2024-03-21T10:58:37.350 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-21T10:58:37.350 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} --name vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288 --owner scheduled_vshankar@teuthology --verbose -- /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288/orig.config.yaml 2024-03-21T10:58:37.406 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T10:58:37.497 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288 branch: wip-vshankar-testing-20240320.053002 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} email: vshankar@redhat.com first_in_suite: false job_id: '7612288' kernel: client: branch: testing flavor: default kdb: true sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240320.053002 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240320.053002 sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8073 sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 sleep_before_teardown: 0 subset: 203/1024 suite: fs suite_branch: wip-vshankar-testing-20240320.053002 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLgNA2Cv+/e+vSQiK6PBnccFdtzJeLcrO6X8o2IA6GnIzz2YmyjPyY9VqM/h7kdjJ994waGlpuljCHye3vrB+ms= smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMAKhlcMYYiUa8l05rssHJxz+MWdLxH5Db3TrI5GQrNK2JqHXA+5lxfmLOUMCPxtgSIk0zt/rI19pc1bJml8CjM= smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHGc7E7fcehHfHo3EXyrmhbGUh5qN0aAYfNGR18GsApBPg7ec4LJmJtKZIFZzxmkPI3YFipQT0YCUyvUM838azE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-20_11:21:18 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-21T10:58:37.497 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa; will attempt to use it 2024-03-21T10:58:37.498 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks 2024-03-21T10:58:37.498 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-21T10:58:37.500 INFO:teuthology.task.internal:Checking packages... 2024-03-21T10:58:37.524 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2edfee68ad7243bed422a049b6f5f58619300f11' 2024-03-21T10:58:37.524 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-21T10:58:37.524 INFO:teuthology.packaging:ref: None 2024-03-21T10:58:37.524 INFO:teuthology.packaging:tag: None 2024-03-21T10:58:37.524 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240320.053002 2024-03-21T10:58:37.525 INFO:teuthology.packaging:sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:58:37.525 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240320.053002 2024-03-21T10:58:37.663 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2231.g2edfee68 2024-03-21T10:58:37.665 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-21T10:58:37.672 INFO:teuthology.task.internal:no buildpackages task found 2024-03-21T10:58:37.672 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-21T10:58:37.716 INFO:teuthology.task.internal:Saving configuration 2024-03-21T10:58:37.737 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-21T10:58:37.743 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T10:58:37.769 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 10:53:37.281124', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLgNA2Cv+/e+vSQiK6PBnccFdtzJeLcrO6X8o2IA6GnIzz2YmyjPyY9VqM/h7kdjJ994waGlpuljCHye3vrB+ms='} 2024-03-21T10:58:37.788 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 10:53:37.278927', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMAKhlcMYYiUa8l05rssHJxz+MWdLxH5Db3TrI5GQrNK2JqHXA+5lxfmLOUMCPxtgSIk0zt/rI19pc1bJml8CjM='} 2024-03-21T10:58:37.807 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 10:53:37.280099', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHGc7E7fcehHfHo3EXyrmhbGUh5qN0aAYfNGR18GsApBPg7ec4LJmJtKZIFZzxmkPI3YFipQT0YCUyvUM838azE='} 2024-03-21T10:58:37.807 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-21T10:58:37.813 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-21T10:58:37.813 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-21T10:58:37.813 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-21T10:58:37.813 INFO:teuthology.run_tasks:Running task console_log... 2024-03-21T10:58:37.925 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8e7a72f160>, signals=[15]) 2024-03-21T10:58:37.926 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-21T10:58:37.932 INFO:teuthology.task.internal:Opening connections... 2024-03-21T10:58:37.933 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T10:58:37.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T10:58:38.044 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-03-21T10:58:38.045 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T10:58:38.136 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-03-21T10:58:38.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T10:58:38.232 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-21T10:58:38.242 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-03-21T10:58:38.262 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-03-21T10:58:38.262 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-03-21T10:58:38.321 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-03-21T10:58:38.321 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-03-21T10:58:38.321 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-03-21T10:58:38.321 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-03-21T10:58:38.322 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-03-21T10:58:38.322 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-03-21T10:58:38.322 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T10:58:38.322 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-03-21T10:58:38.322 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-03-21T10:58:38.322 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T10:58:38.323 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-03-21T10:58:38.323 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T10:58:38.323 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T10:58:38.323 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T10:58:38.323 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-03-21T10:58:38.350 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-03-21T10:58:38.367 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-03-21T10:58:38.368 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-03-21T10:58:38.425 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-03-21T10:58:38.425 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-03-21T10:58:38.425 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-03-21T10:58:38.426 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-03-21T10:58:38.426 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-03-21T10:58:38.426 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-03-21T10:58:38.426 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T10:58:38.426 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-03-21T10:58:38.426 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-03-21T10:58:38.426 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T10:58:38.426 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-03-21T10:58:38.426 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T10:58:38.426 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T10:58:38.426 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T10:58:38.427 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-03-21T10:58:38.449 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-03-21T10:58:38.466 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-03-21T10:58:38.466 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-03-21T10:58:38.523 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-03-21T10:58:38.523 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="9" 2024-03-21T10:58:38.523 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-03-21T10:58:38.523 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-03-21T10:58:38.523 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="9" 2024-03-21T10:58:38.523 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el9" 2024-03-21T10:58:38.523 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T10:58:38.524 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-03-21T10:58:38.524 INFO:teuthology.orchestra.run.smithi102.stdout:LOGO="fedora-logo-icon" 2024-03-21T10:58:38.524 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T10:58:38.524 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-03-21T10:58:38.524 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T10:58:38.524 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T10:58:38.524 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T10:58:38.524 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-03-21T10:58:38.549 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-21T10:58:38.561 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-21T10:58:38.579 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-21T10:58:38.579 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-03-21T10:58:38.582 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-03-21T10:58:38.584 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-03-21T10:58:38.599 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-21T10:58:38.605 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-21T10:58:38.605 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-03-21T10:58:38.639 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-03-21T10:58:38.641 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-03-21T10:58:38.750 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-21T10:58:38.831 INFO:teuthology.run_tasks:Running task kernel... 2024-03-21T10:58:38.847 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'} 2024-03-21T10:58:38.847 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-03-21T10:58:38.847 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-03-21T10:58:38.848 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T10:58:38.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T10:58:38.866 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-21T10:58:39.184 DEBUG:teuthology.task.kernel:sha1 for client.0 is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T10:58:39.184 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-21T10:58:39.184 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-03-21T10:58:39.202 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-361.el9.x86_64 2024-03-21T10:58:39.202 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T10:58:39.203 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-21T10:58:39.203 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum upgrade -y linux-firmware 2024-03-21T10:58:41.431 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-21T10:58:42.873 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - BaseOS 7.0 MB/s | 2.0 MB 00:00 2024-03-21T10:58:43.764 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-03-21T10:58:45.473 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-03-21T10:58:49.907 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 6.1 MB 00:00 2024-03-21T10:58:51.616 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-03-21T10:58:53.012 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-21T10:58:58.278 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-03-21T10:59:00.600 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-03-21T10:59:00.608 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-03-21T10:59:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-03-21T10:59:00.608 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-03-21T10:59:00.608 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-03-21T10:59:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-21T10:59:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:59:00.608 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-03-21T10:59:00.608 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-03-21T10:59:00.608 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 1 Package 2024-03-21T10:59:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:59:00.609 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 375 M 2024-03-21T10:59:00.609 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-03-21T10:59:09.802 INFO:teuthology.orchestra.run.smithi047.stdout:linux-firmware-20240219-143.el9.noarch.rpm 41 MB/s | 375 MB 00:09 2024-03-21T10:59:09.803 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-03-21T10:59:09.803 INFO:teuthology.orchestra.run.smithi047.stdout:Total 41 MB/s | 375 MB 00:09 2024-03-21T10:59:09.804 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-03-21T10:59:09.906 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-03-21T10:59:09.906 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-03-21T10:59:12.444 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-03-21T10:59:12.444 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-03-21T10:59:18.921 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-03-21T10:59:19.133 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T10:59:19.222 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T10:59:20.630 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T10:59:20.631 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T10:59:21.012 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T10:59:21.012 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:59:21.012 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-03-21T10:59:21.012 INFO:teuthology.orchestra.run.smithi047.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-21T10:59:21.012 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:59:21.012 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-03-21T10:59:21.291 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.0... 2024-03-21T10:59:21.292 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T10:59:21.514 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-21T10:59:21.514 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-21T10:59:21.516 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi047.front.sepia.ceph.com to complete... 2024-03-21T10:59:33.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21 10:59:33 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-21T10:59:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:FINISHED --2024-03-21 10:59:33-- 2024-03-21T10:59:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:Total wall clock time: 12s 2024-03-21T10:59:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:Downloaded: 1 files, 321M in 12s (27.9 MB/s) 2024-03-21T10:59:33.443 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.0... 2024-03-21T10:59:33.444 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-21T10:59:35.471 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying... ######################################## 2024-03-21T10:59:35.534 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing... ######################################## 2024-03-21T10:59:35.617 INFO:teuthology.orchestra.run.smithi047.stdout:Updating / installing... 2024-03-21T10:59:43.446 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-21T10:59:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T11:00:22.266 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-21T11:00:22.266 DEBUG:teuthology.orchestra.run.smithi047:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-21T11:00:22.342 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-21T11:00:22.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-21T11:00:22.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-21T11:00:22.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-21T11:00:22.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-21T11:00:22.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-21T11:00:22.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-21T11:00:22.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-21T11:00:22.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-21T11:00:22.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-21T11:00:22.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-21T11:00:22.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-21T11:00:22.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-21T11:00:22.343 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-21T11:00:22.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-21T11:00:22.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-21T11:00:22.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-21T11:00:22.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-21T11:00:22.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-21T11:00:22.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-21T11:00:22.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-21T11:00:22.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-21T11:00:22.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-21T11:00:22.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-21T11:00:22.344 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-21T11:00:22.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-21T11:00:22.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-21T11:00:22.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-21T11:00:22.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-21T11:00:22.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-21T11:00:22.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-21T11:00:22.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-21T11:00:22.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-21T11:00:22.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-21T11:00:22.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-21T11:00:22.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-21T11:00:22.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-21T11:00:22.345 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-21T11:00:22.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-21T11:00:22.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-21T11:00:22.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-21T11:00:22.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-21T11:00:22.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-21T11:00:22.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-21T11:00:22.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-21T11:00:22.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-21T11:00:22.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-21T11:00:22.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-21T11:00:22.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-21T11:00:22.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-21T11:00:22.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-21T11:00:22.346 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-21T11:00:22.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-21T11:00:22.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-21T11:00:22.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-21T11:00:22.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-21T11:00:22.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-21T11:00:22.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-21T11:00:22.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-21T11:00:22.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-21T11:00:22.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-21T11:00:22.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-21T11:00:22.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-21T11:00:22.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-21T11:00:22.347 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-21T11:00:22.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-21T11:00:22.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-21T11:00:22.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-21T11:00:22.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-21T11:00:22.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-21T11:00:22.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-21T11:00:22.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-21T11:00:22.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-21T11:00:22.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-21T11:00:22.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-21T11:00:22.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-21T11:00:22.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-21T11:00:22.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-21T11:00:22.348 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-21T11:00:22.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-21T11:00:22.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-21T11:00:22.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-21T11:00:22.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-21T11:00:22.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-21T11:00:22.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-21T11:00:22.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-21T11:00:22.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-21T11:00:22.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-21T11:00:22.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-21T11:00:22.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-21T11:00:22.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-21T11:00:22.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-21T11:00:22.349 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-21T11:00:22.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-21T11:00:22.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-21T11:00:22.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-21T11:00:22.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-21T11:00:22.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-21T11:00:22.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-21T11:00:22.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-21T11:00:22.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-21T11:00:22.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-21T11:00:22.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-21T11:00:22.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-21T11:00:22.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-21T11:00:22.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-21T11:00:22.350 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-21T11:00:22.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-21T11:00:22.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-21T11:00:22.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-21T11:00:22.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-21T11:00:22.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-21T11:00:22.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-21T11:00:22.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-21T11:00:22.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-21T11:00:22.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-21T11:00:22.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-21T11:00:22.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-21T11:00:22.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-21T11:00:22.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-21T11:00:22.351 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-21T11:00:22.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-21T11:00:22.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-21T11:00:22.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-21T11:00:22.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-21T11:00:22.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-21T11:00:22.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-21T11:00:22.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-21T11:00:22.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-21T11:00:22.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-21T11:00:22.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-21T11:00:22.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-21T11:00:22.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-21T11:00:22.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-21T11:00:22.352 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-21T11:00:22.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-21T11:00:22.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-21T11:00:22.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-21T11:00:22.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-21T11:00:22.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-21T11:00:22.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-21T11:00:22.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-21T11:00:22.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-21T11:00:22.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-21T11:00:22.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-21T11:00:22.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-21T11:00:22.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-21T11:00:22.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-21T11:00:22.353 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-21T11:00:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-21T11:00:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-21T11:00:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-21T11:00:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-21T11:00:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-21T11:00:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-21T11:00:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-21T11:00:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-21T11:00:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-21T11:00:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-21T11:00:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-21T11:00:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-21T11:00:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-21T11:00:22.354 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-21T11:00:22.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-21T11:00:22.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-21T11:00:22.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-21T11:00:22.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-21T11:00:22.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-21T11:00:22.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-21T11:00:22.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-21T11:00:22.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-21T11:00:22.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-21T11:00:22.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-21T11:00:22.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-21T11:00:22.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-21T11:00:22.355 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-21T11:00:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-21T11:00:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-21T11:00:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-21T11:00:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-21T11:00:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-21T11:00:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-21T11:00:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-21T11:00:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-21T11:00:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-21T11:00:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-21T11:00:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-21T11:00:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-21T11:00:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-21T11:00:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-21T11:00:22.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-21T11:00:22.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-21T11:00:22.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-21T11:00:22.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-21T11:00:22.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-21T11:00:22.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-21T11:00:22.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-21T11:00:22.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-21T11:00:22.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-21T11:00:22.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-21T11:00:22.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-21T11:00:22.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-21T11:00:22.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-21T11:00:22.357 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-21T11:00:22.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-21T11:00:22.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-21T11:00:22.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-21T11:00:22.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-21T11:00:22.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-21T11:00:22.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-21T11:00:22.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-21T11:00:22.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-21T11:00:22.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-21T11:00:22.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-21T11:00:22.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-21T11:00:22.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-21T11:00:22.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-21T11:00:22.358 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-21T11:00:22.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-21T11:00:22.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-21T11:00:22.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-21T11:00:22.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-21T11:00:22.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-21T11:00:22.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-21T11:00:22.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-21T11:00:22.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-21T11:00:22.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-21T11:00:22.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-21T11:00:22.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-21T11:00:22.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-21T11:00:22.359 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-21T11:00:22.360 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-21T11:00:22.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-21T11:00:22.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-21T11:00:22.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-21T11:00:22.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-21T11:00:22.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-21T11:00:22.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-21T11:00:22.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-21T11:00:22.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-21T11:00:22.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-21T11:00:22.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-21T11:00:22.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-21T11:00:22.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-21T11:00:22.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-21T11:00:22.361 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-21T11:00:22.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-21T11:00:22.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-21T11:00:22.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-21T11:00:22.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-21T11:00:22.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-21T11:00:22.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-21T11:00:22.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-21T11:00:22.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-21T11:00:22.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-21T11:00:22.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-21T11:00:22.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-21T11:00:22.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-21T11:00:22.362 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-21T11:00:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-21T11:00:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-21T11:00:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-21T11:00:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-21T11:00:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-21T11:00:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-21T11:00:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-21T11:00:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-21T11:00:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-21T11:00:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-21T11:00:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-21T11:00:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-21T11:00:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-21T11:00:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-21T11:00:22.364 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-21T11:00:22.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-21T11:00:22.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-21T11:00:22.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-21T11:00:22.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-21T11:00:22.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-21T11:00:22.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-21T11:00:22.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-21T11:00:22.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-21T11:00:22.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-21T11:00:22.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-21T11:00:22.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-21T11:00:22.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-21T11:00:22.365 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-21T11:00:22.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-21T11:00:22.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-21T11:00:22.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-21T11:00:22.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-21T11:00:22.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-21T11:00:22.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-21T11:00:22.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-21T11:00:22.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-21T11:00:22.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-21T11:00:22.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-21T11:00:22.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-21T11:00:22.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-21T11:00:22.366 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-21T11:00:22.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-21T11:00:22.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-21T11:00:22.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-21T11:00:22.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-21T11:00:22.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-21T11:00:22.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-21T11:00:22.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-21T11:00:22.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-21T11:00:22.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-21T11:00:22.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-21T11:00:22.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-21T11:00:22.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-21T11:00:22.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-21T11:00:22.367 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-21T11:00:22.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-21T11:00:22.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-21T11:00:22.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-21T11:00:22.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-21T11:00:22.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-21T11:00:22.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-21T11:00:22.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-21T11:00:22.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-21T11:00:22.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-21T11:00:22.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-21T11:00:22.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-21T11:00:22.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-21T11:00:22.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-21T11:00:22.368 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-21T11:00:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-21T11:00:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-21T11:00:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-21T11:00:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-21T11:00:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-21T11:00:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-21T11:00:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-21T11:00:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-21T11:00:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-21T11:00:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-21T11:00:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-21T11:00:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-21T11:00:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-21T11:00:22.369 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-21T11:00:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-21T11:00:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-21T11:00:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-21T11:00:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-21T11:00:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-21T11:00:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-21T11:00:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-21T11:00:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-21T11:00:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-21T11:00:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-21T11:00:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-21T11:00:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-21T11:00:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-21T11:00:22.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-21T11:00:22.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-21T11:00:22.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-21T11:00:22.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-21T11:00:22.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-21T11:00:22.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-21T11:00:22.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-21T11:00:22.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-21T11:00:22.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-21T11:00:22.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-21T11:00:22.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-21T11:00:22.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-21T11:00:22.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-21T11:00:22.371 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-21T11:00:22.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-21T11:00:22.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-21T11:00:22.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-21T11:00:22.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-21T11:00:22.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-21T11:00:22.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-21T11:00:22.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-21T11:00:22.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-21T11:00:22.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-21T11:00:22.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-21T11:00:22.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-21T11:00:22.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-21T11:00:22.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-21T11:00:22.372 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-21T11:00:22.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-21T11:00:22.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-21T11:00:22.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-21T11:00:22.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-21T11:00:22.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-21T11:00:22.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-21T11:00:22.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-21T11:00:22.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-21T11:00:22.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-21T11:00:22.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-21T11:00:22.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-21T11:00:22.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-21T11:00:22.373 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-21T11:00:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-21T11:00:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-21T11:00:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-21T11:00:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-21T11:00:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-21T11:00:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-21T11:00:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-21T11:00:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-21T11:00:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-21T11:00:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-21T11:00:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-21T11:00:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-21T11:00:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-21T11:00:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-21T11:00:22.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-21T11:00:22.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-21T11:00:22.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-21T11:00:22.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-21T11:00:22.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-21T11:00:22.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-21T11:00:22.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-21T11:00:22.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-21T11:00:22.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-21T11:00:22.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-21T11:00:22.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-21T11:00:22.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-21T11:00:22.375 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-21T11:00:22.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-21T11:00:22.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-21T11:00:22.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-21T11:00:22.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-21T11:00:22.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-21T11:00:22.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-21T11:00:22.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-21T11:00:22.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-21T11:00:22.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-21T11:00:22.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-21T11:00:22.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-21T11:00:22.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-21T11:00:22.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-21T11:00:22.376 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-21T11:00:22.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-21T11:00:22.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-21T11:00:22.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-21T11:00:22.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-21T11:00:22.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-21T11:00:22.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-21T11:00:22.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-21T11:00:22.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-21T11:00:22.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-21T11:00:22.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-21T11:00:22.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-21T11:00:22.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-21T11:00:22.377 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-21T11:00:22.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-21T11:00:22.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-21T11:00:22.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-21T11:00:22.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-21T11:00:22.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-21T11:00:22.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-21T11:00:22.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-21T11:00:22.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-21T11:00:22.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-21T11:00:22.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-21T11:00:22.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-21T11:00:22.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-21T11:00:22.378 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-21T11:00:22.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-21T11:00:22.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-21T11:00:22.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-21T11:00:22.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-21T11:00:22.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-21T11:00:22.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-21T11:00:22.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-21T11:00:22.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-21T11:00:22.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-21T11:00:22.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-21T11:00:22.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-21T11:00:22.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-21T11:00:22.379 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-21T11:00:22.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-21T11:00:22.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-21T11:00:22.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-21T11:00:22.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-21T11:00:22.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-21T11:00:22.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-21T11:00:22.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-21T11:00:22.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-21T11:00:22.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-21T11:00:22.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-21T11:00:22.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-21T11:00:22.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-21T11:00:22.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-21T11:00:22.380 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-21T11:00:22.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-21T11:00:22.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-21T11:00:22.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-21T11:00:22.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-21T11:00:22.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-21T11:00:22.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-21T11:00:22.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-21T11:00:22.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-21T11:00:22.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-21T11:00:22.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-21T11:00:22.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-21T11:00:22.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-21T11:00:22.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-21T11:00:22.381 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-21T11:00:22.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-21T11:00:22.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-21T11:00:22.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-21T11:00:22.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-21T11:00:22.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-21T11:00:22.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-21T11:00:22.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-21T11:00:22.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-21T11:00:22.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-21T11:00:22.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-21T11:00:22.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-21T11:00:22.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-21T11:00:22.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-21T11:00:22.382 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-21T11:00:22.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-21T11:00:22.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-21T11:00:22.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-21T11:00:22.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-21T11:00:22.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-21T11:00:22.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-21T11:00:22.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-21T11:00:22.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-21T11:00:22.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-21T11:00:22.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-21T11:00:22.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-21T11:00:22.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-21T11:00:22.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-21T11:00:22.383 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-21T11:00:22.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-21T11:00:22.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-21T11:00:22.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-21T11:00:22.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-21T11:00:22.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-21T11:00:22.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-21T11:00:22.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-21T11:00:22.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-21T11:00:22.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-21T11:00:22.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-21T11:00:22.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-21T11:00:22.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-21T11:00:22.384 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-21T11:00:22.385 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-21T11:00:22.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-21T11:00:22.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-21T11:00:22.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-21T11:00:22.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-21T11:00:22.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-21T11:00:22.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-21T11:00:22.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-21T11:00:22.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-21T11:00:22.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-21T11:00:22.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-21T11:00:22.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-21T11:00:22.386 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-21T11:00:22.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-21T11:00:22.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-21T11:00:22.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-21T11:00:22.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-21T11:00:22.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-21T11:00:22.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-21T11:00:22.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-21T11:00:22.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-21T11:00:22.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-21T11:00:22.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-21T11:00:22.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-21T11:00:22.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-21T11:00:22.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-21T11:00:22.387 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-21T11:00:22.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-21T11:00:22.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-21T11:00:22.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-21T11:00:22.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-21T11:00:22.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-21T11:00:22.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-21T11:00:22.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-21T11:00:22.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-21T11:00:22.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-21T11:00:22.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-21T11:00:22.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-21T11:00:22.388 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-21T11:00:22.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-21T11:00:22.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-21T11:00:22.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-21T11:00:22.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-21T11:00:22.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-21T11:00:22.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-21T11:00:22.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-21T11:00:22.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-21T11:00:22.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-21T11:00:22.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-21T11:00:22.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-21T11:00:22.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-21T11:00:22.389 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-21T11:00:22.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-21T11:00:22.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-21T11:00:22.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-21T11:00:22.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-21T11:00:22.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-21T11:00:22.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-21T11:00:22.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-21T11:00:22.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-21T11:00:22.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-21T11:00:22.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-21T11:00:22.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-21T11:00:22.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-21T11:00:22.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-21T11:00:22.390 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-21T11:00:22.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-21T11:00:22.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-21T11:00:22.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-21T11:00:22.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-21T11:00:22.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-21T11:00:22.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-21T11:00:22.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-21T11:00:22.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-21T11:00:22.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-21T11:00:22.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-21T11:00:22.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-21T11:00:22.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-21T11:00:22.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-21T11:00:22.391 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-21T11:00:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-21T11:00:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-21T11:00:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-21T11:00:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-21T11:00:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-21T11:00:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-21T11:00:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-21T11:00:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-21T11:00:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-21T11:00:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-21T11:00:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-21T11:00:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-21T11:00:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-21T11:00:22.392 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-21T11:00:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-21T11:00:22.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-21T11:00:22.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-21T11:00:22.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-21T11:00:22.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-21T11:00:22.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-21T11:00:22.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-21T11:00:22.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-21T11:00:22.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-21T11:00:22.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-21T11:00:22.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-21T11:00:22.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-21T11:00:22.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-21T11:00:22.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-21T11:00:22.394 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-21T11:00:22.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-21T11:00:22.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-21T11:00:22.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-21T11:00:22.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-21T11:00:22.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-21T11:00:22.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-21T11:00:22.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-21T11:00:22.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-21T11:00:22.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-21T11:00:22.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-21T11:00:22.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-21T11:00:22.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-21T11:00:22.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-21T11:00:22.395 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-21T11:00:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-21T11:00:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-21T11:00:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-21T11:00:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-21T11:00:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-21T11:00:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-21T11:00:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-21T11:00:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-21T11:00:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-21T11:00:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-21T11:00:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-21T11:00:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-21T11:00:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-21T11:00:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-21T11:00:22.397 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-21T11:00:22.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-21T11:00:22.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-21T11:00:22.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-21T11:00:22.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-21T11:00:22.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-21T11:00:22.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-21T11:00:22.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-21T11:00:22.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-21T11:00:22.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-21T11:00:22.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-21T11:00:22.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-21T11:00:22.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-21T11:00:22.398 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-21T11:00:22.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-21T11:00:22.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-21T11:00:22.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-21T11:00:22.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-21T11:00:22.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-21T11:00:22.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-21T11:00:22.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-21T11:00:22.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-21T11:00:22.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-21T11:00:22.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-21T11:00:22.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-21T11:00:22.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-21T11:00:22.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-21T11:00:22.399 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-21T11:00:22.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-21T11:00:22.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-21T11:00:22.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-21T11:00:22.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-21T11:00:22.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-21T11:00:22.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-21T11:00:22.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-21T11:00:22.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-21T11:00:22.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-21T11:00:22.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-21T11:00:22.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-21T11:00:22.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-21T11:00:22.400 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-21T11:00:22.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-21T11:00:22.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-21T11:00:22.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-21T11:00:22.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-21T11:00:22.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-21T11:00:22.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-21T11:00:22.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-21T11:00:22.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-21T11:00:22.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-21T11:00:22.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-21T11:00:22.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-21T11:00:22.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-21T11:00:22.401 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-21T11:00:22.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-21T11:00:22.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-21T11:00:22.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-21T11:00:22.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-21T11:00:22.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-21T11:00:22.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-21T11:00:22.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-21T11:00:22.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-21T11:00:22.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-21T11:00:22.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-21T11:00:22.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-21T11:00:22.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-21T11:00:22.402 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-21T11:00:22.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-21T11:00:22.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-21T11:00:22.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-21T11:00:22.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-21T11:00:22.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-21T11:00:22.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-21T11:00:22.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-21T11:00:22.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-21T11:00:22.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-21T11:00:22.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-21T11:00:22.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-21T11:00:22.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-21T11:00:22.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-21T11:00:22.403 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-21T11:00:22.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-21T11:00:22.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-21T11:00:22.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-21T11:00:22.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-21T11:00:22.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-21T11:00:22.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-21T11:00:22.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-21T11:00:22.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-21T11:00:22.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-21T11:00:22.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-21T11:00:22.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-21T11:00:22.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-21T11:00:22.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-21T11:00:22.404 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-21T11:00:22.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-21T11:00:22.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-21T11:00:22.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-21T11:00:22.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-21T11:00:22.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-21T11:00:22.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-21T11:00:22.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-21T11:00:22.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-21T11:00:22.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-21T11:00:22.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-21T11:00:22.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-21T11:00:22.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-21T11:00:22.405 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-21T11:00:22.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-21T11:00:22.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-21T11:00:22.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-21T11:00:22.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-21T11:00:22.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-21T11:00:22.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-21T11:00:22.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-21T11:00:22.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-21T11:00:22.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-21T11:00:22.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-21T11:00:22.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-21T11:00:22.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-21T11:00:22.406 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-21T11:00:22.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-21T11:00:22.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-21T11:00:22.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-21T11:00:22.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-21T11:00:22.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-21T11:00:22.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-21T11:00:22.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-21T11:00:22.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-21T11:00:22.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-21T11:00:22.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-21T11:00:22.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-21T11:00:22.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-21T11:00:22.407 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-21T11:00:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-21T11:00:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-21T11:00:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-21T11:00:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-21T11:00:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-21T11:00:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-21T11:00:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-21T11:00:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-21T11:00:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-21T11:00:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-21T11:00:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-21T11:00:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-21T11:00:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-21T11:00:22.408 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-21T11:00:22.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-21T11:00:22.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-21T11:00:22.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-21T11:00:22.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-21T11:00:22.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-21T11:00:22.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-21T11:00:22.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-21T11:00:22.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-21T11:00:22.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-21T11:00:22.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-21T11:00:22.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-21T11:00:22.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-21T11:00:22.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-21T11:00:22.409 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-21T11:00:22.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-21T11:00:22.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-21T11:00:22.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-21T11:00:22.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-21T11:00:22.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-21T11:00:22.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-21T11:00:22.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-21T11:00:22.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-21T11:00:22.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-21T11:00:22.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-21T11:00:22.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-21T11:00:22.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-21T11:00:22.410 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-21T11:00:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-21T11:00:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-21T11:00:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-21T11:00:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-21T11:00:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-21T11:00:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-21T11:00:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-21T11:00:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-21T11:00:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-21T11:00:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-21T11:00:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-21T11:00:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-21T11:00:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-21T11:00:22.411 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-21T11:00:22.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-21T11:00:22.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-21T11:00:22.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-21T11:00:22.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-21T11:00:22.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-21T11:00:22.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-21T11:00:22.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-21T11:00:22.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-21T11:00:22.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-21T11:00:22.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-21T11:00:22.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-21T11:00:22.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-21T11:00:22.412 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-21T11:00:22.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-21T11:00:22.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-21T11:00:22.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-21T11:00:22.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-21T11:00:22.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-21T11:00:22.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-21T11:00:22.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-21T11:00:22.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-21T11:00:22.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-21T11:00:22.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-21T11:00:22.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-21T11:00:22.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-21T11:00:22.413 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-21T11:00:22.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-21T11:00:22.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-21T11:00:22.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-21T11:00:22.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-21T11:00:22.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-21T11:00:22.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-21T11:00:22.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-21T11:00:22.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-21T11:00:22.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-21T11:00:22.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-21T11:00:22.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-21T11:00:22.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-21T11:00:22.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-21T11:00:22.414 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-21T11:00:22.415 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-21T11:00:22.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-21T11:00:22.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-21T11:00:22.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-21T11:00:22.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-21T11:00:22.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-21T11:00:22.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-21T11:00:22.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-21T11:00:22.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-21T11:00:22.416 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-21T11:00:22.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-21T11:00:22.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-21T11:00:22.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-21T11:00:22.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-21T11:00:22.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-21T11:00:22.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-21T11:00:22.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-21T11:00:22.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-21T11:00:22.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-21T11:00:22.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-21T11:00:22.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-21T11:00:22.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-21T11:00:22.417 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-21T11:00:22.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-21T11:00:22.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-21T11:00:22.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-21T11:00:22.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-21T11:00:22.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-21T11:00:22.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-21T11:00:22.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-21T11:00:22.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-21T11:00:22.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-21T11:00:22.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-21T11:00:22.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-21T11:00:22.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-21T11:00:22.418 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-21T11:00:22.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-21T11:00:22.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-21T11:00:22.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-21T11:00:22.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-21T11:00:22.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-21T11:00:22.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-21T11:00:22.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-21T11:00:22.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-21T11:00:22.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-21T11:00:22.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-21T11:00:22.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-21T11:00:22.419 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-21T11:00:22.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-21T11:00:22.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-21T11:00:22.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-21T11:00:22.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-21T11:00:22.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-21T11:00:22.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-21T11:00:22.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-21T11:00:22.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-21T11:00:22.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-21T11:00:22.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-21T11:00:22.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-21T11:00:22.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-21T11:00:22.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-21T11:00:22.420 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-21T11:00:22.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-21T11:00:22.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-21T11:00:22.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-21T11:00:22.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-21T11:00:22.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-21T11:00:22.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-21T11:00:22.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-21T11:00:22.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-21T11:00:22.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-21T11:00:22.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-21T11:00:22.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-21T11:00:22.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-21T11:00:22.421 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-21T11:00:22.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-21T11:00:22.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-21T11:00:22.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-21T11:00:22.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-21T11:00:22.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-21T11:00:22.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-21T11:00:22.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-21T11:00:22.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-21T11:00:22.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-21T11:00:22.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-21T11:00:22.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-21T11:00:22.422 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-21T11:00:22.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-21T11:00:22.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-21T11:00:22.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-21T11:00:22.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-21T11:00:22.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-21T11:00:22.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-21T11:00:22.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-21T11:00:22.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-21T11:00:22.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-21T11:00:22.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-21T11:00:22.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-21T11:00:22.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-21T11:00:22.423 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-21T11:00:22.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-21T11:00:22.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-21T11:00:22.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-21T11:00:22.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-21T11:00:22.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-21T11:00:22.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-21T11:00:22.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-21T11:00:22.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-21T11:00:22.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-21T11:00:22.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-21T11:00:22.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-21T11:00:22.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-21T11:00:22.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-21T11:00:22.424 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-21T11:00:22.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-21T11:00:22.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-21T11:00:22.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-21T11:00:22.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-21T11:00:22.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-21T11:00:22.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-21T11:00:22.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-21T11:00:22.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-21T11:00:22.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-21T11:00:22.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-21T11:00:22.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-21T11:00:22.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-21T11:00:22.425 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-21T11:00:22.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-21T11:00:22.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-21T11:00:22.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-21T11:00:22.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-21T11:00:22.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-21T11:00:22.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-21T11:00:22.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-21T11:00:22.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-21T11:00:22.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-21T11:00:22.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-21T11:00:22.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-21T11:00:22.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-21T11:00:22.426 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-21T11:00:22.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-21T11:00:22.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-21T11:00:22.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-21T11:00:22.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-21T11:00:22.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-21T11:00:22.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-21T11:00:22.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-21T11:00:22.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-21T11:00:22.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-21T11:00:22.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-21T11:00:22.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-21T11:00:22.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-21T11:00:22.427 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-21T11:00:22.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-21T11:00:22.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-21T11:00:22.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-21T11:00:22.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-21T11:00:22.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-21T11:00:22.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-21T11:00:22.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-21T11:00:22.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-21T11:00:22.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-21T11:00:22.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-21T11:00:22.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-21T11:00:22.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-21T11:00:22.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-21T11:00:22.428 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-21T11:00:22.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-21T11:00:22.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-21T11:00:22.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-21T11:00:22.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-21T11:00:22.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-21T11:00:22.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-21T11:00:22.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-21T11:00:22.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-21T11:00:22.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-21T11:00:22.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-21T11:00:22.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-21T11:00:22.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-21T11:00:22.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-21T11:00:22.429 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-21T11:00:22.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-21T11:00:22.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-21T11:00:22.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-21T11:00:22.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-21T11:00:22.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-21T11:00:22.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-21T11:00:22.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-21T11:00:22.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-21T11:00:22.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-21T11:00:22.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-21T11:00:22.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-21T11:00:22.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-21T11:00:22.430 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-21T11:00:22.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-21T11:00:22.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-21T11:00:22.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-21T11:00:22.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-21T11:00:22.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-21T11:00:22.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-21T11:00:22.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-21T11:00:22.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-21T11:00:22.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-21T11:00:22.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-21T11:00:22.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-21T11:00:22.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-21T11:00:22.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-21T11:00:22.431 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-21T11:00:22.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-21T11:00:22.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-21T11:00:22.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-21T11:00:22.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-21T11:00:22.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-21T11:00:22.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-21T11:00:22.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-21T11:00:22.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-21T11:00:22.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-21T11:00:22.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-21T11:00:22.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-21T11:00:22.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-21T11:00:22.432 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-21T11:00:22.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-21T11:00:22.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-21T11:00:22.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-21T11:00:22.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-21T11:00:22.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-21T11:00:22.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-21T11:00:22.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-21T11:00:22.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-21T11:00:22.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-21T11:00:22.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-21T11:00:22.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-21T11:00:22.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-21T11:00:22.433 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-21T11:00:22.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-21T11:00:22.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-21T11:00:22.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-21T11:00:22.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-21T11:00:22.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-21T11:00:22.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-21T11:00:22.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-21T11:00:22.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-21T11:00:22.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-21T11:00:22.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-21T11:00:22.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-21T11:00:22.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-21T11:00:22.434 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-21T11:00:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-21T11:00:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-21T11:00:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-21T11:00:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-21T11:00:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-21T11:00:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-21T11:00:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-21T11:00:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-21T11:00:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-21T11:00:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-21T11:00:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-21T11:00:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-21T11:00:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-21T11:00:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-21T11:00:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-21T11:00:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-21T11:00:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-21T11:00:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-21T11:00:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-21T11:00:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-21T11:00:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-21T11:00:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-21T11:00:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-21T11:00:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-21T11:00:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-21T11:00:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-21T11:00:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-21T11:00:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-21T11:00:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-21T11:00:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-21T11:00:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-21T11:00:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-21T11:00:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-21T11:00:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-21T11:00:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-21T11:00:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-21T11:00:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-21T11:00:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-21T11:00:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-21T11:00:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-21T11:00:22.437 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-21T11:00:22.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-21T11:00:22.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-21T11:00:22.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-21T11:00:22.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-21T11:00:22.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-21T11:00:22.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-21T11:00:22.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-21T11:00:22.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-21T11:00:22.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-21T11:00:22.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-21T11:00:22.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-21T11:00:22.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-21T11:00:22.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-21T11:00:22.438 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-21T11:00:22.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-21T11:00:22.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-21T11:00:22.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-21T11:00:22.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-21T11:00:22.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-21T11:00:22.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-21T11:00:22.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-21T11:00:22.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-21T11:00:22.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-21T11:00:22.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-21T11:00:22.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-21T11:00:22.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-21T11:00:22.439 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-21T11:00:22.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-21T11:00:22.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-21T11:00:22.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-21T11:00:22.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-21T11:00:22.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-21T11:00:22.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-21T11:00:22.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-21T11:00:22.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-21T11:00:22.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-21T11:00:22.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-21T11:00:22.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-21T11:00:22.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-21T11:00:22.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-21T11:00:22.440 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-21T11:00:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-21T11:00:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-21T11:00:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-21T11:00:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-21T11:00:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-21T11:00:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-21T11:00:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-21T11:00:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-21T11:00:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-21T11:00:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-21T11:00:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-21T11:00:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-21T11:00:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-21T11:00:22.441 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-21T11:00:22.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-21T11:00:22.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-21T11:00:22.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-21T11:00:22.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-21T11:00:22.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-21T11:00:22.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-21T11:00:22.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-21T11:00:22.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-21T11:00:22.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-21T11:00:22.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-21T11:00:22.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-21T11:00:22.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-21T11:00:22.442 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-21T11:00:22.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-21T11:00:22.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-21T11:00:22.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-21T11:00:22.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-21T11:00:22.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-21T11:00:22.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-21T11:00:22.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-21T11:00:22.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-21T11:00:22.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-21T11:00:22.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-21T11:00:22.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-21T11:00:22.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-21T11:00:22.443 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-21T11:00:22.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-21T11:00:22.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-21T11:00:22.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-21T11:00:22.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-21T11:00:22.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-21T11:00:22.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-21T11:00:22.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-21T11:00:22.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-21T11:00:22.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-21T11:00:22.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-21T11:00:22.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-21T11:00:22.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-21T11:00:22.444 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-21T11:00:22.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-21T11:00:22.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-21T11:00:22.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-21T11:00:22.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-21T11:00:22.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-21T11:00:22.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-21T11:00:22.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-21T11:00:22.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-21T11:00:22.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-21T11:00:22.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-21T11:00:22.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-21T11:00:22.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-21T11:00:22.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-21T11:00:22.445 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-21T11:00:22.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-21T11:00:22.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-21T11:00:22.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-21T11:00:22.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-21T11:00:22.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-21T11:00:22.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-21T11:00:22.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-21T11:00:22.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-21T11:00:22.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-21T11:00:22.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-21T11:00:22.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-21T11:00:22.446 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-21T11:00:22.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-21T11:00:22.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-21T11:00:22.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-21T11:00:22.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-21T11:00:22.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-21T11:00:22.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-21T11:00:22.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-21T11:00:22.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-21T11:00:22.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-21T11:00:22.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-21T11:00:22.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-21T11:00:22.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-21T11:00:22.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-21T11:00:22.447 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-21T11:00:22.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-21T11:00:22.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-21T11:00:22.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-21T11:00:22.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-21T11:00:22.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-21T11:00:22.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-21T11:00:22.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-21T11:00:22.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-21T11:00:22.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-21T11:00:22.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-21T11:00:22.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-21T11:00:22.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-21T11:00:22.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-21T11:00:22.448 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-21T11:00:22.449 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-21T11:00:22.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-21T11:00:22.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-21T11:00:22.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-21T11:00:22.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-21T11:00:22.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-21T11:00:22.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-21T11:00:22.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-21T11:00:22.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-21T11:00:22.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-21T11:00:22.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-21T11:00:22.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-21T11:00:22.450 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-21T11:00:22.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-21T11:00:22.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-21T11:00:22.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-21T11:00:22.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-21T11:00:22.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-21T11:00:22.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-21T11:00:22.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-21T11:00:22.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-21T11:00:22.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-21T11:00:22.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-21T11:00:22.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-21T11:00:22.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-21T11:00:22.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-21T11:00:22.451 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-21T11:00:22.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-21T11:00:22.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-21T11:00:22.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-21T11:00:22.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-21T11:00:22.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-21T11:00:22.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-21T11:00:22.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-21T11:00:22.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-21T11:00:22.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-21T11:00:22.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-21T11:00:22.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-21T11:00:22.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-21T11:00:22.452 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-21T11:00:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-21T11:00:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-21T11:00:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-21T11:00:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-21T11:00:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-21T11:00:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-21T11:00:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-21T11:00:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-21T11:00:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-21T11:00:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-21T11:00:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-21T11:00:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-21T11:00:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-21T11:00:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-21T11:00:22.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-21T11:00:22.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-21T11:00:22.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-21T11:00:22.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-21T11:00:22.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-21T11:00:22.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-21T11:00:22.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-21T11:00:22.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-21T11:00:22.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-21T11:00:22.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-21T11:00:22.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-21T11:00:22.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-21T11:00:22.454 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-21T11:00:22.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-21T11:00:22.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-21T11:00:22.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-21T11:00:22.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-21T11:00:22.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-21T11:00:22.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-21T11:00:22.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-21T11:00:22.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-21T11:00:22.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-21T11:00:22.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-21T11:00:22.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-21T11:00:22.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-21T11:00:22.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-21T11:00:22.455 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-21T11:00:22.456 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-21T11:00:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-21T11:00:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-21T11:00:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-21T11:00:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-21T11:00:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-21T11:00:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-21T11:00:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-21T11:00:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-21T11:00:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-21T11:00:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-21T11:00:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-21T11:00:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-21T11:00:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-21T11:00:22.457 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-21T11:00:22.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-21T11:00:22.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-21T11:00:22.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-21T11:00:22.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-21T11:00:22.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-21T11:00:22.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-21T11:00:22.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-21T11:00:22.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-21T11:00:22.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-21T11:00:22.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-21T11:00:22.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-21T11:00:22.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-21T11:00:22.458 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-21T11:00:22.459 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-21T11:00:22.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-21T11:00:22.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-21T11:00:22.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-21T11:00:22.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-21T11:00:22.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-21T11:00:22.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-21T11:00:22.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-21T11:00:22.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-21T11:00:22.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-21T11:00:22.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-21T11:00:22.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-21T11:00:22.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-21T11:00:22.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-21T11:00:22.460 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-21T11:00:22.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-21T11:00:22.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-21T11:00:22.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-21T11:00:22.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-21T11:00:22.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-21T11:00:22.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-21T11:00:22.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-21T11:00:22.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-21T11:00:22.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-21T11:00:22.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-21T11:00:22.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-21T11:00:22.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-21T11:00:22.461 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-21T11:00:22.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-21T11:00:22.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-21T11:00:22.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-21T11:00:22.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-21T11:00:22.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-21T11:00:22.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-21T11:00:22.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-21T11:00:22.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-21T11:00:22.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-21T11:00:22.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-21T11:00:22.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-21T11:00:22.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-21T11:00:22.462 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-21T11:00:22.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-21T11:00:22.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-21T11:00:22.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-21T11:00:22.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-21T11:00:22.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-21T11:00:22.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-21T11:00:22.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-21T11:00:22.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-21T11:00:22.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-21T11:00:22.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-21T11:00:22.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-21T11:00:22.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-21T11:00:22.463 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-21T11:00:22.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-21T11:00:22.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-21T11:00:22.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-21T11:00:22.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-21T11:00:22.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-21T11:00:22.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-21T11:00:22.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-21T11:00:22.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-21T11:00:22.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-21T11:00:22.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-21T11:00:22.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-21T11:00:22.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-21T11:00:22.464 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-21T11:00:22.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-21T11:00:22.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-21T11:00:22.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-21T11:00:22.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-21T11:00:22.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-21T11:00:22.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-21T11:00:22.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-21T11:00:22.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-21T11:00:22.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-21T11:00:22.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-21T11:00:22.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-21T11:00:22.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-21T11:00:22.465 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-21T11:00:22.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-21T11:00:22.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-21T11:00:22.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-21T11:00:22.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-21T11:00:22.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-21T11:00:22.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-21T11:00:22.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-21T11:00:22.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-21T11:00:22.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-21T11:00:22.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-21T11:00:22.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-21T11:00:22.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-21T11:00:22.466 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-21T11:00:22.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-21T11:00:22.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-21T11:00:22.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-21T11:00:22.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-21T11:00:22.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-21T11:00:22.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-21T11:00:22.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-21T11:00:22.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-21T11:00:22.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-21T11:00:22.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-21T11:00:22.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-21T11:00:22.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-21T11:00:22.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-21T11:00:22.467 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-21T11:00:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-21T11:00:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-21T11:00:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-21T11:00:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-21T11:00:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-21T11:00:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-21T11:00:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-21T11:00:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-21T11:00:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-21T11:00:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-21T11:00:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-21T11:00:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-21T11:00:22.468 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-21T11:00:22.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-21T11:00:22.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-21T11:00:22.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-21T11:00:22.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-21T11:00:22.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-21T11:00:22.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-21T11:00:22.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-21T11:00:22.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-21T11:00:22.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-21T11:00:22.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-21T11:00:22.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-21T11:00:22.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-21T11:00:22.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-21T11:00:22.469 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-21T11:00:22.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-21T11:00:22.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-21T11:00:22.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-21T11:00:22.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-21T11:00:22.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-21T11:00:22.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-21T11:00:22.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-21T11:00:22.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-21T11:00:22.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-21T11:00:22.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-21T11:00:22.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-21T11:00:22.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-21T11:00:22.470 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-21T11:00:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-21T11:00:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-21T11:00:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-21T11:00:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-21T11:00:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-21T11:00:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-21T11:00:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-21T11:00:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-21T11:00:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-21T11:00:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-21T11:00:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-21T11:00:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-21T11:00:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-21T11:00:22.471 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-21T11:00:22.472 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-21T11:00:22.472 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-21T11:00:22.472 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-21T11:00:22.472 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-21T11:00:22.472 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-21T11:00:22.472 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-21T11:00:22.472 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-21T11:00:22.472 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-21T11:00:22.472 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-21T11:00:22.472 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-21T11:00:22.472 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-21T11:00:22.472 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-21T11:00:22.472 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-21T11:00:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-21T11:00:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-21T11:00:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-21T11:00:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-21T11:00:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-21T11:00:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-21T11:00:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-21T11:00:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-21T11:00:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-21T11:00:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-21T11:00:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-21T11:00:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-21T11:00:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-21T11:00:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-21T11:00:22.474 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-21T11:00:22.474 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-21T11:00:22.474 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-21T11:00:22.474 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-21T11:00:22.474 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-21T11:00:22.474 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-21T11:00:22.474 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-21T11:00:22.474 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-21T11:00:22.474 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-21T11:00:22.474 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-21T11:00:22.474 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-21T11:00:22.474 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-21T11:00:22.475 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-21T11:00:22.475 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-21T11:00:22.475 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-21T11:00:22.475 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-21T11:00:22.475 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-21T11:00:22.475 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-21T11:00:22.475 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-21T11:00:22.475 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-21T11:00:22.475 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-21T11:00:22.475 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-21T11:00:22.475 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-21T11:00:22.475 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-21T11:00:22.475 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-21T11:00:22.475 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-21T11:00:22.476 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-21T11:00:22.476 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-21T11:00:22.476 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-21T11:00:22.476 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-21T11:00:22.476 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-21T11:00:22.476 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-21T11:00:22.476 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-21T11:00:22.476 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-21T11:00:22.476 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-21T11:00:22.476 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-21T11:00:22.476 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-21T11:00:22.476 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-21T11:00:22.476 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-21T11:00:22.476 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-21T11:00:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-21T11:00:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-21T11:00:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-21T11:00:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-21T11:00:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-21T11:00:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-21T11:00:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-21T11:00:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-21T11:00:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-21T11:00:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-21T11:00:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-21T11:00:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-21T11:00:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-21T11:00:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-21T11:00:22.478 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-21T11:00:22.478 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-21T11:00:22.478 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-21T11:00:22.478 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-21T11:00:22.478 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-21T11:00:22.478 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-21T11:00:22.478 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-21T11:00:22.478 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-21T11:00:22.478 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-21T11:00:22.478 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-21T11:00:22.478 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-21T11:00:22.478 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-21T11:00:22.479 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-21T11:00:22.479 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-21T11:00:22.479 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-21T11:00:22.479 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-21T11:00:22.479 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-21T11:00:22.479 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-21T11:00:22.479 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-21T11:00:22.479 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-21T11:00:22.479 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-21T11:00:22.479 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-21T11:00:22.479 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-21T11:00:22.479 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-21T11:00:22.479 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-21T11:00:22.479 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-21T11:00:22.480 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-21T11:00:22.480 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-21T11:00:22.480 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-21T11:00:22.480 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-21T11:00:22.480 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-21T11:00:22.480 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-21T11:00:22.480 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-21T11:00:22.480 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-21T11:00:22.480 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-21T11:00:22.480 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-21T11:00:22.480 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-21T11:00:22.480 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-21T11:00:22.480 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-21T11:00:22.480 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-21T11:00:22.481 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-21T11:00:22.481 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-21T11:00:22.481 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-21T11:00:22.481 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-21T11:00:22.481 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-21T11:00:22.481 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-21T11:00:22.481 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-21T11:00:22.481 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-21T11:00:22.481 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-21T11:00:22.481 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-21T11:00:22.481 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-21T11:00:22.481 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-21T11:00:22.481 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-21T11:00:22.481 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-21T11:00:22.482 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-21T11:00:22.482 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-21T11:00:22.482 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-21T11:00:22.482 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-21T11:00:22.482 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-21T11:00:22.482 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-21T11:00:22.482 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-21T11:00:22.482 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-21T11:00:22.482 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-21T11:00:22.482 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-21T11:00:22.482 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-21T11:00:22.482 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-21T11:00:22.482 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-21T11:00:22.483 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-21T11:00:22.483 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-21T11:00:22.483 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-21T11:00:22.483 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-21T11:00:22.483 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-21T11:00:22.483 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-21T11:00:22.483 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-21T11:00:22.483 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-21T11:00:22.483 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-21T11:00:22.483 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-21T11:00:22.483 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-21T11:00:22.483 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-21T11:00:22.483 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-21T11:00:22.484 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-21T11:00:22.484 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-21T11:00:22.484 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-21T11:00:22.484 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-21T11:00:22.484 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-21T11:00:22.484 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-21T11:00:22.484 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-21T11:00:22.484 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-21T11:00:22.484 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-21T11:00:22.484 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-21T11:00:22.484 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-21T11:00:22.484 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-21T11:00:22.484 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-21T11:00:22.484 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-21T11:00:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-21T11:00:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-21T11:00:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-21T11:00:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-21T11:00:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-21T11:00:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-21T11:00:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-21T11:00:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-21T11:00:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-21T11:00:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-21T11:00:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-21T11:00:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-21T11:00:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-21T11:00:22.485 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-21T11:00:22.486 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-21T11:00:22.486 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-21T11:00:22.486 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-21T11:00:22.486 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-21T11:00:22.486 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-21T11:00:22.486 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-21T11:00:22.486 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-21T11:00:22.486 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-21T11:00:22.486 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-21T11:00:22.486 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-21T11:00:22.486 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-21T11:00:22.486 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T11:00:22.490 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-21T11:00:22.491 DEBUG:teuthology.orchestra.run.smithi047:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-21T11:00:22.515 INFO:teuthology.orchestra.run.smithi047.stdout:postinstall scriptlet (using /bin/sh): 2024-03-21T11:00:22.515 INFO:teuthology.orchestra.run.smithi047.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-21T11:00:22.515 INFO:teuthology.orchestra.run.smithi047.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T11:00:22.515 INFO:teuthology.orchestra.run.smithi047.stdout:fi 2024-03-21T11:00:22.515 INFO:teuthology.orchestra.run.smithi047.stdout:for file in vmlinuz System.map config; do 2024-03-21T11:00:22.515 INFO:teuthology.orchestra.run.smithi047.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-21T11:00:22.515 INFO:teuthology.orchestra.run.smithi047.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-21T11:00:22.515 INFO:teuthology.orchestra.run.smithi047.stdout: fi 2024-03-21T11:00:22.516 INFO:teuthology.orchestra.run.smithi047.stdout:done 2024-03-21T11:00:22.516 INFO:teuthology.orchestra.run.smithi047.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-21T11:00:22.516 INFO:teuthology.orchestra.run.smithi047.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-21T11:00:22.516 INFO:teuthology.orchestra.run.smithi047.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-21T11:00:22.516 INFO:teuthology.orchestra.run.smithi047.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-21T11:00:22.516 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-21T11:00:22.516 INFO:teuthology.orchestra.run.smithi047.stdout:fi 2024-03-21T11:00:22.516 INFO:teuthology.orchestra.run.smithi047.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-21T11:00:22.516 INFO:teuthology.orchestra.run.smithi047.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-21T11:00:22.516 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-21T11:00:22.516 INFO:teuthology.orchestra.run.smithi047.stdout:fi 2024-03-21T11:00:22.517 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -qi grub2-tools 2024-03-21T11:00:22.599 INFO:teuthology.orchestra.run.smithi047.stdout:Name : grub2-tools 2024-03-21T11:00:22.599 INFO:teuthology.orchestra.run.smithi047.stdout:Epoch : 1 2024-03-21T11:00:22.599 INFO:teuthology.orchestra.run.smithi047.stdout:Version : 2.06 2024-03-21T11:00:22.599 INFO:teuthology.orchestra.run.smithi047.stdout:Release : 68.el9 2024-03-21T11:00:22.599 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: x86_64 2024-03-21T11:00:22.599 INFO:teuthology.orchestra.run.smithi047.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T11:00:22.599 INFO:teuthology.orchestra.run.smithi047.stdout:Group : Unspecified 2024-03-21T11:00:22.599 INFO:teuthology.orchestra.run.smithi047.stdout:Size : 8264254 2024-03-21T11:00:22.600 INFO:teuthology.orchestra.run.smithi047.stdout:License : GPLv3+ 2024-03-21T11:00:22.600 INFO:teuthology.orchestra.run.smithi047.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T11:00:22.600 INFO:teuthology.orchestra.run.smithi047.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T11:00:22.600 INFO:teuthology.orchestra.run.smithi047.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T11:00:22.600 INFO:teuthology.orchestra.run.smithi047.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T11:00:22.600 INFO:teuthology.orchestra.run.smithi047.stdout:Packager : builder@centos.org 2024-03-21T11:00:22.600 INFO:teuthology.orchestra.run.smithi047.stdout:Vendor : CentOS 2024-03-21T11:00:22.600 INFO:teuthology.orchestra.run.smithi047.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T11:00:22.600 INFO:teuthology.orchestra.run.smithi047.stdout:Summary : Support tools for GRUB. 2024-03-21T11:00:22.600 INFO:teuthology.orchestra.run.smithi047.stdout:Description : 2024-03-21T11:00:22.600 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:00:22.600 INFO:teuthology.orchestra.run.smithi047.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T11:00:22.600 INFO:teuthology.orchestra.run.smithi047.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T11:00:22.600 INFO:teuthology.orchestra.run.smithi047.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T11:00:22.601 INFO:teuthology.orchestra.run.smithi047.stdout:hardware devices. 2024-03-21T11:00:22.601 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:00:22.601 INFO:teuthology.orchestra.run.smithi047.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T11:00:22.602 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T11:00:22.602 INFO:teuthology.task.kernel:Updating grub on smithi047 to boot 6.8.0-g5ec3e6e88b2e 2024-03-21T11:00:22.602 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T11:00:23.235 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2024-03-21T11:00:24.676 INFO:teuthology.orchestra.run.smithi047.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T11:00:24.724 INFO:teuthology.orchestra.run.smithi047.stderr:done 2024-03-21T11:00:24.726 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-03-21T11:00:24.741 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.2oQziu4ufD 2024-03-21T11:00:24.741 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2oQziu4ufD 2024-03-21T11:00:24.813 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.2oQziu4ufD 2024-03-21T11:00:24.957 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.2oQziu4ufD is 6KB 2024-03-21T11:00:25.006 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.2oQziu4ufD 2024-03-21T11:00:25.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T11:00:25.091 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T11:00:25.091 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T11:00:25.091 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T11:00:25.091 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-21T11:00:25.093 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-21T11:00:25.244 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2024-03-21T11:00:55.247 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-21T11:00:55.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-21T11:00:55.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T11:01:13.630 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-03-21T11:01:22.631 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-21T11:01:22.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T11:01:25.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-03-21T11:01:37.695 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-21T11:01:37.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T11:01:40.767 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-03-21T11:01:55.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-21T11:01:55.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T11:02:27.849 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-03-21T11:02:28.507 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-21T11:02:28.507 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g5ec3e6e88b2e"... 2024-03-21T11:02:28.507 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-03-21T11:02:28.554 INFO:teuthology.orchestra.run.smithi047.stdout:6.8.0-g5ec3e6e88b2e 2024-03-21T11:02:28.554 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-21T11:02:28.554 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T11:02:28.555 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T11:02:29.556 DEBUG:teuthology.parallel:result is None 2024-03-21T11:02:29.556 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-21T11:02:29.564 INFO:teuthology.task.internal:Creating test directory... 2024-03-21T11:02:29.564 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T11:02:29.570 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T11:02:29.573 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T11:02:29.615 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-21T11:02:29.621 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-21T11:02:29.645 INFO:teuthology.task.internal:Creating archive directory... 2024-03-21T11:02:29.646 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T11:02:29.652 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T11:02:29.654 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T11:02:29.706 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-21T11:02:29.712 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-21T11:02:29.713 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T11:02:29.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T11:02:29.732 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T11:02:29.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T11:02:29.748 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T11:02:29.752 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T11:02:29.775 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T11:02:29.841 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T11:02:29.849 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T11:02:29.862 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T11:02:29.870 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T11:02:29.873 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T11:02:29.952 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T11:02:29.959 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-21T11:02:29.967 INFO:teuthology.task.internal:Configuring sudo... 2024-03-21T11:02:29.967 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T11:02:29.972 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T11:02:29.974 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T11:02:30.049 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-21T11:02:30.059 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-21T11:02:30.059 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T11:02:30.064 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T11:02:30.066 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T11:02:30.103 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T11:02:30.172 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T11:02:30.269 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T11:02:30.347 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T11:02:30.423 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:02:30.423 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T11:02:30.522 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T11:02:30.568 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T11:02:30.625 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T11:02:30.712 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T11:02:30.754 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T11:02:30.754 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T11:02:30.823 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T11:02:30.867 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T11:02:30.919 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T11:02:31.004 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T11:02:31.046 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-21T11:02:31.046 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T11:02:31.115 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-03-21T11:02:31.121 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-03-21T11:02:31.123 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-03-21T11:02:31.161 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T11:02:31.187 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T11:02:31.249 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T11:02:31.736 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-21T11:02:31.744 INFO:teuthology.task.internal:Starting timer... 2024-03-21T11:02:31.745 INFO:teuthology.run_tasks:Running task pcp... 2024-03-21T11:02:31.790 INFO:teuthology.run_tasks:Running task selinux... 2024-03-21T11:02:31.831 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-21T11:02:31.831 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-03-21T11:02:31.937 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-03-21T11:02:31.942 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-03-21T11:02:32.010 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-03-21T11:02:32.012 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-03-21T11:02:32.076 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-03-21T11:02:32.078 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-21T11:02:32.078 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-03-21T11:02:32.127 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-03-21T11:02:32.127 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-03-21T11:02:32.147 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-03-21T11:02:32.147 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-03-21T11:02:32.177 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-03-21T11:02:32.177 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi100.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi102.front.sepia.ceph.com': 'permissive'} 2024-03-21T11:02:32.177 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T11:02:32.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T11:02:32.249 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T11:02:32.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T11:02:32.278 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T11:02:32.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T11:02:32.307 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-21T11:02:32.307 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-03-21T11:02:32.396 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-03-21T11:02:32.432 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-03-21T11:02:32.474 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-21T11:02:32.486 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-21T11:02:32.492 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-21T11:02:32.621 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-21T11:02:32.637 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-21T11:02:32.638 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi100.front.sepia.ceph.com,smithi102.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-21T11:08:36.699 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi102.front.sepia.ceph.com')] 2024-03-21T11:08:36.700 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-21T11:08:36.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T11:08:36.854 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-03-21T11:08:37.015 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-21T11:08:37.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-21T11:08:37.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T11:08:37.103 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-03-21T11:08:37.174 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-21T11:08:37.175 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-03-21T11:08:37.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T11:08:37.263 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-03-21T11:08:37.329 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-03-21T11:08:37.330 INFO:teuthology.run_tasks:Running task clock... 2024-03-21T11:08:37.344 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-21T11:08:37.345 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T11:08:37.345 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T11:08:37.356 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T11:08:37.356 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T11:08:37.358 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T11:08:37.358 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T11:08:37.390 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T11:08:37.401 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T11:08:37.408 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T11:08:37.417 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T11:08:37.438 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-03-21T11:08:37.448 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-03-21T11:08:37.452 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-03-21T11:08:37.457 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T11:08:37.461 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-03-21T11:08:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T11:08:37.478 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T11:08:37.486 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T11:08:37.494 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T11:08:37.512 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T11:08:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-03-21T11:08:37.533 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T11:08:37.533 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-03-21T11:08:37.533 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T11:08:37.534 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T11:08:37.534 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T11:08:37.534 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T11:08:37.537 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-03-21T11:08:37.541 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T11:08:37.541 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-03-21T11:08:37.541 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T11:08:37.541 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T11:08:37.541 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T11:08:37.541 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T11:08:37.615 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-03-21T11:08:37.664 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-03-21T11:08:37.721 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T11:08:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T11:08:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-03-21T11:08:37.926 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T11:08:37.926 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-03-21T11:08:37.926 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T11:08:37.926 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T11:08:37.926 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T11:08:37.926 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T11:08:37.927 INFO:teuthology.run_tasks:Running task install... 2024-03-21T11:08:37.936 DEBUG:teuthology.task.install:project ceph 2024-03-21T11:08:37.936 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11'}} 2024-03-21T11:08:37.936 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11'} 2024-03-21T11:08:37.936 INFO:teuthology.task.install:Using flavor: default 2024-03-21T11:08:37.942 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-21T11:08:37.942 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-21T11:08:37.942 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T11:08:37.943 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T11:08:37.945 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T11:08:37.945 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T11:08:37.946 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T11:08:37.946 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T11:08:38.089 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T11:08:38.089 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T11:08:38.171 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T11:08:38.171 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T11:08:38.175 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T11:08:38.175 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T11:08:38.260 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T11:08:38.261 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-21T11:08:38.261 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T11:08:38.276 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T11:08:38.276 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:08:38.276 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T11:08:38.290 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T11:08:38.290 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T11:08:38.341 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T11:08:38.341 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T11:08:38.341 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T11:08:38.359 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T11:08:38.373 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T11:08:38.374 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T11:08:38.383 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T11:08:38.383 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T11:08:38.439 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T11:08:38.444 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T11:08:38.506 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-03-21T11:08:38.507 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-03-21T11:08:38.524 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T11:08:38.526 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T11:08:38.593 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-03-21T11:08:38.594 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-03-21T11:08:38.671 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T11:08:38.742 INFO:teuthology.orchestra.run.smithi102.stdout:74 files removed 2024-03-21T11:08:38.768 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T11:08:38.769 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-03-21T11:08:38.774 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-03-21T11:08:38.828 INFO:teuthology.orchestra.run.smithi100.stdout:74 files removed 2024-03-21T11:08:38.854 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T11:08:39.134 INFO:teuthology.orchestra.run.smithi047.stdout:74 files removed 2024-03-21T11:08:39.175 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T11:08:39.462 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 205 kB/s | 83 kB 00:00 2024-03-21T11:08:39.468 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 255 kB/s | 83 kB 00:00 2024-03-21T11:08:39.852 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 29 kB/s | 10 kB 00:00 2024-03-21T11:08:39.938 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 23 kB/s | 10 kB 00:00 2024-03-21T11:08:40.252 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-03-21T11:08:40.258 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 148 kB/s | 83 kB 00:00 2024-03-21T11:08:40.330 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-03-21T11:08:40.734 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-21T11:08:40.772 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-21T11:08:41.216 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS-9 - AppStream 7.8 MB/s | 6.7 MB 00:00 2024-03-21T11:08:41.242 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-03-21T11:08:41.871 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-21T11:08:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2024-03-21T11:08:42.616 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-03-21T11:08:43.340 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.0 MB 00:00 2024-03-21T11:08:43.413 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.0 MB 00:00 2024-03-21T11:08:43.478 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2024-03-21T11:08:45.133 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-03-21T11:08:45.275 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.0 MB 00:01 2024-03-21T11:08:45.497 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-03-21T11:08:47.227 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-03-21T11:08:49.201 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.1 MB 00:00 2024-03-21T11:08:50.610 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-03-21T11:08:51.677 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-21T11:08:52.848 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.1 MB 00:03 2024-03-21T11:08:53.211 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 16 kB 00:00 2024-03-21T11:08:54.265 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-03-21T11:08:54.513 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-03-21T11:08:55.183 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-03-21T11:09:00.075 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-03-21T11:09:00.521 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-21T11:09:01.302 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 2.0 MB/s | 21 MB 00:10 2024-03-21T11:09:02.412 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T11:09:02.412 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T11:09:02.424 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T11:09:02.425 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T11:09:02.426 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T11:09:02.427 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T11:09:02.429 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T11:09:02.431 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T11:09:02.432 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T11:09:02.510 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-21T11:09:02.531 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-03-21T11:09:02.531 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-03-21T11:09:02.531 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-03-21T11:09:02.531 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-21T11:09:02.531 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T11:09:02.531 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T11:09:02.532 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T11:09:02.532 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T11:09:02.532 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T11:09:02.532 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T11:09:02.532 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T11:09:02.532 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T11:09:02.532 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T11:09:02.532 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T11:09:02.532 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T11:09:02.532 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T11:09:02.532 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T11:09:02.532 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T11:09:02.532 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T11:09:02.533 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T11:09:02.533 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T11:09:02.533 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T11:09:02.533 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T11:09:02.533 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T11:09:02.533 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T11:09:02.533 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T11:09:02.533 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T11:09:02.533 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T11:09:02.533 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T11:09:02.533 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T11:09:02.534 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T11:09:02.534 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T11:09:02.534 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T11:09:02.534 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T11:09:02.534 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T11:09:02.534 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T11:09:02.534 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T11:09:02.534 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T11:09:02.534 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T11:09:02.534 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T11:09:02.534 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T11:09:02.535 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-03-21T11:09:02.535 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T11:09:02.535 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T11:09:02.535 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T11:09:02.535 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T11:09:02.535 INFO:teuthology.orchestra.run.smithi100.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T11:09:02.535 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T11:09:02.535 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T11:09:02.535 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T11:09:02.535 INFO:teuthology.orchestra.run.smithi100.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T11:09:02.535 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T11:09:02.535 INFO:teuthology.orchestra.run.smithi100.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T11:09:02.535 INFO:teuthology.orchestra.run.smithi100.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T11:09:02.536 INFO:teuthology.orchestra.run.smithi100.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T11:09:02.536 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T11:09:02.536 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T11:09:02.536 INFO:teuthology.orchestra.run.smithi100.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T11:09:02.536 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T11:09:02.536 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T11:09:02.536 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T11:09:02.536 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T11:09:02.536 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T11:09:02.536 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T11:09:02.536 INFO:teuthology.orchestra.run.smithi100.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T11:09:02.536 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T11:09:02.536 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-03-21T11:09:02.536 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T11:09:02.537 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T11:09:02.537 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T11:09:02.537 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T11:09:02.537 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T11:09:02.537 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T11:09:02.537 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T11:09:02.537 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T11:09:02.537 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T11:09:02.537 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T11:09:02.537 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T11:09:02.537 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T11:09:02.537 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T11:09:02.538 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T11:09:02.538 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T11:09:02.538 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T11:09:02.538 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T11:09:02.538 INFO:teuthology.orchestra.run.smithi100.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T11:09:02.538 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T11:09:02.538 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T11:09:02.538 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T11:09:02.538 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T11:09:02.538 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T11:09:02.538 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T11:09:02.538 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T11:09:02.538 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T11:09:02.539 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T11:09:02.539 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T11:09:02.539 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T11:09:02.539 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T11:09:02.539 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T11:09:02.539 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T11:09:02.539 INFO:teuthology.orchestra.run.smithi100.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T11:09:02.539 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T11:09:02.539 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T11:09:02.539 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T11:09:02.539 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T11:09:02.539 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T11:09:02.539 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T11:09:02.539 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T11:09:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T11:09:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T11:09:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T11:09:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T11:09:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T11:09:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T11:09:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T11:09:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T11:09:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T11:09:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T11:09:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T11:09:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T11:09:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T11:09:02.540 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T11:09:02.541 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T11:09:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T11:09:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T11:09:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T11:09:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T11:09:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T11:09:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T11:09:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T11:09:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T11:09:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T11:09:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T11:09:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T11:09:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T11:09:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T11:09:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T11:09:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T11:09:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T11:09:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T11:09:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T11:09:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T11:09:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T11:09:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T11:09:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T11:09:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T11:09:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T11:09:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T11:09:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T11:09:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T11:09:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T11:09:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T11:09:02.545 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-03-21T11:09:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T11:09:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:09:02.545 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-21T11:09:02.545 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-03-21T11:09:02.545 INFO:teuthology.orchestra.run.smithi100.stdout:Install 150 Packages 2024-03-21T11:09:02.545 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 24 Packages 2024-03-21T11:09:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:09:02.546 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 224 M 2024-03-21T11:09:02.546 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-21T11:09:03.118 INFO:teuthology.orchestra.run.smithi100.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 41 kB/s | 6.6 kB 00:00 2024-03-21T11:09:03.167 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T11:09:03.170 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T11:09:03.192 INFO:teuthology.orchestra.run.smithi047.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T11:09:03.193 INFO:teuthology.orchestra.run.smithi047.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T11:09:03.195 INFO:teuthology.orchestra.run.smithi047.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T11:09:03.197 INFO:teuthology.orchestra.run.smithi047.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T11:09:03.199 INFO:teuthology.orchestra.run.smithi047.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T11:09:03.203 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T11:09:03.205 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T11:09:03.300 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-03-21T11:09:03.309 INFO:teuthology.orchestra.run.smithi100.stdout:(2/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 4.5 MB/s | 890 kB 00:00 2024-03-21T11:09:03.320 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T11:09:03.321 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T11:09:03.322 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T11:09:03.322 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T11:09:03.322 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T11:09:03.322 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T11:09:03.322 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T11:09:03.322 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T11:09:03.322 INFO:teuthology.orchestra.run.smithi047.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T11:09:03.322 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T11:09:03.322 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T11:09:03.322 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T11:09:03.322 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T11:09:03.322 INFO:teuthology.orchestra.run.smithi047.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T11:09:03.323 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T11:09:03.323 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T11:09:03.323 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T11:09:03.323 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T11:09:03.323 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T11:09:03.323 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T11:09:03.323 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T11:09:03.323 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T11:09:03.323 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T11:09:03.323 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T11:09:03.323 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T11:09:03.323 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T11:09:03.324 INFO:teuthology.orchestra.run.smithi047.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T11:09:03.325 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T11:09:03.326 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T11:09:03.327 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T11:09:03.327 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T11:09:03.327 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T11:09:03.327 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T11:09:03.327 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T11:09:03.327 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T11:09:03.327 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T11:09:03.327 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T11:09:03.327 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T11:09:03.327 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T11:09:03.327 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T11:09:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T11:09:03.329 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T11:09:03.330 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T11:09:03.330 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T11:09:03.330 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T11:09:03.330 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T11:09:03.330 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T11:09:03.330 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T11:09:03.330 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T11:09:03.330 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T11:09:03.330 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T11:09:03.330 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T11:09:03.330 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T11:09:03.330 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T11:09:03.330 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T11:09:03.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T11:09:03.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T11:09:03.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T11:09:03.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T11:09:03.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T11:09:03.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T11:09:03.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T11:09:03.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T11:09:03.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T11:09:03.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T11:09:03.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T11:09:03.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T11:09:03.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T11:09:03.331 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T11:09:03.332 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T11:09:03.332 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T11:09:03.332 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T11:09:03.332 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T11:09:03.332 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T11:09:03.332 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T11:09:03.332 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T11:09:03.332 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T11:09:03.332 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T11:09:03.332 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T11:09:03.332 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T11:09:03.332 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T11:09:03.332 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T11:09:03.332 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T11:09:03.333 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T11:09:03.333 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T11:09:03.333 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T11:09:03.333 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T11:09:03.333 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T11:09:03.333 INFO:teuthology.orchestra.run.smithi047.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T11:09:03.333 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T11:09:03.333 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T11:09:03.333 INFO:teuthology.orchestra.run.smithi047.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T11:09:03.333 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T11:09:03.333 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T11:09:03.333 INFO:teuthology.orchestra.run.smithi047.stdout:Installing weak dependencies: 2024-03-21T11:09:03.334 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T11:09:03.334 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:09:03.334 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-03-21T11:09:03.334 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-03-21T11:09:03.334 INFO:teuthology.orchestra.run.smithi047.stdout:Install 150 Packages 2024-03-21T11:09:03.334 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 24 Packages 2024-03-21T11:09:03.334 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:09:03.334 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 224 M 2024-03-21T11:09:03.334 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-03-21T11:09:03.368 INFO:teuthology.orchestra.run.smithi100.stdout:(3/174): ceph-immutable-object-cache-19.0.0-223 2.4 MB/s | 145 kB 00:00 2024-03-21T11:09:03.652 INFO:teuthology.orchestra.run.smithi100.stdout:(4/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 8.2 MB/s | 2.3 MB 00:00 2024-03-21T11:09:03.860 INFO:teuthology.orchestra.run.smithi100.stdout:(5/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 5.9 MB/s | 5.4 MB 00:00 2024-03-21T11:09:03.902 INFO:teuthology.orchestra.run.smithi100.stdout:(6/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 6.0 MB/s | 1.5 MB 00:00 2024-03-21T11:09:04.244 INFO:teuthology.orchestra.run.smithi100.stdout:(7/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x86 12 MB/s | 4.7 MB 00:00 2024-03-21T11:09:04.986 INFO:teuthology.orchestra.run.smithi100.stdout:(8/174): ceph-radosgw-19.0.0-2231.g2edfee68.el9 13 MB/s | 9.9 MB 00:00 2024-03-21T11:09:05.070 INFO:teuthology.orchestra.run.smithi100.stdout:(9/174): ceph-selinux-19.0.0-2231.g2edfee68.el9 302 kB/s | 25 kB 00:00 2024-03-21T11:09:05.248 INFO:teuthology.orchestra.run.smithi047.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-03-21T11:09:05.612 INFO:teuthology.orchestra.run.smithi100.stdout:(10/174): ceph-common-19.0.0-2231.g2edfee68.el9 7.7 MB/s | 21 MB 00:02 2024-03-21T11:09:05.695 INFO:teuthology.orchestra.run.smithi100.stdout:(11/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 2.6 MB/s | 220 kB 00:00 2024-03-21T11:09:05.723 INFO:teuthology.orchestra.run.smithi047.stdout:(2/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 1.8 MB/s | 890 kB 00:00 2024-03-21T11:09:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:(12/174): cephfs-shell-19.0.0-2231.g2edfee68.el 232 kB/s | 27 kB 00:00 2024-03-21T11:09:05.864 INFO:teuthology.orchestra.run.smithi047.stdout:(3/174): ceph-immutable-object-cache-19.0.0-223 1.0 MB/s | 145 kB 00:00 2024-03-21T11:09:05.896 INFO:teuthology.orchestra.run.smithi100.stdout:(13/174): libcephfs-devel-19.0.0-2231.g2edfee68 376 kB/s | 31 kB 00:00 2024-03-21T11:09:06.096 INFO:teuthology.orchestra.run.smithi100.stdout:(14/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 3.6 MB/s | 747 kB 00:00 2024-03-21T11:09:06.196 INFO:teuthology.orchestra.run.smithi100.stdout:(15/174): libcephsqlite-19.0.0-2231.g2edfee68.e 1.6 MB/s | 163 kB 00:00 2024-03-21T11:09:06.227 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-03-21T11:09:06.305 INFO:teuthology.orchestra.run.smithi100.stdout:(16/174): librados-devel-19.0.0-2231.g2edfee68. 1.1 MB/s | 127 kB 00:00 2024-03-21T11:09:06.425 INFO:teuthology.orchestra.run.smithi100.stdout:(17/174): libradosstriper1-19.0.0-2231.g2edfee6 4.1 MB/s | 501 kB 00:00 2024-03-21T11:09:06.440 INFO:teuthology.orchestra.run.smithi047.stdout:(4/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 3.3 MB/s | 5.4 MB 00:01 2024-03-21T11:09:06.990 INFO:teuthology.orchestra.run.smithi047.stdout:(5/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 2.1 MB/s | 2.3 MB 00:01 2024-03-21T11:09:07.090 INFO:teuthology.orchestra.run.smithi047.stdout:(6/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 2.3 MB/s | 1.5 MB 00:00 2024-03-21T11:09:07.417 INFO:teuthology.orchestra.run.smithi100.stdout:(18/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x8 4.8 MB/s | 17 MB 00:03 2024-03-21T11:09:07.509 INFO:teuthology.orchestra.run.smithi100.stdout:(19/174): python3-ceph-argparse-19.0.0-2231.g2e 494 kB/s | 45 kB 00:00 2024-03-21T11:09:07.626 INFO:teuthology.orchestra.run.smithi100.stdout:(20/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 4.3 MB/s | 5.1 MB 00:01 2024-03-21T11:09:07.652 INFO:teuthology.orchestra.run.smithi100.stdout:(21/174): python3-ceph-common-19.0.0-2231.g2edf 924 kB/s | 131 kB 00:00 2024-03-21T11:09:07.735 INFO:teuthology.orchestra.run.smithi100.stdout:(22/174): python3-cephfs-19.0.0-2231.g2edfee68. 1.5 MB/s | 161 kB 00:00 2024-03-21T11:09:07.827 INFO:teuthology.orchestra.run.smithi100.stdout:(23/174): python3-rados-19.0.0-2231.g2edfee68.e 1.8 MB/s | 320 kB 00:00 2024-03-21T11:09:07.911 INFO:teuthology.orchestra.run.smithi100.stdout:(24/174): python3-rgw-19.0.0-2231.g2edfee68.el9 1.2 MB/s | 99 kB 00:00 2024-03-21T11:09:07.936 INFO:teuthology.orchestra.run.smithi100.stdout:(25/174): python3-rbd-19.0.0-2231.g2edfee68.el9 1.4 MB/s | 297 kB 00:00 2024-03-21T11:09:07.995 INFO:teuthology.orchestra.run.smithi100.stdout:(26/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-21T11:09:08.078 INFO:teuthology.orchestra.run.smithi100.stdout:(27/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 2.0 MB/s | 171 kB 00:00 2024-03-21T11:09:08.120 INFO:teuthology.orchestra.run.smithi047.stdout:(7/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x86 4.2 MB/s | 4.7 MB 00:01 2024-03-21T11:09:08.154 INFO:teuthology.orchestra.run.smithi100.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2231.g 453 kB/s | 34 kB 00:00 2024-03-21T11:09:08.237 INFO:teuthology.orchestra.run.smithi100.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 1.6 MB/s | 140 kB 00:00 2024-03-21T11:09:08.549 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T11:09:08.550 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T11:09:08.565 INFO:teuthology.orchestra.run.smithi102.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T11:09:08.566 INFO:teuthology.orchestra.run.smithi102.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T11:09:08.567 INFO:teuthology.orchestra.run.smithi102.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T11:09:08.568 INFO:teuthology.orchestra.run.smithi102.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T11:09:08.570 INFO:teuthology.orchestra.run.smithi102.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T11:09:08.573 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T11:09:08.574 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T11:09:08.655 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-03-21T11:09:08.663 INFO:teuthology.orchestra.run.smithi100.stdout:(30/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 13 MB/s | 48 MB 00:03 2024-03-21T11:09:08.675 INFO:teuthology.orchestra.run.smithi102.stdout:============================================================================================ 2024-03-21T11:09:08.675 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-03-21T11:09:08.675 INFO:teuthology.orchestra.run.smithi102.stdout:============================================================================================ 2024-03-21T11:09:08.675 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-03-21T11:09:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T11:09:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T11:09:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T11:09:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T11:09:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T11:09:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T11:09:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T11:09:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T11:09:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T11:09:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T11:09:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T11:09:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T11:09:08.676 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T11:09:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T11:09:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T11:09:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T11:09:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T11:09:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T11:09:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T11:09:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T11:09:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T11:09:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T11:09:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T11:09:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T11:09:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T11:09:08.677 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T11:09:08.678 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T11:09:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T11:09:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T11:09:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T11:09:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T11:09:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T11:09:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T11:09:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T11:09:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T11:09:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T11:09:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T11:09:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T11:09:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T11:09:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T11:09:08.679 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T11:09:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T11:09:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T11:09:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T11:09:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T11:09:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T11:09:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T11:09:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T11:09:08.680 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-03-21T11:09:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T11:09:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T11:09:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T11:09:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T11:09:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T11:09:08.680 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T11:09:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T11:09:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T11:09:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T11:09:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T11:09:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T11:09:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T11:09:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T11:09:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T11:09:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T11:09:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T11:09:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T11:09:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T11:09:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T11:09:08.681 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T11:09:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T11:09:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T11:09:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T11:09:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T11:09:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T11:09:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T11:09:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T11:09:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T11:09:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T11:09:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T11:09:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T11:09:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T11:09:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T11:09:08.682 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T11:09:08.683 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T11:09:08.684 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T11:09:08.685 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T11:09:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T11:09:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T11:09:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T11:09:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T11:09:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T11:09:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T11:09:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T11:09:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T11:09:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T11:09:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T11:09:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T11:09:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T11:09:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T11:09:08.686 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T11:09:08.687 INFO:teuthology.orchestra.run.smithi102.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T11:09:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T11:09:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T11:09:08.688 INFO:teuthology.orchestra.run.smithi102.stdout:Installing weak dependencies: 2024-03-21T11:09:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T11:09:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:09:08.688 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-03-21T11:09:08.688 INFO:teuthology.orchestra.run.smithi102.stdout:============================================================================================ 2024-03-21T11:09:08.688 INFO:teuthology.orchestra.run.smithi102.stdout:Install 150 Packages 2024-03-21T11:09:08.688 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 24 Packages 2024-03-21T11:09:08.688 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:09:08.688 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 224 M 2024-03-21T11:09:08.689 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-03-21T11:09:08.821 INFO:teuthology.orchestra.run.smithi100.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 3.1 MB/s | 1.8 MB 00:00 2024-03-21T11:09:08.872 INFO:teuthology.orchestra.run.smithi100.stdout:(32/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 3.3 MB/s | 3.1 MB 00:00 2024-03-21T11:09:08.922 INFO:teuthology.orchestra.run.smithi100.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2231.g2e 2.4 MB/s | 248 kB 00:00 2024-03-21T11:09:08.939 INFO:teuthology.orchestra.run.smithi100.stdout:(34/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 739 kB/s | 49 kB 00:00 2024-03-21T11:09:08.972 INFO:teuthology.orchestra.run.smithi100.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2231.g2 327 kB/s | 16 kB 00:00 2024-03-21T11:09:09.014 INFO:teuthology.orchestra.run.smithi100.stdout:(36/174): ceph-volume-19.0.0-2231.g2edfee68.el9 3.4 MB/s | 264 kB 00:00 2024-03-21T11:09:09.016 INFO:teuthology.orchestra.run.smithi047.stdout:(8/174): ceph-radosgw-19.0.0-2231.g2edfee68.el9 11 MB/s | 9.9 MB 00:00 2024-03-21T11:09:09.081 INFO:teuthology.orchestra.run.smithi100.stdout:(37/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 274 kB/s | 18 kB 00:00 2024-03-21T11:09:09.083 INFO:teuthology.orchestra.run.smithi047.stdout:(9/174): ceph-selinux-19.0.0-2231.g2edfee68.el9 384 kB/s | 25 kB 00:00 2024-03-21T11:09:09.148 INFO:teuthology.orchestra.run.smithi100.stdout:(38/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 4.2 MB/s | 750 kB 00:00 2024-03-21T11:09:09.240 INFO:teuthology.orchestra.run.smithi100.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 9.9 MB/s | 920 kB 00:00 2024-03-21T11:09:09.274 INFO:teuthology.orchestra.run.smithi100.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2024-03-21T11:09:09.299 INFO:teuthology.orchestra.run.smithi100.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-21T11:09:09.324 INFO:teuthology.orchestra.run.smithi100.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-21T11:09:09.341 INFO:teuthology.orchestra.run.smithi100.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 763 kB/s | 32 kB 00:00 2024-03-21T11:09:09.425 INFO:teuthology.orchestra.run.smithi100.stdout:(44/174): ceph-mgr-diskprediction-local-19.0.0- 9.7 MB/s | 7.4 MB 00:00 2024-03-21T11:09:09.475 INFO:teuthology.orchestra.run.smithi100.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-03-21T11:09:09.492 INFO:teuthology.orchestra.run.smithi100.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 115 kB/s | 17 kB 00:00 2024-03-21T11:09:09.509 INFO:teuthology.orchestra.run.smithi100.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 124 kB/s | 10 kB 00:00 2024-03-21T11:09:09.526 INFO:teuthology.orchestra.run.smithi100.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-21T11:09:09.537 INFO:teuthology.orchestra.run.smithi102.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-03-21T11:09:09.560 INFO:teuthology.orchestra.run.smithi100.stdout:(49/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 159 kB 00:00 2024-03-21T11:09:09.593 INFO:teuthology.orchestra.run.smithi100.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-03-21T11:09:09.627 INFO:teuthology.orchestra.run.smithi100.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 578 kB 00:00 2024-03-21T11:09:09.644 INFO:teuthology.orchestra.run.smithi100.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 938 kB/s | 46 kB 00:00 2024-03-21T11:09:09.669 INFO:teuthology.orchestra.run.smithi100.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-21T11:09:09.694 INFO:teuthology.orchestra.run.smithi100.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2024-03-21T11:09:09.719 INFO:teuthology.orchestra.run.smithi100.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 686 kB/s | 34 kB 00:00 2024-03-21T11:09:09.745 INFO:teuthology.orchestra.run.smithi100.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-21T11:09:09.828 INFO:teuthology.orchestra.run.smithi100.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2024-03-21T11:09:09.879 INFO:teuthology.orchestra.run.smithi100.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 9.8 MB/s | 1.5 MB 00:00 2024-03-21T11:09:09.912 INFO:teuthology.orchestra.run.smithi100.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 4.8 MB/s | 811 kB 00:00 2024-03-21T11:09:09.937 INFO:teuthology.orchestra.run.smithi100.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.2 MB/s | 134 kB 00:00 2024-03-21T11:09:09.971 INFO:teuthology.orchestra.run.smithi100.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-03-21T11:09:10.079 INFO:teuthology.orchestra.run.smithi102.stdout:(2/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 1.6 MB/s | 890 kB 00:00 2024-03-21T11:09:10.096 INFO:teuthology.orchestra.run.smithi100.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2024-03-21T11:09:10.122 INFO:teuthology.orchestra.run.smithi100.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 422 kB/s | 78 kB 00:00 2024-03-21T11:09:10.196 INFO:teuthology.orchestra.run.smithi102.stdout:(3/174): ceph-immutable-object-cache-19.0.0-223 1.2 MB/s | 145 kB 00:00 2024-03-21T11:09:10.222 INFO:teuthology.orchestra.run.smithi100.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-03-21T11:09:10.247 INFO:teuthology.orchestra.run.smithi100.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 24 kB 00:00 2024-03-21T11:09:10.272 INFO:teuthology.orchestra.run.smithi100.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 122 kB/s | 18 kB 00:00 2024-03-21T11:09:10.297 INFO:teuthology.orchestra.run.smithi100.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-21T11:09:10.323 INFO:teuthology.orchestra.run.smithi100.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 632 kB/s | 32 kB 00:00 2024-03-21T11:09:10.431 INFO:teuthology.orchestra.run.smithi100.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2024-03-21T11:09:10.466 INFO:teuthology.orchestra.run.smithi100.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-21T11:09:10.491 INFO:teuthology.orchestra.run.smithi100.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-03-21T11:09:10.516 INFO:teuthology.orchestra.run.smithi100.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 819 kB/s | 69 kB 00:00 2024-03-21T11:09:10.541 INFO:teuthology.orchestra.run.smithi100.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2024-03-21T11:09:10.575 INFO:teuthology.orchestra.run.smithi100.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-03-21T11:09:10.600 INFO:teuthology.orchestra.run.smithi100.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-03-21T11:09:10.625 INFO:teuthology.orchestra.run.smithi100.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 951 kB/s | 47 kB 00:00 2024-03-21T11:09:10.659 INFO:teuthology.orchestra.run.smithi100.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2024-03-21T11:09:10.685 INFO:teuthology.orchestra.run.smithi100.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 512 kB/s | 43 kB 00:00 2024-03-21T11:09:11.118 INFO:teuthology.orchestra.run.smithi100.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-03-21T11:09:11.135 INFO:teuthology.orchestra.run.smithi100.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 86 kB/s | 39 kB 00:00 2024-03-21T11:09:11.152 INFO:teuthology.orchestra.run.smithi100.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 88 kB/s | 41 kB 00:00 2024-03-21T11:09:11.177 INFO:teuthology.orchestra.run.smithi100.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-03-21T11:09:11.188 INFO:teuthology.orchestra.run.smithi102.stdout:(4/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 2.3 MB/s | 2.3 MB 00:00 2024-03-21T11:09:11.194 INFO:teuthology.orchestra.run.smithi100.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-03-21T11:09:11.211 INFO:teuthology.orchestra.run.smithi100.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 63 kB 00:00 2024-03-21T11:09:11.228 INFO:teuthology.orchestra.run.smithi100.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 877 kB/s | 44 kB 00:00 2024-03-21T11:09:11.253 INFO:teuthology.orchestra.run.smithi100.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.7 MB/s | 241 kB 00:00 2024-03-21T11:09:11.279 INFO:teuthology.orchestra.run.smithi100.stdout:(87/174): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 79 kB 00:00 2024-03-21T11:09:11.320 INFO:teuthology.orchestra.run.smithi100.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-21T11:09:11.346 INFO:teuthology.orchestra.run.smithi100.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-21T11:09:11.371 INFO:teuthology.orchestra.run.smithi100.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-21T11:09:11.396 INFO:teuthology.orchestra.run.smithi100.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-21T11:09:11.505 INFO:teuthology.orchestra.run.smithi100.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-21T11:09:11.530 INFO:teuthology.orchestra.run.smithi100.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-03-21T11:09:11.555 INFO:teuthology.orchestra.run.smithi100.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-21T11:09:11.589 INFO:teuthology.orchestra.run.smithi100.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-03-21T11:09:11.781 INFO:teuthology.orchestra.run.smithi100.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 113 kB/s | 25 kB 00:00 2024-03-21T11:09:11.812 INFO:teuthology.orchestra.run.smithi102.stdout:(5/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 2.4 MB/s | 1.5 MB 00:00 2024-03-21T11:09:11.839 INFO:teuthology.orchestra.run.smithi100.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 69 kB/s | 21 kB 00:00 2024-03-21T11:09:11.896 INFO:teuthology.orchestra.run.smithi102.stdout:(6/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-03-21T11:09:11.940 INFO:teuthology.orchestra.run.smithi100.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-03-21T11:09:11.965 INFO:teuthology.orchestra.run.smithi100.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-03-21T11:09:12.032 INFO:teuthology.orchestra.run.smithi100.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.3 MB/s | 216 kB 00:00 2024-03-21T11:09:12.057 INFO:teuthology.orchestra.run.smithi100.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 379 kB/s | 35 kB 00:00 2024-03-21T11:09:12.099 INFO:teuthology.orchestra.run.smithi100.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 665 kB/s | 44 kB 00:00 2024-03-21T11:09:12.124 INFO:teuthology.orchestra.run.smithi100.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-03-21T11:09:12.166 INFO:teuthology.orchestra.run.smithi100.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 334 kB/s | 22 kB 00:00 2024-03-21T11:09:12.283 INFO:teuthology.orchestra.run.smithi100.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.3 MB/s | 863 kB 00:00 2024-03-21T11:09:12.513 INFO:teuthology.orchestra.run.smithi102.stdout:(7/174): ceph-common-19.0.0-2231.g2edfee68.el9. 6.5 MB/s | 21 MB 00:03 2024-03-21T11:09:12.550 INFO:teuthology.orchestra.run.smithi100.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.3 MB/s | 4.1 MB 00:00 2024-03-21T11:09:12.575 INFO:teuthology.orchestra.run.smithi100.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-03-21T11:09:12.609 INFO:teuthology.orchestra.run.smithi100.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-03-21T11:09:12.642 INFO:teuthology.orchestra.run.smithi100.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 211 kB/s | 19 kB 00:00 2024-03-21T11:09:12.668 INFO:teuthology.orchestra.run.smithi100.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 442 kB/s | 26 kB 00:00 2024-03-21T11:09:12.735 INFO:teuthology.orchestra.run.smithi100.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-03-21T11:09:12.810 INFO:teuthology.orchestra.run.smithi100.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-03-21T11:09:12.844 INFO:teuthology.orchestra.run.smithi100.stdout:(113/174): liboath-2.6.7-2.el9.x86_64.rpm 242 kB/s | 49 kB 00:00 2024-03-21T11:09:13.000 INFO:teuthology.orchestra.run.smithi047.stdout:(10/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 12 MB/s | 48 MB 00:03 2024-03-21T11:09:13.019 INFO:teuthology.orchestra.run.smithi100.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-03-21T11:09:13.061 INFO:teuthology.orchestra.run.smithi100.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2024-03-21T11:09:13.086 INFO:teuthology.orchestra.run.smithi100.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 650 kB/s | 43 kB 00:00 2024-03-21T11:09:13.111 INFO:teuthology.orchestra.run.smithi100.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-03-21T11:09:13.130 INFO:teuthology.orchestra.run.smithi102.stdout:(8/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x86 3.6 MB/s | 4.7 MB 00:01 2024-03-21T11:09:13.137 INFO:teuthology.orchestra.run.smithi100.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-03-21T11:09:13.196 INFO:teuthology.orchestra.run.smithi100.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-03-21T11:09:13.258 INFO:teuthology.orchestra.run.smithi047.stdout:(11/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x8 2.7 MB/s | 17 MB 00:06 2024-03-21T11:09:13.264 INFO:teuthology.orchestra.run.smithi102.stdout:(9/174): ceph-radosgw-19.0.0-2231.g2edfee68.el9 13 MB/s | 9.9 MB 00:00 2024-03-21T11:09:13.271 INFO:teuthology.orchestra.run.smithi100.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-03-21T11:09:13.289 INFO:teuthology.orchestra.run.smithi102.stdout:(10/174): ceph-selinux-19.0.0-2231.g2edfee68.el 159 kB/s | 25 kB 00:00 2024-03-21T11:09:13.291 INFO:teuthology.orchestra.run.smithi047.stdout:(12/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 758 kB/s | 220 kB 00:00 2024-03-21T11:09:13.313 INFO:teuthology.orchestra.run.smithi100.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-03-21T11:09:13.317 INFO:teuthology.orchestra.run.smithi047.stdout:(13/174): cephfs-shell-19.0.0-2231.g2edfee68.el 472 kB/s | 27 kB 00:00 2024-03-21T11:09:13.330 INFO:teuthology.orchestra.run.smithi100.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 603 kB/s | 35 kB 00:00 2024-03-21T11:09:13.348 INFO:teuthology.orchestra.run.smithi102.stdout:(11/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 3.7 MB/s | 220 kB 00:00 2024-03-21T11:09:13.350 INFO:teuthology.orchestra.run.smithi047.stdout:(14/174): libcephfs-devel-19.0.0-2231.g2edfee68 543 kB/s | 31 kB 00:00 2024-03-21T11:09:13.388 INFO:teuthology.orchestra.run.smithi100.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-21T11:09:13.406 INFO:teuthology.orchestra.run.smithi102.stdout:(12/174): cephfs-shell-19.0.0-2231.g2edfee68.el 462 kB/s | 27 kB 00:00 2024-03-21T11:09:13.417 INFO:teuthology.orchestra.run.smithi047.stdout:(15/174): libcephsqlite-19.0.0-2231.g2edfee68.e 2.4 MB/s | 163 kB 00:00 2024-03-21T11:09:13.447 INFO:teuthology.orchestra.run.smithi100.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-03-21T11:09:13.465 INFO:teuthology.orchestra.run.smithi102.stdout:(13/174): libcephfs-devel-19.0.0-2231.g2edfee68 538 kB/s | 31 kB 00:00 2024-03-21T11:09:13.489 INFO:teuthology.orchestra.run.smithi100.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 258 kB/s | 11 kB 00:00 2024-03-21T11:09:13.492 INFO:teuthology.orchestra.run.smithi047.stdout:(16/174): librados-devel-19.0.0-2231.g2edfee68. 1.7 MB/s | 127 kB 00:00 2024-03-21T11:09:13.531 INFO:teuthology.orchestra.run.smithi100.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 424 kB/s | 18 kB 00:00 2024-03-21T11:09:13.573 INFO:teuthology.orchestra.run.smithi100.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 558 kB/s | 23 kB 00:00 2024-03-21T11:09:13.576 INFO:teuthology.orchestra.run.smithi047.stdout:(17/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 2.8 MB/s | 747 kB 00:00 2024-03-21T11:09:13.615 INFO:teuthology.orchestra.run.smithi100.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 469 kB/s | 19 kB 00:00 2024-03-21T11:09:13.651 INFO:teuthology.orchestra.run.smithi047.stdout:(18/174): libradosstriper1-19.0.0-2231.g2edfee6 3.1 MB/s | 501 kB 00:00 2024-03-21T11:09:13.657 INFO:teuthology.orchestra.run.smithi100.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 473 kB/s | 20 kB 00:00 2024-03-21T11:09:13.690 INFO:teuthology.orchestra.run.smithi102.stdout:(14/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 3.2 MB/s | 747 kB 00:00 2024-03-21T11:09:13.715 INFO:teuthology.orchestra.run.smithi100.stdout:(130/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-03-21T11:09:13.726 INFO:teuthology.orchestra.run.smithi047.stdout:(19/174): python3-ceph-argparse-19.0.0-2231.g2e 610 kB/s | 45 kB 00:00 2024-03-21T11:09:13.757 INFO:teuthology.orchestra.run.smithi100.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-03-21T11:09:13.765 INFO:teuthology.orchestra.run.smithi102.stdout:(15/174): libcephsqlite-19.0.0-2231.g2edfee68.e 2.1 MB/s | 163 kB 00:00 2024-03-21T11:09:13.807 INFO:teuthology.orchestra.run.smithi100.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-03-21T11:09:13.810 INFO:teuthology.orchestra.run.smithi047.stdout:(20/174): python3-ceph-common-19.0.0-2231.g2edf 1.6 MB/s | 131 kB 00:00 2024-03-21T11:09:13.849 INFO:teuthology.orchestra.run.smithi100.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-03-21T11:09:13.857 INFO:teuthology.orchestra.run.smithi102.stdout:(16/174): librados-devel-19.0.0-2231.g2edfee68. 1.4 MB/s | 127 kB 00:00 2024-03-21T11:09:13.894 INFO:teuthology.orchestra.run.smithi047.stdout:(21/174): python3-cephfs-19.0.0-2231.g2edfee68. 1.9 MB/s | 161 kB 00:00 2024-03-21T11:09:13.933 INFO:teuthology.orchestra.run.smithi100.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-21T11:09:13.975 INFO:teuthology.orchestra.run.smithi100.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 396 kB/s | 16 kB 00:00 2024-03-21T11:09:13.986 INFO:teuthology.orchestra.run.smithi047.stdout:(22/174): python3-rados-19.0.0-2231.g2edfee68.e 3.4 MB/s | 320 kB 00:00 2024-03-21T11:09:14.034 INFO:teuthology.orchestra.run.smithi100.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-03-21T11:09:14.057 INFO:teuthology.orchestra.run.smithi102.stdout:(17/174): libradosstriper1-19.0.0-2231.g2edfee6 2.5 MB/s | 501 kB 00:00 2024-03-21T11:09:14.069 INFO:teuthology.orchestra.run.smithi047.stdout:(23/174): python3-rbd-19.0.0-2231.g2edfee68.el9 3.5 MB/s | 297 kB 00:00 2024-03-21T11:09:14.084 INFO:teuthology.orchestra.run.smithi100.stdout:(137/174): python3-kubernetes-26.1.0-2.el9.noar 2.4 MB/s | 1.0 MB 00:00 2024-03-21T11:09:14.109 INFO:teuthology.orchestra.run.smithi100.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 376 kB/s | 28 kB 00:00 2024-03-21T11:09:14.134 INFO:teuthology.orchestra.run.smithi100.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-03-21T11:09:14.153 INFO:teuthology.orchestra.run.smithi047.stdout:(24/174): python3-rgw-19.0.0-2231.g2edfee68.el9 1.2 MB/s | 99 kB 00:00 2024-03-21T11:09:14.176 INFO:teuthology.orchestra.run.smithi100.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-03-21T11:09:14.201 INFO:teuthology.orchestra.run.smithi100.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-03-21T11:09:14.227 INFO:teuthology.orchestra.run.smithi100.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-03-21T11:09:14.252 INFO:teuthology.orchestra.run.smithi100.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-03-21T11:09:14.278 INFO:teuthology.orchestra.run.smithi047.stdout:(25/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 687 kB/s | 85 kB 00:00 2024-03-21T11:09:14.311 INFO:teuthology.orchestra.run.smithi100.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-21T11:09:14.336 INFO:teuthology.orchestra.run.smithi100.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-21T11:09:14.379 INFO:teuthology.orchestra.run.smithi100.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 481 kB/s | 20 kB 00:00 2024-03-21T11:09:14.445 INFO:teuthology.orchestra.run.smithi100.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-03-21T11:09:14.487 INFO:teuthology.orchestra.run.smithi100.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-03-21T11:09:14.720 INFO:teuthology.orchestra.run.smithi047.stdout:(26/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 7.0 MB/s | 3.1 MB 00:00 2024-03-21T11:09:14.796 INFO:teuthology.orchestra.run.smithi047.stdout:(27/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 2.2 MB/s | 171 kB 00:00 2024-03-21T11:09:14.812 INFO:teuthology.orchestra.run.smithi100.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 4.3 MB/s | 1.6 MB 00:00 2024-03-21T11:09:14.871 INFO:teuthology.orchestra.run.smithi047.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2231.g 458 kB/s | 34 kB 00:00 2024-03-21T11:09:14.955 INFO:teuthology.orchestra.run.smithi047.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 1.7 MB/s | 140 kB 00:00 2024-03-21T11:09:15.113 INFO:teuthology.orchestra.run.smithi100.stdout:(150/174): python3-grpcio-1.46.7-10.el9.x86_64. 1.1 MB/s | 2.0 MB 00:01 2024-03-21T11:09:15.213 INFO:teuthology.orchestra.run.smithi047.stdout:(30/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 3.1 MB/s | 5.1 MB 00:01 2024-03-21T11:09:15.238 INFO:teuthology.orchestra.run.smithi100.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.5 MB/s | 321 kB 00:00 2024-03-21T11:09:15.272 INFO:teuthology.orchestra.run.smithi047.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 5.7 MB/s | 1.8 MB 00:00 2024-03-21T11:09:15.341 INFO:teuthology.orchestra.run.smithi102.stdout:(18/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 4.0 MB/s | 5.1 MB 00:01 2024-03-21T11:09:15.347 INFO:teuthology.orchestra.run.smithi047.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2231.g2e 3.3 MB/s | 248 kB 00:00 2024-03-21T11:09:15.416 INFO:teuthology.orchestra.run.smithi102.stdout:(19/174): python3-ceph-argparse-19.0.0-2231.g2e 603 kB/s | 45 kB 00:00 2024-03-21T11:09:15.430 INFO:teuthology.orchestra.run.smithi047.stdout:(33/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 598 kB/s | 49 kB 00:00 2024-03-21T11:09:15.446 INFO:teuthology.orchestra.run.smithi100.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-03-21T11:09:15.483 INFO:teuthology.orchestra.run.smithi102.stdout:(20/174): python3-ceph-common-19.0.0-2231.g2edf 1.9 MB/s | 131 kB 00:00 2024-03-21T11:09:15.506 INFO:teuthology.orchestra.run.smithi047.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2231.g2 222 kB/s | 16 kB 00:00 2024-03-21T11:09:15.530 INFO:teuthology.orchestra.run.smithi100.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-03-21T11:09:15.572 INFO:teuthology.orchestra.run.smithi100.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-21T11:09:15.584 INFO:teuthology.orchestra.run.smithi102.stdout:(21/174): python3-cephfs-19.0.0-2231.g2edfee68. 1.6 MB/s | 161 kB 00:00 2024-03-21T11:09:15.606 INFO:teuthology.orchestra.run.smithi047.stdout:(35/174): ceph-volume-19.0.0-2231.g2edfee68.el9 2.6 MB/s | 264 kB 00:00 2024-03-21T11:09:15.614 INFO:teuthology.orchestra.run.smithi100.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-03-21T11:09:15.656 INFO:teuthology.orchestra.run.smithi100.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-21T11:09:15.698 INFO:teuthology.orchestra.run.smithi100.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 879 kB/s | 36 kB 00:00 2024-03-21T11:09:15.740 INFO:teuthology.orchestra.run.smithi100.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2024-03-21T11:09:15.782 INFO:teuthology.orchestra.run.smithi100.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 256 kB 00:00 2024-03-21T11:09:15.790 INFO:teuthology.orchestra.run.smithi047.stdout:(36/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 4.0 MB/s | 750 kB 00:00 2024-03-21T11:09:15.815 INFO:teuthology.orchestra.run.smithi100.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 692 kB/s | 23 kB 00:00 2024-03-21T11:09:15.858 INFO:teuthology.orchestra.run.smithi100.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 108 kB 00:00 2024-03-21T11:09:15.865 INFO:teuthology.orchestra.run.smithi047.stdout:(37/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 245 kB/s | 18 kB 00:00 2024-03-21T11:09:15.908 INFO:teuthology.orchestra.run.smithi100.stdout:(162/174): librados2-19.0.0-2231.g2edfee68.el9. 2.4 MB/s | 3.4 MB 00:01 2024-03-21T11:09:15.933 INFO:teuthology.orchestra.run.smithi100.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-21T11:09:15.959 INFO:teuthology.orchestra.run.smithi102.stdout:(22/174): python3-rados-19.0.0-2231.g2edfee68.e 855 kB/s | 320 kB 00:00 2024-03-21T11:09:15.975 INFO:teuthology.orchestra.run.smithi100.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 64 kB 00:00 2024-03-21T11:09:16.017 INFO:teuthology.orchestra.run.smithi100.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 688 kB/s | 29 kB 00:00 2024-03-21T11:09:16.042 INFO:teuthology.orchestra.run.smithi100.stdout:(166/174): libmount-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 136 kB 00:00 2024-03-21T11:09:16.065 INFO:teuthology.orchestra.run.smithi047.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.3 MB/s | 662 kB 00:00 2024-03-21T11:09:16.068 INFO:teuthology.orchestra.run.smithi102.stdout:(23/174): python3-rbd-19.0.0-2231.g2edfee68.el9 2.7 MB/s | 297 kB 00:00 2024-03-21T11:09:16.092 INFO:teuthology.orchestra.run.smithi100.stdout:(167/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-03-21T11:09:16.134 INFO:teuthology.orchestra.run.smithi102.stdout:(24/174): python3-rgw-19.0.0-2231.g2edfee68.el9 1.5 MB/s | 99 kB 00:00 2024-03-21T11:09:16.185 INFO:teuthology.orchestra.run.smithi102.stdout:(25/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-21T11:09:16.191 INFO:teuthology.orchestra.run.smithi047.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 7.3 MB/s | 920 kB 00:00 2024-03-21T11:09:16.219 INFO:teuthology.orchestra.run.smithi100.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 6.0 MB/s | 1.2 MB 00:00 2024-03-21T11:09:16.232 INFO:teuthology.orchestra.run.smithi047.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2024-03-21T11:09:16.291 INFO:teuthology.orchestra.run.smithi047.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-03-21T11:09:16.318 INFO:teuthology.orchestra.run.smithi100.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 4.6 MB/s | 465 kB 00:00 2024-03-21T11:09:16.333 INFO:teuthology.orchestra.run.smithi047.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 776 kB/s | 32 kB 00:00 2024-03-21T11:09:16.371 INFO:teuthology.orchestra.run.smithi100.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 6.6 MB/s | 2.2 MB 00:00 2024-03-21T11:09:16.396 INFO:teuthology.orchestra.run.smithi100.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 403 kB/s | 31 kB 00:00 2024-03-21T11:09:16.438 INFO:teuthology.orchestra.run.smithi100.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-03-21T11:09:16.458 INFO:teuthology.orchestra.run.smithi047.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-03-21T11:09:16.492 INFO:teuthology.orchestra.run.smithi047.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 527 kB/s | 17 kB 00:00 2024-03-21T11:09:16.522 INFO:teuthology.orchestra.run.smithi100.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.3 MB/s | 2.3 MB 00:00 2024-03-21T11:09:16.534 INFO:teuthology.orchestra.run.smithi047.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 247 kB/s | 10 kB 00:00 2024-03-21T11:09:16.563 INFO:teuthology.orchestra.run.smithi100.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 445 kB/s | 74 kB 00:00 2024-03-21T11:09:16.570 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-21T11:09:16.570 INFO:teuthology.orchestra.run.smithi100.stdout:Total 16 MB/s | 224 MB 00:14 2024-03-21T11:09:16.576 INFO:teuthology.orchestra.run.smithi047.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 159 kB 00:00 2024-03-21T11:09:16.610 INFO:teuthology.orchestra.run.smithi047.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-21T11:09:16.610 INFO:teuthology.orchestra.run.smithi102.stdout:(26/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x8 3.6 MB/s | 17 MB 00:04 2024-03-21T11:09:16.668 INFO:teuthology.orchestra.run.smithi047.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-21T11:09:16.685 INFO:teuthology.orchestra.run.smithi102.stdout:(27/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 2.2 MB/s | 171 kB 00:00 2024-03-21T11:09:16.718 INFO:teuthology.orchestra.run.smithi047.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-03-21T11:09:16.752 INFO:teuthology.orchestra.run.smithi047.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-03-21T11:09:16.761 INFO:teuthology.orchestra.run.smithi102.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2231.g 454 kB/s | 34 kB 00:00 2024-03-21T11:09:16.836 INFO:teuthology.orchestra.run.smithi102.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 1.8 MB/s | 140 kB 00:00 2024-03-21T11:09:16.878 INFO:teuthology.orchestra.run.smithi047.stdout:(51/174): ceph-mgr-diskprediction-local-19.0.0- 4.4 MB/s | 7.4 MB 00:01 2024-03-21T11:09:16.895 INFO:teuthology.orchestra.run.smithi102.stdout:(30/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 4.4 MB/s | 3.1 MB 00:00 2024-03-21T11:09:16.911 INFO:teuthology.orchestra.run.smithi047.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 282 kB 00:00 2024-03-21T11:09:16.945 INFO:teuthology.orchestra.run.smithi047.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-21T11:09:16.970 INFO:teuthology.orchestra.run.smithi047.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.8 MB/s | 217 kB 00:00 2024-03-21T11:09:16.995 INFO:teuthology.orchestra.run.smithi047.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 720 kB/s | 34 kB 00:00 2024-03-21T11:09:17.053 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-21T11:09:17.271 INFO:teuthology.orchestra.run.smithi047.stdout:(56/174): ceph-common-19.0.0-2231.g2edfee68.el9 1.6 MB/s | 21 MB 00:12 2024-03-21T11:09:17.324 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-21T11:09:17.324 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-21T11:09:17.396 INFO:teuthology.orchestra.run.smithi047.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-03-21T11:09:17.454 INFO:teuthology.orchestra.run.smithi047.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 3.4 MB/s | 1.5 MB 00:00 2024-03-21T11:09:17.488 INFO:teuthology.orchestra.run.smithi047.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 3.9 MB/s | 811 kB 00:00 2024-03-21T11:09:17.504 INFO:teuthology.orchestra.run.smithi047.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2024-03-21T11:09:17.597 INFO:teuthology.orchestra.run.smithi047.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 9.5 MB/s | 1.0 MB 00:00 2024-03-21T11:09:17.603 INFO:teuthology.orchestra.run.smithi102.stdout:(31/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 11 MB/s | 48 MB 00:04 2024-03-21T11:09:17.645 INFO:teuthology.orchestra.run.smithi102.stdout:(32/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 2.2 MB/s | 1.8 MB 00:00 2024-03-21T11:09:17.670 INFO:teuthology.orchestra.run.smithi102.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2231.g2e 3.7 MB/s | 248 kB 00:00 2024-03-21T11:09:17.713 INFO:teuthology.orchestra.run.smithi102.stdout:(34/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 739 kB/s | 49 kB 00:00 2024-03-21T11:09:17.713 INFO:teuthology.orchestra.run.smithi047.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 25 MB/s | 5.7 MB 00:00 2024-03-21T11:09:17.737 INFO:teuthology.orchestra.run.smithi102.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2231.g2 246 kB/s | 16 kB 00:00 2024-03-21T11:09:17.739 INFO:teuthology.orchestra.run.smithi047.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 357 kB/s | 78 kB 00:00 2024-03-21T11:09:17.830 INFO:teuthology.orchestra.run.smithi102.stdout:(36/174): ceph-volume-19.0.0-2231.g2edfee68.el9 2.2 MB/s | 264 kB 00:00 2024-03-21T11:09:17.855 INFO:teuthology.orchestra.run.smithi102.stdout:(37/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 6.3 MB/s | 750 kB 00:00 2024-03-21T11:09:17.906 INFO:teuthology.orchestra.run.smithi047.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-03-21T11:09:17.931 INFO:teuthology.orchestra.run.smithi047.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 139 kB/s | 24 kB 00:00 2024-03-21T11:09:17.938 INFO:teuthology.orchestra.run.smithi102.stdout:(38/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 168 kB/s | 18 kB 00:00 2024-03-21T11:09:17.956 INFO:teuthology.orchestra.run.smithi047.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 90 kB/s | 18 kB 00:00 2024-03-21T11:09:17.998 INFO:teuthology.orchestra.run.smithi047.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.5 MB/s | 144 kB 00:00 2024-03-21T11:09:18.023 INFO:teuthology.orchestra.run.smithi047.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 485 kB/s | 32 kB 00:00 2024-03-21T11:09:18.155 INFO:teuthology.orchestra.run.smithi102.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 662 kB 00:00 2024-03-21T11:09:18.165 INFO:teuthology.orchestra.run.smithi047.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2024-03-21T11:09:18.199 INFO:teuthology.orchestra.run.smithi047.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-21T11:09:18.224 INFO:teuthology.orchestra.run.smithi047.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 949 kB/s | 174 kB 00:00 2024-03-21T11:09:18.249 INFO:teuthology.orchestra.run.smithi047.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 867 kB/s | 69 kB 00:00 2024-03-21T11:09:18.255 INFO:teuthology.orchestra.run.smithi102.stdout:(40/174): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-03-21T11:09:18.308 INFO:teuthology.orchestra.run.smithi047.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 503 kB/s | 41 kB 00:00 2024-03-21T11:09:18.333 INFO:teuthology.orchestra.run.smithi047.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 238 kB 00:00 2024-03-21T11:09:18.339 INFO:teuthology.orchestra.run.smithi102.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 2.3 MB/s | 920 kB 00:00 2024-03-21T11:09:18.358 INFO:teuthology.orchestra.run.smithi047.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 132 kB 00:00 2024-03-21T11:09:18.364 INFO:teuthology.orchestra.run.smithi102.stdout:(42/174): boost-program-options-1.75.0-8.el9.x8 507 kB/s | 106 kB 00:00 2024-03-21T11:09:18.390 INFO:teuthology.orchestra.run.smithi102.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 307 kB 00:00 2024-03-21T11:09:18.392 INFO:teuthology.orchestra.run.smithi047.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.6 MB/s | 211 kB 00:00 2024-03-21T11:09:18.425 INFO:teuthology.orchestra.run.smithi047.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 741 kB/s | 47 kB 00:00 2024-03-21T11:09:18.432 INFO:teuthology.orchestra.run.smithi102.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 477 kB/s | 32 kB 00:00 2024-03-21T11:09:18.451 INFO:teuthology.orchestra.run.smithi047.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 482 kB/s | 43 kB 00:00 2024-03-21T11:09:18.607 INFO:teuthology.orchestra.run.smithi102.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 79 kB/s | 17 kB 00:00 2024-03-21T11:09:18.709 INFO:teuthology.orchestra.run.smithi047.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 159 kB/s | 39 kB 00:00 2024-03-21T11:09:18.734 INFO:teuthology.orchestra.run.smithi047.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 153 kB/s | 41 kB 00:00 2024-03-21T11:09:18.757 INFO:teuthology.orchestra.run.smithi102.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 31 kB/s | 10 kB 00:00 2024-03-21T11:09:18.857 INFO:teuthology.orchestra.run.smithi102.stdout:(47/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.1 MB/s | 3.0 MB 00:00 2024-03-21T11:09:18.883 INFO:teuthology.orchestra.run.smithi102.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 578 kB/s | 159 kB 00:00 2024-03-21T11:09:18.908 INFO:teuthology.orchestra.run.smithi102.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 294 kB/s | 44 kB 00:00 2024-03-21T11:09:18.951 INFO:teuthology.orchestra.run.smithi047.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-03-21T11:09:19.000 INFO:teuthology.orchestra.run.smithi102.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 661 kB 00:00 2024-03-21T11:09:19.008 INFO:teuthology.orchestra.run.smithi047.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 300 kB 00:00 2024-03-21T11:09:19.025 INFO:teuthology.orchestra.run.smithi102.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 397 kB/s | 46 kB 00:00 2024-03-21T11:09:19.026 INFO:teuthology.orchestra.run.smithi047.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 204 kB/s | 56 kB 00:00 2024-03-21T11:09:19.029 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-21T11:09:19.029 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-21T11:09:19.052 INFO:teuthology.orchestra.run.smithi047.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 652 kB/s | 63 kB 00:00 2024-03-21T11:09:19.085 INFO:teuthology.orchestra.run.smithi047.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 774 kB/s | 44 kB 00:00 2024-03-21T11:09:19.089 INFO:teuthology.orchestra.run.smithi102.stdout:(52/174): libtool-2.4.6-45.el9.x86_64.rpm 3.1 MB/s | 578 kB 00:00 2024-03-21T11:09:19.111 INFO:teuthology.orchestra.run.smithi047.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 959 kB/s | 79 kB 00:00 2024-03-21T11:09:19.123 INFO:teuthology.orchestra.run.smithi102.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.2 MB/s | 282 kB 00:00 2024-03-21T11:09:19.144 INFO:teuthology.orchestra.run.smithi047.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2024-03-21T11:09:19.148 INFO:teuthology.orchestra.run.smithi102.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-03-21T11:09:19.174 INFO:teuthology.orchestra.run.smithi102.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.5 MB/s | 217 kB 00:00 2024-03-21T11:09:19.194 INFO:teuthology.orchestra.run.smithi047.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-21T11:09:19.199 INFO:teuthology.orchestra.run.smithi102.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 454 kB/s | 34 kB 00:00 2024-03-21T11:09:19.211 INFO:teuthology.orchestra.run.smithi047.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-21T11:09:19.228 INFO:teuthology.orchestra.run.smithi047.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.7 MB/s | 124 kB 00:00 2024-03-21T11:09:19.262 INFO:teuthology.orchestra.run.smithi047.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.8 MB/s | 187 kB 00:00 2024-03-21T11:09:19.404 INFO:teuthology.orchestra.run.smithi047.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 986 kB/s | 187 kB 00:00 2024-03-21T11:09:19.438 INFO:teuthology.orchestra.run.smithi047.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-03-21T11:09:19.471 INFO:teuthology.orchestra.run.smithi047.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-21T11:09:19.513 INFO:teuthology.orchestra.run.smithi047.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-03-21T11:09:19.516 INFO:teuthology.orchestra.run.smithi102.stdout:(57/174): postgresql-contrib-13.11-1.el9.x86_64 2.5 MB/s | 811 kB 00:00 2024-03-21T11:09:19.539 INFO:teuthology.orchestra.run.smithi047.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 215 kB/s | 21 kB 00:00 2024-03-21T11:09:19.550 INFO:teuthology.orchestra.run.smithi102.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 4.0 MB/s | 1.5 MB 00:00 2024-03-21T11:09:19.572 INFO:teuthology.orchestra.run.smithi047.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 255 kB/s | 25 kB 00:00 2024-03-21T11:09:19.617 INFO:teuthology.orchestra.run.smithi102.stdout:(59/174): openblas-openmp-0.3.21-2.el9.x86_64.r 10 MB/s | 4.7 MB 00:00 2024-03-21T11:09:19.642 INFO:teuthology.orchestra.run.smithi102.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.1 MB/s | 134 kB 00:00 2024-03-21T11:09:19.731 INFO:teuthology.orchestra.run.smithi047.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-03-21T11:09:19.747 INFO:teuthology.orchestra.run.smithi047.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-03-21T11:09:19.767 INFO:teuthology.orchestra.run.smithi102.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 26 MB/s | 5.7 MB 00:00 2024-03-21T11:09:19.784 INFO:teuthology.orchestra.run.smithi102.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 558 kB/s | 78 kB 00:00 2024-03-21T11:09:19.806 INFO:teuthology.orchestra.run.smithi047.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 611 kB/s | 35 kB 00:00 2024-03-21T11:09:19.826 INFO:teuthology.orchestra.run.smithi102.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.6 MB/s | 1.0 MB 00:00 2024-03-21T11:09:19.840 INFO:teuthology.orchestra.run.smithi047.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.3 MB/s | 216 kB 00:00 2024-03-21T11:09:19.882 INFO:teuthology.orchestra.run.smithi047.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 602 kB/s | 44 kB 00:00 2024-03-21T11:09:19.940 INFO:teuthology.orchestra.run.smithi047.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 388 kB/s | 22 kB 00:00 2024-03-21T11:09:19.943 INFO:teuthology.orchestra.run.smithi102.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 5.8 MB 00:00 2024-03-21T11:09:19.968 INFO:teuthology.orchestra.run.smithi102.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 135 kB/s | 24 kB 00:00 2024-03-21T11:09:19.985 INFO:teuthology.orchestra.run.smithi102.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 116 kB/s | 18 kB 00:00 2024-03-21T11:09:20.010 INFO:teuthology.orchestra.run.smithi102.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-03-21T11:09:20.035 INFO:teuthology.orchestra.run.smithi102.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 635 kB/s | 32 kB 00:00 2024-03-21T11:09:20.049 INFO:teuthology.orchestra.run.smithi047.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 450 kB/s | 94 kB 00:00 2024-03-21T11:09:20.219 INFO:teuthology.orchestra.run.smithi102.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-03-21T11:09:20.244 INFO:teuthology.orchestra.run.smithi102.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 931 kB/s | 217 kB 00:00 2024-03-21T11:09:20.266 INFO:teuthology.orchestra.run.smithi047.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-03-21T11:09:20.269 INFO:teuthology.orchestra.run.smithi102.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 752 kB/s | 174 kB 00:00 2024-03-21T11:09:20.328 INFO:teuthology.orchestra.run.smithi102.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-03-21T11:09:20.354 INFO:teuthology.orchestra.run.smithi102.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 492 kB/s | 41 kB 00:00 2024-03-21T11:09:20.379 INFO:teuthology.orchestra.run.smithi102.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 238 kB 00:00 2024-03-21T11:09:20.395 INFO:teuthology.orchestra.run.smithi102.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-21T11:09:20.454 INFO:teuthology.orchestra.run.smithi102.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 633 kB/s | 47 kB 00:00 2024-03-21T11:09:20.513 INFO:teuthology.orchestra.run.smithi102.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 211 kB 00:00 2024-03-21T11:09:20.538 INFO:teuthology.orchestra.run.smithi102.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 301 kB/s | 43 kB 00:00 2024-03-21T11:09:20.708 INFO:teuthology.orchestra.run.smithi047.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 833 kB/s | 548 kB 00:00 2024-03-21T11:09:20.775 INFO:teuthology.orchestra.run.smithi047.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 295 kB/s | 19 kB 00:00 2024-03-21T11:09:20.800 INFO:teuthology.orchestra.run.smithi047.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 579 kB/s | 309 kB 00:00 2024-03-21T11:09:20.883 INFO:teuthology.orchestra.run.smithi047.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 314 kB/s | 26 kB 00:00 2024-03-21T11:09:20.997 INFO:teuthology.orchestra.run.smithi102.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-03-21T11:09:21.022 INFO:teuthology.orchestra.run.smithi102.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 80 kB/s | 39 kB 00:00 2024-03-21T11:09:21.033 INFO:teuthology.orchestra.run.smithi047.stdout:(110/174): openssl-devel-3.0.7-27.el9.x86_64.rp 2.7 MB/s | 4.1 MB 00:01 2024-03-21T11:09:21.047 INFO:teuthology.orchestra.run.smithi102.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 81 kB/s | 41 kB 00:00 2024-03-21T11:09:21.059 INFO:teuthology.orchestra.run.smithi047.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 278 kB/s | 49 kB 00:00 2024-03-21T11:09:21.072 INFO:teuthology.orchestra.run.smithi102.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-03-21T11:09:21.106 INFO:teuthology.orchestra.run.smithi102.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 977 kB/s | 56 kB 00:00 2024-03-21T11:09:21.131 INFO:teuthology.orchestra.run.smithi102.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 754 kB/s | 63 kB 00:00 2024-03-21T11:09:21.156 INFO:teuthology.orchestra.run.smithi102.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 525 kB/s | 44 kB 00:00 2024-03-21T11:09:21.181 INFO:teuthology.orchestra.run.smithi102.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-21T11:09:21.184 INFO:teuthology.orchestra.run.smithi047.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-03-21T11:09:21.207 INFO:teuthology.orchestra.run.smithi102.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-21T11:09:21.218 INFO:teuthology.orchestra.run.smithi047.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 960 kB/s | 151 kB 00:00 2024-03-21T11:09:21.240 INFO:teuthology.orchestra.run.smithi102.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-21T11:09:21.265 INFO:teuthology.orchestra.run.smithi102.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-21T11:09:21.268 INFO:teuthology.orchestra.run.smithi047.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.9 MB/s | 838 kB 00:00 2024-03-21T11:09:21.291 INFO:teuthology.orchestra.run.smithi102.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-21T11:09:21.302 INFO:teuthology.orchestra.run.smithi047.stdout:(115/174): libunwind-1.6.2-1.el9.x86_64.rpm 252 kB/s | 67 kB 00:00 2024-03-21T11:09:21.316 INFO:teuthology.orchestra.run.smithi102.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-21T11:09:21.327 INFO:teuthology.orchestra.run.smithi047.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 753 kB/s | 43 kB 00:00 2024-03-21T11:09:21.377 INFO:teuthology.orchestra.run.smithi047.stdout:(117/174): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-03-21T11:09:21.402 INFO:teuthology.orchestra.run.smithi047.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 201 kB/s | 15 kB 00:00 2024-03-21T11:09:21.416 INFO:teuthology.orchestra.run.smithi102.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-21T11:09:21.428 INFO:teuthology.orchestra.run.smithi047.stdout:(119/174): python3-cachetools-4.2.4-1.el9.noarc 258 kB/s | 32 kB 00:00 2024-03-21T11:09:21.433 INFO:teuthology.orchestra.run.smithi102.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-03-21T11:09:21.461 INFO:teuthology.orchestra.run.smithi047.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-03-21T11:09:21.467 INFO:teuthology.orchestra.run.smithi102.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-03-21T11:09:21.484 INFO:teuthology.orchestra.run.smithi102.stdout:(95/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-21T11:09:21.495 INFO:teuthology.orchestra.run.smithi047.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-03-21T11:09:21.520 INFO:teuthology.orchestra.run.smithi047.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 611 kB/s | 35 kB 00:00 2024-03-21T11:09:21.542 INFO:teuthology.orchestra.run.smithi102.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 197 kB/s | 21 kB 00:00 2024-03-21T11:09:21.568 INFO:teuthology.orchestra.run.smithi102.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-03-21T11:09:21.595 INFO:teuthology.orchestra.run.smithi047.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-03-21T11:09:21.620 INFO:teuthology.orchestra.run.smithi047.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-03-21T11:09:21.679 INFO:teuthology.orchestra.run.smithi047.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-03-21T11:09:21.696 INFO:teuthology.orchestra.run.smithi047.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 143 kB/s | 11 kB 00:00 2024-03-21T11:09:21.751 INFO:teuthology.orchestra.run.smithi102.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-03-21T11:09:21.755 INFO:teuthology.orchestra.run.smithi047.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-03-21T11:09:21.777 INFO:teuthology.orchestra.run.smithi102.stdout:(99/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2024-03-21T11:09:21.780 INFO:teuthology.orchestra.run.smithi047.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-03-21T11:09:21.868 INFO:teuthology.orchestra.run.smithi102.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 381 kB/s | 35 kB 00:00 2024-03-21T11:09:21.872 INFO:teuthology.orchestra.run.smithi047.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2024-03-21T11:09:21.947 INFO:teuthology.orchestra.run.smithi047.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 118 kB/s | 20 kB 00:00 2024-03-21T11:09:21.952 INFO:teuthology.orchestra.run.smithi102.stdout:(101/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 534 kB/s | 44 kB 00:00 2024-03-21T11:09:21.981 INFO:teuthology.orchestra.run.smithi047.stdout:(131/174): python3-jaraco-collections-3.0.0-8.e 82 kB/s | 23 kB 00:00 2024-03-21T11:09:21.986 INFO:teuthology.orchestra.run.smithi102.stdout:(102/174): qatlib-23.11.0-1.el9.x86_64.rpm 925 kB/s | 216 kB 00:00 2024-03-21T11:09:22.014 INFO:teuthology.orchestra.run.smithi047.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 700 kB/s | 46 kB 00:00 2024-03-21T11:09:22.044 INFO:teuthology.orchestra.run.smithi102.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.0 MB/s | 94 kB 00:00 2024-03-21T11:09:22.048 INFO:teuthology.orchestra.run.smithi047.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-03-21T11:09:22.082 INFO:teuthology.orchestra.run.smithi047.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 788 kB/s | 79 kB 00:00 2024-03-21T11:09:22.115 INFO:teuthology.orchestra.run.smithi047.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 576 kB/s | 58 kB 00:00 2024-03-21T11:09:22.149 INFO:teuthology.orchestra.run.smithi047.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-03-21T11:09:22.174 INFO:teuthology.orchestra.run.smithi047.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-03-21T11:09:22.199 INFO:teuthology.orchestra.run.smithi047.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-03-21T11:09:22.225 INFO:teuthology.orchestra.run.smithi047.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 378 kB/s | 28 kB 00:00 2024-03-21T11:09:22.250 INFO:teuthology.orchestra.run.smithi047.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 412 kB/s | 31 kB 00:00 2024-03-21T11:09:22.283 INFO:teuthology.orchestra.run.smithi047.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-03-21T11:09:22.309 INFO:teuthology.orchestra.run.smithi047.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 618 kB/s | 36 kB 00:00 2024-03-21T11:09:22.334 INFO:teuthology.orchestra.run.smithi047.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-03-21T11:09:22.359 INFO:teuthology.orchestra.run.smithi047.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 683 kB/s | 51 kB 00:00 2024-03-21T11:09:22.393 INFO:teuthology.orchestra.run.smithi047.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-21T11:09:22.418 INFO:teuthology.orchestra.run.smithi047.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-03-21T11:09:22.477 INFO:teuthology.orchestra.run.smithi047.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-03-21T11:09:22.502 INFO:teuthology.orchestra.run.smithi047.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 185 kB/s | 20 kB 00:00 2024-03-21T11:09:22.537 INFO:teuthology.orchestra.run.smithi047.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-03-21T11:09:22.686 INFO:teuthology.orchestra.run.smithi102.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 32 kB/s | 22 kB 00:00 2024-03-21T11:09:22.761 INFO:teuthology.orchestra.run.smithi047.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-03-21T11:09:22.811 INFO:teuthology.orchestra.run.smithi047.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.4 MB/s | 321 kB 00:00 2024-03-21T11:09:23.028 INFO:teuthology.orchestra.run.smithi047.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-03-21T11:09:23.112 INFO:teuthology.orchestra.run.smithi047.stdout:(153/174): librados2-19.0.0-2231.g2edfee68.el9. 5.6 MB/s | 3.4 MB 00:00 2024-03-21T11:09:23.195 INFO:teuthology.orchestra.run.smithi047.stdout:(154/174): NetworkManager-libnm-1.47.2-1.el9.x8 11 MB/s | 1.8 MB 00:00 2024-03-21T11:09:23.221 INFO:teuthology.orchestra.run.smithi047.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 391 kB/s | 42 kB 00:00 2024-03-21T11:09:23.228 INFO:teuthology.orchestra.run.smithi102.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-03-21T11:09:23.254 INFO:teuthology.orchestra.run.smithi047.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-03-21T11:09:23.279 INFO:teuthology.orchestra.run.smithi047.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-21T11:09:23.306 INFO:teuthology.orchestra.run.smithi047.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 724 kB/s | 36 kB 00:00 2024-03-21T11:09:23.339 INFO:teuthology.orchestra.run.smithi047.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-03-21T11:09:23.345 INFO:teuthology.orchestra.run.smithi102.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-03-21T11:09:23.373 INFO:teuthology.orchestra.run.smithi047.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.8 MB/s | 256 kB 00:00 2024-03-21T11:09:23.390 INFO:teuthology.orchestra.run.smithi047.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 466 kB/s | 23 kB 00:00 2024-03-21T11:09:23.420 INFO:teuthology.orchestra.run.smithi102.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-03-21T11:09:23.424 INFO:teuthology.orchestra.run.smithi047.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-21T11:09:23.457 INFO:teuthology.orchestra.run.smithi047.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 155 kB 00:00 2024-03-21T11:09:23.491 INFO:teuthology.orchestra.run.smithi047.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-03-21T11:09:23.516 INFO:teuthology.orchestra.run.smithi047.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-21T11:09:23.541 INFO:teuthology.orchestra.run.smithi047.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 579 kB/s | 29 kB 00:00 2024-03-21T11:09:23.633 INFO:teuthology.orchestra.run.smithi047.stdout:(167/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 2.9 MB/s | 3.1 MB 00:01 2024-03-21T11:09:23.701 INFO:teuthology.orchestra.run.smithi047.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 6.6 MB/s | 1.2 MB 00:00 2024-03-21T11:09:23.792 INFO:teuthology.orchestra.run.smithi047.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 8.6 MB/s | 2.2 MB 00:00 2024-03-21T11:09:23.826 INFO:teuthology.orchestra.run.smithi047.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.7 MB/s | 465 kB 00:00 2024-03-21T11:09:23.851 INFO:teuthology.orchestra.run.smithi047.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 539 kB/s | 31 kB 00:00 2024-03-21T11:09:23.862 INFO:teuthology.orchestra.run.smithi102.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-03-21T11:09:23.918 INFO:teuthology.orchestra.run.smithi047.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-03-21T11:09:23.939 INFO:teuthology.orchestra.run.smithi102.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 340 kB/s | 26 kB 00:00 2024-03-21T11:09:23.952 INFO:teuthology.orchestra.run.smithi047.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 747 kB/s | 74 kB 00:00 2024-03-21T11:09:23.981 INFO:teuthology.orchestra.run.smithi102.stdout:(110/174): protobuf-compiler-3.14.0-13.el9.x86_ 446 kB/s | 863 kB 00:01 2024-03-21T11:09:24.010 INFO:teuthology.orchestra.run.smithi047.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 6.1 MB/s | 2.3 MB 00:00 2024-03-21T11:09:24.014 INFO:teuthology.orchestra.run.smithi102.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 655 kB/s | 49 kB 00:00 2024-03-21T11:09:24.018 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-03-21T11:09:24.019 INFO:teuthology.orchestra.run.smithi047.stdout:Total 11 MB/s | 224 MB 00:20 2024-03-21T11:09:24.098 INFO:teuthology.orchestra.run.smithi102.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-03-21T11:09:24.157 INFO:teuthology.orchestra.run.smithi102.stdout:(113/174): openssl-devel-3.0.7-27.el9.x86_64.rp 1.5 MB/s | 4.1 MB 00:02 2024-03-21T11:09:24.216 INFO:teuthology.orchestra.run.smithi102.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.0 MB/s | 838 kB 00:00 2024-03-21T11:09:24.316 INFO:teuthology.orchestra.run.smithi102.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 589 kB/s | 43 kB 00:00 2024-03-21T11:09:24.349 INFO:teuthology.orchestra.run.smithi102.stdout:(116/174): libunwind-1.6.2-1.el9.x86_64.rpm 183 kB/s | 67 kB 00:00 2024-03-21T11:09:24.383 INFO:teuthology.orchestra.run.smithi102.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 481 kB/s | 32 kB 00:00 2024-03-21T11:09:24.425 INFO:teuthology.orchestra.run.smithi102.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-03-21T11:09:24.500 INFO:teuthology.orchestra.run.smithi102.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-03-21T11:09:24.592 INFO:teuthology.orchestra.run.smithi102.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-03-21T11:09:24.676 INFO:teuthology.orchestra.run.smithi102.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-03-21T11:09:24.701 INFO:teuthology.orchestra.run.smithi102.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 325 kB/s | 35 kB 00:00 2024-03-21T11:09:24.726 INFO:teuthology.orchestra.run.smithi102.stdout:(123/174): python3-asyncssh-2.13.2-5.el9.noarch 965 kB/s | 548 kB 00:00 2024-03-21T11:09:24.802 INFO:teuthology.orchestra.run.smithi102.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-03-21T11:09:24.894 INFO:teuthology.orchestra.run.smithi102.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-03-21T11:09:24.919 INFO:teuthology.orchestra.run.smithi102.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 752 kB/s | 144 kB 00:00 2024-03-21T11:09:24.961 INFO:teuthology.orchestra.run.smithi102.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 7.9 MB/s | 2.0 MB 00:00 2024-03-21T11:09:24.986 INFO:teuthology.orchestra.run.smithi102.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-03-21T11:09:25.011 INFO:teuthology.orchestra.run.smithi102.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-03-21T11:09:25.045 INFO:teuthology.orchestra.run.smithi102.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-03-21T11:09:25.070 INFO:teuthology.orchestra.run.smithi102.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 234 kB/s | 20 kB 00:00 2024-03-21T11:09:25.145 INFO:teuthology.orchestra.run.smithi102.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-03-21T11:09:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 787 kB/s | 79 kB 00:00 2024-03-21T11:09:25.213 INFO:teuthology.orchestra.run.smithi102.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-03-21T11:09:25.255 INFO:teuthology.orchestra.run.smithi102.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-03-21T11:09:25.280 INFO:teuthology.orchestra.run.smithi102.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-03-21T11:09:25.305 INFO:teuthology.orchestra.run.smithi102.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 177 kB/s | 16 kB 00:00 2024-03-21T11:09:25.356 INFO:teuthology.orchestra.run.smithi102.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-03-21T11:09:25.381 INFO:teuthology.orchestra.run.smithi102.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-03-21T11:09:25.406 INFO:teuthology.orchestra.run.smithi102.stdout:(140/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 595 kB/s | 90 kB 00:00 2024-03-21T11:09:25.448 INFO:teuthology.orchestra.run.smithi102.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-03-21T11:09:25.482 INFO:teuthology.orchestra.run.smithi102.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-03-21T11:09:25.498 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-03-21T11:09:25.507 INFO:teuthology.orchestra.run.smithi102.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 470 kB/s | 59 kB 00:00 2024-03-21T11:09:25.532 INFO:teuthology.orchestra.run.smithi102.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 608 kB/s | 51 kB 00:00 2024-03-21T11:09:25.566 INFO:teuthology.orchestra.run.smithi102.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-21T11:09:25.633 INFO:teuthology.orchestra.run.smithi102.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 716 kB/s | 90 kB 00:00 2024-03-21T11:09:25.649 INFO:teuthology.orchestra.run.smithi102.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-03-21T11:09:25.675 INFO:teuthology.orchestra.run.smithi102.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-03-21T11:09:25.792 INFO:teuthology.orchestra.run.smithi102.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-21T11:09:25.817 INFO:teuthology.orchestra.run.smithi102.stdout:(150/174): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2024-03-21T11:09:25.867 INFO:teuthology.orchestra.run.smithi102.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-03-21T11:09:25.904 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-03-21T11:09:25.904 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-03-21T11:09:25.930 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-21T11:09:26.042 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T11:09:26.059 INFO:teuthology.orchestra.run.smithi102.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-03-21T11:09:26.076 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T11:09:26.115 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T11:09:26.125 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T11:09:26.135 INFO:teuthology.orchestra.run.smithi102.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-03-21T11:09:26.173 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T11:09:26.176 INFO:teuthology.orchestra.run.smithi102.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-21T11:09:26.218 INFO:teuthology.orchestra.run.smithi102.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-03-21T11:09:26.226 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T11:09:26.260 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T11:09:26.268 INFO:teuthology.orchestra.run.smithi102.stdout:(156/174): librados2-19.0.0-2231.g2edfee68.el9. 5.7 MB/s | 3.4 MB 00:00 2024-03-21T11:09:26.286 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T11:09:26.294 INFO:teuthology.orchestra.run.smithi102.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 950 kB/s | 71 kB 00:00 2024-03-21T11:09:26.328 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T11:09:26.336 INFO:teuthology.orchestra.run.smithi102.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 545 kB/s | 36 kB 00:00 2024-03-21T11:09:26.361 INFO:teuthology.orchestra.run.smithi102.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-03-21T11:09:26.403 INFO:teuthology.orchestra.run.smithi102.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 554 kB/s | 23 kB 00:00 2024-03-21T11:09:26.428 INFO:teuthology.orchestra.run.smithi102.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.7 MB/s | 256 kB 00:00 2024-03-21T11:09:26.470 INFO:teuthology.orchestra.run.smithi102.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 108 kB 00:00 2024-03-21T11:09:26.525 INFO:teuthology.orchestra.run.smithi102.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 155 kB 00:00 2024-03-21T11:09:26.554 INFO:teuthology.orchestra.run.smithi102.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-03-21T11:09:26.580 INFO:teuthology.orchestra.run.smithi102.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-21T11:09:26.605 INFO:teuthology.orchestra.run.smithi102.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 571 kB/s | 29 kB 00:00 2024-03-21T11:09:26.648 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T11:09:26.672 INFO:teuthology.orchestra.run.smithi102.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-03-21T11:09:26.712 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T11:09:26.722 INFO:teuthology.orchestra.run.smithi102.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 18 MB/s | 2.2 MB 00:00 2024-03-21T11:09:26.773 INFO:teuthology.orchestra.run.smithi102.stdout:(169/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 3.2 MB/s | 3.1 MB 00:00 2024-03-21T11:09:26.798 INFO:teuthology.orchestra.run.smithi102.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.0 MB/s | 465 kB 00:00 2024-03-21T11:09:26.834 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T11:09:26.848 INFO:teuthology.orchestra.run.smithi102.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 13 MB/s | 2.3 MB 00:00 2024-03-21T11:09:26.863 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T11:09:26.873 INFO:teuthology.orchestra.run.smithi102.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 310 kB/s | 31 kB 00:00 2024-03-21T11:09:26.907 INFO:teuthology.orchestra.run.smithi102.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.9 MB/s | 1.1 MB 00:00 2024-03-21T11:09:26.949 INFO:teuthology.orchestra.run.smithi102.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 741 kB/s | 74 kB 00:00 2024-03-21T11:09:26.955 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T11:09:26.955 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-03-21T11:09:26.955 INFO:teuthology.orchestra.run.smithi102.stdout:Total 12 MB/s | 224 MB 00:18 2024-03-21T11:09:26.984 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T11:09:27.025 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T11:09:27.198 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T11:09:27.249 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T11:09:27.277 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T11:09:27.372 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T11:09:27.403 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T11:09:27.433 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-03-21T11:09:27.450 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T11:09:27.490 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T11:09:27.513 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T11:09:27.529 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T11:09:27.703 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-03-21T11:09:27.703 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-03-21T11:09:27.721 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T11:09:27.753 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T11:09:27.792 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T11:09:27.899 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T11:09:28.038 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T11:09:28.089 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T11:09:28.158 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T11:09:28.232 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T11:09:28.413 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-03-21T11:09:28.414 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-03-21T11:09:28.456 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T11:09:28.572 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T11:09:28.674 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T11:09:28.712 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T11:09:28.738 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T11:09:28.774 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T11:09:28.849 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T11:09:28.903 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T11:09:28.949 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T11:09:29.013 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T11:09:29.056 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T11:09:29.100 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T11:09:29.132 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T11:09:29.160 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T11:09:29.242 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T11:09:29.274 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T11:09:29.317 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T11:09:29.346 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T11:09:29.411 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-03-21T11:09:29.411 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-03-21T11:09:29.415 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T11:09:29.468 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T11:09:29.551 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T11:09:29.687 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T11:09:29.747 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T11:09:30.273 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T11:09:30.367 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T11:09:30.466 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T11:09:30.520 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T11:09:30.528 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T11:09:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T11:09:30.609 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T11:09:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T11:09:30.677 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T11:09:30.904 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T11:09:31.109 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T11:09:31.173 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T11:09:31.217 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T11:09:31.354 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T11:09:31.412 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T11:09:31.467 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T11:09:31.623 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T11:09:31.805 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T11:09:31.842 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T11:09:31.898 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T11:09:31.933 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T11:09:31.976 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T11:09:32.018 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T11:09:32.160 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T11:09:32.215 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T11:09:32.245 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T11:09:32.286 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T11:09:32.381 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T11:09:32.420 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T11:09:32.460 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T11:09:32.486 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T11:09:32.510 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T11:09:32.583 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T11:09:32.703 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T11:09:32.814 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T11:09:32.861 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T11:09:32.909 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T11:09:32.964 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T11:09:33.149 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T11:09:33.390 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T11:09:33.527 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T11:09:33.573 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T11:09:33.603 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T11:09:33.634 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T11:09:33.698 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T11:09:33.739 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T11:09:33.777 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T11:09:33.809 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T11:09:33.848 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T11:09:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T11:09:33.958 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T11:09:33.991 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T11:09:34.033 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T11:09:34.067 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T11:09:34.157 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T11:09:34.238 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T11:09:34.298 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T11:09:34.339 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T11:09:34.386 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T11:09:34.482 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T11:09:34.531 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T11:09:35.259 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T11:09:35.312 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T11:09:35.346 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T11:09:35.402 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T11:09:35.489 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T11:09:35.530 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T11:09:35.707 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T11:09:35.736 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T11:09:35.814 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T11:09:35.980 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-03-21T11:09:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T11:09:36.125 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T11:09:36.162 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T11:09:36.207 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T11:09:36.285 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-03-21T11:09:36.306 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T11:09:36.386 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T11:09:36.418 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T11:09:36.420 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T11:09:36.451 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T11:09:36.452 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T11:09:36.460 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T11:09:36.473 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T11:09:36.500 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T11:09:36.511 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T11:09:36.553 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T11:09:36.588 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T11:09:36.592 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T11:09:36.608 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T11:09:36.613 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T11:09:36.657 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T11:09:37.034 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T11:09:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T11:09:37.066 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T11:09:37.084 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T11:09:37.179 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T11:09:37.201 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T11:09:37.243 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T11:09:37.277 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T11:09:37.292 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T11:09:37.321 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T11:09:37.361 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T11:09:37.411 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T11:09:37.468 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T11:09:37.537 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T11:09:37.553 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T11:09:37.594 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T11:09:37.622 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T11:09:37.734 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T11:09:37.757 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T11:09:37.803 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T11:09:37.804 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T11:09:37.836 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T11:09:37.858 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T11:09:37.868 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T11:09:37.874 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T11:09:37.887 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T11:09:37.918 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T11:09:37.926 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T11:09:37.949 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T11:09:37.950 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T11:09:37.977 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T11:09:38.066 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T11:09:38.098 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T11:09:38.110 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T11:09:38.129 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T11:09:38.135 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T11:09:38.225 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T11:09:38.235 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T11:09:38.289 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T11:09:38.332 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T11:09:38.332 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T11:09:38.392 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T11:09:38.405 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T11:09:38.445 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T11:09:38.461 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T11:09:38.481 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T11:09:38.510 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T11:09:38.538 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T11:09:38.603 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T11:09:38.661 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T11:09:38.719 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T11:09:38.824 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T11:09:38.931 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T11:09:38.941 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T11:09:38.948 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T11:09:38.964 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T11:09:38.977 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T11:09:38.982 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T11:09:38.991 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T11:09:39.018 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T11:09:39.026 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T11:09:39.030 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T11:09:39.054 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T11:09:39.100 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T11:09:39.154 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T11:09:39.193 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T11:09:39.255 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T11:09:39.256 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T11:09:39.291 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T11:09:39.335 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T11:09:39.367 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T11:09:39.394 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T11:09:39.396 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T11:09:39.465 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T11:09:39.482 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T11:09:39.501 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T11:09:39.544 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T11:09:39.573 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T11:09:39.642 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T11:09:39.674 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T11:09:39.687 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T11:09:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T11:09:39.760 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T11:09:39.894 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T11:09:39.939 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T11:09:40.107 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T11:09:40.315 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T11:09:40.455 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T11:09:40.458 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T11:09:40.495 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T11:09:40.536 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T11:09:40.541 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T11:09:40.607 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T11:09:40.621 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T11:09:40.664 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T11:09:40.671 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T11:09:40.703 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T11:09:40.727 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T11:09:40.744 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T11:09:40.782 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T11:09:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T11:09:40.811 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T11:09:40.846 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T11:09:40.978 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T11:09:41.027 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T11:09:41.051 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T11:09:41.092 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T11:09:41.141 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T11:09:41.180 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T11:09:41.213 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T11:09:41.264 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T11:09:41.297 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T11:09:41.303 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T11:09:41.307 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T11:09:41.342 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T11:09:41.350 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T11:09:41.402 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T11:09:41.441 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T11:09:41.564 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T11:09:41.621 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T11:09:41.650 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T11:09:41.732 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T11:09:41.844 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T11:09:41.883 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T11:09:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T11:09:41.920 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T11:09:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T11:09:41.985 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T11:09:41.985 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T11:09:41.985 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T11:09:41.985 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:09:41.993 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T11:09:42.013 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T11:09:42.065 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T11:09:42.095 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T11:09:42.132 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T11:09:42.178 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T11:09:42.214 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T11:09:42.257 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T11:09:42.299 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T11:09:42.435 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T11:09:42.490 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T11:09:42.520 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T11:09:42.561 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T11:09:42.655 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T11:09:42.694 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T11:09:42.695 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T11:09:42.735 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T11:09:42.761 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T11:09:42.786 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T11:09:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T11:09:42.857 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T11:09:42.929 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T11:09:42.977 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T11:09:42.977 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T11:09:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T11:09:43.052 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T11:09:43.086 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T11:09:43.127 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T11:09:43.130 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T11:09:43.176 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T11:09:43.190 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T11:09:43.230 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T11:09:43.243 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T11:09:43.414 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T11:09:43.457 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T11:09:43.585 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T11:09:43.621 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T11:09:43.653 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T11:09:43.684 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T11:09:43.704 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T11:09:43.712 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T11:09:43.746 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T11:09:43.785 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T11:09:43.817 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T11:09:43.847 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T11:09:43.917 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T11:09:43.924 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T11:09:43.958 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T11:09:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T11:09:43.992 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T11:09:44.032 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T11:09:44.060 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T11:09:44.067 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T11:09:44.156 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T11:09:44.247 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T11:09:44.274 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T11:09:44.289 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T11:09:44.321 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T11:09:44.352 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T11:09:44.368 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T11:09:44.435 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T11:09:44.466 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T11:09:44.522 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T11:09:44.530 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T11:09:44.578 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T11:09:44.618 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T11:09:44.689 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T11:09:44.734 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T11:09:44.798 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T11:09:44.865 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T11:09:45.069 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T11:09:45.132 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T11:09:45.170 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T11:09:45.229 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T11:09:45.249 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T11:09:45.295 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T11:09:45.329 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T11:09:45.373 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T11:09:45.377 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T11:09:45.418 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T11:09:45.457 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T11:09:45.467 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T11:09:45.496 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T11:09:45.518 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T11:09:45.551 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T11:09:45.673 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T11:09:45.703 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T11:09:45.744 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T11:09:45.788 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T11:09:45.925 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T11:09:46.061 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T11:09:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T11:09:46.181 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T11:09:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T11:09:46.417 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T11:09:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T11:09:46.516 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T11:09:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T11:09:46.730 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T11:09:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T11:09:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T11:09:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T11:09:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T11:09:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T11:09:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T11:09:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T11:09:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T11:09:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T11:09:47.316 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T11:09:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T11:09:47.418 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T11:09:47.464 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T11:09:47.614 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T11:09:47.726 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T11:09:47.734 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T11:09:47.775 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T11:09:47.812 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T11:09:47.828 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T11:09:47.843 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T11:09:47.874 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T11:09:47.887 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T11:09:47.901 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T11:09:48.030 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T11:09:48.095 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T11:09:48.214 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T11:09:48.240 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T11:09:48.329 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T11:09:48.370 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T11:09:48.406 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T11:09:48.868 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T11:09:48.902 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T11:09:48.957 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T11:09:48.987 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T11:09:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T11:09:49.130 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T11:09:49.161 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T11:09:49.219 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T11:09:49.335 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T11:09:49.378 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T11:09:49.624 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T11:09:49.649 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T11:09:49.752 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T11:09:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T11:09:50.742 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T11:09:51.255 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T11:09:51.291 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T11:09:51.832 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T11:09:51.870 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T11:09:51.951 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T11:09:51.951 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T11:09:51.951 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T11:09:51.951 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:09:51.979 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T11:09:52.350 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T11:09:52.454 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T11:09:52.492 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T11:09:52.523 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T11:09:52.556 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T11:09:52.953 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T11:09:52.986 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T11:09:53.107 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T11:09:53.142 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T11:09:53.198 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T11:09:53.627 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T11:09:53.627 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-03-21T11:09:53.627 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-03-21T11:09:53.627 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-03-21T11:09:53.627 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-03-21T11:09:53.628 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-03-21T11:09:53.628 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-03-21T11:09:53.628 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-03-21T11:09:53.628 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:09:53.742 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T11:09:53.772 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T11:09:53.792 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T11:09:53.807 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T11:09:53.808 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:09:53.808 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T11:09:53.808 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:09:53.872 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T11:09:53.899 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T11:09:55.297 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T11:09:55.323 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T11:09:55.323 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:09:55.323 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T11:09:55.323 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T11:09:55.323 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T11:09:55.323 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:09:55.548 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T11:09:55.570 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T11:09:55.570 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:09:55.570 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T11:09:55.570 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T11:09:55.570 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T11:09:55.570 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:09:56.015 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T11:09:56.038 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T11:09:56.038 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:09:56.038 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T11:09:56.038 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T11:09:56.038 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T11:09:56.038 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:09:56.134 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T11:09:56.601 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T11:09:56.634 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T11:09:56.640 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T11:09:56.671 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T11:09:56.672 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T11:09:56.733 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T11:09:56.763 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T11:09:56.997 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T11:09:57.033 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T11:09:57.326 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T11:09:57.366 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T11:09:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T11:09:57.541 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T11:09:57.541 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T11:09:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:09:57.583 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T11:09:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T11:09:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T11:09:58.125 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T11:09:58.288 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T11:09:58.319 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T11:09:58.342 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T11:09:58.342 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:09:58.342 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T11:09:58.342 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T11:09:58.342 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T11:09:58.342 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:09:58.386 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T11:09:58.397 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T11:09:58.442 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T11:09:59.319 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T11:09:59.363 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T11:09:59.386 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T11:09:59.386 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:09:59.386 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T11:09:59.386 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T11:09:59.386 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T11:09:59.386 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:09:59.439 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T11:09:59.461 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T11:09:59.461 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:09:59.461 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T11:09:59.461 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:09:59.506 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T11:09:59.529 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T11:09:59.529 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:09:59.529 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T11:09:59.529 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T11:09:59.529 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T11:09:59.529 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:09:59.840 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T11:09:59.863 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T11:09:59.863 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:09:59.863 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T11:09:59.863 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T11:09:59.863 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T11:09:59.863 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:10:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T11:10:03.784 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-03-21T11:10:03.784 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-03-21T11:10:03.784 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-03-21T11:10:03.784 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-03-21T11:10:03.784 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-03-21T11:10:03.784 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /root 2024-03-21T11:10:03.784 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-03-21T11:10:03.784 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:10:03.960 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T11:10:03.975 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T11:10:03.975 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:03.975 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T11:10:03.976 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:10:04.321 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T11:10:04.362 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T11:10:04.392 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T11:10:04.419 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T11:10:04.599 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T11:10:04.719 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T11:10:04.771 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T11:10:05.071 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T11:10:05.259 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T11:10:05.302 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T11:10:05.335 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T11:10:05.376 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T11:10:05.405 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T11:10:05.439 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T11:10:05.463 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T11:10:05.464 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:05.464 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T11:10:05.464 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T11:10:05.464 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T11:10:05.464 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:10:05.469 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T11:10:05.520 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T11:10:05.562 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T11:10:05.680 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T11:10:05.703 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T11:10:05.703 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:05.703 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T11:10:05.703 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T11:10:05.703 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T11:10:05.703 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:10:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T11:10:06.156 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T11:10:06.177 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T11:10:06.179 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T11:10:06.179 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:06.179 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T11:10:06.179 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T11:10:06.179 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T11:10:06.179 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:10:06.197 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T11:10:06.266 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T11:10:06.701 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T11:10:06.732 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T11:10:06.762 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T11:10:06.817 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T11:10:06.846 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T11:10:07.011 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T11:10:07.082 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T11:10:07.097 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T11:10:07.114 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T11:10:07.138 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T11:10:07.220 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T11:10:07.251 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T11:10:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T11:10:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T11:10:07.327 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T11:10:07.327 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T11:10:07.415 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T11:10:07.453 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T11:10:07.454 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T11:10:07.546 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T11:10:07.573 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T11:10:07.623 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T11:10:07.648 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T11:10:07.673 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T11:10:07.706 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T11:10:07.746 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T11:10:07.786 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T11:10:07.812 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T11:10:07.837 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T11:10:07.862 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T11:10:07.887 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T11:10:07.894 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T11:10:07.894 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T11:10:08.074 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T11:10:08.074 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T11:10:08.077 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T11:10:08.106 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T11:10:08.165 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T11:10:08.189 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T11:10:08.206 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T11:10:08.221 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T11:10:08.249 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T11:10:08.274 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T11:10:08.338 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T11:10:08.369 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T11:10:08.392 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T11:10:08.414 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T11:10:08.415 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:08.415 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T11:10:08.415 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T11:10:08.415 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T11:10:08.415 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:10:08.459 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T11:10:08.469 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T11:10:08.515 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T11:10:09.389 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T11:10:09.420 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T11:10:09.442 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T11:10:09.442 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:09.442 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T11:10:09.442 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T11:10:09.442 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T11:10:09.442 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:10:09.487 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T11:10:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T11:10:09.509 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:09.509 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T11:10:09.509 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:10:09.554 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T11:10:09.576 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T11:10:09.576 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:09.577 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T11:10:09.577 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T11:10:09.577 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T11:10:09.577 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:10:09.888 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T11:10:09.910 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T11:10:09.911 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:09.911 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T11:10:09.911 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T11:10:09.911 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T11:10:09.911 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:10:10.985 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T11:10:10.985 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-03-21T11:10:10.985 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-03-21T11:10:10.985 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-03-21T11:10:10.985 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-03-21T11:10:10.986 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-03-21T11:10:10.986 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /root 2024-03-21T11:10:10.986 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-03-21T11:10:10.986 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:10:11.216 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T11:10:11.265 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T11:10:11.265 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:11.265 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T11:10:11.265 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:10:12.935 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T11:10:13.020 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T11:10:13.021 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:13.021 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T11:10:13.021 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T11:10:13.021 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T11:10:13.021 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:10:13.268 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T11:10:13.288 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T11:10:13.288 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T11:10:13.288 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T11:10:13.288 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T11:10:13.288 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T11:10:13.288 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T11:10:13.288 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T11:10:13.288 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T11:10:13.289 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T11:10:13.289 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T11:10:13.289 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T11:10:13.289 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T11:10:13.289 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T11:10:13.289 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T11:10:13.289 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T11:10:13.289 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T11:10:13.289 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T11:10:13.289 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T11:10:13.289 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T11:10:13.289 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T11:10:13.289 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T11:10:13.290 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T11:10:13.290 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T11:10:13.290 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T11:10:13.290 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T11:10:13.290 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T11:10:13.290 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T11:10:13.290 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T11:10:13.290 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T11:10:13.290 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T11:10:13.290 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T11:10:13.290 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T11:10:13.290 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T11:10:13.290 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T11:10:13.291 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T11:10:13.291 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T11:10:13.291 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T11:10:13.291 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T11:10:13.291 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T11:10:13.291 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T11:10:13.291 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T11:10:13.291 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T11:10:13.291 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T11:10:13.291 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T11:10:13.291 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T11:10:13.291 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T11:10:13.291 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T11:10:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T11:10:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T11:10:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T11:10:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T11:10:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T11:10:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T11:10:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T11:10:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T11:10:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T11:10:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T11:10:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T11:10:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T11:10:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T11:10:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T11:10:13.293 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T11:10:13.293 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T11:10:13.293 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T11:10:13.293 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T11:10:13.293 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T11:10:13.293 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T11:10:13.293 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T11:10:13.293 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T11:10:13.293 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T11:10:13.293 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T11:10:13.293 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T11:10:13.293 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T11:10:13.293 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T11:10:13.293 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T11:10:13.294 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T11:10:13.294 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T11:10:13.294 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T11:10:13.294 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T11:10:13.294 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T11:10:13.294 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T11:10:13.294 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T11:10:13.294 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T11:10:13.294 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T11:10:13.294 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T11:10:13.294 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T11:10:13.294 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T11:10:13.294 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T11:10:13.295 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T11:10:13.296 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T11:10:13.296 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T11:10:13.296 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T11:10:13.296 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T11:10:13.296 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T11:10:13.296 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T11:10:13.296 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T11:10:13.296 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T11:10:13.296 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T11:10:13.296 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T11:10:13.296 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T11:10:13.296 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T11:10:13.296 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T11:10:13.296 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T11:10:13.297 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T11:10:13.297 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T11:10:13.297 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T11:10:13.297 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T11:10:13.297 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T11:10:13.297 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T11:10:13.297 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T11:10:13.297 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T11:10:13.297 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T11:10:13.297 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T11:10:13.297 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T11:10:13.297 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T11:10:13.297 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T11:10:13.297 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T11:10:13.298 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T11:10:13.298 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T11:10:13.298 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T11:10:13.298 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T11:10:13.298 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T11:10:13.298 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T11:10:13.298 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T11:10:13.298 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T11:10:13.298 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T11:10:13.298 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T11:10:13.298 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T11:10:13.298 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T11:10:13.298 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T11:10:13.299 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T11:10:13.299 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T11:10:13.299 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T11:10:13.299 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T11:10:13.299 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T11:10:13.299 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T11:10:13.299 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T11:10:13.299 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T11:10:13.299 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T11:10:13.299 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T11:10:13.299 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T11:10:13.299 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T11:10:13.299 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T11:10:13.299 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T11:10:13.300 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T11:10:13.301 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T11:10:13.301 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T11:10:13.301 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T11:10:13.301 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T11:10:13.301 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T11:10:13.301 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T11:10:13.301 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T11:10:13.301 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T11:10:13.301 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T11:10:13.301 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T11:10:13.301 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T11:10:13.301 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T11:10:13.301 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T11:10:13.301 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T11:10:13.302 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T11:10:13.302 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T11:10:13.302 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T11:10:13.302 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T11:10:13.302 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T11:10:13.302 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T11:10:13.302 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T11:10:13.302 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T11:10:13.302 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T11:10:13.302 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T11:10:13.302 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T11:10:13.338 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T11:10:13.338 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:13.338 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T11:10:13.338 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T11:10:13.338 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T11:10:13.339 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:10:13.844 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T11:10:13.915 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T11:10:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T11:10:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T11:10:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T11:10:13.915 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:10:14.029 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T11:10:14.351 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T11:10:14.393 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T11:10:14.423 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T11:10:14.450 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T11:10:14.631 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T11:10:14.672 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T11:10:14.707 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T11:10:14.749 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T11:10:14.758 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T11:10:14.798 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T11:10:14.839 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T11:10:14.862 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T11:10:14.892 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T11:10:14.894 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T11:10:14.935 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T11:10:14.968 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T11:10:15.000 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T11:10:15.030 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T11:10:15.084 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T11:10:15.146 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T11:10:15.178 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T11:10:15.234 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T11:10:15.278 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T11:10:15.685 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T11:10:15.726 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T11:10:15.745 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T11:10:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T11:10:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:10:16.107 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-03-21T11:10:16.107 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T11:10:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T11:10:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T11:10:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T11:10:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T11:10:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T11:10:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T11:10:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T11:10:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T11:10:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T11:10:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T11:10:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T11:10:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T11:10:16.108 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T11:10:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T11:10:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T11:10:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T11:10:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T11:10:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T11:10:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T11:10:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T11:10:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T11:10:16.109 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-21T11:10:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T11:10:16.109 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T11:10:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T11:10:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T11:10:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:16.110 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:16.111 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T11:10:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T11:10:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T11:10:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T11:10:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T11:10:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T11:10:16.114 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T11:10:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T11:10:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: jq-1.6-16.el9.x86_64 2024-03-21T11:10:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T11:10:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T11:10:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T11:10:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T11:10:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T11:10:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T11:10:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T11:10:16.115 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T11:10:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T11:10:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T11:10:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T11:10:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T11:10:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T11:10:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T11:10:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T11:10:16.116 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T11:10:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T11:10:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T11:10:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T11:10:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T11:10:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T11:10:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T11:10:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T11:10:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T11:10:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T11:10:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T11:10:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T11:10:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T11:10:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T11:10:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T11:10:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T11:10:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T11:10:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T11:10:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T11:10:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T11:10:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T11:10:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T11:10:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T11:10:16.118 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T11:10:16.119 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T11:10:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T11:10:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T11:10:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T11:10:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T11:10:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T11:10:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T11:10:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T11:10:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T11:10:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T11:10:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T11:10:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T11:10:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T11:10:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T11:10:16.120 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T11:10:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T11:10:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T11:10:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T11:10:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T11:10:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T11:10:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T11:10:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T11:10:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T11:10:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T11:10:16.121 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T11:10:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T11:10:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T11:10:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T11:10:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T11:10:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T11:10:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T11:10:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T11:10:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T11:10:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T11:10:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T11:10:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T11:10:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T11:10:16.122 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T11:10:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T11:10:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T11:10:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T11:10:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T11:10:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T11:10:16.123 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T11:10:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T11:10:16.124 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T11:10:16.124 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-21T11:10:16.433 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T11:10:16.462 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T11:10:16.580 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T11:10:16.605 DEBUG:teuthology.parallel:result is None 2024-03-21T11:10:16.613 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T11:10:16.672 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T11:10:16.712 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T11:10:16.793 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T11:10:16.793 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T11:10:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T11:10:16.833 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T11:10:16.858 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T11:10:16.884 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T11:10:16.896 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T11:10:16.896 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:16.896 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T11:10:16.896 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T11:10:16.896 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T11:10:16.896 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:10:16.910 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T11:10:16.910 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T11:10:16.957 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T11:10:16.981 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T11:10:16.995 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T11:10:17.011 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T11:10:17.011 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T11:10:17.082 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T11:10:17.096 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T11:10:17.122 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T11:10:17.155 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T11:10:17.206 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T11:10:17.230 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T11:10:17.263 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T11:10:17.303 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T11:10:17.344 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T11:10:17.369 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T11:10:17.394 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T11:10:17.419 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T11:10:17.436 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T11:10:17.443 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T11:10:17.443 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T11:10:17.622 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T11:10:17.623 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T11:10:17.706 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T11:10:17.731 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T11:10:17.763 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T11:10:17.790 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T11:10:17.815 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T11:10:17.880 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T11:10:18.103 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T11:10:18.133 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T11:10:18.208 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T11:10:18.208 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:18.208 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T11:10:18.208 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T11:10:18.208 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T11:10:18.208 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:10:18.251 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T11:10:18.320 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T11:10:18.320 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:18.320 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T11:10:18.320 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:10:18.393 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T11:10:18.472 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T11:10:18.472 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:18.472 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T11:10:18.472 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T11:10:18.472 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T11:10:18.472 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:10:18.861 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T11:10:18.954 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T11:10:18.954 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T11:10:18.954 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T11:10:18.954 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T11:10:18.954 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T11:10:18.954 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:10:22.756 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T11:10:22.756 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T11:10:22.756 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T11:10:22.756 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T11:10:22.756 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T11:10:22.756 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T11:10:22.756 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T11:10:22.756 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T11:10:22.756 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T11:10:22.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T11:10:22.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T11:10:22.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T11:10:22.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T11:10:22.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T11:10:22.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T11:10:22.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T11:10:22.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T11:10:22.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T11:10:22.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T11:10:22.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T11:10:22.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T11:10:22.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T11:10:22.757 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T11:10:22.758 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T11:10:22.758 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T11:10:22.758 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T11:10:22.758 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T11:10:22.758 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T11:10:22.758 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T11:10:22.758 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T11:10:22.758 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T11:10:22.758 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T11:10:22.759 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T11:10:22.759 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T11:10:22.759 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T11:10:22.759 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T11:10:22.759 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T11:10:22.759 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T11:10:22.759 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T11:10:22.759 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T11:10:22.759 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T11:10:22.760 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T11:10:22.760 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T11:10:22.760 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T11:10:22.760 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T11:10:22.760 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T11:10:22.760 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T11:10:22.760 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T11:10:22.760 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T11:10:22.760 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T11:10:22.760 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T11:10:22.760 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T11:10:22.760 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T11:10:22.760 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T11:10:22.761 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T11:10:22.761 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T11:10:22.761 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T11:10:22.761 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T11:10:22.761 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T11:10:22.761 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T11:10:22.761 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T11:10:22.761 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T11:10:22.761 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T11:10:22.761 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T11:10:22.761 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T11:10:22.762 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T11:10:22.762 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T11:10:22.762 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T11:10:22.762 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T11:10:22.762 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T11:10:22.762 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T11:10:22.762 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T11:10:22.762 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T11:10:22.762 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T11:10:22.762 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T11:10:22.762 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T11:10:22.762 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T11:10:22.763 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T11:10:22.763 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T11:10:22.763 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T11:10:22.763 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T11:10:22.763 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T11:10:22.763 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T11:10:22.763 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T11:10:22.763 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T11:10:22.763 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T11:10:22.763 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T11:10:22.763 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T11:10:22.764 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T11:10:22.764 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T11:10:22.764 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T11:10:22.764 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T11:10:22.764 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T11:10:22.764 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T11:10:22.764 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T11:10:22.764 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T11:10:22.764 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T11:10:22.764 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T11:10:22.764 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T11:10:22.764 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T11:10:22.764 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T11:10:22.764 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T11:10:22.765 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T11:10:22.765 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T11:10:22.765 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T11:10:22.765 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T11:10:22.765 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T11:10:22.765 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T11:10:22.765 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T11:10:22.765 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T11:10:22.765 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T11:10:22.765 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T11:10:22.765 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T11:10:22.765 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T11:10:22.765 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T11:10:22.766 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T11:10:22.766 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T11:10:22.766 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T11:10:22.766 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T11:10:22.766 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T11:10:22.766 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T11:10:22.766 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T11:10:22.766 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T11:10:22.766 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T11:10:22.766 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T11:10:22.766 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T11:10:22.766 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T11:10:22.766 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T11:10:22.767 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T11:10:22.767 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T11:10:22.767 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T11:10:22.767 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T11:10:22.767 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T11:10:22.767 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T11:10:22.767 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T11:10:22.767 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T11:10:22.767 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T11:10:22.767 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T11:10:22.767 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T11:10:22.767 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T11:10:22.767 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T11:10:22.767 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T11:10:22.768 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T11:10:22.768 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T11:10:22.768 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T11:10:22.768 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T11:10:22.768 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T11:10:22.768 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T11:10:22.768 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T11:10:22.768 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T11:10:22.768 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T11:10:22.768 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T11:10:22.768 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T11:10:22.768 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T11:10:22.768 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T11:10:22.768 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T11:10:22.769 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T11:10:22.769 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T11:10:22.769 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T11:10:22.769 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T11:10:22.769 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T11:10:22.769 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T11:10:22.769 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T11:10:22.769 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T11:10:22.769 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T11:10:22.769 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T11:10:22.769 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T11:10:22.770 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T11:10:22.770 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T11:10:22.770 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T11:10:22.770 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T11:10:22.770 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T11:10:22.770 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T11:10:22.770 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T11:10:22.770 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T11:10:22.770 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T11:10:22.770 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T11:10:22.770 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T11:10:22.770 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T11:10:22.770 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T11:10:22.770 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T11:10:22.771 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T11:10:22.771 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T11:10:22.771 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T11:10:22.771 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T11:10:22.771 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T11:10:22.771 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T11:10:22.771 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T11:10:22.771 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T11:10:22.771 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T11:10:22.771 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T11:10:22.771 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T11:10:22.771 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T11:10:22.771 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T11:10:22.771 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T11:10:22.772 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T11:10:22.772 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T11:10:22.772 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T11:10:24.035 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T11:10:24.079 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T11:10:24.116 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T11:10:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T11:10:24.437 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T11:10:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T11:10:24.626 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T11:10:24.686 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T11:10:24.897 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T11:10:24.946 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T11:10:24.991 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T11:10:25.034 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T11:10:25.086 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T11:10:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T11:10:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T11:10:25.333 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T11:10:25.705 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T11:10:25.706 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:10:25.706 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-03-21T11:10:25.706 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T11:10:25.706 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T11:10:25.706 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T11:10:25.706 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T11:10:25.706 INFO:teuthology.orchestra.run.smithi102.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T11:10:25.706 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T11:10:25.706 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T11:10:25.706 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T11:10:25.706 INFO:teuthology.orchestra.run.smithi102.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T11:10:25.707 INFO:teuthology.orchestra.run.smithi102.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T11:10:25.707 INFO:teuthology.orchestra.run.smithi102.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T11:10:25.707 INFO:teuthology.orchestra.run.smithi102.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T11:10:25.707 INFO:teuthology.orchestra.run.smithi102.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T11:10:25.707 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.707 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.707 INFO:teuthology.orchestra.run.smithi102.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T11:10:25.707 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T11:10:25.707 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T11:10:25.707 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T11:10:25.707 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T11:10:25.707 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T11:10:25.707 INFO:teuthology.orchestra.run.smithi102.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T11:10:25.708 INFO:teuthology.orchestra.run.smithi102.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T11:10:25.708 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T11:10:25.708 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-03-21T11:10:25.708 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T11:10:25.708 INFO:teuthology.orchestra.run.smithi102.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T11:10:25.708 INFO:teuthology.orchestra.run.smithi102.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T11:10:25.708 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T11:10:25.708 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.708 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.708 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.708 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.708 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:25.708 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.709 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.709 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.709 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:25.709 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:25.709 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:25.709 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:25.709 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:25.709 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.709 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.709 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:25.709 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.709 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.709 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.709 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:25.710 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:25.710 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.710 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.710 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:25.710 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T11:10:25.710 INFO:teuthology.orchestra.run.smithi102.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T11:10:25.710 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T11:10:25.710 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T11:10:25.710 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T11:10:25.710 INFO:teuthology.orchestra.run.smithi102.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T11:10:25.710 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T11:10:25.711 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T11:10:25.711 INFO:teuthology.orchestra.run.smithi102.stdout: jq-1.6-16.el9.x86_64 2024-03-21T11:10:25.711 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T11:10:25.711 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T11:10:25.711 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T11:10:25.711 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.711 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.711 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.711 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T11:10:25.711 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T11:10:25.711 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T11:10:25.711 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T11:10:25.711 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T11:10:25.712 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.712 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.712 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T11:10:25.712 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.712 INFO:teuthology.orchestra.run.smithi102.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T11:10:25.712 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T11:10:25.712 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T11:10:25.712 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T11:10:25.712 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T11:10:25.712 INFO:teuthology.orchestra.run.smithi102.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T11:10:25.712 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T11:10:25.712 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T11:10:25.712 INFO:teuthology.orchestra.run.smithi102.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T11:10:25.713 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T11:10:25.713 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T11:10:25.713 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T11:10:25.713 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T11:10:25.713 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T11:10:25.713 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T11:10:25.713 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T11:10:25.713 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T11:10:25.713 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T11:10:25.713 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T11:10:25.713 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T11:10:25.713 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T11:10:25.714 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T11:10:25.714 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T11:10:25.714 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T11:10:25.714 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.714 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.714 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.714 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T11:10:25.714 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T11:10:25.714 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T11:10:25.714 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T11:10:25.714 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T11:10:25.714 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T11:10:25.715 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T11:10:25.715 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T11:10:25.715 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T11:10:25.715 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T11:10:25.715 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T11:10:25.715 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T11:10:25.715 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T11:10:25.715 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T11:10:25.715 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T11:10:25.715 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T11:10:25.716 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T11:10:25.716 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T11:10:25.716 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T11:10:25.716 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T11:10:25.716 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T11:10:25.716 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T11:10:25.716 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T11:10:25.716 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T11:10:25.716 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T11:10:25.716 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T11:10:25.717 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T11:10:25.717 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T11:10:25.717 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T11:10:25.717 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T11:10:25.717 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T11:10:25.717 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T11:10:25.717 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T11:10:25.717 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T11:10:25.717 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T11:10:25.717 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T11:10:25.717 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T11:10:25.718 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T11:10:25.718 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T11:10:25.718 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T11:10:25.718 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.718 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.718 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T11:10:25.718 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T11:10:25.718 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T11:10:25.719 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.719 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T11:10:25.719 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T11:10:25.719 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T11:10:25.719 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T11:10:25.719 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T11:10:25.719 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T11:10:25.719 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T11:10:25.719 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T11:10:25.719 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T11:10:25.719 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T11:10:25.720 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T11:10:25.720 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T11:10:25.720 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T11:10:25.720 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T11:10:25.720 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T11:10:25.720 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.720 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.720 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:25.720 INFO:teuthology.orchestra.run.smithi102.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T11:10:25.720 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T11:10:25.720 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T11:10:25.720 INFO:teuthology.orchestra.run.smithi102.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T11:10:25.721 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T11:10:25.721 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T11:10:25.721 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T11:10:25.721 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:10:25.721 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-03-21T11:10:26.146 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T11:10:26.189 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T11:10:26.200 DEBUG:teuthology.parallel:result is None 2024-03-21T11:10:26.248 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T11:10:27.684 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T11:10:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T11:10:27.828 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T11:10:27.963 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T11:10:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T11:10:28.030 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T11:10:28.064 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T11:10:28.107 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T11:10:28.107 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T11:10:28.244 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T11:10:28.293 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T11:10:28.293 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T11:10:28.465 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T11:10:28.510 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T11:10:28.556 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T11:10:28.597 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T11:10:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T11:10:28.713 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T11:10:28.786 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T11:10:28.834 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T11:10:28.867 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T11:10:28.901 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T11:10:28.934 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T11:10:28.968 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T11:10:29.006 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T11:10:29.007 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T11:10:29.343 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T11:10:29.344 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T11:10:29.482 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T11:10:29.538 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T11:10:29.592 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T11:10:29.624 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T11:10:29.664 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T11:10:29.797 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T11:10:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T11:10:30.062 INFO:teuthology.orchestra.run.smithi047.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-21T11:10:30.062 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:10:35.025 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T11:10:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T11:10:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T11:10:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T11:10:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T11:10:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T11:10:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T11:10:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T11:10:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T11:10:35.026 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T11:10:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T11:10:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T11:10:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T11:10:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T11:10:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T11:10:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T11:10:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T11:10:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T11:10:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T11:10:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T11:10:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T11:10:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T11:10:35.027 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T11:10:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T11:10:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T11:10:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T11:10:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T11:10:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T11:10:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T11:10:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T11:10:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T11:10:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T11:10:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T11:10:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T11:10:35.028 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T11:10:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T11:10:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T11:10:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T11:10:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T11:10:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T11:10:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T11:10:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T11:10:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T11:10:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T11:10:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T11:10:35.029 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T11:10:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T11:10:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T11:10:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T11:10:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T11:10:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T11:10:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T11:10:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T11:10:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T11:10:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T11:10:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T11:10:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T11:10:35.030 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T11:10:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T11:10:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T11:10:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T11:10:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T11:10:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T11:10:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T11:10:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T11:10:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T11:10:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T11:10:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T11:10:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T11:10:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T11:10:35.031 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T11:10:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T11:10:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T11:10:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T11:10:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T11:10:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T11:10:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T11:10:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T11:10:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T11:10:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T11:10:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T11:10:35.032 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T11:10:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T11:10:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T11:10:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T11:10:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T11:10:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T11:10:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T11:10:35.033 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T11:10:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T11:10:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T11:10:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T11:10:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T11:10:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T11:10:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T11:10:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T11:10:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T11:10:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T11:10:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T11:10:35.034 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T11:10:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T11:10:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T11:10:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T11:10:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T11:10:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T11:10:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T11:10:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T11:10:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T11:10:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T11:10:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T11:10:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T11:10:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T11:10:35.035 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T11:10:35.036 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T11:10:35.036 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T11:10:35.036 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T11:10:35.036 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T11:10:35.036 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T11:10:35.036 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T11:10:35.036 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T11:10:35.036 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T11:10:35.036 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T11:10:35.036 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T11:10:35.036 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T11:10:35.036 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T11:10:35.036 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T11:10:35.037 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T11:10:35.037 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T11:10:35.037 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T11:10:35.037 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T11:10:35.037 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T11:10:35.037 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T11:10:35.037 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T11:10:35.037 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T11:10:35.037 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T11:10:35.037 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T11:10:35.037 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T11:10:35.038 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T11:10:35.038 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T11:10:35.038 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T11:10:35.038 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T11:10:35.038 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T11:10:35.038 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T11:10:35.038 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T11:10:35.038 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T11:10:35.038 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T11:10:35.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T11:10:35.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T11:10:35.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T11:10:35.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T11:10:35.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T11:10:35.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T11:10:35.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T11:10:35.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T11:10:35.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T11:10:35.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T11:10:35.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T11:10:35.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T11:10:35.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T11:10:35.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T11:10:35.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T11:10:35.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T11:10:35.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T11:10:35.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T11:10:35.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T11:10:35.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T11:10:35.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T11:10:35.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T11:10:35.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T11:10:35.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T11:10:35.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T11:10:35.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T11:10:35.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T11:10:35.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T11:10:35.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T11:10:35.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T11:10:35.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T11:10:35.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T11:10:35.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T11:10:35.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T11:10:35.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T11:10:35.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T11:10:35.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T11:10:35.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T11:10:35.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T11:10:35.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T11:10:35.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T11:10:35.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T11:10:35.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T11:10:35.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T11:10:35.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T11:10:35.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T11:10:35.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T11:10:35.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T11:10:35.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T11:10:35.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T11:10:35.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T11:10:37.476 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T11:10:37.476 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:10:37.476 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-03-21T11:10:37.476 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T11:10:37.476 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T11:10:37.476 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T11:10:37.476 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T11:10:37.476 INFO:teuthology.orchestra.run.smithi047.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T11:10:37.476 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T11:10:37.477 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T11:10:37.477 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T11:10:37.477 INFO:teuthology.orchestra.run.smithi047.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T11:10:37.477 INFO:teuthology.orchestra.run.smithi047.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T11:10:37.477 INFO:teuthology.orchestra.run.smithi047.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T11:10:37.477 INFO:teuthology.orchestra.run.smithi047.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T11:10:37.477 INFO:teuthology.orchestra.run.smithi047.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T11:10:37.477 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.477 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.477 INFO:teuthology.orchestra.run.smithi047.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T11:10:37.477 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T11:10:37.477 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T11:10:37.478 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T11:10:37.478 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T11:10:37.478 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T11:10:37.478 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T11:10:37.478 INFO:teuthology.orchestra.run.smithi047.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T11:10:37.478 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T11:10:37.478 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-03-21T11:10:37.478 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T11:10:37.478 INFO:teuthology.orchestra.run.smithi047.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T11:10:37.478 INFO:teuthology.orchestra.run.smithi047.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T11:10:37.478 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T11:10:37.479 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.479 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.479 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.479 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.479 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:37.479 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.479 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.479 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.479 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:37.479 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:37.479 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:37.479 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:37.480 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:37.480 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.480 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.480 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:37.480 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.480 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.480 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.480 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:37.480 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:37.480 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.480 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.481 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T11:10:37.481 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T11:10:37.481 INFO:teuthology.orchestra.run.smithi047.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T11:10:37.481 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T11:10:37.481 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T11:10:37.481 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T11:10:37.481 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T11:10:37.481 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T11:10:37.481 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T11:10:37.481 INFO:teuthology.orchestra.run.smithi047.stdout: jq-1.6-16.el9.x86_64 2024-03-21T11:10:37.481 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T11:10:37.481 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T11:10:37.482 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T11:10:37.482 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.482 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.482 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.482 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T11:10:37.482 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T11:10:37.482 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T11:10:37.482 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T11:10:37.482 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T11:10:37.482 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.482 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.483 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T11:10:37.483 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.483 INFO:teuthology.orchestra.run.smithi047.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T11:10:37.483 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T11:10:37.483 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T11:10:37.483 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T11:10:37.483 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T11:10:37.483 INFO:teuthology.orchestra.run.smithi047.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T11:10:37.483 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T11:10:37.483 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T11:10:37.483 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T11:10:37.483 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T11:10:37.484 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T11:10:37.484 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T11:10:37.484 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T11:10:37.484 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T11:10:37.484 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T11:10:37.484 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T11:10:37.484 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T11:10:37.484 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T11:10:37.484 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T11:10:37.484 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T11:10:37.484 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T11:10:37.484 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T11:10:37.484 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T11:10:37.485 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T11:10:37.485 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.485 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.485 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.485 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T11:10:37.485 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T11:10:37.485 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T11:10:37.485 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T11:10:37.485 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T11:10:37.485 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T11:10:37.485 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T11:10:37.485 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T11:10:37.486 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T11:10:37.486 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T11:10:37.486 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T11:10:37.486 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T11:10:37.486 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T11:10:37.486 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T11:10:37.486 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T11:10:37.486 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T11:10:37.486 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T11:10:37.486 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T11:10:37.486 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T11:10:37.487 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T11:10:37.487 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T11:10:37.487 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T11:10:37.487 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T11:10:37.487 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T11:10:37.487 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T11:10:37.487 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T11:10:37.487 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T11:10:37.487 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T11:10:37.487 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T11:10:37.487 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T11:10:37.487 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T11:10:37.488 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T11:10:37.488 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T11:10:37.488 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T11:10:37.488 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T11:10:37.488 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T11:10:37.488 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T11:10:37.488 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T11:10:37.488 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T11:10:37.488 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T11:10:37.488 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.488 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.488 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T11:10:37.489 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T11:10:37.489 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T11:10:37.489 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.489 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T11:10:37.489 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T11:10:37.489 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T11:10:37.489 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T11:10:37.489 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T11:10:37.489 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T11:10:37.489 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T11:10:37.489 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T11:10:37.489 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T11:10:37.490 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T11:10:37.490 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T11:10:37.490 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T11:10:37.490 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T11:10:37.490 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T11:10:37.490 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T11:10:37.490 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.490 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.490 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T11:10:37.490 INFO:teuthology.orchestra.run.smithi047.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T11:10:37.490 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T11:10:37.490 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T11:10:37.491 INFO:teuthology.orchestra.run.smithi047.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T11:10:37.491 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T11:10:37.491 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T11:10:37.491 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T11:10:37.491 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:10:37.491 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-03-21T11:10:38.182 DEBUG:teuthology.parallel:result is None 2024-03-21T11:10:38.183 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T11:10:38.331 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T11:10:38.388 INFO:teuthology.orchestra.run.smithi047.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T11:10:38.389 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T11:10:38.389 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T11:10:38.391 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T11:10:38.648 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T11:10:38.674 INFO:teuthology.orchestra.run.smithi100.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T11:10:38.674 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T11:10:38.674 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T11:10:38.676 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T11:10:38.900 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T11:10:38.925 INFO:teuthology.orchestra.run.smithi102.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T11:10:38.925 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T11:10:38.925 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T11:10:38.927 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-21T11:10:38.927 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:10:38.927 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T11:10:39.007 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T11:10:39.007 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T11:10:39.042 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-21T11:10:39.042 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T11:10:39.076 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-21T11:10:39.076 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:10:39.076 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T11:10:39.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T11:10:39.269 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T11:10:39.270 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T11:10:39.301 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T11:10:39.370 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-21T11:10:39.370 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T11:10:39.398 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T11:10:39.466 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-21T11:10:39.466 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:10:39.466 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T11:10:39.547 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T11:10:39.663 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T11:10:39.664 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T11:10:39.690 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T11:10:39.757 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-21T11:10:39.757 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T11:10:39.784 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T11:10:39.851 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-21T11:10:39.851 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:10:39.851 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T11:10:39.927 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T11:10:40.040 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T11:10:40.040 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T11:10:40.067 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T11:10:40.134 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-21T11:10:40.134 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T11:10:40.160 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T11:10:40.227 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-21T11:10:40.327 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'subvols': {'create': 2}} 2024-03-21T11:10:40.327 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T11:10:40.328 INFO:tasks.cephadm:Cluster fsid is a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:10:40.328 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-21T11:10:40.328 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.47', 'mon.b': '172.21.15.100', 'mon.c': '172.21.15.102'} 2024-03-21T11:10:40.329 INFO:tasks.cephadm:First mon is mon.a on smithi047 2024-03-21T11:10:40.329 INFO:tasks.cephadm:First mgr is x 2024-03-21T11:10:40.329 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-21T11:10:40.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname $(hostname -s) 2024-03-21T11:10:40.400 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname $(hostname -s) 2024-03-21T11:10:40.427 DEBUG:teuthology.orchestra.run.smithi102:> sudo hostname $(hostname -s) 2024-03-21T11:10:40.454 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-21T11:10:40.454 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T11:10:40.671 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77791/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2231-g2edfee68', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2231.g2edfee68'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-20 07:25:08.579979', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240320.053002', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-21T11:10:40.812 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-vshankar-testing-20240320.053002, sha1 2edfee68ad7243bed422a049b6f5f58619300f11 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T11:10:40.814 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm 2024-03-21T11:10:40.814 DEBUG:teuthology.orchestra.run.smithi047:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T11:10:41.307 INFO:teuthology.orchestra.run.smithi047.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 11:10 /home/ubuntu/cephtest/cephadm 2024-03-21T11:10:41.307 DEBUG:teuthology.orchestra.run.smithi100:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T11:10:41.756 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 11:10 /home/ubuntu/cephtest/cephadm 2024-03-21T11:10:41.757 DEBUG:teuthology.orchestra.run.smithi102:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T11:10:42.439 INFO:teuthology.orchestra.run.smithi102.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 11:10 /home/ubuntu/cephtest/cephadm 2024-03-21T11:10:42.439 DEBUG:teuthology.orchestra.run.smithi047:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T11:10:42.495 DEBUG:teuthology.orchestra.run.smithi100:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T11:10:42.513 DEBUG:teuthology.orchestra.run.smithi102:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T11:10:42.547 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 on all hosts... 2024-03-21T11:10:42.547 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T11:10:42.552 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T11:10:42.556 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T11:10:43.441 INFO:teuthology.orchestra.run.smithi102.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T11:10:43.695 INFO:teuthology.orchestra.run.smithi100.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T11:10:44.020 INFO:teuthology.orchestra.run.smithi047.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T11:11:15.530 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-03-21T11:11:15.531 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T11:11:15.531 INFO:teuthology.orchestra.run.smithi102.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T11:11:15.531 INFO:teuthology.orchestra.run.smithi102.stdout: "repo_digests": [ 2024-03-21T11:11:15.531 INFO:teuthology.orchestra.run.smithi102.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T11:11:15.531 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-03-21T11:11:15.531 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-03-21T11:11:20.805 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T11:11:20.805 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T11:11:20.805 INFO:teuthology.orchestra.run.smithi100.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T11:11:20.805 INFO:teuthology.orchestra.run.smithi100.stdout: "repo_digests": [ 2024-03-21T11:11:20.805 INFO:teuthology.orchestra.run.smithi100.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T11:11:20.806 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-21T11:11:20.806 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T11:11:21.401 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:11:21.401 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T11:11:21.401 INFO:teuthology.orchestra.run.smithi047.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T11:11:21.401 INFO:teuthology.orchestra.run.smithi047.stdout: "repo_digests": [ 2024-03-21T11:11:21.401 INFO:teuthology.orchestra.run.smithi047.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T11:11:21.401 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T11:11:21.401 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:11:21.468 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph 2024-03-21T11:11:21.574 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph 2024-03-21T11:11:21.610 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph 2024-03-21T11:11:21.644 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /etc/ceph 2024-03-21T11:11:21.747 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /etc/ceph 2024-03-21T11:11:21.777 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /etc/ceph 2024-03-21T11:11:21.809 INFO:tasks.cephadm:Writing seed config... 2024-03-21T11:11:21.810 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-21T11:11:21.810 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-21T11:11:21.810 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-21T11:11:21.810 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-21T11:11:21.810 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-21T11:11:21.810 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-21T11:11:21.810 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-21T11:11:21.810 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-21T11:11:21.811 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-21T11:11:21.811 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-21T11:11:21.811 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-21T11:11:21.811 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-21T11:11:21.811 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-21T11:11:21.811 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-21T11:11:21.811 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-21T11:11:21.811 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-21T11:11:21.811 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-21T11:11:21.811 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-21T11:11:21.811 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-21T11:11:21.811 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-21T11:11:21.812 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-21T11:11:21.812 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-21T11:11:21.812 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-21T11:11:21.812 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-21T11:11:21.812 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-21T11:11:21.812 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-21T11:11:21.812 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-21T11:11:21.812 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-21T11:11:21.812 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-21T11:11:21.812 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-21T11:11:21.812 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-21T11:11:21.812 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-21T11:11:21.812 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-21T11:11:21.812 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-21T11:11:21.813 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-21T11:11:21.813 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-21T11:11:21.813 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-21T11:11:21.813 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-21T11:11:21.813 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-21T11:11:21.813 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-21T11:11:21.813 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-21T11:11:21.813 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-21T11:11:21.813 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-21T11:11:21.813 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-21T11:11:21.813 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-21T11:11:21.814 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:11:21.814 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-21T11:11:21.872 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a6f35682-e773-11ee-95cd-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-21T11:11:21.872 DEBUG:teuthology.orchestra.run.smithi047:mon.a> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.a.service 2024-03-21T11:11:21.919 DEBUG:teuthology.orchestra.run.smithi047:mgr.x> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mgr.x.service 2024-03-21T11:11:21.965 INFO:tasks.cephadm:Bootstrapping... 2024-03-21T11:11:21.965 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 -v bootstrap --fsid a6f35682-e773-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.47 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-21T11:11:22.243 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-03-21T11:11:22.243 INFO:teuthology.orchestra.run.smithi047.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11', '-v', 'bootstrap', '--fsid', 'a6f35682-e773-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.47', '--skip-admin-label'] 2024-03-21T11:11:22.243 INFO:teuthology.orchestra.run.smithi047.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-21T11:11:22.244 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying podman|docker is present... 2024-03-21T11:11:22.304 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T11:11:22.304 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying lvm2 is present... 2024-03-21T11:11:22.304 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying time synchronization is in place... 2024-03-21T11:11:22.321 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T11:11:22.321 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T11:11:22.337 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T11:11:22.337 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-03-21T11:11:22.356 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout enabled 2024-03-21T11:11:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout active 2024-03-21T11:11:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:Unit chronyd.service is enabled and running 2024-03-21T11:11:22.374 INFO:teuthology.orchestra.run.smithi047.stdout:Repeating the final host check... 2024-03-21T11:11:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T11:11:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-21T11:11:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl is present 2024-03-21T11:11:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:lvcreate is present 2024-03-21T11:11:22.452 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T11:11:22.453 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T11:11:22.472 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T11:11:22.473 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-03-21T11:11:22.499 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout enabled 2024-03-21T11:11:22.524 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout active 2024-03-21T11:11:22.525 INFO:teuthology.orchestra.run.smithi047.stdout:Unit chronyd.service is enabled and running 2024-03-21T11:11:22.525 INFO:teuthology.orchestra.run.smithi047.stdout:Host looks OK 2024-03-21T11:11:22.525 INFO:teuthology.orchestra.run.smithi047.stdout:Cluster fsid: a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:11:22.527 INFO:teuthology.orchestra.run.smithi047.stdout:Acquiring lock 140273214800848 on /run/cephadm/a6f35682-e773-11ee-95cd-87774f69a715.lock 2024-03-21T11:11:22.527 INFO:teuthology.orchestra.run.smithi047.stdout:Lock 140273214800848 acquired on /run/cephadm/a6f35682-e773-11ee-95cd-87774f69a715.lock 2024-03-21T11:11:22.529 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying IP 172.21.15.47 port 3300 ... 2024-03-21T11:11:22.531 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying IP 172.21.15.47 port 6789 ... 2024-03-21T11:11:22.531 INFO:teuthology.orchestra.run.smithi047.stdout:Base mon IP(s) is [172.21.15.47:3300, 172.21.15.47:6789], mon addrv is [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-03-21T11:11:22.550 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.47 metric 100 2024-03-21T11:11:22.550 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.47 metric 100 2024-03-21T11:11:22.563 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-21T11:11:22.564 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-21T11:11:22.564 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-21T11:11:22.575 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-21T11:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-21T11:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T11:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-21T11:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-21T11:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T11:11:22.579 INFO:teuthology.orchestra.run.smithi047.stdout:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-03-21T11:11:22.579 INFO:teuthology.orchestra.run.smithi047.stdout:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-03-21T11:11:22.579 INFO:teuthology.orchestra.run.smithi047.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-21T11:11:22.582 INFO:teuthology.orchestra.run.smithi047.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-21T11:11:22.585 INFO:teuthology.orchestra.run.smithi047.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T11:11:30.581 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 2024-03-21T11:11:30.581 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T11:11:30.582 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Getting image source signatures 2024-03-21T11:11:30.582 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying blob sha256:96a207f569d15e36f8e9e3b2feafd8af7b93861f24c2b60ada6c9667dd5405be 2024-03-21T11:11:30.582 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-21T11:11:30.582 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying config sha256:237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 2024-03-21T11:11:30.582 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-21T11:11:31.884 INFO:teuthology.orchestra.run.smithi047.stdout:ceph: stdout ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev) 2024-03-21T11:11:31.885 INFO:teuthology.orchestra.run.smithi047.stdout:Ceph version: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev) 2024-03-21T11:11:31.885 INFO:teuthology.orchestra.run.smithi047.stdout:Extracting ceph user uid/gid from container image... 2024-03-21T11:11:33.024 INFO:teuthology.orchestra.run.smithi047.stdout:stat: stdout 167 167 2024-03-21T11:11:33.025 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial keys... 2024-03-21T11:11:34.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQDlFfxlwflqHBAAiImndnjy59mAgI5bDugYdg== 2024-03-21T11:11:35.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQDmFfxl6oLcIhAAlPUx12VZw/wwHoEq7UFRBA== 2024-03-21T11:11:37.207 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQDnFfxlJFbALRAAZV4gwsDXd6g4A722V46DjA== 2024-03-21T11:11:37.212 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial monmap... 2024-03-21T11:11:38.321 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T11:11:38.321 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-21T11:11:38.321 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:11:38.322 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T11:11:38.322 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool for a [v2:172.21.15.47:3300,v1:172.21.15.47:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T11:11:38.322 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = quincy 2024-03-21T11:11:38.322 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: set fsid to a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:11:38.322 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T11:11:38.322 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:11:38.322 INFO:teuthology.orchestra.run.smithi047.stdout:Creating mon... 2024-03-21T11:11:39.414 INFO:teuthology.orchestra.run.smithi047.stdout:create mon.a on 2024-03-21T11:11:39.784 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-21T11:11:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-21T11:11:40.321 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a6f35682-e773-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-a6f35682-e773-11ee-95cd-87774f69a715.target. 2024-03-21T11:11:40.321 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a6f35682-e773-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-a6f35682-e773-11ee-95cd-87774f69a715.target. 2024-03-21T11:11:40.742 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.a 2024-03-21T11:11:40.742 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-21T11:11:41.029 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a6f35682-e773-11ee-95cd-87774f69a715.target.wants/ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-a6f35682-e773-11ee-95cd-87774f69a715@.service. 2024-03-21T11:11:41.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:41 smithi047 podman[30961]: 2024-03-21 11:11:41.647039335 +0000 UTC m=+0.343422789 container start 1e510d91fb6db85c247f394ceb7a90eb08ed549cb5ad51035d8bab9a32f4ca2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-21T11:11:41.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:41 smithi047 bash[30961]: 1e510d91fb6db85c247f394ceb7a90eb08ed549cb5ad51035d8bab9a32f4ca2c 2024-03-21T11:11:41.747 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T11:11:41.747 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-03-21T11:11:41.762 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T11:11:41.762 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-03-21T11:11:41.762 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-03-21T11:11:41.762 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T11:11:41.762 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mon to start... 2024-03-21T11:11:41.763 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mon... 2024-03-21T11:11:42.004 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:41 smithi047 systemd[1]: Started Ceph mon.a for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:11:42.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:41 smithi047 ceph-mon[30980]: mkfs a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:11:42.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:41 smithi047 ceph-mon[30980]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T11:11:43.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:42 smithi047 ceph-mon[30980]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T11:11:43.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:42 smithi047 ceph-mon[30980]: monmap e1: 1 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T11:11:43.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:42 smithi047 ceph-mon[30980]: fsmap 2024-03-21T11:11:43.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:42 smithi047 ceph-mon[30980]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T11:11:43.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:42 smithi047 ceph-mon[30980]: mgrmap e1: no daemons active 2024-03-21T11:11:43.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:42 smithi047 ceph-mon[30980]: from='client.? 172.21.15.47:0/1007336755' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-21T11:11:43.095 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout cluster: 2024-03-21T11:11:43.105 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout id: a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:11:43.105 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-21T11:11:43.105 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T11:11:43.105 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout services: 2024-03-21T11:11:43.105 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.67427s) 2024-03-21T11:11:43.105 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-21T11:11:43.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-21T11:11:43.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T11:11:43.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout data: 2024-03-21T11:11:43.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-21T11:11:43.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-21T11:11:43.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-21T11:11:43.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout pgs: 2024-03-21T11:11:43.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T11:11:43.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.304+0000 7f52b2000700 1 Processor -- start 2024-03-21T11:11:43.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.305+0000 7f52b2000700 1 -- start start 2024-03-21T11:11:43.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.305+0000 7f52b2000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac07e880 0x7f52ac07eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:11:43.107 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.305+0000 7f52b2000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac07f270 con 0x7f52ac07e880 2024-03-21T11:11:43.107 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.306+0000 7f52ab400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac07e880 0x7f52ac07eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:11:43.107 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.306+0000 7f52ab400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac07e880 0x7f52ac07eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57640/0 (socket says 172.21.15.47:57640) 2024-03-21T11:11:43.107 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.306+0000 7f52ab400700 1 -- 172.21.15.47:0/671144877 learned_addr learned my addr 172.21.15.47:0/671144877 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:11:43.107 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.307+0000 7f52ab400700 1 -- 172.21.15.47:0/671144877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac07fa30 con 0x7f52ac07e880 2024-03-21T11:11:43.107 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.307+0000 7f52ab400700 1 --2- 172.21.15.47:0/671144877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac07e880 0x7f52ac07eca0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f529c0098a0 tx=0x7f529c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9b55b72dc71e45 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:11:43.107 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.307+0000 7f52aaa00700 1 -- 172.21.15.47:0/671144877 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f529c004160 con 0x7f52ac07e880 2024-03-21T11:11:43.107 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.308+0000 7f52aaa00700 1 -- 172.21.15.47:0/671144877 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c004300 con 0x7f52ac07e880 2024-03-21T11:11:43.107 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.308+0000 7f52aaa00700 1 -- 172.21.15.47:0/671144877 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f529c0036a0 con 0x7f52ac07e880 2024-03-21T11:11:43.107 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.308+0000 7f52b2000700 1 -- 172.21.15.47:0/671144877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac07e880 msgr2=0x7f52ac07eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:11:43.108 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.308+0000 7f52b2000700 1 --2- 172.21.15.47:0/671144877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac07e880 0x7f52ac07eca0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f529c0098a0 tx=0x7f529c009bb0 comp rx=0 tx=0).stop 2024-03-21T11:11:43.108 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.308+0000 7f52b2000700 1 -- 172.21.15.47:0/671144877 shutdown_connections 2024-03-21T11:11:43.108 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.308+0000 7f52b2000700 1 --2- 172.21.15.47:0/671144877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac07e880 0x7f52ac07eca0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:11:43.108 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.308+0000 7f52b2000700 1 -- 172.21.15.47:0/671144877 >> 172.21.15.47:0/671144877 conn(0x7f52ac07d220 msgr2=0x7f52ac07d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:11:43.108 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.309+0000 7f52b2000700 1 -- 172.21.15.47:0/671144877 shutdown_connections 2024-03-21T11:11:43.108 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.309+0000 7f52b2000700 1 -- 172.21.15.47:0/671144877 wait complete. 2024-03-21T11:11:43.108 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.310+0000 7f52b2000700 1 Processor -- start 2024-03-21T11:11:43.108 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.310+0000 7f52b2000700 1 -- start start 2024-03-21T11:11:43.108 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.310+0000 7f52b2000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac07e880 0x7f52ac1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:11:43.108 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.311+0000 7f52b2000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac07ff60 con 0x7f52ac07e880 2024-03-21T11:11:43.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.311+0000 7f52ab400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac07e880 0x7f52ac1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:11:43.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.311+0000 7f52ab400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac07e880 0x7f52ac1a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57654/0 (socket says 172.21.15.47:57654) 2024-03-21T11:11:43.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.311+0000 7f52ab400700 1 -- 172.21.15.47:0/1007336755 learned_addr learned my addr 172.21.15.47:0/1007336755 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:11:43.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.311+0000 7f52ab400700 1 -- 172.21.15.47:0/1007336755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac1a1d70 con 0x7f52ac07e880 2024-03-21T11:11:43.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.312+0000 7f52ab400700 1 --2- 172.21.15.47:0/1007336755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac07e880 0x7f52ac1a1830 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f529c000c00 tx=0x7f529c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:11:43.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.312+0000 7f52a8c00700 1 -- 172.21.15.47:0/1007336755 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f529c003db0 con 0x7f52ac07e880 2024-03-21T11:11:43.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.312+0000 7f52a8c00700 1 -- 172.21.15.47:0/1007336755 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c011630 con 0x7f52ac07e880 2024-03-21T11:11:43.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.312+0000 7f52a8c00700 1 -- 172.21.15.47:0/1007336755 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f529c011950 con 0x7f52ac07e880 2024-03-21T11:11:43.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.312+0000 7f52b2000700 1 -- 172.21.15.47:0/1007336755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ac1a2000 con 0x7f52ac07e880 2024-03-21T11:11:43.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.312+0000 7f52b2000700 1 -- 172.21.15.47:0/1007336755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ac1a4ca0 con 0x7f52ac07e880 2024-03-21T11:11:43.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.313+0000 7f52a8c00700 1 -- 172.21.15.47:0/1007336755 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f529c01a460 con 0x7f52ac07e880 2024-03-21T11:11:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.313+0000 7f52a8c00700 1 -- 172.21.15.47:0/1007336755 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f529c01b780 con 0x7f52ac07e880 2024-03-21T11:11:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.313+0000 7f52b2000700 1 -- 172.21.15.47:0/1007336755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ac0637c0 con 0x7f52ac07e880 2024-03-21T11:11:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.317+0000 7f52a8c00700 1 -- 172.21.15.47:0/1007336755 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f529c01ba20 con 0x7f52ac07e880 2024-03-21T11:11:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.397+0000 7f52b2000700 1 -- 172.21.15.47:0/1007336755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f52ac02dd30 con 0x7f52ac07e880 2024-03-21T11:11:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.398+0000 7f52a8c00700 1 -- 172.21.15.47:0/1007336755 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+316 (secure 0 0 0) 0x7f529c01bc00 con 0x7f52ac07e880 2024-03-21T11:11:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.399+0000 7f52b2000700 1 -- 172.21.15.47:0/1007336755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac07e880 msgr2=0x7f52ac1a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:11:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.399+0000 7f52b2000700 1 --2- 172.21.15.47:0/1007336755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac07e880 0x7f52ac1a1830 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f529c000c00 tx=0x7f529c004750 comp rx=0 tx=0).stop 2024-03-21T11:11:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.399+0000 7f52b2000700 1 -- 172.21.15.47:0/1007336755 shutdown_connections 2024-03-21T11:11:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.399+0000 7f52b2000700 1 --2- 172.21.15.47:0/1007336755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52ac07e880 0x7f52ac1a1830 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:11:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.399+0000 7f52b2000700 1 -- 172.21.15.47:0/1007336755 >> 172.21.15.47:0/1007336755 conn(0x7f52ac07d220 msgr2=0x7f52ac107830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:11:43.111 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.399+0000 7f52b2000700 1 -- 172.21.15.47:0/1007336755 shutdown_connections 2024-03-21T11:11:43.111 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:42.399+0000 7f52b2000700 1 -- 172.21.15.47:0/1007336755 wait complete. 2024-03-21T11:11:43.111 INFO:teuthology.orchestra.run.smithi047.stdout:mon is available 2024-03-21T11:11:43.111 INFO:teuthology.orchestra.run.smithi047.stdout:Assimilating anything we can from ceph.conf... 2024-03-21T11:11:44.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:43 smithi047 ceph-mon[30980]: from='client.? 172.21.15.47:0/3760278906' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T11:11:44.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:43 smithi047 ceph-mon[30980]: from='client.? 172.21.15.47:0/3760278906' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-21T11:11:44.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T11:11:44.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T11:11:44.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout fsid = a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:11:44.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-03-21T11:11:44.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T11:11:44.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T11:11:44.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T11:11:44.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T11:11:44.398 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T11:11:44.399 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T11:11:44.399 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T11:11:44.399 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T11:11:44.399 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T11:11:44.399 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-21T11:11:44.399 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-21T11:11:44.399 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-21T11:11:44.399 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T11:11:44.400 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T11:11:44.400 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.675+0000 7f8ff1e00700 1 Processor -- start 2024-03-21T11:11:44.400 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.676+0000 7f8ff1e00700 1 -- start start 2024-03-21T11:11:44.400 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.676+0000 7f8ff1e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fec0822f0 0x7f8fec110d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:11:44.401 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.676+0000 7f8ff1e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec1112d0 con 0x7f8fec0822f0 2024-03-21T11:11:44.401 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.677+0000 7f8feaa00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fec0822f0 0x7f8fec110d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:11:44.401 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.677+0000 7f8feaa00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fec0822f0 0x7f8fec110d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57670/0 (socket says 172.21.15.47:57670) 2024-03-21T11:11:44.401 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.677+0000 7f8feaa00700 1 -- 172.21.15.47:0/3941831707 learned_addr learned my addr 172.21.15.47:0/3941831707 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:11:44.402 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.677+0000 7f8feaa00700 1 -- 172.21.15.47:0/3941831707 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec111450 con 0x7f8fec0822f0 2024-03-21T11:11:44.402 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.677+0000 7f8feaa00700 1 --2- 172.21.15.47:0/3941831707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fec0822f0 0x7f8fec110d00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0098a0 tx=0x7f8fdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74dc7f1f9ccb9db8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:11:44.403 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.678+0000 7f8fe9600700 1 -- 172.21.15.47:0/3941831707 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fdc004160 con 0x7f8fec0822f0 2024-03-21T11:11:44.403 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.678+0000 7f8fe9600700 1 -- 172.21.15.47:0/3941831707 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fdc004300 con 0x7f8fec0822f0 2024-03-21T11:11:44.403 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.678+0000 7f8fe9600700 1 -- 172.21.15.47:0/3941831707 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fdc0036a0 con 0x7f8fec0822f0 2024-03-21T11:11:44.403 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.678+0000 7f8ff1e00700 1 -- 172.21.15.47:0/3941831707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fec0822f0 msgr2=0x7f8fec110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:11:44.403 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.678+0000 7f8ff1e00700 1 --2- 172.21.15.47:0/3941831707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fec0822f0 0x7f8fec110d00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0098a0 tx=0x7f8fdc009bb0 comp rx=0 tx=0).stop 2024-03-21T11:11:44.403 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.679+0000 7f8ff1e00700 1 -- 172.21.15.47:0/3941831707 shutdown_connections 2024-03-21T11:11:44.403 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.679+0000 7f8ff1e00700 1 --2- 172.21.15.47:0/3941831707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fec0822f0 0x7f8fec110d00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:11:44.403 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.679+0000 7f8ff1e00700 1 -- 172.21.15.47:0/3941831707 >> 172.21.15.47:0/3941831707 conn(0x7f8fec07d220 msgr2=0x7f8fec07f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:11:44.403 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.679+0000 7f8ff1e00700 1 -- 172.21.15.47:0/3941831707 shutdown_connections 2024-03-21T11:11:44.403 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.679+0000 7f8ff1e00700 1 -- 172.21.15.47:0/3941831707 wait complete. 2024-03-21T11:11:44.405 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.680+0000 7f8ff1e00700 1 Processor -- start 2024-03-21T11:11:44.405 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.680+0000 7f8ff1e00700 1 -- start start 2024-03-21T11:11:44.405 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.680+0000 7f8ff1e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fec1a1a40 0x7f8fec1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:11:44.405 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.680+0000 7f8ff1e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec111980 con 0x7f8fec1a1a40 2024-03-21T11:11:44.405 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.681+0000 7f8feaa00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fec1a1a40 0x7f8fec1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:11:44.406 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.681+0000 7f8feaa00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fec1a1a40 0x7f8fec1a1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57676/0 (socket says 172.21.15.47:57676) 2024-03-21T11:11:44.406 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.681+0000 7f8feaa00700 1 -- 172.21.15.47:0/3760278906 learned_addr learned my addr 172.21.15.47:0/3760278906 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:11:44.406 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.681+0000 7f8feaa00700 1 -- 172.21.15.47:0/3760278906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec1a23a0 con 0x7f8fec1a1a40 2024-03-21T11:11:44.406 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.681+0000 7f8feaa00700 1 --2- 172.21.15.47:0/3760278906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fec1a1a40 0x7f8fec1a1e60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc000c00 tx=0x7f8fdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:11:44.406 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.682+0000 7f8ff0a00700 1 -- 172.21.15.47:0/3760278906 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fdc003ca0 con 0x7f8fec1a1a40 2024-03-21T11:11:44.406 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.682+0000 7f8ff0a00700 1 -- 172.21.15.47:0/3760278906 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fdc003e40 con 0x7f8fec1a1a40 2024-03-21T11:11:44.406 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.682+0000 7f8ff0a00700 1 -- 172.21.15.47:0/3760278906 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fdc0117a0 con 0x7f8fec1a1a40 2024-03-21T11:11:44.406 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.682+0000 7f8ff1e00700 1 -- 172.21.15.47:0/3760278906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec1a2630 con 0x7f8fec1a1a40 2024-03-21T11:11:44.406 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.682+0000 7f8ff1e00700 1 -- 172.21.15.47:0/3760278906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec1a5110 con 0x7f8fec1a1a40 2024-03-21T11:11:44.406 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.682+0000 7f8ff0a00700 1 -- 172.21.15.47:0/3760278906 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8fdc011cd0 con 0x7f8fec1a1a40 2024-03-21T11:11:44.406 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.682+0000 7f8ff0a00700 1 -- 172.21.15.47:0/3760278906 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8fdc01a7a0 con 0x7f8fec1a1a40 2024-03-21T11:11:44.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.683+0000 7f8ff1e00700 1 -- 172.21.15.47:0/3760278906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec0637c0 con 0x7f8fec1a1a40 2024-03-21T11:11:44.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.686+0000 7f8ff0a00700 1 -- 172.21.15.47:0/3760278906 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8fdc006e20 con 0x7f8fec1a1a40 2024-03-21T11:11:44.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.764+0000 7f8ff1e00700 1 -- 172.21.15.47:0/3760278906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8fec02dde0 con 0x7f8fec1a1a40 2024-03-21T11:11:44.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.771+0000 7f8ff0a00700 1 -- 172.21.15.47:0/3760278906 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f8fdc024d30 con 0x7f8fec1a1a40 2024-03-21T11:11:44.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.773+0000 7f8ff1e00700 1 -- 172.21.15.47:0/3760278906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fec1a1a40 msgr2=0x7f8fec1a1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:11:44.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.773+0000 7f8ff1e00700 1 --2- 172.21.15.47:0/3760278906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fec1a1a40 0x7f8fec1a1e60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc000c00 tx=0x7f8fdc004750 comp rx=0 tx=0).stop 2024-03-21T11:11:44.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.773+0000 7f8ff1e00700 1 -- 172.21.15.47:0/3760278906 shutdown_connections 2024-03-21T11:11:44.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.773+0000 7f8ff1e00700 1 --2- 172.21.15.47:0/3760278906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fec1a1a40 0x7f8fec1a1e60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:11:44.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.773+0000 7f8ff1e00700 1 -- 172.21.15.47:0/3760278906 >> 172.21.15.47:0/3760278906 conn(0x7f8fec07d220 msgr2=0x7f8fec07f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:11:44.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.773+0000 7f8ff1e00700 1 -- 172.21.15.47:0/3760278906 shutdown_connections 2024-03-21T11:11:44.408 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:43.773+0000 7f8ff1e00700 1 -- 172.21.15.47:0/3760278906 wait complete. 2024-03-21T11:11:44.409 INFO:teuthology.orchestra.run.smithi047.stdout:Generating new minimal ceph.conf... 2024-03-21T11:11:45.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:45 smithi047 ceph-mon[30980]: from='client.? 172.21.15.47:0/3283107828' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:11:45.672 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.041+0000 7f0f53e00700 1 Processor -- start 2024-03-21T11:11:45.672 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.041+0000 7f0f53e00700 1 -- start start 2024-03-21T11:11:45.672 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.042+0000 7f0f53e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f54108440 0x7f0f5410a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:11:45.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.042+0000 7f0f53e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f54075840 con 0x7f0f54108440 2024-03-21T11:11:45.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.042+0000 7f0f52a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f54108440 0x7f0f5410a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:11:45.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.042+0000 7f0f52a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f54108440 0x7f0f5410a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57684/0 (socket says 172.21.15.47:57684) 2024-03-21T11:11:45.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.043+0000 7f0f52a00700 1 -- 172.21.15.47:0/1896548403 learned_addr learned my addr 172.21.15.47:0/1896548403 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:11:45.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.043+0000 7f0f52a00700 1 -- 172.21.15.47:0/1896548403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5410ad70 con 0x7f0f54108440 2024-03-21T11:11:45.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.043+0000 7f0f52a00700 1 --2- 172.21.15.47:0/1896548403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f54108440 0x7f0f5410a830 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0f440098a0 tx=0x7f0f44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e98807b61d7062 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:11:45.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.044+0000 7f0f52000700 1 -- 172.21.15.47:0/1896548403 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f44004160 con 0x7f0f54108440 2024-03-21T11:11:45.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.044+0000 7f0f52000700 1 -- 172.21.15.47:0/1896548403 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f44004300 con 0x7f0f54108440 2024-03-21T11:11:45.675 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.044+0000 7f0f52000700 1 -- 172.21.15.47:0/1896548403 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f44011b50 con 0x7f0f54108440 2024-03-21T11:11:45.675 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.044+0000 7f0f53e00700 1 -- 172.21.15.47:0/1896548403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f54108440 msgr2=0x7f0f5410a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:11:45.675 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.044+0000 7f0f53e00700 1 --2- 172.21.15.47:0/1896548403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f54108440 0x7f0f5410a830 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0f440098a0 tx=0x7f0f44009bb0 comp rx=0 tx=0).stop 2024-03-21T11:11:45.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.044+0000 7f0f53e00700 1 -- 172.21.15.47:0/1896548403 shutdown_connections 2024-03-21T11:11:45.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.044+0000 7f0f53e00700 1 --2- 172.21.15.47:0/1896548403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f54108440 0x7f0f5410a830 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:11:45.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.044+0000 7f0f53e00700 1 -- 172.21.15.47:0/1896548403 >> 172.21.15.47:0/1896548403 conn(0x7f0f54101e30 msgr2=0x7f0f54104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:11:45.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.045+0000 7f0f53e00700 1 -- 172.21.15.47:0/1896548403 shutdown_connections 2024-03-21T11:11:45.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.045+0000 7f0f53e00700 1 -- 172.21.15.47:0/1896548403 wait complete. 2024-03-21T11:11:45.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.046+0000 7f0f53e00700 1 Processor -- start 2024-03-21T11:11:45.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.046+0000 7f0f53e00700 1 -- start start 2024-03-21T11:11:45.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.046+0000 7f0f53e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f54108440 0x7f0f54107cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:11:45.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.046+0000 7f0f53e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5410bce0 con 0x7f0f54108440 2024-03-21T11:11:45.679 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.046+0000 7f0f52a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f54108440 0x7f0f54107cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:11:45.679 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.046+0000 7f0f52a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f54108440 0x7f0f54107cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57692/0 (socket says 172.21.15.47:57692) 2024-03-21T11:11:45.679 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.046+0000 7f0f52a00700 1 -- 172.21.15.47:0/3283107828 learned_addr learned my addr 172.21.15.47:0/3283107828 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:11:45.679 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.047+0000 7f0f52a00700 1 -- 172.21.15.47:0/3283107828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f54108200 con 0x7f0f54108440 2024-03-21T11:11:45.679 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.047+0000 7f0f52a00700 1 --2- 172.21.15.47:0/3283107828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f54108440 0x7f0f54107cc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0f44011b60 tx=0x7f0f44011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:11:45.679 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.047+0000 7f0f4b400700 1 -- 172.21.15.47:0/3283107828 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f44003b20 con 0x7f0f54108440 2024-03-21T11:11:45.679 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.047+0000 7f0f4b400700 1 -- 172.21.15.47:0/3283107828 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f44003cc0 con 0x7f0f54108440 2024-03-21T11:11:45.679 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.047+0000 7f0f4b400700 1 -- 172.21.15.47:0/3283107828 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f44024840 con 0x7f0f54108440 2024-03-21T11:11:45.679 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.047+0000 7f0f53e00700 1 -- 172.21.15.47:0/3283107828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f541063f0 con 0x7f0f54108440 2024-03-21T11:11:45.679 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.048+0000 7f0f53e00700 1 -- 172.21.15.47:0/3283107828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f54106930 con 0x7f0f54108440 2024-03-21T11:11:45.679 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.048+0000 7f0f4b400700 1 -- 172.21.15.47:0/3283107828 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0f44022020 con 0x7f0f54108440 2024-03-21T11:11:45.679 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.048+0000 7f0f4b400700 1 -- 172.21.15.47:0/3283107828 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0f44011e00 con 0x7f0f54108440 2024-03-21T11:11:45.679 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.049+0000 7f0f53e00700 1 -- 172.21.15.47:0/3283107828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f540637c0 con 0x7f0f54108440 2024-03-21T11:11:45.680 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.052+0000 7f0f4b400700 1 -- 172.21.15.47:0/3283107828 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0f44043dd0 con 0x7f0f54108440 2024-03-21T11:11:45.680 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.129+0000 7f0f53e00700 1 -- 172.21.15.47:0/3283107828 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f0f54106c70 con 0x7f0f54108440 2024-03-21T11:11:45.680 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.130+0000 7f0f4b400700 1 -- 172.21.15.47:0/3283107828 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f0f44032020 con 0x7f0f54108440 2024-03-21T11:11:45.680 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.131+0000 7f0f53e00700 1 -- 172.21.15.47:0/3283107828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f54108440 msgr2=0x7f0f54107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:11:45.680 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.131+0000 7f0f53e00700 1 --2- 172.21.15.47:0/3283107828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f54108440 0x7f0f54107cc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0f44011b60 tx=0x7f0f44011c40 comp rx=0 tx=0).stop 2024-03-21T11:11:45.680 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.131+0000 7f0f53e00700 1 -- 172.21.15.47:0/3283107828 shutdown_connections 2024-03-21T11:11:45.681 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.131+0000 7f0f53e00700 1 --2- 172.21.15.47:0/3283107828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f54108440 0x7f0f54107cc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:11:45.681 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.132+0000 7f0f53e00700 1 -- 172.21.15.47:0/3283107828 >> 172.21.15.47:0/3283107828 conn(0x7f0f54101e30 msgr2=0x7f0f54102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:11:45.681 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.132+0000 7f0f53e00700 1 -- 172.21.15.47:0/3283107828 shutdown_connections 2024-03-21T11:11:45.681 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:45.132+0000 7f0f53e00700 1 -- 172.21.15.47:0/3283107828 wait complete. 2024-03-21T11:11:45.681 INFO:teuthology.orchestra.run.smithi047.stdout:Restarting the monitor... 2024-03-21T11:11:46.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:45 smithi047 systemd[1]: Stopping Ceph mon.a for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:11:46.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:46 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a[30976]: 2024-03-21T11:11:46.171+0000 7f5046000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:11:46.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:46 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a[30976]: 2024-03-21T11:11:46.171+0000 7f5046000700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-21T11:11:46.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:46 smithi047 podman[31223]: 2024-03-21 11:11:46.367541564 +0000 UTC m=+0.316264391 container died 1e510d91fb6db85c247f394ceb7a90eb08ed549cb5ad51035d8bab9a32f4ca2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-21T11:11:47.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:46 smithi047 podman[31223]: 2024-03-21 11:11:46.747045192 +0000 UTC m=+0.695768012 container cleanup 1e510d91fb6db85c247f394ceb7a90eb08ed549cb5ad51035d8bab9a32f4ca2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-21T11:11:47.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:46 smithi047 bash[31223]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a 2024-03-21T11:11:47.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:47 smithi047 podman[31236]: 2024-03-21 11:11:47.139768917 +0000 UTC m=+0.749272982 container remove 1e510d91fb6db85c247f394ceb7a90eb08ed549cb5ad51035d8bab9a32f4ca2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:11:47.933 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:47 smithi047 systemd[1]: ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-21T11:11:47.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:47 smithi047 systemd[1]: Stopped Ceph mon.a for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:11:47.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:47 smithi047 systemd[1]: ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.a.service: Consumed 1.992s CPU time. 2024-03-21T11:11:47.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:47 smithi047 systemd[1]: Starting Ceph mon.a for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:11:48.206 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 podman[31332]: 2024-03-21 11:11:47.961031895 +0000 UTC m=+0.040372788 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T11:11:48.206 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 podman[31332]: 2024-03-21 11:11:48.065486302 +0000 UTC m=+0.144827242 container create af33427a459adb8d1f6af38ac16b3b39ae04f6cb4b32fc99b556d7f1a6beb74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-21T11:11:48.427 INFO:teuthology.orchestra.run.smithi047.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-21T11:11:48.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 podman[31332]: 2024-03-21 11:11:48.335990679 +0000 UTC m=+0.415331609 container init af33427a459adb8d1f6af38ac16b3b39ae04f6cb4b32fc99b556d7f1a6beb74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-21T11:11:48.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 podman[31332]: 2024-03-21 11:11:48.346756385 +0000 UTC m=+0.426097291 container start af33427a459adb8d1f6af38ac16b3b39ae04f6cb4b32fc99b556d7f1a6beb74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T11:11:48.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T11:11:48.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T11:11:48.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: pidfile_write: ignore empty --pid-file 2024-03-21T11:11:48.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: load: jerasure load: lrc 2024-03-21T11:11:48.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: RocksDB version: 7.9.2 2024-03-21T11:11:48.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Git sha 0 2024-03-21T11:11:48.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T11:11:48.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: DB SUMMARY 2024-03-21T11:11:48.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: DB Session ID: SOPH8JTGIG0NBNZYA2L9 2024-03-21T11:11:48.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: CURRENT file: CURRENT 2024-03-21T11:11:48.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T11:11:48.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-21T11:11:48.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-21T11:11:48.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100772 ; 2024-03-21T11:11:48.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.error_if_exists: 0 2024-03-21T11:11:48.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.create_if_missing: 0 2024-03-21T11:11:48.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.paranoid_checks: 1 2024-03-21T11:11:48.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T11:11:48.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T11:11:48.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T11:11:48.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.env: 0x5623c57d5b20 2024-03-21T11:11:48.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T11:11:48.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.info_log: 0x5623c66c9960 2024-03-21T11:11:48.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T11:11:48.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.statistics: (nil) 2024-03-21T11:11:48.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.use_fsync: 0 2024-03-21T11:11:48.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_log_file_size: 0 2024-03-21T11:11:48.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T11:11:48.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T11:11:48.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T11:11:48.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T11:11:48.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.allow_fallocate: 1 2024-03-21T11:11:48.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T11:11:48.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T11:11:48.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.use_direct_reads: 0 2024-03-21T11:11:48.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T11:11:48.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T11:11:48.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.db_log_dir: 2024-03-21T11:11:48.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.wal_dir: 2024-03-21T11:11:48.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T11:11:48.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T11:11:48.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T11:11:48.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T11:11:48.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T11:11:48.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T11:11:48.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T11:11:48.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T11:11:48.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.write_buffer_manager: 0x5623c669c1e0 2024-03-21T11:11:48.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T11:11:48.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T11:11:48.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T11:11:48.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T11:11:48.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T11:11:48.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T11:11:48.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T11:11:48.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T11:11:48.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.unordered_write: 0 2024-03-21T11:11:48.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T11:11:48.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T11:11:48.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T11:11:48.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T11:11:48.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.row_cache: None 2024-03-21T11:11:48.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.wal_filter: None 2024-03-21T11:11:48.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T11:11:48.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T11:11:48.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.two_write_queues: 0 2024-03-21T11:11:48.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T11:11:48.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.wal_compression: 0 2024-03-21T11:11:48.704 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.atomic_flush: 0 2024-03-21T11:11:48.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T11:11:48.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T11:11:48.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T11:11:48.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.log_readahead_size: 0 2024-03-21T11:11:48.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T11:11:48.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T11:11:48.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T11:11:48.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T11:11:48.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T11:11:48.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T11:11:48.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T11:11:48.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_background_jobs: 2 2024-03-21T11:11:48.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_background_compactions: -1 2024-03-21T11:11:48.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_subcompactions: 1 2024-03-21T11:11:48.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T11:11:48.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T11:11:48.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T11:11:48.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T11:11:48.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T11:11:48.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T11:11:48.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T11:11:48.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T11:11:48.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_open_files: -1 2024-03-21T11:11:48.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T11:11:48.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T11:11:48.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T11:11:48.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T11:11:48.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_background_flushes: -1 2024-03-21T11:11:48.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Compression algorithms supported: 2024-03-21T11:11:48.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T11:11:48.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: kZSTD supported: 0 2024-03-21T11:11:48.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: kXpressCompression supported: 0 2024-03-21T11:11:48.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T11:11:48.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: kZlibCompression supported: 1 2024-03-21T11:11:48.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: kSnappyCompression supported: 1 2024-03-21T11:11:48.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: kLZ4Compression supported: 1 2024-03-21T11:11:48.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: kBZip2Compression supported: 0 2024-03-21T11:11:48.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T11:11:48.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T11:11:48.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-21T11:11:48.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T11:11:48.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T11:11:48.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.merge_operator: 2024-03-21T11:11:48.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compaction_filter: None 2024-03-21T11:11:48.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T11:11:48.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T11:11:48.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T11:11:48.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T11:11:48.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5623c66c9a60) 2024-03-21T11:11:48.708 INFO:journalctl@ceph.mon.a.smithi047.stdout: cache_index_and_filter_blocks: 1 2024-03-21T11:11:48.708 INFO:journalctl@ceph.mon.a.smithi047.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T11:11:48.708 INFO:journalctl@ceph.mon.a.smithi047.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T11:11:48.708 INFO:journalctl@ceph.mon.a.smithi047.stdout: pin_top_level_index_and_filter: 1 2024-03-21T11:11:48.708 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_type: 0 2024-03-21T11:11:48.709 INFO:journalctl@ceph.mon.a.smithi047.stdout: data_block_index_type: 0 2024-03-21T11:11:48.709 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_shortening: 1 2024-03-21T11:11:48.709 INFO:journalctl@ceph.mon.a.smithi047.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T11:11:48.709 INFO:journalctl@ceph.mon.a.smithi047.stdout: checksum: 4 2024-03-21T11:11:48.709 INFO:journalctl@ceph.mon.a.smithi047.stdout: no_block_cache: 0 2024-03-21T11:11:48.709 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache: 0x5623c66cf090 2024-03-21T11:11:48.709 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_name: BinnedLRUCache 2024-03-21T11:11:48.709 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_options: 2024-03-21T11:11:48.709 INFO:journalctl@ceph.mon.a.smithi047.stdout: capacity : 536870912 2024-03-21T11:11:48.709 INFO:journalctl@ceph.mon.a.smithi047.stdout: num_shard_bits : 4 2024-03-21T11:11:48.709 INFO:journalctl@ceph.mon.a.smithi047.stdout: strict_capacity_limit : 0 2024-03-21T11:11:48.709 INFO:journalctl@ceph.mon.a.smithi047.stdout: high_pri_pool_ratio: 0.000 2024-03-21T11:11:48.709 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_compressed: (nil) 2024-03-21T11:11:48.709 INFO:journalctl@ceph.mon.a.smithi047.stdout: persistent_cache: (nil) 2024-03-21T11:11:48.710 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_size: 4096 2024-03-21T11:11:48.710 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_size_deviation: 10 2024-03-21T11:11:48.710 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_restart_interval: 16 2024-03-21T11:11:48.710 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_block_restart_interval: 1 2024-03-21T11:11:48.710 INFO:journalctl@ceph.mon.a.smithi047.stdout: metadata_block_size: 4096 2024-03-21T11:11:48.710 INFO:journalctl@ceph.mon.a.smithi047.stdout: partition_filters: 0 2024-03-21T11:11:48.710 INFO:journalctl@ceph.mon.a.smithi047.stdout: use_delta_encoding: 1 2024-03-21T11:11:48.710 INFO:journalctl@ceph.mon.a.smithi047.stdout: filter_policy: bloomfilter 2024-03-21T11:11:48.710 INFO:journalctl@ceph.mon.a.smithi047.stdout: whole_key_filtering: 1 2024-03-21T11:11:48.710 INFO:journalctl@ceph.mon.a.smithi047.stdout: verify_compression: 0 2024-03-21T11:11:48.710 INFO:journalctl@ceph.mon.a.smithi047.stdout: read_amp_bytes_per_bit: 0 2024-03-21T11:11:48.710 INFO:journalctl@ceph.mon.a.smithi047.stdout: format_version: 5 2024-03-21T11:11:48.710 INFO:journalctl@ceph.mon.a.smithi047.stdout: enable_index_compression: 1 2024-03-21T11:11:48.711 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_align: 0 2024-03-21T11:11:48.711 INFO:journalctl@ceph.mon.a.smithi047.stdout: max_auto_readahead_size: 262144 2024-03-21T11:11:48.711 INFO:journalctl@ceph.mon.a.smithi047.stdout: prepopulate_block_cache: 0 2024-03-21T11:11:48.711 INFO:journalctl@ceph.mon.a.smithi047.stdout: initial_auto_readahead_size: 8192 2024-03-21T11:11:48.711 INFO:journalctl@ceph.mon.a.smithi047.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T11:11:48.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T11:11:48.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T11:11:48.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compression: NoCompression 2024-03-21T11:11:48.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T11:11:48.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T11:11:48.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T11:11:48.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.num_levels: 7 2024-03-21T11:11:48.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T11:11:48.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T11:11:48.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T11:11:48.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T11:11:48.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T11:11:48.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T11:11:48.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T11:11:48.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T11:11:48.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T11:11:48.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T11:11:48.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T11:11:48.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T11:11:48.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T11:11:48.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T11:11:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T11:11:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T11:11:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T11:11:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T11:11:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T11:11:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T11:11:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T11:11:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T11:11:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T11:11:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T11:11:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T11:11:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T11:11:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T11:11:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T11:11:48.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T11:11:48.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T11:11:48.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T11:11:48.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T11:11:48.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T11:11:48.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T11:11:48.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T11:11:48.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T11:11:48.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T11:11:48.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T11:11:48.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T11:11:48.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T11:11:48.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T11:11:48.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T11:11:48.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T11:11:48.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T11:11:48.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T11:11:48.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T11:11:48.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T11:11:48.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T11:11:48.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T11:11:48.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T11:11:48.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T11:11:48.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T11:11:48.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T11:11:48.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.table_properties_collectors: 2024-03-21T11:11:48.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.inplace_update_support: 0 2024-03-21T11:11:48.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T11:11:48.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T11:11:48.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T11:11:48.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T11:11:48.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.bloom_locality: 0 2024-03-21T11:11:48.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.max_successive_merges: 0 2024-03-21T11:11:48.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T11:11:48.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T11:11:48.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T11:11:48.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T11:11:48.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.ttl: 2592000 2024-03-21T11:11:48.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T11:11:48.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T11:11:48.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T11:11:48.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.enable_blob_files: false 2024-03-21T11:11:48.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.min_blob_size: 0 2024-03-21T11:11:48.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T11:11:48.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T11:11:48.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T11:11:48.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T11:11:48.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T11:11:48.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T11:11:48.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T11:11:48.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T11:11:48.718 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-21T11:11:48.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-21T11:11:48.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: eea76e3c-ab3a-4abf-8a35-69f5cfbbf394 2024-03-21T11:11:48.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711019508404057, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-21T11:11:48.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-21T11:11:48.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711019508406286, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95891, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 93881, "index_size": 292, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13934, "raw_average_key_size": 50, "raw_value_size": 86309, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711019508, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "eea76e3c-ab3a-4abf-8a35-69f5cfbbf394", "db_session_id": "SOPH8JTGIG0NBNZYA2L9", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-21T11:11:48.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711019508406438, "job": 1, "event": "recovery_finished"} 2024-03-21T11:11:48.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-21T11:11:48.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T11:11:48.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5623c67c2000 2024-03-21T11:11:48.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: DB pointer 0x5623c67ac000 2024-03-21T11:11:48.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T11:11:48.719 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T11:11:48.719 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** DB Stats ** 2024-03-21T11:11:48.719 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T11:11:48.720 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T11:11:48.720 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T11:11:48.720 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T11:11:48.720 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T11:11:48.720 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T11:11:48.720 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T11:11:48.720 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-03-21T11:11:48.720 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-03-21T11:11:48.720 INFO:journalctl@ceph.mon.a.smithi047.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T11:11:48.720 INFO:journalctl@ceph.mon.a.smithi047.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T11:11:48.720 INFO:journalctl@ceph.mon.a.smithi047.stdout: L0 2/0 95.48 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 57.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-21T11:11:48.720 INFO:journalctl@ceph.mon.a.smithi047.stdout: Sum 2/0 95.48 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 57.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-21T11:11:48.720 INFO:journalctl@ceph.mon.a.smithi047.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 57.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-21T11:11:48.720 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 57.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative compaction: 0.00 GB write, 13.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T11:11:48.721 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval compaction: 0.00 GB write, 13.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T11:11:48.722 INFO:journalctl@ceph.mon.a.smithi047.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T11:11:48.722 INFO:journalctl@ceph.mon.a.smithi047.stdout: Block cache BinnedLRUCache@0x5623c66cf090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-21T11:11:48.722 INFO:journalctl@ceph.mon.a.smithi047.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-21T11:11:48.722 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-03-21T11:11:48.722 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T11:11:48.722 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: starting mon.a rank 0 at public addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] at bind addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:11:48.722 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: mon.a@-1(???) e1 preinit fsid a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:11:48.722 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: mon.a@-1(???).mds e1 new map 2024-03-21T11:11:48.722 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: mon.a@-1(???).mds e1 print_map 2024-03-21T11:11:48.722 INFO:journalctl@ceph.mon.a.smithi047.stdout: e1 2024-03-21T11:11:48.722 INFO:journalctl@ceph.mon.a.smithi047.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T11:11:48.722 INFO:journalctl@ceph.mon.a.smithi047.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T11:11:48.722 INFO:journalctl@ceph.mon.a.smithi047.stdout: legacy client fscid: -1 2024-03-21T11:11:48.722 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-03-21T11:11:48.723 INFO:journalctl@ceph.mon.a.smithi047.stdout: No filesystems configured 2024-03-21T11:11:48.723 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T11:11:48.723 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T11:11:48.723 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T11:11:48.723 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T11:11:48.723 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-21T11:11:48.723 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 bash[31332]: af33427a459adb8d1f6af38ac16b3b39ae04f6cb4b32fc99b556d7f1a6beb74a 2024-03-21T11:11:48.723 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 systemd[1]: Started Ceph mon.a for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:11:48.723 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T11:11:48.723 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: monmap e1: 1 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T11:11:48.723 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: fsmap 2024-03-21T11:11:48.723 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T11:11:48.723 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:48 smithi047 ceph-mon[31351]: mgrmap e1: no daemons active 2024-03-21T11:11:49.733 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.011+0000 7f72ef000700 1 Processor -- start 2024-03-21T11:11:49.733 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.011+0000 7f72ef000700 1 -- start start 2024-03-21T11:11:49.733 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.012+0000 7f72ef000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e8107ae0 0x7f72e8107f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:11:49.733 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.012+0000 7f72ef000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e81084d0 con 0x7f72e8107ae0 2024-03-21T11:11:49.733 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.012+0000 7f72e7e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e8107ae0 0x7f72e8107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:11:49.733 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.013+0000 7f72e7e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e8107ae0 0x7f72e8107f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60828/0 (socket says 172.21.15.47:60828) 2024-03-21T11:11:49.733 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.013+0000 7f72e7e00700 1 -- 172.21.15.47:0/1239807270 learned_addr learned my addr 172.21.15.47:0/1239807270 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:11:49.733 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.013+0000 7f72e7e00700 1 -- 172.21.15.47:0/1239807270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e8108ce0 con 0x7f72e8107ae0 2024-03-21T11:11:49.733 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.013+0000 7f72e7e00700 1 --2- 172.21.15.47:0/1239807270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e8107ae0 0x7f72e8107f00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f72d80098a0 tx=0x7f72d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad54ad4c6f2afc99 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:11:49.734 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.014+0000 7f72e7400700 1 -- 172.21.15.47:0/1239807270 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72d8004160 con 0x7f72e8107ae0 2024-03-21T11:11:49.734 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.014+0000 7f72e7400700 1 -- 172.21.15.47:0/1239807270 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72d8004300 con 0x7f72e8107ae0 2024-03-21T11:11:49.734 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.014+0000 7f72e7400700 1 -- 172.21.15.47:0/1239807270 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72d8011b50 con 0x7f72e8107ae0 2024-03-21T11:11:49.734 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.014+0000 7f72ef000700 1 -- 172.21.15.47:0/1239807270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e8107ae0 msgr2=0x7f72e8107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:11:49.734 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.014+0000 7f72ef000700 1 --2- 172.21.15.47:0/1239807270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e8107ae0 0x7f72e8107f00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f72d80098a0 tx=0x7f72d8009bb0 comp rx=0 tx=0).stop 2024-03-21T11:11:49.734 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.015+0000 7f72ef000700 1 -- 172.21.15.47:0/1239807270 shutdown_connections 2024-03-21T11:11:49.734 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.015+0000 7f72ef000700 1 --2- 172.21.15.47:0/1239807270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e8107ae0 0x7f72e8107f00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:11:49.734 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.015+0000 7f72ef000700 1 -- 172.21.15.47:0/1239807270 >> 172.21.15.47:0/1239807270 conn(0x7f72e8103080 msgr2=0x7f72e81054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:11:49.734 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.015+0000 7f72ef000700 1 -- 172.21.15.47:0/1239807270 shutdown_connections 2024-03-21T11:11:49.734 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.015+0000 7f72ef000700 1 -- 172.21.15.47:0/1239807270 wait complete. 2024-03-21T11:11:49.734 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.016+0000 7f72ef000700 1 Processor -- start 2024-03-21T11:11:49.734 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.016+0000 7f72ef000700 1 -- start start 2024-03-21T11:11:49.734 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.017+0000 7f72ef000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e8107ae0 0x7f72e819dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:11:49.735 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.017+0000 7f72ef000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e810c090 con 0x7f72e8107ae0 2024-03-21T11:11:49.735 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.017+0000 7f72e7e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e8107ae0 0x7f72e819dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:11:49.735 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.017+0000 7f72e7e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e8107ae0 0x7f72e819dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60836/0 (socket says 172.21.15.47:60836) 2024-03-21T11:11:49.735 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.017+0000 7f72e7e00700 1 -- 172.21.15.47:0/374655637 learned_addr learned my addr 172.21.15.47:0/374655637 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:11:49.735 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.017+0000 7f72e7e00700 1 -- 172.21.15.47:0/374655637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e819e270 con 0x7f72e8107ae0 2024-03-21T11:11:49.735 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.018+0000 7f72e7e00700 1 --2- 172.21.15.47:0/374655637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e8107ae0 0x7f72e819dd30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f72d8000c00 tx=0x7f72d8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:11:49.735 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.018+0000 7f72e5600700 1 -- 172.21.15.47:0/374655637 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72d8003b20 con 0x7f72e8107ae0 2024-03-21T11:11:49.735 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.018+0000 7f72e5600700 1 -- 172.21.15.47:0/374655637 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72d8003cc0 con 0x7f72e8107ae0 2024-03-21T11:11:49.735 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.018+0000 7f72ef000700 1 -- 172.21.15.47:0/374655637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e819e500 con 0x7f72e8107ae0 2024-03-21T11:11:49.735 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.018+0000 7f72e5600700 1 -- 172.21.15.47:0/374655637 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72d8024910 con 0x7f72e8107ae0 2024-03-21T11:11:49.735 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.018+0000 7f72ef000700 1 -- 172.21.15.47:0/374655637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e81a11f0 con 0x7f72e8107ae0 2024-03-21T11:11:49.735 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.019+0000 7f72e5600700 1 -- 172.21.15.47:0/374655637 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f72d8022020 con 0x7f72e8107ae0 2024-03-21T11:11:49.735 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.020+0000 7f72e5600700 1 -- 172.21.15.47:0/374655637 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f72d8011e00 con 0x7f72e8107ae0 2024-03-21T11:11:49.736 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.020+0000 7f72de000700 1 -- 172.21.15.47:0/374655637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e80637c0 con 0x7f72e8107ae0 2024-03-21T11:11:49.736 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.023+0000 7f72e5600700 1 -- 172.21.15.47:0/374655637 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f72d8042dd0 con 0x7f72e8107ae0 2024-03-21T11:11:49.736 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.099+0000 7f72de000700 1 -- 172.21.15.47:0/374655637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f72e8062da0 con 0x7f72e8107ae0 2024-03-21T11:11:49.736 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.102+0000 7f72e5600700 1 -- 172.21.15.47:0/374655637 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f72d8031020 con 0x7f72e8107ae0 2024-03-21T11:11:49.736 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.103+0000 7f72de000700 1 -- 172.21.15.47:0/374655637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e8107ae0 msgr2=0x7f72e819dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:11:49.736 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.103+0000 7f72de000700 1 --2- 172.21.15.47:0/374655637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e8107ae0 0x7f72e819dd30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f72d8000c00 tx=0x7f72d8011c10 comp rx=0 tx=0).stop 2024-03-21T11:11:49.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.104+0000 7f72de000700 1 -- 172.21.15.47:0/374655637 shutdown_connections 2024-03-21T11:11:49.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.104+0000 7f72de000700 1 --2- 172.21.15.47:0/374655637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e8107ae0 0x7f72e819dd30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:11:49.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.104+0000 7f72de000700 1 -- 172.21.15.47:0/374655637 >> 172.21.15.47:0/374655637 conn(0x7f72e8103080 msgr2=0x7f72e8103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:11:49.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.104+0000 7f72de000700 1 -- 172.21.15.47:0/374655637 shutdown_connections 2024-03-21T11:11:49.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:49.104+0000 7f72de000700 1 -- 172.21.15.47:0/374655637 wait complete. 2024-03-21T11:11:49.737 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-21T11:11:49.763 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-21T11:11:49.763 INFO:teuthology.orchestra.run.smithi047.stdout:Creating mgr... 2024-03-21T11:11:49.764 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-21T11:11:49.765 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-21T11:11:50.192 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a6f35682-e773-11ee-95cd-87774f69a715@mgr.x 2024-03-21T11:11:50.192 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a6f35682-e773-11ee-95cd-87774f69a715@mgr.x.service: Unit ceph-a6f35682-e773-11ee-95cd-87774f69a715@mgr.x.service not loaded. 2024-03-21T11:11:50.472 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a6f35682-e773-11ee-95cd-87774f69a715.target.wants/ceph-a6f35682-e773-11ee-95cd-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-a6f35682-e773-11ee-95cd-87774f69a715@.service. 2024-03-21T11:11:50.503 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:50 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/374655637' entity='client.admin' 2024-03-21T11:11:50.814 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:50 smithi047 systemd[1]: Starting Ceph mgr.x for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:11:51.160 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:50 smithi047 podman[31566]: 2024-03-21 11:11:50.770234401 +0000 UTC m=+0.035204477 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T11:11:51.160 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:50 smithi047 podman[31566]: 2024-03-21 11:11:50.867006019 +0000 UTC m=+0.131976131 container create 34a3d6ac0ad59a1753ed4986446d79db805c3c956b4bb36893995c7a48b04f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:11:51.160 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:51 smithi047 podman[31566]: 2024-03-21 11:11:51.078474943 +0000 UTC m=+0.343445062 container init 34a3d6ac0ad59a1753ed4986446d79db805c3c956b4bb36893995c7a48b04f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-21T11:11:51.160 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:51 smithi047 podman[31566]: 2024-03-21 11:11:51.088200606 +0000 UTC m=+0.353170701 container start 34a3d6ac0ad59a1753ed4986446d79db805c3c956b4bb36893995c7a48b04f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:11:51.198 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T11:11:51.198 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-03-21T11:11:51.218 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T11:11:51.218 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-03-21T11:11:51.218 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-03-21T11:11:51.218 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T11:11:51.240 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T11:11:51.240 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-03-21T11:11:51.258 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T11:11:51.258 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-03-21T11:11:51.258 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-03-21T11:11:51.258 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-21T11:11:51.258 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr to start... 2024-03-21T11:11:51.258 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr... 2024-03-21T11:11:51.411 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:51 smithi047 bash[31566]: 34a3d6ac0ad59a1753ed4986446d79db805c3c956b4bb36893995c7a48b04f0e 2024-03-21T11:11:51.412 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:51 smithi047 systemd[1]: Started Ceph mgr.x for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:11:51.975 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:51 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:51.901+0000 7ff38427f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T11:11:52.313 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:52 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:52.032+0000 7ff38427f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T11:11:52.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:52 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/4021450978' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T11:11:52.626 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T11:11:52.627 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T11:11:52.627 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:11:52.627 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T11:11:52.627 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T11:11:52.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T11:11:52.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T11:11:52.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:52.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T11:11:52.629 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T11:11:52.629 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-03-21T11:11:52.629 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T11:11:52.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T11:11:52.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T11:11:52.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T11:11:52.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-21T11:11:52.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T11:11:52.630 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:11:52.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T11:11:52.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T11:11:52.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:52.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T11:11:52.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:11:52.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T11:11:52.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T11:11:52.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T11:11:52.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T11:11:52.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T11:11:52.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T11:11:52.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:52.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T11:11:52.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T11:11:52.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T11:11:52.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T11:11:52.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T11:11:52.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T11:11:52.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T11:11:52.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T11:11:52.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T11:11:52.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:52.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T11:11:52.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:11:52.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T11:11:52.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T11:11:52.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:52.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T11:11:52.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T11:11:52.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T11:11:52.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T11:11:52.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T11:11:52.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T11:11:52.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T11:11:52.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T11:11:52.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T11:11:52.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:52.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T11:11:52.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:11:52.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T11:11:41.726635+0000", 2024-03-21T11:11:52.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T11:11:52.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:52.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T11:11:52.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T11:11:52.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.884+0000 7fe8f8400700 1 Processor -- start 2024-03-21T11:11:52.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.884+0000 7fe8f8400700 1 -- start start 2024-03-21T11:11:52.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.884+0000 7fe8f8400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8f01069f0 0x7fe8f0106e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:11:52.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.885+0000 7fe8f8400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f01073e0 con 0x7fe8f01069f0 2024-03-21T11:11:52.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.885+0000 7fe8f5a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8f01069f0 0x7fe8f0106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:11:52.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.885+0000 7fe8f5a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8f01069f0 0x7fe8f0106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60858/0 (socket says 172.21.15.47:60858) 2024-03-21T11:11:52.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.885+0000 7fe8f5a00700 1 -- 172.21.15.47:0/4253516637 learned_addr learned my addr 172.21.15.47:0/4253516637 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:11:52.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.886+0000 7fe8f5a00700 1 -- 172.21.15.47:0/4253516637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f0107bf0 con 0x7fe8f01069f0 2024-03-21T11:11:52.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.886+0000 7fe8f5a00700 1 --2- 172.21.15.47:0/4253516637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8f01069f0 0x7fe8f0106e10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40098a0 tx=0x7fe8e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b819ee887c9196fd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:11:52.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.886+0000 7fe8efe00700 1 -- 172.21.15.47:0/4253516637 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8e4004160 con 0x7fe8f01069f0 2024-03-21T11:11:52.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.886+0000 7fe8efe00700 1 -- 172.21.15.47:0/4253516637 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8e4004300 con 0x7fe8f01069f0 2024-03-21T11:11:52.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.886+0000 7fe8efe00700 1 -- 172.21.15.47:0/4253516637 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8e4011b50 con 0x7fe8f01069f0 2024-03-21T11:11:52.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.887+0000 7fe8f8400700 1 -- 172.21.15.47:0/4253516637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8f01069f0 msgr2=0x7fe8f0106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:11:52.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.887+0000 7fe8f8400700 1 --2- 172.21.15.47:0/4253516637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8f01069f0 0x7fe8f0106e10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40098a0 tx=0x7fe8e4009bb0 comp rx=0 tx=0).stop 2024-03-21T11:11:52.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.887+0000 7fe8f8400700 1 -- 172.21.15.47:0/4253516637 shutdown_connections 2024-03-21T11:11:52.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.887+0000 7fe8f8400700 1 --2- 172.21.15.47:0/4253516637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8f01069f0 0x7fe8f0106e10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:11:52.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.887+0000 7fe8f8400700 1 -- 172.21.15.47:0/4253516637 >> 172.21.15.47:0/4253516637 conn(0x7fe8f0101fb0 msgr2=0x7fe8f01043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:11:52.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.887+0000 7fe8f8400700 1 -- 172.21.15.47:0/4253516637 shutdown_connections 2024-03-21T11:11:52.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.887+0000 7fe8f8400700 1 -- 172.21.15.47:0/4253516637 wait complete. 2024-03-21T11:11:52.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.888+0000 7fe8f8400700 1 Processor -- start 2024-03-21T11:11:52.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.889+0000 7fe8f8400700 1 -- start start 2024-03-21T11:11:52.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.889+0000 7fe8f8400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8f01069f0 0x7fe8f019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:11:52.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.889+0000 7fe8f8400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f010afa0 con 0x7fe8f01069f0 2024-03-21T11:11:52.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.889+0000 7fe8f5a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8f01069f0 0x7fe8f019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:11:52.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.889+0000 7fe8f5a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8f01069f0 0x7fe8f019bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60866/0 (socket says 172.21.15.47:60866) 2024-03-21T11:11:52.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.889+0000 7fe8f5a00700 1 -- 172.21.15.47:0/4021450978 learned_addr learned my addr 172.21.15.47:0/4021450978 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:11:52.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.890+0000 7fe8f5a00700 1 -- 172.21.15.47:0/4021450978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f019c0f0 con 0x7fe8f01069f0 2024-03-21T11:11:52.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.890+0000 7fe8f5a00700 1 --2- 172.21.15.47:0/4021450978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8f01069f0 0x7fe8f019bbb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4011850 tx=0x7fe8e4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:11:52.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.890+0000 7fe8ee000700 1 -- 172.21.15.47:0/4021450978 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8e4003af0 con 0x7fe8f01069f0 2024-03-21T11:11:52.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.890+0000 7fe8ee000700 1 -- 172.21.15.47:0/4021450978 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8e4020070 con 0x7fe8f01069f0 2024-03-21T11:11:52.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.890+0000 7fe8f8400700 1 -- 172.21.15.47:0/4021450978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f019c380 con 0x7fe8f01069f0 2024-03-21T11:11:52.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.890+0000 7fe8ee000700 1 -- 172.21.15.47:0/4021450978 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8e40195a0 con 0x7fe8f01069f0 2024-03-21T11:11:52.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.890+0000 7fe8f8400700 1 -- 172.21.15.47:0/4021450978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f019f020 con 0x7fe8f01069f0 2024-03-21T11:11:52.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.891+0000 7fe8ee000700 1 -- 172.21.15.47:0/4021450978 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe8e4019c00 con 0x7fe8f01069f0 2024-03-21T11:11:52.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.891+0000 7fe8ee000700 1 -- 172.21.15.47:0/4021450978 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe8e4011e00 con 0x7fe8f01069f0 2024-03-21T11:11:52.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.892+0000 7fe8f8400700 1 -- 172.21.15.47:0/4021450978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f00637c0 con 0x7fe8f01069f0 2024-03-21T11:11:52.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.894+0000 7fe8ee000700 1 -- 172.21.15.47:0/4021450978 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe8e4003c90 con 0x7fe8f01069f0 2024-03-21T11:11:52.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.975+0000 7fe8f8400700 1 -- 172.21.15.47:0/4021450978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe8f002dc30 con 0x7fe8f01069f0 2024-03-21T11:11:52.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.976+0000 7fe8ee000700 1 -- 172.21.15.47:0/4021450978 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fe8e4025460 con 0x7fe8f01069f0 2024-03-21T11:11:52.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.977+0000 7fe8f8400700 1 -- 172.21.15.47:0/4021450978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8f01069f0 msgr2=0x7fe8f019bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:11:52.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.977+0000 7fe8f8400700 1 --2- 172.21.15.47:0/4021450978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8f01069f0 0x7fe8f019bbb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4011850 tx=0x7fe8e4011c10 comp rx=0 tx=0).stop 2024-03-21T11:11:52.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.977+0000 7fe8f8400700 1 -- 172.21.15.47:0/4021450978 shutdown_connections 2024-03-21T11:11:52.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.978+0000 7fe8f8400700 1 --2- 172.21.15.47:0/4021450978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8f01069f0 0x7fe8f019bbb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:11:52.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.978+0000 7fe8f8400700 1 -- 172.21.15.47:0/4021450978 >> 172.21.15.47:0/4021450978 conn(0x7fe8f0101fb0 msgr2=0x7fe8f0102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:11:52.646 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.978+0000 7fe8f8400700 1 -- 172.21.15.47:0/4021450978 shutdown_connections 2024-03-21T11:11:52.646 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:51.978+0000 7fe8f8400700 1 -- 172.21.15.47:0/4021450978 wait complete. 2024-03-21T11:11:52.646 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (1/15)... 2024-03-21T11:11:53.230 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:52 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:52.924+0000 7ff38427f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T11:11:53.231 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:53 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:53.050+0000 7ff38427f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T11:11:53.231 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:53 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:53.230+0000 7ff38427f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T11:11:53.563 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:53 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:53.358+0000 7ff38427f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T11:11:53.563 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:53 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:53.471+0000 7ff38427f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T11:11:54.063 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:53 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:53.811+0000 7ff38427f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T11:11:54.063 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:53 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:53.955+0000 7ff38427f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T11:11:54.418 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:54 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:54.069+0000 7ff38427f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T11:11:54.418 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:54 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:54.196+0000 7ff38427f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T11:11:54.418 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:54 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:54.417+0000 7ff38427f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T11:11:55.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:55 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2110704548' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T11:11:55.955 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:55 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:55.820+0000 7ff38427f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T11:11:55.965 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T11:11:55.965 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T11:11:55.965 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:11:55.965 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T11:11:55.965 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T11:11:55.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T11:11:55.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T11:11:55.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:55.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T11:11:55.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T11:11:55.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-03-21T11:11:55.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T11:11:55.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T11:11:55.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T11:11:55.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T11:11:55.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-03-21T11:11:55.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T11:11:55.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:11:55.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T11:11:55.967 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T11:11:55.967 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:55.967 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T11:11:55.967 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:11:55.967 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T11:11:55.967 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T11:11:55.975 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T11:11:55.975 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T11:11:55.976 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T11:11:55.976 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T11:11:55.976 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:55.976 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T11:11:55.976 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T11:11:55.976 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T11:11:55.976 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T11:11:55.976 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T11:11:55.976 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T11:11:55.976 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T11:11:55.976 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T11:11:55.976 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T11:11:55.976 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:55.977 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T11:11:55.977 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:11:55.977 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T11:11:55.977 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T11:11:55.977 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:55.977 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T11:11:55.977 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T11:11:55.977 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T11:11:55.977 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T11:11:55.977 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T11:11:55.977 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T11:11:55.977 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T11:11:55.977 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T11:11:55.977 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T11:11:55.978 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:55.978 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T11:11:55.978 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:11:55.978 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T11:11:41.726635+0000", 2024-03-21T11:11:55.978 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T11:11:55.978 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:55.978 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T11:11:55.978 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T11:11:55.978 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.279+0000 7fb60c800700 1 Processor -- start 2024-03-21T11:11:55.978 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.280+0000 7fb60c800700 1 -- start start 2024-03-21T11:11:55.978 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.280+0000 7fb60c800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb60407cdf0 0x7fb60407b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:11:55.978 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.280+0000 7fb60c800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60407b820 con 0x7fb60407cdf0 2024-03-21T11:11:55.979 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.280+0000 7fb609e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb60407cdf0 0x7fb60407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:11:55.979 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.281+0000 7fb609e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb60407cdf0 0x7fb60407b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60878/0 (socket says 172.21.15.47:60878) 2024-03-21T11:11:55.979 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.281+0000 7fb609e00700 1 -- 172.21.15.47:0/451109530 learned_addr learned my addr 172.21.15.47:0/451109530 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:11:55.979 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.281+0000 7fb609e00700 1 -- 172.21.15.47:0/451109530 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60407b9a0 con 0x7fb60407cdf0 2024-03-21T11:11:55.979 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.281+0000 7fb609e00700 1 --2- 172.21.15.47:0/451109530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb60407cdf0 0x7fb60407b250 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80098a0 tx=0x7fb5f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20556eb11091ae8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:11:55.979 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.282+0000 7fb608a00700 1 -- 172.21.15.47:0/451109530 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5f8004160 con 0x7fb60407cdf0 2024-03-21T11:11:55.979 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.282+0000 7fb608a00700 1 -- 172.21.15.47:0/451109530 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5f8004300 con 0x7fb60407cdf0 2024-03-21T11:11:55.979 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.282+0000 7fb608a00700 1 -- 172.21.15.47:0/451109530 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5f8011b50 con 0x7fb60407cdf0 2024-03-21T11:11:55.979 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.282+0000 7fb60c800700 1 -- 172.21.15.47:0/451109530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb60407cdf0 msgr2=0x7fb60407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:11:55.979 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.282+0000 7fb60c800700 1 --2- 172.21.15.47:0/451109530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb60407cdf0 0x7fb60407b250 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80098a0 tx=0x7fb5f8009bb0 comp rx=0 tx=0).stop 2024-03-21T11:11:55.979 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.283+0000 7fb60c800700 1 -- 172.21.15.47:0/451109530 shutdown_connections 2024-03-21T11:11:55.979 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.283+0000 7fb60c800700 1 --2- 172.21.15.47:0/451109530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb60407cdf0 0x7fb60407b250 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:11:55.979 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.283+0000 7fb60c800700 1 -- 172.21.15.47:0/451109530 >> 172.21.15.47:0/451109530 conn(0x7fb6041030c0 msgr2=0x7fb604105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:11:55.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.283+0000 7fb60c800700 1 -- 172.21.15.47:0/451109530 shutdown_connections 2024-03-21T11:11:55.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.283+0000 7fb60c800700 1 -- 172.21.15.47:0/451109530 wait complete. 2024-03-21T11:11:55.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.284+0000 7fb60c800700 1 Processor -- start 2024-03-21T11:11:55.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.284+0000 7fb60c800700 1 -- start start 2024-03-21T11:11:55.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.284+0000 7fb60c800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb60407cdf0 0x7fb60419dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:11:55.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.284+0000 7fb60c800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60410e160 con 0x7fb60407cdf0 2024-03-21T11:11:55.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.284+0000 7fb609e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb60407cdf0 0x7fb60419dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:11:55.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.284+0000 7fb609e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb60407cdf0 0x7fb60419dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60886/0 (socket says 172.21.15.47:60886) 2024-03-21T11:11:55.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.285+0000 7fb609e00700 1 -- 172.21.15.47:0/2110704548 learned_addr learned my addr 172.21.15.47:0/2110704548 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:11:55.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.285+0000 7fb609e00700 1 -- 172.21.15.47:0/2110704548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60419e210 con 0x7fb60407cdf0 2024-03-21T11:11:55.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.285+0000 7fb609e00700 1 --2- 172.21.15.47:0/2110704548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb60407cdf0 0x7fb60419dcd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8000c00 tx=0x7fb5f8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:11:55.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.285+0000 7fb5fea00700 1 -- 172.21.15.47:0/2110704548 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5f8003b20 con 0x7fb60407cdf0 2024-03-21T11:11:55.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.285+0000 7fb5fea00700 1 -- 172.21.15.47:0/2110704548 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5f8003cc0 con 0x7fb60407cdf0 2024-03-21T11:11:55.980 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.286+0000 7fb60c800700 1 -- 172.21.15.47:0/2110704548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60419e4a0 con 0x7fb60407cdf0 2024-03-21T11:11:55.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.286+0000 7fb5fea00700 1 -- 172.21.15.47:0/2110704548 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5f8024910 con 0x7fb60407cdf0 2024-03-21T11:11:55.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.286+0000 7fb60c800700 1 -- 172.21.15.47:0/2110704548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6041a1180 con 0x7fb60407cdf0 2024-03-21T11:11:55.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.286+0000 7fb5fea00700 1 -- 172.21.15.47:0/2110704548 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb5f8022020 con 0x7fb60407cdf0 2024-03-21T11:11:55.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.286+0000 7fb5fea00700 1 -- 172.21.15.47:0/2110704548 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb5f8011e00 con 0x7fb60407cdf0 2024-03-21T11:11:55.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.287+0000 7fb60c800700 1 -- 172.21.15.47:0/2110704548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6040637c0 con 0x7fb60407cdf0 2024-03-21T11:11:55.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.289+0000 7fb5fea00700 1 -- 172.21.15.47:0/2110704548 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb5f801acd0 con 0x7fb60407cdf0 2024-03-21T11:11:55.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.370+0000 7fb60c800700 1 -- 172.21.15.47:0/2110704548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb6041a1350 con 0x7fb60407cdf0 2024-03-21T11:11:55.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.371+0000 7fb5fea00700 1 -- 172.21.15.47:0/2110704548 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fb5f8032030 con 0x7fb60407cdf0 2024-03-21T11:11:55.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.372+0000 7fb60c800700 1 -- 172.21.15.47:0/2110704548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb60407cdf0 msgr2=0x7fb60419dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:11:55.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.372+0000 7fb60c800700 1 --2- 172.21.15.47:0/2110704548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb60407cdf0 0x7fb60419dcd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8000c00 tx=0x7fb5f8011c10 comp rx=0 tx=0).stop 2024-03-21T11:11:55.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.372+0000 7fb60c800700 1 -- 172.21.15.47:0/2110704548 shutdown_connections 2024-03-21T11:11:55.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.372+0000 7fb60c800700 1 --2- 172.21.15.47:0/2110704548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb60407cdf0 0x7fb60419dcd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:11:55.981 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.372+0000 7fb60c800700 1 -- 172.21.15.47:0/2110704548 >> 172.21.15.47:0/2110704548 conn(0x7fb6041030c0 msgr2=0x7fb604103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:11:55.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.372+0000 7fb60c800700 1 -- 172.21.15.47:0/2110704548 shutdown_connections 2024-03-21T11:11:55.982 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:55.372+0000 7fb60c800700 1 -- 172.21.15.47:0/2110704548 wait complete. 2024-03-21T11:11:55.982 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (2/15)... 2024-03-21T11:11:56.313 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:56 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:56.005+0000 7ff38427f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T11:11:56.808 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:56 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:56.430+0000 7ff38427f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T11:11:56.808 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:56 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:56.548+0000 7ff38427f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T11:11:57.061 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:56 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:56.808+0000 7ff38427f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T11:11:57.313 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:57 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:57.061+0000 7ff38427f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T11:11:57.763 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:57 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:57.442+0000 7ff38427f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T11:11:58.063 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:57 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:57.763+0000 7ff38427f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T11:11:58.367 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:58 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:58.078+0000 7ff38427f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T11:11:58.986 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:11:58 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/626073357' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T11:11:59.278 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:59 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:59.166+0000 7ff38427f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T11:11:59.296 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T11:11:59.296 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T11:11:59.296 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:11:59.296 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T11:11:59.296 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T11:11:59.296 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T11:11:59.296 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T11:11:59.296 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:59.297 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T11:11:59.297 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T11:11:59.297 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-03-21T11:11:59.297 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T11:11:59.297 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T11:11:59.297 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T11:11:59.297 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T11:11:59.297 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-21T11:11:59.297 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T11:11:59.297 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:11:59.297 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T11:11:59.297 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T11:11:59.298 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:59.298 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T11:11:59.298 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:11:59.298 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T11:11:59.298 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T11:11:59.298 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T11:11:59.298 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T11:11:59.298 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T11:11:59.298 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T11:11:59.298 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:59.298 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T11:11:59.298 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T11:11:59.298 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T11:11:59.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T11:11:59.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T11:11:59.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T11:11:59.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T11:11:59.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T11:11:59.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T11:11:59.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:59.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T11:11:59.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:11:59.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T11:11:59.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T11:11:59.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:59.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T11:11:59.299 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T11:11:59.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T11:11:59.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T11:11:59.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T11:11:59.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T11:11:59.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T11:11:59.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T11:11:59.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T11:11:59.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:59.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T11:11:59.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:11:59.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T11:11:41.726635+0000", 2024-03-21T11:11:59.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T11:11:59.300 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:11:59.301 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T11:11:59.301 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T11:11:59.301 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.522+0000 7f668c400700 1 Processor -- start 2024-03-21T11:11:59.301 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.523+0000 7f668c400700 1 -- start start 2024-03-21T11:11:59.301 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.523+0000 7f668c400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66841063b0 0x7f66841067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:11:59.301 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.523+0000 7f668c400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66840758a0 con 0x7f66841063b0 2024-03-21T11:11:59.301 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.523+0000 7f6689a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66841063b0 0x7f66841067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:11:59.301 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.523+0000 7f6689a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66841063b0 0x7f66841067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60894/0 (socket says 172.21.15.47:60894) 2024-03-21T11:11:59.301 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.524+0000 7f6689a00700 1 -- 172.21.15.47:0/1163794407 learned_addr learned my addr 172.21.15.47:0/1163794407 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:11:59.301 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.524+0000 7f6689a00700 1 -- 172.21.15.47:0/1163794407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6684106d10 con 0x7f66841063b0 2024-03-21T11:11:59.301 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.524+0000 7f6689a00700 1 --2- 172.21.15.47:0/1163794407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66841063b0 0x7f66841067d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f66780098a0 tx=0x7f6678009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8695662b5aa6bf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:11:59.301 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.525+0000 7f6683e00700 1 -- 172.21.15.47:0/1163794407 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6678004160 con 0x7f66841063b0 2024-03-21T11:11:59.301 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.525+0000 7f6683e00700 1 -- 172.21.15.47:0/1163794407 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6678004300 con 0x7f66841063b0 2024-03-21T11:11:59.302 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.525+0000 7f6683e00700 1 -- 172.21.15.47:0/1163794407 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6678011b50 con 0x7f66841063b0 2024-03-21T11:11:59.302 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.525+0000 7f668c400700 1 -- 172.21.15.47:0/1163794407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66841063b0 msgr2=0x7f66841067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:11:59.302 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.525+0000 7f668c400700 1 --2- 172.21.15.47:0/1163794407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66841063b0 0x7f66841067d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f66780098a0 tx=0x7f6678009bb0 comp rx=0 tx=0).stop 2024-03-21T11:11:59.302 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.525+0000 7f668c400700 1 -- 172.21.15.47:0/1163794407 shutdown_connections 2024-03-21T11:11:59.302 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.525+0000 7f668c400700 1 --2- 172.21.15.47:0/1163794407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66841063b0 0x7f66841067d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:11:59.302 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.525+0000 7f668c400700 1 -- 172.21.15.47:0/1163794407 >> 172.21.15.47:0/1163794407 conn(0x7f6684101fb0 msgr2=0x7f66841043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:11:59.302 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.526+0000 7f668c400700 1 -- 172.21.15.47:0/1163794407 shutdown_connections 2024-03-21T11:11:59.302 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.526+0000 7f668c400700 1 -- 172.21.15.47:0/1163794407 wait complete. 2024-03-21T11:11:59.302 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.527+0000 7f668c400700 1 Processor -- start 2024-03-21T11:11:59.302 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.527+0000 7f668c400700 1 -- start start 2024-03-21T11:11:59.302 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.527+0000 7f668c400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66841063b0 0x7f668419dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:11:59.302 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.527+0000 7f668c400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6684107da0 con 0x7f66841063b0 2024-03-21T11:11:59.303 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.527+0000 7f6689a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66841063b0 0x7f668419dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:11:59.303 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.527+0000 7f6689a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66841063b0 0x7f668419dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60898/0 (socket says 172.21.15.47:60898) 2024-03-21T11:11:59.303 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.527+0000 7f6689a00700 1 -- 172.21.15.47:0/626073357 learned_addr learned my addr 172.21.15.47:0/626073357 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:11:59.303 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.528+0000 7f6689a00700 1 -- 172.21.15.47:0/626073357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668419e210 con 0x7f66841063b0 2024-03-21T11:11:59.303 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.528+0000 7f6689a00700 1 --2- 172.21.15.47:0/626073357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66841063b0 0x7f668419dcd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6678009fb0 tx=0x7f6678011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:11:59.303 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.528+0000 7f6682000700 1 -- 172.21.15.47:0/626073357 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66780038b0 con 0x7f66841063b0 2024-03-21T11:11:59.303 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.528+0000 7f6682000700 1 -- 172.21.15.47:0/626073357 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6678011e50 con 0x7f66841063b0 2024-03-21T11:11:59.303 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.528+0000 7f6682000700 1 -- 172.21.15.47:0/626073357 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6678019420 con 0x7f66841063b0 2024-03-21T11:11:59.303 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.528+0000 7f668c400700 1 -- 172.21.15.47:0/626073357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668419e4a0 con 0x7f66841063b0 2024-03-21T11:11:59.303 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.528+0000 7f668c400700 1 -- 172.21.15.47:0/626073357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66841a1180 con 0x7f66841063b0 2024-03-21T11:11:59.303 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.529+0000 7f6682000700 1 -- 172.21.15.47:0/626073357 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6678019a60 con 0x7f66841063b0 2024-03-21T11:11:59.303 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.529+0000 7f6682000700 1 -- 172.21.15.47:0/626073357 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6678024730 con 0x7f66841063b0 2024-03-21T11:11:59.304 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.530+0000 7f668c400700 1 -- 172.21.15.47:0/626073357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66840637c0 con 0x7f66841063b0 2024-03-21T11:11:59.304 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.534+0000 7f6682000700 1 -- 172.21.15.47:0/626073357 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6678043dd0 con 0x7f66841063b0 2024-03-21T11:11:59.304 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.613+0000 7f668c400700 1 -- 172.21.15.47:0/626073357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f668402dc80 con 0x7f66841063b0 2024-03-21T11:11:59.304 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.614+0000 7f6682000700 1 -- 172.21.15.47:0/626073357 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f6678032030 con 0x7f66841063b0 2024-03-21T11:11:59.304 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.615+0000 7f668c400700 1 -- 172.21.15.47:0/626073357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66841063b0 msgr2=0x7f668419dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:11:59.304 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.615+0000 7f668c400700 1 --2- 172.21.15.47:0/626073357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66841063b0 0x7f668419dcd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6678009fb0 tx=0x7f6678011bf0 comp rx=0 tx=0).stop 2024-03-21T11:11:59.304 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.615+0000 7f668c400700 1 -- 172.21.15.47:0/626073357 shutdown_connections 2024-03-21T11:11:59.304 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.615+0000 7f668c400700 1 --2- 172.21.15.47:0/626073357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66841063b0 0x7f668419dcd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:11:59.304 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.615+0000 7f668c400700 1 -- 172.21.15.47:0/626073357 >> 172.21.15.47:0/626073357 conn(0x7f6684101fb0 msgr2=0x7f6684194960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:11:59.304 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.616+0000 7f668c400700 1 -- 172.21.15.47:0/626073357 shutdown_connections 2024-03-21T11:11:59.304 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:11:58.616+0000 7f668c400700 1 -- 172.21.15.47:0/626073357 wait complete. 2024-03-21T11:11:59.304 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (3/15)... 2024-03-21T11:12:00.300 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:11:59 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:11:59.881+0000 7ff38427f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T11:12:00.300 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:00 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:00.013+0000 7ff38427f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T11:12:00.563 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:00 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:00.300+0000 7ff38427f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T11:12:00.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:00 smithi047 ceph-mon[31351]: Activating manager daemon x 2024-03-21T11:12:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:00 smithi047 ceph-mon[31351]: mgrmap e2: x(active, starting, since 0.0111002s) 2024-03-21T11:12:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:00 smithi047 ceph-mon[31351]: from='mgr.14100 172.21.15.47:0/1700352992' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T11:12:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:00 smithi047 ceph-mon[31351]: from='mgr.14100 172.21.15.47:0/1700352992' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T11:12:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:00 smithi047 ceph-mon[31351]: from='mgr.14100 172.21.15.47:0/1700352992' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T11:12:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:00 smithi047 ceph-mon[31351]: from='mgr.14100 172.21.15.47:0/1700352992' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:12:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:00 smithi047 ceph-mon[31351]: from='mgr.14100 172.21.15.47:0/1700352992' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T11:12:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:00 smithi047 ceph-mon[31351]: Manager daemon x is now available 2024-03-21T11:12:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:00 smithi047 ceph-mon[31351]: from='mgr.14100 172.21.15.47:0/1700352992' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T11:12:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:00 smithi047 ceph-mon[31351]: from='mgr.14100 172.21.15.47:0/1700352992' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T11:12:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:00 smithi047 ceph-mon[31351]: from='mgr.14100 172.21.15.47:0/1700352992' entity='mgr.x' 2024-03-21T11:12:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:00 smithi047 ceph-mon[31351]: from='mgr.14100 172.21.15.47:0/1700352992' entity='mgr.x' 2024-03-21T11:12:01.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:01 smithi047 ceph-mon[31351]: from='mgr.14100 172.21.15.47:0/1700352992' entity='mgr.x' 2024-03-21T11:12:01.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:01 smithi047 ceph-mon[31351]: mgrmap e3: x(active, since 1.01807s) 2024-03-21T11:12:02.501 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:02 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2529998665' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T11:12:03.123 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T11:12:03.123 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T11:12:03.123 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:12:03.123 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T11:12:03.123 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T11:12:03.123 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T11:12:03.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T11:12:03.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:12:03.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T11:12:03.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T11:12:03.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-03-21T11:12:03.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T11:12:03.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T11:12:03.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T11:12:03.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T11:12:03.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-03-21T11:12:03.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T11:12:03.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:12:03.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T11:12:03.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T11:12:03.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:12:03.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T11:12:03.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:12:03.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T11:12:03.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T11:12:03.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T11:12:03.137 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T11:12:03.137 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T11:12:03.137 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T11:12:03.137 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:12:03.137 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T11:12:03.137 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T11:12:03.137 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T11:12:03.137 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T11:12:03.138 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T11:12:03.138 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T11:12:03.138 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T11:12:03.138 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T11:12:03.138 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T11:12:03.138 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:12:03.138 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T11:12:03.138 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:12:03.138 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T11:12:03.138 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T11:12:03.139 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:12:03.139 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T11:12:03.139 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T11:12:03.139 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T11:12:03.139 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T11:12:03.139 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T11:12:03.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T11:12:03.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T11:12:03.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T11:12:03.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T11:12:03.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:12:03.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T11:12:03.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T11:12:03.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T11:11:41.726635+0000", 2024-03-21T11:12:03.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T11:12:03.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T11:12:03.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T11:12:03.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T11:12:03.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.877+0000 7f4588000700 1 Processor -- start 2024-03-21T11:12:03.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.878+0000 7f4588000700 1 -- start start 2024-03-21T11:12:03.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.878+0000 7f4588000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45800822f0 0x7f4580110d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:03.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.878+0000 7f4588000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45801112d0 con 0x7f45800822f0 2024-03-21T11:12:03.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.879+0000 7f4585600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45800822f0 0x7f4580110d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:03.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.879+0000 7f4585600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45800822f0 0x7f4580110d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48536/0 (socket says 172.21.15.47:48536) 2024-03-21T11:12:03.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.879+0000 7f4585600700 1 -- 172.21.15.47:0/1033856642 learned_addr learned my addr 172.21.15.47:0/1033856642 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:03.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.879+0000 7f4585600700 1 -- 172.21.15.47:0/1033856642 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4580111450 con 0x7f45800822f0 2024-03-21T11:12:03.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.880+0000 7f4585600700 1 --2- 172.21.15.47:0/1033856642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45800822f0 0x7f4580110d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f45740098a0 tx=0x7f4574009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2057106de69431a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:03.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.880+0000 7f457fe00700 1 -- 172.21.15.47:0/1033856642 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4574004160 con 0x7f45800822f0 2024-03-21T11:12:03.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.880+0000 7f457fe00700 1 -- 172.21.15.47:0/1033856642 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4574004300 con 0x7f45800822f0 2024-03-21T11:12:03.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.880+0000 7f457fe00700 1 -- 172.21.15.47:0/1033856642 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4574011b50 con 0x7f45800822f0 2024-03-21T11:12:03.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.880+0000 7f4588000700 1 -- 172.21.15.47:0/1033856642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45800822f0 msgr2=0x7f4580110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:03.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.880+0000 7f4588000700 1 --2- 172.21.15.47:0/1033856642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45800822f0 0x7f4580110d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f45740098a0 tx=0x7f4574009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:03.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.881+0000 7f4588000700 1 -- 172.21.15.47:0/1033856642 shutdown_connections 2024-03-21T11:12:03.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.881+0000 7f4588000700 1 --2- 172.21.15.47:0/1033856642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45800822f0 0x7f4580110d00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:03.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.881+0000 7f4588000700 1 -- 172.21.15.47:0/1033856642 >> 172.21.15.47:0/1033856642 conn(0x7f458007d220 msgr2=0x7f458007f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:03.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.881+0000 7f4588000700 1 -- 172.21.15.47:0/1033856642 shutdown_connections 2024-03-21T11:12:03.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.881+0000 7f4588000700 1 -- 172.21.15.47:0/1033856642 wait complete. 2024-03-21T11:12:03.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.882+0000 7f4588000700 1 Processor -- start 2024-03-21T11:12:03.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.882+0000 7f4588000700 1 -- start start 2024-03-21T11:12:03.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.883+0000 7f4588000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45800822f0 0x7f45801a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:03.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.883+0000 7f4588000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45801124e0 con 0x7f45800822f0 2024-03-21T11:12:03.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.883+0000 7f4585600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45800822f0 0x7f45801a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:03.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.883+0000 7f4585600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45800822f0 0x7f45801a2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48548/0 (socket says 172.21.15.47:48548) 2024-03-21T11:12:03.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.883+0000 7f4585600700 1 -- 172.21.15.47:0/2529998665 learned_addr learned my addr 172.21.15.47:0/2529998665 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:03.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.883+0000 7f4585600700 1 -- 172.21.15.47:0/2529998665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45801a2650 con 0x7f45800822f0 2024-03-21T11:12:03.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.884+0000 7f4585600700 1 --2- 172.21.15.47:0/2529998665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45800822f0 0x7f45801a2110 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4574011850 tx=0x7f4574011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:03.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.884+0000 7f457e000700 1 -- 172.21.15.47:0/2529998665 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4574003af0 con 0x7f45800822f0 2024-03-21T11:12:03.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.884+0000 7f457e000700 1 -- 172.21.15.47:0/2529998665 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4574020070 con 0x7f45800822f0 2024-03-21T11:12:03.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.884+0000 7f4588000700 1 -- 172.21.15.47:0/2529998665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45801a28e0 con 0x7f45800822f0 2024-03-21T11:12:03.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.884+0000 7f457e000700 1 -- 172.21.15.47:0/2529998665 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45740195a0 con 0x7f45800822f0 2024-03-21T11:12:03.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.884+0000 7f4588000700 1 -- 172.21.15.47:0/2529998665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45801a55d0 con 0x7f45800822f0 2024-03-21T11:12:03.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.885+0000 7f4588000700 1 -- 172.21.15.47:0/2529998665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45800637c0 con 0x7f45800822f0 2024-03-21T11:12:03.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.885+0000 7f457e000700 1 -- 172.21.15.47:0/2529998665 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 3) v1 ==== 46177+0+0 (secure 0 0 0) 0x7f4574019850 con 0x7f45800822f0 2024-03-21T11:12:03.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.886+0000 7f457e000700 1 --2- 172.21.15.47:0/2529998665 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f4568039060 0x7f456803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:03.145 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.886+0000 7f457e000700 1 -- 172.21.15.47:0/2529998665 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4574011e00 con 0x7f45800822f0 2024-03-21T11:12:03.145 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.886+0000 7f4584c00700 1 --2- 172.21.15.47:0/2529998665 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f4568039060 0x7f456803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:03.145 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.887+0000 7f4584c00700 1 --2- 172.21.15.47:0/2529998665 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f4568039060 0x7f456803b520 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f456c006fd0 tx=0x7f456c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:03.145 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:01.893+0000 7f457e000700 1 -- 172.21.15.47:0/2529998665 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4574003c90 con 0x7f45800822f0 2024-03-21T11:12:03.145 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:02.203+0000 7f4588000700 1 -- 172.21.15.47:0/2529998665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f45801a57a0 con 0x7f45800822f0 2024-03-21T11:12:03.145 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:02.204+0000 7f457e000700 1 -- 172.21.15.47:0/2529998665 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f45801a57a0 con 0x7f45800822f0 2024-03-21T11:12:03.145 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:02.206+0000 7f4588000700 1 -- 172.21.15.47:0/2529998665 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f4568039060 msgr2=0x7f456803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:03.145 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:02.206+0000 7f4588000700 1 --2- 172.21.15.47:0/2529998665 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f4568039060 0x7f456803b520 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f456c006fd0 tx=0x7f456c006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:03.145 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:02.206+0000 7f4588000700 1 -- 172.21.15.47:0/2529998665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45800822f0 msgr2=0x7f45801a2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:03.145 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:02.206+0000 7f4588000700 1 --2- 172.21.15.47:0/2529998665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45800822f0 0x7f45801a2110 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4574011850 tx=0x7f4574011c10 comp rx=0 tx=0).stop 2024-03-21T11:12:03.145 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:02.207+0000 7f4588000700 1 -- 172.21.15.47:0/2529998665 shutdown_connections 2024-03-21T11:12:03.146 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:02.207+0000 7f4588000700 1 --2- 172.21.15.47:0/2529998665 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f4568039060 0x7f456803b520 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:03.146 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:02.207+0000 7f4588000700 1 --2- 172.21.15.47:0/2529998665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45800822f0 0x7f45801a2110 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:03.146 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:02.207+0000 7f4588000700 1 -- 172.21.15.47:0/2529998665 >> 172.21.15.47:0/2529998665 conn(0x7f458007d220 msgr2=0x7f458007de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:03.146 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:02.207+0000 7f4588000700 1 -- 172.21.15.47:0/2529998665 shutdown_connections 2024-03-21T11:12:03.146 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:02.207+0000 7f4588000700 1 -- 172.21.15.47:0/2529998665 wait complete. 2024-03-21T11:12:03.146 INFO:teuthology.orchestra.run.smithi047.stdout:mgr is available 2024-03-21T11:12:03.635 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:03 smithi047 ceph-mon[31351]: mgrmap e4: x(active, since 2s) 2024-03-21T11:12:04.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:04 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1253630731' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T11:12:04.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T11:12:04.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T11:12:04.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout fsid = a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:12:04.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T11:12:04.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T11:12:04.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T11:12:04.631 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T11:12:04.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T11:12:04.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T11:12:04.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T11:12:04.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T11:12:04.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T11:12:04.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-21T11:12:04.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-21T11:12:04.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-21T11:12:04.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T11:12:04.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T11:12:04.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.727+0000 7f9655a00700 1 Processor -- start 2024-03-21T11:12:04.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.728+0000 7f9655a00700 1 -- start start 2024-03-21T11:12:04.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.728+0000 7f9655a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650109db0 0x7f965010a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:04.632 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.728+0000 7f9655a00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965010a7a0 con 0x7f9650109db0 2024-03-21T11:12:04.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.728+0000 7f964ea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650109db0 0x7f965010a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:04.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.729+0000 7f964ea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650109db0 0x7f965010a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48552/0 (socket says 172.21.15.47:48552) 2024-03-21T11:12:04.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.729+0000 7f964ea00700 1 -- 172.21.15.47:0/750739225 learned_addr learned my addr 172.21.15.47:0/750739225 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:04.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.729+0000 7f964ea00700 1 -- 172.21.15.47:0/750739225 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965010af10 con 0x7f9650109db0 2024-03-21T11:12:04.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.729+0000 7f964ea00700 1 --2- 172.21.15.47:0/750739225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650109db0 0x7f965010a1d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f96400098a0 tx=0x7f9640009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea271779b05a62d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:04.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.730+0000 7f964e000700 1 -- 172.21.15.47:0/750739225 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9640004160 con 0x7f9650109db0 2024-03-21T11:12:04.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.730+0000 7f964e000700 1 -- 172.21.15.47:0/750739225 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9640004300 con 0x7f9650109db0 2024-03-21T11:12:04.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.730+0000 7f964e000700 1 -- 172.21.15.47:0/750739225 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9640011b50 con 0x7f9650109db0 2024-03-21T11:12:04.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.730+0000 7f9655a00700 1 -- 172.21.15.47:0/750739225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650109db0 msgr2=0x7f965010a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:04.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.730+0000 7f9655a00700 1 --2- 172.21.15.47:0/750739225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650109db0 0x7f965010a1d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f96400098a0 tx=0x7f9640009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:04.633 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.730+0000 7f9655a00700 1 -- 172.21.15.47:0/750739225 shutdown_connections 2024-03-21T11:12:04.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.730+0000 7f9655a00700 1 --2- 172.21.15.47:0/750739225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650109db0 0x7f965010a1d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:04.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.730+0000 7f9655a00700 1 -- 172.21.15.47:0/750739225 >> 172.21.15.47:0/750739225 conn(0x7f965007d130 msgr2=0x7f965007d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:04.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.731+0000 7f9655a00700 1 -- 172.21.15.47:0/750739225 shutdown_connections 2024-03-21T11:12:04.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.731+0000 7f9655a00700 1 -- 172.21.15.47:0/750739225 wait complete. 2024-03-21T11:12:04.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.732+0000 7f9655a00700 1 Processor -- start 2024-03-21T11:12:04.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.732+0000 7f9655a00700 1 -- start start 2024-03-21T11:12:04.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.732+0000 7f9655a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650109db0 0x7f965019dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:04.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.732+0000 7f9655a00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965010e2c0 con 0x7f9650109db0 2024-03-21T11:12:04.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.732+0000 7f964ea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650109db0 0x7f965019dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:04.634 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.732+0000 7f964ea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650109db0 0x7f965019dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48554/0 (socket says 172.21.15.47:48554) 2024-03-21T11:12:04.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.732+0000 7f964ea00700 1 -- 172.21.15.47:0/1253630731 learned_addr learned my addr 172.21.15.47:0/1253630731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:04.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.733+0000 7f964ea00700 1 -- 172.21.15.47:0/1253630731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965019e2a0 con 0x7f9650109db0 2024-03-21T11:12:04.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.733+0000 7f964ea00700 1 --2- 172.21.15.47:0/1253630731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650109db0 0x7f965019dd60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9640011850 tx=0x7f9640011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:04.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.733+0000 7f9647e00700 1 -- 172.21.15.47:0/1253630731 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9640003af0 con 0x7f9650109db0 2024-03-21T11:12:04.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.733+0000 7f9647e00700 1 -- 172.21.15.47:0/1253630731 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9640020070 con 0x7f9650109db0 2024-03-21T11:12:04.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.733+0000 7f9655a00700 1 -- 172.21.15.47:0/1253630731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965019e530 con 0x7f9650109db0 2024-03-21T11:12:04.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.734+0000 7f9647e00700 1 -- 172.21.15.47:0/1253630731 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96400195a0 con 0x7f9650109db0 2024-03-21T11:12:04.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.734+0000 7f9655a00700 1 -- 172.21.15.47:0/1253630731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96501a1220 con 0x7f9650109db0 2024-03-21T11:12:04.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.735+0000 7f9647e00700 1 -- 172.21.15.47:0/1253630731 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 46283+0+0 (secure 0 0 0) 0x7f96400198a0 con 0x7f9650109db0 2024-03-21T11:12:04.635 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.735+0000 7f9655a00700 1 -- 172.21.15.47:0/1253630731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96500637c0 con 0x7f9650109db0 2024-03-21T11:12:04.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.735+0000 7f9647e00700 1 --2- 172.21.15.47:0/1253630731 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f9634039130 0x7f963403b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:04.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.735+0000 7f9647e00700 1 -- 172.21.15.47:0/1253630731 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9640011e00 con 0x7f9650109db0 2024-03-21T11:12:04.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.736+0000 7f9646000700 1 --2- 172.21.15.47:0/1253630731 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f9634039130 0x7f963403b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:04.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.736+0000 7f9646000700 1 --2- 172.21.15.47:0/1253630731 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f9634039130 0x7f963403b5f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9638006fd0 tx=0x7f9638006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:04.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.741+0000 7f9647e00700 1 -- 172.21.15.47:0/1253630731 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9640003c90 con 0x7f9650109db0 2024-03-21T11:12:04.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.964+0000 7f9655a00700 1 -- 172.21.15.47:0/1253630731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f96501a13f0 con 0x7f9650109db0 2024-03-21T11:12:04.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.966+0000 7f9647e00700 1 -- 172.21.15.47:0/1253630731 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f96501a13f0 con 0x7f9650109db0 2024-03-21T11:12:04.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.968+0000 7f9655a00700 1 -- 172.21.15.47:0/1253630731 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f9634039130 msgr2=0x7f963403b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:04.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.968+0000 7f9655a00700 1 --2- 172.21.15.47:0/1253630731 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f9634039130 0x7f963403b5f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9638006fd0 tx=0x7f9638006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:04.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.969+0000 7f9655a00700 1 -- 172.21.15.47:0/1253630731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650109db0 msgr2=0x7f965019dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:04.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.969+0000 7f9655a00700 1 --2- 172.21.15.47:0/1253630731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650109db0 0x7f965019dd60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9640011850 tx=0x7f9640011c10 comp rx=0 tx=0).stop 2024-03-21T11:12:04.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.969+0000 7f9655a00700 1 -- 172.21.15.47:0/1253630731 shutdown_connections 2024-03-21T11:12:04.636 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.969+0000 7f9655a00700 1 --2- 172.21.15.47:0/1253630731 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f9634039130 0x7f963403b5f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:04.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.969+0000 7f9655a00700 1 --2- 172.21.15.47:0/1253630731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9650109db0 0x7f965019dd60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:04.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.969+0000 7f9655a00700 1 -- 172.21.15.47:0/1253630731 >> 172.21.15.47:0/1253630731 conn(0x7f965007d130 msgr2=0x7f9650107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:04.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.969+0000 7f9655a00700 1 -- 172.21.15.47:0/1253630731 shutdown_connections 2024-03-21T11:12:04.637 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:03.969+0000 7f9655a00700 1 -- 172.21.15.47:0/1253630731 wait complete. 2024-03-21T11:12:04.637 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling cephadm module... 2024-03-21T11:12:05.813 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:05 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: ignoring --setuser ceph since I am not root 2024-03-21T11:12:05.813 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:05 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: ignoring --setgroup ceph since I am not root 2024-03-21T11:12:05.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:05 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2247776692' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-21T11:12:06.257 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.212+0000 7fa69d600700 1 Processor -- start 2024-03-21T11:12:06.258 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.212+0000 7fa69d600700 1 -- start start 2024-03-21T11:12:06.258 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.213+0000 7fa69d600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981063f0 0x7fa698106810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:06.258 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.213+0000 7fa69d600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69807be70 con 0x7fa6981063f0 2024-03-21T11:12:06.258 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.214+0000 7fa696a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981063f0 0x7fa698106810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:06.258 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.214+0000 7fa696a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981063f0 0x7fa698106810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48564/0 (socket says 172.21.15.47:48564) 2024-03-21T11:12:06.258 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.214+0000 7fa696a00700 1 -- 172.21.15.47:0/4277483967 learned_addr learned my addr 172.21.15.47:0/4277483967 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:06.258 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.214+0000 7fa696a00700 1 -- 172.21.15.47:0/4277483967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa698106d50 con 0x7fa6981063f0 2024-03-21T11:12:06.258 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.215+0000 7fa696a00700 1 --2- 172.21.15.47:0/4277483967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981063f0 0x7fa698106810 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa6880098a0 tx=0x7fa688009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3619df07e09c4b1b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:06.258 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.215+0000 7fa696000700 1 -- 172.21.15.47:0/4277483967 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa688004160 con 0x7fa6981063f0 2024-03-21T11:12:06.258 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.215+0000 7fa696000700 1 -- 172.21.15.47:0/4277483967 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa688004300 con 0x7fa6981063f0 2024-03-21T11:12:06.258 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.215+0000 7fa696000700 1 -- 172.21.15.47:0/4277483967 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa688011b50 con 0x7fa6981063f0 2024-03-21T11:12:06.259 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.216+0000 7fa69d600700 1 -- 172.21.15.47:0/4277483967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981063f0 msgr2=0x7fa698106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:06.259 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.216+0000 7fa69d600700 1 --2- 172.21.15.47:0/4277483967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981063f0 0x7fa698106810 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa6880098a0 tx=0x7fa688009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:06.259 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.216+0000 7fa69d600700 1 -- 172.21.15.47:0/4277483967 shutdown_connections 2024-03-21T11:12:06.259 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.216+0000 7fa69d600700 1 --2- 172.21.15.47:0/4277483967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981063f0 0x7fa698106810 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:06.259 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.216+0000 7fa69d600700 1 -- 172.21.15.47:0/4277483967 >> 172.21.15.47:0/4277483967 conn(0x7fa698101fd0 msgr2=0x7fa698104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:06.259 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.216+0000 7fa69d600700 1 -- 172.21.15.47:0/4277483967 shutdown_connections 2024-03-21T11:12:06.259 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.216+0000 7fa69d600700 1 -- 172.21.15.47:0/4277483967 wait complete. 2024-03-21T11:12:06.259 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.217+0000 7fa69d600700 1 Processor -- start 2024-03-21T11:12:06.259 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.218+0000 7fa69d600700 1 -- start start 2024-03-21T11:12:06.259 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.218+0000 7fa69d600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981063f0 0x7fa69819dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:06.259 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.218+0000 7fa69d600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa698107de0 con 0x7fa6981063f0 2024-03-21T11:12:06.259 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.218+0000 7fa696a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981063f0 0x7fa69819dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:06.260 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.219+0000 7fa696a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981063f0 0x7fa69819dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48568/0 (socket says 172.21.15.47:48568) 2024-03-21T11:12:06.260 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.219+0000 7fa696a00700 1 -- 172.21.15.47:0/2247776692 learned_addr learned my addr 172.21.15.47:0/2247776692 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:06.260 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.219+0000 7fa696a00700 1 -- 172.21.15.47:0/2247776692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69819e1e0 con 0x7fa6981063f0 2024-03-21T11:12:06.260 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.219+0000 7fa696a00700 1 --2- 172.21.15.47:0/2247776692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981063f0 0x7fa69819dca0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa688011b60 tx=0x7fa688011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:06.260 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.219+0000 7fa68f400700 1 -- 172.21.15.47:0/2247776692 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa688003b20 con 0x7fa6981063f0 2024-03-21T11:12:06.260 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.220+0000 7fa68f400700 1 -- 172.21.15.47:0/2247776692 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa688003cc0 con 0x7fa6981063f0 2024-03-21T11:12:06.260 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.220+0000 7fa69d600700 1 -- 172.21.15.47:0/2247776692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69819e470 con 0x7fa6981063f0 2024-03-21T11:12:06.260 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.220+0000 7fa68f400700 1 -- 172.21.15.47:0/2247776692 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa688024840 con 0x7fa6981063f0 2024-03-21T11:12:06.260 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.220+0000 7fa69d600700 1 -- 172.21.15.47:0/2247776692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6981a1160 con 0x7fa6981063f0 2024-03-21T11:12:06.260 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.221+0000 7fa69d600700 1 -- 172.21.15.47:0/2247776692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6980637c0 con 0x7fa6981063f0 2024-03-21T11:12:06.260 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.221+0000 7fa68f400700 1 -- 172.21.15.47:0/2247776692 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 46283+0+0 (secure 0 0 0) 0x7fa688022020 con 0x7fa6981063f0 2024-03-21T11:12:06.260 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.221+0000 7fa68f400700 1 --2- 172.21.15.47:0/2247776692 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7fa67c039180 0x7fa67c03b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:06.260 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.222+0000 7fa68f400700 1 -- 172.21.15.47:0/2247776692 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa688011e00 con 0x7fa6981063f0 2024-03-21T11:12:06.261 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.222+0000 7fa68fe00700 1 --2- 172.21.15.47:0/2247776692 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7fa67c039180 0x7fa67c03b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:06.261 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.223+0000 7fa68fe00700 1 --2- 172.21.15.47:0/2247776692 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7fa67c039180 0x7fa67c03b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa680006fd0 tx=0x7fa680006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:06.261 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.227+0000 7fa68f400700 1 -- 172.21.15.47:0/2247776692 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa68800b3e0 con 0x7fa6981063f0 2024-03-21T11:12:06.261 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.517+0000 7fa69d600700 1 -- 172.21.15.47:0/2247776692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fa6980008d0 con 0x7fa6981063f0 2024-03-21T11:12:06.261 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.571+0000 7fa68f400700 1 -- 172.21.15.47:0/2247776692 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fa68804cc30 con 0x7fa6981063f0 2024-03-21T11:12:06.261 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.575+0000 7fa68f400700 1 -- 172.21.15.47:0/2247776692 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fa68802c350 con 0x7fa6981063f0 2024-03-21T11:12:06.261 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.577+0000 7fa69d600700 1 -- 172.21.15.47:0/2247776692 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7fa67c039180 msgr2=0x7fa67c03b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:06.261 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.577+0000 7fa69d600700 1 --2- 172.21.15.47:0/2247776692 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7fa67c039180 0x7fa67c03b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa680006fd0 tx=0x7fa680006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:06.261 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.577+0000 7fa69d600700 1 -- 172.21.15.47:0/2247776692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981063f0 msgr2=0x7fa69819dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:06.261 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.577+0000 7fa69d600700 1 --2- 172.21.15.47:0/2247776692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981063f0 0x7fa69819dca0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa688011b60 tx=0x7fa688011c40 comp rx=0 tx=0).stop 2024-03-21T11:12:06.261 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.577+0000 7fa69d600700 1 -- 172.21.15.47:0/2247776692 shutdown_connections 2024-03-21T11:12:06.261 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.577+0000 7fa69d600700 1 --2- 172.21.15.47:0/2247776692 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7fa67c039180 0x7fa67c03b640 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:06.262 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.577+0000 7fa69d600700 1 --2- 172.21.15.47:0/2247776692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6981063f0 0x7fa69819dca0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:06.262 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.577+0000 7fa69d600700 1 -- 172.21.15.47:0/2247776692 >> 172.21.15.47:0/2247776692 conn(0x7fa698101fd0 msgr2=0x7fa698194900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:06.262 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.577+0000 7fa69d600700 1 -- 172.21.15.47:0/2247776692 shutdown_connections 2024-03-21T11:12:06.262 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:05.577+0000 7fa69d600700 1 -- 172.21.15.47:0/2247776692 wait complete. 2024-03-21T11:12:06.516 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:06 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:06.386+0000 7f94c588f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T11:12:06.516 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:06 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:06.516+0000 7f94c588f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T11:12:06.773 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:06 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2247776692' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-21T11:12:06.773 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:06 smithi047 ceph-mon[31351]: mgrmap e5: x(active, since 5s) 2024-03-21T11:12:07.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:07 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/859371191' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T11:12:07.686 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:07 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:07.373+0000 7f94c588f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T11:12:07.686 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:07 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:07.504+0000 7f94c588f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T11:12:07.686 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:07 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:07.686+0000 7f94c588f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T11:12:07.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T11:12:07.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-21T11:12:07.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T11:12:07.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-21T11:12:07.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T11:12:07.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T11:12:07.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.840+0000 7f6832000700 1 Processor -- start 2024-03-21T11:12:07.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.840+0000 7f6832000700 1 -- start start 2024-03-21T11:12:07.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.840+0000 7f6832000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f682c109db0 0x7f682c10a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:07.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.840+0000 7f6832000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c10a7a0 con 0x7f682c109db0 2024-03-21T11:12:07.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.841+0000 7f682b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f682c109db0 0x7f682c10a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:07.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.841+0000 7f682b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f682c109db0 0x7f682c10a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48588/0 (socket says 172.21.15.47:48588) 2024-03-21T11:12:07.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.841+0000 7f682b400700 1 -- 172.21.15.47:0/884776566 learned_addr learned my addr 172.21.15.47:0/884776566 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:07.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.841+0000 7f682b400700 1 -- 172.21.15.47:0/884776566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c10af10 con 0x7f682c109db0 2024-03-21T11:12:07.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.841+0000 7f682b400700 1 --2- 172.21.15.47:0/884776566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f682c109db0 0x7f682c10a1d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f681c0098a0 tx=0x7f681c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb24c01e87053d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:07.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.842+0000 7f682a000700 1 -- 172.21.15.47:0/884776566 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f681c004160 con 0x7f682c109db0 2024-03-21T11:12:07.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.842+0000 7f682a000700 1 -- 172.21.15.47:0/884776566 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f681c004300 con 0x7f682c109db0 2024-03-21T11:12:07.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.842+0000 7f682a000700 1 -- 172.21.15.47:0/884776566 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f681c011b50 con 0x7f682c109db0 2024-03-21T11:12:07.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.843+0000 7f6832000700 1 -- 172.21.15.47:0/884776566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f682c109db0 msgr2=0x7f682c10a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:07.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.843+0000 7f6832000700 1 --2- 172.21.15.47:0/884776566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f682c109db0 0x7f682c10a1d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f681c0098a0 tx=0x7f681c009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:07.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.843+0000 7f6832000700 1 -- 172.21.15.47:0/884776566 shutdown_connections 2024-03-21T11:12:07.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.843+0000 7f6832000700 1 --2- 172.21.15.47:0/884776566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f682c109db0 0x7f682c10a1d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:07.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.843+0000 7f6832000700 1 -- 172.21.15.47:0/884776566 >> 172.21.15.47:0/884776566 conn(0x7f682c07d220 msgr2=0x7f682c07d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:07.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.843+0000 7f6832000700 1 -- 172.21.15.47:0/884776566 shutdown_connections 2024-03-21T11:12:07.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.843+0000 7f6832000700 1 -- 172.21.15.47:0/884776566 wait complete. 2024-03-21T11:12:07.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.844+0000 7f6832000700 1 Processor -- start 2024-03-21T11:12:07.951 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.845+0000 7f6832000700 1 -- start start 2024-03-21T11:12:07.951 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.845+0000 7f6832000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f682c109db0 0x7f682c19dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:07.951 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.845+0000 7f6832000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c10e2c0 con 0x7f682c109db0 2024-03-21T11:12:07.951 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.845+0000 7f682b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f682c109db0 0x7f682c19dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:07.951 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.845+0000 7f682b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f682c109db0 0x7f682c19dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48594/0 (socket says 172.21.15.47:48594) 2024-03-21T11:12:07.951 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.846+0000 7f682b400700 1 -- 172.21.15.47:0/859371191 learned_addr learned my addr 172.21.15.47:0/859371191 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:07.951 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.846+0000 7f682b400700 1 -- 172.21.15.47:0/859371191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c19e270 con 0x7f682c109db0 2024-03-21T11:12:07.951 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.846+0000 7f682b400700 1 --2- 172.21.15.47:0/859371191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f682c109db0 0x7f682c19dd30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f681c009fb0 tx=0x7f681c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:07.951 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.847+0000 7f6823e00700 1 -- 172.21.15.47:0/859371191 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f681c0038b0 con 0x7f682c109db0 2024-03-21T11:12:07.951 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.847+0000 7f6823e00700 1 -- 172.21.15.47:0/859371191 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f681c011e50 con 0x7f682c109db0 2024-03-21T11:12:07.951 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.847+0000 7f6832000700 1 -- 172.21.15.47:0/859371191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f682c19e500 con 0x7f682c109db0 2024-03-21T11:12:07.952 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.847+0000 7f6823e00700 1 -- 172.21.15.47:0/859371191 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f681c019550 con 0x7f682c109db0 2024-03-21T11:12:07.952 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.847+0000 7f6832000700 1 -- 172.21.15.47:0/859371191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f682c1a11f0 con 0x7f682c109db0 2024-03-21T11:12:07.952 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.848+0000 7f6823e00700 1 -- 172.21.15.47:0/859371191 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f681c003a50 con 0x7f682c109db0 2024-03-21T11:12:07.952 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.848+0000 7f6823e00700 1 --2- 172.21.15.47:0/859371191 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f68100391e0 0x7f681003b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:07.952 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.848+0000 7f6823e00700 1 -- 172.21.15.47:0/859371191 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f681c04d3c0 con 0x7f682c109db0 2024-03-21T11:12:07.952 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.848+0000 7f682aa00700 1 -- 172.21.15.47:0/859371191 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f68100391e0 msgr2=0x7f681003b6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1584030551 2024-03-21T11:12:07.952 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.848+0000 7f6832000700 1 -- 172.21.15.47:0/859371191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f682c0637c0 con 0x7f682c109db0 2024-03-21T11:12:07.952 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.849+0000 7f682aa00700 1 --2- 172.21.15.47:0/859371191 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f68100391e0 0x7f681003b6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T11:12:07.952 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:06.858+0000 7f6823e00700 1 -- 172.21.15.47:0/859371191 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f681c04cde0 con 0x7f682c109db0 2024-03-21T11:12:07.952 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:07.050+0000 7f682aa00700 1 -- 172.21.15.47:0/859371191 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f68100391e0 msgr2=0x7f681003b6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1584030551 2024-03-21T11:12:07.952 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:07.050+0000 7f682aa00700 1 --2- 172.21.15.47:0/859371191 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f68100391e0 0x7f681003b6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T11:12:07.953 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:07.153+0000 7f6832000700 1 -- 172.21.15.47:0/859371191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f682c1a1720 con 0x7f682c109db0 2024-03-21T11:12:07.953 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:07.154+0000 7f6823e00700 1 -- 172.21.15.47:0/859371191 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f681c029d50 con 0x7f682c109db0 2024-03-21T11:12:07.953 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:07.156+0000 7f6832000700 1 -- 172.21.15.47:0/859371191 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f68100391e0 msgr2=0x7f681003b6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:12:07.953 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:07.156+0000 7f6832000700 1 --2- 172.21.15.47:0/859371191 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f68100391e0 0x7f681003b6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:07.953 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:07.156+0000 7f6832000700 1 -- 172.21.15.47:0/859371191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f682c109db0 msgr2=0x7f682c19dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:07.953 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:07.156+0000 7f6832000700 1 --2- 172.21.15.47:0/859371191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f682c109db0 0x7f682c19dd30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f681c009fb0 tx=0x7f681c011bf0 comp rx=0 tx=0).stop 2024-03-21T11:12:07.953 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:07.157+0000 7f6832000700 1 -- 172.21.15.47:0/859371191 shutdown_connections 2024-03-21T11:12:07.953 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:07.157+0000 7f6832000700 1 --2- 172.21.15.47:0/859371191 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7f68100391e0 0x7f681003b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:07.953 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:07.157+0000 7f6832000700 1 --2- 172.21.15.47:0/859371191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f682c109db0 0x7f682c19dd30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:07.953 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:07.157+0000 7f6832000700 1 -- 172.21.15.47:0/859371191 >> 172.21.15.47:0/859371191 conn(0x7f682c07d220 msgr2=0x7f682c1074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:07.953 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:07.157+0000 7f6832000700 1 -- 172.21.15.47:0/859371191 shutdown_connections 2024-03-21T11:12:07.954 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:07.157+0000 7f6832000700 1 -- 172.21.15.47:0/859371191 wait complete. 2024-03-21T11:12:07.954 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for the mgr to restart... 2024-03-21T11:12:07.954 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr epoch 5... 2024-03-21T11:12:08.063 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:07 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:07.815+0000 7f94c588f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T11:12:08.063 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:07 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:07.931+0000 7f94c588f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T11:12:08.372 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:08 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:08.286+0000 7f94c588f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T11:12:08.683 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:08 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:08.435+0000 7f94c588f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T11:12:08.683 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:08 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:08.555+0000 7f94c588f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T11:12:08.683 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:08 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:08.683+0000 7f94c588f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T11:12:09.063 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:08 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:08.907+0000 7f94c588f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T11:12:10.563 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:10 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:10.232+0000 7f94c588f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T11:12:10.563 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:10 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:10.411+0000 7f94c588f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T11:12:11.226 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:10 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:10.838+0000 7f94c588f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T11:12:11.226 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:10 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:10.960+0000 7f94c588f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T11:12:11.478 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:11 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:11.226+0000 7f94c588f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T11:12:11.813 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:11 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:11.477+0000 7f94c588f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T11:12:12.187 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:11 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:11.863+0000 7f94c588f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T11:12:12.500 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:12 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:12.186+0000 7f94c588f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T11:12:12.813 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:12 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:12.499+0000 7f94c588f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T11:12:14.063 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:13 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:13.566+0000 7f94c588f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T11:12:14.563 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:14 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:14.284+0000 7f94c588f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T11:12:14.563 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:14 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:14.417+0000 7f94c588f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T11:12:15.063 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:14 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:14.709+0000 7f94c588f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T11:12:15.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:14 smithi047 ceph-mon[31351]: Active manager daemon x restarted 2024-03-21T11:12:15.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:14 smithi047 ceph-mon[31351]: Activating manager daemon x 2024-03-21T11:12:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:14 smithi047 ceph-mon[31351]: osdmap e2: 0 total, 0 up, 0 in 2024-03-21T11:12:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:14 smithi047 ceph-mon[31351]: mgrmap e6: x(active, starting, since 0.00932661s) 2024-03-21T11:12:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:14 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:12:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:14 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T11:12:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:14 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T11:12:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:14 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T11:12:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:14 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T11:12:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:14 smithi047 ceph-mon[31351]: Manager daemon x is now available 2024-03-21T11:12:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:14 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' 2024-03-21T11:12:16.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:15 smithi047 ceph-mon[31351]: Found migration_current of "None". Setting to last migration. 2024-03-21T11:12:16.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:15 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' 2024-03-21T11:12:16.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:15 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:12:16.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:15 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:12:16.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:15 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T11:12:16.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:15 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T11:12:16.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:15 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' 2024-03-21T11:12:16.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:15 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' 2024-03-21T11:12:16.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:15 smithi047 ceph-mon[31351]: [21/Mar/2024:11:12:15] ENGINE Bus STARTING 2024-03-21T11:12:16.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:15 smithi047 ceph-mon[31351]: mgrmap e7: x(active, since 1.01711s) 2024-03-21T11:12:16.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T11:12:16.566 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-21T11:12:16.566 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T11:12:16.566 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T11:12:16.566 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.530+0000 7ff33fe00700 1 Processor -- start 2024-03-21T11:12:16.566 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.530+0000 7ff33fe00700 1 -- start start 2024-03-21T11:12:16.566 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.530+0000 7ff33fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3381085e0 0x7ff33810a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:16.566 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.531+0000 7ff33fe00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff338075880 con 0x7ff3381085e0 2024-03-21T11:12:16.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.531+0000 7ff33d400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3381085e0 0x7ff33810a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:16.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.531+0000 7ff33d400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3381085e0 0x7ff33810a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48600/0 (socket says 172.21.15.47:48600) 2024-03-21T11:12:16.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.531+0000 7ff33d400700 1 -- 172.21.15.47:0/2365799891 learned_addr learned my addr 172.21.15.47:0/2365799891 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:16.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.532+0000 7ff33d400700 1 -- 172.21.15.47:0/2365799891 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33810af10 con 0x7ff3381085e0 2024-03-21T11:12:16.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.532+0000 7ff33d400700 1 --2- 172.21.15.47:0/2365799891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3381085e0 0x7ff33810a9d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0098a0 tx=0x7ff32c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0defb3dbb33732 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:16.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.532+0000 7ff337e00700 1 -- 172.21.15.47:0/2365799891 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff32c004160 con 0x7ff3381085e0 2024-03-21T11:12:16.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.532+0000 7ff337e00700 1 -- 172.21.15.47:0/2365799891 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff32c004300 con 0x7ff3381085e0 2024-03-21T11:12:16.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.532+0000 7ff337e00700 1 -- 172.21.15.47:0/2365799891 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff32c011b50 con 0x7ff3381085e0 2024-03-21T11:12:16.567 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.533+0000 7ff33fe00700 1 -- 172.21.15.47:0/2365799891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3381085e0 msgr2=0x7ff33810a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:16.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.533+0000 7ff33fe00700 1 --2- 172.21.15.47:0/2365799891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3381085e0 0x7ff33810a9d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0098a0 tx=0x7ff32c009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:16.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.533+0000 7ff33fe00700 1 -- 172.21.15.47:0/2365799891 shutdown_connections 2024-03-21T11:12:16.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.533+0000 7ff33fe00700 1 --2- 172.21.15.47:0/2365799891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3381085e0 0x7ff33810a9d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:16.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.533+0000 7ff33fe00700 1 -- 172.21.15.47:0/2365799891 >> 172.21.15.47:0/2365799891 conn(0x7ff338101fb0 msgr2=0x7ff3381043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:16.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.533+0000 7ff33fe00700 1 -- 172.21.15.47:0/2365799891 shutdown_connections 2024-03-21T11:12:16.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.533+0000 7ff33fe00700 1 -- 172.21.15.47:0/2365799891 wait complete. 2024-03-21T11:12:16.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.534+0000 7ff33fe00700 1 Processor -- start 2024-03-21T11:12:16.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.535+0000 7ff33fe00700 1 -- start start 2024-03-21T11:12:16.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.535+0000 7ff33fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3381085e0 0x7ff33819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:16.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.535+0000 7ff33fe00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33810bed0 con 0x7ff3381085e0 2024-03-21T11:12:16.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.535+0000 7ff33d400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3381085e0 0x7ff33819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:16.568 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.535+0000 7ff33d400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3381085e0 0x7ff33819dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48614/0 (socket says 172.21.15.47:48614) 2024-03-21T11:12:16.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.535+0000 7ff33d400700 1 -- 172.21.15.47:0/2291391082 learned_addr learned my addr 172.21.15.47:0/2291391082 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:16.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.536+0000 7ff33d400700 1 -- 172.21.15.47:0/2291391082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33819e280 con 0x7ff3381085e0 2024-03-21T11:12:16.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.536+0000 7ff33d400700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3381085e0 0x7ff33819dd40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff32c011b60 tx=0x7ff32c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:16.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.536+0000 7ff336000700 1 -- 172.21.15.47:0/2291391082 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff32c003970 con 0x7ff3381085e0 2024-03-21T11:12:16.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.536+0000 7ff336000700 1 -- 172.21.15.47:0/2291391082 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff32c003b10 con 0x7ff3381085e0 2024-03-21T11:12:16.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.536+0000 7ff336000700 1 -- 172.21.15.47:0/2291391082 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff32c024610 con 0x7ff3381085e0 2024-03-21T11:12:16.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.536+0000 7ff33fe00700 1 -- 172.21.15.47:0/2291391082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33819e510 con 0x7ff3381085e0 2024-03-21T11:12:16.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.537+0000 7ff33fe00700 1 -- 172.21.15.47:0/2291391082 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3381a1200 con 0x7ff3381085e0 2024-03-21T11:12:16.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.538+0000 7ff336000700 1 -- 172.21.15.47:0/2291391082 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7ff32c01a9f0 con 0x7ff3381085e0 2024-03-21T11:12:16.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.538+0000 7ff336000700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7ff3200391d0 0x7ff32003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:16.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.538+0000 7ff336000700 1 -- 172.21.15.47:0/2291391082 --> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff32003bda0 con 0x7ff3200391d0 2024-03-21T11:12:16.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.539+0000 7ff336000700 1 -- 172.21.15.47:0/2291391082 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff32c011e00 con 0x7ff3381085e0 2024-03-21T11:12:16.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.539+0000 7ff33ca00700 1 -- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7ff3200391d0 msgr2=0x7ff32003b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1584030551 2024-03-21T11:12:16.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.539+0000 7ff33ca00700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7ff3200391d0 0x7ff32003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T11:12:16.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.740+0000 7ff33ca00700 1 -- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7ff3200391d0 msgr2=0x7ff32003b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1584030551 2024-03-21T11:12:16.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:08.740+0000 7ff33ca00700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7ff3200391d0 0x7ff32003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T11:12:16.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:09.141+0000 7ff33ca00700 1 -- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7ff3200391d0 msgr2=0x7ff32003b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1584030551 2024-03-21T11:12:16.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:09.141+0000 7ff33ca00700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7ff3200391d0 0x7ff32003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T11:12:16.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:09.942+0000 7ff33ca00700 1 -- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7ff3200391d0 msgr2=0x7ff32003b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1584030551 2024-03-21T11:12:16.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:09.942+0000 7ff33ca00700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7ff3200391d0 0x7ff32003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T11:12:16.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:11.543+0000 7ff33ca00700 1 -- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7ff3200391d0 msgr2=0x7ff32003b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/1584030551 2024-03-21T11:12:16.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:11.543+0000 7ff33ca00700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7ff3200391d0 0x7ff32003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T11:12:16.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:14.718+0000 7ff336000700 1 -- 172.21.15.47:0/2291391082 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mgrmap(e 6) v1 ==== 46061+0+0 (secure 0 0 0) 0x7ff32c024c40 con 0x7ff3381085e0 2024-03-21T11:12:16.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:14.719+0000 7ff336000700 1 -- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7ff3200391d0 msgr2=0x7ff32003b690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:12:16.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:14.719+0000 7ff336000700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7ff3200391d0 0x7ff32003b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:16.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.723+0000 7ff336000700 1 -- 172.21.15.47:0/2291391082 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 7) v1 ==== 46188+0+0 (secure 0 0 0) 0x7ff32c04ed00 con 0x7ff3381085e0 2024-03-21T11:12:16.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.723+0000 7ff336000700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7ff32003cac0 0x7ff32003eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:16.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.723+0000 7ff336000700 1 -- 172.21.15.47:0/2291391082 --> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff32c012ea0 con 0x7ff32003cac0 2024-03-21T11:12:16.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.725+0000 7ff33ca00700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7ff32003cac0 0x7ff32003eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:16.574 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.728+0000 7ff33ca00700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7ff32003cac0 0x7ff32003eeb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff324003a10 tx=0x7ff3240092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:16.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.730+0000 7ff336000700 1 -- 172.21.15.47:0/2291391082 <== mgr.14120 v2:172.21.15.47:6800/185270192 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7ff32c012ea0 con 0x7ff32003cac0 2024-03-21T11:12:16.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.739+0000 7ff33fe00700 1 -- 172.21.15.47:0/2291391082 --> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff3380637c0 con 0x7ff32003cac0 2024-03-21T11:12:16.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.739+0000 7ff336000700 1 -- 172.21.15.47:0/2291391082 <== mgr.14120 v2:172.21.15.47:6800/185270192 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff3380637c0 con 0x7ff32003cac0 2024-03-21T11:12:16.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.740+0000 7ff33fe00700 1 -- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7ff32003cac0 msgr2=0x7ff32003eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:16.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.740+0000 7ff33fe00700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7ff32003cac0 0x7ff32003eeb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff324003a10 tx=0x7ff3240092b0 comp rx=0 tx=0).stop 2024-03-21T11:12:16.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.740+0000 7ff33fe00700 1 -- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3381085e0 msgr2=0x7ff33819dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:16.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.740+0000 7ff33fe00700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3381085e0 0x7ff33819dd40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff32c011b60 tx=0x7ff32c011c40 comp rx=0 tx=0).stop 2024-03-21T11:12:16.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.740+0000 7ff33fe00700 1 -- 172.21.15.47:0/2291391082 shutdown_connections 2024-03-21T11:12:16.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.740+0000 7ff33fe00700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7ff32003cac0 0x7ff32003eeb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:16.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.740+0000 7ff33fe00700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3381085e0 0x7ff33819dd40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:16.575 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.740+0000 7ff33fe00700 1 --2- 172.21.15.47:0/2291391082 >> [v2:172.21.15.47:6800/1584030551,v1:172.21.15.47:6801/1584030551] conn(0x7ff3200391d0 0x7ff32003b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:16.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.740+0000 7ff33fe00700 1 -- 172.21.15.47:0/2291391082 >> 172.21.15.47:0/2291391082 conn(0x7ff338101fb0 msgr2=0x7ff338102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:16.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.741+0000 7ff33fe00700 1 -- 172.21.15.47:0/2291391082 shutdown_connections 2024-03-21T11:12:16.576 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:15.741+0000 7ff33fe00700 1 -- 172.21.15.47:0/2291391082 wait complete. 2024-03-21T11:12:16.576 INFO:teuthology.orchestra.run.smithi047.stdout:mgr epoch 5 is available 2024-03-21T11:12:16.576 INFO:teuthology.orchestra.run.smithi047.stdout:Setting orchestrator backend to cephadm... 2024-03-21T11:12:16.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:16 smithi047 ceph-mon[31351]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T11:12:17.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:16 smithi047 ceph-mon[31351]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T11:12:17.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:16 smithi047 ceph-mon[31351]: [21/Mar/2024:11:12:15] ENGINE Serving on http://172.21.15.47:8765 2024-03-21T11:12:17.627 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:16 smithi047 ceph-mon[31351]: [21/Mar/2024:11:12:15] ENGINE Serving on https://172.21.15.47:7150 2024-03-21T11:12:17.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:16 smithi047 ceph-mon[31351]: [21/Mar/2024:11:12:15] ENGINE Bus STARTED 2024-03-21T11:12:17.628 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:16 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:12:18.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:17 smithi047 ceph-mon[31351]: mgrmap e8: x(active, since 2s) 2024-03-21T11:12:18.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:17 smithi047 ceph-mon[31351]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:12:18.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:17 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' 2024-03-21T11:12:18.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:17 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:12:18.124 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.156+0000 7f4977200700 1 Processor -- start 2024-03-21T11:12:18.124 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.156+0000 7f4977200700 1 -- start start 2024-03-21T11:12:18.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.157+0000 7f4977200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4978109bf0 0x7f497810a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:18.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.157+0000 7f4977200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497810a5e0 con 0x7f4978109bf0 2024-03-21T11:12:18.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.157+0000 7f4975e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4978109bf0 0x7f497810a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:18.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.157+0000 7f4975e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4978109bf0 0x7f497810a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59364/0 (socket says 172.21.15.47:59364) 2024-03-21T11:12:18.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.157+0000 7f4975e00700 1 -- 172.21.15.47:0/3170107571 learned_addr learned my addr 172.21.15.47:0/3170107571 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:18.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.158+0000 7f4975e00700 1 -- 172.21.15.47:0/3170107571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497810ad50 con 0x7f4978109bf0 2024-03-21T11:12:18.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.158+0000 7f4975e00700 1 --2- 172.21.15.47:0/3170107571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4978109bf0 0x7f497810a010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f49680098a0 tx=0x7f4968009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfb108d3a0d4be9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:18.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.158+0000 7f4974a00700 1 -- 172.21.15.47:0/3170107571 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4968004160 con 0x7f4978109bf0 2024-03-21T11:12:18.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.158+0000 7f4974a00700 1 -- 172.21.15.47:0/3170107571 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4968004300 con 0x7f4978109bf0 2024-03-21T11:12:18.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.158+0000 7f4974a00700 1 -- 172.21.15.47:0/3170107571 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4968011b50 con 0x7f4978109bf0 2024-03-21T11:12:18.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.159+0000 7f4977200700 1 -- 172.21.15.47:0/3170107571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4978109bf0 msgr2=0x7f497810a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:18.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.159+0000 7f4977200700 1 --2- 172.21.15.47:0/3170107571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4978109bf0 0x7f497810a010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f49680098a0 tx=0x7f4968009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:18.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.159+0000 7f4977200700 1 -- 172.21.15.47:0/3170107571 shutdown_connections 2024-03-21T11:12:18.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.159+0000 7f4977200700 1 --2- 172.21.15.47:0/3170107571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4978109bf0 0x7f497810a010 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:18.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.159+0000 7f4977200700 1 -- 172.21.15.47:0/3170107571 >> 172.21.15.47:0/3170107571 conn(0x7f49781053d0 msgr2=0x7f49781077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:18.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.159+0000 7f4977200700 1 -- 172.21.15.47:0/3170107571 shutdown_connections 2024-03-21T11:12:18.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.159+0000 7f4977200700 1 -- 172.21.15.47:0/3170107571 wait complete. 2024-03-21T11:12:18.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.160+0000 7f4977200700 1 Processor -- start 2024-03-21T11:12:18.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.161+0000 7f4977200700 1 -- start start 2024-03-21T11:12:18.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.161+0000 7f4977200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4978109bf0 0x7f497819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:18.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.161+0000 7f4977200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497810e100 con 0x7f4978109bf0 2024-03-21T11:12:18.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.161+0000 7f4975e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4978109bf0 0x7f497819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:18.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.161+0000 7f4975e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4978109bf0 0x7f497819db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59376/0 (socket says 172.21.15.47:59376) 2024-03-21T11:12:18.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.161+0000 7f4975e00700 1 -- 172.21.15.47:0/524607370 learned_addr learned my addr 172.21.15.47:0/524607370 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:18.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.161+0000 7f4975e00700 1 -- 172.21.15.47:0/524607370 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497819e0b0 con 0x7f4978109bf0 2024-03-21T11:12:18.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.162+0000 7f4975e00700 1 --2- 172.21.15.47:0/524607370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4978109bf0 0x7f497819db70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4968011850 tx=0x7f4968011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:18.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.162+0000 7f496ea00700 1 -- 172.21.15.47:0/524607370 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4968003af0 con 0x7f4978109bf0 2024-03-21T11:12:18.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.162+0000 7f496ea00700 1 -- 172.21.15.47:0/524607370 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4968020070 con 0x7f4978109bf0 2024-03-21T11:12:18.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.162+0000 7f496ea00700 1 -- 172.21.15.47:0/524607370 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49680195a0 con 0x7f4978109bf0 2024-03-21T11:12:18.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.162+0000 7f4977200700 1 -- 172.21.15.47:0/524607370 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497819e340 con 0x7f4978109bf0 2024-03-21T11:12:18.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.162+0000 7f4977200700 1 -- 172.21.15.47:0/524607370 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49781a1030 con 0x7f4978109bf0 2024-03-21T11:12:18.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.163+0000 7f496ea00700 1 -- 172.21.15.47:0/524607370 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f4968019850 con 0x7f4978109bf0 2024-03-21T11:12:18.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.163+0000 7f4977200700 1 -- 172.21.15.47:0/524607370 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49780637c0 con 0x7f4978109bf0 2024-03-21T11:12:18.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.164+0000 7f496ea00700 1 --2- 172.21.15.47:0/524607370 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f495c0391d0 0x7f495c03b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:18.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.164+0000 7f496ea00700 1 -- 172.21.15.47:0/524607370 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4968011e00 con 0x7f4978109bf0 2024-03-21T11:12:18.127 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.164+0000 7f4975400700 1 --2- 172.21.15.47:0/524607370 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f495c0391d0 0x7f495c03b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:18.128 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.165+0000 7f4975400700 1 --2- 172.21.15.47:0/524607370 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f495c0391d0 0x7f495c03b690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4964006fd0 tx=0x7f4964006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:18.128 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.170+0000 7f496ea00700 1 -- 172.21.15.47:0/524607370 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4968019b40 con 0x7f4978109bf0 2024-03-21T11:12:18.128 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.413+0000 7f4977200700 1 -- 172.21.15.47:0/524607370 --> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f4978107610 con 0x7f495c0391d0 2024-03-21T11:12:18.128 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.432+0000 7f496ea00700 1 -- 172.21.15.47:0/524607370 <== mgr.14120 v2:172.21.15.47:6800/185270192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4978107610 con 0x7f495c0391d0 2024-03-21T11:12:18.128 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.435+0000 7f4977200700 1 -- 172.21.15.47:0/524607370 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f495c0391d0 msgr2=0x7f495c03b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:18.128 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.435+0000 7f4977200700 1 --2- 172.21.15.47:0/524607370 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f495c0391d0 0x7f495c03b690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4964006fd0 tx=0x7f4964006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:18.128 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.435+0000 7f4977200700 1 -- 172.21.15.47:0/524607370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4978109bf0 msgr2=0x7f497819db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:18.128 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.435+0000 7f4977200700 1 --2- 172.21.15.47:0/524607370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4978109bf0 0x7f497819db70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4968011850 tx=0x7f4968011c10 comp rx=0 tx=0).stop 2024-03-21T11:12:18.128 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.435+0000 7f4977200700 1 -- 172.21.15.47:0/524607370 shutdown_connections 2024-03-21T11:12:18.128 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.435+0000 7f4977200700 1 --2- 172.21.15.47:0/524607370 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f495c0391d0 0x7f495c03b690 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:18.128 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.435+0000 7f4977200700 1 --2- 172.21.15.47:0/524607370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4978109bf0 0x7f497819db70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:18.128 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.435+0000 7f4977200700 1 -- 172.21.15.47:0/524607370 >> 172.21.15.47:0/524607370 conn(0x7f49781053d0 msgr2=0x7f4978106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:18.128 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.436+0000 7f4977200700 1 -- 172.21.15.47:0/524607370 shutdown_connections 2024-03-21T11:12:18.129 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:17.436+0000 7f4977200700 1 -- 172.21.15.47:0/524607370 wait complete. 2024-03-21T11:12:19.618 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-21T11:12:19.619 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.710+0000 7fdc2d400700 1 Processor -- start 2024-03-21T11:12:19.619 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.710+0000 7fdc2d400700 1 -- start start 2024-03-21T11:12:19.620 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.711+0000 7fdc2d400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2807d770 0x7fdc2807db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:19.620 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.711+0000 7fdc2d400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2807e160 con 0x7fdc2807d770 2024-03-21T11:12:19.620 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.711+0000 7fdc26800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2807d770 0x7fdc2807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:19.620 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.712+0000 7fdc26800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2807d770 0x7fdc2807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59392/0 (socket says 172.21.15.47:59392) 2024-03-21T11:12:19.621 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.712+0000 7fdc26800700 1 -- 172.21.15.47:0/1659253710 learned_addr learned my addr 172.21.15.47:0/1659253710 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:19.621 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.712+0000 7fdc26800700 1 -- 172.21.15.47:0/1659253710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2807e9c0 con 0x7fdc2807d770 2024-03-21T11:12:19.621 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.712+0000 7fdc26800700 1 --2- 172.21.15.47:0/1659253710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2807d770 0x7fdc2807db90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdc180098a0 tx=0x7fdc18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3d843b1a74725c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:19.621 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.713+0000 7fdc25400700 1 -- 172.21.15.47:0/1659253710 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc18004160 con 0x7fdc2807d770 2024-03-21T11:12:19.621 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.713+0000 7fdc25400700 1 -- 172.21.15.47:0/1659253710 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc18004300 con 0x7fdc2807d770 2024-03-21T11:12:19.623 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.713+0000 7fdc25400700 1 -- 172.21.15.47:0/1659253710 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc18011b50 con 0x7fdc2807d770 2024-03-21T11:12:19.623 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.714+0000 7fdc2d400700 1 -- 172.21.15.47:0/1659253710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2807d770 msgr2=0x7fdc2807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:19.623 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.714+0000 7fdc2d400700 1 --2- 172.21.15.47:0/1659253710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2807d770 0x7fdc2807db90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdc180098a0 tx=0x7fdc18009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:19.623 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.714+0000 7fdc2d400700 1 -- 172.21.15.47:0/1659253710 shutdown_connections 2024-03-21T11:12:19.623 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.714+0000 7fdc2d400700 1 --2- 172.21.15.47:0/1659253710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2807d770 0x7fdc2807db90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:19.623 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.714+0000 7fdc2d400700 1 -- 172.21.15.47:0/1659253710 >> 172.21.15.47:0/1659253710 conn(0x7fdc281053d0 msgr2=0x7fdc281077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:19.624 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.714+0000 7fdc2d400700 1 -- 172.21.15.47:0/1659253710 shutdown_connections 2024-03-21T11:12:19.624 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.714+0000 7fdc2d400700 1 -- 172.21.15.47:0/1659253710 wait complete. 2024-03-21T11:12:19.624 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.715+0000 7fdc2d400700 1 Processor -- start 2024-03-21T11:12:19.624 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.715+0000 7fdc2d400700 1 -- start start 2024-03-21T11:12:19.624 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.716+0000 7fdc2d400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2807d770 0x7fdc281a2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:19.624 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.716+0000 7fdc2d400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2807fd80 con 0x7fdc2807d770 2024-03-21T11:12:19.624 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.716+0000 7fdc26800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2807d770 0x7fdc281a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:19.624 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.716+0000 7fdc26800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2807d770 0x7fdc281a2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59404/0 (socket says 172.21.15.47:59404) 2024-03-21T11:12:19.624 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.716+0000 7fdc26800700 1 -- 172.21.15.47:0/3830791405 learned_addr learned my addr 172.21.15.47:0/3830791405 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:19.624 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.716+0000 7fdc26800700 1 -- 172.21.15.47:0/3830791405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc281a25c0 con 0x7fdc2807d770 2024-03-21T11:12:19.624 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.717+0000 7fdc26800700 1 --2- 172.21.15.47:0/3830791405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2807d770 0x7fdc281a2080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdc18011b60 tx=0x7fdc18011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:19.624 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.717+0000 7fdc1f400700 1 -- 172.21.15.47:0/3830791405 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc18003b20 con 0x7fdc2807d770 2024-03-21T11:12:19.627 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.717+0000 7fdc1f400700 1 -- 172.21.15.47:0/3830791405 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc18003cc0 con 0x7fdc2807d770 2024-03-21T11:12:19.627 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.717+0000 7fdc2d400700 1 -- 172.21.15.47:0/3830791405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc281a2850 con 0x7fdc2807d770 2024-03-21T11:12:19.627 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.717+0000 7fdc1f400700 1 -- 172.21.15.47:0/3830791405 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc18024840 con 0x7fdc2807d770 2024-03-21T11:12:19.627 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.717+0000 7fdc2d400700 1 -- 172.21.15.47:0/3830791405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc281a5540 con 0x7fdc2807d770 2024-03-21T11:12:19.627 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.719+0000 7fdc1f400700 1 -- 172.21.15.47:0/3830791405 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fdc18022020 con 0x7fdc2807d770 2024-03-21T11:12:19.627 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.719+0000 7fdc2d400700 1 -- 172.21.15.47:0/3830791405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc280637c0 con 0x7fdc2807d770 2024-03-21T11:12:19.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.719+0000 7fdc1f400700 1 --2- 172.21.15.47:0/3830791405 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fdc0c0391d0 0x7fdc0c03b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:19.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.719+0000 7fdc1f400700 1 -- 172.21.15.47:0/3830791405 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdc18011e00 con 0x7fdc2807d770 2024-03-21T11:12:19.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.720+0000 7fdc25e00700 1 --2- 172.21.15.47:0/3830791405 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fdc0c0391d0 0x7fdc0c03b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:19.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.721+0000 7fdc25e00700 1 --2- 172.21.15.47:0/3830791405 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fdc0c0391d0 0x7fdc0c03b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdc14006fd0 tx=0x7fdc14006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:19.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.725+0000 7fdc1f400700 1 -- 172.21.15.47:0/3830791405 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdc1804ccd0 con 0x7fdc2807d770 2024-03-21T11:12:19.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.953+0000 7fdc2d400700 1 -- 172.21.15.47:0/3830791405 --> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc280008d0 con 0x7fdc0c0391d0 2024-03-21T11:12:19.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.955+0000 7fdc1f400700 1 -- 172.21.15.47:0/3830791405 <== mgr.14120 v2:172.21.15.47:6800/185270192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fdc280008d0 con 0x7fdc0c0391d0 2024-03-21T11:12:19.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.958+0000 7fdc2d400700 1 -- 172.21.15.47:0/3830791405 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fdc0c0391d0 msgr2=0x7fdc0c03b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:19.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.958+0000 7fdc2d400700 1 --2- 172.21.15.47:0/3830791405 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fdc0c0391d0 0x7fdc0c03b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdc14006fd0 tx=0x7fdc14006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:19.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.959+0000 7fdc2d400700 1 -- 172.21.15.47:0/3830791405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2807d770 msgr2=0x7fdc281a2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:19.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.959+0000 7fdc2d400700 1 --2- 172.21.15.47:0/3830791405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2807d770 0x7fdc281a2080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdc18011b60 tx=0x7fdc18011c40 comp rx=0 tx=0).stop 2024-03-21T11:12:19.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.959+0000 7fdc2d400700 1 -- 172.21.15.47:0/3830791405 shutdown_connections 2024-03-21T11:12:19.628 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.959+0000 7fdc2d400700 1 --2- 172.21.15.47:0/3830791405 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fdc0c0391d0 0x7fdc0c03b690 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:19.629 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.959+0000 7fdc2d400700 1 --2- 172.21.15.47:0/3830791405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc2807d770 0x7fdc281a2080 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:19.629 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.959+0000 7fdc2d400700 1 -- 172.21.15.47:0/3830791405 >> 172.21.15.47:0/3830791405 conn(0x7fdc281053d0 msgr2=0x7fdc28198d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:19.629 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.959+0000 7fdc2d400700 1 -- 172.21.15.47:0/3830791405 shutdown_connections 2024-03-21T11:12:19.629 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:18.959+0000 7fdc2d400700 1 -- 172.21.15.47:0/3830791405 wait complete. 2024-03-21T11:12:19.629 INFO:teuthology.orchestra.run.smithi047.stdout:Generating ssh key... 2024-03-21T11:12:19.922 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:19 smithi047 ceph-mon[31351]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:12:21.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-mon[31351]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:12:21.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-mon[31351]: Generating ssh key... 2024-03-21T11:12:21.064 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: Generating public/private rsa key pair. 2024-03-21T11:12:21.064 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: Your identification has been saved in /tmp/tmplqc6z6au/key. 2024-03-21T11:12:21.064 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: Your public key has been saved in /tmp/tmplqc6z6au/key.pub. 2024-03-21T11:12:21.064 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: The key fingerprint is: 2024-03-21T11:12:21.064 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: SHA256:0tGvYnU5YqWF34sYdZb6VkqNqncDEBIBMgPivMDkLmw ceph-a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:12:21.064 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: The key's randomart image is: 2024-03-21T11:12:21.064 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: +---[RSA 3072]----+ 2024-03-21T11:12:21.064 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: |....+ ..oo | 2024-03-21T11:12:21.064 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: |*. + .... . | 2024-03-21T11:12:21.065 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: |.= ..o.+ + | 2024-03-21T11:12:21.065 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: |+ . . ..B * o | 2024-03-21T11:12:21.065 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: |.E . S *.B + o| 2024-03-21T11:12:21.065 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: |o . o *.* + | 2024-03-21T11:12:21.065 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: | o o o.= | 2024-03-21T11:12:21.065 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: | . . ...o | 2024-03-21T11:12:21.065 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: | .. . . | 2024-03-21T11:12:21.065 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: +----[SHA256]-----+ 2024-03-21T11:12:21.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.224+0000 7fa49f000700 1 Processor -- start 2024-03-21T11:12:21.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.225+0000 7fa49f000700 1 -- start start 2024-03-21T11:12:21.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.225+0000 7fa49f000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa498107b30 0x7fa498107f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:21.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.225+0000 7fa49f000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa498108520 con 0x7fa498107b30 2024-03-21T11:12:21.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.226+0000 7fa497e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa498107b30 0x7fa498107f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:21.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.226+0000 7fa497e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa498107b30 0x7fa498107f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43748/0 (socket says 172.21.15.47:43748) 2024-03-21T11:12:21.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.226+0000 7fa497e00700 1 -- 172.21.15.47:0/3476091534 learned_addr learned my addr 172.21.15.47:0/3476091534 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:21.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.226+0000 7fa497e00700 1 -- 172.21.15.47:0/3476091534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa498108d30 con 0x7fa498107b30 2024-03-21T11:12:21.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.227+0000 7fa497e00700 1 --2- 172.21.15.47:0/3476091534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa498107b30 0x7fa498107f50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa4880098a0 tx=0x7fa488009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0c6915eb5f6f6a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:21.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.227+0000 7fa496a00700 1 -- 172.21.15.47:0/3476091534 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa488004160 con 0x7fa498107b30 2024-03-21T11:12:21.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.227+0000 7fa496a00700 1 -- 172.21.15.47:0/3476091534 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa488004300 con 0x7fa498107b30 2024-03-21T11:12:21.579 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.227+0000 7fa496a00700 1 -- 172.21.15.47:0/3476091534 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa488011b50 con 0x7fa498107b30 2024-03-21T11:12:21.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.228+0000 7fa49f000700 1 -- 172.21.15.47:0/3476091534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa498107b30 msgr2=0x7fa498107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:21.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.228+0000 7fa49f000700 1 --2- 172.21.15.47:0/3476091534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa498107b30 0x7fa498107f50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa4880098a0 tx=0x7fa488009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:21.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.228+0000 7fa49f000700 1 -- 172.21.15.47:0/3476091534 shutdown_connections 2024-03-21T11:12:21.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.228+0000 7fa49f000700 1 --2- 172.21.15.47:0/3476091534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa498107b30 0x7fa498107f50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:21.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.228+0000 7fa49f000700 1 -- 172.21.15.47:0/3476091534 >> 172.21.15.47:0/3476091534 conn(0x7fa498103150 msgr2=0x7fa498105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:21.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.228+0000 7fa49f000700 1 -- 172.21.15.47:0/3476091534 shutdown_connections 2024-03-21T11:12:21.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.228+0000 7fa49f000700 1 -- 172.21.15.47:0/3476091534 wait complete. 2024-03-21T11:12:21.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.229+0000 7fa49f000700 1 Processor -- start 2024-03-21T11:12:21.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.229+0000 7fa49f000700 1 -- start start 2024-03-21T11:12:21.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.230+0000 7fa49f000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa498107b30 0x7fa49807db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:21.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.230+0000 7fa49f000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49810c0e0 con 0x7fa498107b30 2024-03-21T11:12:21.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.230+0000 7fa497e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa498107b30 0x7fa49807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:21.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.230+0000 7fa497e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa498107b30 0x7fa49807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43754/0 (socket says 172.21.15.47:43754) 2024-03-21T11:12:21.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.230+0000 7fa497e00700 1 -- 172.21.15.47:0/2182458979 learned_addr learned my addr 172.21.15.47:0/2182458979 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:21.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.230+0000 7fa497e00700 1 -- 172.21.15.47:0/2182458979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49807e0d0 con 0x7fa498107b30 2024-03-21T11:12:21.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.231+0000 7fa497e00700 1 --2- 172.21.15.47:0/2182458979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa498107b30 0x7fa49807db90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa488009fb0 tx=0x7fa488011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:21.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.231+0000 7fa494c00700 1 -- 172.21.15.47:0/2182458979 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4880038b0 con 0x7fa498107b30 2024-03-21T11:12:21.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.231+0000 7fa494c00700 1 -- 172.21.15.47:0/2182458979 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa488011e50 con 0x7fa498107b30 2024-03-21T11:12:21.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.231+0000 7fa494c00700 1 -- 172.21.15.47:0/2182458979 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa488019550 con 0x7fa498107b30 2024-03-21T11:12:21.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.231+0000 7fa49f000700 1 -- 172.21.15.47:0/2182458979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49807c2c0 con 0x7fa498107b30 2024-03-21T11:12:21.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.231+0000 7fa49f000700 1 -- 172.21.15.47:0/2182458979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49807c800 con 0x7fa498107b30 2024-03-21T11:12:21.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.232+0000 7fa494c00700 1 -- 172.21.15.47:0/2182458979 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fa488003a50 con 0x7fa498107b30 2024-03-21T11:12:21.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.232+0000 7fa49f000700 1 -- 172.21.15.47:0/2182458979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4980637c0 con 0x7fa498107b30 2024-03-21T11:12:21.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.233+0000 7fa494c00700 1 --2- 172.21.15.47:0/2182458979 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fa47c039180 0x7fa47c03b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:21.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.233+0000 7fa494c00700 1 -- 172.21.15.47:0/2182458979 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa48804d390 con 0x7fa498107b30 2024-03-21T11:12:21.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.233+0000 7fa497400700 1 --2- 172.21.15.47:0/2182458979 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fa47c039180 0x7fa47c03b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:21.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.234+0000 7fa497400700 1 --2- 172.21.15.47:0/2182458979 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fa47c039180 0x7fa47c03b640 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa484006fd0 tx=0x7fa484006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:21.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.238+0000 7fa494c00700 1 -- 172.21.15.47:0/2182458979 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa488016dd0 con 0x7fa498107b30 2024-03-21T11:12:21.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.464+0000 7fa49f000700 1 -- 172.21.15.47:0/2182458979 --> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fa49807d050 con 0x7fa47c039180 2024-03-21T11:12:21.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.881+0000 7fa494c00700 1 -- 172.21.15.47:0/2182458979 <== mgr.14120 v2:172.21.15.47:6800/185270192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa49807d050 con 0x7fa47c039180 2024-03-21T11:12:21.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.884+0000 7fa49f000700 1 -- 172.21.15.47:0/2182458979 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fa47c039180 msgr2=0x7fa47c03b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:21.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.884+0000 7fa49f000700 1 --2- 172.21.15.47:0/2182458979 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fa47c039180 0x7fa47c03b640 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa484006fd0 tx=0x7fa484006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:21.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.884+0000 7fa49f000700 1 -- 172.21.15.47:0/2182458979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa498107b30 msgr2=0x7fa49807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:21.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.884+0000 7fa49f000700 1 --2- 172.21.15.47:0/2182458979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa498107b30 0x7fa49807db90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa488009fb0 tx=0x7fa488011bf0 comp rx=0 tx=0).stop 2024-03-21T11:12:21.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.884+0000 7fa49f000700 1 -- 172.21.15.47:0/2182458979 shutdown_connections 2024-03-21T11:12:21.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.884+0000 7fa49f000700 1 --2- 172.21.15.47:0/2182458979 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fa47c039180 0x7fa47c03b640 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:21.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.884+0000 7fa49f000700 1 --2- 172.21.15.47:0/2182458979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa498107b30 0x7fa49807db90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:21.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.884+0000 7fa49f000700 1 -- 172.21.15.47:0/2182458979 >> 172.21.15.47:0/2182458979 conn(0x7fa498103150 msgr2=0x7fa498103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:21.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.884+0000 7fa49f000700 1 -- 172.21.15.47:0/2182458979 shutdown_connections 2024-03-21T11:12:21.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:20.885+0000 7fa49f000700 1 -- 172.21.15.47:0/2182458979 wait complete. 2024-03-21T11:12:22.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:21 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' 2024-03-21T11:12:22.140 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:21 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' 2024-03-21T11:12:23.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:22 smithi047 ceph-mon[31351]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:12:23.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCd2A9bkrlxpsHKSFuRY2i055vBovL/3eSqEm9uziN0m7WLUWLcpzmKdfcHuxR/yMjjLSD7CquJ1zJbVEZllAAT8biRqSZSfVUo+SHiV6FVaARsdkCmgbpbQPBdW6Z1m0ibz9/ggehOMBMd9ZPhSGIKq4Jl5SdAxYJzODe1xB6S6bOEM+LJjBFY7lnRgzm9lKqZvlw+iZFYQ3jTblXDMI1L6tpP6zSkxyKRg32XA1A9RbbtI/+WvLg24P8fGQS16Lf1J+WW0ee8JHZRkxgi27s9/1YXadLjFXHuFzIzRnojjwd3vlb/z7K81saQd+4wuQa/EH4aG4qsKHpfwPCYemPCWnMU7CKDKF2ChQjMYbTIk5twD5zj/y3GB4GA7TyKuy8tPhQtq59HI6Z7tmFtt5/WGm0hqua7NXwcPAjc5YE0DVzS9ym1f9UIWNJ7SvFD9U5iltnU+65nOTwyYkA1LqcSKBrjkPHwyv+aEdiSDoXvoS7lf/Tk7JfOwF2/KWRFxoE= ceph-a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:12:23.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.139+0000 7fd49b200700 1 Processor -- start 2024-03-21T11:12:23.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.139+0000 7fd49b200700 1 -- start start 2024-03-21T11:12:23.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.140+0000 7fd49b200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49c1061f0 0x7fd49c106610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:23.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.140+0000 7fd49b200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c075880 con 0x7fd49c1061f0 2024-03-21T11:12:23.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.140+0000 7fd499e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49c1061f0 0x7fd49c106610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:23.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.140+0000 7fd499e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49c1061f0 0x7fd49c106610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43768/0 (socket says 172.21.15.47:43768) 2024-03-21T11:12:23.069 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.140+0000 7fd499e00700 1 -- 172.21.15.47:0/590210749 learned_addr learned my addr 172.21.15.47:0/590210749 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:23.070 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.141+0000 7fd499e00700 1 -- 172.21.15.47:0/590210749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c106b50 con 0x7fd49c1061f0 2024-03-21T11:12:23.070 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.141+0000 7fd499e00700 1 --2- 172.21.15.47:0/590210749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49c1061f0 0x7fd49c106610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0098a0 tx=0x7fd48c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4ab814d5ce5ee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:23.070 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.141+0000 7fd498a00700 1 -- 172.21.15.47:0/590210749 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd48c004160 con 0x7fd49c1061f0 2024-03-21T11:12:23.070 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.142+0000 7fd498a00700 1 -- 172.21.15.47:0/590210749 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd48c004300 con 0x7fd49c1061f0 2024-03-21T11:12:23.070 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.142+0000 7fd498a00700 1 -- 172.21.15.47:0/590210749 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd48c011b50 con 0x7fd49c1061f0 2024-03-21T11:12:23.070 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.142+0000 7fd49b200700 1 -- 172.21.15.47:0/590210749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49c1061f0 msgr2=0x7fd49c106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:23.074 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.142+0000 7fd49b200700 1 --2- 172.21.15.47:0/590210749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49c1061f0 0x7fd49c106610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0098a0 tx=0x7fd48c009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:23.075 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.142+0000 7fd49b200700 1 -- 172.21.15.47:0/590210749 shutdown_connections 2024-03-21T11:12:23.075 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.142+0000 7fd49b200700 1 --2- 172.21.15.47:0/590210749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49c1061f0 0x7fd49c106610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:23.075 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.142+0000 7fd49b200700 1 -- 172.21.15.47:0/590210749 >> 172.21.15.47:0/590210749 conn(0x7fd49c101e30 msgr2=0x7fd49c104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:23.075 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.143+0000 7fd49b200700 1 -- 172.21.15.47:0/590210749 shutdown_connections 2024-03-21T11:12:23.075 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.143+0000 7fd49b200700 1 -- 172.21.15.47:0/590210749 wait complete. 2024-03-21T11:12:23.075 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.144+0000 7fd49b200700 1 Processor -- start 2024-03-21T11:12:23.075 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.144+0000 7fd49b200700 1 -- start start 2024-03-21T11:12:23.075 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.144+0000 7fd49b200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49c1061f0 0x7fd49c19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:23.075 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.144+0000 7fd49b200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c107be0 con 0x7fd49c1061f0 2024-03-21T11:12:23.075 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.144+0000 7fd499e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49c1061f0 0x7fd49c19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:23.075 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.144+0000 7fd499e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49c1061f0 0x7fd49c19da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43776/0 (socket says 172.21.15.47:43776) 2024-03-21T11:12:23.075 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.144+0000 7fd499e00700 1 -- 172.21.15.47:0/2231436938 learned_addr learned my addr 172.21.15.47:0/2231436938 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:23.075 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.145+0000 7fd499e00700 1 -- 172.21.15.47:0/2231436938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c19dfd0 con 0x7fd49c1061f0 2024-03-21T11:12:23.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.145+0000 7fd499e00700 1 --2- 172.21.15.47:0/2231436938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49c1061f0 0x7fd49c19da90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd48c011b60 tx=0x7fd48c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:23.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.145+0000 7fd492a00700 1 -- 172.21.15.47:0/2231436938 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd48c003b20 con 0x7fd49c1061f0 2024-03-21T11:12:23.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.145+0000 7fd492a00700 1 -- 172.21.15.47:0/2231436938 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd48c003cc0 con 0x7fd49c1061f0 2024-03-21T11:12:23.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.145+0000 7fd49b200700 1 -- 172.21.15.47:0/2231436938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49c19e260 con 0x7fd49c1061f0 2024-03-21T11:12:23.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.145+0000 7fd492a00700 1 -- 172.21.15.47:0/2231436938 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd48c024840 con 0x7fd49c1061f0 2024-03-21T11:12:23.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.146+0000 7fd49b200700 1 -- 172.21.15.47:0/2231436938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49c1a0f50 con 0x7fd49c1061f0 2024-03-21T11:12:23.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.147+0000 7fd492a00700 1 -- 172.21.15.47:0/2231436938 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fd48c022020 con 0x7fd49c1061f0 2024-03-21T11:12:23.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.147+0000 7fd49b200700 1 -- 172.21.15.47:0/2231436938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49c0637c0 con 0x7fd49c1061f0 2024-03-21T11:12:23.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.147+0000 7fd492a00700 1 --2- 172.21.15.47:0/2231436938 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fd4800391d0 0x7fd48003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:23.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.147+0000 7fd492a00700 1 -- 172.21.15.47:0/2231436938 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd48c011e00 con 0x7fd49c1061f0 2024-03-21T11:12:23.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.148+0000 7fd499400700 1 --2- 172.21.15.47:0/2231436938 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fd4800391d0 0x7fd48003b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:23.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.148+0000 7fd499400700 1 --2- 172.21.15.47:0/2231436938 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fd4800391d0 0x7fd48003b690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd488006fd0 tx=0x7fd488006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:23.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.153+0000 7fd492a00700 1 -- 172.21.15.47:0/2231436938 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd48c04ccd0 con 0x7fd49c1061f0 2024-03-21T11:12:23.076 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.376+0000 7fd49b200700 1 -- 172.21.15.47:0/2231436938 --> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd49c0008d0 con 0x7fd4800391d0 2024-03-21T11:12:23.077 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.377+0000 7fd492a00700 1 -- 172.21.15.47:0/2231436938 <== mgr.14120 v2:172.21.15.47:6800/185270192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fd49c0008d0 con 0x7fd4800391d0 2024-03-21T11:12:23.077 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.380+0000 7fd49b200700 1 -- 172.21.15.47:0/2231436938 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fd4800391d0 msgr2=0x7fd48003b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:23.077 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.380+0000 7fd49b200700 1 --2- 172.21.15.47:0/2231436938 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fd4800391d0 0x7fd48003b690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd488006fd0 tx=0x7fd488006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:23.077 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.380+0000 7fd49b200700 1 -- 172.21.15.47:0/2231436938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49c1061f0 msgr2=0x7fd49c19da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:23.077 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.380+0000 7fd49b200700 1 --2- 172.21.15.47:0/2231436938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49c1061f0 0x7fd49c19da90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd48c011b60 tx=0x7fd48c011c40 comp rx=0 tx=0).stop 2024-03-21T11:12:23.077 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.380+0000 7fd49b200700 1 -- 172.21.15.47:0/2231436938 shutdown_connections 2024-03-21T11:12:23.077 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.380+0000 7fd49b200700 1 --2- 172.21.15.47:0/2231436938 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7fd4800391d0 0x7fd48003b690 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:23.077 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.380+0000 7fd49b200700 1 --2- 172.21.15.47:0/2231436938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49c1061f0 0x7fd49c19da90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:23.077 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.380+0000 7fd49b200700 1 -- 172.21.15.47:0/2231436938 >> 172.21.15.47:0/2231436938 conn(0x7fd49c101e30 msgr2=0x7fd49c1946b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:23.077 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.381+0000 7fd49b200700 1 -- 172.21.15.47:0/2231436938 shutdown_connections 2024-03-21T11:12:23.077 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:22.381+0000 7fd49b200700 1 -- 172.21.15.47:0/2231436938 wait complete. 2024-03-21T11:12:23.077 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-21T11:12:23.077 INFO:teuthology.orchestra.run.smithi047.stdout:Adding key to root@localhost authorized_keys... 2024-03-21T11:12:23.078 INFO:teuthology.orchestra.run.smithi047.stdout:Adding host smithi047... 2024-03-21T11:12:25.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:24 smithi047 ceph-mon[31351]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi047", "addr": "172.21.15.47", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:12:26.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:25 smithi047 ceph-mon[31351]: Deploying cephadm binary to smithi047 2024-03-21T11:12:31.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Added host 'smithi047' with addr '172.21.15.47' 2024-03-21T11:12:31.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.640+0000 7f9d50800700 1 Processor -- start 2024-03-21T11:12:31.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.641+0000 7f9d50800700 1 -- start start 2024-03-21T11:12:31.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.641+0000 7f9d50800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d48108560 0x7f9d4810a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:31.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.641+0000 7f9d50800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d48075900 con 0x7f9d48108560 2024-03-21T11:12:31.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.642+0000 7f9d4de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d48108560 0x7f9d4810a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:31.356 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.642+0000 7f9d4de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d48108560 0x7f9d4810a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43784/0 (socket says 172.21.15.47:43784) 2024-03-21T11:12:31.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.642+0000 7f9d4de00700 1 -- 172.21.15.47:0/621324130 learned_addr learned my addr 172.21.15.47:0/621324130 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:31.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.642+0000 7f9d4de00700 1 -- 172.21.15.47:0/621324130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4810ae90 con 0x7f9d48108560 2024-03-21T11:12:31.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.643+0000 7f9d4de00700 1 --2- 172.21.15.47:0/621324130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d48108560 0x7f9d4810a950 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0098a0 tx=0x7f9d3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383abbbecf0dbf30 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:31.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.643+0000 7f9d4ca00700 1 -- 172.21.15.47:0/621324130 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d3c004160 con 0x7f9d48108560 2024-03-21T11:12:31.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.643+0000 7f9d4ca00700 1 -- 172.21.15.47:0/621324130 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d3c004300 con 0x7f9d48108560 2024-03-21T11:12:31.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.643+0000 7f9d4ca00700 1 -- 172.21.15.47:0/621324130 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d3c011b50 con 0x7f9d48108560 2024-03-21T11:12:31.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.644+0000 7f9d50800700 1 -- 172.21.15.47:0/621324130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d48108560 msgr2=0x7f9d4810a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:31.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.644+0000 7f9d50800700 1 --2- 172.21.15.47:0/621324130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d48108560 0x7f9d4810a950 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0098a0 tx=0x7f9d3c009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:31.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.644+0000 7f9d50800700 1 -- 172.21.15.47:0/621324130 shutdown_connections 2024-03-21T11:12:31.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.644+0000 7f9d50800700 1 --2- 172.21.15.47:0/621324130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d48108560 0x7f9d4810a950 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:31.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.644+0000 7f9d50800700 1 -- 172.21.15.47:0/621324130 >> 172.21.15.47:0/621324130 conn(0x7f9d48101fb0 msgr2=0x7f9d481043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:31.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.644+0000 7f9d50800700 1 -- 172.21.15.47:0/621324130 shutdown_connections 2024-03-21T11:12:31.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.644+0000 7f9d50800700 1 -- 172.21.15.47:0/621324130 wait complete. 2024-03-21T11:12:31.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.645+0000 7f9d50800700 1 Processor -- start 2024-03-21T11:12:31.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.645+0000 7f9d50800700 1 -- start start 2024-03-21T11:12:31.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.646+0000 7f9d50800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d48108560 0x7f9d4819dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:31.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.646+0000 7f9d50800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4810bec0 con 0x7f9d48108560 2024-03-21T11:12:31.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.646+0000 7f9d4de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d48108560 0x7f9d4819dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:31.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.646+0000 7f9d4de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d48108560 0x7f9d4819dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43786/0 (socket says 172.21.15.47:43786) 2024-03-21T11:12:31.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.646+0000 7f9d4de00700 1 -- 172.21.15.47:0/3464598772 learned_addr learned my addr 172.21.15.47:0/3464598772 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:31.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.646+0000 7f9d4de00700 1 -- 172.21.15.47:0/3464598772 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4819e270 con 0x7f9d48108560 2024-03-21T11:12:31.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.647+0000 7f9d4de00700 1 --2- 172.21.15.47:0/3464598772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d48108560 0x7f9d4819dd30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009fb0 tx=0x7f9d3c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:31.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.647+0000 7f9d42a00700 1 -- 172.21.15.47:0/3464598772 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d3c0038b0 con 0x7f9d48108560 2024-03-21T11:12:31.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.647+0000 7f9d42a00700 1 -- 172.21.15.47:0/3464598772 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d3c011e50 con 0x7f9d48108560 2024-03-21T11:12:31.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.647+0000 7f9d42a00700 1 -- 172.21.15.47:0/3464598772 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d3c019420 con 0x7f9d48108560 2024-03-21T11:12:31.358 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.647+0000 7f9d50800700 1 -- 172.21.15.47:0/3464598772 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4819e500 con 0x7f9d48108560 2024-03-21T11:12:31.359 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.647+0000 7f9d50800700 1 -- 172.21.15.47:0/3464598772 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d481a11f0 con 0x7f9d48108560 2024-03-21T11:12:31.359 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.648+0000 7f9d42a00700 1 -- 172.21.15.47:0/3464598772 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f9d3c003a50 con 0x7f9d48108560 2024-03-21T11:12:31.359 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.649+0000 7f9d50800700 1 -- 172.21.15.47:0/3464598772 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d480637c0 con 0x7f9d48108560 2024-03-21T11:12:31.359 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.649+0000 7f9d42a00700 1 --2- 172.21.15.47:0/3464598772 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f9d30039180 0x7f9d3003b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:31.359 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.649+0000 7f9d42a00700 1 -- 172.21.15.47:0/3464598772 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9d3c04d3a0 con 0x7f9d48108560 2024-03-21T11:12:31.359 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.649+0000 7f9d4d400700 1 --2- 172.21.15.47:0/3464598772 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f9d30039180 0x7f9d3003b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:31.359 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.650+0000 7f9d4d400700 1 --2- 172.21.15.47:0/3464598772 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f9d30039180 0x7f9d3003b640 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9d38006fd0 tx=0x7f9d38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:31.359 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.655+0000 7f9d42a00700 1 -- 172.21.15.47:0/3464598772 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9d3c0172b0 con 0x7f9d48108560 2024-03-21T11:12:31.359 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:23.885+0000 7f9d50800700 1 -- 172.21.15.47:0/3464598772 --> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi047", "addr": "172.21.15.47", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d48103190 con 0x7f9d30039180 2024-03-21T11:12:31.359 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:30.584+0000 7f9d42a00700 1 -- 172.21.15.47:0/3464598772 <== mgr.14120 v2:172.21.15.47:6800/185270192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f9d48103190 con 0x7f9d30039180 2024-03-21T11:12:31.359 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:30.588+0000 7f9d50800700 1 -- 172.21.15.47:0/3464598772 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f9d30039180 msgr2=0x7f9d3003b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:31.359 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:30.588+0000 7f9d50800700 1 --2- 172.21.15.47:0/3464598772 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f9d30039180 0x7f9d3003b640 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9d38006fd0 tx=0x7f9d38006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:31.359 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:30.588+0000 7f9d50800700 1 -- 172.21.15.47:0/3464598772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d48108560 msgr2=0x7f9d4819dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:31.360 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:30.588+0000 7f9d50800700 1 --2- 172.21.15.47:0/3464598772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d48108560 0x7f9d4819dd30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009fb0 tx=0x7f9d3c011bf0 comp rx=0 tx=0).stop 2024-03-21T11:12:31.360 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:30.588+0000 7f9d50800700 1 -- 172.21.15.47:0/3464598772 shutdown_connections 2024-03-21T11:12:31.360 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:30.588+0000 7f9d50800700 1 --2- 172.21.15.47:0/3464598772 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f9d30039180 0x7f9d3003b640 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:31.360 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:30.588+0000 7f9d50800700 1 --2- 172.21.15.47:0/3464598772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d48108560 0x7f9d4819dd30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:31.360 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:30.588+0000 7f9d50800700 1 -- 172.21.15.47:0/3464598772 >> 172.21.15.47:0/3464598772 conn(0x7f9d48101fb0 msgr2=0x7f9d48102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:31.360 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:30.589+0000 7f9d50800700 1 -- 172.21.15.47:0/3464598772 shutdown_connections 2024-03-21T11:12:31.360 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:30.589+0000 7f9d50800700 1 -- 172.21.15.47:0/3464598772 wait complete. 2024-03-21T11:12:31.360 INFO:teuthology.orchestra.run.smithi047.stdout:Deploying unmanaged mon service... 2024-03-21T11:12:31.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:31 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' 2024-03-21T11:12:31.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:31 smithi047 ceph-mon[31351]: Added host smithi047 2024-03-21T11:12:31.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:31 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:12:33.552 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-21T11:12:33.552 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.034+0000 7f5c3a000700 1 Processor -- start 2024-03-21T11:12:33.552 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.034+0000 7f5c3a000700 1 -- start start 2024-03-21T11:12:33.552 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.035+0000 7f5c3a000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c341085a0 0x7f5c3410a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:33.552 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.035+0000 7f5c3a000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c340758c0 con 0x7f5c341085a0 2024-03-21T11:12:33.552 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.035+0000 7f5c33400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c341085a0 0x7f5c3410a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:33.552 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.036+0000 7f5c33400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c341085a0 0x7f5c3410a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57626/0 (socket says 172.21.15.47:57626) 2024-03-21T11:12:33.553 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.036+0000 7f5c33400700 1 -- 172.21.15.47:0/4074559905 learned_addr learned my addr 172.21.15.47:0/4074559905 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:33.553 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.036+0000 7f5c33400700 1 -- 172.21.15.47:0/4074559905 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3410aed0 con 0x7f5c341085a0 2024-03-21T11:12:33.553 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.036+0000 7f5c33400700 1 --2- 172.21.15.47:0/4074559905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c341085a0 0x7f5c3410a990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c240098a0 tx=0x7f5c24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5681fc2385c7cfe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:33.553 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.037+0000 7f5c32000700 1 -- 172.21.15.47:0/4074559905 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c24004160 con 0x7f5c341085a0 2024-03-21T11:12:33.553 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.037+0000 7f5c32000700 1 -- 172.21.15.47:0/4074559905 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c24004300 con 0x7f5c341085a0 2024-03-21T11:12:33.553 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.037+0000 7f5c32000700 1 -- 172.21.15.47:0/4074559905 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c24011b50 con 0x7f5c341085a0 2024-03-21T11:12:33.553 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.037+0000 7f5c3a000700 1 -- 172.21.15.47:0/4074559905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c341085a0 msgr2=0x7f5c3410a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:33.553 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.037+0000 7f5c3a000700 1 --2- 172.21.15.47:0/4074559905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c341085a0 0x7f5c3410a990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c240098a0 tx=0x7f5c24009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:33.553 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.038+0000 7f5c3a000700 1 -- 172.21.15.47:0/4074559905 shutdown_connections 2024-03-21T11:12:33.553 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.038+0000 7f5c3a000700 1 --2- 172.21.15.47:0/4074559905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c341085a0 0x7f5c3410a990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:33.553 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.038+0000 7f5c3a000700 1 -- 172.21.15.47:0/4074559905 >> 172.21.15.47:0/4074559905 conn(0x7f5c34101fb0 msgr2=0x7f5c341043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:33.553 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.038+0000 7f5c3a000700 1 -- 172.21.15.47:0/4074559905 shutdown_connections 2024-03-21T11:12:33.553 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.038+0000 7f5c3a000700 1 -- 172.21.15.47:0/4074559905 wait complete. 2024-03-21T11:12:33.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.039+0000 7f5c3a000700 1 Processor -- start 2024-03-21T11:12:33.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.039+0000 7f5c3a000700 1 -- start start 2024-03-21T11:12:33.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.040+0000 7f5c3a000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c341085a0 0x7f5c3419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:33.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.040+0000 7f5c3a000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3410bef0 con 0x7f5c341085a0 2024-03-21T11:12:33.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.040+0000 7f5c33400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c341085a0 0x7f5c3419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:33.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.040+0000 7f5c33400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c341085a0 0x7f5c3419bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57634/0 (socket says 172.21.15.47:57634) 2024-03-21T11:12:33.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.040+0000 7f5c33400700 1 -- 172.21.15.47:0/570655359 learned_addr learned my addr 172.21.15.47:0/570655359 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:33.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.040+0000 7f5c33400700 1 -- 172.21.15.47:0/570655359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3419c080 con 0x7f5c341085a0 2024-03-21T11:12:33.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.041+0000 7f5c33400700 1 --2- 172.21.15.47:0/570655359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c341085a0 0x7f5c3419bb40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5c24011b60 tx=0x7f5c24011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:33.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.041+0000 7f5c2be00700 1 -- 172.21.15.47:0/570655359 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c24003970 con 0x7f5c341085a0 2024-03-21T11:12:33.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.041+0000 7f5c2be00700 1 -- 172.21.15.47:0/570655359 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c24003b10 con 0x7f5c341085a0 2024-03-21T11:12:33.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.041+0000 7f5c3a000700 1 -- 172.21.15.47:0/570655359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3419c310 con 0x7f5c341085a0 2024-03-21T11:12:33.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.041+0000 7f5c2be00700 1 -- 172.21.15.47:0/570655359 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c24024610 con 0x7f5c341085a0 2024-03-21T11:12:33.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.041+0000 7f5c3a000700 1 -- 172.21.15.47:0/570655359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3419f000 con 0x7f5c341085a0 2024-03-21T11:12:33.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.043+0000 7f5c2be00700 1 -- 172.21.15.47:0/570655359 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f5c2401a9f0 con 0x7f5c341085a0 2024-03-21T11:12:33.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.042+0000 7f5c3a000700 1 -- 172.21.15.47:0/570655359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c340637c0 con 0x7f5c341085a0 2024-03-21T11:12:33.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.043+0000 7f5c2be00700 1 --2- 172.21.15.47:0/570655359 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f5c18039180 0x7f5c1803b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:33.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.043+0000 7f5c2be00700 1 -- 172.21.15.47:0/570655359 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5c24011e00 con 0x7f5c341085a0 2024-03-21T11:12:33.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.043+0000 7f5c32a00700 1 --2- 172.21.15.47:0/570655359 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f5c18039180 0x7f5c1803b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:33.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.044+0000 7f5c32a00700 1 --2- 172.21.15.47:0/570655359 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f5c18039180 0x7f5c1803b640 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5c20006fd0 tx=0x7f5c20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:33.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.049+0000 7f5c2be00700 1 -- 172.21.15.47:0/570655359 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5c24019dd0 con 0x7f5c341085a0 2024-03-21T11:12:33.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.312+0000 7f5c3a000700 1 -- 172.21.15.47:0/570655359 --> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5c341031d0 con 0x7f5c18039180 2024-03-21T11:12:33.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.322+0000 7f5c2be00700 1 -- 172.21.15.47:0/570655359 <== mgr.14120 v2:172.21.15.47:6800/185270192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5c341031d0 con 0x7f5c18039180 2024-03-21T11:12:33.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.325+0000 7f5c3a000700 1 -- 172.21.15.47:0/570655359 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f5c18039180 msgr2=0x7f5c1803b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:33.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.325+0000 7f5c3a000700 1 --2- 172.21.15.47:0/570655359 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f5c18039180 0x7f5c1803b640 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5c20006fd0 tx=0x7f5c20006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:33.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.325+0000 7f5c3a000700 1 -- 172.21.15.47:0/570655359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c341085a0 msgr2=0x7f5c3419bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:33.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.325+0000 7f5c3a000700 1 --2- 172.21.15.47:0/570655359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c341085a0 0x7f5c3419bb40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5c24011b60 tx=0x7f5c24011c40 comp rx=0 tx=0).stop 2024-03-21T11:12:33.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.325+0000 7f5c3a000700 1 -- 172.21.15.47:0/570655359 shutdown_connections 2024-03-21T11:12:33.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.325+0000 7f5c3a000700 1 --2- 172.21.15.47:0/570655359 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f5c18039180 0x7f5c1803b640 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:33.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.325+0000 7f5c3a000700 1 --2- 172.21.15.47:0/570655359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c341085a0 0x7f5c3419bb40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:33.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.325+0000 7f5c3a000700 1 -- 172.21.15.47:0/570655359 >> 172.21.15.47:0/570655359 conn(0x7f5c34101fb0 msgr2=0x7f5c34102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:33.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.325+0000 7f5c3a000700 1 -- 172.21.15.47:0/570655359 shutdown_connections 2024-03-21T11:12:33.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:32.325+0000 7f5c3a000700 1 -- 172.21.15.47:0/570655359 wait complete. 2024-03-21T11:12:33.556 INFO:teuthology.orchestra.run.smithi047.stdout:Deploying unmanaged mgr service... 2024-03-21T11:12:33.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:33 smithi047 ceph-mon[31351]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:12:33.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:33 smithi047 ceph-mon[31351]: Saving service mon spec with placement count:5 2024-03-21T11:12:33.705 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:33 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' 2024-03-21T11:12:35.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-21T11:12:35.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.439+0000 7f2397200700 1 Processor -- start 2024-03-21T11:12:35.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.440+0000 7f2397200700 1 -- start start 2024-03-21T11:12:35.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.440+0000 7f2397200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23901085f0 0x7f239010a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:35.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.440+0000 7f2397200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239007be90 con 0x7f23901085f0 2024-03-21T11:12:35.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.440+0000 7f238fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23901085f0 0x7f239010a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:35.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.441+0000 7f238fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23901085f0 0x7f239010a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57650/0 (socket says 172.21.15.47:57650) 2024-03-21T11:12:35.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.441+0000 7f238fe00700 1 -- 172.21.15.47:0/1948384218 learned_addr learned my addr 172.21.15.47:0/1948384218 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:35.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.441+0000 7f238fe00700 1 -- 172.21.15.47:0/1948384218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239010af20 con 0x7f23901085f0 2024-03-21T11:12:35.384 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.441+0000 7f238fe00700 1 --2- 172.21.15.47:0/1948384218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23901085f0 0x7f239010a9e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f23800098a0 tx=0x7f2380009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd12778d1a914a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:35.385 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.442+0000 7f238f400700 1 -- 172.21.15.47:0/1948384218 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2380004160 con 0x7f23901085f0 2024-03-21T11:12:35.385 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.442+0000 7f238f400700 1 -- 172.21.15.47:0/1948384218 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2380004300 con 0x7f23901085f0 2024-03-21T11:12:35.385 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.442+0000 7f238f400700 1 -- 172.21.15.47:0/1948384218 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2380011b50 con 0x7f23901085f0 2024-03-21T11:12:35.385 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.442+0000 7f2397200700 1 -- 172.21.15.47:0/1948384218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23901085f0 msgr2=0x7f239010a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:35.385 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.442+0000 7f2397200700 1 --2- 172.21.15.47:0/1948384218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23901085f0 0x7f239010a9e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f23800098a0 tx=0x7f2380009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:35.385 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.442+0000 7f2397200700 1 -- 172.21.15.47:0/1948384218 shutdown_connections 2024-03-21T11:12:35.385 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.442+0000 7f2397200700 1 --2- 172.21.15.47:0/1948384218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23901085f0 0x7f239010a9e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:35.385 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.442+0000 7f2397200700 1 -- 172.21.15.47:0/1948384218 >> 172.21.15.47:0/1948384218 conn(0x7f2390102000 msgr2=0x7f2390104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:35.385 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.443+0000 7f2397200700 1 -- 172.21.15.47:0/1948384218 shutdown_connections 2024-03-21T11:12:35.385 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.443+0000 7f2397200700 1 -- 172.21.15.47:0/1948384218 wait complete. 2024-03-21T11:12:35.385 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.444+0000 7f2397200700 1 Processor -- start 2024-03-21T11:12:35.385 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.444+0000 7f2397200700 1 -- start start 2024-03-21T11:12:35.385 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.444+0000 7f2397200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23901085f0 0x7f2390107e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:35.386 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.444+0000 7f2397200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239010bf40 con 0x7f23901085f0 2024-03-21T11:12:35.386 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.445+0000 7f238fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23901085f0 0x7f2390107e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:35.386 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.445+0000 7f238fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23901085f0 0x7f2390107e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57658/0 (socket says 172.21.15.47:57658) 2024-03-21T11:12:35.386 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.445+0000 7f238fe00700 1 -- 172.21.15.47:0/2626658875 learned_addr learned my addr 172.21.15.47:0/2626658875 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:35.386 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.445+0000 7f238fe00700 1 -- 172.21.15.47:0/2626658875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23901083b0 con 0x7f23901085f0 2024-03-21T11:12:35.386 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.445+0000 7f238fe00700 1 --2- 172.21.15.47:0/2626658875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23901085f0 0x7f2390107e70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2380011b60 tx=0x7f2380011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:35.386 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.446+0000 7f238d600700 1 -- 172.21.15.47:0/2626658875 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2380003970 con 0x7f23901085f0 2024-03-21T11:12:35.386 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.446+0000 7f238d600700 1 -- 172.21.15.47:0/2626658875 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2380003b10 con 0x7f23901085f0 2024-03-21T11:12:35.386 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.446+0000 7f2397200700 1 -- 172.21.15.47:0/2626658875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23901065a0 con 0x7f23901085f0 2024-03-21T11:12:35.386 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.446+0000 7f238d600700 1 -- 172.21.15.47:0/2626658875 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2380024610 con 0x7f23901085f0 2024-03-21T11:12:35.386 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.446+0000 7f2397200700 1 -- 172.21.15.47:0/2626658875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2390106ae0 con 0x7f23901085f0 2024-03-21T11:12:35.386 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.447+0000 7f238d600700 1 -- 172.21.15.47:0/2626658875 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f238001a430 con 0x7f23901085f0 2024-03-21T11:12:35.386 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.447+0000 7f2397200700 1 -- 172.21.15.47:0/2626658875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23900637c0 con 0x7f23901085f0 2024-03-21T11:12:35.387 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.447+0000 7f238d600700 1 --2- 172.21.15.47:0/2626658875 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f23740391d0 0x7f237403b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:35.387 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.447+0000 7f238d600700 1 -- 172.21.15.47:0/2626658875 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2380011e00 con 0x7f23901085f0 2024-03-21T11:12:35.387 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.448+0000 7f2387e00700 1 --2- 172.21.15.47:0/2626658875 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f23740391d0 0x7f237403b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:35.387 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.448+0000 7f2387e00700 1 --2- 172.21.15.47:0/2626658875 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f23740391d0 0x7f237403b690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2378006fd0 tx=0x7f2378006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:35.387 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.454+0000 7f238d600700 1 -- 172.21.15.47:0/2626658875 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f238004cd00 con 0x7f23901085f0 2024-03-21T11:12:35.387 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.665+0000 7f238d600700 1 -- 172.21.15.47:0/2626658875 <== mon.0 v2:172.21.15.47:3300/0 7 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f238002ca40 con 0x7f23901085f0 2024-03-21T11:12:35.387 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.731+0000 7f2397200700 1 -- 172.21.15.47:0/2626658875 --> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2390103310 con 0x7f23740391d0 2024-03-21T11:12:35.387 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.737+0000 7f238d600700 1 -- 172.21.15.47:0/2626658875 <== mgr.14120 v2:172.21.15.47:6800/185270192 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2390103310 con 0x7f23740391d0 2024-03-21T11:12:35.387 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.739+0000 7f2397200700 1 -- 172.21.15.47:0/2626658875 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f23740391d0 msgr2=0x7f237403b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:35.387 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.739+0000 7f2397200700 1 --2- 172.21.15.47:0/2626658875 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f23740391d0 0x7f237403b690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2378006fd0 tx=0x7f2378006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:35.387 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.739+0000 7f2397200700 1 -- 172.21.15.47:0/2626658875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23901085f0 msgr2=0x7f2390107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:35.387 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.739+0000 7f2397200700 1 --2- 172.21.15.47:0/2626658875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23901085f0 0x7f2390107e70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2380011b60 tx=0x7f2380011c40 comp rx=0 tx=0).stop 2024-03-21T11:12:35.388 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.739+0000 7f2397200700 1 -- 172.21.15.47:0/2626658875 shutdown_connections 2024-03-21T11:12:35.388 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.739+0000 7f2397200700 1 --2- 172.21.15.47:0/2626658875 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f23740391d0 0x7f237403b690 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:35.388 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.739+0000 7f2397200700 1 --2- 172.21.15.47:0/2626658875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23901085f0 0x7f2390107e70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:35.388 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.739+0000 7f2397200700 1 -- 172.21.15.47:0/2626658875 >> 172.21.15.47:0/2626658875 conn(0x7f2390102000 msgr2=0x7f2390102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:35.388 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.740+0000 7f2397200700 1 -- 172.21.15.47:0/2626658875 shutdown_connections 2024-03-21T11:12:35.388 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:34.740+0000 7f2397200700 1 -- 172.21.15.47:0/2626658875 wait complete. 2024-03-21T11:12:35.924 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:35 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' 2024-03-21T11:12:35.924 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:35 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' 2024-03-21T11:12:35.924 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:35 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' 2024-03-21T11:12:36.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:36 smithi047 ceph-mon[31351]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:12:36.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:36 smithi047 ceph-mon[31351]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:12:36.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:36 smithi047 ceph-mon[31351]: Saving service mgr spec with placement count:2 2024-03-21T11:12:36.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:36 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3558243898' entity='client.admin' 2024-03-21T11:12:36.888 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.006+0000 7f1160c00700 1 Processor -- start 2024-03-21T11:12:36.888 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.006+0000 7f1160c00700 1 -- start start 2024-03-21T11:12:36.888 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.007+0000 7f1160c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f115c107af0 0x7f115c107f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:36.888 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.007+0000 7f1160c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c1084e0 con 0x7f115c107af0 2024-03-21T11:12:36.889 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.007+0000 7f1159e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f115c107af0 0x7f115c107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:36.889 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.007+0000 7f1159e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f115c107af0 0x7f115c107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57668/0 (socket says 172.21.15.47:57668) 2024-03-21T11:12:36.890 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.007+0000 7f1159e00700 1 -- 172.21.15.47:0/285959802 learned_addr learned my addr 172.21.15.47:0/285959802 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:36.890 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.008+0000 7f1159e00700 1 -- 172.21.15.47:0/285959802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c108cf0 con 0x7f115c107af0 2024-03-21T11:12:36.890 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.008+0000 7f1159e00700 1 --2- 172.21.15.47:0/285959802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f115c107af0 0x7f115c107f10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f114c0098a0 tx=0x7f114c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60f47adf21b8a3c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:36.890 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.008+0000 7f1158a00700 1 -- 172.21.15.47:0/285959802 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f114c004160 con 0x7f115c107af0 2024-03-21T11:12:36.890 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.008+0000 7f1158a00700 1 -- 172.21.15.47:0/285959802 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f114c004300 con 0x7f115c107af0 2024-03-21T11:12:36.891 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.008+0000 7f1158a00700 1 -- 172.21.15.47:0/285959802 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f114c011b20 con 0x7f115c107af0 2024-03-21T11:12:36.891 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.009+0000 7f1160c00700 1 -- 172.21.15.47:0/285959802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f115c107af0 msgr2=0x7f115c107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:36.891 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.009+0000 7f1160c00700 1 --2- 172.21.15.47:0/285959802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f115c107af0 0x7f115c107f10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f114c0098a0 tx=0x7f114c009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:36.891 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.009+0000 7f1160c00700 1 -- 172.21.15.47:0/285959802 shutdown_connections 2024-03-21T11:12:36.891 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.009+0000 7f1160c00700 1 --2- 172.21.15.47:0/285959802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f115c107af0 0x7f115c107f10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:36.891 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.009+0000 7f1160c00700 1 -- 172.21.15.47:0/285959802 >> 172.21.15.47:0/285959802 conn(0x7f115c103070 msgr2=0x7f115c1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:36.891 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.010+0000 7f1160c00700 1 -- 172.21.15.47:0/285959802 shutdown_connections 2024-03-21T11:12:36.893 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.010+0000 7f1160c00700 1 -- 172.21.15.47:0/285959802 wait complete. 2024-03-21T11:12:36.893 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.011+0000 7f1160c00700 1 Processor -- start 2024-03-21T11:12:36.893 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.011+0000 7f1160c00700 1 -- start start 2024-03-21T11:12:36.893 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.011+0000 7f1160c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f115c107af0 0x7f115c07ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:36.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.011+0000 7f1160c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c10c0a0 con 0x7f115c107af0 2024-03-21T11:12:36.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.011+0000 7f1159e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f115c107af0 0x7f115c07ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:36.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.011+0000 7f1159e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f115c107af0 0x7f115c07ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57678/0 (socket says 172.21.15.47:57678) 2024-03-21T11:12:36.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.011+0000 7f1159e00700 1 -- 172.21.15.47:0/3558243898 learned_addr learned my addr 172.21.15.47:0/3558243898 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:36.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.012+0000 7f1159e00700 1 -- 172.21.15.47:0/3558243898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c07cfc0 con 0x7f115c107af0 2024-03-21T11:12:36.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.012+0000 7f1159e00700 1 --2- 172.21.15.47:0/3558243898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f115c107af0 0x7f115c07ca80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f114c0117e0 tx=0x7f114c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:36.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.013+0000 7f1152a00700 1 -- 172.21.15.47:0/3558243898 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f114c003bf0 con 0x7f115c107af0 2024-03-21T11:12:36.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.013+0000 7f1152a00700 1 -- 172.21.15.47:0/3558243898 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f114c01a910 con 0x7f115c107af0 2024-03-21T11:12:36.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.013+0000 7f1160c00700 1 -- 172.21.15.47:0/3558243898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115c07b1b0 con 0x7f115c107af0 2024-03-21T11:12:36.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.013+0000 7f1152a00700 1 -- 172.21.15.47:0/3558243898 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f114c024970 con 0x7f115c107af0 2024-03-21T11:12:36.896 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.013+0000 7f1160c00700 1 -- 172.21.15.47:0/3558243898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c07b670 con 0x7f115c107af0 2024-03-21T11:12:36.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.014+0000 7f1152a00700 1 -- 172.21.15.47:0/3558243898 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f114c022020 con 0x7f115c107af0 2024-03-21T11:12:36.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.014+0000 7f1160c00700 1 -- 172.21.15.47:0/3558243898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115c0637c0 con 0x7f115c107af0 2024-03-21T11:12:36.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.014+0000 7f1152a00700 1 --2- 172.21.15.47:0/3558243898 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f11400391d0 0x7f114003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:36.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.015+0000 7f1152a00700 1 -- 172.21.15.47:0/3558243898 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f114c04d5c0 con 0x7f115c107af0 2024-03-21T11:12:36.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.015+0000 7f1159400700 1 --2- 172.21.15.47:0/3558243898 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f11400391d0 0x7f114003b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:36.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.016+0000 7f1159400700 1 --2- 172.21.15.47:0/3558243898 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f11400391d0 0x7f114003b690 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1148006fd0 tx=0x7f1148006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:36.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.020+0000 7f1152a00700 1 -- 172.21.15.47:0/3558243898 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f114c00f2e0 con 0x7f115c107af0 2024-03-21T11:12:36.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.249+0000 7f1160c00700 1 -- 172.21.15.47:0/3558243898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f115c07bf10 con 0x7f115c107af0 2024-03-21T11:12:36.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.253+0000 7f1152a00700 1 -- 172.21.15.47:0/3558243898 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f114c04d320 con 0x7f115c107af0 2024-03-21T11:12:36.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.256+0000 7f1160c00700 1 -- 172.21.15.47:0/3558243898 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f11400391d0 msgr2=0x7f114003b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:36.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.256+0000 7f1160c00700 1 --2- 172.21.15.47:0/3558243898 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f11400391d0 0x7f114003b690 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1148006fd0 tx=0x7f1148006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:36.897 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.256+0000 7f1160c00700 1 -- 172.21.15.47:0/3558243898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f115c107af0 msgr2=0x7f115c07ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:36.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.256+0000 7f1160c00700 1 --2- 172.21.15.47:0/3558243898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f115c107af0 0x7f115c07ca80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f114c0117e0 tx=0x7f114c011ba0 comp rx=0 tx=0).stop 2024-03-21T11:12:36.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.256+0000 7f1160c00700 1 -- 172.21.15.47:0/3558243898 shutdown_connections 2024-03-21T11:12:36.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.256+0000 7f1160c00700 1 --2- 172.21.15.47:0/3558243898 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f11400391d0 0x7f114003b690 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:36.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.256+0000 7f1160c00700 1 --2- 172.21.15.47:0/3558243898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f115c107af0 0x7f115c07ca80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:36.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.256+0000 7f1160c00700 1 -- 172.21.15.47:0/3558243898 >> 172.21.15.47:0/3558243898 conn(0x7f115c103070 msgr2=0x7f115c103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:36.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.257+0000 7f1160c00700 1 -- 172.21.15.47:0/3558243898 shutdown_connections 2024-03-21T11:12:36.898 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:36.257+0000 7f1160c00700 1 -- 172.21.15.47:0/3558243898 wait complete. 2024-03-21T11:12:38.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:38 smithi047 ceph-mon[31351]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:12:38.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:38 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1806184289' entity='client.admin' 2024-03-21T11:12:38.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:38 smithi047 ceph-mon[31351]: from='mgr.14120 172.21.15.47:0/4229010486' entity='mgr.x' 2024-03-21T11:12:38.719 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.464+0000 7f5b06400700 1 Processor -- start 2024-03-21T11:12:38.720 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.464+0000 7f5b06400700 1 -- start start 2024-03-21T11:12:38.720 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.465+0000 7f5b06400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b001069f0 0x7f5b00106e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:38.720 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.465+0000 7f5b06400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b001073e0 con 0x7f5b001069f0 2024-03-21T11:12:38.720 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.465+0000 7f5aff400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b001069f0 0x7f5b00106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:38.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.465+0000 7f5aff400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b001069f0 0x7f5b00106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57682/0 (socket says 172.21.15.47:57682) 2024-03-21T11:12:38.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.465+0000 7f5aff400700 1 -- 172.21.15.47:0/4222059915 learned_addr learned my addr 172.21.15.47:0/4222059915 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:38.721 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.466+0000 7f5aff400700 1 -- 172.21.15.47:0/4222059915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b00107bf0 con 0x7f5b001069f0 2024-03-21T11:12:38.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.466+0000 7f5aff400700 1 --2- 172.21.15.47:0/4222059915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b001069f0 0x7f5b00106e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5af00098a0 tx=0x7f5af0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec77e6bdb5c2008 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:38.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.466+0000 7f5afe000700 1 -- 172.21.15.47:0/4222059915 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5af0004160 con 0x7f5b001069f0 2024-03-21T11:12:38.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.466+0000 7f5afe000700 1 -- 172.21.15.47:0/4222059915 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5af0004300 con 0x7f5b001069f0 2024-03-21T11:12:38.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.466+0000 7f5afe000700 1 -- 172.21.15.47:0/4222059915 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5af0011b20 con 0x7f5b001069f0 2024-03-21T11:12:38.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.467+0000 7f5b06400700 1 -- 172.21.15.47:0/4222059915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b001069f0 msgr2=0x7f5b00106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:38.722 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.467+0000 7f5b06400700 1 --2- 172.21.15.47:0/4222059915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b001069f0 0x7f5b00106e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5af00098a0 tx=0x7f5af0009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:38.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.467+0000 7f5b06400700 1 -- 172.21.15.47:0/4222059915 shutdown_connections 2024-03-21T11:12:38.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.467+0000 7f5b06400700 1 --2- 172.21.15.47:0/4222059915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b001069f0 0x7f5b00106e10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:38.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.467+0000 7f5b06400700 1 -- 172.21.15.47:0/4222059915 >> 172.21.15.47:0/4222059915 conn(0x7f5b00101fb0 msgr2=0x7f5b001043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:38.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.467+0000 7f5b06400700 1 -- 172.21.15.47:0/4222059915 shutdown_connections 2024-03-21T11:12:38.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.467+0000 7f5b06400700 1 -- 172.21.15.47:0/4222059915 wait complete. 2024-03-21T11:12:38.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.469+0000 7f5b06400700 1 Processor -- start 2024-03-21T11:12:38.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.469+0000 7f5b06400700 1 -- start start 2024-03-21T11:12:38.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.469+0000 7f5b06400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b001069f0 0x7f5b00075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:38.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.469+0000 7f5b06400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0010afa0 con 0x7f5b001069f0 2024-03-21T11:12:38.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.470+0000 7f5aff400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b001069f0 0x7f5b00075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:38.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.470+0000 7f5aff400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b001069f0 0x7f5b00075500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57686/0 (socket says 172.21.15.47:57686) 2024-03-21T11:12:38.727 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.470+0000 7f5aff400700 1 -- 172.21.15.47:0/1806184289 learned_addr learned my addr 172.21.15.47:0/1806184289 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:38.728 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.470+0000 7f5aff400700 1 -- 172.21.15.47:0/1806184289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0019e800 con 0x7f5b001069f0 2024-03-21T11:12:38.728 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.470+0000 7f5aff400700 1 --2- 172.21.15.47:0/1806184289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b001069f0 0x7f5b00075500 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5af0000c00 tx=0x7f5af0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:38.728 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.471+0000 7f5af7e00700 1 -- 172.21.15.47:0/1806184289 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5af0003b60 con 0x7f5b001069f0 2024-03-21T11:12:38.728 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.471+0000 7f5af7e00700 1 -- 172.21.15.47:0/1806184289 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5af001a910 con 0x7f5b001069f0 2024-03-21T11:12:38.728 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.471+0000 7f5af7e00700 1 -- 172.21.15.47:0/1806184289 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5af0024930 con 0x7f5b001069f0 2024-03-21T11:12:38.728 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.471+0000 7f5b06400700 1 -- 172.21.15.47:0/1806184289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0019e9d0 con 0x7f5b001069f0 2024-03-21T11:12:38.728 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.471+0000 7f5b06400700 1 -- 172.21.15.47:0/1806184289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0019ed70 con 0x7f5b001069f0 2024-03-21T11:12:38.728 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.472+0000 7f5af7e00700 1 -- 172.21.15.47:0/1806184289 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f5af0022020 con 0x7f5b001069f0 2024-03-21T11:12:38.728 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.472+0000 7f5b06400700 1 -- 172.21.15.47:0/1806184289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b000637c0 con 0x7f5b001069f0 2024-03-21T11:12:38.728 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.473+0000 7f5af7e00700 1 --2- 172.21.15.47:0/1806184289 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f5ae40391d0 0x7f5ae403b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:38.728 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.473+0000 7f5af7e00700 1 -- 172.21.15.47:0/1806184289 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5af004d400 con 0x7f5b001069f0 2024-03-21T11:12:38.728 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.473+0000 7f5afea00700 1 --2- 172.21.15.47:0/1806184289 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f5ae40391d0 0x7f5ae403b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:38.729 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.474+0000 7f5afea00700 1 --2- 172.21.15.47:0/1806184289 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f5ae40391d0 0x7f5ae403b690 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5aec006fd0 tx=0x7f5aec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:38.729 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.478+0000 7f5af7e00700 1 -- 172.21.15.47:0/1806184289 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5af004cd50 con 0x7f5b001069f0 2024-03-21T11:12:38.729 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.715+0000 7f5b06400700 1 -- 172.21.15.47:0/1806184289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f5b0019f5d0 con 0x7f5b001069f0 2024-03-21T11:12:38.729 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.718+0000 7f5af7e00700 1 -- 172.21.15.47:0/1806184289 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f5af001ab70 con 0x7f5b001069f0 2024-03-21T11:12:38.729 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.722+0000 7f5b06400700 1 -- 172.21.15.47:0/1806184289 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f5ae40391d0 msgr2=0x7f5ae403b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:38.729 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.722+0000 7f5b06400700 1 --2- 172.21.15.47:0/1806184289 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f5ae40391d0 0x7f5ae403b690 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5aec006fd0 tx=0x7f5aec006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:38.729 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.722+0000 7f5b06400700 1 -- 172.21.15.47:0/1806184289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b001069f0 msgr2=0x7f5b00075500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:38.729 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.722+0000 7f5b06400700 1 --2- 172.21.15.47:0/1806184289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b001069f0 0x7f5b00075500 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5af0000c00 tx=0x7f5af0011c10 comp rx=0 tx=0).stop 2024-03-21T11:12:38.729 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.722+0000 7f5b06400700 1 -- 172.21.15.47:0/1806184289 shutdown_connections 2024-03-21T11:12:38.729 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.722+0000 7f5b06400700 1 --2- 172.21.15.47:0/1806184289 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f5ae40391d0 0x7f5ae403b690 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:38.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.722+0000 7f5b06400700 1 --2- 172.21.15.47:0/1806184289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b001069f0 0x7f5b00075500 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:38.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.722+0000 7f5b06400700 1 -- 172.21.15.47:0/1806184289 >> 172.21.15.47:0/1806184289 conn(0x7f5b00101fb0 msgr2=0x7f5b00102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:38.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.722+0000 7f5b06400700 1 -- 172.21.15.47:0/1806184289 shutdown_connections 2024-03-21T11:12:38.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:37.723+0000 7f5b06400700 1 -- 172.21.15.47:0/1806184289 wait complete. 2024-03-21T11:12:38.730 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling the dashboard module... 2024-03-21T11:12:40.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:39 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1092664858' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-21T11:12:40.064 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:39 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: ignoring --setuser ceph since I am not root 2024-03-21T11:12:40.064 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:39 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: ignoring --setgroup ceph since I am not root 2024-03-21T11:12:40.415 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.323+0000 7f69bba00700 1 Processor -- start 2024-03-21T11:12:40.415 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.323+0000 7f69bba00700 1 -- start start 2024-03-21T11:12:40.415 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.323+0000 7f69bba00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69b4107b30 0x7f69b4107f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:40.415 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.323+0000 7f69bba00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b4108520 con 0x7f69b4107b30 2024-03-21T11:12:40.415 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.323+0000 7f69b9000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69b4107b30 0x7f69b4107f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:40.415 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.323+0000 7f69b9000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69b4107b30 0x7f69b4107f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41562/0 (socket says 172.21.15.47:41562) 2024-03-21T11:12:40.416 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.323+0000 7f69b9000700 1 -- 172.21.15.47:0/3282043289 learned_addr learned my addr 172.21.15.47:0/3282043289 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:40.416 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.324+0000 7f69b9000700 1 -- 172.21.15.47:0/3282043289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b4108d30 con 0x7f69b4107b30 2024-03-21T11:12:40.416 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.324+0000 7f69b9000700 1 --2- 172.21.15.47:0/3282043289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69b4107b30 0x7f69b4107f50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f69a40098a0 tx=0x7f69a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338aa32b5793253e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:40.416 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.324+0000 7f69b3e00700 1 -- 172.21.15.47:0/3282043289 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69a4004160 con 0x7f69b4107b30 2024-03-21T11:12:40.416 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.324+0000 7f69b3e00700 1 -- 172.21.15.47:0/3282043289 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69a4004300 con 0x7f69b4107b30 2024-03-21T11:12:40.416 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.324+0000 7f69b3e00700 1 -- 172.21.15.47:0/3282043289 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69a4011b20 con 0x7f69b4107b30 2024-03-21T11:12:40.416 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.325+0000 7f69bba00700 1 -- 172.21.15.47:0/3282043289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69b4107b30 msgr2=0x7f69b4107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:40.416 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.325+0000 7f69bba00700 1 --2- 172.21.15.47:0/3282043289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69b4107b30 0x7f69b4107f50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f69a40098a0 tx=0x7f69a4009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:40.416 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.325+0000 7f69bba00700 1 -- 172.21.15.47:0/3282043289 shutdown_connections 2024-03-21T11:12:40.416 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.325+0000 7f69bba00700 1 --2- 172.21.15.47:0/3282043289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69b4107b30 0x7f69b4107f50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:40.416 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.325+0000 7f69bba00700 1 -- 172.21.15.47:0/3282043289 >> 172.21.15.47:0/3282043289 conn(0x7f69b4103150 msgr2=0x7f69b4105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:40.416 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.325+0000 7f69bba00700 1 -- 172.21.15.47:0/3282043289 shutdown_connections 2024-03-21T11:12:40.416 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.325+0000 7f69bba00700 1 -- 172.21.15.47:0/3282043289 wait complete. 2024-03-21T11:12:40.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.326+0000 7f69bba00700 1 Processor -- start 2024-03-21T11:12:40.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.326+0000 7f69bba00700 1 -- start start 2024-03-21T11:12:40.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.327+0000 7f69bba00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69b4107b30 0x7f69b419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:40.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.327+0000 7f69bba00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b410c0e0 con 0x7f69b4107b30 2024-03-21T11:12:40.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.327+0000 7f69b9000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69b4107b30 0x7f69b419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:40.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.327+0000 7f69b9000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69b4107b30 0x7f69b419dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41568/0 (socket says 172.21.15.47:41568) 2024-03-21T11:12:40.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.327+0000 7f69b9000700 1 -- 172.21.15.47:0/1092664858 learned_addr learned my addr 172.21.15.47:0/1092664858 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:40.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.327+0000 7f69b9000700 1 -- 172.21.15.47:0/1092664858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b419e260 con 0x7f69b4107b30 2024-03-21T11:12:40.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.328+0000 7f69b9000700 1 --2- 172.21.15.47:0/1092664858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69b4107b30 0x7f69b419dd20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f69a40116a0 tx=0x7f69a4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:40.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.328+0000 7f69b2000700 1 -- 172.21.15.47:0/1092664858 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69a4003a40 con 0x7f69b4107b30 2024-03-21T11:12:40.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.328+0000 7f69b2000700 1 -- 172.21.15.47:0/1092664858 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69a4003be0 con 0x7f69b4107b30 2024-03-21T11:12:40.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.328+0000 7f69b2000700 1 -- 172.21.15.47:0/1092664858 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69a401a720 con 0x7f69b4107b30 2024-03-21T11:12:40.417 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.328+0000 7f69bba00700 1 -- 172.21.15.47:0/1092664858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b419e4f0 con 0x7f69b4107b30 2024-03-21T11:12:40.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.328+0000 7f69bba00700 1 -- 172.21.15.47:0/1092664858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b419e950 con 0x7f69b4107b30 2024-03-21T11:12:40.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.329+0000 7f69b2000700 1 -- 172.21.15.47:0/1092664858 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f69a4024aa0 con 0x7f69b4107b30 2024-03-21T11:12:40.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.329+0000 7f69bba00700 1 -- 172.21.15.47:0/1092664858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b40637c0 con 0x7f69b4107b30 2024-03-21T11:12:40.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.330+0000 7f69b2000700 1 --2- 172.21.15.47:0/1092664858 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f6998039180 0x7f699803b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:40.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.330+0000 7f69b2000700 1 -- 172.21.15.47:0/1092664858 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f69a404d770 con 0x7f69b4107b30 2024-03-21T11:12:40.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.330+0000 7f69abe00700 1 --2- 172.21.15.47:0/1092664858 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f6998039180 0x7f699803b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:40.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.331+0000 7f69abe00700 1 --2- 172.21.15.47:0/1092664858 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f6998039180 0x7f699803b640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f699c006fd0 tx=0x7f699c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:40.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.335+0000 7f69b2000700 1 -- 172.21.15.47:0/1092664858 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f69a4011e20 con 0x7f69b4107b30 2024-03-21T11:12:40.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.626+0000 7f69bba00700 1 -- 172.21.15.47:0/1092664858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f69b41a13b0 con 0x7f69b4107b30 2024-03-21T11:12:40.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.674+0000 7f69b2000700 1 -- 172.21.15.47:0/1092664858 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f69a400f2e0 con 0x7f69b4107b30 2024-03-21T11:12:40.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.679+0000 7f69b2000700 1 -- 172.21.15.47:0/1092664858 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f69a402d770 con 0x7f69b4107b30 2024-03-21T11:12:40.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.679+0000 7f69bba00700 1 -- 172.21.15.47:0/1092664858 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f6998039180 msgr2=0x7f699803b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:40.418 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.679+0000 7f69bba00700 1 --2- 172.21.15.47:0/1092664858 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f6998039180 0x7f699803b640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f699c006fd0 tx=0x7f699c006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:40.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.679+0000 7f69bba00700 1 -- 172.21.15.47:0/1092664858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69b4107b30 msgr2=0x7f69b419dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:40.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.679+0000 7f69bba00700 1 --2- 172.21.15.47:0/1092664858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69b4107b30 0x7f69b419dd20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f69a40116a0 tx=0x7f69a4011ba0 comp rx=0 tx=0).stop 2024-03-21T11:12:40.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.679+0000 7f69bba00700 1 -- 172.21.15.47:0/1092664858 shutdown_connections 2024-03-21T11:12:40.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.679+0000 7f69bba00700 1 --2- 172.21.15.47:0/1092664858 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f6998039180 0x7f699803b640 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:40.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.680+0000 7f69bba00700 1 --2- 172.21.15.47:0/1092664858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69b4107b30 0x7f69b419dd20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:40.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.680+0000 7f69bba00700 1 -- 172.21.15.47:0/1092664858 >> 172.21.15.47:0/1092664858 conn(0x7f69b4103150 msgr2=0x7f69b4103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:40.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.680+0000 7f69bba00700 1 -- 172.21.15.47:0/1092664858 shutdown_connections 2024-03-21T11:12:40.419 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:39.680+0000 7f69bba00700 1 -- 172.21.15.47:0/1092664858 wait complete. 2024-03-21T11:12:40.674 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:40 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:40.540+0000 7f77d7e06200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T11:12:40.674 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:40 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:40.673+0000 7f77d7e06200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T11:12:40.970 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:40 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1092664858' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-21T11:12:40.970 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:40 smithi047 ceph-mon[31351]: mgrmap e9: x(active, since 24s) 2024-03-21T11:12:41.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:41 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1831677334' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T11:12:41.813 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:41 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:41.511+0000 7f77d7e06200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T11:12:41.813 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:41 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:41.638+0000 7f77d7e06200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T11:12:42.047 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T11:12:42.047 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-21T11:12:42.047 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T11:12:42.047 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-21T11:12:42.047 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T11:12:42.048 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T11:12:42.048 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.045+0000 7f956f200700 1 Processor -- start 2024-03-21T11:12:42.048 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.045+0000 7f956f200700 1 -- start start 2024-03-21T11:12:42.048 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.045+0000 7f956f200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9570107960 0x7f9570107d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:42.048 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.045+0000 7f956f200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9570108350 con 0x7f9570107960 2024-03-21T11:12:42.048 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.046+0000 7f956de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9570107960 0x7f9570107d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:42.048 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.046+0000 7f956de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9570107960 0x7f9570107d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41592/0 (socket says 172.21.15.47:41592) 2024-03-21T11:12:42.048 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.046+0000 7f956de00700 1 -- 172.21.15.47:0/1625663048 learned_addr learned my addr 172.21.15.47:0/1625663048 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:42.048 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.046+0000 7f956de00700 1 -- 172.21.15.47:0/1625663048 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9570108b10 con 0x7f9570107960 2024-03-21T11:12:42.048 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.047+0000 7f956de00700 1 --2- 172.21.15.47:0/1625663048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9570107960 0x7f9570107d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f95600098a0 tx=0x7f9560009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edda40e9f05247c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:42.048 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.047+0000 7f956ca00700 1 -- 172.21.15.47:0/1625663048 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9560004160 con 0x7f9570107960 2024-03-21T11:12:42.049 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.047+0000 7f956ca00700 1 -- 172.21.15.47:0/1625663048 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9560004300 con 0x7f9570107960 2024-03-21T11:12:42.049 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.047+0000 7f956ca00700 1 -- 172.21.15.47:0/1625663048 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9560011b20 con 0x7f9570107960 2024-03-21T11:12:42.049 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.048+0000 7f956f200700 1 -- 172.21.15.47:0/1625663048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9570107960 msgr2=0x7f9570107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:42.049 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.048+0000 7f956f200700 1 --2- 172.21.15.47:0/1625663048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9570107960 0x7f9570107d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f95600098a0 tx=0x7f9560009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:42.049 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.048+0000 7f956f200700 1 -- 172.21.15.47:0/1625663048 shutdown_connections 2024-03-21T11:12:42.049 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.048+0000 7f956f200700 1 --2- 172.21.15.47:0/1625663048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9570107960 0x7f9570107d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:42.049 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.048+0000 7f956f200700 1 -- 172.21.15.47:0/1625663048 >> 172.21.15.47:0/1625663048 conn(0x7f9570102f00 msgr2=0x7f9570105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:42.049 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.048+0000 7f956f200700 1 -- 172.21.15.47:0/1625663048 shutdown_connections 2024-03-21T11:12:42.049 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.048+0000 7f956f200700 1 -- 172.21.15.47:0/1625663048 wait complete. 2024-03-21T11:12:42.049 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.049+0000 7f956f200700 1 Processor -- start 2024-03-21T11:12:42.049 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.049+0000 7f956f200700 1 -- start start 2024-03-21T11:12:42.049 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.050+0000 7f956f200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9570107960 0x7f957019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:42.049 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.050+0000 7f956f200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957007aff0 con 0x7f9570107960 2024-03-21T11:12:42.050 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.050+0000 7f956de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9570107960 0x7f957019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:42.050 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.050+0000 7f956de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9570107960 0x7f957019db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41608/0 (socket says 172.21.15.47:41608) 2024-03-21T11:12:42.050 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.050+0000 7f956de00700 1 -- 172.21.15.47:0/1831677334 learned_addr learned my addr 172.21.15.47:0/1831677334 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:42.050 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.050+0000 7f956de00700 1 -- 172.21.15.47:0/1831677334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957019e0d0 con 0x7f9570107960 2024-03-21T11:12:42.050 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.051+0000 7f956de00700 1 --2- 172.21.15.47:0/1831677334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9570107960 0x7f957019db90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9560000c00 tx=0x7f9560011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:42.050 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.051+0000 7f9566a00700 1 -- 172.21.15.47:0/1831677334 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9560003b60 con 0x7f9570107960 2024-03-21T11:12:42.050 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.051+0000 7f9566a00700 1 -- 172.21.15.47:0/1831677334 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f956001a910 con 0x7f9570107960 2024-03-21T11:12:42.050 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.051+0000 7f956f200700 1 -- 172.21.15.47:0/1831677334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957019e360 con 0x7f9570107960 2024-03-21T11:12:42.050 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.051+0000 7f9566a00700 1 -- 172.21.15.47:0/1831677334 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9560024930 con 0x7f9570107960 2024-03-21T11:12:42.050 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.051+0000 7f956f200700 1 -- 172.21.15.47:0/1831677334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957019e7c0 con 0x7f9570107960 2024-03-21T11:12:42.050 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.052+0000 7f956f200700 1 -- 172.21.15.47:0/1831677334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95700637c0 con 0x7f9570107960 2024-03-21T11:12:42.050 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.053+0000 7f9566a00700 1 -- 172.21.15.47:0/1831677334 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f9560022020 con 0x7f9570107960 2024-03-21T11:12:42.051 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.053+0000 7f9566a00700 1 --2- 172.21.15.47:0/1831677334 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f95540391d0 0x7f955403b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:42.051 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.053+0000 7f9566a00700 1 -- 172.21.15.47:0/1831677334 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f956004d470 con 0x7f9570107960 2024-03-21T11:12:42.051 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.053+0000 7f956d400700 1 -- 172.21.15.47:0/1831677334 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f95540391d0 msgr2=0x7f955403b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/185270192 2024-03-21T11:12:42.051 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.053+0000 7f956d400700 1 --2- 172.21.15.47:0/1831677334 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f95540391d0 0x7f955403b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T11:12:42.051 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.059+0000 7f9566a00700 1 -- 172.21.15.47:0/1831677334 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9560017ca0 con 0x7f9570107960 2024-03-21T11:12:42.051 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.254+0000 7f956d400700 1 -- 172.21.15.47:0/1831677334 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f95540391d0 msgr2=0x7f955403b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/185270192 2024-03-21T11:12:42.051 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.254+0000 7f956d400700 1 --2- 172.21.15.47:0/1831677334 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f95540391d0 0x7f955403b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T11:12:42.051 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.338+0000 7f956f200700 1 -- 172.21.15.47:0/1831677334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9570062da0 con 0x7f9570107960 2024-03-21T11:12:42.051 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.338+0000 7f9566a00700 1 -- 172.21.15.47:0/1831677334 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f956004cdc0 con 0x7f9570107960 2024-03-21T11:12:42.051 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.341+0000 7f956f200700 1 -- 172.21.15.47:0/1831677334 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f95540391d0 msgr2=0x7f955403b690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:12:42.051 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.341+0000 7f956f200700 1 --2- 172.21.15.47:0/1831677334 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f95540391d0 0x7f955403b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:42.051 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.341+0000 7f956f200700 1 -- 172.21.15.47:0/1831677334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9570107960 msgr2=0x7f957019db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:42.051 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.341+0000 7f956f200700 1 --2- 172.21.15.47:0/1831677334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9570107960 0x7f957019db90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9560000c00 tx=0x7f9560011c10 comp rx=0 tx=0).stop 2024-03-21T11:12:42.052 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.341+0000 7f956f200700 1 -- 172.21.15.47:0/1831677334 shutdown_connections 2024-03-21T11:12:42.052 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.341+0000 7f956f200700 1 --2- 172.21.15.47:0/1831677334 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f95540391d0 0x7f955403b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:42.052 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.341+0000 7f956f200700 1 --2- 172.21.15.47:0/1831677334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9570107960 0x7f957019db90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:42.052 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.341+0000 7f956f200700 1 -- 172.21.15.47:0/1831677334 >> 172.21.15.47:0/1831677334 conn(0x7f9570102f00 msgr2=0x7f9570105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:42.052 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.341+0000 7f956f200700 1 -- 172.21.15.47:0/1831677334 shutdown_connections 2024-03-21T11:12:42.052 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:41.342+0000 7f956f200700 1 -- 172.21.15.47:0/1831677334 wait complete. 2024-03-21T11:12:42.052 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for the mgr to restart... 2024-03-21T11:12:42.052 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr epoch 9... 2024-03-21T11:12:42.191 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:41 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:41.824+0000 7f77d7e06200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T11:12:42.191 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:41 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:41.952+0000 7f77d7e06200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T11:12:42.191 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:42 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:42.069+0000 7f77d7e06200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T11:12:42.460 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:42 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:42.420+0000 7f77d7e06200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T11:12:42.812 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:42 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:42.567+0000 7f77d7e06200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T11:12:42.812 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:42 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:42.684+0000 7f77d7e06200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T11:12:42.813 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:42 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:42.812+0000 7f77d7e06200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T11:12:43.063 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:43 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:43.035+0000 7f77d7e06200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T11:12:44.813 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:44 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:44.353+0000 7f77d7e06200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T11:12:44.813 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:44 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:44.531+0000 7f77d7e06200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T11:12:45.313 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:44 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:44.960+0000 7f77d7e06200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T11:12:45.313 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:45 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:45.081+0000 7f77d7e06200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T11:12:45.600 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:45 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:45.345+0000 7f77d7e06200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T11:12:45.988 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:45 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:45.599+0000 7f77d7e06200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T11:12:46.312 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:45 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:45.987+0000 7f77d7e06200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T11:12:46.563 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:46 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:46.311+0000 7f77d7e06200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T11:12:47.063 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:46 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:46.623+0000 7f77d7e06200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T11:12:48.063 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:47 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:47.689+0000 7f77d7e06200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T11:12:48.813 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:48 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:48.407+0000 7f77d7e06200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T11:12:48.813 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:48 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:48.541+0000 7f77d7e06200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T11:12:49.313 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:12:48 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:12:48.832+0000 7f77d7e06200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T11:12:49.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:48 smithi047 ceph-mon[31351]: Active manager daemon x restarted 2024-03-21T11:12:49.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:48 smithi047 ceph-mon[31351]: Activating manager daemon x 2024-03-21T11:12:49.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:48 smithi047 ceph-mon[31351]: osdmap e3: 0 total, 0 up, 0 in 2024-03-21T11:12:49.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:48 smithi047 ceph-mon[31351]: mgrmap e10: x(active, starting, since 0.00744165s) 2024-03-21T11:12:49.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:12:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T11:12:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T11:12:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T11:12:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T11:12:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:48 smithi047 ceph-mon[31351]: Manager daemon x is now available 2024-03-21T11:12:50.200 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:12:50.200 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T11:12:50.200 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T11:12:50.200 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:49 smithi047 ceph-mon[31351]: [21/Mar/2024:11:12:49] ENGINE Bus STARTING 2024-03-21T11:12:50.200 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:49 smithi047 ceph-mon[31351]: [21/Mar/2024:11:12:49] ENGINE Serving on https://172.21.15.47:7150 2024-03-21T11:12:50.201 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:49 smithi047 ceph-mon[31351]: [21/Mar/2024:11:12:49] ENGINE Serving on http://172.21.15.47:8765 2024-03-21T11:12:50.201 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:49 smithi047 ceph-mon[31351]: [21/Mar/2024:11:12:49] ENGINE Bus STARTED 2024-03-21T11:12:50.201 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:49 smithi047 ceph-mon[31351]: mgrmap e11: x(active, since 1.00956s) 2024-03-21T11:12:50.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T11:12:50.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-21T11:12:50.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T11:12:50.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T11:12:50.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.620+0000 7f01aa800700 1 Processor -- start 2024-03-21T11:12:50.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.620+0000 7f01aa800700 1 -- start start 2024-03-21T11:12:50.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.620+0000 7f01aa800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a41083e0 0x7f01a410a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:50.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.620+0000 7f01aa800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a40758a0 con 0x7f01a41083e0 2024-03-21T11:12:50.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.621+0000 7f01a9400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a41083e0 0x7f01a410a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:50.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.621+0000 7f01a9400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a41083e0 0x7f01a410a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41618/0 (socket says 172.21.15.47:41618) 2024-03-21T11:12:50.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.621+0000 7f01a9400700 1 -- 172.21.15.47:0/786680192 learned_addr learned my addr 172.21.15.47:0/786680192 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:50.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.621+0000 7f01a9400700 1 -- 172.21.15.47:0/786680192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a410ad10 con 0x7f01a41083e0 2024-03-21T11:12:50.595 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.621+0000 7f01a9400700 1 --2- 172.21.15.47:0/786680192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a41083e0 0x7f01a410a7d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f01980098a0 tx=0x7f0198009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c4c099d16ece65 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:50.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.622+0000 7f01a3e00700 1 -- 172.21.15.47:0/786680192 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0198004160 con 0x7f01a41083e0 2024-03-21T11:12:50.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.622+0000 7f01a3e00700 1 -- 172.21.15.47:0/786680192 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0198004300 con 0x7f01a41083e0 2024-03-21T11:12:50.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.622+0000 7f01a3e00700 1 -- 172.21.15.47:0/786680192 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0198011b20 con 0x7f01a41083e0 2024-03-21T11:12:50.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.623+0000 7f01aa800700 1 -- 172.21.15.47:0/786680192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a41083e0 msgr2=0x7f01a410a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:50.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.623+0000 7f01aa800700 1 --2- 172.21.15.47:0/786680192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a41083e0 0x7f01a410a7d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f01980098a0 tx=0x7f0198009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:50.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.623+0000 7f01aa800700 1 -- 172.21.15.47:0/786680192 shutdown_connections 2024-03-21T11:12:50.596 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.623+0000 7f01aa800700 1 --2- 172.21.15.47:0/786680192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a41083e0 0x7f01a410a7d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:50.600 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.623+0000 7f01aa800700 1 -- 172.21.15.47:0/786680192 >> 172.21.15.47:0/786680192 conn(0x7f01a4101e30 msgr2=0x7f01a4104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:50.600 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.623+0000 7f01aa800700 1 -- 172.21.15.47:0/786680192 shutdown_connections 2024-03-21T11:12:50.600 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.623+0000 7f01aa800700 1 -- 172.21.15.47:0/786680192 wait complete. 2024-03-21T11:12:50.600 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.624+0000 7f01aa800700 1 Processor -- start 2024-03-21T11:12:50.600 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.624+0000 7f01aa800700 1 -- start start 2024-03-21T11:12:50.600 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.624+0000 7f01aa800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a41083e0 0x7f01a419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:50.601 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.624+0000 7f01aa800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a410bd20 con 0x7f01a41083e0 2024-03-21T11:12:50.601 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.625+0000 7f01a9400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a41083e0 0x7f01a419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:50.601 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.625+0000 7f01a9400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a41083e0 0x7f01a419dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41626/0 (socket says 172.21.15.47:41626) 2024-03-21T11:12:50.601 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.625+0000 7f01a9400700 1 -- 172.21.15.47:0/3449998555 learned_addr learned my addr 172.21.15.47:0/3449998555 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:50.601 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.625+0000 7f01a9400700 1 -- 172.21.15.47:0/3449998555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a419e010 con 0x7f01a41083e0 2024-03-21T11:12:50.601 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.625+0000 7f01a9400700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a41083e0 0x7f01a419dad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f01980117e0 tx=0x7f0198011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:50.601 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.626+0000 7f01a2000700 1 -- 172.21.15.47:0/3449998555 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0198003ba0 con 0x7f01a41083e0 2024-03-21T11:12:50.601 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.626+0000 7f01a2000700 1 -- 172.21.15.47:0/3449998555 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f019801a910 con 0x7f01a41083e0 2024-03-21T11:12:50.601 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.626+0000 7f01aa800700 1 -- 172.21.15.47:0/3449998555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a419e2a0 con 0x7f01a41083e0 2024-03-21T11:12:50.601 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.626+0000 7f01a2000700 1 -- 172.21.15.47:0/3449998555 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01980248f0 con 0x7f01a41083e0 2024-03-21T11:12:50.601 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.626+0000 7f01aa800700 1 -- 172.21.15.47:0/3449998555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a419e700 con 0x7f01a41083e0 2024-03-21T11:12:50.601 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.627+0000 7f01a2000700 1 -- 172.21.15.47:0/3449998555 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f0198022020 con 0x7f01a41083e0 2024-03-21T11:12:50.601 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.627+0000 7f01a2000700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 0x7f018c03b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:50.601 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.627+0000 7f01a2000700 1 -- 172.21.15.47:0/3449998555 --> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f018c03bda0 con 0x7f018c0391d0 2024-03-21T11:12:50.602 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.627+0000 7f01a2000700 1 -- 172.21.15.47:0/3449998555 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f019804dad0 con 0x7f01a41083e0 2024-03-21T11:12:50.602 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.627+0000 7f01a8a00700 1 -- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 msgr2=0x7f018c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/185270192 2024-03-21T11:12:50.602 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.627+0000 7f01a8a00700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 0x7f018c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T11:12:50.602 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.828+0000 7f01a8a00700 1 -- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 msgr2=0x7f018c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/185270192 2024-03-21T11:12:50.602 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:42.828+0000 7f01a8a00700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 0x7f018c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T11:12:50.602 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:43.229+0000 7f01a8a00700 1 -- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 msgr2=0x7f018c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/185270192 2024-03-21T11:12:50.602 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:43.229+0000 7f01a8a00700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 0x7f018c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T11:12:50.602 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:44.030+0000 7f01a8a00700 1 -- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 msgr2=0x7f018c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/185270192 2024-03-21T11:12:50.602 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:44.030+0000 7f01a8a00700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 0x7f018c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T11:12:50.602 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:45.632+0000 7f01a8a00700 1 -- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 msgr2=0x7f018c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/185270192 2024-03-21T11:12:50.602 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:45.632+0000 7f01a8a00700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 0x7f018c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T11:12:50.602 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:48.835+0000 7f01a8a00700 1 -- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 msgr2=0x7f018c03b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/185270192 2024-03-21T11:12:50.602 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:48.835+0000 7f01a8a00700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 0x7f018c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-21T11:12:50.603 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:48.840+0000 7f01a2000700 1 -- 172.21.15.47:0/3449998555 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mgrmap(e 10) v1 ==== 46074+0+0 (secure 0 0 0) 0x7f019802d3f0 con 0x7f01a41083e0 2024-03-21T11:12:50.603 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:48.841+0000 7f01a2000700 1 -- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 msgr2=0x7f018c03b690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:12:50.603 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:48.841+0000 7f01a2000700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 0x7f018c03b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:50.605 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.842+0000 7f01a2000700 1 -- 172.21.15.47:0/3449998555 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 11) v1 ==== 46201+0+0 (secure 0 0 0) 0x7f019804fd80 con 0x7f01a41083e0 2024-03-21T11:12:50.605 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.843+0000 7f01a2000700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f018c03cc50 0x7f018c03f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:50.605 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.843+0000 7f01a2000700 1 -- 172.21.15.47:0/3449998555 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0198001d30 con 0x7f018c03cc50 2024-03-21T11:12:50.605 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.844+0000 7f01a8a00700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f018c03cc50 0x7f018c03f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:50.605 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.846+0000 7f01a8a00700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f018c03cc50 0x7f018c03f040 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0190003a10 tx=0x7f01900092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:50.606 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.847+0000 7f01a2000700 1 -- 172.21.15.47:0/3449998555 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f0198001d30 con 0x7f018c03cc50 2024-03-21T11:12:50.606 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.854+0000 7f01aa800700 1 -- 172.21.15.47:0/3449998555 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f01a40637c0 con 0x7f018c03cc50 2024-03-21T11:12:50.606 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.855+0000 7f01a2000700 1 -- 172.21.15.47:0/3449998555 <== mgr.14152 v2:172.21.15.47:6800/2057671763 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f01a40637c0 con 0x7f018c03cc50 2024-03-21T11:12:50.606 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.855+0000 7f01aa800700 1 -- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f018c03cc50 msgr2=0x7f018c03f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:50.606 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.855+0000 7f01aa800700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f018c03cc50 0x7f018c03f040 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0190003a10 tx=0x7f01900092b0 comp rx=0 tx=0).stop 2024-03-21T11:12:50.606 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.855+0000 7f01aa800700 1 -- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a41083e0 msgr2=0x7f01a419dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:50.606 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.855+0000 7f01aa800700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a41083e0 0x7f01a419dad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f01980117e0 tx=0x7f0198011ba0 comp rx=0 tx=0).stop 2024-03-21T11:12:50.606 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.856+0000 7f01aa800700 1 -- 172.21.15.47:0/3449998555 shutdown_connections 2024-03-21T11:12:50.606 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.856+0000 7f01aa800700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f018c03cc50 0x7f018c03f040 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:50.606 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.856+0000 7f01aa800700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a41083e0 0x7f01a419dad0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:50.606 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.856+0000 7f01aa800700 1 --2- 172.21.15.47:0/3449998555 >> [v2:172.21.15.47:6800/185270192,v1:172.21.15.47:6801/185270192] conn(0x7f018c0391d0 0x7f018c03b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:50.606 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.856+0000 7f01aa800700 1 -- 172.21.15.47:0/3449998555 >> 172.21.15.47:0/3449998555 conn(0x7f01a4101e30 msgr2=0x7f01a4102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:50.606 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.856+0000 7f01aa800700 1 -- 172.21.15.47:0/3449998555 shutdown_connections 2024-03-21T11:12:50.607 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:49.856+0000 7f01aa800700 1 -- 172.21.15.47:0/3449998555 wait complete. 2024-03-21T11:12:50.607 INFO:teuthology.orchestra.run.smithi047.stdout:mgr epoch 9 is available 2024-03-21T11:12:50.607 INFO:teuthology.orchestra.run.smithi047.stdout:Generating a dashboard self-signed certificate... 2024-03-21T11:12:51.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:50 smithi047 ceph-mon[31351]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T11:12:51.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:50 smithi047 ceph-mon[31351]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T11:12:52.179 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:51 smithi047 ceph-mon[31351]: mgrmap e12: x(active, since 2s) 2024-03-21T11:12:52.179 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:51 smithi047 ceph-mon[31351]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:12:52.179 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:12:52.179 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:12:52.179 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:12:52.518 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-21T11:12:52.518 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.329+0000 7f1421000700 1 Processor -- start 2024-03-21T11:12:52.518 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.329+0000 7f1421000700 1 -- start start 2024-03-21T11:12:52.518 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.329+0000 7f1421000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141c0822f0 0x7f141c110d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:52.519 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.329+0000 7f1421000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c1112d0 con 0x7f141c0822f0 2024-03-21T11:12:52.519 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.329+0000 7f1419e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141c0822f0 0x7f141c110d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:52.519 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.329+0000 7f1419e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141c0822f0 0x7f141c110d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60824/0 (socket says 172.21.15.47:60824) 2024-03-21T11:12:52.519 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.329+0000 7f1419e00700 1 -- 172.21.15.47:0/1471139011 learned_addr learned my addr 172.21.15.47:0/1471139011 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:52.519 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.330+0000 7f1419e00700 1 -- 172.21.15.47:0/1471139011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c111450 con 0x7f141c0822f0 2024-03-21T11:12:52.519 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.330+0000 7f1419e00700 1 --2- 172.21.15.47:0/1471139011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141c0822f0 0x7f141c110d00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f140c0098a0 tx=0x7f140c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23113a69f3dfc7e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:52.519 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.331+0000 7f1419400700 1 -- 172.21.15.47:0/1471139011 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f140c004160 con 0x7f141c0822f0 2024-03-21T11:12:52.519 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.331+0000 7f1419400700 1 -- 172.21.15.47:0/1471139011 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f140c004300 con 0x7f141c0822f0 2024-03-21T11:12:52.519 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.331+0000 7f1419400700 1 -- 172.21.15.47:0/1471139011 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f140c011b20 con 0x7f141c0822f0 2024-03-21T11:12:52.519 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.331+0000 7f1421000700 1 -- 172.21.15.47:0/1471139011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141c0822f0 msgr2=0x7f141c110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:52.519 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.331+0000 7f1421000700 1 --2- 172.21.15.47:0/1471139011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141c0822f0 0x7f141c110d00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f140c0098a0 tx=0x7f140c009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:52.519 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.331+0000 7f1421000700 1 -- 172.21.15.47:0/1471139011 shutdown_connections 2024-03-21T11:12:52.520 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.331+0000 7f1421000700 1 --2- 172.21.15.47:0/1471139011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141c0822f0 0x7f141c110d00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:52.520 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.331+0000 7f1421000700 1 -- 172.21.15.47:0/1471139011 >> 172.21.15.47:0/1471139011 conn(0x7f141c07d220 msgr2=0x7f141c07f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:52.520 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.331+0000 7f1421000700 1 -- 172.21.15.47:0/1471139011 shutdown_connections 2024-03-21T11:12:52.520 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.332+0000 7f1421000700 1 -- 172.21.15.47:0/1471139011 wait complete. 2024-03-21T11:12:52.520 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.333+0000 7f1421000700 1 Processor -- start 2024-03-21T11:12:52.520 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.333+0000 7f1421000700 1 -- start start 2024-03-21T11:12:52.520 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.333+0000 7f1421000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141c0822f0 0x7f141c1a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:52.520 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.333+0000 7f1421000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c112520 con 0x7f141c0822f0 2024-03-21T11:12:52.520 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.333+0000 7f1419e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141c0822f0 0x7f141c1a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:52.520 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.334+0000 7f1419e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141c0822f0 0x7f141c1a20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60840/0 (socket says 172.21.15.47:60840) 2024-03-21T11:12:52.520 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.334+0000 7f1419e00700 1 -- 172.21.15.47:0/3387587481 learned_addr learned my addr 172.21.15.47:0/3387587481 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:52.521 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.334+0000 7f1419e00700 1 -- 172.21.15.47:0/3387587481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c1a2600 con 0x7f141c0822f0 2024-03-21T11:12:52.521 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.334+0000 7f1419e00700 1 --2- 172.21.15.47:0/3387587481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141c0822f0 0x7f141c1a20c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f140c000c00 tx=0x7f140c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:52.521 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.334+0000 7f1412a00700 1 -- 172.21.15.47:0/3387587481 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f140c003b60 con 0x7f141c0822f0 2024-03-21T11:12:52.521 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.334+0000 7f1412a00700 1 -- 172.21.15.47:0/3387587481 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f140c01a910 con 0x7f141c0822f0 2024-03-21T11:12:52.521 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.334+0000 7f1421000700 1 -- 172.21.15.47:0/3387587481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141c1a2890 con 0x7f141c0822f0 2024-03-21T11:12:52.521 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.335+0000 7f1412a00700 1 -- 172.21.15.47:0/3387587481 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f140c024930 con 0x7f141c0822f0 2024-03-21T11:12:52.521 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.335+0000 7f1421000700 1 -- 172.21.15.47:0/3387587481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c1a2cf0 con 0x7f141c0822f0 2024-03-21T11:12:52.521 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.336+0000 7f1421000700 1 -- 172.21.15.47:0/3387587481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141c0637c0 con 0x7f141c0822f0 2024-03-21T11:12:52.521 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.336+0000 7f1412a00700 1 -- 172.21.15.47:0/3387587481 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f140c022020 con 0x7f141c0822f0 2024-03-21T11:12:52.521 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.336+0000 7f1412a00700 1 --2- 172.21.15.47:0/3387587481 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14000391d0 0x7f140003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:52.521 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.336+0000 7f1412a00700 1 -- 172.21.15.47:0/3387587481 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f140c04d5d0 con 0x7f141c0822f0 2024-03-21T11:12:52.521 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.337+0000 7f1413e00700 1 --2- 172.21.15.47:0/3387587481 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14000391d0 0x7f140003b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:52.522 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.337+0000 7f1413e00700 1 --2- 172.21.15.47:0/3387587481 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14000391d0 0x7f140003b690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1404006fd0 tx=0x7f1404006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:52.522 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.342+0000 7f1412a00700 1 -- 172.21.15.47:0/3387587481 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f140c00f2e0 con 0x7f141c0822f0 2024-03-21T11:12:52.522 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.575+0000 7f1421000700 1 -- 172.21.15.47:0/3387587481 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f141c07e490 con 0x7f14000391d0 2024-03-21T11:12:52.522 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.815+0000 7f1412a00700 1 -- 172.21.15.47:0/3387587481 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f141c07e490 con 0x7f14000391d0 2024-03-21T11:12:52.522 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.819+0000 7f1421000700 1 -- 172.21.15.47:0/3387587481 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14000391d0 msgr2=0x7f140003b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:52.522 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.819+0000 7f1421000700 1 --2- 172.21.15.47:0/3387587481 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14000391d0 0x7f140003b690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1404006fd0 tx=0x7f1404006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:52.522 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.819+0000 7f1421000700 1 -- 172.21.15.47:0/3387587481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141c0822f0 msgr2=0x7f141c1a20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:52.522 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.819+0000 7f1421000700 1 --2- 172.21.15.47:0/3387587481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141c0822f0 0x7f141c1a20c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f140c000c00 tx=0x7f140c011c10 comp rx=0 tx=0).stop 2024-03-21T11:12:52.522 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.819+0000 7f1421000700 1 -- 172.21.15.47:0/3387587481 shutdown_connections 2024-03-21T11:12:52.522 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.819+0000 7f1421000700 1 --2- 172.21.15.47:0/3387587481 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14000391d0 0x7f140003b690 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:52.522 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.819+0000 7f1421000700 1 --2- 172.21.15.47:0/3387587481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f141c0822f0 0x7f141c1a20c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:52.522 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.819+0000 7f1421000700 1 -- 172.21.15.47:0/3387587481 >> 172.21.15.47:0/3387587481 conn(0x7f141c07d220 msgr2=0x7f141c07dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:52.523 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.819+0000 7f1421000700 1 -- 172.21.15.47:0/3387587481 shutdown_connections 2024-03-21T11:12:52.523 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:51.819+0000 7f1421000700 1 -- 172.21.15.47:0/3387587481 wait complete. 2024-03-21T11:12:52.523 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial admin user... 2024-03-21T11:12:54.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$igke8O.0YKU5Su5JHQgg5uGilGs5IVRoCwygXwG7tTbBUzlHhxXcm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711019573, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-21T11:12:54.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.115+0000 7f4a8fa00700 1 Processor -- start 2024-03-21T11:12:54.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.116+0000 7f4a8fa00700 1 -- start start 2024-03-21T11:12:54.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.116+0000 7f4a8fa00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a8807df00 0x7f4a8807c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:54.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.116+0000 7f4a8fa00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8807c930 con 0x7f4a8807df00 2024-03-21T11:12:54.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.116+0000 7f4a8d000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a8807df00 0x7f4a8807c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:54.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.117+0000 7f4a8d000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a8807df00 0x7f4a8807c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60856/0 (socket says 172.21.15.47:60856) 2024-03-21T11:12:54.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.117+0000 7f4a8d000700 1 -- 172.21.15.47:0/2851291354 learned_addr learned my addr 172.21.15.47:0/2851291354 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:54.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.117+0000 7f4a8d000700 1 -- 172.21.15.47:0/2851291354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8807cab0 con 0x7f4a8807df00 2024-03-21T11:12:54.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.118+0000 7f4a8d000700 1 --2- 172.21.15.47:0/2851291354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a8807df00 0x7f4a8807c360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0098a0 tx=0x7f4a7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68fb0537cc3c9c78 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:54.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.118+0000 7f4a87400700 1 -- 172.21.15.47:0/2851291354 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a7c004160 con 0x7f4a8807df00 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.118+0000 7f4a87400700 1 -- 172.21.15.47:0/2851291354 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a7c004300 con 0x7f4a8807df00 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.118+0000 7f4a87400700 1 -- 172.21.15.47:0/2851291354 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a7c011ad0 con 0x7f4a8807df00 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.119+0000 7f4a8fa00700 1 -- 172.21.15.47:0/2851291354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a8807df00 msgr2=0x7f4a8807c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.119+0000 7f4a8fa00700 1 --2- 172.21.15.47:0/2851291354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a8807df00 0x7f4a8807c360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0098a0 tx=0x7f4a7c009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.119+0000 7f4a8fa00700 1 -- 172.21.15.47:0/2851291354 shutdown_connections 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.119+0000 7f4a8fa00700 1 --2- 172.21.15.47:0/2851291354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a8807df00 0x7f4a8807c360 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.119+0000 7f4a8fa00700 1 -- 172.21.15.47:0/2851291354 >> 172.21.15.47:0/2851291354 conn(0x7f4a88103150 msgr2=0x7f4a88105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.119+0000 7f4a8fa00700 1 -- 172.21.15.47:0/2851291354 shutdown_connections 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.119+0000 7f4a8fa00700 1 -- 172.21.15.47:0/2851291354 wait complete. 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.120+0000 7f4a8fa00700 1 Processor -- start 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.120+0000 7f4a8fa00700 1 -- start start 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.121+0000 7f4a8fa00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a8807df00 0x7f4a8819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.121+0000 7f4a8fa00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8810e1a0 con 0x7f4a8807df00 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.121+0000 7f4a8d000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a8807df00 0x7f4a8819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:54.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.121+0000 7f4a8d000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a8807df00 0x7f4a8819dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60862/0 (socket says 172.21.15.47:60862) 2024-03-21T11:12:54.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.121+0000 7f4a8d000700 1 -- 172.21.15.47:0/3554148408 learned_addr learned my addr 172.21.15.47:0/3554148408 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:54.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.122+0000 7f4a8d000700 1 -- 172.21.15.47:0/3554148408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8819e240 con 0x7f4a8807df00 2024-03-21T11:12:54.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.122+0000 7f4a8d000700 1 --2- 172.21.15.47:0/3554148408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a8807df00 0x7f4a8819dd00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c000c00 tx=0x7f4a7c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:54.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.122+0000 7f4a85600700 1 -- 172.21.15.47:0/3554148408 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a7c003a70 con 0x7f4a8807df00 2024-03-21T11:12:54.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.122+0000 7f4a85600700 1 -- 172.21.15.47:0/3554148408 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a7c003c10 con 0x7f4a8807df00 2024-03-21T11:12:54.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.122+0000 7f4a85600700 1 -- 172.21.15.47:0/3554148408 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a7c01a6c0 con 0x7f4a8807df00 2024-03-21T11:12:54.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.122+0000 7f4a8fa00700 1 -- 172.21.15.47:0/3554148408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8819e4d0 con 0x7f4a8807df00 2024-03-21T11:12:54.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.122+0000 7f4a8fa00700 1 -- 172.21.15.47:0/3554148408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8819e930 con 0x7f4a8807df00 2024-03-21T11:12:54.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.124+0000 7f4a85600700 1 -- 172.21.15.47:0/3554148408 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f4a7c024ab0 con 0x7f4a8807df00 2024-03-21T11:12:54.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.124+0000 7f4a8fa00700 1 -- 172.21.15.47:0/3554148408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a880637c0 con 0x7f4a8807df00 2024-03-21T11:12:54.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.125+0000 7f4a85600700 1 --2- 172.21.15.47:0/3554148408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4a70039280 0x7f4a7003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:54.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.125+0000 7f4a85600700 1 -- 172.21.15.47:0/3554148408 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4a7c04d6d0 con 0x7f4a8807df00 2024-03-21T11:12:54.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.125+0000 7f4a87e00700 1 --2- 172.21.15.47:0/3554148408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4a70039280 0x7f4a7003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:54.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.126+0000 7f4a87e00700 1 --2- 172.21.15.47:0/3554148408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4a70039280 0x7f4a7003b740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4a74006fd0 tx=0x7f4a74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:54.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.130+0000 7f4a85600700 1 -- 172.21.15.47:0/3554148408 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4a7c00e260 con 0x7f4a8807df00 2024-03-21T11:12:54.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.377+0000 7f4a8fa00700 1 -- 172.21.15.47:0/3554148408 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4a881a12d0 con 0x7f4a70039280 2024-03-21T11:12:54.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.635+0000 7f4a85600700 1 -- 172.21.15.47:0/3554148408 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f4a881a12d0 con 0x7f4a70039280 2024-03-21T11:12:54.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.638+0000 7f4a8fa00700 1 -- 172.21.15.47:0/3554148408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4a70039280 msgr2=0x7f4a7003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:54.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.638+0000 7f4a8fa00700 1 --2- 172.21.15.47:0/3554148408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4a70039280 0x7f4a7003b740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4a74006fd0 tx=0x7f4a74006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:54.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.638+0000 7f4a8fa00700 1 -- 172.21.15.47:0/3554148408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a8807df00 msgr2=0x7f4a8819dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:54.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.638+0000 7f4a8fa00700 1 --2- 172.21.15.47:0/3554148408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a8807df00 0x7f4a8819dd00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c000c00 tx=0x7f4a7c011c10 comp rx=0 tx=0).stop 2024-03-21T11:12:54.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.639+0000 7f4a8fa00700 1 -- 172.21.15.47:0/3554148408 shutdown_connections 2024-03-21T11:12:54.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.639+0000 7f4a8fa00700 1 --2- 172.21.15.47:0/3554148408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4a70039280 0x7f4a7003b740 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:54.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.639+0000 7f4a8fa00700 1 --2- 172.21.15.47:0/3554148408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a8807df00 0x7f4a8819dd00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:54.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.639+0000 7f4a8fa00700 1 -- 172.21.15.47:0/3554148408 >> 172.21.15.47:0/3554148408 conn(0x7f4a88103150 msgr2=0x7f4a88103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:54.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.639+0000 7f4a8fa00700 1 -- 172.21.15.47:0/3554148408 shutdown_connections 2024-03-21T11:12:54.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:53.639+0000 7f4a8fa00700 1 -- 172.21.15.47:0/3554148408 wait complete. 2024-03-21T11:12:54.335 INFO:teuthology.orchestra.run.smithi047.stdout:Fetching dashboard port number... 2024-03-21T11:12:54.939 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:54 smithi047 ceph-mon[31351]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:12:54.939 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:54 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:12:54.939 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:54 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:12:55.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:55 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/364632711' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-21T11:12:55.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 8443 2024-03-21T11:12:55.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.939+0000 7f0fb5a00700 1 Processor -- start 2024-03-21T11:12:55.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.939+0000 7f0fb5a00700 1 -- start start 2024-03-21T11:12:55.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.940+0000 7f0fb5a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb007cd90 0x7f0fb007b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:55.851 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.940+0000 7f0fb5a00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb007b7c0 con 0x7f0fb007cd90 2024-03-21T11:12:55.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.940+0000 7f0faea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb007cd90 0x7f0fb007b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:55.852 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.940+0000 7f0faea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb007cd90 0x7f0fb007b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60866/0 (socket says 172.21.15.47:60866) 2024-03-21T11:12:55.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.940+0000 7f0faea00700 1 -- 172.21.15.47:0/4088597758 learned_addr learned my addr 172.21.15.47:0/4088597758 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:55.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.941+0000 7f0faea00700 1 -- 172.21.15.47:0/4088597758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb007b940 con 0x7f0fb007cd90 2024-03-21T11:12:55.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.941+0000 7f0faea00700 1 --2- 172.21.15.47:0/4088597758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb007cd90 0x7f0fb007b1f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00098a0 tx=0x7f0fa0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d082705d055223b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:55.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.942+0000 7f0fad600700 1 -- 172.21.15.47:0/4088597758 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fa0004160 con 0x7f0fb007cd90 2024-03-21T11:12:55.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.942+0000 7f0fad600700 1 -- 172.21.15.47:0/4088597758 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fa0004300 con 0x7f0fb007cd90 2024-03-21T11:12:55.853 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.942+0000 7f0fad600700 1 -- 172.21.15.47:0/4088597758 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fa0011b20 con 0x7f0fb007cd90 2024-03-21T11:12:55.857 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.943+0000 7f0fb5a00700 1 -- 172.21.15.47:0/4088597758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb007cd90 msgr2=0x7f0fb007b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:55.857 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.943+0000 7f0fb5a00700 1 --2- 172.21.15.47:0/4088597758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb007cd90 0x7f0fb007b1f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00098a0 tx=0x7f0fa0009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:55.857 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.943+0000 7f0fb5a00700 1 -- 172.21.15.47:0/4088597758 shutdown_connections 2024-03-21T11:12:55.857 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.943+0000 7f0fb5a00700 1 --2- 172.21.15.47:0/4088597758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb007cd90 0x7f0fb007b1f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:55.857 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.943+0000 7f0fb5a00700 1 -- 172.21.15.47:0/4088597758 >> 172.21.15.47:0/4088597758 conn(0x7f0fb0103080 msgr2=0x7f0fb01054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:55.858 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.943+0000 7f0fb5a00700 1 -- 172.21.15.47:0/4088597758 shutdown_connections 2024-03-21T11:12:55.858 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.943+0000 7f0fb5a00700 1 -- 172.21.15.47:0/4088597758 wait complete. 2024-03-21T11:12:55.858 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.944+0000 7f0fb5a00700 1 Processor -- start 2024-03-21T11:12:55.858 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.944+0000 7f0fb5a00700 1 -- start start 2024-03-21T11:12:55.858 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.944+0000 7f0fb5a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb007cd90 0x7f0fb019dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:55.858 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.944+0000 7f0fb5a00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb010e160 con 0x7f0fb007cd90 2024-03-21T11:12:55.858 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.945+0000 7f0faea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb007cd90 0x7f0fb019dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:55.858 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.945+0000 7f0faea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb007cd90 0x7f0fb019dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60874/0 (socket says 172.21.15.47:60874) 2024-03-21T11:12:55.858 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.945+0000 7f0faea00700 1 -- 172.21.15.47:0/364632711 learned_addr learned my addr 172.21.15.47:0/364632711 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:55.858 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.945+0000 7f0faea00700 1 -- 172.21.15.47:0/364632711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb019e170 con 0x7f0fb007cd90 2024-03-21T11:12:55.858 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.945+0000 7f0faea00700 1 --2- 172.21.15.47:0/364632711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb007cd90 0x7f0fb019dc30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00117e0 tx=0x7f0fa0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:55.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.946+0000 7f0fa7400700 1 -- 172.21.15.47:0/364632711 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fa0003bf0 con 0x7f0fb007cd90 2024-03-21T11:12:55.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.946+0000 7f0fa7400700 1 -- 172.21.15.47:0/364632711 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fa001a910 con 0x7f0fb007cd90 2024-03-21T11:12:55.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.946+0000 7f0fb5a00700 1 -- 172.21.15.47:0/364632711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb019e400 con 0x7f0fb007cd90 2024-03-21T11:12:55.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.946+0000 7f0fa7400700 1 -- 172.21.15.47:0/364632711 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fa0024970 con 0x7f0fb007cd90 2024-03-21T11:12:55.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.946+0000 7f0fb5a00700 1 -- 172.21.15.47:0/364632711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb019e860 con 0x7f0fb007cd90 2024-03-21T11:12:55.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.948+0000 7f0fa7400700 1 -- 172.21.15.47:0/364632711 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f0fa0022020 con 0x7f0fb007cd90 2024-03-21T11:12:55.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.948+0000 7f0fb5a00700 1 -- 172.21.15.47:0/364632711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb00637c0 con 0x7f0fb007cd90 2024-03-21T11:12:55.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.948+0000 7f0fa7400700 1 --2- 172.21.15.47:0/364632711 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f94039230 0x7f0f9403b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:55.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.948+0000 7f0fa7400700 1 -- 172.21.15.47:0/364632711 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0fa004d690 con 0x7f0fb007cd90 2024-03-21T11:12:55.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.949+0000 7f0fae000700 1 --2- 172.21.15.47:0/364632711 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f94039230 0x7f0f9403b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:55.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.949+0000 7f0fae000700 1 --2- 172.21.15.47:0/364632711 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f94039230 0x7f0f9403b6f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c006fd0 tx=0x7f0f9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:55.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:54.954+0000 7f0fa7400700 1 -- 172.21.15.47:0/364632711 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0fa0011e20 con 0x7f0fb007cd90 2024-03-21T11:12:55.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:55.188+0000 7f0fb5a00700 1 -- 172.21.15.47:0/364632711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f0fb01a12d0 con 0x7f0fb007cd90 2024-03-21T11:12:55.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:55.189+0000 7f0fa7400700 1 -- 172.21.15.47:0/364632711 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f0fa000b3e0 con 0x7f0fb007cd90 2024-03-21T11:12:55.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:55.192+0000 7f0fb5a00700 1 -- 172.21.15.47:0/364632711 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f94039230 msgr2=0x7f0f9403b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:55.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:55.192+0000 7f0fb5a00700 1 --2- 172.21.15.47:0/364632711 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f94039230 0x7f0f9403b6f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c006fd0 tx=0x7f0f9c006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:55.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:55.192+0000 7f0fb5a00700 1 -- 172.21.15.47:0/364632711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb007cd90 msgr2=0x7f0fb019dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:55.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:55.192+0000 7f0fb5a00700 1 --2- 172.21.15.47:0/364632711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb007cd90 0x7f0fb019dc30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00117e0 tx=0x7f0fa0011c10 comp rx=0 tx=0).stop 2024-03-21T11:12:55.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:55.192+0000 7f0fb5a00700 1 -- 172.21.15.47:0/364632711 shutdown_connections 2024-03-21T11:12:55.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:55.192+0000 7f0fb5a00700 1 --2- 172.21.15.47:0/364632711 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f94039230 0x7f0f9403b6f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:55.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:55.192+0000 7f0fb5a00700 1 --2- 172.21.15.47:0/364632711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fb007cd90 0x7f0fb019dc30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:55.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:55.192+0000 7f0fb5a00700 1 -- 172.21.15.47:0/364632711 >> 172.21.15.47:0/364632711 conn(0x7f0fb0103080 msgr2=0x7f0fb0103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:55.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:55.192+0000 7f0fb5a00700 1 -- 172.21.15.47:0/364632711 shutdown_connections 2024-03-21T11:12:55.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:55.192+0000 7f0fb5a00700 1 -- 172.21.15.47:0/364632711 wait complete. 2024-03-21T11:12:55.891 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T11:12:55.892 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-03-21T11:12:55.917 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T11:12:55.917 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-03-21T11:12:55.917 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-03-21T11:12:55.918 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-21T11:12:55.925 INFO:teuthology.orchestra.run.smithi047.stdout:Ceph Dashboard is now available at: 2024-03-21T11:12:55.926 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:12:55.926 INFO:teuthology.orchestra.run.smithi047.stdout: URL: https://smithi047.front.sepia.ceph.com:8443/ 2024-03-21T11:12:55.926 INFO:teuthology.orchestra.run.smithi047.stdout: User: admin 2024-03-21T11:12:55.926 INFO:teuthology.orchestra.run.smithi047.stdout: Password: qfjhsy638x 2024-03-21T11:12:55.926 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:12:55.927 INFO:teuthology.orchestra.run.smithi047.stdout:Saving cluster configuration to /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config directory 2024-03-21T11:12:55.929 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling autotune for osd_memory_target 2024-03-21T11:12:56.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:56 smithi047 ceph-mon[31351]: mgrmap e13: x(active, since 6s) 2024-03-21T11:12:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.951+0000 7ff260000700 1 Processor -- start 2024-03-21T11:12:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.951+0000 7ff260000700 1 -- start start 2024-03-21T11:12:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.951+0000 7ff260000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff258109db0 0x7ff25810a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.951+0000 7ff260000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25810a7a0 con 0x7ff258109db0 2024-03-21T11:12:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.952+0000 7ff25d600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff258109db0 0x7ff25810a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.952+0000 7ff25d600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff258109db0 0x7ff25810a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60882/0 (socket says 172.21.15.47:60882) 2024-03-21T11:12:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.952+0000 7ff25d600700 1 -- 172.21.15.47:0/4293245849 learned_addr learned my addr 172.21.15.47:0/4293245849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.952+0000 7ff25d600700 1 -- 172.21.15.47:0/4293245849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25810af10 con 0x7ff258109db0 2024-03-21T11:12:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.953+0000 7ff25d600700 1 --2- 172.21.15.47:0/4293245849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff258109db0 0x7ff25810a1d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009e80 tx=0x7ff24c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7df04acc60a84b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.953+0000 7ff257e00700 1 -- 172.21.15.47:0/4293245849 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff24c017070 con 0x7ff258109db0 2024-03-21T11:12:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.953+0000 7ff257e00700 1 -- 172.21.15.47:0/4293245849 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff24c00b8c0 con 0x7ff258109db0 2024-03-21T11:12:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.953+0000 7ff257e00700 1 -- 172.21.15.47:0/4293245849 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff24c0045c0 con 0x7ff258109db0 2024-03-21T11:12:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.954+0000 7ff260000700 1 -- 172.21.15.47:0/4293245849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff258109db0 msgr2=0x7ff25810a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.954+0000 7ff260000700 1 --2- 172.21.15.47:0/4293245849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff258109db0 0x7ff25810a1d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009e80 tx=0x7ff24c00b330 comp rx=0 tx=0).stop 2024-03-21T11:12:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.954+0000 7ff260000700 1 -- 172.21.15.47:0/4293245849 shutdown_connections 2024-03-21T11:12:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.954+0000 7ff260000700 1 --2- 172.21.15.47:0/4293245849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff258109db0 0x7ff25810a1d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.954+0000 7ff260000700 1 -- 172.21.15.47:0/4293245849 >> 172.21.15.47:0/4293245849 conn(0x7ff25807d220 msgr2=0x7ff25807d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.954+0000 7ff260000700 1 -- 172.21.15.47:0/4293245849 shutdown_connections 2024-03-21T11:12:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.954+0000 7ff260000700 1 -- 172.21.15.47:0/4293245849 wait complete. 2024-03-21T11:12:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.955+0000 7ff260000700 1 Processor -- start 2024-03-21T11:12:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.955+0000 7ff260000700 1 -- start start 2024-03-21T11:12:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.955+0000 7ff260000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff258109db0 0x7ff25819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.955+0000 7ff260000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25810e2c0 con 0x7ff258109db0 2024-03-21T11:12:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.956+0000 7ff25d600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff258109db0 0x7ff25819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.956+0000 7ff25d600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff258109db0 0x7ff25819dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60892/0 (socket says 172.21.15.47:60892) 2024-03-21T11:12:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.956+0000 7ff25d600700 1 -- 172.21.15.47:0/2383247635 learned_addr learned my addr 172.21.15.47:0/2383247635 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.956+0000 7ff25d600700 1 -- 172.21.15.47:0/2383247635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25819e220 con 0x7ff258109db0 2024-03-21T11:12:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.956+0000 7ff25d600700 1 --2- 172.21.15.47:0/2383247635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff258109db0 0x7ff25819dce0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009530 tx=0x7ff24c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.957+0000 7ff256000700 1 -- 172.21.15.47:0/2383247635 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff24c020020 con 0x7ff258109db0 2024-03-21T11:12:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.957+0000 7ff256000700 1 -- 172.21.15.47:0/2383247635 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff24c00bba0 con 0x7ff258109db0 2024-03-21T11:12:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.957+0000 7ff260000700 1 -- 172.21.15.47:0/2383247635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25819e4b0 con 0x7ff258109db0 2024-03-21T11:12:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.957+0000 7ff256000700 1 -- 172.21.15.47:0/2383247635 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff24c017040 con 0x7ff258109db0 2024-03-21T11:12:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.957+0000 7ff260000700 1 -- 172.21.15.47:0/2383247635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2581a1190 con 0x7ff258109db0 2024-03-21T11:12:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.958+0000 7ff256000700 1 -- 172.21.15.47:0/2383247635 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7ff24c023020 con 0x7ff258109db0 2024-03-21T11:12:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.958+0000 7ff260000700 1 -- 172.21.15.47:0/2383247635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2580637c0 con 0x7ff258109db0 2024-03-21T11:12:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.958+0000 7ff256000700 1 --2- 172.21.15.47:0/2383247635 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2400392d0 0x7ff24003b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.959+0000 7ff256000700 1 -- 172.21.15.47:0/2383247635 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff24c00f050 con 0x7ff258109db0 2024-03-21T11:12:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.959+0000 7ff25cc00700 1 --2- 172.21.15.47:0/2383247635 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2400392d0 0x7ff24003b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.960+0000 7ff25cc00700 1 --2- 172.21.15.47:0/2383247635 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2400392d0 0x7ff24003b790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff244006fd0 tx=0x7ff244006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:56.964+0000 7ff256000700 1 -- 172.21.15.47:0/2383247635 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff24c02a020 con 0x7ff258109db0 2024-03-21T11:12:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:57.196+0000 7ff260000700 1 -- 172.21.15.47:0/2383247635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7ff2581a18f0 con 0x7ff258109db0 2024-03-21T11:12:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:57.197+0000 7ff256000700 1 -- 172.21.15.47:0/2383247635 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7ff24c01bbd0 con 0x7ff258109db0 2024-03-21T11:12:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:57.200+0000 7ff260000700 1 -- 172.21.15.47:0/2383247635 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2400392d0 msgr2=0x7ff24003b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:57.200+0000 7ff260000700 1 --2- 172.21.15.47:0/2383247635 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2400392d0 0x7ff24003b790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff244006fd0 tx=0x7ff244006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:57.200+0000 7ff260000700 1 -- 172.21.15.47:0/2383247635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff258109db0 msgr2=0x7ff25819dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:57.200+0000 7ff260000700 1 --2- 172.21.15.47:0/2383247635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff258109db0 0x7ff25819dce0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009530 tx=0x7ff24c003d00 comp rx=0 tx=0).stop 2024-03-21T11:12:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:57.200+0000 7ff260000700 1 -- 172.21.15.47:0/2383247635 shutdown_connections 2024-03-21T11:12:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:57.200+0000 7ff260000700 1 --2- 172.21.15.47:0/2383247635 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2400392d0 0x7ff24003b790 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:57.200+0000 7ff260000700 1 --2- 172.21.15.47:0/2383247635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff258109db0 0x7ff25819dce0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:57.200+0000 7ff260000700 1 -- 172.21.15.47:0/2383247635 >> 172.21.15.47:0/2383247635 conn(0x7ff25807d220 msgr2=0x7ff2581074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:57.201+0000 7ff260000700 1 -- 172.21.15.47:0/2383247635 shutdown_connections 2024-03-21T11:12:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:57.201+0000 7ff260000700 1 -- 172.21.15.47:0/2383247635 wait complete. 2024-03-21T11:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.494+0000 7f3944e00700 1 Processor -- start 2024-03-21T11:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.494+0000 7f3944e00700 1 -- start start 2024-03-21T11:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.495+0000 7f3944e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3940106440 0x7f3940106860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.495+0000 7f3944e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394007be30 con 0x7f3940106440 2024-03-21T11:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.495+0000 7f393de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3940106440 0x7f3940106860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.495+0000 7f393de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3940106440 0x7f3940106860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60908/0 (socket says 172.21.15.47:60908) 2024-03-21T11:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.495+0000 7f393de00700 1 -- 172.21.15.47:0/3129594871 learned_addr learned my addr 172.21.15.47:0/3129594871 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:59.543 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.496+0000 7f393de00700 1 -- 172.21.15.47:0/3129594871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3940106da0 con 0x7f3940106440 2024-03-21T11:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.496+0000 7f393de00700 1 --2- 172.21.15.47:0/3129594871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3940106440 0x7f3940106860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f39300098a0 tx=0x7f3930009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838f7e7c753697fb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.496+0000 7f393d400700 1 -- 172.21.15.47:0/3129594871 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3930004160 con 0x7f3940106440 2024-03-21T11:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.497+0000 7f393d400700 1 -- 172.21.15.47:0/3129594871 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3930004300 con 0x7f3940106440 2024-03-21T11:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.497+0000 7f393d400700 1 -- 172.21.15.47:0/3129594871 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3930011b20 con 0x7f3940106440 2024-03-21T11:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.497+0000 7f3944e00700 1 -- 172.21.15.47:0/3129594871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3940106440 msgr2=0x7f3940106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.497+0000 7f3944e00700 1 --2- 172.21.15.47:0/3129594871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3940106440 0x7f3940106860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f39300098a0 tx=0x7f3930009bb0 comp rx=0 tx=0).stop 2024-03-21T11:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.497+0000 7f3944e00700 1 -- 172.21.15.47:0/3129594871 shutdown_connections 2024-03-21T11:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.497+0000 7f3944e00700 1 --2- 172.21.15.47:0/3129594871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3940106440 0x7f3940106860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.497+0000 7f3944e00700 1 -- 172.21.15.47:0/3129594871 >> 172.21.15.47:0/3129594871 conn(0x7f3940102000 msgr2=0x7f3940104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:59.544 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.498+0000 7f3944e00700 1 -- 172.21.15.47:0/3129594871 shutdown_connections 2024-03-21T11:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.498+0000 7f3944e00700 1 -- 172.21.15.47:0/3129594871 wait complete. 2024-03-21T11:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.499+0000 7f3944e00700 1 Processor -- start 2024-03-21T11:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.499+0000 7f3944e00700 1 -- start start 2024-03-21T11:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.499+0000 7f3944e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3940106440 0x7f394019dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.499+0000 7f3944e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3940107e70 con 0x7f3940106440 2024-03-21T11:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.499+0000 7f393de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3940106440 0x7f394019dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.499+0000 7f393de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3940106440 0x7f394019dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60922/0 (socket says 172.21.15.47:60922) 2024-03-21T11:12:59.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.499+0000 7f393de00700 1 -- 172.21.15.47:0/2809438016 learned_addr learned my addr 172.21.15.47:0/2809438016 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.500+0000 7f393de00700 1 -- 172.21.15.47:0/2809438016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394019e200 con 0x7f3940106440 2024-03-21T11:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.500+0000 7f393de00700 1 --2- 172.21.15.47:0/2809438016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3940106440 0x7f394019dcc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3930011b60 tx=0x7f3930011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.500+0000 7f3937400700 1 -- 172.21.15.47:0/2809438016 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3930003b60 con 0x7f3940106440 2024-03-21T11:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.500+0000 7f3937400700 1 -- 172.21.15.47:0/2809438016 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f393001a910 con 0x7f3940106440 2024-03-21T11:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.500+0000 7f3944e00700 1 -- 172.21.15.47:0/2809438016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394019e490 con 0x7f3940106440 2024-03-21T11:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.500+0000 7f3937400700 1 -- 172.21.15.47:0/2809438016 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3930024930 con 0x7f3940106440 2024-03-21T11:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.501+0000 7f3944e00700 1 -- 172.21.15.47:0/2809438016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394019e8f0 con 0x7f3940106440 2024-03-21T11:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.502+0000 7f3944e00700 1 -- 172.21.15.47:0/2809438016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39400637c0 con 0x7f3940106440 2024-03-21T11:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.502+0000 7f3937400700 1 -- 172.21.15.47:0/2809438016 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f3930022020 con 0x7f3940106440 2024-03-21T11:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.502+0000 7f3937400700 1 --2- 172.21.15.47:0/2809438016 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3924039270 0x7f392403b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.502+0000 7f3937400700 1 -- 172.21.15.47:0/2809438016 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f393004d560 con 0x7f3940106440 2024-03-21T11:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.503+0000 7f3935400700 1 --2- 172.21.15.47:0/2809438016 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3924039270 0x7f392403b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.503+0000 7f3935400700 1 --2- 172.21.15.47:0/2809438016 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3924039270 0x7f392403b730 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3928006fd0 tx=0x7f3928006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:12:59.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.508+0000 7f3937400700 1 -- 172.21.15.47:0/2809438016 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f393000b3e0 con 0x7f3940106440 2024-03-21T11:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.856+0000 7f3944e00700 1 -- 172.21.15.47:0/2809438016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f3940062da0 con 0x7f3940106440 2024-03-21T11:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.859+0000 7f3937400700 1 -- 172.21.15.47:0/2809438016 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f393002ddb0 con 0x7f3940106440 2024-03-21T11:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.862+0000 7f3944e00700 1 -- 172.21.15.47:0/2809438016 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3924039270 msgr2=0x7f392403b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.862+0000 7f3944e00700 1 --2- 172.21.15.47:0/2809438016 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3924039270 0x7f392403b730 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3928006fd0 tx=0x7f3928006e40 comp rx=0 tx=0).stop 2024-03-21T11:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.862+0000 7f3944e00700 1 -- 172.21.15.47:0/2809438016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3940106440 msgr2=0x7f394019dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.862+0000 7f3944e00700 1 --2- 172.21.15.47:0/2809438016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3940106440 0x7f394019dcc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3930011b60 tx=0x7f3930011c40 comp rx=0 tx=0).stop 2024-03-21T11:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.862+0000 7f3944e00700 1 -- 172.21.15.47:0/2809438016 shutdown_connections 2024-03-21T11:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.862+0000 7f3944e00700 1 --2- 172.21.15.47:0/2809438016 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3924039270 0x7f392403b730 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.862+0000 7f3944e00700 1 --2- 172.21.15.47:0/2809438016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3940106440 0x7f394019dcc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.862+0000 7f3944e00700 1 -- 172.21.15.47:0/2809438016 >> 172.21.15.47:0/2809438016 conn(0x7f3940102000 msgr2=0x7f39401948a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.863+0000 7f3944e00700 1 -- 172.21.15.47:0/2809438016 shutdown_connections 2024-03-21T11:12:59.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T11:12:58.863+0000 7f3944e00700 1 -- 172.21.15.47:0/2809438016 wait complete. 2024-03-21T11:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-21T11:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-21T11:12:59.551 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout:Or, if you are only running a single cluster on this host: 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: ceph telemetry on 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout:For more information see: 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:12:59.552 INFO:teuthology.orchestra.run.smithi047.stdout:Bootstrap complete. 2024-03-21T11:12:59.668 INFO:tasks.cephadm:Fetching config... 2024-03-21T11:12:59.668 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:12:59.668 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-21T11:12:59.723 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-21T11:12:59.723 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:12:59.723 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-21T11:12:59.764 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-21T11:12:59.764 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:12:59.764 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-21T11:12:59.887 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-21T11:12:59.887 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:12:59.887 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-21T11:12:59.983 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-21T11:12:59.984 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCd2A9bkrlxpsHKSFuRY2i055vBovL/3eSqEm9uziN0m7WLUWLcpzmKdfcHuxR/yMjjLSD7CquJ1zJbVEZllAAT8biRqSZSfVUo+SHiV6FVaARsdkCmgbpbQPBdW6Z1m0ibz9/ggehOMBMd9ZPhSGIKq4Jl5SdAxYJzODe1xB6S6bOEM+LJjBFY7lnRgzm9lKqZvlw+iZFYQ3jTblXDMI1L6tpP6zSkxyKRg32XA1A9RbbtI/+WvLg24P8fGQS16Lf1J+WW0ee8JHZRkxgi27s9/1YXadLjFXHuFzIzRnojjwd3vlb/z7K81saQd+4wuQa/EH4aG4qsKHpfwPCYemPCWnMU7CKDKF2ChQjMYbTIk5twD5zj/y3GB4GA7TyKuy8tPhQtq59HI6Z7tmFtt5/WGm0hqua7NXwcPAjc5YE0DVzS9ym1f9UIWNJ7SvFD9U5iltnU+65nOTwyYkA1LqcSKBrjkPHwyv+aEdiSDoXvoS7lf/Tk7JfOwF2/KWRFxoE= ceph-a6f35682-e773-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T11:13:00.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:12:59 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2809438016' entity='client.admin' 2024-03-21T11:13:00.123 INFO:teuthology.orchestra.run.smithi047.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCd2A9bkrlxpsHKSFuRY2i055vBovL/3eSqEm9uziN0m7WLUWLcpzmKdfcHuxR/yMjjLSD7CquJ1zJbVEZllAAT8biRqSZSfVUo+SHiV6FVaARsdkCmgbpbQPBdW6Z1m0ibz9/ggehOMBMd9ZPhSGIKq4Jl5SdAxYJzODe1xB6S6bOEM+LJjBFY7lnRgzm9lKqZvlw+iZFYQ3jTblXDMI1L6tpP6zSkxyKRg32XA1A9RbbtI/+WvLg24P8fGQS16Lf1J+WW0ee8JHZRkxgi27s9/1YXadLjFXHuFzIzRnojjwd3vlb/z7K81saQd+4wuQa/EH4aG4qsKHpfwPCYemPCWnMU7CKDKF2ChQjMYbTIk5twD5zj/y3GB4GA7TyKuy8tPhQtq59HI6Z7tmFtt5/WGm0hqua7NXwcPAjc5YE0DVzS9ym1f9UIWNJ7SvFD9U5iltnU+65nOTwyYkA1LqcSKBrjkPHwyv+aEdiSDoXvoS7lf/Tk7JfOwF2/KWRFxoE= ceph-a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:13:00.200 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCd2A9bkrlxpsHKSFuRY2i055vBovL/3eSqEm9uziN0m7WLUWLcpzmKdfcHuxR/yMjjLSD7CquJ1zJbVEZllAAT8biRqSZSfVUo+SHiV6FVaARsdkCmgbpbQPBdW6Z1m0ibz9/ggehOMBMd9ZPhSGIKq4Jl5SdAxYJzODe1xB6S6bOEM+LJjBFY7lnRgzm9lKqZvlw+iZFYQ3jTblXDMI1L6tpP6zSkxyKRg32XA1A9RbbtI/+WvLg24P8fGQS16Lf1J+WW0ee8JHZRkxgi27s9/1YXadLjFXHuFzIzRnojjwd3vlb/z7K81saQd+4wuQa/EH4aG4qsKHpfwPCYemPCWnMU7CKDKF2ChQjMYbTIk5twD5zj/y3GB4GA7TyKuy8tPhQtq59HI6Z7tmFtt5/WGm0hqua7NXwcPAjc5YE0DVzS9ym1f9UIWNJ7SvFD9U5iltnU+65nOTwyYkA1LqcSKBrjkPHwyv+aEdiSDoXvoS7lf/Tk7JfOwF2/KWRFxoE= ceph-a6f35682-e773-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T11:13:00.248 INFO:teuthology.orchestra.run.smithi100.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCd2A9bkrlxpsHKSFuRY2i055vBovL/3eSqEm9uziN0m7WLUWLcpzmKdfcHuxR/yMjjLSD7CquJ1zJbVEZllAAT8biRqSZSfVUo+SHiV6FVaARsdkCmgbpbQPBdW6Z1m0ibz9/ggehOMBMd9ZPhSGIKq4Jl5SdAxYJzODe1xB6S6bOEM+LJjBFY7lnRgzm9lKqZvlw+iZFYQ3jTblXDMI1L6tpP6zSkxyKRg32XA1A9RbbtI/+WvLg24P8fGQS16Lf1J+WW0ee8JHZRkxgi27s9/1YXadLjFXHuFzIzRnojjwd3vlb/z7K81saQd+4wuQa/EH4aG4qsKHpfwPCYemPCWnMU7CKDKF2ChQjMYbTIk5twD5zj/y3GB4GA7TyKuy8tPhQtq59HI6Z7tmFtt5/WGm0hqua7NXwcPAjc5YE0DVzS9ym1f9UIWNJ7SvFD9U5iltnU+65nOTwyYkA1LqcSKBrjkPHwyv+aEdiSDoXvoS7lf/Tk7JfOwF2/KWRFxoE= ceph-a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:13:00.263 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCd2A9bkrlxpsHKSFuRY2i055vBovL/3eSqEm9uziN0m7WLUWLcpzmKdfcHuxR/yMjjLSD7CquJ1zJbVEZllAAT8biRqSZSfVUo+SHiV6FVaARsdkCmgbpbQPBdW6Z1m0ibz9/ggehOMBMd9ZPhSGIKq4Jl5SdAxYJzODe1xB6S6bOEM+LJjBFY7lnRgzm9lKqZvlw+iZFYQ3jTblXDMI1L6tpP6zSkxyKRg32XA1A9RbbtI/+WvLg24P8fGQS16Lf1J+WW0ee8JHZRkxgi27s9/1YXadLjFXHuFzIzRnojjwd3vlb/z7K81saQd+4wuQa/EH4aG4qsKHpfwPCYemPCWnMU7CKDKF2ChQjMYbTIk5twD5zj/y3GB4GA7TyKuy8tPhQtq59HI6Z7tmFtt5/WGm0hqua7NXwcPAjc5YE0DVzS9ym1f9UIWNJ7SvFD9U5iltnU+65nOTwyYkA1LqcSKBrjkPHwyv+aEdiSDoXvoS7lf/Tk7JfOwF2/KWRFxoE= ceph-a6f35682-e773-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T11:13:00.304 INFO:teuthology.orchestra.run.smithi102.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCd2A9bkrlxpsHKSFuRY2i055vBovL/3eSqEm9uziN0m7WLUWLcpzmKdfcHuxR/yMjjLSD7CquJ1zJbVEZllAAT8biRqSZSfVUo+SHiV6FVaARsdkCmgbpbQPBdW6Z1m0ibz9/ggehOMBMd9ZPhSGIKq4Jl5SdAxYJzODe1xB6S6bOEM+LJjBFY7lnRgzm9lKqZvlw+iZFYQ3jTblXDMI1L6tpP6zSkxyKRg32XA1A9RbbtI/+WvLg24P8fGQS16Lf1J+WW0ee8JHZRkxgi27s9/1YXadLjFXHuFzIzRnojjwd3vlb/z7K81saQd+4wuQa/EH4aG4qsKHpfwPCYemPCWnMU7CKDKF2ChQjMYbTIk5twD5zj/y3GB4GA7TyKuy8tPhQtq59HI6Z7tmFtt5/WGm0hqua7NXwcPAjc5YE0DVzS9ym1f9UIWNJ7SvFD9U5iltnU+65nOTwyYkA1LqcSKBrjkPHwyv+aEdiSDoXvoS7lf/Tk7JfOwF2/KWRFxoE= ceph-a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:13:00.318 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-21T11:13:00.755 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:13:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.414+0000 7ff3ae200700 1 -- 172.21.15.47:0/805902021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a8103e00 msgr2=0x7ff3a8104220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.414+0000 7ff3ae200700 1 --2- 172.21.15.47:0/805902021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a8103e00 0x7ff3a8104220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff398009910 tx=0x7ff398009c20 comp rx=0 tx=0).stop 2024-03-21T11:13:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.414+0000 7ff3ae200700 1 -- 172.21.15.47:0/805902021 shutdown_connections 2024-03-21T11:13:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.414+0000 7ff3ae200700 1 --2- 172.21.15.47:0/805902021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a8103e00 0x7ff3a8104220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.414+0000 7ff3ae200700 1 -- 172.21.15.47:0/805902021 >> 172.21.15.47:0/805902021 conn(0x7ff3a80ff360 msgr2=0x7ff3a81017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.415+0000 7ff3ae200700 1 -- 172.21.15.47:0/805902021 shutdown_connections 2024-03-21T11:13:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.415+0000 7ff3ae200700 1 -- 172.21.15.47:0/805902021 wait complete. 2024-03-21T11:13:03.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.416+0000 7ff3ae200700 1 Processor -- start 2024-03-21T11:13:03.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.416+0000 7ff3ae200700 1 -- start start 2024-03-21T11:13:03.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.416+0000 7ff3ae200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a8103e00 0x7ff3a8199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:03.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.416+0000 7ff3ae200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a8109e10 con 0x7ff3a8103e00 2024-03-21T11:13:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.417+0000 7ff3a7400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a8103e00 0x7ff3a8199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.417+0000 7ff3a7400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a8103e00 0x7ff3a8199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59502/0 (socket says 172.21.15.47:59502) 2024-03-21T11:13:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.417+0000 7ff3a7400700 1 -- 172.21.15.47:0/4082911033 learned_addr learned my addr 172.21.15.47:0/4082911033 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:13:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.417+0000 7ff3a7400700 1 -- 172.21.15.47:0/4082911033 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a8199a60 con 0x7ff3a8103e00 2024-03-21T11:13:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.417+0000 7ff3a7400700 1 --2- 172.21.15.47:0/4082911033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a8103e00 0x7ff3a8199520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff398005950 tx=0x7ff398005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:03.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.418+0000 7ff39fe00700 1 -- 172.21.15.47:0/4082911033 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff398026070 con 0x7ff3a8103e00 2024-03-21T11:13:03.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.418+0000 7ff39fe00700 1 -- 172.21.15.47:0/4082911033 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff39800bc30 con 0x7ff3a8103e00 2024-03-21T11:13:03.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.418+0000 7ff3ae200700 1 -- 172.21.15.47:0/4082911033 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a8199cf0 con 0x7ff3a8103e00 2024-03-21T11:13:03.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.418+0000 7ff39fe00700 1 -- 172.21.15.47:0/4082911033 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff398003b30 con 0x7ff3a8103e00 2024-03-21T11:13:03.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.418+0000 7ff3ae200700 1 -- 172.21.15.47:0/4082911033 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a819c990 con 0x7ff3a8103e00 2024-03-21T11:13:03.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.420+0000 7ff39fe00700 1 -- 172.21.15.47:0/4082911033 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7ff398029070 con 0x7ff3a8103e00 2024-03-21T11:13:03.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.419+0000 7ff3ae200700 1 -- 172.21.15.47:0/4082911033 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a810a590 con 0x7ff3a8103e00 2024-03-21T11:13:03.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.420+0000 7ff39fe00700 1 --2- 172.21.15.47:0/4082911033 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff38c039280 0x7ff38c03b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:03.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.420+0000 7ff39fe00700 1 -- 172.21.15.47:0/4082911033 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff39801e070 con 0x7ff3a8103e00 2024-03-21T11:13:03.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.420+0000 7ff3a6a00700 1 --2- 172.21.15.47:0/4082911033 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff38c039280 0x7ff38c03b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.421+0000 7ff3a6a00700 1 --2- 172.21.15.47:0/4082911033 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff38c039280 0x7ff38c03b740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff394006fd0 tx=0x7ff394006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:03.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.425+0000 7ff39fe00700 1 -- 172.21.15.47:0/4082911033 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff398013070 con 0x7ff3a8103e00 2024-03-21T11:13:03.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.651+0000 7ff3ae200700 1 -- 172.21.15.47:0/4082911033 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7ff3a802db40 con 0x7ff3a8103e00 2024-03-21T11:13:03.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.656+0000 7ff39fe00700 1 -- 172.21.15.47:0/4082911033 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff398030020 con 0x7ff3a8103e00 2024-03-21T11:13:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.659+0000 7ff3ae200700 1 -- 172.21.15.47:0/4082911033 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff38c039280 msgr2=0x7ff38c03b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.659+0000 7ff3ae200700 1 --2- 172.21.15.47:0/4082911033 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff38c039280 0x7ff38c03b740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff394006fd0 tx=0x7ff394006e40 comp rx=0 tx=0).stop 2024-03-21T11:13:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.659+0000 7ff3ae200700 1 -- 172.21.15.47:0/4082911033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a8103e00 msgr2=0x7ff3a8199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.659+0000 7ff3ae200700 1 --2- 172.21.15.47:0/4082911033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a8103e00 0x7ff3a8199520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff398005950 tx=0x7ff398005030 comp rx=0 tx=0).stop 2024-03-21T11:13:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.659+0000 7ff3ae200700 1 -- 172.21.15.47:0/4082911033 shutdown_connections 2024-03-21T11:13:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.659+0000 7ff3ae200700 1 --2- 172.21.15.47:0/4082911033 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff38c039280 0x7ff38c03b740 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.659+0000 7ff3ae200700 1 --2- 172.21.15.47:0/4082911033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3a8103e00 0x7ff3a8199520 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.659+0000 7ff3ae200700 1 -- 172.21.15.47:0/4082911033 >> 172.21.15.47:0/4082911033 conn(0x7ff3a80ff360 msgr2=0x7ff3a80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.660+0000 7ff3ae200700 1 -- 172.21.15.47:0/4082911033 shutdown_connections 2024-03-21T11:13:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:03.660+0000 7ff3ae200700 1 -- 172.21.15.47:0/4082911033 wait complete. 2024-03-21T11:13:04.298 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:04.298 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:04.298 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi047", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:13:04.299 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:04.299 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:04.299 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:04.299 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:03 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/4082911033' entity='client.admin' 2024-03-21T11:13:04.299 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:04.299 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:04.376 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-21T11:13:04.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-21T11:13:04.843 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:13:06.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.811+0000 7fdcfc600700 1 -- 172.21.15.47:0/1934530356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcf4076140 msgr2=0x7fdcf40745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:06.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.811+0000 7fdcfc600700 1 --2- 172.21.15.47:0/1934530356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcf4076140 0x7fdcf40745a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009910 tx=0x7fdce8009c20 comp rx=0 tx=0).stop 2024-03-21T11:13:06.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.812+0000 7fdcfc600700 1 -- 172.21.15.47:0/1934530356 shutdown_connections 2024-03-21T11:13:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.812+0000 7fdcfc600700 1 --2- 172.21.15.47:0/1934530356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcf4076140 0x7fdcf40745a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.812+0000 7fdcfc600700 1 -- 172.21.15.47:0/1934530356 >> 172.21.15.47:0/1934530356 conn(0x7fdcf40fd160 msgr2=0x7fdcf40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.812+0000 7fdcfc600700 1 -- 172.21.15.47:0/1934530356 shutdown_connections 2024-03-21T11:13:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.812+0000 7fdcfc600700 1 -- 172.21.15.47:0/1934530356 wait complete. 2024-03-21T11:13:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.813+0000 7fdcfc600700 1 Processor -- start 2024-03-21T11:13:06.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.813+0000 7fdcfc600700 1 -- start start 2024-03-21T11:13:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.814+0000 7fdcfc600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcf4076140 0x7fdcf4199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.814+0000 7fdcfc600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf4109e40 con 0x7fdcf4076140 2024-03-21T11:13:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.814+0000 7fdcf9c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcf4076140 0x7fdcf4199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.814+0000 7fdcf9c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcf4076140 0x7fdcf4199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59518/0 (socket says 172.21.15.47:59518) 2024-03-21T11:13:06.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.814+0000 7fdcf9c00700 1 -- 172.21.15.47:0/254655365 learned_addr learned my addr 172.21.15.47:0/254655365 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:13:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.815+0000 7fdcf9c00700 1 -- 172.21.15.47:0/254655365 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf41999d0 con 0x7fdcf4076140 2024-03-21T11:13:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.815+0000 7fdcf9c00700 1 --2- 172.21.15.47:0/254655365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcf4076140 0x7fdcf4199490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdce8004fa0 tx=0x7fdce8005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.815+0000 7fdcf2000700 1 -- 172.21.15.47:0/254655365 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdce8027070 con 0x7fdcf4076140 2024-03-21T11:13:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.815+0000 7fdcf2000700 1 -- 172.21.15.47:0/254655365 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdce800be40 con 0x7fdcf4076140 2024-03-21T11:13:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.815+0000 7fdcfc600700 1 -- 172.21.15.47:0/254655365 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf4199c60 con 0x7fdcf4076140 2024-03-21T11:13:06.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.815+0000 7fdcf2000700 1 -- 172.21.15.47:0/254655365 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdce800f460 con 0x7fdcf4076140 2024-03-21T11:13:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.815+0000 7fdcfc600700 1 -- 172.21.15.47:0/254655365 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf419c950 con 0x7fdcf4076140 2024-03-21T11:13:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.816+0000 7fdcf2000700 1 -- 172.21.15.47:0/254655365 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fdce802a070 con 0x7fdcf4076140 2024-03-21T11:13:06.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.817+0000 7fdcfc600700 1 -- 172.21.15.47:0/254655365 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf410a5c0 con 0x7fdcf4076140 2024-03-21T11:13:06.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.817+0000 7fdcf2000700 1 --2- 172.21.15.47:0/254655365 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdcdc039280 0x7fdcdc03b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:06.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.817+0000 7fdcf2000700 1 -- 172.21.15.47:0/254655365 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdce801e030 con 0x7fdcf4076140 2024-03-21T11:13:06.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.817+0000 7fdcf9200700 1 --2- 172.21.15.47:0/254655365 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdcdc039280 0x7fdcdc03b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:06.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.818+0000 7fdcf9200700 1 --2- 172.21.15.47:0/254655365 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdcdc039280 0x7fdcdc03b740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdce0006fd0 tx=0x7fdce0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:06.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:06.823+0000 7fdcf2000700 1 -- 172.21.15.47:0/254655365 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdce8013070 con 0x7fdcf4076140 2024-03-21T11:13:07.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:07.052+0000 7fdcfc600700 1 -- 172.21.15.47:0/254655365 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcf402db40 con 0x7fdcdc039280 2024-03-21T11:13:07.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:07.061+0000 7fdcf2000700 1 -- 172.21.15.47:0/254655365 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdcf402db40 con 0x7fdcdc039280 2024-03-21T11:13:07.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:07.063+0000 7fdcfc600700 1 -- 172.21.15.47:0/254655365 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdcdc039280 msgr2=0x7fdcdc03b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:07.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:07.063+0000 7fdcfc600700 1 --2- 172.21.15.47:0/254655365 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdcdc039280 0x7fdcdc03b740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdce0006fd0 tx=0x7fdce0006e40 comp rx=0 tx=0).stop 2024-03-21T11:13:07.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:07.063+0000 7fdcfc600700 1 -- 172.21.15.47:0/254655365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcf4076140 msgr2=0x7fdcf4199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:07.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:07.063+0000 7fdcfc600700 1 --2- 172.21.15.47:0/254655365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcf4076140 0x7fdcf4199490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdce8004fa0 tx=0x7fdce8005080 comp rx=0 tx=0).stop 2024-03-21T11:13:07.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:07.064+0000 7fdcfc600700 1 -- 172.21.15.47:0/254655365 shutdown_connections 2024-03-21T11:13:07.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:07.064+0000 7fdcfc600700 1 --2- 172.21.15.47:0/254655365 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdcdc039280 0x7fdcdc03b740 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:07.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:07.064+0000 7fdcfc600700 1 --2- 172.21.15.47:0/254655365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcf4076140 0x7fdcf4199490 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:07.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:07.064+0000 7fdcfc600700 1 -- 172.21.15.47:0/254655365 >> 172.21.15.47:0/254655365 conn(0x7fdcf40fd160 msgr2=0x7fdcf40fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:07.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:07.064+0000 7fdcfc600700 1 -- 172.21.15.47:0/254655365 shutdown_connections 2024-03-21T11:13:07.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:07.064+0000 7fdcfc600700 1 -- 172.21.15.47:0/254655365 wait complete. 2024-03-21T11:13:07.806 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi100 2024-03-21T11:13:07.806 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T11:13:07.806 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/etc/ceph/ceph.conf 2024-03-21T11:13:07.825 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T11:13:07.825 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T11:13:07.882 INFO:tasks.cephadm:Adding host smithi100 to orchestrator... 2024-03-21T11:13:07.882 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch host add smithi100 2024-03-21T11:13:08.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:08 smithi047 ceph-mon[31351]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:13:08.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:08 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:08.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:08 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:08.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:08 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:08.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:08 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:13:08.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:08 smithi047 ceph-mon[31351]: Updating smithi047:/etc/ceph/ceph.conf 2024-03-21T11:13:08.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:08 smithi047 ceph-mon[31351]: Updating smithi047:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:13:08.314 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:13:09.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:09 smithi047 ceph-mon[31351]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-03-21T11:13:10.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.322+0000 7f54b2400700 1 -- 172.21.15.47:0/1604056894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0745c0 msgr2=0x7f54ac0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.322+0000 7f54b2400700 1 --2- 172.21.15.47:0/1604056894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0745c0 0x7f54ac0749a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f549c009910 tx=0x7f549c009c20 comp rx=0 tx=0).stop 2024-03-21T11:13:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.322+0000 7f54b2400700 1 -- 172.21.15.47:0/1604056894 shutdown_connections 2024-03-21T11:13:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.322+0000 7f54b2400700 1 --2- 172.21.15.47:0/1604056894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0745c0 0x7f54ac0749a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.322+0000 7f54b2400700 1 -- 172.21.15.47:0/1604056894 >> 172.21.15.47:0/1604056894 conn(0x7f54ac0fd4c0 msgr2=0x7f54ac0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.322+0000 7f54b2400700 1 -- 172.21.15.47:0/1604056894 shutdown_connections 2024-03-21T11:13:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.322+0000 7f54b2400700 1 -- 172.21.15.47:0/1604056894 wait complete. 2024-03-21T11:13:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.324+0000 7f54b2400700 1 Processor -- start 2024-03-21T11:13:10.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.324+0000 7f54b2400700 1 -- start start 2024-03-21T11:13:10.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.324+0000 7f54b2400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0745c0 0x7f54ac19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:10.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.325+0000 7f54b2400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac10e260 con 0x7f54ac0745c0 2024-03-21T11:13:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.325+0000 7f54b1000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0745c0 0x7f54ac19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.325+0000 7f54b1000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0745c0 0x7f54ac19f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36048/0 (socket says 172.21.15.47:36048) 2024-03-21T11:13:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.325+0000 7f54b1000700 1 -- 172.21.15.47:0/1452562944 learned_addr learned my addr 172.21.15.47:0/1452562944 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:13:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.325+0000 7f54b1000700 1 -- 172.21.15.47:0/1452562944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac19f910 con 0x7f54ac0745c0 2024-03-21T11:13:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.325+0000 7f54b1000700 1 --2- 172.21.15.47:0/1452562944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0745c0 0x7f54ac19f3d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f549c006010 tx=0x7f549c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.326+0000 7f54aa000700 1 -- 172.21.15.47:0/1452562944 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f549c026070 con 0x7f54ac0745c0 2024-03-21T11:13:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.326+0000 7f54aa000700 1 -- 172.21.15.47:0/1452562944 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f549c00bc30 con 0x7f54ac0745c0 2024-03-21T11:13:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.326+0000 7f54b2400700 1 -- 172.21.15.47:0/1452562944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ac19fba0 con 0x7f54ac0745c0 2024-03-21T11:13:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.326+0000 7f54aa000700 1 -- 172.21.15.47:0/1452562944 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f549c0039a0 con 0x7f54ac0745c0 2024-03-21T11:13:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.326+0000 7f54b2400700 1 -- 172.21.15.47:0/1452562944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac199590 con 0x7f54ac0745c0 2024-03-21T11:13:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.327+0000 7f54aa000700 1 -- 172.21.15.47:0/1452562944 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f549c029070 con 0x7f54ac0745c0 2024-03-21T11:13:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.327+0000 7f54b2400700 1 -- 172.21.15.47:0/1452562944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ac075c70 con 0x7f54ac0745c0 2024-03-21T11:13:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.328+0000 7f54aa000700 1 --2- 172.21.15.47:0/1452562944 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5490039280 0x7f549003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.328+0000 7f54aa000700 1 -- 172.21.15.47:0/1452562944 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f549c01e070 con 0x7f54ac0745c0 2024-03-21T11:13:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.328+0000 7f54a3e00700 1 --2- 172.21.15.47:0/1452562944 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5490039280 0x7f549003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.329+0000 7f54a3e00700 1 --2- 172.21.15.47:0/1452562944 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5490039280 0x7f549003b740 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5494006fd0 tx=0x7f5494006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.334+0000 7f54aa000700 1 -- 172.21.15.47:0/1452562944 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f549c013070 con 0x7f54ac0745c0 2024-03-21T11:13:10.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:10.565+0000 7f54b2400700 1 -- 172.21.15.47:0/1452562944 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi100", "target": ["mon-mgr", ""]}) v1 -- 0x7f54ac0637c0 con 0x7f5490039280 2024-03-21T11:13:11.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:10 smithi047 ceph-mon[31351]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:11.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:10 smithi047 ceph-mon[31351]: Updating smithi047:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T11:13:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:12.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:11 smithi047 ceph-mon[31351]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi100", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:13:13.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:12 smithi047 ceph-mon[31351]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:13.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:12 smithi047 ceph-mon[31351]: Deploying cephadm binary to smithi100 2024-03-21T11:13:15.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:14 smithi047 ceph-mon[31351]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:15.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:15.688+0000 7f54aa000700 1 -- 172.21.15.47:0/1452562944 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f54ac0637c0 con 0x7f5490039280 2024-03-21T11:13:15.689 INFO:teuthology.orchestra.run.smithi047.stdout:Added host 'smithi100' with addr '172.21.15.100' 2024-03-21T11:13:15.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:15.693+0000 7f54b2400700 1 -- 172.21.15.47:0/1452562944 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5490039280 msgr2=0x7f549003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:15.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:15.693+0000 7f54b2400700 1 --2- 172.21.15.47:0/1452562944 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5490039280 0x7f549003b740 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5494006fd0 tx=0x7f5494006e40 comp rx=0 tx=0).stop 2024-03-21T11:13:15.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:15.693+0000 7f54b2400700 1 -- 172.21.15.47:0/1452562944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0745c0 msgr2=0x7f54ac19f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:15.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:15.693+0000 7f54b2400700 1 --2- 172.21.15.47:0/1452562944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0745c0 0x7f54ac19f3d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f549c006010 tx=0x7f549c005030 comp rx=0 tx=0).stop 2024-03-21T11:13:15.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:15.693+0000 7f54b2400700 1 -- 172.21.15.47:0/1452562944 shutdown_connections 2024-03-21T11:13:15.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:15.693+0000 7f54b2400700 1 --2- 172.21.15.47:0/1452562944 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5490039280 0x7f549003b740 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:15.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:15.693+0000 7f54b2400700 1 --2- 172.21.15.47:0/1452562944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54ac0745c0 0x7f54ac19f3d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:15.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:15.693+0000 7f54b2400700 1 -- 172.21.15.47:0/1452562944 >> 172.21.15.47:0/1452562944 conn(0x7f54ac0fd4c0 msgr2=0x7f54ac105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:15.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:15.693+0000 7f54b2400700 1 -- 172.21.15.47:0/1452562944 shutdown_connections 2024-03-21T11:13:15.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:15.693+0000 7f54b2400700 1 -- 172.21.15.47:0/1452562944 wait complete. 2024-03-21T11:13:16.431 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-21T11:13:16.885 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:13:17.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:16 smithi047 ceph-mon[31351]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:17.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:17.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:16 smithi047 ceph-mon[31351]: Added host smithi100 2024-03-21T11:13:17.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:17.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:18.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:17 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:18.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.902+0000 7ff96fc00700 1 -- 172.21.15.47:0/1953309121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff968101dc0 msgr2=0x7ff9681021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:18.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.902+0000 7ff96fc00700 1 --2- 172.21.15.47:0/1953309121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff968101dc0 0x7ff9681021a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009910 tx=0x7ff95c009c20 comp rx=0 tx=0).stop 2024-03-21T11:13:18.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.902+0000 7ff96fc00700 1 -- 172.21.15.47:0/1953309121 shutdown_connections 2024-03-21T11:13:18.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.902+0000 7ff96fc00700 1 --2- 172.21.15.47:0/1953309121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff968101dc0 0x7ff9681021a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:18.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.902+0000 7ff96fc00700 1 -- 172.21.15.47:0/1953309121 >> 172.21.15.47:0/1953309121 conn(0x7ff9680fd670 msgr2=0x7ff9680ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:18.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.902+0000 7ff96fc00700 1 -- 172.21.15.47:0/1953309121 shutdown_connections 2024-03-21T11:13:18.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.902+0000 7ff96fc00700 1 -- 172.21.15.47:0/1953309121 wait complete. 2024-03-21T11:13:18.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.903+0000 7ff96fc00700 1 Processor -- start 2024-03-21T11:13:18.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.903+0000 7ff96fc00700 1 -- start start 2024-03-21T11:13:18.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.904+0000 7ff96fc00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff968101dc0 0x7ff9681976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:18.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.904+0000 7ff96fc00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96810c190 con 0x7ff968101dc0 2024-03-21T11:13:18.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.904+0000 7ff96d200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff968101dc0 0x7ff9681976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:18.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.904+0000 7ff96d200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff968101dc0 0x7ff9681976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57876/0 (socket says 172.21.15.47:57876) 2024-03-21T11:13:18.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.904+0000 7ff96d200700 1 -- 172.21.15.47:0/2547066387 learned_addr learned my addr 172.21.15.47:0/2547066387 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:13:18.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.904+0000 7ff96d200700 1 -- 172.21.15.47:0/2547066387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff968197c30 con 0x7ff968101dc0 2024-03-21T11:13:18.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.905+0000 7ff96d200700 1 --2- 172.21.15.47:0/2547066387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff968101dc0 0x7ff9681976f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff95c005850 tx=0x7ff95c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:18.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.905+0000 7ff965600700 1 -- 172.21.15.47:0/2547066387 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff95c026070 con 0x7ff968101dc0 2024-03-21T11:13:18.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.905+0000 7ff965600700 1 -- 172.21.15.47:0/2547066387 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff95c00bbd0 con 0x7ff968101dc0 2024-03-21T11:13:18.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.905+0000 7ff96fc00700 1 -- 172.21.15.47:0/2547066387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff968197f20 con 0x7ff968101dc0 2024-03-21T11:13:18.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.905+0000 7ff965600700 1 -- 172.21.15.47:0/2547066387 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff95c003970 con 0x7ff968101dc0 2024-03-21T11:13:18.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.905+0000 7ff96fc00700 1 -- 172.21.15.47:0/2547066387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96819bfc0 con 0x7ff968101dc0 2024-03-21T11:13:18.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.906+0000 7ff965600700 1 -- 172.21.15.47:0/2547066387 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7ff95c029070 con 0x7ff968101dc0 2024-03-21T11:13:18.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.906+0000 7ff96fc00700 1 -- 172.21.15.47:0/2547066387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9681032a0 con 0x7ff968101dc0 2024-03-21T11:13:18.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.907+0000 7ff965600700 1 --2- 172.21.15.47:0/2547066387 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff950039280 0x7ff95003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:18.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.907+0000 7ff965600700 1 -- 172.21.15.47:0/2547066387 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff95c01e070 con 0x7ff968101dc0 2024-03-21T11:13:18.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.907+0000 7ff967e00700 1 --2- 172.21.15.47:0/2547066387 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff950039280 0x7ff95003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:18.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.908+0000 7ff967e00700 1 --2- 172.21.15.47:0/2547066387 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff950039280 0x7ff95003b740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff954006fd0 tx=0x7ff954006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:18.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:18.913+0000 7ff965600700 1 -- 172.21.15.47:0/2547066387 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff95c013070 con 0x7ff968101dc0 2024-03-21T11:13:19.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:18 smithi047 ceph-mon[31351]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:19.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:18 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:19.146+0000 7ff96fc00700 1 -- 172.21.15.47:0/2547066387 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff968198a50 con 0x7ff950039280 2024-03-21T11:13:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:19.148+0000 7ff965600700 1 -- 172.21.15.47:0/2547066387 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7ff968198a50 con 0x7ff950039280 2024-03-21T11:13:19.150 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:13:19.150 INFO:teuthology.orchestra.run.smithi047.stdout:[{"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}] 2024-03-21T11:13:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:19.151+0000 7ff96fc00700 1 -- 172.21.15.47:0/2547066387 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff950039280 msgr2=0x7ff95003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:19.151+0000 7ff96fc00700 1 --2- 172.21.15.47:0/2547066387 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff950039280 0x7ff95003b740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff954006fd0 tx=0x7ff954006e40 comp rx=0 tx=0).stop 2024-03-21T11:13:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:19.151+0000 7ff96fc00700 1 -- 172.21.15.47:0/2547066387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff968101dc0 msgr2=0x7ff9681976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:19.151+0000 7ff96fc00700 1 --2- 172.21.15.47:0/2547066387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff968101dc0 0x7ff9681976f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff95c005850 tx=0x7ff95c005030 comp rx=0 tx=0).stop 2024-03-21T11:13:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:19.152+0000 7ff96fc00700 1 -- 172.21.15.47:0/2547066387 shutdown_connections 2024-03-21T11:13:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:19.152+0000 7ff96fc00700 1 --2- 172.21.15.47:0/2547066387 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff950039280 0x7ff95003b740 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:19.152+0000 7ff96fc00700 1 --2- 172.21.15.47:0/2547066387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff968101dc0 0x7ff9681976f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:19.152+0000 7ff96fc00700 1 -- 172.21.15.47:0/2547066387 >> 172.21.15.47:0/2547066387 conn(0x7ff9680fd670 msgr2=0x7ff968103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:19.152+0000 7ff96fc00700 1 -- 172.21.15.47:0/2547066387 shutdown_connections 2024-03-21T11:13:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:19.152+0000 7ff96fc00700 1 -- 172.21.15.47:0/2547066387 wait complete. 2024-03-21T11:13:19.886 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi102 2024-03-21T11:13:19.887 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-21T11:13:19.887 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/etc/ceph/ceph.conf 2024-03-21T11:13:19.905 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-21T11:13:19.905 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T11:13:19.962 INFO:tasks.cephadm:Adding host smithi102 to orchestrator... 2024-03-21T11:13:19.962 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch host add smithi102 2024-03-21T11:13:20.416 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:13:20.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:20 smithi047 ceph-mon[31351]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:20.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:20 smithi047 ceph-mon[31351]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:13:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.396+0000 7fbe94e00700 1 -- 172.21.15.47:0/2116005202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900744b0 msgr2=0x7fbe90074890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.396+0000 7fbe94e00700 1 --2- 172.21.15.47:0/2116005202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900744b0 0x7fbe90074890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009910 tx=0x7fbe80009c20 comp rx=0 tx=0).stop 2024-03-21T11:13:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.396+0000 7fbe94e00700 1 -- 172.21.15.47:0/2116005202 shutdown_connections 2024-03-21T11:13:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.396+0000 7fbe94e00700 1 --2- 172.21.15.47:0/2116005202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900744b0 0x7fbe90074890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.396+0000 7fbe94e00700 1 -- 172.21.15.47:0/2116005202 >> 172.21.15.47:0/2116005202 conn(0x7fbe900fd670 msgr2=0x7fbe900ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.396+0000 7fbe94e00700 1 -- 172.21.15.47:0/2116005202 shutdown_connections 2024-03-21T11:13:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.396+0000 7fbe94e00700 1 -- 172.21.15.47:0/2116005202 wait complete. 2024-03-21T11:13:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.397+0000 7fbe94e00700 1 Processor -- start 2024-03-21T11:13:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.398+0000 7fbe94e00700 1 -- start start 2024-03-21T11:13:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.398+0000 7fbe94e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900744b0 0x7fbe9019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.398+0000 7fbe94e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9010e3f0 con 0x7fbe900744b0 2024-03-21T11:13:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.398+0000 7fbe8de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900744b0 0x7fbe9019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.398+0000 7fbe8de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900744b0 0x7fbe9019f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57888/0 (socket says 172.21.15.47:57888) 2024-03-21T11:13:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.398+0000 7fbe8de00700 1 -- 172.21.15.47:0/1911141133 learned_addr learned my addr 172.21.15.47:0/1911141133 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:13:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.399+0000 7fbe8de00700 1 -- 172.21.15.47:0/1911141133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9019fbc0 con 0x7fbe900744b0 2024-03-21T11:13:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.399+0000 7fbe8de00700 1 --2- 172.21.15.47:0/1911141133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900744b0 0x7fbe9019f680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe80006010 tx=0x7fbe80005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.399+0000 7fbe86a00700 1 -- 172.21.15.47:0/1911141133 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe80026070 con 0x7fbe900744b0 2024-03-21T11:13:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.399+0000 7fbe86a00700 1 -- 172.21.15.47:0/1911141133 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe8000bc30 con 0x7fbe900744b0 2024-03-21T11:13:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.399+0000 7fbe86a00700 1 -- 172.21.15.47:0/1911141133 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe800039a0 con 0x7fbe900744b0 2024-03-21T11:13:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.400+0000 7fbe94e00700 1 -- 172.21.15.47:0/1911141133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9019fe50 con 0x7fbe900744b0 2024-03-21T11:13:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.400+0000 7fbe94e00700 1 -- 172.21.15.47:0/1911141133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe90199750 con 0x7fbe900744b0 2024-03-21T11:13:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.401+0000 7fbe86a00700 1 -- 172.21.15.47:0/1911141133 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fbe80029070 con 0x7fbe900744b0 2024-03-21T11:13:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.401+0000 7fbe86a00700 1 --2- 172.21.15.47:0/1911141133 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe74039280 0x7fbe7403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.401+0000 7fbe86a00700 1 -- 172.21.15.47:0/1911141133 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbe8001e070 con 0x7fbe900744b0 2024-03-21T11:13:22.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.402+0000 7fbe94e00700 1 -- 172.21.15.47:0/1911141133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe90075bf0 con 0x7fbe900744b0 2024-03-21T11:13:22.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.402+0000 7fbe8d400700 1 --2- 172.21.15.47:0/1911141133 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe74039280 0x7fbe7403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:22.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.403+0000 7fbe8d400700 1 --2- 172.21.15.47:0/1911141133 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe74039280 0x7fbe7403b740 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c006fd0 tx=0x7fbe7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:22.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.407+0000 7fbe86a00700 1 -- 172.21.15.47:0/1911141133 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbe80030070 con 0x7fbe900744b0 2024-03-21T11:13:22.546 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:22 smithi047 ceph-mon[31351]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:22.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:22.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:22.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:22.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:22.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:13:22.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:22.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:13:22.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:22 smithi047 ceph-mon[31351]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-21T11:13:22.547 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:22 smithi047 ceph-mon[31351]: Updating smithi100:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:13:22.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:22.697+0000 7fbe94e00700 1 -- 172.21.15.47:0/1911141133 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe900637c0 con 0x7fbe74039280 2024-03-21T11:13:24.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:23 smithi047 ceph-mon[31351]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-21T11:13:24.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:23 smithi047 ceph-mon[31351]: Updating smithi100:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T11:13:24.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:24.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:24.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:24.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:23 smithi047 ceph-mon[31351]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:13:25.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:24 smithi047 ceph-mon[31351]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:25.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:24 smithi047 ceph-mon[31351]: Deploying cephadm binary to smithi102 2024-03-21T11:13:27.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:26 smithi047 ceph-mon[31351]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:27.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:27.570+0000 7fbe86a00700 1 -- 172.21.15.47:0/1911141133 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbe900637c0 con 0x7fbe74039280 2024-03-21T11:13:27.571 INFO:teuthology.orchestra.run.smithi047.stdout:Added host 'smithi102' with addr '172.21.15.102' 2024-03-21T11:13:27.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:27.573+0000 7fbe94e00700 1 -- 172.21.15.47:0/1911141133 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe74039280 msgr2=0x7fbe7403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:27.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:27.573+0000 7fbe94e00700 1 --2- 172.21.15.47:0/1911141133 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe74039280 0x7fbe7403b740 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c006fd0 tx=0x7fbe7c006e40 comp rx=0 tx=0).stop 2024-03-21T11:13:27.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:27.573+0000 7fbe94e00700 1 -- 172.21.15.47:0/1911141133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900744b0 msgr2=0x7fbe9019f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:27.573+0000 7fbe94e00700 1 --2- 172.21.15.47:0/1911141133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900744b0 0x7fbe9019f680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe80006010 tx=0x7fbe80005030 comp rx=0 tx=0).stop 2024-03-21T11:13:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:27.574+0000 7fbe94e00700 1 -- 172.21.15.47:0/1911141133 shutdown_connections 2024-03-21T11:13:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:27.574+0000 7fbe94e00700 1 --2- 172.21.15.47:0/1911141133 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe74039280 0x7fbe7403b740 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:27.574+0000 7fbe94e00700 1 --2- 172.21.15.47:0/1911141133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe900744b0 0x7fbe9019f680 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:27.574+0000 7fbe94e00700 1 -- 172.21.15.47:0/1911141133 >> 172.21.15.47:0/1911141133 conn(0x7fbe900fd670 msgr2=0x7fbe90105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:27.574+0000 7fbe94e00700 1 -- 172.21.15.47:0/1911141133 shutdown_connections 2024-03-21T11:13:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:27.574+0000 7fbe94e00700 1 -- 172.21.15.47:0/1911141133 wait complete. 2024-03-21T11:13:28.299 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-21T11:13:28.760 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:13:28.917 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:28 smithi047 ceph-mon[31351]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:28.917 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:28 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:28.917 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:28 smithi047 ceph-mon[31351]: Added host smithi102 2024-03-21T11:13:28.917 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:28 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:28.918 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:28 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:29.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.681+0000 7f93b4600700 1 -- 172.21.15.47:0/1580146411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ac102d10 msgr2=0x7f93ac1030f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.681+0000 7f93b4600700 1 --2- 172.21.15.47:0/1580146411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ac102d10 0x7f93ac1030f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009910 tx=0x7f93a0009c20 comp rx=0 tx=0).stop 2024-03-21T11:13:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.681+0000 7f93b4600700 1 -- 172.21.15.47:0/1580146411 shutdown_connections 2024-03-21T11:13:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.681+0000 7f93b4600700 1 --2- 172.21.15.47:0/1580146411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ac102d10 0x7f93ac1030f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.681+0000 7f93b4600700 1 -- 172.21.15.47:0/1580146411 >> 172.21.15.47:0/1580146411 conn(0x7f93ac0fc540 msgr2=0x7f93ac0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.681+0000 7f93b4600700 1 -- 172.21.15.47:0/1580146411 shutdown_connections 2024-03-21T11:13:30.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.681+0000 7f93b4600700 1 -- 172.21.15.47:0/1580146411 wait complete. 2024-03-21T11:13:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.683+0000 7f93b4600700 1 Processor -- start 2024-03-21T11:13:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.683+0000 7f93b4600700 1 -- start start 2024-03-21T11:13:30.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.684+0000 7f93b4600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ac102d10 0x7f93ac100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.684+0000 7f93b4600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac10c200 con 0x7f93ac102d10 2024-03-21T11:13:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.684+0000 7f93b1c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ac102d10 0x7f93ac100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.684+0000 7f93b1c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ac102d10 0x7f93ac100970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51524/0 (socket says 172.21.15.47:51524) 2024-03-21T11:13:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.684+0000 7f93b1c00700 1 -- 172.21.15.47:0/3194961223 learned_addr learned my addr 172.21.15.47:0/3194961223 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:13:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.685+0000 7f93b1c00700 1 -- 172.21.15.47:0/3194961223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac102860 con 0x7f93ac102d10 2024-03-21T11:13:30.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.685+0000 7f93b1c00700 1 --2- 172.21.15.47:0/3194961223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ac102d10 0x7f93ac100970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f93a0006010 tx=0x7f93a0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.685+0000 7f93aa000700 1 -- 172.21.15.47:0/3194961223 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93a0026070 con 0x7f93ac102d10 2024-03-21T11:13:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.685+0000 7f93aa000700 1 -- 172.21.15.47:0/3194961223 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93a000bc30 con 0x7f93ac102d10 2024-03-21T11:13:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.685+0000 7f93aa000700 1 -- 172.21.15.47:0/3194961223 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93a00039a0 con 0x7f93ac102d10 2024-03-21T11:13:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.685+0000 7f93b4600700 1 -- 172.21.15.47:0/3194961223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ac100eb0 con 0x7f93ac102d10 2024-03-21T11:13:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.686+0000 7f93b4600700 1 -- 172.21.15.47:0/3194961223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ac101370 con 0x7f93ac102d10 2024-03-21T11:13:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.687+0000 7f93b4600700 1 -- 172.21.15.47:0/3194961223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ac06a6f0 con 0x7f93ac102d10 2024-03-21T11:13:30.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.687+0000 7f93aa000700 1 -- 172.21.15.47:0/3194961223 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f93a0029070 con 0x7f93ac102d10 2024-03-21T11:13:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.688+0000 7f93aa000700 1 --2- 172.21.15.47:0/3194961223 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9394039280 0x7f939403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.688+0000 7f93aa000700 1 -- 172.21.15.47:0/3194961223 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f93a001e070 con 0x7f93ac102d10 2024-03-21T11:13:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.688+0000 7f93b1200700 1 --2- 172.21.15.47:0/3194961223 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9394039280 0x7f939403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.689+0000 7f93b1200700 1 --2- 172.21.15.47:0/3194961223 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9394039280 0x7f939403b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9398006fd0 tx=0x7f9398006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.693+0000 7f93aa000700 1 -- 172.21.15.47:0/3194961223 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f93a0013070 con 0x7f93ac102d10 2024-03-21T11:13:30.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:30 smithi047 ceph-mon[31351]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:30.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:30 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:30.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.930+0000 7f93b4600700 1 -- 172.21.15.47:0/3194961223 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93ac101f10 con 0x7f9394039280 2024-03-21T11:13:30.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.931+0000 7f93aa000700 1 -- 172.21.15.47:0/3194961223 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f93ac101f10 con 0x7f9394039280 2024-03-21T11:13:30.932 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:13:30.932 INFO:teuthology.orchestra.run.smithi047.stdout:[{"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}, {"addr": "172.21.15.102", "hostname": "smithi102", "labels": [], "status": ""}] 2024-03-21T11:13:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.935+0000 7f93b4600700 1 -- 172.21.15.47:0/3194961223 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9394039280 msgr2=0x7f939403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.935+0000 7f93b4600700 1 --2- 172.21.15.47:0/3194961223 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9394039280 0x7f939403b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9398006fd0 tx=0x7f9398006e40 comp rx=0 tx=0).stop 2024-03-21T11:13:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.935+0000 7f93b4600700 1 -- 172.21.15.47:0/3194961223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ac102d10 msgr2=0x7f93ac100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.935+0000 7f93b4600700 1 --2- 172.21.15.47:0/3194961223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ac102d10 0x7f93ac100970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f93a0006010 tx=0x7f93a0005030 comp rx=0 tx=0).stop 2024-03-21T11:13:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.935+0000 7f93b4600700 1 -- 172.21.15.47:0/3194961223 shutdown_connections 2024-03-21T11:13:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.935+0000 7f93b4600700 1 --2- 172.21.15.47:0/3194961223 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9394039280 0x7f939403b740 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.935+0000 7f93b4600700 1 --2- 172.21.15.47:0/3194961223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93ac102d10 0x7f93ac100970 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.935+0000 7f93b4600700 1 -- 172.21.15.47:0/3194961223 >> 172.21.15.47:0/3194961223 conn(0x7f93ac0fc540 msgr2=0x7f93ac103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.935+0000 7f93b4600700 1 -- 172.21.15.47:0/3194961223 shutdown_connections 2024-03-21T11:13:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:30.935+0000 7f93b4600700 1 -- 172.21.15.47:0/3194961223 wait complete. 2024-03-21T11:13:31.540 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-21T11:13:31.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-21T11:13:31.987 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:13:33.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:32 smithi047 ceph-mon[31351]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:33.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:32 smithi047 ceph-mon[31351]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:13:33.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.971+0000 7f40c5800700 1 -- 172.21.15.47:0/3330594664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c01083f0 msgr2=0x7f40c01087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:33.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.971+0000 7f40c5800700 1 --2- 172.21.15.47:0/3330594664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c01083f0 0x7f40c01087d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f40b000b0a0 tx=0x7f40b000b3b0 comp rx=0 tx=0).stop 2024-03-21T11:13:33.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.971+0000 7f40c5800700 1 -- 172.21.15.47:0/3330594664 shutdown_connections 2024-03-21T11:13:33.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.971+0000 7f40c5800700 1 --2- 172.21.15.47:0/3330594664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c01083f0 0x7f40c01087d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:33.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.971+0000 7f40c5800700 1 -- 172.21.15.47:0/3330594664 >> 172.21.15.47:0/3330594664 conn(0x7f40c0077930 msgr2=0x7f40c0079d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:33.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.971+0000 7f40c5800700 1 -- 172.21.15.47:0/3330594664 shutdown_connections 2024-03-21T11:13:33.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.971+0000 7f40c5800700 1 -- 172.21.15.47:0/3330594664 wait complete. 2024-03-21T11:13:33.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.972+0000 7f40c5800700 1 Processor -- start 2024-03-21T11:13:33.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.972+0000 7f40c5800700 1 -- start start 2024-03-21T11:13:33.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.972+0000 7f40c5800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c01083f0 0x7f40c01a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:33.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.973+0000 7f40c5800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c01127c0 con 0x7f40c01083f0 2024-03-21T11:13:33.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.973+0000 7f40bea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c01083f0 0x7f40c01a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:33.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.973+0000 7f40bea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c01083f0 0x7f40c01a38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51544/0 (socket says 172.21.15.47:51544) 2024-03-21T11:13:33.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.973+0000 7f40bea00700 1 -- 172.21.15.47:0/1463655696 learned_addr learned my addr 172.21.15.47:0/1463655696 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:13:33.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.973+0000 7f40bea00700 1 -- 172.21.15.47:0/1463655696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c01a3de0 con 0x7f40c01083f0 2024-03-21T11:13:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.973+0000 7f40bea00700 1 --2- 172.21.15.47:0/1463655696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c01083f0 0x7f40c01a38a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40b0015040 tx=0x7f40b0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.974+0000 7f40b7400700 1 -- 172.21.15.47:0/1463655696 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40b0017040 con 0x7f40c01083f0 2024-03-21T11:13:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.974+0000 7f40b7400700 1 -- 172.21.15.47:0/1463655696 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40b0022070 con 0x7f40c01083f0 2024-03-21T11:13:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.974+0000 7f40c5800700 1 -- 172.21.15.47:0/1463655696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c01a4070 con 0x7f40c01083f0 2024-03-21T11:13:33.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.974+0000 7f40b7400700 1 -- 172.21.15.47:0/1463655696 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40b001d5d0 con 0x7f40c01083f0 2024-03-21T11:13:33.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.974+0000 7f40c5800700 1 -- 172.21.15.47:0/1463655696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c019dab0 con 0x7f40c01083f0 2024-03-21T11:13:33.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.975+0000 7f40b7400700 1 -- 172.21.15.47:0/1463655696 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f40b0004000 con 0x7f40c01083f0 2024-03-21T11:13:33.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.975+0000 7f40c5800700 1 -- 172.21.15.47:0/1463655696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c01098d0 con 0x7f40c01083f0 2024-03-21T11:13:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.976+0000 7f40b7400700 1 --2- 172.21.15.47:0/1463655696 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40a4039280 0x7f40a403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.976+0000 7f40b7400700 1 -- 172.21.15.47:0/1463655696 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f40b000f050 con 0x7f40c01083f0 2024-03-21T11:13:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.976+0000 7f40be000700 1 --2- 172.21.15.47:0/1463655696 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40a4039280 0x7f40a403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.977+0000 7f40be000700 1 --2- 172.21.15.47:0/1463655696 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40a4039280 0x7f40a403b740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f40ac006fd0 tx=0x7f40ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:33.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:33.982+0000 7f40b7400700 1 -- 172.21.15.47:0/1463655696 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f40b0007e20 con 0x7f40c01083f0 2024-03-21T11:13:34.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:34.220+0000 7f40c5800700 1 -- 172.21.15.47:0/1463655696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f40c019ef90 con 0x7f40c01083f0 2024-03-21T11:13:34.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:34.703+0000 7f40b7400700 1 -- 172.21.15.47:0/1463655696 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f40b000bcc0 con 0x7f40c01083f0 2024-03-21T11:13:34.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:34.707+0000 7f40c5800700 1 -- 172.21.15.47:0/1463655696 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40a4039280 msgr2=0x7f40a403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:34.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:34.707+0000 7f40c5800700 1 --2- 172.21.15.47:0/1463655696 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40a4039280 0x7f40a403b740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f40ac006fd0 tx=0x7f40ac006e40 comp rx=0 tx=0).stop 2024-03-21T11:13:34.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:34.707+0000 7f40c5800700 1 -- 172.21.15.47:0/1463655696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c01083f0 msgr2=0x7f40c01a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:34.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:34.707+0000 7f40c5800700 1 --2- 172.21.15.47:0/1463655696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c01083f0 0x7f40c01a38a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40b0015040 tx=0x7f40b0003d50 comp rx=0 tx=0).stop 2024-03-21T11:13:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:34.707+0000 7f40c5800700 1 -- 172.21.15.47:0/1463655696 shutdown_connections 2024-03-21T11:13:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:34.707+0000 7f40c5800700 1 --2- 172.21.15.47:0/1463655696 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40a4039280 0x7f40a403b740 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:34.707+0000 7f40c5800700 1 --2- 172.21.15.47:0/1463655696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40c01083f0 0x7f40c01a38a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:34.707+0000 7f40c5800700 1 -- 172.21.15.47:0/1463655696 >> 172.21.15.47:0/1463655696 conn(0x7f40c0077930 msgr2=0x7f40c0109dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:34.708+0000 7f40c5800700 1 -- 172.21.15.47:0/1463655696 shutdown_connections 2024-03-21T11:13:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:13:34.708+0000 7f40c5800700 1 -- 172.21.15.47:0/1463655696 wait complete. 2024-03-21T11:13:34.710 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2024-03-21T11:13:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:34 smithi047 ceph-mon[31351]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:34 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:34 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:34 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:34 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:34 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:13:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:34 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:34 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:13:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:34 smithi047 ceph-mon[31351]: Updating smithi102:/etc/ceph/ceph.conf 2024-03-21T11:13:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:34 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1463655696' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T11:13:35.520 INFO:tasks.cephadm:Adding mon.a on smithi047 2024-03-21T11:13:35.520 INFO:tasks.cephadm:Adding mon.b on smithi100 2024-03-21T11:13:35.520 INFO:tasks.cephadm:Adding mon.c on smithi102 2024-03-21T11:13:35.520 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi102:172.21.15.102=c' 2024-03-21T11:13:35.750 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:13:35.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:35 smithi047 ceph-mon[31351]: Updating smithi102:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:13:35.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:35 smithi047 ceph-mon[31351]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-03-21T11:13:35.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:35 smithi047 ceph-mon[31351]: Updating smithi102:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T11:13:35.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:35 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1463655696' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T11:13:35.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:35 smithi047 ceph-mon[31351]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T11:13:35.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:35.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:35.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:37.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:36 smithi047 ceph-mon[31351]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:37.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.141+0000 7f8091652700 1 -- 172.21.15.102:0/1679498271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c101dd0 msgr2=0x7f808c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:37.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.141+0000 7f8091652700 1 --2- 172.21.15.102:0/1679498271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c101dd0 0x7f808c1021b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f807c009910 tx=0x7f807c009c20 comp rx=0 tx=0).stop 2024-03-21T11:13:37.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.141+0000 7f8091652700 1 -- 172.21.15.102:0/1679498271 shutdown_connections 2024-03-21T11:13:37.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.141+0000 7f8091652700 1 --2- 172.21.15.102:0/1679498271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c101dd0 0x7f808c1021b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:37.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.141+0000 7f8091652700 1 -- 172.21.15.102:0/1679498271 >> 172.21.15.102:0/1679498271 conn(0x7f808c0fd6a0 msgr2=0x7f808c0ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:37.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.141+0000 7f8091652700 1 -- 172.21.15.102:0/1679498271 shutdown_connections 2024-03-21T11:13:37.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.141+0000 7f8091652700 1 -- 172.21.15.102:0/1679498271 wait complete. 2024-03-21T11:13:37.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.142+0000 7f8091652700 1 Processor -- start 2024-03-21T11:13:37.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.142+0000 7f8091652700 1 -- start start 2024-03-21T11:13:37.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.142+0000 7f8091652700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c101dd0 0x7f808c1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:37.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.142+0000 7f8091652700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c10c1a0 con 0x7f808c101dd0 2024-03-21T11:13:37.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.143+0000 7f808affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c101dd0 0x7f808c1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:37.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.143+0000 7f808affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c101dd0 0x7f808c1976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.102:38696/0 (socket says 172.21.15.102:38696) 2024-03-21T11:13:37.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.143+0000 7f808affd700 1 -- 172.21.15.102:0/3479875698 learned_addr learned my addr 172.21.15.102:0/3479875698 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-21T11:13:37.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.143+0000 7f808affd700 1 -- 172.21.15.102:0/3479875698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c197be0 con 0x7f808c101dd0 2024-03-21T11:13:37.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.144+0000 7f808affd700 1 --2- 172.21.15.102:0/3479875698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c101dd0 0x7f808c1976a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f807c005850 tx=0x7f807c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:37.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.144+0000 7f8073fff700 1 -- 172.21.15.102:0/3479875698 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f807c026070 con 0x7f808c101dd0 2024-03-21T11:13:37.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.144+0000 7f8091652700 1 -- 172.21.15.102:0/3479875698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808c197ed0 con 0x7f808c101dd0 2024-03-21T11:13:37.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.144+0000 7f8091652700 1 -- 172.21.15.102:0/3479875698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c19bf70 con 0x7f808c101dd0 2024-03-21T11:13:37.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.144+0000 7f8073fff700 1 -- 172.21.15.102:0/3479875698 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f807c00b6f0 con 0x7f808c101dd0 2024-03-21T11:13:37.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.144+0000 7f8073fff700 1 -- 172.21.15.102:0/3479875698 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f807c030060 con 0x7f808c101dd0 2024-03-21T11:13:37.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.145+0000 7f8091652700 1 -- 172.21.15.102:0/3479875698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8078005330 con 0x7f808c101dd0 2024-03-21T11:13:37.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.145+0000 7f8073fff700 1 -- 172.21.15.102:0/3479875698 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f807c029070 con 0x7f808c101dd0 2024-03-21T11:13:37.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.146+0000 7f8073fff700 1 --2- 172.21.15.102:0/3479875698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8074039220 0x7f807403b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:37.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.146+0000 7f8073fff700 1 -- 172.21.15.102:0/3479875698 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f807c01e070 con 0x7f808c101dd0 2024-03-21T11:13:37.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.146+0000 7f808a7fc700 1 --2- 172.21.15.102:0/3479875698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8074039220 0x7f807403b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:37.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.147+0000 7f808a7fc700 1 --2- 172.21.15.102:0/3479875698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8074039220 0x7f807403b6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8080006fd0 tx=0x7f8080006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:37.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.160+0000 7f8073fff700 1 -- 172.21.15.102:0/3479875698 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f807c013070 con 0x7f808c101dd0 2024-03-21T11:13:37.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.366+0000 7f8091652700 1 -- 172.21.15.102:0/3479875698 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi102:172.21.15.102=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f8078000c80 con 0x7f8074039220 2024-03-21T11:13:37.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.381+0000 7f8073fff700 1 -- 172.21.15.102:0/3479875698 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8078000c80 con 0x7f8074039220 2024-03-21T11:13:37.381 INFO:teuthology.orchestra.run.smithi102.stdout:Scheduled mon update... 2024-03-21T11:13:37.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.383+0000 7f8091652700 1 -- 172.21.15.102:0/3479875698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8074039220 msgr2=0x7f807403b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:37.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.383+0000 7f8091652700 1 --2- 172.21.15.102:0/3479875698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8074039220 0x7f807403b6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8080006fd0 tx=0x7f8080006e40 comp rx=0 tx=0).stop 2024-03-21T11:13:37.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.383+0000 7f8091652700 1 -- 172.21.15.102:0/3479875698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c101dd0 msgr2=0x7f808c1976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:37.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.383+0000 7f8091652700 1 --2- 172.21.15.102:0/3479875698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c101dd0 0x7f808c1976a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f807c005850 tx=0x7f807c005030 comp rx=0 tx=0).stop 2024-03-21T11:13:37.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.384+0000 7f8091652700 1 -- 172.21.15.102:0/3479875698 shutdown_connections 2024-03-21T11:13:37.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.384+0000 7f8091652700 1 --2- 172.21.15.102:0/3479875698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8074039220 0x7f807403b6e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:37.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.384+0000 7f8091652700 1 --2- 172.21.15.102:0/3479875698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c101dd0 0x7f808c1976a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:37.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.384+0000 7f8091652700 1 -- 172.21.15.102:0/3479875698 >> 172.21.15.102:0/3479875698 conn(0x7f808c0fd6a0 msgr2=0x7f808c1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:37.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.384+0000 7f8091652700 1 -- 172.21.15.102:0/3479875698 shutdown_connections 2024-03-21T11:13:37.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:37.384+0000 7f8091652700 1 -- 172.21.15.102:0/3479875698 wait complete. 2024-03-21T11:13:37.954 DEBUG:teuthology.orchestra.run.smithi100:mon.b> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.b.service 2024-03-21T11:13:37.957 DEBUG:teuthology.orchestra.run.smithi102:mon.c> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.c.service 2024-03-21T11:13:37.959 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T11:13:37.959 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T11:13:38.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:38 smithi047 ceph-mon[31351]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:38 smithi047 ceph-mon[31351]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi102:172.21.15.102=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:13:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:38 smithi047 ceph-mon[31351]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi102:172.21.15.102=c;count:3 2024-03-21T11:13:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:13:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:13:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:38 smithi047 ceph-mon[31351]: Deploying daemon mon.c on smithi102 2024-03-21T11:13:39.589 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:13:40.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:40 smithi047 ceph-mon[31351]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:41.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:41 smithi102 systemd[1]: Starting Ceph mon.c for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:13:41.782 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:41 smithi102 podman[36703]: 2024-03-21 11:13:41.504022348 +0000 UTC m=+0.147156285 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:13:41.782 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:41 smithi102 podman[36703]: 2024-03-21 11:13:41.658727208 +0000 UTC m=+0.301861133 container create a078e02443605c8f261a67104620ab2ccf2359441cb0ec09f640ca58acbf694d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T11:13:41.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.978+0000 7fb20f87d700 1 -- 172.21.15.102:0/506760203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb208101e00 msgr2=0x7fb2081021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:41.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.978+0000 7fb20f87d700 1 --2- 172.21.15.102:0/506760203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb208101e00 0x7fb2081021e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb204009910 tx=0x7fb204009c20 comp rx=0 tx=0).stop 2024-03-21T11:13:41.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.978+0000 7fb20f87d700 1 -- 172.21.15.102:0/506760203 shutdown_connections 2024-03-21T11:13:41.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.978+0000 7fb20f87d700 1 --2- 172.21.15.102:0/506760203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb208101e00 0x7fb2081021e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:41.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.978+0000 7fb20f87d700 1 -- 172.21.15.102:0/506760203 >> 172.21.15.102:0/506760203 conn(0x7fb2080fd6b0 msgr2=0x7fb2080ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:41.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.978+0000 7fb20f87d700 1 -- 172.21.15.102:0/506760203 shutdown_connections 2024-03-21T11:13:41.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.978+0000 7fb20f87d700 1 -- 172.21.15.102:0/506760203 wait complete. 2024-03-21T11:13:41.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.979+0000 7fb20f87d700 1 Processor -- start 2024-03-21T11:13:41.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.979+0000 7fb20f87d700 1 -- start start 2024-03-21T11:13:41.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.979+0000 7fb20f87d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb208101e00 0x7fb208074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:41.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.979+0000 7fb20f87d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20810c1d0 con 0x7fb208101e00 2024-03-21T11:13:41.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.979+0000 7fb20d619700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb208101e00 0x7fb208074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:41.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.980+0000 7fb20d619700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb208101e00 0x7fb208074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.102:38714/0 (socket says 172.21.15.102:38714) 2024-03-21T11:13:41.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.980+0000 7fb20d619700 1 -- 172.21.15.102:0/3691317992 learned_addr learned my addr 172.21.15.102:0/3691317992 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-21T11:13:41.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.980+0000 7fb20d619700 1 -- 172.21.15.102:0/3691317992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb208076380 con 0x7fb208101e00 2024-03-21T11:13:41.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.980+0000 7fb20d619700 1 --2- 172.21.15.102:0/3691317992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb208101e00 0x7fb208074490 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb204006010 tx=0x7fb204005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:41.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.981+0000 7fb1fe7fc700 1 -- 172.21.15.102:0/3691317992 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb204026070 con 0x7fb208101e00 2024-03-21T11:13:41.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.981+0000 7fb20f87d700 1 -- 172.21.15.102:0/3691317992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb208074a30 con 0x7fb208101e00 2024-03-21T11:13:41.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.981+0000 7fb20f87d700 1 -- 172.21.15.102:0/3691317992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb208074ef0 con 0x7fb208101e00 2024-03-21T11:13:41.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.981+0000 7fb1fe7fc700 1 -- 172.21.15.102:0/3691317992 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb20400bbd0 con 0x7fb208101e00 2024-03-21T11:13:41.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.981+0000 7fb1fe7fc700 1 -- 172.21.15.102:0/3691317992 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb204030060 con 0x7fb208101e00 2024-03-21T11:13:41.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.982+0000 7fb20f87d700 1 -- 172.21.15.102:0/3691317992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2081032e0 con 0x7fb208101e00 2024-03-21T11:13:41.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.982+0000 7fb1fe7fc700 1 -- 172.21.15.102:0/3691317992 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fb204029070 con 0x7fb208101e00 2024-03-21T11:13:41.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.983+0000 7fb1fe7fc700 1 --2- 172.21.15.102:0/3691317992 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1f4039220 0x7fb1f403b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:41.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.983+0000 7fb1fe7fc700 1 -- 172.21.15.102:0/3691317992 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb20401e070 con 0x7fb208101e00 2024-03-21T11:13:41.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.983+0000 7fb20ce18700 1 --2- 172.21.15.102:0/3691317992 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1f4039220 0x7fb1f403b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:41.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.984+0000 7fb20ce18700 1 --2- 172.21.15.102:0/3691317992 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1f4039220 0x7fb1f403b6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8006fd0 tx=0x7fb1f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:41.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:41.998+0000 7fb1fe7fc700 1 -- 172.21.15.102:0/3691317992 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb204013070 con 0x7fb208101e00 2024-03-21T11:13:42.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:42.113+0000 7fb1fe7fc700 1 -- 172.21.15.102:0/3691317992 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fb204021ca0 con 0x7fb208101e00 2024-03-21T11:13:42.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 podman[36703]: 2024-03-21 11:13:42.045235031 +0000 UTC m=+0.688368975 container init a078e02443605c8f261a67104620ab2ccf2359441cb0ec09f640ca58acbf694d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-c, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T11:13:42.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 podman[36703]: 2024-03-21 11:13:42.049141275 +0000 UTC m=+0.692275212 container start a078e02443605c8f261a67104620ab2ccf2359441cb0ec09f640ca58acbf694d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5) 2024-03-21T11:13:42.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T11:13:42.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T11:13:42.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: pidfile_write: ignore empty --pid-file 2024-03-21T11:13:42.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: load: jerasure load: lrc 2024-03-21T11:13:42.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: RocksDB version: 7.9.2 2024-03-21T11:13:42.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Git sha 0 2024-03-21T11:13:42.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T11:13:42.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: DB SUMMARY 2024-03-21T11:13:42.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: DB Session ID: 0OTV6CLHJM5SH275XOYL 2024-03-21T11:13:42.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: CURRENT file: CURRENT 2024-03-21T11:13:42.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T11:13:42.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-21T11:13:42.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-21T11:13:42.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-21T11:13:42.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.error_if_exists: 0 2024-03-21T11:13:42.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.create_if_missing: 0 2024-03-21T11:13:42.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.paranoid_checks: 1 2024-03-21T11:13:42.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T11:13:42.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T11:13:42.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T11:13:42.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.env: 0x55f43ee6ab20 2024-03-21T11:13:42.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T11:13:42.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.info_log: 0x55f440253920 2024-03-21T11:13:42.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T11:13:42.131 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.statistics: (nil) 2024-03-21T11:13:42.132 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.use_fsync: 0 2024-03-21T11:13:42.132 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_log_file_size: 0 2024-03-21T11:13:42.132 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T11:13:42.132 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T11:13:42.132 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T11:13:42.132 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T11:13:42.132 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.allow_fallocate: 1 2024-03-21T11:13:42.132 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T11:13:42.132 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T11:13:42.132 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.use_direct_reads: 0 2024-03-21T11:13:42.132 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T11:13:42.132 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T11:13:42.133 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.db_log_dir: 2024-03-21T11:13:42.133 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.wal_dir: 2024-03-21T11:13:42.133 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T11:13:42.133 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T11:13:42.133 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T11:13:42.133 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T11:13:42.133 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T11:13:42.133 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T11:13:42.133 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T11:13:42.133 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T11:13:42.133 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.write_buffer_manager: 0x55f4402261e0 2024-03-21T11:13:42.133 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T11:13:42.133 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T11:13:42.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T11:13:42.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T11:13:42.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T11:13:42.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T11:13:42.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T11:13:42.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T11:13:42.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.unordered_write: 0 2024-03-21T11:13:42.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T11:13:42.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T11:13:42.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T11:13:42.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T11:13:42.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.row_cache: None 2024-03-21T11:13:42.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.wal_filter: None 2024-03-21T11:13:42.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T11:13:42.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T11:13:42.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.two_write_queues: 0 2024-03-21T11:13:42.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T11:13:42.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.wal_compression: 0 2024-03-21T11:13:42.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.atomic_flush: 0 2024-03-21T11:13:42.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T11:13:42.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T11:13:42.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T11:13:42.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.log_readahead_size: 0 2024-03-21T11:13:42.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T11:13:42.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T11:13:42.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T11:13:42.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T11:13:42.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T11:13:42.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T11:13:42.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T11:13:42.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_background_jobs: 2 2024-03-21T11:13:42.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_background_compactions: -1 2024-03-21T11:13:42.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_subcompactions: 1 2024-03-21T11:13:42.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T11:13:42.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T11:13:42.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T11:13:42.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T11:13:42.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T11:13:42.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T11:13:42.136 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T11:13:42.137 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T11:13:42.137 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_open_files: -1 2024-03-21T11:13:42.137 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T11:13:42.137 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T11:13:42.137 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T11:13:42.137 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T11:13:42.137 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_background_flushes: -1 2024-03-21T11:13:42.137 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Compression algorithms supported: 2024-03-21T11:13:42.137 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T11:13:42.137 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: kZSTD supported: 0 2024-03-21T11:13:42.137 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: kXpressCompression supported: 0 2024-03-21T11:13:42.137 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T11:13:42.137 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: kZlibCompression supported: 1 2024-03-21T11:13:42.137 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: kSnappyCompression supported: 1 2024-03-21T11:13:42.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: kLZ4Compression supported: 1 2024-03-21T11:13:42.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: kBZip2Compression supported: 0 2024-03-21T11:13:42.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T11:13:42.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T11:13:42.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-21T11:13:42.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T11:13:42.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T11:13:42.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.merge_operator: 2024-03-21T11:13:42.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compaction_filter: None 2024-03-21T11:13:42.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T11:13:42.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T11:13:42.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T11:13:42.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T11:13:42.139 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f440253a20) 2024-03-21T11:13:42.139 INFO:journalctl@ceph.mon.c.smithi102.stdout: cache_index_and_filter_blocks: 1 2024-03-21T11:13:42.139 INFO:journalctl@ceph.mon.c.smithi102.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T11:13:42.139 INFO:journalctl@ceph.mon.c.smithi102.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T11:13:42.139 INFO:journalctl@ceph.mon.c.smithi102.stdout: pin_top_level_index_and_filter: 1 2024-03-21T11:13:42.139 INFO:journalctl@ceph.mon.c.smithi102.stdout: index_type: 0 2024-03-21T11:13:42.139 INFO:journalctl@ceph.mon.c.smithi102.stdout: data_block_index_type: 0 2024-03-21T11:13:42.139 INFO:journalctl@ceph.mon.c.smithi102.stdout: index_shortening: 1 2024-03-21T11:13:42.139 INFO:journalctl@ceph.mon.c.smithi102.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T11:13:42.139 INFO:journalctl@ceph.mon.c.smithi102.stdout: checksum: 4 2024-03-21T11:13:42.139 INFO:journalctl@ceph.mon.c.smithi102.stdout: no_block_cache: 0 2024-03-21T11:13:42.140 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache: 0x55f440259090 2024-03-21T11:13:42.140 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache_name: BinnedLRUCache 2024-03-21T11:13:42.140 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache_options: 2024-03-21T11:13:42.140 INFO:journalctl@ceph.mon.c.smithi102.stdout: capacity : 536870912 2024-03-21T11:13:42.140 INFO:journalctl@ceph.mon.c.smithi102.stdout: num_shard_bits : 4 2024-03-21T11:13:42.140 INFO:journalctl@ceph.mon.c.smithi102.stdout: strict_capacity_limit : 0 2024-03-21T11:13:42.140 INFO:journalctl@ceph.mon.c.smithi102.stdout: high_pri_pool_ratio: 0.000 2024-03-21T11:13:42.140 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache_compressed: (nil) 2024-03-21T11:13:42.140 INFO:journalctl@ceph.mon.c.smithi102.stdout: persistent_cache: (nil) 2024-03-21T11:13:42.140 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_size: 4096 2024-03-21T11:13:42.140 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_size_deviation: 10 2024-03-21T11:13:42.140 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_restart_interval: 16 2024-03-21T11:13:42.140 INFO:journalctl@ceph.mon.c.smithi102.stdout: index_block_restart_interval: 1 2024-03-21T11:13:42.141 INFO:journalctl@ceph.mon.c.smithi102.stdout: metadata_block_size: 4096 2024-03-21T11:13:42.141 INFO:journalctl@ceph.mon.c.smithi102.stdout: partition_filters: 0 2024-03-21T11:13:42.141 INFO:journalctl@ceph.mon.c.smithi102.stdout: use_delta_encoding: 1 2024-03-21T11:13:42.141 INFO:journalctl@ceph.mon.c.smithi102.stdout: filter_policy: bloomfilter 2024-03-21T11:13:42.141 INFO:journalctl@ceph.mon.c.smithi102.stdout: whole_key_filtering: 1 2024-03-21T11:13:42.141 INFO:journalctl@ceph.mon.c.smithi102.stdout: verify_compression: 0 2024-03-21T11:13:42.141 INFO:journalctl@ceph.mon.c.smithi102.stdout: read_amp_bytes_per_bit: 0 2024-03-21T11:13:42.141 INFO:journalctl@ceph.mon.c.smithi102.stdout: format_version: 5 2024-03-21T11:13:42.141 INFO:journalctl@ceph.mon.c.smithi102.stdout: enable_index_compression: 1 2024-03-21T11:13:42.141 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_align: 0 2024-03-21T11:13:42.141 INFO:journalctl@ceph.mon.c.smithi102.stdout: max_auto_readahead_size: 262144 2024-03-21T11:13:42.141 INFO:journalctl@ceph.mon.c.smithi102.stdout: prepopulate_block_cache: 0 2024-03-21T11:13:42.141 INFO:journalctl@ceph.mon.c.smithi102.stdout: initial_auto_readahead_size: 8192 2024-03-21T11:13:42.142 INFO:journalctl@ceph.mon.c.smithi102.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T11:13:42.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T11:13:42.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T11:13:42.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compression: NoCompression 2024-03-21T11:13:42.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T11:13:42.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T11:13:42.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T11:13:42.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.num_levels: 7 2024-03-21T11:13:42.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T11:13:42.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T11:13:42.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T11:13:42.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T11:13:42.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T11:13:42.143 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T11:13:42.143 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T11:13:42.143 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T11:13:42.143 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T11:13:42.143 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T11:13:42.143 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T11:13:42.143 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T11:13:42.143 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T11:13:42.143 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T11:13:42.143 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T11:13:42.143 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T11:13:42.143 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T11:13:42.143 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T11:13:42.144 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T11:13:42.144 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T11:13:42.144 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T11:13:42.144 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T11:13:42.144 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T11:13:42.144 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T11:13:42.144 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T11:13:42.144 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T11:13:42.144 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T11:13:42.144 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T11:13:42.144 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T11:13:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T11:13:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T11:13:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T11:13:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T11:13:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T11:13:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T11:13:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T11:13:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T11:13:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T11:13:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T11:13:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T11:13:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T11:13:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T11:13:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T11:13:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T11:13:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T11:13:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T11:13:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T11:13:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T11:13:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T11:13:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T11:13:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T11:13:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T11:13:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T11:13:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.table_properties_collectors: 2024-03-21T11:13:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.inplace_update_support: 0 2024-03-21T11:13:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T11:13:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T11:13:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T11:13:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T11:13:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.bloom_locality: 0 2024-03-21T11:13:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.max_successive_merges: 0 2024-03-21T11:13:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T11:13:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T11:13:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T11:13:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T11:13:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.ttl: 2592000 2024-03-21T11:13:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T11:13:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T11:13:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T11:13:42.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.enable_blob_files: false 2024-03-21T11:13:42.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.min_blob_size: 0 2024-03-21T11:13:42.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T11:13:42.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T11:13:42.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T11:13:42.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T11:13:42.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T11:13:42.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T11:13:42.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T11:13:42.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T11:13:42.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-21T11:13:42.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-21T11:13:42.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e5077d96-14c8-4524-b960-4918be04e94a 2024-03-21T11:13:42.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711019622078904, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-21T11:13:42.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-21T11:13:42.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711019622079324, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711019622, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e5077d96-14c8-4524-b960-4918be04e94a", "db_session_id": "0OTV6CLHJM5SH275XOYL", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-21T11:13:42.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711019622079400, "job": 1, "event": "recovery_finished"} 2024-03-21T11:13:42.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-21T11:13:42.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T11:13:42.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f44034c000 2024-03-21T11:13:42.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: DB pointer 0x55f440336000 2024-03-21T11:13:42.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T11:13:42.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T11:13:42.149 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** DB Stats ** 2024-03-21T11:13:42.149 INFO:journalctl@ceph.mon.c.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T11:13:42.149 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T11:13:42.149 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T11:13:42.150 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T11:13:42.150 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T11:13:42.150 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T11:13:42.150 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T11:13:42.150 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-03-21T11:13:42.150 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** Compaction Stats [default] ** 2024-03-21T11:13:42.150 INFO:journalctl@ceph.mon.c.smithi102.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T11:13:42.150 INFO:journalctl@ceph.mon.c.smithi102.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T11:13:42.150 INFO:journalctl@ceph.mon.c.smithi102.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T11:13:42.150 INFO:journalctl@ceph.mon.c.smithi102.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T11:13:42.150 INFO:journalctl@ceph.mon.c.smithi102.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T11:13:42.150 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-03-21T11:13:42.151 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** Compaction Stats [default] ** 2024-03-21T11:13:42.151 INFO:journalctl@ceph.mon.c.smithi102.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T11:13:42.151 INFO:journalctl@ceph.mon.c.smithi102.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T11:13:42.151 INFO:journalctl@ceph.mon.c.smithi102.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T11:13:42.151 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-03-21T11:13:42.151 INFO:journalctl@ceph.mon.c.smithi102.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T11:13:42.151 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-03-21T11:13:42.151 INFO:journalctl@ceph.mon.c.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T11:13:42.151 INFO:journalctl@ceph.mon.c.smithi102.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T11:13:42.151 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T11:13:42.151 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T11:13:42.151 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T11:13:42.151 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T11:13:42.151 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T11:13:42.152 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T11:13:42.152 INFO:journalctl@ceph.mon.c.smithi102.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T11:13:42.152 INFO:journalctl@ceph.mon.c.smithi102.stdout: Block cache BinnedLRUCache@0x55f440259090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-21T11:13:42.152 INFO:journalctl@ceph.mon.c.smithi102.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-21T11:13:42.152 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-03-21T11:13:42.152 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T11:13:42.152 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-21T11:13:42.152 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: using public_addr v2:172.21.15.102:0/0 -> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] 2024-03-21T11:13:42.152 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: starting mon.c rank -1 at public addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] at bind addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:13:42.152 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c@-1(???) e0 preinit fsid a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:13:42.152 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c@-1(synchronizing).mds e1 new map 2024-03-21T11:13:42.152 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-21T11:13:42.152 INFO:journalctl@ceph.mon.c.smithi102.stdout: e1 2024-03-21T11:13:42.152 INFO:journalctl@ceph.mon.c.smithi102.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T11:13:42.153 INFO:journalctl@ceph.mon.c.smithi102.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T11:13:42.153 INFO:journalctl@ceph.mon.c.smithi102.stdout: legacy client fscid: -1 2024-03-21T11:13:42.153 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-03-21T11:13:42.153 INFO:journalctl@ceph.mon.c.smithi102.stdout: No filesystems configured 2024-03-21T11:13:42.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-21T11:13:42.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-21T11:13:42.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-21T11:13:42.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-21T11:13:42.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-21T11:13:42.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-21T11:13:42.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T11:13:42.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T11:13:42.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T11:13:42.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T11:13:42.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:42.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: Added host smithi100 2024-03-21T11:13:42.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:42.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:42.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:42.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:13:42.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:42.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: Updating smithi100:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: Updating smithi100:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:13:42.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:42.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: Deploying cephadm binary to smithi102 2024-03-21T11:13:42.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:42.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:42.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: Added host smithi102 2024-03-21T11:13:42.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:42.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:42.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:42.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:13:42.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:42.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: Updating smithi102:/etc/ceph/ceph.conf 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1463655696' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: Updating smithi102:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: Updating smithi102:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1463655696' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:42.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:42.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi102:172.21.15.102=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:13:42.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi102:172.21.15.102=c;count:3 2024-03-21T11:13:42.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:42.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:42.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:13:42.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:42.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:13:42.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:42.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: Deploying daemon mon.c on smithi102 2024-03-21T11:13:42.158 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:42.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 ceph-mon[36748]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-21T11:13:42.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 bash[36703]: a078e02443605c8f261a67104620ab2ccf2359441cb0ec09f640ca58acbf694d 2024-03-21T11:13:42.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:42 smithi102 systemd[1]: Started Ceph mon.c for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:13:42.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:42.294+0000 7fb20f87d700 1 -- 172.21.15.102:0/3691317992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb208075a60 con 0x7fb208101e00 2024-03-21T11:13:47.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:47.147+0000 7fb1fe7fc700 1 -- 172.21.15.102:0/3691317992 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fb204020e30 con 0x7fb208101e00 2024-03-21T11:13:47.147 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:13:47.148 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":2,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","modified":"2024-03-21T11:13:42.112093Z","created":"2024-03-21T11:11:37.664890Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T11:13:47.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:47.150+0000 7fb20f87d700 1 -- 172.21.15.102:0/3691317992 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1f4039220 msgr2=0x7fb1f403b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:47.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:47.150+0000 7fb20f87d700 1 --2- 172.21.15.102:0/3691317992 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1f4039220 0x7fb1f403b6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8006fd0 tx=0x7fb1f8006e40 comp rx=0 tx=0).stop 2024-03-21T11:13:47.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:47.150+0000 7fb20f87d700 1 -- 172.21.15.102:0/3691317992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb208101e00 msgr2=0x7fb208074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:47.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:47.150+0000 7fb20f87d700 1 --2- 172.21.15.102:0/3691317992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb208101e00 0x7fb208074490 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb204006010 tx=0x7fb204005030 comp rx=0 tx=0).stop 2024-03-21T11:13:47.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:47.150+0000 7fb20f87d700 1 -- 172.21.15.102:0/3691317992 shutdown_connections 2024-03-21T11:13:47.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:47.150+0000 7fb20f87d700 1 --2- 172.21.15.102:0/3691317992 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1f4039220 0x7fb1f403b6e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:47.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:47.150+0000 7fb20f87d700 1 --2- 172.21.15.102:0/3691317992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb208101e00 0x7fb208074490 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:47.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:47.150+0000 7fb20f87d700 1 -- 172.21.15.102:0/3691317992 >> 172.21.15.102:0/3691317992 conn(0x7fb2080fd6b0 msgr2=0x7fb208104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:47.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:47.150+0000 7fb20f87d700 1 -- 172.21.15.102:0/3691317992 shutdown_connections 2024-03-21T11:13:47.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:47.150+0000 7fb20f87d700 1 -- 172.21.15.102:0/3691317992 wait complete. 2024-03-21T11:13:47.151 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 2 2024-03-21T11:13:47.456 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:13:47.456 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: mon.a calling monitor election 2024-03-21T11:13:47.456 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:47.456 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: from='client.? 172.21.15.102:0/3691317992' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T11:13:47.456 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:47.456 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:47.456 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:47.456 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: mon.c calling monitor election 2024-03-21T11:13:47.456 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:47.457 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:47.457 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:47.457 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:47.457 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T11:13:47.457 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: monmap e2: 2 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],c=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T11:13:47.457 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: fsmap 2024-03-21T11:13:47.457 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T11:13:47.457 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: mgrmap e13: x(active, since 58s) 2024-03-21T11:13:47.457 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: overall HEALTH_OK 2024-03-21T11:13:47.457 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:47.457 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:47.457 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:47.457 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:13:47.457 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:47.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:13:47.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: mon.a calling monitor election 2024-03-21T11:13:47.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:47.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: from='client.? 172.21.15.102:0/3691317992' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T11:13:47.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:47.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:47.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:47.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: mon.c calling monitor election 2024-03-21T11:13:47.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:47.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:47.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:47.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:47.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T11:13:47.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: monmap e2: 2 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],c=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T11:13:47.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: fsmap 2024-03-21T11:13:47.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T11:13:47.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: mgrmap e13: x(active, since 58s) 2024-03-21T11:13:47.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: overall HEALTH_OK 2024-03-21T11:13:47.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:47.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:47.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:47.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:13:47.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:48.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:48 smithi102 ceph-mon[36748]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:48.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:48 smithi102 ceph-mon[36748]: Deploying daemon mon.b on smithi100 2024-03-21T11:13:48.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:48.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:48 smithi047 ceph-mon[31351]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:48.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:48 smithi047 ceph-mon[31351]: Deploying daemon mon.b on smithi100 2024-03-21T11:13:48.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:48.714 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T11:13:48.714 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T11:13:48.953 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:13:50.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.317+0000 7f6d92cba700 1 -- 172.21.15.102:0/1931115366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8c102ea0 msgr2=0x7f6d8c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:50.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.317+0000 7f6d92cba700 1 --2- 172.21.15.102:0/1931115366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8c102ea0 0x7f6d8c103280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009910 tx=0x7f6d80009c20 comp rx=0 tx=0).stop 2024-03-21T11:13:50.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.317+0000 7f6d92cba700 1 -- 172.21.15.102:0/1931115366 shutdown_connections 2024-03-21T11:13:50.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.317+0000 7f6d92cba700 1 --2- 172.21.15.102:0/1931115366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8c102ea0 0x7f6d8c103280 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:50.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.317+0000 7f6d92cba700 1 -- 172.21.15.102:0/1931115366 >> 172.21.15.102:0/1931115366 conn(0x7f6d8c0fc630 msgr2=0x7f6d8c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:50.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.317+0000 7f6d92cba700 1 -- 172.21.15.102:0/1931115366 shutdown_connections 2024-03-21T11:13:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.317+0000 7f6d92cba700 1 -- 172.21.15.102:0/1931115366 wait complete. 2024-03-21T11:13:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.317+0000 7f6d92cba700 1 Processor -- start 2024-03-21T11:13:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.317+0000 7f6d92cba700 1 -- start start 2024-03-21T11:13:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.318+0000 7f6d92cba700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8c102ea0 0x7f6d8c100b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.318+0000 7f6d92cba700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c101040 0x7f6d8c1014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.318+0000 7f6d92cba700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c10c300 con 0x7f6d8c102ea0 2024-03-21T11:13:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.318+0000 7f6d92cba700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c10c180 con 0x7f6d8c101040 2024-03-21T11:13:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.318+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c101040 0x7f6d8c1014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.318+0000 7f6d8bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c101040 0x7f6d8c1014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:46292/0 (socket says 172.21.15.102:46292) 2024-03-21T11:13:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.318+0000 7f6d8bfff700 1 -- 172.21.15.102:0/2170141689 learned_addr learned my addr 172.21.15.102:0/2170141689 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-21T11:13:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.318+0000 7f6d8bfff700 1 -- 172.21.15.102:0/2170141689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c101040 msgr2=0x7f6d8c1014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-21T11:13:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.318+0000 7f6d8bfff700 1 -- 172.21.15.102:0/2170141689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c101040 msgr2=0x7f6d8c1014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-21T11:13:50.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.318+0000 7f6d8bfff700 1 --2- 172.21.15.102:0/2170141689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c101040 0x7f6d8c1014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T11:13:50.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.318+0000 7f6d8bfff700 1 --2- 172.21.15.102:0/2170141689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c101040 0x7f6d8c1014c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T11:13:50.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.318+0000 7f6d90a56700 1 --2- 172.21.15.102:0/2170141689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8c102ea0 0x7f6d8c100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.319+0000 7f6d90a56700 1 -- 172.21.15.102:0/2170141689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c101040 msgr2=0x7f6d8c1014c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:13:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.319+0000 7f6d90a56700 1 --2- 172.21.15.102:0/2170141689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c101040 0x7f6d8c1014c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.319+0000 7f6d90a56700 1 -- 172.21.15.102:0/2170141689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c102a20 con 0x7f6d8c102ea0 2024-03-21T11:13:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.319+0000 7f6d90a56700 1 --2- 172.21.15.102:0/2170141689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8c102ea0 0x7f6d8c100b00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6d80014000 tx=0x7f6d80005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.320+0000 7f6d89ffb700 1 -- 172.21.15.102:0/2170141689 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f6d80005dc0 con 0x7f6d8c102ea0 2024-03-21T11:13:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.320+0000 7f6d92cba700 1 -- 172.21.15.102:0/2170141689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c1a3c60 con 0x7f6d8c102ea0 2024-03-21T11:13:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.320+0000 7f6d92cba700 1 -- 172.21.15.102:0/2170141689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c1a4140 con 0x7f6d8c102ea0 2024-03-21T11:13:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.320+0000 7f6d89ffb700 1 -- 172.21.15.102:0/2170141689 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d8000be40 con 0x7f6d8c102ea0 2024-03-21T11:13:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.320+0000 7f6d89ffb700 1 -- 172.21.15.102:0/2170141689 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f6d8000f460 con 0x7f6d8c102ea0 2024-03-21T11:13:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.321+0000 7f6d92cba700 1 -- 172.21.15.102:0/2170141689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c06abd0 con 0x7f6d8c102ea0 2024-03-21T11:13:50.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.321+0000 7f6d89ffb700 1 -- 172.21.15.102:0/2170141689 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f6d8002a070 con 0x7f6d8c102ea0 2024-03-21T11:13:50.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.322+0000 7f6d89ffb700 1 --2- 172.21.15.102:0/2170141689 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d7c038fc0 0x7f6d7c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:50.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.322+0000 7f6d89ffb700 1 -- 172.21.15.102:0/2170141689 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6d8001e030 con 0x7f6d8c102ea0 2024-03-21T11:13:50.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.322+0000 7f6d8bfff700 1 --2- 172.21.15.102:0/2170141689 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d7c038fc0 0x7f6d7c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:50.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.323+0000 7f6d8bfff700 1 --2- 172.21.15.102:0/2170141689 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d7c038fc0 0x7f6d7c03b480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009180 tx=0x7f6d78008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:50.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.337+0000 7f6d89ffb700 1 -- 172.21.15.102:0/2170141689 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6d80013070 con 0x7f6d8c102ea0 2024-03-21T11:13:50.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:50 smithi102 ceph-mon[36748]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:50.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:50 smithi047 ceph-mon[31351]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:50.614 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.614+0000 7f6d92cba700 1 -- 172.21.15.102:0/2170141689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6d8c1a4370 con 0x7f6d8c102ea0 2024-03-21T11:13:50.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.615+0000 7f6d89ffb700 1 -- 172.21.15.102:0/2170141689 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f6d80017440 con 0x7f6d8c102ea0 2024-03-21T11:13:50.615 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:13:50.615 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":2,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","modified":"2024-03-21T11:13:42.112093Z","created":"2024-03-21T11:11:37.664890Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T11:13:50.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.617+0000 7f6d92cba700 1 -- 172.21.15.102:0/2170141689 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d7c038fc0 msgr2=0x7f6d7c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:50.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.617+0000 7f6d92cba700 1 --2- 172.21.15.102:0/2170141689 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d7c038fc0 0x7f6d7c03b480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009180 tx=0x7f6d78008040 comp rx=0 tx=0).stop 2024-03-21T11:13:50.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.617+0000 7f6d92cba700 1 -- 172.21.15.102:0/2170141689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8c102ea0 msgr2=0x7f6d8c100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:50.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.617+0000 7f6d92cba700 1 --2- 172.21.15.102:0/2170141689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8c102ea0 0x7f6d8c100b00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6d80014000 tx=0x7f6d80005030 comp rx=0 tx=0).stop 2024-03-21T11:13:50.617 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.618+0000 7f6d92cba700 1 -- 172.21.15.102:0/2170141689 shutdown_connections 2024-03-21T11:13:50.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.618+0000 7f6d92cba700 1 --2- 172.21.15.102:0/2170141689 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d7c038fc0 0x7f6d7c03b480 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:50.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.618+0000 7f6d92cba700 1 --2- 172.21.15.102:0/2170141689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8c102ea0 0x7f6d8c100b00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:50.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.618+0000 7f6d92cba700 1 --2- 172.21.15.102:0/2170141689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d8c101040 0x7f6d8c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:50.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.618+0000 7f6d92cba700 1 -- 172.21.15.102:0/2170141689 >> 172.21.15.102:0/2170141689 conn(0x7f6d8c0fc630 msgr2=0x7f6d8c105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:50.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.618+0000 7f6d92cba700 1 -- 172.21.15.102:0/2170141689 shutdown_connections 2024-03-21T11:13:50.618 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:50.618+0000 7f6d92cba700 1 -- 172.21.15.102:0/2170141689 wait complete. 2024-03-21T11:13:50.618 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 2 2024-03-21T11:13:50.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:50 smithi100 podman[36226]: 2024-03-21 11:13:50.730983798 +0000 UTC m=+0.018011926 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:13:51.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 podman[36226]: 2024-03-21 11:13:51.056636204 +0000 UTC m=+0.343664342 container init 5ce4875e32d6a1e20d4151e204ee00a74b0bde9be888e0f1ca4183c8fc239ee9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-21T11:13:51.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 podman[36226]: 2024-03-21 11:13:51.059972945 +0000 UTC m=+0.347001065 container start 5ce4875e32d6a1e20d4151e204ee00a74b0bde9be888e0f1ca4183c8fc239ee9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-b, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-21T11:13:51.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T11:13:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T11:13:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pidfile_write: ignore empty --pid-file 2024-03-21T11:13:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: load: jerasure load: lrc 2024-03-21T11:13:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: RocksDB version: 7.9.2 2024-03-21T11:13:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Git sha 0 2024-03-21T11:13:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T11:13:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: DB SUMMARY 2024-03-21T11:13:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: DB Session ID: Y5SNWPVMXCWUX2KN0YZQ 2024-03-21T11:13:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: CURRENT file: CURRENT 2024-03-21T11:13:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T11:13:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-21T11:13:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-21T11:13:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-21T11:13:51.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.error_if_exists: 0 2024-03-21T11:13:51.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.create_if_missing: 0 2024-03-21T11:13:51.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.paranoid_checks: 1 2024-03-21T11:13:51.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T11:13:51.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T11:13:51.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T11:13:51.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.env: 0x563aace26b20 2024-03-21T11:13:51.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T11:13:51.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.info_log: 0x563aaef6f920 2024-03-21T11:13:51.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T11:13:51.396 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.statistics: (nil) 2024-03-21T11:13:51.396 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.use_fsync: 0 2024-03-21T11:13:51.396 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_log_file_size: 0 2024-03-21T11:13:51.396 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T11:13:51.396 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T11:13:51.396 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T11:13:51.396 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T11:13:51.397 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.allow_fallocate: 1 2024-03-21T11:13:51.397 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T11:13:51.397 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T11:13:51.397 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.use_direct_reads: 0 2024-03-21T11:13:51.397 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T11:13:51.397 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T11:13:51.397 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.db_log_dir: 2024-03-21T11:13:51.397 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.wal_dir: 2024-03-21T11:13:51.397 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T11:13:51.397 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T11:13:51.397 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T11:13:51.397 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T11:13:51.397 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T11:13:51.397 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T11:13:51.398 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T11:13:51.398 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T11:13:51.398 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.write_buffer_manager: 0x563aaef421e0 2024-03-21T11:13:51.398 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T11:13:51.398 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T11:13:51.398 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T11:13:51.398 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T11:13:51.398 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T11:13:51.398 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T11:13:51.398 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T11:13:51.398 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T11:13:51.398 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.unordered_write: 0 2024-03-21T11:13:51.399 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T11:13:51.399 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T11:13:51.399 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T11:13:51.399 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T11:13:51.399 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.row_cache: None 2024-03-21T11:13:51.399 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.wal_filter: None 2024-03-21T11:13:51.399 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T11:13:51.399 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T11:13:51.399 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.two_write_queues: 0 2024-03-21T11:13:51.399 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T11:13:51.399 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.wal_compression: 0 2024-03-21T11:13:51.399 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.atomic_flush: 0 2024-03-21T11:13:51.399 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T11:13:51.399 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T11:13:51.400 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T11:13:51.400 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.log_readahead_size: 0 2024-03-21T11:13:51.400 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T11:13:51.400 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T11:13:51.400 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T11:13:51.400 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T11:13:51.400 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T11:13:51.400 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T11:13:51.400 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T11:13:51.400 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_background_jobs: 2 2024-03-21T11:13:51.400 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_background_compactions: -1 2024-03-21T11:13:51.400 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_subcompactions: 1 2024-03-21T11:13:51.400 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T11:13:51.400 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T11:13:51.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T11:13:51.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T11:13:51.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T11:13:51.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T11:13:51.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T11:13:51.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T11:13:51.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_open_files: -1 2024-03-21T11:13:51.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T11:13:51.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T11:13:51.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T11:13:51.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T11:13:51.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_background_flushes: -1 2024-03-21T11:13:51.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Compression algorithms supported: 2024-03-21T11:13:51.402 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T11:13:51.402 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: kZSTD supported: 0 2024-03-21T11:13:51.402 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: kXpressCompression supported: 0 2024-03-21T11:13:51.402 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T11:13:51.402 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: kZlibCompression supported: 1 2024-03-21T11:13:51.402 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: kSnappyCompression supported: 1 2024-03-21T11:13:51.402 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: kLZ4Compression supported: 1 2024-03-21T11:13:51.402 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: kBZip2Compression supported: 0 2024-03-21T11:13:51.402 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T11:13:51.402 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T11:13:51.402 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-21T11:13:51.402 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T11:13:51.402 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T11:13:51.403 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.merge_operator: 2024-03-21T11:13:51.403 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compaction_filter: None 2024-03-21T11:13:51.403 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T11:13:51.403 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T11:13:51.403 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T11:13:51.403 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T11:13:51.403 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563aaef6fa20) 2024-03-21T11:13:51.403 INFO:journalctl@ceph.mon.b.smithi100.stdout: cache_index_and_filter_blocks: 1 2024-03-21T11:13:51.403 INFO:journalctl@ceph.mon.b.smithi100.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T11:13:51.403 INFO:journalctl@ceph.mon.b.smithi100.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T11:13:51.403 INFO:journalctl@ceph.mon.b.smithi100.stdout: pin_top_level_index_and_filter: 1 2024-03-21T11:13:51.403 INFO:journalctl@ceph.mon.b.smithi100.stdout: index_type: 0 2024-03-21T11:13:51.403 INFO:journalctl@ceph.mon.b.smithi100.stdout: data_block_index_type: 0 2024-03-21T11:13:51.403 INFO:journalctl@ceph.mon.b.smithi100.stdout: index_shortening: 1 2024-03-21T11:13:51.404 INFO:journalctl@ceph.mon.b.smithi100.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T11:13:51.404 INFO:journalctl@ceph.mon.b.smithi100.stdout: checksum: 4 2024-03-21T11:13:51.404 INFO:journalctl@ceph.mon.b.smithi100.stdout: no_block_cache: 0 2024-03-21T11:13:51.404 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_cache: 0x563aaef75090 2024-03-21T11:13:51.404 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_cache_name: BinnedLRUCache 2024-03-21T11:13:51.404 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_cache_options: 2024-03-21T11:13:51.404 INFO:journalctl@ceph.mon.b.smithi100.stdout: capacity : 536870912 2024-03-21T11:13:51.404 INFO:journalctl@ceph.mon.b.smithi100.stdout: num_shard_bits : 4 2024-03-21T11:13:51.404 INFO:journalctl@ceph.mon.b.smithi100.stdout: strict_capacity_limit : 0 2024-03-21T11:13:51.404 INFO:journalctl@ceph.mon.b.smithi100.stdout: high_pri_pool_ratio: 0.000 2024-03-21T11:13:51.404 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_cache_compressed: (nil) 2024-03-21T11:13:51.404 INFO:journalctl@ceph.mon.b.smithi100.stdout: persistent_cache: (nil) 2024-03-21T11:13:51.404 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_size: 4096 2024-03-21T11:13:51.404 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_size_deviation: 10 2024-03-21T11:13:51.405 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_restart_interval: 16 2024-03-21T11:13:51.405 INFO:journalctl@ceph.mon.b.smithi100.stdout: index_block_restart_interval: 1 2024-03-21T11:13:51.405 INFO:journalctl@ceph.mon.b.smithi100.stdout: metadata_block_size: 4096 2024-03-21T11:13:51.405 INFO:journalctl@ceph.mon.b.smithi100.stdout: partition_filters: 0 2024-03-21T11:13:51.405 INFO:journalctl@ceph.mon.b.smithi100.stdout: use_delta_encoding: 1 2024-03-21T11:13:51.405 INFO:journalctl@ceph.mon.b.smithi100.stdout: filter_policy: bloomfilter 2024-03-21T11:13:51.405 INFO:journalctl@ceph.mon.b.smithi100.stdout: whole_key_filtering: 1 2024-03-21T11:13:51.405 INFO:journalctl@ceph.mon.b.smithi100.stdout: verify_compression: 0 2024-03-21T11:13:51.405 INFO:journalctl@ceph.mon.b.smithi100.stdout: read_amp_bytes_per_bit: 0 2024-03-21T11:13:51.405 INFO:journalctl@ceph.mon.b.smithi100.stdout: format_version: 5 2024-03-21T11:13:51.405 INFO:journalctl@ceph.mon.b.smithi100.stdout: enable_index_compression: 1 2024-03-21T11:13:51.405 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_align: 0 2024-03-21T11:13:51.405 INFO:journalctl@ceph.mon.b.smithi100.stdout: max_auto_readahead_size: 262144 2024-03-21T11:13:51.405 INFO:journalctl@ceph.mon.b.smithi100.stdout: prepopulate_block_cache: 0 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout: initial_auto_readahead_size: 8192 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compression: NoCompression 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.num_levels: 7 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T11:13:51.406 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T11:13:51.407 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T11:13:51.407 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T11:13:51.407 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T11:13:51.407 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T11:13:51.407 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T11:13:51.407 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T11:13:51.407 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T11:13:51.407 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T11:13:51.407 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T11:13:51.407 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T11:13:51.407 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T11:13:51.407 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T11:13:51.407 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T11:13:51.407 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T11:13:51.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T11:13:51.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T11:13:51.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T11:13:51.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T11:13:51.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T11:13:51.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T11:13:51.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T11:13:51.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T11:13:51.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T11:13:51.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T11:13:51.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T11:13:51.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T11:13:51.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T11:13:51.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T11:13:51.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T11:13:51.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T11:13:51.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T11:13:51.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T11:13:51.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T11:13:51.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T11:13:51.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T11:13:51.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T11:13:51.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T11:13:51.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T11:13:51.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T11:13:51.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T11:13:51.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T11:13:51.409 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T11:13:51.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T11:13:51.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T11:13:51.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T11:13:51.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T11:13:51.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T11:13:51.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.table_properties_collectors: 2024-03-21T11:13:51.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.inplace_update_support: 0 2024-03-21T11:13:51.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T11:13:51.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T11:13:51.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T11:13:51.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T11:13:51.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.bloom_locality: 0 2024-03-21T11:13:51.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.max_successive_merges: 0 2024-03-21T11:13:51.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.ttl: 2592000 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.enable_blob_files: false 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.min_blob_size: 0 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T11:13:51.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T11:13:51.412 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T11:13:51.412 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T11:13:51.412 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-21T11:13:51.412 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-21T11:13:51.412 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8c141d41-f60e-477c-92ed-e0ff3c35092a 2024-03-21T11:13:51.412 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711019631089140, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-21T11:13:51.412 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-21T11:13:51.412 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711019631089559, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711019631, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8c141d41-f60e-477c-92ed-e0ff3c35092a", "db_session_id": "Y5SNWPVMXCWUX2KN0YZQ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-21T11:13:51.412 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711019631089636, "job": 1, "event": "recovery_finished"} 2024-03-21T11:13:51.412 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-21T11:13:51.412 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T11:13:51.412 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563aaf068000 2024-03-21T11:13:51.412 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: DB pointer 0x563aaf052000 2024-03-21T11:13:51.412 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-21T11:13:51.413 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T11:13:51.413 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T11:13:51.413 INFO:journalctl@ceph.mon.b.smithi100.stdout: ** DB Stats ** 2024-03-21T11:13:51.413 INFO:journalctl@ceph.mon.b.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T11:13:51.413 INFO:journalctl@ceph.mon.b.smithi100.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T11:13:51.413 INFO:journalctl@ceph.mon.b.smithi100.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T11:13:51.413 INFO:journalctl@ceph.mon.b.smithi100.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T11:13:51.413 INFO:journalctl@ceph.mon.b.smithi100.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T11:13:51.413 INFO:journalctl@ceph.mon.b.smithi100.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T11:13:51.413 INFO:journalctl@ceph.mon.b.smithi100.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T11:13:51.413 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-03-21T11:13:51.413 INFO:journalctl@ceph.mon.b.smithi100.stdout: ** Compaction Stats [default] ** 2024-03-21T11:13:51.413 INFO:journalctl@ceph.mon.b.smithi100.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T11:13:51.413 INFO:journalctl@ceph.mon.b.smithi100.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T11:13:51.414 INFO:journalctl@ceph.mon.b.smithi100.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T11:13:51.414 INFO:journalctl@ceph.mon.b.smithi100.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T11:13:51.414 INFO:journalctl@ceph.mon.b.smithi100.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T11:13:51.414 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-03-21T11:13:51.414 INFO:journalctl@ceph.mon.b.smithi100.stdout: ** Compaction Stats [default] ** 2024-03-21T11:13:51.414 INFO:journalctl@ceph.mon.b.smithi100.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T11:13:51.414 INFO:journalctl@ceph.mon.b.smithi100.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T11:13:51.414 INFO:journalctl@ceph.mon.b.smithi100.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T11:13:51.414 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-03-21T11:13:51.414 INFO:journalctl@ceph.mon.b.smithi100.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T11:13:51.414 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-03-21T11:13:51.414 INFO:journalctl@ceph.mon.b.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T11:13:51.414 INFO:journalctl@ceph.mon.b.smithi100.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T11:13:51.414 INFO:journalctl@ceph.mon.b.smithi100.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T11:13:51.415 INFO:journalctl@ceph.mon.b.smithi100.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T11:13:51.415 INFO:journalctl@ceph.mon.b.smithi100.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T11:13:51.415 INFO:journalctl@ceph.mon.b.smithi100.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T11:13:51.415 INFO:journalctl@ceph.mon.b.smithi100.stdout: Cumulative compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T11:13:51.415 INFO:journalctl@ceph.mon.b.smithi100.stdout: Interval compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T11:13:51.415 INFO:journalctl@ceph.mon.b.smithi100.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T11:13:51.415 INFO:journalctl@ceph.mon.b.smithi100.stdout: Block cache BinnedLRUCache@0x563aaef75090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-21T11:13:51.415 INFO:journalctl@ceph.mon.b.smithi100.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-21T11:13:51.415 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-03-21T11:13:51.415 INFO:journalctl@ceph.mon.b.smithi100.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T11:13:51.415 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: using public_addr v2:172.21.15.100:0/0 -> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] 2024-03-21T11:13:51.415 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: starting mon.b rank -1 at public addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] at bind addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:13:51.415 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b@-1(???) e0 preinit fsid a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:13:51.415 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b@-1(synchronizing).mds e1 new map 2024-03-21T11:13:51.416 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-21T11:13:51.416 INFO:journalctl@ceph.mon.b.smithi100.stdout: e1 2024-03-21T11:13:51.416 INFO:journalctl@ceph.mon.b.smithi100.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T11:13:51.416 INFO:journalctl@ceph.mon.b.smithi100.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T11:13:51.416 INFO:journalctl@ceph.mon.b.smithi100.stdout: legacy client fscid: -1 2024-03-21T11:13:51.416 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-03-21T11:13:51.416 INFO:journalctl@ceph.mon.b.smithi100.stdout: No filesystems configured 2024-03-21T11:13:51.416 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-21T11:13:51.416 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-21T11:13:51.416 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-21T11:13:51.416 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-21T11:13:51.416 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-21T11:13:51.416 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-21T11:13:51.416 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: Added host smithi100 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.417 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.418 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.418 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.418 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.418 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:13:51.418 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:51.418 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:13:51.418 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-21T11:13:51.418 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: Updating smithi100:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:13:51.418 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-21T11:13:51.418 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: Updating smithi100:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T11:13:51.418 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.418 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.418 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.419 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:13:51.419 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.419 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: Deploying cephadm binary to smithi102 2024-03-21T11:13:51.419 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.419 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.419 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.419 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: Added host smithi102 2024-03-21T11:13:51.419 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:51.419 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.419 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.419 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.419 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.419 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.419 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:13:51.420 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.420 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.420 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.420 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.420 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.420 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:13:51.420 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:51.420 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:13:51.420 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: Updating smithi102:/etc/ceph/ceph.conf 2024-03-21T11:13:51.420 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1463655696' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T11:13:51.420 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: Updating smithi102:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:13:51.420 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-03-21T11:13:51.420 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: Updating smithi102:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T11:13:51.421 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1463655696' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T11:13:51.421 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T11:13:51.421 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.421 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.421 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.421 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.421 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.421 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi102:172.21.15.102=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:13:51.421 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi102:172.21.15.102=c;count:3 2024-03-21T11:13:51.421 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.421 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:51.421 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:51.421 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:13:51.421 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:13:51.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:51.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: Deploying daemon mon.c on smithi102 2024-03-21T11:13:51.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:13:51.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.a calling monitor election 2024-03-21T11:13:51.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:51.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='client.? 172.21.15.102:0/3691317992' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T11:13:51.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:51.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:51.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.c calling monitor election 2024-03-21T11:13:51.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.422 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:51.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:51.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:51.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T11:13:51.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: monmap e2: 2 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],c=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T11:13:51.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: fsmap 2024-03-21T11:13:51.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T11:13:51.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mgrmap e13: x(active, since 58s) 2024-03-21T11:13:51.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: overall HEALTH_OK 2024-03-21T11:13:51.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:51.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:13:51.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:13:51.423 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.424 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: Deploying daemon mon.b on smithi100 2024-03-21T11:13:51.424 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:51.424 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:51.424 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 ceph-mon[36245]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-21T11:13:51.424 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 bash[36226]: 5ce4875e32d6a1e20d4151e204ee00a74b0bde9be888e0f1ca4183c8fc239ee9 2024-03-21T11:13:51.424 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:51 smithi100 systemd[1]: Started Ceph mon.b for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:13:52.087 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T11:13:52.088 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T11:13:52.328 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:13:56.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:56.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: mon.c calling monitor election 2024-03-21T11:13:56.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:13:56.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: mon.a calling monitor election 2024-03-21T11:13:56.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:56.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:56.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:56.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:56.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:56.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:56.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:56.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:56.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:56.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T11:13:56.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T11:13:56.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: fsmap 2024-03-21T11:13:56.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T11:13:56.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: mgrmap e13: x(active, since 67s) 2024-03-21T11:13:56.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-21T11:13:56.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T11:13:56.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-21T11:13:56.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: mon.b (rank 2) addr [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] is down (out of quorum) 2024-03-21T11:13:56.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:56.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: mon.c calling monitor election 2024-03-21T11:13:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:13:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: mon.a calling monitor election 2024-03-21T11:13:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:56.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T11:13:56.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T11:13:56.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: fsmap 2024-03-21T11:13:56.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T11:13:56.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: mgrmap e13: x(active, since 67s) 2024-03-21T11:13:56.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-21T11:13:56.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T11:13:56.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-21T11:13:56.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: mon.b (rank 2) addr [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] is down (out of quorum) 2024-03-21T11:13:56.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:56.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:56.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.812+0000 7fefe7ead700 1 -- 172.21.15.102:0/1694979820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefcc0035c0 msgr2=0x7fefcc005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:56.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.812+0000 7fefe7ead700 1 --2- 172.21.15.102:0/1694979820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefcc0035c0 0x7fefcc005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fefe0100ac0 tx=0x7fefdc00feb0 comp rx=0 tx=0).stop 2024-03-21T11:13:56.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.812+0000 7fefe7ead700 1 -- 172.21.15.102:0/1694979820 shutdown_connections 2024-03-21T11:13:56.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.812+0000 7fefe7ead700 1 --2- 172.21.15.102:0/1694979820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefcc0035c0 0x7fefcc005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:56.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.812+0000 7fefe7ead700 1 -- 172.21.15.102:0/1694979820 >> 172.21.15.102:0/1694979820 conn(0x7fefe00fc6a0 msgr2=0x7fefe00feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:56.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.813+0000 7fefe7ead700 1 -- 172.21.15.102:0/1694979820 shutdown_connections 2024-03-21T11:13:56.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.813+0000 7fefe7ead700 1 -- 172.21.15.102:0/1694979820 wait complete. 2024-03-21T11:13:56.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.813+0000 7fefe7ead700 1 Processor -- start 2024-03-21T11:13:56.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.813+0000 7fefe7ead700 1 -- start start 2024-03-21T11:13:56.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe7ead700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefcc0035c0 0x7fefe0198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:56.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe7ead700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefe0198cd0 0x7fefe0199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:56.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe7ead700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe01a1800 0x7fefe01a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:56.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe7ead700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0108bd0 con 0x7fefcc0035c0 2024-03-21T11:13:56.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe7ead700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0108a50 con 0x7fefe01a1800 2024-03-21T11:13:56.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe7ead700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0108d50 con 0x7fefe0198cd0 2024-03-21T11:13:56.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe5448700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefe0198cd0 0x7fefe0199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:56.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe5448700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefe0198cd0 0x7fefe0199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54484/0 (socket says 172.21.15.102:54484) 2024-03-21T11:13:56.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe5448700 1 -- 172.21.15.102:0/3160124185 learned_addr learned my addr 172.21.15.102:0/3160124185 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-21T11:13:56.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe5448700 1 -- 172.21.15.102:0/3160124185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefe0198cd0 msgr2=0x7fefe0199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-21T11:13:56.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe5448700 1 -- 172.21.15.102:0/3160124185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefe0198cd0 msgr2=0x7fefe0199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-21T11:13:56.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe5448700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefe0198cd0 0x7fefe0199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T11:13:56.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe644a700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe01a1800 0x7fefe01a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:56.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe5448700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefe0198cd0 0x7fefe0199150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T11:13:56.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe644a700 1 -- 172.21.15.102:0/3160124185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe01a1800 msgr2=0x7fefe01a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-21T11:13:56.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe644a700 1 -- 172.21.15.102:0/3160124185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe01a1800 msgr2=0x7fefe01a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-21T11:13:56.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe644a700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe01a1800 0x7fefe01a3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T11:13:56.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe644a700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe01a1800 0x7fefe01a3bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T11:13:56.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.814+0000 7fefe5c49700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefcc0035c0 0x7fefe0198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:56.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.815+0000 7fefe5c49700 1 -- 172.21.15.102:0/3160124185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefe0198cd0 msgr2=0x7fefe0199150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:13:56.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.815+0000 7fefe5c49700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefe0198cd0 0x7fefe0199150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:56.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.815+0000 7fefe5c49700 1 -- 172.21.15.102:0/3160124185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe01a1800 msgr2=0x7fefe01a3bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:13:56.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.815+0000 7fefe5c49700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe01a1800 0x7fefe01a3bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:56.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.815+0000 7fefe5c49700 1 -- 172.21.15.102:0/3160124185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe01a4400 con 0x7fefcc0035c0 2024-03-21T11:13:56.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.815+0000 7fefe5c49700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefcc0035c0 0x7fefe0198790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fefdc000c00 tx=0x7fefdc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:56.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.815+0000 7fefd6ffd700 1 -- 172.21.15.102:0/3160124185 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc005300 con 0x7fefcc0035c0 2024-03-21T11:13:56.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.816+0000 7fefe7ead700 1 -- 172.21.15.102:0/3160124185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe01a4690 con 0x7fefcc0035c0 2024-03-21T11:13:56.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.816+0000 7fefe7ead700 1 -- 172.21.15.102:0/3160124185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe01a4c20 con 0x7fefcc0035c0 2024-03-21T11:13:56.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.816+0000 7fefd6ffd700 1 -- 172.21.15.102:0/3160124185 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefdc004510 con 0x7fefcc0035c0 2024-03-21T11:13:56.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.816+0000 7fefd6ffd700 1 -- 172.21.15.102:0/3160124185 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc013b60 con 0x7fefcc0035c0 2024-03-21T11:13:56.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.816+0000 7fefe7ead700 1 -- 172.21.15.102:0/3160124185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefcc002140 con 0x7fefcc0035c0 2024-03-21T11:13:56.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.817+0000 7fefd6ffd700 1 -- 172.21.15.102:0/3160124185 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fefdc023070 con 0x7fefcc0035c0 2024-03-21T11:13:56.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.817+0000 7fefd6ffd700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fefc40396b0 0x7fefc403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:13:56.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.817+0000 7fefd6ffd700 1 -- 172.21.15.102:0/3160124185 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fefdc04e850 con 0x7fefcc0035c0 2024-03-21T11:13:56.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.818+0000 7fefe5448700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fefc40396b0 0x7fefc403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:13:56.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.819+0000 7fefe5448700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fefc40396b0 0x7fefc403bb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009180 tx=0x7fefd0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:13:56.832 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:56.832+0000 7fefd6ffd700 1 -- 172.21.15.102:0/3160124185 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fefdc031070 con 0x7fefcc0035c0 2024-03-21T11:13:57.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:57.129+0000 7fefe7ead700 1 -- 172.21.15.102:0/3160124185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fefcc002c30 con 0x7fefcc0035c0 2024-03-21T11:13:57.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:57.130+0000 7fefd6ffd700 1 -- 172.21.15.102:0/3160124185 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1283 (secure 0 0 0) 0x7fefdc028020 con 0x7fefcc0035c0 2024-03-21T11:13:57.131 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-03-21T11:13:57.131 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":3,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","modified":"2024-03-21T11:13:51.111619Z","created":"2024-03-21T11:11:37.664890Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T11:13:57.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:57.133+0000 7fefe7ead700 1 -- 172.21.15.102:0/3160124185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fefc40396b0 msgr2=0x7fefc403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:57.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:57.133+0000 7fefe7ead700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fefc40396b0 0x7fefc403bb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009180 tx=0x7fefd0008040 comp rx=0 tx=0).stop 2024-03-21T11:13:57.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:57.133+0000 7fefe7ead700 1 -- 172.21.15.102:0/3160124185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefcc0035c0 msgr2=0x7fefe0198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:13:57.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:57.133+0000 7fefe7ead700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefcc0035c0 0x7fefe0198790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fefdc000c00 tx=0x7fefdc005020 comp rx=0 tx=0).stop 2024-03-21T11:13:57.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:57.133+0000 7fefe7ead700 1 -- 172.21.15.102:0/3160124185 shutdown_connections 2024-03-21T11:13:57.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:57.133+0000 7fefe7ead700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefe01a1800 0x7fefe01a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:57.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:57.133+0000 7fefe7ead700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fefc40396b0 0x7fefc403bb70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:57.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:57.133+0000 7fefe7ead700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefcc0035c0 0x7fefe0198790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:57.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:57.133+0000 7fefe7ead700 1 --2- 172.21.15.102:0/3160124185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefe0198cd0 0x7fefe0199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:13:57.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:57.133+0000 7fefe7ead700 1 -- 172.21.15.102:0/3160124185 >> 172.21.15.102:0/3160124185 conn(0x7fefe00fc6a0 msgr2=0x7fefe00ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:13:57.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:57.133+0000 7fefe7ead700 1 -- 172.21.15.102:0/3160124185 shutdown_connections 2024-03-21T11:13:57.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:13:57.133+0000 7fefe7ead700 1 -- 172.21.15.102:0/3160124185 wait complete. 2024-03-21T11:13:57.135 INFO:teuthology.orchestra.run.smithi102.stderr:dumped monmap epoch 3 2024-03-21T11:13:57.450 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:57 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:57.450 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:57 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:57.450 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:57 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:57.450 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:57 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:57.450 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:57 smithi102 ceph-mon[36748]: from='client.? 172.21.15.102:0/3160124185' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T11:13:57.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:57 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:57.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:57 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:57.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:57 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:57.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:57 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:57.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:57 smithi047 ceph-mon[31351]: from='client.? 172.21.15.102:0/3160124185' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T11:13:58.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: mon.c calling monitor election 2024-03-21T11:13:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:13:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: mon.a calling monitor election 2024-03-21T11:13:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:13:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:13:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:58.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T11:13:58.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T11:13:58.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: fsmap 2024-03-21T11:13:58.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T11:13:58.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: mgrmap e13: x(active, since 67s) 2024-03-21T11:13:58.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-21T11:13:58.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T11:13:58.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-21T11:13:58.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: mon.b (rank 2) addr [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] is down (out of quorum) 2024-03-21T11:13:58.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:58.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:58.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:58.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:58.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:13:58.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:58.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='client.? 172.21.15.102:0/3160124185' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T11:13:58.437 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-21T11:13:58.437 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-21T11:13:58.859 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:13:59.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: mon.b calling monitor election 2024-03-21T11:13:59.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: mon.b calling monitor election 2024-03-21T11:13:59.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: mon.c calling monitor election 2024-03-21T11:13:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: mon.a calling monitor election 2024-03-21T11:13:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T11:13:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T11:13:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: fsmap 2024-03-21T11:13:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T11:13:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: mgrmap e13: x(active, since 69s) 2024-03-21T11:13:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T11:13:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: Cluster is now healthy 2024-03-21T11:13:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: overall HEALTH_OK 2024-03-21T11:13:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:58 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: mon.b calling monitor election 2024-03-21T11:13:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: mon.b calling monitor election 2024-03-21T11:13:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: mon.c calling monitor election 2024-03-21T11:13:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: mon.a calling monitor election 2024-03-21T11:13:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T11:13:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T11:13:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: fsmap 2024-03-21T11:13:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T11:13:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: mgrmap e13: x(active, since 69s) 2024-03-21T11:13:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T11:13:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: Cluster is now healthy 2024-03-21T11:13:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: overall HEALTH_OK 2024-03-21T11:13:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: mon.b calling monitor election 2024-03-21T11:13:59.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: mon.b calling monitor election 2024-03-21T11:13:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: mon.c calling monitor election 2024-03-21T11:13:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: mon.a calling monitor election 2024-03-21T11:13:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T11:13:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T11:13:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: fsmap 2024-03-21T11:13:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T11:13:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: mgrmap e13: x(active, since 69s) 2024-03-21T11:13:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T11:13:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: Cluster is now healthy 2024-03-21T11:13:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: overall HEALTH_OK 2024-03-21T11:13:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:58 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: Updating smithi100:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:13:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: Updating smithi102:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:13:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: Updating smithi047:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:13:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:13:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:13:59.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:13:59.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T11:13:59.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:13:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:00.091 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: Updating smithi100:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:14:00.091 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: Updating smithi102:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:14:00.091 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: Updating smithi047:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:14:00.092 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.092 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:14:00.092 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.092 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.092 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.092 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.092 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.092 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.092 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.092 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:14:00.092 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T11:14:00.092 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:13:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:00.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: Updating smithi100:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:14:00.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: Updating smithi102:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:14:00.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: Updating smithi047:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T11:14:00.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:14:00.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:00.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:14:00.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T11:14:00.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:13:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:00.471 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:14:00 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:14:00.112+0000 7f778d000700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-21T11:14:00.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:00 smithi102 ceph-mon[36748]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:00.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:00 smithi102 ceph-mon[36748]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T11:14:00.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:00 smithi102 ceph-mon[36748]: Reconfiguring daemon mon.a on smithi047 2024-03-21T11:14:01.075 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:00 smithi047 ceph-mon[31351]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:01.075 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:00 smithi047 ceph-mon[31351]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T11:14:01.075 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:00 smithi047 ceph-mon[31351]: Reconfiguring daemon mon.a on smithi047 2024-03-21T11:14:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:00 smithi100 ceph-mon[36245]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:00 smithi100 ceph-mon[36245]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T11:14:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:00 smithi100 ceph-mon[36245]: Reconfiguring daemon mon.a on smithi047 2024-03-21T11:14:01.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.685+0000 7f01df000700 1 -- 172.21.15.47:0/1289284446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d8103ff0 msgr2=0x7f01d81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:01.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.685+0000 7f01df000700 1 --2- 172.21.15.47:0/1289284446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d8103ff0 0x7f01d81043d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009910 tx=0x7f01c8009c20 comp rx=0 tx=0).stop 2024-03-21T11:14:01.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.685+0000 7f01df000700 1 -- 172.21.15.47:0/1289284446 shutdown_connections 2024-03-21T11:14:01.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.685+0000 7f01df000700 1 --2- 172.21.15.47:0/1289284446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d8103ff0 0x7f01d81043d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:01.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.685+0000 7f01df000700 1 -- 172.21.15.47:0/1289284446 >> 172.21.15.47:0/1289284446 conn(0x7f01d80ff880 msgr2=0x7f01d8101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:14:01.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.686+0000 7f01df000700 1 -- 172.21.15.47:0/1289284446 shutdown_connections 2024-03-21T11:14:01.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.686+0000 7f01df000700 1 -- 172.21.15.47:0/1289284446 wait complete. 2024-03-21T11:14:01.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.687+0000 7f01df000700 1 Processor -- start 2024-03-21T11:14:01.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.687+0000 7f01df000700 1 -- start start 2024-03-21T11:14:01.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.688+0000 7f01df000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d819fa40 0x7f01d819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:01.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.688+0000 7f01df000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01d8199b60 0x7f01d8199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:01.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.688+0000 7f01df000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01d819a500 0x7f01d81a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:01.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.688+0000 7f01df000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d810e3c0 con 0x7f01d819fa40 2024-03-21T11:14:01.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.688+0000 7f01df000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d810e240 con 0x7f01d8199b60 2024-03-21T11:14:01.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.688+0000 7f01df000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d810e540 con 0x7f01d819a500 2024-03-21T11:14:01.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.688+0000 7f01d7e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d819fa40 0x7f01d819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:01.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.688+0000 7f01d7e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d819fa40 0x7f01d819fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47824/0 (socket says 172.21.15.47:47824) 2024-03-21T11:14:01.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.688+0000 7f01d7e00700 1 -- 172.21.15.47:0/161400030 learned_addr learned my addr 172.21.15.47:0/161400030 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:14:01.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.688+0000 7f01dd000700 1 --2- 172.21.15.47:0/161400030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01d819a500 0x7f01d81a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:01.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.688+0000 7f01d7400700 1 --2- 172.21.15.47:0/161400030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01d8199b60 0x7f01d8199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:01.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.689+0000 7f01d7e00700 1 -- 172.21.15.47:0/161400030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01d819a500 msgr2=0x7f01d81a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:01.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.689+0000 7f01d7e00700 1 --2- 172.21.15.47:0/161400030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01d819a500 0x7f01d81a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:01.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.689+0000 7f01d7e00700 1 -- 172.21.15.47:0/161400030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01d8199b60 msgr2=0x7f01d8199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:01.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.689+0000 7f01d7e00700 1 --2- 172.21.15.47:0/161400030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01d8199b60 0x7f01d8199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:01.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.689+0000 7f01d7e00700 1 -- 172.21.15.47:0/161400030 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d81a6d30 con 0x7f01d819fa40 2024-03-21T11:14:01.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.689+0000 7f01d7e00700 1 --2- 172.21.15.47:0/161400030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d819fa40 0x7f01d819fe20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f01c8005950 tx=0x7f01c8004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:14:01.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.689+0000 7f01d4c00700 1 -- 172.21.15.47:0/161400030 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8009e60 con 0x7f01d819fa40 2024-03-21T11:14:01.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.689+0000 7f01d4c00700 1 -- 172.21.15.47:0/161400030 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01c800b750 con 0x7f01d819fa40 2024-03-21T11:14:01.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.689+0000 7f01d4c00700 1 -- 172.21.15.47:0/161400030 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c80038e0 con 0x7f01d819fa40 2024-03-21T11:14:01.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.689+0000 7f01df000700 1 -- 172.21.15.47:0/161400030 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d81a6f00 con 0x7f01d819fa40 2024-03-21T11:14:01.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.690+0000 7f01df000700 1 -- 172.21.15.47:0/161400030 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d81a72a0 con 0x7f01d819fa40 2024-03-21T11:14:01.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.691+0000 7f01d4c00700 1 -- 172.21.15.47:0/161400030 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f01c800bc70 con 0x7f01d819fa40 2024-03-21T11:14:01.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.691+0000 7f01ce000700 1 -- 172.21.15.47:0/161400030 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d81054d0 con 0x7f01d819fa40 2024-03-21T11:14:01.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.691+0000 7f01d4c00700 1 --2- 172.21.15.47:0/161400030 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01bc0396b0 0x7f01bc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:01.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.691+0000 7f01d4c00700 1 -- 172.21.15.47:0/161400030 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f01c804d3f0 con 0x7f01d819fa40 2024-03-21T11:14:01.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.691+0000 7f01d7400700 1 --2- 172.21.15.47:0/161400030 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01bc0396b0 0x7f01bc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:01.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.692+0000 7f01d7400700 1 --2- 172.21.15.47:0/161400030 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01bc0396b0 0x7f01bc03bb70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009ad0 tx=0x7f01c4009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:14:01.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.697+0000 7f01d4c00700 1 -- 172.21.15.47:0/161400030 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f01c8013070 con 0x7f01d819fa40 2024-03-21T11:14:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.923+0000 7f01ce000700 1 -- 172.21.15.47:0/161400030 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f01d819afd0 con 0x7f01d819fa40 2024-03-21T11:14:01.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.924+0000 7f01d4c00700 1 -- 172.21.15.47:0/161400030 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f01c800b900 con 0x7f01d819fa40 2024-03-21T11:14:01.924 INFO:teuthology.orchestra.run.smithi047.stdout:# minimal ceph.conf for a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:14:01.924 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-03-21T11:14:01.925 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:14:01.925 INFO:teuthology.orchestra.run.smithi047.stdout: mon_host = [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] 2024-03-21T11:14:01.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.927+0000 7f01ce000700 1 -- 172.21.15.47:0/161400030 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01bc0396b0 msgr2=0x7f01bc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.927+0000 7f01ce000700 1 --2- 172.21.15.47:0/161400030 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01bc0396b0 0x7f01bc03bb70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009ad0 tx=0x7f01c4009210 comp rx=0 tx=0).stop 2024-03-21T11:14:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.927+0000 7f01ce000700 1 -- 172.21.15.47:0/161400030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d819fa40 msgr2=0x7f01d819fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.927+0000 7f01ce000700 1 --2- 172.21.15.47:0/161400030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d819fa40 0x7f01d819fe20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f01c8005950 tx=0x7f01c8004f80 comp rx=0 tx=0).stop 2024-03-21T11:14:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.927+0000 7f01ce000700 1 -- 172.21.15.47:0/161400030 shutdown_connections 2024-03-21T11:14:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.927+0000 7f01ce000700 1 --2- 172.21.15.47:0/161400030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01d819a500 0x7f01d81a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.927+0000 7f01ce000700 1 --2- 172.21.15.47:0/161400030 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01bc0396b0 0x7f01bc03bb70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.927+0000 7f01ce000700 1 --2- 172.21.15.47:0/161400030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01d819fa40 0x7f01d819fe20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.927+0000 7f01ce000700 1 --2- 172.21.15.47:0/161400030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01d8199b60 0x7f01d8199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:01.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.927+0000 7f01ce000700 1 -- 172.21.15.47:0/161400030 >> 172.21.15.47:0/161400030 conn(0x7f01d80ff880 msgr2=0x7f01d8101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:14:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.928+0000 7f01ce000700 1 -- 172.21.15.47:0/161400030 shutdown_connections 2024-03-21T11:14:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:01.928+0000 7f01ce000700 1 -- 172.21.15.47:0/161400030 wait complete. 2024-03-21T11:14:02.851 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-21T11:14:02.852 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:14:02.852 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T11:14:02.957 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:14:02.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T11:14:03.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:02 smithi047 ceph-mon[31351]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:03.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:02 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/161400030' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:03.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:03.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:03.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:14:03.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T11:14:03.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:03.077 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:02 smithi100 ceph-mon[36245]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:03.077 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:02 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/161400030' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:03.077 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:03.078 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:03.078 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:14:03.078 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T11:14:03.078 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:03.104 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T11:14:03.104 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T11:14:03.133 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T11:14:03.134 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T11:14:03.202 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-21T11:14:03.202 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T11:14:03.231 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:02 smithi102 ceph-mon[36748]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:03.231 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:02 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/161400030' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:03.231 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:03.232 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:03.232 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:14:03.232 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T11:14:03.232 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:03.236 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-21T11:14:03.236 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T11:14:03.305 INFO:tasks.cephadm:Adding mgr.x on smithi047 2024-03-21T11:14:03.305 INFO:tasks.cephadm:Adding mgr.y on smithi100 2024-03-21T11:14:03.305 INFO:tasks.cephadm:Adding mgr.z on smithi102 2024-03-21T11:14:03.305 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch apply mgr '3;smithi047=x;smithi100=y;smithi102=z' 2024-03-21T11:14:03.580 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:14:03.988 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:03 smithi102 ceph-mon[36748]: Reconfiguring mon.b (monmap changed)... 2024-03-21T11:14:03.988 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:03 smithi102 ceph-mon[36748]: Reconfiguring daemon mon.b on smithi100 2024-03-21T11:14:03.988 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:03 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:03.988 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:03 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:03.988 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:03 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:14:03.988 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:03 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T11:14:03.989 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:03 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:04.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:03 smithi100 ceph-mon[36245]: Reconfiguring mon.b (monmap changed)... 2024-03-21T11:14:04.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:03 smithi100 ceph-mon[36245]: Reconfiguring daemon mon.b on smithi100 2024-03-21T11:14:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:03 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:03 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:03 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:14:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:03 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T11:14:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:03 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:04.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:03 smithi047 ceph-mon[31351]: Reconfiguring mon.b (monmap changed)... 2024-03-21T11:14:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:03 smithi047 ceph-mon[31351]: Reconfiguring daemon mon.b on smithi100 2024-03-21T11:14:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T11:14:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T11:14:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:05.067 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:04 smithi102 ceph-mon[36748]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:05.067 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:04 smithi102 ceph-mon[36748]: Reconfiguring mon.c (monmap changed)... 2024-03-21T11:14:05.067 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:04 smithi102 ceph-mon[36748]: Reconfiguring daemon mon.c on smithi102 2024-03-21T11:14:05.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:04 smithi100 ceph-mon[36245]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:05.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:04 smithi100 ceph-mon[36245]: Reconfiguring mon.c (monmap changed)... 2024-03-21T11:14:05.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:04 smithi100 ceph-mon[36245]: Reconfiguring daemon mon.c on smithi102 2024-03-21T11:14:05.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:04 smithi047 ceph-mon[31351]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:05.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:04 smithi047 ceph-mon[31351]: Reconfiguring mon.c (monmap changed)... 2024-03-21T11:14:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:04 smithi047 ceph-mon[31351]: Reconfiguring daemon mon.c on smithi102 2024-03-21T11:14:05.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.862+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1339908178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc01033e0 msgr2=0x7f2fc0103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:05.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.862+0000 7f2fc6fc0700 1 --2- 172.21.15.102:0/1339908178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc01033e0 0x7f2fc0103800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009910 tx=0x7f2fb4009c20 comp rx=0 tx=0).stop 2024-03-21T11:14:05.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.862+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1339908178 shutdown_connections 2024-03-21T11:14:05.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.862+0000 7f2fc6fc0700 1 --2- 172.21.15.102:0/1339908178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc01033e0 0x7f2fc0103800 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:05.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.862+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1339908178 >> 172.21.15.102:0/1339908178 conn(0x7f2fc00fd280 msgr2=0x7f2fc00ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:14:05.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.862+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1339908178 shutdown_connections 2024-03-21T11:14:05.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.862+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1339908178 wait complete. 2024-03-21T11:14:05.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.862+0000 7f2fc6fc0700 1 Processor -- start 2024-03-21T11:14:05.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.862+0000 7f2fc6fc0700 1 -- start start 2024-03-21T11:14:05.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.862+0000 7f2fc6fc0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc01033e0 0x7f2fc01975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:05.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.862+0000 7f2fc6fc0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fc0197b30 0x7f2fc019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:05.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.862+0000 7f2fc6fc0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fc0197fb0 0x7f2fc019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:05.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fc6fc0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0107e50 con 0x7f2fc01033e0 2024-03-21T11:14:05.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fc6fc0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0107cd0 con 0x7f2fc0197b30 2024-03-21T11:14:05.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fc6fc0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0107fd0 con 0x7f2fc0197fb0 2024-03-21T11:14:05.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fc555d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fc0197fb0 0x7f2fc019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:05.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fc555d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fc0197fb0 0x7f2fc019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52900/0 (socket says 172.21.15.102:52900) 2024-03-21T11:14:05.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fc555d700 1 -- 172.21.15.102:0/1817342097 learned_addr learned my addr 172.21.15.102:0/1817342097 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-21T11:14:05.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fc4d5c700 1 --2- 172.21.15.102:0/1817342097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc01033e0 0x7f2fc01975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:05.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fbffff700 1 --2- 172.21.15.102:0/1817342097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fc0197b30 0x7f2fc019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:05.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fc555d700 1 -- 172.21.15.102:0/1817342097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fc0197b30 msgr2=0x7f2fc019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:05.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fc555d700 1 --2- 172.21.15.102:0/1817342097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fc0197b30 0x7f2fc019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:05.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fc555d700 1 -- 172.21.15.102:0/1817342097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc01033e0 msgr2=0x7f2fc01975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:05.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fc555d700 1 --2- 172.21.15.102:0/1817342097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc01033e0 0x7f2fc01975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:05.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fc555d700 1 -- 172.21.15.102:0/1817342097 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc019fa00 con 0x7f2fc0197fb0 2024-03-21T11:14:05.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fc555d700 1 --2- 172.21.15.102:0/1817342097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fc0197fb0 0x7f2fc019f130 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800ba50 tx=0x7f2fb800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:14:05.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fbdffb700 1 -- 172.21.15.102:0/1817342097 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8019070 con 0x7f2fc0197fb0 2024-03-21T11:14:05.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1817342097 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc019fcf0 con 0x7f2fc0197fb0 2024-03-21T11:14:05.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fbdffb700 1 -- 172.21.15.102:0/1817342097 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fb8014440 con 0x7f2fc0197fb0 2024-03-21T11:14:05.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.863+0000 7f2fbdffb700 1 -- 172.21.15.102:0/1817342097 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80075b0 con 0x7f2fc0197fb0 2024-03-21T11:14:05.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.864+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1817342097 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc01a0250 con 0x7f2fc0197fb0 2024-03-21T11:14:05.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.864+0000 7f2fbdffb700 1 -- 172.21.15.102:0/1817342097 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f2fb801c070 con 0x7f2fc0197fb0 2024-03-21T11:14:05.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.864+0000 7f2fbdffb700 1 --2- 172.21.15.102:0/1817342097 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2fb00396b0 0x7f2fb003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:05.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.864+0000 7f2fbdffb700 1 -- 172.21.15.102:0/1817342097 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2fb804b940 con 0x7f2fc0197fb0 2024-03-21T11:14:05.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.864+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1817342097 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc00637c0 con 0x7f2fc0197fb0 2024-03-21T11:14:05.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.865+0000 7f2fc4d5c700 1 --2- 172.21.15.102:0/1817342097 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2fb00396b0 0x7f2fb003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:05.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.865+0000 7f2fc4d5c700 1 --2- 172.21.15.102:0/1817342097 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2fb00396b0 0x7f2fb003bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4005200 tx=0x7f2fb401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:14:05.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:05.870+0000 7f2fbdffb700 1 -- 172.21.15.102:0/1817342097 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2fb8012500 con 0x7f2fc0197fb0 2024-03-21T11:14:06.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:06.080+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1817342097 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi100=y;smithi102=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fc002de50 con 0x7f2fb00396b0 2024-03-21T11:14:06.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:06.092+0000 7f2fbdffb700 1 -- 172.21.15.102:0/1817342097 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2fc002de50 con 0x7f2fb00396b0 2024-03-21T11:14:06.093 INFO:teuthology.orchestra.run.smithi102.stdout:Scheduled mgr update... 2024-03-21T11:14:06.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:06.094+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1817342097 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2fb00396b0 msgr2=0x7f2fb003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:06.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:06.094+0000 7f2fc6fc0700 1 --2- 172.21.15.102:0/1817342097 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2fb00396b0 0x7f2fb003bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4005200 tx=0x7f2fb401a040 comp rx=0 tx=0).stop 2024-03-21T11:14:06.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:06.094+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1817342097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fc0197fb0 msgr2=0x7f2fc019f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:06.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:06.094+0000 7f2fc6fc0700 1 --2- 172.21.15.102:0/1817342097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fc0197fb0 0x7f2fc019f130 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800ba50 tx=0x7f2fb800bd60 comp rx=0 tx=0).stop 2024-03-21T11:14:06.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:06.094+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1817342097 shutdown_connections 2024-03-21T11:14:06.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:06.094+0000 7f2fc6fc0700 1 --2- 172.21.15.102:0/1817342097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fc0197fb0 0x7f2fc019f130 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:06.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:06.094+0000 7f2fc6fc0700 1 --2- 172.21.15.102:0/1817342097 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2fb00396b0 0x7f2fb003bb70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:06.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:06.094+0000 7f2fc6fc0700 1 --2- 172.21.15.102:0/1817342097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fc01033e0 0x7f2fc01975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:06.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:06.094+0000 7f2fc6fc0700 1 --2- 172.21.15.102:0/1817342097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fc0197b30 0x7f2fc019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:06.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:06.094+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1817342097 >> 172.21.15.102:0/1817342097 conn(0x7f2fc00fd280 msgr2=0x7f2fc00fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:14:06.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:06.094+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1817342097 shutdown_connections 2024-03-21T11:14:06.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:14:06.094+0000 7f2fc6fc0700 1 -- 172.21.15.102:0/1817342097 wait complete. 2024-03-21T11:14:06.607 DEBUG:teuthology.orchestra.run.smithi100:mgr.y> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mgr.y.service 2024-03-21T11:14:06.610 DEBUG:teuthology.orchestra.run.smithi102:mgr.z> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mgr.z.service 2024-03-21T11:14:06.612 INFO:tasks.cephadm:Deploying OSDs... 2024-03-21T11:14:06.612 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:14:06.612 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T11:14:06.669 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T11:14:06.669 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-03-21T11:14:06.723 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T11:14:06.723 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T11:14:06.723 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-03-21T11:14:06.723 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T11:14:06.723 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T11:14:06.723 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-21 11:13:01.176899820 +0000 2024-03-21T11:14:06.723 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-21 11:08:05.244906761 +0000 2024-03-21T11:14:06.724 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-21 11:08:05.244906761 +0000 2024-03-21T11:14:06.724 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-03-21 11:08:05.244906761 +0000 2024-03-21T11:14:06.724 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T11:14:06.819 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-21T11:14:06.819 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-21T11:14:06.819 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000302345 s, 1.7 MB/s 2024-03-21T11:14:06.823 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T11:14:06.886 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-03-21T11:14:06.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:06.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:06.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:06.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:06.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:06.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T11:14:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T11:14:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T11:14:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:06.941 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T11:14:06.941 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T11:14:06.941 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-21T11:14:06.941 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T11:14:06.941 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T11:14:06.941 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-21 11:13:01.272897726 +0000 2024-03-21T11:14:06.941 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-21 11:08:05.814894172 +0000 2024-03-21T11:14:06.941 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-21 11:08:05.814894172 +0000 2024-03-21T11:14:06.941 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-03-21 11:08:05.814894172 +0000 2024-03-21T11:14:06.942 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T11:14:06.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:06.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T11:14:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T11:14:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T11:14:06.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:07.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:07.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:07.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:07.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:07.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:07.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:07.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:07.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:07.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T11:14:07.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T11:14:07.027 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T11:14:07.027 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:07.035 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-21T11:14:07.035 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-21T11:14:07.035 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000281576 s, 1.8 MB/s 2024-03-21T11:14:07.040 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T11:14:07.099 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-03-21T11:14:07.154 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T11:14:07.155 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T11:14:07.155 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-21T11:14:07.155 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T11:14:07.155 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T11:14:07.155 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-21 11:13:01.335896353 +0000 2024-03-21T11:14:07.155 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-21 11:08:06.401881207 +0000 2024-03-21T11:14:07.155 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-21 11:08:06.401881207 +0000 2024-03-21T11:14:07.155 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-03-21 11:08:06.401881207 +0000 2024-03-21T11:14:07.155 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T11:14:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-21T11:14:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-21T11:14:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000276985 s, 1.8 MB/s 2024-03-21T11:14:07.255 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T11:14:07.315 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-03-21T11:14:07.368 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T11:14:07.368 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T11:14:07.368 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-21T11:14:07.369 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T11:14:07.369 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T11:14:07.369 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-21 11:13:01.398894979 +0000 2024-03-21T11:14:07.369 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-21 11:08:06.969868665 +0000 2024-03-21T11:14:07.369 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-21 11:08:06.969868665 +0000 2024-03-21T11:14:07.369 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-03-21 11:08:06.969868665 +0000 2024-03-21T11:14:07.369 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T11:14:07.463 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-21T11:14:07.463 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-21T11:14:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000271641 s, 1.9 MB/s 2024-03-21T11:14:07.468 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T11:14:07.527 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T11:14:07.527 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T11:14:07.546 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T11:14:07.546 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-03-21T11:14:07.603 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T11:14:07.603 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T11:14:07.603 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-03-21T11:14:07.603 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T11:14:07.603 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T11:14:07.603 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-21 11:13:50.534630084 +0000 2024-03-21T11:14:07.603 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-21 11:06:53.621555561 +0000 2024-03-21T11:14:07.604 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-21 11:06:53.621555561 +0000 2024-03-21T11:14:07.604 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-21 11:06:53.621555561 +0000 2024-03-21T11:14:07.604 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T11:14:07.671 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-21T11:14:07.672 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-21T11:14:07.672 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000282988 s, 1.8 MB/s 2024-03-21T11:14:07.673 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T11:14:07.730 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-03-21T11:14:07.782 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:07 smithi102 ceph-mon[36748]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi100=y;smithi102=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:14:07.782 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:07 smithi102 ceph-mon[36748]: Saving service mgr spec with placement smithi047=x;smithi100=y;smithi102=z;count:3 2024-03-21T11:14:07.782 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:07 smithi102 ceph-mon[36748]: Deploying daemon mgr.z on smithi102 2024-03-21T11:14:07.787 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T11:14:07.787 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T11:14:07.787 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-21T11:14:07.787 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T11:14:07.787 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T11:14:07.787 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-21 11:13:50.534630084 +0000 2024-03-21T11:14:07.787 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-21 11:06:53.953546775 +0000 2024-03-21T11:14:07.788 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-21 11:06:53.953546775 +0000 2024-03-21T11:14:07.788 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-21 11:06:53.953546775 +0000 2024-03-21T11:14:07.788 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T11:14:07.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:07 smithi047 ceph-mon[31351]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi100=y;smithi102=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:14:07.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:07 smithi047 ceph-mon[31351]: Saving service mgr spec with placement smithi047=x;smithi100=y;smithi102=z;count:3 2024-03-21T11:14:07.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:07 smithi047 ceph-mon[31351]: Deploying daemon mgr.z on smithi102 2024-03-21T11:14:07.856 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-21T11:14:07.856 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-21T11:14:07.856 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000230764 s, 2.2 MB/s 2024-03-21T11:14:07.857 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T11:14:07.914 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-03-21T11:14:07.970 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T11:14:07.970 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T11:14:07.970 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-21T11:14:07.970 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T11:14:07.970 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T11:14:07.970 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-21 11:13:50.534630084 +0000 2024-03-21T11:14:07.970 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-21 11:06:54.291537827 +0000 2024-03-21T11:14:07.970 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-21 11:06:54.291537827 +0000 2024-03-21T11:14:07.971 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-21 11:06:54.291537827 +0000 2024-03-21T11:14:07.971 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T11:14:08.034 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:07 smithi100 ceph-mon[36245]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi100=y;smithi102=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:14:08.035 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:07 smithi100 ceph-mon[36245]: Saving service mgr spec with placement smithi047=x;smithi100=y;smithi102=z;count:3 2024-03-21T11:14:08.035 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:07 smithi100 ceph-mon[36245]: Deploying daemon mgr.z on smithi102 2024-03-21T11:14:08.037 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-21T11:14:08.037 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-21T11:14:08.037 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000222491 s, 2.3 MB/s 2024-03-21T11:14:08.038 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T11:14:08.094 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-03-21T11:14:08.149 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T11:14:08.149 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T11:14:08.149 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-21T11:14:08.149 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T11:14:08.149 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T11:14:08.149 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-21 11:13:50.535630058 +0000 2024-03-21T11:14:08.149 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-21 11:06:54.630528853 +0000 2024-03-21T11:14:08.149 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-21 11:06:54.630528853 +0000 2024-03-21T11:14:08.149 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-21 11:06:54.630528853 +0000 2024-03-21T11:14:08.150 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T11:14:08.162 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:07 smithi102 systemd[1]: Starting Ceph mgr.z for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:14:08.215 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-21T11:14:08.215 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-21T11:14:08.215 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000198805 s, 2.6 MB/s 2024-03-21T11:14:08.216 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T11:14:08.272 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-03-21T11:14:08.272 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T11:14:08.288 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T11:14:08.289 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-03-21T11:14:08.343 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T11:14:08.343 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T11:14:08.344 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-21T11:14:08.344 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T11:14:08.344 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T11:14:08.344 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-03-21 11:14:07.885790810 +0000 2024-03-21T11:14:08.344 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-03-21 11:06:52.466139717 +0000 2024-03-21T11:14:08.344 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-03-21 11:06:52.466139717 +0000 2024-03-21T11:14:08.344 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-03-21 11:06:52.466139717 +0000 2024-03-21T11:14:08.344 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T11:14:08.412 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-03-21T11:14:08.412 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-03-21T11:14:08.412 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000195385 s, 2.6 MB/s 2024-03-21T11:14:08.412 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:08 smithi102 podman[38415]: 2024-03-21 11:14:08.064985536 +0000 UTC m=+0.017330100 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:14:08.412 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:08 smithi102 podman[38415]: 2024-03-21 11:14:08.188627341 +0000 UTC m=+0.140971917 container create a64d8e76571a9b7bfbd59dedc2a666ee940a879965e921f6fb77dbb0a0fbbe30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:14:08.413 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:08 smithi102 podman[38415]: 2024-03-21 11:14:08.382417281 +0000 UTC m=+0.334761845 container init a64d8e76571a9b7bfbd59dedc2a666ee940a879965e921f6fb77dbb0a0fbbe30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-21T11:14:08.413 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:08 smithi102 podman[38415]: 2024-03-21 11:14:08.384913482 +0000 UTC m=+0.337258043 container start a64d8e76571a9b7bfbd59dedc2a666ee940a879965e921f6fb77dbb0a0fbbe30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:14:08.413 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:08 smithi102 ceph-mgr[38466]: -- 172.21.15.102:0/278481775 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cc8aab65a0 con 0x55cc8aa8a800 2024-03-21T11:14:08.414 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T11:14:08.469 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-03-21T11:14:08.524 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T11:14:08.524 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T11:14:08.524 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-03-21T11:14:08.524 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T11:14:08.525 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T11:14:08.525 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-03-21 11:14:07.886790791 +0000 2024-03-21T11:14:08.525 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-03-21 11:06:52.816132735 +0000 2024-03-21T11:14:08.525 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-03-21 11:06:52.816132735 +0000 2024-03-21T11:14:08.525 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-03-21 11:06:52.816132735 +0000 2024-03-21T11:14:08.525 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T11:14:08.590 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-03-21T11:14:08.590 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-03-21T11:14:08.590 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000183401 s, 2.8 MB/s 2024-03-21T11:14:08.592 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T11:14:08.647 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-03-21T11:14:08.693 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:08 smithi102 bash[38415]: a64d8e76571a9b7bfbd59dedc2a666ee940a879965e921f6fb77dbb0a0fbbe30 2024-03-21T11:14:08.693 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:08 smithi102 systemd[1]: Started Ceph mgr.z for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:14:08.702 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T11:14:08.702 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T11:14:08.702 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-21T11:14:08.703 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T11:14:08.703 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T11:14:08.703 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-03-21 11:14:07.886790791 +0000 2024-03-21T11:14:08.703 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-03-21 11:06:53.153126012 +0000 2024-03-21T11:14:08.703 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-03-21 11:06:53.153126012 +0000 2024-03-21T11:14:08.703 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-03-21 11:06:53.153126012 +0000 2024-03-21T11:14:08.703 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T11:14:08.768 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-03-21T11:14:08.768 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-03-21T11:14:08.768 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000198871 s, 2.6 MB/s 2024-03-21T11:14:08.769 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T11:14:08.824 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-03-21T11:14:08.879 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T11:14:08.879 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T11:14:08.879 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-21T11:14:08.879 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T11:14:08.880 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T11:14:08.880 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-03-21 11:14:07.886790791 +0000 2024-03-21T11:14:08.880 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-03-21 11:06:53.492119249 +0000 2024-03-21T11:14:08.880 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-03-21 11:06:53.492119249 +0000 2024-03-21T11:14:08.880 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-03-21 11:06:53.492119249 +0000 2024-03-21T11:14:08.880 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T11:14:08.944 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:08 smithi102 ceph-mon[36748]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:08.944 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:08 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:08.944 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:08 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:08.944 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:08 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:08.945 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:08 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T11:14:08.945 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:08 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T11:14:08.945 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:08 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T11:14:08.945 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:08 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:08.945 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-03-21T11:14:08.945 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-03-21T11:14:08.945 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000189083 s, 2.7 MB/s 2024-03-21T11:14:08.946 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T11:14:09.001 INFO:tasks.cephadm:Deploying osd.0 on smithi047 with /dev/vg_nvme/lv_4... 2024-03-21T11:14:09.002 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T11:14:09.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:08 smithi047 ceph-mon[31351]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:09.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:08 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:09.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:08 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:09.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:08 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:09.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:08 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T11:14:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:08 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T11:14:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:08 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T11:14:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:08 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:09.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:08 smithi100 ceph-mon[36245]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:09.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:08 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:09.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:08 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:09.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:08 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:09.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:08 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T11:14:09.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:08 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T11:14:09.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:08 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T11:14:09.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:08 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:09.241 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:08 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:08.952+0000 7f7967261200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T11:14:09.241 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:09 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:09.048+0000 7f7967261200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T11:14:09.460 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:14:09.888 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:09 smithi047 ceph-mon[31351]: Deploying daemon mgr.y on smithi100 2024-03-21T11:14:09.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:09 smithi100 ceph-mon[36245]: Deploying daemon mgr.y on smithi100 2024-03-21T11:14:09.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:09 smithi102 ceph-mon[36748]: Deploying daemon mgr.y on smithi100 2024-03-21T11:14:09.990 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:09 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:09.685+0000 7f7967261200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T11:14:09.991 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:09 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:09.778+0000 7f7967261200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T11:14:09.991 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:09 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:09.918+0000 7f7967261200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T11:14:10.353 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:10 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:10.011+0000 7f7967261200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T11:14:10.353 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:10 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:10.092+0000 7f7967261200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T11:14:10.354 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:10 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:10.352+0000 7f7967261200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T11:14:10.638 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:10 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:10.460+0000 7f7967261200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T11:14:10.638 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:10 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:10.544+0000 7f7967261200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T11:14:10.638 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:10 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:10.637+0000 7f7967261200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T11:14:10.703 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:10 smithi100 systemd[1]: Starting Ceph mgr.y for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:14:10.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:10 smithi102 ceph-mon[36748]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:10.990 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:10 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:10.811+0000 7f7967261200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T11:14:10.995 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:10 smithi047 ceph-mon[31351]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:10.999 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:10 smithi100 ceph-mon[36245]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:10.999 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:10 smithi100 podman[37648]: 2024-03-21 11:14:10.798532004 +0000 UTC m=+0.103385454 container create bdc77f29aa77ba6a1d9099388492869f69fd9cf5be4a63f73f31f5344108fb2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T11:14:10.999 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:10 smithi100 podman[37648]: 2024-03-21 11:14:10.712640049 +0000 UTC m=+0.017493517 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:14:10.999 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:11 smithi100 podman[37648]: 2024-03-21 11:14:11.000583764 +0000 UTC m=+0.305437215 container init bdc77f29aa77ba6a1d9099388492869f69fd9cf5be4a63f73f31f5344108fb2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-21T11:14:11.393 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:11 smithi100 podman[37648]: 2024-03-21 11:14:11.003207594 +0000 UTC m=+0.308061053 container start bdc77f29aa77ba6a1d9099388492869f69fd9cf5be4a63f73f31f5344108fb2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-21T11:14:11.394 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:11 smithi100 ceph-mgr[37668]: -- 172.21.15.100:0/1188540633 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55be20b9e5a0 con 0x55be20b72c00 2024-03-21T11:14:11.394 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:11 smithi100 bash[37648]: bdc77f29aa77ba6a1d9099388492869f69fd9cf5be4a63f73f31f5344108fb2b 2024-03-21T11:14:11.394 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:11 smithi100 systemd[1]: Started Ceph mgr.y for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:14:11.892 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:11 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:11.550+0000 7f50c03be200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T11:14:11.892 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:11 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:11.647+0000 7f50c03be200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T11:14:12.159 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:11 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:11.799+0000 7f7967261200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T11:14:12.159 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:11 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:11.934+0000 7f7967261200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T11:14:12.344 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T11:14:12.344 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T11:14:12.344 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-03-21T11:14:12.344 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-03-21T11:14:12.344 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0382367 s, 274 MB/s 2024-03-21T11:14:12.344 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 2024-03-21T11:14:12.344 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-03-21T11:14:12.431 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:12 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:12.334+0000 7f50c03be200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T11:14:12.431 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:12 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:12.431+0000 7f50c03be200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T11:14:12.431 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:12 smithi100 ceph-mon[36245]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:12.431 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:12.432 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:12.432 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:12.432 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:12.432 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:12.454 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:12 smithi102 ceph-mon[36748]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:12.455 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:12.455 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:12.455 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:12.455 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:12.455 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:12.455 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:12 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:12.284+0000 7f7967261200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T11:14:12.455 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:12 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:12.379+0000 7f7967261200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T11:14:12.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:12 smithi047 ceph-mon[31351]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:12.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:12.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:12.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:12.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:12.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:12.740 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:12 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:12.582+0000 7f7967261200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T11:14:12.755 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:12 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:12.572+0000 7f50c03be200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T11:14:12.756 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:12 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:12.665+0000 7f50c03be200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T11:14:12.756 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:12 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:12.756+0000 7f50c03be200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T11:14:13.049 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:12 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:12.767+0000 7f7967261200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T11:14:13.308 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:13 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:13.049+0000 7f7967261200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T11:14:13.308 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:13 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:13.308+0000 7f7967261200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T11:14:13.310 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi047:vg_nvme/lv_4 2024-03-21T11:14:13.310 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:13.023+0000 7f50c03be200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T11:14:13.310 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:13.133+0000 7f50c03be200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T11:14:13.311 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:13.217+0000 7f50c03be200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T11:14:13.311 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:13.310+0000 7f50c03be200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T11:14:13.642 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:13.482+0000 7f50c03be200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T11:14:13.736 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:14:13.740 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:13 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:13.542+0000 7f7967261200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T11:14:14.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T11:14:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T11:14:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:14.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T11:14:14.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T11:14:14.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:14.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T11:14:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T11:14:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:14.740 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:14 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:14.336+0000 7f7967261200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T11:14:14.800 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:14 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:14.438+0000 7f50c03be200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T11:14:14.800 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:14 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:14.566+0000 7f50c03be200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T11:14:15.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:14 smithi047 ceph-mon[31351]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:14 smithi047 ceph-mon[31351]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T11:14:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:14 smithi047 ceph-mon[31351]: Reconfiguring daemon mgr.x on smithi047 2024-03-21T11:14:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:14 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:14 smithi100 ceph-mon[36245]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:14 smithi100 ceph-mon[36245]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T11:14:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:14 smithi100 ceph-mon[36245]: Reconfiguring daemon mgr.x on smithi047 2024-03-21T11:14:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:14 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:15.143 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:14 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:14.895+0000 7f50c03be200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T11:14:15.143 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:14 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:14.983+0000 7f50c03be200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T11:14:15.205 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:14 smithi102 ceph-mon[36748]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:15.206 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:14 smithi102 ceph-mon[36748]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T11:14:15.206 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:14 smithi102 ceph-mon[36748]: Reconfiguring daemon mgr.x on smithi047 2024-03-21T11:14:15.206 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:14 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:15.206 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:14 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:14.899+0000 7f7967261200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T11:14:15.206 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:14 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:14.995+0000 7f7967261200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T11:14:15.206 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:14:15 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z[38434]: 2024-03-21T11:14:15.205+0000 7f7967261200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T11:14:15.633 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:15 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:15.174+0000 7f50c03be200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T11:14:15.633 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:15 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:15.354+0000 7f50c03be200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T11:14:15.889 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:15 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:15.633+0000 7f50c03be200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T11:14:15.889 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:15 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:15.888+0000 7f50c03be200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T11:14:15.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:15 smithi100 ceph-mon[36245]: from='mgr.? 172.21.15.102:0/2732782697' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T11:14:15.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:15 smithi100 ceph-mon[36245]: from='mgr.? 172.21.15.102:0/2732782697' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T11:14:15.889 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:15 smithi100 ceph-mon[36245]: Standby manager daemon z started 2024-03-21T11:14:15.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:15 smithi100 ceph-mon[36245]: from='mgr.? 172.21.15.102:0/2732782697' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T11:14:15.890 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:15 smithi100 ceph-mon[36245]: from='mgr.? 172.21.15.102:0/2732782697' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T11:14:16.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:15 smithi047 ceph-mon[31351]: from='mgr.? 172.21.15.102:0/2732782697' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T11:14:16.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:15 smithi047 ceph-mon[31351]: from='mgr.? 172.21.15.102:0/2732782697' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T11:14:16.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:15 smithi047 ceph-mon[31351]: Standby manager daemon z started 2024-03-21T11:14:16.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:15 smithi047 ceph-mon[31351]: from='mgr.? 172.21.15.102:0/2732782697' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T11:14:16.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:15 smithi047 ceph-mon[31351]: from='mgr.? 172.21.15.102:0/2732782697' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T11:14:16.142 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:16 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:16.121+0000 7f50c03be200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T11:14:16.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:15 smithi102 ceph-mon[36748]: from='mgr.? 172.21.15.102:0/2732782697' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T11:14:16.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:15 smithi102 ceph-mon[36748]: from='mgr.? 172.21.15.102:0/2732782697' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T11:14:16.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:15 smithi102 ceph-mon[36748]: Standby manager daemon z started 2024-03-21T11:14:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:15 smithi102 ceph-mon[36748]: from='mgr.? 172.21.15.102:0/2732782697' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T11:14:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:15 smithi102 ceph-mon[36748]: from='mgr.? 172.21.15.102:0/2732782697' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T11:14:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.747+0000 7f2f6ee00700 1 -- 172.21.15.47:0/375708496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f68104030 msgr2=0x7f2f68104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:16.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.747+0000 7f2f6ee00700 1 --2- 172.21.15.47:0/375708496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f68104030 0x7f2f68104490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f540099c0 tx=0x7f2f54009cd0 comp rx=0 tx=0).stop 2024-03-21T11:14:16.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.747+0000 7f2f6ee00700 1 -- 172.21.15.47:0/375708496 shutdown_connections 2024-03-21T11:14:16.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.747+0000 7f2f6ee00700 1 --2- 172.21.15.47:0/375708496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f681049d0 0x7f2f6810c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:16.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.747+0000 7f2f6ee00700 1 --2- 172.21.15.47:0/375708496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f68105af0 0x7f2f68105f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:16.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.747+0000 7f2f6ee00700 1 --2- 172.21.15.47:0/375708496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f68104030 0x7f2f68104490 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:16.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.747+0000 7f2f6ee00700 1 -- 172.21.15.47:0/375708496 >> 172.21.15.47:0/375708496 conn(0x7f2f680ff9b0 msgr2=0x7f2f68101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:14:16.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.747+0000 7f2f6ee00700 1 -- 172.21.15.47:0/375708496 shutdown_connections 2024-03-21T11:14:16.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.747+0000 7f2f6ee00700 1 -- 172.21.15.47:0/375708496 wait complete. 2024-03-21T11:14:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.748+0000 7f2f6ee00700 1 Processor -- start 2024-03-21T11:14:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.749+0000 7f2f6ee00700 1 -- start start 2024-03-21T11:14:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.749+0000 7f2f6ee00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f68104030 0x7f2f6819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.749+0000 7f2f6ee00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f681049d0 0x7f2f6819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:16.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.749+0000 7f2f6ee00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f68105af0 0x7f2f681a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:16.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.749+0000 7f2f6ee00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f68110120 con 0x7f2f68104030 2024-03-21T11:14:16.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.749+0000 7f2f6ee00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6810ffa0 con 0x7f2f681049d0 2024-03-21T11:14:16.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.749+0000 7f2f6ee00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f681102a0 con 0x7f2f68105af0 2024-03-21T11:14:16.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.749+0000 7f2f67400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f681049d0 0x7f2f6819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:16.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.749+0000 7f2f67e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f68104030 0x7f2f6819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:16.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.749+0000 7f2f67400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f681049d0 0x7f2f6819ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44992/0 (socket says 172.21.15.47:44992) 2024-03-21T11:14:16.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.749+0000 7f2f67400700 1 -- 172.21.15.47:0/1321643331 learned_addr learned my addr 172.21.15.47:0/1321643331 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:14:16.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.749+0000 7f2f6ce00700 1 --2- 172.21.15.47:0/1321643331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f68105af0 0x7f2f681a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:16.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.750+0000 7f2f67400700 1 -- 172.21.15.47:0/1321643331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f68105af0 msgr2=0x7f2f681a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:16.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.750+0000 7f2f67400700 1 --2- 172.21.15.47:0/1321643331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f68105af0 0x7f2f681a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:16.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.750+0000 7f2f67400700 1 -- 172.21.15.47:0/1321643331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f68104030 msgr2=0x7f2f6819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:16.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.750+0000 7f2f67400700 1 --2- 172.21.15.47:0/1321643331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f68104030 0x7f2f6819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:16.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.750+0000 7f2f67400700 1 -- 172.21.15.47:0/1321643331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f681a2360 con 0x7f2f681049d0 2024-03-21T11:14:16.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.750+0000 7f2f67400700 1 --2- 172.21.15.47:0/1321643331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f681049d0 0x7f2f6819ac40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2f54005b40 tx=0x7f2f54009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:14:16.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.750+0000 7f2f64c00700 1 -- 172.21.15.47:0/1321643331 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f54027070 con 0x7f2f681049d0 2024-03-21T11:14:16.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.750+0000 7f2f64c00700 1 -- 172.21.15.47:0/1321643331 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f5400fb40 con 0x7f2f681049d0 2024-03-21T11:14:16.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.750+0000 7f2f64c00700 1 -- 172.21.15.47:0/1321643331 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f540173f0 con 0x7f2f681049d0 2024-03-21T11:14:16.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.750+0000 7f2f6ee00700 1 -- 172.21.15.47:0/1321643331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f681a25f0 con 0x7f2f681049d0 2024-03-21T11:14:16.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.751+0000 7f2f6ee00700 1 -- 172.21.15.47:0/1321643331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f681a29f0 con 0x7f2f681049d0 2024-03-21T11:14:16.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.752+0000 7f2f6ee00700 1 -- 172.21.15.47:0/1321643331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f68068e50 con 0x7f2f681049d0 2024-03-21T11:14:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.752+0000 7f2f64c00700 1 -- 172.21.15.47:0/1321643331 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 14) v1 ==== 92074+0+0 (secure 0 0 0) 0x7f2f5402a070 con 0x7f2f681049d0 2024-03-21T11:14:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.753+0000 7f2f64c00700 1 --2- 172.21.15.47:0/1321643331 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2f4c06e1c0 0x7f2f4c070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:16.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.754+0000 7f2f64c00700 1 -- 172.21.15.47:0/1321643331 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2f5408db50 con 0x7f2f681049d0 2024-03-21T11:14:16.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.754+0000 7f2f67e00700 1 --2- 172.21.15.47:0/1321643331 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2f4c06e1c0 0x7f2f4c070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:16.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.754+0000 7f2f67e00700 1 --2- 172.21.15.47:0/1321643331 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2f4c06e1c0 0x7f2f4c070680 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f68105470 tx=0x7f2f58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:14:16.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.758+0000 7f2f64c00700 1 -- 172.21.15.47:0/1321643331 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2f54013070 con 0x7f2f681049d0 2024-03-21T11:14:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:16.998+0000 7f2f6ee00700 1 -- 172.21.15.47:0/1321643331 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f6802de10 con 0x7f2f4c06e1c0 2024-03-21T11:14:16.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:16 smithi047 ceph-mon[31351]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:16.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:16 smithi047 ceph-mon[31351]: mgrmap e14: x(active, since 86s), standbys: z 2024-03-21T11:14:16.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T11:14:16.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:16.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:16.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:16.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:16.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:16.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:17.142 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:16 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:16.918+0000 7f50c03be200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T11:14:17.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:16 smithi100 ceph-mon[36245]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:17.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:16 smithi100 ceph-mon[36245]: mgrmap e14: x(active, since 86s), standbys: z 2024-03-21T11:14:17.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T11:14:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:17.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:16 smithi102 ceph-mon[36748]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:17.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:16 smithi102 ceph-mon[36748]: mgrmap e14: x(active, since 86s), standbys: z 2024-03-21T11:14:17.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T11:14:17.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:17.802 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:17.491+0000 7f50c03be200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T11:14:17.802 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:17.587+0000 7f50c03be200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T11:14:17.802 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:14:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y[37664]: 2024-03-21T11:14:17.802+0000 7f50c03be200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T11:14:18.095 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:17 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:14:18.095 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:17 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:14:18.095 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:17 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:18.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:17 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:14:18.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:17 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:14:18.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:17 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:18.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:17 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:14:18.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:17 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:14:18.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:17 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:18.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:18.842+0000 7f2f64c00700 1 -- 172.21.15.47:0/1321643331 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7f2f540586c0 con 0x7f2f681049d0 2024-03-21T11:14:19.084 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:18 smithi102 ceph-mon[36748]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:19.084 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:18 smithi102 ceph-mon[36748]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:14:19.084 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:18 smithi102 ceph-mon[36748]: from='mgr.? 172.21.15.100:0/3245719379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T11:14:19.084 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:18 smithi102 ceph-mon[36748]: from='mgr.? 172.21.15.100:0/3245719379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T11:14:19.084 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:18 smithi102 ceph-mon[36748]: Standby manager daemon y started 2024-03-21T11:14:19.084 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:18 smithi102 ceph-mon[36748]: from='mgr.? 172.21.15.100:0/3245719379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T11:14:19.084 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:18 smithi102 ceph-mon[36748]: from='mgr.? 172.21.15.100:0/3245719379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T11:14:19.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:18 smithi100 ceph-mon[36245]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:18 smithi100 ceph-mon[36245]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:14:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:18 smithi100 ceph-mon[36245]: from='mgr.? 172.21.15.100:0/3245719379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T11:14:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:18 smithi100 ceph-mon[36245]: from='mgr.? 172.21.15.100:0/3245719379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T11:14:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:18 smithi100 ceph-mon[36245]: Standby manager daemon y started 2024-03-21T11:14:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:18 smithi100 ceph-mon[36245]: from='mgr.? 172.21.15.100:0/3245719379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T11:14:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:18 smithi100 ceph-mon[36245]: from='mgr.? 172.21.15.100:0/3245719379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T11:14:19.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:18 smithi047 ceph-mon[31351]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:19.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:18 smithi047 ceph-mon[31351]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:14:19.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:18 smithi047 ceph-mon[31351]: from='mgr.? 172.21.15.100:0/3245719379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T11:14:19.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:18 smithi047 ceph-mon[31351]: from='mgr.? 172.21.15.100:0/3245719379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T11:14:19.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:18 smithi047 ceph-mon[31351]: Standby manager daemon y started 2024-03-21T11:14:19.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:18 smithi047 ceph-mon[31351]: from='mgr.? 172.21.15.100:0/3245719379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T11:14:19.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:18 smithi047 ceph-mon[31351]: from='mgr.? 172.21.15.100:0/3245719379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T11:14:20.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:19 smithi047 ceph-mon[31351]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-03-21T11:14:20.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:19 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T11:14:20.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:19 smithi100 ceph-mon[36245]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-03-21T11:14:20.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:19 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T11:14:20.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:19 smithi102 ceph-mon[36748]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-03-21T11:14:20.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:19 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T11:14:21.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:20 smithi100 ceph-mon[36245]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:21.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:20 smithi102 ceph-mon[36748]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:21.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:20 smithi047 ceph-mon[31351]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:22.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:21 smithi047 ceph-mon[31351]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:22.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:21 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1864609416' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68bd29ba-c4c1-482d-b233-47e42371a12c"}]: dispatch 2024-03-21T11:14:22.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:21 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68bd29ba-c4c1-482d-b233-47e42371a12c"}]: dispatch 2024-03-21T11:14:22.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:21 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68bd29ba-c4c1-482d-b233-47e42371a12c"}]': finished 2024-03-21T11:14:22.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:21 smithi047 ceph-mon[31351]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T11:14:22.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:21 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:22.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:21 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2503389249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:14:22.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:21 smithi100 ceph-mon[36245]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:22.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:21 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1864609416' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68bd29ba-c4c1-482d-b233-47e42371a12c"}]: dispatch 2024-03-21T11:14:22.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:21 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68bd29ba-c4c1-482d-b233-47e42371a12c"}]: dispatch 2024-03-21T11:14:22.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:21 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68bd29ba-c4c1-482d-b233-47e42371a12c"}]': finished 2024-03-21T11:14:22.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:21 smithi100 ceph-mon[36245]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T11:14:22.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:21 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:22.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:21 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2503389249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:14:22.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:21 smithi102 ceph-mon[36748]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:22.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:21 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1864609416' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68bd29ba-c4c1-482d-b233-47e42371a12c"}]: dispatch 2024-03-21T11:14:22.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:21 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68bd29ba-c4c1-482d-b233-47e42371a12c"}]: dispatch 2024-03-21T11:14:22.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:21 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68bd29ba-c4c1-482d-b233-47e42371a12c"}]': finished 2024-03-21T11:14:22.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:21 smithi102 ceph-mon[36748]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T11:14:22.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:21 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:22.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:21 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2503389249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:14:24.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:23 smithi102 ceph-mon[36748]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:24.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:23 smithi047 ceph-mon[31351]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:24.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:23 smithi100 ceph-mon[36245]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:26.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:25 smithi102 ceph-mon[36748]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:26.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:25 smithi047 ceph-mon[31351]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:26.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:25 smithi100 ceph-mon[36245]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:27.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:27 smithi047 ceph-mon[31351]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:28.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:27 smithi102 ceph-mon[36748]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:28.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:27 smithi100 ceph-mon[36245]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:30.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:29 smithi047 ceph-mon[31351]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:30.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T11:14:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:29 smithi047 ceph-mon[31351]: Deploying daemon osd.0 on smithi047 2024-03-21T11:14:30.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:29 smithi102 ceph-mon[36748]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:30.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T11:14:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:29 smithi102 ceph-mon[36748]: Deploying daemon osd.0 on smithi047 2024-03-21T11:14:30.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:29 smithi100 ceph-mon[36245]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:30.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T11:14:30.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:30.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:29 smithi100 ceph-mon[36245]: Deploying daemon osd.0 on smithi047 2024-03-21T11:14:32.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:31 smithi102 ceph-mon[36748]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:32.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:31 smithi047 ceph-mon[31351]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:32.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:31 smithi100 ceph-mon[36245]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:34.234 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:33 smithi047 ceph-mon[31351]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:34.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:33 smithi102 ceph-mon[36748]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:34.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:33 smithi100 ceph-mon[36245]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:36.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:35 smithi102 ceph-mon[36748]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:35 smithi047 ceph-mon[31351]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:35 smithi100 ceph-mon[36245]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:38.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:37 smithi047 ceph-mon[31351]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:38.213 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:37 smithi100 ceph-mon[36245]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:38.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:37 smithi102 ceph-mon[36748]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:39.292 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:39 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:39.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:39 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:39.293 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:39 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:39.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:39 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:39.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:39 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:39.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:39 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:39.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:39 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:39.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:39 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:39.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:39 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:40.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:40 smithi047 ceph-mon[31351]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:40.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:40 smithi100 ceph-mon[36245]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:40.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:40 smithi102 ceph-mon[36748]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:42.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:42 smithi047 ceph-mon[31351]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:42.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:42.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:42.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:42.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:42.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:42.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:42 smithi047 ceph-mon[31351]: from='osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T11:14:42.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:42 smithi100 ceph-mon[36245]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:42.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:42.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:42.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:42.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:42.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:42.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:42 smithi100 ceph-mon[36245]: from='osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T11:14:42.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:42 smithi102 ceph-mon[36748]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:42 smithi102 ceph-mon[36748]: from='osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T11:14:42.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:42.802+0000 7f2f64c00700 1 -- 172.21.15.47:0/1321643331 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2f6802de10 con 0x7f2f4c06e1c0 2024-03-21T11:14:42.804 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 0 on host 'smithi047' 2024-03-21T11:14:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:42.806+0000 7f2f6ee00700 1 -- 172.21.15.47:0/1321643331 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2f4c06e1c0 msgr2=0x7f2f4c070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:42.806+0000 7f2f6ee00700 1 --2- 172.21.15.47:0/1321643331 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2f4c06e1c0 0x7f2f4c070680 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f68105470 tx=0x7f2f58009260 comp rx=0 tx=0).stop 2024-03-21T11:14:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:42.806+0000 7f2f6ee00700 1 -- 172.21.15.47:0/1321643331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f681049d0 msgr2=0x7f2f6819ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:42.806+0000 7f2f6ee00700 1 --2- 172.21.15.47:0/1321643331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f681049d0 0x7f2f6819ac40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2f54005b40 tx=0x7f2f54009580 comp rx=0 tx=0).stop 2024-03-21T11:14:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:42.806+0000 7f2f6ee00700 1 -- 172.21.15.47:0/1321643331 shutdown_connections 2024-03-21T11:14:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:42.806+0000 7f2f6ee00700 1 --2- 172.21.15.47:0/1321643331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f68105af0 0x7f2f681a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:42.806+0000 7f2f6ee00700 1 --2- 172.21.15.47:0/1321643331 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2f4c06e1c0 0x7f2f4c070680 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:42.806+0000 7f2f6ee00700 1 --2- 172.21.15.47:0/1321643331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f68104030 0x7f2f6819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:42.806+0000 7f2f6ee00700 1 --2- 172.21.15.47:0/1321643331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f681049d0 0x7f2f6819ac40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:42.806+0000 7f2f6ee00700 1 -- 172.21.15.47:0/1321643331 >> 172.21.15.47:0/1321643331 conn(0x7f2f680ff9b0 msgr2=0x7f2f681088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:14:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:42.807+0000 7f2f6ee00700 1 -- 172.21.15.47:0/1321643331 shutdown_connections 2024-03-21T11:14:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:14:42.807+0000 7f2f6ee00700 1 -- 172.21.15.47:0/1321643331 wait complete. 2024-03-21T11:14:43.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:43 smithi047 ceph-mon[31351]: from='osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T11:14:43.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:43 smithi047 ceph-mon[31351]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T11:14:43.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:43 smithi047 ceph-mon[31351]: from='osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:14:43.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:43 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:43.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:43 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:43.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:43 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:43.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:43 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:43.604 DEBUG:teuthology.orchestra.run.smithi047:osd.0> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.0.service 2024-03-21T11:14:43.608 INFO:tasks.cephadm:Deploying osd.1 on smithi100 with /dev/vg_nvme/lv_4... 2024-03-21T11:14:43.608 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T11:14:43.639 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:43 smithi100 ceph-mon[36245]: from='osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T11:14:43.639 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:43 smithi100 ceph-mon[36245]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T11:14:43.639 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:43 smithi100 ceph-mon[36245]: from='osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:14:43.639 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:43 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:43.639 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:43 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:43.639 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:43 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:43.639 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:43 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:43.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:43 smithi102 ceph-mon[36748]: from='osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T11:14:43.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:43 smithi102 ceph-mon[36748]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T11:14:43.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:43 smithi102 ceph-mon[36748]: from='osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:14:43.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:43 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:43.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:43 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:43.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:43 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:43.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:43 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:14:43.852 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T11:14:44.639 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:44 smithi100 ceph-mon[36245]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:44.640 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:44 smithi100 ceph-mon[36245]: from='osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T11:14:44.640 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:44 smithi100 ceph-mon[36245]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T11:14:44.640 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:44 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:44.640 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:44 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:44.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:44 smithi102 ceph-mon[36748]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:44.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:44 smithi102 ceph-mon[36748]: from='osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T11:14:44.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:44 smithi102 ceph-mon[36748]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T11:14:44.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:44 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:44.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:44 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:44.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:44 smithi047 ceph-mon[31351]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T11:14:44.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:44 smithi047 ceph-mon[31351]: from='osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T11:14:44.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:44 smithi047 ceph-mon[31351]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T11:14:44.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:44 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:44.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:44 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:45.444 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:45 smithi047 ceph-mon[31351]: purged_snaps scrub starts 2024-03-21T11:14:45.444 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:45 smithi047 ceph-mon[31351]: purged_snaps scrub ok 2024-03-21T11:14:45.444 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:45 smithi047 ceph-mon[31351]: osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460] boot 2024-03-21T11:14:45.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:45 smithi047 ceph-mon[31351]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T11:14:45.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:45 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:45.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:45 smithi100 ceph-mon[36245]: purged_snaps scrub starts 2024-03-21T11:14:45.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:45 smithi100 ceph-mon[36245]: purged_snaps scrub ok 2024-03-21T11:14:45.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:45 smithi100 ceph-mon[36245]: osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460] boot 2024-03-21T11:14:45.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:45 smithi100 ceph-mon[36245]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T11:14:45.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:45 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:45.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:45 smithi102 ceph-mon[36748]: purged_snaps scrub starts 2024-03-21T11:14:45.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:45 smithi102 ceph-mon[36748]: purged_snaps scrub ok 2024-03-21T11:14:45.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:45 smithi102 ceph-mon[36748]: osd.0 [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460] boot 2024-03-21T11:14:45.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:45 smithi102 ceph-mon[36748]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T11:14:45.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:45 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T11:14:45.763 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T11:14:45.763 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T11:14:45.763 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-21T11:14:45.763 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-21T11:14:45.763 INFO:teuthology.orchestra.run.smithi100.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011201 s, 936 MB/s 2024-03-21T11:14:45.763 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 2024-03-21T11:14:45.763 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-21T11:14:46.300 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_4 2024-03-21T11:14:46.573 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T11:14:46.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:46 smithi100 ceph-mon[36245]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:46.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:46 smithi100 ceph-mon[36245]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T11:14:46.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:46 smithi102 ceph-mon[36748]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:46.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:46 smithi102 ceph-mon[36748]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T11:14:46.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:46 smithi047 ceph-mon[31351]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:46.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:46 smithi047 ceph-mon[31351]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T11:14:48.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.066+0000 7fca8f370700 1 -- 172.21.15.100:0/2753069828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca881055b0 msgr2=0x7fca881079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:48.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.066+0000 7fca8f370700 1 --2- 172.21.15.100:0/2753069828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca881055b0 0x7fca881079a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009990 tx=0x7fca7c009ca0 comp rx=0 tx=0).stop 2024-03-21T11:14:48.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.066+0000 7fca8f370700 1 -- 172.21.15.100:0/2753069828 shutdown_connections 2024-03-21T11:14:48.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.066+0000 7fca8f370700 1 --2- 172.21.15.100:0/2753069828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca88107ee0 0x7fca8810a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:48.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.066+0000 7fca8f370700 1 --2- 172.21.15.100:0/2753069828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca88102c80 0x7fca88105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:48.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.066+0000 7fca8f370700 1 --2- 172.21.15.100:0/2753069828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca881055b0 0x7fca881079a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:48.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.066+0000 7fca8f370700 1 -- 172.21.15.100:0/2753069828 >> 172.21.15.100:0/2753069828 conn(0x7fca880fc690 msgr2=0x7fca880fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:14:48.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.066+0000 7fca8f370700 1 -- 172.21.15.100:0/2753069828 shutdown_connections 2024-03-21T11:14:48.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.066+0000 7fca8f370700 1 -- 172.21.15.100:0/2753069828 wait complete. 2024-03-21T11:14:48.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.066+0000 7fca8f370700 1 Processor -- start 2024-03-21T11:14:48.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8f370700 1 -- start start 2024-03-21T11:14:48.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8f370700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca88102c80 0x7fca88198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:48.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8f370700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca881055b0 0x7fca88198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:48.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8f370700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca88107ee0 0x7fca8819fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:48.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8f370700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8810dec0 con 0x7fca881055b0 2024-03-21T11:14:48.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8f370700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8810dd40 con 0x7fca88102c80 2024-03-21T11:14:48.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8f370700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8810e040 con 0x7fca88107ee0 2024-03-21T11:14:48.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8d10c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca88102c80 0x7fca88198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:48.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8d10c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca88102c80 0x7fca88198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44036/0 (socket says 172.21.15.100:44036) 2024-03-21T11:14:48.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8d10c700 1 -- 172.21.15.100:0/2886434582 learned_addr learned my addr 172.21.15.100:0/2886434582 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-21T11:14:48.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8d90d700 1 --2- 172.21.15.100:0/2886434582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca88107ee0 0x7fca8819fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:48.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8d10c700 1 -- 172.21.15.100:0/2886434582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca88107ee0 msgr2=0x7fca8819fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:48.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8d10c700 1 --2- 172.21.15.100:0/2886434582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca88107ee0 0x7fca8819fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:48.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8d10c700 1 -- 172.21.15.100:0/2886434582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca881055b0 msgr2=0x7fca88198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:14:48.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8d10c700 1 --2- 172.21.15.100:0/2886434582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca881055b0 0x7fca88198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:14:48.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8d10c700 1 -- 172.21.15.100:0/2886434582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca881a0160 con 0x7fca88102c80 2024-03-21T11:14:48.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8d10c700 1 --2- 172.21.15.100:0/2886434582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca88102c80 0x7fca88198500 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fca7800b530 tx=0x7fca7800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:14:48.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca867fc700 1 -- 172.21.15.100:0/2886434582 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78015020 con 0x7fca88102c80 2024-03-21T11:14:48.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca867fc700 1 -- 172.21.15.100:0/2886434582 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca78011d10 con 0x7fca88102c80 2024-03-21T11:14:48.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.067+0000 7fca8f370700 1 -- 172.21.15.100:0/2886434582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca881a0450 con 0x7fca88102c80 2024-03-21T11:14:48.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.068+0000 7fca867fc700 1 -- 172.21.15.100:0/2886434582 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7800f2e0 con 0x7fca88102c80 2024-03-21T11:14:48.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.068+0000 7fca8f370700 1 -- 172.21.15.100:0/2886434582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca881a0a10 con 0x7fca88102c80 2024-03-21T11:14:48.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.069+0000 7fca8f370700 1 -- 172.21.15.100:0/2886434582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca88068e50 con 0x7fca88102c80 2024-03-21T11:14:48.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.069+0000 7fca867fc700 1 -- 172.21.15.100:0/2886434582 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7fca78018020 con 0x7fca88102c80 2024-03-21T11:14:48.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.070+0000 7fca867fc700 1 --2- 172.21.15.100:0/2886434582 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fca700a2f20 0x7fca700a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:14:48.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.070+0000 7fca867fc700 1 -- 172.21.15.100:0/2886434582 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fca780cd1a0 con 0x7fca88102c80 2024-03-21T11:14:48.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.071+0000 7fca8c90b700 1 --2- 172.21.15.100:0/2886434582 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fca700a2f20 0x7fca700a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:14:48.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.071+0000 7fca8c90b700 1 --2- 172.21.15.100:0/2886434582 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fca700a2f20 0x7fca700a53e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fca7c005f50 tx=0x7fca7c005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:14:48.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.074+0000 7fca867fc700 1 -- 172.21.15.100:0/2886434582 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fca7801f020 con 0x7fca88102c80 2024-03-21T11:14:48.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:14:48.281+0000 7fca8f370700 1 -- 172.21.15.100:0/2886434582 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fca880625d0 con 0x7fca700a2f20 2024-03-21T11:14:48.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:48 smithi100 ceph-mon[36245]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:48.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:48.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:48.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:14:48.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:48.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:14:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:14:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:48.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:48 smithi102 ceph-mon[36748]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:14:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:14:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:14:48.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:48.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:48 smithi047 ceph-mon[31351]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:48.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:48.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:48.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:14:48.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:48.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:48.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:14:48.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:14:48.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:14:48.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:14:48.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:49.493 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:49 smithi100 ceph-mon[36245]: Detected new or changed devices on smithi047 2024-03-21T11:14:49.493 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:49 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi047 to 17095M 2024-03-21T11:14:49.494 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:49 smithi100 ceph-mon[36245]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:14:49.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:49 smithi102 ceph-mon[36748]: Detected new or changed devices on smithi047 2024-03-21T11:14:49.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:49 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi047 to 17095M 2024-03-21T11:14:49.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:49 smithi102 ceph-mon[36748]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:14:49.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:49 smithi047 ceph-mon[31351]: Detected new or changed devices on smithi047 2024-03-21T11:14:49.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:49 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi047 to 17095M 2024-03-21T11:14:49.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:49 smithi047 ceph-mon[31351]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:14:50.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:50 smithi100 ceph-mon[36245]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:50.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:50 smithi047 ceph-mon[31351]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:50.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:50 smithi102 ceph-mon[36748]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:51.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:51 smithi100 ceph-mon[36245]: from='client.? 172.21.15.100:0/4085770920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c44724d5-59e4-4520-83d9-7e5f4fd085a2"}]: dispatch 2024-03-21T11:14:51.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:51 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c44724d5-59e4-4520-83d9-7e5f4fd085a2"}]: dispatch 2024-03-21T11:14:51.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:51 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c44724d5-59e4-4520-83d9-7e5f4fd085a2"}]': finished 2024-03-21T11:14:51.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:51 smithi100 ceph-mon[36245]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T11:14:51.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:14:51.736 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:51 smithi047 ceph-mon[31351]: from='client.? 172.21.15.100:0/4085770920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c44724d5-59e4-4520-83d9-7e5f4fd085a2"}]: dispatch 2024-03-21T11:14:51.736 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:51 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c44724d5-59e4-4520-83d9-7e5f4fd085a2"}]: dispatch 2024-03-21T11:14:51.736 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:51 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c44724d5-59e4-4520-83d9-7e5f4fd085a2"}]': finished 2024-03-21T11:14:51.736 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:51 smithi047 ceph-mon[31351]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T11:14:51.736 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:14:51.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:51 smithi102 ceph-mon[36748]: from='client.? 172.21.15.100:0/4085770920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c44724d5-59e4-4520-83d9-7e5f4fd085a2"}]: dispatch 2024-03-21T11:14:51.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:51 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c44724d5-59e4-4520-83d9-7e5f4fd085a2"}]: dispatch 2024-03-21T11:14:51.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:51 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c44724d5-59e4-4520-83d9-7e5f4fd085a2"}]': finished 2024-03-21T11:14:51.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:51 smithi102 ceph-mon[36748]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T11:14:51.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:14:52.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:52 smithi102 ceph-mon[36748]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:52.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:52 smithi102 ceph-mon[36748]: from='client.? 172.21.15.100:0/1026941607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:14:52.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:52 smithi047 ceph-mon[31351]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:52.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:52 smithi047 ceph-mon[31351]: from='client.? 172.21.15.100:0/1026941607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:14:52.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:52 smithi100 ceph-mon[36245]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:52.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:52 smithi100 ceph-mon[36245]: from='client.? 172.21.15.100:0/1026941607' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:14:54.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:54 smithi102 ceph-mon[36748]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:54.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:54 smithi047 ceph-mon[31351]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:54.868 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:54 smithi100 ceph-mon[36245]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:56.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:56 smithi100 ceph-mon[36245]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:56.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:56 smithi102 ceph-mon[36748]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:56.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:56 smithi047 ceph-mon[31351]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:58.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:58 smithi102 ceph-mon[36748]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:58.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:58 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T11:14:58.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:58 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:58.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:14:58 smithi102 ceph-mon[36748]: Deploying daemon osd.1 on smithi100 2024-03-21T11:14:58.801 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:58 smithi100 ceph-mon[36245]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:58.801 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T11:14:58.801 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:58.801 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:14:58 smithi100 ceph-mon[36245]: Deploying daemon osd.1 on smithi100 2024-03-21T11:14:58.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:58 smithi047 ceph-mon[31351]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:14:58.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:58 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T11:14:58.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:58 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:14:58.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:14:58 smithi047 ceph-mon[31351]: Deploying daemon osd.1 on smithi100 2024-03-21T11:15:00.675 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:00 smithi100 ceph-mon[36245]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:00.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:00 smithi102 ceph-mon[36748]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:00.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:00 smithi047 ceph-mon[31351]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:02.510 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:02 smithi100 ceph-mon[36245]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:02.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:02 smithi102 ceph-mon[36748]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:02.801 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:02 smithi047 ceph-mon[31351]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:04.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:04 smithi047 ceph-mon[31351]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:04 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:04 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:15:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:04 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:04.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:04 smithi100 ceph-mon[36245]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:04.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:04 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:04.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:04 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:15:04.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:04 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:04.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:04 smithi102 ceph-mon[36748]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:04.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:04 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:04.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:04 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:15:04.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:04 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:06.620 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:06 smithi100 ceph-mon[36245]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:06.620 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:06.620 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:06.620 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:06.620 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:15:06.620 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:06.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:15:06.713+0000 7fca867fc700 1 -- 172.21.15.100:0/2886434582 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fca880625d0 con 0x7fca700a2f20 2024-03-21T11:15:06.713 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 1 on host 'smithi100' 2024-03-21T11:15:06.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:15:06.715+0000 7fca8f370700 1 -- 172.21.15.100:0/2886434582 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fca700a2f20 msgr2=0x7fca700a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:15:06.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:15:06.715+0000 7fca8f370700 1 --2- 172.21.15.100:0/2886434582 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fca700a2f20 0x7fca700a53e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fca7c005f50 tx=0x7fca7c005e80 comp rx=0 tx=0).stop 2024-03-21T11:15:06.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:15:06.715+0000 7fca8f370700 1 -- 172.21.15.100:0/2886434582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca88102c80 msgr2=0x7fca88198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:15:06.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:15:06.715+0000 7fca8f370700 1 --2- 172.21.15.100:0/2886434582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca88102c80 0x7fca88198500 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fca7800b530 tx=0x7fca7800b840 comp rx=0 tx=0).stop 2024-03-21T11:15:06.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:15:06.715+0000 7fca8f370700 1 -- 172.21.15.100:0/2886434582 shutdown_connections 2024-03-21T11:15:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:15:06.715+0000 7fca8f370700 1 --2- 172.21.15.100:0/2886434582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca88107ee0 0x7fca8819fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:15:06.715+0000 7fca8f370700 1 --2- 172.21.15.100:0/2886434582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca88102c80 0x7fca88198500 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:15:06.715+0000 7fca8f370700 1 --2- 172.21.15.100:0/2886434582 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fca700a2f20 0x7fca700a53e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:15:06.715+0000 7fca8f370700 1 --2- 172.21.15.100:0/2886434582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca881055b0 0x7fca88198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:15:06.715+0000 7fca8f370700 1 -- 172.21.15.100:0/2886434582 >> 172.21.15.100:0/2886434582 conn(0x7fca880fc690 msgr2=0x7fca88103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:15:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:15:06.715+0000 7fca8f370700 1 -- 172.21.15.100:0/2886434582 shutdown_connections 2024-03-21T11:15:06.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:15:06.715+0000 7fca8f370700 1 -- 172.21.15.100:0/2886434582 wait complete. 2024-03-21T11:15:06.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:06 smithi102 ceph-mon[36748]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:06.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:06.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:06.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:06.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:15:06.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:06.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:06 smithi047 ceph-mon[31351]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:06.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:06.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:06.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:06.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:15:06.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:07.339 DEBUG:teuthology.orchestra.run.smithi100:osd.1> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.1.service 2024-03-21T11:15:07.342 INFO:tasks.cephadm:Deploying osd.2 on smithi102 with /dev/vg_nvme/lv_4... 2024-03-21T11:15:07.342 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T11:15:07.583 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:15:07.926 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:07 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:07.927 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:07 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:07.927 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:07 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:15:07.927 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:07 smithi100 ceph-mon[36245]: from='osd.1 [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T11:15:07.927 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:07 smithi100 ceph-mon[36245]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T11:15:07.927 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:07 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:07.927 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:07 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:07.927 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:07 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:15:07.927 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:07 smithi102 ceph-mon[36748]: from='osd.1 [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T11:15:07.928 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:07 smithi102 ceph-mon[36748]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T11:15:08.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:07 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:07 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:07 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:15:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:07 smithi047 ceph-mon[31351]: from='osd.1 [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T11:15:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:07 smithi047 ceph-mon[31351]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T11:15:08.859 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:08 smithi100 ceph-mon[36245]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:08.859 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:08 smithi100 ceph-mon[36245]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T11:15:08.859 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:08 smithi100 ceph-mon[36245]: from='osd.1 [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:15:08.860 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:08 smithi100 ceph-mon[36245]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T11:15:08.860 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:08 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:08.860 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:08 smithi100 ceph-mon[36245]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:15:08.868 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:08 smithi102 ceph-mon[36748]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:08.868 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:08 smithi102 ceph-mon[36748]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T11:15:08.868 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:08 smithi102 ceph-mon[36748]: from='osd.1 [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:15:08.869 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:08 smithi102 ceph-mon[36748]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T11:15:08.869 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:08 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:08.869 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:08 smithi102 ceph-mon[36748]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:15:09.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:08 smithi047 ceph-mon[31351]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:08 smithi047 ceph-mon[31351]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T11:15:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:08 smithi047 ceph-mon[31351]: from='osd.1 [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:15:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:08 smithi047 ceph-mon[31351]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T11:15:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:08 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:08 smithi047 ceph-mon[31351]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:15:09.431 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T11:15:09.431 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T11:15:09.431 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-03-21T11:15:09.431 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-03-21T11:15:09.431 INFO:teuthology.orchestra.run.smithi102.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113928 s, 920 MB/s 2024-03-21T11:15:09.432 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-03-21T11:15:09.642 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:15:09 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-1[40856]: 2024-03-21T11:15:09.142+0000 7f1bb9d0a700 -1 osd.1 0 waiting for initial osdmap 2024-03-21T11:15:09.643 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:15:09 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-1[40856]: 2024-03-21T11:15:09.145+0000 7f1bb1af5700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T11:15:09.948 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:09 smithi100 ceph-mon[36245]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T11:15:09.949 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:09 smithi100 ceph-mon[36245]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T11:15:09.949 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:09 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:09.949 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:09 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:09.949 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:09 smithi100 ceph-mon[36245]: from='osd.1 ' entity='osd.1' 2024-03-21T11:15:09.970 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:09 smithi102 ceph-mon[36748]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T11:15:09.970 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:09 smithi102 ceph-mon[36748]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T11:15:09.970 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:09 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:09.970 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:09 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:09.970 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:09 smithi102 ceph-mon[36748]: from='osd.1 ' entity='osd.1' 2024-03-21T11:15:09.996 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi102:vg_nvme/lv_4 2024-03-21T11:15:10.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:09 smithi047 ceph-mon[31351]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T11:15:10.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:09 smithi047 ceph-mon[31351]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T11:15:10.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:09 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:10.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:09 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:10.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:09 smithi047 ceph-mon[31351]: from='osd.1 ' entity='osd.1' 2024-03-21T11:15:10.235 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:15:10.917 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:10 smithi102 ceph-mon[36748]: purged_snaps scrub starts 2024-03-21T11:15:10.917 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:10 smithi102 ceph-mon[36748]: purged_snaps scrub ok 2024-03-21T11:15:10.917 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:10 smithi102 ceph-mon[36748]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:10.917 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:10.918 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:10 smithi102 ceph-mon[36748]: osd.1 [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436] boot 2024-03-21T11:15:10.918 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:10 smithi102 ceph-mon[36748]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T11:15:10.918 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:10.918 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:10.918 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:10.918 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:15:10.918 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:10.918 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:10.918 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:15:10.918 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:10 smithi047 ceph-mon[31351]: purged_snaps scrub starts 2024-03-21T11:15:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:10 smithi047 ceph-mon[31351]: purged_snaps scrub ok 2024-03-21T11:15:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:10 smithi047 ceph-mon[31351]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:10 smithi047 ceph-mon[31351]: osd.1 [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436] boot 2024-03-21T11:15:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:10 smithi047 ceph-mon[31351]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T11:15:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:11.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:11.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:15:11.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:11.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:11.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:15:11.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:11.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:10 smithi100 ceph-mon[36245]: purged_snaps scrub starts 2024-03-21T11:15:11.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:10 smithi100 ceph-mon[36245]: purged_snaps scrub ok 2024-03-21T11:15:11.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:10 smithi100 ceph-mon[36245]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T11:15:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:10 smithi100 ceph-mon[36245]: osd.1 [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436] boot 2024-03-21T11:15:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:10 smithi100 ceph-mon[36245]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T11:15:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T11:15:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:15:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:15:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:11.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.658+0000 7fdc902ac700 1 -- 172.21.15.102:0/2721462327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc88102360 msgr2=0x7fdc881027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:15:11.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.658+0000 7fdc902ac700 1 --2- 172.21.15.102:0/2721462327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc88102360 0x7fdc881027c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009990 tx=0x7fdc78009ca0 comp rx=0 tx=0).stop 2024-03-21T11:15:11.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 -- 172.21.15.102:0/2721462327 shutdown_connections 2024-03-21T11:15:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 --2- 172.21.15.102:0/2721462327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88102d00 0x7fdc881095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 --2- 172.21.15.102:0/2721462327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc88101160 0x7fdc88101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 --2- 172.21.15.102:0/2721462327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc88102360 0x7fdc881027c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 -- 172.21.15.102:0/2721462327 >> 172.21.15.102:0/2721462327 conn(0x7fdc880fc720 msgr2=0x7fdc880feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:15:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 -- 172.21.15.102:0/2721462327 shutdown_connections 2024-03-21T11:15:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 -- 172.21.15.102:0/2721462327 wait complete. 2024-03-21T11:15:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 Processor -- start 2024-03-21T11:15:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 -- start start 2024-03-21T11:15:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc88101160 0x7fdc88198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:15:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc88102360 0x7fdc88198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:15:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88102d00 0x7fdc8819fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:15:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8810cdd0 con 0x7fdc88102360 2024-03-21T11:15:11.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8810cc50 con 0x7fdc88102d00 2024-03-21T11:15:11.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc902ac700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8810cf50 con 0x7fdc88101160 2024-03-21T11:15:11.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc8e048700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc88101160 0x7fdc88198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:15:11.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.659+0000 7fdc8e048700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc88101160 0x7fdc88198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52500/0 (socket says 172.21.15.102:52500) 2024-03-21T11:15:11.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.660+0000 7fdc8e048700 1 -- 172.21.15.102:0/2411240451 learned_addr learned my addr 172.21.15.102:0/2411240451 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-21T11:15:11.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.660+0000 7fdc8e849700 1 --2- 172.21.15.102:0/2411240451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88102d00 0x7fdc8819fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:15:11.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.660+0000 7fdc8e048700 1 -- 172.21.15.102:0/2411240451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88102d00 msgr2=0x7fdc8819fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:15:11.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.660+0000 7fdc8e048700 1 --2- 172.21.15.102:0/2411240451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88102d00 0x7fdc8819fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:11.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.660+0000 7fdc8e048700 1 -- 172.21.15.102:0/2411240451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc88102360 msgr2=0x7fdc88198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:15:11.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.660+0000 7fdc8e048700 1 --2- 172.21.15.102:0/2411240451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc88102360 0x7fdc88198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:11.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.660+0000 7fdc8e048700 1 -- 172.21.15.102:0/2411240451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc881a01a0 con 0x7fdc88101160 2024-03-21T11:15:11.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.660+0000 7fdc8e048700 1 --2- 172.21.15.102:0/2411240451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc88101160 0x7fdc88198590 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400b530 tx=0x7fdc8400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:15:11.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.660+0000 7fdc7f7fe700 1 -- 172.21.15.102:0/2411240451 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84015020 con 0x7fdc88101160 2024-03-21T11:15:11.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.660+0000 7fdc7f7fe700 1 -- 172.21.15.102:0/2411240451 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc84011d10 con 0x7fdc88101160 2024-03-21T11:15:11.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.660+0000 7fdc902ac700 1 -- 172.21.15.102:0/2411240451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc881a0490 con 0x7fdc88101160 2024-03-21T11:15:11.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.660+0000 7fdc7f7fe700 1 -- 172.21.15.102:0/2411240451 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8400f2e0 con 0x7fdc88101160 2024-03-21T11:15:11.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.660+0000 7fdc902ac700 1 -- 172.21.15.102:0/2411240451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc881a0a50 con 0x7fdc88101160 2024-03-21T11:15:11.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.661+0000 7fdc902ac700 1 -- 172.21.15.102:0/2411240451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc88068e50 con 0x7fdc88101160 2024-03-21T11:15:11.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.662+0000 7fdc7f7fe700 1 -- 172.21.15.102:0/2411240451 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7fdc84018020 con 0x7fdc88101160 2024-03-21T11:15:11.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.663+0000 7fdc7f7fe700 1 --2- 172.21.15.102:0/2411240451 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc740a2f20 0x7fdc740a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:15:11.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.663+0000 7fdc7f7fe700 1 -- 172.21.15.102:0/2411240451 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fdc840cd300 con 0x7fdc88101160 2024-03-21T11:15:11.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.663+0000 7fdc8d847700 1 --2- 172.21.15.102:0/2411240451 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc740a2f20 0x7fdc740a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:15:11.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.664+0000 7fdc8d847700 1 --2- 172.21.15.102:0/2411240451 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc740a2f20 0x7fdc740a53e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc780098e0 tx=0x7fdc78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:15:11.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.667+0000 7fdc7f7fe700 1 -- 172.21.15.102:0/2411240451 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdc8401f020 con 0x7fdc88101160 2024-03-21T11:15:11.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:11.873+0000 7fdc902ac700 1 -- 172.21.15.102:0/2411240451 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc880fe900 con 0x7fdc740a2f20 2024-03-21T11:15:11.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:11 smithi102 ceph-mon[36748]: Detected new or changed devices on smithi100 2024-03-21T11:15:11.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:11 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi100 to 17080M 2024-03-21T11:15:12.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:11 smithi047 ceph-mon[31351]: Detected new or changed devices on smithi100 2024-03-21T11:15:12.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:11 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi100 to 17080M 2024-03-21T11:15:12.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:11 smithi100 ceph-mon[36245]: Detected new or changed devices on smithi100 2024-03-21T11:15:12.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:11 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi100 to 17080M 2024-03-21T11:15:12.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:12 smithi102 ceph-mon[36748]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:12.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:12 smithi102 ceph-mon[36748]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T11:15:12.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:15:12.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:15:12.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:13.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:12 smithi047 ceph-mon[31351]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:13.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:12 smithi047 ceph-mon[31351]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T11:15:13.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:15:13.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:15:13.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:13.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:12 smithi100 ceph-mon[36245]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:13.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:12 smithi100 ceph-mon[36245]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T11:15:13.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:15:13.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:15:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:13.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:13 smithi102 ceph-mon[36748]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:15:14.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:13 smithi047 ceph-mon[31351]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:15:14.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:13 smithi100 ceph-mon[36245]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:15:14.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:14 smithi102 ceph-mon[36748]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:14.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:14 smithi102 ceph-mon[36748]: from='client.? 172.21.15.102:0/2668685799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "712cd8e5-7760-4534-8f80-71e7acd5fae9"}]: dispatch 2024-03-21T11:15:14.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:14 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "712cd8e5-7760-4534-8f80-71e7acd5fae9"}]: dispatch 2024-03-21T11:15:14.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:14 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "712cd8e5-7760-4534-8f80-71e7acd5fae9"}]': finished 2024-03-21T11:15:14.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:14 smithi102 ceph-mon[36748]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T11:15:14.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:14 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:15.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:14 smithi047 ceph-mon[31351]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:15.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:14 smithi047 ceph-mon[31351]: from='client.? 172.21.15.102:0/2668685799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "712cd8e5-7760-4534-8f80-71e7acd5fae9"}]: dispatch 2024-03-21T11:15:15.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:14 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "712cd8e5-7760-4534-8f80-71e7acd5fae9"}]: dispatch 2024-03-21T11:15:15.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:14 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "712cd8e5-7760-4534-8f80-71e7acd5fae9"}]': finished 2024-03-21T11:15:15.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:14 smithi047 ceph-mon[31351]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T11:15:15.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:14 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:14 smithi100 ceph-mon[36245]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:14 smithi100 ceph-mon[36245]: from='client.? 172.21.15.102:0/2668685799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "712cd8e5-7760-4534-8f80-71e7acd5fae9"}]: dispatch 2024-03-21T11:15:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:14 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "712cd8e5-7760-4534-8f80-71e7acd5fae9"}]: dispatch 2024-03-21T11:15:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:14 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "712cd8e5-7760-4534-8f80-71e7acd5fae9"}]': finished 2024-03-21T11:15:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:14 smithi100 ceph-mon[36245]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T11:15:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:14 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:15.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:15 smithi102 ceph-mon[36748]: from='client.? 172.21.15.102:0/2949051015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:15:16.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:15 smithi047 ceph-mon[31351]: from='client.? 172.21.15.102:0/2949051015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:15:16.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:15 smithi100 ceph-mon[36245]: from='client.? 172.21.15.102:0/2949051015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:15:16.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:16 smithi102 ceph-mon[36748]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:17.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:16 smithi047 ceph-mon[31351]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:17.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:16 smithi100 ceph-mon[36245]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:18.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:18 smithi102 ceph-mon[36748]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:19.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:18 smithi047 ceph-mon[31351]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:19.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:18 smithi100 ceph-mon[36245]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:20.829 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:20 smithi102 ceph-mon[36748]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:21.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:20 smithi047 ceph-mon[31351]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:21.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:20 smithi100 ceph-mon[36245]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:22.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:21 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T11:15:22.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:21 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:22.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:21 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T11:15:22.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:21 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:22.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:21 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T11:15:22.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:21 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:22.811 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:22 smithi102 ceph-mon[36748]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:22.812 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:22 smithi102 ceph-mon[36748]: Deploying daemon osd.2 on smithi102 2024-03-21T11:15:22.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:22 smithi047 ceph-mon[31351]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:22.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:22 smithi047 ceph-mon[31351]: Deploying daemon osd.2 on smithi102 2024-03-21T11:15:23.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:22 smithi100 ceph-mon[36245]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:23.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:22 smithi100 ceph-mon[36245]: Deploying daemon osd.2 on smithi102 2024-03-21T11:15:24.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:24 smithi102 ceph-mon[36748]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:25.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:24 smithi047 ceph-mon[31351]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:25.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:24 smithi100 ceph-mon[36245]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:26.909 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:26 smithi102 ceph-mon[36748]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:26.909 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:26.909 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:26.909 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:15:27.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:26 smithi047 ceph-mon[31351]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:15:27.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:26 smithi100 ceph-mon[36245]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:27.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:27.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:27.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:15:28.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:28 smithi102 ceph-mon[36748]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:29.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:28 smithi047 ceph-mon[31351]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:29.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:28 smithi100 ceph-mon[36245]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:29.400 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 2 on host 'smithi102' 2024-03-21T11:15:29.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:29.400+0000 7fdc7f7fe700 1 -- 172.21.15.102:0/2411240451 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdc880fe900 con 0x7fdc740a2f20 2024-03-21T11:15:29.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:29.402+0000 7fdc902ac700 1 -- 172.21.15.102:0/2411240451 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc740a2f20 msgr2=0x7fdc740a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:15:29.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:29.402+0000 7fdc902ac700 1 --2- 172.21.15.102:0/2411240451 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc740a2f20 0x7fdc740a53e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc780098e0 tx=0x7fdc78005fb0 comp rx=0 tx=0).stop 2024-03-21T11:15:29.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:29.402+0000 7fdc902ac700 1 -- 172.21.15.102:0/2411240451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc88101160 msgr2=0x7fdc88198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:15:29.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:29.402+0000 7fdc902ac700 1 --2- 172.21.15.102:0/2411240451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc88101160 0x7fdc88198590 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400b530 tx=0x7fdc8400b840 comp rx=0 tx=0).stop 2024-03-21T11:15:29.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:29.402+0000 7fdc902ac700 1 -- 172.21.15.102:0/2411240451 shutdown_connections 2024-03-21T11:15:29.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:29.402+0000 7fdc902ac700 1 --2- 172.21.15.102:0/2411240451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88102d00 0x7fdc8819fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:29.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:29.402+0000 7fdc902ac700 1 --2- 172.21.15.102:0/2411240451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc88101160 0x7fdc88198590 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:29.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:29.402+0000 7fdc902ac700 1 --2- 172.21.15.102:0/2411240451 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc740a2f20 0x7fdc740a53e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:29.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:29.402+0000 7fdc902ac700 1 --2- 172.21.15.102:0/2411240451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc88102360 0x7fdc88198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:29.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:29.402+0000 7fdc902ac700 1 -- 172.21.15.102:0/2411240451 >> 172.21.15.102:0/2411240451 conn(0x7fdc880fc720 msgr2=0x7fdc880fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:15:29.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:29.402+0000 7fdc902ac700 1 -- 172.21.15.102:0/2411240451 shutdown_connections 2024-03-21T11:15:29.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:15:29.402+0000 7fdc902ac700 1 -- 172.21.15.102:0/2411240451 wait complete. 2024-03-21T11:15:29.990 DEBUG:teuthology.orchestra.run.smithi102:osd.2> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.2.service 2024-03-21T11:15:29.993 INFO:tasks.cephadm:Deploying osd.3 on smithi047 with /dev/vg_nvme/lv_3... 2024-03-21T11:15:29.993 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T11:15:30.240 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:15:30 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2[41926]: 2024-03-21T11:15:30.090+0000 7f96669ca7c0 -1 osd.2 0 log_to_monitors true 2024-03-21T11:15:30.472 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:15:30.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:30 smithi047 ceph-mon[31351]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:30.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:30 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:30.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:30 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:30.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:30 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:30.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:30 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:30.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:30 smithi047 ceph-mon[31351]: from='osd.2 [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T11:15:30.625 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:30 smithi047 ceph-mon[31351]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T11:15:30.635 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:30 smithi102 ceph-mon[36748]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:30.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:30 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:30.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:30 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:30.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:30 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:30.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:30 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:30.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:30 smithi102 ceph-mon[36748]: from='osd.2 [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T11:15:30.636 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:30 smithi102 ceph-mon[36748]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T11:15:30.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:30 smithi100 ceph-mon[36245]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:30.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:30 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:30.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:30 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:30.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:30 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:30.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:30 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:30.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:30 smithi100 ceph-mon[36245]: from='osd.2 [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T11:15:30.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:30 smithi100 ceph-mon[36245]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T11:15:31.895 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:31 smithi047 ceph-mon[31351]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T11:15:31.895 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:31 smithi047 ceph-mon[31351]: from='osd.2 [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:15:31.895 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:31 smithi047 ceph-mon[31351]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T11:15:31.895 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:31 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:31.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:31 smithi047 ceph-mon[31351]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:15:32.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:31 smithi100 ceph-mon[36245]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T11:15:32.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:31 smithi100 ceph-mon[36245]: from='osd.2 [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:15:32.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:31 smithi100 ceph-mon[36245]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T11:15:32.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:31 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:32.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:31 smithi100 ceph-mon[36245]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:15:32.214 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:31 smithi102 ceph-mon[36748]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T11:15:32.215 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:31 smithi102 ceph-mon[36748]: from='osd.2 [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:15:32.215 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:31 smithi102 ceph-mon[36748]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T11:15:32.215 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:31 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:32.215 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:31 smithi102 ceph-mon[36748]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:15:32.491 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:15:32 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2[41926]: 2024-03-21T11:15:32.232+0000 7f965c02f700 -1 osd.2 0 waiting for initial osdmap 2024-03-21T11:15:32.491 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:15:32 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2[41926]: 2024-03-21T11:15:32.238+0000 7f9653e1a700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T11:15:32.828 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:32 smithi102 ceph-mon[36748]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:32.829 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:32 smithi102 ceph-mon[36748]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-03-21T11:15:32.829 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:32 smithi102 ceph-mon[36748]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T11:15:32.829 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:32 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:32.829 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:32 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:32.829 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:32 smithi102 ceph-mon[36748]: from='osd.2 ' entity='osd.2' 2024-03-21T11:15:33.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:32 smithi047 ceph-mon[31351]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:33.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:32 smithi047 ceph-mon[31351]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-03-21T11:15:33.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:32 smithi047 ceph-mon[31351]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T11:15:33.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:32 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:33.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:32 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:33.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:32 smithi047 ceph-mon[31351]: from='osd.2 ' entity='osd.2' 2024-03-21T11:15:33.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:32 smithi100 ceph-mon[36245]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:33.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:32 smithi100 ceph-mon[36245]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-03-21T11:15:33.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:32 smithi100 ceph-mon[36245]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T11:15:33.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:32 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:33.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:32 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:33.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:32 smithi100 ceph-mon[36245]: from='osd.2 ' entity='osd.2' 2024-03-21T11:15:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T11:15:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T11:15:33.539 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-03-21T11:15:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-03-21T11:15:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.038197 s, 275 MB/s 2024-03-21T11:15:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-03-21T11:15:34.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: purged_snaps scrub starts 2024-03-21T11:15:34.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: purged_snaps scrub ok 2024-03-21T11:15:34.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:34.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:15:34.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:34.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:15:34.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:15:34.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:34.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:15:34.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: osd.2 [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261] boot 2024-03-21T11:15:34.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T11:15:34.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:34.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: purged_snaps scrub starts 2024-03-21T11:15:34.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: purged_snaps scrub ok 2024-03-21T11:15:34.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:15:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:15:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:15:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:15:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: osd.2 [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261] boot 2024-03-21T11:15:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T11:15:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:34.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: purged_snaps scrub starts 2024-03-21T11:15:34.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: purged_snaps scrub ok 2024-03-21T11:15:34.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:34.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:15:34.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:34.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:15:34.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:15:34.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:34.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:15:34.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:34.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: osd.2 [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261] boot 2024-03-21T11:15:34.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T11:15:34.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T11:15:34.573 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi047:vg_nvme/lv_3 2024-03-21T11:15:35.048 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:15:35.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:34 smithi047 ceph-mon[31351]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:35.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:34 smithi047 ceph-mon[31351]: Detected new or changed devices on smithi102 2024-03-21T11:15:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:34 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi102 to 17080M 2024-03-21T11:15:35.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:34 smithi100 ceph-mon[36245]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:35.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:34 smithi100 ceph-mon[36245]: Detected new or changed devices on smithi102 2024-03-21T11:15:35.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:34 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi102 to 17080M 2024-03-21T11:15:35.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:34 smithi102 ceph-mon[36748]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T11:15:35.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:34 smithi102 ceph-mon[36748]: Detected new or changed devices on smithi102 2024-03-21T11:15:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:34 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi102 to 17080M 2024-03-21T11:15:36.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:35 smithi047 ceph-mon[31351]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T11:15:36.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T11:15:36.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:35 smithi100 ceph-mon[36245]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T11:15:36.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:35 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T11:15:36.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:35 smithi102 ceph-mon[36748]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T11:15:36.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:35 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T11:15:37.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:36 smithi047 ceph-mon[31351]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:37.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T11:15:37.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:36 smithi047 ceph-mon[31351]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T11:15:37.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T11:15:37.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:36 smithi100 ceph-mon[36245]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:37.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T11:15:37.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:36 smithi100 ceph-mon[36245]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T11:15:37.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T11:15:37.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:36 smithi102 ceph-mon[36748]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:37.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T11:15:37.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:36 smithi102 ceph-mon[36748]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T11:15:37.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T11:15:37.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.423+0000 7fa324400700 1 -- 172.21.15.47:0/1608341497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa31c104410 msgr2=0x7fa31c104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:15:37.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.423+0000 7fa324400700 1 --2- 172.21.15.47:0/1608341497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa31c104410 0x7fa31c104830 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa310009990 tx=0x7fa310009ca0 comp rx=0 tx=0).stop 2024-03-21T11:15:37.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.423+0000 7fa324400700 1 -- 172.21.15.47:0/1608341497 shutdown_connections 2024-03-21T11:15:37.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.423+0000 7fa324400700 1 --2- 172.21.15.47:0/1608341497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c105fb0 0x7fa31c10c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:37.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.423+0000 7fa324400700 1 --2- 172.21.15.47:0/1608341497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa31c104410 0x7fa31c104830 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:37.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.423+0000 7fa324400700 1 --2- 172.21.15.47:0/1608341497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa31c105610 0x7fa31c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:37.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.423+0000 7fa324400700 1 -- 172.21.15.47:0/1608341497 >> 172.21.15.47:0/1608341497 conn(0x7fa31c0ff9b0 msgr2=0x7fa31c101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:15:37.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.424+0000 7fa324400700 1 -- 172.21.15.47:0/1608341497 shutdown_connections 2024-03-21T11:15:37.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.424+0000 7fa324400700 1 -- 172.21.15.47:0/1608341497 wait complete. 2024-03-21T11:15:37.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.425+0000 7fa324400700 1 Processor -- start 2024-03-21T11:15:37.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.425+0000 7fa324400700 1 -- start start 2024-03-21T11:15:37.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.425+0000 7fa324400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa31c104410 0x7fa31c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:15:37.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.425+0000 7fa324400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c105610 0x7fa31c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:15:37.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.425+0000 7fa324400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa31c105fb0 0x7fa31c1a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:15:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.425+0000 7fa324400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c110040 con 0x7fa31c104410 2024-03-21T11:15:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.425+0000 7fa324400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c10fec0 con 0x7fa31c105610 2024-03-21T11:15:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.425+0000 7fa324400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c1101c0 con 0x7fa31c105fb0 2024-03-21T11:15:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.426+0000 7fa321a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa31c104410 0x7fa31c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:15:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.426+0000 7fa321a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa31c104410 0x7fa31c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40558/0 (socket says 172.21.15.47:40558) 2024-03-21T11:15:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.426+0000 7fa321000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c105610 0x7fa31c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:15:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.426+0000 7fa321a00700 1 -- 172.21.15.47:0/2950874158 learned_addr learned my addr 172.21.15.47:0/2950874158 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:15:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.426+0000 7fa322400700 1 --2- 172.21.15.47:0/2950874158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa31c105fb0 0x7fa31c1a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:15:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.426+0000 7fa321a00700 1 -- 172.21.15.47:0/2950874158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa31c105fb0 msgr2=0x7fa31c1a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:15:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.426+0000 7fa321a00700 1 --2- 172.21.15.47:0/2950874158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa31c105fb0 0x7fa31c1a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.426+0000 7fa321a00700 1 -- 172.21.15.47:0/2950874158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c105610 msgr2=0x7fa31c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:15:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.426+0000 7fa321a00700 1 --2- 172.21.15.47:0/2950874158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c105610 0x7fa31c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:15:37.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.426+0000 7fa321a00700 1 -- 172.21.15.47:0/2950874158 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c1a22d0 con 0x7fa31c104410 2024-03-21T11:15:37.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.426+0000 7fa321000700 1 --2- 172.21.15.47:0/2950874158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c105610 0x7fa31c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:15:37.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.426+0000 7fa321a00700 1 --2- 172.21.15.47:0/2950874158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa31c104410 0x7fa31c19a670 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa310000c00 tx=0x7fa31000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:15:37.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.427+0000 7fa31a000700 1 -- 172.21.15.47:0/2950874158 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310026070 con 0x7fa31c104410 2024-03-21T11:15:37.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.427+0000 7fa31a000700 1 -- 172.21.15.47:0/2950874158 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa31000fda0 con 0x7fa31c104410 2024-03-21T11:15:37.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.427+0000 7fa324400700 1 -- 172.21.15.47:0/2950874158 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c1a2560 con 0x7fa31c104410 2024-03-21T11:15:37.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.427+0000 7fa31a000700 1 -- 172.21.15.47:0/2950874158 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310017650 con 0x7fa31c104410 2024-03-21T11:15:37.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.427+0000 7fa324400700 1 -- 172.21.15.47:0/2950874158 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c1a2a10 con 0x7fa31c104410 2024-03-21T11:15:37.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.428+0000 7fa324400700 1 -- 172.21.15.47:0/2950874158 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31c068e50 con 0x7fa31c104410 2024-03-21T11:15:37.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.431+0000 7fa31a000700 1 -- 172.21.15.47:0/2950874158 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7fa310029070 con 0x7fa31c104410 2024-03-21T11:15:37.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.432+0000 7fa31a000700 1 --2- 172.21.15.47:0/2950874158 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa3040a2fa0 0x7fa3040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:15:37.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.433+0000 7fa31a000700 1 -- 172.21.15.47:0/2950874158 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3237+0+0 (secure 0 0 0) 0x7fa3100cf5a0 con 0x7fa31c104410 2024-03-21T11:15:37.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.433+0000 7fa321000700 1 --2- 172.21.15.47:0/2950874158 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa3040a2fa0 0x7fa3040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:15:37.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.434+0000 7fa321000700 1 --2- 172.21.15.47:0/2950874158 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa3040a2fa0 0x7fa3040a5460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa308011490 tx=0x7fa308009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:15:37.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.435+0000 7fa31a000700 1 -- 172.21.15.47:0/2950874158 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa310013070 con 0x7fa31c104410 2024-03-21T11:15:37.632 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:15:37 smithi047 sudo[48217]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T11:15:37.633 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:15:37 smithi047 sudo[48217]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T11:15:37.633 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:15:37 smithi047 sudo[48217]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T11:15:37.633 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:15:37 smithi047 sudo[48217]: pam_unix(sudo:session): session closed for user root 2024-03-21T11:15:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:37.677+0000 7fa324400700 1 -- 172.21.15.47:0/2950874158 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa31c0625d0 con 0x7fa3040a2fa0 2024-03-21T11:15:38.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T11:15:38.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:37 smithi047 ceph-mon[31351]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T11:15:38.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:15:38.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:15:38.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:38.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T11:15:38.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:37 smithi100 ceph-mon[36245]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T11:15:38.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:15:38.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:15:38.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:38.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T11:15:38.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:37 smithi102 ceph-mon[36748]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T11:15:38.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:15:38.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:15:38.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:38.560 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:15:38 smithi047 sudo[48287]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T11:15:38.560 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:15:38 smithi047 sudo[48287]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T11:15:38.560 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:15:38 smithi047 sudo[48287]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T11:15:38.561 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:15:38 smithi047 sudo[48287]: pam_unix(sudo:session): session closed for user root 2024-03-21T11:15:38.642 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:15:38 smithi100 sudo[44578]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T11:15:38.642 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:15:38 smithi100 sudo[44578]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T11:15:38.642 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:15:38 smithi100 sudo[44578]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T11:15:38.642 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:15:38 smithi100 sudo[44578]: pam_unix(sudo:session): session closed for user root 2024-03-21T11:15:39.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:38 smithi100 ceph-mon[36245]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:39.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:38 smithi100 ceph-mon[36245]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:15:39.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:38 smithi100 ceph-mon[36245]: osdmap e22: 3 total, 3 up, 3 in 2024-03-21T11:15:39.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:38 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:39.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:38 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:39.143 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:15:38 smithi100 sudo[44584]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T11:15:39.143 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:15:38 smithi100 sudo[44584]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T11:15:39.143 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:15:38 smithi100 sudo[44584]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T11:15:39.143 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:15:38 smithi100 sudo[44584]: pam_unix(sudo:session): session closed for user root 2024-03-21T11:15:39.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:38 smithi102 ceph-mon[36748]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:39.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:38 smithi102 ceph-mon[36748]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:15:39.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:38 smithi102 ceph-mon[36748]: osdmap e22: 3 total, 3 up, 3 in 2024-03-21T11:15:39.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:38 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:38 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:39.241 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:15:39 smithi102 sudo[45582]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T11:15:39.241 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:15:39 smithi102 sudo[45582]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T11:15:39.241 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:15:39 smithi102 sudo[45582]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T11:15:39.241 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:15:39 smithi102 sudo[45582]: pam_unix(sudo:session): session closed for user root 2024-03-21T11:15:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:38 smithi047 ceph-mon[31351]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:38 smithi047 ceph-mon[31351]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:15:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:38 smithi047 ceph-mon[31351]: osdmap e22: 3 total, 3 up, 3 in 2024-03-21T11:15:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:39.741 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:15:39 smithi102 sudo[45585]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T11:15:39.741 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:15:39 smithi102 sudo[45585]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T11:15:39.741 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:15:39 smithi102 sudo[45585]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T11:15:39.741 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:15:39 smithi102 sudo[45585]: pam_unix(sudo:session): session closed for user root 2024-03-21T11:15:40.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:39 smithi047 sudo[48368]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T11:15:40.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:39 smithi047 sudo[48368]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T11:15:40.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:39 smithi047 sudo[48368]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T11:15:40.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:39 smithi047 sudo[48368]: pam_unix(sudo:session): session closed for user root 2024-03-21T11:15:40.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:15:40.537+0000 7fa31a000700 1 -- 172.21.15.47:0/2950874158 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa310009d10 con 0x7fa31c104410 2024-03-21T11:15:40.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:40 smithi047 sudo[48394]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T11:15:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:40 smithi047 sudo[48394]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T11:15:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:40 smithi047 sudo[48394]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T11:15:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:40 smithi047 sudo[48394]: pam_unix(sudo:session): session closed for user root 2024-03-21T11:15:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:40 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:40 smithi047 ceph-mon[31351]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T11:15:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:40 smithi047 ceph-mon[31351]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T11:15:40.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:40 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:40.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:40 smithi100 ceph-mon[36245]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T11:15:40.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:40 smithi100 ceph-mon[36245]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T11:15:40.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:40 smithi102 sudo[45588]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T11:15:40.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:40 smithi102 sudo[45588]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T11:15:40.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:40 smithi102 sudo[45588]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T11:15:40.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:40 smithi102 sudo[45588]: pam_unix(sudo:session): session closed for user root 2024-03-21T11:15:41.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 sudo[45591]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T11:15:41.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 sudo[45591]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T11:15:41.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 sudo[45591]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T11:15:41.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 sudo[45591]: pam_unix(sudo:session): session closed for user root 2024-03-21T11:15:41.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:15:41.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T11:15:41.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T11:15:41.509 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 sudo[44588]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T11:15:41.509 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 sudo[44588]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T11:15:41.509 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 sudo[44588]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T11:15:41.510 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 sudo[44588]: pam_unix(sudo:session): session closed for user root 2024-03-21T11:15:41.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:41.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:15:41.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:15:41.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:15:41.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T11:15:41.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:15:41.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-21T11:15:41.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:15:41.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:15:41.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T11:15:41.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T11:15:41.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:15:41.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:15:41.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:15:41.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:41 smithi047 ceph-mon[31351]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T11:15:41.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 sudo[44591]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T11:15:41.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 sudo[44591]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T11:15:41.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 sudo[44591]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T11:15:41.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 sudo[44591]: pam_unix(sudo:session): session closed for user root 2024-03-21T11:15:41.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:41.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:15:41.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:15:41.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:15:41.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T11:15:41.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:15:41.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-21T11:15:41.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:15:41.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:15:41.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T11:15:41.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T11:15:41.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:15:41.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:15:41.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:15:41.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:41 smithi100 ceph-mon[36245]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T11:15:41.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:41.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:15:41.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:15:41.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:15:41.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T11:15:41.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:15:41.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-21T11:15:41.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:15:41.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:15:41.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T11:15:41.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T11:15:41.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T11:15:41.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T11:15:41.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T11:15:41.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:41 smithi102 ceph-mon[36748]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T11:15:42.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:42 smithi047 ceph-mon[31351]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:42.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:42 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1036208236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "182c6d29-67dd-4cc1-b79f-7d106f2cc0ff"}]: dispatch 2024-03-21T11:15:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:42 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "182c6d29-67dd-4cc1-b79f-7d106f2cc0ff"}]: dispatch 2024-03-21T11:15:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:42 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "182c6d29-67dd-4cc1-b79f-7d106f2cc0ff"}]': finished 2024-03-21T11:15:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:42 smithi047 ceph-mon[31351]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T11:15:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:15:42.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:42 smithi100 ceph-mon[36245]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:42.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:42 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1036208236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "182c6d29-67dd-4cc1-b79f-7d106f2cc0ff"}]: dispatch 2024-03-21T11:15:42.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:42 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "182c6d29-67dd-4cc1-b79f-7d106f2cc0ff"}]: dispatch 2024-03-21T11:15:42.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:42 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "182c6d29-67dd-4cc1-b79f-7d106f2cc0ff"}]': finished 2024-03-21T11:15:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:42 smithi100 ceph-mon[36245]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T11:15:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:15:42.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:42 smithi102 ceph-mon[36748]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:42 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1036208236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "182c6d29-67dd-4cc1-b79f-7d106f2cc0ff"}]: dispatch 2024-03-21T11:15:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:42 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "182c6d29-67dd-4cc1-b79f-7d106f2cc0ff"}]: dispatch 2024-03-21T11:15:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:42 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "182c6d29-67dd-4cc1-b79f-7d106f2cc0ff"}]': finished 2024-03-21T11:15:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:42 smithi102 ceph-mon[36748]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T11:15:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:15:43.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:43 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3938503855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:15:43.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:43 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3938503855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:15:43.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:43 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3938503855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:15:44.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:44 smithi047 ceph-mon[31351]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:44.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:44 smithi100 ceph-mon[36245]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:44.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:44 smithi102 ceph-mon[36748]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:46.800 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:46 smithi047 ceph-mon[31351]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:46.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:46 smithi100 ceph-mon[36245]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:46.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:46 smithi102 ceph-mon[36748]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:15:48.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:48 smithi047 ceph-mon[31351]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-03-21T11:15:48.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:48 smithi100 ceph-mon[36245]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-03-21T11:15:48.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:48 smithi102 ceph-mon[36748]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-03-21T11:15:50.684 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:50 smithi047 ceph-mon[31351]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-21T11:15:50.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:50 smithi100 ceph-mon[36245]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-21T11:15:50.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:50 smithi102 ceph-mon[36748]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-21T11:15:51.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T11:15:51.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:51.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T11:15:51.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:52.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T11:15:52.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:15:52.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:52 smithi047 ceph-mon[31351]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-21T11:15:52.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:52 smithi047 ceph-mon[31351]: Deploying daemon osd.3 on smithi047 2024-03-21T11:15:52.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:52 smithi100 ceph-mon[36245]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-21T11:15:52.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:52 smithi100 ceph-mon[36245]: Deploying daemon osd.3 on smithi047 2024-03-21T11:15:52.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:52 smithi102 ceph-mon[36748]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-21T11:15:52.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:52 smithi102 ceph-mon[36748]: Deploying daemon osd.3 on smithi047 2024-03-21T11:15:54.757 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:54 smithi047 ceph-mon[31351]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 828 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-21T11:15:54.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:54 smithi100 ceph-mon[36245]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 828 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-21T11:15:54.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:54 smithi102 ceph-mon[36748]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 828 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-21T11:15:56.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:56 smithi047 ceph-mon[31351]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-21T11:15:56.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:56 smithi100 ceph-mon[36245]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-21T11:15:56.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:56 smithi102 ceph-mon[36748]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-21T11:15:58.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:15:58 smithi047 ceph-mon[31351]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-21T11:15:58.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:15:58 smithi100 ceph-mon[36245]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-21T11:15:58.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:15:58 smithi102 ceph-mon[36748]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-21T11:16:00.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:00 smithi102 ceph-mon[36748]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:16:00.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:00 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:00.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:00 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:00.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:00 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:01.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:00 smithi047 ceph-mon[31351]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:16:01.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:00 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:01.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:00 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:01.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:00 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:00 smithi100 ceph-mon[36245]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:16:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:00 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:00 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:00 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:02.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:02 smithi047 ceph-mon[31351]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:16:02.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:02 smithi102 ceph-mon[36748]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:16:03.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:02 smithi100 ceph-mon[36245]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:16:04.379 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 3 on host 'smithi047' 2024-03-21T11:16:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:16:04.378+0000 7fa31a000700 1 -- 172.21.15.47:0/2950874158 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa31c0625d0 con 0x7fa3040a2fa0 2024-03-21T11:16:04.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:16:04.380+0000 7fa324400700 1 -- 172.21.15.47:0/2950874158 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa3040a2fa0 msgr2=0x7fa3040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:16:04.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:16:04.380+0000 7fa324400700 1 --2- 172.21.15.47:0/2950874158 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa3040a2fa0 0x7fa3040a5460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa308011490 tx=0x7fa308009260 comp rx=0 tx=0).stop 2024-03-21T11:16:04.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:16:04.381+0000 7fa324400700 1 -- 172.21.15.47:0/2950874158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa31c104410 msgr2=0x7fa31c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:16:04.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:16:04.381+0000 7fa324400700 1 --2- 172.21.15.47:0/2950874158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa31c104410 0x7fa31c19a670 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa310000c00 tx=0x7fa31000f690 comp rx=0 tx=0).stop 2024-03-21T11:16:04.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:16:04.381+0000 7fa324400700 1 -- 172.21.15.47:0/2950874158 shutdown_connections 2024-03-21T11:16:04.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:16:04.381+0000 7fa324400700 1 --2- 172.21.15.47:0/2950874158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa31c105fb0 0x7fa31c1a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:04.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:16:04.381+0000 7fa324400700 1 --2- 172.21.15.47:0/2950874158 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa3040a2fa0 0x7fa3040a5460 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:04.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:16:04.381+0000 7fa324400700 1 --2- 172.21.15.47:0/2950874158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa31c104410 0x7fa31c19a670 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:04.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:16:04.381+0000 7fa324400700 1 --2- 172.21.15.47:0/2950874158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa31c105610 0x7fa31c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:04.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:16:04.381+0000 7fa324400700 1 -- 172.21.15.47:0/2950874158 >> 172.21.15.47:0/2950874158 conn(0x7fa31c0ff9b0 msgr2=0x7fa31c1088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:16:04.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:16:04.381+0000 7fa324400700 1 -- 172.21.15.47:0/2950874158 shutdown_connections 2024-03-21T11:16:04.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:16:04.381+0000 7fa324400700 1 -- 172.21.15.47:0/2950874158 wait complete. 2024-03-21T11:16:04.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:04 smithi047 ceph-mon[31351]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:16:04.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:04 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:04.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:04 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:04.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:04 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:04.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:04 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:04.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:04 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:04.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:04 smithi047 ceph-mon[31351]: from='osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T11:16:04.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:04 smithi100 ceph-mon[36245]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:16:04.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:04 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:04.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:04 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:04.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:04 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:04.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:04 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:04.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:04 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:04.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:04 smithi100 ceph-mon[36245]: from='osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T11:16:04.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:04 smithi102 ceph-mon[36748]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:16:04.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:04 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:04.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:04 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:04.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:04 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:04.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:04 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:04.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:04 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:04.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:04 smithi102 ceph-mon[36748]: from='osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T11:16:05.105 DEBUG:teuthology.orchestra.run.smithi047:osd.3> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.3.service 2024-03-21T11:16:05.111 INFO:tasks.cephadm:Deploying osd.4 on smithi100 with /dev/vg_nvme/lv_3... 2024-03-21T11:16:05.111 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T11:16:05.356 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T11:16:05.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:05 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:05.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:05 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:05.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:05 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:05.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:05 smithi100 ceph-mon[36245]: from='osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T11:16:05.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:05 smithi100 ceph-mon[36245]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T11:16:05.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:05 smithi100 ceph-mon[36245]: from='osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:16:05.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:05 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:16:05.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:05 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:05.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:05 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:05.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:05 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:05.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:05 smithi047 ceph-mon[31351]: from='osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T11:16:05.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:05 smithi047 ceph-mon[31351]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T11:16:05.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:05 smithi047 ceph-mon[31351]: from='osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:16:05.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:05 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:16:05.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:05 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:05.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:05 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:05.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:05 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:05.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:05 smithi102 ceph-mon[36748]: from='osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T11:16:05.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:05 smithi102 ceph-mon[36748]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T11:16:05.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:05 smithi102 ceph-mon[36748]: from='osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:16:05.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:05 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:16:06.065 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:16:05 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3[50461]: 2024-03-21T11:16:05.756+0000 7f511d200700 -1 osd.3 0 waiting for initial osdmap 2024-03-21T11:16:06.065 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:16:05 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3[50461]: 2024-03-21T11:16:05.767+0000 7f5118000700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T11:16:06.775 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:06 smithi100 ceph-mon[36245]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:16:06.775 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:06 smithi100 ceph-mon[36245]: from='osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T11:16:06.775 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:06 smithi100 ceph-mon[36245]: osdmap e25: 4 total, 3 up, 4 in 2024-03-21T11:16:06.776 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:16:06.776 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:16:06.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:06 smithi047 ceph-mon[31351]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:16:06.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:06 smithi047 ceph-mon[31351]: from='osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T11:16:06.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:06 smithi047 ceph-mon[31351]: osdmap e25: 4 total, 3 up, 4 in 2024-03-21T11:16:06.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:16:06.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:16:06.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:06 smithi102 ceph-mon[36748]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T11:16:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:06 smithi102 ceph-mon[36748]: from='osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T11:16:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:06 smithi102 ceph-mon[36748]: osdmap e25: 4 total, 3 up, 4 in 2024-03-21T11:16:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:16:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:16:07.386 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T11:16:07.387 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T11:16:07.387 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-21T11:16:07.387 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-21T11:16:07.387 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112943 s, 928 MB/s 2024-03-21T11:16:07.387 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-21T11:16:08.005 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:07 smithi100 ceph-mon[36245]: purged_snaps scrub starts 2024-03-21T11:16:08.005 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:07 smithi100 ceph-mon[36245]: purged_snaps scrub ok 2024-03-21T11:16:08.005 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:07 smithi100 ceph-mon[36245]: osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877] boot 2024-03-21T11:16:08.005 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:07 smithi100 ceph-mon[36245]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T11:16:08.005 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:07 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:16:08.032 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_3 2024-03-21T11:16:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:07 smithi047 ceph-mon[31351]: purged_snaps scrub starts 2024-03-21T11:16:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:07 smithi047 ceph-mon[31351]: purged_snaps scrub ok 2024-03-21T11:16:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:07 smithi047 ceph-mon[31351]: osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877] boot 2024-03-21T11:16:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:07 smithi047 ceph-mon[31351]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T11:16:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:07 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:16:08.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:07 smithi102 ceph-mon[36748]: purged_snaps scrub starts 2024-03-21T11:16:08.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:07 smithi102 ceph-mon[36748]: purged_snaps scrub ok 2024-03-21T11:16:08.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:07 smithi102 ceph-mon[36748]: osd.3 [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877] boot 2024-03-21T11:16:08.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:07 smithi102 ceph-mon[36748]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T11:16:08.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:07 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T11:16:08.267 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T11:16:08.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:08 smithi100 ceph-mon[36245]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:08.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:08 smithi100 ceph-mon[36245]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T11:16:09.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:08 smithi047 ceph-mon[31351]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:08 smithi047 ceph-mon[31351]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T11:16:09.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:08 smithi102 ceph-mon[36748]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:09.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:08 smithi102 ceph-mon[36748]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T11:16:09.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.894+0000 7f05e52fb700 1 -- 172.21.15.100:0/3326294325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05e0100880 msgr2=0x7f05e0100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:16:09.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.894+0000 7f05e52fb700 1 --2- 172.21.15.100:0/3326294325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05e0100880 0x7f05e0100ca0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f05d80099c0 tx=0x7f05d8009cd0 comp rx=0 tx=0).stop 2024-03-21T11:16:09.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.894+0000 7f05e52fb700 1 -- 172.21.15.100:0/3326294325 shutdown_connections 2024-03-21T11:16:09.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.894+0000 7f05e52fb700 1 --2- 172.21.15.100:0/3326294325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e0102420 0x7f05e0108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:09.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.894+0000 7f05e52fb700 1 --2- 172.21.15.100:0/3326294325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05e0100880 0x7f05e0100ca0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:09.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.894+0000 7f05e52fb700 1 --2- 172.21.15.100:0/3326294325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05e0101a80 0x7f05e0101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:09.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.894+0000 7f05e52fb700 1 -- 172.21.15.100:0/3326294325 >> 172.21.15.100:0/3326294325 conn(0x7f05e00fbe20 msgr2=0x7f05e00fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:16:09.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.894+0000 7f05e52fb700 1 -- 172.21.15.100:0/3326294325 shutdown_connections 2024-03-21T11:16:09.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.894+0000 7f05e52fb700 1 -- 172.21.15.100:0/3326294325 wait complete. 2024-03-21T11:16:09.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05e52fb700 1 Processor -- start 2024-03-21T11:16:09.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05e52fb700 1 -- start start 2024-03-21T11:16:09.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05e52fb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05e0100880 0x7f05e010a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:16:09.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05e52fb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e0101a80 0x7f05e019d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:16:09.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05e52fb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05e0102420 0x7f05e019f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:16:09.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05e52fb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e010ce10 con 0x7f05e0101a80 2024-03-21T11:16:09.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05e52fb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e010cc90 con 0x7f05e0100880 2024-03-21T11:16:09.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05e52fb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e010cf90 con 0x7f05e0102420 2024-03-21T11:16:09.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05deffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05e0100880 0x7f05e010a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:16:09.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05deffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05e0100880 0x7f05e010a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43758/0 (socket says 172.21.15.100:43758) 2024-03-21T11:16:09.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05deffd700 1 -- 172.21.15.100:0/144848657 learned_addr learned my addr 172.21.15.100:0/144848657 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-21T11:16:09.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05df7fe700 1 --2- 172.21.15.100:0/144848657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05e0102420 0x7f05e019f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:16:09.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05deffd700 1 -- 172.21.15.100:0/144848657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05e0102420 msgr2=0x7f05e019f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:16:09.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05deffd700 1 --2- 172.21.15.100:0/144848657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05e0102420 0x7f05e019f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:09.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05deffd700 1 -- 172.21.15.100:0/144848657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e0101a80 msgr2=0x7f05e019d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:16:09.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05deffd700 1 --2- 172.21.15.100:0/144848657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e0101a80 0x7f05e019d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:09.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05deffd700 1 -- 172.21.15.100:0/144848657 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e0197080 con 0x7f05e0100880 2024-03-21T11:16:09.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.895+0000 7f05df7fe700 1 --2- 172.21.15.100:0/144848657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05e0102420 0x7f05e019f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:16:09.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.896+0000 7f05deffd700 1 --2- 172.21.15.100:0/144848657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05e0100880 0x7f05e010a260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f05d80098e0 tx=0x7f05d80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:16:09.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.896+0000 7f05c7fff700 1 -- 172.21.15.100:0/144848657 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8027070 con 0x7f05e0100880 2024-03-21T11:16:09.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.896+0000 7f05e52fb700 1 -- 172.21.15.100:0/144848657 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e0197310 con 0x7f05e0100880 2024-03-21T11:16:09.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.896+0000 7f05c7fff700 1 -- 172.21.15.100:0/144848657 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05d8022470 con 0x7f05e0100880 2024-03-21T11:16:09.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.896+0000 7f05e52fb700 1 -- 172.21.15.100:0/144848657 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e0197870 con 0x7f05e0100880 2024-03-21T11:16:09.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.896+0000 7f05c7fff700 1 -- 172.21.15.100:0/144848657 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8031060 con 0x7f05e0100880 2024-03-21T11:16:09.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.897+0000 7f05e52fb700 1 -- 172.21.15.100:0/144848657 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e0068e50 con 0x7f05e0100880 2024-03-21T11:16:09.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.898+0000 7f05c7fff700 1 -- 172.21.15.100:0/144848657 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f05d802a070 con 0x7f05e0100880 2024-03-21T11:16:09.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:09 smithi100 ceph-mon[36245]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T11:16:09.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.898+0000 7f05c7fff700 1 --2- 172.21.15.100:0/144848657 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05c80a2fb0 0x7f05c80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:16:09.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.898+0000 7f05c7fff700 1 -- 172.21.15.100:0/144848657 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3669+0+0 (secure 0 0 0) 0x7f05d80cf540 con 0x7f05e0100880 2024-03-21T11:16:09.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.899+0000 7f05de7fc700 1 --2- 172.21.15.100:0/144848657 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05c80a2fb0 0x7f05c80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:16:09.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.900+0000 7f05de7fc700 1 --2- 172.21.15.100:0/144848657 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05c80a2fb0 0x7f05c80a5470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f05e010a7c0 tx=0x7f05d4006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:16:09.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:09.904+0000 7f05c7fff700 1 -- 172.21.15.100:0/144848657 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f05d8013070 con 0x7f05e0100880 2024-03-21T11:16:10.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:09 smithi047 ceph-mon[31351]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T11:16:10.074 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:09 smithi102 ceph-mon[36748]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T11:16:10.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:10.110+0000 7f05e52fb700 1 -- 172.21.15.100:0/144848657 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f05e0107830 con 0x7f05c80a2fb0 2024-03-21T11:16:11.009 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:10 smithi100 ceph-mon[36245]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:11.009 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:10 smithi100 ceph-mon[36245]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T11:16:11.009 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:10 smithi100 ceph-mon[36245]: Detected new or changed devices on smithi047 2024-03-21T11:16:11.009 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:11.010 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:11.010 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:11.010 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:11.010 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:11.010 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:11.010 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:11.010 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:11.010 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:16:11.010 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:16:11.010 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:10 smithi047 ceph-mon[31351]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:10 smithi047 ceph-mon[31351]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T11:16:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:10 smithi047 ceph-mon[31351]: Detected new or changed devices on smithi047 2024-03-21T11:16:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:11.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:11.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:11.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:11.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:11.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:16:11.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:16:11.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:10 smithi102 ceph-mon[36748]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:10 smithi102 ceph-mon[36748]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T11:16:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:10 smithi102 ceph-mon[36748]: Detected new or changed devices on smithi047 2024-03-21T11:16:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:11.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:16:11.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:16:11.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:12.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:11 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi047 to 8547M 2024-03-21T11:16:12.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:11 smithi047 ceph-mon[31351]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:16:12.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:11 smithi047 ceph-mon[31351]: osdmap e30: 4 total, 4 up, 4 in 2024-03-21T11:16:12.075 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:11 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi047 to 8547M 2024-03-21T11:16:12.075 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:11 smithi100 ceph-mon[36245]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:16:12.075 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:11 smithi100 ceph-mon[36245]: osdmap e30: 4 total, 4 up, 4 in 2024-03-21T11:16:12.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:11 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi047 to 8547M 2024-03-21T11:16:12.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:11 smithi102 ceph-mon[36748]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:16:12.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:11 smithi102 ceph-mon[36748]: osdmap e30: 4 total, 4 up, 4 in 2024-03-21T11:16:13.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:12 smithi047 ceph-mon[31351]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T11:16:13.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:12 smithi100 ceph-mon[36245]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T11:16:13.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:12 smithi102 ceph-mon[36748]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T11:16:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:13 smithi047 ceph-mon[31351]: from='client.? 172.21.15.100:0/1214551807' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65901f21-1ccb-45d6-90a9-8f40d2dfaf28"}]: dispatch 2024-03-21T11:16:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:13 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65901f21-1ccb-45d6-90a9-8f40d2dfaf28"}]: dispatch 2024-03-21T11:16:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:13 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65901f21-1ccb-45d6-90a9-8f40d2dfaf28"}]': finished 2024-03-21T11:16:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:13 smithi047 ceph-mon[31351]: osdmap e31: 5 total, 4 up, 5 in 2024-03-21T11:16:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:13 smithi047 ceph-mon[31351]: from='client.? 172.21.15.100:0/3857267921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:16:14.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:13 smithi100 ceph-mon[36245]: from='client.? 172.21.15.100:0/1214551807' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65901f21-1ccb-45d6-90a9-8f40d2dfaf28"}]: dispatch 2024-03-21T11:16:14.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:13 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65901f21-1ccb-45d6-90a9-8f40d2dfaf28"}]: dispatch 2024-03-21T11:16:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:13 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65901f21-1ccb-45d6-90a9-8f40d2dfaf28"}]': finished 2024-03-21T11:16:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:13 smithi100 ceph-mon[36245]: osdmap e31: 5 total, 4 up, 5 in 2024-03-21T11:16:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:13 smithi100 ceph-mon[36245]: from='client.? 172.21.15.100:0/3857267921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:16:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:13 smithi102 ceph-mon[36748]: from='client.? 172.21.15.100:0/1214551807' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65901f21-1ccb-45d6-90a9-8f40d2dfaf28"}]: dispatch 2024-03-21T11:16:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:13 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65901f21-1ccb-45d6-90a9-8f40d2dfaf28"}]: dispatch 2024-03-21T11:16:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:13 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65901f21-1ccb-45d6-90a9-8f40d2dfaf28"}]': finished 2024-03-21T11:16:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:13 smithi102 ceph-mon[36748]: osdmap e31: 5 total, 4 up, 5 in 2024-03-21T11:16:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:13 smithi102 ceph-mon[36748]: from='client.? 172.21.15.100:0/3857267921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:16:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:14 smithi047 ceph-mon[31351]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-03-21T11:16:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:14 smithi100 ceph-mon[36245]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-03-21T11:16:15.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:14 smithi102 ceph-mon[36748]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-03-21T11:16:17.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:16 smithi047 ceph-mon[31351]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T11:16:17.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:16 smithi100 ceph-mon[36245]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T11:16:17.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:16 smithi102 ceph-mon[36748]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T11:16:19.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:18 smithi047 ceph-mon[31351]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-21T11:16:19.140 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:18 smithi100 ceph-mon[36245]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-21T11:16:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:18 smithi102 ceph-mon[36748]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-21T11:16:21.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:20 smithi047 ceph-mon[31351]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:21.067 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:20 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T11:16:21.067 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:20 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:21.091 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:20 smithi100 ceph-mon[36245]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:21.091 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:20 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T11:16:21.091 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:20 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:20 smithi102 ceph-mon[36748]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:20 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T11:16:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:20 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:22.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:21 smithi047 ceph-mon[31351]: Deploying daemon osd.4 on smithi100 2024-03-21T11:16:22.087 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:21 smithi100 ceph-mon[36245]: Deploying daemon osd.4 on smithi100 2024-03-21T11:16:22.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:21 smithi102 ceph-mon[36748]: Deploying daemon osd.4 on smithi100 2024-03-21T11:16:23.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:22 smithi100 ceph-mon[36245]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:23.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:22 smithi102 ceph-mon[36748]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:22 smithi047 ceph-mon[31351]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:25.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:24 smithi100 ceph-mon[36245]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:25.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:24 smithi102 ceph-mon[36748]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:25.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:24 smithi047 ceph-mon[31351]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:26.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:26 smithi100 ceph-mon[36245]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:26.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:26.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:26.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:26.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:26 smithi102 ceph-mon[36748]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:26 smithi047 ceph-mon[31351]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:28.842 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:28 smithi100 ceph-mon[36245]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:28.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:28 smithi102 ceph-mon[36748]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:29.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:28 smithi047 ceph-mon[31351]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:29.656 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 4 on host 'smithi100' 2024-03-21T11:16:29.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:29.655+0000 7f05c7fff700 1 -- 172.21.15.100:0/144848657 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f05e0107830 con 0x7f05c80a2fb0 2024-03-21T11:16:29.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:29.657+0000 7f05e52fb700 1 -- 172.21.15.100:0/144848657 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05c80a2fb0 msgr2=0x7f05c80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:16:29.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:29.657+0000 7f05e52fb700 1 --2- 172.21.15.100:0/144848657 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05c80a2fb0 0x7f05c80a5470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f05e010a7c0 tx=0x7f05d4006d20 comp rx=0 tx=0).stop 2024-03-21T11:16:29.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:29.657+0000 7f05e52fb700 1 -- 172.21.15.100:0/144848657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05e0100880 msgr2=0x7f05e010a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:16:29.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:29.657+0000 7f05e52fb700 1 --2- 172.21.15.100:0/144848657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05e0100880 0x7f05e010a260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f05d80098e0 tx=0x7f05d80047b0 comp rx=0 tx=0).stop 2024-03-21T11:16:29.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:29.657+0000 7f05e52fb700 1 -- 172.21.15.100:0/144848657 shutdown_connections 2024-03-21T11:16:29.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:29.657+0000 7f05e52fb700 1 --2- 172.21.15.100:0/144848657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05e0102420 0x7f05e019f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:29.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:29.657+0000 7f05e52fb700 1 --2- 172.21.15.100:0/144848657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05e0100880 0x7f05e010a260 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:29.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:29.657+0000 7f05e52fb700 1 --2- 172.21.15.100:0/144848657 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05c80a2fb0 0x7f05c80a5470 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:29.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:29.657+0000 7f05e52fb700 1 --2- 172.21.15.100:0/144848657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05e0101a80 0x7f05e019d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:29.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:29.657+0000 7f05e52fb700 1 -- 172.21.15.100:0/144848657 >> 172.21.15.100:0/144848657 conn(0x7f05e00fbe20 msgr2=0x7f05e0104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:16:29.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:29.657+0000 7f05e52fb700 1 -- 172.21.15.100:0/144848657 shutdown_connections 2024-03-21T11:16:29.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:16:29.657+0000 7f05e52fb700 1 -- 172.21.15.100:0/144848657 wait complete. 2024-03-21T11:16:29.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:29.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:29.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:29.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:29.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:29.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:29 smithi100 ceph-mon[36245]: from='osd.4 [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T11:16:29.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:29 smithi100 ceph-mon[36245]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T11:16:29.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:29.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:29.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:29.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:29.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:29.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:29.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:29.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:29.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:29 smithi102 ceph-mon[36748]: from='osd.4 [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T11:16:29.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:29 smithi102 ceph-mon[36748]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T11:16:29.999 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:30.000 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:30.000 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:29 smithi047 ceph-mon[31351]: from='osd.4 [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T11:16:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:29 smithi047 ceph-mon[31351]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T11:16:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:30.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:30.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:30.322 DEBUG:teuthology.orchestra.run.smithi100:osd.4> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.4.service 2024-03-21T11:16:30.324 INFO:tasks.cephadm:Deploying osd.5 on smithi102 with /dev/vg_nvme/lv_3... 2024-03-21T11:16:30.324 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T11:16:30.558 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:16:30.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:30 smithi100 ceph-mon[36245]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:30.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:30 smithi100 ceph-mon[36245]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T11:16:30.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:30 smithi100 ceph-mon[36245]: from='osd.4 [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:16:30.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:30 smithi100 ceph-mon[36245]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T11:16:30.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:30 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:30.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:30 smithi100 ceph-mon[36245]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:16:30.893 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:16:30 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4[47405]: 2024-03-21T11:16:30.784+0000 7f9c57ccc700 -1 osd.4 0 waiting for initial osdmap 2024-03-21T11:16:30.893 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:16:30 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4[47405]: 2024-03-21T11:16:30.791+0000 7f9c53abf700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T11:16:30.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:30 smithi102 ceph-mon[36748]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:30.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:30 smithi102 ceph-mon[36748]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T11:16:30.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:30 smithi102 ceph-mon[36748]: from='osd.4 [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:16:30.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:30 smithi102 ceph-mon[36748]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T11:16:30.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:30 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:30.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:30 smithi102 ceph-mon[36748]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:16:31.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:30 smithi047 ceph-mon[31351]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:31.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:30 smithi047 ceph-mon[31351]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T11:16:31.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:30 smithi047 ceph-mon[31351]: from='osd.4 [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:16:31.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:30 smithi047 ceph-mon[31351]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T11:16:31.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:30 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:31.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:30 smithi047 ceph-mon[31351]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:16:31.882 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:31 smithi100 ceph-mon[36245]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T11:16:31.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:31 smithi100 ceph-mon[36245]: osdmap e33: 5 total, 4 up, 5 in 2024-03-21T11:16:31.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:31 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:31.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:31 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:31.952 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:31 smithi102 ceph-mon[36748]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T11:16:31.952 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:31 smithi102 ceph-mon[36748]: osdmap e33: 5 total, 4 up, 5 in 2024-03-21T11:16:31.952 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:31 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:31.952 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:31 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:32.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:31 smithi047 ceph-mon[31351]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T11:16:32.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:31 smithi047 ceph-mon[31351]: osdmap e33: 5 total, 4 up, 5 in 2024-03-21T11:16:32.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:31 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:32.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:31 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:32.472 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T11:16:32.473 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T11:16:32.473 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-03-21T11:16:32.473 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-03-21T11:16:32.473 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011263 s, 931 MB/s 2024-03-21T11:16:32.473 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-03-21T11:16:33.032 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:32 smithi102 ceph-mon[36748]: purged_snaps scrub starts 2024-03-21T11:16:33.032 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:32 smithi102 ceph-mon[36748]: purged_snaps scrub ok 2024-03-21T11:16:33.032 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:32 smithi102 ceph-mon[36748]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:33.033 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:32 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:33.033 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:32 smithi102 ceph-mon[36748]: osd.4 [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971] boot 2024-03-21T11:16:33.033 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:32 smithi102 ceph-mon[36748]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T11:16:33.033 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:32 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:33.058 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi102:vg_nvme/lv_3 2024-03-21T11:16:33.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:32 smithi047 ceph-mon[31351]: purged_snaps scrub starts 2024-03-21T11:16:33.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:32 smithi047 ceph-mon[31351]: purged_snaps scrub ok 2024-03-21T11:16:33.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:32 smithi047 ceph-mon[31351]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:33.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:32 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:33.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:32 smithi047 ceph-mon[31351]: osd.4 [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971] boot 2024-03-21T11:16:33.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:32 smithi047 ceph-mon[31351]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T11:16:33.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:32 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:33.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:32 smithi100 ceph-mon[36245]: purged_snaps scrub starts 2024-03-21T11:16:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:32 smithi100 ceph-mon[36245]: purged_snaps scrub ok 2024-03-21T11:16:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:32 smithi100 ceph-mon[36245]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-21T11:16:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:32 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:32 smithi100 ceph-mon[36245]: osd.4 [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971] boot 2024-03-21T11:16:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:32 smithi100 ceph-mon[36245]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T11:16:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:32 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T11:16:33.293 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:16:33.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:33 smithi102 ceph-mon[36748]: osdmap e35: 5 total, 5 up, 5 in 2024-03-21T11:16:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:34.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:33 smithi100 ceph-mon[36245]: osdmap e35: 5 total, 5 up, 5 in 2024-03-21T11:16:34.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:34.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:34.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:33 smithi047 ceph-mon[31351]: osdmap e35: 5 total, 5 up, 5 in 2024-03-21T11:16:34.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:34.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:34.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:34.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:34.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:34.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:34.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:34.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:34.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.935+0000 7fe6a953e700 1 -- 172.21.15.102:0/2098826388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6a4107ea0 msgr2=0x7fe6a410a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:16:34.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.935+0000 7fe6a953e700 1 --2- 172.21.15.102:0/2098826388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6a4107ea0 0x7fe6a410a310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009990 tx=0x7fe69c009ca0 comp rx=0 tx=0).stop 2024-03-21T11:16:34.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.935+0000 7fe6a953e700 1 -- 172.21.15.102:0/2098826388 shutdown_connections 2024-03-21T11:16:34.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.935+0000 7fe6a953e700 1 --2- 172.21.15.102:0/2098826388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6a4107ea0 0x7fe6a410a310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:34.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.935+0000 7fe6a953e700 1 --2- 172.21.15.102:0/2098826388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6a4102c40 0x7fe6a4105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:34.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.935+0000 7fe6a953e700 1 --2- 172.21.15.102:0/2098826388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6a4105570 0x7fe6a4107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:34.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.935+0000 7fe6a953e700 1 -- 172.21.15.102:0/2098826388 >> 172.21.15.102:0/2098826388 conn(0x7fe6a40fc650 msgr2=0x7fe6a40fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:16:34.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.935+0000 7fe6a953e700 1 -- 172.21.15.102:0/2098826388 shutdown_connections 2024-03-21T11:16:34.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.935+0000 7fe6a953e700 1 -- 172.21.15.102:0/2098826388 wait complete. 2024-03-21T11:16:34.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.936+0000 7fe6a953e700 1 Processor -- start 2024-03-21T11:16:34.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.936+0000 7fe6a953e700 1 -- start start 2024-03-21T11:16:34.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.936+0000 7fe6a953e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6a4102c40 0x7fe6a41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:16:34.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.936+0000 7fe6a953e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6a4105570 0x7fe6a41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:16:34.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.936+0000 7fe6a953e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6a4107ea0 0x7fe6a419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:16:34.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.936+0000 7fe6a953e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a410de60 con 0x7fe6a4105570 2024-03-21T11:16:34.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.936+0000 7fe6a953e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a410dce0 con 0x7fe6a4102c40 2024-03-21T11:16:34.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.936+0000 7fe6a953e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a410dfe0 con 0x7fe6a4107ea0 2024-03-21T11:16:34.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.936+0000 7fe6a37fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6a4107ea0 0x7fe6a419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:16:34.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.936+0000 7fe6a37fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6a4107ea0 0x7fe6a419fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56452/0 (socket says 172.21.15.102:56452) 2024-03-21T11:16:34.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.936+0000 7fe6a37fe700 1 -- 172.21.15.102:0/1866940366 learned_addr learned my addr 172.21.15.102:0/1866940366 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-21T11:16:34.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.936+0000 7fe6a2ffd700 1 --2- 172.21.15.102:0/1866940366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6a4102c40 0x7fe6a41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:16:34.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.937+0000 7fe6a37fe700 1 -- 172.21.15.102:0/1866940366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6a4102c40 msgr2=0x7fe6a41984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:16:34.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.937+0000 7fe6a37fe700 1 --2- 172.21.15.102:0/1866940366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6a4102c40 0x7fe6a41984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:34.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.937+0000 7fe6a37fe700 1 -- 172.21.15.102:0/1866940366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6a4105570 msgr2=0x7fe6a41989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:16:34.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.937+0000 7fe6a37fe700 1 --2- 172.21.15.102:0/1866940366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6a4105570 0x7fe6a41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:34.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.937+0000 7fe6a37fe700 1 -- 172.21.15.102:0/1866940366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a4100a30 con 0x7fe6a4107ea0 2024-03-21T11:16:34.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.937+0000 7fe6a37fe700 1 --2- 172.21.15.102:0/1866940366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6a4107ea0 0x7fe6a419fa60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe69c006010 tx=0x7fe69c005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:16:34.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.937+0000 7fe69bfff700 1 -- 172.21.15.102:0/1866940366 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c027070 con 0x7fe6a4107ea0 2024-03-21T11:16:34.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.937+0000 7fe6a953e700 1 -- 172.21.15.102:0/1866940366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a4100cc0 con 0x7fe6a4107ea0 2024-03-21T11:16:34.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.937+0000 7fe69bfff700 1 -- 172.21.15.102:0/1866940366 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe69c0041f0 con 0x7fe6a4107ea0 2024-03-21T11:16:34.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.937+0000 7fe69bfff700 1 -- 172.21.15.102:0/1866940366 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00f460 con 0x7fe6a4107ea0 2024-03-21T11:16:34.941 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.937+0000 7fe6a953e700 1 -- 172.21.15.102:0/1866940366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a4101270 con 0x7fe6a4107ea0 2024-03-21T11:16:34.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.938+0000 7fe6a953e700 1 -- 172.21.15.102:0/1866940366 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a4068e50 con 0x7fe6a4107ea0 2024-03-21T11:16:34.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.939+0000 7fe69bfff700 1 -- 172.21.15.102:0/1866940366 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe69c02a070 con 0x7fe6a4107ea0 2024-03-21T11:16:34.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.939+0000 7fe69bfff700 1 --2- 172.21.15.102:0/1866940366 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6900a2fb0 0x7fe6900a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:16:34.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.940+0000 7fe69bfff700 1 -- 172.21.15.102:0/1866940366 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fe69c0cf350 con 0x7fe6a4107ea0 2024-03-21T11:16:34.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.940+0000 7fe6a2ffd700 1 --2- 172.21.15.102:0/1866940366 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6900a2fb0 0x7fe6900a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:16:34.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.941+0000 7fe6a2ffd700 1 --2- 172.21.15.102:0/1866940366 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6900a2fb0 0x7fe6900a5470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe69400a400 tx=0x7fe69400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:16:34.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:34.944+0000 7fe69bfff700 1 -- 172.21.15.102:0/1866940366 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe69c05e740 con 0x7fe6a4107ea0 2024-03-21T11:16:34.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:34 smithi102 ceph-mon[36748]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:34.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:34 smithi102 ceph-mon[36748]: Detected new or changed devices on smithi100 2024-03-21T11:16:34.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:34 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi100 to 8540M 2024-03-21T11:16:35.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:34 smithi100 ceph-mon[36245]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:35.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:34 smithi100 ceph-mon[36245]: Detected new or changed devices on smithi100 2024-03-21T11:16:35.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:34 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi100 to 8540M 2024-03-21T11:16:35.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:35.150+0000 7fe6a953e700 1 -- 172.21.15.102:0/1866940366 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6a40fdcc0 con 0x7fe6900a2fb0 2024-03-21T11:16:35.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:34 smithi047 ceph-mon[31351]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:35.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:34 smithi047 ceph-mon[31351]: Detected new or changed devices on smithi100 2024-03-21T11:16:35.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:34 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi100 to 8540M 2024-03-21T11:16:36.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:35 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:16:36.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:35 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:16:36.138 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:35 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:36.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:35 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:16:36.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:35 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:16:36.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:35 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:36.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:16:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:16:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:36.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:36 smithi102 ceph-mon[36748]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:36.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:36 smithi102 ceph-mon[36748]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:16:37.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:36 smithi100 ceph-mon[36245]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:37.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:36 smithi100 ceph-mon[36245]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:16:37.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:36 smithi047 ceph-mon[31351]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:37.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:36 smithi047 ceph-mon[31351]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:16:38.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:37 smithi100 ceph-mon[36245]: from='client.? 172.21.15.102:0/1152040368' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d02dcbc9-f19e-43ef-9b63-9c7039111849"}]: dispatch 2024-03-21T11:16:38.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:37 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d02dcbc9-f19e-43ef-9b63-9c7039111849"}]: dispatch 2024-03-21T11:16:38.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:37 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d02dcbc9-f19e-43ef-9b63-9c7039111849"}]': finished 2024-03-21T11:16:38.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:37 smithi100 ceph-mon[36245]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T11:16:38.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T11:16:38.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:37 smithi102 ceph-mon[36748]: from='client.? 172.21.15.102:0/1152040368' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d02dcbc9-f19e-43ef-9b63-9c7039111849"}]: dispatch 2024-03-21T11:16:38.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:37 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d02dcbc9-f19e-43ef-9b63-9c7039111849"}]: dispatch 2024-03-21T11:16:38.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:37 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d02dcbc9-f19e-43ef-9b63-9c7039111849"}]': finished 2024-03-21T11:16:38.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:37 smithi102 ceph-mon[36748]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T11:16:38.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T11:16:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:37 smithi047 ceph-mon[31351]: from='client.? 172.21.15.102:0/1152040368' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d02dcbc9-f19e-43ef-9b63-9c7039111849"}]: dispatch 2024-03-21T11:16:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:37 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d02dcbc9-f19e-43ef-9b63-9c7039111849"}]: dispatch 2024-03-21T11:16:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:37 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d02dcbc9-f19e-43ef-9b63-9c7039111849"}]': finished 2024-03-21T11:16:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:37 smithi047 ceph-mon[31351]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T11:16:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T11:16:39.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:38 smithi100 ceph-mon[36245]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:39.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:38 smithi100 ceph-mon[36245]: from='client.? 172.21.15.102:0/3679275261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:16:39.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:38 smithi102 ceph-mon[36748]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:38 smithi102 ceph-mon[36748]: from='client.? 172.21.15.102:0/3679275261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:16:39.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:38 smithi047 ceph-mon[31351]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:38 smithi047 ceph-mon[31351]: from='client.? 172.21.15.102:0/3679275261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:16:41.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:40 smithi100 ceph-mon[36245]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:41.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:40 smithi102 ceph-mon[36748]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:41.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:40 smithi047 ceph-mon[31351]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:42.946 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:42 smithi102 ceph-mon[36748]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:43.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:42 smithi100 ceph-mon[36245]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:43.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:42 smithi047 ceph-mon[31351]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:45.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:44 smithi100 ceph-mon[36245]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:45.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:44 smithi102 ceph-mon[36748]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:45.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:44 smithi047 ceph-mon[31351]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:46.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:45 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T11:16:46.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:45 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:46.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:45 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T11:16:46.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:45 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:46.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:45 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T11:16:46.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:45 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:47.124 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:46 smithi102 ceph-mon[36748]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:47.125 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:46 smithi102 ceph-mon[36748]: Deploying daemon osd.5 on smithi102 2024-03-21T11:16:47.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:46 smithi100 ceph-mon[36245]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:47.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:46 smithi100 ceph-mon[36245]: Deploying daemon osd.5 on smithi102 2024-03-21T11:16:47.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:46 smithi047 ceph-mon[31351]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:46 smithi047 ceph-mon[31351]: Deploying daemon osd.5 on smithi102 2024-03-21T11:16:48.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:47 smithi100 ceph-mon[36245]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:48.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:47 smithi102 ceph-mon[36748]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:48.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:47 smithi047 ceph-mon[31351]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:50.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:49 smithi102 ceph-mon[36748]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:50.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:49 smithi047 ceph-mon[31351]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:50.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:49 smithi100 ceph-mon[36245]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:51.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:51.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:51.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:51.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:51.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:51.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:51.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:51.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:51.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:52.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:52 smithi102 ceph-mon[36748]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:52.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:52 smithi047 ceph-mon[31351]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:52.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:52 smithi100 ceph-mon[36245]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:53.893 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 5 on host 'smithi102' 2024-03-21T11:16:53.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:53.892+0000 7fe69bfff700 1 -- 172.21.15.102:0/1866940366 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe6a40fdcc0 con 0x7fe6900a2fb0 2024-03-21T11:16:53.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:53.894+0000 7fe6a953e700 1 -- 172.21.15.102:0/1866940366 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6900a2fb0 msgr2=0x7fe6900a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:16:53.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:53.894+0000 7fe6a953e700 1 --2- 172.21.15.102:0/1866940366 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6900a2fb0 0x7fe6900a5470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe69400a400 tx=0x7fe69400d040 comp rx=0 tx=0).stop 2024-03-21T11:16:53.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:53.894+0000 7fe6a953e700 1 -- 172.21.15.102:0/1866940366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6a4107ea0 msgr2=0x7fe6a419fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:16:53.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:53.894+0000 7fe6a953e700 1 --2- 172.21.15.102:0/1866940366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6a4107ea0 0x7fe6a419fa60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe69c006010 tx=0x7fe69c005ee0 comp rx=0 tx=0).stop 2024-03-21T11:16:53.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:53.894+0000 7fe6a953e700 1 -- 172.21.15.102:0/1866940366 shutdown_connections 2024-03-21T11:16:53.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:53.895+0000 7fe6a953e700 1 --2- 172.21.15.102:0/1866940366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6a4107ea0 0x7fe6a419fa60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:53.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:53.895+0000 7fe6a953e700 1 --2- 172.21.15.102:0/1866940366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6a4102c40 0x7fe6a41984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:53.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:53.895+0000 7fe6a953e700 1 --2- 172.21.15.102:0/1866940366 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6900a2fb0 0x7fe6900a5470 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:53.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:53.895+0000 7fe6a953e700 1 --2- 172.21.15.102:0/1866940366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6a4105570 0x7fe6a41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:16:53.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:53.895+0000 7fe6a953e700 1 -- 172.21.15.102:0/1866940366 >> 172.21.15.102:0/1866940366 conn(0x7fe6a40fc650 msgr2=0x7fe6a4103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:16:53.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:53.895+0000 7fe6a953e700 1 -- 172.21.15.102:0/1866940366 shutdown_connections 2024-03-21T11:16:53.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:16:53.895+0000 7fe6a953e700 1 -- 172.21.15.102:0/1866940366 wait complete. 2024-03-21T11:16:54.545 DEBUG:teuthology.orchestra.run.smithi102:osd.5> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.5.service 2024-03-21T11:16:54.548 INFO:tasks.cephadm:Deploying osd.6 on smithi047 with /dev/vg_nvme/lv_2... 2024-03-21T11:16:54.548 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T11:16:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:16:55.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:54 smithi047 ceph-mon[31351]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:55.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:54 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:55.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:54 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:55.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:54 smithi047 ceph-mon[31351]: from='osd.5 [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T11:16:55.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:54 smithi047 ceph-mon[31351]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T11:16:55.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:54 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:55.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:54 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:55.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:54 smithi100 ceph-mon[36245]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:55.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:54 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:55.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:54 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:55.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:54 smithi100 ceph-mon[36245]: from='osd.5 [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T11:16:55.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:54 smithi100 ceph-mon[36245]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T11:16:55.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:54 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:55.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:54 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:55.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:54 smithi102 ceph-mon[36748]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:55.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:54 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:54 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:54 smithi102 ceph-mon[36748]: from='osd.5 [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T11:16:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:54 smithi102 ceph-mon[36748]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T11:16:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:54 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:54 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:56.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:55 smithi100 ceph-mon[36245]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T11:16:56.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:55 smithi100 ceph-mon[36245]: from='osd.5 [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:16:56.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:55 smithi100 ceph-mon[36245]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T11:16:56.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:55 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T11:16:56.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:55 smithi100 ceph-mon[36245]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:16:56.194 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:55 smithi102 ceph-mon[36748]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T11:16:56.195 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:55 smithi102 ceph-mon[36748]: from='osd.5 [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:16:56.195 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:55 smithi102 ceph-mon[36748]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T11:16:56.195 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:55 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T11:16:56.195 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:55 smithi102 ceph-mon[36748]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:16:56.195 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:16:55 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:16:55.866+0000 7f23a0e4a700 -1 osd.5 0 waiting for initial osdmap 2024-03-21T11:16:56.195 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:16:55 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:16:55.872+0000 7f239cc3d700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T11:16:56.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:55 smithi047 ceph-mon[31351]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T11:16:56.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:55 smithi047 ceph-mon[31351]: from='osd.5 [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:16:56.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:55 smithi047 ceph-mon[31351]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T11:16:56.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:55 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T11:16:56.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:55 smithi047 ceph-mon[31351]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:16:57.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:56 smithi100 ceph-mon[36245]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:57.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:56 smithi100 ceph-mon[36245]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-03-21T11:16:57.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:56 smithi100 ceph-mon[36245]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T11:16:57.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:56 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T11:16:57.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:56 smithi047 ceph-mon[31351]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:57.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:56 smithi047 ceph-mon[31351]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-03-21T11:16:57.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:56 smithi047 ceph-mon[31351]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T11:16:57.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T11:16:57.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:56 smithi102 ceph-mon[36748]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T11:16:57.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:56 smithi102 ceph-mon[36748]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-03-21T11:16:57.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:56 smithi102 ceph-mon[36748]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T11:16:57.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T11:16:57.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:57 smithi102 ceph-mon[36748]: purged_snaps scrub starts 2024-03-21T11:16:57.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:57 smithi102 ceph-mon[36748]: purged_snaps scrub ok 2024-03-21T11:16:57.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:57 smithi102 ceph-mon[36748]: osd.5 [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052] boot 2024-03-21T11:16:57.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:57 smithi102 ceph-mon[36748]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T11:16:57.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:57 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T11:16:57.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:57 smithi102 ceph-mon[36748]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:16:57.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:57 smithi102 ceph-mon[36748]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T11:16:58.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:57 smithi047 ceph-mon[31351]: purged_snaps scrub starts 2024-03-21T11:16:58.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:57 smithi047 ceph-mon[31351]: purged_snaps scrub ok 2024-03-21T11:16:58.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:57 smithi047 ceph-mon[31351]: osd.5 [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052] boot 2024-03-21T11:16:58.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:57 smithi047 ceph-mon[31351]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T11:16:58.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:57 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T11:16:58.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:57 smithi047 ceph-mon[31351]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:16:58.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:57 smithi047 ceph-mon[31351]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T11:16:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T11:16:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T11:16:58.060 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-03-21T11:16:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-03-21T11:16:58.060 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0412408 s, 254 MB/s 2024-03-21T11:16:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-03-21T11:16:58.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:57 smithi100 ceph-mon[36245]: purged_snaps scrub starts 2024-03-21T11:16:58.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:57 smithi100 ceph-mon[36245]: purged_snaps scrub ok 2024-03-21T11:16:58.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:57 smithi100 ceph-mon[36245]: osd.5 [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052] boot 2024-03-21T11:16:58.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:57 smithi100 ceph-mon[36245]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T11:16:58.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:57 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T11:16:58.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:57 smithi100 ceph-mon[36245]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:16:58.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:57 smithi100 ceph-mon[36245]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T11:16:59.008 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi047:vg_nvme/lv_2 2024-03-21T11:16:59.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: Detected new or changed devices on smithi102 2024-03-21T11:16:59.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi102 to 8540M 2024-03-21T11:16:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:59.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:59.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:16:59 smithi047 ceph-mon[31351]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T11:16:59.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: Detected new or changed devices on smithi102 2024-03-21T11:16:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:59.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi102 to 8540M 2024-03-21T11:16:59.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:59.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:59.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:59.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:59.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:59.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:16:59 smithi100 ceph-mon[36245]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T11:16:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:16:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: Detected new or changed devices on smithi102 2024-03-21T11:16:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:16:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi102 to 8540M 2024-03-21T11:16:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:16:59.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:16:59.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:16:59.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:16:59.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:16:59 smithi102 ceph-mon[36748]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T11:17:00.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:00 smithi102 ceph-mon[36748]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:00.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:00 smithi047 ceph-mon[31351]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:00.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:00 smithi100 ceph-mon[36245]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:01.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:01 smithi047 ceph-mon[31351]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T11:17:01.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:01 smithi100 ceph-mon[36245]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T11:17:01.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:01 smithi102 ceph-mon[36748]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T11:17:01.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.669+0000 7f60a9a00700 1 -- 172.21.15.47:0/3150508827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a41043d0 msgr2=0x7f60a41047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:17:01.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.669+0000 7f60a9a00700 1 --2- 172.21.15.47:0/3150508827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a41043d0 0x7f60a41047f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f60940098f0 tx=0x7f6094009c00 comp rx=0 tx=0).stop 2024-03-21T11:17:01.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.669+0000 7f60a9a00700 1 -- 172.21.15.47:0/3150508827 shutdown_connections 2024-03-21T11:17:01.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.669+0000 7f60a9a00700 1 --2- 172.21.15.47:0/3150508827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a4105f70 0x7f60a410c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:01.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.669+0000 7f60a9a00700 1 --2- 172.21.15.47:0/3150508827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a41043d0 0x7f60a41047f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:01.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.669+0000 7f60a9a00700 1 --2- 172.21.15.47:0/3150508827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60a41055d0 0x7f60a4105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.669+0000 7f60a9a00700 1 -- 172.21.15.47:0/3150508827 >> 172.21.15.47:0/3150508827 conn(0x7f60a40ff950 msgr2=0x7f60a4101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:17:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.669+0000 7f60a9a00700 1 -- 172.21.15.47:0/3150508827 shutdown_connections 2024-03-21T11:17:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.669+0000 7f60a9a00700 1 -- 172.21.15.47:0/3150508827 wait complete. 2024-03-21T11:17:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.670+0000 7f60a9a00700 1 Processor -- start 2024-03-21T11:17:01.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.671+0000 7f60a9a00700 1 -- start start 2024-03-21T11:17:01.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.671+0000 7f60a9a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60a41043d0 0x7f60a419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:17:01.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.671+0000 7f60a9a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a41055d0 0x7f60a419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:17:01.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.671+0000 7f60a9a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a4105f70 0x7f60a41a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:17:01.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.671+0000 7f60a9a00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a4110000 con 0x7f60a41055d0 2024-03-21T11:17:01.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.671+0000 7f60a9a00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a410fe80 con 0x7f60a41043d0 2024-03-21T11:17:01.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.671+0000 7f60a9a00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a4110180 con 0x7f60a4105f70 2024-03-21T11:17:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.672+0000 7f60a2000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a41055d0 0x7f60a419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:17:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.672+0000 7f60a3400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a4105f70 0x7f60a41a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:17:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.672+0000 7f60a2a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60a41043d0 0x7f60a419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:17:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.672+0000 7f60a3400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a4105f70 0x7f60a41a1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43408/0 (socket says 172.21.15.47:43408) 2024-03-21T11:17:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.672+0000 7f60a2000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a41055d0 0x7f60a419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48466/0 (socket says 172.21.15.47:48466) 2024-03-21T11:17:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.672+0000 7f60a3400700 1 -- 172.21.15.47:0/1557262130 learned_addr learned my addr 172.21.15.47:0/1557262130 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:17:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.672+0000 7f60a2a00700 1 -- 172.21.15.47:0/1557262130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a4105f70 msgr2=0x7f60a41a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:17:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.672+0000 7f60a2a00700 1 --2- 172.21.15.47:0/1557262130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a4105f70 0x7f60a41a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.672+0000 7f60a2a00700 1 -- 172.21.15.47:0/1557262130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a41055d0 msgr2=0x7f60a419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:17:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.672+0000 7f60a2a00700 1 --2- 172.21.15.47:0/1557262130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a41055d0 0x7f60a419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.672+0000 7f60a2a00700 1 -- 172.21.15.47:0/1557262130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a41a22f0 con 0x7f60a41043d0 2024-03-21T11:17:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.672+0000 7f60a3400700 1 --2- 172.21.15.47:0/1557262130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a4105f70 0x7f60a41a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:17:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.672+0000 7f60a2000700 1 --2- 172.21.15.47:0/1557262130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a41055d0 0x7f60a419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:17:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.673+0000 7f60a2a00700 1 --2- 172.21.15.47:0/1557262130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60a41043d0 0x7f60a419a690 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6094000c00 tx=0x7f60940052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:17:01.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.673+0000 7f609b400700 1 -- 172.21.15.47:0/1557262130 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609400b560 con 0x7f60a41043d0 2024-03-21T11:17:01.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.673+0000 7f60a9a00700 1 -- 172.21.15.47:0/1557262130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a41a2580 con 0x7f60a41043d0 2024-03-21T11:17:01.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.673+0000 7f609b400700 1 -- 172.21.15.47:0/1557262130 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f609400bbe0 con 0x7f60a41043d0 2024-03-21T11:17:01.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.673+0000 7f609b400700 1 -- 172.21.15.47:0/1557262130 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6094021460 con 0x7f60a41043d0 2024-03-21T11:17:01.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.673+0000 7f60a9a00700 1 -- 172.21.15.47:0/1557262130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a41a2a30 con 0x7f60a41043d0 2024-03-21T11:17:01.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.674+0000 7f60a9a00700 1 -- 172.21.15.47:0/1557262130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a4068e50 con 0x7f60a41043d0 2024-03-21T11:17:01.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.676+0000 7f609b400700 1 -- 172.21.15.47:0/1557262130 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6094003680 con 0x7f60a41043d0 2024-03-21T11:17:01.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.678+0000 7f609b400700 1 --2- 172.21.15.47:0/1557262130 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60880a2fb0 0x7f60880a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:17:01.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.678+0000 7f609b400700 1 -- 172.21.15.47:0/1557262130 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4533+0+0 (secure 0 0 0) 0x7f60940cf220 con 0x7f60a41043d0 2024-03-21T11:17:01.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.678+0000 7f60a2000700 1 --2- 172.21.15.47:0/1557262130 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60880a2fb0 0x7f60880a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:17:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.679+0000 7f60a2000700 1 --2- 172.21.15.47:0/1557262130 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60880a2fb0 0x7f60880a5470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6090011490 tx=0x7f6090009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:17:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.682+0000 7f609b400700 1 -- 172.21.15.47:0/1557262130 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6094013030 con 0x7f60a41043d0 2024-03-21T11:17:01.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:01.923+0000 7f60a9a00700 1 -- 172.21.15.47:0/1557262130 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f60a40625d0 con 0x7f60880a2fb0 2024-03-21T11:17:02.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:02 smithi047 ceph-mon[31351]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-03-21T11:17:02.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:02 smithi047 ceph-mon[31351]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T11:17:02.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:17:02.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:17:02.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:02.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:02 smithi100 ceph-mon[36245]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-03-21T11:17:02.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:02 smithi100 ceph-mon[36245]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T11:17:02.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:17:02.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:17:02.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:02.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:02 smithi102 ceph-mon[36748]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-03-21T11:17:02.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:02 smithi102 ceph-mon[36748]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T11:17:02.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:17:02.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:17:02.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:03.229 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:03 smithi047 ceph-mon[31351]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:17:03.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:03 smithi100 ceph-mon[36245]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:17:03.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:03 smithi102 ceph-mon[36748]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:17:04.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:04 smithi100 ceph-mon[36245]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-21T11:17:04.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:04 smithi102 ceph-mon[36748]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-21T11:17:04.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:04 smithi047 ceph-mon[31351]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-21T11:17:06.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:06 smithi100 ceph-mon[36245]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-21T11:17:06.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:06 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2692808520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbe3b7c0-943f-4849-a8b5-5d792e1f494b"}]: dispatch 2024-03-21T11:17:06.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:06 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbe3b7c0-943f-4849-a8b5-5d792e1f494b"}]: dispatch 2024-03-21T11:17:06.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:06 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbe3b7c0-943f-4849-a8b5-5d792e1f494b"}]': finished 2024-03-21T11:17:06.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:06 smithi100 ceph-mon[36245]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T11:17:06.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:06.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:06 smithi102 ceph-mon[36748]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-21T11:17:06.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:06 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2692808520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbe3b7c0-943f-4849-a8b5-5d792e1f494b"}]: dispatch 2024-03-21T11:17:06.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:06 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbe3b7c0-943f-4849-a8b5-5d792e1f494b"}]: dispatch 2024-03-21T11:17:06.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:06 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbe3b7c0-943f-4849-a8b5-5d792e1f494b"}]': finished 2024-03-21T11:17:06.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:06 smithi102 ceph-mon[36748]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T11:17:06.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:06.498 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:06 smithi047 ceph-mon[31351]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-21T11:17:06.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:06 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2692808520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbe3b7c0-943f-4849-a8b5-5d792e1f494b"}]: dispatch 2024-03-21T11:17:06.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:06 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fbe3b7c0-943f-4849-a8b5-5d792e1f494b"}]: dispatch 2024-03-21T11:17:06.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:06 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbe3b7c0-943f-4849-a8b5-5d792e1f494b"}]': finished 2024-03-21T11:17:06.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:06 smithi047 ceph-mon[31351]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T11:17:06.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:07.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:07 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/115175562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:17:07.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:07 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/115175562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:17:07.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:07 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/115175562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:17:08.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:08 smithi100 ceph-mon[36245]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-21T11:17:08.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:08 smithi102 ceph-mon[36748]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-21T11:17:08.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:08 smithi047 ceph-mon[31351]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-21T11:17:10.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:10 smithi100 ceph-mon[36245]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:10.403 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:10 smithi047 ceph-mon[31351]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:10.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:10 smithi102 ceph-mon[36748]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:12.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:12 smithi100 ceph-mon[36245]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:12.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:12 smithi047 ceph-mon[31351]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:12.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:12 smithi102 ceph-mon[36748]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:14.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:14 smithi100 ceph-mon[36245]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:14.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:14 smithi102 ceph-mon[36748]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:14.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:14 smithi047 ceph-mon[31351]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:16.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:16 smithi100 ceph-mon[36245]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:16.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T11:17:16.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:16.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:16 smithi100 ceph-mon[36245]: Deploying daemon osd.6 on smithi047 2024-03-21T11:17:16.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:16 smithi102 ceph-mon[36748]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:16.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T11:17:16.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:16.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:16 smithi102 ceph-mon[36748]: Deploying daemon osd.6 on smithi047 2024-03-21T11:17:16.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:16 smithi047 ceph-mon[31351]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:16.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T11:17:16.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:16.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:16 smithi047 ceph-mon[31351]: Deploying daemon osd.6 on smithi047 2024-03-21T11:17:18.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:18 smithi047 ceph-mon[31351]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:18.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:18 smithi100 ceph-mon[36245]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:18.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:18 smithi102 ceph-mon[36748]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:20.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:20 smithi047 ceph-mon[31351]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:20.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:20 smithi102 ceph-mon[36748]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:20.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:20 smithi100 ceph-mon[36245]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:22.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:22 smithi102 ceph-mon[36748]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:22.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:22 smithi047 ceph-mon[31351]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:22.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:22 smithi100 ceph-mon[36245]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:24.440 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:24 smithi047 ceph-mon[31351]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:24.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:24 smithi102 ceph-mon[36748]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:24.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:24 smithi100 ceph-mon[36245]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:25.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:25 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:25.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:25 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:25.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:25 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:17:25.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:25 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:25.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:25 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:25.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:25 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:17:25.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:25 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:25.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:25 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:25.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:25 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:17:26.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:26 smithi047 ceph-mon[31351]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:26.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:26 smithi100 ceph-mon[36245]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:26.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:26 smithi102 ceph-mon[36748]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:28.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:28 smithi047 ceph-mon[31351]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:28.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:28 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:28.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:28 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:28.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:28 smithi047 ceph-mon[31351]: from='osd.6 [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T11:17:28.492 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:28 smithi047 ceph-mon[31351]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T11:17:28.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:28 smithi100 ceph-mon[36245]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:28.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:28 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:28.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:28 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:28.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:28 smithi100 ceph-mon[36245]: from='osd.6 [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T11:17:28.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:28 smithi100 ceph-mon[36245]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T11:17:28.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:28 smithi102 ceph-mon[36748]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:28.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:28 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:28.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:28 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:28.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:28 smithi102 ceph-mon[36748]: from='osd.6 [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T11:17:28.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:28 smithi102 ceph-mon[36748]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T11:17:28.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:28.971+0000 7f609b400700 1 -- 172.21.15.47:0/1557262130 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f60a40625d0 con 0x7f60880a2fb0 2024-03-21T11:17:28.973 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 6 on host 'smithi047' 2024-03-21T11:17:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:28.974+0000 7f60a9a00700 1 -- 172.21.15.47:0/1557262130 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60880a2fb0 msgr2=0x7f60880a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:17:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:28.974+0000 7f60a9a00700 1 --2- 172.21.15.47:0/1557262130 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60880a2fb0 0x7f60880a5470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6090011490 tx=0x7f6090009260 comp rx=0 tx=0).stop 2024-03-21T11:17:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:28.974+0000 7f60a9a00700 1 -- 172.21.15.47:0/1557262130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60a41043d0 msgr2=0x7f60a419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:17:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:28.974+0000 7f60a9a00700 1 --2- 172.21.15.47:0/1557262130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60a41043d0 0x7f60a419a690 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6094000c00 tx=0x7f60940052f0 comp rx=0 tx=0).stop 2024-03-21T11:17:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:28.975+0000 7f60a9a00700 1 -- 172.21.15.47:0/1557262130 shutdown_connections 2024-03-21T11:17:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:28.975+0000 7f60a9a00700 1 --2- 172.21.15.47:0/1557262130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60a4105f70 0x7f60a41a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:28.975+0000 7f60a9a00700 1 --2- 172.21.15.47:0/1557262130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60a41043d0 0x7f60a419a690 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:28.975+0000 7f60a9a00700 1 --2- 172.21.15.47:0/1557262130 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60880a2fb0 0x7f60880a5470 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:28.975+0000 7f60a9a00700 1 --2- 172.21.15.47:0/1557262130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60a41055d0 0x7f60a419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:28.975+0000 7f60a9a00700 1 -- 172.21.15.47:0/1557262130 >> 172.21.15.47:0/1557262130 conn(0x7f60a40ff950 msgr2=0x7f60a41088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:17:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:28.975+0000 7f60a9a00700 1 -- 172.21.15.47:0/1557262130 shutdown_connections 2024-03-21T11:17:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:17:28.975+0000 7f60a9a00700 1 -- 172.21.15.47:0/1557262130 wait complete. 2024-03-21T11:17:29.725 DEBUG:teuthology.orchestra.run.smithi047:osd.6> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.6.service 2024-03-21T11:17:29.732 INFO:tasks.cephadm:Deploying osd.7 on smithi100 with /dev/vg_nvme/lv_2... 2024-03-21T11:17:29.732 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T11:17:29.980 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T11:17:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:29 smithi047 ceph-mon[31351]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T11:17:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:29 smithi047 ceph-mon[31351]: from='osd.6 [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:17:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:29 smithi047 ceph-mon[31351]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T11:17:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:29 smithi047 ceph-mon[31351]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:17:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:29 smithi047 ceph-mon[31351]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:30.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:30.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:30.101 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:29 smithi100 ceph-mon[36245]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T11:17:30.101 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:29 smithi100 ceph-mon[36245]: from='osd.6 [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:17:30.101 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:29 smithi100 ceph-mon[36245]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T11:17:30.101 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:30.101 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:29 smithi100 ceph-mon[36245]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:17:30.101 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:29 smithi100 ceph-mon[36245]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:30.101 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:30.101 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:29 smithi102 ceph-mon[36748]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T11:17:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:29 smithi102 ceph-mon[36748]: from='osd.6 [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:17:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:29 smithi102 ceph-mon[36748]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T11:17:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:29 smithi102 ceph-mon[36748]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:17:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:29 smithi102 ceph-mon[36748]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:30.774 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:17:30 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:17:30.544+0000 7ff861c00700 -1 osd.6 0 waiting for initial osdmap 2024-03-21T11:17:30.774 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:17:30 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:17:30.585+0000 7ff857a00700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T11:17:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:30 smithi102 ceph-mon[36748]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T11:17:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:30 smithi102 ceph-mon[36748]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T11:17:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:30 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:30 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:30 smithi102 ceph-mon[36748]: from='osd.6 ' entity='osd.6' 2024-03-21T11:17:31.296 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:30 smithi100 ceph-mon[36245]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T11:17:31.296 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:30 smithi100 ceph-mon[36245]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T11:17:31.297 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:30 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:31.297 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:30 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:31.297 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:30 smithi100 ceph-mon[36245]: from='osd.6 ' entity='osd.6' 2024-03-21T11:17:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:30 smithi047 ceph-mon[31351]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T11:17:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:30 smithi047 ceph-mon[31351]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T11:17:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:30 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:30 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:30 smithi047 ceph-mon[31351]: from='osd.6 ' entity='osd.6' 2024-03-21T11:17:32.012 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T11:17:32.012 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T11:17:32.012 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-21T11:17:32.012 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-21T11:17:32.013 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011214 s, 935 MB/s 2024-03-21T11:17:32.013 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-21T11:17:32.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:31 smithi047 ceph-mon[31351]: purged_snaps scrub starts 2024-03-21T11:17:32.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:31 smithi047 ceph-mon[31351]: purged_snaps scrub ok 2024-03-21T11:17:32.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:31 smithi047 ceph-mon[31351]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:32.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:31 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:32.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:31 smithi047 ceph-mon[31351]: osd.6 [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491] boot 2024-03-21T11:17:32.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:31 smithi047 ceph-mon[31351]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T11:17:32.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:31 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:32.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:31 smithi102 ceph-mon[36748]: purged_snaps scrub starts 2024-03-21T11:17:32.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:31 smithi102 ceph-mon[36748]: purged_snaps scrub ok 2024-03-21T11:17:32.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:31 smithi102 ceph-mon[36748]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:32.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:31 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:32.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:31 smithi102 ceph-mon[36748]: osd.6 [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491] boot 2024-03-21T11:17:32.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:31 smithi102 ceph-mon[36748]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T11:17:32.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:31 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:32.308 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:31 smithi100 ceph-mon[36245]: purged_snaps scrub starts 2024-03-21T11:17:32.308 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:31 smithi100 ceph-mon[36245]: purged_snaps scrub ok 2024-03-21T11:17:32.308 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:31 smithi100 ceph-mon[36245]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-21T11:17:32.308 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:31 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:32.308 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:31 smithi100 ceph-mon[36245]: osd.6 [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491] boot 2024-03-21T11:17:32.308 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:31 smithi100 ceph-mon[36245]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T11:17:32.308 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:31 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T11:17:32.760 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_2 2024-03-21T11:17:33.006 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T11:17:34.122 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:33 smithi100 ceph-mon[36245]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:34.172 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:33 smithi100 ceph-mon[36245]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T11:17:34.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:33 smithi102 ceph-mon[36748]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:34.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:33 smithi102 ceph-mon[36748]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T11:17:34.309 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:33 smithi047 ceph-mon[31351]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:34.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:33 smithi047 ceph-mon[31351]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T11:17:34.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.688+0000 7ff7abe0c700 1 -- 172.21.15.100:0/3853885721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a4102300 msgr2=0x7ff7a4102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:17:34.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.688+0000 7ff7abe0c700 1 --2- 172.21.15.100:0/3853885721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a4102300 0x7ff7a4102760 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff7940099c0 tx=0x7ff794009cd0 comp rx=0 tx=0).stop 2024-03-21T11:17:34.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.688+0000 7ff7abe0c700 1 -- 172.21.15.100:0/3853885721 shutdown_connections 2024-03-21T11:17:34.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.688+0000 7ff7abe0c700 1 --2- 172.21.15.100:0/3853885721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a4102ca0 0x7ff7a4109550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:34.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.688+0000 7ff7abe0c700 1 --2- 172.21.15.100:0/3853885721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7a4101100 0x7ff7a4101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:34.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.688+0000 7ff7abe0c700 1 --2- 172.21.15.100:0/3853885721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a4102300 0x7ff7a4102760 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:34.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.688+0000 7ff7abe0c700 1 -- 172.21.15.100:0/3853885721 >> 172.21.15.100:0/3853885721 conn(0x7ff7a40fc680 msgr2=0x7ff7a40feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:17:34.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.688+0000 7ff7abe0c700 1 -- 172.21.15.100:0/3853885721 shutdown_connections 2024-03-21T11:17:34.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.688+0000 7ff7abe0c700 1 -- 172.21.15.100:0/3853885721 wait complete. 2024-03-21T11:17:34.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.688+0000 7ff7abe0c700 1 Processor -- start 2024-03-21T11:17:34.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.688+0000 7ff7abe0c700 1 -- start start 2024-03-21T11:17:34.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7abe0c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7a4101100 0x7ff7a406b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:17:34.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7abe0c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a4102300 0x7ff7a419d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:17:34.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7abe0c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a4102ca0 0x7ff7a419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:17:34.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7abe0c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a410cd60 con 0x7ff7a4101100 2024-03-21T11:17:34.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7abe0c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a410cbe0 con 0x7ff7a4102300 2024-03-21T11:17:34.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7abe0c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a410cee0 con 0x7ff7a4102ca0 2024-03-21T11:17:34.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7a93a7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a4102300 0x7ff7a419d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:17:34.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7a93a7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a4102300 0x7ff7a419d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49502/0 (socket says 172.21.15.100:49502) 2024-03-21T11:17:34.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7a93a7700 1 -- 172.21.15.100:0/603602332 learned_addr learned my addr 172.21.15.100:0/603602332 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-21T11:17:34.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7aa3a9700 1 --2- 172.21.15.100:0/603602332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a4102ca0 0x7ff7a419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:17:34.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7a93a7700 1 -- 172.21.15.100:0/603602332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a4102ca0 msgr2=0x7ff7a419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:17:34.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7a93a7700 1 --2- 172.21.15.100:0/603602332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a4102ca0 0x7ff7a419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:34.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7a93a7700 1 -- 172.21.15.100:0/603602332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7a4101100 msgr2=0x7ff7a406b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:17:34.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7a93a7700 1 --2- 172.21.15.100:0/603602332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7a4101100 0x7ff7a406b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:34.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7a93a7700 1 -- 172.21.15.100:0/603602332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a41a00a0 con 0x7ff7a4102300 2024-03-21T11:17:34.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7a93a7700 1 --2- 172.21.15.100:0/603602332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a4102300 0x7ff7a419d140 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff794006010 tx=0x7ff7940057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:17:34.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff79affd700 1 -- 172.21.15.100:0/603602332 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794027070 con 0x7ff7a4102300 2024-03-21T11:17:34.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7abe0c700 1 -- 172.21.15.100:0/603602332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a41a0330 con 0x7ff7a4102300 2024-03-21T11:17:34.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff79affd700 1 -- 172.21.15.100:0/603602332 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff794004150 con 0x7ff7a4102300 2024-03-21T11:17:34.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff79affd700 1 -- 172.21.15.100:0/603602332 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79400f460 con 0x7ff7a4102300 2024-03-21T11:17:34.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.689+0000 7ff7abe0c700 1 -- 172.21.15.100:0/603602332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a41a07e0 con 0x7ff7a4102300 2024-03-21T11:17:34.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.690+0000 7ff7abe0c700 1 -- 172.21.15.100:0/603602332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a4068e50 con 0x7ff7a4102300 2024-03-21T11:17:34.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.691+0000 7ff79affd700 1 -- 172.21.15.100:0/603602332 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff79402a070 con 0x7ff7a4102300 2024-03-21T11:17:34.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.692+0000 7ff79affd700 1 --2- 172.21.15.100:0/603602332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7900a2fb0 0x7ff7900a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:17:34.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.692+0000 7ff79affd700 1 -- 172.21.15.100:0/603602332 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7ff7940cfe60 con 0x7ff7a4102300 2024-03-21T11:17:34.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.693+0000 7ff7a9ba8700 1 --2- 172.21.15.100:0/603602332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7900a2fb0 0x7ff7900a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:17:34.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.693+0000 7ff7a9ba8700 1 --2- 172.21.15.100:0/603602332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7900a2fb0 0x7ff7900a5470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4102160 tx=0x7ff7a0006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:17:34.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.696+0000 7ff79affd700 1 -- 172.21.15.100:0/603602332 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff794031020 con 0x7ff7a4102300 2024-03-21T11:17:34.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:34.897+0000 7ff7abe0c700 1 -- 172.21.15.100:0/603602332 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7a40625d0 con 0x7ff7900a2fb0 2024-03-21T11:17:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:34 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:17:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:34 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:17:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:34 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:35.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:34 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:17:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:34 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:17:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:34 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:35.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:34 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:17:35.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:34 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:17:35.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:34 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:36.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:17:36.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:36.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: Detected new or changed devices on smithi047 2024-03-21T11:17:36.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:36.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:36.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:17:36.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:17:36.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:17:36.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi047 to 5698M 2024-03-21T11:17:36.556 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:36.557 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:36.557 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:17:36.557 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:36.557 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:17:36.557 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:36.557 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:17:36.557 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:17:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: Detected new or changed devices on smithi047 2024-03-21T11:17:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:17:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:17:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:17:36.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi047 to 5698M 2024-03-21T11:17:36.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:36.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:36.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:17:36.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:36.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:17:36.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:36.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:17:36.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:17:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: Detected new or changed devices on smithi047 2024-03-21T11:17:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:17:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:17:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:17:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi047 to 5698M 2024-03-21T11:17:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:36.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:17:36.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:36.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:17:36.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:36.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:17:36.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:38.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:38 smithi100 ceph-mon[36245]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:38.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:38 smithi100 ceph-mon[36245]: from='client.? 172.21.15.100:0/1958356751' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53cea2b7-6bbe-4b15-b28e-7fdddd70607e"}]: dispatch 2024-03-21T11:17:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:38 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53cea2b7-6bbe-4b15-b28e-7fdddd70607e"}]: dispatch 2024-03-21T11:17:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:38 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53cea2b7-6bbe-4b15-b28e-7fdddd70607e"}]': finished 2024-03-21T11:17:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:38 smithi100 ceph-mon[36245]: osdmap e49: 8 total, 7 up, 8 in 2024-03-21T11:17:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:38 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:38.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:38 smithi047 ceph-mon[31351]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:38.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:38 smithi047 ceph-mon[31351]: from='client.? 172.21.15.100:0/1958356751' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53cea2b7-6bbe-4b15-b28e-7fdddd70607e"}]: dispatch 2024-03-21T11:17:38.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:38 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53cea2b7-6bbe-4b15-b28e-7fdddd70607e"}]: dispatch 2024-03-21T11:17:38.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:38 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53cea2b7-6bbe-4b15-b28e-7fdddd70607e"}]': finished 2024-03-21T11:17:38.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:38 smithi047 ceph-mon[31351]: osdmap e49: 8 total, 7 up, 8 in 2024-03-21T11:17:38.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:38.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:38 smithi102 ceph-mon[36748]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:38.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:38 smithi102 ceph-mon[36748]: from='client.? 172.21.15.100:0/1958356751' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53cea2b7-6bbe-4b15-b28e-7fdddd70607e"}]: dispatch 2024-03-21T11:17:38.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:38 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "53cea2b7-6bbe-4b15-b28e-7fdddd70607e"}]: dispatch 2024-03-21T11:17:38.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:38 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "53cea2b7-6bbe-4b15-b28e-7fdddd70607e"}]': finished 2024-03-21T11:17:38.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:38 smithi102 ceph-mon[36748]: osdmap e49: 8 total, 7 up, 8 in 2024-03-21T11:17:38.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:38 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:39.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:39 smithi047 ceph-mon[31351]: from='client.? 172.21.15.100:0/778874397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:17:39.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:39 smithi100 ceph-mon[36245]: from='client.? 172.21.15.100:0/778874397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:17:39.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:39 smithi102 ceph-mon[36748]: from='client.? 172.21.15.100:0/778874397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:17:40.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:40 smithi047 ceph-mon[31351]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:40.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:40 smithi100 ceph-mon[36245]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:40.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:40 smithi102 ceph-mon[36748]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:42.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:42 smithi047 ceph-mon[31351]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:42.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:42 smithi100 ceph-mon[36245]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:42.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:42 smithi102 ceph-mon[36748]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:44.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:44 smithi047 ceph-mon[31351]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:44.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:44 smithi100 ceph-mon[36245]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:44.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:44 smithi102 ceph-mon[36748]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:45.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:45 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T11:17:45.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:45 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:45.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:45 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T11:17:45.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:45 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:45.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:45 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T11:17:45.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:45 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:46.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:46 smithi100 ceph-mon[36245]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:46.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:46 smithi100 ceph-mon[36245]: Deploying daemon osd.7 on smithi100 2024-03-21T11:17:46.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:46 smithi047 ceph-mon[31351]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:46.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:46 smithi047 ceph-mon[31351]: Deploying daemon osd.7 on smithi100 2024-03-21T11:17:46.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:46 smithi102 ceph-mon[36748]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:46.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:46 smithi102 ceph-mon[36748]: Deploying daemon osd.7 on smithi100 2024-03-21T11:17:48.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:48 smithi047 ceph-mon[31351]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:48.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:48 smithi100 ceph-mon[36245]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:48.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:48 smithi102 ceph-mon[36748]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:50.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:50 smithi047 ceph-mon[31351]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:50.600 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:50 smithi100 ceph-mon[36245]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:50.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:50 smithi102 ceph-mon[36748]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:52.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:52.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:52.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:17:52.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:17:52.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:52.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:52.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:17:53.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:52 smithi100 ceph-mon[36245]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:53.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:52 smithi102 ceph-mon[36748]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:53.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:52 smithi047 ceph-mon[31351]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:54.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:54.432+0000 7ff79affd700 1 -- 172.21.15.100:0/603602332 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff7a40625d0 con 0x7ff7900a2fb0 2024-03-21T11:17:54.433 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 7 on host 'smithi100' 2024-03-21T11:17:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:54.434+0000 7ff7abe0c700 1 -- 172.21.15.100:0/603602332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7900a2fb0 msgr2=0x7ff7900a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:17:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:54.434+0000 7ff7abe0c700 1 --2- 172.21.15.100:0/603602332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7900a2fb0 0x7ff7900a5470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4102160 tx=0x7ff7a0006d20 comp rx=0 tx=0).stop 2024-03-21T11:17:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:54.434+0000 7ff7abe0c700 1 -- 172.21.15.100:0/603602332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a4102300 msgr2=0x7ff7a419d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:17:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:54.434+0000 7ff7abe0c700 1 --2- 172.21.15.100:0/603602332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a4102300 0x7ff7a419d140 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff794006010 tx=0x7ff7940057c0 comp rx=0 tx=0).stop 2024-03-21T11:17:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:54.434+0000 7ff7abe0c700 1 -- 172.21.15.100:0/603602332 shutdown_connections 2024-03-21T11:17:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:54.434+0000 7ff7abe0c700 1 --2- 172.21.15.100:0/603602332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a4102ca0 0x7ff7a419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:54.434+0000 7ff7abe0c700 1 --2- 172.21.15.100:0/603602332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7900a2fb0 0x7ff7900a5470 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:54.434+0000 7ff7abe0c700 1 --2- 172.21.15.100:0/603602332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7a4101100 0x7ff7a406b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:54.434+0000 7ff7abe0c700 1 --2- 172.21.15.100:0/603602332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7a4102300 0x7ff7a419d140 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:54.434+0000 7ff7abe0c700 1 -- 172.21.15.100:0/603602332 >> 172.21.15.100:0/603602332 conn(0x7ff7a40fc680 msgr2=0x7ff7a4105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:17:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:54.435+0000 7ff7abe0c700 1 -- 172.21.15.100:0/603602332 shutdown_connections 2024-03-21T11:17:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:17:54.435+0000 7ff7abe0c700 1 -- 172.21.15.100:0/603602332 wait complete. 2024-03-21T11:17:55.112 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:54 smithi100 ceph-mon[36245]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:55.112 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:54 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:55.112 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:54 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:55.113 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:54 smithi100 ceph-mon[36245]: from='osd.7 [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T11:17:55.113 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:54 smithi100 ceph-mon[36245]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T11:17:55.139 DEBUG:teuthology.orchestra.run.smithi100:osd.7> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.7.service 2024-03-21T11:17:55.141 INFO:tasks.cephadm:Deploying osd.8 on smithi102 with /dev/vg_nvme/lv_2... 2024-03-21T11:17:55.141 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T11:17:55.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:54 smithi102 ceph-mon[36748]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:55.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:54 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:55.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:54 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:55.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:54 smithi102 ceph-mon[36748]: from='osd.7 [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T11:17:55.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:54 smithi102 ceph-mon[36748]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T11:17:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:54 smithi047 ceph-mon[31351]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:54 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:54 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:54 smithi047 ceph-mon[31351]: from='osd.7 [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T11:17:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:54 smithi047 ceph-mon[31351]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T11:17:55.395 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:17:56.545 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:56 smithi102 ceph-mon[36748]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:56.545 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:56.545 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:56.545 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:56 smithi102 ceph-mon[36748]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T11:17:56.545 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:56 smithi102 ceph-mon[36748]: from='osd.7 [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:17:56.545 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:56 smithi102 ceph-mon[36748]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T11:17:56.545 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:56.545 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:56 smithi102 ceph-mon[36748]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:17:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:56 smithi047 ceph-mon[31351]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:56 smithi047 ceph-mon[31351]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T11:17:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:56 smithi047 ceph-mon[31351]: from='osd.7 [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:17:56.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:56 smithi047 ceph-mon[31351]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T11:17:56.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:56.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:56 smithi047 ceph-mon[31351]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:17:56.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:56 smithi100 ceph-mon[36245]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:56.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:56 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:56.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:56 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:56.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:56 smithi100 ceph-mon[36245]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T11:17:56.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:56 smithi100 ceph-mon[36245]: from='osd.7 [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:17:56.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:56 smithi100 ceph-mon[36245]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T11:17:56.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:56 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:56.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:56 smithi100 ceph-mon[36245]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:17:57.078 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:17:56 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:17:56.807+0000 7f8fb0259700 -1 osd.7 0 waiting for initial osdmap 2024-03-21T11:17:57.078 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:17:56 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:17:56.817+0000 7f8fa8044700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T11:17:57.345 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T11:17:57.345 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T11:17:57.345 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-03-21T11:17:57.345 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-03-21T11:17:57.345 INFO:teuthology.orchestra.run.smithi102.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112861 s, 929 MB/s 2024-03-21T11:17:57.345 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-03-21T11:17:57.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:57 smithi100 ceph-mon[36245]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T11:17:57.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:57 smithi100 ceph-mon[36245]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T11:17:57.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:57 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:57.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:57 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:57.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:57 smithi100 ceph-mon[36245]: from='osd.7 ' entity='osd.7' 2024-03-21T11:17:57.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:57 smithi102 ceph-mon[36748]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T11:17:57.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:57 smithi102 ceph-mon[36748]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T11:17:57.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:57 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:57.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:57 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:57.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:57 smithi102 ceph-mon[36748]: from='osd.7 ' entity='osd.7' 2024-03-21T11:17:57.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:57 smithi047 ceph-mon[31351]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T11:17:57.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:57 smithi047 ceph-mon[31351]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T11:17:57.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:57 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:57.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:57 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:57.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:57 smithi047 ceph-mon[31351]: from='osd.7 ' entity='osd.7' 2024-03-21T11:17:57.959 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi102:vg_nvme/lv_2 2024-03-21T11:17:58.240 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:17:58.602 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:58 smithi102 ceph-mon[36748]: purged_snaps scrub starts 2024-03-21T11:17:58.602 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:58 smithi102 ceph-mon[36748]: purged_snaps scrub ok 2024-03-21T11:17:58.602 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:58 smithi102 ceph-mon[36748]: pgmap v195: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:58.602 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:58 smithi102 ceph-mon[36748]: osd.7 [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551] boot 2024-03-21T11:17:58.602 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:58 smithi102 ceph-mon[36748]: osdmap e52: 8 total, 8 up, 8 in 2024-03-21T11:17:58.602 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:58 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:58.602 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:58 smithi102 ceph-mon[36748]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-21T11:17:58.602 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:58 smithi102 ceph-mon[36748]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T11:17:58.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:58 smithi100 ceph-mon[36245]: purged_snaps scrub starts 2024-03-21T11:17:58.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:58 smithi100 ceph-mon[36245]: purged_snaps scrub ok 2024-03-21T11:17:58.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:58 smithi100 ceph-mon[36245]: pgmap v195: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:58.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:58 smithi100 ceph-mon[36245]: osd.7 [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551] boot 2024-03-21T11:17:58.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:58 smithi100 ceph-mon[36245]: osdmap e52: 8 total, 8 up, 8 in 2024-03-21T11:17:58.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:58 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:58.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:58 smithi100 ceph-mon[36245]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-21T11:17:58.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:58 smithi100 ceph-mon[36245]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T11:17:58.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:58 smithi047 ceph-mon[31351]: purged_snaps scrub starts 2024-03-21T11:17:58.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:58 smithi047 ceph-mon[31351]: purged_snaps scrub ok 2024-03-21T11:17:58.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:58 smithi047 ceph-mon[31351]: pgmap v195: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-21T11:17:58.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:58 smithi047 ceph-mon[31351]: osd.7 [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551] boot 2024-03-21T11:17:58.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:58 smithi047 ceph-mon[31351]: osdmap e52: 8 total, 8 up, 8 in 2024-03-21T11:17:58.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:58 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T11:17:58.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:58 smithi047 ceph-mon[31351]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-21T11:17:58.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:58 smithi047 ceph-mon[31351]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T11:17:59.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.905+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1154232851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8107f20 msgr2=0x7f80c810a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:17:59.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.905+0000 7f80cd7dd700 1 --2- 172.21.15.102:0/1154232851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8107f20 0x7f80c810a310 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f80c000b0a0 tx=0x7f80c000b3b0 comp rx=0 tx=0).stop 2024-03-21T11:17:59.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.905+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1154232851 shutdown_connections 2024-03-21T11:17:59.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.905+0000 7f80cd7dd700 1 --2- 172.21.15.102:0/1154232851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8107f20 0x7f80c810a310 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:59.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.905+0000 7f80cd7dd700 1 --2- 172.21.15.102:0/1154232851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c8102cc0 0x7f80c81050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:59.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.905+0000 7f80cd7dd700 1 --2- 172.21.15.102:0/1154232851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80c81055f0 0x7f80c81079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:59.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.905+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1154232851 >> 172.21.15.102:0/1154232851 conn(0x7f80c80fc650 msgr2=0x7f80c80fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:17:59.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.905+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1154232851 shutdown_connections 2024-03-21T11:17:59.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.905+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1154232851 wait complete. 2024-03-21T11:17:59.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80cd7dd700 1 Processor -- start 2024-03-21T11:17:59.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80cd7dd700 1 -- start start 2024-03-21T11:17:59.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80cd7dd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8102cc0 0x7f80c8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:17:59.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80cd7dd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c81055f0 0x7f80c8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:17:59.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80cd7dd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80c8107f20 0x7f80c819f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:17:59.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80cd7dd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c810de60 con 0x7f80c8107f20 2024-03-21T11:17:59.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80cd7dd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c810dce0 con 0x7f80c81055f0 2024-03-21T11:17:59.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80cd7dd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c810dfe0 con 0x7f80c8102cc0 2024-03-21T11:17:59.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80c6ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8102cc0 0x7f80c8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:17:59.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80c6ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8102cc0 0x7f80c8198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33700/0 (socket says 172.21.15.102:33700) 2024-03-21T11:17:59.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80c6ffd700 1 -- 172.21.15.102:0/1625078490 learned_addr learned my addr 172.21.15.102:0/1625078490 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-21T11:17:59.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80be7fc700 1 --2- 172.21.15.102:0/1625078490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c81055f0 0x7f80c8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:17:59.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80c6ffd700 1 -- 172.21.15.102:0/1625078490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c81055f0 msgr2=0x7f80c8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:17:59.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80c6ffd700 1 --2- 172.21.15.102:0/1625078490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c81055f0 0x7f80c8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:59.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80c6ffd700 1 -- 172.21.15.102:0/1625078490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80c8107f20 msgr2=0x7f80c819f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:17:59.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80c6ffd700 1 --2- 172.21.15.102:0/1625078490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80c8107f20 0x7f80c819f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:17:59.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.906+0000 7f80c6ffd700 1 -- 172.21.15.102:0/1625078490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c81a0070 con 0x7f80c8102cc0 2024-03-21T11:17:59.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.907+0000 7f80c6ffd700 1 --2- 172.21.15.102:0/1625078490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8102cc0 0x7f80c8198410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f80b800b560 tx=0x7f80b800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:17:59.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.907+0000 7f80c4ff9700 1 -- 172.21.15.102:0/1625078490 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8015020 con 0x7f80c8102cc0 2024-03-21T11:17:59.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.907+0000 7f80c4ff9700 1 -- 172.21.15.102:0/1625078490 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80b8011d10 con 0x7f80c8102cc0 2024-03-21T11:17:59.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.907+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1625078490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c81a0360 con 0x7f80c8102cc0 2024-03-21T11:17:59.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.907+0000 7f80c4ff9700 1 -- 172.21.15.102:0/1625078490 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8010e50 con 0x7f80c8102cc0 2024-03-21T11:17:59.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.907+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1625078490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c81a08d0 con 0x7f80c8102cc0 2024-03-21T11:17:59.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.908+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1625078490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c8068e50 con 0x7f80c8102cc0 2024-03-21T11:17:59.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.909+0000 7f80c4ff9700 1 -- 172.21.15.102:0/1625078490 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f80b8018020 con 0x7f80c8102cc0 2024-03-21T11:17:59.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.909+0000 7f80c4ff9700 1 --2- 172.21.15.102:0/1625078490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80b40a3150 0x7f80b40a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:17:59.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.910+0000 7f80c4ff9700 1 -- 172.21.15.102:0/1625078490 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f80b80ce0b0 con 0x7f80c8102cc0 2024-03-21T11:17:59.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.910+0000 7f80be7fc700 1 --2- 172.21.15.102:0/1625078490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80b40a3150 0x7f80b40a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:17:59.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.911+0000 7f80be7fc700 1 --2- 172.21.15.102:0/1625078490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80b40a3150 0x7f80b40a5610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009a10 tx=0x7f80b0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:17:59.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:17:59.914+0000 7f80c4ff9700 1 -- 172.21.15.102:0/1625078490 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f80b801f020 con 0x7f80c8102cc0 2024-03-21T11:17:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: Detected new or changed devices on smithi100 2024-03-21T11:17:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:17:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:17:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:17:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi100 to 5693M 2024-03-21T11:17:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:17:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:17:59.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:17:59.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:17:59.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:17:59.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:17:59 smithi102 ceph-mon[36748]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T11:18:00.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: Detected new or changed devices on smithi100 2024-03-21T11:18:00.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:00.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:00.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:00.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:00.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:00.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi100 to 5693M 2024-03-21T11:18:00.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:00.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:00.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:18:00.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:00.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:18:00.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:00.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:18:00.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:00.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:17:59 smithi047 ceph-mon[31351]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T11:18:00.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:18:00.118+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1625078490 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f80c80625d0 con 0x7f80b40a3150 2024-03-21T11:18:00.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: Detected new or changed devices on smithi100 2024-03-21T11:18:00.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:00.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:00.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:00.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:00.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:00.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi100 to 5693M 2024-03-21T11:18:00.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:00.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:00.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:18:00.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:00.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:18:00.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:00.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:18:00.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:00.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:17:59 smithi100 ceph-mon[36245]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T11:18:01.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:00 smithi047 ceph-mon[31351]: pgmap v198: 1 pgs: 1 unknown; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:01.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:00 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:18:01.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:00 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:18:01.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:00 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:01.097 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:00 smithi102 ceph-mon[36748]: pgmap v198: 1 pgs: 1 unknown; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:01.097 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:00 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:18:01.097 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:00 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:18:01.097 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:00 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:00 smithi100 ceph-mon[36245]: pgmap v198: 1 pgs: 1 unknown; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:00 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:18:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:00 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:18:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:00 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:02.021 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:01 smithi102 ceph-mon[36748]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:18:02.021 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:01 smithi102 ceph-mon[36748]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T11:18:02.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:01 smithi047 ceph-mon[31351]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:18:02.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:01 smithi047 ceph-mon[31351]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T11:18:02.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:01 smithi100 ceph-mon[36245]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:18:02.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:01 smithi100 ceph-mon[36245]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T11:18:03.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:02 smithi100 ceph-mon[36245]: pgmap v201: 1 pgs: 1 unknown; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:03.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:02 smithi100 ceph-mon[36245]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T11:18:03.209 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:02 smithi102 ceph-mon[36748]: pgmap v201: 1 pgs: 1 unknown; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:03.210 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:02 smithi102 ceph-mon[36748]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T11:18:03.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:02 smithi047 ceph-mon[31351]: pgmap v201: 1 pgs: 1 unknown; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:03.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:02 smithi047 ceph-mon[31351]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T11:18:04.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:03 smithi100 ceph-mon[36245]: from='client.? 172.21.15.102:0/2554217242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20bf15de-3238-4613-9cd2-6bcc5ba6d60f"}]: dispatch 2024-03-21T11:18:04.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:03 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20bf15de-3238-4613-9cd2-6bcc5ba6d60f"}]: dispatch 2024-03-21T11:18:04.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:03 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20bf15de-3238-4613-9cd2-6bcc5ba6d60f"}]': finished 2024-03-21T11:18:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:03 smithi100 ceph-mon[36245]: osdmap e57: 9 total, 8 up, 9 in 2024-03-21T11:18:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:03 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:03 smithi100 ceph-mon[36245]: from='client.? 172.21.15.102:0/3695005451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:18:04.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:03 smithi102 ceph-mon[36748]: from='client.? 172.21.15.102:0/2554217242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20bf15de-3238-4613-9cd2-6bcc5ba6d60f"}]: dispatch 2024-03-21T11:18:04.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:03 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20bf15de-3238-4613-9cd2-6bcc5ba6d60f"}]: dispatch 2024-03-21T11:18:04.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:03 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20bf15de-3238-4613-9cd2-6bcc5ba6d60f"}]': finished 2024-03-21T11:18:04.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:03 smithi102 ceph-mon[36748]: osdmap e57: 9 total, 8 up, 9 in 2024-03-21T11:18:04.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:03 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:04.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:03 smithi102 ceph-mon[36748]: from='client.? 172.21.15.102:0/3695005451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:18:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:03 smithi047 ceph-mon[31351]: from='client.? 172.21.15.102:0/2554217242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20bf15de-3238-4613-9cd2-6bcc5ba6d60f"}]: dispatch 2024-03-21T11:18:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:03 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20bf15de-3238-4613-9cd2-6bcc5ba6d60f"}]: dispatch 2024-03-21T11:18:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:03 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20bf15de-3238-4613-9cd2-6bcc5ba6d60f"}]': finished 2024-03-21T11:18:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:03 smithi047 ceph-mon[31351]: osdmap e57: 9 total, 8 up, 9 in 2024-03-21T11:18:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:03 smithi047 ceph-mon[31351]: from='client.? 172.21.15.102:0/3695005451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:18:05.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:04 smithi100 ceph-mon[36245]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:05.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:04 smithi100 ceph-mon[36245]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-21T11:18:05.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:04 smithi100 ceph-mon[36245]: Cluster is now healthy 2024-03-21T11:18:05.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:04 smithi102 ceph-mon[36748]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:04 smithi102 ceph-mon[36748]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-21T11:18:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:04 smithi102 ceph-mon[36748]: Cluster is now healthy 2024-03-21T11:18:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:04 smithi047 ceph-mon[31351]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:04 smithi047 ceph-mon[31351]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-21T11:18:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:04 smithi047 ceph-mon[31351]: Cluster is now healthy 2024-03-21T11:18:07.110 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:06 smithi102 ceph-mon[36748]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:07.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:06 smithi100 ceph-mon[36245]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:07.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:06 smithi047 ceph-mon[31351]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:08.989 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:08 smithi102 ceph-mon[36748]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:09.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:08 smithi100 ceph-mon[36245]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:09.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:08 smithi047 ceph-mon[31351]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:10.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:10 smithi102 ceph-mon[36748]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:10.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T11:18:10.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:11.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:10 smithi100 ceph-mon[36245]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T11:18:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:11.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:10 smithi047 ceph-mon[31351]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:11.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T11:18:11.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:12.033 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:11 smithi102 ceph-mon[36748]: Deploying daemon osd.8 on smithi102 2024-03-21T11:18:12.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:11 smithi100 ceph-mon[36245]: Deploying daemon osd.8 on smithi102 2024-03-21T11:18:12.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:11 smithi047 ceph-mon[31351]: Deploying daemon osd.8 on smithi102 2024-03-21T11:18:13.131 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:12 smithi102 ceph-mon[36748]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:13.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:12 smithi100 ceph-mon[36245]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:13.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:12 smithi047 ceph-mon[31351]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:14 smithi100 ceph-mon[36245]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:15.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:14 smithi102 ceph-mon[36748]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:15.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:14 smithi047 ceph-mon[31351]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:17.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:16 smithi047 ceph-mon[31351]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:17.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:17.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:18:17.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:17.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:16 smithi100 ceph-mon[36245]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:17.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:17.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:18:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:17.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:16 smithi102 ceph-mon[36748]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:18:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:19.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:18 smithi047 ceph-mon[31351]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:19.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:18 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:18 smithi100 ceph-mon[36245]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:18 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:19.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:18 smithi102 ceph-mon[36748]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:18 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:19.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:18:19.561+0000 7f80c4ff9700 1 -- 172.21.15.102:0/1625078490 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f80c80625d0 con 0x7f80b40a3150 2024-03-21T11:18:19.562 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 8 on host 'smithi102' 2024-03-21T11:18:19.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:18:19.563+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1625078490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80b40a3150 msgr2=0x7f80b40a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:18:19.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:18:19.563+0000 7f80cd7dd700 1 --2- 172.21.15.102:0/1625078490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80b40a3150 0x7f80b40a5610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009a10 tx=0x7f80b0009190 comp rx=0 tx=0).stop 2024-03-21T11:18:19.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:18:19.563+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1625078490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8102cc0 msgr2=0x7f80c8198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:18:19.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:18:19.563+0000 7f80cd7dd700 1 --2- 172.21.15.102:0/1625078490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8102cc0 0x7f80c8198410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f80b800b560 tx=0x7f80b800b870 comp rx=0 tx=0).stop 2024-03-21T11:18:19.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:18:19.563+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1625078490 shutdown_connections 2024-03-21T11:18:19.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:18:19.563+0000 7f80cd7dd700 1 --2- 172.21.15.102:0/1625078490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80b40a3150 0x7f80b40a5610 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:18:19.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:18:19.563+0000 7f80cd7dd700 1 --2- 172.21.15.102:0/1625078490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80c8107f20 0x7f80c819f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:18:19.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:18:19.563+0000 7f80cd7dd700 1 --2- 172.21.15.102:0/1625078490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8102cc0 0x7f80c8198410 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:18:19.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:18:19.563+0000 7f80cd7dd700 1 --2- 172.21.15.102:0/1625078490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c81055f0 0x7f80c8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:18:19.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:18:19.563+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1625078490 >> 172.21.15.102:0/1625078490 conn(0x7f80c80fc650 msgr2=0x7f80c81035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:18:19.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:18:19.563+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1625078490 shutdown_connections 2024-03-21T11:18:19.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:18:19.563+0000 7f80cd7dd700 1 -- 172.21.15.102:0/1625078490 wait complete. 2024-03-21T11:18:19.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:19 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:19.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:19 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:19.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:19 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:18:19.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:19 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:19.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:19 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:19.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:19 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:18:19.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:19 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:19.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:19 smithi100 ceph-mon[36245]: from='osd.8 [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T11:18:19.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:19 smithi100 ceph-mon[36245]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T11:18:19.934 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:19 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:19.935 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:19 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:19.935 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:19 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:18:19.935 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:19 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:19.935 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:19 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:19.935 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:19 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:18:19.935 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:19 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:19.935 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:19 smithi102 ceph-mon[36748]: from='osd.8 [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T11:18:19.935 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:19 smithi102 ceph-mon[36748]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T11:18:20.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:19 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:20.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:19 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:20.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:19 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:18:20.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:19 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:20.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:19 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:20.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:19 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:18:20.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:19 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:20.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:19 smithi047 ceph-mon[31351]: from='osd.8 [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T11:18:20.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:19 smithi047 ceph-mon[31351]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T11:18:20.286 DEBUG:teuthology.orchestra.run.smithi102:osd.8> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.8.service 2024-03-21T11:18:20.288 INFO:tasks.cephadm:Deploying osd.9 on smithi047 with /dev/vg_nvme/lv_1... 2024-03-21T11:18:20.288 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T11:18:20.768 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:18:21.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:20 smithi047 ceph-mon[31351]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:21.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:20 smithi047 ceph-mon[31351]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T11:18:21.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:20 smithi047 ceph-mon[31351]: from='osd.8 [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:18:21.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:20 smithi047 ceph-mon[31351]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T11:18:21.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:20 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:21.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:20 smithi047 ceph-mon[31351]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:18:21.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:20 smithi100 ceph-mon[36245]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:21.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:20 smithi100 ceph-mon[36245]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T11:18:21.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:20 smithi100 ceph-mon[36245]: from='osd.8 [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:18:21.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:20 smithi100 ceph-mon[36245]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T11:18:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:20 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:20 smithi100 ceph-mon[36245]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:18:21.192 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:20 smithi102 ceph-mon[36748]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:21.192 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:20 smithi102 ceph-mon[36748]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T11:18:21.192 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:20 smithi102 ceph-mon[36748]: from='osd.8 [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:18:21.192 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:20 smithi102 ceph-mon[36748]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T11:18:21.192 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:20 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:21.192 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:20 smithi102 ceph-mon[36748]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:18:22.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:18:21 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:18:21.974+0000 7f71c81e4700 -1 osd.8 0 waiting for initial osdmap 2024-03-21T11:18:22.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:18:21 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:18:21.987+0000 7f71bffcf700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T11:18:22.808 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:22 smithi047 ceph-mon[31351]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:22.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:22 smithi047 ceph-mon[31351]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-03-21T11:18:22.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:22 smithi047 ceph-mon[31351]: osdmap e59: 9 total, 8 up, 9 in 2024-03-21T11:18:22.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:22.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:22.809 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:22 smithi047 ceph-mon[31351]: from='osd.8 ' entity='osd.8' 2024-03-21T11:18:22.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:22 smithi100 ceph-mon[36245]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:22.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:22 smithi100 ceph-mon[36245]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-03-21T11:18:22.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:22 smithi100 ceph-mon[36245]: osdmap e59: 9 total, 8 up, 9 in 2024-03-21T11:18:22.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:22.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:22.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:22 smithi100 ceph-mon[36245]: from='osd.8 ' entity='osd.8' 2024-03-21T11:18:22.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:22 smithi102 ceph-mon[36748]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T11:18:22.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:22 smithi102 ceph-mon[36748]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-03-21T11:18:22.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:22 smithi102 ceph-mon[36748]: osdmap e59: 9 total, 8 up, 9 in 2024-03-21T11:18:22.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:22.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:22.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:22 smithi102 ceph-mon[36748]: from='osd.8 ' entity='osd.8' 2024-03-21T11:18:23.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: purged_snaps scrub starts 2024-03-21T11:18:23.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: purged_snaps scrub ok 2024-03-21T11:18:23.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: osd.8 [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917] boot 2024-03-21T11:18:23.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: osdmap e60: 9 total, 9 up, 9 in 2024-03-21T11:18:23.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:23.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:23.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:23.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:23.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:23.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:23.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:23.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:23.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:18:23.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:23.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:23 smithi047 ceph-mon[31351]: osdmap e61: 9 total, 9 up, 9 in 2024-03-21T11:18:23.847 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T11:18:23.847 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T11:18:23.847 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-03-21T11:18:23.847 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-03-21T11:18:23.848 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.037074 s, 283 MB/s 2024-03-21T11:18:23.848 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-03-21T11:18:23.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: purged_snaps scrub starts 2024-03-21T11:18:23.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: purged_snaps scrub ok 2024-03-21T11:18:23.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: osd.8 [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917] boot 2024-03-21T11:18:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: osdmap e60: 9 total, 9 up, 9 in 2024-03-21T11:18:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:18:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:23 smithi100 ceph-mon[36245]: osdmap e61: 9 total, 9 up, 9 in 2024-03-21T11:18:23.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: purged_snaps scrub starts 2024-03-21T11:18:23.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: purged_snaps scrub ok 2024-03-21T11:18:23.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: osd.8 [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917] boot 2024-03-21T11:18:23.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: osdmap e60: 9 total, 9 up, 9 in 2024-03-21T11:18:23.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T11:18:23.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:23.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:23.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:23.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:23.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:18:23.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:23.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:23.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:18:23.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:23.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:23 smithi102 ceph-mon[36748]: osdmap e61: 9 total, 9 up, 9 in 2024-03-21T11:18:24.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:24 smithi047 ceph-mon[31351]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:24.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:24 smithi047 ceph-mon[31351]: Detected new or changed devices on smithi102 2024-03-21T11:18:24.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:24 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi102 to 5693M 2024-03-21T11:18:24.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:24 smithi047 ceph-mon[31351]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T11:18:24.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:24 smithi100 ceph-mon[36245]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:24.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:24 smithi100 ceph-mon[36245]: Detected new or changed devices on smithi102 2024-03-21T11:18:24.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:24 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi102 to 5693M 2024-03-21T11:18:24.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:24 smithi100 ceph-mon[36245]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T11:18:24.977 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi047:vg_nvme/lv_1 2024-03-21T11:18:24.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:24 smithi102 ceph-mon[36748]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:24.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:24 smithi102 ceph-mon[36748]: Detected new or changed devices on smithi102 2024-03-21T11:18:24.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:24 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi102 to 5693M 2024-03-21T11:18:24.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:24 smithi102 ceph-mon[36748]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T11:18:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:18:26.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:26 smithi047 ceph-mon[31351]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:26.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:26 smithi047 ceph-mon[31351]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T11:18:26.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:26 smithi100 ceph-mon[36245]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:26.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:26 smithi100 ceph-mon[36245]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T11:18:26.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:26 smithi102 ceph-mon[36748]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:26 smithi102 ceph-mon[36748]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T11:18:27.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.802+0000 7f3a0e400700 1 -- 172.21.15.47:0/1311977391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a08105610 msgr2=0x7f3a08105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:18:27.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.802+0000 7f3a0e400700 1 --2- 172.21.15.47:0/1311977391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a08105610 0x7f3a08105a70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f39f40099c0 tx=0x7f39f4009cd0 comp rx=0 tx=0).stop 2024-03-21T11:18:27.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.802+0000 7f3a0e400700 1 -- 172.21.15.47:0/1311977391 shutdown_connections 2024-03-21T11:18:27.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.802+0000 7f3a0e400700 1 --2- 172.21.15.47:0/1311977391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a08105fb0 0x7f3a0810c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:18:27.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.802+0000 7f3a0e400700 1 --2- 172.21.15.47:0/1311977391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a08104410 0x7f3a08104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:18:27.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.802+0000 7f3a0e400700 1 --2- 172.21.15.47:0/1311977391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a08105610 0x7f3a08105a70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:18:27.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.802+0000 7f3a0e400700 1 -- 172.21.15.47:0/1311977391 >> 172.21.15.47:0/1311977391 conn(0x7f3a080ff9b0 msgr2=0x7f3a08101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:18:27.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.802+0000 7f3a0e400700 1 -- 172.21.15.47:0/1311977391 shutdown_connections 2024-03-21T11:18:27.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.802+0000 7f3a0e400700 1 -- 172.21.15.47:0/1311977391 wait complete. 2024-03-21T11:18:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.804+0000 7f3a0e400700 1 Processor -- start 2024-03-21T11:18:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.804+0000 7f3a0e400700 1 -- start start 2024-03-21T11:18:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.804+0000 7f3a0e400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a08104410 0x7f3a0819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:18:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.804+0000 7f3a0e400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a08105610 0x7f3a0819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:18:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.804+0000 7f3a0e400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a08105fb0 0x7f3a081a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:18:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.804+0000 7f3a0e400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a08110030 con 0x7f3a08104410 2024-03-21T11:18:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.804+0000 7f3a0e400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0810feb0 con 0x7f3a08105fb0 2024-03-21T11:18:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.804+0000 7f3a0e400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a081101b0 con 0x7f3a08105610 2024-03-21T11:18:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.804+0000 7f3a06a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a08105610 0x7f3a0819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:18:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.804+0000 7f3a07400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a08104410 0x7f3a0819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:18:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.805+0000 7f3a06a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a08105610 0x7f3a0819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:39822/0 (socket says 172.21.15.47:39822) 2024-03-21T11:18:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.805+0000 7f3a06a00700 1 -- 172.21.15.47:0/3710730288 learned_addr learned my addr 172.21.15.47:0/3710730288 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:18:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.805+0000 7f3a07e00700 1 --2- 172.21.15.47:0/3710730288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a08105fb0 0x7f3a081a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:18:27.806 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:27 smithi047 ceph-mon[31351]: osdmap e64: 9 total, 9 up, 9 in 2024-03-21T11:18:27.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.805+0000 7f3a06a00700 1 -- 172.21.15.47:0/3710730288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a08105fb0 msgr2=0x7f3a081a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:18:27.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.805+0000 7f3a06a00700 1 --2- 172.21.15.47:0/3710730288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a08105fb0 0x7f3a081a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:18:27.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.805+0000 7f3a06a00700 1 -- 172.21.15.47:0/3710730288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a08104410 msgr2=0x7f3a0819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:18:27.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.805+0000 7f3a06a00700 1 --2- 172.21.15.47:0/3710730288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a08104410 0x7f3a0819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:18:27.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.805+0000 7f3a06a00700 1 -- 172.21.15.47:0/3710730288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a081a2270 con 0x7f3a08105610 2024-03-21T11:18:27.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.805+0000 7f3a07e00700 1 --2- 172.21.15.47:0/3710730288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a08105fb0 0x7f3a081a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:18:27.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.805+0000 7f3a06a00700 1 --2- 172.21.15.47:0/3710730288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a08105610 0x7f3a0819ab50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f39f400f690 tx=0x7f39f4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:18:27.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.806+0000 7f39ffe00700 1 -- 172.21.15.47:0/3710730288 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4027070 con 0x7f3a08105610 2024-03-21T11:18:27.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.806+0000 7f3a0e400700 1 -- 172.21.15.47:0/3710730288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a081a2500 con 0x7f3a08105610 2024-03-21T11:18:27.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.806+0000 7f39ffe00700 1 -- 172.21.15.47:0/3710730288 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39f400fb30 con 0x7f3a08105610 2024-03-21T11:18:27.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.806+0000 7f39ffe00700 1 -- 172.21.15.47:0/3710730288 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4017650 con 0x7f3a08105610 2024-03-21T11:18:27.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.806+0000 7f3a0e400700 1 -- 172.21.15.47:0/3710730288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a081a29b0 con 0x7f3a08105610 2024-03-21T11:18:27.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.808+0000 7f3a0e400700 1 -- 172.21.15.47:0/3710730288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a08068e50 con 0x7f3a08105610 2024-03-21T11:18:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.810+0000 7f39ffe00700 1 -- 172.21.15.47:0/3710730288 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f39f402a070 con 0x7f3a08105610 2024-03-21T11:18:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.811+0000 7f39ffe00700 1 --2- 172.21.15.47:0/3710730288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39ec0a3030 0x7f39ec0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:18:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.811+0000 7f39ffe00700 1 -- 172.21.15.47:0/3710730288 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f39f40cf970 con 0x7f3a08105610 2024-03-21T11:18:27.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.811+0000 7f3a07400700 1 --2- 172.21.15.47:0/3710730288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39ec0a3030 0x7f39ec0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:18:27.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.812+0000 7f3a07400700 1 --2- 172.21.15.47:0/3710730288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39ec0a3030 0x7f39ec0a54f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a08105470 tx=0x7f39f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:18:27.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:27.815+0000 7f39ffe00700 1 -- 172.21.15.47:0/3710730288 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f39f405e740 con 0x7f3a08105610 2024-03-21T11:18:27.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:27 smithi100 ceph-mon[36245]: osdmap e64: 9 total, 9 up, 9 in 2024-03-21T11:18:27.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:27 smithi102 ceph-mon[36748]: osdmap e64: 9 total, 9 up, 9 in 2024-03-21T11:18:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:28.056+0000 7f3a0e400700 1 -- 172.21.15.47:0/3710730288 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a080625d0 con 0x7f39ec0a3030 2024-03-21T11:18:28.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:28 smithi047 ceph-mon[31351]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:28.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:28 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:18:28.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:28 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:18:28.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:28 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:28.892 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:28 smithi102 ceph-mon[36748]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:28.892 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:28 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:18:28.892 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:28 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:18:28.892 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:28 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:28.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:28 smithi100 ceph-mon[36245]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:28.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:28 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:18:28.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:28 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:18:28.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:28 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:29.872 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:29 smithi047 ceph-mon[31351]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:18:29.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:29 smithi100 ceph-mon[36245]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:18:29.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:29 smithi102 ceph-mon[36748]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:18:30.820 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:30 smithi047 ceph-mon[31351]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-03-21T11:18:30.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:30 smithi100 ceph-mon[36245]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-03-21T11:18:30.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:30 smithi102 ceph-mon[36748]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-03-21T11:18:32.751 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:32 smithi047 ceph-mon[31351]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-21T11:18:32.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:32 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1808062140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f22490f-23d7-4822-b6f6-57f2da5e0514"}]: dispatch 2024-03-21T11:18:32.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:32 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f22490f-23d7-4822-b6f6-57f2da5e0514"}]: dispatch 2024-03-21T11:18:32.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:32 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f22490f-23d7-4822-b6f6-57f2da5e0514"}]': finished 2024-03-21T11:18:32.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:32 smithi047 ceph-mon[31351]: osdmap e65: 10 total, 9 up, 10 in 2024-03-21T11:18:32.752 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:32 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:32.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:32 smithi100 ceph-mon[36245]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-21T11:18:32.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:32 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1808062140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f22490f-23d7-4822-b6f6-57f2da5e0514"}]: dispatch 2024-03-21T11:18:32.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:32 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f22490f-23d7-4822-b6f6-57f2da5e0514"}]: dispatch 2024-03-21T11:18:32.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:32 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f22490f-23d7-4822-b6f6-57f2da5e0514"}]': finished 2024-03-21T11:18:32.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:32 smithi100 ceph-mon[36245]: osdmap e65: 10 total, 9 up, 10 in 2024-03-21T11:18:32.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:32 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:32.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:32 smithi102 ceph-mon[36748]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-21T11:18:32.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:32 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1808062140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f22490f-23d7-4822-b6f6-57f2da5e0514"}]: dispatch 2024-03-21T11:18:32.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:32 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f22490f-23d7-4822-b6f6-57f2da5e0514"}]: dispatch 2024-03-21T11:18:32.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:32 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f22490f-23d7-4822-b6f6-57f2da5e0514"}]': finished 2024-03-21T11:18:32.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:32 smithi102 ceph-mon[36748]: osdmap e65: 10 total, 9 up, 10 in 2024-03-21T11:18:32.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:32 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:33.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:33 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3248571479' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:18:33.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:33 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3248571479' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:18:34.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:33 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3248571479' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:18:34.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:34 smithi100 ceph-mon[36245]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-21T11:18:34.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:34 smithi102 ceph-mon[36748]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-21T11:18:35.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:34 smithi047 ceph-mon[31351]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-21T11:18:36.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:36 smithi100 ceph-mon[36245]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-21T11:18:36.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:36 smithi102 ceph-mon[36748]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-21T11:18:37.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:36 smithi047 ceph-mon[31351]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-21T11:18:38.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:38 smithi100 ceph-mon[36245]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T11:18:38.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:38 smithi047 ceph-mon[31351]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T11:18:38.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:38 smithi102 ceph-mon[36748]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T11:18:40.886 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:40 smithi047 ceph-mon[31351]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:40.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:40 smithi100 ceph-mon[36245]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:40.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:40 smithi102 ceph-mon[36748]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:42.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:42 smithi047 ceph-mon[31351]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:42.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T11:18:42.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:42.861 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:42 smithi047 ceph-mon[31351]: Deploying daemon osd.9 on smithi047 2024-03-21T11:18:42.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:42 smithi100 ceph-mon[36245]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:42.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T11:18:42.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:42.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:42 smithi100 ceph-mon[36245]: Deploying daemon osd.9 on smithi047 2024-03-21T11:18:42.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:42 smithi102 ceph-mon[36748]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T11:18:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:18:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:42 smithi102 ceph-mon[36748]: Deploying daemon osd.9 on smithi047 2024-03-21T11:18:44.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:44 smithi100 ceph-mon[36245]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:44.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:44 smithi102 ceph-mon[36748]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:45.013 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:44 smithi047 ceph-mon[31351]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:46.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:46 smithi100 ceph-mon[36245]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:46.912 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:46 smithi047 ceph-mon[31351]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:46.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:46 smithi102 ceph-mon[36748]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:48.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:48 smithi047 ceph-mon[31351]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:48.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:48 smithi100 ceph-mon[36245]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:48.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:48 smithi102 ceph-mon[36748]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:50.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:50 smithi100 ceph-mon[36245]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:50.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:50.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:50.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:18:50.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:50 smithi102 ceph-mon[36748]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:18:51.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:50 smithi047 ceph-mon[31351]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:51.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:51.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:51.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:18:52.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:52 smithi100 ceph-mon[36245]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:52.910 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:52 smithi047 ceph-mon[31351]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:52.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:52 smithi102 ceph-mon[36748]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:54.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:54 smithi047 ceph-mon[31351]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:54.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:54 smithi047 ceph-mon[31351]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T11:18:54.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:54 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:54.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:54 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:54.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:54 smithi100 ceph-mon[36245]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:54.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:54 smithi100 ceph-mon[36245]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T11:18:54.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:54 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:54.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:54 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:54.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:54 smithi102 ceph-mon[36748]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:54.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:54 smithi102 ceph-mon[36748]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T11:18:54.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:54 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:54.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:54 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:55.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:55.356+0000 7f39ffe00700 1 -- 172.21.15.47:0/3710730288 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3a080625d0 con 0x7f39ec0a3030 2024-03-21T11:18:55.357 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 9 on host 'smithi047' 2024-03-21T11:18:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:55.360+0000 7f3a0e400700 1 -- 172.21.15.47:0/3710730288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39ec0a3030 msgr2=0x7f39ec0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:18:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:55.360+0000 7f3a0e400700 1 --2- 172.21.15.47:0/3710730288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39ec0a3030 0x7f39ec0a54f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a08105470 tx=0x7f39f8009260 comp rx=0 tx=0).stop 2024-03-21T11:18:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:55.360+0000 7f3a0e400700 1 -- 172.21.15.47:0/3710730288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a08105610 msgr2=0x7f3a0819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:18:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:55.360+0000 7f3a0e400700 1 --2- 172.21.15.47:0/3710730288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a08105610 0x7f3a0819ab50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f39f400f690 tx=0x7f39f4009580 comp rx=0 tx=0).stop 2024-03-21T11:18:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:55.360+0000 7f3a0e400700 1 -- 172.21.15.47:0/3710730288 shutdown_connections 2024-03-21T11:18:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:55.360+0000 7f3a0e400700 1 --2- 172.21.15.47:0/3710730288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a08105fb0 0x7f3a081a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:18:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:55.360+0000 7f3a0e400700 1 --2- 172.21.15.47:0/3710730288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39ec0a3030 0x7f39ec0a54f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:18:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:55.360+0000 7f3a0e400700 1 --2- 172.21.15.47:0/3710730288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a08104410 0x7f3a0819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:18:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:55.360+0000 7f3a0e400700 1 --2- 172.21.15.47:0/3710730288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a08105610 0x7f3a0819ab50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:18:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:55.360+0000 7f3a0e400700 1 -- 172.21.15.47:0/3710730288 >> 172.21.15.47:0/3710730288 conn(0x7f3a080ff9b0 msgr2=0x7f3a081088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:18:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:55.360+0000 7f3a0e400700 1 -- 172.21.15.47:0/3710730288 shutdown_connections 2024-03-21T11:18:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:18:55.360+0000 7f3a0e400700 1 -- 172.21.15.47:0/3710730288 wait complete. 2024-03-21T11:18:55.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:55 smithi100 ceph-mon[36245]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T11:18:55.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:55 smithi100 ceph-mon[36245]: osdmap e66: 10 total, 9 up, 10 in 2024-03-21T11:18:55.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:55 smithi100 ceph-mon[36245]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:18:55.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:55 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:55.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:55 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:55.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:55 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:55.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:55 smithi100 ceph-mon[36245]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T11:18:55.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:55 smithi100 ceph-mon[36245]: osdmap e67: 10 total, 9 up, 10 in 2024-03-21T11:18:55.902 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:55 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:55.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:55 smithi102 ceph-mon[36748]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T11:18:55.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:55 smithi102 ceph-mon[36748]: osdmap e66: 10 total, 9 up, 10 in 2024-03-21T11:18:55.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:55 smithi102 ceph-mon[36748]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:18:55.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:55 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:55.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:55 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:55.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:55 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:55.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:55 smithi102 ceph-mon[36748]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T11:18:55.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:55 smithi102 ceph-mon[36748]: osdmap e67: 10 total, 9 up, 10 in 2024-03-21T11:18:55.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:55 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:56.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:55 smithi047 ceph-mon[31351]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T11:18:56.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:55 smithi047 ceph-mon[31351]: osdmap e66: 10 total, 9 up, 10 in 2024-03-21T11:18:56.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:55 smithi047 ceph-mon[31351]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T11:18:56.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:55 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:56.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:55 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:56.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:55 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:18:56.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:55 smithi047 ceph-mon[31351]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T11:18:56.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:55 smithi047 ceph-mon[31351]: osdmap e67: 10 total, 9 up, 10 in 2024-03-21T11:18:56.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:55 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:56.152 DEBUG:teuthology.orchestra.run.smithi047:osd.9> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.9.service 2024-03-21T11:18:56.156 INFO:tasks.cephadm:Deploying osd.10 on smithi100 with /dev/vg_nvme/lv_1... 2024-03-21T11:18:56.157 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T11:18:56.402 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T11:18:56.656 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:18:56 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:18:56.374+0000 7f53dbe00700 -1 osd.9 0 waiting for initial osdmap 2024-03-21T11:18:56.656 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:18:56 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:18:56.405+0000 7f53d1c00700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T11:18:56.776 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:56 smithi100 ceph-mon[36245]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:56.776 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:56 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:56.776 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:56 smithi100 ceph-mon[36245]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' 2024-03-21T11:18:56.777 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:56 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:56.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:56 smithi102 ceph-mon[36748]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:56.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:56.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:56 smithi102 ceph-mon[36748]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' 2024-03-21T11:18:56.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:56 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:56 smithi047 ceph-mon[31351]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:56 smithi047 ceph-mon[31351]: from='osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812]' entity='osd.9' 2024-03-21T11:18:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:56 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:57.827 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:57 smithi100 ceph-mon[36245]: purged_snaps scrub starts 2024-03-21T11:18:57.828 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:57 smithi100 ceph-mon[36245]: purged_snaps scrub ok 2024-03-21T11:18:57.828 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:57 smithi100 ceph-mon[36245]: osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812] boot 2024-03-21T11:18:57.828 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:57 smithi100 ceph-mon[36245]: osdmap e68: 10 total, 10 up, 10 in 2024-03-21T11:18:57.828 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:57 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:57.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:57 smithi102 ceph-mon[36748]: purged_snaps scrub starts 2024-03-21T11:18:57.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:57 smithi102 ceph-mon[36748]: purged_snaps scrub ok 2024-03-21T11:18:57.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:57 smithi102 ceph-mon[36748]: osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812] boot 2024-03-21T11:18:57.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:57 smithi102 ceph-mon[36748]: osdmap e68: 10 total, 10 up, 10 in 2024-03-21T11:18:57.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:57 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:58.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:57 smithi047 ceph-mon[31351]: purged_snaps scrub starts 2024-03-21T11:18:58.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:57 smithi047 ceph-mon[31351]: purged_snaps scrub ok 2024-03-21T11:18:58.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:57 smithi047 ceph-mon[31351]: osd.9 [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812] boot 2024-03-21T11:18:58.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:57 smithi047 ceph-mon[31351]: osdmap e68: 10 total, 10 up, 10 in 2024-03-21T11:18:58.017 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:57 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T11:18:58.554 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T11:18:58.554 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T11:18:58.554 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-21T11:18:58.554 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-21T11:18:58.554 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011226 s, 934 MB/s 2024-03-21T11:18:58.554 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-21T11:18:58.929 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:58 smithi047 ceph-mon[31351]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:58.960 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:58 smithi100 ceph-mon[36245]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:58.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:58 smithi102 ceph-mon[36748]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-21T11:18:59.288 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_1 2024-03-21T11:18:59.527 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T11:18:59.915 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:18:59 smithi100 ceph-mon[36245]: osdmap e69: 10 total, 10 up, 10 in 2024-03-21T11:18:59.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:18:59 smithi102 ceph-mon[36748]: osdmap e69: 10 total, 10 up, 10 in 2024-03-21T11:19:00.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:18:59 smithi047 ceph-mon[31351]: osdmap e69: 10 total, 10 up, 10 in 2024-03-21T11:19:00.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:00 smithi100 ceph-mon[36245]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:00.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:00 smithi102 ceph-mon[36748]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:01.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:00 smithi047 ceph-mon[31351]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.200+0000 7f95d33a6700 1 -- 172.21.15.100:0/683754118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95cc1021c0 msgr2=0x7f95cc1025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.200+0000 7f95d33a6700 1 --2- 172.21.15.100:0/683754118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95cc1021c0 0x7f95cc1025e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f95c80099c0 tx=0x7f95c8009cd0 comp rx=0 tx=0).stop 2024-03-21T11:19:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.200+0000 7f95d33a6700 1 -- 172.21.15.100:0/683754118 shutdown_connections 2024-03-21T11:19:01.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.200+0000 7f95d33a6700 1 --2- 172.21.15.100:0/683754118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95cc103d60 0x7f95cc10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.200+0000 7f95d33a6700 1 --2- 172.21.15.100:0/683754118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95cc1021c0 0x7f95cc1025e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.200+0000 7f95d33a6700 1 --2- 172.21.15.100:0/683754118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95cc1033c0 0x7f95cc103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.200+0000 7f95d33a6700 1 -- 172.21.15.100:0/683754118 >> 172.21.15.100:0/683754118 conn(0x7f95cc0fd7a0 msgr2=0x7f95cc0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:19:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.200+0000 7f95d33a6700 1 -- 172.21.15.100:0/683754118 shutdown_connections 2024-03-21T11:19:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.200+0000 7f95d33a6700 1 -- 172.21.15.100:0/683754118 wait complete. 2024-03-21T11:19:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.201+0000 7f95d33a6700 1 Processor -- start 2024-03-21T11:19:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.201+0000 7f95d33a6700 1 -- start start 2024-03-21T11:19:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.201+0000 7f95d33a6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95cc1021c0 0x7f95cc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.201+0000 7f95d33a6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95cc1033c0 0x7f95cc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.201+0000 7f95d33a6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95cc103d60 0x7f95cc19fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.201+0000 7f95d33a6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc10de00 con 0x7f95cc1021c0 2024-03-21T11:19:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.201+0000 7f95d33a6700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc10dc80 con 0x7f95cc103d60 2024-03-21T11:19:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.201+0000 7f95d33a6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc10df80 con 0x7f95cc1033c0 2024-03-21T11:19:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.201+0000 7f95d1943700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95cc103d60 0x7f95cc19fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.201+0000 7f95d1943700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95cc103d60 0x7f95cc19fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36250/0 (socket says 172.21.15.100:36250) 2024-03-21T11:19:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.201+0000 7f95d1943700 1 -- 172.21.15.100:0/2313951321 learned_addr learned my addr 172.21.15.100:0/2313951321 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-21T11:19:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.202+0000 7f95d0941700 1 --2- 172.21.15.100:0/2313951321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95cc1033c0 0x7f95cc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.202+0000 7f95d1943700 1 -- 172.21.15.100:0/2313951321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95cc1033c0 msgr2=0x7f95cc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.202+0000 7f95d1943700 1 --2- 172.21.15.100:0/2313951321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95cc1033c0 0x7f95cc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.202+0000 7f95d1943700 1 -- 172.21.15.100:0/2313951321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95cc1021c0 msgr2=0x7f95cc198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.202+0000 7f95d1943700 1 --2- 172.21.15.100:0/2313951321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95cc1021c0 0x7f95cc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.202+0000 7f95d1943700 1 -- 172.21.15.100:0/2313951321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc1a00d0 con 0x7f95cc103d60 2024-03-21T11:19:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.202+0000 7f95d1943700 1 --2- 172.21.15.100:0/2313951321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95cc103d60 0x7f95cc19fa30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f95c4007ee0 tx=0x7f95c400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:19:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.202+0000 7f95c27fc700 1 -- 172.21.15.100:0/2313951321 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c400dbd0 con 0x7f95cc103d60 2024-03-21T11:19:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.202+0000 7f95d33a6700 1 -- 172.21.15.100:0/2313951321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc1a03c0 con 0x7f95cc103d60 2024-03-21T11:19:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.202+0000 7f95c27fc700 1 -- 172.21.15.100:0/2313951321 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95c400dd70 con 0x7f95cc103d60 2024-03-21T11:19:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.202+0000 7f95c27fc700 1 -- 172.21.15.100:0/2313951321 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4005a40 con 0x7f95cc103d60 2024-03-21T11:19:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.202+0000 7f95d33a6700 1 -- 172.21.15.100:0/2313951321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc1a0930 con 0x7f95cc103d60 2024-03-21T11:19:01.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.203+0000 7f95d33a6700 1 -- 172.21.15.100:0/2313951321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc068e50 con 0x7f95cc103d60 2024-03-21T11:19:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.205+0000 7f95c27fc700 1 -- 172.21.15.100:0/2313951321 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f95c401a020 con 0x7f95cc103d60 2024-03-21T11:19:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.205+0000 7f95c27fc700 1 --2- 172.21.15.100:0/2313951321 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95b80a2fb0 0x7f95b80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.206+0000 7f95c27fc700 1 -- 172.21.15.100:0/2313951321 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f95c40ce960 con 0x7f95cc103d60 2024-03-21T11:19:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.206+0000 7f95d1142700 1 --2- 172.21.15.100:0/2313951321 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95b80a2fb0 0x7f95b80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:01.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.207+0000 7f95d1142700 1 --2- 172.21.15.100:0/2313951321 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95b80a2fb0 0x7f95b80a5470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95c8003890 tx=0x7f95c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:19:01.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.209+0000 7f95c27fc700 1 -- 172.21.15.100:0/2313951321 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f95c4010070 con 0x7f95cc103d60 2024-03-21T11:19:01.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:01.412+0000 7f95d33a6700 1 -- 172.21.15.100:0/2313951321 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f95cc062690 con 0x7f95b80a2fb0 2024-03-21T11:19:01.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:01 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:19:01.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:01 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:19:01.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:01 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:02.050 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:01 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:19:02.051 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:01 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:19:02.051 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:01 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:02.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:01 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:19:02.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:01 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:19:02.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:01 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:03.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:03.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:19:03.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: Detected new or changed devices on smithi047 2024-03-21T11:19:03.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:03.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:03.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:03.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:03.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:03.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:03.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi047 to 4273M 2024-03-21T11:19:03.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:03.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:03.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:03.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:03.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:03.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:03.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:03.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:03.077 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:03.077 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:19:03.078 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: Detected new or changed devices on smithi047 2024-03-21T11:19:03.078 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:03.078 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:03.078 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:03.078 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:03.078 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:03.078 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:03.078 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi047 to 4273M 2024-03-21T11:19:03.078 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:03.078 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:03.079 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:03.079 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:03.079 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:03.079 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:03.079 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:03.079 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:19:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: Detected new or changed devices on smithi047 2024-03-21T11:19:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:03.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:03.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:03.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi047 to 4273M 2024-03-21T11:19:03.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:03.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:03.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:03.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:03.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:03.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:03.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:03.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:04 smithi047 ceph-mon[31351]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:04 smithi047 ceph-mon[31351]: from='client.? 172.21.15.100:0/1434352467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22ad02d6-55e9-4ec3-96e1-b24d6d1222f5"}]: dispatch 2024-03-21T11:19:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:04 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22ad02d6-55e9-4ec3-96e1-b24d6d1222f5"}]: dispatch 2024-03-21T11:19:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:04 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22ad02d6-55e9-4ec3-96e1-b24d6d1222f5"}]': finished 2024-03-21T11:19:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:04 smithi047 ceph-mon[31351]: osdmap e70: 11 total, 10 up, 11 in 2024-03-21T11:19:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:04 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:05.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:04 smithi100 ceph-mon[36245]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:05.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:04 smithi100 ceph-mon[36245]: from='client.? 172.21.15.100:0/1434352467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22ad02d6-55e9-4ec3-96e1-b24d6d1222f5"}]: dispatch 2024-03-21T11:19:05.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:04 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22ad02d6-55e9-4ec3-96e1-b24d6d1222f5"}]: dispatch 2024-03-21T11:19:05.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:04 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22ad02d6-55e9-4ec3-96e1-b24d6d1222f5"}]': finished 2024-03-21T11:19:05.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:04 smithi100 ceph-mon[36245]: osdmap e70: 11 total, 10 up, 11 in 2024-03-21T11:19:05.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:04 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:04 smithi102 ceph-mon[36748]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:04 smithi102 ceph-mon[36748]: from='client.? 172.21.15.100:0/1434352467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22ad02d6-55e9-4ec3-96e1-b24d6d1222f5"}]: dispatch 2024-03-21T11:19:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:04 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22ad02d6-55e9-4ec3-96e1-b24d6d1222f5"}]: dispatch 2024-03-21T11:19:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:04 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22ad02d6-55e9-4ec3-96e1-b24d6d1222f5"}]': finished 2024-03-21T11:19:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:04 smithi102 ceph-mon[36748]: osdmap e70: 11 total, 10 up, 11 in 2024-03-21T11:19:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:04 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:06.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:05 smithi047 ceph-mon[31351]: from='client.? 172.21.15.100:0/3559155817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:19:06.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:05 smithi100 ceph-mon[36245]: from='client.? 172.21.15.100:0/3559155817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:19:06.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:05 smithi102 ceph-mon[36748]: from='client.? 172.21.15.100:0/3559155817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:19:07.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:06 smithi047 ceph-mon[31351]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:07.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:06 smithi100 ceph-mon[36245]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:07.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:06 smithi102 ceph-mon[36748]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:09.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:08 smithi047 ceph-mon[31351]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:09.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:08 smithi100 ceph-mon[36245]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:08 smithi102 ceph-mon[36748]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:10.991 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:10 smithi100 ceph-mon[36245]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:11.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:10 smithi047 ceph-mon[31351]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:11.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:10 smithi102 ceph-mon[36748]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:12.968 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:12 smithi100 ceph-mon[36245]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:12.970 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T11:19:12.970 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:12 smithi102 ceph-mon[36748]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T11:19:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:12 smithi047 ceph-mon[31351]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T11:19:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:13.972 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:13 smithi100 ceph-mon[36245]: Deploying daemon osd.10 on smithi100 2024-03-21T11:19:14.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:13 smithi102 ceph-mon[36748]: Deploying daemon osd.10 on smithi100 2024-03-21T11:19:14.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:13 smithi047 ceph-mon[31351]: Deploying daemon osd.10 on smithi100 2024-03-21T11:19:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:14 smithi100 ceph-mon[36245]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:15.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:14 smithi102 ceph-mon[36748]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:15.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:14 smithi047 ceph-mon[31351]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:17.080 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:16 smithi100 ceph-mon[36245]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:17.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:16 smithi102 ceph-mon[36748]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:17.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:16 smithi047 ceph-mon[31351]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:18.970 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:18 smithi100 ceph-mon[36245]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:18.970 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:18 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:18.970 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:18 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:18.971 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:18 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:18 smithi102 ceph-mon[36748]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:18 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:18 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:18 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:18 smithi047 ceph-mon[31351]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:18 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:18 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:18 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:21.117 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:20 smithi047 ceph-mon[31351]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:21.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:20 smithi100 ceph-mon[36245]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:21.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:20 smithi102 ceph-mon[36748]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:21.307 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 10 on host 'smithi100' 2024-03-21T11:19:21.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:21.307+0000 7f95c27fc700 1 -- 172.21.15.100:0/2313951321 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f95cc062690 con 0x7f95b80a2fb0 2024-03-21T11:19:21.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:21.309+0000 7f95d33a6700 1 -- 172.21.15.100:0/2313951321 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95b80a2fb0 msgr2=0x7f95b80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:21.309+0000 7f95d33a6700 1 --2- 172.21.15.100:0/2313951321 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95b80a2fb0 0x7f95b80a5470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95c8003890 tx=0x7f95c8000bc0 comp rx=0 tx=0).stop 2024-03-21T11:19:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:21.309+0000 7f95d33a6700 1 -- 172.21.15.100:0/2313951321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95cc103d60 msgr2=0x7f95cc19fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:21.309+0000 7f95d33a6700 1 --2- 172.21.15.100:0/2313951321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95cc103d60 0x7f95cc19fa30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f95c4007ee0 tx=0x7f95c400d3b0 comp rx=0 tx=0).stop 2024-03-21T11:19:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:21.309+0000 7f95d33a6700 1 -- 172.21.15.100:0/2313951321 shutdown_connections 2024-03-21T11:19:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:21.309+0000 7f95d33a6700 1 --2- 172.21.15.100:0/2313951321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95cc103d60 0x7f95cc19fa30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:21.309+0000 7f95d33a6700 1 --2- 172.21.15.100:0/2313951321 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95b80a2fb0 0x7f95b80a5470 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:21.309+0000 7f95d33a6700 1 --2- 172.21.15.100:0/2313951321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95cc1021c0 0x7f95cc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:21.309+0000 7f95d33a6700 1 --2- 172.21.15.100:0/2313951321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95cc1033c0 0x7f95cc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:21.309+0000 7f95d33a6700 1 -- 172.21.15.100:0/2313951321 >> 172.21.15.100:0/2313951321 conn(0x7f95cc0fd7a0 msgr2=0x7f95cc1066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:19:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:21.309+0000 7f95d33a6700 1 -- 172.21.15.100:0/2313951321 shutdown_connections 2024-03-21T11:19:21.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T11:19:21.309+0000 7f95d33a6700 1 -- 172.21.15.100:0/2313951321 wait complete. 2024-03-21T11:19:22.095 DEBUG:teuthology.orchestra.run.smithi100:osd.10> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.10.service 2024-03-21T11:19:22.097 INFO:tasks.cephadm:Deploying osd.11 on smithi102 with /dev/vg_nvme/lv_1... 2024-03-21T11:19:22.098 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T11:19:22.350 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:19:22.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:22 smithi100 ceph-mon[36245]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:22.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:22.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:22.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:22 smithi100 ceph-mon[36245]: from='osd.10 [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T11:19:22.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:22 smithi100 ceph-mon[36245]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T11:19:22.471 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:22 smithi102 ceph-mon[36748]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:22.472 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:22.472 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:22.472 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:22 smithi102 ceph-mon[36748]: from='osd.10 [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T11:19:22.472 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:22 smithi102 ceph-mon[36748]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T11:19:22.507 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:22 smithi047 ceph-mon[31351]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:22.507 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:22.507 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:22.508 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:22 smithi047 ceph-mon[31351]: from='osd.10 [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T11:19:22.508 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:22 smithi047 ceph-mon[31351]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T11:19:23.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:23 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:23 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:23 smithi100 ceph-mon[36245]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T11:19:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:23 smithi100 ceph-mon[36245]: from='osd.10 [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:19:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:23 smithi100 ceph-mon[36245]: osdmap e71: 11 total, 10 up, 11 in 2024-03-21T11:19:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:23 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:23.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:23 smithi100 ceph-mon[36245]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:19:23.903 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:23 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:23.903 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:23 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:23.903 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:23 smithi102 ceph-mon[36748]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T11:19:23.903 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:23 smithi102 ceph-mon[36748]: from='osd.10 [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:19:23.904 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:23 smithi102 ceph-mon[36748]: osdmap e71: 11 total, 10 up, 11 in 2024-03-21T11:19:23.904 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:23 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:23.904 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:23 smithi102 ceph-mon[36748]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:19:24.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:24.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:24.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:23 smithi047 ceph-mon[31351]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T11:19:24.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:23 smithi047 ceph-mon[31351]: from='osd.10 [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:19:24.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:23 smithi047 ceph-mon[31351]: osdmap e71: 11 total, 10 up, 11 in 2024-03-21T11:19:24.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:24.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:23 smithi047 ceph-mon[31351]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T11:19:24.497 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T11:19:24.497 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T11:19:24.497 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-03-21T11:19:24.497 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-03-21T11:19:24.498 INFO:teuthology.orchestra.run.smithi102.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0149907 s, 699 MB/s 2024-03-21T11:19:24.498 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-03-21T11:19:24.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:19:24 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:19:24.269+0000 7fc119eaa700 -1 osd.10 0 waiting for initial osdmap 2024-03-21T11:19:24.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:19:24 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:19:24.283+0000 7fc111c95700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T11:19:24.952 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:24 smithi100 ceph-mon[36245]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:24.953 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:24 smithi100 ceph-mon[36245]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T11:19:24.953 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:24 smithi100 ceph-mon[36245]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T11:19:24.953 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:24 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:24.953 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:24 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:24.953 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:24 smithi100 ceph-mon[36245]: from='osd.10 ' entity='osd.10' 2024-03-21T11:19:25.216 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:24 smithi102 ceph-mon[36748]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:25.216 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:24 smithi102 ceph-mon[36748]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T11:19:25.216 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:24 smithi102 ceph-mon[36748]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T11:19:25.216 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:24 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:25.216 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:24 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:25.216 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:24 smithi102 ceph-mon[36748]: from='osd.10 ' entity='osd.10' 2024-03-21T11:19:25.243 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi102:vg_nvme/lv_1 2024-03-21T11:19:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:24 smithi047 ceph-mon[31351]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:24 smithi047 ceph-mon[31351]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T11:19:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:24 smithi047 ceph-mon[31351]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T11:19:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:24 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:24 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:24 smithi047 ceph-mon[31351]: from='osd.10 ' entity='osd.10' 2024-03-21T11:19:25.483 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:19:26.121 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:25 smithi100 ceph-mon[36245]: purged_snaps scrub starts 2024-03-21T11:19:26.121 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:25 smithi100 ceph-mon[36245]: purged_snaps scrub ok 2024-03-21T11:19:26.121 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:25 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:26.121 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:25 smithi100 ceph-mon[36245]: osd.10 [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291] boot 2024-03-21T11:19:26.121 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:25 smithi100 ceph-mon[36245]: osdmap e73: 11 total, 11 up, 11 in 2024-03-21T11:19:26.121 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:25 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:26.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:25 smithi102 ceph-mon[36748]: purged_snaps scrub starts 2024-03-21T11:19:26.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:25 smithi102 ceph-mon[36748]: purged_snaps scrub ok 2024-03-21T11:19:26.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:25 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:26.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:25 smithi102 ceph-mon[36748]: osd.10 [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291] boot 2024-03-21T11:19:26.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:25 smithi102 ceph-mon[36748]: osdmap e73: 11 total, 11 up, 11 in 2024-03-21T11:19:26.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:25 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:26.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:25 smithi047 ceph-mon[31351]: purged_snaps scrub starts 2024-03-21T11:19:26.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:25 smithi047 ceph-mon[31351]: purged_snaps scrub ok 2024-03-21T11:19:26.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:25 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:26.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:25 smithi047 ceph-mon[31351]: osd.10 [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291] boot 2024-03-21T11:19:26.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:25 smithi047 ceph-mon[31351]: osdmap e73: 11 total, 11 up, 11 in 2024-03-21T11:19:26.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:25 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T11:19:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:26 smithi102 ceph-mon[36748]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:26 smithi102 ceph-mon[36748]: osdmap e74: 11 total, 11 up, 11 in 2024-03-21T11:19:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:26.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:26.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:27.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.063+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/3330821010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a58103e50 msgr2=0x7f1a5810a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:27.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.063+0000 7f1a5dcb5700 1 --2- 172.21.15.102:0/3330821010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a58103e50 0x7f1a5810a700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000b0a0 tx=0x7f1a5000b3b0 comp rx=0 tx=0).stop 2024-03-21T11:19:27.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.063+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/3330821010 shutdown_connections 2024-03-21T11:19:27.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.063+0000 7f1a5dcb5700 1 --2- 172.21.15.102:0/3330821010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a58103e50 0x7f1a5810a700 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:27.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.063+0000 7f1a5dcb5700 1 --2- 172.21.15.102:0/3330821010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a581004d0 0x7f1a581008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:27.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.063+0000 7f1a5dcb5700 1 --2- 172.21.15.102:0/3330821010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a581034b0 0x7f1a58103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:27.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.063+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/3330821010 >> 172.21.15.102:0/3330821010 conn(0x7f1a580fbe30 msgr2=0x7f1a580fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:19:27.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.063+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/3330821010 shutdown_connections 2024-03-21T11:19:27.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.063+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/3330821010 wait complete. 2024-03-21T11:19:27.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.064+0000 7f1a5dcb5700 1 Processor -- start 2024-03-21T11:19:27.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.064+0000 7f1a5dcb5700 1 -- start start 2024-03-21T11:19:27.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.064+0000 7f1a5dcb5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a581004d0 0x7f1a58102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:27.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.064+0000 7f1a5dcb5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a581034b0 0x7f1a58101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:27.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.064+0000 7f1a5dcb5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a58103e50 0x7f1a581019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:27.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.064+0000 7f1a5dcb5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5810de70 con 0x7f1a581004d0 2024-03-21T11:19:27.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.064+0000 7f1a5dcb5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5810dcf0 con 0x7f1a581034b0 2024-03-21T11:19:27.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.064+0000 7f1a5dcb5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5810dff0 con 0x7f1a58103e50 2024-03-21T11:19:27.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.064+0000 7f1a57fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a58103e50 0x7f1a581019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:27.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.064+0000 7f1a57fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a58103e50 0x7f1a581019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42428/0 (socket says 172.21.15.102:42428) 2024-03-21T11:19:27.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.064+0000 7f1a57fff700 1 -- 172.21.15.102:0/886083338 learned_addr learned my addr 172.21.15.102:0/886083338 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-21T11:19:27.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.064+0000 7f1a56ffd700 1 --2- 172.21.15.102:0/886083338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a581034b0 0x7f1a58101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:27.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.065+0000 7f1a57fff700 1 -- 172.21.15.102:0/886083338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a581034b0 msgr2=0x7f1a58101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:27.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.065+0000 7f1a57fff700 1 --2- 172.21.15.102:0/886083338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a581034b0 0x7f1a58101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:27.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.065+0000 7f1a57fff700 1 -- 172.21.15.102:0/886083338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a581004d0 msgr2=0x7f1a58102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:27.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.065+0000 7f1a57fff700 1 --2- 172.21.15.102:0/886083338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a581004d0 0x7f1a58102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:27.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.065+0000 7f1a57fff700 1 -- 172.21.15.102:0/886083338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a58102250 con 0x7f1a58103e50 2024-03-21T11:19:27.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.065+0000 7f1a56ffd700 1 --2- 172.21.15.102:0/886083338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a581034b0 0x7f1a58101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:19:27.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.065+0000 7f1a57fff700 1 --2- 172.21.15.102:0/886083338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a58103e50 0x7f1a581019a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a50007f00 tx=0x7f1a50004e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:19:27.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.065+0000 7f1a54ff9700 1 -- 172.21.15.102:0/886083338 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50021070 con 0x7f1a58103e50 2024-03-21T11:19:27.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.065+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/886083338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a581a24f0 con 0x7f1a58103e50 2024-03-21T11:19:27.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.065+0000 7f1a54ff9700 1 -- 172.21.15.102:0/886083338 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a50004020 con 0x7f1a58103e50 2024-03-21T11:19:27.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.065+0000 7f1a54ff9700 1 -- 172.21.15.102:0/886083338 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5001d6d0 con 0x7f1a58103e50 2024-03-21T11:19:27.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.065+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/886083338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a581a29a0 con 0x7f1a58103e50 2024-03-21T11:19:27.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.066+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/886083338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a58068e50 con 0x7f1a58103e50 2024-03-21T11:19:27.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.067+0000 7f1a54ff9700 1 -- 172.21.15.102:0/886083338 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1a500041d0 con 0x7f1a58103e50 2024-03-21T11:19:27.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.067+0000 7f1a54ff9700 1 --2- 172.21.15.102:0/886083338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a400a2fb0 0x7f1a400a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:27.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.068+0000 7f1a54ff9700 1 -- 172.21.15.102:0/886083338 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f1a5000f020 con 0x7f1a58103e50 2024-03-21T11:19:27.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.068+0000 7f1a577fe700 1 --2- 172.21.15.102:0/886083338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a400a2fb0 0x7f1a400a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:27.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.069+0000 7f1a577fe700 1 --2- 172.21.15.102:0/886083338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a400a2fb0 0x7f1a400a5470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1a5806a670 tx=0x7f1a48009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:19:27.077 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.071+0000 7f1a54ff9700 1 -- 172.21.15.102:0/886083338 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1a5002a020 con 0x7f1a58103e50 2024-03-21T11:19:27.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:26 smithi100 ceph-mon[36245]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:26 smithi100 ceph-mon[36245]: osdmap e74: 11 total, 11 up, 11 in 2024-03-21T11:19:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:27.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:27.281+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/886083338 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a580625d0 con 0x7f1a400a2fb0 2024-03-21T11:19:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:26 smithi047 ceph-mon[31351]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-21T11:19:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:26 smithi047 ceph-mon[31351]: osdmap e74: 11 total, 11 up, 11 in 2024-03-21T11:19:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:27.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:27.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:27.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:27.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:27.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:27.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:27.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:27.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:28.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:27 smithi100 ceph-mon[36245]: Detected new or changed devices on smithi100 2024-03-21T11:19:28.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:27 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi100 to 4270M 2024-03-21T11:19:28.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:27 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:28.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:27 smithi100 ceph-mon[36245]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T11:19:28.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:27 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:28.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:27 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:28.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:27 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:19:28.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:27 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:19:28.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:27 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:28.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:27 smithi102 ceph-mon[36748]: Detected new or changed devices on smithi100 2024-03-21T11:19:28.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:27 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi100 to 4270M 2024-03-21T11:19:28.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:27 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:28.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:27 smithi102 ceph-mon[36748]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T11:19:28.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:27 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:28.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:27 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:28.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:27 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:19:28.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:27 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:19:28.182 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:27 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:27 smithi047 ceph-mon[31351]: Detected new or changed devices on smithi100 2024-03-21T11:19:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:27 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi100 to 4270M 2024-03-21T11:19:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:27 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:27 smithi047 ceph-mon[31351]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T11:19:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:27 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:27 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:27 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T11:19:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:27 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T11:19:28.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:27 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:28.962 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:28 smithi102 ceph-mon[36748]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:28.963 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:28 smithi102 ceph-mon[36748]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:19:28.963 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:28 smithi102 ceph-mon[36748]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T11:19:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:28 smithi047 ceph-mon[31351]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:28 smithi047 ceph-mon[31351]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:19:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:28 smithi047 ceph-mon[31351]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T11:19:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:28 smithi100 ceph-mon[36245]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:28 smithi100 ceph-mon[36245]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:19:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:28 smithi100 ceph-mon[36245]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T11:19:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:29 smithi102 ceph-mon[36748]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T11:19:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:29 smithi102 ceph-mon[36748]: pgmap v267: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T11:19:30.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:29 smithi047 ceph-mon[31351]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T11:19:30.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:29 smithi047 ceph-mon[31351]: pgmap v267: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T11:19:30.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:29 smithi100 ceph-mon[36245]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T11:19:30.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:29 smithi100 ceph-mon[36245]: pgmap v267: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T11:19:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:30 smithi102 ceph-mon[36748]: from='client.? 172.21.15.102:0/3091982639' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "756cdfb6-361e-494f-89a0-ad0d560f9675"}]: dispatch 2024-03-21T11:19:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:30 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "756cdfb6-361e-494f-89a0-ad0d560f9675"}]: dispatch 2024-03-21T11:19:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:30 smithi102 ceph-mon[36748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "756cdfb6-361e-494f-89a0-ad0d560f9675"}]': finished 2024-03-21T11:19:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:30 smithi102 ceph-mon[36748]: osdmap e78: 12 total, 11 up, 12 in 2024-03-21T11:19:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:30 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:30 smithi102 ceph-mon[36748]: from='client.? 172.21.15.102:0/3471416907' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:19:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:30 smithi047 ceph-mon[31351]: from='client.? 172.21.15.102:0/3091982639' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "756cdfb6-361e-494f-89a0-ad0d560f9675"}]: dispatch 2024-03-21T11:19:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:30 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "756cdfb6-361e-494f-89a0-ad0d560f9675"}]: dispatch 2024-03-21T11:19:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:30 smithi047 ceph-mon[31351]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "756cdfb6-361e-494f-89a0-ad0d560f9675"}]': finished 2024-03-21T11:19:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:30 smithi047 ceph-mon[31351]: osdmap e78: 12 total, 11 up, 12 in 2024-03-21T11:19:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:30 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:30 smithi047 ceph-mon[31351]: from='client.? 172.21.15.102:0/3471416907' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:19:31.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:30 smithi100 ceph-mon[36245]: from='client.? 172.21.15.102:0/3091982639' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "756cdfb6-361e-494f-89a0-ad0d560f9675"}]: dispatch 2024-03-21T11:19:31.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:30 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "756cdfb6-361e-494f-89a0-ad0d560f9675"}]: dispatch 2024-03-21T11:19:31.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:30 smithi100 ceph-mon[36245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "756cdfb6-361e-494f-89a0-ad0d560f9675"}]': finished 2024-03-21T11:19:31.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:30 smithi100 ceph-mon[36245]: osdmap e78: 12 total, 11 up, 12 in 2024-03-21T11:19:31.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:30 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:31.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:30 smithi100 ceph-mon[36245]: from='client.? 172.21.15.102:0/3471416907' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T11:19:32.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:31 smithi102 ceph-mon[36748]: pgmap v269: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-03-21T11:19:32.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:31 smithi047 ceph-mon[31351]: pgmap v269: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-03-21T11:19:32.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:31 smithi100 ceph-mon[36245]: pgmap v269: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-03-21T11:19:34.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:34 smithi047 ceph-mon[31351]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T11:19:34.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:34 smithi100 ceph-mon[36245]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T11:19:34.406 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:34 smithi102 ceph-mon[36748]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T11:19:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:36 smithi047 ceph-mon[31351]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:36.354 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:36 smithi102 ceph-mon[36748]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:36.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:36 smithi100 ceph-mon[36245]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:38.278 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:38 smithi102 ceph-mon[36748]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:38.278 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:38 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T11:19:38.278 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:38 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:38.278 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:38 smithi102 ceph-mon[36748]: Deploying daemon osd.11 on smithi102 2024-03-21T11:19:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:38 smithi047 ceph-mon[31351]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T11:19:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:38 smithi047 ceph-mon[31351]: Deploying daemon osd.11 on smithi102 2024-03-21T11:19:38.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:38 smithi100 ceph-mon[36245]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:38 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T11:19:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:38 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:38 smithi100 ceph-mon[36245]: Deploying daemon osd.11 on smithi102 2024-03-21T11:19:40.126 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:40 smithi102 ceph-mon[36748]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:40.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:40 smithi047 ceph-mon[31351]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:40.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:40 smithi100 ceph-mon[36245]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:42.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:42 smithi102 ceph-mon[36748]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:42.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:42 smithi047 ceph-mon[31351]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:42.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:42 smithi100 ceph-mon[36245]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:44.198 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:44 smithi102 ceph-mon[36748]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:44.198 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:44 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:44.198 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:44 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:44.198 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:44 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:44.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:44 smithi047 ceph-mon[31351]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:44.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:44 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:44.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:44 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:44.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:44 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:44.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:44 smithi100 ceph-mon[36245]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:44.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:44 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:44.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:44 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:44.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:44 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:46.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:46 smithi102 ceph-mon[36748]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:46.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:46 smithi047 ceph-mon[31351]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:46.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:46 smithi100 ceph-mon[36245]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:47.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:47.169+0000 7f1a54ff9700 1 -- 172.21.15.102:0/886083338 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f1a580625d0 con 0x7f1a400a2fb0 2024-03-21T11:19:47.170 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 11 on host 'smithi102' 2024-03-21T11:19:47.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:47.172+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/886083338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a400a2fb0 msgr2=0x7f1a400a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:47.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:47.172+0000 7f1a5dcb5700 1 --2- 172.21.15.102:0/886083338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a400a2fb0 0x7f1a400a5470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1a5806a670 tx=0x7f1a48009470 comp rx=0 tx=0).stop 2024-03-21T11:19:47.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:47.172+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/886083338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a58103e50 msgr2=0x7f1a581019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:47.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:47.172+0000 7f1a5dcb5700 1 --2- 172.21.15.102:0/886083338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a58103e50 0x7f1a581019a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a50007f00 tx=0x7f1a50004e80 comp rx=0 tx=0).stop 2024-03-21T11:19:47.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:47.172+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/886083338 shutdown_connections 2024-03-21T11:19:47.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:47.172+0000 7f1a5dcb5700 1 --2- 172.21.15.102:0/886083338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a58103e50 0x7f1a581019a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:47.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:47.172+0000 7f1a5dcb5700 1 --2- 172.21.15.102:0/886083338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a400a2fb0 0x7f1a400a5470 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:47.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:47.172+0000 7f1a5dcb5700 1 --2- 172.21.15.102:0/886083338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a581004d0 0x7f1a58102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:47.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:47.172+0000 7f1a5dcb5700 1 --2- 172.21.15.102:0/886083338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a581034b0 0x7f1a58101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:47.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:47.172+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/886083338 >> 172.21.15.102:0/886083338 conn(0x7f1a580fbe30 msgr2=0x7f1a581066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:19:47.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:47.172+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/886083338 shutdown_connections 2024-03-21T11:19:47.173 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:19:47.172+0000 7f1a5dcb5700 1 -- 172.21.15.102:0/886083338 wait complete. 2024-03-21T11:19:47.485 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.486 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.486 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:47.486 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:47.486 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.486 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:47 smithi102 ceph-mon[36748]: from='osd.11 [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T11:19:47.486 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:47 smithi102 ceph-mon[36748]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T11:19:47.486 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.486 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.486 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:47.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:47.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:47.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:47 smithi047 ceph-mon[31351]: from='osd.11 [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T11:19:47.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:47 smithi047 ceph-mon[31351]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T11:19:47.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:47.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:47.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:47.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:47 smithi100 ceph-mon[36245]: from='osd.11 [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T11:19:47.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:47 smithi100 ceph-mon[36245]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T11:19:47.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:47.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:19:47.846 DEBUG:teuthology.orchestra.run.smithi102:osd.11> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.11.service 2024-03-21T11:19:47.849 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-21T11:19:47.849 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-21T11:19:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:19:48.366 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:48 smithi102 ceph-mon[36748]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:48.366 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:48 smithi102 ceph-mon[36748]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T11:19:48.366 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:48 smithi102 ceph-mon[36748]: from='osd.11 [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:19:48.366 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:48 smithi102 ceph-mon[36748]: osdmap e79: 12 total, 11 up, 12 in 2024-03-21T11:19:48.366 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:48.366 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:48 smithi102 ceph-mon[36748]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:19:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:48 smithi047 ceph-mon[31351]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:48 smithi047 ceph-mon[31351]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T11:19:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:48 smithi047 ceph-mon[31351]: from='osd.11 [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:19:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:48 smithi047 ceph-mon[31351]: osdmap e79: 12 total, 11 up, 12 in 2024-03-21T11:19:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:48 smithi047 ceph-mon[31351]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:19:48.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:48 smithi100 ceph-mon[36245]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:48.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:48 smithi100 ceph-mon[36245]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T11:19:48.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:48 smithi100 ceph-mon[36245]: from='osd.11 [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:19:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:48 smithi100 ceph-mon[36245]: osdmap e79: 12 total, 11 up, 12 in 2024-03-21T11:19:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:48 smithi100 ceph-mon[36245]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-03-21T11:19:48.970 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:19:48 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:19:48.701+0000 7f4820e42700 -1 osd.11 0 waiting for initial osdmap 2024-03-21T11:19:48.970 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:19:48 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:19:48.715+0000 7f4818c2d700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T11:19:49.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:49 smithi047 ceph-mon[31351]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-03-21T11:19:49.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:49 smithi047 ceph-mon[31351]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T11:19:49.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:49.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:49.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:49 smithi047 ceph-mon[31351]: from='osd.11 ' entity='osd.11' 2024-03-21T11:19:49.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:49 smithi100 ceph-mon[36245]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-03-21T11:19:49.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:49 smithi100 ceph-mon[36245]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T11:19:49.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:49.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:49.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:49 smithi100 ceph-mon[36245]: from='osd.11 ' entity='osd.11' 2024-03-21T11:19:49.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:49 smithi102 ceph-mon[36748]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-03-21T11:19:49.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:49 smithi102 ceph-mon[36748]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T11:19:49.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:49.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:49.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:49 smithi102 ceph-mon[36748]: from='osd.11 ' entity='osd.11' 2024-03-21T11:19:50.419 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:50 smithi102 ceph-mon[36748]: purged_snaps scrub starts 2024-03-21T11:19:50.419 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:50 smithi102 ceph-mon[36748]: purged_snaps scrub ok 2024-03-21T11:19:50.420 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:50 smithi102 ceph-mon[36748]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:50.420 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:50.420 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:50 smithi102 ceph-mon[36748]: osd.11 [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355] boot 2024-03-21T11:19:50.420 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:50 smithi102 ceph-mon[36748]: osdmap e81: 12 total, 12 up, 12 in 2024-03-21T11:19:50.420 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:50.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:50 smithi047 ceph-mon[31351]: purged_snaps scrub starts 2024-03-21T11:19:50.555 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:50 smithi047 ceph-mon[31351]: purged_snaps scrub ok 2024-03-21T11:19:50.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:50 smithi047 ceph-mon[31351]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:50.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:50.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:50 smithi047 ceph-mon[31351]: osd.11 [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355] boot 2024-03-21T11:19:50.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:50 smithi047 ceph-mon[31351]: osdmap e81: 12 total, 12 up, 12 in 2024-03-21T11:19:50.556 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:50.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:50 smithi100 ceph-mon[36245]: purged_snaps scrub starts 2024-03-21T11:19:50.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:50 smithi100 ceph-mon[36245]: purged_snaps scrub ok 2024-03-21T11:19:50.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:50 smithi100 ceph-mon[36245]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-21T11:19:50.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:50.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:50 smithi100 ceph-mon[36245]: osd.11 [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355] boot 2024-03-21T11:19:50.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:50 smithi100 ceph-mon[36245]: osdmap e81: 12 total, 12 up, 12 in 2024-03-21T11:19:50.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T11:19:50.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.887+0000 7fab0c800700 1 -- 172.21.15.47:0/2671290947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab04107ea0 msgr2=0x7fab0410a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:50.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.887+0000 7fab0c800700 1 --2- 172.21.15.47:0/2671290947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab04107ea0 0x7fab0410a310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fab00009f90 tx=0x7fab0000b330 comp rx=0 tx=0).stop 2024-03-21T11:19:50.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.887+0000 7fab0c800700 1 -- 172.21.15.47:0/2671290947 shutdown_connections 2024-03-21T11:19:50.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.887+0000 7fab0c800700 1 --2- 172.21.15.47:0/2671290947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab04107ea0 0x7fab0410a310 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:50.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.887+0000 7fab0c800700 1 --2- 172.21.15.47:0/2671290947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab04102c40 0x7fab04105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:50.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.887+0000 7fab0c800700 1 --2- 172.21.15.47:0/2671290947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab04105570 0x7fab04107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:50.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.887+0000 7fab0c800700 1 -- 172.21.15.47:0/2671290947 >> 172.21.15.47:0/2671290947 conn(0x7fab040fc650 msgr2=0x7fab040fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:19:50.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.887+0000 7fab0c800700 1 -- 172.21.15.47:0/2671290947 shutdown_connections 2024-03-21T11:19:50.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.887+0000 7fab0c800700 1 -- 172.21.15.47:0/2671290947 wait complete. 2024-03-21T11:19:50.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.888+0000 7fab0c800700 1 Processor -- start 2024-03-21T11:19:50.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.889+0000 7fab0c800700 1 -- start start 2024-03-21T11:19:50.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.889+0000 7fab0c800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab04102c40 0x7fab041025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:50.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.889+0000 7fab0c800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab04105570 0x7fab04100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:50.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.889+0000 7fab0c800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab04107ea0 0x7fab04101130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:50.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.889+0000 7fab0c800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0410dd90 con 0x7fab04105570 2024-03-21T11:19:50.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.889+0000 7fab0c800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0410dc10 con 0x7fab04102c40 2024-03-21T11:19:50.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.889+0000 7fab0c800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0410df10 con 0x7fab04107ea0 2024-03-21T11:19:50.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.889+0000 7fab0a800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab04107ea0 0x7fab04101130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:50.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.889+0000 7fab0a800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab04107ea0 0x7fab04101130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44302/0 (socket says 172.21.15.47:44302) 2024-03-21T11:19:50.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.889+0000 7fab0a800700 1 -- 172.21.15.47:0/1325741655 learned_addr learned my addr 172.21.15.47:0/1325741655 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:19:50.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.889+0000 7fab09e00700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab04102c40 0x7fab041025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:50.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.890+0000 7fab09400700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab04105570 0x7fab04100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:50.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.890+0000 7fab0a800700 1 -- 172.21.15.47:0/1325741655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab04102c40 msgr2=0x7fab041025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:50.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.890+0000 7fab0a800700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab04102c40 0x7fab041025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:50.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.890+0000 7fab0a800700 1 -- 172.21.15.47:0/1325741655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab04105570 msgr2=0x7fab04100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:50.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.890+0000 7fab0a800700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab04105570 0x7fab04100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:50.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.890+0000 7fab0a800700 1 -- 172.21.15.47:0/1325741655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab041019e0 con 0x7fab04107ea0 2024-03-21T11:19:50.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.890+0000 7fab09400700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab04105570 0x7fab04100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:19:50.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.890+0000 7fab0a800700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab04107ea0 0x7fab04101130 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fab00007c70 tx=0x7fab00007ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:19:50.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.891+0000 7faafea00700 1 -- 172.21.15.47:0/1325741655 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00021020 con 0x7fab04107ea0 2024-03-21T11:19:50.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.891+0000 7fab0c800700 1 -- 172.21.15.47:0/1325741655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab041a2480 con 0x7fab04107ea0 2024-03-21T11:19:50.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.891+0000 7faafea00700 1 -- 172.21.15.47:0/1325741655 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab00004e40 con 0x7fab04107ea0 2024-03-21T11:19:50.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.891+0000 7faafea00700 1 -- 172.21.15.47:0/1325741655 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab000052b0 con 0x7fab04107ea0 2024-03-21T11:19:50.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.891+0000 7fab0c800700 1 -- 172.21.15.47:0/1325741655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab041a2930 con 0x7fab04107ea0 2024-03-21T11:19:50.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.892+0000 7fab0c800700 1 -- 172.21.15.47:0/1325741655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab04068e50 con 0x7fab04107ea0 2024-03-21T11:19:50.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.895+0000 7faafea00700 1 -- 172.21.15.47:0/1325741655 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fab00024020 con 0x7fab04107ea0 2024-03-21T11:19:50.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.896+0000 7faafea00700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faaec0a3030 0x7faaec0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:50.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.897+0000 7faafea00700 1 -- 172.21.15.47:0/1325741655 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fab0000f020 con 0x7fab04107ea0 2024-03-21T11:19:50.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.897+0000 7fab09e00700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faaec0a3030 0x7faaec0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:50.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.898+0000 7fab09e00700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faaec0a3030 0x7faaec0a54f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009c60 tx=0x7faaf8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:19:50.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:50.900+0000 7faafea00700 1 -- 172.21.15.47:0/1325741655 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fab0009d420 con 0x7fab04107ea0 2024-03-21T11:19:51.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:51.140+0000 7fab0c800700 1 -- 172.21.15.47:0/1325741655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fab04062da0 con 0x7fab04107ea0 2024-03-21T11:19:51.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:51.141+0000 7faafea00700 1 -- 172.21.15.47:0/1325741655 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) v1 ==== 74+0+133 (secure 0 0 0) 0x7fab0009cd20 con 0x7fab04107ea0 2024-03-21T11:19:51.142 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:19:51.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:51.146+0000 7fab0c800700 1 -- 172.21.15.47:0/1325741655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faaec0a3030 msgr2=0x7faaec0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:51.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:51.146+0000 7fab0c800700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faaec0a3030 0x7faaec0a54f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009c60 tx=0x7faaf8009260 comp rx=0 tx=0).stop 2024-03-21T11:19:51.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:51.146+0000 7fab0c800700 1 -- 172.21.15.47:0/1325741655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab04107ea0 msgr2=0x7fab04101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:51.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:51.146+0000 7fab0c800700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab04107ea0 0x7fab04101130 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fab00007c70 tx=0x7fab00007ca0 comp rx=0 tx=0).stop 2024-03-21T11:19:51.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:51.146+0000 7fab0c800700 1 -- 172.21.15.47:0/1325741655 shutdown_connections 2024-03-21T11:19:51.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:51.146+0000 7fab0c800700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab04107ea0 0x7fab04101130 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:51.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:51.146+0000 7fab0c800700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab04102c40 0x7fab041025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:51.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:51.146+0000 7fab0c800700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faaec0a3030 0x7faaec0a54f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:51.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:51.146+0000 7fab0c800700 1 --2- 172.21.15.47:0/1325741655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab04105570 0x7fab04100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:51.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:51.146+0000 7fab0c800700 1 -- 172.21.15.47:0/1325741655 >> 172.21.15.47:0/1325741655 conn(0x7fab040fc650 msgr2=0x7fab041034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:19:51.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:51.146+0000 7fab0c800700 1 -- 172.21.15.47:0/1325741655 shutdown_connections 2024-03-21T11:19:51.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:51.146+0000 7fab0c800700 1 -- 172.21.15.47:0/1325741655 wait complete. 2024-03-21T11:19:51.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:51 smithi047 ceph-mon[31351]: osdmap e82: 12 total, 12 up, 12 in 2024-03-21T11:19:51.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:51 smithi047 ceph-mon[31351]: Detected new or changed devices on smithi102 2024-03-21T11:19:51.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:51.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:51.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:51.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:51.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:51.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:51.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:51 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi102 to 4270M 2024-03-21T11:19:51.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:51.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:51.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:51.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:51.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:51 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1325741655' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T11:19:51.909 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1711019989,"num_in_osds":12,"osd_in_since":1711019970,"num_remapped_pgs":0} 2024-03-21T11:19:51.910 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T11:19:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:51 smithi102 ceph-mon[36748]: osdmap e82: 12 total, 12 up, 12 in 2024-03-21T11:19:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:51 smithi102 ceph-mon[36748]: Detected new or changed devices on smithi102 2024-03-21T11:19:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:51.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:51 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi102 to 4270M 2024-03-21T11:19:51.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:51.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:51.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:51.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:51.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:51 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1325741655' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T11:19:52.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:51 smithi100 ceph-mon[36245]: osdmap e82: 12 total, 12 up, 12 in 2024-03-21T11:19:52.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:51 smithi100 ceph-mon[36245]: Detected new or changed devices on smithi102 2024-03-21T11:19:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:19:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:51 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi102 to 4270M 2024-03-21T11:19:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:19:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:19:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:19:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:51 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1325741655' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T11:19:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:19:53.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:52 smithi047 ceph-mon[31351]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:19:53.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:52 smithi100 ceph-mon[36245]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:19:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:52 smithi102 ceph-mon[36748]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:19:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.763+0000 7fa9fda00700 1 -- 172.21.15.47:0/3903577649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9f81095c0 msgr2=0x7fa9f8111190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.763+0000 7fa9fda00700 1 --2- 172.21.15.47:0/3903577649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9f81095c0 0x7fa9f8111190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00099c0 tx=0x7fa9f0009cd0 comp rx=0 tx=0).stop 2024-03-21T11:19:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.763+0000 7fa9fda00700 1 -- 172.21.15.47:0/3903577649 shutdown_connections 2024-03-21T11:19:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.763+0000 7fa9fda00700 1 --2- 172.21.15.47:0/3903577649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9f81095c0 0x7fa9f8111190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.763+0000 7fa9fda00700 1 --2- 172.21.15.47:0/3903577649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f81044e0 0x7fa9f81048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.763+0000 7fa9fda00700 1 --2- 172.21.15.47:0/3903577649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f8104e90 0x7fa9f8108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.763+0000 7fa9fda00700 1 -- 172.21.15.47:0/3903577649 >> 172.21.15.47:0/3903577649 conn(0x7fa9f80ffdb0 msgr2=0x7fa9f81021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:19:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.763+0000 7fa9fda00700 1 -- 172.21.15.47:0/3903577649 shutdown_connections 2024-03-21T11:19:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.763+0000 7fa9fda00700 1 -- 172.21.15.47:0/3903577649 wait complete. 2024-03-21T11:19:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.764+0000 7fa9fda00700 1 Processor -- start 2024-03-21T11:19:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.764+0000 7fa9fda00700 1 -- start start 2024-03-21T11:19:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9fda00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f81044e0 0x7fa9f81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9fda00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9f8104e90 0x7fa9f81a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9fda00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f81095c0 0x7fa9f819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9fda00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f8114620 con 0x7fa9f81095c0 2024-03-21T11:19:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9fda00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f81144a0 con 0x7fa9f81044e0 2024-03-21T11:19:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9fda00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f81147a0 con 0x7fa9f8104e90 2024-03-21T11:19:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9f7e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f81044e0 0x7fa9f81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9fd000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f81095c0 0x7fa9f819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9f7e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f81044e0 0x7fa9f81a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32778/0 (socket says 172.21.15.47:32778) 2024-03-21T11:19:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9f7e00700 1 -- 172.21.15.47:0/206781565 learned_addr learned my addr 172.21.15.47:0/206781565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:19:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9ef400700 1 --2- 172.21.15.47:0/206781565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9f8104e90 0x7fa9f81a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9f7e00700 1 -- 172.21.15.47:0/206781565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9f8104e90 msgr2=0x7fa9f81a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9f7e00700 1 --2- 172.21.15.47:0/206781565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9f8104e90 0x7fa9f81a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9f7e00700 1 -- 172.21.15.47:0/206781565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f81095c0 msgr2=0x7fa9f819a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9f7e00700 1 --2- 172.21.15.47:0/206781565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f81095c0 0x7fa9f819a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.765+0000 7fa9f7e00700 1 -- 172.21.15.47:0/206781565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f819afc0 con 0x7fa9f81044e0 2024-03-21T11:19:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.766+0000 7fa9f7e00700 1 --2- 172.21.15.47:0/206781565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f81044e0 0x7fa9f81a0660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800e9c0 tx=0x7fa9e800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:19:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.766+0000 7fa9f5600700 1 -- 172.21.15.47:0/206781565 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800cbd0 con 0x7fa9f81044e0 2024-03-21T11:19:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.766+0000 7fa9f5600700 1 -- 172.21.15.47:0/206781565 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa9e800ee60 con 0x7fa9f81044e0 2024-03-21T11:19:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.766+0000 7fa9fda00700 1 -- 172.21.15.47:0/206781565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f819b2b0 con 0x7fa9f81044e0 2024-03-21T11:19:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.766+0000 7fa9f5600700 1 -- 172.21.15.47:0/206781565 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8005630 con 0x7fa9f81044e0 2024-03-21T11:19:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.766+0000 7fa9fda00700 1 -- 172.21.15.47:0/206781565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f81a74c0 con 0x7fa9f81044e0 2024-03-21T11:19:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.767+0000 7fa9fda00700 1 -- 172.21.15.47:0/206781565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f81059c0 con 0x7fa9f81044e0 2024-03-21T11:19:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.769+0000 7fa9f5600700 1 -- 172.21.15.47:0/206781565 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa9e8004750 con 0x7fa9f81044e0 2024-03-21T11:19:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.771+0000 7fa9f5600700 1 --2- 172.21.15.47:0/206781565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa9dc0a3030 0x7fa9dc0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.771+0000 7fa9f5600700 1 -- 172.21.15.47:0/206781565 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa9e80cf1e0 con 0x7fa9f81044e0 2024-03-21T11:19:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.771+0000 7fa9ef400700 1 --2- 172.21.15.47:0/206781565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa9dc0a3030 0x7fa9dc0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.772+0000 7fa9ef400700 1 --2- 172.21.15.47:0/206781565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa9dc0a3030 0x7fa9dc0a54f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8103410 tx=0x7fa9e000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:19:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:54.775+0000 7fa9f5600700 1 -- 172.21.15.47:0/206781565 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa9e809c410 con 0x7fa9f81044e0 2024-03-21T11:19:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:55.010+0000 7fa9fda00700 1 -- 172.21.15.47:0/206781565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9f81a7750 con 0x7fa9f81044e0 2024-03-21T11:19:55.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:55.012+0000 7fa9f5600700 1 -- 172.21.15.47:0/206781565 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18455 (secure 0 0 0) 0x7fa9e809bd10 con 0x7fa9f81044e0 2024-03-21T11:19:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:19:55.013 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":82,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:19:50.716171+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:19:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:55.015+0000 7fa9fda00700 1 -- 172.21.15.47:0/206781565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa9dc0a3030 msgr2=0x7fa9dc0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:55.015+0000 7fa9fda00700 1 --2- 172.21.15.47:0/206781565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa9dc0a3030 0x7fa9dc0a54f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8103410 tx=0x7fa9e000b040 comp rx=0 tx=0).stop 2024-03-21T11:19:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:55.015+0000 7fa9fda00700 1 -- 172.21.15.47:0/206781565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f81044e0 msgr2=0x7fa9f81a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:55.015+0000 7fa9fda00700 1 --2- 172.21.15.47:0/206781565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f81044e0 0x7fa9f81a0660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800e9c0 tx=0x7fa9e800ecd0 comp rx=0 tx=0).stop 2024-03-21T11:19:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:55.015+0000 7fa9fda00700 1 -- 172.21.15.47:0/206781565 shutdown_connections 2024-03-21T11:19:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:55.015+0000 7fa9fda00700 1 --2- 172.21.15.47:0/206781565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa9dc0a3030 0x7fa9dc0a54f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:55.015+0000 7fa9fda00700 1 --2- 172.21.15.47:0/206781565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9f81095c0 0x7fa9f819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:55.015+0000 7fa9fda00700 1 --2- 172.21.15.47:0/206781565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f81044e0 0x7fa9f81a0660 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:55.015+0000 7fa9fda00700 1 --2- 172.21.15.47:0/206781565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9f8104e90 0x7fa9f81a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:55.015+0000 7fa9fda00700 1 -- 172.21.15.47:0/206781565 >> 172.21.15.47:0/206781565 conn(0x7fa9f80ffdb0 msgr2=0x7fa9f8100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:19:55.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:55.015+0000 7fa9fda00700 1 -- 172.21.15.47:0/206781565 shutdown_connections 2024-03-21T11:19:55.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:55.015+0000 7fa9fda00700 1 -- 172.21.15.47:0/206781565 wait complete. 2024-03-21T11:19:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:54 smithi102 ceph-mon[36748]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:19:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:54 smithi047 ceph-mon[31351]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:19:55.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:54 smithi100 ceph-mon[36245]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:19:55.766 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T11:15:35.545245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T11:19:55.767 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-21T11:19:56.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:55 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/206781565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:19:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:19:56.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:55 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/206781565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:19:56.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:55 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/206781565' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:19:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:56 smithi047 ceph-mon[31351]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:19:57.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:56 smithi102 ceph-mon[36748]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:19:57.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:56 smithi100 ceph-mon[36245]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:19:58.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:57 smithi047 ceph-mon[31351]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:19:58.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:57 smithi102 ceph-mon[36748]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:19:58.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.389+0000 7fbc6b800700 1 -- 172.21.15.47:0/2210372801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc64104b90 msgr2=0x7fbc64105020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:58.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.389+0000 7fbc6b800700 1 --2- 172.21.15.47:0/2210372801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc64104b90 0x7fbc64105020 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00b0a0 tx=0x7fbc5c00b3b0 comp rx=0 tx=0).stop 2024-03-21T11:19:58.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.389+0000 7fbc6b800700 1 -- 172.21.15.47:0/2210372801 shutdown_connections 2024-03-21T11:19:58.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.389+0000 7fbc6b800700 1 --2- 172.21.15.47:0/2210372801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc64104b90 0x7fbc64105020 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:58.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.389+0000 7fbc6b800700 1 --2- 172.21.15.47:0/2210372801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc64107080 0x7fbc64069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:58.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.389+0000 7fbc6b800700 1 --2- 172.21.15.47:0/2210372801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc6406a380 0x7fbc6406a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:58.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.389+0000 7fbc6b800700 1 -- 172.21.15.47:0/2210372801 >> 172.21.15.47:0/2210372801 conn(0x7fbc64077fc0 msgr2=0x7fbc640783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:19:58.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.389+0000 7fbc6b800700 1 -- 172.21.15.47:0/2210372801 shutdown_connections 2024-03-21T11:19:58.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.389+0000 7fbc6b800700 1 -- 172.21.15.47:0/2210372801 wait complete. 2024-03-21T11:19:58.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.391+0000 7fbc6b800700 1 Processor -- start 2024-03-21T11:19:58.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.391+0000 7fbc6b800700 1 -- start start 2024-03-21T11:19:58.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.391+0000 7fbc6b800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc6406a380 0x7fbc6419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:58.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.391+0000 7fbc6b800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc64104b90 0x7fbc6419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:58.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.391+0000 7fbc6b800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc64107080 0x7fbc641a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:58.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.391+0000 7fbc6b800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc64118a60 con 0x7fbc6406a380 2024-03-21T11:19:58.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.391+0000 7fbc6b800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc641188e0 con 0x7fbc64107080 2024-03-21T11:19:58.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.391+0000 7fbc6b800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc64118be0 con 0x7fbc64104b90 2024-03-21T11:19:58.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.392+0000 7fbc5be00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc64104b90 0x7fbc6419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:58.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.392+0000 7fbc68e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc6406a380 0x7fbc6419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:58.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.392+0000 7fbc5be00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc64104b90 0x7fbc6419f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44336/0 (socket says 172.21.15.47:44336) 2024-03-21T11:19:58.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.392+0000 7fbc5be00700 1 -- 172.21.15.47:0/2585413765 learned_addr learned my addr 172.21.15.47:0/2585413765 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:19:58.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.392+0000 7fbc69800700 1 --2- 172.21.15.47:0/2585413765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc64107080 0x7fbc641a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:58.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.392+0000 7fbc5be00700 1 -- 172.21.15.47:0/2585413765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc64107080 msgr2=0x7fbc641a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:58.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.392+0000 7fbc5be00700 1 --2- 172.21.15.47:0/2585413765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc64107080 0x7fbc641a3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:58.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.392+0000 7fbc5be00700 1 -- 172.21.15.47:0/2585413765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc6406a380 msgr2=0x7fbc6419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:58.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.392+0000 7fbc5be00700 1 --2- 172.21.15.47:0/2585413765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc6406a380 0x7fbc6419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:58.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.392+0000 7fbc5be00700 1 -- 172.21.15.47:0/2585413765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc641a3e80 con 0x7fbc64104b90 2024-03-21T11:19:58.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:57 smithi100 ceph-mon[36245]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:19:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.392+0000 7fbc5be00700 1 --2- 172.21.15.47:0/2585413765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc64104b90 0x7fbc6419f3b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400d980 tx=0x7fbc5400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:19:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.393+0000 7fbc62000700 1 -- 172.21.15.47:0/2585413765 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc54016070 con 0x7fbc64104b90 2024-03-21T11:19:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.393+0000 7fbc62000700 1 -- 172.21.15.47:0/2585413765 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc54004d10 con 0x7fbc64104b90 2024-03-21T11:19:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.393+0000 7fbc6b800700 1 -- 172.21.15.47:0/2585413765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc641a4170 con 0x7fbc64104b90 2024-03-21T11:19:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.393+0000 7fbc62000700 1 -- 172.21.15.47:0/2585413765 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc540055f0 con 0x7fbc64104b90 2024-03-21T11:19:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.393+0000 7fbc6b800700 1 -- 172.21.15.47:0/2585413765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc641ab9f0 con 0x7fbc64104b90 2024-03-21T11:19:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.394+0000 7fbc6b800700 1 -- 172.21.15.47:0/2585413765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc64068e50 con 0x7fbc64104b90 2024-03-21T11:19:58.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.396+0000 7fbc62000700 1 -- 172.21.15.47:0/2585413765 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbc54004750 con 0x7fbc64104b90 2024-03-21T11:19:58.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.397+0000 7fbc62000700 1 --2- 172.21.15.47:0/2585413765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbc440a2fb0 0x7fbc440a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:19:58.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.397+0000 7fbc62000700 1 -- 172.21.15.47:0/2585413765 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbc540cee20 con 0x7fbc64104b90 2024-03-21T11:19:58.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.398+0000 7fbc68e00700 1 --2- 172.21.15.47:0/2585413765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbc440a2fb0 0x7fbc440a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:19:58.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.398+0000 7fbc68e00700 1 --2- 172.21.15.47:0/2585413765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbc440a2fb0 0x7fbc440a5470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c007b60 tx=0x7fbc4c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:19:58.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.405+0000 7fbc62000700 1 -- 172.21.15.47:0/2585413765 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbc5401f020 con 0x7fbc64104b90 2024-03-21T11:19:58.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.632+0000 7fbc6b800700 1 -- 172.21.15.47:0/2585413765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc641abbc0 con 0x7fbc64104b90 2024-03-21T11:19:58.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.633+0000 7fbc62000700 1 -- 172.21.15.47:0/2585413765 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc5409c050 con 0x7fbc64104b90 2024-03-21T11:19:58.634 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T11:19:58.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.636+0000 7fbc6b800700 1 -- 172.21.15.47:0/2585413765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbc440a2fb0 msgr2=0x7fbc440a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:58.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.636+0000 7fbc6b800700 1 --2- 172.21.15.47:0/2585413765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbc440a2fb0 0x7fbc440a5470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c007b60 tx=0x7fbc4c0073d0 comp rx=0 tx=0).stop 2024-03-21T11:19:58.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.636+0000 7fbc6b800700 1 -- 172.21.15.47:0/2585413765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc64104b90 msgr2=0x7fbc6419f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:19:58.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.636+0000 7fbc6b800700 1 --2- 172.21.15.47:0/2585413765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc64104b90 0x7fbc6419f3b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400d980 tx=0x7fbc5400dc90 comp rx=0 tx=0).stop 2024-03-21T11:19:58.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.636+0000 7fbc6b800700 1 -- 172.21.15.47:0/2585413765 shutdown_connections 2024-03-21T11:19:58.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.636+0000 7fbc6b800700 1 --2- 172.21.15.47:0/2585413765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc64107080 0x7fbc641a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:58.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.636+0000 7fbc6b800700 1 --2- 172.21.15.47:0/2585413765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbc440a2fb0 0x7fbc440a5470 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:58.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.636+0000 7fbc6b800700 1 --2- 172.21.15.47:0/2585413765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc6406a380 0x7fbc6419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:58.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.636+0000 7fbc6b800700 1 --2- 172.21.15.47:0/2585413765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc64104b90 0x7fbc6419f3b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:19:58.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.636+0000 7fbc6b800700 1 -- 172.21.15.47:0/2585413765 >> 172.21.15.47:0/2585413765 conn(0x7fbc64077fc0 msgr2=0x7fbc64103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:19:58.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.636+0000 7fbc6b800700 1 -- 172.21.15.47:0/2585413765 shutdown_connections 2024-03-21T11:19:58.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:19:58.636+0000 7fbc6b800700 1 -- 172.21.15.47:0/2585413765 wait complete. 2024-03-21T11:19:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:58 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2585413765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:19:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:58 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2585413765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:19:59.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:58 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2585413765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:19:59.617 INFO:tasks.cephadm:Adding mds.a on smithi047 2024-03-21T11:19:59.617 INFO:tasks.cephadm:Adding mds.d on smithi047 2024-03-21T11:19:59.617 INFO:tasks.cephadm:Adding mds.g on smithi047 2024-03-21T11:19:59.617 INFO:tasks.cephadm:Adding mds.j on smithi047 2024-03-21T11:19:59.618 INFO:tasks.cephadm:Adding mds.b on smithi100 2024-03-21T11:19:59.618 INFO:tasks.cephadm:Adding mds.e on smithi100 2024-03-21T11:19:59.618 INFO:tasks.cephadm:Adding mds.h on smithi100 2024-03-21T11:19:59.618 INFO:tasks.cephadm:Adding mds.k on smithi100 2024-03-21T11:19:59.618 INFO:tasks.cephadm:Adding mds.c on smithi102 2024-03-21T11:19:59.618 INFO:tasks.cephadm:Adding mds.f on smithi102 2024-03-21T11:19:59.618 INFO:tasks.cephadm:Adding mds.i on smithi102 2024-03-21T11:19:59.618 INFO:tasks.cephadm:Adding mds.l on smithi102 2024-03-21T11:19:59.618 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph orch apply mds all '12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l' 2024-03-21T11:19:59.857 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:20:00.213 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:19:59 smithi102 ceph-mon[36748]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:00.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:19:59 smithi047 ceph-mon[31351]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:00.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:19:59 smithi100 ceph-mon[36245]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:00 smithi102 ceph-mon[36748]: overall HEALTH_OK 2024-03-21T11:20:01.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:00 smithi047 ceph-mon[31351]: overall HEALTH_OK 2024-03-21T11:20:01.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:00 smithi100 ceph-mon[36245]: overall HEALTH_OK 2024-03-21T11:20:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.551+0000 7f7714179700 1 -- 172.21.15.102:0/466465201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f770c102330 msgr2=0x7f770c102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.551+0000 7f7714179700 1 --2- 172.21.15.102:0/466465201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f770c102330 0x7f770c102790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009990 tx=0x7f76fc009ca0 comp rx=0 tx=0).stop 2024-03-21T11:20:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.552+0000 7f7714179700 1 -- 172.21.15.102:0/466465201 shutdown_connections 2024-03-21T11:20:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.552+0000 7f7714179700 1 --2- 172.21.15.102:0/466465201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f770c102cd0 0x7f770c109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.552+0000 7f7714179700 1 --2- 172.21.15.102:0/466465201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f770c101130 0x7f770c101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.552+0000 7f7714179700 1 --2- 172.21.15.102:0/466465201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f770c102330 0x7f770c102790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.552+0000 7f7714179700 1 -- 172.21.15.102:0/466465201 >> 172.21.15.102:0/466465201 conn(0x7f770c0fc6f0 msgr2=0x7f770c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.552+0000 7f7714179700 1 -- 172.21.15.102:0/466465201 shutdown_connections 2024-03-21T11:20:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.552+0000 7f7714179700 1 -- 172.21.15.102:0/466465201 wait complete. 2024-03-21T11:20:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.552+0000 7f7714179700 1 Processor -- start 2024-03-21T11:20:01.555 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.552+0000 7f7714179700 1 -- start start 2024-03-21T11:20:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.552+0000 7f7714179700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f770c101130 0x7f770c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.552+0000 7f7714179700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f770c102330 0x7f770c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.552+0000 7f7714179700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f770c102cd0 0x7f770c19fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7714179700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c10cd50 con 0x7f770c102330 2024-03-21T11:20:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7714179700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c10cbd0 con 0x7f770c102cd0 2024-03-21T11:20:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7714179700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c10ced0 con 0x7f770c101130 2024-03-21T11:20:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7711f15700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f770c101130 0x7f770c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7711f15700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f770c101130 0x7f770c198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34828/0 (socket says 172.21.15.102:34828) 2024-03-21T11:20:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7711f15700 1 -- 172.21.15.102:0/4114366990 learned_addr learned my addr 172.21.15.102:0/4114366990 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-03-21T11:20:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7711f15700 1 -- 172.21.15.102:0/4114366990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f770c102cd0 msgr2=0x7f770c19fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7712716700 1 --2- 172.21.15.102:0/4114366990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f770c102cd0 0x7f770c19fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7711f15700 1 --2- 172.21.15.102:0/4114366990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f770c102cd0 0x7f770c19fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:01.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7711f15700 1 -- 172.21.15.102:0/4114366990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f770c102330 msgr2=0x7f770c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:01.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7711f15700 1 --2- 172.21.15.102:0/4114366990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f770c102330 0x7f770c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:01.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7711f15700 1 -- 172.21.15.102:0/4114366990 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c1a00b0 con 0x7f770c101130 2024-03-21T11:20:01.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7712716700 1 --2- 172.21.15.102:0/4114366990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f770c102cd0 0x7f770c19fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:20:01.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7711f15700 1 --2- 172.21.15.102:0/4114366990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f770c101130 0x7f770c198450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f770800b530 tx=0x7f770800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:01.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7702ffd700 1 -- 172.21.15.102:0/4114366990 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708015020 con 0x7f770c101130 2024-03-21T11:20:01.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7702ffd700 1 -- 172.21.15.102:0/4114366990 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7708011d10 con 0x7f770c101130 2024-03-21T11:20:01.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7714179700 1 -- 172.21.15.102:0/4114366990 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770c1a03a0 con 0x7f770c101130 2024-03-21T11:20:01.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7702ffd700 1 -- 172.21.15.102:0/4114366990 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770800f2e0 con 0x7f770c101130 2024-03-21T11:20:01.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.553+0000 7f7714179700 1 -- 172.21.15.102:0/4114366990 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c1a0960 con 0x7f770c101130 2024-03-21T11:20:01.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.554+0000 7f7714179700 1 -- 172.21.15.102:0/4114366990 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770c068e50 con 0x7f770c101130 2024-03-21T11:20:01.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.555+0000 7f7702ffd700 1 -- 172.21.15.102:0/4114366990 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7708018020 con 0x7f770c101130 2024-03-21T11:20:01.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.556+0000 7f7702ffd700 1 --2- 172.21.15.102:0/4114366990 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f76f80a2fb0 0x7f76f80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:01.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.556+0000 7f7702ffd700 1 -- 172.21.15.102:0/4114366990 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f77080ceca0 con 0x7f770c101130 2024-03-21T11:20:01.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.556+0000 7f7711714700 1 --2- 172.21.15.102:0/4114366990 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f76f80a2fb0 0x7f76f80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:01.558 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.557+0000 7f7711714700 1 --2- 172.21.15.102:0/4114366990 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f76f80a2fb0 0x7f76f80a5470 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0098e0 tx=0x7f76fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:01.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.559+0000 7f7702ffd700 1 -- 172.21.15.102:0/4114366990 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f770801f020 con 0x7f770c101130 2024-03-21T11:20:01.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.766+0000 7f7714179700 1 -- 172.21.15.102:0/4114366990 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f770c0fe990 con 0x7f76f80a2fb0 2024-03-21T11:20:01.784 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.783+0000 7f7702ffd700 1 -- 172.21.15.102:0/4114366990 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f770c0fe990 con 0x7f76f80a2fb0 2024-03-21T11:20:01.784 INFO:teuthology.orchestra.run.smithi102.stdout:Scheduled mds.all update... 2024-03-21T11:20:01.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.785+0000 7f7714179700 1 -- 172.21.15.102:0/4114366990 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f76f80a2fb0 msgr2=0x7f76f80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:01.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.785+0000 7f7714179700 1 --2- 172.21.15.102:0/4114366990 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f76f80a2fb0 0x7f76f80a5470 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0098e0 tx=0x7f76fc005fb0 comp rx=0 tx=0).stop 2024-03-21T11:20:01.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.785+0000 7f7714179700 1 -- 172.21.15.102:0/4114366990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f770c101130 msgr2=0x7f770c198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:01.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.785+0000 7f7714179700 1 --2- 172.21.15.102:0/4114366990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f770c101130 0x7f770c198450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f770800b530 tx=0x7f770800b8f0 comp rx=0 tx=0).stop 2024-03-21T11:20:01.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.785+0000 7f7714179700 1 -- 172.21.15.102:0/4114366990 shutdown_connections 2024-03-21T11:20:01.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.785+0000 7f7714179700 1 --2- 172.21.15.102:0/4114366990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f770c102cd0 0x7f770c19fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:01.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.785+0000 7f7714179700 1 --2- 172.21.15.102:0/4114366990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f770c101130 0x7f770c198450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:01.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.785+0000 7f7714179700 1 --2- 172.21.15.102:0/4114366990 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f76f80a2fb0 0x7f76f80a5470 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:01.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.785+0000 7f7714179700 1 --2- 172.21.15.102:0/4114366990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f770c102330 0x7f770c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:01.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.785+0000 7f7714179700 1 -- 172.21.15.102:0/4114366990 >> 172.21.15.102:0/4114366990 conn(0x7f770c0fc6f0 msgr2=0x7f770c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:01.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.785+0000 7f7714179700 1 -- 172.21.15.102:0/4114366990 shutdown_connections 2024-03-21T11:20:01.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-03-21T11:20:01.785+0000 7f7714179700 1 -- 172.21.15.102:0/4114366990 wait complete. 2024-03-21T11:20:02.407 DEBUG:teuthology.orchestra.run.smithi047:mds.a> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.a.service 2024-03-21T11:20:02.415 DEBUG:teuthology.orchestra.run.smithi047:mds.d> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.d.service 2024-03-21T11:20:02.461 DEBUG:teuthology.orchestra.run.smithi047:mds.g> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.g.service 2024-03-21T11:20:02.506 DEBUG:teuthology.orchestra.run.smithi047:mds.j> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.j.service 2024-03-21T11:20:02.551 DEBUG:teuthology.orchestra.run.smithi100:mds.b> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.b.service 2024-03-21T11:20:02.553 DEBUG:teuthology.orchestra.run.smithi100:mds.e> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.e.service 2024-03-21T11:20:02.596 DEBUG:teuthology.orchestra.run.smithi100:mds.h> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.h.service 2024-03-21T11:20:02.639 DEBUG:teuthology.orchestra.run.smithi100:mds.k> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.k.service 2024-03-21T11:20:02.682 DEBUG:teuthology.orchestra.run.smithi102:mds.c> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.c.service 2024-03-21T11:20:02.685 DEBUG:teuthology.orchestra.run.smithi102:mds.f> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.f.service 2024-03-21T11:20:02.728 DEBUG:teuthology.orchestra.run.smithi102:mds.i> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.i.service 2024-03-21T11:20:02.771 DEBUG:teuthology.orchestra.run.smithi102:mds.l> sudo journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.l.service 2024-03-21T11:20:02.813 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-21T11:20:02.814 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:20:02.892 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:02 smithi047 ceph-mon[31351]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:02.892 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:02 smithi047 ceph-mon[31351]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:20:02.892 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:02 smithi047 ceph-mon[31351]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-03-21T11:20:02.892 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:02.892 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:20:02.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:02 smithi100 ceph-mon[36245]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:02.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:02 smithi100 ceph-mon[36245]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:20:02.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:02 smithi100 ceph-mon[36245]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-03-21T11:20:02.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:02.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:20:02.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:02 smithi102 ceph-mon[36748]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:02.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:02 smithi102 ceph-mon[36748]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:20:02.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:02 smithi102 ceph-mon[36748]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-03-21T11:20:02.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:02.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:20:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.040+0000 7fbb53200640 1 -- 172.21.15.47:0/662891150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb541026f0 msgr2=0x7fbb54102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.040+0000 7fbb53200640 1 --2- 172.21.15.47:0/662891150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb541026f0 0x7fbb54102b70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009900 tx=0x7fbb3c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:20:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.040+0000 7fbb53200640 1 -- 172.21.15.47:0/662891150 shutdown_connections 2024-03-21T11:20:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.040+0000 7fbb53200640 1 --2- 172.21.15.47:0/662891150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb541030b0 0x7fbb54109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.040+0000 7fbb53200640 1 --2- 172.21.15.47:0/662891150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb541026f0 0x7fbb54102b70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.040+0000 7fbb53200640 1 --2- 172.21.15.47:0/662891150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb54101500 0x7fbb54101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.040+0000 7fbb53200640 1 -- 172.21.15.47:0/662891150 >> 172.21.15.47:0/662891150 conn(0x7fbb540fca70 msgr2=0x7fbb540feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.040+0000 7fbb53200640 1 -- 172.21.15.47:0/662891150 shutdown_connections 2024-03-21T11:20:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.040+0000 7fbb53200640 1 -- 172.21.15.47:0/662891150 wait complete. 2024-03-21T11:20:03.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.041+0000 7fbb53200640 1 Processor -- start 2024-03-21T11:20:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.041+0000 7fbb53200640 1 -- start start 2024-03-21T11:20:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb53200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb54101500 0x7fbb54197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb53200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb541026f0 0x7fbb54197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb53200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb541030b0 0x7fbb5419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb53200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5410d1e0 con 0x7fbb541030b0 2024-03-21T11:20:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb53200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5410d060 con 0x7fbb541026f0 2024-03-21T11:20:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb53200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5410d360 con 0x7fbb54101500 2024-03-21T11:20:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb51e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb54101500 0x7fbb54197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb52800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb541030b0 0x7fbb5419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb51e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb54101500 0x7fbb54197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37896/0 (socket says 172.21.15.47:37896) 2024-03-21T11:20:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb51e00640 1 -- 172.21.15.47:0/853649644 learned_addr learned my addr 172.21.15.47:0/853649644 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb4be00640 1 --2- 172.21.15.47:0/853649644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb541026f0 0x7fbb54197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb51e00640 1 -- 172.21.15.47:0/853649644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb541026f0 msgr2=0x7fbb54197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb51e00640 1 --2- 172.21.15.47:0/853649644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb541026f0 0x7fbb54197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb51e00640 1 -- 172.21.15.47:0/853649644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb541030b0 msgr2=0x7fbb5419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb51e00640 1 --2- 172.21.15.47:0/853649644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb541030b0 0x7fbb5419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb51e00640 1 -- 172.21.15.47:0/853649644 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5419f420 con 0x7fbb54101500 2024-03-21T11:20:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.042+0000 7fbb4be00640 1 --2- 172.21.15.47:0/853649644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb541026f0 0x7fbb54197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:20:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.043+0000 7fbb51e00640 1 --2- 172.21.15.47:0/853649644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb54101500 0x7fbb54197800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400e9c0 tx=0x7fbb4400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.043+0000 7fbb4aa00640 1 -- 172.21.15.47:0/853649644 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400cde0 con 0x7fbb54101500 2024-03-21T11:20:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.043+0000 7fbb4aa00640 1 -- 172.21.15.47:0/853649644 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb44010910 con 0x7fbb54101500 2024-03-21T11:20:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.043+0000 7fbb4aa00640 1 -- 172.21.15.47:0/853649644 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44018a00 con 0x7fbb54101500 2024-03-21T11:20:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.043+0000 7fbb53200640 1 -- 172.21.15.47:0/853649644 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb5419f710 con 0x7fbb54101500 2024-03-21T11:20:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.043+0000 7fbb53200640 1 -- 172.21.15.47:0/853649644 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5419fbe0 con 0x7fbb54101500 2024-03-21T11:20:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.044+0000 7fbb53200640 1 -- 172.21.15.47:0/853649644 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb541087c0 con 0x7fbb54101500 2024-03-21T11:20:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.046+0000 7fbb4aa00640 1 -- 172.21.15.47:0/853649644 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbb4401f020 con 0x7fbb54101500 2024-03-21T11:20:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.047+0000 7fbb4aa00640 1 --2- 172.21.15.47:0/853649644 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb380a2fa0 0x7fbb380a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.048+0000 7fbb4aa00640 1 -- 172.21.15.47:0/853649644 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbb440cf1f0 con 0x7fbb54101500 2024-03-21T11:20:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.048+0000 7fbb4be00640 1 --2- 172.21.15.47:0/853649644 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb380a2fa0 0x7fbb380a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:03.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.048+0000 7fbb4be00640 1 --2- 172.21.15.47:0/853649644 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb380a2fa0 0x7fbb380a5460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c006010 tx=0x7fbb3c02c890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:03.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.051+0000 7fbb4aa00640 1 -- 172.21.15.47:0/853649644 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbb4409c420 con 0x7fbb54101500 2024-03-21T11:20:03.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.257+0000 7fbb53200640 1 -- 172.21.15.47:0/853649644 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb540622d0 con 0x7fbb54101500 2024-03-21T11:20:03.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.259+0000 7fbb4aa00640 1 -- 172.21.15.47:0/853649644 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18455 (secure 0 0 0) 0x7fbb4409bd20 con 0x7fbb54101500 2024-03-21T11:20:03.260 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:03.260 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":82,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:19:50.716171+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:20:03.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.261+0000 7fbb53200640 1 -- 172.21.15.47:0/853649644 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb380a2fa0 msgr2=0x7fbb380a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:03.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.261+0000 7fbb53200640 1 --2- 172.21.15.47:0/853649644 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb380a2fa0 0x7fbb380a5460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c006010 tx=0x7fbb3c02c890 comp rx=0 tx=0).stop 2024-03-21T11:20:03.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.261+0000 7fbb53200640 1 -- 172.21.15.47:0/853649644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb54101500 msgr2=0x7fbb54197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.261+0000 7fbb53200640 1 --2- 172.21.15.47:0/853649644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb54101500 0x7fbb54197800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400e9c0 tx=0x7fbb4400ee80 comp rx=0 tx=0).stop 2024-03-21T11:20:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.262+0000 7fbb53200640 1 -- 172.21.15.47:0/853649644 shutdown_connections 2024-03-21T11:20:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.262+0000 7fbb53200640 1 --2- 172.21.15.47:0/853649644 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb380a2fa0 0x7fbb380a5460 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.262+0000 7fbb53200640 1 --2- 172.21.15.47:0/853649644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb541030b0 0x7fbb5419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.262+0000 7fbb53200640 1 --2- 172.21.15.47:0/853649644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb541026f0 0x7fbb54197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.262+0000 7fbb53200640 1 --2- 172.21.15.47:0/853649644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb54101500 0x7fbb54197800 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.262+0000 7fbb53200640 1 -- 172.21.15.47:0/853649644 >> 172.21.15.47:0/853649644 conn(0x7fbb540fca70 msgr2=0x7fbb54105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.262+0000 7fbb53200640 1 -- 172.21.15.47:0/853649644 shutdown_connections 2024-03-21T11:20:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.262+0000 7fbb53200640 1 -- 172.21.15.47:0/853649644 wait complete. 2024-03-21T11:20:03.284 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T11:15:35.545245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T11:20:03.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T11:20:03.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.519+0000 7f414b200640 1 -- 172.21.15.47:0/78419628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f414c107260 msgr2=0x7f414c1096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:03.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.519+0000 7f414b200640 1 --2- 172.21.15.47:0/78419628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f414c107260 0x7f414c1096b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4144009ca0 tx=0x7f414402c100 comp rx=0 tx=0).stop 2024-03-21T11:20:03.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.519+0000 7f414b200640 1 -- 172.21.15.47:0/78419628 shutdown_connections 2024-03-21T11:20:03.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.519+0000 7f414b200640 1 --2- 172.21.15.47:0/78419628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f414c107260 0x7f414c1096b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.519+0000 7f414b200640 1 --2- 172.21.15.47:0/78419628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c104930 0x7f414c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.519+0000 7f414b200640 1 --2- 172.21.15.47:0/78419628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f414c102000 0x7f414c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.519+0000 7f414b200640 1 -- 172.21.15.47:0/78419628 >> 172.21.15.47:0/78419628 conn(0x7f414c0fba10 msgr2=0x7f414c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:03.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.519+0000 7f414b200640 1 -- 172.21.15.47:0/78419628 shutdown_connections 2024-03-21T11:20:03.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.519+0000 7f414b200640 1 -- 172.21.15.47:0/78419628 wait complete. 2024-03-21T11:20:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.520+0000 7f414b200640 1 Processor -- start 2024-03-21T11:20:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.520+0000 7f414b200640 1 -- start start 2024-03-21T11:20:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.520+0000 7f414b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f414c102000 0x7f414c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.520+0000 7f414b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c104930 0x7f414c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.520+0000 7f414b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f414c107260 0x7f414c1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:03.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.520+0000 7f414b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c10d210 con 0x7f414c104930 2024-03-21T11:20:03.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.520+0000 7f414b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c10d090 con 0x7f414c102000 2024-03-21T11:20:03.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.520+0000 7f414b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c10d390 con 0x7f414c107260 2024-03-21T11:20:03.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.521+0000 7f4149400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c104930 0x7f414c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:03.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.521+0000 7f414a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f414c107260 0x7f414c1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:03.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.521+0000 7f4149400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c104930 0x7f414c199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35034/0 (socket says 172.21.15.47:35034) 2024-03-21T11:20:03.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.521+0000 7f4149400640 1 -- 172.21.15.47:0/3228372382 learned_addr learned my addr 172.21.15.47:0/3228372382 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:03.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.521+0000 7f414a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f414c107260 0x7f414c1a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37910/0 (socket says 172.21.15.47:37910) 2024-03-21T11:20:03.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.521+0000 7f4149e00640 1 --2- 172.21.15.47:0/3228372382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f414c102000 0x7f414c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:03.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.521+0000 7f414a800640 1 -- 172.21.15.47:0/3228372382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f414c102000 msgr2=0x7f414c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:03.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.521+0000 7f414a800640 1 --2- 172.21.15.47:0/3228372382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f414c102000 0x7f414c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.521+0000 7f414a800640 1 -- 172.21.15.47:0/3228372382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c104930 msgr2=0x7f414c199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:03.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.521+0000 7f414a800640 1 --2- 172.21.15.47:0/3228372382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c104930 0x7f414c199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.521+0000 7f414a800640 1 -- 172.21.15.47:0/3228372382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414c1a1640 con 0x7f414c107260 2024-03-21T11:20:03.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.521+0000 7f414a800640 1 --2- 172.21.15.47:0/3228372382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f414c107260 0x7f414c1a0fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4144037040 tx=0x7f4144035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:03.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.522+0000 7f4142a00640 1 -- 172.21.15.47:0/3228372382 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144039040 con 0x7f414c107260 2024-03-21T11:20:03.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.522+0000 7f4142a00640 1 -- 172.21.15.47:0/3228372382 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4144044070 con 0x7f414c107260 2024-03-21T11:20:03.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.522+0000 7f4142a00640 1 -- 172.21.15.47:0/3228372382 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144004350 con 0x7f414c107260 2024-03-21T11:20:03.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.522+0000 7f414b200640 1 -- 172.21.15.47:0/3228372382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414c1a18d0 con 0x7f414c107260 2024-03-21T11:20:03.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.522+0000 7f414b200640 1 -- 172.21.15.47:0/3228372382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414c1a1d40 con 0x7f414c107260 2024-03-21T11:20:03.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.523+0000 7f414b200640 1 -- 172.21.15.47:0/3228372382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f414c108d50 con 0x7f414c107260 2024-03-21T11:20:03.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.525+0000 7f4142a00640 1 -- 172.21.15.47:0/3228372382 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f41440044f0 con 0x7f414c107260 2024-03-21T11:20:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.526+0000 7f4142a00640 1 --2- 172.21.15.47:0/3228372382 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41300a2fa0 0x7f41300a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.526+0000 7f4142a00640 1 -- 172.21.15.47:0/3228372382 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f41440f1460 con 0x7f414c107260 2024-03-21T11:20:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.527+0000 7f4149e00640 1 --2- 172.21.15.47:0/3228372382 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41300a2fa0 0x7f41300a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.527+0000 7f4149e00640 1 --2- 172.21.15.47:0/3228372382 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41300a2fa0 0x7f41300a5460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f413c009bd0 tx=0x7f413c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:03.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.531+0000 7f4142a00640 1 -- 172.21.15.47:0/3228372382 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f41440be690 con 0x7f414c107260 2024-03-21T11:20:03.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.733+0000 7f414b200640 1 -- 172.21.15.47:0/3228372382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f414c0622a0 con 0x7f414c107260 2024-03-21T11:20:03.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.733+0000 7f4142a00640 1 -- 172.21.15.47:0/3228372382 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41440098b0 con 0x7f414c107260 2024-03-21T11:20:03.734 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T11:20:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.736+0000 7f414b200640 1 -- 172.21.15.47:0/3228372382 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41300a2fa0 msgr2=0x7f41300a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.736+0000 7f414b200640 1 --2- 172.21.15.47:0/3228372382 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41300a2fa0 0x7f41300a5460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f413c009bd0 tx=0x7f413c005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.736+0000 7f414b200640 1 -- 172.21.15.47:0/3228372382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f414c107260 msgr2=0x7f414c1a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.736+0000 7f414b200640 1 --2- 172.21.15.47:0/3228372382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f414c107260 0x7f414c1a0fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4144037040 tx=0x7f4144035d10 comp rx=0 tx=0).stop 2024-03-21T11:20:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.736+0000 7f414b200640 1 -- 172.21.15.47:0/3228372382 shutdown_connections 2024-03-21T11:20:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.736+0000 7f414b200640 1 --2- 172.21.15.47:0/3228372382 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41300a2fa0 0x7f41300a5460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.736+0000 7f414b200640 1 --2- 172.21.15.47:0/3228372382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f414c107260 0x7f414c1a0fd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.736+0000 7f414b200640 1 --2- 172.21.15.47:0/3228372382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f414c104930 0x7f414c199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.736+0000 7f414b200640 1 --2- 172.21.15.47:0/3228372382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f414c102000 0x7f414c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.736+0000 7f414b200640 1 -- 172.21.15.47:0/3228372382 >> 172.21.15.47:0/3228372382 conn(0x7f414c0fba10 msgr2=0x7f414c1016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:03.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.737+0000 7f414b200640 1 -- 172.21.15.47:0/3228372382 shutdown_connections 2024-03-21T11:20:03.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.737+0000 7f414b200640 1 -- 172.21.15.47:0/3228372382 wait complete. 2024-03-21T11:20:03.757 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:03.901 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:03 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:03.901 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:03 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:20:03.901 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:03 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:03.901 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:03 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:03.901 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:03 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:03.901 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:03 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:03.901 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:03 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/853649644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:03.901 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:03 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3228372382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:20:03.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.981+0000 7f93ae800640 1 -- 172.21.15.47:0/3643057066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a80772a0 msgr2=0x7f93a8077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:03.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.981+0000 7f93ae800640 1 --2- 172.21.15.47:0/3643057066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a80772a0 0x7f93a8077700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9390009860 tx=0x7f939002bb70 comp rx=0 tx=0).stop 2024-03-21T11:20:03.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.982+0000 7f93ae800640 1 -- 172.21.15.47:0/3643057066 shutdown_connections 2024-03-21T11:20:03.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.982+0000 7f93ae800640 1 --2- 172.21.15.47:0/3643057066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93a8077c40 0x7f93a80784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.982+0000 7f93ae800640 1 --2- 172.21.15.47:0/3643057066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a80772a0 0x7f93a8077700 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.982+0000 7f93ae800640 1 --2- 172.21.15.47:0/3643057066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a8076060 0x7f93a8076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.982+0000 7f93ae800640 1 -- 172.21.15.47:0/3643057066 >> 172.21.15.47:0/3643057066 conn(0x7f93a80fee90 msgr2=0x7f93a81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:03.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.982+0000 7f93ae800640 1 -- 172.21.15.47:0/3643057066 shutdown_connections 2024-03-21T11:20:03.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.982+0000 7f93ae800640 1 -- 172.21.15.47:0/3643057066 wait complete. 2024-03-21T11:20:03.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.983+0000 7f93ae800640 1 Processor -- start 2024-03-21T11:20:03.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.983+0000 7f93ae800640 1 -- start start 2024-03-21T11:20:03.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93ae800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a8076060 0x7f93a819dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:03.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93ae800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a80772a0 0x7f93a819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:03.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93ae800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93a8077c40 0x7f93a81a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:03.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93ae800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a8113970 con 0x7f93a8077c40 2024-03-21T11:20:03.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93ae800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a81137f0 con 0x7f93a80772a0 2024-03-21T11:20:03.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93ae800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a8113af0 con 0x7f93a8076060 2024-03-21T11:20:03.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93a7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a8076060 0x7f93a819dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:03.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93a7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a8076060 0x7f93a819dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37928/0 (socket says 172.21.15.47:37928) 2024-03-21T11:20:03.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93a7400640 1 -- 172.21.15.47:0/1708844214 learned_addr learned my addr 172.21.15.47:0/1708844214 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:03.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93a6a00640 1 --2- 172.21.15.47:0/1708844214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a80772a0 0x7f93a819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:03.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93a7e00640 1 --2- 172.21.15.47:0/1708844214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93a8077c40 0x7f93a81a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:03.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93a7400640 1 -- 172.21.15.47:0/1708844214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a80772a0 msgr2=0x7f93a819e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:03.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93a7400640 1 --2- 172.21.15.47:0/1708844214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a80772a0 0x7f93a819e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93a7400640 1 -- 172.21.15.47:0/1708844214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93a8077c40 msgr2=0x7f93a81a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:03.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93a7400640 1 --2- 172.21.15.47:0/1708844214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93a8077c40 0x7f93a81a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:03.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.984+0000 7f93a7400640 1 -- 172.21.15.47:0/1708844214 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a81a5c10 con 0x7f93a8076060 2024-03-21T11:20:03.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.985+0000 7f93a7400640 1 --2- 172.21.15.47:0/1708844214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a8076060 0x7f93a819dff0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f939800c420 tx=0x7f939800c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:03.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.985+0000 7f939fe00640 1 -- 172.21.15.47:0/1708844214 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398004b20 con 0x7f93a8076060 2024-03-21T11:20:03.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.985+0000 7f939fe00640 1 -- 172.21.15.47:0/1708844214 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9398003e50 con 0x7f93a8076060 2024-03-21T11:20:03.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.985+0000 7f939fe00640 1 -- 172.21.15.47:0/1708844214 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939800b630 con 0x7f93a8076060 2024-03-21T11:20:03.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.985+0000 7f93ae800640 1 -- 172.21.15.47:0/1708844214 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a81a5f00 con 0x7f93a8076060 2024-03-21T11:20:03.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.985+0000 7f93ae800640 1 -- 172.21.15.47:0/1708844214 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a81a6360 con 0x7f93a8076060 2024-03-21T11:20:03.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.986+0000 7f93ae800640 1 -- 172.21.15.47:0/1708844214 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a8110430 con 0x7f93a8076060 2024-03-21T11:20:03.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.988+0000 7f939fe00640 1 -- 172.21.15.47:0/1708844214 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f939801d020 con 0x7f93a8076060 2024-03-21T11:20:03.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.990+0000 7f939fe00640 1 --2- 172.21.15.47:0/1708844214 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f938c0a2fa0 0x7f938c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:03.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.990+0000 7f939fe00640 1 -- 172.21.15.47:0/1708844214 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f93980cf080 con 0x7f93a8076060 2024-03-21T11:20:03.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.990+0000 7f93a6a00640 1 --2- 172.21.15.47:0/1708844214 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f938c0a2fa0 0x7f938c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:03.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.991+0000 7f93a6a00640 1 --2- 172.21.15.47:0/1708844214 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f938c0a2fa0 0x7f938c0a5460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9390009830 tx=0x7f9390034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:03.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:03.993+0000 7f939fe00640 1 -- 172.21.15.47:0/1708844214 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f939809c2b0 con 0x7f93a8076060 2024-03-21T11:20:04.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:04.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:20:04.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:04.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:04.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:04.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:03 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:04.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:03 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/853649644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:04.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:03 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3228372382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:20:04.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:03 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:04.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:03 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:20:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:03 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:03 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:03 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:03 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:03 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/853649644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:03 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3228372382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:20:04.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:04.236+0000 7f93ae800640 1 -- 172.21.15.47:0/1708844214 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93a80622a0 con 0x7f93a8076060 2024-03-21T11:20:04.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:04.237+0000 7f939fe00640 1 -- 172.21.15.47:0/1708844214 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f939809bbb0 con 0x7f93a8076060 2024-03-21T11:20:04.238 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-21T11:20:04.238 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 1 2024-03-21T11:20:04.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:04.240+0000 7f93ae800640 1 -- 172.21.15.47:0/1708844214 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f938c0a2fa0 msgr2=0x7f938c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:04.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:04.240+0000 7f93ae800640 1 --2- 172.21.15.47:0/1708844214 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f938c0a2fa0 0x7f938c0a5460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9390009830 tx=0x7f9390034ad0 comp rx=0 tx=0).stop 2024-03-21T11:20:04.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:04.240+0000 7f93ae800640 1 -- 172.21.15.47:0/1708844214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a8076060 msgr2=0x7f93a819dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:04.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:04.240+0000 7f93ae800640 1 --2- 172.21.15.47:0/1708844214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a8076060 0x7f93a819dff0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f939800c420 tx=0x7f939800c8e0 comp rx=0 tx=0).stop 2024-03-21T11:20:04.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:04.241+0000 7f93ae800640 1 -- 172.21.15.47:0/1708844214 shutdown_connections 2024-03-21T11:20:04.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:04.241+0000 7f93ae800640 1 --2- 172.21.15.47:0/1708844214 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f938c0a2fa0 0x7f938c0a5460 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:04.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:04.241+0000 7f93ae800640 1 --2- 172.21.15.47:0/1708844214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93a8077c40 0x7f93a81a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:04.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:04.241+0000 7f93ae800640 1 --2- 172.21.15.47:0/1708844214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a80772a0 0x7f93a819e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:04.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:04.241+0000 7f93ae800640 1 --2- 172.21.15.47:0/1708844214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a8076060 0x7f93a819dff0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:04.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:04.241+0000 7f93ae800640 1 -- 172.21.15.47:0/1708844214 >> 172.21.15.47:0/1708844214 conn(0x7f93a80fee90 msgr2=0x7f93a8100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:04.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:04.241+0000 7f93ae800640 1 -- 172.21.15.47:0/1708844214 shutdown_connections 2024-03-21T11:20:04.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:04.241+0000 7f93ae800640 1 -- 172.21.15.47:0/1708844214 wait complete. 2024-03-21T11:20:04.951 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:04 smithi102 ceph-mon[36748]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:04.951 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:04 smithi102 ceph-mon[36748]: Deploying daemon mds.l on smithi102 2024-03-21T11:20:04.951 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:04 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1708844214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:04 smithi047 ceph-mon[31351]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:04 smithi047 ceph-mon[31351]: Deploying daemon mds.l on smithi102 2024-03-21T11:20:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:04 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1708844214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:05.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:04 smithi100 ceph-mon[36245]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:05.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:04 smithi100 ceph-mon[36245]: Deploying daemon mds.l on smithi102 2024-03-21T11:20:05.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:04 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1708844214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:05.241 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:20:05 smithi102 systemd[1]: Starting Ceph mds.l for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:20:05.636 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:20:05 smithi102 podman[67046]: 2024-03-21 11:20:05.255538523 +0000 UTC m=+0.017525722 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:20:05.636 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:20:05 smithi102 podman[67046]: 2024-03-21 11:20:05.38422885 +0000 UTC m=+0.146216090 container create caa307b0576baaddcff0d4d1953f8d76d0d3e120e5ebdf7ac8d48f5122cb0eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-21T11:20:05.991 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:20:05 smithi102 podman[67046]: 2024-03-21 11:20:05.636959956 +0000 UTC m=+0.398947135 container init caa307b0576baaddcff0d4d1953f8d76d0d3e120e5ebdf7ac8d48f5122cb0eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-l, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-21T11:20:05.991 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:20:05 smithi102 podman[67046]: 2024-03-21 11:20:05.639665034 +0000 UTC m=+0.401652213 container start caa307b0576baaddcff0d4d1953f8d76d0d3e120e5ebdf7ac8d48f5122cb0eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:20:05.991 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:20:05 smithi102 ceph-mds[67066]: -- 172.21.15.102:0/3771288853 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b46ca0ba40 con 0x55b46c9dec00 2024-03-21T11:20:05.991 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:20:05 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-l[67062]: starting mds.l at 2024-03-21T11:20:05.991 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:20:05 smithi102 bash[67046]: caa307b0576baaddcff0d4d1953f8d76d0d3e120e5ebdf7ac8d48f5122cb0eb9 2024-03-21T11:20:05.991 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:20:05 smithi102 systemd[1]: Started Ceph mds.l for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:20:06.263 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:06.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.496+0000 7f636cc00640 1 -- 172.21.15.47:0/211218132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6368107ce0 msgr2=0x7f6368108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:06.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.496+0000 7f636cc00640 1 --2- 172.21.15.47:0/211218132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6368107ce0 0x7f6368108100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6358009900 tx=0x7f635802bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.497+0000 7f636cc00640 1 -- 172.21.15.47:0/211218132 shutdown_connections 2024-03-21T11:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.497+0000 7f636cc00640 1 --2- 172.21.15.47:0/211218132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6368109890 0x7f6368110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.497+0000 7f636cc00640 1 --2- 172.21.15.47:0/211218132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6368108ed0 0x7f6368109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.497+0000 7f636cc00640 1 --2- 172.21.15.47:0/211218132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6368107ce0 0x7f6368108100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.497+0000 7f636cc00640 1 -- 172.21.15.47:0/211218132 >> 172.21.15.47:0/211218132 conn(0x7f6368076cb0 msgr2=0x7f63680790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.497+0000 7f636cc00640 1 -- 172.21.15.47:0/211218132 shutdown_connections 2024-03-21T11:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.497+0000 7f636cc00640 1 -- 172.21.15.47:0/211218132 wait complete. 2024-03-21T11:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.498+0000 7f636cc00640 1 Processor -- start 2024-03-21T11:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.498+0000 7f636cc00640 1 -- start start 2024-03-21T11:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.498+0000 7f636cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6368107ce0 0x7f636819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.498+0000 7f636cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6368108ed0 0x7f636819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.498+0000 7f636cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6368109890 0x7f63681a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.498+0000 7f636cc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6368113930 con 0x7f6368109890 2024-03-21T11:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.498+0000 7f636cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63681137b0 con 0x7f6368108ed0 2024-03-21T11:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.498+0000 7f636cc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6368113ab0 con 0x7f6368107ce0 2024-03-21T11:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.498+0000 7f6366800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6368109890 0x7f63681a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.498+0000 7f6365e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6368107ce0 0x7f636819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.498+0000 7f6365e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6368107ce0 0x7f636819dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37950/0 (socket says 172.21.15.47:37950) 2024-03-21T11:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.498+0000 7f6365400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6368108ed0 0x7f636819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.498+0000 7f6365e00640 1 -- 172.21.15.47:0/369544179 learned_addr learned my addr 172.21.15.47:0/369544179 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.499+0000 7f6365e00640 1 -- 172.21.15.47:0/369544179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6368108ed0 msgr2=0x7f636819e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.499+0000 7f6365e00640 1 --2- 172.21.15.47:0/369544179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6368108ed0 0x7f636819e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.499+0000 7f6365e00640 1 -- 172.21.15.47:0/369544179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6368109890 msgr2=0x7f63681a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.499+0000 7f6365e00640 1 --2- 172.21.15.47:0/369544179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6368109890 0x7f63681a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.499+0000 7f6365e00640 1 -- 172.21.15.47:0/369544179 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63681a5bf0 con 0x7f6368107ce0 2024-03-21T11:20:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.499+0000 7f6365e00640 1 --2- 172.21.15.47:0/369544179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6368107ce0 0x7f636819dfd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6358000c00 tx=0x7f635802c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.499+0000 7f635ea00640 1 -- 172.21.15.47:0/369544179 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358041070 con 0x7f6368107ce0 2024-03-21T11:20:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.499+0000 7f635ea00640 1 -- 172.21.15.47:0/369544179 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6358004450 con 0x7f6368107ce0 2024-03-21T11:20:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.499+0000 7f636cc00640 1 -- 172.21.15.47:0/369544179 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63681a5e80 con 0x7f6368107ce0 2024-03-21T11:20:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.499+0000 7f635ea00640 1 -- 172.21.15.47:0/369544179 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358034440 con 0x7f6368107ce0 2024-03-21T11:20:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.499+0000 7f636cc00640 1 -- 172.21.15.47:0/369544179 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63681a62e0 con 0x7f6368107ce0 2024-03-21T11:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.500+0000 7f636cc00640 1 -- 172.21.15.47:0/369544179 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636810f000 con 0x7f6368107ce0 2024-03-21T11:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.503+0000 7f635ea00640 1 -- 172.21.15.47:0/369544179 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6358044070 con 0x7f6368107ce0 2024-03-21T11:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.504+0000 7f635ea00640 1 --2- 172.21.15.47:0/369544179 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f634c0a2fa0 0x7f634c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.504+0000 7f635ea00640 1 -- 172.21.15.47:0/369544179 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f63580f0bc0 con 0x7f6368107ce0 2024-03-21T11:20:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.504+0000 7f6365400640 1 --2- 172.21.15.47:0/369544179 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f634c0a2fa0 0x7f634c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.505+0000 7f6365400640 1 --2- 172.21.15.47:0/369544179 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f634c0a2fa0 0x7f634c0a5460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f636819f4e0 tx=0x7f6350005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.506+0000 7f635ea00640 1 -- 172.21.15.47:0/369544179 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f63580bd830 con 0x7f6368107ce0 2024-03-21T11:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.754+0000 7f636cc00640 1 -- 172.21.15.47:0/369544179 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f636802d190 con 0x7f6368107ce0 2024-03-21T11:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.755+0000 7f635ea00640 1 -- 172.21.15.47:0/369544179 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f63580c18d0 con 0x7f6368107ce0 2024-03-21T11:20:06.756 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:06.756 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T11:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 2 2024-03-21T11:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.758+0000 7f636cc00640 1 -- 172.21.15.47:0/369544179 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f634c0a2fa0 msgr2=0x7f634c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.758+0000 7f636cc00640 1 --2- 172.21.15.47:0/369544179 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f634c0a2fa0 0x7f634c0a5460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f636819f4e0 tx=0x7f6350005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.758+0000 7f636cc00640 1 -- 172.21.15.47:0/369544179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6368107ce0 msgr2=0x7f636819dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.758+0000 7f636cc00640 1 --2- 172.21.15.47:0/369544179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6368107ce0 0x7f636819dfd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6358000c00 tx=0x7f635802c640 comp rx=0 tx=0).stop 2024-03-21T11:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.759+0000 7f636cc00640 1 -- 172.21.15.47:0/369544179 shutdown_connections 2024-03-21T11:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.759+0000 7f636cc00640 1 --2- 172.21.15.47:0/369544179 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f634c0a2fa0 0x7f634c0a5460 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.759+0000 7f636cc00640 1 --2- 172.21.15.47:0/369544179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6368109890 0x7f63681a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.759+0000 7f636cc00640 1 --2- 172.21.15.47:0/369544179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6368108ed0 0x7f636819e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.759+0000 7f636cc00640 1 --2- 172.21.15.47:0/369544179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6368107ce0 0x7f636819dfd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.759+0000 7f636cc00640 1 -- 172.21.15.47:0/369544179 >> 172.21.15.47:0/369544179 conn(0x7f6368076cb0 msgr2=0x7f636810c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.759+0000 7f636cc00640 1 -- 172.21.15.47:0/369544179 shutdown_connections 2024-03-21T11:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:06.759+0000 7f636cc00640 1 -- 172.21.15.47:0/369544179 wait complete. 2024-03-21T11:20:06.790 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:06 smithi100 ceph-mon[36245]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:06.790 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:06 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.102:6832/1794606435,v1:172.21.15.102:6833/1794606435] up:boot 2024-03-21T11:20:06.791 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:06 smithi100 ceph-mon[36245]: fsmap 1 up:standby 2024-03-21T11:20:06.791 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T11:20:06.791 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T11:20:06.791 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:06.791 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:06.791 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:06.791 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:06.791 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:06.791 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:06 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:06.791 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:06 smithi100 ceph-mon[36245]: Deploying daemon mds.k on smithi100 2024-03-21T11:20:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:06 smithi102 ceph-mon[36748]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:06 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.102:6832/1794606435,v1:172.21.15.102:6833/1794606435] up:boot 2024-03-21T11:20:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:06 smithi102 ceph-mon[36748]: fsmap 1 up:standby 2024-03-21T11:20:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T11:20:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T11:20:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:06.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:06.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:06 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:06.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:06 smithi102 ceph-mon[36748]: Deploying daemon mds.k on smithi100 2024-03-21T11:20:07.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:06 smithi047 ceph-mon[31351]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:07.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:06 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.102:6832/1794606435,v1:172.21.15.102:6833/1794606435] up:boot 2024-03-21T11:20:07.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:06 smithi047 ceph-mon[31351]: fsmap 1 up:standby 2024-03-21T11:20:07.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T11:20:07.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T11:20:07.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:07.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:07.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:07.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:07.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:07.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:06 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:07.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:06 smithi047 ceph-mon[31351]: Deploying daemon mds.k on smithi100 2024-03-21T11:20:07.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:07 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/369544179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:07.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:07 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/369544179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:07 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/369544179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:08.317 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:20:08 smithi100 systemd[1]: Starting Ceph mds.k for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:20:08.568 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:20:08 smithi100 podman[65356]: 2024-03-21 11:20:08.219982194 +0000 UTC m=+0.017594763 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:20:08.569 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:20:08 smithi100 podman[65356]: 2024-03-21 11:20:08.340990826 +0000 UTC m=+0.138603414 container create 17c5e3dda4ad7c4b6a4aaf08fef89d4139e774972e351a069581d509377d31a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-k, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-21T11:20:08.569 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:20:08 smithi100 podman[65356]: 2024-03-21 11:20:08.56900717 +0000 UTC m=+0.366619748 container init 17c5e3dda4ad7c4b6a4aaf08fef89d4139e774972e351a069581d509377d31a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-k, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-21T11:20:08.781 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:08.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:08 smithi100 ceph-mon[36245]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:08.893 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:20:08 smithi100 podman[65356]: 2024-03-21 11:20:08.572067194 +0000 UTC m=+0.369679759 container start 17c5e3dda4ad7c4b6a4aaf08fef89d4139e774972e351a069581d509377d31a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-k, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True) 2024-03-21T11:20:08.893 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:20:08 smithi100 ceph-mds[65375]: -- 172.21.15.100:0/2615492375 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559b415a1a40 con 0x559b41574c00 2024-03-21T11:20:08.893 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:20:08 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-k[65371]: starting mds.k at 2024-03-21T11:20:08.893 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:20:08 smithi100 bash[65356]: 17c5e3dda4ad7c4b6a4aaf08fef89d4139e774972e351a069581d509377d31a3 2024-03-21T11:20:08.893 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:20:08 smithi100 systemd[1]: Started Ceph mds.k for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:20:08.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:08 smithi102 ceph-mon[36748]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:08.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.993+0000 7f8783000640 1 -- 172.21.15.47:0/3514858668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f877c103260 msgr2=0x7f877c109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:08.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.993+0000 7f8783000640 1 --2- 172.21.15.47:0/3514858668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f877c103260 0x7f877c109af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8774009ca0 tx=0x7f877402c0f0 comp rx=0 tx=0).stop 2024-03-21T11:20:08.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.993+0000 7f8783000640 1 -- 172.21.15.47:0/3514858668 shutdown_connections 2024-03-21T11:20:08.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.993+0000 7f8783000640 1 --2- 172.21.15.47:0/3514858668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f877c103260 0x7f877c109af0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:08.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.993+0000 7f8783000640 1 --2- 172.21.15.47:0/3514858668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f877c1028a0 0x7f877c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:08.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.993+0000 7f8783000640 1 --2- 172.21.15.47:0/3514858668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877c1016b0 0x7f877c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:08.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.993+0000 7f8783000640 1 -- 172.21.15.47:0/3514858668 >> 172.21.15.47:0/3514858668 conn(0x7f877c0fcc80 msgr2=0x7f877c0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:08.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.993+0000 7f8783000640 1 -- 172.21.15.47:0/3514858668 shutdown_connections 2024-03-21T11:20:08.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.993+0000 7f8783000640 1 -- 172.21.15.47:0/3514858668 wait complete. 2024-03-21T11:20:08.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.994+0000 7f8783000640 1 Processor -- start 2024-03-21T11:20:08.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.994+0000 7f8783000640 1 -- start start 2024-03-21T11:20:08.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f8783000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f877c1016b0 0x7f877c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:08.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f8783000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877c1028a0 0x7f877c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:08.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f8783000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f877c103260 0x7f877c1a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:08.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f8783000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c10d310 con 0x7f877c1016b0 2024-03-21T11:20:08.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f8783000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c10d190 con 0x7f877c103260 2024-03-21T11:20:08.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f8783000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c10d490 con 0x7f877c1028a0 2024-03-21T11:20:08.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f877be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f877c1016b0 0x7f877c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:08.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f8781000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f877c103260 0x7f877c1a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:08.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f877be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f877c1016b0 0x7f877c199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34842/0 (socket says 172.21.15.47:34842) 2024-03-21T11:20:08.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f877be00640 1 -- 172.21.15.47:0/2131349147 learned_addr learned my addr 172.21.15.47:0/2131349147 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:08.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f877b400640 1 --2- 172.21.15.47:0/2131349147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877c1028a0 0x7f877c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:08.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f8781000640 1 -- 172.21.15.47:0/2131349147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877c1028a0 msgr2=0x7f877c19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:08.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f8781000640 1 --2- 172.21.15.47:0/2131349147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877c1028a0 0x7f877c19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:08.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f8781000640 1 -- 172.21.15.47:0/2131349147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f877c1016b0 msgr2=0x7f877c199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:08.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f8781000640 1 --2- 172.21.15.47:0/2131349147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f877c1016b0 0x7f877c199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f8781000640 1 -- 172.21.15.47:0/2131349147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c1a1790 con 0x7f877c103260 2024-03-21T11:20:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.995+0000 7f877be00640 1 --2- 172.21.15.47:0/2131349147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f877c1016b0 0x7f877c199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:20:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.996+0000 7f8781000640 1 --2- 172.21.15.47:0/2131349147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f877c103260 0x7f877c1a1120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8774037040 tx=0x7f8774032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.996+0000 7f8778c00640 1 -- 172.21.15.47:0/2131349147 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774041020 con 0x7f877c103260 2024-03-21T11:20:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.996+0000 7f8778c00640 1 -- 172.21.15.47:0/2131349147 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8774034cd0 con 0x7f877c103260 2024-03-21T11:20:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.996+0000 7f8783000640 1 -- 172.21.15.47:0/2131349147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877c1a1a20 con 0x7f877c103260 2024-03-21T11:20:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.996+0000 7f8778c00640 1 -- 172.21.15.47:0/2131349147 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774039040 con 0x7f877c103260 2024-03-21T11:20:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.996+0000 7f8783000640 1 -- 172.21.15.47:0/2131349147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877c1a1f30 con 0x7f877c103260 2024-03-21T11:20:09.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.997+0000 7f8783000640 1 -- 172.21.15.47:0/2131349147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877c1a2160 con 0x7f877c103260 2024-03-21T11:20:09.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:08.999+0000 7f8778c00640 1 -- 172.21.15.47:0/2131349147 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8774044020 con 0x7f877c103260 2024-03-21T11:20:09.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.001+0000 7f8778c00640 1 --2- 172.21.15.47:0/2131349147 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f87600a2fa0 0x7f87600a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:09.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.001+0000 7f8778c00640 1 -- 172.21.15.47:0/2131349147 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f87740f0d00 con 0x7f877c103260 2024-03-21T11:20:09.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.001+0000 7f877be00640 1 --2- 172.21.15.47:0/2131349147 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f87600a2fa0 0x7f87600a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:09.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.002+0000 7f877be00640 1 --2- 172.21.15.47:0/2131349147 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f87600a2fa0 0x7f87600a5460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f876c0058c0 tx=0x7f876c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.005+0000 7f8778c00640 1 -- 172.21.15.47:0/2131349147 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f877404b020 con 0x7f877c103260 2024-03-21T11:20:09.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:08 smithi047 ceph-mon[31351]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:09.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.252+0000 7f8783000640 1 -- 172.21.15.47:0/2131349147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f877c1a2330 con 0x7f877c103260 2024-03-21T11:20:09.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.253+0000 7f8778c00640 1 -- 172.21.15.47:0/2131349147 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f87740bdfc0 con 0x7f877c103260 2024-03-21T11:20:09.254 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:09.254 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T11:20:09.254 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 3 2024-03-21T11:20:09.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.256+0000 7f8783000640 1 -- 172.21.15.47:0/2131349147 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f87600a2fa0 msgr2=0x7f87600a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:09.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.256+0000 7f8783000640 1 --2- 172.21.15.47:0/2131349147 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f87600a2fa0 0x7f87600a5460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f876c0058c0 tx=0x7f876c005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:09.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.256+0000 7f8783000640 1 -- 172.21.15.47:0/2131349147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f877c103260 msgr2=0x7f877c1a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:09.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.256+0000 7f8783000640 1 --2- 172.21.15.47:0/2131349147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f877c103260 0x7f877c1a1120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8774037040 tx=0x7f8774032ea0 comp rx=0 tx=0).stop 2024-03-21T11:20:09.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.256+0000 7f8783000640 1 -- 172.21.15.47:0/2131349147 shutdown_connections 2024-03-21T11:20:09.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.256+0000 7f8783000640 1 --2- 172.21.15.47:0/2131349147 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f87600a2fa0 0x7f87600a5460 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:09.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.256+0000 7f8783000640 1 --2- 172.21.15.47:0/2131349147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f877c103260 0x7f877c1a1120 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:09.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.256+0000 7f8783000640 1 --2- 172.21.15.47:0/2131349147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877c1028a0 0x7f877c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:09.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.256+0000 7f8783000640 1 --2- 172.21.15.47:0/2131349147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f877c1016b0 0x7f877c199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:09.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.256+0000 7f8783000640 1 -- 172.21.15.47:0/2131349147 >> 172.21.15.47:0/2131349147 conn(0x7f877c0fcc80 msgr2=0x7f877c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:09.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.257+0000 7f8783000640 1 -- 172.21.15.47:0/2131349147 shutdown_connections 2024-03-21T11:20:09.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:09.257+0000 7f8783000640 1 -- 172.21.15.47:0/2131349147 wait complete. 2024-03-21T11:20:09.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:09 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.100:6832/904714287,v1:172.21.15.100:6833/904714287] up:boot 2024-03-21T11:20:09.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:09 smithi102 ceph-mon[36748]: fsmap 2 up:standby 2024-03-21T11:20:09.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:09 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T11:20:09.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:09 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:09.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:09 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:09.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:09 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:09.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:09 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:09.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:09 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:09.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:09 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:09.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:09 smithi102 ceph-mon[36748]: Deploying daemon mds.j on smithi047 2024-03-21T11:20:09.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:09 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2131349147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:10.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:09 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.100:6832/904714287,v1:172.21.15.100:6833/904714287] up:boot 2024-03-21T11:20:10.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:09 smithi047 ceph-mon[31351]: fsmap 2 up:standby 2024-03-21T11:20:10.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:09 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T11:20:10.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:09 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:10.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:09 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:10.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:09 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:10.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:09 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:10.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:09 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:10.006 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:09 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:10.006 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:09 smithi047 ceph-mon[31351]: Deploying daemon mds.j on smithi047 2024-03-21T11:20:10.006 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:09 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2131349147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:10.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:09 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.100:6832/904714287,v1:172.21.15.100:6833/904714287] up:boot 2024-03-21T11:20:10.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:09 smithi100 ceph-mon[36245]: fsmap 2 up:standby 2024-03-21T11:20:10.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:09 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T11:20:10.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:09 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:10.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:09 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:10.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:09 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:10.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:09 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:10.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:09 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:10.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:09 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:10.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:09 smithi100 ceph-mon[36245]: Deploying daemon mds.j on smithi047 2024-03-21T11:20:10.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:09 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2131349147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:10.853 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:10 smithi047 ceph-mon[31351]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:11.026 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:10 smithi100 ceph-mon[36245]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:10 smithi102 ceph-mon[36748]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:11.280 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:11.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.500+0000 7fd125c00640 1 -- 172.21.15.47:0/225717815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120105490 msgr2=0x7fd12010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:11.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.500+0000 7fd125c00640 1 --2- 172.21.15.47:0/225717815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120105490 0x7fd12010bd20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd118009ca0 tx=0x7fd11802c0f0 comp rx=0 tx=0).stop 2024-03-21T11:20:11.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.500+0000 7fd125c00640 1 -- 172.21.15.47:0/225717815 shutdown_connections 2024-03-21T11:20:11.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.500+0000 7fd125c00640 1 --2- 172.21.15.47:0/225717815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120105490 0x7fd12010bd20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:11.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.500+0000 7fd125c00640 1 --2- 172.21.15.47:0/225717815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd120104ad0 0x7fd120104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:11.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.500+0000 7fd125c00640 1 --2- 172.21.15.47:0/225717815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1201038e0 0x7fd120103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:11.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.500+0000 7fd125c00640 1 -- 172.21.15.47:0/225717815 >> 172.21.15.47:0/225717815 conn(0x7fd1200fee90 msgr2=0x7fd1201012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:11.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.500+0000 7fd125c00640 1 -- 172.21.15.47:0/225717815 shutdown_connections 2024-03-21T11:20:11.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.500+0000 7fd125c00640 1 -- 172.21.15.47:0/225717815 wait complete. 2024-03-21T11:20:11.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.501+0000 7fd125c00640 1 Processor -- start 2024-03-21T11:20:11.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.502+0000 7fd125c00640 1 -- start start 2024-03-21T11:20:11.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.502+0000 7fd125c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1201038e0 0x7fd120199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:11.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.502+0000 7fd125c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120104ad0 0x7fd12019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:11.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.502+0000 7fd125c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd120105490 0x7fd1201a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:11.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.502+0000 7fd125c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12010f540 con 0x7fd120105490 2024-03-21T11:20:11.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.502+0000 7fd125c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12010f3c0 con 0x7fd120104ad0 2024-03-21T11:20:11.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.502+0000 7fd125c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12010f6c0 con 0x7fd1201038e0 2024-03-21T11:20:11.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.502+0000 7fd11e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120104ad0 0x7fd12019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:11.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.502+0000 7fd11f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd120105490 0x7fd1201a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:11.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.502+0000 7fd11e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120104ad0 0x7fd12019a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52126/0 (socket says 172.21.15.47:52126) 2024-03-21T11:20:11.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.502+0000 7fd11e000640 1 -- 172.21.15.47:0/3805729049 learned_addr learned my addr 172.21.15.47:0/3805729049 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.502+0000 7fd11ea00640 1 --2- 172.21.15.47:0/3805729049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1201038e0 0x7fd120199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.503+0000 7fd11e000640 1 -- 172.21.15.47:0/3805729049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1201038e0 msgr2=0x7fd120199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.503+0000 7fd11e000640 1 --2- 172.21.15.47:0/3805729049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1201038e0 0x7fd120199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.503+0000 7fd11e000640 1 -- 172.21.15.47:0/3805729049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd120105490 msgr2=0x7fd1201a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.503+0000 7fd11e000640 1 --2- 172.21.15.47:0/3805729049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd120105490 0x7fd1201a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.503+0000 7fd11e000640 1 -- 172.21.15.47:0/3805729049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1201a17b0 con 0x7fd120104ad0 2024-03-21T11:20:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.503+0000 7fd11e000640 1 --2- 172.21.15.47:0/3805729049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120104ad0 0x7fd12019a0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00e9f0 tx=0x7fd10c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.503+0000 7fd117400640 1 -- 172.21.15.47:0/3805729049 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10c00cd80 con 0x7fd120104ad0 2024-03-21T11:20:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.503+0000 7fd117400640 1 -- 172.21.15.47:0/3805729049 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd10c010910 con 0x7fd120104ad0 2024-03-21T11:20:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.503+0000 7fd117400640 1 -- 172.21.15.47:0/3805729049 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10c018990 con 0x7fd120104ad0 2024-03-21T11:20:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.503+0000 7fd125c00640 1 -- 172.21.15.47:0/3805729049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1201a1aa0 con 0x7fd120104ad0 2024-03-21T11:20:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.504+0000 7fd125c00640 1 -- 172.21.15.47:0/3805729049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1201a1ff0 con 0x7fd120104ad0 2024-03-21T11:20:11.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.505+0000 7fd125c00640 1 -- 172.21.15.47:0/3805729049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12010ac00 con 0x7fd120104ad0 2024-03-21T11:20:11.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.507+0000 7fd117400640 1 -- 172.21.15.47:0/3805729049 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd10c010ac0 con 0x7fd120104ad0 2024-03-21T11:20:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.508+0000 7fd117400640 1 --2- 172.21.15.47:0/3805729049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1040a2f20 0x7fd1040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.508+0000 7fd117400640 1 -- 172.21.15.47:0/3805729049 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd10c0cf260 con 0x7fd120104ad0 2024-03-21T11:20:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.509+0000 7fd11ea00640 1 --2- 172.21.15.47:0/3805729049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1040a2f20 0x7fd1040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.509+0000 7fd11ea00640 1 --2- 172.21.15.47:0/3805729049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1040a2f20 0x7fd1040a53e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd110005fd0 tx=0x7fd11000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:11.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.512+0000 7fd117400640 1 -- 172.21.15.47:0/3805729049 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd10c09be40 con 0x7fd120104ad0 2024-03-21T11:20:11.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.759+0000 7fd125c00640 1 -- 172.21.15.47:0/3805729049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd12002d070 con 0x7fd120104ad0 2024-03-21T11:20:11.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.760+0000 7fd117400640 1 -- 172.21.15.47:0/3805729049 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7fd10c09fee0 con 0x7fd120104ad0 2024-03-21T11:20:11.761 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:11.761 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T11:20:11.761 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 3 2024-03-21T11:20:11.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.763+0000 7fd125c00640 1 -- 172.21.15.47:0/3805729049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1040a2f20 msgr2=0x7fd1040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:11.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.763+0000 7fd125c00640 1 --2- 172.21.15.47:0/3805729049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1040a2f20 0x7fd1040a53e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd110005fd0 tx=0x7fd11000b410 comp rx=0 tx=0).stop 2024-03-21T11:20:11.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.763+0000 7fd125c00640 1 -- 172.21.15.47:0/3805729049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120104ad0 msgr2=0x7fd12019a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:11.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.763+0000 7fd125c00640 1 --2- 172.21.15.47:0/3805729049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120104ad0 0x7fd12019a0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00e9f0 tx=0x7fd10c00eeb0 comp rx=0 tx=0).stop 2024-03-21T11:20:11.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.763+0000 7fd125c00640 1 -- 172.21.15.47:0/3805729049 shutdown_connections 2024-03-21T11:20:11.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.763+0000 7fd125c00640 1 --2- 172.21.15.47:0/3805729049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1040a2f20 0x7fd1040a53e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:11.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.763+0000 7fd125c00640 1 --2- 172.21.15.47:0/3805729049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd120105490 0x7fd1201a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:11.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.763+0000 7fd125c00640 1 --2- 172.21.15.47:0/3805729049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120104ad0 0x7fd12019a0d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:11.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.763+0000 7fd125c00640 1 --2- 172.21.15.47:0/3805729049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1201038e0 0x7fd120199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:11.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.763+0000 7fd125c00640 1 -- 172.21.15.47:0/3805729049 >> 172.21.15.47:0/3805729049 conn(0x7fd1200fee90 msgr2=0x7fd120107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:11.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.763+0000 7fd125c00640 1 -- 172.21.15.47:0/3805729049 shutdown_connections 2024-03-21T11:20:11.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:11.763+0000 7fd125c00640 1 -- 172.21.15.47:0/3805729049 wait complete. 2024-03-21T11:20:12.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:11 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3805729049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:12.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:11 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3805729049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:12.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:11 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3805729049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:12.447 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 11:20:12 smithi047 podman[69186]: 2024-03-21 11:20:12.13539705 +0000 UTC m=+0.173694896 container create 8a6c3bc78d0f8b529308a4f2b49123bdaece972607138ff58af06c5ad572028b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-j, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, ceph=True) 2024-03-21T11:20:12.447 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 11:20:12 smithi047 podman[69186]: 2024-03-21 11:20:12.446607275 +0000 UTC m=+0.484905108 container init 8a6c3bc78d0f8b529308a4f2b49123bdaece972607138ff58af06c5ad572028b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-j, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T11:20:12.814 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 11:20:12 smithi047 podman[69186]: 2024-03-21 11:20:12.458752709 +0000 UTC m=+0.497050571 container start 8a6c3bc78d0f8b529308a4f2b49123bdaece972607138ff58af06c5ad572028b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-j, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-21T11:20:12.814 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 11:20:12 smithi047 ceph-mds[69204]: -- 172.21.15.47:0/2305515631 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564bbc2bfa40 con 0x564bbc292800 2024-03-21T11:20:12.814 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 11:20:12 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-j[69200]: starting mds.j at 2024-03-21T11:20:12.814 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 11:20:12 smithi047 bash[69186]: 8a6c3bc78d0f8b529308a4f2b49123bdaece972607138ff58af06c5ad572028b 2024-03-21T11:20:12.815 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 11:20:12 smithi047 systemd[1]: Started Ceph mds.j for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:20:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:12 smithi100 ceph-mon[36245]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:12 smithi102 ceph-mon[36748]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:12 smithi047 ceph-mon[31351]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:13.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:13.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:13.783 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.016+0000 7f6cea200640 1 -- 172.21.15.47:0/328897061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ce40756e0 msgr2=0x7f6ce4073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:14.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.016+0000 7f6cea200640 1 --2- 172.21.15.47:0/328897061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ce40756e0 0x7f6ce4073b30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009900 tx=0x7f6cd802bc40 comp rx=0 tx=0).stop 2024-03-21T11:20:14.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.016+0000 7f6cea200640 1 -- 172.21.15.47:0/328897061 shutdown_connections 2024-03-21T11:20:14.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.016+0000 7f6cea200640 1 --2- 172.21.15.47:0/328897061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ce41093a0 0x7f6ce410b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:14.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.016+0000 7f6cea200640 1 --2- 172.21.15.47:0/328897061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce4074100 0x7f6ce4074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:14.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.016+0000 7f6cea200640 1 --2- 172.21.15.47:0/328897061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ce40756e0 0x7f6ce4073b30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:14.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.016+0000 7f6cea200640 1 -- 172.21.15.47:0/328897061 >> 172.21.15.47:0/328897061 conn(0x7f6ce40fca30 msgr2=0x7f6ce40fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:14.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.016+0000 7f6cea200640 1 -- 172.21.15.47:0/328897061 shutdown_connections 2024-03-21T11:20:14.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.016+0000 7f6cea200640 1 -- 172.21.15.47:0/328897061 wait complete. 2024-03-21T11:20:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.017+0000 7f6cea200640 1 Processor -- start 2024-03-21T11:20:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.017+0000 7f6cea200640 1 -- start start 2024-03-21T11:20:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.017+0000 7f6cea200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ce4074100 0x7f6ce4199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.017+0000 7f6cea200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce40756e0 0x7f6ce4199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.017+0000 7f6cea200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ce41093a0 0x7f6ce41a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.017+0000 7f6cea200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce410f390 con 0x7f6ce40756e0 2024-03-21T11:20:14.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.017+0000 7f6cea200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce410f210 con 0x7f6ce4074100 2024-03-21T11:20:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.017+0000 7f6cea200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce410f510 con 0x7f6ce41093a0 2024-03-21T11:20:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.018+0000 7f6ce3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce40756e0 0x7f6ce4199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.018+0000 7f6ce8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ce4074100 0x7f6ce4199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.018+0000 7f6ce8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ce4074100 0x7f6ce4199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52164/0 (socket says 172.21.15.47:52164) 2024-03-21T11:20:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.018+0000 7f6ce8e00640 1 -- 172.21.15.47:0/4129686886 learned_addr learned my addr 172.21.15.47:0/4129686886 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.018+0000 7f6ce9800640 1 --2- 172.21.15.47:0/4129686886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ce41093a0 0x7f6ce41a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.018+0000 7f6ce8e00640 1 -- 172.21.15.47:0/4129686886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ce41093a0 msgr2=0x7f6ce41a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.018+0000 7f6ce8e00640 1 --2- 172.21.15.47:0/4129686886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ce41093a0 0x7f6ce41a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.018+0000 7f6ce8e00640 1 -- 172.21.15.47:0/4129686886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce40756e0 msgr2=0x7f6ce4199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.018+0000 7f6ce8e00640 1 --2- 172.21.15.47:0/4129686886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce40756e0 0x7f6ce4199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.018+0000 7f6ce8e00640 1 -- 172.21.15.47:0/4129686886 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce41a1640 con 0x7f6ce4074100 2024-03-21T11:20:14.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.018+0000 7f6ce3e00640 1 --2- 172.21.15.47:0/4129686886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce40756e0 0x7f6ce4199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:20:14.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.019+0000 7f6ce8e00640 1 --2- 172.21.15.47:0/4129686886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ce4074100 0x7f6ce4199a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6cd802c8f0 tx=0x7f6cd802c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:14.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.019+0000 7f6ce1600640 1 -- 172.21.15.47:0/4129686886 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8041070 con 0x7f6ce4074100 2024-03-21T11:20:14.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.019+0000 7f6ce1600640 1 -- 172.21.15.47:0/4129686886 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cd8004450 con 0x7f6ce4074100 2024-03-21T11:20:14.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.019+0000 7f6cea200640 1 -- 172.21.15.47:0/4129686886 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce41a18d0 con 0x7f6ce4074100 2024-03-21T11:20:14.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.019+0000 7f6ce1600640 1 -- 172.21.15.47:0/4129686886 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8034440 con 0x7f6ce4074100 2024-03-21T11:20:14.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.019+0000 7f6cea200640 1 -- 172.21.15.47:0/4129686886 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce41a1dc0 con 0x7f6ce4074100 2024-03-21T11:20:14.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.020+0000 7f6cea200640 1 -- 172.21.15.47:0/4129686886 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce41080d0 con 0x7f6ce4074100 2024-03-21T11:20:14.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.022+0000 7f6ce1600640 1 -- 172.21.15.47:0/4129686886 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6cd8044070 con 0x7f6ce4074100 2024-03-21T11:20:14.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.023+0000 7f6ce1600640 1 --2- 172.21.15.47:0/4129686886 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ccc0a2f20 0x7f6ccc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:14.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.024+0000 7f6ce1600640 1 -- 172.21.15.47:0/4129686886 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6cd80f0b10 con 0x7f6ce4074100 2024-03-21T11:20:14.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.024+0000 7f6ce3e00640 1 --2- 172.21.15.47:0/4129686886 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ccc0a2f20 0x7f6ccc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:14.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.024+0000 7f6ce3e00640 1 --2- 172.21.15.47:0/4129686886 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ccc0a2f20 0x7f6ccc0a53e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6ce419af30 tx=0x7f6cd0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:14.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.027+0000 7f6ce1600640 1 -- 172.21.15.47:0/4129686886 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6cd80bd6f0 con 0x7f6ce4074100 2024-03-21T11:20:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:13 smithi047 ceph-mon[31351]: Deploying daemon mds.d on smithi047 2024-03-21T11:20:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:13 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.47:6834/4028210726,v1:172.21.15.47:6835/4028210726] up:boot 2024-03-21T11:20:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:13 smithi047 ceph-mon[31351]: fsmap 3 up:standby 2024-03-21T11:20:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T11:20:14.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:13 smithi100 ceph-mon[36245]: Deploying daemon mds.d on smithi047 2024-03-21T11:20:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:13 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.47:6834/4028210726,v1:172.21.15.47:6835/4028210726] up:boot 2024-03-21T11:20:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:13 smithi100 ceph-mon[36245]: fsmap 3 up:standby 2024-03-21T11:20:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T11:20:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:13 smithi102 ceph-mon[36748]: Deploying daemon mds.d on smithi047 2024-03-21T11:20:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:13 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.47:6834/4028210726,v1:172.21.15.47:6835/4028210726] up:boot 2024-03-21T11:20:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:13 smithi102 ceph-mon[36748]: fsmap 3 up:standby 2024-03-21T11:20:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T11:20:14.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.279+0000 7f6cea200640 1 -- 172.21.15.47:0/4129686886 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ce40622a0 con 0x7f6ce4074100 2024-03-21T11:20:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.280+0000 7f6ce1600640 1 -- 172.21.15.47:0/4129686886 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f6cd80c1790 con 0x7f6ce4074100 2024-03-21T11:20:14.281 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:14.281 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T11:20:14.282 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 4 2024-03-21T11:20:14.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.283+0000 7f6cea200640 1 -- 172.21.15.47:0/4129686886 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ccc0a2f20 msgr2=0x7f6ccc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:14.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.283+0000 7f6cea200640 1 --2- 172.21.15.47:0/4129686886 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ccc0a2f20 0x7f6ccc0a53e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6ce419af30 tx=0x7f6cd0005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:14.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.283+0000 7f6cea200640 1 -- 172.21.15.47:0/4129686886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ce4074100 msgr2=0x7f6ce4199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:14.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.283+0000 7f6cea200640 1 --2- 172.21.15.47:0/4129686886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ce4074100 0x7f6ce4199a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6cd802c8f0 tx=0x7f6cd802c660 comp rx=0 tx=0).stop 2024-03-21T11:20:14.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.283+0000 7f6cea200640 1 -- 172.21.15.47:0/4129686886 shutdown_connections 2024-03-21T11:20:14.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.283+0000 7f6cea200640 1 --2- 172.21.15.47:0/4129686886 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ccc0a2f20 0x7f6ccc0a53e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:14.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.283+0000 7f6cea200640 1 --2- 172.21.15.47:0/4129686886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ce41093a0 0x7f6ce41a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:14.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.283+0000 7f6cea200640 1 --2- 172.21.15.47:0/4129686886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce40756e0 0x7f6ce4199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:14.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.283+0000 7f6cea200640 1 --2- 172.21.15.47:0/4129686886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ce4074100 0x7f6ce4199a20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:14.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.283+0000 7f6cea200640 1 -- 172.21.15.47:0/4129686886 >> 172.21.15.47:0/4129686886 conn(0x7f6ce40fca30 msgr2=0x7f6ce41052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:14.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.284+0000 7f6cea200640 1 -- 172.21.15.47:0/4129686886 shutdown_connections 2024-03-21T11:20:14.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:14.284+0000 7f6cea200640 1 -- 172.21.15.47:0/4129686886 wait complete. 2024-03-21T11:20:15.127 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:14 smithi047 ceph-mon[31351]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:15.127 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:14 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/4129686886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:14 smithi100 ceph-mon[36245]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:14 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/4129686886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:15.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:14 smithi102 ceph-mon[36748]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:15.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:14 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/4129686886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:16.211 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:20:15 smithi047 systemd[1]: Starting Ceph mds.d for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:20:16.303 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:16.462 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:20:16 smithi047 podman[69535]: 2024-03-21 11:20:16.11738371 +0000 UTC m=+0.039491365 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:20:16.462 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:20:16 smithi047 podman[69535]: 2024-03-21 11:20:16.232073118 +0000 UTC m=+0.154180805 container create f6f4b37d470db8f093be83e64edacdd1fdbc502f0101310c55a25ce43850511d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-d, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-21T11:20:16.462 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:20:16 smithi047 podman[69535]: 2024-03-21 11:20:16.461626672 +0000 UTC m=+0.383734412 container init f6f4b37d470db8f093be83e64edacdd1fdbc502f0101310c55a25ce43850511d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-d, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-21T11:20:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.527+0000 7f7ceee00640 1 -- 172.21.15.47:0/3245222177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce8104ad0 msgr2=0x7f7ce8104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.527+0000 7f7ceee00640 1 --2- 172.21.15.47:0/3245222177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce8104ad0 0x7f7ce8104f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009900 tx=0x7f7cd402bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.527+0000 7f7ceee00640 1 -- 172.21.15.47:0/3245222177 shutdown_connections 2024-03-21T11:20:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.527+0000 7f7ceee00640 1 --2- 172.21.15.47:0/3245222177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ce8105490 0x7f7ce810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.527+0000 7f7ceee00640 1 --2- 172.21.15.47:0/3245222177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce8104ad0 0x7f7ce8104f50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.527+0000 7f7ceee00640 1 --2- 172.21.15.47:0/3245222177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce81038e0 0x7f7ce8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.527+0000 7f7ceee00640 1 -- 172.21.15.47:0/3245222177 >> 172.21.15.47:0/3245222177 conn(0x7f7ce80fee90 msgr2=0x7f7ce81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.527+0000 7f7ceee00640 1 -- 172.21.15.47:0/3245222177 shutdown_connections 2024-03-21T11:20:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.527+0000 7f7ceee00640 1 -- 172.21.15.47:0/3245222177 wait complete. 2024-03-21T11:20:16.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.528+0000 7f7ceee00640 1 Processor -- start 2024-03-21T11:20:16.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.528+0000 7f7ceee00640 1 -- start start 2024-03-21T11:20:16.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.528+0000 7f7ceee00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce81038e0 0x7f7ce8199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:16.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.528+0000 7f7ceee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ce8104ad0 0x7f7ce819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:16.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.528+0000 7f7ceee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce8105490 0x7f7ce81a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:16.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.528+0000 7f7ceee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce810f540 con 0x7f7ce8105490 2024-03-21T11:20:16.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.528+0000 7f7ceee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce810f3c0 con 0x7f7ce8104ad0 2024-03-21T11:20:16.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.528+0000 7f7ceee00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce810f6c0 con 0x7f7ce81038e0 2024-03-21T11:20:16.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.529+0000 7f7ce7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce81038e0 0x7f7ce8199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:16.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.529+0000 7f7ce7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce81038e0 0x7f7ce8199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36828/0 (socket says 172.21.15.47:36828) 2024-03-21T11:20:16.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.529+0000 7f7ce7e00640 1 -- 172.21.15.47:0/719178544 learned_addr learned my addr 172.21.15.47:0/719178544 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:16.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.529+0000 7f7ce7400640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ce8104ad0 0x7f7ce819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:16.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.529+0000 7f7cece00640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce8105490 0x7f7ce81a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:16.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.529+0000 7f7ce7400640 1 -- 172.21.15.47:0/719178544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce81038e0 msgr2=0x7f7ce8199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:16.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.529+0000 7f7ce7400640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce81038e0 0x7f7ce8199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:16.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.529+0000 7f7ce7400640 1 -- 172.21.15.47:0/719178544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce8105490 msgr2=0x7f7ce81a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:16.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.529+0000 7f7ce7400640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce8105490 0x7f7ce81a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:16.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.529+0000 7f7ce7400640 1 -- 172.21.15.47:0/719178544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce81a1810 con 0x7f7ce8104ad0 2024-03-21T11:20:16.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.529+0000 7f7cece00640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce8105490 0x7f7ce81a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:20:16.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.529+0000 7f7ce7400640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ce8104ad0 0x7f7ce819a130 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009230 tx=0x7f7cd402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:16.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.530+0000 7f7ce4c00640 1 -- 172.21.15.47:0/719178544 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4041070 con 0x7f7ce8104ad0 2024-03-21T11:20:16.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.530+0000 7f7ce4c00640 1 -- 172.21.15.47:0/719178544 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cd4004450 con 0x7f7ce8104ad0 2024-03-21T11:20:16.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.530+0000 7f7ce4c00640 1 -- 172.21.15.47:0/719178544 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4003d80 con 0x7f7ce8104ad0 2024-03-21T11:20:16.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.530+0000 7f7ceee00640 1 -- 172.21.15.47:0/719178544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce81a1aa0 con 0x7f7ce8104ad0 2024-03-21T11:20:16.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.530+0000 7f7ceee00640 1 -- 172.21.15.47:0/719178544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce81a1f40 con 0x7f7ce8104ad0 2024-03-21T11:20:16.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.531+0000 7f7ceee00640 1 -- 172.21.15.47:0/719178544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce810ac70 con 0x7f7ce8104ad0 2024-03-21T11:20:16.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.533+0000 7f7ce4c00640 1 -- 172.21.15.47:0/719178544 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7cd4044070 con 0x7f7ce8104ad0 2024-03-21T11:20:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.534+0000 7f7ce4c00640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ccc0a2f20 0x7f7ccc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.534+0000 7f7ce4c00640 1 -- 172.21.15.47:0/719178544 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7cd40f09a0 con 0x7f7ce8104ad0 2024-03-21T11:20:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.535+0000 7f7ce7e00640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ccc0a2f20 0x7f7ccc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.535+0000 7f7ce7e00640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ccc0a2f20 0x7f7ccc0a53e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7cd80058c0 tx=0x7f7cd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:16.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.539+0000 7f7ce4c00640 1 -- 172.21.15.47:0/719178544 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7cd40bdbd0 con 0x7f7ce8104ad0 2024-03-21T11:20:16.814 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:20:16 smithi047 podman[69535]: 2024-03-21 11:20:16.470627934 +0000 UTC m=+0.392735584 container start f6f4b37d470db8f093be83e64edacdd1fdbc502f0101310c55a25ce43850511d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-d, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux ) 2024-03-21T11:20:16.814 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:20:16 smithi047 ceph-mds[69574]: -- 172.21.15.47:0/2888005018 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5570352e1a40 con 0x5570352b4800 2024-03-21T11:20:16.815 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:20:16 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-d[69563]: starting mds.d at 2024-03-21T11:20:16.815 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:20:16 smithi047 bash[69535]: f6f4b37d470db8f093be83e64edacdd1fdbc502f0101310c55a25ce43850511d 2024-03-21T11:20:16.815 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:20:16 smithi047 systemd[1]: Started Ceph mds.d for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:20:16.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.818+0000 7f7ceee00640 1 -- 172.21.15.47:0/719178544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ce802d070 con 0x7f7ce8104ad0 2024-03-21T11:20:16.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.819+0000 7f7ce4c00640 1 -- 172.21.15.47:0/719178544 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f7cd40bd4d0 con 0x7f7ce8104ad0 2024-03-21T11:20:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:16.820 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T11:20:16.820 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 4 2024-03-21T11:20:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.822+0000 7f7ceee00640 1 -- 172.21.15.47:0/719178544 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ccc0a2f20 msgr2=0x7f7ccc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.822+0000 7f7ceee00640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ccc0a2f20 0x7f7ccc0a53e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7cd80058c0 tx=0x7f7cd8005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.822+0000 7f7ceee00640 1 -- 172.21.15.47:0/719178544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ce8104ad0 msgr2=0x7f7ce819a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.822+0000 7f7ceee00640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ce8104ad0 0x7f7ce819a130 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009230 tx=0x7f7cd402c650 comp rx=0 tx=0).stop 2024-03-21T11:20:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.822+0000 7f7ceee00640 1 -- 172.21.15.47:0/719178544 shutdown_connections 2024-03-21T11:20:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.822+0000 7f7ceee00640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ccc0a2f20 0x7f7ccc0a53e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.822+0000 7f7ceee00640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ce8105490 0x7f7ce81a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.822+0000 7f7ceee00640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ce8104ad0 0x7f7ce819a130 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.822+0000 7f7ceee00640 1 --2- 172.21.15.47:0/719178544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce81038e0 0x7f7ce8199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.822+0000 7f7ceee00640 1 -- 172.21.15.47:0/719178544 >> 172.21.15.47:0/719178544 conn(0x7f7ce80fee90 msgr2=0x7f7ce8107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.823+0000 7f7ceee00640 1 -- 172.21.15.47:0/719178544 shutdown_connections 2024-03-21T11:20:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:16.823+0000 7f7ceee00640 1 -- 172.21.15.47:0/719178544 wait complete. 2024-03-21T11:20:17.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:16 smithi100 ceph-mon[36245]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:17.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:16 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/719178544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:16 smithi102 ceph-mon[36748]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:16 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/719178544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:16 smithi047 ceph-mon[31351]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:16 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/719178544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:18.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:17 smithi100 ceph-mon[36245]: Deploying daemon mds.e on smithi100 2024-03-21T11:20:18.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:17 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.47:6836/1846721633,v1:172.21.15.47:6837/1846721633] up:boot 2024-03-21T11:20:18.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:17 smithi100 ceph-mon[36245]: fsmap 4 up:standby 2024-03-21T11:20:18.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:17 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T11:20:18.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:17 smithi102 ceph-mon[36748]: Deploying daemon mds.e on smithi100 2024-03-21T11:20:18.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:17 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.47:6836/1846721633,v1:172.21.15.47:6837/1846721633] up:boot 2024-03-21T11:20:18.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:17 smithi102 ceph-mon[36748]: fsmap 4 up:standby 2024-03-21T11:20:18.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:17 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T11:20:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:17 smithi047 ceph-mon[31351]: Deploying daemon mds.e on smithi100 2024-03-21T11:20:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:17 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.47:6836/1846721633,v1:172.21.15.47:6837/1846721633] up:boot 2024-03-21T11:20:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:17 smithi047 ceph-mon[31351]: fsmap 4 up:standby 2024-03-21T11:20:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:17 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T11:20:18.846 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.080+0000 7f121f000640 1 -- 172.21.15.47:0/2481662215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1218108cf0 msgr2=0x7f1218109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.080+0000 7f121f000640 1 --2- 172.21.15.47:0/2481662215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1218108cf0 0x7f1218109170 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1204009900 tx=0x7f120402bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.080+0000 7f121f000640 1 -- 172.21.15.47:0/2481662215 shutdown_connections 2024-03-21T11:20:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.080+0000 7f121f000640 1 --2- 172.21.15.47:0/2481662215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12181096b0 0x7f121810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.080+0000 7f121f000640 1 --2- 172.21.15.47:0/2481662215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1218108cf0 0x7f1218109170 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.080+0000 7f121f000640 1 --2- 172.21.15.47:0/2481662215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1218107b00 0x7f1218107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.080+0000 7f121f000640 1 -- 172.21.15.47:0/2481662215 >> 172.21.15.47:0/2481662215 conn(0x7f1218076ab0 msgr2=0x7f1218078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.080+0000 7f121f000640 1 -- 172.21.15.47:0/2481662215 shutdown_connections 2024-03-21T11:20:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.080+0000 7f121f000640 1 -- 172.21.15.47:0/2481662215 wait complete. 2024-03-21T11:20:19.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.081+0000 7f121f000640 1 Processor -- start 2024-03-21T11:20:19.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.081+0000 7f121f000640 1 -- start start 2024-03-21T11:20:19.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.081+0000 7f121f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1218107b00 0x7f121819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:19.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.081+0000 7f121f000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1218108cf0 0x7f121819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:19.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121f000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12181096b0 0x7f12181a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:19.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121f000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1218113780 con 0x7f12181096b0 2024-03-21T11:20:19.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1218113600 con 0x7f1218107b00 2024-03-21T11:20:19.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121f000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1218113900 con 0x7f1218108cf0 2024-03-21T11:20:19.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1218107b00 0x7f121819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:19.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1218107b00 0x7f121819ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40248/0 (socket says 172.21.15.47:40248) 2024-03-21T11:20:19.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121dc00640 1 -- 172.21.15.47:0/2695258553 learned_addr learned my addr 172.21.15.47:0/2695258553 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121d200640 1 --2- 172.21.15.47:0/2695258553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1218108cf0 0x7f121819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121e600640 1 --2- 172.21.15.47:0/2695258553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12181096b0 0x7f12181a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121dc00640 1 -- 172.21.15.47:0/2695258553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1218108cf0 msgr2=0x7f121819e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121dc00640 1 --2- 172.21.15.47:0/2695258553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1218108cf0 0x7f121819e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121dc00640 1 -- 172.21.15.47:0/2695258553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12181096b0 msgr2=0x7f12181a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121dc00640 1 --2- 172.21.15.47:0/2695258553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12181096b0 0x7f12181a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121dc00640 1 -- 172.21.15.47:0/2695258553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12181a59f0 con 0x7f1218107b00 2024-03-21T11:20:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.082+0000 7f121dc00640 1 --2- 172.21.15.47:0/2695258553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1218107b00 0x7f121819ddd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f120c00e9c0 tx=0x7f120c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.083+0000 7f1216000640 1 -- 172.21.15.47:0/2695258553 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c00cde0 con 0x7f1218107b00 2024-03-21T11:20:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.083+0000 7f1216000640 1 -- 172.21.15.47:0/2695258553 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f120c010910 con 0x7f1218107b00 2024-03-21T11:20:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.083+0000 7f1216000640 1 -- 172.21.15.47:0/2695258553 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c018a00 con 0x7f1218107b00 2024-03-21T11:20:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.083+0000 7f121f000640 1 -- 172.21.15.47:0/2695258553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12181a5ce0 con 0x7f1218107b00 2024-03-21T11:20:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.083+0000 7f121f000640 1 -- 172.21.15.47:0/2695258553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12181a61b0 con 0x7f1218107b00 2024-03-21T11:20:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.084+0000 7f121f000640 1 -- 172.21.15.47:0/2695258553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121810ee20 con 0x7f1218107b00 2024-03-21T11:20:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.086+0000 7f1216000640 1 -- 172.21.15.47:0/2695258553 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f120c01f020 con 0x7f1218107b00 2024-03-21T11:20:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.087+0000 7f1216000640 1 --2- 172.21.15.47:0/2695258553 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12000a2f20 0x7f12000a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.087+0000 7f1216000640 1 -- 172.21.15.47:0/2695258553 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f120c0cf1c0 con 0x7f1218107b00 2024-03-21T11:20:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.088+0000 7f121d200640 1 --2- 172.21.15.47:0/2695258553 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12000a2f20 0x7f12000a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.088+0000 7f121d200640 1 --2- 172.21.15.47:0/2695258553 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12000a2f20 0x7f12000a53e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f120402c7b0 tx=0x7f1204005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:19.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.091+0000 7f1216000640 1 -- 172.21.15.47:0/2695258553 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f120c09c3f0 con 0x7f1218107b00 2024-03-21T11:20:19.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:18 smithi100 ceph-mon[36245]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:19.143 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:20:18 smithi100 systemd[1]: Starting Ceph mds.e for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:20:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:18 smithi102 ceph-mon[36748]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:18 smithi047 ceph-mon[31351]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:19.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.337+0000 7f121f000640 1 -- 172.21.15.47:0/2695258553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f121802d070 con 0x7f1218107b00 2024-03-21T11:20:19.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.338+0000 7f1216000640 1 -- 172.21.15.47:0/2695258553 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f120c09bcf0 con 0x7f1218107b00 2024-03-21T11:20:19.339 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:19.339 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[]} 2024-03-21T11:20:19.339 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 5 2024-03-21T11:20:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.341+0000 7f121f000640 1 -- 172.21.15.47:0/2695258553 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12000a2f20 msgr2=0x7f12000a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.341+0000 7f121f000640 1 --2- 172.21.15.47:0/2695258553 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12000a2f20 0x7f12000a53e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f120402c7b0 tx=0x7f1204005fd0 comp rx=0 tx=0).stop 2024-03-21T11:20:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.341+0000 7f121f000640 1 -- 172.21.15.47:0/2695258553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1218107b00 msgr2=0x7f121819ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.341+0000 7f121f000640 1 --2- 172.21.15.47:0/2695258553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1218107b00 0x7f121819ddd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f120c00e9c0 tx=0x7f120c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:20:19.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.341+0000 7f121f000640 1 -- 172.21.15.47:0/2695258553 shutdown_connections 2024-03-21T11:20:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.341+0000 7f121f000640 1 --2- 172.21.15.47:0/2695258553 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12000a2f20 0x7f12000a53e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.342+0000 7f121f000640 1 --2- 172.21.15.47:0/2695258553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12181096b0 0x7f12181a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.342+0000 7f121f000640 1 --2- 172.21.15.47:0/2695258553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1218108cf0 0x7f121819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.342+0000 7f121f000640 1 --2- 172.21.15.47:0/2695258553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1218107b00 0x7f121819ddd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.342+0000 7f121f000640 1 -- 172.21.15.47:0/2695258553 >> 172.21.15.47:0/2695258553 conn(0x7f1218076ab0 msgr2=0x7f121810bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.342+0000 7f121f000640 1 -- 172.21.15.47:0/2695258553 shutdown_connections 2024-03-21T11:20:19.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:19.342+0000 7f121f000640 1 -- 172.21.15.47:0/2695258553 wait complete. 2024-03-21T11:20:19.434 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:20:19 smithi100 podman[65678]: 2024-03-21 11:20:19.076858605 +0000 UTC m=+0.018108341 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:20:19.435 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:20:19 smithi100 podman[65678]: 2024-03-21 11:20:19.181333779 +0000 UTC m=+0.122583502 container create d771cc7b5d6f1300df4a23364332a8b84cd50f211714daa2fb267c0ec8317e0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-03-21T11:20:19.435 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:20:19 smithi100 podman[65678]: 2024-03-21 11:20:19.40941247 +0000 UTC m=+0.350662201 container init d771cc7b5d6f1300df4a23364332a8b84cd50f211714daa2fb267c0ec8317e0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:20:19.435 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:20:19 smithi100 podman[65678]: 2024-03-21 11:20:19.41195721 +0000 UTC m=+0.353206933 container start d771cc7b5d6f1300df4a23364332a8b84cd50f211714daa2fb267c0ec8317e0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-21T11:20:19.879 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:20:19 smithi100 ceph-mds[65697]: -- 172.21.15.100:0/624288972 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563929a17a40 con 0x5639299eac00 2024-03-21T11:20:19.879 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:20:19 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e[65693]: starting mds.e at 2024-03-21T11:20:19.879 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:20:19 smithi100 bash[65678]: d771cc7b5d6f1300df4a23364332a8b84cd50f211714daa2fb267c0ec8317e0e 2024-03-21T11:20:19.879 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:20:19 smithi100 systemd[1]: Started Ceph mds.e for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:20:20.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:19 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2695258553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:20.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:19 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:20.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:19 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:20.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:19 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:20.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:19 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:20.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:19 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:20.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:19 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:20.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:19 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2695258553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:20.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:19 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:20.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:19 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:20.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:19 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:20.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:19 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:20.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:19 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:20.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:19 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:20.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:19 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2695258553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:20.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:19 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:20.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:19 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:20.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:19 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:20.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:19 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:20.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:19 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:20.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:19 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:21.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:20 smithi100 ceph-mon[36245]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:20 smithi100 ceph-mon[36245]: Deploying daemon mds.f on smithi102 2024-03-21T11:20:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:20 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.100:6834/2785789126,v1:172.21.15.100:6835/2785789126] up:boot 2024-03-21T11:20:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:20 smithi100 ceph-mon[36245]: fsmap 5 up:standby 2024-03-21T11:20:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:20 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T11:20:21.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:20 smithi047 ceph-mon[31351]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:21.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:20 smithi047 ceph-mon[31351]: Deploying daemon mds.f on smithi102 2024-03-21T11:20:21.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:20 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.100:6834/2785789126,v1:172.21.15.100:6835/2785789126] up:boot 2024-03-21T11:20:21.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:20 smithi047 ceph-mon[31351]: fsmap 5 up:standby 2024-03-21T11:20:21.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:20 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T11:20:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:20 smithi102 ceph-mon[36748]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:20 smithi102 ceph-mon[36748]: Deploying daemon mds.f on smithi102 2024-03-21T11:20:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:20 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.100:6834/2785789126,v1:172.21.15.100:6835/2785789126] up:boot 2024-03-21T11:20:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:20 smithi102 ceph-mon[36748]: fsmap 5 up:standby 2024-03-21T11:20:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:20 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T11:20:21.364 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.602+0000 7f9791000640 1 -- 172.21.15.47:0/3028418343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c1015e0 msgr2=0x7f978c101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.602+0000 7f9791000640 1 --2- 172.21.15.47:0/3028418343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c1015e0 0x7f978c101a60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9778009900 tx=0x7f977802bc40 comp rx=0 tx=0).stop 2024-03-21T11:20:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.602+0000 7f9791000640 1 -- 172.21.15.47:0/3028418343 shutdown_connections 2024-03-21T11:20:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.602+0000 7f9791000640 1 --2- 172.21.15.47:0/3028418343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f978c101fa0 0x7f978c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.602+0000 7f9791000640 1 --2- 172.21.15.47:0/3028418343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c1015e0 0x7f978c101a60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.602+0000 7f9791000640 1 --2- 172.21.15.47:0/3028418343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f978c1003f0 0x7f978c100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.602+0000 7f9791000640 1 -- 172.21.15.47:0/3028418343 >> 172.21.15.47:0/3028418343 conn(0x7f978c0fb980 msgr2=0x7f978c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.602+0000 7f9791000640 1 -- 172.21.15.47:0/3028418343 shutdown_connections 2024-03-21T11:20:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.602+0000 7f9791000640 1 -- 172.21.15.47:0/3028418343 wait complete. 2024-03-21T11:20:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.603+0000 7f9791000640 1 Processor -- start 2024-03-21T11:20:21.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.604+0000 7f9791000640 1 -- start start 2024-03-21T11:20:21.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.604+0000 7f9791000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f978c1003f0 0x7f978c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:21.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.604+0000 7f9791000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f978c1015e0 0x7f978c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:21.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.604+0000 7f9791000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101fa0 0x7f978c1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:21.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.604+0000 7f9791000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c10c0d0 con 0x7f978c101fa0 2024-03-21T11:20:21.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.604+0000 7f9791000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c10bf50 con 0x7f978c1003f0 2024-03-21T11:20:21.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.604+0000 7f9791000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c10c250 con 0x7f978c1015e0 2024-03-21T11:20:21.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.604+0000 7f978aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f978c1015e0 0x7f978c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:21.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.604+0000 7f978aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f978c1015e0 0x7f978c199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45142/0 (socket says 172.21.15.47:45142) 2024-03-21T11:20:21.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.604+0000 7f978aa00640 1 -- 172.21.15.47:0/1057002427 learned_addr learned my addr 172.21.15.47:0/1057002427 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:21.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.604+0000 7f978b400640 1 --2- 172.21.15.47:0/1057002427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f978c1003f0 0x7f978c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:21.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.604+0000 7f978be00640 1 --2- 172.21.15.47:0/1057002427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101fa0 0x7f978c1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:21.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.605+0000 7f978aa00640 1 -- 172.21.15.47:0/1057002427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f978c1003f0 msgr2=0x7f978c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:21.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.605+0000 7f978aa00640 1 --2- 172.21.15.47:0/1057002427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f978c1003f0 0x7f978c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:21.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.605+0000 7f978aa00640 1 -- 172.21.15.47:0/1057002427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101fa0 msgr2=0x7f978c1a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:21.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.605+0000 7f978aa00640 1 --2- 172.21.15.47:0/1057002427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101fa0 0x7f978c1a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:21.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.605+0000 7f978aa00640 1 -- 172.21.15.47:0/1057002427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c1a1640 con 0x7f978c1015e0 2024-03-21T11:20:21.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.605+0000 7f978aa00640 1 --2- 172.21.15.47:0/1057002427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f978c1015e0 0x7f978c199f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f977802c8f0 tx=0x7f977802c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:21.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.605+0000 7f9783e00640 1 -- 172.21.15.47:0/1057002427 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9778041070 con 0x7f978c1015e0 2024-03-21T11:20:21.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.605+0000 7f9783e00640 1 -- 172.21.15.47:0/1057002427 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9778004450 con 0x7f978c1015e0 2024-03-21T11:20:21.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.605+0000 7f9783e00640 1 -- 172.21.15.47:0/1057002427 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9778034440 con 0x7f978c1015e0 2024-03-21T11:20:21.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.605+0000 7f9791000640 1 -- 172.21.15.47:0/1057002427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f978c1a18d0 con 0x7f978c1015e0 2024-03-21T11:20:21.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.606+0000 7f9791000640 1 -- 172.21.15.47:0/1057002427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978c1a1d70 con 0x7f978c1015e0 2024-03-21T11:20:21.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.607+0000 7f9791000640 1 -- 172.21.15.47:0/1057002427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978c1a1fd0 con 0x7f978c1015e0 2024-03-21T11:20:21.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.611+0000 7f9783e00640 1 -- 172.21.15.47:0/1057002427 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9778044070 con 0x7f978c1015e0 2024-03-21T11:20:21.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.613+0000 7f9783e00640 1 --2- 172.21.15.47:0/1057002427 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97700a2fa0 0x7f97700a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:21.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.613+0000 7f9783e00640 1 -- 172.21.15.47:0/1057002427 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f97780f0bc0 con 0x7f978c1015e0 2024-03-21T11:20:21.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.613+0000 7f978b400640 1 --2- 172.21.15.47:0/1057002427 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97700a2fa0 0x7f97700a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:21.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.613+0000 7f9783e00640 1 -- 172.21.15.47:0/1057002427 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f97780bd7a0 con 0x7f978c1015e0 2024-03-21T11:20:21.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.614+0000 7f978b400640 1 --2- 172.21.15.47:0/1057002427 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97700a2fa0 0x7f97700a5460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f977c0095f0 tx=0x7f977c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:21.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.860+0000 7f9791000640 1 -- 172.21.15.47:0/1057002427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f978c1a21a0 con 0x7f978c1015e0 2024-03-21T11:20:21.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.861+0000 7f9783e00640 1 -- 172.21.15.47:0/1057002427 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4495 (secure 0 0 0) 0x7f97780c1840 con 0x7f978c1015e0 2024-03-21T11:20:21.862 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:21.862 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-21T11:20:21.863 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2024-03-21T11:20:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.864+0000 7f9791000640 1 -- 172.21.15.47:0/1057002427 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97700a2fa0 msgr2=0x7f97700a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.864+0000 7f9791000640 1 --2- 172.21.15.47:0/1057002427 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97700a2fa0 0x7f97700a5460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f977c0095f0 tx=0x7f977c005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.864+0000 7f9791000640 1 -- 172.21.15.47:0/1057002427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f978c1015e0 msgr2=0x7f978c199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.865+0000 7f9791000640 1 --2- 172.21.15.47:0/1057002427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f978c1015e0 0x7f978c199f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f977802c8f0 tx=0x7f977802c660 comp rx=0 tx=0).stop 2024-03-21T11:20:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.865+0000 7f9791000640 1 -- 172.21.15.47:0/1057002427 shutdown_connections 2024-03-21T11:20:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.865+0000 7f9791000640 1 --2- 172.21.15.47:0/1057002427 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97700a2fa0 0x7f97700a5460 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.865+0000 7f9791000640 1 --2- 172.21.15.47:0/1057002427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101fa0 0x7f978c1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.865+0000 7f9791000640 1 --2- 172.21.15.47:0/1057002427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f978c1015e0 0x7f978c199f60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.865+0000 7f9791000640 1 --2- 172.21.15.47:0/1057002427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f978c1003f0 0x7f978c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.865+0000 7f9791000640 1 -- 172.21.15.47:0/1057002427 >> 172.21.15.47:0/1057002427 conn(0x7f978c0fb980 msgr2=0x7f978c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.865+0000 7f9791000640 1 -- 172.21.15.47:0/1057002427 shutdown_connections 2024-03-21T11:20:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:21.865+0000 7f9791000640 1 -- 172.21.15.47:0/1057002427 wait complete. 2024-03-21T11:20:21.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:21 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1057002427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:22.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:21 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1057002427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:22.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:21 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1057002427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:22.320 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:20:22 smithi102 podman[67355]: 2024-03-21 11:20:22.317661129 +0000 UTC m=+0.363720813 container init f8203788be54de48b755f29a98e32430bc5f8bee388b8169d16133ce6729409c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-03-21T11:20:22.320 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:20:22 smithi102 podman[67355]: 2024-03-21 11:20:22.320712382 +0000 UTC m=+0.366772067 container start f8203788be54de48b755f29a98e32430bc5f8bee388b8169d16133ce6729409c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-f, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-21T11:20:22.741 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:20:22 smithi102 ceph-mds[67375]: -- 172.21.15.102:0/696490125 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558e12e85a40 con 0x558e12e59000 2024-03-21T11:20:22.741 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:20:22 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-f[67371]: starting mds.f at 2024-03-21T11:20:22.741 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:20:22 smithi102 bash[67355]: f8203788be54de48b755f29a98e32430bc5f8bee388b8169d16133ce6729409c 2024-03-21T11:20:22.741 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:20:22 smithi102 systemd[1]: Started Ceph mds.f for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:20:23.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:22 smithi100 ceph-mon[36245]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:23.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:22 smithi102 ceph-mon[36748]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:23.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:23.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:23.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:23.177 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:23.177 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:23.177 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:22 smithi047 ceph-mon[31351]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:23.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:23.884 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:24.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.118+0000 7f6a29400640 1 -- 172.21.15.47:0/1051335272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a24107b00 msgr2=0x7f6a24107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:24.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.118+0000 7f6a29400640 1 --2- 172.21.15.47:0/1051335272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a24107b00 0x7f6a24107f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009900 tx=0x7f6a1402bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:24.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.118+0000 7f6a29400640 1 -- 172.21.15.47:0/1051335272 shutdown_connections 2024-03-21T11:20:24.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.118+0000 7f6a29400640 1 --2- 172.21.15.47:0/1051335272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a241096b0 0x7f6a2410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:24.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.118+0000 7f6a29400640 1 --2- 172.21.15.47:0/1051335272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a24108cf0 0x7f6a24109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:24.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.118+0000 7f6a29400640 1 --2- 172.21.15.47:0/1051335272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a24107b00 0x7f6a24107f20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:24.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.118+0000 7f6a29400640 1 -- 172.21.15.47:0/1051335272 >> 172.21.15.47:0/1051335272 conn(0x7f6a24076ab0 msgr2=0x7f6a24078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:24.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.118+0000 7f6a29400640 1 -- 172.21.15.47:0/1051335272 shutdown_connections 2024-03-21T11:20:24.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.118+0000 7f6a29400640 1 -- 172.21.15.47:0/1051335272 wait complete. 2024-03-21T11:20:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.119+0000 7f6a29400640 1 Processor -- start 2024-03-21T11:20:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.119+0000 7f6a29400640 1 -- start start 2024-03-21T11:20:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.119+0000 7f6a29400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a24107b00 0x7f6a2419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.119+0000 7f6a29400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a24108cf0 0x7f6a2419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.119+0000 7f6a29400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a241096b0 0x7f6a241a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.119+0000 7f6a29400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241137c0 con 0x7f6a24107b00 2024-03-21T11:20:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.119+0000 7f6a29400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24113640 con 0x7f6a24108cf0 2024-03-21T11:20:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.119+0000 7f6a29400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24113940 con 0x7f6a241096b0 2024-03-21T11:20:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.120+0000 7f6a23e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a24107b00 0x7f6a2419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.120+0000 7f6a1b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a24108cf0 0x7f6a2419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.120+0000 7f6a23e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a24107b00 0x7f6a2419de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57232/0 (socket says 172.21.15.47:57232) 2024-03-21T11:20:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.120+0000 7f6a23e00640 1 -- 172.21.15.47:0/2435493408 learned_addr learned my addr 172.21.15.47:0/2435493408 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.120+0000 7f6a28a00640 1 --2- 172.21.15.47:0/2435493408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a241096b0 0x7f6a241a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.120+0000 7f6a1b400640 1 -- 172.21.15.47:0/2435493408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a241096b0 msgr2=0x7f6a241a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.120+0000 7f6a1b400640 1 --2- 172.21.15.47:0/2435493408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a241096b0 0x7f6a241a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.120+0000 7f6a1b400640 1 -- 172.21.15.47:0/2435493408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a24107b00 msgr2=0x7f6a2419de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.120+0000 7f6a1b400640 1 --2- 172.21.15.47:0/2435493408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a24107b00 0x7f6a2419de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.120+0000 7f6a1b400640 1 -- 172.21.15.47:0/2435493408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a241a5a80 con 0x7f6a24108cf0 2024-03-21T11:20:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.120+0000 7f6a1b400640 1 --2- 172.21.15.47:0/2435493408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a24108cf0 0x7f6a2419e3a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00e9f0 tx=0x7f6a0c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.121+0000 7f6a21600640 1 -- 172.21.15.47:0/2435493408 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00ccc0 con 0x7f6a24108cf0 2024-03-21T11:20:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.121+0000 7f6a21600640 1 -- 172.21.15.47:0/2435493408 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a0c005020 con 0x7f6a24108cf0 2024-03-21T11:20:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.121+0000 7f6a29400640 1 -- 172.21.15.47:0/2435493408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a241a5d70 con 0x7f6a24108cf0 2024-03-21T11:20:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.121+0000 7f6a21600640 1 -- 172.21.15.47:0/2435493408 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c004a00 con 0x7f6a24108cf0 2024-03-21T11:20:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.121+0000 7f6a29400640 1 -- 172.21.15.47:0/2435493408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a241a6260 con 0x7f6a24108cf0 2024-03-21T11:20:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.122+0000 7f6a29400640 1 -- 172.21.15.47:0/2435493408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a2410ee70 con 0x7f6a24108cf0 2024-03-21T11:20:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.124+0000 7f6a21600640 1 -- 172.21.15.47:0/2435493408 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6a0c0051d0 con 0x7f6a24108cf0 2024-03-21T11:20:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.125+0000 7f6a21600640 1 --2- 172.21.15.47:0/2435493408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a080a2fa0 0x7f6a080a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.125+0000 7f6a21600640 1 -- 172.21.15.47:0/2435493408 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6a0c0ceea0 con 0x7f6a24108cf0 2024-03-21T11:20:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.125+0000 7f6a23e00640 1 --2- 172.21.15.47:0/2435493408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a080a2fa0 0x7f6a080a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.126+0000 7f6a23e00640 1 --2- 172.21.15.47:0/2435493408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a080a2fa0 0x7f6a080a5460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6a14006010 tx=0x7f6a1402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.130+0000 7f6a21600640 1 -- 172.21.15.47:0/2435493408 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6a0c09c0d0 con 0x7f6a24108cf0 2024-03-21T11:20:24.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:23 smithi102 ceph-mon[36748]: Deploying daemon mds.c on smithi102 2024-03-21T11:20:24.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:23 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] up:boot 2024-03-21T11:20:24.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:23 smithi102 ceph-mon[36748]: fsmap 6 up:standby 2024-03-21T11:20:24.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:23 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T11:20:24.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:23 smithi047 ceph-mon[31351]: Deploying daemon mds.c on smithi102 2024-03-21T11:20:24.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:23 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] up:boot 2024-03-21T11:20:24.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:23 smithi047 ceph-mon[31351]: fsmap 6 up:standby 2024-03-21T11:20:24.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:23 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T11:20:24.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.374+0000 7f6a29400640 1 -- 172.21.15.47:0/2435493408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a2402d070 con 0x7f6a24108cf0 2024-03-21T11:20:24.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.376+0000 7f6a21600640 1 -- 172.21.15.47:0/2435493408 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7f6a0c09b9d0 con 0x7f6a24108cf0 2024-03-21T11:20:24.375 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:24.376 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-21T11:20:24.376 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2024-03-21T11:20:24.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.377+0000 7f6a29400640 1 -- 172.21.15.47:0/2435493408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a080a2fa0 msgr2=0x7f6a080a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:24.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.377+0000 7f6a29400640 1 --2- 172.21.15.47:0/2435493408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a080a2fa0 0x7f6a080a5460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6a14006010 tx=0x7f6a1402df90 comp rx=0 tx=0).stop 2024-03-21T11:20:24.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.377+0000 7f6a29400640 1 -- 172.21.15.47:0/2435493408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a24108cf0 msgr2=0x7f6a2419e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:24.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.377+0000 7f6a29400640 1 --2- 172.21.15.47:0/2435493408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a24108cf0 0x7f6a2419e3a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00e9f0 tx=0x7f6a0c00edf0 comp rx=0 tx=0).stop 2024-03-21T11:20:24.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.378+0000 7f6a29400640 1 -- 172.21.15.47:0/2435493408 shutdown_connections 2024-03-21T11:20:24.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.378+0000 7f6a29400640 1 --2- 172.21.15.47:0/2435493408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a080a2fa0 0x7f6a080a5460 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:24.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.378+0000 7f6a29400640 1 --2- 172.21.15.47:0/2435493408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a241096b0 0x7f6a241a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:24.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.378+0000 7f6a29400640 1 --2- 172.21.15.47:0/2435493408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a24108cf0 0x7f6a2419e3a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:24.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.378+0000 7f6a29400640 1 --2- 172.21.15.47:0/2435493408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a24107b00 0x7f6a2419de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:24.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.378+0000 7f6a29400640 1 -- 172.21.15.47:0/2435493408 >> 172.21.15.47:0/2435493408 conn(0x7f6a24076ab0 msgr2=0x7f6a2410c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:24.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.378+0000 7f6a29400640 1 -- 172.21.15.47:0/2435493408 shutdown_connections 2024-03-21T11:20:24.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:24.378+0000 7f6a29400640 1 -- 172.21.15.47:0/2435493408 wait complete. 2024-03-21T11:20:24.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:23 smithi100 ceph-mon[36245]: Deploying daemon mds.c on smithi102 2024-03-21T11:20:24.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:23 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] up:boot 2024-03-21T11:20:24.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:23 smithi100 ceph-mon[36245]: fsmap 6 up:standby 2024-03-21T11:20:24.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:23 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T11:20:24.907 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:20:24 smithi102 systemd[1]: Starting Ceph mds.c for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:20:25.207 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:24 smithi102 ceph-mon[36748]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:25.207 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:24 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2435493408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:25.208 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:20:24 smithi102 podman[67667]: 2024-03-21 11:20:24.843706653 +0000 UTC m=+0.017583745 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:20:25.208 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:20:24 smithi102 podman[67667]: 2024-03-21 11:20:24.972079678 +0000 UTC m=+0.145956775 container create 7eee422f14642b5a1d3c9a9a6b2e08d905230ef8440f1d5423d4c1d8e7bbefc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-21T11:20:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:24 smithi047 ceph-mon[31351]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:24 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2435493408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:25.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:24 smithi100 ceph-mon[36245]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:25.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:24 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2435493408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:25.491 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:20:25 smithi102 podman[67667]: 2024-03-21 11:20:25.208056875 +0000 UTC m=+0.381933959 container init 7eee422f14642b5a1d3c9a9a6b2e08d905230ef8440f1d5423d4c1d8e7bbefc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-c, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-03-21T11:20:25.491 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:20:25 smithi102 podman[67667]: 2024-03-21 11:20:25.21064523 +0000 UTC m=+0.384522309 container start 7eee422f14642b5a1d3c9a9a6b2e08d905230ef8440f1d5423d4c1d8e7bbefc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-c, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-21T11:20:25.491 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:20:25 smithi102 ceph-mds[67686]: -- 172.21.15.102:0/210644513 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b391e5a40 con 0x561b391b9000 2024-03-21T11:20:25.491 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:20:25 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-c[67682]: starting mds.c at 2024-03-21T11:20:25.491 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:20:25 smithi102 bash[67667]: 7eee422f14642b5a1d3c9a9a6b2e08d905230ef8440f1d5423d4c1d8e7bbefc3 2024-03-21T11:20:25.491 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:20:25 smithi102 systemd[1]: Started Ceph mds.c for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:20:26.397 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:26.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.630+0000 7ff545c00640 1 -- 172.21.15.47:0/4010148063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff540105490 msgr2=0x7ff54010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:26.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.630+0000 7ff545c00640 1 --2- 172.21.15.47:0/4010148063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff540105490 0x7ff54010bd20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff538009be0 tx=0x7ff53802bef0 comp rx=0 tx=0).stop 2024-03-21T11:20:26.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.630+0000 7ff545c00640 1 -- 172.21.15.47:0/4010148063 shutdown_connections 2024-03-21T11:20:26.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.630+0000 7ff545c00640 1 --2- 172.21.15.47:0/4010148063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff540105490 0x7ff54010bd20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:26.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.630+0000 7ff545c00640 1 --2- 172.21.15.47:0/4010148063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff540104ad0 0x7ff540104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:26.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.630+0000 7ff545c00640 1 --2- 172.21.15.47:0/4010148063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5401038e0 0x7ff540103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:26.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.630+0000 7ff545c00640 1 -- 172.21.15.47:0/4010148063 >> 172.21.15.47:0/4010148063 conn(0x7ff5400fee50 msgr2=0x7ff540101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:26.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.630+0000 7ff545c00640 1 -- 172.21.15.47:0/4010148063 shutdown_connections 2024-03-21T11:20:26.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.630+0000 7ff545c00640 1 -- 172.21.15.47:0/4010148063 wait complete. 2024-03-21T11:20:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.631+0000 7ff545c00640 1 Processor -- start 2024-03-21T11:20:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.631+0000 7ff545c00640 1 -- start start 2024-03-21T11:20:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff545c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5401038e0 0x7ff540199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff545c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff540104ad0 0x7ff54019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff545c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff540105490 0x7ff5401a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff545c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54010f590 con 0x7ff540105490 2024-03-21T11:20:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff545c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54010f410 con 0x7ff5401038e0 2024-03-21T11:20:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff545c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54010f710 con 0x7ff540104ad0 2024-03-21T11:20:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff53ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5401038e0 0x7ff540199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff53f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff540105490 0x7ff5401a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff53ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5401038e0 0x7ff540199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40294/0 (socket says 172.21.15.47:40294) 2024-03-21T11:20:26.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff53ea00640 1 -- 172.21.15.47:0/3307060013 learned_addr learned my addr 172.21.15.47:0/3307060013 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff53e000640 1 --2- 172.21.15.47:0/3307060013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff540104ad0 0x7ff54019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff53ea00640 1 -- 172.21.15.47:0/3307060013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff540104ad0 msgr2=0x7ff54019a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff53ea00640 1 --2- 172.21.15.47:0/3307060013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff540104ad0 0x7ff54019a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff53ea00640 1 -- 172.21.15.47:0/3307060013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff540105490 msgr2=0x7ff5401a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff53ea00640 1 --2- 172.21.15.47:0/3307060013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff540105490 0x7ff5401a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.632+0000 7ff53ea00640 1 -- 172.21.15.47:0/3307060013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5401a1770 con 0x7ff5401038e0 2024-03-21T11:20:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.633+0000 7ff53ea00640 1 --2- 172.21.15.47:0/3307060013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5401038e0 0x7ff540199b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff53000e9f0 tx=0x7ff53000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.633+0000 7ff537400640 1 -- 172.21.15.47:0/3307060013 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53000ccc0 con 0x7ff5401038e0 2024-03-21T11:20:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.633+0000 7ff545c00640 1 -- 172.21.15.47:0/3307060013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5401a1a60 con 0x7ff5401038e0 2024-03-21T11:20:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.633+0000 7ff537400640 1 -- 172.21.15.47:0/3307060013 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5300035f0 con 0x7ff5401038e0 2024-03-21T11:20:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.633+0000 7ff537400640 1 -- 172.21.15.47:0/3307060013 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53000ce60 con 0x7ff5401038e0 2024-03-21T11:20:26.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.633+0000 7ff545c00640 1 -- 172.21.15.47:0/3307060013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5401a2000 con 0x7ff5401038e0 2024-03-21T11:20:26.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.635+0000 7ff545c00640 1 -- 172.21.15.47:0/3307060013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5200052c0 con 0x7ff5401038e0 2024-03-21T11:20:26.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.636+0000 7ff537400640 1 -- 172.21.15.47:0/3307060013 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff5300037a0 con 0x7ff5401038e0 2024-03-21T11:20:26.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.637+0000 7ff537400640 1 --2- 172.21.15.47:0/3307060013 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff5240a2fa0 0x7ff5240a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:26.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.637+0000 7ff537400640 1 -- 172.21.15.47:0/3307060013 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff5300cf1c0 con 0x7ff5401038e0 2024-03-21T11:20:26.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.638+0000 7ff53e000640 1 --2- 172.21.15.47:0/3307060013 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff5240a2fa0 0x7ff5240a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:26.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.638+0000 7ff53e000640 1 --2- 172.21.15.47:0/3307060013 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff5240a2fa0 0x7ff5240a5460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff54019b060 tx=0x7ff52c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:26.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.641+0000 7ff537400640 1 -- 172.21.15.47:0/3307060013 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff53009c3f0 con 0x7ff5401038e0 2024-03-21T11:20:26.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:26 smithi100 ceph-mon[36245]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:26.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:26.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:26 smithi100 ceph-mon[36245]: Deploying daemon mds.h on smithi100 2024-03-21T11:20:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:26 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] up:boot 2024-03-21T11:20:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:26 smithi100 ceph-mon[36245]: fsmap 7 up:standby 2024-03-21T11:20:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T11:20:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:26 smithi102 ceph-mon[36748]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:26 smithi102 ceph-mon[36748]: Deploying daemon mds.h on smithi100 2024-03-21T11:20:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:26 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] up:boot 2024-03-21T11:20:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:26 smithi102 ceph-mon[36748]: fsmap 7 up:standby 2024-03-21T11:20:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T11:20:26.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:26 smithi047 ceph-mon[31351]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:26.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:26.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:26.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:26.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:26.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:26.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:26.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:26 smithi047 ceph-mon[31351]: Deploying daemon mds.h on smithi100 2024-03-21T11:20:26.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:26 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] up:boot 2024-03-21T11:20:26.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:26 smithi047 ceph-mon[31351]: fsmap 7 up:standby 2024-03-21T11:20:26.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T11:20:26.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.891+0000 7ff545c00640 1 -- 172.21.15.47:0/3307060013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5200055b0 con 0x7ff5401038e0 2024-03-21T11:20:26.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.893+0000 7ff537400640 1 -- 172.21.15.47:0/3307060013 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7ff53009bcf0 con 0x7ff5401038e0 2024-03-21T11:20:26.893 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:26.893 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-21T11:20:26.893 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 8 2024-03-21T11:20:26.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.896+0000 7ff545c00640 1 -- 172.21.15.47:0/3307060013 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff5240a2fa0 msgr2=0x7ff5240a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:26.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.896+0000 7ff545c00640 1 --2- 172.21.15.47:0/3307060013 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff5240a2fa0 0x7ff5240a5460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff54019b060 tx=0x7ff52c005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:26.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.896+0000 7ff545c00640 1 -- 172.21.15.47:0/3307060013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5401038e0 msgr2=0x7ff540199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:26.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.896+0000 7ff545c00640 1 --2- 172.21.15.47:0/3307060013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5401038e0 0x7ff540199b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff53000e9f0 tx=0x7ff53000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:20:26.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.896+0000 7ff545c00640 1 -- 172.21.15.47:0/3307060013 shutdown_connections 2024-03-21T11:20:26.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.896+0000 7ff545c00640 1 --2- 172.21.15.47:0/3307060013 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff5240a2fa0 0x7ff5240a5460 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:26.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.896+0000 7ff545c00640 1 --2- 172.21.15.47:0/3307060013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff540105490 0x7ff5401a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:26.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.896+0000 7ff545c00640 1 --2- 172.21.15.47:0/3307060013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff540104ad0 0x7ff54019a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:26.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.896+0000 7ff545c00640 1 --2- 172.21.15.47:0/3307060013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5401038e0 0x7ff540199b50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:26.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.896+0000 7ff545c00640 1 -- 172.21.15.47:0/3307060013 >> 172.21.15.47:0/3307060013 conn(0x7ff5400fee50 msgr2=0x7ff540107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:26.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.896+0000 7ff545c00640 1 -- 172.21.15.47:0/3307060013 shutdown_connections 2024-03-21T11:20:26.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:26.896+0000 7ff545c00640 1 -- 172.21.15.47:0/3307060013 wait complete. 2024-03-21T11:20:27.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:27 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3307060013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:27.643 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:20:27 smithi100 systemd[1]: Starting Ceph mds.h for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:20:27.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:27 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3307060013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:27.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:27 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3307060013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:28.099 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:20:27 smithi100 podman[65986]: 2024-03-21 11:20:27.729381303 +0000 UTC m=+0.017510363 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:20:28.099 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:20:27 smithi100 podman[65986]: 2024-03-21 11:20:27.845894479 +0000 UTC m=+0.134023516 container create 987da945c2f1dc6a6312d4880d23021a18b7a06261d3dc915844ab449c11eaba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-h, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-21T11:20:28.099 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:20:28 smithi100 podman[65986]: 2024-03-21 11:20:28.07382939 +0000 UTC m=+0.361958439 container init 987da945c2f1dc6a6312d4880d23021a18b7a06261d3dc915844ab449c11eaba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-h, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-21T11:20:28.099 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:20:28 smithi100 podman[65986]: 2024-03-21 11:20:28.076487424 +0000 UTC m=+0.364616469 container start 987da945c2f1dc6a6312d4880d23021a18b7a06261d3dc915844ab449c11eaba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T11:20:28.392 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:20:28 smithi100 ceph-mds[66006]: -- 172.21.15.100:0/4025648755 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562a77511a40 con 0x562a774e4c00 2024-03-21T11:20:28.392 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:20:28 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-h[66002]: starting mds.h at 2024-03-21T11:20:28.393 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:20:28 smithi100 bash[65986]: 987da945c2f1dc6a6312d4880d23021a18b7a06261d3dc915844ab449c11eaba 2024-03-21T11:20:28.393 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:20:28 smithi100 systemd[1]: Started Ceph mds.h for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:20:28.656 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:28 smithi102 ceph-mon[36748]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:28.656 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:28 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:28.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:28 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:28.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:28 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:28.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:28 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:28.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:28 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:28.657 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:28 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:28.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:28 smithi047 ceph-mon[31351]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:28.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:28 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:28.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:28 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:28.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:28 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:28.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:28 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:28.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:28 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:28.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:28 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:28.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:28 smithi100 ceph-mon[36245]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:28.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:28 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:28.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:28 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:28.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:28 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:28.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:28 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:28.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:28 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:28.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:28 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:28.915 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.151+0000 7fc616a00640 1 -- 172.21.15.47:0/2113869778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6100ff980 msgr2=0x7fc6100ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.151+0000 7fc616a00640 1 --2- 172.21.15.47:0/2113869778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6100ff980 0x7fc6100ffda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc600009900 tx=0x7fc60002bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.152+0000 7fc616a00640 1 -- 172.21.15.47:0/2113869778 shutdown_connections 2024-03-21T11:20:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.152+0000 7fc616a00640 1 --2- 172.21.15.47:0/2113869778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6101010a0 0x7fc610105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.152+0000 7fc616a00640 1 --2- 172.21.15.47:0/2113869778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6101006e0 0x7fc610100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.152+0000 7fc616a00640 1 --2- 172.21.15.47:0/2113869778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6100ff980 0x7fc6100ffda0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.152+0000 7fc616a00640 1 -- 172.21.15.47:0/2113869778 >> 172.21.15.47:0/2113869778 conn(0x7fc6100fb2d0 msgr2=0x7fc6100fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.152+0000 7fc616a00640 1 -- 172.21.15.47:0/2113869778 shutdown_connections 2024-03-21T11:20:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.152+0000 7fc616a00640 1 -- 172.21.15.47:0/2113869778 wait complete. 2024-03-21T11:20:29.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.153+0000 7fc616a00640 1 Processor -- start 2024-03-21T11:20:29.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.153+0000 7fc616a00640 1 -- start start 2024-03-21T11:20:29.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.154+0000 7fc616a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6100ff980 0x7fc61019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:29.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.154+0000 7fc616a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6101006e0 0x7fc61019e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:29.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.154+0000 7fc616a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6101010a0 0x7fc6101a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.154+0000 7fc616a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6101139c0 con 0x7fc6101010a0 2024-03-21T11:20:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.154+0000 7fc616a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc610113840 con 0x7fc6101006e0 2024-03-21T11:20:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.154+0000 7fc616a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc610113b40 con 0x7fc6100ff980 2024-03-21T11:20:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.154+0000 7fc60fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6100ff980 0x7fc61019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.154+0000 7fc60fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6100ff980 0x7fc61019dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54214/0 (socket says 172.21.15.47:54214) 2024-03-21T11:20:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.154+0000 7fc60fe00640 1 -- 172.21.15.47:0/3043451194 learned_addr learned my addr 172.21.15.47:0/3043451194 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.154+0000 7fc614a00640 1 --2- 172.21.15.47:0/3043451194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6101010a0 0x7fc6101a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.154+0000 7fc60f400640 1 --2- 172.21.15.47:0/3043451194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6101006e0 0x7fc61019e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.155+0000 7fc60fe00640 1 -- 172.21.15.47:0/3043451194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6101006e0 msgr2=0x7fc61019e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.155+0000 7fc60fe00640 1 --2- 172.21.15.47:0/3043451194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6101006e0 0x7fc61019e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.155+0000 7fc60fe00640 1 -- 172.21.15.47:0/3043451194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6101010a0 msgr2=0x7fc6101a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.155+0000 7fc60fe00640 1 --2- 172.21.15.47:0/3043451194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6101010a0 0x7fc6101a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.155+0000 7fc60fe00640 1 -- 172.21.15.47:0/3043451194 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6101a5c10 con 0x7fc6100ff980 2024-03-21T11:20:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.155+0000 7fc60fe00640 1 --2- 172.21.15.47:0/3043451194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6100ff980 0x7fc61019dff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc600000c00 tx=0x7fc60002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.155+0000 7fc60cc00640 1 -- 172.21.15.47:0/3043451194 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600043020 con 0x7fc6100ff980 2024-03-21T11:20:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.155+0000 7fc60cc00640 1 -- 172.21.15.47:0/3043451194 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6000054e0 con 0x7fc6100ff980 2024-03-21T11:20:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.155+0000 7fc616a00640 1 -- 172.21.15.47:0/3043451194 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6101a5ea0 con 0x7fc6100ff980 2024-03-21T11:20:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.155+0000 7fc60cc00640 1 -- 172.21.15.47:0/3043451194 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600003bd0 con 0x7fc6100ff980 2024-03-21T11:20:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.156+0000 7fc616a00640 1 -- 172.21.15.47:0/3043451194 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6101a6360 con 0x7fc6100ff980 2024-03-21T11:20:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.157+0000 7fc616a00640 1 -- 172.21.15.47:0/3043451194 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc610105040 con 0x7fc6100ff980 2024-03-21T11:20:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.158+0000 7fc60cc00640 1 -- 172.21.15.47:0/3043451194 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc600046020 con 0x7fc6100ff980 2024-03-21T11:20:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.160+0000 7fc60cc00640 1 --2- 172.21.15.47:0/3043451194 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5f40a2fa0 0x7fc5f40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.160+0000 7fc60cc00640 1 -- 172.21.15.47:0/3043451194 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc6000f0ff0 con 0x7fc6100ff980 2024-03-21T11:20:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.160+0000 7fc60f400640 1 --2- 172.21.15.47:0/3043451194 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5f40a2fa0 0x7fc5f40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.161+0000 7fc60f400640 1 --2- 172.21.15.47:0/3043451194 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5f40a2fa0 0x7fc5f40a5460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc61019f500 tx=0x7fc5fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.164+0000 7fc60cc00640 1 -- 172.21.15.47:0/3043451194 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc60004d020 con 0x7fc6100ff980 2024-03-21T11:20:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.414+0000 7fc616a00640 1 -- 172.21.15.47:0/3043451194 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc61002d070 con 0x7fc6100ff980 2024-03-21T11:20:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.415+0000 7fc60cc00640 1 -- 172.21.15.47:0/3043451194 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7fc6000be280 con 0x7fc6100ff980 2024-03-21T11:20:29.415 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:29.415 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-21T11:20:29.415 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-03-21T11:20:29.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.418+0000 7fc616a00640 1 -- 172.21.15.47:0/3043451194 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5f40a2fa0 msgr2=0x7fc5f40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.418+0000 7fc616a00640 1 --2- 172.21.15.47:0/3043451194 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5f40a2fa0 0x7fc5f40a5460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc61019f500 tx=0x7fc5fc005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.418+0000 7fc616a00640 1 -- 172.21.15.47:0/3043451194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6100ff980 msgr2=0x7fc61019dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.418+0000 7fc616a00640 1 --2- 172.21.15.47:0/3043451194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6100ff980 0x7fc61019dff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc600000c00 tx=0x7fc60002c650 comp rx=0 tx=0).stop 2024-03-21T11:20:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.418+0000 7fc616a00640 1 -- 172.21.15.47:0/3043451194 shutdown_connections 2024-03-21T11:20:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.418+0000 7fc616a00640 1 --2- 172.21.15.47:0/3043451194 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5f40a2fa0 0x7fc5f40a5460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.418+0000 7fc616a00640 1 --2- 172.21.15.47:0/3043451194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6101010a0 0x7fc6101a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.418+0000 7fc616a00640 1 --2- 172.21.15.47:0/3043451194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6101006e0 0x7fc61019e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.418+0000 7fc616a00640 1 --2- 172.21.15.47:0/3043451194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6100ff980 0x7fc61019dff0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.418+0000 7fc616a00640 1 -- 172.21.15.47:0/3043451194 >> 172.21.15.47:0/3043451194 conn(0x7fc6100fb2d0 msgr2=0x7fc6100fbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.418+0000 7fc616a00640 1 -- 172.21.15.47:0/3043451194 shutdown_connections 2024-03-21T11:20:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:29.418+0000 7fc616a00640 1 -- 172.21.15.47:0/3043451194 wait complete. 2024-03-21T11:20:29.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:29 smithi102 ceph-mon[36748]: Deploying daemon mds.i on smithi102 2024-03-21T11:20:29.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:29 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] up:boot 2024-03-21T11:20:29.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:29 smithi102 ceph-mon[36748]: fsmap 8 up:standby 2024-03-21T11:20:29.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T11:20:29.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:29 smithi047 ceph-mon[31351]: Deploying daemon mds.i on smithi102 2024-03-21T11:20:29.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:29 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] up:boot 2024-03-21T11:20:29.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:29 smithi047 ceph-mon[31351]: fsmap 8 up:standby 2024-03-21T11:20:29.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T11:20:29.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:29 smithi100 ceph-mon[36245]: Deploying daemon mds.i on smithi102 2024-03-21T11:20:29.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:29 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] up:boot 2024-03-21T11:20:29.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:29 smithi100 ceph-mon[36245]: fsmap 8 up:standby 2024-03-21T11:20:29.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T11:20:30.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:30 smithi102 ceph-mon[36748]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:30.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:30 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3043451194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:30.741 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:20:30 smithi102 systemd[1]: Starting Ceph mds.i for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:20:30.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:30 smithi047 ceph-mon[31351]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:30.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:30 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3043451194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:30.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:30 smithi100 ceph-mon[36245]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:30.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:30 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3043451194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:31.122 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:20:30 smithi102 podman[67982]: 2024-03-21 11:20:30.767013599 +0000 UTC m=+0.017193764 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:20:31.123 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:20:30 smithi102 podman[67982]: 2024-03-21 11:20:30.869898 +0000 UTC m=+0.120078174 container create 19a552d32b82ab30d9e409aecd2ccd2e0370c2ba87b1a1a971d698cd490cceda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-i, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:20:31.123 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:20:31 smithi102 podman[67982]: 2024-03-21 11:20:31.123121307 +0000 UTC m=+0.373301470 container init 19a552d32b82ab30d9e409aecd2ccd2e0370c2ba87b1a1a971d698cd490cceda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:20:31.436 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:31.491 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:20:31 smithi102 podman[67982]: 2024-03-21 11:20:31.126117242 +0000 UTC m=+0.376297397 container start 19a552d32b82ab30d9e409aecd2ccd2e0370c2ba87b1a1a971d698cd490cceda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-i, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-21T11:20:31.491 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:20:31 smithi102 ceph-mds[68001]: -- 172.21.15.102:0/4125345631 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55835a525a40 con 0x55835a4f8c00 2024-03-21T11:20:31.491 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:20:31 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-i[67997]: starting mds.i at 2024-03-21T11:20:31.491 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:20:31 smithi102 bash[67982]: 19a552d32b82ab30d9e409aecd2ccd2e0370c2ba87b1a1a971d698cd490cceda 2024-03-21T11:20:31.491 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:20:31 smithi102 systemd[1]: Started Ceph mds.i for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:20:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.634+0000 7f52c2000640 1 -- 172.21.15.47:0/3604481058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52bc0fff90 msgr2=0x7f52bc1003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.634+0000 7f52c2000640 1 --2- 172.21.15.47:0/3604481058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52bc0fff90 0x7f52bc1003b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009ca0 tx=0x7f52ac02c100 comp rx=0 tx=0).stop 2024-03-21T11:20:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.635+0000 7f52c2000640 1 -- 172.21.15.47:0/3604481058 shutdown_connections 2024-03-21T11:20:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.635+0000 7f52c2000640 1 --2- 172.21.15.47:0/3604481058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52bc101290 0x7f52bc101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.635+0000 7f52c2000640 1 --2- 172.21.15.47:0/3604481058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52bc1008f0 0x7f52bc100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.635+0000 7f52c2000640 1 --2- 172.21.15.47:0/3604481058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52bc0fff90 0x7f52bc1003b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.635+0000 7f52c2000640 1 -- 172.21.15.47:0/3604481058 >> 172.21.15.47:0/3604481058 conn(0x7f52bc0fbb30 msgr2=0x7f52bc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.635+0000 7f52c2000640 1 -- 172.21.15.47:0/3604481058 shutdown_connections 2024-03-21T11:20:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.635+0000 7f52c2000640 1 -- 172.21.15.47:0/3604481058 wait complete. 2024-03-21T11:20:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.636+0000 7f52c2000640 1 Processor -- start 2024-03-21T11:20:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.636+0000 7f52c2000640 1 -- start start 2024-03-21T11:20:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.636+0000 7f52c2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52bc0fff90 0x7f52bc199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.636+0000 7f52c2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52bc1008f0 0x7f52bc19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.636+0000 7f52c2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52bc101290 0x7f52bc1a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.636+0000 7f52c2000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc10f520 con 0x7f52bc0fff90 2024-03-21T11:20:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.636+0000 7f52c2000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc10f3a0 con 0x7f52bc101290 2024-03-21T11:20:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.636+0000 7f52c2000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc10f6a0 con 0x7f52bc1008f0 2024-03-21T11:20:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.637+0000 7f52bbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52bc101290 0x7f52bc1a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.637+0000 7f52bb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52bc0fff90 0x7f52bc199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.637+0000 7f52baa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52bc1008f0 0x7f52bc19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.637+0000 7f52bbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52bc101290 0x7f52bc1a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37950/0 (socket says 172.21.15.47:37950) 2024-03-21T11:20:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.637+0000 7f52bbe00640 1 -- 172.21.15.47:0/3430626054 learned_addr learned my addr 172.21.15.47:0/3430626054 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.637+0000 7f52bbe00640 1 -- 172.21.15.47:0/3430626054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52bc1008f0 msgr2=0x7f52bc19a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.637+0000 7f52bbe00640 1 --2- 172.21.15.47:0/3430626054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52bc1008f0 0x7f52bc19a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.637+0000 7f52bbe00640 1 -- 172.21.15.47:0/3430626054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52bc0fff90 msgr2=0x7f52bc199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.637+0000 7f52bbe00640 1 --2- 172.21.15.47:0/3430626054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52bc0fff90 0x7f52bc199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.637+0000 7f52bbe00640 1 -- 172.21.15.47:0/3430626054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc1a17f0 con 0x7f52bc101290 2024-03-21T11:20:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.637+0000 7f52baa00640 1 --2- 172.21.15.47:0/3430626054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52bc1008f0 0x7f52bc19a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:20:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.637+0000 7f52bb400640 1 --2- 172.21.15.47:0/3430626054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52bc0fff90 0x7f52bc199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:20:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.637+0000 7f52bbe00640 1 --2- 172.21.15.47:0/3430626054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52bc101290 0x7f52bc1a1180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f52b400e9b0 tx=0x7f52b400ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.638+0000 7f52b3e00640 1 -- 172.21.15.47:0/3430626054 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b400cd80 con 0x7f52bc101290 2024-03-21T11:20:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.638+0000 7f52b3e00640 1 -- 172.21.15.47:0/3430626054 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52b40035f0 con 0x7f52bc101290 2024-03-21T11:20:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.638+0000 7f52b3e00640 1 -- 172.21.15.47:0/3430626054 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4010670 con 0x7f52bc101290 2024-03-21T11:20:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.638+0000 7f52c2000640 1 -- 172.21.15.47:0/3430626054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52bc1a1ae0 con 0x7f52bc101290 2024-03-21T11:20:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.638+0000 7f52c2000640 1 -- 172.21.15.47:0/3430626054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52bc1a2030 con 0x7f52bc101290 2024-03-21T11:20:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.639+0000 7f52c2000640 1 -- 172.21.15.47:0/3430626054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52bc10aec0 con 0x7f52bc101290 2024-03-21T11:20:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.641+0000 7f52b3e00640 1 -- 172.21.15.47:0/3430626054 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f52b40037a0 con 0x7f52bc101290 2024-03-21T11:20:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.642+0000 7f52b3e00640 1 --2- 172.21.15.47:0/3430626054 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f52a00a2fa0 0x7f52a00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.642+0000 7f52b3e00640 1 -- 172.21.15.47:0/3430626054 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f52b40cf260 con 0x7f52bc101290 2024-03-21T11:20:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.643+0000 7f52bb400640 1 --2- 172.21.15.47:0/3430626054 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f52a00a2fa0 0x7f52a00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:31.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.643+0000 7f52bb400640 1 --2- 172.21.15.47:0/3430626054 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f52a00a2fa0 0x7f52a00a5460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f52ac006010 tx=0x7f52ac032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.646+0000 7f52b3e00640 1 -- 172.21.15.47:0/3430626054 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f52b409c490 con 0x7f52bc101290 2024-03-21T11:20:31.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.893+0000 7f52c2000640 1 -- 172.21.15.47:0/3430626054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52bc0622a0 con 0x7f52bc101290 2024-03-21T11:20:31.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.894+0000 7f52b3e00640 1 -- 172.21.15.47:0/3430626054 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7f52b409bd90 con 0x7f52bc101290 2024-03-21T11:20:31.895 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:31.895 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-21T11:20:31.895 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 10 2024-03-21T11:20:31.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.897+0000 7f52c2000640 1 -- 172.21.15.47:0/3430626054 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f52a00a2fa0 msgr2=0x7f52a00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:31.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.897+0000 7f52c2000640 1 --2- 172.21.15.47:0/3430626054 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f52a00a2fa0 0x7f52a00a5460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f52ac006010 tx=0x7f52ac032590 comp rx=0 tx=0).stop 2024-03-21T11:20:31.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.897+0000 7f52c2000640 1 -- 172.21.15.47:0/3430626054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52bc101290 msgr2=0x7f52bc1a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:31.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.897+0000 7f52c2000640 1 --2- 172.21.15.47:0/3430626054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52bc101290 0x7f52bc1a1180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f52b400e9b0 tx=0x7f52b400ee70 comp rx=0 tx=0).stop 2024-03-21T11:20:31.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.897+0000 7f52c2000640 1 -- 172.21.15.47:0/3430626054 shutdown_connections 2024-03-21T11:20:31.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.897+0000 7f52c2000640 1 --2- 172.21.15.47:0/3430626054 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f52a00a2fa0 0x7f52a00a5460 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:31.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.897+0000 7f52c2000640 1 --2- 172.21.15.47:0/3430626054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52bc101290 0x7f52bc1a1180 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:31.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.897+0000 7f52c2000640 1 --2- 172.21.15.47:0/3430626054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52bc1008f0 0x7f52bc19a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:31.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.897+0000 7f52c2000640 1 --2- 172.21.15.47:0/3430626054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52bc0fff90 0x7f52bc199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:31.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.897+0000 7f52c2000640 1 -- 172.21.15.47:0/3430626054 >> 172.21.15.47:0/3430626054 conn(0x7f52bc0fbb30 msgr2=0x7f52bc0fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:31.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.897+0000 7f52c2000640 1 -- 172.21.15.47:0/3430626054 shutdown_connections 2024-03-21T11:20:31.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:31.897+0000 7f52c2000640 1 -- 172.21.15.47:0/3430626054 wait complete. 2024-03-21T11:20:32.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:32 smithi047 ceph-mon[31351]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:32.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:32 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:32.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:32 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:32.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:32 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:32.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:32 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:32.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:32 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:32.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:32 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:32.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:32 smithi047 ceph-mon[31351]: Deploying daemon mds.g on smithi047 2024-03-21T11:20:32.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:32 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] up:boot 2024-03-21T11:20:32.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:32 smithi047 ceph-mon[31351]: fsmap 9 up:standby 2024-03-21T11:20:32.550 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:32 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T11:20:32.550 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:32 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3430626054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:32.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:32 smithi100 ceph-mon[36245]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:32 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:32 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:32 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:32 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:32 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:32 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:32 smithi100 ceph-mon[36245]: Deploying daemon mds.g on smithi047 2024-03-21T11:20:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:32 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] up:boot 2024-03-21T11:20:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:32 smithi100 ceph-mon[36245]: fsmap 9 up:standby 2024-03-21T11:20:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:32 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T11:20:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:32 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3430626054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:32.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:32 smithi102 ceph-mon[36748]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:32.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:32 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:32.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:32 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:32.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:32 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:32.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:32 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:32.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:32 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:32.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:32 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:32.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:32 smithi102 ceph-mon[36748]: Deploying daemon mds.g on smithi047 2024-03-21T11:20:32.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:32 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] up:boot 2024-03-21T11:20:32.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:32 smithi102 ceph-mon[36748]: fsmap 9 up:standby 2024-03-21T11:20:32.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:32 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T11:20:32.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:32 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3430626054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:33.919 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:34.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.196+0000 7f6e76600640 1 -- 172.21.15.47:0/3691882599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7010b010 msgr2=0x7f6e7010d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:34.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.196+0000 7f6e76600640 1 --2- 172.21.15.47:0/3691882599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7010b010 0x7f6e7010d4a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009900 tx=0x7f6e5c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:34.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.196+0000 7f6e76600640 1 -- 172.21.15.47:0/3691882599 shutdown_connections 2024-03-21T11:20:34.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.196+0000 7f6e76600640 1 --2- 172.21.15.47:0/3691882599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7010d9e0 0x7f6e7010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:34.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.196+0000 7f6e76600640 1 --2- 172.21.15.47:0/3691882599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7010b010 0x7f6e7010d4a0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009900 tx=0x7f6e5c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:34.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.196+0000 7f6e76600640 1 --2- 172.21.15.47:0/3691882599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e701086a0 0x7f6e7010aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:34.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.196+0000 7f6e76600640 1 -- 172.21.15.47:0/3691882599 >> 172.21.15.47:0/3691882599 conn(0x7f6e7006e510 msgr2=0x7f6e70070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:34.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.199+0000 7f6e76600640 1 -- 172.21.15.47:0/3691882599 shutdown_connections 2024-03-21T11:20:34.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.199+0000 7f6e76600640 1 -- 172.21.15.47:0/3691882599 wait complete. 2024-03-21T11:20:34.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.200+0000 7f6e76600640 1 Processor -- start 2024-03-21T11:20:34.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.200+0000 7f6e76600640 1 -- start start 2024-03-21T11:20:34.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.201+0000 7f6e76600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e701086a0 0x7f6e701a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:34.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.201+0000 7f6e76600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7010b010 0x7f6e701a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:34.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.201+0000 7f6e76600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7010d9e0 0x7f6e701adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:34.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.201+0000 7f6e76600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7011c1e0 con 0x7f6e7010b010 2024-03-21T11:20:34.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.201+0000 7f6e76600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7011c060 con 0x7f6e7010d9e0 2024-03-21T11:20:34.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.201+0000 7f6e76600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7011c360 con 0x7f6e701086a0 2024-03-21T11:20:34.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.201+0000 7f6e6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7010d9e0 0x7f6e701adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:34.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.201+0000 7f6e6ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7010b010 0x7f6e701a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:34.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.201+0000 7f6e6f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e701086a0 0x7f6e701a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:34.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.201+0000 7f6e6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7010d9e0 0x7f6e701adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37970/0 (socket says 172.21.15.47:37970) 2024-03-21T11:20:34.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.201+0000 7f6e6fe00640 1 -- 172.21.15.47:0/311722604 learned_addr learned my addr 172.21.15.47:0/311722604 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:34.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.202+0000 7f6e6fe00640 1 -- 172.21.15.47:0/311722604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e701086a0 msgr2=0x7f6e701a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:34.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.202+0000 7f6e6fe00640 1 --2- 172.21.15.47:0/311722604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e701086a0 0x7f6e701a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:34.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.202+0000 7f6e6fe00640 1 -- 172.21.15.47:0/311722604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7010b010 msgr2=0x7f6e701a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:34.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.202+0000 7f6e6fe00640 1 --2- 172.21.15.47:0/311722604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7010b010 0x7f6e701a6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:34.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.202+0000 7f6e6fe00640 1 -- 172.21.15.47:0/311722604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e701ae410 con 0x7f6e7010d9e0 2024-03-21T11:20:34.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.202+0000 7f6e6fe00640 1 --2- 172.21.15.47:0/311722604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7010d9e0 0x7f6e701adda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009c90 tx=0x7f6e6800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:34.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.202+0000 7f6e67e00640 1 -- 172.21.15.47:0/311722604 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68003b70 con 0x7f6e7010d9e0 2024-03-21T11:20:34.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.203+0000 7f6e67e00640 1 -- 172.21.15.47:0/311722604 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e68003d10 con 0x7f6e7010d9e0 2024-03-21T11:20:34.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.203+0000 7f6e67e00640 1 -- 172.21.15.47:0/311722604 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68010960 con 0x7f6e7010d9e0 2024-03-21T11:20:34.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.203+0000 7f6e76600640 1 -- 172.21.15.47:0/311722604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e701ae700 con 0x7f6e7010d9e0 2024-03-21T11:20:34.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.203+0000 7f6e76600640 1 -- 172.21.15.47:0/311722604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e701aec20 con 0x7f6e7010d9e0 2024-03-21T11:20:34.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.204+0000 7f6e76600640 1 -- 172.21.15.47:0/311722604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e70071010 con 0x7f6e7010d9e0 2024-03-21T11:20:34.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.206+0000 7f6e67e00640 1 -- 172.21.15.47:0/311722604 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6e68004020 con 0x7f6e7010d9e0 2024-03-21T11:20:34.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.207+0000 7f6e67e00640 1 --2- 172.21.15.47:0/311722604 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6e540a2ff0 0x7f6e540a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:34.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.208+0000 7f6e67e00640 1 -- 172.21.15.47:0/311722604 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6e680cf550 con 0x7f6e7010d9e0 2024-03-21T11:20:34.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.208+0000 7f6e6f400640 1 --2- 172.21.15.47:0/311722604 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6e540a2ff0 0x7f6e540a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:34.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.209+0000 7f6e6f400640 1 --2- 172.21.15.47:0/311722604 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6e540a2ff0 0x7f6e540a54b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6e70072790 tx=0x7f6e60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:34.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.212+0000 7f6e67e00640 1 -- 172.21.15.47:0/311722604 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6e6809c780 con 0x7f6e7010d9e0 2024-03-21T11:20:34.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:34 smithi047 ceph-mon[31351]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:34.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.471+0000 7f6e76600640 1 -- 172.21.15.47:0/311722604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e700622a0 con 0x7f6e7010d9e0 2024-03-21T11:20:34.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.472+0000 7f6e67e00640 1 -- 172.21.15.47:0/311722604 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7f6e6809c080 con 0x7f6e7010d9e0 2024-03-21T11:20:34.473 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:34.473 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-21T11:20:34.473 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 10 2024-03-21T11:20:34.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.475+0000 7f6e76600640 1 -- 172.21.15.47:0/311722604 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6e540a2ff0 msgr2=0x7f6e540a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:34.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.475+0000 7f6e76600640 1 --2- 172.21.15.47:0/311722604 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6e540a2ff0 0x7f6e540a54b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6e70072790 tx=0x7f6e60005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:34.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.475+0000 7f6e76600640 1 -- 172.21.15.47:0/311722604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7010d9e0 msgr2=0x7f6e701adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:34.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.475+0000 7f6e76600640 1 --2- 172.21.15.47:0/311722604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7010d9e0 0x7f6e701adda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009c90 tx=0x7f6e6800c6a0 comp rx=0 tx=0).stop 2024-03-21T11:20:34.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.475+0000 7f6e76600640 1 -- 172.21.15.47:0/311722604 shutdown_connections 2024-03-21T11:20:34.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.475+0000 7f6e76600640 1 --2- 172.21.15.47:0/311722604 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6e540a2ff0 0x7f6e540a54b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:34.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.475+0000 7f6e76600640 1 --2- 172.21.15.47:0/311722604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7010d9e0 0x7f6e701adda0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:34.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.475+0000 7f6e76600640 1 --2- 172.21.15.47:0/311722604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e7010b010 0x7f6e701a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:34.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.475+0000 7f6e76600640 1 --2- 172.21.15.47:0/311722604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e701086a0 0x7f6e701a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:34.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.475+0000 7f6e76600640 1 -- 172.21.15.47:0/311722604 >> 172.21.15.47:0/311722604 conn(0x7f6e7006e510 msgr2=0x7f6e70108ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:34.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.475+0000 7f6e76600640 1 -- 172.21.15.47:0/311722604 shutdown_connections 2024-03-21T11:20:34.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:34.475+0000 7f6e76600640 1 -- 172.21.15.47:0/311722604 wait complete. 2024-03-21T11:20:34.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:34 smithi100 ceph-mon[36245]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:34.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:34 smithi102 ceph-mon[36748]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:35.065 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 11:20:34 smithi047 podman[70175]: 2024-03-21 11:20:34.838468866 +0000 UTC m=+0.159542533 container create d14b154447d8c489c4672eea2e32eae8d23c6c934e5ac3fbbc7429c297ef2b22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-g, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-03-21T11:20:35.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:35 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/311722604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:35.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:35.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:35.564 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 11:20:35 smithi047 podman[70175]: 2024-03-21 11:20:35.083899916 +0000 UTC m=+0.404973589 container init d14b154447d8c489c4672eea2e32eae8d23c6c934e5ac3fbbc7429c297ef2b22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-g, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-21T11:20:35.564 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 11:20:35 smithi047 podman[70175]: 2024-03-21 11:20:35.094281795 +0000 UTC m=+0.415355450 container start d14b154447d8c489c4672eea2e32eae8d23c6c934e5ac3fbbc7429c297ef2b22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-g, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T11:20:35.564 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 11:20:35 smithi047 ceph-mds[70193]: -- 172.21.15.47:0/241932956 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561fa391ba40 con 0x561fa38ee800 2024-03-21T11:20:35.564 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 11:20:35 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-g[70189]: starting mds.g at 2024-03-21T11:20:35.564 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 11:20:35 smithi047 bash[70175]: d14b154447d8c489c4672eea2e32eae8d23c6c934e5ac3fbbc7429c297ef2b22 2024-03-21T11:20:35.565 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 11:20:35 smithi047 systemd[1]: Started Ceph mds.g for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:20:35.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:35 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/311722604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:35.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:35 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:35.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:35 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:35.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:35 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/311722604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:35.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:35 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:35.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:35 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:36.495 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:36 smithi047 ceph-mon[31351]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:36 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.47:6838/1314095873,v1:172.21.15.47:6839/1314095873] up:boot 2024-03-21T11:20:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:36 smithi047 ceph-mon[31351]: fsmap 10 up:standby 2024-03-21T11:20:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T11:20:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:36 smithi047 ceph-mon[31351]: Deploying daemon mds.a on smithi047 2024-03-21T11:20:36.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:36 smithi100 ceph-mon[36245]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:36.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:36 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.47:6838/1314095873,v1:172.21.15.47:6839/1314095873] up:boot 2024-03-21T11:20:36.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:36 smithi100 ceph-mon[36245]: fsmap 10 up:standby 2024-03-21T11:20:36.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T11:20:36.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:36.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:36.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:36.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:36.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:36 smithi100 ceph-mon[36245]: Deploying daemon mds.a on smithi047 2024-03-21T11:20:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:36 smithi102 ceph-mon[36748]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:36 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.47:6838/1314095873,v1:172.21.15.47:6839/1314095873] up:boot 2024-03-21T11:20:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:36 smithi102 ceph-mon[36748]: fsmap 10 up:standby 2024-03-21T11:20:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T11:20:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:36 smithi102 ceph-mon[36748]: Deploying daemon mds.a on smithi047 2024-03-21T11:20:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.741+0000 7f8089e00640 1 -- 172.21.15.47:0/3312840306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80841030b0 msgr2=0x7f8084109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.741+0000 7f8089e00640 1 --2- 172.21.15.47:0/3312840306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80841030b0 0x7f8084109940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f807c009900 tx=0x7f807c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.741+0000 7f8089e00640 1 -- 172.21.15.47:0/3312840306 shutdown_connections 2024-03-21T11:20:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.741+0000 7f8089e00640 1 --2- 172.21.15.47:0/3312840306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80841030b0 0x7f8084109940 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.741+0000 7f8089e00640 1 --2- 172.21.15.47:0/3312840306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80841026f0 0x7f8084102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.741+0000 7f8089e00640 1 --2- 172.21.15.47:0/3312840306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8084101500 0x7f8084101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.741+0000 7f8089e00640 1 -- 172.21.15.47:0/3312840306 >> 172.21.15.47:0/3312840306 conn(0x7f80840fca70 msgr2=0x7f80840feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.741+0000 7f8089e00640 1 -- 172.21.15.47:0/3312840306 shutdown_connections 2024-03-21T11:20:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.742+0000 7f8089e00640 1 -- 172.21.15.47:0/3312840306 wait complete. 2024-03-21T11:20:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.742+0000 7f8089e00640 1 Processor -- start 2024-03-21T11:20:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.743+0000 7f8089e00640 1 -- start start 2024-03-21T11:20:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.743+0000 7f8089e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8084101500 0x7f8084199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.743+0000 7f8089e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80841026f0 0x7f8084199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.743+0000 7f8089e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80841030b0 0x7f80841a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.743+0000 7f8089e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808410d1e0 con 0x7f80841030b0 2024-03-21T11:20:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.743+0000 7f8089e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808410d060 con 0x7f8084101500 2024-03-21T11:20:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.743+0000 7f8089e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808410d360 con 0x7f80841026f0 2024-03-21T11:20:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.743+0000 7f8089400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80841030b0 0x7f80841a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.743+0000 7f8083e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80841026f0 0x7f8084199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.743+0000 7f8088a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8084101500 0x7f8084199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.743+0000 7f8089400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80841030b0 0x7f80841a0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59614/0 (socket says 172.21.15.47:59614) 2024-03-21T11:20:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.743+0000 7f8089400640 1 -- 172.21.15.47:0/1087883475 learned_addr learned my addr 172.21.15.47:0/1087883475 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.744+0000 7f8083e00640 1 -- 172.21.15.47:0/1087883475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8084101500 msgr2=0x7f8084199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.744+0000 7f8083e00640 1 --2- 172.21.15.47:0/1087883475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8084101500 0x7f8084199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.744+0000 7f8083e00640 1 -- 172.21.15.47:0/1087883475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80841030b0 msgr2=0x7f80841a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.744+0000 7f8083e00640 1 --2- 172.21.15.47:0/1087883475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80841030b0 0x7f80841a0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.744+0000 7f8083e00640 1 -- 172.21.15.47:0/1087883475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80841a1590 con 0x7f80841026f0 2024-03-21T11:20:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.744+0000 7f8083e00640 1 --2- 172.21.15.47:0/1087883475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80841026f0 0x7f8084199eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f807000e9f0 tx=0x7f807000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.744+0000 7f8081600640 1 -- 172.21.15.47:0/1087883475 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807000ccc0 con 0x7f80841026f0 2024-03-21T11:20:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.744+0000 7f8081600640 1 -- 172.21.15.47:0/1087883475 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8070005020 con 0x7f80841026f0 2024-03-21T11:20:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.744+0000 7f8081600640 1 -- 172.21.15.47:0/1087883475 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80700049e0 con 0x7f80841026f0 2024-03-21T11:20:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.745+0000 7f8089e00640 1 -- 172.21.15.47:0/1087883475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80841a1880 con 0x7f80841026f0 2024-03-21T11:20:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.745+0000 7f8089e00640 1 -- 172.21.15.47:0/1087883475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80841a1d80 con 0x7f80841026f0 2024-03-21T11:20:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.746+0000 7f8089e00640 1 -- 172.21.15.47:0/1087883475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80841a2040 con 0x7f80841026f0 2024-03-21T11:20:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.748+0000 7f8081600640 1 -- 172.21.15.47:0/1087883475 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8070005560 con 0x7f80841026f0 2024-03-21T11:20:36.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.750+0000 7f8081600640 1 --2- 172.21.15.47:0/1087883475 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80680a2f20 0x7f80680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:36.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.750+0000 7f8081600640 1 -- 172.21.15.47:0/1087883475 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f80700cef70 con 0x7f80841026f0 2024-03-21T11:20:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.750+0000 7f8088a00640 1 --2- 172.21.15.47:0/1087883475 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80680a2f20 0x7f80680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.751+0000 7f8088a00640 1 --2- 172.21.15.47:0/1087883475 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80680a2f20 0x7f80680a53e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8084102550 tx=0x7f8074009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:36.754+0000 7f8081600640 1 -- 172.21.15.47:0/1087883475 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f80700051d0 con 0x7f80841026f0 2024-03-21T11:20:37.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:37.003+0000 7f8089e00640 1 -- 172.21.15.47:0/1087883475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80841a2210 con 0x7f80841026f0 2024-03-21T11:20:37.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:37.004+0000 7f8081600640 1 -- 172.21.15.47:0/1087883475 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8465 (secure 0 0 0) 0x7f807009c1a0 con 0x7f80841026f0 2024-03-21T11:20:37.005 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:37.005 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-21T11:20:37.005 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2024-03-21T11:20:37.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:37.007+0000 7f8089e00640 1 -- 172.21.15.47:0/1087883475 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80680a2f20 msgr2=0x7f80680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:37.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:37.007+0000 7f8089e00640 1 --2- 172.21.15.47:0/1087883475 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80680a2f20 0x7f80680a53e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8084102550 tx=0x7f8074009500 comp rx=0 tx=0).stop 2024-03-21T11:20:37.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:37.007+0000 7f8089e00640 1 -- 172.21.15.47:0/1087883475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80841026f0 msgr2=0x7f8084199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:37.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:37.007+0000 7f8089e00640 1 --2- 172.21.15.47:0/1087883475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80841026f0 0x7f8084199eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f807000e9f0 tx=0x7f807000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:20:37.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:37.007+0000 7f8089e00640 1 -- 172.21.15.47:0/1087883475 shutdown_connections 2024-03-21T11:20:37.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:37.007+0000 7f8089e00640 1 --2- 172.21.15.47:0/1087883475 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80680a2f20 0x7f80680a53e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:37.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:37.007+0000 7f8089e00640 1 --2- 172.21.15.47:0/1087883475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80841030b0 0x7f80841a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:37.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:37.007+0000 7f8089e00640 1 --2- 172.21.15.47:0/1087883475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80841026f0 0x7f8084199eb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:37.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:37.007+0000 7f8089e00640 1 --2- 172.21.15.47:0/1087883475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8084101500 0x7f8084199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:37.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:37.007+0000 7f8089e00640 1 -- 172.21.15.47:0/1087883475 >> 172.21.15.47:0/1087883475 conn(0x7f80840fca70 msgr2=0x7f80840738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:37.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:37.007+0000 7f8089e00640 1 -- 172.21.15.47:0/1087883475 shutdown_connections 2024-03-21T11:20:37.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:37.007+0000 7f8089e00640 1 -- 172.21.15.47:0/1087883475 wait complete. 2024-03-21T11:20:37.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:37 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1087883475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:37.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:37 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1087883475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:37.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:37 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1087883475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:38.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:38 smithi047 ceph-mon[31351]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:38.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:38 smithi100 ceph-mon[36245]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:38.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:38 smithi102 ceph-mon[36748]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:39.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:39.064 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:20:38 smithi047 systemd[1]: Starting Ceph mds.a for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:20:39.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.247+0000 7f86af000640 1 -- 172.21.15.47:0/1433089527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a81096b0 msgr2=0x7f86a810ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.247+0000 7f86af000640 1 --2- 172.21.15.47:0/1433089527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a81096b0 0x7f86a810ff40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009ca0 tx=0x7f86a002c0f0 comp rx=0 tx=0).stop 2024-03-21T11:20:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.247+0000 7f86af000640 1 -- 172.21.15.47:0/1433089527 shutdown_connections 2024-03-21T11:20:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.247+0000 7f86af000640 1 --2- 172.21.15.47:0/1433089527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a81096b0 0x7f86a810ff40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.247+0000 7f86af000640 1 --2- 172.21.15.47:0/1433089527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86a8108cf0 0x7f86a8109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.247+0000 7f86af000640 1 --2- 172.21.15.47:0/1433089527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a8107b00 0x7f86a8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.247+0000 7f86af000640 1 -- 172.21.15.47:0/1433089527 >> 172.21.15.47:0/1433089527 conn(0x7f86a8076ab0 msgr2=0x7f86a8078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.247+0000 7f86af000640 1 -- 172.21.15.47:0/1433089527 shutdown_connections 2024-03-21T11:20:39.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.247+0000 7f86af000640 1 -- 172.21.15.47:0/1433089527 wait complete. 2024-03-21T11:20:39.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.248+0000 7f86af000640 1 Processor -- start 2024-03-21T11:20:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.248+0000 7f86af000640 1 -- start start 2024-03-21T11:20:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.248+0000 7f86af000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8107b00 0x7f86a819dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.248+0000 7f86af000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a8108cf0 0x7f86a819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.248+0000 7f86af000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86a81096b0 0x7f86a81a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.248+0000 7f86af000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8113760 con 0x7f86a8107b00 2024-03-21T11:20:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.248+0000 7f86af000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a81135e0 con 0x7f86a8108cf0 2024-03-21T11:20:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.248+0000 7f86af000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a81138e0 con 0x7f86a81096b0 2024-03-21T11:20:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.249+0000 7f86adc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8107b00 0x7f86a819dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.249+0000 7f86adc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8107b00 0x7f86a819dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35202/0 (socket says 172.21.15.47:35202) 2024-03-21T11:20:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.249+0000 7f86adc00640 1 -- 172.21.15.47:0/584012755 learned_addr learned my addr 172.21.15.47:0/584012755 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.249+0000 7f86ae600640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86a81096b0 0x7f86a81a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:39.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.249+0000 7f86ad200640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a8108cf0 0x7f86a819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:39.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.249+0000 7f86ae600640 1 -- 172.21.15.47:0/584012755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a8108cf0 msgr2=0x7f86a819e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:39.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.249+0000 7f86ae600640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a8108cf0 0x7f86a819e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:39.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.249+0000 7f86ae600640 1 -- 172.21.15.47:0/584012755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8107b00 msgr2=0x7f86a819dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:39.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.249+0000 7f86ae600640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8107b00 0x7f86a819dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:39.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.249+0000 7f86ae600640 1 -- 172.21.15.47:0/584012755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a81a5970 con 0x7f86a81096b0 2024-03-21T11:20:39.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.249+0000 7f86adc00640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8107b00 0x7f86a819dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:20:39.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.249+0000 7f86ae600640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86a81096b0 0x7f86a81a5300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f86a0037040 tx=0x7f86a0032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:39.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.250+0000 7f86a6000640 1 -- 172.21.15.47:0/584012755 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a0041020 con 0x7f86a81096b0 2024-03-21T11:20:39.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.250+0000 7f86af000640 1 -- 172.21.15.47:0/584012755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a81a5c00 con 0x7f86a81096b0 2024-03-21T11:20:39.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.250+0000 7f86a6000640 1 -- 172.21.15.47:0/584012755 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86a0034cd0 con 0x7f86a81096b0 2024-03-21T11:20:39.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.250+0000 7f86a6000640 1 -- 172.21.15.47:0/584012755 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a0039040 con 0x7f86a81096b0 2024-03-21T11:20:39.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.250+0000 7f86af000640 1 -- 172.21.15.47:0/584012755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a81a60c0 con 0x7f86a81096b0 2024-03-21T11:20:39.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.251+0000 7f86af000640 1 -- 172.21.15.47:0/584012755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a810edc0 con 0x7f86a81096b0 2024-03-21T11:20:39.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.253+0000 7f86a6000640 1 -- 172.21.15.47:0/584012755 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f86a0044020 con 0x7f86a81096b0 2024-03-21T11:20:39.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.254+0000 7f86a6000640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86900a2f20 0x7f86900a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:39.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.254+0000 7f86a6000640 1 -- 172.21.15.47:0/584012755 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f86a00f0d70 con 0x7f86a81096b0 2024-03-21T11:20:39.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.254+0000 7f86adc00640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86900a2f20 0x7f86900a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:39.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.255+0000 7f86adc00640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86900a2f20 0x7f86900a53e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f86980095f0 tx=0x7f8698005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:39.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.259+0000 7f86a6000640 1 -- 172.21.15.47:0/584012755 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f86a004b020 con 0x7f86a81096b0 2024-03-21T11:20:39.325 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:20:39 smithi047 podman[70514]: 2024-03-21 11:20:38.97045014 +0000 UTC m=+0.039473464 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:20:39.326 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:20:39 smithi047 podman[70514]: 2024-03-21 11:20:39.070060564 +0000 UTC m=+0.139083894 container create 0463adc15a5cfe7fb9e31c6e21d1a8f8cd73f07f39da46c0fc1e3d8f851dee1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-21T11:20:39.326 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:20:39 smithi047 podman[70514]: 2024-03-21 11:20:39.324635832 +0000 UTC m=+0.393659200 container init 0463adc15a5cfe7fb9e31c6e21d1a8f8cd73f07f39da46c0fc1e3d8f851dee1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-21T11:20:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.518+0000 7f86af000640 1 -- 172.21.15.47:0/584012755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86a8062340 con 0x7f86a81096b0 2024-03-21T11:20:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.519+0000 7f86a6000640 1 -- 172.21.15.47:0/584012755 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8465 (secure 0 0 0) 0x7f86a00be030 con 0x7f86a81096b0 2024-03-21T11:20:39.520 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:39.520 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-21T11:20:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2024-03-21T11:20:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.521+0000 7f86af000640 1 -- 172.21.15.47:0/584012755 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86900a2f20 msgr2=0x7f86900a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.522+0000 7f86af000640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86900a2f20 0x7f86900a53e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f86980095f0 tx=0x7f8698005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.522+0000 7f86af000640 1 -- 172.21.15.47:0/584012755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86a81096b0 msgr2=0x7f86a81a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.522+0000 7f86af000640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86a81096b0 0x7f86a81a5300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f86a0037040 tx=0x7f86a0032ea0 comp rx=0 tx=0).stop 2024-03-21T11:20:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.522+0000 7f86af000640 1 -- 172.21.15.47:0/584012755 shutdown_connections 2024-03-21T11:20:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.522+0000 7f86af000640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86900a2f20 0x7f86900a53e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.522+0000 7f86af000640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86a81096b0 0x7f86a81a5300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.522+0000 7f86af000640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a8108cf0 0x7f86a819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.522+0000 7f86af000640 1 --2- 172.21.15.47:0/584012755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8107b00 0x7f86a819dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.522+0000 7f86af000640 1 -- 172.21.15.47:0/584012755 >> 172.21.15.47:0/584012755 conn(0x7f86a8076ab0 msgr2=0x7f86a810bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.522+0000 7f86af000640 1 -- 172.21.15.47:0/584012755 shutdown_connections 2024-03-21T11:20:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:39.522+0000 7f86af000640 1 -- 172.21.15.47:0/584012755 wait complete. 2024-03-21T11:20:39.815 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:20:39 smithi047 podman[70514]: 2024-03-21 11:20:39.336735889 +0000 UTC m=+0.405759233 container start 0463adc15a5cfe7fb9e31c6e21d1a8f8cd73f07f39da46c0fc1e3d8f851dee1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux ) 2024-03-21T11:20:39.815 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:20:39 smithi047 ceph-mds[70572]: -- 172.21.15.47:0/3698647934 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563ef7efda40 con 0x563ef7ed0c00 2024-03-21T11:20:39.815 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:20:39 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-a[70549]: starting mds.a at 2024-03-21T11:20:39.815 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:20:39 smithi047 bash[70514]: 0463adc15a5cfe7fb9e31c6e21d1a8f8cd73f07f39da46c0fc1e3d8f851dee1e 2024-03-21T11:20:39.815 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:20:39 smithi047 systemd[1]: Started Ceph mds.a for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:20:40.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:40 smithi102 ceph-mon[36748]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:40.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:40 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:40.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:40 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:40.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:40 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:40.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:40 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/584012755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:40.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:40 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:40.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:40 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:40.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:40 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:40.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:40 smithi102 ceph-mon[36748]: Deploying daemon mds.b on smithi100 2024-03-21T11:20:40.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:40 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.47:6840/1742084729,v1:172.21.15.47:6841/1742084729] up:boot 2024-03-21T11:20:40.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:40 smithi102 ceph-mon[36748]: fsmap 11 up:standby 2024-03-21T11:20:40.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:40 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T11:20:40.764 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:40 smithi100 ceph-mon[36245]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:40.764 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:40 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:40.764 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:40 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:40.764 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:40 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:40.764 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:40 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/584012755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:40.764 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:40 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:40.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:40 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:40.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:40 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:40.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:40 smithi100 ceph-mon[36245]: Deploying daemon mds.b on smithi100 2024-03-21T11:20:40.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:40 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.47:6840/1742084729,v1:172.21.15.47:6841/1742084729] up:boot 2024-03-21T11:20:40.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:40 smithi100 ceph-mon[36245]: fsmap 11 up:standby 2024-03-21T11:20:40.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:40 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T11:20:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:40 smithi047 ceph-mon[31351]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:40 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:40 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:40 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:40.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:40 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/584012755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:40.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:40 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T11:20:40.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:40 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T11:20:40.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:40 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:40.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:40 smithi047 ceph-mon[31351]: Deploying daemon mds.b on smithi100 2024-03-21T11:20:40.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:40 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.47:6840/1742084729,v1:172.21.15.47:6841/1742084729] up:boot 2024-03-21T11:20:40.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:40 smithi047 ceph-mon[31351]: fsmap 11 up:standby 2024-03-21T11:20:40.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:40 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T11:20:41.543 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.767+0000 7ff033200640 1 -- 172.21.15.47:0/153532716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c1026d0 msgr2=0x7ff02c102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.767+0000 7ff033200640 1 --2- 172.21.15.47:0/153532716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c1026d0 0x7ff02c102b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009900 tx=0x7ff01c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:41.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.767+0000 7ff033200640 1 -- 172.21.15.47:0/153532716 shutdown_connections 2024-03-21T11:20:41.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.767+0000 7ff033200640 1 --2- 172.21.15.47:0/153532716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff02c103090 0x7ff02c109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:41.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.767+0000 7ff033200640 1 --2- 172.21.15.47:0/153532716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c1026d0 0x7ff02c102b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:41.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.767+0000 7ff033200640 1 --2- 172.21.15.47:0/153532716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff02c1014e0 0x7ff02c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:41.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.767+0000 7ff033200640 1 -- 172.21.15.47:0/153532716 >> 172.21.15.47:0/153532716 conn(0x7ff02c0fca30 msgr2=0x7ff02c0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:41.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.767+0000 7ff033200640 1 -- 172.21.15.47:0/153532716 shutdown_connections 2024-03-21T11:20:41.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.767+0000 7ff033200640 1 -- 172.21.15.47:0/153532716 wait complete. 2024-03-21T11:20:41.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.768+0000 7ff033200640 1 Processor -- start 2024-03-21T11:20:41.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.768+0000 7ff033200640 1 -- start start 2024-03-21T11:20:41.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.768+0000 7ff033200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff02c1014e0 0x7ff02c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:41.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.768+0000 7ff033200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c1026d0 0x7ff02c197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:41.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.768+0000 7ff033200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff02c103090 0x7ff02c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:41.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.768+0000 7ff033200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c10d1c0 con 0x7ff02c103090 2024-03-21T11:20:41.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.768+0000 7ff033200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c10d040 con 0x7ff02c1026d0 2024-03-21T11:20:41.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.768+0000 7ff033200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c10d340 con 0x7ff02c1014e0 2024-03-21T11:20:41.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.769+0000 7ff031400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c1026d0 0x7ff02c197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:41.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.769+0000 7ff031400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c1026d0 0x7ff02c197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38228/0 (socket says 172.21.15.47:38228) 2024-03-21T11:20:41.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.769+0000 7ff031400640 1 -- 172.21.15.47:0/1733076436 learned_addr learned my addr 172.21.15.47:0/1733076436 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:41.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.769+0000 7ff031e00640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff02c1014e0 0x7ff02c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:41.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.769+0000 7ff032800640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff02c103090 0x7ff02c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:41.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.769+0000 7ff031400640 1 -- 172.21.15.47:0/1733076436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff02c1014e0 msgr2=0x7ff02c1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.769+0000 7ff031400640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff02c1014e0 0x7ff02c1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.769+0000 7ff031400640 1 -- 172.21.15.47:0/1733076436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff02c103090 msgr2=0x7ff02c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.769+0000 7ff031400640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff02c103090 0x7ff02c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.769+0000 7ff031400640 1 -- 172.21.15.47:0/1733076436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c19f410 con 0x7ff02c1026d0 2024-03-21T11:20:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.769+0000 7ff031e00640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff02c1014e0 0x7ff02c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:20:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.770+0000 7ff031400640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c1026d0 0x7ff02c197d30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff01c000c00 tx=0x7ff01c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.770+0000 7ff026a00640 1 -- 172.21.15.47:0/1733076436 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c043020 con 0x7ff02c1026d0 2024-03-21T11:20:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.770+0000 7ff026a00640 1 -- 172.21.15.47:0/1733076436 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff01c0054e0 con 0x7ff02c1026d0 2024-03-21T11:20:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.770+0000 7ff026a00640 1 -- 172.21.15.47:0/1733076436 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c003c40 con 0x7ff02c1026d0 2024-03-21T11:20:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.770+0000 7ff033200640 1 -- 172.21.15.47:0/1733076436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02c19f6a0 con 0x7ff02c1026d0 2024-03-21T11:20:41.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.770+0000 7ff033200640 1 -- 172.21.15.47:0/1733076436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02c19fba0 con 0x7ff02c1026d0 2024-03-21T11:20:41.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.771+0000 7ff033200640 1 -- 172.21.15.47:0/1733076436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02c108800 con 0x7ff02c1026d0 2024-03-21T11:20:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.773+0000 7ff026a00640 1 -- 172.21.15.47:0/1733076436 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff01c046020 con 0x7ff02c1026d0 2024-03-21T11:20:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.775+0000 7ff026a00640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff0140a2fa0 0x7ff0140a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.775+0000 7ff026a00640 1 -- 172.21.15.47:0/1733076436 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff01c0f12b0 con 0x7ff02c1026d0 2024-03-21T11:20:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.775+0000 7ff031e00640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff0140a2fa0 0x7ff0140a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.775+0000 7ff031e00640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff0140a2fa0 0x7ff0140a5460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff02c102530 tx=0x7ff020005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:41.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:41.777+0000 7ff026a00640 1 -- 172.21.15.47:0/1733076436 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff01c04d020 con 0x7ff02c1026d0 2024-03-21T11:20:41.893 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:20:41 smithi100 systemd[1]: Starting Ceph mds.b for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:20:42.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:42.020+0000 7ff033200640 1 -- 172.21.15.47:0/1733076436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff02c02d2c0 con 0x7ff02c1026d0 2024-03-21T11:20:42.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:42.021+0000 7ff026a00640 1 -- 172.21.15.47:0/1733076436 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9257 (secure 0 0 0) 0x7ff01c0be4e0 con 0x7ff02c1026d0 2024-03-21T11:20:42.022 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:42.022 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-21T11:20:42.022 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2024-03-21T11:20:42.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:42.024+0000 7ff033200640 1 -- 172.21.15.47:0/1733076436 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff0140a2fa0 msgr2=0x7ff0140a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:42.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:42.024+0000 7ff033200640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff0140a2fa0 0x7ff0140a5460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff02c102530 tx=0x7ff020005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:42.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:42.024+0000 7ff033200640 1 -- 172.21.15.47:0/1733076436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c1026d0 msgr2=0x7ff02c197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:42.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:42.024+0000 7ff033200640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c1026d0 0x7ff02c197d30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff01c000c00 tx=0x7ff01c02c650 comp rx=0 tx=0).stop 2024-03-21T11:20:42.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:42.025+0000 7ff033200640 1 -- 172.21.15.47:0/1733076436 shutdown_connections 2024-03-21T11:20:42.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:42.025+0000 7ff033200640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff0140a2fa0 0x7ff0140a5460 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:42.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:42.025+0000 7ff033200640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff02c103090 0x7ff02c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:42.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:42.025+0000 7ff033200640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c1026d0 0x7ff02c197d30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:42.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:42.025+0000 7ff033200640 1 --2- 172.21.15.47:0/1733076436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff02c1014e0 0x7ff02c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:42.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:42.025+0000 7ff033200640 1 -- 172.21.15.47:0/1733076436 >> 172.21.15.47:0/1733076436 conn(0x7ff02c0fca30 msgr2=0x7ff02c1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:42.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:42.025+0000 7ff033200640 1 -- 172.21.15.47:0/1733076436 shutdown_connections 2024-03-21T11:20:42.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:42.025+0000 7ff033200640 1 -- 172.21.15.47:0/1733076436 wait complete. 2024-03-21T11:20:42.188 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:20:41 smithi100 podman[66309]: 2024-03-21 11:20:41.832397178 +0000 UTC m=+0.017218991 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:20:42.188 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:20:41 smithi100 podman[66309]: 2024-03-21 11:20:41.952373866 +0000 UTC m=+0.137195655 container create 3fa90a15426b691dcf39a2f16a6fce372d8eaa4085f74f5ea55d99b95aca0a5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T11:20:42.499 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:20:42 smithi100 podman[66309]: 2024-03-21 11:20:42.18886225 +0000 UTC m=+0.373684062 container init 3fa90a15426b691dcf39a2f16a6fce372d8eaa4085f74f5ea55d99b95aca0a5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T11:20:42.499 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:20:42 smithi100 podman[66309]: 2024-03-21 11:20:42.19222382 +0000 UTC m=+0.377045622 container start 3fa90a15426b691dcf39a2f16a6fce372d8eaa4085f74f5ea55d99b95aca0a5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-21T11:20:42.500 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:20:42 smithi100 ceph-mds[66328]: -- 172.21.15.100:0/473043240 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55655e359a40 con 0x55655e32cc00 2024-03-21T11:20:42.500 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:20:42 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b[66324]: starting mds.b at 2024-03-21T11:20:42.500 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:20:42 smithi100 bash[66309]: 3fa90a15426b691dcf39a2f16a6fce372d8eaa4085f74f5ea55d99b95aca0a5b 2024-03-21T11:20:42.500 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:20:42 smithi100 systemd[1]: Started Ceph mds.b for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:20:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:42 smithi102 ceph-mon[36748]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:42 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1733076436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:42 smithi047 ceph-mon[31351]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:42 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1733076436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:42.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:42 smithi100 ceph-mon[36245]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:42 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1733076436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:43.793 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:43 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] up:boot 2024-03-21T11:20:43.793 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:43 smithi102 ceph-mon[36748]: fsmap 12 up:standby 2024-03-21T11:20:43.793 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:43 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T11:20:43.793 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:43 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:20:43.808 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:43 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] up:boot 2024-03-21T11:20:43.808 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:43 smithi100 ceph-mon[36245]: fsmap 12 up:standby 2024-03-21T11:20:43.808 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:43 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T11:20:43.808 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:43 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:20:43.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:43 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] up:boot 2024-03-21T11:20:43.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:43 smithi047 ceph-mon[31351]: fsmap 12 up:standby 2024-03-21T11:20:43.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:43 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T11:20:43.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:43 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:20:44.045 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:44.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.290+0000 7f9538e00640 1 -- 172.21.15.47:0/383441640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9534104a90 msgr2=0x7f9534106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:44.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.290+0000 7f9538e00640 1 --2- 172.21.15.47:0/383441640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9534104a90 0x7f9534106e80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f951c009900 tx=0x7f951c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:44.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.290+0000 7f9538e00640 1 -- 172.21.15.47:0/383441640 shutdown_connections 2024-03-21T11:20:44.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.290+0000 7f9538e00640 1 --2- 172.21.15.47:0/383441640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95341073c0 0x7f9534109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.290+0000 7f9538e00640 1 --2- 172.21.15.47:0/383441640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9534104a90 0x7f9534106e80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.290+0000 7f9538e00640 1 --2- 172.21.15.47:0/383441640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9534102160 0x7f9534104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.290+0000 7f9538e00640 1 -- 172.21.15.47:0/383441640 >> 172.21.15.47:0/383441640 conn(0x7f95340fbb50 msgr2=0x7f95340fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:44.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.290+0000 7f9538e00640 1 -- 172.21.15.47:0/383441640 shutdown_connections 2024-03-21T11:20:44.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.291+0000 7f9538e00640 1 -- 172.21.15.47:0/383441640 wait complete. 2024-03-21T11:20:44.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.292+0000 7f9538e00640 1 Processor -- start 2024-03-21T11:20:44.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.292+0000 7f9538e00640 1 -- start start 2024-03-21T11:20:44.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.292+0000 7f9538e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9534102160 0x7f9534199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.292+0000 7f9538e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9534104a90 0x7f953419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9538e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95341073c0 0x7f95341a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9538e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953410d3a0 con 0x7f95341073c0 2024-03-21T11:20:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9538e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953410d220 con 0x7f9534104a90 2024-03-21T11:20:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9538e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953410d520 con 0x7f9534102160 2024-03-21T11:20:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9531400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9534104a90 0x7f953419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9531e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9534102160 0x7f9534199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9531400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9534104a90 0x7f953419a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38256/0 (socket says 172.21.15.47:38256) 2024-03-21T11:20:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9531400640 1 -- 172.21.15.47:0/3691147054 learned_addr learned my addr 172.21.15.47:0/3691147054 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9532800640 1 --2- 172.21.15.47:0/3691147054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95341073c0 0x7f95341a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9531400640 1 -- 172.21.15.47:0/3691147054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9534102160 msgr2=0x7f9534199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9531400640 1 --2- 172.21.15.47:0/3691147054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9534102160 0x7f9534199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9531400640 1 -- 172.21.15.47:0/3691147054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95341073c0 msgr2=0x7f95341a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:44.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9531400640 1 --2- 172.21.15.47:0/3691147054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95341073c0 0x7f95341a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9531400640 1 -- 172.21.15.47:0/3691147054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95341a1860 con 0x7f9534104a90 2024-03-21T11:20:44.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.293+0000 7f9532800640 1 --2- 172.21.15.47:0/3691147054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95341073c0 0x7f95341a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:20:44.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.294+0000 7f9531400640 1 --2- 172.21.15.47:0/3691147054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9534104a90 0x7f953419a190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f951c000c00 tx=0x7f951c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:44.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.294+0000 7f952aa00640 1 -- 172.21.15.47:0/3691147054 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c043020 con 0x7f9534104a90 2024-03-21T11:20:44.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.294+0000 7f952aa00640 1 -- 172.21.15.47:0/3691147054 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f951c0054e0 con 0x7f9534104a90 2024-03-21T11:20:44.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.294+0000 7f9538e00640 1 -- 172.21.15.47:0/3691147054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95341a1af0 con 0x7f9534104a90 2024-03-21T11:20:44.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.294+0000 7f952aa00640 1 -- 172.21.15.47:0/3691147054 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c003c40 con 0x7f9534104a90 2024-03-21T11:20:44.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.294+0000 7f9538e00640 1 -- 172.21.15.47:0/3691147054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95341a2040 con 0x7f9534104a90 2024-03-21T11:20:44.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.295+0000 7f9538e00640 1 -- 172.21.15.47:0/3691147054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9534108f10 con 0x7f9534104a90 2024-03-21T11:20:44.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.297+0000 7f952aa00640 1 -- 172.21.15.47:0/3691147054 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f951c046020 con 0x7f9534104a90 2024-03-21T11:20:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.298+0000 7f952aa00640 1 --2- 172.21.15.47:0/3691147054 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95180a2f20 0x7f95180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.299+0000 7f952aa00640 1 -- 172.21.15.47:0/3691147054 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f951c0f1110 con 0x7f9534104a90 2024-03-21T11:20:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.299+0000 7f9531e00640 1 --2- 172.21.15.47:0/3691147054 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95180a2f20 0x7f95180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:44.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.299+0000 7f9531e00640 1 --2- 172.21.15.47:0/3691147054 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95180a2f20 0x7f95180a53e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f953419b120 tx=0x7f9524009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.302+0000 7f952aa00640 1 -- 172.21.15.47:0/3691147054 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f951c04d020 con 0x7f9534104a90 2024-03-21T11:20:44.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.572+0000 7f9538e00640 1 -- 172.21.15.47:0/3691147054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f953402d070 con 0x7f9534104a90 2024-03-21T11:20:44.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.574+0000 7f952aa00640 1 -- 172.21.15.47:0/3691147054 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10052 (secure 0 0 0) 0x7f951c0be340 con 0x7f9534104a90 2024-03-21T11:20:44.574 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:44.575 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-21T11:20:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 13 2024-03-21T11:20:44.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.576+0000 7f9538e00640 1 -- 172.21.15.47:0/3691147054 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95180a2f20 msgr2=0x7f95180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:44.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.576+0000 7f9538e00640 1 --2- 172.21.15.47:0/3691147054 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95180a2f20 0x7f95180a53e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f953419b120 tx=0x7f9524009500 comp rx=0 tx=0).stop 2024-03-21T11:20:44.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.576+0000 7f9538e00640 1 -- 172.21.15.47:0/3691147054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9534104a90 msgr2=0x7f953419a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:44.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.576+0000 7f9538e00640 1 --2- 172.21.15.47:0/3691147054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9534104a90 0x7f953419a190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f951c000c00 tx=0x7f951c02c650 comp rx=0 tx=0).stop 2024-03-21T11:20:44.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.577+0000 7f9538e00640 1 -- 172.21.15.47:0/3691147054 shutdown_connections 2024-03-21T11:20:44.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.577+0000 7f9538e00640 1 --2- 172.21.15.47:0/3691147054 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95180a2f20 0x7f95180a53e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.577+0000 7f9538e00640 1 --2- 172.21.15.47:0/3691147054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95341073c0 0x7f95341a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.577+0000 7f9538e00640 1 --2- 172.21.15.47:0/3691147054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9534104a90 0x7f953419a190 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.577+0000 7f9538e00640 1 --2- 172.21.15.47:0/3691147054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9534102160 0x7f9534199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.577+0000 7f9538e00640 1 -- 172.21.15.47:0/3691147054 >> 172.21.15.47:0/3691147054 conn(0x7f95340fbb50 msgr2=0x7f9534101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:44.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.577+0000 7f9538e00640 1 -- 172.21.15.47:0/3691147054 shutdown_connections 2024-03-21T11:20:44.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.577+0000 7f9538e00640 1 -- 172.21.15.47:0/3691147054 wait complete. 2024-03-21T11:20:44.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:20:44.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:44 smithi047 ceph-mon[31351]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:44.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:44 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:44.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:44 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:44.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:44 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:44.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:44 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:44.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:44 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.830+0000 7f254b200640 1 -- 172.21.15.47:0/3620367403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f254c1030c0 msgr2=0x7f254c109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.830+0000 7f254b200640 1 --2- 172.21.15.47:0/3620367403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f254c1030c0 0x7f254c109950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2544009be0 tx=0x7f254402bef0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.830+0000 7f254b200640 1 -- 172.21.15.47:0/3620367403 shutdown_connections 2024-03-21T11:20:44.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.830+0000 7f254b200640 1 --2- 172.21.15.47:0/3620367403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f254c1030c0 0x7f254c109950 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.830+0000 7f254b200640 1 --2- 172.21.15.47:0/3620367403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102700 0x7f254c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.830+0000 7f254b200640 1 --2- 172.21.15.47:0/3620367403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254c101510 0x7f254c101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.830+0000 7f254b200640 1 -- 172.21.15.47:0/3620367403 >> 172.21.15.47:0/3620367403 conn(0x7f254c0fca80 msgr2=0x7f254c0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:44.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.831+0000 7f254b200640 1 -- 172.21.15.47:0/3620367403 shutdown_connections 2024-03-21T11:20:44.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.831+0000 7f254b200640 1 -- 172.21.15.47:0/3620367403 wait complete. 2024-03-21T11:20:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.832+0000 7f254b200640 1 Processor -- start 2024-03-21T11:20:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.832+0000 7f254b200640 1 -- start start 2024-03-21T11:20:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.832+0000 7f254b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254c101510 0x7f254c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.832+0000 7f254b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102700 0x7f254c199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.832+0000 7f254b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f254c1030c0 0x7f254c1a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.832+0000 7f254b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c10d1c0 con 0x7f254c101510 2024-03-21T11:20:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.832+0000 7f254b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c10d040 con 0x7f254c102700 2024-03-21T11:20:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.832+0000 7f254b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c10d340 con 0x7f254c1030c0 2024-03-21T11:20:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.832+0000 7f2549e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254c101510 0x7f254c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.832+0000 7f2549400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102700 0x7f254c199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.832+0000 7f2549e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254c101510 0x7f254c199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35272/0 (socket says 172.21.15.47:35272) 2024-03-21T11:20:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.832+0000 7f2549e00640 1 -- 172.21.15.47:0/687565589 learned_addr learned my addr 172.21.15.47:0/687565589 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.832+0000 7f254a800640 1 --2- 172.21.15.47:0/687565589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f254c1030c0 0x7f254c1a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.833+0000 7f2549400640 1 -- 172.21.15.47:0/687565589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f254c1030c0 msgr2=0x7f254c1a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.833+0000 7f2549400640 1 --2- 172.21.15.47:0/687565589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f254c1030c0 0x7f254c1a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.833+0000 7f2549400640 1 -- 172.21.15.47:0/687565589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254c101510 msgr2=0x7f254c199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.833+0000 7f2549400640 1 --2- 172.21.15.47:0/687565589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254c101510 0x7f254c199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.833+0000 7f2549400640 1 -- 172.21.15.47:0/687565589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c1a15b0 con 0x7f254c102700 2024-03-21T11:20:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.833+0000 7f2549400640 1 --2- 172.21.15.47:0/687565589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102700 0x7f254c199ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f253800e9f0 tx=0x7f253800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.833+0000 7f2542a00640 1 -- 172.21.15.47:0/687565589 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253800ccc0 con 0x7f254c102700 2024-03-21T11:20:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.833+0000 7f2542a00640 1 -- 172.21.15.47:0/687565589 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2538010910 con 0x7f254c102700 2024-03-21T11:20:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.833+0000 7f2542a00640 1 -- 172.21.15.47:0/687565589 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538018960 con 0x7f254c102700 2024-03-21T11:20:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.833+0000 7f254b200640 1 -- 172.21.15.47:0/687565589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c1a18a0 con 0x7f254c102700 2024-03-21T11:20:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.833+0000 7f254b200640 1 -- 172.21.15.47:0/687565589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c1a1e40 con 0x7f254c102700 2024-03-21T11:20:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.835+0000 7f254b200640 1 -- 172.21.15.47:0/687565589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254c1a2070 con 0x7f254c102700 2024-03-21T11:20:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.837+0000 7f2542a00640 1 -- 172.21.15.47:0/687565589 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2538010ac0 con 0x7f254c102700 2024-03-21T11:20:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.838+0000 7f2542a00640 1 --2- 172.21.15.47:0/687565589 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f25300a2fa0 0x7f25300a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.838+0000 7f2542a00640 1 -- 172.21.15.47:0/687565589 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f25380cf2c0 con 0x7f254c102700 2024-03-21T11:20:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.839+0000 7f2549e00640 1 --2- 172.21.15.47:0/687565589 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f25300a2fa0 0x7f25300a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.839+0000 7f2549e00640 1 --2- 172.21.15.47:0/687565589 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f25300a2fa0 0x7f25300a5460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f253c0096c0 tx=0x7f253c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:44.842+0000 7f2542a00640 1 -- 172.21.15.47:0/687565589 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f253809bea0 con 0x7f254c102700 2024-03-21T11:20:44.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:44 smithi100 ceph-mon[36245]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:44.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:44 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:44.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:44 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:44.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:44 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:44.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:44 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:44.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:44 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:44.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:44 smithi102 ceph-mon[36748]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:44.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:44 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:44.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:44 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:44.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:44 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:44.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:44 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:44.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:44 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:45.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.053+0000 7f254b200640 1 -- 172.21.15.47:0/687565589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f254c1a2240 con 0x7f254c102700 2024-03-21T11:20:45.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.055+0000 7f2542a00640 1 -- 172.21.15.47:0/687565589 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18455 (secure 0 0 0) 0x7f253809ff40 con 0x7f254c102700 2024-03-21T11:20:45.056 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:45.056 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":82,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:19:50.716171+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:20:45.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.058+0000 7f254b200640 1 -- 172.21.15.47:0/687565589 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f25300a2fa0 msgr2=0x7f25300a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:45.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.058+0000 7f254b200640 1 --2- 172.21.15.47:0/687565589 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f25300a2fa0 0x7f25300a5460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f253c0096c0 tx=0x7f253c009260 comp rx=0 tx=0).stop 2024-03-21T11:20:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.058+0000 7f254b200640 1 -- 172.21.15.47:0/687565589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102700 msgr2=0x7f254c199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.058+0000 7f254b200640 1 --2- 172.21.15.47:0/687565589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102700 0x7f254c199ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f253800e9f0 tx=0x7f253800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.058+0000 7f254b200640 1 -- 172.21.15.47:0/687565589 shutdown_connections 2024-03-21T11:20:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.058+0000 7f254b200640 1 --2- 172.21.15.47:0/687565589 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f25300a2fa0 0x7f25300a5460 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.058+0000 7f254b200640 1 --2- 172.21.15.47:0/687565589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f254c1030c0 0x7f254c1a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.058+0000 7f254b200640 1 --2- 172.21.15.47:0/687565589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f254c102700 0x7f254c199ed0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.058+0000 7f254b200640 1 --2- 172.21.15.47:0/687565589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f254c101510 0x7f254c199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.058+0000 7f254b200640 1 -- 172.21.15.47:0/687565589 >> 172.21.15.47:0/687565589 conn(0x7f254c0fca80 msgr2=0x7f254c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.058+0000 7f254b200640 1 -- 172.21.15.47:0/687565589 shutdown_connections 2024-03-21T11:20:45.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.058+0000 7f254b200640 1 -- 172.21.15.47:0/687565589 wait complete. 2024-03-21T11:20:45.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T11:15:35.545245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T11:20:45.078 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T11:20:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.272+0000 7fc0d2000640 1 -- 172.21.15.47:0/1781981517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0cc1028a0 msgr2=0x7fc0cc102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.272+0000 7fc0d2000640 1 --2- 172.21.15.47:0/1781981517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0cc1028a0 0x7fc0cc102d20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009860 tx=0x7fc0b802bb70 comp rx=0 tx=0).stop 2024-03-21T11:20:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.272+0000 7fc0d2000640 1 -- 172.21.15.47:0/1781981517 shutdown_connections 2024-03-21T11:20:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.272+0000 7fc0d2000640 1 --2- 172.21.15.47:0/1781981517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc103260 0x7fc0cc109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.272+0000 7fc0d2000640 1 --2- 172.21.15.47:0/1781981517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0cc1028a0 0x7fc0cc102d20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.272+0000 7fc0d2000640 1 --2- 172.21.15.47:0/1781981517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0cc1016b0 0x7fc0cc101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.272+0000 7fc0d2000640 1 -- 172.21.15.47:0/1781981517 >> 172.21.15.47:0/1781981517 conn(0x7fc0cc0fcc20 msgr2=0x7fc0cc0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.272+0000 7fc0d2000640 1 -- 172.21.15.47:0/1781981517 shutdown_connections 2024-03-21T11:20:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.272+0000 7fc0d2000640 1 -- 172.21.15.47:0/1781981517 wait complete. 2024-03-21T11:20:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.273+0000 7fc0d2000640 1 Processor -- start 2024-03-21T11:20:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.273+0000 7fc0d2000640 1 -- start start 2024-03-21T11:20:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.273+0000 7fc0d2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0cc1016b0 0x7fc0cc199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.273+0000 7fc0d2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0cc1028a0 0x7fc0cc19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.273+0000 7fc0d2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc103260 0x7fc0cc1a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.273+0000 7fc0d2000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc10d310 con 0x7fc0cc1028a0 2024-03-21T11:20:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.273+0000 7fc0d2000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc10d190 con 0x7fc0cc103260 2024-03-21T11:20:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.273+0000 7fc0d2000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc10d490 con 0x7fc0cc1016b0 2024-03-21T11:20:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.274+0000 7fc0caa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0cc1028a0 0x7fc0cc19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.274+0000 7fc0cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc103260 0x7fc0cc1a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.274+0000 7fc0cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc103260 0x7fc0cc1a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38284/0 (socket says 172.21.15.47:38284) 2024-03-21T11:20:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.274+0000 7fc0cbe00640 1 -- 172.21.15.47:0/2215264931 learned_addr learned my addr 172.21.15.47:0/2215264931 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.274+0000 7fc0cb400640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0cc1016b0 0x7fc0cc199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.274+0000 7fc0caa00640 1 -- 172.21.15.47:0/2215264931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0cc1016b0 msgr2=0x7fc0cc199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.274+0000 7fc0caa00640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0cc1016b0 0x7fc0cc199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.274+0000 7fc0caa00640 1 -- 172.21.15.47:0/2215264931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc103260 msgr2=0x7fc0cc1a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.274+0000 7fc0caa00640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc103260 0x7fc0cc1a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.274+0000 7fc0caa00640 1 -- 172.21.15.47:0/2215264931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0cc1a1890 con 0x7fc0cc1028a0 2024-03-21T11:20:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.274+0000 7fc0cbe00640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc103260 0x7fc0cc1a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:20:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.274+0000 7fc0cb400640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0cc1016b0 0x7fc0cc199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:20:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.275+0000 7fc0caa00640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0cc1028a0 0x7fc0cc19a1c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc0b802c0b0 tx=0x7fc0b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.275+0000 7fc0c3e00640 1 -- 172.21.15.47:0/2215264931 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8041020 con 0x7fc0cc1028a0 2024-03-21T11:20:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.275+0000 7fc0c3e00640 1 -- 172.21.15.47:0/2215264931 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0b80323b0 con 0x7fc0cc1028a0 2024-03-21T11:20:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.275+0000 7fc0d2000640 1 -- 172.21.15.47:0/2215264931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0cc1a1b20 con 0x7fc0cc1028a0 2024-03-21T11:20:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.275+0000 7fc0c3e00640 1 -- 172.21.15.47:0/2215264931 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8039040 con 0x7fc0cc1028a0 2024-03-21T11:20:45.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.275+0000 7fc0d2000640 1 -- 172.21.15.47:0/2215264931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0cc1a1fe0 con 0x7fc0cc1028a0 2024-03-21T11:20:45.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.276+0000 7fc0d2000640 1 -- 172.21.15.47:0/2215264931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0cc1a2270 con 0x7fc0cc1028a0 2024-03-21T11:20:45.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.279+0000 7fc0c3e00640 1 -- 172.21.15.47:0/2215264931 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc0b8044020 con 0x7fc0cc1028a0 2024-03-21T11:20:45.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.281+0000 7fc0c3e00640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc0b00a2f20 0x7fc0b00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:45.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.281+0000 7fc0c3e00640 1 -- 172.21.15.47:0/2215264931 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc0b80f0cf0 con 0x7fc0cc1028a0 2024-03-21T11:20:45.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.281+0000 7fc0cb400640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc0b00a2f20 0x7fc0b00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:45.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.282+0000 7fc0cb400640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc0b00a2f20 0x7fc0b00a53e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc19b150 tx=0x7fc0bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:45.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.285+0000 7fc0c3e00640 1 -- 172.21.15.47:0/2215264931 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc0b804b020 con 0x7fc0cc1028a0 2024-03-21T11:20:45.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.532+0000 7fc0d2000640 1 -- 172.21.15.47:0/2215264931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0cc1a2440 con 0x7fc0cc1028a0 2024-03-21T11:20:45.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.533+0000 7fc0c3e00640 1 -- 172.21.15.47:0/2215264931 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc0b80bdf20 con 0x7fc0cc1028a0 2024-03-21T11:20:45.534 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T11:20:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.536+0000 7fc0d2000640 1 -- 172.21.15.47:0/2215264931 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc0b00a2f20 msgr2=0x7fc0b00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.536+0000 7fc0d2000640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc0b00a2f20 0x7fc0b00a53e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc19b150 tx=0x7fc0bc009260 comp rx=0 tx=0).stop 2024-03-21T11:20:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.536+0000 7fc0d2000640 1 -- 172.21.15.47:0/2215264931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0cc1028a0 msgr2=0x7fc0cc19a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.536+0000 7fc0d2000640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0cc1028a0 0x7fc0cc19a1c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc0b802c0b0 tx=0x7fc0b80042f0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.536+0000 7fc0d2000640 1 -- 172.21.15.47:0/2215264931 shutdown_connections 2024-03-21T11:20:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.536+0000 7fc0d2000640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc0b00a2f20 0x7fc0b00a53e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.536+0000 7fc0d2000640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0cc103260 0x7fc0cc1a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.536+0000 7fc0d2000640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0cc1028a0 0x7fc0cc19a1c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.536+0000 7fc0d2000640 1 --2- 172.21.15.47:0/2215264931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0cc1016b0 0x7fc0cc199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.536+0000 7fc0d2000640 1 -- 172.21.15.47:0/2215264931 >> 172.21.15.47:0/2215264931 conn(0x7fc0cc0fcc20 msgr2=0x7fc0cc0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.536+0000 7fc0d2000640 1 -- 172.21.15.47:0/2215264931 shutdown_connections 2024-03-21T11:20:45.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.536+0000 7fc0d2000640 1 -- 172.21.15.47:0/2215264931 wait complete. 2024-03-21T11:20:45.556 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-21T11:20:45.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.768+0000 7fcb2d200640 1 -- 172.21.15.47:0/3491983039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb28100640 msgr2=0x7fcb28100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:45.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.768+0000 7fcb2d200640 1 --2- 172.21.15.47:0/3491983039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb28100640 0x7fcb28100a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009860 tx=0x7fcb1802bb70 comp rx=0 tx=0).stop 2024-03-21T11:20:45.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.769+0000 7fcb2d200640 1 -- 172.21.15.47:0/3491983039 shutdown_connections 2024-03-21T11:20:45.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.769+0000 7fcb2d200640 1 --2- 172.21.15.47:0/3491983039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb281021f0 0x7fcb28108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.769+0000 7fcb2d200640 1 --2- 172.21.15.47:0/3491983039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb28101830 0x7fcb28101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.769+0000 7fcb2d200640 1 --2- 172.21.15.47:0/3491983039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb28100640 0x7fcb28100a60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.769+0000 7fcb2d200640 1 -- 172.21.15.47:0/3491983039 >> 172.21.15.47:0/3491983039 conn(0x7fcb280fbbd0 msgr2=0x7fcb280fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:45.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.769+0000 7fcb2d200640 1 -- 172.21.15.47:0/3491983039 shutdown_connections 2024-03-21T11:20:45.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.769+0000 7fcb2d200640 1 -- 172.21.15.47:0/3491983039 wait complete. 2024-03-21T11:20:45.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.770+0000 7fcb2d200640 1 Processor -- start 2024-03-21T11:20:45.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.770+0000 7fcb2d200640 1 -- start start 2024-03-21T11:20:45.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.770+0000 7fcb2d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb28100640 0x7fcb28197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:45.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.770+0000 7fcb2d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb28101830 0x7fcb28197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:45.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.770+0000 7fcb2d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb281021f0 0x7fcb2819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:45.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.770+0000 7fcb2d200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2810c240 con 0x7fcb281021f0 2024-03-21T11:20:45.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.770+0000 7fcb2d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2810c0c0 con 0x7fcb28101830 2024-03-21T11:20:45.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.770+0000 7fcb2d200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2810c3c0 con 0x7fcb28100640 2024-03-21T11:20:45.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.771+0000 7fcb25e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb28100640 0x7fcb28197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:45.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.771+0000 7fcb25e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb28100640 0x7fcb28197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50632/0 (socket says 172.21.15.47:50632) 2024-03-21T11:20:45.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.771+0000 7fcb1fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb28101830 0x7fcb28197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:45.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.771+0000 7fcb26800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb281021f0 0x7fcb2819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:45.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.771+0000 7fcb25e00640 1 -- 172.21.15.47:0/2182638637 learned_addr learned my addr 172.21.15.47:0/2182638637 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.771+0000 7fcb25e00640 1 -- 172.21.15.47:0/2182638637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb28101830 msgr2=0x7fcb28197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.771+0000 7fcb25e00640 1 --2- 172.21.15.47:0/2182638637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb28101830 0x7fcb28197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.771+0000 7fcb25e00640 1 -- 172.21.15.47:0/2182638637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb281021f0 msgr2=0x7fcb2819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.771+0000 7fcb25e00640 1 --2- 172.21.15.47:0/2182638637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb281021f0 0x7fcb2819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.771+0000 7fcb25e00640 1 -- 172.21.15.47:0/2182638637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2819f6a0 con 0x7fcb28100640 2024-03-21T11:20:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.771+0000 7fcb25e00640 1 --2- 172.21.15.47:0/2182638637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb28100640 0x7fcb28197a80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb18037040 tx=0x7fcb18034760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.772+0000 7fcb1ea00640 1 -- 172.21.15.47:0/2182638637 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18039040 con 0x7fcb28100640 2024-03-21T11:20:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.772+0000 7fcb2d200640 1 -- 172.21.15.47:0/2182638637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2819f930 con 0x7fcb28100640 2024-03-21T11:20:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.772+0000 7fcb1ea00640 1 -- 172.21.15.47:0/2182638637 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb18042070 con 0x7fcb28100640 2024-03-21T11:20:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.772+0000 7fcb1ea00640 1 -- 172.21.15.47:0/2182638637 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18032c70 con 0x7fcb28100640 2024-03-21T11:20:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.772+0000 7fcb2d200640 1 -- 172.21.15.47:0/2182638637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2819fe70 con 0x7fcb28100640 2024-03-21T11:20:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.773+0000 7fcb2d200640 1 -- 172.21.15.47:0/2182638637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb281a0040 con 0x7fcb28100640 2024-03-21T11:20:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.775+0000 7fcb1ea00640 1 -- 172.21.15.47:0/2182638637 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcb1802c510 con 0x7fcb28100640 2024-03-21T11:20:45.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.776+0000 7fcb1ea00640 1 --2- 172.21.15.47:0/2182638637 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcb0c0a2f20 0x7fcb0c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:45.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.776+0000 7fcb1ea00640 1 -- 172.21.15.47:0/2182638637 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcb180f0730 con 0x7fcb28100640 2024-03-21T11:20:45.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.776+0000 7fcb1fe00640 1 --2- 172.21.15.47:0/2182638637 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcb0c0a2f20 0x7fcb0c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:45.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.777+0000 7fcb1fe00640 1 --2- 172.21.15.47:0/2182638637 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcb0c0a2f20 0x7fcb0c0a53e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcb28198f90 tx=0x7fcb10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:45.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.780+0000 7fcb1ea00640 1 -- 172.21.15.47:0/2182638637 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcb180bd960 con 0x7fcb28100640 2024-03-21T11:20:45.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:45 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3691147054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:45.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:45 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/687565589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:45.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:45 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3691147054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:45.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:45 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/687565589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:45.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:45 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3691147054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:45.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:45 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/687565589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.997+0000 7fcb2d200640 1 -- 172.21.15.47:0/2182638637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcb281a0210 con 0x7fcb28100640 2024-03-21T11:20:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:45.998+0000 7fcb1ea00640 1 -- 172.21.15.47:0/2182638637 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcb180bd260 con 0x7fcb28100640 2024-03-21T11:20:45.998 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:45.999 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-03-21T11:20:45.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:20:45.999 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-03-21T11:20:45.999 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-03-21T11:20:45.999 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:20:45.999 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-03-21T11:20:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.000+0000 7fcb2d200640 1 -- 172.21.15.47:0/2182638637 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcb0c0a2f20 msgr2=0x7fcb0c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.000+0000 7fcb2d200640 1 --2- 172.21.15.47:0/2182638637 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcb0c0a2f20 0x7fcb0c0a53e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcb28198f90 tx=0x7fcb10005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.000+0000 7fcb2d200640 1 -- 172.21.15.47:0/2182638637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb28100640 msgr2=0x7fcb28197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.000+0000 7fcb2d200640 1 --2- 172.21.15.47:0/2182638637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb28100640 0x7fcb28197a80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb18037040 tx=0x7fcb18034760 comp rx=0 tx=0).stop 2024-03-21T11:20:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.000+0000 7fcb2d200640 1 -- 172.21.15.47:0/2182638637 shutdown_connections 2024-03-21T11:20:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.000+0000 7fcb2d200640 1 --2- 172.21.15.47:0/2182638637 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcb0c0a2f20 0x7fcb0c0a53e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.000+0000 7fcb2d200640 1 --2- 172.21.15.47:0/2182638637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb281021f0 0x7fcb2819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.000+0000 7fcb2d200640 1 --2- 172.21.15.47:0/2182638637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb28101830 0x7fcb28197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.000+0000 7fcb2d200640 1 --2- 172.21.15.47:0/2182638637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb28100640 0x7fcb28197a80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.000+0000 7fcb2d200640 1 -- 172.21.15.47:0/2182638637 >> 172.21.15.47:0/2182638637 conn(0x7fcb280fbbd0 msgr2=0x7fcb280fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.001+0000 7fcb2d200640 1 -- 172.21.15.47:0/2182638637 shutdown_connections 2024-03-21T11:20:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.001+0000 7fcb2d200640 1 -- 172.21.15.47:0/2182638637 wait complete. 2024-03-21T11:20:46.021 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-21T11:20:46.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-21T11:20:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.221+0000 7f92eb200640 1 -- 172.21.15.47:0/764091270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92e40ffe70 msgr2=0x7f92e4100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.221+0000 7f92eb200640 1 --2- 172.21.15.47:0/764091270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92e40ffe70 0x7f92e4100290 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009900 tx=0x7f92d802bcd0 comp rx=0 tx=0).stop 2024-03-21T11:20:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.221+0000 7f92eb200640 1 -- 172.21.15.47:0/764091270 shutdown_connections 2024-03-21T11:20:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.221+0000 7f92eb200640 1 --2- 172.21.15.47:0/764091270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92e4101170 0x7f92e41015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.221+0000 7f92eb200640 1 --2- 172.21.15.47:0/764091270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92e41007d0 0x7f92e4100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.221+0000 7f92eb200640 1 --2- 172.21.15.47:0/764091270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92e40ffe70 0x7f92e4100290 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.221+0000 7f92eb200640 1 -- 172.21.15.47:0/764091270 >> 172.21.15.47:0/764091270 conn(0x7f92e40fba10 msgr2=0x7f92e40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.221+0000 7f92eb200640 1 -- 172.21.15.47:0/764091270 shutdown_connections 2024-03-21T11:20:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.221+0000 7f92eb200640 1 -- 172.21.15.47:0/764091270 wait complete. 2024-03-21T11:20:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.222+0000 7f92eb200640 1 Processor -- start 2024-03-21T11:20:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.222+0000 7f92eb200640 1 -- start start 2024-03-21T11:20:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.222+0000 7f92eb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92e40ffe70 0x7f92e4199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.222+0000 7f92eb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92e41007d0 0x7f92e419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92eb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92e4101170 0x7f92e41a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92eb200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e410f3d0 con 0x7f92e4101170 2024-03-21T11:20:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92eb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e410f250 con 0x7f92e40ffe70 2024-03-21T11:20:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92eb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e410f550 con 0x7f92e41007d0 2024-03-21T11:20:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92e40ffe70 0x7f92e4199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92e9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92e41007d0 0x7f92e419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92e40ffe70 0x7f92e4199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38328/0 (socket says 172.21.15.47:38328) 2024-03-21T11:20:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92e9e00640 1 -- 172.21.15.47:0/1500465955 learned_addr learned my addr 172.21.15.47:0/1500465955 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92ea800640 1 --2- 172.21.15.47:0/1500465955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92e4101170 0x7f92e41a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92e9400640 1 -- 172.21.15.47:0/1500465955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92e40ffe70 msgr2=0x7f92e4199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92e9400640 1 --2- 172.21.15.47:0/1500465955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92e40ffe70 0x7f92e4199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92e9400640 1 -- 172.21.15.47:0/1500465955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92e4101170 msgr2=0x7f92e41a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92e9400640 1 --2- 172.21.15.47:0/1500465955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92e4101170 0x7f92e41a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92e9400640 1 -- 172.21.15.47:0/1500465955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e41a16f0 con 0x7f92e41007d0 2024-03-21T11:20:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.223+0000 7f92e9400640 1 --2- 172.21.15.47:0/1500465955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92e41007d0 0x7f92e419a010 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f92d400e9f0 tx=0x7f92d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.224+0000 7f92dea00640 1 -- 172.21.15.47:0/1500465955 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d400ccc0 con 0x7f92e41007d0 2024-03-21T11:20:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.224+0000 7f92dea00640 1 -- 172.21.15.47:0/1500465955 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92d4010910 con 0x7f92e41007d0 2024-03-21T11:20:46.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.224+0000 7f92eb200640 1 -- 172.21.15.47:0/1500465955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e41a19e0 con 0x7f92e41007d0 2024-03-21T11:20:46.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.224+0000 7f92dea00640 1 -- 172.21.15.47:0/1500465955 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4018960 con 0x7f92e41007d0 2024-03-21T11:20:46.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.224+0000 7f92eb200640 1 -- 172.21.15.47:0/1500465955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e41a1f80 con 0x7f92e41007d0 2024-03-21T11:20:46.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.225+0000 7f92eb200640 1 -- 172.21.15.47:0/1500465955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e410abc0 con 0x7f92e41007d0 2024-03-21T11:20:46.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.228+0000 7f92dea00640 1 -- 172.21.15.47:0/1500465955 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f92d4010ac0 con 0x7f92e41007d0 2024-03-21T11:20:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.229+0000 7f92dea00640 1 --2- 172.21.15.47:0/1500465955 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92cc0a2fa0 0x7f92cc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.229+0000 7f92dea00640 1 -- 172.21.15.47:0/1500465955 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f92d40cf350 con 0x7f92e41007d0 2024-03-21T11:20:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.229+0000 7f92e9e00640 1 --2- 172.21.15.47:0/1500465955 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92cc0a2fa0 0x7f92cc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.230+0000 7f92e9e00640 1 --2- 172.21.15.47:0/1500465955 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92cc0a2fa0 0x7f92cc0a5460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f92d8006010 tx=0x7f92d802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:46.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.237+0000 7f92dea00640 1 -- 172.21.15.47:0/1500465955 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f92d409c580 con 0x7f92e41007d0 2024-03-21T11:20:46.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:46.444+0000 7f92eb200640 1 -- 172.21.15.47:0/1500465955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f92e402d070 con 0x7f92e41007d0 2024-03-21T11:20:47.047 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:47 smithi100 ceph-mon[36245]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:47.047 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:47 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2215264931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:20:47.047 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:47.047 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:47.047 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:47.047 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:20:47.048 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:47.048 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:20:47.048 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:47 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2182638637' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T11:20:47.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.073+0000 7f92dea00640 1 -- 172.21.15.47:0/1500465955 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f92d409be80 con 0x7f92e41007d0 2024-03-21T11:20:47.075 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-03-21T11:20:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.079+0000 7f92eb200640 1 -- 172.21.15.47:0/1500465955 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92cc0a2fa0 msgr2=0x7f92cc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.079+0000 7f92eb200640 1 --2- 172.21.15.47:0/1500465955 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92cc0a2fa0 0x7f92cc0a5460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f92d8006010 tx=0x7f92d802cf90 comp rx=0 tx=0).stop 2024-03-21T11:20:47.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.079+0000 7f92eb200640 1 -- 172.21.15.47:0/1500465955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92e41007d0 msgr2=0x7f92e419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:47.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.079+0000 7f92eb200640 1 --2- 172.21.15.47:0/1500465955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92e41007d0 0x7f92e419a010 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f92d400e9f0 tx=0x7f92d400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:20:47.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.080+0000 7f92eb200640 1 -- 172.21.15.47:0/1500465955 shutdown_connections 2024-03-21T11:20:47.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.080+0000 7f92eb200640 1 --2- 172.21.15.47:0/1500465955 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92cc0a2fa0 0x7f92cc0a5460 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:47.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.080+0000 7f92eb200640 1 --2- 172.21.15.47:0/1500465955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92e4101170 0x7f92e41a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:47.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.080+0000 7f92eb200640 1 --2- 172.21.15.47:0/1500465955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92e41007d0 0x7f92e419a010 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:47.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.080+0000 7f92eb200640 1 --2- 172.21.15.47:0/1500465955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92e40ffe70 0x7f92e4199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:47.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.080+0000 7f92eb200640 1 -- 172.21.15.47:0/1500465955 >> 172.21.15.47:0/1500465955 conn(0x7f92e40fba10 msgr2=0x7f92e40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:47.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.080+0000 7f92eb200640 1 -- 172.21.15.47:0/1500465955 shutdown_connections 2024-03-21T11:20:47.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.081+0000 7f92eb200640 1 -- 172.21.15.47:0/1500465955 wait complete. 2024-03-21T11:20:47.103 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-21T11:20:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:47 smithi102 ceph-mon[36748]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:47 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2215264931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:20:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:47.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:20:47.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:47.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:20:47.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:47 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2182638637' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T11:20:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:47 smithi047 ceph-mon[31351]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:47 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2215264931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:20:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:20:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:20:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:47 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2182638637' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T11:20:47.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.313+0000 7f8afb200640 1 -- 172.21.15.47:0/1308996580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8afc102000 msgr2=0x7f8afc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:47.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.313+0000 7f8afb200640 1 --2- 172.21.15.47:0/1308996580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8afc102000 0x7f8afc1043f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009900 tx=0x7f8aec02bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:47.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.314+0000 7f8afb200640 1 -- 172.21.15.47:0/1308996580 shutdown_connections 2024-03-21T11:20:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.314+0000 7f8afb200640 1 --2- 172.21.15.47:0/1308996580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8afc107260 0x7f8afc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.314+0000 7f8afb200640 1 --2- 172.21.15.47:0/1308996580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8afc104930 0x7f8afc106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.314+0000 7f8afb200640 1 --2- 172.21.15.47:0/1308996580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8afc102000 0x7f8afc1043f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.314+0000 7f8afb200640 1 -- 172.21.15.47:0/1308996580 >> 172.21.15.47:0/1308996580 conn(0x7f8afc0fba10 msgr2=0x7f8afc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.314+0000 7f8afb200640 1 -- 172.21.15.47:0/1308996580 shutdown_connections 2024-03-21T11:20:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.314+0000 7f8afb200640 1 -- 172.21.15.47:0/1308996580 wait complete. 2024-03-21T11:20:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.315+0000 7f8afb200640 1 Processor -- start 2024-03-21T11:20:47.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.315+0000 7f8afb200640 1 -- start start 2024-03-21T11:20:47.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.315+0000 7f8afb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8afc102000 0x7f8afc199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:47.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.315+0000 7f8afb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8afc104930 0x7f8afc199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:47.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.315+0000 7f8afb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8afc107260 0x7f8afc1a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:47.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.315+0000 7f8afb200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc10d210 con 0x7f8afc104930 2024-03-21T11:20:47.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.315+0000 7f8afb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc10d090 con 0x7f8afc102000 2024-03-21T11:20:47.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.315+0000 7f8afb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc10d390 con 0x7f8afc107260 2024-03-21T11:20:47.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.316+0000 7f8af9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8afc102000 0x7f8afc199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:47.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.316+0000 7f8af9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8afc102000 0x7f8afc199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38344/0 (socket says 172.21.15.47:38344) 2024-03-21T11:20:47.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.316+0000 7f8af9e00640 1 -- 172.21.15.47:0/106224542 learned_addr learned my addr 172.21.15.47:0/106224542 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:47.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.316+0000 7f8afa800640 1 --2- 172.21.15.47:0/106224542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8afc107260 0x7f8afc1a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:47.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.316+0000 7f8af9400640 1 --2- 172.21.15.47:0/106224542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8afc104930 0x7f8afc199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:47.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.316+0000 7f8af9e00640 1 -- 172.21.15.47:0/106224542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8afc107260 msgr2=0x7f8afc1a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:47.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.316+0000 7f8af9e00640 1 --2- 172.21.15.47:0/106224542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8afc107260 0x7f8afc1a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:47.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.316+0000 7f8af9e00640 1 -- 172.21.15.47:0/106224542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8afc104930 msgr2=0x7f8afc199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:47.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.316+0000 7f8af9e00640 1 --2- 172.21.15.47:0/106224542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8afc104930 0x7f8afc199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:47.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.316+0000 7f8af9e00640 1 -- 172.21.15.47:0/106224542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc1a16a0 con 0x7f8afc102000 2024-03-21T11:20:47.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.316+0000 7f8af9e00640 1 --2- 172.21.15.47:0/106224542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8afc102000 0x7f8afc199a80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009230 tx=0x7f8aec005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:47.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.317+0000 7f8af2a00640 1 -- 172.21.15.47:0/106224542 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec041070 con 0x7f8afc102000 2024-03-21T11:20:47.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.317+0000 7f8afb200640 1 -- 172.21.15.47:0/106224542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8afc1a1930 con 0x7f8afc102000 2024-03-21T11:20:47.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.317+0000 7f8af2a00640 1 -- 172.21.15.47:0/106224542 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8aec0044d0 con 0x7f8afc102000 2024-03-21T11:20:47.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.317+0000 7f8af2a00640 1 -- 172.21.15.47:0/106224542 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec034440 con 0x7f8afc102000 2024-03-21T11:20:47.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.317+0000 7f8afb200640 1 -- 172.21.15.47:0/106224542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc1a1e20 con 0x7f8afc102000 2024-03-21T11:20:47.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.318+0000 7f8afb200640 1 -- 172.21.15.47:0/106224542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8afc108db0 con 0x7f8afc102000 2024-03-21T11:20:47.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.320+0000 7f8af2a00640 1 -- 172.21.15.47:0/106224542 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8aec044070 con 0x7f8afc102000 2024-03-21T11:20:47.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.321+0000 7f8af2a00640 1 --2- 172.21.15.47:0/106224542 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8ae00a2f20 0x7f8ae00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:47.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.321+0000 7f8af2a00640 1 -- 172.21.15.47:0/106224542 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f8aec0f0d30 con 0x7f8afc102000 2024-03-21T11:20:47.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.322+0000 7f8af9400640 1 --2- 172.21.15.47:0/106224542 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8ae00a2f20 0x7f8ae00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:47.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.322+0000 7f8af9400640 1 --2- 172.21.15.47:0/106224542 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8ae00a2f20 0x7f8ae00a53e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8afc19af90 tx=0x7f8ae8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:47.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.325+0000 7f8af2a00640 1 -- 172.21.15.47:0/106224542 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8aec0bd7a0 con 0x7f8afc102000 2024-03-21T11:20:47.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:47.532+0000 7f8afb200640 1 -- 172.21.15.47:0/106224542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f8afc02d070 con 0x7f8afc102000 2024-03-21T11:20:48.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.122+0000 7f8af2a00640 1 -- 172.21.15.47:0/106224542 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) v1 ==== 171+0+0 (secure 0 0 0) 0x7f8aec0c1840 con 0x7f8afc102000 2024-03-21T11:20:48.125 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-03-21T11:20:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.127+0000 7f8afb200640 1 -- 172.21.15.47:0/106224542 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8ae00a2f20 msgr2=0x7f8ae00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.127+0000 7f8afb200640 1 --2- 172.21.15.47:0/106224542 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8ae00a2f20 0x7f8ae00a53e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8afc19af90 tx=0x7f8ae8005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.127+0000 7f8afb200640 1 -- 172.21.15.47:0/106224542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8afc102000 msgr2=0x7f8afc199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.127+0000 7f8afb200640 1 --2- 172.21.15.47:0/106224542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8afc102000 0x7f8afc199a80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009230 tx=0x7f8aec005d50 comp rx=0 tx=0).stop 2024-03-21T11:20:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.127+0000 7f8afb200640 1 -- 172.21.15.47:0/106224542 shutdown_connections 2024-03-21T11:20:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.127+0000 7f8afb200640 1 --2- 172.21.15.47:0/106224542 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8ae00a2f20 0x7f8ae00a53e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.127+0000 7f8afb200640 1 --2- 172.21.15.47:0/106224542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8afc107260 0x7f8afc1a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.127+0000 7f8afb200640 1 --2- 172.21.15.47:0/106224542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8afc104930 0x7f8afc199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.127+0000 7f8afb200640 1 --2- 172.21.15.47:0/106224542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8afc102000 0x7f8afc199a80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:48.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.127+0000 7f8afb200640 1 -- 172.21.15.47:0/106224542 >> 172.21.15.47:0/106224542 conn(0x7f8afc0fba10 msgr2=0x7f8afc1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.128+0000 7f8afb200640 1 -- 172.21.15.47:0/106224542 shutdown_connections 2024-03-21T11:20:48.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.128+0000 7f8afb200640 1 -- 172.21.15.47:0/106224542 wait complete. 2024-03-21T11:20:48.165 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-21T11:20:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.356+0000 7f4133a00640 1 -- 172.21.15.47:0/4065135461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f412c1016b0 msgr2=0x7f412c101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.356+0000 7f4133a00640 1 --2- 172.21.15.47:0/4065135461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f412c1016b0 0x7f412c101ad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4120009900 tx=0x7f412002bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.356+0000 7f4133a00640 1 -- 172.21.15.47:0/4065135461 shutdown_connections 2024-03-21T11:20:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.356+0000 7f4133a00640 1 --2- 172.21.15.47:0/4065135461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f412c103260 0x7f412c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.356+0000 7f4133a00640 1 --2- 172.21.15.47:0/4065135461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412c1028a0 0x7f412c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.356+0000 7f4133a00640 1 --2- 172.21.15.47:0/4065135461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f412c1016b0 0x7f412c101ad0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.356+0000 7f4133a00640 1 -- 172.21.15.47:0/4065135461 >> 172.21.15.47:0/4065135461 conn(0x7f412c0fcc20 msgr2=0x7f412c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.356+0000 7f4133a00640 1 -- 172.21.15.47:0/4065135461 shutdown_connections 2024-03-21T11:20:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.356+0000 7f4133a00640 1 -- 172.21.15.47:0/4065135461 wait complete. 2024-03-21T11:20:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.357+0000 7f4133a00640 1 Processor -- start 2024-03-21T11:20:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.357+0000 7f4133a00640 1 -- start start 2024-03-21T11:20:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.357+0000 7f4133a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412c1016b0 0x7f412c075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.357+0000 7f4133a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f412c1028a0 0x7f412c073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.358+0000 7f4133a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f412c103260 0x7f412c073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.358+0000 7f4133a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c10d370 con 0x7f412c1028a0 2024-03-21T11:20:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.358+0000 7f4133a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c10d1f0 con 0x7f412c1016b0 2024-03-21T11:20:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.358+0000 7f4133a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c10d4f0 con 0x7f412c103260 2024-03-21T11:20:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.358+0000 7f412be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f412c1028a0 0x7f412c073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.358+0000 7f4131000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412c1016b0 0x7f412c075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.358+0000 7f4131000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412c1016b0 0x7f412c075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38366/0 (socket says 172.21.15.47:38366) 2024-03-21T11:20:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.358+0000 7f4131000640 1 -- 172.21.15.47:0/3619959028 learned_addr learned my addr 172.21.15.47:0/3619959028 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.358+0000 7f4131a00640 1 --2- 172.21.15.47:0/3619959028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f412c103260 0x7f412c073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.358+0000 7f4131000640 1 -- 172.21.15.47:0/3619959028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f412c103260 msgr2=0x7f412c073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.358+0000 7f4131000640 1 --2- 172.21.15.47:0/3619959028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f412c103260 0x7f412c073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.358+0000 7f4131000640 1 -- 172.21.15.47:0/3619959028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f412c1028a0 msgr2=0x7f412c073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.358+0000 7f4131000640 1 --2- 172.21.15.47:0/3619959028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f412c1028a0 0x7f412c073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.358+0000 7f4131000640 1 -- 172.21.15.47:0/3619959028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c074860 con 0x7f412c1016b0 2024-03-21T11:20:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.359+0000 7f4131000640 1 --2- 172.21.15.47:0/3619959028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412c1016b0 0x7f412c075450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4120000c00 tx=0x7f412002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.359+0000 7f4129600640 1 -- 172.21.15.47:0/3619959028 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120043020 con 0x7f412c1016b0 2024-03-21T11:20:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.359+0000 7f4129600640 1 -- 172.21.15.47:0/3619959028 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41200054e0 con 0x7f412c1016b0 2024-03-21T11:20:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.359+0000 7f4129600640 1 -- 172.21.15.47:0/3619959028 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120003c40 con 0x7f412c1016b0 2024-03-21T11:20:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.359+0000 7f4133a00640 1 -- 172.21.15.47:0/3619959028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412c1a1b20 con 0x7f412c1016b0 2024-03-21T11:20:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.359+0000 7f4133a00640 1 -- 172.21.15.47:0/3619959028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412c1a2070 con 0x7f412c1016b0 2024-03-21T11:20:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.361+0000 7f4133a00640 1 -- 172.21.15.47:0/3619959028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412c108a70 con 0x7f412c1016b0 2024-03-21T11:20:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.362+0000 7f4129600640 1 -- 172.21.15.47:0/3619959028 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4120046020 con 0x7f412c1016b0 2024-03-21T11:20:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.363+0000 7f4129600640 1 --2- 172.21.15.47:0/3619959028 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41140a2f20 0x7f41140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.363+0000 7f4129600640 1 -- 172.21.15.47:0/3619959028 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f41200f1370 con 0x7f412c1016b0 2024-03-21T11:20:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.363+0000 7f412be00640 1 --2- 172.21.15.47:0/3619959028 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41140a2f20 0x7f41140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.364+0000 7f412be00640 1 --2- 172.21.15.47:0/3619959028 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41140a2f20 0x7f41140a53e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f412c0751c0 tx=0x7f4118005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.367+0000 7f4129600640 1 -- 172.21.15.47:0/3619959028 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f412004d020 con 0x7f412c1016b0 2024-03-21T11:20:48.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:48 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1500465955' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T11:20:48.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:48 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T11:20:48.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:48 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T11:20:48.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:48 smithi100 ceph-mon[36245]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T11:20:48.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:48 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1500465955' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T11:20:48.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:48 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T11:20:48.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:48 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T11:20:48.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:48 smithi102 ceph-mon[36748]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T11:20:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:48 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1500465955' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T11:20:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:48 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T11:20:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:48 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T11:20:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:48 smithi047 ceph-mon[31351]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T11:20:48.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:48.622+0000 7f4133a00640 1 -- 172.21.15.47:0/3619959028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f412c0622a0 con 0x7f412c1016b0 2024-03-21T11:20:49.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/106224542' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T11:20:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T11:20:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-21T11:20:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T11:20:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3619959028' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T11:20:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T11:20:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-21T11:20:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T11:20:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:20:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/106224542' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T11:20:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T11:20:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-21T11:20:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T11:20:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3619959028' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T11:20:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T11:20:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-21T11:20:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T11:20:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:20:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:49.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/106224542' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T11:20:49.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T11:20:49.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-21T11:20:49.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T11:20:49.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3619959028' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T11:20:49.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T11:20:49.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-21T11:20:49.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T11:20:49.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:20:49.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:20:49.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:20:49.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.591+0000 7f4129600640 1 -- 172.21.15.47:0/3619959028 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f41200be2b0 con 0x7f412c1016b0 2024-03-21T11:20:49.592 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 2 and data pool 3 2024-03-21T11:20:49.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.596+0000 7f4133a00640 1 -- 172.21.15.47:0/3619959028 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41140a2f20 msgr2=0x7f41140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:49.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.596+0000 7f4133a00640 1 --2- 172.21.15.47:0/3619959028 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41140a2f20 0x7f41140a53e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f412c0751c0 tx=0x7f4118005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.596+0000 7f4133a00640 1 -- 172.21.15.47:0/3619959028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412c1016b0 msgr2=0x7f412c075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.596+0000 7f4133a00640 1 --2- 172.21.15.47:0/3619959028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412c1016b0 0x7f412c075450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4120000c00 tx=0x7f412002c650 comp rx=0 tx=0).stop 2024-03-21T11:20:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.596+0000 7f4133a00640 1 -- 172.21.15.47:0/3619959028 shutdown_connections 2024-03-21T11:20:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.596+0000 7f4133a00640 1 --2- 172.21.15.47:0/3619959028 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41140a2f20 0x7f41140a53e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.596+0000 7f4133a00640 1 --2- 172.21.15.47:0/3619959028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f412c103260 0x7f412c073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.596+0000 7f4133a00640 1 --2- 172.21.15.47:0/3619959028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f412c1028a0 0x7f412c073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:49.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.596+0000 7f4133a00640 1 --2- 172.21.15.47:0/3619959028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412c1016b0 0x7f412c075450 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4120000c00 tx=0x7f412002c650 comp rx=0 tx=0).stop 2024-03-21T11:20:49.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.596+0000 7f4133a00640 1 -- 172.21.15.47:0/3619959028 >> 172.21.15.47:0/3619959028 conn(0x7f412c0fcc20 msgr2=0x7f412c105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:49.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.597+0000 7f4133a00640 1 -- 172.21.15.47:0/3619959028 shutdown_connections 2024-03-21T11:20:49.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.597+0000 7f4133a00640 1 -- 172.21.15.47:0/3619959028 wait complete. 2024-03-21T11:20:49.622 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-21T11:20:49.622 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-21T11:20:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.842+0000 7fb01f200640 1 -- 172.21.15.47:0/880465711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb020109770 msgr2=0x7fb020110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.842+0000 7fb01f200640 1 --2- 172.21.15.47:0/880465711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb020109770 0x7fb020110000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb018009ca0 tx=0x7fb01802c0f0 comp rx=0 tx=0).stop 2024-03-21T11:20:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.843+0000 7fb01f200640 1 -- 172.21.15.47:0/880465711 shutdown_connections 2024-03-21T11:20:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.843+0000 7fb01f200640 1 --2- 172.21.15.47:0/880465711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb020109770 0x7fb020110000 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.843+0000 7fb01f200640 1 --2- 172.21.15.47:0/880465711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb020108db0 0x7fb020109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.843+0000 7fb01f200640 1 --2- 172.21.15.47:0/880465711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb020107bc0 0x7fb020107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.843+0000 7fb01f200640 1 -- 172.21.15.47:0/880465711 >> 172.21.15.47:0/880465711 conn(0x7fb020076b70 msgr2=0x7fb020078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.843+0000 7fb01f200640 1 -- 172.21.15.47:0/880465711 shutdown_connections 2024-03-21T11:20:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.843+0000 7fb01f200640 1 -- 172.21.15.47:0/880465711 wait complete. 2024-03-21T11:20:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.844+0000 7fb01f200640 1 Processor -- start 2024-03-21T11:20:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.844+0000 7fb01f200640 1 -- start start 2024-03-21T11:20:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb020107bc0 0x7fb02019de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb020108db0 0x7fb02019e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb020109770 0x7fb0201a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb020113820 con 0x7fb020109770 2024-03-21T11:20:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0201136a0 con 0x7fb020107bc0 2024-03-21T11:20:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0201139a0 con 0x7fb020108db0 2024-03-21T11:20:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb020107bc0 0x7fb02019de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb020107bc0 0x7fb02019de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48298/0 (socket says 172.21.15.47:48298) 2024-03-21T11:20:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01de00640 1 -- 172.21.15.47:0/4145020258 learned_addr learned my addr 172.21.15.47:0/4145020258 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01e800640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb020109770 0x7fb0201a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb017e00640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb020108db0 0x7fb02019e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01de00640 1 -- 172.21.15.47:0/4145020258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb020108db0 msgr2=0x7fb02019e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01de00640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb020108db0 0x7fb02019e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01de00640 1 -- 172.21.15.47:0/4145020258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb020109770 msgr2=0x7fb0201a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01de00640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb020109770 0x7fb0201a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb01de00640 1 -- 172.21.15.47:0/4145020258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0201a5a20 con 0x7fb020107bc0 2024-03-21T11:20:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.845+0000 7fb017e00640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb020108db0 0x7fb02019e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:20:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.846+0000 7fb01de00640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb020107bc0 0x7fb02019de00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb01000e9c0 tx=0x7fb01000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.846+0000 7fb016a00640 1 -- 172.21.15.47:0/4145020258 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01000cd80 con 0x7fb020107bc0 2024-03-21T11:20:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.846+0000 7fb016a00640 1 -- 172.21.15.47:0/4145020258 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0100035f0 con 0x7fb020107bc0 2024-03-21T11:20:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.846+0000 7fb01f200640 1 -- 172.21.15.47:0/4145020258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0201a5d10 con 0x7fb020107bc0 2024-03-21T11:20:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.846+0000 7fb016a00640 1 -- 172.21.15.47:0/4145020258 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb010010690 con 0x7fb020107bc0 2024-03-21T11:20:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.846+0000 7fb01f200640 1 -- 172.21.15.47:0/4145020258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0201a61b0 con 0x7fb020107bc0 2024-03-21T11:20:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.847+0000 7fb01f200640 1 -- 172.21.15.47:0/4145020258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02010eee0 con 0x7fb020107bc0 2024-03-21T11:20:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.849+0000 7fb016a00640 1 -- 172.21.15.47:0/4145020258 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb0100037a0 con 0x7fb020107bc0 2024-03-21T11:20:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.850+0000 7fb016a00640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0040a2f20 0x7fb0040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.851+0000 7fb016a00640 1 -- 172.21.15.47:0/4145020258 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb0100cf780 con 0x7fb020107bc0 2024-03-21T11:20:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.851+0000 7fb017e00640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0040a2f20 0x7fb0040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.852+0000 7fb017e00640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0040a2f20 0x7fb0040a53e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb02019f310 tx=0x7fb008005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:49.855+0000 7fb016a00640 1 -- 172.21.15.47:0/4145020258 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb01009c660 con 0x7fb020107bc0 2024-03-21T11:20:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.070+0000 7fb01f200640 1 -- 172.21.15.47:0/4145020258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fb02002d070 con 0x7fb020107bc0 2024-03-21T11:20:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.690+0000 7fb016a00640 1 -- 172.21.15.47:0/4145020258 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v87) v1 ==== 157+0+0 (secure 0 0 0) 0x7fb02002d070 con 0x7fb020107bc0 2024-03-21T11:20:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.695+0000 7fb01f200640 1 -- 172.21.15.47:0/4145020258 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0040a2f20 msgr2=0x7fb0040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.695+0000 7fb01f200640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0040a2f20 0x7fb0040a53e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb02019f310 tx=0x7fb008005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.695+0000 7fb01f200640 1 -- 172.21.15.47:0/4145020258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb020107bc0 msgr2=0x7fb02019de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.695+0000 7fb01f200640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb020107bc0 0x7fb02019de00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb01000e9c0 tx=0x7fb01000ee80 comp rx=0 tx=0).stop 2024-03-21T11:20:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.696+0000 7fb01f200640 1 -- 172.21.15.47:0/4145020258 shutdown_connections 2024-03-21T11:20:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.696+0000 7fb01f200640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0040a2f20 0x7fb0040a53e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.696+0000 7fb01f200640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb020109770 0x7fb0201a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.696+0000 7fb01f200640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb020108db0 0x7fb02019e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.696+0000 7fb01f200640 1 --2- 172.21.15.47:0/4145020258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb020107bc0 0x7fb02019de00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.696+0000 7fb01f200640 1 -- 172.21.15.47:0/4145020258 >> 172.21.15.47:0/4145020258 conn(0x7fb020076b70 msgr2=0x7fb02010c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.696+0000 7fb01f200640 1 -- 172.21.15.47:0/4145020258 shutdown_connections 2024-03-21T11:20:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.696+0000 7fb01f200640 1 -- 172.21.15.47:0/4145020258 wait complete. 2024-03-21T11:20:50.720 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-21T11:20:50.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:50 smithi100 ceph-mon[36245]: pgmap v315: 97 pgs: 15 creating+peering, 74 unknown, 8 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:50.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:50 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T11:20:50.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:50 smithi100 ceph-mon[36245]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-21T11:20:50.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:50 smithi100 ceph-mon[36245]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-21T11:20:50.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:50 smithi100 ceph-mon[36245]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T11:20:50.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:50 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/4145020258' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T11:20:50.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:50 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T11:20:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.958+0000 7f0814a00640 1 -- 172.21.15.47:0/3825622556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0810105280 msgr2=0x7f081010bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.958+0000 7f0814a00640 1 --2- 172.21.15.47:0/3825622556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0810105280 0x7f081010bb10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0808009ca0 tx=0x7f080802c0f0 comp rx=0 tx=0).stop 2024-03-21T11:20:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.958+0000 7f0814a00640 1 -- 172.21.15.47:0/3825622556 shutdown_connections 2024-03-21T11:20:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.958+0000 7f0814a00640 1 --2- 172.21.15.47:0/3825622556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0810105280 0x7f081010bb10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:50.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.958+0000 7f0814a00640 1 --2- 172.21.15.47:0/3825622556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08101048c0 0x7f0810104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:50.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.958+0000 7f0814a00640 1 --2- 172.21.15.47:0/3825622556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08101036d0 0x7f0810103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:50.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.958+0000 7f0814a00640 1 -- 172.21.15.47:0/3825622556 >> 172.21.15.47:0/3825622556 conn(0x7f08100fec40 msgr2=0x7f0810101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:50.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.958+0000 7f0814a00640 1 -- 172.21.15.47:0/3825622556 shutdown_connections 2024-03-21T11:20:50.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.958+0000 7f0814a00640 1 -- 172.21.15.47:0/3825622556 wait complete. 2024-03-21T11:20:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.959+0000 7f0814a00640 1 Processor -- start 2024-03-21T11:20:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.959+0000 7f0814a00640 1 -- start start 2024-03-21T11:20:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.960+0000 7f0814a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08101036d0 0x7f08101998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.960+0000 7f0814a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08101048c0 0x7f0810199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.960+0000 7f0814a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0810105280 0x7f08101a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.960+0000 7f0814a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081010f330 con 0x7f08101036d0 2024-03-21T11:20:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.960+0000 7f0814a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081010f1b0 con 0x7f08101048c0 2024-03-21T11:20:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.960+0000 7f0814a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081010f4b0 con 0x7f0810105280 2024-03-21T11:20:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.960+0000 7f080e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08101036d0 0x7f08101998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.960+0000 7f080de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08101048c0 0x7f0810199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.960+0000 7f080e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08101036d0 0x7f08101998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60644/0 (socket says 172.21.15.47:60644) 2024-03-21T11:20:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.960+0000 7f080e800640 1 -- 172.21.15.47:0/1170930 learned_addr learned my addr 172.21.15.47:0/1170930 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.960+0000 7f080f200640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0810105280 0x7f08101a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.961+0000 7f080e800640 1 -- 172.21.15.47:0/1170930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0810105280 msgr2=0x7f08101a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.961+0000 7f080e800640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0810105280 0x7f08101a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.961+0000 7f080e800640 1 -- 172.21.15.47:0/1170930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08101048c0 msgr2=0x7f0810199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.961+0000 7f080e800640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08101048c0 0x7f0810199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.961+0000 7f080e800640 1 -- 172.21.15.47:0/1170930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08101a1510 con 0x7f08101036d0 2024-03-21T11:20:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.961+0000 7f080f200640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0810105280 0x7f08101a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:20:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.961+0000 7f080de00640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08101048c0 0x7f0810199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:20:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.961+0000 7f080e800640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08101036d0 0x7f08101998f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f080000e9c0 tx=0x7f080000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.961+0000 7f0807400640 1 -- 172.21.15.47:0/1170930 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080000cd80 con 0x7f08101036d0 2024-03-21T11:20:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.961+0000 7f0807400640 1 -- 172.21.15.47:0/1170930 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08000035f0 con 0x7f08101036d0 2024-03-21T11:20:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.961+0000 7f0807400640 1 -- 172.21.15.47:0/1170930 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800010690 con 0x7f08101036d0 2024-03-21T11:20:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.961+0000 7f0814a00640 1 -- 172.21.15.47:0/1170930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08101a1800 con 0x7f08101036d0 2024-03-21T11:20:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.962+0000 7f0814a00640 1 -- 172.21.15.47:0/1170930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08101a1d00 con 0x7f08101036d0 2024-03-21T11:20:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.964+0000 7f0814a00640 1 -- 172.21.15.47:0/1170930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081010a990 con 0x7f08101036d0 2024-03-21T11:20:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.966+0000 7f0807400640 1 -- 172.21.15.47:0/1170930 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f08000037a0 con 0x7f08101036d0 2024-03-21T11:20:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.967+0000 7f0807400640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07f40a2f20 0x7f07f40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.968+0000 7f0807400640 1 -- 172.21.15.47:0/1170930 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f08000cf830 con 0x7f08101036d0 2024-03-21T11:20:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.968+0000 7f080de00640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07f40a2f20 0x7f07f40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.968+0000 7f080de00640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07f40a2f20 0x7f07f40a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f081019ae00 tx=0x7f07fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:50.972+0000 7f0807400640 1 -- 172.21.15.47:0/1170930 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f080009c5d0 con 0x7f08101036d0 2024-03-21T11:20:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:50 smithi102 ceph-mon[36748]: pgmap v315: 97 pgs: 15 creating+peering, 74 unknown, 8 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:50 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T11:20:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:50 smithi102 ceph-mon[36748]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-21T11:20:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:50 smithi102 ceph-mon[36748]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-21T11:20:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:50 smithi102 ceph-mon[36748]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T11:20:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:50 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/4145020258' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T11:20:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:50 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T11:20:51.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:50 smithi047 ceph-mon[31351]: pgmap v315: 97 pgs: 15 creating+peering, 74 unknown, 8 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:20:51.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:50 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T11:20:51.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:50 smithi047 ceph-mon[31351]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-21T11:20:51.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:50 smithi047 ceph-mon[31351]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-21T11:20:51.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:50 smithi047 ceph-mon[31351]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T11:20:51.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:50 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/4145020258' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T11:20:51.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:50 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T11:20:51.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:51.179+0000 7f0814a00640 1 -- 172.21.15.47:0/1170930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f081002d070 con 0x7f08101036d0 2024-03-21T11:20:51.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:51 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] up:active 2024-03-21T11:20:51.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:51 smithi100 ceph-mon[36245]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:20:51.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:51 smithi100 ceph-mon[36245]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:20:51.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:51 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-21T11:20:51.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:51 smithi100 ceph-mon[36245]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T11:20:51.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:51 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1170930' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T11:20:51.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:51 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] up:active 2024-03-21T11:20:51.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:51 smithi047 ceph-mon[31351]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:20:51.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:51 smithi047 ceph-mon[31351]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:20:51.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:51 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-21T11:20:51.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:51 smithi047 ceph-mon[31351]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T11:20:51.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:51 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1170930' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T11:20:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:51 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] up:active 2024-03-21T11:20:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:51 smithi102 ceph-mon[36748]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:20:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:51 smithi102 ceph-mon[36748]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:20:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:51 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-21T11:20:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:51 smithi102 ceph-mon[36748]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T11:20:51.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:51 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1170930' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T11:20:52.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.710+0000 7f0807400640 1 -- 172.21.15.47:0/1170930 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v89) v1 ==== 228+0+0 (secure 0 0 0) 0x7f080009bed0 con 0x7f08101036d0 2024-03-21T11:20:52.711 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data_ec' created 2024-03-21T11:20:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.715+0000 7f0814a00640 1 -- 172.21.15.47:0/1170930 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07f40a2f20 msgr2=0x7f07f40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.715+0000 7f0814a00640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07f40a2f20 0x7f07f40a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f081019ae00 tx=0x7f07fc005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.715+0000 7f0814a00640 1 -- 172.21.15.47:0/1170930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08101036d0 msgr2=0x7f08101998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.715+0000 7f0814a00640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08101036d0 0x7f08101998f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f080000e9c0 tx=0x7f080000ee80 comp rx=0 tx=0).stop 2024-03-21T11:20:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.715+0000 7f0814a00640 1 -- 172.21.15.47:0/1170930 shutdown_connections 2024-03-21T11:20:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.715+0000 7f0814a00640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07f40a2f20 0x7f07f40a53e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.715+0000 7f0814a00640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0810105280 0x7f08101a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.715+0000 7f0814a00640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08101048c0 0x7f0810199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.715+0000 7f0814a00640 1 --2- 172.21.15.47:0/1170930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08101036d0 0x7f08101998f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.715+0000 7f0814a00640 1 -- 172.21.15.47:0/1170930 >> 172.21.15.47:0/1170930 conn(0x7f08100fec40 msgr2=0x7f0810107b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.716+0000 7f0814a00640 1 -- 172.21.15.47:0/1170930 shutdown_connections 2024-03-21T11:20:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.716+0000 7f0814a00640 1 -- 172.21.15.47:0/1170930 wait complete. 2024-03-21T11:20:52.744 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-21T11:20:52.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:52 smithi100 ceph-mon[36245]: pgmap v318: 97 pgs: 15 creating+peering, 45 unknown, 37 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 786 B/s wr, 1 op/s 2024-03-21T11:20:52.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:52 smithi100 ceph-mon[36245]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T11:20:52.893 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:20:52 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b[66324]: 2024-03-21T11:20:52.499+0000 7f3f74d62700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-21T11:20:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.933+0000 7fb970800640 1 -- 172.21.15.47:0/3270377111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968104ad0 msgr2=0x7fb968104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.933+0000 7fb970800640 1 --2- 172.21.15.47:0/3270377111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968104ad0 0x7fb968104f50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb958009900 tx=0x7fb95802bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.933+0000 7fb970800640 1 -- 172.21.15.47:0/3270377111 shutdown_connections 2024-03-21T11:20:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.933+0000 7fb970800640 1 --2- 172.21.15.47:0/3270377111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb968105490 0x7fb96810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.933+0000 7fb970800640 1 --2- 172.21.15.47:0/3270377111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968104ad0 0x7fb968104f50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.933+0000 7fb970800640 1 --2- 172.21.15.47:0/3270377111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9681038e0 0x7fb968103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.933+0000 7fb970800640 1 -- 172.21.15.47:0/3270377111 >> 172.21.15.47:0/3270377111 conn(0x7fb9680fee50 msgr2=0x7fb968101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.933+0000 7fb970800640 1 -- 172.21.15.47:0/3270377111 shutdown_connections 2024-03-21T11:20:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.933+0000 7fb970800640 1 -- 172.21.15.47:0/3270377111 wait complete. 2024-03-21T11:20:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.934+0000 7fb970800640 1 Processor -- start 2024-03-21T11:20:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.934+0000 7fb970800640 1 -- start start 2024-03-21T11:20:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb970800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9681038e0 0x7fb968199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb970800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb968104ad0 0x7fb96819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb970800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968105490 0x7fb9681a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb970800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96810f5b0 con 0x7fb968105490 2024-03-21T11:20:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb970800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96810f430 con 0x7fb9681038e0 2024-03-21T11:20:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb970800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96810f730 con 0x7fb968104ad0 2024-03-21T11:20:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb96e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968105490 0x7fb9681a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb96de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9681038e0 0x7fb968199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb96e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968105490 0x7fb9681a11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60670/0 (socket says 172.21.15.47:60670) 2024-03-21T11:20:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb96e800640 1 -- 172.21.15.47:0/3364530907 learned_addr learned my addr 172.21.15.47:0/3364530907 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb96d400640 1 --2- 172.21.15.47:0/3364530907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb968104ad0 0x7fb96819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb96de00640 1 -- 172.21.15.47:0/3364530907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb968104ad0 msgr2=0x7fb96819a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:52.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb96de00640 1 --2- 172.21.15.47:0/3364530907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb968104ad0 0x7fb96819a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:52.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb96de00640 1 -- 172.21.15.47:0/3364530907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968105490 msgr2=0x7fb9681a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:52.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb96de00640 1 --2- 172.21.15.47:0/3364530907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968105490 0x7fb9681a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:52.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb96de00640 1 -- 172.21.15.47:0/3364530907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9681a1860 con 0x7fb9681038e0 2024-03-21T11:20:52.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.935+0000 7fb96e800640 1 --2- 172.21.15.47:0/3364530907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968105490 0x7fb9681a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:20:52.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.936+0000 7fb96de00640 1 --2- 172.21.15.47:0/3364530907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9681038e0 0x7fb968199c50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00e9c0 tx=0x7fb95c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:52.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.936+0000 7fb962a00640 1 -- 172.21.15.47:0/3364530907 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c00ccc0 con 0x7fb9681038e0 2024-03-21T11:20:52.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.936+0000 7fb962a00640 1 -- 172.21.15.47:0/3364530907 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb95c0035f0 con 0x7fb9681038e0 2024-03-21T11:20:52.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.936+0000 7fb962a00640 1 -- 172.21.15.47:0/3364530907 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c00ce60 con 0x7fb9681038e0 2024-03-21T11:20:52.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.936+0000 7fb970800640 1 -- 172.21.15.47:0/3364530907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9681a1b50 con 0x7fb9681038e0 2024-03-21T11:20:52.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.936+0000 7fb970800640 1 -- 172.21.15.47:0/3364530907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9681a2050 con 0x7fb9681038e0 2024-03-21T11:20:52.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.937+0000 7fb970800640 1 -- 172.21.15.47:0/3364530907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb96810acd0 con 0x7fb9681038e0 2024-03-21T11:20:52.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.940+0000 7fb962a00640 1 -- 172.21.15.47:0/3364530907 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb95c0037a0 con 0x7fb9681038e0 2024-03-21T11:20:52.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.941+0000 7fb962a00640 1 --2- 172.21.15.47:0/3364530907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9500a2f20 0x7fb9500a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:52.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.941+0000 7fb962a00640 1 -- 172.21.15.47:0/3364530907 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8770+0+0 (secure 0 0 0) 0x7fb95c0cfc10 con 0x7fb9681038e0 2024-03-21T11:20:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.941+0000 7fb96d400640 1 --2- 172.21.15.47:0/3364530907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9500a2f20 0x7fb9500a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.942+0000 7fb96d400640 1 --2- 172.21.15.47:0/3364530907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9500a2f20 0x7fb9500a53e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb95802c240 tx=0x7fb95802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:52.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:52.945+0000 7fb962a00640 1 -- 172.21.15.47:0/3364530907 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb95c09c7c0 con 0x7fb9681038e0 2024-03-21T11:20:52.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:52 smithi102 ceph-mon[36748]: pgmap v318: 97 pgs: 15 creating+peering, 45 unknown, 37 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 786 B/s wr, 1 op/s 2024-03-21T11:20:52.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:52 smithi102 ceph-mon[36748]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T11:20:53.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:52 smithi047 ceph-mon[31351]: pgmap v318: 97 pgs: 15 creating+peering, 45 unknown, 37 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 786 B/s wr, 1 op/s 2024-03-21T11:20:53.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:52 smithi047 ceph-mon[31351]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T11:20:53.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.147+0000 7fb970800640 1 -- 172.21.15.47:0/3364530907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fb96802d070 con 0x7fb9681038e0 2024-03-21T11:20:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:set pool 4 allow_ec_overwrites to true 2024-03-21T11:20:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.719+0000 7fb962a00640 1 -- 172.21.15.47:0/3364530907 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v90) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb95c09c0c0 con 0x7fb9681038e0 2024-03-21T11:20:53.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.724+0000 7fb970800640 1 -- 172.21.15.47:0/3364530907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9500a2f20 msgr2=0x7fb9500a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:53.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.724+0000 7fb970800640 1 --2- 172.21.15.47:0/3364530907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9500a2f20 0x7fb9500a53e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb95802c240 tx=0x7fb95802df90 comp rx=0 tx=0).stop 2024-03-21T11:20:53.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.724+0000 7fb970800640 1 -- 172.21.15.47:0/3364530907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9681038e0 msgr2=0x7fb968199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:53.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.724+0000 7fb970800640 1 --2- 172.21.15.47:0/3364530907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9681038e0 0x7fb968199c50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00e9c0 tx=0x7fb95c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:20:53.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.725+0000 7fb970800640 1 -- 172.21.15.47:0/3364530907 shutdown_connections 2024-03-21T11:20:53.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.725+0000 7fb970800640 1 --2- 172.21.15.47:0/3364530907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9500a2f20 0x7fb9500a53e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:53.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.725+0000 7fb970800640 1 --2- 172.21.15.47:0/3364530907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968105490 0x7fb9681a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:53.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.725+0000 7fb970800640 1 --2- 172.21.15.47:0/3364530907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb968104ad0 0x7fb96819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.725+0000 7fb970800640 1 --2- 172.21.15.47:0/3364530907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9681038e0 0x7fb968199c50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.725+0000 7fb970800640 1 -- 172.21.15.47:0/3364530907 >> 172.21.15.47:0/3364530907 conn(0x7fb9680fee50 msgr2=0x7fb968107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.725+0000 7fb970800640 1 -- 172.21.15.47:0/3364530907 shutdown_connections 2024-03-21T11:20:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.725+0000 7fb970800640 1 -- 172.21.15.47:0/3364530907 wait complete. 2024-03-21T11:20:53.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:53 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1170930' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T11:20:53.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:53 smithi047 ceph-mon[31351]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T11:20:53.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:53 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3364530907' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T11:20:53.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:53 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T11:20:53.756 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-21T11:20:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.938+0000 7f9399200640 1 -- 172.21.15.47:0/762806752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941015a0 msgr2=0x7f93941019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.938+0000 7f9399200640 1 --2- 172.21.15.47:0/762806752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941015a0 0x7f93941019c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9384009ca0 tx=0x7f938402c100 comp rx=0 tx=0).stop 2024-03-21T11:20:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.938+0000 7f9399200640 1 -- 172.21.15.47:0/762806752 shutdown_connections 2024-03-21T11:20:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.938+0000 7f9399200640 1 --2- 172.21.15.47:0/762806752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9394103150 0x7f93941099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.938+0000 7f9399200640 1 --2- 172.21.15.47:0/762806752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9394102790 0x7f9394102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.938+0000 7f9399200640 1 --2- 172.21.15.47:0/762806752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941015a0 0x7f93941019c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.938+0000 7f9399200640 1 -- 172.21.15.47:0/762806752 >> 172.21.15.47:0/762806752 conn(0x7f93940fcb10 msgr2=0x7f93940fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.938+0000 7f9399200640 1 -- 172.21.15.47:0/762806752 shutdown_connections 2024-03-21T11:20:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.938+0000 7f9399200640 1 -- 172.21.15.47:0/762806752 wait complete. 2024-03-21T11:20:53.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.939+0000 7f9399200640 1 Processor -- start 2024-03-21T11:20:53.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.939+0000 7f9399200640 1 -- start start 2024-03-21T11:20:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.939+0000 7f9399200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941015a0 0x7f9394075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.939+0000 7f9399200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9394102790 0x7f9394073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.939+0000 7f9399200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9394103150 0x7f9394073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.939+0000 7f9399200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939410d230 con 0x7f9394103150 2024-03-21T11:20:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.939+0000 7f9399200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939410d0b0 con 0x7f93941015a0 2024-03-21T11:20:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.939+0000 7f9399200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939410d3b0 con 0x7f9394102790 2024-03-21T11:20:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.939+0000 7f9393400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941015a0 0x7f9394075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.939+0000 7f9393400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941015a0 0x7f9394075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48336/0 (socket says 172.21.15.47:48336) 2024-03-21T11:20:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.939+0000 7f9393400640 1 -- 172.21.15.47:0/4030515883 learned_addr learned my addr 172.21.15.47:0/4030515883 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.940+0000 7f9393e00640 1 --2- 172.21.15.47:0/4030515883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9394103150 0x7f9394073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.940+0000 7f9392a00640 1 --2- 172.21.15.47:0/4030515883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9394102790 0x7f9394073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.940+0000 7f9393400640 1 -- 172.21.15.47:0/4030515883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9394102790 msgr2=0x7f9394073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.940+0000 7f9393400640 1 --2- 172.21.15.47:0/4030515883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9394102790 0x7f9394073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.940+0000 7f9393400640 1 -- 172.21.15.47:0/4030515883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9394103150 msgr2=0x7f9394073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.940+0000 7f9393400640 1 --2- 172.21.15.47:0/4030515883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9394103150 0x7f9394073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.940+0000 7f9393400640 1 -- 172.21.15.47:0/4030515883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9394074860 con 0x7f93941015a0 2024-03-21T11:20:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.940+0000 7f9393400640 1 --2- 172.21.15.47:0/4030515883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941015a0 0x7f9394075450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9384000f80 tx=0x7f9384035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.940+0000 7f938be00640 1 -- 172.21.15.47:0/4030515883 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384039040 con 0x7f93941015a0 2024-03-21T11:20:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.940+0000 7f938be00640 1 -- 172.21.15.47:0/4030515883 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9384044070 con 0x7f93941015a0 2024-03-21T11:20:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.940+0000 7f938be00640 1 -- 172.21.15.47:0/4030515883 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93840045a0 con 0x7f93941015a0 2024-03-21T11:20:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.940+0000 7f9399200640 1 -- 172.21.15.47:0/4030515883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93941a1980 con 0x7f93941015a0 2024-03-21T11:20:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.941+0000 7f9399200640 1 -- 172.21.15.47:0/4030515883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93941a1e50 con 0x7f93941015a0 2024-03-21T11:20:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.941+0000 7f9399200640 1 -- 172.21.15.47:0/4030515883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9394108990 con 0x7f93941015a0 2024-03-21T11:20:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.943+0000 7f938be00640 1 -- 172.21.15.47:0/4030515883 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9384032890 con 0x7f93941015a0 2024-03-21T11:20:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.944+0000 7f938be00640 1 --2- 172.21.15.47:0/4030515883 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93780a2f20 0x7f93780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.945+0000 7f938be00640 1 -- 172.21.15.47:0/4030515883 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f93840f1e20 con 0x7f93941015a0 2024-03-21T11:20:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.945+0000 7f9392a00640 1 --2- 172.21.15.47:0/4030515883 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93780a2f20 0x7f93780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.946+0000 7f9392a00640 1 --2- 172.21.15.47:0/4030515883 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93780a2f20 0x7f93780a53e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9380005fd0 tx=0x7f9380005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:53.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:53.948+0000 7f938be00640 1 -- 172.21.15.47:0/4030515883 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f93840be9d0 con 0x7f93941015a0 2024-03-21T11:20:53.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:53 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1170930' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T11:20:53.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:53 smithi102 ceph-mon[36748]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T11:20:53.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:53 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3364530907' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T11:20:53.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:53 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T11:20:54.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:53 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1170930' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T11:20:54.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:53 smithi100 ceph-mon[36245]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T11:20:54.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:53 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3364530907' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T11:20:54.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:53 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T11:20:54.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.188+0000 7f9399200640 1 -- 172.21.15.47:0/4030515883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f93940622a0 con 0x7f93941015a0 2024-03-21T11:20:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.746+0000 7f938be00640 1 -- 172.21.15.47:0/4030515883 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f93840be2d0 con 0x7f93941015a0 2024-03-21T11:20:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:added data pool 4 to fsmap 2024-03-21T11:20:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.750+0000 7f9399200640 1 -- 172.21.15.47:0/4030515883 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93780a2f20 msgr2=0x7f93780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.750+0000 7f9399200640 1 --2- 172.21.15.47:0/4030515883 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93780a2f20 0x7f93780a53e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9380005fd0 tx=0x7f9380005850 comp rx=0 tx=0).stop 2024-03-21T11:20:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.750+0000 7f9399200640 1 -- 172.21.15.47:0/4030515883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941015a0 msgr2=0x7f9394075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.750+0000 7f9399200640 1 --2- 172.21.15.47:0/4030515883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941015a0 0x7f9394075450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9384000f80 tx=0x7f9384035d10 comp rx=0 tx=0).stop 2024-03-21T11:20:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.750+0000 7f9399200640 1 -- 172.21.15.47:0/4030515883 shutdown_connections 2024-03-21T11:20:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.750+0000 7f9399200640 1 --2- 172.21.15.47:0/4030515883 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93780a2f20 0x7f93780a53e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.750+0000 7f9399200640 1 --2- 172.21.15.47:0/4030515883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9394103150 0x7f9394073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.750+0000 7f9399200640 1 --2- 172.21.15.47:0/4030515883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9394102790 0x7f9394073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.750+0000 7f9399200640 1 --2- 172.21.15.47:0/4030515883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941015a0 0x7f9394075450 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.750+0000 7f9399200640 1 -- 172.21.15.47:0/4030515883 >> 172.21.15.47:0/4030515883 conn(0x7f93940fcb10 msgr2=0x7f9394105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.750+0000 7f9399200640 1 -- 172.21.15.47:0/4030515883 shutdown_connections 2024-03-21T11:20:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.750+0000 7f9399200640 1 -- 172.21.15.47:0/4030515883 wait complete. 2024-03-21T11:20:54.770 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.990+0000 7fccc6400640 1 -- 172.21.15.47:0/2572095466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc01073a0 msgr2=0x7fccc01097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.990+0000 7fccc6400640 1 --2- 172.21.15.47:0/2572095466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc01073a0 0x7fccc01097f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009ca0 tx=0x7fccb802c100 comp rx=0 tx=0).stop 2024-03-21T11:20:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.990+0000 7fccc6400640 1 -- 172.21.15.47:0/2572095466 shutdown_connections 2024-03-21T11:20:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.990+0000 7fccc6400640 1 --2- 172.21.15.47:0/2572095466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc01073a0 0x7fccc01097f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.990+0000 7fccc6400640 1 --2- 172.21.15.47:0/2572095466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccc0104a70 0x7fccc0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.990+0000 7fccc6400640 1 --2- 172.21.15.47:0/2572095466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccc0102140 0x7fccc0104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.990+0000 7fccc6400640 1 -- 172.21.15.47:0/2572095466 >> 172.21.15.47:0/2572095466 conn(0x7fccc00fbb30 msgr2=0x7fccc00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.990+0000 7fccc6400640 1 -- 172.21.15.47:0/2572095466 shutdown_connections 2024-03-21T11:20:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.990+0000 7fccc6400640 1 -- 172.21.15.47:0/2572095466 wait complete. 2024-03-21T11:20:54.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:54 smithi102 ceph-mon[36748]: pgmap v321: 129 pgs: 7 creating+peering, 25 unknown, 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2024-03-21T11:20:54.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:54 smithi102 ceph-mon[36748]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T11:20:54.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:54 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-21T11:20:54.993 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:54 smithi102 ceph-mon[36748]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T11:20:54.993 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:54 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/4030515883' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T11:20:54.993 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:54 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T11:20:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.991+0000 7fccc6400640 1 Processor -- start 2024-03-21T11:20:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.991+0000 7fccc6400640 1 -- start start 2024-03-21T11:20:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.991+0000 7fccc6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0102140 0x7fccc0199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.991+0000 7fccc6400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccc0104a70 0x7fccc019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.991+0000 7fccc6400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccc01073a0 0x7fccc01a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.991+0000 7fccc6400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc010d340 con 0x7fccc0104a70 2024-03-21T11:20:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.991+0000 7fccc6400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc010d1c0 con 0x7fccc0102140 2024-03-21T11:20:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.991+0000 7fccc6400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc010d4c0 con 0x7fccc01073a0 2024-03-21T11:20:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.992+0000 7fccbf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0102140 0x7fccc0199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.992+0000 7fccbf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0102140 0x7fccc0199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48354/0 (socket says 172.21.15.47:48354) 2024-03-21T11:20:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.992+0000 7fccbf400640 1 -- 172.21.15.47:0/973913159 learned_addr learned my addr 172.21.15.47:0/973913159 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.992+0000 7fccbea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccc0104a70 0x7fccc019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.992+0000 7fccbfe00640 1 --2- 172.21.15.47:0/973913159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccc01073a0 0x7fccc01a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.992+0000 7fccbf400640 1 -- 172.21.15.47:0/973913159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccc01073a0 msgr2=0x7fccc01a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.992+0000 7fccbf400640 1 --2- 172.21.15.47:0/973913159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccc01073a0 0x7fccc01a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.992+0000 7fccbf400640 1 -- 172.21.15.47:0/973913159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccc0104a70 msgr2=0x7fccc019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.992+0000 7fccbf400640 1 --2- 172.21.15.47:0/973913159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccc0104a70 0x7fccc019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.992+0000 7fccbf400640 1 -- 172.21.15.47:0/973913159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc01a1790 con 0x7fccc0102140 2024-03-21T11:20:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.992+0000 7fccbf400640 1 --2- 172.21.15.47:0/973913159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0102140 0x7fccc0199b70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fccb000e9f0 tx=0x7fccb000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.993+0000 7fccb7e00640 1 -- 172.21.15.47:0/973913159 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb000cd80 con 0x7fccc0102140 2024-03-21T11:20:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.993+0000 7fccb7e00640 1 -- 172.21.15.47:0/973913159 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccb0010910 con 0x7fccc0102140 2024-03-21T11:20:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.993+0000 7fccb7e00640 1 -- 172.21.15.47:0/973913159 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0018990 con 0x7fccc0102140 2024-03-21T11:20:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.993+0000 7fccc6400640 1 -- 172.21.15.47:0/973913159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc01a1a80 con 0x7fccc0102140 2024-03-21T11:20:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.993+0000 7fccc6400640 1 -- 172.21.15.47:0/973913159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc01a1f80 con 0x7fccc0102140 2024-03-21T11:20:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.994+0000 7fccc6400640 1 -- 172.21.15.47:0/973913159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc0068490 con 0x7fccc0102140 2024-03-21T11:20:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.995+0000 7fccb7e00640 1 -- 172.21.15.47:0/973913159 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fccb0010ac0 con 0x7fccc0102140 2024-03-21T11:20:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.997+0000 7fccb7e00640 1 --2- 172.21.15.47:0/973913159 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcca40a2fa0 0x7fcca40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.997+0000 7fccb7e00640 1 -- 172.21.15.47:0/973913159 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fccb00cf8f0 con 0x7fccc0102140 2024-03-21T11:20:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.997+0000 7fccbea00640 1 --2- 172.21.15.47:0/973913159 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcca40a2fa0 0x7fcca40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:54.997+0000 7fccbea00640 1 --2- 172.21.15.47:0/973913159 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcca40a2fa0 0x7fcca40a5460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fccc019b080 tx=0x7fccac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.000+0000 7fccb7e00640 1 -- 172.21.15.47:0/973913159 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fccb009be30 con 0x7fccc0102140 2024-03-21T11:20:55.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:54 smithi047 ceph-mon[31351]: pgmap v321: 129 pgs: 7 creating+peering, 25 unknown, 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2024-03-21T11:20:55.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:54 smithi047 ceph-mon[31351]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T11:20:55.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:54 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-21T11:20:55.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:54 smithi047 ceph-mon[31351]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T11:20:55.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:54 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/4030515883' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T11:20:55.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:54 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T11:20:55.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:54 smithi100 ceph-mon[36245]: pgmap v321: 129 pgs: 7 creating+peering, 25 unknown, 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2024-03-21T11:20:55.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:54 smithi100 ceph-mon[36245]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T11:20:55.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:54 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-21T11:20:55.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:54 smithi100 ceph-mon[36245]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T11:20:55.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:54 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/4030515883' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T11:20:55.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:54 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T11:20:55.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.236+0000 7fccc6400640 1 -- 172.21.15.47:0/973913159 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccc0108e90 con 0x7fccc0102140 2024-03-21T11:20:55.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.238+0000 7fccb7e00640 1 -- 172.21.15.47:0/973913159 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11336 (secure 0 0 0) 0x7fccc0108e90 con 0x7fccc0102140 2024-03-21T11:20:55.238 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:55.239 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:20:54.191693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-03-21T11:20:55.239 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2024-03-21T11:20:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.240+0000 7fccc6400640 1 -- 172.21.15.47:0/973913159 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcca40a2fa0 msgr2=0x7fcca40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.240+0000 7fccc6400640 1 --2- 172.21.15.47:0/973913159 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcca40a2fa0 0x7fcca40a5460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fccc019b080 tx=0x7fccac00b410 comp rx=0 tx=0).stop 2024-03-21T11:20:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.240+0000 7fccc6400640 1 -- 172.21.15.47:0/973913159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0102140 msgr2=0x7fccc0199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.240+0000 7fccc6400640 1 --2- 172.21.15.47:0/973913159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0102140 0x7fccc0199b70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fccb000e9f0 tx=0x7fccb000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.240+0000 7fccc6400640 1 -- 172.21.15.47:0/973913159 shutdown_connections 2024-03-21T11:20:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.240+0000 7fccc6400640 1 --2- 172.21.15.47:0/973913159 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcca40a2fa0 0x7fcca40a5460 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.240+0000 7fccc6400640 1 --2- 172.21.15.47:0/973913159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccc01073a0 0x7fccc01a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.240+0000 7fccc6400640 1 --2- 172.21.15.47:0/973913159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccc0104a70 0x7fccc019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.240+0000 7fccc6400640 1 --2- 172.21.15.47:0/973913159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccc0102140 0x7fccc0199b70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.240+0000 7fccc6400640 1 -- 172.21.15.47:0/973913159 >> 172.21.15.47:0/973913159 conn(0x7fccc00fbb30 msgr2=0x7fccc0101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.240+0000 7fccc6400640 1 -- 172.21.15.47:0/973913159 shutdown_connections 2024-03-21T11:20:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.240+0000 7fccc6400640 1 -- 172.21.15.47:0/973913159 wait complete. 2024-03-21T11:20:55.260 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:20:55.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.476+0000 7f4d61600640 1 -- 172.21.15.47:0/2310186875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5c101f70 msgr2=0x7f4d5c104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:55.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.476+0000 7f4d61600640 1 --2- 172.21.15.47:0/2310186875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5c101f70 0x7f4d5c104360 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009900 tx=0x7f4d4c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:55.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.476+0000 7f4d61600640 1 -- 172.21.15.47:0/2310186875 shutdown_connections 2024-03-21T11:20:55.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.476+0000 7f4d61600640 1 --2- 172.21.15.47:0/2310186875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d5c1071d0 0x7f4d5c109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.477+0000 7f4d61600640 1 --2- 172.21.15.47:0/2310186875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d5c1048a0 0x7f4d5c106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.477+0000 7f4d61600640 1 --2- 172.21.15.47:0/2310186875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5c101f70 0x7f4d5c104360 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.477+0000 7f4d61600640 1 -- 172.21.15.47:0/2310186875 >> 172.21.15.47:0/2310186875 conn(0x7f4d5c0fb980 msgr2=0x7f4d5c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:55.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.477+0000 7f4d61600640 1 -- 172.21.15.47:0/2310186875 shutdown_connections 2024-03-21T11:20:55.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.477+0000 7f4d61600640 1 -- 172.21.15.47:0/2310186875 wait complete. 2024-03-21T11:20:55.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.478+0000 7f4d61600640 1 Processor -- start 2024-03-21T11:20:55.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.478+0000 7f4d61600640 1 -- start start 2024-03-21T11:20:55.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.478+0000 7f4d61600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d5c101f70 0x7f4d5c1018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:55.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.478+0000 7f4d61600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5c1048a0 0x7f4d5c0fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:55.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.478+0000 7f4d61600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d5c1071d0 0x7f4d5c100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:55.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.478+0000 7f4d61600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c10d1d0 con 0x7f4d5c1071d0 2024-03-21T11:20:55.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.478+0000 7f4d61600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c10d050 con 0x7f4d5c1048a0 2024-03-21T11:20:55.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.478+0000 7f4d61600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c10d350 con 0x7f4d5c101f70 2024-03-21T11:20:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.478+0000 7f4d60c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d5c1071d0 0x7f4d5c100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.478+0000 7f4d5b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5c1048a0 0x7f4d5c0fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.478+0000 7f4d5b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5c1048a0 0x7f4d5c0fff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48380/0 (socket says 172.21.15.47:48380) 2024-03-21T11:20:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.478+0000 7f4d5b400640 1 -- 172.21.15.47:0/1558804051 learned_addr learned my addr 172.21.15.47:0/1558804051 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.478+0000 7f4d60c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d5c1071d0 0x7f4d5c100460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60726/0 (socket says 172.21.15.47:60726) 2024-03-21T11:20:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.478+0000 7f4d5be00640 1 --2- 172.21.15.47:0/1558804051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d5c101f70 0x7f4d5c1018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.479+0000 7f4d5b400640 1 -- 172.21.15.47:0/1558804051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d5c101f70 msgr2=0x7f4d5c1018e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.479+0000 7f4d5b400640 1 --2- 172.21.15.47:0/1558804051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d5c101f70 0x7f4d5c1018e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.479+0000 7f4d5b400640 1 -- 172.21.15.47:0/1558804051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d5c1071d0 msgr2=0x7f4d5c100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.479+0000 7f4d5b400640 1 --2- 172.21.15.47:0/1558804051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d5c1071d0 0x7f4d5c100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.479+0000 7f4d5b400640 1 -- 172.21.15.47:0/1558804051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c100cf0 con 0x7f4d5c1048a0 2024-03-21T11:20:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.479+0000 7f4d5b400640 1 --2- 172.21.15.47:0/1558804051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5c1048a0 0x7f4d5c0fff20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800e9c0 tx=0x7f4d4800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.479+0000 7f4d58c00640 1 -- 172.21.15.47:0/1558804051 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4800cd80 con 0x7f4d5c1048a0 2024-03-21T11:20:55.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.479+0000 7f4d58c00640 1 -- 172.21.15.47:0/1558804051 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d48005500 con 0x7f4d5c1048a0 2024-03-21T11:20:55.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.479+0000 7f4d61600640 1 -- 172.21.15.47:0/1558804051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5c1a1910 con 0x7f4d5c1048a0 2024-03-21T11:20:55.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.479+0000 7f4d58c00640 1 -- 172.21.15.47:0/1558804051 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48010690 con 0x7f4d5c1048a0 2024-03-21T11:20:55.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.479+0000 7f4d61600640 1 -- 172.21.15.47:0/1558804051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5c1a1eb0 con 0x7f4d5c1048a0 2024-03-21T11:20:55.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.480+0000 7f4d61600640 1 -- 172.21.15.47:0/1558804051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5c0fd470 con 0x7f4d5c1048a0 2024-03-21T11:20:55.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.483+0000 7f4d58c00640 1 -- 172.21.15.47:0/1558804051 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4d480046c0 con 0x7f4d5c1048a0 2024-03-21T11:20:55.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.485+0000 7f4d58c00640 1 --2- 172.21.15.47:0/1558804051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4d400a2fa0 0x7f4d400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:55.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.485+0000 7f4d58c00640 1 -- 172.21.15.47:0/1558804051 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4d480cfbd0 con 0x7f4d5c1048a0 2024-03-21T11:20:55.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.485+0000 7f4d5be00640 1 --2- 172.21.15.47:0/1558804051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4d400a2fa0 0x7f4d400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:55.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.485+0000 7f4d5be00640 1 --2- 172.21.15.47:0/1558804051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4d400a2fa0 0x7f4d400a5460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c02c240 tx=0x7f4d4c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:55.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.486+0000 7f4d58c00640 1 -- 172.21.15.47:0/1558804051 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4d48053740 con 0x7f4d5c1048a0 2024-03-21T11:20:55.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.702+0000 7f4d61600640 1 -- 172.21.15.47:0/1558804051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d5c02d2c0 con 0x7f4d5c1048a0 2024-03-21T11:20:55.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.704+0000 7f4d58c00640 1 -- 172.21.15.47:0/1558804051 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24127 (secure 0 0 0) 0x7f4d4809c760 con 0x7f4d5c1048a0 2024-03-21T11:20:55.705 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:55.705 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:20:54.191030+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:20:55.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.707+0000 7f4d61600640 1 -- 172.21.15.47:0/1558804051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4d400a2fa0 msgr2=0x7f4d400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:55.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.707+0000 7f4d61600640 1 --2- 172.21.15.47:0/1558804051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4d400a2fa0 0x7f4d400a5460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c02c240 tx=0x7f4d4c02df90 comp rx=0 tx=0).stop 2024-03-21T11:20:55.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.707+0000 7f4d61600640 1 -- 172.21.15.47:0/1558804051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5c1048a0 msgr2=0x7f4d5c0fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.707+0000 7f4d61600640 1 --2- 172.21.15.47:0/1558804051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5c1048a0 0x7f4d5c0fff20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800e9c0 tx=0x7f4d4800ee80 comp rx=0 tx=0).stop 2024-03-21T11:20:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.707+0000 7f4d61600640 1 -- 172.21.15.47:0/1558804051 shutdown_connections 2024-03-21T11:20:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.707+0000 7f4d61600640 1 --2- 172.21.15.47:0/1558804051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4d400a2fa0 0x7f4d400a5460 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.707+0000 7f4d61600640 1 --2- 172.21.15.47:0/1558804051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d5c1071d0 0x7f4d5c100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.707+0000 7f4d61600640 1 --2- 172.21.15.47:0/1558804051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5c1048a0 0x7f4d5c0fff20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.707+0000 7f4d61600640 1 --2- 172.21.15.47:0/1558804051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d5c101f70 0x7f4d5c1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.707+0000 7f4d61600640 1 -- 172.21.15.47:0/1558804051 >> 172.21.15.47:0/1558804051 conn(0x7f4d5c0fb980 msgr2=0x7f4d5c1028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.707+0000 7f4d61600640 1 -- 172.21.15.47:0/1558804051 shutdown_connections 2024-03-21T11:20:55.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.707+0000 7f4d61600640 1 -- 172.21.15.47:0/1558804051 wait complete. 2024-03-21T11:20:55.728 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:20:55.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.953+0000 7fa124400640 1 -- 172.21.15.47:0/225806519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c1038e0 msgr2=0x7fa11c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.953+0000 7fa124400640 1 --2- 172.21.15.47:0/225806519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c1038e0 0x7fa11c103d00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa110009900 tx=0x7fa11002bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.954+0000 7fa124400640 1 -- 172.21.15.47:0/225806519 shutdown_connections 2024-03-21T11:20:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.954+0000 7fa124400640 1 --2- 172.21.15.47:0/225806519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa11c105490 0x7fa11c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.954+0000 7fa124400640 1 --2- 172.21.15.47:0/225806519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa11c104ad0 0x7fa11c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.954+0000 7fa124400640 1 --2- 172.21.15.47:0/225806519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c1038e0 0x7fa11c103d00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.954+0000 7fa124400640 1 -- 172.21.15.47:0/225806519 >> 172.21.15.47:0/225806519 conn(0x7fa11c0fee50 msgr2=0x7fa11c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.954+0000 7fa124400640 1 -- 172.21.15.47:0/225806519 shutdown_connections 2024-03-21T11:20:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.954+0000 7fa124400640 1 -- 172.21.15.47:0/225806519 wait complete. 2024-03-21T11:20:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.955+0000 7fa124400640 1 Processor -- start 2024-03-21T11:20:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.955+0000 7fa124400640 1 -- start start 2024-03-21T11:20:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.955+0000 7fa124400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa11c1038e0 0x7fa11c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa124400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa11c104ad0 0x7fa11c19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa124400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c105490 0x7fa11c1a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa124400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c10f5a0 con 0x7fa11c1038e0 2024-03-21T11:20:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa124400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c10f420 con 0x7fa11c105490 2024-03-21T11:20:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa124400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c10f720 con 0x7fa11c104ad0 2024-03-21T11:20:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa121a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa11c1038e0 0x7fa11c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa121a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa11c1038e0 0x7fa11c199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60746/0 (socket says 172.21.15.47:60746) 2024-03-21T11:20:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa121a00640 1 -- 172.21.15.47:0/2215179313 learned_addr learned my addr 172.21.15.47:0/2215179313 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa121000640 1 --2- 172.21.15.47:0/2215179313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa11c104ad0 0x7fa11c19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa122400640 1 --2- 172.21.15.47:0/2215179313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c105490 0x7fa11c1a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa122400640 1 -- 172.21.15.47:0/2215179313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa11c104ad0 msgr2=0x7fa11c19a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa122400640 1 --2- 172.21.15.47:0/2215179313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa11c104ad0 0x7fa11c19a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa122400640 1 -- 172.21.15.47:0/2215179313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa11c1038e0 msgr2=0x7fa11c199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa122400640 1 --2- 172.21.15.47:0/2215179313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa11c1038e0 0x7fa11c199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.956+0000 7fa122400640 1 -- 172.21.15.47:0/2215179313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c1a18b0 con 0x7fa11c105490 2024-03-21T11:20:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.957+0000 7fa122400640 1 --2- 172.21.15.47:0/2215179313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c105490 0x7fa11c1a1210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa11400da10 tx=0x7fa11400ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.957+0000 7fa11a000640 1 -- 172.21.15.47:0/2215179313 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11400bd70 con 0x7fa11c105490 2024-03-21T11:20:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.957+0000 7fa11a000640 1 -- 172.21.15.47:0/2215179313 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa114003e50 con 0x7fa11c105490 2024-03-21T11:20:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.957+0000 7fa11a000640 1 -- 172.21.15.47:0/2215179313 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1140106a0 con 0x7fa11c105490 2024-03-21T11:20:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.957+0000 7fa124400640 1 -- 172.21.15.47:0/2215179313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c1a1ba0 con 0x7fa11c105490 2024-03-21T11:20:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.957+0000 7fa124400640 1 -- 172.21.15.47:0/2215179313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c1a20a0 con 0x7fa11c105490 2024-03-21T11:20:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.958+0000 7fa124400640 1 -- 172.21.15.47:0/2215179313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c10ac80 con 0x7fa11c105490 2024-03-21T11:20:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.960+0000 7fa11a000640 1 -- 172.21.15.47:0/2215179313 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa114003970 con 0x7fa11c105490 2024-03-21T11:20:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.961+0000 7fa11a000640 1 --2- 172.21.15.47:0/2215179313 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa1040a2fa0 0x7fa1040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.961+0000 7fa11a000640 1 -- 172.21.15.47:0/2215179313 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa1140cfa10 con 0x7fa11c105490 2024-03-21T11:20:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.961+0000 7fa121a00640 1 --2- 172.21.15.47:0/2215179313 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa1040a2fa0 0x7fa1040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.962+0000 7fa121a00640 1 --2- 172.21.15.47:0/2215179313 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa1040a2fa0 0x7fa1040a5460 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa11c19ab40 tx=0x7fa11002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:55.965+0000 7fa11a000640 1 -- 172.21.15.47:0/2215179313 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa11409c5a0 con 0x7fa11c105490 2024-03-21T11:20:55.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:55 smithi102 ceph-mon[36748]: osdmap e91: 12 total, 12 up, 12 in 2024-03-21T11:20:55.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:55 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-21T11:20:55.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:55 smithi102 ceph-mon[36748]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:20:55.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:55 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/973913159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:55.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:55 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1558804051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:56.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:55 smithi047 ceph-mon[31351]: osdmap e91: 12 total, 12 up, 12 in 2024-03-21T11:20:56.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:55 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-21T11:20:56.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:55 smithi047 ceph-mon[31351]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:20:56.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:55 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/973913159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:56.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:55 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1558804051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:56.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:55 smithi100 ceph-mon[36245]: osdmap e91: 12 total, 12 up, 12 in 2024-03-21T11:20:56.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:55 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-21T11:20:56.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:55 smithi100 ceph-mon[36245]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:20:56.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:55 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/973913159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:56.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:55 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1558804051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.172+0000 7fa124400640 1 -- 172.21.15.47:0/2215179313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa11c0622a0 con 0x7fa11c105490 2024-03-21T11:20:56.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.174+0000 7fa11a000640 1 -- 172.21.15.47:0/2215179313 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24127 (secure 0 0 0) 0x7fa11409bea0 con 0x7fa11c105490 2024-03-21T11:20:56.176 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:56.176 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:20:54.191030+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:20:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.177+0000 7fa124400640 1 -- 172.21.15.47:0/2215179313 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa1040a2fa0 msgr2=0x7fa1040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.177+0000 7fa124400640 1 --2- 172.21.15.47:0/2215179313 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa1040a2fa0 0x7fa1040a5460 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa11c19ab40 tx=0x7fa11002df90 comp rx=0 tx=0).stop 2024-03-21T11:20:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.177+0000 7fa124400640 1 -- 172.21.15.47:0/2215179313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c105490 msgr2=0x7fa11c1a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.177+0000 7fa124400640 1 --2- 172.21.15.47:0/2215179313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c105490 0x7fa11c1a1210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa11400da10 tx=0x7fa11400ded0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.177+0000 7fa124400640 1 -- 172.21.15.47:0/2215179313 shutdown_connections 2024-03-21T11:20:56.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.177+0000 7fa124400640 1 --2- 172.21.15.47:0/2215179313 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa1040a2fa0 0x7fa1040a5460 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.177+0000 7fa124400640 1 --2- 172.21.15.47:0/2215179313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c105490 0x7fa11c1a1210 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.177+0000 7fa124400640 1 --2- 172.21.15.47:0/2215179313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa11c104ad0 0x7fa11c19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.177+0000 7fa124400640 1 --2- 172.21.15.47:0/2215179313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa11c1038e0 0x7fa11c199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.177+0000 7fa124400640 1 -- 172.21.15.47:0/2215179313 >> 172.21.15.47:0/2215179313 conn(0x7fa11c0fee50 msgr2=0x7fa11c107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:56.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.178+0000 7fa124400640 1 -- 172.21.15.47:0/2215179313 shutdown_connections 2024-03-21T11:20:56.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.178+0000 7fa124400640 1 -- 172.21.15.47:0/2215179313 wait complete. 2024-03-21T11:20:56.215 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:20:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.447+0000 7efdd0200640 1 -- 172.21.15.47:0/3258261053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdc81038e0 msgr2=0x7efdc8103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.447+0000 7efdd0200640 1 --2- 172.21.15.47:0/3258261053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdc81038e0 0x7efdc8103d00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009ca0 tx=0x7efdbc02c100 comp rx=0 tx=0).stop 2024-03-21T11:20:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.447+0000 7efdd0200640 1 -- 172.21.15.47:0/3258261053 shutdown_connections 2024-03-21T11:20:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.447+0000 7efdd0200640 1 --2- 172.21.15.47:0/3258261053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc8105490 0x7efdc810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.447+0000 7efdd0200640 1 --2- 172.21.15.47:0/3258261053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdc8104ad0 0x7efdc8104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.447+0000 7efdd0200640 1 --2- 172.21.15.47:0/3258261053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdc81038e0 0x7efdc8103d00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.447+0000 7efdd0200640 1 -- 172.21.15.47:0/3258261053 >> 172.21.15.47:0/3258261053 conn(0x7efdc80fee90 msgr2=0x7efdc81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.447+0000 7efdd0200640 1 -- 172.21.15.47:0/3258261053 shutdown_connections 2024-03-21T11:20:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.447+0000 7efdd0200640 1 -- 172.21.15.47:0/3258261053 wait complete. 2024-03-21T11:20:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.449+0000 7efdd0200640 1 Processor -- start 2024-03-21T11:20:56.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.449+0000 7efdd0200640 1 -- start start 2024-03-21T11:20:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.449+0000 7efdd0200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc81038e0 0x7efdc8199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.449+0000 7efdd0200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdc8104ad0 0x7efdc819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.449+0000 7efdd0200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdc8105490 0x7efdc81a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.449+0000 7efdd0200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc810f560 con 0x7efdc8104ad0 2024-03-21T11:20:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.449+0000 7efdd0200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc810f3e0 con 0x7efdc81038e0 2024-03-21T11:20:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.449+0000 7efdd0200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc810f6e0 con 0x7efdc8105490 2024-03-21T11:20:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.450+0000 7efdcd800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc81038e0 0x7efdc8199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.450+0000 7efdcd800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc81038e0 0x7efdc8199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48408/0 (socket says 172.21.15.47:48408) 2024-03-21T11:20:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.450+0000 7efdcd800640 1 -- 172.21.15.47:0/1509145549 learned_addr learned my addr 172.21.15.47:0/1509145549 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.450+0000 7efdcce00640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdc8104ad0 0x7efdc819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.450+0000 7efdce200640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdc8105490 0x7efdc81a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.450+0000 7efdcd800640 1 -- 172.21.15.47:0/1509145549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdc8105490 msgr2=0x7efdc81a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.450+0000 7efdcd800640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdc8105490 0x7efdc81a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.450+0000 7efdcd800640 1 -- 172.21.15.47:0/1509145549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdc8104ad0 msgr2=0x7efdc819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.450+0000 7efdcd800640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdc8104ad0 0x7efdc819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.450+0000 7efdcd800640 1 -- 172.21.15.47:0/1509145549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc81a17c0 con 0x7efdc81038e0 2024-03-21T11:20:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.450+0000 7efdcce00640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdc8104ad0 0x7efdc819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:20:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.450+0000 7efdce200640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdc8105490 0x7efdc81a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:20:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.450+0000 7efdcd800640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc81038e0 0x7efdc8199ba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0324c0 tx=0x7efdbc035d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.450+0000 7efdc6000640 1 -- 172.21.15.47:0/1509145549 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc039040 con 0x7efdc81038e0 2024-03-21T11:20:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.451+0000 7efdc6000640 1 -- 172.21.15.47:0/1509145549 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efdbc044070 con 0x7efdc81038e0 2024-03-21T11:20:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.451+0000 7efdd0200640 1 -- 172.21.15.47:0/1509145549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc81a1a50 con 0x7efdc81038e0 2024-03-21T11:20:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.451+0000 7efdc6000640 1 -- 172.21.15.47:0/1509145549 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc0044a0 con 0x7efdc81038e0 2024-03-21T11:20:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.451+0000 7efdd0200640 1 -- 172.21.15.47:0/1509145549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc81a1f90 con 0x7efdc81038e0 2024-03-21T11:20:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.452+0000 7efdd0200640 1 -- 172.21.15.47:0/1509145549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc810aba0 con 0x7efdc81038e0 2024-03-21T11:20:56.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.454+0000 7efdc6000640 1 -- 172.21.15.47:0/1509145549 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7efdbc004640 con 0x7efdc81038e0 2024-03-21T11:20:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.455+0000 7efdc6000640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efdb00a2fa0 0x7efdb00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.455+0000 7efdc6000640 1 -- 172.21.15.47:0/1509145549 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efdbc0f1dd0 con 0x7efdc81038e0 2024-03-21T11:20:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.455+0000 7efdcce00640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efdb00a2fa0 0x7efdb00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.456+0000 7efdcce00640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efdb00a2fa0 0x7efdb00a5460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efdb4005d00 tx=0x7efdb4005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.459+0000 7efdc6000640 1 -- 172.21.15.47:0/1509145549 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efdbc0be960 con 0x7efdc81038e0 2024-03-21T11:20:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.710+0000 7efdd0200640 1 -- 172.21.15.47:0/1509145549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdc802d070 con 0x7efdc81038e0 2024-03-21T11:20:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.711+0000 7efdc6000640 1 -- 172.21.15.47:0/1509145549 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11336 (secure 0 0 0) 0x7efdc802d070 con 0x7efdc81038e0 2024-03-21T11:20:56.712 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:56.712 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:20:54.191693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-03-21T11:20:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2024-03-21T11:20:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.714+0000 7efdd0200640 1 -- 172.21.15.47:0/1509145549 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efdb00a2fa0 msgr2=0x7efdb00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.714+0000 7efdd0200640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efdb00a2fa0 0x7efdb00a5460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efdb4005d00 tx=0x7efdb4005c70 comp rx=0 tx=0).stop 2024-03-21T11:20:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.714+0000 7efdd0200640 1 -- 172.21.15.47:0/1509145549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc81038e0 msgr2=0x7efdc8199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.714+0000 7efdd0200640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc81038e0 0x7efdc8199ba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0324c0 tx=0x7efdbc035d90 comp rx=0 tx=0).stop 2024-03-21T11:20:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.714+0000 7efdd0200640 1 -- 172.21.15.47:0/1509145549 shutdown_connections 2024-03-21T11:20:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.714+0000 7efdd0200640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efdb00a2fa0 0x7efdb00a5460 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.714+0000 7efdd0200640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdc8105490 0x7efdc81a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.714+0000 7efdd0200640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdc8104ad0 0x7efdc819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.714+0000 7efdd0200640 1 --2- 172.21.15.47:0/1509145549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdc81038e0 0x7efdc8199ba0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.714+0000 7efdd0200640 1 -- 172.21.15.47:0/1509145549 >> 172.21.15.47:0/1509145549 conn(0x7efdc80fee90 msgr2=0x7efdc8107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.715+0000 7efdd0200640 1 -- 172.21.15.47:0/1509145549 shutdown_connections 2024-03-21T11:20:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.715+0000 7efdd0200640 1 -- 172.21.15.47:0/1509145549 wait complete. 2024-03-21T11:20:56.734 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:20:56.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.958+0000 7fb12fe00640 1 -- 172.21.15.47:0/1244477690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb130108cf0 msgr2=0x7fb130109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:56.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.958+0000 7fb12fe00640 1 --2- 172.21.15.47:0/1244477690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb130108cf0 0x7fb130109170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb118009900 tx=0x7fb11802bc30 comp rx=0 tx=0).stop 2024-03-21T11:20:56.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.958+0000 7fb12fe00640 1 -- 172.21.15.47:0/1244477690 shutdown_connections 2024-03-21T11:20:56.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.958+0000 7fb12fe00640 1 --2- 172.21.15.47:0/1244477690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1301096b0 0x7fb13010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.958+0000 7fb12fe00640 1 --2- 172.21.15.47:0/1244477690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb130108cf0 0x7fb130109170 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.958+0000 7fb12fe00640 1 --2- 172.21.15.47:0/1244477690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb130107b00 0x7fb130107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.958+0000 7fb12fe00640 1 -- 172.21.15.47:0/1244477690 >> 172.21.15.47:0/1244477690 conn(0x7fb130076ab0 msgr2=0x7fb130078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:56.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.958+0000 7fb12fe00640 1 -- 172.21.15.47:0/1244477690 shutdown_connections 2024-03-21T11:20:56.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.958+0000 7fb12fe00640 1 -- 172.21.15.47:0/1244477690 wait complete. 2024-03-21T11:20:56.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.959+0000 7fb12fe00640 1 Processor -- start 2024-03-21T11:20:56.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.959+0000 7fb12fe00640 1 -- start start 2024-03-21T11:20:56.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.960+0000 7fb12fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb130107b00 0x7fb13019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:56.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.960+0000 7fb12fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb130108cf0 0x7fb13019e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:56.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.960+0000 7fb12fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1301096b0 0x7fb1301a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:56.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.960+0000 7fb12fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1301137b0 con 0x7fb130108cf0 2024-03-21T11:20:56.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.960+0000 7fb12fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb130113630 con 0x7fb1301096b0 2024-03-21T11:20:56.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.960+0000 7fb12fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb130113930 con 0x7fb130107b00 2024-03-21T11:20:56.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.960+0000 7fb127e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb130108cf0 0x7fb13019e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:56.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.960+0000 7fb12f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1301096b0 0x7fb1301a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:56.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.960+0000 7fb12f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1301096b0 0x7fb1301a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48422/0 (socket says 172.21.15.47:48422) 2024-03-21T11:20:56.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.960+0000 7fb12f400640 1 -- 172.21.15.47:0/695948391 learned_addr learned my addr 172.21.15.47:0/695948391 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:56.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.960+0000 7fb127e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb130108cf0 0x7fb13019e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60778/0 (socket says 172.21.15.47:60778) 2024-03-21T11:20:56.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.960+0000 7fb12ea00640 1 --2- 172.21.15.47:0/695948391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb130107b00 0x7fb13019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:56.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.961+0000 7fb12f400640 1 -- 172.21.15.47:0/695948391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb130107b00 msgr2=0x7fb13019de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:56.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.961+0000 7fb12f400640 1 --2- 172.21.15.47:0/695948391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb130107b00 0x7fb13019de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.961+0000 7fb12f400640 1 -- 172.21.15.47:0/695948391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb130108cf0 msgr2=0x7fb13019e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:56.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.961+0000 7fb12f400640 1 --2- 172.21.15.47:0/695948391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb130108cf0 0x7fb13019e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:56.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.961+0000 7fb12f400640 1 -- 172.21.15.47:0/695948391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1301a5a40 con 0x7fb1301096b0 2024-03-21T11:20:56.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.961+0000 7fb12ea00640 1 --2- 172.21.15.47:0/695948391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb130107b00 0x7fb13019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:20:56.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.961+0000 7fb12f400640 1 --2- 172.21.15.47:0/695948391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1301096b0 0x7fb1301a53d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb128009fd0 tx=0x7fb12800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:56.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.961+0000 7fb134a00640 1 -- 172.21.15.47:0/695948391 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128017070 con 0x7fb1301096b0 2024-03-21T11:20:56.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.961+0000 7fb134a00640 1 -- 172.21.15.47:0/695948391 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb128003e50 con 0x7fb1301096b0 2024-03-21T11:20:56.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.961+0000 7fb12fe00640 1 -- 172.21.15.47:0/695948391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1301a5d30 con 0x7fb1301096b0 2024-03-21T11:20:56.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.961+0000 7fb134a00640 1 -- 172.21.15.47:0/695948391 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1280106c0 con 0x7fb1301096b0 2024-03-21T11:20:56.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.961+0000 7fb12fe00640 1 -- 172.21.15.47:0/695948391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1301a61f0 con 0x7fb1301096b0 2024-03-21T11:20:56.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.962+0000 7fb12fe00640 1 -- 172.21.15.47:0/695948391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13010ee70 con 0x7fb1301096b0 2024-03-21T11:20:56.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.964+0000 7fb134a00640 1 -- 172.21.15.47:0/695948391 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb128003970 con 0x7fb1301096b0 2024-03-21T11:20:56.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.965+0000 7fb134a00640 1 --2- 172.21.15.47:0/695948391 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1140a2fa0 0x7fb1140a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:56.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.965+0000 7fb134a00640 1 -- 172.21.15.47:0/695948391 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb1280cf3d0 con 0x7fb1301096b0 2024-03-21T11:20:56.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.966+0000 7fb12ea00640 1 --2- 172.21.15.47:0/695948391 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1140a2fa0 0x7fb1140a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:56.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.966+0000 7fb12ea00640 1 --2- 172.21.15.47:0/695948391 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1140a2fa0 0x7fb1140a5460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb130108b50 tx=0x7fb120005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:56.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:56.970+0000 7fb134a00640 1 -- 172.21.15.47:0/695948391 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb12809bf60 con 0x7fb1301096b0 2024-03-21T11:20:56.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:56 smithi102 ceph-mon[36748]: pgmap v324: 129 pgs: 7 creating+peering, 19 unknown, 103 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-21T11:20:56.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:56 smithi102 ceph-mon[36748]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T11:20:56.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:56 smithi102 ceph-mon[36748]: Cluster is now healthy 2024-03-21T11:20:56.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:56 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2215179313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:56.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:56 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1509145549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:56 smithi047 ceph-mon[31351]: pgmap v324: 129 pgs: 7 creating+peering, 19 unknown, 103 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-21T11:20:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:56 smithi047 ceph-mon[31351]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T11:20:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:56 smithi047 ceph-mon[31351]: Cluster is now healthy 2024-03-21T11:20:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:56 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2215179313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:56 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1509145549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:57.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:56 smithi100 ceph-mon[36245]: pgmap v324: 129 pgs: 7 creating+peering, 19 unknown, 103 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-21T11:20:57.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:56 smithi100 ceph-mon[36245]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T11:20:57.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:56 smithi100 ceph-mon[36245]: Cluster is now healthy 2024-03-21T11:20:57.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:56 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2215179313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:57.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:56 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1509145549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:20:57.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.173+0000 7fb12fe00640 1 -- 172.21.15.47:0/695948391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1300622a0 con 0x7fb1301096b0 2024-03-21T11:20:57.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.175+0000 7fb134a00640 1 -- 172.21.15.47:0/695948391 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24127 (secure 0 0 0) 0x7fb12809b860 con 0x7fb1301096b0 2024-03-21T11:20:57.176 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:20:57.177 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:20:54.191030+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:20:57.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.178+0000 7fb12fe00640 1 -- 172.21.15.47:0/695948391 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1140a2fa0 msgr2=0x7fb1140a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:57.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.178+0000 7fb12fe00640 1 --2- 172.21.15.47:0/695948391 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1140a2fa0 0x7fb1140a5460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb130108b50 tx=0x7fb120005f90 comp rx=0 tx=0).stop 2024-03-21T11:20:57.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.178+0000 7fb12fe00640 1 -- 172.21.15.47:0/695948391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1301096b0 msgr2=0x7fb1301a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:57.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.178+0000 7fb12fe00640 1 --2- 172.21.15.47:0/695948391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1301096b0 0x7fb1301a53d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb128009fd0 tx=0x7fb12800b260 comp rx=0 tx=0).stop 2024-03-21T11:20:57.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.178+0000 7fb12fe00640 1 -- 172.21.15.47:0/695948391 shutdown_connections 2024-03-21T11:20:57.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.178+0000 7fb12fe00640 1 --2- 172.21.15.47:0/695948391 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1140a2fa0 0x7fb1140a5460 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.178+0000 7fb12fe00640 1 --2- 172.21.15.47:0/695948391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1301096b0 0x7fb1301a53d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.178+0000 7fb12fe00640 1 --2- 172.21.15.47:0/695948391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb130108cf0 0x7fb13019e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.178+0000 7fb12fe00640 1 --2- 172.21.15.47:0/695948391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb130107b00 0x7fb13019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.178+0000 7fb12fe00640 1 -- 172.21.15.47:0/695948391 >> 172.21.15.47:0/695948391 conn(0x7fb130076ab0 msgr2=0x7fb13010c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:57.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.178+0000 7fb12fe00640 1 -- 172.21.15.47:0/695948391 shutdown_connections 2024-03-21T11:20:57.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.178+0000 7fb12fe00640 1 -- 172.21.15.47:0/695948391 wait complete. 2024-03-21T11:20:57.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-21T11:20:48.626416+0000', 'modified': '2024-03-21T11:20:54.191693+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24581}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6839/2708345792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 2708345792}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 2708345792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24581, 'qdb_cluster': [24581]} 2024-03-21T11:20:57.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-21T11:20:57.198 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-21T11:20:57.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.427+0000 7f3158c00640 1 -- 172.21.15.47:0/1282435506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3154074100 msgr2=0x7f3154074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:57.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.427+0000 7f3158c00640 1 --2- 172.21.15.47:0/1282435506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3154074100 0x7f3154074580 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f313c009930 tx=0x7f313c02bc20 comp rx=0 tx=0).stop 2024-03-21T11:20:57.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.427+0000 7f3158c00640 1 -- 172.21.15.47:0/1282435506 shutdown_connections 2024-03-21T11:20:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.427+0000 7f3158c00640 1 --2- 172.21.15.47:0/1282435506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f315410db70 0x7f315410ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.427+0000 7f3158c00640 1 --2- 172.21.15.47:0/1282435506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3154074100 0x7f3154074580 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.427+0000 7f3158c00640 1 --2- 172.21.15.47:0/1282435506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31540756e0 0x7f3154073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.427+0000 7f3158c00640 1 -- 172.21.15.47:0/1282435506 >> 172.21.15.47:0/1282435506 conn(0x7f31540fcb10 msgr2=0x7f31540fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.427+0000 7f3158c00640 1 -- 172.21.15.47:0/1282435506 shutdown_connections 2024-03-21T11:20:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.427+0000 7f3158c00640 1 -- 172.21.15.47:0/1282435506 wait complete. 2024-03-21T11:20:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.428+0000 7f3158c00640 1 Processor -- start 2024-03-21T11:20:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.428+0000 7f3158c00640 1 -- start start 2024-03-21T11:20:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.429+0000 7f3158c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3154074100 0x7f315419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.429+0000 7f3158c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31540756e0 0x7f315419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.429+0000 7f3158c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f315410db70 0x7f31541a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.429+0000 7f3158c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3154113840 con 0x7f3154074100 2024-03-21T11:20:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.429+0000 7f3158c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31541136c0 con 0x7f315410db70 2024-03-21T11:20:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.429+0000 7f3158c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31541139c0 con 0x7f31540756e0 2024-03-21T11:20:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.429+0000 7f3153400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3154074100 0x7f315419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.429+0000 7f3153e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f315410db70 0x7f31541a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.429+0000 7f3153400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3154074100 0x7f315419df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60798/0 (socket says 172.21.15.47:60798) 2024-03-21T11:20:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.429+0000 7f3153400640 1 -- 172.21.15.47:0/4212346521 learned_addr learned my addr 172.21.15.47:0/4212346521 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.429+0000 7f3152a00640 1 --2- 172.21.15.47:0/4212346521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31540756e0 0x7f315419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.430+0000 7f3153e00640 1 -- 172.21.15.47:0/4212346521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31540756e0 msgr2=0x7f315419e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.430+0000 7f3153e00640 1 --2- 172.21.15.47:0/4212346521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31540756e0 0x7f315419e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.430+0000 7f3153e00640 1 -- 172.21.15.47:0/4212346521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3154074100 msgr2=0x7f315419df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.430+0000 7f3153e00640 1 --2- 172.21.15.47:0/4212346521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3154074100 0x7f315419df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.430+0000 7f3153e00640 1 -- 172.21.15.47:0/4212346521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31541a5b50 con 0x7f315410db70 2024-03-21T11:20:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.430+0000 7f3152a00640 1 --2- 172.21.15.47:0/4212346521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31540756e0 0x7f315419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:20:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.430+0000 7f3153e00640 1 --2- 172.21.15.47:0/4212346521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f315410db70 0x7f31541a54e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f314400ca30 tx=0x7f314400cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.430+0000 7f314be00640 1 -- 172.21.15.47:0/4212346521 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3144007d90 con 0x7f315410db70 2024-03-21T11:20:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.430+0000 7f314be00640 1 -- 172.21.15.47:0/4212346521 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3144005020 con 0x7f315410db70 2024-03-21T11:20:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.430+0000 7f314be00640 1 -- 172.21.15.47:0/4212346521 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314400f780 con 0x7f315410db70 2024-03-21T11:20:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.430+0000 7f3158c00640 1 -- 172.21.15.47:0/4212346521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31541a5e40 con 0x7f315410db70 2024-03-21T11:20:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.431+0000 7f3158c00640 1 -- 172.21.15.47:0/4212346521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31541a62b0 con 0x7f315410db70 2024-03-21T11:20:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.434+0000 7f314be00640 1 -- 172.21.15.47:0/4212346521 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f31440051d0 con 0x7f315410db70 2024-03-21T11:20:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.434+0000 7f315cf8c740 1 Processor -- start 2024-03-21T11:20:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.434+0000 7f315cf8c740 1 -- start start 2024-03-21T11:20:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.434+0000 7f315cf8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e065d80 0x556d3e0661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.434+0000 7f315cf8c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e066770 0x556d3e06b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.434+0000 7f315cf8c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 0x556d3e06dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.434+0000 7f315cf8c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3dfcac90 con 0x556d3e06b8e0 2024-03-21T11:20:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.434+0000 7f315cf8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3df73980 con 0x556d3e065d80 2024-03-21T11:20:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.434+0000 7f315cf8c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3dfc6d30 con 0x556d3e066770 2024-03-21T11:20:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.435+0000 7f3152a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e065d80 0x556d3e0661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.435+0000 7f3152a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e065d80 0x556d3e0661a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48454/0 (socket says 172.21.15.47:48454) 2024-03-21T11:20:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.435+0000 7f3153400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e066770 0x556d3e06b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.435+0000 7f3152a00640 1 -- 172.21.15.47:0/1853600274 learned_addr learned my addr 172.21.15.47:0/1853600274 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.435+0000 7f3152a00640 1 --2- 172.21.15.47:0/1853600274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 0x556d3e06dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.435+0000 7f3153400640 1 -- 172.21.15.47:0/1853600274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e065d80 msgr2=0x556d3e0661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:20:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.435+0000 7f3153400640 1 --2- 172.21.15.47:0/1853600274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e065d80 0x556d3e0661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.435+0000 7f3153400640 1 -- 172.21.15.47:0/1853600274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 msgr2=0x556d3e06dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:20:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.435+0000 7f3153400640 1 --2- 172.21.15.47:0/1853600274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 0x556d3e06dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.435+0000 7f3153400640 1 -- 172.21.15.47:0/1853600274 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d3df8b510 con 0x556d3e066770 2024-03-21T11:20:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.435+0000 7f314be00640 1 --2- 172.21.15.47:0/4212346521 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31380a2fa0 0x7f31380a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.435+0000 7f3153400640 1 --2- 172.21.15.47:0/1853600274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e066770 0x556d3e06b270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f314c004610 tx=0x7f314c00bc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ede71673854d5e0d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.436+0000 7f314be00640 1 -- 172.21.15.47:0/4212346521 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f31440cf810 con 0x7f315410db70 2024-03-21T11:20:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.436+0000 7f3143e00640 1 -- 172.21.15.47:0/1853600274 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c003dd0 con 0x556d3e066770 2024-03-21T11:20:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.436+0000 7f3143e00640 1 -- 172.21.15.47:0/1853600274 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f314c015070 con 0x556d3e066770 2024-03-21T11:20:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.436+0000 7f3152a00640 1 --2- 172.21.15.47:0/4212346521 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31380a2fa0 0x7f31380a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.436+0000 7f3143e00640 1 -- 172.21.15.47:0/1853600274 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c004e30 con 0x556d3e066770 2024-03-21T11:20:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.437+0000 7f315cf8c740 1 -- 172.21.15.47:0/1853600274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e066770 msgr2=0x556d3e06b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.437+0000 7f315cf8c740 1 --2- 172.21.15.47:0/1853600274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e066770 0x556d3e06b270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f314c004610 tx=0x7f314c00bc70 comp rx=0 tx=0).stop 2024-03-21T11:20:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.437+0000 7f3152a00640 1 --2- 172.21.15.47:0/4212346521 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31380a2fa0 0x7f31380a5460 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f313c005ec0 tx=0x7f313c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.437+0000 7f315cf8c740 1 -- 172.21.15.47:0/1853600274 shutdown_connections 2024-03-21T11:20:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.437+0000 7f315cf8c740 1 --2- 172.21.15.47:0/1853600274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 0x556d3e06dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.437+0000 7f315cf8c740 1 --2- 172.21.15.47:0/1853600274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e066770 0x556d3e06b270 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.437+0000 7f315cf8c740 1 --2- 172.21.15.47:0/1853600274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e065d80 0x556d3e0661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.437+0000 7f315cf8c740 1 -- 172.21.15.47:0/1853600274 >> 172.21.15.47:0/1853600274 conn(0x556d3e0612f0 msgr2=0x556d3e0616f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.437+0000 7f315cf8c740 1 -- 172.21.15.47:0/1853600274 shutdown_connections 2024-03-21T11:20:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.437+0000 7f315cf8c740 1 -- 172.21.15.47:0/1853600274 wait complete. 2024-03-21T11:20:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.437+0000 7f315cf8c740 1 -- start start 2024-03-21T11:20:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.438+0000 7f315cf8c740 1 Processor -- start 2024-03-21T11:20:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.438+0000 7f315cf8c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e065d80 0x556d3e0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.438+0000 7f315cf8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e066770 0x556d3e070800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.438+0000 7f315cf8c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e06b8e0 0x556d3e077840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.438+0000 7f315cf8c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3e06e890 con 0x556d3e065d80 2024-03-21T11:20:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.438+0000 7f315cf8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3df86dd0 con 0x556d3e066770 2024-03-21T11:20:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.438+0000 7f315cf8c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3e06ea10 con 0x556d3e06b8e0 2024-03-21T11:20:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.438+0000 7f315cf8c740 10 client.0 _ll_register_callbacks cb 0x556d3dcdeac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:20:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.438+0000 7f3152a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e066770 0x556d3e070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.438+0000 7f3153400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e065d80 0x556d3e0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.438+0000 7f3153e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e06b8e0 0x556d3e077840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.438+0000 7f3152a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e066770 0x556d3e070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48468/0 (socket says 172.21.15.47:48468) 2024-03-21T11:20:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.438+0000 7f3152a00640 1 -- 172.21.15.47:0/4224818927 learned_addr learned my addr 172.21.15.47:0/4224818927 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.439+0000 7f3152a00640 1 -- 172.21.15.47:0/4224818927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e06b8e0 msgr2=0x556d3e077840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:20:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.439+0000 7f3152a00640 1 --2- 172.21.15.47:0/4224818927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e06b8e0 0x556d3e077840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.439+0000 7f3152a00640 1 -- 172.21.15.47:0/4224818927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e065d80 msgr2=0x556d3e0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:20:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.439+0000 7f3152a00640 1 --2- 172.21.15.47:0/4224818927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e065d80 0x556d3e0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.439+0000 7f3152a00640 1 -- 172.21.15.47:0/4224818927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d3dcdf2d0 con 0x556d3e066770 2024-03-21T11:20:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.439+0000 7f3152a00640 1 --2- 172.21.15.47:0/4224818927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e066770 0x556d3e070800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f313c009930 tx=0x7f313c02bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14dd99576d8670d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.439+0000 7f3140c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:20:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.439+0000 7f3140c00640 1 -- 172.21.15.47:0/4224818927 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c052070 con 0x556d3e066770 2024-03-21T11:20:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.439+0000 7f3140c00640 1 -- 172.21.15.47:0/4224818927 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f313c004c10 con 0x556d3e066770 2024-03-21T11:20:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.439+0000 7f315cf8c740 1 -- 172.21.15.47:0/4224818927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f314c015070 con 0x556d3e066770 2024-03-21T11:20:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.440+0000 7f3140c00640 1 -- 172.21.15.47:0/4224818927 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f313c05e050 con 0x556d3e066770 2024-03-21T11:20:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.440+0000 7f315cf8c740 10 client.24689 fetch_fsmap learned FSMap version 17 2024-03-21T11:20:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.440+0000 7f315cf8c740 1 -- 172.21.15.47:0/4224818927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x556d3dda96b0 con 0x556d3e066770 2024-03-21T11:20:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.440+0000 7f3140c00640 1 -- 172.21.15.47:0/4224818927 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f314c015070 con 0x556d3e066770 2024-03-21T11:20:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.440+0000 7f3140c00640 1 -- 172.21.15.47:0/4224818927 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f313c04d070 con 0x556d3e066770 2024-03-21T11:20:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.440+0000 7f315cf8c740 10 client.24689 fetch_fsmap finished waiting for FSMap version 17 2024-03-21T11:20:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.440+0000 7f315cf8c740 10 client.24689 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T11:20:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.440+0000 7f315cf8c740 20 client.24689 populate_metadata read hostname 'smithi047' 2024-03-21T11:20:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.440+0000 7f315cf8c740 1 --2- 172.21.15.47:0/4224818927 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d3e07caa0 0x556d3e07bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.440+0000 7f315cf8c740 4 client.24689 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:20:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.440+0000 7f315cf8c740 1 -- 172.21.15.47:0/4224818927 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d3df6c520 con 0x556d3e07caa0 2024-03-21T11:20:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.440+0000 7f3153400640 1 --2- 172.21.15.47:0/4224818927 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d3e07caa0 0x556d3e07bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.441+0000 7f3153400640 1 --2- 172.21.15.47:0/4224818927 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d3e07caa0 0x556d3e07bad0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc99e2e28398efbe server_cookie=73fa5706a0e73a96 in_seq=0 out_seq=0 2024-03-21T11:20:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.441+0000 7f3140c00640 10 client.24689 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:20:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f3140c00640 1 -- 172.21.15.47:0/4224818927 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556d3df6c520 con 0x556d3e07caa0 2024-03-21T11:20:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f3140c00640 10 client.24689 handle_command_reply: tid=0 2024-03-21T11:20:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f315cf8c740 1 client.24689 shutdown 2024-03-21T11:20:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f315cf8c740 1 -- 172.21.15.47:0/4224818927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e066770 msgr2=0x556d3e070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f315cf8c740 1 --2- 172.21.15.47:0/4224818927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e066770 0x556d3e070800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f313c009930 tx=0x7f313c02bc20 comp rx=0 tx=0).stop 2024-03-21T11:20:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f315cf8c740 1 -- 172.21.15.47:0/4224818927 shutdown_connections 2024-03-21T11:20:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f315cf8c740 1 --2- 172.21.15.47:0/4224818927 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d3e07caa0 0x556d3e07bad0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f315cf8c740 1 --2- 172.21.15.47:0/4224818927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e06b8e0 0x556d3e077840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f315cf8c740 1 --2- 172.21.15.47:0/4224818927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e066770 0x556d3e070800 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f315cf8c740 1 --2- 172.21.15.47:0/4224818927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e065d80 0x556d3e0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f315cf8c740 1 -- 172.21.15.47:0/4224818927 >> 172.21.15.47:0/4224818927 conn(0x556d3e0612f0 msgr2=0x556d3e0604f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f3140c00640 0 client.24689 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:20:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f315cf8c740 1 -- 172.21.15.47:0/4224818927 shutdown_connections 2024-03-21T11:20:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f315cf8c740 1 -- 172.21.15.47:0/4224818927 wait complete. 2024-03-21T11:20:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.442+0000 7f315cf8c740 20 client.24689 trim_cache size 0 max 16384 2024-03-21T11:20:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.461+0000 7f315cf8c740 1 Processor -- start 2024-03-21T11:20:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.461+0000 7f315cf8c740 1 -- start start 2024-03-21T11:20:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.461+0000 7f315cf8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e065d80 0x556d3e067c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.461+0000 7f315cf8c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e066770 0x556d3e068150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.461+0000 7f315cf8c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 0x556d3e077800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.461+0000 7f315cf8c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3e07c2c0 con 0x556d3e06b8e0 2024-03-21T11:20:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.461+0000 7f315cf8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3e07c920 con 0x556d3e065d80 2024-03-21T11:20:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.461+0000 7f315cf8c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3e07cf80 con 0x556d3e066770 2024-03-21T11:20:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.461+0000 7f3153e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 0x556d3e077800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.461+0000 7f3153e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 0x556d3e077800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60828/0 (socket says 172.21.15.47:60828) 2024-03-21T11:20:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.461+0000 7f3153e00640 1 -- 172.21.15.47:0/2662559795 learned_addr learned my addr 172.21.15.47:0/2662559795 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.461+0000 7f3153400640 1 --2- 172.21.15.47:0/2662559795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e065d80 0x556d3e067c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.461+0000 7f3152a00640 1 --2- 172.21.15.47:0/2662559795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e066770 0x556d3e068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.462+0000 7f3153e00640 1 -- 172.21.15.47:0/2662559795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e066770 msgr2=0x556d3e068150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:20:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.462+0000 7f3153e00640 1 --2- 172.21.15.47:0/2662559795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e066770 0x556d3e068150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.462+0000 7f3153e00640 1 -- 172.21.15.47:0/2662559795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e065d80 msgr2=0x556d3e067c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:20:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.462+0000 7f3153e00640 1 --2- 172.21.15.47:0/2662559795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e065d80 0x556d3e067c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.462+0000 7f3153e00640 1 -- 172.21.15.47:0/2662559795 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d3e065b70 con 0x556d3e06b8e0 2024-03-21T11:20:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.462+0000 7f3153400640 1 --2- 172.21.15.47:0/2662559795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e065d80 0x556d3e067c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:20:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.462+0000 7f3153e00640 1 --2- 172.21.15.47:0/2662559795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 0x556d3e077800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f31440cd070 tx=0x7f314409bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff93a9eef0a84461 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.462+0000 7f3143400640 1 -- 172.21.15.47:0/2662559795 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314401d040 con 0x556d3e06b8e0 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.462+0000 7f3143400640 1 -- 172.21.15.47:0/2662559795 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3144099fa0 con 0x556d3e06b8e0 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.462+0000 7f3143400640 1 -- 172.21.15.47:0/2662559795 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314409b0f0 con 0x556d3e06b8e0 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.463+0000 7f315cf8c740 1 -- 172.21.15.47:0/2662559795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 msgr2=0x556d3e077800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.463+0000 7f315cf8c740 1 --2- 172.21.15.47:0/2662559795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 0x556d3e077800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f31440cd070 tx=0x7f314409bca0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.463+0000 7f315cf8c740 1 -- 172.21.15.47:0/2662559795 shutdown_connections 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.463+0000 7f315cf8c740 1 --2- 172.21.15.47:0/2662559795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 0x556d3e077800 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.463+0000 7f315cf8c740 1 --2- 172.21.15.47:0/2662559795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e066770 0x556d3e068150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.463+0000 7f315cf8c740 1 --2- 172.21.15.47:0/2662559795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e065d80 0x556d3e067c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.463+0000 7f315cf8c740 1 -- 172.21.15.47:0/2662559795 >> 172.21.15.47:0/2662559795 conn(0x556d3e0612f0 msgr2=0x556d3e0616d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.463+0000 7f315cf8c740 1 -- 172.21.15.47:0/2662559795 shutdown_connections 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.463+0000 7f315cf8c740 1 -- 172.21.15.47:0/2662559795 wait complete. 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.464+0000 7f315cf8c740 1 -- start start 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.464+0000 7f315cf8c740 1 Processor -- start 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.464+0000 7f315cf8c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e065d80 0x556d3e077480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.464+0000 7f315cf8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e066770 0x556d3e073f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.464+0000 7f315cf8c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 0x556d3e074460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.464+0000 7f315cf8c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3e07dc40 con 0x556d3e06b8e0 2024-03-21T11:20:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.464+0000 7f315cf8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3e07d5e0 con 0x556d3e066770 2024-03-21T11:20:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.464+0000 7f315cf8c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3e07e2a0 con 0x556d3e065d80 2024-03-21T11:20:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.464+0000 7f315cf8c740 10 client.0 _ll_register_callbacks cb 0x7f313c005400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:20:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3152a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e066770 0x556d3e073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3152a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e066770 0x556d3e073f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48486/0 (socket says 172.21.15.47:48486) 2024-03-21T11:20:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3153e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 0x556d3e074460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3152a00640 1 -- 172.21.15.47:0/2641959351 learned_addr learned my addr 172.21.15.47:0/2641959351 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:20:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3153400640 1 --2- 172.21.15.47:0/2641959351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e065d80 0x556d3e077480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3152a00640 1 -- 172.21.15.47:0/2641959351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e065d80 msgr2=0x556d3e077480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:20:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3152a00640 1 --2- 172.21.15.47:0/2641959351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e065d80 0x556d3e077480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3152a00640 1 -- 172.21.15.47:0/2641959351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 msgr2=0x556d3e074460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:20:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3152a00640 1 --2- 172.21.15.47:0/2641959351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 0x556d3e074460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3152a00640 1 -- 172.21.15.47:0/2641959351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d3e077ab0 con 0x556d3e066770 2024-03-21T11:20:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3153400640 1 --2- 172.21.15.47:0/2641959351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e065d80 0x556d3e077480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:20:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3152a00640 1 --2- 172.21.15.47:0/2641959351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e066770 0x556d3e073f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f313c02cea0 tx=0x7f313c02cd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e565456b21da964 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:20:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3140c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:20:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3140c00640 1 -- 172.21.15.47:0/2641959351 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c052040 con 0x556d3e066770 2024-03-21T11:20:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f3140c00640 1 -- 172.21.15.47:0/2641959351 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f313c043cc0 con 0x556d3e066770 2024-03-21T11:20:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.465+0000 7f315cf8c740 1 -- 172.21.15.47:0/2641959351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3144099fa0 con 0x556d3e066770 2024-03-21T11:20:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.466+0000 7f3140c00640 1 -- 172.21.15.47:0/2641959351 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f313c004a90 con 0x556d3e066770 2024-03-21T11:20:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.466+0000 7f315cf8c740 10 client.24695 fetch_fsmap learned FSMap version 17 2024-03-21T11:20:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.466+0000 7f315cf8c740 1 -- 172.21.15.47:0/2641959351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x556d3e0d8ae0 con 0x556d3e066770 2024-03-21T11:20:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.466+0000 7f3140c00640 1 -- 172.21.15.47:0/2641959351 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3144099fa0 con 0x556d3e066770 2024-03-21T11:20:57.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.466+0000 7f3140c00640 1 -- 172.21.15.47:0/2641959351 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f313c04d040 con 0x556d3e066770 2024-03-21T11:20:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.466+0000 7f315cf8c740 10 client.24695 fetch_fsmap finished waiting for FSMap version 17 2024-03-21T11:20:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.466+0000 7f315cf8c740 10 client.24695 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T11:20:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.466+0000 7f315cf8c740 20 client.24695 populate_metadata read hostname 'smithi047' 2024-03-21T11:20:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.466+0000 7f315cf8c740 1 --2- 172.21.15.47:0/2641959351 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d3e0dd120 0x556d3e0d8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:20:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.466+0000 7f315cf8c740 4 client.24695 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T11:20:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.466+0000 7f315cf8c740 1 -- 172.21.15.47:0/2641959351 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556d3dfd81d0 con 0x556d3e0dd120 2024-03-21T11:20:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.466+0000 7f3153400640 1 --2- 172.21.15.47:0/2641959351 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d3e0dd120 0x556d3e0d8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:20:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f3153400640 1 --2- 172.21.15.47:0/2641959351 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d3e0dd120 0x556d3e0d8d70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f72ce1bcea80c74f server_cookie=30bf006c546e5a2a in_seq=0 out_seq=0 2024-03-21T11:20:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f3140c00640 10 client.24695 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:20:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f3140c00640 1 -- 172.21.15.47:0/2641959351 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x556d3dfd81d0 con 0x556d3e0dd120 2024-03-21T11:20:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f3140c00640 10 client.24695 handle_command_reply: tid=0 2024-03-21T11:20:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f315cf8c740 1 client.24695 shutdown 2024-03-21T11:20:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f315cf8c740 1 -- 172.21.15.47:0/2641959351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e066770 msgr2=0x556d3e073f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f315cf8c740 1 --2- 172.21.15.47:0/2641959351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e066770 0x556d3e073f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f313c02cea0 tx=0x7f313c02cd20 comp rx=0 tx=0).stop 2024-03-21T11:20:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f315cf8c740 1 -- 172.21.15.47:0/2641959351 shutdown_connections 2024-03-21T11:20:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f315cf8c740 1 --2- 172.21.15.47:0/2641959351 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d3e0dd120 0x556d3e0d8d70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f315cf8c740 1 --2- 172.21.15.47:0/2641959351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3e06b8e0 0x556d3e074460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f3140c00640 0 client.24695 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:20:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f315cf8c740 1 --2- 172.21.15.47:0/2641959351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d3e066770 0x556d3e073f20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f315cf8c740 1 --2- 172.21.15.47:0/2641959351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d3e065d80 0x556d3e077480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f315cf8c740 1 -- 172.21.15.47:0/2641959351 >> 172.21.15.47:0/2641959351 conn(0x556d3e0612f0 msgr2=0x556d3e0702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f315cf8c740 1 -- 172.21.15.47:0/2641959351 shutdown_connections 2024-03-21T11:20:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.467+0000 7f315cf8c740 1 -- 172.21.15.47:0/2641959351 wait complete. 2024-03-21T11:20:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.468+0000 7f315cf8c740 20 client.24695 trim_cache size 0 max 16384 2024-03-21T11:20:57.476 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:20:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:20:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-21T11:20:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24581, 2024-03-21T11:20:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T11:20:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T11:20:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T11:20:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 7.8782995419999997, 2024-03-21T11:20:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 17, 2024-03-21T11:20:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 91, 2024-03-21T11:20:57.476 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-03-21T11:20:57.477 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 15.251367991 2024-03-21T11:20:57.477 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:20:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.468+0000 7f3158c00640 1 -- 172.21.15.47:0/4212346521 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31380a2fa0 msgr2=0x7f31380a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:57.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.468+0000 7f3158c00640 1 --2- 172.21.15.47:0/4212346521 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31380a2fa0 0x7f31380a5460 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f313c005ec0 tx=0x7f313c000bc0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.468+0000 7f3158c00640 1 -- 172.21.15.47:0/4212346521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f315410db70 msgr2=0x7f31541a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:20:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.468+0000 7f3158c00640 1 --2- 172.21.15.47:0/4212346521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f315410db70 0x7f31541a54e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f314400ca30 tx=0x7f314400cef0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.468+0000 7f3158c00640 1 -- 172.21.15.47:0/4212346521 shutdown_connections 2024-03-21T11:20:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.468+0000 7f3158c00640 1 --2- 172.21.15.47:0/4212346521 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31380a2fa0 0x7f31380a5460 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.468+0000 7f3158c00640 1 --2- 172.21.15.47:0/4212346521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f315410db70 0x7f31541a54e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.468+0000 7f3158c00640 1 --2- 172.21.15.47:0/4212346521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31540756e0 0x7f315419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.468+0000 7f3158c00640 1 --2- 172.21.15.47:0/4212346521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3154074100 0x7f315419df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:20:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.468+0000 7f3158c00640 1 -- 172.21.15.47:0/4212346521 >> 172.21.15.47:0/4212346521 conn(0x7f31540fcb10 msgr2=0x7f3154100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:20:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.468+0000 7f3158c00640 1 -- 172.21.15.47:0/4212346521 shutdown_connections 2024-03-21T11:20:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:20:57.468+0000 7f3158c00640 1 -- 172.21.15.47:0/4212346521 wait complete. 2024-03-21T11:20:57.486 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-21T11:20:57.486 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-21T11:20:57.486 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-21T11:20:57.945 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:20:58.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:57 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/695948391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:58.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:57 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/695948391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:58.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:57 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/695948391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:20:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:20:58 smithi047 ceph-mon[31351]: pgmap v325: 129 pgs: 7 creating+peering, 6 unknown, 116 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 7 op/s 2024-03-21T11:20:59.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:20:58 smithi102 ceph-mon[36748]: pgmap v325: 129 pgs: 7 creating+peering, 6 unknown, 116 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 7 op/s 2024-03-21T11:20:59.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:20:58 smithi100 ceph-mon[36245]: pgmap v325: 129 pgs: 7 creating+peering, 6 unknown, 116 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 7 op/s 2024-03-21T11:21:00.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.348+0000 7fdc20800700 1 -- 172.21.15.47:0/4196887212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18102bd0 msgr2=0x7fdc1810f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:00.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.348+0000 7fdc20800700 1 --2- 172.21.15.47:0/4196887212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18102bd0 0x7fdc1810f060 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400b0d0 tx=0x7fdc1400b3e0 comp rx=0 tx=0).stop 2024-03-21T11:21:00.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.349+0000 7fdc20800700 1 -- 172.21.15.47:0/4196887212 shutdown_connections 2024-03-21T11:21:00.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.349+0000 7fdc20800700 1 --2- 172.21.15.47:0/4196887212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18102bd0 0x7fdc1810f060 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:00.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.349+0000 7fdc20800700 1 --2- 172.21.15.47:0/4196887212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc18108260 0x7fdc18108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:00.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.349+0000 7fdc20800700 1 --2- 172.21.15.47:0/4196887212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc18102210 0x7fdc18102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:00.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.349+0000 7fdc20800700 1 -- 172.21.15.47:0/4196887212 >> 172.21.15.47:0/4196887212 conn(0x7fdc180792e0 msgr2=0x7fdc180796f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:00.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.349+0000 7fdc20800700 1 -- 172.21.15.47:0/4196887212 shutdown_connections 2024-03-21T11:21:00.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.349+0000 7fdc20800700 1 -- 172.21.15.47:0/4196887212 wait complete. 2024-03-21T11:21:00.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.350+0000 7fdc20800700 1 Processor -- start 2024-03-21T11:21:00.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.350+0000 7fdc20800700 1 -- start start 2024-03-21T11:21:00.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.350+0000 7fdc20800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18102210 0x7fdc181a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:00.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.350+0000 7fdc20800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc18102bd0 0x7fdc181a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:00.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc20800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc18108260 0x7fdc1819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:00.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc20800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc181124f0 con 0x7fdc18102bd0 2024-03-21T11:21:00.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc20800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc18112370 con 0x7fdc18102210 2024-03-21T11:21:00.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc20800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc18112670 con 0x7fdc18108260 2024-03-21T11:21:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc1de00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18102210 0x7fdc181a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc1de00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18102210 0x7fdc181a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45210/0 (socket says 172.21.15.47:45210) 2024-03-21T11:21:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc1de00700 1 -- 172.21.15.47:0/3806755699 learned_addr learned my addr 172.21.15.47:0/3806755699 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc1d400700 1 --2- 172.21.15.47:0/3806755699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc18102bd0 0x7fdc181a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc1e800700 1 --2- 172.21.15.47:0/3806755699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc18108260 0x7fdc1819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc1de00700 1 -- 172.21.15.47:0/3806755699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc18108260 msgr2=0x7fdc1819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc1de00700 1 --2- 172.21.15.47:0/3806755699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc18108260 0x7fdc1819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc1de00700 1 -- 172.21.15.47:0/3806755699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc18102bd0 msgr2=0x7fdc181a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc1de00700 1 --2- 172.21.15.47:0/3806755699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc18102bd0 0x7fdc181a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc1de00700 1 -- 172.21.15.47:0/3806755699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1819b080 con 0x7fdc18102210 2024-03-21T11:21:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.351+0000 7fdc1de00700 1 --2- 172.21.15.47:0/3806755699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18102210 0x7fdc181a0760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00e9c0 tx=0x7fdc0c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.352+0000 7fdc12a00700 1 -- 172.21.15.47:0/3806755699 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c00cbd0 con 0x7fdc18102210 2024-03-21T11:21:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.352+0000 7fdc12a00700 1 -- 172.21.15.47:0/3806755699 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc0c00cd70 con 0x7fdc18102210 2024-03-21T11:21:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.352+0000 7fdc12a00700 1 -- 172.21.15.47:0/3806755699 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c010430 con 0x7fdc18102210 2024-03-21T11:21:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.352+0000 7fdc20800700 1 -- 172.21.15.47:0/3806755699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1819b370 con 0x7fdc18102210 2024-03-21T11:21:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.352+0000 7fdc20800700 1 -- 172.21.15.47:0/3806755699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc181a7630 con 0x7fdc18102210 2024-03-21T11:21:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.353+0000 7fdc20800700 1 -- 172.21.15.47:0/3806755699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc18068e50 con 0x7fdc18102210 2024-03-21T11:21:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.355+0000 7fdc12a00700 1 -- 172.21.15.47:0/3806755699 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdc0c004750 con 0x7fdc18102210 2024-03-21T11:21:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.356+0000 7fdc12a00700 1 --2- 172.21.15.47:0/3806755699 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc000a3030 0x7fdc000a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.356+0000 7fdc12a00700 1 -- 172.21.15.47:0/3806755699 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fdc0c0cf840 con 0x7fdc18102210 2024-03-21T11:21:00.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.356+0000 7fdc1d400700 1 --2- 172.21.15.47:0/3806755699 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc000a3030 0x7fdc000a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:00.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.357+0000 7fdc1d400700 1 --2- 172.21.15.47:0/3806755699 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc000a3030 0x7fdc000a54f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdc18079b90 tx=0x7fdc08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.364+0000 7fdc12a00700 1 -- 172.21.15.47:0/3806755699 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdc0c09c3d0 con 0x7fdc18102210 2024-03-21T11:21:00.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.639+0000 7fdc20800700 1 -- 172.21.15.47:0/3806755699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdc181a7990 con 0x7fdc18102210 2024-03-21T11:21:00.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.639+0000 7fdc12a00700 1 -- 172.21.15.47:0/3806755699 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fdc0c09bcd0 con 0x7fdc18102210 2024-03-21T11:21:00.640 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-21T11:21:00.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.643+0000 7fdc20800700 1 -- 172.21.15.47:0/3806755699 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc000a3030 msgr2=0x7fdc000a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:00.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.643+0000 7fdc20800700 1 --2- 172.21.15.47:0/3806755699 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc000a3030 0x7fdc000a54f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdc18079b90 tx=0x7fdc08009260 comp rx=0 tx=0).stop 2024-03-21T11:21:00.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.643+0000 7fdc20800700 1 -- 172.21.15.47:0/3806755699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18102210 msgr2=0x7fdc181a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:00.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.643+0000 7fdc20800700 1 --2- 172.21.15.47:0/3806755699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18102210 0x7fdc181a0760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00e9c0 tx=0x7fdc0c00ecd0 comp rx=0 tx=0).stop 2024-03-21T11:21:00.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.643+0000 7fdc20800700 1 -- 172.21.15.47:0/3806755699 shutdown_connections 2024-03-21T11:21:00.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.643+0000 7fdc20800700 1 --2- 172.21.15.47:0/3806755699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc18108260 0x7fdc1819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:00.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.643+0000 7fdc20800700 1 --2- 172.21.15.47:0/3806755699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18102210 0x7fdc181a0760 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:00.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.643+0000 7fdc20800700 1 --2- 172.21.15.47:0/3806755699 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc000a3030 0x7fdc000a54f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:00.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.643+0000 7fdc20800700 1 --2- 172.21.15.47:0/3806755699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc18102bd0 0x7fdc181a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:00.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.643+0000 7fdc20800700 1 -- 172.21.15.47:0/3806755699 >> 172.21.15.47:0/3806755699 conn(0x7fdc180792e0 msgr2=0x7fdc1810d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:00.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.643+0000 7fdc20800700 1 -- 172.21.15.47:0/3806755699 shutdown_connections 2024-03-21T11:21:00.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:00.643+0000 7fdc20800700 1 -- 172.21.15.47:0/3806755699 wait complete. 2024-03-21T11:21:01.032 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:00 smithi047 ceph-mon[31351]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2024-03-21T11:21:01.032 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:00 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3806755699' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T11:21:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:00 smithi100 ceph-mon[36245]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2024-03-21T11:21:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:00 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3806755699' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T11:21:01.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:00 smithi102 ceph-mon[36748]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2024-03-21T11:21:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:00 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3806755699' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T11:21:01.499 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-21T11:21:01.499 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-21T11:21:01.500 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-21T11:21:01.500 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-21T11:21:01.500 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-21T11:21:01.500 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-21T11:21:01.500 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-21T11:21:01.500 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-21T11:21:01.500 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-21T11:21:01.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:01.500 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-21T11:21:01.560 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-21T11:21:01.560 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-21T11:21:01.560 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-21T11:21:01.560 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-21T11:21:01.561 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-21T11:21:01.561 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-21T11:21:01.561 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T11:21:01.561 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-21T11:21:01.561 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-21T11:21:01.561 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T11:21:01.561 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-21T11:21:01.561 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-21T11:21:01.561 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-21T11:21:01.561 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-21T11:21:01.562 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-21T11:21:01.562 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-21T11:21:01.562 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42083sec preferred_lft 42083sec 2024-03-21T11:21:01.562 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-21T11:21:01.562 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-21T11:21:01.562 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T11:21:01.562 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-21T11:21:01.562 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-21T11:21:01.564 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-21T11:21:01.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:01.564 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T11:21:01.564 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T11:21:01.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add name ceph-brx type bridge 2024-03-21T11:21:01.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr flush dev ceph-brx 2024-03-21T11:21:01.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set ceph-brx up 2024-03-21T11:21:01.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-21T11:21:01.564 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T11:21:02.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:02.026 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-21T11:21:02.114 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-21T11:21:02.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:02.117 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-21T11:21:02.176 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-21T11:21:02.176 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-21T11:21:02.176 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-21T11:21:02.176 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-21T11:21:02.176 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-21T11:21:02.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:02.177 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T11:21:02.177 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T11:21:02.177 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-21T11:21:02.177 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-21T11:21:02.177 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-21T11:21:02.177 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T11:21:02.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:02 smithi047 ceph-mon[31351]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 0 op/s 2024-03-21T11:21:02.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:02.897 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-21T11:21:02.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:02.957 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-03-21T11:21:03.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:03.014 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T11:21:03.014 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T11:21:03.014 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T11:21:03.014 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-21T11:21:03.014 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T11:21:03.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:02 smithi100 ceph-mon[36245]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 0 op/s 2024-03-21T11:21:03.171 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-21T11:21:03.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:03.171 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T11:21:03.171 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T11:21:03.171 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-21T11:21:03.171 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-21T11:21:03.171 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-21T11:21:03.171 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-21T11:21:03.171 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-21T11:21:03.171 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T11:21:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:02 smithi102 ceph-mon[36748]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 0 op/s 2024-03-21T11:21:03.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:03.458 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T11:21:03.458 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T11:21:03.458 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-21T11:21:03.458 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.0 master ceph-brx 2024-03-21T11:21:03.458 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T11:21:03.638 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-21T11:21:03.638 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T11:21:03.638 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-21T11:21:03.688 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-21T11:21:03.688 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T11:21:03.688 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-21T11:21:03.737 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-21T11:21:03.852 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T11:21:03.900 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-03-21T11:21:03.900 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-03-21T11:21:03.900 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-03-21T11:21:03.900 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-03-21T11:21:03.900 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-03-21T11:21:03.900 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T11:21:03.901 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T11:21:03.902 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T11:21:03.902 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T11:21:03.902 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-03-21T11:21:03.902 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-03-21T11:21:03.902 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay 2024-03-21T11:21:03.902 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/b0959b2f4a41a6d827ca88f7c3f56c3892e2681b7e9718294300c48d3a296bc8/merged 2024-03-21T11:21:03.902 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/b0fe289b9f994f8ae47cf40c5e2627ac47b56dc1922bc4456f7de391b48f4690/merged 2024-03-21T11:21:03.902 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/0 2024-03-21T11:21:03.902 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T11:21:03.902 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/04f5c698e9378d31898c95f388062eacc12b7b767470b3aae2fca804ed0db614/merged 2024-03-21T11:21:03.903 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/94ad49511c4d29ce29f7b91e99bb1fc4c0433c7bed6bbde1f718b28c1d5fa556/merged 2024-03-21T11:21:03.903 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/42d8f3e78dab8f60216fd46f623ebc8b58b206927c36ead622c89cd07829729b/merged 2024-03-21T11:21:03.903 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/57cbfb58334b963472bf36de072f7a87f157a6a73251680a0de0dddb606d5aa2/merged 2024-03-21T11:21:03.903 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/4e1a0a78a04575d57d31202af0169fc47a3f2fb03a716384571a443f5f7a588c/merged 2024-03-21T11:21:03.903 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/196dc14855258dcecbdf4d85c37d78946fb1a22a8fa4a00776e0e6c7e03cb6cd/merged 2024-03-21T11:21:03.903 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/806af9c150bbdff470894e192b7ab16d1afa3071ca2e0016ce55af1c6c809839/merged 2024-03-21T11:21:03.903 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/6d9716b2edcf62816b1660eeeb0001cb12fa32efaa0c18c5b79ad32ab6b3bacf/merged 2024-03-21T11:21:03.903 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-03-21T11:21:03.903 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T11:21:03.903 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T11:21:03.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:03.904 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-21T11:21:03.944 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-21T11:21:03.945 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-21T11:21:03.992 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-21T11:21:04.073 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T11:21:04.117 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-03-21T11:21:04.117 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-03-21T11:21:04.117 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-03-21T11:21:04.117 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-03-21T11:21:04.117 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-03-21T11:21:04.117 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-03-21T11:21:04.117 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-03-21T11:21:04.117 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-03-21T11:21:04.117 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-03-21T11:21:04.118 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-03-21T11:21:04.118 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-03-21T11:21:04.118 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-03-21T11:21:04.118 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T11:21:04.118 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-03-21T11:21:04.118 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-03-21T11:21:04.118 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-03-21T11:21:04.118 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-03-21T11:21:04.118 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-03-21T11:21:04.118 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-03-21T11:21:04.118 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T11:21:04.118 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T11:21:04.118 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/b0959b2f4a41a6d827ca88f7c3f56c3892e2681b7e9718294300c48d3a296bc8/merged 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/b0fe289b9f994f8ae47cf40c5e2627ac47b56dc1922bc4456f7de391b48f4690/merged 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/0 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/04f5c698e9378d31898c95f388062eacc12b7b767470b3aae2fca804ed0db614/merged 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/94ad49511c4d29ce29f7b91e99bb1fc4c0433c7bed6bbde1f718b28c1d5fa556/merged 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/42d8f3e78dab8f60216fd46f623ebc8b58b206927c36ead622c89cd07829729b/merged 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/57cbfb58334b963472bf36de072f7a87f157a6a73251680a0de0dddb606d5aa2/merged 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/4e1a0a78a04575d57d31202af0169fc47a3f2fb03a716384571a443f5f7a588c/merged 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/196dc14855258dcecbdf4d85c37d78946fb1a22a8fa4a00776e0e6c7e03cb6cd/merged 2024-03-21T11:21:04.119 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/806af9c150bbdff470894e192b7ab16d1afa3071ca2e0016ce55af1c6c809839/merged 2024-03-21T11:21:04.120 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/6d9716b2edcf62816b1660eeeb0001cb12fa32efaa0c18c5b79ad32ab6b3bacf/merged 2024-03-21T11:21:04.120 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-03-21T11:21:04.120 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T11:21:04.120 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T11:21:04.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:04.121 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-21T11:21:04.214 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:2024-03-21T11:21:04.214+0000 7f745143e480 -1 init, newargv = 0x562209c74f50 newargc=15 2024-03-21T11:21:04.215 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:ceph-fuse[72690]: starting ceph client 2024-03-21T11:21:04.224 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:ceph-fuse[72690]: starting fuse 2024-03-21T11:21:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:04 smithi047 ceph-mon[31351]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 773 B/s wr, 0 op/s 2024-03-21T11:21:05.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:04 smithi100 ceph-mon[36245]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 773 B/s wr, 0 op/s 2024-03-21T11:21:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:04 smithi102 ceph-mon[36748]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 773 B/s wr, 0 op/s 2024-03-21T11:21:05.304 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-21T11:21:05.412 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T11:21:05.467 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-03-21T11:21:05.467 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-03-21T11:21:05.468 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-03-21T11:21:05.468 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-03-21T11:21:05.468 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-03-21T11:21:05.468 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-03-21T11:21:05.468 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-03-21T11:21:05.468 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-03-21T11:21:05.468 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-03-21T11:21:05.468 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-03-21T11:21:05.468 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-03-21T11:21:05.468 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-03-21T11:21:05.468 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T11:21:05.469 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-03-21T11:21:05.469 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-03-21T11:21:05.469 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-03-21T11:21:05.469 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-03-21T11:21:05.469 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-03-21T11:21:05.469 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-03-21T11:21:05.469 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T11:21:05.469 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T11:21:05.469 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T11:21:05.469 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T11:21:05.469 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-03-21T11:21:05.469 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-03-21T11:21:05.469 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay 2024-03-21T11:21:05.470 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/b0959b2f4a41a6d827ca88f7c3f56c3892e2681b7e9718294300c48d3a296bc8/merged 2024-03-21T11:21:05.470 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/b0fe289b9f994f8ae47cf40c5e2627ac47b56dc1922bc4456f7de391b48f4690/merged 2024-03-21T11:21:05.470 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/0 2024-03-21T11:21:05.470 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T11:21:05.470 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/04f5c698e9378d31898c95f388062eacc12b7b767470b3aae2fca804ed0db614/merged 2024-03-21T11:21:05.470 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/94ad49511c4d29ce29f7b91e99bb1fc4c0433c7bed6bbde1f718b28c1d5fa556/merged 2024-03-21T11:21:05.470 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/42d8f3e78dab8f60216fd46f623ebc8b58b206927c36ead622c89cd07829729b/merged 2024-03-21T11:21:05.470 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/57cbfb58334b963472bf36de072f7a87f157a6a73251680a0de0dddb606d5aa2/merged 2024-03-21T11:21:05.470 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/4e1a0a78a04575d57d31202af0169fc47a3f2fb03a716384571a443f5f7a588c/merged 2024-03-21T11:21:05.470 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/196dc14855258dcecbdf4d85c37d78946fb1a22a8fa4a00776e0e6c7e03cb6cd/merged 2024-03-21T11:21:05.470 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/806af9c150bbdff470894e192b7ab16d1afa3071ca2e0016ce55af1c6c809839/merged 2024-03-21T11:21:05.470 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/6d9716b2edcf62816b1660eeeb0001cb12fa32efaa0c18c5b79ad32ab6b3bacf/merged 2024-03-21T11:21:05.470 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-03-21T11:21:05.471 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T11:21:05.471 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T11:21:05.471 INFO:teuthology.orchestra.run.smithi047.stdout:/home/ubuntu/cephtest/mnt.admin 2024-03-21T11:21:05.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:05.472 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-21T11:21:05.530 INFO:teuthology.orchestra.run.smithi047.stdout:118 2024-03-21T11:21:05.530 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-21T11:21:05.530 DEBUG:teuthology.orchestra.run.smithi047:> sudo stdin-killer -- python3 -c ' 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-21T11:21:05.531 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-21T11:21:05.532 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-21T11:21:05.532 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-21T11:21:05.532 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-21T11:21:05.532 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-21T11:21:05.532 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-21T11:21:05.532 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-21T11:21:05.532 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-21T11:21:05.532 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-21T11:21:05.532 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.admin")) 2024-03-21T11:21:05.532 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-21T11:21:05.718 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.admin.72690.asok 2024-03-21T11:21:05.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-21T11:21:05.738 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.72690.asok 2024-03-21T11:21:05.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:05.738 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.72690.asok status 2024-03-21T11:21:05.987 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:21:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-21T11:21:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "2edfee68ad7243bed422a049b6f5f58619300f11", 2024-03-21T11:21:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T11:21:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "admin", 2024-03-21T11:21:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-21T11:21:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-21T11:21:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "72690", 2024-03-21T11:21:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-21T11:21:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:21:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-21T11:21:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-21T11:21:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 14595, 2024-03-21T11:21:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-21T11:21:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-21T11:21:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-21T11:21:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 14595 2024-03-21T11:21:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:21:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-21T11:21:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-21T11:21:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-03-21T11:21:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1142751110 2024-03-21T11:21:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:21:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:21:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-21T11:21:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-21T11:21:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-03-21T11:21:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1142751110 2024-03-21T11:21:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:21:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.14595 172.21.15.47:0/1142751110", 2024-03-21T11:21:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "172.21.15.47:0/1142751110", 2024-03-21T11:21:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-21T11:21:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 17, 2024-03-21T11:21:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 91, 2024-03-21T11:21:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-21T11:21:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-21T11:21:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-21T11:21:05.989 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:21:06.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:06.002 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-21T11:21:06.002 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-21T11:21:06.057 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-21T11:21:06.057 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-21T11:21:06.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:06.058 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-21T11:21:06.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:06.153 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-21T11:21:06.237 INFO:teuthology.orchestra.run.smithi047.stdout:# file: . 2024-03-21T11:21:06.237 INFO:teuthology.orchestra.run.smithi047.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-21T11:21:06.237 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:21:06.238 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:21:06.239 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T11:21:06.295 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:21:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.520+0000 7f1d08c00640 1 -- 172.21.15.47:0/2848396181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d04108cf0 msgr2=0x7f1d04109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.520+0000 7f1d08c00640 1 --2- 172.21.15.47:0/2848396181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d04108cf0 0x7f1d04109170 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009900 tx=0x7f1cec02bc30 comp rx=0 tx=0).stop 2024-03-21T11:21:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.520+0000 7f1d08c00640 1 -- 172.21.15.47:0/2848396181 shutdown_connections 2024-03-21T11:21:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.520+0000 7f1d08c00640 1 --2- 172.21.15.47:0/2848396181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d041096b0 0x7f1d0410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.520+0000 7f1d08c00640 1 --2- 172.21.15.47:0/2848396181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d04108cf0 0x7f1d04109170 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.520+0000 7f1d08c00640 1 --2- 172.21.15.47:0/2848396181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d04107b00 0x7f1d04107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.520+0000 7f1d08c00640 1 -- 172.21.15.47:0/2848396181 >> 172.21.15.47:0/2848396181 conn(0x7f1d04076ab0 msgr2=0x7f1d04078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.520+0000 7f1d08c00640 1 -- 172.21.15.47:0/2848396181 shutdown_connections 2024-03-21T11:21:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.520+0000 7f1d08c00640 1 -- 172.21.15.47:0/2848396181 wait complete. 2024-03-21T11:21:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.521+0000 7f1d08c00640 1 Processor -- start 2024-03-21T11:21:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.522+0000 7f1d08c00640 1 -- start start 2024-03-21T11:21:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.522+0000 7f1d08c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d04107b00 0x7f1d0419ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.522+0000 7f1d08c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d04108cf0 0x7f1d0419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.522+0000 7f1d08c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d041096b0 0x7f1d041a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.522+0000 7f1d08c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d041137b0 con 0x7f1d04108cf0 2024-03-21T11:21:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.522+0000 7f1d08c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04113630 con 0x7f1d041096b0 2024-03-21T11:21:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.522+0000 7f1d08c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04113930 con 0x7f1d04107b00 2024-03-21T11:21:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.522+0000 7f1cfbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d04108cf0 0x7f1d0419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.522+0000 7f1d03400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d04107b00 0x7f1d0419ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.522+0000 7f1cfbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d04108cf0 0x7f1d0419e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33004/0 (socket says 172.21.15.47:33004) 2024-03-21T11:21:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.522+0000 7f1cfbe00640 1 -- 172.21.15.47:0/3232247804 learned_addr learned my addr 172.21.15.47:0/3232247804 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.523+0000 7f1d03e00640 1 --2- 172.21.15.47:0/3232247804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d041096b0 0x7f1d041a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.523+0000 7f1d03400640 1 -- 172.21.15.47:0/3232247804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d041096b0 msgr2=0x7f1d041a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.523+0000 7f1d03400640 1 --2- 172.21.15.47:0/3232247804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d041096b0 0x7f1d041a5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.523+0000 7f1d03400640 1 -- 172.21.15.47:0/3232247804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d04108cf0 msgr2=0x7f1d0419e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.523+0000 7f1d03400640 1 --2- 172.21.15.47:0/3232247804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d04108cf0 0x7f1d0419e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.523+0000 7f1d03400640 1 -- 172.21.15.47:0/3232247804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d041a59e0 con 0x7f1d04107b00 2024-03-21T11:21:06.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.523+0000 7f1d03400640 1 --2- 172.21.15.47:0/3232247804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d04107b00 0x7f1d0419ddc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400e9c0 tx=0x7f1cf400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:06.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.524+0000 7f1d00c00640 1 -- 172.21.15.47:0/3232247804 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400ccc0 con 0x7f1d04107b00 2024-03-21T11:21:06.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.524+0000 7f1d00c00640 1 -- 172.21.15.47:0/3232247804 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1cf40035f0 con 0x7f1d04107b00 2024-03-21T11:21:06.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.524+0000 7f1d00c00640 1 -- 172.21.15.47:0/3232247804 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400ce60 con 0x7f1d04107b00 2024-03-21T11:21:06.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.524+0000 7f1d08c00640 1 -- 172.21.15.47:0/3232247804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d041a5cd0 con 0x7f1d04107b00 2024-03-21T11:21:06.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.524+0000 7f1d08c00640 1 -- 172.21.15.47:0/3232247804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d041a61a0 con 0x7f1d04107b00 2024-03-21T11:21:06.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.525+0000 7f1d08c00640 1 -- 172.21.15.47:0/3232247804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0410ee20 con 0x7f1d04107b00 2024-03-21T11:21:06.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.527+0000 7f1d00c00640 1 -- 172.21.15.47:0/3232247804 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1cf40037a0 con 0x7f1d04107b00 2024-03-21T11:21:06.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.529+0000 7f1d00c00640 1 --2- 172.21.15.47:0/3232247804 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1ce80a2fa0 0x7f1ce80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:06.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.529+0000 7f1d00c00640 1 -- 172.21.15.47:0/3232247804 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f1cf40cf960 con 0x7f1d04107b00 2024-03-21T11:21:06.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.529+0000 7f1cfbe00640 1 --2- 172.21.15.47:0/3232247804 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1ce80a2fa0 0x7f1ce80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:06.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.530+0000 7f1cfbe00640 1 --2- 172.21.15.47:0/3232247804 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1ce80a2fa0 0x7f1ce80a5460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1cec02c8e0 tx=0x7f1cec02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:06.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.531+0000 7f1d00c00640 1 -- 172.21.15.47:0/3232247804 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1cf409c4f0 con 0x7f1d04107b00 2024-03-21T11:21:06.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.736+0000 7f1d08c00640 1 -- 172.21.15.47:0/3232247804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d0402d240 con 0x7f1d04107b00 2024-03-21T11:21:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.739+0000 7f1d00c00640 1 -- 172.21.15.47:0/3232247804 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24127 (secure 0 0 0) 0x7f1cf409bdf0 con 0x7f1d04107b00 2024-03-21T11:21:06.740 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:21:06.740 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:20:54.191030+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:21:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.742+0000 7f1d08c00640 1 -- 172.21.15.47:0/3232247804 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1ce80a2fa0 msgr2=0x7f1ce80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.742+0000 7f1d08c00640 1 --2- 172.21.15.47:0/3232247804 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1ce80a2fa0 0x7f1ce80a5460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1cec02c8e0 tx=0x7f1cec02df90 comp rx=0 tx=0).stop 2024-03-21T11:21:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.742+0000 7f1d08c00640 1 -- 172.21.15.47:0/3232247804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d04107b00 msgr2=0x7f1d0419ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.742+0000 7f1d08c00640 1 --2- 172.21.15.47:0/3232247804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d04107b00 0x7f1d0419ddc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400e9c0 tx=0x7f1cf400ee80 comp rx=0 tx=0).stop 2024-03-21T11:21:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.742+0000 7f1d08c00640 1 -- 172.21.15.47:0/3232247804 shutdown_connections 2024-03-21T11:21:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.742+0000 7f1d08c00640 1 --2- 172.21.15.47:0/3232247804 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1ce80a2fa0 0x7f1ce80a5460 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.742+0000 7f1d08c00640 1 --2- 172.21.15.47:0/3232247804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d041096b0 0x7f1d041a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.742+0000 7f1d08c00640 1 --2- 172.21.15.47:0/3232247804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d04108cf0 0x7f1d0419e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.742+0000 7f1d08c00640 1 --2- 172.21.15.47:0/3232247804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d04107b00 0x7f1d0419ddc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.742+0000 7f1d08c00640 1 -- 172.21.15.47:0/3232247804 >> 172.21.15.47:0/3232247804 conn(0x7f1d04076ab0 msgr2=0x7f1d0410bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.742+0000 7f1d08c00640 1 -- 172.21.15.47:0/3232247804 shutdown_connections 2024-03-21T11:21:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.742+0000 7f1d08c00640 1 -- 172.21.15.47:0/3232247804 wait complete. 2024-03-21T11:21:06.763 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T11:15:35.545245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T11:20:47.016174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T11:20:48.108506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T11:20:52.254154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T11:21:06.764 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T11:21:06.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.991+0000 7f06a5600640 1 -- 172.21.15.47:0/2222820157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a0104a50 msgr2=0x7f06a0106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:06.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.991+0000 7f06a5600640 1 --2- 172.21.15.47:0/2222820157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a0104a50 0x7f06a0106e40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f068c009900 tx=0x7f068c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:21:06.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.992+0000 7f06a5600640 1 -- 172.21.15.47:0/2222820157 shutdown_connections 2024-03-21T11:21:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.992+0000 7f06a5600640 1 --2- 172.21.15.47:0/2222820157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06a0107380 0x7f06a01097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.992+0000 7f06a5600640 1 --2- 172.21.15.47:0/2222820157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a0104a50 0x7f06a0106e40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.992+0000 7f06a5600640 1 --2- 172.21.15.47:0/2222820157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06a0102120 0x7f06a0104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.992+0000 7f06a5600640 1 -- 172.21.15.47:0/2222820157 >> 172.21.15.47:0/2222820157 conn(0x7f06a00fbb30 msgr2=0x7f06a00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.992+0000 7f06a5600640 1 -- 172.21.15.47:0/2222820157 shutdown_connections 2024-03-21T11:21:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.992+0000 7f06a5600640 1 -- 172.21.15.47:0/2222820157 wait complete. 2024-03-21T11:21:06.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.993+0000 7f06a5600640 1 Processor -- start 2024-03-21T11:21:06.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.993+0000 7f06a5600640 1 -- start start 2024-03-21T11:21:06.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.993+0000 7f06a5600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06a0102120 0x7f06a0199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.993+0000 7f06a5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06a0104a50 0x7f06a019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.993+0000 7f06a5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a0107380 0x7f06a01a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.993+0000 7f06a5600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a010d360 con 0x7f06a0102120 2024-03-21T11:21:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.993+0000 7f06a5600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a010d1e0 con 0x7f06a0107380 2024-03-21T11:21:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.993+0000 7f06a5600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a010d4e0 con 0x7f06a0104a50 2024-03-21T11:21:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.993+0000 7f069ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06a0102120 0x7f06a0199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.994+0000 7f069f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a0107380 0x7f06a01a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.994+0000 7f069f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a0107380 0x7f06a01a11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45244/0 (socket says 172.21.15.47:45244) 2024-03-21T11:21:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.994+0000 7f069f400640 1 -- 172.21.15.47:0/3309839611 learned_addr learned my addr 172.21.15.47:0/3309839611 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.994+0000 7f069e000640 1 --2- 172.21.15.47:0/3309839611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06a0104a50 0x7f06a019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.994+0000 7f069f400640 1 -- 172.21.15.47:0/3309839611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06a0104a50 msgr2=0x7f06a019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.994+0000 7f069f400640 1 --2- 172.21.15.47:0/3309839611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06a0104a50 0x7f06a019a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:06.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.994+0000 7f069f400640 1 -- 172.21.15.47:0/3309839611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06a0102120 msgr2=0x7f06a0199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:06.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.994+0000 7f069f400640 1 --2- 172.21.15.47:0/3309839611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06a0102120 0x7f06a0199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:06.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.994+0000 7f069f400640 1 -- 172.21.15.47:0/3309839611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a01a1860 con 0x7f06a0107380 2024-03-21T11:21:06.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.994+0000 7f069f400640 1 --2- 172.21.15.47:0/3309839611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a0107380 0x7f06a01a11c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f069800eb10 tx=0x7f069800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:06.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.994+0000 7f0697400640 1 -- 172.21.15.47:0/3309839611 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0698004b20 con 0x7f06a0107380 2024-03-21T11:21:06.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.995+0000 7f0697400640 1 -- 172.21.15.47:0/3309839611 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0698004cc0 con 0x7f06a0107380 2024-03-21T11:21:06.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.995+0000 7f0697400640 1 -- 172.21.15.47:0/3309839611 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0698010760 con 0x7f06a0107380 2024-03-21T11:21:06.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.995+0000 7f06a5600640 1 -- 172.21.15.47:0/3309839611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a01a1b50 con 0x7f06a0107380 2024-03-21T11:21:06.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.995+0000 7f06a5600640 1 -- 172.21.15.47:0/3309839611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a01a20a0 con 0x7f06a0107380 2024-03-21T11:21:06.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.996+0000 7f06a5600640 1 -- 172.21.15.47:0/3309839611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a0108ed0 con 0x7f06a0107380 2024-03-21T11:21:06.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.998+0000 7f0697400640 1 -- 172.21.15.47:0/3309839611 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0698005020 con 0x7f06a0107380 2024-03-21T11:21:06.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.999+0000 7f0697400640 1 --2- 172.21.15.47:0/3309839611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f06840a2fa0 0x7f06840a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:06.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.999+0000 7f0697400640 1 -- 172.21.15.47:0/3309839611 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f06980cfc20 con 0x7f06a0107380 2024-03-21T11:21:07.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:06.999+0000 7f069ea00640 1 --2- 172.21.15.47:0/3309839611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f06840a2fa0 0x7f06840a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:07.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.000+0000 7f069ea00640 1 --2- 172.21.15.47:0/3309839611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f06840a2fa0 0x7f06840a5460 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f06a019b120 tx=0x7f0690005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:07.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.004+0000 7f0697400640 1 -- 172.21.15.47:0/3309839611 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f069809c7b0 con 0x7f06a0107380 2024-03-21T11:21:07.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:06 smithi047 ceph-mon[31351]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:21:07.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:06 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3232247804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:07.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:06 smithi100 ceph-mon[36245]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:21:07.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:06 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3232247804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:07.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.207+0000 7f06a5600640 1 -- 172.21.15.47:0/3309839611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06a0062340 con 0x7f06a0107380 2024-03-21T11:21:07.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.207+0000 7f0697400640 1 -- 172.21.15.47:0/3309839611 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06980bae20 con 0x7f06a0107380 2024-03-21T11:21:07.208 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T11:21:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.210+0000 7f06a5600640 1 -- 172.21.15.47:0/3309839611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f06840a2fa0 msgr2=0x7f06840a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.210+0000 7f06a5600640 1 --2- 172.21.15.47:0/3309839611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f06840a2fa0 0x7f06840a5460 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f06a019b120 tx=0x7f0690005f90 comp rx=0 tx=0).stop 2024-03-21T11:21:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.210+0000 7f06a5600640 1 -- 172.21.15.47:0/3309839611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a0107380 msgr2=0x7f06a01a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.210+0000 7f06a5600640 1 --2- 172.21.15.47:0/3309839611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a0107380 0x7f06a01a11c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f069800eb10 tx=0x7f069800c2d0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.210+0000 7f06a5600640 1 -- 172.21.15.47:0/3309839611 shutdown_connections 2024-03-21T11:21:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.210+0000 7f06a5600640 1 --2- 172.21.15.47:0/3309839611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f06840a2fa0 0x7f06840a5460 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.210+0000 7f06a5600640 1 --2- 172.21.15.47:0/3309839611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a0107380 0x7f06a01a11c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.210+0000 7f06a5600640 1 --2- 172.21.15.47:0/3309839611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06a0104a50 0x7f06a019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.210+0000 7f06a5600640 1 --2- 172.21.15.47:0/3309839611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06a0102120 0x7f06a0199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.210+0000 7f06a5600640 1 -- 172.21.15.47:0/3309839611 >> 172.21.15.47:0/3309839611 conn(0x7f06a00fbb30 msgr2=0x7f06a0101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.210+0000 7f06a5600640 1 -- 172.21.15.47:0/3309839611 shutdown_connections 2024-03-21T11:21:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.210+0000 7f06a5600640 1 -- 172.21.15.47:0/3309839611 wait complete. 2024-03-21T11:21:07.229 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T11:21:07.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:06 smithi102 ceph-mon[36748]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:21:07.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:06 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3232247804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:07.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.448+0000 7f1198a00640 1 -- 172.21.15.47:0/3914324841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11940772a0 msgr2=0x7f1194077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:07.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.448+0000 7f1198a00640 1 --2- 172.21.15.47:0/3914324841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11940772a0 0x7f1194077700 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f117c009ca0 tx=0x7f117c02c0f0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.448+0000 7f1198a00640 1 -- 172.21.15.47:0/3914324841 shutdown_connections 2024-03-21T11:21:07.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.448+0000 7f1198a00640 1 --2- 172.21.15.47:0/3914324841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1194077c40 0x7f11940784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.448+0000 7f1198a00640 1 --2- 172.21.15.47:0/3914324841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11940772a0 0x7f1194077700 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.448+0000 7f1198a00640 1 --2- 172.21.15.47:0/3914324841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1194076060 0x7f1194076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.448+0000 7f1198a00640 1 -- 172.21.15.47:0/3914324841 >> 172.21.15.47:0/3914324841 conn(0x7f11940fee50 msgr2=0x7f1194101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:07.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.448+0000 7f1198a00640 1 -- 172.21.15.47:0/3914324841 shutdown_connections 2024-03-21T11:21:07.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.448+0000 7f1198a00640 1 -- 172.21.15.47:0/3914324841 wait complete. 2024-03-21T11:21:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.450+0000 7f1198a00640 1 Processor -- start 2024-03-21T11:21:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.450+0000 7f1198a00640 1 -- start start 2024-03-21T11:21:07.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.450+0000 7f1198a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1194076060 0x7f119419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.450+0000 7f1198a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11940772a0 0x7f119419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.450+0000 7f1198a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1194077c40 0x7f11941a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.451+0000 7f1198a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1194113970 con 0x7f11940772a0 2024-03-21T11:21:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.451+0000 7f1198a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11941137f0 con 0x7f1194076060 2024-03-21T11:21:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.451+0000 7f1198a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1194113af0 con 0x7f1194077c40 2024-03-21T11:21:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.451+0000 7f1191e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1194076060 0x7f119419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.451+0000 7f1191e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1194076060 0x7f119419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45262/0 (socket says 172.21.15.47:45262) 2024-03-21T11:21:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.451+0000 7f1191e00640 1 -- 172.21.15.47:0/1320669447 learned_addr learned my addr 172.21.15.47:0/1320669447 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.451+0000 7f1192800640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1194077c40 0x7f11941a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.451+0000 7f1191400640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11940772a0 0x7f119419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.451+0000 7f1191e00640 1 -- 172.21.15.47:0/1320669447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1194077c40 msgr2=0x7f11941a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.451+0000 7f1191e00640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1194077c40 0x7f11941a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.451+0000 7f1191e00640 1 -- 172.21.15.47:0/1320669447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11940772a0 msgr2=0x7f119419e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.451+0000 7f1191e00640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11940772a0 0x7f119419e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.451+0000 7f1191e00640 1 -- 172.21.15.47:0/1320669447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11941a5bc0 con 0x7f1194076060 2024-03-21T11:21:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.451+0000 7f1191400640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11940772a0 0x7f119419e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:21:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.452+0000 7f1191e00640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1194076060 0x7f119419dfa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f118400e9f0 tx=0x7f118400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.452+0000 7f118aa00640 1 -- 172.21.15.47:0/1320669447 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118400cd80 con 0x7f1194076060 2024-03-21T11:21:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.452+0000 7f118aa00640 1 -- 172.21.15.47:0/1320669447 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1184010910 con 0x7f1194076060 2024-03-21T11:21:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.452+0000 7f118aa00640 1 -- 172.21.15.47:0/1320669447 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184018990 con 0x7f1194076060 2024-03-21T11:21:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.452+0000 7f1198a00640 1 -- 172.21.15.47:0/1320669447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11941a5eb0 con 0x7f1194076060 2024-03-21T11:21:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.452+0000 7f1198a00640 1 -- 172.21.15.47:0/1320669447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11941a62c0 con 0x7f1194076060 2024-03-21T11:21:07.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.453+0000 7f1198a00640 1 -- 172.21.15.47:0/1320669447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11941103e0 con 0x7f1194076060 2024-03-21T11:21:07.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.455+0000 7f118aa00640 1 -- 172.21.15.47:0/1320669447 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1184010ac0 con 0x7f1194076060 2024-03-21T11:21:07.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.456+0000 7f118aa00640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f11740a2fa0 0x7f11740a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:07.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.457+0000 7f118aa00640 1 -- 172.21.15.47:0/1320669447 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f11840cf9a0 con 0x7f1194076060 2024-03-21T11:21:07.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.457+0000 7f1191400640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f11740a2fa0 0x7f11740a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:07.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.457+0000 7f1191400640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f11740a2fa0 0x7f11740a5460 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f117c02c700 tx=0x7f117c032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.460+0000 7f118aa00640 1 -- 172.21.15.47:0/1320669447 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f118409c530 con 0x7f1194076060 2024-03-21T11:21:07.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.660+0000 7f1198a00640 1 -- 172.21.15.47:0/1320669447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1194061ae0 con 0x7f1194076060 2024-03-21T11:21:07.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.661+0000 7f118aa00640 1 -- 172.21.15.47:0/1320669447 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f118409be30 con 0x7f1194076060 2024-03-21T11:21:07.662 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:21:07.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.663+0000 7f1198a00640 1 -- 172.21.15.47:0/1320669447 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f11740a2fa0 msgr2=0x7f11740a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.663+0000 7f1198a00640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f11740a2fa0 0x7f11740a5460 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f117c02c700 tx=0x7f117c032590 comp rx=0 tx=0).stop 2024-03-21T11:21:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.664+0000 7f1198a00640 1 -- 172.21.15.47:0/1320669447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1194076060 msgr2=0x7f119419dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.664+0000 7f1198a00640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1194076060 0x7f119419dfa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f118400e9f0 tx=0x7f118400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.664+0000 7f1198a00640 1 -- 172.21.15.47:0/1320669447 shutdown_connections 2024-03-21T11:21:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.664+0000 7f1198a00640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f11740a2fa0 0x7f11740a5460 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.664+0000 7f1198a00640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1194077c40 0x7f11941a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.664+0000 7f1198a00640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11940772a0 0x7f119419e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.664+0000 7f1198a00640 1 --2- 172.21.15.47:0/1320669447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1194076060 0x7f119419dfa0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.664+0000 7f1198a00640 1 -- 172.21.15.47:0/1320669447 >> 172.21.15.47:0/1320669447 conn(0x7f11940fee50 msgr2=0x7f1194100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.664+0000 7f1198a00640 1 -- 172.21.15.47:0/1320669447 shutdown_connections 2024-03-21T11:21:07.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.664+0000 7f1198a00640 1 -- 172.21.15.47:0/1320669447 wait complete. 2024-03-21T11:21:07.683 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T11:21:07.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.903+0000 7eff6a000640 1 -- 172.21.15.47:0/2656070105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff641037c0 msgr2=0x7eff64103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:07.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.903+0000 7eff6a000640 1 --2- 172.21.15.47:0/2656070105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff641037c0 0x7eff64103be0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7eff58009900 tx=0x7eff5802bc30 comp rx=0 tx=0).stop 2024-03-21T11:21:07.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.903+0000 7eff6a000640 1 -- 172.21.15.47:0/2656070105 shutdown_connections 2024-03-21T11:21:07.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.903+0000 7eff6a000640 1 --2- 172.21.15.47:0/2656070105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff64105370 0x7eff6410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.903+0000 7eff6a000640 1 --2- 172.21.15.47:0/2656070105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff641049b0 0x7eff64104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.903+0000 7eff6a000640 1 --2- 172.21.15.47:0/2656070105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff641037c0 0x7eff64103be0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.903+0000 7eff6a000640 1 -- 172.21.15.47:0/2656070105 >> 172.21.15.47:0/2656070105 conn(0x7eff640fed50 msgr2=0x7eff64101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:07.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.903+0000 7eff6a000640 1 -- 172.21.15.47:0/2656070105 shutdown_connections 2024-03-21T11:21:07.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.903+0000 7eff6a000640 1 -- 172.21.15.47:0/2656070105 wait complete. 2024-03-21T11:21:07.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.904+0000 7eff6a000640 1 Processor -- start 2024-03-21T11:21:07.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.905+0000 7eff6a000640 1 -- start start 2024-03-21T11:21:07.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.905+0000 7eff6a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff641037c0 0x7eff64199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:07.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.905+0000 7eff6a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff641049b0 0x7eff6419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:07.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.905+0000 7eff6a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff64105370 0x7eff641a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:07.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.905+0000 7eff6a000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6410f410 con 0x7eff64105370 2024-03-21T11:21:07.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.905+0000 7eff6a000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6410f290 con 0x7eff641049b0 2024-03-21T11:21:07.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.905+0000 7eff6a000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6410f590 con 0x7eff641037c0 2024-03-21T11:21:07.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.905+0000 7eff63e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff641049b0 0x7eff6419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:07.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.905+0000 7eff69600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff64105370 0x7eff641a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:07.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.905+0000 7eff63e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff641049b0 0x7eff6419a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45276/0 (socket says 172.21.15.47:45276) 2024-03-21T11:21:07.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.905+0000 7eff68c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff641037c0 0x7eff64199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:07.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.905+0000 7eff63e00640 1 -- 172.21.15.47:0/942099899 learned_addr learned my addr 172.21.15.47:0/942099899 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:07.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.906+0000 7eff63e00640 1 -- 172.21.15.47:0/942099899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff641037c0 msgr2=0x7eff64199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:07.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.906+0000 7eff63e00640 1 --2- 172.21.15.47:0/942099899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff641037c0 0x7eff64199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.906+0000 7eff63e00640 1 -- 172.21.15.47:0/942099899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff64105370 msgr2=0x7eff641a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:07.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.906+0000 7eff63e00640 1 --2- 172.21.15.47:0/942099899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff64105370 0x7eff641a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:07.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.906+0000 7eff63e00640 1 -- 172.21.15.47:0/942099899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff641a1730 con 0x7eff641049b0 2024-03-21T11:21:07.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.906+0000 7eff63e00640 1 --2- 172.21.15.47:0/942099899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff641049b0 0x7eff6419a0f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7eff5000ea60 tx=0x7eff5000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:07.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.906+0000 7eff61600640 1 -- 172.21.15.47:0/942099899 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5000cde0 con 0x7eff641049b0 2024-03-21T11:21:07.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.906+0000 7eff61600640 1 -- 172.21.15.47:0/942099899 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff500035f0 con 0x7eff641049b0 2024-03-21T11:21:07.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.906+0000 7eff6a000640 1 -- 172.21.15.47:0/942099899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff641a1a20 con 0x7eff641049b0 2024-03-21T11:21:07.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.907+0000 7eff61600640 1 -- 172.21.15.47:0/942099899 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50010670 con 0x7eff641049b0 2024-03-21T11:21:07.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.907+0000 7eff6a000640 1 -- 172.21.15.47:0/942099899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff641a1fc0 con 0x7eff641049b0 2024-03-21T11:21:07.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.908+0000 7eff6a000640 1 -- 172.21.15.47:0/942099899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6410ab00 con 0x7eff641049b0 2024-03-21T11:21:07.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.909+0000 7eff61600640 1 -- 172.21.15.47:0/942099899 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7eff500037a0 con 0x7eff641049b0 2024-03-21T11:21:07.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.912+0000 7eff61600640 1 --2- 172.21.15.47:0/942099899 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff4c0a2fa0 0x7eff4c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:07.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.912+0000 7eff61600640 1 -- 172.21.15.47:0/942099899 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7eff500cf8c0 con 0x7eff641049b0 2024-03-21T11:21:07.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.912+0000 7eff68c00640 1 --2- 172.21.15.47:0/942099899 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff4c0a2fa0 0x7eff4c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:07.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.913+0000 7eff68c00640 1 --2- 172.21.15.47:0/942099899 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff4c0a2fa0 0x7eff4c0a5460 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7eff58000c00 tx=0x7eff58005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:07.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:07.915+0000 7eff61600640 1 -- 172.21.15.47:0/942099899 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7eff5009c450 con 0x7eff641049b0 2024-03-21T11:21:08.044 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:07 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3309839611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:21:08.044 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:07 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1320669447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:21:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:07 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3309839611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:21:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:07 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1320669447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:21:08.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.116+0000 7eff6a000640 1 -- 172.21.15.47:0/942099899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7eff640622a0 con 0x7eff641049b0 2024-03-21T11:21:08.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.117+0000 7eff61600640 1 -- 172.21.15.47:0/942099899 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7eff5009bd50 con 0x7eff641049b0 2024-03-21T11:21:08.118 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T11:21:08.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.120+0000 7eff6a000640 1 -- 172.21.15.47:0/942099899 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff4c0a2fa0 msgr2=0x7eff4c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.120+0000 7eff6a000640 1 --2- 172.21.15.47:0/942099899 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff4c0a2fa0 0x7eff4c0a5460 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7eff58000c00 tx=0x7eff58005fd0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.120+0000 7eff6a000640 1 -- 172.21.15.47:0/942099899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff641049b0 msgr2=0x7eff6419a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.120+0000 7eff6a000640 1 --2- 172.21.15.47:0/942099899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff641049b0 0x7eff6419a0f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7eff5000ea60 tx=0x7eff5000ef20 comp rx=0 tx=0).stop 2024-03-21T11:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.120+0000 7eff6a000640 1 -- 172.21.15.47:0/942099899 shutdown_connections 2024-03-21T11:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.120+0000 7eff6a000640 1 --2- 172.21.15.47:0/942099899 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff4c0a2fa0 0x7eff4c0a5460 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.120+0000 7eff6a000640 1 --2- 172.21.15.47:0/942099899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff64105370 0x7eff641a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.120+0000 7eff6a000640 1 --2- 172.21.15.47:0/942099899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff641049b0 0x7eff6419a0f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.120+0000 7eff6a000640 1 --2- 172.21.15.47:0/942099899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff641037c0 0x7eff64199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.120+0000 7eff6a000640 1 -- 172.21.15.47:0/942099899 >> 172.21.15.47:0/942099899 conn(0x7eff640fed50 msgr2=0x7eff64107cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.120+0000 7eff6a000640 1 -- 172.21.15.47:0/942099899 shutdown_connections 2024-03-21T11:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.120+0000 7eff6a000640 1 -- 172.21.15.47:0/942099899 wait complete. 2024-03-21T11:21:08.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T11:21:08.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:07 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3309839611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:21:08.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:07 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1320669447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:21:08.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.367+0000 7feb0f000640 1 -- 172.21.15.47:0/1620408622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb08103130 msgr2=0x7feb081099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:08.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.367+0000 7feb0f000640 1 --2- 172.21.15.47:0/1620408622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb08103130 0x7feb081099c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7feb00009ca0 tx=0x7feb0002c100 comp rx=0 tx=0).stop 2024-03-21T11:21:08.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.367+0000 7feb0f000640 1 -- 172.21.15.47:0/1620408622 shutdown_connections 2024-03-21T11:21:08.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.367+0000 7feb0f000640 1 --2- 172.21.15.47:0/1620408622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb08103130 0x7feb081099c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.367+0000 7feb0f000640 1 --2- 172.21.15.47:0/1620408622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb08102770 0x7feb08102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.367+0000 7feb0f000640 1 --2- 172.21.15.47:0/1620408622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb08101580 0x7feb081019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.367+0000 7feb0f000640 1 -- 172.21.15.47:0/1620408622 >> 172.21.15.47:0/1620408622 conn(0x7feb080fcb10 msgr2=0x7feb080fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:08.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.367+0000 7feb0f000640 1 -- 172.21.15.47:0/1620408622 shutdown_connections 2024-03-21T11:21:08.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.367+0000 7feb0f000640 1 -- 172.21.15.47:0/1620408622 wait complete. 2024-03-21T11:21:08.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.368+0000 7feb0f000640 1 Processor -- start 2024-03-21T11:21:08.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.368+0000 7feb0f000640 1 -- start start 2024-03-21T11:21:08.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.369+0000 7feb0f000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb08101580 0x7feb08075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:08.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.369+0000 7feb0f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb08102770 0x7feb08073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:08.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.369+0000 7feb0f000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb08103130 0x7feb08073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:08.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.369+0000 7feb0f000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0810d1e0 con 0x7feb08103130 2024-03-21T11:21:08.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.369+0000 7feb0f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0810d060 con 0x7feb08102770 2024-03-21T11:21:08.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.369+0000 7feb0f000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0810d360 con 0x7feb08101580 2024-03-21T11:21:08.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.369+0000 7feb0d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb08102770 0x7feb08073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:08.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.369+0000 7feb0dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb08101580 0x7feb08075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:08.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.369+0000 7feb0d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb08102770 0x7feb08073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45294/0 (socket says 172.21.15.47:45294) 2024-03-21T11:21:08.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.369+0000 7feb0d200640 1 -- 172.21.15.47:0/65050325 learned_addr learned my addr 172.21.15.47:0/65050325 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.369+0000 7feb0e600640 1 --2- 172.21.15.47:0/65050325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb08103130 0x7feb08073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.369+0000 7feb0d200640 1 -- 172.21.15.47:0/65050325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb08101580 msgr2=0x7feb08075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.369+0000 7feb0d200640 1 --2- 172.21.15.47:0/65050325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb08101580 0x7feb08075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.369+0000 7feb0d200640 1 -- 172.21.15.47:0/65050325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb08103130 msgr2=0x7feb08073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.370+0000 7feb0d200640 1 --2- 172.21.15.47:0/65050325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb08103130 0x7feb08073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.370+0000 7feb0d200640 1 -- 172.21.15.47:0/65050325 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb08074860 con 0x7feb08102770 2024-03-21T11:21:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.370+0000 7feb0d200640 1 --2- 172.21.15.47:0/65050325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb08102770 0x7feb08073a90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feaf400e9f0 tx=0x7feaf400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.370+0000 7feb06000640 1 -- 172.21.15.47:0/65050325 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf400cd80 con 0x7feb08102770 2024-03-21T11:21:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.370+0000 7feb06000640 1 -- 172.21.15.47:0/65050325 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feaf4010910 con 0x7feb08102770 2024-03-21T11:21:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.370+0000 7feb0f000640 1 -- 172.21.15.47:0/65050325 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb081a1940 con 0x7feb08102770 2024-03-21T11:21:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.370+0000 7feb06000640 1 -- 172.21.15.47:0/65050325 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4018990 con 0x7feb08102770 2024-03-21T11:21:08.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.370+0000 7feb0f000640 1 -- 172.21.15.47:0/65050325 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb081a1e90 con 0x7feb08102770 2024-03-21T11:21:08.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.371+0000 7feb0f000640 1 -- 172.21.15.47:0/65050325 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb08108840 con 0x7feb08102770 2024-03-21T11:21:08.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.374+0000 7feb06000640 1 -- 172.21.15.47:0/65050325 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7feaf4010ac0 con 0x7feb08102770 2024-03-21T11:21:08.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.375+0000 7feb06000640 1 --2- 172.21.15.47:0/65050325 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feaf00a2f20 0x7feaf00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:08.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.375+0000 7feb06000640 1 -- 172.21.15.47:0/65050325 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7feaf40cf990 con 0x7feb08102770 2024-03-21T11:21:08.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.376+0000 7feb0dc00640 1 --2- 172.21.15.47:0/65050325 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feaf00a2f20 0x7feaf00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:08.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.376+0000 7feb0dc00640 1 --2- 172.21.15.47:0/65050325 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feaf00a2f20 0x7feaf00a53e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feb081025d0 tx=0x7feafc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:08.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.379+0000 7feb06000640 1 -- 172.21.15.47:0/65050325 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feaf409bed0 con 0x7feb08102770 2024-03-21T11:21:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.583+0000 7feb0f000640 1 -- 172.21.15.47:0/65050325 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7feb080622a0 con 0x7feb08102770 2024-03-21T11:21:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.584+0000 7feb06000640 1 -- 172.21.15.47:0/65050325 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7feaf409ff70 con 0x7feb08102770 2024-03-21T11:21:08.585 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:21:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.587+0000 7feb0f000640 1 -- 172.21.15.47:0/65050325 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feaf00a2f20 msgr2=0x7feaf00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.587+0000 7feb0f000640 1 --2- 172.21.15.47:0/65050325 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feaf00a2f20 0x7feaf00a53e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feb081025d0 tx=0x7feafc005f90 comp rx=0 tx=0).stop 2024-03-21T11:21:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.587+0000 7feb0f000640 1 -- 172.21.15.47:0/65050325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb08102770 msgr2=0x7feb08073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.587+0000 7feb0f000640 1 --2- 172.21.15.47:0/65050325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb08102770 0x7feb08073a90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feaf400e9f0 tx=0x7feaf400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.587+0000 7feb0f000640 1 -- 172.21.15.47:0/65050325 shutdown_connections 2024-03-21T11:21:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.587+0000 7feb0f000640 1 --2- 172.21.15.47:0/65050325 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feaf00a2f20 0x7feaf00a53e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.587+0000 7feb0f000640 1 --2- 172.21.15.47:0/65050325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb08103130 0x7feb08073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.587+0000 7feb0f000640 1 --2- 172.21.15.47:0/65050325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb08102770 0x7feb08073a90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.587+0000 7feb0f000640 1 --2- 172.21.15.47:0/65050325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb08101580 0x7feb08075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.587+0000 7feb0f000640 1 -- 172.21.15.47:0/65050325 >> 172.21.15.47:0/65050325 conn(0x7feb080fcb10 msgr2=0x7feb08105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.587+0000 7feb0f000640 1 -- 172.21.15.47:0/65050325 shutdown_connections 2024-03-21T11:21:08.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.587+0000 7feb0f000640 1 -- 172.21.15.47:0/65050325 wait complete. 2024-03-21T11:21:08.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T11:21:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.837+0000 7f3b61e00640 1 -- 172.21.15.47:0/1114992209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b5c0772a0 msgr2=0x7f3b5c077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.837+0000 7f3b61e00640 1 --2- 172.21.15.47:0/1114992209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b5c0772a0 0x7f3b5c077700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009900 tx=0x7f3b4402bc30 comp rx=0 tx=0).stop 2024-03-21T11:21:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.837+0000 7f3b61e00640 1 -- 172.21.15.47:0/1114992209 shutdown_connections 2024-03-21T11:21:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.837+0000 7f3b61e00640 1 --2- 172.21.15.47:0/1114992209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b5c077c40 0x7f3b5c0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.837+0000 7f3b61e00640 1 --2- 172.21.15.47:0/1114992209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b5c0772a0 0x7f3b5c077700 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.837+0000 7f3b61e00640 1 --2- 172.21.15.47:0/1114992209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b5c076060 0x7f3b5c076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.837+0000 7f3b61e00640 1 -- 172.21.15.47:0/1114992209 >> 172.21.15.47:0/1114992209 conn(0x7f3b5c0fee50 msgr2=0x7f3b5c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.837+0000 7f3b61e00640 1 -- 172.21.15.47:0/1114992209 shutdown_connections 2024-03-21T11:21:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.837+0000 7f3b61e00640 1 -- 172.21.15.47:0/1114992209 wait complete. 2024-03-21T11:21:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.838+0000 7f3b61e00640 1 Processor -- start 2024-03-21T11:21:08.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.838+0000 7f3b61e00640 1 -- start start 2024-03-21T11:21:08.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.838+0000 7f3b61e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b5c076060 0x7f3b5c19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:08.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b61e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b5c0772a0 0x7f3b5c19e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:08.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b61e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b5c077c40 0x7f3b5c1a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:08.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b61e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c113930 con 0x7f3b5c0772a0 2024-03-21T11:21:08.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b61e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c1137b0 con 0x7f3b5c077c40 2024-03-21T11:21:08.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b61e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c113ab0 con 0x7f3b5c076060 2024-03-21T11:21:08.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b53e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b5c0772a0 0x7f3b5c19e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:08.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b53e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b5c0772a0 0x7f3b5c19e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33076/0 (socket says 172.21.15.47:33076) 2024-03-21T11:21:08.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b53e00640 1 -- 172.21.15.47:0/2426907492 learned_addr learned my addr 172.21.15.47:0/2426907492 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:08.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b5b400640 1 --2- 172.21.15.47:0/2426907492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b5c077c40 0x7f3b5c1a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:08.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b5aa00640 1 --2- 172.21.15.47:0/2426907492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b5c076060 0x7f3b5c19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:08.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b5b400640 1 -- 172.21.15.47:0/2426907492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b5c076060 msgr2=0x7f3b5c19e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b5b400640 1 --2- 172.21.15.47:0/2426907492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b5c076060 0x7f3b5c19e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b5b400640 1 -- 172.21.15.47:0/2426907492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b5c0772a0 msgr2=0x7f3b5c19e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b5b400640 1 --2- 172.21.15.47:0/2426907492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b5c0772a0 0x7f3b5c19e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.839+0000 7f3b5b400640 1 -- 172.21.15.47:0/2426907492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c1a5c20 con 0x7f3b5c077c40 2024-03-21T11:21:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.840+0000 7f3b5b400640 1 --2- 172.21.15.47:0/2426907492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b5c077c40 0x7f3b5c1a55b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3b54009fd0 tx=0x7f3b5400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.840+0000 7f3b60a00640 1 -- 172.21.15.47:0/2426907492 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5400ce20 con 0x7f3b5c077c40 2024-03-21T11:21:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.840+0000 7f3b60a00640 1 -- 172.21.15.47:0/2426907492 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b54005020 con 0x7f3b5c077c40 2024-03-21T11:21:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.840+0000 7f3b60a00640 1 -- 172.21.15.47:0/2426907492 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b54010720 con 0x7f3b5c077c40 2024-03-21T11:21:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.840+0000 7f3b61e00640 1 -- 172.21.15.47:0/2426907492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c1a5f10 con 0x7f3b5c077c40 2024-03-21T11:21:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.840+0000 7f3b61e00640 1 -- 172.21.15.47:0/2426907492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c1a6380 con 0x7f3b5c077c40 2024-03-21T11:21:08.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.841+0000 7f3b61e00640 1 -- 172.21.15.47:0/2426907492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5c110250 con 0x7f3b5c077c40 2024-03-21T11:21:08.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.843+0000 7f3b60a00640 1 -- 172.21.15.47:0/2426907492 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3b54005540 con 0x7f3b5c077c40 2024-03-21T11:21:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.844+0000 7f3b60a00640 1 --2- 172.21.15.47:0/2426907492 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b400a2f20 0x7f3b400a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.844+0000 7f3b60a00640 1 -- 172.21.15.47:0/2426907492 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f3b540cfd50 con 0x7f3b5c077c40 2024-03-21T11:21:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.845+0000 7f3b5aa00640 1 --2- 172.21.15.47:0/2426907492 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b400a2f20 0x7f3b400a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.845+0000 7f3b5aa00640 1 --2- 172.21.15.47:0/2426907492 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b400a2f20 0x7f3b400a53e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009bd0 tx=0x7f3b4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:08.849+0000 7f3b60a00640 1 -- 172.21.15.47:0/2426907492 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3b5409c8e0 con 0x7f3b5c077c40 2024-03-21T11:21:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:08 smithi047 ceph-mon[31351]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-21T11:21:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:08 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/942099899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:21:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:08 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/65050325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:21:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.067+0000 7f3b61e00640 1 -- 172.21.15.47:0/2426907492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3b5c0622a0 con 0x7f3b5c077c40 2024-03-21T11:21:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.068+0000 7f3b60a00640 1 -- 172.21.15.47:0/2426907492 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) v1 ==== 81+0+589 (secure 0 0 0) 0x7f3b5409c1e0 con 0x7f3b5c077c40 2024-03-21T11:21:09.068 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/4277663939 2024-03-22T11:12:48.837005+0000 2024-03-21T11:21:09.069 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/185270192 2024-03-22T11:12:48.837005+0000 2024-03-21T11:21:09.069 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/185270192 2024-03-22T11:12:48.837005+0000 2024-03-21T11:21:09.069 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1272635669 2024-03-22T11:12:48.837005+0000 2024-03-21T11:21:09.069 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2508263463 2024-03-22T11:12:14.713084+0000 2024-03-21T11:21:09.069 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/3773889790 2024-03-22T11:12:48.837005+0000 2024-03-21T11:21:09.069 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/1584030551 2024-03-22T11:12:14.713084+0000 2024-03-21T11:21:09.069 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/3130329909 2024-03-22T11:12:14.713084+0000 2024-03-21T11:21:09.069 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/1584030551 2024-03-22T11:12:14.713084+0000 2024-03-21T11:21:09.069 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/335539055 2024-03-22T11:12:14.713084+0000 2024-03-21T11:21:09.069 INFO:teuthology.orchestra.run.smithi047.stderr:listed 10 entries 2024-03-21T11:21:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.071+0000 7f3b61e00640 1 -- 172.21.15.47:0/2426907492 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b400a2f20 msgr2=0x7f3b400a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.071+0000 7f3b61e00640 1 --2- 172.21.15.47:0/2426907492 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b400a2f20 0x7f3b400a53e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009bd0 tx=0x7f3b4c005f90 comp rx=0 tx=0).stop 2024-03-21T11:21:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.071+0000 7f3b61e00640 1 -- 172.21.15.47:0/2426907492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b5c077c40 msgr2=0x7f3b5c1a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.071+0000 7f3b61e00640 1 --2- 172.21.15.47:0/2426907492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b5c077c40 0x7f3b5c1a55b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3b54009fd0 tx=0x7f3b5400ef10 comp rx=0 tx=0).stop 2024-03-21T11:21:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.071+0000 7f3b61e00640 1 -- 172.21.15.47:0/2426907492 shutdown_connections 2024-03-21T11:21:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.071+0000 7f3b61e00640 1 --2- 172.21.15.47:0/2426907492 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b400a2f20 0x7f3b400a53e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.071+0000 7f3b61e00640 1 --2- 172.21.15.47:0/2426907492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b5c077c40 0x7f3b5c1a55b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.071+0000 7f3b61e00640 1 --2- 172.21.15.47:0/2426907492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b5c0772a0 0x7f3b5c19e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.071+0000 7f3b61e00640 1 --2- 172.21.15.47:0/2426907492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b5c076060 0x7f3b5c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.071+0000 7f3b61e00640 1 -- 172.21.15.47:0/2426907492 >> 172.21.15.47:0/2426907492 conn(0x7f3b5c0fee50 msgr2=0x7f3b5c100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.071+0000 7f3b61e00640 1 -- 172.21.15.47:0/2426907492 shutdown_connections 2024-03-21T11:21:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.071+0000 7f3b61e00640 1 -- 172.21.15.47:0/2426907492 wait complete. 2024-03-21T11:21:09.089 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:21:09.090 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T11:21:09.141 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:21:09.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:08 smithi100 ceph-mon[36245]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-21T11:21:09.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:08 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/942099899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:21:09.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:08 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/65050325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:21:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:08 smithi102 ceph-mon[36748]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-21T11:21:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:08 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/942099899' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:21:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:08 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/65050325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:21:09.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.409+0000 7fd9cb200640 1 -- 172.21.15.47:0/3694147560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9cc1036f0 msgr2=0x7fd9cc103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:09.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.409+0000 7fd9cb200640 1 --2- 172.21.15.47:0/3694147560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9cc1036f0 0x7fd9cc103b10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009900 tx=0x7fd9bc02bc40 comp rx=0 tx=0).stop 2024-03-21T11:21:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.409+0000 7fd9cb200640 1 -- 172.21.15.47:0/3694147560 shutdown_connections 2024-03-21T11:21:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.409+0000 7fd9cb200640 1 --2- 172.21.15.47:0/3694147560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9cc1052a0 0x7fd9cc10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.409+0000 7fd9cb200640 1 --2- 172.21.15.47:0/3694147560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9cc1048e0 0x7fd9cc104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.409+0000 7fd9cb200640 1 --2- 172.21.15.47:0/3694147560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9cc1036f0 0x7fd9cc103b10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.409+0000 7fd9cb200640 1 -- 172.21.15.47:0/3694147560 >> 172.21.15.47:0/3694147560 conn(0x7fd9cc0fec80 msgr2=0x7fd9cc1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.409+0000 7fd9cb200640 1 -- 172.21.15.47:0/3694147560 shutdown_connections 2024-03-21T11:21:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.410+0000 7fd9cb200640 1 -- 172.21.15.47:0/3694147560 wait complete. 2024-03-21T11:21:09.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.410+0000 7fd9cb200640 1 Processor -- start 2024-03-21T11:21:09.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.411+0000 7fd9cb200640 1 -- start start 2024-03-21T11:21:09.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.411+0000 7fd9cb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9cc1036f0 0x7fd9cc199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:09.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.411+0000 7fd9cb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9cc1048e0 0x7fd9cc199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:09.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.411+0000 7fd9cb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9cc1052a0 0x7fd9cc1a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:09.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.411+0000 7fd9cb200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc10f390 con 0x7fd9cc1052a0 2024-03-21T11:21:09.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.411+0000 7fd9cb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc10f210 con 0x7fd9cc1036f0 2024-03-21T11:21:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.411+0000 7fd9cb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc10f510 con 0x7fd9cc1048e0 2024-03-21T11:21:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.411+0000 7fd9c9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9cc1036f0 0x7fd9cc199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.411+0000 7fd9c9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9cc1036f0 0x7fd9cc199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47900/0 (socket says 172.21.15.47:47900) 2024-03-21T11:21:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.411+0000 7fd9c9e00640 1 -- 172.21.15.47:0/3814282956 learned_addr learned my addr 172.21.15.47:0/3814282956 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.411+0000 7fd9ca800640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9cc1052a0 0x7fd9cc1a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.411+0000 7fd9c1400640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9cc1048e0 0x7fd9cc199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.412+0000 7fd9c9e00640 1 -- 172.21.15.47:0/3814282956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9cc1048e0 msgr2=0x7fd9cc199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.412+0000 7fd9c9e00640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9cc1048e0 0x7fd9cc199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.412+0000 7fd9c9e00640 1 -- 172.21.15.47:0/3814282956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9cc1052a0 msgr2=0x7fd9cc1a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.412+0000 7fd9c9e00640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9cc1052a0 0x7fd9cc1a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.412+0000 7fd9c9e00640 1 -- 172.21.15.47:0/3814282956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc1a1660 con 0x7fd9cc1036f0 2024-03-21T11:21:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.412+0000 7fd9c1400640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9cc1048e0 0x7fd9cc199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:21:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.412+0000 7fd9ca800640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9cc1052a0 0x7fd9cc1a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:21:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.412+0000 7fd9c9e00640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9cc1036f0 0x7fd9cc199a40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc02c8f0 tx=0x7fd9bc02c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.412+0000 7fd9c3400640 1 -- 172.21.15.47:0/3814282956 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc041070 con 0x7fd9cc1036f0 2024-03-21T11:21:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.412+0000 7fd9c3400640 1 -- 172.21.15.47:0/3814282956 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9bc004450 con 0x7fd9cc1036f0 2024-03-21T11:21:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.412+0000 7fd9cb200640 1 -- 172.21.15.47:0/3814282956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9cc1a18f0 con 0x7fd9cc1036f0 2024-03-21T11:21:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.412+0000 7fd9c3400640 1 -- 172.21.15.47:0/3814282956 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc034440 con 0x7fd9cc1036f0 2024-03-21T11:21:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.413+0000 7fd9cb200640 1 -- 172.21.15.47:0/3814282956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9cc1a1de0 con 0x7fd9cc1036f0 2024-03-21T11:21:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.414+0000 7fd9cb200640 1 -- 172.21.15.47:0/3814282956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9cc10a9b0 con 0x7fd9cc1036f0 2024-03-21T11:21:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.415+0000 7fd9c3400640 1 -- 172.21.15.47:0/3814282956 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd9bc044070 con 0x7fd9cc1036f0 2024-03-21T11:21:09.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.417+0000 7fd9c3400640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9b00a2f20 0x7fd9b00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:09.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.417+0000 7fd9c3400640 1 -- 172.21.15.47:0/3814282956 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd9bc0f12b0 con 0x7fd9cc1036f0 2024-03-21T11:21:09.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.417+0000 7fd9c1400640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9b00a2f20 0x7fd9b00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.418+0000 7fd9c1400640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9b00a2f20 0x7fd9b00a53e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc19af50 tx=0x7fd9b4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:09.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.420+0000 7fd9c3400640 1 -- 172.21.15.47:0/3814282956 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd9bc0bdea0 con 0x7fd9cc1036f0 2024-03-21T11:21:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.628+0000 7fd9cb200640 1 -- 172.21.15.47:0/3814282956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9cc0622d0 con 0x7fd9cc1036f0 2024-03-21T11:21:09.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.631+0000 7fd9c3400640 1 -- 172.21.15.47:0/3814282956 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24127 (secure 0 0 0) 0x7fd9bc0bd7a0 con 0x7fd9cc1036f0 2024-03-21T11:21:09.631 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:21:09.631 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:20:54.191030+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:21:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.633+0000 7fd9cb200640 1 -- 172.21.15.47:0/3814282956 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9b00a2f20 msgr2=0x7fd9b00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.633+0000 7fd9cb200640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9b00a2f20 0x7fd9b00a53e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc19af50 tx=0x7fd9b4009500 comp rx=0 tx=0).stop 2024-03-21T11:21:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.633+0000 7fd9cb200640 1 -- 172.21.15.47:0/3814282956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9cc1036f0 msgr2=0x7fd9cc199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.633+0000 7fd9cb200640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9cc1036f0 0x7fd9cc199a40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc02c8f0 tx=0x7fd9bc02c660 comp rx=0 tx=0).stop 2024-03-21T11:21:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.633+0000 7fd9cb200640 1 -- 172.21.15.47:0/3814282956 shutdown_connections 2024-03-21T11:21:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.633+0000 7fd9cb200640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9b00a2f20 0x7fd9b00a53e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.633+0000 7fd9cb200640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9cc1052a0 0x7fd9cc1a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.633+0000 7fd9cb200640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9cc1048e0 0x7fd9cc199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.633+0000 7fd9cb200640 1 --2- 172.21.15.47:0/3814282956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9cc1036f0 0x7fd9cc199a40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.633+0000 7fd9cb200640 1 -- 172.21.15.47:0/3814282956 >> 172.21.15.47:0/3814282956 conn(0x7fd9cc0fec80 msgr2=0x7fd9cc107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.633+0000 7fd9cb200640 1 -- 172.21.15.47:0/3814282956 shutdown_connections 2024-03-21T11:21:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.633+0000 7fd9cb200640 1 -- 172.21.15.47:0/3814282956 wait complete. 2024-03-21T11:21:09.654 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T11:15:35.545245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T11:20:47.016174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T11:20:48.108506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T11:20:52.254154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T11:21:09.655 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T11:21:09.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.890+0000 7f496a000640 1 -- 172.21.15.47:0/576353589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4964107bc0 msgr2=0x7f4964107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.890+0000 7f496a000640 1 --2- 172.21.15.47:0/576353589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4964107bc0 0x7f4964107fe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4958009900 tx=0x7f495802bc30 comp rx=0 tx=0).stop 2024-03-21T11:21:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.891+0000 7f496a000640 1 -- 172.21.15.47:0/576353589 shutdown_connections 2024-03-21T11:21:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.891+0000 7f496a000640 1 --2- 172.21.15.47:0/576353589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4964109770 0x7f4964110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.891+0000 7f496a000640 1 --2- 172.21.15.47:0/576353589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4964108db0 0x7f4964109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.891+0000 7f496a000640 1 --2- 172.21.15.47:0/576353589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4964107bc0 0x7f4964107fe0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.891+0000 7f496a000640 1 -- 172.21.15.47:0/576353589 >> 172.21.15.47:0/576353589 conn(0x7f4964076b70 msgr2=0x7f4964078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.891+0000 7f496a000640 1 -- 172.21.15.47:0/576353589 shutdown_connections 2024-03-21T11:21:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.891+0000 7f496a000640 1 -- 172.21.15.47:0/576353589 wait complete. 2024-03-21T11:21:09.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.892+0000 7f496a000640 1 Processor -- start 2024-03-21T11:21:09.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.892+0000 7f496a000640 1 -- start start 2024-03-21T11:21:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.892+0000 7f496a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4964107bc0 0x7f496419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.892+0000 7f496a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4964108db0 0x7f496419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.892+0000 7f496a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4964109770 0x7f49641a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.892+0000 7f496a000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4964113890 con 0x7f4964107bc0 2024-03-21T11:21:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.892+0000 7f496a000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4964113710 con 0x7f4964108db0 2024-03-21T11:21:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.892+0000 7f496a000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4964113a10 con 0x7f4964109770 2024-03-21T11:21:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.893+0000 7f4963e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4964108db0 0x7f496419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.893+0000 7f4968c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4964107bc0 0x7f496419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.893+0000 7f4963e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4964108db0 0x7f496419e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47916/0 (socket says 172.21.15.47:47916) 2024-03-21T11:21:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.893+0000 7f4963e00640 1 -- 172.21.15.47:0/1152301390 learned_addr learned my addr 172.21.15.47:0/1152301390 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.893+0000 7f4969600640 1 --2- 172.21.15.47:0/1152301390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4964109770 0x7f49641a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.893+0000 7f4963e00640 1 -- 172.21.15.47:0/1152301390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4964109770 msgr2=0x7f49641a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.893+0000 7f4963e00640 1 --2- 172.21.15.47:0/1152301390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4964109770 0x7f49641a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.893+0000 7f4963e00640 1 -- 172.21.15.47:0/1152301390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4964107bc0 msgr2=0x7f496419dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:09.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.893+0000 7f4963e00640 1 --2- 172.21.15.47:0/1152301390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4964107bc0 0x7f496419dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:09.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.893+0000 7f4963e00640 1 -- 172.21.15.47:0/1152301390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49641a5ac0 con 0x7f4964108db0 2024-03-21T11:21:09.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.893+0000 7f4968c00640 1 --2- 172.21.15.47:0/1152301390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4964107bc0 0x7f496419dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:21:09.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.893+0000 7f4963e00640 1 --2- 172.21.15.47:0/1152301390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4964108db0 0x7f496419e3e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f495000e9f0 tx=0x7f495000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:09.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.894+0000 7f4961600640 1 -- 172.21.15.47:0/1152301390 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495000cd80 con 0x7f4964108db0 2024-03-21T11:21:09.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.894+0000 7f4961600640 1 -- 172.21.15.47:0/1152301390 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4950010910 con 0x7f4964108db0 2024-03-21T11:21:09.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.894+0000 7f4961600640 1 -- 172.21.15.47:0/1152301390 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4950018990 con 0x7f4964108db0 2024-03-21T11:21:09.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.894+0000 7f496a000640 1 -- 172.21.15.47:0/1152301390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49641a5db0 con 0x7f4964108db0 2024-03-21T11:21:09.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.894+0000 7f496a000640 1 -- 172.21.15.47:0/1152301390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49641a6220 con 0x7f4964108db0 2024-03-21T11:21:09.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.895+0000 7f496a000640 1 -- 172.21.15.47:0/1152301390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f496410eee0 con 0x7f4964108db0 2024-03-21T11:21:09.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.897+0000 7f4961600640 1 -- 172.21.15.47:0/1152301390 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4950010ac0 con 0x7f4964108db0 2024-03-21T11:21:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.899+0000 7f4961600640 1 --2- 172.21.15.47:0/1152301390 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f494c0a2fa0 0x7f494c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.899+0000 7f4961600640 1 -- 172.21.15.47:0/1152301390 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f49500cf8c0 con 0x7f4964108db0 2024-03-21T11:21:09.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.899+0000 7f4968c00640 1 --2- 172.21.15.47:0/1152301390 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f494c0a2fa0 0x7f494c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.900+0000 7f4968c00640 1 --2- 172.21.15.47:0/1152301390 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f494c0a2fa0 0x7f494c0a5460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f495802c170 tx=0x7f495802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:09.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:09.903+0000 7f4961600640 1 -- 172.21.15.47:0/1152301390 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f495009be00 con 0x7f4964108db0 2024-03-21T11:21:10.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:09 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2426907492' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T11:21:10.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:09 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3814282956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:10.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.108+0000 7f496a000640 1 -- 172.21.15.47:0/1152301390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4964062340 con 0x7f4964108db0 2024-03-21T11:21:10.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.109+0000 7f4961600640 1 -- 172.21.15.47:0/1152301390 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f495009fea0 con 0x7f4964108db0 2024-03-21T11:21:10.110 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T11:21:10.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.111+0000 7f496a000640 1 -- 172.21.15.47:0/1152301390 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f494c0a2fa0 msgr2=0x7f494c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:10.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.111+0000 7f496a000640 1 --2- 172.21.15.47:0/1152301390 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f494c0a2fa0 0x7f494c0a5460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f495802c170 tx=0x7f495802df90 comp rx=0 tx=0).stop 2024-03-21T11:21:10.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.112+0000 7f496a000640 1 -- 172.21.15.47:0/1152301390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4964108db0 msgr2=0x7f496419e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:10.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.112+0000 7f496a000640 1 --2- 172.21.15.47:0/1152301390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4964108db0 0x7f496419e3e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f495000e9f0 tx=0x7f495000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.112+0000 7f496a000640 1 -- 172.21.15.47:0/1152301390 shutdown_connections 2024-03-21T11:21:10.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.112+0000 7f496a000640 1 --2- 172.21.15.47:0/1152301390 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f494c0a2fa0 0x7f494c0a5460 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.112+0000 7f496a000640 1 --2- 172.21.15.47:0/1152301390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4964109770 0x7f49641a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.112+0000 7f496a000640 1 --2- 172.21.15.47:0/1152301390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4964108db0 0x7f496419e3e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.112+0000 7f496a000640 1 --2- 172.21.15.47:0/1152301390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4964107bc0 0x7f496419dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.112+0000 7f496a000640 1 -- 172.21.15.47:0/1152301390 >> 172.21.15.47:0/1152301390 conn(0x7f4964076b70 msgr2=0x7f496410c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:10.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.112+0000 7f496a000640 1 -- 172.21.15.47:0/1152301390 shutdown_connections 2024-03-21T11:21:10.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.112+0000 7f496a000640 1 -- 172.21.15.47:0/1152301390 wait complete. 2024-03-21T11:21:10.134 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T11:21:10.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:09 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2426907492' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T11:21:10.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:09 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3814282956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:10.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:09 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2426907492' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T11:21:10.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:09 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3814282956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:10.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.368+0000 7fbb76600640 1 -- 172.21.15.47:0/1847878538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb70101580 msgr2=0x7fbb701019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:10.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.368+0000 7fbb76600640 1 --2- 172.21.15.47:0/1847878538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb70101580 0x7fbb701019a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009ca0 tx=0x7fbb6002c0f0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.369+0000 7fbb76600640 1 -- 172.21.15.47:0/1847878538 shutdown_connections 2024-03-21T11:21:10.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.369+0000 7fbb76600640 1 --2- 172.21.15.47:0/1847878538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb70103130 0x7fbb701099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.369+0000 7fbb76600640 1 --2- 172.21.15.47:0/1847878538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb70102770 0x7fbb70102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.369+0000 7fbb76600640 1 --2- 172.21.15.47:0/1847878538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb70101580 0x7fbb701019a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.369+0000 7fbb76600640 1 -- 172.21.15.47:0/1847878538 >> 172.21.15.47:0/1847878538 conn(0x7fbb700fcb10 msgr2=0x7fbb700fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:10.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.369+0000 7fbb76600640 1 -- 172.21.15.47:0/1847878538 shutdown_connections 2024-03-21T11:21:10.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.369+0000 7fbb76600640 1 -- 172.21.15.47:0/1847878538 wait complete. 2024-03-21T11:21:10.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.370+0000 7fbb76600640 1 Processor -- start 2024-03-21T11:21:10.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.370+0000 7fbb76600640 1 -- start start 2024-03-21T11:21:10.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.370+0000 7fbb76600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb70101580 0x7fbb701978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:10.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.370+0000 7fbb76600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb70102770 0x7fbb70197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:10.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb76600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb70103130 0x7fbb7019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:10.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb76600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7010d210 con 0x7fbb70101580 2024-03-21T11:21:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb76600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7010d090 con 0x7fbb70102770 2024-03-21T11:21:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb76600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7010d390 con 0x7fbb70103130 2024-03-21T11:21:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb70102770 0x7fbb70197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb70102770 0x7fbb70197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47920/0 (socket says 172.21.15.47:47920) 2024-03-21T11:21:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb6fe00640 1 -- 172.21.15.47:0/1082981253 learned_addr learned my addr 172.21.15.47:0/1082981253 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb75200640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb70101580 0x7fbb701978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb75c00640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb70103130 0x7fbb7019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb6fe00640 1 -- 172.21.15.47:0/1082981253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb70103130 msgr2=0x7fbb7019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb6fe00640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb70103130 0x7fbb7019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb6fe00640 1 -- 172.21.15.47:0/1082981253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb70101580 msgr2=0x7fbb701978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb6fe00640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb70101580 0x7fbb701978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb6fe00640 1 -- 172.21.15.47:0/1082981253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7019f4e0 con 0x7fbb70102770 2024-03-21T11:21:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.371+0000 7fbb75200640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb70101580 0x7fbb701978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:21:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.372+0000 7fbb6fe00640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb70102770 0x7fbb70197e00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00e9f0 tx=0x7fbb5c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:10.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.372+0000 7fbb6d600640 1 -- 172.21.15.47:0/1082981253 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c00cd80 con 0x7fbb70102770 2024-03-21T11:21:10.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.372+0000 7fbb6d600640 1 -- 172.21.15.47:0/1082981253 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb5c005500 con 0x7fbb70102770 2024-03-21T11:21:10.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.372+0000 7fbb6d600640 1 -- 172.21.15.47:0/1082981253 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c010690 con 0x7fbb70102770 2024-03-21T11:21:10.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.372+0000 7fbb76600640 1 -- 172.21.15.47:0/1082981253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7019f7d0 con 0x7fbb70102770 2024-03-21T11:21:10.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.372+0000 7fbb76600640 1 -- 172.21.15.47:0/1082981253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7019fd20 con 0x7fbb70102770 2024-03-21T11:21:10.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.373+0000 7fbb76600640 1 -- 172.21.15.47:0/1082981253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb701088a0 con 0x7fbb70102770 2024-03-21T11:21:10.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.376+0000 7fbb6d600640 1 -- 172.21.15.47:0/1082981253 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbb5c0046c0 con 0x7fbb70102770 2024-03-21T11:21:10.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.378+0000 7fbb6d600640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb540a2fa0 0x7fbb540a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:10.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.378+0000 7fbb6d600640 1 -- 172.21.15.47:0/1082981253 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fbb5c0cfb40 con 0x7fbb70102770 2024-03-21T11:21:10.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.378+0000 7fbb75200640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb540a2fa0 0x7fbb540a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:10.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.378+0000 7fbb75200640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb540a2fa0 0x7fbb540a5460 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbb6002c700 tx=0x7fbb600325b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.382+0000 7fbb6d600640 1 -- 172.21.15.47:0/1082981253 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbb5c09c6d0 con 0x7fbb70102770 2024-03-21T11:21:10.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.585+0000 7fbb76600640 1 -- 172.21.15.47:0/1082981253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb700622a0 con 0x7fbb70102770 2024-03-21T11:21:10.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.585+0000 7fbb6d600640 1 -- 172.21.15.47:0/1082981253 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbb5c09bfd0 con 0x7fbb70102770 2024-03-21T11:21:10.586 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:21:10.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.588+0000 7fbb76600640 1 -- 172.21.15.47:0/1082981253 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb540a2fa0 msgr2=0x7fbb540a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:10.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.588+0000 7fbb76600640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb540a2fa0 0x7fbb540a5460 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbb6002c700 tx=0x7fbb600325b0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.588+0000 7fbb76600640 1 -- 172.21.15.47:0/1082981253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb70102770 msgr2=0x7fbb70197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:10.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.588+0000 7fbb76600640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb70102770 0x7fbb70197e00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00e9f0 tx=0x7fbb5c00edf0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.588+0000 7fbb76600640 1 -- 172.21.15.47:0/1082981253 shutdown_connections 2024-03-21T11:21:10.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.588+0000 7fbb76600640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb540a2fa0 0x7fbb540a5460 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.588+0000 7fbb76600640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb70103130 0x7fbb7019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.588+0000 7fbb76600640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb70102770 0x7fbb70197e00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.588+0000 7fbb76600640 1 --2- 172.21.15.47:0/1082981253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb70101580 0x7fbb701978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.588+0000 7fbb76600640 1 -- 172.21.15.47:0/1082981253 >> 172.21.15.47:0/1082981253 conn(0x7fbb700fcb10 msgr2=0x7fbb70105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:10.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.589+0000 7fbb76600640 1 -- 172.21.15.47:0/1082981253 shutdown_connections 2024-03-21T11:21:10.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.589+0000 7fbb76600640 1 -- 172.21.15.47:0/1082981253 wait complete. 2024-03-21T11:21:10.609 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T11:21:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.834+0000 7fd3a1000640 1 -- 172.21.15.47:0/996874152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd39c105350 msgr2=0x7fd39c10bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.834+0000 7fd3a1000640 1 --2- 172.21.15.47:0/996874152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd39c105350 0x7fd39c10bbe0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd394009ca0 tx=0x7fd39402c0f0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.834+0000 7fd3a1000640 1 -- 172.21.15.47:0/996874152 shutdown_connections 2024-03-21T11:21:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.834+0000 7fd3a1000640 1 --2- 172.21.15.47:0/996874152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd39c105350 0x7fd39c10bbe0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.834+0000 7fd3a1000640 1 --2- 172.21.15.47:0/996874152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd39c104990 0x7fd39c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.834+0000 7fd3a1000640 1 --2- 172.21.15.47:0/996874152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd39c1037a0 0x7fd39c103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.834+0000 7fd3a1000640 1 -- 172.21.15.47:0/996874152 >> 172.21.15.47:0/996874152 conn(0x7fd39c0fed10 msgr2=0x7fd39c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.835+0000 7fd3a1000640 1 -- 172.21.15.47:0/996874152 shutdown_connections 2024-03-21T11:21:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.835+0000 7fd3a1000640 1 -- 172.21.15.47:0/996874152 wait complete. 2024-03-21T11:21:10.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.835+0000 7fd3a1000640 1 Processor -- start 2024-03-21T11:21:10.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.836+0000 7fd3a1000640 1 -- start start 2024-03-21T11:21:10.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.836+0000 7fd3a1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd39c1037a0 0x7fd39c199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:10.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.836+0000 7fd3a1000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd39c104990 0x7fd39c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:10.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.836+0000 7fd3a1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd39c105350 0x7fd39c1a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:10.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.836+0000 7fd3a1000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c10f420 con 0x7fd39c104990 2024-03-21T11:21:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.836+0000 7fd3a1000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c10f2a0 con 0x7fd39c105350 2024-03-21T11:21:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.836+0000 7fd3a1000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c10f5a0 con 0x7fd39c1037a0 2024-03-21T11:21:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.836+0000 7fd39aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd39c104990 0x7fd39c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.836+0000 7fd39b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd39c1037a0 0x7fd39c199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.836+0000 7fd39be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd39c105350 0x7fd39c1a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.836+0000 7fd39aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd39c104990 0x7fd39c199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37486/0 (socket says 172.21.15.47:37486) 2024-03-21T11:21:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.836+0000 7fd39be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd39c105350 0x7fd39c1a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47936/0 (socket says 172.21.15.47:47936) 2024-03-21T11:21:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.836+0000 7fd39aa00640 1 -- 172.21.15.47:0/1180895981 learned_addr learned my addr 172.21.15.47:0/1180895981 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.837+0000 7fd39be00640 1 -- 172.21.15.47:0/1180895981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd39c1037a0 msgr2=0x7fd39c199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.837+0000 7fd39be00640 1 --2- 172.21.15.47:0/1180895981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd39c1037a0 0x7fd39c199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.837+0000 7fd39be00640 1 -- 172.21.15.47:0/1180895981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd39c104990 msgr2=0x7fd39c199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.837+0000 7fd39be00640 1 --2- 172.21.15.47:0/1180895981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd39c104990 0x7fd39c199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.837+0000 7fd39be00640 1 -- 172.21.15.47:0/1180895981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39c1a1680 con 0x7fd39c105350 2024-03-21T11:21:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.837+0000 7fd39be00640 1 --2- 172.21.15.47:0/1180895981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd39c105350 0x7fd39c1a1010 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd394037040 tx=0x7fd394032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.837+0000 7fd393e00640 1 -- 172.21.15.47:0/1180895981 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd394041020 con 0x7fd39c105350 2024-03-21T11:21:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.837+0000 7fd393e00640 1 -- 172.21.15.47:0/1180895981 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd394034cd0 con 0x7fd39c105350 2024-03-21T11:21:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.837+0000 7fd393e00640 1 -- 172.21.15.47:0/1180895981 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd394039040 con 0x7fd39c105350 2024-03-21T11:21:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.837+0000 7fd3a1000640 1 -- 172.21.15.47:0/1180895981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39c1a1910 con 0x7fd39c105350 2024-03-21T11:21:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.837+0000 7fd3a1000640 1 -- 172.21.15.47:0/1180895981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39c1a1e20 con 0x7fd39c105350 2024-03-21T11:21:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.839+0000 7fd3a1000640 1 -- 172.21.15.47:0/1180895981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39c10aac0 con 0x7fd39c105350 2024-03-21T11:21:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.840+0000 7fd393e00640 1 -- 172.21.15.47:0/1180895981 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd394044020 con 0x7fd39c105350 2024-03-21T11:21:10.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.842+0000 7fd393e00640 1 --2- 172.21.15.47:0/1180895981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd3800a2fa0 0x7fd3800a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:10.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.842+0000 7fd393e00640 1 -- 172.21.15.47:0/1180895981 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd3940f13a0 con 0x7fd39c105350 2024-03-21T11:21:10.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.842+0000 7fd39b400640 1 --2- 172.21.15.47:0/1180895981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd3800a2fa0 0x7fd3800a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:10.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.843+0000 7fd39b400640 1 --2- 172.21.15.47:0/1180895981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd3800a2fa0 0x7fd3800a5460 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd39c1047f0 tx=0x7fd38c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:10.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:10.846+0000 7fd393e00640 1 -- 172.21.15.47:0/1180895981 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd39404b020 con 0x7fd39c105350 2024-03-21T11:21:11.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.050+0000 7fd3a1000640 1 -- 172.21.15.47:0/1180895981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd39c0622a0 con 0x7fd39c105350 2024-03-21T11:21:11.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.051+0000 7fd393e00640 1 -- 172.21.15.47:0/1180895981 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd3940bdfc0 con 0x7fd39c105350 2024-03-21T11:21:11.051 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T11:21:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.054+0000 7fd3a1000640 1 -- 172.21.15.47:0/1180895981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd3800a2fa0 msgr2=0x7fd3800a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.054+0000 7fd3a1000640 1 --2- 172.21.15.47:0/1180895981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd3800a2fa0 0x7fd3800a5460 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd39c1047f0 tx=0x7fd38c005f90 comp rx=0 tx=0).stop 2024-03-21T11:21:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.054+0000 7fd3a1000640 1 -- 172.21.15.47:0/1180895981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd39c105350 msgr2=0x7fd39c1a1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:11.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.054+0000 7fd3a1000640 1 --2- 172.21.15.47:0/1180895981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd39c105350 0x7fd39c1a1010 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd394037040 tx=0x7fd394032ea0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.054+0000 7fd3a1000640 1 -- 172.21.15.47:0/1180895981 shutdown_connections 2024-03-21T11:21:11.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.054+0000 7fd3a1000640 1 --2- 172.21.15.47:0/1180895981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd3800a2fa0 0x7fd3800a5460 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.054+0000 7fd3a1000640 1 --2- 172.21.15.47:0/1180895981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd39c105350 0x7fd39c1a1010 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.054+0000 7fd3a1000640 1 --2- 172.21.15.47:0/1180895981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd39c104990 0x7fd39c199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.054+0000 7fd3a1000640 1 --2- 172.21.15.47:0/1180895981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd39c1037a0 0x7fd39c199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.054+0000 7fd3a1000640 1 -- 172.21.15.47:0/1180895981 >> 172.21.15.47:0/1180895981 conn(0x7fd39c0fed10 msgr2=0x7fd39c107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:11.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.054+0000 7fd3a1000640 1 -- 172.21.15.47:0/1180895981 shutdown_connections 2024-03-21T11:21:11.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.054+0000 7fd3a1000640 1 -- 172.21.15.47:0/1180895981 wait complete. 2024-03-21T11:21:11.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:10 smithi047 ceph-mon[31351]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-21T11:21:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:10 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1152301390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:21:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:10 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1082981253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:21:11.074 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T11:21:11.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:10 smithi100 ceph-mon[36245]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-21T11:21:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:10 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1152301390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:21:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:10 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1082981253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:21:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:10 smithi102 ceph-mon[36748]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-21T11:21:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:10 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1152301390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:21:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:10 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1082981253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:21:11.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.300+0000 7f9a20a00640 1 -- 172.21.15.47:0/2271771716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a1c105490 msgr2=0x7f9a1c10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:11.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.300+0000 7f9a20a00640 1 --2- 172.21.15.47:0/2271771716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a1c105490 0x7f9a1c10bd20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009ca0 tx=0x7f9a1402c100 comp rx=0 tx=0).stop 2024-03-21T11:21:11.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.300+0000 7f9a20a00640 1 -- 172.21.15.47:0/2271771716 shutdown_connections 2024-03-21T11:21:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.300+0000 7f9a20a00640 1 --2- 172.21.15.47:0/2271771716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a1c105490 0x7f9a1c10bd20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.300+0000 7f9a20a00640 1 --2- 172.21.15.47:0/2271771716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c104ad0 0x7f9a1c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.300+0000 7f9a20a00640 1 --2- 172.21.15.47:0/2271771716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a1c1038e0 0x7f9a1c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.300+0000 7f9a20a00640 1 -- 172.21.15.47:0/2271771716 >> 172.21.15.47:0/2271771716 conn(0x7f9a1c0fee50 msgr2=0x7f9a1c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.300+0000 7f9a20a00640 1 -- 172.21.15.47:0/2271771716 shutdown_connections 2024-03-21T11:21:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.300+0000 7f9a20a00640 1 -- 172.21.15.47:0/2271771716 wait complete. 2024-03-21T11:21:11.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.301+0000 7f9a20a00640 1 Processor -- start 2024-03-21T11:21:11.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.302+0000 7f9a20a00640 1 -- start start 2024-03-21T11:21:11.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.302+0000 7f9a20a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c1038e0 0x7f9a1c199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:11.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.302+0000 7f9a20a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a1c104ad0 0x7f9a1c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:11.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.302+0000 7f9a20a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a1c105490 0x7f9a1c1a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:11.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.302+0000 7f9a20a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c10f540 con 0x7f9a1c1038e0 2024-03-21T11:21:11.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.302+0000 7f9a20a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c10f3c0 con 0x7f9a1c104ad0 2024-03-21T11:21:11.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.302+0000 7f9a20a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c10f6c0 con 0x7f9a1c105490 2024-03-21T11:21:11.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.302+0000 7f9a19400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a1c104ad0 0x7f9a1c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:11.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.302+0000 7f9a19e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c1038e0 0x7f9a1c199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:11.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.302+0000 7f9a19400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a1c104ad0 0x7f9a1c19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47952/0 (socket says 172.21.15.47:47952) 2024-03-21T11:21:11.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.302+0000 7f9a19400640 1 -- 172.21.15.47:0/2518814734 learned_addr learned my addr 172.21.15.47:0/2518814734 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:11.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.302+0000 7f9a1a800640 1 --2- 172.21.15.47:0/2518814734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a1c105490 0x7f9a1c1a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.303+0000 7f9a19400640 1 -- 172.21.15.47:0/2518814734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a1c105490 msgr2=0x7f9a1c1a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.303+0000 7f9a19400640 1 --2- 172.21.15.47:0/2518814734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a1c105490 0x7f9a1c1a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.303+0000 7f9a19400640 1 -- 172.21.15.47:0/2518814734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c1038e0 msgr2=0x7f9a1c199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.303+0000 7f9a19400640 1 --2- 172.21.15.47:0/2518814734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c1038e0 0x7f9a1c199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.303+0000 7f9a19400640 1 -- 172.21.15.47:0/2518814734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c1a1770 con 0x7f9a1c104ad0 2024-03-21T11:21:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.303+0000 7f9a19400640 1 --2- 172.21.15.47:0/2518814734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a1c104ad0 0x7f9a1c19a090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400d950 tx=0x7f9a0400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.303+0000 7f9a12a00640 1 -- 172.21.15.47:0/2518814734 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04017030 con 0x7f9a1c104ad0 2024-03-21T11:21:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.303+0000 7f9a12a00640 1 -- 172.21.15.47:0/2518814734 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a04004020 con 0x7f9a1c104ad0 2024-03-21T11:21:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.303+0000 7f9a20a00640 1 -- 172.21.15.47:0/2518814734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c1a1a60 con 0x7f9a1c104ad0 2024-03-21T11:21:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.303+0000 7f9a12a00640 1 -- 172.21.15.47:0/2518814734 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040047a0 con 0x7f9a1c104ad0 2024-03-21T11:21:11.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.303+0000 7f9a20a00640 1 -- 172.21.15.47:0/2518814734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c1a2000 con 0x7f9a1c104ad0 2024-03-21T11:21:11.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.305+0000 7f9a20a00640 1 -- 172.21.15.47:0/2518814734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c10aba0 con 0x7f9a1c104ad0 2024-03-21T11:21:11.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.306+0000 7f9a12a00640 1 -- 172.21.15.47:0/2518814734 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9a040041d0 con 0x7f9a1c104ad0 2024-03-21T11:21:11.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.308+0000 7f9a12a00640 1 --2- 172.21.15.47:0/2518814734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9a000a2fa0 0x7f9a000a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:11.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.308+0000 7f9a12a00640 1 -- 172.21.15.47:0/2518814734 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9a040cef60 con 0x7f9a1c104ad0 2024-03-21T11:21:11.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.308+0000 7f9a19e00640 1 --2- 172.21.15.47:0/2518814734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9a000a2fa0 0x7f9a000a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:11.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.309+0000 7f9a19e00640 1 --2- 172.21.15.47:0/2518814734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9a000a2fa0 0x7f9a000a5460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c104930 tx=0x7f9a0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:11.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.312+0000 7f9a12a00640 1 -- 172.21.15.47:0/2518814734 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9a0409baf0 con 0x7f9a1c104ad0 2024-03-21T11:21:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.515+0000 7f9a20a00640 1 -- 172.21.15.47:0/2518814734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9a1c062340 con 0x7f9a1c104ad0 2024-03-21T11:21:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.516+0000 7f9a12a00640 1 -- 172.21.15.47:0/2518814734 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9a0409b3f0 con 0x7f9a1c104ad0 2024-03-21T11:21:11.515 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:21:11.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.518+0000 7f9a20a00640 1 -- 172.21.15.47:0/2518814734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9a000a2fa0 msgr2=0x7f9a000a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:11.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.518+0000 7f9a20a00640 1 --2- 172.21.15.47:0/2518814734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9a000a2fa0 0x7f9a000a5460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c104930 tx=0x7f9a0c005f90 comp rx=0 tx=0).stop 2024-03-21T11:21:11.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.518+0000 7f9a20a00640 1 -- 172.21.15.47:0/2518814734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a1c104ad0 msgr2=0x7f9a1c19a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:11.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.518+0000 7f9a20a00640 1 --2- 172.21.15.47:0/2518814734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a1c104ad0 0x7f9a1c19a090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400d950 tx=0x7f9a0400de10 comp rx=0 tx=0).stop 2024-03-21T11:21:11.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.519+0000 7f9a20a00640 1 -- 172.21.15.47:0/2518814734 shutdown_connections 2024-03-21T11:21:11.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.519+0000 7f9a20a00640 1 --2- 172.21.15.47:0/2518814734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9a000a2fa0 0x7f9a000a5460 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.519+0000 7f9a20a00640 1 --2- 172.21.15.47:0/2518814734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a1c105490 0x7f9a1c1a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.519+0000 7f9a20a00640 1 --2- 172.21.15.47:0/2518814734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a1c104ad0 0x7f9a1c19a090 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.519+0000 7f9a20a00640 1 --2- 172.21.15.47:0/2518814734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a1c1038e0 0x7f9a1c199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.519+0000 7f9a20a00640 1 -- 172.21.15.47:0/2518814734 >> 172.21.15.47:0/2518814734 conn(0x7f9a1c0fee50 msgr2=0x7f9a1c107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:11.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.519+0000 7f9a20a00640 1 -- 172.21.15.47:0/2518814734 shutdown_connections 2024-03-21T11:21:11.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.519+0000 7f9a20a00640 1 -- 172.21.15.47:0/2518814734 wait complete. 2024-03-21T11:21:11.538 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T11:21:11.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.765+0000 7f6973200640 1 -- 172.21.15.47:0/3196722822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6974104930 msgr2=0x7f6974106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:11.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.765+0000 7f6973200640 1 --2- 172.21.15.47:0/3196722822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6974104930 0x7f6974106d20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f695c009900 tx=0x7f695c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:21:11.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.765+0000 7f6973200640 1 -- 172.21.15.47:0/3196722822 shutdown_connections 2024-03-21T11:21:11.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.765+0000 7f6973200640 1 --2- 172.21.15.47:0/3196722822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6974107260 0x7f69741096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.765+0000 7f6973200640 1 --2- 172.21.15.47:0/3196722822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6974104930 0x7f6974106d20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.765+0000 7f6973200640 1 --2- 172.21.15.47:0/3196722822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6974102000 0x7f69741043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.765+0000 7f6973200640 1 -- 172.21.15.47:0/3196722822 >> 172.21.15.47:0/3196722822 conn(0x7f69740fba10 msgr2=0x7f69740fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:11.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.765+0000 7f6973200640 1 -- 172.21.15.47:0/3196722822 shutdown_connections 2024-03-21T11:21:11.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.765+0000 7f6973200640 1 -- 172.21.15.47:0/3196722822 wait complete. 2024-03-21T11:21:11.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.767+0000 7f6973200640 1 Processor -- start 2024-03-21T11:21:11.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.767+0000 7f6973200640 1 -- start start 2024-03-21T11:21:11.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.767+0000 7f6973200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6974102000 0x7f6974101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:11.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.767+0000 7f6973200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6974104930 0x7f69740fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:11.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.767+0000 7f6973200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6974107260 0x7f69741004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:11.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.767+0000 7f6973200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697410d260 con 0x7f6974107260 2024-03-21T11:21:11.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.767+0000 7f6973200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697410d0e0 con 0x7f6974104930 2024-03-21T11:21:11.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.767+0000 7f6973200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697410d3e0 con 0x7f6974102000 2024-03-21T11:21:11.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.767+0000 7f6971400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6974104930 0x7f69740fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:11.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.768+0000 7f6971400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6974104930 0x7f69740fffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47966/0 (socket says 172.21.15.47:47966) 2024-03-21T11:21:11.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.768+0000 7f6971400640 1 -- 172.21.15.47:0/2951319445 learned_addr learned my addr 172.21.15.47:0/2951319445 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:11.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.768+0000 7f6971e00640 1 --2- 172.21.15.47:0/2951319445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6974102000 0x7f6974101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:11.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.768+0000 7f6972800640 1 --2- 172.21.15.47:0/2951319445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6974107260 0x7f69741004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:11.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.768+0000 7f6971400640 1 -- 172.21.15.47:0/2951319445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6974102000 msgr2=0x7f6974101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:11.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.768+0000 7f6971400640 1 --2- 172.21.15.47:0/2951319445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6974102000 0x7f6974101970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.768+0000 7f6971400640 1 -- 172.21.15.47:0/2951319445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6974107260 msgr2=0x7f69741004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:11.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.768+0000 7f6971400640 1 --2- 172.21.15.47:0/2951319445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6974107260 0x7f69741004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.768+0000 7f6971400640 1 -- 172.21.15.47:0/2951319445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6974100d80 con 0x7f6974104930 2024-03-21T11:21:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.768+0000 7f6971400640 1 --2- 172.21.15.47:0/2951319445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6974104930 0x7f69740fffb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f695c000c00 tx=0x7f695c02c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.769+0000 7f696aa00640 1 -- 172.21.15.47:0/2951319445 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c043020 con 0x7f6974104930 2024-03-21T11:21:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.769+0000 7f696aa00640 1 -- 172.21.15.47:0/2951319445 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f695c0054e0 con 0x7f6974104930 2024-03-21T11:21:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.769+0000 7f696aa00640 1 -- 172.21.15.47:0/2951319445 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c003ce0 con 0x7f6974104930 2024-03-21T11:21:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.769+0000 7f6973200640 1 -- 172.21.15.47:0/2951319445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69741a19e0 con 0x7f6974104930 2024-03-21T11:21:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.769+0000 7f6973200640 1 -- 172.21.15.47:0/2951319445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69741a1eb0 con 0x7f6974104930 2024-03-21T11:21:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.770+0000 7f6973200640 1 -- 172.21.15.47:0/2951319445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69740fd500 con 0x7f6974104930 2024-03-21T11:21:11.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.772+0000 7f696aa00640 1 -- 172.21.15.47:0/2951319445 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f695c046020 con 0x7f6974104930 2024-03-21T11:21:11.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.773+0000 7f696aa00640 1 --2- 172.21.15.47:0/2951319445 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f69580a2fa0 0x7f69580a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:11.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.773+0000 7f696aa00640 1 -- 172.21.15.47:0/2951319445 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f695c0f1720 con 0x7f6974104930 2024-03-21T11:21:11.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.773+0000 7f6971e00640 1 --2- 172.21.15.47:0/2951319445 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f69580a2fa0 0x7f69580a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:11.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.774+0000 7f6971e00640 1 --2- 172.21.15.47:0/2951319445 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f69580a2fa0 0x7f69580a5460 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6964009bd0 tx=0x7f6964005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:11.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:11.777+0000 7f696aa00640 1 -- 172.21.15.47:0/2951319445 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f695c04d020 con 0x7f6974104930 2024-03-21T11:21:12.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.001+0000 7f6973200640 1 -- 172.21.15.47:0/2951319445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f697402d190 con 0x7f6974104930 2024-03-21T11:21:12.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.002+0000 7f696aa00640 1 -- 172.21.15.47:0/2951319445 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) v1 ==== 81+0+589 (secure 0 0 0) 0x7f695c0be2b0 con 0x7f6974104930 2024-03-21T11:21:12.002 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/4277663939 2024-03-22T11:12:48.837005+0000 2024-03-21T11:21:12.002 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/185270192 2024-03-22T11:12:48.837005+0000 2024-03-21T11:21:12.002 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/185270192 2024-03-22T11:12:48.837005+0000 2024-03-21T11:21:12.002 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1272635669 2024-03-22T11:12:48.837005+0000 2024-03-21T11:21:12.002 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2508263463 2024-03-22T11:12:14.713084+0000 2024-03-21T11:21:12.002 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/3773889790 2024-03-22T11:12:48.837005+0000 2024-03-21T11:21:12.002 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/1584030551 2024-03-22T11:12:14.713084+0000 2024-03-21T11:21:12.002 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/3130329909 2024-03-22T11:12:14.713084+0000 2024-03-21T11:21:12.003 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/1584030551 2024-03-22T11:12:14.713084+0000 2024-03-21T11:21:12.003 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/335539055 2024-03-22T11:12:14.713084+0000 2024-03-21T11:21:12.003 INFO:teuthology.orchestra.run.smithi047.stderr:listed 10 entries 2024-03-21T11:21:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.005+0000 7f6973200640 1 -- 172.21.15.47:0/2951319445 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f69580a2fa0 msgr2=0x7f69580a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.005+0000 7f6973200640 1 --2- 172.21.15.47:0/2951319445 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f69580a2fa0 0x7f69580a5460 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6964009bd0 tx=0x7f6964005f90 comp rx=0 tx=0).stop 2024-03-21T11:21:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.005+0000 7f6973200640 1 -- 172.21.15.47:0/2951319445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6974104930 msgr2=0x7f69740fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.005+0000 7f6973200640 1 --2- 172.21.15.47:0/2951319445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6974104930 0x7f69740fffb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f695c000c00 tx=0x7f695c02c6c0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.005+0000 7f6973200640 1 -- 172.21.15.47:0/2951319445 shutdown_connections 2024-03-21T11:21:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.005+0000 7f6973200640 1 --2- 172.21.15.47:0/2951319445 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f69580a2fa0 0x7f69580a5460 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.005+0000 7f6973200640 1 --2- 172.21.15.47:0/2951319445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6974107260 0x7f69741004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.005+0000 7f6973200640 1 --2- 172.21.15.47:0/2951319445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6974104930 0x7f69740fffb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.005+0000 7f6973200640 1 --2- 172.21.15.47:0/2951319445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6974102000 0x7f6974101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.005+0000 7f6973200640 1 -- 172.21.15.47:0/2951319445 >> 172.21.15.47:0/2951319445 conn(0x7f69740fba10 msgr2=0x7f69741028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.005+0000 7f6973200640 1 -- 172.21.15.47:0/2951319445 shutdown_connections 2024-03-21T11:21:12.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.005+0000 7f6973200640 1 -- 172.21.15.47:0/2951319445 wait complete. 2024-03-21T11:21:12.024 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-21T11:21:12.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:12.025 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-21T11:21:12.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:11 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1180895981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:21:12.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:11 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2518814734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:21:12.123 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:ceph-fuse[72690]: fuse finished with error 0 and tester_r 0 2024-03-21T11:21:12.128 INFO:teuthology.orchestra.run:waiting for 300 2024-03-21T11:21:12.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:11 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1180895981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:21:12.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:11 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2518814734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:21:12.195 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T11:21:12.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:21:12.196 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-21T11:21:12.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:11 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1180895981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:21:12.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:11 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2518814734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:21:12.256 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:21:12.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.488+0000 7f472f200640 1 -- 172.21.15.47:0/4189220642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47301048e0 msgr2=0x7f4730104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:12.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.488+0000 7f472f200640 1 --2- 172.21.15.47:0/4189220642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47301048e0 0x7f4730104d60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f471c009900 tx=0x7f471c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:21:12.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.488+0000 7f472f200640 1 -- 172.21.15.47:0/4189220642 shutdown_connections 2024-03-21T11:21:12.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.488+0000 7f472f200640 1 --2- 172.21.15.47:0/4189220642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47301052a0 0x7f473010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.488+0000 7f472f200640 1 --2- 172.21.15.47:0/4189220642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47301048e0 0x7f4730104d60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.488+0000 7f472f200640 1 --2- 172.21.15.47:0/4189220642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47301036f0 0x7f4730103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.488+0000 7f472f200640 1 -- 172.21.15.47:0/4189220642 >> 172.21.15.47:0/4189220642 conn(0x7f47300fec80 msgr2=0x7f47301010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:12.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.488+0000 7f472f200640 1 -- 172.21.15.47:0/4189220642 shutdown_connections 2024-03-21T11:21:12.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.488+0000 7f472f200640 1 -- 172.21.15.47:0/4189220642 wait complete. 2024-03-21T11:21:12.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.489+0000 7f472f200640 1 Processor -- start 2024-03-21T11:21:12.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.489+0000 7f472f200640 1 -- start start 2024-03-21T11:21:12.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.489+0000 7f472f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47301036f0 0x7f47301999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:12.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47301048e0 0x7f4730199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:12.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47301052a0 0x7f47301a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:12.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473010f3c0 con 0x7f47301048e0 2024-03-21T11:21:12.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473010f240 con 0x7f47301036f0 2024-03-21T11:21:12.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473010f540 con 0x7f47301052a0 2024-03-21T11:21:12.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47301036f0 0x7f47301999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:12.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47301036f0 0x7f47301999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47984/0 (socket says 172.21.15.47:47984) 2024-03-21T11:21:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472de00640 1 -- 172.21.15.47:0/4127619605 learned_addr learned my addr 172.21.15.47:0/4127619605 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472d400640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47301048e0 0x7f4730199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472e800640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47301052a0 0x7f47301a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472de00640 1 -- 172.21.15.47:0/4127619605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47301052a0 msgr2=0x7f47301a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472de00640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47301052a0 0x7f47301a0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472de00640 1 -- 172.21.15.47:0/4127619605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47301048e0 msgr2=0x7f4730199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472de00640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47301048e0 0x7f4730199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472de00640 1 -- 172.21.15.47:0/4127619605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47301a15c0 con 0x7f47301036f0 2024-03-21T11:21:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.490+0000 7f472d400640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47301048e0 0x7f4730199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:21:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.491+0000 7f472de00640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47301036f0 0x7f47301999a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f472000d770 tx=0x7f472000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:12.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.491+0000 7f4726a00640 1 -- 172.21.15.47:0/4127619605 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720017070 con 0x7f47301036f0 2024-03-21T11:21:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.491+0000 7f4726a00640 1 -- 172.21.15.47:0/4127619605 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4720009c30 con 0x7f47301036f0 2024-03-21T11:21:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.491+0000 7f472f200640 1 -- 172.21.15.47:0/4127619605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47301a18b0 con 0x7f47301036f0 2024-03-21T11:21:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.491+0000 7f4726a00640 1 -- 172.21.15.47:0/4127619605 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720003dd0 con 0x7f47301036f0 2024-03-21T11:21:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.491+0000 7f472f200640 1 -- 172.21.15.47:0/4127619605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47301a1e20 con 0x7f47301036f0 2024-03-21T11:21:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.492+0000 7f472f200640 1 -- 172.21.15.47:0/4127619605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473010a9b0 con 0x7f47301036f0 2024-03-21T11:21:12.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.494+0000 7f4726a00640 1 -- 172.21.15.47:0/4127619605 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4720003fb0 con 0x7f47301036f0 2024-03-21T11:21:12.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.495+0000 7f4726a00640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47140a2fa0 0x7f47140a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:12.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.496+0000 7f4726a00640 1 -- 172.21.15.47:0/4127619605 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f47200ceff0 con 0x7f47301036f0 2024-03-21T11:21:12.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.496+0000 7f472d400640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47140a2fa0 0x7f47140a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:12.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.496+0000 7f472d400640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47140a2fa0 0x7f47140a5460 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f471c02c250 tx=0x7f471c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:12.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.499+0000 7f4726a00640 1 -- 172.21.15.47:0/4127619605 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4720009de0 con 0x7f47301036f0 2024-03-21T11:21:12.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.711+0000 7f472f200640 1 -- 172.21.15.47:0/4127619605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4730062340 con 0x7f47301036f0 2024-03-21T11:21:12.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.713+0000 7f4726a00640 1 -- 172.21.15.47:0/4127619605 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24127 (secure 0 0 0) 0x7f472009bbe0 con 0x7f47301036f0 2024-03-21T11:21:12.713 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:21:12.713 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:20:54.191030+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:21:12.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.716+0000 7f472f200640 1 -- 172.21.15.47:0/4127619605 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47140a2fa0 msgr2=0x7f47140a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.716+0000 7f472f200640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47140a2fa0 0x7f47140a5460 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f471c02c250 tx=0x7f471c005fd0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.716+0000 7f472f200640 1 -- 172.21.15.47:0/4127619605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47301036f0 msgr2=0x7f47301999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.716+0000 7f472f200640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47301036f0 0x7f47301999a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f472000d770 tx=0x7f472000dc30 comp rx=0 tx=0).stop 2024-03-21T11:21:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.716+0000 7f472f200640 1 -- 172.21.15.47:0/4127619605 shutdown_connections 2024-03-21T11:21:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.716+0000 7f472f200640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47140a2fa0 0x7f47140a5460 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.716+0000 7f472f200640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47301052a0 0x7f47301a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.716+0000 7f472f200640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47301048e0 0x7f4730199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.716+0000 7f472f200640 1 --2- 172.21.15.47:0/4127619605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47301036f0 0x7f47301999a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.716+0000 7f472f200640 1 -- 172.21.15.47:0/4127619605 >> 172.21.15.47:0/4127619605 conn(0x7f47300fec80 msgr2=0x7f4730107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.716+0000 7f472f200640 1 -- 172.21.15.47:0/4127619605 shutdown_connections 2024-03-21T11:21:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.716+0000 7f472f200640 1 -- 172.21.15.47:0/4127619605 wait complete. 2024-03-21T11:21:12.734 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:21:12.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.962+0000 7fa2e2000640 1 -- 172.21.15.47:0/2931835642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2dc108cf0 msgr2=0x7fa2dc109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:12.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.962+0000 7fa2e2000640 1 --2- 172.21.15.47:0/2931835642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2dc108cf0 0x7fa2dc109170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009900 tx=0x7fa2c802bc40 comp rx=0 tx=0).stop 2024-03-21T11:21:12.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.962+0000 7fa2e2000640 1 -- 172.21.15.47:0/2931835642 shutdown_connections 2024-03-21T11:21:12.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.962+0000 7fa2e2000640 1 --2- 172.21.15.47:0/2931835642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2dc1096b0 0x7fa2dc10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.962+0000 7fa2e2000640 1 --2- 172.21.15.47:0/2931835642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2dc108cf0 0x7fa2dc109170 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.962+0000 7fa2e2000640 1 --2- 172.21.15.47:0/2931835642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2dc107b00 0x7fa2dc107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.962+0000 7fa2e2000640 1 -- 172.21.15.47:0/2931835642 >> 172.21.15.47:0/2931835642 conn(0x7fa2dc076ab0 msgr2=0x7fa2dc078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:12.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.962+0000 7fa2e2000640 1 -- 172.21.15.47:0/2931835642 shutdown_connections 2024-03-21T11:21:12.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.962+0000 7fa2e2000640 1 -- 172.21.15.47:0/2931835642 wait complete. 2024-03-21T11:21:12.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.963+0000 7fa2e2000640 1 Processor -- start 2024-03-21T11:21:12.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.963+0000 7fa2e2000640 1 -- start start 2024-03-21T11:21:12.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.964+0000 7fa2e2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2dc107b00 0x7fa2dc19ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:12.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.964+0000 7fa2e2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2dc108cf0 0x7fa2dc19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:12.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.964+0000 7fa2e2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2dc1096b0 0x7fa2dc1a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:12.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.964+0000 7fa2e2000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc1137b0 con 0x7fa2dc108cf0 2024-03-21T11:21:12.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.964+0000 7fa2e2000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc113630 con 0x7fa2dc107b00 2024-03-21T11:21:12.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.964+0000 7fa2e2000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc113930 con 0x7fa2dc1096b0 2024-03-21T11:21:12.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.964+0000 7fa2e0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2dc107b00 0x7fa2dc19ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:12.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.964+0000 7fa2e0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2dc107b00 0x7fa2dc19ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47998/0 (socket says 172.21.15.47:47998) 2024-03-21T11:21:12.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.964+0000 7fa2e0c00640 1 -- 172.21.15.47:0/778806763 learned_addr learned my addr 172.21.15.47:0/778806763 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:12.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.964+0000 7fa2dbe00640 1 --2- 172.21.15.47:0/778806763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2dc108cf0 0x7fa2dc19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:12.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.964+0000 7fa2e1600640 1 --2- 172.21.15.47:0/778806763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2dc1096b0 0x7fa2dc1a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:12.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.965+0000 7fa2e0c00640 1 -- 172.21.15.47:0/778806763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2dc1096b0 msgr2=0x7fa2dc1a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:12.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.965+0000 7fa2e0c00640 1 --2- 172.21.15.47:0/778806763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2dc1096b0 0x7fa2dc1a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.965+0000 7fa2e0c00640 1 -- 172.21.15.47:0/778806763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2dc108cf0 msgr2=0x7fa2dc19e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:12.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.965+0000 7fa2e0c00640 1 --2- 172.21.15.47:0/778806763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2dc108cf0 0x7fa2dc19e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:12.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.965+0000 7fa2e0c00640 1 -- 172.21.15.47:0/778806763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc1a59f0 con 0x7fa2dc107b00 2024-03-21T11:21:12.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.965+0000 7fa2e0c00640 1 --2- 172.21.15.47:0/778806763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2dc107b00 0x7fa2dc19ddd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000e9c0 tx=0x7fa2d000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:12.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.965+0000 7fa2d9600640 1 -- 172.21.15.47:0/778806763 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d000ccc0 con 0x7fa2dc107b00 2024-03-21T11:21:12.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.965+0000 7fa2d9600640 1 -- 172.21.15.47:0/778806763 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2d00035f0 con 0x7fa2dc107b00 2024-03-21T11:21:12.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.965+0000 7fa2d9600640 1 -- 172.21.15.47:0/778806763 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d000ce60 con 0x7fa2dc107b00 2024-03-21T11:21:12.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.965+0000 7fa2e2000640 1 -- 172.21.15.47:0/778806763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2dc1a5ce0 con 0x7fa2dc107b00 2024-03-21T11:21:12.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.966+0000 7fa2e2000640 1 -- 172.21.15.47:0/778806763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2dc1a61b0 con 0x7fa2dc107b00 2024-03-21T11:21:12.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.967+0000 7fa2e2000640 1 -- 172.21.15.47:0/778806763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2dc10ee20 con 0x7fa2dc107b00 2024-03-21T11:21:12.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.969+0000 7fa2d9600640 1 -- 172.21.15.47:0/778806763 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa2d00037a0 con 0x7fa2dc107b00 2024-03-21T11:21:12.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.970+0000 7fa2d9600640 1 --2- 172.21.15.47:0/778806763 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2c40a2f20 0x7fa2c40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:12.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.970+0000 7fa2d9600640 1 -- 172.21.15.47:0/778806763 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa2d00cf9b0 con 0x7fa2dc107b00 2024-03-21T11:21:12.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.970+0000 7fa2dbe00640 1 --2- 172.21.15.47:0/778806763 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2c40a2f20 0x7fa2c40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:12.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.971+0000 7fa2dbe00640 1 --2- 172.21.15.47:0/778806763 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2c40a2f20 0x7fa2c40a53e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2c802c250 tx=0x7fa2c802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:12.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:12.974+0000 7fa2d9600640 1 -- 172.21.15.47:0/778806763 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa2d009c540 con 0x7fa2dc107b00 2024-03-21T11:21:13.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:12 smithi047 ceph-mon[31351]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T11:21:13.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:12 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2951319445' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T11:21:13.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:12 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/4127619605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:12 smithi100 ceph-mon[36245]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T11:21:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:12 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2951319445' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T11:21:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:12 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/4127619605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:13.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.184+0000 7fa2e2000640 1 -- 172.21.15.47:0/778806763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2dc0622a0 con 0x7fa2dc107b00 2024-03-21T11:21:13.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.186+0000 7fa2d9600640 1 -- 172.21.15.47:0/778806763 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24127 (secure 0 0 0) 0x7fa2d009be40 con 0x7fa2dc107b00 2024-03-21T11:21:13.187 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:21:13.187 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:20:54.191030+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:21:13.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.189+0000 7fa2e2000640 1 -- 172.21.15.47:0/778806763 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2c40a2f20 msgr2=0x7fa2c40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.189+0000 7fa2e2000640 1 --2- 172.21.15.47:0/778806763 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2c40a2f20 0x7fa2c40a53e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2c802c250 tx=0x7fa2c802cf90 comp rx=0 tx=0).stop 2024-03-21T11:21:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.189+0000 7fa2e2000640 1 -- 172.21.15.47:0/778806763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2dc107b00 msgr2=0x7fa2dc19ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.189+0000 7fa2e2000640 1 --2- 172.21.15.47:0/778806763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2dc107b00 0x7fa2dc19ddd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000e9c0 tx=0x7fa2d000edc0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.189+0000 7fa2e2000640 1 -- 172.21.15.47:0/778806763 shutdown_connections 2024-03-21T11:21:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.189+0000 7fa2e2000640 1 --2- 172.21.15.47:0/778806763 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2c40a2f20 0x7fa2c40a53e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.189+0000 7fa2e2000640 1 --2- 172.21.15.47:0/778806763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2dc1096b0 0x7fa2dc1a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.189+0000 7fa2e2000640 1 --2- 172.21.15.47:0/778806763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2dc108cf0 0x7fa2dc19e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.189+0000 7fa2e2000640 1 --2- 172.21.15.47:0/778806763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2dc107b00 0x7fa2dc19ddd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.189+0000 7fa2e2000640 1 -- 172.21.15.47:0/778806763 >> 172.21.15.47:0/778806763 conn(0x7fa2dc076ab0 msgr2=0x7fa2dc10bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.189+0000 7fa2e2000640 1 -- 172.21.15.47:0/778806763 shutdown_connections 2024-03-21T11:21:13.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.189+0000 7fa2e2000640 1 -- 172.21.15.47:0/778806763 wait complete. 2024-03-21T11:21:13.210 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-21T11:21:13.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:12 smithi102 ceph-mon[36748]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T11:21:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:12 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2951319445' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T11:21:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:12 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/4127619605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.441+0000 7f86f3200640 1 -- 172.21.15.47:0/570022587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f4108db0 msgr2=0x7f86f4109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.441+0000 7f86f3200640 1 --2- 172.21.15.47:0/570022587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f4108db0 0x7f86f4109230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009900 tx=0x7f86dc02bc50 comp rx=0 tx=0).stop 2024-03-21T11:21:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.441+0000 7f86f3200640 1 -- 172.21.15.47:0/570022587 shutdown_connections 2024-03-21T11:21:13.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.441+0000 7f86f3200640 1 --2- 172.21.15.47:0/570022587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86f4109770 0x7f86f4110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.441+0000 7f86f3200640 1 --2- 172.21.15.47:0/570022587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f4108db0 0x7f86f4109230 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.441+0000 7f86f3200640 1 --2- 172.21.15.47:0/570022587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86f4107bc0 0x7f86f4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.441+0000 7f86f3200640 1 -- 172.21.15.47:0/570022587 >> 172.21.15.47:0/570022587 conn(0x7f86f4076b70 msgr2=0x7f86f4078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:13.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.442+0000 7f86f3200640 1 -- 172.21.15.47:0/570022587 shutdown_connections 2024-03-21T11:21:13.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.442+0000 7f86f3200640 1 -- 172.21.15.47:0/570022587 wait complete. 2024-03-21T11:21:13.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.443+0000 7f86f3200640 1 Processor -- start 2024-03-21T11:21:13.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.443+0000 7f86f3200640 1 -- start start 2024-03-21T11:21:13.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.443+0000 7f86f3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f4107bc0 0x7f86f419de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:13.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.443+0000 7f86f3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86f4108db0 0x7f86f419e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.443+0000 7f86f3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86f4109770 0x7f86f41a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.443+0000 7f86f3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f4113870 con 0x7f86f4109770 2024-03-21T11:21:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.443+0000 7f86f3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f41136f0 con 0x7f86f4107bc0 2024-03-21T11:21:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.443+0000 7f86f3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f41139f0 con 0x7f86f4108db0 2024-03-21T11:21:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.444+0000 7f86f1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f4107bc0 0x7f86f419de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.444+0000 7f86f1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f4107bc0 0x7f86f419de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48012/0 (socket says 172.21.15.47:48012) 2024-03-21T11:21:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.444+0000 7f86f1e00640 1 -- 172.21.15.47:0/2093687959 learned_addr learned my addr 172.21.15.47:0/2093687959 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.444+0000 7f86ebe00640 1 --2- 172.21.15.47:0/2093687959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86f4108db0 0x7f86f419e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.444+0000 7f86f2800640 1 --2- 172.21.15.47:0/2093687959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86f4109770 0x7f86f41a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.444+0000 7f86f1e00640 1 -- 172.21.15.47:0/2093687959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86f4108db0 msgr2=0x7f86f419e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.444+0000 7f86f1e00640 1 --2- 172.21.15.47:0/2093687959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86f4108db0 0x7f86f419e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.444+0000 7f86f1e00640 1 -- 172.21.15.47:0/2093687959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86f4109770 msgr2=0x7f86f41a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.444+0000 7f86f1e00640 1 --2- 172.21.15.47:0/2093687959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86f4109770 0x7f86f41a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.444+0000 7f86f1e00640 1 -- 172.21.15.47:0/2093687959 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f41a5a70 con 0x7f86f4107bc0 2024-03-21T11:21:13.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.444+0000 7f86f1e00640 1 --2- 172.21.15.47:0/2093687959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f4107bc0 0x7f86f419de50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f86e400e9c0 tx=0x7f86e400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:13.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.445+0000 7f86eaa00640 1 -- 172.21.15.47:0/2093687959 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e400cd80 con 0x7f86f4107bc0 2024-03-21T11:21:13.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.445+0000 7f86eaa00640 1 -- 172.21.15.47:0/2093687959 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86e40035f0 con 0x7f86f4107bc0 2024-03-21T11:21:13.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.445+0000 7f86f3200640 1 -- 172.21.15.47:0/2093687959 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f41a5d60 con 0x7f86f4107bc0 2024-03-21T11:21:13.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.445+0000 7f86eaa00640 1 -- 172.21.15.47:0/2093687959 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4010690 con 0x7f86f4107bc0 2024-03-21T11:21:13.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.445+0000 7f86f3200640 1 -- 172.21.15.47:0/2093687959 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f41a6210 con 0x7f86f4107bc0 2024-03-21T11:21:13.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.446+0000 7f86f3200640 1 -- 172.21.15.47:0/2093687959 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f410eee0 con 0x7f86f4107bc0 2024-03-21T11:21:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.448+0000 7f86eaa00640 1 -- 172.21.15.47:0/2093687959 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f86e40037a0 con 0x7f86f4107bc0 2024-03-21T11:21:13.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.449+0000 7f86eaa00640 1 --2- 172.21.15.47:0/2093687959 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86d80a2fa0 0x7f86d80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:13.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.449+0000 7f86eaa00640 1 -- 172.21.15.47:0/2093687959 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f86e40cfaa0 con 0x7f86f4107bc0 2024-03-21T11:21:13.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.449+0000 7f86ebe00640 1 --2- 172.21.15.47:0/2093687959 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86d80a2fa0 0x7f86d80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:13.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.450+0000 7f86ebe00640 1 --2- 172.21.15.47:0/2093687959 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86d80a2fa0 0x7f86d80a5460 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f86dc02c260 tx=0x7f86dc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:13.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.453+0000 7f86eaa00640 1 -- 172.21.15.47:0/2093687959 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f86e409c630 con 0x7f86f4107bc0 2024-03-21T11:21:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.697+0000 7f86f3200640 1 -- 172.21.15.47:0/2093687959 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f86f402d070 con 0x7f86f4107bc0 2024-03-21T11:21:13.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.816+0000 7f86eaa00640 1 -- 172.21.15.47:0/2093687959 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f86e409bf30 con 0x7f86f4107bc0 2024-03-21T11:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.819+0000 7f86f3200640 1 -- 172.21.15.47:0/2093687959 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86d80a2fa0 msgr2=0x7f86d80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.819+0000 7f86f3200640 1 --2- 172.21.15.47:0/2093687959 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86d80a2fa0 0x7f86d80a5460 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f86dc02c260 tx=0x7f86dc000bc0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.819+0000 7f86f3200640 1 -- 172.21.15.47:0/2093687959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f4107bc0 msgr2=0x7f86f419de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.819+0000 7f86f3200640 1 --2- 172.21.15.47:0/2093687959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f4107bc0 0x7f86f419de50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f86e400e9c0 tx=0x7f86e400ee80 comp rx=0 tx=0).stop 2024-03-21T11:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.819+0000 7f86f3200640 1 -- 172.21.15.47:0/2093687959 shutdown_connections 2024-03-21T11:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.819+0000 7f86f3200640 1 --2- 172.21.15.47:0/2093687959 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86d80a2fa0 0x7f86d80a5460 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.819+0000 7f86f3200640 1 --2- 172.21.15.47:0/2093687959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86f4109770 0x7f86f41a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.819+0000 7f86f3200640 1 --2- 172.21.15.47:0/2093687959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86f4108db0 0x7f86f419e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.819+0000 7f86f3200640 1 --2- 172.21.15.47:0/2093687959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86f4107bc0 0x7f86f419de50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.819+0000 7f86f3200640 1 -- 172.21.15.47:0/2093687959 >> 172.21.15.47:0/2093687959 conn(0x7f86f4076b70 msgr2=0x7f86f410c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:13.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.819+0000 7f86f3200640 1 -- 172.21.15.47:0/2093687959 shutdown_connections 2024-03-21T11:21:13.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:13.819+0000 7f86f3200640 1 -- 172.21.15.47:0/2093687959 wait complete. 2024-03-21T11:21:13.841 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-03-21T11:21:13.841 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-21T11:21:14.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.061+0000 7fb335400640 1 -- 172.21.15.47:0/150592310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3300772a0 msgr2=0x7fb330077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:14.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.061+0000 7fb335400640 1 --2- 172.21.15.47:0/150592310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3300772a0 0x7fb330077700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb318009900 tx=0x7fb31802bc40 comp rx=0 tx=0).stop 2024-03-21T11:21:14.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.061+0000 7fb335400640 1 -- 172.21.15.47:0/150592310 shutdown_connections 2024-03-21T11:21:14.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.061+0000 7fb335400640 1 --2- 172.21.15.47:0/150592310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb330077c40 0x7fb3300784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:14.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.061+0000 7fb335400640 1 --2- 172.21.15.47:0/150592310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3300772a0 0x7fb330077700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:14.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.061+0000 7fb335400640 1 --2- 172.21.15.47:0/150592310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb330076060 0x7fb330076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:14.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.061+0000 7fb335400640 1 -- 172.21.15.47:0/150592310 >> 172.21.15.47:0/150592310 conn(0x7fb3300fed10 msgr2=0x7fb330101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:14.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.061+0000 7fb335400640 1 -- 172.21.15.47:0/150592310 shutdown_connections 2024-03-21T11:21:14.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.061+0000 7fb335400640 1 -- 172.21.15.47:0/150592310 wait complete. 2024-03-21T11:21:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.062+0000 7fb335400640 1 Processor -- start 2024-03-21T11:21:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.062+0000 7fb335400640 1 -- start start 2024-03-21T11:21:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb335400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb330076060 0x7fb33019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb335400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3300772a0 0x7fb33019e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb335400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb330077c40 0x7fb3301a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb335400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb330113830 con 0x7fb330076060 2024-03-21T11:21:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb335400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3301136b0 con 0x7fb330077c40 2024-03-21T11:21:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb335400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3301139b0 con 0x7fb3300772a0 2024-03-21T11:21:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb32fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb330076060 0x7fb33019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb32fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb330076060 0x7fb33019de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37590/0 (socket says 172.21.15.47:37590) 2024-03-21T11:21:14.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb32fe00640 1 -- 172.21.15.47:0/1388443981 learned_addr learned my addr 172.21.15.47:0/1388443981 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:14.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb334a00640 1 --2- 172.21.15.47:0/1388443981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb330077c40 0x7fb3301a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:14.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb327400640 1 --2- 172.21.15.47:0/1388443981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3300772a0 0x7fb33019e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:14.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb32fe00640 1 -- 172.21.15.47:0/1388443981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3300772a0 msgr2=0x7fb33019e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:14.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb32fe00640 1 --2- 172.21.15.47:0/1388443981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3300772a0 0x7fb33019e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:14.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb32fe00640 1 -- 172.21.15.47:0/1388443981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb330077c40 msgr2=0x7fb3301a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:14.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb32fe00640 1 --2- 172.21.15.47:0/1388443981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb330077c40 0x7fb3301a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:14.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.063+0000 7fb32fe00640 1 -- 172.21.15.47:0/1388443981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3301a5ab0 con 0x7fb330076060 2024-03-21T11:21:14.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:13 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/778806763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:14.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:13 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2093687959' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T11:21:14.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:13 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T11:21:14.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.064+0000 7fb32fe00640 1 --2- 172.21.15.47:0/1388443981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb330076060 0x7fb33019de90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb32000d950 tx=0x7fb32000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:14.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.064+0000 7fb32d600640 1 -- 172.21.15.47:0/1388443981 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320017070 con 0x7fb330076060 2024-03-21T11:21:14.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.064+0000 7fb32d600640 1 -- 172.21.15.47:0/1388443981 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb32000de80 con 0x7fb330076060 2024-03-21T11:21:14.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.064+0000 7fb335400640 1 -- 172.21.15.47:0/1388443981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3301a5da0 con 0x7fb330076060 2024-03-21T11:21:14.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.064+0000 7fb32d600640 1 -- 172.21.15.47:0/1388443981 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32000f620 con 0x7fb330076060 2024-03-21T11:21:14.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.064+0000 7fb335400640 1 -- 172.21.15.47:0/1388443981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3301a6200 con 0x7fb330076060 2024-03-21T11:21:14.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.066+0000 7fb335400640 1 -- 172.21.15.47:0/1388443981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb330068800 con 0x7fb330076060 2024-03-21T11:21:14.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.068+0000 7fb32d600640 1 -- 172.21.15.47:0/1388443981 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb320009c30 con 0x7fb330076060 2024-03-21T11:21:14.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.070+0000 7fb32d600640 1 --2- 172.21.15.47:0/1388443981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb3140a2f20 0x7fb3140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:14.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.071+0000 7fb32d600640 1 -- 172.21.15.47:0/1388443981 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb3200cf1d0 con 0x7fb330076060 2024-03-21T11:21:14.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.071+0000 7fb327400640 1 --2- 172.21.15.47:0/1388443981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb3140a2f20 0x7fb3140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:14.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.071+0000 7fb327400640 1 --2- 172.21.15.47:0/1388443981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb3140a2f20 0x7fb3140a53e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb31802c7c0 tx=0x7fb318005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:14.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.074+0000 7fb32d600640 1 -- 172.21.15.47:0/1388443981 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb32009bdc0 con 0x7fb330076060 2024-03-21T11:21:14.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:13 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/778806763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:14.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:13 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2093687959' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T11:21:14.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:13 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T11:21:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:13 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/778806763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:13 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2093687959' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T11:21:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:13 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T11:21:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.322+0000 7fb335400640 1 -- 172.21.15.47:0/1388443981 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fb3300622a0 con 0x7fb330076060 2024-03-21T11:21:14.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.815+0000 7fb32d600640 1 -- 172.21.15.47:0/1388443981 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fb32009b6c0 con 0x7fb330076060 2024-03-21T11:21:14.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.819+0000 7fb335400640 1 -- 172.21.15.47:0/1388443981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb3140a2f20 msgr2=0x7fb3140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:14.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.819+0000 7fb335400640 1 --2- 172.21.15.47:0/1388443981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb3140a2f20 0x7fb3140a53e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb31802c7c0 tx=0x7fb318005fd0 comp rx=0 tx=0).stop 2024-03-21T11:21:14.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.819+0000 7fb335400640 1 -- 172.21.15.47:0/1388443981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb330076060 msgr2=0x7fb33019de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:14.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.819+0000 7fb335400640 1 --2- 172.21.15.47:0/1388443981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb330076060 0x7fb33019de90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb32000d950 tx=0x7fb32000dd50 comp rx=0 tx=0).stop 2024-03-21T11:21:14.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.819+0000 7fb335400640 1 -- 172.21.15.47:0/1388443981 shutdown_connections 2024-03-21T11:21:14.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.819+0000 7fb335400640 1 --2- 172.21.15.47:0/1388443981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb3140a2f20 0x7fb3140a53e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:14.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.819+0000 7fb335400640 1 --2- 172.21.15.47:0/1388443981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb330077c40 0x7fb3301a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:14.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.819+0000 7fb335400640 1 --2- 172.21.15.47:0/1388443981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3300772a0 0x7fb33019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:14.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.819+0000 7fb335400640 1 --2- 172.21.15.47:0/1388443981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb330076060 0x7fb33019de90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:14.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.819+0000 7fb335400640 1 -- 172.21.15.47:0/1388443981 >> 172.21.15.47:0/1388443981 conn(0x7fb3300fed10 msgr2=0x7fb330100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:14.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.819+0000 7fb335400640 1 -- 172.21.15.47:0/1388443981 shutdown_connections 2024-03-21T11:21:14.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:14.819+0000 7fb335400640 1 -- 172.21.15.47:0/1388443981 wait complete. 2024-03-21T11:21:14.847 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-21T11:21:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.059+0000 7f5650200640 1 -- 172.21.15.47:0/213206926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5648105490 msgr2=0x7f564810bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.059+0000 7f5650200640 1 --2- 172.21.15.47:0/213206926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5648105490 0x7f564810bd20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5640009ca0 tx=0x7f564002c100 comp rx=0 tx=0).stop 2024-03-21T11:21:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.059+0000 7f5650200640 1 -- 172.21.15.47:0/213206926 shutdown_connections 2024-03-21T11:21:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.059+0000 7f5650200640 1 --2- 172.21.15.47:0/213206926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5648105490 0x7f564810bd20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.059+0000 7f5650200640 1 --2- 172.21.15.47:0/213206926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5648104ad0 0x7f5648104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.059+0000 7f5650200640 1 --2- 172.21.15.47:0/213206926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56481038e0 0x7f5648103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:15.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.059+0000 7f5650200640 1 -- 172.21.15.47:0/213206926 >> 172.21.15.47:0/213206926 conn(0x7f56480fee50 msgr2=0x7f5648101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:15.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.059+0000 7f5650200640 1 -- 172.21.15.47:0/213206926 shutdown_connections 2024-03-21T11:21:15.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.059+0000 7f5650200640 1 -- 172.21.15.47:0/213206926 wait complete. 2024-03-21T11:21:15.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.060+0000 7f5650200640 1 Processor -- start 2024-03-21T11:21:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.060+0000 7f5650200640 1 -- start start 2024-03-21T11:21:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f5650200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56481038e0 0x7f5648199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f5650200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5648104ad0 0x7f564819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f5650200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5648105490 0x7f56481a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f5650200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564810f560 con 0x7f56481038e0 2024-03-21T11:21:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f5650200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564810f3e0 con 0x7f5648105490 2024-03-21T11:21:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f5650200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564810f6e0 con 0x7f5648104ad0 2024-03-21T11:21:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f564d800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56481038e0 0x7f5648199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f564e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5648105490 0x7f56481a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f564d800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56481038e0 0x7f5648199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37622/0 (socket says 172.21.15.47:37622) 2024-03-21T11:21:15.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f564e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5648105490 0x7f56481a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48028/0 (socket says 172.21.15.47:48028) 2024-03-21T11:21:15.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f564d800640 1 -- 172.21.15.47:0/3228166067 learned_addr learned my addr 172.21.15.47:0/3228166067 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:15.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f564ce00640 1 --2- 172.21.15.47:0/3228166067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5648104ad0 0x7f564819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:15.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f564e200640 1 -- 172.21.15.47:0/3228166067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5648104ad0 msgr2=0x7f564819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:15.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f564e200640 1 --2- 172.21.15.47:0/3228166067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5648104ad0 0x7f564819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:15.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f564e200640 1 -- 172.21.15.47:0/3228166067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56481038e0 msgr2=0x7f5648199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:15.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f564e200640 1 --2- 172.21.15.47:0/3228166067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56481038e0 0x7f5648199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:15.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f564e200640 1 -- 172.21.15.47:0/3228166067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56481a1790 con 0x7f5648105490 2024-03-21T11:21:15.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.061+0000 7f564d800640 1 --2- 172.21.15.47:0/3228166067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56481038e0 0x7f5648199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:21:15.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.062+0000 7f564e200640 1 --2- 172.21.15.47:0/3228166067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5648105490 0x7f56481a1120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5640037040 tx=0x7f5640035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:15.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.062+0000 7f5646000640 1 -- 172.21.15.47:0/3228166067 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640039040 con 0x7f5648105490 2024-03-21T11:21:15.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.062+0000 7f5646000640 1 -- 172.21.15.47:0/3228166067 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5640044070 con 0x7f5648105490 2024-03-21T11:21:15.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.062+0000 7f5646000640 1 -- 172.21.15.47:0/3228166067 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640004350 con 0x7f5648105490 2024-03-21T11:21:15.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.062+0000 7f5650200640 1 -- 172.21.15.47:0/3228166067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56481a1a20 con 0x7f5648105490 2024-03-21T11:21:15.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.062+0000 7f5650200640 1 -- 172.21.15.47:0/3228166067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56481a1e90 con 0x7f5648105490 2024-03-21T11:21:15.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.063+0000 7f5650200640 1 -- 172.21.15.47:0/3228166067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564810ac00 con 0x7f5648105490 2024-03-21T11:21:15.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.065+0000 7f5646000640 1 -- 172.21.15.47:0/3228166067 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f56400044f0 con 0x7f5648105490 2024-03-21T11:21:15.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.066+0000 7f5646000640 1 --2- 172.21.15.47:0/3228166067 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56300a2f20 0x7f56300a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:15.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.067+0000 7f5646000640 1 -- 172.21.15.47:0/3228166067 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f56400f1b50 con 0x7f5648105490 2024-03-21T11:21:15.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.067+0000 7f564d800640 1 --2- 172.21.15.47:0/3228166067 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56300a2f20 0x7f56300a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:15.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.068+0000 7f564d800640 1 --2- 172.21.15.47:0/3228166067 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56300a2f20 0x7f56300a53e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f563c0058c0 tx=0x7f563c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:15.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.070+0000 7f5646000640 1 -- 172.21.15.47:0/3228166067 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f56400be6e0 con 0x7f5648105490 2024-03-21T11:21:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:14 smithi100 ceph-mon[36245]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T11:21:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:14 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T11:21:15.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:14 smithi100 ceph-mon[36245]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:21:15.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:14 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1388443981' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-21T11:21:15.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:14 smithi102 ceph-mon[36748]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T11:21:15.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:14 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T11:21:15.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:14 smithi102 ceph-mon[36748]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:21:15.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:14 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1388443981' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-21T11:21:15.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:14 smithi047 ceph-mon[31351]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T11:21:15.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:14 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T11:21:15.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:14 smithi047 ceph-mon[31351]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:21:15.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:14 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1388443981' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-21T11:21:15.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.317+0000 7f5650200640 1 -- 172.21.15.47:0/3228166067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f56480622a0 con 0x7f5648105490 2024-03-21T11:21:15.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.829+0000 7f5646000640 1 -- 172.21.15.47:0/3228166067 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f56400bdfe0 con 0x7f5648105490 2024-03-21T11:21:15.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.833+0000 7f5650200640 1 -- 172.21.15.47:0/3228166067 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56300a2f20 msgr2=0x7f56300a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:15.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.833+0000 7f5650200640 1 --2- 172.21.15.47:0/3228166067 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56300a2f20 0x7f56300a53e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f563c0058c0 tx=0x7f563c005f90 comp rx=0 tx=0).stop 2024-03-21T11:21:15.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.833+0000 7f5650200640 1 -- 172.21.15.47:0/3228166067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5648105490 msgr2=0x7f56481a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:15.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.833+0000 7f5650200640 1 --2- 172.21.15.47:0/3228166067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5648105490 0x7f56481a1120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5640037040 tx=0x7f5640035d10 comp rx=0 tx=0).stop 2024-03-21T11:21:15.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.833+0000 7f5650200640 1 -- 172.21.15.47:0/3228166067 shutdown_connections 2024-03-21T11:21:15.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.833+0000 7f5650200640 1 --2- 172.21.15.47:0/3228166067 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56300a2f20 0x7f56300a53e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:15.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.833+0000 7f5650200640 1 --2- 172.21.15.47:0/3228166067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5648105490 0x7f56481a1120 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:15.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.833+0000 7f5650200640 1 --2- 172.21.15.47:0/3228166067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5648104ad0 0x7f564819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:15.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.833+0000 7f5650200640 1 --2- 172.21.15.47:0/3228166067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56481038e0 0x7f5648199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:15.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.833+0000 7f5650200640 1 -- 172.21.15.47:0/3228166067 >> 172.21.15.47:0/3228166067 conn(0x7f56480fee50 msgr2=0x7f5648107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:15.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.834+0000 7f5650200640 1 -- 172.21.15.47:0/3228166067 shutdown_connections 2024-03-21T11:21:15.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:15.834+0000 7f5650200640 1 -- 172.21.15.47:0/3228166067 wait complete. 2024-03-21T11:21:15.865 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-21T11:21:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.081+0000 7fd2aa000640 1 -- 172.21.15.47:0/797807494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2a4108db0 msgr2=0x7fd2a4109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:16.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.081+0000 7fd2aa000640 1 --2- 172.21.15.47:0/797807494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2a4108db0 0x7fd2a4109230 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009900 tx=0x7fd28c02bc50 comp rx=0 tx=0).stop 2024-03-21T11:21:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.081+0000 7fd2aa000640 1 -- 172.21.15.47:0/797807494 shutdown_connections 2024-03-21T11:21:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.081+0000 7fd2aa000640 1 --2- 172.21.15.47:0/797807494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a4109770 0x7fd2a4110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.081+0000 7fd2aa000640 1 --2- 172.21.15.47:0/797807494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2a4108db0 0x7fd2a4109230 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.081+0000 7fd2aa000640 1 --2- 172.21.15.47:0/797807494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2a4107bc0 0x7fd2a4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.081+0000 7fd2aa000640 1 -- 172.21.15.47:0/797807494 >> 172.21.15.47:0/797807494 conn(0x7fd2a4076b70 msgr2=0x7fd2a4078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.081+0000 7fd2aa000640 1 -- 172.21.15.47:0/797807494 shutdown_connections 2024-03-21T11:21:16.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.081+0000 7fd2aa000640 1 -- 172.21.15.47:0/797807494 wait complete. 2024-03-21T11:21:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.083+0000 7fd2aa000640 1 Processor -- start 2024-03-21T11:21:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.083+0000 7fd2aa000640 1 -- start start 2024-03-21T11:21:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.083+0000 7fd2aa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2a4107bc0 0x7fd2a419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.083+0000 7fd2aa000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a4108db0 0x7fd2a419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.083+0000 7fd2aa000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2a4109770 0x7fd2a41a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.083+0000 7fd2aa000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a4113820 con 0x7fd2a4108db0 2024-03-21T11:21:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.083+0000 7fd2aa000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a41136a0 con 0x7fd2a4109770 2024-03-21T11:21:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.083+0000 7fd2aa000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a41139a0 con 0x7fd2a4107bc0 2024-03-21T11:21:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.083+0000 7fd2a8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2a4107bc0 0x7fd2a419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.083+0000 7fd2a3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a4108db0 0x7fd2a419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.083+0000 7fd2a9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2a4109770 0x7fd2a41a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.083+0000 7fd2a9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2a4109770 0x7fd2a41a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48056/0 (socket says 172.21.15.47:48056) 2024-03-21T11:21:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.083+0000 7fd2a9600640 1 -- 172.21.15.47:0/2666795657 learned_addr learned my addr 172.21.15.47:0/2666795657 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.084+0000 7fd2a9600640 1 -- 172.21.15.47:0/2666795657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2a4107bc0 msgr2=0x7fd2a419df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.084+0000 7fd2a9600640 1 --2- 172.21.15.47:0/2666795657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2a4107bc0 0x7fd2a419df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.084+0000 7fd2a9600640 1 -- 172.21.15.47:0/2666795657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a4108db0 msgr2=0x7fd2a419e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:16.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.084+0000 7fd2a9600640 1 --2- 172.21.15.47:0/2666795657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a4108db0 0x7fd2a419e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.084+0000 7fd2a9600640 1 -- 172.21.15.47:0/2666795657 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a41a5b50 con 0x7fd2a4109770 2024-03-21T11:21:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.084+0000 7fd2a8c00640 1 --2- 172.21.15.47:0/2666795657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2a4107bc0 0x7fd2a419df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:21:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.084+0000 7fd2a9600640 1 --2- 172.21.15.47:0/2666795657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2a4109770 0x7fd2a41a54e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00eaa0 tx=0x7fd29c00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.084+0000 7fd2a1600640 1 -- 172.21.15.47:0/2666795657 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c00cdb0 con 0x7fd2a4109770 2024-03-21T11:21:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.085+0000 7fd2a1600640 1 -- 172.21.15.47:0/2666795657 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd29c005020 con 0x7fd2a4109770 2024-03-21T11:21:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.085+0000 7fd2aa000640 1 -- 172.21.15.47:0/2666795657 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a41a5e40 con 0x7fd2a4109770 2024-03-21T11:21:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.085+0000 7fd2a1600640 1 -- 172.21.15.47:0/2666795657 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c0116b0 con 0x7fd2a4109770 2024-03-21T11:21:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.085+0000 7fd2aa000640 1 -- 172.21.15.47:0/2666795657 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a41a6300 con 0x7fd2a4109770 2024-03-21T11:21:16.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.086+0000 7fd2aa000640 1 -- 172.21.15.47:0/2666795657 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a410ef60 con 0x7fd2a4109770 2024-03-21T11:21:16.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.088+0000 7fd2a1600640 1 -- 172.21.15.47:0/2666795657 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd29c005560 con 0x7fd2a4109770 2024-03-21T11:21:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.089+0000 7fd2a1600640 1 --2- 172.21.15.47:0/2666795657 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2880a2fa0 0x7fd2880a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.089+0000 7fd2a1600640 1 -- 172.21.15.47:0/2666795657 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd29c0cf7b0 con 0x7fd2a4109770 2024-03-21T11:21:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.089+0000 7fd2a8c00640 1 --2- 172.21.15.47:0/2666795657 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2880a2fa0 0x7fd2880a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.090+0000 7fd2a8c00640 1 --2- 172.21.15.47:0/2666795657 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2880a2fa0 0x7fd2880a5460 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4108c10 tx=0x7fd294005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.093+0000 7fd2a1600640 1 -- 172.21.15.47:0/2666795657 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd29c016030 con 0x7fd2a4109770 2024-03-21T11:21:16.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:15 smithi100 ceph-mon[36245]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T11:21:16.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:15 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1388443981' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-21T11:21:16.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:15 smithi100 ceph-mon[36245]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T11:21:16.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:15 smithi100 ceph-mon[36245]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:21:16.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:15 smithi100 ceph-mon[36245]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-21T11:21:16.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:15 smithi100 ceph-mon[36245]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-21T11:21:16.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:15 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3228166067' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T11:21:16.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:15 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T11:21:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:15 smithi102 ceph-mon[36748]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T11:21:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:15 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1388443981' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-21T11:21:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:15 smithi102 ceph-mon[36748]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T11:21:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:15 smithi102 ceph-mon[36748]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:21:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:15 smithi102 ceph-mon[36748]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-21T11:21:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:15 smithi102 ceph-mon[36748]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-21T11:21:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:15 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3228166067' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T11:21:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:15 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T11:21:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:15 smithi047 ceph-mon[31351]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T11:21:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:15 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1388443981' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-21T11:21:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:15 smithi047 ceph-mon[31351]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T11:21:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:15 smithi047 ceph-mon[31351]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T11:21:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:15 smithi047 ceph-mon[31351]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-21T11:21:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:15 smithi047 ceph-mon[31351]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-21T11:21:16.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:15 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3228166067' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T11:21:16.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:15 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T11:21:16.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.338+0000 7fd2aa000640 1 -- 172.21.15.47:0/2666795657 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fd2a40622a0 con 0x7fd2a4109770 2024-03-21T11:21:16.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.852+0000 7fd2a1600640 1 -- 172.21.15.47:0/2666795657 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7fd29c09bcf0 con 0x7fd2a4109770 2024-03-21T11:21:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.856+0000 7fd2aa000640 1 -- 172.21.15.47:0/2666795657 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2880a2fa0 msgr2=0x7fd2880a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.856+0000 7fd2aa000640 1 --2- 172.21.15.47:0/2666795657 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2880a2fa0 0x7fd2880a5460 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4108c10 tx=0x7fd294005f90 comp rx=0 tx=0).stop 2024-03-21T11:21:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.856+0000 7fd2aa000640 1 -- 172.21.15.47:0/2666795657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2a4109770 msgr2=0x7fd2a41a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.856+0000 7fd2aa000640 1 --2- 172.21.15.47:0/2666795657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2a4109770 0x7fd2a41a54e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00eaa0 tx=0x7fd29c00ef60 comp rx=0 tx=0).stop 2024-03-21T11:21:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.856+0000 7fd2aa000640 1 -- 172.21.15.47:0/2666795657 shutdown_connections 2024-03-21T11:21:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.856+0000 7fd2aa000640 1 --2- 172.21.15.47:0/2666795657 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2880a2fa0 0x7fd2880a5460 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.856+0000 7fd2aa000640 1 --2- 172.21.15.47:0/2666795657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2a4109770 0x7fd2a41a54e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.856+0000 7fd2aa000640 1 --2- 172.21.15.47:0/2666795657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a4108db0 0x7fd2a419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.856+0000 7fd2aa000640 1 --2- 172.21.15.47:0/2666795657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2a4107bc0 0x7fd2a419df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.856+0000 7fd2aa000640 1 -- 172.21.15.47:0/2666795657 >> 172.21.15.47:0/2666795657 conn(0x7fd2a4076b70 msgr2=0x7fd2a410c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.856+0000 7fd2aa000640 1 -- 172.21.15.47:0/2666795657 shutdown_connections 2024-03-21T11:21:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:16.856+0000 7fd2aa000640 1 -- 172.21.15.47:0/2666795657 wait complete. 2024-03-21T11:21:16.884 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-03-21T11:21:16.884 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-21T11:21:17.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.083+0000 7f949fe00640 1 -- 172.21.15.47:0/3448146701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94981038b0 msgr2=0x7f9498103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:17.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.083+0000 7f949fe00640 1 --2- 172.21.15.47:0/3448146701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94981038b0 0x7f9498103cd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9488009ca0 tx=0x7f948802c0f0 comp rx=0 tx=0).stop 2024-03-21T11:21:17.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.083+0000 7f949fe00640 1 -- 172.21.15.47:0/3448146701 shutdown_connections 2024-03-21T11:21:17.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.083+0000 7f949fe00640 1 --2- 172.21.15.47:0/3448146701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9498105460 0x7f949810bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:17.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.083+0000 7f949fe00640 1 --2- 172.21.15.47:0/3448146701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9498104aa0 0x7f9498104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:17.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.083+0000 7f949fe00640 1 --2- 172.21.15.47:0/3448146701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94981038b0 0x7f9498103cd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:17.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.083+0000 7f949fe00640 1 -- 172.21.15.47:0/3448146701 >> 172.21.15.47:0/3448146701 conn(0x7f94980fee60 msgr2=0x7f9498101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:17.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.084+0000 7f949fe00640 1 -- 172.21.15.47:0/3448146701 shutdown_connections 2024-03-21T11:21:17.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.084+0000 7f949fe00640 1 -- 172.21.15.47:0/3448146701 wait complete. 2024-03-21T11:21:17.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.085+0000 7f949fe00640 1 Processor -- start 2024-03-21T11:21:17.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.085+0000 7f949fe00640 1 -- start start 2024-03-21T11:21:17.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.085+0000 7f949fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94981038b0 0x7f9498079450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:17.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9498104aa0 0x7f9498079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:17.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9498105460 0x7f9498075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:17.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949810f500 con 0x7f9498105460 2024-03-21T11:21:17.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949810f380 con 0x7f9498104aa0 2024-03-21T11:21:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949810f680 con 0x7f94981038b0 2024-03-21T11:21:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94981038b0 0x7f9498079450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9498104aa0 0x7f9498079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94981038b0 0x7f9498079450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40586/0 (socket says 172.21.15.47:40586) 2024-03-21T11:21:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949d400640 1 -- 172.21.15.47:0/3020159578 learned_addr learned my addr 172.21.15.47:0/3020159578 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949de00640 1 --2- 172.21.15.47:0/3020159578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9498105460 0x7f9498075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949d400640 1 -- 172.21.15.47:0/3020159578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9498104aa0 msgr2=0x7f9498079990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949d400640 1 --2- 172.21.15.47:0/3020159578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9498104aa0 0x7f9498079990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949d400640 1 -- 172.21.15.47:0/3020159578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9498105460 msgr2=0x7f9498075fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949d400640 1 --2- 172.21.15.47:0/3020159578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9498105460 0x7f9498075fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949d400640 1 -- 172.21.15.47:0/3020159578 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9498076880 con 0x7f94981038b0 2024-03-21T11:21:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.086+0000 7f949ca00640 1 --2- 172.21.15.47:0/3020159578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9498104aa0 0x7f9498079990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:21:17.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.087+0000 7f949d400640 1 --2- 172.21.15.47:0/3020159578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94981038b0 0x7f9498079450 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f94880324c0 tx=0x7f9488032f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:17.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.087+0000 7f9496000640 1 -- 172.21.15.47:0/3020159578 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488041020 con 0x7f94981038b0 2024-03-21T11:21:17.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.087+0000 7f9496000640 1 -- 172.21.15.47:0/3020159578 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9488034cd0 con 0x7f94981038b0 2024-03-21T11:21:17.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.087+0000 7f9496000640 1 -- 172.21.15.47:0/3020159578 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488039040 con 0x7f94981038b0 2024-03-21T11:21:17.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.087+0000 7f949fe00640 1 -- 172.21.15.47:0/3020159578 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94981a6000 con 0x7f94981038b0 2024-03-21T11:21:17.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.087+0000 7f949fe00640 1 -- 172.21.15.47:0/3020159578 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94981a63b0 con 0x7f94981038b0 2024-03-21T11:21:17.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.088+0000 7f949fe00640 1 -- 172.21.15.47:0/3020159578 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949810ac50 con 0x7f94981038b0 2024-03-21T11:21:17.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.091+0000 7f9496000640 1 -- 172.21.15.47:0/3020159578 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9488044020 con 0x7f94981038b0 2024-03-21T11:21:17.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.092+0000 7f9496000640 1 --2- 172.21.15.47:0/3020159578 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f947c0a2f20 0x7f947c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:17.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.092+0000 7f9496000640 1 -- 172.21.15.47:0/3020159578 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f94880f15a0 con 0x7f94981038b0 2024-03-21T11:21:17.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.093+0000 7f949ca00640 1 --2- 172.21.15.47:0/3020159578 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f947c0a2f20 0x7f947c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:17.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.093+0000 7f949ca00640 1 --2- 172.21.15.47:0/3020159578 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f947c0a2f20 0x7f947c0a53e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f94980773d0 tx=0x7f9484005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:17.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.100+0000 7f9496000640 1 -- 172.21.15.47:0/3020159578 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f948804b020 con 0x7f94981038b0 2024-03-21T11:21:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:16 smithi100 ceph-mon[36245]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T11:21:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:16 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T11:21:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:16 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] up:active 2024-03-21T11:21:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:16 smithi100 ceph-mon[36245]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T11:21:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:16 smithi100 ceph-mon[36245]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-21T11:21:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:16 smithi100 ceph-mon[36245]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-03-21T11:21:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:16 smithi100 ceph-mon[36245]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-21T11:21:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:16 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2666795657' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T11:21:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:16 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T11:21:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:16 smithi102 ceph-mon[36748]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T11:21:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:16 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T11:21:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:16 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] up:active 2024-03-21T11:21:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:16 smithi102 ceph-mon[36748]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T11:21:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:16 smithi102 ceph-mon[36748]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-21T11:21:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:16 smithi102 ceph-mon[36748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-03-21T11:21:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:16 smithi102 ceph-mon[36748]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-21T11:21:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:16 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2666795657' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T11:21:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:16 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T11:21:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:16 smithi047 ceph-mon[31351]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T11:21:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:16 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T11:21:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:16 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] up:active 2024-03-21T11:21:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:16 smithi047 ceph-mon[31351]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T11:21:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:16 smithi047 ceph-mon[31351]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-21T11:21:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:16 smithi047 ceph-mon[31351]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-03-21T11:21:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:16 smithi047 ceph-mon[31351]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-21T11:21:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:16 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2666795657' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T11:21:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:16 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T11:21:17.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.334+0000 7f949fe00640 1 -- 172.21.15.47:0/3020159578 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f9498061ae0 con 0x7f947c0a2f20 2024-03-21T11:21:17.814 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:21:17 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:21:17.346+0000 7f772d800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T11:21:17.815 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:21:17 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:21:17.346+0000 7f772d800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T11:21:17.815 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:21:17 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:21:17.346+0000 7f772d800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T11:21:17.815 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:21:17 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:21:17.346+0000 7f772d800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T11:21:17.815 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:21:17 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:21:17.346+0000 7f772d800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T11:21:17.853 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:21:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b[66324]: 2024-03-21T11:21:17.501+0000 7f3f74d62700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-21T11:21:17.853 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:21:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b[66324]: 2024-03-21T11:21:17.501+0000 7f3f74d62700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-21T11:21:17.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.906+0000 7f9496000640 1 -- 172.21.15.47:0/3020159578 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9498061ae0 con 0x7f947c0a2f20 2024-03-21T11:21:17.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.910+0000 7f949fe00640 1 -- 172.21.15.47:0/3020159578 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f947c0a2f20 msgr2=0x7f947c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:17.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.910+0000 7f949fe00640 1 --2- 172.21.15.47:0/3020159578 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f947c0a2f20 0x7f947c0a53e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f94980773d0 tx=0x7f9484005f90 comp rx=0 tx=0).stop 2024-03-21T11:21:17.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.910+0000 7f949fe00640 1 -- 172.21.15.47:0/3020159578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94981038b0 msgr2=0x7f9498079450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:17.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.910+0000 7f949fe00640 1 --2- 172.21.15.47:0/3020159578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94981038b0 0x7f9498079450 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f94880324c0 tx=0x7f9488032f40 comp rx=0 tx=0).stop 2024-03-21T11:21:17.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.910+0000 7f949fe00640 1 -- 172.21.15.47:0/3020159578 shutdown_connections 2024-03-21T11:21:17.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.910+0000 7f949fe00640 1 --2- 172.21.15.47:0/3020159578 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f947c0a2f20 0x7f947c0a53e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:17.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.910+0000 7f949fe00640 1 --2- 172.21.15.47:0/3020159578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9498105460 0x7f9498075fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:17.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.910+0000 7f949fe00640 1 --2- 172.21.15.47:0/3020159578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9498104aa0 0x7f9498079990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:17.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.910+0000 7f949fe00640 1 --2- 172.21.15.47:0/3020159578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94981038b0 0x7f9498079450 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:17.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.910+0000 7f949fe00640 1 -- 172.21.15.47:0/3020159578 >> 172.21.15.47:0/3020159578 conn(0x7f94980fee60 msgr2=0x7f9498107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:17.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.910+0000 7f949fe00640 1 -- 172.21.15.47:0/3020159578 shutdown_connections 2024-03-21T11:21:17.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:17.911+0000 7f949fe00640 1 -- 172.21.15.47:0/3020159578 wait complete. 2024-03-21T11:21:17.933 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-21T11:21:18.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:17 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T11:21:18.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:17 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] up:active 2024-03-21T11:21:18.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:17 smithi100 ceph-mon[36245]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-21T11:21:18.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:17 smithi100 ceph-mon[36245]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-03-21T11:21:18.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:17 smithi100 ceph-mon[36245]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-03-21T11:21:18.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:17 smithi100 ceph-mon[36245]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-21T11:21:18.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.149+0000 7fa284400640 1 -- 172.21.15.47:0/3674170620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c104ab0 msgr2=0x7fa27c104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:18.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.149+0000 7fa284400640 1 --2- 172.21.15.47:0/3674170620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c104ab0 0x7fa27c104f30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa270009900 tx=0x7fa27002bc50 comp rx=0 tx=0).stop 2024-03-21T11:21:18.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.149+0000 7fa284400640 1 -- 172.21.15.47:0/3674170620 shutdown_connections 2024-03-21T11:21:18.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.149+0000 7fa284400640 1 --2- 172.21.15.47:0/3674170620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27c105470 0x7fa27c10bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.149+0000 7fa284400640 1 --2- 172.21.15.47:0/3674170620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c104ab0 0x7fa27c104f30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.149+0000 7fa284400640 1 --2- 172.21.15.47:0/3674170620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c1038c0 0x7fa27c103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.149+0000 7fa284400640 1 -- 172.21.15.47:0/3674170620 >> 172.21.15.47:0/3674170620 conn(0x7fa27c0fee50 msgr2=0x7fa27c101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:18.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.149+0000 7fa284400640 1 -- 172.21.15.47:0/3674170620 shutdown_connections 2024-03-21T11:21:18.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.149+0000 7fa284400640 1 -- 172.21.15.47:0/3674170620 wait complete. 2024-03-21T11:21:18.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.150+0000 7fa284400640 1 Processor -- start 2024-03-21T11:21:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.150+0000 7fa284400640 1 -- start start 2024-03-21T11:21:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa284400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c1038c0 0x7fa27c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa284400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27c104ab0 0x7fa27c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa284400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c105470 0x7fa27c1a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa284400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c10f570 con 0x7fa27c104ab0 2024-03-21T11:21:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa284400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c10f3f0 con 0x7fa27c105470 2024-03-21T11:21:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa284400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c10f6f0 con 0x7fa27c1038c0 2024-03-21T11:21:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa281a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c1038c0 0x7fa27c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa281000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27c104ab0 0x7fa27c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa282400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c105470 0x7fa27c1a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa281a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c1038c0 0x7fa27c199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40622/0 (socket says 172.21.15.47:40622) 2024-03-21T11:21:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa281a00640 1 -- 172.21.15.47:0/2318815083 learned_addr learned my addr 172.21.15.47:0/2318815083 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:18.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa281a00640 1 -- 172.21.15.47:0/2318815083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c105470 msgr2=0x7fa27c1a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa281a00640 1 --2- 172.21.15.47:0/2318815083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c105470 0x7fa27c1a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa281a00640 1 -- 172.21.15.47:0/2318815083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27c104ab0 msgr2=0x7fa27c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa281a00640 1 --2- 172.21.15.47:0/2318815083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27c104ab0 0x7fa27c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.151+0000 7fa281a00640 1 -- 172.21.15.47:0/2318815083 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c1a17e0 con 0x7fa27c1038c0 2024-03-21T11:21:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.152+0000 7fa281a00640 1 --2- 172.21.15.47:0/2318815083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c1038c0 0x7fa27c199bc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa26800e9c0 tx=0x7fa26800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.152+0000 7fa27a000640 1 -- 172.21.15.47:0/2318815083 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26800cde0 con 0x7fa27c1038c0 2024-03-21T11:21:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.152+0000 7fa27a000640 1 -- 172.21.15.47:0/2318815083 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa268010910 con 0x7fa27c1038c0 2024-03-21T11:21:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.152+0000 7fa284400640 1 -- 172.21.15.47:0/2318815083 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c1a1ad0 con 0x7fa27c1038c0 2024-03-21T11:21:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.152+0000 7fa27a000640 1 -- 172.21.15.47:0/2318815083 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268018a00 con 0x7fa27c1038c0 2024-03-21T11:21:18.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.152+0000 7fa284400640 1 -- 172.21.15.47:0/2318815083 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c1a2040 con 0x7fa27c1038c0 2024-03-21T11:21:18.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.153+0000 7fa284400640 1 -- 172.21.15.47:0/2318815083 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c10ac30 con 0x7fa27c1038c0 2024-03-21T11:21:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.156+0000 7fa27a000640 1 -- 172.21.15.47:0/2318815083 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa26801f020 con 0x7fa27c1038c0 2024-03-21T11:21:18.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.157+0000 7fa27a000640 1 --2- 172.21.15.47:0/2318815083 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2640a2fa0 0x7fa2640a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:18.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.157+0000 7fa27a000640 1 -- 172.21.15.47:0/2318815083 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa2680cf7e0 con 0x7fa27c1038c0 2024-03-21T11:21:18.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.157+0000 7fa281000640 1 --2- 172.21.15.47:0/2318815083 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2640a2fa0 0x7fa2640a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:18.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.158+0000 7fa281000640 1 --2- 172.21.15.47:0/2318815083 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2640a2fa0 0x7fa2640a5460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa27002c260 tx=0x7fa270000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:18.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.161+0000 7fa27a000640 1 -- 172.21.15.47:0/2318815083 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa26809c370 con 0x7fa27c1038c0 2024-03-21T11:21:18.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:17 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T11:21:18.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:17 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] up:active 2024-03-21T11:21:18.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:17 smithi102 ceph-mon[36748]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-21T11:21:18.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:17 smithi102 ceph-mon[36748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-03-21T11:21:18.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:17 smithi102 ceph-mon[36748]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-03-21T11:21:18.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:17 smithi102 ceph-mon[36748]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-21T11:21:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:17 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T11:21:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:17 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] up:active 2024-03-21T11:21:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:17 smithi047 ceph-mon[31351]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-21T11:21:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:17 smithi047 ceph-mon[31351]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-03-21T11:21:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:17 smithi047 ceph-mon[31351]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-03-21T11:21:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:17 smithi047 ceph-mon[31351]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-21T11:21:18.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.394+0000 7fa284400640 1 -- 172.21.15.47:0/2318815083 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fa27c061b70 con 0x7fa2640a2fa0 2024-03-21T11:21:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.421+0000 7fa27a000640 1 -- 172.21.15.47:0/2318815083 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa27c061b70 con 0x7fa2640a2fa0 2024-03-21T11:21:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.424+0000 7fa284400640 1 -- 172.21.15.47:0/2318815083 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2640a2fa0 msgr2=0x7fa2640a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.424+0000 7fa284400640 1 --2- 172.21.15.47:0/2318815083 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2640a2fa0 0x7fa2640a5460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa27002c260 tx=0x7fa270000bc0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.424+0000 7fa284400640 1 -- 172.21.15.47:0/2318815083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c1038c0 msgr2=0x7fa27c199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.424+0000 7fa284400640 1 --2- 172.21.15.47:0/2318815083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c1038c0 0x7fa27c199bc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa26800e9c0 tx=0x7fa26800ee80 comp rx=0 tx=0).stop 2024-03-21T11:21:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.424+0000 7fa284400640 1 -- 172.21.15.47:0/2318815083 shutdown_connections 2024-03-21T11:21:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.424+0000 7fa284400640 1 --2- 172.21.15.47:0/2318815083 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2640a2fa0 0x7fa2640a5460 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.424+0000 7fa284400640 1 --2- 172.21.15.47:0/2318815083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c105470 0x7fa27c1a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.424+0000 7fa284400640 1 --2- 172.21.15.47:0/2318815083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27c104ab0 0x7fa27c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.424+0000 7fa284400640 1 --2- 172.21.15.47:0/2318815083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa27c1038c0 0x7fa27c199bc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.424+0000 7fa284400640 1 -- 172.21.15.47:0/2318815083 >> 172.21.15.47:0/2318815083 conn(0x7fa27c0fee50 msgr2=0x7fa27c107e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.424+0000 7fa284400640 1 -- 172.21.15.47:0/2318815083 shutdown_connections 2024-03-21T11:21:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.424+0000 7fa284400640 1 -- 172.21.15.47:0/2318815083 wait complete. 2024-03-21T11:21:18.446 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-21T11:21:18.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.676+0000 7f4afc600640 1 -- 172.21.15.47:0/2569375633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af4104ad0 msgr2=0x7f4af4104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:18.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.676+0000 7f4afc600640 1 --2- 172.21.15.47:0/2569375633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af4104ad0 0x7f4af4104f50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009860 tx=0x7f4ae002bb70 comp rx=0 tx=0).stop 2024-03-21T11:21:18.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.676+0000 7f4afc600640 1 -- 172.21.15.47:0/2569375633 shutdown_connections 2024-03-21T11:21:18.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.676+0000 7f4afc600640 1 --2- 172.21.15.47:0/2569375633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4af4105490 0x7f4af410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.676+0000 7f4afc600640 1 --2- 172.21.15.47:0/2569375633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af4104ad0 0x7f4af4104f50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.676+0000 7f4afc600640 1 --2- 172.21.15.47:0/2569375633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4af41038e0 0x7f4af4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.676+0000 7f4afc600640 1 -- 172.21.15.47:0/2569375633 >> 172.21.15.47:0/2569375633 conn(0x7f4af40fee50 msgr2=0x7f4af4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:18.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.676+0000 7f4afc600640 1 -- 172.21.15.47:0/2569375633 shutdown_connections 2024-03-21T11:21:18.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.676+0000 7f4afc600640 1 -- 172.21.15.47:0/2569375633 wait complete. 2024-03-21T11:21:18.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.677+0000 7f4afc600640 1 Processor -- start 2024-03-21T11:21:18.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.678+0000 7f4afc600640 1 -- start start 2024-03-21T11:21:18.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.678+0000 7f4afc600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4af41038e0 0x7f4af4199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:18.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.678+0000 7f4afc600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4af4104ad0 0x7f4af419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:18.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.678+0000 7f4afc600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af4105490 0x7f4af41a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:18.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.678+0000 7f4afc600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af410f560 con 0x7f4af4104ad0 2024-03-21T11:21:18.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.678+0000 7f4afc600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af410f3e0 con 0x7f4af4105490 2024-03-21T11:21:18.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.678+0000 7f4afc600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af410f6e0 con 0x7f4af41038e0 2024-03-21T11:21:18.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.678+0000 7f4afa600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af4105490 0x7f4af41a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:18.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.678+0000 7f4af9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4af41038e0 0x7f4af4199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:18.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.678+0000 7f4af9200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4af4104ad0 0x7f4af419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:18.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.678+0000 7f4afa600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af4105490 0x7f4af41a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48124/0 (socket says 172.21.15.47:48124) 2024-03-21T11:21:18.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.678+0000 7f4afa600640 1 -- 172.21.15.47:0/1554338353 learned_addr learned my addr 172.21.15.47:0/1554338353 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:18.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.679+0000 7f4afa600640 1 -- 172.21.15.47:0/1554338353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4af41038e0 msgr2=0x7f4af4199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:18.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.679+0000 7f4afa600640 1 --2- 172.21.15.47:0/1554338353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4af41038e0 0x7f4af4199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.679+0000 7f4afa600640 1 -- 172.21.15.47:0/1554338353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4af4104ad0 msgr2=0x7f4af419a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:18.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.679+0000 7f4afa600640 1 --2- 172.21.15.47:0/1554338353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4af4104ad0 0x7f4af419a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.679+0000 7f4afa600640 1 -- 172.21.15.47:0/1554338353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af41a1800 con 0x7f4af4105490 2024-03-21T11:21:18.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.679+0000 7f4af9200640 1 --2- 172.21.15.47:0/1554338353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4af4104ad0 0x7f4af419a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:21:18.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.679+0000 7f4afa600640 1 --2- 172.21.15.47:0/1554338353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af4105490 0x7f4af41a1190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00e970 tx=0x7f4aec00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:18.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.679+0000 7f4af2000640 1 -- 172.21.15.47:0/1554338353 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec00ccc0 con 0x7f4af4105490 2024-03-21T11:21:18.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.679+0000 7f4af2000640 1 -- 172.21.15.47:0/1554338353 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4aec00ce60 con 0x7f4af4105490 2024-03-21T11:21:18.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.679+0000 7f4af2000640 1 -- 172.21.15.47:0/1554338353 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec010490 con 0x7f4af4105490 2024-03-21T11:21:18.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.679+0000 7f4afc600640 1 -- 172.21.15.47:0/1554338353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af41a1af0 con 0x7f4af4105490 2024-03-21T11:21:18.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.679+0000 7f4afc600640 1 -- 172.21.15.47:0/1554338353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af41a1fc0 con 0x7f4af4105490 2024-03-21T11:21:18.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.680+0000 7f4afc600640 1 -- 172.21.15.47:0/1554338353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af410acd0 con 0x7f4af4105490 2024-03-21T11:21:18.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.682+0000 7f4af2000640 1 -- 172.21.15.47:0/1554338353 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4aec005020 con 0x7f4af4105490 2024-03-21T11:21:18.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.684+0000 7f4af2000640 1 --2- 172.21.15.47:0/1554338353 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4adc0a2f20 0x7f4adc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:18.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.684+0000 7f4af2000640 1 -- 172.21.15.47:0/1554338353 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4aec0cfad0 con 0x7f4af4105490 2024-03-21T11:21:18.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.684+0000 7f4af9c00640 1 --2- 172.21.15.47:0/1554338353 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4adc0a2f20 0x7f4adc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:18.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.685+0000 7f4af9c00640 1 --2- 172.21.15.47:0/1554338353 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4adc0a2f20 0x7f4adc0a53e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4af4104930 tx=0x7f4ae8010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:18.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.688+0000 7f4af2000640 1 -- 172.21.15.47:0/1554338353 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4aec09c660 con 0x7f4af4105490 2024-03-21T11:21:18.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.923+0000 7f4afc600640 1 -- 172.21.15.47:0/1554338353 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f4af4061ae0 con 0x7f4adc0a2f20 2024-03-21T11:21:18.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.929+0000 7f4af2000640 1 -- 172.21.15.47:0/1554338353 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f4af4061ae0 con 0x7f4adc0a2f20 2024-03-21T11:21:18.930 INFO:teuthology.orchestra.run.smithi047.stdout:/volumes/qa/sv_0/23e9986f-ff9d-43f2-a292-41bb15c54f08 2024-03-21T11:21:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.932+0000 7f4afc600640 1 -- 172.21.15.47:0/1554338353 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4adc0a2f20 msgr2=0x7f4adc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.932+0000 7f4afc600640 1 --2- 172.21.15.47:0/1554338353 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4adc0a2f20 0x7f4adc0a53e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4af4104930 tx=0x7f4ae8010040 comp rx=0 tx=0).stop 2024-03-21T11:21:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.932+0000 7f4afc600640 1 -- 172.21.15.47:0/1554338353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af4105490 msgr2=0x7f4af41a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:18.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.932+0000 7f4afc600640 1 --2- 172.21.15.47:0/1554338353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af4105490 0x7f4af41a1190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00e970 tx=0x7f4aec00ed70 comp rx=0 tx=0).stop 2024-03-21T11:21:18.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.932+0000 7f4afc600640 1 -- 172.21.15.47:0/1554338353 shutdown_connections 2024-03-21T11:21:18.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.932+0000 7f4afc600640 1 --2- 172.21.15.47:0/1554338353 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4adc0a2f20 0x7f4adc0a53e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.932+0000 7f4afc600640 1 --2- 172.21.15.47:0/1554338353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af4105490 0x7f4af41a1190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.932+0000 7f4afc600640 1 --2- 172.21.15.47:0/1554338353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4af4104ad0 0x7f4af419a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.932+0000 7f4afc600640 1 --2- 172.21.15.47:0/1554338353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4af41038e0 0x7f4af4199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:18.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.932+0000 7f4afc600640 1 -- 172.21.15.47:0/1554338353 >> 172.21.15.47:0/1554338353 conn(0x7f4af40fee50 msgr2=0x7f4af4107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:18.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.932+0000 7f4afc600640 1 -- 172.21.15.47:0/1554338353 shutdown_connections 2024-03-21T11:21:18.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:18.932+0000 7f4afc600640 1 -- 172.21.15.47:0/1554338353 wait complete. 2024-03-21T11:21:18.954 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-21T11:21:19.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:18 smithi100 ceph-mon[36245]: pgmap v335: 129 pgs: 129 active+clean; 583 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-03-21T11:21:19.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:18 smithi100 ceph-mon[36245]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:18 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] up:active 2024-03-21T11:21:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:18 smithi100 ceph-mon[36245]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-21T11:21:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:18 smithi100 ceph-mon[36245]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T11:21:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:18 smithi100 ceph-mon[36245]: Cluster is now healthy 2024-03-21T11:21:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:18 smithi100 ceph-mon[36245]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-03-21T11:21:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:18 smithi100 ceph-mon[36245]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-21T11:21:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:18 smithi100 ceph-mon[36245]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-03-21T11:21:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:18 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] up:active 2024-03-21T11:21:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:18 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:21:19.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.185+0000 7f0c3fc00640 1 -- 172.21.15.47:0/3206479121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c38075660 msgr2=0x7f0c38073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:19.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.185+0000 7f0c3fc00640 1 --2- 172.21.15.47:0/3206479121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c38075660 0x7f0c38073ab0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009900 tx=0x7f0c2802bcc0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.185+0000 7f0c3fc00640 1 -- 172.21.15.47:0/3206479121 shutdown_connections 2024-03-21T11:21:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.185+0000 7f0c3fc00640 1 --2- 172.21.15.47:0/3206479121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c38109560 0x7f0c3810b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.185+0000 7f0c3fc00640 1 --2- 172.21.15.47:0/3206479121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38074080 0x7f0c380744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.185+0000 7f0c3fc00640 1 --2- 172.21.15.47:0/3206479121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c38075660 0x7f0c38073ab0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.185+0000 7f0c3fc00640 1 -- 172.21.15.47:0/3206479121 >> 172.21.15.47:0/3206479121 conn(0x7f0c380fcc40 msgr2=0x7f0c380ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.186+0000 7f0c3fc00640 1 -- 172.21.15.47:0/3206479121 shutdown_connections 2024-03-21T11:21:19.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.186+0000 7f0c3fc00640 1 -- 172.21.15.47:0/3206479121 wait complete. 2024-03-21T11:21:19.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.186+0000 7f0c3fc00640 1 Processor -- start 2024-03-21T11:21:19.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.187+0000 7f0c3fc00640 1 -- start start 2024-03-21T11:21:19.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.187+0000 7f0c3fc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38074080 0x7f0c38199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:19.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.187+0000 7f0c3fc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c38075660 0x7f0c3819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:19.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.187+0000 7f0c3fc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c38109560 0x7f0c381a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:19.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.187+0000 7f0c3fc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3810f560 con 0x7f0c38074080 2024-03-21T11:21:19.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.187+0000 7f0c3fc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3810f3e0 con 0x7f0c38109560 2024-03-21T11:21:19.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.187+0000 7f0c3fc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3810f6e0 con 0x7f0c38075660 2024-03-21T11:21:19.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.187+0000 7f0c3dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c38109560 0x7f0c381a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:19.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.187+0000 7f0c3dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c38109560 0x7f0c381a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40604/0 (socket says 172.21.15.47:40604) 2024-03-21T11:21:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.187+0000 7f0c3dc00640 1 -- 172.21.15.47:0/1658086 learned_addr learned my addr 172.21.15.47:0/1658086 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.187+0000 7f0c37e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c38075660 0x7f0c3819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.187+0000 7f0c3d200640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38074080 0x7f0c38199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.188+0000 7f0c3dc00640 1 -- 172.21.15.47:0/1658086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c38075660 msgr2=0x7f0c3819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.188+0000 7f0c3dc00640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c38075660 0x7f0c3819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.188+0000 7f0c3dc00640 1 -- 172.21.15.47:0/1658086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38074080 msgr2=0x7f0c38199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.188+0000 7f0c3dc00640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38074080 0x7f0c38199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.188+0000 7f0c3dc00640 1 -- 172.21.15.47:0/1658086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c381a1830 con 0x7f0c38109560 2024-03-21T11:21:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.188+0000 7f0c3d200640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38074080 0x7f0c38199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:21:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.188+0000 7f0c37e00640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c38075660 0x7f0c3819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:21:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.188+0000 7f0c3dc00640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c38109560 0x7f0c381a1190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000efd0 tx=0x7f0c3000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.188+0000 7f0c35600640 1 -- 172.21.15.47:0/1658086 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30003b70 con 0x7f0c38109560 2024-03-21T11:21:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.188+0000 7f0c35600640 1 -- 172.21.15.47:0/1658086 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c30003d10 con 0x7f0c38109560 2024-03-21T11:21:19.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.188+0000 7f0c3fc00640 1 -- 172.21.15.47:0/1658086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c381a1b20 con 0x7f0c38109560 2024-03-21T11:21:19.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.188+0000 7f0c35600640 1 -- 172.21.15.47:0/1658086 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c300109f0 con 0x7f0c38109560 2024-03-21T11:21:19.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.188+0000 7f0c3fc00640 1 -- 172.21.15.47:0/1658086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c381a1ff0 con 0x7f0c38109560 2024-03-21T11:21:19.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.189+0000 7f0c3fc00640 1 -- 172.21.15.47:0/1658086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c38108320 con 0x7f0c38109560 2024-03-21T11:21:19.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.191+0000 7f0c35600640 1 -- 172.21.15.47:0/1658086 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0c30004020 con 0x7f0c38109560 2024-03-21T11:21:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.192+0000 7f0c35600640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c1c0a2f20 0x7f0c1c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.193+0000 7f0c35600640 1 -- 172.21.15.47:0/1658086 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f0c300cfc10 con 0x7f0c38109560 2024-03-21T11:21:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.193+0000 7f0c3d200640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c1c0a2f20 0x7f0c1c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:19.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.194+0000 7f0c3d200640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c1c0a2f20 0x7f0c1c0a53e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c2802c2d0 tx=0x7f0c2802c910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.201+0000 7f0c35600640 1 -- 172.21.15.47:0/1658086 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0c3009c7a0 con 0x7f0c38109560 2024-03-21T11:21:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:18 smithi102 ceph-mon[36748]: pgmap v335: 129 pgs: 129 active+clean; 583 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-03-21T11:21:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:18 smithi102 ceph-mon[36748]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:18 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] up:active 2024-03-21T11:21:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:18 smithi102 ceph-mon[36748]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-21T11:21:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:18 smithi102 ceph-mon[36748]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T11:21:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:18 smithi102 ceph-mon[36748]: Cluster is now healthy 2024-03-21T11:21:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:18 smithi102 ceph-mon[36748]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-03-21T11:21:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:18 smithi102 ceph-mon[36748]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-21T11:21:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:18 smithi102 ceph-mon[36748]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-03-21T11:21:19.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:18 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] up:active 2024-03-21T11:21:19.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:18 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:21:19.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:18 smithi047 ceph-mon[31351]: pgmap v335: 129 pgs: 129 active+clean; 583 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-03-21T11:21:19.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:18 smithi047 ceph-mon[31351]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:19.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:18 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] up:active 2024-03-21T11:21:19.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:18 smithi047 ceph-mon[31351]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-21T11:21:19.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:18 smithi047 ceph-mon[31351]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T11:21:19.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:18 smithi047 ceph-mon[31351]: Cluster is now healthy 2024-03-21T11:21:19.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:18 smithi047 ceph-mon[31351]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-03-21T11:21:19.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:18 smithi047 ceph-mon[31351]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-21T11:21:19.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:18 smithi047 ceph-mon[31351]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-03-21T11:21:19.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:18 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] up:active 2024-03-21T11:21:19.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:18 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:21:19.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.433+0000 7f0c3fc00640 1 -- 172.21.15.47:0/1658086 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c38061ae0 con 0x7f0c1c0a2f20 2024-03-21T11:21:19.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.467+0000 7f0c35600640 1 -- 172.21.15.47:0/1658086 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0c38061ae0 con 0x7f0c1c0a2f20 2024-03-21T11:21:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.470+0000 7f0c3fc00640 1 -- 172.21.15.47:0/1658086 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c1c0a2f20 msgr2=0x7f0c1c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:19.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.470+0000 7f0c3fc00640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c1c0a2f20 0x7f0c1c0a53e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c2802c2d0 tx=0x7f0c2802c910 comp rx=0 tx=0).stop 2024-03-21T11:21:19.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.470+0000 7f0c3fc00640 1 -- 172.21.15.47:0/1658086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c38109560 msgr2=0x7f0c381a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:19.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.470+0000 7f0c3fc00640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c38109560 0x7f0c381a1190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000efd0 tx=0x7f0c3000c6a0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.471+0000 7f0c3fc00640 1 -- 172.21.15.47:0/1658086 shutdown_connections 2024-03-21T11:21:19.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.471+0000 7f0c3fc00640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c1c0a2f20 0x7f0c1c0a53e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.471+0000 7f0c3fc00640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c38109560 0x7f0c381a1190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.471+0000 7f0c3fc00640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c38075660 0x7f0c3819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.471+0000 7f0c3fc00640 1 --2- 172.21.15.47:0/1658086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38074080 0x7f0c38199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.471+0000 7f0c3fc00640 1 -- 172.21.15.47:0/1658086 >> 172.21.15.47:0/1658086 conn(0x7f0c380fcc40 msgr2=0x7f0c381054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:19.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.471+0000 7f0c3fc00640 1 -- 172.21.15.47:0/1658086 shutdown_connections 2024-03-21T11:21:19.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.471+0000 7f0c3fc00640 1 -- 172.21.15.47:0/1658086 wait complete. 2024-03-21T11:21:19.491 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-21T11:21:19.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.718+0000 7fdf15000640 1 -- 172.21.15.47:0/4155338344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf10074100 msgr2=0x7fdf10074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.718+0000 7fdf15000640 1 --2- 172.21.15.47:0/4155338344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf10074100 0x7fdf10074580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009930 tx=0x7fdef802bc30 comp rx=0 tx=0).stop 2024-03-21T11:21:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.718+0000 7fdf15000640 1 -- 172.21.15.47:0/4155338344 shutdown_connections 2024-03-21T11:21:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.718+0000 7fdf15000640 1 --2- 172.21.15.47:0/4155338344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1010db70 0x7fdf1010ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.718+0000 7fdf15000640 1 --2- 172.21.15.47:0/4155338344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf10074100 0x7fdf10074580 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.718+0000 7fdf15000640 1 --2- 172.21.15.47:0/4155338344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf100756e0 0x7fdf10073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.718+0000 7fdf15000640 1 -- 172.21.15.47:0/4155338344 >> 172.21.15.47:0/4155338344 conn(0x7fdf100fcb10 msgr2=0x7fdf100fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.718+0000 7fdf15000640 1 -- 172.21.15.47:0/4155338344 shutdown_connections 2024-03-21T11:21:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.718+0000 7fdf15000640 1 -- 172.21.15.47:0/4155338344 wait complete. 2024-03-21T11:21:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.719+0000 7fdf15000640 1 Processor -- start 2024-03-21T11:21:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.719+0000 7fdf15000640 1 -- start start 2024-03-21T11:21:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.720+0000 7fdf15000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf10074100 0x7fdf1019dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.720+0000 7fdf15000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf100756e0 0x7fdf1019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.720+0000 7fdf15000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1010db70 0x7fdf101a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.720+0000 7fdf15000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf10113860 con 0x7fdf1010db70 2024-03-21T11:21:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.720+0000 7fdf15000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf101136e0 con 0x7fdf100756e0 2024-03-21T11:21:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.720+0000 7fdf15000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf101139e0 con 0x7fdf10074100 2024-03-21T11:21:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.720+0000 7fdf0f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf10074100 0x7fdf1019dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.720+0000 7fdf0f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf10074100 0x7fdf1019dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32966/0 (socket says 172.21.15.47:32966) 2024-03-21T11:21:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.720+0000 7fdf0f400640 1 -- 172.21.15.47:0/1436712312 learned_addr learned my addr 172.21.15.47:0/1436712312 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.720+0000 7fdf0ea00640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf100756e0 0x7fdf1019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.720+0000 7fdf0fe00640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1010db70 0x7fdf101a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.721+0000 7fdf0f400640 1 -- 172.21.15.47:0/1436712312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf100756e0 msgr2=0x7fdf1019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.721+0000 7fdf0f400640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf100756e0 0x7fdf1019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.721+0000 7fdf0f400640 1 -- 172.21.15.47:0/1436712312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1010db70 msgr2=0x7fdf101a5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.721+0000 7fdf0f400640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1010db70 0x7fdf101a5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.721+0000 7fdf0f400640 1 -- 172.21.15.47:0/1436712312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf101a5b00 con 0x7fdf10074100 2024-03-21T11:21:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.721+0000 7fdf0fe00640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1010db70 0x7fdf101a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:21:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.721+0000 7fdf0f400640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf10074100 0x7fdf1019dee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000eb90 tx=0x7fdf0000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.721+0000 7fdf07e00640 1 -- 172.21.15.47:0/1436712312 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00004d20 con 0x7fdf10074100 2024-03-21T11:21:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.721+0000 7fdf07e00640 1 -- 172.21.15.47:0/1436712312 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf0000ee90 con 0x7fdf10074100 2024-03-21T11:21:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.721+0000 7fdf15000640 1 -- 172.21.15.47:0/1436712312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf101a5df0 con 0x7fdf10074100 2024-03-21T11:21:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.721+0000 7fdf07e00640 1 -- 172.21.15.47:0/1436712312 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00010830 con 0x7fdf10074100 2024-03-21T11:21:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.721+0000 7fdf15000640 1 -- 172.21.15.47:0/1436712312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf101a6200 con 0x7fdf10074100 2024-03-21T11:21:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.722+0000 7fdf15000640 1 -- 172.21.15.47:0/1436712312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf10103c30 con 0x7fdf10074100 2024-03-21T11:21:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.724+0000 7fdf07e00640 1 -- 172.21.15.47:0/1436712312 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdf00044b40 con 0x7fdf10074100 2024-03-21T11:21:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.726+0000 7fdf07e00640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdef40a2f20 0x7fdef40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.726+0000 7fdf07e00640 1 -- 172.21.15.47:0/1436712312 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fdf000cfd70 con 0x7fdf10074100 2024-03-21T11:21:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.726+0000 7fdf0ea00640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdef40a2f20 0x7fdef40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.727+0000 7fdf0ea00640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdef40a2f20 0x7fdef40a53e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdef8036040 tx=0x7fdef802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:19.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.732+0000 7fdf07e00640 1 -- 172.21.15.47:0/1436712312 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdf0009c900 con 0x7fdf10074100 2024-03-21T11:21:19.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.893+0000 7fdf07e00640 1 -- 172.21.15.47:0/1436712312 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fdf00098870 con 0x7fdf10074100 2024-03-21T11:21:19.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.975+0000 7fdf15000640 1 -- 172.21.15.47:0/1436712312 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf10061ae0 con 0x7fdef40a2f20 2024-03-21T11:21:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.983+0000 7fdf07e00640 1 -- 172.21.15.47:0/1436712312 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fdf10061ae0 con 0x7fdef40a2f20 2024-03-21T11:21:19.983 INFO:teuthology.orchestra.run.smithi047.stdout:/volumes/qa/sv_1/0b5c929a-10c2-48bb-bf4c-fe3d6041cfdc 2024-03-21T11:21:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.985+0000 7fdf15000640 1 -- 172.21.15.47:0/1436712312 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdef40a2f20 msgr2=0x7fdef40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.985+0000 7fdf15000640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdef40a2f20 0x7fdef40a53e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdef8036040 tx=0x7fdef802df90 comp rx=0 tx=0).stop 2024-03-21T11:21:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.985+0000 7fdf15000640 1 -- 172.21.15.47:0/1436712312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf10074100 msgr2=0x7fdf1019dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.985+0000 7fdf15000640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf10074100 0x7fdf1019dee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000eb90 tx=0x7fdf0000c6a0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.985+0000 7fdf15000640 1 -- 172.21.15.47:0/1436712312 shutdown_connections 2024-03-21T11:21:19.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.985+0000 7fdf15000640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdef40a2f20 0x7fdef40a53e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.986+0000 7fdf15000640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf1010db70 0x7fdf101a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.986+0000 7fdf15000640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf100756e0 0x7fdf1019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.986+0000 7fdf15000640 1 --2- 172.21.15.47:0/1436712312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf10074100 0x7fdf1019dee0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:19.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.986+0000 7fdf15000640 1 -- 172.21.15.47:0/1436712312 >> 172.21.15.47:0/1436712312 conn(0x7fdf100fcb10 msgr2=0x7fdf10100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:19.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.986+0000 7fdf15000640 1 -- 172.21.15.47:0/1436712312 shutdown_connections 2024-03-21T11:21:19.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:19.986+0000 7fdf15000640 1 -- 172.21.15.47:0/1436712312 wait complete. 2024-03-21T11:21:20.006 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:21:20.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:19 smithi100 ceph-mon[36245]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:20.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.226+0000 7fddbac00640 1 -- 172.21.15.47:0/1803711218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb4107b00 msgr2=0x7fddb4107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:20.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.226+0000 7fddbac00640 1 --2- 172.21.15.47:0/1803711218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb4107b00 0x7fddb4107f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009900 tx=0x7fdda802bc30 comp rx=0 tx=0).stop 2024-03-21T11:21:20.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.226+0000 7fddbac00640 1 -- 172.21.15.47:0/1803711218 shutdown_connections 2024-03-21T11:21:20.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.226+0000 7fddbac00640 1 --2- 172.21.15.47:0/1803711218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddb41096b0 0x7fddb410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.226+0000 7fddbac00640 1 --2- 172.21.15.47:0/1803711218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb4108cf0 0x7fddb4109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.226+0000 7fddbac00640 1 --2- 172.21.15.47:0/1803711218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb4107b00 0x7fddb4107f20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.226+0000 7fddbac00640 1 -- 172.21.15.47:0/1803711218 >> 172.21.15.47:0/1803711218 conn(0x7fddb4076ab0 msgr2=0x7fddb4078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:20.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.226+0000 7fddbac00640 1 -- 172.21.15.47:0/1803711218 shutdown_connections 2024-03-21T11:21:20.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.227+0000 7fddbac00640 1 -- 172.21.15.47:0/1803711218 wait complete. 2024-03-21T11:21:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.228+0000 7fddbac00640 1 Processor -- start 2024-03-21T11:21:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.228+0000 7fddbac00640 1 -- start start 2024-03-21T11:21:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.228+0000 7fddbac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb4107b00 0x7fddb419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.228+0000 7fddbac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddb4108cf0 0x7fddb419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.228+0000 7fddbac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb41096b0 0x7fddb41a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.228+0000 7fddbac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb4113770 con 0x7fddb4108cf0 2024-03-21T11:21:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.228+0000 7fddbac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb41135f0 con 0x7fddb41096b0 2024-03-21T11:21:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.228+0000 7fddbac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb41138f0 con 0x7fddb4107b00 2024-03-21T11:21:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.228+0000 7fddb9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb4107b00 0x7fddb419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.228+0000 7fddb9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb4107b00 0x7fddb419de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32976/0 (socket says 172.21.15.47:32976) 2024-03-21T11:21:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.228+0000 7fddb9800640 1 -- 172.21.15.47:0/1369373967 learned_addr learned my addr 172.21.15.47:0/1369373967 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.229+0000 7fddb8e00640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddb4108cf0 0x7fddb419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.229+0000 7fddba200640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb41096b0 0x7fddb41a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.229+0000 7fddb9800640 1 -- 172.21.15.47:0/1369373967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb41096b0 msgr2=0x7fddb41a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.229+0000 7fddb9800640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb41096b0 0x7fddb41a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.229+0000 7fddb9800640 1 -- 172.21.15.47:0/1369373967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddb4108cf0 msgr2=0x7fddb419e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.229+0000 7fddb9800640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddb4108cf0 0x7fddb419e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.229+0000 7fddb9800640 1 -- 172.21.15.47:0/1369373967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb41a5a30 con 0x7fddb4107b00 2024-03-21T11:21:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.229+0000 7fddba200640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb41096b0 0x7fddb41a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:21:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.229+0000 7fddb9800640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb4107b00 0x7fddb419de10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdda8000c00 tx=0x7fdda8005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.229+0000 7fddb2000640 1 -- 172.21.15.47:0/1369373967 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda8041070 con 0x7fddb4107b00 2024-03-21T11:21:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.229+0000 7fddb2000640 1 -- 172.21.15.47:0/1369373967 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdda80044d0 con 0x7fddb4107b00 2024-03-21T11:21:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.229+0000 7fddb2000640 1 -- 172.21.15.47:0/1369373967 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda8034440 con 0x7fddb4107b00 2024-03-21T11:21:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.229+0000 7fddbac00640 1 -- 172.21.15.47:0/1369373967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb41a5cc0 con 0x7fddb4107b00 2024-03-21T11:21:20.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.230+0000 7fddbac00640 1 -- 172.21.15.47:0/1369373967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb41a6160 con 0x7fddb4107b00 2024-03-21T11:21:20.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.231+0000 7fddbac00640 1 -- 172.21.15.47:0/1369373967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb410eea0 con 0x7fddb4107b00 2024-03-21T11:21:20.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.233+0000 7fddb2000640 1 -- 172.21.15.47:0/1369373967 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fdda8044070 con 0x7fddb4107b00 2024-03-21T11:21:20.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.234+0000 7fddb2000640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd9c0a3030 0x7fdd9c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:20.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.234+0000 7fddb2000640 1 -- 172.21.15.47:0/1369373967 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fdda80f1420 con 0x7fddb4107b00 2024-03-21T11:21:20.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.234+0000 7fddb8e00640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd9c0a3030 0x7fdd9c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:20.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.235+0000 7fddb8e00640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd9c0a3030 0x7fdd9c0a54f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fddb419f320 tx=0x7fdda0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:20.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.236+0000 7fddb2000640 1 -- 172.21.15.47:0/1369373967 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdda80bdf60 con 0x7fddb4107b00 2024-03-21T11:21:20.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:19 smithi102 ceph-mon[36748]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:20.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:19 smithi047 ceph-mon[31351]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:20.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.482+0000 7fddbac00640 1 -- 172.21.15.47:0/1369373967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddb402d190 con 0x7fddb4107b00 2024-03-21T11:21:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.483+0000 7fddb2000640 1 -- 172.21.15.47:0/1369373967 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fdda80bd860 con 0x7fddb4107b00 2024-03-21T11:21:20.484 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:21:20.484 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:21:18.592954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24524,24523,24535,24581,24553]},"id":1}]} 2024-03-21T11:21:20.485 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 27 2024-03-21T11:21:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.486+0000 7fddbac00640 1 -- 172.21.15.47:0/1369373967 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd9c0a3030 msgr2=0x7fdd9c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.486+0000 7fddbac00640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd9c0a3030 0x7fdd9c0a54f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fddb419f320 tx=0x7fdda0005f90 comp rx=0 tx=0).stop 2024-03-21T11:21:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.486+0000 7fddbac00640 1 -- 172.21.15.47:0/1369373967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb4107b00 msgr2=0x7fddb419de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.486+0000 7fddbac00640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb4107b00 0x7fddb419de10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdda8000c00 tx=0x7fdda8005d50 comp rx=0 tx=0).stop 2024-03-21T11:21:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.487+0000 7fddbac00640 1 -- 172.21.15.47:0/1369373967 shutdown_connections 2024-03-21T11:21:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.487+0000 7fddbac00640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd9c0a3030 0x7fdd9c0a54f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.487+0000 7fddbac00640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddb41096b0 0x7fddb41a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.487+0000 7fddbac00640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddb4108cf0 0x7fddb419e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.487+0000 7fddbac00640 1 --2- 172.21.15.47:0/1369373967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb4107b00 0x7fddb419de10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.487+0000 7fddbac00640 1 -- 172.21.15.47:0/1369373967 >> 172.21.15.47:0/1369373967 conn(0x7fddb4076ab0 msgr2=0x7fddb410c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.487+0000 7fddbac00640 1 -- 172.21.15.47:0/1369373967 shutdown_connections 2024-03-21T11:21:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.487+0000 7fddbac00640 1 -- 172.21.15.47:0/1369373967 wait complete. 2024-03-21T11:21:20.506 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:21:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.739+0000 7f88ba000640 1 -- 172.21.15.47:0/1130485477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b4109890 msgr2=0x7f88b4110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.739+0000 7f88ba000640 1 --2- 172.21.15.47:0/1130485477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b4109890 0x7f88b4110120 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009900 tx=0x7f88ac02bc30 comp rx=0 tx=0).stop 2024-03-21T11:21:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.739+0000 7f88ba000640 1 -- 172.21.15.47:0/1130485477 shutdown_connections 2024-03-21T11:21:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.739+0000 7f88ba000640 1 --2- 172.21.15.47:0/1130485477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b4109890 0x7f88b4110120 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.739+0000 7f88ba000640 1 --2- 172.21.15.47:0/1130485477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b4108ed0 0x7f88b4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.739+0000 7f88ba000640 1 --2- 172.21.15.47:0/1130485477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b4107ce0 0x7f88b4108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.739+0000 7f88ba000640 1 -- 172.21.15.47:0/1130485477 >> 172.21.15.47:0/1130485477 conn(0x7f88b4076cb0 msgr2=0x7f88b40790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:20.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.739+0000 7f88ba000640 1 -- 172.21.15.47:0/1130485477 shutdown_connections 2024-03-21T11:21:20.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.739+0000 7f88ba000640 1 -- 172.21.15.47:0/1130485477 wait complete. 2024-03-21T11:21:20.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.740+0000 7f88ba000640 1 Processor -- start 2024-03-21T11:21:20.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.740+0000 7f88ba000640 1 -- start start 2024-03-21T11:21:20.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88ba000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b4107ce0 0x7f88b419dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:20.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88ba000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b4108ed0 0x7f88b419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:20.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88ba000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b4109890 0x7f88b41a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:20.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88ba000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b4113960 con 0x7f88b4107ce0 2024-03-21T11:21:20.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88ba000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b41137e0 con 0x7f88b4109890 2024-03-21T11:21:20.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88ba000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b4113ae0 con 0x7f88b4108ed0 2024-03-21T11:21:20.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88b2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b4108ed0 0x7f88b419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:20.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88b3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b4107ce0 0x7f88b419dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:20.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88b2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b4108ed0 0x7f88b419e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33000/0 (socket says 172.21.15.47:33000) 2024-03-21T11:21:20.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88b2a00640 1 -- 172.21.15.47:0/1184893204 learned_addr learned my addr 172.21.15.47:0/1184893204 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:20.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88b3e00640 1 --2- 172.21.15.47:0/1184893204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b4109890 0x7f88b41a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:20.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88b2a00640 1 -- 172.21.15.47:0/1184893204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b4109890 msgr2=0x7f88b41a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:20.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88b2a00640 1 --2- 172.21.15.47:0/1184893204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b4109890 0x7f88b41a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88b2a00640 1 -- 172.21.15.47:0/1184893204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b4107ce0 msgr2=0x7f88b419dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:20.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88b2a00640 1 --2- 172.21.15.47:0/1184893204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b4107ce0 0x7f88b419dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.741+0000 7f88b2a00640 1 -- 172.21.15.47:0/1184893204 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b41a5bf0 con 0x7f88b4108ed0 2024-03-21T11:21:20.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.742+0000 7f88b2a00640 1 --2- 172.21.15.47:0/1184893204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b4108ed0 0x7f88b419e510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f889c00eb90 tx=0x7f889c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:20.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.742+0000 7f88abe00640 1 -- 172.21.15.47:0/1184893204 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c004d20 con 0x7f88b4108ed0 2024-03-21T11:21:20.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.742+0000 7f88abe00640 1 -- 172.21.15.47:0/1184893204 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f889c00ee90 con 0x7f88b4108ed0 2024-03-21T11:21:20.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.742+0000 7f88ba000640 1 -- 172.21.15.47:0/1184893204 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b41a5ee0 con 0x7f88b4108ed0 2024-03-21T11:21:20.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.742+0000 7f88abe00640 1 -- 172.21.15.47:0/1184893204 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c010830 con 0x7f88b4108ed0 2024-03-21T11:21:20.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.742+0000 7f88ba000640 1 -- 172.21.15.47:0/1184893204 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b41a63a0 con 0x7f88b4108ed0 2024-03-21T11:21:20.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.743+0000 7f88ba000640 1 -- 172.21.15.47:0/1184893204 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b410ef30 con 0x7f88b4108ed0 2024-03-21T11:21:20.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.745+0000 7f88abe00640 1 -- 172.21.15.47:0/1184893204 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f889c005020 con 0x7f88b4108ed0 2024-03-21T11:21:20.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.746+0000 7f88abe00640 1 --2- 172.21.15.47:0/1184893204 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f88980a3030 0x7f88980a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:20.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.746+0000 7f88abe00640 1 -- 172.21.15.47:0/1184893204 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f889c0cfd30 con 0x7f88b4108ed0 2024-03-21T11:21:20.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.747+0000 7f88b3400640 1 --2- 172.21.15.47:0/1184893204 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f88980a3030 0x7f88980a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:20.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.747+0000 7f88b3400640 1 --2- 172.21.15.47:0/1184893204 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f88980a3030 0x7f88980a54f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f88a4005fd0 tx=0x7f88a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:20.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.753+0000 7f88abe00640 1 -- 172.21.15.47:0/1184893204 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f889c09c760 con 0x7f88b4108ed0 2024-03-21T11:21:20.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.961+0000 7f88ba000640 1 -- 172.21.15.47:0/1184893204 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88b402d240 con 0x7f88b4108ed0 2024-03-21T11:21:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.963+0000 7f88abe00640 1 -- 172.21.15.47:0/1184893204 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24127 (secure 0 0 0) 0x7f889c09c060 con 0x7f88b4108ed0 2024-03-21T11:21:20.964 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:21:20.964 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:20:54.191030+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:21:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.965+0000 7f88ba000640 1 -- 172.21.15.47:0/1184893204 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f88980a3030 msgr2=0x7f88980a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.965+0000 7f88ba000640 1 --2- 172.21.15.47:0/1184893204 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f88980a3030 0x7f88980a54f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f88a4005fd0 tx=0x7f88a400b410 comp rx=0 tx=0).stop 2024-03-21T11:21:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.965+0000 7f88ba000640 1 -- 172.21.15.47:0/1184893204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b4108ed0 msgr2=0x7f88b419e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.965+0000 7f88ba000640 1 --2- 172.21.15.47:0/1184893204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b4108ed0 0x7f88b419e510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f889c00eb90 tx=0x7f889c00c6a0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.965+0000 7f88ba000640 1 -- 172.21.15.47:0/1184893204 shutdown_connections 2024-03-21T11:21:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.965+0000 7f88ba000640 1 --2- 172.21.15.47:0/1184893204 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f88980a3030 0x7f88980a54f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.965+0000 7f88ba000640 1 --2- 172.21.15.47:0/1184893204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88b4109890 0x7f88b41a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.965+0000 7f88ba000640 1 --2- 172.21.15.47:0/1184893204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88b4108ed0 0x7f88b419e510 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.965+0000 7f88ba000640 1 --2- 172.21.15.47:0/1184893204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b4107ce0 0x7f88b419dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.965+0000 7f88ba000640 1 -- 172.21.15.47:0/1184893204 >> 172.21.15.47:0/1184893204 conn(0x7f88b4076cb0 msgr2=0x7f88b410c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.966+0000 7f88ba000640 1 -- 172.21.15.47:0/1184893204 shutdown_connections 2024-03-21T11:21:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:20.966+0000 7f88ba000640 1 -- 172.21.15.47:0/1184893204 wait complete. 2024-03-21T11:21:20.988 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-21T11:21:20.988 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T11:21:21.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:20 smithi100 ceph-mon[36245]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:21.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:20 smithi100 ceph-mon[36245]: pgmap v336: 129 pgs: 129 active+clean; 585 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 3.3 KiB/s wr, 9 op/s 2024-03-21T11:21:21.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:20 smithi100 ceph-mon[36245]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:20 smithi100 ceph-mon[36245]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-21T11:21:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:20 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1369373967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:21:21.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.215+0000 7fa557200640 1 -- 172.21.15.47:0/2893979292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5581071d0 msgr2=0x7fa558109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.215+0000 7fa557200640 1 --2- 172.21.15.47:0/2893979292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5581071d0 0x7fa558109620 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa550009900 tx=0x7fa55002bc40 comp rx=0 tx=0).stop 2024-03-21T11:21:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.216+0000 7fa557200640 1 -- 172.21.15.47:0/2893979292 shutdown_connections 2024-03-21T11:21:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.216+0000 7fa557200640 1 --2- 172.21.15.47:0/2893979292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5581071d0 0x7fa558109620 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.216+0000 7fa557200640 1 --2- 172.21.15.47:0/2893979292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5581048a0 0x7fa558106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.216+0000 7fa557200640 1 --2- 172.21.15.47:0/2893979292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa558101f70 0x7fa558104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.216+0000 7fa557200640 1 -- 172.21.15.47:0/2893979292 >> 172.21.15.47:0/2893979292 conn(0x7fa5580fb980 msgr2=0x7fa5580fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.216+0000 7fa557200640 1 -- 172.21.15.47:0/2893979292 shutdown_connections 2024-03-21T11:21:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.216+0000 7fa557200640 1 -- 172.21.15.47:0/2893979292 wait complete. 2024-03-21T11:21:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.217+0000 7fa557200640 1 Processor -- start 2024-03-21T11:21:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.217+0000 7fa557200640 1 -- start start 2024-03-21T11:21:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.217+0000 7fa557200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa558101f70 0x7fa5581018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.217+0000 7fa557200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5581048a0 0x7fa5580fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.217+0000 7fa557200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5581071d0 0x7fa558100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.218+0000 7fa557200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55810d1d0 con 0x7fa5581048a0 2024-03-21T11:21:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.218+0000 7fa557200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55810d050 con 0x7fa5581071d0 2024-03-21T11:21:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.218+0000 7fa557200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55810d350 con 0x7fa558101f70 2024-03-21T11:21:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.218+0000 7fa555e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa558101f70 0x7fa5581018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.218+0000 7fa555400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5581048a0 0x7fa5580fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.218+0000 7fa555e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa558101f70 0x7fa5581018e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33022/0 (socket says 172.21.15.47:33022) 2024-03-21T11:21:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.218+0000 7fa555e00640 1 -- 172.21.15.47:0/3820243776 learned_addr learned my addr 172.21.15.47:0/3820243776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:21.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.218+0000 7fa556800640 1 --2- 172.21.15.47:0/3820243776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5581071d0 0x7fa558100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:21.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.218+0000 7fa555e00640 1 -- 172.21.15.47:0/3820243776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5581071d0 msgr2=0x7fa558100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:21.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.218+0000 7fa555e00640 1 --2- 172.21.15.47:0/3820243776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5581071d0 0x7fa558100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.218+0000 7fa555e00640 1 -- 172.21.15.47:0/3820243776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5581048a0 msgr2=0x7fa5580fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:21.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.218+0000 7fa555e00640 1 --2- 172.21.15.47:0/3820243776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5581048a0 0x7fa5580fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.218+0000 7fa555e00640 1 -- 172.21.15.47:0/3820243776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa558100cf0 con 0x7fa558101f70 2024-03-21T11:21:21.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.219+0000 7fa555e00640 1 --2- 172.21.15.47:0/3820243776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa558101f70 0x7fa5581018e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa54000e9f0 tx=0x7fa54000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:21.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.219+0000 7fa54ea00640 1 -- 172.21.15.47:0/3820243776 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54000cd80 con 0x7fa558101f70 2024-03-21T11:21:21.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.219+0000 7fa54ea00640 1 -- 172.21.15.47:0/3820243776 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa540005500 con 0x7fa558101f70 2024-03-21T11:21:21.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.219+0000 7fa54ea00640 1 -- 172.21.15.47:0/3820243776 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540010690 con 0x7fa558101f70 2024-03-21T11:21:21.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.219+0000 7fa557200640 1 -- 172.21.15.47:0/3820243776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5581a18d0 con 0x7fa558101f70 2024-03-21T11:21:21.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.219+0000 7fa557200640 1 -- 172.21.15.47:0/3820243776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5581a1e20 con 0x7fa558101f70 2024-03-21T11:21:21.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.220+0000 7fa557200640 1 -- 172.21.15.47:0/3820243776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5580fd470 con 0x7fa558101f70 2024-03-21T11:21:21.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.222+0000 7fa54ea00640 1 -- 172.21.15.47:0/3820243776 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fa5400046c0 con 0x7fa558101f70 2024-03-21T11:21:21.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.223+0000 7fa54ea00640 1 --2- 172.21.15.47:0/3820243776 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa53c0a3030 0x7fa53c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:21.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.223+0000 7fa54ea00640 1 -- 172.21.15.47:0/3820243776 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa5400cfc00 con 0x7fa558101f70 2024-03-21T11:21:21.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.224+0000 7fa555400640 1 --2- 172.21.15.47:0/3820243776 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa53c0a3030 0x7fa53c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:21.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.224+0000 7fa555400640 1 --2- 172.21.15.47:0/3820243776 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa53c0a3030 0x7fa53c0a54f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa558101650 tx=0x7fa548009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:21.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.231+0000 7fa54ea00640 1 -- 172.21.15.47:0/3820243776 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa54009c630 con 0x7fa558101f70 2024-03-21T11:21:21.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:20 smithi102 ceph-mon[36748]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:21.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:20 smithi102 ceph-mon[36748]: pgmap v336: 129 pgs: 129 active+clean; 585 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 3.3 KiB/s wr, 9 op/s 2024-03-21T11:21:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:20 smithi102 ceph-mon[36748]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:20 smithi102 ceph-mon[36748]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-21T11:21:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:20 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1369373967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:21:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:20 smithi047 ceph-mon[31351]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:20 smithi047 ceph-mon[31351]: pgmap v336: 129 pgs: 129 active+clean; 585 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 3.3 KiB/s wr, 9 op/s 2024-03-21T11:21:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:20 smithi047 ceph-mon[31351]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:20 smithi047 ceph-mon[31351]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-21T11:21:21.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:20 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1369373967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:21:21.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.430+0000 7fa557200640 1 -- 172.21.15.47:0/3820243776 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa558061ae0 con 0x7fa53c0a3030 2024-03-21T11:21:21.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.467+0000 7fa54ea00640 1 -- 172.21.15.47:0/3820243776 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340657 (secure 0 0 0) 0x7fa558061ae0 con 0x7fa53c0a3030 2024-03-21T11:21:21.470 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:21:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-21T11:21:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.474+0000 7fa557200640 1 -- 172.21.15.47:0/3820243776 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa53c0a3030 msgr2=0x7fa53c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.474+0000 7fa557200640 1 --2- 172.21.15.47:0/3820243776 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa53c0a3030 0x7fa53c0a54f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa558101650 tx=0x7fa548009500 comp rx=0 tx=0).stop 2024-03-21T11:21:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.474+0000 7fa557200640 1 -- 172.21.15.47:0/3820243776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa558101f70 msgr2=0x7fa5581018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.474+0000 7fa557200640 1 --2- 172.21.15.47:0/3820243776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa558101f70 0x7fa5581018e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa54000e9f0 tx=0x7fa54000edf0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.474+0000 7fa557200640 1 -- 172.21.15.47:0/3820243776 shutdown_connections 2024-03-21T11:21:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.474+0000 7fa557200640 1 --2- 172.21.15.47:0/3820243776 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa53c0a3030 0x7fa53c0a54f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.474+0000 7fa557200640 1 --2- 172.21.15.47:0/3820243776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5581071d0 0x7fa558100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.474+0000 7fa557200640 1 --2- 172.21.15.47:0/3820243776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5581048a0 0x7fa5580fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.474+0000 7fa557200640 1 --2- 172.21.15.47:0/3820243776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa558101f70 0x7fa5581018e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.474+0000 7fa557200640 1 -- 172.21.15.47:0/3820243776 >> 172.21.15.47:0/3820243776 conn(0x7fa5580fb980 msgr2=0x7fa5581028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.474+0000 7fa557200640 1 -- 172.21.15.47:0/3820243776 shutdown_connections 2024-03-21T11:21:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.474+0000 7fa557200640 1 -- 172.21.15.47:0/3820243776 wait complete. 2024-03-21T11:21:21.499 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-03-21T11:21:20.992389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":637681,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":146,"num_read_kb":164,"num_write":363,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":297,"ondisk_log_size":297,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373680,"kb_used_data":12048,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568144,"statfs":{"total":1151940427776,"available":1151557779456,"internally_reserved":0,"allocated":12337152,"data_stored":7748025,"data_compressed":289357,"data_compressed_allocated":389120,"data_compressed_original":2277376,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":44454,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":20,"num_read_kb":55,"num_write":136,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005128"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042323+0000","last_change":"2024-03-21T11:20:48.667185+0000","last_active":"2024-03-21T11:20:55.042323+0000","last_peered":"2024-03-21T11:20:55.042323+0000","last_clean":"2024-03-21T11:20:55.042323+0000","last_became_active":"2024-03-21T11:20:48.666855+0000","last_became_peered":"2024-03-21T11:20:48.666855+0000","last_unstale":"2024-03-21T11:20:55.042323+0000","last_undegraded":"2024-03-21T11:20:55.042323+0000","last_fullsized":"2024-03-21T11:20:55.042323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:30:37.393096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.722850+0000","last_change":"2024-03-21T11:20:48.671613+0000","last_active":"2024-03-21T11:20:55.722850+0000","last_peered":"2024-03-21T11:20:55.722850+0000","last_clean":"2024-03-21T11:20:55.722850+0000","last_became_active":"2024-03-21T11:20:48.671211+0000","last_became_peered":"2024-03-21T11:20:48.671211+0000","last_unstale":"2024-03-21T11:20:55.722850+0000","last_undegraded":"2024-03-21T11:20:55.722850+0000","last_fullsized":"2024-03-21T11:20:55.722850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:20.167922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721820+0000","last_change":"2024-03-21T11:20:48.655734+0000","last_active":"2024-03-21T11:20:55.721820+0000","last_peered":"2024-03-21T11:20:55.721820+0000","last_clean":"2024-03-21T11:20:55.721820+0000","last_became_active":"2024-03-21T11:20:48.653798+0000","last_became_peered":"2024-03-21T11:20:48.653798+0000","last_unstale":"2024-03-21T11:20:55.721820+0000","last_undegraded":"2024-03-21T11:20:55.721820+0000","last_fullsized":"2024-03-21T11:20:55.721820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:44.796405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734826+0000","last_change":"2024-03-21T11:20:48.648784+0000","last_active":"2024-03-21T11:20:54.734826+0000","last_peered":"2024-03-21T11:20:54.734826+0000","last_clean":"2024-03-21T11:20:54.734826+0000","last_became_active":"2024-03-21T11:20:48.648040+0000","last_became_peered":"2024-03-21T11:20:48.648040+0000","last_unstale":"2024-03-21T11:20:54.734826+0000","last_undegraded":"2024-03-21T11:20:54.734826+0000","last_fullsized":"2024-03-21T11:20:54.734826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:48:51.786346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721353+0000","last_change":"2024-03-21T11:20:48.672084+0000","last_active":"2024-03-21T11:20:55.721353+0000","last_peered":"2024-03-21T11:20:55.721353+0000","last_clean":"2024-03-21T11:20:55.721353+0000","last_became_active":"2024-03-21T11:20:48.671441+0000","last_became_peered":"2024-03-21T11:20:48.671441+0000","last_unstale":"2024-03-21T11:20:55.721353+0000","last_undegraded":"2024-03-21T11:20:55.721353+0000","last_fullsized":"2024-03-21T11:20:55.721353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:20:17.638270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.639858+0000","last_change":"2024-03-21T11:20:48.659782+0000","last_active":"2024-03-21T11:20:55.639858+0000","last_peered":"2024-03-21T11:20:55.639858+0000","last_clean":"2024-03-21T11:20:55.639858+0000","last_became_active":"2024-03-21T11:20:48.659460+0000","last_became_peered":"2024-03-21T11:20:48.659460+0000","last_unstale":"2024-03-21T11:20:55.639858+0000","last_undegraded":"2024-03-21T11:20:55.639858+0000","last_fullsized":"2024-03-21T11:20:55.639858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:57.712654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721315+0000","last_change":"2024-03-21T11:20:48.659401+0000","last_active":"2024-03-21T11:20:55.721315+0000","last_peered":"2024-03-21T11:20:55.721315+0000","last_clean":"2024-03-21T11:20:55.721315+0000","last_became_active":"2024-03-21T11:20:48.659075+0000","last_became_peered":"2024-03-21T11:20:48.659075+0000","last_unstale":"2024-03-21T11:20:55.721315+0000","last_undegraded":"2024-03-21T11:20:55.721315+0000","last_fullsized":"2024-03-21T11:20:55.721315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:54.342236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.570267+0000","last_change":"2024-03-21T11:20:48.667175+0000","last_active":"2024-03-21T11:20:55.570267+0000","last_peered":"2024-03-21T11:20:55.570267+0000","last_clean":"2024-03-21T11:20:55.570267+0000","last_became_active":"2024-03-21T11:20:48.666756+0000","last_became_peered":"2024-03-21T11:20:48.666756+0000","last_unstale":"2024-03-21T11:20:55.570267+0000","last_undegraded":"2024-03-21T11:20:55.570267+0000","last_fullsized":"2024-03-21T11:20:55.570267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:25:39.694699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.720882+0000","last_change":"2024-03-21T11:20:48.650584+0000","last_active":"2024-03-21T11:20:55.720882+0000","last_peered":"2024-03-21T11:20:55.720882+0000","last_clean":"2024-03-21T11:20:55.720882+0000","last_became_active":"2024-03-21T11:20:48.650251+0000","last_became_peered":"2024-03-21T11:20:48.650251+0000","last_unstale":"2024-03-21T11:20:55.720882+0000","last_undegraded":"2024-03-21T11:20:55.720882+0000","last_fullsized":"2024-03-21T11:20:55.720882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:35:59.000522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.640107+0000","last_change":"2024-03-21T11:20:48.652945+0000","last_active":"2024-03-21T11:20:55.640107+0000","last_peered":"2024-03-21T11:20:55.640107+0000","last_clean":"2024-03-21T11:20:55.640107+0000","last_became_active":"2024-03-21T11:20:48.652626+0000","last_became_peered":"2024-03-21T11:20:48.652626+0000","last_unstale":"2024-03-21T11:20:55.640107+0000","last_undegraded":"2024-03-21T11:20:55.640107+0000","last_fullsized":"2024-03-21T11:20:55.640107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:43.586907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721709+0000","last_change":"2024-03-21T11:20:48.651520+0000","last_active":"2024-03-21T11:20:55.721709+0000","last_peered":"2024-03-21T11:20:55.721709+0000","last_clean":"2024-03-21T11:20:55.721709+0000","last_became_active":"2024-03-21T11:20:48.651210+0000","last_became_peered":"2024-03-21T11:20:48.651210+0000","last_unstale":"2024-03-21T11:20:55.721709+0000","last_undegraded":"2024-03-21T11:20:55.721709+0000","last_fullsized":"2024-03-21T11:20:55.721709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:29.506732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734732+0000","last_change":"2024-03-21T11:20:48.649844+0000","last_active":"2024-03-21T11:20:54.734732+0000","last_peered":"2024-03-21T11:20:54.734732+0000","last_clean":"2024-03-21T11:20:54.734732+0000","last_became_active":"2024-03-21T11:20:48.649503+0000","last_became_peered":"2024-03-21T11:20:48.649503+0000","last_unstale":"2024-03-21T11:20:54.734732+0000","last_undegraded":"2024-03-21T11:20:54.734732+0000","last_fullsized":"2024-03-21T11:20:54.734732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:31:55.734583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.832602+0000","last_change":"2024-03-21T11:20:48.662012+0000","last_active":"2024-03-21T11:20:55.832602+0000","last_peered":"2024-03-21T11:20:55.832602+0000","last_clean":"2024-03-21T11:20:55.832602+0000","last_became_active":"2024-03-21T11:20:48.661444+0000","last_became_peered":"2024-03-21T11:20:48.661444+0000","last_unstale":"2024-03-21T11:20:55.832602+0000","last_undegraded":"2024-03-21T11:20:55.832602+0000","last_fullsized":"2024-03-21T11:20:55.832602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:52.493244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721684+0000","last_change":"2024-03-21T11:20:48.663087+0000","last_active":"2024-03-21T11:20:55.721684+0000","last_peered":"2024-03-21T11:20:55.721684+0000","last_clean":"2024-03-21T11:20:55.721684+0000","last_became_active":"2024-03-21T11:20:48.662039+0000","last_became_peered":"2024-03-21T11:20:48.662039+0000","last_unstale":"2024-03-21T11:20:55.721684+0000","last_undegraded":"2024-03-21T11:20:55.721684+0000","last_fullsized":"2024-03-21T11:20:55.721684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:47:01.962314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721686+0000","last_change":"2024-03-21T11:20:48.663478+0000","last_active":"2024-03-21T11:20:55.721686+0000","last_peered":"2024-03-21T11:20:55.721686+0000","last_clean":"2024-03-21T11:20:55.721686+0000","last_became_active":"2024-03-21T11:20:48.663135+0000","last_became_peered":"2024-03-21T11:20:48.663135+0000","last_unstale":"2024-03-21T11:20:55.721686+0000","last_undegraded":"2024-03-21T11:20:55.721686+0000","last_fullsized":"2024-03-21T11:20:55.721686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:29.139760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.720174+0000","last_change":"2024-03-21T11:20:48.653893+0000","last_active":"2024-03-21T11:20:55.720174+0000","last_peered":"2024-03-21T11:20:55.720174+0000","last_clean":"2024-03-21T11:20:55.720174+0000","last_became_active":"2024-03-21T11:20:48.653578+0000","last_became_peered":"2024-03-21T11:20:48.653578+0000","last_unstale":"2024-03-21T11:20:55.720174+0000","last_undegraded":"2024-03-21T11:20:55.720174+0000","last_fullsized":"2024-03-21T11:20:55.720174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:54:31.254030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042048+0000","last_change":"2024-03-21T11:20:48.667388+0000","last_active":"2024-03-21T11:20:55.042048+0000","last_peered":"2024-03-21T11:20:55.042048+0000","last_clean":"2024-03-21T11:20:55.042048+0000","last_became_active":"2024-03-21T11:20:48.667077+0000","last_became_peered":"2024-03-21T11:20:48.667077+0000","last_unstale":"2024-03-21T11:20:55.042048+0000","last_undegraded":"2024-03-21T11:20:55.042048+0000","last_fullsized":"2024-03-21T11:20:55.042048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:13:32.093457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.739566+0000","last_change":"2024-03-21T11:20:48.658479+0000","last_active":"2024-03-21T11:20:54.739566+0000","last_peered":"2024-03-21T11:20:54.739566+0000","last_clean":"2024-03-21T11:20:54.739566+0000","last_became_active":"2024-03-21T11:20:48.658011+0000","last_became_peered":"2024-03-21T11:20:48.658011+0000","last_unstale":"2024-03-21T11:20:54.739566+0000","last_undegraded":"2024-03-21T11:20:54.739566+0000","last_fullsized":"2024-03-21T11:20:54.739566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:33.313537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734846+0000","last_change":"2024-03-21T11:20:48.668648+0000","last_active":"2024-03-21T11:20:54.734846+0000","last_peered":"2024-03-21T11:20:54.734846+0000","last_clean":"2024-03-21T11:20:54.734846+0000","last_became_active":"2024-03-21T11:20:48.668376+0000","last_became_peered":"2024-03-21T11:20:48.668376+0000","last_unstale":"2024-03-21T11:20:54.734846+0000","last_undegraded":"2024-03-21T11:20:54.734846+0000","last_fullsized":"2024-03-21T11:20:54.734846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:49:47.601724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734412+0000","last_change":"2024-03-21T11:20:53.755600+0000","last_active":"2024-03-21T11:20:54.734412+0000","last_peered":"2024-03-21T11:20:54.734412+0000","last_clean":"2024-03-21T11:20:54.734412+0000","last_became_active":"2024-03-21T11:20:53.755321+0000","last_became_peered":"2024-03-21T11:20:53.755321+0000","last_unstale":"2024-03-21T11:20:54.734412+0000","last_undegraded":"2024-03-21T11:20:54.734412+0000","last_fullsized":"2024-03-21T11:20:54.734412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:04.905910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:16.876749+0000","last_change":"2024-03-21T11:20:48.161001+0000","last_active":"2024-03-21T11:21:16.876749+0000","last_peered":"2024-03-21T11:21:16.876749+0000","last_clean":"2024-03-21T11:21:16.876749+0000","last_became_active":"2024-03-21T11:20:48.153053+0000","last_became_peered":"2024-03-21T11:20:48.153053+0000","last_unstale":"2024-03-21T11:21:16.876749+0000","last_undegraded":"2024-03-21T11:21:16.876749+0000","last_fullsized":"2024-03-21T11:21:16.876749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:26.010581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.571618+0000","last_change":"2024-03-21T11:20:48.671430+0000","last_active":"2024-03-21T11:20:55.571618+0000","last_peered":"2024-03-21T11:20:55.571618+0000","last_clean":"2024-03-21T11:20:55.571618+0000","last_became_active":"2024-03-21T11:20:48.670192+0000","last_became_peered":"2024-03-21T11:20:48.670192+0000","last_unstale":"2024-03-21T11:20:55.571618+0000","last_undegraded":"2024-03-21T11:20:55.571618+0000","last_fullsized":"2024-03-21T11:20:55.571618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:00:53.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734304+0000","last_change":"2024-03-21T11:20:53.750313+0000","last_active":"2024-03-21T11:20:54.734304+0000","last_peered":"2024-03-21T11:20:54.734304+0000","last_clean":"2024-03-21T11:20:54.734304+0000","last_became_active":"2024-03-21T11:20:53.749913+0000","last_became_peered":"2024-03-21T11:20:53.749913+0000","last_unstale":"2024-03-21T11:20:54.734304+0000","last_undegraded":"2024-03-21T11:20:54.734304+0000","last_fullsized":"2024-03-21T11:20:54.734304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:39:09.007674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.572099+0000","last_change":"2024-03-21T11:20:48.159934+0000","last_active":"2024-03-21T11:20:55.572099+0000","last_peered":"2024-03-21T11:20:55.572099+0000","last_clean":"2024-03-21T11:20:55.572099+0000","last_became_active":"2024-03-21T11:20:48.159650+0000","last_became_peered":"2024-03-21T11:20:48.159650+0000","last_unstale":"2024-03-21T11:20:55.572099+0000","last_undegraded":"2024-03-21T11:20:55.572099+0000","last_fullsized":"2024-03-21T11:20:55.572099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:49:56.891733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.722279+0000","last_change":"2024-03-21T11:20:48.673012+0000","last_active":"2024-03-21T11:20:55.722279+0000","last_peered":"2024-03-21T11:20:55.722279+0000","last_clean":"2024-03-21T11:20:55.722279+0000","last_became_active":"2024-03-21T11:20:48.672584+0000","last_became_peered":"2024-03-21T11:20:48.672584+0000","last_unstale":"2024-03-21T11:20:55.722279+0000","last_undegraded":"2024-03-21T11:20:55.722279+0000","last_fullsized":"2024-03-21T11:20:55.722279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:48.213569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734302+0000","last_change":"2024-03-21T11:20:53.750641+0000","last_active":"2024-03-21T11:20:54.734302+0000","last_peered":"2024-03-21T11:20:54.734302+0000","last_clean":"2024-03-21T11:20:54.734302+0000","last_became_active":"2024-03-21T11:20:53.750258+0000","last_became_peered":"2024-03-21T11:20:53.750258+0000","last_unstale":"2024-03-21T11:20:54.734302+0000","last_undegraded":"2024-03-21T11:20:54.734302+0000","last_fullsized":"2024-03-21T11:20:54.734302+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:58:17.348835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:16.868273+0000","last_change":"2024-03-21T11:20:48.134383+0000","last_active":"2024-03-21T11:21:16.868273+0000","last_peered":"2024-03-21T11:21:16.868273+0000","last_clean":"2024-03-21T11:21:16.868273+0000","last_became_active":"2024-03-21T11:20:48.134012+0000","last_became_peered":"2024-03-21T11:20:48.134012+0000","last_unstale":"2024-03-21T11:21:16.868273+0000","last_undegraded":"2024-03-21T11:21:16.868273+0000","last_fullsized":"2024-03-21T11:21:16.868273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:24:47.460145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.720548+0000","last_change":"2024-03-21T11:20:48.662392+0000","last_active":"2024-03-21T11:20:55.720548+0000","last_peered":"2024-03-21T11:20:55.720548+0000","last_clean":"2024-03-21T11:20:55.720548+0000","last_became_active":"2024-03-21T11:20:48.662042+0000","last_became_peered":"2024-03-21T11:20:48.662042+0000","last_unstale":"2024-03-21T11:20:55.720548+0000","last_undegraded":"2024-03-21T11:20:55.720548+0000","last_fullsized":"2024-03-21T11:20:55.720548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:59:34.456582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734256+0000","last_change":"2024-03-21T11:20:53.749693+0000","last_active":"2024-03-21T11:20:54.734256+0000","last_peered":"2024-03-21T11:20:54.734256+0000","last_clean":"2024-03-21T11:20:54.734256+0000","last_became_active":"2024-03-21T11:20:53.749438+0000","last_became_peered":"2024-03-21T11:20:53.749438+0000","last_unstale":"2024-03-21T11:20:54.734256+0000","last_undegraded":"2024-03-21T11:20:54.734256+0000","last_fullsized":"2024-03-21T11:20:54.734256+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:37.714471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721742+0000","last_change":"2024-03-21T11:20:48.132587+0000","last_active":"2024-03-21T11:20:55.721742+0000","last_peered":"2024-03-21T11:20:55.721742+0000","last_clean":"2024-03-21T11:20:55.721742+0000","last_became_active":"2024-03-21T11:20:48.132232+0000","last_became_peered":"2024-03-21T11:20:48.132232+0000","last_unstale":"2024-03-21T11:20:55.721742+0000","last_undegraded":"2024-03-21T11:20:55.721742+0000","last_fullsized":"2024-03-21T11:20:55.721742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:20:45.217545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330584+0000","last_change":"2024-03-21T11:20:48.656881+0000","last_active":"2024-03-21T11:20:55.330584+0000","last_peered":"2024-03-21T11:20:55.330584+0000","last_clean":"2024-03-21T11:20:55.330584+0000","last_became_active":"2024-03-21T11:20:48.656426+0000","last_became_peered":"2024-03-21T11:20:48.656426+0000","last_unstale":"2024-03-21T11:20:55.330584+0000","last_undegraded":"2024-03-21T11:20:55.330584+0000","last_fullsized":"2024-03-21T11:20:55.330584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:06:27.398062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734697+0000","last_change":"2024-03-21T11:20:53.733396+0000","last_active":"2024-03-21T11:20:54.734697+0000","last_peered":"2024-03-21T11:20:54.734697+0000","last_clean":"2024-03-21T11:20:54.734697+0000","last_became_active":"2024-03-21T11:20:53.732985+0000","last_became_peered":"2024-03-21T11:20:53.732985+0000","last_unstale":"2024-03-21T11:20:54.734697+0000","last_undegraded":"2024-03-21T11:20:54.734697+0000","last_fullsized":"2024-03-21T11:20:54.734697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:54:14.064761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.837912+0000","last_change":"2024-03-21T11:20:48.133362+0000","last_active":"2024-03-21T11:21:14.837912+0000","last_peered":"2024-03-21T11:21:14.837912+0000","last_clean":"2024-03-21T11:21:14.837912+0000","last_became_active":"2024-03-21T11:20:48.132405+0000","last_became_peered":"2024-03-21T11:20:48.132405+0000","last_unstale":"2024-03-21T11:21:14.837912+0000","last_undegraded":"2024-03-21T11:21:14.837912+0000","last_fullsized":"2024-03-21T11:21:14.837912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:48:52.688948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721405+0000","last_change":"2024-03-21T11:20:48.659610+0000","last_active":"2024-03-21T11:20:55.721405+0000","last_peered":"2024-03-21T11:20:55.721405+0000","last_clean":"2024-03-21T11:20:55.721405+0000","last_became_active":"2024-03-21T11:20:48.659257+0000","last_became_peered":"2024-03-21T11:20:48.659257+0000","last_unstale":"2024-03-21T11:20:55.721405+0000","last_undegraded":"2024-03-21T11:20:55.721405+0000","last_fullsized":"2024-03-21T11:20:55.721405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:17:30.744434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734497+0000","last_change":"2024-03-21T11:20:53.750014+0000","last_active":"2024-03-21T11:20:54.734497+0000","last_peered":"2024-03-21T11:20:54.734497+0000","last_clean":"2024-03-21T11:20:54.734497+0000","last_became_active":"2024-03-21T11:20:53.749572+0000","last_became_peered":"2024-03-21T11:20:53.749572+0000","last_unstale":"2024-03-21T11:20:54.734497+0000","last_undegraded":"2024-03-21T11:20:54.734497+0000","last_fullsized":"2024-03-21T11:20:54.734497+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:29.954384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:16.874890+0000","last_change":"2024-03-21T11:20:48.142729+0000","last_active":"2024-03-21T11:21:16.874890+0000","last_peered":"2024-03-21T11:21:16.874890+0000","last_clean":"2024-03-21T11:21:16.874890+0000","last_became_active":"2024-03-21T11:20:48.142509+0000","last_became_peered":"2024-03-21T11:20:48.142509+0000","last_unstale":"2024-03-21T11:21:16.874890+0000","last_undegraded":"2024-03-21T11:21:16.874890+0000","last_fullsized":"2024-03-21T11:21:16.874890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:49:28.681159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330466+0000","last_change":"2024-03-21T11:20:48.671221+0000","last_active":"2024-03-21T11:20:55.330466+0000","last_peered":"2024-03-21T11:20:55.330466+0000","last_clean":"2024-03-21T11:20:55.330466+0000","last_became_active":"2024-03-21T11:20:48.670837+0000","last_became_peered":"2024-03-21T11:20:48.670837+0000","last_unstale":"2024-03-21T11:20:55.330466+0000","last_undegraded":"2024-03-21T11:20:55.330466+0000","last_fullsized":"2024-03-21T11:20:55.330466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:38.680409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042277+0000","last_change":"2024-03-21T11:20:53.747501+0000","last_active":"2024-03-21T11:20:55.042277+0000","last_peered":"2024-03-21T11:20:55.042277+0000","last_clean":"2024-03-21T11:20:55.042277+0000","last_became_active":"2024-03-21T11:20:53.747062+0000","last_became_peered":"2024-03-21T11:20:53.747062+0000","last_unstale":"2024-03-21T11:20:55.042277+0000","last_undegraded":"2024-03-21T11:20:55.042277+0000","last_fullsized":"2024-03-21T11:20:55.042277+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:58:36.904221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:15.854302+0000","last_change":"2024-03-21T11:20:48.156175+0000","last_active":"2024-03-21T11:21:15.854302+0000","last_peered":"2024-03-21T11:21:15.854302+0000","last_clean":"2024-03-21T11:21:15.854302+0000","last_became_active":"2024-03-21T11:20:48.154271+0000","last_became_peered":"2024-03-21T11:20:48.154271+0000","last_unstale":"2024-03-21T11:21:15.854302+0000","last_undegraded":"2024-03-21T11:21:15.854302+0000","last_fullsized":"2024-03-21T11:21:15.854302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:04.216391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.570565+0000","last_change":"2024-03-21T11:20:48.670011+0000","last_active":"2024-03-21T11:20:55.570565+0000","last_peered":"2024-03-21T11:20:55.570565+0000","last_clean":"2024-03-21T11:20:55.570565+0000","last_became_active":"2024-03-21T11:20:48.669016+0000","last_became_peered":"2024-03-21T11:20:48.669016+0000","last_unstale":"2024-03-21T11:20:55.570565+0000","last_undegraded":"2024-03-21T11:20:55.570565+0000","last_fullsized":"2024-03-21T11:20:55.570565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:43:18.906490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.723049+0000","last_change":"2024-03-21T11:20:53.751934+0000","last_active":"2024-03-21T11:20:55.723049+0000","last_peered":"2024-03-21T11:20:55.723049+0000","last_clean":"2024-03-21T11:20:55.723049+0000","last_became_active":"2024-03-21T11:20:53.751413+0000","last_became_peered":"2024-03-21T11:20:53.751413+0000","last_unstale":"2024-03-21T11:20:55.723049+0000","last_undegraded":"2024-03-21T11:20:55.723049+0000","last_fullsized":"2024-03-21T11:20:55.723049+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:12:33.880376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.837188+0000","last_change":"2024-03-21T11:20:48.131796+0000","last_active":"2024-03-21T11:21:14.837188+0000","last_peered":"2024-03-21T11:21:14.837188+0000","last_clean":"2024-03-21T11:21:14.837188+0000","last_became_active":"2024-03-21T11:20:48.131478+0000","last_became_peered":"2024-03-21T11:20:48.131478+0000","last_unstale":"2024-03-21T11:21:14.837188+0000","last_undegraded":"2024-03-21T11:21:14.837188+0000","last_fullsized":"2024-03-21T11:21:14.837188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:59:24.871300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.737515+0000","last_change":"2024-03-21T11:20:48.660046+0000","last_active":"2024-03-21T11:20:54.737515+0000","last_peered":"2024-03-21T11:20:54.737515+0000","last_clean":"2024-03-21T11:20:54.737515+0000","last_became_active":"2024-03-21T11:20:48.659621+0000","last_became_peered":"2024-03-21T11:20:48.659621+0000","last_unstale":"2024-03-21T11:20:54.737515+0000","last_undegraded":"2024-03-21T11:20:54.737515+0000","last_fullsized":"2024-03-21T11:20:54.737515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:47:01.923285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.570862+0000","last_change":"2024-03-21T11:20:53.754458+0000","last_active":"2024-03-21T11:20:55.570862+0000","last_peered":"2024-03-21T11:20:55.570862+0000","last_clean":"2024-03-21T11:20:55.570862+0000","last_became_active":"2024-03-21T11:20:53.753927+0000","last_became_peered":"2024-03-21T11:20:53.753927+0000","last_unstale":"2024-03-21T11:20:55.570862+0000","last_undegraded":"2024-03-21T11:20:55.570862+0000","last_fullsized":"2024-03-21T11:20:55.570862+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:00:47.392699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:20.607041+0000","last_change":"2024-03-21T11:20:48.154125+0000","last_active":"2024-03-21T11:21:20.607041+0000","last_peered":"2024-03-21T11:21:20.607041+0000","last_clean":"2024-03-21T11:21:20.607041+0000","last_became_active":"2024-03-21T11:20:48.153857+0000","last_became_peered":"2024-03-21T11:20:48.153857+0000","last_unstale":"2024-03-21T11:21:20.607041+0000","last_undegraded":"2024-03-21T11:21:20.607041+0000","last_fullsized":"2024-03-21T11:21:20.607041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:54:06.525335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.720901+0000","last_change":"2024-03-21T11:20:48.650143+0000","last_active":"2024-03-21T11:20:55.720901+0000","last_peered":"2024-03-21T11:20:55.720901+0000","last_clean":"2024-03-21T11:20:55.720901+0000","last_became_active":"2024-03-21T11:20:48.649825+0000","last_became_peered":"2024-03-21T11:20:48.649825+0000","last_unstale":"2024-03-21T11:20:55.720901+0000","last_undegraded":"2024-03-21T11:20:55.720901+0000","last_fullsized":"2024-03-21T11:20:55.720901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:09:15.735280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.737610+0000","last_change":"2024-03-21T11:20:53.743006+0000","last_active":"2024-03-21T11:20:54.737610+0000","last_peered":"2024-03-21T11:20:54.737610+0000","last_clean":"2024-03-21T11:20:54.737610+0000","last_became_active":"2024-03-21T11:20:53.742587+0000","last_became_peered":"2024-03-21T11:20:53.742587+0000","last_unstale":"2024-03-21T11:20:54.737610+0000","last_undegraded":"2024-03-21T11:20:54.737610+0000","last_fullsized":"2024-03-21T11:20:54.737610+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:06:06.845791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.835888+0000","last_change":"2024-03-21T11:20:48.137137+0000","last_active":"2024-03-21T11:21:14.835888+0000","last_peered":"2024-03-21T11:21:14.835888+0000","last_clean":"2024-03-21T11:21:14.835888+0000","last_became_active":"2024-03-21T11:20:48.136790+0000","last_became_peered":"2024-03-21T11:20:48.136790+0000","last_unstale":"2024-03-21T11:21:14.835888+0000","last_undegraded":"2024-03-21T11:21:14.835888+0000","last_fullsized":"2024-03-21T11:21:14.835888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:22:47.223969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.722441+0000","last_change":"2024-03-21T11:20:48.673568+0000","last_active":"2024-03-21T11:20:55.722441+0000","last_peered":"2024-03-21T11:20:55.722441+0000","last_clean":"2024-03-21T11:20:55.722441+0000","last_became_active":"2024-03-21T11:20:48.672613+0000","last_became_peered":"2024-03-21T11:20:48.672613+0000","last_unstale":"2024-03-21T11:20:55.722441+0000","last_undegraded":"2024-03-21T11:20:55.722441+0000","last_fullsized":"2024-03-21T11:20:55.722441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:43:38.193802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734470+0000","last_change":"2024-03-21T11:20:53.739228+0000","last_active":"2024-03-21T11:20:54.734470+0000","last_peered":"2024-03-21T11:20:54.734470+0000","last_clean":"2024-03-21T11:20:54.734470+0000","last_became_active":"2024-03-21T11:20:53.738750+0000","last_became_peered":"2024-03-21T11:20:53.738750+0000","last_unstale":"2024-03-21T11:20:54.734470+0000","last_undegraded":"2024-03-21T11:20:54.734470+0000","last_fullsized":"2024-03-21T11:20:54.734470+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:58:33.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.840277+0000","last_change":"2024-03-21T11:20:48.144451+0000","last_active":"2024-03-21T11:21:14.840277+0000","last_peered":"2024-03-21T11:21:14.840277+0000","last_clean":"2024-03-21T11:21:14.840277+0000","last_became_active":"2024-03-21T11:20:48.144074+0000","last_became_peered":"2024-03-21T11:20:48.144074+0000","last_unstale":"2024-03-21T11:21:14.840277+0000","last_undegraded":"2024-03-21T11:21:14.840277+0000","last_fullsized":"2024-03-21T11:21:14.840277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:33:17.032461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.720684+0000","last_change":"2024-03-21T11:20:48.654472+0000","last_active":"2024-03-21T11:20:55.720684+0000","last_peered":"2024-03-21T11:20:55.720684+0000","last_clean":"2024-03-21T11:20:55.720684+0000","last_became_active":"2024-03-21T11:20:48.654129+0000","last_became_peered":"2024-03-21T11:20:48.654129+0000","last_unstale":"2024-03-21T11:20:55.720684+0000","last_undegraded":"2024-03-21T11:20:55.720684+0000","last_fullsized":"2024-03-21T11:20:55.720684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:28:31.224296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.738529+0000","last_change":"2024-03-21T11:20:53.739994+0000","last_active":"2024-03-21T11:20:54.738529+0000","last_peered":"2024-03-21T11:20:54.738529+0000","last_clean":"2024-03-21T11:20:54.738529+0000","last_became_active":"2024-03-21T11:20:53.738319+0000","last_became_peered":"2024-03-21T11:20:53.738319+0000","last_unstale":"2024-03-21T11:20:54.738529+0000","last_undegraded":"2024-03-21T11:20:54.738529+0000","last_fullsized":"2024-03-21T11:20:54.738529+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:55.694986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'25","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:20.609592+0000","last_change":"2024-03-21T11:20:48.131461+0000","last_active":"2024-03-21T11:21:20.609592+0000","last_peered":"2024-03-21T11:21:20.609592+0000","last_clean":"2024-03-21T11:21:20.609592+0000","last_became_active":"2024-03-21T11:20:48.131079+0000","last_became_peered":"2024-03-21T11:20:48.131079+0000","last_unstale":"2024-03-21T11:21:20.609592+0000","last_undegraded":"2024-03-21T11:21:20.609592+0000","last_fullsized":"2024-03-21T11:21:20.609592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:48:57.528444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":45,"num_write":27,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721555+0000","last_change":"2024-03-21T11:20:48.653696+0000","last_active":"2024-03-21T11:20:55.721555+0000","last_peered":"2024-03-21T11:20:55.721555+0000","last_clean":"2024-03-21T11:20:55.721555+0000","last_became_active":"2024-03-21T11:20:48.653218+0000","last_became_peered":"2024-03-21T11:20:48.653218+0000","last_unstale":"2024-03-21T11:20:55.721555+0000","last_undegraded":"2024-03-21T11:20:55.721555+0000","last_fullsized":"2024-03-21T11:20:55.721555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:11:59.419040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330931+0000","last_change":"2024-03-21T11:20:53.754019+0000","last_active":"2024-03-21T11:20:55.330931+0000","last_peered":"2024-03-21T11:20:55.330931+0000","last_clean":"2024-03-21T11:20:55.330931+0000","last_became_active":"2024-03-21T11:20:53.753606+0000","last_became_peered":"2024-03-21T11:20:53.753606+0000","last_unstale":"2024-03-21T11:20:55.330931+0000","last_undegraded":"2024-03-21T11:20:55.330931+0000","last_fullsized":"2024-03-21T11:20:55.330931+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:23:27.898596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:18.601797+0000","last_change":"2024-03-21T11:20:48.153544+0000","last_active":"2024-03-21T11:21:18.601797+0000","last_peered":"2024-03-21T11:21:18.601797+0000","last_clean":"2024-03-21T11:21:18.601797+0000","last_became_active":"2024-03-21T11:20:48.153279+0000","last_became_peered":"2024-03-21T11:20:48.153279+0000","last_unstale":"2024-03-21T11:21:18.601797+0000","last_undegraded":"2024-03-21T11:21:18.601797+0000","last_fullsized":"2024-03-21T11:21:18.601797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:31:31.911844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.832250+0000","last_change":"2024-03-21T11:20:48.665140+0000","last_active":"2024-03-21T11:20:55.832250+0000","last_peered":"2024-03-21T11:20:55.832250+0000","last_clean":"2024-03-21T11:20:55.832250+0000","last_became_active":"2024-03-21T11:20:48.664226+0000","last_became_peered":"2024-03-21T11:20:48.664226+0000","last_unstale":"2024-03-21T11:20:55.832250+0000","last_undegraded":"2024-03-21T11:20:55.832250+0000","last_fullsized":"2024-03-21T11:20:55.832250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:47:38.951403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042226+0000","last_change":"2024-03-21T11:20:53.751887+0000","last_active":"2024-03-21T11:20:55.042226+0000","last_peered":"2024-03-21T11:20:55.042226+0000","last_clean":"2024-03-21T11:20:55.042226+0000","last_became_active":"2024-03-21T11:20:53.751353+0000","last_became_peered":"2024-03-21T11:20:53.751353+0000","last_unstale":"2024-03-21T11:20:55.042226+0000","last_undegraded":"2024-03-21T11:20:55.042226+0000","last_fullsized":"2024-03-21T11:20:55.042226+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:21:36.689786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.839973+0000","last_change":"2024-03-21T11:20:48.162150+0000","last_active":"2024-03-21T11:21:14.839973+0000","last_peered":"2024-03-21T11:21:14.839973+0000","last_clean":"2024-03-21T11:21:14.839973+0000","last_became_active":"2024-03-21T11:20:48.161825+0000","last_became_peered":"2024-03-21T11:20:48.161825+0000","last_unstale":"2024-03-21T11:21:14.839973+0000","last_undegraded":"2024-03-21T11:21:14.839973+0000","last_fullsized":"2024-03-21T11:21:14.839973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:10:47.097540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721494+0000","last_change":"2024-03-21T11:20:48.663856+0000","last_active":"2024-03-21T11:20:55.721494+0000","last_peered":"2024-03-21T11:20:55.721494+0000","last_clean":"2024-03-21T11:20:55.721494+0000","last_became_active":"2024-03-21T11:20:48.662908+0000","last_became_peered":"2024-03-21T11:20:48.662908+0000","last_unstale":"2024-03-21T11:20:55.721494+0000","last_undegraded":"2024-03-21T11:20:55.721494+0000","last_fullsized":"2024-03-21T11:20:55.721494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:11:12.255265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.739719+0000","last_change":"2024-03-21T11:20:53.755942+0000","last_active":"2024-03-21T11:20:54.739719+0000","last_peered":"2024-03-21T11:20:54.739719+0000","last_clean":"2024-03-21T11:20:54.739719+0000","last_became_active":"2024-03-21T11:20:53.755141+0000","last_became_peered":"2024-03-21T11:20:53.755141+0000","last_unstale":"2024-03-21T11:20:54.739719+0000","last_undegraded":"2024-03-21T11:20:54.739719+0000","last_fullsized":"2024-03-21T11:20:54.739719+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:30.141117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:16.867878+0000","last_change":"2024-03-21T11:20:48.132460+0000","last_active":"2024-03-21T11:21:16.867878+0000","last_peered":"2024-03-21T11:21:16.867878+0000","last_clean":"2024-03-21T11:21:16.867878+0000","last_became_active":"2024-03-21T11:20:48.131573+0000","last_became_peered":"2024-03-21T11:20:48.131573+0000","last_unstale":"2024-03-21T11:21:16.867878+0000","last_undegraded":"2024-03-21T11:21:16.867878+0000","last_fullsized":"2024-03-21T11:21:16.867878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:40:02.821536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721245+0000","last_change":"2024-03-21T11:20:48.656262+0000","last_active":"2024-03-21T11:20:55.721245+0000","last_peered":"2024-03-21T11:20:55.721245+0000","last_clean":"2024-03-21T11:20:55.721245+0000","last_became_active":"2024-03-21T11:20:48.655924+0000","last_became_peered":"2024-03-21T11:20:48.655924+0000","last_unstale":"2024-03-21T11:20:55.721245+0000","last_undegraded":"2024-03-21T11:20:55.721245+0000","last_fullsized":"2024-03-21T11:20:55.721245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:24:20.412185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.833712+0000","last_change":"2024-03-21T11:20:53.758666+0000","last_active":"2024-03-21T11:20:55.833712+0000","last_peered":"2024-03-21T11:20:55.833712+0000","last_clean":"2024-03-21T11:20:55.833712+0000","last_became_active":"2024-03-21T11:20:53.757398+0000","last_became_peered":"2024-03-21T11:20:53.757398+0000","last_unstale":"2024-03-21T11:20:55.833712+0000","last_undegraded":"2024-03-21T11:20:55.833712+0000","last_fullsized":"2024-03-21T11:20:55.833712+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:34.198388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'16","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:15.853646+0000","last_change":"2024-03-21T11:20:48.132725+0000","last_active":"2024-03-21T11:21:15.853646+0000","last_peered":"2024-03-21T11:21:15.853646+0000","last_clean":"2024-03-21T11:21:15.853646+0000","last_became_active":"2024-03-21T11:20:48.132367+0000","last_became_peered":"2024-03-21T11:20:48.132367+0000","last_unstale":"2024-03-21T11:21:15.853646+0000","last_undegraded":"2024-03-21T11:21:15.853646+0000","last_fullsized":"2024-03-21T11:21:15.853646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:19:18.334962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330145+0000","last_change":"2024-03-21T11:20:48.656302+0000","last_active":"2024-03-21T11:20:55.330145+0000","last_peered":"2024-03-21T11:20:55.330145+0000","last_clean":"2024-03-21T11:20:55.330145+0000","last_became_active":"2024-03-21T11:20:48.654305+0000","last_became_peered":"2024-03-21T11:20:48.654305+0000","last_unstale":"2024-03-21T11:20:55.330145+0000","last_undegraded":"2024-03-21T11:20:55.330145+0000","last_fullsized":"2024-03-21T11:20:55.330145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:59:40.766829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330319+0000","last_change":"2024-03-21T11:20:53.750710+0000","last_active":"2024-03-21T11:20:55.330319+0000","last_peered":"2024-03-21T11:20:55.330319+0000","last_clean":"2024-03-21T11:20:55.330319+0000","last_became_active":"2024-03-21T11:20:53.750255+0000","last_became_peered":"2024-03-21T11:20:53.750255+0000","last_unstale":"2024-03-21T11:20:55.330319+0000","last_undegraded":"2024-03-21T11:20:55.330319+0000","last_fullsized":"2024-03-21T11:20:55.330319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:36:28.611838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721788+0000","last_change":"2024-03-21T11:20:48.133524+0000","last_active":"2024-03-21T11:20:55.721788+0000","last_peered":"2024-03-21T11:20:55.721788+0000","last_clean":"2024-03-21T11:20:55.721788+0000","last_became_active":"2024-03-21T11:20:48.133186+0000","last_became_peered":"2024-03-21T11:20:48.133186+0000","last_unstale":"2024-03-21T11:20:55.721788+0000","last_undegraded":"2024-03-21T11:20:55.721788+0000","last_fullsized":"2024-03-21T11:20:55.721788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:02.018639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.041999+0000","last_change":"2024-03-21T11:20:48.654126+0000","last_active":"2024-03-21T11:20:55.041999+0000","last_peered":"2024-03-21T11:20:55.041999+0000","last_clean":"2024-03-21T11:20:55.041999+0000","last_became_active":"2024-03-21T11:20:48.653835+0000","last_became_peered":"2024-03-21T11:20:48.653835+0000","last_unstale":"2024-03-21T11:20:55.041999+0000","last_undegraded":"2024-03-21T11:20:55.041999+0000","last_fullsized":"2024-03-21T11:20:55.041999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:12:03.332076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721501+0000","last_change":"2024-03-21T11:20:53.916645+0000","last_active":"2024-03-21T11:20:55.721501+0000","last_peered":"2024-03-21T11:20:55.721501+0000","last_clean":"2024-03-21T11:20:55.721501+0000","last_became_active":"2024-03-21T11:20:53.916178+0000","last_became_peered":"2024-03-21T11:20:53.916178+0000","last_unstale":"2024-03-21T11:20:55.721501+0000","last_undegraded":"2024-03-21T11:20:55.721501+0000","last_fullsized":"2024-03-21T11:20:55.721501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:56:15.805231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.738813+0000","last_change":"2024-03-21T11:20:48.159877+0000","last_active":"2024-03-21T11:20:54.738813+0000","last_peered":"2024-03-21T11:20:54.738813+0000","last_clean":"2024-03-21T11:20:54.738813+0000","last_became_active":"2024-03-21T11:20:48.159529+0000","last_became_peered":"2024-03-21T11:20:48.159529+0000","last_unstale":"2024-03-21T11:20:54.738813+0000","last_undegraded":"2024-03-21T11:20:54.738813+0000","last_fullsized":"2024-03-21T11:20:54.738813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:17:09.662462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.640028+0000","last_change":"2024-03-21T11:20:48.663311+0000","last_active":"2024-03-21T11:20:55.640028+0000","last_peered":"2024-03-21T11:20:55.640028+0000","last_clean":"2024-03-21T11:20:55.640028+0000","last_became_active":"2024-03-21T11:20:48.662961+0000","last_became_peered":"2024-03-21T11:20:48.662961+0000","last_unstale":"2024-03-21T11:20:55.640028+0000","last_undegraded":"2024-03-21T11:20:55.640028+0000","last_fullsized":"2024-03-21T11:20:55.640028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:14:46.616326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.833090+0000","last_change":"2024-03-21T11:20:53.751804+0000","last_active":"2024-03-21T11:20:55.833090+0000","last_peered":"2024-03-21T11:20:55.833090+0000","last_clean":"2024-03-21T11:20:55.833090+0000","last_became_active":"2024-03-21T11:20:53.751323+0000","last_became_peered":"2024-03-21T11:20:53.751323+0000","last_unstale":"2024-03-21T11:20:55.833090+0000","last_undegraded":"2024-03-21T11:20:55.833090+0000","last_fullsized":"2024-03-21T11:20:55.833090+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:38:25.677345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721346+0000","last_change":"2024-03-21T11:20:48.147580+0000","last_active":"2024-03-21T11:20:55.721346+0000","last_peered":"2024-03-21T11:20:55.721346+0000","last_clean":"2024-03-21T11:20:55.721346+0000","last_became_active":"2024-03-21T11:20:48.147222+0000","last_became_peered":"2024-03-21T11:20:48.147222+0000","last_unstale":"2024-03-21T11:20:55.721346+0000","last_undegraded":"2024-03-21T11:20:55.721346+0000","last_fullsized":"2024-03-21T11:20:55.721346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:49:04.204129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.639876+0000","last_change":"2024-03-21T11:20:48.667940+0000","last_active":"2024-03-21T11:20:55.639876+0000","last_peered":"2024-03-21T11:20:55.639876+0000","last_clean":"2024-03-21T11:20:55.639876+0000","last_became_active":"2024-03-21T11:20:48.667595+0000","last_became_peered":"2024-03-21T11:20:48.667595+0000","last_unstale":"2024-03-21T11:20:55.639876+0000","last_undegraded":"2024-03-21T11:20:55.639876+0000","last_fullsized":"2024-03-21T11:20:55.639876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:08:14.007589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.833727+0000","last_change":"2024-03-21T11:20:53.916931+0000","last_active":"2024-03-21T11:20:55.833727+0000","last_peered":"2024-03-21T11:20:55.833727+0000","last_clean":"2024-03-21T11:20:55.833727+0000","last_became_active":"2024-03-21T11:20:53.916495+0000","last_became_peered":"2024-03-21T11:20:53.916495+0000","last_unstale":"2024-03-21T11:20:55.833727+0000","last_undegraded":"2024-03-21T11:20:55.833727+0000","last_fullsized":"2024-03-21T11:20:55.833727+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:18:33.046052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330279+0000","last_change":"2024-03-21T11:20:48.652997+0000","last_active":"2024-03-21T11:20:55.330279+0000","last_peered":"2024-03-21T11:20:55.330279+0000","last_clean":"2024-03-21T11:20:55.330279+0000","last_became_active":"2024-03-21T11:20:48.652680+0000","last_became_peered":"2024-03-21T11:20:48.652680+0000","last_unstale":"2024-03-21T11:20:55.330279+0000","last_undegraded":"2024-03-21T11:20:55.330279+0000","last_fullsized":"2024-03-21T11:20:55.330279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:00:19.516607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:15.858857+0000","last_change":"2024-03-21T11:20:48.164607+0000","last_active":"2024-03-21T11:21:15.858857+0000","last_peered":"2024-03-21T11:21:15.858857+0000","last_clean":"2024-03-21T11:21:15.858857+0000","last_became_active":"2024-03-21T11:20:48.164222+0000","last_became_peered":"2024-03-21T11:20:48.164222+0000","last_unstale":"2024-03-21T11:21:15.858857+0000","last_undegraded":"2024-03-21T11:21:15.858857+0000","last_fullsized":"2024-03-21T11:21:15.858857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:52:24.359155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721577+0000","last_change":"2024-03-21T11:20:53.733695+0000","last_active":"2024-03-21T11:20:55.721577+0000","last_peered":"2024-03-21T11:20:55.721577+0000","last_clean":"2024-03-21T11:20:55.721577+0000","last_became_active":"2024-03-21T11:20:53.733246+0000","last_became_peered":"2024-03-21T11:20:53.733246+0000","last_unstale":"2024-03-21T11:20:55.721577+0000","last_undegraded":"2024-03-21T11:20:55.721577+0000","last_fullsized":"2024-03-21T11:20:55.721577+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:16:17.689387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.570582+0000","last_change":"2024-03-21T11:20:48.667212+0000","last_active":"2024-03-21T11:20:55.570582+0000","last_peered":"2024-03-21T11:20:55.570582+0000","last_clean":"2024-03-21T11:20:55.570582+0000","last_became_active":"2024-03-21T11:20:48.662839+0000","last_became_peered":"2024-03-21T11:20:48.662839+0000","last_unstale":"2024-03-21T11:20:55.570582+0000","last_undegraded":"2024-03-21T11:20:55.570582+0000","last_fullsized":"2024-03-21T11:20:55.570582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:27:41.490922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:16.873914+0000","last_change":"2024-03-21T11:20:48.151185+0000","last_active":"2024-03-21T11:21:16.873914+0000","last_peered":"2024-03-21T11:21:16.873914+0000","last_clean":"2024-03-21T11:21:16.873914+0000","last_became_active":"2024-03-21T11:20:48.150859+0000","last_became_peered":"2024-03-21T11:20:48.150859+0000","last_unstale":"2024-03-21T11:21:16.873914+0000","last_undegraded":"2024-03-21T11:21:16.873914+0000","last_fullsized":"2024-03-21T11:21:16.873914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:52:45.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721118+0000","last_change":"2024-03-21T11:19:28.916140+0000","last_active":"2024-03-21T11:20:55.721118+0000","last_peered":"2024-03-21T11:20:55.721118+0000","last_clean":"2024-03-21T11:20:55.721118+0000","last_became_active":"2024-03-21T11:19:28.915634+0000","last_became_peered":"2024-03-21T11:19:28.915634+0000","last_unstale":"2024-03-21T11:20:55.721118+0000","last_undegraded":"2024-03-21T11:20:55.721118+0000","last_fullsized":"2024-03-21T11:20:55.721118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:15:35.803515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:15:35.803515+0000","last_clean_scrub_stamp":"2024-03-21T11:15:35.803515+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:08:31.509894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330760+0000","last_change":"2024-03-21T11:20:53.917119+0000","last_active":"2024-03-21T11:20:55.330760+0000","last_peered":"2024-03-21T11:20:55.330760+0000","last_clean":"2024-03-21T11:20:55.330760+0000","last_became_active":"2024-03-21T11:20:53.916213+0000","last_became_peered":"2024-03-21T11:20:53.916213+0000","last_unstale":"2024-03-21T11:20:55.330760+0000","last_undegraded":"2024-03-21T11:20:55.330760+0000","last_fullsized":"2024-03-21T11:20:55.330760+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:48:50.874202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.831939+0000","last_change":"2024-03-21T11:20:48.657878+0000","last_active":"2024-03-21T11:20:55.831939+0000","last_peered":"2024-03-21T11:20:55.831939+0000","last_clean":"2024-03-21T11:20:55.831939+0000","last_became_active":"2024-03-21T11:20:48.657212+0000","last_became_peered":"2024-03-21T11:20:48.657212+0000","last_unstale":"2024-03-21T11:20:55.831939+0000","last_undegraded":"2024-03-21T11:20:55.831939+0000","last_fullsized":"2024-03-21T11:20:55.831939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:43:11.993577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:15.850471+0000","last_change":"2024-03-21T11:20:48.135513+0000","last_active":"2024-03-21T11:21:15.850471+0000","last_peered":"2024-03-21T11:21:15.850471+0000","last_clean":"2024-03-21T11:21:15.850471+0000","last_became_active":"2024-03-21T11:20:48.135223+0000","last_became_peered":"2024-03-21T11:20:48.135223+0000","last_unstale":"2024-03-21T11:21:15.850471+0000","last_undegraded":"2024-03-21T11:21:15.850471+0000","last_fullsized":"2024-03-21T11:21:15.850471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:37:38.834284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042119+0000","last_change":"2024-03-21T11:20:53.751790+0000","last_active":"2024-03-21T11:20:55.042119+0000","last_peered":"2024-03-21T11:20:55.042119+0000","last_clean":"2024-03-21T11:20:55.042119+0000","last_became_active":"2024-03-21T11:20:53.751312+0000","last_became_peered":"2024-03-21T11:20:53.751312+0000","last_unstale":"2024-03-21T11:20:55.042119+0000","last_undegraded":"2024-03-21T11:20:55.042119+0000","last_fullsized":"2024-03-21T11:20:55.042119+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:21:41.599985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:16.876078+0000","last_change":"2024-03-21T11:20:48.154119+0000","last_active":"2024-03-21T11:21:16.876078+0000","last_peered":"2024-03-21T11:21:16.876078+0000","last_clean":"2024-03-21T11:21:16.876078+0000","last_became_active":"2024-03-21T11:20:48.153781+0000","last_became_peered":"2024-03-21T11:20:48.153781+0000","last_unstale":"2024-03-21T11:21:16.876078+0000","last_undegraded":"2024-03-21T11:21:16.876078+0000","last_fullsized":"2024-03-21T11:21:16.876078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:44:13.937836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.833154+0000","last_change":"2024-03-21T11:20:48.661282+0000","last_active":"2024-03-21T11:20:55.833154+0000","last_peered":"2024-03-21T11:20:55.833154+0000","last_clean":"2024-03-21T11:20:55.833154+0000","last_became_active":"2024-03-21T11:20:48.660569+0000","last_became_peered":"2024-03-21T11:20:48.660569+0000","last_unstale":"2024-03-21T11:20:55.833154+0000","last_undegraded":"2024-03-21T11:20:55.833154+0000","last_fullsized":"2024-03-21T11:20:55.833154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:14.875700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.570710+0000","last_change":"2024-03-21T11:20:53.918016+0000","last_active":"2024-03-21T11:20:55.570710+0000","last_peered":"2024-03-21T11:20:55.570710+0000","last_clean":"2024-03-21T11:20:55.570710+0000","last_became_active":"2024-03-21T11:20:53.916891+0000","last_became_peered":"2024-03-21T11:20:53.916891+0000","last_unstale":"2024-03-21T11:20:55.570710+0000","last_undegraded":"2024-03-21T11:20:55.570710+0000","last_fullsized":"2024-03-21T11:20:55.570710+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:40:25.206778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:16.877158+0000","last_change":"2024-03-21T11:20:48.167560+0000","last_active":"2024-03-21T11:21:16.877158+0000","last_peered":"2024-03-21T11:21:16.877158+0000","last_clean":"2024-03-21T11:21:16.877158+0000","last_became_active":"2024-03-21T11:20:48.164986+0000","last_became_peered":"2024-03-21T11:20:48.164986+0000","last_unstale":"2024-03-21T11:21:16.877158+0000","last_undegraded":"2024-03-21T11:21:16.877158+0000","last_fullsized":"2024-03-21T11:21:16.877158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:59.930626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.571946+0000","last_change":"2024-03-21T11:20:48.675250+0000","last_active":"2024-03-21T11:20:55.571946+0000","last_peered":"2024-03-21T11:20:55.571946+0000","last_clean":"2024-03-21T11:20:55.571946+0000","last_became_active":"2024-03-21T11:20:48.674638+0000","last_became_peered":"2024-03-21T11:20:48.674638+0000","last_unstale":"2024-03-21T11:20:55.571946+0000","last_undegraded":"2024-03-21T11:20:55.571946+0000","last_fullsized":"2024-03-21T11:20:55.571946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:46:34.842363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734240+0000","last_change":"2024-03-21T11:20:53.733809+0000","last_active":"2024-03-21T11:20:54.734240+0000","last_peered":"2024-03-21T11:20:54.734240+0000","last_clean":"2024-03-21T11:20:54.734240+0000","last_became_active":"2024-03-21T11:20:53.733393+0000","last_became_peered":"2024-03-21T11:20:53.733393+0000","last_unstale":"2024-03-21T11:20:54.734240+0000","last_undegraded":"2024-03-21T11:20:54.734240+0000","last_fullsized":"2024-03-21T11:20:54.734240+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:02:46.270716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.837529+0000","last_change":"2024-03-21T11:20:48.139475+0000","last_active":"2024-03-21T11:21:14.837529+0000","last_peered":"2024-03-21T11:21:14.837529+0000","last_clean":"2024-03-21T11:21:14.837529+0000","last_became_active":"2024-03-21T11:20:48.139249+0000","last_became_peered":"2024-03-21T11:20:48.139249+0000","last_unstale":"2024-03-21T11:21:14.837529+0000","last_undegraded":"2024-03-21T11:21:14.837529+0000","last_fullsized":"2024-03-21T11:21:14.837529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:45:18.811829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.722997+0000","last_change":"2024-03-21T11:20:48.673975+0000","last_active":"2024-03-21T11:20:55.722997+0000","last_peered":"2024-03-21T11:20:55.722997+0000","last_clean":"2024-03-21T11:20:55.722997+0000","last_became_active":"2024-03-21T11:20:48.673587+0000","last_became_peered":"2024-03-21T11:20:48.673587+0000","last_unstale":"2024-03-21T11:20:55.722997+0000","last_undegraded":"2024-03-21T11:20:55.722997+0000","last_fullsized":"2024-03-21T11:20:55.722997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:08:52.354780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734877+0000","last_change":"2024-03-21T11:20:53.917493+0000","last_active":"2024-03-21T11:20:54.734877+0000","last_peered":"2024-03-21T11:20:54.734877+0000","last_clean":"2024-03-21T11:20:54.734877+0000","last_became_active":"2024-03-21T11:20:53.917007+0000","last_became_peered":"2024-03-21T11:20:53.917007+0000","last_unstale":"2024-03-21T11:20:54.734877+0000","last_undegraded":"2024-03-21T11:20:54.734877+0000","last_fullsized":"2024-03-21T11:20:54.734877+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:39:20.280899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.840422+0000","last_change":"2024-03-21T11:20:48.140914+0000","last_active":"2024-03-21T11:21:14.840422+0000","last_peered":"2024-03-21T11:21:14.840422+0000","last_clean":"2024-03-21T11:21:14.840422+0000","last_became_active":"2024-03-21T11:20:48.140558+0000","last_became_peered":"2024-03-21T11:20:48.140558+0000","last_unstale":"2024-03-21T11:21:14.840422+0000","last_undegraded":"2024-03-21T11:21:14.840422+0000","last_fullsized":"2024-03-21T11:21:14.840422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:59:52.065431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721230+0000","last_change":"2024-03-21T11:20:48.651190+0000","last_active":"2024-03-21T11:20:55.721230+0000","last_peered":"2024-03-21T11:20:55.721230+0000","last_clean":"2024-03-21T11:20:55.721230+0000","last_became_active":"2024-03-21T11:20:48.650818+0000","last_became_peered":"2024-03-21T11:20:48.650818+0000","last_unstale":"2024-03-21T11:20:55.721230+0000","last_undegraded":"2024-03-21T11:20:55.721230+0000","last_fullsized":"2024-03-21T11:20:55.721230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:47:17.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.739717+0000","last_change":"2024-03-21T11:20:53.756844+0000","last_active":"2024-03-21T11:20:54.739717+0000","last_peered":"2024-03-21T11:20:54.739717+0000","last_clean":"2024-03-21T11:20:54.739717+0000","last_became_active":"2024-03-21T11:20:53.756380+0000","last_became_peered":"2024-03-21T11:20:53.756380+0000","last_unstale":"2024-03-21T11:20:54.739717+0000","last_undegraded":"2024-03-21T11:20:54.739717+0000","last_fullsized":"2024-03-21T11:20:54.739717+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:51:55.327798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.840519+0000","last_change":"2024-03-21T11:20:48.160749+0000","last_active":"2024-03-21T11:21:14.840519+0000","last_peered":"2024-03-21T11:21:14.840519+0000","last_clean":"2024-03-21T11:21:14.840519+0000","last_became_active":"2024-03-21T11:20:48.158507+0000","last_became_peered":"2024-03-21T11:20:48.158507+0000","last_unstale":"2024-03-21T11:21:14.840519+0000","last_undegraded":"2024-03-21T11:21:14.840519+0000","last_fullsized":"2024-03-21T11:21:14.840519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:55.375529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.041775+0000","last_change":"2024-03-21T11:20:48.667153+0000","last_active":"2024-03-21T11:20:55.041775+0000","last_peered":"2024-03-21T11:20:55.041775+0000","last_clean":"2024-03-21T11:20:55.041775+0000","last_became_active":"2024-03-21T11:20:48.666823+0000","last_became_peered":"2024-03-21T11:20:48.666823+0000","last_unstale":"2024-03-21T11:20:55.041775+0000","last_undegraded":"2024-03-21T11:20:55.041775+0000","last_fullsized":"2024-03-21T11:20:55.041775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:43:54.183372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734557+0000","last_change":"2024-03-21T11:20:53.739188+0000","last_active":"2024-03-21T11:20:54.734557+0000","last_peered":"2024-03-21T11:20:54.734557+0000","last_clean":"2024-03-21T11:20:54.734557+0000","last_became_active":"2024-03-21T11:20:53.738770+0000","last_became_peered":"2024-03-21T11:20:53.738770+0000","last_unstale":"2024-03-21T11:20:54.734557+0000","last_undegraded":"2024-03-21T11:20:54.734557+0000","last_fullsized":"2024-03-21T11:20:54.734557+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:53:48.897963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:15.833224+0000","last_change":"2024-03-21T11:20:48.162279+0000","last_active":"2024-03-21T11:21:15.833224+0000","last_peered":"2024-03-21T11:21:15.833224+0000","last_clean":"2024-03-21T11:21:15.833224+0000","last_became_active":"2024-03-21T11:20:48.161878+0000","last_became_peered":"2024-03-21T11:20:48.161878+0000","last_unstale":"2024-03-21T11:21:15.833224+0000","last_undegraded":"2024-03-21T11:21:15.833224+0000","last_fullsized":"2024-03-21T11:21:15.833224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:34:15.144710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.571021+0000","last_change":"2024-03-21T11:20:48.677303+0000","last_active":"2024-03-21T11:20:55.571021+0000","last_peered":"2024-03-21T11:20:55.571021+0000","last_clean":"2024-03-21T11:20:55.571021+0000","last_became_active":"2024-03-21T11:20:48.676903+0000","last_became_peered":"2024-03-21T11:20:48.676903+0000","last_unstale":"2024-03-21T11:20:55.571021+0000","last_undegraded":"2024-03-21T11:20:55.571021+0000","last_fullsized":"2024-03-21T11:20:55.571021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:06:59.922274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734605+0000","last_change":"2024-03-21T11:20:53.916691+0000","last_active":"2024-03-21T11:20:54.734605+0000","last_peered":"2024-03-21T11:20:54.734605+0000","last_clean":"2024-03-21T11:20:54.734605+0000","last_became_active":"2024-03-21T11:20:53.916275+0000","last_became_peered":"2024-03-21T11:20:53.916275+0000","last_unstale":"2024-03-21T11:20:54.734605+0000","last_undegraded":"2024-03-21T11:20:54.734605+0000","last_fullsized":"2024-03-21T11:20:54.734605+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:35.848589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.835756+0000","last_change":"2024-03-21T11:20:48.134292+0000","last_active":"2024-03-21T11:21:14.835756+0000","last_peered":"2024-03-21T11:21:14.835756+0000","last_clean":"2024-03-21T11:21:14.835756+0000","last_became_active":"2024-03-21T11:20:48.133957+0000","last_became_peered":"2024-03-21T11:20:48.133957+0000","last_unstale":"2024-03-21T11:21:14.835756+0000","last_undegraded":"2024-03-21T11:21:14.835756+0000","last_fullsized":"2024-03-21T11:21:14.835756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:19:06.066423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721338+0000","last_change":"2024-03-21T11:20:48.661486+0000","last_active":"2024-03-21T11:20:55.721338+0000","last_peered":"2024-03-21T11:20:55.721338+0000","last_clean":"2024-03-21T11:20:55.721338+0000","last_became_active":"2024-03-21T11:20:48.661109+0000","last_became_peered":"2024-03-21T11:20:48.661109+0000","last_unstale":"2024-03-21T11:20:55.721338+0000","last_undegraded":"2024-03-21T11:20:55.721338+0000","last_fullsized":"2024-03-21T11:20:55.721338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:15:27.093616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.720414+0000","last_change":"2024-03-21T11:20:53.747223+0000","last_active":"2024-03-21T11:20:55.720414+0000","last_peered":"2024-03-21T11:20:55.720414+0000","last_clean":"2024-03-21T11:20:55.720414+0000","last_became_active":"2024-03-21T11:20:53.746805+0000","last_became_peered":"2024-03-21T11:20:53.746805+0000","last_unstale":"2024-03-21T11:20:55.720414+0000","last_undegraded":"2024-03-21T11:20:55.720414+0000","last_fullsized":"2024-03-21T11:20:55.720414+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:29.186745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:18.604767+0000","last_change":"2024-03-21T11:20:48.151172+0000","last_active":"2024-03-21T11:21:18.604767+0000","last_peered":"2024-03-21T11:21:18.604767+0000","last_clean":"2024-03-21T11:21:18.604767+0000","last_became_active":"2024-03-21T11:20:48.150832+0000","last_became_peered":"2024-03-21T11:20:48.150832+0000","last_unstale":"2024-03-21T11:21:18.604767+0000","last_undegraded":"2024-03-21T11:21:18.604767+0000","last_fullsized":"2024-03-21T11:21:18.604767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:34:46.367713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721772+0000","last_change":"2024-03-21T11:20:48.652275+0000","last_active":"2024-03-21T11:20:55.721772+0000","last_peered":"2024-03-21T11:20:55.721772+0000","last_clean":"2024-03-21T11:20:55.721772+0000","last_became_active":"2024-03-21T11:20:48.651965+0000","last_became_peered":"2024-03-21T11:20:48.651965+0000","last_unstale":"2024-03-21T11:20:55.721772+0000","last_undegraded":"2024-03-21T11:20:55.721772+0000","last_fullsized":"2024-03-21T11:20:55.721772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:56:18.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721946+0000","last_change":"2024-03-21T11:20:53.748050+0000","last_active":"2024-03-21T11:20:55.721946+0000","last_peered":"2024-03-21T11:20:55.721946+0000","last_clean":"2024-03-21T11:20:55.721946+0000","last_became_active":"2024-03-21T11:20:53.747636+0000","last_became_peered":"2024-03-21T11:20:53.747636+0000","last_unstale":"2024-03-21T11:20:55.721946+0000","last_undegraded":"2024-03-21T11:20:55.721946+0000","last_fullsized":"2024-03-21T11:20:55.721946+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:09:15.202183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330698+0000","last_change":"2024-03-21T11:20:48.156928+0000","last_active":"2024-03-21T11:20:55.330698+0000","last_peered":"2024-03-21T11:20:55.330698+0000","last_clean":"2024-03-21T11:20:55.330698+0000","last_became_active":"2024-03-21T11:20:48.156625+0000","last_became_peered":"2024-03-21T11:20:48.156625+0000","last_unstale":"2024-03-21T11:20:55.330698+0000","last_undegraded":"2024-03-21T11:20:55.330698+0000","last_fullsized":"2024-03-21T11:20:55.330698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:19:42.035565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721547+0000","last_change":"2024-03-21T11:20:48.663655+0000","last_active":"2024-03-21T11:20:55.721547+0000","last_peered":"2024-03-21T11:20:55.721547+0000","last_clean":"2024-03-21T11:20:55.721547+0000","last_became_active":"2024-03-21T11:20:48.663336+0000","last_became_peered":"2024-03-21T11:20:48.663336+0000","last_unstale":"2024-03-21T11:20:55.721547+0000","last_undegraded":"2024-03-21T11:20:55.721547+0000","last_fullsized":"2024-03-21T11:20:55.721547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:29:05.119283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.329990+0000","last_change":"2024-03-21T11:20:53.732584+0000","last_active":"2024-03-21T11:20:55.329990+0000","last_peered":"2024-03-21T11:20:55.329990+0000","last_clean":"2024-03-21T11:20:55.329990+0000","last_became_active":"2024-03-21T11:20:53.731975+0000","last_became_peered":"2024-03-21T11:20:53.731975+0000","last_unstale":"2024-03-21T11:20:55.329990+0000","last_undegraded":"2024-03-21T11:20:55.329990+0000","last_fullsized":"2024-03-21T11:20:55.329990+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:51:06.336241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.833544+0000","last_change":"2024-03-21T11:20:48.165594+0000","last_active":"2024-03-21T11:20:55.833544+0000","last_peered":"2024-03-21T11:20:55.833544+0000","last_clean":"2024-03-21T11:20:55.833544+0000","last_became_active":"2024-03-21T11:20:48.164394+0000","last_became_peered":"2024-03-21T11:20:48.164394+0000","last_unstale":"2024-03-21T11:20:55.833544+0000","last_undegraded":"2024-03-21T11:20:55.833544+0000","last_fullsized":"2024-03-21T11:20:55.833544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:13.317767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721612+0000","last_change":"2024-03-21T11:20:48.649359+0000","last_active":"2024-03-21T11:20:55.721612+0000","last_peered":"2024-03-21T11:20:55.721612+0000","last_clean":"2024-03-21T11:20:55.721612+0000","last_became_active":"2024-03-21T11:20:48.649031+0000","last_became_peered":"2024-03-21T11:20:48.649031+0000","last_unstale":"2024-03-21T11:20:55.721612+0000","last_undegraded":"2024-03-21T11:20:55.721612+0000","last_fullsized":"2024-03-21T11:20:55.721612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:17.055732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042041+0000","last_change":"2024-03-21T11:20:48.654999+0000","last_active":"2024-03-21T11:20:55.042041+0000","last_peered":"2024-03-21T11:20:55.042041+0000","last_clean":"2024-03-21T11:20:55.042041+0000","last_became_active":"2024-03-21T11:20:48.654638+0000","last_became_peered":"2024-03-21T11:20:48.654638+0000","last_unstale":"2024-03-21T11:20:55.042041+0000","last_undegraded":"2024-03-21T11:20:55.042041+0000","last_fullsized":"2024-03-21T11:20:55.042041+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:49:48.711420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734678+0000","last_change":"2024-03-21T11:20:48.651864+0000","last_active":"2024-03-21T11:20:54.734678+0000","last_peered":"2024-03-21T11:20:54.734678+0000","last_clean":"2024-03-21T11:20:54.734678+0000","last_became_active":"2024-03-21T11:20:48.651565+0000","last_became_peered":"2024-03-21T11:20:48.651565+0000","last_unstale":"2024-03-21T11:20:54.734678+0000","last_undegraded":"2024-03-21T11:20:54.734678+0000","last_fullsized":"2024-03-21T11:20:54.734678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:58.659889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734266+0000","last_change":"2024-03-21T11:20:48.667038+0000","last_active":"2024-03-21T11:20:54.734266+0000","last_peered":"2024-03-21T11:20:54.734266+0000","last_clean":"2024-03-21T11:20:54.734266+0000","last_became_active":"2024-03-21T11:20:48.666570+0000","last_became_peered":"2024-03-21T11:20:48.666570+0000","last_unstale":"2024-03-21T11:20:54.734266+0000","last_undegraded":"2024-03-21T11:20:54.734266+0000","last_fullsized":"2024-03-21T11:20:54.734266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:36:39.043697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.722507+0000","last_change":"2024-03-21T11:20:48.676216+0000","last_active":"2024-03-21T11:20:55.722507+0000","last_peered":"2024-03-21T11:20:55.722507+0000","last_clean":"2024-03-21T11:20:55.722507+0000","last_became_active":"2024-03-21T11:20:48.675269+0000","last_became_peered":"2024-03-21T11:20:48.675269+0000","last_unstale":"2024-03-21T11:20:55.722507+0000","last_undegraded":"2024-03-21T11:20:55.722507+0000","last_fullsized":"2024-03-21T11:20:55.722507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:42:47.006371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.833311+0000","last_change":"2024-03-21T11:20:48.670139+0000","last_active":"2024-03-21T11:20:55.833311+0000","last_peered":"2024-03-21T11:20:55.833311+0000","last_clean":"2024-03-21T11:20:55.833311+0000","last_became_active":"2024-03-21T11:20:48.669410+0000","last_became_peered":"2024-03-21T11:20:48.669410+0000","last_unstale":"2024-03-21T11:20:55.833311+0000","last_undegraded":"2024-03-21T11:20:55.833311+0000","last_fullsized":"2024-03-21T11:20:55.833311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:35:18.799840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042314+0000","last_change":"2024-03-21T11:20:48.655356+0000","last_active":"2024-03-21T11:20:55.042314+0000","last_peered":"2024-03-21T11:20:55.042314+0000","last_clean":"2024-03-21T11:20:55.042314+0000","last_became_active":"2024-03-21T11:20:48.655028+0000","last_became_peered":"2024-03-21T11:20:48.655028+0000","last_unstale":"2024-03-21T11:20:55.042314+0000","last_undegraded":"2024-03-21T11:20:55.042314+0000","last_fullsized":"2024-03-21T11:20:55.042314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:46:39.948982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734834+0000","last_change":"2024-03-21T11:20:48.648840+0000","last_active":"2024-03-21T11:20:54.734834+0000","last_peered":"2024-03-21T11:20:54.734834+0000","last_clean":"2024-03-21T11:20:54.734834+0000","last_became_active":"2024-03-21T11:20:48.648502+0000","last_became_peered":"2024-03-21T11:20:48.648502+0000","last_unstale":"2024-03-21T11:20:54.734834+0000","last_undegraded":"2024-03-21T11:20:54.734834+0000","last_fullsized":"2024-03-21T11:20:54.734834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:33:14.824880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.572065+0000","last_change":"2024-03-21T11:20:48.672015+0000","last_active":"2024-03-21T11:20:55.572065+0000","last_peered":"2024-03-21T11:20:55.572065+0000","last_clean":"2024-03-21T11:20:55.572065+0000","last_became_active":"2024-03-21T11:20:48.671663+0000","last_became_peered":"2024-03-21T11:20:48.671663+0000","last_unstale":"2024-03-21T11:20:55.572065+0000","last_undegraded":"2024-03-21T11:20:55.572065+0000","last_fullsized":"2024-03-21T11:20:55.572065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:21:25.328698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330140+0000","last_change":"2024-03-21T11:20:48.664114+0000","last_active":"2024-03-21T11:20:55.330140+0000","last_peered":"2024-03-21T11:20:55.330140+0000","last_clean":"2024-03-21T11:20:55.330140+0000","last_became_active":"2024-03-21T11:20:48.663819+0000","last_became_peered":"2024-03-21T11:20:48.663819+0000","last_unstale":"2024-03-21T11:20:55.330140+0000","last_undegraded":"2024-03-21T11:20:55.330140+0000","last_fullsized":"2024-03-21T11:20:55.330140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:08:08.201383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.570381+0000","last_change":"2024-03-21T11:20:48.671336+0000","last_active":"2024-03-21T11:20:55.570381+0000","last_peered":"2024-03-21T11:20:55.570381+0000","last_clean":"2024-03-21T11:20:55.570381+0000","last_became_active":"2024-03-21T11:20:48.670918+0000","last_became_peered":"2024-03-21T11:20:48.670918+0000","last_unstale":"2024-03-21T11:20:55.570381+0000","last_undegraded":"2024-03-21T11:20:55.570381+0000","last_fullsized":"2024-03-21T11:20:55.570381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:01:44.764584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.720534+0000","last_change":"2024-03-21T11:20:48.659339+0000","last_active":"2024-03-21T11:20:55.720534+0000","last_peered":"2024-03-21T11:20:55.720534+0000","last_clean":"2024-03-21T11:20:55.720534+0000","last_became_active":"2024-03-21T11:20:48.658989+0000","last_became_peered":"2024-03-21T11:20:48.658989+0000","last_unstale":"2024-03-21T11:20:55.720534+0000","last_undegraded":"2024-03-21T11:20:55.720534+0000","last_fullsized":"2024-03-21T11:20:55.720534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:44:09.599836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.832465+0000","last_change":"2024-03-21T11:20:48.668679+0000","last_active":"2024-03-21T11:20:55.832465+0000","last_peered":"2024-03-21T11:20:55.832465+0000","last_clean":"2024-03-21T11:20:55.832465+0000","last_became_active":"2024-03-21T11:20:48.667956+0000","last_became_peered":"2024-03-21T11:20:48.667956+0000","last_unstale":"2024-03-21T11:20:55.832465+0000","last_undegraded":"2024-03-21T11:20:55.832465+0000","last_fullsized":"2024-03-21T11:20:55.832465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:04.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042177+0000","last_change":"2024-03-21T11:20:48.650213+0000","last_active":"2024-03-21T11:20:55.042177+0000","last_peered":"2024-03-21T11:20:55.042177+0000","last_clean":"2024-03-21T11:20:55.042177+0000","last_became_active":"2024-03-21T11:20:48.649693+0000","last_became_peered":"2024-03-21T11:20:48.649693+0000","last_unstale":"2024-03-21T11:20:55.042177+0000","last_undegraded":"2024-03-21T11:20:55.042177+0000","last_fullsized":"2024-03-21T11:20:55.042177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:26:45.180690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247498,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47313,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":20,"num_read_kb":55,"num_write":182,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":125148,"data_compressed":3658,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350997,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":494124,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612634,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1126400,"data_stored":1122846,"data_compressed":52683,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776158,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":489263,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037797,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":489183,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299434,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":1081277,"data_compressed":50854,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462960,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":488535,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":532103,"data_compressed":16622,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":491014,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":501641,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":488639,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888122,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":489111,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724597,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":1080289,"data_compressed":50854,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":43658,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":44033,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T11:21:21.506 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T11:21:21.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.772+0000 7f980f200640 1 -- 172.21.15.47:0/1245420329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98101026d0 msgr2=0x7f9810104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:21.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.772+0000 7f980f200640 1 --2- 172.21.15.47:0/1245420329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98101026d0 0x7f9810104b40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009900 tx=0x7f97fc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:21:21.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.772+0000 7f980f200640 1 -- 172.21.15.47:0/1245420329 shutdown_connections 2024-03-21T11:21:21.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.772+0000 7f980f200640 1 --2- 172.21.15.47:0/1245420329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9810105080 0x7f9810105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.772+0000 7f980f200640 1 --2- 172.21.15.47:0/1245420329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98101026d0 0x7f9810104b40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.772+0000 7f980f200640 1 --2- 172.21.15.47:0/1245420329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98100ffd60 0x7f9810102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.772+0000 7f980f200640 1 -- 172.21.15.47:0/1245420329 >> 172.21.15.47:0/1245420329 conn(0x7f98100fb980 msgr2=0x7f98100fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:21.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.772+0000 7f980f200640 1 -- 172.21.15.47:0/1245420329 shutdown_connections 2024-03-21T11:21:21.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.772+0000 7f980f200640 1 -- 172.21.15.47:0/1245420329 wait complete. 2024-03-21T11:21:21.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.773+0000 7f980f200640 1 Processor -- start 2024-03-21T11:21:21.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.773+0000 7f980f200640 1 -- start start 2024-03-21T11:21:21.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.773+0000 7f980f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98100ffd60 0x7f981019ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:21.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.773+0000 7f980f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98101026d0 0x7f981019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:21.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.773+0000 7f980f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9810105080 0x7f98101a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:21.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.773+0000 7f980f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98101137b0 con 0x7f98100ffd60 2024-03-21T11:21:21.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.773+0000 7f980f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9810113630 con 0x7f9810105080 2024-03-21T11:21:21.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.773+0000 7f980f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9810113930 con 0x7f98101026d0 2024-03-21T11:21:21.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.774+0000 7f980e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9810105080 0x7f98101a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:21.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.774+0000 7f980d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98101026d0 0x7f981019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:21.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.774+0000 7f980e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9810105080 0x7f98101a53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40706/0 (socket says 172.21.15.47:40706) 2024-03-21T11:21:21.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.774+0000 7f980e800640 1 -- 172.21.15.47:0/872385314 learned_addr learned my addr 172.21.15.47:0/872385314 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:21.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.774+0000 7f980de00640 1 --2- 172.21.15.47:0/872385314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98100ffd60 0x7f981019ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:21.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.774+0000 7f980e800640 1 -- 172.21.15.47:0/872385314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98101026d0 msgr2=0x7f981019e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:21.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.774+0000 7f980e800640 1 --2- 172.21.15.47:0/872385314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98101026d0 0x7f981019e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.774+0000 7f980e800640 1 -- 172.21.15.47:0/872385314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98100ffd60 msgr2=0x7f981019ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:21.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.774+0000 7f980e800640 1 --2- 172.21.15.47:0/872385314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98100ffd60 0x7f981019ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:21.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.774+0000 7f980e800640 1 -- 172.21.15.47:0/872385314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98101a5a10 con 0x7f9810105080 2024-03-21T11:21:21.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.774+0000 7f980e800640 1 --2- 172.21.15.47:0/872385314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9810105080 0x7f98101a53a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f980800eb10 tx=0x7f980800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:21.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.775+0000 7f9806a00640 1 -- 172.21.15.47:0/872385314 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808004b20 con 0x7f9810105080 2024-03-21T11:21:21.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.775+0000 7f9806a00640 1 -- 172.21.15.47:0/872385314 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9808004cc0 con 0x7f9810105080 2024-03-21T11:21:21.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.775+0000 7f9806a00640 1 -- 172.21.15.47:0/872385314 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808010760 con 0x7f9810105080 2024-03-21T11:21:21.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.775+0000 7f980f200640 1 -- 172.21.15.47:0/872385314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98101a5d00 con 0x7f9810105080 2024-03-21T11:21:21.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.775+0000 7f980f200640 1 -- 172.21.15.47:0/872385314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98101a61d0 con 0x7f9810105080 2024-03-21T11:21:21.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.776+0000 7f980f200640 1 -- 172.21.15.47:0/872385314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98100fd470 con 0x7f9810105080 2024-03-21T11:21:21.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.778+0000 7f9806a00640 1 -- 172.21.15.47:0/872385314 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f9808005020 con 0x7f9810105080 2024-03-21T11:21:21.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.779+0000 7f9806a00640 1 --2- 172.21.15.47:0/872385314 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97f40a2fb0 0x7f97f40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:21.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.779+0000 7f9806a00640 1 -- 172.21.15.47:0/872385314 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f98080cfd80 con 0x7f9810105080 2024-03-21T11:21:21.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.779+0000 7f980de00640 1 --2- 172.21.15.47:0/872385314 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97f40a2fb0 0x7f97f40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:21.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.780+0000 7f980de00640 1 --2- 172.21.15.47:0/872385314 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97f40a2fb0 0x7f97f40a5470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9800009bd0 tx=0x7f9800005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:21.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.784+0000 7f9806a00640 1 -- 172.21.15.47:0/872385314 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f980809c7b0 con 0x7f9810105080 2024-03-21T11:21:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:21.983+0000 7f980f200640 1 -- 172.21.15.47:0/872385314 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9810061ae0 con 0x7f97f40a2fb0 2024-03-21T11:21:22.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:22.014+0000 7f9806a00640 1 -- 172.21.15.47:0/872385314 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340657 (secure 0 0 0) 0x7f9810061ae0 con 0x7f97f40a2fb0 2024-03-21T11:21:22.019 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:21:22.021 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-21T11:21:22.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:22.020+0000 7f980f200640 1 -- 172.21.15.47:0/872385314 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97f40a2fb0 msgr2=0x7f97f40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:22.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:22.020+0000 7f980f200640 1 --2- 172.21.15.47:0/872385314 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97f40a2fb0 0x7f97f40a5470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9800009bd0 tx=0x7f9800005f90 comp rx=0 tx=0).stop 2024-03-21T11:21:22.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:22.020+0000 7f980f200640 1 -- 172.21.15.47:0/872385314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9810105080 msgr2=0x7f98101a53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:22.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:22.020+0000 7f980f200640 1 --2- 172.21.15.47:0/872385314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9810105080 0x7f98101a53a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f980800eb10 tx=0x7f980800c2d0 comp rx=0 tx=0).stop 2024-03-21T11:21:22.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:22.020+0000 7f980f200640 1 -- 172.21.15.47:0/872385314 shutdown_connections 2024-03-21T11:21:22.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:22.020+0000 7f980f200640 1 --2- 172.21.15.47:0/872385314 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97f40a2fb0 0x7f97f40a5470 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:22.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:22.020+0000 7f980f200640 1 --2- 172.21.15.47:0/872385314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9810105080 0x7f98101a53a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:22.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:22.020+0000 7f980f200640 1 --2- 172.21.15.47:0/872385314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98101026d0 0x7f981019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:22.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:22.020+0000 7f980f200640 1 --2- 172.21.15.47:0/872385314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98100ffd60 0x7f981019ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:22.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:22.020+0000 7f980f200640 1 -- 172.21.15.47:0/872385314 >> 172.21.15.47:0/872385314 conn(0x7f98100fb980 msgr2=0x7f9810100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:22.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:22.020+0000 7f980f200640 1 -- 172.21.15.47:0/872385314 shutdown_connections 2024-03-21T11:21:22.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:22.021+0000 7f980f200640 1 -- 172.21.15.47:0/872385314 wait complete. 2024-03-21T11:21:22.041 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-03-21T11:21:20.992389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":637681,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":146,"num_read_kb":164,"num_write":363,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":297,"ondisk_log_size":297,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373680,"kb_used_data":12048,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568144,"statfs":{"total":1151940427776,"available":1151557779456,"internally_reserved":0,"allocated":12337152,"data_stored":7748025,"data_compressed":289357,"data_compressed_allocated":389120,"data_compressed_original":2277376,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":44454,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":20,"num_read_kb":55,"num_write":136,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005128"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042323+0000","last_change":"2024-03-21T11:20:48.667185+0000","last_active":"2024-03-21T11:20:55.042323+0000","last_peered":"2024-03-21T11:20:55.042323+0000","last_clean":"2024-03-21T11:20:55.042323+0000","last_became_active":"2024-03-21T11:20:48.666855+0000","last_became_peered":"2024-03-21T11:20:48.666855+0000","last_unstale":"2024-03-21T11:20:55.042323+0000","last_undegraded":"2024-03-21T11:20:55.042323+0000","last_fullsized":"2024-03-21T11:20:55.042323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:30:37.393096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.722850+0000","last_change":"2024-03-21T11:20:48.671613+0000","last_active":"2024-03-21T11:20:55.722850+0000","last_peered":"2024-03-21T11:20:55.722850+0000","last_clean":"2024-03-21T11:20:55.722850+0000","last_became_active":"2024-03-21T11:20:48.671211+0000","last_became_peered":"2024-03-21T11:20:48.671211+0000","last_unstale":"2024-03-21T11:20:55.722850+0000","last_undegraded":"2024-03-21T11:20:55.722850+0000","last_fullsized":"2024-03-21T11:20:55.722850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:20.167922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721820+0000","last_change":"2024-03-21T11:20:48.655734+0000","last_active":"2024-03-21T11:20:55.721820+0000","last_peered":"2024-03-21T11:20:55.721820+0000","last_clean":"2024-03-21T11:20:55.721820+0000","last_became_active":"2024-03-21T11:20:48.653798+0000","last_became_peered":"2024-03-21T11:20:48.653798+0000","last_unstale":"2024-03-21T11:20:55.721820+0000","last_undegraded":"2024-03-21T11:20:55.721820+0000","last_fullsized":"2024-03-21T11:20:55.721820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:44.796405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734826+0000","last_change":"2024-03-21T11:20:48.648784+0000","last_active":"2024-03-21T11:20:54.734826+0000","last_peered":"2024-03-21T11:20:54.734826+0000","last_clean":"2024-03-21T11:20:54.734826+0000","last_became_active":"2024-03-21T11:20:48.648040+0000","last_became_peered":"2024-03-21T11:20:48.648040+0000","last_unstale":"2024-03-21T11:20:54.734826+0000","last_undegraded":"2024-03-21T11:20:54.734826+0000","last_fullsized":"2024-03-21T11:20:54.734826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:48:51.786346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721353+0000","last_change":"2024-03-21T11:20:48.672084+0000","last_active":"2024-03-21T11:20:55.721353+0000","last_peered":"2024-03-21T11:20:55.721353+0000","last_clean":"2024-03-21T11:20:55.721353+0000","last_became_active":"2024-03-21T11:20:48.671441+0000","last_became_peered":"2024-03-21T11:20:48.671441+0000","last_unstale":"2024-03-21T11:20:55.721353+0000","last_undegraded":"2024-03-21T11:20:55.721353+0000","last_fullsized":"2024-03-21T11:20:55.721353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:20:17.638270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.639858+0000","last_change":"2024-03-21T11:20:48.659782+0000","last_active":"2024-03-21T11:20:55.639858+0000","last_peered":"2024-03-21T11:20:55.639858+0000","last_clean":"2024-03-21T11:20:55.639858+0000","last_became_active":"2024-03-21T11:20:48.659460+0000","last_became_peered":"2024-03-21T11:20:48.659460+0000","last_unstale":"2024-03-21T11:20:55.639858+0000","last_undegraded":"2024-03-21T11:20:55.639858+0000","last_fullsized":"2024-03-21T11:20:55.639858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:57.712654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721315+0000","last_change":"2024-03-21T11:20:48.659401+0000","last_active":"2024-03-21T11:20:55.721315+0000","last_peered":"2024-03-21T11:20:55.721315+0000","last_clean":"2024-03-21T11:20:55.721315+0000","last_became_active":"2024-03-21T11:20:48.659075+0000","last_became_peered":"2024-03-21T11:20:48.659075+0000","last_unstale":"2024-03-21T11:20:55.721315+0000","last_undegraded":"2024-03-21T11:20:55.721315+0000","last_fullsized":"2024-03-21T11:20:55.721315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:54.342236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.570267+0000","last_change":"2024-03-21T11:20:48.667175+0000","last_active":"2024-03-21T11:20:55.570267+0000","last_peered":"2024-03-21T11:20:55.570267+0000","last_clean":"2024-03-21T11:20:55.570267+0000","last_became_active":"2024-03-21T11:20:48.666756+0000","last_became_peered":"2024-03-21T11:20:48.666756+0000","last_unstale":"2024-03-21T11:20:55.570267+0000","last_undegraded":"2024-03-21T11:20:55.570267+0000","last_fullsized":"2024-03-21T11:20:55.570267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:25:39.694699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.720882+0000","last_change":"2024-03-21T11:20:48.650584+0000","last_active":"2024-03-21T11:20:55.720882+0000","last_peered":"2024-03-21T11:20:55.720882+0000","last_clean":"2024-03-21T11:20:55.720882+0000","last_became_active":"2024-03-21T11:20:48.650251+0000","last_became_peered":"2024-03-21T11:20:48.650251+0000","last_unstale":"2024-03-21T11:20:55.720882+0000","last_undegraded":"2024-03-21T11:20:55.720882+0000","last_fullsized":"2024-03-21T11:20:55.720882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:35:59.000522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.640107+0000","last_change":"2024-03-21T11:20:48.652945+0000","last_active":"2024-03-21T11:20:55.640107+0000","last_peered":"2024-03-21T11:20:55.640107+0000","last_clean":"2024-03-21T11:20:55.640107+0000","last_became_active":"2024-03-21T11:20:48.652626+0000","last_became_peered":"2024-03-21T11:20:48.652626+0000","last_unstale":"2024-03-21T11:20:55.640107+0000","last_undegraded":"2024-03-21T11:20:55.640107+0000","last_fullsized":"2024-03-21T11:20:55.640107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:43.586907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721709+0000","last_change":"2024-03-21T11:20:48.651520+0000","last_active":"2024-03-21T11:20:55.721709+0000","last_peered":"2024-03-21T11:20:55.721709+0000","last_clean":"2024-03-21T11:20:55.721709+0000","last_became_active":"2024-03-21T11:20:48.651210+0000","last_became_peered":"2024-03-21T11:20:48.651210+0000","last_unstale":"2024-03-21T11:20:55.721709+0000","last_undegraded":"2024-03-21T11:20:55.721709+0000","last_fullsized":"2024-03-21T11:20:55.721709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:29.506732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734732+0000","last_change":"2024-03-21T11:20:48.649844+0000","last_active":"2024-03-21T11:20:54.734732+0000","last_peered":"2024-03-21T11:20:54.734732+0000","last_clean":"2024-03-21T11:20:54.734732+0000","last_became_active":"2024-03-21T11:20:48.649503+0000","last_became_peered":"2024-03-21T11:20:48.649503+0000","last_unstale":"2024-03-21T11:20:54.734732+0000","last_undegraded":"2024-03-21T11:20:54.734732+0000","last_fullsized":"2024-03-21T11:20:54.734732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:31:55.734583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.832602+0000","last_change":"2024-03-21T11:20:48.662012+0000","last_active":"2024-03-21T11:20:55.832602+0000","last_peered":"2024-03-21T11:20:55.832602+0000","last_clean":"2024-03-21T11:20:55.832602+0000","last_became_active":"2024-03-21T11:20:48.661444+0000","last_became_peered":"2024-03-21T11:20:48.661444+0000","last_unstale":"2024-03-21T11:20:55.832602+0000","last_undegraded":"2024-03-21T11:20:55.832602+0000","last_fullsized":"2024-03-21T11:20:55.832602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:52.493244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721684+0000","last_change":"2024-03-21T11:20:48.663087+0000","last_active":"2024-03-21T11:20:55.721684+0000","last_peered":"2024-03-21T11:20:55.721684+0000","last_clean":"2024-03-21T11:20:55.721684+0000","last_became_active":"2024-03-21T11:20:48.662039+0000","last_became_peered":"2024-03-21T11:20:48.662039+0000","last_unstale":"2024-03-21T11:20:55.721684+0000","last_undegraded":"2024-03-21T11:20:55.721684+0000","last_fullsized":"2024-03-21T11:20:55.721684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:47:01.962314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721686+0000","last_change":"2024-03-21T11:20:48.663478+0000","last_active":"2024-03-21T11:20:55.721686+0000","last_peered":"2024-03-21T11:20:55.721686+0000","last_clean":"2024-03-21T11:20:55.721686+0000","last_became_active":"2024-03-21T11:20:48.663135+0000","last_became_peered":"2024-03-21T11:20:48.663135+0000","last_unstale":"2024-03-21T11:20:55.721686+0000","last_undegraded":"2024-03-21T11:20:55.721686+0000","last_fullsized":"2024-03-21T11:20:55.721686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:29.139760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.720174+0000","last_change":"2024-03-21T11:20:48.653893+0000","last_active":"2024-03-21T11:20:55.720174+0000","last_peered":"2024-03-21T11:20:55.720174+0000","last_clean":"2024-03-21T11:20:55.720174+0000","last_became_active":"2024-03-21T11:20:48.653578+0000","last_became_peered":"2024-03-21T11:20:48.653578+0000","last_unstale":"2024-03-21T11:20:55.720174+0000","last_undegraded":"2024-03-21T11:20:55.720174+0000","last_fullsized":"2024-03-21T11:20:55.720174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:54:31.254030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042048+0000","last_change":"2024-03-21T11:20:48.667388+0000","last_active":"2024-03-21T11:20:55.042048+0000","last_peered":"2024-03-21T11:20:55.042048+0000","last_clean":"2024-03-21T11:20:55.042048+0000","last_became_active":"2024-03-21T11:20:48.667077+0000","last_became_peered":"2024-03-21T11:20:48.667077+0000","last_unstale":"2024-03-21T11:20:55.042048+0000","last_undegraded":"2024-03-21T11:20:55.042048+0000","last_fullsized":"2024-03-21T11:20:55.042048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:13:32.093457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.739566+0000","last_change":"2024-03-21T11:20:48.658479+0000","last_active":"2024-03-21T11:20:54.739566+0000","last_peered":"2024-03-21T11:20:54.739566+0000","last_clean":"2024-03-21T11:20:54.739566+0000","last_became_active":"2024-03-21T11:20:48.658011+0000","last_became_peered":"2024-03-21T11:20:48.658011+0000","last_unstale":"2024-03-21T11:20:54.739566+0000","last_undegraded":"2024-03-21T11:20:54.739566+0000","last_fullsized":"2024-03-21T11:20:54.739566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:33.313537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734846+0000","last_change":"2024-03-21T11:20:48.668648+0000","last_active":"2024-03-21T11:20:54.734846+0000","last_peered":"2024-03-21T11:20:54.734846+0000","last_clean":"2024-03-21T11:20:54.734846+0000","last_became_active":"2024-03-21T11:20:48.668376+0000","last_became_peered":"2024-03-21T11:20:48.668376+0000","last_unstale":"2024-03-21T11:20:54.734846+0000","last_undegraded":"2024-03-21T11:20:54.734846+0000","last_fullsized":"2024-03-21T11:20:54.734846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:49:47.601724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734412+0000","last_change":"2024-03-21T11:20:53.755600+0000","last_active":"2024-03-21T11:20:54.734412+0000","last_peered":"2024-03-21T11:20:54.734412+0000","last_clean":"2024-03-21T11:20:54.734412+0000","last_became_active":"2024-03-21T11:20:53.755321+0000","last_became_peered":"2024-03-21T11:20:53.755321+0000","last_unstale":"2024-03-21T11:20:54.734412+0000","last_undegraded":"2024-03-21T11:20:54.734412+0000","last_fullsized":"2024-03-21T11:20:54.734412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:04.905910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:16.876749+0000","last_change":"2024-03-21T11:20:48.161001+0000","last_active":"2024-03-21T11:21:16.876749+0000","last_peered":"2024-03-21T11:21:16.876749+0000","last_clean":"2024-03-21T11:21:16.876749+0000","last_became_active":"2024-03-21T11:20:48.153053+0000","last_became_peered":"2024-03-21T11:20:48.153053+0000","last_unstale":"2024-03-21T11:21:16.876749+0000","last_undegraded":"2024-03-21T11:21:16.876749+0000","last_fullsized":"2024-03-21T11:21:16.876749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:26.010581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.571618+0000","last_change":"2024-03-21T11:20:48.671430+0000","last_active":"2024-03-21T11:20:55.571618+0000","last_peered":"2024-03-21T11:20:55.571618+0000","last_clean":"2024-03-21T11:20:55.571618+0000","last_became_active":"2024-03-21T11:20:48.670192+0000","last_became_peered":"2024-03-21T11:20:48.670192+0000","last_unstale":"2024-03-21T11:20:55.571618+0000","last_undegraded":"2024-03-21T11:20:55.571618+0000","last_fullsized":"2024-03-21T11:20:55.571618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:00:53.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734304+0000","last_change":"2024-03-21T11:20:53.750313+0000","last_active":"2024-03-21T11:20:54.734304+0000","last_peered":"2024-03-21T11:20:54.734304+0000","last_clean":"2024-03-21T11:20:54.734304+0000","last_became_active":"2024-03-21T11:20:53.749913+0000","last_became_peered":"2024-03-21T11:20:53.749913+0000","last_unstale":"2024-03-21T11:20:54.734304+0000","last_undegraded":"2024-03-21T11:20:54.734304+0000","last_fullsized":"2024-03-21T11:20:54.734304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:39:09.007674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.572099+0000","last_change":"2024-03-21T11:20:48.159934+0000","last_active":"2024-03-21T11:20:55.572099+0000","last_peered":"2024-03-21T11:20:55.572099+0000","last_clean":"2024-03-21T11:20:55.572099+0000","last_became_active":"2024-03-21T11:20:48.159650+0000","last_became_peered":"2024-03-21T11:20:48.159650+0000","last_unstale":"2024-03-21T11:20:55.572099+0000","last_undegraded":"2024-03-21T11:20:55.572099+0000","last_fullsized":"2024-03-21T11:20:55.572099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:49:56.891733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.722279+0000","last_change":"2024-03-21T11:20:48.673012+0000","last_active":"2024-03-21T11:20:55.722279+0000","last_peered":"2024-03-21T11:20:55.722279+0000","last_clean":"2024-03-21T11:20:55.722279+0000","last_became_active":"2024-03-21T11:20:48.672584+0000","last_became_peered":"2024-03-21T11:20:48.672584+0000","last_unstale":"2024-03-21T11:20:55.722279+0000","last_undegraded":"2024-03-21T11:20:55.722279+0000","last_fullsized":"2024-03-21T11:20:55.722279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:48.213569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734302+0000","last_change":"2024-03-21T11:20:53.750641+0000","last_active":"2024-03-21T11:20:54.734302+0000","last_peered":"2024-03-21T11:20:54.734302+0000","last_clean":"2024-03-21T11:20:54.734302+0000","last_became_active":"2024-03-21T11:20:53.750258+0000","last_became_peered":"2024-03-21T11:20:53.750258+0000","last_unstale":"2024-03-21T11:20:54.734302+0000","last_undegraded":"2024-03-21T11:20:54.734302+0000","last_fullsized":"2024-03-21T11:20:54.734302+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:58:17.348835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:16.868273+0000","last_change":"2024-03-21T11:20:48.134383+0000","last_active":"2024-03-21T11:21:16.868273+0000","last_peered":"2024-03-21T11:21:16.868273+0000","last_clean":"2024-03-21T11:21:16.868273+0000","last_became_active":"2024-03-21T11:20:48.134012+0000","last_became_peered":"2024-03-21T11:20:48.134012+0000","last_unstale":"2024-03-21T11:21:16.868273+0000","last_undegraded":"2024-03-21T11:21:16.868273+0000","last_fullsized":"2024-03-21T11:21:16.868273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:24:47.460145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.720548+0000","last_change":"2024-03-21T11:20:48.662392+0000","last_active":"2024-03-21T11:20:55.720548+0000","last_peered":"2024-03-21T11:20:55.720548+0000","last_clean":"2024-03-21T11:20:55.720548+0000","last_became_active":"2024-03-21T11:20:48.662042+0000","last_became_peered":"2024-03-21T11:20:48.662042+0000","last_unstale":"2024-03-21T11:20:55.720548+0000","last_undegraded":"2024-03-21T11:20:55.720548+0000","last_fullsized":"2024-03-21T11:20:55.720548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:59:34.456582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734256+0000","last_change":"2024-03-21T11:20:53.749693+0000","last_active":"2024-03-21T11:20:54.734256+0000","last_peered":"2024-03-21T11:20:54.734256+0000","last_clean":"2024-03-21T11:20:54.734256+0000","last_became_active":"2024-03-21T11:20:53.749438+0000","last_became_peered":"2024-03-21T11:20:53.749438+0000","last_unstale":"2024-03-21T11:20:54.734256+0000","last_undegraded":"2024-03-21T11:20:54.734256+0000","last_fullsized":"2024-03-21T11:20:54.734256+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:37.714471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721742+0000","last_change":"2024-03-21T11:20:48.132587+0000","last_active":"2024-03-21T11:20:55.721742+0000","last_peered":"2024-03-21T11:20:55.721742+0000","last_clean":"2024-03-21T11:20:55.721742+0000","last_became_active":"2024-03-21T11:20:48.132232+0000","last_became_peered":"2024-03-21T11:20:48.132232+0000","last_unstale":"2024-03-21T11:20:55.721742+0000","last_undegraded":"2024-03-21T11:20:55.721742+0000","last_fullsized":"2024-03-21T11:20:55.721742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:20:45.217545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330584+0000","last_change":"2024-03-21T11:20:48.656881+0000","last_active":"2024-03-21T11:20:55.330584+0000","last_peered":"2024-03-21T11:20:55.330584+0000","last_clean":"2024-03-21T11:20:55.330584+0000","last_became_active":"2024-03-21T11:20:48.656426+0000","last_became_peered":"2024-03-21T11:20:48.656426+0000","last_unstale":"2024-03-21T11:20:55.330584+0000","last_undegraded":"2024-03-21T11:20:55.330584+0000","last_fullsized":"2024-03-21T11:20:55.330584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:06:27.398062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734697+0000","last_change":"2024-03-21T11:20:53.733396+0000","last_active":"2024-03-21T11:20:54.734697+0000","last_peered":"2024-03-21T11:20:54.734697+0000","last_clean":"2024-03-21T11:20:54.734697+0000","last_became_active":"2024-03-21T11:20:53.732985+0000","last_became_peered":"2024-03-21T11:20:53.732985+0000","last_unstale":"2024-03-21T11:20:54.734697+0000","last_undegraded":"2024-03-21T11:20:54.734697+0000","last_fullsized":"2024-03-21T11:20:54.734697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:54:14.064761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.837912+0000","last_change":"2024-03-21T11:20:48.133362+0000","last_active":"2024-03-21T11:21:14.837912+0000","last_peered":"2024-03-21T11:21:14.837912+0000","last_clean":"2024-03-21T11:21:14.837912+0000","last_became_active":"2024-03-21T11:20:48.132405+0000","last_became_peered":"2024-03-21T11:20:48.132405+0000","last_unstale":"2024-03-21T11:21:14.837912+0000","last_undegraded":"2024-03-21T11:21:14.837912+0000","last_fullsized":"2024-03-21T11:21:14.837912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:48:52.688948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721405+0000","last_change":"2024-03-21T11:20:48.659610+0000","last_active":"2024-03-21T11:20:55.721405+0000","last_peered":"2024-03-21T11:20:55.721405+0000","last_clean":"2024-03-21T11:20:55.721405+0000","last_became_active":"2024-03-21T11:20:48.659257+0000","last_became_peered":"2024-03-21T11:20:48.659257+0000","last_unstale":"2024-03-21T11:20:55.721405+0000","last_undegraded":"2024-03-21T11:20:55.721405+0000","last_fullsized":"2024-03-21T11:20:55.721405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:17:30.744434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734497+0000","last_change":"2024-03-21T11:20:53.750014+0000","last_active":"2024-03-21T11:20:54.734497+0000","last_peered":"2024-03-21T11:20:54.734497+0000","last_clean":"2024-03-21T11:20:54.734497+0000","last_became_active":"2024-03-21T11:20:53.749572+0000","last_became_peered":"2024-03-21T11:20:53.749572+0000","last_unstale":"2024-03-21T11:20:54.734497+0000","last_undegraded":"2024-03-21T11:20:54.734497+0000","last_fullsized":"2024-03-21T11:20:54.734497+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:29.954384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:16.874890+0000","last_change":"2024-03-21T11:20:48.142729+0000","last_active":"2024-03-21T11:21:16.874890+0000","last_peered":"2024-03-21T11:21:16.874890+0000","last_clean":"2024-03-21T11:21:16.874890+0000","last_became_active":"2024-03-21T11:20:48.142509+0000","last_became_peered":"2024-03-21T11:20:48.142509+0000","last_unstale":"2024-03-21T11:21:16.874890+0000","last_undegraded":"2024-03-21T11:21:16.874890+0000","last_fullsized":"2024-03-21T11:21:16.874890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:49:28.681159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330466+0000","last_change":"2024-03-21T11:20:48.671221+0000","last_active":"2024-03-21T11:20:55.330466+0000","last_peered":"2024-03-21T11:20:55.330466+0000","last_clean":"2024-03-21T11:20:55.330466+0000","last_became_active":"2024-03-21T11:20:48.670837+0000","last_became_peered":"2024-03-21T11:20:48.670837+0000","last_unstale":"2024-03-21T11:20:55.330466+0000","last_undegraded":"2024-03-21T11:20:55.330466+0000","last_fullsized":"2024-03-21T11:20:55.330466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:38.680409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042277+0000","last_change":"2024-03-21T11:20:53.747501+0000","last_active":"2024-03-21T11:20:55.042277+0000","last_peered":"2024-03-21T11:20:55.042277+0000","last_clean":"2024-03-21T11:20:55.042277+0000","last_became_active":"2024-03-21T11:20:53.747062+0000","last_became_peered":"2024-03-21T11:20:53.747062+0000","last_unstale":"2024-03-21T11:20:55.042277+0000","last_undegraded":"2024-03-21T11:20:55.042277+0000","last_fullsized":"2024-03-21T11:20:55.042277+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:58:36.904221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:15.854302+0000","last_change":"2024-03-21T11:20:48.156175+0000","last_active":"2024-03-21T11:21:15.854302+0000","last_peered":"2024-03-21T11:21:15.854302+0000","last_clean":"2024-03-21T11:21:15.854302+0000","last_became_active":"2024-03-21T11:20:48.154271+0000","last_became_peered":"2024-03-21T11:20:48.154271+0000","last_unstale":"2024-03-21T11:21:15.854302+0000","last_undegraded":"2024-03-21T11:21:15.854302+0000","last_fullsized":"2024-03-21T11:21:15.854302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:04.216391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.570565+0000","last_change":"2024-03-21T11:20:48.670011+0000","last_active":"2024-03-21T11:20:55.570565+0000","last_peered":"2024-03-21T11:20:55.570565+0000","last_clean":"2024-03-21T11:20:55.570565+0000","last_became_active":"2024-03-21T11:20:48.669016+0000","last_became_peered":"2024-03-21T11:20:48.669016+0000","last_unstale":"2024-03-21T11:20:55.570565+0000","last_undegraded":"2024-03-21T11:20:55.570565+0000","last_fullsized":"2024-03-21T11:20:55.570565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:43:18.906490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.723049+0000","last_change":"2024-03-21T11:20:53.751934+0000","last_active":"2024-03-21T11:20:55.723049+0000","last_peered":"2024-03-21T11:20:55.723049+0000","last_clean":"2024-03-21T11:20:55.723049+0000","last_became_active":"2024-03-21T11:20:53.751413+0000","last_became_peered":"2024-03-21T11:20:53.751413+0000","last_unstale":"2024-03-21T11:20:55.723049+0000","last_undegraded":"2024-03-21T11:20:55.723049+0000","last_fullsized":"2024-03-21T11:20:55.723049+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:12:33.880376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.837188+0000","last_change":"2024-03-21T11:20:48.131796+0000","last_active":"2024-03-21T11:21:14.837188+0000","last_peered":"2024-03-21T11:21:14.837188+0000","last_clean":"2024-03-21T11:21:14.837188+0000","last_became_active":"2024-03-21T11:20:48.131478+0000","last_became_peered":"2024-03-21T11:20:48.131478+0000","last_unstale":"2024-03-21T11:21:14.837188+0000","last_undegraded":"2024-03-21T11:21:14.837188+0000","last_fullsized":"2024-03-21T11:21:14.837188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:59:24.871300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.737515+0000","last_change":"2024-03-21T11:20:48.660046+0000","last_active":"2024-03-21T11:20:54.737515+0000","last_peered":"2024-03-21T11:20:54.737515+0000","last_clean":"2024-03-21T11:20:54.737515+0000","last_became_active":"2024-03-21T11:20:48.659621+0000","last_became_peered":"2024-03-21T11:20:48.659621+0000","last_unstale":"2024-03-21T11:20:54.737515+0000","last_undegraded":"2024-03-21T11:20:54.737515+0000","last_fullsized":"2024-03-21T11:20:54.737515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:47:01.923285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.570862+0000","last_change":"2024-03-21T11:20:53.754458+0000","last_active":"2024-03-21T11:20:55.570862+0000","last_peered":"2024-03-21T11:20:55.570862+0000","last_clean":"2024-03-21T11:20:55.570862+0000","last_became_active":"2024-03-21T11:20:53.753927+0000","last_became_peered":"2024-03-21T11:20:53.753927+0000","last_unstale":"2024-03-21T11:20:55.570862+0000","last_undegraded":"2024-03-21T11:20:55.570862+0000","last_fullsized":"2024-03-21T11:20:55.570862+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:00:47.392699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:20.607041+0000","last_change":"2024-03-21T11:20:48.154125+0000","last_active":"2024-03-21T11:21:20.607041+0000","last_peered":"2024-03-21T11:21:20.607041+0000","last_clean":"2024-03-21T11:21:20.607041+0000","last_became_active":"2024-03-21T11:20:48.153857+0000","last_became_peered":"2024-03-21T11:20:48.153857+0000","last_unstale":"2024-03-21T11:21:20.607041+0000","last_undegraded":"2024-03-21T11:21:20.607041+0000","last_fullsized":"2024-03-21T11:21:20.607041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:54:06.525335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.720901+0000","last_change":"2024-03-21T11:20:48.650143+0000","last_active":"2024-03-21T11:20:55.720901+0000","last_peered":"2024-03-21T11:20:55.720901+0000","last_clean":"2024-03-21T11:20:55.720901+0000","last_became_active":"2024-03-21T11:20:48.649825+0000","last_became_peered":"2024-03-21T11:20:48.649825+0000","last_unstale":"2024-03-21T11:20:55.720901+0000","last_undegraded":"2024-03-21T11:20:55.720901+0000","last_fullsized":"2024-03-21T11:20:55.720901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:09:15.735280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.737610+0000","last_change":"2024-03-21T11:20:53.743006+0000","last_active":"2024-03-21T11:20:54.737610+0000","last_peered":"2024-03-21T11:20:54.737610+0000","last_clean":"2024-03-21T11:20:54.737610+0000","last_became_active":"2024-03-21T11:20:53.742587+0000","last_became_peered":"2024-03-21T11:20:53.742587+0000","last_unstale":"2024-03-21T11:20:54.737610+0000","last_undegraded":"2024-03-21T11:20:54.737610+0000","last_fullsized":"2024-03-21T11:20:54.737610+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:06:06.845791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.835888+0000","last_change":"2024-03-21T11:20:48.137137+0000","last_active":"2024-03-21T11:21:14.835888+0000","last_peered":"2024-03-21T11:21:14.835888+0000","last_clean":"2024-03-21T11:21:14.835888+0000","last_became_active":"2024-03-21T11:20:48.136790+0000","last_became_peered":"2024-03-21T11:20:48.136790+0000","last_unstale":"2024-03-21T11:21:14.835888+0000","last_undegraded":"2024-03-21T11:21:14.835888+0000","last_fullsized":"2024-03-21T11:21:14.835888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:22:47.223969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.722441+0000","last_change":"2024-03-21T11:20:48.673568+0000","last_active":"2024-03-21T11:20:55.722441+0000","last_peered":"2024-03-21T11:20:55.722441+0000","last_clean":"2024-03-21T11:20:55.722441+0000","last_became_active":"2024-03-21T11:20:48.672613+0000","last_became_peered":"2024-03-21T11:20:48.672613+0000","last_unstale":"2024-03-21T11:20:55.722441+0000","last_undegraded":"2024-03-21T11:20:55.722441+0000","last_fullsized":"2024-03-21T11:20:55.722441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:43:38.193802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734470+0000","last_change":"2024-03-21T11:20:53.739228+0000","last_active":"2024-03-21T11:20:54.734470+0000","last_peered":"2024-03-21T11:20:54.734470+0000","last_clean":"2024-03-21T11:20:54.734470+0000","last_became_active":"2024-03-21T11:20:53.738750+0000","last_became_peered":"2024-03-21T11:20:53.738750+0000","last_unstale":"2024-03-21T11:20:54.734470+0000","last_undegraded":"2024-03-21T11:20:54.734470+0000","last_fullsized":"2024-03-21T11:20:54.734470+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:58:33.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.840277+0000","last_change":"2024-03-21T11:20:48.144451+0000","last_active":"2024-03-21T11:21:14.840277+0000","last_peered":"2024-03-21T11:21:14.840277+0000","last_clean":"2024-03-21T11:21:14.840277+0000","last_became_active":"2024-03-21T11:20:48.144074+0000","last_became_peered":"2024-03-21T11:20:48.144074+0000","last_unstale":"2024-03-21T11:21:14.840277+0000","last_undegraded":"2024-03-21T11:21:14.840277+0000","last_fullsized":"2024-03-21T11:21:14.840277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:33:17.032461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.720684+0000","last_change":"2024-03-21T11:20:48.654472+0000","last_active":"2024-03-21T11:20:55.720684+0000","last_peered":"2024-03-21T11:20:55.720684+0000","last_clean":"2024-03-21T11:20:55.720684+0000","last_became_active":"2024-03-21T11:20:48.654129+0000","last_became_peered":"2024-03-21T11:20:48.654129+0000","last_unstale":"2024-03-21T11:20:55.720684+0000","last_undegraded":"2024-03-21T11:20:55.720684+0000","last_fullsized":"2024-03-21T11:20:55.720684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:28:31.224296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.738529+0000","last_change":"2024-03-21T11:20:53.739994+0000","last_active":"2024-03-21T11:20:54.738529+0000","last_peered":"2024-03-21T11:20:54.738529+0000","last_clean":"2024-03-21T11:20:54.738529+0000","last_became_active":"2024-03-21T11:20:53.738319+0000","last_became_peered":"2024-03-21T11:20:53.738319+0000","last_unstale":"2024-03-21T11:20:54.738529+0000","last_undegraded":"2024-03-21T11:20:54.738529+0000","last_fullsized":"2024-03-21T11:20:54.738529+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:55.694986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'25","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:20.609592+0000","last_change":"2024-03-21T11:20:48.131461+0000","last_active":"2024-03-21T11:21:20.609592+0000","last_peered":"2024-03-21T11:21:20.609592+0000","last_clean":"2024-03-21T11:21:20.609592+0000","last_became_active":"2024-03-21T11:20:48.131079+0000","last_became_peered":"2024-03-21T11:20:48.131079+0000","last_unstale":"2024-03-21T11:21:20.609592+0000","last_undegraded":"2024-03-21T11:21:20.609592+0000","last_fullsized":"2024-03-21T11:21:20.609592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:48:57.528444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":45,"num_write":27,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721555+0000","last_change":"2024-03-21T11:20:48.653696+0000","last_active":"2024-03-21T11:20:55.721555+0000","last_peered":"2024-03-21T11:20:55.721555+0000","last_clean":"2024-03-21T11:20:55.721555+0000","last_became_active":"2024-03-21T11:20:48.653218+0000","last_became_peered":"2024-03-21T11:20:48.653218+0000","last_unstale":"2024-03-21T11:20:55.721555+0000","last_undegraded":"2024-03-21T11:20:55.721555+0000","last_fullsized":"2024-03-21T11:20:55.721555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:11:59.419040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330931+0000","last_change":"2024-03-21T11:20:53.754019+0000","last_active":"2024-03-21T11:20:55.330931+0000","last_peered":"2024-03-21T11:20:55.330931+0000","last_clean":"2024-03-21T11:20:55.330931+0000","last_became_active":"2024-03-21T11:20:53.753606+0000","last_became_peered":"2024-03-21T11:20:53.753606+0000","last_unstale":"2024-03-21T11:20:55.330931+0000","last_undegraded":"2024-03-21T11:20:55.330931+0000","last_fullsized":"2024-03-21T11:20:55.330931+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:23:27.898596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:18.601797+0000","last_change":"2024-03-21T11:20:48.153544+0000","last_active":"2024-03-21T11:21:18.601797+0000","last_peered":"2024-03-21T11:21:18.601797+0000","last_clean":"2024-03-21T11:21:18.601797+0000","last_became_active":"2024-03-21T11:20:48.153279+0000","last_became_peered":"2024-03-21T11:20:48.153279+0000","last_unstale":"2024-03-21T11:21:18.601797+0000","last_undegraded":"2024-03-21T11:21:18.601797+0000","last_fullsized":"2024-03-21T11:21:18.601797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:31:31.911844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.832250+0000","last_change":"2024-03-21T11:20:48.665140+0000","last_active":"2024-03-21T11:20:55.832250+0000","last_peered":"2024-03-21T11:20:55.832250+0000","last_clean":"2024-03-21T11:20:55.832250+0000","last_became_active":"2024-03-21T11:20:48.664226+0000","last_became_peered":"2024-03-21T11:20:48.664226+0000","last_unstale":"2024-03-21T11:20:55.832250+0000","last_undegraded":"2024-03-21T11:20:55.832250+0000","last_fullsized":"2024-03-21T11:20:55.832250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:47:38.951403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042226+0000","last_change":"2024-03-21T11:20:53.751887+0000","last_active":"2024-03-21T11:20:55.042226+0000","last_peered":"2024-03-21T11:20:55.042226+0000","last_clean":"2024-03-21T11:20:55.042226+0000","last_became_active":"2024-03-21T11:20:53.751353+0000","last_became_peered":"2024-03-21T11:20:53.751353+0000","last_unstale":"2024-03-21T11:20:55.042226+0000","last_undegraded":"2024-03-21T11:20:55.042226+0000","last_fullsized":"2024-03-21T11:20:55.042226+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:21:36.689786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.839973+0000","last_change":"2024-03-21T11:20:48.162150+0000","last_active":"2024-03-21T11:21:14.839973+0000","last_peered":"2024-03-21T11:21:14.839973+0000","last_clean":"2024-03-21T11:21:14.839973+0000","last_became_active":"2024-03-21T11:20:48.161825+0000","last_became_peered":"2024-03-21T11:20:48.161825+0000","last_unstale":"2024-03-21T11:21:14.839973+0000","last_undegraded":"2024-03-21T11:21:14.839973+0000","last_fullsized":"2024-03-21T11:21:14.839973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:10:47.097540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721494+0000","last_change":"2024-03-21T11:20:48.663856+0000","last_active":"2024-03-21T11:20:55.721494+0000","last_peered":"2024-03-21T11:20:55.721494+0000","last_clean":"2024-03-21T11:20:55.721494+0000","last_became_active":"2024-03-21T11:20:48.662908+0000","last_became_peered":"2024-03-21T11:20:48.662908+0000","last_unstale":"2024-03-21T11:20:55.721494+0000","last_undegraded":"2024-03-21T11:20:55.721494+0000","last_fullsized":"2024-03-21T11:20:55.721494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:11:12.255265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.739719+0000","last_change":"2024-03-21T11:20:53.755942+0000","last_active":"2024-03-21T11:20:54.739719+0000","last_peered":"2024-03-21T11:20:54.739719+0000","last_clean":"2024-03-21T11:20:54.739719+0000","last_became_active":"2024-03-21T11:20:53.755141+0000","last_became_peered":"2024-03-21T11:20:53.755141+0000","last_unstale":"2024-03-21T11:20:54.739719+0000","last_undegraded":"2024-03-21T11:20:54.739719+0000","last_fullsized":"2024-03-21T11:20:54.739719+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:30.141117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:16.867878+0000","last_change":"2024-03-21T11:20:48.132460+0000","last_active":"2024-03-21T11:21:16.867878+0000","last_peered":"2024-03-21T11:21:16.867878+0000","last_clean":"2024-03-21T11:21:16.867878+0000","last_became_active":"2024-03-21T11:20:48.131573+0000","last_became_peered":"2024-03-21T11:20:48.131573+0000","last_unstale":"2024-03-21T11:21:16.867878+0000","last_undegraded":"2024-03-21T11:21:16.867878+0000","last_fullsized":"2024-03-21T11:21:16.867878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:40:02.821536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721245+0000","last_change":"2024-03-21T11:20:48.656262+0000","last_active":"2024-03-21T11:20:55.721245+0000","last_peered":"2024-03-21T11:20:55.721245+0000","last_clean":"2024-03-21T11:20:55.721245+0000","last_became_active":"2024-03-21T11:20:48.655924+0000","last_became_peered":"2024-03-21T11:20:48.655924+0000","last_unstale":"2024-03-21T11:20:55.721245+0000","last_undegraded":"2024-03-21T11:20:55.721245+0000","last_fullsized":"2024-03-21T11:20:55.721245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:24:20.412185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.833712+0000","last_change":"2024-03-21T11:20:53.758666+0000","last_active":"2024-03-21T11:20:55.833712+0000","last_peered":"2024-03-21T11:20:55.833712+0000","last_clean":"2024-03-21T11:20:55.833712+0000","last_became_active":"2024-03-21T11:20:53.757398+0000","last_became_peered":"2024-03-21T11:20:53.757398+0000","last_unstale":"2024-03-21T11:20:55.833712+0000","last_undegraded":"2024-03-21T11:20:55.833712+0000","last_fullsized":"2024-03-21T11:20:55.833712+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:34.198388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'16","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:15.853646+0000","last_change":"2024-03-21T11:20:48.132725+0000","last_active":"2024-03-21T11:21:15.853646+0000","last_peered":"2024-03-21T11:21:15.853646+0000","last_clean":"2024-03-21T11:21:15.853646+0000","last_became_active":"2024-03-21T11:20:48.132367+0000","last_became_peered":"2024-03-21T11:20:48.132367+0000","last_unstale":"2024-03-21T11:21:15.853646+0000","last_undegraded":"2024-03-21T11:21:15.853646+0000","last_fullsized":"2024-03-21T11:21:15.853646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:19:18.334962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330145+0000","last_change":"2024-03-21T11:20:48.656302+0000","last_active":"2024-03-21T11:20:55.330145+0000","last_peered":"2024-03-21T11:20:55.330145+0000","last_clean":"2024-03-21T11:20:55.330145+0000","last_became_active":"2024-03-21T11:20:48.654305+0000","last_became_peered":"2024-03-21T11:20:48.654305+0000","last_unstale":"2024-03-21T11:20:55.330145+0000","last_undegraded":"2024-03-21T11:20:55.330145+0000","last_fullsized":"2024-03-21T11:20:55.330145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:59:40.766829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330319+0000","last_change":"2024-03-21T11:20:53.750710+0000","last_active":"2024-03-21T11:20:55.330319+0000","last_peered":"2024-03-21T11:20:55.330319+0000","last_clean":"2024-03-21T11:20:55.330319+0000","last_became_active":"2024-03-21T11:20:53.750255+0000","last_became_peered":"2024-03-21T11:20:53.750255+0000","last_unstale":"2024-03-21T11:20:55.330319+0000","last_undegraded":"2024-03-21T11:20:55.330319+0000","last_fullsized":"2024-03-21T11:20:55.330319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:36:28.611838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721788+0000","last_change":"2024-03-21T11:20:48.133524+0000","last_active":"2024-03-21T11:20:55.721788+0000","last_peered":"2024-03-21T11:20:55.721788+0000","last_clean":"2024-03-21T11:20:55.721788+0000","last_became_active":"2024-03-21T11:20:48.133186+0000","last_became_peered":"2024-03-21T11:20:48.133186+0000","last_unstale":"2024-03-21T11:20:55.721788+0000","last_undegraded":"2024-03-21T11:20:55.721788+0000","last_fullsized":"2024-03-21T11:20:55.721788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:02.018639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.041999+0000","last_change":"2024-03-21T11:20:48.654126+0000","last_active":"2024-03-21T11:20:55.041999+0000","last_peered":"2024-03-21T11:20:55.041999+0000","last_clean":"2024-03-21T11:20:55.041999+0000","last_became_active":"2024-03-21T11:20:48.653835+0000","last_became_peered":"2024-03-21T11:20:48.653835+0000","last_unstale":"2024-03-21T11:20:55.041999+0000","last_undegraded":"2024-03-21T11:20:55.041999+0000","last_fullsized":"2024-03-21T11:20:55.041999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:12:03.332076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721501+0000","last_change":"2024-03-21T11:20:53.916645+0000","last_active":"2024-03-21T11:20:55.721501+0000","last_peered":"2024-03-21T11:20:55.721501+0000","last_clean":"2024-03-21T11:20:55.721501+0000","last_became_active":"2024-03-21T11:20:53.916178+0000","last_became_peered":"2024-03-21T11:20:53.916178+0000","last_unstale":"2024-03-21T11:20:55.721501+0000","last_undegraded":"2024-03-21T11:20:55.721501+0000","last_fullsized":"2024-03-21T11:20:55.721501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:56:15.805231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.738813+0000","last_change":"2024-03-21T11:20:48.159877+0000","last_active":"2024-03-21T11:20:54.738813+0000","last_peered":"2024-03-21T11:20:54.738813+0000","last_clean":"2024-03-21T11:20:54.738813+0000","last_became_active":"2024-03-21T11:20:48.159529+0000","last_became_peered":"2024-03-21T11:20:48.159529+0000","last_unstale":"2024-03-21T11:20:54.738813+0000","last_undegraded":"2024-03-21T11:20:54.738813+0000","last_fullsized":"2024-03-21T11:20:54.738813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:17:09.662462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.640028+0000","last_change":"2024-03-21T11:20:48.663311+0000","last_active":"2024-03-21T11:20:55.640028+0000","last_peered":"2024-03-21T11:20:55.640028+0000","last_clean":"2024-03-21T11:20:55.640028+0000","last_became_active":"2024-03-21T11:20:48.662961+0000","last_became_peered":"2024-03-21T11:20:48.662961+0000","last_unstale":"2024-03-21T11:20:55.640028+0000","last_undegraded":"2024-03-21T11:20:55.640028+0000","last_fullsized":"2024-03-21T11:20:55.640028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:14:46.616326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.833090+0000","last_change":"2024-03-21T11:20:53.751804+0000","last_active":"2024-03-21T11:20:55.833090+0000","last_peered":"2024-03-21T11:20:55.833090+0000","last_clean":"2024-03-21T11:20:55.833090+0000","last_became_active":"2024-03-21T11:20:53.751323+0000","last_became_peered":"2024-03-21T11:20:53.751323+0000","last_unstale":"2024-03-21T11:20:55.833090+0000","last_undegraded":"2024-03-21T11:20:55.833090+0000","last_fullsized":"2024-03-21T11:20:55.833090+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:38:25.677345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721346+0000","last_change":"2024-03-21T11:20:48.147580+0000","last_active":"2024-03-21T11:20:55.721346+0000","last_peered":"2024-03-21T11:20:55.721346+0000","last_clean":"2024-03-21T11:20:55.721346+0000","last_became_active":"2024-03-21T11:20:48.147222+0000","last_became_peered":"2024-03-21T11:20:48.147222+0000","last_unstale":"2024-03-21T11:20:55.721346+0000","last_undegraded":"2024-03-21T11:20:55.721346+0000","last_fullsized":"2024-03-21T11:20:55.721346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:49:04.204129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.639876+0000","last_change":"2024-03-21T11:20:48.667940+0000","last_active":"2024-03-21T11:20:55.639876+0000","last_peered":"2024-03-21T11:20:55.639876+0000","last_clean":"2024-03-21T11:20:55.639876+0000","last_became_active":"2024-03-21T11:20:48.667595+0000","last_became_peered":"2024-03-21T11:20:48.667595+0000","last_unstale":"2024-03-21T11:20:55.639876+0000","last_undegraded":"2024-03-21T11:20:55.639876+0000","last_fullsized":"2024-03-21T11:20:55.639876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:08:14.007589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.833727+0000","last_change":"2024-03-21T11:20:53.916931+0000","last_active":"2024-03-21T11:20:55.833727+0000","last_peered":"2024-03-21T11:20:55.833727+0000","last_clean":"2024-03-21T11:20:55.833727+0000","last_became_active":"2024-03-21T11:20:53.916495+0000","last_became_peered":"2024-03-21T11:20:53.916495+0000","last_unstale":"2024-03-21T11:20:55.833727+0000","last_undegraded":"2024-03-21T11:20:55.833727+0000","last_fullsized":"2024-03-21T11:20:55.833727+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:18:33.046052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330279+0000","last_change":"2024-03-21T11:20:48.652997+0000","last_active":"2024-03-21T11:20:55.330279+0000","last_peered":"2024-03-21T11:20:55.330279+0000","last_clean":"2024-03-21T11:20:55.330279+0000","last_became_active":"2024-03-21T11:20:48.652680+0000","last_became_peered":"2024-03-21T11:20:48.652680+0000","last_unstale":"2024-03-21T11:20:55.330279+0000","last_undegraded":"2024-03-21T11:20:55.330279+0000","last_fullsized":"2024-03-21T11:20:55.330279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:00:19.516607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:15.858857+0000","last_change":"2024-03-21T11:20:48.164607+0000","last_active":"2024-03-21T11:21:15.858857+0000","last_peered":"2024-03-21T11:21:15.858857+0000","last_clean":"2024-03-21T11:21:15.858857+0000","last_became_active":"2024-03-21T11:20:48.164222+0000","last_became_peered":"2024-03-21T11:20:48.164222+0000","last_unstale":"2024-03-21T11:21:15.858857+0000","last_undegraded":"2024-03-21T11:21:15.858857+0000","last_fullsized":"2024-03-21T11:21:15.858857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:52:24.359155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721577+0000","last_change":"2024-03-21T11:20:53.733695+0000","last_active":"2024-03-21T11:20:55.721577+0000","last_peered":"2024-03-21T11:20:55.721577+0000","last_clean":"2024-03-21T11:20:55.721577+0000","last_became_active":"2024-03-21T11:20:53.733246+0000","last_became_peered":"2024-03-21T11:20:53.733246+0000","last_unstale":"2024-03-21T11:20:55.721577+0000","last_undegraded":"2024-03-21T11:20:55.721577+0000","last_fullsized":"2024-03-21T11:20:55.721577+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:16:17.689387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.570582+0000","last_change":"2024-03-21T11:20:48.667212+0000","last_active":"2024-03-21T11:20:55.570582+0000","last_peered":"2024-03-21T11:20:55.570582+0000","last_clean":"2024-03-21T11:20:55.570582+0000","last_became_active":"2024-03-21T11:20:48.662839+0000","last_became_peered":"2024-03-21T11:20:48.662839+0000","last_unstale":"2024-03-21T11:20:55.570582+0000","last_undegraded":"2024-03-21T11:20:55.570582+0000","last_fullsized":"2024-03-21T11:20:55.570582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:27:41.490922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:16.873914+0000","last_change":"2024-03-21T11:20:48.151185+0000","last_active":"2024-03-21T11:21:16.873914+0000","last_peered":"2024-03-21T11:21:16.873914+0000","last_clean":"2024-03-21T11:21:16.873914+0000","last_became_active":"2024-03-21T11:20:48.150859+0000","last_became_peered":"2024-03-21T11:20:48.150859+0000","last_unstale":"2024-03-21T11:21:16.873914+0000","last_undegraded":"2024-03-21T11:21:16.873914+0000","last_fullsized":"2024-03-21T11:21:16.873914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:52:45.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721118+0000","last_change":"2024-03-21T11:19:28.916140+0000","last_active":"2024-03-21T11:20:55.721118+0000","last_peered":"2024-03-21T11:20:55.721118+0000","last_clean":"2024-03-21T11:20:55.721118+0000","last_became_active":"2024-03-21T11:19:28.915634+0000","last_became_peered":"2024-03-21T11:19:28.915634+0000","last_unstale":"2024-03-21T11:20:55.721118+0000","last_undegraded":"2024-03-21T11:20:55.721118+0000","last_fullsized":"2024-03-21T11:20:55.721118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:15:35.803515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:15:35.803515+0000","last_clean_scrub_stamp":"2024-03-21T11:15:35.803515+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:08:31.509894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330760+0000","last_change":"2024-03-21T11:20:53.917119+0000","last_active":"2024-03-21T11:20:55.330760+0000","last_peered":"2024-03-21T11:20:55.330760+0000","last_clean":"2024-03-21T11:20:55.330760+0000","last_became_active":"2024-03-21T11:20:53.916213+0000","last_became_peered":"2024-03-21T11:20:53.916213+0000","last_unstale":"2024-03-21T11:20:55.330760+0000","last_undegraded":"2024-03-21T11:20:55.330760+0000","last_fullsized":"2024-03-21T11:20:55.330760+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:48:50.874202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.831939+0000","last_change":"2024-03-21T11:20:48.657878+0000","last_active":"2024-03-21T11:20:55.831939+0000","last_peered":"2024-03-21T11:20:55.831939+0000","last_clean":"2024-03-21T11:20:55.831939+0000","last_became_active":"2024-03-21T11:20:48.657212+0000","last_became_peered":"2024-03-21T11:20:48.657212+0000","last_unstale":"2024-03-21T11:20:55.831939+0000","last_undegraded":"2024-03-21T11:20:55.831939+0000","last_fullsized":"2024-03-21T11:20:55.831939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:43:11.993577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:15.850471+0000","last_change":"2024-03-21T11:20:48.135513+0000","last_active":"2024-03-21T11:21:15.850471+0000","last_peered":"2024-03-21T11:21:15.850471+0000","last_clean":"2024-03-21T11:21:15.850471+0000","last_became_active":"2024-03-21T11:20:48.135223+0000","last_became_peered":"2024-03-21T11:20:48.135223+0000","last_unstale":"2024-03-21T11:21:15.850471+0000","last_undegraded":"2024-03-21T11:21:15.850471+0000","last_fullsized":"2024-03-21T11:21:15.850471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:37:38.834284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042119+0000","last_change":"2024-03-21T11:20:53.751790+0000","last_active":"2024-03-21T11:20:55.042119+0000","last_peered":"2024-03-21T11:20:55.042119+0000","last_clean":"2024-03-21T11:20:55.042119+0000","last_became_active":"2024-03-21T11:20:53.751312+0000","last_became_peered":"2024-03-21T11:20:53.751312+0000","last_unstale":"2024-03-21T11:20:55.042119+0000","last_undegraded":"2024-03-21T11:20:55.042119+0000","last_fullsized":"2024-03-21T11:20:55.042119+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:21:41.599985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:16.876078+0000","last_change":"2024-03-21T11:20:48.154119+0000","last_active":"2024-03-21T11:21:16.876078+0000","last_peered":"2024-03-21T11:21:16.876078+0000","last_clean":"2024-03-21T11:21:16.876078+0000","last_became_active":"2024-03-21T11:20:48.153781+0000","last_became_peered":"2024-03-21T11:20:48.153781+0000","last_unstale":"2024-03-21T11:21:16.876078+0000","last_undegraded":"2024-03-21T11:21:16.876078+0000","last_fullsized":"2024-03-21T11:21:16.876078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:44:13.937836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.833154+0000","last_change":"2024-03-21T11:20:48.661282+0000","last_active":"2024-03-21T11:20:55.833154+0000","last_peered":"2024-03-21T11:20:55.833154+0000","last_clean":"2024-03-21T11:20:55.833154+0000","last_became_active":"2024-03-21T11:20:48.660569+0000","last_became_peered":"2024-03-21T11:20:48.660569+0000","last_unstale":"2024-03-21T11:20:55.833154+0000","last_undegraded":"2024-03-21T11:20:55.833154+0000","last_fullsized":"2024-03-21T11:20:55.833154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:14.875700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.570710+0000","last_change":"2024-03-21T11:20:53.918016+0000","last_active":"2024-03-21T11:20:55.570710+0000","last_peered":"2024-03-21T11:20:55.570710+0000","last_clean":"2024-03-21T11:20:55.570710+0000","last_became_active":"2024-03-21T11:20:53.916891+0000","last_became_peered":"2024-03-21T11:20:53.916891+0000","last_unstale":"2024-03-21T11:20:55.570710+0000","last_undegraded":"2024-03-21T11:20:55.570710+0000","last_fullsized":"2024-03-21T11:20:55.570710+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:40:25.206778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:16.877158+0000","last_change":"2024-03-21T11:20:48.167560+0000","last_active":"2024-03-21T11:21:16.877158+0000","last_peered":"2024-03-21T11:21:16.877158+0000","last_clean":"2024-03-21T11:21:16.877158+0000","last_became_active":"2024-03-21T11:20:48.164986+0000","last_became_peered":"2024-03-21T11:20:48.164986+0000","last_unstale":"2024-03-21T11:21:16.877158+0000","last_undegraded":"2024-03-21T11:21:16.877158+0000","last_fullsized":"2024-03-21T11:21:16.877158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:59.930626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.571946+0000","last_change":"2024-03-21T11:20:48.675250+0000","last_active":"2024-03-21T11:20:55.571946+0000","last_peered":"2024-03-21T11:20:55.571946+0000","last_clean":"2024-03-21T11:20:55.571946+0000","last_became_active":"2024-03-21T11:20:48.674638+0000","last_became_peered":"2024-03-21T11:20:48.674638+0000","last_unstale":"2024-03-21T11:20:55.571946+0000","last_undegraded":"2024-03-21T11:20:55.571946+0000","last_fullsized":"2024-03-21T11:20:55.571946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:46:34.842363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734240+0000","last_change":"2024-03-21T11:20:53.733809+0000","last_active":"2024-03-21T11:20:54.734240+0000","last_peered":"2024-03-21T11:20:54.734240+0000","last_clean":"2024-03-21T11:20:54.734240+0000","last_became_active":"2024-03-21T11:20:53.733393+0000","last_became_peered":"2024-03-21T11:20:53.733393+0000","last_unstale":"2024-03-21T11:20:54.734240+0000","last_undegraded":"2024-03-21T11:20:54.734240+0000","last_fullsized":"2024-03-21T11:20:54.734240+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:02:46.270716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.837529+0000","last_change":"2024-03-21T11:20:48.139475+0000","last_active":"2024-03-21T11:21:14.837529+0000","last_peered":"2024-03-21T11:21:14.837529+0000","last_clean":"2024-03-21T11:21:14.837529+0000","last_became_active":"2024-03-21T11:20:48.139249+0000","last_became_peered":"2024-03-21T11:20:48.139249+0000","last_unstale":"2024-03-21T11:21:14.837529+0000","last_undegraded":"2024-03-21T11:21:14.837529+0000","last_fullsized":"2024-03-21T11:21:14.837529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:45:18.811829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.722997+0000","last_change":"2024-03-21T11:20:48.673975+0000","last_active":"2024-03-21T11:20:55.722997+0000","last_peered":"2024-03-21T11:20:55.722997+0000","last_clean":"2024-03-21T11:20:55.722997+0000","last_became_active":"2024-03-21T11:20:48.673587+0000","last_became_peered":"2024-03-21T11:20:48.673587+0000","last_unstale":"2024-03-21T11:20:55.722997+0000","last_undegraded":"2024-03-21T11:20:55.722997+0000","last_fullsized":"2024-03-21T11:20:55.722997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:08:52.354780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734877+0000","last_change":"2024-03-21T11:20:53.917493+0000","last_active":"2024-03-21T11:20:54.734877+0000","last_peered":"2024-03-21T11:20:54.734877+0000","last_clean":"2024-03-21T11:20:54.734877+0000","last_became_active":"2024-03-21T11:20:53.917007+0000","last_became_peered":"2024-03-21T11:20:53.917007+0000","last_unstale":"2024-03-21T11:20:54.734877+0000","last_undegraded":"2024-03-21T11:20:54.734877+0000","last_fullsized":"2024-03-21T11:20:54.734877+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:39:20.280899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.840422+0000","last_change":"2024-03-21T11:20:48.140914+0000","last_active":"2024-03-21T11:21:14.840422+0000","last_peered":"2024-03-21T11:21:14.840422+0000","last_clean":"2024-03-21T11:21:14.840422+0000","last_became_active":"2024-03-21T11:20:48.140558+0000","last_became_peered":"2024-03-21T11:20:48.140558+0000","last_unstale":"2024-03-21T11:21:14.840422+0000","last_undegraded":"2024-03-21T11:21:14.840422+0000","last_fullsized":"2024-03-21T11:21:14.840422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:59:52.065431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721230+0000","last_change":"2024-03-21T11:20:48.651190+0000","last_active":"2024-03-21T11:20:55.721230+0000","last_peered":"2024-03-21T11:20:55.721230+0000","last_clean":"2024-03-21T11:20:55.721230+0000","last_became_active":"2024-03-21T11:20:48.650818+0000","last_became_peered":"2024-03-21T11:20:48.650818+0000","last_unstale":"2024-03-21T11:20:55.721230+0000","last_undegraded":"2024-03-21T11:20:55.721230+0000","last_fullsized":"2024-03-21T11:20:55.721230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:47:17.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.739717+0000","last_change":"2024-03-21T11:20:53.756844+0000","last_active":"2024-03-21T11:20:54.739717+0000","last_peered":"2024-03-21T11:20:54.739717+0000","last_clean":"2024-03-21T11:20:54.739717+0000","last_became_active":"2024-03-21T11:20:53.756380+0000","last_became_peered":"2024-03-21T11:20:53.756380+0000","last_unstale":"2024-03-21T11:20:54.739717+0000","last_undegraded":"2024-03-21T11:20:54.739717+0000","last_fullsized":"2024-03-21T11:20:54.739717+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:51:55.327798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.840519+0000","last_change":"2024-03-21T11:20:48.160749+0000","last_active":"2024-03-21T11:21:14.840519+0000","last_peered":"2024-03-21T11:21:14.840519+0000","last_clean":"2024-03-21T11:21:14.840519+0000","last_became_active":"2024-03-21T11:20:48.158507+0000","last_became_peered":"2024-03-21T11:20:48.158507+0000","last_unstale":"2024-03-21T11:21:14.840519+0000","last_undegraded":"2024-03-21T11:21:14.840519+0000","last_fullsized":"2024-03-21T11:21:14.840519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:55.375529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.041775+0000","last_change":"2024-03-21T11:20:48.667153+0000","last_active":"2024-03-21T11:20:55.041775+0000","last_peered":"2024-03-21T11:20:55.041775+0000","last_clean":"2024-03-21T11:20:55.041775+0000","last_became_active":"2024-03-21T11:20:48.666823+0000","last_became_peered":"2024-03-21T11:20:48.666823+0000","last_unstale":"2024-03-21T11:20:55.041775+0000","last_undegraded":"2024-03-21T11:20:55.041775+0000","last_fullsized":"2024-03-21T11:20:55.041775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:43:54.183372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734557+0000","last_change":"2024-03-21T11:20:53.739188+0000","last_active":"2024-03-21T11:20:54.734557+0000","last_peered":"2024-03-21T11:20:54.734557+0000","last_clean":"2024-03-21T11:20:54.734557+0000","last_became_active":"2024-03-21T11:20:53.738770+0000","last_became_peered":"2024-03-21T11:20:53.738770+0000","last_unstale":"2024-03-21T11:20:54.734557+0000","last_undegraded":"2024-03-21T11:20:54.734557+0000","last_fullsized":"2024-03-21T11:20:54.734557+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:53:48.897963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:15.833224+0000","last_change":"2024-03-21T11:20:48.162279+0000","last_active":"2024-03-21T11:21:15.833224+0000","last_peered":"2024-03-21T11:21:15.833224+0000","last_clean":"2024-03-21T11:21:15.833224+0000","last_became_active":"2024-03-21T11:20:48.161878+0000","last_became_peered":"2024-03-21T11:20:48.161878+0000","last_unstale":"2024-03-21T11:21:15.833224+0000","last_undegraded":"2024-03-21T11:21:15.833224+0000","last_fullsized":"2024-03-21T11:21:15.833224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:34:15.144710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.571021+0000","last_change":"2024-03-21T11:20:48.677303+0000","last_active":"2024-03-21T11:20:55.571021+0000","last_peered":"2024-03-21T11:20:55.571021+0000","last_clean":"2024-03-21T11:20:55.571021+0000","last_became_active":"2024-03-21T11:20:48.676903+0000","last_became_peered":"2024-03-21T11:20:48.676903+0000","last_unstale":"2024-03-21T11:20:55.571021+0000","last_undegraded":"2024-03-21T11:20:55.571021+0000","last_fullsized":"2024-03-21T11:20:55.571021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:06:59.922274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734605+0000","last_change":"2024-03-21T11:20:53.916691+0000","last_active":"2024-03-21T11:20:54.734605+0000","last_peered":"2024-03-21T11:20:54.734605+0000","last_clean":"2024-03-21T11:20:54.734605+0000","last_became_active":"2024-03-21T11:20:53.916275+0000","last_became_peered":"2024-03-21T11:20:53.916275+0000","last_unstale":"2024-03-21T11:20:54.734605+0000","last_undegraded":"2024-03-21T11:20:54.734605+0000","last_fullsized":"2024-03-21T11:20:54.734605+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:35.848589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:14.835756+0000","last_change":"2024-03-21T11:20:48.134292+0000","last_active":"2024-03-21T11:21:14.835756+0000","last_peered":"2024-03-21T11:21:14.835756+0000","last_clean":"2024-03-21T11:21:14.835756+0000","last_became_active":"2024-03-21T11:20:48.133957+0000","last_became_peered":"2024-03-21T11:20:48.133957+0000","last_unstale":"2024-03-21T11:21:14.835756+0000","last_undegraded":"2024-03-21T11:21:14.835756+0000","last_fullsized":"2024-03-21T11:21:14.835756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:19:06.066423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721338+0000","last_change":"2024-03-21T11:20:48.661486+0000","last_active":"2024-03-21T11:20:55.721338+0000","last_peered":"2024-03-21T11:20:55.721338+0000","last_clean":"2024-03-21T11:20:55.721338+0000","last_became_active":"2024-03-21T11:20:48.661109+0000","last_became_peered":"2024-03-21T11:20:48.661109+0000","last_unstale":"2024-03-21T11:20:55.721338+0000","last_undegraded":"2024-03-21T11:20:55.721338+0000","last_fullsized":"2024-03-21T11:20:55.721338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:15:27.093616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.720414+0000","last_change":"2024-03-21T11:20:53.747223+0000","last_active":"2024-03-21T11:20:55.720414+0000","last_peered":"2024-03-21T11:20:55.720414+0000","last_clean":"2024-03-21T11:20:55.720414+0000","last_became_active":"2024-03-21T11:20:53.746805+0000","last_became_peered":"2024-03-21T11:20:53.746805+0000","last_unstale":"2024-03-21T11:20:55.720414+0000","last_undegraded":"2024-03-21T11:20:55.720414+0000","last_fullsized":"2024-03-21T11:20:55.720414+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:29.186745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:21:18.604767+0000","last_change":"2024-03-21T11:20:48.151172+0000","last_active":"2024-03-21T11:21:18.604767+0000","last_peered":"2024-03-21T11:21:18.604767+0000","last_clean":"2024-03-21T11:21:18.604767+0000","last_became_active":"2024-03-21T11:20:48.150832+0000","last_became_peered":"2024-03-21T11:20:48.150832+0000","last_unstale":"2024-03-21T11:21:18.604767+0000","last_undegraded":"2024-03-21T11:21:18.604767+0000","last_fullsized":"2024-03-21T11:21:18.604767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:34:46.367713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721772+0000","last_change":"2024-03-21T11:20:48.652275+0000","last_active":"2024-03-21T11:20:55.721772+0000","last_peered":"2024-03-21T11:20:55.721772+0000","last_clean":"2024-03-21T11:20:55.721772+0000","last_became_active":"2024-03-21T11:20:48.651965+0000","last_became_peered":"2024-03-21T11:20:48.651965+0000","last_unstale":"2024-03-21T11:20:55.721772+0000","last_undegraded":"2024-03-21T11:20:55.721772+0000","last_fullsized":"2024-03-21T11:20:55.721772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:56:18.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721946+0000","last_change":"2024-03-21T11:20:53.748050+0000","last_active":"2024-03-21T11:20:55.721946+0000","last_peered":"2024-03-21T11:20:55.721946+0000","last_clean":"2024-03-21T11:20:55.721946+0000","last_became_active":"2024-03-21T11:20:53.747636+0000","last_became_peered":"2024-03-21T11:20:53.747636+0000","last_unstale":"2024-03-21T11:20:55.721946+0000","last_undegraded":"2024-03-21T11:20:55.721946+0000","last_fullsized":"2024-03-21T11:20:55.721946+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:09:15.202183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330698+0000","last_change":"2024-03-21T11:20:48.156928+0000","last_active":"2024-03-21T11:20:55.330698+0000","last_peered":"2024-03-21T11:20:55.330698+0000","last_clean":"2024-03-21T11:20:55.330698+0000","last_became_active":"2024-03-21T11:20:48.156625+0000","last_became_peered":"2024-03-21T11:20:48.156625+0000","last_unstale":"2024-03-21T11:20:55.330698+0000","last_undegraded":"2024-03-21T11:20:55.330698+0000","last_fullsized":"2024-03-21T11:20:55.330698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:19:42.035565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721547+0000","last_change":"2024-03-21T11:20:48.663655+0000","last_active":"2024-03-21T11:20:55.721547+0000","last_peered":"2024-03-21T11:20:55.721547+0000","last_clean":"2024-03-21T11:20:55.721547+0000","last_became_active":"2024-03-21T11:20:48.663336+0000","last_became_peered":"2024-03-21T11:20:48.663336+0000","last_unstale":"2024-03-21T11:20:55.721547+0000","last_undegraded":"2024-03-21T11:20:55.721547+0000","last_fullsized":"2024-03-21T11:20:55.721547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:29:05.119283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.329990+0000","last_change":"2024-03-21T11:20:53.732584+0000","last_active":"2024-03-21T11:20:55.329990+0000","last_peered":"2024-03-21T11:20:55.329990+0000","last_clean":"2024-03-21T11:20:55.329990+0000","last_became_active":"2024-03-21T11:20:53.731975+0000","last_became_peered":"2024-03-21T11:20:53.731975+0000","last_unstale":"2024-03-21T11:20:55.329990+0000","last_undegraded":"2024-03-21T11:20:55.329990+0000","last_fullsized":"2024-03-21T11:20:55.329990+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:51:06.336241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.833544+0000","last_change":"2024-03-21T11:20:48.165594+0000","last_active":"2024-03-21T11:20:55.833544+0000","last_peered":"2024-03-21T11:20:55.833544+0000","last_clean":"2024-03-21T11:20:55.833544+0000","last_became_active":"2024-03-21T11:20:48.164394+0000","last_became_peered":"2024-03-21T11:20:48.164394+0000","last_unstale":"2024-03-21T11:20:55.833544+0000","last_undegraded":"2024-03-21T11:20:55.833544+0000","last_fullsized":"2024-03-21T11:20:55.833544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:13.317767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.721612+0000","last_change":"2024-03-21T11:20:48.649359+0000","last_active":"2024-03-21T11:20:55.721612+0000","last_peered":"2024-03-21T11:20:55.721612+0000","last_clean":"2024-03-21T11:20:55.721612+0000","last_became_active":"2024-03-21T11:20:48.649031+0000","last_became_peered":"2024-03-21T11:20:48.649031+0000","last_unstale":"2024-03-21T11:20:55.721612+0000","last_undegraded":"2024-03-21T11:20:55.721612+0000","last_fullsized":"2024-03-21T11:20:55.721612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:17.055732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042041+0000","last_change":"2024-03-21T11:20:48.654999+0000","last_active":"2024-03-21T11:20:55.042041+0000","last_peered":"2024-03-21T11:20:55.042041+0000","last_clean":"2024-03-21T11:20:55.042041+0000","last_became_active":"2024-03-21T11:20:48.654638+0000","last_became_peered":"2024-03-21T11:20:48.654638+0000","last_unstale":"2024-03-21T11:20:55.042041+0000","last_undegraded":"2024-03-21T11:20:55.042041+0000","last_fullsized":"2024-03-21T11:20:55.042041+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:49:48.711420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734678+0000","last_change":"2024-03-21T11:20:48.651864+0000","last_active":"2024-03-21T11:20:54.734678+0000","last_peered":"2024-03-21T11:20:54.734678+0000","last_clean":"2024-03-21T11:20:54.734678+0000","last_became_active":"2024-03-21T11:20:48.651565+0000","last_became_peered":"2024-03-21T11:20:48.651565+0000","last_unstale":"2024-03-21T11:20:54.734678+0000","last_undegraded":"2024-03-21T11:20:54.734678+0000","last_fullsized":"2024-03-21T11:20:54.734678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:58.659889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734266+0000","last_change":"2024-03-21T11:20:48.667038+0000","last_active":"2024-03-21T11:20:54.734266+0000","last_peered":"2024-03-21T11:20:54.734266+0000","last_clean":"2024-03-21T11:20:54.734266+0000","last_became_active":"2024-03-21T11:20:48.666570+0000","last_became_peered":"2024-03-21T11:20:48.666570+0000","last_unstale":"2024-03-21T11:20:54.734266+0000","last_undegraded":"2024-03-21T11:20:54.734266+0000","last_fullsized":"2024-03-21T11:20:54.734266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:36:39.043697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.722507+0000","last_change":"2024-03-21T11:20:48.676216+0000","last_active":"2024-03-21T11:20:55.722507+0000","last_peered":"2024-03-21T11:20:55.722507+0000","last_clean":"2024-03-21T11:20:55.722507+0000","last_became_active":"2024-03-21T11:20:48.675269+0000","last_became_peered":"2024-03-21T11:20:48.675269+0000","last_unstale":"2024-03-21T11:20:55.722507+0000","last_undegraded":"2024-03-21T11:20:55.722507+0000","last_fullsized":"2024-03-21T11:20:55.722507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:42:47.006371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.833311+0000","last_change":"2024-03-21T11:20:48.670139+0000","last_active":"2024-03-21T11:20:55.833311+0000","last_peered":"2024-03-21T11:20:55.833311+0000","last_clean":"2024-03-21T11:20:55.833311+0000","last_became_active":"2024-03-21T11:20:48.669410+0000","last_became_peered":"2024-03-21T11:20:48.669410+0000","last_unstale":"2024-03-21T11:20:55.833311+0000","last_undegraded":"2024-03-21T11:20:55.833311+0000","last_fullsized":"2024-03-21T11:20:55.833311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:35:18.799840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042314+0000","last_change":"2024-03-21T11:20:48.655356+0000","last_active":"2024-03-21T11:20:55.042314+0000","last_peered":"2024-03-21T11:20:55.042314+0000","last_clean":"2024-03-21T11:20:55.042314+0000","last_became_active":"2024-03-21T11:20:48.655028+0000","last_became_peered":"2024-03-21T11:20:48.655028+0000","last_unstale":"2024-03-21T11:20:55.042314+0000","last_undegraded":"2024-03-21T11:20:55.042314+0000","last_fullsized":"2024-03-21T11:20:55.042314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:46:39.948982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:54.734834+0000","last_change":"2024-03-21T11:20:48.648840+0000","last_active":"2024-03-21T11:20:54.734834+0000","last_peered":"2024-03-21T11:20:54.734834+0000","last_clean":"2024-03-21T11:20:54.734834+0000","last_became_active":"2024-03-21T11:20:48.648502+0000","last_became_peered":"2024-03-21T11:20:48.648502+0000","last_unstale":"2024-03-21T11:20:54.734834+0000","last_undegraded":"2024-03-21T11:20:54.734834+0000","last_fullsized":"2024-03-21T11:20:54.734834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:33:14.824880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.572065+0000","last_change":"2024-03-21T11:20:48.672015+0000","last_active":"2024-03-21T11:20:55.572065+0000","last_peered":"2024-03-21T11:20:55.572065+0000","last_clean":"2024-03-21T11:20:55.572065+0000","last_became_active":"2024-03-21T11:20:48.671663+0000","last_became_peered":"2024-03-21T11:20:48.671663+0000","last_unstale":"2024-03-21T11:20:55.572065+0000","last_undegraded":"2024-03-21T11:20:55.572065+0000","last_fullsized":"2024-03-21T11:20:55.572065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:21:25.328698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.330140+0000","last_change":"2024-03-21T11:20:48.664114+0000","last_active":"2024-03-21T11:20:55.330140+0000","last_peered":"2024-03-21T11:20:55.330140+0000","last_clean":"2024-03-21T11:20:55.330140+0000","last_became_active":"2024-03-21T11:20:48.663819+0000","last_became_peered":"2024-03-21T11:20:48.663819+0000","last_unstale":"2024-03-21T11:20:55.330140+0000","last_undegraded":"2024-03-21T11:20:55.330140+0000","last_fullsized":"2024-03-21T11:20:55.330140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:08:08.201383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.570381+0000","last_change":"2024-03-21T11:20:48.671336+0000","last_active":"2024-03-21T11:20:55.570381+0000","last_peered":"2024-03-21T11:20:55.570381+0000","last_clean":"2024-03-21T11:20:55.570381+0000","last_became_active":"2024-03-21T11:20:48.670918+0000","last_became_peered":"2024-03-21T11:20:48.670918+0000","last_unstale":"2024-03-21T11:20:55.570381+0000","last_undegraded":"2024-03-21T11:20:55.570381+0000","last_fullsized":"2024-03-21T11:20:55.570381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:01:44.764584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.720534+0000","last_change":"2024-03-21T11:20:48.659339+0000","last_active":"2024-03-21T11:20:55.720534+0000","last_peered":"2024-03-21T11:20:55.720534+0000","last_clean":"2024-03-21T11:20:55.720534+0000","last_became_active":"2024-03-21T11:20:48.658989+0000","last_became_peered":"2024-03-21T11:20:48.658989+0000","last_unstale":"2024-03-21T11:20:55.720534+0000","last_undegraded":"2024-03-21T11:20:55.720534+0000","last_fullsized":"2024-03-21T11:20:55.720534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:44:09.599836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.832465+0000","last_change":"2024-03-21T11:20:48.668679+0000","last_active":"2024-03-21T11:20:55.832465+0000","last_peered":"2024-03-21T11:20:55.832465+0000","last_clean":"2024-03-21T11:20:55.832465+0000","last_became_active":"2024-03-21T11:20:48.667956+0000","last_became_peered":"2024-03-21T11:20:48.667956+0000","last_unstale":"2024-03-21T11:20:55.832465+0000","last_undegraded":"2024-03-21T11:20:55.832465+0000","last_fullsized":"2024-03-21T11:20:55.832465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:04.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-21T11:20:55.042177+0000","last_change":"2024-03-21T11:20:48.650213+0000","last_active":"2024-03-21T11:20:55.042177+0000","last_peered":"2024-03-21T11:20:55.042177+0000","last_clean":"2024-03-21T11:20:55.042177+0000","last_became_active":"2024-03-21T11:20:48.649693+0000","last_became_peered":"2024-03-21T11:20:48.649693+0000","last_unstale":"2024-03-21T11:20:55.042177+0000","last_undegraded":"2024-03-21T11:20:55.042177+0000","last_fullsized":"2024-03-21T11:20:55.042177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:26:45.180690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247498,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47313,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":20,"num_read_kb":55,"num_write":182,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":125148,"data_compressed":3658,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350997,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":494124,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612634,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1126400,"data_stored":1122846,"data_compressed":52683,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776158,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":489263,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037797,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":489183,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299434,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":1081277,"data_compressed":50854,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462960,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":488535,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":532103,"data_compressed":16622,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":491014,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":501641,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":488639,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888122,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":489111,"data_compressed":14793,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724597,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":1080289,"data_compressed":50854,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":43658,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":44033,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T11:21:22.047 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-21T11:21:22.047 INFO:tasks.cephadm:Setting up client nodes... 2024-03-21T11:21:22.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-21T11:21:22.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:21 smithi100 ceph-mon[36245]: from='client.24859 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:22.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:21 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1184893204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:22.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:21 smithi047 ceph-mon[31351]: from='client.24859 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:22.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:21 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1184893204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:22.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:21 smithi102 ceph-mon[36748]: from='client.24859 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:21:22.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:21 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1184893204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:22.560 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:23.019 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:22 smithi047 ceph-mon[31351]: pgmap v337: 129 pgs: 129 active+clean; 623 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.9 KiB/s wr, 12 op/s 2024-03-21T11:21:23.019 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:22 smithi047 ceph-mon[31351]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:21:23.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:22 smithi102 ceph-mon[36748]: pgmap v337: 129 pgs: 129 active+clean; 623 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.9 KiB/s wr, 12 op/s 2024-03-21T11:21:23.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:22 smithi102 ceph-mon[36748]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:21:23.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:22 smithi100 ceph-mon[36245]: pgmap v337: 129 pgs: 129 active+clean; 623 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.9 KiB/s wr, 12 op/s 2024-03-21T11:21:23.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:22 smithi100 ceph-mon[36245]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:21:24.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:23 smithi047 ceph-mon[31351]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:21:24.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:23 smithi102 ceph-mon[36748]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:21:24.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:23 smithi100 ceph-mon[36245]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:21:24.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.829+0000 7f2d0fe00700 1 -- 172.21.15.47:0/282521475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d10074b70 msgr2=0x7f2d10074ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:24.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.829+0000 7f2d0fe00700 1 --2- 172.21.15.47:0/282521475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d10074b70 0x7f2d10074ff0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0098f0 tx=0x7f2cfc009c00 comp rx=0 tx=0).stop 2024-03-21T11:21:24.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.829+0000 7f2d0fe00700 1 -- 172.21.15.47:0/282521475 shutdown_connections 2024-03-21T11:21:24.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.829+0000 7f2d0fe00700 1 --2- 172.21.15.47:0/282521475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d10109f30 0x7f2d1010c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:24.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.829+0000 7f2d0fe00700 1 --2- 172.21.15.47:0/282521475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d10076140 0x7f2d100745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:24.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.829+0000 7f2d0fe00700 1 --2- 172.21.15.47:0/282521475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d10074b70 0x7f2d10074ff0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:24.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.829+0000 7f2d0fe00700 1 -- 172.21.15.47:0/282521475 >> 172.21.15.47:0/282521475 conn(0x7f2d100fd5d0 msgr2=0x7f2d100ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:24.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.829+0000 7f2d0fe00700 1 -- 172.21.15.47:0/282521475 shutdown_connections 2024-03-21T11:21:24.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.830+0000 7f2d0fe00700 1 -- 172.21.15.47:0/282521475 wait complete. 2024-03-21T11:21:24.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.830+0000 7f2d0fe00700 1 Processor -- start 2024-03-21T11:21:24.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.831+0000 7f2d0fe00700 1 -- start start 2024-03-21T11:21:24.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.831+0000 7f2d0fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d10074b70 0x7f2d1019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:24.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.831+0000 7f2d0fe00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d10076140 0x7f2d1019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:24.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.831+0000 7f2d0fe00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d10109f30 0x7f2d101a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:24.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.831+0000 7f2d0fe00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1010fe40 con 0x7f2d10074b70 2024-03-21T11:21:24.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.831+0000 7f2d0fe00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1010fcc0 con 0x7f2d10109f30 2024-03-21T11:21:24.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.831+0000 7f2d0fe00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1010ffc0 con 0x7f2d10076140 2024-03-21T11:21:24.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.831+0000 7f2d0e000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d10076140 0x7f2d1019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:24.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.831+0000 7f2d0ea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d10074b70 0x7f2d1019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:24.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.831+0000 7f2d0e000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d10076140 0x7f2d1019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33064/0 (socket says 172.21.15.47:33064) 2024-03-21T11:21:24.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.831+0000 7f2d0e000700 1 -- 172.21.15.47:0/2886095898 learned_addr learned my addr 172.21.15.47:0/2886095898 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:24.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.832+0000 7f2d0f400700 1 --2- 172.21.15.47:0/2886095898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d10109f30 0x7f2d101a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:24.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.832+0000 7f2d0e000700 1 -- 172.21.15.47:0/2886095898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d10109f30 msgr2=0x7f2d101a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:24.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.832+0000 7f2d0e000700 1 --2- 172.21.15.47:0/2886095898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d10109f30 0x7f2d101a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:24.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.832+0000 7f2d0e000700 1 -- 172.21.15.47:0/2886095898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d10074b70 msgr2=0x7f2d1019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:24.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.832+0000 7f2d0e000700 1 --2- 172.21.15.47:0/2886095898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d10074b70 0x7f2d1019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:24.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.832+0000 7f2d0e000700 1 -- 172.21.15.47:0/2886095898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d101a2020 con 0x7f2d10076140 2024-03-21T11:21:24.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.832+0000 7f2d0e000700 1 --2- 172.21.15.47:0/2886095898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d10076140 0x7f2d1019a900 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00fca0 tx=0x7f2cfc005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:24.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.832+0000 7f2d07400700 1 -- 172.21.15.47:0/2886095898 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc025070 con 0x7f2d10076140 2024-03-21T11:21:24.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.832+0000 7f2d07400700 1 -- 172.21.15.47:0/2886095898 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cfc00ba40 con 0x7f2d10076140 2024-03-21T11:21:24.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.832+0000 7f2d07400700 1 -- 172.21.15.47:0/2886095898 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc003d10 con 0x7f2d10076140 2024-03-21T11:21:24.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.832+0000 7f2d0fe00700 1 -- 172.21.15.47:0/2886095898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d101a22b0 con 0x7f2d10076140 2024-03-21T11:21:24.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.833+0000 7f2d0fe00700 1 -- 172.21.15.47:0/2886095898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d101a2760 con 0x7f2d10076140 2024-03-21T11:21:24.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.834+0000 7f2d0fe00700 1 -- 172.21.15.47:0/2886095898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d100637c0 con 0x7f2d10076140 2024-03-21T11:21:24.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.836+0000 7f2d07400700 1 -- 172.21.15.47:0/2886095898 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f2cfc028070 con 0x7f2d10076140 2024-03-21T11:21:24.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.837+0000 7f2d07400700 1 --2- 172.21.15.47:0/2886095898 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2cf40a3170 0x7f2cf40a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:24.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.837+0000 7f2d07400700 1 -- 172.21.15.47:0/2886095898 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2cfc0d00c0 con 0x7f2d10076140 2024-03-21T11:21:24.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.837+0000 7f2d0ea00700 1 --2- 172.21.15.47:0/2886095898 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2cf40a3170 0x7f2cf40a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:24.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.838+0000 7f2d0ea00700 1 --2- 172.21.15.47:0/2886095898 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2cf40a3170 0x7f2cf40a5630 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2d00005fd0 tx=0x7f2d00009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:24.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:24.841+0000 7f2d07400700 1 -- 172.21.15.47:0/2886095898 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2cfc05c740 con 0x7f2d10076140 2024-03-21T11:21:25.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:25.162+0000 7f2d0fe00700 1 -- 172.21.15.47:0/2886095898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f2d1002da60 con 0x7f2d10076140 2024-03-21T11:21:25.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:24 smithi047 ceph-mon[31351]: pgmap v338: 129 pgs: 129 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.2 KiB/s wr, 20 op/s 2024-03-21T11:21:25.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:25.169+0000 7f2d07400700 1 -- 172.21.15.47:0/2886095898 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f2cfc013030 con 0x7f2d10076140 2024-03-21T11:21:25.170 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-03-21T11:21:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQA1GPxlIu/RCRAAJyvBs5/28OyOwirY1UhnBQ== 2024-03-21T11:21:25.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:25.172+0000 7f2d0fe00700 1 -- 172.21.15.47:0/2886095898 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2cf40a3170 msgr2=0x7f2cf40a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:25.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:25.172+0000 7f2d0fe00700 1 --2- 172.21.15.47:0/2886095898 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2cf40a3170 0x7f2cf40a5630 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2d00005fd0 tx=0x7f2d00009310 comp rx=0 tx=0).stop 2024-03-21T11:21:25.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:25.172+0000 7f2d0fe00700 1 -- 172.21.15.47:0/2886095898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d10076140 msgr2=0x7f2d1019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:25.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:25.172+0000 7f2d0fe00700 1 --2- 172.21.15.47:0/2886095898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d10076140 0x7f2d1019a900 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00fca0 tx=0x7f2cfc005250 comp rx=0 tx=0).stop 2024-03-21T11:21:25.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:25.172+0000 7f2d0fe00700 1 -- 172.21.15.47:0/2886095898 shutdown_connections 2024-03-21T11:21:25.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:25.172+0000 7f2d0fe00700 1 --2- 172.21.15.47:0/2886095898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d10109f30 0x7f2d101a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:25.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:25.172+0000 7f2d0fe00700 1 --2- 172.21.15.47:0/2886095898 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2cf40a3170 0x7f2cf40a5630 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:25.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:25.172+0000 7f2d0fe00700 1 --2- 172.21.15.47:0/2886095898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d10074b70 0x7f2d1019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:25.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:25.172+0000 7f2d0fe00700 1 --2- 172.21.15.47:0/2886095898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d10076140 0x7f2d1019a900 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:25.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:25.172+0000 7f2d0fe00700 1 -- 172.21.15.47:0/2886095898 >> 172.21.15.47:0/2886095898 conn(0x7f2d100fd5d0 msgr2=0x7f2d10105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:25.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:25.173+0000 7f2d0fe00700 1 -- 172.21.15.47:0/2886095898 shutdown_connections 2024-03-21T11:21:25.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:25.173+0000 7f2d0fe00700 1 -- 172.21.15.47:0/2886095898 wait complete. 2024-03-21T11:21:25.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:24 smithi102 ceph-mon[36748]: pgmap v338: 129 pgs: 129 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.2 KiB/s wr, 20 op/s 2024-03-21T11:21:25.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:24 smithi100 ceph-mon[36245]: pgmap v338: 129 pgs: 129 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.2 KiB/s wr, 20 op/s 2024-03-21T11:21:26.071 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:21:26.071 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-21T11:21:26.071 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-21T11:21:26.212 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:25 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2886095898' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T11:21:26.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:25 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T11:21:26.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:25 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T11:21:26.231 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-21T11:21:26.232 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-21T11:21:26.232 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-21T11:21:26.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:25 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2886095898' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T11:21:26.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:25 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T11:21:26.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:25 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T11:21:26.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:25 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2886095898' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T11:21:26.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:25 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T11:21:26.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:25 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T11:21:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:27.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:26 smithi102 ceph-mon[36748]: pgmap v339: 129 pgs: 129 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.2 KiB/s wr, 21 op/s 2024-03-21T11:21:27.271 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:26 smithi047 ceph-mon[31351]: pgmap v339: 129 pgs: 129 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.2 KiB/s wr, 21 op/s 2024-03-21T11:21:27.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:26 smithi100 ceph-mon[36245]: pgmap v339: 129 pgs: 129 active+clean; 625 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.2 KiB/s wr, 21 op/s 2024-03-21T11:21:28.149 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:27 smithi047 ceph-mon[31351]: pgmap v340: 129 pgs: 129 active+clean; 630 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-03-21T11:21:28.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:27 smithi102 ceph-mon[36748]: pgmap v340: 129 pgs: 129 active+clean; 630 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-03-21T11:21:28.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:27 smithi100 ceph-mon[36245]: pgmap v340: 129 pgs: 129 active+clean; 630 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-03-21T11:21:28.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.963+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2413962249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f41081e0 msgr2=0x7fe0f41085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:28.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.963+0000 7fe0f8e00700 1 --2- 172.21.15.47:0/2413962249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f41081e0 0x7fe0f41085c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009990 tx=0x7fe0e4009ca0 comp rx=0 tx=0).stop 2024-03-21T11:21:28.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.964+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2413962249 shutdown_connections 2024-03-21T11:21:28.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.964+0000 7fe0f8e00700 1 --2- 172.21.15.47:0/2413962249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0f406a7e0 0x7fe0f410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:28.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.964+0000 7fe0f8e00700 1 --2- 172.21.15.47:0/2413962249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f41081e0 0x7fe0f41085c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:28.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.964+0000 7fe0f8e00700 1 --2- 172.21.15.47:0/2413962249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0f4069e40 0x7fe0f406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:28.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.964+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2413962249 >> 172.21.15.47:0/2413962249 conn(0x7fe0f4077fc0 msgr2=0x7fe0f40783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:28.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.964+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2413962249 shutdown_connections 2024-03-21T11:21:28.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.964+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2413962249 wait complete. 2024-03-21T11:21:28.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.965+0000 7fe0f8e00700 1 Processor -- start 2024-03-21T11:21:28.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.965+0000 7fe0f8e00700 1 -- start start 2024-03-21T11:21:28.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.965+0000 7fe0f8e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0f4069e40 0x7fe0f41a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:28.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f8e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f406a7e0 0x7fe0f41a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:28.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f8e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0f41081e0 0x7fe0f419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:28.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f8e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f4112420 con 0x7fe0f4069e40 2024-03-21T11:21:28.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f8e00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f41122a0 con 0x7fe0f406a7e0 2024-03-21T11:21:28.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f8e00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f41125a0 con 0x7fe0f41081e0 2024-03-21T11:21:28.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f1400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f406a7e0 0x7fe0f41a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f1e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0f4069e40 0x7fe0f41a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f1400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f406a7e0 0x7fe0f41a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40696/0 (socket says 172.21.15.47:40696) 2024-03-21T11:21:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f1400700 1 -- 172.21.15.47:0/2885826140 learned_addr learned my addr 172.21.15.47:0/2885826140 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f2800700 1 --2- 172.21.15.47:0/2885826140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0f41081e0 0x7fe0f419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f1400700 1 -- 172.21.15.47:0/2885826140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0f41081e0 msgr2=0x7fe0f419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f1400700 1 --2- 172.21.15.47:0/2885826140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0f41081e0 0x7fe0f419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f1400700 1 -- 172.21.15.47:0/2885826140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0f4069e40 msgr2=0x7fe0f41a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f1400700 1 --2- 172.21.15.47:0/2885826140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0f4069e40 0x7fe0f41a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.966+0000 7fe0f1400700 1 -- 172.21.15.47:0/2885826140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f419b000 con 0x7fe0f406a7e0 2024-03-21T11:21:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.967+0000 7fe0f1400700 1 --2- 172.21.15.47:0/2885826140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f406a7e0 0x7fe0f41a0c20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000e9a0 tx=0x7fe0e000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.967+0000 7fe0eaa00700 1 -- 172.21.15.47:0/2885826140 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e000cbd0 con 0x7fe0f406a7e0 2024-03-21T11:21:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.967+0000 7fe0eaa00700 1 -- 172.21.15.47:0/2885826140 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0e000cd70 con 0x7fe0f406a7e0 2024-03-21T11:21:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.967+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2885826140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f419b2f0 con 0x7fe0f406a7e0 2024-03-21T11:21:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.967+0000 7fe0eaa00700 1 -- 172.21.15.47:0/2885826140 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0018750 con 0x7fe0f406a7e0 2024-03-21T11:21:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.967+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2885826140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f41a75b0 con 0x7fe0f406a7e0 2024-03-21T11:21:28.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.968+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2885826140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f4068e50 con 0x7fe0f406a7e0 2024-03-21T11:21:28.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.970+0000 7fe0eaa00700 1 -- 172.21.15.47:0/2885826140 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fe0e0010430 con 0x7fe0f406a7e0 2024-03-21T11:21:28.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.972+0000 7fe0eaa00700 1 --2- 172.21.15.47:0/2885826140 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe0d80a30c0 0x7fe0d80a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:28.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.972+0000 7fe0eaa00700 1 -- 172.21.15.47:0/2885826140 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe0e00cfa60 con 0x7fe0f406a7e0 2024-03-21T11:21:28.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.972+0000 7fe0f1e00700 1 --2- 172.21.15.47:0/2885826140 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe0d80a30c0 0x7fe0d80a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:28.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.973+0000 7fe0f1e00700 1 --2- 172.21.15.47:0/2885826140 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe0d80a30c0 0x7fe0d80a5580 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe0f419b9e0 tx=0x7fe0e4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:28.976+0000 7fe0eaa00700 1 -- 172.21.15.47:0/2885826140 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe0e009c490 con 0x7fe0f406a7e0 2024-03-21T11:21:29.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:29.268+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2885826140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe0f41a7810 con 0x7fe0f406a7e0 2024-03-21T11:21:29.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:29.273+0000 7fe0eaa00700 1 -- 172.21.15.47:0/2885826140 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+263551 (secure 0 0 0) 0x7fe0e009bd90 con 0x7fe0f406a7e0 2024-03-21T11:21:29.276 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:21:29.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:29.281+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2885826140 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe0d80a30c0 msgr2=0x7fe0d80a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:29.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:29.281+0000 7fe0f8e00700 1 --2- 172.21.15.47:0/2885826140 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe0d80a30c0 0x7fe0d80a5580 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe0f419b9e0 tx=0x7fe0e4005960 comp rx=0 tx=0).stop 2024-03-21T11:21:29.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:29.281+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2885826140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f406a7e0 msgr2=0x7fe0f41a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:29.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:29.281+0000 7fe0f8e00700 1 --2- 172.21.15.47:0/2885826140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f406a7e0 0x7fe0f41a0c20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000e9a0 tx=0x7fe0e000ecb0 comp rx=0 tx=0).stop 2024-03-21T11:21:29.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:29.281+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2885826140 shutdown_connections 2024-03-21T11:21:29.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:29.281+0000 7fe0f8e00700 1 --2- 172.21.15.47:0/2885826140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0f41081e0 0x7fe0f419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:29.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:29.281+0000 7fe0f8e00700 1 --2- 172.21.15.47:0/2885826140 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe0d80a30c0 0x7fe0d80a5580 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:29.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:29.281+0000 7fe0f8e00700 1 --2- 172.21.15.47:0/2885826140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0f4069e40 0x7fe0f41a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:29.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:29.281+0000 7fe0f8e00700 1 --2- 172.21.15.47:0/2885826140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f406a7e0 0x7fe0f41a0c20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:29.281+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2885826140 >> 172.21.15.47:0/2885826140 conn(0x7fe0f4077fc0 msgr2=0x7fe0f410d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:29.281+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2885826140 shutdown_connections 2024-03-21T11:21:29.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:29.281+0000 7fe0f8e00700 1 -- 172.21.15.47:0/2885826140 wait complete. 2024-03-21T11:21:30.093 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2057671763},{"type":"v1","addr":"172.21.15.47:6801","nonce":2057671763}]},"active_addr":"172.21.15.47:6801/2057671763","active_change":"2024-03-21T11:12:48.837174+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.47:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":4154356547}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":1443923713}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":2690553164}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":595972321}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":4267926278}]}]} 2024-03-21T11:21:30.098 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-21T11:21:30.098 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-21T11:21:30.099 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T11:21:30.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:30 smithi047 ceph-mon[31351]: pgmap v341: 129 pgs: 129 active+clean; 630 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 23 op/s 2024-03-21T11:21:30.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:30 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2885826140' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T11:21:30.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:30 smithi100 ceph-mon[36245]: pgmap v341: 129 pgs: 129 active+clean; 630 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 23 op/s 2024-03-21T11:21:30.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:30 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2885826140' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T11:21:30.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:30 smithi102 ceph-mon[36748]: pgmap v341: 129 pgs: 129 active+clean; 630 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 23 op/s 2024-03-21T11:21:30.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:30 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2885826140' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T11:21:30.584 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:32.319 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:32 smithi047 ceph-mon[31351]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 KiB/s wr, 18 op/s 2024-03-21T11:21:32.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:32 smithi100 ceph-mon[36245]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 KiB/s wr, 18 op/s 2024-03-21T11:21:32.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:32 smithi102 ceph-mon[36748]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 KiB/s wr, 18 op/s 2024-03-21T11:21:32.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.727+0000 7fd318400700 1 -- 172.21.15.47:0/2094598293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd310112e50 msgr2=0x7fd310115240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:32.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.727+0000 7fd318400700 1 --2- 172.21.15.47:0/2094598293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd310112e50 0x7fd310115240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd3080098f0 tx=0x7fd308009c00 comp rx=0 tx=0).stop 2024-03-21T11:21:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.728+0000 7fd318400700 1 -- 172.21.15.47:0/2094598293 shutdown_connections 2024-03-21T11:21:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.728+0000 7fd318400700 1 --2- 172.21.15.47:0/2094598293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd310112e50 0x7fd310115240 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.728+0000 7fd318400700 1 --2- 172.21.15.47:0/2094598293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3100761a0 0x7fd3100743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.728+0000 7fd318400700 1 --2- 172.21.15.47:0/2094598293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd310074900 0x7fd310074d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.728+0000 7fd318400700 1 -- 172.21.15.47:0/2094598293 >> 172.21.15.47:0/2094598293 conn(0x7fd310079220 msgr2=0x7fd310079630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.728+0000 7fd318400700 1 -- 172.21.15.47:0/2094598293 shutdown_connections 2024-03-21T11:21:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.728+0000 7fd318400700 1 -- 172.21.15.47:0/2094598293 wait complete. 2024-03-21T11:21:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.729+0000 7fd318400700 1 Processor -- start 2024-03-21T11:21:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.730+0000 7fd318400700 1 -- start start 2024-03-21T11:21:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.730+0000 7fd318400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd310074900 0x7fd31019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.730+0000 7fd318400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3100761a0 0x7fd31019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.730+0000 7fd318400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd310112e50 0x7fd3101a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.730+0000 7fd318400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd310118a90 con 0x7fd3100761a0 2024-03-21T11:21:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.730+0000 7fd318400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd310118910 con 0x7fd310112e50 2024-03-21T11:21:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.730+0000 7fd318400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd310118c10 con 0x7fd310074900 2024-03-21T11:21:32.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.730+0000 7fd315a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd310074900 0x7fd31019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:32.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.731+0000 7fd315a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd310074900 0x7fd31019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45592/0 (socket says 172.21.15.47:45592) 2024-03-21T11:21:32.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.731+0000 7fd315a00700 1 -- 172.21.15.47:0/3687984873 learned_addr learned my addr 172.21.15.47:0/3687984873 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:32.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.731+0000 7fd315000700 1 --2- 172.21.15.47:0/3687984873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3100761a0 0x7fd31019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:32.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.731+0000 7fd315a00700 1 -- 172.21.15.47:0/3687984873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd310112e50 msgr2=0x7fd3101a3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:32.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.731+0000 7fd315a00700 1 --2- 172.21.15.47:0/3687984873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd310112e50 0x7fd3101a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:32.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.731+0000 7fd315a00700 1 -- 172.21.15.47:0/3687984873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3100761a0 msgr2=0x7fd31019f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:32.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.731+0000 7fd315a00700 1 --2- 172.21.15.47:0/3687984873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3100761a0 0x7fd31019f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:32.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.731+0000 7fd315a00700 1 -- 172.21.15.47:0/3687984873 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3101a3f50 con 0x7fd310074900 2024-03-21T11:21:32.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.731+0000 7fd315a00700 1 --2- 172.21.15.47:0/3687984873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd310074900 0x7fd31019ef40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd30400ef10 tx=0x7fd30400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:32.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.732+0000 7fd30e000700 1 -- 172.21.15.47:0/3687984873 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30400ca70 con 0x7fd310074900 2024-03-21T11:21:32.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.732+0000 7fd30e000700 1 -- 172.21.15.47:0/3687984873 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd304019070 con 0x7fd310074900 2024-03-21T11:21:32.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.732+0000 7fd30e000700 1 -- 172.21.15.47:0/3687984873 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3040146e0 con 0x7fd310074900 2024-03-21T11:21:32.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.732+0000 7fd318400700 1 -- 172.21.15.47:0/3687984873 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3101a41e0 con 0x7fd310074900 2024-03-21T11:21:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.732+0000 7fd318400700 1 -- 172.21.15.47:0/3687984873 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3101abac0 con 0x7fd310074900 2024-03-21T11:21:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.733+0000 7fd318400700 1 -- 172.21.15.47:0/3687984873 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd310068e50 con 0x7fd310074900 2024-03-21T11:21:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.735+0000 7fd30e000700 1 -- 172.21.15.47:0/3687984873 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fd304012050 con 0x7fd310074900 2024-03-21T11:21:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.736+0000 7fd30e000700 1 --2- 172.21.15.47:0/3687984873 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2f80a30c0 0x7fd2f80a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.736+0000 7fd30e000700 1 -- 172.21.15.47:0/3687984873 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd3040cf800 con 0x7fd310074900 2024-03-21T11:21:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.737+0000 7fd315000700 1 --2- 172.21.15.47:0/3687984873 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2f80a30c0 0x7fd2f80a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:32.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.737+0000 7fd315000700 1 --2- 172.21.15.47:0/3687984873 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2f80a30c0 0x7fd2f80a5580 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009750 tx=0x7fd2fc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:32.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.740+0000 7fd30e000700 1 -- 172.21.15.47:0/3687984873 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd304010040 con 0x7fd310074900 2024-03-21T11:21:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.970+0000 7fd318400700 1 -- 172.21.15.47:0/3687984873 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3101abd60 con 0x7fd310074900 2024-03-21T11:21:32.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.972+0000 7fd30e000700 1 -- 172.21.15.47:0/3687984873 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24127 (secure 0 0 0) 0x7fd30409c230 con 0x7fd310074900 2024-03-21T11:21:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:21:32.973 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:20:54.191030+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:21:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.975+0000 7fd318400700 1 -- 172.21.15.47:0/3687984873 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2f80a30c0 msgr2=0x7fd2f80a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.975+0000 7fd318400700 1 --2- 172.21.15.47:0/3687984873 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2f80a30c0 0x7fd2f80a5580 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009750 tx=0x7fd2fc009310 comp rx=0 tx=0).stop 2024-03-21T11:21:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.975+0000 7fd318400700 1 -- 172.21.15.47:0/3687984873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd310074900 msgr2=0x7fd31019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.975+0000 7fd318400700 1 --2- 172.21.15.47:0/3687984873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd310074900 0x7fd31019ef40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd30400ef10 tx=0x7fd30400c3a0 comp rx=0 tx=0).stop 2024-03-21T11:21:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.975+0000 7fd318400700 1 -- 172.21.15.47:0/3687984873 shutdown_connections 2024-03-21T11:21:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.975+0000 7fd318400700 1 --2- 172.21.15.47:0/3687984873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd310112e50 0x7fd3101a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.975+0000 7fd318400700 1 --2- 172.21.15.47:0/3687984873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd310074900 0x7fd31019ef40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.975+0000 7fd318400700 1 --2- 172.21.15.47:0/3687984873 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2f80a30c0 0x7fd2f80a5580 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.975+0000 7fd318400700 1 --2- 172.21.15.47:0/3687984873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3100761a0 0x7fd31019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.975+0000 7fd318400700 1 -- 172.21.15.47:0/3687984873 >> 172.21.15.47:0/3687984873 conn(0x7fd310079220 msgr2=0x7fd310111bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.975+0000 7fd318400700 1 -- 172.21.15.47:0/3687984873 shutdown_connections 2024-03-21T11:21:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:32.975+0000 7fd318400700 1 -- 172.21.15.47:0/3687984873 wait complete. 2024-03-21T11:21:33.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:33 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3687984873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:33.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:33 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3687984873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:33.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:33 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3687984873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:33.665 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-21T11:21:33.665 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T11:21:34.145 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:34.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:34 smithi100 ceph-mon[36245]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.3 KiB/s wr, 17 op/s 2024-03-21T11:21:34.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:34 smithi047 ceph-mon[31351]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.3 KiB/s wr, 17 op/s 2024-03-21T11:21:34.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:34 smithi102 ceph-mon[36748]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.3 KiB/s wr, 17 op/s 2024-03-21T11:21:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.256+0000 7f5dec400700 1 -- 172.21.15.47:0/1505507298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de4074ea0 msgr2=0x7f5de410e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.256+0000 7f5dec400700 1 --2- 172.21.15.47:0/1505507298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de4074ea0 0x7f5de410e540 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009990 tx=0x7f5dd0009ca0 comp rx=0 tx=0).stop 2024-03-21T11:21:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.256+0000 7f5dec400700 1 -- 172.21.15.47:0/1505507298 shutdown_connections 2024-03-21T11:21:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.256+0000 7f5dec400700 1 --2- 172.21.15.47:0/1505507298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5de410ea80 0x7f5de4110eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.256+0000 7f5dec400700 1 --2- 172.21.15.47:0/1505507298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5de4074580 0x7f5de4074960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.256+0000 7f5dec400700 1 --2- 172.21.15.47:0/1505507298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de4074ea0 0x7f5de410e540 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.256+0000 7f5dec400700 1 -- 172.21.15.47:0/1505507298 >> 172.21.15.47:0/1505507298 conn(0x7f5de40fdcd0 msgr2=0x7f5de41000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.257+0000 7f5dec400700 1 -- 172.21.15.47:0/1505507298 shutdown_connections 2024-03-21T11:21:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.257+0000 7f5dec400700 1 -- 172.21.15.47:0/1505507298 wait complete. 2024-03-21T11:21:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.258+0000 7f5dec400700 1 Processor -- start 2024-03-21T11:21:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.258+0000 7f5dec400700 1 -- start start 2024-03-21T11:21:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.258+0000 7f5dec400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5de4074580 0x7f5de41a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.258+0000 7f5dec400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5de4074ea0 0x7f5de41a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.258+0000 7f5dec400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de410ea80 0x7f5de419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.258+0000 7f5dec400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de41147a0 con 0x7f5de4074580 2024-03-21T11:21:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.258+0000 7f5dec400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de4114620 con 0x7f5de410ea80 2024-03-21T11:21:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.258+0000 7f5dec400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de4114920 con 0x7f5de4074ea0 2024-03-21T11:21:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.258+0000 7f5de9a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5de4074580 0x7f5de41a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.259+0000 7f5de9a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5de4074580 0x7f5de41a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42854/0 (socket says 172.21.15.47:42854) 2024-03-21T11:21:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.259+0000 7f5de9a00700 1 -- 172.21.15.47:0/145622664 learned_addr learned my addr 172.21.15.47:0/145622664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.259+0000 7f5de9000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5de4074ea0 0x7f5de41a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.259+0000 7f5dea400700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de410ea80 0x7f5de419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.259+0000 7f5de9a00700 1 -- 172.21.15.47:0/145622664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5de4074ea0 msgr2=0x7f5de41a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.259+0000 7f5de9a00700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5de4074ea0 0x7f5de41a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.259+0000 7f5de9a00700 1 -- 172.21.15.47:0/145622664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de410ea80 msgr2=0x7f5de419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.259+0000 7f5de9a00700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de410ea80 0x7f5de419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.259+0000 7f5de9a00700 1 -- 172.21.15.47:0/145622664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de419b220 con 0x7f5de4074580 2024-03-21T11:21:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.259+0000 7f5dea400700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de410ea80 0x7f5de419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:21:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.259+0000 7f5de9000700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5de4074ea0 0x7f5de41a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:21:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.260+0000 7f5de9a00700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5de4074580 0x7f5de41a0900 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800e9a0 tx=0x7f5dd800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.260+0000 7f5de2000700 1 -- 172.21.15.47:0/145622664 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800ca10 con 0x7f5de4074580 2024-03-21T11:21:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.260+0000 7f5de2000700 1 -- 172.21.15.47:0/145622664 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5dd800cbb0 con 0x7f5de4074580 2024-03-21T11:21:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.260+0000 7f5dec400700 1 -- 172.21.15.47:0/145622664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de419b510 con 0x7f5de4074580 2024-03-21T11:21:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.260+0000 7f5de2000700 1 -- 172.21.15.47:0/145622664 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8018680 con 0x7f5de4074580 2024-03-21T11:21:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.260+0000 7f5dec400700 1 -- 172.21.15.47:0/145622664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de41a77d0 con 0x7f5de4074580 2024-03-21T11:21:36.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.261+0000 7f5dec400700 1 -- 172.21.15.47:0/145622664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de4068e50 con 0x7f5de4074580 2024-03-21T11:21:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.264+0000 7f5de2000700 1 -- 172.21.15.47:0/145622664 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f5dd8010430 con 0x7f5de4074580 2024-03-21T11:21:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.267+0000 7f5de2000700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5dcc0a3040 0x7f5dcc0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.267+0000 7f5de2000700 1 -- 172.21.15.47:0/145622664 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f5dd80cfef0 con 0x7f5de4074580 2024-03-21T11:21:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.267+0000 7f5de9000700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5dcc0a3040 0x7f5dcc0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.268+0000 7f5de9000700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5dcc0a3040 0x7f5dcc0a5500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5de41018e0 tx=0x7f5dd000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.268+0000 7f5de2000700 1 -- 172.21.15.47:0/145622664 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5dd809c920 con 0x7f5de4074580 2024-03-21T11:21:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:36 smithi047 ceph-mon[31351]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-21T11:21:36.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:36 smithi100 ceph-mon[36245]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-21T11:21:36.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:36 smithi102 ceph-mon[36748]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-21T11:21:36.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.494+0000 7f5dec400700 1 -- 172.21.15.47:0/145622664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5de402da30 con 0x7f5de4074580 2024-03-21T11:21:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.499+0000 7f5de2000700 1 -- 172.21.15.47:0/145622664 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24127 (secure 0 0 0) 0x7f5dd809c220 con 0x7f5de4074580 2024-03-21T11:21:36.500 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:21:36.500 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:20:54.191030+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:21:36.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.502+0000 7f5dec400700 1 -- 172.21.15.47:0/145622664 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5dcc0a3040 msgr2=0x7f5dcc0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:36.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.502+0000 7f5dec400700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5dcc0a3040 0x7f5dcc0a5500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5de41018e0 tx=0x7f5dd000b260 comp rx=0 tx=0).stop 2024-03-21T11:21:36.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.502+0000 7f5dec400700 1 -- 172.21.15.47:0/145622664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5de4074580 msgr2=0x7f5de41a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:36.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.502+0000 7f5dec400700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5de4074580 0x7f5de41a0900 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800e9a0 tx=0x7f5dd800ecb0 comp rx=0 tx=0).stop 2024-03-21T11:21:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.502+0000 7f5dec400700 1 -- 172.21.15.47:0/145622664 shutdown_connections 2024-03-21T11:21:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.502+0000 7f5dec400700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5de410ea80 0x7f5de419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.502+0000 7f5dec400700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5dcc0a3040 0x7f5dcc0a5500 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.502+0000 7f5dec400700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5de4074580 0x7f5de41a0900 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.502+0000 7f5dec400700 1 --2- 172.21.15.47:0/145622664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5de4074ea0 0x7f5de41a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.502+0000 7f5dec400700 1 -- 172.21.15.47:0/145622664 >> 172.21.15.47:0/145622664 conn(0x7f5de40fdcd0 msgr2=0x7f5de40fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.502+0000 7f5dec400700 1 -- 172.21.15.47:0/145622664 shutdown_connections 2024-03-21T11:21:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:36.502+0000 7f5dec400700 1 -- 172.21.15.47:0/145622664 wait complete. 2024-03-21T11:21:37.247 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-21T11:21:37.248 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-21T11:21:37.248 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-21T11:21:37.248 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-21T11:21:37.249 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-21T11:21:37.249 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-21T11:21:37.250 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-21T11:21:37.250 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-21T11:21:37.251 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-21T11:21:37.251 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-21T11:21:37.251 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-21T11:21:37.252 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-21T11:21:37.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:37 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/145622664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:37.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:37 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/145622664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:37.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:37 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/145622664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:21:38.251 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:38.366 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:38.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:38 smithi100 ceph-mon[36245]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-21T11:21:38.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:38 smithi102 ceph-mon[36748]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-21T11:21:38.506 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:38 smithi047 ceph-mon[31351]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-21T11:21:38.717 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:38.717 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:38.717 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:38.717 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:38.719 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:38.720 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:40.231 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:40.236 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:40.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:40 smithi047 ceph-mon[31351]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T11:21:40.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:40 smithi100 ceph-mon[36245]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T11:21:40.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:40 smithi102 ceph-mon[36748]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T11:21:42.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:42 smithi100 ceph-mon[36245]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-21T11:21:42.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:42 smithi047 ceph-mon[31351]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-21T11:21:42.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:42 smithi102 ceph-mon[36748]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-21T11:21:42.888 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:21:44.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:44 smithi100 ceph-mon[36245]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T11:21:44.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:44 smithi102 ceph-mon[36748]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T11:21:44.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:44 smithi047 ceph-mon[31351]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T11:21:46.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:46 smithi047 ceph-mon[31351]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T11:21:46.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:46 smithi100 ceph-mon[36245]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T11:21:46.990 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:46 smithi102 ceph-mon[36748]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T11:21:48.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.720+0000 7fa0cb800700 1 -- 172.21.15.47:0/3130333040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0c4074ce0 msgr2=0x7fa0c410e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.720+0000 7fa0cb800700 1 --2- 172.21.15.47:0/3130333040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0c4074ce0 0x7fa0c410e650 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009990 tx=0x7fa0b0009ca0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.721+0000 7fa0cb800700 1 -- 172.21.15.47:0/3130333040 shutdown_connections 2024-03-21T11:21:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.721+0000 7fa0cb800700 1 --2- 172.21.15.47:0/3130333040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c410eb90 0x7fa0c4110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.721+0000 7fa0cb800700 1 --2- 172.21.15.47:0/3130333040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c40743c0 0x7fa0c40747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.721+0000 7fa0cb800700 1 --2- 172.21.15.47:0/3130333040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0c4074ce0 0x7fa0c410e650 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.721+0000 7fa0cb800700 1 -- 172.21.15.47:0/3130333040 >> 172.21.15.47:0/3130333040 conn(0x7fa0c40792c0 msgr2=0x7fa0c40796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.721+0000 7fa0cb800700 1 -- 172.21.15.47:0/3130333040 shutdown_connections 2024-03-21T11:21:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.721+0000 7fa0cb800700 1 -- 172.21.15.47:0/3130333040 wait complete. 2024-03-21T11:21:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.722+0000 7fa0cb800700 1 Processor -- start 2024-03-21T11:21:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.722+0000 7fa0cb800700 1 -- start start 2024-03-21T11:21:48.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.722+0000 7fa0cb800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0c40743c0 0x7fa0c41a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.722+0000 7fa0cb800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c4074ce0 0x7fa0c41a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.722+0000 7fa0cb800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c410eb90 0x7fa0c419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.722+0000 7fa0cb800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c41146a0 con 0x7fa0c4074ce0 2024-03-21T11:21:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.722+0000 7fa0cb800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4114520 con 0x7fa0c410eb90 2024-03-21T11:21:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.722+0000 7fa0cb800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4114820 con 0x7fa0c40743c0 2024-03-21T11:21:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.723+0000 7fa0c8e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0c40743c0 0x7fa0c41a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.723+0000 7fa0c8e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0c40743c0 0x7fa0c41a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55578/0 (socket says 172.21.15.47:55578) 2024-03-21T11:21:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.723+0000 7fa0c8e00700 1 -- 172.21.15.47:0/1526656042 learned_addr learned my addr 172.21.15.47:0/1526656042 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.723+0000 7fa0c9800700 1 --2- 172.21.15.47:0/1526656042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c410eb90 0x7fa0c419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.723+0000 7fa0c3e00700 1 --2- 172.21.15.47:0/1526656042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c4074ce0 0x7fa0c41a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.723+0000 7fa0c8e00700 1 -- 172.21.15.47:0/1526656042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c410eb90 msgr2=0x7fa0c419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:48.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.723+0000 7fa0c8e00700 1 --2- 172.21.15.47:0/1526656042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c410eb90 0x7fa0c419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.723+0000 7fa0c8e00700 1 -- 172.21.15.47:0/1526656042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c4074ce0 msgr2=0x7fa0c41a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.723+0000 7fa0c8e00700 1 --2- 172.21.15.47:0/1526656042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c4074ce0 0x7fa0c41a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.723+0000 7fa0c8e00700 1 -- 172.21.15.47:0/1526656042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c419b0c0 con 0x7fa0c40743c0 2024-03-21T11:21:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.723+0000 7fa0c8e00700 1 --2- 172.21.15.47:0/1526656042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0c40743c0 0x7fa0c41a0750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800e9a0 tx=0x7fa0b800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.724+0000 7fa0c1600700 1 -- 172.21.15.47:0/1526656042 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800ca10 con 0x7fa0c40743c0 2024-03-21T11:21:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.724+0000 7fa0c1600700 1 -- 172.21.15.47:0/1526656042 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0b800cbb0 con 0x7fa0c40743c0 2024-03-21T11:21:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.724+0000 7fa0c1600700 1 -- 172.21.15.47:0/1526656042 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8018680 con 0x7fa0c40743c0 2024-03-21T11:21:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.724+0000 7fa0cb800700 1 -- 172.21.15.47:0/1526656042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c419b3b0 con 0x7fa0c40743c0 2024-03-21T11:21:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.724+0000 7fa0cb800700 1 -- 172.21.15.47:0/1526656042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c41a7620 con 0x7fa0c40743c0 2024-03-21T11:21:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.725+0000 7fa0cb800700 1 -- 172.21.15.47:0/1526656042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0c4062920 con 0x7fa0c40743c0 2024-03-21T11:21:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.727+0000 7fa0c1600700 1 -- 172.21.15.47:0/1526656042 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fa0b8010430 con 0x7fa0c40743c0 2024-03-21T11:21:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.728+0000 7fa0c1600700 1 --2- 172.21.15.47:0/1526656042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa0ac0a30c0 0x7fa0ac0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:48.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.728+0000 7fa0c1600700 1 -- 172.21.15.47:0/1526656042 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa0b80d0230 con 0x7fa0c40743c0 2024-03-21T11:21:48.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.729+0000 7fa0c3e00700 1 --2- 172.21.15.47:0/1526656042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa0ac0a30c0 0x7fa0ac0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:48.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.729+0000 7fa0c1600700 1 --2- 172.21.15.47:0/1526656042 >> [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460] conn(0x7fa0ac0add80 0x7fa0ac0b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:48.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.729+0000 7fa0c1600700 1 -- 172.21.15.47:0/1526656042 --> [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0b801bec0 con 0x7fa0ac0add80 2024-03-21T11:21:48.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.729+0000 7fa0c1600700 1 -- 172.21.15.47:0/1526656042 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0b80d05e0 con 0x7fa0c40743c0 2024-03-21T11:21:48.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.729+0000 7fa0c9800700 1 --2- 172.21.15.47:0/1526656042 >> [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460] conn(0x7fa0ac0add80 0x7fa0ac0b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:48.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.729+0000 7fa0c3e00700 1 --2- 172.21.15.47:0/1526656042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa0ac0a30c0 0x7fa0ac0a5580 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa0c41108b0 tx=0x7fa0b0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:48.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.730+0000 7fa0c9800700 1 --2- 172.21.15.47:0/1526656042 >> [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460] conn(0x7fa0ac0add80 0x7fa0ac0b01c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.731+0000 7fa0c1600700 1 -- 172.21.15.47:0/1526656042 <== osd.0 v2:172.21.15.47:6802/3777145460 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa0b801bec0 con 0x7fa0ac0add80 2024-03-21T11:21:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.749+0000 7fa0cb800700 1 -- 172.21.15.47:0/1526656042 --> [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa0c4062d20 con 0x7fa0ac0add80 2024-03-21T11:21:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.750+0000 7fa0c1600700 1 -- 172.21.15.47:0/1526656042 <== osd.0 v2:172.21.15.47:6802/3777145460 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa0c4062d20 con 0x7fa0ac0add80 2024-03-21T11:21:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.750+0000 7fa0cb800700 1 -- 172.21.15.47:0/1526656042 >> [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460] conn(0x7fa0ac0add80 msgr2=0x7fa0ac0b01c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.750+0000 7fa0cb800700 1 --2- 172.21.15.47:0/1526656042 >> [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460] conn(0x7fa0ac0add80 0x7fa0ac0b01c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.751+0000 7fa0cb800700 1 -- 172.21.15.47:0/1526656042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa0ac0a30c0 msgr2=0x7fa0ac0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.751+0000 7fa0cb800700 1 --2- 172.21.15.47:0/1526656042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa0ac0a30c0 0x7fa0ac0a5580 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa0c41108b0 tx=0x7fa0b0005fd0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.751+0000 7fa0cb800700 1 -- 172.21.15.47:0/1526656042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0c40743c0 msgr2=0x7fa0c41a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.751+0000 7fa0cb800700 1 --2- 172.21.15.47:0/1526656042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0c40743c0 0x7fa0c41a0750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800e9a0 tx=0x7fa0b800ecb0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.751+0000 7fa0c9800700 1 -- 172.21.15.47:0/1526656042 reap_dead start 2024-03-21T11:21:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.751+0000 7fa0cb800700 1 -- 172.21.15.47:0/1526656042 shutdown_connections 2024-03-21T11:21:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.751+0000 7fa0cb800700 1 -- 172.21.15.47:0/1526656042 >> 172.21.15.47:0/1526656042 conn(0x7fa0c40792c0 msgr2=0x7fa0c40ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:48.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.751+0000 7fa0cb800700 1 -- 172.21.15.47:0/1526656042 shutdown_connections 2024-03-21T11:21:48.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:45.751+0000 7fa0cb800700 1 -- 172.21.15.47:0/1526656042 wait complete. 2024-03-21T11:21:48.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:48 smithi100 ceph-mon[36245]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T11:21:48.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:48 smithi047 ceph-mon[31351]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T11:21:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.985+0000 7fb076c00700 1 -- 172.21.15.47:0/2850934036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0701081c0 msgr2=0x7fb0701085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.985+0000 7fb076c00700 1 --2- 172.21.15.47:0/2850934036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0701081c0 0x7fb0701085a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb0600099c0 tx=0x7fb060009cd0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.985+0000 7fb076c00700 1 -- 172.21.15.47:0/2850934036 shutdown_connections 2024-03-21T11:21:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.985+0000 7fb076c00700 1 --2- 172.21.15.47:0/2850934036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb070102ba0 0x7fb07010efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.985+0000 7fb076c00700 1 --2- 172.21.15.47:0/2850934036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0701081c0 0x7fb0701085a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.985+0000 7fb076c00700 1 --2- 172.21.15.47:0/2850934036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0701021e0 0x7fb070102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.985+0000 7fb076c00700 1 -- 172.21.15.47:0/2850934036 >> 172.21.15.47:0/2850934036 conn(0x7fb0700792c0 msgr2=0x7fb0700796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.985+0000 7fb076c00700 1 -- 172.21.15.47:0/2850934036 shutdown_connections 2024-03-21T11:21:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.985+0000 7fb076c00700 1 -- 172.21.15.47:0/2850934036 wait complete. 2024-03-21T11:21:48.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.986+0000 7fb076c00700 1 Processor -- start 2024-03-21T11:21:48.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.986+0000 7fb076c00700 1 -- start start 2024-03-21T11:21:48.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.987+0000 7fb076c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0701021e0 0x7fb070074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:48.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.987+0000 7fb076c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb070102ba0 0x7fb0700749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:48.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.987+0000 7fb076c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0701081c0 0x7fb070074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:48.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.987+0000 7fb076c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0701124c0 con 0x7fb070102ba0 2024-03-21T11:21:48.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.987+0000 7fb076c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb070112340 con 0x7fb0701081c0 2024-03-21T11:21:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.987+0000 7fb076c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb070112640 con 0x7fb0701021e0 2024-03-21T11:21:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.987+0000 7fb06fe00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0701021e0 0x7fb070074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.987+0000 7fb074c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0701081c0 0x7fb070074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.987+0000 7fb06fe00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0701021e0 0x7fb070074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51982/0 (socket says 172.21.15.47:51982) 2024-03-21T11:21:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.987+0000 7fb06fe00700 1 -- 172.21.15.47:0/586213317 learned_addr learned my addr 172.21.15.47:0/586213317 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.988+0000 7fb06f400700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb070102ba0 0x7fb0700749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.988+0000 7fb06fe00700 1 -- 172.21.15.47:0/586213317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0701081c0 msgr2=0x7fb070074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.988+0000 7fb06fe00700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0701081c0 0x7fb070074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.988+0000 7fb06fe00700 1 -- 172.21.15.47:0/586213317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb070102ba0 msgr2=0x7fb0700749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.988+0000 7fb06fe00700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb070102ba0 0x7fb0700749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.988+0000 7fb06fe00700 1 -- 172.21.15.47:0/586213317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0701a6de0 con 0x7fb0701021e0 2024-03-21T11:21:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.988+0000 7fb06f400700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb070102ba0 0x7fb0700749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:21:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.988+0000 7fb074c00700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0701081c0 0x7fb070074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:21:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.988+0000 7fb06fe00700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0701021e0 0x7fb070074490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb060005910 tx=0x7fb06000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.989+0000 7fb06cc00700 1 -- 172.21.15.47:0/586213317 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060027070 con 0x7fb0701021e0 2024-03-21T11:21:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.989+0000 7fb06cc00700 1 -- 172.21.15.47:0/586213317 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb060003680 con 0x7fb0701021e0 2024-03-21T11:21:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.989+0000 7fb06cc00700 1 -- 172.21.15.47:0/586213317 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0600173f0 con 0x7fb0701021e0 2024-03-21T11:21:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.989+0000 7fb076c00700 1 -- 172.21.15.47:0/586213317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0701a7070 con 0x7fb0701021e0 2024-03-21T11:21:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.989+0000 7fb076c00700 1 -- 172.21.15.47:0/586213317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0701a7520 con 0x7fb0701021e0 2024-03-21T11:21:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.990+0000 7fb076c00700 1 -- 172.21.15.47:0/586213317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb0700628e0 con 0x7fb0701021e0 2024-03-21T11:21:48.995 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:48 smithi102 ceph-mon[36748]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T11:21:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.994+0000 7fb06cc00700 1 -- 172.21.15.47:0/586213317 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fb06000fcd0 con 0x7fb0701021e0 2024-03-21T11:21:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.995+0000 7fb06cc00700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0540a30c0 0x7fb0540a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.995+0000 7fb06cc00700 1 -- 172.21.15.47:0/586213317 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb0600d0420 con 0x7fb0701021e0 2024-03-21T11:21:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.995+0000 7fb06f400700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0540a30c0 0x7fb0540a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.996+0000 7fb06cc00700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971] conn(0x7fb0540adc50 0x7fb0540b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.996+0000 7fb06cc00700 1 -- 172.21.15.47:0/586213317 --> [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb06000b0b0 con 0x7fb0540adc50 2024-03-21T11:21:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.996+0000 7fb06cc00700 1 -- 172.21.15.47:0/586213317 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0600d07d0 con 0x7fb0701021e0 2024-03-21T11:21:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.996+0000 7fb074c00700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971] conn(0x7fb0540adc50 0x7fb0540b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.997+0000 7fb06f400700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0540a30c0 0x7fb0540a5580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb070079850 tx=0x7fb05c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.997+0000 7fb074c00700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971] conn(0x7fb0540adc50 0x7fb0540b0090 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:48.998+0000 7fb06cc00700 1 -- 172.21.15.47:0/586213317 <== osd.4 v2:172.21.15.100:6808/1323994971 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb06000b0b0 con 0x7fb0540adc50 2024-03-21T11:21:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:49.023+0000 7fb076c00700 1 -- 172.21.15.47:0/586213317 --> [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb070062d10 con 0x7fb0540adc50 2024-03-21T11:21:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:49.024+0000 7fb06cc00700 1 -- 172.21.15.47:0/586213317 <== osd.4 v2:172.21.15.100:6808/1323994971 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb070062d10 con 0x7fb0540adc50 2024-03-21T11:21:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:49.024+0000 7fb076c00700 1 -- 172.21.15.47:0/586213317 >> [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971] conn(0x7fb0540adc50 msgr2=0x7fb0540b0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:49.024+0000 7fb076c00700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971] conn(0x7fb0540adc50 0x7fb0540b0090 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:49.024+0000 7fb076c00700 1 -- 172.21.15.47:0/586213317 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0540a30c0 msgr2=0x7fb0540a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:49.024+0000 7fb076c00700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0540a30c0 0x7fb0540a5580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb070079850 tx=0x7fb05c009260 comp rx=0 tx=0).stop 2024-03-21T11:21:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:49.024+0000 7fb076c00700 1 -- 172.21.15.47:0/586213317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0701021e0 msgr2=0x7fb070074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:49.024+0000 7fb076c00700 1 --2- 172.21.15.47:0/586213317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0701021e0 0x7fb070074490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb060005910 tx=0x7fb06000f690 comp rx=0 tx=0).stop 2024-03-21T11:21:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:49.024+0000 7fb074c00700 1 -- 172.21.15.47:0/586213317 reap_dead start 2024-03-21T11:21:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:49.025+0000 7fb076c00700 1 -- 172.21.15.47:0/586213317 shutdown_connections 2024-03-21T11:21:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:49.025+0000 7fb076c00700 1 -- 172.21.15.47:0/586213317 >> 172.21.15.47:0/586213317 conn(0x7fb0700792c0 msgr2=0x7fb070100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:49.025+0000 7fb076c00700 1 -- 172.21.15.47:0/586213317 shutdown_connections 2024-03-21T11:21:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:49.025+0000 7fb076c00700 1 -- 172.21.15.47:0/586213317 wait complete. 2024-03-21T11:21:49.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-21T11:21:49.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-21T11:21:49.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-21T11:21:49.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-21T11:21:49.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-21T11:21:49.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-21T11:21:49.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2024-03-21T11:21:49.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-03-21T11:21:49.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-03-21T11:21:49.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-21T11:21:49.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:21:49.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-21T11:21:49.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-21T11:21:49.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-21T11:21:49.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-21T11:21:49.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-21T11:21:49.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-21T11:21:49.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2024-03-21T11:21:49.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-03-21T11:21:49.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-03-21T11:21:49.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-21T11:21:49.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:21:49.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-21T11:21:49.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-21T11:21:49.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-21T11:21:49.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-21T11:21:49.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-21T11:21:49.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-21T11:21:49.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2024-03-21T11:21:49.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-03-21T11:21:49.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-03-21T11:21:49.993 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-21T11:21:49.994 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:21:50.989 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:50 smithi047 ceph-mon[31351]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T11:21:50.989 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-21T11:21:50.989 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-21T11:21:50.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-21T11:21:50.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-21T11:21:50.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-21T11:21:50.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-21T11:21:50.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2024-03-21T11:21:50.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-03-21T11:21:50.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-03-21T11:21:50.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-21T11:21:50.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:50 smithi047 ceph-mon[31351]: osdmap e92: 12 total, 12 up, 12 in 2024-03-21T11:21:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:50 smithi102 ceph-mon[36748]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T11:21:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-21T11:21:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-21T11:21:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-21T11:21:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-21T11:21:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-21T11:21:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-21T11:21:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2024-03-21T11:21:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-03-21T11:21:50.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-03-21T11:21:50.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-21T11:21:50.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:50 smithi102 ceph-mon[36748]: osdmap e92: 12 total, 12 up, 12 in 2024-03-21T11:21:51.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:50 smithi100 ceph-mon[36245]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T11:21:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-21T11:21:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-21T11:21:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-21T11:21:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-21T11:21:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-21T11:21:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-21T11:21:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2024-03-21T11:21:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-03-21T11:21:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-03-21T11:21:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-21T11:21:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:50 smithi100 ceph-mon[36245]: osdmap e92: 12 total, 12 up, 12 in 2024-03-21T11:21:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:51 smithi102 ceph-mon[36748]: osdmap e93: 12 total, 12 up, 12 in 2024-03-21T11:21:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:21:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:21:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:21:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:21:52.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:51 smithi047 ceph-mon[31351]: osdmap e93: 12 total, 12 up, 12 in 2024-03-21T11:21:52.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:21:52.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:21:52.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:21:52.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:21:52.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:51 smithi100 ceph-mon[36245]: osdmap e93: 12 total, 12 up, 12 in 2024-03-21T11:21:52.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:21:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:21:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:21:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:21:53.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:52 smithi047 ceph-mon[31351]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-21T11:21:53.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:52 smithi100 ceph-mon[36245]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-21T11:21:53.221 INFO:teuthology.orchestra.run.smithi047.stdout:34359738455 2024-03-21T11:21:53.222 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-21T11:21:53.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:52 smithi102 ceph-mon[36748]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-21T11:21:53.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.444+0000 7f6f06c00700 1 -- 172.21.15.47:0/1104694663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f0006a7e0 msgr2=0x7f6f0010f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:53.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.444+0000 7f6f06c00700 1 --2- 172.21.15.47:0/1104694663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f0006a7e0 0x7f6f0010f020 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b0a0 tx=0x7f6ef800b3b0 comp rx=0 tx=0).stop 2024-03-21T11:21:53.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.445+0000 7f6f06c00700 1 -- 172.21.15.47:0/1104694663 shutdown_connections 2024-03-21T11:21:53.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.445+0000 7f6f06c00700 1 --2- 172.21.15.47:0/1104694663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f0006a7e0 0x7f6f0010f020 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:53.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.445+0000 7f6f06c00700 1 --2- 172.21.15.47:0/1104694663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f001081e0 0x7f6f001085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:53.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.445+0000 7f6f06c00700 1 --2- 172.21.15.47:0/1104694663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f00069e40 0x7f6f0006a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:53.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.445+0000 7f6f06c00700 1 -- 172.21.15.47:0/1104694663 >> 172.21.15.47:0/1104694663 conn(0x7f6f00077fc0 msgr2=0x7f6f000783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:53.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.445+0000 7f6f06c00700 1 -- 172.21.15.47:0/1104694663 shutdown_connections 2024-03-21T11:21:53.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.445+0000 7f6f06c00700 1 -- 172.21.15.47:0/1104694663 wait complete. 2024-03-21T11:21:53.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.446+0000 7f6f06c00700 1 Processor -- start 2024-03-21T11:21:53.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.446+0000 7f6f06c00700 1 -- start start 2024-03-21T11:21:53.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.447+0000 7f6f06c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f00069e40 0x7f6f0019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:53.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.447+0000 7f6f06c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f0006a7e0 0x7f6f0019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:53.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.447+0000 7f6f06c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f001081e0 0x7f6f001987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:53.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.447+0000 7f6f06c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f00112480 con 0x7f6f00069e40 2024-03-21T11:21:53.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.447+0000 7f6f06c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f00112300 con 0x7f6f001081e0 2024-03-21T11:21:53.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.447+0000 7f6f06c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f00112600 con 0x7f6f0006a7e0 2024-03-21T11:21:53.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.447+0000 7f6effe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f00069e40 0x7f6f0019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:53.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.447+0000 7f6eff400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f0006a7e0 0x7f6f0019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:53.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.447+0000 7f6effe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f00069e40 0x7f6f0019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49962/0 (socket says 172.21.15.47:49962) 2024-03-21T11:21:53.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.447+0000 7f6effe00700 1 -- 172.21.15.47:0/2163897573 learned_addr learned my addr 172.21.15.47:0/2163897573 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:53.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.447+0000 7f6f04c00700 1 --2- 172.21.15.47:0/2163897573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f001081e0 0x7f6f001987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:53.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.448+0000 7f6f04c00700 1 -- 172.21.15.47:0/2163897573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f0006a7e0 msgr2=0x7f6f0019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:53.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.448+0000 7f6f04c00700 1 --2- 172.21.15.47:0/2163897573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f0006a7e0 0x7f6f0019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:53.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.448+0000 7f6f04c00700 1 -- 172.21.15.47:0/2163897573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f00069e40 msgr2=0x7f6f0019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:53.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.448+0000 7f6f04c00700 1 --2- 172.21.15.47:0/2163897573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f00069e40 0x7f6f0019e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:53.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.448+0000 7f6f04c00700 1 -- 172.21.15.47:0/2163897573 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f00198fc0 con 0x7f6f001081e0 2024-03-21T11:21:53.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.448+0000 7f6effe00700 1 --2- 172.21.15.47:0/2163897573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f00069e40 0x7f6f0019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:21:53.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.448+0000 7f6f04c00700 1 --2- 172.21.15.47:0/2163897573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f001081e0 0x7f6f001987e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80062a0 tx=0x7f6ef8004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:53.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.448+0000 7f6efcc00700 1 -- 172.21.15.47:0/2163897573 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8021070 con 0x7f6f001081e0 2024-03-21T11:21:53.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.448+0000 7f6efcc00700 1 -- 172.21.15.47:0/2163897573 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ef8003c00 con 0x7f6f001081e0 2024-03-21T11:21:53.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.448+0000 7f6f06c00700 1 -- 172.21.15.47:0/2163897573 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f00199250 con 0x7f6f001081e0 2024-03-21T11:21:53.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.448+0000 7f6efcc00700 1 -- 172.21.15.47:0/2163897573 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef80045a0 con 0x7f6f001081e0 2024-03-21T11:21:53.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.448+0000 7f6f06c00700 1 -- 172.21.15.47:0/2163897573 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f001a5470 con 0x7f6f001081e0 2024-03-21T11:21:53.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.449+0000 7f6f06c00700 1 -- 172.21.15.47:0/2163897573 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6f00062860 con 0x7f6f001081e0 2024-03-21T11:21:53.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.452+0000 7f6efcc00700 1 -- 172.21.15.47:0/2163897573 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f6ef8004740 con 0x7f6f001081e0 2024-03-21T11:21:53.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.453+0000 7f6efcc00700 1 --2- 172.21.15.47:0/2163897573 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ee40a30c0 0x7f6ee40a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:53.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.453+0000 7f6efcc00700 1 -- 172.21.15.47:0/2163897573 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f6ef800f020 con 0x7f6f001081e0 2024-03-21T11:21:53.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.454+0000 7f6effe00700 1 --2- 172.21.15.47:0/2163897573 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ee40a30c0 0x7f6ee40a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:53.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.454+0000 7f6efcc00700 1 --2- 172.21.15.47:0/2163897573 >> [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436] conn(0x7f6ee40ae220 0x7f6ee40b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:53.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.454+0000 7f6efcc00700 1 -- 172.21.15.47:0/2163897573 --> [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ef8012e50 con 0x7f6ee40ae220 2024-03-21T11:21:53.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.454+0000 7f6efcc00700 1 -- 172.21.15.47:0/2163897573 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ef80d10e0 con 0x7f6f001081e0 2024-03-21T11:21:53.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.454+0000 7f6eff400700 1 --2- 172.21.15.47:0/2163897573 >> [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436] conn(0x7f6ee40ae220 0x7f6ee40b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:53.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.454+0000 7f6effe00700 1 --2- 172.21.15.47:0/2163897573 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ee40a30c0 0x7f6ee40a5580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0011550 tx=0x7f6ef0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:53.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.455+0000 7f6eff400700 1 --2- 172.21.15.47:0/2163897573 >> [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436] conn(0x7f6ee40ae220 0x7f6ee40b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:53.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.456+0000 7f6efcc00700 1 -- 172.21.15.47:0/2163897573 <== osd.1 v2:172.21.15.100:6800/2612500436 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6ef8012e50 con 0x7f6ee40ae220 2024-03-21T11:21:53.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.475+0000 7f6f06c00700 1 -- 172.21.15.47:0/2163897573 --> [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6f00062c60 con 0x7f6ee40ae220 2024-03-21T11:21:53.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.476+0000 7f6efcc00700 1 -- 172.21.15.47:0/2163897573 <== osd.1 v2:172.21.15.100:6800/2612500436 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6f00062c60 con 0x7f6ee40ae220 2024-03-21T11:21:53.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.476+0000 7f6f06c00700 1 -- 172.21.15.47:0/2163897573 >> [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436] conn(0x7f6ee40ae220 msgr2=0x7f6ee40b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:53.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.476+0000 7f6f06c00700 1 --2- 172.21.15.47:0/2163897573 >> [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436] conn(0x7f6ee40ae220 0x7f6ee40b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:53.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.477+0000 7f6f06c00700 1 -- 172.21.15.47:0/2163897573 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ee40a30c0 msgr2=0x7f6ee40a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:53.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.477+0000 7f6f06c00700 1 --2- 172.21.15.47:0/2163897573 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ee40a30c0 0x7f6ee40a5580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0011550 tx=0x7f6ef0009260 comp rx=0 tx=0).stop 2024-03-21T11:21:53.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.477+0000 7f6f06c00700 1 -- 172.21.15.47:0/2163897573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f001081e0 msgr2=0x7f6f001987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:53.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.477+0000 7f6f06c00700 1 --2- 172.21.15.47:0/2163897573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f001081e0 0x7f6f001987e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80062a0 tx=0x7f6ef8004c60 comp rx=0 tx=0).stop 2024-03-21T11:21:53.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.477+0000 7f6f04c00700 1 -- 172.21.15.47:0/2163897573 reap_dead start 2024-03-21T11:21:53.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.477+0000 7f6f06c00700 1 -- 172.21.15.47:0/2163897573 shutdown_connections 2024-03-21T11:21:53.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.477+0000 7f6f06c00700 1 -- 172.21.15.47:0/2163897573 >> 172.21.15.47:0/2163897573 conn(0x7f6f00077fc0 msgr2=0x7f6f000ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:53.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.477+0000 7f6f06c00700 1 -- 172.21.15.47:0/2163897573 shutdown_connections 2024-03-21T11:21:53.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:50.477+0000 7f6f06c00700 1 -- 172.21.15.47:0/2163897573 wait complete. 2024-03-21T11:21:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.061+0000 7f63aac00700 1 -- 172.21.15.47:0/3510154294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a4104f90 msgr2=0x7f63a4108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.061+0000 7f63aac00700 1 --2- 172.21.15.47:0/3510154294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a4104f90 0x7f63a4108fe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6390009990 tx=0x7f6390009ca0 comp rx=0 tx=0).stop 2024-03-21T11:21:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.061+0000 7f63aac00700 1 -- 172.21.15.47:0/3510154294 shutdown_connections 2024-03-21T11:21:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.061+0000 7f63aac00700 1 --2- 172.21.15.47:0/3510154294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63a41096c0 0x7f63a4111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.061+0000 7f63aac00700 1 --2- 172.21.15.47:0/3510154294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63a41045e0 0x7f63a41049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.061+0000 7f63aac00700 1 --2- 172.21.15.47:0/3510154294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a4104f90 0x7f63a4108fe0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.061+0000 7f63aac00700 1 -- 172.21.15.47:0/3510154294 >> 172.21.15.47:0/3510154294 conn(0x7f63a40ffe70 msgr2=0x7f63a4102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.061+0000 7f63aac00700 1 -- 172.21.15.47:0/3510154294 shutdown_connections 2024-03-21T11:21:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.061+0000 7f63aac00700 1 -- 172.21.15.47:0/3510154294 wait complete. 2024-03-21T11:21:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.063+0000 7f63aac00700 1 Processor -- start 2024-03-21T11:21:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.063+0000 7f63aac00700 1 -- start start 2024-03-21T11:21:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.064+0000 7f63aac00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a41045e0 0x7f63a41a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.064+0000 7f63aac00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63a4104f90 0x7f63a41a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.064+0000 7f63aac00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63a41096c0 0x7f63a419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.064+0000 7f63aac00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a41146d0 con 0x7f63a4104f90 2024-03-21T11:21:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.064+0000 7f63aac00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a4114550 con 0x7f63a41045e0 2024-03-21T11:21:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.064+0000 7f63aac00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a4114850 con 0x7f63a41096c0 2024-03-21T11:21:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.064+0000 7f63a3e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a41045e0 0x7f63a41a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.064+0000 7f63a3e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a41045e0 0x7f63a41a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46436/0 (socket says 172.21.15.47:46436) 2024-03-21T11:21:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.064+0000 7f63a3e00700 1 -- 172.21.15.47:0/3957179591 learned_addr learned my addr 172.21.15.47:0/3957179591 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.064+0000 7f63a3400700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63a4104f90 0x7f63a41a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.064+0000 7f63a8c00700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63a41096c0 0x7f63a419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.064+0000 7f63a3e00700 1 -- 172.21.15.47:0/3957179591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63a41096c0 msgr2=0x7f63a419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.065+0000 7f63a3e00700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63a41096c0 0x7f63a419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.065+0000 7f63a3e00700 1 -- 172.21.15.47:0/3957179591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63a4104f90 msgr2=0x7f63a41a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.065+0000 7f63a3e00700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63a4104f90 0x7f63a41a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.065+0000 7f63a3e00700 1 -- 172.21.15.47:0/3957179591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a419b0d0 con 0x7f63a41045e0 2024-03-21T11:21:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.065+0000 7f63a3400700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63a4104f90 0x7f63a41a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:21:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.065+0000 7f63a3e00700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a41045e0 0x7f63a41a0760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f639400e9a0 tx=0x7f639400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.065+0000 7f63a0c00700 1 -- 172.21.15.47:0/3957179591 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639400cbd0 con 0x7f63a41045e0 2024-03-21T11:21:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.065+0000 7f63a0c00700 1 -- 172.21.15.47:0/3957179591 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f639400cd70 con 0x7f63a41045e0 2024-03-21T11:21:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.065+0000 7f63a0c00700 1 -- 172.21.15.47:0/3957179591 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394018750 con 0x7f63a41045e0 2024-03-21T11:21:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.065+0000 7f63aac00700 1 -- 172.21.15.47:0/3957179591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a419b3c0 con 0x7f63a41045e0 2024-03-21T11:21:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.065+0000 7f63aac00700 1 -- 172.21.15.47:0/3957179591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a41a7630 con 0x7f63a41045e0 2024-03-21T11:21:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.066+0000 7f63aac00700 1 -- 172.21.15.47:0/3957179591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f63a40628e0 con 0x7f63a41045e0 2024-03-21T11:21:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.068+0000 7f63a0c00700 1 -- 172.21.15.47:0/3957179591 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f6394010430 con 0x7f63a41045e0 2024-03-21T11:21:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.070+0000 7f63a0c00700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f63880a30c0 0x7f63880a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.070+0000 7f63a0c00700 1 -- 172.21.15.47:0/3957179591 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f63940cfbb0 con 0x7f63a41045e0 2024-03-21T11:21:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.070+0000 7f63a3400700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f63880a30c0 0x7f63880a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.070+0000 7f63a0c00700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491] conn(0x7f63880ae220 0x7f63880b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.070+0000 7f63a0c00700 1 -- 172.21.15.47:0/3957179591 --> [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f63880b0d90 con 0x7f63880ae220 2024-03-21T11:21:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.070+0000 7f63a0c00700 1 -- 172.21.15.47:0/3957179591 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63940d42a0 con 0x7f63a41045e0 2024-03-21T11:21:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.071+0000 7f63a8c00700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491] conn(0x7f63880ae220 0x7f63880b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.071+0000 7f63a3400700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f63880a30c0 0x7f63880a5580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6390006010 tx=0x7f639001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.071+0000 7f63a8c00700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491] conn(0x7f63880ae220 0x7f63880b06c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.072+0000 7f63a0c00700 1 -- 172.21.15.47:0/3957179591 <== osd.6 v2:172.21.15.47:6818/2684410491 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f63880b0d90 con 0x7f63880ae220 2024-03-21T11:21:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.091+0000 7f63aac00700 1 -- 172.21.15.47:0/3957179591 --> [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f63a4062d30 con 0x7f63880ae220 2024-03-21T11:21:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.092+0000 7f63a0c00700 1 -- 172.21.15.47:0/3957179591 <== osd.6 v2:172.21.15.47:6818/2684410491 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f63a4062d30 con 0x7f63880ae220 2024-03-21T11:21:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.093+0000 7f63aac00700 1 -- 172.21.15.47:0/3957179591 >> [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491] conn(0x7f63880ae220 msgr2=0x7f63880b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.093+0000 7f63aac00700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491] conn(0x7f63880ae220 0x7f63880b06c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.093+0000 7f63aac00700 1 -- 172.21.15.47:0/3957179591 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f63880a30c0 msgr2=0x7f63880a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.093+0000 7f63aac00700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f63880a30c0 0x7f63880a5580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6390006010 tx=0x7f639001a040 comp rx=0 tx=0).stop 2024-03-21T11:21:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.093+0000 7f63aac00700 1 -- 172.21.15.47:0/3957179591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a41045e0 msgr2=0x7f63a41a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.093+0000 7f63aac00700 1 --2- 172.21.15.47:0/3957179591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a41045e0 0x7f63a41a0760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f639400e9a0 tx=0x7f639400ecb0 comp rx=0 tx=0).stop 2024-03-21T11:21:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.093+0000 7f63a8c00700 1 -- 172.21.15.47:0/3957179591 reap_dead start 2024-03-21T11:21:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.093+0000 7f63aac00700 1 -- 172.21.15.47:0/3957179591 shutdown_connections 2024-03-21T11:21:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.093+0000 7f63aac00700 1 -- 172.21.15.47:0/3957179591 >> 172.21.15.47:0/3957179591 conn(0x7f63a40ffe70 msgr2=0x7f63a41009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.093+0000 7f63aac00700 1 -- 172.21.15.47:0/3957179591 shutdown_connections 2024-03-21T11:21:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:53.093+0000 7f63aac00700 1 -- 172.21.15.47:0/3957179591 wait complete. 2024-03-21T11:21:55.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:54 smithi047 ceph-mon[31351]: pgmap v355: 129 pgs: 3 peering, 126 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T11:21:55.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:54 smithi100 ceph-mon[36245]: pgmap v355: 129 pgs: 3 peering, 126 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T11:21:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.694+0000 7fe67e200700 1 -- 172.21.15.47:0/1655880904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6781045e0 msgr2=0x7fe6781049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.694+0000 7fe67e200700 1 --2- 172.21.15.47:0/1655880904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6781045e0 0x7fe6781049c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe66800b0a0 tx=0x7fe66800b3b0 comp rx=0 tx=0).stop 2024-03-21T11:21:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.694+0000 7fe67e200700 1 -- 172.21.15.47:0/1655880904 shutdown_connections 2024-03-21T11:21:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.694+0000 7fe67e200700 1 --2- 172.21.15.47:0/1655880904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6781096c0 0x7fe678111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.694+0000 7fe67e200700 1 --2- 172.21.15.47:0/1655880904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6781045e0 0x7fe6781049c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.694+0000 7fe67e200700 1 --2- 172.21.15.47:0/1655880904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe678104f90 0x7fe678108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.694+0000 7fe67e200700 1 -- 172.21.15.47:0/1655880904 >> 172.21.15.47:0/1655880904 conn(0x7fe6780ffe70 msgr2=0x7fe678102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.694+0000 7fe67e200700 1 -- 172.21.15.47:0/1655880904 shutdown_connections 2024-03-21T11:21:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.694+0000 7fe67e200700 1 -- 172.21.15.47:0/1655880904 wait complete. 2024-03-21T11:21:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.696+0000 7fe67e200700 1 Processor -- start 2024-03-21T11:21:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.696+0000 7fe67e200700 1 -- start start 2024-03-21T11:21:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.696+0000 7fe67e200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6781045e0 0x7fe6781a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.696+0000 7fe67e200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe678104f90 0x7fe6781a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.696+0000 7fe67e200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6781096c0 0x7fe67819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.696+0000 7fe67e200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe678114710 con 0x7fe678104f90 2024-03-21T11:21:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.696+0000 7fe67e200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe678114590 con 0x7fe6781045e0 2024-03-21T11:21:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.696+0000 7fe67e200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe678114890 con 0x7fe6781096c0 2024-03-21T11:21:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.697+0000 7fe677e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6781096c0 0x7fe67819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.697+0000 7fe677400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6781045e0 0x7fe6781a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.697+0000 7fe677e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6781096c0 0x7fe67819a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52044/0 (socket says 172.21.15.47:52044) 2024-03-21T11:21:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.697+0000 7fe677e00700 1 -- 172.21.15.47:0/1539116388 learned_addr learned my addr 172.21.15.47:0/1539116388 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.697+0000 7fe677400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6781045e0 0x7fe6781a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46462/0 (socket says 172.21.15.47:46462) 2024-03-21T11:21:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.697+0000 7fe66ea00700 1 --2- 172.21.15.47:0/1539116388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe678104f90 0x7fe6781a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.697+0000 7fe677e00700 1 -- 172.21.15.47:0/1539116388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6781045e0 msgr2=0x7fe6781a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.697+0000 7fe677e00700 1 --2- 172.21.15.47:0/1539116388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6781045e0 0x7fe6781a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.697+0000 7fe677e00700 1 -- 172.21.15.47:0/1539116388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe678104f90 msgr2=0x7fe6781a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.697+0000 7fe677e00700 1 --2- 172.21.15.47:0/1539116388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe678104f90 0x7fe6781a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.697+0000 7fe677e00700 1 -- 172.21.15.47:0/1539116388 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67819b070 con 0x7fe6781096c0 2024-03-21T11:21:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.697+0000 7fe677400700 1 --2- 172.21.15.47:0/1539116388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6781045e0 0x7fe6781a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:21:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.697+0000 7fe677e00700 1 --2- 172.21.15.47:0/1539116388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6781096c0 0x7fe67819a910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe67000e940 tx=0x7fe67000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.698+0000 7fe674c00700 1 -- 172.21.15.47:0/1539116388 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670004a30 con 0x7fe6781096c0 2024-03-21T11:21:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.698+0000 7fe674c00700 1 -- 172.21.15.47:0/1539116388 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe670005500 con 0x7fe6781096c0 2024-03-21T11:21:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.698+0000 7fe674c00700 1 -- 172.21.15.47:0/1539116388 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67000ce20 con 0x7fe6781096c0 2024-03-21T11:21:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.698+0000 7fe67e200700 1 -- 172.21.15.47:0/1539116388 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67819b360 con 0x7fe6781096c0 2024-03-21T11:21:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.698+0000 7fe67e200700 1 -- 172.21.15.47:0/1539116388 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6781a7620 con 0x7fe6781096c0 2024-03-21T11:21:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.699+0000 7fe67e200700 1 -- 172.21.15.47:0/1539116388 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe678062920 con 0x7fe6781096c0 2024-03-21T11:21:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.701+0000 7fe674c00700 1 -- 172.21.15.47:0/1539116388 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fe670005000 con 0x7fe6781096c0 2024-03-21T11:21:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.702+0000 7fe674c00700 1 --2- 172.21.15.47:0/1539116388 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe65c0a3040 0x7fe65c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.702+0000 7fe674c00700 1 -- 172.21.15.47:0/1539116388 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fe6700cfe20 con 0x7fe6781096c0 2024-03-21T11:21:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.703+0000 7fe674c00700 1 --2- 172.21.15.47:0/1539116388 >> [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917] conn(0x7fe65c0ae1a0 0x7fe65c0b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.703+0000 7fe674c00700 1 -- 172.21.15.47:0/1539116388 --> [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe65c0b0d10 con 0x7fe65c0ae1a0 2024-03-21T11:21:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.703+0000 7fe677400700 1 --2- 172.21.15.47:0/1539116388 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe65c0a3040 0x7fe65c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.703+0000 7fe674c00700 1 -- 172.21.15.47:0/1539116388 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6700d01d0 con 0x7fe6781096c0 2024-03-21T11:21:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.703+0000 7fe66ea00700 1 --2- 172.21.15.47:0/1539116388 >> [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917] conn(0x7fe65c0ae1a0 0x7fe65c0b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.703+0000 7fe677400700 1 --2- 172.21.15.47:0/1539116388 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe65c0a3040 0x7fe65c0a5500 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe668009fd0 tx=0x7fe66800b740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.704+0000 7fe66ea00700 1 --2- 172.21.15.47:0/1539116388 >> [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917] conn(0x7fe65c0ae1a0 0x7fe65c0b0640 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.704+0000 7fe674c00700 1 -- 172.21.15.47:0/1539116388 <== osd.8 v2:172.21.15.102:6816/1509099917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe65c0b0d10 con 0x7fe65c0ae1a0 2024-03-21T11:21:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.722+0000 7fe67e200700 1 -- 172.21.15.47:0/1539116388 --> [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe678062d20 con 0x7fe65c0ae1a0 2024-03-21T11:21:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.723+0000 7fe674c00700 1 -- 172.21.15.47:0/1539116388 <== osd.8 v2:172.21.15.102:6816/1509099917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe678062d20 con 0x7fe65c0ae1a0 2024-03-21T11:21:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.723+0000 7fe67e200700 1 -- 172.21.15.47:0/1539116388 >> [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917] conn(0x7fe65c0ae1a0 msgr2=0x7fe65c0b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.723+0000 7fe67e200700 1 --2- 172.21.15.47:0/1539116388 >> [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917] conn(0x7fe65c0ae1a0 0x7fe65c0b0640 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.723+0000 7fe67e200700 1 -- 172.21.15.47:0/1539116388 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe65c0a3040 msgr2=0x7fe65c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.723+0000 7fe67e200700 1 --2- 172.21.15.47:0/1539116388 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe65c0a3040 0x7fe65c0a5500 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe668009fd0 tx=0x7fe66800b740 comp rx=0 tx=0).stop 2024-03-21T11:21:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.723+0000 7fe67e200700 1 -- 172.21.15.47:0/1539116388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6781096c0 msgr2=0x7fe67819a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.723+0000 7fe67e200700 1 --2- 172.21.15.47:0/1539116388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6781096c0 0x7fe67819a910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe67000e940 tx=0x7fe67000ec50 comp rx=0 tx=0).stop 2024-03-21T11:21:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.724+0000 7fe677e00700 1 -- 172.21.15.47:0/1539116388 reap_dead start 2024-03-21T11:21:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.724+0000 7fe67e200700 1 -- 172.21.15.47:0/1539116388 shutdown_connections 2024-03-21T11:21:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.724+0000 7fe67e200700 1 -- 172.21.15.47:0/1539116388 >> 172.21.15.47:0/1539116388 conn(0x7fe6780ffe70 msgr2=0x7fe678100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.724+0000 7fe67e200700 1 -- 172.21.15.47:0/1539116388 shutdown_connections 2024-03-21T11:21:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.724+0000 7fe67e200700 1 -- 172.21.15.47:0/1539116388 wait complete. 2024-03-21T11:21:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:54 smithi102 ceph-mon[36748]: pgmap v355: 129 pgs: 3 peering, 126 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T11:21:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.790+0000 7f67e0800700 1 -- 172.21.15.47:0/2575959891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67d81045e0 msgr2=0x7f67d81049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.790+0000 7f67e0800700 1 --2- 172.21.15.47:0/2575959891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67d81045e0 0x7f67d81049c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009920 tx=0x7f67cc009c30 comp rx=0 tx=0).stop 2024-03-21T11:21:56.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.790+0000 7f67e0800700 1 -- 172.21.15.47:0/2575959891 shutdown_connections 2024-03-21T11:21:56.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.790+0000 7f67e0800700 1 --2- 172.21.15.47:0/2575959891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d81096c0 0x7f67d8111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:56.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.790+0000 7f67e0800700 1 --2- 172.21.15.47:0/2575959891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67d81045e0 0x7f67d81049c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:56.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.790+0000 7f67e0800700 1 --2- 172.21.15.47:0/2575959891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d8104f90 0x7f67d8108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:56.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.790+0000 7f67e0800700 1 -- 172.21.15.47:0/2575959891 >> 172.21.15.47:0/2575959891 conn(0x7f67d80ffe90 msgr2=0x7f67d81022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:56.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.790+0000 7f67e0800700 1 -- 172.21.15.47:0/2575959891 shutdown_connections 2024-03-21T11:21:56.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.790+0000 7f67e0800700 1 -- 172.21.15.47:0/2575959891 wait complete. 2024-03-21T11:21:56.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.791+0000 7f67e0800700 1 Processor -- start 2024-03-21T11:21:56.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.791+0000 7f67e0800700 1 -- start start 2024-03-21T11:21:56.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.791+0000 7f67e0800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d81045e0 0x7f67d81a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:56.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.791+0000 7f67e0800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d8104f90 0x7f67d81a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:56.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67e0800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67d81096c0 0x7f67d819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:56.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67e0800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d81146f0 con 0x7f67d81045e0 2024-03-21T11:21:56.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67e0800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d8114570 con 0x7f67d8104f90 2024-03-21T11:21:56.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67e0800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d8114870 con 0x7f67d81096c0 2024-03-21T11:21:56.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67dde00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d81045e0 0x7f67d81a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:56.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67dd400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d8104f90 0x7f67d81a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:56.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67dde00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d81045e0 0x7f67d81a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50010/0 (socket says 172.21.15.47:50010) 2024-03-21T11:21:56.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67dd400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d8104f90 0x7f67d81a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46486/0 (socket says 172.21.15.47:46486) 2024-03-21T11:21:56.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67dde00700 1 -- 172.21.15.47:0/460250491 learned_addr learned my addr 172.21.15.47:0/460250491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:56.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67de800700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67d81096c0 0x7f67d819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:56.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67dd400700 1 -- 172.21.15.47:0/460250491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67d81096c0 msgr2=0x7f67d819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:56.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67dd400700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67d81096c0 0x7f67d819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:56.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67dd400700 1 -- 172.21.15.47:0/460250491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d81045e0 msgr2=0x7f67d81a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:56.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67dd400700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d81045e0 0x7f67d81a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:56.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67dd400700 1 -- 172.21.15.47:0/460250491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d819b170 con 0x7f67d8104f90 2024-03-21T11:21:56.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67dde00700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d81045e0 0x7f67d81a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:21:56.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.792+0000 7f67de800700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67d81096c0 0x7f67d819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:21:56.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.793+0000 7f67dd400700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d8104f90 0x7f67d81a0d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f67c8005fd0 tx=0x7f67c800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:56.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.793+0000 7f67d2a00700 1 -- 172.21.15.47:0/460250491 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c800cbd0 con 0x7f67d8104f90 2024-03-21T11:21:56.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.793+0000 7f67d2a00700 1 -- 172.21.15.47:0/460250491 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67c8004500 con 0x7f67d8104f90 2024-03-21T11:21:56.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.793+0000 7f67e0800700 1 -- 172.21.15.47:0/460250491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d819b460 con 0x7f67d8104f90 2024-03-21T11:21:56.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.793+0000 7f67d2a00700 1 -- 172.21.15.47:0/460250491 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c8010470 con 0x7f67d8104f90 2024-03-21T11:21:56.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.793+0000 7f67e0800700 1 -- 172.21.15.47:0/460250491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d81a7720 con 0x7f67d8104f90 2024-03-21T11:21:56.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.794+0000 7f67e0800700 1 -- 172.21.15.47:0/460250491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f67d8062920 con 0x7f67d8104f90 2024-03-21T11:21:56.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.796+0000 7f67d2a00700 1 -- 172.21.15.47:0/460250491 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f67c8004000 con 0x7f67d8104f90 2024-03-21T11:21:56.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.797+0000 7f67d2a00700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f67c00a30c0 0x7f67c00a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:56.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.797+0000 7f67d2a00700 1 -- 172.21.15.47:0/460250491 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f67c80cfd80 con 0x7f67d8104f90 2024-03-21T11:21:56.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.798+0000 7f67dde00700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f67c00a30c0 0x7f67c00a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:56.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.798+0000 7f67d2a00700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052] conn(0x7f67c00ae220 0x7f67c00b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:56.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.798+0000 7f67d2a00700 1 -- 172.21.15.47:0/460250491 --> [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67c8012ec0 con 0x7f67c00ae220 2024-03-21T11:21:56.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.798+0000 7f67d2a00700 1 -- 172.21.15.47:0/460250491 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67c80d0130 con 0x7f67d8104f90 2024-03-21T11:21:56.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.798+0000 7f67de800700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052] conn(0x7f67c00ae220 0x7f67c00b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:56.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.798+0000 7f67dde00700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f67c00a30c0 0x7f67c00a5580 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f67cc011040 tx=0x7f67cc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:56.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.798+0000 7f67de800700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052] conn(0x7f67c00ae220 0x7f67c00b06c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:56.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.799+0000 7f67d2a00700 1 -- 172.21.15.47:0/460250491 <== osd.5 v2:172.21.15.102:6808/2128847052 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f67c8012ec0 con 0x7f67c00ae220 2024-03-21T11:21:56.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.817+0000 7f67e0800700 1 -- 172.21.15.47:0/460250491 --> [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f67d8062d20 con 0x7f67c00ae220 2024-03-21T11:21:56.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.817+0000 7f67d2a00700 1 -- 172.21.15.47:0/460250491 <== osd.5 v2:172.21.15.102:6808/2128847052 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f67d8062d20 con 0x7f67c00ae220 2024-03-21T11:21:56.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.818+0000 7f67e0800700 1 -- 172.21.15.47:0/460250491 >> [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052] conn(0x7f67c00ae220 msgr2=0x7f67c00b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:56.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.818+0000 7f67e0800700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052] conn(0x7f67c00ae220 0x7f67c00b06c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:56.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.818+0000 7f67e0800700 1 -- 172.21.15.47:0/460250491 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f67c00a30c0 msgr2=0x7f67c00a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:56.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.818+0000 7f67e0800700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f67c00a30c0 0x7f67c00a5580 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f67cc011040 tx=0x7f67cc019040 comp rx=0 tx=0).stop 2024-03-21T11:21:56.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.818+0000 7f67e0800700 1 -- 172.21.15.47:0/460250491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d8104f90 msgr2=0x7f67d81a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:56.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.818+0000 7f67e0800700 1 --2- 172.21.15.47:0/460250491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d8104f90 0x7f67d81a0d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f67c8005fd0 tx=0x7f67c800ee70 comp rx=0 tx=0).stop 2024-03-21T11:21:56.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.818+0000 7f67de800700 1 -- 172.21.15.47:0/460250491 reap_dead start 2024-03-21T11:21:56.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.818+0000 7f67e0800700 1 -- 172.21.15.47:0/460250491 shutdown_connections 2024-03-21T11:21:56.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.818+0000 7f67e0800700 1 -- 172.21.15.47:0/460250491 >> 172.21.15.47:0/460250491 conn(0x7f67d80ffe90 msgr2=0x7f67d81012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:56.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.818+0000 7f67e0800700 1 -- 172.21.15.47:0/460250491 shutdown_connections 2024-03-21T11:21:56.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:54.818+0000 7f67e0800700 1 -- 172.21.15.47:0/460250491 wait complete. 2024-03-21T11:21:57.207 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:56 smithi047 ceph-mon[31351]: pgmap v356: 129 pgs: 3 peering, 126 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T11:21:57.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:56 smithi102 ceph-mon[36748]: pgmap v356: 129 pgs: 3 peering, 126 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T11:21:57.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:56 smithi100 ceph-mon[36245]: pgmap v356: 129 pgs: 3 peering, 126 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T11:21:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.666+0000 7f99f2200700 1 -- 172.21.15.47:0/3433428055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99ec1081e0 msgr2=0x7f99ec1085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.666+0000 7f99f2200700 1 --2- 172.21.15.47:0/3433428055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99ec1081e0 0x7f99ec1085c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0099c0 tx=0x7f99dc009cd0 comp rx=0 tx=0).stop 2024-03-21T11:21:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.666+0000 7f99f2200700 1 -- 172.21.15.47:0/3433428055 shutdown_connections 2024-03-21T11:21:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.666+0000 7f99f2200700 1 --2- 172.21.15.47:0/3433428055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ec06a7e0 0x7f99ec10f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.666+0000 7f99f2200700 1 --2- 172.21.15.47:0/3433428055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99ec1081e0 0x7f99ec1085c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.666+0000 7f99f2200700 1 --2- 172.21.15.47:0/3433428055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99ec069e40 0x7f99ec06a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.666+0000 7f99f2200700 1 -- 172.21.15.47:0/3433428055 >> 172.21.15.47:0/3433428055 conn(0x7f99ec077fc0 msgr2=0x7f99ec0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.667+0000 7f99f2200700 1 -- 172.21.15.47:0/3433428055 shutdown_connections 2024-03-21T11:21:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.667+0000 7f99f2200700 1 -- 172.21.15.47:0/3433428055 wait complete. 2024-03-21T11:21:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.669+0000 7f99f2200700 1 Processor -- start 2024-03-21T11:21:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.669+0000 7f99f2200700 1 -- start start 2024-03-21T11:21:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.669+0000 7f99f2200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99ec069e40 0x7f99ec10e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.669+0000 7f99f2200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ec06a7e0 0x7f99ec10e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.669+0000 7f99f2200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99ec1081e0 0x7f99ec1a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.670+0000 7f99f2200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec112490 con 0x7f99ec1081e0 2024-03-21T11:21:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.670+0000 7f99f2200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec112310 con 0x7f99ec06a7e0 2024-03-21T11:21:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.670+0000 7f99f2200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec112610 con 0x7f99ec069e40 2024-03-21T11:21:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.670+0000 7f99eb400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99ec069e40 0x7f99ec10e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.670+0000 7f99eb400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99ec069e40 0x7f99ec10e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52112/0 (socket says 172.21.15.47:52112) 2024-03-21T11:21:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.670+0000 7f99eb400700 1 -- 172.21.15.47:0/974204900 learned_addr learned my addr 172.21.15.47:0/974204900 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.670+0000 7f99eaa00700 1 --2- 172.21.15.47:0/974204900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ec06a7e0 0x7f99ec10e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.670+0000 7f99ebe00700 1 --2- 172.21.15.47:0/974204900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99ec1081e0 0x7f99ec1a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.671+0000 7f99eb400700 1 -- 172.21.15.47:0/974204900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ec06a7e0 msgr2=0x7f99ec10e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.671+0000 7f99eb400700 1 --2- 172.21.15.47:0/974204900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99ec06a7e0 0x7f99ec10e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.671+0000 7f99eb400700 1 -- 172.21.15.47:0/974204900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99ec1081e0 msgr2=0x7f99ec1a3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.671+0000 7f99eb400700 1 --2- 172.21.15.47:0/974204900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99ec1081e0 0x7f99ec1a3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.671+0000 7f99eb400700 1 -- 172.21.15.47:0/974204900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec1a4020 con 0x7f99ec069e40 2024-03-21T11:21:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.671+0000 7f99eb400700 1 --2- 172.21.15.47:0/974204900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99ec069e40 0x7f99ec10e3d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f99dc006010 tx=0x7f99dc003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.671+0000 7f99e3e00700 1 -- 172.21.15.47:0/974204900 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc0050c0 con 0x7f99ec069e40 2024-03-21T11:21:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.671+0000 7f99f2200700 1 -- 172.21.15.47:0/974204900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ec1a42b0 con 0x7f99ec069e40 2024-03-21T11:21:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.671+0000 7f99f2200700 1 -- 172.21.15.47:0/974204900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ec1abb90 con 0x7f99ec069e40 2024-03-21T11:21:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.672+0000 7f99e3e00700 1 -- 172.21.15.47:0/974204900 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99dc00f460 con 0x7f99ec069e40 2024-03-21T11:21:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.672+0000 7f99e3e00700 1 -- 172.21.15.47:0/974204900 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc017650 con 0x7f99ec069e40 2024-03-21T11:21:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.673+0000 7f99f2200700 1 -- 172.21.15.47:0/974204900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f99cc001030 con 0x7f99ec069e40 2024-03-21T11:21:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.675+0000 7f99e3e00700 1 -- 172.21.15.47:0/974204900 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f99dc04eb80 con 0x7f99ec069e40 2024-03-21T11:21:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.678+0000 7f99e3e00700 1 --2- 172.21.15.47:0/974204900 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f99d00a30c0 0x7f99d00a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.678+0000 7f99eaa00700 1 --2- 172.21.15.47:0/974204900 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f99d00a30c0 0x7f99d00a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.679+0000 7f99e3e00700 1 -- 172.21.15.47:0/974204900 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f99dc0d0520 con 0x7f99ec069e40 2024-03-21T11:21:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.679+0000 7f99eaa00700 1 --2- 172.21.15.47:0/974204900 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f99d00a30c0 0x7f99d00a5580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f99ec1030f0 tx=0x7f99d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.680+0000 7f99e3e00700 1 --2- 172.21.15.47:0/974204900 >> [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551] conn(0x7f99d00ae220 0x7f99d00b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.680+0000 7f99e3e00700 1 -- 172.21.15.47:0/974204900 --> [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99dc009030 con 0x7f99d00ae220 2024-03-21T11:21:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.680+0000 7f99e3e00700 1 -- 172.21.15.47:0/974204900 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99dc0d08d0 con 0x7f99ec069e40 2024-03-21T11:21:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.680+0000 7f99ebe00700 1 --2- 172.21.15.47:0/974204900 >> [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551] conn(0x7f99d00ae220 0x7f99d00b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.681+0000 7f99ebe00700 1 --2- 172.21.15.47:0/974204900 >> [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551] conn(0x7f99d00ae220 0x7f99d00b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.681+0000 7f99e3e00700 1 -- 172.21.15.47:0/974204900 <== osd.7 v2:172.21.15.100:6816/3018830551 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f99dc009030 con 0x7f99d00ae220 2024-03-21T11:21:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.703+0000 7f99f2200700 1 -- 172.21.15.47:0/974204900 --> [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f99cc002e60 con 0x7f99d00ae220 2024-03-21T11:21:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.703+0000 7f99e3e00700 1 -- 172.21.15.47:0/974204900 <== osd.7 v2:172.21.15.100:6816/3018830551 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f99cc002e60 con 0x7f99d00ae220 2024-03-21T11:21:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.704+0000 7f99f2200700 1 -- 172.21.15.47:0/974204900 >> [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551] conn(0x7f99d00ae220 msgr2=0x7f99d00b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.704+0000 7f99f2200700 1 --2- 172.21.15.47:0/974204900 >> [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551] conn(0x7f99d00ae220 0x7f99d00b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.704+0000 7f99f2200700 1 -- 172.21.15.47:0/974204900 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f99d00a30c0 msgr2=0x7f99d00a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.704+0000 7f99f2200700 1 --2- 172.21.15.47:0/974204900 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f99d00a30c0 0x7f99d00a5580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f99ec1030f0 tx=0x7f99d8009260 comp rx=0 tx=0).stop 2024-03-21T11:21:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.704+0000 7f99f2200700 1 -- 172.21.15.47:0/974204900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99ec069e40 msgr2=0x7f99ec10e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.704+0000 7f99f2200700 1 --2- 172.21.15.47:0/974204900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99ec069e40 0x7f99ec10e3d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f99dc006010 tx=0x7f99dc003a90 comp rx=0 tx=0).stop 2024-03-21T11:21:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.704+0000 7f99ebe00700 1 -- 172.21.15.47:0/974204900 reap_dead start 2024-03-21T11:21:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.704+0000 7f99f2200700 1 -- 172.21.15.47:0/974204900 shutdown_connections 2024-03-21T11:21:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.704+0000 7f99f2200700 1 -- 172.21.15.47:0/974204900 >> 172.21.15.47:0/974204900 conn(0x7f99ec077fc0 msgr2=0x7f99ec0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.704+0000 7f99f2200700 1 -- 172.21.15.47:0/974204900 shutdown_connections 2024-03-21T11:21:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.705+0000 7f99f2200700 1 -- 172.21.15.47:0/974204900 wait complete. 2024-03-21T11:21:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:55834574931 2024-03-21T11:21:57.848 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-21T11:21:58.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.562+0000 7f94f8200700 1 -- 172.21.15.47:0/4211217458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f0074ce0 msgr2=0x7f94f010e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:58.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.562+0000 7f94f8200700 1 --2- 172.21.15.47:0/4211217458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f0074ce0 0x7f94f010e650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009990 tx=0x7f94dc009ca0 comp rx=0 tx=0).stop 2024-03-21T11:21:58.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.562+0000 7f94f8200700 1 -- 172.21.15.47:0/4211217458 shutdown_connections 2024-03-21T11:21:58.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.562+0000 7f94f8200700 1 --2- 172.21.15.47:0/4211217458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f010eb90 0x7f94f0110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:58.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.562+0000 7f94f8200700 1 --2- 172.21.15.47:0/4211217458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94f00743c0 0x7f94f00747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:58.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.562+0000 7f94f8200700 1 --2- 172.21.15.47:0/4211217458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f0074ce0 0x7f94f010e650 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:58.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.562+0000 7f94f8200700 1 -- 172.21.15.47:0/4211217458 >> 172.21.15.47:0/4211217458 conn(0x7f94f00792c0 msgr2=0x7f94f00796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:58.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.562+0000 7f94f8200700 1 -- 172.21.15.47:0/4211217458 shutdown_connections 2024-03-21T11:21:58.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.562+0000 7f94f8200700 1 -- 172.21.15.47:0/4211217458 wait complete. 2024-03-21T11:21:58.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.563+0000 7f94f8200700 1 Processor -- start 2024-03-21T11:21:58.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.563+0000 7f94f8200700 1 -- start start 2024-03-21T11:21:58.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f8200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f00743c0 0x7f94f01a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:58.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f8200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94f0074ce0 0x7f94f01a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:58.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f8200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f010eb90 0x7f94f019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:58.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f8200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0114690 con 0x7f94f0074ce0 2024-03-21T11:21:58.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f8200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0114510 con 0x7f94f00743c0 2024-03-21T11:21:58.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f8200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0114810 con 0x7f94f010eb90 2024-03-21T11:21:58.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f5800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f00743c0 0x7f94f01a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:58.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f5800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f00743c0 0x7f94f01a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46500/0 (socket says 172.21.15.47:46500) 2024-03-21T11:21:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f5800700 1 -- 172.21.15.47:0/1924578179 learned_addr learned my addr 172.21.15.47:0/1924578179 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:21:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f6200700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f010eb90 0x7f94f019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f4e00700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94f0074ce0 0x7f94f01a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f5800700 1 -- 172.21.15.47:0/1924578179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f010eb90 msgr2=0x7f94f019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f5800700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f010eb90 0x7f94f019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f5800700 1 -- 172.21.15.47:0/1924578179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94f0074ce0 msgr2=0x7f94f01a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f5800700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94f0074ce0 0x7f94f01a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.564+0000 7f94f5800700 1 -- 172.21.15.47:0/1924578179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f019b0f0 con 0x7f94f00743c0 2024-03-21T11:21:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.565+0000 7f94f6200700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f010eb90 0x7f94f019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:21:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.565+0000 7f94f5800700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f00743c0 0x7f94f01a0780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f94e400d950 tx=0x7f94e400dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.565+0000 7f94ee000700 1 -- 172.21.15.47:0/1924578179 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4016070 con 0x7f94f00743c0 2024-03-21T11:21:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.565+0000 7f94ee000700 1 -- 172.21.15.47:0/1924578179 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94e4009b40 con 0x7f94f00743c0 2024-03-21T11:21:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.565+0000 7f94f8200700 1 -- 172.21.15.47:0/1924578179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f019b3e0 con 0x7f94f00743c0 2024-03-21T11:21:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.565+0000 7f94ee000700 1 -- 172.21.15.47:0/1924578179 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e400f620 con 0x7f94f00743c0 2024-03-21T11:21:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.565+0000 7f94f8200700 1 -- 172.21.15.47:0/1924578179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f01a7650 con 0x7f94f00743c0 2024-03-21T11:21:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.566+0000 7f94f8200700 1 -- 172.21.15.47:0/1924578179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94f00628e0 con 0x7f94f00743c0 2024-03-21T11:21:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.568+0000 7f94ee000700 1 -- 172.21.15.47:0/1924578179 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f94e4019070 con 0x7f94f00743c0 2024-03-21T11:21:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.569+0000 7f94ee000700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f94d80a30c0 0x7f94d80a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.570+0000 7f94ee000700 1 -- 172.21.15.47:0/1924578179 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f94e40cf8e0 con 0x7f94f00743c0 2024-03-21T11:21:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.570+0000 7f94f4e00700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f94d80a30c0 0x7f94d80a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.570+0000 7f94ee000700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355] conn(0x7f94d80ae220 0x7f94d80b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:21:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.570+0000 7f94ee000700 1 -- 172.21.15.47:0/1924578179 --> [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94d80b0d90 con 0x7f94d80ae220 2024-03-21T11:21:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.570+0000 7f94ee000700 1 -- 172.21.15.47:0/1924578179 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94e40d42a0 con 0x7f94f00743c0 2024-03-21T11:21:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.570+0000 7f94f6200700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355] conn(0x7f94d80ae220 0x7f94d80b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:21:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.571+0000 7f94f4e00700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f94d80a30c0 0x7f94d80a5580 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00b2e0 tx=0x7f94dc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.571+0000 7f94f6200700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355] conn(0x7f94d80ae220 0x7f94d80b06c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:21:58.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.572+0000 7f94ee000700 1 -- 172.21.15.47:0/1924578179 <== osd.11 v2:172.21.15.102:6824/557348355 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f94d80b0d90 con 0x7f94d80ae220 2024-03-21T11:21:58.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.591+0000 7f94f8200700 1 -- 172.21.15.47:0/1924578179 --> [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f94f0062d10 con 0x7f94d80ae220 2024-03-21T11:21:58.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.591+0000 7f94ee000700 1 -- 172.21.15.47:0/1924578179 <== osd.11 v2:172.21.15.102:6824/557348355 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f94f0062d10 con 0x7f94d80ae220 2024-03-21T11:21:58.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.592+0000 7f94f8200700 1 -- 172.21.15.47:0/1924578179 >> [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355] conn(0x7f94d80ae220 msgr2=0x7f94d80b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:58.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.592+0000 7f94f8200700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355] conn(0x7f94d80ae220 0x7f94d80b06c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:21:58.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.592+0000 7f94f8200700 1 -- 172.21.15.47:0/1924578179 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f94d80a30c0 msgr2=0x7f94d80a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:58.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.592+0000 7f94f8200700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f94d80a30c0 0x7f94d80a5580 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00b2e0 tx=0x7f94dc01a040 comp rx=0 tx=0).stop 2024-03-21T11:21:58.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.592+0000 7f94f8200700 1 -- 172.21.15.47:0/1924578179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f00743c0 msgr2=0x7f94f01a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:21:58.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.592+0000 7f94f8200700 1 --2- 172.21.15.47:0/1924578179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f00743c0 0x7f94f01a0780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f94e400d950 tx=0x7f94e400dd10 comp rx=0 tx=0).stop 2024-03-21T11:21:58.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.592+0000 7f94f6200700 1 -- 172.21.15.47:0/1924578179 reap_dead start 2024-03-21T11:21:58.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.593+0000 7f94f8200700 1 -- 172.21.15.47:0/1924578179 shutdown_connections 2024-03-21T11:21:58.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.593+0000 7f94f8200700 1 -- 172.21.15.47:0/1924578179 >> 172.21.15.47:0/1924578179 conn(0x7f94f00792c0 msgr2=0x7f94f00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:21:58.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.593+0000 7f94f8200700 1 -- 172.21.15.47:0/1924578179 shutdown_connections 2024-03-21T11:21:58.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:57.593+0000 7f94f8200700 1 -- 172.21.15.47:0/1924578179 wait complete. 2024-03-21T11:21:59.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:21:58 smithi102 ceph-mon[36748]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 9 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T11:21:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:21:58 smithi047 ceph-mon[31351]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 9 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T11:21:59.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:21:58 smithi100 ceph-mon[36245]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 9 op/s; 14 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T11:22:00.334 INFO:teuthology.orchestra.run.smithi047.stdout:201863462968 2024-03-21T11:22:00.335 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-21T11:22:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.980+0000 7fca43c00700 1 -- 172.21.15.47:0/3747649310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca3c107510 msgr2=0x7fca3c10f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.980+0000 7fca43c00700 1 --2- 172.21.15.47:0/3747649310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca3c107510 0x7fca3c10f090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fca3400b0a0 tx=0x7fca3400b3b0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.981+0000 7fca43c00700 1 -- 172.21.15.47:0/3747649310 shutdown_connections 2024-03-21T11:22:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.981+0000 7fca43c00700 1 --2- 172.21.15.47:0/3747649310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca3c107510 0x7fca3c10f090 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.981+0000 7fca43c00700 1 --2- 172.21.15.47:0/3747649310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca3c06a930 0x7fca3c06ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.981+0000 7fca43c00700 1 --2- 172.21.15.47:0/3747649310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca3c06b2e0 0x7fca3c106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.981+0000 7fca43c00700 1 -- 172.21.15.47:0/3747649310 >> 172.21.15.47:0/3747649310 conn(0x7fca3c0770e0 msgr2=0x7fca3c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.981+0000 7fca43c00700 1 -- 172.21.15.47:0/3747649310 shutdown_connections 2024-03-21T11:22:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.982+0000 7fca43c00700 1 -- 172.21.15.47:0/3747649310 wait complete. 2024-03-21T11:22:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.983+0000 7fca43c00700 1 Processor -- start 2024-03-21T11:22:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.983+0000 7fca43c00700 1 -- start start 2024-03-21T11:22:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.983+0000 7fca43c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca3c06a930 0x7fca3c101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.983+0000 7fca43c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca3c06b2e0 0x7fca3c1016d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.983+0000 7fca43c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca3c107510 0x7fca3c101c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.983+0000 7fca43c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c112500 con 0x7fca3c06b2e0 2024-03-21T11:22:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.983+0000 7fca43c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c112380 con 0x7fca3c107510 2024-03-21T11:22:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.983+0000 7fca43c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c112680 con 0x7fca3c06a930 2024-03-21T11:22:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.983+0000 7fca41200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca3c06a930 0x7fca3c101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.984+0000 7fca3be00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca3c06b2e0 0x7fca3c1016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.984+0000 7fca41c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca3c107510 0x7fca3c101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.984+0000 7fca41200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca3c06a930 0x7fca3c101190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48168/0 (socket says 172.21.15.47:48168) 2024-03-21T11:22:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.984+0000 7fca41200700 1 -- 172.21.15.47:0/2292023703 learned_addr learned my addr 172.21.15.47:0/2292023703 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.984+0000 7fca41c00700 1 -- 172.21.15.47:0/2292023703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca3c06a930 msgr2=0x7fca3c101190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.984+0000 7fca41c00700 1 --2- 172.21.15.47:0/2292023703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fca3c06a930 0x7fca3c101190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.984+0000 7fca41c00700 1 -- 172.21.15.47:0/2292023703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca3c06b2e0 msgr2=0x7fca3c1016d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.984+0000 7fca41c00700 1 --2- 172.21.15.47:0/2292023703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca3c06b2e0 0x7fca3c1016d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.984+0000 7fca41c00700 1 -- 172.21.15.47:0/2292023703 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c1a6ee0 con 0x7fca3c107510 2024-03-21T11:22:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.984+0000 7fca41c00700 1 --2- 172.21.15.47:0/2292023703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca3c107510 0x7fca3c101c10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fca340062a0 tx=0x7fca34004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.984+0000 7fca39600700 1 -- 172.21.15.47:0/2292023703 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34021070 con 0x7fca3c107510 2024-03-21T11:22:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.984+0000 7fca39600700 1 -- 172.21.15.47:0/2292023703 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca34003c00 con 0x7fca3c107510 2024-03-21T11:22:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.984+0000 7fca39600700 1 -- 172.21.15.47:0/2292023703 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca340045a0 con 0x7fca3c107510 2024-03-21T11:22:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.985+0000 7fca43c00700 1 -- 172.21.15.47:0/2292023703 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c1a7170 con 0x7fca3c107510 2024-03-21T11:22:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.985+0000 7fca43c00700 1 -- 172.21.15.47:0/2292023703 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c1a75d0 con 0x7fca3c107510 2024-03-21T11:22:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.986+0000 7fca43c00700 1 -- 172.21.15.47:0/2292023703 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca3c0628a0 con 0x7fca3c107510 2024-03-21T11:22:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.987+0000 7fca39600700 1 -- 172.21.15.47:0/2292023703 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fca34004740 con 0x7fca3c107510 2024-03-21T11:22:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.989+0000 7fca39600700 1 --2- 172.21.15.47:0/2292023703 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fca240a3040 0x7fca240a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.989+0000 7fca39600700 1 -- 172.21.15.47:0/2292023703 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fca3400f020 con 0x7fca3c107510 2024-03-21T11:22:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.989+0000 7fca41200700 1 --2- 172.21.15.47:0/2292023703 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fca240a3040 0x7fca240a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.989+0000 7fca39600700 1 --2- 172.21.15.47:0/2292023703 >> [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291] conn(0x7fca240ae1a0 0x7fca240b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:00.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.989+0000 7fca39600700 1 -- 172.21.15.47:0/2292023703 --> [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca34009ce0 con 0x7fca240ae1a0 2024-03-21T11:22:00.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.989+0000 7fca39600700 1 -- 172.21.15.47:0/2292023703 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca340d52a0 con 0x7fca3c107510 2024-03-21T11:22:00.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.989+0000 7fca3be00700 1 --2- 172.21.15.47:0/2292023703 >> [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291] conn(0x7fca240ae1a0 0x7fca240b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:00.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.990+0000 7fca41200700 1 --2- 172.21.15.47:0/2292023703 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fca240a3040 0x7fca240a5500 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fca30009d20 tx=0x7fca30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.990+0000 7fca3be00700 1 --2- 172.21.15.47:0/2292023703 >> [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291] conn(0x7fca240ae1a0 0x7fca240b0640 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:21:59.991+0000 7fca39600700 1 -- 172.21.15.47:0/2292023703 <== osd.10 v2:172.21.15.100:6824/1645855291 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fca34009ce0 con 0x7fca240ae1a0 2024-03-21T11:22:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.009+0000 7fca43c00700 1 -- 172.21.15.47:0/2292023703 --> [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fca3c062ca0 con 0x7fca240ae1a0 2024-03-21T11:22:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.010+0000 7fca39600700 1 -- 172.21.15.47:0/2292023703 <== osd.10 v2:172.21.15.100:6824/1645855291 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fca3c062ca0 con 0x7fca240ae1a0 2024-03-21T11:22:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.010+0000 7fca43c00700 1 -- 172.21.15.47:0/2292023703 >> [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291] conn(0x7fca240ae1a0 msgr2=0x7fca240b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.010+0000 7fca43c00700 1 --2- 172.21.15.47:0/2292023703 >> [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291] conn(0x7fca240ae1a0 0x7fca240b0640 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.010+0000 7fca43c00700 1 -- 172.21.15.47:0/2292023703 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fca240a3040 msgr2=0x7fca240a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.010+0000 7fca43c00700 1 --2- 172.21.15.47:0/2292023703 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fca240a3040 0x7fca240a5500 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fca30009d20 tx=0x7fca30009260 comp rx=0 tx=0).stop 2024-03-21T11:22:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.010+0000 7fca43c00700 1 -- 172.21.15.47:0/2292023703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca3c107510 msgr2=0x7fca3c101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.010+0000 7fca43c00700 1 --2- 172.21.15.47:0/2292023703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca3c107510 0x7fca3c101c10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fca340062a0 tx=0x7fca34004c60 comp rx=0 tx=0).stop 2024-03-21T11:22:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.010+0000 7fca41c00700 1 -- 172.21.15.47:0/2292023703 reap_dead start 2024-03-21T11:22:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.011+0000 7fca43c00700 1 -- 172.21.15.47:0/2292023703 shutdown_connections 2024-03-21T11:22:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.011+0000 7fca43c00700 1 -- 172.21.15.47:0/2292023703 >> 172.21.15.47:0/2292023703 conn(0x7fca3c0770e0 msgr2=0x7fca3c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.011+0000 7fca43c00700 1 -- 172.21.15.47:0/2292023703 shutdown_connections 2024-03-21T11:22:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.011+0000 7fca43c00700 1 -- 172.21.15.47:0/2292023703 wait complete. 2024-03-21T11:22:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.090+0000 7fadab000700 1 -- 172.21.15.47:0/651018320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fada4074ce0 msgr2=0x7fada4112e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.090+0000 7fadab000700 1 --2- 172.21.15.47:0/651018320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fada4074ce0 0x7fada4112e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fad90009990 tx=0x7fad90009ca0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.090+0000 7fadab000700 1 -- 172.21.15.47:0/651018320 shutdown_connections 2024-03-21T11:22:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.090+0000 7fadab000700 1 --2- 172.21.15.47:0/651018320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fada4113360 0x7fada4115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.090+0000 7fadab000700 1 --2- 172.21.15.47:0/651018320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fada40743c0 0x7fada40747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.090+0000 7fadab000700 1 --2- 172.21.15.47:0/651018320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fada4074ce0 0x7fada4112e20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.090+0000 7fadab000700 1 -- 172.21.15.47:0/651018320 >> 172.21.15.47:0/651018320 conn(0x7fada40792f0 msgr2=0x7fada4079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.090+0000 7fadab000700 1 -- 172.21.15.47:0/651018320 shutdown_connections 2024-03-21T11:22:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.090+0000 7fadab000700 1 -- 172.21.15.47:0/651018320 wait complete. 2024-03-21T11:22:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.091+0000 7fadab000700 1 Processor -- start 2024-03-21T11:22:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.091+0000 7fadab000700 1 -- start start 2024-03-21T11:22:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fadab000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fada40743c0 0x7fada419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fadab000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fada4074ce0 0x7fada419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fadab000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fada4113360 0x7fada41a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fadab000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada4118ba0 con 0x7fada4074ce0 2024-03-21T11:22:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fadab000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada4118a20 con 0x7fada4113360 2024-03-21T11:22:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fadab000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada4118d20 con 0x7fada40743c0 2024-03-21T11:22:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fada3e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fada40743c0 0x7fada419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fada3e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fada40743c0 0x7fada419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48188/0 (socket says 172.21.15.47:48188) 2024-03-21T11:22:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fada3e00700 1 -- 172.21.15.47:0/1943123338 learned_addr learned my addr 172.21.15.47:0/1943123338 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fada9000700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fada4113360 0x7fada41a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fada3400700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fada4074ce0 0x7fada419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fada3e00700 1 -- 172.21.15.47:0/1943123338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fada4113360 msgr2=0x7fada41a38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fada3e00700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fada4113360 0x7fada41a38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fada3e00700 1 -- 172.21.15.47:0/1943123338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fada4074ce0 msgr2=0x7fada419f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fada3e00700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fada4074ce0 0x7fada419f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fada3e00700 1 -- 172.21.15.47:0/1943123338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada41a4020 con 0x7fada40743c0 2024-03-21T11:22:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.092+0000 7fada3400700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fada4074ce0 0x7fada419f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:22:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.093+0000 7fada3e00700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fada40743c0 0x7fada419f010 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fad9400e9a0 tx=0x7fad9400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.093+0000 7fada0c00700 1 -- 172.21.15.47:0/1943123338 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9400cbd0 con 0x7fada40743c0 2024-03-21T11:22:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.093+0000 7fadab000700 1 -- 172.21.15.47:0/1943123338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada41a4310 con 0x7fada40743c0 2024-03-21T11:22:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.093+0000 7fada0c00700 1 -- 172.21.15.47:0/1943123338 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad9400cd70 con 0x7fada40743c0 2024-03-21T11:22:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.093+0000 7fada0c00700 1 -- 172.21.15.47:0/1943123338 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94018750 con 0x7fada40743c0 2024-03-21T11:22:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.093+0000 7fadab000700 1 -- 172.21.15.47:0/1943123338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada41abb90 con 0x7fada40743c0 2024-03-21T11:22:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.094+0000 7fadab000700 1 -- 172.21.15.47:0/1943123338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fada4112b00 con 0x7fada40743c0 2024-03-21T11:22:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.096+0000 7fada0c00700 1 -- 172.21.15.47:0/1943123338 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fad94010430 con 0x7fada40743c0 2024-03-21T11:22:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.097+0000 7fada0c00700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad880a3040 0x7fad880a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.098+0000 7fada0c00700 1 -- 172.21.15.47:0/1943123338 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fad940d03f0 con 0x7fada40743c0 2024-03-21T11:22:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.098+0000 7fada3400700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad880a3040 0x7fad880a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:00.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.098+0000 7fada0c00700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812] conn(0x7fad880ae1a0 0x7fad880b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:00.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.098+0000 7fada0c00700 1 -- 172.21.15.47:0/1943123338 --> [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad880b0d10 con 0x7fad880ae1a0 2024-03-21T11:22:00.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.098+0000 7fada0c00700 1 -- 172.21.15.47:0/1943123338 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad940d0840 con 0x7fada40743c0 2024-03-21T11:22:00.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.098+0000 7fada9000700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812] conn(0x7fad880ae1a0 0x7fad880b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:00.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.099+0000 7fada3400700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad880a3040 0x7fad880a5500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fad9000b2e0 tx=0x7fad9001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:00.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.099+0000 7fada9000700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812] conn(0x7fad880ae1a0 0x7fad880b0640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:00.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.100+0000 7fada0c00700 1 -- 172.21.15.47:0/1943123338 <== osd.9 v2:172.21.15.47:6826/3319373812 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fad880b0d10 con 0x7fad880ae1a0 2024-03-21T11:22:00.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.118+0000 7fadab000700 1 -- 172.21.15.47:0/1943123338 --> [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fada4075af0 con 0x7fad880ae1a0 2024-03-21T11:22:00.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.119+0000 7fada0c00700 1 -- 172.21.15.47:0/1943123338 <== osd.9 v2:172.21.15.47:6826/3319373812 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fada4075af0 con 0x7fad880ae1a0 2024-03-21T11:22:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.119+0000 7fadab000700 1 -- 172.21.15.47:0/1943123338 >> [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812] conn(0x7fad880ae1a0 msgr2=0x7fad880b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.119+0000 7fadab000700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812] conn(0x7fad880ae1a0 0x7fad880b0640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.119+0000 7fadab000700 1 -- 172.21.15.47:0/1943123338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad880a3040 msgr2=0x7fad880a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.119+0000 7fadab000700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad880a3040 0x7fad880a5500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fad9000b2e0 tx=0x7fad9001a040 comp rx=0 tx=0).stop 2024-03-21T11:22:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.119+0000 7fadab000700 1 -- 172.21.15.47:0/1943123338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fada40743c0 msgr2=0x7fada419f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.119+0000 7fadab000700 1 --2- 172.21.15.47:0/1943123338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fada40743c0 0x7fada419f010 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fad9400e9a0 tx=0x7fad9400ecb0 comp rx=0 tx=0).stop 2024-03-21T11:22:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.119+0000 7fada9000700 1 -- 172.21.15.47:0/1943123338 reap_dead start 2024-03-21T11:22:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.120+0000 7fadab000700 1 -- 172.21.15.47:0/1943123338 shutdown_connections 2024-03-21T11:22:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.120+0000 7fadab000700 1 -- 172.21.15.47:0/1943123338 >> 172.21.15.47:0/1943123338 conn(0x7fada40792f0 msgr2=0x7fada4112120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.120+0000 7fadab000700 1 -- 172.21.15.47:0/1943123338 shutdown_connections 2024-03-21T11:22:00.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:00.120+0000 7fadab000700 1 -- 172.21.15.47:0/1943123338 wait complete. 2024-03-21T11:22:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:00 smithi102 ceph-mon[36748]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 218 B/s wr, 10 op/s; 12 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T11:22:01.256 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:00 smithi047 ceph-mon[31351]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 218 B/s wr, 10 op/s; 12 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T11:22:01.318 INFO:teuthology.orchestra.run.smithi047.stdout:146028888129 2024-03-21T11:22:01.319 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-21T11:22:01.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:00 smithi100 ceph-mon[36245]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 218 B/s wr, 10 op/s; 12 B/s, 1 keys/s, 1 objects/s recovering 2024-03-21T11:22:02.007 INFO:teuthology.orchestra.run.smithi047.stdout:257698037805 2024-03-21T11:22:02.008 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-21T11:22:02.398 INFO:teuthology.orchestra.run.smithi047.stdout:167503724605 2024-03-21T11:22:02.399 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-21T11:22:02.824 INFO:teuthology.orchestra.run.smithi047.stdout:313532612642 2024-03-21T11:22:02.824 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-21T11:22:03.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:02 smithi047 ceph-mon[31351]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 197 B/s wr, 11 op/s; 11 B/s, 0 keys/s, 1 objects/s recovering 2024-03-21T11:22:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.075+0000 7fab23000700 1 -- 172.21.15.47:0/2700588821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c10c660 msgr2=0x7fab1c10eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.075+0000 7fab23000700 1 --2- 172.21.15.47:0/2700588821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c10c660 0x7fab1c10eb20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fab14009f90 tx=0x7fab1400b330 comp rx=0 tx=0).stop 2024-03-21T11:22:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.075+0000 7fab23000700 1 -- 172.21.15.47:0/2700588821 shutdown_connections 2024-03-21T11:22:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.075+0000 7fab23000700 1 --2- 172.21.15.47:0/2700588821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c10c660 0x7fab1c10eb20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.075+0000 7fab23000700 1 --2- 172.21.15.47:0/2700588821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c103230 0x7fab1c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.075+0000 7fab23000700 1 --2- 172.21.15.47:0/2700588821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab1c103b50 0x7fab1c10c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.075+0000 7fab23000700 1 -- 172.21.15.47:0/2700588821 >> 172.21.15.47:0/2700588821 conn(0x7fab1c0fca60 msgr2=0x7fab1c0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.075+0000 7fab23000700 1 -- 172.21.15.47:0/2700588821 shutdown_connections 2024-03-21T11:22:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.075+0000 7fab23000700 1 -- 172.21.15.47:0/2700588821 wait complete. 2024-03-21T11:22:03.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.076+0000 7fab23000700 1 Processor -- start 2024-03-21T11:22:03.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.076+0000 7fab23000700 1 -- start start 2024-03-21T11:22:03.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab23000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c103230 0x7fab1c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:03.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab23000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c103b50 0x7fab1c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:03.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab23000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab1c10c660 0x7fab1c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:03.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab23000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c112420 con 0x7fab1c103230 2024-03-21T11:22:03.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab23000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1122a0 con 0x7fab1c103b50 2024-03-21T11:22:03.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab23000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1125a0 con 0x7fab1c10c660 2024-03-21T11:22:03.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab21200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c103b50 0x7fab1c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:03.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab21c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c103230 0x7fab1c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:03.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab21200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c103b50 0x7fab1c19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38962/0 (socket says 172.21.15.47:38962) 2024-03-21T11:22:03.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab21200700 1 -- 172.21.15.47:0/4024234215 learned_addr learned my addr 172.21.15.47:0/4024234215 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:03.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab22600700 1 --2- 172.21.15.47:0/4024234215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab1c10c660 0x7fab1c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:03.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab21200700 1 -- 172.21.15.47:0/4024234215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab1c10c660 msgr2=0x7fab1c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:03.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab21200700 1 --2- 172.21.15.47:0/4024234215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab1c10c660 0x7fab1c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:03.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab21200700 1 -- 172.21.15.47:0/4024234215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c103230 msgr2=0x7fab1c19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:03.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab21200700 1 --2- 172.21.15.47:0/4024234215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1c103230 0x7fab1c19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:03.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.077+0000 7fab21200700 1 -- 172.21.15.47:0/4024234215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c198fe0 con 0x7fab1c103b50 2024-03-21T11:22:03.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.078+0000 7fab21200700 1 --2- 172.21.15.47:0/4024234215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c103b50 0x7fab1c19eb70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fab0800e9c0 tx=0x7fab0800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:03.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.078+0000 7fab1a000700 1 -- 172.21.15.47:0/4024234215 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08004a30 con 0x7fab1c103b50 2024-03-21T11:22:03.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.078+0000 7fab1a000700 1 -- 172.21.15.47:0/4024234215 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab0800cca0 con 0x7fab1c103b50 2024-03-21T11:22:03.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.078+0000 7fab1a000700 1 -- 172.21.15.47:0/4024234215 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08005650 con 0x7fab1c103b50 2024-03-21T11:22:03.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.078+0000 7fab23000700 1 -- 172.21.15.47:0/4024234215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c1992d0 con 0x7fab1c103b50 2024-03-21T11:22:03.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.078+0000 7fab23000700 1 -- 172.21.15.47:0/4024234215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c1a55b0 con 0x7fab1c103b50 2024-03-21T11:22:03.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.079+0000 7fab23000700 1 -- 172.21.15.47:0/4024234215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab1c062860 con 0x7fab1c103b50 2024-03-21T11:22:03.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.081+0000 7fab1a000700 1 -- 172.21.15.47:0/4024234215 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fab0800c7c0 con 0x7fab1c103b50 2024-03-21T11:22:03.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.082+0000 7fab1a000700 1 --2- 172.21.15.47:0/4024234215 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab040a30c0 0x7fab040a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:03.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.082+0000 7fab1a000700 1 -- 172.21.15.47:0/4024234215 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fab080cf870 con 0x7fab1c103b50 2024-03-21T11:22:03.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.083+0000 7fab21c00700 1 --2- 172.21.15.47:0/4024234215 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab040a30c0 0x7fab040a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:03.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.083+0000 7fab1a000700 1 --2- 172.21.15.47:0/4024234215 >> [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877] conn(0x7fab040ae220 0x7fab040b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:03.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.083+0000 7fab1a000700 1 -- 172.21.15.47:0/4024234215 --> [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab040b0d90 con 0x7fab040ae220 2024-03-21T11:22:03.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.083+0000 7fab1a000700 1 -- 172.21.15.47:0/4024234215 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab080d42a0 con 0x7fab1c103b50 2024-03-21T11:22:03.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.083+0000 7fab22600700 1 --2- 172.21.15.47:0/4024234215 >> [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877] conn(0x7fab040ae220 0x7fab040b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:03.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.083+0000 7fab21c00700 1 --2- 172.21.15.47:0/4024234215 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab040a30c0 0x7fab040a5580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fab10009c60 tx=0x7fab10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:03.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.084+0000 7fab22600700 1 --2- 172.21.15.47:0/4024234215 >> [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877] conn(0x7fab040ae220 0x7fab040b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:03.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.086+0000 7fab1a000700 1 -- 172.21.15.47:0/4024234215 <== osd.3 v2:172.21.15.47:6810/534982877 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fab040b0d90 con 0x7fab040ae220 2024-03-21T11:22:03.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.104+0000 7fab23000700 1 -- 172.21.15.47:0/4024234215 --> [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab1c062ce0 con 0x7fab040ae220 2024-03-21T11:22:03.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.105+0000 7fab1a000700 1 -- 172.21.15.47:0/4024234215 <== osd.3 v2:172.21.15.47:6810/534982877 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fab1c062ce0 con 0x7fab040ae220 2024-03-21T11:22:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.106+0000 7fab23000700 1 -- 172.21.15.47:0/4024234215 >> [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877] conn(0x7fab040ae220 msgr2=0x7fab040b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.106+0000 7fab23000700 1 --2- 172.21.15.47:0/4024234215 >> [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877] conn(0x7fab040ae220 0x7fab040b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:03.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.106+0000 7fab23000700 1 -- 172.21.15.47:0/4024234215 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab040a30c0 msgr2=0x7fab040a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:03.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.106+0000 7fab23000700 1 --2- 172.21.15.47:0/4024234215 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab040a30c0 0x7fab040a5580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fab10009c60 tx=0x7fab10009260 comp rx=0 tx=0).stop 2024-03-21T11:22:03.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.106+0000 7fab23000700 1 -- 172.21.15.47:0/4024234215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c103b50 msgr2=0x7fab1c19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:03.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.106+0000 7fab23000700 1 --2- 172.21.15.47:0/4024234215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab1c103b50 0x7fab1c19eb70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fab0800e9c0 tx=0x7fab0800ecd0 comp rx=0 tx=0).stop 2024-03-21T11:22:03.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.106+0000 7fab22600700 1 -- 172.21.15.47:0/4024234215 reap_dead start 2024-03-21T11:22:03.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.106+0000 7fab23000700 1 -- 172.21.15.47:0/4024234215 shutdown_connections 2024-03-21T11:22:03.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.106+0000 7fab23000700 1 -- 172.21.15.47:0/4024234215 >> 172.21.15.47:0/4024234215 conn(0x7fab1c0fca60 msgr2=0x7fab1c0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:03.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.106+0000 7fab23000700 1 -- 172.21.15.47:0/4024234215 shutdown_connections 2024-03-21T11:22:03.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:03.106+0000 7fab23000700 1 -- 172.21.15.47:0/4024234215 wait complete. 2024-03-21T11:22:03.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:02 smithi102 ceph-mon[36748]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 197 B/s wr, 11 op/s; 11 B/s, 0 keys/s, 1 objects/s recovering 2024-03-21T11:22:03.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:02 smithi100 ceph-mon[36245]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 197 B/s wr, 11 op/s; 11 B/s, 0 keys/s, 1 objects/s recovering 2024-03-21T11:22:04.585 INFO:teuthology.orchestra.run.smithi047.stdout:292057776167 2024-03-21T11:22:04.585 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-21T11:22:04.926 INFO:teuthology.orchestra.run.smithi047.stdout:223338299443 2024-03-21T11:22:04.926 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-21T11:22:05.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:04 smithi102 ceph-mon[36748]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T11:22:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:04 smithi047 ceph-mon[31351]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T11:22:05.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:04 smithi100 ceph-mon[36245]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T11:22:05.442 INFO:teuthology.orchestra.run.smithi047.stdout:347892351005 2024-03-21T11:22:05.443 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-21T11:22:06.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.001+0000 7f450aa00700 1 -- 172.21.15.47:0/2863365905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4504069e40 msgr2=0x7f450406a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:06.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.001+0000 7f450aa00700 1 --2- 172.21.15.47:0/2863365905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4504069e40 0x7f450406a2a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f44f00099c0 tx=0x7f44f0009cd0 comp rx=0 tx=0).stop 2024-03-21T11:22:06.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.001+0000 7f450aa00700 1 -- 172.21.15.47:0/2863365905 shutdown_connections 2024-03-21T11:22:06.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.001+0000 7f450aa00700 1 --2- 172.21.15.47:0/2863365905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450406a7e0 0x7f450410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:06.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.001+0000 7f450aa00700 1 --2- 172.21.15.47:0/2863365905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45041081e0 0x7f45041085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:06.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.001+0000 7f450aa00700 1 --2- 172.21.15.47:0/2863365905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4504069e40 0x7f450406a2a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:06.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.001+0000 7f450aa00700 1 -- 172.21.15.47:0/2863365905 >> 172.21.15.47:0/2863365905 conn(0x7f4504077fc0 msgr2=0x7f45040783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:06.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.001+0000 7f450aa00700 1 -- 172.21.15.47:0/2863365905 shutdown_connections 2024-03-21T11:22:06.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.001+0000 7f450aa00700 1 -- 172.21.15.47:0/2863365905 wait complete. 2024-03-21T11:22:06.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.003+0000 7f450aa00700 1 Processor -- start 2024-03-21T11:22:06.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.003+0000 7f450aa00700 1 -- start start 2024-03-21T11:22:06.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.003+0000 7f450aa00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4504069e40 0x7f45041a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:06.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.003+0000 7f450aa00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450406a7e0 0x7f45041a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:06.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.003+0000 7f450aa00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45041081e0 0x7f450419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:06.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.003+0000 7f450aa00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4504112490 con 0x7f45041081e0 2024-03-21T11:22:06.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.003+0000 7f450aa00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4504112310 con 0x7f450406a7e0 2024-03-21T11:22:06.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.003+0000 7f450aa00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4504112610 con 0x7f4504069e40 2024-03-21T11:22:06.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.004+0000 7f4503400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450406a7e0 0x7f45041a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:06.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.004+0000 7f4503e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4504069e40 0x7f45041a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:06.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.004+0000 7f4503400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450406a7e0 0x7f45041a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38984/0 (socket says 172.21.15.47:38984) 2024-03-21T11:22:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.004+0000 7f4503400700 1 -- 172.21.15.47:0/1880590241 learned_addr learned my addr 172.21.15.47:0/1880590241 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.004+0000 7f4503e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4504069e40 0x7f45041a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48228/0 (socket says 172.21.15.47:48228) 2024-03-21T11:22:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.004+0000 7f4503400700 1 -- 172.21.15.47:0/1880590241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4504069e40 msgr2=0x7f45041a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.004+0000 7f4503400700 1 --2- 172.21.15.47:0/1880590241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4504069e40 0x7f45041a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.004+0000 7f4503400700 1 -- 172.21.15.47:0/1880590241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45041081e0 msgr2=0x7f450419aa70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:22:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.004+0000 7f4503400700 1 --2- 172.21.15.47:0/1880590241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45041081e0 0x7f450419aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.004+0000 7f4503400700 1 -- 172.21.15.47:0/1880590241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450419b1d0 con 0x7f450406a7e0 2024-03-21T11:22:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.004+0000 7f4503400700 1 --2- 172.21.15.47:0/1880590241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450406a7e0 0x7f45041a0df0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f44f0005910 tx=0x7f44f000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.005+0000 7f4500c00700 1 -- 172.21.15.47:0/1880590241 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f0027070 con 0x7f450406a7e0 2024-03-21T11:22:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.005+0000 7f4500c00700 1 -- 172.21.15.47:0/1880590241 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44f0003680 con 0x7f450406a7e0 2024-03-21T11:22:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.005+0000 7f4500c00700 1 -- 172.21.15.47:0/1880590241 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f00173f0 con 0x7f450406a7e0 2024-03-21T11:22:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.005+0000 7f450aa00700 1 -- 172.21.15.47:0/1880590241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450419b460 con 0x7f450406a7e0 2024-03-21T11:22:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.005+0000 7f450aa00700 1 -- 172.21.15.47:0/1880590241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45041a7780 con 0x7f450406a7e0 2024-03-21T11:22:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.006+0000 7f450aa00700 1 -- 172.21.15.47:0/1880590241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f450410dd50 con 0x7f450406a7e0 2024-03-21T11:22:06.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.008+0000 7f4500c00700 1 -- 172.21.15.47:0/1880590241 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f44f002a070 con 0x7f450406a7e0 2024-03-21T11:22:06.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.010+0000 7f4500c00700 1 --2- 172.21.15.47:0/1880590241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44e80a30c0 0x7f44e80a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:06.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.011+0000 7f4500c00700 1 -- 172.21.15.47:0/1880590241 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f44f00d0600 con 0x7f450406a7e0 2024-03-21T11:22:06.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.011+0000 7f4503e00700 1 --2- 172.21.15.47:0/1880590241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44e80a30c0 0x7f44e80a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:06.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.011+0000 7f4500c00700 1 --2- 172.21.15.47:0/1880590241 >> [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261] conn(0x7f44e80ae0f0 0x7f44e80b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.011+0000 7f4500c00700 1 -- 172.21.15.47:0/1880590241 --> [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44e80b0c60 con 0x7f44e80ae0f0 2024-03-21T11:22:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.011+0000 7f4500c00700 1 -- 172.21.15.47:0/1880590241 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44f00d7050 con 0x7f450406a7e0 2024-03-21T11:22:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.011+0000 7f4508a00700 1 --2- 172.21.15.47:0/1880590241 >> [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261] conn(0x7f44e80ae0f0 0x7f44e80b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.012+0000 7f4503e00700 1 --2- 172.21.15.47:0/1880590241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44e80a30c0 0x7f44e80a5580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f450419bbb0 tx=0x7f44f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.012+0000 7f4508a00700 1 --2- 172.21.15.47:0/1880590241 >> [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261] conn(0x7f44e80ae0f0 0x7f44e80b0590 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.014+0000 7f4500c00700 1 -- 172.21.15.47:0/1880590241 <== osd.2 v2:172.21.15.102:6800/644548261 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f44e80b0c60 con 0x7f44e80ae0f0 2024-03-21T11:22:06.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.033+0000 7f450aa00700 1 -- 172.21.15.47:0/1880590241 --> [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4504103890 con 0x7f44e80ae0f0 2024-03-21T11:22:06.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.033+0000 7f4500c00700 1 -- 172.21.15.47:0/1880590241 <== osd.2 v2:172.21.15.102:6800/644548261 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4504103890 con 0x7f44e80ae0f0 2024-03-21T11:22:06.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.034+0000 7f450aa00700 1 -- 172.21.15.47:0/1880590241 >> [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261] conn(0x7f44e80ae0f0 msgr2=0x7f44e80b0590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:06.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.034+0000 7f450aa00700 1 --2- 172.21.15.47:0/1880590241 >> [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261] conn(0x7f44e80ae0f0 0x7f44e80b0590 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:06.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.034+0000 7f450aa00700 1 -- 172.21.15.47:0/1880590241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44e80a30c0 msgr2=0x7f44e80a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:06.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.034+0000 7f450aa00700 1 --2- 172.21.15.47:0/1880590241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44e80a30c0 0x7f44e80a5580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f450419bbb0 tx=0x7f44f4009260 comp rx=0 tx=0).stop 2024-03-21T11:22:06.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.034+0000 7f450aa00700 1 -- 172.21.15.47:0/1880590241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450406a7e0 msgr2=0x7f45041a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:06.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.034+0000 7f450aa00700 1 --2- 172.21.15.47:0/1880590241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f450406a7e0 0x7f45041a0df0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f44f0005910 tx=0x7f44f000f690 comp rx=0 tx=0).stop 2024-03-21T11:22:06.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.034+0000 7f4508a00700 1 -- 172.21.15.47:0/1880590241 reap_dead start 2024-03-21T11:22:06.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.034+0000 7f450aa00700 1 -- 172.21.15.47:0/1880590241 shutdown_connections 2024-03-21T11:22:06.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.034+0000 7f450aa00700 1 -- 172.21.15.47:0/1880590241 >> 172.21.15.47:0/1880590241 conn(0x7f4504077fc0 msgr2=0x7f450410d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:06.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.034+0000 7f450aa00700 1 -- 172.21.15.47:0/1880590241 shutdown_connections 2024-03-21T11:22:06.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:06.035+0000 7f450aa00700 1 -- 172.21.15.47:0/1880590241 wait complete. 2024-03-21T11:22:06.535 INFO:teuthology.orchestra.run.smithi047.stdout:111669149770 2024-03-21T11:22:06.535 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-21T11:22:07.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:06 smithi047 ceph-mon[31351]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T11:22:07.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:06 smithi102 ceph-mon[36748]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T11:22:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:07.261 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:07.261 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:07.262 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:07.263 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:07.292 INFO:teuthology.orchestra.run.smithi047.stdout:77309411409 2024-03-21T11:22:07.293 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-21T11:22:07.312 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:07.349 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:07.381 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:07.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:06 smithi100 ceph-mon[36245]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T11:22:09.194 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:08 smithi100 ceph-mon[36245]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T11:22:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:08 smithi102 ceph-mon[36748]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T11:22:09.267 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:08 smithi047 ceph-mon[31351]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-21T11:22:11.197 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:10 smithi047 ceph-mon[31351]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:10 smithi102 ceph-mon[36748]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:11.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:10 smithi100 ceph-mon[36245]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:12 smithi102 ceph-mon[36748]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:22:13.241 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:12 smithi047 ceph-mon[31351]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:22:13.262 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:13.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:12 smithi100 ceph-mon[36245]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:22:14.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:13 smithi102 ceph-mon[36748]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-21T11:22:14.299 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:13 smithi047 ceph-mon[31351]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-21T11:22:14.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:13 smithi100 ceph-mon[36245]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-21T11:22:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:16 smithi047 ceph-mon[31351]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:22:16.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:16 smithi100 ceph-mon[36245]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:22:16.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:16 smithi102 ceph-mon[36748]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:22:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.821+0000 7f3a96400700 1 -- 172.21.15.47:0/1742247545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a90069e40 msgr2=0x7f3a9006a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.821+0000 7f3a96400700 1 --2- 172.21.15.47:0/1742247545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a90069e40 0x7f3a9006a220 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009990 tx=0x7f3a80009ca0 comp rx=0 tx=0).stop 2024-03-21T11:22:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.821+0000 7f3a96400700 1 -- 172.21.15.47:0/1742247545 shutdown_connections 2024-03-21T11:22:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.821+0000 7f3a96400700 1 --2- 172.21.15.47:0/1742247545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9010ec40 0x7f3a90111030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:17.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.821+0000 7f3a96400700 1 --2- 172.21.15.47:0/1742247545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a90069e40 0x7f3a9006a220 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.821+0000 7f3a96400700 1 --2- 172.21.15.47:0/1742247545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9006a760 0x7f3a9010e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.821+0000 7f3a96400700 1 -- 172.21.15.47:0/1742247545 >> 172.21.15.47:0/1742247545 conn(0x7f3a90077fc0 msgr2=0x7f3a900783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.821+0000 7f3a96400700 1 -- 172.21.15.47:0/1742247545 shutdown_connections 2024-03-21T11:22:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.821+0000 7f3a96400700 1 -- 172.21.15.47:0/1742247545 wait complete. 2024-03-21T11:22:17.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.822+0000 7f3a96400700 1 Processor -- start 2024-03-21T11:22:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.822+0000 7f3a96400700 1 -- start start 2024-03-21T11:22:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.823+0000 7f3a96400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a90069e40 0x7f3a90103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.823+0000 7f3a96400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9006a760 0x7f3a901038b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.823+0000 7f3a96400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a9010ec40 0x7f3a90103df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.823+0000 7f3a96400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a901146a0 con 0x7f3a9006a760 2024-03-21T11:22:17.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.823+0000 7f3a96400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a90114520 con 0x7f3a90069e40 2024-03-21T11:22:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.823+0000 7f3a96400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a90114820 con 0x7f3a9010ec40 2024-03-21T11:22:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.823+0000 7f3a8f400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a90069e40 0x7f3a90103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.823+0000 7f3a8f400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a90069e40 0x7f3a90103370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49802/0 (socket says 172.21.15.47:49802) 2024-03-21T11:22:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.823+0000 7f3a8f400700 1 -- 172.21.15.47:0/2205846747 learned_addr learned my addr 172.21.15.47:0/2205846747 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.823+0000 7f3a8fe00700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a9010ec40 0x7f3a90103df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.823+0000 7f3a8ea00700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9006a760 0x7f3a901038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.824+0000 7f3a8f400700 1 -- 172.21.15.47:0/2205846747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a9010ec40 msgr2=0x7f3a90103df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.824+0000 7f3a8f400700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a9010ec40 0x7f3a90103df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.824+0000 7f3a8f400700 1 -- 172.21.15.47:0/2205846747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9006a760 msgr2=0x7f3a901038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.824+0000 7f3a8f400700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9006a760 0x7f3a901038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.824+0000 7f3a8f400700 1 -- 172.21.15.47:0/2205846747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a901ab2c0 con 0x7f3a90069e40 2024-03-21T11:22:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.824+0000 7f3a8fe00700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a9010ec40 0x7f3a90103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:22:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.824+0000 7f3a8ea00700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9006a760 0x7f3a901038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:22:17.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.824+0000 7f3a8f400700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a90069e40 0x7f3a90103370 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000bab0 tx=0x7f3a8000bb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:17.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.824+0000 7f3a87e00700 1 -- 172.21.15.47:0/2205846747 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80027070 con 0x7f3a90069e40 2024-03-21T11:22:17.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.824+0000 7f3a87e00700 1 -- 172.21.15.47:0/2205846747 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a80003d70 con 0x7f3a90069e40 2024-03-21T11:22:17.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.825+0000 7f3a87e00700 1 -- 172.21.15.47:0/2205846747 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000f460 con 0x7f3a90069e40 2024-03-21T11:22:17.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.825+0000 7f3a96400700 1 -- 172.21.15.47:0/2205846747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a901ab550 con 0x7f3a90069e40 2024-03-21T11:22:17.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.825+0000 7f3a96400700 1 -- 172.21.15.47:0/2205846747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a901ab9e0 con 0x7f3a90069e40 2024-03-21T11:22:17.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.826+0000 7f3a96400700 1 -- 172.21.15.47:0/2205846747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a90101670 con 0x7f3a90069e40 2024-03-21T11:22:17.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.828+0000 7f3a87e00700 1 -- 172.21.15.47:0/2205846747 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f3a8002a070 con 0x7f3a90069e40 2024-03-21T11:22:17.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.829+0000 7f3a87e00700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a740a30c0 0x7f3a740a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.829+0000 7f3a87e00700 1 -- 172.21.15.47:0/2205846747 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f3a800d06a0 con 0x7f3a90069e40 2024-03-21T11:22:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.830+0000 7f3a8ea00700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a740a30c0 0x7f3a740a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:17.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.830+0000 7f3a8ea00700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a740a30c0 0x7f3a740a5580 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3a90104b30 tx=0x7f3a7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.833+0000 7f3a87e00700 1 -- 172.21.15.47:0/2205846747 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3a80013070 con 0x7f3a90069e40 2024-03-21T11:22:18.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:18.086+0000 7f3a96400700 1 -- 172.21.15.47:0/2205846747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3a9002da30 con 0x7f3a90069e40 2024-03-21T11:22:18.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:18.086+0000 7f3a87e00700 1 -- 172.21.15.47:0/2205846747 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a8009d040 con 0x7f3a90069e40 2024-03-21T11:22:18.087 INFO:teuthology.orchestra.run.smithi047.stdout:55834574937 2024-03-21T11:22:18.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:18.089+0000 7f3a96400700 1 -- 172.21.15.47:0/2205846747 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a740a30c0 msgr2=0x7f3a740a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:18.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:18.089+0000 7f3a96400700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a740a30c0 0x7f3a740a5580 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3a90104b30 tx=0x7f3a7c009260 comp rx=0 tx=0).stop 2024-03-21T11:22:18.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:18.089+0000 7f3a96400700 1 -- 172.21.15.47:0/2205846747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a90069e40 msgr2=0x7f3a90103370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:18.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:18.089+0000 7f3a96400700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a90069e40 0x7f3a90103370 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000bab0 tx=0x7f3a8000bb90 comp rx=0 tx=0).stop 2024-03-21T11:22:18.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:18.089+0000 7f3a96400700 1 -- 172.21.15.47:0/2205846747 shutdown_connections 2024-03-21T11:22:18.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:18.089+0000 7f3a96400700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a9010ec40 0x7f3a90103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:18.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:18.089+0000 7f3a96400700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a90069e40 0x7f3a90103370 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:18.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:18.089+0000 7f3a96400700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a740a30c0 0x7f3a740a5580 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:18.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:18.089+0000 7f3a96400700 1 --2- 172.21.15.47:0/2205846747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9006a760 0x7f3a901038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:18.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:18.089+0000 7f3a96400700 1 -- 172.21.15.47:0/2205846747 >> 172.21.15.47:0/2205846747 conn(0x7f3a90077fc0 msgr2=0x7f3a90110220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:18.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:18.090+0000 7f3a96400700 1 -- 172.21.15.47:0/2205846747 shutdown_connections 2024-03-21T11:22:18.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:18.090+0000 7f3a96400700 1 -- 172.21.15.47:0/2205846747 wait complete. 2024-03-21T11:22:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:18 smithi047 ceph-mon[31351]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:22:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:18 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/601246720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T11:22:18.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:18 smithi100 ceph-mon[36245]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:22:18.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:18 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/601246720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T11:22:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:18 smithi102 ceph-mon[36748]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:22:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:18 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/601246720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T11:22:18.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.497+0000 7f606b200700 1 -- 172.21.15.47:0/1895863336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60641073a0 msgr2=0x7f606410ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:18.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.497+0000 7f606b200700 1 --2- 172.21.15.47:0/1895863336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60641073a0 0x7f606410ef70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f606000b0a0 tx=0x7f606000b3b0 comp rx=0 tx=0).stop 2024-03-21T11:22:18.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.497+0000 7f606b200700 1 -- 172.21.15.47:0/1895863336 shutdown_connections 2024-03-21T11:22:18.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.497+0000 7f606b200700 1 --2- 172.21.15.47:0/1895863336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60641073a0 0x7f606410ef70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:18.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.497+0000 7f606b200700 1 --2- 172.21.15.47:0/1895863336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60641022c0 0x7f60641026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:18.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.497+0000 7f606b200700 1 --2- 172.21.15.47:0/1895863336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6064102c70 0x7f6064106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:18.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.497+0000 7f606b200700 1 -- 172.21.15.47:0/1895863336 >> 172.21.15.47:0/1895863336 conn(0x7f60640fdbb0 msgr2=0x7f60640fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:18.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.497+0000 7f606b200700 1 -- 172.21.15.47:0/1895863336 shutdown_connections 2024-03-21T11:22:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.497+0000 7f606b200700 1 -- 172.21.15.47:0/1895863336 wait complete. 2024-03-21T11:22:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.499+0000 7f606b200700 1 Processor -- start 2024-03-21T11:22:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.499+0000 7f606b200700 1 -- start start 2024-03-21T11:22:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.499+0000 7f606b200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60641022c0 0x7f606419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.499+0000 7f606b200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6064102c70 0x7f606419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.499+0000 7f606b200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60641073a0 0x7f6064198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.499+0000 7f606b200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60641123e0 con 0x7f60641022c0 2024-03-21T11:22:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.499+0000 7f606b200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6064112260 con 0x7f60641073a0 2024-03-21T11:22:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.499+0000 7f606b200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6064112560 con 0x7f6064102c70 2024-03-21T11:22:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.500+0000 7f6069400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6064102c70 0x7f606419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.500+0000 7f6069400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6064102c70 0x7f606419eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57360/0 (socket says 172.21.15.47:57360) 2024-03-21T11:22:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.500+0000 7f6069400700 1 -- 172.21.15.47:0/601246720 learned_addr learned my addr 172.21.15.47:0/601246720 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.500+0000 7f6069e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60641022c0 0x7f606419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.500+0000 7f606a800700 1 --2- 172.21.15.47:0/601246720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60641073a0 0x7f6064198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.500+0000 7f6069400700 1 -- 172.21.15.47:0/601246720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60641073a0 msgr2=0x7f6064198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.500+0000 7f6069400700 1 --2- 172.21.15.47:0/601246720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60641073a0 0x7f6064198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.500+0000 7f6069400700 1 -- 172.21.15.47:0/601246720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60641022c0 msgr2=0x7f606419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.500+0000 7f6069400700 1 --2- 172.21.15.47:0/601246720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60641022c0 0x7f606419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.500+0000 7f6069400700 1 -- 172.21.15.47:0/601246720 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6064199000 con 0x7f6064102c70 2024-03-21T11:22:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.500+0000 7f606a800700 1 --2- 172.21.15.47:0/601246720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60641073a0 0x7f6064198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.500+0000 7f6069400700 1 --2- 172.21.15.47:0/601246720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6064102c70 0x7f606419eb40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f605400d980 tx=0x7f605400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.501+0000 7f605ea00700 1 -- 172.21.15.47:0/601246720 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6054017070 con 0x7f6064102c70 2024-03-21T11:22:18.988 INFO:teuthology.orchestra.run.smithi047.stdout:146028888134 2024-03-21T11:22:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.501+0000 7f605ea00700 1 -- 172.21.15.47:0/601246720 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f605400fd70 con 0x7f6064102c70 2024-03-21T11:22:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.501+0000 7f605ea00700 1 -- 172.21.15.47:0/601246720 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605400e3b0 con 0x7f6064102c70 2024-03-21T11:22:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.501+0000 7f606b200700 1 -- 172.21.15.47:0/601246720 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60641992f0 con 0x7f6064102c70 2024-03-21T11:22:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.501+0000 7f606b200700 1 -- 172.21.15.47:0/601246720 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60641a5580 con 0x7f6064102c70 2024-03-21T11:22:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.502+0000 7f606b200700 1 -- 172.21.15.47:0/601246720 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60641037a0 con 0x7f6064102c70 2024-03-21T11:22:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.504+0000 7f605ea00700 1 -- 172.21.15.47:0/601246720 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f605400f890 con 0x7f6064102c70 2024-03-21T11:22:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.505+0000 7f605ea00700 1 --2- 172.21.15.47:0/601246720 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f604c0a3040 0x7f604c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:18.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.506+0000 7f605ea00700 1 -- 172.21.15.47:0/601246720 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f60540cf380 con 0x7f6064102c70 2024-03-21T11:22:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.506+0000 7f6069e00700 1 --2- 172.21.15.47:0/601246720 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f604c0a3040 0x7f604c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.506+0000 7f6069e00700 1 --2- 172.21.15.47:0/601246720 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f604c0a3040 0x7f604c0a5500 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6058009c60 tx=0x7f6058009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.512+0000 7f605ea00700 1 -- 172.21.15.47:0/601246720 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f605409bc90 con 0x7f6064102c70 2024-03-21T11:22:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.747+0000 7f606b200700 1 -- 172.21.15.47:0/601246720 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6064199520 con 0x7f6064102c70 2024-03-21T11:22:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.748+0000 7f605ea00700 1 -- 172.21.15.47:0/601246720 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f605409b590 con 0x7f6064102c70 2024-03-21T11:22:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.752+0000 7f606b200700 1 -- 172.21.15.47:0/601246720 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f604c0a3040 msgr2=0x7f604c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.752+0000 7f606b200700 1 --2- 172.21.15.47:0/601246720 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f604c0a3040 0x7f604c0a5500 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6058009c60 tx=0x7f6058009260 comp rx=0 tx=0).stop 2024-03-21T11:22:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.752+0000 7f606b200700 1 -- 172.21.15.47:0/601246720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6064102c70 msgr2=0x7f606419eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.752+0000 7f606b200700 1 --2- 172.21.15.47:0/601246720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6064102c70 0x7f606419eb40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f605400d980 tx=0x7f605400dc90 comp rx=0 tx=0).stop 2024-03-21T11:22:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.752+0000 7f606b200700 1 -- 172.21.15.47:0/601246720 shutdown_connections 2024-03-21T11:22:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.752+0000 7f606b200700 1 --2- 172.21.15.47:0/601246720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60641073a0 0x7f6064198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.752+0000 7f606b200700 1 --2- 172.21.15.47:0/601246720 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f604c0a3040 0x7f604c0a5500 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.752+0000 7f606b200700 1 --2- 172.21.15.47:0/601246720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60641022c0 0x7f606419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.752+0000 7f606b200700 1 --2- 172.21.15.47:0/601246720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6064102c70 0x7f606419eb40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.752+0000 7f606b200700 1 -- 172.21.15.47:0/601246720 >> 172.21.15.47:0/601246720 conn(0x7f60640fdbb0 msgr2=0x7f60640fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.752+0000 7f606b200700 1 -- 172.21.15.47:0/601246720 shutdown_connections 2024-03-21T11:22:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:17.753+0000 7f606b200700 1 -- 172.21.15.47:0/601246720 wait complete. 2024-03-21T11:22:19.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:19 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2205846747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T11:22:19.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:19 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2205846747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T11:22:19.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:19 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2205846747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T11:22:20.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:20 smithi047 ceph-mon[31351]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:20.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:20 smithi100 ceph-mon[36245]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:20.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:20 smithi102 ceph-mon[36748]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:21.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.377+0000 7f7db5600700 1 -- 172.21.15.47:0/3539741156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7db0102df0 msgr2=0x7f7db0106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.377+0000 7f7db5600700 1 --2- 172.21.15.47:0/3539741156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7db0102df0 0x7f7db0106e30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009990 tx=0x7f7d98009ca0 comp rx=0 tx=0).stop 2024-03-21T11:22:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.378+0000 7f7db5600700 1 -- 172.21.15.47:0/3539741156 shutdown_connections 2024-03-21T11:22:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.378+0000 7f7db5600700 1 --2- 172.21.15.47:0/3539741156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7db0107510 0x7f7db010f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.378+0000 7f7db5600700 1 --2- 172.21.15.47:0/3539741156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7db0102440 0x7f7db0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.378+0000 7f7db5600700 1 --2- 172.21.15.47:0/3539741156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7db0102df0 0x7f7db0106e30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.378+0000 7f7db5600700 1 -- 172.21.15.47:0/3539741156 >> 172.21.15.47:0/3539741156 conn(0x7f7db00792f0 msgr2=0x7f7db0079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.378+0000 7f7db5600700 1 -- 172.21.15.47:0/3539741156 shutdown_connections 2024-03-21T11:22:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.378+0000 7f7db5600700 1 -- 172.21.15.47:0/3539741156 wait complete. 2024-03-21T11:22:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.379+0000 7f7db5600700 1 Processor -- start 2024-03-21T11:22:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.379+0000 7f7db5600700 1 -- start start 2024-03-21T11:22:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.380+0000 7f7db5600700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7db0102440 0x7f7db01a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.380+0000 7f7db5600700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7db0102df0 0x7f7db01a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.380+0000 7f7db5600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7db0107510 0x7f7db019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.380+0000 7f7db5600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0112520 con 0x7f7db0107510 2024-03-21T11:22:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.380+0000 7f7db5600700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db01123a0 con 0x7f7db0102440 2024-03-21T11:22:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.380+0000 7f7db5600700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db01126a0 con 0x7f7db0102df0 2024-03-21T11:22:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.380+0000 7f7da6000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7db0102df0 0x7f7db01a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.380+0000 7f7daea00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7db0102440 0x7f7db01a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.380+0000 7f7da6000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7db0102df0 0x7f7db01a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54784/0 (socket says 172.21.15.47:54784) 2024-03-21T11:22:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.381+0000 7f7da6000700 1 -- 172.21.15.47:0/1286633704 learned_addr learned my addr 172.21.15.47:0/1286633704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.381+0000 7f7daf400700 1 --2- 172.21.15.47:0/1286633704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7db0107510 0x7f7db019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.381+0000 7f7daea00700 1 -- 172.21.15.47:0/1286633704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7db0102df0 msgr2=0x7f7db01a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.381+0000 7f7daea00700 1 --2- 172.21.15.47:0/1286633704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7db0102df0 0x7f7db01a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.381+0000 7f7daea00700 1 -- 172.21.15.47:0/1286633704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7db0107510 msgr2=0x7f7db019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:21.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.381+0000 7f7daea00700 1 --2- 172.21.15.47:0/1286633704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7db0107510 0x7f7db019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.381+0000 7f7daea00700 1 -- 172.21.15.47:0/1286633704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db019b1c0 con 0x7f7db0102440 2024-03-21T11:22:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.381+0000 7f7da6000700 1 --2- 172.21.15.47:0/1286633704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7db0102df0 0x7f7db01a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.381+0000 7f7daea00700 1 --2- 172.21.15.47:0/1286633704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7db0102440 0x7f7db01a08a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7da000ef10 tx=0x7f7da000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.381+0000 7f7da7e00700 1 -- 172.21.15.47:0/1286633704 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da000c9c0 con 0x7f7db0102440 2024-03-21T11:22:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.382+0000 7f7da7e00700 1 -- 172.21.15.47:0/1286633704 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7da0019070 con 0x7f7db0102440 2024-03-21T11:22:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.382+0000 7f7db5600700 1 -- 172.21.15.47:0/1286633704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db019b450 con 0x7f7db0102440 2024-03-21T11:22:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.382+0000 7f7da7e00700 1 -- 172.21.15.47:0/1286633704 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da00146e0 con 0x7f7db0102440 2024-03-21T11:22:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.382+0000 7f7db5600700 1 -- 172.21.15.47:0/1286633704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db01a7770 con 0x7f7db0102440 2024-03-21T11:22:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.383+0000 7f7db5600700 1 -- 172.21.15.47:0/1286633704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db0068e50 con 0x7f7db0102440 2024-03-21T11:22:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.385+0000 7f7da7e00700 1 -- 172.21.15.47:0/1286633704 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f7da0012050 con 0x7f7db0102440 2024-03-21T11:22:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.386+0000 7f7da7e00700 1 --2- 172.21.15.47:0/1286633704 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d940a30c0 0x7f7d940a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.386+0000 7f7da7e00700 1 -- 172.21.15.47:0/1286633704 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f7da00cf810 con 0x7f7db0102440 2024-03-21T11:22:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.386+0000 7f7da6000700 1 --2- 172.21.15.47:0/1286633704 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d940a30c0 0x7f7d940a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:21.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.387+0000 7f7da6000700 1 --2- 172.21.15.47:0/1286633704 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d940a30c0 0x7f7d940a5580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7db01020b0 tx=0x7f7d9801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.390+0000 7f7da7e00700 1 -- 172.21.15.47:0/1286633704 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7da0010040 con 0x7f7db0102440 2024-03-21T11:22:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.649+0000 7f7db5600700 1 -- 172.21.15.47:0/1286633704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7db019bf90 con 0x7f7db0102440 2024-03-21T11:22:21.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.650+0000 7f7da7e00700 1 -- 172.21.15.47:0/1286633704 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7da009c120 con 0x7f7db0102440 2024-03-21T11:22:21.651 INFO:teuthology.orchestra.run.smithi047.stdout:223338299447 2024-03-21T11:22:21.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.654+0000 7f7db5600700 1 -- 172.21.15.47:0/1286633704 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d940a30c0 msgr2=0x7f7d940a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:21.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.654+0000 7f7db5600700 1 --2- 172.21.15.47:0/1286633704 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d940a30c0 0x7f7d940a5580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7db01020b0 tx=0x7f7d9801a040 comp rx=0 tx=0).stop 2024-03-21T11:22:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.654+0000 7f7db5600700 1 -- 172.21.15.47:0/1286633704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7db0102440 msgr2=0x7f7db01a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.654+0000 7f7db5600700 1 --2- 172.21.15.47:0/1286633704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7db0102440 0x7f7db01a08a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7da000ef10 tx=0x7f7da000c3a0 comp rx=0 tx=0).stop 2024-03-21T11:22:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.654+0000 7f7db5600700 1 -- 172.21.15.47:0/1286633704 shutdown_connections 2024-03-21T11:22:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.654+0000 7f7db5600700 1 --2- 172.21.15.47:0/1286633704 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d940a30c0 0x7f7d940a5580 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.654+0000 7f7db5600700 1 --2- 172.21.15.47:0/1286633704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7db0107510 0x7f7db019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:21.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.654+0000 7f7db5600700 1 --2- 172.21.15.47:0/1286633704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7db0102440 0x7f7db01a08a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.654+0000 7f7db5600700 1 --2- 172.21.15.47:0/1286633704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7db0102df0 0x7f7db01a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.654+0000 7f7db5600700 1 -- 172.21.15.47:0/1286633704 >> 172.21.15.47:0/1286633704 conn(0x7f7db00792f0 msgr2=0x7f7db010d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.654+0000 7f7db5600700 1 -- 172.21.15.47:0/1286633704 shutdown_connections 2024-03-21T11:22:21.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:21.655+0000 7f7db5600700 1 -- 172.21.15.47:0/1286633704 wait complete. 2024-03-21T11:22:22.032 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888129 got 146028888134 for osd.4 2024-03-21T11:22:22.032 DEBUG:teuthology.parallel:result is None 2024-03-21T11:22:22.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:22 smithi047 ceph-mon[31351]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:22:22.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:22 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1286633704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T11:22:22.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:22 smithi100 ceph-mon[36245]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:22:22.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:22 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1286633704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T11:22:22.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:22 smithi102 ceph-mon[36748]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:22:22.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:22 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1286633704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T11:22:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.679+0000 7ff277400700 1 -- 172.21.15.47:0/2765832044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270102440 msgr2=0x7ff270102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.679+0000 7ff277400700 1 --2- 172.21.15.47:0/2765832044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270102440 0x7ff270102820 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff264009990 tx=0x7ff264009ca0 comp rx=0 tx=0).stop 2024-03-21T11:22:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.679+0000 7ff277400700 1 -- 172.21.15.47:0/2765832044 shutdown_connections 2024-03-21T11:22:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.679+0000 7ff277400700 1 --2- 172.21.15.47:0/2765832044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff270107510 0x7ff27010f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.679+0000 7ff277400700 1 --2- 172.21.15.47:0/2765832044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270102440 0x7ff270102820 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.679+0000 7ff277400700 1 --2- 172.21.15.47:0/2765832044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff270102df0 0x7ff270106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.679+0000 7ff277400700 1 -- 172.21.15.47:0/2765832044 >> 172.21.15.47:0/2765832044 conn(0x7ff2700792f0 msgr2=0x7ff270079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.679+0000 7ff277400700 1 -- 172.21.15.47:0/2765832044 shutdown_connections 2024-03-21T11:22:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.679+0000 7ff277400700 1 -- 172.21.15.47:0/2765832044 wait complete. 2024-03-21T11:22:22.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.680+0000 7ff277400700 1 Processor -- start 2024-03-21T11:22:22.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.680+0000 7ff277400700 1 -- start start 2024-03-21T11:22:22.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.680+0000 7ff277400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff270102440 0x7ff2701a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:22.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.680+0000 7ff277400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff270102df0 0x7ff2701a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:22.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff277400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270107510 0x7ff27019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:22.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff277400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff270112560 con 0x7ff270102df0 2024-03-21T11:22:22.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff277400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2701123e0 con 0x7ff270107510 2024-03-21T11:22:22.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff277400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2701126e0 con 0x7ff270102440 2024-03-21T11:22:22.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff274a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff270102440 0x7ff2701a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff274a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff270102440 0x7ff2701a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54800/0 (socket says 172.21.15.47:54800) 2024-03-21T11:22:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff274a00700 1 -- 172.21.15.47:0/3172158372 learned_addr learned my addr 172.21.15.47:0/3172158372 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff275400700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270107510 0x7ff27019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff26fe00700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff270102df0 0x7ff2701a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff275400700 1 -- 172.21.15.47:0/3172158372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff270102440 msgr2=0x7ff2701a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff275400700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff270102440 0x7ff2701a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff275400700 1 -- 172.21.15.47:0/3172158372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff270102df0 msgr2=0x7ff2701a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff275400700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff270102df0 0x7ff2701a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff275400700 1 -- 172.21.15.47:0/3172158372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27019b1c0 con 0x7ff270107510 2024-03-21T11:22:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff274a00700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff270102440 0x7ff2701a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.681+0000 7ff26fe00700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff270102df0 0x7ff2701a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:22:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.682+0000 7ff275400700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270107510 0x7ff27019aa60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff26800efc0 tx=0x7ff26800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:22.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.682+0000 7ff26d600700 1 -- 172.21.15.47:0/3172158372 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2680092e0 con 0x7ff270107510 2024-03-21T11:22:22.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.682+0000 7ff26d600700 1 -- 172.21.15.47:0/3172158372 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff268004bb0 con 0x7ff270107510 2024-03-21T11:22:22.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.682+0000 7ff277400700 1 -- 172.21.15.47:0/3172158372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27019b450 con 0x7ff270107510 2024-03-21T11:22:22.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.682+0000 7ff26d600700 1 -- 172.21.15.47:0/3172158372 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268004280 con 0x7ff270107510 2024-03-21T11:22:22.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.682+0000 7ff277400700 1 -- 172.21.15.47:0/3172158372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2701a7770 con 0x7ff270107510 2024-03-21T11:22:22.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.683+0000 7ff277400700 1 -- 172.21.15.47:0/3172158372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff270068e50 con 0x7ff270107510 2024-03-21T11:22:22.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.685+0000 7ff26d600700 1 -- 172.21.15.47:0/3172158372 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7ff26801b030 con 0x7ff270107510 2024-03-21T11:22:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.686+0000 7ff26d600700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2580a30c0 0x7ff2580a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.686+0000 7ff26d600700 1 -- 172.21.15.47:0/3172158372 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ff2680cf320 con 0x7ff270107510 2024-03-21T11:22:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.687+0000 7ff274a00700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2580a30c0 0x7ff2580a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:22.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.687+0000 7ff274a00700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2580a30c0 0x7ff2580a5580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff264006010 tx=0x7ff26401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.692+0000 7ff26d600700 1 -- 172.21.15.47:0/3172158372 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff268022020 con 0x7ff270107510 2024-03-21T11:22:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.941+0000 7ff277400700 1 -- 172.21.15.47:0/3172158372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7ff2701a7940 con 0x7ff270107510 2024-03-21T11:22:22.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.941+0000 7ff26d600700 1 -- 172.21.15.47:0/3172158372 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7ff268010040 con 0x7ff270107510 2024-03-21T11:22:22.943 INFO:teuthology.orchestra.run.smithi047.stdout:347892351010 2024-03-21T11:22:22.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.944+0000 7ff277400700 1 -- 172.21.15.47:0/3172158372 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2580a30c0 msgr2=0x7ff2580a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:22.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.944+0000 7ff277400700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2580a30c0 0x7ff2580a5580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff264006010 tx=0x7ff26401a040 comp rx=0 tx=0).stop 2024-03-21T11:22:22.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.944+0000 7ff277400700 1 -- 172.21.15.47:0/3172158372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270107510 msgr2=0x7ff27019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:22.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.944+0000 7ff277400700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270107510 0x7ff27019aa60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff26800efc0 tx=0x7ff26800c3a0 comp rx=0 tx=0).stop 2024-03-21T11:22:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.944+0000 7ff277400700 1 -- 172.21.15.47:0/3172158372 shutdown_connections 2024-03-21T11:22:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.945+0000 7ff277400700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff270107510 0x7ff27019aa60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.945+0000 7ff277400700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff270102440 0x7ff2701a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.945+0000 7ff277400700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2580a30c0 0x7ff2580a5580 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.945+0000 7ff277400700 1 --2- 172.21.15.47:0/3172158372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff270102df0 0x7ff2701a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.945+0000 7ff277400700 1 -- 172.21.15.47:0/3172158372 >> 172.21.15.47:0/3172158372 conn(0x7ff2700792f0 msgr2=0x7ff2701019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.945+0000 7ff277400700 1 -- 172.21.15.47:0/3172158372 shutdown_connections 2024-03-21T11:22:22.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:22.945+0000 7ff277400700 1 -- 172.21.15.47:0/3172158372 wait complete. 2024-03-21T11:22:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:23 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3172158372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T11:22:23.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:23 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3172158372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T11:22:23.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:23 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3172158372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T11:22:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.530+0000 7fcda2a00700 1 -- 172.21.15.47:0/1336040155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd9c1081c0 msgr2=0x7fcd9c1085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.530+0000 7fcda2a00700 1 --2- 172.21.15.47:0/1336040155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd9c1081c0 0x7fcd9c1085a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0099c0 tx=0x7fcd8c009cd0 comp rx=0 tx=0).stop 2024-03-21T11:22:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.530+0000 7fcda2a00700 1 -- 172.21.15.47:0/1336040155 shutdown_connections 2024-03-21T11:22:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.530+0000 7fcda2a00700 1 --2- 172.21.15.47:0/1336040155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd9c102ba0 0x7fcd9c10efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.530+0000 7fcda2a00700 1 --2- 172.21.15.47:0/1336040155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd9c1081c0 0x7fcd9c1085a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.530+0000 7fcda2a00700 1 --2- 172.21.15.47:0/1336040155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd9c1021e0 0x7fcd9c102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.530+0000 7fcda2a00700 1 -- 172.21.15.47:0/1336040155 >> 172.21.15.47:0/1336040155 conn(0x7fcd9c0792c0 msgr2=0x7fcd9c0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.530+0000 7fcda2a00700 1 -- 172.21.15.47:0/1336040155 shutdown_connections 2024-03-21T11:22:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.530+0000 7fcda2a00700 1 -- 172.21.15.47:0/1336040155 wait complete. 2024-03-21T11:22:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.532+0000 7fcda2a00700 1 Processor -- start 2024-03-21T11:22:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.532+0000 7fcda2a00700 1 -- start start 2024-03-21T11:22:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcda2a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd9c1021e0 0x7fcd9c1a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcda2a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd9c102ba0 0x7fcd9c1a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcda2a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd9c1081c0 0x7fcd9c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcda2a00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c112490 con 0x7fcd9c1081c0 2024-03-21T11:22:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcda2a00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c112310 con 0x7fcd9c1021e0 2024-03-21T11:22:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcda2a00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c112610 con 0x7fcd9c102ba0 2024-03-21T11:22:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcd9b400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd9c102ba0 0x7fcd9c1a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcd9be00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd9c1021e0 0x7fcd9c1a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcd9b400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd9c102ba0 0x7fcd9c1a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54830/0 (socket says 172.21.15.47:54830) 2024-03-21T11:22:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcd9b400700 1 -- 172.21.15.47:0/2693960135 learned_addr learned my addr 172.21.15.47:0/2693960135 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcda0a00700 1 --2- 172.21.15.47:0/2693960135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd9c1081c0 0x7fcd9c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcd9be00700 1 -- 172.21.15.47:0/2693960135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd9c102ba0 msgr2=0x7fcd9c1a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcd9be00700 1 --2- 172.21.15.47:0/2693960135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd9c102ba0 0x7fcd9c1a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcd9be00700 1 -- 172.21.15.47:0/2693960135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd9c1081c0 msgr2=0x7fcd9c19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.533+0000 7fcd9be00700 1 --2- 172.21.15.47:0/2693960135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd9c1081c0 0x7fcd9c19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:23.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.534+0000 7fcd9be00700 1 -- 172.21.15.47:0/2693960135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c19b0d0 con 0x7fcd9c1021e0 2024-03-21T11:22:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.534+0000 7fcd9be00700 1 --2- 172.21.15.47:0/2693960135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd9c1021e0 0x7fcd9c1a07b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0098e0 tx=0x7fcd8c005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.534+0000 7fcd98c00700 1 -- 172.21.15.47:0/2693960135 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c027070 con 0x7fcd9c1021e0 2024-03-21T11:22:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.534+0000 7fcd98c00700 1 -- 172.21.15.47:0/2693960135 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd8c003680 con 0x7fcd9c1021e0 2024-03-21T11:22:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.534+0000 7fcda2a00700 1 -- 172.21.15.47:0/2693960135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9c19b360 con 0x7fcd9c1021e0 2024-03-21T11:22:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.534+0000 7fcd98c00700 1 -- 172.21.15.47:0/2693960135 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c0173f0 con 0x7fcd9c1021e0 2024-03-21T11:22:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.534+0000 7fcda2a00700 1 -- 172.21.15.47:0/2693960135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9c1a7680 con 0x7fcd9c1021e0 2024-03-21T11:22:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.536+0000 7fcda2a00700 1 -- 172.21.15.47:0/2693960135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9c068e50 con 0x7fcd9c1021e0 2024-03-21T11:22:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.537+0000 7fcd98c00700 1 -- 172.21.15.47:0/2693960135 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fcd8c02a070 con 0x7fcd9c1021e0 2024-03-21T11:22:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.538+0000 7fcd98c00700 1 --2- 172.21.15.47:0/2693960135 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcd800a30c0 0x7fcd800a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.539+0000 7fcd98c00700 1 -- 172.21.15.47:0/2693960135 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fcd8c0d03c0 con 0x7fcd9c1021e0 2024-03-21T11:22:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.539+0000 7fcd9b400700 1 --2- 172.21.15.47:0/2693960135 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcd800a30c0 0x7fcd800a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.539+0000 7fcd9b400700 1 --2- 172.21.15.47:0/2693960135 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcd800a30c0 0x7fcd800a5580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c079b50 tx=0x7fcd88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.542+0000 7fcd98c00700 1 -- 172.21.15.47:0/2693960135 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcd8c05e740 con 0x7fcd9c1021e0 2024-03-21T11:22:23.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.768+0000 7fcda2a00700 1 -- 172.21.15.47:0/2693960135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fcd9c1a7850 con 0x7fcd9c1021e0 2024-03-21T11:22:23.770 INFO:teuthology.orchestra.run.smithi047.stdout:257698037811 2024-03-21T11:22:23.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.769+0000 7fcd98c00700 1 -- 172.21.15.47:0/2693960135 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd8c013070 con 0x7fcd9c1021e0 2024-03-21T11:22:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.771+0000 7fcda2a00700 1 -- 172.21.15.47:0/2693960135 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcd800a30c0 msgr2=0x7fcd800a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:23.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.771+0000 7fcda2a00700 1 --2- 172.21.15.47:0/2693960135 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcd800a30c0 0x7fcd800a5580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c079b50 tx=0x7fcd88009260 comp rx=0 tx=0).stop 2024-03-21T11:22:23.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.771+0000 7fcda2a00700 1 -- 172.21.15.47:0/2693960135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd9c1021e0 msgr2=0x7fcd9c1a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:23.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.771+0000 7fcda2a00700 1 --2- 172.21.15.47:0/2693960135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd9c1021e0 0x7fcd9c1a07b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0098e0 tx=0x7fcd8c005f10 comp rx=0 tx=0).stop 2024-03-21T11:22:23.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.772+0000 7fcda2a00700 1 -- 172.21.15.47:0/2693960135 shutdown_connections 2024-03-21T11:22:23.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.772+0000 7fcda2a00700 1 --2- 172.21.15.47:0/2693960135 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcd800a30c0 0x7fcd800a5580 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:23.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.772+0000 7fcda2a00700 1 --2- 172.21.15.47:0/2693960135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd9c1081c0 0x7fcd9c19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:23.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.772+0000 7fcda2a00700 1 --2- 172.21.15.47:0/2693960135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd9c1021e0 0x7fcd9c1a07b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:23.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.772+0000 7fcda2a00700 1 --2- 172.21.15.47:0/2693960135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd9c102ba0 0x7fcd9c1a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:23.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.772+0000 7fcda2a00700 1 -- 172.21.15.47:0/2693960135 >> 172.21.15.47:0/2693960135 conn(0x7fcd9c0792c0 msgr2=0x7fcd9c074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:23.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.772+0000 7fcda2a00700 1 -- 172.21.15.47:0/2693960135 shutdown_connections 2024-03-21T11:22:23.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:23.772+0000 7fcda2a00700 1 -- 172.21.15.47:0/2693960135 wait complete. 2024-03-21T11:22:24.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.281+0000 7f4ec1e00700 1 -- 172.21.15.47:0/597769570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ebc074580 msgr2=0x7f4ebc074960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:24.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.281+0000 7f4ec1e00700 1 --2- 172.21.15.47:0/597769570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ebc074580 0x7f4ebc074960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009990 tx=0x7f4eac009ca0 comp rx=0 tx=0).stop 2024-03-21T11:22:24.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.281+0000 7f4ec1e00700 1 -- 172.21.15.47:0/597769570 shutdown_connections 2024-03-21T11:22:24.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.281+0000 7f4ec1e00700 1 --2- 172.21.15.47:0/597769570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc10ea30 0x7f4ebc110e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:24.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.281+0000 7f4ec1e00700 1 --2- 172.21.15.47:0/597769570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ebc074580 0x7f4ebc074960 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:24.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.281+0000 7f4ec1e00700 1 --2- 172.21.15.47:0/597769570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ebc074ea0 0x7f4ebc10e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:24.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.281+0000 7f4ec1e00700 1 -- 172.21.15.47:0/597769570 >> 172.21.15.47:0/597769570 conn(0x7f4ebc0fdc60 msgr2=0x7f4ebc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:24.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.281+0000 7f4ec1e00700 1 -- 172.21.15.47:0/597769570 shutdown_connections 2024-03-21T11:22:24.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.282+0000 7f4ec1e00700 1 -- 172.21.15.47:0/597769570 wait complete. 2024-03-21T11:22:24.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.282+0000 7f4ec1e00700 1 Processor -- start 2024-03-21T11:22:24.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.283+0000 7f4ec1e00700 1 -- start start 2024-03-21T11:22:24.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.283+0000 7f4ec1e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc074580 0x7f4ebc1a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:24.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.283+0000 7f4ec1e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ebc074ea0 0x7f4ebc1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:24.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.283+0000 7f4ec1e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ebc10ea30 0x7f4ebc19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.283+0000 7f4ec1e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc114760 con 0x7f4ebc074580 2024-03-21T11:22:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.283+0000 7f4ec1e00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc1145e0 con 0x7f4ebc10ea30 2024-03-21T11:22:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.283+0000 7f4ec1e00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc1148e0 con 0x7f4ebc074ea0 2024-03-21T11:22:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.283+0000 7f4eba000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ebc074ea0 0x7f4ebc1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.283+0000 7f4ebb400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ebc10ea30 0x7f4ebc19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.283+0000 7f4ebaa00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc074580 0x7f4ebc1a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.283+0000 7f4eba000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ebc074ea0 0x7f4ebc1a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54842/0 (socket says 172.21.15.47:54842) 2024-03-21T11:22:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.283+0000 7f4eba000700 1 -- 172.21.15.47:0/2059907085 learned_addr learned my addr 172.21.15.47:0/2059907085 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.284+0000 7f4eba000700 1 -- 172.21.15.47:0/2059907085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ebc10ea30 msgr2=0x7f4ebc19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.284+0000 7f4eba000700 1 --2- 172.21.15.47:0/2059907085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ebc10ea30 0x7f4ebc19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.284+0000 7f4eba000700 1 -- 172.21.15.47:0/2059907085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc074580 msgr2=0x7f4ebc1a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.284+0000 7f4eba000700 1 --2- 172.21.15.47:0/2059907085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc074580 0x7f4ebc1a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.284+0000 7f4eba000700 1 -- 172.21.15.47:0/2059907085 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc19b0e0 con 0x7f4ebc074ea0 2024-03-21T11:22:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.284+0000 7f4eba000700 1 --2- 172.21.15.47:0/2059907085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ebc074ea0 0x7f4ebc1a0d00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800e9c0 tx=0x7f4ea800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.284+0000 7f4ec0a00700 1 -- 172.21.15.47:0/2059907085 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea800cbd0 con 0x7f4ebc074ea0 2024-03-21T11:22:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.284+0000 7f4ec0a00700 1 -- 172.21.15.47:0/2059907085 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ea800ee60 con 0x7f4ebc074ea0 2024-03-21T11:22:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.284+0000 7f4ec0a00700 1 -- 172.21.15.47:0/2059907085 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8005630 con 0x7f4ebc074ea0 2024-03-21T11:22:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.284+0000 7f4ec1e00700 1 -- 172.21.15.47:0/2059907085 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ebc19b3d0 con 0x7f4ebc074ea0 2024-03-21T11:22:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.285+0000 7f4ec1e00700 1 -- 172.21.15.47:0/2059907085 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ebc1a7690 con 0x7f4ebc074ea0 2024-03-21T11:22:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.286+0000 7f4ec1e00700 1 -- 172.21.15.47:0/2059907085 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ebc068e50 con 0x7f4ebc074ea0 2024-03-21T11:22:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.288+0000 7f4ec0a00700 1 -- 172.21.15.47:0/2059907085 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f4ea8044b80 con 0x7f4ebc074ea0 2024-03-21T11:22:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.289+0000 7f4ec0a00700 1 --2- 172.21.15.47:0/2059907085 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4ea00a3040 0x7f4ea00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.289+0000 7f4ec0a00700 1 -- 172.21.15.47:0/2059907085 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f4ea80cfbd0 con 0x7f4ebc074ea0 2024-03-21T11:22:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.289+0000 7f4ebaa00700 1 --2- 172.21.15.47:0/2059907085 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4ea00a3040 0x7f4ea00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.290+0000 7f4ebaa00700 1 --2- 172.21.15.47:0/2059907085 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4ea00a3040 0x7f4ea00a5500 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc19bac0 tx=0x7f4eac00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:24.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.291+0000 7f4ec0a00700 1 -- 172.21.15.47:0/2059907085 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4ea809c4e0 con 0x7f4ebc074ea0 2024-03-21T11:22:24.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:24 smithi100 ceph-mon[36245]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:22:24.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:24 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2693960135' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T11:22:24.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:24 smithi047 ceph-mon[31351]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:22:24.436 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:24 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2693960135' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T11:22:24.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:24 smithi102 ceph-mon[36748]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:22:24.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:24 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2693960135' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T11:22:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.524+0000 7f4ec1e00700 1 -- 172.21.15.47:0/2059907085 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f4ebc02da30 con 0x7f4ebc074ea0 2024-03-21T11:22:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.524+0000 7f4ec0a00700 1 -- 172.21.15.47:0/2059907085 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f4ea809bde0 con 0x7f4ebc074ea0 2024-03-21T11:22:24.525 INFO:teuthology.orchestra.run.smithi047.stdout:313532612647 2024-03-21T11:22:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.527+0000 7f4ec1e00700 1 -- 172.21.15.47:0/2059907085 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4ea00a3040 msgr2=0x7f4ea00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.527+0000 7f4ec1e00700 1 --2- 172.21.15.47:0/2059907085 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4ea00a3040 0x7f4ea00a5500 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc19bac0 tx=0x7f4eac00b260 comp rx=0 tx=0).stop 2024-03-21T11:22:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.527+0000 7f4ec1e00700 1 -- 172.21.15.47:0/2059907085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ebc074ea0 msgr2=0x7f4ebc1a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.527+0000 7f4ec1e00700 1 --2- 172.21.15.47:0/2059907085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ebc074ea0 0x7f4ebc1a0d00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800e9c0 tx=0x7f4ea800ecd0 comp rx=0 tx=0).stop 2024-03-21T11:22:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.528+0000 7f4ec1e00700 1 -- 172.21.15.47:0/2059907085 shutdown_connections 2024-03-21T11:22:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.528+0000 7f4ec1e00700 1 --2- 172.21.15.47:0/2059907085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ebc10ea30 0x7f4ebc19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.528+0000 7f4ec1e00700 1 --2- 172.21.15.47:0/2059907085 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4ea00a3040 0x7f4ea00a5500 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.528+0000 7f4ec1e00700 1 --2- 172.21.15.47:0/2059907085 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ebc074580 0x7f4ebc1a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.528+0000 7f4ec1e00700 1 --2- 172.21.15.47:0/2059907085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ebc074ea0 0x7f4ebc1a0d00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.528+0000 7f4ec1e00700 1 -- 172.21.15.47:0/2059907085 >> 172.21.15.47:0/2059907085 conn(0x7f4ebc0fdc60 msgr2=0x7f4ebc0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.528+0000 7f4ec1e00700 1 -- 172.21.15.47:0/2059907085 shutdown_connections 2024-03-21T11:22:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:24.528+0000 7f4ec1e00700 1 -- 172.21.15.47:0/2059907085 wait complete. 2024-03-21T11:22:25.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:25 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2059907085' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T11:22:25.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.256+0000 7f01cc400700 1 -- 172.21.15.47:0/4219099077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c4102dc0 msgr2=0x7f01c4106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:25.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.256+0000 7f01cc400700 1 --2- 172.21.15.47:0/4219099077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c4102dc0 0x7f01c4106e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f01b00099c0 tx=0x7f01b0009cd0 comp rx=0 tx=0).stop 2024-03-21T11:22:25.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.256+0000 7f01cc400700 1 -- 172.21.15.47:0/4219099077 shutdown_connections 2024-03-21T11:22:25.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.256+0000 7f01cc400700 1 --2- 172.21.15.47:0/4219099077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c41074f0 0x7f01c410f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:25.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.256+0000 7f01cc400700 1 --2- 172.21.15.47:0/4219099077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c4102410 0x7f01c41027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:25.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.256+0000 7f01cc400700 1 --2- 172.21.15.47:0/4219099077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c4102dc0 0x7f01c4106e10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:25.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.256+0000 7f01cc400700 1 -- 172.21.15.47:0/4219099077 >> 172.21.15.47:0/4219099077 conn(0x7f01c40fdc60 msgr2=0x7f01c4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:25.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.256+0000 7f01cc400700 1 -- 172.21.15.47:0/4219099077 shutdown_connections 2024-03-21T11:22:25.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.256+0000 7f01cc400700 1 -- 172.21.15.47:0/4219099077 wait complete. 2024-03-21T11:22:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.257+0000 7f01cc400700 1 Processor -- start 2024-03-21T11:22:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.258+0000 7f01cc400700 1 -- start start 2024-03-21T11:22:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.258+0000 7f01cc400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c4102410 0x7f01c4074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.258+0000 7f01cc400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c4102dc0 0x7f01c40749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.258+0000 7f01cc400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c41074f0 0x7f01c4074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.258+0000 7f01cc400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4112500 con 0x7f01c4102dc0 2024-03-21T11:22:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.258+0000 7f01cc400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4112380 con 0x7f01c4102410 2024-03-21T11:22:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.258+0000 7f01cc400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4112680 con 0x7f01c41074f0 2024-03-21T11:22:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.258+0000 7f01c9000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c4102dc0 0x7f01c40749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.258+0000 7f01c9a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c4102410 0x7f01c4074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.258+0000 7f01ca400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c41074f0 0x7f01c4074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.258+0000 7f01c9000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c4102dc0 0x7f01c40749d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52002/0 (socket says 172.21.15.47:52002) 2024-03-21T11:22:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.258+0000 7f01ca400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c41074f0 0x7f01c4074f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54864/0 (socket says 172.21.15.47:54864) 2024-03-21T11:22:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.258+0000 7f01c9a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c4102410 0x7f01c4074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38134/0 (socket says 172.21.15.47:38134) 2024-03-21T11:22:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.258+0000 7f01c9000700 1 -- 172.21.15.47:0/2919862981 learned_addr learned my addr 172.21.15.47:0/2919862981 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.259+0000 7f01c9a00700 1 -- 172.21.15.47:0/2919862981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c41074f0 msgr2=0x7f01c4074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.259+0000 7f01c9a00700 1 --2- 172.21.15.47:0/2919862981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c41074f0 0x7f01c4074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.259+0000 7f01c9a00700 1 -- 172.21.15.47:0/2919862981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c4102dc0 msgr2=0x7f01c40749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.259+0000 7f01c9a00700 1 --2- 172.21.15.47:0/2919862981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c4102dc0 0x7f01c40749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.259+0000 7f01c9a00700 1 -- 172.21.15.47:0/2919862981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c41a6ee0 con 0x7f01c4102410 2024-03-21T11:22:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.259+0000 7f01ca400700 1 --2- 172.21.15.47:0/2919862981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c41074f0 0x7f01c4074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.259+0000 7f01c9000700 1 --2- 172.21.15.47:0/2919862981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c4102dc0 0x7f01c40749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.259+0000 7f01c9a00700 1 --2- 172.21.15.47:0/2919862981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c4102410 0x7f01c4074490 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f01b800ef10 tx=0x7f01b800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.259+0000 7f01c2000700 1 -- 172.21.15.47:0/2919862981 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b800c9b0 con 0x7f01c4102410 2024-03-21T11:22:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.260+0000 7f01c2000700 1 -- 172.21.15.47:0/2919862981 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01b8019070 con 0x7f01c4102410 2024-03-21T11:22:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.260+0000 7f01cc400700 1 -- 172.21.15.47:0/2919862981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c41a7170 con 0x7f01c4102410 2024-03-21T11:22:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.260+0000 7f01c2000700 1 -- 172.21.15.47:0/2919862981 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b80146e0 con 0x7f01c4102410 2024-03-21T11:22:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.260+0000 7f01cc400700 1 -- 172.21.15.47:0/2919862981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c41a7700 con 0x7f01c4102410 2024-03-21T11:22:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.261+0000 7f01cc400700 1 -- 172.21.15.47:0/2919862981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c4068e50 con 0x7f01c4102410 2024-03-21T11:22:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.262+0000 7f01c2000700 1 -- 172.21.15.47:0/2919862981 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f01b8012050 con 0x7f01c4102410 2024-03-21T11:22:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.264+0000 7f01c2000700 1 --2- 172.21.15.47:0/2919862981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01ac0a30c0 0x7f01ac0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.264+0000 7f01c2000700 1 -- 172.21.15.47:0/2919862981 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f01b80cf870 con 0x7f01c4102410 2024-03-21T11:22:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.264+0000 7f01c9000700 1 --2- 172.21.15.47:0/2919862981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01ac0a30c0 0x7f01ac0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.265+0000 7f01c9000700 1 --2- 172.21.15.47:0/2919862981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01ac0a30c0 0x7f01ac0a5580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f01c4101340 tx=0x7f01b0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.271+0000 7f01c2000700 1 -- 172.21.15.47:0/2919862981 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f01b8010040 con 0x7f01c4102410 2024-03-21T11:22:25.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:25 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2059907085' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T11:22:25.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:25 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2059907085' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T11:22:25.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.510+0000 7f01cc400700 1 -- 172.21.15.47:0/2919862981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f01c402da30 con 0x7f01c4102410 2024-03-21T11:22:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.511+0000 7f01c2000700 1 -- 172.21.15.47:0/2919862981 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01b809c180 con 0x7f01c4102410 2024-03-21T11:22:25.512 INFO:teuthology.orchestra.run.smithi047.stdout:34359738463 2024-03-21T11:22:25.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.514+0000 7f01cc400700 1 -- 172.21.15.47:0/2919862981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01ac0a30c0 msgr2=0x7f01ac0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:25.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.514+0000 7f01cc400700 1 --2- 172.21.15.47:0/2919862981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01ac0a30c0 0x7f01ac0a5580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f01c4101340 tx=0x7f01b0005fb0 comp rx=0 tx=0).stop 2024-03-21T11:22:25.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.514+0000 7f01cc400700 1 -- 172.21.15.47:0/2919862981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c4102410 msgr2=0x7f01c4074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:25.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.514+0000 7f01cc400700 1 --2- 172.21.15.47:0/2919862981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c4102410 0x7f01c4074490 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f01b800ef10 tx=0x7f01b800c3a0 comp rx=0 tx=0).stop 2024-03-21T11:22:25.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.514+0000 7f01cc400700 1 -- 172.21.15.47:0/2919862981 shutdown_connections 2024-03-21T11:22:25.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.514+0000 7f01cc400700 1 --2- 172.21.15.47:0/2919862981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c41074f0 0x7f01c4074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:25.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.514+0000 7f01cc400700 1 --2- 172.21.15.47:0/2919862981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c4102410 0x7f01c4074490 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:25.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.514+0000 7f01cc400700 1 --2- 172.21.15.47:0/2919862981 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01ac0a30c0 0x7f01ac0a5580 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:25.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.514+0000 7f01cc400700 1 --2- 172.21.15.47:0/2919862981 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c4102dc0 0x7f01c40749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:25.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.514+0000 7f01cc400700 1 -- 172.21.15.47:0/2919862981 >> 172.21.15.47:0/2919862981 conn(0x7f01c40fdc60 msgr2=0x7f01c40fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:25.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.515+0000 7f01cc400700 1 -- 172.21.15.47:0/2919862981 shutdown_connections 2024-03-21T11:22:25.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:25.515+0000 7f01cc400700 1 -- 172.21.15.47:0/2919862981 wait complete. 2024-03-21T11:22:26.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:26 smithi100 ceph-mon[36245]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:22:26.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:26 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2919862981' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T11:22:26.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:26 smithi102 ceph-mon[36748]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:22:26.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:26 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2919862981' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T11:22:26.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:26 smithi047 ceph-mon[31351]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:22:26.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:26 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2919862981' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T11:22:26.759 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299443 got 223338299447 for osd.7 2024-03-21T11:22:26.759 DEBUG:teuthology.parallel:result is None 2024-03-21T11:22:26.873 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574931 got 55834574937 for osd.1 2024-03-21T11:22:26.874 DEBUG:teuthology.parallel:result is None 2024-03-21T11:22:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.059+0000 7f8c44000700 1 -- 172.21.15.47:0/2423421754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3c102440 msgr2=0x7f8c3c102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.059+0000 7f8c44000700 1 --2- 172.21.15.47:0/2423421754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3c102440 0x7f8c3c102820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009990 tx=0x7f8c30009ca0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.059+0000 7f8c44000700 1 -- 172.21.15.47:0/2423421754 shutdown_connections 2024-03-21T11:22:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.059+0000 7f8c44000700 1 --2- 172.21.15.47:0/2423421754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c3c107510 0x7f8c3c10f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.059+0000 7f8c44000700 1 --2- 172.21.15.47:0/2423421754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3c102440 0x7f8c3c102820 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.059+0000 7f8c44000700 1 --2- 172.21.15.47:0/2423421754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c3c102df0 0x7f8c3c106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.059+0000 7f8c44000700 1 -- 172.21.15.47:0/2423421754 >> 172.21.15.47:0/2423421754 conn(0x7f8c3c0792f0 msgr2=0x7f8c3c079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.059+0000 7f8c44000700 1 -- 172.21.15.47:0/2423421754 shutdown_connections 2024-03-21T11:22:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.059+0000 7f8c44000700 1 -- 172.21.15.47:0/2423421754 wait complete. 2024-03-21T11:22:28.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.061+0000 7f8c44000700 1 Processor -- start 2024-03-21T11:22:28.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.061+0000 7f8c44000700 1 -- start start 2024-03-21T11:22:28.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.061+0000 7f8c44000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c3c102440 0x7f8c3c1a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:28.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.061+0000 7f8c44000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3c102df0 0x7f8c3c1a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:28.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.061+0000 7f8c44000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c3c107510 0x7f8c3c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:28.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.061+0000 7f8c44000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c112540 con 0x7f8c3c102440 2024-03-21T11:22:28.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.061+0000 7f8c44000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c1123c0 con 0x7f8c3c102df0 2024-03-21T11:22:28.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.061+0000 7f8c44000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c1126c0 con 0x7f8c3c107510 2024-03-21T11:22:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.061+0000 7f8c41600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c3c102440 0x7f8c3c1a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.061+0000 7f8c40c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3c102df0 0x7f8c3c1a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.061+0000 7f8c42000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c3c107510 0x7f8c3c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.062+0000 7f8c40c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3c102df0 0x7f8c3c1a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38162/0 (socket says 172.21.15.47:38162) 2024-03-21T11:22:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.062+0000 7f8c40c00700 1 -- 172.21.15.47:0/2368017210 learned_addr learned my addr 172.21.15.47:0/2368017210 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.062+0000 7f8c41600700 1 --2- 172.21.15.47:0/2368017210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c3c102440 0x7f8c3c1a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52042/0 (socket says 172.21.15.47:52042) 2024-03-21T11:22:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.062+0000 7f8c40c00700 1 -- 172.21.15.47:0/2368017210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c3c107510 msgr2=0x7f8c3c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.062+0000 7f8c40c00700 1 --2- 172.21.15.47:0/2368017210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c3c107510 0x7f8c3c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.062+0000 7f8c40c00700 1 -- 172.21.15.47:0/2368017210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c3c102440 msgr2=0x7f8c3c1a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.062+0000 7f8c40c00700 1 --2- 172.21.15.47:0/2368017210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c3c102440 0x7f8c3c1a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.062+0000 7f8c40c00700 1 -- 172.21.15.47:0/2368017210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c19b160 con 0x7f8c3c102df0 2024-03-21T11:22:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.062+0000 7f8c42000700 1 --2- 172.21.15.47:0/2368017210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c3c107510 0x7f8c3c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.062+0000 7f8c40c00700 1 --2- 172.21.15.47:0/2368017210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3c102df0 0x7f8c3c1a0d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800e970 tx=0x7f8c2800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.063+0000 7f8c3a000700 1 -- 172.21.15.47:0/2368017210 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28004a30 con 0x7f8c3c102df0 2024-03-21T11:22:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.063+0000 7f8c3a000700 1 -- 172.21.15.47:0/2368017210 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c2800ccc0 con 0x7f8c3c102df0 2024-03-21T11:22:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.063+0000 7f8c3a000700 1 -- 172.21.15.47:0/2368017210 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28005650 con 0x7f8c3c102df0 2024-03-21T11:22:28.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.063+0000 7f8c44000700 1 -- 172.21.15.47:0/2368017210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c19b450 con 0x7f8c3c102df0 2024-03-21T11:22:28.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.063+0000 7f8c44000700 1 -- 172.21.15.47:0/2368017210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c1a7710 con 0x7f8c3c102df0 2024-03-21T11:22:28.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.064+0000 7f8c44000700 1 -- 172.21.15.47:0/2368017210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c068e50 con 0x7f8c3c102df0 2024-03-21T11:22:28.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.066+0000 7f8c3a000700 1 -- 172.21.15.47:0/2368017210 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f8c28004bd0 con 0x7f8c3c102df0 2024-03-21T11:22:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.067+0000 7f8c3a000700 1 --2- 172.21.15.47:0/2368017210 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8c240a3040 0x7f8c240a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.068+0000 7f8c3a000700 1 -- 172.21.15.47:0/2368017210 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f8c280d0230 con 0x7f8c3c102df0 2024-03-21T11:22:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.068+0000 7f8c41600700 1 --2- 172.21.15.47:0/2368017210 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8c240a3040 0x7f8c240a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.069+0000 7f8c41600700 1 --2- 172.21.15.47:0/2368017210 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8c240a3040 0x7f8c240a5500 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c19bb40 tx=0x7f8c3001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.074+0000 7f8c3a000700 1 -- 172.21.15.47:0/2368017210 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8c2809cb40 con 0x7f8c3c102df0 2024-03-21T11:22:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.307+0000 7f8c44000700 1 -- 172.21.15.47:0/2368017210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8c3c19bf30 con 0x7f8c3c102df0 2024-03-21T11:22:28.308 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:28 smithi047 ceph-mon[31351]: pgmap v372: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:22:28.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.307+0000 7f8c3a000700 1 -- 172.21.15.47:0/2368017210 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8c3c19bf30 con 0x7f8c3c102df0 2024-03-21T11:22:28.308 INFO:teuthology.orchestra.run.smithi047.stdout:167503724612 2024-03-21T11:22:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.311+0000 7f8c44000700 1 -- 172.21.15.47:0/2368017210 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8c240a3040 msgr2=0x7f8c240a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.311+0000 7f8c44000700 1 --2- 172.21.15.47:0/2368017210 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8c240a3040 0x7f8c240a5500 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c19bb40 tx=0x7f8c3001a040 comp rx=0 tx=0).stop 2024-03-21T11:22:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.311+0000 7f8c44000700 1 -- 172.21.15.47:0/2368017210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3c102df0 msgr2=0x7f8c3c1a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.311+0000 7f8c44000700 1 --2- 172.21.15.47:0/2368017210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3c102df0 0x7f8c3c1a0d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800e970 tx=0x7f8c2800ec80 comp rx=0 tx=0).stop 2024-03-21T11:22:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.311+0000 7f8c44000700 1 -- 172.21.15.47:0/2368017210 shutdown_connections 2024-03-21T11:22:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.311+0000 7f8c44000700 1 --2- 172.21.15.47:0/2368017210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c3c107510 0x7f8c3c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.311+0000 7f8c44000700 1 --2- 172.21.15.47:0/2368017210 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8c240a3040 0x7f8c240a5500 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.311+0000 7f8c44000700 1 --2- 172.21.15.47:0/2368017210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c3c102440 0x7f8c3c1a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.311+0000 7f8c44000700 1 --2- 172.21.15.47:0/2368017210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c3c102df0 0x7f8c3c1a0d80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.311+0000 7f8c44000700 1 -- 172.21.15.47:0/2368017210 >> 172.21.15.47:0/2368017210 conn(0x7f8c3c0792f0 msgr2=0x7f8c3c075570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.312+0000 7f8c44000700 1 -- 172.21.15.47:0/2368017210 shutdown_connections 2024-03-21T11:22:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.312+0000 7f8c44000700 1 -- 172.21.15.47:0/2368017210 wait complete. 2024-03-21T11:22:28.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:28 smithi100 ceph-mon[36245]: pgmap v372: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:22:28.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:28 smithi102 ceph-mon[36748]: pgmap v372: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:22:28.525 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612642 got 313532612647 for osd.10 2024-03-21T11:22:28.525 DEBUG:teuthology.parallel:result is None 2024-03-21T11:22:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.937+0000 7f0a72c00700 1 -- 172.21.15.47:0/3798415320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a6c10eb90 msgr2=0x7f0a6c110f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.937+0000 7f0a72c00700 1 --2- 172.21.15.47:0/3798415320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a6c10eb90 0x7f0a6c110f80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400b0a0 tx=0x7f0a6400b3b0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.937+0000 7f0a72c00700 1 -- 172.21.15.47:0/3798415320 shutdown_connections 2024-03-21T11:22:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.937+0000 7f0a72c00700 1 --2- 172.21.15.47:0/3798415320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a6c10eb90 0x7f0a6c110f80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.937+0000 7f0a72c00700 1 --2- 172.21.15.47:0/3798415320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a6c0743c0 0x7f0a6c0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.937+0000 7f0a72c00700 1 --2- 172.21.15.47:0/3798415320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a6c074ce0 0x7f0a6c10e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.937+0000 7f0a72c00700 1 -- 172.21.15.47:0/3798415320 >> 172.21.15.47:0/3798415320 conn(0x7f0a6c0792f0 msgr2=0x7f0a6c079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.938+0000 7f0a72c00700 1 -- 172.21.15.47:0/3798415320 shutdown_connections 2024-03-21T11:22:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.938+0000 7f0a72c00700 1 -- 172.21.15.47:0/3798415320 wait complete. 2024-03-21T11:22:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.939+0000 7f0a72c00700 1 Processor -- start 2024-03-21T11:22:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.939+0000 7f0a72c00700 1 -- start start 2024-03-21T11:22:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.940+0000 7f0a72c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a6c0743c0 0x7f0a6c1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.940+0000 7f0a72c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a6c074ce0 0x7f0a6c1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.940+0000 7f0a72c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a6c10eb90 0x7f0a6c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.940+0000 7f0a72c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c114770 con 0x7f0a6c074ce0 2024-03-21T11:22:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.940+0000 7f0a72c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1145f0 con 0x7f0a6c0743c0 2024-03-21T11:22:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.940+0000 7f0a72c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1148f0 con 0x7f0a6c10eb90 2024-03-21T11:22:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.940+0000 7f0a6be00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a6c0743c0 0x7f0a6c1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.940+0000 7f0a70c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a6c10eb90 0x7f0a6c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.940+0000 7f0a6be00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a6c0743c0 0x7f0a6c1a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38146/0 (socket says 172.21.15.47:38146) 2024-03-21T11:22:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.940+0000 7f0a70c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a6c10eb90 0x7f0a6c19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54874/0 (socket says 172.21.15.47:54874) 2024-03-21T11:22:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.940+0000 7f0a6be00700 1 -- 172.21.15.47:0/1453264362 learned_addr learned my addr 172.21.15.47:0/1453264362 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.940+0000 7f0a6b400700 1 --2- 172.21.15.47:0/1453264362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a6c074ce0 0x7f0a6c1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.940+0000 7f0a70c00700 1 -- 172.21.15.47:0/1453264362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a6c0743c0 msgr2=0x7f0a6c1a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.940+0000 7f0a70c00700 1 --2- 172.21.15.47:0/1453264362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a6c0743c0 0x7f0a6c1a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.941+0000 7f0a70c00700 1 -- 172.21.15.47:0/1453264362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a6c074ce0 msgr2=0x7f0a6c1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.941+0000 7f0a70c00700 1 --2- 172.21.15.47:0/1453264362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a6c074ce0 0x7f0a6c1a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.941+0000 7f0a70c00700 1 -- 172.21.15.47:0/1453264362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c19b190 con 0x7f0a6c10eb90 2024-03-21T11:22:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.941+0000 7f0a6be00700 1 --2- 172.21.15.47:0/1453264362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a6c0743c0 0x7f0a6c1a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.941+0000 7f0a70c00700 1 --2- 172.21.15.47:0/1453264362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a6c10eb90 0x7f0a6c19aa30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400b070 tx=0x7f0a64004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.941+0000 7f0a68c00700 1 -- 172.21.15.47:0/1453264362 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a64021070 con 0x7f0a6c10eb90 2024-03-21T11:22:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.941+0000 7f0a68c00700 1 -- 172.21.15.47:0/1453264362 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a64007b80 con 0x7f0a6c10eb90 2024-03-21T11:22:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.941+0000 7f0a68c00700 1 -- 172.21.15.47:0/1453264362 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a64005040 con 0x7f0a6c10eb90 2024-03-21T11:22:28.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.941+0000 7f0a72c00700 1 -- 172.21.15.47:0/1453264362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6c19b420 con 0x7f0a6c10eb90 2024-03-21T11:22:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.942+0000 7f0a72c00700 1 -- 172.21.15.47:0/1453264362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6c1a76f0 con 0x7f0a6c10eb90 2024-03-21T11:22:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.943+0000 7f0a72c00700 1 -- 172.21.15.47:0/1453264362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6c068e50 con 0x7f0a6c10eb90 2024-03-21T11:22:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.946+0000 7f0a68c00700 1 -- 172.21.15.47:0/1453264362 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f0a64024070 con 0x7f0a6c10eb90 2024-03-21T11:22:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.947+0000 7f0a68c00700 1 --2- 172.21.15.47:0/1453264362 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a500a3040 0x7f0a500a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.948+0000 7f0a68c00700 1 -- 172.21.15.47:0/1453264362 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f0a640d0720 con 0x7f0a6c10eb90 2024-03-21T11:22:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.948+0000 7f0a6be00700 1 --2- 172.21.15.47:0/1453264362 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a500a3040 0x7f0a500a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.948+0000 7f0a6be00700 1 --2- 172.21.15.47:0/1453264362 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a500a3040 0x7f0a500a5500 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c101ec0 tx=0x7f0a5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:27.951+0000 7f0a68c00700 1 -- 172.21.15.47:0/1453264362 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0a64007d30 con 0x7f0a6c10eb90 2024-03-21T11:22:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.188+0000 7f0a72c00700 1 -- 172.21.15.47:0/1453264362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0a6c02da30 con 0x7f0a6c10eb90 2024-03-21T11:22:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.188+0000 7f0a68c00700 1 -- 172.21.15.47:0/1453264362 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a6409d0c0 con 0x7f0a6c10eb90 2024-03-21T11:22:28.787 INFO:teuthology.orchestra.run.smithi047.stdout:201863462975 2024-03-21T11:22:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.191+0000 7f0a72c00700 1 -- 172.21.15.47:0/1453264362 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a500a3040 msgr2=0x7f0a500a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.191+0000 7f0a72c00700 1 --2- 172.21.15.47:0/1453264362 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a500a3040 0x7f0a500a5500 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c101ec0 tx=0x7f0a5c009260 comp rx=0 tx=0).stop 2024-03-21T11:22:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.191+0000 7f0a72c00700 1 -- 172.21.15.47:0/1453264362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a6c10eb90 msgr2=0x7f0a6c19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.191+0000 7f0a72c00700 1 --2- 172.21.15.47:0/1453264362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a6c10eb90 0x7f0a6c19aa30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400b070 tx=0x7f0a64004c80 comp rx=0 tx=0).stop 2024-03-21T11:22:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.192+0000 7f0a72c00700 1 -- 172.21.15.47:0/1453264362 shutdown_connections 2024-03-21T11:22:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.192+0000 7f0a72c00700 1 --2- 172.21.15.47:0/1453264362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a6c10eb90 0x7f0a6c19aa30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.192+0000 7f0a72c00700 1 --2- 172.21.15.47:0/1453264362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a6c0743c0 0x7f0a6c1a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.192+0000 7f0a72c00700 1 --2- 172.21.15.47:0/1453264362 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a500a3040 0x7f0a500a5500 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.192+0000 7f0a72c00700 1 --2- 172.21.15.47:0/1453264362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a6c074ce0 0x7f0a6c1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:28.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.192+0000 7f0a72c00700 1 -- 172.21.15.47:0/1453264362 >> 172.21.15.47:0/1453264362 conn(0x7f0a6c0792f0 msgr2=0x7f0a6c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:28.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.192+0000 7f0a72c00700 1 -- 172.21.15.47:0/1453264362 shutdown_connections 2024-03-21T11:22:28.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:28.192+0000 7f0a72c00700 1 -- 172.21.15.47:0/1453264362 wait complete. 2024-03-21T11:22:29.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:29 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1453264362' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T11:22:29.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:29 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2368017210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:29 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1453264362' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:29 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2368017210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T11:22:29.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:29 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1453264362' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T11:22:29.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:29 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2368017210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T11:22:29.539 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462968 got 201863462975 for osd.6 2024-03-21T11:22:29.539 DEBUG:teuthology.parallel:result is None 2024-03-21T11:22:29.896 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738455 got 34359738463 for osd.0 2024-03-21T11:22:29.896 DEBUG:teuthology.parallel:result is None 2024-03-21T11:22:30.021 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:30.162 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037805 got 257698037811 for osd.8 2024-03-21T11:22:30.162 DEBUG:teuthology.parallel:result is None 2024-03-21T11:22:30.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:30 smithi100 ceph-mon[36245]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:30.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:30 smithi102 ceph-mon[36748]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:30.495 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:30 smithi047 ceph-mon[31351]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:31.820 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351005 got 347892351010 for osd.11 2024-03-21T11:22:31.821 DEBUG:teuthology.parallel:result is None 2024-03-21T11:22:32.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.051+0000 7fb27e800700 1 -- 172.21.15.47:0/3933463442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2781096c0 msgr2=0x7fb278111290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:32.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.051+0000 7fb27e800700 1 --2- 172.21.15.47:0/3933463442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2781096c0 0x7fb278111290 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb27000b0d0 tx=0x7fb27000b3e0 comp rx=0 tx=0).stop 2024-03-21T11:22:32.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.051+0000 7fb27e800700 1 -- 172.21.15.47:0/3933463442 shutdown_connections 2024-03-21T11:22:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.051+0000 7fb27e800700 1 --2- 172.21.15.47:0/3933463442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2781096c0 0x7fb278111290 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.051+0000 7fb27e800700 1 --2- 172.21.15.47:0/3933463442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2781045e0 0x7fb2781049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.051+0000 7fb27e800700 1 --2- 172.21.15.47:0/3933463442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb278104f90 0x7fb278108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.051+0000 7fb27e800700 1 -- 172.21.15.47:0/3933463442 >> 172.21.15.47:0/3933463442 conn(0x7fb2780ffe70 msgr2=0x7fb278102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.051+0000 7fb27e800700 1 -- 172.21.15.47:0/3933463442 shutdown_connections 2024-03-21T11:22:32.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.051+0000 7fb27e800700 1 -- 172.21.15.47:0/3933463442 wait complete. 2024-03-21T11:22:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.052+0000 7fb27e800700 1 Processor -- start 2024-03-21T11:22:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.053+0000 7fb27e800700 1 -- start start 2024-03-21T11:22:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.053+0000 7fb27e800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2781045e0 0x7fb2781a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.053+0000 7fb27e800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb278104f90 0x7fb2781a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.053+0000 7fb27e800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2781096c0 0x7fb27819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.053+0000 7fb27e800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb278114700 con 0x7fb2781096c0 2024-03-21T11:22:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.053+0000 7fb27e800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb278114580 con 0x7fb2781045e0 2024-03-21T11:22:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.053+0000 7fb27e800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb278114880 con 0x7fb278104f90 2024-03-21T11:22:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.053+0000 7fb26ea00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb278104f90 0x7fb2781a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.053+0000 7fb277400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2781045e0 0x7fb2781a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.053+0000 7fb26ea00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb278104f90 0x7fb2781a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46116/0 (socket says 172.21.15.47:46116) 2024-03-21T11:22:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.053+0000 7fb26ea00700 1 -- 172.21.15.47:0/2363766520 learned_addr learned my addr 172.21.15.47:0/2363766520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:32.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.053+0000 7fb277400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2781045e0 0x7fb2781a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43624/0 (socket says 172.21.15.47:43624) 2024-03-21T11:22:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.054+0000 7fb277e00700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2781096c0 0x7fb27819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.054+0000 7fb26ea00700 1 -- 172.21.15.47:0/2363766520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2781045e0 msgr2=0x7fb2781a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.054+0000 7fb26ea00700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2781045e0 0x7fb2781a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.054+0000 7fb26ea00700 1 -- 172.21.15.47:0/2363766520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2781096c0 msgr2=0x7fb27819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.054+0000 7fb26ea00700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2781096c0 0x7fb27819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.054+0000 7fb26ea00700 1 -- 172.21.15.47:0/2363766520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27819b020 con 0x7fb278104f90 2024-03-21T11:22:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.054+0000 7fb277400700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2781045e0 0x7fb2781a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.054+0000 7fb277e00700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2781096c0 0x7fb27819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:22:32.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.054+0000 7fb26ea00700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb278104f90 0x7fb2781a0c40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb26000e970 tx=0x7fb26000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:32.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.055+0000 7fb274c00700 1 -- 172.21.15.47:0/2363766520 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26000cbd0 con 0x7fb278104f90 2024-03-21T11:22:32.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.055+0000 7fb274c00700 1 -- 172.21.15.47:0/2363766520 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb26000cd70 con 0x7fb278104f90 2024-03-21T11:22:32.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.055+0000 7fb27e800700 1 -- 172.21.15.47:0/2363766520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27819b310 con 0x7fb278104f90 2024-03-21T11:22:32.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.055+0000 7fb274c00700 1 -- 172.21.15.47:0/2363766520 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2600056b0 con 0x7fb278104f90 2024-03-21T11:22:32.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.055+0000 7fb27e800700 1 -- 172.21.15.47:0/2363766520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2781a75d0 con 0x7fb278104f90 2024-03-21T11:22:32.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.056+0000 7fb27e800700 1 -- 172.21.15.47:0/2363766520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb278068e50 con 0x7fb278104f90 2024-03-21T11:22:32.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.058+0000 7fb274c00700 1 -- 172.21.15.47:0/2363766520 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fb260004750 con 0x7fb278104f90 2024-03-21T11:22:32.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.060+0000 7fb274c00700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb25c0a30c0 0x7fb25c0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:32.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.060+0000 7fb274c00700 1 -- 172.21.15.47:0/2363766520 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fb2600d0060 con 0x7fb278104f90 2024-03-21T11:22:32.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.060+0000 7fb277400700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb25c0a30c0 0x7fb25c0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:32.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.061+0000 7fb277400700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb25c0a30c0 0x7fb25c0a5580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb27819ba00 tx=0x7fb268009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:32.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.062+0000 7fb274c00700 1 -- 172.21.15.47:0/2363766520 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb26009c970 con 0x7fb278104f90 2024-03-21T11:22:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.301+0000 7fb27e800700 1 -- 172.21.15.47:0/2363766520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fb27802da30 con 0x7fb278104f90 2024-03-21T11:22:32.302 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:32 smithi047 ceph-mon[31351]: pgmap v374: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:22:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.301+0000 7fb274c00700 1 -- 172.21.15.47:0/2363766520 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb26009c270 con 0x7fb278104f90 2024-03-21T11:22:32.303 INFO:teuthology.orchestra.run.smithi047.stdout:292057776173 2024-03-21T11:22:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.304+0000 7fb27e800700 1 -- 172.21.15.47:0/2363766520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb25c0a30c0 msgr2=0x7fb25c0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.304+0000 7fb27e800700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb25c0a30c0 0x7fb25c0a5580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb27819ba00 tx=0x7fb268009260 comp rx=0 tx=0).stop 2024-03-21T11:22:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.304+0000 7fb27e800700 1 -- 172.21.15.47:0/2363766520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb278104f90 msgr2=0x7fb2781a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.304+0000 7fb27e800700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb278104f90 0x7fb2781a0c40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb26000e970 tx=0x7fb26000ec80 comp rx=0 tx=0).stop 2024-03-21T11:22:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.304+0000 7fb27e800700 1 -- 172.21.15.47:0/2363766520 shutdown_connections 2024-03-21T11:22:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.304+0000 7fb27e800700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb25c0a30c0 0x7fb25c0a5580 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.304+0000 7fb27e800700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2781096c0 0x7fb27819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.304+0000 7fb27e800700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2781045e0 0x7fb2781a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.304+0000 7fb27e800700 1 --2- 172.21.15.47:0/2363766520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb278104f90 0x7fb2781a0c40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.304+0000 7fb27e800700 1 -- 172.21.15.47:0/2363766520 >> 172.21.15.47:0/2363766520 conn(0x7fb2780ffe70 msgr2=0x7fb278100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.305+0000 7fb27e800700 1 -- 172.21.15.47:0/2363766520 shutdown_connections 2024-03-21T11:22:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:32.305+0000 7fb27e800700 1 -- 172.21.15.47:0/2363766520 wait complete. 2024-03-21T11:22:32.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:32 smithi102 ceph-mon[36748]: pgmap v374: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:22:32.614 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:32.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:32 smithi100 ceph-mon[36245]: pgmap v374: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:22:32.653 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724605 got 167503724612 for osd.5 2024-03-21T11:22:32.653 DEBUG:teuthology.parallel:result is None 2024-03-21T11:22:33.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:33 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2363766520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T11:22:33.471 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776167 got 292057776173 for osd.9 2024-03-21T11:22:33.472 DEBUG:teuthology.parallel:result is None 2024-03-21T11:22:33.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:33 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2363766520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T11:22:33.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:33 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2363766520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T11:22:34.980 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:34 smithi047 ceph-mon[31351]: pgmap v375: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:22:34.980 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:34 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:22:34.980 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:34 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:22:35.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:34 smithi100 ceph-mon[36245]: pgmap v375: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:22:35.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:34 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:22:35.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:34 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:22:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:34 smithi102 ceph-mon[36748]: pgmap v375: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:22:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:34 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:22:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:34 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:22:35.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:22:35.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:22:35.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:22:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.883+0000 7f253d200700 1 -- 172.21.15.47:0/3084403993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2538074ce0 msgr2=0x7f253810e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.883+0000 7f253d200700 1 --2- 172.21.15.47:0/3084403993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2538074ce0 0x7f253810e650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2520009990 tx=0x7f2520009ca0 comp rx=0 tx=0).stop 2024-03-21T11:22:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.884+0000 7f253d200700 1 -- 172.21.15.47:0/3084403993 shutdown_connections 2024-03-21T11:22:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.884+0000 7f253d200700 1 --2- 172.21.15.47:0/3084403993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f253810eb90 0x7f2538110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.884+0000 7f253d200700 1 --2- 172.21.15.47:0/3084403993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25380743c0 0x7f25380747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.884+0000 7f253d200700 1 --2- 172.21.15.47:0/3084403993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2538074ce0 0x7f253810e650 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.884+0000 7f253d200700 1 -- 172.21.15.47:0/3084403993 >> 172.21.15.47:0/3084403993 conn(0x7f25380792c0 msgr2=0x7f25380796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.884+0000 7f253d200700 1 -- 172.21.15.47:0/3084403993 shutdown_connections 2024-03-21T11:22:35.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.884+0000 7f253d200700 1 -- 172.21.15.47:0/3084403993 wait complete. 2024-03-21T11:22:35.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.885+0000 7f253d200700 1 Processor -- start 2024-03-21T11:22:35.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.885+0000 7f253d200700 1 -- start start 2024-03-21T11:22:35.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.885+0000 7f253d200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25380743c0 0x7f25381a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:35.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.885+0000 7f253d200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2538074ce0 0x7f25381a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:35.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.885+0000 7f253d200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f253810eb90 0x7f253819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:35.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.885+0000 7f253d200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25381146b0 con 0x7f2538074ce0 2024-03-21T11:22:35.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.885+0000 7f253d200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2538114530 con 0x7f25380743c0 2024-03-21T11:22:35.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.885+0000 7f253d200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2538114830 con 0x7f253810eb90 2024-03-21T11:22:35.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.886+0000 7f2535e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25380743c0 0x7f25381a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:35.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.886+0000 7f252d400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2538074ce0 0x7f25381a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:35.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.886+0000 7f2535e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25380743c0 0x7f25381a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43632/0 (socket says 172.21.15.47:43632) 2024-03-21T11:22:35.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.886+0000 7f2535e00700 1 -- 172.21.15.47:0/3909981150 learned_addr learned my addr 172.21.15.47:0/3909981150 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:35.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.886+0000 7f2536800700 1 --2- 172.21.15.47:0/3909981150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f253810eb90 0x7f253819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:35.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.886+0000 7f2535e00700 1 -- 172.21.15.47:0/3909981150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f253810eb90 msgr2=0x7f253819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:35.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.886+0000 7f2535e00700 1 --2- 172.21.15.47:0/3909981150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f253810eb90 0x7f253819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:35.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.886+0000 7f2535e00700 1 -- 172.21.15.47:0/3909981150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2538074ce0 msgr2=0x7f25381a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:35.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.886+0000 7f2535e00700 1 --2- 172.21.15.47:0/3909981150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2538074ce0 0x7f25381a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:35.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.886+0000 7f2535e00700 1 -- 172.21.15.47:0/3909981150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253819b1b0 con 0x7f25380743c0 2024-03-21T11:22:35.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.886+0000 7f252d400700 1 --2- 172.21.15.47:0/3909981150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2538074ce0 0x7f25381a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:35.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.886+0000 7f2535e00700 1 --2- 172.21.15.47:0/3909981150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25380743c0 0x7f25381a0840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f252800e9c0 tx=0x7f252800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:35.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.887+0000 7f252f400700 1 -- 172.21.15.47:0/3909981150 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252800cbd0 con 0x7f25380743c0 2024-03-21T11:22:35.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.887+0000 7f253d200700 1 -- 172.21.15.47:0/3909981150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253819b4a0 con 0x7f25380743c0 2024-03-21T11:22:35.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.887+0000 7f252f400700 1 -- 172.21.15.47:0/3909981150 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f252800ee60 con 0x7f25380743c0 2024-03-21T11:22:35.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.887+0000 7f252f400700 1 -- 172.21.15.47:0/3909981150 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528005630 con 0x7f25380743c0 2024-03-21T11:22:35.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.887+0000 7f253d200700 1 -- 172.21.15.47:0/3909981150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25381a7710 con 0x7f25380743c0 2024-03-21T11:22:35.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.888+0000 7f253d200700 1 -- 172.21.15.47:0/3909981150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2538068e50 con 0x7f25380743c0 2024-03-21T11:22:35.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.890+0000 7f252f400700 1 -- 172.21.15.47:0/3909981150 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f2528004750 con 0x7f25380743c0 2024-03-21T11:22:35.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.891+0000 7f252f400700 1 --2- 172.21.15.47:0/3909981150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f251c0a3040 0x7f251c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:35.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.891+0000 7f252f400700 1 -- 172.21.15.47:0/3909981150 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f25280cfb90 con 0x7f25380743c0 2024-03-21T11:22:35.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.891+0000 7f252d400700 1 --2- 172.21.15.47:0/3909981150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f251c0a3040 0x7f251c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:35.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.892+0000 7f252d400700 1 --2- 172.21.15.47:0/3909981150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f251c0a3040 0x7f251c0a5500 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2538110910 tx=0x7f2520005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:35.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:35.897+0000 7f252f400700 1 -- 172.21.15.47:0/3909981150 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f252809c4a0 con 0x7f25380743c0 2024-03-21T11:22:36.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.073+0000 7fc8bec00700 1 -- 172.21.15.47:0/331941444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8b806a930 msgr2=0x7fc8b806ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:36.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.073+0000 7fc8bec00700 1 --2- 172.21.15.47:0/331941444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8b806a930 0x7fc8b806ad10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009990 tx=0x7fc8a8009ca0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.073+0000 7fc8bec00700 1 -- 172.21.15.47:0/331941444 shutdown_connections 2024-03-21T11:22:36.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.073+0000 7fc8bec00700 1 --2- 172.21.15.47:0/331941444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8b8107510 0x7fc8b810f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.073+0000 7fc8bec00700 1 --2- 172.21.15.47:0/331941444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8b806a930 0x7fc8b806ad10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.073+0000 7fc8bec00700 1 --2- 172.21.15.47:0/331941444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8b806b2e0 0x7fc8b8106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.073+0000 7fc8bec00700 1 -- 172.21.15.47:0/331941444 >> 172.21.15.47:0/331941444 conn(0x7fc8b80770e0 msgr2=0x7fc8b80774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:36.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.073+0000 7fc8bec00700 1 -- 172.21.15.47:0/331941444 shutdown_connections 2024-03-21T11:22:36.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.073+0000 7fc8bec00700 1 -- 172.21.15.47:0/331941444 wait complete. 2024-03-21T11:22:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.074+0000 7fc8bec00700 1 Processor -- start 2024-03-21T11:22:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.074+0000 7fc8bec00700 1 -- start start 2024-03-21T11:22:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.075+0000 7fc8bec00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8b806a930 0x7fc8b8101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.075+0000 7fc8bec00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8b806b2e0 0x7fc8b81016d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.075+0000 7fc8bec00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8b8107510 0x7fc8b8101c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.075+0000 7fc8bec00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b81124f0 con 0x7fc8b806b2e0 2024-03-21T11:22:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.075+0000 7fc8bec00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b8112370 con 0x7fc8b806a930 2024-03-21T11:22:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.075+0000 7fc8bec00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b8112670 con 0x7fc8b8107510 2024-03-21T11:22:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.075+0000 7fc8bcc00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8b8107510 0x7fc8b8101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.075+0000 7fc8bcc00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8b8107510 0x7fc8b8101c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46140/0 (socket says 172.21.15.47:46140) 2024-03-21T11:22:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.075+0000 7fc8bcc00700 1 -- 172.21.15.47:0/2121497501 learned_addr learned my addr 172.21.15.47:0/2121497501 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.075+0000 7fc8b7e00700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8b806a930 0x7fc8b8101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.076+0000 7fc8b7400700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8b806b2e0 0x7fc8b81016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.076+0000 7fc8bcc00700 1 -- 172.21.15.47:0/2121497501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8b806a930 msgr2=0x7fc8b8101190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.076+0000 7fc8bcc00700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8b806a930 0x7fc8b8101190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.076+0000 7fc8bcc00700 1 -- 172.21.15.47:0/2121497501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8b806b2e0 msgr2=0x7fc8b81016d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.076+0000 7fc8bcc00700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8b806b2e0 0x7fc8b81016d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.076+0000 7fc8bcc00700 1 -- 172.21.15.47:0/2121497501 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b81a6ef0 con 0x7fc8b8107510 2024-03-21T11:22:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.076+0000 7fc8b7400700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8b806b2e0 0x7fc8b81016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:22:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.076+0000 7fc8bcc00700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8b8107510 0x7fc8b8101c10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000ef30 tx=0x7fc8b000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.076+0000 7fc8b4c00700 1 -- 172.21.15.47:0/2121497501 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00092e0 con 0x7fc8b8107510 2024-03-21T11:22:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.076+0000 7fc8b4c00700 1 -- 172.21.15.47:0/2121497501 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8b0004bb0 con 0x7fc8b8107510 2024-03-21T11:22:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.076+0000 7fc8bec00700 1 -- 172.21.15.47:0/2121497501 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b81a7180 con 0x7fc8b8107510 2024-03-21T11:22:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.076+0000 7fc8bec00700 1 -- 172.21.15.47:0/2121497501 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b81a7670 con 0x7fc8b8107510 2024-03-21T11:22:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.077+0000 7fc8b4c00700 1 -- 172.21.15.47:0/2121497501 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0004280 con 0x7fc8b8107510 2024-03-21T11:22:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.077+0000 7fc8bec00700 1 -- 172.21.15.47:0/2121497501 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc898005330 con 0x7fc8b8107510 2024-03-21T11:22:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.079+0000 7fc8b4c00700 1 -- 172.21.15.47:0/2121497501 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fc8b001b030 con 0x7fc8b8107510 2024-03-21T11:22:36.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.080+0000 7fc8b4c00700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc89c0a3040 0x7fc89c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:36.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.081+0000 7fc8b4c00700 1 -- 172.21.15.47:0/2121497501 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fc8b00cf2e0 con 0x7fc8b8107510 2024-03-21T11:22:36.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.081+0000 7fc8b7e00700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc89c0a3040 0x7fc89c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:36.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.081+0000 7fc8b7e00700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc89c0a3040 0x7fc89c0a5500 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800b2e0 tx=0x7fc8a8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:36.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.084+0000 7fc8b4c00700 1 -- 172.21.15.47:0/2121497501 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc8b0022020 con 0x7fc8b8107510 2024-03-21T11:22:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.131+0000 7f253d200700 1 -- 172.21.15.47:0/3909981150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f253819bf30 con 0x7f25380743c0 2024-03-21T11:22:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.131+0000 7f252f400700 1 -- 172.21.15.47:0/3909981150 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f252809bda0 con 0x7f25380743c0 2024-03-21T11:22:36.132 INFO:teuthology.orchestra.run.smithi047.stdout:111669149776 2024-03-21T11:22:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.134+0000 7f253d200700 1 -- 172.21.15.47:0/3909981150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f251c0a3040 msgr2=0x7f251c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.134+0000 7f253d200700 1 --2- 172.21.15.47:0/3909981150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f251c0a3040 0x7f251c0a5500 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2538110910 tx=0x7f2520005fd0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.134+0000 7f253d200700 1 -- 172.21.15.47:0/3909981150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25380743c0 msgr2=0x7f25381a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.134+0000 7f253d200700 1 --2- 172.21.15.47:0/3909981150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25380743c0 0x7f25381a0840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f252800e9c0 tx=0x7f252800ecd0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.134+0000 7f253d200700 1 -- 172.21.15.47:0/3909981150 shutdown_connections 2024-03-21T11:22:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.134+0000 7f253d200700 1 --2- 172.21.15.47:0/3909981150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f253810eb90 0x7f253819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.134+0000 7f253d200700 1 --2- 172.21.15.47:0/3909981150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25380743c0 0x7f25381a0840 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.134+0000 7f253d200700 1 --2- 172.21.15.47:0/3909981150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f251c0a3040 0x7f251c0a5500 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.134+0000 7f253d200700 1 --2- 172.21.15.47:0/3909981150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2538074ce0 0x7f25381a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.134+0000 7f253d200700 1 -- 172.21.15.47:0/3909981150 >> 172.21.15.47:0/3909981150 conn(0x7f25380792c0 msgr2=0x7f2538100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.134+0000 7f253d200700 1 -- 172.21.15.47:0/3909981150 shutdown_connections 2024-03-21T11:22:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.134+0000 7f253d200700 1 -- 172.21.15.47:0/3909981150 wait complete. 2024-03-21T11:22:36.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:35 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:22:36.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:35 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:22:36.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:35 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:22:36.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:35 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:22:36.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:35 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:22:36.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:35 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:22:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.309+0000 7fc8bec00700 1 -- 172.21.15.47:0/2121497501 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc8980058f0 con 0x7fc8b8107510 2024-03-21T11:22:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.310+0000 7fc8b4c00700 1 -- 172.21.15.47:0/2121497501 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc8b0010040 con 0x7fc8b8107510 2024-03-21T11:22:36.311 INFO:teuthology.orchestra.run.smithi047.stdout:77309411415 2024-03-21T11:22:36.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.313+0000 7fc8bec00700 1 -- 172.21.15.47:0/2121497501 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc89c0a3040 msgr2=0x7fc89c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.313+0000 7fc8bec00700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc89c0a3040 0x7fc89c0a5500 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800b2e0 tx=0x7fc8a8005fd0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.313+0000 7fc8bec00700 1 -- 172.21.15.47:0/2121497501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8b8107510 msgr2=0x7fc8b8101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.313+0000 7fc8bec00700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8b8107510 0x7fc8b8101c10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000ef30 tx=0x7fc8b000c3a0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.313+0000 7fc8bec00700 1 -- 172.21.15.47:0/2121497501 shutdown_connections 2024-03-21T11:22:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.313+0000 7fc8bec00700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8b8107510 0x7fc8b8101c10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.313+0000 7fc8bec00700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8b806a930 0x7fc8b8101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.313+0000 7fc8bec00700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc89c0a3040 0x7fc89c0a5500 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.313+0000 7fc8bec00700 1 --2- 172.21.15.47:0/2121497501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8b806b2e0 0x7fc8b81016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.313+0000 7fc8bec00700 1 -- 172.21.15.47:0/2121497501 >> 172.21.15.47:0/2121497501 conn(0x7fc8b80770e0 msgr2=0x7fc8b80ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.313+0000 7fc8bec00700 1 -- 172.21.15.47:0/2121497501 shutdown_connections 2024-03-21T11:22:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:36.313+0000 7fc8bec00700 1 -- 172.21.15.47:0/2121497501 wait complete. 2024-03-21T11:22:37.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:36 smithi047 ceph-mon[31351]: pgmap v376: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:22:37.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:36 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3909981150' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T11:22:37.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:36 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2121497501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T11:22:37.133 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411409 got 77309411415 for osd.2 2024-03-21T11:22:37.133 DEBUG:teuthology.parallel:result is None 2024-03-21T11:22:37.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:36 smithi100 ceph-mon[36245]: pgmap v376: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:22:37.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:36 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3909981150' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T11:22:37.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:36 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2121497501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T11:22:37.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:36 smithi102 ceph-mon[36748]: pgmap v376: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:22:37.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:36 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3909981150' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T11:22:37.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:36 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2121497501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T11:22:37.756 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149770 got 111669149776 for osd.3 2024-03-21T11:22:37.756 DEBUG:teuthology.parallel:result is None 2024-03-21T11:22:37.756 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-21T11:22:37.756 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T11:22:38.238 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:38 smithi047 ceph-mon[31351]: pgmap v377: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:22:39.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:38 smithi100 ceph-mon[36245]: pgmap v377: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:22:39.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:38 smithi102 ceph-mon[36748]: pgmap v377: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.317+0000 7f6aca200700 1 -- 172.21.15.47:0/2174918370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ac4069e40 msgr2=0x7f6ac406a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:40.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.317+0000 7f6aca200700 1 --2- 172.21.15.47:0/2174918370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ac4069e40 0x7f6ac406a2a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0099c0 tx=0x7f6aac009cd0 comp rx=0 tx=0).stop 2024-03-21T11:22:40.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.317+0000 7f6aca200700 1 -- 172.21.15.47:0/2174918370 shutdown_connections 2024-03-21T11:22:40.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.317+0000 7f6aca200700 1 --2- 172.21.15.47:0/2174918370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ac406a7e0 0x7f6ac410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:40.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.317+0000 7f6aca200700 1 --2- 172.21.15.47:0/2174918370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ac41081e0 0x7f6ac41085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:40.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.317+0000 7f6aca200700 1 --2- 172.21.15.47:0/2174918370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ac4069e40 0x7f6ac406a2a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:40.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.317+0000 7f6aca200700 1 -- 172.21.15.47:0/2174918370 >> 172.21.15.47:0/2174918370 conn(0x7f6ac4077fc0 msgr2=0x7f6ac40783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:40.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.318+0000 7f6aca200700 1 -- 172.21.15.47:0/2174918370 shutdown_connections 2024-03-21T11:22:40.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.318+0000 7f6aca200700 1 -- 172.21.15.47:0/2174918370 wait complete. 2024-03-21T11:22:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.319+0000 7f6aca200700 1 Processor -- start 2024-03-21T11:22:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.319+0000 7f6aca200700 1 -- start start 2024-03-21T11:22:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.319+0000 7f6aca200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ac4069e40 0x7f6ac419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.319+0000 7f6aca200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ac406a7e0 0x7f6ac419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.319+0000 7f6aca200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ac41081e0 0x7f6ac41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.319+0000 7f6aca200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac4112440 con 0x7f6ac41081e0 2024-03-21T11:22:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.319+0000 7f6aca200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac41122c0 con 0x7f6ac4069e40 2024-03-21T11:22:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.319+0000 7f6aca200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac41125c0 con 0x7f6ac406a7e0 2024-03-21T11:22:40.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.319+0000 7f6ac3400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ac4069e40 0x7f6ac419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:40.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.319+0000 7f6abaa00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ac406a7e0 0x7f6ac419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:40.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.320+0000 7f6abaa00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ac406a7e0 0x7f6ac419eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37834/0 (socket says 172.21.15.47:37834) 2024-03-21T11:22:40.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.320+0000 7f6abaa00700 1 -- 172.21.15.47:0/286199396 learned_addr learned my addr 172.21.15.47:0/286199396 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:40.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.320+0000 7f6abaa00700 1 -- 172.21.15.47:0/286199396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ac4069e40 msgr2=0x7f6ac419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:40.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.320+0000 7f6abaa00700 1 --2- 172.21.15.47:0/286199396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ac4069e40 0x7f6ac419e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:40.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.320+0000 7f6abaa00700 1 -- 172.21.15.47:0/286199396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ac41081e0 msgr2=0x7f6ac41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:40.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.320+0000 7f6abaa00700 1 --2- 172.21.15.47:0/286199396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ac41081e0 0x7f6ac41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:40.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.320+0000 7f6abaa00700 1 -- 172.21.15.47:0/286199396 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac4198fa0 con 0x7f6ac406a7e0 2024-03-21T11:22:40.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.320+0000 7f6abaa00700 1 --2- 172.21.15.47:0/286199396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ac406a7e0 0x7f6ac419eb30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6aac006010 tx=0x7f6aac003b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:40.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.321+0000 7f6ac0c00700 1 -- 172.21.15.47:0/286199396 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac003680 con 0x7f6ac406a7e0 2024-03-21T11:22:40.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.321+0000 7f6ac0c00700 1 -- 172.21.15.47:0/286199396 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6aac0055a0 con 0x7f6ac406a7e0 2024-03-21T11:22:40.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.321+0000 7f6aca200700 1 -- 172.21.15.47:0/286199396 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac4199230 con 0x7f6ac406a7e0 2024-03-21T11:22:40.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.321+0000 7f6ac0c00700 1 -- 172.21.15.47:0/286199396 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac0173f0 con 0x7f6ac406a7e0 2024-03-21T11:22:40.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.321+0000 7f6aca200700 1 -- 172.21.15.47:0/286199396 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac41a53a0 con 0x7f6ac406a7e0 2024-03-21T11:22:40.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.322+0000 7f6aca200700 1 -- 172.21.15.47:0/286199396 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac4103890 con 0x7f6ac406a7e0 2024-03-21T11:22:40.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.324+0000 7f6ac0c00700 1 -- 172.21.15.47:0/286199396 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f6aac00f680 con 0x7f6ac406a7e0 2024-03-21T11:22:40.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.326+0000 7f6ac0c00700 1 --2- 172.21.15.47:0/286199396 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6aa80a31e0 0x7f6aa80a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:40.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.326+0000 7f6ac0c00700 1 -- 172.21.15.47:0/286199396 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f6aac0d0760 con 0x7f6ac406a7e0 2024-03-21T11:22:40.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.326+0000 7f6ac3400700 1 --2- 172.21.15.47:0/286199396 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6aa80a31e0 0x7f6aa80a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:40.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.327+0000 7f6ac3400700 1 --2- 172.21.15.47:0/286199396 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6aa80a31e0 0x7f6aa80a56a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009bd0 tx=0x7f6ab4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.333+0000 7f6ac0c00700 1 -- 172.21.15.47:0/286199396 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6aac013070 con 0x7f6ac406a7e0 2024-03-21T11:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.553+0000 7f6aca200700 1 -- 172.21.15.47:0/286199396 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ac40625d0 con 0x7f6aa80a31e0 2024-03-21T11:22:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.568+0000 7f6ac0c00700 1 -- 172.21.15.47:0/286199396 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340720 (secure 0 0 0) 0x7f6ac40625d0 con 0x7f6aa80a31e0 2024-03-21T11:22:40.572 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:22:40.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.575+0000 7f6aca200700 1 -- 172.21.15.47:0/286199396 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6aa80a31e0 msgr2=0x7f6aa80a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:40.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.575+0000 7f6aca200700 1 --2- 172.21.15.47:0/286199396 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6aa80a31e0 0x7f6aa80a56a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009bd0 tx=0x7f6ab4009260 comp rx=0 tx=0).stop 2024-03-21T11:22:40.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.575+0000 7f6aca200700 1 -- 172.21.15.47:0/286199396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ac406a7e0 msgr2=0x7f6ac419eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:40.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.575+0000 7f6aca200700 1 --2- 172.21.15.47:0/286199396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ac406a7e0 0x7f6ac419eb30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6aac006010 tx=0x7f6aac003b40 comp rx=0 tx=0).stop 2024-03-21T11:22:40.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.575+0000 7f6aca200700 1 -- 172.21.15.47:0/286199396 shutdown_connections 2024-03-21T11:22:40.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.575+0000 7f6aca200700 1 --2- 172.21.15.47:0/286199396 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6aa80a31e0 0x7f6aa80a56a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:40.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.575+0000 7f6aca200700 1 --2- 172.21.15.47:0/286199396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ac41081e0 0x7f6ac41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:40.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.575+0000 7f6aca200700 1 --2- 172.21.15.47:0/286199396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ac4069e40 0x7f6ac419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:40.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.575+0000 7f6aca200700 1 --2- 172.21.15.47:0/286199396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ac406a7e0 0x7f6ac419eb30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:40.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.575+0000 7f6aca200700 1 -- 172.21.15.47:0/286199396 >> 172.21.15.47:0/286199396 conn(0x7f6ac4077fc0 msgr2=0x7f6ac40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:40.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.576+0000 7f6aca200700 1 -- 172.21.15.47:0/286199396 shutdown_connections 2024-03-21T11:22:40.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:40.576+0000 7f6aca200700 1 -- 172.21.15.47:0/286199396 wait complete. 2024-03-21T11:22:40.580 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-21T11:22:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:40 smithi047 ceph-mon[31351]: pgmap v378: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:41.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:40 smithi100 ceph-mon[36245]: pgmap v378: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:40 smithi102 ceph-mon[36748]: pgmap v378: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:41.390 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-03-21T11:22:39.030381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662403,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":950,"num_read_kb":595,"num_write":426,"num_write_kb":3083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374260,"kb_used_data":12536,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567564,"statfs":{"total":1151940427776,"available":1151557185536,"internally_reserved":0,"allocated":12836864,"data_stored":8107871,"data_compressed":363522,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005089"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640403+0000","last_change":"2024-03-21T11:20:48.667185+0000","last_active":"2024-03-21T11:21:50.640403+0000","last_peered":"2024-03-21T11:21:50.640403+0000","last_clean":"2024-03-21T11:21:50.640403+0000","last_became_active":"2024-03-21T11:20:48.666855+0000","last_became_peered":"2024-03-21T11:20:48.666855+0000","last_unstale":"2024-03-21T11:21:50.640403+0000","last_undegraded":"2024-03-21T11:21:50.640403+0000","last_fullsized":"2024-03-21T11:21:50.640403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:30:37.393096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.646777+0000","last_change":"2024-03-21T11:20:48.671613+0000","last_active":"2024-03-21T11:21:50.646777+0000","last_peered":"2024-03-21T11:21:50.646777+0000","last_clean":"2024-03-21T11:21:50.646777+0000","last_became_active":"2024-03-21T11:20:48.671211+0000","last_became_peered":"2024-03-21T11:20:48.671211+0000","last_unstale":"2024-03-21T11:21:50.646777+0000","last_undegraded":"2024-03-21T11:21:50.646777+0000","last_fullsized":"2024-03-21T11:21:50.646777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:20.167922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647345+0000","last_change":"2024-03-21T11:20:48.655734+0000","last_active":"2024-03-21T11:21:50.647345+0000","last_peered":"2024-03-21T11:21:50.647345+0000","last_clean":"2024-03-21T11:21:50.647345+0000","last_became_active":"2024-03-21T11:20:48.653798+0000","last_became_peered":"2024-03-21T11:20:48.653798+0000","last_unstale":"2024-03-21T11:21:50.647345+0000","last_undegraded":"2024-03-21T11:21:50.647345+0000","last_fullsized":"2024-03-21T11:21:50.647345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:44.796405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.637299+0000","last_change":"2024-03-21T11:20:48.648784+0000","last_active":"2024-03-21T11:21:50.637299+0000","last_peered":"2024-03-21T11:21:50.637299+0000","last_clean":"2024-03-21T11:21:50.637299+0000","last_became_active":"2024-03-21T11:20:48.648040+0000","last_became_peered":"2024-03-21T11:20:48.648040+0000","last_unstale":"2024-03-21T11:21:50.637299+0000","last_undegraded":"2024-03-21T11:21:50.637299+0000","last_fullsized":"2024-03-21T11:21:50.637299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:48:51.786346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645770+0000","last_change":"2024-03-21T11:20:48.672084+0000","last_active":"2024-03-21T11:21:50.645770+0000","last_peered":"2024-03-21T11:21:50.645770+0000","last_clean":"2024-03-21T11:21:50.645770+0000","last_became_active":"2024-03-21T11:20:48.671441+0000","last_became_peered":"2024-03-21T11:20:48.671441+0000","last_unstale":"2024-03-21T11:21:50.645770+0000","last_undegraded":"2024-03-21T11:21:50.645770+0000","last_fullsized":"2024-03-21T11:21:50.645770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:20:17.638270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644602+0000","last_change":"2024-03-21T11:20:48.659782+0000","last_active":"2024-03-21T11:21:50.644602+0000","last_peered":"2024-03-21T11:21:50.644602+0000","last_clean":"2024-03-21T11:21:50.644602+0000","last_became_active":"2024-03-21T11:20:48.659460+0000","last_became_peered":"2024-03-21T11:20:48.659460+0000","last_unstale":"2024-03-21T11:21:50.644602+0000","last_undegraded":"2024-03-21T11:21:50.644602+0000","last_fullsized":"2024-03-21T11:21:50.644602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:57.712654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.641814+0000","last_change":"2024-03-21T11:20:48.659401+0000","last_active":"2024-03-21T11:21:50.641814+0000","last_peered":"2024-03-21T11:21:50.641814+0000","last_clean":"2024-03-21T11:21:50.641814+0000","last_became_active":"2024-03-21T11:20:48.659075+0000","last_became_peered":"2024-03-21T11:20:48.659075+0000","last_unstale":"2024-03-21T11:21:50.641814+0000","last_undegraded":"2024-03-21T11:21:50.641814+0000","last_fullsized":"2024-03-21T11:21:50.641814+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:54.342236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651418+0000","last_change":"2024-03-21T11:20:48.667175+0000","last_active":"2024-03-21T11:21:50.651418+0000","last_peered":"2024-03-21T11:21:50.651418+0000","last_clean":"2024-03-21T11:21:50.651418+0000","last_became_active":"2024-03-21T11:20:48.666756+0000","last_became_peered":"2024-03-21T11:20:48.666756+0000","last_unstale":"2024-03-21T11:21:50.651418+0000","last_undegraded":"2024-03-21T11:21:50.651418+0000","last_fullsized":"2024-03-21T11:21:50.651418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:25:39.694699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689988+0000","last_change":"2024-03-21T11:20:48.650584+0000","last_active":"2024-03-21T11:21:50.689988+0000","last_peered":"2024-03-21T11:21:50.689988+0000","last_clean":"2024-03-21T11:21:50.689988+0000","last_became_active":"2024-03-21T11:20:48.650251+0000","last_became_peered":"2024-03-21T11:20:48.650251+0000","last_unstale":"2024-03-21T11:21:50.689988+0000","last_undegraded":"2024-03-21T11:21:50.689988+0000","last_fullsized":"2024-03-21T11:21:50.689988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:35:59.000522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645180+0000","last_change":"2024-03-21T11:20:48.652945+0000","last_active":"2024-03-21T11:21:50.645180+0000","last_peered":"2024-03-21T11:21:50.645180+0000","last_clean":"2024-03-21T11:21:50.645180+0000","last_became_active":"2024-03-21T11:20:48.652626+0000","last_became_peered":"2024-03-21T11:20:48.652626+0000","last_unstale":"2024-03-21T11:21:50.645180+0000","last_undegraded":"2024-03-21T11:21:50.645180+0000","last_fullsized":"2024-03-21T11:21:50.645180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:43.586907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.643473+0000","last_change":"2024-03-21T11:20:48.651520+0000","last_active":"2024-03-21T11:21:50.643473+0000","last_peered":"2024-03-21T11:21:50.643473+0000","last_clean":"2024-03-21T11:21:50.643473+0000","last_became_active":"2024-03-21T11:20:48.651210+0000","last_became_peered":"2024-03-21T11:20:48.651210+0000","last_unstale":"2024-03-21T11:21:50.643473+0000","last_undegraded":"2024-03-21T11:21:50.643473+0000","last_fullsized":"2024-03-21T11:21:50.643473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:29.506732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.637181+0000","last_change":"2024-03-21T11:20:48.649844+0000","last_active":"2024-03-21T11:21:50.637181+0000","last_peered":"2024-03-21T11:21:50.637181+0000","last_clean":"2024-03-21T11:21:50.637181+0000","last_became_active":"2024-03-21T11:20:48.649503+0000","last_became_peered":"2024-03-21T11:20:48.649503+0000","last_unstale":"2024-03-21T11:21:50.637181+0000","last_undegraded":"2024-03-21T11:21:50.637181+0000","last_fullsized":"2024-03-21T11:21:50.637181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:31:55.734583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.654174+0000","last_change":"2024-03-21T11:20:48.662012+0000","last_active":"2024-03-21T11:21:50.654174+0000","last_peered":"2024-03-21T11:21:50.654174+0000","last_clean":"2024-03-21T11:21:50.654174+0000","last_became_active":"2024-03-21T11:20:48.661444+0000","last_became_peered":"2024-03-21T11:20:48.661444+0000","last_unstale":"2024-03-21T11:21:50.654174+0000","last_undegraded":"2024-03-21T11:21:50.654174+0000","last_fullsized":"2024-03-21T11:21:50.654174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:52.493244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644560+0000","last_change":"2024-03-21T11:20:48.663087+0000","last_active":"2024-03-21T11:21:50.644560+0000","last_peered":"2024-03-21T11:21:50.644560+0000","last_clean":"2024-03-21T11:21:50.644560+0000","last_became_active":"2024-03-21T11:20:48.662039+0000","last_became_peered":"2024-03-21T11:20:48.662039+0000","last_unstale":"2024-03-21T11:21:50.644560+0000","last_undegraded":"2024-03-21T11:21:50.644560+0000","last_fullsized":"2024-03-21T11:21:50.644560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:47:01.962314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645475+0000","last_change":"2024-03-21T11:20:48.663478+0000","last_active":"2024-03-21T11:21:50.645475+0000","last_peered":"2024-03-21T11:21:50.645475+0000","last_clean":"2024-03-21T11:21:50.645475+0000","last_became_active":"2024-03-21T11:20:48.663135+0000","last_became_peered":"2024-03-21T11:20:48.663135+0000","last_unstale":"2024-03-21T11:21:50.645475+0000","last_undegraded":"2024-03-21T11:21:50.645475+0000","last_fullsized":"2024-03-21T11:21:50.645475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:29.139760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689199+0000","last_change":"2024-03-21T11:20:48.653893+0000","last_active":"2024-03-21T11:21:50.689199+0000","last_peered":"2024-03-21T11:21:50.689199+0000","last_clean":"2024-03-21T11:21:50.689199+0000","last_became_active":"2024-03-21T11:20:48.653578+0000","last_became_peered":"2024-03-21T11:20:48.653578+0000","last_unstale":"2024-03-21T11:21:50.689199+0000","last_undegraded":"2024-03-21T11:21:50.689199+0000","last_fullsized":"2024-03-21T11:21:50.689199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:54:31.254030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640175+0000","last_change":"2024-03-21T11:20:48.667388+0000","last_active":"2024-03-21T11:21:50.640175+0000","last_peered":"2024-03-21T11:21:50.640175+0000","last_clean":"2024-03-21T11:21:50.640175+0000","last_became_active":"2024-03-21T11:20:48.667077+0000","last_became_peered":"2024-03-21T11:20:48.667077+0000","last_unstale":"2024-03-21T11:21:50.640175+0000","last_undegraded":"2024-03-21T11:21:50.640175+0000","last_fullsized":"2024-03-21T11:21:50.640175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:13:32.093457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.646546+0000","last_change":"2024-03-21T11:20:48.658479+0000","last_active":"2024-03-21T11:21:50.646546+0000","last_peered":"2024-03-21T11:21:50.646546+0000","last_clean":"2024-03-21T11:21:50.646546+0000","last_became_active":"2024-03-21T11:20:48.658011+0000","last_became_peered":"2024-03-21T11:20:48.658011+0000","last_unstale":"2024-03-21T11:21:50.646546+0000","last_undegraded":"2024-03-21T11:21:50.646546+0000","last_fullsized":"2024-03-21T11:21:50.646546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:33.313537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.646244+0000","last_change":"2024-03-21T11:20:48.668648+0000","last_active":"2024-03-21T11:21:50.646244+0000","last_peered":"2024-03-21T11:21:50.646244+0000","last_clean":"2024-03-21T11:21:50.646244+0000","last_became_active":"2024-03-21T11:20:48.668376+0000","last_became_peered":"2024-03-21T11:20:48.668376+0000","last_unstale":"2024-03-21T11:21:50.646244+0000","last_undegraded":"2024-03-21T11:21:50.646244+0000","last_fullsized":"2024-03-21T11:21:50.646244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:49:47.601724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644436+0000","last_change":"2024-03-21T11:20:53.755600+0000","last_active":"2024-03-21T11:21:50.644436+0000","last_peered":"2024-03-21T11:21:50.644436+0000","last_clean":"2024-03-21T11:21:50.644436+0000","last_became_active":"2024-03-21T11:20:53.755321+0000","last_became_peered":"2024-03-21T11:20:53.755321+0000","last_unstale":"2024-03-21T11:21:50.644436+0000","last_undegraded":"2024-03-21T11:21:50.644436+0000","last_fullsized":"2024-03-21T11:21:50.644436+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:04.905910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":122,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:35.771623+0000","last_change":"2024-03-21T11:20:48.161001+0000","last_active":"2024-03-21T11:22:35.771623+0000","last_peered":"2024-03-21T11:22:35.771623+0000","last_clean":"2024-03-21T11:22:35.771623+0000","last_became_active":"2024-03-21T11:20:48.153053+0000","last_became_peered":"2024-03-21T11:20:48.153053+0000","last_unstale":"2024-03-21T11:22:35.771623+0000","last_undegraded":"2024-03-21T11:22:35.771623+0000","last_fullsized":"2024-03-21T11:22:35.771623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:26.010581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.652181+0000","last_change":"2024-03-21T11:20:48.671430+0000","last_active":"2024-03-21T11:21:50.652181+0000","last_peered":"2024-03-21T11:21:50.652181+0000","last_clean":"2024-03-21T11:21:50.652181+0000","last_became_active":"2024-03-21T11:20:48.670192+0000","last_became_peered":"2024-03-21T11:20:48.670192+0000","last_unstale":"2024-03-21T11:21:50.652181+0000","last_undegraded":"2024-03-21T11:21:50.652181+0000","last_fullsized":"2024-03-21T11:21:50.652181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:00:53.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.636538+0000","last_change":"2024-03-21T11:20:53.750313+0000","last_active":"2024-03-21T11:21:50.636538+0000","last_peered":"2024-03-21T11:21:50.636538+0000","last_clean":"2024-03-21T11:21:50.636538+0000","last_became_active":"2024-03-21T11:20:53.749913+0000","last_became_peered":"2024-03-21T11:20:53.749913+0000","last_unstale":"2024-03-21T11:21:50.636538+0000","last_undegraded":"2024-03-21T11:21:50.636538+0000","last_fullsized":"2024-03-21T11:21:50.636538+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:39:09.007674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.652592+0000","last_change":"2024-03-21T11:20:48.159934+0000","last_active":"2024-03-21T11:21:50.652592+0000","last_peered":"2024-03-21T11:21:50.652592+0000","last_clean":"2024-03-21T11:21:50.652592+0000","last_became_active":"2024-03-21T11:20:48.159650+0000","last_became_peered":"2024-03-21T11:20:48.159650+0000","last_unstale":"2024-03-21T11:21:50.652592+0000","last_undegraded":"2024-03-21T11:21:50.652592+0000","last_fullsized":"2024-03-21T11:21:50.652592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:49:56.891733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647823+0000","last_change":"2024-03-21T11:20:48.673012+0000","last_active":"2024-03-21T11:21:50.647823+0000","last_peered":"2024-03-21T11:21:50.647823+0000","last_clean":"2024-03-21T11:21:50.647823+0000","last_became_active":"2024-03-21T11:20:48.672584+0000","last_became_peered":"2024-03-21T11:20:48.672584+0000","last_unstale":"2024-03-21T11:21:50.647823+0000","last_undegraded":"2024-03-21T11:21:50.647823+0000","last_fullsized":"2024-03-21T11:21:50.647823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:48.213569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.655957+0000","last_change":"2024-03-21T11:21:50.655957+0000","last_active":"2024-03-21T11:21:50.655957+0000","last_peered":"2024-03-21T11:21:50.655957+0000","last_clean":"2024-03-21T11:21:50.655957+0000","last_became_active":"2024-03-21T11:21:50.655477+0000","last_became_peered":"2024-03-21T11:21:50.655477+0000","last_unstale":"2024-03-21T11:21:50.655957+0000","last_undegraded":"2024-03-21T11:21:50.655957+0000","last_fullsized":"2024-03-21T11:21:50.655957+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:58:28.476186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.664575+0000","last_change":"2024-03-21T11:21:50.664575+0000","last_active":"2024-03-21T11:21:50.664575+0000","last_peered":"2024-03-21T11:21:50.664575+0000","last_clean":"2024-03-21T11:21:50.664575+0000","last_became_active":"2024-03-21T11:21:50.647335+0000","last_became_peered":"2024-03-21T11:21:50.647335+0000","last_unstale":"2024-03-21T11:21:50.664575+0000","last_undegraded":"2024-03-21T11:21:50.664575+0000","last_fullsized":"2024-03-21T11:21:50.664575+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:50:28.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689390+0000","last_change":"2024-03-21T11:20:48.662392+0000","last_active":"2024-03-21T11:21:50.689390+0000","last_peered":"2024-03-21T11:21:50.689390+0000","last_clean":"2024-03-21T11:21:50.689390+0000","last_became_active":"2024-03-21T11:20:48.662042+0000","last_became_peered":"2024-03-21T11:20:48.662042+0000","last_unstale":"2024-03-21T11:21:50.689390+0000","last_undegraded":"2024-03-21T11:21:50.689390+0000","last_fullsized":"2024-03-21T11:21:50.689390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:59:34.456582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.636723+0000","last_change":"2024-03-21T11:20:53.749693+0000","last_active":"2024-03-21T11:21:50.636723+0000","last_peered":"2024-03-21T11:21:50.636723+0000","last_clean":"2024-03-21T11:21:50.636723+0000","last_became_active":"2024-03-21T11:20:53.749438+0000","last_became_peered":"2024-03-21T11:20:53.749438+0000","last_unstale":"2024-03-21T11:21:50.636723+0000","last_undegraded":"2024-03-21T11:21:50.636723+0000","last_fullsized":"2024-03-21T11:21:50.636723+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:37.714471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647072+0000","last_change":"2024-03-21T11:20:48.132587+0000","last_active":"2024-03-21T11:21:50.647072+0000","last_peered":"2024-03-21T11:21:50.647072+0000","last_clean":"2024-03-21T11:21:50.647072+0000","last_became_active":"2024-03-21T11:20:48.132232+0000","last_became_peered":"2024-03-21T11:20:48.132232+0000","last_unstale":"2024-03-21T11:21:50.647072+0000","last_undegraded":"2024-03-21T11:21:50.647072+0000","last_fullsized":"2024-03-21T11:21:50.647072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:20:45.217545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639574+0000","last_change":"2024-03-21T11:20:48.656881+0000","last_active":"2024-03-21T11:21:50.639574+0000","last_peered":"2024-03-21T11:21:50.639574+0000","last_clean":"2024-03-21T11:21:50.639574+0000","last_became_active":"2024-03-21T11:20:48.656426+0000","last_became_peered":"2024-03-21T11:20:48.656426+0000","last_unstale":"2024-03-21T11:21:50.639574+0000","last_undegraded":"2024-03-21T11:21:50.639574+0000","last_fullsized":"2024-03-21T11:21:50.639574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:06:27.398062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645967+0000","last_change":"2024-03-21T11:20:53.733396+0000","last_active":"2024-03-21T11:21:50.645967+0000","last_peered":"2024-03-21T11:21:50.645967+0000","last_clean":"2024-03-21T11:21:50.645967+0000","last_became_active":"2024-03-21T11:20:53.732985+0000","last_became_peered":"2024-03-21T11:20:53.732985+0000","last_unstale":"2024-03-21T11:21:50.645967+0000","last_undegraded":"2024-03-21T11:21:50.645967+0000","last_fullsized":"2024-03-21T11:21:50.645967+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:54:14.064761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":115,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:35.736427+0000","last_change":"2024-03-21T11:20:48.133362+0000","last_active":"2024-03-21T11:22:35.736427+0000","last_peered":"2024-03-21T11:22:35.736427+0000","last_clean":"2024-03-21T11:22:35.736427+0000","last_became_active":"2024-03-21T11:20:48.132405+0000","last_became_peered":"2024-03-21T11:20:48.132405+0000","last_unstale":"2024-03-21T11:22:35.736427+0000","last_undegraded":"2024-03-21T11:22:35.736427+0000","last_fullsized":"2024-03-21T11:22:35.736427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:48:52.688948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644945+0000","last_change":"2024-03-21T11:20:48.659610+0000","last_active":"2024-03-21T11:21:50.644945+0000","last_peered":"2024-03-21T11:21:50.644945+0000","last_clean":"2024-03-21T11:21:50.644945+0000","last_became_active":"2024-03-21T11:20:48.659257+0000","last_became_peered":"2024-03-21T11:20:48.659257+0000","last_unstale":"2024-03-21T11:21:50.644945+0000","last_undegraded":"2024-03-21T11:21:50.644945+0000","last_fullsized":"2024-03-21T11:21:50.644945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:17:30.744434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.656047+0000","last_change":"2024-03-21T11:21:50.656047+0000","last_active":"2024-03-21T11:21:50.656047+0000","last_peered":"2024-03-21T11:21:50.656047+0000","last_clean":"2024-03-21T11:21:50.656047+0000","last_became_active":"2024-03-21T11:21:50.655554+0000","last_became_peered":"2024-03-21T11:21:50.655554+0000","last_unstale":"2024-03-21T11:21:50.656047+0000","last_undegraded":"2024-03-21T11:21:50.656047+0000","last_fullsized":"2024-03-21T11:21:50.656047+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:35:15.858237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.695331+0000","last_change":"2024-03-21T11:21:50.695331+0000","last_active":"2024-03-21T11:21:50.695331+0000","last_peered":"2024-03-21T11:21:50.695331+0000","last_clean":"2024-03-21T11:21:50.695331+0000","last_became_active":"2024-03-21T11:21:50.690368+0000","last_became_peered":"2024-03-21T11:21:50.690368+0000","last_unstale":"2024-03-21T11:21:50.695331+0000","last_undegraded":"2024-03-21T11:21:50.695331+0000","last_fullsized":"2024-03-21T11:21:50.695331+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:50:03.835364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640552+0000","last_change":"2024-03-21T11:20:48.671221+0000","last_active":"2024-03-21T11:21:50.640552+0000","last_peered":"2024-03-21T11:21:50.640552+0000","last_clean":"2024-03-21T11:21:50.640552+0000","last_became_active":"2024-03-21T11:20:48.670837+0000","last_became_peered":"2024-03-21T11:20:48.670837+0000","last_unstale":"2024-03-21T11:21:50.640552+0000","last_undegraded":"2024-03-21T11:21:50.640552+0000","last_fullsized":"2024-03-21T11:21:50.640552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:38.680409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640079+0000","last_change":"2024-03-21T11:20:53.747501+0000","last_active":"2024-03-21T11:21:50.640079+0000","last_peered":"2024-03-21T11:21:50.640079+0000","last_clean":"2024-03-21T11:21:50.640079+0000","last_became_active":"2024-03-21T11:20:53.747062+0000","last_became_peered":"2024-03-21T11:20:53.747062+0000","last_unstale":"2024-03-21T11:21:50.640079+0000","last_undegraded":"2024-03-21T11:21:50.640079+0000","last_fullsized":"2024-03-21T11:21:50.640079+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:58:36.904221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.652004+0000","last_change":"2024-03-21T11:20:48.156175+0000","last_active":"2024-03-21T11:21:50.652004+0000","last_peered":"2024-03-21T11:21:50.652004+0000","last_clean":"2024-03-21T11:21:50.652004+0000","last_became_active":"2024-03-21T11:20:48.154271+0000","last_became_peered":"2024-03-21T11:20:48.154271+0000","last_unstale":"2024-03-21T11:21:50.652004+0000","last_undegraded":"2024-03-21T11:21:50.652004+0000","last_fullsized":"2024-03-21T11:21:50.652004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:04.216391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651905+0000","last_change":"2024-03-21T11:20:48.670011+0000","last_active":"2024-03-21T11:21:50.651905+0000","last_peered":"2024-03-21T11:21:50.651905+0000","last_clean":"2024-03-21T11:21:50.651905+0000","last_became_active":"2024-03-21T11:20:48.669016+0000","last_became_peered":"2024-03-21T11:20:48.669016+0000","last_unstale":"2024-03-21T11:21:50.651905+0000","last_undegraded":"2024-03-21T11:21:50.651905+0000","last_fullsized":"2024-03-21T11:21:50.651905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:43:18.906490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.646486+0000","last_change":"2024-03-21T11:20:53.751934+0000","last_active":"2024-03-21T11:21:50.646486+0000","last_peered":"2024-03-21T11:21:50.646486+0000","last_clean":"2024-03-21T11:21:50.646486+0000","last_became_active":"2024-03-21T11:20:53.751413+0000","last_became_peered":"2024-03-21T11:20:53.751413+0000","last_unstale":"2024-03-21T11:21:50.646486+0000","last_undegraded":"2024-03-21T11:21:50.646486+0000","last_fullsized":"2024-03-21T11:21:50.646486+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:12:33.880376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639167+0000","last_change":"2024-03-21T11:20:48.131796+0000","last_active":"2024-03-21T11:21:50.639167+0000","last_peered":"2024-03-21T11:21:50.639167+0000","last_clean":"2024-03-21T11:21:50.639167+0000","last_became_active":"2024-03-21T11:20:48.131478+0000","last_became_peered":"2024-03-21T11:20:48.131478+0000","last_unstale":"2024-03-21T11:21:50.639167+0000","last_undegraded":"2024-03-21T11:21:50.639167+0000","last_fullsized":"2024-03-21T11:21:50.639167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:59:24.871300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640780+0000","last_change":"2024-03-21T11:20:48.660046+0000","last_active":"2024-03-21T11:21:50.640780+0000","last_peered":"2024-03-21T11:21:50.640780+0000","last_clean":"2024-03-21T11:21:50.640780+0000","last_became_active":"2024-03-21T11:20:48.659621+0000","last_became_peered":"2024-03-21T11:20:48.659621+0000","last_unstale":"2024-03-21T11:21:50.640780+0000","last_undegraded":"2024-03-21T11:21:50.640780+0000","last_fullsized":"2024-03-21T11:21:50.640780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:47:01.923285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.653445+0000","last_change":"2024-03-21T11:20:53.754458+0000","last_active":"2024-03-21T11:21:50.653445+0000","last_peered":"2024-03-21T11:21:50.653445+0000","last_clean":"2024-03-21T11:21:50.653445+0000","last_became_active":"2024-03-21T11:20:53.753927+0000","last_became_peered":"2024-03-21T11:20:53.753927+0000","last_unstale":"2024-03-21T11:21:50.653445+0000","last_undegraded":"2024-03-21T11:21:50.653445+0000","last_fullsized":"2024-03-21T11:21:50.653445+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:00:47.392699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":189,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:35.691586+0000","last_change":"2024-03-21T11:20:48.154125+0000","last_active":"2024-03-21T11:22:35.691586+0000","last_peered":"2024-03-21T11:22:35.691586+0000","last_clean":"2024-03-21T11:22:35.691586+0000","last_became_active":"2024-03-21T11:20:48.153857+0000","last_became_peered":"2024-03-21T11:20:48.153857+0000","last_unstale":"2024-03-21T11:22:35.691586+0000","last_undegraded":"2024-03-21T11:22:35.691586+0000","last_fullsized":"2024-03-21T11:22:35.691586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:54:06.525335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689811+0000","last_change":"2024-03-21T11:20:48.650143+0000","last_active":"2024-03-21T11:21:50.689811+0000","last_peered":"2024-03-21T11:21:50.689811+0000","last_clean":"2024-03-21T11:21:50.689811+0000","last_became_active":"2024-03-21T11:20:48.649825+0000","last_became_peered":"2024-03-21T11:20:48.649825+0000","last_unstale":"2024-03-21T11:21:50.689811+0000","last_undegraded":"2024-03-21T11:21:50.689811+0000","last_fullsized":"2024-03-21T11:21:50.689811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:09:15.735280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639235+0000","last_change":"2024-03-21T11:20:53.743006+0000","last_active":"2024-03-21T11:21:50.639235+0000","last_peered":"2024-03-21T11:21:50.639235+0000","last_clean":"2024-03-21T11:21:50.639235+0000","last_became_active":"2024-03-21T11:20:53.742587+0000","last_became_peered":"2024-03-21T11:20:53.742587+0000","last_unstale":"2024-03-21T11:21:50.639235+0000","last_undegraded":"2024-03-21T11:21:50.639235+0000","last_fullsized":"2024-03-21T11:21:50.639235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:06:06.845791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.642935+0000","last_change":"2024-03-21T11:20:48.137137+0000","last_active":"2024-03-21T11:21:50.642935+0000","last_peered":"2024-03-21T11:21:50.642935+0000","last_clean":"2024-03-21T11:21:50.642935+0000","last_became_active":"2024-03-21T11:20:48.136790+0000","last_became_peered":"2024-03-21T11:20:48.136790+0000","last_unstale":"2024-03-21T11:21:50.642935+0000","last_undegraded":"2024-03-21T11:21:50.642935+0000","last_fullsized":"2024-03-21T11:21:50.642935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:22:47.223969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647962+0000","last_change":"2024-03-21T11:20:48.673568+0000","last_active":"2024-03-21T11:21:50.647962+0000","last_peered":"2024-03-21T11:21:50.647962+0000","last_clean":"2024-03-21T11:21:50.647962+0000","last_became_active":"2024-03-21T11:20:48.672613+0000","last_became_peered":"2024-03-21T11:20:48.672613+0000","last_unstale":"2024-03-21T11:21:50.647962+0000","last_undegraded":"2024-03-21T11:21:50.647962+0000","last_fullsized":"2024-03-21T11:21:50.647962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:43:38.193802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.637415+0000","last_change":"2024-03-21T11:20:53.739228+0000","last_active":"2024-03-21T11:21:50.637415+0000","last_peered":"2024-03-21T11:21:50.637415+0000","last_clean":"2024-03-21T11:21:50.637415+0000","last_became_active":"2024-03-21T11:20:53.738750+0000","last_became_peered":"2024-03-21T11:20:53.738750+0000","last_unstale":"2024-03-21T11:21:50.637415+0000","last_undegraded":"2024-03-21T11:21:50.637415+0000","last_fullsized":"2024-03-21T11:21:50.637415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:58:33.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639461+0000","last_change":"2024-03-21T11:20:48.144451+0000","last_active":"2024-03-21T11:21:50.639461+0000","last_peered":"2024-03-21T11:21:50.639461+0000","last_clean":"2024-03-21T11:21:50.639461+0000","last_became_active":"2024-03-21T11:20:48.144074+0000","last_became_peered":"2024-03-21T11:20:48.144074+0000","last_unstale":"2024-03-21T11:21:50.639461+0000","last_undegraded":"2024-03-21T11:21:50.639461+0000","last_fullsized":"2024-03-21T11:21:50.639461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:33:17.032461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689581+0000","last_change":"2024-03-21T11:20:48.654472+0000","last_active":"2024-03-21T11:21:50.689581+0000","last_peered":"2024-03-21T11:21:50.689581+0000","last_clean":"2024-03-21T11:21:50.689581+0000","last_became_active":"2024-03-21T11:20:48.654129+0000","last_became_peered":"2024-03-21T11:20:48.654129+0000","last_unstale":"2024-03-21T11:21:50.689581+0000","last_undegraded":"2024-03-21T11:21:50.689581+0000","last_fullsized":"2024-03-21T11:21:50.689581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:28:31.224296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.641411+0000","last_change":"2024-03-21T11:20:53.739994+0000","last_active":"2024-03-21T11:21:50.641411+0000","last_peered":"2024-03-21T11:21:50.641411+0000","last_clean":"2024-03-21T11:21:50.641411+0000","last_became_active":"2024-03-21T11:20:53.738319+0000","last_became_peered":"2024-03-21T11:20:53.738319+0000","last_unstale":"2024-03-21T11:21:50.641411+0000","last_undegraded":"2024-03-21T11:21:50.641411+0000","last_fullsized":"2024-03-21T11:21:50.641411+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:55.694986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":227,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:35.691500+0000","last_change":"2024-03-21T11:20:48.131461+0000","last_active":"2024-03-21T11:22:35.691500+0000","last_peered":"2024-03-21T11:22:35.691500+0000","last_clean":"2024-03-21T11:22:35.691500+0000","last_became_active":"2024-03-21T11:20:48.131079+0000","last_became_peered":"2024-03-21T11:20:48.131079+0000","last_unstale":"2024-03-21T11:22:35.691500+0000","last_undegraded":"2024-03-21T11:22:35.691500+0000","last_fullsized":"2024-03-21T11:22:35.691500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:48:57.528444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64206,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":163,"num_read_kb":144,"num_write":37,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647182+0000","last_change":"2024-03-21T11:20:48.653696+0000","last_active":"2024-03-21T11:21:50.647182+0000","last_peered":"2024-03-21T11:21:50.647182+0000","last_clean":"2024-03-21T11:21:50.647182+0000","last_became_active":"2024-03-21T11:20:48.653218+0000","last_became_peered":"2024-03-21T11:20:48.653218+0000","last_unstale":"2024-03-21T11:21:50.647182+0000","last_undegraded":"2024-03-21T11:21:50.647182+0000","last_fullsized":"2024-03-21T11:21:50.647182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:11:59.419040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639982+0000","last_change":"2024-03-21T11:20:53.754019+0000","last_active":"2024-03-21T11:21:50.639982+0000","last_peered":"2024-03-21T11:21:50.639982+0000","last_clean":"2024-03-21T11:21:50.639982+0000","last_became_active":"2024-03-21T11:20:53.753606+0000","last_became_peered":"2024-03-21T11:20:53.753606+0000","last_unstale":"2024-03-21T11:21:50.639982+0000","last_undegraded":"2024-03-21T11:21:50.639982+0000","last_fullsized":"2024-03-21T11:21:50.639982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:23:27.898596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645028+0000","last_change":"2024-03-21T11:20:48.153544+0000","last_active":"2024-03-21T11:21:50.645028+0000","last_peered":"2024-03-21T11:21:50.645028+0000","last_clean":"2024-03-21T11:21:50.645028+0000","last_became_active":"2024-03-21T11:20:48.153279+0000","last_became_peered":"2024-03-21T11:20:48.153279+0000","last_unstale":"2024-03-21T11:21:50.645028+0000","last_undegraded":"2024-03-21T11:21:50.645028+0000","last_fullsized":"2024-03-21T11:21:50.645028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:31:31.911844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.654309+0000","last_change":"2024-03-21T11:20:48.665140+0000","last_active":"2024-03-21T11:21:50.654309+0000","last_peered":"2024-03-21T11:21:50.654309+0000","last_clean":"2024-03-21T11:21:50.654309+0000","last_became_active":"2024-03-21T11:20:48.664226+0000","last_became_peered":"2024-03-21T11:20:48.664226+0000","last_unstale":"2024-03-21T11:21:50.654309+0000","last_undegraded":"2024-03-21T11:21:50.654309+0000","last_fullsized":"2024-03-21T11:21:50.654309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:47:38.951403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.641466+0000","last_change":"2024-03-21T11:20:53.751887+0000","last_active":"2024-03-21T11:21:50.641466+0000","last_peered":"2024-03-21T11:21:50.641466+0000","last_clean":"2024-03-21T11:21:50.641466+0000","last_became_active":"2024-03-21T11:20:53.751353+0000","last_became_peered":"2024-03-21T11:20:53.751353+0000","last_unstale":"2024-03-21T11:21:50.641466+0000","last_undegraded":"2024-03-21T11:21:50.641466+0000","last_fullsized":"2024-03-21T11:21:50.641466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:21:36.689786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639455+0000","last_change":"2024-03-21T11:20:48.162150+0000","last_active":"2024-03-21T11:21:50.639455+0000","last_peered":"2024-03-21T11:21:50.639455+0000","last_clean":"2024-03-21T11:21:50.639455+0000","last_became_active":"2024-03-21T11:20:48.161825+0000","last_became_peered":"2024-03-21T11:20:48.161825+0000","last_unstale":"2024-03-21T11:21:50.639455+0000","last_undegraded":"2024-03-21T11:21:50.639455+0000","last_fullsized":"2024-03-21T11:21:50.639455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:10:47.097540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.642620+0000","last_change":"2024-03-21T11:20:48.663856+0000","last_active":"2024-03-21T11:21:50.642620+0000","last_peered":"2024-03-21T11:21:50.642620+0000","last_clean":"2024-03-21T11:21:50.642620+0000","last_became_active":"2024-03-21T11:20:48.662908+0000","last_became_peered":"2024-03-21T11:20:48.662908+0000","last_unstale":"2024-03-21T11:21:50.642620+0000","last_undegraded":"2024-03-21T11:21:50.642620+0000","last_fullsized":"2024-03-21T11:21:50.642620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:11:12.255265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.643577+0000","last_change":"2024-03-21T11:20:53.755942+0000","last_active":"2024-03-21T11:21:50.643577+0000","last_peered":"2024-03-21T11:21:50.643577+0000","last_clean":"2024-03-21T11:21:50.643577+0000","last_became_active":"2024-03-21T11:20:53.755141+0000","last_became_peered":"2024-03-21T11:20:53.755141+0000","last_unstale":"2024-03-21T11:21:50.643577+0000","last_undegraded":"2024-03-21T11:21:50.643577+0000","last_fullsized":"2024-03-21T11:21:50.643577+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:30.141117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":116,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:36.727880+0000","last_change":"2024-03-21T11:20:48.132460+0000","last_active":"2024-03-21T11:22:36.727880+0000","last_peered":"2024-03-21T11:22:36.727880+0000","last_clean":"2024-03-21T11:22:36.727880+0000","last_became_active":"2024-03-21T11:20:48.131573+0000","last_became_peered":"2024-03-21T11:20:48.131573+0000","last_unstale":"2024-03-21T11:22:36.727880+0000","last_undegraded":"2024-03-21T11:22:36.727880+0000","last_fullsized":"2024-03-21T11:22:36.727880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:40:02.821536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":83,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.641878+0000","last_change":"2024-03-21T11:20:48.656262+0000","last_active":"2024-03-21T11:21:50.641878+0000","last_peered":"2024-03-21T11:21:50.641878+0000","last_clean":"2024-03-21T11:21:50.641878+0000","last_became_active":"2024-03-21T11:20:48.655924+0000","last_became_peered":"2024-03-21T11:20:48.655924+0000","last_unstale":"2024-03-21T11:21:50.641878+0000","last_undegraded":"2024-03-21T11:21:50.641878+0000","last_fullsized":"2024-03-21T11:21:50.641878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:24:20.412185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651635+0000","last_change":"2024-03-21T11:20:53.758666+0000","last_active":"2024-03-21T11:21:50.651635+0000","last_peered":"2024-03-21T11:21:50.651635+0000","last_clean":"2024-03-21T11:21:50.651635+0000","last_became_active":"2024-03-21T11:20:53.757398+0000","last_became_peered":"2024-03-21T11:20:53.757398+0000","last_unstale":"2024-03-21T11:21:50.651635+0000","last_undegraded":"2024-03-21T11:21:50.651635+0000","last_fullsized":"2024-03-21T11:21:50.651635+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:34.198388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644797+0000","last_change":"2024-03-21T11:20:48.132725+0000","last_active":"2024-03-21T11:21:50.644797+0000","last_peered":"2024-03-21T11:21:50.644797+0000","last_clean":"2024-03-21T11:21:50.644797+0000","last_became_active":"2024-03-21T11:20:48.132367+0000","last_became_peered":"2024-03-21T11:20:48.132367+0000","last_unstale":"2024-03-21T11:21:50.644797+0000","last_undegraded":"2024-03-21T11:21:50.644797+0000","last_fullsized":"2024-03-21T11:21:50.644797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:19:18.334962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640266+0000","last_change":"2024-03-21T11:20:48.656302+0000","last_active":"2024-03-21T11:21:50.640266+0000","last_peered":"2024-03-21T11:21:50.640266+0000","last_clean":"2024-03-21T11:21:50.640266+0000","last_became_active":"2024-03-21T11:20:48.654305+0000","last_became_peered":"2024-03-21T11:20:48.654305+0000","last_unstale":"2024-03-21T11:21:50.640266+0000","last_undegraded":"2024-03-21T11:21:50.640266+0000","last_fullsized":"2024-03-21T11:21:50.640266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:59:40.766829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639309+0000","last_change":"2024-03-21T11:20:53.750710+0000","last_active":"2024-03-21T11:21:50.639309+0000","last_peered":"2024-03-21T11:21:50.639309+0000","last_clean":"2024-03-21T11:21:50.639309+0000","last_became_active":"2024-03-21T11:20:53.750255+0000","last_became_peered":"2024-03-21T11:20:53.750255+0000","last_unstale":"2024-03-21T11:21:50.639309+0000","last_undegraded":"2024-03-21T11:21:50.639309+0000","last_fullsized":"2024-03-21T11:21:50.639309+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:36:28.611838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647379+0000","last_change":"2024-03-21T11:20:48.133524+0000","last_active":"2024-03-21T11:21:50.647379+0000","last_peered":"2024-03-21T11:21:50.647379+0000","last_clean":"2024-03-21T11:21:50.647379+0000","last_became_active":"2024-03-21T11:20:48.133186+0000","last_became_peered":"2024-03-21T11:20:48.133186+0000","last_unstale":"2024-03-21T11:21:50.647379+0000","last_undegraded":"2024-03-21T11:21:50.647379+0000","last_fullsized":"2024-03-21T11:21:50.647379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:02.018639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639075+0000","last_change":"2024-03-21T11:20:48.654126+0000","last_active":"2024-03-21T11:21:50.639075+0000","last_peered":"2024-03-21T11:21:50.639075+0000","last_clean":"2024-03-21T11:21:50.639075+0000","last_became_active":"2024-03-21T11:20:48.653835+0000","last_became_peered":"2024-03-21T11:20:48.653835+0000","last_unstale":"2024-03-21T11:21:50.639075+0000","last_undegraded":"2024-03-21T11:21:50.639075+0000","last_fullsized":"2024-03-21T11:21:50.639075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:12:03.332076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644944+0000","last_change":"2024-03-21T11:20:53.916645+0000","last_active":"2024-03-21T11:21:50.644944+0000","last_peered":"2024-03-21T11:21:50.644944+0000","last_clean":"2024-03-21T11:21:50.644944+0000","last_became_active":"2024-03-21T11:20:53.916178+0000","last_became_peered":"2024-03-21T11:20:53.916178+0000","last_unstale":"2024-03-21T11:21:50.644944+0000","last_undegraded":"2024-03-21T11:21:50.644944+0000","last_fullsized":"2024-03-21T11:21:50.644944+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:56:15.805231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640468+0000","last_change":"2024-03-21T11:20:48.159877+0000","last_active":"2024-03-21T11:21:50.640468+0000","last_peered":"2024-03-21T11:21:50.640468+0000","last_clean":"2024-03-21T11:21:50.640468+0000","last_became_active":"2024-03-21T11:20:48.159529+0000","last_became_peered":"2024-03-21T11:20:48.159529+0000","last_unstale":"2024-03-21T11:21:50.640468+0000","last_undegraded":"2024-03-21T11:21:50.640468+0000","last_fullsized":"2024-03-21T11:21:50.640468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:17:09.662462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645078+0000","last_change":"2024-03-21T11:20:48.663311+0000","last_active":"2024-03-21T11:21:50.645078+0000","last_peered":"2024-03-21T11:21:50.645078+0000","last_clean":"2024-03-21T11:21:50.645078+0000","last_became_active":"2024-03-21T11:20:48.662961+0000","last_became_peered":"2024-03-21T11:20:48.662961+0000","last_unstale":"2024-03-21T11:21:50.645078+0000","last_undegraded":"2024-03-21T11:21:50.645078+0000","last_fullsized":"2024-03-21T11:21:50.645078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:14:46.616326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651173+0000","last_change":"2024-03-21T11:20:53.751804+0000","last_active":"2024-03-21T11:21:50.651173+0000","last_peered":"2024-03-21T11:21:50.651173+0000","last_clean":"2024-03-21T11:21:50.651173+0000","last_became_active":"2024-03-21T11:20:53.751323+0000","last_became_peered":"2024-03-21T11:20:53.751323+0000","last_unstale":"2024-03-21T11:21:50.651173+0000","last_undegraded":"2024-03-21T11:21:50.651173+0000","last_fullsized":"2024-03-21T11:21:50.651173+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:38:25.677345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":114,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:35.772320+0000","last_change":"2024-03-21T11:20:48.147580+0000","last_active":"2024-03-21T11:22:35.772320+0000","last_peered":"2024-03-21T11:22:35.772320+0000","last_clean":"2024-03-21T11:22:35.772320+0000","last_became_active":"2024-03-21T11:20:48.147222+0000","last_became_peered":"2024-03-21T11:20:48.147222+0000","last_unstale":"2024-03-21T11:22:35.772320+0000","last_undegraded":"2024-03-21T11:22:35.772320+0000","last_fullsized":"2024-03-21T11:22:35.772320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:49:04.204129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644790+0000","last_change":"2024-03-21T11:20:48.667940+0000","last_active":"2024-03-21T11:21:50.644790+0000","last_peered":"2024-03-21T11:21:50.644790+0000","last_clean":"2024-03-21T11:21:50.644790+0000","last_became_active":"2024-03-21T11:20:48.667595+0000","last_became_peered":"2024-03-21T11:20:48.667595+0000","last_unstale":"2024-03-21T11:21:50.644790+0000","last_undegraded":"2024-03-21T11:21:50.644790+0000","last_fullsized":"2024-03-21T11:21:50.644790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:08:14.007589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.652571+0000","last_change":"2024-03-21T11:20:53.916931+0000","last_active":"2024-03-21T11:21:50.652571+0000","last_peered":"2024-03-21T11:21:50.652571+0000","last_clean":"2024-03-21T11:21:50.652571+0000","last_became_active":"2024-03-21T11:20:53.916495+0000","last_became_peered":"2024-03-21T11:20:53.916495+0000","last_unstale":"2024-03-21T11:21:50.652571+0000","last_undegraded":"2024-03-21T11:21:50.652571+0000","last_fullsized":"2024-03-21T11:21:50.652571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:18:33.046052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639370+0000","last_change":"2024-03-21T11:20:48.652997+0000","last_active":"2024-03-21T11:21:50.639370+0000","last_peered":"2024-03-21T11:21:50.639370+0000","last_clean":"2024-03-21T11:21:50.639370+0000","last_became_active":"2024-03-21T11:20:48.652680+0000","last_became_peered":"2024-03-21T11:20:48.652680+0000","last_unstale":"2024-03-21T11:21:50.639370+0000","last_undegraded":"2024-03-21T11:21:50.639370+0000","last_fullsized":"2024-03-21T11:21:50.639370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:00:19.516607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":113,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:35.725659+0000","last_change":"2024-03-21T11:20:48.164607+0000","last_active":"2024-03-21T11:22:35.725659+0000","last_peered":"2024-03-21T11:22:35.725659+0000","last_clean":"2024-03-21T11:22:35.725659+0000","last_became_active":"2024-03-21T11:20:48.164222+0000","last_became_peered":"2024-03-21T11:20:48.164222+0000","last_unstale":"2024-03-21T11:22:35.725659+0000","last_undegraded":"2024-03-21T11:22:35.725659+0000","last_fullsized":"2024-03-21T11:22:35.725659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:52:24.359155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.658412+0000","last_change":"2024-03-21T11:21:50.658412+0000","last_active":"2024-03-21T11:21:50.658412+0000","last_peered":"2024-03-21T11:21:50.658412+0000","last_clean":"2024-03-21T11:21:50.658412+0000","last_became_active":"2024-03-21T11:21:50.649270+0000","last_became_peered":"2024-03-21T11:21:50.649270+0000","last_unstale":"2024-03-21T11:21:50.658412+0000","last_undegraded":"2024-03-21T11:21:50.658412+0000","last_fullsized":"2024-03-21T11:21:50.658412+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:11:06.006302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651571+0000","last_change":"2024-03-21T11:20:48.667212+0000","last_active":"2024-03-21T11:21:50.651571+0000","last_peered":"2024-03-21T11:21:50.651571+0000","last_clean":"2024-03-21T11:21:50.651571+0000","last_became_active":"2024-03-21T11:20:48.662839+0000","last_became_peered":"2024-03-21T11:20:48.662839+0000","last_unstale":"2024-03-21T11:21:50.651571+0000","last_undegraded":"2024-03-21T11:21:50.651571+0000","last_fullsized":"2024-03-21T11:21:50.651571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:27:41.490922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.637316+0000","last_change":"2024-03-21T11:20:48.151185+0000","last_active":"2024-03-21T11:21:50.637316+0000","last_peered":"2024-03-21T11:21:50.637316+0000","last_clean":"2024-03-21T11:21:50.637316+0000","last_became_active":"2024-03-21T11:20:48.150859+0000","last_became_peered":"2024-03-21T11:20:48.150859+0000","last_unstale":"2024-03-21T11:21:50.637316+0000","last_undegraded":"2024-03-21T11:21:50.637316+0000","last_fullsized":"2024-03-21T11:21:50.637316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:52:45.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.641662+0000","last_change":"2024-03-21T11:19:28.916140+0000","last_active":"2024-03-21T11:21:50.641662+0000","last_peered":"2024-03-21T11:21:50.641662+0000","last_clean":"2024-03-21T11:21:50.641662+0000","last_became_active":"2024-03-21T11:19:28.915634+0000","last_became_peered":"2024-03-21T11:19:28.915634+0000","last_unstale":"2024-03-21T11:21:50.641662+0000","last_undegraded":"2024-03-21T11:21:50.641662+0000","last_fullsized":"2024-03-21T11:21:50.641662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:15:35.803515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:15:35.803515+0000","last_clean_scrub_stamp":"2024-03-21T11:15:35.803515+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:08:31.509894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.656708+0000","last_change":"2024-03-21T11:21:50.656708+0000","last_active":"2024-03-21T11:21:50.656708+0000","last_peered":"2024-03-21T11:21:50.656708+0000","last_clean":"2024-03-21T11:21:50.656708+0000","last_became_active":"2024-03-21T11:21:50.656236+0000","last_became_peered":"2024-03-21T11:21:50.656236+0000","last_unstale":"2024-03-21T11:21:50.656708+0000","last_undegraded":"2024-03-21T11:21:50.656708+0000","last_fullsized":"2024-03-21T11:21:50.656708+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:17:44.114512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651700+0000","last_change":"2024-03-21T11:20:48.657878+0000","last_active":"2024-03-21T11:21:50.651700+0000","last_peered":"2024-03-21T11:21:50.651700+0000","last_clean":"2024-03-21T11:21:50.651700+0000","last_became_active":"2024-03-21T11:20:48.657212+0000","last_became_peered":"2024-03-21T11:20:48.657212+0000","last_unstale":"2024-03-21T11:21:50.651700+0000","last_undegraded":"2024-03-21T11:21:50.651700+0000","last_fullsized":"2024-03-21T11:21:50.651700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:43:11.993577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.641781+0000","last_change":"2024-03-21T11:20:48.135513+0000","last_active":"2024-03-21T11:21:50.641781+0000","last_peered":"2024-03-21T11:21:50.641781+0000","last_clean":"2024-03-21T11:21:50.641781+0000","last_became_active":"2024-03-21T11:20:48.135223+0000","last_became_peered":"2024-03-21T11:20:48.135223+0000","last_unstale":"2024-03-21T11:21:50.641781+0000","last_undegraded":"2024-03-21T11:21:50.641781+0000","last_fullsized":"2024-03-21T11:21:50.641781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:37:38.834284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640721+0000","last_change":"2024-03-21T11:20:53.751790+0000","last_active":"2024-03-21T11:21:50.640721+0000","last_peered":"2024-03-21T11:21:50.640721+0000","last_clean":"2024-03-21T11:21:50.640721+0000","last_became_active":"2024-03-21T11:20:53.751312+0000","last_became_peered":"2024-03-21T11:20:53.751312+0000","last_unstale":"2024-03-21T11:21:50.640721+0000","last_undegraded":"2024-03-21T11:21:50.640721+0000","last_fullsized":"2024-03-21T11:21:50.640721+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:21:41.599985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647236+0000","last_change":"2024-03-21T11:20:48.154119+0000","last_active":"2024-03-21T11:21:50.647236+0000","last_peered":"2024-03-21T11:21:50.647236+0000","last_clean":"2024-03-21T11:21:50.647236+0000","last_became_active":"2024-03-21T11:20:48.153781+0000","last_became_peered":"2024-03-21T11:20:48.153781+0000","last_unstale":"2024-03-21T11:21:50.647236+0000","last_undegraded":"2024-03-21T11:21:50.647236+0000","last_fullsized":"2024-03-21T11:21:50.647236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:44:13.937836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651405+0000","last_change":"2024-03-21T11:20:48.661282+0000","last_active":"2024-03-21T11:21:50.651405+0000","last_peered":"2024-03-21T11:21:50.651405+0000","last_clean":"2024-03-21T11:21:50.651405+0000","last_became_active":"2024-03-21T11:20:48.660569+0000","last_became_peered":"2024-03-21T11:20:48.660569+0000","last_unstale":"2024-03-21T11:21:50.651405+0000","last_undegraded":"2024-03-21T11:21:50.651405+0000","last_fullsized":"2024-03-21T11:21:50.651405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:14.875700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651671+0000","last_change":"2024-03-21T11:20:53.918016+0000","last_active":"2024-03-21T11:21:50.651671+0000","last_peered":"2024-03-21T11:21:50.651671+0000","last_clean":"2024-03-21T11:21:50.651671+0000","last_became_active":"2024-03-21T11:20:53.916891+0000","last_became_peered":"2024-03-21T11:20:53.916891+0000","last_unstale":"2024-03-21T11:21:50.651671+0000","last_undegraded":"2024-03-21T11:21:50.651671+0000","last_fullsized":"2024-03-21T11:21:50.651671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:40:25.206778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'12","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.643805+0000","last_change":"2024-03-21T11:20:48.167560+0000","last_active":"2024-03-21T11:21:50.643805+0000","last_peered":"2024-03-21T11:21:50.643805+0000","last_clean":"2024-03-21T11:21:50.643805+0000","last_became_active":"2024-03-21T11:20:48.164986+0000","last_became_peered":"2024-03-21T11:20:48.164986+0000","last_unstale":"2024-03-21T11:21:50.643805+0000","last_undegraded":"2024-03-21T11:21:50.643805+0000","last_fullsized":"2024-03-21T11:21:50.643805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:59.930626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.653352+0000","last_change":"2024-03-21T11:20:48.675250+0000","last_active":"2024-03-21T11:21:50.653352+0000","last_peered":"2024-03-21T11:21:50.653352+0000","last_clean":"2024-03-21T11:21:50.653352+0000","last_became_active":"2024-03-21T11:20:48.674638+0000","last_became_peered":"2024-03-21T11:20:48.674638+0000","last_unstale":"2024-03-21T11:21:50.653352+0000","last_undegraded":"2024-03-21T11:21:50.653352+0000","last_fullsized":"2024-03-21T11:21:50.653352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:46:34.842363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644270+0000","last_change":"2024-03-21T11:20:53.733809+0000","last_active":"2024-03-21T11:21:50.644270+0000","last_peered":"2024-03-21T11:21:50.644270+0000","last_clean":"2024-03-21T11:21:50.644270+0000","last_became_active":"2024-03-21T11:20:53.733393+0000","last_became_peered":"2024-03-21T11:20:53.733393+0000","last_unstale":"2024-03-21T11:21:50.644270+0000","last_undegraded":"2024-03-21T11:21:50.644270+0000","last_fullsized":"2024-03-21T11:21:50.644270+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:02:46.270716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.637327+0000","last_change":"2024-03-21T11:20:48.139475+0000","last_active":"2024-03-21T11:21:50.637327+0000","last_peered":"2024-03-21T11:21:50.637327+0000","last_clean":"2024-03-21T11:21:50.637327+0000","last_became_active":"2024-03-21T11:20:48.139249+0000","last_became_peered":"2024-03-21T11:20:48.139249+0000","last_unstale":"2024-03-21T11:21:50.637327+0000","last_undegraded":"2024-03-21T11:21:50.637327+0000","last_fullsized":"2024-03-21T11:21:50.637327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:45:18.811829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647070+0000","last_change":"2024-03-21T11:20:48.673975+0000","last_active":"2024-03-21T11:21:50.647070+0000","last_peered":"2024-03-21T11:21:50.647070+0000","last_clean":"2024-03-21T11:21:50.647070+0000","last_became_active":"2024-03-21T11:20:48.673587+0000","last_became_peered":"2024-03-21T11:20:48.673587+0000","last_unstale":"2024-03-21T11:21:50.647070+0000","last_undegraded":"2024-03-21T11:21:50.647070+0000","last_fullsized":"2024-03-21T11:21:50.647070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:08:52.354780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.636920+0000","last_change":"2024-03-21T11:20:53.917493+0000","last_active":"2024-03-21T11:21:50.636920+0000","last_peered":"2024-03-21T11:21:50.636920+0000","last_clean":"2024-03-21T11:21:50.636920+0000","last_became_active":"2024-03-21T11:20:53.917007+0000","last_became_peered":"2024-03-21T11:20:53.917007+0000","last_unstale":"2024-03-21T11:21:50.636920+0000","last_undegraded":"2024-03-21T11:21:50.636920+0000","last_fullsized":"2024-03-21T11:21:50.636920+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:39:20.280899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689227+0000","last_change":"2024-03-21T11:20:48.140914+0000","last_active":"2024-03-21T11:21:50.689227+0000","last_peered":"2024-03-21T11:21:50.689227+0000","last_clean":"2024-03-21T11:21:50.689227+0000","last_became_active":"2024-03-21T11:20:48.140558+0000","last_became_peered":"2024-03-21T11:20:48.140558+0000","last_unstale":"2024-03-21T11:21:50.689227+0000","last_undegraded":"2024-03-21T11:21:50.689227+0000","last_fullsized":"2024-03-21T11:21:50.689227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:59:52.065431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644834+0000","last_change":"2024-03-21T11:20:48.651190+0000","last_active":"2024-03-21T11:21:50.644834+0000","last_peered":"2024-03-21T11:21:50.644834+0000","last_clean":"2024-03-21T11:21:50.644834+0000","last_became_active":"2024-03-21T11:20:48.650818+0000","last_became_peered":"2024-03-21T11:20:48.650818+0000","last_unstale":"2024-03-21T11:21:50.644834+0000","last_undegraded":"2024-03-21T11:21:50.644834+0000","last_fullsized":"2024-03-21T11:21:50.644834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:47:17.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644034+0000","last_change":"2024-03-21T11:20:53.756844+0000","last_active":"2024-03-21T11:21:50.644034+0000","last_peered":"2024-03-21T11:21:50.644034+0000","last_clean":"2024-03-21T11:21:50.644034+0000","last_became_active":"2024-03-21T11:20:53.756380+0000","last_became_peered":"2024-03-21T11:20:53.756380+0000","last_unstale":"2024-03-21T11:21:50.644034+0000","last_undegraded":"2024-03-21T11:21:50.644034+0000","last_fullsized":"2024-03-21T11:21:50.644034+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:51:55.327798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.653167+0000","last_change":"2024-03-21T11:20:48.160749+0000","last_active":"2024-03-21T11:21:50.653167+0000","last_peered":"2024-03-21T11:21:50.653167+0000","last_clean":"2024-03-21T11:21:50.653167+0000","last_became_active":"2024-03-21T11:20:48.158507+0000","last_became_peered":"2024-03-21T11:20:48.158507+0000","last_unstale":"2024-03-21T11:21:50.653167+0000","last_undegraded":"2024-03-21T11:21:50.653167+0000","last_fullsized":"2024-03-21T11:21:50.653167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:55.375529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639079+0000","last_change":"2024-03-21T11:20:48.667153+0000","last_active":"2024-03-21T11:21:50.639079+0000","last_peered":"2024-03-21T11:21:50.639079+0000","last_clean":"2024-03-21T11:21:50.639079+0000","last_became_active":"2024-03-21T11:20:48.666823+0000","last_became_peered":"2024-03-21T11:20:48.666823+0000","last_unstale":"2024-03-21T11:21:50.639079+0000","last_undegraded":"2024-03-21T11:21:50.639079+0000","last_fullsized":"2024-03-21T11:21:50.639079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:43:54.183372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.637008+0000","last_change":"2024-03-21T11:20:53.739188+0000","last_active":"2024-03-21T11:21:50.637008+0000","last_peered":"2024-03-21T11:21:50.637008+0000","last_clean":"2024-03-21T11:21:50.637008+0000","last_became_active":"2024-03-21T11:20:53.738770+0000","last_became_peered":"2024-03-21T11:20:53.738770+0000","last_unstale":"2024-03-21T11:21:50.637008+0000","last_undegraded":"2024-03-21T11:21:50.637008+0000","last_fullsized":"2024-03-21T11:21:50.637008+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:53:48.897963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.668860+0000","last_change":"2024-03-21T11:21:50.668860+0000","last_active":"2024-03-21T11:21:50.668860+0000","last_peered":"2024-03-21T11:21:50.668860+0000","last_clean":"2024-03-21T11:21:50.668860+0000","last_became_active":"2024-03-21T11:21:50.655255+0000","last_became_peered":"2024-03-21T11:21:50.655255+0000","last_unstale":"2024-03-21T11:21:50.668860+0000","last_undegraded":"2024-03-21T11:21:50.668860+0000","last_fullsized":"2024-03-21T11:21:50.668860+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:27.634508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.650514+0000","last_change":"2024-03-21T11:20:48.677303+0000","last_active":"2024-03-21T11:21:50.650514+0000","last_peered":"2024-03-21T11:21:50.650514+0000","last_clean":"2024-03-21T11:21:50.650514+0000","last_became_active":"2024-03-21T11:20:48.676903+0000","last_became_peered":"2024-03-21T11:20:48.676903+0000","last_unstale":"2024-03-21T11:21:50.650514+0000","last_undegraded":"2024-03-21T11:21:50.650514+0000","last_fullsized":"2024-03-21T11:21:50.650514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:06:59.922274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.636868+0000","last_change":"2024-03-21T11:20:53.916691+0000","last_active":"2024-03-21T11:21:50.636868+0000","last_peered":"2024-03-21T11:21:50.636868+0000","last_clean":"2024-03-21T11:21:50.636868+0000","last_became_active":"2024-03-21T11:20:53.916275+0000","last_became_peered":"2024-03-21T11:20:53.916275+0000","last_unstale":"2024-03-21T11:21:50.636868+0000","last_undegraded":"2024-03-21T11:21:50.636868+0000","last_fullsized":"2024-03-21T11:21:50.636868+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:35.848589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.668467+0000","last_change":"2024-03-21T11:21:50.668467+0000","last_active":"2024-03-21T11:21:50.668467+0000","last_peered":"2024-03-21T11:21:50.668467+0000","last_clean":"2024-03-21T11:21:50.668467+0000","last_became_active":"2024-03-21T11:21:50.649598+0000","last_became_peered":"2024-03-21T11:21:50.649598+0000","last_unstale":"2024-03-21T11:21:50.668467+0000","last_undegraded":"2024-03-21T11:21:50.668467+0000","last_fullsized":"2024-03-21T11:21:50.668467+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:01:48.186997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.642157+0000","last_change":"2024-03-21T11:20:48.661486+0000","last_active":"2024-03-21T11:21:50.642157+0000","last_peered":"2024-03-21T11:21:50.642157+0000","last_clean":"2024-03-21T11:21:50.642157+0000","last_became_active":"2024-03-21T11:20:48.661109+0000","last_became_peered":"2024-03-21T11:20:48.661109+0000","last_unstale":"2024-03-21T11:21:50.642157+0000","last_undegraded":"2024-03-21T11:21:50.642157+0000","last_fullsized":"2024-03-21T11:21:50.642157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:15:27.093616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689652+0000","last_change":"2024-03-21T11:20:53.747223+0000","last_active":"2024-03-21T11:21:50.689652+0000","last_peered":"2024-03-21T11:21:50.689652+0000","last_clean":"2024-03-21T11:21:50.689652+0000","last_became_active":"2024-03-21T11:20:53.746805+0000","last_became_peered":"2024-03-21T11:20:53.746805+0000","last_unstale":"2024-03-21T11:21:50.689652+0000","last_undegraded":"2024-03-21T11:21:50.689652+0000","last_fullsized":"2024-03-21T11:21:50.689652+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:29.186745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'6","reported_seq":117,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:37.693923+0000","last_change":"2024-03-21T11:20:48.151172+0000","last_active":"2024-03-21T11:22:37.693923+0000","last_peered":"2024-03-21T11:22:37.693923+0000","last_clean":"2024-03-21T11:22:37.693923+0000","last_became_active":"2024-03-21T11:20:48.150832+0000","last_became_peered":"2024-03-21T11:20:48.150832+0000","last_unstale":"2024-03-21T11:22:37.693923+0000","last_undegraded":"2024-03-21T11:22:37.693923+0000","last_fullsized":"2024-03-21T11:22:37.693923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:34:46.367713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":83,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.643555+0000","last_change":"2024-03-21T11:20:48.652275+0000","last_active":"2024-03-21T11:21:50.643555+0000","last_peered":"2024-03-21T11:21:50.643555+0000","last_clean":"2024-03-21T11:21:50.643555+0000","last_became_active":"2024-03-21T11:20:48.651965+0000","last_became_peered":"2024-03-21T11:20:48.651965+0000","last_unstale":"2024-03-21T11:21:50.643555+0000","last_undegraded":"2024-03-21T11:21:50.643555+0000","last_fullsized":"2024-03-21T11:21:50.643555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:56:18.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647381+0000","last_change":"2024-03-21T11:20:53.748050+0000","last_active":"2024-03-21T11:21:50.647381+0000","last_peered":"2024-03-21T11:21:50.647381+0000","last_clean":"2024-03-21T11:21:50.647381+0000","last_became_active":"2024-03-21T11:20:53.747636+0000","last_became_peered":"2024-03-21T11:20:53.747636+0000","last_unstale":"2024-03-21T11:21:50.647381+0000","last_undegraded":"2024-03-21T11:21:50.647381+0000","last_fullsized":"2024-03-21T11:21:50.647381+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:09:15.202183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639671+0000","last_change":"2024-03-21T11:20:48.156928+0000","last_active":"2024-03-21T11:21:50.639671+0000","last_peered":"2024-03-21T11:21:50.639671+0000","last_clean":"2024-03-21T11:21:50.639671+0000","last_became_active":"2024-03-21T11:20:48.156625+0000","last_became_peered":"2024-03-21T11:20:48.156625+0000","last_unstale":"2024-03-21T11:21:50.639671+0000","last_undegraded":"2024-03-21T11:21:50.639671+0000","last_fullsized":"2024-03-21T11:21:50.639671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:19:42.035565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.642846+0000","last_change":"2024-03-21T11:20:48.663655+0000","last_active":"2024-03-21T11:21:50.642846+0000","last_peered":"2024-03-21T11:21:50.642846+0000","last_clean":"2024-03-21T11:21:50.642846+0000","last_became_active":"2024-03-21T11:20:48.663336+0000","last_became_peered":"2024-03-21T11:20:48.663336+0000","last_unstale":"2024-03-21T11:21:50.642846+0000","last_undegraded":"2024-03-21T11:21:50.642846+0000","last_fullsized":"2024-03-21T11:21:50.642846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:29:05.119283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.638869+0000","last_change":"2024-03-21T11:20:53.732584+0000","last_active":"2024-03-21T11:21:50.638869+0000","last_peered":"2024-03-21T11:21:50.638869+0000","last_clean":"2024-03-21T11:21:50.638869+0000","last_became_active":"2024-03-21T11:20:53.731975+0000","last_became_peered":"2024-03-21T11:20:53.731975+0000","last_unstale":"2024-03-21T11:21:50.638869+0000","last_undegraded":"2024-03-21T11:21:50.638869+0000","last_fullsized":"2024-03-21T11:21:50.638869+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:51:06.336241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651482+0000","last_change":"2024-03-21T11:20:48.165594+0000","last_active":"2024-03-21T11:21:50.651482+0000","last_peered":"2024-03-21T11:21:50.651482+0000","last_clean":"2024-03-21T11:21:50.651482+0000","last_became_active":"2024-03-21T11:20:48.164394+0000","last_became_peered":"2024-03-21T11:20:48.164394+0000","last_unstale":"2024-03-21T11:21:50.651482+0000","last_undegraded":"2024-03-21T11:21:50.651482+0000","last_fullsized":"2024-03-21T11:21:50.651482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:13.317767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.643170+0000","last_change":"2024-03-21T11:20:48.649359+0000","last_active":"2024-03-21T11:21:50.643170+0000","last_peered":"2024-03-21T11:21:50.643170+0000","last_clean":"2024-03-21T11:21:50.643170+0000","last_became_active":"2024-03-21T11:20:48.649031+0000","last_became_peered":"2024-03-21T11:20:48.649031+0000","last_unstale":"2024-03-21T11:21:50.643170+0000","last_undegraded":"2024-03-21T11:21:50.643170+0000","last_fullsized":"2024-03-21T11:21:50.643170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:17.055732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639185+0000","last_change":"2024-03-21T11:20:48.654999+0000","last_active":"2024-03-21T11:21:50.639185+0000","last_peered":"2024-03-21T11:21:50.639185+0000","last_clean":"2024-03-21T11:21:50.639185+0000","last_became_active":"2024-03-21T11:20:48.654638+0000","last_became_peered":"2024-03-21T11:20:48.654638+0000","last_unstale":"2024-03-21T11:21:50.639185+0000","last_undegraded":"2024-03-21T11:21:50.639185+0000","last_fullsized":"2024-03-21T11:21:50.639185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:49:48.711420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644656+0000","last_change":"2024-03-21T11:20:48.651864+0000","last_active":"2024-03-21T11:21:50.644656+0000","last_peered":"2024-03-21T11:21:50.644656+0000","last_clean":"2024-03-21T11:21:50.644656+0000","last_became_active":"2024-03-21T11:20:48.651565+0000","last_became_peered":"2024-03-21T11:20:48.651565+0000","last_unstale":"2024-03-21T11:21:50.644656+0000","last_undegraded":"2024-03-21T11:21:50.644656+0000","last_fullsized":"2024-03-21T11:21:50.644656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:58.659889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645659+0000","last_change":"2024-03-21T11:20:48.667038+0000","last_active":"2024-03-21T11:21:50.645659+0000","last_peered":"2024-03-21T11:21:50.645659+0000","last_clean":"2024-03-21T11:21:50.645659+0000","last_became_active":"2024-03-21T11:20:48.666570+0000","last_became_peered":"2024-03-21T11:20:48.666570+0000","last_unstale":"2024-03-21T11:21:50.645659+0000","last_undegraded":"2024-03-21T11:21:50.645659+0000","last_fullsized":"2024-03-21T11:21:50.645659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:36:39.043697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.658024+0000","last_change":"2024-03-21T11:21:50.658024+0000","last_active":"2024-03-21T11:21:50.658024+0000","last_peered":"2024-03-21T11:21:50.658024+0000","last_clean":"2024-03-21T11:21:50.658024+0000","last_became_active":"2024-03-21T11:21:50.657390+0000","last_became_peered":"2024-03-21T11:21:50.657390+0000","last_unstale":"2024-03-21T11:21:50.658024+0000","last_undegraded":"2024-03-21T11:21:50.658024+0000","last_fullsized":"2024-03-21T11:21:50.658024+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:29.578366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651546+0000","last_change":"2024-03-21T11:20:48.670139+0000","last_active":"2024-03-21T11:21:50.651546+0000","last_peered":"2024-03-21T11:21:50.651546+0000","last_clean":"2024-03-21T11:21:50.651546+0000","last_became_active":"2024-03-21T11:20:48.669410+0000","last_became_peered":"2024-03-21T11:20:48.669410+0000","last_unstale":"2024-03-21T11:21:50.651546+0000","last_undegraded":"2024-03-21T11:21:50.651546+0000","last_fullsized":"2024-03-21T11:21:50.651546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:35:18.799840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640617+0000","last_change":"2024-03-21T11:20:48.655356+0000","last_active":"2024-03-21T11:21:50.640617+0000","last_peered":"2024-03-21T11:21:50.640617+0000","last_clean":"2024-03-21T11:21:50.640617+0000","last_became_active":"2024-03-21T11:20:48.655028+0000","last_became_peered":"2024-03-21T11:20:48.655028+0000","last_unstale":"2024-03-21T11:21:50.640617+0000","last_undegraded":"2024-03-21T11:21:50.640617+0000","last_fullsized":"2024-03-21T11:21:50.640617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:46:39.948982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645395+0000","last_change":"2024-03-21T11:20:48.648840+0000","last_active":"2024-03-21T11:21:50.645395+0000","last_peered":"2024-03-21T11:21:50.645395+0000","last_clean":"2024-03-21T11:21:50.645395+0000","last_became_active":"2024-03-21T11:20:48.648502+0000","last_became_peered":"2024-03-21T11:20:48.648502+0000","last_unstale":"2024-03-21T11:21:50.645395+0000","last_undegraded":"2024-03-21T11:21:50.645395+0000","last_fullsized":"2024-03-21T11:21:50.645395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:33:14.824880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.653563+0000","last_change":"2024-03-21T11:20:48.672015+0000","last_active":"2024-03-21T11:21:50.653563+0000","last_peered":"2024-03-21T11:21:50.653563+0000","last_clean":"2024-03-21T11:21:50.653563+0000","last_became_active":"2024-03-21T11:20:48.671663+0000","last_became_peered":"2024-03-21T11:20:48.671663+0000","last_unstale":"2024-03-21T11:21:50.653563+0000","last_undegraded":"2024-03-21T11:21:50.653563+0000","last_fullsized":"2024-03-21T11:21:50.653563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:21:25.328698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639265+0000","last_change":"2024-03-21T11:20:48.664114+0000","last_active":"2024-03-21T11:21:50.639265+0000","last_peered":"2024-03-21T11:21:50.639265+0000","last_clean":"2024-03-21T11:21:50.639265+0000","last_became_active":"2024-03-21T11:20:48.663819+0000","last_became_peered":"2024-03-21T11:20:48.663819+0000","last_unstale":"2024-03-21T11:21:50.639265+0000","last_undegraded":"2024-03-21T11:21:50.639265+0000","last_fullsized":"2024-03-21T11:21:50.639265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:08:08.201383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.653508+0000","last_change":"2024-03-21T11:20:48.671336+0000","last_active":"2024-03-21T11:21:50.653508+0000","last_peered":"2024-03-21T11:21:50.653508+0000","last_clean":"2024-03-21T11:21:50.653508+0000","last_became_active":"2024-03-21T11:20:48.670918+0000","last_became_peered":"2024-03-21T11:20:48.670918+0000","last_unstale":"2024-03-21T11:21:50.653508+0000","last_undegraded":"2024-03-21T11:21:50.653508+0000","last_fullsized":"2024-03-21T11:21:50.653508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:01:44.764584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689840+0000","last_change":"2024-03-21T11:20:48.659339+0000","last_active":"2024-03-21T11:21:50.689840+0000","last_peered":"2024-03-21T11:21:50.689840+0000","last_clean":"2024-03-21T11:21:50.689840+0000","last_became_active":"2024-03-21T11:20:48.658989+0000","last_became_peered":"2024-03-21T11:20:48.658989+0000","last_unstale":"2024-03-21T11:21:50.689840+0000","last_undegraded":"2024-03-21T11:21:50.689840+0000","last_fullsized":"2024-03-21T11:21:50.689840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:44:09.599836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.653931+0000","last_change":"2024-03-21T11:20:48.668679+0000","last_active":"2024-03-21T11:21:50.653931+0000","last_peered":"2024-03-21T11:21:50.653931+0000","last_clean":"2024-03-21T11:21:50.653931+0000","last_became_active":"2024-03-21T11:20:48.667956+0000","last_became_peered":"2024-03-21T11:20:48.667956+0000","last_unstale":"2024-03-21T11:21:50.653931+0000","last_undegraded":"2024-03-21T11:21:50.653931+0000","last_fullsized":"2024-03-21T11:21:50.653931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:04.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.650600+0000","last_change":"2024-03-21T11:21:50.650600+0000","last_active":"2024-03-21T11:21:50.650600+0000","last_peered":"2024-03-21T11:21:50.650600+0000","last_clean":"2024-03-21T11:21:50.650600+0000","last_became_active":"2024-03-21T11:21:50.650221+0000","last_became_peered":"2024-03-21T11:21:50.650221+0000","last_unstale":"2024-03-21T11:21:50.650600+0000","last_undegraded":"2024-03-21T11:21:50.650600+0000","last_fullsized":"2024-03-21T11:21:50.650600+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:29:52.245281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247498,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71801,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":824,"num_read_kb":486,"num_write":243,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":215403,"data_compressed":5487,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":513062,"data_compressed":20821,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1175552,"data_stored":1168180,"data_compressed":58711,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":516697,"data_compressed":20821,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":512979,"data_compressed":20821,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":1109867,"data_compressed":56882,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462977,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":508105,"data_compressed":20821,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":573341,"data_compressed":22650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":509952,"data_compressed":20821,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":574343,"data_compressed":22650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":507813,"data_compressed":20821,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888139,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":512570,"data_compressed":20821,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1100962,"data_compressed":56882,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65958,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66960,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":66333,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T11:22:41.398 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T11:22:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:42.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:41 smithi047 ceph-mon[31351]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:22:42.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:41 smithi100 ceph-mon[36245]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:22:42.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:41 smithi102 ceph-mon[36748]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:22:43.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:42 smithi047 ceph-mon[31351]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:22:43.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:42 smithi100 ceph-mon[36245]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:22:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:42 smithi102 ceph-mon[36748]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:22:44.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.100+0000 7f540d400700 1 -- 172.21.15.47:0/1312658883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5408101090 msgr2=0x7f54081014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:44.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.100+0000 7f540d400700 1 --2- 172.21.15.47:0/1312658883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5408101090 0x7f54081014f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009990 tx=0x7f53f0009ca0 comp rx=0 tx=0).stop 2024-03-21T11:22:44.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.100+0000 7f540d400700 1 -- 172.21.15.47:0/1312658883 shutdown_connections 2024-03-21T11:22:44.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.100+0000 7f540d400700 1 --2- 172.21.15.47:0/1312658883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5408101a30 0x7f540810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.100+0000 7f540d400700 1 --2- 172.21.15.47:0/1312658883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54081070d0 0x7f54081074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.100+0000 7f540d400700 1 --2- 172.21.15.47:0/1312658883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5408101090 0x7f54081014f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.100+0000 7f540d400700 1 -- 172.21.15.47:0/1312658883 >> 172.21.15.47:0/1312658883 conn(0x7f5408077fc0 msgr2=0x7f54080783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:44.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.100+0000 7f540d400700 1 -- 172.21.15.47:0/1312658883 shutdown_connections 2024-03-21T11:22:44.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.100+0000 7f540d400700 1 -- 172.21.15.47:0/1312658883 wait complete. 2024-03-21T11:22:44.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.101+0000 7f540d400700 1 Processor -- start 2024-03-21T11:22:44.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.101+0000 7f540d400700 1 -- start start 2024-03-21T11:22:44.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.102+0000 7f540d400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5408101090 0x7f5408102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:44.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.102+0000 7f540d400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5408101a30 0x7f5408103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.102+0000 7f540d400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54081070d0 0x7f54081037c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.102+0000 7f540d400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5408111380 con 0x7f5408101090 2024-03-21T11:22:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.102+0000 7f540d400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5408111200 con 0x7f5408101a30 2024-03-21T11:22:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.102+0000 7f540d400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5408111500 con 0x7f54081070d0 2024-03-21T11:22:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.102+0000 7f53fde00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5408101a30 0x7f5408103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.102+0000 7f5406800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5408101090 0x7f5408102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:44.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.102+0000 7f53fde00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5408101a30 0x7f5408103280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35400/0 (socket says 172.21.15.47:35400) 2024-03-21T11:22:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.102+0000 7f53fde00700 1 -- 172.21.15.47:0/1380644978 learned_addr learned my addr 172.21.15.47:0/1380644978 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.102+0000 7f5407200700 1 --2- 172.21.15.47:0/1380644978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54081070d0 0x7f54081037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.103+0000 7f53fde00700 1 -- 172.21.15.47:0/1380644978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54081070d0 msgr2=0x7f54081037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.103+0000 7f53fde00700 1 --2- 172.21.15.47:0/1380644978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54081070d0 0x7f54081037c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.103+0000 7f53fde00700 1 -- 172.21.15.47:0/1380644978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5408101090 msgr2=0x7f5408102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.103+0000 7f53fde00700 1 --2- 172.21.15.47:0/1380644978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5408101090 0x7f5408102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.103+0000 7f53fde00700 1 -- 172.21.15.47:0/1380644978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54081a6e60 con 0x7f5408101a30 2024-03-21T11:22:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.103+0000 7f53fde00700 1 --2- 172.21.15.47:0/1380644978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5408101a30 0x7f5408103280 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f53f0005910 tx=0x7f53f000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.103+0000 7f53ffe00700 1 -- 172.21.15.47:0/1380644978 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0027070 con 0x7f5408101a30 2024-03-21T11:22:44.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.103+0000 7f53ffe00700 1 -- 172.21.15.47:0/1380644978 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53f0003d70 con 0x7f5408101a30 2024-03-21T11:22:44.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.103+0000 7f53ffe00700 1 -- 172.21.15.47:0/1380644978 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f000f460 con 0x7f5408101a30 2024-03-21T11:22:44.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.103+0000 7f540d400700 1 -- 172.21.15.47:0/1380644978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54081a70f0 con 0x7f5408101a30 2024-03-21T11:22:44.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.103+0000 7f540d400700 1 -- 172.21.15.47:0/1380644978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54081a75a0 con 0x7f5408101a30 2024-03-21T11:22:44.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.104+0000 7f540d400700 1 -- 172.21.15.47:0/1380644978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5408068e50 con 0x7f5408101a30 2024-03-21T11:22:44.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.106+0000 7f53ffe00700 1 -- 172.21.15.47:0/1380644978 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f53f002a070 con 0x7f5408101a30 2024-03-21T11:22:44.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.107+0000 7f53ffe00700 1 --2- 172.21.15.47:0/1380644978 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53ec0a30c0 0x7f53ec0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:44.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.108+0000 7f53ffe00700 1 -- 172.21.15.47:0/1380644978 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f53f00d06f0 con 0x7f5408101a30 2024-03-21T11:22:44.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.108+0000 7f5406800700 1 --2- 172.21.15.47:0/1380644978 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53ec0a30c0 0x7f53ec0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:44.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.108+0000 7f5406800700 1 --2- 172.21.15.47:0/1380644978 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53ec0a30c0 0x7f53ec0a5580 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009550 tx=0x7f53f80091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:44.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.116+0000 7f53ffe00700 1 -- 172.21.15.47:0/1380644978 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f53f005e740 con 0x7f5408101a30 2024-03-21T11:22:44.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.344+0000 7f540d400700 1 -- 172.21.15.47:0/1380644978 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5408104e80 con 0x7f53ec0a30c0 2024-03-21T11:22:44.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.373+0000 7f53ffe00700 1 -- 172.21.15.47:0/1380644978 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340720 (secure 0 0 0) 0x7f5408104e80 con 0x7f53ec0a30c0 2024-03-21T11:22:44.378 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:22:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.378+0000 7f540d400700 1 -- 172.21.15.47:0/1380644978 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53ec0a30c0 msgr2=0x7f53ec0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.379+0000 7f540d400700 1 --2- 172.21.15.47:0/1380644978 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53ec0a30c0 0x7f53ec0a5580 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009550 tx=0x7f53f80091b0 comp rx=0 tx=0).stop 2024-03-21T11:22:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.379+0000 7f540d400700 1 -- 172.21.15.47:0/1380644978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5408101a30 msgr2=0x7f5408103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.379+0000 7f540d400700 1 --2- 172.21.15.47:0/1380644978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5408101a30 0x7f5408103280 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f53f0005910 tx=0x7f53f000bab0 comp rx=0 tx=0).stop 2024-03-21T11:22:44.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.379+0000 7f540d400700 1 -- 172.21.15.47:0/1380644978 shutdown_connections 2024-03-21T11:22:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.379+0000 7f540d400700 1 --2- 172.21.15.47:0/1380644978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54081070d0 0x7f54081037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.379+0000 7f540d400700 1 --2- 172.21.15.47:0/1380644978 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53ec0a30c0 0x7f53ec0a5580 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.379+0000 7f540d400700 1 --2- 172.21.15.47:0/1380644978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5408101090 0x7f5408102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.379+0000 7f540d400700 1 --2- 172.21.15.47:0/1380644978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5408101a30 0x7f5408103280 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.379+0000 7f540d400700 1 -- 172.21.15.47:0/1380644978 >> 172.21.15.47:0/1380644978 conn(0x7f5408077fc0 msgr2=0x7f5408100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.379+0000 7f540d400700 1 -- 172.21.15.47:0/1380644978 shutdown_connections 2024-03-21T11:22:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:44.379+0000 7f540d400700 1 -- 172.21.15.47:0/1380644978 wait complete. 2024-03-21T11:22:44.382 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-21T11:22:45.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:44 smithi047 ceph-mon[31351]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:22:45.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:44 smithi100 ceph-mon[36245]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:22:45.198 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-03-21T11:22:43.032766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662403,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":995,"num_read_kb":620,"num_write":426,"num_write_kb":3083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374260,"kb_used_data":12536,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567564,"statfs":{"total":1151940427776,"available":1151557185536,"internally_reserved":0,"allocated":12836864,"data_stored":8107871,"data_compressed":363522,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005590"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640403+0000","last_change":"2024-03-21T11:20:48.667185+0000","last_active":"2024-03-21T11:21:50.640403+0000","last_peered":"2024-03-21T11:21:50.640403+0000","last_clean":"2024-03-21T11:21:50.640403+0000","last_became_active":"2024-03-21T11:20:48.666855+0000","last_became_peered":"2024-03-21T11:20:48.666855+0000","last_unstale":"2024-03-21T11:21:50.640403+0000","last_undegraded":"2024-03-21T11:21:50.640403+0000","last_fullsized":"2024-03-21T11:21:50.640403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:30:37.393096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.646777+0000","last_change":"2024-03-21T11:20:48.671613+0000","last_active":"2024-03-21T11:21:50.646777+0000","last_peered":"2024-03-21T11:21:50.646777+0000","last_clean":"2024-03-21T11:21:50.646777+0000","last_became_active":"2024-03-21T11:20:48.671211+0000","last_became_peered":"2024-03-21T11:20:48.671211+0000","last_unstale":"2024-03-21T11:21:50.646777+0000","last_undegraded":"2024-03-21T11:21:50.646777+0000","last_fullsized":"2024-03-21T11:21:50.646777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:20.167922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647345+0000","last_change":"2024-03-21T11:20:48.655734+0000","last_active":"2024-03-21T11:21:50.647345+0000","last_peered":"2024-03-21T11:21:50.647345+0000","last_clean":"2024-03-21T11:21:50.647345+0000","last_became_active":"2024-03-21T11:20:48.653798+0000","last_became_peered":"2024-03-21T11:20:48.653798+0000","last_unstale":"2024-03-21T11:21:50.647345+0000","last_undegraded":"2024-03-21T11:21:50.647345+0000","last_fullsized":"2024-03-21T11:21:50.647345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:44.796405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.637299+0000","last_change":"2024-03-21T11:20:48.648784+0000","last_active":"2024-03-21T11:21:50.637299+0000","last_peered":"2024-03-21T11:21:50.637299+0000","last_clean":"2024-03-21T11:21:50.637299+0000","last_became_active":"2024-03-21T11:20:48.648040+0000","last_became_peered":"2024-03-21T11:20:48.648040+0000","last_unstale":"2024-03-21T11:21:50.637299+0000","last_undegraded":"2024-03-21T11:21:50.637299+0000","last_fullsized":"2024-03-21T11:21:50.637299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:48:51.786346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645770+0000","last_change":"2024-03-21T11:20:48.672084+0000","last_active":"2024-03-21T11:21:50.645770+0000","last_peered":"2024-03-21T11:21:50.645770+0000","last_clean":"2024-03-21T11:21:50.645770+0000","last_became_active":"2024-03-21T11:20:48.671441+0000","last_became_peered":"2024-03-21T11:20:48.671441+0000","last_unstale":"2024-03-21T11:21:50.645770+0000","last_undegraded":"2024-03-21T11:21:50.645770+0000","last_fullsized":"2024-03-21T11:21:50.645770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:20:17.638270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644602+0000","last_change":"2024-03-21T11:20:48.659782+0000","last_active":"2024-03-21T11:21:50.644602+0000","last_peered":"2024-03-21T11:21:50.644602+0000","last_clean":"2024-03-21T11:21:50.644602+0000","last_became_active":"2024-03-21T11:20:48.659460+0000","last_became_peered":"2024-03-21T11:20:48.659460+0000","last_unstale":"2024-03-21T11:21:50.644602+0000","last_undegraded":"2024-03-21T11:21:50.644602+0000","last_fullsized":"2024-03-21T11:21:50.644602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:57.712654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.641814+0000","last_change":"2024-03-21T11:20:48.659401+0000","last_active":"2024-03-21T11:21:50.641814+0000","last_peered":"2024-03-21T11:21:50.641814+0000","last_clean":"2024-03-21T11:21:50.641814+0000","last_became_active":"2024-03-21T11:20:48.659075+0000","last_became_peered":"2024-03-21T11:20:48.659075+0000","last_unstale":"2024-03-21T11:21:50.641814+0000","last_undegraded":"2024-03-21T11:21:50.641814+0000","last_fullsized":"2024-03-21T11:21:50.641814+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:54.342236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651418+0000","last_change":"2024-03-21T11:20:48.667175+0000","last_active":"2024-03-21T11:21:50.651418+0000","last_peered":"2024-03-21T11:21:50.651418+0000","last_clean":"2024-03-21T11:21:50.651418+0000","last_became_active":"2024-03-21T11:20:48.666756+0000","last_became_peered":"2024-03-21T11:20:48.666756+0000","last_unstale":"2024-03-21T11:21:50.651418+0000","last_undegraded":"2024-03-21T11:21:50.651418+0000","last_fullsized":"2024-03-21T11:21:50.651418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:25:39.694699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689988+0000","last_change":"2024-03-21T11:20:48.650584+0000","last_active":"2024-03-21T11:21:50.689988+0000","last_peered":"2024-03-21T11:21:50.689988+0000","last_clean":"2024-03-21T11:21:50.689988+0000","last_became_active":"2024-03-21T11:20:48.650251+0000","last_became_peered":"2024-03-21T11:20:48.650251+0000","last_unstale":"2024-03-21T11:21:50.689988+0000","last_undegraded":"2024-03-21T11:21:50.689988+0000","last_fullsized":"2024-03-21T11:21:50.689988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:35:59.000522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645180+0000","last_change":"2024-03-21T11:20:48.652945+0000","last_active":"2024-03-21T11:21:50.645180+0000","last_peered":"2024-03-21T11:21:50.645180+0000","last_clean":"2024-03-21T11:21:50.645180+0000","last_became_active":"2024-03-21T11:20:48.652626+0000","last_became_peered":"2024-03-21T11:20:48.652626+0000","last_unstale":"2024-03-21T11:21:50.645180+0000","last_undegraded":"2024-03-21T11:21:50.645180+0000","last_fullsized":"2024-03-21T11:21:50.645180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:43.586907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.643473+0000","last_change":"2024-03-21T11:20:48.651520+0000","last_active":"2024-03-21T11:21:50.643473+0000","last_peered":"2024-03-21T11:21:50.643473+0000","last_clean":"2024-03-21T11:21:50.643473+0000","last_became_active":"2024-03-21T11:20:48.651210+0000","last_became_peered":"2024-03-21T11:20:48.651210+0000","last_unstale":"2024-03-21T11:21:50.643473+0000","last_undegraded":"2024-03-21T11:21:50.643473+0000","last_fullsized":"2024-03-21T11:21:50.643473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:29.506732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.637181+0000","last_change":"2024-03-21T11:20:48.649844+0000","last_active":"2024-03-21T11:21:50.637181+0000","last_peered":"2024-03-21T11:21:50.637181+0000","last_clean":"2024-03-21T11:21:50.637181+0000","last_became_active":"2024-03-21T11:20:48.649503+0000","last_became_peered":"2024-03-21T11:20:48.649503+0000","last_unstale":"2024-03-21T11:21:50.637181+0000","last_undegraded":"2024-03-21T11:21:50.637181+0000","last_fullsized":"2024-03-21T11:21:50.637181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:31:55.734583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.654174+0000","last_change":"2024-03-21T11:20:48.662012+0000","last_active":"2024-03-21T11:21:50.654174+0000","last_peered":"2024-03-21T11:21:50.654174+0000","last_clean":"2024-03-21T11:21:50.654174+0000","last_became_active":"2024-03-21T11:20:48.661444+0000","last_became_peered":"2024-03-21T11:20:48.661444+0000","last_unstale":"2024-03-21T11:21:50.654174+0000","last_undegraded":"2024-03-21T11:21:50.654174+0000","last_fullsized":"2024-03-21T11:21:50.654174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:52.493244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644560+0000","last_change":"2024-03-21T11:20:48.663087+0000","last_active":"2024-03-21T11:21:50.644560+0000","last_peered":"2024-03-21T11:21:50.644560+0000","last_clean":"2024-03-21T11:21:50.644560+0000","last_became_active":"2024-03-21T11:20:48.662039+0000","last_became_peered":"2024-03-21T11:20:48.662039+0000","last_unstale":"2024-03-21T11:21:50.644560+0000","last_undegraded":"2024-03-21T11:21:50.644560+0000","last_fullsized":"2024-03-21T11:21:50.644560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:47:01.962314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645475+0000","last_change":"2024-03-21T11:20:48.663478+0000","last_active":"2024-03-21T11:21:50.645475+0000","last_peered":"2024-03-21T11:21:50.645475+0000","last_clean":"2024-03-21T11:21:50.645475+0000","last_became_active":"2024-03-21T11:20:48.663135+0000","last_became_peered":"2024-03-21T11:20:48.663135+0000","last_unstale":"2024-03-21T11:21:50.645475+0000","last_undegraded":"2024-03-21T11:21:50.645475+0000","last_fullsized":"2024-03-21T11:21:50.645475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:29.139760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689199+0000","last_change":"2024-03-21T11:20:48.653893+0000","last_active":"2024-03-21T11:21:50.689199+0000","last_peered":"2024-03-21T11:21:50.689199+0000","last_clean":"2024-03-21T11:21:50.689199+0000","last_became_active":"2024-03-21T11:20:48.653578+0000","last_became_peered":"2024-03-21T11:20:48.653578+0000","last_unstale":"2024-03-21T11:21:50.689199+0000","last_undegraded":"2024-03-21T11:21:50.689199+0000","last_fullsized":"2024-03-21T11:21:50.689199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:54:31.254030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640175+0000","last_change":"2024-03-21T11:20:48.667388+0000","last_active":"2024-03-21T11:21:50.640175+0000","last_peered":"2024-03-21T11:21:50.640175+0000","last_clean":"2024-03-21T11:21:50.640175+0000","last_became_active":"2024-03-21T11:20:48.667077+0000","last_became_peered":"2024-03-21T11:20:48.667077+0000","last_unstale":"2024-03-21T11:21:50.640175+0000","last_undegraded":"2024-03-21T11:21:50.640175+0000","last_fullsized":"2024-03-21T11:21:50.640175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:13:32.093457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.646546+0000","last_change":"2024-03-21T11:20:48.658479+0000","last_active":"2024-03-21T11:21:50.646546+0000","last_peered":"2024-03-21T11:21:50.646546+0000","last_clean":"2024-03-21T11:21:50.646546+0000","last_became_active":"2024-03-21T11:20:48.658011+0000","last_became_peered":"2024-03-21T11:20:48.658011+0000","last_unstale":"2024-03-21T11:21:50.646546+0000","last_undegraded":"2024-03-21T11:21:50.646546+0000","last_fullsized":"2024-03-21T11:21:50.646546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:33.313537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.646244+0000","last_change":"2024-03-21T11:20:48.668648+0000","last_active":"2024-03-21T11:21:50.646244+0000","last_peered":"2024-03-21T11:21:50.646244+0000","last_clean":"2024-03-21T11:21:50.646244+0000","last_became_active":"2024-03-21T11:20:48.668376+0000","last_became_peered":"2024-03-21T11:20:48.668376+0000","last_unstale":"2024-03-21T11:21:50.646244+0000","last_undegraded":"2024-03-21T11:21:50.646244+0000","last_fullsized":"2024-03-21T11:21:50.646244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:49:47.601724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644436+0000","last_change":"2024-03-21T11:20:53.755600+0000","last_active":"2024-03-21T11:21:50.644436+0000","last_peered":"2024-03-21T11:21:50.644436+0000","last_clean":"2024-03-21T11:21:50.644436+0000","last_became_active":"2024-03-21T11:20:53.755321+0000","last_became_peered":"2024-03-21T11:20:53.755321+0000","last_unstale":"2024-03-21T11:21:50.644436+0000","last_undegraded":"2024-03-21T11:21:50.644436+0000","last_fullsized":"2024-03-21T11:21:50.644436+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:04.905910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":127,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:40.782207+0000","last_change":"2024-03-21T11:20:48.161001+0000","last_active":"2024-03-21T11:22:40.782207+0000","last_peered":"2024-03-21T11:22:40.782207+0000","last_clean":"2024-03-21T11:22:40.782207+0000","last_became_active":"2024-03-21T11:20:48.153053+0000","last_became_peered":"2024-03-21T11:20:48.153053+0000","last_unstale":"2024-03-21T11:22:40.782207+0000","last_undegraded":"2024-03-21T11:22:40.782207+0000","last_fullsized":"2024-03-21T11:22:40.782207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:26.010581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.652181+0000","last_change":"2024-03-21T11:20:48.671430+0000","last_active":"2024-03-21T11:21:50.652181+0000","last_peered":"2024-03-21T11:21:50.652181+0000","last_clean":"2024-03-21T11:21:50.652181+0000","last_became_active":"2024-03-21T11:20:48.670192+0000","last_became_peered":"2024-03-21T11:20:48.670192+0000","last_unstale":"2024-03-21T11:21:50.652181+0000","last_undegraded":"2024-03-21T11:21:50.652181+0000","last_fullsized":"2024-03-21T11:21:50.652181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:00:53.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.636538+0000","last_change":"2024-03-21T11:20:53.750313+0000","last_active":"2024-03-21T11:21:50.636538+0000","last_peered":"2024-03-21T11:21:50.636538+0000","last_clean":"2024-03-21T11:21:50.636538+0000","last_became_active":"2024-03-21T11:20:53.749913+0000","last_became_peered":"2024-03-21T11:20:53.749913+0000","last_unstale":"2024-03-21T11:21:50.636538+0000","last_undegraded":"2024-03-21T11:21:50.636538+0000","last_fullsized":"2024-03-21T11:21:50.636538+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:39:09.007674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.652592+0000","last_change":"2024-03-21T11:20:48.159934+0000","last_active":"2024-03-21T11:21:50.652592+0000","last_peered":"2024-03-21T11:21:50.652592+0000","last_clean":"2024-03-21T11:21:50.652592+0000","last_became_active":"2024-03-21T11:20:48.159650+0000","last_became_peered":"2024-03-21T11:20:48.159650+0000","last_unstale":"2024-03-21T11:21:50.652592+0000","last_undegraded":"2024-03-21T11:21:50.652592+0000","last_fullsized":"2024-03-21T11:21:50.652592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:49:56.891733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647823+0000","last_change":"2024-03-21T11:20:48.673012+0000","last_active":"2024-03-21T11:21:50.647823+0000","last_peered":"2024-03-21T11:21:50.647823+0000","last_clean":"2024-03-21T11:21:50.647823+0000","last_became_active":"2024-03-21T11:20:48.672584+0000","last_became_peered":"2024-03-21T11:20:48.672584+0000","last_unstale":"2024-03-21T11:21:50.647823+0000","last_undegraded":"2024-03-21T11:21:50.647823+0000","last_fullsized":"2024-03-21T11:21:50.647823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:48.213569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.655957+0000","last_change":"2024-03-21T11:21:50.655957+0000","last_active":"2024-03-21T11:21:50.655957+0000","last_peered":"2024-03-21T11:21:50.655957+0000","last_clean":"2024-03-21T11:21:50.655957+0000","last_became_active":"2024-03-21T11:21:50.655477+0000","last_became_peered":"2024-03-21T11:21:50.655477+0000","last_unstale":"2024-03-21T11:21:50.655957+0000","last_undegraded":"2024-03-21T11:21:50.655957+0000","last_fullsized":"2024-03-21T11:21:50.655957+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:58:28.476186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.664575+0000","last_change":"2024-03-21T11:21:50.664575+0000","last_active":"2024-03-21T11:21:50.664575+0000","last_peered":"2024-03-21T11:21:50.664575+0000","last_clean":"2024-03-21T11:21:50.664575+0000","last_became_active":"2024-03-21T11:21:50.647335+0000","last_became_peered":"2024-03-21T11:21:50.647335+0000","last_unstale":"2024-03-21T11:21:50.664575+0000","last_undegraded":"2024-03-21T11:21:50.664575+0000","last_fullsized":"2024-03-21T11:21:50.664575+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:50:28.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689390+0000","last_change":"2024-03-21T11:20:48.662392+0000","last_active":"2024-03-21T11:21:50.689390+0000","last_peered":"2024-03-21T11:21:50.689390+0000","last_clean":"2024-03-21T11:21:50.689390+0000","last_became_active":"2024-03-21T11:20:48.662042+0000","last_became_peered":"2024-03-21T11:20:48.662042+0000","last_unstale":"2024-03-21T11:21:50.689390+0000","last_undegraded":"2024-03-21T11:21:50.689390+0000","last_fullsized":"2024-03-21T11:21:50.689390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:59:34.456582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.636723+0000","last_change":"2024-03-21T11:20:53.749693+0000","last_active":"2024-03-21T11:21:50.636723+0000","last_peered":"2024-03-21T11:21:50.636723+0000","last_clean":"2024-03-21T11:21:50.636723+0000","last_became_active":"2024-03-21T11:20:53.749438+0000","last_became_peered":"2024-03-21T11:20:53.749438+0000","last_unstale":"2024-03-21T11:21:50.636723+0000","last_undegraded":"2024-03-21T11:21:50.636723+0000","last_fullsized":"2024-03-21T11:21:50.636723+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:37.714471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647072+0000","last_change":"2024-03-21T11:20:48.132587+0000","last_active":"2024-03-21T11:21:50.647072+0000","last_peered":"2024-03-21T11:21:50.647072+0000","last_clean":"2024-03-21T11:21:50.647072+0000","last_became_active":"2024-03-21T11:20:48.132232+0000","last_became_peered":"2024-03-21T11:20:48.132232+0000","last_unstale":"2024-03-21T11:21:50.647072+0000","last_undegraded":"2024-03-21T11:21:50.647072+0000","last_fullsized":"2024-03-21T11:21:50.647072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:20:45.217545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639574+0000","last_change":"2024-03-21T11:20:48.656881+0000","last_active":"2024-03-21T11:21:50.639574+0000","last_peered":"2024-03-21T11:21:50.639574+0000","last_clean":"2024-03-21T11:21:50.639574+0000","last_became_active":"2024-03-21T11:20:48.656426+0000","last_became_peered":"2024-03-21T11:20:48.656426+0000","last_unstale":"2024-03-21T11:21:50.639574+0000","last_undegraded":"2024-03-21T11:21:50.639574+0000","last_fullsized":"2024-03-21T11:21:50.639574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:06:27.398062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645967+0000","last_change":"2024-03-21T11:20:53.733396+0000","last_active":"2024-03-21T11:21:50.645967+0000","last_peered":"2024-03-21T11:21:50.645967+0000","last_clean":"2024-03-21T11:21:50.645967+0000","last_became_active":"2024-03-21T11:20:53.732985+0000","last_became_peered":"2024-03-21T11:20:53.732985+0000","last_unstale":"2024-03-21T11:21:50.645967+0000","last_undegraded":"2024-03-21T11:21:50.645967+0000","last_fullsized":"2024-03-21T11:21:50.645967+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:54:14.064761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":120,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:40.744786+0000","last_change":"2024-03-21T11:20:48.133362+0000","last_active":"2024-03-21T11:22:40.744786+0000","last_peered":"2024-03-21T11:22:40.744786+0000","last_clean":"2024-03-21T11:22:40.744786+0000","last_became_active":"2024-03-21T11:20:48.132405+0000","last_became_peered":"2024-03-21T11:20:48.132405+0000","last_unstale":"2024-03-21T11:22:40.744786+0000","last_undegraded":"2024-03-21T11:22:40.744786+0000","last_fullsized":"2024-03-21T11:22:40.744786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:48:52.688948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644945+0000","last_change":"2024-03-21T11:20:48.659610+0000","last_active":"2024-03-21T11:21:50.644945+0000","last_peered":"2024-03-21T11:21:50.644945+0000","last_clean":"2024-03-21T11:21:50.644945+0000","last_became_active":"2024-03-21T11:20:48.659257+0000","last_became_peered":"2024-03-21T11:20:48.659257+0000","last_unstale":"2024-03-21T11:21:50.644945+0000","last_undegraded":"2024-03-21T11:21:50.644945+0000","last_fullsized":"2024-03-21T11:21:50.644945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:17:30.744434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.656047+0000","last_change":"2024-03-21T11:21:50.656047+0000","last_active":"2024-03-21T11:21:50.656047+0000","last_peered":"2024-03-21T11:21:50.656047+0000","last_clean":"2024-03-21T11:21:50.656047+0000","last_became_active":"2024-03-21T11:21:50.655554+0000","last_became_peered":"2024-03-21T11:21:50.655554+0000","last_unstale":"2024-03-21T11:21:50.656047+0000","last_undegraded":"2024-03-21T11:21:50.656047+0000","last_fullsized":"2024-03-21T11:21:50.656047+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:35:15.858237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.695331+0000","last_change":"2024-03-21T11:21:50.695331+0000","last_active":"2024-03-21T11:21:50.695331+0000","last_peered":"2024-03-21T11:21:50.695331+0000","last_clean":"2024-03-21T11:21:50.695331+0000","last_became_active":"2024-03-21T11:21:50.690368+0000","last_became_peered":"2024-03-21T11:21:50.690368+0000","last_unstale":"2024-03-21T11:21:50.695331+0000","last_undegraded":"2024-03-21T11:21:50.695331+0000","last_fullsized":"2024-03-21T11:21:50.695331+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:50:03.835364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640552+0000","last_change":"2024-03-21T11:20:48.671221+0000","last_active":"2024-03-21T11:21:50.640552+0000","last_peered":"2024-03-21T11:21:50.640552+0000","last_clean":"2024-03-21T11:21:50.640552+0000","last_became_active":"2024-03-21T11:20:48.670837+0000","last_became_peered":"2024-03-21T11:20:48.670837+0000","last_unstale":"2024-03-21T11:21:50.640552+0000","last_undegraded":"2024-03-21T11:21:50.640552+0000","last_fullsized":"2024-03-21T11:21:50.640552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:38.680409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640079+0000","last_change":"2024-03-21T11:20:53.747501+0000","last_active":"2024-03-21T11:21:50.640079+0000","last_peered":"2024-03-21T11:21:50.640079+0000","last_clean":"2024-03-21T11:21:50.640079+0000","last_became_active":"2024-03-21T11:20:53.747062+0000","last_became_peered":"2024-03-21T11:20:53.747062+0000","last_unstale":"2024-03-21T11:21:50.640079+0000","last_undegraded":"2024-03-21T11:21:50.640079+0000","last_fullsized":"2024-03-21T11:21:50.640079+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:58:36.904221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.652004+0000","last_change":"2024-03-21T11:20:48.156175+0000","last_active":"2024-03-21T11:21:50.652004+0000","last_peered":"2024-03-21T11:21:50.652004+0000","last_clean":"2024-03-21T11:21:50.652004+0000","last_became_active":"2024-03-21T11:20:48.154271+0000","last_became_peered":"2024-03-21T11:20:48.154271+0000","last_unstale":"2024-03-21T11:21:50.652004+0000","last_undegraded":"2024-03-21T11:21:50.652004+0000","last_fullsized":"2024-03-21T11:21:50.652004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:04.216391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651905+0000","last_change":"2024-03-21T11:20:48.670011+0000","last_active":"2024-03-21T11:21:50.651905+0000","last_peered":"2024-03-21T11:21:50.651905+0000","last_clean":"2024-03-21T11:21:50.651905+0000","last_became_active":"2024-03-21T11:20:48.669016+0000","last_became_peered":"2024-03-21T11:20:48.669016+0000","last_unstale":"2024-03-21T11:21:50.651905+0000","last_undegraded":"2024-03-21T11:21:50.651905+0000","last_fullsized":"2024-03-21T11:21:50.651905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:43:18.906490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.646486+0000","last_change":"2024-03-21T11:20:53.751934+0000","last_active":"2024-03-21T11:21:50.646486+0000","last_peered":"2024-03-21T11:21:50.646486+0000","last_clean":"2024-03-21T11:21:50.646486+0000","last_became_active":"2024-03-21T11:20:53.751413+0000","last_became_peered":"2024-03-21T11:20:53.751413+0000","last_unstale":"2024-03-21T11:21:50.646486+0000","last_undegraded":"2024-03-21T11:21:50.646486+0000","last_fullsized":"2024-03-21T11:21:50.646486+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:12:33.880376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639167+0000","last_change":"2024-03-21T11:20:48.131796+0000","last_active":"2024-03-21T11:21:50.639167+0000","last_peered":"2024-03-21T11:21:50.639167+0000","last_clean":"2024-03-21T11:21:50.639167+0000","last_became_active":"2024-03-21T11:20:48.131478+0000","last_became_peered":"2024-03-21T11:20:48.131478+0000","last_unstale":"2024-03-21T11:21:50.639167+0000","last_undegraded":"2024-03-21T11:21:50.639167+0000","last_fullsized":"2024-03-21T11:21:50.639167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:59:24.871300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640780+0000","last_change":"2024-03-21T11:20:48.660046+0000","last_active":"2024-03-21T11:21:50.640780+0000","last_peered":"2024-03-21T11:21:50.640780+0000","last_clean":"2024-03-21T11:21:50.640780+0000","last_became_active":"2024-03-21T11:20:48.659621+0000","last_became_peered":"2024-03-21T11:20:48.659621+0000","last_unstale":"2024-03-21T11:21:50.640780+0000","last_undegraded":"2024-03-21T11:21:50.640780+0000","last_fullsized":"2024-03-21T11:21:50.640780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:47:01.923285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.653445+0000","last_change":"2024-03-21T11:20:53.754458+0000","last_active":"2024-03-21T11:21:50.653445+0000","last_peered":"2024-03-21T11:21:50.653445+0000","last_clean":"2024-03-21T11:21:50.653445+0000","last_became_active":"2024-03-21T11:20:53.753927+0000","last_became_peered":"2024-03-21T11:20:53.753927+0000","last_unstale":"2024-03-21T11:21:50.653445+0000","last_undegraded":"2024-03-21T11:21:50.653445+0000","last_fullsized":"2024-03-21T11:21:50.653445+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:00:47.392699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":199,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:40.696452+0000","last_change":"2024-03-21T11:20:48.154125+0000","last_active":"2024-03-21T11:22:40.696452+0000","last_peered":"2024-03-21T11:22:40.696452+0000","last_clean":"2024-03-21T11:22:40.696452+0000","last_became_active":"2024-03-21T11:20:48.153857+0000","last_became_peered":"2024-03-21T11:20:48.153857+0000","last_unstale":"2024-03-21T11:22:40.696452+0000","last_undegraded":"2024-03-21T11:22:40.696452+0000","last_fullsized":"2024-03-21T11:22:40.696452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:54:06.525335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":166,"num_read_kb":166,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689811+0000","last_change":"2024-03-21T11:20:48.650143+0000","last_active":"2024-03-21T11:21:50.689811+0000","last_peered":"2024-03-21T11:21:50.689811+0000","last_clean":"2024-03-21T11:21:50.689811+0000","last_became_active":"2024-03-21T11:20:48.649825+0000","last_became_peered":"2024-03-21T11:20:48.649825+0000","last_unstale":"2024-03-21T11:21:50.689811+0000","last_undegraded":"2024-03-21T11:21:50.689811+0000","last_fullsized":"2024-03-21T11:21:50.689811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:09:15.735280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639235+0000","last_change":"2024-03-21T11:20:53.743006+0000","last_active":"2024-03-21T11:21:50.639235+0000","last_peered":"2024-03-21T11:21:50.639235+0000","last_clean":"2024-03-21T11:21:50.639235+0000","last_became_active":"2024-03-21T11:20:53.742587+0000","last_became_peered":"2024-03-21T11:20:53.742587+0000","last_unstale":"2024-03-21T11:21:50.639235+0000","last_undegraded":"2024-03-21T11:21:50.639235+0000","last_fullsized":"2024-03-21T11:21:50.639235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:06:06.845791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.642935+0000","last_change":"2024-03-21T11:20:48.137137+0000","last_active":"2024-03-21T11:21:50.642935+0000","last_peered":"2024-03-21T11:21:50.642935+0000","last_clean":"2024-03-21T11:21:50.642935+0000","last_became_active":"2024-03-21T11:20:48.136790+0000","last_became_peered":"2024-03-21T11:20:48.136790+0000","last_unstale":"2024-03-21T11:21:50.642935+0000","last_undegraded":"2024-03-21T11:21:50.642935+0000","last_fullsized":"2024-03-21T11:21:50.642935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:22:47.223969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647962+0000","last_change":"2024-03-21T11:20:48.673568+0000","last_active":"2024-03-21T11:21:50.647962+0000","last_peered":"2024-03-21T11:21:50.647962+0000","last_clean":"2024-03-21T11:21:50.647962+0000","last_became_active":"2024-03-21T11:20:48.672613+0000","last_became_peered":"2024-03-21T11:20:48.672613+0000","last_unstale":"2024-03-21T11:21:50.647962+0000","last_undegraded":"2024-03-21T11:21:50.647962+0000","last_fullsized":"2024-03-21T11:21:50.647962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:43:38.193802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.637415+0000","last_change":"2024-03-21T11:20:53.739228+0000","last_active":"2024-03-21T11:21:50.637415+0000","last_peered":"2024-03-21T11:21:50.637415+0000","last_clean":"2024-03-21T11:21:50.637415+0000","last_became_active":"2024-03-21T11:20:53.738750+0000","last_became_peered":"2024-03-21T11:20:53.738750+0000","last_unstale":"2024-03-21T11:21:50.637415+0000","last_undegraded":"2024-03-21T11:21:50.637415+0000","last_fullsized":"2024-03-21T11:21:50.637415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:58:33.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639461+0000","last_change":"2024-03-21T11:20:48.144451+0000","last_active":"2024-03-21T11:21:50.639461+0000","last_peered":"2024-03-21T11:21:50.639461+0000","last_clean":"2024-03-21T11:21:50.639461+0000","last_became_active":"2024-03-21T11:20:48.144074+0000","last_became_peered":"2024-03-21T11:20:48.144074+0000","last_unstale":"2024-03-21T11:21:50.639461+0000","last_undegraded":"2024-03-21T11:21:50.639461+0000","last_fullsized":"2024-03-21T11:21:50.639461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:33:17.032461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689581+0000","last_change":"2024-03-21T11:20:48.654472+0000","last_active":"2024-03-21T11:21:50.689581+0000","last_peered":"2024-03-21T11:21:50.689581+0000","last_clean":"2024-03-21T11:21:50.689581+0000","last_became_active":"2024-03-21T11:20:48.654129+0000","last_became_peered":"2024-03-21T11:20:48.654129+0000","last_unstale":"2024-03-21T11:21:50.689581+0000","last_undegraded":"2024-03-21T11:21:50.689581+0000","last_fullsized":"2024-03-21T11:21:50.689581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:28:31.224296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.641411+0000","last_change":"2024-03-21T11:20:53.739994+0000","last_active":"2024-03-21T11:21:50.641411+0000","last_peered":"2024-03-21T11:21:50.641411+0000","last_clean":"2024-03-21T11:21:50.641411+0000","last_became_active":"2024-03-21T11:20:53.738319+0000","last_became_peered":"2024-03-21T11:20:53.738319+0000","last_unstale":"2024-03-21T11:21:50.641411+0000","last_undegraded":"2024-03-21T11:21:50.641411+0000","last_fullsized":"2024-03-21T11:21:50.641411+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:55.694986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":237,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:40.696522+0000","last_change":"2024-03-21T11:20:48.131461+0000","last_active":"2024-03-21T11:22:40.696522+0000","last_peered":"2024-03-21T11:22:40.696522+0000","last_clean":"2024-03-21T11:22:40.696522+0000","last_became_active":"2024-03-21T11:20:48.131079+0000","last_became_peered":"2024-03-21T11:20:48.131079+0000","last_unstale":"2024-03-21T11:22:40.696522+0000","last_undegraded":"2024-03-21T11:22:40.696522+0000","last_fullsized":"2024-03-21T11:22:40.696522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:48:57.528444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64206,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":173,"num_read_kb":149,"num_write":37,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647182+0000","last_change":"2024-03-21T11:20:48.653696+0000","last_active":"2024-03-21T11:21:50.647182+0000","last_peered":"2024-03-21T11:21:50.647182+0000","last_clean":"2024-03-21T11:21:50.647182+0000","last_became_active":"2024-03-21T11:20:48.653218+0000","last_became_peered":"2024-03-21T11:20:48.653218+0000","last_unstale":"2024-03-21T11:21:50.647182+0000","last_undegraded":"2024-03-21T11:21:50.647182+0000","last_fullsized":"2024-03-21T11:21:50.647182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:11:59.419040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639982+0000","last_change":"2024-03-21T11:20:53.754019+0000","last_active":"2024-03-21T11:21:50.639982+0000","last_peered":"2024-03-21T11:21:50.639982+0000","last_clean":"2024-03-21T11:21:50.639982+0000","last_became_active":"2024-03-21T11:20:53.753606+0000","last_became_peered":"2024-03-21T11:20:53.753606+0000","last_unstale":"2024-03-21T11:21:50.639982+0000","last_undegraded":"2024-03-21T11:21:50.639982+0000","last_fullsized":"2024-03-21T11:21:50.639982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:23:27.898596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645028+0000","last_change":"2024-03-21T11:20:48.153544+0000","last_active":"2024-03-21T11:21:50.645028+0000","last_peered":"2024-03-21T11:21:50.645028+0000","last_clean":"2024-03-21T11:21:50.645028+0000","last_became_active":"2024-03-21T11:20:48.153279+0000","last_became_peered":"2024-03-21T11:20:48.153279+0000","last_unstale":"2024-03-21T11:21:50.645028+0000","last_undegraded":"2024-03-21T11:21:50.645028+0000","last_fullsized":"2024-03-21T11:21:50.645028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:31:31.911844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.654309+0000","last_change":"2024-03-21T11:20:48.665140+0000","last_active":"2024-03-21T11:21:50.654309+0000","last_peered":"2024-03-21T11:21:50.654309+0000","last_clean":"2024-03-21T11:21:50.654309+0000","last_became_active":"2024-03-21T11:20:48.664226+0000","last_became_peered":"2024-03-21T11:20:48.664226+0000","last_unstale":"2024-03-21T11:21:50.654309+0000","last_undegraded":"2024-03-21T11:21:50.654309+0000","last_fullsized":"2024-03-21T11:21:50.654309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:47:38.951403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.641466+0000","last_change":"2024-03-21T11:20:53.751887+0000","last_active":"2024-03-21T11:21:50.641466+0000","last_peered":"2024-03-21T11:21:50.641466+0000","last_clean":"2024-03-21T11:21:50.641466+0000","last_became_active":"2024-03-21T11:20:53.751353+0000","last_became_peered":"2024-03-21T11:20:53.751353+0000","last_unstale":"2024-03-21T11:21:50.641466+0000","last_undegraded":"2024-03-21T11:21:50.641466+0000","last_fullsized":"2024-03-21T11:21:50.641466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:21:36.689786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639455+0000","last_change":"2024-03-21T11:20:48.162150+0000","last_active":"2024-03-21T11:21:50.639455+0000","last_peered":"2024-03-21T11:21:50.639455+0000","last_clean":"2024-03-21T11:21:50.639455+0000","last_became_active":"2024-03-21T11:20:48.161825+0000","last_became_peered":"2024-03-21T11:20:48.161825+0000","last_unstale":"2024-03-21T11:21:50.639455+0000","last_undegraded":"2024-03-21T11:21:50.639455+0000","last_fullsized":"2024-03-21T11:21:50.639455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:10:47.097540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.642620+0000","last_change":"2024-03-21T11:20:48.663856+0000","last_active":"2024-03-21T11:21:50.642620+0000","last_peered":"2024-03-21T11:21:50.642620+0000","last_clean":"2024-03-21T11:21:50.642620+0000","last_became_active":"2024-03-21T11:20:48.662908+0000","last_became_peered":"2024-03-21T11:20:48.662908+0000","last_unstale":"2024-03-21T11:21:50.642620+0000","last_undegraded":"2024-03-21T11:21:50.642620+0000","last_fullsized":"2024-03-21T11:21:50.642620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:11:12.255265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.643577+0000","last_change":"2024-03-21T11:20:53.755942+0000","last_active":"2024-03-21T11:21:50.643577+0000","last_peered":"2024-03-21T11:21:50.643577+0000","last_clean":"2024-03-21T11:21:50.643577+0000","last_became_active":"2024-03-21T11:20:53.755141+0000","last_became_peered":"2024-03-21T11:20:53.755141+0000","last_unstale":"2024-03-21T11:21:50.643577+0000","last_undegraded":"2024-03-21T11:21:50.643577+0000","last_fullsized":"2024-03-21T11:21:50.643577+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:30.141117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":121,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:41.735056+0000","last_change":"2024-03-21T11:20:48.132460+0000","last_active":"2024-03-21T11:22:41.735056+0000","last_peered":"2024-03-21T11:22:41.735056+0000","last_clean":"2024-03-21T11:22:41.735056+0000","last_became_active":"2024-03-21T11:20:48.131573+0000","last_became_peered":"2024-03-21T11:20:48.131573+0000","last_unstale":"2024-03-21T11:22:41.735056+0000","last_undegraded":"2024-03-21T11:22:41.735056+0000","last_fullsized":"2024-03-21T11:22:41.735056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:40:02.821536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":88,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.641878+0000","last_change":"2024-03-21T11:20:48.656262+0000","last_active":"2024-03-21T11:21:50.641878+0000","last_peered":"2024-03-21T11:21:50.641878+0000","last_clean":"2024-03-21T11:21:50.641878+0000","last_became_active":"2024-03-21T11:20:48.655924+0000","last_became_peered":"2024-03-21T11:20:48.655924+0000","last_unstale":"2024-03-21T11:21:50.641878+0000","last_undegraded":"2024-03-21T11:21:50.641878+0000","last_fullsized":"2024-03-21T11:21:50.641878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:24:20.412185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651635+0000","last_change":"2024-03-21T11:20:53.758666+0000","last_active":"2024-03-21T11:21:50.651635+0000","last_peered":"2024-03-21T11:21:50.651635+0000","last_clean":"2024-03-21T11:21:50.651635+0000","last_became_active":"2024-03-21T11:20:53.757398+0000","last_became_peered":"2024-03-21T11:20:53.757398+0000","last_unstale":"2024-03-21T11:21:50.651635+0000","last_undegraded":"2024-03-21T11:21:50.651635+0000","last_fullsized":"2024-03-21T11:21:50.651635+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:34.198388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644797+0000","last_change":"2024-03-21T11:20:48.132725+0000","last_active":"2024-03-21T11:21:50.644797+0000","last_peered":"2024-03-21T11:21:50.644797+0000","last_clean":"2024-03-21T11:21:50.644797+0000","last_became_active":"2024-03-21T11:20:48.132367+0000","last_became_peered":"2024-03-21T11:20:48.132367+0000","last_unstale":"2024-03-21T11:21:50.644797+0000","last_undegraded":"2024-03-21T11:21:50.644797+0000","last_fullsized":"2024-03-21T11:21:50.644797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:19:18.334962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640266+0000","last_change":"2024-03-21T11:20:48.656302+0000","last_active":"2024-03-21T11:21:50.640266+0000","last_peered":"2024-03-21T11:21:50.640266+0000","last_clean":"2024-03-21T11:21:50.640266+0000","last_became_active":"2024-03-21T11:20:48.654305+0000","last_became_peered":"2024-03-21T11:20:48.654305+0000","last_unstale":"2024-03-21T11:21:50.640266+0000","last_undegraded":"2024-03-21T11:21:50.640266+0000","last_fullsized":"2024-03-21T11:21:50.640266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:59:40.766829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639309+0000","last_change":"2024-03-21T11:20:53.750710+0000","last_active":"2024-03-21T11:21:50.639309+0000","last_peered":"2024-03-21T11:21:50.639309+0000","last_clean":"2024-03-21T11:21:50.639309+0000","last_became_active":"2024-03-21T11:20:53.750255+0000","last_became_peered":"2024-03-21T11:20:53.750255+0000","last_unstale":"2024-03-21T11:21:50.639309+0000","last_undegraded":"2024-03-21T11:21:50.639309+0000","last_fullsized":"2024-03-21T11:21:50.639309+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:36:28.611838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647379+0000","last_change":"2024-03-21T11:20:48.133524+0000","last_active":"2024-03-21T11:21:50.647379+0000","last_peered":"2024-03-21T11:21:50.647379+0000","last_clean":"2024-03-21T11:21:50.647379+0000","last_became_active":"2024-03-21T11:20:48.133186+0000","last_became_peered":"2024-03-21T11:20:48.133186+0000","last_unstale":"2024-03-21T11:21:50.647379+0000","last_undegraded":"2024-03-21T11:21:50.647379+0000","last_fullsized":"2024-03-21T11:21:50.647379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:02.018639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639075+0000","last_change":"2024-03-21T11:20:48.654126+0000","last_active":"2024-03-21T11:21:50.639075+0000","last_peered":"2024-03-21T11:21:50.639075+0000","last_clean":"2024-03-21T11:21:50.639075+0000","last_became_active":"2024-03-21T11:20:48.653835+0000","last_became_peered":"2024-03-21T11:20:48.653835+0000","last_unstale":"2024-03-21T11:21:50.639075+0000","last_undegraded":"2024-03-21T11:21:50.639075+0000","last_fullsized":"2024-03-21T11:21:50.639075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:12:03.332076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644944+0000","last_change":"2024-03-21T11:20:53.916645+0000","last_active":"2024-03-21T11:21:50.644944+0000","last_peered":"2024-03-21T11:21:50.644944+0000","last_clean":"2024-03-21T11:21:50.644944+0000","last_became_active":"2024-03-21T11:20:53.916178+0000","last_became_peered":"2024-03-21T11:20:53.916178+0000","last_unstale":"2024-03-21T11:21:50.644944+0000","last_undegraded":"2024-03-21T11:21:50.644944+0000","last_fullsized":"2024-03-21T11:21:50.644944+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:56:15.805231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640468+0000","last_change":"2024-03-21T11:20:48.159877+0000","last_active":"2024-03-21T11:21:50.640468+0000","last_peered":"2024-03-21T11:21:50.640468+0000","last_clean":"2024-03-21T11:21:50.640468+0000","last_became_active":"2024-03-21T11:20:48.159529+0000","last_became_peered":"2024-03-21T11:20:48.159529+0000","last_unstale":"2024-03-21T11:21:50.640468+0000","last_undegraded":"2024-03-21T11:21:50.640468+0000","last_fullsized":"2024-03-21T11:21:50.640468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:17:09.662462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645078+0000","last_change":"2024-03-21T11:20:48.663311+0000","last_active":"2024-03-21T11:21:50.645078+0000","last_peered":"2024-03-21T11:21:50.645078+0000","last_clean":"2024-03-21T11:21:50.645078+0000","last_became_active":"2024-03-21T11:20:48.662961+0000","last_became_peered":"2024-03-21T11:20:48.662961+0000","last_unstale":"2024-03-21T11:21:50.645078+0000","last_undegraded":"2024-03-21T11:21:50.645078+0000","last_fullsized":"2024-03-21T11:21:50.645078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:14:46.616326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651173+0000","last_change":"2024-03-21T11:20:53.751804+0000","last_active":"2024-03-21T11:21:50.651173+0000","last_peered":"2024-03-21T11:21:50.651173+0000","last_clean":"2024-03-21T11:21:50.651173+0000","last_became_active":"2024-03-21T11:20:53.751323+0000","last_became_peered":"2024-03-21T11:20:53.751323+0000","last_unstale":"2024-03-21T11:21:50.651173+0000","last_undegraded":"2024-03-21T11:21:50.651173+0000","last_fullsized":"2024-03-21T11:21:50.651173+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:38:25.677345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":119,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:40.783156+0000","last_change":"2024-03-21T11:20:48.147580+0000","last_active":"2024-03-21T11:22:40.783156+0000","last_peered":"2024-03-21T11:22:40.783156+0000","last_clean":"2024-03-21T11:22:40.783156+0000","last_became_active":"2024-03-21T11:20:48.147222+0000","last_became_peered":"2024-03-21T11:20:48.147222+0000","last_unstale":"2024-03-21T11:22:40.783156+0000","last_undegraded":"2024-03-21T11:22:40.783156+0000","last_fullsized":"2024-03-21T11:22:40.783156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:49:04.204129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644790+0000","last_change":"2024-03-21T11:20:48.667940+0000","last_active":"2024-03-21T11:21:50.644790+0000","last_peered":"2024-03-21T11:21:50.644790+0000","last_clean":"2024-03-21T11:21:50.644790+0000","last_became_active":"2024-03-21T11:20:48.667595+0000","last_became_peered":"2024-03-21T11:20:48.667595+0000","last_unstale":"2024-03-21T11:21:50.644790+0000","last_undegraded":"2024-03-21T11:21:50.644790+0000","last_fullsized":"2024-03-21T11:21:50.644790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:08:14.007589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.652571+0000","last_change":"2024-03-21T11:20:53.916931+0000","last_active":"2024-03-21T11:21:50.652571+0000","last_peered":"2024-03-21T11:21:50.652571+0000","last_clean":"2024-03-21T11:21:50.652571+0000","last_became_active":"2024-03-21T11:20:53.916495+0000","last_became_peered":"2024-03-21T11:20:53.916495+0000","last_unstale":"2024-03-21T11:21:50.652571+0000","last_undegraded":"2024-03-21T11:21:50.652571+0000","last_fullsized":"2024-03-21T11:21:50.652571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:18:33.046052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639370+0000","last_change":"2024-03-21T11:20:48.652997+0000","last_active":"2024-03-21T11:21:50.639370+0000","last_peered":"2024-03-21T11:21:50.639370+0000","last_clean":"2024-03-21T11:21:50.639370+0000","last_became_active":"2024-03-21T11:20:48.652680+0000","last_became_peered":"2024-03-21T11:20:48.652680+0000","last_unstale":"2024-03-21T11:21:50.639370+0000","last_undegraded":"2024-03-21T11:21:50.639370+0000","last_fullsized":"2024-03-21T11:21:50.639370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:00:19.516607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":118,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:40.733172+0000","last_change":"2024-03-21T11:20:48.164607+0000","last_active":"2024-03-21T11:22:40.733172+0000","last_peered":"2024-03-21T11:22:40.733172+0000","last_clean":"2024-03-21T11:22:40.733172+0000","last_became_active":"2024-03-21T11:20:48.164222+0000","last_became_peered":"2024-03-21T11:20:48.164222+0000","last_unstale":"2024-03-21T11:22:40.733172+0000","last_undegraded":"2024-03-21T11:22:40.733172+0000","last_fullsized":"2024-03-21T11:22:40.733172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:52:24.359155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.658412+0000","last_change":"2024-03-21T11:21:50.658412+0000","last_active":"2024-03-21T11:21:50.658412+0000","last_peered":"2024-03-21T11:21:50.658412+0000","last_clean":"2024-03-21T11:21:50.658412+0000","last_became_active":"2024-03-21T11:21:50.649270+0000","last_became_peered":"2024-03-21T11:21:50.649270+0000","last_unstale":"2024-03-21T11:21:50.658412+0000","last_undegraded":"2024-03-21T11:21:50.658412+0000","last_fullsized":"2024-03-21T11:21:50.658412+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:11:06.006302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651571+0000","last_change":"2024-03-21T11:20:48.667212+0000","last_active":"2024-03-21T11:21:50.651571+0000","last_peered":"2024-03-21T11:21:50.651571+0000","last_clean":"2024-03-21T11:21:50.651571+0000","last_became_active":"2024-03-21T11:20:48.662839+0000","last_became_peered":"2024-03-21T11:20:48.662839+0000","last_unstale":"2024-03-21T11:21:50.651571+0000","last_undegraded":"2024-03-21T11:21:50.651571+0000","last_fullsized":"2024-03-21T11:21:50.651571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:27:41.490922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.637316+0000","last_change":"2024-03-21T11:20:48.151185+0000","last_active":"2024-03-21T11:21:50.637316+0000","last_peered":"2024-03-21T11:21:50.637316+0000","last_clean":"2024-03-21T11:21:50.637316+0000","last_became_active":"2024-03-21T11:20:48.150859+0000","last_became_peered":"2024-03-21T11:20:48.150859+0000","last_unstale":"2024-03-21T11:21:50.637316+0000","last_undegraded":"2024-03-21T11:21:50.637316+0000","last_fullsized":"2024-03-21T11:21:50.637316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:52:45.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.641662+0000","last_change":"2024-03-21T11:19:28.916140+0000","last_active":"2024-03-21T11:21:50.641662+0000","last_peered":"2024-03-21T11:21:50.641662+0000","last_clean":"2024-03-21T11:21:50.641662+0000","last_became_active":"2024-03-21T11:19:28.915634+0000","last_became_peered":"2024-03-21T11:19:28.915634+0000","last_unstale":"2024-03-21T11:21:50.641662+0000","last_undegraded":"2024-03-21T11:21:50.641662+0000","last_fullsized":"2024-03-21T11:21:50.641662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:15:35.803515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:15:35.803515+0000","last_clean_scrub_stamp":"2024-03-21T11:15:35.803515+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:08:31.509894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.656708+0000","last_change":"2024-03-21T11:21:50.656708+0000","last_active":"2024-03-21T11:21:50.656708+0000","last_peered":"2024-03-21T11:21:50.656708+0000","last_clean":"2024-03-21T11:21:50.656708+0000","last_became_active":"2024-03-21T11:21:50.656236+0000","last_became_peered":"2024-03-21T11:21:50.656236+0000","last_unstale":"2024-03-21T11:21:50.656708+0000","last_undegraded":"2024-03-21T11:21:50.656708+0000","last_fullsized":"2024-03-21T11:21:50.656708+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:17:44.114512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651700+0000","last_change":"2024-03-21T11:20:48.657878+0000","last_active":"2024-03-21T11:21:50.651700+0000","last_peered":"2024-03-21T11:21:50.651700+0000","last_clean":"2024-03-21T11:21:50.651700+0000","last_became_active":"2024-03-21T11:20:48.657212+0000","last_became_peered":"2024-03-21T11:20:48.657212+0000","last_unstale":"2024-03-21T11:21:50.651700+0000","last_undegraded":"2024-03-21T11:21:50.651700+0000","last_fullsized":"2024-03-21T11:21:50.651700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:43:11.993577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.641781+0000","last_change":"2024-03-21T11:20:48.135513+0000","last_active":"2024-03-21T11:21:50.641781+0000","last_peered":"2024-03-21T11:21:50.641781+0000","last_clean":"2024-03-21T11:21:50.641781+0000","last_became_active":"2024-03-21T11:20:48.135223+0000","last_became_peered":"2024-03-21T11:20:48.135223+0000","last_unstale":"2024-03-21T11:21:50.641781+0000","last_undegraded":"2024-03-21T11:21:50.641781+0000","last_fullsized":"2024-03-21T11:21:50.641781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:37:38.834284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640721+0000","last_change":"2024-03-21T11:20:53.751790+0000","last_active":"2024-03-21T11:21:50.640721+0000","last_peered":"2024-03-21T11:21:50.640721+0000","last_clean":"2024-03-21T11:21:50.640721+0000","last_became_active":"2024-03-21T11:20:53.751312+0000","last_became_peered":"2024-03-21T11:20:53.751312+0000","last_unstale":"2024-03-21T11:21:50.640721+0000","last_undegraded":"2024-03-21T11:21:50.640721+0000","last_fullsized":"2024-03-21T11:21:50.640721+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:21:41.599985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647236+0000","last_change":"2024-03-21T11:20:48.154119+0000","last_active":"2024-03-21T11:21:50.647236+0000","last_peered":"2024-03-21T11:21:50.647236+0000","last_clean":"2024-03-21T11:21:50.647236+0000","last_became_active":"2024-03-21T11:20:48.153781+0000","last_became_peered":"2024-03-21T11:20:48.153781+0000","last_unstale":"2024-03-21T11:21:50.647236+0000","last_undegraded":"2024-03-21T11:21:50.647236+0000","last_fullsized":"2024-03-21T11:21:50.647236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:44:13.937836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651405+0000","last_change":"2024-03-21T11:20:48.661282+0000","last_active":"2024-03-21T11:21:50.651405+0000","last_peered":"2024-03-21T11:21:50.651405+0000","last_clean":"2024-03-21T11:21:50.651405+0000","last_became_active":"2024-03-21T11:20:48.660569+0000","last_became_peered":"2024-03-21T11:20:48.660569+0000","last_unstale":"2024-03-21T11:21:50.651405+0000","last_undegraded":"2024-03-21T11:21:50.651405+0000","last_fullsized":"2024-03-21T11:21:50.651405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:14.875700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651671+0000","last_change":"2024-03-21T11:20:53.918016+0000","last_active":"2024-03-21T11:21:50.651671+0000","last_peered":"2024-03-21T11:21:50.651671+0000","last_clean":"2024-03-21T11:21:50.651671+0000","last_became_active":"2024-03-21T11:20:53.916891+0000","last_became_peered":"2024-03-21T11:20:53.916891+0000","last_unstale":"2024-03-21T11:21:50.651671+0000","last_undegraded":"2024-03-21T11:21:50.651671+0000","last_fullsized":"2024-03-21T11:21:50.651671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:40:25.206778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'12","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.643805+0000","last_change":"2024-03-21T11:20:48.167560+0000","last_active":"2024-03-21T11:21:50.643805+0000","last_peered":"2024-03-21T11:21:50.643805+0000","last_clean":"2024-03-21T11:21:50.643805+0000","last_became_active":"2024-03-21T11:20:48.164986+0000","last_became_peered":"2024-03-21T11:20:48.164986+0000","last_unstale":"2024-03-21T11:21:50.643805+0000","last_undegraded":"2024-03-21T11:21:50.643805+0000","last_fullsized":"2024-03-21T11:21:50.643805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:59.930626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.653352+0000","last_change":"2024-03-21T11:20:48.675250+0000","last_active":"2024-03-21T11:21:50.653352+0000","last_peered":"2024-03-21T11:21:50.653352+0000","last_clean":"2024-03-21T11:21:50.653352+0000","last_became_active":"2024-03-21T11:20:48.674638+0000","last_became_peered":"2024-03-21T11:20:48.674638+0000","last_unstale":"2024-03-21T11:21:50.653352+0000","last_undegraded":"2024-03-21T11:21:50.653352+0000","last_fullsized":"2024-03-21T11:21:50.653352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:46:34.842363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644270+0000","last_change":"2024-03-21T11:20:53.733809+0000","last_active":"2024-03-21T11:21:50.644270+0000","last_peered":"2024-03-21T11:21:50.644270+0000","last_clean":"2024-03-21T11:21:50.644270+0000","last_became_active":"2024-03-21T11:20:53.733393+0000","last_became_peered":"2024-03-21T11:20:53.733393+0000","last_unstale":"2024-03-21T11:21:50.644270+0000","last_undegraded":"2024-03-21T11:21:50.644270+0000","last_fullsized":"2024-03-21T11:21:50.644270+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:02:46.270716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.637327+0000","last_change":"2024-03-21T11:20:48.139475+0000","last_active":"2024-03-21T11:21:50.637327+0000","last_peered":"2024-03-21T11:21:50.637327+0000","last_clean":"2024-03-21T11:21:50.637327+0000","last_became_active":"2024-03-21T11:20:48.139249+0000","last_became_peered":"2024-03-21T11:20:48.139249+0000","last_unstale":"2024-03-21T11:21:50.637327+0000","last_undegraded":"2024-03-21T11:21:50.637327+0000","last_fullsized":"2024-03-21T11:21:50.637327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:45:18.811829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647070+0000","last_change":"2024-03-21T11:20:48.673975+0000","last_active":"2024-03-21T11:21:50.647070+0000","last_peered":"2024-03-21T11:21:50.647070+0000","last_clean":"2024-03-21T11:21:50.647070+0000","last_became_active":"2024-03-21T11:20:48.673587+0000","last_became_peered":"2024-03-21T11:20:48.673587+0000","last_unstale":"2024-03-21T11:21:50.647070+0000","last_undegraded":"2024-03-21T11:21:50.647070+0000","last_fullsized":"2024-03-21T11:21:50.647070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:08:52.354780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.636920+0000","last_change":"2024-03-21T11:20:53.917493+0000","last_active":"2024-03-21T11:21:50.636920+0000","last_peered":"2024-03-21T11:21:50.636920+0000","last_clean":"2024-03-21T11:21:50.636920+0000","last_became_active":"2024-03-21T11:20:53.917007+0000","last_became_peered":"2024-03-21T11:20:53.917007+0000","last_unstale":"2024-03-21T11:21:50.636920+0000","last_undegraded":"2024-03-21T11:21:50.636920+0000","last_fullsized":"2024-03-21T11:21:50.636920+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:39:20.280899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689227+0000","last_change":"2024-03-21T11:20:48.140914+0000","last_active":"2024-03-21T11:21:50.689227+0000","last_peered":"2024-03-21T11:21:50.689227+0000","last_clean":"2024-03-21T11:21:50.689227+0000","last_became_active":"2024-03-21T11:20:48.140558+0000","last_became_peered":"2024-03-21T11:20:48.140558+0000","last_unstale":"2024-03-21T11:21:50.689227+0000","last_undegraded":"2024-03-21T11:21:50.689227+0000","last_fullsized":"2024-03-21T11:21:50.689227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:59:52.065431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644834+0000","last_change":"2024-03-21T11:20:48.651190+0000","last_active":"2024-03-21T11:21:50.644834+0000","last_peered":"2024-03-21T11:21:50.644834+0000","last_clean":"2024-03-21T11:21:50.644834+0000","last_became_active":"2024-03-21T11:20:48.650818+0000","last_became_peered":"2024-03-21T11:20:48.650818+0000","last_unstale":"2024-03-21T11:21:50.644834+0000","last_undegraded":"2024-03-21T11:21:50.644834+0000","last_fullsized":"2024-03-21T11:21:50.644834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:47:17.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644034+0000","last_change":"2024-03-21T11:20:53.756844+0000","last_active":"2024-03-21T11:21:50.644034+0000","last_peered":"2024-03-21T11:21:50.644034+0000","last_clean":"2024-03-21T11:21:50.644034+0000","last_became_active":"2024-03-21T11:20:53.756380+0000","last_became_peered":"2024-03-21T11:20:53.756380+0000","last_unstale":"2024-03-21T11:21:50.644034+0000","last_undegraded":"2024-03-21T11:21:50.644034+0000","last_fullsized":"2024-03-21T11:21:50.644034+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:51:55.327798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.653167+0000","last_change":"2024-03-21T11:20:48.160749+0000","last_active":"2024-03-21T11:21:50.653167+0000","last_peered":"2024-03-21T11:21:50.653167+0000","last_clean":"2024-03-21T11:21:50.653167+0000","last_became_active":"2024-03-21T11:20:48.158507+0000","last_became_peered":"2024-03-21T11:20:48.158507+0000","last_unstale":"2024-03-21T11:21:50.653167+0000","last_undegraded":"2024-03-21T11:21:50.653167+0000","last_fullsized":"2024-03-21T11:21:50.653167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:55.375529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639079+0000","last_change":"2024-03-21T11:20:48.667153+0000","last_active":"2024-03-21T11:21:50.639079+0000","last_peered":"2024-03-21T11:21:50.639079+0000","last_clean":"2024-03-21T11:21:50.639079+0000","last_became_active":"2024-03-21T11:20:48.666823+0000","last_became_peered":"2024-03-21T11:20:48.666823+0000","last_unstale":"2024-03-21T11:21:50.639079+0000","last_undegraded":"2024-03-21T11:21:50.639079+0000","last_fullsized":"2024-03-21T11:21:50.639079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:43:54.183372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.637008+0000","last_change":"2024-03-21T11:20:53.739188+0000","last_active":"2024-03-21T11:21:50.637008+0000","last_peered":"2024-03-21T11:21:50.637008+0000","last_clean":"2024-03-21T11:21:50.637008+0000","last_became_active":"2024-03-21T11:20:53.738770+0000","last_became_peered":"2024-03-21T11:20:53.738770+0000","last_unstale":"2024-03-21T11:21:50.637008+0000","last_undegraded":"2024-03-21T11:21:50.637008+0000","last_fullsized":"2024-03-21T11:21:50.637008+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:53:48.897963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.668860+0000","last_change":"2024-03-21T11:21:50.668860+0000","last_active":"2024-03-21T11:21:50.668860+0000","last_peered":"2024-03-21T11:21:50.668860+0000","last_clean":"2024-03-21T11:21:50.668860+0000","last_became_active":"2024-03-21T11:21:50.655255+0000","last_became_peered":"2024-03-21T11:21:50.655255+0000","last_unstale":"2024-03-21T11:21:50.668860+0000","last_undegraded":"2024-03-21T11:21:50.668860+0000","last_fullsized":"2024-03-21T11:21:50.668860+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:27.634508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.650514+0000","last_change":"2024-03-21T11:20:48.677303+0000","last_active":"2024-03-21T11:21:50.650514+0000","last_peered":"2024-03-21T11:21:50.650514+0000","last_clean":"2024-03-21T11:21:50.650514+0000","last_became_active":"2024-03-21T11:20:48.676903+0000","last_became_peered":"2024-03-21T11:20:48.676903+0000","last_unstale":"2024-03-21T11:21:50.650514+0000","last_undegraded":"2024-03-21T11:21:50.650514+0000","last_fullsized":"2024-03-21T11:21:50.650514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:06:59.922274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.636868+0000","last_change":"2024-03-21T11:20:53.916691+0000","last_active":"2024-03-21T11:21:50.636868+0000","last_peered":"2024-03-21T11:21:50.636868+0000","last_clean":"2024-03-21T11:21:50.636868+0000","last_became_active":"2024-03-21T11:20:53.916275+0000","last_became_peered":"2024-03-21T11:20:53.916275+0000","last_unstale":"2024-03-21T11:21:50.636868+0000","last_undegraded":"2024-03-21T11:21:50.636868+0000","last_fullsized":"2024-03-21T11:21:50.636868+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:35.848589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.668467+0000","last_change":"2024-03-21T11:21:50.668467+0000","last_active":"2024-03-21T11:21:50.668467+0000","last_peered":"2024-03-21T11:21:50.668467+0000","last_clean":"2024-03-21T11:21:50.668467+0000","last_became_active":"2024-03-21T11:21:50.649598+0000","last_became_peered":"2024-03-21T11:21:50.649598+0000","last_unstale":"2024-03-21T11:21:50.668467+0000","last_undegraded":"2024-03-21T11:21:50.668467+0000","last_fullsized":"2024-03-21T11:21:50.668467+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:01:48.186997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.642157+0000","last_change":"2024-03-21T11:20:48.661486+0000","last_active":"2024-03-21T11:21:50.642157+0000","last_peered":"2024-03-21T11:21:50.642157+0000","last_clean":"2024-03-21T11:21:50.642157+0000","last_became_active":"2024-03-21T11:20:48.661109+0000","last_became_peered":"2024-03-21T11:20:48.661109+0000","last_unstale":"2024-03-21T11:21:50.642157+0000","last_undegraded":"2024-03-21T11:21:50.642157+0000","last_fullsized":"2024-03-21T11:21:50.642157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:15:27.093616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689652+0000","last_change":"2024-03-21T11:20:53.747223+0000","last_active":"2024-03-21T11:21:50.689652+0000","last_peered":"2024-03-21T11:21:50.689652+0000","last_clean":"2024-03-21T11:21:50.689652+0000","last_became_active":"2024-03-21T11:20:53.746805+0000","last_became_peered":"2024-03-21T11:20:53.746805+0000","last_unstale":"2024-03-21T11:21:50.689652+0000","last_undegraded":"2024-03-21T11:21:50.689652+0000","last_fullsized":"2024-03-21T11:21:50.689652+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:29.186745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'6","reported_seq":117,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:22:37.693923+0000","last_change":"2024-03-21T11:20:48.151172+0000","last_active":"2024-03-21T11:22:37.693923+0000","last_peered":"2024-03-21T11:22:37.693923+0000","last_clean":"2024-03-21T11:22:37.693923+0000","last_became_active":"2024-03-21T11:20:48.150832+0000","last_became_peered":"2024-03-21T11:20:48.150832+0000","last_unstale":"2024-03-21T11:22:37.693923+0000","last_undegraded":"2024-03-21T11:22:37.693923+0000","last_fullsized":"2024-03-21T11:22:37.693923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:34:46.367713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":83,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.643555+0000","last_change":"2024-03-21T11:20:48.652275+0000","last_active":"2024-03-21T11:21:50.643555+0000","last_peered":"2024-03-21T11:21:50.643555+0000","last_clean":"2024-03-21T11:21:50.643555+0000","last_became_active":"2024-03-21T11:20:48.651965+0000","last_became_peered":"2024-03-21T11:20:48.651965+0000","last_unstale":"2024-03-21T11:21:50.643555+0000","last_undegraded":"2024-03-21T11:21:50.643555+0000","last_fullsized":"2024-03-21T11:21:50.643555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:56:18.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.647381+0000","last_change":"2024-03-21T11:20:53.748050+0000","last_active":"2024-03-21T11:21:50.647381+0000","last_peered":"2024-03-21T11:21:50.647381+0000","last_clean":"2024-03-21T11:21:50.647381+0000","last_became_active":"2024-03-21T11:20:53.747636+0000","last_became_peered":"2024-03-21T11:20:53.747636+0000","last_unstale":"2024-03-21T11:21:50.647381+0000","last_undegraded":"2024-03-21T11:21:50.647381+0000","last_fullsized":"2024-03-21T11:21:50.647381+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:09:15.202183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639671+0000","last_change":"2024-03-21T11:20:48.156928+0000","last_active":"2024-03-21T11:21:50.639671+0000","last_peered":"2024-03-21T11:21:50.639671+0000","last_clean":"2024-03-21T11:21:50.639671+0000","last_became_active":"2024-03-21T11:20:48.156625+0000","last_became_peered":"2024-03-21T11:20:48.156625+0000","last_unstale":"2024-03-21T11:21:50.639671+0000","last_undegraded":"2024-03-21T11:21:50.639671+0000","last_fullsized":"2024-03-21T11:21:50.639671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:19:42.035565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.642846+0000","last_change":"2024-03-21T11:20:48.663655+0000","last_active":"2024-03-21T11:21:50.642846+0000","last_peered":"2024-03-21T11:21:50.642846+0000","last_clean":"2024-03-21T11:21:50.642846+0000","last_became_active":"2024-03-21T11:20:48.663336+0000","last_became_peered":"2024-03-21T11:20:48.663336+0000","last_unstale":"2024-03-21T11:21:50.642846+0000","last_undegraded":"2024-03-21T11:21:50.642846+0000","last_fullsized":"2024-03-21T11:21:50.642846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:29:05.119283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.638869+0000","last_change":"2024-03-21T11:20:53.732584+0000","last_active":"2024-03-21T11:21:50.638869+0000","last_peered":"2024-03-21T11:21:50.638869+0000","last_clean":"2024-03-21T11:21:50.638869+0000","last_became_active":"2024-03-21T11:20:53.731975+0000","last_became_peered":"2024-03-21T11:20:53.731975+0000","last_unstale":"2024-03-21T11:21:50.638869+0000","last_undegraded":"2024-03-21T11:21:50.638869+0000","last_fullsized":"2024-03-21T11:21:50.638869+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:51:06.336241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651482+0000","last_change":"2024-03-21T11:20:48.165594+0000","last_active":"2024-03-21T11:21:50.651482+0000","last_peered":"2024-03-21T11:21:50.651482+0000","last_clean":"2024-03-21T11:21:50.651482+0000","last_became_active":"2024-03-21T11:20:48.164394+0000","last_became_peered":"2024-03-21T11:20:48.164394+0000","last_unstale":"2024-03-21T11:21:50.651482+0000","last_undegraded":"2024-03-21T11:21:50.651482+0000","last_fullsized":"2024-03-21T11:21:50.651482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:13.317767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.643170+0000","last_change":"2024-03-21T11:20:48.649359+0000","last_active":"2024-03-21T11:21:50.643170+0000","last_peered":"2024-03-21T11:21:50.643170+0000","last_clean":"2024-03-21T11:21:50.643170+0000","last_became_active":"2024-03-21T11:20:48.649031+0000","last_became_peered":"2024-03-21T11:20:48.649031+0000","last_unstale":"2024-03-21T11:21:50.643170+0000","last_undegraded":"2024-03-21T11:21:50.643170+0000","last_fullsized":"2024-03-21T11:21:50.643170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:17.055732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639185+0000","last_change":"2024-03-21T11:20:48.654999+0000","last_active":"2024-03-21T11:21:50.639185+0000","last_peered":"2024-03-21T11:21:50.639185+0000","last_clean":"2024-03-21T11:21:50.639185+0000","last_became_active":"2024-03-21T11:20:48.654638+0000","last_became_peered":"2024-03-21T11:20:48.654638+0000","last_unstale":"2024-03-21T11:21:50.639185+0000","last_undegraded":"2024-03-21T11:21:50.639185+0000","last_fullsized":"2024-03-21T11:21:50.639185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:49:48.711420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.644656+0000","last_change":"2024-03-21T11:20:48.651864+0000","last_active":"2024-03-21T11:21:50.644656+0000","last_peered":"2024-03-21T11:21:50.644656+0000","last_clean":"2024-03-21T11:21:50.644656+0000","last_became_active":"2024-03-21T11:20:48.651565+0000","last_became_peered":"2024-03-21T11:20:48.651565+0000","last_unstale":"2024-03-21T11:21:50.644656+0000","last_undegraded":"2024-03-21T11:21:50.644656+0000","last_fullsized":"2024-03-21T11:21:50.644656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:58.659889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645659+0000","last_change":"2024-03-21T11:20:48.667038+0000","last_active":"2024-03-21T11:21:50.645659+0000","last_peered":"2024-03-21T11:21:50.645659+0000","last_clean":"2024-03-21T11:21:50.645659+0000","last_became_active":"2024-03-21T11:20:48.666570+0000","last_became_peered":"2024-03-21T11:20:48.666570+0000","last_unstale":"2024-03-21T11:21:50.645659+0000","last_undegraded":"2024-03-21T11:21:50.645659+0000","last_fullsized":"2024-03-21T11:21:50.645659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:36:39.043697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.658024+0000","last_change":"2024-03-21T11:21:50.658024+0000","last_active":"2024-03-21T11:21:50.658024+0000","last_peered":"2024-03-21T11:21:50.658024+0000","last_clean":"2024-03-21T11:21:50.658024+0000","last_became_active":"2024-03-21T11:21:50.657390+0000","last_became_peered":"2024-03-21T11:21:50.657390+0000","last_unstale":"2024-03-21T11:21:50.658024+0000","last_undegraded":"2024-03-21T11:21:50.658024+0000","last_fullsized":"2024-03-21T11:21:50.658024+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:29.578366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.651546+0000","last_change":"2024-03-21T11:20:48.670139+0000","last_active":"2024-03-21T11:21:50.651546+0000","last_peered":"2024-03-21T11:21:50.651546+0000","last_clean":"2024-03-21T11:21:50.651546+0000","last_became_active":"2024-03-21T11:20:48.669410+0000","last_became_peered":"2024-03-21T11:20:48.669410+0000","last_unstale":"2024-03-21T11:21:50.651546+0000","last_undegraded":"2024-03-21T11:21:50.651546+0000","last_fullsized":"2024-03-21T11:21:50.651546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:35:18.799840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.640617+0000","last_change":"2024-03-21T11:20:48.655356+0000","last_active":"2024-03-21T11:21:50.640617+0000","last_peered":"2024-03-21T11:21:50.640617+0000","last_clean":"2024-03-21T11:21:50.640617+0000","last_became_active":"2024-03-21T11:20:48.655028+0000","last_became_peered":"2024-03-21T11:20:48.655028+0000","last_unstale":"2024-03-21T11:21:50.640617+0000","last_undegraded":"2024-03-21T11:21:50.640617+0000","last_fullsized":"2024-03-21T11:21:50.640617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:46:39.948982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.645395+0000","last_change":"2024-03-21T11:20:48.648840+0000","last_active":"2024-03-21T11:21:50.645395+0000","last_peered":"2024-03-21T11:21:50.645395+0000","last_clean":"2024-03-21T11:21:50.645395+0000","last_became_active":"2024-03-21T11:20:48.648502+0000","last_became_peered":"2024-03-21T11:20:48.648502+0000","last_unstale":"2024-03-21T11:21:50.645395+0000","last_undegraded":"2024-03-21T11:21:50.645395+0000","last_fullsized":"2024-03-21T11:21:50.645395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:33:14.824880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.653563+0000","last_change":"2024-03-21T11:20:48.672015+0000","last_active":"2024-03-21T11:21:50.653563+0000","last_peered":"2024-03-21T11:21:50.653563+0000","last_clean":"2024-03-21T11:21:50.653563+0000","last_became_active":"2024-03-21T11:20:48.671663+0000","last_became_peered":"2024-03-21T11:20:48.671663+0000","last_unstale":"2024-03-21T11:21:50.653563+0000","last_undegraded":"2024-03-21T11:21:50.653563+0000","last_fullsized":"2024-03-21T11:21:50.653563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:21:25.328698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.639265+0000","last_change":"2024-03-21T11:20:48.664114+0000","last_active":"2024-03-21T11:21:50.639265+0000","last_peered":"2024-03-21T11:21:50.639265+0000","last_clean":"2024-03-21T11:21:50.639265+0000","last_became_active":"2024-03-21T11:20:48.663819+0000","last_became_peered":"2024-03-21T11:20:48.663819+0000","last_unstale":"2024-03-21T11:21:50.639265+0000","last_undegraded":"2024-03-21T11:21:50.639265+0000","last_fullsized":"2024-03-21T11:21:50.639265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:08:08.201383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.653508+0000","last_change":"2024-03-21T11:20:48.671336+0000","last_active":"2024-03-21T11:21:50.653508+0000","last_peered":"2024-03-21T11:21:50.653508+0000","last_clean":"2024-03-21T11:21:50.653508+0000","last_became_active":"2024-03-21T11:20:48.670918+0000","last_became_peered":"2024-03-21T11:20:48.670918+0000","last_unstale":"2024-03-21T11:21:50.653508+0000","last_undegraded":"2024-03-21T11:21:50.653508+0000","last_fullsized":"2024-03-21T11:21:50.653508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:01:44.764584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.689840+0000","last_change":"2024-03-21T11:20:48.659339+0000","last_active":"2024-03-21T11:21:50.689840+0000","last_peered":"2024-03-21T11:21:50.689840+0000","last_clean":"2024-03-21T11:21:50.689840+0000","last_became_active":"2024-03-21T11:20:48.658989+0000","last_became_peered":"2024-03-21T11:20:48.658989+0000","last_unstale":"2024-03-21T11:21:50.689840+0000","last_undegraded":"2024-03-21T11:21:50.689840+0000","last_fullsized":"2024-03-21T11:21:50.689840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:44:09.599836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.653931+0000","last_change":"2024-03-21T11:20:48.668679+0000","last_active":"2024-03-21T11:21:50.653931+0000","last_peered":"2024-03-21T11:21:50.653931+0000","last_clean":"2024-03-21T11:21:50.653931+0000","last_became_active":"2024-03-21T11:20:48.667956+0000","last_became_peered":"2024-03-21T11:20:48.667956+0000","last_unstale":"2024-03-21T11:21:50.653931+0000","last_undegraded":"2024-03-21T11:21:50.653931+0000","last_fullsized":"2024-03-21T11:21:50.653931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:04.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T11:21:50.650600+0000","last_change":"2024-03-21T11:21:50.650600+0000","last_active":"2024-03-21T11:21:50.650600+0000","last_peered":"2024-03-21T11:21:50.650600+0000","last_clean":"2024-03-21T11:21:50.650600+0000","last_became_active":"2024-03-21T11:21:50.650221+0000","last_became_peered":"2024-03-21T11:21:50.650221+0000","last_unstale":"2024-03-21T11:21:50.650600+0000","last_undegraded":"2024-03-21T11:21:50.650600+0000","last_fullsized":"2024-03-21T11:21:50.650600+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:29:52.245281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247498,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71801,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":869,"num_read_kb":511,"num_write":243,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":215403,"data_compressed":5487,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":513062,"data_compressed":20821,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1175552,"data_stored":1168180,"data_compressed":58711,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776176,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":516697,"data_compressed":20821,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037815,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":512979,"data_compressed":20821,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299452,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":1109867,"data_compressed":56882,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462978,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":508105,"data_compressed":20821,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":573341,"data_compressed":22650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":509952,"data_compressed":20821,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":574343,"data_compressed":22650,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":507813,"data_compressed":20821,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888140,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":512570,"data_compressed":20821,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724615,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1100962,"data_compressed":56882,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65958,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66960,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":66333,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T11:22:45.203 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-21T11:22:45.203 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-21T11:22:45.203 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-21T11:22:45.203 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-21T11:22:45.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:44 smithi102 ceph-mon[36748]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:22:45.719 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:46.086 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:45 smithi047 ceph-mon[31351]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:22:46.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:45 smithi100 ceph-mon[36245]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:22:46.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:45 smithi102 ceph-mon[36748]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:22:47.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:46 smithi047 ceph-mon[31351]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:22:47.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:46 smithi100 ceph-mon[36245]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:22:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:46 smithi102 ceph-mon[36748]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:22:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.896+0000 7fbd3ea00700 1 -- 172.21.15.47:0/2258771786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd381074f0 msgr2=0x7fbd3810f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.896+0000 7fbd3ea00700 1 --2- 172.21.15.47:0/2258771786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd381074f0 0x7fbd3810f0c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000b0a0 tx=0x7fbd3000b3b0 comp rx=0 tx=0).stop 2024-03-21T11:22:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.896+0000 7fbd3ea00700 1 -- 172.21.15.47:0/2258771786 shutdown_connections 2024-03-21T11:22:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.896+0000 7fbd3ea00700 1 --2- 172.21.15.47:0/2258771786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd381074f0 0x7fbd3810f0c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.896+0000 7fbd3ea00700 1 --2- 172.21.15.47:0/2258771786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd38102410 0x7fbd381027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.896+0000 7fbd3ea00700 1 --2- 172.21.15.47:0/2258771786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd38102dc0 0x7fbd38106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.896+0000 7fbd3ea00700 1 -- 172.21.15.47:0/2258771786 >> 172.21.15.47:0/2258771786 conn(0x7fbd380fdc60 msgr2=0x7fbd38100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.896+0000 7fbd3ea00700 1 -- 172.21.15.47:0/2258771786 shutdown_connections 2024-03-21T11:22:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.896+0000 7fbd3ea00700 1 -- 172.21.15.47:0/2258771786 wait complete. 2024-03-21T11:22:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.897+0000 7fbd3ea00700 1 Processor -- start 2024-03-21T11:22:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.898+0000 7fbd3ea00700 1 -- start start 2024-03-21T11:22:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.898+0000 7fbd3ea00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd38102410 0x7fbd381a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.898+0000 7fbd3ea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd38102dc0 0x7fbd381a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.898+0000 7fbd3ea00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd381074f0 0x7fbd3819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.898+0000 7fbd3ea00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd38112530 con 0x7fbd38102dc0 2024-03-21T11:22:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.898+0000 7fbd3ea00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd381123b0 con 0x7fbd38102410 2024-03-21T11:22:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.898+0000 7fbd3ea00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd381126b0 con 0x7fbd381074f0 2024-03-21T11:22:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.898+0000 7fbd37e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd38102410 0x7fbd381a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.898+0000 7fbd3ca00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd381074f0 0x7fbd3819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.898+0000 7fbd37e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd38102410 0x7fbd381a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35418/0 (socket says 172.21.15.47:35418) 2024-03-21T11:22:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.898+0000 7fbd37400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd38102dc0 0x7fbd381a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.898+0000 7fbd37e00700 1 -- 172.21.15.47:0/3418683833 learned_addr learned my addr 172.21.15.47:0/3418683833 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.899+0000 7fbd37e00700 1 -- 172.21.15.47:0/3418683833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd381074f0 msgr2=0x7fbd3819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.899+0000 7fbd37e00700 1 --2- 172.21.15.47:0/3418683833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd381074f0 0x7fbd3819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.899+0000 7fbd37e00700 1 -- 172.21.15.47:0/3418683833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd38102dc0 msgr2=0x7fbd381a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.899+0000 7fbd37e00700 1 --2- 172.21.15.47:0/3418683833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd38102dc0 0x7fbd381a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.899+0000 7fbd37e00700 1 -- 172.21.15.47:0/3418683833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3819b0a0 con 0x7fbd38102410 2024-03-21T11:22:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.899+0000 7fbd37e00700 1 --2- 172.21.15.47:0/3418683833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd38102410 0x7fbd381a0780 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800e9a0 tx=0x7fbd2800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.899+0000 7fbd34c00700 1 -- 172.21.15.47:0/3418683833 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2800ca10 con 0x7fbd38102410 2024-03-21T11:22:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.899+0000 7fbd34c00700 1 -- 172.21.15.47:0/3418683833 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd2800cbb0 con 0x7fbd38102410 2024-03-21T11:22:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.899+0000 7fbd34c00700 1 -- 172.21.15.47:0/3418683833 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28018680 con 0x7fbd38102410 2024-03-21T11:22:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.899+0000 7fbd3ea00700 1 -- 172.21.15.47:0/3418683833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3819b390 con 0x7fbd38102410 2024-03-21T11:22:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.900+0000 7fbd3ea00700 1 -- 172.21.15.47:0/3418683833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd381a7650 con 0x7fbd38102410 2024-03-21T11:22:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.901+0000 7fbd3ea00700 1 -- 172.21.15.47:0/3418683833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd38068e50 con 0x7fbd38102410 2024-03-21T11:22:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.903+0000 7fbd34c00700 1 -- 172.21.15.47:0/3418683833 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fbd28010430 con 0x7fbd38102410 2024-03-21T11:22:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.904+0000 7fbd34c00700 1 --2- 172.21.15.47:0/3418683833 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbd1c0a30c0 0x7fbd1c0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.904+0000 7fbd34c00700 1 -- 172.21.15.47:0/3418683833 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fbd280cfaf0 con 0x7fbd38102410 2024-03-21T11:22:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.904+0000 7fbd37400700 1 --2- 172.21.15.47:0/3418683833 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbd1c0a30c0 0x7fbd1c0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.905+0000 7fbd37400700 1 --2- 172.21.15.47:0/3418683833 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbd1c0a30c0 0x7fbd1c0a5580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009680 tx=0x7fbd24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:47.912+0000 7fbd34c00700 1 -- 172.21.15.47:0/3418683833 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbd2809c400 con 0x7fbd38102410 2024-03-21T11:22:48.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:48.200+0000 7fbd3ea00700 1 -- 172.21.15.47:0/3418683833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbd381a78c0 con 0x7fbd38102410 2024-03-21T11:22:48.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:48.201+0000 7fbd34c00700 1 -- 172.21.15.47:0/3418683833 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fbd2809bd00 con 0x7fbd38102410 2024-03-21T11:22:48.202 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:22:48.202 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-21T11:22:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:48.204+0000 7fbd3ea00700 1 -- 172.21.15.47:0/3418683833 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbd1c0a30c0 msgr2=0x7fbd1c0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:48.204+0000 7fbd3ea00700 1 --2- 172.21.15.47:0/3418683833 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbd1c0a30c0 0x7fbd1c0a5580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009680 tx=0x7fbd24009260 comp rx=0 tx=0).stop 2024-03-21T11:22:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:48.204+0000 7fbd3ea00700 1 -- 172.21.15.47:0/3418683833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd38102410 msgr2=0x7fbd381a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:48.204+0000 7fbd3ea00700 1 --2- 172.21.15.47:0/3418683833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd38102410 0x7fbd381a0780 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800e9a0 tx=0x7fbd2800ecb0 comp rx=0 tx=0).stop 2024-03-21T11:22:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:48.204+0000 7fbd3ea00700 1 -- 172.21.15.47:0/3418683833 shutdown_connections 2024-03-21T11:22:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:48.204+0000 7fbd3ea00700 1 --2- 172.21.15.47:0/3418683833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd381074f0 0x7fbd3819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:48.204+0000 7fbd3ea00700 1 --2- 172.21.15.47:0/3418683833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd38102410 0x7fbd381a0780 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:48.204+0000 7fbd3ea00700 1 --2- 172.21.15.47:0/3418683833 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbd1c0a30c0 0x7fbd1c0a5580 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:48.204+0000 7fbd3ea00700 1 --2- 172.21.15.47:0/3418683833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbd38102dc0 0x7fbd381a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:48.204+0000 7fbd3ea00700 1 -- 172.21.15.47:0/3418683833 >> 172.21.15.47:0/3418683833 conn(0x7fbd380fdc60 msgr2=0x7fbd3810d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:48.204+0000 7fbd3ea00700 1 -- 172.21.15.47:0/3418683833 shutdown_connections 2024-03-21T11:22:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:48.204+0000 7fbd3ea00700 1 -- 172.21.15.47:0/3418683833 wait complete. 2024-03-21T11:22:48.828 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:48 smithi047 ceph-mon[31351]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:22:48.944 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-21T11:22:48.945 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:22:49.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:48 smithi100 ceph-mon[36245]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:22:49.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:48 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3418683833' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T11:22:49.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.170+0000 7fa919e00640 1 -- 172.21.15.47:0/448915739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa91410ea60 msgr2=0x7fa914110ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:49.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.170+0000 7fa919e00640 1 --2- 172.21.15.47:0/448915739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa91410ea60 0x7fa914110ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009ca0 tx=0x7fa90c02c100 comp rx=0 tx=0).stop 2024-03-21T11:22:49.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.170+0000 7fa919e00640 1 -- 172.21.15.47:0/448915739 shutdown_connections 2024-03-21T11:22:49.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.170+0000 7fa919e00640 1 --2- 172.21.15.47:0/448915739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa91410ea60 0x7fa914110ef0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.170+0000 7fa919e00640 1 --2- 172.21.15.47:0/448915739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa91410c0e0 0x7fa91410e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.170+0000 7fa919e00640 1 --2- 172.21.15.47:0/448915739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9141097b0 0x7fa91410bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.170+0000 7fa919e00640 1 -- 172.21.15.47:0/448915739 >> 172.21.15.47:0/448915739 conn(0x7fa91406e8b0 msgr2=0x7fa914070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:49.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.170+0000 7fa919e00640 1 -- 172.21.15.47:0/448915739 shutdown_connections 2024-03-21T11:22:49.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.170+0000 7fa919e00640 1 -- 172.21.15.47:0/448915739 wait complete. 2024-03-21T11:22:49.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.171+0000 7fa919e00640 1 Processor -- start 2024-03-21T11:22:49.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.171+0000 7fa919e00640 1 -- start start 2024-03-21T11:22:49.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.171+0000 7fa919e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9141097b0 0x7fa9141a2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:49.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.171+0000 7fa919e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa91410c0e0 0x7fa9141a2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:49.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.171+0000 7fa919e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa91410ea60 0x7fa9141a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:49.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.171+0000 7fa919e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa914115b80 con 0x7fa9141097b0 2024-03-21T11:22:49.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.171+0000 7fa919e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa914115a00 con 0x7fa91410ea60 2024-03-21T11:22:49.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.171+0000 7fa919e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa914115d00 con 0x7fa91410c0e0 2024-03-21T11:22:49.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.172+0000 7fa912000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa91410c0e0 0x7fa9141a2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:49.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.172+0000 7fa913400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa91410ea60 0x7fa9141a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:49.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.172+0000 7fa912a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9141097b0 0x7fa9141a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:49.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.172+0000 7fa912000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa91410c0e0 0x7fa9141a2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41456/0 (socket says 172.21.15.47:41456) 2024-03-21T11:22:49.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.172+0000 7fa912000640 1 -- 172.21.15.47:0/3450759571 learned_addr learned my addr 172.21.15.47:0/3450759571 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:49.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.172+0000 7fa912000640 1 -- 172.21.15.47:0/3450759571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa91410ea60 msgr2=0x7fa9141a99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:49.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.172+0000 7fa912000640 1 --2- 172.21.15.47:0/3450759571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa91410ea60 0x7fa9141a99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.172+0000 7fa912000640 1 -- 172.21.15.47:0/3450759571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9141097b0 msgr2=0x7fa9141a2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:49.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.172+0000 7fa912000640 1 --2- 172.21.15.47:0/3450759571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9141097b0 0x7fa9141a2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.172+0000 7fa912000640 1 -- 172.21.15.47:0/3450759571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9141aa050 con 0x7fa91410c0e0 2024-03-21T11:22:49.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.172+0000 7fa913400640 1 --2- 172.21.15.47:0/3450759571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa91410ea60 0x7fa9141a99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:49.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.172+0000 7fa912000640 1 --2- 172.21.15.47:0/3450759571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa91410c0e0 0x7fa9141a2970 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa90000e9c0 tx=0x7fa90000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:49.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.173+0000 7fa918a00640 1 -- 172.21.15.47:0/3450759571 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90000cd80 con 0x7fa91410c0e0 2024-03-21T11:22:49.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.173+0000 7fa918a00640 1 -- 172.21.15.47:0/3450759571 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa900005500 con 0x7fa91410c0e0 2024-03-21T11:22:49.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.173+0000 7fa919e00640 1 -- 172.21.15.47:0/3450759571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9141aa340 con 0x7fa91410c0e0 2024-03-21T11:22:49.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.173+0000 7fa918a00640 1 -- 172.21.15.47:0/3450759571 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900010690 con 0x7fa91410c0e0 2024-03-21T11:22:49.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.173+0000 7fa919e00640 1 -- 172.21.15.47:0/3450759571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9141aa8e0 con 0x7fa91410c0e0 2024-03-21T11:22:49.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.174+0000 7fa919e00640 1 -- 172.21.15.47:0/3450759571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa914110550 con 0x7fa91410c0e0 2024-03-21T11:22:49.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.177+0000 7fa918a00640 1 -- 172.21.15.47:0/3450759571 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fa9000046c0 con 0x7fa91410c0e0 2024-03-21T11:22:49.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.178+0000 7fa918a00640 1 --2- 172.21.15.47:0/3450759571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8f80a3030 0x7fa8f80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:49.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.178+0000 7fa918a00640 1 -- 172.21.15.47:0/3450759571 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa9000cfc80 con 0x7fa91410c0e0 2024-03-21T11:22:49.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.178+0000 7fa912a00640 1 --2- 172.21.15.47:0/3450759571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8f80a3030 0x7fa8f80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:49.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.179+0000 7fa912a00640 1 --2- 172.21.15.47:0/3450759571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8f80a3030 0x7fa8f80a54f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa9140738a0 tx=0x7fa904005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:49.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.179+0000 7fa918a00640 1 -- 172.21.15.47:0/3450759571 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa90009c590 con 0x7fa91410c0e0 2024-03-21T11:22:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:48 smithi102 ceph-mon[36748]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:22:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:48 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3418683833' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T11:22:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:48 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3418683833' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T11:22:49.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.387+0000 7fa919e00640 1 -- 172.21.15.47:0/3450759571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa91402d2c0 con 0x7fa91410c0e0 2024-03-21T11:22:49.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.389+0000 7fa918a00640 1 -- 172.21.15.47:0/3450759571 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24655 (secure 0 0 0) 0x7fa90009be90 con 0x7fa91410c0e0 2024-03-21T11:22:49.390 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:22:49.390 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":93,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:21:50.631013+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:22:49.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.392+0000 7fa919e00640 1 -- 172.21.15.47:0/3450759571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8f80a3030 msgr2=0x7fa8f80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:49.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.392+0000 7fa919e00640 1 --2- 172.21.15.47:0/3450759571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8f80a3030 0x7fa8f80a54f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa9140738a0 tx=0x7fa904005f90 comp rx=0 tx=0).stop 2024-03-21T11:22:49.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.392+0000 7fa919e00640 1 -- 172.21.15.47:0/3450759571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa91410c0e0 msgr2=0x7fa9141a2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:49.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.392+0000 7fa919e00640 1 --2- 172.21.15.47:0/3450759571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa91410c0e0 0x7fa9141a2970 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa90000e9c0 tx=0x7fa90000ee80 comp rx=0 tx=0).stop 2024-03-21T11:22:49.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.392+0000 7fa919e00640 1 -- 172.21.15.47:0/3450759571 shutdown_connections 2024-03-21T11:22:49.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.392+0000 7fa919e00640 1 --2- 172.21.15.47:0/3450759571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8f80a3030 0x7fa8f80a54f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.392+0000 7fa919e00640 1 --2- 172.21.15.47:0/3450759571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa91410ea60 0x7fa9141a99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.392+0000 7fa919e00640 1 --2- 172.21.15.47:0/3450759571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa91410c0e0 0x7fa9141a2970 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.392+0000 7fa919e00640 1 --2- 172.21.15.47:0/3450759571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9141097b0 0x7fa9141a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.392+0000 7fa919e00640 1 -- 172.21.15.47:0/3450759571 >> 172.21.15.47:0/3450759571 conn(0x7fa91406e8b0 msgr2=0x7fa914113520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:49.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.392+0000 7fa919e00640 1 -- 172.21.15.47:0/3450759571 shutdown_connections 2024-03-21T11:22:49.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.392+0000 7fa919e00640 1 -- 172.21.15.47:0/3450759571 wait complete. 2024-03-21T11:22:49.413 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T11:15:35.545245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T11:20:47.016174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T11:20:48.108506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T11:20:52.254154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T11:22:49.413 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T11:22:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.638+0000 7f1741c00640 1 -- 172.21.15.47:0/3582705724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173c105350 msgr2=0x7f173c10bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.638+0000 7f1741c00640 1 --2- 172.21.15.47:0/3582705724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173c105350 0x7f173c10bbe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1734009ca0 tx=0x7f173402c100 comp rx=0 tx=0).stop 2024-03-21T11:22:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.638+0000 7f1741c00640 1 -- 172.21.15.47:0/3582705724 shutdown_connections 2024-03-21T11:22:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.638+0000 7f1741c00640 1 --2- 172.21.15.47:0/3582705724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173c105350 0x7f173c10bbe0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.638+0000 7f1741c00640 1 --2- 172.21.15.47:0/3582705724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f173c104990 0x7f173c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.638+0000 7f1741c00640 1 --2- 172.21.15.47:0/3582705724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f173c1037a0 0x7f173c103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.638+0000 7f1741c00640 1 -- 172.21.15.47:0/3582705724 >> 172.21.15.47:0/3582705724 conn(0x7f173c0fed10 msgr2=0x7f173c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.638+0000 7f1741c00640 1 -- 172.21.15.47:0/3582705724 shutdown_connections 2024-03-21T11:22:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.638+0000 7f1741c00640 1 -- 172.21.15.47:0/3582705724 wait complete. 2024-03-21T11:22:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.639+0000 7f1741c00640 1 Processor -- start 2024-03-21T11:22:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.640+0000 7f1741c00640 1 -- start start 2024-03-21T11:22:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.640+0000 7f1741c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f173c1037a0 0x7f173c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.640+0000 7f1741c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f173c104990 0x7f173c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.640+0000 7f1741c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173c105350 0x7f173c1a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.640+0000 7f1741c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c10f400 con 0x7f173c1037a0 2024-03-21T11:22:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.640+0000 7f1741c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c10f280 con 0x7f173c105350 2024-03-21T11:22:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.640+0000 7f1741c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c10f580 con 0x7f173c104990 2024-03-21T11:22:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.640+0000 7f1741200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173c105350 0x7f173c1a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.640+0000 7f173be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f173c1037a0 0x7f173c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.640+0000 7f1741200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173c105350 0x7f173c1a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52684/0 (socket says 172.21.15.47:52684) 2024-03-21T11:22:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.640+0000 7f1741200640 1 -- 172.21.15.47:0/3487024963 learned_addr learned my addr 172.21.15.47:0/3487024963 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.640+0000 7f173b400640 1 --2- 172.21.15.47:0/3487024963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f173c104990 0x7f173c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.641+0000 7f1741200640 1 -- 172.21.15.47:0/3487024963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f173c104990 msgr2=0x7f173c199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.641+0000 7f1741200640 1 --2- 172.21.15.47:0/3487024963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f173c104990 0x7f173c199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.641+0000 7f1741200640 1 -- 172.21.15.47:0/3487024963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f173c1037a0 msgr2=0x7f173c199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.641+0000 7f1741200640 1 --2- 172.21.15.47:0/3487024963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f173c1037a0 0x7f173c199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.641+0000 7f1741200640 1 -- 172.21.15.47:0/3487024963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c1a1630 con 0x7f173c105350 2024-03-21T11:22:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.641+0000 7f1741200640 1 --2- 172.21.15.47:0/3487024963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173c105350 0x7f173c1a0fc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1734037040 tx=0x7f1734035d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.641+0000 7f1738c00640 1 -- 172.21.15.47:0/3487024963 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734039040 con 0x7f173c105350 2024-03-21T11:22:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.641+0000 7f1738c00640 1 -- 172.21.15.47:0/3487024963 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1734044070 con 0x7f173c105350 2024-03-21T11:22:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.641+0000 7f1738c00640 1 -- 172.21.15.47:0/3487024963 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734004460 con 0x7f173c105350 2024-03-21T11:22:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.641+0000 7f1741c00640 1 -- 172.21.15.47:0/3487024963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c1a18c0 con 0x7f173c105350 2024-03-21T11:22:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.642+0000 7f1741c00640 1 -- 172.21.15.47:0/3487024963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c1a1e00 con 0x7f173c105350 2024-03-21T11:22:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.643+0000 7f1741c00640 1 -- 172.21.15.47:0/3487024963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173c10aac0 con 0x7f173c105350 2024-03-21T11:22:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.646+0000 7f1738c00640 1 -- 172.21.15.47:0/3487024963 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f1734004600 con 0x7f173c105350 2024-03-21T11:22:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.648+0000 7f1738c00640 1 --2- 172.21.15.47:0/3487024963 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17200a3030 0x7f17200a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.648+0000 7f1738c00640 1 -- 172.21.15.47:0/3487024963 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f17340f1da0 con 0x7f173c105350 2024-03-21T11:22:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.648+0000 7f173be00640 1 --2- 172.21.15.47:0/3487024963 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17200a3030 0x7f17200a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.649+0000 7f173be00640 1 --2- 172.21.15.47:0/3487024963 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17200a3030 0x7f17200a54f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f173c1047f0 tx=0x7f172c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.651+0000 7f1738c00640 1 -- 172.21.15.47:0/3487024963 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f17340be820 con 0x7f173c105350 2024-03-21T11:22:49.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.856+0000 7f1738c00640 1 -- 172.21.15.47:0/3487024963 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f17340f7020 con 0x7f173c105350 2024-03-21T11:22:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.867+0000 7f1741c00640 1 -- 172.21.15.47:0/3487024963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f173c02d240 con 0x7f173c105350 2024-03-21T11:22:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.868+0000 7f1738c00640 1 -- 172.21.15.47:0/3487024963 <== mon.2 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17340b3450 con 0x7f173c105350 2024-03-21T11:22:49.903 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T11:22:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.871+0000 7f1741c00640 1 -- 172.21.15.47:0/3487024963 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17200a3030 msgr2=0x7f17200a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.871+0000 7f1741c00640 1 --2- 172.21.15.47:0/3487024963 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17200a3030 0x7f17200a54f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f173c1047f0 tx=0x7f172c005f90 comp rx=0 tx=0).stop 2024-03-21T11:22:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.871+0000 7f1741c00640 1 -- 172.21.15.47:0/3487024963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173c105350 msgr2=0x7f173c1a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.871+0000 7f1741c00640 1 --2- 172.21.15.47:0/3487024963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173c105350 0x7f173c1a0fc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1734037040 tx=0x7f1734035d90 comp rx=0 tx=0).stop 2024-03-21T11:22:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.871+0000 7f1741c00640 1 -- 172.21.15.47:0/3487024963 shutdown_connections 2024-03-21T11:22:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.871+0000 7f1741c00640 1 --2- 172.21.15.47:0/3487024963 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17200a3030 0x7f17200a54f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.871+0000 7f1741c00640 1 --2- 172.21.15.47:0/3487024963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f173c105350 0x7f173c1a0fc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.871+0000 7f1741c00640 1 --2- 172.21.15.47:0/3487024963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f173c104990 0x7f173c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.871+0000 7f1741c00640 1 --2- 172.21.15.47:0/3487024963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f173c1037a0 0x7f173c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.871+0000 7f1741c00640 1 -- 172.21.15.47:0/3487024963 >> 172.21.15.47:0/3487024963 conn(0x7f173c0fed10 msgr2=0x7f173c107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.871+0000 7f1741c00640 1 -- 172.21.15.47:0/3487024963 shutdown_connections 2024-03-21T11:22:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:49.872+0000 7f1741c00640 1 -- 172.21.15.47:0/3487024963 wait complete. 2024-03-21T11:22:49.906 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T11:22:50.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.122+0000 7efead400640 1 -- 172.21.15.47:0/2008212338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efea8105370 msgr2=0x7efea810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:50.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.122+0000 7efead400640 1 --2- 172.21.15.47:0/2008212338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efea8105370 0x7efea810bc00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efea0009ca0 tx=0x7efea002c100 comp rx=0 tx=0).stop 2024-03-21T11:22:50.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.122+0000 7efead400640 1 -- 172.21.15.47:0/2008212338 shutdown_connections 2024-03-21T11:22:50.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.122+0000 7efead400640 1 --2- 172.21.15.47:0/2008212338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efea8105370 0x7efea810bc00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.122+0000 7efead400640 1 --2- 172.21.15.47:0/2008212338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efea81049b0 0x7efea8104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.122+0000 7efead400640 1 --2- 172.21.15.47:0/2008212338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efea81037c0 0x7efea8103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.122+0000 7efead400640 1 -- 172.21.15.47:0/2008212338 >> 172.21.15.47:0/2008212338 conn(0x7efea80fed50 msgr2=0x7efea8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:50.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.122+0000 7efead400640 1 -- 172.21.15.47:0/2008212338 shutdown_connections 2024-03-21T11:22:50.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.122+0000 7efead400640 1 -- 172.21.15.47:0/2008212338 wait complete. 2024-03-21T11:22:50.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.124+0000 7efead400640 1 Processor -- start 2024-03-21T11:22:50.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.124+0000 7efead400640 1 -- start start 2024-03-21T11:22:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.124+0000 7efead400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efea81037c0 0x7efea8199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.124+0000 7efead400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efea81049b0 0x7efea8199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.124+0000 7efead400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efea8105370 0x7efea81a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.124+0000 7efead400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea810f420 con 0x7efea81037c0 2024-03-21T11:22:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.124+0000 7efead400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea810f2a0 con 0x7efea81049b0 2024-03-21T11:22:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.124+0000 7efead400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea810f5a0 con 0x7efea8105370 2024-03-21T11:22:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.125+0000 7efe9fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efea81049b0 0x7efea8199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.125+0000 7efea7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efea81037c0 0x7efea8199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.125+0000 7efe9fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efea81049b0 0x7efea8199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52700/0 (socket says 172.21.15.47:52700) 2024-03-21T11:22:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.125+0000 7efe9fe00640 1 -- 172.21.15.47:0/2484421776 learned_addr learned my addr 172.21.15.47:0/2484421776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.125+0000 7efeaca00640 1 --2- 172.21.15.47:0/2484421776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efea8105370 0x7efea81a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.125+0000 7efe9fe00640 1 -- 172.21.15.47:0/2484421776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efea8105370 msgr2=0x7efea81a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:50.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.125+0000 7efe9fe00640 1 --2- 172.21.15.47:0/2484421776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efea8105370 0x7efea81a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.125+0000 7efe9fe00640 1 -- 172.21.15.47:0/2484421776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efea81037c0 msgr2=0x7efea8199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:50.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.125+0000 7efe9fe00640 1 --2- 172.21.15.47:0/2484421776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efea81037c0 0x7efea8199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.125+0000 7efe9fe00640 1 -- 172.21.15.47:0/2484421776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea81a1660 con 0x7efea81049b0 2024-03-21T11:22:50.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.125+0000 7efeaca00640 1 --2- 172.21.15.47:0/2484421776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efea8105370 0x7efea81a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:50.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.126+0000 7efe9fe00640 1 --2- 172.21.15.47:0/2484421776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efea81049b0 0x7efea8199f80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efe9000e9f0 tx=0x7efe9000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:50.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.126+0000 7efea5600640 1 -- 172.21.15.47:0/2484421776 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9000cd80 con 0x7efea81049b0 2024-03-21T11:22:50.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.126+0000 7efea5600640 1 -- 172.21.15.47:0/2484421776 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe90010910 con 0x7efea81049b0 2024-03-21T11:22:50.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.126+0000 7efead400640 1 -- 172.21.15.47:0/2484421776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea81a1950 con 0x7efea81049b0 2024-03-21T11:22:50.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.126+0000 7efea5600640 1 -- 172.21.15.47:0/2484421776 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90018990 con 0x7efea81049b0 2024-03-21T11:22:50.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.126+0000 7efead400640 1 -- 172.21.15.47:0/2484421776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea81a1ea0 con 0x7efea81049b0 2024-03-21T11:22:50.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.127+0000 7efead400640 1 -- 172.21.15.47:0/2484421776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea810aae0 con 0x7efea81049b0 2024-03-21T11:22:50.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.129+0000 7efea5600640 1 -- 172.21.15.47:0/2484421776 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7efe90010ac0 con 0x7efea81049b0 2024-03-21T11:22:50.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.130+0000 7efea5600640 1 --2- 172.21.15.47:0/2484421776 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe8c0a2f20 0x7efe8c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:50.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.130+0000 7efea5600640 1 -- 172.21.15.47:0/2484421776 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efe900cfad0 con 0x7efea81049b0 2024-03-21T11:22:50.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.130+0000 7efea7e00640 1 --2- 172.21.15.47:0/2484421776 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe8c0a2f20 0x7efe8c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:50.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.131+0000 7efea7e00640 1 --2- 172.21.15.47:0/2484421776 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe8c0a2f20 0x7efe8c0a53e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efea8104810 tx=0x7efe98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:50.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.134+0000 7efea5600640 1 -- 172.21.15.47:0/2484421776 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efe9009bec0 con 0x7efea81049b0 2024-03-21T11:22:50.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:49 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3450759571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:22:50.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-21T11:22:50.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-21T11:22:50.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:49 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3450759571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:22:50.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-21T11:22:50.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-21T11:22:50.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:49 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3450759571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:22:50.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-21T11:22:50.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-21T11:22:50.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.337+0000 7efead400640 1 -- 172.21.15.47:0/2484421776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efea802d070 con 0x7efea81049b0 2024-03-21T11:22:50.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.338+0000 7efea5600640 1 -- 172.21.15.47:0/2484421776 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe9009ff60 con 0x7efea81049b0 2024-03-21T11:22:50.339 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:22:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.341+0000 7efead400640 1 -- 172.21.15.47:0/2484421776 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe8c0a2f20 msgr2=0x7efe8c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.341+0000 7efead400640 1 --2- 172.21.15.47:0/2484421776 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe8c0a2f20 0x7efe8c0a53e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efea8104810 tx=0x7efe98005f90 comp rx=0 tx=0).stop 2024-03-21T11:22:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.341+0000 7efead400640 1 -- 172.21.15.47:0/2484421776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efea81049b0 msgr2=0x7efea8199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.341+0000 7efead400640 1 --2- 172.21.15.47:0/2484421776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efea81049b0 0x7efea8199f80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efe9000e9f0 tx=0x7efe9000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.341+0000 7efead400640 1 -- 172.21.15.47:0/2484421776 shutdown_connections 2024-03-21T11:22:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.341+0000 7efead400640 1 --2- 172.21.15.47:0/2484421776 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe8c0a2f20 0x7efe8c0a53e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.341+0000 7efead400640 1 --2- 172.21.15.47:0/2484421776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efea8105370 0x7efea81a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.341+0000 7efead400640 1 --2- 172.21.15.47:0/2484421776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efea81049b0 0x7efea8199f80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.341+0000 7efead400640 1 --2- 172.21.15.47:0/2484421776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efea81037c0 0x7efea8199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.341+0000 7efead400640 1 -- 172.21.15.47:0/2484421776 >> 172.21.15.47:0/2484421776 conn(0x7efea80fed50 msgr2=0x7efea8107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:50.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.341+0000 7efead400640 1 -- 172.21.15.47:0/2484421776 shutdown_connections 2024-03-21T11:22:50.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.341+0000 7efead400640 1 -- 172.21.15.47:0/2484421776 wait complete. 2024-03-21T11:22:50.362 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T11:22:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.589+0000 7f21b7c00640 1 -- 172.21.15.47:0/3165955318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21b0108eb0 msgr2=0x7f21b0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.589+0000 7f21b7c00640 1 --2- 172.21.15.47:0/3165955318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21b0108eb0 0x7f21b0109330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2198009900 tx=0x7f219802bc30 comp rx=0 tx=0).stop 2024-03-21T11:22:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.589+0000 7f21b7c00640 1 -- 172.21.15.47:0/3165955318 shutdown_connections 2024-03-21T11:22:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.589+0000 7f21b7c00640 1 --2- 172.21.15.47:0/3165955318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b0109870 0x7f21b0110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.589+0000 7f21b7c00640 1 --2- 172.21.15.47:0/3165955318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21b0108eb0 0x7f21b0109330 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.589+0000 7f21b7c00640 1 --2- 172.21.15.47:0/3165955318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21b0107cc0 0x7f21b01080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.589+0000 7f21b7c00640 1 -- 172.21.15.47:0/3165955318 >> 172.21.15.47:0/3165955318 conn(0x7f21b0076c60 msgr2=0x7f21b00790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.589+0000 7f21b7c00640 1 -- 172.21.15.47:0/3165955318 shutdown_connections 2024-03-21T11:22:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.589+0000 7f21b7c00640 1 -- 172.21.15.47:0/3165955318 wait complete. 2024-03-21T11:22:50.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.590+0000 7f21b7c00640 1 Processor -- start 2024-03-21T11:22:50.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.591+0000 7f21b7c00640 1 -- start start 2024-03-21T11:22:50.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.591+0000 7f21b7c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21b0107cc0 0x7f21b019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:50.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.591+0000 7f21b7c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b0108eb0 0x7f21b019e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:50.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.591+0000 7f21b7c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21b0109870 0x7f21b01a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:50.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.591+0000 7f21b7c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b0113970 con 0x7f21b0107cc0 2024-03-21T11:22:50.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.591+0000 7f21b7c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b01137f0 con 0x7f21b0108eb0 2024-03-21T11:22:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.591+0000 7f21b7c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b0113af0 con 0x7f21b0109870 2024-03-21T11:22:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.591+0000 7f21b5200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21b0107cc0 0x7f21b019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.592+0000 7f21b5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21b0109870 0x7f21b01a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.592+0000 7f21afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b0108eb0 0x7f21b019e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.592+0000 7f21b5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21b0109870 0x7f21b01a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41524/0 (socket says 172.21.15.47:41524) 2024-03-21T11:22:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.592+0000 7f21b5c00640 1 -- 172.21.15.47:0/3963509451 learned_addr learned my addr 172.21.15.47:0/3963509451 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:50.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.592+0000 7f21b5c00640 1 -- 172.21.15.47:0/3963509451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b0108eb0 msgr2=0x7f21b019e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.592+0000 7f21b5c00640 1 --2- 172.21.15.47:0/3963509451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b0108eb0 0x7f21b019e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.592+0000 7f21b5c00640 1 -- 172.21.15.47:0/3963509451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21b0107cc0 msgr2=0x7f21b019df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.592+0000 7f21b5c00640 1 --2- 172.21.15.47:0/3963509451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21b0107cc0 0x7f21b019df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.592+0000 7f21b5c00640 1 -- 172.21.15.47:0/3963509451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b01a5bb0 con 0x7f21b0109870 2024-03-21T11:22:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.592+0000 7f21b5c00640 1 --2- 172.21.15.47:0/3963509451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21b0109870 0x7f21b01a5540 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f21a800e970 tx=0x7f21a800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.592+0000 7f21ad600640 1 -- 172.21.15.47:0/3963509451 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a800ccc0 con 0x7f21b0109870 2024-03-21T11:22:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.593+0000 7f21ad600640 1 -- 172.21.15.47:0/3963509451 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21a800ce60 con 0x7f21b0109870 2024-03-21T11:22:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.593+0000 7f21ad600640 1 -- 172.21.15.47:0/3963509451 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8010490 con 0x7f21b0109870 2024-03-21T11:22:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.593+0000 7f21b7c00640 1 -- 172.21.15.47:0/3963509451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b01a5ea0 con 0x7f21b0109870 2024-03-21T11:22:50.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.593+0000 7f21b7c00640 1 -- 172.21.15.47:0/3963509451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b01a6360 con 0x7f21b0109870 2024-03-21T11:22:50.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.594+0000 7f21b7c00640 1 -- 172.21.15.47:0/3963509451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b010efe0 con 0x7f21b0109870 2024-03-21T11:22:50.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.596+0000 7f21ad600640 1 -- 172.21.15.47:0/3963509451 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f21a8044b40 con 0x7f21b0109870 2024-03-21T11:22:50.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.597+0000 7f21ad600640 1 --2- 172.21.15.47:0/3963509451 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f21940a2fa0 0x7f21940a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:50.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.598+0000 7f21ad600640 1 -- 172.21.15.47:0/3963509451 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f21a80cfbc0 con 0x7f21b0109870 2024-03-21T11:22:50.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.598+0000 7f21b5200640 1 --2- 172.21.15.47:0/3963509451 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f21940a2fa0 0x7f21940a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:50.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.598+0000 7f21b5200640 1 --2- 172.21.15.47:0/3963509451 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f21940a2fa0 0x7f21940a5460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f21b0108d10 tx=0x7f21a0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:50.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.599+0000 7f21ad600640 1 -- 172.21.15.47:0/3963509451 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f21a809c600 con 0x7f21b0109870 2024-03-21T11:22:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.803+0000 7f21b7c00640 1 -- 172.21.15.47:0/3963509451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f21b002d200 con 0x7f21b0109870 2024-03-21T11:22:50.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.804+0000 7f21ad600640 1 -- 172.21.15.47:0/3963509451 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f21a809bf00 con 0x7f21b0109870 2024-03-21T11:22:50.805 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T11:22:50.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.807+0000 7f21b7c00640 1 -- 172.21.15.47:0/3963509451 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f21940a2fa0 msgr2=0x7f21940a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.807+0000 7f21b7c00640 1 --2- 172.21.15.47:0/3963509451 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f21940a2fa0 0x7f21940a5460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f21b0108d10 tx=0x7f21a0009500 comp rx=0 tx=0).stop 2024-03-21T11:22:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.807+0000 7f21b7c00640 1 -- 172.21.15.47:0/3963509451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21b0109870 msgr2=0x7f21b01a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.807+0000 7f21b7c00640 1 --2- 172.21.15.47:0/3963509451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21b0109870 0x7f21b01a5540 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f21a800e970 tx=0x7f21a800ed70 comp rx=0 tx=0).stop 2024-03-21T11:22:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.807+0000 7f21b7c00640 1 -- 172.21.15.47:0/3963509451 shutdown_connections 2024-03-21T11:22:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.807+0000 7f21b7c00640 1 --2- 172.21.15.47:0/3963509451 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f21940a2fa0 0x7f21940a5460 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.807+0000 7f21b7c00640 1 --2- 172.21.15.47:0/3963509451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21b0109870 0x7f21b01a5540 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.807+0000 7f21b7c00640 1 --2- 172.21.15.47:0/3963509451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b0108eb0 0x7f21b019e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.807+0000 7f21b7c00640 1 --2- 172.21.15.47:0/3963509451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21b0107cc0 0x7f21b019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:50.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.807+0000 7f21b7c00640 1 -- 172.21.15.47:0/3963509451 >> 172.21.15.47:0/3963509451 conn(0x7f21b0076c60 msgr2=0x7f21b010c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.807+0000 7f21b7c00640 1 -- 172.21.15.47:0/3963509451 shutdown_connections 2024-03-21T11:22:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:50.808+0000 7f21b7c00640 1 -- 172.21.15.47:0/3963509451 wait complete. 2024-03-21T11:22:50.830 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T11:22:51.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.043+0000 7f6a65200640 1 -- 172.21.15.47:0/4196716357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a60077c40 msgr2=0x7f6a600784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:51.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.044+0000 7f6a65200640 1 --2- 172.21.15.47:0/4196716357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a60077c40 0x7f6a600784e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009ca0 tx=0x7f6a5802c100 comp rx=0 tx=0).stop 2024-03-21T11:22:51.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.044+0000 7f6a65200640 1 -- 172.21.15.47:0/4196716357 shutdown_connections 2024-03-21T11:22:51.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.044+0000 7f6a65200640 1 --2- 172.21.15.47:0/4196716357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a60077c40 0x7f6a600784e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.044+0000 7f6a65200640 1 --2- 172.21.15.47:0/4196716357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a600772a0 0x7f6a60077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.044+0000 7f6a65200640 1 --2- 172.21.15.47:0/4196716357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a60076060 0x7f6a60076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.044+0000 7f6a65200640 1 -- 172.21.15.47:0/4196716357 >> 172.21.15.47:0/4196716357 conn(0x7f6a600fee50 msgr2=0x7f6a60101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:51.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.044+0000 7f6a65200640 1 -- 172.21.15.47:0/4196716357 shutdown_connections 2024-03-21T11:22:51.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.044+0000 7f6a65200640 1 -- 172.21.15.47:0/4196716357 wait complete. 2024-03-21T11:22:51.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.045+0000 7f6a65200640 1 Processor -- start 2024-03-21T11:22:51.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.045+0000 7f6a65200640 1 -- start start 2024-03-21T11:22:51.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.045+0000 7f6a65200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a60076060 0x7f6a6019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:51.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.045+0000 7f6a65200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a600772a0 0x7f6a6019e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:51.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.045+0000 7f6a65200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a60077c40 0x7f6a601a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:51.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.045+0000 7f6a65200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a60113970 con 0x7f6a600772a0 2024-03-21T11:22:51.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.045+0000 7f6a65200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a601137f0 con 0x7f6a60076060 2024-03-21T11:22:51.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.045+0000 7f6a65200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a60113af0 con 0x7f6a60077c40 2024-03-21T11:22:51.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.046+0000 7f6a5de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a60076060 0x7f6a6019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:51.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.046+0000 7f6a5de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a60076060 0x7f6a6019df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52752/0 (socket says 172.21.15.47:52752) 2024-03-21T11:22:51.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.046+0000 7f6a5de00640 1 -- 172.21.15.47:0/1627945632 learned_addr learned my addr 172.21.15.47:0/1627945632 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:51.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.046+0000 7f6a5e800640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a60077c40 0x7f6a601a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:51.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.046+0000 7f6a57e00640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a600772a0 0x7f6a6019e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:51.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.046+0000 7f6a5de00640 1 -- 172.21.15.47:0/1627945632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a60077c40 msgr2=0x7f6a601a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:51.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.046+0000 7f6a5de00640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a60077c40 0x7f6a601a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.046+0000 7f6a5de00640 1 -- 172.21.15.47:0/1627945632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a600772a0 msgr2=0x7f6a6019e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:51.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.046+0000 7f6a5de00640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a600772a0 0x7f6a6019e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.046+0000 7f6a5de00640 1 -- 172.21.15.47:0/1627945632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a601a5ba0 con 0x7f6a60076060 2024-03-21T11:22:51.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.046+0000 7f6a5e800640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a60077c40 0x7f6a601a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:51.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.046+0000 7f6a5de00640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a60076060 0x7f6a6019df80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000ea60 tx=0x7f6a5000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:51.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.047+0000 7f6a56a00640 1 -- 172.21.15.47:0/1627945632 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5000cde0 con 0x7f6a60076060 2024-03-21T11:22:51.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.047+0000 7f6a56a00640 1 -- 172.21.15.47:0/1627945632 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a500035f0 con 0x7f6a60076060 2024-03-21T11:22:51.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.047+0000 7f6a56a00640 1 -- 172.21.15.47:0/1627945632 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50010670 con 0x7f6a60076060 2024-03-21T11:22:51.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.047+0000 7f6a65200640 1 -- 172.21.15.47:0/1627945632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a601a5e90 con 0x7f6a60076060 2024-03-21T11:22:51.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.047+0000 7f6a65200640 1 -- 172.21.15.47:0/1627945632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a601a6350 con 0x7f6a60076060 2024-03-21T11:22:51.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.048+0000 7f6a65200640 1 -- 172.21.15.47:0/1627945632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a60110250 con 0x7f6a60076060 2024-03-21T11:22:51.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.050+0000 7f6a56a00640 1 -- 172.21.15.47:0/1627945632 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6a500037a0 con 0x7f6a60076060 2024-03-21T11:22:51.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.051+0000 7f6a56a00640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a440a2fa0 0x7f6a440a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:51.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.051+0000 7f6a56a00640 1 -- 172.21.15.47:0/1627945632 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6a500cfa50 con 0x7f6a60076060 2024-03-21T11:22:51.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.052+0000 7f6a57e00640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a440a2fa0 0x7f6a440a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:51.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.052+0000 7f6a57e00640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a440a2fa0 0x7f6a440a5460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6a480095f0 tx=0x7f6a48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:51.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.055+0000 7f6a56a00640 1 -- 172.21.15.47:0/1627945632 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6a5009c490 con 0x7f6a60076060 2024-03-21T11:22:51.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:50 smithi100 ceph-mon[36245]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-21T11:22:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-21T11:22:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:50 smithi100 ceph-mon[36245]: osdmap e94: 12 total, 12 up, 12 in 2024-03-21T11:22:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:50 smithi100 ceph-mon[36245]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-21T11:22:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:50 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3487024963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:22:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:50 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2484421776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:22:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:50 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3963509451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:22:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:50 smithi102 ceph-mon[36748]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-21T11:22:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-21T11:22:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:50 smithi102 ceph-mon[36748]: osdmap e94: 12 total, 12 up, 12 in 2024-03-21T11:22:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:50 smithi102 ceph-mon[36748]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-21T11:22:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:50 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3487024963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:22:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:50 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2484421776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:22:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:50 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3963509451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:22:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.258+0000 7f6a65200640 1 -- 172.21.15.47:0/1627945632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6a60062340 con 0x7f6a60076060 2024-03-21T11:22:51.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.258+0000 7f6a56a00640 1 -- 172.21.15.47:0/1627945632 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6a5009bd90 con 0x7f6a60076060 2024-03-21T11:22:51.259 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:22:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.261+0000 7f6a65200640 1 -- 172.21.15.47:0/1627945632 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a440a2fa0 msgr2=0x7f6a440a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.261+0000 7f6a65200640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a440a2fa0 0x7f6a440a5460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6a480095f0 tx=0x7f6a48005f90 comp rx=0 tx=0).stop 2024-03-21T11:22:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.261+0000 7f6a65200640 1 -- 172.21.15.47:0/1627945632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a60076060 msgr2=0x7f6a6019df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.261+0000 7f6a65200640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a60076060 0x7f6a6019df80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000ea60 tx=0x7f6a5000ef20 comp rx=0 tx=0).stop 2024-03-21T11:22:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.261+0000 7f6a65200640 1 -- 172.21.15.47:0/1627945632 shutdown_connections 2024-03-21T11:22:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.261+0000 7f6a65200640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a440a2fa0 0x7f6a440a5460 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.261+0000 7f6a65200640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a60077c40 0x7f6a601a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.261+0000 7f6a65200640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a600772a0 0x7f6a6019e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.261+0000 7f6a65200640 1 --2- 172.21.15.47:0/1627945632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a60076060 0x7f6a6019df80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.261+0000 7f6a65200640 1 -- 172.21.15.47:0/1627945632 >> 172.21.15.47:0/1627945632 conn(0x7f6a600fee50 msgr2=0x7f6a60100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.261+0000 7f6a65200640 1 -- 172.21.15.47:0/1627945632 shutdown_connections 2024-03-21T11:22:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.261+0000 7f6a65200640 1 -- 172.21.15.47:0/1627945632 wait complete. 2024-03-21T11:22:51.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:50 smithi047 ceph-mon[31351]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:22:51.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-21T11:22:51.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-21T11:22:51.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:50 smithi047 ceph-mon[31351]: osdmap e94: 12 total, 12 up, 12 in 2024-03-21T11:22:51.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:50 smithi047 ceph-mon[31351]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-21T11:22:51.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:50 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3487024963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:22:51.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:50 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2484421776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:22:51.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:50 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3963509451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:22:51.282 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:22:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.487+0000 7fd926e00640 1 -- 172.21.15.47:0/884141217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9201052a0 msgr2=0x7fd92010bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.487+0000 7fd926e00640 1 --2- 172.21.15.47:0/884141217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9201052a0 0x7fd92010bb30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd918009ca0 tx=0x7fd91802c0f0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.487+0000 7fd926e00640 1 -- 172.21.15.47:0/884141217 shutdown_connections 2024-03-21T11:22:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.487+0000 7fd926e00640 1 --2- 172.21.15.47:0/884141217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9201052a0 0x7fd92010bb30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.487+0000 7fd926e00640 1 --2- 172.21.15.47:0/884141217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9201048e0 0x7fd920104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.487+0000 7fd926e00640 1 --2- 172.21.15.47:0/884141217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9201036f0 0x7fd920103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.487+0000 7fd926e00640 1 -- 172.21.15.47:0/884141217 >> 172.21.15.47:0/884141217 conn(0x7fd9200fec80 msgr2=0x7fd9201010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:51.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.487+0000 7fd926e00640 1 -- 172.21.15.47:0/884141217 shutdown_connections 2024-03-21T11:22:51.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.487+0000 7fd926e00640 1 -- 172.21.15.47:0/884141217 wait complete. 2024-03-21T11:22:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.488+0000 7fd926e00640 1 Processor -- start 2024-03-21T11:22:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.488+0000 7fd926e00640 1 -- start start 2024-03-21T11:22:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd926e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9201036f0 0x7fd920079450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd926e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9201048e0 0x7fd920079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd926e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9201052a0 0x7fd920075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd926e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92010f350 con 0x7fd9201052a0 2024-03-21T11:22:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd926e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92010f1d0 con 0x7fd9201036f0 2024-03-21T11:22:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd926e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92010f4d0 con 0x7fd9201048e0 2024-03-21T11:22:51.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd925a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9201036f0 0x7fd920079450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:51.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd925000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9201048e0 0x7fd920079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:51.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd925a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9201036f0 0x7fd920079450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52758/0 (socket says 172.21.15.47:52758) 2024-03-21T11:22:51.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd925a00640 1 -- 172.21.15.47:0/3347324170 learned_addr learned my addr 172.21.15.47:0/3347324170 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:51.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd926400640 1 --2- 172.21.15.47:0/3347324170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9201052a0 0x7fd920075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:51.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd925a00640 1 -- 172.21.15.47:0/3347324170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9201048e0 msgr2=0x7fd920079990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:51.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd925a00640 1 --2- 172.21.15.47:0/3347324170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9201048e0 0x7fd920079990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd925a00640 1 -- 172.21.15.47:0/3347324170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9201052a0 msgr2=0x7fd920075fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:51.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd925a00640 1 --2- 172.21.15.47:0/3347324170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9201052a0 0x7fd920075fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd925a00640 1 -- 172.21.15.47:0/3347324170 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd920076880 con 0x7fd9201036f0 2024-03-21T11:22:51.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.489+0000 7fd926400640 1 --2- 172.21.15.47:0/3347324170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9201052a0 0x7fd920075fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:22:51.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.490+0000 7fd925a00640 1 --2- 172.21.15.47:0/3347324170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9201036f0 0x7fd920079450 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd91400e9c0 tx=0x7fd91400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:51.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.490+0000 7fd91e000640 1 -- 172.21.15.47:0/3347324170 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91400cd80 con 0x7fd9201036f0 2024-03-21T11:22:51.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.490+0000 7fd91e000640 1 -- 172.21.15.47:0/3347324170 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9140035f0 con 0x7fd9201036f0 2024-03-21T11:22:51.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.490+0000 7fd926e00640 1 -- 172.21.15.47:0/3347324170 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9201a5db0 con 0x7fd9201036f0 2024-03-21T11:22:51.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.490+0000 7fd91e000640 1 -- 172.21.15.47:0/3347324170 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd914010690 con 0x7fd9201036f0 2024-03-21T11:22:51.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.490+0000 7fd926e00640 1 -- 172.21.15.47:0/3347324170 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9201a61c0 con 0x7fd9201036f0 2024-03-21T11:22:51.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.492+0000 7fd926e00640 1 -- 172.21.15.47:0/3347324170 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92010a9b0 con 0x7fd9201036f0 2024-03-21T11:22:51.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.493+0000 7fd91e000640 1 -- 172.21.15.47:0/3347324170 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd9140037a0 con 0x7fd9201036f0 2024-03-21T11:22:51.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.494+0000 7fd91e000640 1 --2- 172.21.15.47:0/3347324170 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9080a2f20 0x7fd9080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:51.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.495+0000 7fd91e000640 1 -- 172.21.15.47:0/3347324170 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd9140cfc60 con 0x7fd9201036f0 2024-03-21T11:22:51.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.495+0000 7fd925000640 1 --2- 172.21.15.47:0/3347324170 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9080a2f20 0x7fd9080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:51.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.496+0000 7fd925000640 1 --2- 172.21.15.47:0/3347324170 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9080a2f20 0x7fd9080a53e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd9200773d0 tx=0x7fd910005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:51.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.503+0000 7fd91e000640 1 -- 172.21.15.47:0/3347324170 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd91409c6a0 con 0x7fd9201036f0 2024-03-21T11:22:51.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.745+0000 7fd926e00640 1 -- 172.21.15.47:0/3347324170 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd92002d070 con 0x7fd9201036f0 2024-03-21T11:22:51.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.746+0000 7fd91e000640 1 -- 172.21.15.47:0/3347324170 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fd92002d070 con 0x7fd9201036f0 2024-03-21T11:22:51.747 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:22:51.747 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:21:18.592954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24524,24523,24535,24581,24553]},"id":1}]} 2024-03-21T11:22:51.748 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 27 2024-03-21T11:22:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.751+0000 7fd926e00640 1 -- 172.21.15.47:0/3347324170 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9080a2f20 msgr2=0x7fd9080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.751+0000 7fd926e00640 1 --2- 172.21.15.47:0/3347324170 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9080a2f20 0x7fd9080a53e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd9200773d0 tx=0x7fd910005f90 comp rx=0 tx=0).stop 2024-03-21T11:22:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.751+0000 7fd926e00640 1 -- 172.21.15.47:0/3347324170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9201036f0 msgr2=0x7fd920079450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.751+0000 7fd926e00640 1 --2- 172.21.15.47:0/3347324170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9201036f0 0x7fd920079450 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd91400e9c0 tx=0x7fd91400ee80 comp rx=0 tx=0).stop 2024-03-21T11:22:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.751+0000 7fd926e00640 1 -- 172.21.15.47:0/3347324170 shutdown_connections 2024-03-21T11:22:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.751+0000 7fd926e00640 1 --2- 172.21.15.47:0/3347324170 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9080a2f20 0x7fd9080a53e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.751+0000 7fd926e00640 1 --2- 172.21.15.47:0/3347324170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9201052a0 0x7fd920075fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.751+0000 7fd926e00640 1 --2- 172.21.15.47:0/3347324170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9201048e0 0x7fd920079990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.751+0000 7fd926e00640 1 --2- 172.21.15.47:0/3347324170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9201036f0 0x7fd920079450 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.751+0000 7fd926e00640 1 -- 172.21.15.47:0/3347324170 >> 172.21.15.47:0/3347324170 conn(0x7fd9200fec80 msgr2=0x7fd920107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.751+0000 7fd926e00640 1 -- 172.21.15.47:0/3347324170 shutdown_connections 2024-03-21T11:22:51.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.751+0000 7fd926e00640 1 -- 172.21.15.47:0/3347324170 wait complete. 2024-03-21T11:22:51.771 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:22:51.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.961+0000 7fc0f8c00640 1 -- 172.21.15.47:0/3359196453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f4103270 msgr2=0x7fc0f4109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:51.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.961+0000 7fc0f8c00640 1 --2- 172.21.15.47:0/3359196453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f4103270 0x7fc0f4109b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009be0 tx=0x7fc0ec02bef0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.961+0000 7fc0f8c00640 1 -- 172.21.15.47:0/3359196453 shutdown_connections 2024-03-21T11:22:51.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.961+0000 7fc0f8c00640 1 --2- 172.21.15.47:0/3359196453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f4103270 0x7fc0f4109b00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.961+0000 7fc0f8c00640 1 --2- 172.21.15.47:0/3359196453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0f41028b0 0x7fc0f4102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.961+0000 7fc0f8c00640 1 --2- 172.21.15.47:0/3359196453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f41016c0 0x7fc0f4101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.961+0000 7fc0f8c00640 1 -- 172.21.15.47:0/3359196453 >> 172.21.15.47:0/3359196453 conn(0x7fc0f40fcc50 msgr2=0x7fc0f40ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.961+0000 7fc0f8c00640 1 -- 172.21.15.47:0/3359196453 shutdown_connections 2024-03-21T11:22:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.962+0000 7fc0f8c00640 1 -- 172.21.15.47:0/3359196453 wait complete. 2024-03-21T11:22:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.962+0000 7fc0f8c00640 1 Processor -- start 2024-03-21T11:22:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.962+0000 7fc0f8c00640 1 -- start start 2024-03-21T11:22:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f8c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f41016c0 0x7fc0f4075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0f41028b0 0x7fc0f4073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f4103270 0x7fc0f4073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f8c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f410d340 con 0x7fc0f41016c0 2024-03-21T11:22:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f8c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f410d1c0 con 0x7fc0f4103270 2024-03-21T11:22:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f8c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f410d4c0 con 0x7fc0f41028b0 2024-03-21T11:22:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f41016c0 0x7fc0f4075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0f41028b0 0x7fc0f4073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f4103270 0x7fc0f4073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0f41028b0 0x7fc0f4073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41580/0 (socket says 172.21.15.47:41580) 2024-03-21T11:22:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f41016c0 0x7fc0f4075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52472/0 (socket says 172.21.15.47:52472) 2024-03-21T11:22:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0ebe00640 1 -- 172.21.15.47:0/350161185 learned_addr learned my addr 172.21.15.47:0/350161185 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f4103270 0x7fc0f4073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52778/0 (socket says 172.21.15.47:52778) 2024-03-21T11:22:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f2800640 1 -- 172.21.15.47:0/350161185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0f41028b0 msgr2=0x7fc0f4073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f2800640 1 --2- 172.21.15.47:0/350161185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0f41028b0 0x7fc0f4073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f2800640 1 -- 172.21.15.47:0/350161185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f41016c0 msgr2=0x7fc0f4075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f2800640 1 --2- 172.21.15.47:0/350161185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f41016c0 0x7fc0f4075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f2800640 1 -- 172.21.15.47:0/350161185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f4074860 con 0x7fc0f4103270 2024-03-21T11:22:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.963+0000 7fc0f1e00640 1 --2- 172.21.15.47:0/350161185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f41016c0 0x7fc0f4075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.964+0000 7fc0f2800640 1 --2- 172.21.15.47:0/350161185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f4103270 0x7fc0f4073fd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec02c430 tx=0x7fc0ec02cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.964+0000 7fc0eaa00640 1 -- 172.21.15.47:0/350161185 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec043020 con 0x7fc0f4103270 2024-03-21T11:22:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.964+0000 7fc0eaa00640 1 -- 172.21.15.47:0/350161185 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0ec003e50 con 0x7fc0f4103270 2024-03-21T11:22:51.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.964+0000 7fc0eaa00640 1 -- 172.21.15.47:0/350161185 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec034430 con 0x7fc0f4103270 2024-03-21T11:22:51.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.964+0000 7fc0f8c00640 1 -- 172.21.15.47:0/350161185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f41a1ab0 con 0x7fc0f4103270 2024-03-21T11:22:51.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.964+0000 7fc0f8c00640 1 -- 172.21.15.47:0/350161185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f41a1fb0 con 0x7fc0f4103270 2024-03-21T11:22:51.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.965+0000 7fc0f8c00640 1 -- 172.21.15.47:0/350161185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f4108980 con 0x7fc0f4103270 2024-03-21T11:22:51.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:51 smithi047 ceph-mon[31351]: osdmap e95: 12 total, 12 up, 12 in 2024-03-21T11:22:51.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:51 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1627945632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:22:51.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:51 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3347324170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:22:51.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.967+0000 7fc0eaa00640 1 -- 172.21.15.47:0/350161185 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc0ec046020 con 0x7fc0f4103270 2024-03-21T11:22:51.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.969+0000 7fc0eaa00640 1 --2- 172.21.15.47:0/350161185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc0d80a2fa0 0x7fc0d80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:51.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.969+0000 7fc0eaa00640 1 -- 172.21.15.47:0/350161185 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc0ec0f1c40 con 0x7fc0f4103270 2024-03-21T11:22:51.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.970+0000 7fc0f1e00640 1 --2- 172.21.15.47:0/350161185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc0d80a2fa0 0x7fc0d80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:51.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.970+0000 7fc0f1e00640 1 --2- 172.21.15.47:0/350161185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc0d80a2fa0 0x7fc0d80a5460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4102710 tx=0x7fc0e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:51.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:51.972+0000 7fc0eaa00640 1 -- 172.21.15.47:0/350161185 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc0ec04d020 con 0x7fc0f4103270 2024-03-21T11:22:52.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:51 smithi100 ceph-mon[36245]: osdmap e95: 12 total, 12 up, 12 in 2024-03-21T11:22:52.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:51 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1627945632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:22:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:51 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3347324170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:22:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.174+0000 7fc0f8c00640 1 -- 172.21.15.47:0/350161185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0f40622a0 con 0x7fc0f4103270 2024-03-21T11:22:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.175+0000 7fc0eaa00640 1 -- 172.21.15.47:0/350161185 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24747 (secure 0 0 0) 0x7fc0ec0be6e0 con 0x7fc0f4103270 2024-03-21T11:22:52.176 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:22:52.177 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:22:50.841892+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:22:52.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.178+0000 7fc0f8c00640 1 -- 172.21.15.47:0/350161185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc0d80a2fa0 msgr2=0x7fc0d80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:52.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.178+0000 7fc0f8c00640 1 --2- 172.21.15.47:0/350161185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc0d80a2fa0 0x7fc0d80a5460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4102710 tx=0x7fc0e4009260 comp rx=0 tx=0).stop 2024-03-21T11:22:52.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.178+0000 7fc0f8c00640 1 -- 172.21.15.47:0/350161185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f4103270 msgr2=0x7fc0f4073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:52.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.178+0000 7fc0f8c00640 1 --2- 172.21.15.47:0/350161185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f4103270 0x7fc0f4073fd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec02c430 tx=0x7fc0ec02cb90 comp rx=0 tx=0).stop 2024-03-21T11:22:52.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.178+0000 7fc0f8c00640 1 -- 172.21.15.47:0/350161185 shutdown_connections 2024-03-21T11:22:52.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.179+0000 7fc0f8c00640 1 --2- 172.21.15.47:0/350161185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc0d80a2fa0 0x7fc0d80a5460 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.179+0000 7fc0f8c00640 1 --2- 172.21.15.47:0/350161185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f4103270 0x7fc0f4073fd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.179+0000 7fc0f8c00640 1 --2- 172.21.15.47:0/350161185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0f41028b0 0x7fc0f4073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.179+0000 7fc0f8c00640 1 --2- 172.21.15.47:0/350161185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f41016c0 0x7fc0f4075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.179+0000 7fc0f8c00640 1 -- 172.21.15.47:0/350161185 >> 172.21.15.47:0/350161185 conn(0x7fc0f40fcc50 msgr2=0x7fc0f4105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:52.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.179+0000 7fc0f8c00640 1 -- 172.21.15.47:0/350161185 shutdown_connections 2024-03-21T11:22:52.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.179+0000 7fc0f8c00640 1 -- 172.21.15.47:0/350161185 wait complete. 2024-03-21T11:22:52.196 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T11:15:35.545245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T11:20:47.016174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T11:20:48.108506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T11:20:52.254154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T11:22:52.197 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T11:22:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:51 smithi102 ceph-mon[36748]: osdmap e95: 12 total, 12 up, 12 in 2024-03-21T11:22:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:51 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1627945632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:22:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:51 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3347324170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:22:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.403+0000 7fbf07000640 1 -- 172.21.15.47:0/2807643369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf00109450 msgr2=0x7fbf0010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.403+0000 7fbf07000640 1 --2- 172.21.15.47:0/2807643369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf00109450 0x7fbf0010b8e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009ca0 tx=0x7fbef802c190 comp rx=0 tx=0).stop 2024-03-21T11:22:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.403+0000 7fbf07000640 1 -- 172.21.15.47:0/2807643369 shutdown_connections 2024-03-21T11:22:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.403+0000 7fbf07000640 1 --2- 172.21.15.47:0/2807643369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf00109450 0x7fbf0010b8e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.403+0000 7fbf07000640 1 --2- 172.21.15.47:0/2807643369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf00074080 0x7fbf000744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.403+0000 7fbf07000640 1 --2- 172.21.15.47:0/2807643369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf00075660 0x7fbf00073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.403+0000 7fbf07000640 1 -- 172.21.15.47:0/2807643369 >> 172.21.15.47:0/2807643369 conn(0x7fbf000fcb10 msgr2=0x7fbf000fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.403+0000 7fbf07000640 1 -- 172.21.15.47:0/2807643369 shutdown_connections 2024-03-21T11:22:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.403+0000 7fbf07000640 1 -- 172.21.15.47:0/2807643369 wait complete. 2024-03-21T11:22:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.404+0000 7fbf07000640 1 Processor -- start 2024-03-21T11:22:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.404+0000 7fbf07000640 1 -- start start 2024-03-21T11:22:52.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf07000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf00074080 0x7fbf001999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:52.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf07000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf00075660 0x7fbf00199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:52.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf07000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf00109450 0x7fbf001a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:52.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf07000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0010f410 con 0x7fbf00109450 2024-03-21T11:22:52.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf07000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0010f290 con 0x7fbf00075660 2024-03-21T11:22:52.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf07000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0010f590 con 0x7fbf00074080 2024-03-21T11:22:52.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf05c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf00074080 0x7fbf001999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:52.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf05c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf00074080 0x7fbf001999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41608/0 (socket says 172.21.15.47:41608) 2024-03-21T11:22:52.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf05c00640 1 -- 172.21.15.47:0/376191037 learned_addr learned my addr 172.21.15.47:0/376191037 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:52.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf05200640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf00075660 0x7fbf00199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:52.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf06600640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf00109450 0x7fbf001a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf05c00640 1 -- 172.21.15.47:0/376191037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf00075660 msgr2=0x7fbf00199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf05c00640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf00075660 0x7fbf00199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf05c00640 1 -- 172.21.15.47:0/376191037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf00109450 msgr2=0x7fbf001a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf05c00640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf00109450 0x7fbf001a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf05c00640 1 -- 172.21.15.47:0/376191037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf001a1610 con 0x7fbf00074080 2024-03-21T11:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf06600640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf00109450 0x7fbf001a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.405+0000 7fbf05200640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf00075660 0x7fbf00199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.406+0000 7fbf05c00640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf00074080 0x7fbf001999f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbef400ea60 tx=0x7fbef400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.406+0000 7fbefe000640 1 -- 172.21.15.47:0/376191037 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef400cde0 con 0x7fbf00074080 2024-03-21T11:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.406+0000 7fbefe000640 1 -- 172.21.15.47:0/376191037 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbef40035f0 con 0x7fbf00074080 2024-03-21T11:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.406+0000 7fbefe000640 1 -- 172.21.15.47:0/376191037 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4010670 con 0x7fbf00074080 2024-03-21T11:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.406+0000 7fbf07000640 1 -- 172.21.15.47:0/376191037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf001a1900 con 0x7fbf00074080 2024-03-21T11:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.406+0000 7fbf07000640 1 -- 172.21.15.47:0/376191037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf001a1e50 con 0x7fbf00074080 2024-03-21T11:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.407+0000 7fbf07000640 1 -- 172.21.15.47:0/376191037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf00108210 con 0x7fbf00074080 2024-03-21T11:22:52.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.409+0000 7fbefe000640 1 -- 172.21.15.47:0/376191037 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbef40037a0 con 0x7fbf00074080 2024-03-21T11:22:52.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.410+0000 7fbefe000640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbee80a2f20 0x7fbee80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:52.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.410+0000 7fbefe000640 1 -- 172.21.15.47:0/376191037 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbef40cfb70 con 0x7fbf00074080 2024-03-21T11:22:52.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.411+0000 7fbf05200640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbee80a2f20 0x7fbee80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:52.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.411+0000 7fbf05200640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbee80a2f20 0x7fbee80a53e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0095f0 tx=0x7fbeec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:52.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.414+0000 7fbefe000640 1 -- 172.21.15.47:0/376191037 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbef409c5b0 con 0x7fbf00074080 2024-03-21T11:22:52.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.616+0000 7fbf07000640 1 -- 172.21.15.47:0/376191037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf0002d070 con 0x7fbf00074080 2024-03-21T11:22:52.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.617+0000 7fbefe000640 1 -- 172.21.15.47:0/376191037 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf0002d070 con 0x7fbf00074080 2024-03-21T11:22:52.617 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T11:22:52.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.619+0000 7fbf07000640 1 -- 172.21.15.47:0/376191037 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbee80a2f20 msgr2=0x7fbee80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:52.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.619+0000 7fbf07000640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbee80a2f20 0x7fbee80a53e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0095f0 tx=0x7fbeec005f90 comp rx=0 tx=0).stop 2024-03-21T11:22:52.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.619+0000 7fbf07000640 1 -- 172.21.15.47:0/376191037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf00074080 msgr2=0x7fbf001999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:52.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.619+0000 7fbf07000640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf00074080 0x7fbf001999f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbef400ea60 tx=0x7fbef400ef20 comp rx=0 tx=0).stop 2024-03-21T11:22:52.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.619+0000 7fbf07000640 1 -- 172.21.15.47:0/376191037 shutdown_connections 2024-03-21T11:22:52.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.619+0000 7fbf07000640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbee80a2f20 0x7fbee80a53e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.619+0000 7fbf07000640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf00109450 0x7fbf001a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.619+0000 7fbf07000640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf00075660 0x7fbf00199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.619+0000 7fbf07000640 1 --2- 172.21.15.47:0/376191037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf00074080 0x7fbf001999f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.619+0000 7fbf07000640 1 -- 172.21.15.47:0/376191037 >> 172.21.15.47:0/376191037 conn(0x7fbf000fcb10 msgr2=0x7fbf001053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.619+0000 7fbf07000640 1 -- 172.21.15.47:0/376191037 shutdown_connections 2024-03-21T11:22:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.619+0000 7fbf07000640 1 -- 172.21.15.47:0/376191037 wait complete. 2024-03-21T11:22:52.639 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T11:22:52.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.853+0000 7fe4f7c00640 1 -- 172.21.15.47:0/3949394568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01028a0 msgr2=0x7fe4f0102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:52.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.853+0000 7fe4f7c00640 1 --2- 172.21.15.47:0/3949394568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01028a0 0x7fe4f0102d20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009900 tx=0x7fe4dc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:22:52.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.853+0000 7fe4f7c00640 1 -- 172.21.15.47:0/3949394568 shutdown_connections 2024-03-21T11:22:52.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.853+0000 7fe4f7c00640 1 --2- 172.21.15.47:0/3949394568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4f0103260 0x7fe4f0109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.853+0000 7fe4f7c00640 1 --2- 172.21.15.47:0/3949394568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01028a0 0x7fe4f0102d20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.853+0000 7fe4f7c00640 1 --2- 172.21.15.47:0/3949394568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4f01016b0 0x7fe4f0101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.853+0000 7fe4f7c00640 1 -- 172.21.15.47:0/3949394568 >> 172.21.15.47:0/3949394568 conn(0x7fe4f00fcc40 msgr2=0x7fe4f00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:52.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.853+0000 7fe4f7c00640 1 -- 172.21.15.47:0/3949394568 shutdown_connections 2024-03-21T11:22:52.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.853+0000 7fe4f7c00640 1 -- 172.21.15.47:0/3949394568 wait complete. 2024-03-21T11:22:52.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.854+0000 7fe4f7c00640 1 Processor -- start 2024-03-21T11:22:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.854+0000 7fe4f7c00640 1 -- start start 2024-03-21T11:22:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.854+0000 7fe4f7c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4f01016b0 0x7fe4f0075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4f7c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01028a0 0x7fe4f0073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4f7c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4f0103260 0x7fe4f0073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4f7c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f010d330 con 0x7fe4f01016b0 2024-03-21T11:22:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4f7c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f010d1b0 con 0x7fe4f01028a0 2024-03-21T11:22:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4f7c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f010d4b0 con 0x7fe4f0103260 2024-03-21T11:22:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4efe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01028a0 0x7fe4f0073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4efe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01028a0 0x7fe4f0073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52832/0 (socket says 172.21.15.47:52832) 2024-03-21T11:22:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4efe00640 1 -- 172.21.15.47:0/1540380405 learned_addr learned my addr 172.21.15.47:0/1540380405 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:52.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4f5c00640 1 --2- 172.21.15.47:0/1540380405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4f0103260 0x7fe4f0073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4f5200640 1 --2- 172.21.15.47:0/1540380405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4f01016b0 0x7fe4f0075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4efe00640 1 -- 172.21.15.47:0/1540380405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4f0103260 msgr2=0x7fe4f0073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4efe00640 1 --2- 172.21.15.47:0/1540380405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4f0103260 0x7fe4f0073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4efe00640 1 -- 172.21.15.47:0/1540380405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4f01016b0 msgr2=0x7fe4f0075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4efe00640 1 --2- 172.21.15.47:0/1540380405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4f01016b0 0x7fe4f0075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.855+0000 7fe4efe00640 1 -- 172.21.15.47:0/1540380405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f0074860 con 0x7fe4f01028a0 2024-03-21T11:22:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.856+0000 7fe4efe00640 1 --2- 172.21.15.47:0/1540380405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01028a0 0x7fe4f0073a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009230 tx=0x7fe4dc02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.856+0000 7fe4ed600640 1 -- 172.21.15.47:0/1540380405 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc041070 con 0x7fe4f01028a0 2024-03-21T11:22:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.856+0000 7fe4ed600640 1 -- 172.21.15.47:0/1540380405 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc004450 con 0x7fe4f01028a0 2024-03-21T11:22:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.856+0000 7fe4ed600640 1 -- 172.21.15.47:0/1540380405 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc003e20 con 0x7fe4f01028a0 2024-03-21T11:22:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.856+0000 7fe4f7c00640 1 -- 172.21.15.47:0/1540380405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f01a1a70 con 0x7fe4f01028a0 2024-03-21T11:22:52.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.856+0000 7fe4f7c00640 1 -- 172.21.15.47:0/1540380405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f01a1f70 con 0x7fe4f01028a0 2024-03-21T11:22:52.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.857+0000 7fe4f7c00640 1 -- 172.21.15.47:0/1540380405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f0108a50 con 0x7fe4f01028a0 2024-03-21T11:22:52.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.859+0000 7fe4ed600640 1 -- 172.21.15.47:0/1540380405 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe4dc044070 con 0x7fe4f01028a0 2024-03-21T11:22:52.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.861+0000 7fe4ed600640 1 --2- 172.21.15.47:0/1540380405 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4d80a2fa0 0x7fe4d80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:52.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.861+0000 7fe4ed600640 1 -- 172.21.15.47:0/1540380405 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe4dc0f1280 con 0x7fe4f01028a0 2024-03-21T11:22:52.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.861+0000 7fe4f5200640 1 --2- 172.21.15.47:0/1540380405 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4d80a2fa0 0x7fe4d80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:52.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.862+0000 7fe4f5200640 1 --2- 172.21.15.47:0/1540380405 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4d80a2fa0 0x7fe4d80a5460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0102700 tx=0x7fe4e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:52.864+0000 7fe4ed600640 1 -- 172.21.15.47:0/1540380405 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe4dc0bd670 con 0x7fe4f01028a0 2024-03-21T11:22:52.887 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:52 smithi047 ceph-mon[31351]: pgmap v386: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 639 B/s wr, 10 op/s 2024-03-21T11:22:52.887 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:52 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/350161185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:22:52.887 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:52 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/376191037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:22:53.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.061+0000 7fe4f7c00640 1 -- 172.21.15.47:0/1540380405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe4f0062340 con 0x7fe4f01028a0 2024-03-21T11:22:53.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.062+0000 7fe4ed600640 1 -- 172.21.15.47:0/1540380405 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe4dc0c1710 con 0x7fe4f01028a0 2024-03-21T11:22:53.062 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:22:53.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.064+0000 7fe4f7c00640 1 -- 172.21.15.47:0/1540380405 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4d80a2fa0 msgr2=0x7fe4d80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.064+0000 7fe4f7c00640 1 --2- 172.21.15.47:0/1540380405 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4d80a2fa0 0x7fe4d80a5460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0102700 tx=0x7fe4e4005f90 comp rx=0 tx=0).stop 2024-03-21T11:22:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.064+0000 7fe4f7c00640 1 -- 172.21.15.47:0/1540380405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01028a0 msgr2=0x7fe4f0073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.064+0000 7fe4f7c00640 1 --2- 172.21.15.47:0/1540380405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01028a0 0x7fe4f0073a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009230 tx=0x7fe4dc02c650 comp rx=0 tx=0).stop 2024-03-21T11:22:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.064+0000 7fe4f7c00640 1 -- 172.21.15.47:0/1540380405 shutdown_connections 2024-03-21T11:22:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.064+0000 7fe4f7c00640 1 --2- 172.21.15.47:0/1540380405 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4d80a2fa0 0x7fe4d80a5460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.064+0000 7fe4f7c00640 1 --2- 172.21.15.47:0/1540380405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4f0103260 0x7fe4f0073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.064+0000 7fe4f7c00640 1 --2- 172.21.15.47:0/1540380405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01028a0 0x7fe4f0073a90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.064+0000 7fe4f7c00640 1 --2- 172.21.15.47:0/1540380405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4f01016b0 0x7fe4f0075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.064+0000 7fe4f7c00640 1 -- 172.21.15.47:0/1540380405 >> 172.21.15.47:0/1540380405 conn(0x7fe4f00fcc40 msgr2=0x7fe4f0105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.064+0000 7fe4f7c00640 1 -- 172.21.15.47:0/1540380405 shutdown_connections 2024-03-21T11:22:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.064+0000 7fe4f7c00640 1 -- 172.21.15.47:0/1540380405 wait complete. 2024-03-21T11:22:53.085 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T11:22:53.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:52 smithi100 ceph-mon[36245]: pgmap v386: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 639 B/s wr, 10 op/s 2024-03-21T11:22:53.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:52 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/350161185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:22:53.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:52 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/376191037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:22:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:52 smithi102 ceph-mon[36748]: pgmap v386: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 639 B/s wr, 10 op/s 2024-03-21T11:22:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:52 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/350161185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:22:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:52 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/376191037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:22:53.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.304+0000 7f646b000640 1 -- 172.21.15.47:0/1703991517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6464109770 msgr2=0x7f6464110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:53.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.304+0000 7f646b000640 1 --2- 172.21.15.47:0/1703991517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6464109770 0x7f6464110000 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f645c009be0 tx=0x7f645c02bef0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.304+0000 7f646b000640 1 -- 172.21.15.47:0/1703991517 shutdown_connections 2024-03-21T11:22:53.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.304+0000 7f646b000640 1 --2- 172.21.15.47:0/1703991517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6464109770 0x7f6464110000 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.304+0000 7f646b000640 1 --2- 172.21.15.47:0/1703991517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6464108db0 0x7f6464109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.304+0000 7f646b000640 1 --2- 172.21.15.47:0/1703991517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6464107bc0 0x7f6464107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.304+0000 7f646b000640 1 -- 172.21.15.47:0/1703991517 >> 172.21.15.47:0/1703991517 conn(0x7f6464076b70 msgr2=0x7f6464078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:53.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.304+0000 7f646b000640 1 -- 172.21.15.47:0/1703991517 shutdown_connections 2024-03-21T11:22:53.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.304+0000 7f646b000640 1 -- 172.21.15.47:0/1703991517 wait complete. 2024-03-21T11:22:53.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.305+0000 7f646b000640 1 Processor -- start 2024-03-21T11:22:53.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.305+0000 7f646b000640 1 -- start start 2024-03-21T11:22:53.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.305+0000 7f646b000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6464107bc0 0x7f646419de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:53.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.305+0000 7f646b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6464108db0 0x7f646419e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:53.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f646b000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6464109770 0x7f64641a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:53.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f646b000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6464113890 con 0x7f6464109770 2024-03-21T11:22:53.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f646b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6464113710 con 0x7f6464108db0 2024-03-21T11:22:53.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f646b000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6464113a10 con 0x7f6464107bc0 2024-03-21T11:22:53.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f6469200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6464108db0 0x7f646419e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f6469c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6464107bc0 0x7f646419de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f6469c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6464107bc0 0x7f646419de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41630/0 (socket says 172.21.15.47:41630) 2024-03-21T11:22:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f6469200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6464108db0 0x7f646419e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52846/0 (socket says 172.21.15.47:52846) 2024-03-21T11:22:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f6469c00640 1 -- 172.21.15.47:0/4195264940 learned_addr learned my addr 172.21.15.47:0/4195264940 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f646a600640 1 --2- 172.21.15.47:0/4195264940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6464109770 0x7f64641a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f6469c00640 1 -- 172.21.15.47:0/4195264940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6464108db0 msgr2=0x7f646419e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f6469c00640 1 --2- 172.21.15.47:0/4195264940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6464108db0 0x7f646419e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f6469c00640 1 -- 172.21.15.47:0/4195264940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6464109770 msgr2=0x7f64641a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f6469c00640 1 --2- 172.21.15.47:0/4195264940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6464109770 0x7f64641a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f6469c00640 1 -- 172.21.15.47:0/4195264940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64641a5a40 con 0x7f6464107bc0 2024-03-21T11:22:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.306+0000 7f6469200640 1 --2- 172.21.15.47:0/4195264940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6464108db0 0x7f646419e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.307+0000 7f6469c00640 1 --2- 172.21.15.47:0/4195264940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6464107bc0 0x7f646419de20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f645800e9f0 tx=0x7f645800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.307+0000 7f6462000640 1 -- 172.21.15.47:0/4195264940 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645800ccc0 con 0x7f6464107bc0 2024-03-21T11:22:53.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.307+0000 7f6462000640 1 -- 172.21.15.47:0/4195264940 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64580035f0 con 0x7f6464107bc0 2024-03-21T11:22:53.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.307+0000 7f6462000640 1 -- 172.21.15.47:0/4195264940 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645800ce60 con 0x7f6464107bc0 2024-03-21T11:22:53.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.307+0000 7f646b000640 1 -- 172.21.15.47:0/4195264940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64641a5d30 con 0x7f6464107bc0 2024-03-21T11:22:53.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.307+0000 7f646b000640 1 -- 172.21.15.47:0/4195264940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64641a61d0 con 0x7f6464107bc0 2024-03-21T11:22:53.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.308+0000 7f646b000640 1 -- 172.21.15.47:0/4195264940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646410ee80 con 0x7f6464107bc0 2024-03-21T11:22:53.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.310+0000 7f6462000640 1 -- 172.21.15.47:0/4195264940 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6458003b10 con 0x7f6464107bc0 2024-03-21T11:22:53.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.311+0000 7f6462000640 1 --2- 172.21.15.47:0/4195264940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f644c0a2fa0 0x7f644c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:53.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.311+0000 7f6462000640 1 -- 172.21.15.47:0/4195264940 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f64580cfac0 con 0x7f6464107bc0 2024-03-21T11:22:53.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.312+0000 7f6469200640 1 --2- 172.21.15.47:0/4195264940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f644c0a2fa0 0x7f644c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:53.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.312+0000 7f6469200640 1 --2- 172.21.15.47:0/4195264940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f644c0a2fa0 0x7f644c0a5460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f646419f330 tx=0x7f645000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:53.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.318+0000 7f6462000640 1 -- 172.21.15.47:0/4195264940 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f645809c500 con 0x7f6464107bc0 2024-03-21T11:22:53.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.518+0000 7f646b000640 1 -- 172.21.15.47:0/4195264940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f646402d220 con 0x7f6464107bc0 2024-03-21T11:22:53.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.519+0000 7f6462000640 1 -- 172.21.15.47:0/4195264940 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f645809be00 con 0x7f6464107bc0 2024-03-21T11:22:53.520 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T11:22:53.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.522+0000 7f646b000640 1 -- 172.21.15.47:0/4195264940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f644c0a2fa0 msgr2=0x7f644c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:53.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.522+0000 7f646b000640 1 --2- 172.21.15.47:0/4195264940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f644c0a2fa0 0x7f644c0a5460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f646419f330 tx=0x7f645000b390 comp rx=0 tx=0).stop 2024-03-21T11:22:53.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.522+0000 7f646b000640 1 -- 172.21.15.47:0/4195264940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6464107bc0 msgr2=0x7f646419de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:53.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.522+0000 7f646b000640 1 --2- 172.21.15.47:0/4195264940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6464107bc0 0x7f646419de20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f645800e9f0 tx=0x7f645800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.522+0000 7f646b000640 1 -- 172.21.15.47:0/4195264940 shutdown_connections 2024-03-21T11:22:53.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.522+0000 7f646b000640 1 --2- 172.21.15.47:0/4195264940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f644c0a2fa0 0x7f644c0a5460 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.522+0000 7f646b000640 1 --2- 172.21.15.47:0/4195264940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6464109770 0x7f64641a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.522+0000 7f646b000640 1 --2- 172.21.15.47:0/4195264940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6464108db0 0x7f646419e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.522+0000 7f646b000640 1 --2- 172.21.15.47:0/4195264940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6464107bc0 0x7f646419de20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.522+0000 7f646b000640 1 -- 172.21.15.47:0/4195264940 >> 172.21.15.47:0/4195264940 conn(0x7f6464076b70 msgr2=0x7f646410c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:53.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.523+0000 7f646b000640 1 -- 172.21.15.47:0/4195264940 shutdown_connections 2024-03-21T11:22:53.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.523+0000 7f646b000640 1 -- 172.21.15.47:0/4195264940 wait complete. 2024-03-21T11:22:53.543 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T11:22:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.775+0000 7fc3f8a00640 1 -- 172.21.15.47:0/2009649304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f4108db0 msgr2=0x7fc3f4109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.775+0000 7fc3f8a00640 1 --2- 172.21.15.47:0/2009649304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f4108db0 0x7fc3f4109230 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009900 tx=0x7fc3dc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:22:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.775+0000 7fc3f8a00640 1 -- 172.21.15.47:0/2009649304 shutdown_connections 2024-03-21T11:22:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.775+0000 7fc3f8a00640 1 --2- 172.21.15.47:0/2009649304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3f4109770 0x7fc3f4110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.775+0000 7fc3f8a00640 1 --2- 172.21.15.47:0/2009649304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f4108db0 0x7fc3f4109230 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.775+0000 7fc3f8a00640 1 --2- 172.21.15.47:0/2009649304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f4107bc0 0x7fc3f4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.775+0000 7fc3f8a00640 1 -- 172.21.15.47:0/2009649304 >> 172.21.15.47:0/2009649304 conn(0x7fc3f4076b70 msgr2=0x7fc3f4078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.775+0000 7fc3f8a00640 1 -- 172.21.15.47:0/2009649304 shutdown_connections 2024-03-21T11:22:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.775+0000 7fc3f8a00640 1 -- 172.21.15.47:0/2009649304 wait complete. 2024-03-21T11:22:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.777+0000 7fc3f8a00640 1 Processor -- start 2024-03-21T11:22:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.777+0000 7fc3f8a00640 1 -- start start 2024-03-21T11:22:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.777+0000 7fc3f8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f4107bc0 0x7fc3f419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.777+0000 7fc3f8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f4108db0 0x7fc3f419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.777+0000 7fc3f8a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3f4109770 0x7fc3f41a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.777+0000 7fc3f8a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f4113820 con 0x7fc3f4109770 2024-03-21T11:22:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.777+0000 7fc3f8a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41136a0 con 0x7fc3f4108db0 2024-03-21T11:22:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.777+0000 7fc3f8a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41139a0 con 0x7fc3f4107bc0 2024-03-21T11:22:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.778+0000 7fc3f1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f4108db0 0x7fc3f419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.778+0000 7fc3f2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f4107bc0 0x7fc3f419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.778+0000 7fc3f1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f4108db0 0x7fc3f419e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52868/0 (socket says 172.21.15.47:52868) 2024-03-21T11:22:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.778+0000 7fc3f1e00640 1 -- 172.21.15.47:0/1151664851 learned_addr learned my addr 172.21.15.47:0/1151664851 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.778+0000 7fc3f3200640 1 --2- 172.21.15.47:0/1151664851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3f4109770 0x7fc3f41a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.778+0000 7fc3f1e00640 1 -- 172.21.15.47:0/1151664851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f4107bc0 msgr2=0x7fc3f419dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.778+0000 7fc3f1e00640 1 --2- 172.21.15.47:0/1151664851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f4107bc0 0x7fc3f419dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.778+0000 7fc3f1e00640 1 -- 172.21.15.47:0/1151664851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3f4109770 msgr2=0x7fc3f41a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.778+0000 7fc3f1e00640 1 --2- 172.21.15.47:0/1151664851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3f4109770 0x7fc3f41a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.778+0000 7fc3f1e00640 1 -- 172.21.15.47:0/1151664851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f41a5ac0 con 0x7fc3f4108db0 2024-03-21T11:22:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.778+0000 7fc3f3200640 1 --2- 172.21.15.47:0/1151664851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3f4109770 0x7fc3f41a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:22:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.778+0000 7fc3f1e00640 1 --2- 172.21.15.47:0/1151664851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f4108db0 0x7fc3f419e3e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009230 tx=0x7fc3dc005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.778+0000 7fc3eb400640 1 -- 172.21.15.47:0/1151664851 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc041070 con 0x7fc3f4108db0 2024-03-21T11:22:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.779+0000 7fc3eb400640 1 -- 172.21.15.47:0/1151664851 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3dc0044d0 con 0x7fc3f4108db0 2024-03-21T11:22:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.779+0000 7fc3eb400640 1 -- 172.21.15.47:0/1151664851 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc034440 con 0x7fc3f4108db0 2024-03-21T11:22:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.779+0000 7fc3f8a00640 1 -- 172.21.15.47:0/1151664851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f41a5d50 con 0x7fc3f4108db0 2024-03-21T11:22:53.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.779+0000 7fc3f8a00640 1 -- 172.21.15.47:0/1151664851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f41a6190 con 0x7fc3f4108db0 2024-03-21T11:22:53.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.780+0000 7fc3f8a00640 1 -- 172.21.15.47:0/1151664851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f410ef60 con 0x7fc3f4108db0 2024-03-21T11:22:53.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.782+0000 7fc3eb400640 1 -- 172.21.15.47:0/1151664851 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc3dc044070 con 0x7fc3f4108db0 2024-03-21T11:22:53.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.783+0000 7fc3eb400640 1 --2- 172.21.15.47:0/1151664851 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3d80a2fa0 0x7fc3d80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:53.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.783+0000 7fc3eb400640 1 -- 172.21.15.47:0/1151664851 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc3dc0f13a0 con 0x7fc3f4108db0 2024-03-21T11:22:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.784+0000 7fc3f2800640 1 --2- 172.21.15.47:0/1151664851 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3d80a2fa0 0x7fc3d80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.784+0000 7fc3f2800640 1 --2- 172.21.15.47:0/1151664851 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3d80a2fa0 0x7fc3d80a5460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4108c10 tx=0x7fc3e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:53.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.787+0000 7fc3eb400640 1 -- 172.21.15.47:0/1151664851 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc3dc0bdde0 con 0x7fc3f4108db0 2024-03-21T11:22:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.993+0000 7fc3f8a00640 1 -- 172.21.15.47:0/1151664851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc3f4062340 con 0x7fc3f4108db0 2024-03-21T11:22:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.993+0000 7fc3eb400640 1 -- 172.21.15.47:0/1151664851 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc3dc0bd6e0 con 0x7fc3f4108db0 2024-03-21T11:22:53.994 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:22:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.997+0000 7fc3f8a00640 1 -- 172.21.15.47:0/1151664851 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3d80a2fa0 msgr2=0x7fc3d80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.997+0000 7fc3f8a00640 1 --2- 172.21.15.47:0/1151664851 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3d80a2fa0 0x7fc3d80a5460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4108c10 tx=0x7fc3e4005f90 comp rx=0 tx=0).stop 2024-03-21T11:22:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.997+0000 7fc3f8a00640 1 -- 172.21.15.47:0/1151664851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f4108db0 msgr2=0x7fc3f419e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.997+0000 7fc3f8a00640 1 --2- 172.21.15.47:0/1151664851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f4108db0 0x7fc3f419e3e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009230 tx=0x7fc3dc005d50 comp rx=0 tx=0).stop 2024-03-21T11:22:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.997+0000 7fc3f8a00640 1 -- 172.21.15.47:0/1151664851 shutdown_connections 2024-03-21T11:22:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.997+0000 7fc3f8a00640 1 --2- 172.21.15.47:0/1151664851 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3d80a2fa0 0x7fc3d80a5460 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.997+0000 7fc3f8a00640 1 --2- 172.21.15.47:0/1151664851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3f4109770 0x7fc3f41a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.997+0000 7fc3f8a00640 1 --2- 172.21.15.47:0/1151664851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3f4108db0 0x7fc3f419e3e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.997+0000 7fc3f8a00640 1 --2- 172.21.15.47:0/1151664851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3f4107bc0 0x7fc3f419dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.997+0000 7fc3f8a00640 1 -- 172.21.15.47:0/1151664851 >> 172.21.15.47:0/1151664851 conn(0x7fc3f4076b70 msgr2=0x7fc3f410c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.997+0000 7fc3f8a00640 1 -- 172.21.15.47:0/1151664851 shutdown_connections 2024-03-21T11:22:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:53.997+0000 7fc3f8a00640 1 -- 172.21.15.47:0/1151664851 wait complete. 2024-03-21T11:22:54.019 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:22:54.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:53 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1540380405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:22:54.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:53 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/4195264940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:22:54.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:53 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1540380405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:22:54.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:53 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/4195264940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:22:54.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.244+0000 7fef82600640 1 -- 172.21.15.47:0/2496040229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c074100 msgr2=0x7fef7c074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:54.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.244+0000 7fef82600640 1 --2- 172.21.15.47:0/2496040229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c074100 0x7fef7c074580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009890 tx=0x7fef6c02bb70 comp rx=0 tx=0).stop 2024-03-21T11:22:54.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.244+0000 7fef82600640 1 -- 172.21.15.47:0/2496040229 shutdown_connections 2024-03-21T11:22:54.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.244+0000 7fef82600640 1 --2- 172.21.15.47:0/2496040229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7c10dab0 0x7fef7c10fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.244+0000 7fef82600640 1 --2- 172.21.15.47:0/2496040229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c074100 0x7fef7c074580 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.244+0000 7fef82600640 1 --2- 172.21.15.47:0/2496040229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef7c0756e0 0x7fef7c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.244+0000 7fef82600640 1 -- 172.21.15.47:0/2496040229 >> 172.21.15.47:0/2496040229 conn(0x7fef7c0fca30 msgr2=0x7fef7c0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:54.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.244+0000 7fef82600640 1 -- 172.21.15.47:0/2496040229 shutdown_connections 2024-03-21T11:22:54.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.244+0000 7fef82600640 1 -- 172.21.15.47:0/2496040229 wait complete. 2024-03-21T11:22:54.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.245+0000 7fef82600640 1 Processor -- start 2024-03-21T11:22:54.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.245+0000 7fef82600640 1 -- start start 2024-03-21T11:22:54.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.246+0000 7fef82600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c074100 0x7fef7c19de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:54.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.246+0000 7fef82600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7c0756e0 0x7fef7c19e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:54.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.246+0000 7fef82600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef7c10dab0 0x7fef7c1a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:54.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.246+0000 7fef82600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c113730 con 0x7fef7c0756e0 2024-03-21T11:22:54.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.246+0000 7fef82600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1135b0 con 0x7fef7c074100 2024-03-21T11:22:54.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.246+0000 7fef82600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1138b0 con 0x7fef7c10dab0 2024-03-21T11:22:54.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.246+0000 7fef81200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c074100 0x7fef7c19de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:54.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.246+0000 7fef7be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7c0756e0 0x7fef7c19e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:54.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.246+0000 7fef81200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c074100 0x7fef7c19de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52878/0 (socket says 172.21.15.47:52878) 2024-03-21T11:22:54.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.246+0000 7fef81200640 1 -- 172.21.15.47:0/2434496946 learned_addr learned my addr 172.21.15.47:0/2434496946 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:54.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.246+0000 7fef81c00640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef7c10dab0 0x7fef7c1a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:54.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.247+0000 7fef81200640 1 -- 172.21.15.47:0/2434496946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef7c10dab0 msgr2=0x7fef7c1a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:54.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.247+0000 7fef81200640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef7c10dab0 0x7fef7c1a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.247+0000 7fef81200640 1 -- 172.21.15.47:0/2434496946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7c0756e0 msgr2=0x7fef7c19e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:54.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.247+0000 7fef81200640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7c0756e0 0x7fef7c19e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.247+0000 7fef81200640 1 -- 172.21.15.47:0/2434496946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c1a5a60 con 0x7fef7c074100 2024-03-21T11:22:54.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.247+0000 7fef7be00640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7c0756e0 0x7fef7c19e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:54.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.247+0000 7fef81c00640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef7c10dab0 0x7fef7c1a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:54.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.247+0000 7fef81200640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c074100 0x7fef7c19de40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fef7000e9c0 tx=0x7fef7000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:54.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.247+0000 7fef79600640 1 -- 172.21.15.47:0/2434496946 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7000cde0 con 0x7fef7c074100 2024-03-21T11:22:54.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.247+0000 7fef79600640 1 -- 172.21.15.47:0/2434496946 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef700035f0 con 0x7fef7c074100 2024-03-21T11:22:54.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.247+0000 7fef82600640 1 -- 172.21.15.47:0/2434496946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c1a5d50 con 0x7fef7c074100 2024-03-21T11:22:54.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.247+0000 7fef79600640 1 -- 172.21.15.47:0/2434496946 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70010690 con 0x7fef7c074100 2024-03-21T11:22:54.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.248+0000 7fef82600640 1 -- 172.21.15.47:0/2434496946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c1a6200 con 0x7fef7c074100 2024-03-21T11:22:54.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.249+0000 7fef82600640 1 -- 172.21.15.47:0/2434496946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7c103c80 con 0x7fef7c074100 2024-03-21T11:22:54.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.251+0000 7fef79600640 1 -- 172.21.15.47:0/2434496946 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fef700037a0 con 0x7fef7c074100 2024-03-21T11:22:54.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.252+0000 7fef79600640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef640a2fa0 0x7fef640a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:54.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.252+0000 7fef79600640 1 -- 172.21.15.47:0/2434496946 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fef700cfa70 con 0x7fef7c074100 2024-03-21T11:22:54.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.253+0000 7fef7be00640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef640a2fa0 0x7fef640a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:54.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.253+0000 7fef7be00640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef640a2fa0 0x7fef640a5460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fef6c02c180 tx=0x7fef6c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:54.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.256+0000 7fef79600640 1 -- 172.21.15.47:0/2434496946 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fef7009c4b0 con 0x7fef7c074100 2024-03-21T11:22:54.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:53 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1540380405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:22:54.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:53 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/4195264940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:22:54.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.503+0000 7fef82600640 1 -- 172.21.15.47:0/2434496946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef7c02d430 con 0x7fef7c074100 2024-03-21T11:22:54.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.504+0000 7fef79600640 1 -- 172.21.15.47:0/2434496946 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fef7009bdb0 con 0x7fef7c074100 2024-03-21T11:22:54.505 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:22:54.505 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:21:18.592954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24524,24523,24535,24581,24553]},"id":1}]} 2024-03-21T11:22:54.505 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 27 2024-03-21T11:22:54.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.506+0000 7fef82600640 1 -- 172.21.15.47:0/2434496946 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef640a2fa0 msgr2=0x7fef640a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:54.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.506+0000 7fef82600640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef640a2fa0 0x7fef640a5460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fef6c02c180 tx=0x7fef6c02e040 comp rx=0 tx=0).stop 2024-03-21T11:22:54.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.507+0000 7fef82600640 1 -- 172.21.15.47:0/2434496946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c074100 msgr2=0x7fef7c19de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:54.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.507+0000 7fef82600640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c074100 0x7fef7c19de40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fef7000e9c0 tx=0x7fef7000ee80 comp rx=0 tx=0).stop 2024-03-21T11:22:54.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.507+0000 7fef82600640 1 -- 172.21.15.47:0/2434496946 shutdown_connections 2024-03-21T11:22:54.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.507+0000 7fef82600640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef640a2fa0 0x7fef640a5460 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.507+0000 7fef82600640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef7c10dab0 0x7fef7c1a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.507+0000 7fef82600640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7c0756e0 0x7fef7c19e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.507+0000 7fef82600640 1 --2- 172.21.15.47:0/2434496946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c074100 0x7fef7c19de40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.507+0000 7fef82600640 1 -- 172.21.15.47:0/2434496946 >> 172.21.15.47:0/2434496946 conn(0x7fef7c0fca30 msgr2=0x7fef7c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:54.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.507+0000 7fef82600640 1 -- 172.21.15.47:0/2434496946 shutdown_connections 2024-03-21T11:22:54.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.507+0000 7fef82600640 1 -- 172.21.15.47:0/2434496946 wait complete. 2024-03-21T11:22:54.528 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:22:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.749+0000 7f79e3e00640 1 -- 172.21.15.47:0/1180484586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79e4101500 msgr2=0x7f79e4101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.749+0000 7f79e3e00640 1 --2- 172.21.15.47:0/1180484586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79e4101500 0x7f79e4101920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009900 tx=0x7f79d402bc40 comp rx=0 tx=0).stop 2024-03-21T11:22:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.749+0000 7f79e3e00640 1 -- 172.21.15.47:0/1180484586 shutdown_connections 2024-03-21T11:22:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.749+0000 7f79e3e00640 1 --2- 172.21.15.47:0/1180484586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79e41030b0 0x7f79e4109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.749+0000 7f79e3e00640 1 --2- 172.21.15.47:0/1180484586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79e41026f0 0x7f79e4102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.749+0000 7f79e3e00640 1 --2- 172.21.15.47:0/1180484586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79e4101500 0x7f79e4101920 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.749+0000 7f79e3e00640 1 -- 172.21.15.47:0/1180484586 >> 172.21.15.47:0/1180484586 conn(0x7f79e40fca70 msgr2=0x7f79e40feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.749+0000 7f79e3e00640 1 -- 172.21.15.47:0/1180484586 shutdown_connections 2024-03-21T11:22:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.749+0000 7f79e3e00640 1 -- 172.21.15.47:0/1180484586 wait complete. 2024-03-21T11:22:54.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.750+0000 7f79e3e00640 1 Processor -- start 2024-03-21T11:22:54.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.750+0000 7f79e3e00640 1 -- start start 2024-03-21T11:22:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.751+0000 7f79e3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79e4101500 0x7f79e41999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.751+0000 7f79e3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79e41026f0 0x7f79e4199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.751+0000 7f79e3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79e41030b0 0x7f79e41a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.751+0000 7f79e3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e410d1e0 con 0x7f79e4101500 2024-03-21T11:22:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.751+0000 7f79e3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e410d060 con 0x7f79e41026f0 2024-03-21T11:22:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.751+0000 7f79e3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e410d360 con 0x7f79e41030b0 2024-03-21T11:22:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.751+0000 7f79e2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79e4101500 0x7f79e41999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.751+0000 7f79e3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79e41030b0 0x7f79e41a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.751+0000 7f79e2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79e4101500 0x7f79e41999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52560/0 (socket says 172.21.15.47:52560) 2024-03-21T11:22:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.751+0000 7f79e2a00640 1 -- 172.21.15.47:0/847713134 learned_addr learned my addr 172.21.15.47:0/847713134 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.751+0000 7f79e2000640 1 --2- 172.21.15.47:0/847713134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79e41026f0 0x7f79e4199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.751+0000 7f79e3400640 1 -- 172.21.15.47:0/847713134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79e41026f0 msgr2=0x7f79e4199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.751+0000 7f79e3400640 1 --2- 172.21.15.47:0/847713134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79e41026f0 0x7f79e4199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.752+0000 7f79e3400640 1 -- 172.21.15.47:0/847713134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79e4101500 msgr2=0x7f79e41999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.752+0000 7f79e3400640 1 --2- 172.21.15.47:0/847713134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79e4101500 0x7f79e41999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.752+0000 7f79e3400640 1 -- 172.21.15.47:0/847713134 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e41a1610 con 0x7f79e41030b0 2024-03-21T11:22:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.752+0000 7f79e2a00640 1 --2- 172.21.15.47:0/847713134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79e4101500 0x7f79e41999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.752+0000 7f79e3400640 1 --2- 172.21.15.47:0/847713134 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79e41030b0 0x7f79e41a0fa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009fd0 tx=0x7f79dc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.752+0000 7f79db400640 1 -- 172.21.15.47:0/847713134 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc00ce20 con 0x7f79e41030b0 2024-03-21T11:22:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.752+0000 7f79db400640 1 -- 172.21.15.47:0/847713134 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79dc005020 con 0x7f79e41030b0 2024-03-21T11:22:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.752+0000 7f79db400640 1 -- 172.21.15.47:0/847713134 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc010720 con 0x7f79e41030b0 2024-03-21T11:22:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.752+0000 7f79e3e00640 1 -- 172.21.15.47:0/847713134 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e41a1900 con 0x7f79e41030b0 2024-03-21T11:22:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.752+0000 7f79e3e00640 1 -- 172.21.15.47:0/847713134 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e41a1e50 con 0x7f79e41030b0 2024-03-21T11:22:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.753+0000 7f79e3e00640 1 -- 172.21.15.47:0/847713134 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e41a2140 con 0x7f79e41030b0 2024-03-21T11:22:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.755+0000 7f79db400640 1 -- 172.21.15.47:0/847713134 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f79dc005540 con 0x7f79e41030b0 2024-03-21T11:22:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.757+0000 7f79db400640 1 --2- 172.21.15.47:0/847713134 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f79c80a2f20 0x7f79c80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.757+0000 7f79db400640 1 -- 172.21.15.47:0/847713134 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f79dc0cfe20 con 0x7f79e41030b0 2024-03-21T11:22:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.757+0000 7f79e2a00640 1 --2- 172.21.15.47:0/847713134 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f79c80a2f20 0x7f79c80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.758+0000 7f79e2a00640 1 --2- 172.21.15.47:0/847713134 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f79c80a2f20 0x7f79c80a53e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f79d4005ab0 tx=0x7f79d402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.758+0000 7f79db400640 1 -- 172.21.15.47:0/847713134 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f79dc09c860 con 0x7f79e41030b0 2024-03-21T11:22:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.969+0000 7f79e3e00640 1 -- 172.21.15.47:0/847713134 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79e41a2310 con 0x7f79e41030b0 2024-03-21T11:22:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.971+0000 7f79db400640 1 -- 172.21.15.47:0/847713134 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24747 (secure 0 0 0) 0x7f79dc09c160 con 0x7f79e41030b0 2024-03-21T11:22:54.977 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:22:54.977 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:22:50.841892+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:22:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.974+0000 7f79e3e00640 1 -- 172.21.15.47:0/847713134 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f79c80a2f20 msgr2=0x7f79c80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.974+0000 7f79e3e00640 1 --2- 172.21.15.47:0/847713134 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f79c80a2f20 0x7f79c80a53e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f79d4005ab0 tx=0x7f79d402cf90 comp rx=0 tx=0).stop 2024-03-21T11:22:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.974+0000 7f79e3e00640 1 -- 172.21.15.47:0/847713134 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79e41030b0 msgr2=0x7f79e41a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.974+0000 7f79e3e00640 1 --2- 172.21.15.47:0/847713134 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79e41030b0 0x7f79e41a0fa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009fd0 tx=0x7f79dc00ef10 comp rx=0 tx=0).stop 2024-03-21T11:22:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.975+0000 7f79e3e00640 1 -- 172.21.15.47:0/847713134 shutdown_connections 2024-03-21T11:22:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.975+0000 7f79e3e00640 1 --2- 172.21.15.47:0/847713134 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f79c80a2f20 0x7f79c80a53e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.975+0000 7f79e3e00640 1 --2- 172.21.15.47:0/847713134 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79e41030b0 0x7f79e41a0fa0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.975+0000 7f79e3e00640 1 --2- 172.21.15.47:0/847713134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79e41026f0 0x7f79e4199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.975+0000 7f79e3e00640 1 --2- 172.21.15.47:0/847713134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79e4101500 0x7f79e41999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.975+0000 7f79e3e00640 1 -- 172.21.15.47:0/847713134 >> 172.21.15.47:0/847713134 conn(0x7f79e40fca70 msgr2=0x7f79e40738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.975+0000 7f79e3e00640 1 -- 172.21.15.47:0/847713134 shutdown_connections 2024-03-21T11:22:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:54.975+0000 7f79e3e00640 1 -- 172.21.15.47:0/847713134 wait complete. 2024-03-21T11:22:54.997 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:22:55.158 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:54 smithi100 ceph-mon[36245]: pgmap v387: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1023 B/s wr, 13 op/s 2024-03-21T11:22:55.158 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:54 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1151664851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:22:55.158 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:54 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2434496946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:22:55.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.230+0000 7f0a23000640 1 -- 172.21.15.47:0/1716196365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a1c103700 msgr2=0x7f0a1c103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:55.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.230+0000 7f0a23000640 1 --2- 172.21.15.47:0/1716196365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a1c103700 0x7f0a1c103b20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009900 tx=0x7f0a0c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:22:55.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.230+0000 7f0a23000640 1 -- 172.21.15.47:0/1716196365 shutdown_connections 2024-03-21T11:22:55.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.230+0000 7f0a23000640 1 --2- 172.21.15.47:0/1716196365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1c1052b0 0x7f0a1c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.230+0000 7f0a23000640 1 --2- 172.21.15.47:0/1716196365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a1c1048f0 0x7f0a1c104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.230+0000 7f0a23000640 1 --2- 172.21.15.47:0/1716196365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a1c103700 0x7f0a1c103b20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.230+0000 7f0a23000640 1 -- 172.21.15.47:0/1716196365 >> 172.21.15.47:0/1716196365 conn(0x7f0a1c0fec70 msgr2=0x7f0a1c1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:55.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.230+0000 7f0a23000640 1 -- 172.21.15.47:0/1716196365 shutdown_connections 2024-03-21T11:22:55.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.230+0000 7f0a23000640 1 -- 172.21.15.47:0/1716196365 wait complete. 2024-03-21T11:22:55.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.232+0000 7f0a23000640 1 Processor -- start 2024-03-21T11:22:55.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.232+0000 7f0a23000640 1 -- start start 2024-03-21T11:22:55.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.232+0000 7f0a23000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a1c103700 0x7f0a1c199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.232+0000 7f0a23000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a1c1048f0 0x7f0a1c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.232+0000 7f0a23000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1c1052b0 0x7f0a1c1a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.232+0000 7f0a23000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c10f3a0 con 0x7f0a1c1052b0 2024-03-21T11:22:55.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.232+0000 7f0a23000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c10f220 con 0x7f0a1c1048f0 2024-03-21T11:22:55.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.232+0000 7f0a23000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c10f520 con 0x7f0a1c103700 2024-03-21T11:22:55.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.233+0000 7f0a21c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a1c103700 0x7f0a1c199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.233+0000 7f0a21c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a1c103700 0x7f0a1c199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41702/0 (socket says 172.21.15.47:41702) 2024-03-21T11:22:55.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.233+0000 7f0a21c00640 1 -- 172.21.15.47:0/3896736218 learned_addr learned my addr 172.21.15.47:0/3896736218 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:55.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.233+0000 7f0a21200640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a1c1048f0 0x7f0a1c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.233+0000 7f0a22600640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1c1052b0 0x7f0a1c1a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.233+0000 7f0a21c00640 1 -- 172.21.15.47:0/3896736218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a1c1048f0 msgr2=0x7f0a1c199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:55.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.233+0000 7f0a21c00640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a1c1048f0 0x7f0a1c199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.233+0000 7f0a21c00640 1 -- 172.21.15.47:0/3896736218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1c1052b0 msgr2=0x7f0a1c1a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:55.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.233+0000 7f0a21c00640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1c1052b0 0x7f0a1c1a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.233+0000 7f0a21c00640 1 -- 172.21.15.47:0/3896736218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c1a1660 con 0x7f0a1c103700 2024-03-21T11:22:55.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.233+0000 7f0a22600640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1c1052b0 0x7f0a1c1a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:22:55.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.233+0000 7f0a21c00640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a1c103700 0x7f0a1c199a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c02efd0 tx=0x7f0a0c02c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:55.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.234+0000 7f0a1a000640 1 -- 172.21.15.47:0/3896736218 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c041070 con 0x7f0a1c103700 2024-03-21T11:22:55.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.234+0000 7f0a1a000640 1 -- 172.21.15.47:0/3896736218 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a0c004450 con 0x7f0a1c103700 2024-03-21T11:22:55.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.234+0000 7f0a1a000640 1 -- 172.21.15.47:0/3896736218 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c003e50 con 0x7f0a1c103700 2024-03-21T11:22:55.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.234+0000 7f0a23000640 1 -- 172.21.15.47:0/3896736218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1c1a18f0 con 0x7f0a1c103700 2024-03-21T11:22:55.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.234+0000 7f0a23000640 1 -- 172.21.15.47:0/3896736218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1c1a1e00 con 0x7f0a1c103700 2024-03-21T11:22:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.235+0000 7f0a23000640 1 -- 172.21.15.47:0/3896736218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1c0683b0 con 0x7f0a1c103700 2024-03-21T11:22:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.237+0000 7f0a1a000640 1 -- 172.21.15.47:0/3896736218 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0a0c044070 con 0x7f0a1c103700 2024-03-21T11:22:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.238+0000 7f0a1a000640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a000a3140 0x7f0a000a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.238+0000 7f0a1a000640 1 -- 172.21.15.47:0/3896736218 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0a0c0f1380 con 0x7f0a1c103700 2024-03-21T11:22:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.238+0000 7f0a21200640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a000a3140 0x7f0a000a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.239+0000 7f0a21200640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a000a3140 0x7f0a000a5600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c19af50 tx=0x7f0a08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:54 smithi102 ceph-mon[36748]: pgmap v387: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1023 B/s wr, 13 op/s 2024-03-21T11:22:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:54 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1151664851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:22:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:54 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2434496946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:22:55.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.244+0000 7f0a1a000640 1 -- 172.21.15.47:0/3896736218 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0a0c0bddc0 con 0x7f0a1c103700 2024-03-21T11:22:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:54 smithi047 ceph-mon[31351]: pgmap v387: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1023 B/s wr, 13 op/s 2024-03-21T11:22:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:54 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1151664851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:22:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:54 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2434496946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:22:55.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.489+0000 7f0a23000640 1 -- 172.21.15.47:0/3896736218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a1c0622a0 con 0x7f0a1c103700 2024-03-21T11:22:55.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.491+0000 7f0a1a000640 1 -- 172.21.15.47:0/3896736218 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f0a0c0bd6c0 con 0x7f0a1c103700 2024-03-21T11:22:55.492 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:22:55.492 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:21:18.592954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24524,24523,24535,24581,24553]},"id":1}]} 2024-03-21T11:22:55.492 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 27 2024-03-21T11:22:55.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.495+0000 7f0a23000640 1 -- 172.21.15.47:0/3896736218 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a000a3140 msgr2=0x7f0a000a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.495+0000 7f0a23000640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a000a3140 0x7f0a000a5600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c19af50 tx=0x7f0a08005f90 comp rx=0 tx=0).stop 2024-03-21T11:22:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.495+0000 7f0a23000640 1 -- 172.21.15.47:0/3896736218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a1c103700 msgr2=0x7f0a1c199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.495+0000 7f0a23000640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a1c103700 0x7f0a1c199a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c02efd0 tx=0x7f0a0c02c660 comp rx=0 tx=0).stop 2024-03-21T11:22:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.495+0000 7f0a23000640 1 -- 172.21.15.47:0/3896736218 shutdown_connections 2024-03-21T11:22:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.495+0000 7f0a23000640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a000a3140 0x7f0a000a5600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.495+0000 7f0a23000640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a1c1052b0 0x7f0a1c1a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.495+0000 7f0a23000640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a1c1048f0 0x7f0a1c199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.495+0000 7f0a23000640 1 --2- 172.21.15.47:0/3896736218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a1c103700 0x7f0a1c199a40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.495+0000 7f0a23000640 1 -- 172.21.15.47:0/3896736218 >> 172.21.15.47:0/3896736218 conn(0x7f0a1c0fec70 msgr2=0x7f0a1c107c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.495+0000 7f0a23000640 1 -- 172.21.15.47:0/3896736218 shutdown_connections 2024-03-21T11:22:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.495+0000 7f0a23000640 1 -- 172.21.15.47:0/3896736218 wait complete. 2024-03-21T11:22:55.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-21T11:20:48.626416+0000', 'modified': '2024-03-21T11:21:18.592954+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24581, 'mds_1': 24553, 'mds_2': 24535, 'mds_3': 24524, 'mds_4': 24523}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14466': {'gid': 14466, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.47:6841/1742084729', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6840', 'nonce': 1742084729}, {'type': 'v1', 'addr': '172.21.15.47:6841', 'nonce': 1742084729}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24446': {'gid': 24446, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6833/904714287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6832', 'nonce': 904714287}, {'type': 'v1', 'addr': '172.21.15.100:6833', 'nonce': 904714287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24457': {'gid': 24457, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.102:6833/1794606435', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6832', 'nonce': 1794606435}, {'type': 'v1', 'addr': '172.21.15.102:6833', 'nonce': 1794606435}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24476': {'gid': 24476, 'name': 'd', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.47:6837/1846721633', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 1846721633}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 1846721633}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6835/2785789126', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 2785789126}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 2785789126}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.102:6835/1182573386', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6834', 'nonce': 1182573386}, {'type': 'v1', 'addr': '172.21.15.102:6835', 'nonce': 1182573386}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.100:6837/1118767699', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 1118767699}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 1118767699}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.102:6837/1198140504', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6836', 'nonce': 1198140504}, {'type': 'v1', 'addr': '172.21.15.102:6837', 'nonce': 1198140504}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24553': {'gid': 24553, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.102:6839/2885246980', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 2885246980}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 2885246980}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6839/2708345792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 2708345792}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 2708345792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24581, 'qdb_cluster': [24524, 24523, 24535, 24581, 24553]} 2024-03-21T11:22:55.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-21T11:22:55.515 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-21T11:22:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.742+0000 7f2d67e00640 1 -- 172.21.15.47:0/2349806563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d681049b0 msgr2=0x7f2d68104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.742+0000 7f2d67e00640 1 --2- 172.21.15.47:0/2349806563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d681049b0 0x7f2d68104e30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009900 tx=0x7f2d5002bc40 comp rx=0 tx=0).stop 2024-03-21T11:22:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.742+0000 7f2d67e00640 1 -- 172.21.15.47:0/2349806563 shutdown_connections 2024-03-21T11:22:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.742+0000 7f2d67e00640 1 --2- 172.21.15.47:0/2349806563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d68105370 0x7f2d6810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.742+0000 7f2d67e00640 1 --2- 172.21.15.47:0/2349806563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d681049b0 0x7f2d68104e30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.742+0000 7f2d67e00640 1 --2- 172.21.15.47:0/2349806563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d681037c0 0x7f2d68103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.742+0000 7f2d67e00640 1 -- 172.21.15.47:0/2349806563 >> 172.21.15.47:0/2349806563 conn(0x7f2d680fed50 msgr2=0x7f2d68101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.742+0000 7f2d67e00640 1 -- 172.21.15.47:0/2349806563 shutdown_connections 2024-03-21T11:22:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.742+0000 7f2d67e00640 1 -- 172.21.15.47:0/2349806563 wait complete. 2024-03-21T11:22:55.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.743+0000 7f2d67e00640 1 Processor -- start 2024-03-21T11:22:55.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.744+0000 7f2d67e00640 1 -- start start 2024-03-21T11:22:55.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.744+0000 7f2d67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d681037c0 0x7f2d68199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.744+0000 7f2d67e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d681049b0 0x7f2d68199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.744+0000 7f2d67e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d68105370 0x7f2d681a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.744+0000 7f2d67e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6810f420 con 0x7f2d68105370 2024-03-21T11:22:55.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.744+0000 7f2d67e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6810f2a0 con 0x7f2d681049b0 2024-03-21T11:22:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.744+0000 7f2d67e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6810f5a0 con 0x7f2d681037c0 2024-03-21T11:22:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.744+0000 7f2d66a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d681037c0 0x7f2d68199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.744+0000 7f2d66a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d681037c0 0x7f2d68199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41720/0 (socket says 172.21.15.47:41720) 2024-03-21T11:22:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.744+0000 7f2d66a00640 1 -- 172.21.15.47:0/1712701556 learned_addr learned my addr 172.21.15.47:0/1712701556 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.744+0000 7f2d5e000640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d681049b0 0x7f2d68199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.745+0000 7f2d67400640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d68105370 0x7f2d681a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.745+0000 7f2d66a00640 1 -- 172.21.15.47:0/1712701556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d681049b0 msgr2=0x7f2d68199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.745+0000 7f2d66a00640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d681049b0 0x7f2d68199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.745+0000 7f2d66a00640 1 -- 172.21.15.47:0/1712701556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d68105370 msgr2=0x7f2d681a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.745+0000 7f2d66a00640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d68105370 0x7f2d681a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.745+0000 7f2d66a00640 1 -- 172.21.15.47:0/1712701556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d681a1690 con 0x7f2d681037c0 2024-03-21T11:22:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.745+0000 7f2d5e000640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d681049b0 0x7f2d68199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.745+0000 7f2d66a00640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d681037c0 0x7f2d68199a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800e9c0 tx=0x7f2d5800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.745+0000 7f2d5fe00640 1 -- 172.21.15.47:0/1712701556 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5800cde0 con 0x7f2d681037c0 2024-03-21T11:22:55.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.745+0000 7f2d5fe00640 1 -- 172.21.15.47:0/1712701556 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d58010910 con 0x7f2d681037c0 2024-03-21T11:22:55.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.745+0000 7f2d5fe00640 1 -- 172.21.15.47:0/1712701556 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58018a00 con 0x7f2d681037c0 2024-03-21T11:22:55.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.746+0000 7f2d67e00640 1 -- 172.21.15.47:0/1712701556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d681a1980 con 0x7f2d681037c0 2024-03-21T11:22:55.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.746+0000 7f2d67e00640 1 -- 172.21.15.47:0/1712701556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d681a1e50 con 0x7f2d681037c0 2024-03-21T11:22:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.749+0000 7f2d5fe00640 1 -- 172.21.15.47:0/1712701556 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2d5801f020 con 0x7f2d681037c0 2024-03-21T11:22:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.749+0000 7f2d7063a740 1 Processor -- start 2024-03-21T11:22:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.749+0000 7f2d7063a740 1 -- start start 2024-03-21T11:22:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.749+0000 7f2d7063a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 0x555f71ef81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.749+0000 7f2d7063a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71efd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.749+0000 7f2d7063a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 0x555f71effd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.749+0000 7f2d7063a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f71e5cc90 con 0x555f71efd8e0 2024-03-21T11:22:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.749+0000 7f2d7063a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f71e05980 con 0x555f71ef7d80 2024-03-21T11:22:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.749+0000 7f2d7063a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f71e58d30 con 0x555f71ef8770 2024-03-21T11:22:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.750+0000 7f2d67400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 0x555f71ef81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.750+0000 7f2d66a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71efd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.750+0000 7f2d67400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 0x555f71ef81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52928/0 (socket says 172.21.15.47:52928) 2024-03-21T11:22:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.750+0000 7f2d67400640 1 -- 172.21.15.47:0/518667984 learned_addr learned my addr 172.21.15.47:0/518667984 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.750+0000 7f2d5e000640 1 --2- 172.21.15.47:0/518667984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 0x555f71effd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.750+0000 7f2d67400640 1 -- 172.21.15.47:0/518667984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 msgr2=0x555f71efd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.750+0000 7f2d67400640 1 --2- 172.21.15.47:0/518667984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71efd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.750+0000 7f2d67400640 1 -- 172.21.15.47:0/518667984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 msgr2=0x555f71effd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.750+0000 7f2d67400640 1 --2- 172.21.15.47:0/518667984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 0x555f71effd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.750+0000 7f2d67400640 1 -- 172.21.15.47:0/518667984 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f71e1d510 con 0x555f71ef7d80 2024-03-21T11:22:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.750+0000 7f2d5fe00640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d4c0a2fa0 0x7f2d4c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.750+0000 7f2d67400640 1 --2- 172.21.15.47:0/518667984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 0x555f71ef81a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009fd0 tx=0x7f2d600118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7536fb1d04662ddc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.751+0000 7f2d57e00640 1 -- 172.21.15.47:0/518667984 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60004d20 con 0x555f71ef7d80 2024-03-21T11:22:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.751+0000 7f2d5fe00640 1 -- 172.21.15.47:0/1712701556 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2d580cf930 con 0x7f2d681037c0 2024-03-21T11:22:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.751+0000 7f2d57e00640 1 -- 172.21.15.47:0/518667984 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d60004050 con 0x555f71ef7d80 2024-03-21T11:22:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.751+0000 7f2d57e00640 1 -- 172.21.15.47:0/518667984 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60005260 con 0x555f71ef7d80 2024-03-21T11:22:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.751+0000 7f2d66a00640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d4c0a2fa0 0x7f2d4c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.751+0000 7f2d7063a740 1 -- 172.21.15.47:0/518667984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 msgr2=0x555f71ef81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.751+0000 7f2d66a00640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d4c0a2fa0 0x7f2d4c0a5460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2d580a4150 tx=0x7f2d580d5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.751+0000 7f2d7063a740 1 --2- 172.21.15.47:0/518667984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 0x555f71ef81a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009fd0 tx=0x7f2d600118d0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.752+0000 7f2d7063a740 1 -- 172.21.15.47:0/518667984 shutdown_connections 2024-03-21T11:22:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.752+0000 7f2d7063a740 1 --2- 172.21.15.47:0/518667984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 0x555f71effd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.752+0000 7f2d7063a740 1 --2- 172.21.15.47:0/518667984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71efd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.752+0000 7f2d7063a740 1 --2- 172.21.15.47:0/518667984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 0x555f71ef81a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.752+0000 7f2d7063a740 1 -- 172.21.15.47:0/518667984 >> 172.21.15.47:0/518667984 conn(0x555f71ef32f0 msgr2=0x555f71ef36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.752+0000 7f2d7063a740 1 -- 172.21.15.47:0/518667984 shutdown_connections 2024-03-21T11:22:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.752+0000 7f2d7063a740 1 -- 172.21.15.47:0/518667984 wait complete. 2024-03-21T11:22:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.752+0000 7f2d7063a740 1 -- start start 2024-03-21T11:22:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.752+0000 7f2d7063a740 1 Processor -- start 2024-03-21T11:22:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d7063a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 0x555f71f02270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d7063a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71f027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d7063a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 0x555f71f097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d7063a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f71f00890 con 0x555f71efd8e0 2024-03-21T11:22:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d7063a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f71e18dd0 con 0x555f71ef7d80 2024-03-21T11:22:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d7063a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f71f00a10 con 0x555f71ef8770 2024-03-21T11:22:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d7063a740 10 client.0 _ll_register_callbacks cb 0x555f71b70ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:22:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d5e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71f027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d5e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71f027b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41736/0 (socket says 172.21.15.47:41736) 2024-03-21T11:22:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d5e000640 1 -- 172.21.15.47:0/3449964957 learned_addr learned my addr 172.21.15.47:0/3449964957 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d67400640 1 --2- 172.21.15.47:0/3449964957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 0x555f71f02270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d67400640 1 --2- 172.21.15.47:0/3449964957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 0x555f71f097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d5e000640 1 -- 172.21.15.47:0/3449964957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 msgr2=0x555f71f02270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d5e000640 1 --2- 172.21.15.47:0/3449964957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 0x555f71f02270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d5e000640 1 -- 172.21.15.47:0/3449964957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 msgr2=0x555f71f097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d5e000640 1 --2- 172.21.15.47:0/3449964957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 0x555f71f097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.753+0000 7f2d5e000640 1 -- 172.21.15.47:0/3449964957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f71b712d0 con 0x555f71ef8770 2024-03-21T11:22:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.754+0000 7f2d5e000640 1 --2- 172.21.15.47:0/3449964957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71f027b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009900 tx=0x7f2d50005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c8bddaf72e09ad1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.754+0000 7f2d54c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:22:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.754+0000 7f2d54c00640 1 -- 172.21.15.47:0/3449964957 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50005d50 con 0x555f71ef8770 2024-03-21T11:22:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.754+0000 7f2d54c00640 1 -- 172.21.15.47:0/3449964957 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2d500048c0 con 0x555f71ef8770 2024-03-21T11:22:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.754+0000 7f2d7063a740 1 -- 172.21.15.47:0/3449964957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d60004050 con 0x555f71ef8770 2024-03-21T11:22:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.754+0000 7f2d54c00640 1 -- 172.21.15.47:0/3449964957 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d50050050 con 0x555f71ef8770 2024-03-21T11:22:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.754+0000 7f2d7063a740 10 client.25165 fetch_fsmap learned FSMap version 27 2024-03-21T11:22:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.754+0000 7f2d7063a740 1 -- 172.21.15.47:0/3449964957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555f71c3b6b0 con 0x555f71ef8770 2024-03-21T11:22:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.755+0000 7f2d54c00640 1 -- 172.21.15.47:0/3449964957 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d60004050 con 0x555f71ef8770 2024-03-21T11:22:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.755+0000 7f2d54c00640 1 -- 172.21.15.47:0/3449964957 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2d50003870 con 0x555f71ef8770 2024-03-21T11:22:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.755+0000 7f2d7063a740 10 client.25165 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:22:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.755+0000 7f2d7063a740 10 client.25165 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-21T11:22:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.755+0000 7f2d7063a740 20 client.25165 populate_metadata read hostname 'smithi047' 2024-03-21T11:22:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.755+0000 7f2d7063a740 1 --2- 172.21.15.47:0/3449964957 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555f71f0ea40 0x555f71f0dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.755+0000 7f2d7063a740 4 client.25165 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:22:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.755+0000 7f2d7063a740 1 -- 172.21.15.47:0/3449964957 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f71dfe520 con 0x555f71f0ea40 2024-03-21T11:22:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.755+0000 7f2d67400640 1 --2- 172.21.15.47:0/3449964957 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555f71f0ea40 0x555f71f0dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.756+0000 7f2d67400640 1 --2- 172.21.15.47:0/3449964957 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555f71f0ea40 0x555f71f0dad0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3869f6ee07fde7f6 server_cookie=ddc6621c1f9065b7 in_seq=0 out_seq=0 2024-03-21T11:22:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.756+0000 7f2d54c00640 10 client.25165 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:22:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d54c00640 1 -- 172.21.15.47:0/3449964957 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555f71dfe520 con 0x555f71f0ea40 2024-03-21T11:22:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d54c00640 10 client.25165 handle_command_reply: tid=0 2024-03-21T11:22:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d7063a740 1 client.25165 shutdown 2024-03-21T11:22:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d7063a740 1 -- 172.21.15.47:0/3449964957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 msgr2=0x555f71f027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d7063a740 1 --2- 172.21.15.47:0/3449964957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71f027b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009900 tx=0x7f2d50005270 comp rx=0 tx=0).stop 2024-03-21T11:22:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d7063a740 1 -- 172.21.15.47:0/3449964957 shutdown_connections 2024-03-21T11:22:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d7063a740 1 --2- 172.21.15.47:0/3449964957 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555f71f0ea40 0x555f71f0dad0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d7063a740 1 --2- 172.21.15.47:0/3449964957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 0x555f71f097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d7063a740 1 --2- 172.21.15.47:0/3449964957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71f027b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d54c00640 0 client.25165 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:22:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d7063a740 1 --2- 172.21.15.47:0/3449964957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 0x555f71f02270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d7063a740 1 -- 172.21.15.47:0/3449964957 >> 172.21.15.47:0/3449964957 conn(0x555f71ef32f0 msgr2=0x555f71ef24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d7063a740 1 -- 172.21.15.47:0/3449964957 shutdown_connections 2024-03-21T11:22:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d7063a740 1 -- 172.21.15.47:0/3449964957 wait complete. 2024-03-21T11:22:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.757+0000 7f2d7063a740 20 client.25165 trim_cache size 0 max 16384 2024-03-21T11:22:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.776+0000 7f2d7063a740 1 Processor -- start 2024-03-21T11:22:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.777+0000 7f2d7063a740 1 -- start start 2024-03-21T11:22:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.777+0000 7f2d7063a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef7d80 0x555f71f00db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.777+0000 7f2d7063a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71ef8770 0x555f71f097b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.777+0000 7f2d7063a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71efd8e0 0x555f71f0dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.777+0000 7f2d7063a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f71f0e260 con 0x555f71ef8770 2024-03-21T11:22:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.777+0000 7f2d7063a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f71f0e8c0 con 0x555f71efd8e0 2024-03-21T11:22:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.777+0000 7f2d7063a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f71f0ef20 con 0x555f71ef7d80 2024-03-21T11:22:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.777+0000 7f2d67400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef7d80 0x555f71f00db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.777+0000 7f2d67400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef7d80 0x555f71f00db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41738/0 (socket says 172.21.15.47:41738) 2024-03-21T11:22:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.777+0000 7f2d67400640 1 -- 172.21.15.47:0/1205767235 learned_addr learned my addr 172.21.15.47:0/1205767235 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.777+0000 7f2d67400640 1 --2- 172.21.15.47:0/1205767235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71efd8e0 0x555f71f0dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.777+0000 7f2d5e000640 1 --2- 172.21.15.47:0/1205767235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71ef8770 0x555f71f097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.778+0000 7f2d67400640 1 -- 172.21.15.47:0/1205767235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71efd8e0 msgr2=0x555f71f0dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.778+0000 7f2d67400640 1 --2- 172.21.15.47:0/1205767235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71efd8e0 0x555f71f0dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.778+0000 7f2d67400640 1 -- 172.21.15.47:0/1205767235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71ef8770 msgr2=0x555f71f097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.778+0000 7f2d67400640 1 --2- 172.21.15.47:0/1205767235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71ef8770 0x555f71f097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.778+0000 7f2d67400640 1 -- 172.21.15.47:0/1205767235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f71ee5170 con 0x555f71ef7d80 2024-03-21T11:22:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.778+0000 7f2d67400640 1 --2- 172.21.15.47:0/1205767235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef7d80 0x555f71f00db0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2d60014040 tx=0x7f2d60009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d40e1748820186c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.778+0000 7f2d57400640 1 -- 172.21.15.47:0/1205767235 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60009c70 con 0x555f71ef7d80 2024-03-21T11:22:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.778+0000 7f2d57400640 1 -- 172.21.15.47:0/1205767235 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d600050e0 con 0x555f71ef7d80 2024-03-21T11:22:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.778+0000 7f2d57400640 1 -- 172.21.15.47:0/1205767235 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000f740 con 0x555f71ef7d80 2024-03-21T11:22:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.779+0000 7f2d7063a740 1 -- 172.21.15.47:0/1205767235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef7d80 msgr2=0x555f71f00db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.779+0000 7f2d7063a740 1 --2- 172.21.15.47:0/1205767235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef7d80 0x555f71f00db0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2d60014040 tx=0x7f2d60009500 comp rx=0 tx=0).stop 2024-03-21T11:22:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.779+0000 7f2d7063a740 1 -- 172.21.15.47:0/1205767235 shutdown_connections 2024-03-21T11:22:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.779+0000 7f2d7063a740 1 --2- 172.21.15.47:0/1205767235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71efd8e0 0x555f71f0dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.779+0000 7f2d7063a740 1 --2- 172.21.15.47:0/1205767235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71ef8770 0x555f71f097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.779+0000 7f2d7063a740 1 --2- 172.21.15.47:0/1205767235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef7d80 0x555f71f00db0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.779+0000 7f2d7063a740 1 -- 172.21.15.47:0/1205767235 >> 172.21.15.47:0/1205767235 conn(0x555f71ef32f0 msgr2=0x555f71ef9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.779+0000 7f2d7063a740 1 -- 172.21.15.47:0/1205767235 shutdown_connections 2024-03-21T11:22:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.779+0000 7f2d7063a740 1 -- 172.21.15.47:0/1205767235 wait complete. 2024-03-21T11:22:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.780+0000 7f2d7063a740 1 -- start start 2024-03-21T11:22:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.780+0000 7f2d7063a740 1 Processor -- start 2024-03-21T11:22:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.780+0000 7f2d7063a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 0x555f71f090c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.780+0000 7f2d7063a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71f09600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.780+0000 7f2d7063a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 0x555f71f05b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.780+0000 7f2d7063a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f71f0fbe0 con 0x555f71efd8e0 2024-03-21T11:22:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.780+0000 7f2d7063a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f71f0f580 con 0x555f71ef7d80 2024-03-21T11:22:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.780+0000 7f2d7063a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f71f10240 con 0x555f71ef8770 2024-03-21T11:22:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.780+0000 7f2d7063a740 10 client.0 _ll_register_callbacks cb 0x7f2d58005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:22:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.781+0000 7f2d5e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71f09600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.781+0000 7f2d5e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71f09600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41754/0 (socket says 172.21.15.47:41754) 2024-03-21T11:22:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.781+0000 7f2d5e000640 1 -- 172.21.15.47:0/1204217382 learned_addr learned my addr 172.21.15.47:0/1204217382 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.781+0000 7f2d67400640 1 --2- 172.21.15.47:0/1204217382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 0x555f71f090c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.781+0000 7f2d67400640 1 --2- 172.21.15.47:0/1204217382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 0x555f71f05b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.781+0000 7f2d5e000640 1 -- 172.21.15.47:0/1204217382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 msgr2=0x555f71f090c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.781+0000 7f2d5e000640 1 --2- 172.21.15.47:0/1204217382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 0x555f71f090c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.781+0000 7f2d5e000640 1 -- 172.21.15.47:0/1204217382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 msgr2=0x555f71f05b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.781+0000 7f2d5e000640 1 --2- 172.21.15.47:0/1204217382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 0x555f71f05b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.781+0000 7f2d5e000640 1 -- 172.21.15.47:0/1204217382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f71f062d0 con 0x555f71ef8770 2024-03-21T11:22:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.781+0000 7f2d5e000640 1 --2- 172.21.15.47:0/1204217382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71f09600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d50000c00 tx=0x7f2d5002c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29ff2fa439255fc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.781+0000 7f2d54c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:22:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.782+0000 7f2d54c00640 1 -- 172.21.15.47:0/1204217382 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50005d50 con 0x555f71ef8770 2024-03-21T11:22:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.782+0000 7f2d54c00640 1 -- 172.21.15.47:0/1204217382 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2d50003800 con 0x555f71ef8770 2024-03-21T11:22:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.782+0000 7f2d7063a740 1 -- 172.21.15.47:0/1204217382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d600050e0 con 0x555f71ef8770 2024-03-21T11:22:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.782+0000 7f2d54c00640 1 -- 172.21.15.47:0/1204217382 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d50061050 con 0x555f71ef8770 2024-03-21T11:22:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.782+0000 7f2d7063a740 10 client.25171 fetch_fsmap learned FSMap version 27 2024-03-21T11:22:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.782+0000 7f2d7063a740 1 -- 172.21.15.47:0/1204217382 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555f71f6a780 con 0x555f71ef8770 2024-03-21T11:22:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.782+0000 7f2d54c00640 1 -- 172.21.15.47:0/1204217382 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d600050e0 con 0x555f71ef8770 2024-03-21T11:22:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.782+0000 7f2d54c00640 1 -- 172.21.15.47:0/1204217382 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2d5005c030 con 0x555f71ef8770 2024-03-21T11:22:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.782+0000 7f2d7063a740 10 client.25171 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:22:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.782+0000 7f2d7063a740 10 client.25171 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-21T11:22:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.782+0000 7f2d7063a740 20 client.25171 populate_metadata read hostname 'smithi047' 2024-03-21T11:22:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.782+0000 7f2d7063a740 1 --2- 172.21.15.47:0/1204217382 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555f71f6ee60 0x555f71f6aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.782+0000 7f2d7063a740 4 client.25171 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T11:22:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.782+0000 7f2d7063a740 1 -- 172.21.15.47:0/1204217382 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555f71e6a1d0 con 0x555f71f6ee60 2024-03-21T11:22:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.783+0000 7f2d67400640 1 --2- 172.21.15.47:0/1204217382 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555f71f6ee60 0x555f71f6aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.783+0000 7f2d67400640 1 --2- 172.21.15.47:0/1204217382 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555f71f6ee60 0x555f71f6aa70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de91bdffea5e60ca server_cookie=5374cf8765f7afdf in_seq=0 out_seq=0 2024-03-21T11:22:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.783+0000 7f2d54c00640 10 client.25171 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:22:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.783+0000 7f2d54c00640 1 -- 172.21.15.47:0/1204217382 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x555f71e6a1d0 con 0x555f71f6ee60 2024-03-21T11:22:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.783+0000 7f2d54c00640 10 client.25171 handle_command_reply: tid=0 2024-03-21T11:22:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.783+0000 7f2d7063a740 1 client.25171 shutdown 2024-03-21T11:22:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.783+0000 7f2d7063a740 1 -- 172.21.15.47:0/1204217382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 msgr2=0x555f71f09600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.783+0000 7f2d7063a740 1 --2- 172.21.15.47:0/1204217382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71f09600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d50000c00 tx=0x7f2d5002c6d0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.784+0000 7f2d7063a740 1 -- 172.21.15.47:0/1204217382 shutdown_connections 2024-03-21T11:22:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.784+0000 7f2d7063a740 1 --2- 172.21.15.47:0/1204217382 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555f71f6ee60 0x555f71f6aa70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.784+0000 7f2d7063a740 1 --2- 172.21.15.47:0/1204217382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f71efd8e0 0x555f71f05b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.784+0000 7f2d7063a740 1 --2- 172.21.15.47:0/1204217382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f71ef8770 0x555f71f09600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.784+0000 7f2d7063a740 1 --2- 172.21.15.47:0/1204217382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f71ef7d80 0x555f71f090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.784+0000 7f2d54c00640 0 client.25171 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:22:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.784+0000 7f2d7063a740 1 -- 172.21.15.47:0/1204217382 >> 172.21.15.47:0/1204217382 conn(0x555f71ef32f0 msgr2=0x555f71f00db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.784+0000 7f2d7063a740 1 -- 172.21.15.47:0/1204217382 shutdown_connections 2024-03-21T11:22:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.784+0000 7f2d7063a740 1 -- 172.21.15.47:0/1204217382 wait complete. 2024-03-21T11:22:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.784+0000 7f2d7063a740 20 client.25171 trim_cache size 0 max 16384 2024-03-21T11:22:55.791 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:22:55.792 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:22:55.792 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 4, 2024-03-21T11:22:55.792 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24523, 2024-03-21T11:22:55.793 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T11:22:55.793 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T11:22:55.793 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T11:22:55.793 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 97.910350206000004, 2024-03-21T11:22:55.793 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 27, 2024-03-21T11:22:55.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T11:22:55.794 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-03-21T11:22:55.794 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 153.43826625299999 2024-03-21T11:22:55.794 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:22:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.784+0000 7f2d67e00640 1 -- 172.21.15.47:0/1712701556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d4c0a2fa0 msgr2=0x7f2d4c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.784+0000 7f2d67e00640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d4c0a2fa0 0x7f2d4c0a5460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2d580a4150 tx=0x7f2d580d5530 comp rx=0 tx=0).stop 2024-03-21T11:22:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.784+0000 7f2d67e00640 1 -- 172.21.15.47:0/1712701556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d681037c0 msgr2=0x7f2d68199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.784+0000 7f2d67e00640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d681037c0 0x7f2d68199a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800e9c0 tx=0x7f2d5800ee80 comp rx=0 tx=0).stop 2024-03-21T11:22:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.785+0000 7f2d67e00640 1 -- 172.21.15.47:0/1712701556 shutdown_connections 2024-03-21T11:22:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.785+0000 7f2d67e00640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d4c0a2fa0 0x7f2d4c0a5460 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.785+0000 7f2d67e00640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d68105370 0x7f2d681a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.785+0000 7f2d67e00640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d681049b0 0x7f2d68199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.785+0000 7f2d67e00640 1 --2- 172.21.15.47:0/1712701556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d681037c0 0x7f2d68199a70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.785+0000 7f2d67e00640 1 -- 172.21.15.47:0/1712701556 >> 172.21.15.47:0/1712701556 conn(0x7f2d680fed50 msgr2=0x7f2d68107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.785+0000 7f2d67e00640 1 -- 172.21.15.47:0/1712701556 shutdown_connections 2024-03-21T11:22:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:55.785+0000 7f2d67e00640 1 -- 172.21.15.47:0/1712701556 wait complete. 2024-03-21T11:22:55.803 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-21T11:22:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.015+0000 7f6fde400640 1 -- 172.21.15.47:0/3747261400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fd80772a0 msgr2=0x7f6fd8077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.015+0000 7f6fde400640 1 --2- 172.21.15.47:0/3747261400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fd80772a0 0x7f6fd8077700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009900 tx=0x7f6fc402bc30 comp rx=0 tx=0).stop 2024-03-21T11:22:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.016+0000 7f6fde400640 1 -- 172.21.15.47:0/3747261400 shutdown_connections 2024-03-21T11:22:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.016+0000 7f6fde400640 1 --2- 172.21.15.47:0/3747261400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8077c40 0x7f6fd80784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.016+0000 7f6fde400640 1 --2- 172.21.15.47:0/3747261400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fd80772a0 0x7f6fd8077700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.016+0000 7f6fde400640 1 --2- 172.21.15.47:0/3747261400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fd8076060 0x7f6fd8076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.016+0000 7f6fde400640 1 -- 172.21.15.47:0/3747261400 >> 172.21.15.47:0/3747261400 conn(0x7f6fd80fee50 msgr2=0x7f6fd8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.016+0000 7f6fde400640 1 -- 172.21.15.47:0/3747261400 shutdown_connections 2024-03-21T11:22:56.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.016+0000 7f6fde400640 1 -- 172.21.15.47:0/3747261400 wait complete. 2024-03-21T11:22:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.017+0000 7f6fde400640 1 Processor -- start 2024-03-21T11:22:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.017+0000 7f6fde400640 1 -- start start 2024-03-21T11:22:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.017+0000 7f6fde400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fd8076060 0x7f6fd819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.017+0000 7f6fde400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fd80772a0 0x7f6fd819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.017+0000 7f6fde400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8077c40 0x7f6fd81a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.017+0000 7f6fde400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd8113970 con 0x7f6fd80772a0 2024-03-21T11:22:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.017+0000 7f6fde400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd81137f0 con 0x7f6fd8077c40 2024-03-21T11:22:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.017+0000 7f6fde400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd8113af0 con 0x7f6fd8076060 2024-03-21T11:22:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.018+0000 7f6fd7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fd8076060 0x7f6fd819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.018+0000 7f6fd7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fd8076060 0x7f6fd819e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41774/0 (socket says 172.21.15.47:41774) 2024-03-21T11:22:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.018+0000 7f6fd7400640 1 -- 172.21.15.47:0/3068211719 learned_addr learned my addr 172.21.15.47:0/3068211719 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.018+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8077c40 0x7f6fd81a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.018+0000 7f6fd6a00640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fd80772a0 0x7f6fd819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.018+0000 7f6fd7400640 1 -- 172.21.15.47:0/3068211719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8077c40 msgr2=0x7f6fd81a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.018+0000 7f6fd7400640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8077c40 0x7f6fd81a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.018+0000 7f6fd7400640 1 -- 172.21.15.47:0/3068211719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fd80772a0 msgr2=0x7f6fd819e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.018+0000 7f6fd7400640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fd80772a0 0x7f6fd819e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.018+0000 7f6fd7400640 1 -- 172.21.15.47:0/3068211719 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd81a5c40 con 0x7f6fd8076060 2024-03-21T11:22:56.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.018+0000 7f6fd6a00640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fd80772a0 0x7f6fd819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:22:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.018+0000 7f6fd7400640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fd8076060 0x7f6fd819e020 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800d950 tx=0x7f6fc800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.018+0000 7f6fcfe00640 1 -- 172.21.15.47:0/3068211719 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8017070 con 0x7f6fd8076060 2024-03-21T11:22:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.019+0000 7f6fcfe00640 1 -- 172.21.15.47:0/3068211719 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fc8009c30 con 0x7f6fd8076060 2024-03-21T11:22:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.019+0000 7f6fcfe00640 1 -- 172.21.15.47:0/3068211719 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc800f6b0 con 0x7f6fd8076060 2024-03-21T11:22:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.019+0000 7f6fde400640 1 -- 172.21.15.47:0/3068211719 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd81a5f30 con 0x7f6fd8076060 2024-03-21T11:22:56.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.019+0000 7f6fde400640 1 -- 172.21.15.47:0/3068211719 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd81a6390 con 0x7f6fd8076060 2024-03-21T11:22:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.022+0000 7f6fe101d740 1 Processor -- start 2024-03-21T11:22:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.022+0000 7f6fe101d740 1 -- start start 2024-03-21T11:22:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.022+0000 7f6fcfe00640 1 -- 172.21.15.47:0/3068211719 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6fc80109d0 con 0x7f6fd8076060 2024-03-21T11:22:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.022+0000 7f6fe101d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30805ba0 0x55aa30805fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.022+0000 7f6fe101d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30806590 0x55aa3080b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.022+0000 7f6fe101d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 0x55aa3080db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.022+0000 7f6fe101d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3076ac90 con 0x55aa3080b700 2024-03-21T11:22:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.022+0000 7f6fe101d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa30713980 con 0x55aa30806590 2024-03-21T11:22:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.022+0000 7f6fe101d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa30766d30 con 0x55aa30805ba0 2024-03-21T11:22:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.022+0000 7f6fd7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30805ba0 0x55aa30805fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.023+0000 7f6fd7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30806590 0x55aa3080b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.023+0000 7f6fd7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30805ba0 0x55aa30805fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41778/0 (socket says 172.21.15.47:41778) 2024-03-21T11:22:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.023+0000 7f6fd7e00640 1 -- 172.21.15.47:0/939551 learned_addr learned my addr 172.21.15.47:0/939551 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.023+0000 7f6fd6a00640 1 --2- 172.21.15.47:0/939551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 0x55aa3080db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.023+0000 7f6fd7e00640 1 -- 172.21.15.47:0/939551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30806590 msgr2=0x55aa3080b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.023+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/939551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30806590 0x55aa3080b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.023+0000 7f6fd7e00640 1 -- 172.21.15.47:0/939551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 msgr2=0x55aa3080db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.023+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/939551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 0x55aa3080db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.023+0000 7f6fd7e00640 1 -- 172.21.15.47:0/939551 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa3072b510 con 0x55aa30805ba0 2024-03-21T11:22:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.023+0000 7f6fd6a00640 1 --2- 172.21.15.47:0/939551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 0x55aa3080db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:22:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.023+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/939551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30805ba0 0x55aa30805fc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009fd0 tx=0x7f6fd00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3c75d50dc65ed3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.023+0000 7f6fccc00640 1 -- 172.21.15.47:0/939551 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0004d20 con 0x55aa30805ba0 2024-03-21T11:22:56.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.023+0000 7f6fccc00640 1 -- 172.21.15.47:0/939551 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fd0004050 con 0x55aa30805ba0 2024-03-21T11:22:56.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.024+0000 7f6fccc00640 1 -- 172.21.15.47:0/939551 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0005260 con 0x55aa30805ba0 2024-03-21T11:22:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.024+0000 7f6fcfe00640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6fbc0a2f20 0x7f6fbc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.024+0000 7f6fe101d740 1 -- 172.21.15.47:0/939551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30805ba0 msgr2=0x55aa30805fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.024+0000 7f6fe101d740 1 --2- 172.21.15.47:0/939551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30805ba0 0x55aa30805fc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009fd0 tx=0x7f6fd00118d0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.024+0000 7f6fcfe00640 1 -- 172.21.15.47:0/3068211719 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6fc80cf3d0 con 0x7f6fd8076060 2024-03-21T11:22:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.024+0000 7f6fe101d740 1 -- 172.21.15.47:0/939551 shutdown_connections 2024-03-21T11:22:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.024+0000 7f6fe101d740 1 --2- 172.21.15.47:0/939551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 0x55aa3080db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.024+0000 7f6fe101d740 1 --2- 172.21.15.47:0/939551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30806590 0x55aa3080b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.024+0000 7f6fe101d740 1 --2- 172.21.15.47:0/939551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30805ba0 0x55aa30805fc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.024+0000 7f6fe101d740 1 -- 172.21.15.47:0/939551 >> 172.21.15.47:0/939551 conn(0x55aa30801110 msgr2=0x55aa30801510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.024+0000 7f6fd7400640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6fbc0a2f20 0x7f6fbc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.025+0000 7f6fe101d740 1 -- 172.21.15.47:0/939551 shutdown_connections 2024-03-21T11:22:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.025+0000 7f6fe101d740 1 -- 172.21.15.47:0/939551 wait complete. 2024-03-21T11:22:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.025+0000 7f6fd7400640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6fbc0a2f20 0x7f6fbc0a53e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8097b70 tx=0x7f6fc80d2e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.025+0000 7f6fe101d740 1 -- start start 2024-03-21T11:22:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.025+0000 7f6fe101d740 1 Processor -- start 2024-03-21T11:22:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.026+0000 7f6fe101d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30805ba0 0x55aa30810110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.026+0000 7f6fe101d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30806590 0x55aa30810650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.026+0000 7f6fe101d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 0x55aa30817690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.026+0000 7f6fe101d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3080e6b0 con 0x55aa3080b700 2024-03-21T11:22:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.026+0000 7f6fe101d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa30726dd0 con 0x55aa30805ba0 2024-03-21T11:22:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.026+0000 7f6fe101d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3080e830 con 0x55aa30806590 2024-03-21T11:22:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.026+0000 7f6fe101d740 10 client.0 _ll_register_callbacks cb 0x55aa3047eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:22:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.026+0000 7f6fd6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30806590 0x55aa30810650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.026+0000 7f6fd6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30806590 0x55aa30810650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41792/0 (socket says 172.21.15.47:41792) 2024-03-21T11:22:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.026+0000 7f6fd6a00640 1 -- 172.21.15.47:0/2009241039 learned_addr learned my addr 172.21.15.47:0/2009241039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.026+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/2009241039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30805ba0 0x55aa30810110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.026+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/2009241039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 0x55aa30817690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.027+0000 7f6fd6a00640 1 -- 172.21.15.47:0/2009241039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30805ba0 msgr2=0x55aa30810110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.027+0000 7f6fd6a00640 1 --2- 172.21.15.47:0/2009241039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30805ba0 0x55aa30810110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.027+0000 7f6fd6a00640 1 -- 172.21.15.47:0/2009241039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 msgr2=0x55aa30817690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.027+0000 7f6fd6a00640 1 --2- 172.21.15.47:0/2009241039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 0x55aa30817690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.027+0000 7f6fd6a00640 1 -- 172.21.15.47:0/2009241039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa3047f2d0 con 0x55aa30806590 2024-03-21T11:22:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.027+0000 7f6fd6a00640 1 --2- 172.21.15.47:0/2009241039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30806590 0x55aa30810650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4005ab0 tx=0x7f6fc40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97710bb29fac1ffb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.027+0000 7f6fc1600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:22:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.027+0000 7f6fc1600640 1 -- 172.21.15.47:0/2009241039 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4042070 con 0x55aa30806590 2024-03-21T11:22:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.027+0000 7f6fc1600640 1 -- 172.21.15.47:0/2009241039 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6fc4005450 con 0x55aa30806590 2024-03-21T11:22:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.027+0000 7f6fe101d740 1 -- 172.21.15.47:0/2009241039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fd0004050 con 0x55aa30806590 2024-03-21T11:22:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.028+0000 7f6fc1600640 1 -- 172.21.15.47:0/2009241039 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fc404e050 con 0x55aa30806590 2024-03-21T11:22:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.028+0000 7f6fe101d740 10 client.25183 fetch_fsmap learned FSMap version 27 2024-03-21T11:22:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.028+0000 7f6fe101d740 1 -- 172.21.15.47:0/2009241039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aa305496b0 con 0x55aa30806590 2024-03-21T11:22:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.028+0000 7f6fc1600640 1 -- 172.21.15.47:0/2009241039 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fd0004050 con 0x55aa30806590 2024-03-21T11:22:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.028+0000 7f6fc1600640 1 -- 172.21.15.47:0/2009241039 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6fc40035f0 con 0x55aa30806590 2024-03-21T11:22:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.028+0000 7f6fe101d740 10 client.25183 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:22:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.028+0000 7f6fe101d740 10 client.25183 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-21T11:22:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.028+0000 7f6fe101d740 20 client.25183 populate_metadata read hostname 'smithi047' 2024-03-21T11:22:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.028+0000 7f6fe101d740 1 --2- 172.21.15.47:0/2009241039 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aa3081c810 0x55aa3081b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.028+0000 7f6fe101d740 4 client.25183 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:22:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.028+0000 7f6fe101d740 1 -- 172.21.15.47:0/2009241039 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa3070c520 con 0x55aa3081c810 2024-03-21T11:22:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.028+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/2009241039 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aa3081c810 0x55aa3081b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.029+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/2009241039 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aa3081c810 0x55aa3081b8f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=be629b0cfddb493c server_cookie=508fadd0cf92a0 in_seq=0 out_seq=0 2024-03-21T11:22:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.029+0000 7f6fc1600640 10 client.25183 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:22:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fc1600640 1 -- 172.21.15.47:0/2009241039 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55aa3070c520 con 0x55aa3081c810 2024-03-21T11:22:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fc1600640 10 client.25183 handle_command_reply: tid=0 2024-03-21T11:22:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fe101d740 1 client.25183 shutdown 2024-03-21T11:22:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fe101d740 1 -- 172.21.15.47:0/2009241039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30806590 msgr2=0x55aa30810650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fe101d740 1 --2- 172.21.15.47:0/2009241039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30806590 0x55aa30810650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4005ab0 tx=0x7f6fc40042f0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fe101d740 1 -- 172.21.15.47:0/2009241039 shutdown_connections 2024-03-21T11:22:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fe101d740 1 --2- 172.21.15.47:0/2009241039 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aa3081c810 0x55aa3081b8f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fe101d740 1 --2- 172.21.15.47:0/2009241039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 0x55aa30817690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fe101d740 1 --2- 172.21.15.47:0/2009241039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30806590 0x55aa30810650 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fe101d740 1 --2- 172.21.15.47:0/2009241039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30805ba0 0x55aa30810110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fc1600640 0 client.25183 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:22:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fe101d740 1 -- 172.21.15.47:0/2009241039 >> 172.21.15.47:0/2009241039 conn(0x55aa30801110 msgr2=0x55aa30800310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fe101d740 1 -- 172.21.15.47:0/2009241039 shutdown_connections 2024-03-21T11:22:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fe101d740 1 -- 172.21.15.47:0/2009241039 wait complete. 2024-03-21T11:22:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.030+0000 7f6fe101d740 20 client.25183 trim_cache size 0 max 16384 2024-03-21T11:22:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.049+0000 7f6fe101d740 1 Processor -- start 2024-03-21T11:22:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.049+0000 7f6fe101d740 1 -- start start 2024-03-21T11:22:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.049+0000 7f6fe101d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa30805ba0 0x55aa3080ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.049+0000 7f6fe101d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30806590 0x55aa30817650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.049+0000 7f6fe101d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa3080b700 0x55aa307fbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.049+0000 7f6fe101d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3081c030 con 0x55aa30805ba0 2024-03-21T11:22:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.049+0000 7f6fe101d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3081c690 con 0x55aa3080b700 2024-03-21T11:22:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.049+0000 7f6fe101d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3081ccf0 con 0x55aa30806590 2024-03-21T11:22:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.050+0000 7f6fd6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30806590 0x55aa30817650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.050+0000 7f6fd6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30806590 0x55aa30817650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41808/0 (socket says 172.21.15.47:41808) 2024-03-21T11:22:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.050+0000 7f6fd6a00640 1 -- 172.21.15.47:0/2903056594 learned_addr learned my addr 172.21.15.47:0/2903056594 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.050+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/2903056594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa30805ba0 0x55aa3080ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.050+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/2903056594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa3080b700 0x55aa307fbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.050+0000 7f6fd6a00640 1 -- 172.21.15.47:0/2903056594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa3080b700 msgr2=0x55aa307fbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.050+0000 7f6fd6a00640 1 --2- 172.21.15.47:0/2903056594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa3080b700 0x55aa307fbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.050+0000 7f6fd6a00640 1 -- 172.21.15.47:0/2903056594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa30805ba0 msgr2=0x55aa3080ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.050+0000 7f6fd6a00640 1 --2- 172.21.15.47:0/2903056594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa30805ba0 0x55aa3080ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.050+0000 7f6fd6a00640 1 -- 172.21.15.47:0/2903056594 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa30806e20 con 0x55aa30806590 2024-03-21T11:22:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.050+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/2903056594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa30805ba0 0x55aa3080ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:22:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.050+0000 7f6fd6a00640 1 --2- 172.21.15.47:0/2903056594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30806590 0x55aa30817650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6fc402cfd0 tx=0x7f6fc402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3b6f8e074418784 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.050+0000 7f6fc3e00640 1 -- 172.21.15.47:0/2903056594 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc404b020 con 0x55aa30806590 2024-03-21T11:22:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.051+0000 7f6fc3e00640 1 -- 172.21.15.47:0/2903056594 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fc4009520 con 0x55aa30806590 2024-03-21T11:22:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.051+0000 7f6fc3e00640 1 -- 172.21.15.47:0/2903056594 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4042070 con 0x55aa30806590 2024-03-21T11:22:56.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.051+0000 7f6fe101d740 1 -- 172.21.15.47:0/2903056594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30806590 msgr2=0x55aa30817650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.051+0000 7f6fe101d740 1 --2- 172.21.15.47:0/2903056594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30806590 0x55aa30817650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6fc402cfd0 tx=0x7f6fc402bc30 comp rx=0 tx=0).stop 2024-03-21T11:22:56.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.051+0000 7f6fe101d740 1 -- 172.21.15.47:0/2903056594 shutdown_connections 2024-03-21T11:22:56.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.051+0000 7f6fe101d740 1 --2- 172.21.15.47:0/2903056594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa3080b700 0x55aa307fbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.051+0000 7f6fe101d740 1 --2- 172.21.15.47:0/2903056594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30806590 0x55aa30817650 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.051+0000 7f6fe101d740 1 --2- 172.21.15.47:0/2903056594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa30805ba0 0x55aa3080ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.051+0000 7f6fe101d740 1 -- 172.21.15.47:0/2903056594 >> 172.21.15.47:0/2903056594 conn(0x55aa30801110 msgr2=0x55aa30807030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.051+0000 7f6fe101d740 1 -- 172.21.15.47:0/2903056594 shutdown_connections 2024-03-21T11:22:56.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.051+0000 7f6fe101d740 1 -- 172.21.15.47:0/2903056594 wait complete. 2024-03-21T11:22:56.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.052+0000 7f6fe101d740 1 -- start start 2024-03-21T11:22:56.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.052+0000 7f6fe101d740 1 Processor -- start 2024-03-21T11:22:56.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.052+0000 7f6fe101d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30805ba0 0x55aa308138f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.052+0000 7f6fe101d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30806590 0x55aa30813e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.052+0000 7f6fe101d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 0x55aa30816e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.052+0000 7f6fe101d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3081d9b0 con 0x55aa3080b700 2024-03-21T11:22:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.052+0000 7f6fe101d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3081d350 con 0x55aa30806590 2024-03-21T11:22:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.052+0000 7f6fe101d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3081e010 con 0x55aa30805ba0 2024-03-21T11:22:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.052+0000 7f6fe101d740 10 client.0 _ll_register_callbacks cb 0x7f6fd00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:22:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.053+0000 7f6fd6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30806590 0x55aa30813e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.053+0000 7f6fd6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30806590 0x55aa30813e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52994/0 (socket says 172.21.15.47:52994) 2024-03-21T11:22:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.053+0000 7f6fd6a00640 1 -- 172.21.15.47:0/1081757410 learned_addr learned my addr 172.21.15.47:0/1081757410 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.053+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/1081757410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30805ba0 0x55aa308138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.053+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/1081757410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 0x55aa30816e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.053+0000 7f6fd6a00640 1 -- 172.21.15.47:0/1081757410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30805ba0 msgr2=0x55aa308138f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.053+0000 7f6fd6a00640 1 --2- 172.21.15.47:0/1081757410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30805ba0 0x55aa308138f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.053+0000 7f6fd6a00640 1 -- 172.21.15.47:0/1081757410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 msgr2=0x55aa30816e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.053+0000 7f6fd6a00640 1 --2- 172.21.15.47:0/1081757410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 0x55aa30816e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.053+0000 7f6fd6a00640 1 -- 172.21.15.47:0/1081757410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa30817440 con 0x55aa30806590 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.053+0000 7f6fd6a00640 1 --2- 172.21.15.47:0/1081757410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30806590 0x55aa30813e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4003a40 tx=0x7f6fc4005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae18d25806d29535 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.053+0000 7f6fc1600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.053+0000 7f6fc1600640 1 -- 172.21.15.47:0/1081757410 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc404b030 con 0x55aa30806590 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.053+0000 7f6fe101d740 1 -- 172.21.15.47:0/1081757410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fc4009520 con 0x55aa30806590 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.054+0000 7f6fc1600640 1 -- 172.21.15.47:0/1081757410 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6fc4005450 con 0x55aa30806590 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.054+0000 7f6fc1600640 1 -- 172.21.15.47:0/1081757410 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fc4042040 con 0x55aa30806590 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.054+0000 7f6fe101d740 10 client.25178 fetch_fsmap learned FSMap version 27 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.054+0000 7f6fe101d740 1 -- 172.21.15.47:0/1081757410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aa30878390 con 0x55aa30806590 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.054+0000 7f6fc1600640 1 -- 172.21.15.47:0/1081757410 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6fc4009520 con 0x55aa30806590 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.054+0000 7f6fc1600640 1 -- 172.21.15.47:0/1081757410 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6fc4047580 con 0x55aa30806590 2024-03-21T11:22:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.054+0000 7f6fe101d740 10 client.25178 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:22:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.054+0000 7f6fe101d740 10 client.25178 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-21T11:22:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.054+0000 7f6fe101d740 20 client.25178 populate_metadata read hostname 'smithi047' 2024-03-21T11:22:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.054+0000 7f6fe101d740 1 --2- 172.21.15.47:0/1081757410 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aa3087c7f0 0x55aa30878560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.054+0000 7f6fe101d740 4 client.25178 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T11:22:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.054+0000 7f6fe101d740 1 -- 172.21.15.47:0/1081757410 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aa307781d0 con 0x55aa3087c7f0 2024-03-21T11:22:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.055+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/1081757410 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aa3087c7f0 0x55aa30878560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.055+0000 7f6fd7e00640 1 --2- 172.21.15.47:0/1081757410 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aa3087c7f0 0x55aa30878560 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=aac703c2b4765a16 server_cookie=1a7191d9c27713cb in_seq=0 out_seq=0 2024-03-21T11:22:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.055+0000 7f6fc1600640 10 client.25178 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:22:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.055+0000 7f6fc1600640 1 -- 172.21.15.47:0/1081757410 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55aa307781d0 con 0x55aa3087c7f0 2024-03-21T11:22:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.055+0000 7f6fc1600640 10 client.25178 handle_command_reply: tid=0 2024-03-21T11:22:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.055+0000 7f6fe101d740 1 client.25178 shutdown 2024-03-21T11:22:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.055+0000 7f6fe101d740 1 -- 172.21.15.47:0/1081757410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30806590 msgr2=0x55aa30813e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.055+0000 7f6fe101d740 1 --2- 172.21.15.47:0/1081757410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30806590 0x55aa30813e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4003a40 tx=0x7f6fc4005e40 comp rx=0 tx=0).stop 2024-03-21T11:22:56.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.056+0000 7f6fe101d740 1 -- 172.21.15.47:0/1081757410 shutdown_connections 2024-03-21T11:22:56.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.056+0000 7f6fe101d740 1 --2- 172.21.15.47:0/1081757410 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aa3087c7f0 0x55aa30878560 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.056+0000 7f6fe101d740 1 --2- 172.21.15.47:0/1081757410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3080b700 0x55aa30816e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.056+0000 7f6fe101d740 1 --2- 172.21.15.47:0/1081757410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa30806590 0x55aa30813e30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.056+0000 7f6fc1600640 0 client.25178 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:22:56.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.056+0000 7f6fe101d740 1 --2- 172.21.15.47:0/1081757410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa30805ba0 0x55aa308138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.056+0000 7f6fe101d740 1 -- 172.21.15.47:0/1081757410 >> 172.21.15.47:0/1081757410 conn(0x55aa30801110 msgr2=0x55aa3080ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.056+0000 7f6fe101d740 1 -- 172.21.15.47:0/1081757410 shutdown_connections 2024-03-21T11:22:56.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.056+0000 7f6fe101d740 1 -- 172.21.15.47:0/1081757410 wait complete. 2024-03-21T11:22:56.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.056+0000 7f6fe101d740 20 client.25178 trim_cache size 0 max 16384 2024-03-21T11:22:56.063 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:22:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:22:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 3, 2024-03-21T11:22:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24524, 2024-03-21T11:22:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T11:22:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T11:22:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T11:22:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 99.195494409999995, 2024-03-21T11:22:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 27, 2024-03-21T11:22:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T11:22:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-03-21T11:22:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 147.95484205700001 2024-03-21T11:22:56.064 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:22:56.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.056+0000 7f6fde400640 1 -- 172.21.15.47:0/3068211719 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6fbc0a2f20 msgr2=0x7f6fbc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.056+0000 7f6fde400640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6fbc0a2f20 0x7f6fbc0a53e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8097b70 tx=0x7f6fc80d2e10 comp rx=0 tx=0).stop 2024-03-21T11:22:56.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.056+0000 7f6fde400640 1 -- 172.21.15.47:0/3068211719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fd8076060 msgr2=0x7f6fd819e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.056+0000 7f6fde400640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fd8076060 0x7f6fd819e020 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800d950 tx=0x7f6fc800de10 comp rx=0 tx=0).stop 2024-03-21T11:22:56.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.057+0000 7f6fde400640 1 -- 172.21.15.47:0/3068211719 shutdown_connections 2024-03-21T11:22:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.057+0000 7f6fde400640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6fbc0a2f20 0x7f6fbc0a53e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.057+0000 7f6fde400640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd8077c40 0x7f6fd81a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.057+0000 7f6fde400640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fd80772a0 0x7f6fd819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.057+0000 7f6fde400640 1 --2- 172.21.15.47:0/3068211719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6fd8076060 0x7f6fd819e020 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.057+0000 7f6fde400640 1 -- 172.21.15.47:0/3068211719 >> 172.21.15.47:0/3068211719 conn(0x7f6fd80fee50 msgr2=0x7f6fd8100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.057+0000 7f6fde400640 1 -- 172.21.15.47:0/3068211719 shutdown_connections 2024-03-21T11:22:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.057+0000 7f6fde400640 1 -- 172.21.15.47:0/3068211719 wait complete. 2024-03-21T11:22:56.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:55 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/847713134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:22:56.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:55 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3896736218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:22:56.076 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-21T11:22:56.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:55 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/847713134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:22:56.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:55 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3896736218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:22:56.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:55 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/847713134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:22:56.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:55 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3896736218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:22:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.283+0000 7f4e31600640 1 -- 172.21.15.47:0/3488699638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2c077c40 msgr2=0x7f4e2c0784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.283+0000 7f4e31600640 1 --2- 172.21.15.47:0/3488699638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2c077c40 0x7f4e2c0784e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009860 tx=0x7f4e2402bb70 comp rx=0 tx=0).stop 2024-03-21T11:22:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.283+0000 7f4e31600640 1 -- 172.21.15.47:0/3488699638 shutdown_connections 2024-03-21T11:22:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.283+0000 7f4e31600640 1 --2- 172.21.15.47:0/3488699638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2c077c40 0x7f4e2c0784e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.283+0000 7f4e31600640 1 --2- 172.21.15.47:0/3488699638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e2c0772a0 0x7f4e2c077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.283+0000 7f4e31600640 1 --2- 172.21.15.47:0/3488699638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e2c076060 0x7f4e2c076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.283+0000 7f4e31600640 1 -- 172.21.15.47:0/3488699638 >> 172.21.15.47:0/3488699638 conn(0x7f4e2c0fee50 msgr2=0x7f4e2c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.283+0000 7f4e31600640 1 -- 172.21.15.47:0/3488699638 shutdown_connections 2024-03-21T11:22:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.283+0000 7f4e31600640 1 -- 172.21.15.47:0/3488699638 wait complete. 2024-03-21T11:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.284+0000 7f4e31600640 1 Processor -- start 2024-03-21T11:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.284+0000 7f4e31600640 1 -- start start 2024-03-21T11:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e31600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e2c076060 0x7f4e2c19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e31600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e2c0772a0 0x7f4e2c19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e31600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2c077c40 0x7f4e2c1a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e31600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c113970 con 0x7f4e2c0772a0 2024-03-21T11:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e31600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c1137f0 con 0x7f4e2c076060 2024-03-21T11:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e31600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c113af0 con 0x7f4e2c077c40 2024-03-21T11:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e2c076060 0x7f4e2c19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e2c076060 0x7f4e2c19dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53016/0 (socket says 172.21.15.47:53016) 2024-03-21T11:22:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e2aa00640 1 -- 172.21.15.47:0/3701077549 learned_addr learned my addr 172.21.15.47:0/3701077549 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e2b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2c077c40 0x7f4e2c1a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e2a000640 1 --2- 172.21.15.47:0/3701077549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e2c0772a0 0x7f4e2c19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e2aa00640 1 -- 172.21.15.47:0/3701077549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2c077c40 msgr2=0x7f4e2c1a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/3701077549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2c077c40 0x7f4e2c1a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e2aa00640 1 -- 172.21.15.47:0/3701077549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e2c0772a0 msgr2=0x7f4e2c19e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/3701077549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e2c0772a0 0x7f4e2c19e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.285+0000 7f4e2aa00640 1 -- 172.21.15.47:0/3701077549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c1a5b70 con 0x7f4e2c076060 2024-03-21T11:22:56.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.286+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/3701077549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e2c076060 0x7f4e2c19dfa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00e9c0 tx=0x7f4e1c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.286+0000 7f4e23400640 1 -- 172.21.15.47:0/3701077549 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00ccc0 con 0x7f4e2c076060 2024-03-21T11:22:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.286+0000 7f4e23400640 1 -- 172.21.15.47:0/3701077549 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e1c005500 con 0x7f4e2c076060 2024-03-21T11:22:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.286+0000 7f4e31600640 1 -- 172.21.15.47:0/3701077549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2c1a5e60 con 0x7f4e2c076060 2024-03-21T11:22:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.286+0000 7f4e23400640 1 -- 172.21.15.47:0/3701077549 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c010430 con 0x7f4e2c076060 2024-03-21T11:22:56.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.286+0000 7f4e31600640 1 -- 172.21.15.47:0/3701077549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2c1a6320 con 0x7f4e2c076060 2024-03-21T11:22:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.289+0000 7f4e23400640 1 -- 172.21.15.47:0/3701077549 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4e1c0046c0 con 0x7f4e2c076060 2024-03-21T11:22:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.289+0000 7f4e343ec740 1 Processor -- start 2024-03-21T11:22:56.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.290+0000 7f4e343ec740 1 -- start start 2024-03-21T11:22:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.290+0000 7f4e343ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6eaba0 0x55a42d6eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.290+0000 7f4e343ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eb590 0x55a42d6f0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.290+0000 7f4e343ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6f0700 0x55a42d6f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.290+0000 7f4e343ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42d64fc90 con 0x55a42d6eb590 2024-03-21T11:22:56.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.290+0000 7f4e343ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42d5f8980 con 0x55a42d6eaba0 2024-03-21T11:22:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.290+0000 7f4e343ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42d64bd30 con 0x55a42d6f0700 2024-03-21T11:22:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.290+0000 7f4e2b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6eaba0 0x55a42d6eafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.290+0000 7f4e2b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6eaba0 0x55a42d6eafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53028/0 (socket says 172.21.15.47:53028) 2024-03-21T11:22:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.290+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eb590 0x55a42d6f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.290+0000 7f4e2b400640 1 -- 172.21.15.47:0/996013722 learned_addr learned my addr 172.21.15.47:0/996013722 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.290+0000 7f4e2a000640 1 --2- 172.21.15.47:0/996013722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6f0700 0x55a42d6f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.291+0000 7f4e2b400640 1 -- 172.21.15.47:0/996013722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6f0700 msgr2=0x55a42d6f2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.291+0000 7f4e2b400640 1 --2- 172.21.15.47:0/996013722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6f0700 0x55a42d6f2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.291+0000 7f4e2b400640 1 -- 172.21.15.47:0/996013722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eb590 msgr2=0x55a42d6f0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.291+0000 7f4e2b400640 1 --2- 172.21.15.47:0/996013722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eb590 0x55a42d6f0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.291+0000 7f4e2b400640 1 -- 172.21.15.47:0/996013722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a42d610510 con 0x55a42d6eaba0 2024-03-21T11:22:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.291+0000 7f4e2a000640 1 --2- 172.21.15.47:0/996013722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6f0700 0x55a42d6f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.291+0000 7f4e2b400640 1 --2- 172.21.15.47:0/996013722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6eaba0 0x55a42d6eafc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4e24034b10 tx=0x7f4e24005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a59b314e4cf9ca server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.291+0000 7f4e23400640 1 --2- 172.21.15.47:0/3701077549 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e0c0a2fa0 0x7f4e0c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.291+0000 7f4e1b400640 1 -- 172.21.15.47:0/996013722 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e24043020 con 0x55a42d6eaba0 2024-03-21T11:22:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.291+0000 7f4e1b400640 1 -- 172.21.15.47:0/996013722 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e24034db0 con 0x55a42d6eaba0 2024-03-21T11:22:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.291+0000 7f4e1b400640 1 -- 172.21.15.47:0/996013722 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e24039040 con 0x55a42d6eaba0 2024-03-21T11:22:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.292+0000 7f4e23400640 1 -- 172.21.15.47:0/3701077549 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4e1c0cfa30 con 0x7f4e2c076060 2024-03-21T11:22:56.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.292+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/3701077549 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e0c0a2fa0 0x7f4e0c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.292+0000 7f4e343ec740 1 -- 172.21.15.47:0/996013722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6eaba0 msgr2=0x55a42d6eafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.292+0000 7f4e343ec740 1 --2- 172.21.15.47:0/996013722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6eaba0 0x55a42d6eafc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4e24034b10 tx=0x7f4e24005b20 comp rx=0 tx=0).stop 2024-03-21T11:22:56.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.292+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/3701077549 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e0c0a2fa0 0x7f4e0c0a5460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0cd140 tx=0x7f4e1c0d54d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.292+0000 7f4e343ec740 1 -- 172.21.15.47:0/996013722 shutdown_connections 2024-03-21T11:22:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.292+0000 7f4e343ec740 1 --2- 172.21.15.47:0/996013722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6f0700 0x55a42d6f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.292+0000 7f4e343ec740 1 --2- 172.21.15.47:0/996013722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eb590 0x55a42d6f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.292+0000 7f4e343ec740 1 --2- 172.21.15.47:0/996013722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6eaba0 0x55a42d6eafc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.292+0000 7f4e343ec740 1 -- 172.21.15.47:0/996013722 >> 172.21.15.47:0/996013722 conn(0x55a42d6e6110 msgr2=0x55a42d6e6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.292+0000 7f4e343ec740 1 -- 172.21.15.47:0/996013722 shutdown_connections 2024-03-21T11:22:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.293+0000 7f4e343ec740 1 -- 172.21.15.47:0/996013722 wait complete. 2024-03-21T11:22:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.293+0000 7f4e343ec740 1 -- start start 2024-03-21T11:22:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.293+0000 7f4e343ec740 1 Processor -- start 2024-03-21T11:22:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e343ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eaba0 0x55a42d6f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e343ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eb590 0x55a42d6f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e343ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6f0700 0x55a42d6fc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e343ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42d6f3650 con 0x55a42d6eaba0 2024-03-21T11:22:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e343ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42d60bdd0 con 0x55a42d6f0700 2024-03-21T11:22:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e343ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42d6f37d0 con 0x55a42d6eb590 2024-03-21T11:22:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e343ec740 10 client.0 _ll_register_callbacks cb 0x55a42d363ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:22:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e2a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eb590 0x55a42d6f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e2a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eb590 0x55a42d6f55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41844/0 (socket says 172.21.15.47:41844) 2024-03-21T11:22:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e2a000640 1 -- 172.21.15.47:0/1597748760 learned_addr learned my addr 172.21.15.47:0/1597748760 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e2b400640 1 --2- 172.21.15.47:0/1597748760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eaba0 0x55a42d6f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e2b400640 1 --2- 172.21.15.47:0/1597748760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6f0700 0x55a42d6fc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e2a000640 1 -- 172.21.15.47:0/1597748760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6f0700 msgr2=0x55a42d6fc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e2a000640 1 --2- 172.21.15.47:0/1597748760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6f0700 0x55a42d6fc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e2a000640 1 -- 172.21.15.47:0/1597748760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eaba0 msgr2=0x55a42d6f50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e2a000640 1 --2- 172.21.15.47:0/1597748760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eaba0 0x55a42d6f50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.294+0000 7f4e2a000640 1 -- 172.21.15.47:0/1597748760 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a42d3642d0 con 0x55a42d6eb590 2024-03-21T11:22:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.295+0000 7f4e2a000640 1 --2- 172.21.15.47:0/1597748760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eb590 0x55a42d6f55f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e14011d10 tx=0x7f4e1400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58768f4e3d9bf988 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.295+0000 7f4e13e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:22:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.295+0000 7f4e13e00640 1 -- 172.21.15.47:0/1597748760 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1401f060 con 0x55a42d6eb590 2024-03-21T11:22:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.295+0000 7f4e343ec740 1 -- 172.21.15.47:0/1597748760 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e24034db0 con 0x55a42d6eb590 2024-03-21T11:22:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.295+0000 7f4e13e00640 1 -- 172.21.15.47:0/1597748760 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4e14003970 con 0x55a42d6eb590 2024-03-21T11:22:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.295+0000 7f4e13e00640 1 -- 172.21.15.47:0/1597748760 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e14026050 con 0x55a42d6eb590 2024-03-21T11:22:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.295+0000 7f4e343ec740 10 client.25201 fetch_fsmap learned FSMap version 27 2024-03-21T11:22:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.295+0000 7f4e343ec740 1 -- 172.21.15.47:0/1597748760 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a42d42e6b0 con 0x55a42d6eb590 2024-03-21T11:22:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.295+0000 7f4e13e00640 1 -- 172.21.15.47:0/1597748760 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e24034db0 con 0x55a42d6eb590 2024-03-21T11:22:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.295+0000 7f4e13e00640 1 -- 172.21.15.47:0/1597748760 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4e14019030 con 0x55a42d6eb590 2024-03-21T11:22:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.296+0000 7f4e343ec740 10 client.25201 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:22:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.296+0000 7f4e343ec740 10 client.25201 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-21T11:22:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.296+0000 7f4e343ec740 20 client.25201 populate_metadata read hostname 'smithi047' 2024-03-21T11:22:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.296+0000 7f4e343ec740 1 --2- 172.21.15.47:0/1597748760 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55a42d701950 0x55a42d7009d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.296+0000 7f4e343ec740 4 client.25201 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:22:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.296+0000 7f4e343ec740 1 -- 172.21.15.47:0/1597748760 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a42d5f1520 con 0x55a42d701950 2024-03-21T11:22:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.296+0000 7f4e2b400640 1 --2- 172.21.15.47:0/1597748760 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55a42d701950 0x55a42d7009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.296+0000 7f4e2b400640 1 --2- 172.21.15.47:0/1597748760 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55a42d701950 0x55a42d7009d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ad00bbf9628bdc9 server_cookie=631d049d7fc19bf0 in_seq=0 out_seq=0 2024-03-21T11:22:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.296+0000 7f4e13e00640 10 client.25201 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:22:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e13e00640 1 -- 172.21.15.47:0/1597748760 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a42d5f1520 con 0x55a42d701950 2024-03-21T11:22:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e13e00640 10 client.25201 handle_command_reply: tid=0 2024-03-21T11:22:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e343ec740 1 client.25201 shutdown 2024-03-21T11:22:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e343ec740 1 -- 172.21.15.47:0/1597748760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eb590 msgr2=0x55a42d6f55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e343ec740 1 --2- 172.21.15.47:0/1597748760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eb590 0x55a42d6f55f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e14011d10 tx=0x7f4e1400f790 comp rx=0 tx=0).stop 2024-03-21T11:22:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e343ec740 1 -- 172.21.15.47:0/1597748760 shutdown_connections 2024-03-21T11:22:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e343ec740 1 --2- 172.21.15.47:0/1597748760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6f0700 0x55a42d6fc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e343ec740 1 --2- 172.21.15.47:0/1597748760 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55a42d701950 0x55a42d7009d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e343ec740 1 --2- 172.21.15.47:0/1597748760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eb590 0x55a42d6f55f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e343ec740 1 --2- 172.21.15.47:0/1597748760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eaba0 0x55a42d6f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e343ec740 1 -- 172.21.15.47:0/1597748760 >> 172.21.15.47:0/1597748760 conn(0x55a42d6e6110 msgr2=0x55a42d6e5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e13e00640 0 client.25201 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:22:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e343ec740 1 -- 172.21.15.47:0/1597748760 shutdown_connections 2024-03-21T11:22:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e343ec740 1 -- 172.21.15.47:0/1597748760 wait complete. 2024-03-21T11:22:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.298+0000 7f4e343ec740 20 client.25201 trim_cache size 0 max 16384 2024-03-21T11:22:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.317+0000 7f4e343ec740 1 Processor -- start 2024-03-21T11:22:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.317+0000 7f4e343ec740 1 -- start start 2024-03-21T11:22:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.317+0000 7f4e343ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eaba0 0x55a42d6f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.317+0000 7f4e343ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6eb590 0x55a42d6fc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.317+0000 7f4e343ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6f0700 0x55a42d7009d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.317+0000 7f4e343ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42d701170 con 0x55a42d6f0700 2024-03-21T11:22:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.317+0000 7f4e343ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42d7017d0 con 0x55a42d6eb590 2024-03-21T11:22:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.317+0000 7f4e343ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42d701e30 con 0x55a42d6eaba0 2024-03-21T11:22:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.317+0000 7f4e2a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6eb590 0x55a42d6fc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.317+0000 7f4e2b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eaba0 0x55a42d6f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.317+0000 7f4e2a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6eb590 0x55a42d6fc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53052/0 (socket says 172.21.15.47:53052) 2024-03-21T11:22:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.317+0000 7f4e2a000640 1 -- 172.21.15.47:0/161714489 learned_addr learned my addr 172.21.15.47:0/161714489 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.318+0000 7f4e2b400640 1 --2- 172.21.15.47:0/161714489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6f0700 0x55a42d7009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.318+0000 7f4e2a000640 1 -- 172.21.15.47:0/161714489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eaba0 msgr2=0x55a42d6f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.318+0000 7f4e2a000640 1 --2- 172.21.15.47:0/161714489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eaba0 0x55a42d6f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.318+0000 7f4e2a000640 1 -- 172.21.15.47:0/161714489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6f0700 msgr2=0x55a42d7009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.318+0000 7f4e2a000640 1 --2- 172.21.15.47:0/161714489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6f0700 0x55a42d7009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.318+0000 7f4e2a000640 1 -- 172.21.15.47:0/161714489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a42d6ebe20 con 0x55a42d6eb590 2024-03-21T11:22:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.318+0000 7f4e2b400640 1 --2- 172.21.15.47:0/161714489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6f0700 0x55a42d7009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:22:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.318+0000 7f4e2a000640 1 --2- 172.21.15.47:0/161714489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6eb590 0x55a42d6fc5f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009a00 tx=0x7f4e14011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfec2ff9f6d63208 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.318+0000 7f4e1aa00640 1 -- 172.21.15.47:0/161714489 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1400fe80 con 0x55a42d6eb590 2024-03-21T11:22:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.318+0000 7f4e1aa00640 1 -- 172.21.15.47:0/161714489 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e1400e5f0 con 0x55a42d6eb590 2024-03-21T11:22:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.318+0000 7f4e1aa00640 1 -- 172.21.15.47:0/161714489 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1402abf0 con 0x55a42d6eb590 2024-03-21T11:22:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.319+0000 7f4e343ec740 1 -- 172.21.15.47:0/161714489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6eb590 msgr2=0x55a42d6fc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.319+0000 7f4e343ec740 1 --2- 172.21.15.47:0/161714489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6eb590 0x55a42d6fc5f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009a00 tx=0x7f4e14011900 comp rx=0 tx=0).stop 2024-03-21T11:22:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.319+0000 7f4e343ec740 1 -- 172.21.15.47:0/161714489 shutdown_connections 2024-03-21T11:22:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.319+0000 7f4e343ec740 1 --2- 172.21.15.47:0/161714489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6f0700 0x55a42d7009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.319+0000 7f4e343ec740 1 --2- 172.21.15.47:0/161714489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6eb590 0x55a42d6fc5f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.319+0000 7f4e343ec740 1 --2- 172.21.15.47:0/161714489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eaba0 0x55a42d6f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.319+0000 7f4e343ec740 1 -- 172.21.15.47:0/161714489 >> 172.21.15.47:0/161714489 conn(0x55a42d6e6110 msgr2=0x55a42d6f3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.319+0000 7f4e343ec740 1 -- 172.21.15.47:0/161714489 shutdown_connections 2024-03-21T11:22:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.319+0000 7f4e343ec740 1 -- 172.21.15.47:0/161714489 wait complete. 2024-03-21T11:22:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.319+0000 7f4e343ec740 1 -- start start 2024-03-21T11:22:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.320+0000 7f4e343ec740 1 Processor -- start 2024-03-21T11:22:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.320+0000 7f4e343ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eaba0 0x55a42d6f8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.320+0000 7f4e343ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eb590 0x55a42d6f8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.320+0000 7f4e343ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6f0700 0x55a42d6fbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.320+0000 7f4e343ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42d702af0 con 0x55a42d6eaba0 2024-03-21T11:22:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.320+0000 7f4e343ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42d702490 con 0x55a42d6f0700 2024-03-21T11:22:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.320+0000 7f4e343ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42d703150 con 0x55a42d6eb590 2024-03-21T11:22:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.320+0000 7f4e343ec740 10 client.0 _ll_register_callbacks cb 0x7f4e14009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:22:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.320+0000 7f4e2a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eb590 0x55a42d6f8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.320+0000 7f4e2a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eb590 0x55a42d6f8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41854/0 (socket says 172.21.15.47:41854) 2024-03-21T11:22:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.320+0000 7f4e2a000640 1 -- 172.21.15.47:0/2165515930 learned_addr learned my addr 172.21.15.47:0/2165515930 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.320+0000 7f4e2b400640 1 --2- 172.21.15.47:0/2165515930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eaba0 0x55a42d6f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.321+0000 7f4e2b400640 1 --2- 172.21.15.47:0/2165515930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6f0700 0x55a42d6fbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.321+0000 7f4e2a000640 1 -- 172.21.15.47:0/2165515930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6f0700 msgr2=0x55a42d6fbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.321+0000 7f4e2a000640 1 --2- 172.21.15.47:0/2165515930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6f0700 0x55a42d6fbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.321+0000 7f4e2a000640 1 -- 172.21.15.47:0/2165515930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eaba0 msgr2=0x55a42d6f8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.321+0000 7f4e2a000640 1 --2- 172.21.15.47:0/2165515930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eaba0 0x55a42d6f8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.321+0000 7f4e2a000640 1 -- 172.21.15.47:0/2165515930 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a42d6fc0b0 con 0x55a42d6eb590 2024-03-21T11:22:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.321+0000 7f4e2b400640 1 --2- 172.21.15.47:0/2165515930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eaba0 0x55a42d6f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:22:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.321+0000 7f4e2a000640 1 --2- 172.21.15.47:0/2165515930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eb590 0x55a42d6f8b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400e480 tx=0x7f4e1402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e5722d3fc04ca1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.321+0000 7f4e13e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:22:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.321+0000 7f4e13e00640 1 -- 172.21.15.47:0/2165515930 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14037070 con 0x55a42d6eb590 2024-03-21T11:22:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.321+0000 7f4e343ec740 1 -- 172.21.15.47:0/2165515930 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e1400e5f0 con 0x55a42d6eb590 2024-03-21T11:22:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.321+0000 7f4e13e00640 1 -- 172.21.15.47:0/2165515930 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4e14004b80 con 0x55a42d6eb590 2024-03-21T11:22:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.321+0000 7f4e13e00640 1 -- 172.21.15.47:0/2165515930 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e1401f020 con 0x55a42d6eb590 2024-03-21T11:22:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.321+0000 7f4e343ec740 10 client.25207 fetch_fsmap learned FSMap version 27 2024-03-21T11:22:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.322+0000 7f4e343ec740 1 -- 172.21.15.47:0/2165515930 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a42d75d1f0 con 0x55a42d6eb590 2024-03-21T11:22:56.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.322+0000 7f4e13e00640 1 -- 172.21.15.47:0/2165515930 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e1400e5f0 con 0x55a42d6eb590 2024-03-21T11:22:56.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.322+0000 7f4e13e00640 1 -- 172.21.15.47:0/2165515930 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4e14019050 con 0x55a42d6eb590 2024-03-21T11:22:56.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.322+0000 7f4e343ec740 10 client.25207 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:22:56.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.322+0000 7f4e343ec740 10 client.25207 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-21T11:22:56.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.322+0000 7f4e343ec740 20 client.25207 populate_metadata read hostname 'smithi047' 2024-03-21T11:22:56.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.322+0000 7f4e343ec740 1 --2- 172.21.15.47:0/2165515930 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55a42d761450 0x55a42d75d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.322+0000 7f4e343ec740 4 client.25207 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T11:22:56.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.322+0000 7f4e343ec740 1 -- 172.21.15.47:0/2165515930 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a42d65d1d0 con 0x55a42d761450 2024-03-21T11:22:56.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.322+0000 7f4e2b400640 1 --2- 172.21.15.47:0/2165515930 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55a42d761450 0x55a42d75d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.323+0000 7f4e2b400640 1 --2- 172.21.15.47:0/2165515930 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55a42d761450 0x55a42d75d480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15842feddcbcfc85 server_cookie=1bf8686b333d8feb in_seq=0 out_seq=0 2024-03-21T11:22:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.323+0000 7f4e13e00640 10 client.25207 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:22:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.323+0000 7f4e13e00640 1 -- 172.21.15.47:0/2165515930 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a42d65d1d0 con 0x55a42d761450 2024-03-21T11:22:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.323+0000 7f4e13e00640 10 client.25207 handle_command_reply: tid=0 2024-03-21T11:22:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.323+0000 7f4e343ec740 1 client.25207 shutdown 2024-03-21T11:22:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.323+0000 7f4e343ec740 1 -- 172.21.15.47:0/2165515930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eb590 msgr2=0x55a42d6f8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.323+0000 7f4e343ec740 1 --2- 172.21.15.47:0/2165515930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eb590 0x55a42d6f8b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400e480 tx=0x7f4e1402a660 comp rx=0 tx=0).stop 2024-03-21T11:22:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.323+0000 7f4e343ec740 1 -- 172.21.15.47:0/2165515930 shutdown_connections 2024-03-21T11:22:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.323+0000 7f4e343ec740 1 --2- 172.21.15.47:0/2165515930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a42d6f0700 0x55a42d6fbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.323+0000 7f4e343ec740 1 --2- 172.21.15.47:0/2165515930 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55a42d761450 0x55a42d75d480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.323+0000 7f4e343ec740 1 --2- 172.21.15.47:0/2165515930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42d6eb590 0x55a42d6f8b40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.323+0000 7f4e343ec740 1 --2- 172.21.15.47:0/2165515930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a42d6eaba0 0x55a42d6f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.323+0000 7f4e13e00640 0 client.25207 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:22:56.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.323+0000 7f4e343ec740 1 -- 172.21.15.47:0/2165515930 >> 172.21.15.47:0/2165515930 conn(0x55a42d6e6110 msgr2=0x55a42d7009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.324+0000 7f4e343ec740 1 -- 172.21.15.47:0/2165515930 shutdown_connections 2024-03-21T11:22:56.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.324+0000 7f4e343ec740 1 -- 172.21.15.47:0/2165515930 wait complete. 2024-03-21T11:22:56.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.324+0000 7f4e343ec740 20 client.25207 trim_cache size 0 max 16384 2024-03-21T11:22:56.331 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:22:56.331 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:22:56.331 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 2, 2024-03-21T11:22:56.331 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24535, 2024-03-21T11:22:56.331 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T11:22:56.331 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T11:22:56.332 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T11:22:56.332 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 100.484297187, 2024-03-21T11:22:56.332 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 27, 2024-03-21T11:22:56.332 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T11:22:56.332 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-03-21T11:22:56.332 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 151.08849780400001 2024-03-21T11:22:56.332 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:22:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.324+0000 7f4e31600640 1 -- 172.21.15.47:0/3701077549 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e0c0a2fa0 msgr2=0x7f4e0c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.324+0000 7f4e31600640 1 --2- 172.21.15.47:0/3701077549 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e0c0a2fa0 0x7f4e0c0a5460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0cd140 tx=0x7f4e1c0d54d0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.324+0000 7f4e31600640 1 -- 172.21.15.47:0/3701077549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e2c076060 msgr2=0x7f4e2c19dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.324+0000 7f4e31600640 1 --2- 172.21.15.47:0/3701077549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e2c076060 0x7f4e2c19dfa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00e9c0 tx=0x7f4e1c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:22:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.324+0000 7f4e31600640 1 -- 172.21.15.47:0/3701077549 shutdown_connections 2024-03-21T11:22:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.324+0000 7f4e31600640 1 --2- 172.21.15.47:0/3701077549 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e0c0a2fa0 0x7f4e0c0a5460 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.324+0000 7f4e31600640 1 --2- 172.21.15.47:0/3701077549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e2c077c40 0x7f4e2c1a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.324+0000 7f4e31600640 1 --2- 172.21.15.47:0/3701077549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e2c0772a0 0x7f4e2c19e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.324+0000 7f4e31600640 1 --2- 172.21.15.47:0/3701077549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e2c076060 0x7f4e2c19dfa0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.324+0000 7f4e31600640 1 -- 172.21.15.47:0/3701077549 >> 172.21.15.47:0/3701077549 conn(0x7f4e2c0fee50 msgr2=0x7f4e2c100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.325+0000 7f4e31600640 1 -- 172.21.15.47:0/3701077549 shutdown_connections 2024-03-21T11:22:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.325+0000 7f4e31600640 1 -- 172.21.15.47:0/3701077549 wait complete. 2024-03-21T11:22:56.344 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-21T11:22:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.565+0000 7f8fe3200640 1 -- 172.21.15.47:0/1786439551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fdc108db0 msgr2=0x7f8fdc109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.565+0000 7f8fe3200640 1 --2- 172.21.15.47:0/1786439551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fdc108db0 0x7f8fdc109230 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009900 tx=0x7f8fc802bc30 comp rx=0 tx=0).stop 2024-03-21T11:22:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.565+0000 7f8fe3200640 1 -- 172.21.15.47:0/1786439551 shutdown_connections 2024-03-21T11:22:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.565+0000 7f8fe3200640 1 --2- 172.21.15.47:0/1786439551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fdc109770 0x7f8fdc110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.565+0000 7f8fe3200640 1 --2- 172.21.15.47:0/1786439551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fdc108db0 0x7f8fdc109230 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.565+0000 7f8fe3200640 1 --2- 172.21.15.47:0/1786439551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc107bc0 0x7f8fdc107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.565+0000 7f8fe3200640 1 -- 172.21.15.47:0/1786439551 >> 172.21.15.47:0/1786439551 conn(0x7f8fdc076b70 msgr2=0x7f8fdc078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.565+0000 7f8fe3200640 1 -- 172.21.15.47:0/1786439551 shutdown_connections 2024-03-21T11:22:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.565+0000 7f8fe3200640 1 -- 172.21.15.47:0/1786439551 wait complete. 2024-03-21T11:22:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.566+0000 7f8fe3200640 1 Processor -- start 2024-03-21T11:22:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.566+0000 7f8fe3200640 1 -- start start 2024-03-21T11:22:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.567+0000 7f8fe3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fdc107bc0 0x7f8fdc19de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.567+0000 7f8fe3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fdc108db0 0x7f8fdc19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.567+0000 7f8fe3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc109770 0x7f8fdc1a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.567+0000 7f8fe3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc113870 con 0x7f8fdc107bc0 2024-03-21T11:22:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.567+0000 7f8fe3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1136f0 con 0x7f8fdc108db0 2024-03-21T11:22:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.567+0000 7f8fe3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1139f0 con 0x7f8fdc109770 2024-03-21T11:22:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.567+0000 7f8fe1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fdc107bc0 0x7f8fdc19de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.567+0000 7f8fe1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fdc107bc0 0x7f8fdc19de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52750/0 (socket says 172.21.15.47:52750) 2024-03-21T11:22:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.567+0000 7f8fe1e00640 1 -- 172.21.15.47:0/4159312869 learned_addr learned my addr 172.21.15.47:0/4159312869 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.567+0000 7f8fe1400640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fdc108db0 0x7f8fdc19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.567+0000 7f8fe2800640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc109770 0x7f8fdc1a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.568+0000 7f8fe1e00640 1 -- 172.21.15.47:0/4159312869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc109770 msgr2=0x7f8fdc1a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.568+0000 7f8fe1e00640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc109770 0x7f8fdc1a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.568+0000 7f8fe1e00640 1 -- 172.21.15.47:0/4159312869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fdc108db0 msgr2=0x7f8fdc19e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.568+0000 7f8fe1e00640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fdc108db0 0x7f8fdc19e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.568+0000 7f8fe1e00640 1 -- 172.21.15.47:0/4159312869 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc1a5ab0 con 0x7f8fdc107bc0 2024-03-21T11:22:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.568+0000 7f8fe1400640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fdc108db0 0x7f8fdc19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.568+0000 7f8fe1e00640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fdc107bc0 0x7f8fdc19de90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000e9c0 tx=0x7f8fd000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.568+0000 7f8fd6a00640 1 -- 172.21.15.47:0/4159312869 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd000cd80 con 0x7f8fdc107bc0 2024-03-21T11:22:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.568+0000 7f8fd6a00640 1 -- 172.21.15.47:0/4159312869 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fd00035f0 con 0x7f8fdc107bc0 2024-03-21T11:22:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.568+0000 7f8fe3200640 1 -- 172.21.15.47:0/4159312869 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fdc1a5da0 con 0x7f8fdc107bc0 2024-03-21T11:22:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.569+0000 7f8fd6a00640 1 -- 172.21.15.47:0/4159312869 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0010690 con 0x7f8fdc107bc0 2024-03-21T11:22:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.569+0000 7f8fe3200640 1 -- 172.21.15.47:0/4159312869 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc1a6200 con 0x7f8fdc107bc0 2024-03-21T11:22:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.572+0000 7f8fe760a740 1 Processor -- start 2024-03-21T11:22:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.572+0000 7f8fd6a00640 1 -- 172.21.15.47:0/4159312869 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8fd00037a0 con 0x7f8fdc107bc0 2024-03-21T11:22:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.572+0000 7f8fe760a740 1 -- start start 2024-03-21T11:22:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.572+0000 7f8fe760a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9dd80 0x55b627c9e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.572+0000 7f8fe760a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 0x55b627ca3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.572+0000 7f8fe760a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627ca38e0 0x55b627ca5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.572+0000 7f8fe760a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b627c02c90 con 0x55b627c9dd80 2024-03-21T11:22:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.572+0000 7f8fe760a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b627bab980 con 0x55b627c9e770 2024-03-21T11:22:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.572+0000 7f8fe760a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b627bfed30 con 0x55b627ca38e0 2024-03-21T11:22:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.572+0000 7f8fe1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 0x55b627ca3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.572+0000 7f8fe1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627ca38e0 0x55b627ca5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.573+0000 7f8fe1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627ca38e0 0x55b627ca5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41884/0 (socket says 172.21.15.47:41884) 2024-03-21T11:22:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.573+0000 7f8fe1400640 1 -- 172.21.15.47:0/2261478419 learned_addr learned my addr 172.21.15.47:0/2261478419 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.573+0000 7f8fe2800640 1 --2- 172.21.15.47:0/2261478419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9dd80 0x55b627c9e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.573+0000 7f8fe1400640 1 -- 172.21.15.47:0/2261478419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 msgr2=0x55b627ca3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.573+0000 7f8fe1400640 1 --2- 172.21.15.47:0/2261478419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 0x55b627ca3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.573+0000 7f8fe1400640 1 -- 172.21.15.47:0/2261478419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9dd80 msgr2=0x55b627c9e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.573+0000 7f8fe1400640 1 --2- 172.21.15.47:0/2261478419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9dd80 0x55b627c9e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.573+0000 7f8fe1400640 1 -- 172.21.15.47:0/2261478419 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b627bc3510 con 0x55b627ca38e0 2024-03-21T11:22:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.573+0000 7f8fe1400640 1 --2- 172.21.15.47:0/2261478419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627ca38e0 0x55b627ca5d70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8005ab0 tx=0x7f8fc80095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e71746ffb62b1e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.573+0000 7f8fcf000640 1 -- 172.21.15.47:0/2261478419 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8045070 con 0x55b627ca38e0 2024-03-21T11:22:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.574+0000 7f8fcf000640 1 -- 172.21.15.47:0/2261478419 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fc802cd30 con 0x55b627ca38e0 2024-03-21T11:22:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.574+0000 7f8fcf000640 1 -- 172.21.15.47:0/2261478419 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8005110 con 0x55b627ca38e0 2024-03-21T11:22:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.574+0000 7f8fd6a00640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8fc40a2f20 0x7f8fc40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.574+0000 7f8fd6a00640 1 -- 172.21.15.47:0/4159312869 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8fd00cfb50 con 0x7f8fdc107bc0 2024-03-21T11:22:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.574+0000 7f8fe2800640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8fc40a2f20 0x7f8fc40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.574+0000 7f8fe760a740 1 -- 172.21.15.47:0/2261478419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627ca38e0 msgr2=0x55b627ca5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.574+0000 7f8fe760a740 1 --2- 172.21.15.47:0/2261478419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627ca38e0 0x55b627ca5d70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8005ab0 tx=0x7f8fc80095e0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.574+0000 7f8fe760a740 1 -- 172.21.15.47:0/2261478419 shutdown_connections 2024-03-21T11:22:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.574+0000 7f8fe760a740 1 --2- 172.21.15.47:0/2261478419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627ca38e0 0x55b627ca5d70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.574+0000 7f8fe760a740 1 --2- 172.21.15.47:0/2261478419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 0x55b627ca3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.574+0000 7f8fe760a740 1 --2- 172.21.15.47:0/2261478419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9dd80 0x55b627c9e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.574+0000 7f8fe760a740 1 -- 172.21.15.47:0/2261478419 >> 172.21.15.47:0/2261478419 conn(0x55b627c992f0 msgr2=0x55b627c996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.575+0000 7f8fe760a740 1 -- 172.21.15.47:0/2261478419 shutdown_connections 2024-03-21T11:22:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.575+0000 7f8fe760a740 1 -- 172.21.15.47:0/2261478419 wait complete. 2024-03-21T11:22:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.575+0000 7f8fe2800640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8fc40a2f20 0x7f8fc40a53e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55b627c9efd0 tx=0x7f8fd80096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.575+0000 7f8fe760a740 1 -- start start 2024-03-21T11:22:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.575+0000 7f8fe760a740 1 Processor -- start 2024-03-21T11:22:56.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.576+0000 7f8fe760a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627c9dd80 0x55b627ca81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.576+0000 7f8fe760a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 0x55b627ca8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.576+0000 7f8fe760a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627ca38e0 0x55b627caf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.576+0000 7f8fe760a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b627ca6830 con 0x55b627ca38e0 2024-03-21T11:22:56.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.576+0000 7f8fe760a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b627bbedd0 con 0x55b627c9e770 2024-03-21T11:22:56.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.576+0000 7f8fe760a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b627ca69b0 con 0x55b627c9dd80 2024-03-21T11:22:56.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.576+0000 7f8fe1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627c9dd80 0x55b627ca81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.576+0000 7f8fe1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627c9dd80 0x55b627ca81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41890/0 (socket says 172.21.15.47:41890) 2024-03-21T11:22:56.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.576+0000 7f8fe1e00640 1 -- 172.21.15.47:0/4111139927 learned_addr learned my addr 172.21.15.47:0/4111139927 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.576+0000 7f8fe760a740 10 client.0 _ll_register_callbacks cb 0x55b627916ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:22:56.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.576+0000 7f8fe2800640 1 --2- 172.21.15.47:0/4111139927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627ca38e0 0x55b627caf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.576+0000 7f8fe1400640 1 --2- 172.21.15.47:0/4111139927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 0x55b627ca8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.577+0000 7f8fe1e00640 1 -- 172.21.15.47:0/4111139927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 msgr2=0x55b627ca8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.577+0000 7f8fe1e00640 1 --2- 172.21.15.47:0/4111139927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 0x55b627ca8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.577+0000 7f8fe1e00640 1 -- 172.21.15.47:0/4111139927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627ca38e0 msgr2=0x55b627caf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.577+0000 7f8fe1e00640 1 --2- 172.21.15.47:0/4111139927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627ca38e0 0x55b627caf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.577+0000 7f8fe1e00640 1 -- 172.21.15.47:0/4111139927 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6279172d0 con 0x55b627c9dd80 2024-03-21T11:22:56.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.577+0000 7f8fe1e00640 1 --2- 172.21.15.47:0/4111139927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627c9dd80 0x55b627ca81f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00058c0 tx=0x7f8fd0005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca8564a04d105679 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.577+0000 7f8fc3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.577+0000 7f8fc3400640 1 -- 172.21.15.47:0/4111139927 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd009ba30 con 0x55b627c9dd80 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.577+0000 7f8fc3400640 1 -- 172.21.15.47:0/4111139927 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8fd009b340 con 0x55b627c9dd80 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.577+0000 7f8fe760a740 1 -- 172.21.15.47:0/4111139927 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fc802cd30 con 0x55b627c9dd80 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.577+0000 7f8fc3400640 1 -- 172.21.15.47:0/4111139927 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fd009b850 con 0x55b627c9dd80 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.577+0000 7f8fe760a740 10 client.25219 fetch_fsmap learned FSMap version 27 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.577+0000 7f8fe760a740 1 -- 172.21.15.47:0/4111139927 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b6279e16b0 con 0x55b627c9dd80 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.578+0000 7f8fc3400640 1 -- 172.21.15.47:0/4111139927 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fc802cd30 con 0x55b627c9dd80 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.578+0000 7f8fc3400640 1 -- 172.21.15.47:0/4111139927 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8fd00b8080 con 0x55b627c9dd80 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.578+0000 7f8fe760a740 10 client.25219 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.578+0000 7f8fe760a740 10 client.25219 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.578+0000 7f8fe760a740 20 client.25219 populate_metadata read hostname 'smithi047' 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.578+0000 7f8fe760a740 1 --2- 172.21.15.47:0/4111139927 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b627cb4990 0x55b627cb3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.578+0000 7f8fe760a740 4 client.25219 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.578+0000 7f8fe760a740 1 -- 172.21.15.47:0/4111139927 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b627ba4520 con 0x55b627cb4990 2024-03-21T11:22:56.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.578+0000 7f8fe1400640 1 --2- 172.21.15.47:0/4111139927 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b627cb4990 0x55b627cb3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.579+0000 7f8fe1400640 1 --2- 172.21.15.47:0/4111139927 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b627cb4990 0x55b627cb3a60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e0fe9ee155073ff server_cookie=2a92064a876ea000 in_seq=0 out_seq=0 2024-03-21T11:22:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.579+0000 7f8fc3400640 10 client.25219 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:22:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.579+0000 7f8fc3400640 1 -- 172.21.15.47:0/4111139927 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b627ba4520 con 0x55b627cb4990 2024-03-21T11:22:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.579+0000 7f8fc3400640 10 client.25219 handle_command_reply: tid=0 2024-03-21T11:22:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.579+0000 7f8fe760a740 1 client.25219 shutdown 2024-03-21T11:22:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.580+0000 7f8fe760a740 1 -- 172.21.15.47:0/4111139927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627c9dd80 msgr2=0x55b627ca81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.580+0000 7f8fe760a740 1 --2- 172.21.15.47:0/4111139927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627c9dd80 0x55b627ca81f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00058c0 tx=0x7f8fd0005ab0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.580+0000 7f8fe760a740 1 -- 172.21.15.47:0/4111139927 shutdown_connections 2024-03-21T11:22:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.580+0000 7f8fe760a740 1 --2- 172.21.15.47:0/4111139927 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b627cb4990 0x55b627cb3a60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.580+0000 7f8fe760a740 1 --2- 172.21.15.47:0/4111139927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627ca38e0 0x55b627caf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.580+0000 7f8fe760a740 1 --2- 172.21.15.47:0/4111139927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 0x55b627ca8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.580+0000 7f8fe760a740 1 --2- 172.21.15.47:0/4111139927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627c9dd80 0x55b627ca81f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.580+0000 7f8fe760a740 1 -- 172.21.15.47:0/4111139927 >> 172.21.15.47:0/4111139927 conn(0x55b627c992f0 msgr2=0x55b627c984f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.580+0000 7f8fc3400640 0 client.25219 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:22:56.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.580+0000 7f8fe760a740 1 -- 172.21.15.47:0/4111139927 shutdown_connections 2024-03-21T11:22:56.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.580+0000 7f8fe760a740 1 -- 172.21.15.47:0/4111139927 wait complete. 2024-03-21T11:22:56.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.580+0000 7f8fe760a740 20 client.25219 trim_cache size 0 max 16384 2024-03-21T11:22:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.598+0000 7f8fe760a740 1 Processor -- start 2024-03-21T11:22:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.598+0000 7f8fe760a740 1 -- start start 2024-03-21T11:22:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.599+0000 7f8fe760a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9dd80 0x55b627ca8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.599+0000 7f8fe760a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 0x55b627caf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.599+0000 7f8fe760a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627ca38e0 0x55b627cb3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.599+0000 7f8fe760a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b627cb41b0 con 0x55b627c9dd80 2024-03-21T11:22:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.599+0000 7f8fe760a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b627cb4810 con 0x55b627c9e770 2024-03-21T11:22:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.599+0000 7f8fe760a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b627cb4e70 con 0x55b627ca38e0 2024-03-21T11:22:56.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.599+0000 7f8fe2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627ca38e0 0x55b627cb3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.599+0000 7f8fe2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627ca38e0 0x55b627cb3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41904/0 (socket says 172.21.15.47:41904) 2024-03-21T11:22:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.599+0000 7f8fe1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 0x55b627caf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.599+0000 7f8fe2800640 1 -- 172.21.15.47:0/182230761 learned_addr learned my addr 172.21.15.47:0/182230761 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.599+0000 7f8fe1e00640 1 --2- 172.21.15.47:0/182230761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9dd80 0x55b627ca8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.599+0000 7f8fe2800640 1 -- 172.21.15.47:0/182230761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 msgr2=0x55b627caf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.600+0000 7f8fe2800640 1 --2- 172.21.15.47:0/182230761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 0x55b627caf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.600+0000 7f8fe2800640 1 -- 172.21.15.47:0/182230761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9dd80 msgr2=0x55b627ca8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.600+0000 7f8fe2800640 1 --2- 172.21.15.47:0/182230761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9dd80 0x55b627ca8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.600+0000 7f8fe2800640 1 -- 172.21.15.47:0/182230761 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b627c8b170 con 0x55b627ca38e0 2024-03-21T11:22:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.600+0000 7f8fe1e00640 1 --2- 172.21.15.47:0/182230761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9dd80 0x55b627ca8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:22:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.600+0000 7f8fe2800640 1 --2- 172.21.15.47:0/182230761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627ca38e0 0x55b627cb3a60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8fd801e770 tx=0x7f8fd8019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4652095b4e819408 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.600+0000 7f8fce600640 1 -- 172.21.15.47:0/182230761 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8011640 con 0x55b627ca38e0 2024-03-21T11:22:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.600+0000 7f8fce600640 1 -- 172.21.15.47:0/182230761 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fd8011cc0 con 0x55b627ca38e0 2024-03-21T11:22:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.600+0000 7f8fce600640 1 -- 172.21.15.47:0/182230761 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd801cdc0 con 0x55b627ca38e0 2024-03-21T11:22:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.601+0000 7f8fe760a740 1 -- 172.21.15.47:0/182230761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627ca38e0 msgr2=0x55b627cb3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.601+0000 7f8fe760a740 1 --2- 172.21.15.47:0/182230761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627ca38e0 0x55b627cb3a60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8fd801e770 tx=0x7f8fd8019000 comp rx=0 tx=0).stop 2024-03-21T11:22:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.601+0000 7f8fe760a740 1 -- 172.21.15.47:0/182230761 shutdown_connections 2024-03-21T11:22:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.601+0000 7f8fe760a740 1 --2- 172.21.15.47:0/182230761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627ca38e0 0x55b627cb3a60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.601+0000 7f8fe760a740 1 --2- 172.21.15.47:0/182230761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627c9e770 0x55b627caf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.601+0000 7f8fe760a740 1 --2- 172.21.15.47:0/182230761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9dd80 0x55b627ca8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.601+0000 7f8fe760a740 1 -- 172.21.15.47:0/182230761 >> 172.21.15.47:0/182230761 conn(0x55b627c992f0 msgr2=0x55b627ca6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.601+0000 7f8fe760a740 1 -- 172.21.15.47:0/182230761 shutdown_connections 2024-03-21T11:22:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.601+0000 7f8fe760a740 1 -- 172.21.15.47:0/182230761 wait complete. 2024-03-21T11:22:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.601+0000 7f8fe760a740 1 -- start start 2024-03-21T11:22:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.602+0000 7f8fe760a740 1 Processor -- start 2024-03-21T11:22:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.602+0000 7f8fe760a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627c9dd80 0x55b627caf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.602+0000 7f8fe760a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9e770 0x55b627caf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.602+0000 7f8fe760a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627ca38e0 0x55b627cabd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.602+0000 7f8fe760a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b627cb5b30 con 0x55b627c9e770 2024-03-21T11:22:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.602+0000 7f8fe760a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b627cb54d0 con 0x55b627ca38e0 2024-03-21T11:22:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.602+0000 7f8fe760a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b627cb6190 con 0x55b627c9dd80 2024-03-21T11:22:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.602+0000 7f8fe760a740 10 client.0 _ll_register_callbacks cb 0x7f8fd00d3310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:22:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.602+0000 7f8fe2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627ca38e0 0x55b627cabd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.602+0000 7f8fe1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9e770 0x55b627caf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.602+0000 7f8fe1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627c9dd80 0x55b627caf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.602+0000 7f8fe2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627ca38e0 0x55b627cabd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53100/0 (socket says 172.21.15.47:53100) 2024-03-21T11:22:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.602+0000 7f8fe2800640 1 -- 172.21.15.47:0/1170746378 learned_addr learned my addr 172.21.15.47:0/1170746378 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.603+0000 7f8fe2800640 1 -- 172.21.15.47:0/1170746378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627c9dd80 msgr2=0x55b627caf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.603+0000 7f8fe2800640 1 --2- 172.21.15.47:0/1170746378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627c9dd80 0x55b627caf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.603+0000 7f8fe2800640 1 -- 172.21.15.47:0/1170746378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9e770 msgr2=0x55b627caf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.603+0000 7f8fe2800640 1 --2- 172.21.15.47:0/1170746378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9e770 0x55b627caf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.603+0000 7f8fe2800640 1 -- 172.21.15.47:0/1170746378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b627cac590 con 0x55b627ca38e0 2024-03-21T11:22:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.603+0000 7f8fe1e00640 1 --2- 172.21.15.47:0/1170746378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627c9dd80 0x55b627caf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:22:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.603+0000 7f8fe2800640 1 --2- 172.21.15.47:0/1170746378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627ca38e0 0x55b627cabd60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8018040 tx=0x7f8fd8005ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b20a525abf29ce0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.603+0000 7f8fc3400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:22:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.603+0000 7f8fc3400640 1 -- 172.21.15.47:0/1170746378 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8036060 con 0x55b627ca38e0 2024-03-21T11:22:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.603+0000 7f8fe760a740 1 -- 172.21.15.47:0/1170746378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fd8011cc0 con 0x55b627ca38e0 2024-03-21T11:22:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.603+0000 7f8fc3400640 1 -- 172.21.15.47:0/1170746378 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8fd801c430 con 0x55b627ca38e0 2024-03-21T11:22:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.603+0000 7f8fc3400640 1 -- 172.21.15.47:0/1170746378 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fd803d050 con 0x55b627ca38e0 2024-03-21T11:22:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.603+0000 7f8fe760a740 10 client.25211 fetch_fsmap learned FSMap version 27 2024-03-21T11:22:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.603+0000 7f8fe760a740 1 -- 172.21.15.47:0/1170746378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b627d107c0 con 0x55b627ca38e0 2024-03-21T11:22:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.604+0000 7f8fc3400640 1 -- 172.21.15.47:0/1170746378 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fd8011cc0 con 0x55b627ca38e0 2024-03-21T11:22:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.604+0000 7f8fc3400640 1 -- 172.21.15.47:0/1170746378 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8fd8029500 con 0x55b627ca38e0 2024-03-21T11:22:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.604+0000 7f8fe760a740 10 client.25211 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:22:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.604+0000 7f8fe760a740 10 client.25211 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-21T11:22:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.604+0000 7f8fe760a740 20 client.25211 populate_metadata read hostname 'smithi047' 2024-03-21T11:22:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.604+0000 7f8fe760a740 1 --2- 172.21.15.47:0/1170746378 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b627d149d0 0x55b627d10a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.604+0000 7f8fe760a740 4 client.25211 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T11:22:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.604+0000 7f8fe760a740 1 -- 172.21.15.47:0/1170746378 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b627c101d0 con 0x55b627d149d0 2024-03-21T11:22:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.604+0000 7f8fe1e00640 1 --2- 172.21.15.47:0/1170746378 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b627d149d0 0x55b627d10a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fe1e00640 1 --2- 172.21.15.47:0/1170746378 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b627d149d0 0x55b627d10a50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f778683a2627e064 server_cookie=e2252af2e3bc3bb3 in_seq=0 out_seq=0 2024-03-21T11:22:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fc3400640 10 client.25211 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:22:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fc3400640 1 -- 172.21.15.47:0/1170746378 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b627c101d0 con 0x55b627d149d0 2024-03-21T11:22:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fc3400640 10 client.25211 handle_command_reply: tid=0 2024-03-21T11:22:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fe760a740 1 client.25211 shutdown 2024-03-21T11:22:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fe760a740 1 -- 172.21.15.47:0/1170746378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627ca38e0 msgr2=0x55b627cabd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fe760a740 1 --2- 172.21.15.47:0/1170746378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627ca38e0 0x55b627cabd60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8018040 tx=0x7f8fd8005ae0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fe760a740 1 -- 172.21.15.47:0/1170746378 shutdown_connections 2024-03-21T11:22:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fe760a740 1 --2- 172.21.15.47:0/1170746378 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b627d149d0 0x55b627d10a50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fe760a740 1 --2- 172.21.15.47:0/1170746378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b627ca38e0 0x55b627cabd60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fe760a740 1 --2- 172.21.15.47:0/1170746378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b627c9e770 0x55b627caf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fe760a740 1 --2- 172.21.15.47:0/1170746378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b627c9dd80 0x55b627caf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fe760a740 1 -- 172.21.15.47:0/1170746378 >> 172.21.15.47:0/1170746378 conn(0x55b627c992f0 msgr2=0x55b627c984f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fc3400640 0 client.25211 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:22:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fe760a740 1 -- 172.21.15.47:0/1170746378 shutdown_connections 2024-03-21T11:22:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fe760a740 1 -- 172.21.15.47:0/1170746378 wait complete. 2024-03-21T11:22:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.605+0000 7f8fe760a740 20 client.25211 trim_cache size 0 max 16384 2024-03-21T11:22:56.611 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:22:56.611 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:22:56.611 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 1, 2024-03-21T11:22:56.612 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24553, 2024-03-21T11:22:56.612 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T11:22:56.612 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T11:22:56.612 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T11:22:56.612 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 101.77833025699999, 2024-03-21T11:22:56.612 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 27, 2024-03-21T11:22:56.612 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T11:22:56.612 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-03-21T11:22:56.613 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 145.45518802199999 2024-03-21T11:22:56.613 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:22:56.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.606+0000 7f8fe3200640 1 -- 172.21.15.47:0/4159312869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8fc40a2f20 msgr2=0x7f8fc40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.606+0000 7f8fe3200640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8fc40a2f20 0x7f8fc40a53e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55b627c9efd0 tx=0x7f8fd80096a0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.606+0000 7f8fe3200640 1 -- 172.21.15.47:0/4159312869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fdc107bc0 msgr2=0x7f8fdc19de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.606+0000 7f8fe3200640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fdc107bc0 0x7f8fdc19de90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000e9c0 tx=0x7f8fd000ee80 comp rx=0 tx=0).stop 2024-03-21T11:22:56.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.606+0000 7f8fe3200640 1 -- 172.21.15.47:0/4159312869 shutdown_connections 2024-03-21T11:22:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.606+0000 7f8fe3200640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8fc40a2f20 0x7f8fc40a53e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.606+0000 7f8fe3200640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fdc109770 0x7f8fdc1a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.606+0000 7f8fe3200640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fdc108db0 0x7f8fdc19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.606+0000 7f8fe3200640 1 --2- 172.21.15.47:0/4159312869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fdc107bc0 0x7f8fdc19de90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.606+0000 7f8fe3200640 1 -- 172.21.15.47:0/4159312869 >> 172.21.15.47:0/4159312869 conn(0x7f8fdc076b70 msgr2=0x7f8fdc10c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.606+0000 7f8fe3200640 1 -- 172.21.15.47:0/4159312869 shutdown_connections 2024-03-21T11:22:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.606+0000 7f8fe3200640 1 -- 172.21.15.47:0/4159312869 wait complete. 2024-03-21T11:22:56.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-21T11:22:56.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.858+0000 7f5a86200640 1 -- 172.21.15.47:0/1635771903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a80108db0 msgr2=0x7f5a80109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.858+0000 7f5a86200640 1 --2- 172.21.15.47:0/1635771903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a80108db0 0x7f5a80109230 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009900 tx=0x7f5a6c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:22:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.858+0000 7f5a86200640 1 -- 172.21.15.47:0/1635771903 shutdown_connections 2024-03-21T11:22:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.858+0000 7f5a86200640 1 --2- 172.21.15.47:0/1635771903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a80109770 0x7f5a80110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.858+0000 7f5a86200640 1 --2- 172.21.15.47:0/1635771903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a80108db0 0x7f5a80109230 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.858+0000 7f5a86200640 1 --2- 172.21.15.47:0/1635771903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a80107bc0 0x7f5a80107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.858+0000 7f5a86200640 1 -- 172.21.15.47:0/1635771903 >> 172.21.15.47:0/1635771903 conn(0x7f5a80076b70 msgr2=0x7f5a80078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.858+0000 7f5a86200640 1 -- 172.21.15.47:0/1635771903 shutdown_connections 2024-03-21T11:22:56.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.858+0000 7f5a86200640 1 -- 172.21.15.47:0/1635771903 wait complete. 2024-03-21T11:22:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.859+0000 7f5a86200640 1 Processor -- start 2024-03-21T11:22:56.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.859+0000 7f5a86200640 1 -- start start 2024-03-21T11:22:56.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.860+0000 7f5a86200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a80107bc0 0x7f5a8019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.860+0000 7f5a86200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a80108db0 0x7f5a8019e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.860+0000 7f5a86200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a80109770 0x7f5a801a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.860+0000 7f5a86200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a80113870 con 0x7f5a80108db0 2024-03-21T11:22:56.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.860+0000 7f5a86200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a801136f0 con 0x7f5a80109770 2024-03-21T11:22:56.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.860+0000 7f5a86200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a801139f0 con 0x7f5a80107bc0 2024-03-21T11:22:56.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.860+0000 7f5a84e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a80107bc0 0x7f5a8019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.860+0000 7f5a85800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a80109770 0x7f5a801a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.860+0000 7f5a84e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a80107bc0 0x7f5a8019de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41932/0 (socket says 172.21.15.47:41932) 2024-03-21T11:22:56.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.860+0000 7f5a84e00640 1 -- 172.21.15.47:0/2727874846 learned_addr learned my addr 172.21.15.47:0/2727874846 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.860+0000 7f5a7fe00640 1 --2- 172.21.15.47:0/2727874846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a80108db0 0x7f5a8019e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.861+0000 7f5a84e00640 1 -- 172.21.15.47:0/2727874846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a80109770 msgr2=0x7f5a801a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.861+0000 7f5a84e00640 1 --2- 172.21.15.47:0/2727874846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a80109770 0x7f5a801a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.861+0000 7f5a84e00640 1 -- 172.21.15.47:0/2727874846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a80108db0 msgr2=0x7f5a8019e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.861+0000 7f5a84e00640 1 --2- 172.21.15.47:0/2727874846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a80108db0 0x7f5a8019e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.861+0000 7f5a84e00640 1 -- 172.21.15.47:0/2727874846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a801a5ab0 con 0x7f5a80107bc0 2024-03-21T11:22:56.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.861+0000 7f5a84e00640 1 --2- 172.21.15.47:0/2727874846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a80107bc0 0x7f5a8019de90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400e9c0 tx=0x7f5a7400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.861+0000 7f5a7d600640 1 -- 172.21.15.47:0/2727874846 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7400cd80 con 0x7f5a80107bc0 2024-03-21T11:22:56.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.861+0000 7f5a7d600640 1 -- 172.21.15.47:0/2727874846 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a740035f0 con 0x7f5a80107bc0 2024-03-21T11:22:56.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.861+0000 7f5a7d600640 1 -- 172.21.15.47:0/2727874846 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74010690 con 0x7f5a80107bc0 2024-03-21T11:22:56.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.861+0000 7f5a86200640 1 -- 172.21.15.47:0/2727874846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a801a5da0 con 0x7f5a80107bc0 2024-03-21T11:22:56.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.862+0000 7f5a86200640 1 -- 172.21.15.47:0/2727874846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a801a6150 con 0x7f5a80107bc0 2024-03-21T11:22:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.864+0000 7f5a7d600640 1 -- 172.21.15.47:0/2727874846 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5a740037a0 con 0x7f5a80107bc0 2024-03-21T11:22:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.865+0000 7f5a8a591740 1 Processor -- start 2024-03-21T11:22:56.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.865+0000 7f5a8a591740 1 -- start start 2024-03-21T11:22:56.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.865+0000 7f5a8a591740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564115618d80 0x5641156191a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.865+0000 7f5a8a591740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x56411561e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.865+0000 7f5a8a591740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56411561e8e0 0x564115620d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.865+0000 7f5a8a591740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56411557dc90 con 0x56411561e8e0 2024-03-21T11:22:56.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.865+0000 7f5a8a591740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115526980 con 0x564115619770 2024-03-21T11:22:56.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.865+0000 7f5a8a591740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115579d30 con 0x564115618d80 2024-03-21T11:22:56.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.865+0000 7f5a85800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564115618d80 0x5641156191a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.865+0000 7f5a85800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564115618d80 0x5641156191a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41936/0 (socket says 172.21.15.47:41936) 2024-03-21T11:22:56.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.865+0000 7f5a85800640 1 -- 172.21.15.47:0/2648249903 learned_addr learned my addr 172.21.15.47:0/2648249903 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.866+0000 7f5a84e00640 1 --2- 172.21.15.47:0/2648249903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x56411561e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.866+0000 7f5a7fe00640 1 --2- 172.21.15.47:0/2648249903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56411561e8e0 0x564115620d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.866+0000 7f5a85800640 1 -- 172.21.15.47:0/2648249903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 msgr2=0x56411561e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.866+0000 7f5a85800640 1 --2- 172.21.15.47:0/2648249903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x56411561e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.866+0000 7f5a85800640 1 -- 172.21.15.47:0/2648249903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56411561e8e0 msgr2=0x564115620d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.866+0000 7f5a85800640 1 --2- 172.21.15.47:0/2648249903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56411561e8e0 0x564115620d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.866+0000 7f5a85800640 1 -- 172.21.15.47:0/2648249903 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56411553e510 con 0x564115618d80 2024-03-21T11:22:56.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.866+0000 7f5a85800640 1 --2- 172.21.15.47:0/2648249903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564115618d80 0x5641156191a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009fd0 tx=0x7f5a780118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b89043c82c65fbf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.866+0000 7f5a7d600640 1 --2- 172.21.15.47:0/2727874846 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5a680a2fa0 0x7f5a680a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.866+0000 7f5a72000640 1 -- 172.21.15.47:0/2648249903 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78004d20 con 0x564115618d80 2024-03-21T11:22:56.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.866+0000 7f5a7d600640 1 -- 172.21.15.47:0/2727874846 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5a740cfbf0 con 0x7f5a80107bc0 2024-03-21T11:22:56.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.866+0000 7f5a72000640 1 -- 172.21.15.47:0/2648249903 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a78004050 con 0x564115618d80 2024-03-21T11:22:56.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.866+0000 7f5a72000640 1 -- 172.21.15.47:0/2648249903 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78005260 con 0x564115618d80 2024-03-21T11:22:56.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.867+0000 7f5a84e00640 1 --2- 172.21.15.47:0/2727874846 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5a680a2fa0 0x7f5a680a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.867+0000 7f5a8a591740 1 -- 172.21.15.47:0/2648249903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564115618d80 msgr2=0x5641156191a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.867+0000 7f5a8a591740 1 --2- 172.21.15.47:0/2648249903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564115618d80 0x5641156191a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009fd0 tx=0x7f5a780118d0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.867+0000 7f5a84e00640 1 --2- 172.21.15.47:0/2727874846 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5a680a2fa0 0x7f5a680a5460 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5a74098390 tx=0x7f5a740d3630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.867+0000 7f5a8a591740 1 -- 172.21.15.47:0/2648249903 shutdown_connections 2024-03-21T11:22:56.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.867+0000 7f5a8a591740 1 --2- 172.21.15.47:0/2648249903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56411561e8e0 0x564115620d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.867+0000 7f5a8a591740 1 --2- 172.21.15.47:0/2648249903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x56411561e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.867+0000 7f5a8a591740 1 --2- 172.21.15.47:0/2648249903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564115618d80 0x5641156191a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.867+0000 7f5a8a591740 1 -- 172.21.15.47:0/2648249903 >> 172.21.15.47:0/2648249903 conn(0x5641156142f0 msgr2=0x5641156146f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.867+0000 7f5a8a591740 1 -- 172.21.15.47:0/2648249903 shutdown_connections 2024-03-21T11:22:56.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.867+0000 7f5a8a591740 1 -- 172.21.15.47:0/2648249903 wait complete. 2024-03-21T11:22:56.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.868+0000 7f5a8a591740 1 -- start start 2024-03-21T11:22:56.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.868+0000 7f5a8a591740 1 Processor -- start 2024-03-21T11:22:56.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a8a591740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564115618d80 0x564115623270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a8a591740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x5641156237b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a8a591740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56411561e8e0 0x56411562a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a8a591740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115621890 con 0x564115618d80 2024-03-21T11:22:56.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a8a591740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115539dd0 con 0x564115619770 2024-03-21T11:22:56.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a8a591740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115621a10 con 0x56411561e8e0 2024-03-21T11:22:56.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a8a591740 10 client.0 _ll_register_callbacks cb 0x564115291ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:22:56.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x5641156237b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x5641156237b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53132/0 (socket says 172.21.15.47:53132) 2024-03-21T11:22:56.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a7fe00640 1 -- 172.21.15.47:0/2500611388 learned_addr learned my addr 172.21.15.47:0/2500611388 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a85800640 1 --2- 172.21.15.47:0/2500611388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564115618d80 0x564115623270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a7fe00640 1 -- 172.21.15.47:0/2500611388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56411561e8e0 msgr2=0x56411562a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a85800640 1 --2- 172.21.15.47:0/2500611388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56411561e8e0 0x56411562a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a7fe00640 1 --2- 172.21.15.47:0/2500611388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56411561e8e0 0x56411562a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a7fe00640 1 -- 172.21.15.47:0/2500611388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564115618d80 msgr2=0x564115623270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a7fe00640 1 --2- 172.21.15.47:0/2500611388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564115618d80 0x564115623270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a7fe00640 1 -- 172.21.15.47:0/2500611388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641152922d0 con 0x564115619770 2024-03-21T11:22:56.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.869+0000 7f5a85800640 1 --2- 172.21.15.47:0/2500611388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56411561e8e0 0x56411562a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:22:56.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.870+0000 7f5a7fe00640 1 --2- 172.21.15.47:0/2500611388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x5641156237b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c000c00 tx=0x7f5a6c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6259c5cdde2d0307 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.870+0000 7f5a66a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:22:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.870+0000 7f5a66a00640 1 -- 172.21.15.47:0/2500611388 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c005d50 con 0x564115619770 2024-03-21T11:22:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.870+0000 7f5a66a00640 1 -- 172.21.15.47:0/2500611388 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5a6c005230 con 0x564115619770 2024-03-21T11:22:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.870+0000 7f5a8a591740 1 -- 172.21.15.47:0/2500611388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a78004050 con 0x564115619770 2024-03-21T11:22:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.870+0000 7f5a66a00640 1 -- 172.21.15.47:0/2500611388 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a6c050050 con 0x564115619770 2024-03-21T11:22:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.870+0000 7f5a8a591740 10 client.25223 fetch_fsmap learned FSMap version 27 2024-03-21T11:22:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.870+0000 7f5a8a591740 1 -- 172.21.15.47:0/2500611388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56411535c6b0 con 0x564115619770 2024-03-21T11:22:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.871+0000 7f5a66a00640 1 -- 172.21.15.47:0/2500611388 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a78004050 con 0x564115619770 2024-03-21T11:22:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.871+0000 7f5a66a00640 1 -- 172.21.15.47:0/2500611388 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5a6c003710 con 0x564115619770 2024-03-21T11:22:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.871+0000 7f5a8a591740 10 client.25223 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:22:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.871+0000 7f5a8a591740 10 client.25223 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T11:22:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.871+0000 7f5a8a591740 20 client.25223 populate_metadata read hostname 'smithi047' 2024-03-21T11:22:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.871+0000 7f5a8a591740 1 --2- 172.21.15.47:0/2500611388 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56411562fa10 0x56411562ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.871+0000 7f5a8a591740 4 client.25223 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:22:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.871+0000 7f5a8a591740 1 -- 172.21.15.47:0/2500611388 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56411551f520 con 0x56411562fa10 2024-03-21T11:22:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.871+0000 7f5a85800640 1 --2- 172.21.15.47:0/2500611388 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56411562fa10 0x56411562ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.872+0000 7f5a85800640 1 --2- 172.21.15.47:0/2500611388 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56411562fa10 0x56411562ead0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87bf23728c662d62 server_cookie=b4e68ff94655115f in_seq=0 out_seq=0 2024-03-21T11:22:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.872+0000 7f5a66a00640 10 client.25223 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:22:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.872+0000 7f5a66a00640 1 -- 172.21.15.47:0/2500611388 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56411551f520 con 0x56411562fa10 2024-03-21T11:22:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.872+0000 7f5a66a00640 10 client.25223 handle_command_reply: tid=0 2024-03-21T11:22:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.872+0000 7f5a8a591740 1 client.25223 shutdown 2024-03-21T11:22:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.873+0000 7f5a8a591740 1 -- 172.21.15.47:0/2500611388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 msgr2=0x5641156237b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.873+0000 7f5a8a591740 1 --2- 172.21.15.47:0/2500611388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x5641156237b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c000c00 tx=0x7f5a6c004270 comp rx=0 tx=0).stop 2024-03-21T11:22:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.873+0000 7f5a8a591740 1 -- 172.21.15.47:0/2500611388 shutdown_connections 2024-03-21T11:22:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.873+0000 7f5a8a591740 1 --2- 172.21.15.47:0/2500611388 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56411562fa10 0x56411562ead0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.873+0000 7f5a8a591740 1 --2- 172.21.15.47:0/2500611388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56411561e8e0 0x56411562a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.873+0000 7f5a8a591740 1 --2- 172.21.15.47:0/2500611388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x5641156237b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.873+0000 7f5a8a591740 1 --2- 172.21.15.47:0/2500611388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564115618d80 0x564115623270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.873+0000 7f5a66a00640 0 client.25223 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:22:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.873+0000 7f5a8a591740 1 -- 172.21.15.47:0/2500611388 >> 172.21.15.47:0/2500611388 conn(0x5641156142f0 msgr2=0x5641156134f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.873+0000 7f5a8a591740 1 -- 172.21.15.47:0/2500611388 shutdown_connections 2024-03-21T11:22:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.873+0000 7f5a8a591740 1 -- 172.21.15.47:0/2500611388 wait complete. 2024-03-21T11:22:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.873+0000 7f5a8a591740 20 client.25223 trim_cache size 0 max 16384 2024-03-21T11:22:56.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.892+0000 7f5a8a591740 1 Processor -- start 2024-03-21T11:22:56.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.893+0000 7f5a8a591740 1 -- start start 2024-03-21T11:22:56.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.893+0000 7f5a8a591740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564115618d80 0x564115621db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.893+0000 7f5a8a591740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x56411562a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.893+0000 7f5a8a591740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56411561e8e0 0x5641156146d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.893+0000 7f5a8a591740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56411562f230 con 0x564115618d80 2024-03-21T11:22:56.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.893+0000 7f5a8a591740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56411562f890 con 0x564115619770 2024-03-21T11:22:56.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.893+0000 7f5a8a591740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56411562fef0 con 0x56411561e8e0 2024-03-21T11:22:56.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.893+0000 7f5a7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x56411562a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.893+0000 7f5a85800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564115618d80 0x564115621db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.893+0000 7f5a7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x56411562a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53134/0 (socket says 172.21.15.47:53134) 2024-03-21T11:22:56.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.893+0000 7f5a7fe00640 1 -- 172.21.15.47:0/435189068 learned_addr learned my addr 172.21.15.47:0/435189068 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.893+0000 7f5a85800640 1 --2- 172.21.15.47:0/435189068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56411561e8e0 0x5641156146d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.894+0000 7f5a7fe00640 1 -- 172.21.15.47:0/435189068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56411561e8e0 msgr2=0x5641156146d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.894+0000 7f5a7fe00640 1 --2- 172.21.15.47:0/435189068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56411561e8e0 0x5641156146d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.894+0000 7f5a7fe00640 1 -- 172.21.15.47:0/435189068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564115618d80 msgr2=0x564115621db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.894+0000 7f5a7fe00640 1 --2- 172.21.15.47:0/435189068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564115618d80 0x564115621db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.894+0000 7f5a7fe00640 1 -- 172.21.15.47:0/435189068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56411561a000 con 0x564115619770 2024-03-21T11:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.894+0000 7f5a7fe00640 1 --2- 172.21.15.47:0/435189068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x56411562a7b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c003b80 tx=0x7f5a6c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0fe920f0e573bca server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.894+0000 7f5a71600640 1 -- 172.21.15.47:0/435189068 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c058020 con 0x564115619770 2024-03-21T11:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.894+0000 7f5a71600640 1 -- 172.21.15.47:0/435189068 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a6c04d070 con 0x564115619770 2024-03-21T11:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.894+0000 7f5a71600640 1 -- 172.21.15.47:0/435189068 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0349a0 con 0x564115619770 2024-03-21T11:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.895+0000 7f5a8a591740 1 -- 172.21.15.47:0/435189068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 msgr2=0x56411562a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.895+0000 7f5a8a591740 1 --2- 172.21.15.47:0/435189068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x56411562a7b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c003b80 tx=0x7f5a6c009780 comp rx=0 tx=0).stop 2024-03-21T11:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.895+0000 7f5a8a591740 1 -- 172.21.15.47:0/435189068 shutdown_connections 2024-03-21T11:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.895+0000 7f5a8a591740 1 --2- 172.21.15.47:0/435189068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56411561e8e0 0x5641156146d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.895+0000 7f5a8a591740 1 --2- 172.21.15.47:0/435189068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115619770 0x56411562a7b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.895+0000 7f5a8a591740 1 --2- 172.21.15.47:0/435189068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564115618d80 0x564115621db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.895+0000 7f5a8a591740 1 -- 172.21.15.47:0/435189068 >> 172.21.15.47:0/435189068 conn(0x5641156142f0 msgr2=0x56411562ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.895+0000 7f5a8a591740 1 -- 172.21.15.47:0/435189068 shutdown_connections 2024-03-21T11:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.895+0000 7f5a8a591740 1 -- 172.21.15.47:0/435189068 wait complete. 2024-03-21T11:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.896+0000 7f5a8a591740 1 -- start start 2024-03-21T11:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.896+0000 7f5a8a591740 1 Processor -- start 2024-03-21T11:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.896+0000 7f5a8a591740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115618d80 0x56411562a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.896+0000 7f5a8a591740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564115619770 0x56411562a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.896+0000 7f5a8a591740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56411561e8e0 0x564115626fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.896+0000 7f5a8a591740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115630bb0 con 0x56411561e8e0 2024-03-21T11:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.896+0000 7f5a8a591740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115630550 con 0x564115618d80 2024-03-21T11:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.896+0000 7f5a8a591740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115631210 con 0x564115619770 2024-03-21T11:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.896+0000 7f5a8a591740 10 client.0 _ll_register_callbacks cb 0x7f5a780093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.897+0000 7f5a7fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564115619770 0x56411562a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.897+0000 7f5a7fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564115619770 0x56411562a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41958/0 (socket says 172.21.15.47:41958) 2024-03-21T11:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.897+0000 7f5a7fe00640 1 -- 172.21.15.47:0/3072373890 learned_addr learned my addr 172.21.15.47:0/3072373890 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.897+0000 7f5a85800640 1 --2- 172.21.15.47:0/3072373890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115618d80 0x56411562a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.897+0000 7f5a85800640 1 --2- 172.21.15.47:0/3072373890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56411561e8e0 0x564115626fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.897+0000 7f5a7fe00640 1 -- 172.21.15.47:0/3072373890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115618d80 msgr2=0x56411562a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.897+0000 7f5a7fe00640 1 --2- 172.21.15.47:0/3072373890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115618d80 0x56411562a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.897+0000 7f5a7fe00640 1 -- 172.21.15.47:0/3072373890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56411561e8e0 msgr2=0x564115626fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.897+0000 7f5a7fe00640 1 --2- 172.21.15.47:0/3072373890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56411561e8e0 0x564115626fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.897+0000 7f5a7fe00640 1 -- 172.21.15.47:0/3072373890 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641156277f0 con 0x564115619770 2024-03-21T11:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.897+0000 7f5a85800640 1 --2- 172.21.15.47:0/3072373890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115618d80 0x56411562a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.897+0000 7f5a7fe00640 1 --2- 172.21.15.47:0/3072373890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564115619770 0x56411562a820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c005820 tx=0x7f5a6c003f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6012d5bf6909c99 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.897+0000 7f5a66a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.897+0000 7f5a66a00640 1 -- 172.21.15.47:0/3072373890 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c060020 con 0x564115619770 2024-03-21T11:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.898+0000 7f5a8a591740 1 -- 172.21.15.47:0/3072373890 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a6c04d070 con 0x564115619770 2024-03-21T11:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.898+0000 7f5a66a00640 1 -- 172.21.15.47:0/3072373890 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5a6c034640 con 0x564115619770 2024-03-21T11:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.898+0000 7f5a66a00640 1 -- 172.21.15.47:0/3072373890 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a6c068050 con 0x564115619770 2024-03-21T11:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.898+0000 7f5a8a591740 10 client.25240 fetch_fsmap learned FSMap version 27 2024-03-21T11:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.898+0000 7f5a8a591740 1 -- 172.21.15.47:0/3072373890 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56411568b860 con 0x564115619770 2024-03-21T11:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.898+0000 7f5a66a00640 1 -- 172.21.15.47:0/3072373890 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a6c04d070 con 0x564115619770 2024-03-21T11:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.898+0000 7f5a66a00640 1 -- 172.21.15.47:0/3072373890 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5a6c03e400 con 0x564115619770 2024-03-21T11:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.898+0000 7f5a8a591740 10 client.25240 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.898+0000 7f5a8a591740 10 client.25240 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T11:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.898+0000 7f5a8a591740 20 client.25240 populate_metadata read hostname 'smithi047' 2024-03-21T11:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.898+0000 7f5a8a591740 1 --2- 172.21.15.47:0/3072373890 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56411568fbb0 0x56411568ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.898+0000 7f5a8a591740 4 client.25240 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T11:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.898+0000 7f5a8a591740 1 -- 172.21.15.47:0/3072373890 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56411558b1d0 con 0x56411568fbb0 2024-03-21T11:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.899+0000 7f5a85800640 1 --2- 172.21.15.47:0/3072373890 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56411568fbb0 0x56411568ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.899+0000 7f5a85800640 1 --2- 172.21.15.47:0/3072373890 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56411568fbb0 0x56411568ba90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2099a7750d1beb8f server_cookie=471561c87d392cf8 in_seq=0 out_seq=0 2024-03-21T11:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.899+0000 7f5a66a00640 10 client.25240 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.899+0000 7f5a66a00640 1 -- 172.21.15.47:0/3072373890 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x56411558b1d0 con 0x56411568fbb0 2024-03-21T11:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.899+0000 7f5a66a00640 10 client.25240 handle_command_reply: tid=0 2024-03-21T11:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.899+0000 7f5a8a591740 1 client.25240 shutdown 2024-03-21T11:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.900+0000 7f5a8a591740 1 -- 172.21.15.47:0/3072373890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564115619770 msgr2=0x56411562a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.900+0000 7f5a8a591740 1 --2- 172.21.15.47:0/3072373890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564115619770 0x56411562a820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c005820 tx=0x7f5a6c003f20 comp rx=0 tx=0).stop 2024-03-21T11:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.900+0000 7f5a8a591740 1 -- 172.21.15.47:0/3072373890 shutdown_connections 2024-03-21T11:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.900+0000 7f5a8a591740 1 --2- 172.21.15.47:0/3072373890 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56411568fbb0 0x56411568ba90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.900+0000 7f5a8a591740 1 --2- 172.21.15.47:0/3072373890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56411561e8e0 0x564115626fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.900+0000 7f5a8a591740 1 --2- 172.21.15.47:0/3072373890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564115619770 0x56411562a820 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.900+0000 7f5a8a591740 1 --2- 172.21.15.47:0/3072373890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564115618d80 0x56411562a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.900+0000 7f5a66a00640 0 client.25240 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.900+0000 7f5a8a591740 1 -- 172.21.15.47:0/3072373890 >> 172.21.15.47:0/3072373890 conn(0x5641156142f0 msgr2=0x564115621db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.900+0000 7f5a8a591740 1 -- 172.21.15.47:0/3072373890 shutdown_connections 2024-03-21T11:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.900+0000 7f5a8a591740 1 -- 172.21.15.47:0/3072373890 wait complete. 2024-03-21T11:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.900+0000 7f5a8a591740 20 client.25240 trim_cache size 0 max 16384 2024-03-21T11:22:56.905 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:22:56.905 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:22:56.905 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-21T11:22:56.906 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24581, 2024-03-21T11:22:56.906 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T11:22:56.906 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T11:22:56.906 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T11:22:56.906 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 127.31063118, 2024-03-21T11:22:56.906 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 27, 2024-03-21T11:22:56.906 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T11:22:56.906 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-03-21T11:22:56.906 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 134.68369998599999 2024-03-21T11:22:56.906 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.901+0000 7f5a86200640 1 -- 172.21.15.47:0/2727874846 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5a680a2fa0 msgr2=0x7f5a680a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.901+0000 7f5a86200640 1 --2- 172.21.15.47:0/2727874846 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5a680a2fa0 0x7f5a680a5460 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5a74098390 tx=0x7f5a740d3630 comp rx=0 tx=0).stop 2024-03-21T11:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.901+0000 7f5a86200640 1 -- 172.21.15.47:0/2727874846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a80107bc0 msgr2=0x7f5a8019de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.901+0000 7f5a86200640 1 --2- 172.21.15.47:0/2727874846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a80107bc0 0x7f5a8019de90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400e9c0 tx=0x7f5a7400ee80 comp rx=0 tx=0).stop 2024-03-21T11:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.901+0000 7f5a86200640 1 -- 172.21.15.47:0/2727874846 shutdown_connections 2024-03-21T11:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.901+0000 7f5a86200640 1 --2- 172.21.15.47:0/2727874846 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5a680a2fa0 0x7f5a680a5460 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.901+0000 7f5a86200640 1 --2- 172.21.15.47:0/2727874846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a80109770 0x7f5a801a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.901+0000 7f5a86200640 1 --2- 172.21.15.47:0/2727874846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a80108db0 0x7f5a8019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.901+0000 7f5a86200640 1 --2- 172.21.15.47:0/2727874846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a80107bc0 0x7f5a8019de90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.901+0000 7f5a86200640 1 -- 172.21.15.47:0/2727874846 >> 172.21.15.47:0/2727874846 conn(0x7f5a80076b70 msgr2=0x7f5a8010c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.901+0000 7f5a86200640 1 -- 172.21.15.47:0/2727874846 shutdown_connections 2024-03-21T11:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:56.901+0000 7f5a86200640 1 -- 172.21.15.47:0/2727874846 wait complete. 2024-03-21T11:22:56.922 INFO:tasks.cephadm:Setup complete, yielding 2024-03-21T11:22:56.922 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T11:22:56.935 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T11:22:56.935 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-21T11:22:57.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:56 smithi100 ceph-mon[36245]: pgmap v388: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-03-21T11:22:57.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:56 smithi102 ceph-mon[36748]: pgmap v388: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-03-21T11:22:57.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:56 smithi047 ceph-mon[31351]: pgmap v388: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-03-21T11:22:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:22:58.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:22:58 smithi047 ceph-mon[31351]: pgmap v389: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-03-21T11:22:59.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:22:58 smithi100 ceph-mon[36245]: pgmap v389: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-03-21T11:22:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:22:58 smithi102 ceph-mon[36748]: pgmap v389: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-03-21T11:22:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.560+0000 7f9f89a00700 1 -- 172.21.15.47:0/1450074382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f84107dd0 msgr2=0x7f9f8410a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.560+0000 7f9f89a00700 1 --2- 172.21.15.47:0/1450074382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f84107dd0 0x7f9f8410a240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00b0a0 tx=0x7f9f7c00b3b0 comp rx=0 tx=0).stop 2024-03-21T11:22:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.560+0000 7f9f89a00700 1 -- 172.21.15.47:0/1450074382 shutdown_connections 2024-03-21T11:22:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.560+0000 7f9f89a00700 1 --2- 172.21.15.47:0/1450074382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f84107dd0 0x7f9f8410a240 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.560+0000 7f9f89a00700 1 --2- 172.21.15.47:0/1450074382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f84102b70 0x7f9f84104f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.560+0000 7f9f89a00700 1 --2- 172.21.15.47:0/1450074382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f841054a0 0x7f9f84107890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.560+0000 7f9f89a00700 1 -- 172.21.15.47:0/1450074382 >> 172.21.15.47:0/1450074382 conn(0x7f9f840fc560 msgr2=0x7f9f840fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.560+0000 7f9f89a00700 1 -- 172.21.15.47:0/1450074382 shutdown_connections 2024-03-21T11:22:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.560+0000 7f9f89a00700 1 -- 172.21.15.47:0/1450074382 wait complete. 2024-03-21T11:22:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.561+0000 7f9f89a00700 1 Processor -- start 2024-03-21T11:22:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.561+0000 7f9f89a00700 1 -- start start 2024-03-21T11:22:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.562+0000 7f9f89a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f84102b70 0x7f9f8419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.562+0000 7f9f89a00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f841054a0 0x7f9f8419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.562+0000 7f9f89a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f84107dd0 0x7f9f841a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.562+0000 7f9f89a00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8410dce0 con 0x7f9f84107dd0 2024-03-21T11:22:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.562+0000 7f9f89a00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8410db60 con 0x7f9f84102b70 2024-03-21T11:22:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.562+0000 7f9f89a00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8410de60 con 0x7f9f841054a0 2024-03-21T11:22:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.562+0000 7f9f83400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f841054a0 0x7f9f8419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.562+0000 7f9f83e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f84102b70 0x7f9f8419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.562+0000 7f9f83400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f841054a0 0x7f9f8419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38734/0 (socket says 172.21.15.47:38734) 2024-03-21T11:22:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.562+0000 7f9f83400700 1 -- 172.21.15.47:0/1803557879 learned_addr learned my addr 172.21.15.47:0/1803557879 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:22:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.562+0000 7f9f83e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f84102b70 0x7f9f8419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35946/0 (socket says 172.21.15.47:35946) 2024-03-21T11:22:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.562+0000 7f9f89000700 1 --2- 172.21.15.47:0/1803557879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f84107dd0 0x7f9f841a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.562+0000 7f9f83e00700 1 -- 172.21.15.47:0/1803557879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f841054a0 msgr2=0x7f9f8419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.562+0000 7f9f83e00700 1 --2- 172.21.15.47:0/1803557879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f841054a0 0x7f9f8419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.563+0000 7f9f83e00700 1 -- 172.21.15.47:0/1803557879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f84107dd0 msgr2=0x7f9f841a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.563+0000 7f9f83e00700 1 --2- 172.21.15.47:0/1803557879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f84107dd0 0x7f9f841a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.563+0000 7f9f83e00700 1 -- 172.21.15.47:0/1803557879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f841a2100 con 0x7f9f84102b70 2024-03-21T11:22:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.563+0000 7f9f83e00700 1 --2- 172.21.15.47:0/1803557879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f84102b70 0x7f9f8419a4a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400d950 tx=0x7f9f7400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.563+0000 7f9f80c00700 1 -- 172.21.15.47:0/1803557879 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f74017070 con 0x7f9f84102b70 2024-03-21T11:22:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.563+0000 7f9f80c00700 1 -- 172.21.15.47:0/1803557879 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f74009e10 con 0x7f9f84102b70 2024-03-21T11:22:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.563+0000 7f9f89a00700 1 -- 172.21.15.47:0/1803557879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f841a23f0 con 0x7f9f84102b70 2024-03-21T11:22:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.563+0000 7f9f80c00700 1 -- 172.21.15.47:0/1803557879 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7400f3c0 con 0x7f9f84102b70 2024-03-21T11:22:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.563+0000 7f9f89a00700 1 -- 172.21.15.47:0/1803557879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f841a2930 con 0x7f9f84102b70 2024-03-21T11:22:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.564+0000 7f9f89a00700 1 -- 172.21.15.47:0/1803557879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f84068e50 con 0x7f9f84102b70 2024-03-21T11:22:59.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.566+0000 7f9f80c00700 1 -- 172.21.15.47:0/1803557879 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9f7401a070 con 0x7f9f84102b70 2024-03-21T11:22:59.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.568+0000 7f9f80c00700 1 --2- 172.21.15.47:0/1803557879 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9f680a2fb0 0x7f9f680a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:22:59.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.568+0000 7f9f80c00700 1 -- 172.21.15.47:0/1803557879 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9f740cf1b0 con 0x7f9f84102b70 2024-03-21T11:22:59.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.568+0000 7f9f83400700 1 --2- 172.21.15.47:0/1803557879 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9f680a2fb0 0x7f9f680a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:22:59.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.569+0000 7f9f83400700 1 --2- 172.21.15.47:0/1803557879 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9f680a2fb0 0x7f9f680a5470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9f70007c90 tx=0x7f9f70007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:22:59.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.576+0000 7f9f80c00700 1 -- 172.21.15.47:0/1803557879 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9f7409bbf0 con 0x7f9f84102b70 2024-03-21T11:22:59.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.830+0000 7f9f89a00700 1 -- 172.21.15.47:0/1803557879 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f840625d0 con 0x7f9f680a2fb0 2024-03-21T11:22:59.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.832+0000 7f9f80c00700 1 -- 172.21.15.47:0/1803557879 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f9f840625d0 con 0x7f9f680a2fb0 2024-03-21T11:22:59.834 INFO:teuthology.orchestra.run.smithi047.stdout:Backend: cephadm 2024-03-21T11:22:59.834 INFO:teuthology.orchestra.run.smithi047.stdout:Available: Yes 2024-03-21T11:22:59.834 INFO:teuthology.orchestra.run.smithi047.stdout:Paused: No 2024-03-21T11:22:59.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.835+0000 7f9f89a00700 1 -- 172.21.15.47:0/1803557879 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9f680a2fb0 msgr2=0x7f9f680a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.835+0000 7f9f89a00700 1 --2- 172.21.15.47:0/1803557879 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9f680a2fb0 0x7f9f680a5470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9f70007c90 tx=0x7f9f70007480 comp rx=0 tx=0).stop 2024-03-21T11:22:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.835+0000 7f9f89a00700 1 -- 172.21.15.47:0/1803557879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f84102b70 msgr2=0x7f9f8419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:22:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.835+0000 7f9f89a00700 1 --2- 172.21.15.47:0/1803557879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f84102b70 0x7f9f8419a4a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400d950 tx=0x7f9f7400dc60 comp rx=0 tx=0).stop 2024-03-21T11:22:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.836+0000 7f9f89a00700 1 -- 172.21.15.47:0/1803557879 shutdown_connections 2024-03-21T11:22:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.836+0000 7f9f89a00700 1 --2- 172.21.15.47:0/1803557879 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9f680a2fb0 0x7f9f680a5470 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.836+0000 7f9f89a00700 1 --2- 172.21.15.47:0/1803557879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f84107dd0 0x7f9f841a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.836+0000 7f9f89a00700 1 --2- 172.21.15.47:0/1803557879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f84102b70 0x7f9f8419a4a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.836+0000 7f9f89a00700 1 --2- 172.21.15.47:0/1803557879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f841054a0 0x7f9f8419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:22:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.836+0000 7f9f89a00700 1 -- 172.21.15.47:0/1803557879 >> 172.21.15.47:0/1803557879 conn(0x7f9f840fc560 msgr2=0x7f9f84102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:22:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.836+0000 7f9f89a00700 1 -- 172.21.15.47:0/1803557879 shutdown_connections 2024-03-21T11:22:59.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:22:59.836+0000 7f9f89a00700 1 -- 172.21.15.47:0/1803557879 wait complete. 2024-03-21T11:23:00.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-21T11:23:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:00 smithi100 ceph-mon[36245]: pgmap v390: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1002 B/s wr, 12 op/s 2024-03-21T11:23:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:00 smithi100 ceph-mon[36245]: from='client.25235 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:01.162 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:23:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:00 smithi102 ceph-mon[36748]: pgmap v390: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1002 B/s wr, 12 op/s 2024-03-21T11:23:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:00 smithi102 ceph-mon[36748]: from='client.25235 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:01.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:00 smithi047 ceph-mon[31351]: pgmap v390: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1002 B/s wr, 12 op/s 2024-03-21T11:23:01.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:00 smithi047 ceph-mon[31351]: from='client.25235 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:03.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:02 smithi047 ceph-mon[31351]: pgmap v391: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 903 B/s wr, 13 op/s 2024-03-21T11:23:03.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:02 smithi102 ceph-mon[36748]: pgmap v391: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 903 B/s wr, 13 op/s 2024-03-21T11:23:03.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.352+0000 7f4742200700 1 -- 172.21.15.47:0/1881733777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f473c102c50 msgr2=0x7f473c105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:03.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.352+0000 7f4742200700 1 --2- 172.21.15.47:0/1881733777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f473c102c50 0x7f473c105040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f472c009990 tx=0x7f472c009ca0 comp rx=0 tx=0).stop 2024-03-21T11:23:03.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.352+0000 7f4742200700 1 -- 172.21.15.47:0/1881733777 shutdown_connections 2024-03-21T11:23:03.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.352+0000 7f4742200700 1 --2- 172.21.15.47:0/1881733777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f473c107eb0 0x7f473c10a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:03.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.352+0000 7f4742200700 1 --2- 172.21.15.47:0/1881733777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f473c102c50 0x7f473c105040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:03.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.352+0000 7f4742200700 1 --2- 172.21.15.47:0/1881733777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c105580 0x7f473c107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:03.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.352+0000 7f4742200700 1 -- 172.21.15.47:0/1881733777 >> 172.21.15.47:0/1881733777 conn(0x7f473c0fc660 msgr2=0x7f473c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:03.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.353+0000 7f4742200700 1 -- 172.21.15.47:0/1881733777 shutdown_connections 2024-03-21T11:23:03.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.353+0000 7f4742200700 1 -- 172.21.15.47:0/1881733777 wait complete. 2024-03-21T11:23:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.354+0000 7f4742200700 1 Processor -- start 2024-03-21T11:23:03.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.354+0000 7f4742200700 1 -- start start 2024-03-21T11:23:03.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.354+0000 7f4742200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f473c102c50 0x7f473c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:03.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.354+0000 7f4742200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c105580 0x7f473c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:03.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.355+0000 7f4742200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f473c107eb0 0x7f473c1a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:03.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.355+0000 7f4742200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c10de20 con 0x7f473c102c50 2024-03-21T11:23:03.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.355+0000 7f4742200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c10dca0 con 0x7f473c105580 2024-03-21T11:23:03.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.355+0000 7f4742200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c10dfa0 con 0x7f473c107eb0 2024-03-21T11:23:03.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.355+0000 7f473be00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f473c107eb0 0x7f473c1a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:03.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.355+0000 7f473b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f473c102c50 0x7f473c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.355+0000 7f473be00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f473c107eb0 0x7f473c1a1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38746/0 (socket says 172.21.15.47:38746) 2024-03-21T11:23:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.355+0000 7f473be00700 1 -- 172.21.15.47:0/3266928165 learned_addr learned my addr 172.21.15.47:0/3266928165 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.355+0000 7f473aa00700 1 --2- 172.21.15.47:0/3266928165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c105580 0x7f473c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.355+0000 7f473be00700 1 -- 172.21.15.47:0/3266928165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c105580 msgr2=0x7f473c19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.355+0000 7f473be00700 1 --2- 172.21.15.47:0/3266928165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c105580 0x7f473c19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.355+0000 7f473be00700 1 -- 172.21.15.47:0/3266928165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f473c102c50 msgr2=0x7f473c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.355+0000 7f473be00700 1 --2- 172.21.15.47:0/3266928165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f473c102c50 0x7f473c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.355+0000 7f473be00700 1 -- 172.21.15.47:0/3266928165 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c1a2300 con 0x7f473c107eb0 2024-03-21T11:23:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.356+0000 7f473be00700 1 --2- 172.21.15.47:0/3266928165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f473c107eb0 0x7f473c1a1c60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f473400add0 tx=0x7f473400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.356+0000 7f4733e00700 1 -- 172.21.15.47:0/3266928165 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4734017020 con 0x7f473c107eb0 2024-03-21T11:23:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.356+0000 7f4733e00700 1 -- 172.21.15.47:0/3266928165 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f473400ae00 con 0x7f473c107eb0 2024-03-21T11:23:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.356+0000 7f4742200700 1 -- 172.21.15.47:0/3266928165 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473c1a25f0 con 0x7f473c107eb0 2024-03-21T11:23:03.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.356+0000 7f4733e00700 1 -- 172.21.15.47:0/3266928165 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47340053a0 con 0x7f473c107eb0 2024-03-21T11:23:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.356+0000 7f4742200700 1 -- 172.21.15.47:0/3266928165 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c1a2b30 con 0x7f473c107eb0 2024-03-21T11:23:03.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.358+0000 7f4742200700 1 -- 172.21.15.47:0/3266928165 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473c068e50 con 0x7f473c107eb0 2024-03-21T11:23:03.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.359+0000 7f4733e00700 1 -- 172.21.15.47:0/3266928165 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f473400c890 con 0x7f473c107eb0 2024-03-21T11:23:03.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.361+0000 7f4733e00700 1 --2- 172.21.15.47:0/3266928165 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47200a3030 0x7f47200a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:03.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.361+0000 7f4733e00700 1 -- 172.21.15.47:0/3266928165 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f47340cede0 con 0x7f473c107eb0 2024-03-21T11:23:03.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.361+0000 7f473b400700 1 --2- 172.21.15.47:0/3266928165 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47200a3030 0x7f47200a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:03.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.362+0000 7f473b400700 1 --2- 172.21.15.47:0/3266928165 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47200a3030 0x7f47200a54f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f472c00b7b0 tx=0x7f472c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:03.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.365+0000 7f4733e00700 1 -- 172.21.15.47:0/3266928165 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4734021020 con 0x7f473c107eb0 2024-03-21T11:23:03.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:02 smithi100 ceph-mon[36245]: pgmap v391: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 903 B/s wr, 13 op/s 2024-03-21T11:23:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.601+0000 7f4742200700 1 -- 172.21.15.47:0/3266928165 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f473c0625d0 con 0x7f47200a3030 2024-03-21T11:23:03.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.643+0000 7f4733e00700 1 -- 172.21.15.47:0/3266928165 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4123 (secure 0 0 0) 0x7f473c0625d0 con 0x7f47200a3030 2024-03-21T11:23:03.645 INFO:teuthology.orchestra.run.smithi047.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T11:23:03.645 INFO:teuthology.orchestra.run.smithi047.stdout:mds.a smithi047 running (2m) 29s ago 2m 15.2M - 19.0.0-2231-g2edfee68 237e43c7a65f 0463adc15a5c 2024-03-21T11:23:03.645 INFO:teuthology.orchestra.run.smithi047.stdout:mds.b smithi100 running (2m) 72s ago 2m 26.4M - 19.0.0-2231-g2edfee68 237e43c7a65f 3fa90a15426b 2024-03-21T11:23:03.645 INFO:teuthology.orchestra.run.smithi047.stdout:mds.c smithi102 running (2m) 72s ago 2m 19.5M - 19.0.0-2231-g2edfee68 237e43c7a65f 7eee422f1464 2024-03-21T11:23:03.645 INFO:teuthology.orchestra.run.smithi047.stdout:mds.d smithi047 running (2m) 29s ago 2m 15.2M - 19.0.0-2231-g2edfee68 237e43c7a65f f6f4b37d470d 2024-03-21T11:23:03.645 INFO:teuthology.orchestra.run.smithi047.stdout:mds.e smithi100 running (2m) 72s ago 2m 19.6M - 19.0.0-2231-g2edfee68 237e43c7a65f d771cc7b5d6f 2024-03-21T11:23:03.645 INFO:teuthology.orchestra.run.smithi047.stdout:mds.f smithi102 running (2m) 72s ago 2m 18.4M - 19.0.0-2231-g2edfee68 237e43c7a65f f8203788be54 2024-03-21T11:23:03.645 INFO:teuthology.orchestra.run.smithi047.stdout:mds.g smithi047 running (2m) 29s ago 2m 12.4M - 19.0.0-2231-g2edfee68 237e43c7a65f d14b154447d8 2024-03-21T11:23:03.645 INFO:teuthology.orchestra.run.smithi047.stdout:mds.h smithi100 running (2m) 72s ago 2m 16.4M - 19.0.0-2231-g2edfee68 237e43c7a65f 987da945c2f1 2024-03-21T11:23:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:mds.i smithi102 running (2m) 72s ago 2m 16.4M - 19.0.0-2231-g2edfee68 237e43c7a65f 19a552d32b82 2024-03-21T11:23:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:mds.j smithi047 running (2m) 29s ago 2m 12.5M - 19.0.0-2231-g2edfee68 237e43c7a65f 8a6c3bc78d0f 2024-03-21T11:23:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:mds.k smithi100 running (2m) 72s ago 2m 16.1M - 19.0.0-2231-g2edfee68 237e43c7a65f 17c5e3dda4ad 2024-03-21T11:23:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:mds.l smithi102 running (2m) 72s ago 2m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f caa307b0576b 2024-03-21T11:23:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.x smithi047 *:9283,8765 running (11m) 29s ago 11m 515M - 19.0.0-2231-g2edfee68 237e43c7a65f 34a3d6ac0ad5 2024-03-21T11:23:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.y smithi100 *:8443,8765 running (8m) 72s ago 8m 453M - 19.0.0-2231-g2edfee68 237e43c7a65f bdc77f29aa77 2024-03-21T11:23:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.z smithi102 *:8443,8765 running (8m) 72s ago 8m 453M - 19.0.0-2231-g2edfee68 237e43c7a65f a64d8e76571a 2024-03-21T11:23:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:mon.a smithi047 running (11m) 29s ago 11m 48.5M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f af33427a459a 2024-03-21T11:23:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:mon.b smithi100 running (9m) 72s ago 9m 41.6M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 5ce4875e32d6 2024-03-21T11:23:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:mon.c smithi102 running (9m) 72s ago 9m 42.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f a078e0244360 2024-03-21T11:23:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0 smithi047 running (8m) 29s ago 8m 45.2M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f 8c62707ad3ec 2024-03-21T11:23:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1 smithi100 running (8m) 72s ago 8m 57.8M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f ba975c2a84e7 2024-03-21T11:23:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2 smithi102 running (7m) 72s ago 7m 57.9M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 62adc2cf2f9e 2024-03-21T11:23:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3 smithi047 running (7m) 29s ago 7m 43.1M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f 8418de722bbb 2024-03-21T11:23:03.647 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4 smithi100 running (6m) 72s ago 6m 45.7M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 1fe6513c03be 2024-03-21T11:23:03.647 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5 smithi102 running (6m) 72s ago 6m 47.2M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 27fc6e14ed65 2024-03-21T11:23:03.647 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6 smithi047 running (5m) 29s ago 5m 51.9M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f 038d97750d0e 2024-03-21T11:23:03.647 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7 smithi100 running (5m) 72s ago 5m 55.1M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 2f005f201338 2024-03-21T11:23:03.647 INFO:teuthology.orchestra.run.smithi047.stdout:osd.8 smithi102 running (4m) 72s ago 4m 53.3M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 9cabba3f6551 2024-03-21T11:23:03.647 INFO:teuthology.orchestra.run.smithi047.stdout:osd.9 smithi047 running (4m) 29s ago 4m 52.9M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f 6debc0f9d05a 2024-03-21T11:23:03.647 INFO:teuthology.orchestra.run.smithi047.stdout:osd.10 smithi100 running (3m) 72s ago 3m 53.3M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f a397990779ce 2024-03-21T11:23:03.647 INFO:teuthology.orchestra.run.smithi047.stdout:osd.11 smithi102 running (3m) 72s ago 3m 50.4M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 002b68a8f12c 2024-03-21T11:23:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.646+0000 7f4742200700 1 -- 172.21.15.47:0/3266928165 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47200a3030 msgr2=0x7f47200a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.646+0000 7f4742200700 1 --2- 172.21.15.47:0/3266928165 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47200a3030 0x7f47200a54f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f472c00b7b0 tx=0x7f472c005fd0 comp rx=0 tx=0).stop 2024-03-21T11:23:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.646+0000 7f4742200700 1 -- 172.21.15.47:0/3266928165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f473c107eb0 msgr2=0x7f473c1a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.646+0000 7f4742200700 1 --2- 172.21.15.47:0/3266928165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f473c107eb0 0x7f473c1a1c60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f473400add0 tx=0x7f473400c330 comp rx=0 tx=0).stop 2024-03-21T11:23:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.647+0000 7f4742200700 1 -- 172.21.15.47:0/3266928165 shutdown_connections 2024-03-21T11:23:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.647+0000 7f4742200700 1 --2- 172.21.15.47:0/3266928165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f473c107eb0 0x7f473c1a1c60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.647+0000 7f4742200700 1 --2- 172.21.15.47:0/3266928165 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47200a3030 0x7f47200a54f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.647+0000 7f4742200700 1 --2- 172.21.15.47:0/3266928165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f473c102c50 0x7f473c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.647+0000 7f4742200700 1 --2- 172.21.15.47:0/3266928165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f473c105580 0x7f473c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:03.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.647+0000 7f4742200700 1 -- 172.21.15.47:0/3266928165 >> 172.21.15.47:0/3266928165 conn(0x7f473c0fc660 msgr2=0x7f473c102370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:03.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.647+0000 7f4742200700 1 -- 172.21.15.47:0/3266928165 shutdown_connections 2024-03-21T11:23:03.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:03.647+0000 7f4742200700 1 -- 172.21.15.47:0/3266928165 wait complete. 2024-03-21T11:23:04.370 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-21T11:23:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:23:05.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:04 smithi047 ceph-mon[31351]: pgmap v392: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-21T11:23:05.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:04 smithi047 ceph-mon[31351]: from='client.25252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:04 smithi102 ceph-mon[36748]: pgmap v392: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-21T11:23:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:04 smithi102 ceph-mon[36748]: from='client.25252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:05.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:04 smithi100 ceph-mon[36245]: pgmap v392: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-21T11:23:05.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:04 smithi100 ceph-mon[36245]: from='client.25252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.952+0000 7fb596c00700 1 -- 172.21.15.47:0/2172647611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590102c50 msgr2=0x7fb590105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.952+0000 7fb596c00700 1 --2- 172.21.15.47:0/2172647611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590102c50 0x7fb590105040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb580009990 tx=0x7fb580009ca0 comp rx=0 tx=0).stop 2024-03-21T11:23:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.952+0000 7fb596c00700 1 -- 172.21.15.47:0/2172647611 shutdown_connections 2024-03-21T11:23:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.952+0000 7fb596c00700 1 --2- 172.21.15.47:0/2172647611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb590107eb0 0x7fb59010a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.952+0000 7fb596c00700 1 --2- 172.21.15.47:0/2172647611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590102c50 0x7fb590105040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.952+0000 7fb596c00700 1 --2- 172.21.15.47:0/2172647611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb590105580 0x7fb590107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.952+0000 7fb596c00700 1 -- 172.21.15.47:0/2172647611 >> 172.21.15.47:0/2172647611 conn(0x7fb5900fc660 msgr2=0x7fb5900feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.952+0000 7fb596c00700 1 -- 172.21.15.47:0/2172647611 shutdown_connections 2024-03-21T11:23:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.952+0000 7fb596c00700 1 -- 172.21.15.47:0/2172647611 wait complete. 2024-03-21T11:23:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.953+0000 7fb596c00700 1 Processor -- start 2024-03-21T11:23:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.954+0000 7fb596c00700 1 -- start start 2024-03-21T11:23:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.954+0000 7fb596c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590102c50 0x7fb5901025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.954+0000 7fb596c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb590105580 0x7fb590100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.954+0000 7fb596c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb590107eb0 0x7fb590101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.954+0000 7fb596c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59010de20 con 0x7fb590105580 2024-03-21T11:23:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.954+0000 7fb596c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59010dca0 con 0x7fb590107eb0 2024-03-21T11:23:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.954+0000 7fb596c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59010dfa0 con 0x7fb590102c50 2024-03-21T11:23:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.954+0000 7fb58fe00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590102c50 0x7fb5901025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.954+0000 7fb58fe00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590102c50 0x7fb5901025b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38766/0 (socket says 172.21.15.47:38766) 2024-03-21T11:23:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.954+0000 7fb58fe00700 1 -- 172.21.15.47:0/1175639294 learned_addr learned my addr 172.21.15.47:0/1175639294 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.954+0000 7fb587e00700 1 --2- 172.21.15.47:0/1175639294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb590105580 0x7fb590100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.954+0000 7fb594c00700 1 --2- 172.21.15.47:0/1175639294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb590107eb0 0x7fb590101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.955+0000 7fb58fe00700 1 -- 172.21.15.47:0/1175639294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb590107eb0 msgr2=0x7fb590101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.955+0000 7fb58fe00700 1 --2- 172.21.15.47:0/1175639294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb590107eb0 0x7fb590101140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.955+0000 7fb58fe00700 1 -- 172.21.15.47:0/1175639294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb590105580 msgr2=0x7fb590100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.955+0000 7fb58fe00700 1 --2- 172.21.15.47:0/1175639294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb590105580 0x7fb590100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.955+0000 7fb58fe00700 1 -- 172.21.15.47:0/1175639294 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5901019f0 con 0x7fb590102c50 2024-03-21T11:23:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.955+0000 7fb58fe00700 1 --2- 172.21.15.47:0/1175639294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590102c50 0x7fb5901025b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb580005910 tx=0x7fb58000bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.955+0000 7fb58d600700 1 -- 172.21.15.47:0/1175639294 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580026070 con 0x7fb590102c50 2024-03-21T11:23:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.955+0000 7fb58d600700 1 -- 172.21.15.47:0/1175639294 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb580003d70 con 0x7fb590102c50 2024-03-21T11:23:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.955+0000 7fb58d600700 1 -- 172.21.15.47:0/1175639294 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580021410 con 0x7fb590102c50 2024-03-21T11:23:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.955+0000 7fb596c00700 1 -- 172.21.15.47:0/1175639294 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5901a2570 con 0x7fb590102c50 2024-03-21T11:23:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.955+0000 7fb596c00700 1 -- 172.21.15.47:0/1175639294 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5901a2970 con 0x7fb590102c50 2024-03-21T11:23:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.956+0000 7fb596c00700 1 -- 172.21.15.47:0/1175639294 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb590068e50 con 0x7fb590102c50 2024-03-21T11:23:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.958+0000 7fb58d600700 1 -- 172.21.15.47:0/1175639294 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb580029070 con 0x7fb590102c50 2024-03-21T11:23:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.960+0000 7fb58d600700 1 --2- 172.21.15.47:0/1175639294 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5740a2fb0 0x7fb5740a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.960+0000 7fb58d600700 1 -- 172.21.15.47:0/1175639294 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb5800d04c0 con 0x7fb590102c50 2024-03-21T11:23:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.960+0000 7fb587e00700 1 --2- 172.21.15.47:0/1175639294 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5740a2fb0 0x7fb5740a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.961+0000 7fb587e00700 1 --2- 172.21.15.47:0/1175639294 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5740a2fb0 0x7fb5740a5470 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb578009c60 tx=0x7fb578009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:06.963+0000 7fb58d600700 1 -- 172.21.15.47:0/1175639294 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb580013070 con 0x7fb590102c50 2024-03-21T11:23:07.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:06 smithi047 ceph-mon[31351]: pgmap v393: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T11:23:07.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:07.190+0000 7fb596c00700 1 -- 172.21.15.47:0/1175639294 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5900625d0 con 0x7fb5740a2fb0 2024-03-21T11:23:07.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:07.207+0000 7fb58d600700 1 -- 172.21.15.47:0/1175639294 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7fb5900625d0 con 0x7fb5740a2fb0 2024-03-21T11:23:07.208 INFO:teuthology.orchestra.run.smithi047.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-21T11:23:07.209 INFO:teuthology.orchestra.run.smithi047.stdout:mds.all 12/12 76s ago 3m smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi102=c;smithi102=f;smithi102=i;smithi102=l;count:12 2024-03-21T11:23:07.209 INFO:teuthology.orchestra.run.smithi047.stdout:mgr 3/3 76s ago 9m smithi047=x;smithi100=y;smithi102=z;count:3 2024-03-21T11:23:07.209 INFO:teuthology.orchestra.run.smithi047.stdout:mon 3/3 76s ago 9m smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi102:172.21.15.102=c;count:3 2024-03-21T11:23:07.209 INFO:teuthology.orchestra.run.smithi047.stdout:osd 12 76s ago - 2024-03-21T11:23:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:07.210+0000 7fb596c00700 1 -- 172.21.15.47:0/1175639294 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5740a2fb0 msgr2=0x7fb5740a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:07.210+0000 7fb596c00700 1 --2- 172.21.15.47:0/1175639294 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5740a2fb0 0x7fb5740a5470 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb578009c60 tx=0x7fb578009260 comp rx=0 tx=0).stop 2024-03-21T11:23:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:07.210+0000 7fb596c00700 1 -- 172.21.15.47:0/1175639294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590102c50 msgr2=0x7fb5901025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:07.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:07.210+0000 7fb596c00700 1 --2- 172.21.15.47:0/1175639294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590102c50 0x7fb5901025b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb580005910 tx=0x7fb58000bbb0 comp rx=0 tx=0).stop 2024-03-21T11:23:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:07.210+0000 7fb596c00700 1 -- 172.21.15.47:0/1175639294 shutdown_connections 2024-03-21T11:23:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:07.210+0000 7fb596c00700 1 --2- 172.21.15.47:0/1175639294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb590107eb0 0x7fb590101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:07.210+0000 7fb596c00700 1 --2- 172.21.15.47:0/1175639294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590102c50 0x7fb5901025b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:07.210+0000 7fb596c00700 1 --2- 172.21.15.47:0/1175639294 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5740a2fb0 0x7fb5740a5470 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:07.210+0000 7fb596c00700 1 --2- 172.21.15.47:0/1175639294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb590105580 0x7fb590100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:07.210+0000 7fb596c00700 1 -- 172.21.15.47:0/1175639294 >> 172.21.15.47:0/1175639294 conn(0x7fb5900fc660 msgr2=0x7fb5901034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:07.211+0000 7fb596c00700 1 -- 172.21.15.47:0/1175639294 shutdown_connections 2024-03-21T11:23:07.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:07.211+0000 7fb596c00700 1 -- 172.21.15.47:0/1175639294 wait complete. 2024-03-21T11:23:07.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:06 smithi102 ceph-mon[36748]: pgmap v393: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T11:23:07.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:06 smithi100 ceph-mon[36245]: pgmap v393: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T11:23:08.028 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-21T11:23:08.503 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:23:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:08 smithi102 ceph-mon[36748]: pgmap v394: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:23:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:08 smithi102 ceph-mon[36748]: from='client.25258 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:09.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:08 smithi047 ceph-mon[31351]: pgmap v394: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:23:09.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:08 smithi047 ceph-mon[31351]: from='client.25258 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:09.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:08 smithi100 ceph-mon[36245]: pgmap v394: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:23:09.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:08 smithi100 ceph-mon[36245]: from='client.25258 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:10.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.698+0000 7fd8b5c00700 1 -- 172.21.15.47:0/4145775476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8b0108810 msgr2=0x7fd8b0108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:10.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.698+0000 7fd8b5c00700 1 --2- 172.21.15.47:0/4145775476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8b0108810 0x7fd8b0108c30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00099c0 tx=0x7fd8a0009cd0 comp rx=0 tx=0).stop 2024-03-21T11:23:10.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.698+0000 7fd8b5c00700 1 -- 172.21.15.47:0/4145775476 shutdown_connections 2024-03-21T11:23:10.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.698+0000 7fd8b5c00700 1 --2- 172.21.15.47:0/4145775476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8b010a3b0 0x7fd8b0110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:10.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.698+0000 7fd8b5c00700 1 --2- 172.21.15.47:0/4145775476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8b0108810 0x7fd8b0108c30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.698+0000 7fd8b5c00700 1 --2- 172.21.15.47:0/4145775476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b0109a10 0x7fd8b0109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.698+0000 7fd8b5c00700 1 -- 172.21.15.47:0/4145775476 >> 172.21.15.47:0/4145775476 conn(0x7fd8b00777e0 msgr2=0x7fd8b0079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.698+0000 7fd8b5c00700 1 -- 172.21.15.47:0/4145775476 shutdown_connections 2024-03-21T11:23:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.698+0000 7fd8b5c00700 1 -- 172.21.15.47:0/4145775476 wait complete. 2024-03-21T11:23:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.699+0000 7fd8b5c00700 1 Processor -- start 2024-03-21T11:23:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.699+0000 7fd8b5c00700 1 -- start start 2024-03-21T11:23:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.699+0000 7fd8b5c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8b0108810 0x7fd8b019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8b5c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b0109a10 0x7fd8b019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8b5c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8b010a3b0 0x7fd8b01a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8b5c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0114420 con 0x7fd8b0109a10 2024-03-21T11:23:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8b5c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b01142a0 con 0x7fd8b010a3b0 2024-03-21T11:23:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8b5c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b01145a0 con 0x7fd8b0108810 2024-03-21T11:23:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8aea00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8b0108810 0x7fd8b019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8aea00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8b0108810 0x7fd8b019ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59300/0 (socket says 172.21.15.47:59300) 2024-03-21T11:23:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8aea00700 1 -- 172.21.15.47:0/319055759 learned_addr learned my addr 172.21.15.47:0/319055759 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8ae000700 1 --2- 172.21.15.47:0/319055759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b0109a10 0x7fd8b019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8af400700 1 --2- 172.21.15.47:0/319055759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8b010a3b0 0x7fd8b01a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8aea00700 1 -- 172.21.15.47:0/319055759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8b010a3b0 msgr2=0x7fd8b01a5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8aea00700 1 --2- 172.21.15.47:0/319055759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8b010a3b0 0x7fd8b01a5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8aea00700 1 -- 172.21.15.47:0/319055759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b0109a10 msgr2=0x7fd8b019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8aea00700 1 --2- 172.21.15.47:0/319055759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b0109a10 0x7fd8b019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.700+0000 7fd8aea00700 1 -- 172.21.15.47:0/319055759 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b01a6660 con 0x7fd8b0108810 2024-03-21T11:23:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.701+0000 7fd8aea00700 1 --2- 172.21.15.47:0/319055759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8b0108810 0x7fd8b019ea00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000bee0 tx=0x7fd8a000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.701+0000 7fd8a7400700 1 -- 172.21.15.47:0/319055759 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0009e60 con 0x7fd8b0108810 2024-03-21T11:23:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.701+0000 7fd8a7400700 1 -- 172.21.15.47:0/319055759 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8a0003e30 con 0x7fd8b0108810 2024-03-21T11:23:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.701+0000 7fd8a7400700 1 -- 172.21.15.47:0/319055759 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00173f0 con 0x7fd8b0108810 2024-03-21T11:23:10.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.701+0000 7fd8b5c00700 1 -- 172.21.15.47:0/319055759 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b01a68f0 con 0x7fd8b0108810 2024-03-21T11:23:10.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.701+0000 7fd8b5c00700 1 -- 172.21.15.47:0/319055759 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b01a6da0 con 0x7fd8b0108810 2024-03-21T11:23:10.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.702+0000 7fd8b5c00700 1 -- 172.21.15.47:0/319055759 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b0068e50 con 0x7fd8b0108810 2024-03-21T11:23:10.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.704+0000 7fd8a7400700 1 -- 172.21.15.47:0/319055759 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd8a000f660 con 0x7fd8b0108810 2024-03-21T11:23:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.706+0000 7fd8a7400700 1 --2- 172.21.15.47:0/319055759 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8940a3030 0x7fd8940a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.706+0000 7fd8a7400700 1 -- 172.21.15.47:0/319055759 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd8a00d0510 con 0x7fd8b0108810 2024-03-21T11:23:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.706+0000 7fd8ae000700 1 --2- 172.21.15.47:0/319055759 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8940a3030 0x7fd8940a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.707+0000 7fd8ae000700 1 --2- 172.21.15.47:0/319055759 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8940a3030 0x7fd8940a54f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009c60 tx=0x7fd89c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.712+0000 7fd8a7400700 1 -- 172.21.15.47:0/319055759 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd8a0013070 con 0x7fd8b0108810 2024-03-21T11:23:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.951+0000 7fd8b5c00700 1 -- 172.21.15.47:0/319055759 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8b00625d0 con 0x7fd8940a3030 2024-03-21T11:23:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.954+0000 7fd8a7400700 1 -- 172.21.15.47:0/319055759 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fd8b00625d0 con 0x7fd8940a3030 2024-03-21T11:23:10.955 INFO:teuthology.orchestra.run.smithi047.stdout:HOST ADDR LABELS STATUS 2024-03-21T11:23:10.955 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 172.21.15.47 2024-03-21T11:23:10.955 INFO:teuthology.orchestra.run.smithi047.stdout:smithi100 172.21.15.100 2024-03-21T11:23:10.956 INFO:teuthology.orchestra.run.smithi047.stdout:smithi102 172.21.15.102 2024-03-21T11:23:10.956 INFO:teuthology.orchestra.run.smithi047.stdout:3 hosts in cluster 2024-03-21T11:23:10.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.957+0000 7fd8b5c00700 1 -- 172.21.15.47:0/319055759 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8940a3030 msgr2=0x7fd8940a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:10.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.957+0000 7fd8b5c00700 1 --2- 172.21.15.47:0/319055759 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8940a3030 0x7fd8940a54f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009c60 tx=0x7fd89c009260 comp rx=0 tx=0).stop 2024-03-21T11:23:10.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.957+0000 7fd8b5c00700 1 -- 172.21.15.47:0/319055759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8b0108810 msgr2=0x7fd8b019ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:10.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.957+0000 7fd8b5c00700 1 --2- 172.21.15.47:0/319055759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8b0108810 0x7fd8b019ea00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000bee0 tx=0x7fd8a000bf10 comp rx=0 tx=0).stop 2024-03-21T11:23:10.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.957+0000 7fd8b5c00700 1 -- 172.21.15.47:0/319055759 shutdown_connections 2024-03-21T11:23:10.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.957+0000 7fd8b5c00700 1 --2- 172.21.15.47:0/319055759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8b010a3b0 0x7fd8b01a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:10.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.957+0000 7fd8b5c00700 1 --2- 172.21.15.47:0/319055759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8b0108810 0x7fd8b019ea00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:10.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.957+0000 7fd8b5c00700 1 --2- 172.21.15.47:0/319055759 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8940a3030 0x7fd8940a54f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:10.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.957+0000 7fd8b5c00700 1 --2- 172.21.15.47:0/319055759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8b0109a10 0x7fd8b019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:10.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.957+0000 7fd8b5c00700 1 -- 172.21.15.47:0/319055759 >> 172.21.15.47:0/319055759 conn(0x7fd8b00777e0 msgr2=0x7fd8b010cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:10.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.958+0000 7fd8b5c00700 1 -- 172.21.15.47:0/319055759 shutdown_connections 2024-03-21T11:23:10.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:10.958+0000 7fd8b5c00700 1 -- 172.21.15.47:0/319055759 wait complete. 2024-03-21T11:23:11.194 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:10 smithi100 ceph-mon[36245]: pgmap v395: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:23:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:10 smithi102 ceph-mon[36748]: pgmap v395: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:23:11.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:10 smithi047 ceph-mon[31351]: pgmap v395: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:23:11.757 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-21T11:23:12.237 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:23:13.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:12 smithi047 ceph-mon[31351]: from='client.25264 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:13.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:12 smithi047 ceph-mon[31351]: pgmap v396: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:23:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:12 smithi102 ceph-mon[36748]: from='client.25264 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:12 smithi102 ceph-mon[36748]: pgmap v396: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:23:13.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:12 smithi100 ceph-mon[36245]: from='client.25264 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:13.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:12 smithi100 ceph-mon[36245]: pgmap v396: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:23:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.489+0000 7f6209400700 1 -- 172.21.15.47:0/2259459670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6204109900 msgr2=0x7f6204109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.489+0000 7f6209400700 1 --2- 172.21.15.47:0/2259459670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6204109900 0x7f6204109d60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009990 tx=0x7f61ec009ca0 comp rx=0 tx=0).stop 2024-03-21T11:23:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.489+0000 7f6209400700 1 -- 172.21.15.47:0/2259459670 shutdown_connections 2024-03-21T11:23:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.489+0000 7f6209400700 1 --2- 172.21.15.47:0/2259459670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f620410a2a0 0x7f6204110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.489+0000 7f6209400700 1 --2- 172.21.15.47:0/2259459670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6204108700 0x7f6204108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.489+0000 7f6209400700 1 --2- 172.21.15.47:0/2259459670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6204109900 0x7f6204109d60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.489+0000 7f6209400700 1 -- 172.21.15.47:0/2259459670 >> 172.21.15.47:0/2259459670 conn(0x7f62040776d0 msgr2=0x7f6204079b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.489+0000 7f6209400700 1 -- 172.21.15.47:0/2259459670 shutdown_connections 2024-03-21T11:23:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.489+0000 7f6209400700 1 -- 172.21.15.47:0/2259459670 wait complete. 2024-03-21T11:23:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.491+0000 7f6209400700 1 Processor -- start 2024-03-21T11:23:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.491+0000 7f6209400700 1 -- start start 2024-03-21T11:23:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.492+0000 7f6209400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6204108700 0x7f620419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.492+0000 7f6209400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6204109900 0x7f620419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.492+0000 7f6209400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f620410a2a0 0x7f62041a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.492+0000 7f6209400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62041142c0 con 0x7f6204109900 2024-03-21T11:23:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.492+0000 7f6209400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6204114140 con 0x7f6204108700 2024-03-21T11:23:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.492+0000 7f6209400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6204114440 con 0x7f620410a2a0 2024-03-21T11:23:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.492+0000 7f6203e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6204108700 0x7f620419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.492+0000 7f6203e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6204108700 0x7f620419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47410/0 (socket says 172.21.15.47:47410) 2024-03-21T11:23:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.492+0000 7f61fb400700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6204109900 0x7f620419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.492+0000 7f6203e00700 1 -- 172.21.15.47:0/3538615390 learned_addr learned my addr 172.21.15.47:0/3538615390 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.492+0000 7f6208a00700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f620410a2a0 0x7f62041a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:14.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.493+0000 7f6203e00700 1 -- 172.21.15.47:0/3538615390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f620410a2a0 msgr2=0x7f62041a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:14.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.493+0000 7f6203e00700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f620410a2a0 0x7f62041a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:14.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.493+0000 7f6203e00700 1 -- 172.21.15.47:0/3538615390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6204109900 msgr2=0x7f620419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:14.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.493+0000 7f6203e00700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6204109900 0x7f620419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:14.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.493+0000 7f6203e00700 1 -- 172.21.15.47:0/3538615390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62041a6510 con 0x7f6204108700 2024-03-21T11:23:14.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.493+0000 7f61fb400700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6204109900 0x7f620419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.493+0000 7f6208a00700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f620410a2a0 0x7f62041a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.493+0000 7f6203e00700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6204108700 0x7f620419e8b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f61f400c140 tx=0x7f61f400c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.493+0000 7f6201600700 1 -- 172.21.15.47:0/3538615390 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f400ccd0 con 0x7f6204108700 2024-03-21T11:23:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.493+0000 7f6201600700 1 -- 172.21.15.47:0/3538615390 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f61f4018070 con 0x7f6204108700 2024-03-21T11:23:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.493+0000 7f6201600700 1 -- 172.21.15.47:0/3538615390 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f40136f0 con 0x7f6204108700 2024-03-21T11:23:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.493+0000 7f6209400700 1 -- 172.21.15.47:0/3538615390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62041a67a0 con 0x7f6204108700 2024-03-21T11:23:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.493+0000 7f6209400700 1 -- 172.21.15.47:0/3538615390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62041a6c90 con 0x7f6204108700 2024-03-21T11:23:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.494+0000 7f6209400700 1 -- 172.21.15.47:0/3538615390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6204068e50 con 0x7f6204108700 2024-03-21T11:23:14.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.496+0000 7f6201600700 1 -- 172.21.15.47:0/3538615390 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f61f4012070 con 0x7f6204108700 2024-03-21T11:23:14.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.498+0000 7f6201600700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f61e80a3030 0x7f61e80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:14.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.498+0000 7f6201600700 1 -- 172.21.15.47:0/3538615390 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f61f40cf240 con 0x7f6204108700 2024-03-21T11:23:14.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.498+0000 7f61fb400700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f61e80a3030 0x7f61e80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:14.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.499+0000 7f61fb400700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f61e80a3030 0x7f61e80a54f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00b2e0 tx=0x7f61ec00b8f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:14.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.502+0000 7f6201600700 1 -- 172.21.15.47:0/3538615390 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f61f4022020 con 0x7f6204108700 2024-03-21T11:23:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.738+0000 7f6209400700 1 -- 172.21.15.47:0/3538615390 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f62040625d0 con 0x7f61e80a3030 2024-03-21T11:23:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.741+0000 7f6201600700 1 -- 172.21.15.47:0/3538615390 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f62040625d0 con 0x7f61e80a3030 2024-03-21T11:23:14.742 INFO:teuthology.orchestra.run.smithi047.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-21T11:23:14.742 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130022480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-03-21T11:23:14.742 INFO:teuthology.orchestra.run.smithi047.stdout:smithi100 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-21T11:23:14.742 INFO:teuthology.orchestra.run.smithi047.stdout:smithi102 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-21T11:23:14.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.744+0000 7f6209400700 1 -- 172.21.15.47:0/3538615390 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f61e80a3030 msgr2=0x7f61e80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:14.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.744+0000 7f6209400700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f61e80a3030 0x7f61e80a54f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f61ec00b2e0 tx=0x7f61ec00b8f0 comp rx=0 tx=0).stop 2024-03-21T11:23:14.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.744+0000 7f6209400700 1 -- 172.21.15.47:0/3538615390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6204108700 msgr2=0x7f620419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:14.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.744+0000 7f6209400700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6204108700 0x7f620419e8b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f61f400c140 tx=0x7f61f400c450 comp rx=0 tx=0).stop 2024-03-21T11:23:14.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.744+0000 7f6209400700 1 -- 172.21.15.47:0/3538615390 shutdown_connections 2024-03-21T11:23:14.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.744+0000 7f6209400700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f620410a2a0 0x7f62041a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:14.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.744+0000 7f6209400700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6204108700 0x7f620419e8b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.744+0000 7f6209400700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f61e80a3030 0x7f61e80a54f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.744+0000 7f6209400700 1 --2- 172.21.15.47:0/3538615390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6204109900 0x7f620419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.744+0000 7f6209400700 1 -- 172.21.15.47:0/3538615390 >> 172.21.15.47:0/3538615390 conn(0x7f62040776d0 msgr2=0x7f620410cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.745+0000 7f6209400700 1 -- 172.21.15.47:0/3538615390 shutdown_connections 2024-03-21T11:23:14.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:14.745+0000 7f6209400700 1 -- 172.21.15.47:0/3538615390 wait complete. 2024-03-21T11:23:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:14 smithi047 ceph-mon[31351]: pgmap v397: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:23:15.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:14 smithi102 ceph-mon[36748]: pgmap v397: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:23:15.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:14 smithi100 ceph-mon[36245]: pgmap v397: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:23:15.598 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T11:23:15.608 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T11:23:15.609 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-21T11:23:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:23:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:15 smithi102 ceph-mon[36748]: from='client.25259 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:16.250 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:15 smithi047 ceph-mon[31351]: from='client.25259 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:16.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:15 smithi100 ceph-mon[36245]: from='client.25259 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:16 smithi102 ceph-mon[36748]: pgmap v398: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:23:17.281 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:16 smithi047 ceph-mon[31351]: pgmap v398: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:23:17.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:16 smithi100 ceph-mon[36245]: pgmap v398: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:23:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.289+0000 7f352f200700 1 -- 172.21.15.47:0/2943371736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3528105580 msgr2=0x7f3528107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.289+0000 7f352f200700 1 --2- 172.21.15.47:0/2943371736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3528105580 0x7f3528107970 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3510009990 tx=0x7f3510009ca0 comp rx=0 tx=0).stop 2024-03-21T11:23:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.289+0000 7f352f200700 1 -- 172.21.15.47:0/2943371736 shutdown_connections 2024-03-21T11:23:18.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.289+0000 7f352f200700 1 --2- 172.21.15.47:0/2943371736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3528107eb0 0x7f352810a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:18.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.289+0000 7f352f200700 1 --2- 172.21.15.47:0/2943371736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3528102c50 0x7f3528105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:18.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.289+0000 7f352f200700 1 --2- 172.21.15.47:0/2943371736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3528105580 0x7f3528107970 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:18.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.289+0000 7f352f200700 1 -- 172.21.15.47:0/2943371736 >> 172.21.15.47:0/2943371736 conn(0x7f35280fc660 msgr2=0x7f35280feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:18.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.289+0000 7f352f200700 1 -- 172.21.15.47:0/2943371736 shutdown_connections 2024-03-21T11:23:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.289+0000 7f352f200700 1 -- 172.21.15.47:0/2943371736 wait complete. 2024-03-21T11:23:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.291+0000 7f352f200700 1 Processor -- start 2024-03-21T11:23:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.291+0000 7f352f200700 1 -- start start 2024-03-21T11:23:18.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.292+0000 7f352f200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3528102c50 0x7f352819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:18.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.292+0000 7f352f200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3528105580 0x7f352819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.292+0000 7f352f200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3528107eb0 0x7f35281a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.292+0000 7f352f200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352810ddf0 con 0x7f3528105580 2024-03-21T11:23:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.292+0000 7f352f200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352810dc70 con 0x7f3528107eb0 2024-03-21T11:23:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.292+0000 7f352f200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352810df70 con 0x7f3528102c50 2024-03-21T11:23:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.292+0000 7f3527e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3528102c50 0x7f352819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.292+0000 7f3527e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3528102c50 0x7f352819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59338/0 (socket says 172.21.15.47:59338) 2024-03-21T11:23:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.292+0000 7f3527e00700 1 -- 172.21.15.47:0/1862896878 learned_addr learned my addr 172.21.15.47:0/1862896878 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.292+0000 7f351fe00700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3528105580 0x7f352819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.292+0000 7f352d200700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3528107eb0 0x7f35281a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.293+0000 7f3527e00700 1 -- 172.21.15.47:0/1862896878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3528107eb0 msgr2=0x7f35281a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.293+0000 7f3527e00700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3528107eb0 0x7f35281a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.293+0000 7f3527e00700 1 -- 172.21.15.47:0/1862896878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3528105580 msgr2=0x7f352819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.293+0000 7f3527e00700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3528105580 0x7f352819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.293+0000 7f3527e00700 1 -- 172.21.15.47:0/1862896878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35281a2310 con 0x7f3528102c50 2024-03-21T11:23:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.293+0000 7f351fe00700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3528105580 0x7f352819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.293+0000 7f3527e00700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3528102c50 0x7f352819a6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f351800d980 tx=0x7f351800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.293+0000 7f3525600700 1 -- 172.21.15.47:0/1862896878 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518016070 con 0x7f3528102c50 2024-03-21T11:23:18.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.293+0000 7f3525600700 1 -- 172.21.15.47:0/1862896878 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f351800de70 con 0x7f3528102c50 2024-03-21T11:23:18.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.293+0000 7f352f200700 1 -- 172.21.15.47:0/1862896878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35281a2600 con 0x7f3528102c50 2024-03-21T11:23:18.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.293+0000 7f3525600700 1 -- 172.21.15.47:0/1862896878 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351800f3c0 con 0x7f3528102c50 2024-03-21T11:23:18.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.293+0000 7f352f200700 1 -- 172.21.15.47:0/1862896878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35281a2b90 con 0x7f3528102c50 2024-03-21T11:23:18.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.295+0000 7f352f200700 1 -- 172.21.15.47:0/1862896878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3528109a30 con 0x7f3528102c50 2024-03-21T11:23:18.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.296+0000 7f3525600700 1 -- 172.21.15.47:0/1862896878 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3518019070 con 0x7f3528102c50 2024-03-21T11:23:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.298+0000 7f3525600700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f350c0a3030 0x7f350c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.298+0000 7f3525600700 1 -- 172.21.15.47:0/1862896878 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f35180cf270 con 0x7f3528102c50 2024-03-21T11:23:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.298+0000 7f351fe00700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f350c0a3030 0x7f350c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.299+0000 7f351fe00700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f350c0a3030 0x7f350c0a54f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f351000b7b0 tx=0x7f351001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.306+0000 7f3525600700 1 -- 172.21.15.47:0/1862896878 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f351809bcb0 con 0x7f3528102c50 2024-03-21T11:23:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.585+0000 7f352f200700 1 -- 172.21.15.47:0/1862896878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3528062da0 con 0x7f3528102c50 2024-03-21T11:23:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.586+0000 7f3525600700 1 -- 172.21.15.47:0/1862896878 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+3046 (secure 0 0 0) 0x7f351809b5b0 con 0x7f3528102c50 2024-03-21T11:23:18.587 INFO:teuthology.orchestra.run.smithi047.stdout:e27 2024-03-21T11:23:18.587 INFO:teuthology.orchestra.run.smithi047.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T11:23:18.587 INFO:teuthology.orchestra.run.smithi047.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T11:23:18.587 INFO:teuthology.orchestra.run.smithi047.stdout:legacy client fscid: 1 2024-03-21T11:23:18.588 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:18.588 INFO:teuthology.orchestra.run.smithi047.stdout:Filesystem 'cephfs' (1) 2024-03-21T11:23:18.588 INFO:teuthology.orchestra.run.smithi047.stdout:fs_name cephfs 2024-03-21T11:23:18.588 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 27 2024-03-21T11:23:18.588 INFO:teuthology.orchestra.run.smithi047.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T11:23:18.588 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-03-21T11:20:48.626416+0000 2024-03-21T11:23:18.588 INFO:teuthology.orchestra.run.smithi047.stdout:modified 2024-03-21T11:21:18.592954+0000 2024-03-21T11:23:18.588 INFO:teuthology.orchestra.run.smithi047.stdout:tableserver 0 2024-03-21T11:23:18.588 INFO:teuthology.orchestra.run.smithi047.stdout:root 0 2024-03-21T11:23:18.588 INFO:teuthology.orchestra.run.smithi047.stdout:session_timeout 300 2024-03-21T11:23:18.588 INFO:teuthology.orchestra.run.smithi047.stdout:session_autoclose 300 2024-03-21T11:23:18.588 INFO:teuthology.orchestra.run.smithi047.stdout:max_file_size 1099511627776 2024-03-21T11:23:18.588 INFO:teuthology.orchestra.run.smithi047.stdout:max_xattr_size 65536 2024-03-21T11:23:18.588 INFO:teuthology.orchestra.run.smithi047.stdout:required_client_features {} 2024-03-21T11:23:18.589 INFO:teuthology.orchestra.run.smithi047.stdout:last_failure 0 2024-03-21T11:23:18.589 INFO:teuthology.orchestra.run.smithi047.stdout:last_failure_osd_epoch 0 2024-03-21T11:23:18.589 INFO:teuthology.orchestra.run.smithi047.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T11:23:18.589 INFO:teuthology.orchestra.run.smithi047.stdout:max_mds 5 2024-03-21T11:23:18.589 INFO:teuthology.orchestra.run.smithi047.stdout:in 0,1,2,3,4 2024-03-21T11:23:18.589 INFO:teuthology.orchestra.run.smithi047.stdout:up {0=24581,1=24553,2=24535,3=24524,4=24523} 2024-03-21T11:23:18.589 INFO:teuthology.orchestra.run.smithi047.stdout:failed 2024-03-21T11:23:18.589 INFO:teuthology.orchestra.run.smithi047.stdout:damaged 2024-03-21T11:23:18.589 INFO:teuthology.orchestra.run.smithi047.stdout:stopped 2024-03-21T11:23:18.589 INFO:teuthology.orchestra.run.smithi047.stdout:data_pools [3,4] 2024-03-21T11:23:18.589 INFO:teuthology.orchestra.run.smithi047.stdout:metadata_pool 2 2024-03-21T11:23:18.589 INFO:teuthology.orchestra.run.smithi047.stdout:inline_data disabled 2024-03-21T11:23:18.589 INFO:teuthology.orchestra.run.smithi047.stdout:balancer 2024-03-21T11:23:18.589 INFO:teuthology.orchestra.run.smithi047.stdout:bal_rank_mask -1 2024-03-21T11:23:18.590 INFO:teuthology.orchestra.run.smithi047.stdout:standby_count_wanted 0 2024-03-21T11:23:18.590 INFO:teuthology.orchestra.run.smithi047.stdout:qdb_cluster leader: 24581 members: 24524,24523,24535,24581,24553 2024-03-21T11:23:18.590 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b{0:24581} state up:active seq 3 addr [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T11:23:18.590 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.e{0:24494} state up:standby-replay seq 1 addr [v2:172.21.15.100:6834/2785789126,v1:172.21.15.100:6835/2785789126] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T11:23:18.590 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.i{1:24553} state up:active seq 12 addr [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T11:23:18.590 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d{1:24476} state up:standby-replay seq 1 addr [v2:172.21.15.47:6836/1846721633,v1:172.21.15.47:6837/1846721633] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T11:23:18.590 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c{2:24535} state up:active seq 14 addr [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T11:23:18.590 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.l{2:24457} state up:standby-replay seq 1 addr [v2:172.21.15.102:6832/1794606435,v1:172.21.15.102:6833/1794606435] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T11:23:18.590 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.h{3:24524} state up:active seq 14 addr [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T11:23:18.590 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.k{3:24446} state up:standby-replay seq 1 addr [v2:172.21.15.100:6832/904714287,v1:172.21.15.100:6833/904714287] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T11:23:18.590 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.f{4:24523} state up:active seq 15 addr [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T11:23:18.590 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a{4:14466} state up:standby-replay seq 1 addr [v2:172.21.15.47:6840/1742084729,v1:172.21.15.47:6841/1742084729] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T11:23:18.590 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:18.590 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:18.591 INFO:teuthology.orchestra.run.smithi047.stdout:Standby daemons: 2024-03-21T11:23:18.591 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:18.591 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.j{-1:14382} state up:standby seq 1 addr [v2:172.21.15.47:6834/4028210726,v1:172.21.15.47:6835/4028210726] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T11:23:18.591 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.g{-1:14448} state up:standby seq 1 addr [v2:172.21.15.47:6838/1314095873,v1:172.21.15.47:6839/1314095873] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T11:23:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.589+0000 7f352f200700 1 -- 172.21.15.47:0/1862896878 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f350c0a3030 msgr2=0x7f350c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.589+0000 7f352f200700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f350c0a3030 0x7f350c0a54f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f351000b7b0 tx=0x7f351001a040 comp rx=0 tx=0).stop 2024-03-21T11:23:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.589+0000 7f352f200700 1 -- 172.21.15.47:0/1862896878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3528102c50 msgr2=0x7f352819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.589+0000 7f352f200700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3528102c50 0x7f352819a6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f351800d980 tx=0x7f351800dc90 comp rx=0 tx=0).stop 2024-03-21T11:23:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.589+0000 7f352f200700 1 -- 172.21.15.47:0/1862896878 shutdown_connections 2024-03-21T11:23:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.589+0000 7f352f200700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3528107eb0 0x7f35281a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.589+0000 7f352f200700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3528102c50 0x7f352819a6b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.589+0000 7f352f200700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f350c0a3030 0x7f350c0a54f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.589+0000 7f352f200700 1 --2- 172.21.15.47:0/1862896878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3528105580 0x7f352819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.589+0000 7f352f200700 1 -- 172.21.15.47:0/1862896878 >> 172.21.15.47:0/1862896878 conn(0x7f35280fc660 msgr2=0x7f35281024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.589+0000 7f352f200700 1 -- 172.21.15.47:0/1862896878 shutdown_connections 2024-03-21T11:23:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:18.589+0000 7f352f200700 1 -- 172.21.15.47:0/1862896878 wait complete. 2024-03-21T11:23:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 27 2024-03-21T11:23:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:18 smithi102 ceph-mon[36748]: pgmap v399: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:23:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:18 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1862896878' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T11:23:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:18 smithi047 ceph-mon[31351]: pgmap v399: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:23:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:18 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1862896878' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T11:23:19.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:18 smithi100 ceph-mon[36245]: pgmap v399: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:23:19.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:18 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1862896878' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T11:23:19.445 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-21T11:23:19.931 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:23:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:20 smithi102 ceph-mon[36748]: pgmap v400: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:23:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:20 smithi047 ceph-mon[31351]: pgmap v400: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:23:21.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:20 smithi100 ceph-mon[36245]: pgmap v400: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:23:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.274+0000 7fcc8b400700 1 -- 172.21.15.47:0/1685118507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc84104410 msgr2=0x7fcc84104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:22.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.274+0000 7fcc8b400700 1 --2- 172.21.15.47:0/1685118507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc84104410 0x7fcc84104830 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcc780099c0 tx=0x7fcc78009cd0 comp rx=0 tx=0).stop 2024-03-21T11:23:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.274+0000 7fcc8b400700 1 -- 172.21.15.47:0/1685118507 shutdown_connections 2024-03-21T11:23:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.274+0000 7fcc8b400700 1 --2- 172.21.15.47:0/1685118507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc84105fb0 0x7fcc8410c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.274+0000 7fcc8b400700 1 --2- 172.21.15.47:0/1685118507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc84104410 0x7fcc84104830 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.274+0000 7fcc8b400700 1 --2- 172.21.15.47:0/1685118507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc84105610 0x7fcc84105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.274+0000 7fcc8b400700 1 -- 172.21.15.47:0/1685118507 >> 172.21.15.47:0/1685118507 conn(0x7fcc840ff9b0 msgr2=0x7fcc84101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.274+0000 7fcc8b400700 1 -- 172.21.15.47:0/1685118507 shutdown_connections 2024-03-21T11:23:22.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.274+0000 7fcc8b400700 1 -- 172.21.15.47:0/1685118507 wait complete. 2024-03-21T11:23:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.275+0000 7fcc8b400700 1 Processor -- start 2024-03-21T11:23:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.276+0000 7fcc8b400700 1 -- start start 2024-03-21T11:23:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.276+0000 7fcc8b400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc84104410 0x7fcc8419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.276+0000 7fcc8b400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc84105610 0x7fcc8419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.276+0000 7fcc8b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc84105fb0 0x7fcc841a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.276+0000 7fcc8b400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8410ffe0 con 0x7fcc84105fb0 2024-03-21T11:23:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.276+0000 7fcc8b400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8410fe60 con 0x7fcc84104410 2024-03-21T11:23:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.276+0000 7fcc8b400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc84110160 con 0x7fcc84105610 2024-03-21T11:23:22.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.276+0000 7fcc83e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc84105610 0x7fcc8419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:22.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.276+0000 7fcc88a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc84104410 0x7fcc8419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:22.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.276+0000 7fcc83e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc84105610 0x7fcc8419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32818/0 (socket says 172.21.15.47:32818) 2024-03-21T11:23:22.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.276+0000 7fcc83e00700 1 -- 172.21.15.47:0/1773970807 learned_addr learned my addr 172.21.15.47:0/1773970807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:22.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.276+0000 7fcc89400700 1 --2- 172.21.15.47:0/1773970807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc84105fb0 0x7fcc841a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:22.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.277+0000 7fcc88a00700 1 -- 172.21.15.47:0/1773970807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc84105610 msgr2=0x7fcc8419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:22.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.277+0000 7fcc88a00700 1 --2- 172.21.15.47:0/1773970807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc84105610 0x7fcc8419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:22.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.277+0000 7fcc88a00700 1 -- 172.21.15.47:0/1773970807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc84105fb0 msgr2=0x7fcc841a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:22.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.277+0000 7fcc88a00700 1 --2- 172.21.15.47:0/1773970807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc84105fb0 0x7fcc841a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:22.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.277+0000 7fcc88a00700 1 -- 172.21.15.47:0/1773970807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc841a2220 con 0x7fcc84104410 2024-03-21T11:23:22.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.277+0000 7fcc83e00700 1 --2- 172.21.15.47:0/1773970807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc84105610 0x7fcc8419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:22.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.277+0000 7fcc88a00700 1 --2- 172.21.15.47:0/1773970807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc84104410 0x7fcc8419a5c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcc78003790 tx=0x7fcc78009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:22.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.277+0000 7fcc81600700 1 -- 172.21.15.47:0/1773970807 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78009e60 con 0x7fcc84104410 2024-03-21T11:23:22.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.277+0000 7fcc81600700 1 -- 172.21.15.47:0/1773970807 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc78003960 con 0x7fcc84104410 2024-03-21T11:23:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.277+0000 7fcc81600700 1 -- 172.21.15.47:0/1773970807 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78005560 con 0x7fcc84104410 2024-03-21T11:23:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.277+0000 7fcc8b400700 1 -- 172.21.15.47:0/1773970807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc841a24b0 con 0x7fcc84104410 2024-03-21T11:23:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.278+0000 7fcc8b400700 1 -- 172.21.15.47:0/1773970807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc841a2990 con 0x7fcc84104410 2024-03-21T11:23:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.279+0000 7fcc8b400700 1 -- 172.21.15.47:0/1773970807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc84068e50 con 0x7fcc84104410 2024-03-21T11:23:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.281+0000 7fcc81600700 1 -- 172.21.15.47:0/1773970807 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcc78003b10 con 0x7fcc84104410 2024-03-21T11:23:22.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.282+0000 7fcc81600700 1 --2- 172.21.15.47:0/1773970807 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc6c0a2fb0 0x7fcc6c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:22.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.282+0000 7fcc81600700 1 -- 172.21.15.47:0/1773970807 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcc780d0520 con 0x7fcc84104410 2024-03-21T11:23:22.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.282+0000 7fcc83e00700 1 --2- 172.21.15.47:0/1773970807 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc6c0a2fb0 0x7fcc6c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:22.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.283+0000 7fcc83e00700 1 --2- 172.21.15.47:0/1773970807 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc6c0a2fb0 0x7fcc6c0a5470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcc74011630 tx=0x7fcc74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:22.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.284+0000 7fcc81600700 1 -- 172.21.15.47:0/1773970807 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcc78013070 con 0x7fcc84104410 2024-03-21T11:23:22.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.512+0000 7fcc8b400700 1 -- 172.21.15.47:0/1773970807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fcc84062da0 con 0x7fcc84104410 2024-03-21T11:23:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.513+0000 7fcc81600700 1 -- 172.21.15.47:0/1773970807 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v95) v1 ==== 56+0+5652 (secure 0 0 0) 0x7fcc7809cff0 con 0x7fcc84104410 2024-03-21T11:23:22.517 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 95 2024-03-21T11:23:22.517 INFO:teuthology.orchestra.run.smithi047.stdout:fsid a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:23:22.517 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-03-21T11:11:41.726008+0000 2024-03-21T11:23:22.517 INFO:teuthology.orchestra.run.smithi047.stdout:modified 2024-03-21T11:22:50.841892+0000 2024-03-21T11:23:22.517 INFO:teuthology.orchestra.run.smithi047.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-21T11:23:22.517 INFO:teuthology.orchestra.run.smithi047.stdout:crush_version 27 2024-03-21T11:23:22.517 INFO:teuthology.orchestra.run.smithi047.stdout:full_ratio 0.95 2024-03-21T11:23:22.517 INFO:teuthology.orchestra.run.smithi047.stdout:backfillfull_ratio 0.9 2024-03-21T11:23:22.517 INFO:teuthology.orchestra.run.smithi047.stdout:nearfull_ratio 0.85 2024-03-21T11:23:22.517 INFO:teuthology.orchestra.run.smithi047.stdout:require_min_compat_client luminous 2024-03-21T11:23:22.517 INFO:teuthology.orchestra.run.smithi047.stdout:min_compat_client luminous 2024-03-21T11:23:22.518 INFO:teuthology.orchestra.run.smithi047.stdout:require_osd_release squid 2024-03-21T11:23:22.518 INFO:teuthology.orchestra.run.smithi047.stdout:stretch_mode_enabled false 2024-03-21T11:23:22.518 INFO:teuthology.orchestra.run.smithi047.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-21T11:23:22.518 INFO:teuthology.orchestra.run.smithi047.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-21T11:23:22.518 INFO:teuthology.orchestra.run.smithi047.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-21T11:23:22.518 INFO:teuthology.orchestra.run.smithi047.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 91 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-21T11:23:22.518 INFO:teuthology.orchestra.run.smithi047.stdout:max_osd 12 2024-03-21T11:23:22.518 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0 up in weight 1 up_from 8 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6802/3777145460,v1:172.21.15.47:6803/3777145460] [v2:172.21.15.47:6804/3777145460,v1:172.21.15.47:6805/3777145460] exists,up 68bd29ba-c4c1-482d-b233-47e42371a12c 2024-03-21T11:23:22.518 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6800/2612500436,v1:172.21.15.100:6801/2612500436] [v2:172.21.15.100:6802/2612500436,v1:172.21.15.100:6803/2612500436] exists,up c44724d5-59e4-4520-83d9-7e5f4fd085a2 2024-03-21T11:23:22.518 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2 up in weight 1 up_from 18 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6800/644548261,v1:172.21.15.102:6801/644548261] [v2:172.21.15.102:6802/644548261,v1:172.21.15.102:6803/644548261] exists,up 712cd8e5-7760-4534-8f80-71e7acd5fae9 2024-03-21T11:23:22.518 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3 up in weight 1 up_from 26 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6810/534982877,v1:172.21.15.47:6811/534982877] [v2:172.21.15.47:6812/534982877,v1:172.21.15.47:6813/534982877] exists,up 182c6d29-67dd-4cc1-b79f-7d106f2cc0ff 2024-03-21T11:23:22.518 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4 up in weight 1 up_from 34 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6808/1323994971,v1:172.21.15.100:6809/1323994971] [v2:172.21.15.100:6810/1323994971,v1:172.21.15.100:6811/1323994971] exists,up 65901f21-1ccb-45d6-90a9-8f40d2dfaf28 2024-03-21T11:23:22.518 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5 up in weight 1 up_from 39 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6808/2128847052,v1:172.21.15.102:6809/2128847052] [v2:172.21.15.102:6810/2128847052,v1:172.21.15.102:6811/2128847052] exists,up d02dcbc9-f19e-43ef-9b63-9c7039111849 2024-03-21T11:23:22.519 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6 up in weight 1 up_from 47 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6818/2684410491,v1:172.21.15.47:6819/2684410491] [v2:172.21.15.47:6820/2684410491,v1:172.21.15.47:6821/2684410491] exists,up fbe3b7c0-943f-4849-a8b5-5d792e1f494b 2024-03-21T11:23:22.519 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7 up in weight 1 up_from 52 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6816/3018830551,v1:172.21.15.100:6817/3018830551] [v2:172.21.15.100:6818/3018830551,v1:172.21.15.100:6819/3018830551] exists,up 53cea2b7-6bbe-4b15-b28e-7fdddd70607e 2024-03-21T11:23:22.519 INFO:teuthology.orchestra.run.smithi047.stdout:osd.8 up in weight 1 up_from 60 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6816/1509099917,v1:172.21.15.102:6817/1509099917] [v2:172.21.15.102:6818/1509099917,v1:172.21.15.102:6819/1509099917] exists,up 20bf15de-3238-4613-9cd2-6bcc5ba6d60f 2024-03-21T11:23:22.519 INFO:teuthology.orchestra.run.smithi047.stdout:osd.9 up in weight 1 up_from 68 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6826/3319373812,v1:172.21.15.47:6827/3319373812] [v2:172.21.15.47:6828/3319373812,v1:172.21.15.47:6829/3319373812] exists,up 9f22490f-23d7-4822-b6f6-57f2da5e0514 2024-03-21T11:23:22.519 INFO:teuthology.orchestra.run.smithi047.stdout:osd.10 up in weight 1 up_from 73 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6824/1645855291,v1:172.21.15.100:6825/1645855291] [v2:172.21.15.100:6826/1645855291,v1:172.21.15.100:6827/1645855291] exists,up 22ad02d6-55e9-4ec3-96e1-b24d6d1222f5 2024-03-21T11:23:22.519 INFO:teuthology.orchestra.run.smithi047.stdout:osd.11 up in weight 1 up_from 81 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6824/557348355,v1:172.21.15.102:6825/557348355] [v2:172.21.15.102:6826/557348355,v1:172.21.15.102:6827/557348355] exists,up 756cdfb6-361e-494f-89a0-ad0d560f9675 2024-03-21T11:23:22.520 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.a [2,11] 2024-03-21T11:23:22.520 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.b [2,4] 2024-03-21T11:23:22.520 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.1a [8,1] 2024-03-21T11:23:22.520 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.1d [8,11] 2024-03-21T11:23:22.520 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.0 [5,9] 2024-03-21T11:23:22.520 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.3 [4,9] 2024-03-21T11:23:22.520 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.23 [4,8] 2024-03-21T11:23:22.520 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.2c [4,9] 2024-03-21T11:23:22.520 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 4.5 [5,10] 2024-03-21T11:23:22.520 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 4.6 [9,11] 2024-03-21T11:23:22.521 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 4.1b [9,8] 2024-03-21T11:23:22.521 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 4.1c [9,8] 2024-03-21T11:23:22.521 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/4277663939 expires 2024-03-22T11:12:48.837005+0000 2024-03-21T11:23:22.521 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6801/185270192 expires 2024-03-22T11:12:48.837005+0000 2024-03-21T11:23:22.521 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6800/185270192 expires 2024-03-22T11:12:48.837005+0000 2024-03-21T11:23:22.521 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/1272635669 expires 2024-03-22T11:12:48.837005+0000 2024-03-21T11:23:22.521 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/2508263463 expires 2024-03-22T11:12:14.713084+0000 2024-03-21T11:23:22.521 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/3773889790 expires 2024-03-22T11:12:48.837005+0000 2024-03-21T11:23:22.521 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6801/1584030551 expires 2024-03-22T11:12:14.713084+0000 2024-03-21T11:23:22.521 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/3130329909 expires 2024-03-22T11:12:14.713084+0000 2024-03-21T11:23:22.521 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6800/1584030551 expires 2024-03-22T11:12:14.713084+0000 2024-03-21T11:23:22.521 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/335539055 expires 2024-03-22T11:12:14.713084+0000 2024-03-21T11:23:22.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.516+0000 7fcc8b400700 1 -- 172.21.15.47:0/1773970807 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc6c0a2fb0 msgr2=0x7fcc6c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:22.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.516+0000 7fcc8b400700 1 --2- 172.21.15.47:0/1773970807 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc6c0a2fb0 0x7fcc6c0a5470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcc74011630 tx=0x7fcc74009260 comp rx=0 tx=0).stop 2024-03-21T11:23:22.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.516+0000 7fcc8b400700 1 -- 172.21.15.47:0/1773970807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc84104410 msgr2=0x7fcc8419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:22.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.516+0000 7fcc8b400700 1 --2- 172.21.15.47:0/1773970807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc84104410 0x7fcc8419a5c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcc78003790 tx=0x7fcc78009580 comp rx=0 tx=0).stop 2024-03-21T11:23:22.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.516+0000 7fcc8b400700 1 -- 172.21.15.47:0/1773970807 shutdown_connections 2024-03-21T11:23:22.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.516+0000 7fcc8b400700 1 --2- 172.21.15.47:0/1773970807 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc6c0a2fb0 0x7fcc6c0a5470 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:22.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.516+0000 7fcc8b400700 1 --2- 172.21.15.47:0/1773970807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc84105fb0 0x7fcc841a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:22.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.516+0000 7fcc8b400700 1 --2- 172.21.15.47:0/1773970807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc84104410 0x7fcc8419a5c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:22.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.516+0000 7fcc8b400700 1 --2- 172.21.15.47:0/1773970807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc84105610 0x7fcc8419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:22.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.516+0000 7fcc8b400700 1 -- 172.21.15.47:0/1773970807 >> 172.21.15.47:0/1773970807 conn(0x7fcc840ff9b0 msgr2=0x7fcc84108910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:22.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.517+0000 7fcc8b400700 1 -- 172.21.15.47:0/1773970807 shutdown_connections 2024-03-21T11:23:22.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:22.517+0000 7fcc8b400700 1 -- 172.21.15.47:0/1773970807 wait complete. 2024-03-21T11:23:23.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:22 smithi102 ceph-mon[36748]: pgmap v401: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:23:23.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:22 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1773970807' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T11:23:23.255 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:22 smithi047 ceph-mon[31351]: pgmap v401: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:23:23.255 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:22 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1773970807' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T11:23:23.380 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-21T11:23:23.393 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:23:23.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:22 smithi100 ceph-mon[36245]: pgmap v401: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:23:23.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:22 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1773970807' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T11:23:23.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.621+0000 7fbe87200640 1 -- 172.21.15.47:0/2684177140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe88101f70 msgr2=0x7fbe88104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:23.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.621+0000 7fbe87200640 1 --2- 172.21.15.47:0/2684177140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe88101f70 0x7fbe88104360 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009900 tx=0x7fbe7802bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:23.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.621+0000 7fbe87200640 1 -- 172.21.15.47:0/2684177140 shutdown_connections 2024-03-21T11:23:23.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.621+0000 7fbe87200640 1 --2- 172.21.15.47:0/2684177140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe881071d0 0x7fbe88109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:23.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.621+0000 7fbe87200640 1 --2- 172.21.15.47:0/2684177140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe881048a0 0x7fbe88106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:23.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.621+0000 7fbe87200640 1 --2- 172.21.15.47:0/2684177140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe88101f70 0x7fbe88104360 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:23.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.621+0000 7fbe87200640 1 -- 172.21.15.47:0/2684177140 >> 172.21.15.47:0/2684177140 conn(0x7fbe880fb980 msgr2=0x7fbe880fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:23.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.621+0000 7fbe87200640 1 -- 172.21.15.47:0/2684177140 shutdown_connections 2024-03-21T11:23:23.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.621+0000 7fbe87200640 1 -- 172.21.15.47:0/2684177140 wait complete. 2024-03-21T11:23:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.622+0000 7fbe87200640 1 Processor -- start 2024-03-21T11:23:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.622+0000 7fbe87200640 1 -- start start 2024-03-21T11:23:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe87200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe88101f70 0x7fbe88199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:23.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe87200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe881048a0 0x7fbe88199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:23.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe87200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe881071d0 0x7fbe881a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:23.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe87200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8810d180 con 0x7fbe88101f70 2024-03-21T11:23:23.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe87200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8810d000 con 0x7fbe881048a0 2024-03-21T11:23:23.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe87200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8810d300 con 0x7fbe881071d0 2024-03-21T11:23:23.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe85400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe881048a0 0x7fbe88199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:23.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe85e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe88101f70 0x7fbe88199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:23.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe85400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe881048a0 0x7fbe88199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38526/0 (socket says 172.21.15.47:38526) 2024-03-21T11:23:23.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe85400640 1 -- 172.21.15.47:0/257200470 learned_addr learned my addr 172.21.15.47:0/257200470 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:23.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe86800640 1 --2- 172.21.15.47:0/257200470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe881071d0 0x7fbe881a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:23.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe85400640 1 -- 172.21.15.47:0/257200470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe881071d0 msgr2=0x7fbe881a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:23.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe85400640 1 --2- 172.21.15.47:0/257200470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe881071d0 0x7fbe881a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:23.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe85400640 1 -- 172.21.15.47:0/257200470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe88101f70 msgr2=0x7fbe88199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:23.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe85400640 1 --2- 172.21.15.47:0/257200470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe88101f70 0x7fbe88199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:23.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe85400640 1 -- 172.21.15.47:0/257200470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe881a1630 con 0x7fbe881048a0 2024-03-21T11:23:23.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.623+0000 7fbe85e00640 1 --2- 172.21.15.47:0/257200470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe88101f70 0x7fbe88199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:23.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.624+0000 7fbe85400640 1 --2- 172.21.15.47:0/257200470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe881048a0 0x7fbe88199f50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400e9f0 tx=0x7fbe7400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:23.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.624+0000 7fbe7ea00640 1 -- 172.21.15.47:0/257200470 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7400ccc0 con 0x7fbe881048a0 2024-03-21T11:23:23.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.624+0000 7fbe87200640 1 -- 172.21.15.47:0/257200470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe881a1920 con 0x7fbe881048a0 2024-03-21T11:23:23.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.624+0000 7fbe7ea00640 1 -- 172.21.15.47:0/257200470 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe74005500 con 0x7fbe881048a0 2024-03-21T11:23:23.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.624+0000 7fbe7ea00640 1 -- 172.21.15.47:0/257200470 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74010690 con 0x7fbe881048a0 2024-03-21T11:23:23.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.624+0000 7fbe87200640 1 -- 172.21.15.47:0/257200470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe881a1e70 con 0x7fbe881048a0 2024-03-21T11:23:23.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.625+0000 7fbe87200640 1 -- 172.21.15.47:0/257200470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe88108d20 con 0x7fbe881048a0 2024-03-21T11:23:23.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.628+0000 7fbe7ea00640 1 -- 172.21.15.47:0/257200470 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbe740046c0 con 0x7fbe881048a0 2024-03-21T11:23:23.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.629+0000 7fbe7ea00640 1 --2- 172.21.15.47:0/257200470 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe6c0a2f20 0x7fbe6c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.629+0000 7fbe7ea00640 1 -- 172.21.15.47:0/257200470 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbe740cfd40 con 0x7fbe881048a0 2024-03-21T11:23:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.630+0000 7fbe85e00640 1 --2- 172.21.15.47:0/257200470 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe6c0a2f20 0x7fbe6c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.630+0000 7fbe85e00640 1 --2- 172.21.15.47:0/257200470 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe6c0a2f20 0x7fbe6c0a53e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbe7802c7b0 tx=0x7fbe78005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:23.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.635+0000 7fbe7ea00640 1 -- 172.21.15.47:0/257200470 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbe74053740 con 0x7fbe881048a0 2024-03-21T11:23:23.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.843+0000 7fbe87200640 1 -- 172.21.15.47:0/257200470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe8802d230 con 0x7fbe881048a0 2024-03-21T11:23:23.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.846+0000 7fbe7ea00640 1 -- 172.21.15.47:0/257200470 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24747 (secure 0 0 0) 0x7fbe7409c780 con 0x7fbe881048a0 2024-03-21T11:23:23.848 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:23.848 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:22:50.841892+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:23:23.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.850+0000 7fbe87200640 1 -- 172.21.15.47:0/257200470 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe6c0a2f20 msgr2=0x7fbe6c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:23.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.850+0000 7fbe87200640 1 --2- 172.21.15.47:0/257200470 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe6c0a2f20 0x7fbe6c0a53e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbe7802c7b0 tx=0x7fbe78005fd0 comp rx=0 tx=0).stop 2024-03-21T11:23:23.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.850+0000 7fbe87200640 1 -- 172.21.15.47:0/257200470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe881048a0 msgr2=0x7fbe88199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:23.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.850+0000 7fbe87200640 1 --2- 172.21.15.47:0/257200470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe881048a0 0x7fbe88199f50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400e9f0 tx=0x7fbe7400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:23:23.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.850+0000 7fbe87200640 1 -- 172.21.15.47:0/257200470 shutdown_connections 2024-03-21T11:23:23.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.850+0000 7fbe87200640 1 --2- 172.21.15.47:0/257200470 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe6c0a2f20 0x7fbe6c0a53e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:23.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.850+0000 7fbe87200640 1 --2- 172.21.15.47:0/257200470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe881071d0 0x7fbe881a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:23.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.850+0000 7fbe87200640 1 --2- 172.21.15.47:0/257200470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe881048a0 0x7fbe88199f50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:23.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.850+0000 7fbe87200640 1 --2- 172.21.15.47:0/257200470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe88101f70 0x7fbe88199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:23.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.850+0000 7fbe87200640 1 -- 172.21.15.47:0/257200470 >> 172.21.15.47:0/257200470 conn(0x7fbe880fb980 msgr2=0x7fbe881016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:23.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.850+0000 7fbe87200640 1 -- 172.21.15.47:0/257200470 shutdown_connections 2024-03-21T11:23:23.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:23.850+0000 7fbe87200640 1 -- 172.21.15.47:0/257200470 wait complete. 2024-03-21T11:23:23.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T11:15:35.545245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T11:20:47.016174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T11:20:48.108506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T11:20:52.254154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T11:23:23.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T11:23:24.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.077+0000 7f5fbfe00640 1 -- 172.21.15.47:0/889357454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fc0102790 msgr2=0x7f5fc0102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:24.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.077+0000 7f5fbfe00640 1 --2- 172.21.15.47:0/889357454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fc0102790 0x7f5fc0102c10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009900 tx=0x7f5fac02bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:24.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.077+0000 7f5fbfe00640 1 -- 172.21.15.47:0/889357454 shutdown_connections 2024-03-21T11:23:24.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.077+0000 7f5fbfe00640 1 --2- 172.21.15.47:0/889357454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc0103150 0x7f5fc01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.077+0000 7f5fbfe00640 1 --2- 172.21.15.47:0/889357454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fc0102790 0x7f5fc0102c10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.077+0000 7f5fbfe00640 1 --2- 172.21.15.47:0/889357454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fc01015a0 0x7f5fc01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.077+0000 7f5fbfe00640 1 -- 172.21.15.47:0/889357454 >> 172.21.15.47:0/889357454 conn(0x7f5fc00fcb10 msgr2=0x7f5fc00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:24.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.077+0000 7f5fbfe00640 1 -- 172.21.15.47:0/889357454 shutdown_connections 2024-03-21T11:23:24.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.077+0000 7f5fbfe00640 1 -- 172.21.15.47:0/889357454 wait complete. 2024-03-21T11:23:24.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.078+0000 7f5fbfe00640 1 Processor -- start 2024-03-21T11:23:24.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.078+0000 7f5fbfe00640 1 -- start start 2024-03-21T11:23:24.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fc01015a0 0x7f5fc0199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc0102790 0x7f5fc019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fc0103150 0x7f5fc01a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbfe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc010d250 con 0x7f5fc01015a0 2024-03-21T11:23:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbfe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc010d0d0 con 0x7f5fc0102790 2024-03-21T11:23:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbfe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc010d3d0 con 0x7f5fc0103150 2024-03-21T11:23:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc0102790 0x7f5fc019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fc01015a0 0x7f5fc0199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc0102790 0x7f5fc019a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38544/0 (socket says 172.21.15.47:38544) 2024-03-21T11:23:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbe000640 1 -- 172.21.15.47:0/3809562757 learned_addr learned my addr 172.21.15.47:0/3809562757 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbf400640 1 --2- 172.21.15.47:0/3809562757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fc0103150 0x7f5fc01a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbe000640 1 -- 172.21.15.47:0/3809562757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fc0103150 msgr2=0x7f5fc01a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbe000640 1 --2- 172.21.15.47:0/3809562757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fc0103150 0x7f5fc01a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbe000640 1 -- 172.21.15.47:0/3809562757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fc01015a0 msgr2=0x7f5fc0199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbe000640 1 --2- 172.21.15.47:0/3809562757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fc01015a0 0x7f5fc0199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.079+0000 7f5fbe000640 1 -- 172.21.15.47:0/3809562757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc01a1700 con 0x7f5fc0102790 2024-03-21T11:23:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.080+0000 7f5fbe000640 1 --2- 172.21.15.47:0/3809562757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc0102790 0x7f5fc019a020 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5fac000c00 tx=0x7f5fac02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.080+0000 7f5fc4a00640 1 -- 172.21.15.47:0/3809562757 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac043020 con 0x7f5fc0102790 2024-03-21T11:23:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.080+0000 7f5fbfe00640 1 -- 172.21.15.47:0/3809562757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc01a1990 con 0x7f5fc0102790 2024-03-21T11:23:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.080+0000 7f5fc4a00640 1 -- 172.21.15.47:0/3809562757 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fac0054e0 con 0x7f5fc0102790 2024-03-21T11:23:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.080+0000 7f5fc4a00640 1 -- 172.21.15.47:0/3809562757 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac003c40 con 0x7f5fc0102790 2024-03-21T11:23:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.080+0000 7f5fbfe00640 1 -- 172.21.15.47:0/3809562757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc01a1ee0 con 0x7f5fc0102790 2024-03-21T11:23:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.081+0000 7f5fbfe00640 1 -- 172.21.15.47:0/3809562757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc01a20b0 con 0x7f5fc0102790 2024-03-21T11:23:24.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.083+0000 7f5fc4a00640 1 -- 172.21.15.47:0/3809562757 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5fac046020 con 0x7f5fc0102790 2024-03-21T11:23:24.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.084+0000 7f5fc4a00640 1 --2- 172.21.15.47:0/3809562757 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5fa40a2f20 0x7f5fa40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:24.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.084+0000 7f5fc4a00640 1 -- 172.21.15.47:0/3809562757 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5fac0f1820 con 0x7f5fc0102790 2024-03-21T11:23:24.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.084+0000 7f5fbea00640 1 --2- 172.21.15.47:0/3809562757 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5fa40a2f20 0x7f5fa40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:24.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.085+0000 7f5fbea00640 1 --2- 172.21.15.47:0/3809562757 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5fa40a2f20 0x7f5fa40a53e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5fc01025f0 tx=0x7f5fb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:24.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.092+0000 7f5fc4a00640 1 -- 172.21.15.47:0/3809562757 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5fac04d020 con 0x7f5fc0102790 2024-03-21T11:23:24.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:23 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/257200470' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:24.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:23 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/257200470' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:24.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.293+0000 7f5fbfe00640 1 -- 172.21.15.47:0/3809562757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5fc01a26f0 con 0x7f5fc0102790 2024-03-21T11:23:24.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.294+0000 7f5fc4a00640 1 -- 172.21.15.47:0/3809562757 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5fac0be260 con 0x7f5fc0102790 2024-03-21T11:23:24.295 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T11:23:24.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.296+0000 7f5fbfe00640 1 -- 172.21.15.47:0/3809562757 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5fa40a2f20 msgr2=0x7f5fa40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:24.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.296+0000 7f5fbfe00640 1 --2- 172.21.15.47:0/3809562757 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5fa40a2f20 0x7f5fa40a53e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5fc01025f0 tx=0x7f5fb0005f90 comp rx=0 tx=0).stop 2024-03-21T11:23:24.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.296+0000 7f5fbfe00640 1 -- 172.21.15.47:0/3809562757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc0102790 msgr2=0x7f5fc019a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:24.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.296+0000 7f5fbfe00640 1 --2- 172.21.15.47:0/3809562757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc0102790 0x7f5fc019a020 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5fac000c00 tx=0x7f5fac02c650 comp rx=0 tx=0).stop 2024-03-21T11:23:24.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.296+0000 7f5fbfe00640 1 -- 172.21.15.47:0/3809562757 shutdown_connections 2024-03-21T11:23:24.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.296+0000 7f5fbfe00640 1 --2- 172.21.15.47:0/3809562757 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5fa40a2f20 0x7f5fa40a53e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.296+0000 7f5fbfe00640 1 --2- 172.21.15.47:0/3809562757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fc0103150 0x7f5fc01a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.296+0000 7f5fbfe00640 1 --2- 172.21.15.47:0/3809562757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc0102790 0x7f5fc019a020 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.296+0000 7f5fbfe00640 1 --2- 172.21.15.47:0/3809562757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fc01015a0 0x7f5fc0199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.296+0000 7f5fbfe00640 1 -- 172.21.15.47:0/3809562757 >> 172.21.15.47:0/3809562757 conn(0x7f5fc00fcb10 msgr2=0x7f5fc00738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:24.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.296+0000 7f5fbfe00640 1 -- 172.21.15.47:0/3809562757 shutdown_connections 2024-03-21T11:23:24.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.296+0000 7f5fbfe00640 1 -- 172.21.15.47:0/3809562757 wait complete. 2024-03-21T11:23:24.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T11:23:24.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:23 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/257200470' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.539+0000 7fe6e4e00640 1 -- 172.21.15.47:0/416614394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e00772a0 msgr2=0x7fe6e0077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.539+0000 7fe6e4e00640 1 --2- 172.21.15.47:0/416614394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e00772a0 0x7fe6e0077700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009900 tx=0x7fe6c802bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.539+0000 7fe6e4e00640 1 -- 172.21.15.47:0/416614394 shutdown_connections 2024-03-21T11:23:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.539+0000 7fe6e4e00640 1 --2- 172.21.15.47:0/416614394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6e0077c40 0x7fe6e00784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.539+0000 7fe6e4e00640 1 --2- 172.21.15.47:0/416614394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e00772a0 0x7fe6e0077700 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.539+0000 7fe6e4e00640 1 --2- 172.21.15.47:0/416614394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6e0076060 0x7fe6e0076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.539+0000 7fe6e4e00640 1 -- 172.21.15.47:0/416614394 >> 172.21.15.47:0/416614394 conn(0x7fe6e00fee50 msgr2=0x7fe6e0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.539+0000 7fe6e4e00640 1 -- 172.21.15.47:0/416614394 shutdown_connections 2024-03-21T11:23:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.539+0000 7fe6e4e00640 1 -- 172.21.15.47:0/416614394 wait complete. 2024-03-21T11:23:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.540+0000 7fe6e4e00640 1 Processor -- start 2024-03-21T11:23:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.540+0000 7fe6e4e00640 1 -- start start 2024-03-21T11:23:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.541+0000 7fe6e4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e0076060 0x7fe6e019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.541+0000 7fe6e4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6e00772a0 0x7fe6e019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.541+0000 7fe6e4e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6e0077c40 0x7fe6e01a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.541+0000 7fe6e4e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e01139b0 con 0x7fe6e0077c40 2024-03-21T11:23:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.541+0000 7fe6e4e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e0113830 con 0x7fe6e00772a0 2024-03-21T11:23:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.541+0000 7fe6e4e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e0113b30 con 0x7fe6e0076060 2024-03-21T11:23:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.541+0000 7fe6dde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e0076060 0x7fe6e019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.541+0000 7fe6dd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6e00772a0 0x7fe6e019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.541+0000 7fe6dde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e0076060 0x7fe6e019dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32862/0 (socket says 172.21.15.47:32862) 2024-03-21T11:23:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.541+0000 7fe6dde00640 1 -- 172.21.15.47:0/1784524572 learned_addr learned my addr 172.21.15.47:0/1784524572 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.541+0000 7fe6de800640 1 --2- 172.21.15.47:0/1784524572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6e0077c40 0x7fe6e01a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.542+0000 7fe6dd400640 1 -- 172.21.15.47:0/1784524572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e0076060 msgr2=0x7fe6e019dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.542+0000 7fe6dd400640 1 --2- 172.21.15.47:0/1784524572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e0076060 0x7fe6e019dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.542+0000 7fe6dd400640 1 -- 172.21.15.47:0/1784524572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6e0077c40 msgr2=0x7fe6e01a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.542+0000 7fe6dd400640 1 --2- 172.21.15.47:0/1784524572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6e0077c40 0x7fe6e01a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.542+0000 7fe6dd400640 1 -- 172.21.15.47:0/1784524572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e01a5c00 con 0x7fe6e00772a0 2024-03-21T11:23:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.542+0000 7fe6dde00640 1 --2- 172.21.15.47:0/1784524572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e0076060 0x7fe6e019dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.542+0000 7fe6dd400640 1 --2- 172.21.15.47:0/1784524572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6e00772a0 0x7fe6e019e520 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8036040 tx=0x7fe6c802c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.542+0000 7fe6d6a00640 1 -- 172.21.15.47:0/1784524572 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8043020 con 0x7fe6e00772a0 2024-03-21T11:23:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.542+0000 7fe6d6a00640 1 -- 172.21.15.47:0/1784524572 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6c80054e0 con 0x7fe6e00772a0 2024-03-21T11:23:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.542+0000 7fe6e4e00640 1 -- 172.21.15.47:0/1784524572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e01a5e90 con 0x7fe6e00772a0 2024-03-21T11:23:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.542+0000 7fe6d6a00640 1 -- 172.21.15.47:0/1784524572 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8003ca0 con 0x7fe6e00772a0 2024-03-21T11:23:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.542+0000 7fe6e4e00640 1 -- 172.21.15.47:0/1784524572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e01a6350 con 0x7fe6e00772a0 2024-03-21T11:23:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.543+0000 7fe6e4e00640 1 -- 172.21.15.47:0/1784524572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e0110250 con 0x7fe6e00772a0 2024-03-21T11:23:24.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.545+0000 7fe6d6a00640 1 -- 172.21.15.47:0/1784524572 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe6c8046020 con 0x7fe6e00772a0 2024-03-21T11:23:24.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.547+0000 7fe6d6a00640 1 --2- 172.21.15.47:0/1784524572 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6c40a2f20 0x7fe6c40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:24.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.547+0000 7fe6d6a00640 1 -- 172.21.15.47:0/1784524572 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe6c80f1820 con 0x7fe6e00772a0 2024-03-21T11:23:24.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.547+0000 7fe6dde00640 1 --2- 172.21.15.47:0/1784524572 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6c40a2f20 0x7fe6c40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.548+0000 7fe6dde00640 1 --2- 172.21.15.47:0/1784524572 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6c40a2f20 0x7fe6c40a53e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800f7d0 tx=0x7fe6d8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.551+0000 7fe6d6a00640 1 -- 172.21.15.47:0/1784524572 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe6c804d020 con 0x7fe6e00772a0 2024-03-21T11:23:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.745+0000 7fe6e4e00640 1 -- 172.21.15.47:0/1784524572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe6e00622a0 con 0x7fe6e00772a0 2024-03-21T11:23:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.746+0000 7fe6d6a00640 1 -- 172.21.15.47:0/1784524572 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe6c80be260 con 0x7fe6e00772a0 2024-03-21T11:23:24.747 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:23:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.748+0000 7fe6e4e00640 1 -- 172.21.15.47:0/1784524572 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6c40a2f20 msgr2=0x7fe6c40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.748+0000 7fe6e4e00640 1 --2- 172.21.15.47:0/1784524572 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6c40a2f20 0x7fe6c40a53e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800f7d0 tx=0x7fe6d8009500 comp rx=0 tx=0).stop 2024-03-21T11:23:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.748+0000 7fe6e4e00640 1 -- 172.21.15.47:0/1784524572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6e00772a0 msgr2=0x7fe6e019e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.748+0000 7fe6e4e00640 1 --2- 172.21.15.47:0/1784524572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6e00772a0 0x7fe6e019e520 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8036040 tx=0x7fe6c802c6c0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.748+0000 7fe6e4e00640 1 -- 172.21.15.47:0/1784524572 shutdown_connections 2024-03-21T11:23:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.748+0000 7fe6e4e00640 1 --2- 172.21.15.47:0/1784524572 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6c40a2f20 0x7fe6c40a53e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.748+0000 7fe6e4e00640 1 --2- 172.21.15.47:0/1784524572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6e0077c40 0x7fe6e01a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.748+0000 7fe6e4e00640 1 --2- 172.21.15.47:0/1784524572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6e00772a0 0x7fe6e019e520 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.748+0000 7fe6e4e00640 1 --2- 172.21.15.47:0/1784524572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e0076060 0x7fe6e019dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.748+0000 7fe6e4e00640 1 -- 172.21.15.47:0/1784524572 >> 172.21.15.47:0/1784524572 conn(0x7fe6e00fee50 msgr2=0x7fe6e0100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.748+0000 7fe6e4e00640 1 -- 172.21.15.47:0/1784524572 shutdown_connections 2024-03-21T11:23:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.748+0000 7fe6e4e00640 1 -- 172.21.15.47:0/1784524572 wait complete. 2024-03-21T11:23:24.769 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T11:23:24.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.988+0000 7fac30000640 1 -- 172.21.15.47:0/553728212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac281021e0 msgr2=0x7fac281045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:24.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.988+0000 7fac30000640 1 --2- 172.21.15.47:0/553728212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac281021e0 0x7fac281045d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009ca0 tx=0x7fac1c02c100 comp rx=0 tx=0).stop 2024-03-21T11:23:24.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.989+0000 7fac30000640 1 -- 172.21.15.47:0/553728212 shutdown_connections 2024-03-21T11:23:24.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.989+0000 7fac30000640 1 --2- 172.21.15.47:0/553728212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac28107440 0x7fac28109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.989+0000 7fac30000640 1 --2- 172.21.15.47:0/553728212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac28104b10 0x7fac28106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.989+0000 7fac30000640 1 --2- 172.21.15.47:0/553728212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac281021e0 0x7fac281045d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.989+0000 7fac30000640 1 -- 172.21.15.47:0/553728212 >> 172.21.15.47:0/553728212 conn(0x7fac280fbb30 msgr2=0x7fac280fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:24.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.989+0000 7fac30000640 1 -- 172.21.15.47:0/553728212 shutdown_connections 2024-03-21T11:23:24.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.989+0000 7fac30000640 1 -- 172.21.15.47:0/553728212 wait complete. 2024-03-21T11:23:24.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.990+0000 7fac30000640 1 Processor -- start 2024-03-21T11:23:24.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.990+0000 7fac30000640 1 -- start start 2024-03-21T11:23:24.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac30000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac281021e0 0x7fac28199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:24.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac30000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac28104b10 0x7fac2819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:24.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac30000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac28107440 0x7fac281a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:24.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac30000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2810d350 con 0x7fac281021e0 2024-03-21T11:23:24.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac30000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2810d1d0 con 0x7fac28104b10 2024-03-21T11:23:24.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac30000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2810d4d0 con 0x7fac28107440 2024-03-21T11:23:24.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac2d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac281021e0 0x7fac28199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:24.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac2e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac28107440 0x7fac281a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:24.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac2e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac28107440 0x7fac281a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32884/0 (socket says 172.21.15.47:32884) 2024-03-21T11:23:24.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac2cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac28104b10 0x7fac2819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:24.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac2e000640 1 -- 172.21.15.47:0/2703065706 learned_addr learned my addr 172.21.15.47:0/2703065706 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:24.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac2e000640 1 -- 172.21.15.47:0/2703065706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac28104b10 msgr2=0x7fac2819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:24.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac2e000640 1 --2- 172.21.15.47:0/2703065706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac28104b10 0x7fac2819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac2e000640 1 -- 172.21.15.47:0/2703065706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac281021e0 msgr2=0x7fac28199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:24.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.991+0000 7fac2e000640 1 --2- 172.21.15.47:0/2703065706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac281021e0 0x7fac28199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:24.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.992+0000 7fac2e000640 1 -- 172.21.15.47:0/2703065706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac281a17c0 con 0x7fac28107440 2024-03-21T11:23:24.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.992+0000 7fac2e000640 1 --2- 172.21.15.47:0/2703065706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac28107440 0x7fac281a1150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac2000c970 tx=0x7fac2000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:24.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.992+0000 7fac26000640 1 -- 172.21.15.47:0/2703065706 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20007bb0 con 0x7fac28107440 2024-03-21T11:23:24.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.992+0000 7fac26000640 1 -- 172.21.15.47:0/2703065706 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac20007d50 con 0x7fac28107440 2024-03-21T11:23:24.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.992+0000 7fac26000640 1 -- 172.21.15.47:0/2703065706 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20018800 con 0x7fac28107440 2024-03-21T11:23:24.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.992+0000 7fac30000640 1 -- 172.21.15.47:0/2703065706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac281a1ab0 con 0x7fac28107440 2024-03-21T11:23:24.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.992+0000 7fac30000640 1 -- 172.21.15.47:0/2703065706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac281a1fb0 con 0x7fac28107440 2024-03-21T11:23:24.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.993+0000 7fac30000640 1 -- 172.21.15.47:0/2703065706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac28108f30 con 0x7fac28107440 2024-03-21T11:23:24.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.995+0000 7fac26000640 1 -- 172.21.15.47:0/2703065706 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fac2000fa50 con 0x7fac28107440 2024-03-21T11:23:24.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.996+0000 7fac26000640 1 --2- 172.21.15.47:0/2703065706 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac100a2f20 0x7fac100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:24.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.997+0000 7fac26000640 1 -- 172.21.15.47:0/2703065706 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fac200cf790 con 0x7fac28107440 2024-03-21T11:23:24.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.997+0000 7fac2d600640 1 --2- 172.21.15.47:0/2703065706 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac100a2f20 0x7fac100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:24.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:24.997+0000 7fac2d600640 1 --2- 172.21.15.47:0/2703065706 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac100a2f20 0x7fac100a53e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fac1c02c710 tx=0x7fac1c0323c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:25.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.005+0000 7fac26000640 1 -- 172.21.15.47:0/2703065706 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fac20014030 con 0x7fac28107440 2024-03-21T11:23:25.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:24 smithi047 ceph-mon[31351]: pgmap v402: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:23:25.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:24 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3809562757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:25.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:24 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1784524572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:25.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.211+0000 7fac30000640 1 -- 172.21.15.47:0/2703065706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac280622a0 con 0x7fac28107440 2024-03-21T11:23:25.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.211+0000 7fac26000640 1 -- 172.21.15.47:0/2703065706 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac2009c1d0 con 0x7fac28107440 2024-03-21T11:23:25.212 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T11:23:25.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.214+0000 7fac30000640 1 -- 172.21.15.47:0/2703065706 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac100a2f20 msgr2=0x7fac100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:25.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.214+0000 7fac30000640 1 --2- 172.21.15.47:0/2703065706 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac100a2f20 0x7fac100a53e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fac1c02c710 tx=0x7fac1c0323c0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.214+0000 7fac30000640 1 -- 172.21.15.47:0/2703065706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac28107440 msgr2=0x7fac281a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:25.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.214+0000 7fac30000640 1 --2- 172.21.15.47:0/2703065706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac28107440 0x7fac281a1150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac2000c970 tx=0x7fac2000ce30 comp rx=0 tx=0).stop 2024-03-21T11:23:25.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.215+0000 7fac30000640 1 -- 172.21.15.47:0/2703065706 shutdown_connections 2024-03-21T11:23:25.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.215+0000 7fac30000640 1 --2- 172.21.15.47:0/2703065706 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac100a2f20 0x7fac100a53e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.215+0000 7fac30000640 1 --2- 172.21.15.47:0/2703065706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac28107440 0x7fac281a1150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.215+0000 7fac30000640 1 --2- 172.21.15.47:0/2703065706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac28104b10 0x7fac2819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.215+0000 7fac30000640 1 --2- 172.21.15.47:0/2703065706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac281021e0 0x7fac28199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.215+0000 7fac30000640 1 -- 172.21.15.47:0/2703065706 >> 172.21.15.47:0/2703065706 conn(0x7fac280fbb30 msgr2=0x7fac28101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:25.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.215+0000 7fac30000640 1 -- 172.21.15.47:0/2703065706 shutdown_connections 2024-03-21T11:23:25.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.215+0000 7fac30000640 1 -- 172.21.15.47:0/2703065706 wait complete. 2024-03-21T11:23:25.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T11:23:25.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:24 smithi102 ceph-mon[36748]: pgmap v402: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:23:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:24 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3809562757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:24 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1784524572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:25.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:24 smithi100 ceph-mon[36245]: pgmap v402: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:23:25.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:24 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3809562757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:25.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:24 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1784524572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.464+0000 7fa687200640 1 -- 172.21.15.47:0/1989958515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6881049b0 msgr2=0x7fa688104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.464+0000 7fa687200640 1 --2- 172.21.15.47:0/1989958515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6881049b0 0x7fa688104e30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa674009900 tx=0x7fa67402bc40 comp rx=0 tx=0).stop 2024-03-21T11:23:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.464+0000 7fa687200640 1 -- 172.21.15.47:0/1989958515 shutdown_connections 2024-03-21T11:23:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.464+0000 7fa687200640 1 --2- 172.21.15.47:0/1989958515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa688105370 0x7fa68810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.464+0000 7fa687200640 1 --2- 172.21.15.47:0/1989958515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6881049b0 0x7fa688104e30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.464+0000 7fa687200640 1 --2- 172.21.15.47:0/1989958515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6881037c0 0x7fa688103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.464+0000 7fa687200640 1 -- 172.21.15.47:0/1989958515 >> 172.21.15.47:0/1989958515 conn(0x7fa6880fed50 msgr2=0x7fa688101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.464+0000 7fa687200640 1 -- 172.21.15.47:0/1989958515 shutdown_connections 2024-03-21T11:23:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.464+0000 7fa687200640 1 -- 172.21.15.47:0/1989958515 wait complete. 2024-03-21T11:23:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.466+0000 7fa687200640 1 Processor -- start 2024-03-21T11:23:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.466+0000 7fa687200640 1 -- start start 2024-03-21T11:23:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.466+0000 7fa687200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6881037c0 0x7fa688199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.466+0000 7fa687200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6881049b0 0x7fa68819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.466+0000 7fa687200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa688105370 0x7fa6881a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.466+0000 7fa687200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68810f440 con 0x7fa688105370 2024-03-21T11:23:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.466+0000 7fa687200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68810f2c0 con 0x7fa6881037c0 2024-03-21T11:23:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.466+0000 7fa687200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68810f5c0 con 0x7fa6881049b0 2024-03-21T11:23:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.466+0000 7fa685e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6881037c0 0x7fa688199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.467+0000 7fa685400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6881049b0 0x7fa68819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.467+0000 7fa685e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6881037c0 0x7fa688199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38588/0 (socket says 172.21.15.47:38588) 2024-03-21T11:23:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.467+0000 7fa685e00640 1 -- 172.21.15.47:0/1223349109 learned_addr learned my addr 172.21.15.47:0/1223349109 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.467+0000 7fa685400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6881049b0 0x7fa68819a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32894/0 (socket says 172.21.15.47:32894) 2024-03-21T11:23:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.467+0000 7fa686800640 1 --2- 172.21.15.47:0/1223349109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa688105370 0x7fa6881a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.467+0000 7fa685e00640 1 -- 172.21.15.47:0/1223349109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6881049b0 msgr2=0x7fa68819a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.467+0000 7fa685e00640 1 --2- 172.21.15.47:0/1223349109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6881049b0 0x7fa68819a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.467+0000 7fa685e00640 1 -- 172.21.15.47:0/1223349109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa688105370 msgr2=0x7fa6881a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.467+0000 7fa685e00640 1 --2- 172.21.15.47:0/1223349109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa688105370 0x7fa6881a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.467+0000 7fa685e00640 1 -- 172.21.15.47:0/1223349109 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6881a1700 con 0x7fa6881037c0 2024-03-21T11:23:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.467+0000 7fa686800640 1 --2- 172.21.15.47:0/1223349109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa688105370 0x7fa6881a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.467+0000 7fa685e00640 1 --2- 172.21.15.47:0/1223349109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6881037c0 0x7fa688199ae0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa67800d950 tx=0x7fa67800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.468+0000 7fa67ea00640 1 -- 172.21.15.47:0/1223349109 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678017070 con 0x7fa6881037c0 2024-03-21T11:23:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.468+0000 7fa67ea00640 1 -- 172.21.15.47:0/1223349109 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa678009c30 con 0x7fa6881037c0 2024-03-21T11:23:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.468+0000 7fa687200640 1 -- 172.21.15.47:0/1223349109 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6881a19f0 con 0x7fa6881037c0 2024-03-21T11:23:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.468+0000 7fa67ea00640 1 -- 172.21.15.47:0/1223349109 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67800f600 con 0x7fa6881037c0 2024-03-21T11:23:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.468+0000 7fa687200640 1 -- 172.21.15.47:0/1223349109 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6881a1f10 con 0x7fa6881037c0 2024-03-21T11:23:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.469+0000 7fa687200640 1 -- 172.21.15.47:0/1223349109 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68810aa80 con 0x7fa6881037c0 2024-03-21T11:23:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.471+0000 7fa67ea00640 1 -- 172.21.15.47:0/1223349109 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa67801a070 con 0x7fa6881037c0 2024-03-21T11:23:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.472+0000 7fa67ea00640 1 --2- 172.21.15.47:0/1223349109 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa66c0a2fa0 0x7fa66c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.472+0000 7fa67ea00640 1 -- 172.21.15.47:0/1223349109 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa6780cf150 con 0x7fa6881037c0 2024-03-21T11:23:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.472+0000 7fa685400640 1 --2- 172.21.15.47:0/1223349109 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa66c0a2fa0 0x7fa66c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.473+0000 7fa685400640 1 --2- 172.21.15.47:0/1223349109 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa66c0a2fa0 0x7fa66c0a5460 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa67402c7c0 tx=0x7fa674005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:25.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.479+0000 7fa67ea00640 1 -- 172.21.15.47:0/1223349109 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa678009de0 con 0x7fa6881037c0 2024-03-21T11:23:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.684+0000 7fa687200640 1 -- 172.21.15.47:0/1223349109 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa6880622a0 con 0x7fa6881037c0 2024-03-21T11:23:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.685+0000 7fa67ea00640 1 -- 172.21.15.47:0/1223349109 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa67809bbf0 con 0x7fa6881037c0 2024-03-21T11:23:25.686 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:23:25.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.688+0000 7fa687200640 1 -- 172.21.15.47:0/1223349109 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa66c0a2fa0 msgr2=0x7fa66c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:25.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.688+0000 7fa687200640 1 --2- 172.21.15.47:0/1223349109 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa66c0a2fa0 0x7fa66c0a5460 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa67402c7c0 tx=0x7fa674005fd0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.688+0000 7fa687200640 1 -- 172.21.15.47:0/1223349109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6881037c0 msgr2=0x7fa688199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.688+0000 7fa687200640 1 --2- 172.21.15.47:0/1223349109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6881037c0 0x7fa688199ae0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa67800d950 tx=0x7fa67800de10 comp rx=0 tx=0).stop 2024-03-21T11:23:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.688+0000 7fa687200640 1 -- 172.21.15.47:0/1223349109 shutdown_connections 2024-03-21T11:23:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.688+0000 7fa687200640 1 --2- 172.21.15.47:0/1223349109 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa66c0a2fa0 0x7fa66c0a5460 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.688+0000 7fa687200640 1 --2- 172.21.15.47:0/1223349109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa688105370 0x7fa6881a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.688+0000 7fa687200640 1 --2- 172.21.15.47:0/1223349109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6881049b0 0x7fa68819a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.688+0000 7fa687200640 1 --2- 172.21.15.47:0/1223349109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6881037c0 0x7fa688199ae0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.688+0000 7fa687200640 1 -- 172.21.15.47:0/1223349109 >> 172.21.15.47:0/1223349109 conn(0x7fa6880fed50 msgr2=0x7fa688107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.688+0000 7fa687200640 1 -- 172.21.15.47:0/1223349109 shutdown_connections 2024-03-21T11:23:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.688+0000 7fa687200640 1 -- 172.21.15.47:0/1223349109 wait complete. 2024-03-21T11:23:25.708 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:23:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.932+0000 7f8c26400640 1 -- 172.21.15.47:0/840445862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c200772a0 msgr2=0x7f8c20077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.932+0000 7f8c26400640 1 --2- 172.21.15.47:0/840445862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c200772a0 0x7f8c20077700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009ca0 tx=0x7f8c0802c0f0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.932+0000 7f8c26400640 1 -- 172.21.15.47:0/840445862 shutdown_connections 2024-03-21T11:23:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.932+0000 7f8c26400640 1 --2- 172.21.15.47:0/840445862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c20077c40 0x7f8c200784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.932+0000 7f8c26400640 1 --2- 172.21.15.47:0/840445862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c200772a0 0x7f8c20077700 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.932+0000 7f8c26400640 1 --2- 172.21.15.47:0/840445862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c20076060 0x7f8c20076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.932+0000 7f8c26400640 1 -- 172.21.15.47:0/840445862 >> 172.21.15.47:0/840445862 conn(0x7f8c200fee50 msgr2=0x7f8c20101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.932+0000 7f8c26400640 1 -- 172.21.15.47:0/840445862 shutdown_connections 2024-03-21T11:23:25.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.932+0000 7f8c26400640 1 -- 172.21.15.47:0/840445862 wait complete. 2024-03-21T11:23:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.933+0000 7f8c26400640 1 Processor -- start 2024-03-21T11:23:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.933+0000 7f8c26400640 1 -- start start 2024-03-21T11:23:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c26400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c20076060 0x7f8c2019dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c26400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c200772a0 0x7f8c2019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c26400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c20077c40 0x7f8c201a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c26400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c20113930 con 0x7f8c20076060 2024-03-21T11:23:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c26400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c201137b0 con 0x7f8c20077c40 2024-03-21T11:23:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c26400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c20113ab0 con 0x7f8c200772a0 2024-03-21T11:23:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c1ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c200772a0 0x7f8c2019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c1fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c20077c40 0x7f8c201a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c1f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c20076060 0x7f8c2019dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c1ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c200772a0 0x7f8c2019e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32908/0 (socket says 172.21.15.47:32908) 2024-03-21T11:23:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c1ea00640 1 -- 172.21.15.47:0/3208184873 learned_addr learned my addr 172.21.15.47:0/3208184873 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c1ea00640 1 -- 172.21.15.47:0/3208184873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c20077c40 msgr2=0x7f8c201a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c1ea00640 1 --2- 172.21.15.47:0/3208184873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c20077c40 0x7f8c201a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c1ea00640 1 -- 172.21.15.47:0/3208184873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c20076060 msgr2=0x7f8c2019dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c1ea00640 1 --2- 172.21.15.47:0/3208184873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c20076060 0x7f8c2019dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c1ea00640 1 -- 172.21.15.47:0/3208184873 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c201a5bd0 con 0x7f8c200772a0 2024-03-21T11:23:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.934+0000 7f8c1fe00640 1 --2- 172.21.15.47:0/3208184873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c20077c40 0x7f8c201a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.935+0000 7f8c1ea00640 1 --2- 172.21.15.47:0/3208184873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c200772a0 0x7f8c2019e4f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8c080325f0 tx=0x7f8c08032ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.935+0000 7f8c17e00640 1 -- 172.21.15.47:0/3208184873 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08041020 con 0x7f8c200772a0 2024-03-21T11:23:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.935+0000 7f8c17e00640 1 -- 172.21.15.47:0/3208184873 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c08034cd0 con 0x7f8c200772a0 2024-03-21T11:23:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.935+0000 7f8c17e00640 1 -- 172.21.15.47:0/3208184873 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08039040 con 0x7f8c200772a0 2024-03-21T11:23:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.935+0000 7f8c26400640 1 -- 172.21.15.47:0/3208184873 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c201a5e60 con 0x7f8c200772a0 2024-03-21T11:23:25.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.935+0000 7f8c26400640 1 -- 172.21.15.47:0/3208184873 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c201a6210 con 0x7f8c200772a0 2024-03-21T11:23:25.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.936+0000 7f8c26400640 1 -- 172.21.15.47:0/3208184873 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c20110400 con 0x7f8c200772a0 2024-03-21T11:23:25.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.938+0000 7f8c17e00640 1 -- 172.21.15.47:0/3208184873 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8c08044020 con 0x7f8c200772a0 2024-03-21T11:23:25.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.940+0000 7f8c17e00640 1 --2- 172.21.15.47:0/3208184873 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8c000a2f20 0x7f8c000a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:25.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.940+0000 7f8c17e00640 1 -- 172.21.15.47:0/3208184873 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8c080f1580 con 0x7f8c200772a0 2024-03-21T11:23:25.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.940+0000 7f8c1f400640 1 --2- 172.21.15.47:0/3208184873 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8c000a2f20 0x7f8c000a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:25.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.941+0000 7f8c1f400640 1 --2- 172.21.15.47:0/3208184873 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8c000a2f20 0x7f8c000a53e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009bd0 tx=0x7f8c10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:25.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:25.949+0000 7f8c17e00640 1 -- 172.21.15.47:0/3208184873 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8c0804b020 con 0x7f8c200772a0 2024-03-21T11:23:26.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.199+0000 7f8c26400640 1 -- 172.21.15.47:0/3208184873 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c2002d070 con 0x7f8c200772a0 2024-03-21T11:23:26.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.201+0000 7f8c17e00640 1 -- 172.21.15.47:0/3208184873 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f8c080be050 con 0x7f8c200772a0 2024-03-21T11:23:26.202 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:26.202 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:21:18.592954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24524,24523,24535,24581,24553]},"id":1}]} 2024-03-21T11:23:26.202 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 27 2024-03-21T11:23:26.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.203+0000 7f8c26400640 1 -- 172.21.15.47:0/3208184873 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8c000a2f20 msgr2=0x7f8c000a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:26.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.203+0000 7f8c26400640 1 --2- 172.21.15.47:0/3208184873 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8c000a2f20 0x7f8c000a53e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009bd0 tx=0x7f8c10005f90 comp rx=0 tx=0).stop 2024-03-21T11:23:26.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.203+0000 7f8c26400640 1 -- 172.21.15.47:0/3208184873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c200772a0 msgr2=0x7f8c2019e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:26.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.203+0000 7f8c26400640 1 --2- 172.21.15.47:0/3208184873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c200772a0 0x7f8c2019e4f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8c080325f0 tx=0x7f8c08032ec0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.204+0000 7f8c26400640 1 -- 172.21.15.47:0/3208184873 shutdown_connections 2024-03-21T11:23:26.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.204+0000 7f8c26400640 1 --2- 172.21.15.47:0/3208184873 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8c000a2f20 0x7f8c000a53e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.204+0000 7f8c26400640 1 --2- 172.21.15.47:0/3208184873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c20077c40 0x7f8c201a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.204+0000 7f8c26400640 1 --2- 172.21.15.47:0/3208184873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c200772a0 0x7f8c2019e4f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.204+0000 7f8c26400640 1 --2- 172.21.15.47:0/3208184873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c20076060 0x7f8c2019dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.204+0000 7f8c26400640 1 -- 172.21.15.47:0/3208184873 >> 172.21.15.47:0/3208184873 conn(0x7f8c200fee50 msgr2=0x7f8c20100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:26.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.204+0000 7f8c26400640 1 -- 172.21.15.47:0/3208184873 shutdown_connections 2024-03-21T11:23:26.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.204+0000 7f8c26400640 1 -- 172.21.15.47:0/3208184873 wait complete. 2024-03-21T11:23:26.220 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:25 smithi047 ceph-mon[31351]: pgmap v403: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:23:26.220 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:25 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2703065706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:26.221 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:25 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1223349109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:26.223 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:23:26.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:25 smithi102 ceph-mon[36748]: pgmap v403: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:23:26.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:25 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2703065706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:26.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:25 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1223349109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:26.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:25 smithi100 ceph-mon[36245]: pgmap v403: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:23:26.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:25 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2703065706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:26.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:25 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1223349109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.442+0000 7f3cc2e00640 1 -- 172.21.15.47:0/2486584512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cbc108cf0 msgr2=0x7f3cbc109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:26.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.442+0000 7f3cc2e00640 1 --2- 172.21.15.47:0/2486584512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cbc108cf0 0x7f3cbc109170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009900 tx=0x7f3ca802bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:26.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.442+0000 7f3cc2e00640 1 -- 172.21.15.47:0/2486584512 shutdown_connections 2024-03-21T11:23:26.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.442+0000 7f3cc2e00640 1 --2- 172.21.15.47:0/2486584512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cbc1096b0 0x7f3cbc10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.442+0000 7f3cc2e00640 1 --2- 172.21.15.47:0/2486584512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cbc108cf0 0x7f3cbc109170 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.442+0000 7f3cc2e00640 1 --2- 172.21.15.47:0/2486584512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cbc107b00 0x7f3cbc107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.442+0000 7f3cc2e00640 1 -- 172.21.15.47:0/2486584512 >> 172.21.15.47:0/2486584512 conn(0x7f3cbc076ab0 msgr2=0x7f3cbc078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:26.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.442+0000 7f3cc2e00640 1 -- 172.21.15.47:0/2486584512 shutdown_connections 2024-03-21T11:23:26.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.442+0000 7f3cc2e00640 1 -- 172.21.15.47:0/2486584512 wait complete. 2024-03-21T11:23:26.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.443+0000 7f3cc2e00640 1 Processor -- start 2024-03-21T11:23:26.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.443+0000 7f3cc2e00640 1 -- start start 2024-03-21T11:23:26.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc2e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cbc107b00 0x7f3cbc19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:26.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc2e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cbc108cf0 0x7f3cbc19e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:26.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc2e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cbc1096b0 0x7f3cbc1a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:26.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc2e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc1137d0 con 0x7f3cbc108cf0 2024-03-21T11:23:26.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc2e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc113650 con 0x7f3cbc107b00 2024-03-21T11:23:26.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc2e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc113950 con 0x7f3cbc1096b0 2024-03-21T11:23:26.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cbc107b00 0x7f3cbc19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:26.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cbc107b00 0x7f3cbc19dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38638/0 (socket says 172.21.15.47:38638) 2024-03-21T11:23:26.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc1a00640 1 -- 172.21.15.47:0/1811893862 learned_addr learned my addr 172.21.15.47:0/1811893862 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:26.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc2400640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cbc1096b0 0x7f3cbc1a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc1000640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cbc108cf0 0x7f3cbc19e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc1a00640 1 -- 172.21.15.47:0/1811893862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cbc1096b0 msgr2=0x7f3cbc1a5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc1a00640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cbc1096b0 0x7f3cbc1a5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc1a00640 1 -- 172.21.15.47:0/1811893862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cbc108cf0 msgr2=0x7f3cbc19e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc1a00640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cbc108cf0 0x7f3cbc19e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc1a00640 1 -- 172.21.15.47:0/1811893862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc1a5970 con 0x7f3cbc107b00 2024-03-21T11:23:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.444+0000 7f3cc1000640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cbc108cf0 0x7f3cbc19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.445+0000 7f3cc1a00640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cbc107b00 0x7f3cbc19dd50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000c970 tx=0x7f3cb000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.445+0000 7f3cba000640 1 -- 172.21.15.47:0/1811893862 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0007c10 con 0x7f3cbc107b00 2024-03-21T11:23:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.445+0000 7f3cba000640 1 -- 172.21.15.47:0/1811893862 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cb0007db0 con 0x7f3cbc107b00 2024-03-21T11:23:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.445+0000 7f3cba000640 1 -- 172.21.15.47:0/1811893862 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb000f940 con 0x7f3cbc107b00 2024-03-21T11:23:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.445+0000 7f3cc2e00640 1 -- 172.21.15.47:0/1811893862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cbc1a5c60 con 0x7f3cbc107b00 2024-03-21T11:23:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.445+0000 7f3cc2e00640 1 -- 172.21.15.47:0/1811893862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cbc1a61d0 con 0x7f3cbc107b00 2024-03-21T11:23:26.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.446+0000 7f3cc2e00640 1 -- 172.21.15.47:0/1811893862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cbc10ed50 con 0x7f3cbc107b00 2024-03-21T11:23:26.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.448+0000 7f3cba000640 1 -- 172.21.15.47:0/1811893862 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3cb000fae0 con 0x7f3cbc107b00 2024-03-21T11:23:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.449+0000 7f3cba000640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ca40a2fa0 0x7f3ca40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.450+0000 7f3cba000640 1 -- 172.21.15.47:0/1811893862 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3cb00cfb30 con 0x7f3cbc107b00 2024-03-21T11:23:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.450+0000 7f3cc1000640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ca40a2fa0 0x7f3ca40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.451+0000 7f3cc1000640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ca40a2fa0 0x7f3ca40a5460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3ca802c240 tx=0x7f3ca802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.453+0000 7f3cba000640 1 -- 172.21.15.47:0/1811893862 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3cb009c600 con 0x7f3cbc107b00 2024-03-21T11:23:26.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.656+0000 7f3cc2e00640 1 -- 172.21.15.47:0/1811893862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cbc062340 con 0x7f3cbc107b00 2024-03-21T11:23:26.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.658+0000 7f3cba000640 1 -- 172.21.15.47:0/1811893862 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24747 (secure 0 0 0) 0x7f3cb009bf00 con 0x7f3cbc107b00 2024-03-21T11:23:26.658 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:26.658 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:22:50.841892+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:23:26.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.660+0000 7f3cc2e00640 1 -- 172.21.15.47:0/1811893862 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ca40a2fa0 msgr2=0x7f3ca40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.660+0000 7f3cc2e00640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ca40a2fa0 0x7f3ca40a5460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3ca802c240 tx=0x7f3ca802df90 comp rx=0 tx=0).stop 2024-03-21T11:23:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.660+0000 7f3cc2e00640 1 -- 172.21.15.47:0/1811893862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cbc107b00 msgr2=0x7f3cbc19dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.660+0000 7f3cc2e00640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cbc107b00 0x7f3cbc19dd50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000c970 tx=0x7f3cb000ce30 comp rx=0 tx=0).stop 2024-03-21T11:23:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.661+0000 7f3cc2e00640 1 -- 172.21.15.47:0/1811893862 shutdown_connections 2024-03-21T11:23:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.661+0000 7f3cc2e00640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ca40a2fa0 0x7f3ca40a5460 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.661+0000 7f3cc2e00640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cbc1096b0 0x7f3cbc1a5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.661+0000 7f3cc2e00640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cbc108cf0 0x7f3cbc19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.661+0000 7f3cc2e00640 1 --2- 172.21.15.47:0/1811893862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cbc107b00 0x7f3cbc19dd50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.661+0000 7f3cc2e00640 1 -- 172.21.15.47:0/1811893862 >> 172.21.15.47:0/1811893862 conn(0x7f3cbc076ab0 msgr2=0x7f3cbc10bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.661+0000 7f3cc2e00640 1 -- 172.21.15.47:0/1811893862 shutdown_connections 2024-03-21T11:23:26.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.661+0000 7f3cc2e00640 1 -- 172.21.15.47:0/1811893862 wait complete. 2024-03-21T11:23:26.679 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T11:15:35.545245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T11:20:47.016174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T11:20:48.108506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T11:20:52.254154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T11:23:26.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T11:23:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.907+0000 7f4a67400640 1 -- 172.21.15.47:0/2723205841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a60102120 msgr2=0x7f4a60104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.907+0000 7f4a67400640 1 --2- 172.21.15.47:0/2723205841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a60102120 0x7f4a60104510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009900 tx=0x7f4a5002bc40 comp rx=0 tx=0).stop 2024-03-21T11:23:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.907+0000 7f4a67400640 1 -- 172.21.15.47:0/2723205841 shutdown_connections 2024-03-21T11:23:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.907+0000 7f4a67400640 1 --2- 172.21.15.47:0/2723205841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a60107380 0x7f4a601097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.907+0000 7f4a67400640 1 --2- 172.21.15.47:0/2723205841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a60104a50 0x7f4a60106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.907+0000 7f4a67400640 1 --2- 172.21.15.47:0/2723205841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a60102120 0x7f4a60104510 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.907+0000 7f4a67400640 1 -- 172.21.15.47:0/2723205841 >> 172.21.15.47:0/2723205841 conn(0x7f4a600fbb30 msgr2=0x7f4a600fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.907+0000 7f4a67400640 1 -- 172.21.15.47:0/2723205841 shutdown_connections 2024-03-21T11:23:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.907+0000 7f4a67400640 1 -- 172.21.15.47:0/2723205841 wait complete. 2024-03-21T11:23:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.908+0000 7f4a67400640 1 Processor -- start 2024-03-21T11:23:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.908+0000 7f4a67400640 1 -- start start 2024-03-21T11:23:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a67400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a60102120 0x7f4a60199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a67400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a60104a50 0x7f4a6019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a67400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a60107380 0x7f4a601a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a67400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6010d380 con 0x7f4a60104a50 2024-03-21T11:23:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a67400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6010d200 con 0x7f4a60107380 2024-03-21T11:23:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a67400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6010d500 con 0x7f4a60102120 2024-03-21T11:23:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a64a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a60102120 0x7f4a60199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a64a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a60102120 0x7f4a60199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32944/0 (socket says 172.21.15.47:32944) 2024-03-21T11:23:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a64a00640 1 -- 172.21.15.47:0/673723428 learned_addr learned my addr 172.21.15.47:0/673723428 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a65400640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a60107380 0x7f4a601a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a5fe00640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a60104a50 0x7f4a6019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a64a00640 1 -- 172.21.15.47:0/673723428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a60107380 msgr2=0x7f4a601a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a64a00640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a60107380 0x7f4a601a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a64a00640 1 -- 172.21.15.47:0/673723428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a60104a50 msgr2=0x7f4a6019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.909+0000 7f4a64a00640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a60104a50 0x7f4a6019a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.910+0000 7f4a64a00640 1 -- 172.21.15.47:0/673723428 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a601a1860 con 0x7f4a60102120 2024-03-21T11:23:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.910+0000 7f4a65400640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a60107380 0x7f4a601a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.910+0000 7f4a64a00640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a60102120 0x7f4a60199c50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4a5002c8f0 tx=0x7f4a5002c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.910+0000 7f4a5d600640 1 -- 172.21.15.47:0/673723428 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50041070 con 0x7f4a60102120 2024-03-21T11:23:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.910+0000 7f4a67400640 1 -- 172.21.15.47:0/673723428 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a601a1af0 con 0x7f4a60102120 2024-03-21T11:23:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.910+0000 7f4a5d600640 1 -- 172.21.15.47:0/673723428 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a50004450 con 0x7f4a60102120 2024-03-21T11:23:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.910+0000 7f4a5d600640 1 -- 172.21.15.47:0/673723428 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50003df0 con 0x7f4a60102120 2024-03-21T11:23:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.910+0000 7f4a67400640 1 -- 172.21.15.47:0/673723428 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a601a1fe0 con 0x7f4a60102120 2024-03-21T11:23:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.912+0000 7f4a67400640 1 -- 172.21.15.47:0/673723428 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a60108ed0 con 0x7f4a60102120 2024-03-21T11:23:26.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.913+0000 7f4a5d600640 1 -- 172.21.15.47:0/673723428 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4a50044070 con 0x7f4a60102120 2024-03-21T11:23:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.914+0000 7f4a5d600640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4a400a2f20 0x7f4a400a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.915+0000 7f4a5d600640 1 -- 172.21.15.47:0/673723428 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4a500f1370 con 0x7f4a60102120 2024-03-21T11:23:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.915+0000 7f4a5fe00640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4a400a2f20 0x7f4a400a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:26.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.915+0000 7f4a5fe00640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4a400a2f20 0x7f4a400a53e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4a60069360 tx=0x7f4a48009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:26.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:26.921+0000 7f4a5d600640 1 -- 172.21.15.47:0/673723428 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4a500bddb0 con 0x7f4a60102120 2024-03-21T11:23:27.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.125+0000 7f4a67400640 1 -- 172.21.15.47:0/673723428 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4a600622a0 con 0x7f4a60102120 2024-03-21T11:23:27.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.125+0000 7f4a5d600640 1 -- 172.21.15.47:0/673723428 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4a500bd6b0 con 0x7f4a60102120 2024-03-21T11:23:27.125 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T11:23:27.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.128+0000 7f4a67400640 1 -- 172.21.15.47:0/673723428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4a400a2f20 msgr2=0x7f4a400a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:27.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.128+0000 7f4a67400640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4a400a2f20 0x7f4a400a53e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4a60069360 tx=0x7f4a48009500 comp rx=0 tx=0).stop 2024-03-21T11:23:27.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.128+0000 7f4a67400640 1 -- 172.21.15.47:0/673723428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a60102120 msgr2=0x7f4a60199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:27.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.128+0000 7f4a67400640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a60102120 0x7f4a60199c50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4a5002c8f0 tx=0x7f4a5002c660 comp rx=0 tx=0).stop 2024-03-21T11:23:27.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.128+0000 7f4a67400640 1 -- 172.21.15.47:0/673723428 shutdown_connections 2024-03-21T11:23:27.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.128+0000 7f4a67400640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4a400a2f20 0x7f4a400a53e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.128+0000 7f4a67400640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a60107380 0x7f4a601a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.128+0000 7f4a67400640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a60104a50 0x7f4a6019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.128+0000 7f4a67400640 1 --2- 172.21.15.47:0/673723428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a60102120 0x7f4a60199c50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.128+0000 7f4a67400640 1 -- 172.21.15.47:0/673723428 >> 172.21.15.47:0/673723428 conn(0x7f4a600fbb30 msgr2=0x7f4a60101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:27.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.128+0000 7f4a67400640 1 -- 172.21.15.47:0/673723428 shutdown_connections 2024-03-21T11:23:27.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.128+0000 7f4a67400640 1 -- 172.21.15.47:0/673723428 wait complete. 2024-03-21T11:23:27.146 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T11:23:27.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:26 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3208184873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:23:27.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:26 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1811893862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:27.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:26 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3208184873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:23:27.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:26 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1811893862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.371+0000 7fd495c00640 1 -- 172.21.15.47:0/2909164151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd490104a50 msgr2=0x7fd490106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.371+0000 7fd495c00640 1 --2- 172.21.15.47:0/2909164151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd490104a50 0x7fd490106e40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd478009900 tx=0x7fd47802bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.371+0000 7fd495c00640 1 -- 172.21.15.47:0/2909164151 shutdown_connections 2024-03-21T11:23:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.371+0000 7fd495c00640 1 --2- 172.21.15.47:0/2909164151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd490107380 0x7fd4901097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.371+0000 7fd495c00640 1 --2- 172.21.15.47:0/2909164151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd490104a50 0x7fd490106e40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.371+0000 7fd495c00640 1 --2- 172.21.15.47:0/2909164151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd490102120 0x7fd490104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.371+0000 7fd495c00640 1 -- 172.21.15.47:0/2909164151 >> 172.21.15.47:0/2909164151 conn(0x7fd4900fbb30 msgr2=0x7fd4900fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:27.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.371+0000 7fd495c00640 1 -- 172.21.15.47:0/2909164151 shutdown_connections 2024-03-21T11:23:27.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.371+0000 7fd495c00640 1 -- 172.21.15.47:0/2909164151 wait complete. 2024-03-21T11:23:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.372+0000 7fd495c00640 1 Processor -- start 2024-03-21T11:23:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.372+0000 7fd495c00640 1 -- start start 2024-03-21T11:23:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.373+0000 7fd495c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd490102120 0x7fd490101a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.373+0000 7fd495c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd490104a50 0x7fd4901000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.373+0000 7fd495c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd490107380 0x7fd490100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.373+0000 7fd495c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49010d360 con 0x7fd490107380 2024-03-21T11:23:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.373+0000 7fd495c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49010d1e0 con 0x7fd490104a50 2024-03-21T11:23:27.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.373+0000 7fd495c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49010d4e0 con 0x7fd490102120 2024-03-21T11:23:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.373+0000 7fd48ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd490102120 0x7fd490101a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.373+0000 7fd48ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd490102120 0x7fd490101a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32958/0 (socket says 172.21.15.47:32958) 2024-03-21T11:23:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.373+0000 7fd48ea00640 1 -- 172.21.15.47:0/1266484322 learned_addr learned my addr 172.21.15.47:0/1266484322 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.373+0000 7fd48e000640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd490104a50 0x7fd4901000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.373+0000 7fd48f400640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd490107380 0x7fd490100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.374+0000 7fd48ea00640 1 -- 172.21.15.47:0/1266484322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd490104a50 msgr2=0x7fd4901000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.374+0000 7fd48ea00640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd490104a50 0x7fd4901000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.374+0000 7fd48ea00640 1 -- 172.21.15.47:0/1266484322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd490107380 msgr2=0x7fd490100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.374+0000 7fd48ea00640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd490107380 0x7fd490100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.374+0000 7fd48ea00640 1 -- 172.21.15.47:0/1266484322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd490100ea0 con 0x7fd490102120 2024-03-21T11:23:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.374+0000 7fd48e000640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd490104a50 0x7fd4901000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.374+0000 7fd48ea00640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd490102120 0x7fd490101a90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd48000e9f0 tx=0x7fd48000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.374+0000 7fd487400640 1 -- 172.21.15.47:0/1266484322 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48000cd80 con 0x7fd490102120 2024-03-21T11:23:27.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.374+0000 7fd487400640 1 -- 172.21.15.47:0/1266484322 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd480010910 con 0x7fd490102120 2024-03-21T11:23:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.374+0000 7fd495c00640 1 -- 172.21.15.47:0/1266484322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4901a1b20 con 0x7fd490102120 2024-03-21T11:23:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.374+0000 7fd487400640 1 -- 172.21.15.47:0/1266484322 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480018950 con 0x7fd490102120 2024-03-21T11:23:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.375+0000 7fd495c00640 1 -- 172.21.15.47:0/1266484322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4901a2090 con 0x7fd490102120 2024-03-21T11:23:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.376+0000 7fd495c00640 1 -- 172.21.15.47:0/1266484322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4900fd620 con 0x7fd490102120 2024-03-21T11:23:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.378+0000 7fd487400640 1 -- 172.21.15.47:0/1266484322 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd480010ac0 con 0x7fd490102120 2024-03-21T11:23:27.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.379+0000 7fd487400640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd4740a2fa0 0x7fd4740a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:27.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.379+0000 7fd487400640 1 -- 172.21.15.47:0/1266484322 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd4800cfab0 con 0x7fd490102120 2024-03-21T11:23:27.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.379+0000 7fd48e000640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd4740a2fa0 0x7fd4740a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:27.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.380+0000 7fd48e000640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd4740a2fa0 0x7fd4740a5460 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd47802c170 tx=0x7fd47802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:27.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.388+0000 7fd487400640 1 -- 172.21.15.47:0/1266484322 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd48009bea0 con 0x7fd490102120 2024-03-21T11:23:27.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:26 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3208184873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:23:27.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:26 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1811893862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:27.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.579+0000 7fd495c00640 1 -- 172.21.15.47:0/1266484322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd4900622d0 con 0x7fd490102120 2024-03-21T11:23:27.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.579+0000 7fd487400640 1 -- 172.21.15.47:0/1266484322 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd48009ff40 con 0x7fd490102120 2024-03-21T11:23:27.579 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:23:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.582+0000 7fd495c00640 1 -- 172.21.15.47:0/1266484322 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd4740a2fa0 msgr2=0x7fd4740a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.582+0000 7fd495c00640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd4740a2fa0 0x7fd4740a5460 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd47802c170 tx=0x7fd47802df90 comp rx=0 tx=0).stop 2024-03-21T11:23:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.582+0000 7fd495c00640 1 -- 172.21.15.47:0/1266484322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd490102120 msgr2=0x7fd490101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.582+0000 7fd495c00640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd490102120 0x7fd490101a90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd48000e9f0 tx=0x7fd48000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.582+0000 7fd495c00640 1 -- 172.21.15.47:0/1266484322 shutdown_connections 2024-03-21T11:23:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.582+0000 7fd495c00640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd4740a2fa0 0x7fd4740a5460 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.582+0000 7fd495c00640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd490107380 0x7fd490100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.582+0000 7fd495c00640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd490104a50 0x7fd4901000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.582+0000 7fd495c00640 1 --2- 172.21.15.47:0/1266484322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd490102120 0x7fd490101a90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.582+0000 7fd495c00640 1 -- 172.21.15.47:0/1266484322 >> 172.21.15.47:0/1266484322 conn(0x7fd4900fbb30 msgr2=0x7fd490102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:27.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.582+0000 7fd495c00640 1 -- 172.21.15.47:0/1266484322 shutdown_connections 2024-03-21T11:23:27.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.582+0000 7fd495c00640 1 -- 172.21.15.47:0/1266484322 wait complete. 2024-03-21T11:23:27.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T11:23:27.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.812+0000 7fbad5200640 1 -- 172.21.15.47:0/2242018376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbad010c0e0 msgr2=0x7fbad010e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.812+0000 7fbad5200640 1 --2- 172.21.15.47:0/2242018376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbad010c0e0 0x7fbad010e4d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009900 tx=0x7fbab802bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.813+0000 7fbad5200640 1 -- 172.21.15.47:0/2242018376 shutdown_connections 2024-03-21T11:23:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.813+0000 7fbad5200640 1 --2- 172.21.15.47:0/2242018376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbad010eaa0 0x7fbad0110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.813+0000 7fbad5200640 1 --2- 172.21.15.47:0/2242018376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbad010c0e0 0x7fbad010e4d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.813+0000 7fbad5200640 1 --2- 172.21.15.47:0/2242018376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad01097b0 0x7fbad010bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.813+0000 7fbad5200640 1 -- 172.21.15.47:0/2242018376 >> 172.21.15.47:0/2242018376 conn(0x7fbad006e8a0 msgr2=0x7fbad0070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.813+0000 7fbad5200640 1 -- 172.21.15.47:0/2242018376 shutdown_connections 2024-03-21T11:23:27.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.813+0000 7fbad5200640 1 -- 172.21.15.47:0/2242018376 wait complete. 2024-03-21T11:23:27.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.814+0000 7fbad5200640 1 Processor -- start 2024-03-21T11:23:27.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.814+0000 7fbad5200640 1 -- start start 2024-03-21T11:23:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbad5200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbad01097b0 0x7fbad01a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbad5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbad010c0e0 0x7fbad01a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbad5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad010eaa0 0x7fbad01a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbad5200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad0115bb0 con 0x7fbad01097b0 2024-03-21T11:23:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbad5200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad0115a30 con 0x7fbad010eaa0 2024-03-21T11:23:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbad5200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad0115d30 con 0x7fbad010c0e0 2024-03-21T11:23:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbacde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbad01097b0 0x7fbad01a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbacde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbad01097b0 0x7fbad01a24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33776/0 (socket says 172.21.15.47:33776) 2024-03-21T11:23:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbacde00640 1 -- 172.21.15.47:0/2357613185 learned_addr learned my addr 172.21.15.47:0/2357613185 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbac7e00640 1 --2- 172.21.15.47:0/2357613185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbad010c0e0 0x7fbad01a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbace800640 1 --2- 172.21.15.47:0/2357613185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad010eaa0 0x7fbad01a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:27.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbac7e00640 1 -- 172.21.15.47:0/2357613185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad010eaa0 msgr2=0x7fbad01a9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbac7e00640 1 --2- 172.21.15.47:0/2357613185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad010eaa0 0x7fbad01a9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbac7e00640 1 -- 172.21.15.47:0/2357613185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbad01097b0 msgr2=0x7fbad01a24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbac7e00640 1 --2- 172.21.15.47:0/2357613185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbad01097b0 0x7fbad01a24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.815+0000 7fbac7e00640 1 -- 172.21.15.47:0/2357613185 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad01aa0d0 con 0x7fbad010c0e0 2024-03-21T11:23:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.816+0000 7fbac7e00640 1 --2- 172.21.15.47:0/2357613185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbad010c0e0 0x7fbad01a29f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009230 tx=0x7fbab802c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.816+0000 7fbac6a00640 1 -- 172.21.15.47:0/2357613185 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab8043020 con 0x7fbad010c0e0 2024-03-21T11:23:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.816+0000 7fbac6a00640 1 -- 172.21.15.47:0/2357613185 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbab80054e0 con 0x7fbad010c0e0 2024-03-21T11:23:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.816+0000 7fbac6a00640 1 -- 172.21.15.47:0/2357613185 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab8003c60 con 0x7fbad010c0e0 2024-03-21T11:23:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.816+0000 7fbad5200640 1 -- 172.21.15.47:0/2357613185 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad01aa360 con 0x7fbad010c0e0 2024-03-21T11:23:27.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.816+0000 7fbad5200640 1 -- 172.21.15.47:0/2357613185 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad01aa8d0 con 0x7fbad010c0e0 2024-03-21T11:23:27.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.817+0000 7fbad5200640 1 -- 172.21.15.47:0/2357613185 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad0110620 con 0x7fbad010c0e0 2024-03-21T11:23:27.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.819+0000 7fbac6a00640 1 -- 172.21.15.47:0/2357613185 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbab8046020 con 0x7fbad010c0e0 2024-03-21T11:23:27.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.821+0000 7fbac6a00640 1 --2- 172.21.15.47:0/2357613185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbab40a2fa0 0x7fbab40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:27.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.821+0000 7fbac6a00640 1 -- 172.21.15.47:0/2357613185 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbab80f1aa0 con 0x7fbad010c0e0 2024-03-21T11:23:27.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.821+0000 7fbacde00640 1 --2- 172.21.15.47:0/2357613185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbab40a2fa0 0x7fbab40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:27.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.822+0000 7fbacde00640 1 --2- 172.21.15.47:0/2357613185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbab40a2fa0 0x7fbab40a5460 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009c70 tx=0x7fbac0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:27.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:27.826+0000 7fbac6a00640 1 -- 172.21.15.47:0/2357613185 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbab804d020 con 0x7fbad010c0e0 2024-03-21T11:23:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.034+0000 7fbad5200640 1 -- 172.21.15.47:0/2357613185 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbad002d2c0 con 0x7fbad010c0e0 2024-03-21T11:23:28.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.035+0000 7fbac6a00640 1 -- 172.21.15.47:0/2357613185 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbab80be4e0 con 0x7fbad010c0e0 2024-03-21T11:23:28.035 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T11:23:28.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.037+0000 7fbad5200640 1 -- 172.21.15.47:0/2357613185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbab40a2fa0 msgr2=0x7fbab40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:28.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.037+0000 7fbad5200640 1 --2- 172.21.15.47:0/2357613185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbab40a2fa0 0x7fbab40a5460 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009c70 tx=0x7fbac0005f90 comp rx=0 tx=0).stop 2024-03-21T11:23:28.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.037+0000 7fbad5200640 1 -- 172.21.15.47:0/2357613185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbad010c0e0 msgr2=0x7fbad01a29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:28.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.037+0000 7fbad5200640 1 --2- 172.21.15.47:0/2357613185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbad010c0e0 0x7fbad01a29f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009230 tx=0x7fbab802c6c0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.038+0000 7fbad5200640 1 -- 172.21.15.47:0/2357613185 shutdown_connections 2024-03-21T11:23:28.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.038+0000 7fbad5200640 1 --2- 172.21.15.47:0/2357613185 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbab40a2fa0 0x7fbab40a5460 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.038+0000 7fbad5200640 1 --2- 172.21.15.47:0/2357613185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad010eaa0 0x7fbad01a9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.038+0000 7fbad5200640 1 --2- 172.21.15.47:0/2357613185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbad010c0e0 0x7fbad01a29f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.038+0000 7fbad5200640 1 --2- 172.21.15.47:0/2357613185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbad01097b0 0x7fbad01a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.038+0000 7fbad5200640 1 -- 172.21.15.47:0/2357613185 >> 172.21.15.47:0/2357613185 conn(0x7fbad006e8a0 msgr2=0x7fbad01134d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:28.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.038+0000 7fbad5200640 1 -- 172.21.15.47:0/2357613185 shutdown_connections 2024-03-21T11:23:28.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.038+0000 7fbad5200640 1 -- 172.21.15.47:0/2357613185 wait complete. 2024-03-21T11:23:28.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T11:23:28.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:27 smithi102 ceph-mon[36748]: pgmap v404: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:23:28.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:27 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/673723428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:28.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:27 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1266484322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:28.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.291+0000 7f4186800640 1 -- 172.21.15.47:0/196337855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801037c0 msgr2=0x7f4180103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:28.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.291+0000 7f4186800640 1 --2- 172.21.15.47:0/196337855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801037c0 0x7f4180103be0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4174009900 tx=0x7f417402bc40 comp rx=0 tx=0).stop 2024-03-21T11:23:28.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.291+0000 7f4186800640 1 -- 172.21.15.47:0/196337855 shutdown_connections 2024-03-21T11:23:28.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.291+0000 7f4186800640 1 --2- 172.21.15.47:0/196337855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4180105370 0x7f418010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.291+0000 7f4186800640 1 --2- 172.21.15.47:0/196337855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41801049b0 0x7f4180104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.291+0000 7f4186800640 1 --2- 172.21.15.47:0/196337855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801037c0 0x7f4180103be0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.291+0000 7f4186800640 1 -- 172.21.15.47:0/196337855 >> 172.21.15.47:0/196337855 conn(0x7f41800fed50 msgr2=0x7f4180101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:28.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.291+0000 7f4186800640 1 -- 172.21.15.47:0/196337855 shutdown_connections 2024-03-21T11:23:28.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.291+0000 7f4186800640 1 -- 172.21.15.47:0/196337855 wait complete. 2024-03-21T11:23:28.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.292+0000 7f4186800640 1 Processor -- start 2024-03-21T11:23:28.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.292+0000 7f4186800640 1 -- start start 2024-03-21T11:23:28.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.292+0000 7f4186800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801037c0 0x7f4180199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:28.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.292+0000 7f4186800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41801049b0 0x7f4180199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:28.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.292+0000 7f4186800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4180105370 0x7f41801a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:28.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.292+0000 7f4186800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418010f440 con 0x7f41801049b0 2024-03-21T11:23:28.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.292+0000 7f4186800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418010f2c0 con 0x7f4180105370 2024-03-21T11:23:28.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.292+0000 7f4186800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418010f5c0 con 0x7f41801037c0 2024-03-21T11:23:28.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.293+0000 7f4185400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801037c0 0x7f4180199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:28.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.293+0000 7f4185400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801037c0 0x7f4180199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33012/0 (socket says 172.21.15.47:33012) 2024-03-21T11:23:28.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.293+0000 7f4185400640 1 -- 172.21.15.47:0/3024365884 learned_addr learned my addr 172.21.15.47:0/3024365884 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.293+0000 7f4184a00640 1 --2- 172.21.15.47:0/3024365884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41801049b0 0x7f4180199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.293+0000 7f4185e00640 1 --2- 172.21.15.47:0/3024365884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4180105370 0x7f41801a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.293+0000 7f4185400640 1 -- 172.21.15.47:0/3024365884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4180105370 msgr2=0x7f41801a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.293+0000 7f4185400640 1 --2- 172.21.15.47:0/3024365884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4180105370 0x7f41801a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.293+0000 7f4185400640 1 -- 172.21.15.47:0/3024365884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41801049b0 msgr2=0x7f4180199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.293+0000 7f4185400640 1 --2- 172.21.15.47:0/3024365884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41801049b0 0x7f4180199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.293+0000 7f4185400640 1 -- 172.21.15.47:0/3024365884 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41801a16d0 con 0x7f41801037c0 2024-03-21T11:23:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.293+0000 7f4185400640 1 --2- 172.21.15.47:0/3024365884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801037c0 0x7f4180199ab0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4174000c00 tx=0x7f4174005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.294+0000 7f417e000640 1 -- 172.21.15.47:0/3024365884 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174041070 con 0x7f41801037c0 2024-03-21T11:23:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.294+0000 7f417e000640 1 -- 172.21.15.47:0/3024365884 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41740044d0 con 0x7f41801037c0 2024-03-21T11:23:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.294+0000 7f417e000640 1 -- 172.21.15.47:0/3024365884 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174034440 con 0x7f41801037c0 2024-03-21T11:23:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.294+0000 7f4186800640 1 -- 172.21.15.47:0/3024365884 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41801a1960 con 0x7f41801037c0 2024-03-21T11:23:28.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.294+0000 7f4186800640 1 -- 172.21.15.47:0/3024365884 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41801a1e00 con 0x7f41801037c0 2024-03-21T11:23:28.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.295+0000 7f4186800640 1 -- 172.21.15.47:0/3024365884 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418010aae0 con 0x7f41801037c0 2024-03-21T11:23:28.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.297+0000 7f417e000640 1 -- 172.21.15.47:0/3024365884 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4174044070 con 0x7f41801037c0 2024-03-21T11:23:28.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.298+0000 7f417e000640 1 --2- 172.21.15.47:0/3024365884 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41680a2fa0 0x7f41680a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:28.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.298+0000 7f417e000640 1 -- 172.21.15.47:0/3024365884 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f41740f1270 con 0x7f41801037c0 2024-03-21T11:23:28.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.298+0000 7f4184a00640 1 --2- 172.21.15.47:0/3024365884 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41680a2fa0 0x7f41680a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:28.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.299+0000 7f4184a00640 1 --2- 172.21.15.47:0/3024365884 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41680a2fa0 0x7f41680a5460 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f418019afc0 tx=0x7f416c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:28.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.302+0000 7f417e000640 1 -- 172.21.15.47:0/3024365884 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f41740bd660 con 0x7f41801037c0 2024-03-21T11:23:28.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:27 smithi047 ceph-mon[31351]: pgmap v404: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:23:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:27 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/673723428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:27 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1266484322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:28.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:27 smithi100 ceph-mon[36245]: pgmap v404: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:23:28.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:27 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/673723428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:28.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:27 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1266484322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:28.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.503+0000 7f4186800640 1 -- 172.21.15.47:0/3024365884 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f41800622a0 con 0x7f41801037c0 2024-03-21T11:23:28.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.503+0000 7f417e000640 1 -- 172.21.15.47:0/3024365884 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f41740c1700 con 0x7f41801037c0 2024-03-21T11:23:28.503 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:23:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.506+0000 7f4186800640 1 -- 172.21.15.47:0/3024365884 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41680a2fa0 msgr2=0x7f41680a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.506+0000 7f4186800640 1 --2- 172.21.15.47:0/3024365884 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41680a2fa0 0x7f41680a5460 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f418019afc0 tx=0x7f416c00b390 comp rx=0 tx=0).stop 2024-03-21T11:23:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.506+0000 7f4186800640 1 -- 172.21.15.47:0/3024365884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801037c0 msgr2=0x7f4180199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.506+0000 7f4186800640 1 --2- 172.21.15.47:0/3024365884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801037c0 0x7f4180199ab0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4174000c00 tx=0x7f4174005d50 comp rx=0 tx=0).stop 2024-03-21T11:23:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.506+0000 7f4186800640 1 -- 172.21.15.47:0/3024365884 shutdown_connections 2024-03-21T11:23:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.506+0000 7f4186800640 1 --2- 172.21.15.47:0/3024365884 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41680a2fa0 0x7f41680a5460 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.506+0000 7f4186800640 1 --2- 172.21.15.47:0/3024365884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4180105370 0x7f41801a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.506+0000 7f4186800640 1 --2- 172.21.15.47:0/3024365884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41801049b0 0x7f4180199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.506+0000 7f4186800640 1 --2- 172.21.15.47:0/3024365884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801037c0 0x7f4180199ab0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.506+0000 7f4186800640 1 -- 172.21.15.47:0/3024365884 >> 172.21.15.47:0/3024365884 conn(0x7f41800fed50 msgr2=0x7f4180107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:28.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.506+0000 7f4186800640 1 -- 172.21.15.47:0/3024365884 shutdown_connections 2024-03-21T11:23:28.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.506+0000 7f4186800640 1 -- 172.21.15.47:0/3024365884 wait complete. 2024-03-21T11:23:28.524 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:23:28.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.752+0000 7fe4abe00640 1 -- 172.21.15.47:0/2854835973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4ac1052a0 msgr2=0x7fe4ac10bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:28.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.752+0000 7fe4abe00640 1 --2- 172.21.15.47:0/2854835973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4ac1052a0 0x7fe4ac10bb30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009ca0 tx=0x7fe4a402c100 comp rx=0 tx=0).stop 2024-03-21T11:23:28.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.753+0000 7fe4abe00640 1 -- 172.21.15.47:0/2854835973 shutdown_connections 2024-03-21T11:23:28.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.753+0000 7fe4abe00640 1 --2- 172.21.15.47:0/2854835973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4ac1052a0 0x7fe4ac10bb30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.753+0000 7fe4abe00640 1 --2- 172.21.15.47:0/2854835973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac1048e0 0x7fe4ac104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.753+0000 7fe4abe00640 1 --2- 172.21.15.47:0/2854835973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac1036f0 0x7fe4ac103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.753+0000 7fe4abe00640 1 -- 172.21.15.47:0/2854835973 >> 172.21.15.47:0/2854835973 conn(0x7fe4ac0fec80 msgr2=0x7fe4ac1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:28.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.753+0000 7fe4abe00640 1 -- 172.21.15.47:0/2854835973 shutdown_connections 2024-03-21T11:23:28.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.753+0000 7fe4abe00640 1 -- 172.21.15.47:0/2854835973 wait complete. 2024-03-21T11:23:28.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.754+0000 7fe4abe00640 1 Processor -- start 2024-03-21T11:23:28.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.754+0000 7fe4abe00640 1 -- start start 2024-03-21T11:23:28.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.754+0000 7fe4abe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac1036f0 0x7fe4ac1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:28.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.754+0000 7fe4abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4ac1048e0 0x7fe4ac199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:28.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.754+0000 7fe4abe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac1052a0 0x7fe4ac1a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:28.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.754+0000 7fe4abe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac10f350 con 0x7fe4ac1052a0 2024-03-21T11:23:28.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.754+0000 7fe4abe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac10f1d0 con 0x7fe4ac1048e0 2024-03-21T11:23:28.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.754+0000 7fe4abe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac10f4d0 con 0x7fe4ac1036f0 2024-03-21T11:23:28.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.755+0000 7fe4aa000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4ac1048e0 0x7fe4ac199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:28.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.755+0000 7fe4aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac1036f0 0x7fe4ac1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:28.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.755+0000 7fe4aa000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4ac1048e0 0x7fe4ac199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38726/0 (socket says 172.21.15.47:38726) 2024-03-21T11:23:28.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.755+0000 7fe4aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac1036f0 0x7fe4ac1998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33044/0 (socket says 172.21.15.47:33044) 2024-03-21T11:23:28.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.755+0000 7fe4aa000640 1 -- 172.21.15.47:0/2464867992 learned_addr learned my addr 172.21.15.47:0/2464867992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:28.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.755+0000 7fe4ab400640 1 --2- 172.21.15.47:0/2464867992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac1052a0 0x7fe4ac1a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.755+0000 7fe4aaa00640 1 -- 172.21.15.47:0/2464867992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4ac1048e0 msgr2=0x7fe4ac199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.755+0000 7fe4aaa00640 1 --2- 172.21.15.47:0/2464867992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4ac1048e0 0x7fe4ac199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.755+0000 7fe4aaa00640 1 -- 172.21.15.47:0/2464867992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac1052a0 msgr2=0x7fe4ac1a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.755+0000 7fe4aaa00640 1 --2- 172.21.15.47:0/2464867992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac1052a0 0x7fe4ac1a0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.755+0000 7fe4aaa00640 1 -- 172.21.15.47:0/2464867992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac1a14f0 con 0x7fe4ac1036f0 2024-03-21T11:23:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.755+0000 7fe4ab400640 1 --2- 172.21.15.47:0/2464867992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac1052a0 0x7fe4ac1a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.755+0000 7fe4aaa00640 1 --2- 172.21.15.47:0/2464867992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac1036f0 0x7fe4ac1998d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00e9c0 tx=0x7fe49c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.756+0000 7fe4b0a00640 1 -- 172.21.15.47:0/2464867992 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c00ccc0 con 0x7fe4ac1036f0 2024-03-21T11:23:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.756+0000 7fe4b0a00640 1 -- 172.21.15.47:0/2464867992 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe49c0035f0 con 0x7fe4ac1036f0 2024-03-21T11:23:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.756+0000 7fe4abe00640 1 -- 172.21.15.47:0/2464867992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ac1a17e0 con 0x7fe4ac1036f0 2024-03-21T11:23:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.756+0000 7fe4b0a00640 1 -- 172.21.15.47:0/2464867992 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c00ce60 con 0x7fe4ac1036f0 2024-03-21T11:23:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.756+0000 7fe4abe00640 1 -- 172.21.15.47:0/2464867992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ac1a1d80 con 0x7fe4ac1036f0 2024-03-21T11:23:28.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.757+0000 7fe4abe00640 1 -- 172.21.15.47:0/2464867992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ac10a940 con 0x7fe4ac1036f0 2024-03-21T11:23:28.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.759+0000 7fe4b0a00640 1 -- 172.21.15.47:0/2464867992 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe49c0037a0 con 0x7fe4ac1036f0 2024-03-21T11:23:28.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.760+0000 7fe4b0a00640 1 --2- 172.21.15.47:0/2464867992 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4900a2fa0 0x7fe4900a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:28.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.760+0000 7fe4b0a00640 1 -- 172.21.15.47:0/2464867992 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe49c0cfb50 con 0x7fe4ac1036f0 2024-03-21T11:23:28.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.760+0000 7fe4aa000640 1 --2- 172.21.15.47:0/2464867992 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4900a2fa0 0x7fe4900a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:28.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.761+0000 7fe4aa000640 1 --2- 172.21.15.47:0/2464867992 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4900a2fa0 0x7fe4900a5460 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac19ade0 tx=0x7fe49800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:28.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:28.768+0000 7fe4b0a00640 1 -- 172.21.15.47:0/2464867992 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe49c09c590 con 0x7fe4ac1036f0 2024-03-21T11:23:29.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.016+0000 7fe4abe00640 1 -- 172.21.15.47:0/2464867992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4ac02d070 con 0x7fe4ac1036f0 2024-03-21T11:23:29.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.017+0000 7fe4b0a00640 1 -- 172.21.15.47:0/2464867992 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fe49c09be90 con 0x7fe4ac1036f0 2024-03-21T11:23:29.019 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:29.019 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:21:18.592954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24524,24523,24535,24581,24553]},"id":1}]} 2024-03-21T11:23:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 27 2024-03-21T11:23:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.019+0000 7fe4abe00640 1 -- 172.21.15.47:0/2464867992 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4900a2fa0 msgr2=0x7fe4900a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.019+0000 7fe4abe00640 1 --2- 172.21.15.47:0/2464867992 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4900a2fa0 0x7fe4900a5460 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac19ade0 tx=0x7fe49800a380 comp rx=0 tx=0).stop 2024-03-21T11:23:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.020+0000 7fe4abe00640 1 -- 172.21.15.47:0/2464867992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac1036f0 msgr2=0x7fe4ac1998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.020+0000 7fe4abe00640 1 --2- 172.21.15.47:0/2464867992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac1036f0 0x7fe4ac1998d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00e9c0 tx=0x7fe49c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:23:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.020+0000 7fe4abe00640 1 -- 172.21.15.47:0/2464867992 shutdown_connections 2024-03-21T11:23:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.020+0000 7fe4abe00640 1 --2- 172.21.15.47:0/2464867992 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4900a2fa0 0x7fe4900a5460 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.020+0000 7fe4abe00640 1 --2- 172.21.15.47:0/2464867992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4ac1052a0 0x7fe4ac1a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.020+0000 7fe4abe00640 1 --2- 172.21.15.47:0/2464867992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4ac1048e0 0x7fe4ac199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.020+0000 7fe4abe00640 1 --2- 172.21.15.47:0/2464867992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac1036f0 0x7fe4ac1998d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.020+0000 7fe4abe00640 1 -- 172.21.15.47:0/2464867992 >> 172.21.15.47:0/2464867992 conn(0x7fe4ac0fec80 msgr2=0x7fe4ac107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.020+0000 7fe4abe00640 1 -- 172.21.15.47:0/2464867992 shutdown_connections 2024-03-21T11:23:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.020+0000 7fe4abe00640 1 -- 172.21.15.47:0/2464867992 wait complete. 2024-03-21T11:23:29.040 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:23:29.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:28 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2357613185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:29.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:28 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3024365884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:29.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.262+0000 7fbb91200640 1 -- 172.21.15.47:0/2018822630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb8c104af0 msgr2=0x7fbb8c106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.262+0000 7fbb91200640 1 --2- 172.21.15.47:0/2018822630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb8c104af0 0x7fbb8c106ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009900 tx=0x7fbb7402bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:29.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.262+0000 7fbb91200640 1 -- 172.21.15.47:0/2018822630 shutdown_connections 2024-03-21T11:23:29.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.262+0000 7fbb91200640 1 --2- 172.21.15.47:0/2018822630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb8c107420 0x7fbb8c109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.262+0000 7fbb91200640 1 --2- 172.21.15.47:0/2018822630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb8c104af0 0x7fbb8c106ee0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.262+0000 7fbb91200640 1 --2- 172.21.15.47:0/2018822630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb8c1021c0 0x7fbb8c1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.262+0000 7fbb91200640 1 -- 172.21.15.47:0/2018822630 >> 172.21.15.47:0/2018822630 conn(0x7fbb8c0fbbd0 msgr2=0x7fbb8c0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:29.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.262+0000 7fbb91200640 1 -- 172.21.15.47:0/2018822630 shutdown_connections 2024-03-21T11:23:29.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.262+0000 7fbb91200640 1 -- 172.21.15.47:0/2018822630 wait complete. 2024-03-21T11:23:29.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.263+0000 7fbb91200640 1 Processor -- start 2024-03-21T11:23:29.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.264+0000 7fbb91200640 1 -- start start 2024-03-21T11:23:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.264+0000 7fbb91200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb8c1021c0 0x7fbb8c101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.264+0000 7fbb91200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb8c104af0 0x7fbb8c100170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.264+0000 7fbb91200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb8c107420 0x7fbb8c1006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.264+0000 7fbb91200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c10d390 con 0x7fbb8c1021c0 2024-03-21T11:23:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.264+0000 7fbb91200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c10d210 con 0x7fbb8c104af0 2024-03-21T11:23:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.264+0000 7fbb91200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c10d510 con 0x7fbb8c107420 2024-03-21T11:23:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.264+0000 7fbb89e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb8c1021c0 0x7fbb8c101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.264+0000 7fbb89e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb8c1021c0 0x7fbb8c101b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46406/0 (socket says 172.21.15.47:46406) 2024-03-21T11:23:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.264+0000 7fbb89e00640 1 -- 172.21.15.47:0/611095911 learned_addr learned my addr 172.21.15.47:0/611095911 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.264+0000 7fbb8a800640 1 --2- 172.21.15.47:0/611095911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb8c107420 0x7fbb8c1006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.264+0000 7fbb83e00640 1 --2- 172.21.15.47:0/611095911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb8c104af0 0x7fbb8c100170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.265+0000 7fbb89e00640 1 -- 172.21.15.47:0/611095911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb8c107420 msgr2=0x7fbb8c1006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.265+0000 7fbb89e00640 1 --2- 172.21.15.47:0/611095911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb8c107420 0x7fbb8c1006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.265+0000 7fbb89e00640 1 -- 172.21.15.47:0/611095911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb8c104af0 msgr2=0x7fbb8c100170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.265+0000 7fbb89e00640 1 --2- 172.21.15.47:0/611095911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb8c104af0 0x7fbb8c100170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.265+0000 7fbb89e00640 1 -- 172.21.15.47:0/611095911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c100f40 con 0x7fbb8c1021c0 2024-03-21T11:23:29.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.265+0000 7fbb89e00640 1 --2- 172.21.15.47:0/611095911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb8c1021c0 0x7fbb8c101b30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00ea60 tx=0x7fbb7c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:29.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.265+0000 7fbb82a00640 1 -- 172.21.15.47:0/611095911 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c00cde0 con 0x7fbb8c1021c0 2024-03-21T11:23:29.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.266+0000 7fbb82a00640 1 -- 172.21.15.47:0/611095911 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb7c0035f0 con 0x7fbb8c1021c0 2024-03-21T11:23:29.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.266+0000 7fbb82a00640 1 -- 172.21.15.47:0/611095911 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c010670 con 0x7fbb8c1021c0 2024-03-21T11:23:29.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.266+0000 7fbb91200640 1 -- 172.21.15.47:0/611095911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c1a1b20 con 0x7fbb8c1021c0 2024-03-21T11:23:29.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.266+0000 7fbb91200640 1 -- 172.21.15.47:0/611095911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c1a1ff0 con 0x7fbb8c1021c0 2024-03-21T11:23:29.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.267+0000 7fbb91200640 1 -- 172.21.15.47:0/611095911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8c0688e0 con 0x7fbb8c1021c0 2024-03-21T11:23:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.269+0000 7fbb82a00640 1 -- 172.21.15.47:0/611095911 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbb7c0037a0 con 0x7fbb8c1021c0 2024-03-21T11:23:29.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.270+0000 7fbb82a00640 1 --2- 172.21.15.47:0/611095911 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb700a2fa0 0x7fbb700a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.271+0000 7fbb82a00640 1 -- 172.21.15.47:0/611095911 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbb7c0cfa70 con 0x7fbb8c1021c0 2024-03-21T11:23:29.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.271+0000 7fbb83e00640 1 --2- 172.21.15.47:0/611095911 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb700a2fa0 0x7fbb700a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.271+0000 7fbb83e00640 1 --2- 172.21.15.47:0/611095911 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb700a2fa0 0x7fbb700a5460 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbb7402c170 tx=0x7fbb7402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:29.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.275+0000 7fbb82a00640 1 -- 172.21.15.47:0/611095911 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbb7c09c4b0 con 0x7fbb8c1021c0 2024-03-21T11:23:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:28 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2357613185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:28 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3024365884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:29.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:28 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2357613185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:29.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:28 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3024365884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:29.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.480+0000 7fbb91200640 1 -- 172.21.15.47:0/611095911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb8c0622a0 con 0x7fbb8c1021c0 2024-03-21T11:23:29.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.485+0000 7fbb82a00640 1 -- 172.21.15.47:0/611095911 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24747 (secure 0 0 0) 0x7fbb7c09bdb0 con 0x7fbb8c1021c0 2024-03-21T11:23:29.485 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:29.485 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:22:50.841892+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:23:29.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.488+0000 7fbb91200640 1 -- 172.21.15.47:0/611095911 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb700a2fa0 msgr2=0x7fbb700a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.488+0000 7fbb91200640 1 --2- 172.21.15.47:0/611095911 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb700a2fa0 0x7fbb700a5460 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbb7402c170 tx=0x7fbb7402df90 comp rx=0 tx=0).stop 2024-03-21T11:23:29.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.488+0000 7fbb91200640 1 -- 172.21.15.47:0/611095911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb8c1021c0 msgr2=0x7fbb8c101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.488+0000 7fbb91200640 1 --2- 172.21.15.47:0/611095911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb8c1021c0 0x7fbb8c101b30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00ea60 tx=0x7fbb7c00ef20 comp rx=0 tx=0).stop 2024-03-21T11:23:29.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.488+0000 7fbb91200640 1 -- 172.21.15.47:0/611095911 shutdown_connections 2024-03-21T11:23:29.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.488+0000 7fbb91200640 1 --2- 172.21.15.47:0/611095911 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbb700a2fa0 0x7fbb700a5460 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.488+0000 7fbb91200640 1 --2- 172.21.15.47:0/611095911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb8c107420 0x7fbb8c1006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.488+0000 7fbb91200640 1 --2- 172.21.15.47:0/611095911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb8c104af0 0x7fbb8c100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.488+0000 7fbb91200640 1 --2- 172.21.15.47:0/611095911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb8c1021c0 0x7fbb8c101b30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.488+0000 7fbb91200640 1 -- 172.21.15.47:0/611095911 >> 172.21.15.47:0/611095911 conn(0x7fbb8c0fbbd0 msgr2=0x7fbb8c102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:29.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.488+0000 7fbb91200640 1 -- 172.21.15.47:0/611095911 shutdown_connections 2024-03-21T11:23:29.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.488+0000 7fbb91200640 1 -- 172.21.15.47:0/611095911 wait complete. 2024-03-21T11:23:29.510 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-21T11:20:48.626416+0000', 'modified': '2024-03-21T11:21:18.592954+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24581, 'mds_1': 24553, 'mds_2': 24535, 'mds_3': 24524, 'mds_4': 24523}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14466': {'gid': 14466, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.47:6841/1742084729', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6840', 'nonce': 1742084729}, {'type': 'v1', 'addr': '172.21.15.47:6841', 'nonce': 1742084729}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24446': {'gid': 24446, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6833/904714287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6832', 'nonce': 904714287}, {'type': 'v1', 'addr': '172.21.15.100:6833', 'nonce': 904714287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24457': {'gid': 24457, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.102:6833/1794606435', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6832', 'nonce': 1794606435}, {'type': 'v1', 'addr': '172.21.15.102:6833', 'nonce': 1794606435}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24476': {'gid': 24476, 'name': 'd', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.47:6837/1846721633', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6836', 'nonce': 1846721633}, {'type': 'v1', 'addr': '172.21.15.47:6837', 'nonce': 1846721633}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6835/2785789126', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 2785789126}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 2785789126}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.102:6835/1182573386', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6834', 'nonce': 1182573386}, {'type': 'v1', 'addr': '172.21.15.102:6835', 'nonce': 1182573386}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.100:6837/1118767699', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 1118767699}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 1118767699}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.102:6837/1198140504', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6836', 'nonce': 1198140504}, {'type': 'v1', 'addr': '172.21.15.102:6837', 'nonce': 1198140504}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24553': {'gid': 24553, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.102:6839/2885246980', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 2885246980}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 2885246980}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6839/2708345792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 2708345792}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 2708345792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24581, 'qdb_cluster': [24524, 24523, 24535, 24581, 24553]} 2024-03-21T11:23:29.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-21T11:23:29.511 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-21T11:23:29.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.734+0000 7f2321200640 1 -- 172.21.15.47:0/276231093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f231c109770 msgr2=0x7f231c110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.734+0000 7f2321200640 1 --2- 172.21.15.47:0/276231093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f231c109770 0x7f231c110000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2314009ca0 tx=0x7f231402c100 comp rx=0 tx=0).stop 2024-03-21T11:23:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.734+0000 7f2321200640 1 -- 172.21.15.47:0/276231093 shutdown_connections 2024-03-21T11:23:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.734+0000 7f2321200640 1 --2- 172.21.15.47:0/276231093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f231c109770 0x7f231c110000 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.734+0000 7f2321200640 1 --2- 172.21.15.47:0/276231093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f231c108db0 0x7f231c109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.734+0000 7f2321200640 1 --2- 172.21.15.47:0/276231093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f231c107bc0 0x7f231c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.734+0000 7f2321200640 1 -- 172.21.15.47:0/276231093 >> 172.21.15.47:0/276231093 conn(0x7f231c076b70 msgr2=0x7f231c078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.734+0000 7f2321200640 1 -- 172.21.15.47:0/276231093 shutdown_connections 2024-03-21T11:23:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.734+0000 7f2321200640 1 -- 172.21.15.47:0/276231093 wait complete. 2024-03-21T11:23:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.735+0000 7f2321200640 1 Processor -- start 2024-03-21T11:23:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.736+0000 7f2321200640 1 -- start start 2024-03-21T11:23:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.736+0000 7f2321200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f231c107bc0 0x7f231c19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.736+0000 7f2321200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f231c108db0 0x7f231c19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.736+0000 7f2321200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f231c109770 0x7f231c1a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.736+0000 7f2321200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c113840 con 0x7f231c109770 2024-03-21T11:23:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.736+0000 7f2321200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c1136c0 con 0x7f231c108db0 2024-03-21T11:23:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.736+0000 7f2321200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c1139c0 con 0x7f231c107bc0 2024-03-21T11:23:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.736+0000 7f231b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f231c107bc0 0x7f231c19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.736+0000 7f2313e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f231c108db0 0x7f231c19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.736+0000 7f231be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f231c109770 0x7f231c1a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.736+0000 7f2313e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f231c108db0 0x7f231c19e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59492/0 (socket says 172.21.15.47:59492) 2024-03-21T11:23:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.736+0000 7f2313e00640 1 -- 172.21.15.47:0/3964711525 learned_addr learned my addr 172.21.15.47:0/3964711525 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.737+0000 7f2313e00640 1 -- 172.21.15.47:0/3964711525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f231c107bc0 msgr2=0x7f231c19de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.737+0000 7f2313e00640 1 --2- 172.21.15.47:0/3964711525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f231c107bc0 0x7f231c19de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.737+0000 7f2313e00640 1 -- 172.21.15.47:0/3964711525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f231c109770 msgr2=0x7f231c1a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.737+0000 7f2313e00640 1 --2- 172.21.15.47:0/3964711525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f231c109770 0x7f231c1a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.737+0000 7f2313e00640 1 -- 172.21.15.47:0/3964711525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231c1a5a70 con 0x7f231c108db0 2024-03-21T11:23:29.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.737+0000 7f231be00640 1 --2- 172.21.15.47:0/3964711525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f231c109770 0x7f231c1a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:29.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.737+0000 7f2313e00640 1 --2- 172.21.15.47:0/3964711525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f231c108db0 0x7f231c19e390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f230400e9f0 tx=0x7f230400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:29.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.737+0000 7f2318c00640 1 -- 172.21.15.47:0/3964711525 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230400cd80 con 0x7f231c108db0 2024-03-21T11:23:29.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.737+0000 7f2318c00640 1 -- 172.21.15.47:0/3964711525 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2304010910 con 0x7f231c108db0 2024-03-21T11:23:29.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.737+0000 7f2318c00640 1 -- 172.21.15.47:0/3964711525 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304018990 con 0x7f231c108db0 2024-03-21T11:23:29.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.737+0000 7f2321200640 1 -- 172.21.15.47:0/3964711525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231c1a5d60 con 0x7f231c108db0 2024-03-21T11:23:29.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.737+0000 7f2321200640 1 -- 172.21.15.47:0/3964711525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231c1a6200 con 0x7f231c108db0 2024-03-21T11:23:29.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.740+0000 7f2318c00640 1 -- 172.21.15.47:0/3964711525 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2304010ac0 con 0x7f231c108db0 2024-03-21T11:23:29.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.741+0000 7f23255ff740 1 Processor -- start 2024-03-21T11:23:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.741+0000 7f23255ff740 1 -- start start 2024-03-21T11:23:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.741+0000 7f23255ff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da563d80 0x5564da5641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.741+0000 7f23255ff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da564770 0x5564da569270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.741+0000 7f23255ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da5698e0 0x5564da56bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.741+0000 7f23255ff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564da4c8c90 con 0x5564da564770 2024-03-21T11:23:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.741+0000 7f23255ff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564da471980 con 0x5564da563d80 2024-03-21T11:23:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.741+0000 7f23255ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564da4c4d30 con 0x5564da5698e0 2024-03-21T11:23:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.742+0000 7f231be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da563d80 0x5564da5641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.742+0000 7f231be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da563d80 0x5564da5641a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59494/0 (socket says 172.21.15.47:59494) 2024-03-21T11:23:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.742+0000 7f231be00640 1 -- 172.21.15.47:0/3221170541 learned_addr learned my addr 172.21.15.47:0/3221170541 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.742+0000 7f2313e00640 1 --2- 172.21.15.47:0/3221170541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da5698e0 0x5564da56bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.742+0000 7f231b400640 1 --2- 172.21.15.47:0/3221170541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da564770 0x5564da569270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.742+0000 7f231be00640 1 -- 172.21.15.47:0/3221170541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da5698e0 msgr2=0x5564da56bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:29.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.742+0000 7f231be00640 1 --2- 172.21.15.47:0/3221170541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da5698e0 0x5564da56bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.742+0000 7f231be00640 1 -- 172.21.15.47:0/3221170541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da564770 msgr2=0x5564da569270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:29.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.742+0000 7f231be00640 1 --2- 172.21.15.47:0/3221170541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da564770 0x5564da569270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.742+0000 7f231be00640 1 -- 172.21.15.47:0/3221170541 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564da489510 con 0x5564da563d80 2024-03-21T11:23:29.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.742+0000 7f231b400640 1 --2- 172.21.15.47:0/3221170541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da564770 0x5564da569270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:29.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.742+0000 7f231be00640 1 --2- 172.21.15.47:0/3221170541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da563d80 0x5564da5641a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f231402c710 tx=0x7f23140323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20457dbce36e7263 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:29.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.743+0000 7f2318c00640 1 --2- 172.21.15.47:0/3964711525 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f23000a2fa0 0x7f23000a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.743+0000 7f2318c00640 1 -- 172.21.15.47:0/3964711525 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f23040cfa40 con 0x7f231c108db0 2024-03-21T11:23:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.743+0000 7f2310c00640 1 -- 172.21.15.47:0/3221170541 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314043070 con 0x5564da563d80 2024-03-21T11:23:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.743+0000 7f2310c00640 1 -- 172.21.15.47:0/3221170541 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2314032c90 con 0x5564da563d80 2024-03-21T11:23:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.743+0000 7f2310c00640 1 -- 172.21.15.47:0/3221170541 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314039040 con 0x5564da563d80 2024-03-21T11:23:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.743+0000 7f231b400640 1 --2- 172.21.15.47:0/3964711525 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f23000a2fa0 0x7f23000a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.743+0000 7f23255ff740 1 -- 172.21.15.47:0/3221170541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da563d80 msgr2=0x5564da5641a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.743+0000 7f23255ff740 1 --2- 172.21.15.47:0/3221170541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da563d80 0x5564da5641a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f231402c710 tx=0x7f23140323d0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.744+0000 7f23255ff740 1 -- 172.21.15.47:0/3221170541 shutdown_connections 2024-03-21T11:23:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.744+0000 7f23255ff740 1 --2- 172.21.15.47:0/3221170541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da5698e0 0x5564da56bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.744+0000 7f23255ff740 1 --2- 172.21.15.47:0/3221170541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da564770 0x5564da569270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.744+0000 7f231b400640 1 --2- 172.21.15.47:0/3964711525 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f23000a2fa0 0x7f23000a5460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x5564da564b70 tx=0x7f230c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.744+0000 7f23255ff740 1 --2- 172.21.15.47:0/3221170541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da563d80 0x5564da5641a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.744+0000 7f23255ff740 1 -- 172.21.15.47:0/3221170541 >> 172.21.15.47:0/3221170541 conn(0x5564da55f2f0 msgr2=0x5564da55f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.744+0000 7f23255ff740 1 -- 172.21.15.47:0/3221170541 shutdown_connections 2024-03-21T11:23:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.744+0000 7f23255ff740 1 -- 172.21.15.47:0/3221170541 wait complete. 2024-03-21T11:23:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.744+0000 7f23255ff740 1 -- start start 2024-03-21T11:23:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.744+0000 7f23255ff740 1 Processor -- start 2024-03-21T11:23:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.745+0000 7f23255ff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da563d80 0x5564da56e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.745+0000 7f23255ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da564770 0x5564da56e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.745+0000 7f23255ff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da5698e0 0x5564da5757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.745+0000 7f23255ff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564da56c830 con 0x5564da563d80 2024-03-21T11:23:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.745+0000 7f23255ff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564da484dd0 con 0x5564da5698e0 2024-03-21T11:23:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.745+0000 7f23255ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564da56c9b0 con 0x5564da564770 2024-03-21T11:23:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.745+0000 7f23255ff740 10 client.0 _ll_register_callbacks cb 0x5564da1dcac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:23:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.745+0000 7f231be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da5698e0 0x5564da5757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.745+0000 7f2313e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da564770 0x5564da56e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.745+0000 7f231b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da563d80 0x5564da56e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.745+0000 7f231be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da5698e0 0x5564da5757f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59502/0 (socket says 172.21.15.47:59502) 2024-03-21T11:23:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.745+0000 7f231be00640 1 -- 172.21.15.47:0/2503527736 learned_addr learned my addr 172.21.15.47:0/2503527736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.746+0000 7f231be00640 1 -- 172.21.15.47:0/2503527736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da564770 msgr2=0x5564da56e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.746+0000 7f231be00640 1 --2- 172.21.15.47:0/2503527736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da564770 0x5564da56e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.746+0000 7f231be00640 1 -- 172.21.15.47:0/2503527736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da563d80 msgr2=0x5564da56e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.746+0000 7f231be00640 1 --2- 172.21.15.47:0/2503527736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da563d80 0x5564da56e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.746+0000 7f231be00640 1 -- 172.21.15.47:0/2503527736 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564da1dd2d0 con 0x5564da5698e0 2024-03-21T11:23:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.746+0000 7f2313e00640 1 --2- 172.21.15.47:0/2503527736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da564770 0x5564da56e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.746+0000 7f231be00640 1 --2- 172.21.15.47:0/2503527736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da5698e0 0x5564da5757f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2314006010 tx=0x7f23140093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16df88f77fb49728 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.746+0000 7f2309600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:23:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.746+0000 7f2309600640 1 -- 172.21.15.47:0/2503527736 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231404c030 con 0x5564da5698e0 2024-03-21T11:23:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.746+0000 7f2309600640 1 -- 172.21.15.47:0/2503527736 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2314003970 con 0x5564da5698e0 2024-03-21T11:23:29.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.746+0000 7f23255ff740 1 -- 172.21.15.47:0/2503527736 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2314032c90 con 0x5564da5698e0 2024-03-21T11:23:29.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.747+0000 7f2309600640 1 -- 172.21.15.47:0/2503527736 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2314059050 con 0x5564da5698e0 2024-03-21T11:23:29.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.747+0000 7f23255ff740 10 client.25361 fetch_fsmap learned FSMap version 27 2024-03-21T11:23:29.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.747+0000 7f23255ff740 1 -- 172.21.15.47:0/2503527736 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5564da2a76b0 con 0x5564da5698e0 2024-03-21T11:23:29.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.747+0000 7f2309600640 1 -- 172.21.15.47:0/2503527736 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2314032c90 con 0x5564da5698e0 2024-03-21T11:23:29.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.747+0000 7f2309600640 1 -- 172.21.15.47:0/2503527736 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2314004530 con 0x5564da5698e0 2024-03-21T11:23:29.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.747+0000 7f23255ff740 10 client.25361 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:23:29.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.747+0000 7f23255ff740 10 client.25361 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-21T11:23:29.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.747+0000 7f23255ff740 20 client.25361 populate_metadata read hostname 'smithi047' 2024-03-21T11:23:29.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.747+0000 7f23255ff740 1 --2- 172.21.15.47:0/2503527736 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564da57a450 0x5564da57e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.747+0000 7f23255ff740 4 client.25361 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:23:29.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.747+0000 7f23255ff740 1 -- 172.21.15.47:0/2503527736 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564da46a520 con 0x5564da57a450 2024-03-21T11:23:29.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.747+0000 7f231b400640 1 --2- 172.21.15.47:0/2503527736 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564da57a450 0x5564da57e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.748+0000 7f231b400640 1 --2- 172.21.15.47:0/2503527736 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564da57a450 0x5564da57e8d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c719b511b5361f4 server_cookie=e3ce2ae1598d43d8 in_seq=0 out_seq=0 2024-03-21T11:23:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.748+0000 7f2309600640 10 client.25361 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:23:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.749+0000 7f2309600640 1 -- 172.21.15.47:0/2503527736 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5564da46a520 con 0x5564da57a450 2024-03-21T11:23:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.749+0000 7f2309600640 10 client.25361 handle_command_reply: tid=0 2024-03-21T11:23:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.749+0000 7f23255ff740 1 client.25361 shutdown 2024-03-21T11:23:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.749+0000 7f23255ff740 1 -- 172.21.15.47:0/2503527736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da5698e0 msgr2=0x5564da5757f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.749+0000 7f23255ff740 1 --2- 172.21.15.47:0/2503527736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da5698e0 0x5564da5757f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2314006010 tx=0x7f23140093e0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.749+0000 7f23255ff740 1 -- 172.21.15.47:0/2503527736 shutdown_connections 2024-03-21T11:23:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.749+0000 7f23255ff740 1 --2- 172.21.15.47:0/2503527736 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564da57a450 0x5564da57e8d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.749+0000 7f23255ff740 1 --2- 172.21.15.47:0/2503527736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da5698e0 0x5564da5757f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.749+0000 7f23255ff740 1 --2- 172.21.15.47:0/2503527736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da564770 0x5564da56e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.749+0000 7f23255ff740 1 --2- 172.21.15.47:0/2503527736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da563d80 0x5564da56e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.749+0000 7f2309600640 0 client.25361 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:23:29.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.749+0000 7f23255ff740 1 -- 172.21.15.47:0/2503527736 >> 172.21.15.47:0/2503527736 conn(0x5564da55f2f0 msgr2=0x5564da55e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:29.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.750+0000 7f23255ff740 1 -- 172.21.15.47:0/2503527736 shutdown_connections 2024-03-21T11:23:29.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.750+0000 7f23255ff740 1 -- 172.21.15.47:0/2503527736 wait complete. 2024-03-21T11:23:29.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.750+0000 7f23255ff740 20 client.25361 trim_cache size 0 max 16384 2024-03-21T11:23:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.768+0000 7f23255ff740 1 Processor -- start 2024-03-21T11:23:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.769+0000 7f23255ff740 1 -- start start 2024-03-21T11:23:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.769+0000 7f23255ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da563d80 0x5564da55f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.769+0000 7f23255ff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da564770 0x5564da575710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.769+0000 7f23255ff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da5698e0 0x5564da55dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.769+0000 7f23255ff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564da579c70 con 0x5564da564770 2024-03-21T11:23:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.769+0000 7f23255ff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564da57a2d0 con 0x5564da5698e0 2024-03-21T11:23:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.769+0000 7f23255ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564da57a930 con 0x5564da563d80 2024-03-21T11:23:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.769+0000 7f231b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da563d80 0x5564da55f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.769+0000 7f231b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da563d80 0x5564da55f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53220/0 (socket says 172.21.15.47:53220) 2024-03-21T11:23:29.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.769+0000 7f231be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da5698e0 0x5564da55dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.769+0000 7f231b400640 1 -- 172.21.15.47:0/3936386710 learned_addr learned my addr 172.21.15.47:0/3936386710 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:29.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.769+0000 7f2313e00640 1 --2- 172.21.15.47:0/3936386710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da564770 0x5564da575710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.770+0000 7f231b400640 1 -- 172.21.15.47:0/3936386710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da5698e0 msgr2=0x5564da55dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:29.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.770+0000 7f231b400640 1 --2- 172.21.15.47:0/3936386710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da5698e0 0x5564da55dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.770+0000 7f231b400640 1 -- 172.21.15.47:0/3936386710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da564770 msgr2=0x5564da575710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.770+0000 7f231b400640 1 --2- 172.21.15.47:0/3936386710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da564770 0x5564da575710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.770+0000 7f231b400640 1 -- 172.21.15.47:0/3936386710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564da56c8f0 con 0x5564da563d80 2024-03-21T11:23:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.770+0000 7f231be00640 1 --2- 172.21.15.47:0/3936386710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da5698e0 0x5564da55dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.770+0000 7f231b400640 1 --2- 172.21.15.47:0/3936386710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da563d80 0x5564da55f6d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x5564da560260 tx=0x7f230c01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35cd12aa7dd55f45 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.770+0000 7f230be00640 1 -- 172.21.15.47:0/3936386710 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c003c80 con 0x5564da563d80 2024-03-21T11:23:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.770+0000 7f230be00640 1 -- 172.21.15.47:0/3936386710 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f230c030070 con 0x5564da563d80 2024-03-21T11:23:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.770+0000 7f230be00640 1 -- 172.21.15.47:0/3936386710 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c01f690 con 0x5564da563d80 2024-03-21T11:23:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.771+0000 7f23255ff740 1 -- 172.21.15.47:0/3936386710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da563d80 msgr2=0x5564da55f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.771+0000 7f23255ff740 1 --2- 172.21.15.47:0/3936386710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da563d80 0x5564da55f6d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x5564da560260 tx=0x7f230c01b000 comp rx=0 tx=0).stop 2024-03-21T11:23:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.771+0000 7f23255ff740 1 -- 172.21.15.47:0/3936386710 shutdown_connections 2024-03-21T11:23:29.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.771+0000 7f23255ff740 1 --2- 172.21.15.47:0/3936386710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da5698e0 0x5564da55dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.771+0000 7f23255ff740 1 --2- 172.21.15.47:0/3936386710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da564770 0x5564da575710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.771+0000 7f23255ff740 1 --2- 172.21.15.47:0/3936386710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da563d80 0x5564da55f6d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.771+0000 7f23255ff740 1 -- 172.21.15.47:0/3936386710 >> 172.21.15.47:0/3936386710 conn(0x5564da55f2f0 msgr2=0x5564da56cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.771+0000 7f23255ff740 1 -- 172.21.15.47:0/3936386710 shutdown_connections 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.771+0000 7f23255ff740 1 -- 172.21.15.47:0/3936386710 wait complete. 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.772+0000 7f23255ff740 1 -- start start 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.772+0000 7f23255ff740 1 Processor -- start 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.772+0000 7f23255ff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da563d80 0x5564da571930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.772+0000 7f23255ff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da564770 0x5564da571e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.772+0000 7f23255ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da5698e0 0x5564da574e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.772+0000 7f23255ff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564da57b5f0 con 0x5564da563d80 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.772+0000 7f23255ff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564da57af90 con 0x5564da564770 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.772+0000 7f23255ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564da57bc50 con 0x5564da5698e0 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.772+0000 7f23255ff740 10 client.0 _ll_register_callbacks cb 0x7f23040d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:23:29.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.772+0000 7f231be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da5698e0 0x5564da574e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f2313e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da564770 0x5564da571e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f231be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da5698e0 0x5564da574e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53222/0 (socket says 172.21.15.47:53222) 2024-03-21T11:23:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f231be00640 1 -- 172.21.15.47:0/438326471 learned_addr learned my addr 172.21.15.47:0/438326471 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f231b400640 1 --2- 172.21.15.47:0/438326471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da563d80 0x5564da571930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f231be00640 1 -- 172.21.15.47:0/438326471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da564770 msgr2=0x5564da571e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f231be00640 1 --2- 172.21.15.47:0/438326471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da564770 0x5564da571e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f231be00640 1 -- 172.21.15.47:0/438326471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da563d80 msgr2=0x5564da571930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f231be00640 1 --2- 172.21.15.47:0/438326471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da563d80 0x5564da571930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f231be00640 1 -- 172.21.15.47:0/438326471 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564da575410 con 0x5564da5698e0 2024-03-21T11:23:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f2313e00640 1 --2- 172.21.15.47:0/438326471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da564770 0x5564da571e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f231be00640 1 --2- 172.21.15.47:0/438326471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da5698e0 0x5564da574e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2314004900 tx=0x7f2314048a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e99805b7d0e24f49 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f2309600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:23:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f2309600640 1 -- 172.21.15.47:0/438326471 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231404c040 con 0x5564da5698e0 2024-03-21T11:23:29.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f2309600640 1 -- 172.21.15.47:0/438326471 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f23140349f0 con 0x5564da5698e0 2024-03-21T11:23:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.773+0000 7f23255ff740 1 -- 172.21.15.47:0/438326471 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f230c030070 con 0x5564da5698e0 2024-03-21T11:23:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.774+0000 7f2309600640 1 -- 172.21.15.47:0/438326471 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2314039070 con 0x5564da5698e0 2024-03-21T11:23:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.774+0000 7f23255ff740 10 client.25378 fetch_fsmap learned FSMap version 27 2024-03-21T11:23:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.774+0000 7f23255ff740 1 -- 172.21.15.47:0/438326471 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5564da5d6330 con 0x5564da5698e0 2024-03-21T11:23:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.774+0000 7f2309600640 1 -- 172.21.15.47:0/438326471 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f230c030070 con 0x5564da5698e0 2024-03-21T11:23:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.774+0000 7f2309600640 1 -- 172.21.15.47:0/438326471 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2314034c80 con 0x5564da5698e0 2024-03-21T11:23:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.774+0000 7f23255ff740 10 client.25378 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:23:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.774+0000 7f23255ff740 10 client.25378 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-21T11:23:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.774+0000 7f23255ff740 20 client.25378 populate_metadata read hostname 'smithi047' 2024-03-21T11:23:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.774+0000 7f23255ff740 1 --2- 172.21.15.47:0/438326471 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564da5da810 0x5564da5d65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.774+0000 7f23255ff740 4 client.25378 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T11:23:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.774+0000 7f23255ff740 1 -- 172.21.15.47:0/438326471 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5564da4d61d0 con 0x5564da5da810 2024-03-21T11:23:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.775+0000 7f231b400640 1 --2- 172.21.15.47:0/438326471 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564da5da810 0x5564da5d65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:29.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.775+0000 7f231b400640 1 --2- 172.21.15.47:0/438326471 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564da5da810 0x5564da5d65b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c101b4aba9f510f server_cookie=657c3fedeaa2f524 in_seq=0 out_seq=0 2024-03-21T11:23:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.775+0000 7f2309600640 10 client.25378 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:23:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.775+0000 7f2309600640 1 -- 172.21.15.47:0/438326471 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5564da4d61d0 con 0x5564da5da810 2024-03-21T11:23:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.775+0000 7f2309600640 10 client.25378 handle_command_reply: tid=0 2024-03-21T11:23:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.775+0000 7f23255ff740 1 client.25378 shutdown 2024-03-21T11:23:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.775+0000 7f23255ff740 1 -- 172.21.15.47:0/438326471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da5698e0 msgr2=0x5564da574e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.775+0000 7f23255ff740 1 --2- 172.21.15.47:0/438326471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da5698e0 0x5564da574e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2314004900 tx=0x7f2314048a80 comp rx=0 tx=0).stop 2024-03-21T11:23:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.776+0000 7f23255ff740 1 -- 172.21.15.47:0/438326471 shutdown_connections 2024-03-21T11:23:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.776+0000 7f23255ff740 1 --2- 172.21.15.47:0/438326471 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564da5da810 0x5564da5d65b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.776+0000 7f23255ff740 1 --2- 172.21.15.47:0/438326471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564da5698e0 0x5564da574e20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.776+0000 7f23255ff740 1 --2- 172.21.15.47:0/438326471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564da564770 0x5564da571e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.776+0000 7f23255ff740 1 --2- 172.21.15.47:0/438326471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564da563d80 0x5564da571930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.776+0000 7f2309600640 0 client.25378 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:23:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.776+0000 7f23255ff740 1 -- 172.21.15.47:0/438326471 >> 172.21.15.47:0/438326471 conn(0x5564da55f2f0 msgr2=0x5564da55f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:29.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.776+0000 7f23255ff740 1 -- 172.21.15.47:0/438326471 shutdown_connections 2024-03-21T11:23:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.776+0000 7f23255ff740 1 -- 172.21.15.47:0/438326471 wait complete. 2024-03-21T11:23:29.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.776+0000 7f23255ff740 20 client.25378 trim_cache size 0 max 16384 2024-03-21T11:23:29.781 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:23:29.781 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:23:29.781 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 4, 2024-03-21T11:23:29.781 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24523, 2024-03-21T11:23:29.781 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T11:23:29.781 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T11:23:29.781 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T11:23:29.782 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 131.901254585, 2024-03-21T11:23:29.782 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 27, 2024-03-21T11:23:29.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T11:23:29.782 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-03-21T11:23:29.782 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 187.42916658799999 2024-03-21T11:23:29.782 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:23:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.776+0000 7f2321200640 1 -- 172.21.15.47:0/3964711525 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f23000a2fa0 msgr2=0x7f23000a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.776+0000 7f2321200640 1 --2- 172.21.15.47:0/3964711525 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f23000a2fa0 0x7f23000a5460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x5564da564b70 tx=0x7f230c005f90 comp rx=0 tx=0).stop 2024-03-21T11:23:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.776+0000 7f2321200640 1 -- 172.21.15.47:0/3964711525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f231c108db0 msgr2=0x7f231c19e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.776+0000 7f2321200640 1 --2- 172.21.15.47:0/3964711525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f231c108db0 0x7f231c19e390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f230400e9f0 tx=0x7f230400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.777+0000 7f2321200640 1 -- 172.21.15.47:0/3964711525 shutdown_connections 2024-03-21T11:23:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.777+0000 7f2321200640 1 --2- 172.21.15.47:0/3964711525 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f23000a2fa0 0x7f23000a5460 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.777+0000 7f2321200640 1 --2- 172.21.15.47:0/3964711525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f231c109770 0x7f231c1a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.777+0000 7f2321200640 1 --2- 172.21.15.47:0/3964711525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f231c108db0 0x7f231c19e390 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.777+0000 7f2321200640 1 --2- 172.21.15.47:0/3964711525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f231c107bc0 0x7f231c19de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.777+0000 7f2321200640 1 -- 172.21.15.47:0/3964711525 >> 172.21.15.47:0/3964711525 conn(0x7f231c076b70 msgr2=0x7f231c10bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.777+0000 7f2321200640 1 -- 172.21.15.47:0/3964711525 shutdown_connections 2024-03-21T11:23:29.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:29.777+0000 7f2321200640 1 -- 172.21.15.47:0/3964711525 wait complete. 2024-03-21T11:23:29.793 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-21T11:23:30.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.002+0000 7f4c50a00640 1 -- 172.21.15.47:0/2179448742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c1038c0 msgr2=0x7f4c4c103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.002+0000 7f4c50a00640 1 --2- 172.21.15.47:0/2179448742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c1038c0 0x7f4c4c103ce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009900 tx=0x7f4c3c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:30.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.002+0000 7f4c50a00640 1 -- 172.21.15.47:0/2179448742 shutdown_connections 2024-03-21T11:23:30.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.002+0000 7f4c50a00640 1 --2- 172.21.15.47:0/2179448742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c4c105470 0x7f4c4c10bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.002+0000 7f4c50a00640 1 --2- 172.21.15.47:0/2179448742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c4c104ab0 0x7f4c4c104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.002+0000 7f4c50a00640 1 --2- 172.21.15.47:0/2179448742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c1038c0 0x7f4c4c103ce0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.002+0000 7f4c50a00640 1 -- 172.21.15.47:0/2179448742 >> 172.21.15.47:0/2179448742 conn(0x7f4c4c0fee50 msgr2=0x7f4c4c101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.002+0000 7f4c50a00640 1 -- 172.21.15.47:0/2179448742 shutdown_connections 2024-03-21T11:23:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.002+0000 7f4c50a00640 1 -- 172.21.15.47:0/2179448742 wait complete. 2024-03-21T11:23:30.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.003+0000 7f4c50a00640 1 Processor -- start 2024-03-21T11:23:30.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.003+0000 7f4c50a00640 1 -- start start 2024-03-21T11:23:30.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c50a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c1038c0 0x7f4c4c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c50a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c4c104ab0 0x7f4c4c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c50a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c4c105470 0x7f4c4c1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c50a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c10f560 con 0x7f4c4c104ab0 2024-03-21T11:23:30.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c50a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c10f3e0 con 0x7f4c4c1038c0 2024-03-21T11:23:30.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c50a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c10f6e0 con 0x7f4c4c105470 2024-03-21T11:23:30.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c49e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c1038c0 0x7f4c4c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c49400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c4c104ab0 0x7f4c4c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c4a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c4c105470 0x7f4c4c1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c49e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c1038c0 0x7f4c4c199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59534/0 (socket says 172.21.15.47:59534) 2024-03-21T11:23:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c49e00640 1 -- 172.21.15.47:0/1373646255 learned_addr learned my addr 172.21.15.47:0/1373646255 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c4a800640 1 -- 172.21.15.47:0/1373646255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c1038c0 msgr2=0x7f4c4c199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c4a800640 1 --2- 172.21.15.47:0/1373646255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c1038c0 0x7f4c4c199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c4a800640 1 -- 172.21.15.47:0/1373646255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c4c104ab0 msgr2=0x7f4c4c19a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c4a800640 1 --2- 172.21.15.47:0/1373646255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c4c104ab0 0x7f4c4c19a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c4a800640 1 -- 172.21.15.47:0/1373646255 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c1a1800 con 0x7f4c4c105470 2024-03-21T11:23:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.004+0000 7f4c49400640 1 --2- 172.21.15.47:0/1373646255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c4c104ab0 0x7f4c4c19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.005+0000 7f4c4a800640 1 --2- 172.21.15.47:0/1373646255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c4c105470 0x7f4c4c1a1190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009fd0 tx=0x7f4c4400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.005+0000 7f4c42a00640 1 -- 172.21.15.47:0/1373646255 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400ce20 con 0x7f4c4c105470 2024-03-21T11:23:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.005+0000 7f4c50a00640 1 -- 172.21.15.47:0/1373646255 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c1a1af0 con 0x7f4c4c105470 2024-03-21T11:23:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.005+0000 7f4c42a00640 1 -- 172.21.15.47:0/1373646255 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c44005020 con 0x7f4c4c105470 2024-03-21T11:23:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.005+0000 7f4c42a00640 1 -- 172.21.15.47:0/1373646255 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44010720 con 0x7f4c4c105470 2024-03-21T11:23:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.005+0000 7f4c50a00640 1 -- 172.21.15.47:0/1373646255 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4c1a1ff0 con 0x7f4c4c105470 2024-03-21T11:23:30.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.008+0000 7f4c42a00640 1 -- 172.21.15.47:0/1373646255 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4c44005540 con 0x7f4c4c105470 2024-03-21T11:23:30.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.008+0000 7f4c537da740 1 Processor -- start 2024-03-21T11:23:30.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.008+0000 7f4c537da740 1 -- start start 2024-03-21T11:23:30.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c537da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe23ba0 0x5603bbe23fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c537da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe29090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c537da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe29700 0x5603bbe2bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c537da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bbd88c90 con 0x5603bbe23ba0 2024-03-21T11:23:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c537da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bbd31980 con 0x5603bbe29700 2024-03-21T11:23:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c537da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bbd84d30 con 0x5603bbe24590 2024-03-21T11:23:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c49e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe29090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c49400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe23ba0 0x5603bbe23fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c49e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe29090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53246/0 (socket says 172.21.15.47:53246) 2024-03-21T11:23:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c49e00640 1 -- 172.21.15.47:0/705647438 learned_addr learned my addr 172.21.15.47:0/705647438 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c49400640 1 --2- 172.21.15.47:0/705647438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe29700 0x5603bbe2bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c49e00640 1 -- 172.21.15.47:0/705647438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe29700 msgr2=0x5603bbe2bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c49e00640 1 --2- 172.21.15.47:0/705647438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe29700 0x5603bbe2bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c49e00640 1 -- 172.21.15.47:0/705647438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe23ba0 msgr2=0x5603bbe23fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c49e00640 1 --2- 172.21.15.47:0/705647438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe23ba0 0x5603bbe23fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c49e00640 1 -- 172.21.15.47:0/705647438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603bbd49510 con 0x5603bbe24590 2024-03-21T11:23:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.009+0000 7f4c49400640 1 --2- 172.21.15.47:0/705647438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe29700 0x5603bbe2bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:23:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.010+0000 7f4c49e00640 1 --2- 172.21.15.47:0/705647438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe29090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c02c170 tx=0x7f4c3c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a36e3c2ea6b5e61 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.010+0000 7f4c42a00640 1 --2- 172.21.15.47:0/1373646255 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c300a2f20 0x7f4c300a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.010+0000 7f4c3b000640 1 -- 172.21.15.47:0/705647438 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c045070 con 0x5603bbe24590 2024-03-21T11:23:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.010+0000 7f4c3b000640 1 -- 172.21.15.47:0/705647438 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c3c004040 con 0x5603bbe24590 2024-03-21T11:23:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.010+0000 7f4c3b000640 1 -- 172.21.15.47:0/705647438 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c004ef0 con 0x5603bbe24590 2024-03-21T11:23:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.010+0000 7f4c42a00640 1 -- 172.21.15.47:0/1373646255 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4c440cfed0 con 0x7f4c4c105470 2024-03-21T11:23:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.010+0000 7f4c49400640 1 --2- 172.21.15.47:0/1373646255 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c300a2f20 0x7f4c300a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.011+0000 7f4c537da740 1 -- 172.21.15.47:0/705647438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 msgr2=0x5603bbe29090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.011+0000 7f4c537da740 1 --2- 172.21.15.47:0/705647438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe29090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c02c170 tx=0x7f4c3c009560 comp rx=0 tx=0).stop 2024-03-21T11:23:30.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.011+0000 7f4c49400640 1 --2- 172.21.15.47:0/1373646255 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c300a2f20 0x7f4c300a53e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5603bbe24df0 tx=0x7f4c34005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.011+0000 7f4c537da740 1 -- 172.21.15.47:0/705647438 shutdown_connections 2024-03-21T11:23:30.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.011+0000 7f4c537da740 1 --2- 172.21.15.47:0/705647438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe29700 0x5603bbe2bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.011+0000 7f4c537da740 1 --2- 172.21.15.47:0/705647438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe29090 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.011+0000 7f4c537da740 1 --2- 172.21.15.47:0/705647438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe23ba0 0x5603bbe23fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.011+0000 7f4c537da740 1 -- 172.21.15.47:0/705647438 >> 172.21.15.47:0/705647438 conn(0x5603bbe1f110 msgr2=0x5603bbe1f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.011+0000 7f4c537da740 1 -- 172.21.15.47:0/705647438 shutdown_connections 2024-03-21T11:23:30.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.011+0000 7f4c537da740 1 -- 172.21.15.47:0/705647438 wait complete. 2024-03-21T11:23:30.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.011+0000 7f4c537da740 1 -- start start 2024-03-21T11:23:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.012+0000 7f4c537da740 1 Processor -- start 2024-03-21T11:23:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.012+0000 7f4c537da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe23ba0 0x5603bbe2dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.012+0000 7f4c537da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe24590 0x5603bbe2e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.012+0000 7f4c537da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe29700 0x5603bbe35560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.012+0000 7f4c537da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bbe2c650 con 0x5603bbe29700 2024-03-21T11:23:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.012+0000 7f4c537da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bbd44dd0 con 0x5603bbe24590 2024-03-21T11:23:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.012+0000 7f4c537da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bbe2c7d0 con 0x5603bbe23ba0 2024-03-21T11:23:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.012+0000 7f4c49e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe23ba0 0x5603bbe2dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.012+0000 7f4c4a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe29700 0x5603bbe35560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.012+0000 7f4c537da740 10 client.0 _ll_register_callbacks cb 0x5603bba9cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:23:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.012+0000 7f4c49e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe23ba0 0x5603bbe2dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53248/0 (socket says 172.21.15.47:53248) 2024-03-21T11:23:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.012+0000 7f4c49e00640 1 -- 172.21.15.47:0/4130084680 learned_addr learned my addr 172.21.15.47:0/4130084680 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.012+0000 7f4c49400640 1 --2- 172.21.15.47:0/4130084680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe24590 0x5603bbe2e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.013+0000 7f4c49e00640 1 -- 172.21.15.47:0/4130084680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe24590 msgr2=0x5603bbe2e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.013+0000 7f4c49e00640 1 --2- 172.21.15.47:0/4130084680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe24590 0x5603bbe2e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.013+0000 7f4c49e00640 1 -- 172.21.15.47:0/4130084680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe29700 msgr2=0x5603bbe35560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.013+0000 7f4c49e00640 1 --2- 172.21.15.47:0/4130084680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe29700 0x5603bbe35560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.013+0000 7f4c49e00640 1 -- 172.21.15.47:0/4130084680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603bba9d2d0 con 0x5603bbe23ba0 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.013+0000 7f4c4a800640 1 --2- 172.21.15.47:0/4130084680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe29700 0x5603bbe35560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.013+0000 7f4c49e00640 1 --2- 172.21.15.47:0/4130084680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe23ba0 0x5603bbe2dfe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c02c240 tx=0x7f4c3c02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c2df9038bb81a40 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.013+0000 7f4c2f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.013+0000 7f4c2f400640 1 -- 172.21.15.47:0/4130084680 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c045040 con 0x5603bbe23ba0 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.013+0000 7f4c537da740 1 -- 172.21.15.47:0/4130084680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c3c004040 con 0x5603bbe23ba0 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.013+0000 7f4c2f400640 1 -- 172.21.15.47:0/4130084680 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4c3c004ef0 con 0x5603bbe23ba0 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.014+0000 7f4c2f400640 1 -- 172.21.15.47:0/4130084680 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c3c05b050 con 0x5603bbe23ba0 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.014+0000 7f4c537da740 10 client.25390 fetch_fsmap learned FSMap version 27 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.014+0000 7f4c537da740 1 -- 172.21.15.47:0/4130084680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5603bbb676b0 con 0x5603bbe23ba0 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.014+0000 7f4c2f400640 1 -- 172.21.15.47:0/4130084680 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c3c004040 con 0x5603bbe23ba0 2024-03-21T11:23:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.014+0000 7f4c2f400640 1 -- 172.21.15.47:0/4130084680 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4c3c003bf0 con 0x5603bbe23ba0 2024-03-21T11:23:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.014+0000 7f4c537da740 10 client.25390 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:23:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.014+0000 7f4c537da740 10 client.25390 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-21T11:23:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.014+0000 7f4c537da740 20 client.25390 populate_metadata read hostname 'smithi047' 2024-03-21T11:23:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.014+0000 7f4c537da740 1 --2- 172.21.15.47:0/4130084680 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5603bbe3a320 0x5603bbe3e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.014+0000 7f4c537da740 4 client.25390 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:23:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.014+0000 7f4c537da740 1 -- 172.21.15.47:0/4130084680 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603bbd2a520 con 0x5603bbe3a320 2024-03-21T11:23:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.015+0000 7f4c49400640 1 --2- 172.21.15.47:0/4130084680 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5603bbe3a320 0x5603bbe3e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.015+0000 7f4c49400640 1 --2- 172.21.15.47:0/4130084680 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5603bbe3a320 0x5603bbe3e7a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f771f7d258c62a4d server_cookie=5fd85d516c4d16f8 in_seq=0 out_seq=0 2024-03-21T11:23:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.015+0000 7f4c2f400640 10 client.25390 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:23:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.016+0000 7f4c2f400640 1 -- 172.21.15.47:0/4130084680 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5603bbd2a520 con 0x5603bbe3a320 2024-03-21T11:23:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.016+0000 7f4c2f400640 10 client.25390 handle_command_reply: tid=0 2024-03-21T11:23:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.016+0000 7f4c537da740 1 client.25390 shutdown 2024-03-21T11:23:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.016+0000 7f4c537da740 1 -- 172.21.15.47:0/4130084680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe23ba0 msgr2=0x5603bbe2dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.016+0000 7f4c537da740 1 --2- 172.21.15.47:0/4130084680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe23ba0 0x5603bbe2dfe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c02c240 tx=0x7f4c3c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.016+0000 7f4c537da740 1 -- 172.21.15.47:0/4130084680 shutdown_connections 2024-03-21T11:23:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.016+0000 7f4c537da740 1 --2- 172.21.15.47:0/4130084680 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5603bbe3a320 0x5603bbe3e7a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.016+0000 7f4c537da740 1 --2- 172.21.15.47:0/4130084680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe29700 0x5603bbe35560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.016+0000 7f4c537da740 1 --2- 172.21.15.47:0/4130084680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe24590 0x5603bbe2e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.016+0000 7f4c537da740 1 --2- 172.21.15.47:0/4130084680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe23ba0 0x5603bbe2dfe0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.016+0000 7f4c537da740 1 -- 172.21.15.47:0/4130084680 >> 172.21.15.47:0/4130084680 conn(0x5603bbe1f110 msgr2=0x5603bbe1e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.016+0000 7f4c2f400640 0 client.25390 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:23:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.017+0000 7f4c537da740 1 -- 172.21.15.47:0/4130084680 shutdown_connections 2024-03-21T11:23:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.017+0000 7f4c537da740 1 -- 172.21.15.47:0/4130084680 wait complete. 2024-03-21T11:23:30.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.017+0000 7f4c537da740 20 client.25390 trim_cache size 0 max 16384 2024-03-21T11:23:30.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.035+0000 7f4c537da740 1 Processor -- start 2024-03-21T11:23:30.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.035+0000 7f4c537da740 1 -- start start 2024-03-21T11:23:30.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.035+0000 7f4c537da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe23ba0 0x5603bbe35bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c537da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe2cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c537da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe29700 0x5603bbe2baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c537da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bbe39b40 con 0x5603bbe23ba0 2024-03-21T11:23:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c537da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bbe3a1a0 con 0x5603bbe29700 2024-03-21T11:23:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c537da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bbe3a800 con 0x5603bbe24590 2024-03-21T11:23:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c49e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe23ba0 0x5603bbe35bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c4a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe29700 0x5603bbe2baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c4a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe29700 0x5603bbe2baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59570/0 (socket says 172.21.15.47:59570) 2024-03-21T11:23:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c4a800640 1 -- 172.21.15.47:0/26869672 learned_addr learned my addr 172.21.15.47:0/26869672 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c49400640 1 --2- 172.21.15.47:0/26869672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe2cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c4a800640 1 -- 172.21.15.47:0/26869672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 msgr2=0x5603bbe2cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c4a800640 1 --2- 172.21.15.47:0/26869672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe2cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c4a800640 1 -- 172.21.15.47:0/26869672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe23ba0 msgr2=0x5603bbe35bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c4a800640 1 --2- 172.21.15.47:0/26869672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe23ba0 0x5603bbe35bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.036+0000 7f4c4a800640 1 -- 172.21.15.47:0/26869672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603bbe2f110 con 0x5603bbe29700 2024-03-21T11:23:30.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.037+0000 7f4c4a800640 1 --2- 172.21.15.47:0/26869672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe29700 0x5603bbe2baf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c760 tx=0x7f4c440cd4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33af641c30edfc68 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.037+0000 7f4c3a600640 1 -- 172.21.15.47:0/26869672 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c440a0360 con 0x5603bbe29700 2024-03-21T11:23:30.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.037+0000 7f4c3a600640 1 -- 172.21.15.47:0/26869672 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c440b5070 con 0x5603bbe29700 2024-03-21T11:23:30.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.037+0000 7f4c3a600640 1 -- 172.21.15.47:0/26869672 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c440a4790 con 0x5603bbe29700 2024-03-21T11:23:30.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.037+0000 7f4c537da740 1 -- 172.21.15.47:0/26869672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe29700 msgr2=0x5603bbe2baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.037+0000 7f4c537da740 1 --2- 172.21.15.47:0/26869672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe29700 0x5603bbe2baf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c760 tx=0x7f4c440cd4b0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.038+0000 7f4c537da740 1 -- 172.21.15.47:0/26869672 shutdown_connections 2024-03-21T11:23:30.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.038+0000 7f4c537da740 1 --2- 172.21.15.47:0/26869672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe29700 0x5603bbe2baf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.038+0000 7f4c537da740 1 --2- 172.21.15.47:0/26869672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe2cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.038+0000 7f4c537da740 1 --2- 172.21.15.47:0/26869672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe23ba0 0x5603bbe35bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.038+0000 7f4c537da740 1 -- 172.21.15.47:0/26869672 >> 172.21.15.47:0/26869672 conn(0x5603bbe1f110 msgr2=0x5603bbe25000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.038+0000 7f4c537da740 1 -- 172.21.15.47:0/26869672 shutdown_connections 2024-03-21T11:23:30.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.038+0000 7f4c537da740 1 -- 172.21.15.47:0/26869672 wait complete. 2024-03-21T11:23:30.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.038+0000 7f4c537da740 1 -- start start 2024-03-21T11:23:30.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.038+0000 7f4c537da740 1 Processor -- start 2024-03-21T11:23:30.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c537da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe23ba0 0x5603bbe27410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c537da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe27950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c537da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe29700 0x5603bbe33510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c537da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bbe3b4c0 con 0x5603bbe29700 2024-03-21T11:23:30.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c537da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bbe3ae60 con 0x5603bbe23ba0 2024-03-21T11:23:30.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c537da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603bbe3bb20 con 0x5603bbe24590 2024-03-21T11:23:30.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c537da740 10 client.0 _ll_register_callbacks cb 0x7f4c3c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:23:30.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c49400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe27950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c49400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe27950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53274/0 (socket says 172.21.15.47:53274) 2024-03-21T11:23:30.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c49400640 1 -- 172.21.15.47:0/1267789212 learned_addr learned my addr 172.21.15.47:0/1267789212 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c49e00640 1 --2- 172.21.15.47:0/1267789212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe23ba0 0x5603bbe27410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c4a800640 1 --2- 172.21.15.47:0/1267789212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe29700 0x5603bbe33510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c49400640 1 -- 172.21.15.47:0/1267789212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe23ba0 msgr2=0x5603bbe27410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c49400640 1 --2- 172.21.15.47:0/1267789212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe23ba0 0x5603bbe27410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c49400640 1 -- 172.21.15.47:0/1267789212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe29700 msgr2=0x5603bbe33510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c49400640 1 --2- 172.21.15.47:0/1267789212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe29700 0x5603bbe33510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c49400640 1 -- 172.21.15.47:0/1267789212 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603bbe33b00 con 0x5603bbe24590 2024-03-21T11:23:30.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c4a800640 1 --2- 172.21.15.47:0/1267789212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe29700 0x5603bbe33510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:30.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.039+0000 7f4c49400640 1 --2- 172.21.15.47:0/1267789212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe27950 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c340058c0 tx=0x7f4c34004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a48b7b44f06d44a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c2f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:23:30.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c2f400640 1 -- 172.21.15.47:0/1267789212 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3402c050 con 0x5603bbe24590 2024-03-21T11:23:30.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c2f400640 1 -- 172.21.15.47:0/1267789212 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4c3402c1f0 con 0x5603bbe24590 2024-03-21T11:23:30.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c537da740 1 -- 172.21.15.47:0/1267789212 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c440b5070 con 0x5603bbe24590 2024-03-21T11:23:30.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c2f400640 1 -- 172.21.15.47:0/1267789212 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c3403a050 con 0x5603bbe24590 2024-03-21T11:23:30.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c537da740 10 client.25396 fetch_fsmap learned FSMap version 27 2024-03-21T11:23:30.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c537da740 1 -- 172.21.15.47:0/1267789212 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5603bbe34ea0 con 0x5603bbe24590 2024-03-21T11:23:30.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c2f400640 1 -- 172.21.15.47:0/1267789212 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c440b5070 con 0x5603bbe24590 2024-03-21T11:23:30.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c2f400640 1 -- 172.21.15.47:0/1267789212 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4c34004bb0 con 0x5603bbe24590 2024-03-21T11:23:30.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c537da740 10 client.25396 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:23:30.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c537da740 10 client.25396 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-21T11:23:30.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c537da740 20 client.25396 populate_metadata read hostname 'smithi047' 2024-03-21T11:23:30.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c537da740 1 --2- 172.21.15.47:0/1267789212 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5603bbea2410 0x5603bbe9e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c537da740 4 client.25396 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T11:23:30.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.040+0000 7f4c537da740 1 -- 172.21.15.47:0/1267789212 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5603bbd961d0 con 0x5603bbea2410 2024-03-21T11:23:30.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.041+0000 7f4c49e00640 1 --2- 172.21.15.47:0/1267789212 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5603bbea2410 0x5603bbe9e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.041+0000 7f4c49e00640 1 --2- 172.21.15.47:0/1267789212 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5603bbea2410 0x5603bbe9e0d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=db84da7c70378a4 server_cookie=2c37848830de97f7 in_seq=0 out_seq=0 2024-03-21T11:23:30.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.041+0000 7f4c2f400640 10 client.25396 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:23:30.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.041+0000 7f4c2f400640 1 -- 172.21.15.47:0/1267789212 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5603bbd961d0 con 0x5603bbea2410 2024-03-21T11:23:30.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.041+0000 7f4c2f400640 10 client.25396 handle_command_reply: tid=0 2024-03-21T11:23:30.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.041+0000 7f4c537da740 1 client.25396 shutdown 2024-03-21T11:23:30.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.042+0000 7f4c537da740 1 -- 172.21.15.47:0/1267789212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 msgr2=0x5603bbe27950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.042+0000 7f4c537da740 1 --2- 172.21.15.47:0/1267789212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe27950 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c340058c0 tx=0x7f4c34004730 comp rx=0 tx=0).stop 2024-03-21T11:23:30.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.042+0000 7f4c537da740 1 -- 172.21.15.47:0/1267789212 shutdown_connections 2024-03-21T11:23:30.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.042+0000 7f4c537da740 1 --2- 172.21.15.47:0/1267789212 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5603bbea2410 0x5603bbe9e0d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.042+0000 7f4c537da740 1 --2- 172.21.15.47:0/1267789212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603bbe29700 0x5603bbe33510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.042+0000 7f4c537da740 1 --2- 172.21.15.47:0/1267789212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603bbe24590 0x5603bbe27950 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.042+0000 7f4c537da740 1 --2- 172.21.15.47:0/1267789212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603bbe23ba0 0x5603bbe27410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.042+0000 7f4c537da740 1 -- 172.21.15.47:0/1267789212 >> 172.21.15.47:0/1267789212 conn(0x5603bbe1f110 msgr2=0x5603bbe1f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.042+0000 7f4c2f400640 0 client.25396 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:23:30.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.042+0000 7f4c537da740 1 -- 172.21.15.47:0/1267789212 shutdown_connections 2024-03-21T11:23:30.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.042+0000 7f4c537da740 1 -- 172.21.15.47:0/1267789212 wait complete. 2024-03-21T11:23:30.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.042+0000 7f4c537da740 20 client.25396 trim_cache size 0 max 16384 2024-03-21T11:23:30.053 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:23:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:23:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 3, 2024-03-21T11:23:30.053 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24524, 2024-03-21T11:23:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T11:23:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T11:23:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T11:23:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 133.18056630000001, 2024-03-21T11:23:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 27, 2024-03-21T11:23:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T11:23:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-03-21T11:23:30.054 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 181.939914926 2024-03-21T11:23:30.054 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:23:30.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.043+0000 7f4c50a00640 1 -- 172.21.15.47:0/1373646255 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c300a2f20 msgr2=0x7f4c300a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.043+0000 7f4c50a00640 1 --2- 172.21.15.47:0/1373646255 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c300a2f20 0x7f4c300a53e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5603bbe24df0 tx=0x7f4c34005850 comp rx=0 tx=0).stop 2024-03-21T11:23:30.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.043+0000 7f4c50a00640 1 -- 172.21.15.47:0/1373646255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c4c105470 msgr2=0x7f4c4c1a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.043+0000 7f4c50a00640 1 --2- 172.21.15.47:0/1373646255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c4c105470 0x7f4c4c1a1190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009fd0 tx=0x7f4c4400ef10 comp rx=0 tx=0).stop 2024-03-21T11:23:30.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.043+0000 7f4c50a00640 1 -- 172.21.15.47:0/1373646255 shutdown_connections 2024-03-21T11:23:30.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.043+0000 7f4c50a00640 1 --2- 172.21.15.47:0/1373646255 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c300a2f20 0x7f4c300a53e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.043+0000 7f4c50a00640 1 --2- 172.21.15.47:0/1373646255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c4c105470 0x7f4c4c1a1190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.043+0000 7f4c50a00640 1 --2- 172.21.15.47:0/1373646255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c4c104ab0 0x7f4c4c19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.043+0000 7f4c50a00640 1 --2- 172.21.15.47:0/1373646255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c4c1038c0 0x7f4c4c199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.043+0000 7f4c50a00640 1 -- 172.21.15.47:0/1373646255 >> 172.21.15.47:0/1373646255 conn(0x7f4c4c0fee50 msgr2=0x7f4c4c107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.043+0000 7f4c50a00640 1 -- 172.21.15.47:0/1373646255 shutdown_connections 2024-03-21T11:23:30.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.043+0000 7f4c50a00640 1 -- 172.21.15.47:0/1373646255 wait complete. 2024-03-21T11:23:30.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:29 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2464867992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:23:30.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:29 smithi047 ceph-mon[31351]: pgmap v405: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:23:30.059 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:29 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/611095911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:30.063 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-21T11:23:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:29 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2464867992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:23:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:29 smithi102 ceph-mon[36748]: pgmap v405: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:23:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:29 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/611095911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.288+0000 7f152d200640 1 -- 172.21.15.47:0/3454471673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1528105370 msgr2=0x7f152810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.288+0000 7f152d200640 1 --2- 172.21.15.47:0/3454471673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1528105370 0x7f152810bc00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1520009ca0 tx=0x7f152002c100 comp rx=0 tx=0).stop 2024-03-21T11:23:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.288+0000 7f152d200640 1 -- 172.21.15.47:0/3454471673 shutdown_connections 2024-03-21T11:23:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.288+0000 7f152d200640 1 --2- 172.21.15.47:0/3454471673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1528105370 0x7f152810bc00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.288+0000 7f152d200640 1 --2- 172.21.15.47:0/3454471673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15281049b0 0x7f1528104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.288+0000 7f152d200640 1 --2- 172.21.15.47:0/3454471673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15281037c0 0x7f1528103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.288+0000 7f152d200640 1 -- 172.21.15.47:0/3454471673 >> 172.21.15.47:0/3454471673 conn(0x7f15280fed50 msgr2=0x7f1528101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.288+0000 7f152d200640 1 -- 172.21.15.47:0/3454471673 shutdown_connections 2024-03-21T11:23:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.288+0000 7f152d200640 1 -- 172.21.15.47:0/3454471673 wait complete. 2024-03-21T11:23:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.289+0000 7f152d200640 1 Processor -- start 2024-03-21T11:23:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.289+0000 7f152d200640 1 -- start start 2024-03-21T11:23:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f152d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15281037c0 0x7f1528199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f152d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15281049b0 0x7f1528199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f152d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1528105370 0x7f15281a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f152d200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152810f440 con 0x7f15281049b0 2024-03-21T11:23:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f152d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152810f2c0 con 0x7f1528105370 2024-03-21T11:23:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f152d200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152810f5c0 con 0x7f15281037c0 2024-03-21T11:23:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f1527400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15281037c0 0x7f1528199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f1527400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15281037c0 0x7f1528199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53294/0 (socket says 172.21.15.47:53294) 2024-03-21T11:23:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f1527400640 1 -- 172.21.15.47:0/432810752 learned_addr learned my addr 172.21.15.47:0/432810752 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f1526a00640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15281049b0 0x7f1528199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f1527e00640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1528105370 0x7f15281a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f1527400640 1 -- 172.21.15.47:0/432810752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1528105370 msgr2=0x7f15281a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f1527400640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1528105370 0x7f15281a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f1527400640 1 -- 172.21.15.47:0/432810752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15281049b0 msgr2=0x7f1528199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f1527400640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15281049b0 0x7f1528199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f1527400640 1 -- 172.21.15.47:0/432810752 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15281a1670 con 0x7f15281037c0 2024-03-21T11:23:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.290+0000 7f1526a00640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15281049b0 0x7f1528199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.291+0000 7f1527400640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15281037c0 0x7f1528199a50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f151800d950 tx=0x7f151800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.291+0000 7f151fe00640 1 -- 172.21.15.47:0/432810752 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518017020 con 0x7f15281037c0 2024-03-21T11:23:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.291+0000 7f152d200640 1 -- 172.21.15.47:0/432810752 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15281a1960 con 0x7f15281037c0 2024-03-21T11:23:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.291+0000 7f151fe00640 1 -- 172.21.15.47:0/432810752 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1518009c30 con 0x7f15281037c0 2024-03-21T11:23:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.291+0000 7f151fe00640 1 -- 172.21.15.47:0/432810752 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151800f6b0 con 0x7f15281037c0 2024-03-21T11:23:30.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.291+0000 7f152d200640 1 -- 172.21.15.47:0/432810752 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15281a1f00 con 0x7f15281037c0 2024-03-21T11:23:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.294+0000 7f151fe00640 1 -- 172.21.15.47:0/432810752 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f151801a020 con 0x7f15281037c0 2024-03-21T11:23:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.294+0000 7f1531602740 1 Processor -- start 2024-03-21T11:23:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.295+0000 7f1531602740 1 -- start start 2024-03-21T11:23:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.295+0000 7f1531602740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80ad80 0x55c4dd80b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.295+0000 7f1531602740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80b770 0x55c4dd810270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.295+0000 7f1531602740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 0x55c4dd812d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.295+0000 7f1531602740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dd76fc90 con 0x55c4dd8108e0 2024-03-21T11:23:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.295+0000 7f1531602740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dd718980 con 0x55c4dd80ad80 2024-03-21T11:23:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.295+0000 7f1531602740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dd76bd30 con 0x55c4dd80b770 2024-03-21T11:23:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.295+0000 7f1527e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80ad80 0x55c4dd80b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.295+0000 7f1527400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80b770 0x55c4dd810270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.295+0000 7f1527e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80ad80 0x55c4dd80b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59610/0 (socket says 172.21.15.47:59610) 2024-03-21T11:23:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.295+0000 7f1527e00640 1 -- 172.21.15.47:0/400102161 learned_addr learned my addr 172.21.15.47:0/400102161 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.295+0000 7f1526a00640 1 --2- 172.21.15.47:0/400102161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 0x55c4dd812d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.296+0000 7f1527e00640 1 -- 172.21.15.47:0/400102161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80b770 msgr2=0x55c4dd810270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.296+0000 7f1527e00640 1 --2- 172.21.15.47:0/400102161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80b770 0x55c4dd810270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.296+0000 7f1527e00640 1 -- 172.21.15.47:0/400102161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 msgr2=0x55c4dd812d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.296+0000 7f1527e00640 1 --2- 172.21.15.47:0/400102161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 0x55c4dd812d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.296+0000 7f1527e00640 1 -- 172.21.15.47:0/400102161 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4dd730510 con 0x55c4dd80ad80 2024-03-21T11:23:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.296+0000 7f1527e00640 1 --2- 172.21.15.47:0/400102161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80ad80 0x55c4dd80b1a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f152002c710 tx=0x7f15200323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35770215da789680 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.296+0000 7f151fe00640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f150c0a2fa0 0x7f150c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.296+0000 7f151cc00640 1 -- 172.21.15.47:0/400102161 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520043070 con 0x55c4dd80ad80 2024-03-21T11:23:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.296+0000 7f151fe00640 1 -- 172.21.15.47:0/432810752 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f15180cf740 con 0x7f15281037c0 2024-03-21T11:23:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.296+0000 7f151cc00640 1 -- 172.21.15.47:0/400102161 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1520032c90 con 0x55c4dd80ad80 2024-03-21T11:23:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.296+0000 7f151cc00640 1 -- 172.21.15.47:0/400102161 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520039040 con 0x55c4dd80ad80 2024-03-21T11:23:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.297+0000 7f1527400640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f150c0a2fa0 0x7f150c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.297+0000 7f1531602740 1 -- 172.21.15.47:0/400102161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80ad80 msgr2=0x55c4dd80b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.297+0000 7f1527400640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f150c0a2fa0 0x7f150c0a5460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1518097ee0 tx=0x7f15180d5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.297+0000 7f1531602740 1 --2- 172.21.15.47:0/400102161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80ad80 0x55c4dd80b1a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f152002c710 tx=0x7f15200323d0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.297+0000 7f1531602740 1 -- 172.21.15.47:0/400102161 shutdown_connections 2024-03-21T11:23:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.297+0000 7f1531602740 1 --2- 172.21.15.47:0/400102161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 0x55c4dd812d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.297+0000 7f1531602740 1 --2- 172.21.15.47:0/400102161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80b770 0x55c4dd810270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.297+0000 7f1531602740 1 --2- 172.21.15.47:0/400102161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80ad80 0x55c4dd80b1a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.297+0000 7f1531602740 1 -- 172.21.15.47:0/400102161 >> 172.21.15.47:0/400102161 conn(0x55c4dd8062f0 msgr2=0x55c4dd8066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.298+0000 7f1531602740 1 -- 172.21.15.47:0/400102161 shutdown_connections 2024-03-21T11:23:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.298+0000 7f1531602740 1 -- 172.21.15.47:0/400102161 wait complete. 2024-03-21T11:23:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.298+0000 7f1531602740 1 -- start start 2024-03-21T11:23:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.298+0000 7f1531602740 1 Processor -- start 2024-03-21T11:23:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1531602740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80ad80 0x55c4dd815240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1531602740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80b770 0x55c4dd815780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1531602740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 0x55c4dd81c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1531602740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dd813830 con 0x55c4dd8108e0 2024-03-21T11:23:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1531602740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dd72bdd0 con 0x55c4dd80b770 2024-03-21T11:23:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1531602740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dd8139b0 con 0x55c4dd80ad80 2024-03-21T11:23:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1531602740 10 client.0 _ll_register_callbacks cb 0x55c4dd483ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:23:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1526a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80b770 0x55c4dd815780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1527e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80ad80 0x55c4dd815240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1526a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80b770 0x55c4dd815780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59614/0 (socket says 172.21.15.47:59614) 2024-03-21T11:23:30.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1526a00640 1 -- 172.21.15.47:0/4212666101 learned_addr learned my addr 172.21.15.47:0/4212666101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1527e00640 1 --2- 172.21.15.47:0/4212666101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 0x55c4dd81c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1526a00640 1 -- 172.21.15.47:0/4212666101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80ad80 msgr2=0x55c4dd815240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1526a00640 1 --2- 172.21.15.47:0/4212666101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80ad80 0x55c4dd815240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1526a00640 1 -- 172.21.15.47:0/4212666101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 msgr2=0x55c4dd81c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1526a00640 1 --2- 172.21.15.47:0/4212666101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 0x55c4dd81c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1526a00640 1 -- 172.21.15.47:0/4212666101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4dd4842d0 con 0x55c4dd80b770 2024-03-21T11:23:30.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.299+0000 7f1527e00640 1 --2- 172.21.15.47:0/4212666101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 0x55c4dd81c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:30.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.300+0000 7f1526a00640 1 --2- 172.21.15.47:0/4212666101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80b770 0x55c4dd815780 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1514012700 tx=0x7f1514012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=755672fc27d5f172 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.300+0000 7f1511600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:23:30.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.300+0000 7f1511600640 1 -- 172.21.15.47:0/4212666101 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514011c70 con 0x55c4dd80b770 2024-03-21T11:23:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.300+0000 7f1511600640 1 -- 172.21.15.47:0/4212666101 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1514011e10 con 0x55c4dd80b770 2024-03-21T11:23:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.300+0000 7f1531602740 1 -- 172.21.15.47:0/4212666101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1520032c90 con 0x55c4dd80b770 2024-03-21T11:23:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.300+0000 7f1511600640 1 -- 172.21.15.47:0/4212666101 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f151400f460 con 0x55c4dd80b770 2024-03-21T11:23:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.300+0000 7f1531602740 10 client.25394 fetch_fsmap learned FSMap version 27 2024-03-21T11:23:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.300+0000 7f1531602740 1 -- 172.21.15.47:0/4212666101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c4dd54e6b0 con 0x55c4dd80b770 2024-03-21T11:23:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.301+0000 7f1511600640 1 -- 172.21.15.47:0/4212666101 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1520032c90 con 0x55c4dd80b770 2024-03-21T11:23:30.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.301+0000 7f1511600640 1 -- 172.21.15.47:0/4212666101 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f151401c910 con 0x55c4dd80b770 2024-03-21T11:23:30.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.301+0000 7f1531602740 10 client.25394 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:23:30.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.301+0000 7f1531602740 10 client.25394 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-21T11:23:30.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.301+0000 7f1531602740 20 client.25394 populate_metadata read hostname 'smithi047' 2024-03-21T11:23:30.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.301+0000 7f1531602740 1 --2- 172.21.15.47:0/4212666101 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c4dd821a10 0x55c4dd820aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.301+0000 7f1531602740 4 client.25394 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:23:30.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.301+0000 7f1531602740 1 -- 172.21.15.47:0/4212666101 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4dd711520 con 0x55c4dd821a10 2024-03-21T11:23:30.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.301+0000 7f1527e00640 1 --2- 172.21.15.47:0/4212666101 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c4dd821a10 0x55c4dd820aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.302+0000 7f1527e00640 1 --2- 172.21.15.47:0/4212666101 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c4dd821a10 0x55c4dd820aa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79001d600912621c server_cookie=d60f84c0d16e8eda in_seq=0 out_seq=0 2024-03-21T11:23:30.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.302+0000 7f1511600640 10 client.25394 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:23:30.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1511600640 1 -- 172.21.15.47:0/4212666101 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c4dd711520 con 0x55c4dd821a10 2024-03-21T11:23:30.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1511600640 10 client.25394 handle_command_reply: tid=0 2024-03-21T11:23:30.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1531602740 1 client.25394 shutdown 2024-03-21T11:23:30.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1531602740 1 -- 172.21.15.47:0/4212666101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80b770 msgr2=0x55c4dd815780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1531602740 1 --2- 172.21.15.47:0/4212666101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80b770 0x55c4dd815780 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1514012700 tx=0x7f1514012bc0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1531602740 1 -- 172.21.15.47:0/4212666101 shutdown_connections 2024-03-21T11:23:30.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1531602740 1 --2- 172.21.15.47:0/4212666101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 0x55c4dd81c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1531602740 1 --2- 172.21.15.47:0/4212666101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80b770 0x55c4dd815780 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1531602740 1 --2- 172.21.15.47:0/4212666101 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c4dd821a10 0x55c4dd820aa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1531602740 1 --2- 172.21.15.47:0/4212666101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80ad80 0x55c4dd815240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1511600640 0 client.25394 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:23:30.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1531602740 1 -- 172.21.15.47:0/4212666101 >> 172.21.15.47:0/4212666101 conn(0x55c4dd8062f0 msgr2=0x55c4dd8054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1531602740 1 -- 172.21.15.47:0/4212666101 shutdown_connections 2024-03-21T11:23:30.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1531602740 1 -- 172.21.15.47:0/4212666101 wait complete. 2024-03-21T11:23:30.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.303+0000 7f1531602740 20 client.25394 trim_cache size 0 max 16384 2024-03-21T11:23:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.322+0000 7f1531602740 1 Processor -- start 2024-03-21T11:23:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.322+0000 7f1531602740 1 -- start start 2024-03-21T11:23:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.322+0000 7f1531602740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80ad80 0x55c4dd8102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.322+0000 7f1531602740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd80b770 0x55c4dd81c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.322+0000 7f1531602740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd8108e0 0x55c4dd80c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.323+0000 7f1531602740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dd821230 con 0x55c4dd80b770 2024-03-21T11:23:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.323+0000 7f1531602740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dd821890 con 0x55c4dd80ad80 2024-03-21T11:23:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.323+0000 7f1531602740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dd821ef0 con 0x55c4dd8108e0 2024-03-21T11:23:30.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.323+0000 7f1527e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80ad80 0x55c4dd8102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.323+0000 7f1526a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd80b770 0x55c4dd81c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.323+0000 7f1527e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80ad80 0x55c4dd8102a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59618/0 (socket says 172.21.15.47:59618) 2024-03-21T11:23:30.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.323+0000 7f1527e00640 1 -- 172.21.15.47:0/3818560258 learned_addr learned my addr 172.21.15.47:0/3818560258 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.323+0000 7f1527e00640 1 --2- 172.21.15.47:0/3818560258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd8108e0 0x55c4dd80c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.323+0000 7f1527e00640 1 -- 172.21.15.47:0/3818560258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd8108e0 msgr2=0x55c4dd80c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.323+0000 7f1527e00640 1 --2- 172.21.15.47:0/3818560258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd8108e0 0x55c4dd80c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.323+0000 7f1527e00640 1 -- 172.21.15.47:0/3818560258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd80b770 msgr2=0x55c4dd81c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.323+0000 7f1527e00640 1 --2- 172.21.15.47:0/3818560258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd80b770 0x55c4dd81c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.323+0000 7f1527e00640 1 -- 172.21.15.47:0/3818560258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4dd807060 con 0x55c4dd80ad80 2024-03-21T11:23:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.323+0000 7f1527e00640 1 --2- 172.21.15.47:0/3818560258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80ad80 0x55c4dd8102a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1520009ca0 tx=0x7f152002e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8e255f9436f580d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.324+0000 7f1513e00640 1 -- 172.21.15.47:0/3818560258 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520039040 con 0x55c4dd80ad80 2024-03-21T11:23:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.324+0000 7f1513e00640 1 -- 172.21.15.47:0/3818560258 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15200552d0 con 0x55c4dd80ad80 2024-03-21T11:23:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.324+0000 7f1513e00640 1 -- 172.21.15.47:0/3818560258 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520043040 con 0x55c4dd80ad80 2024-03-21T11:23:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.324+0000 7f1531602740 1 -- 172.21.15.47:0/3818560258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80ad80 msgr2=0x55c4dd8102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.324+0000 7f1531602740 1 --2- 172.21.15.47:0/3818560258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80ad80 0x55c4dd8102a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1520009ca0 tx=0x7f152002e040 comp rx=0 tx=0).stop 2024-03-21T11:23:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.324+0000 7f1531602740 1 -- 172.21.15.47:0/3818560258 shutdown_connections 2024-03-21T11:23:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.324+0000 7f1531602740 1 --2- 172.21.15.47:0/3818560258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd8108e0 0x55c4dd80c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.324+0000 7f1531602740 1 --2- 172.21.15.47:0/3818560258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd80b770 0x55c4dd81c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.324+0000 7f1531602740 1 --2- 172.21.15.47:0/3818560258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80ad80 0x55c4dd8102a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.324+0000 7f1531602740 1 -- 172.21.15.47:0/3818560258 >> 172.21.15.47:0/3818560258 conn(0x55c4dd8062f0 msgr2=0x55c4dd813d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.324+0000 7f1531602740 1 -- 172.21.15.47:0/3818560258 shutdown_connections 2024-03-21T11:23:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.325+0000 7f1531602740 1 -- 172.21.15.47:0/3818560258 wait complete. 2024-03-21T11:23:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.325+0000 7f1531602740 1 -- start start 2024-03-21T11:23:30.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.325+0000 7f1531602740 1 Processor -- start 2024-03-21T11:23:30.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1531602740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80ad80 0x55c4dd81a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1531602740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80b770 0x55c4dd878a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1531602740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 0x55c4dd87afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1531602740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dd822bb0 con 0x55c4dd8108e0 2024-03-21T11:23:30.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1531602740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dd822550 con 0x55c4dd80b770 2024-03-21T11:23:30.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1531602740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dd823210 con 0x55c4dd80ad80 2024-03-21T11:23:30.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1531602740 10 client.0 _ll_register_callbacks cb 0x7f1514009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:23:30.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1526a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80b770 0x55c4dd878a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1527e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80ad80 0x55c4dd81a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1526a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80b770 0x55c4dd878a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59624/0 (socket says 172.21.15.47:59624) 2024-03-21T11:23:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1526a00640 1 -- 172.21.15.47:0/1574447775 learned_addr learned my addr 172.21.15.47:0/1574447775 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1527e00640 1 --2- 172.21.15.47:0/1574447775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 0x55c4dd87afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1526a00640 1 -- 172.21.15.47:0/1574447775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80ad80 msgr2=0x55c4dd81a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1526a00640 1 --2- 172.21.15.47:0/1574447775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80ad80 0x55c4dd81a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1526a00640 1 -- 172.21.15.47:0/1574447775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 msgr2=0x55c4dd87afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1526a00640 1 --2- 172.21.15.47:0/1574447775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 0x55c4dd87afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1526a00640 1 -- 172.21.15.47:0/1574447775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4dd87b780 con 0x55c4dd80b770 2024-03-21T11:23:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.326+0000 7f1527e00640 1 --2- 172.21.15.47:0/1574447775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 0x55c4dd87afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.327+0000 7f1526a00640 1 --2- 172.21.15.47:0/1574447775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80b770 0x55c4dd878a50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f15140126d0 tx=0x7f1514011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fa31def7d7ce38 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.327+0000 7f1511600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:23:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.327+0000 7f1511600640 1 -- 172.21.15.47:0/1574447775 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151402e070 con 0x55c4dd80b770 2024-03-21T11:23:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.327+0000 7f1511600640 1 -- 172.21.15.47:0/1574447775 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f151401c630 con 0x55c4dd80b770 2024-03-21T11:23:30.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.327+0000 7f1531602740 1 -- 172.21.15.47:0/1574447775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15200552d0 con 0x55c4dd80b770 2024-03-21T11:23:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.327+0000 7f1511600640 1 -- 172.21.15.47:0/1574447775 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1514011e60 con 0x55c4dd80b770 2024-03-21T11:23:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.327+0000 7f1531602740 10 client.25400 fetch_fsmap learned FSMap version 27 2024-03-21T11:23:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.327+0000 7f1531602740 1 -- 172.21.15.47:0/1574447775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c4dd87caf0 con 0x55c4dd80b770 2024-03-21T11:23:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.327+0000 7f1511600640 1 -- 172.21.15.47:0/1574447775 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15200552d0 con 0x55c4dd80b770 2024-03-21T11:23:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.328+0000 7f1511600640 1 -- 172.21.15.47:0/1574447775 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1514033460 con 0x55c4dd80b770 2024-03-21T11:23:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.328+0000 7f1531602740 10 client.25400 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:23:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.328+0000 7f1531602740 10 client.25400 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-21T11:23:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.328+0000 7f1531602740 20 client.25400 populate_metadata read hostname 'smithi047' 2024-03-21T11:23:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.328+0000 7f1531602740 1 --2- 172.21.15.47:0/1574447775 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c4dd881160 0x55c4dd87cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.328+0000 7f1531602740 4 client.25400 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T11:23:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.328+0000 7f1531602740 1 -- 172.21.15.47:0/1574447775 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c4dd77d1d0 con 0x55c4dd881160 2024-03-21T11:23:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.328+0000 7f1527e00640 1 --2- 172.21.15.47:0/1574447775 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c4dd881160 0x55c4dd87cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.328+0000 7f1527e00640 1 --2- 172.21.15.47:0/1574447775 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c4dd881160 0x55c4dd87cd80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c872c064a9c276c server_cookie=2ba562aa3f4c3335 in_seq=0 out_seq=0 2024-03-21T11:23:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.328+0000 7f1511600640 10 client.25400 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:23:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1511600640 1 -- 172.21.15.47:0/1574447775 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c4dd77d1d0 con 0x55c4dd881160 2024-03-21T11:23:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1511600640 10 client.25400 handle_command_reply: tid=0 2024-03-21T11:23:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1531602740 1 client.25400 shutdown 2024-03-21T11:23:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1531602740 1 -- 172.21.15.47:0/1574447775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80b770 msgr2=0x55c4dd878a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1531602740 1 --2- 172.21.15.47:0/1574447775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80b770 0x55c4dd878a50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f15140126d0 tx=0x7f1514011c70 comp rx=0 tx=0).stop 2024-03-21T11:23:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1531602740 1 -- 172.21.15.47:0/1574447775 shutdown_connections 2024-03-21T11:23:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1531602740 1 --2- 172.21.15.47:0/1574447775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4dd8108e0 0x55c4dd87afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1531602740 1 --2- 172.21.15.47:0/1574447775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4dd80b770 0x55c4dd878a50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1531602740 1 --2- 172.21.15.47:0/1574447775 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c4dd881160 0x55c4dd87cd80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1531602740 1 --2- 172.21.15.47:0/1574447775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4dd80ad80 0x55c4dd81a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1531602740 1 -- 172.21.15.47:0/1574447775 >> 172.21.15.47:0/1574447775 conn(0x55c4dd8062f0 msgr2=0x55c4dd80c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1511600640 0 client.25400 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:23:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1531602740 1 -- 172.21.15.47:0/1574447775 shutdown_connections 2024-03-21T11:23:30.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1531602740 1 -- 172.21.15.47:0/1574447775 wait complete. 2024-03-21T11:23:30.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.329+0000 7f1531602740 20 client.25400 trim_cache size 0 max 16384 2024-03-21T11:23:30.335 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:23:30.335 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:23:30.335 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 2, 2024-03-21T11:23:30.335 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24535, 2024-03-21T11:23:30.335 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T11:23:30.335 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T11:23:30.335 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T11:23:30.336 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 134.48885090799999, 2024-03-21T11:23:30.336 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 27, 2024-03-21T11:23:30.336 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T11:23:30.336 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-03-21T11:23:30.336 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 185.09305570699999 2024-03-21T11:23:30.336 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:23:30.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.330+0000 7f152d200640 1 -- 172.21.15.47:0/432810752 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f150c0a2fa0 msgr2=0x7f150c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.330+0000 7f152d200640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f150c0a2fa0 0x7f150c0a5460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1518097ee0 tx=0x7f15180d5530 comp rx=0 tx=0).stop 2024-03-21T11:23:30.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.330+0000 7f152d200640 1 -- 172.21.15.47:0/432810752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15281037c0 msgr2=0x7f1528199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.330+0000 7f152d200640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15281037c0 0x7f1528199a50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f151800d950 tx=0x7f151800de10 comp rx=0 tx=0).stop 2024-03-21T11:23:30.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.330+0000 7f152d200640 1 -- 172.21.15.47:0/432810752 shutdown_connections 2024-03-21T11:23:30.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.330+0000 7f152d200640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f150c0a2fa0 0x7f150c0a5460 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.330+0000 7f152d200640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1528105370 0x7f15281a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.330+0000 7f152d200640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15281049b0 0x7f1528199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.330+0000 7f152d200640 1 --2- 172.21.15.47:0/432810752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15281037c0 0x7f1528199a50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.330+0000 7f152d200640 1 -- 172.21.15.47:0/432810752 >> 172.21.15.47:0/432810752 conn(0x7f15280fed50 msgr2=0x7f1528107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.330+0000 7f152d200640 1 -- 172.21.15.47:0/432810752 shutdown_connections 2024-03-21T11:23:30.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.330+0000 7f152d200640 1 -- 172.21.15.47:0/432810752 wait complete. 2024-03-21T11:23:30.348 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-21T11:23:30.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:29 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2464867992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:23:30.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:29 smithi100 ceph-mon[36245]: pgmap v405: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T11:23:30.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:29 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/611095911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.553+0000 7fba7c800640 1 -- 172.21.15.47:0/3421236830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba740772a0 msgr2=0x7fba74077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.553+0000 7fba7c800640 1 --2- 172.21.15.47:0/3421236830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba740772a0 0x7fba74077700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fba64009900 tx=0x7fba6402bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.554+0000 7fba7c800640 1 -- 172.21.15.47:0/3421236830 shutdown_connections 2024-03-21T11:23:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.554+0000 7fba7c800640 1 --2- 172.21.15.47:0/3421236830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba74077c40 0x7fba740784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.554+0000 7fba7c800640 1 --2- 172.21.15.47:0/3421236830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba740772a0 0x7fba74077700 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.554+0000 7fba7c800640 1 --2- 172.21.15.47:0/3421236830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba74076060 0x7fba74076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.554+0000 7fba7c800640 1 -- 172.21.15.47:0/3421236830 >> 172.21.15.47:0/3421236830 conn(0x7fba740fee50 msgr2=0x7fba74101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.554+0000 7fba7c800640 1 -- 172.21.15.47:0/3421236830 shutdown_connections 2024-03-21T11:23:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.554+0000 7fba7c800640 1 -- 172.21.15.47:0/3421236830 wait complete. 2024-03-21T11:23:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.555+0000 7fba7c800640 1 Processor -- start 2024-03-21T11:23:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.555+0000 7fba7c800640 1 -- start start 2024-03-21T11:23:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.556+0000 7fba7c800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba74076060 0x7fba7419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.556+0000 7fba7c800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba740772a0 0x7fba7419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.556+0000 7fba7c800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba74077c40 0x7fba741a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.556+0000 7fba7c800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba74113970 con 0x7fba74076060 2024-03-21T11:23:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.556+0000 7fba7c800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba741137f0 con 0x7fba740772a0 2024-03-21T11:23:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.556+0000 7fba7c800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba74113af0 con 0x7fba74077c40 2024-03-21T11:23:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.556+0000 7fba79e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba74076060 0x7fba7419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.556+0000 7fba79400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba740772a0 0x7fba7419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.556+0000 7fba79e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba74076060 0x7fba7419e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46564/0 (socket says 172.21.15.47:46564) 2024-03-21T11:23:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.556+0000 7fba79e00640 1 -- 172.21.15.47:0/4232364192 learned_addr learned my addr 172.21.15.47:0/4232364192 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.556+0000 7fba7a800640 1 --2- 172.21.15.47:0/4232364192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba74077c40 0x7fba741a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.556+0000 7fba79400640 1 -- 172.21.15.47:0/4232364192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba74077c40 msgr2=0x7fba741a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.556+0000 7fba79400640 1 --2- 172.21.15.47:0/4232364192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba74077c40 0x7fba741a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.556+0000 7fba79400640 1 -- 172.21.15.47:0/4232364192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba74076060 msgr2=0x7fba7419e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.557+0000 7fba79400640 1 --2- 172.21.15.47:0/4232364192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba74076060 0x7fba7419e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.557+0000 7fba79400640 1 -- 172.21.15.47:0/4232364192 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba741a5c50 con 0x7fba740772a0 2024-03-21T11:23:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.557+0000 7fba79e00640 1 --2- 172.21.15.47:0/4232364192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba74076060 0x7fba7419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.557+0000 7fba79400640 1 --2- 172.21.15.47:0/4232364192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba740772a0 0x7fba7419e570 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fba64036040 tx=0x7fba6402c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.557+0000 7fba6ea00640 1 -- 172.21.15.47:0/4232364192 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64043020 con 0x7fba740772a0 2024-03-21T11:23:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.557+0000 7fba6ea00640 1 -- 172.21.15.47:0/4232364192 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba640054e0 con 0x7fba740772a0 2024-03-21T11:23:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.557+0000 7fba7c800640 1 -- 172.21.15.47:0/4232364192 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba741a5ee0 con 0x7fba740772a0 2024-03-21T11:23:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.557+0000 7fba6ea00640 1 -- 172.21.15.47:0/4232364192 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64003ce0 con 0x7fba740772a0 2024-03-21T11:23:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.557+0000 7fba7c800640 1 -- 172.21.15.47:0/4232364192 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba741a63f0 con 0x7fba740772a0 2024-03-21T11:23:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.560+0000 7fba6ea00640 1 -- 172.21.15.47:0/4232364192 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fba64046020 con 0x7fba740772a0 2024-03-21T11:23:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.560+0000 7fba7f5ec740 1 Processor -- start 2024-03-21T11:23:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.561+0000 7fba7f5ec740 1 -- start start 2024-03-21T11:23:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.561+0000 7fba7f5ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.561+0000 7fba7f5ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c3590 0x5566ad8c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.561+0000 7fba7f5ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c8700 0x5566ad8cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.561+0000 7fba7f5ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ad827c90 con 0x5566ad8c2ba0 2024-03-21T11:23:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.561+0000 7fba7f5ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ad7d0980 con 0x5566ad8c8700 2024-03-21T11:23:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.561+0000 7fba7f5ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ad823d30 con 0x5566ad8c3590 2024-03-21T11:23:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.561+0000 7fba79e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c3590 0x5566ad8c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.561+0000 7fba7a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.561+0000 7fba79400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c8700 0x5566ad8cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.561+0000 7fba79e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c3590 0x5566ad8c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53372/0 (socket says 172.21.15.47:53372) 2024-03-21T11:23:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.561+0000 7fba79e00640 1 -- 172.21.15.47:0/2256673238 learned_addr learned my addr 172.21.15.47:0/2256673238 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.562+0000 7fba79e00640 1 -- 172.21.15.47:0/2256673238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c8700 msgr2=0x5566ad8cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.562+0000 7fba79e00640 1 --2- 172.21.15.47:0/2256673238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c8700 0x5566ad8cab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.562+0000 7fba79e00640 1 -- 172.21.15.47:0/2256673238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 msgr2=0x5566ad8c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.562+0000 7fba79e00640 1 --2- 172.21.15.47:0/2256673238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8c2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.562+0000 7fba79e00640 1 -- 172.21.15.47:0/2256673238 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566ad7e8510 con 0x5566ad8c3590 2024-03-21T11:23:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.562+0000 7fba79e00640 1 --2- 172.21.15.47:0/2256673238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c3590 0x5566ad8c8090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fba68011650 tx=0x7fba68009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aaecd9609bc8290 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.562+0000 7fba63000640 1 -- 172.21.15.47:0/2256673238 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6801e070 con 0x5566ad8c3590 2024-03-21T11:23:30.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.562+0000 7fba63000640 1 -- 172.21.15.47:0/2256673238 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba6800fba0 con 0x5566ad8c3590 2024-03-21T11:23:30.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.562+0000 7fba63000640 1 -- 172.21.15.47:0/2256673238 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68019cd0 con 0x5566ad8c3590 2024-03-21T11:23:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.562+0000 7fba6ea00640 1 --2- 172.21.15.47:0/4232364192 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fba5c0a2fa0 0x7fba5c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.563+0000 7fba6ea00640 1 -- 172.21.15.47:0/4232364192 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fba640f17c0 con 0x7fba740772a0 2024-03-21T11:23:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.563+0000 7fba7a800640 1 --2- 172.21.15.47:0/4232364192 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fba5c0a2fa0 0x7fba5c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.563+0000 7fba7f5ec740 1 -- 172.21.15.47:0/2256673238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c3590 msgr2=0x5566ad8c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.563+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/2256673238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c3590 0x5566ad8c8090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fba68011650 tx=0x7fba68009f90 comp rx=0 tx=0).stop 2024-03-21T11:23:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.563+0000 7fba7f5ec740 1 -- 172.21.15.47:0/2256673238 shutdown_connections 2024-03-21T11:23:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.563+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/2256673238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c8700 0x5566ad8cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.563+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/2256673238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c3590 0x5566ad8c8090 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.563+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/2256673238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.563+0000 7fba7f5ec740 1 -- 172.21.15.47:0/2256673238 >> 172.21.15.47:0/2256673238 conn(0x5566ad8be110 msgr2=0x5566ad8be510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.563+0000 7fba7f5ec740 1 -- 172.21.15.47:0/2256673238 shutdown_connections 2024-03-21T11:23:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.563+0000 7fba7f5ec740 1 -- 172.21.15.47:0/2256673238 wait complete. 2024-03-21T11:23:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.563+0000 7fba7a800640 1 --2- 172.21.15.47:0/4232364192 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fba5c0a2fa0 0x7fba5c0a5460 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5566ad8c3df0 tx=0x7fba700096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.564+0000 7fba7f5ec740 1 -- start start 2024-03-21T11:23:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.564+0000 7fba7f5ec740 1 Processor -- start 2024-03-21T11:23:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.564+0000 7fba7f5ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c2ba0 0x5566ad8cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.564+0000 7fba7f5ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c3590 0x5566ad8cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.564+0000 7fba7f5ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 0x5566ad8d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.564+0000 7fba7f5ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ad8cb6b0 con 0x5566ad8c3590 2024-03-21T11:23:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.564+0000 7fba7f5ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ad7e3dd0 con 0x5566ad8c2ba0 2024-03-21T11:23:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.564+0000 7fba7f5ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ad8cb830 con 0x5566ad8c8700 2024-03-21T11:23:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.564+0000 7fba7f5ec740 10 client.0 _ll_register_callbacks cb 0x5566ad53bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:23:30.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.565+0000 7fba79e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c2ba0 0x5566ad8cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.565+0000 7fba7a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 0x5566ad8d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.565+0000 7fba79e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c2ba0 0x5566ad8cd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59678/0 (socket says 172.21.15.47:59678) 2024-03-21T11:23:30.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.565+0000 7fba79e00640 1 -- 172.21.15.47:0/1238428608 learned_addr learned my addr 172.21.15.47:0/1238428608 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.565+0000 7fba79400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c3590 0x5566ad8cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.565+0000 7fba79e00640 1 -- 172.21.15.47:0/1238428608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 msgr2=0x5566ad8d4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.565+0000 7fba79e00640 1 --2- 172.21.15.47:0/1238428608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 0x5566ad8d4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.565+0000 7fba79e00640 1 -- 172.21.15.47:0/1238428608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c3590 msgr2=0x5566ad8cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.565+0000 7fba79e00640 1 --2- 172.21.15.47:0/1238428608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c3590 0x5566ad8cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.565+0000 7fba79e00640 1 -- 172.21.15.47:0/1238428608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566ad53c2d0 con 0x5566ad8c2ba0 2024-03-21T11:23:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.565+0000 7fba79e00640 1 --2- 172.21.15.47:0/1238428608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c2ba0 0x5566ad8cd0f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fba6801b000 tx=0x7fba6800faf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=358e4b8fcbcd41dc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.565+0000 7fba57400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:23:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.566+0000 7fba57400640 1 -- 172.21.15.47:0/1238428608 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6801e040 con 0x5566ad8c2ba0 2024-03-21T11:23:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.566+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1238428608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba6800fba0 con 0x5566ad8c2ba0 2024-03-21T11:23:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.566+0000 7fba57400640 1 -- 172.21.15.47:0/1238428608 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fba6800fdd0 con 0x5566ad8c2ba0 2024-03-21T11:23:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.566+0000 7fba57400640 1 -- 172.21.15.47:0/1238428608 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba6802d050 con 0x5566ad8c2ba0 2024-03-21T11:23:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.566+0000 7fba7f5ec740 10 client.25412 fetch_fsmap learned FSMap version 27 2024-03-21T11:23:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.566+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1238428608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5566ad6066b0 con 0x5566ad8c2ba0 2024-03-21T11:23:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.566+0000 7fba57400640 1 -- 172.21.15.47:0/1238428608 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba6800fba0 con 0x5566ad8c2ba0 2024-03-21T11:23:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.566+0000 7fba57400640 1 -- 172.21.15.47:0/1238428608 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fba68021030 con 0x5566ad8c2ba0 2024-03-21T11:23:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.566+0000 7fba7f5ec740 10 client.25412 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:23:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.567+0000 7fba7f5ec740 10 client.25412 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-21T11:23:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.567+0000 7fba7f5ec740 20 client.25412 populate_metadata read hostname 'smithi047' 2024-03-21T11:23:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.567+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1238428608 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5566ad8d9660 0x5566ad8d88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.567+0000 7fba7f5ec740 4 client.25412 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:23:30.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.567+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1238428608 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566ad7c9520 con 0x5566ad8d9660 2024-03-21T11:23:30.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.567+0000 7fba79400640 1 --2- 172.21.15.47:0/1238428608 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5566ad8d9660 0x5566ad8d88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.567+0000 7fba79400640 1 --2- 172.21.15.47:0/1238428608 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5566ad8d9660 0x5566ad8d88d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19957e8364f43868 server_cookie=8c886ef370390895 in_seq=0 out_seq=0 2024-03-21T11:23:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.567+0000 7fba57400640 10 client.25412 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:23:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.568+0000 7fba57400640 1 -- 172.21.15.47:0/1238428608 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5566ad7c9520 con 0x5566ad8d9660 2024-03-21T11:23:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.568+0000 7fba57400640 10 client.25412 handle_command_reply: tid=0 2024-03-21T11:23:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.568+0000 7fba7f5ec740 1 client.25412 shutdown 2024-03-21T11:23:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.568+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1238428608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c2ba0 msgr2=0x5566ad8cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.568+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1238428608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c2ba0 0x5566ad8cd0f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fba6801b000 tx=0x7fba6800faf0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.569+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1238428608 shutdown_connections 2024-03-21T11:23:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.569+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1238428608 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5566ad8d9660 0x5566ad8d88d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.569+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1238428608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 0x5566ad8d4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.569+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1238428608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c3590 0x5566ad8cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.569+0000 7fba57400640 0 client.25412 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:23:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.569+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1238428608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c2ba0 0x5566ad8cd0f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.569+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1238428608 >> 172.21.15.47:0/1238428608 conn(0x5566ad8be110 msgr2=0x5566ad8bd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.569+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1238428608 shutdown_connections 2024-03-21T11:23:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.569+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1238428608 wait complete. 2024-03-21T11:23:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.569+0000 7fba7f5ec740 20 client.25412 trim_cache size 0 max 16384 2024-03-21T11:23:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.587+0000 7fba7f5ec740 1 Processor -- start 2024-03-21T11:23:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.588+0000 7fba7f5ec740 1 -- start start 2024-03-21T11:23:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.588+0000 7fba7f5ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.588+0000 7fba7f5ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c3590 0x5566ad8d4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.588+0000 7fba7f5ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 0x5566ad8be4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.588+0000 7fba7f5ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ad8d8e80 con 0x5566ad8c2ba0 2024-03-21T11:23:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.588+0000 7fba7f5ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ad8d94e0 con 0x5566ad8c3590 2024-03-21T11:23:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.588+0000 7fba7f5ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ad8d9b40 con 0x5566ad8c8700 2024-03-21T11:23:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.588+0000 7fba79400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c3590 0x5566ad8d4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.588+0000 7fba79400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c3590 0x5566ad8d4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59692/0 (socket says 172.21.15.47:59692) 2024-03-21T11:23:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.588+0000 7fba79400640 1 -- 172.21.15.47:0/1208901450 learned_addr learned my addr 172.21.15.47:0/1208901450 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.588+0000 7fba7a800640 1 --2- 172.21.15.47:0/1208901450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 0x5566ad8be4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.588+0000 7fba79e00640 1 --2- 172.21.15.47:0/1208901450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.589+0000 7fba79400640 1 -- 172.21.15.47:0/1208901450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 msgr2=0x5566ad8be4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.589+0000 7fba79400640 1 --2- 172.21.15.47:0/1208901450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 0x5566ad8be4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.589+0000 7fba79400640 1 -- 172.21.15.47:0/1208901450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 msgr2=0x5566ad8cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.589+0000 7fba79400640 1 --2- 172.21.15.47:0/1208901450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.589+0000 7fba79400640 1 -- 172.21.15.47:0/1208901450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566ad8cc170 con 0x5566ad8c3590 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.589+0000 7fba79e00640 1 --2- 172.21.15.47:0/1208901450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.589+0000 7fba79400640 1 --2- 172.21.15.47:0/1208901450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c3590 0x5566ad8d4590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fba64006010 tx=0x7fba640eeda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79386368e52d3bf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.589+0000 7fba62600640 1 -- 172.21.15.47:0/1208901450 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640bd8d0 con 0x5566ad8c3590 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.589+0000 7fba62600640 1 -- 172.21.15.47:0/1208901450 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba640035f0 con 0x5566ad8c3590 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.589+0000 7fba62600640 1 -- 172.21.15.47:0/1208901450 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640bd460 con 0x5566ad8c3590 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.590+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1208901450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c3590 msgr2=0x5566ad8d4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.590+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1208901450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c3590 0x5566ad8d4590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fba64006010 tx=0x7fba640eeda0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.590+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1208901450 shutdown_connections 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.590+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1208901450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 0x5566ad8be4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.590+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1208901450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c3590 0x5566ad8d4590 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.590+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1208901450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.590+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1208901450 >> 172.21.15.47:0/1208901450 conn(0x5566ad8be110 msgr2=0x5566ad8c4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.590+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1208901450 shutdown_connections 2024-03-21T11:23:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.590+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1208901450 wait complete. 2024-03-21T11:23:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.590+0000 7fba7f5ec740 1 -- start start 2024-03-21T11:23:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.591+0000 7fba7f5ec740 1 Processor -- start 2024-03-21T11:23:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.591+0000 7fba7f5ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.591+0000 7fba7f5ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c3590 0x5566ad8d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.591+0000 7fba7f5ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 0x5566ad8d3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.591+0000 7fba7f5ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ad8da800 con 0x5566ad8c2ba0 2024-03-21T11:23:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.591+0000 7fba7f5ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ad8da1a0 con 0x5566ad8c3590 2024-03-21T11:23:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.591+0000 7fba7f5ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ad8dae60 con 0x5566ad8c8700 2024-03-21T11:23:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.591+0000 7fba7f5ec740 10 client.0 _ll_register_callbacks cb 0x7fba6801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:23:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.591+0000 7fba79e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.591+0000 7fba79400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c3590 0x5566ad8d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.591+0000 7fba79e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46602/0 (socket says 172.21.15.47:46602) 2024-03-21T11:23:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.591+0000 7fba7a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 0x5566ad8d3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.591+0000 7fba79e00640 1 -- 172.21.15.47:0/1321243487 learned_addr learned my addr 172.21.15.47:0/1321243487 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.591+0000 7fba79400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c3590 0x5566ad8d0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59696/0 (socket says 172.21.15.47:59696) 2024-03-21T11:23:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.592+0000 7fba79e00640 1 -- 172.21.15.47:0/1321243487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 msgr2=0x5566ad8d3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.592+0000 7fba79e00640 1 --2- 172.21.15.47:0/1321243487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 0x5566ad8d3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.592+0000 7fba79e00640 1 -- 172.21.15.47:0/1321243487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c3590 msgr2=0x5566ad8d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.592+0000 7fba79e00640 1 --2- 172.21.15.47:0/1321243487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c3590 0x5566ad8d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.592+0000 7fba79e00640 1 -- 172.21.15.47:0/1321243487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566ad8d4080 con 0x5566ad8c2ba0 2024-03-21T11:23:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.592+0000 7fba79400640 1 --2- 172.21.15.47:0/1321243487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c3590 0x5566ad8d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.592+0000 7fba79e00640 1 --2- 172.21.15.47:0/1321243487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8d0580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fba68009230 tx=0x7fba68005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a48203be16565a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.592+0000 7fba57400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:23:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.592+0000 7fba57400640 1 -- 172.21.15.47:0/1321243487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6801e040 con 0x5566ad8c2ba0 2024-03-21T11:23:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.592+0000 7fba57400640 1 -- 172.21.15.47:0/1321243487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fba68009e10 con 0x5566ad8c2ba0 2024-03-21T11:23:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.592+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1321243487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566ad8351d0 con 0x5566ad8c2ba0 2024-03-21T11:23:30.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.592+0000 7fba57400640 1 -- 172.21.15.47:0/1321243487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba6800fb60 con 0x5566ad8c2ba0 2024-03-21T11:23:30.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.593+0000 7fba7f5ec740 10 client.15282 fetch_fsmap learned FSMap version 27 2024-03-21T11:23:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.593+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1321243487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5566ad934de0 con 0x5566ad8c2ba0 2024-03-21T11:23:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.593+0000 7fba57400640 1 -- 172.21.15.47:0/1321243487 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5566ad8351d0 con 0x5566ad8c2ba0 2024-03-21T11:23:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.593+0000 7fba57400640 1 -- 172.21.15.47:0/1321243487 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fba68021040 con 0x5566ad8c2ba0 2024-03-21T11:23:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.593+0000 7fba7f5ec740 10 client.15282 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:23:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.593+0000 7fba7f5ec740 10 client.15282 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-21T11:23:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.593+0000 7fba7f5ec740 20 client.15282 populate_metadata read hostname 'smithi047' 2024-03-21T11:23:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.593+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1321243487 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5566ad939450 0x5566ad935070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.593+0000 7fba7f5ec740 4 client.15282 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T11:23:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.593+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1321243487 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5566ad82f520 con 0x5566ad939450 2024-03-21T11:23:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.593+0000 7fba79400640 1 --2- 172.21.15.47:0/1321243487 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5566ad939450 0x5566ad935070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba79400640 1 --2- 172.21.15.47:0/1321243487 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5566ad939450 0x5566ad935070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1fe5b74d45340746 server_cookie=88c0fb11c844df60 in_seq=0 out_seq=0 2024-03-21T11:23:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba57400640 10 client.15282 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:23:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba57400640 1 -- 172.21.15.47:0/1321243487 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5566ad82f520 con 0x5566ad939450 2024-03-21T11:23:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba57400640 10 client.15282 handle_command_reply: tid=0 2024-03-21T11:23:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba7f5ec740 1 client.15282 shutdown 2024-03-21T11:23:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1321243487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 msgr2=0x5566ad8d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1321243487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8d0580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fba68009230 tx=0x7fba68005f10 comp rx=0 tx=0).stop 2024-03-21T11:23:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1321243487 shutdown_connections 2024-03-21T11:23:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1321243487 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5566ad939450 0x5566ad935070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1321243487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566ad8c8700 0x5566ad8d3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1321243487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ad8c3590 0x5566ad8d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba57400640 0 client.15282 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:23:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba7f5ec740 1 --2- 172.21.15.47:0/1321243487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ad8c2ba0 0x5566ad8d0580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1321243487 >> 172.21.15.47:0/1321243487 conn(0x5566ad8be110 msgr2=0x5566ad8bca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.594+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1321243487 shutdown_connections 2024-03-21T11:23:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.595+0000 7fba7f5ec740 1 -- 172.21.15.47:0/1321243487 wait complete. 2024-03-21T11:23:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.595+0000 7fba7f5ec740 20 client.15282 trim_cache size 0 max 16384 2024-03-21T11:23:30.601 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:23:30.601 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:23:30.601 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 1, 2024-03-21T11:23:30.601 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24553, 2024-03-21T11:23:30.602 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T11:23:30.602 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T11:23:30.602 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T11:23:30.602 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 135.766482309, 2024-03-21T11:23:30.602 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 27, 2024-03-21T11:23:30.602 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T11:23:30.602 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-03-21T11:23:30.602 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 179.44334378299999 2024-03-21T11:23:30.602 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:23:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.595+0000 7fba7c800640 1 -- 172.21.15.47:0/4232364192 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fba5c0a2fa0 msgr2=0x7fba5c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.595+0000 7fba7c800640 1 --2- 172.21.15.47:0/4232364192 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fba5c0a2fa0 0x7fba5c0a5460 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5566ad8c3df0 tx=0x7fba700096a0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.595+0000 7fba7c800640 1 -- 172.21.15.47:0/4232364192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba740772a0 msgr2=0x7fba7419e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.595+0000 7fba7c800640 1 --2- 172.21.15.47:0/4232364192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba740772a0 0x7fba7419e570 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fba64036040 tx=0x7fba6402c6c0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.595+0000 7fba7c800640 1 -- 172.21.15.47:0/4232364192 shutdown_connections 2024-03-21T11:23:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.595+0000 7fba7c800640 1 --2- 172.21.15.47:0/4232364192 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fba5c0a2fa0 0x7fba5c0a5460 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.595+0000 7fba7c800640 1 --2- 172.21.15.47:0/4232364192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba74077c40 0x7fba741a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.595+0000 7fba7c800640 1 --2- 172.21.15.47:0/4232364192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba740772a0 0x7fba7419e570 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.595+0000 7fba7c800640 1 --2- 172.21.15.47:0/4232364192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba74076060 0x7fba7419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.595+0000 7fba7c800640 1 -- 172.21.15.47:0/4232364192 >> 172.21.15.47:0/4232364192 conn(0x7fba740fee50 msgr2=0x7fba74100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.596+0000 7fba7c800640 1 -- 172.21.15.47:0/4232364192 shutdown_connections 2024-03-21T11:23:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.596+0000 7fba7c800640 1 -- 172.21.15.47:0/4232364192 wait complete. 2024-03-21T11:23:30.614 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-21T11:23:30.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.840+0000 7f44a3200640 1 -- 172.21.15.47:0/3029114513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a4075680 msgr2=0x7f44a4073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.840+0000 7f44a3200640 1 --2- 172.21.15.47:0/3029114513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a4075680 0x7f44a4073ad0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f449c009ca0 tx=0x7f449c02c180 comp rx=0 tx=0).stop 2024-03-21T11:23:30.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.840+0000 7f44a3200640 1 -- 172.21.15.47:0/3029114513 shutdown_connections 2024-03-21T11:23:30.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.840+0000 7f44a3200640 1 --2- 172.21.15.47:0/3029114513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a4109430 0x7f44a410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.841+0000 7f44a3200640 1 --2- 172.21.15.47:0/3029114513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44a40740a0 0x7f44a4074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.841+0000 7f44a3200640 1 --2- 172.21.15.47:0/3029114513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a4075680 0x7f44a4073ad0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.841+0000 7f44a3200640 1 -- 172.21.15.47:0/3029114513 >> 172.21.15.47:0/3029114513 conn(0x7f44a40fcb10 msgr2=0x7f44a40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.841+0000 7f44a3200640 1 -- 172.21.15.47:0/3029114513 shutdown_connections 2024-03-21T11:23:30.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.841+0000 7f44a3200640 1 -- 172.21.15.47:0/3029114513 wait complete. 2024-03-21T11:23:30.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.842+0000 7f44a3200640 1 Processor -- start 2024-03-21T11:23:30.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.842+0000 7f44a3200640 1 -- start start 2024-03-21T11:23:30.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.842+0000 7f44a3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44a40740a0 0x7f44a4199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.842+0000 7f44a3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a4075680 0x7f44a4199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.842+0000 7f44a3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a4109430 0x7f44a41a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.842+0000 7f44a3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a410f410 con 0x7f44a4109430 2024-03-21T11:23:30.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.842+0000 7f44a3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a410f290 con 0x7f44a40740a0 2024-03-21T11:23:30.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.843+0000 7f44a3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a410f590 con 0x7f44a4075680 2024-03-21T11:23:30.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.843+0000 7f44a1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44a40740a0 0x7f44a4199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.843+0000 7f44a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a4075680 0x7f44a4199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.843+0000 7f44a1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44a40740a0 0x7f44a4199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59714/0 (socket says 172.21.15.47:59714) 2024-03-21T11:23:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.843+0000 7f44a1e00640 1 -- 172.21.15.47:0/4257475797 learned_addr learned my addr 172.21.15.47:0/4257475797 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.843+0000 7f44a2800640 1 --2- 172.21.15.47:0/4257475797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a4109430 0x7f44a41a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.843+0000 7f44a1e00640 1 -- 172.21.15.47:0/4257475797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a4075680 msgr2=0x7f44a4199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.843+0000 7f44a1e00640 1 --2- 172.21.15.47:0/4257475797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a4075680 0x7f44a4199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.843+0000 7f44a1e00640 1 -- 172.21.15.47:0/4257475797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a4109430 msgr2=0x7f44a41a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.843+0000 7f44a1e00640 1 --2- 172.21.15.47:0/4257475797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a4109430 0x7f44a41a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.843+0000 7f44a1e00640 1 -- 172.21.15.47:0/4257475797 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a41a16c0 con 0x7f44a40740a0 2024-03-21T11:23:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.843+0000 7f44a2800640 1 --2- 172.21.15.47:0/4257475797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a4109430 0x7f44a41a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.844+0000 7f44a1e00640 1 --2- 172.21.15.47:0/4257475797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44a40740a0 0x7f44a4199aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f449c009dc0 tx=0x7f449c0093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.844+0000 7f449aa00640 1 -- 172.21.15.47:0/4257475797 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c039040 con 0x7f44a40740a0 2024-03-21T11:23:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.844+0000 7f449aa00640 1 -- 172.21.15.47:0/4257475797 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f449c044070 con 0x7f44a40740a0 2024-03-21T11:23:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.844+0000 7f449aa00640 1 -- 172.21.15.47:0/4257475797 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c004460 con 0x7f44a40740a0 2024-03-21T11:23:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.844+0000 7f44a3200640 1 -- 172.21.15.47:0/4257475797 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a41a1950 con 0x7f44a40740a0 2024-03-21T11:23:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.844+0000 7f44a3200640 1 -- 172.21.15.47:0/4257475797 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a41a1df0 con 0x7f44a40740a0 2024-03-21T11:23:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.847+0000 7f44abfdd740 1 Processor -- start 2024-03-21T11:23:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.847+0000 7f449aa00640 1 -- 172.21.15.47:0/4257475797 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f449c004600 con 0x7f44a40740a0 2024-03-21T11:23:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.847+0000 7f44abfdd740 1 -- start start 2024-03-21T11:23:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.848+0000 7f44abfdd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39ded80 0x5579b39df1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.848+0000 7f44abfdd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39df770 0x5579b39e4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.848+0000 7f44abfdd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39e48e0 0x5579b39e6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.848+0000 7f44abfdd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b3943c90 con 0x5579b39df770 2024-03-21T11:23:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.848+0000 7f44abfdd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b38ec980 con 0x5579b39e48e0 2024-03-21T11:23:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.848+0000 7f44abfdd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b393fd30 con 0x5579b39ded80 2024-03-21T11:23:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.848+0000 7f44a2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39ded80 0x5579b39df1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.848+0000 7f44a1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39e48e0 0x5579b39e6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.848+0000 7f44a2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39ded80 0x5579b39df1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53424/0 (socket says 172.21.15.47:53424) 2024-03-21T11:23:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.848+0000 7f44a1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39e48e0 0x5579b39e6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59722/0 (socket says 172.21.15.47:59722) 2024-03-21T11:23:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.848+0000 7f44a2800640 1 -- 172.21.15.47:0/3566435510 learned_addr learned my addr 172.21.15.47:0/3566435510 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.848+0000 7f44a1e00640 1 --2- 172.21.15.47:0/3566435510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39df770 0x5579b39e4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.849+0000 7f44a2800640 1 -- 172.21.15.47:0/3566435510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39e48e0 msgr2=0x5579b39e6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.849+0000 7f44a2800640 1 --2- 172.21.15.47:0/3566435510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39e48e0 0x5579b39e6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.849+0000 7f44a2800640 1 -- 172.21.15.47:0/3566435510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39df770 msgr2=0x5579b39e4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.849+0000 7f44a2800640 1 --2- 172.21.15.47:0/3566435510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39df770 0x5579b39e4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.849+0000 7f44a2800640 1 -- 172.21.15.47:0/3566435510 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579b3904510 con 0x5579b39ded80 2024-03-21T11:23:30.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.849+0000 7f44a1400640 1 --2- 172.21.15.47:0/3566435510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39e48e0 0x5579b39e6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:30.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.849+0000 7f449aa00640 1 --2- 172.21.15.47:0/4257475797 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44880a2fa0 0x7f44880a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.849+0000 7f44a2800640 1 --2- 172.21.15.47:0/3566435510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39ded80 0x5579b39df1a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4494009cc0 tx=0x7f44940117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ad5d42e0d653940 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.849+0000 7f449aa00640 1 -- 172.21.15.47:0/4257475797 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f449c0f1cf0 con 0x7f44a40740a0 2024-03-21T11:23:30.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.849+0000 7f4493400640 1 -- 172.21.15.47:0/3566435510 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494012450 con 0x5579b39ded80 2024-03-21T11:23:30.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.849+0000 7f4493400640 1 -- 172.21.15.47:0/3566435510 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4494012ad0 con 0x5579b39ded80 2024-03-21T11:23:30.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.849+0000 7f44a1e00640 1 --2- 172.21.15.47:0/4257475797 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44880a2fa0 0x7f44880a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.849+0000 7f4493400640 1 -- 172.21.15.47:0/3566435510 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449400fc50 con 0x5579b39ded80 2024-03-21T11:23:30.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.850+0000 7f44abfdd740 1 -- 172.21.15.47:0/3566435510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39ded80 msgr2=0x5579b39df1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.850+0000 7f44abfdd740 1 --2- 172.21.15.47:0/3566435510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39ded80 0x5579b39df1a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4494009cc0 tx=0x7f44940117b0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.850+0000 7f44a1e00640 1 --2- 172.21.15.47:0/4257475797 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44880a2fa0 0x7f44880a5460 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f449c0ba490 tx=0x7f449c009a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.850+0000 7f44abfdd740 1 -- 172.21.15.47:0/3566435510 shutdown_connections 2024-03-21T11:23:30.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.850+0000 7f44abfdd740 1 --2- 172.21.15.47:0/3566435510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39e48e0 0x5579b39e6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.850+0000 7f44abfdd740 1 --2- 172.21.15.47:0/3566435510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39df770 0x5579b39e4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.850+0000 7f44abfdd740 1 --2- 172.21.15.47:0/3566435510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39ded80 0x5579b39df1a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.850+0000 7f44abfdd740 1 -- 172.21.15.47:0/3566435510 >> 172.21.15.47:0/3566435510 conn(0x5579b39da2f0 msgr2=0x5579b39da6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.850+0000 7f44abfdd740 1 -- 172.21.15.47:0/3566435510 shutdown_connections 2024-03-21T11:23:30.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.850+0000 7f44abfdd740 1 -- 172.21.15.47:0/3566435510 wait complete. 2024-03-21T11:23:30.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.851+0000 7f44abfdd740 1 -- start start 2024-03-21T11:23:30.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.851+0000 7f44abfdd740 1 Processor -- start 2024-03-21T11:23:30.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.851+0000 7f44abfdd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39ded80 0x5579b39e9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.851+0000 7f44abfdd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39df770 0x5579b39e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.851+0000 7f44abfdd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 0x5579b39f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.851+0000 7f44abfdd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b39e78e0 con 0x5579b39e48e0 2024-03-21T11:23:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.851+0000 7f44abfdd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b38ffdd0 con 0x5579b39df770 2024-03-21T11:23:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.851+0000 7f44abfdd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b39e7a60 con 0x5579b39ded80 2024-03-21T11:23:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.851+0000 7f44abfdd740 10 client.0 _ll_register_callbacks cb 0x5579b3657ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:23:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.852+0000 7f44a2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39ded80 0x5579b39e9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.852+0000 7f44a1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39df770 0x5579b39e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.852+0000 7f44a2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39ded80 0x5579b39e9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53436/0 (socket says 172.21.15.47:53436) 2024-03-21T11:23:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.852+0000 7f44a1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39df770 0x5579b39e97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59730/0 (socket says 172.21.15.47:59730) 2024-03-21T11:23:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.852+0000 7f44a2800640 1 -- 172.21.15.47:0/2908856118 learned_addr learned my addr 172.21.15.47:0/2908856118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.852+0000 7f44a2800640 1 --2- 172.21.15.47:0/2908856118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 0x5579b39f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.852+0000 7f44a1400640 1 -- 172.21.15.47:0/2908856118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39ded80 msgr2=0x5579b39e9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.852+0000 7f44a1400640 1 --2- 172.21.15.47:0/2908856118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39ded80 0x5579b39e9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.852+0000 7f44a1400640 1 -- 172.21.15.47:0/2908856118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 msgr2=0x5579b39f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.852+0000 7f44a1400640 1 --2- 172.21.15.47:0/2908856118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 0x5579b39f0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.852+0000 7f44a1400640 1 -- 172.21.15.47:0/2908856118 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579b36582d0 con 0x5579b39df770 2024-03-21T11:23:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.852+0000 7f44a2800640 1 --2- 172.21.15.47:0/2908856118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 0x5579b39f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.852+0000 7f44a1400640 1 --2- 172.21.15.47:0/2908856118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39df770 0x5579b39e97d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f44a419afb0 tx=0x7f448c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ff83574411a739f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.852+0000 7f4487e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:23:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.853+0000 7f4487e00640 1 -- 172.21.15.47:0/2908856118 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c004030 con 0x5579b39df770 2024-03-21T11:23:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.853+0000 7f4487e00640 1 -- 172.21.15.47:0/2908856118 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f448c0041d0 con 0x5579b39df770 2024-03-21T11:23:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.853+0000 7f44abfdd740 1 -- 172.21.15.47:0/2908856118 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4494012ad0 con 0x5579b39df770 2024-03-21T11:23:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.853+0000 7f4487e00640 1 -- 172.21.15.47:0/2908856118 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f448c025050 con 0x5579b39df770 2024-03-21T11:23:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.853+0000 7f44abfdd740 10 client.25430 fetch_fsmap learned FSMap version 27 2024-03-21T11:23:30.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.853+0000 7f44abfdd740 1 -- 172.21.15.47:0/2908856118 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5579b37226b0 con 0x5579b39df770 2024-03-21T11:23:30.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.853+0000 7f4487e00640 1 -- 172.21.15.47:0/2908856118 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4494012ad0 con 0x5579b39df770 2024-03-21T11:23:30.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.853+0000 7f4487e00640 1 -- 172.21.15.47:0/2908856118 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f448c003a20 con 0x5579b39df770 2024-03-21T11:23:30.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.853+0000 7f44abfdd740 10 client.25430 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:23:30.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.854+0000 7f44abfdd740 10 client.25430 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T11:23:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.854+0000 7f44abfdd740 20 client.25430 populate_metadata read hostname 'smithi047' 2024-03-21T11:23:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.854+0000 7f44abfdd740 1 --2- 172.21.15.47:0/2908856118 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579b39f5a50 0x5579b39f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.854+0000 7f44abfdd740 4 client.25430 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:23:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.854+0000 7f44abfdd740 1 -- 172.21.15.47:0/2908856118 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579b38e5520 con 0x5579b39f5a50 2024-03-21T11:23:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.854+0000 7f44a2800640 1 --2- 172.21.15.47:0/2908856118 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579b39f5a50 0x5579b39f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.854+0000 7f44a2800640 1 --2- 172.21.15.47:0/2908856118 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579b39f5a50 0x5579b39f4b50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf4f23636bff8a66 server_cookie=4964e25ed7011cb2 in_seq=0 out_seq=0 2024-03-21T11:23:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.854+0000 7f4487e00640 10 client.25430 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:23:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.855+0000 7f4487e00640 1 -- 172.21.15.47:0/2908856118 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5579b38e5520 con 0x5579b39f5a50 2024-03-21T11:23:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.855+0000 7f4487e00640 10 client.25430 handle_command_reply: tid=0 2024-03-21T11:23:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.855+0000 7f44abfdd740 1 client.25430 shutdown 2024-03-21T11:23:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.855+0000 7f44abfdd740 1 -- 172.21.15.47:0/2908856118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39df770 msgr2=0x5579b39e97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.855+0000 7f44abfdd740 1 --2- 172.21.15.47:0/2908856118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39df770 0x5579b39e97d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f44a419afb0 tx=0x7f448c012b60 comp rx=0 tx=0).stop 2024-03-21T11:23:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.855+0000 7f44abfdd740 1 -- 172.21.15.47:0/2908856118 shutdown_connections 2024-03-21T11:23:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.855+0000 7f44abfdd740 1 --2- 172.21.15.47:0/2908856118 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579b39f5a50 0x5579b39f4b50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.855+0000 7f44abfdd740 1 --2- 172.21.15.47:0/2908856118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 0x5579b39f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.855+0000 7f44abfdd740 1 --2- 172.21.15.47:0/2908856118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39df770 0x5579b39e97d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.855+0000 7f44abfdd740 1 --2- 172.21.15.47:0/2908856118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39ded80 0x5579b39e9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.855+0000 7f44abfdd740 1 -- 172.21.15.47:0/2908856118 >> 172.21.15.47:0/2908856118 conn(0x5579b39da2f0 msgr2=0x5579b39d94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.855+0000 7f4487e00640 0 client.25430 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:23:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.856+0000 7f44abfdd740 1 -- 172.21.15.47:0/2908856118 shutdown_connections 2024-03-21T11:23:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.856+0000 7f44abfdd740 1 -- 172.21.15.47:0/2908856118 wait complete. 2024-03-21T11:23:30.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.856+0000 7f44abfdd740 20 client.25430 trim_cache size 0 max 16384 2024-03-21T11:23:30.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.875+0000 7f44abfdd740 1 Processor -- start 2024-03-21T11:23:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.875+0000 7f44abfdd740 1 -- start start 2024-03-21T11:23:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.875+0000 7f44abfdd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39ded80 0x5579b39f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.875+0000 7f44abfdd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39df770 0x5579b39f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.875+0000 7f44abfdd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 0x5579b39ddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.875+0000 7f44abfdd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b39f5270 con 0x5579b39e48e0 2024-03-21T11:23:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.875+0000 7f44abfdd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b39f58d0 con 0x5579b39ded80 2024-03-21T11:23:30.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.875+0000 7f44abfdd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b39f5f30 con 0x5579b39df770 2024-03-21T11:23:30.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.875+0000 7f44a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39df770 0x5579b39f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.876+0000 7f44a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39df770 0x5579b39f07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53446/0 (socket says 172.21.15.47:53446) 2024-03-21T11:23:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.876+0000 7f44a1400640 1 -- 172.21.15.47:0/1512537967 learned_addr learned my addr 172.21.15.47:0/1512537967 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.876+0000 7f44a2800640 1 --2- 172.21.15.47:0/1512537967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39ded80 0x5579b39f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.876+0000 7f44a2800640 1 --2- 172.21.15.47:0/1512537967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 0x5579b39ddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.876+0000 7f44a1400640 1 -- 172.21.15.47:0/1512537967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39ded80 msgr2=0x5579b39f4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.876+0000 7f44a1400640 1 --2- 172.21.15.47:0/1512537967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39ded80 0x5579b39f4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.876+0000 7f44a1400640 1 -- 172.21.15.47:0/1512537967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 msgr2=0x5579b39ddde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.876+0000 7f44a1400640 1 --2- 172.21.15.47:0/1512537967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 0x5579b39ddde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.876+0000 7f44a1400640 1 -- 172.21.15.47:0/1512537967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579b39d9e90 con 0x5579b39df770 2024-03-21T11:23:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.876+0000 7f44a1400640 1 --2- 172.21.15.47:0/1512537967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39df770 0x5579b39f07d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f448c003e90 tx=0x7f448c009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b2d675fb7066fb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.876+0000 7f4492a00640 1 -- 172.21.15.47:0/1512537967 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c02e070 con 0x5579b39df770 2024-03-21T11:23:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.877+0000 7f4492a00640 1 -- 172.21.15.47:0/1512537967 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f448c00fe00 con 0x5579b39df770 2024-03-21T11:23:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.877+0000 7f4492a00640 1 -- 172.21.15.47:0/1512537967 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c022040 con 0x5579b39df770 2024-03-21T11:23:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.877+0000 7f44abfdd740 1 -- 172.21.15.47:0/1512537967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39df770 msgr2=0x5579b39f07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.877+0000 7f44abfdd740 1 --2- 172.21.15.47:0/1512537967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39df770 0x5579b39f07d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f448c003e90 tx=0x7f448c009830 comp rx=0 tx=0).stop 2024-03-21T11:23:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.877+0000 7f44abfdd740 1 -- 172.21.15.47:0/1512537967 shutdown_connections 2024-03-21T11:23:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.877+0000 7f44abfdd740 1 --2- 172.21.15.47:0/1512537967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 0x5579b39ddde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.877+0000 7f44abfdd740 1 --2- 172.21.15.47:0/1512537967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39df770 0x5579b39f07d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.877+0000 7f44abfdd740 1 --2- 172.21.15.47:0/1512537967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39ded80 0x5579b39f4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.877+0000 7f44abfdd740 1 -- 172.21.15.47:0/1512537967 >> 172.21.15.47:0/1512537967 conn(0x5579b39da2f0 msgr2=0x5579b39e7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.877+0000 7f44abfdd740 1 -- 172.21.15.47:0/1512537967 shutdown_connections 2024-03-21T11:23:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.877+0000 7f44abfdd740 1 -- 172.21.15.47:0/1512537967 wait complete. 2024-03-21T11:23:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.878+0000 7f44abfdd740 1 -- start start 2024-03-21T11:23:30.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.878+0000 7f44abfdd740 1 Processor -- start 2024-03-21T11:23:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.878+0000 7f44abfdd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39ded80 0x5579b39f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.878+0000 7f44abfdd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39df770 0x5579b39ecf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44abfdd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 0x5579b39ed490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44abfdd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b39f6bf0 con 0x5579b39e48e0 2024-03-21T11:23:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44abfdd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b39f6590 con 0x5579b39ded80 2024-03-21T11:23:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44abfdd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579b39f7250 con 0x5579b39df770 2024-03-21T11:23:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44abfdd740 10 client.0 _ll_register_callbacks cb 0x7f448c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:23:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39df770 0x5579b39ecf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39df770 0x5579b39ecf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53456/0 (socket says 172.21.15.47:53456) 2024-03-21T11:23:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44a1400640 1 -- 172.21.15.47:0/3335419435 learned_addr learned my addr 172.21.15.47:0/3335419435 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44a2800640 1 --2- 172.21.15.47:0/3335419435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39ded80 0x5579b39f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44a2800640 1 --2- 172.21.15.47:0/3335419435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 0x5579b39ed490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44a1400640 1 -- 172.21.15.47:0/3335419435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39ded80 msgr2=0x5579b39f04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44a1400640 1 --2- 172.21.15.47:0/3335419435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39ded80 0x5579b39f04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44a1400640 1 -- 172.21.15.47:0/3335419435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 msgr2=0x5579b39ed490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44a1400640 1 --2- 172.21.15.47:0/3335419435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 0x5579b39ed490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44a1400640 1 -- 172.21.15.47:0/3335419435 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579b39edad0 con 0x5579b39df770 2024-03-21T11:23:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44a2800640 1 --2- 172.21.15.47:0/3335419435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 0x5579b39ed490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f44a1400640 1 --2- 172.21.15.47:0/3335419435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39df770 0x5579b39ecf50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f448c0106a0 tx=0x7f448c003800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9382ee9aa41c6c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.879+0000 7f4487e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:23:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.880+0000 7f4487e00640 1 -- 172.21.15.47:0/3335419435 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c02e070 con 0x5579b39df770 2024-03-21T11:23:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.880+0000 7f44abfdd740 1 -- 172.21.15.47:0/3335419435 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f448c00fe00 con 0x5579b39df770 2024-03-21T11:23:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.880+0000 7f4487e00640 1 -- 172.21.15.47:0/3335419435 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f448c00fb30 con 0x5579b39df770 2024-03-21T11:23:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.880+0000 7f4487e00640 1 -- 172.21.15.47:0/3335419435 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f448c03a050 con 0x5579b39df770 2024-03-21T11:23:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.880+0000 7f44abfdd740 10 client.25450 fetch_fsmap learned FSMap version 27 2024-03-21T11:23:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.880+0000 7f44abfdd740 1 -- 172.21.15.47:0/3335419435 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5579b3a51b40 con 0x5579b39df770 2024-03-21T11:23:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.880+0000 7f4487e00640 1 -- 172.21.15.47:0/3335419435 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f448c00fe00 con 0x5579b39df770 2024-03-21T11:23:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.880+0000 7f4487e00640 1 -- 172.21.15.47:0/3335419435 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 27) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f448c010b80 con 0x5579b39df770 2024-03-21T11:23:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.880+0000 7f44abfdd740 10 client.25450 fetch_fsmap finished waiting for FSMap version 27 2024-03-21T11:23:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.880+0000 7f44abfdd740 10 client.25450 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T11:23:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.880+0000 7f44abfdd740 20 client.25450 populate_metadata read hostname 'smithi047' 2024-03-21T11:23:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.880+0000 7f44abfdd740 1 --2- 172.21.15.47:0/3335419435 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579b3a55d90 0x5579b3a51d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.880+0000 7f44abfdd740 4 client.25450 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T11:23:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.880+0000 7f44abfdd740 1 -- 172.21.15.47:0/3335419435 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5579b39511d0 con 0x5579b3a55d90 2024-03-21T11:23:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.881+0000 7f44a2800640 1 --2- 172.21.15.47:0/3335419435 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579b3a55d90 0x5579b3a51d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.881+0000 7f44a2800640 1 --2- 172.21.15.47:0/3335419435 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579b3a55d90 0x5579b3a51d10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42378f38125a58b5 server_cookie=390d34eb3fe68c65 in_seq=0 out_seq=0 2024-03-21T11:23:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.881+0000 7f4487e00640 10 client.25450 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:23:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.881+0000 7f4487e00640 1 -- 172.21.15.47:0/3335419435 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5579b39511d0 con 0x5579b3a55d90 2024-03-21T11:23:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.881+0000 7f4487e00640 10 client.25450 handle_command_reply: tid=0 2024-03-21T11:23:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.881+0000 7f44abfdd740 1 client.25450 shutdown 2024-03-21T11:23:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.882+0000 7f44abfdd740 1 -- 172.21.15.47:0/3335419435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39df770 msgr2=0x5579b39ecf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.882+0000 7f44abfdd740 1 --2- 172.21.15.47:0/3335419435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39df770 0x5579b39ecf50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f448c0106a0 tx=0x7f448c003800 comp rx=0 tx=0).stop 2024-03-21T11:23:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.882+0000 7f44abfdd740 1 -- 172.21.15.47:0/3335419435 shutdown_connections 2024-03-21T11:23:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.882+0000 7f44abfdd740 1 --2- 172.21.15.47:0/3335419435 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579b3a55d90 0x5579b3a51d10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.882+0000 7f44abfdd740 1 --2- 172.21.15.47:0/3335419435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579b39e48e0 0x5579b39ed490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.882+0000 7f44abfdd740 1 --2- 172.21.15.47:0/3335419435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579b39df770 0x5579b39ecf50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.882+0000 7f4487e00640 0 client.25450 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:23:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.882+0000 7f44abfdd740 1 --2- 172.21.15.47:0/3335419435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579b39ded80 0x5579b39f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.882+0000 7f44abfdd740 1 -- 172.21.15.47:0/3335419435 >> 172.21.15.47:0/3335419435 conn(0x5579b39da2f0 msgr2=0x5579b39f4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.882+0000 7f44abfdd740 1 -- 172.21.15.47:0/3335419435 shutdown_connections 2024-03-21T11:23:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.882+0000 7f44abfdd740 1 -- 172.21.15.47:0/3335419435 wait complete. 2024-03-21T11:23:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.882+0000 7f44abfdd740 20 client.25450 trim_cache size 0 max 16384 2024-03-21T11:23:30.888 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:23:30.888 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "a6f35682-e773-11ee-95cd-87774f69a715", 2024-03-21T11:23:30.888 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-21T11:23:30.888 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24581, 2024-03-21T11:23:30.888 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T11:23:30.888 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T11:23:30.888 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T11:23:30.888 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 161.291633577, 2024-03-21T11:23:30.888 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 27, 2024-03-21T11:23:30.888 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T11:23:30.889 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-03-21T11:23:30.889 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 168.66470693799999 2024-03-21T11:23:30.889 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:23:30.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.883+0000 7f44a3200640 1 -- 172.21.15.47:0/4257475797 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44880a2fa0 msgr2=0x7f44880a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.883+0000 7f44a3200640 1 --2- 172.21.15.47:0/4257475797 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44880a2fa0 0x7f44880a5460 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f449c0ba490 tx=0x7f449c009a20 comp rx=0 tx=0).stop 2024-03-21T11:23:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.883+0000 7f44a3200640 1 -- 172.21.15.47:0/4257475797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44a40740a0 msgr2=0x7f44a4199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.883+0000 7f44a3200640 1 --2- 172.21.15.47:0/4257475797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44a40740a0 0x7f44a4199aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f449c009dc0 tx=0x7f449c0093e0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.883+0000 7f44a3200640 1 -- 172.21.15.47:0/4257475797 shutdown_connections 2024-03-21T11:23:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.883+0000 7f44a3200640 1 --2- 172.21.15.47:0/4257475797 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44880a2fa0 0x7f44880a5460 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.883+0000 7f44a3200640 1 --2- 172.21.15.47:0/4257475797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a4109430 0x7f44a41a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.883+0000 7f44a3200640 1 --2- 172.21.15.47:0/4257475797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a4075680 0x7f44a4199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.883+0000 7f44a3200640 1 --2- 172.21.15.47:0/4257475797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44a40740a0 0x7f44a4199aa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.883+0000 7f44a3200640 1 -- 172.21.15.47:0/4257475797 >> 172.21.15.47:0/4257475797 conn(0x7f44a40fcb10 msgr2=0x7f44a41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.883+0000 7f44a3200640 1 -- 172.21.15.47:0/4257475797 shutdown_connections 2024-03-21T11:23:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:30.883+0000 7f44a3200640 1 -- 172.21.15.47:0/4257475797 wait complete. 2024-03-21T11:23:30.900 INFO:teuthology.run_tasks:Running task kclient... 2024-03-21T11:23:30.914 INFO:tasks.kclient:Mounting kernel clients... 2024-03-21T11:23:30.914 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-21T11:23:30.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:30.915 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-21T11:23:30.971 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-21T11:23:30.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:30.973 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T11:23:31.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:31.076 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-03-21T11:23:31.207 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-03-21T11:23:31.207 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-21T11:23:31.207 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-21T11:23:31.207 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-21T11:23:31.673 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:23:32.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:32 smithi100 ceph-mon[36245]: pgmap v406: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:23:32.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:32 smithi102 ceph-mon[36748]: pgmap v406: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:23:32.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:32 smithi047 ceph-mon[31351]: pgmap v406: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:23:33.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.954+0000 7ff4c9e00700 1 -- 172.21.15.47:0/2113087286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4c4104da0 msgr2=0x7ff4c41112b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:33.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.954+0000 7ff4c9e00700 1 --2- 172.21.15.47:0/2113087286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4c4104da0 0x7ff4c41112b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00b0a0 tx=0x7ff4bc00b3b0 comp rx=0 tx=0).stop 2024-03-21T11:23:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.954+0000 7ff4c9e00700 1 -- 172.21.15.47:0/2113087286 shutdown_connections 2024-03-21T11:23:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.954+0000 7ff4c9e00700 1 --2- 172.21.15.47:0/2113087286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4c4104da0 0x7ff4c41112b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.954+0000 7ff4c9e00700 1 --2- 172.21.15.47:0/2113087286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c410a470 0x7ff4c410a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.954+0000 7ff4c9e00700 1 --2- 172.21.15.47:0/2113087286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c41043e0 0x7ff4c4104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.954+0000 7ff4c9e00700 1 -- 172.21.15.47:0/2113087286 >> 172.21.15.47:0/2113087286 conn(0x7ff4c40fff30 msgr2=0x7ff4c4102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.954+0000 7ff4c9e00700 1 -- 172.21.15.47:0/2113087286 shutdown_connections 2024-03-21T11:23:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.954+0000 7ff4c9e00700 1 -- 172.21.15.47:0/2113087286 wait complete. 2024-03-21T11:23:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.956+0000 7ff4c9e00700 1 Processor -- start 2024-03-21T11:23:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.956+0000 7ff4c9e00700 1 -- start start 2024-03-21T11:23:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.957+0000 7ff4c9e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c41043e0 0x7ff4c41a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.957+0000 7ff4c9e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4c4104da0 0x7ff4c41a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.957+0000 7ff4c9e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c410a470 0x7ff4c419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.957+0000 7ff4c9e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c4114720 con 0x7ff4c41043e0 2024-03-21T11:23:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.957+0000 7ff4c9e00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c41145a0 con 0x7ff4c410a470 2024-03-21T11:23:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.957+0000 7ff4c9e00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c41148a0 con 0x7ff4c4104da0 2024-03-21T11:23:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.957+0000 7ff4c2a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c41043e0 0x7ff4c41a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.957+0000 7ff4c2000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4c4104da0 0x7ff4c41a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.957+0000 7ff4c2a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c41043e0 0x7ff4c41a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46688/0 (socket says 172.21.15.47:46688) 2024-03-21T11:23:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.957+0000 7ff4c2000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4c4104da0 0x7ff4c41a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53466/0 (socket says 172.21.15.47:53466) 2024-03-21T11:23:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.957+0000 7ff4c2a00700 1 -- 172.21.15.47:0/17620333 learned_addr learned my addr 172.21.15.47:0/17620333 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:33.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.957+0000 7ff4c3400700 1 --2- 172.21.15.47:0/17620333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c410a470 0x7ff4c419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:33.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.958+0000 7ff4c2000700 1 -- 172.21.15.47:0/17620333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c410a470 msgr2=0x7ff4c419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:33.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.958+0000 7ff4c2000700 1 --2- 172.21.15.47:0/17620333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c410a470 0x7ff4c419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.958+0000 7ff4c2000700 1 -- 172.21.15.47:0/17620333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c41043e0 msgr2=0x7ff4c41a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.958+0000 7ff4c2000700 1 --2- 172.21.15.47:0/17620333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c41043e0 0x7ff4c41a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.958+0000 7ff4c2000700 1 -- 172.21.15.47:0/17620333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c419b100 con 0x7ff4c4104da0 2024-03-21T11:23:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.958+0000 7ff4c3400700 1 --2- 172.21.15.47:0/17620333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c410a470 0x7ff4c419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.958+0000 7ff4c2000700 1 --2- 172.21.15.47:0/17620333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4c4104da0 0x7ff4c41a0d20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000e9f0 tx=0x7ff4b000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.958+0000 7ff4c8a00700 1 -- 172.21.15.47:0/17620333 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b000cb70 con 0x7ff4c4104da0 2024-03-21T11:23:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.958+0000 7ff4c8a00700 1 -- 172.21.15.47:0/17620333 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4b000ee90 con 0x7ff4c4104da0 2024-03-21T11:23:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.958+0000 7ff4c8a00700 1 -- 172.21.15.47:0/17620333 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0003dc0 con 0x7ff4c4104da0 2024-03-21T11:23:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.958+0000 7ff4c9e00700 1 -- 172.21.15.47:0/17620333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c419b3f0 con 0x7ff4c4104da0 2024-03-21T11:23:33.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.959+0000 7ff4c9e00700 1 -- 172.21.15.47:0/17620333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c41a76b0 con 0x7ff4c4104da0 2024-03-21T11:23:33.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.960+0000 7ff4c9e00700 1 -- 172.21.15.47:0/17620333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c4068e50 con 0x7ff4c4104da0 2024-03-21T11:23:33.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.962+0000 7ff4c8a00700 1 -- 172.21.15.47:0/17620333 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff4b0003fa0 con 0x7ff4c4104da0 2024-03-21T11:23:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.963+0000 7ff4c8a00700 1 --2- 172.21.15.47:0/17620333 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff4a80a3030 0x7ff4a80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.963+0000 7ff4c8a00700 1 -- 172.21.15.47:0/17620333 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff4b00cf890 con 0x7ff4c4104da0 2024-03-21T11:23:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.963+0000 7ff4c2a00700 1 --2- 172.21.15.47:0/17620333 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff4a80a3030 0x7ff4a80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.964+0000 7ff4c2a00700 1 --2- 172.21.15.47:0/17620333 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff4a80a3030 0x7ff4a80a54f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff4c419bae0 tx=0x7ff4b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:33.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:33.967+0000 7ff4c8a00700 1 -- 172.21.15.47:0/17620333 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff4b009c2d0 con 0x7ff4c4104da0 2024-03-21T11:23:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:34.248+0000 7ff4c9e00700 1 -- 172.21.15.47:0/17620333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff4c419bf00 con 0x7ff4c4104da0 2024-03-21T11:23:34.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:34.249+0000 7ff4c8a00700 1 -- 172.21.15.47:0/17620333 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff4c419bf00 con 0x7ff4c4104da0 2024-03-21T11:23:34.250 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-21T11:23:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:34.252+0000 7ff4c9e00700 1 -- 172.21.15.47:0/17620333 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff4a80a3030 msgr2=0x7ff4a80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:34.252+0000 7ff4c9e00700 1 --2- 172.21.15.47:0/17620333 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff4a80a3030 0x7ff4a80a54f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff4c419bae0 tx=0x7ff4b4009260 comp rx=0 tx=0).stop 2024-03-21T11:23:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:34.252+0000 7ff4c9e00700 1 -- 172.21.15.47:0/17620333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4c4104da0 msgr2=0x7ff4c41a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:34.252+0000 7ff4c9e00700 1 --2- 172.21.15.47:0/17620333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4c4104da0 0x7ff4c41a0d20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000e9f0 tx=0x7ff4b000ed00 comp rx=0 tx=0).stop 2024-03-21T11:23:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:34.252+0000 7ff4c9e00700 1 -- 172.21.15.47:0/17620333 shutdown_connections 2024-03-21T11:23:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:34.252+0000 7ff4c9e00700 1 --2- 172.21.15.47:0/17620333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4c410a470 0x7ff4c419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:34.252+0000 7ff4c9e00700 1 --2- 172.21.15.47:0/17620333 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff4a80a3030 0x7ff4a80a54f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:34.252+0000 7ff4c9e00700 1 --2- 172.21.15.47:0/17620333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4c41043e0 0x7ff4c41a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:34.252+0000 7ff4c9e00700 1 --2- 172.21.15.47:0/17620333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4c4104da0 0x7ff4c41a0d20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:34.252+0000 7ff4c9e00700 1 -- 172.21.15.47:0/17620333 >> 172.21.15.47:0/17620333 conn(0x7ff4c40fff30 msgr2=0x7ff4c4100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:34.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:34.252+0000 7ff4c9e00700 1 -- 172.21.15.47:0/17620333 shutdown_connections 2024-03-21T11:23:34.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:34.252+0000 7ff4c9e00700 1 -- 172.21.15.47:0/17620333 wait complete. 2024-03-21T11:23:34.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:34 smithi100 ceph-mon[36245]: pgmap v407: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:23:34.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:34 smithi102 ceph-mon[36748]: pgmap v407: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:23:34.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:34 smithi047 ceph-mon[31351]: pgmap v407: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:23:35.095 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-21T11:23:35.095 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-21T11:23:35.095 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-21T11:23:35.095 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-21T11:23:35.095 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-21T11:23:35.095 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-21T11:23:35.095 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-21T11:23:35.096 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-21T11:23:35.096 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-21T11:23:35.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:35.096 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-21T11:23:35.158 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-21T11:23:35.158 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-21T11:23:35.158 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-21T11:23:35.158 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-21T11:23:35.158 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-21T11:23:35.158 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-21T11:23:35.158 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T11:23:35.158 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-21T11:23:35.158 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-21T11:23:35.158 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T11:23:35.158 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-21T11:23:35.159 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-21T11:23:35.159 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-21T11:23:35.159 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-21T11:23:35.159 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-21T11:23:35.159 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-21T11:23:35.159 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 41929sec preferred_lft 41929sec 2024-03-21T11:23:35.159 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-21T11:23:35.159 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-21T11:23:35.159 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T11:23:35.159 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-21T11:23:35.159 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-21T11:23:35.160 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-21T11:23:35.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:35.160 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T11:23:35.160 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T11:23:35.160 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add name ceph-brx type bridge 2024-03-21T11:23:35.160 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr flush dev ceph-brx 2024-03-21T11:23:35.160 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set ceph-brx up 2024-03-21T11:23:35.160 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-21T11:23:35.160 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T11:23:35.267 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:35 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/17620333' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T11:23:35.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:35 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/17620333' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T11:23:35.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:35.397 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-21T11:23:35.473 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-21T11:23:35.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:35.479 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-21T11:23:35.490 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:35 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/17620333' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T11:23:35.522 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-21T11:23:35.522 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-21T11:23:35.522 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-21T11:23:35.522 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-21T11:23:35.522 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-21T11:23:35.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:35.523 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T11:23:35.523 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T11:23:35.523 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-21T11:23:35.523 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-21T11:23:35.523 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-21T11:23:35.523 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T11:23:35.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:35.746 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-21T11:23:35.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:35.797 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-03-21T11:23:35.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:35.847 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T11:23:35.847 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T11:23:35.847 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T11:23:35.847 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-21T11:23:35.847 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T11:23:35.987 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-21T11:23:35.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:35.987 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T11:23:35.987 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T11:23:35.988 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-21T11:23:35.988 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-21T11:23:35.988 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-21T11:23:35.988 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-21T11:23:35.988 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-21T11:23:35.988 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T11:23:36.202 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:36 smithi047 ceph-mon[31351]: pgmap v408: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:23:36.202 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:23:36.203 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:36.203 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:36.203 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:36.203 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:36 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:36.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:36.261 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T11:23:36.261 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T11:23:36.261 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-21T11:23:36.261 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.0 master ceph-brx 2024-03-21T11:23:36.261 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T11:23:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:36 smithi100 ceph-mon[36245]: pgmap v408: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:23:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:23:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:36 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:36.413 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T11:23:36.413 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-21T11:23:36.452 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-21T11:23:36.452 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T11:23:36.453 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-21T11:23:36.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:36 smithi102 ceph-mon[36748]: pgmap v408: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:23:36.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:23:36.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:36.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:36.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:36.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:36 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:36.544 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-21T11:23:36.544 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-21T11:23:36.544 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/0b5c929a-10c2-48bb-bf4c-fe3d6041cfdc 2024-03-21T11:23:36.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:36.544 DEBUG:teuthology.orchestra.run.smithi047:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/0b5c929a-10c2-48bb-bf4c-fe3d6041cfdc /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-03-21T11:23:36.664 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-03-21T11:23:36.664 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-03-21T11:23:37.307 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-03-21T11:23:37.307 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-03-21T11:23:37.307 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:6789,172.21.15.100:6789,172.21.15.102:6789" 2024-03-21T11:23:37.307 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: trying mount with new device syntax: 0@a6f35682-e773-11ee-95cd-87774f69a715.cephfs=/volumes/qa/sv_1/0b5c929a-10c2-48bb-bf4c-fe3d6041cfdc 2024-03-21T11:23:37.307 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.47:6789/172.21.15.100:6789/172.21.15.102:6789" will pass to kernel 2024-03-21T11:23:37.310 INFO:teuthology.orchestra.run.smithi047.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-21T11:23:37.310 INFO:teuthology.orchestra.run.smithi047.stdout: You just mounted a file system that supports labels which does not 2024-03-21T11:23:37.310 INFO:teuthology.orchestra.run.smithi047.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-21T11:23:37.310 INFO:teuthology.orchestra.run.smithi047.stdout: applications will generate AVC messages and not be allowed access to 2024-03-21T11:23:37.310 INFO:teuthology.orchestra.run.smithi047.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-21T11:23:37.314 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-21T11:23:37.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:37.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-21T11:23:37.372 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:37.372 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:37.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:37.384 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-21T11:23:37.431 INFO:teuthology.orchestra.run.smithi047.stdout:client25459 2024-03-21T11:23:37.432 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:23:37.432 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T11:23:37.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:37.472 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-21T11:23:37.565 INFO:teuthology.orchestra.run.smithi047.stdout:client25459 2024-03-21T11:23:37.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:37.566 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-21T11:23:37.605 INFO:teuthology.orchestra.run.smithi047.stdout:client25459 2024-03-21T11:23:37.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:37.606 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/a6f35682-e773-11ee-95cd-87774f69a715.client25459/status') 2024-03-21T11:23:37.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:37.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:37.728 INFO:teuthology.orchestra.run.smithi047.stdout:instance: client.25459 (3)172.21.15.47:0/3451763792 2024-03-21T11:23:37.728 INFO:teuthology.orchestra.run.smithi047.stdout:blocklisted: false 2024-03-21T11:23:37.728 INFO:teuthology.orchestra.run.smithi047.stderr:0+1 records in 2024-03-21T11:23:37.728 INFO:teuthology.orchestra.run.smithi047.stderr:0+1 records out 2024-03-21T11:23:37.729 INFO:teuthology.orchestra.run.smithi047.stderr:71 bytes copied, 7.0082e-05 s, 1.0 MB/s 2024-03-21T11:23:37.732 INFO:teuthology.run_tasks:Running task exec... 2024-03-21T11:23:37.742 INFO:teuthology.task.exec:Executing custom commands... 2024-03-21T11:23:37.742 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T11:23:37.743 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-21T11:23:37.743 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:37.744 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.929+0000 7f22e9800640 1 -- 172.21.15.47:0/342184305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e4107b20 msgr2=0x7f22e4107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.929+0000 7f22e9800640 1 --2- 172.21.15.47:0/342184305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e4107b20 0x7f22e4107f40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009900 tx=0x7f22d402bc40 comp rx=0 tx=0).stop 2024-03-21T11:23:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.929+0000 7f22e9800640 1 -- 172.21.15.47:0/342184305 shutdown_connections 2024-03-21T11:23:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.929+0000 7f22e9800640 1 --2- 172.21.15.47:0/342184305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e41096d0 0x7f22e410ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.929+0000 7f22e9800640 1 --2- 172.21.15.47:0/342184305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22e4108d10 0x7f22e4109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.929+0000 7f22e9800640 1 --2- 172.21.15.47:0/342184305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e4107b20 0x7f22e4107f40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.929+0000 7f22e9800640 1 -- 172.21.15.47:0/342184305 >> 172.21.15.47:0/342184305 conn(0x7f22e4076ae0 msgr2=0x7f22e4078f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.929+0000 7f22e9800640 1 -- 172.21.15.47:0/342184305 shutdown_connections 2024-03-21T11:23:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.929+0000 7f22e9800640 1 -- 172.21.15.47:0/342184305 wait complete. 2024-03-21T11:23:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.930+0000 7f22e9800640 1 Processor -- start 2024-03-21T11:23:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.931+0000 7f22e9800640 1 -- start start 2024-03-21T11:23:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.931+0000 7f22e9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e4107b20 0x7f22e419de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.931+0000 7f22e9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e4108d10 0x7f22e419e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.931+0000 7f22e9800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22e41096d0 0x7f22e41a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.931+0000 7f22e9800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e41137a0 con 0x7f22e41096d0 2024-03-21T11:23:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.931+0000 7f22e9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4113620 con 0x7f22e4107b20 2024-03-21T11:23:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.931+0000 7f22e9800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4113920 con 0x7f22e4108d10 2024-03-21T11:23:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.931+0000 7f22e3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e4107b20 0x7f22e419de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.931+0000 7f22dbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e4108d10 0x7f22e419e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.931+0000 7f22e3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e4107b20 0x7f22e419de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59770/0 (socket says 172.21.15.47:59770) 2024-03-21T11:23:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.931+0000 7f22e3e00640 1 -- 172.21.15.47:0/1016952259 learned_addr learned my addr 172.21.15.47:0/1016952259 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.931+0000 7f22e8e00640 1 --2- 172.21.15.47:0/1016952259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22e41096d0 0x7f22e41a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.932+0000 7f22e3e00640 1 -- 172.21.15.47:0/1016952259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e4108d10 msgr2=0x7f22e419e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.932+0000 7f22e3e00640 1 --2- 172.21.15.47:0/1016952259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e4108d10 0x7f22e419e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.932+0000 7f22e3e00640 1 -- 172.21.15.47:0/1016952259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22e41096d0 msgr2=0x7f22e41a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.932+0000 7f22e3e00640 1 --2- 172.21.15.47:0/1016952259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22e41096d0 0x7f22e41a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.932+0000 7f22e3e00640 1 -- 172.21.15.47:0/1016952259 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e41a5a40 con 0x7f22e4107b20 2024-03-21T11:23:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.932+0000 7f22dbe00640 1 --2- 172.21.15.47:0/1016952259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e4108d10 0x7f22e419e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.932+0000 7f22e3e00640 1 --2- 172.21.15.47:0/1016952259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e4107b20 0x7f22e419de20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f22d4000c00 tx=0x7f22d402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.932+0000 7f22e1600640 1 -- 172.21.15.47:0/1016952259 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4041070 con 0x7f22e4107b20 2024-03-21T11:23:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.932+0000 7f22e1600640 1 -- 172.21.15.47:0/1016952259 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22d4004450 con 0x7f22e4107b20 2024-03-21T11:23:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.932+0000 7f22e9800640 1 -- 172.21.15.47:0/1016952259 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e41a5cd0 con 0x7f22e4107b20 2024-03-21T11:23:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.932+0000 7f22e1600640 1 -- 172.21.15.47:0/1016952259 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4034440 con 0x7f22e4107b20 2024-03-21T11:23:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.932+0000 7f22e9800640 1 -- 172.21.15.47:0/1016952259 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e41a61c0 con 0x7f22e4107b20 2024-03-21T11:23:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.933+0000 7f22e9800640 1 -- 172.21.15.47:0/1016952259 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e410ee60 con 0x7f22e4107b20 2024-03-21T11:23:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.935+0000 7f22e1600640 1 -- 172.21.15.47:0/1016952259 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f22d4044070 con 0x7f22e4107b20 2024-03-21T11:23:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.937+0000 7f22e1600640 1 --2- 172.21.15.47:0/1016952259 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22c80a2f10 0x7f22c80a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.937+0000 7f22e1600640 1 -- 172.21.15.47:0/1016952259 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f22d40f1440 con 0x7f22e4107b20 2024-03-21T11:23:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.937+0000 7f22dbe00640 1 --2- 172.21.15.47:0/1016952259 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22c80a2f10 0x7f22c80a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:37.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.938+0000 7f22dbe00640 1 --2- 172.21.15.47:0/1016952259 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22c80a2f10 0x7f22c80a53d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f22e419f330 tx=0x7f22cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:37.944+0000 7f22e1600640 1 -- 172.21.15.47:0/1016952259 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f22d40bd920 con 0x7f22e4107b20 2024-03-21T11:23:38.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:38.199+0000 7f22e9800640 1 -- 172.21.15.47:0/1016952259 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f22e402d070 con 0x7f22e4107b20 2024-03-21T11:23:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:38.816+0000 7f22e1600640 1 -- 172.21.15.47:0/1016952259 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) v1 ==== 118+0+0 (secure 0 0 0) 0x7f22d40c19c0 con 0x7f22e4107b20 2024-03-21T11:23:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:38.820+0000 7f22e9800640 1 -- 172.21.15.47:0/1016952259 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22c80a2f10 msgr2=0x7f22c80a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:38.820+0000 7f22e9800640 1 --2- 172.21.15.47:0/1016952259 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22c80a2f10 0x7f22c80a53d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f22e419f330 tx=0x7f22cc005f90 comp rx=0 tx=0).stop 2024-03-21T11:23:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:38.820+0000 7f22e9800640 1 -- 172.21.15.47:0/1016952259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e4107b20 msgr2=0x7f22e419de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:38.820+0000 7f22e9800640 1 --2- 172.21.15.47:0/1016952259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e4107b20 0x7f22e419de20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f22d4000c00 tx=0x7f22d402c650 comp rx=0 tx=0).stop 2024-03-21T11:23:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:38.820+0000 7f22e9800640 1 -- 172.21.15.47:0/1016952259 shutdown_connections 2024-03-21T11:23:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:38.820+0000 7f22e9800640 1 --2- 172.21.15.47:0/1016952259 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22c80a2f10 0x7f22c80a53d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:38.820+0000 7f22e9800640 1 --2- 172.21.15.47:0/1016952259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22e41096d0 0x7f22e41a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:38.820+0000 7f22e9800640 1 --2- 172.21.15.47:0/1016952259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e4108d10 0x7f22e419e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:38.820+0000 7f22e9800640 1 --2- 172.21.15.47:0/1016952259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e4107b20 0x7f22e419de20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:38.820+0000 7f22e9800640 1 -- 172.21.15.47:0/1016952259 >> 172.21.15.47:0/1016952259 conn(0x7f22e4076ae0 msgr2=0x7f22e410c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:38.820+0000 7f22e9800640 1 -- 172.21.15.47:0/1016952259 shutdown_connections 2024-03-21T11:23:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:38.820+0000 7f22e9800640 1 -- 172.21.15.47:0/1016952259 wait complete. 2024-03-21T11:23:38.846 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-21T11:23:39.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.055+0000 7f0b1a800640 1 -- 172.21.15.47:0/3243300913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b14107d00 msgr2=0x7f0b14108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:39.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.055+0000 7f0b1a800640 1 --2- 172.21.15.47:0/3243300913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b14107d00 0x7f0b14108120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009900 tx=0x7f0b0402bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:39.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.055+0000 7f0b1a800640 1 -- 172.21.15.47:0/3243300913 shutdown_connections 2024-03-21T11:23:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.055+0000 7f0b1a800640 1 --2- 172.21.15.47:0/3243300913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b141098b0 0x7f0b14110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.055+0000 7f0b1a800640 1 --2- 172.21.15.47:0/3243300913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b14108ef0 0x7f0b14109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.055+0000 7f0b1a800640 1 --2- 172.21.15.47:0/3243300913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b14107d00 0x7f0b14108120 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.055+0000 7f0b1a800640 1 -- 172.21.15.47:0/3243300913 >> 172.21.15.47:0/3243300913 conn(0x7f0b14076cb0 msgr2=0x7f0b140790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.055+0000 7f0b1a800640 1 -- 172.21.15.47:0/3243300913 shutdown_connections 2024-03-21T11:23:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.055+0000 7f0b1a800640 1 -- 172.21.15.47:0/3243300913 wait complete. 2024-03-21T11:23:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.056+0000 7f0b1a800640 1 Processor -- start 2024-03-21T11:23:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.056+0000 7f0b1a800640 1 -- start start 2024-03-21T11:23:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b1a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b14107d00 0x7f0b1419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b1a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b14108ef0 0x7f0b1419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b1a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b141098b0 0x7f0b141a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b1a800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b141139d0 con 0x7f0b141098b0 2024-03-21T11:23:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b1a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b14113850 con 0x7f0b14108ef0 2024-03-21T11:23:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b1a800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b14113b50 con 0x7f0b14107d00 2024-03-21T11:23:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b13400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b14107d00 0x7f0b1419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b13400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b14107d00 0x7f0b1419dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51736/0 (socket says 172.21.15.47:51736) 2024-03-21T11:23:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b12a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b14108ef0 0x7f0b1419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b13400640 1 -- 172.21.15.47:0/2214381826 learned_addr learned my addr 172.21.15.47:0/2214381826 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b13e00640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b141098b0 0x7f0b141a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b13400640 1 -- 172.21.15.47:0/2214381826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b14108ef0 msgr2=0x7f0b1419e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b13400640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b14108ef0 0x7f0b1419e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b13400640 1 -- 172.21.15.47:0/2214381826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b141098b0 msgr2=0x7f0b141a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b13400640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b141098b0 0x7f0b141a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.057+0000 7f0b13400640 1 -- 172.21.15.47:0/2214381826 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b141a5be0 con 0x7f0b14107d00 2024-03-21T11:23:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.058+0000 7f0b12a00640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b14108ef0 0x7f0b1419e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.058+0000 7f0b13e00640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b141098b0 0x7f0b141a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.058+0000 7f0b13400640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b14107d00 0x7f0b1419dfc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0b04000c00 tx=0x7f0b0402c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.058+0000 7f0b0be00640 1 -- 172.21.15.47:0/2214381826 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b04043020 con 0x7f0b14107d00 2024-03-21T11:23:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.058+0000 7f0b0be00640 1 -- 172.21.15.47:0/2214381826 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b040054e0 con 0x7f0b14107d00 2024-03-21T11:23:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.058+0000 7f0b0be00640 1 -- 172.21.15.47:0/2214381826 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b04003d70 con 0x7f0b14107d00 2024-03-21T11:23:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.058+0000 7f0b1a800640 1 -- 172.21.15.47:0/2214381826 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b141a5e70 con 0x7f0b14107d00 2024-03-21T11:23:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.058+0000 7f0b1a800640 1 -- 172.21.15.47:0/2214381826 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b141a62e0 con 0x7f0b14107d00 2024-03-21T11:23:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.060+0000 7f0b1a800640 1 -- 172.21.15.47:0/2214381826 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1410f020 con 0x7f0b14107d00 2024-03-21T11:23:39.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.062+0000 7f0b0be00640 1 -- 172.21.15.47:0/2214381826 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0b04046020 con 0x7f0b14107d00 2024-03-21T11:23:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:38 smithi047 ceph-mon[31351]: pgmap v409: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:23:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:38 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:38 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1016952259' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T11:23:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:38 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T11:23:39.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.064+0000 7f0b0be00640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0af80a3040 0x7f0af80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:39.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.064+0000 7f0b0be00640 1 -- 172.21.15.47:0/2214381826 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0b040f18c0 con 0x7f0b14107d00 2024-03-21T11:23:39.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.064+0000 7f0b12a00640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0af80a3040 0x7f0af80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:39.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.065+0000 7f0b12a00640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0af80a3040 0x7f0af80a5500 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0b1419f4d0 tx=0x7f0b0000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:39.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.068+0000 7f0b0be00640 1 -- 172.21.15.47:0/2214381826 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0b0404d020 con 0x7f0b14107d00 2024-03-21T11:23:39.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:38 smithi100 ceph-mon[36245]: pgmap v409: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:23:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:38 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:38 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:38 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:38 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:38 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1016952259' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T11:23:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:38 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T11:23:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:38 smithi102 ceph-mon[36748]: pgmap v409: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T11:23:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:38 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:38 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:38 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:38 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:38 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1016952259' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T11:23:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:38 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T11:23:39.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.306+0000 7f0b1a800640 1 -- 172.21.15.47:0/2214381826 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b14061c40 con 0x7f0af80a3040 2024-03-21T11:23:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.331+0000 7f0b0be00640 1 -- 172.21.15.47:0/2214381826 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f0b14061c40 con 0x7f0af80a3040 2024-03-21T11:23:39.332 INFO:teuthology.orchestra.run.smithi047.stdout:{} 2024-03-21T11:23:39.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.334+0000 7f0b1a800640 1 -- 172.21.15.47:0/2214381826 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0af80a3040 msgr2=0x7f0af80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.334+0000 7f0b1a800640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0af80a3040 0x7f0af80a5500 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0b1419f4d0 tx=0x7f0b0000b410 comp rx=0 tx=0).stop 2024-03-21T11:23:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.334+0000 7f0b1a800640 1 -- 172.21.15.47:0/2214381826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b14107d00 msgr2=0x7f0b1419dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.334+0000 7f0b1a800640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b14107d00 0x7f0b1419dfc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0b04000c00 tx=0x7f0b0402c6c0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.334+0000 7f0b1a800640 1 -- 172.21.15.47:0/2214381826 shutdown_connections 2024-03-21T11:23:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.334+0000 7f0b1a800640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0af80a3040 0x7f0af80a5500 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.334+0000 7f0b1a800640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b141098b0 0x7f0b141a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.334+0000 7f0b1a800640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b14108ef0 0x7f0b1419e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.334+0000 7f0b1a800640 1 --2- 172.21.15.47:0/2214381826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b14107d00 0x7f0b1419dfc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.334+0000 7f0b1a800640 1 -- 172.21.15.47:0/2214381826 >> 172.21.15.47:0/2214381826 conn(0x7f0b14076cb0 msgr2=0x7f0b1410c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.334+0000 7f0b1a800640 1 -- 172.21.15.47:0/2214381826 shutdown_connections 2024-03-21T11:23:39.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.334+0000 7f0b1a800640 1 -- 172.21.15.47:0/2214381826 wait complete. 2024-03-21T11:23:39.352 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-21T11:23:39.370 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-21T11:23:39.406 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:23:39.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.621+0000 7f6ed2600640 1 -- 172.21.15.47:0/228867725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc1016c0 msgr2=0x7f6ecc101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:39.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.621+0000 7f6ed2600640 1 --2- 172.21.15.47:0/228867725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc1016c0 0x7f6ecc101ae0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009900 tx=0x7f6ebc02bc40 comp rx=0 tx=0).stop 2024-03-21T11:23:39.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.621+0000 7f6ed2600640 1 -- 172.21.15.47:0/228867725 shutdown_connections 2024-03-21T11:23:39.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.621+0000 7f6ed2600640 1 --2- 172.21.15.47:0/228867725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ecc103270 0x7f6ecc109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.621+0000 7f6ed2600640 1 --2- 172.21.15.47:0/228867725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ecc1028b0 0x7f6ecc102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.621+0000 7f6ed2600640 1 --2- 172.21.15.47:0/228867725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc1016c0 0x7f6ecc101ae0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.621+0000 7f6ed2600640 1 -- 172.21.15.47:0/228867725 >> 172.21.15.47:0/228867725 conn(0x7f6ecc0fcc50 msgr2=0x7f6ecc0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:39.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.621+0000 7f6ed2600640 1 -- 172.21.15.47:0/228867725 shutdown_connections 2024-03-21T11:23:39.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.621+0000 7f6ed2600640 1 -- 172.21.15.47:0/228867725 wait complete. 2024-03-21T11:23:39.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.622+0000 7f6ed2600640 1 Processor -- start 2024-03-21T11:23:39.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.622+0000 7f6ed2600640 1 -- start start 2024-03-21T11:23:39.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ed2600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ecc1016c0 0x7f6ecc1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:39.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ed2600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ecc1028b0 0x7f6ecc197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:39.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ed2600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc103270 0x7f6ecc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:39.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ed2600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc10d390 con 0x7f6ecc103270 2024-03-21T11:23:39.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ed2600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc10d210 con 0x7f6ecc1016c0 2024-03-21T11:23:39.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ed2600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc10d510 con 0x7f6ecc1028b0 2024-03-21T11:23:39.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ecb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ecc1016c0 0x7f6ecc1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:39.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ecb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ecc1016c0 0x7f6ecc1979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38512/0 (socket says 172.21.15.47:38512) 2024-03-21T11:23:39.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ecb400640 1 -- 172.21.15.47:0/1752478490 learned_addr learned my addr 172.21.15.47:0/1752478490 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:39.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ecaa00640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ecc1028b0 0x7f6ecc197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ecbe00640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc103270 0x7f6ecc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ecb400640 1 -- 172.21.15.47:0/1752478490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ecc1028b0 msgr2=0x7f6ecc197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ecb400640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ecc1028b0 0x7f6ecc197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ecb400640 1 -- 172.21.15.47:0/1752478490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc103270 msgr2=0x7f6ecc19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ecb400640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc103270 0x7f6ecc19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ecb400640 1 -- 172.21.15.47:0/1752478490 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc19f5c0 con 0x7f6ecc1016c0 2024-03-21T11:23:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.623+0000 7f6ecbe00640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc103270 0x7f6ecc19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.624+0000 7f6ecb400640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ecc1016c0 0x7f6ecc1979a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc02efd0 tx=0x7f6ebc02c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.624+0000 7f6ec3e00640 1 -- 172.21.15.47:0/1752478490 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc041070 con 0x7f6ecc1016c0 2024-03-21T11:23:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.624+0000 7f6ed2600640 1 -- 172.21.15.47:0/1752478490 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ecc19f850 con 0x7f6ecc1016c0 2024-03-21T11:23:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.624+0000 7f6ec3e00640 1 -- 172.21.15.47:0/1752478490 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ebc004450 con 0x7f6ecc1016c0 2024-03-21T11:23:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.624+0000 7f6ec3e00640 1 -- 172.21.15.47:0/1752478490 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc003e50 con 0x7f6ecc1016c0 2024-03-21T11:23:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.624+0000 7f6ed2600640 1 -- 172.21.15.47:0/1752478490 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ecc19fd60 con 0x7f6ecc1016c0 2024-03-21T11:23:39.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.625+0000 7f6ed2600640 1 -- 172.21.15.47:0/1752478490 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ecc1089e0 con 0x7f6ecc1016c0 2024-03-21T11:23:39.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.627+0000 7f6ec3e00640 1 -- 172.21.15.47:0/1752478490 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6ebc044070 con 0x7f6ecc1016c0 2024-03-21T11:23:39.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.628+0000 7f6ec3e00640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6eb00a30c0 0x7f6eb00a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:39.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.629+0000 7f6ec3e00640 1 -- 172.21.15.47:0/1752478490 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6ebc0f1350 con 0x7f6ecc1016c0 2024-03-21T11:23:39.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.629+0000 7f6ecaa00640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6eb00a30c0 0x7f6eb00a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:39.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.629+0000 7f6ecaa00640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6eb00a30c0 0x7f6eb00a5580 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc198eb0 tx=0x7f6eb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:39.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.637+0000 7f6ec3e00640 1 -- 172.21.15.47:0/1752478490 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6ebc0bdd90 con 0x7f6ecc1016c0 2024-03-21T11:23:39.815 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:23:39 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:23:39.317+0000 7f772d800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T11:23:39.815 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:23:39 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:23:39.318+0000 7f772d800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T11:23:39.815 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:23:39 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:23:39.318+0000 7f772d800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T11:23:39.815 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:23:39 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:23:39.318+0000 7f772d800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T11:23:39.815 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:23:39 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:23:39.318+0000 7f772d800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T11:23:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.843+0000 7f6ed2600640 1 -- 172.21.15.47:0/1752478490 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ecc0622a0 con 0x7f6ecc1016c0 2024-03-21T11:23:39.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.845+0000 7f6ec3e00640 1 -- 172.21.15.47:0/1752478490 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24747 (secure 0 0 0) 0x7f6ebc0bd690 con 0x7f6ecc1016c0 2024-03-21T11:23:39.846 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:39.847 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:22:50.841892+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:23:39.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.848+0000 7f6ed2600640 1 -- 172.21.15.47:0/1752478490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6eb00a30c0 msgr2=0x7f6eb00a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:39.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.848+0000 7f6ed2600640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6eb00a30c0 0x7f6eb00a5580 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc198eb0 tx=0x7f6eb8005f90 comp rx=0 tx=0).stop 2024-03-21T11:23:39.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.848+0000 7f6ed2600640 1 -- 172.21.15.47:0/1752478490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ecc1016c0 msgr2=0x7f6ecc1979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:39.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.848+0000 7f6ed2600640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ecc1016c0 0x7f6ecc1979a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc02efd0 tx=0x7f6ebc02c660 comp rx=0 tx=0).stop 2024-03-21T11:23:39.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.848+0000 7f6ed2600640 1 -- 172.21.15.47:0/1752478490 shutdown_connections 2024-03-21T11:23:39.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.848+0000 7f6ed2600640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6eb00a30c0 0x7f6eb00a5580 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.848+0000 7f6ed2600640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ecc103270 0x7f6ecc19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.848+0000 7f6ed2600640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ecc1028b0 0x7f6ecc197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.848+0000 7f6ed2600640 1 --2- 172.21.15.47:0/1752478490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ecc1016c0 0x7f6ecc1979a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:39.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.848+0000 7f6ed2600640 1 -- 172.21.15.47:0/1752478490 >> 172.21.15.47:0/1752478490 conn(0x7f6ecc0fcc50 msgr2=0x7f6ecc105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:39.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.849+0000 7f6ed2600640 1 -- 172.21.15.47:0/1752478490 shutdown_connections 2024-03-21T11:23:39.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:39.849+0000 7f6ed2600640 1 -- 172.21.15.47:0/1752478490 wait complete. 2024-03-21T11:23:39.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T11:15:35.545245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T11:20:47.016174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T11:20:48.108506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T11:20:52.254154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T11:23:39.869 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T11:23:40.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.087+0000 7f38bc600640 1 -- 172.21.15.47:0/3164660687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b41016b0 msgr2=0x7f38b4101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.087+0000 7f38bc600640 1 --2- 172.21.15.47:0/3164660687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b41016b0 0x7f38b4101ad0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009900 tx=0x7f38a802bc40 comp rx=0 tx=0).stop 2024-03-21T11:23:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.088+0000 7f38bc600640 1 -- 172.21.15.47:0/3164660687 shutdown_connections 2024-03-21T11:23:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.088+0000 7f38bc600640 1 --2- 172.21.15.47:0/3164660687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38b4103260 0x7f38b4109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.088+0000 7f38bc600640 1 --2- 172.21.15.47:0/3164660687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38b41028a0 0x7f38b4102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.088+0000 7f38bc600640 1 --2- 172.21.15.47:0/3164660687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b41016b0 0x7f38b4101ad0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.088+0000 7f38bc600640 1 -- 172.21.15.47:0/3164660687 >> 172.21.15.47:0/3164660687 conn(0x7f38b40fcc40 msgr2=0x7f38b40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.088+0000 7f38bc600640 1 -- 172.21.15.47:0/3164660687 shutdown_connections 2024-03-21T11:23:40.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.088+0000 7f38bc600640 1 -- 172.21.15.47:0/3164660687 wait complete. 2024-03-21T11:23:40.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.089+0000 7f38bc600640 1 Processor -- start 2024-03-21T11:23:40.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.089+0000 7f38bc600640 1 -- start start 2024-03-21T11:23:40.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.089+0000 7f38bc600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b41016b0 0x7f38b41979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:40.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.089+0000 7f38bc600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38b41028a0 0x7f38b4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:40.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.089+0000 7f38bc600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38b4103260 0x7f38b419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:40.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.089+0000 7f38bc600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b410d350 con 0x7f38b41028a0 2024-03-21T11:23:40.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.089+0000 7f38bc600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b410d1d0 con 0x7f38b4103260 2024-03-21T11:23:40.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.089+0000 7f38bc600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b410d4d0 con 0x7f38b41016b0 2024-03-21T11:23:40.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.089+0000 7f38b9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b41016b0 0x7f38b41979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:40.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.089+0000 7f38b9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b41016b0 0x7f38b41979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51786/0 (socket says 172.21.15.47:51786) 2024-03-21T11:23:40.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.089+0000 7f38b9c00640 1 -- 172.21.15.47:0/3660408151 learned_addr learned my addr 172.21.15.47:0/3660408151 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:40.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.089+0000 7f38ba600640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38b4103260 0x7f38b419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:40.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.089+0000 7f38b9200640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38b41028a0 0x7f38b4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.090+0000 7f38b9c00640 1 -- 172.21.15.47:0/3660408151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38b4103260 msgr2=0x7f38b419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.090+0000 7f38b9c00640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38b4103260 0x7f38b419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.090+0000 7f38b9c00640 1 -- 172.21.15.47:0/3660408151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38b41028a0 msgr2=0x7f38b4197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.090+0000 7f38b9c00640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38b41028a0 0x7f38b4197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.090+0000 7f38b9c00640 1 -- 172.21.15.47:0/3660408151 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b419f610 con 0x7f38b41016b0 2024-03-21T11:23:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.090+0000 7f38ba600640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38b4103260 0x7f38b419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.090+0000 7f38b9c00640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b41016b0 0x7f38b41979f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f38a802efd0 tx=0x7f38a802c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.091+0000 7f38b2000640 1 -- 172.21.15.47:0/3660408151 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8041070 con 0x7f38b41016b0 2024-03-21T11:23:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.091+0000 7f38b2000640 1 -- 172.21.15.47:0/3660408151 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38a8004450 con 0x7f38b41016b0 2024-03-21T11:23:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.091+0000 7f38bc600640 1 -- 172.21.15.47:0/3660408151 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b419f8a0 con 0x7f38b41016b0 2024-03-21T11:23:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.091+0000 7f38b2000640 1 -- 172.21.15.47:0/3660408151 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8003e50 con 0x7f38b41016b0 2024-03-21T11:23:40.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.091+0000 7f38bc600640 1 -- 172.21.15.47:0/3660408151 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b419fdb0 con 0x7f38b41016b0 2024-03-21T11:23:40.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.092+0000 7f38bc600640 1 -- 172.21.15.47:0/3660408151 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b4108a20 con 0x7f38b41016b0 2024-03-21T11:23:40.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.095+0000 7f38b2000640 1 -- 172.21.15.47:0/3660408151 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f38a8044070 con 0x7f38b41016b0 2024-03-21T11:23:40.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.097+0000 7f38b2000640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f389c0a3140 0x7f389c0a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:40.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.097+0000 7f38b2000640 1 -- 172.21.15.47:0/3660408151 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f38a80f12a0 con 0x7f38b41016b0 2024-03-21T11:23:40.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.097+0000 7f38b9200640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f389c0a3140 0x7f389c0a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:40.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.098+0000 7f38b9200640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f389c0a3140 0x7f389c0a5600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f38b4198f00 tx=0x7f38a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:40.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.105+0000 7f38b2000640 1 -- 172.21.15.47:0/3660408151 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f38a80bd690 con 0x7f38b41016b0 2024-03-21T11:23:40.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:39 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-21T11:23:40.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:39 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:23:40.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:39 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:40.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:39 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:40.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:39 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-21T11:23:40.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:39 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:23:40.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:39 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:40.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:39 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:40.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:39 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-21T11:23:40.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:39 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:23:40.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:39 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:40.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:39 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.335+0000 7f38bc600640 1 -- 172.21.15.47:0/3660408151 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f38b40622a0 con 0x7f38b41016b0 2024-03-21T11:23:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.335+0000 7f38b2000640 1 -- 172.21.15.47:0/3660408151 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38a80c1730 con 0x7f38b41016b0 2024-03-21T11:23:40.336 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T11:23:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.337+0000 7f38bc600640 1 -- 172.21.15.47:0/3660408151 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f389c0a3140 msgr2=0x7f389c0a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.337+0000 7f38bc600640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f389c0a3140 0x7f389c0a5600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f38b4198f00 tx=0x7f38a0005f90 comp rx=0 tx=0).stop 2024-03-21T11:23:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.337+0000 7f38bc600640 1 -- 172.21.15.47:0/3660408151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b41016b0 msgr2=0x7f38b41979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.337+0000 7f38bc600640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b41016b0 0x7f38b41979f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f38a802efd0 tx=0x7f38a802c660 comp rx=0 tx=0).stop 2024-03-21T11:23:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.338+0000 7f38bc600640 1 -- 172.21.15.47:0/3660408151 shutdown_connections 2024-03-21T11:23:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.338+0000 7f38bc600640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f389c0a3140 0x7f389c0a5600 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.338+0000 7f38bc600640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38b4103260 0x7f38b419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.338+0000 7f38bc600640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38b41028a0 0x7f38b4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.338+0000 7f38bc600640 1 --2- 172.21.15.47:0/3660408151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38b41016b0 0x7f38b41979f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.338+0000 7f38bc600640 1 -- 172.21.15.47:0/3660408151 >> 172.21.15.47:0/3660408151 conn(0x7f38b40fcc40 msgr2=0x7f38b4105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.338+0000 7f38bc600640 1 -- 172.21.15.47:0/3660408151 shutdown_connections 2024-03-21T11:23:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.338+0000 7f38bc600640 1 -- 172.21.15.47:0/3660408151 wait complete. 2024-03-21T11:23:40.355 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T11:23:40.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.571+0000 7febdfe00640 1 -- 172.21.15.47:0/3353745169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd81038e0 msgr2=0x7febd8103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:40.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.571+0000 7febdfe00640 1 --2- 172.21.15.47:0/3353745169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd81038e0 0x7febd8103d00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7febcc009860 tx=0x7febcc02bb70 comp rx=0 tx=0).stop 2024-03-21T11:23:40.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.571+0000 7febdfe00640 1 -- 172.21.15.47:0/3353745169 shutdown_connections 2024-03-21T11:23:40.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.571+0000 7febdfe00640 1 --2- 172.21.15.47:0/3353745169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd8105490 0x7febd810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.571+0000 7febdfe00640 1 --2- 172.21.15.47:0/3353745169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febd8104ad0 0x7febd8104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.571+0000 7febdfe00640 1 --2- 172.21.15.47:0/3353745169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd81038e0 0x7febd8103d00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.571+0000 7febdfe00640 1 -- 172.21.15.47:0/3353745169 >> 172.21.15.47:0/3353745169 conn(0x7febd80fee50 msgr2=0x7febd8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:40.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.571+0000 7febdfe00640 1 -- 172.21.15.47:0/3353745169 shutdown_connections 2024-03-21T11:23:40.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.571+0000 7febdfe00640 1 -- 172.21.15.47:0/3353745169 wait complete. 2024-03-21T11:23:40.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.573+0000 7febdfe00640 1 Processor -- start 2024-03-21T11:23:40.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.573+0000 7febdfe00640 1 -- start start 2024-03-21T11:23:40.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.573+0000 7febdfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd81038e0 0x7febd8199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:40.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.573+0000 7febdfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd8104ad0 0x7febd819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:40.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.573+0000 7febdfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febd8105490 0x7febd81a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:40.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.573+0000 7febdfe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd810f530 con 0x7febd8104ad0 2024-03-21T11:23:40.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.573+0000 7febdfe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd810f3b0 con 0x7febd81038e0 2024-03-21T11:23:40.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.573+0000 7febdfe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd810f6b0 con 0x7febd8105490 2024-03-21T11:23:40.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.573+0000 7febdde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febd8105490 0x7febd81a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:40.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.574+0000 7febdd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd81038e0 0x7febd8199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:40.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.574+0000 7febdde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febd8105490 0x7febd81a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51800/0 (socket says 172.21.15.47:51800) 2024-03-21T11:23:40.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.574+0000 7febdde00640 1 -- 172.21.15.47:0/3683933456 learned_addr learned my addr 172.21.15.47:0/3683933456 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:40.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.574+0000 7febdca00640 1 --2- 172.21.15.47:0/3683933456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd8104ad0 0x7febd819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:40.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.574+0000 7febdde00640 1 -- 172.21.15.47:0/3683933456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd81038e0 msgr2=0x7febd8199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:40.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.574+0000 7febdde00640 1 --2- 172.21.15.47:0/3683933456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd81038e0 0x7febd8199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.574+0000 7febdde00640 1 -- 172.21.15.47:0/3683933456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd8104ad0 msgr2=0x7febd819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:40.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.574+0000 7febdde00640 1 --2- 172.21.15.47:0/3683933456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd8104ad0 0x7febd819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.574+0000 7febdde00640 1 -- 172.21.15.47:0/3683933456 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd81a1830 con 0x7febd8105490 2024-03-21T11:23:40.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.574+0000 7febdd400640 1 --2- 172.21.15.47:0/3683933456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd81038e0 0x7febd8199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:40.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.574+0000 7febdde00640 1 --2- 172.21.15.47:0/3683933456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febd8105490 0x7febd81a1190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7febd0009fd0 tx=0x7febd000def0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:40.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.574+0000 7febd6000640 1 -- 172.21.15.47:0/3683933456 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0017070 con 0x7febd8105490 2024-03-21T11:23:40.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.575+0000 7febdfe00640 1 -- 172.21.15.47:0/3683933456 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd81a1b20 con 0x7febd8105490 2024-03-21T11:23:40.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.575+0000 7febd6000640 1 -- 172.21.15.47:0/3683933456 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febd0003e50 con 0x7febd8105490 2024-03-21T11:23:40.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.575+0000 7febd6000640 1 -- 172.21.15.47:0/3683933456 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0010460 con 0x7febd8105490 2024-03-21T11:23:40.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.575+0000 7febdfe00640 1 -- 172.21.15.47:0/3683933456 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd81a2020 con 0x7febd8105490 2024-03-21T11:23:40.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.576+0000 7febdfe00640 1 -- 172.21.15.47:0/3683933456 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd810ac80 con 0x7febd8105490 2024-03-21T11:23:40.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.578+0000 7febd6000640 1 -- 172.21.15.47:0/3683933456 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7febd0003970 con 0x7febd8105490 2024-03-21T11:23:40.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.579+0000 7febd6000640 1 --2- 172.21.15.47:0/3683933456 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7febc00a2f20 0x7febc00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:40.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.579+0000 7febd6000640 1 -- 172.21.15.47:0/3683933456 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7febd00cf480 con 0x7febd8105490 2024-03-21T11:23:40.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.579+0000 7febdd400640 1 --2- 172.21.15.47:0/3683933456 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7febc00a2f20 0x7febc00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:40.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.580+0000 7febdd400640 1 --2- 172.21.15.47:0/3683933456 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7febc00a2f20 0x7febc00a53e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7febd819b0f0 tx=0x7febcc02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:40.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.587+0000 7febd6000640 1 -- 172.21.15.47:0/3683933456 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7febd009bf20 con 0x7febd8105490 2024-03-21T11:23:40.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.790+0000 7febdfe00640 1 -- 172.21.15.47:0/3683933456 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7febd80622a0 con 0x7febd8105490 2024-03-21T11:23:40.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.791+0000 7febd6000640 1 -- 172.21.15.47:0/3683933456 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7febd009b820 con 0x7febd8105490 2024-03-21T11:23:40.791 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:23:40.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.794+0000 7febdfe00640 1 -- 172.21.15.47:0/3683933456 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7febc00a2f20 msgr2=0x7febc00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:40.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.794+0000 7febdfe00640 1 --2- 172.21.15.47:0/3683933456 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7febc00a2f20 0x7febc00a53e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7febd819b0f0 tx=0x7febcc02e040 comp rx=0 tx=0).stop 2024-03-21T11:23:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.794+0000 7febdfe00640 1 -- 172.21.15.47:0/3683933456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febd8105490 msgr2=0x7febd81a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.794+0000 7febdfe00640 1 --2- 172.21.15.47:0/3683933456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febd8105490 0x7febd81a1190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7febd0009fd0 tx=0x7febd000def0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.794+0000 7febdfe00640 1 -- 172.21.15.47:0/3683933456 shutdown_connections 2024-03-21T11:23:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.794+0000 7febdfe00640 1 --2- 172.21.15.47:0/3683933456 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7febc00a2f20 0x7febc00a53e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.794+0000 7febdfe00640 1 --2- 172.21.15.47:0/3683933456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febd8105490 0x7febd81a1190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.794+0000 7febdfe00640 1 --2- 172.21.15.47:0/3683933456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febd8104ad0 0x7febd819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.794+0000 7febdfe00640 1 --2- 172.21.15.47:0/3683933456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febd81038e0 0x7febd8199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.794+0000 7febdfe00640 1 -- 172.21.15.47:0/3683933456 >> 172.21.15.47:0/3683933456 conn(0x7febd80fee50 msgr2=0x7febd8107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.794+0000 7febdfe00640 1 -- 172.21.15.47:0/3683933456 shutdown_connections 2024-03-21T11:23:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:40.794+0000 7febdfe00640 1 -- 172.21.15.47:0/3683933456 wait complete. 2024-03-21T11:23:40.816 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T11:23:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.042+0000 7fd73ea00640 1 -- 172.21.15.47:0/62518330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7381038c0 msgr2=0x7fd738103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.042+0000 7fd73ea00640 1 --2- 172.21.15.47:0/62518330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7381038c0 0x7fd738103ce0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd728009900 tx=0x7fd72802bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.042+0000 7fd73ea00640 1 -- 172.21.15.47:0/62518330 shutdown_connections 2024-03-21T11:23:41.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.042+0000 7fd73ea00640 1 --2- 172.21.15.47:0/62518330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd738105470 0x7fd73810bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.042+0000 7fd73ea00640 1 --2- 172.21.15.47:0/62518330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd738104ab0 0x7fd738104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.042+0000 7fd73ea00640 1 --2- 172.21.15.47:0/62518330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7381038c0 0x7fd738103ce0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.042+0000 7fd73ea00640 1 -- 172.21.15.47:0/62518330 >> 172.21.15.47:0/62518330 conn(0x7fd7380fee50 msgr2=0x7fd738101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:41.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.042+0000 7fd73ea00640 1 -- 172.21.15.47:0/62518330 shutdown_connections 2024-03-21T11:23:41.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.042+0000 7fd73ea00640 1 -- 172.21.15.47:0/62518330 wait complete. 2024-03-21T11:23:41.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.043+0000 7fd73ea00640 1 Processor -- start 2024-03-21T11:23:41.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.043+0000 7fd73ea00640 1 -- start start 2024-03-21T11:23:41.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd73ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7381038c0 0x7fd738199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:41.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd73ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd738104ab0 0x7fd73819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd73ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd738105470 0x7fd7381a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd73ea00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73810f510 con 0x7fd738105470 2024-03-21T11:23:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd73ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73810f390 con 0x7fd738104ab0 2024-03-21T11:23:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd73ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73810f690 con 0x7fd7381038c0 2024-03-21T11:23:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd737e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7381038c0 0x7fd738199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd737e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7381038c0 0x7fd738199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51822/0 (socket says 172.21.15.47:51822) 2024-03-21T11:23:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd737e00640 1 -- 172.21.15.47:0/2276066872 learned_addr learned my addr 172.21.15.47:0/2276066872 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd72f400640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd738104ab0 0x7fd73819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd73ca00640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd738105470 0x7fd7381a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd737e00640 1 -- 172.21.15.47:0/2276066872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd738104ab0 msgr2=0x7fd73819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd737e00640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd738104ab0 0x7fd73819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd737e00640 1 -- 172.21.15.47:0/2276066872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd738105470 msgr2=0x7fd7381a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd737e00640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd738105470 0x7fd7381a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd737e00640 1 -- 172.21.15.47:0/2276066872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7381a17d0 con 0x7fd7381038c0 2024-03-21T11:23:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.044+0000 7fd73ca00640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd738105470 0x7fd7381a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.045+0000 7fd737e00640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7381038c0 0x7fd738199bb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd728000c00 tx=0x7fd72802c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.045+0000 7fd735600640 1 -- 172.21.15.47:0/2276066872 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728041070 con 0x7fd7381038c0 2024-03-21T11:23:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.045+0000 7fd735600640 1 -- 172.21.15.47:0/2276066872 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd728004450 con 0x7fd7381038c0 2024-03-21T11:23:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.045+0000 7fd735600640 1 -- 172.21.15.47:0/2276066872 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728034440 con 0x7fd7381038c0 2024-03-21T11:23:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.045+0000 7fd73ea00640 1 -- 172.21.15.47:0/2276066872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7381a1a60 con 0x7fd7381038c0 2024-03-21T11:23:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.045+0000 7fd73ea00640 1 -- 172.21.15.47:0/2276066872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7381a1fa0 con 0x7fd7381038c0 2024-03-21T11:23:41.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.046+0000 7fd73ea00640 1 -- 172.21.15.47:0/2276066872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73810ac60 con 0x7fd7381038c0 2024-03-21T11:23:41.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.048+0000 7fd735600640 1 -- 172.21.15.47:0/2276066872 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd728044070 con 0x7fd7381038c0 2024-03-21T11:23:41.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.049+0000 7fd735600640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd71c0a2fa0 0x7fd71c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:41.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.049+0000 7fd735600640 1 -- 172.21.15.47:0/2276066872 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd7280f12c0 con 0x7fd7381038c0 2024-03-21T11:23:41.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.049+0000 7fd72f400640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd71c0a2fa0 0x7fd71c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:41.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.050+0000 7fd72f400640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd71c0a2fa0 0x7fd71c0a5460 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd73819b0c0 tx=0x7fd720005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.053+0000 7fd735600640 1 -- 172.21.15.47:0/2276066872 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd7280bdd90 con 0x7fd7381038c0 2024-03-21T11:23:41.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:40 smithi047 ceph-mon[31351]: pgmap v410: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-21T11:23:41.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:40 smithi047 ceph-mon[31351]: from='client.25471 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:41.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:40 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1752478490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:41.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:40 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:23:41.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:40 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:23:41.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:40 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:41.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:40 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3660408151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:41.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:40 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3683933456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:41.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:40 smithi100 ceph-mon[36245]: pgmap v410: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-21T11:23:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:40 smithi100 ceph-mon[36245]: from='client.25471 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:40 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1752478490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:40 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:23:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:40 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:23:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:40 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:40 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3660408151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:40 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3683933456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:40 smithi102 ceph-mon[36748]: pgmap v410: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-21T11:23:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:40 smithi102 ceph-mon[36748]: from='client.25471 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T11:23:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:40 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1752478490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:40 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:23:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:40 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:23:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:40 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:23:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:40 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3660408151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:40 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3683933456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:41.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.256+0000 7fd73ea00640 1 -- 172.21.15.47:0/2276066872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd73802d070 con 0x7fd7381038c0 2024-03-21T11:23:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.256+0000 7fd735600640 1 -- 172.21.15.47:0/2276066872 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd7280bd690 con 0x7fd7381038c0 2024-03-21T11:23:41.257 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T11:23:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.259+0000 7fd73ea00640 1 -- 172.21.15.47:0/2276066872 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd71c0a2fa0 msgr2=0x7fd71c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.259+0000 7fd73ea00640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd71c0a2fa0 0x7fd71c0a5460 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd73819b0c0 tx=0x7fd720005f90 comp rx=0 tx=0).stop 2024-03-21T11:23:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.259+0000 7fd73ea00640 1 -- 172.21.15.47:0/2276066872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7381038c0 msgr2=0x7fd738199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.259+0000 7fd73ea00640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7381038c0 0x7fd738199bb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd728000c00 tx=0x7fd72802c640 comp rx=0 tx=0).stop 2024-03-21T11:23:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.260+0000 7fd73ea00640 1 -- 172.21.15.47:0/2276066872 shutdown_connections 2024-03-21T11:23:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.260+0000 7fd73ea00640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd71c0a2fa0 0x7fd71c0a5460 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.260+0000 7fd73ea00640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd738105470 0x7fd7381a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.260+0000 7fd73ea00640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd738104ab0 0x7fd73819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.260+0000 7fd73ea00640 1 --2- 172.21.15.47:0/2276066872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7381038c0 0x7fd738199bb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.260+0000 7fd73ea00640 1 -- 172.21.15.47:0/2276066872 >> 172.21.15.47:0/2276066872 conn(0x7fd7380fee50 msgr2=0x7fd738107e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.260+0000 7fd73ea00640 1 -- 172.21.15.47:0/2276066872 shutdown_connections 2024-03-21T11:23:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.260+0000 7fd73ea00640 1 -- 172.21.15.47:0/2276066872 wait complete. 2024-03-21T11:23:41.281 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T11:23:41.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.492+0000 7f1bbea00640 1 -- 172.21.15.47:0/3915191677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb8104990 msgr2=0x7f1bb8104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.492+0000 7f1bbea00640 1 --2- 172.21.15.47:0/3915191677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb8104990 0x7f1bb8104e10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009900 tx=0x7f1ba402bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.492+0000 7f1bbea00640 1 -- 172.21.15.47:0/3915191677 shutdown_connections 2024-03-21T11:23:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.492+0000 7f1bbea00640 1 --2- 172.21.15.47:0/3915191677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bb8105350 0x7f1bb810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.492+0000 7f1bbea00640 1 --2- 172.21.15.47:0/3915191677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb8104990 0x7f1bb8104e10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.492+0000 7f1bbea00640 1 --2- 172.21.15.47:0/3915191677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb81037a0 0x7f1bb8103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.492+0000 7f1bbea00640 1 -- 172.21.15.47:0/3915191677 >> 172.21.15.47:0/3915191677 conn(0x7f1bb80fed10 msgr2=0x7f1bb8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.492+0000 7f1bbea00640 1 -- 172.21.15.47:0/3915191677 shutdown_connections 2024-03-21T11:23:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.492+0000 7f1bbea00640 1 -- 172.21.15.47:0/3915191677 wait complete. 2024-03-21T11:23:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.493+0000 7f1bbea00640 1 Processor -- start 2024-03-21T11:23:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.493+0000 7f1bbea00640 1 -- start start 2024-03-21T11:23:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb81037a0 0x7f1bb8199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bb8104990 0x7f1bb8199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb8105350 0x7f1bb81a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbea00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb810f420 con 0x7f1bb8104990 2024-03-21T11:23:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb810f2a0 con 0x7f1bb81037a0 2024-03-21T11:23:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb810f5a0 con 0x7f1bb8105350 2024-03-21T11:23:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbe000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb8105350 0x7f1bb81a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb81037a0 0x7f1bb8199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbe000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb8105350 0x7f1bb81a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51842/0 (socket says 172.21.15.47:51842) 2024-03-21T11:23:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbe000640 1 -- 172.21.15.47:0/2717984611 learned_addr learned my addr 172.21.15.47:0/2717984611 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbcc00640 1 --2- 172.21.15.47:0/2717984611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bb8104990 0x7f1bb8199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbe000640 1 -- 172.21.15.47:0/2717984611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb81037a0 msgr2=0x7f1bb8199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbe000640 1 --2- 172.21.15.47:0/2717984611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb81037a0 0x7f1bb8199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbe000640 1 -- 172.21.15.47:0/2717984611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bb8104990 msgr2=0x7f1bb8199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbe000640 1 --2- 172.21.15.47:0/2717984611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bb8104990 0x7f1bb8199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.494+0000 7f1bbe000640 1 -- 172.21.15.47:0/2717984611 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb81a16b0 con 0x7f1bb8105350 2024-03-21T11:23:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.495+0000 7f1bbe000640 1 --2- 172.21.15.47:0/2717984611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb8105350 0x7f1bb81a1040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000eaa0 tx=0x7f1bb000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.495+0000 7f1bb6000640 1 -- 172.21.15.47:0/2717984611 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb000cdb0 con 0x7f1bb8105350 2024-03-21T11:23:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.495+0000 7f1bb6000640 1 -- 172.21.15.47:0/2717984611 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bb0005020 con 0x7f1bb8105350 2024-03-21T11:23:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.495+0000 7f1bbea00640 1 -- 172.21.15.47:0/2717984611 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb81a19a0 con 0x7f1bb8105350 2024-03-21T11:23:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.495+0000 7f1bb6000640 1 -- 172.21.15.47:0/2717984611 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00116b0 con 0x7f1bb8105350 2024-03-21T11:23:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.495+0000 7f1bbea00640 1 -- 172.21.15.47:0/2717984611 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb81a1e70 con 0x7f1bb8105350 2024-03-21T11:23:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.496+0000 7f1bbea00640 1 -- 172.21.15.47:0/2717984611 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb810aae0 con 0x7f1bb8105350 2024-03-21T11:23:41.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.498+0000 7f1bb6000640 1 -- 172.21.15.47:0/2717984611 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1bb0005560 con 0x7f1bb8105350 2024-03-21T11:23:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.500+0000 7f1bb6000640 1 --2- 172.21.15.47:0/2717984611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1b9c0a2f20 0x7f1b9c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.500+0000 7f1bb6000640 1 -- 172.21.15.47:0/2717984611 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1bb00cf940 con 0x7f1bb8105350 2024-03-21T11:23:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.500+0000 7f1bbd600640 1 --2- 172.21.15.47:0/2717984611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1b9c0a2f20 0x7f1b9c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.501+0000 7f1bbd600640 1 --2- 172.21.15.47:0/2717984611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1b9c0a2f20 0x7f1b9c0a53e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1bb81047f0 tx=0x7f1ba8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.504+0000 7f1bb6000640 1 -- 172.21.15.47:0/2717984611 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1bb0016030 con 0x7f1bb8105350 2024-03-21T11:23:41.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.705+0000 7f1bbea00640 1 -- 172.21.15.47:0/2717984611 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1bb80622a0 con 0x7f1bb8105350 2024-03-21T11:23:41.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.706+0000 7f1bb6000640 1 -- 172.21.15.47:0/2717984611 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1bb009bd30 con 0x7f1bb8105350 2024-03-21T11:23:41.707 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:23:41.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.709+0000 7f1bbea00640 1 -- 172.21.15.47:0/2717984611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1b9c0a2f20 msgr2=0x7f1b9c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:41.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.709+0000 7f1bbea00640 1 --2- 172.21.15.47:0/2717984611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1b9c0a2f20 0x7f1b9c0a53e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1bb81047f0 tx=0x7f1ba8005f90 comp rx=0 tx=0).stop 2024-03-21T11:23:41.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.709+0000 7f1bbea00640 1 -- 172.21.15.47:0/2717984611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb8105350 msgr2=0x7f1bb81a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:41.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.709+0000 7f1bbea00640 1 --2- 172.21.15.47:0/2717984611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb8105350 0x7f1bb81a1040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000eaa0 tx=0x7f1bb000ef60 comp rx=0 tx=0).stop 2024-03-21T11:23:41.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.709+0000 7f1bbea00640 1 -- 172.21.15.47:0/2717984611 shutdown_connections 2024-03-21T11:23:41.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.709+0000 7f1bbea00640 1 --2- 172.21.15.47:0/2717984611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1b9c0a2f20 0x7f1b9c0a53e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.709+0000 7f1bbea00640 1 --2- 172.21.15.47:0/2717984611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bb8105350 0x7f1bb81a1040 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.709+0000 7f1bbea00640 1 --2- 172.21.15.47:0/2717984611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bb8104990 0x7f1bb8199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.709+0000 7f1bbea00640 1 --2- 172.21.15.47:0/2717984611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb81037a0 0x7f1bb8199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.709+0000 7f1bbea00640 1 -- 172.21.15.47:0/2717984611 >> 172.21.15.47:0/2717984611 conn(0x7f1bb80fed10 msgr2=0x7f1bb8107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.709+0000 7f1bbea00640 1 -- 172.21.15.47:0/2717984611 shutdown_connections 2024-03-21T11:23:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.709+0000 7f1bbea00640 1 -- 172.21.15.47:0/2717984611 wait complete. 2024-03-21T11:23:41.729 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:23:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.954+0000 7f256cc00640 1 -- 172.21.15.47:0/2551841110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2568077c40 msgr2=0x7f25680784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.954+0000 7f256cc00640 1 --2- 172.21.15.47:0/2551841110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2568077c40 0x7f25680784e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2560009ca0 tx=0x7f256002c0f0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.955+0000 7f256cc00640 1 -- 172.21.15.47:0/2551841110 shutdown_connections 2024-03-21T11:23:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.955+0000 7f256cc00640 1 --2- 172.21.15.47:0/2551841110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2568077c40 0x7f25680784e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.955+0000 7f256cc00640 1 --2- 172.21.15.47:0/2551841110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25680772a0 0x7f2568077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.955+0000 7f256cc00640 1 --2- 172.21.15.47:0/2551841110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2568076060 0x7f2568076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.955+0000 7f256cc00640 1 -- 172.21.15.47:0/2551841110 >> 172.21.15.47:0/2551841110 conn(0x7f25680fee50 msgr2=0x7f2568101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.955+0000 7f256cc00640 1 -- 172.21.15.47:0/2551841110 shutdown_connections 2024-03-21T11:23:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.955+0000 7f256cc00640 1 -- 172.21.15.47:0/2551841110 wait complete. 2024-03-21T11:23:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.956+0000 7f256cc00640 1 Processor -- start 2024-03-21T11:23:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.956+0000 7f256cc00640 1 -- start start 2024-03-21T11:23:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f256cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2568076060 0x7f256819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f256cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25680772a0 0x7f256819e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f256cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2568077c40 0x7f25681a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f256cc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2568113970 con 0x7f25680772a0 2024-03-21T11:23:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f256cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25681137f0 con 0x7f2568076060 2024-03-21T11:23:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f256cc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2568113af0 con 0x7f2568077c40 2024-03-21T11:23:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f2565e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2568076060 0x7f256819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f2565e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2568076060 0x7f256819df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38594/0 (socket says 172.21.15.47:38594) 2024-03-21T11:23:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f2565e00640 1 -- 172.21.15.47:0/4061324769 learned_addr learned my addr 172.21.15.47:0/4061324769 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f2565400640 1 --2- 172.21.15.47:0/4061324769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25680772a0 0x7f256819e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f2566800640 1 --2- 172.21.15.47:0/4061324769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2568077c40 0x7f25681a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f2565e00640 1 -- 172.21.15.47:0/4061324769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2568077c40 msgr2=0x7f25681a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f2565e00640 1 --2- 172.21.15.47:0/4061324769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2568077c40 0x7f25681a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f2565e00640 1 -- 172.21.15.47:0/4061324769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25680772a0 msgr2=0x7f256819e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f2565e00640 1 --2- 172.21.15.47:0/4061324769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25680772a0 0x7f256819e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.957+0000 7f2565e00640 1 -- 172.21.15.47:0/4061324769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25681a5b80 con 0x7f2568076060 2024-03-21T11:23:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.958+0000 7f2565e00640 1 --2- 172.21.15.47:0/4061324769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2568076060 0x7f256819df60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f255800e9f0 tx=0x7f255800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.958+0000 7f255ea00640 1 -- 172.21.15.47:0/4061324769 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255800cd80 con 0x7f2568076060 2024-03-21T11:23:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.958+0000 7f255ea00640 1 -- 172.21.15.47:0/4061324769 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2558005500 con 0x7f2568076060 2024-03-21T11:23:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.958+0000 7f256cc00640 1 -- 172.21.15.47:0/4061324769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25681a5e70 con 0x7f2568076060 2024-03-21T11:23:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.958+0000 7f255ea00640 1 -- 172.21.15.47:0/4061324769 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558010690 con 0x7f2568076060 2024-03-21T11:23:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.958+0000 7f256cc00640 1 -- 172.21.15.47:0/4061324769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25681a62d0 con 0x7f2568076060 2024-03-21T11:23:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.959+0000 7f256cc00640 1 -- 172.21.15.47:0/4061324769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2568110390 con 0x7f2568076060 2024-03-21T11:23:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.961+0000 7f255ea00640 1 -- 172.21.15.47:0/4061324769 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f25580046c0 con 0x7f2568076060 2024-03-21T11:23:41.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.962+0000 7f255ea00640 1 --2- 172.21.15.47:0/4061324769 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f254c0a3030 0x7f254c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:41.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.963+0000 7f255ea00640 1 -- 172.21.15.47:0/4061324769 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f25580cfd90 con 0x7f2568076060 2024-03-21T11:23:41.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.963+0000 7f2565400640 1 --2- 172.21.15.47:0/4061324769 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f254c0a3030 0x7f254c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:41.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.964+0000 7f2565400640 1 --2- 172.21.15.47:0/4061324769 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f254c0a3030 0x7f254c0a54f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f256819f470 tx=0x7f2550005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:41.966+0000 7f255ea00640 1 -- 172.21.15.47:0/4061324769 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f255809c670 con 0x7f2568076060 2024-03-21T11:23:42.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:41 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2276066872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:42.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:41 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2717984611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:42.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:41 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2276066872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:42.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:41 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2717984611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.174+0000 7f256cc00640 1 -- 172.21.15.47:0/4061324769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f256802d2c0 con 0x7f2568076060 2024-03-21T11:23:42.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.176+0000 7f255ea00640 1 -- 172.21.15.47:0/4061324769 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24747 (secure 0 0 0) 0x7f255809bf70 con 0x7f2568076060 2024-03-21T11:23:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:42.177 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:22:50.841892+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:23:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.179+0000 7f256cc00640 1 -- 172.21.15.47:0/4061324769 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f254c0a3030 msgr2=0x7f254c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.179+0000 7f256cc00640 1 --2- 172.21.15.47:0/4061324769 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f254c0a3030 0x7f254c0a54f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f256819f470 tx=0x7f2550005f90 comp rx=0 tx=0).stop 2024-03-21T11:23:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.179+0000 7f256cc00640 1 -- 172.21.15.47:0/4061324769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2568076060 msgr2=0x7f256819df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:42.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.179+0000 7f256cc00640 1 --2- 172.21.15.47:0/4061324769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2568076060 0x7f256819df60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f255800e9f0 tx=0x7f255800edf0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.179+0000 7f256cc00640 1 -- 172.21.15.47:0/4061324769 shutdown_connections 2024-03-21T11:23:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.179+0000 7f256cc00640 1 --2- 172.21.15.47:0/4061324769 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f254c0a3030 0x7f254c0a54f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.179+0000 7f256cc00640 1 --2- 172.21.15.47:0/4061324769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2568077c40 0x7f25681a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.179+0000 7f256cc00640 1 --2- 172.21.15.47:0/4061324769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25680772a0 0x7f256819e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.179+0000 7f256cc00640 1 --2- 172.21.15.47:0/4061324769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2568076060 0x7f256819df60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.179+0000 7f256cc00640 1 -- 172.21.15.47:0/4061324769 >> 172.21.15.47:0/4061324769 conn(0x7f25680fee50 msgr2=0x7f2568100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.179+0000 7f256cc00640 1 -- 172.21.15.47:0/4061324769 shutdown_connections 2024-03-21T11:23:42.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.180+0000 7f256cc00640 1 -- 172.21.15.47:0/4061324769 wait complete. 2024-03-21T11:23:42.200 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T11:15:35.545245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T11:20:47.016174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T11:20:48.108506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T11:20:52.254154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T11:23:42.201 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T11:23:42.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:41 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2276066872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:42.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:41 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2717984611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.421+0000 7f15cb200640 1 -- 172.21.15.47:0/364420139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c4105470 msgr2=0x7f15c410bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.421+0000 7f15cb200640 1 --2- 172.21.15.47:0/364420139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c4105470 0x7f15c410bd00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009ca0 tx=0x7f15bc02c100 comp rx=0 tx=0).stop 2024-03-21T11:23:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.422+0000 7f15cb200640 1 -- 172.21.15.47:0/364420139 shutdown_connections 2024-03-21T11:23:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.422+0000 7f15cb200640 1 --2- 172.21.15.47:0/364420139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c4105470 0x7f15c410bd00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.422+0000 7f15cb200640 1 --2- 172.21.15.47:0/364420139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c4104ab0 0x7f15c4104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.422+0000 7f15cb200640 1 --2- 172.21.15.47:0/364420139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15c41038c0 0x7f15c4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.422+0000 7f15cb200640 1 -- 172.21.15.47:0/364420139 >> 172.21.15.47:0/364420139 conn(0x7f15c40fee50 msgr2=0x7f15c4101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.422+0000 7f15cb200640 1 -- 172.21.15.47:0/364420139 shutdown_connections 2024-03-21T11:23:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.422+0000 7f15cb200640 1 -- 172.21.15.47:0/364420139 wait complete. 2024-03-21T11:23:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.423+0000 7f15cb200640 1 Processor -- start 2024-03-21T11:23:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.423+0000 7f15cb200640 1 -- start start 2024-03-21T11:23:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.423+0000 7f15cb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c41038c0 0x7f15c4199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.423+0000 7f15cb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15c4104ab0 0x7f15c419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.423+0000 7f15cb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c4105470 0x7f15c41a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.423+0000 7f15cb200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c410f540 con 0x7f15c4104ab0 2024-03-21T11:23:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.423+0000 7f15cb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c410f3c0 con 0x7f15c4105470 2024-03-21T11:23:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.423+0000 7f15cb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c410f6c0 con 0x7f15c41038c0 2024-03-21T11:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.424+0000 7f15c9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c4105470 0x7f15c41a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.424+0000 7f15c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c41038c0 0x7f15c4199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.424+0000 7f15c9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c4105470 0x7f15c41a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38616/0 (socket says 172.21.15.47:38616) 2024-03-21T11:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.424+0000 7f15c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c41038c0 0x7f15c4199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51872/0 (socket says 172.21.15.47:51872) 2024-03-21T11:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.424+0000 7f15c9200640 1 -- 172.21.15.47:0/4117764616 learned_addr learned my addr 172.21.15.47:0/4117764616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.424+0000 7f15c3400640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15c4104ab0 0x7f15c419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.424+0000 7f15c9200640 1 -- 172.21.15.47:0/4117764616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c41038c0 msgr2=0x7f15c4199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.424+0000 7f15c9200640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c41038c0 0x7f15c4199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.424+0000 7f15c9200640 1 -- 172.21.15.47:0/4117764616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15c4104ab0 msgr2=0x7f15c419a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.424+0000 7f15c9200640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15c4104ab0 0x7f15c419a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.424+0000 7f15c9200640 1 -- 172.21.15.47:0/4117764616 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c41a1810 con 0x7f15c4105470 2024-03-21T11:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.424+0000 7f15c3e00640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c41038c0 0x7f15c4199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.424+0000 7f15c3400640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15c4104ab0 0x7f15c419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:42.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.424+0000 7f15c9200640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c4105470 0x7f15c41a11a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f15bc037040 tx=0x7f15bc035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:42.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.425+0000 7f15c0c00640 1 -- 172.21.15.47:0/4117764616 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc039040 con 0x7f15c4105470 2024-03-21T11:23:42.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.425+0000 7f15cb200640 1 -- 172.21.15.47:0/4117764616 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c41a1aa0 con 0x7f15c4105470 2024-03-21T11:23:42.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.425+0000 7f15c0c00640 1 -- 172.21.15.47:0/4117764616 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15bc044070 con 0x7f15c4105470 2024-03-21T11:23:42.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.425+0000 7f15c0c00640 1 -- 172.21.15.47:0/4117764616 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc004440 con 0x7f15c4105470 2024-03-21T11:23:42.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.425+0000 7f15cb200640 1 -- 172.21.15.47:0/4117764616 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c41a1fe0 con 0x7f15c4105470 2024-03-21T11:23:42.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.426+0000 7f15cb200640 1 -- 172.21.15.47:0/4117764616 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c410ab80 con 0x7f15c4105470 2024-03-21T11:23:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.428+0000 7f15c0c00640 1 -- 172.21.15.47:0/4117764616 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f15bc0045e0 con 0x7f15c4105470 2024-03-21T11:23:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.429+0000 7f15c0c00640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15a80a3030 0x7f15a80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.429+0000 7f15c0c00640 1 -- 172.21.15.47:0/4117764616 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f15bc0f1e00 con 0x7f15c4105470 2024-03-21T11:23:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.429+0000 7f15c3e00640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15a80a3030 0x7f15a80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.430+0000 7f15c3e00640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15a80a3030 0x7f15a80a54f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f15b4005fd0 tx=0x7f15b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:42.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.437+0000 7f15c0c00640 1 -- 172.21.15.47:0/4117764616 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f15bc0be7f0 con 0x7f15c4105470 2024-03-21T11:23:42.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.645+0000 7f15cb200640 1 -- 172.21.15.47:0/4117764616 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15c402d240 con 0x7f15c4105470 2024-03-21T11:23:42.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.645+0000 7f15c0c00640 1 -- 172.21.15.47:0/4117764616 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15bc0be0f0 con 0x7f15c4105470 2024-03-21T11:23:42.646 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T11:23:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.648+0000 7f15cb200640 1 -- 172.21.15.47:0/4117764616 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15a80a3030 msgr2=0x7f15a80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.648+0000 7f15cb200640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15a80a3030 0x7f15a80a54f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f15b4005fd0 tx=0x7f15b400b410 comp rx=0 tx=0).stop 2024-03-21T11:23:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.648+0000 7f15cb200640 1 -- 172.21.15.47:0/4117764616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c4105470 msgr2=0x7f15c41a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.648+0000 7f15cb200640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c4105470 0x7f15c41a11a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f15bc037040 tx=0x7f15bc035d10 comp rx=0 tx=0).stop 2024-03-21T11:23:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.648+0000 7f15cb200640 1 -- 172.21.15.47:0/4117764616 shutdown_connections 2024-03-21T11:23:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.648+0000 7f15cb200640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15a80a3030 0x7f15a80a54f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.648+0000 7f15cb200640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c4105470 0x7f15c41a11a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.648+0000 7f15cb200640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15c4104ab0 0x7f15c419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.648+0000 7f15cb200640 1 --2- 172.21.15.47:0/4117764616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c41038c0 0x7f15c4199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.648+0000 7f15cb200640 1 -- 172.21.15.47:0/4117764616 >> 172.21.15.47:0/4117764616 conn(0x7f15c40fee50 msgr2=0x7f15c4107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.649+0000 7f15cb200640 1 -- 172.21.15.47:0/4117764616 shutdown_connections 2024-03-21T11:23:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.649+0000 7f15cb200640 1 -- 172.21.15.47:0/4117764616 wait complete. 2024-03-21T11:23:42.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T11:23:42.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.900+0000 7f258d000640 1 -- 172.21.15.47:0/1294282823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25881017b0 msgr2=0x7f2588101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:42.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.900+0000 7f258d000640 1 --2- 172.21.15.47:0/1294282823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25881017b0 0x7f2588101c30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2574009900 tx=0x7f257402bc40 comp rx=0 tx=0).stop 2024-03-21T11:23:42.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.900+0000 7f258d000640 1 -- 172.21.15.47:0/1294282823 shutdown_connections 2024-03-21T11:23:42.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.900+0000 7f258d000640 1 --2- 172.21.15.47:0/1294282823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2588102170 0x7f2588108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.900+0000 7f258d000640 1 --2- 172.21.15.47:0/1294282823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25881017b0 0x7f2588101c30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.900+0000 7f258d000640 1 --2- 172.21.15.47:0/1294282823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25881005c0 0x7f25881009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.900+0000 7f258d000640 1 -- 172.21.15.47:0/1294282823 >> 172.21.15.47:0/1294282823 conn(0x7f25880fbb50 msgr2=0x7f25880fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:42.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.900+0000 7f258d000640 1 -- 172.21.15.47:0/1294282823 shutdown_connections 2024-03-21T11:23:42.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.900+0000 7f258d000640 1 -- 172.21.15.47:0/1294282823 wait complete. 2024-03-21T11:23:42.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.901+0000 7f258d000640 1 Processor -- start 2024-03-21T11:23:42.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.901+0000 7f258d000640 1 -- start start 2024-03-21T11:23:42.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.901+0000 7f258d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25881005c0 0x7f2588104550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:42.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.901+0000 7f258d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25881017b0 0x7f2588102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:42.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.901+0000 7f258d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2588102170 0x7f25881030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:42.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.901+0000 7f258d000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258810c280 con 0x7f2588102170 2024-03-21T11:23:42.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.901+0000 7f258d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258810c100 con 0x7f25881017b0 2024-03-21T11:23:42.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.901+0000 7f258d000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258810c400 con 0x7f25881005c0 2024-03-21T11:23:42.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.902+0000 7f2585e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25881005c0 0x7f2588104550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:42.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.902+0000 7f2585400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25881017b0 0x7f2588102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:42.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.902+0000 7f2585e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25881005c0 0x7f2588104550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51884/0 (socket says 172.21.15.47:51884) 2024-03-21T11:23:42.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.902+0000 7f2585e00640 1 -- 172.21.15.47:0/385030909 learned_addr learned my addr 172.21.15.47:0/385030909 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:42.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.902+0000 7f2586800640 1 --2- 172.21.15.47:0/385030909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2588102170 0x7f25881030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:42.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.902+0000 7f2585e00640 1 -- 172.21.15.47:0/385030909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25881017b0 msgr2=0x7f2588102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:42.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.902+0000 7f2585e00640 1 --2- 172.21.15.47:0/385030909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25881017b0 0x7f2588102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.902+0000 7f2585e00640 1 -- 172.21.15.47:0/385030909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2588102170 msgr2=0x7f25881030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:42.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.902+0000 7f2585e00640 1 --2- 172.21.15.47:0/385030909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2588102170 0x7f25881030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:42.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.902+0000 7f2585e00640 1 -- 172.21.15.47:0/385030909 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588103960 con 0x7f25881005c0 2024-03-21T11:23:42.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.902+0000 7f2585e00640 1 --2- 172.21.15.47:0/385030909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25881005c0 0x7f2588104550 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f257800e9c0 tx=0x7f257800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:42.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.903+0000 7f257ea00640 1 -- 172.21.15.47:0/385030909 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800cd80 con 0x7f25881005c0 2024-03-21T11:23:42.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.903+0000 7f257ea00640 1 -- 172.21.15.47:0/385030909 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25780035f0 con 0x7f25881005c0 2024-03-21T11:23:42.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.903+0000 7f258d000640 1 -- 172.21.15.47:0/385030909 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25881a1ad0 con 0x7f25881005c0 2024-03-21T11:23:42.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.903+0000 7f257ea00640 1 -- 172.21.15.47:0/385030909 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578010690 con 0x7f25881005c0 2024-03-21T11:23:42.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.903+0000 7f258d000640 1 -- 172.21.15.47:0/385030909 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25881a2040 con 0x7f25881005c0 2024-03-21T11:23:42.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.904+0000 7f258d000640 1 -- 172.21.15.47:0/385030909 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25880688e0 con 0x7f25881005c0 2024-03-21T11:23:42.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.905+0000 7f257ea00640 1 -- 172.21.15.47:0/385030909 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f25780037a0 con 0x7f25881005c0 2024-03-21T11:23:42.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.907+0000 7f257ea00640 1 --2- 172.21.15.47:0/385030909 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f256c0a3030 0x7f256c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:42.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.907+0000 7f257ea00640 1 -- 172.21.15.47:0/385030909 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f25780cfd30 con 0x7f25881005c0 2024-03-21T11:23:42.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.907+0000 7f2585400640 1 --2- 172.21.15.47:0/385030909 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f256c0a3030 0x7f256c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:42.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.908+0000 7f2585400640 1 --2- 172.21.15.47:0/385030909 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f256c0a3030 0x7f256c0a54f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f257402c250 tx=0x7f257402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:42.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:42.911+0000 7f257ea00640 1 -- 172.21.15.47:0/385030909 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f257809c610 con 0x7f25881005c0 2024-03-21T11:23:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:42 smithi047 ceph-mon[31351]: pgmap v411: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-21T11:23:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:42 smithi047 ceph-mon[31351]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-21T11:23:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:42 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/4061324769' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:42 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/4117764616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.114+0000 7f258d000640 1 -- 172.21.15.47:0/385030909 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25881a23e0 con 0x7f25881005c0 2024-03-21T11:23:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.115+0000 7f257ea00640 1 -- 172.21.15.47:0/385030909 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f257809bf10 con 0x7f25881005c0 2024-03-21T11:23:43.115 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:23:43.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.117+0000 7f258d000640 1 -- 172.21.15.47:0/385030909 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f256c0a3030 msgr2=0x7f256c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:43.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.117+0000 7f258d000640 1 --2- 172.21.15.47:0/385030909 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f256c0a3030 0x7f256c0a54f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f257402c250 tx=0x7f257402cf90 comp rx=0 tx=0).stop 2024-03-21T11:23:43.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.117+0000 7f258d000640 1 -- 172.21.15.47:0/385030909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25881005c0 msgr2=0x7f2588104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:43.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.117+0000 7f258d000640 1 --2- 172.21.15.47:0/385030909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25881005c0 0x7f2588104550 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f257800e9c0 tx=0x7f257800ee80 comp rx=0 tx=0).stop 2024-03-21T11:23:43.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.118+0000 7f258d000640 1 -- 172.21.15.47:0/385030909 shutdown_connections 2024-03-21T11:23:43.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.118+0000 7f258d000640 1 --2- 172.21.15.47:0/385030909 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f256c0a3030 0x7f256c0a54f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.118+0000 7f258d000640 1 --2- 172.21.15.47:0/385030909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2588102170 0x7f25881030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.118+0000 7f258d000640 1 --2- 172.21.15.47:0/385030909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25881017b0 0x7f2588102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.118+0000 7f258d000640 1 --2- 172.21.15.47:0/385030909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25881005c0 0x7f2588104550 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.118+0000 7f258d000640 1 -- 172.21.15.47:0/385030909 >> 172.21.15.47:0/385030909 conn(0x7f25880fbb50 msgr2=0x7f25880fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:43.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.118+0000 7f258d000640 1 -- 172.21.15.47:0/385030909 shutdown_connections 2024-03-21T11:23:43.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.118+0000 7f258d000640 1 -- 172.21.15.47:0/385030909 wait complete. 2024-03-21T11:23:43.137 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T11:23:43.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:42 smithi100 ceph-mon[36245]: pgmap v411: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-21T11:23:43.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:42 smithi100 ceph-mon[36245]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-21T11:23:43.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:42 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/4061324769' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:43.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:42 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/4117764616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:42 smithi102 ceph-mon[36748]: pgmap v411: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-21T11:23:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:42 smithi102 ceph-mon[36748]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-21T11:23:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:42 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/4061324769' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:42 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/4117764616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:43.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.367+0000 7efe89200640 1 -- 172.21.15.47:0/2412798936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe84105370 msgr2=0x7efe8410bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:43.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.367+0000 7efe89200640 1 --2- 172.21.15.47:0/2412798936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe84105370 0x7efe8410bc00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009860 tx=0x7efe7c02bb70 comp rx=0 tx=0).stop 2024-03-21T11:23:43.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.367+0000 7efe89200640 1 -- 172.21.15.47:0/2412798936 shutdown_connections 2024-03-21T11:23:43.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.367+0000 7efe89200640 1 --2- 172.21.15.47:0/2412798936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe84105370 0x7efe8410bc00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.367+0000 7efe89200640 1 --2- 172.21.15.47:0/2412798936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe841049b0 0x7efe84104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.367+0000 7efe89200640 1 --2- 172.21.15.47:0/2412798936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe841037c0 0x7efe84103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.367+0000 7efe89200640 1 -- 172.21.15.47:0/2412798936 >> 172.21.15.47:0/2412798936 conn(0x7efe840fed50 msgr2=0x7efe84101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:43.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.367+0000 7efe89200640 1 -- 172.21.15.47:0/2412798936 shutdown_connections 2024-03-21T11:23:43.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.367+0000 7efe89200640 1 -- 172.21.15.47:0/2412798936 wait complete. 2024-03-21T11:23:43.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.368+0000 7efe89200640 1 Processor -- start 2024-03-21T11:23:43.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.368+0000 7efe89200640 1 -- start start 2024-03-21T11:23:43.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.368+0000 7efe89200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe841037c0 0x7efe84199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:43.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.368+0000 7efe89200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe841049b0 0x7efe84199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:43.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.368+0000 7efe89200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe84105370 0x7efe841a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:43.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.368+0000 7efe89200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8410f440 con 0x7efe841049b0 2024-03-21T11:23:43.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.368+0000 7efe89200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8410f2c0 con 0x7efe841037c0 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.368+0000 7efe89200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8410f5c0 con 0x7efe84105370 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.368+0000 7efe7aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe841049b0 0x7efe84199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.368+0000 7efe83400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe841037c0 0x7efe84199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.369+0000 7efe7aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe841049b0 0x7efe84199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59934/0 (socket says 172.21.15.47:59934) 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.369+0000 7efe7aa00640 1 -- 172.21.15.47:0/2970551389 learned_addr learned my addr 172.21.15.47:0/2970551389 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.369+0000 7efe83e00640 1 --2- 172.21.15.47:0/2970551389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe84105370 0x7efe841a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.369+0000 7efe83400640 1 -- 172.21.15.47:0/2970551389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe84105370 msgr2=0x7efe841a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.369+0000 7efe83400640 1 --2- 172.21.15.47:0/2970551389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe84105370 0x7efe841a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.369+0000 7efe83400640 1 -- 172.21.15.47:0/2970551389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe841049b0 msgr2=0x7efe84199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.369+0000 7efe83400640 1 --2- 172.21.15.47:0/2970551389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe841049b0 0x7efe84199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.369+0000 7efe83400640 1 -- 172.21.15.47:0/2970551389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe841a1640 con 0x7efe841037c0 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.369+0000 7efe83400640 1 --2- 172.21.15.47:0/2970551389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe841037c0 0x7efe84199a20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efe74005fd0 tx=0x7efe7400ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.369+0000 7efe80c00640 1 -- 172.21.15.47:0/2970551389 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7400ccc0 con 0x7efe841037c0 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.369+0000 7efe80c00640 1 -- 172.21.15.47:0/2970551389 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe7400ee70 con 0x7efe841037c0 2024-03-21T11:23:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.369+0000 7efe80c00640 1 -- 172.21.15.47:0/2970551389 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74010430 con 0x7efe841037c0 2024-03-21T11:23:43.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.370+0000 7efe89200640 1 -- 172.21.15.47:0/2970551389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe841a1930 con 0x7efe841037c0 2024-03-21T11:23:43.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.370+0000 7efe89200640 1 -- 172.21.15.47:0/2970551389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe841a1ed0 con 0x7efe841037c0 2024-03-21T11:23:43.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.371+0000 7efe89200640 1 -- 172.21.15.47:0/2970551389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe8410aae0 con 0x7efe841037c0 2024-03-21T11:23:43.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.372+0000 7efe80c00640 1 -- 172.21.15.47:0/2970551389 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7efe74003ef0 con 0x7efe841037c0 2024-03-21T11:23:43.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.374+0000 7efe80c00640 1 --2- 172.21.15.47:0/2970551389 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe680a3030 0x7efe680a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:43.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.374+0000 7efe80c00640 1 -- 172.21.15.47:0/2970551389 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efe740cfae0 con 0x7efe841037c0 2024-03-21T11:23:43.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.374+0000 7efe7aa00640 1 --2- 172.21.15.47:0/2970551389 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe680a3030 0x7efe680a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:43.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.375+0000 7efe7aa00640 1 --2- 172.21.15.47:0/2970551389 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe680a3030 0x7efe680a54f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efe8419af30 tx=0x7efe6c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:43.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.378+0000 7efe80c00640 1 -- 172.21.15.47:0/2970551389 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efe7409c420 con 0x7efe841037c0 2024-03-21T11:23:43.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.581+0000 7efe89200640 1 -- 172.21.15.47:0/2970551389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe840622a0 con 0x7efe841037c0 2024-03-21T11:23:43.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.582+0000 7efe80c00640 1 -- 172.21.15.47:0/2970551389 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe7409bd20 con 0x7efe841037c0 2024-03-21T11:23:43.583 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T11:23:43.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.585+0000 7efe89200640 1 -- 172.21.15.47:0/2970551389 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe680a3030 msgr2=0x7efe680a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:43.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.585+0000 7efe89200640 1 --2- 172.21.15.47:0/2970551389 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe680a3030 0x7efe680a54f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efe8419af30 tx=0x7efe6c009500 comp rx=0 tx=0).stop 2024-03-21T11:23:43.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.585+0000 7efe89200640 1 -- 172.21.15.47:0/2970551389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe841037c0 msgr2=0x7efe84199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:43.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.585+0000 7efe89200640 1 --2- 172.21.15.47:0/2970551389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe841037c0 0x7efe84199a20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efe74005fd0 tx=0x7efe7400ec20 comp rx=0 tx=0).stop 2024-03-21T11:23:43.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.585+0000 7efe89200640 1 -- 172.21.15.47:0/2970551389 shutdown_connections 2024-03-21T11:23:43.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.585+0000 7efe89200640 1 --2- 172.21.15.47:0/2970551389 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe680a3030 0x7efe680a54f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.585+0000 7efe89200640 1 --2- 172.21.15.47:0/2970551389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe84105370 0x7efe841a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.585+0000 7efe89200640 1 --2- 172.21.15.47:0/2970551389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe841049b0 0x7efe84199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.585+0000 7efe89200640 1 --2- 172.21.15.47:0/2970551389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe841037c0 0x7efe84199a20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.585+0000 7efe89200640 1 -- 172.21.15.47:0/2970551389 >> 172.21.15.47:0/2970551389 conn(0x7efe840fed50 msgr2=0x7efe84107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:43.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.585+0000 7efe89200640 1 -- 172.21.15.47:0/2970551389 shutdown_connections 2024-03-21T11:23:43.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.585+0000 7efe89200640 1 -- 172.21.15.47:0/2970551389 wait complete. 2024-03-21T11:23:43.606 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T11:23:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.833+0000 7f2c16c00640 1 -- 172.21.15.47:0/2854627735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c10074100 msgr2=0x7f2c10074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.833+0000 7f2c16c00640 1 --2- 172.21.15.47:0/2854627735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c10074100 0x7f2c10074580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009900 tx=0x7f2bfc02bc40 comp rx=0 tx=0).stop 2024-03-21T11:23:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.833+0000 7f2c16c00640 1 -- 172.21.15.47:0/2854627735 shutdown_connections 2024-03-21T11:23:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.833+0000 7f2c16c00640 1 --2- 172.21.15.47:0/2854627735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c10109460 0x7f2c1010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.833+0000 7f2c16c00640 1 --2- 172.21.15.47:0/2854627735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c10074100 0x7f2c10074580 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.833+0000 7f2c16c00640 1 --2- 172.21.15.47:0/2854627735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c100756e0 0x7f2c10073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.833+0000 7f2c16c00640 1 -- 172.21.15.47:0/2854627735 >> 172.21.15.47:0/2854627735 conn(0x7f2c100fcb10 msgr2=0x7f2c100fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:43.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.833+0000 7f2c16c00640 1 -- 172.21.15.47:0/2854627735 shutdown_connections 2024-03-21T11:23:43.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.833+0000 7f2c16c00640 1 -- 172.21.15.47:0/2854627735 wait complete. 2024-03-21T11:23:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.834+0000 7f2c16c00640 1 Processor -- start 2024-03-21T11:23:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.835+0000 7f2c16c00640 1 -- start start 2024-03-21T11:23:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.835+0000 7f2c16c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c10074100 0x7f2c10199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.835+0000 7f2c16c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c100756e0 0x7f2c10199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.835+0000 7f2c16c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c10109460 0x7f2c101a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.835+0000 7f2c16c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1010f450 con 0x7f2c100756e0 2024-03-21T11:23:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.835+0000 7f2c16c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1010f2d0 con 0x7f2c10109460 2024-03-21T11:23:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.835+0000 7f2c16c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1010f5d0 con 0x7f2c10074100 2024-03-21T11:23:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.835+0000 7f2c15800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c10074100 0x7f2c10199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.835+0000 7f2c15800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c10074100 0x7f2c10199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51916/0 (socket says 172.21.15.47:51916) 2024-03-21T11:23:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.835+0000 7f2c14e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c100756e0 0x7f2c10199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.835+0000 7f2c15800640 1 -- 172.21.15.47:0/873403839 learned_addr learned my addr 172.21.15.47:0/873403839 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.835+0000 7f2c16200640 1 --2- 172.21.15.47:0/873403839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c10109460 0x7f2c101a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.836+0000 7f2c15800640 1 -- 172.21.15.47:0/873403839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c10109460 msgr2=0x7f2c101a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.836+0000 7f2c15800640 1 --2- 172.21.15.47:0/873403839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c10109460 0x7f2c101a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.836+0000 7f2c15800640 1 -- 172.21.15.47:0/873403839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c100756e0 msgr2=0x7f2c10199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.836+0000 7f2c15800640 1 --2- 172.21.15.47:0/873403839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c100756e0 0x7f2c10199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.836+0000 7f2c15800640 1 -- 172.21.15.47:0/873403839 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c101a16c0 con 0x7f2c10074100 2024-03-21T11:23:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.836+0000 7f2c15800640 1 --2- 172.21.15.47:0/873403839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c10074100 0x7f2c10199aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400d950 tx=0x7f2c0400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.837+0000 7f2c0e000640 1 -- 172.21.15.47:0/873403839 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04017030 con 0x7f2c10074100 2024-03-21T11:23:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.837+0000 7f2c0e000640 1 -- 172.21.15.47:0/873403839 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c04004020 con 0x7f2c10074100 2024-03-21T11:23:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.837+0000 7f2c0e000640 1 -- 172.21.15.47:0/873403839 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c040047a0 con 0x7f2c10074100 2024-03-21T11:23:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.837+0000 7f2c16c00640 1 -- 172.21.15.47:0/873403839 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c101a19b0 con 0x7f2c10074100 2024-03-21T11:23:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.837+0000 7f2c16c00640 1 -- 172.21.15.47:0/873403839 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c101a1f20 con 0x7f2c10074100 2024-03-21T11:23:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.838+0000 7f2c16c00640 1 -- 172.21.15.47:0/873403839 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c10108190 con 0x7f2c10074100 2024-03-21T11:23:43.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.840+0000 7f2c0e000640 1 -- 172.21.15.47:0/873403839 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f2c040041d0 con 0x7f2c10074100 2024-03-21T11:23:43.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.841+0000 7f2c0e000640 1 --2- 172.21.15.47:0/873403839 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2bf80a3030 0x7f2bf80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:43.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.841+0000 7f2c0e000640 1 -- 172.21.15.47:0/873403839 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2c040d11a0 con 0x7f2c10074100 2024-03-21T11:23:43.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.842+0000 7f2c14e00640 1 --2- 172.21.15.47:0/873403839 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2bf80a3030 0x7f2bf80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:43.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.842+0000 7f2c14e00640 1 --2- 172.21.15.47:0/873403839 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2bf80a3030 0x7f2bf80a54f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc02c250 tx=0x7f2bfc02cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:43.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:43.849+0000 7f2c0e000640 1 -- 172.21.15.47:0/873403839 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2c0409bbc0 con 0x7f2c10074100 2024-03-21T11:23:44.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.051+0000 7f2c16c00640 1 -- 172.21.15.47:0/873403839 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2c1002d320 con 0x7f2c10074100 2024-03-21T11:23:44.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.052+0000 7f2c0e000640 1 -- 172.21.15.47:0/873403839 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2c0409b4c0 con 0x7f2c10074100 2024-03-21T11:23:44.053 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:23:44.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.055+0000 7f2c16c00640 1 -- 172.21.15.47:0/873403839 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2bf80a3030 msgr2=0x7f2bf80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:44.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.055+0000 7f2c16c00640 1 --2- 172.21.15.47:0/873403839 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2bf80a3030 0x7f2bf80a54f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc02c250 tx=0x7f2bfc02cf90 comp rx=0 tx=0).stop 2024-03-21T11:23:44.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.055+0000 7f2c16c00640 1 -- 172.21.15.47:0/873403839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c10074100 msgr2=0x7f2c10199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:44.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.055+0000 7f2c16c00640 1 --2- 172.21.15.47:0/873403839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c10074100 0x7f2c10199aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400d950 tx=0x7f2c0400de10 comp rx=0 tx=0).stop 2024-03-21T11:23:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.055+0000 7f2c16c00640 1 -- 172.21.15.47:0/873403839 shutdown_connections 2024-03-21T11:23:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.055+0000 7f2c16c00640 1 --2- 172.21.15.47:0/873403839 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2bf80a3030 0x7f2bf80a54f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.055+0000 7f2c16c00640 1 --2- 172.21.15.47:0/873403839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c10109460 0x7f2c101a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.055+0000 7f2c16c00640 1 --2- 172.21.15.47:0/873403839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c100756e0 0x7f2c10199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.055+0000 7f2c16c00640 1 --2- 172.21.15.47:0/873403839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c10074100 0x7f2c10199aa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.055+0000 7f2c16c00640 1 -- 172.21.15.47:0/873403839 >> 172.21.15.47:0/873403839 conn(0x7f2c100fcb10 msgr2=0x7f2c10105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.055+0000 7f2c16c00640 1 -- 172.21.15.47:0/873403839 shutdown_connections 2024-03-21T11:23:44.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.055+0000 7f2c16c00640 1 -- 172.21.15.47:0/873403839 wait complete. 2024-03-21T11:23:44.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:43 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/385030909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:44.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:43 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2970551389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:44.077 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-21T11:23:44.077 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:23:44.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:43 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/385030909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:44.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:43 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2970551389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:44.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:43 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/385030909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:44.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:43 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2970551389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.302+0000 7fd783e00640 1 -- 172.21.15.47:0/1779429428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd784107b00 msgr2=0x7fd784107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.302+0000 7fd783e00640 1 --2- 172.21.15.47:0/1779429428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd784107b00 0x7fd784107f20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd774009900 tx=0x7fd77402bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.302+0000 7fd783e00640 1 -- 172.21.15.47:0/1779429428 shutdown_connections 2024-03-21T11:23:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.302+0000 7fd783e00640 1 --2- 172.21.15.47:0/1779429428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7841096b0 0x7fd78410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.302+0000 7fd783e00640 1 --2- 172.21.15.47:0/1779429428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd784108cf0 0x7fd784109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.302+0000 7fd783e00640 1 --2- 172.21.15.47:0/1779429428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd784107b00 0x7fd784107f20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.302+0000 7fd783e00640 1 -- 172.21.15.47:0/1779429428 >> 172.21.15.47:0/1779429428 conn(0x7fd784076ab0 msgr2=0x7fd784078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.302+0000 7fd783e00640 1 -- 172.21.15.47:0/1779429428 shutdown_connections 2024-03-21T11:23:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.302+0000 7fd783e00640 1 -- 172.21.15.47:0/1779429428 wait complete. 2024-03-21T11:23:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.303+0000 7fd783e00640 1 Processor -- start 2024-03-21T11:23:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.303+0000 7fd783e00640 1 -- start start 2024-03-21T11:23:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd783e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd784107b00 0x7fd78419ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd783e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd784108cf0 0x7fd78419e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd783e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7841096b0 0x7fd7841a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd783e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7841137a0 con 0x7fd784108cf0 2024-03-21T11:23:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd783e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd784113620 con 0x7fd7841096b0 2024-03-21T11:23:44.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd783e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd784113920 con 0x7fd784107b00 2024-03-21T11:23:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd782a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd784107b00 0x7fd78419ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd782a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd784107b00 0x7fd78419ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51922/0 (socket says 172.21.15.47:51922) 2024-03-21T11:23:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd782a00640 1 -- 172.21.15.47:0/961467 learned_addr learned my addr 172.21.15.47:0/961467 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd783400640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7841096b0 0x7fd7841a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd782000640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd784108cf0 0x7fd78419e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd782a00640 1 -- 172.21.15.47:0/961467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7841096b0 msgr2=0x7fd7841a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd782a00640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7841096b0 0x7fd7841a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd782a00640 1 -- 172.21.15.47:0/961467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd784108cf0 msgr2=0x7fd78419e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd782a00640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd784108cf0 0x7fd78419e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd782a00640 1 -- 172.21.15.47:0/961467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7841a5a10 con 0x7fd784107b00 2024-03-21T11:23:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd783400640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7841096b0 0x7fd7841a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.304+0000 7fd782000640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd784108cf0 0x7fd78419e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.305+0000 7fd782a00640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd784107b00 0x7fd78419ddf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd774000c00 tx=0x7fd77402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.305+0000 7fd788a00640 1 -- 172.21.15.47:0/961467 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774043020 con 0x7fd784107b00 2024-03-21T11:23:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.305+0000 7fd788a00640 1 -- 172.21.15.47:0/961467 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7740054e0 con 0x7fd784107b00 2024-03-21T11:23:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.305+0000 7fd788a00640 1 -- 172.21.15.47:0/961467 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774003c40 con 0x7fd784107b00 2024-03-21T11:23:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.305+0000 7fd783e00640 1 -- 172.21.15.47:0/961467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7841a5ca0 con 0x7fd784107b00 2024-03-21T11:23:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.305+0000 7fd783e00640 1 -- 172.21.15.47:0/961467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7841a61a0 con 0x7fd784107b00 2024-03-21T11:23:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.306+0000 7fd783e00640 1 -- 172.21.15.47:0/961467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78410ee20 con 0x7fd784107b00 2024-03-21T11:23:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.308+0000 7fd788a00640 1 -- 172.21.15.47:0/961467 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fd774046020 con 0x7fd784107b00 2024-03-21T11:23:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.310+0000 7fd788a00640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7680a2fb0 0x7fd7680a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.310+0000 7fd788a00640 1 -- 172.21.15.47:0/961467 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd7740f1940 con 0x7fd784107b00 2024-03-21T11:23:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.310+0000 7fd782000640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7680a2fb0 0x7fd7680a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.311+0000 7fd782000640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7680a2fb0 0x7fd7680a5470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd78419f300 tx=0x7fd770005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:44.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.318+0000 7fd788a00640 1 -- 172.21.15.47:0/961467 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd77404d020 con 0x7fd784107b00 2024-03-21T11:23:44.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.566+0000 7fd783e00640 1 -- 172.21.15.47:0/961467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd78402d290 con 0x7fd784107b00 2024-03-21T11:23:44.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.567+0000 7fd788a00640 1 -- 172.21.15.47:0/961467 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fd7740be330 con 0x7fd784107b00 2024-03-21T11:23:44.568 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:44.569 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:23:44.569 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:23:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.570+0000 7fd783e00640 1 -- 172.21.15.47:0/961467 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7680a2fb0 msgr2=0x7fd7680a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.570+0000 7fd783e00640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7680a2fb0 0x7fd7680a5470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd78419f300 tx=0x7fd770005f90 comp rx=0 tx=0).stop 2024-03-21T11:23:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.570+0000 7fd783e00640 1 -- 172.21.15.47:0/961467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd784107b00 msgr2=0x7fd78419ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.570+0000 7fd783e00640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd784107b00 0x7fd78419ddf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd774000c00 tx=0x7fd77402c650 comp rx=0 tx=0).stop 2024-03-21T11:23:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.570+0000 7fd783e00640 1 -- 172.21.15.47:0/961467 shutdown_connections 2024-03-21T11:23:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.570+0000 7fd783e00640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7680a2fb0 0x7fd7680a5470 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.570+0000 7fd783e00640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7841096b0 0x7fd7841a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.570+0000 7fd783e00640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd784108cf0 0x7fd78419e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.570+0000 7fd783e00640 1 --2- 172.21.15.47:0/961467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd784107b00 0x7fd78419ddf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.570+0000 7fd783e00640 1 -- 172.21.15.47:0/961467 >> 172.21.15.47:0/961467 conn(0x7fd784076ab0 msgr2=0x7fd78410bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.571+0000 7fd783e00640 1 -- 172.21.15.47:0/961467 shutdown_connections 2024-03-21T11:23:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.571+0000 7fd783e00640 1 -- 172.21.15.47:0/961467 wait complete. 2024-03-21T11:23:44.591 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-21T11:23:44.592 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-21T11:23:44.592 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T11:23:44.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.812+0000 7f346de00640 1 -- 172.21.15.47:0/282450415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34681036f0 msgr2=0x7f3468103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:44.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.812+0000 7f346de00640 1 --2- 172.21.15.47:0/282450415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34681036f0 0x7f3468103b10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f345c009900 tx=0x7f345c02bc50 comp rx=0 tx=0).stop 2024-03-21T11:23:44.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.812+0000 7f346de00640 1 -- 172.21.15.47:0/282450415 shutdown_connections 2024-03-21T11:23:44.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.812+0000 7f346de00640 1 --2- 172.21.15.47:0/282450415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34681052a0 0x7f346810bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.812+0000 7f346de00640 1 --2- 172.21.15.47:0/282450415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34681048e0 0x7f3468104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.812+0000 7f346de00640 1 --2- 172.21.15.47:0/282450415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34681036f0 0x7f3468103b10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.812+0000 7f346de00640 1 -- 172.21.15.47:0/282450415 >> 172.21.15.47:0/282450415 conn(0x7f34680fec80 msgr2=0x7f34681010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:44.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.812+0000 7f346de00640 1 -- 172.21.15.47:0/282450415 shutdown_connections 2024-03-21T11:23:44.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.812+0000 7f346de00640 1 -- 172.21.15.47:0/282450415 wait complete. 2024-03-21T11:23:44.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.813+0000 7f346de00640 1 Processor -- start 2024-03-21T11:23:44.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.814+0000 7f346de00640 1 -- start start 2024-03-21T11:23:44.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.814+0000 7f346de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34681036f0 0x7f3468199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:44.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.814+0000 7f346de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34681048e0 0x7f346819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:44.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.814+0000 7f346de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34681052a0 0x7f34681a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.814+0000 7f346de00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346810f360 con 0x7f34681052a0 2024-03-21T11:23:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.814+0000 7f346de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346810f1e0 con 0x7f34681048e0 2024-03-21T11:23:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.814+0000 7f346de00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346810f4e0 con 0x7f34681036f0 2024-03-21T11:23:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.814+0000 7f3467e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34681048e0 0x7f346819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.814+0000 7f3467e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34681048e0 0x7f346819a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38702/0 (socket says 172.21.15.47:38702) 2024-03-21T11:23:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.814+0000 7f3467e00640 1 -- 172.21.15.47:0/636866164 learned_addr learned my addr 172.21.15.47:0/636866164 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.814+0000 7f346ca00640 1 --2- 172.21.15.47:0/636866164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34681036f0 0x7f3468199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.815+0000 7f346d400640 1 --2- 172.21.15.47:0/636866164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34681052a0 0x7f34681a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.815+0000 7f3467e00640 1 -- 172.21.15.47:0/636866164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34681036f0 msgr2=0x7f3468199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.815+0000 7f3467e00640 1 --2- 172.21.15.47:0/636866164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34681036f0 0x7f3468199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.815+0000 7f3467e00640 1 -- 172.21.15.47:0/636866164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34681052a0 msgr2=0x7f34681a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.815+0000 7f3467e00640 1 --2- 172.21.15.47:0/636866164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34681052a0 0x7f34681a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.815+0000 7f3467e00640 1 -- 172.21.15.47:0/636866164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34681a1700 con 0x7f34681048e0 2024-03-21T11:23:44.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.815+0000 7f3467e00640 1 --2- 172.21.15.47:0/636866164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34681048e0 0x7f346819a020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f345400e9f0 tx=0x7f345400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:44.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.815+0000 7f3465600640 1 -- 172.21.15.47:0/636866164 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345400cd80 con 0x7f34681048e0 2024-03-21T11:23:44.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.815+0000 7f3465600640 1 -- 172.21.15.47:0/636866164 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3454005500 con 0x7f34681048e0 2024-03-21T11:23:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.815+0000 7f346de00640 1 -- 172.21.15.47:0/636866164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34681a19f0 con 0x7f34681048e0 2024-03-21T11:23:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.815+0000 7f3465600640 1 -- 172.21.15.47:0/636866164 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454010690 con 0x7f34681048e0 2024-03-21T11:23:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.816+0000 7f346de00640 1 -- 172.21.15.47:0/636866164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34681a1ef0 con 0x7f34681048e0 2024-03-21T11:23:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.817+0000 7f346de00640 1 -- 172.21.15.47:0/636866164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346810aa90 con 0x7f34681048e0 2024-03-21T11:23:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.820+0000 7f3465600640 1 -- 172.21.15.47:0/636866164 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f34540046c0 con 0x7f34681048e0 2024-03-21T11:23:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.821+0000 7f3465600640 1 --2- 172.21.15.47:0/636866164 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f34500a2fb0 0x7f34500a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.821+0000 7f3465600640 1 -- 172.21.15.47:0/636866164 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f34540cfdd0 con 0x7f34681048e0 2024-03-21T11:23:44.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.821+0000 7f346ca00640 1 --2- 172.21.15.47:0/636866164 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f34500a2fb0 0x7f34500a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:44.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.822+0000 7f346ca00640 1 --2- 172.21.15.47:0/636866164 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f34500a2fb0 0x7f34500a5470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f345c000c00 tx=0x7f345c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:44.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:44.823+0000 7f3465600640 1 -- 172.21.15.47:0/636866164 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f345409c6b0 con 0x7f34681048e0 2024-03-21T11:23:45.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.023+0000 7f346de00640 1 -- 172.21.15.47:0/636866164 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3468061ae0 con 0x7f34500a2fb0 2024-03-21T11:23:45.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.039+0000 7f3465600640 1 -- 172.21.15.47:0/636866164 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340844 (secure 0 0 0) 0x7f3468061ae0 con 0x7f34500a2fb0 2024-03-21T11:23:45.046 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-21T11:23:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.048+0000 7f346de00640 1 -- 172.21.15.47:0/636866164 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f34500a2fb0 msgr2=0x7f34500a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.048+0000 7f346de00640 1 --2- 172.21.15.47:0/636866164 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f34500a2fb0 0x7f34500a5470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f345c000c00 tx=0x7f345c005fd0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.048+0000 7f346de00640 1 -- 172.21.15.47:0/636866164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34681048e0 msgr2=0x7f346819a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.048+0000 7f346de00640 1 --2- 172.21.15.47:0/636866164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34681048e0 0x7f346819a020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f345400e9f0 tx=0x7f345400edf0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.048+0000 7f346de00640 1 -- 172.21.15.47:0/636866164 shutdown_connections 2024-03-21T11:23:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.048+0000 7f346de00640 1 --2- 172.21.15.47:0/636866164 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f34500a2fb0 0x7f34500a5470 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.048+0000 7f346de00640 1 --2- 172.21.15.47:0/636866164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34681052a0 0x7f34681a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.048+0000 7f346de00640 1 --2- 172.21.15.47:0/636866164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34681048e0 0x7f346819a020 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.048+0000 7f346de00640 1 --2- 172.21.15.47:0/636866164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34681036f0 0x7f3468199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.048+0000 7f346de00640 1 -- 172.21.15.47:0/636866164 >> 172.21.15.47:0/636866164 conn(0x7f34680fec80 msgr2=0x7f3468107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:45.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.048+0000 7f346de00640 1 -- 172.21.15.47:0/636866164 shutdown_connections 2024-03-21T11:23:45.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.048+0000 7f346de00640 1 -- 172.21.15.47:0/636866164 wait complete. 2024-03-21T11:23:45.068 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-03-21T11:23:43.059666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669842,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1602,"num_read_kb":933,"num_write":450,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378664,"kb_used_data":12840,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563160,"statfs":{"total":1151940427776,"available":1151552675840,"internally_reserved":0,"allocated":13148160,"data_stored":8349712,"data_compressed":435654,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6767,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":69,"num_write":13,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004464"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560653+0000","last_change":"2024-03-21T11:20:48.667185+0000","last_active":"2024-03-21T11:22:51.560653+0000","last_peered":"2024-03-21T11:22:51.560653+0000","last_clean":"2024-03-21T11:22:51.560653+0000","last_became_active":"2024-03-21T11:20:48.666855+0000","last_became_peered":"2024-03-21T11:20:48.666855+0000","last_unstale":"2024-03-21T11:22:51.560653+0000","last_undegraded":"2024-03-21T11:22:51.560653+0000","last_fullsized":"2024-03-21T11:22:51.560653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:30:37.393096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855276+0000","last_change":"2024-03-21T11:20:48.671613+0000","last_active":"2024-03-21T11:22:50.855276+0000","last_peered":"2024-03-21T11:22:50.855276+0000","last_clean":"2024-03-21T11:22:50.855276+0000","last_became_active":"2024-03-21T11:20:48.671211+0000","last_became_peered":"2024-03-21T11:20:48.671211+0000","last_unstale":"2024-03-21T11:22:50.855276+0000","last_undegraded":"2024-03-21T11:22:50.855276+0000","last_fullsized":"2024-03-21T11:22:50.855276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:20.167922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.854836+0000","last_change":"2024-03-21T11:20:48.655734+0000","last_active":"2024-03-21T11:22:50.854836+0000","last_peered":"2024-03-21T11:22:50.854836+0000","last_clean":"2024-03-21T11:22:50.854836+0000","last_became_active":"2024-03-21T11:20:48.653798+0000","last_became_peered":"2024-03-21T11:20:48.653798+0000","last_unstale":"2024-03-21T11:22:50.854836+0000","last_undegraded":"2024-03-21T11:22:50.854836+0000","last_fullsized":"2024-03-21T11:22:50.854836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:44.796405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847015+0000","last_change":"2024-03-21T11:20:48.648784+0000","last_active":"2024-03-21T11:22:50.847015+0000","last_peered":"2024-03-21T11:22:50.847015+0000","last_clean":"2024-03-21T11:22:50.847015+0000","last_became_active":"2024-03-21T11:20:48.648040+0000","last_became_peered":"2024-03-21T11:20:48.648040+0000","last_unstale":"2024-03-21T11:22:50.847015+0000","last_undegraded":"2024-03-21T11:22:50.847015+0000","last_fullsized":"2024-03-21T11:22:50.847015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:48:51.786346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846582+0000","last_change":"2024-03-21T11:20:48.672084+0000","last_active":"2024-03-21T11:22:50.846582+0000","last_peered":"2024-03-21T11:22:50.846582+0000","last_clean":"2024-03-21T11:22:50.846582+0000","last_became_active":"2024-03-21T11:20:48.671441+0000","last_became_peered":"2024-03-21T11:20:48.671441+0000","last_unstale":"2024-03-21T11:22:50.846582+0000","last_undegraded":"2024-03-21T11:22:50.846582+0000","last_fullsized":"2024-03-21T11:22:50.846582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:20:17.638270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848462+0000","last_change":"2024-03-21T11:20:48.659782+0000","last_active":"2024-03-21T11:22:50.848462+0000","last_peered":"2024-03-21T11:22:50.848462+0000","last_clean":"2024-03-21T11:22:50.848462+0000","last_became_active":"2024-03-21T11:20:48.659460+0000","last_became_peered":"2024-03-21T11:20:48.659460+0000","last_unstale":"2024-03-21T11:22:50.848462+0000","last_undegraded":"2024-03-21T11:22:50.848462+0000","last_fullsized":"2024-03-21T11:22:50.848462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:57.712654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805934+0000","last_change":"2024-03-21T11:20:48.659401+0000","last_active":"2024-03-21T11:22:51.805934+0000","last_peered":"2024-03-21T11:22:51.805934+0000","last_clean":"2024-03-21T11:22:51.805934+0000","last_became_active":"2024-03-21T11:20:48.659075+0000","last_became_peered":"2024-03-21T11:20:48.659075+0000","last_unstale":"2024-03-21T11:22:51.805934+0000","last_undegraded":"2024-03-21T11:22:51.805934+0000","last_fullsized":"2024-03-21T11:22:51.805934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:54.342236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847869+0000","last_change":"2024-03-21T11:20:48.667175+0000","last_active":"2024-03-21T11:22:50.847869+0000","last_peered":"2024-03-21T11:22:50.847869+0000","last_clean":"2024-03-21T11:22:50.847869+0000","last_became_active":"2024-03-21T11:20:48.666756+0000","last_became_peered":"2024-03-21T11:20:48.666756+0000","last_unstale":"2024-03-21T11:22:50.847869+0000","last_undegraded":"2024-03-21T11:22:50.847869+0000","last_fullsized":"2024-03-21T11:22:50.847869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:25:39.694699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129870+0000","last_change":"2024-03-21T11:20:48.650584+0000","last_active":"2024-03-21T11:22:51.129870+0000","last_peered":"2024-03-21T11:22:51.129870+0000","last_clean":"2024-03-21T11:22:51.129870+0000","last_became_active":"2024-03-21T11:20:48.650251+0000","last_became_peered":"2024-03-21T11:20:48.650251+0000","last_unstale":"2024-03-21T11:22:51.129870+0000","last_undegraded":"2024-03-21T11:22:51.129870+0000","last_fullsized":"2024-03-21T11:22:51.129870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:35:59.000522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848842+0000","last_change":"2024-03-21T11:20:48.652945+0000","last_active":"2024-03-21T11:22:50.848842+0000","last_peered":"2024-03-21T11:22:50.848842+0000","last_clean":"2024-03-21T11:22:50.848842+0000","last_became_active":"2024-03-21T11:20:48.652626+0000","last_became_peered":"2024-03-21T11:20:48.652626+0000","last_unstale":"2024-03-21T11:22:50.848842+0000","last_undegraded":"2024-03-21T11:22:50.848842+0000","last_fullsized":"2024-03-21T11:22:50.848842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:43.586907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806192+0000","last_change":"2024-03-21T11:20:48.651520+0000","last_active":"2024-03-21T11:22:51.806192+0000","last_peered":"2024-03-21T11:22:51.806192+0000","last_clean":"2024-03-21T11:22:51.806192+0000","last_became_active":"2024-03-21T11:20:48.651210+0000","last_became_peered":"2024-03-21T11:20:48.651210+0000","last_unstale":"2024-03-21T11:22:51.806192+0000","last_undegraded":"2024-03-21T11:22:51.806192+0000","last_fullsized":"2024-03-21T11:22:51.806192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:29.506732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846902+0000","last_change":"2024-03-21T11:20:48.649844+0000","last_active":"2024-03-21T11:22:50.846902+0000","last_peered":"2024-03-21T11:22:50.846902+0000","last_clean":"2024-03-21T11:22:50.846902+0000","last_became_active":"2024-03-21T11:20:48.649503+0000","last_became_peered":"2024-03-21T11:20:48.649503+0000","last_unstale":"2024-03-21T11:22:50.846902+0000","last_undegraded":"2024-03-21T11:22:50.846902+0000","last_fullsized":"2024-03-21T11:22:50.846902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:31:55.734583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755540+0000","last_change":"2024-03-21T11:20:48.662012+0000","last_active":"2024-03-21T11:22:51.755540+0000","last_peered":"2024-03-21T11:22:51.755540+0000","last_clean":"2024-03-21T11:22:51.755540+0000","last_became_active":"2024-03-21T11:20:48.661444+0000","last_became_peered":"2024-03-21T11:20:48.661444+0000","last_unstale":"2024-03-21T11:22:51.755540+0000","last_undegraded":"2024-03-21T11:22:51.755540+0000","last_fullsized":"2024-03-21T11:22:51.755540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:52.493244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.854380+0000","last_change":"2024-03-21T11:20:48.663087+0000","last_active":"2024-03-21T11:22:50.854380+0000","last_peered":"2024-03-21T11:22:50.854380+0000","last_clean":"2024-03-21T11:22:50.854380+0000","last_became_active":"2024-03-21T11:20:48.662039+0000","last_became_peered":"2024-03-21T11:20:48.662039+0000","last_unstale":"2024-03-21T11:22:50.854380+0000","last_undegraded":"2024-03-21T11:22:50.854380+0000","last_fullsized":"2024-03-21T11:22:50.854380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:47:01.962314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.854313+0000","last_change":"2024-03-21T11:20:48.663478+0000","last_active":"2024-03-21T11:22:50.854313+0000","last_peered":"2024-03-21T11:22:50.854313+0000","last_clean":"2024-03-21T11:22:50.854313+0000","last_became_active":"2024-03-21T11:20:48.663135+0000","last_became_peered":"2024-03-21T11:20:48.663135+0000","last_unstale":"2024-03-21T11:22:50.854313+0000","last_undegraded":"2024-03-21T11:22:50.854313+0000","last_fullsized":"2024-03-21T11:22:50.854313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:29.139760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129288+0000","last_change":"2024-03-21T11:20:48.653893+0000","last_active":"2024-03-21T11:22:51.129288+0000","last_peered":"2024-03-21T11:22:51.129288+0000","last_clean":"2024-03-21T11:22:51.129288+0000","last_became_active":"2024-03-21T11:20:48.653578+0000","last_became_peered":"2024-03-21T11:20:48.653578+0000","last_unstale":"2024-03-21T11:22:51.129288+0000","last_undegraded":"2024-03-21T11:22:51.129288+0000","last_fullsized":"2024-03-21T11:22:51.129288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:54:31.254030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560545+0000","last_change":"2024-03-21T11:20:48.667388+0000","last_active":"2024-03-21T11:22:51.560545+0000","last_peered":"2024-03-21T11:22:51.560545+0000","last_clean":"2024-03-21T11:22:51.560545+0000","last_became_active":"2024-03-21T11:20:48.667077+0000","last_became_peered":"2024-03-21T11:20:48.667077+0000","last_unstale":"2024-03-21T11:22:51.560545+0000","last_undegraded":"2024-03-21T11:22:51.560545+0000","last_fullsized":"2024-03-21T11:22:51.560545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:13:32.093457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853273+0000","last_change":"2024-03-21T11:20:48.658479+0000","last_active":"2024-03-21T11:22:50.853273+0000","last_peered":"2024-03-21T11:22:50.853273+0000","last_clean":"2024-03-21T11:22:50.853273+0000","last_became_active":"2024-03-21T11:20:48.658011+0000","last_became_peered":"2024-03-21T11:20:48.658011+0000","last_unstale":"2024-03-21T11:22:50.853273+0000","last_undegraded":"2024-03-21T11:22:50.853273+0000","last_fullsized":"2024-03-21T11:22:50.853273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:33.313537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853403+0000","last_change":"2024-03-21T11:20:48.668648+0000","last_active":"2024-03-21T11:22:50.853403+0000","last_peered":"2024-03-21T11:22:50.853403+0000","last_clean":"2024-03-21T11:22:50.853403+0000","last_became_active":"2024-03-21T11:20:48.668376+0000","last_became_peered":"2024-03-21T11:20:48.668376+0000","last_unstale":"2024-03-21T11:22:50.853403+0000","last_undegraded":"2024-03-21T11:22:50.853403+0000","last_fullsized":"2024-03-21T11:22:50.853403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:49:47.601724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852874+0000","last_change":"2024-03-21T11:20:53.755600+0000","last_active":"2024-03-21T11:22:50.852874+0000","last_peered":"2024-03-21T11:22:50.852874+0000","last_clean":"2024-03-21T11:22:50.852874+0000","last_became_active":"2024-03-21T11:20:53.755321+0000","last_became_peered":"2024-03-21T11:20:53.755321+0000","last_unstale":"2024-03-21T11:22:50.852874+0000","last_undegraded":"2024-03-21T11:22:50.852874+0000","last_fullsized":"2024-03-21T11:22:50.852874+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:04.905910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":193,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.907144+0000","last_change":"2024-03-21T11:20:48.161001+0000","last_active":"2024-03-21T11:23:40.907144+0000","last_peered":"2024-03-21T11:23:40.907144+0000","last_clean":"2024-03-21T11:23:40.907144+0000","last_became_active":"2024-03-21T11:20:48.153053+0000","last_became_peered":"2024-03-21T11:20:48.153053+0000","last_unstale":"2024-03-21T11:23:40.907144+0000","last_undegraded":"2024-03-21T11:23:40.907144+0000","last_fullsized":"2024-03-21T11:23:40.907144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:26.010581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849345+0000","last_change":"2024-03-21T11:20:48.671430+0000","last_active":"2024-03-21T11:22:50.849345+0000","last_peered":"2024-03-21T11:22:50.849345+0000","last_clean":"2024-03-21T11:22:50.849345+0000","last_became_active":"2024-03-21T11:20:48.670192+0000","last_became_peered":"2024-03-21T11:20:48.670192+0000","last_unstale":"2024-03-21T11:22:50.849345+0000","last_undegraded":"2024-03-21T11:22:50.849345+0000","last_fullsized":"2024-03-21T11:22:50.849345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:00:53.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846554+0000","last_change":"2024-03-21T11:20:53.750313+0000","last_active":"2024-03-21T11:22:50.846554+0000","last_peered":"2024-03-21T11:22:50.846554+0000","last_clean":"2024-03-21T11:22:50.846554+0000","last_became_active":"2024-03-21T11:20:53.749913+0000","last_became_peered":"2024-03-21T11:20:53.749913+0000","last_unstale":"2024-03-21T11:22:50.846554+0000","last_undegraded":"2024-03-21T11:22:50.846554+0000","last_fullsized":"2024-03-21T11:22:50.846554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:39:09.007674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849723+0000","last_change":"2024-03-21T11:20:48.159934+0000","last_active":"2024-03-21T11:22:50.849723+0000","last_peered":"2024-03-21T11:22:50.849723+0000","last_clean":"2024-03-21T11:22:50.849723+0000","last_became_active":"2024-03-21T11:20:48.159650+0000","last_became_peered":"2024-03-21T11:20:48.159650+0000","last_unstale":"2024-03-21T11:22:50.849723+0000","last_undegraded":"2024-03-21T11:22:50.849723+0000","last_fullsized":"2024-03-21T11:22:50.849723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:49:56.891733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855277+0000","last_change":"2024-03-21T11:20:48.673012+0000","last_active":"2024-03-21T11:22:50.855277+0000","last_peered":"2024-03-21T11:22:50.855277+0000","last_clean":"2024-03-21T11:22:50.855277+0000","last_became_active":"2024-03-21T11:20:48.672584+0000","last_became_peered":"2024-03-21T11:20:48.672584+0000","last_unstale":"2024-03-21T11:22:50.855277+0000","last_undegraded":"2024-03-21T11:22:50.855277+0000","last_fullsized":"2024-03-21T11:22:50.855277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:48.213569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846699+0000","last_change":"2024-03-21T11:21:50.655957+0000","last_active":"2024-03-21T11:22:50.846699+0000","last_peered":"2024-03-21T11:22:50.846699+0000","last_clean":"2024-03-21T11:22:50.846699+0000","last_became_active":"2024-03-21T11:21:50.655477+0000","last_became_peered":"2024-03-21T11:21:50.655477+0000","last_unstale":"2024-03-21T11:22:50.846699+0000","last_undegraded":"2024-03-21T11:22:50.846699+0000","last_fullsized":"2024-03-21T11:22:50.846699+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:58:28.476186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'13","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560446+0000","last_change":"2024-03-21T11:21:50.664575+0000","last_active":"2024-03-21T11:22:51.560446+0000","last_peered":"2024-03-21T11:22:51.560446+0000","last_clean":"2024-03-21T11:22:51.560446+0000","last_became_active":"2024-03-21T11:21:50.647335+0000","last_became_peered":"2024-03-21T11:21:50.647335+0000","last_unstale":"2024-03-21T11:22:51.560446+0000","last_undegraded":"2024-03-21T11:22:51.560446+0000","last_fullsized":"2024-03-21T11:22:51.560446+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:50:28.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129559+0000","last_change":"2024-03-21T11:20:48.662392+0000","last_active":"2024-03-21T11:22:51.129559+0000","last_peered":"2024-03-21T11:22:51.129559+0000","last_clean":"2024-03-21T11:22:51.129559+0000","last_became_active":"2024-03-21T11:20:48.662042+0000","last_became_peered":"2024-03-21T11:20:48.662042+0000","last_unstale":"2024-03-21T11:22:51.129559+0000","last_undegraded":"2024-03-21T11:22:51.129559+0000","last_fullsized":"2024-03-21T11:22:51.129559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:59:34.456582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846900+0000","last_change":"2024-03-21T11:20:53.749693+0000","last_active":"2024-03-21T11:22:50.846900+0000","last_peered":"2024-03-21T11:22:50.846900+0000","last_clean":"2024-03-21T11:22:50.846900+0000","last_became_active":"2024-03-21T11:20:53.749438+0000","last_became_peered":"2024-03-21T11:20:53.749438+0000","last_unstale":"2024-03-21T11:22:50.846900+0000","last_undegraded":"2024-03-21T11:22:50.846900+0000","last_fullsized":"2024-03-21T11:22:50.846900+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:37.714471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846726+0000","last_change":"2024-03-21T11:20:48.132587+0000","last_active":"2024-03-21T11:22:50.846726+0000","last_peered":"2024-03-21T11:22:50.846726+0000","last_clean":"2024-03-21T11:22:50.846726+0000","last_became_active":"2024-03-21T11:20:48.132232+0000","last_became_peered":"2024-03-21T11:20:48.132232+0000","last_unstale":"2024-03-21T11:22:50.846726+0000","last_undegraded":"2024-03-21T11:22:50.846726+0000","last_fullsized":"2024-03-21T11:22:50.846726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:20:45.217545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921559+0000","last_change":"2024-03-21T11:20:48.656881+0000","last_active":"2024-03-21T11:22:50.921559+0000","last_peered":"2024-03-21T11:22:50.921559+0000","last_clean":"2024-03-21T11:22:50.921559+0000","last_became_active":"2024-03-21T11:20:48.656426+0000","last_became_peered":"2024-03-21T11:20:48.656426+0000","last_unstale":"2024-03-21T11:22:50.921559+0000","last_undegraded":"2024-03-21T11:22:50.921559+0000","last_fullsized":"2024-03-21T11:22:50.921559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:06:27.398062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852990+0000","last_change":"2024-03-21T11:20:53.733396+0000","last_active":"2024-03-21T11:22:50.852990+0000","last_peered":"2024-03-21T11:22:50.852990+0000","last_clean":"2024-03-21T11:22:50.852990+0000","last_became_active":"2024-03-21T11:20:53.732985+0000","last_became_peered":"2024-03-21T11:20:53.732985+0000","last_unstale":"2024-03-21T11:22:50.852990+0000","last_undegraded":"2024-03-21T11:22:50.852990+0000","last_fullsized":"2024-03-21T11:22:50.852990+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:54:14.064761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":189,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.854844+0000","last_change":"2024-03-21T11:20:48.133362+0000","last_active":"2024-03-21T11:23:40.854844+0000","last_peered":"2024-03-21T11:23:40.854844+0000","last_clean":"2024-03-21T11:23:40.854844+0000","last_became_active":"2024-03-21T11:20:48.132405+0000","last_became_peered":"2024-03-21T11:20:48.132405+0000","last_unstale":"2024-03-21T11:23:40.854844+0000","last_undegraded":"2024-03-21T11:23:40.854844+0000","last_fullsized":"2024-03-21T11:23:40.854844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:48:52.688948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1460,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":148,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846447+0000","last_change":"2024-03-21T11:20:48.659610+0000","last_active":"2024-03-21T11:22:50.846447+0000","last_peered":"2024-03-21T11:22:50.846447+0000","last_clean":"2024-03-21T11:22:50.846447+0000","last_became_active":"2024-03-21T11:20:48.659257+0000","last_became_peered":"2024-03-21T11:20:48.659257+0000","last_unstale":"2024-03-21T11:22:50.846447+0000","last_undegraded":"2024-03-21T11:22:50.846447+0000","last_fullsized":"2024-03-21T11:22:50.846447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:17:30.744434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852994+0000","last_change":"2024-03-21T11:21:50.656047+0000","last_active":"2024-03-21T11:22:50.852994+0000","last_peered":"2024-03-21T11:22:50.852994+0000","last_clean":"2024-03-21T11:22:50.852994+0000","last_became_active":"2024-03-21T11:21:50.655554+0000","last_became_peered":"2024-03-21T11:21:50.655554+0000","last_unstale":"2024-03-21T11:22:50.852994+0000","last_undegraded":"2024-03-21T11:22:50.852994+0000","last_fullsized":"2024-03-21T11:22:50.852994+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:35:15.858237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848356+0000","last_change":"2024-03-21T11:21:50.695331+0000","last_active":"2024-03-21T11:22:50.848356+0000","last_peered":"2024-03-21T11:22:50.848356+0000","last_clean":"2024-03-21T11:22:50.848356+0000","last_became_active":"2024-03-21T11:21:50.690368+0000","last_became_peered":"2024-03-21T11:21:50.690368+0000","last_unstale":"2024-03-21T11:22:50.848356+0000","last_undegraded":"2024-03-21T11:22:50.848356+0000","last_fullsized":"2024-03-21T11:22:50.848356+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:50:03.835364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921486+0000","last_change":"2024-03-21T11:20:48.671221+0000","last_active":"2024-03-21T11:22:50.921486+0000","last_peered":"2024-03-21T11:22:50.921486+0000","last_clean":"2024-03-21T11:22:50.921486+0000","last_became_active":"2024-03-21T11:20:48.670837+0000","last_became_peered":"2024-03-21T11:20:48.670837+0000","last_unstale":"2024-03-21T11:22:50.921486+0000","last_undegraded":"2024-03-21T11:22:50.921486+0000","last_fullsized":"2024-03-21T11:22:50.921486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:38.680409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560403+0000","last_change":"2024-03-21T11:20:53.747501+0000","last_active":"2024-03-21T11:22:51.560403+0000","last_peered":"2024-03-21T11:22:51.560403+0000","last_clean":"2024-03-21T11:22:51.560403+0000","last_became_active":"2024-03-21T11:20:53.747062+0000","last_became_peered":"2024-03-21T11:20:53.747062+0000","last_unstale":"2024-03-21T11:22:51.560403+0000","last_undegraded":"2024-03-21T11:22:51.560403+0000","last_fullsized":"2024-03-21T11:22:51.560403+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:58:36.904221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.754889+0000","last_change":"2024-03-21T11:20:48.156175+0000","last_active":"2024-03-21T11:22:51.754889+0000","last_peered":"2024-03-21T11:22:51.754889+0000","last_clean":"2024-03-21T11:22:51.754889+0000","last_became_active":"2024-03-21T11:20:48.154271+0000","last_became_peered":"2024-03-21T11:20:48.154271+0000","last_unstale":"2024-03-21T11:22:51.754889+0000","last_undegraded":"2024-03-21T11:22:51.754889+0000","last_fullsized":"2024-03-21T11:22:51.754889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:04.216391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848061+0000","last_change":"2024-03-21T11:20:48.670011+0000","last_active":"2024-03-21T11:22:50.848061+0000","last_peered":"2024-03-21T11:22:50.848061+0000","last_clean":"2024-03-21T11:22:50.848061+0000","last_became_active":"2024-03-21T11:20:48.669016+0000","last_became_peered":"2024-03-21T11:20:48.669016+0000","last_unstale":"2024-03-21T11:22:50.848061+0000","last_undegraded":"2024-03-21T11:22:50.848061+0000","last_fullsized":"2024-03-21T11:22:50.848061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:43:18.906490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855150+0000","last_change":"2024-03-21T11:20:53.751934+0000","last_active":"2024-03-21T11:22:50.855150+0000","last_peered":"2024-03-21T11:22:50.855150+0000","last_clean":"2024-03-21T11:22:50.855150+0000","last_became_active":"2024-03-21T11:20:53.751413+0000","last_became_peered":"2024-03-21T11:20:53.751413+0000","last_unstale":"2024-03-21T11:22:50.855150+0000","last_undegraded":"2024-03-21T11:22:50.855150+0000","last_fullsized":"2024-03-21T11:22:50.855150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:12:33.880376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921223+0000","last_change":"2024-03-21T11:20:48.131796+0000","last_active":"2024-03-21T11:22:50.921223+0000","last_peered":"2024-03-21T11:22:50.921223+0000","last_clean":"2024-03-21T11:22:50.921223+0000","last_became_active":"2024-03-21T11:20:48.131478+0000","last_became_peered":"2024-03-21T11:20:48.131478+0000","last_unstale":"2024-03-21T11:22:50.921223+0000","last_undegraded":"2024-03-21T11:22:50.921223+0000","last_fullsized":"2024-03-21T11:22:50.921223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:59:24.871300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849740+0000","last_change":"2024-03-21T11:20:48.660046+0000","last_active":"2024-03-21T11:22:50.849740+0000","last_peered":"2024-03-21T11:22:50.849740+0000","last_clean":"2024-03-21T11:22:50.849740+0000","last_became_active":"2024-03-21T11:20:48.659621+0000","last_became_peered":"2024-03-21T11:20:48.659621+0000","last_unstale":"2024-03-21T11:22:50.849740+0000","last_undegraded":"2024-03-21T11:22:50.849740+0000","last_fullsized":"2024-03-21T11:22:50.849740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:47:01.923285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848718+0000","last_change":"2024-03-21T11:20:53.754458+0000","last_active":"2024-03-21T11:22:50.848718+0000","last_peered":"2024-03-21T11:22:50.848718+0000","last_clean":"2024-03-21T11:22:50.848718+0000","last_became_active":"2024-03-21T11:20:53.753927+0000","last_became_peered":"2024-03-21T11:20:53.753927+0000","last_unstale":"2024-03-21T11:22:50.848718+0000","last_undegraded":"2024-03-21T11:22:50.848718+0000","last_fullsized":"2024-03-21T11:22:50.848718+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:00:47.392699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'7","reported_seq":327,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.756351+0000","last_change":"2024-03-21T11:20:48.154125+0000","last_active":"2024-03-21T11:23:40.756351+0000","last_peered":"2024-03-21T11:23:40.756351+0000","last_clean":"2024-03-21T11:23:40.756351+0000","last_became_active":"2024-03-21T11:20:48.153857+0000","last_became_peered":"2024-03-21T11:20:48.153857+0000","last_unstale":"2024-03-21T11:23:40.756351+0000","last_undegraded":"2024-03-21T11:23:40.756351+0000","last_fullsized":"2024-03-21T11:23:40.756351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:54:06.525335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":286,"num_read_kb":286,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129808+0000","last_change":"2024-03-21T11:20:48.650143+0000","last_active":"2024-03-21T11:22:51.129808+0000","last_peered":"2024-03-21T11:22:51.129808+0000","last_clean":"2024-03-21T11:22:51.129808+0000","last_became_active":"2024-03-21T11:20:48.649825+0000","last_became_peered":"2024-03-21T11:20:48.649825+0000","last_unstale":"2024-03-21T11:22:51.129808+0000","last_undegraded":"2024-03-21T11:22:51.129808+0000","last_fullsized":"2024-03-21T11:22:51.129808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:09:15.735280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.850023+0000","last_change":"2024-03-21T11:20:53.743006+0000","last_active":"2024-03-21T11:22:50.850023+0000","last_peered":"2024-03-21T11:22:50.850023+0000","last_clean":"2024-03-21T11:22:50.850023+0000","last_became_active":"2024-03-21T11:20:53.742587+0000","last_became_peered":"2024-03-21T11:20:53.742587+0000","last_unstale":"2024-03-21T11:22:50.850023+0000","last_undegraded":"2024-03-21T11:22:50.850023+0000","last_fullsized":"2024-03-21T11:22:50.850023+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:06:06.845791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806093+0000","last_change":"2024-03-21T11:20:48.137137+0000","last_active":"2024-03-21T11:22:51.806093+0000","last_peered":"2024-03-21T11:22:51.806093+0000","last_clean":"2024-03-21T11:22:51.806093+0000","last_became_active":"2024-03-21T11:20:48.136790+0000","last_became_peered":"2024-03-21T11:20:48.136790+0000","last_unstale":"2024-03-21T11:22:51.806093+0000","last_undegraded":"2024-03-21T11:22:51.806093+0000","last_fullsized":"2024-03-21T11:22:51.806093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:22:47.223969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855575+0000","last_change":"2024-03-21T11:20:48.673568+0000","last_active":"2024-03-21T11:22:50.855575+0000","last_peered":"2024-03-21T11:22:50.855575+0000","last_clean":"2024-03-21T11:22:50.855575+0000","last_became_active":"2024-03-21T11:20:48.672613+0000","last_became_peered":"2024-03-21T11:20:48.672613+0000","last_unstale":"2024-03-21T11:22:50.855575+0000","last_undegraded":"2024-03-21T11:22:50.855575+0000","last_fullsized":"2024-03-21T11:22:50.855575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:43:38.193802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846803+0000","last_change":"2024-03-21T11:20:53.739228+0000","last_active":"2024-03-21T11:22:50.846803+0000","last_peered":"2024-03-21T11:22:50.846803+0000","last_clean":"2024-03-21T11:22:50.846803+0000","last_became_active":"2024-03-21T11:20:53.738750+0000","last_became_peered":"2024-03-21T11:20:53.738750+0000","last_unstale":"2024-03-21T11:22:50.846803+0000","last_undegraded":"2024-03-21T11:22:50.846803+0000","last_fullsized":"2024-03-21T11:22:50.846803+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:58:33.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921548+0000","last_change":"2024-03-21T11:20:48.144451+0000","last_active":"2024-03-21T11:22:50.921548+0000","last_peered":"2024-03-21T11:22:50.921548+0000","last_clean":"2024-03-21T11:22:50.921548+0000","last_became_active":"2024-03-21T11:20:48.144074+0000","last_became_peered":"2024-03-21T11:20:48.144074+0000","last_unstale":"2024-03-21T11:22:50.921548+0000","last_undegraded":"2024-03-21T11:22:50.921548+0000","last_fullsized":"2024-03-21T11:22:50.921548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:33:17.032461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129797+0000","last_change":"2024-03-21T11:20:48.654472+0000","last_active":"2024-03-21T11:22:51.129797+0000","last_peered":"2024-03-21T11:22:51.129797+0000","last_clean":"2024-03-21T11:22:51.129797+0000","last_became_active":"2024-03-21T11:20:48.654129+0000","last_became_peered":"2024-03-21T11:20:48.654129+0000","last_unstale":"2024-03-21T11:22:51.129797+0000","last_undegraded":"2024-03-21T11:22:51.129797+0000","last_fullsized":"2024-03-21T11:22:51.129797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:28:31.224296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849451+0000","last_change":"2024-03-21T11:20:53.739994+0000","last_active":"2024-03-21T11:22:50.849451+0000","last_peered":"2024-03-21T11:22:50.849451+0000","last_clean":"2024-03-21T11:22:50.849451+0000","last_became_active":"2024-03-21T11:20:53.738319+0000","last_became_peered":"2024-03-21T11:20:53.738319+0000","last_unstale":"2024-03-21T11:22:50.849451+0000","last_undegraded":"2024-03-21T11:22:50.849451+0000","last_fullsized":"2024-03-21T11:22:50.849451+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:55.694986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":371,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.757016+0000","last_change":"2024-03-21T11:20:48.131461+0000","last_active":"2024-03-21T11:23:40.757016+0000","last_peered":"2024-03-21T11:23:40.757016+0000","last_clean":"2024-03-21T11:23:40.757016+0000","last_became_active":"2024-03-21T11:20:48.131079+0000","last_became_peered":"2024-03-21T11:20:48.131079+0000","last_unstale":"2024-03-21T11:23:40.757016+0000","last_undegraded":"2024-03-21T11:23:40.757016+0000","last_fullsized":"2024-03-21T11:23:40.757016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:48:57.528444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70525,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":296,"num_read_kb":217,"num_write":43,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846832+0000","last_change":"2024-03-21T11:20:48.653696+0000","last_active":"2024-03-21T11:22:50.846832+0000","last_peered":"2024-03-21T11:22:50.846832+0000","last_clean":"2024-03-21T11:22:50.846832+0000","last_became_active":"2024-03-21T11:20:48.653218+0000","last_became_peered":"2024-03-21T11:20:48.653218+0000","last_unstale":"2024-03-21T11:22:50.846832+0000","last_undegraded":"2024-03-21T11:22:50.846832+0000","last_fullsized":"2024-03-21T11:22:50.846832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:11:59.419040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921748+0000","last_change":"2024-03-21T11:20:53.754019+0000","last_active":"2024-03-21T11:22:50.921748+0000","last_peered":"2024-03-21T11:22:50.921748+0000","last_clean":"2024-03-21T11:22:50.921748+0000","last_became_active":"2024-03-21T11:20:53.753606+0000","last_became_peered":"2024-03-21T11:20:53.753606+0000","last_unstale":"2024-03-21T11:22:50.921748+0000","last_undegraded":"2024-03-21T11:22:50.921748+0000","last_fullsized":"2024-03-21T11:22:50.921748+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:23:27.898596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853065+0000","last_change":"2024-03-21T11:20:48.153544+0000","last_active":"2024-03-21T11:22:50.853065+0000","last_peered":"2024-03-21T11:22:50.853065+0000","last_clean":"2024-03-21T11:22:50.853065+0000","last_became_active":"2024-03-21T11:20:48.153279+0000","last_became_peered":"2024-03-21T11:20:48.153279+0000","last_unstale":"2024-03-21T11:22:50.853065+0000","last_undegraded":"2024-03-21T11:22:50.853065+0000","last_fullsized":"2024-03-21T11:22:50.853065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:31:31.911844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755158+0000","last_change":"2024-03-21T11:20:48.665140+0000","last_active":"2024-03-21T11:22:51.755158+0000","last_peered":"2024-03-21T11:22:51.755158+0000","last_clean":"2024-03-21T11:22:51.755158+0000","last_became_active":"2024-03-21T11:20:48.664226+0000","last_became_peered":"2024-03-21T11:20:48.664226+0000","last_unstale":"2024-03-21T11:22:51.755158+0000","last_undegraded":"2024-03-21T11:22:51.755158+0000","last_fullsized":"2024-03-21T11:22:51.755158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:47:38.951403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560135+0000","last_change":"2024-03-21T11:20:53.751887+0000","last_active":"2024-03-21T11:22:51.560135+0000","last_peered":"2024-03-21T11:22:51.560135+0000","last_clean":"2024-03-21T11:22:51.560135+0000","last_became_active":"2024-03-21T11:20:53.751353+0000","last_became_peered":"2024-03-21T11:20:53.751353+0000","last_unstale":"2024-03-21T11:22:51.560135+0000","last_undegraded":"2024-03-21T11:22:51.560135+0000","last_fullsized":"2024-03-21T11:22:51.560135+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:21:36.689786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921390+0000","last_change":"2024-03-21T11:20:48.162150+0000","last_active":"2024-03-21T11:22:50.921390+0000","last_peered":"2024-03-21T11:22:50.921390+0000","last_clean":"2024-03-21T11:22:50.921390+0000","last_became_active":"2024-03-21T11:20:48.161825+0000","last_became_peered":"2024-03-21T11:20:48.161825+0000","last_unstale":"2024-03-21T11:22:50.921390+0000","last_undegraded":"2024-03-21T11:22:50.921390+0000","last_fullsized":"2024-03-21T11:22:50.921390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:10:47.097540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805965+0000","last_change":"2024-03-21T11:20:48.663856+0000","last_active":"2024-03-21T11:22:51.805965+0000","last_peered":"2024-03-21T11:22:51.805965+0000","last_clean":"2024-03-21T11:22:51.805965+0000","last_became_active":"2024-03-21T11:20:48.662908+0000","last_became_peered":"2024-03-21T11:20:48.662908+0000","last_unstale":"2024-03-21T11:22:51.805965+0000","last_undegraded":"2024-03-21T11:22:51.805965+0000","last_fullsized":"2024-03-21T11:22:51.805965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:11:12.255265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.851969+0000","last_change":"2024-03-21T11:20:53.755942+0000","last_active":"2024-03-21T11:22:50.851969+0000","last_peered":"2024-03-21T11:22:50.851969+0000","last_clean":"2024-03-21T11:22:50.851969+0000","last_became_active":"2024-03-21T11:20:53.755141+0000","last_became_peered":"2024-03-21T11:20:53.755141+0000","last_unstale":"2024-03-21T11:22:50.851969+0000","last_undegraded":"2024-03-21T11:22:50.851969+0000","last_fullsized":"2024-03-21T11:22:50.851969+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:30.141117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":187,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.842723+0000","last_change":"2024-03-21T11:20:48.132460+0000","last_active":"2024-03-21T11:23:40.842723+0000","last_peered":"2024-03-21T11:23:40.842723+0000","last_clean":"2024-03-21T11:23:40.842723+0000","last_became_active":"2024-03-21T11:20:48.131573+0000","last_became_peered":"2024-03-21T11:20:48.131573+0000","last_unstale":"2024-03-21T11:23:40.842723+0000","last_undegraded":"2024-03-21T11:23:40.842723+0000","last_fullsized":"2024-03-21T11:23:40.842723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:40:02.821536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805992+0000","last_change":"2024-03-21T11:20:48.656262+0000","last_active":"2024-03-21T11:22:51.805992+0000","last_peered":"2024-03-21T11:22:51.805992+0000","last_clean":"2024-03-21T11:22:51.805992+0000","last_became_active":"2024-03-21T11:20:48.655924+0000","last_became_peered":"2024-03-21T11:20:48.655924+0000","last_unstale":"2024-03-21T11:22:51.805992+0000","last_undegraded":"2024-03-21T11:22:51.805992+0000","last_fullsized":"2024-03-21T11:22:51.805992+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:24:20.412185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755346+0000","last_change":"2024-03-21T11:20:53.758666+0000","last_active":"2024-03-21T11:22:51.755346+0000","last_peered":"2024-03-21T11:22:51.755346+0000","last_clean":"2024-03-21T11:22:51.755346+0000","last_became_active":"2024-03-21T11:20:53.757398+0000","last_became_peered":"2024-03-21T11:20:53.757398+0000","last_unstale":"2024-03-21T11:22:51.755346+0000","last_undegraded":"2024-03-21T11:22:51.755346+0000","last_fullsized":"2024-03-21T11:22:51.755346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:34.198388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846374+0000","last_change":"2024-03-21T11:20:48.132725+0000","last_active":"2024-03-21T11:22:50.846374+0000","last_peered":"2024-03-21T11:22:50.846374+0000","last_clean":"2024-03-21T11:22:50.846374+0000","last_became_active":"2024-03-21T11:20:48.132367+0000","last_became_peered":"2024-03-21T11:20:48.132367+0000","last_unstale":"2024-03-21T11:22:50.846374+0000","last_undegraded":"2024-03-21T11:22:50.846374+0000","last_fullsized":"2024-03-21T11:22:50.846374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:19:18.334962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921197+0000","last_change":"2024-03-21T11:20:48.656302+0000","last_active":"2024-03-21T11:22:50.921197+0000","last_peered":"2024-03-21T11:22:50.921197+0000","last_clean":"2024-03-21T11:22:50.921197+0000","last_became_active":"2024-03-21T11:20:48.654305+0000","last_became_peered":"2024-03-21T11:20:48.654305+0000","last_unstale":"2024-03-21T11:22:50.921197+0000","last_undegraded":"2024-03-21T11:22:50.921197+0000","last_fullsized":"2024-03-21T11:22:50.921197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:59:40.766829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921110+0000","last_change":"2024-03-21T11:20:53.750710+0000","last_active":"2024-03-21T11:22:50.921110+0000","last_peered":"2024-03-21T11:22:50.921110+0000","last_clean":"2024-03-21T11:22:50.921110+0000","last_became_active":"2024-03-21T11:20:53.750255+0000","last_became_peered":"2024-03-21T11:20:53.750255+0000","last_unstale":"2024-03-21T11:22:50.921110+0000","last_undegraded":"2024-03-21T11:22:50.921110+0000","last_fullsized":"2024-03-21T11:22:50.921110+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:36:28.611838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846888+0000","last_change":"2024-03-21T11:20:48.133524+0000","last_active":"2024-03-21T11:22:50.846888+0000","last_peered":"2024-03-21T11:22:50.846888+0000","last_clean":"2024-03-21T11:22:50.846888+0000","last_became_active":"2024-03-21T11:20:48.133186+0000","last_became_peered":"2024-03-21T11:20:48.133186+0000","last_unstale":"2024-03-21T11:22:50.846888+0000","last_undegraded":"2024-03-21T11:22:50.846888+0000","last_fullsized":"2024-03-21T11:22:50.846888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:02.018639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560425+0000","last_change":"2024-03-21T11:20:48.654126+0000","last_active":"2024-03-21T11:22:51.560425+0000","last_peered":"2024-03-21T11:22:51.560425+0000","last_clean":"2024-03-21T11:22:51.560425+0000","last_became_active":"2024-03-21T11:20:48.653835+0000","last_became_peered":"2024-03-21T11:20:48.653835+0000","last_unstale":"2024-03-21T11:22:51.560425+0000","last_undegraded":"2024-03-21T11:22:51.560425+0000","last_fullsized":"2024-03-21T11:22:51.560425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:12:03.332076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846394+0000","last_change":"2024-03-21T11:20:53.916645+0000","last_active":"2024-03-21T11:22:50.846394+0000","last_peered":"2024-03-21T11:22:50.846394+0000","last_clean":"2024-03-21T11:22:50.846394+0000","last_became_active":"2024-03-21T11:20:53.916178+0000","last_became_peered":"2024-03-21T11:20:53.916178+0000","last_unstale":"2024-03-21T11:22:50.846394+0000","last_undegraded":"2024-03-21T11:22:50.846394+0000","last_fullsized":"2024-03-21T11:22:50.846394+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:56:15.805231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.851892+0000","last_change":"2024-03-21T11:20:48.159877+0000","last_active":"2024-03-21T11:22:50.851892+0000","last_peered":"2024-03-21T11:22:50.851892+0000","last_clean":"2024-03-21T11:22:50.851892+0000","last_became_active":"2024-03-21T11:20:48.159529+0000","last_became_peered":"2024-03-21T11:20:48.159529+0000","last_unstale":"2024-03-21T11:22:50.851892+0000","last_undegraded":"2024-03-21T11:22:50.851892+0000","last_fullsized":"2024-03-21T11:22:50.851892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:17:09.662462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848812+0000","last_change":"2024-03-21T11:20:48.663311+0000","last_active":"2024-03-21T11:22:50.848812+0000","last_peered":"2024-03-21T11:22:50.848812+0000","last_clean":"2024-03-21T11:22:50.848812+0000","last_became_active":"2024-03-21T11:20:48.662961+0000","last_became_peered":"2024-03-21T11:20:48.662961+0000","last_unstale":"2024-03-21T11:22:50.848812+0000","last_undegraded":"2024-03-21T11:22:50.848812+0000","last_fullsized":"2024-03-21T11:22:50.848812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:14:46.616326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755080+0000","last_change":"2024-03-21T11:20:53.751804+0000","last_active":"2024-03-21T11:22:51.755080+0000","last_peered":"2024-03-21T11:22:51.755080+0000","last_clean":"2024-03-21T11:22:51.755080+0000","last_became_active":"2024-03-21T11:20:53.751323+0000","last_became_peered":"2024-03-21T11:20:53.751323+0000","last_unstale":"2024-03-21T11:22:51.755080+0000","last_undegraded":"2024-03-21T11:22:51.755080+0000","last_fullsized":"2024-03-21T11:22:51.755080+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:38:25.677345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":186,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.908004+0000","last_change":"2024-03-21T11:20:48.147580+0000","last_active":"2024-03-21T11:23:40.908004+0000","last_peered":"2024-03-21T11:23:40.908004+0000","last_clean":"2024-03-21T11:23:40.908004+0000","last_became_active":"2024-03-21T11:20:48.147222+0000","last_became_peered":"2024-03-21T11:20:48.147222+0000","last_unstale":"2024-03-21T11:23:40.908004+0000","last_undegraded":"2024-03-21T11:23:40.908004+0000","last_fullsized":"2024-03-21T11:23:40.908004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:49:04.204129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.856848+0000","last_change":"2024-03-21T11:22:50.856453+0000","last_active":"2024-03-21T11:22:50.856848+0000","last_peered":"2024-03-21T11:22:50.856848+0000","last_clean":"2024-03-21T11:22:50.856848+0000","last_became_active":"2024-03-21T11:22:50.855987+0000","last_became_peered":"2024-03-21T11:22:50.855987+0000","last_unstale":"2024-03-21T11:22:50.856848+0000","last_undegraded":"2024-03-21T11:22:50.856848+0000","last_fullsized":"2024-03-21T11:22:50.856848+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:32:05.460528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755558+0000","last_change":"2024-03-21T11:20:53.916931+0000","last_active":"2024-03-21T11:22:51.755558+0000","last_peered":"2024-03-21T11:22:51.755558+0000","last_clean":"2024-03-21T11:22:51.755558+0000","last_became_active":"2024-03-21T11:20:53.916495+0000","last_became_peered":"2024-03-21T11:20:53.916495+0000","last_unstale":"2024-03-21T11:22:51.755558+0000","last_undegraded":"2024-03-21T11:22:51.755558+0000","last_fullsized":"2024-03-21T11:22:51.755558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:18:33.046052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855560+0000","last_change":"2024-03-21T11:22:50.854745+0000","last_active":"2024-03-21T11:22:50.855560+0000","last_peered":"2024-03-21T11:22:50.855560+0000","last_clean":"2024-03-21T11:22:50.855560+0000","last_became_active":"2024-03-21T11:22:50.854201+0000","last_became_peered":"2024-03-21T11:22:50.854201+0000","last_unstale":"2024-03-21T11:22:50.855560+0000","last_undegraded":"2024-03-21T11:22:50.855560+0000","last_fullsized":"2024-03-21T11:22:50.855560+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:06:27.288363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":184,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.842225+0000","last_change":"2024-03-21T11:20:48.164607+0000","last_active":"2024-03-21T11:23:40.842225+0000","last_peered":"2024-03-21T11:23:40.842225+0000","last_clean":"2024-03-21T11:23:40.842225+0000","last_became_active":"2024-03-21T11:20:48.164222+0000","last_became_peered":"2024-03-21T11:20:48.164222+0000","last_unstale":"2024-03-21T11:23:40.842225+0000","last_undegraded":"2024-03-21T11:23:40.842225+0000","last_fullsized":"2024-03-21T11:23:40.842225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:52:24.359155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806069+0000","last_change":"2024-03-21T11:21:50.658412+0000","last_active":"2024-03-21T11:22:51.806069+0000","last_peered":"2024-03-21T11:22:51.806069+0000","last_clean":"2024-03-21T11:22:51.806069+0000","last_became_active":"2024-03-21T11:21:50.649270+0000","last_became_peered":"2024-03-21T11:21:50.649270+0000","last_unstale":"2024-03-21T11:22:51.806069+0000","last_undegraded":"2024-03-21T11:22:51.806069+0000","last_fullsized":"2024-03-21T11:22:51.806069+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:11:06.006302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848564+0000","last_change":"2024-03-21T11:20:48.667212+0000","last_active":"2024-03-21T11:22:50.848564+0000","last_peered":"2024-03-21T11:22:50.848564+0000","last_clean":"2024-03-21T11:22:50.848564+0000","last_became_active":"2024-03-21T11:20:48.662839+0000","last_became_peered":"2024-03-21T11:20:48.662839+0000","last_unstale":"2024-03-21T11:22:50.848564+0000","last_undegraded":"2024-03-21T11:22:50.848564+0000","last_fullsized":"2024-03-21T11:22:50.848564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:27:41.490922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846727+0000","last_change":"2024-03-21T11:20:48.151185+0000","last_active":"2024-03-21T11:22:50.846727+0000","last_peered":"2024-03-21T11:22:50.846727+0000","last_clean":"2024-03-21T11:22:50.846727+0000","last_became_active":"2024-03-21T11:20:48.150859+0000","last_became_peered":"2024-03-21T11:20:48.150859+0000","last_unstale":"2024-03-21T11:22:50.846727+0000","last_undegraded":"2024-03-21T11:22:50.846727+0000","last_fullsized":"2024-03-21T11:22:50.846727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:52:45.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":357,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805649+0000","last_change":"2024-03-21T11:19:28.916140+0000","last_active":"2024-03-21T11:22:51.805649+0000","last_peered":"2024-03-21T11:22:51.805649+0000","last_clean":"2024-03-21T11:22:51.805649+0000","last_became_active":"2024-03-21T11:19:28.915634+0000","last_became_peered":"2024-03-21T11:19:28.915634+0000","last_unstale":"2024-03-21T11:22:51.805649+0000","last_undegraded":"2024-03-21T11:22:51.805649+0000","last_fullsized":"2024-03-21T11:22:51.805649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:15:35.803515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:15:35.803515+0000","last_clean_scrub_stamp":"2024-03-21T11:15:35.803515+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:08:31.509894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921613+0000","last_change":"2024-03-21T11:21:50.656708+0000","last_active":"2024-03-21T11:22:50.921613+0000","last_peered":"2024-03-21T11:22:50.921613+0000","last_clean":"2024-03-21T11:22:50.921613+0000","last_became_active":"2024-03-21T11:21:50.656236+0000","last_became_peered":"2024-03-21T11:21:50.656236+0000","last_unstale":"2024-03-21T11:22:50.921613+0000","last_undegraded":"2024-03-21T11:22:50.921613+0000","last_fullsized":"2024-03-21T11:22:50.921613+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:17:44.114512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.754767+0000","last_change":"2024-03-21T11:20:48.657878+0000","last_active":"2024-03-21T11:22:51.754767+0000","last_peered":"2024-03-21T11:22:51.754767+0000","last_clean":"2024-03-21T11:22:51.754767+0000","last_became_active":"2024-03-21T11:20:48.657212+0000","last_became_peered":"2024-03-21T11:20:48.657212+0000","last_unstale":"2024-03-21T11:22:51.754767+0000","last_undegraded":"2024-03-21T11:22:51.754767+0000","last_fullsized":"2024-03-21T11:22:51.754767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:43:11.993577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805832+0000","last_change":"2024-03-21T11:20:48.135513+0000","last_active":"2024-03-21T11:22:51.805832+0000","last_peered":"2024-03-21T11:22:51.805832+0000","last_clean":"2024-03-21T11:22:51.805832+0000","last_became_active":"2024-03-21T11:20:48.135223+0000","last_became_peered":"2024-03-21T11:20:48.135223+0000","last_unstale":"2024-03-21T11:22:51.805832+0000","last_undegraded":"2024-03-21T11:22:51.805832+0000","last_fullsized":"2024-03-21T11:22:51.805832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:37:38.834284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560138+0000","last_change":"2024-03-21T11:20:53.751790+0000","last_active":"2024-03-21T11:22:51.560138+0000","last_peered":"2024-03-21T11:22:51.560138+0000","last_clean":"2024-03-21T11:22:51.560138+0000","last_became_active":"2024-03-21T11:20:53.751312+0000","last_became_peered":"2024-03-21T11:20:53.751312+0000","last_unstale":"2024-03-21T11:22:51.560138+0000","last_undegraded":"2024-03-21T11:22:51.560138+0000","last_fullsized":"2024-03-21T11:22:51.560138+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:21:41.599985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846835+0000","last_change":"2024-03-21T11:20:48.154119+0000","last_active":"2024-03-21T11:22:50.846835+0000","last_peered":"2024-03-21T11:22:50.846835+0000","last_clean":"2024-03-21T11:22:50.846835+0000","last_became_active":"2024-03-21T11:20:48.153781+0000","last_became_peered":"2024-03-21T11:20:48.153781+0000","last_unstale":"2024-03-21T11:22:50.846835+0000","last_undegraded":"2024-03-21T11:22:50.846835+0000","last_fullsized":"2024-03-21T11:22:50.846835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:44:13.937836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755223+0000","last_change":"2024-03-21T11:20:48.661282+0000","last_active":"2024-03-21T11:22:51.755223+0000","last_peered":"2024-03-21T11:22:51.755223+0000","last_clean":"2024-03-21T11:22:51.755223+0000","last_became_active":"2024-03-21T11:20:48.660569+0000","last_became_peered":"2024-03-21T11:20:48.660569+0000","last_unstale":"2024-03-21T11:22:51.755223+0000","last_undegraded":"2024-03-21T11:22:51.755223+0000","last_fullsized":"2024-03-21T11:22:51.755223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:14.875700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847881+0000","last_change":"2024-03-21T11:20:53.918016+0000","last_active":"2024-03-21T11:22:50.847881+0000","last_peered":"2024-03-21T11:22:50.847881+0000","last_clean":"2024-03-21T11:22:50.847881+0000","last_became_active":"2024-03-21T11:20:53.916891+0000","last_became_peered":"2024-03-21T11:20:53.916891+0000","last_unstale":"2024-03-21T11:22:50.847881+0000","last_undegraded":"2024-03-21T11:22:50.847881+0000","last_fullsized":"2024-03-21T11:22:50.847881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:40:25.206778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'15","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:37.609177+0000","last_change":"2024-03-21T11:20:48.167560+0000","last_active":"2024-03-21T11:23:37.609177+0000","last_peered":"2024-03-21T11:23:37.609177+0000","last_clean":"2024-03-21T11:23:37.609177+0000","last_became_active":"2024-03-21T11:20:48.164986+0000","last_became_peered":"2024-03-21T11:20:48.164986+0000","last_unstale":"2024-03-21T11:23:37.609177+0000","last_undegraded":"2024-03-21T11:23:37.609177+0000","last_fullsized":"2024-03-21T11:23:37.609177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:59.930626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849952+0000","last_change":"2024-03-21T11:20:48.675250+0000","last_active":"2024-03-21T11:22:50.849952+0000","last_peered":"2024-03-21T11:22:50.849952+0000","last_clean":"2024-03-21T11:22:50.849952+0000","last_became_active":"2024-03-21T11:20:48.674638+0000","last_became_peered":"2024-03-21T11:20:48.674638+0000","last_unstale":"2024-03-21T11:22:50.849952+0000","last_undegraded":"2024-03-21T11:22:50.849952+0000","last_fullsized":"2024-03-21T11:22:50.849952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:46:34.842363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852772+0000","last_change":"2024-03-21T11:20:53.733809+0000","last_active":"2024-03-21T11:22:50.852772+0000","last_peered":"2024-03-21T11:22:50.852772+0000","last_clean":"2024-03-21T11:22:50.852772+0000","last_became_active":"2024-03-21T11:20:53.733393+0000","last_became_peered":"2024-03-21T11:20:53.733393+0000","last_unstale":"2024-03-21T11:22:50.852772+0000","last_undegraded":"2024-03-21T11:22:50.852772+0000","last_fullsized":"2024-03-21T11:22:50.852772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:02:46.270716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847217+0000","last_change":"2024-03-21T11:20:48.139475+0000","last_active":"2024-03-21T11:22:50.847217+0000","last_peered":"2024-03-21T11:22:50.847217+0000","last_clean":"2024-03-21T11:22:50.847217+0000","last_became_active":"2024-03-21T11:20:48.139249+0000","last_became_peered":"2024-03-21T11:20:48.139249+0000","last_unstale":"2024-03-21T11:22:50.847217+0000","last_undegraded":"2024-03-21T11:22:50.847217+0000","last_fullsized":"2024-03-21T11:22:50.847217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:45:18.811829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855394+0000","last_change":"2024-03-21T11:20:48.673975+0000","last_active":"2024-03-21T11:22:50.855394+0000","last_peered":"2024-03-21T11:22:50.855394+0000","last_clean":"2024-03-21T11:22:50.855394+0000","last_became_active":"2024-03-21T11:20:48.673587+0000","last_became_peered":"2024-03-21T11:20:48.673587+0000","last_unstale":"2024-03-21T11:22:50.855394+0000","last_undegraded":"2024-03-21T11:22:50.855394+0000","last_fullsized":"2024-03-21T11:22:50.855394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:08:52.354780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847109+0000","last_change":"2024-03-21T11:20:53.917493+0000","last_active":"2024-03-21T11:22:50.847109+0000","last_peered":"2024-03-21T11:22:50.847109+0000","last_clean":"2024-03-21T11:22:50.847109+0000","last_became_active":"2024-03-21T11:20:53.917007+0000","last_became_peered":"2024-03-21T11:20:53.917007+0000","last_unstale":"2024-03-21T11:22:50.847109+0000","last_undegraded":"2024-03-21T11:22:50.847109+0000","last_fullsized":"2024-03-21T11:22:50.847109+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:39:20.280899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129330+0000","last_change":"2024-03-21T11:20:48.140914+0000","last_active":"2024-03-21T11:22:51.129330+0000","last_peered":"2024-03-21T11:22:51.129330+0000","last_clean":"2024-03-21T11:22:51.129330+0000","last_became_active":"2024-03-21T11:20:48.140558+0000","last_became_peered":"2024-03-21T11:20:48.140558+0000","last_unstale":"2024-03-21T11:22:51.129330+0000","last_undegraded":"2024-03-21T11:22:51.129330+0000","last_fullsized":"2024-03-21T11:22:51.129330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:59:52.065431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846373+0000","last_change":"2024-03-21T11:20:48.651190+0000","last_active":"2024-03-21T11:22:50.846373+0000","last_peered":"2024-03-21T11:22:50.846373+0000","last_clean":"2024-03-21T11:22:50.846373+0000","last_became_active":"2024-03-21T11:20:48.650818+0000","last_became_peered":"2024-03-21T11:20:48.650818+0000","last_unstale":"2024-03-21T11:22:50.846373+0000","last_undegraded":"2024-03-21T11:22:50.846373+0000","last_fullsized":"2024-03-21T11:22:50.846373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:47:17.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852214+0000","last_change":"2024-03-21T11:20:53.756844+0000","last_active":"2024-03-21T11:22:50.852214+0000","last_peered":"2024-03-21T11:22:50.852214+0000","last_clean":"2024-03-21T11:22:50.852214+0000","last_became_active":"2024-03-21T11:20:53.756380+0000","last_became_peered":"2024-03-21T11:20:53.756380+0000","last_unstale":"2024-03-21T11:22:50.852214+0000","last_undegraded":"2024-03-21T11:22:50.852214+0000","last_fullsized":"2024-03-21T11:22:50.852214+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:51:55.327798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848472+0000","last_change":"2024-03-21T11:20:48.160749+0000","last_active":"2024-03-21T11:22:50.848472+0000","last_peered":"2024-03-21T11:22:50.848472+0000","last_clean":"2024-03-21T11:22:50.848472+0000","last_became_active":"2024-03-21T11:20:48.158507+0000","last_became_peered":"2024-03-21T11:20:48.158507+0000","last_unstale":"2024-03-21T11:22:50.848472+0000","last_undegraded":"2024-03-21T11:22:50.848472+0000","last_fullsized":"2024-03-21T11:22:50.848472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:55.375529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560069+0000","last_change":"2024-03-21T11:20:48.667153+0000","last_active":"2024-03-21T11:22:51.560069+0000","last_peered":"2024-03-21T11:22:51.560069+0000","last_clean":"2024-03-21T11:22:51.560069+0000","last_became_active":"2024-03-21T11:20:48.666823+0000","last_became_peered":"2024-03-21T11:20:48.666823+0000","last_unstale":"2024-03-21T11:22:51.560069+0000","last_undegraded":"2024-03-21T11:22:51.560069+0000","last_fullsized":"2024-03-21T11:22:51.560069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:43:54.183372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846842+0000","last_change":"2024-03-21T11:20:53.739188+0000","last_active":"2024-03-21T11:22:50.846842+0000","last_peered":"2024-03-21T11:22:50.846842+0000","last_clean":"2024-03-21T11:22:50.846842+0000","last_became_active":"2024-03-21T11:20:53.738770+0000","last_became_peered":"2024-03-21T11:20:53.738770+0000","last_unstale":"2024-03-21T11:22:50.846842+0000","last_undegraded":"2024-03-21T11:22:50.846842+0000","last_fullsized":"2024-03-21T11:22:50.846842+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:53:48.897963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560058+0000","last_change":"2024-03-21T11:21:50.668860+0000","last_active":"2024-03-21T11:22:51.560058+0000","last_peered":"2024-03-21T11:22:51.560058+0000","last_clean":"2024-03-21T11:22:51.560058+0000","last_became_active":"2024-03-21T11:21:50.655255+0000","last_became_peered":"2024-03-21T11:21:50.655255+0000","last_unstale":"2024-03-21T11:22:51.560058+0000","last_undegraded":"2024-03-21T11:22:51.560058+0000","last_fullsized":"2024-03-21T11:22:51.560058+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:27.634508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.850136+0000","last_change":"2024-03-21T11:20:48.677303+0000","last_active":"2024-03-21T11:22:50.850136+0000","last_peered":"2024-03-21T11:22:50.850136+0000","last_clean":"2024-03-21T11:22:50.850136+0000","last_became_active":"2024-03-21T11:20:48.676903+0000","last_became_peered":"2024-03-21T11:20:48.676903+0000","last_unstale":"2024-03-21T11:22:50.850136+0000","last_undegraded":"2024-03-21T11:22:50.850136+0000","last_fullsized":"2024-03-21T11:22:50.850136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:06:59.922274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847033+0000","last_change":"2024-03-21T11:20:53.916691+0000","last_active":"2024-03-21T11:22:50.847033+0000","last_peered":"2024-03-21T11:22:50.847033+0000","last_clean":"2024-03-21T11:22:50.847033+0000","last_became_active":"2024-03-21T11:20:53.916275+0000","last_became_peered":"2024-03-21T11:20:53.916275+0000","last_unstale":"2024-03-21T11:22:50.847033+0000","last_undegraded":"2024-03-21T11:22:50.847033+0000","last_fullsized":"2024-03-21T11:22:50.847033+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:35.848589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846526+0000","last_change":"2024-03-21T11:21:50.668467+0000","last_active":"2024-03-21T11:22:50.846526+0000","last_peered":"2024-03-21T11:22:50.846526+0000","last_clean":"2024-03-21T11:22:50.846526+0000","last_became_active":"2024-03-21T11:21:50.649598+0000","last_became_peered":"2024-03-21T11:21:50.649598+0000","last_unstale":"2024-03-21T11:22:50.846526+0000","last_undegraded":"2024-03-21T11:22:50.846526+0000","last_fullsized":"2024-03-21T11:22:50.846526+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:01:48.186997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806108+0000","last_change":"2024-03-21T11:20:48.661486+0000","last_active":"2024-03-21T11:22:51.806108+0000","last_peered":"2024-03-21T11:22:51.806108+0000","last_clean":"2024-03-21T11:22:51.806108+0000","last_became_active":"2024-03-21T11:20:48.661109+0000","last_became_peered":"2024-03-21T11:20:48.661109+0000","last_unstale":"2024-03-21T11:22:51.806108+0000","last_undegraded":"2024-03-21T11:22:51.806108+0000","last_fullsized":"2024-03-21T11:22:51.806108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:15:27.093616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129398+0000","last_change":"2024-03-21T11:20:53.747223+0000","last_active":"2024-03-21T11:22:51.129398+0000","last_peered":"2024-03-21T11:22:51.129398+0000","last_clean":"2024-03-21T11:22:51.129398+0000","last_became_active":"2024-03-21T11:20:53.746805+0000","last_became_peered":"2024-03-21T11:20:53.746805+0000","last_unstale":"2024-03-21T11:22:51.129398+0000","last_undegraded":"2024-03-21T11:22:51.129398+0000","last_fullsized":"2024-03-21T11:22:51.129398+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:29.186745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"93'7","reported_seq":184,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:37.753498+0000","last_change":"2024-03-21T11:20:48.151172+0000","last_active":"2024-03-21T11:23:37.753498+0000","last_peered":"2024-03-21T11:23:37.753498+0000","last_clean":"2024-03-21T11:23:37.753498+0000","last_became_active":"2024-03-21T11:20:48.150832+0000","last_became_peered":"2024-03-21T11:20:48.150832+0000","last_unstale":"2024-03-21T11:23:37.753498+0000","last_undegraded":"2024-03-21T11:23:37.753498+0000","last_fullsized":"2024-03-21T11:23:37.753498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:34:46.367713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806211+0000","last_change":"2024-03-21T11:20:48.652275+0000","last_active":"2024-03-21T11:22:51.806211+0000","last_peered":"2024-03-21T11:22:51.806211+0000","last_clean":"2024-03-21T11:22:51.806211+0000","last_became_active":"2024-03-21T11:20:48.651965+0000","last_became_peered":"2024-03-21T11:20:48.651965+0000","last_unstale":"2024-03-21T11:22:51.806211+0000","last_undegraded":"2024-03-21T11:22:51.806211+0000","last_fullsized":"2024-03-21T11:22:51.806211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:56:18.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.854441+0000","last_change":"2024-03-21T11:20:53.748050+0000","last_active":"2024-03-21T11:22:50.854441+0000","last_peered":"2024-03-21T11:22:50.854441+0000","last_clean":"2024-03-21T11:22:50.854441+0000","last_became_active":"2024-03-21T11:20:53.747636+0000","last_became_peered":"2024-03-21T11:20:53.747636+0000","last_unstale":"2024-03-21T11:22:50.854441+0000","last_undegraded":"2024-03-21T11:22:50.854441+0000","last_fullsized":"2024-03-21T11:22:50.854441+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:09:15.202183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921814+0000","last_change":"2024-03-21T11:20:48.156928+0000","last_active":"2024-03-21T11:22:50.921814+0000","last_peered":"2024-03-21T11:22:50.921814+0000","last_clean":"2024-03-21T11:22:50.921814+0000","last_became_active":"2024-03-21T11:20:48.156625+0000","last_became_peered":"2024-03-21T11:20:48.156625+0000","last_unstale":"2024-03-21T11:22:50.921814+0000","last_undegraded":"2024-03-21T11:22:50.921814+0000","last_fullsized":"2024-03-21T11:22:50.921814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:19:42.035565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805952+0000","last_change":"2024-03-21T11:20:48.663655+0000","last_active":"2024-03-21T11:22:51.805952+0000","last_peered":"2024-03-21T11:22:51.805952+0000","last_clean":"2024-03-21T11:22:51.805952+0000","last_became_active":"2024-03-21T11:20:48.663336+0000","last_became_peered":"2024-03-21T11:20:48.663336+0000","last_unstale":"2024-03-21T11:22:51.805952+0000","last_undegraded":"2024-03-21T11:22:51.805952+0000","last_fullsized":"2024-03-21T11:22:51.805952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:29:05.119283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.920960+0000","last_change":"2024-03-21T11:20:53.732584+0000","last_active":"2024-03-21T11:22:50.920960+0000","last_peered":"2024-03-21T11:22:50.920960+0000","last_clean":"2024-03-21T11:22:50.920960+0000","last_became_active":"2024-03-21T11:20:53.731975+0000","last_became_peered":"2024-03-21T11:20:53.731975+0000","last_unstale":"2024-03-21T11:22:50.920960+0000","last_undegraded":"2024-03-21T11:22:50.920960+0000","last_fullsized":"2024-03-21T11:22:50.920960+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:51:06.336241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755235+0000","last_change":"2024-03-21T11:20:48.165594+0000","last_active":"2024-03-21T11:22:51.755235+0000","last_peered":"2024-03-21T11:22:51.755235+0000","last_clean":"2024-03-21T11:22:51.755235+0000","last_became_active":"2024-03-21T11:20:48.164394+0000","last_became_peered":"2024-03-21T11:20:48.164394+0000","last_unstale":"2024-03-21T11:22:51.755235+0000","last_undegraded":"2024-03-21T11:22:51.755235+0000","last_fullsized":"2024-03-21T11:22:51.755235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:13.317767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806214+0000","last_change":"2024-03-21T11:20:48.649359+0000","last_active":"2024-03-21T11:22:51.806214+0000","last_peered":"2024-03-21T11:22:51.806214+0000","last_clean":"2024-03-21T11:22:51.806214+0000","last_became_active":"2024-03-21T11:20:48.649031+0000","last_became_peered":"2024-03-21T11:20:48.649031+0000","last_unstale":"2024-03-21T11:22:51.806214+0000","last_undegraded":"2024-03-21T11:22:51.806214+0000","last_fullsized":"2024-03-21T11:22:51.806214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:17.055732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560206+0000","last_change":"2024-03-21T11:20:48.654999+0000","last_active":"2024-03-21T11:22:51.560206+0000","last_peered":"2024-03-21T11:22:51.560206+0000","last_clean":"2024-03-21T11:22:51.560206+0000","last_became_active":"2024-03-21T11:20:48.654638+0000","last_became_peered":"2024-03-21T11:20:48.654638+0000","last_unstale":"2024-03-21T11:22:51.560206+0000","last_undegraded":"2024-03-21T11:22:51.560206+0000","last_fullsized":"2024-03-21T11:22:51.560206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:49:48.711420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852998+0000","last_change":"2024-03-21T11:20:48.651864+0000","last_active":"2024-03-21T11:22:50.852998+0000","last_peered":"2024-03-21T11:22:50.852998+0000","last_clean":"2024-03-21T11:22:50.852998+0000","last_became_active":"2024-03-21T11:20:48.651565+0000","last_became_peered":"2024-03-21T11:20:48.651565+0000","last_unstale":"2024-03-21T11:22:50.852998+0000","last_undegraded":"2024-03-21T11:22:50.852998+0000","last_fullsized":"2024-03-21T11:22:50.852998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:58.659889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853152+0000","last_change":"2024-03-21T11:20:48.667038+0000","last_active":"2024-03-21T11:22:50.853152+0000","last_peered":"2024-03-21T11:22:50.853152+0000","last_clean":"2024-03-21T11:22:50.853152+0000","last_became_active":"2024-03-21T11:20:48.666570+0000","last_became_peered":"2024-03-21T11:20:48.666570+0000","last_unstale":"2024-03-21T11:22:50.853152+0000","last_undegraded":"2024-03-21T11:22:50.853152+0000","last_fullsized":"2024-03-21T11:22:50.853152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:36:39.043697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855033+0000","last_change":"2024-03-21T11:21:50.658024+0000","last_active":"2024-03-21T11:22:50.855033+0000","last_peered":"2024-03-21T11:22:50.855033+0000","last_clean":"2024-03-21T11:22:50.855033+0000","last_became_active":"2024-03-21T11:21:50.657390+0000","last_became_peered":"2024-03-21T11:21:50.657390+0000","last_unstale":"2024-03-21T11:22:50.855033+0000","last_undegraded":"2024-03-21T11:22:50.855033+0000","last_fullsized":"2024-03-21T11:22:50.855033+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:29.578366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755343+0000","last_change":"2024-03-21T11:20:48.670139+0000","last_active":"2024-03-21T11:22:51.755343+0000","last_peered":"2024-03-21T11:22:51.755343+0000","last_clean":"2024-03-21T11:22:51.755343+0000","last_became_active":"2024-03-21T11:20:48.669410+0000","last_became_peered":"2024-03-21T11:20:48.669410+0000","last_unstale":"2024-03-21T11:22:51.755343+0000","last_undegraded":"2024-03-21T11:22:51.755343+0000","last_fullsized":"2024-03-21T11:22:51.755343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:35:18.799840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560440+0000","last_change":"2024-03-21T11:20:48.655356+0000","last_active":"2024-03-21T11:22:51.560440+0000","last_peered":"2024-03-21T11:22:51.560440+0000","last_clean":"2024-03-21T11:22:51.560440+0000","last_became_active":"2024-03-21T11:20:48.655028+0000","last_became_peered":"2024-03-21T11:20:48.655028+0000","last_unstale":"2024-03-21T11:22:51.560440+0000","last_undegraded":"2024-03-21T11:22:51.560440+0000","last_fullsized":"2024-03-21T11:22:51.560440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:46:39.948982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853238+0000","last_change":"2024-03-21T11:20:48.648840+0000","last_active":"2024-03-21T11:22:50.853238+0000","last_peered":"2024-03-21T11:22:50.853238+0000","last_clean":"2024-03-21T11:22:50.853238+0000","last_became_active":"2024-03-21T11:20:48.648502+0000","last_became_peered":"2024-03-21T11:20:48.648502+0000","last_unstale":"2024-03-21T11:22:50.853238+0000","last_undegraded":"2024-03-21T11:22:50.853238+0000","last_fullsized":"2024-03-21T11:22:50.853238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:33:14.824880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.850295+0000","last_change":"2024-03-21T11:20:48.672015+0000","last_active":"2024-03-21T11:22:50.850295+0000","last_peered":"2024-03-21T11:22:50.850295+0000","last_clean":"2024-03-21T11:22:50.850295+0000","last_became_active":"2024-03-21T11:20:48.671663+0000","last_became_peered":"2024-03-21T11:20:48.671663+0000","last_unstale":"2024-03-21T11:22:50.850295+0000","last_undegraded":"2024-03-21T11:22:50.850295+0000","last_fullsized":"2024-03-21T11:22:50.850295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:21:25.328698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921220+0000","last_change":"2024-03-21T11:20:48.664114+0000","last_active":"2024-03-21T11:22:50.921220+0000","last_peered":"2024-03-21T11:22:50.921220+0000","last_clean":"2024-03-21T11:22:50.921220+0000","last_became_active":"2024-03-21T11:20:48.663819+0000","last_became_peered":"2024-03-21T11:20:48.663819+0000","last_unstale":"2024-03-21T11:22:50.921220+0000","last_undegraded":"2024-03-21T11:22:50.921220+0000","last_fullsized":"2024-03-21T11:22:50.921220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:08:08.201383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848874+0000","last_change":"2024-03-21T11:20:48.671336+0000","last_active":"2024-03-21T11:22:50.848874+0000","last_peered":"2024-03-21T11:22:50.848874+0000","last_clean":"2024-03-21T11:22:50.848874+0000","last_became_active":"2024-03-21T11:20:48.670918+0000","last_became_peered":"2024-03-21T11:20:48.670918+0000","last_unstale":"2024-03-21T11:22:50.848874+0000","last_undegraded":"2024-03-21T11:22:50.848874+0000","last_fullsized":"2024-03-21T11:22:50.848874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:01:44.764584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129593+0000","last_change":"2024-03-21T11:20:48.659339+0000","last_active":"2024-03-21T11:22:51.129593+0000","last_peered":"2024-03-21T11:22:51.129593+0000","last_clean":"2024-03-21T11:22:51.129593+0000","last_became_active":"2024-03-21T11:20:48.658989+0000","last_became_peered":"2024-03-21T11:20:48.658989+0000","last_unstale":"2024-03-21T11:22:51.129593+0000","last_undegraded":"2024-03-21T11:22:51.129593+0000","last_fullsized":"2024-03-21T11:22:51.129593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:44:09.599836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755291+0000","last_change":"2024-03-21T11:20:48.668679+0000","last_active":"2024-03-21T11:22:51.755291+0000","last_peered":"2024-03-21T11:22:51.755291+0000","last_clean":"2024-03-21T11:22:51.755291+0000","last_became_active":"2024-03-21T11:20:48.667956+0000","last_became_peered":"2024-03-21T11:20:48.667956+0000","last_unstale":"2024-03-21T11:22:51.755291+0000","last_undegraded":"2024-03-21T11:22:51.755291+0000","last_fullsized":"2024-03-21T11:22:51.755291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:04.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560270+0000","last_change":"2024-03-21T11:21:50.650600+0000","last_active":"2024-03-21T11:22:51.560270+0000","last_peered":"2024-03-21T11:22:51.560270+0000","last_clean":"2024-03-21T11:22:51.560270+0000","last_became_active":"2024-03-21T11:21:50.650221+0000","last_became_peered":"2024-03-21T11:21:50.650221+0000","last_unstale":"2024-03-21T11:22:51.560270+0000","last_undegraded":"2024-03-21T11:22:51.560270+0000","last_fullsized":"2024-03-21T11:22:51.560270+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:29:52.245281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247498,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79240,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1476,"num_read_kb":824,"num_write":267,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":231956,"data_compressed":5487,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351026,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":531977,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612663,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32356,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712796,"statfs":{"total":95995035648,"available":95961903104,"internally_reserved":0,"allocated":1208320,"data_stored":1193414,"data_compressed":64722,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776188,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":535471,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037827,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":531753,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299464,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":1129620,"data_compressed":62893,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462990,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":527270,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":598825,"data_compressed":28661,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":528867,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":594260,"data_compressed":28661,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149790,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":526587,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888152,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":531485,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724627,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":1120183,"data_compressed":62893,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72668,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68103,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":72793,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T11:23:45.076 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T11:23:45.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:44 smithi100 ceph-mon[36245]: pgmap v412: 129 pgs: 129 active+clean; 654 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-21T11:23:45.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:44 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/873403839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:45.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:44 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/961467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:23:45.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:44 smithi047 ceph-mon[31351]: pgmap v412: 129 pgs: 129 active+clean; 654 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-21T11:23:45.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:44 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/873403839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:45.196 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:44 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/961467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:23:45.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:44 smithi102 ceph-mon[36748]: pgmap v412: 129 pgs: 129 active+clean; 654 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-21T11:23:45.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:44 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/873403839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:45.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:44 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/961467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:23:45.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.334+0000 7f1bcf000640 1 -- 172.21.15.47:0/4244240109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc81048e0 msgr2=0x7f1bc8104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:45.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.334+0000 7f1bcf000640 1 --2- 172.21.15.47:0/4244240109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc81048e0 0x7f1bc8104d60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009900 tx=0x7f1bb402bc50 comp rx=0 tx=0).stop 2024-03-21T11:23:45.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.334+0000 7f1bcf000640 1 -- 172.21.15.47:0/4244240109 shutdown_connections 2024-03-21T11:23:45.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.334+0000 7f1bcf000640 1 --2- 172.21.15.47:0/4244240109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc81052a0 0x7f1bc810bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.334+0000 7f1bcf000640 1 --2- 172.21.15.47:0/4244240109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc81048e0 0x7f1bc8104d60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.334+0000 7f1bcf000640 1 --2- 172.21.15.47:0/4244240109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bc81036f0 0x7f1bc8103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.334+0000 7f1bcf000640 1 -- 172.21.15.47:0/4244240109 >> 172.21.15.47:0/4244240109 conn(0x7f1bc80fec80 msgr2=0x7f1bc81010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:45.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.334+0000 7f1bcf000640 1 -- 172.21.15.47:0/4244240109 shutdown_connections 2024-03-21T11:23:45.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.334+0000 7f1bcf000640 1 -- 172.21.15.47:0/4244240109 wait complete. 2024-03-21T11:23:45.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.335+0000 7f1bcf000640 1 Processor -- start 2024-03-21T11:23:45.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.336+0000 7f1bcf000640 1 -- start start 2024-03-21T11:23:45.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.336+0000 7f1bcf000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc81036f0 0x7f1bc8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:45.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.336+0000 7f1bcf000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc81048e0 0x7f1bc8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:45.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.336+0000 7f1bcf000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bc81052a0 0x7f1bc81a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:45.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.336+0000 7f1bcf000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc810f370 con 0x7f1bc81052a0 2024-03-21T11:23:45.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.336+0000 7f1bcf000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc810f1f0 con 0x7f1bc81048e0 2024-03-21T11:23:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.336+0000 7f1bcf000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc810f4f0 con 0x7f1bc81036f0 2024-03-21T11:23:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.336+0000 7f1bcdc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc81036f0 0x7f1bc8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.336+0000 7f1bcd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc81048e0 0x7f1bc8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.336+0000 7f1bcdc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc81036f0 0x7f1bc8199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51948/0 (socket says 172.21.15.47:51948) 2024-03-21T11:23:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.336+0000 7f1bcd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc81048e0 0x7f1bc8199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38714/0 (socket says 172.21.15.47:38714) 2024-03-21T11:23:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.336+0000 7f1bcdc00640 1 -- 172.21.15.47:0/3698760123 learned_addr learned my addr 172.21.15.47:0/3698760123 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.336+0000 7f1bce600640 1 --2- 172.21.15.47:0/3698760123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bc81052a0 0x7f1bc81a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.337+0000 7f1bcdc00640 1 -- 172.21.15.47:0/3698760123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc81048e0 msgr2=0x7f1bc8199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.337+0000 7f1bcdc00640 1 --2- 172.21.15.47:0/3698760123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc81048e0 0x7f1bc8199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.337+0000 7f1bcdc00640 1 -- 172.21.15.47:0/3698760123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bc81052a0 msgr2=0x7f1bc81a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.337+0000 7f1bcdc00640 1 --2- 172.21.15.47:0/3698760123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bc81052a0 0x7f1bc81a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.337+0000 7f1bcdc00640 1 -- 172.21.15.47:0/3698760123 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc81a1650 con 0x7f1bc81036f0 2024-03-21T11:23:45.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.337+0000 7f1bcdc00640 1 --2- 172.21.15.47:0/3698760123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc81036f0 0x7f1bc8199a30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800ebc0 tx=0x7f1bb800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:45.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.337+0000 7f1bc6000640 1 -- 172.21.15.47:0/3698760123 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8003b70 con 0x7f1bc81036f0 2024-03-21T11:23:45.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.338+0000 7f1bc6000640 1 -- 172.21.15.47:0/3698760123 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bb8003d10 con 0x7f1bc81036f0 2024-03-21T11:23:45.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.338+0000 7f1bcf000640 1 -- 172.21.15.47:0/3698760123 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc81a1940 con 0x7f1bc81036f0 2024-03-21T11:23:45.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.338+0000 7f1bc6000640 1 -- 172.21.15.47:0/3698760123 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8019980 con 0x7f1bc81036f0 2024-03-21T11:23:45.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.338+0000 7f1bcf000640 1 -- 172.21.15.47:0/3698760123 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc81a1e60 con 0x7f1bc81036f0 2024-03-21T11:23:45.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.339+0000 7f1bcf000640 1 -- 172.21.15.47:0/3698760123 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc810aa10 con 0x7f1bc81036f0 2024-03-21T11:23:45.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.341+0000 7f1bc6000640 1 -- 172.21.15.47:0/3698760123 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f1bb8019b20 con 0x7f1bc81036f0 2024-03-21T11:23:45.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.343+0000 7f1bc6000640 1 --2- 172.21.15.47:0/3698760123 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1bac0a2fb0 0x7f1bac0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:45.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.343+0000 7f1bc6000640 1 -- 172.21.15.47:0/3698760123 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1bb80d0180 con 0x7f1bc81036f0 2024-03-21T11:23:45.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.343+0000 7f1bcd200640 1 --2- 172.21.15.47:0/3698760123 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1bac0a2fb0 0x7f1bac0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:45.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.344+0000 7f1bcd200640 1 --2- 172.21.15.47:0/3698760123 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1bac0a2fb0 0x7f1bac0a5470 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1bb402c7d0 tx=0x7f1bb4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:45.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.347+0000 7f1bc6000640 1 -- 172.21.15.47:0/3698760123 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1bb809ca60 con 0x7f1bc81036f0 2024-03-21T11:23:45.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.548+0000 7f1bcf000640 1 -- 172.21.15.47:0/3698760123 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1bc8061ae0 con 0x7f1bac0a2fb0 2024-03-21T11:23:45.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.578+0000 7f1bc6000640 1 -- 172.21.15.47:0/3698760123 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340844 (secure 0 0 0) 0x7f1bc8061ae0 con 0x7f1bac0a2fb0 2024-03-21T11:23:45.584 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:45.586 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-21T11:23:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.587+0000 7f1bcf000640 1 -- 172.21.15.47:0/3698760123 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1bac0a2fb0 msgr2=0x7f1bac0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.587+0000 7f1bcf000640 1 --2- 172.21.15.47:0/3698760123 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1bac0a2fb0 0x7f1bac0a5470 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1bb402c7d0 tx=0x7f1bb4005fd0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.587+0000 7f1bcf000640 1 -- 172.21.15.47:0/3698760123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc81036f0 msgr2=0x7f1bc8199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.587+0000 7f1bcf000640 1 --2- 172.21.15.47:0/3698760123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc81036f0 0x7f1bc8199a30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800ebc0 tx=0x7f1bb800c6a0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.587+0000 7f1bcf000640 1 -- 172.21.15.47:0/3698760123 shutdown_connections 2024-03-21T11:23:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.587+0000 7f1bcf000640 1 --2- 172.21.15.47:0/3698760123 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1bac0a2fb0 0x7f1bac0a5470 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.587+0000 7f1bcf000640 1 --2- 172.21.15.47:0/3698760123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bc81052a0 0x7f1bc81a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.587+0000 7f1bcf000640 1 --2- 172.21.15.47:0/3698760123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc81048e0 0x7f1bc8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.587+0000 7f1bcf000640 1 --2- 172.21.15.47:0/3698760123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc81036f0 0x7f1bc8199a30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.587+0000 7f1bcf000640 1 -- 172.21.15.47:0/3698760123 >> 172.21.15.47:0/3698760123 conn(0x7f1bc80fec80 msgr2=0x7f1bc8107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.587+0000 7f1bcf000640 1 -- 172.21.15.47:0/3698760123 shutdown_connections 2024-03-21T11:23:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.587+0000 7f1bcf000640 1 -- 172.21.15.47:0/3698760123 wait complete. 2024-03-21T11:23:45.607 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-03-21T11:23:45.060482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669842,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1607,"num_read_kb":933,"num_write":450,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378672,"kb_used_data":12848,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563152,"statfs":{"total":1151940427776,"available":1151552667648,"internally_reserved":0,"allocated":13156352,"data_stored":8357996,"data_compressed":435654,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6767,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":59,"num_write":13,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004592"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560653+0000","last_change":"2024-03-21T11:20:48.667185+0000","last_active":"2024-03-21T11:22:51.560653+0000","last_peered":"2024-03-21T11:22:51.560653+0000","last_clean":"2024-03-21T11:22:51.560653+0000","last_became_active":"2024-03-21T11:20:48.666855+0000","last_became_peered":"2024-03-21T11:20:48.666855+0000","last_unstale":"2024-03-21T11:22:51.560653+0000","last_undegraded":"2024-03-21T11:22:51.560653+0000","last_fullsized":"2024-03-21T11:22:51.560653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:30:37.393096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855276+0000","last_change":"2024-03-21T11:20:48.671613+0000","last_active":"2024-03-21T11:22:50.855276+0000","last_peered":"2024-03-21T11:22:50.855276+0000","last_clean":"2024-03-21T11:22:50.855276+0000","last_became_active":"2024-03-21T11:20:48.671211+0000","last_became_peered":"2024-03-21T11:20:48.671211+0000","last_unstale":"2024-03-21T11:22:50.855276+0000","last_undegraded":"2024-03-21T11:22:50.855276+0000","last_fullsized":"2024-03-21T11:22:50.855276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:20.167922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.854836+0000","last_change":"2024-03-21T11:20:48.655734+0000","last_active":"2024-03-21T11:22:50.854836+0000","last_peered":"2024-03-21T11:22:50.854836+0000","last_clean":"2024-03-21T11:22:50.854836+0000","last_became_active":"2024-03-21T11:20:48.653798+0000","last_became_peered":"2024-03-21T11:20:48.653798+0000","last_unstale":"2024-03-21T11:22:50.854836+0000","last_undegraded":"2024-03-21T11:22:50.854836+0000","last_fullsized":"2024-03-21T11:22:50.854836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:44.796405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847015+0000","last_change":"2024-03-21T11:20:48.648784+0000","last_active":"2024-03-21T11:22:50.847015+0000","last_peered":"2024-03-21T11:22:50.847015+0000","last_clean":"2024-03-21T11:22:50.847015+0000","last_became_active":"2024-03-21T11:20:48.648040+0000","last_became_peered":"2024-03-21T11:20:48.648040+0000","last_unstale":"2024-03-21T11:22:50.847015+0000","last_undegraded":"2024-03-21T11:22:50.847015+0000","last_fullsized":"2024-03-21T11:22:50.847015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:48:51.786346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846582+0000","last_change":"2024-03-21T11:20:48.672084+0000","last_active":"2024-03-21T11:22:50.846582+0000","last_peered":"2024-03-21T11:22:50.846582+0000","last_clean":"2024-03-21T11:22:50.846582+0000","last_became_active":"2024-03-21T11:20:48.671441+0000","last_became_peered":"2024-03-21T11:20:48.671441+0000","last_unstale":"2024-03-21T11:22:50.846582+0000","last_undegraded":"2024-03-21T11:22:50.846582+0000","last_fullsized":"2024-03-21T11:22:50.846582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:20:17.638270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848462+0000","last_change":"2024-03-21T11:20:48.659782+0000","last_active":"2024-03-21T11:22:50.848462+0000","last_peered":"2024-03-21T11:22:50.848462+0000","last_clean":"2024-03-21T11:22:50.848462+0000","last_became_active":"2024-03-21T11:20:48.659460+0000","last_became_peered":"2024-03-21T11:20:48.659460+0000","last_unstale":"2024-03-21T11:22:50.848462+0000","last_undegraded":"2024-03-21T11:22:50.848462+0000","last_fullsized":"2024-03-21T11:22:50.848462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:57.712654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805934+0000","last_change":"2024-03-21T11:20:48.659401+0000","last_active":"2024-03-21T11:22:51.805934+0000","last_peered":"2024-03-21T11:22:51.805934+0000","last_clean":"2024-03-21T11:22:51.805934+0000","last_became_active":"2024-03-21T11:20:48.659075+0000","last_became_peered":"2024-03-21T11:20:48.659075+0000","last_unstale":"2024-03-21T11:22:51.805934+0000","last_undegraded":"2024-03-21T11:22:51.805934+0000","last_fullsized":"2024-03-21T11:22:51.805934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:54.342236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847869+0000","last_change":"2024-03-21T11:20:48.667175+0000","last_active":"2024-03-21T11:22:50.847869+0000","last_peered":"2024-03-21T11:22:50.847869+0000","last_clean":"2024-03-21T11:22:50.847869+0000","last_became_active":"2024-03-21T11:20:48.666756+0000","last_became_peered":"2024-03-21T11:20:48.666756+0000","last_unstale":"2024-03-21T11:22:50.847869+0000","last_undegraded":"2024-03-21T11:22:50.847869+0000","last_fullsized":"2024-03-21T11:22:50.847869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:25:39.694699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129870+0000","last_change":"2024-03-21T11:20:48.650584+0000","last_active":"2024-03-21T11:22:51.129870+0000","last_peered":"2024-03-21T11:22:51.129870+0000","last_clean":"2024-03-21T11:22:51.129870+0000","last_became_active":"2024-03-21T11:20:48.650251+0000","last_became_peered":"2024-03-21T11:20:48.650251+0000","last_unstale":"2024-03-21T11:22:51.129870+0000","last_undegraded":"2024-03-21T11:22:51.129870+0000","last_fullsized":"2024-03-21T11:22:51.129870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:35:59.000522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848842+0000","last_change":"2024-03-21T11:20:48.652945+0000","last_active":"2024-03-21T11:22:50.848842+0000","last_peered":"2024-03-21T11:22:50.848842+0000","last_clean":"2024-03-21T11:22:50.848842+0000","last_became_active":"2024-03-21T11:20:48.652626+0000","last_became_peered":"2024-03-21T11:20:48.652626+0000","last_unstale":"2024-03-21T11:22:50.848842+0000","last_undegraded":"2024-03-21T11:22:50.848842+0000","last_fullsized":"2024-03-21T11:22:50.848842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:43.586907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806192+0000","last_change":"2024-03-21T11:20:48.651520+0000","last_active":"2024-03-21T11:22:51.806192+0000","last_peered":"2024-03-21T11:22:51.806192+0000","last_clean":"2024-03-21T11:22:51.806192+0000","last_became_active":"2024-03-21T11:20:48.651210+0000","last_became_peered":"2024-03-21T11:20:48.651210+0000","last_unstale":"2024-03-21T11:22:51.806192+0000","last_undegraded":"2024-03-21T11:22:51.806192+0000","last_fullsized":"2024-03-21T11:22:51.806192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:29.506732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846902+0000","last_change":"2024-03-21T11:20:48.649844+0000","last_active":"2024-03-21T11:22:50.846902+0000","last_peered":"2024-03-21T11:22:50.846902+0000","last_clean":"2024-03-21T11:22:50.846902+0000","last_became_active":"2024-03-21T11:20:48.649503+0000","last_became_peered":"2024-03-21T11:20:48.649503+0000","last_unstale":"2024-03-21T11:22:50.846902+0000","last_undegraded":"2024-03-21T11:22:50.846902+0000","last_fullsized":"2024-03-21T11:22:50.846902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:31:55.734583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755540+0000","last_change":"2024-03-21T11:20:48.662012+0000","last_active":"2024-03-21T11:22:51.755540+0000","last_peered":"2024-03-21T11:22:51.755540+0000","last_clean":"2024-03-21T11:22:51.755540+0000","last_became_active":"2024-03-21T11:20:48.661444+0000","last_became_peered":"2024-03-21T11:20:48.661444+0000","last_unstale":"2024-03-21T11:22:51.755540+0000","last_undegraded":"2024-03-21T11:22:51.755540+0000","last_fullsized":"2024-03-21T11:22:51.755540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:52.493244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.854380+0000","last_change":"2024-03-21T11:20:48.663087+0000","last_active":"2024-03-21T11:22:50.854380+0000","last_peered":"2024-03-21T11:22:50.854380+0000","last_clean":"2024-03-21T11:22:50.854380+0000","last_became_active":"2024-03-21T11:20:48.662039+0000","last_became_peered":"2024-03-21T11:20:48.662039+0000","last_unstale":"2024-03-21T11:22:50.854380+0000","last_undegraded":"2024-03-21T11:22:50.854380+0000","last_fullsized":"2024-03-21T11:22:50.854380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:47:01.962314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.854313+0000","last_change":"2024-03-21T11:20:48.663478+0000","last_active":"2024-03-21T11:22:50.854313+0000","last_peered":"2024-03-21T11:22:50.854313+0000","last_clean":"2024-03-21T11:22:50.854313+0000","last_became_active":"2024-03-21T11:20:48.663135+0000","last_became_peered":"2024-03-21T11:20:48.663135+0000","last_unstale":"2024-03-21T11:22:50.854313+0000","last_undegraded":"2024-03-21T11:22:50.854313+0000","last_fullsized":"2024-03-21T11:22:50.854313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:29.139760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129288+0000","last_change":"2024-03-21T11:20:48.653893+0000","last_active":"2024-03-21T11:22:51.129288+0000","last_peered":"2024-03-21T11:22:51.129288+0000","last_clean":"2024-03-21T11:22:51.129288+0000","last_became_active":"2024-03-21T11:20:48.653578+0000","last_became_peered":"2024-03-21T11:20:48.653578+0000","last_unstale":"2024-03-21T11:22:51.129288+0000","last_undegraded":"2024-03-21T11:22:51.129288+0000","last_fullsized":"2024-03-21T11:22:51.129288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:54:31.254030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560545+0000","last_change":"2024-03-21T11:20:48.667388+0000","last_active":"2024-03-21T11:22:51.560545+0000","last_peered":"2024-03-21T11:22:51.560545+0000","last_clean":"2024-03-21T11:22:51.560545+0000","last_became_active":"2024-03-21T11:20:48.667077+0000","last_became_peered":"2024-03-21T11:20:48.667077+0000","last_unstale":"2024-03-21T11:22:51.560545+0000","last_undegraded":"2024-03-21T11:22:51.560545+0000","last_fullsized":"2024-03-21T11:22:51.560545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:13:32.093457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853273+0000","last_change":"2024-03-21T11:20:48.658479+0000","last_active":"2024-03-21T11:22:50.853273+0000","last_peered":"2024-03-21T11:22:50.853273+0000","last_clean":"2024-03-21T11:22:50.853273+0000","last_became_active":"2024-03-21T11:20:48.658011+0000","last_became_peered":"2024-03-21T11:20:48.658011+0000","last_unstale":"2024-03-21T11:22:50.853273+0000","last_undegraded":"2024-03-21T11:22:50.853273+0000","last_fullsized":"2024-03-21T11:22:50.853273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:33.313537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853403+0000","last_change":"2024-03-21T11:20:48.668648+0000","last_active":"2024-03-21T11:22:50.853403+0000","last_peered":"2024-03-21T11:22:50.853403+0000","last_clean":"2024-03-21T11:22:50.853403+0000","last_became_active":"2024-03-21T11:20:48.668376+0000","last_became_peered":"2024-03-21T11:20:48.668376+0000","last_unstale":"2024-03-21T11:22:50.853403+0000","last_undegraded":"2024-03-21T11:22:50.853403+0000","last_fullsized":"2024-03-21T11:22:50.853403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:49:47.601724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852874+0000","last_change":"2024-03-21T11:20:53.755600+0000","last_active":"2024-03-21T11:22:50.852874+0000","last_peered":"2024-03-21T11:22:50.852874+0000","last_clean":"2024-03-21T11:22:50.852874+0000","last_became_active":"2024-03-21T11:20:53.755321+0000","last_became_peered":"2024-03-21T11:20:53.755321+0000","last_unstale":"2024-03-21T11:22:50.852874+0000","last_undegraded":"2024-03-21T11:22:50.852874+0000","last_fullsized":"2024-03-21T11:22:50.852874+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:04.905910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":193,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.907144+0000","last_change":"2024-03-21T11:20:48.161001+0000","last_active":"2024-03-21T11:23:40.907144+0000","last_peered":"2024-03-21T11:23:40.907144+0000","last_clean":"2024-03-21T11:23:40.907144+0000","last_became_active":"2024-03-21T11:20:48.153053+0000","last_became_peered":"2024-03-21T11:20:48.153053+0000","last_unstale":"2024-03-21T11:23:40.907144+0000","last_undegraded":"2024-03-21T11:23:40.907144+0000","last_fullsized":"2024-03-21T11:23:40.907144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:26.010581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849345+0000","last_change":"2024-03-21T11:20:48.671430+0000","last_active":"2024-03-21T11:22:50.849345+0000","last_peered":"2024-03-21T11:22:50.849345+0000","last_clean":"2024-03-21T11:22:50.849345+0000","last_became_active":"2024-03-21T11:20:48.670192+0000","last_became_peered":"2024-03-21T11:20:48.670192+0000","last_unstale":"2024-03-21T11:22:50.849345+0000","last_undegraded":"2024-03-21T11:22:50.849345+0000","last_fullsized":"2024-03-21T11:22:50.849345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:00:53.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846554+0000","last_change":"2024-03-21T11:20:53.750313+0000","last_active":"2024-03-21T11:22:50.846554+0000","last_peered":"2024-03-21T11:22:50.846554+0000","last_clean":"2024-03-21T11:22:50.846554+0000","last_became_active":"2024-03-21T11:20:53.749913+0000","last_became_peered":"2024-03-21T11:20:53.749913+0000","last_unstale":"2024-03-21T11:22:50.846554+0000","last_undegraded":"2024-03-21T11:22:50.846554+0000","last_fullsized":"2024-03-21T11:22:50.846554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:39:09.007674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849723+0000","last_change":"2024-03-21T11:20:48.159934+0000","last_active":"2024-03-21T11:22:50.849723+0000","last_peered":"2024-03-21T11:22:50.849723+0000","last_clean":"2024-03-21T11:22:50.849723+0000","last_became_active":"2024-03-21T11:20:48.159650+0000","last_became_peered":"2024-03-21T11:20:48.159650+0000","last_unstale":"2024-03-21T11:22:50.849723+0000","last_undegraded":"2024-03-21T11:22:50.849723+0000","last_fullsized":"2024-03-21T11:22:50.849723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:49:56.891733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855277+0000","last_change":"2024-03-21T11:20:48.673012+0000","last_active":"2024-03-21T11:22:50.855277+0000","last_peered":"2024-03-21T11:22:50.855277+0000","last_clean":"2024-03-21T11:22:50.855277+0000","last_became_active":"2024-03-21T11:20:48.672584+0000","last_became_peered":"2024-03-21T11:20:48.672584+0000","last_unstale":"2024-03-21T11:22:50.855277+0000","last_undegraded":"2024-03-21T11:22:50.855277+0000","last_fullsized":"2024-03-21T11:22:50.855277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:48.213569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846699+0000","last_change":"2024-03-21T11:21:50.655957+0000","last_active":"2024-03-21T11:22:50.846699+0000","last_peered":"2024-03-21T11:22:50.846699+0000","last_clean":"2024-03-21T11:22:50.846699+0000","last_became_active":"2024-03-21T11:21:50.655477+0000","last_became_peered":"2024-03-21T11:21:50.655477+0000","last_unstale":"2024-03-21T11:22:50.846699+0000","last_undegraded":"2024-03-21T11:22:50.846699+0000","last_fullsized":"2024-03-21T11:22:50.846699+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:58:28.476186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'13","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560446+0000","last_change":"2024-03-21T11:21:50.664575+0000","last_active":"2024-03-21T11:22:51.560446+0000","last_peered":"2024-03-21T11:22:51.560446+0000","last_clean":"2024-03-21T11:22:51.560446+0000","last_became_active":"2024-03-21T11:21:50.647335+0000","last_became_peered":"2024-03-21T11:21:50.647335+0000","last_unstale":"2024-03-21T11:22:51.560446+0000","last_undegraded":"2024-03-21T11:22:51.560446+0000","last_fullsized":"2024-03-21T11:22:51.560446+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:50:28.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129559+0000","last_change":"2024-03-21T11:20:48.662392+0000","last_active":"2024-03-21T11:22:51.129559+0000","last_peered":"2024-03-21T11:22:51.129559+0000","last_clean":"2024-03-21T11:22:51.129559+0000","last_became_active":"2024-03-21T11:20:48.662042+0000","last_became_peered":"2024-03-21T11:20:48.662042+0000","last_unstale":"2024-03-21T11:22:51.129559+0000","last_undegraded":"2024-03-21T11:22:51.129559+0000","last_fullsized":"2024-03-21T11:22:51.129559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:59:34.456582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846900+0000","last_change":"2024-03-21T11:20:53.749693+0000","last_active":"2024-03-21T11:22:50.846900+0000","last_peered":"2024-03-21T11:22:50.846900+0000","last_clean":"2024-03-21T11:22:50.846900+0000","last_became_active":"2024-03-21T11:20:53.749438+0000","last_became_peered":"2024-03-21T11:20:53.749438+0000","last_unstale":"2024-03-21T11:22:50.846900+0000","last_undegraded":"2024-03-21T11:22:50.846900+0000","last_fullsized":"2024-03-21T11:22:50.846900+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:37.714471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846726+0000","last_change":"2024-03-21T11:20:48.132587+0000","last_active":"2024-03-21T11:22:50.846726+0000","last_peered":"2024-03-21T11:22:50.846726+0000","last_clean":"2024-03-21T11:22:50.846726+0000","last_became_active":"2024-03-21T11:20:48.132232+0000","last_became_peered":"2024-03-21T11:20:48.132232+0000","last_unstale":"2024-03-21T11:22:50.846726+0000","last_undegraded":"2024-03-21T11:22:50.846726+0000","last_fullsized":"2024-03-21T11:22:50.846726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:20:45.217545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921559+0000","last_change":"2024-03-21T11:20:48.656881+0000","last_active":"2024-03-21T11:22:50.921559+0000","last_peered":"2024-03-21T11:22:50.921559+0000","last_clean":"2024-03-21T11:22:50.921559+0000","last_became_active":"2024-03-21T11:20:48.656426+0000","last_became_peered":"2024-03-21T11:20:48.656426+0000","last_unstale":"2024-03-21T11:22:50.921559+0000","last_undegraded":"2024-03-21T11:22:50.921559+0000","last_fullsized":"2024-03-21T11:22:50.921559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:06:27.398062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852990+0000","last_change":"2024-03-21T11:20:53.733396+0000","last_active":"2024-03-21T11:22:50.852990+0000","last_peered":"2024-03-21T11:22:50.852990+0000","last_clean":"2024-03-21T11:22:50.852990+0000","last_became_active":"2024-03-21T11:20:53.732985+0000","last_became_peered":"2024-03-21T11:20:53.732985+0000","last_unstale":"2024-03-21T11:22:50.852990+0000","last_undegraded":"2024-03-21T11:22:50.852990+0000","last_fullsized":"2024-03-21T11:22:50.852990+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:54:14.064761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":189,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.854844+0000","last_change":"2024-03-21T11:20:48.133362+0000","last_active":"2024-03-21T11:23:40.854844+0000","last_peered":"2024-03-21T11:23:40.854844+0000","last_clean":"2024-03-21T11:23:40.854844+0000","last_became_active":"2024-03-21T11:20:48.132405+0000","last_became_peered":"2024-03-21T11:20:48.132405+0000","last_unstale":"2024-03-21T11:23:40.854844+0000","last_undegraded":"2024-03-21T11:23:40.854844+0000","last_fullsized":"2024-03-21T11:23:40.854844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:48:52.688948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1460,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":148,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846447+0000","last_change":"2024-03-21T11:20:48.659610+0000","last_active":"2024-03-21T11:22:50.846447+0000","last_peered":"2024-03-21T11:22:50.846447+0000","last_clean":"2024-03-21T11:22:50.846447+0000","last_became_active":"2024-03-21T11:20:48.659257+0000","last_became_peered":"2024-03-21T11:20:48.659257+0000","last_unstale":"2024-03-21T11:22:50.846447+0000","last_undegraded":"2024-03-21T11:22:50.846447+0000","last_fullsized":"2024-03-21T11:22:50.846447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:17:30.744434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852994+0000","last_change":"2024-03-21T11:21:50.656047+0000","last_active":"2024-03-21T11:22:50.852994+0000","last_peered":"2024-03-21T11:22:50.852994+0000","last_clean":"2024-03-21T11:22:50.852994+0000","last_became_active":"2024-03-21T11:21:50.655554+0000","last_became_peered":"2024-03-21T11:21:50.655554+0000","last_unstale":"2024-03-21T11:22:50.852994+0000","last_undegraded":"2024-03-21T11:22:50.852994+0000","last_fullsized":"2024-03-21T11:22:50.852994+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:35:15.858237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848356+0000","last_change":"2024-03-21T11:21:50.695331+0000","last_active":"2024-03-21T11:22:50.848356+0000","last_peered":"2024-03-21T11:22:50.848356+0000","last_clean":"2024-03-21T11:22:50.848356+0000","last_became_active":"2024-03-21T11:21:50.690368+0000","last_became_peered":"2024-03-21T11:21:50.690368+0000","last_unstale":"2024-03-21T11:22:50.848356+0000","last_undegraded":"2024-03-21T11:22:50.848356+0000","last_fullsized":"2024-03-21T11:22:50.848356+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:50:03.835364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921486+0000","last_change":"2024-03-21T11:20:48.671221+0000","last_active":"2024-03-21T11:22:50.921486+0000","last_peered":"2024-03-21T11:22:50.921486+0000","last_clean":"2024-03-21T11:22:50.921486+0000","last_became_active":"2024-03-21T11:20:48.670837+0000","last_became_peered":"2024-03-21T11:20:48.670837+0000","last_unstale":"2024-03-21T11:22:50.921486+0000","last_undegraded":"2024-03-21T11:22:50.921486+0000","last_fullsized":"2024-03-21T11:22:50.921486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:38.680409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560403+0000","last_change":"2024-03-21T11:20:53.747501+0000","last_active":"2024-03-21T11:22:51.560403+0000","last_peered":"2024-03-21T11:22:51.560403+0000","last_clean":"2024-03-21T11:22:51.560403+0000","last_became_active":"2024-03-21T11:20:53.747062+0000","last_became_peered":"2024-03-21T11:20:53.747062+0000","last_unstale":"2024-03-21T11:22:51.560403+0000","last_undegraded":"2024-03-21T11:22:51.560403+0000","last_fullsized":"2024-03-21T11:22:51.560403+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:58:36.904221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.754889+0000","last_change":"2024-03-21T11:20:48.156175+0000","last_active":"2024-03-21T11:22:51.754889+0000","last_peered":"2024-03-21T11:22:51.754889+0000","last_clean":"2024-03-21T11:22:51.754889+0000","last_became_active":"2024-03-21T11:20:48.154271+0000","last_became_peered":"2024-03-21T11:20:48.154271+0000","last_unstale":"2024-03-21T11:22:51.754889+0000","last_undegraded":"2024-03-21T11:22:51.754889+0000","last_fullsized":"2024-03-21T11:22:51.754889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:04.216391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848061+0000","last_change":"2024-03-21T11:20:48.670011+0000","last_active":"2024-03-21T11:22:50.848061+0000","last_peered":"2024-03-21T11:22:50.848061+0000","last_clean":"2024-03-21T11:22:50.848061+0000","last_became_active":"2024-03-21T11:20:48.669016+0000","last_became_peered":"2024-03-21T11:20:48.669016+0000","last_unstale":"2024-03-21T11:22:50.848061+0000","last_undegraded":"2024-03-21T11:22:50.848061+0000","last_fullsized":"2024-03-21T11:22:50.848061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:43:18.906490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855150+0000","last_change":"2024-03-21T11:20:53.751934+0000","last_active":"2024-03-21T11:22:50.855150+0000","last_peered":"2024-03-21T11:22:50.855150+0000","last_clean":"2024-03-21T11:22:50.855150+0000","last_became_active":"2024-03-21T11:20:53.751413+0000","last_became_peered":"2024-03-21T11:20:53.751413+0000","last_unstale":"2024-03-21T11:22:50.855150+0000","last_undegraded":"2024-03-21T11:22:50.855150+0000","last_fullsized":"2024-03-21T11:22:50.855150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:12:33.880376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921223+0000","last_change":"2024-03-21T11:20:48.131796+0000","last_active":"2024-03-21T11:22:50.921223+0000","last_peered":"2024-03-21T11:22:50.921223+0000","last_clean":"2024-03-21T11:22:50.921223+0000","last_became_active":"2024-03-21T11:20:48.131478+0000","last_became_peered":"2024-03-21T11:20:48.131478+0000","last_unstale":"2024-03-21T11:22:50.921223+0000","last_undegraded":"2024-03-21T11:22:50.921223+0000","last_fullsized":"2024-03-21T11:22:50.921223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:59:24.871300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849740+0000","last_change":"2024-03-21T11:20:48.660046+0000","last_active":"2024-03-21T11:22:50.849740+0000","last_peered":"2024-03-21T11:22:50.849740+0000","last_clean":"2024-03-21T11:22:50.849740+0000","last_became_active":"2024-03-21T11:20:48.659621+0000","last_became_peered":"2024-03-21T11:20:48.659621+0000","last_unstale":"2024-03-21T11:22:50.849740+0000","last_undegraded":"2024-03-21T11:22:50.849740+0000","last_fullsized":"2024-03-21T11:22:50.849740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:47:01.923285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848718+0000","last_change":"2024-03-21T11:20:53.754458+0000","last_active":"2024-03-21T11:22:50.848718+0000","last_peered":"2024-03-21T11:22:50.848718+0000","last_clean":"2024-03-21T11:22:50.848718+0000","last_became_active":"2024-03-21T11:20:53.753927+0000","last_became_peered":"2024-03-21T11:20:53.753927+0000","last_unstale":"2024-03-21T11:22:50.848718+0000","last_undegraded":"2024-03-21T11:22:50.848718+0000","last_fullsized":"2024-03-21T11:22:50.848718+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:00:47.392699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'7","reported_seq":327,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.756351+0000","last_change":"2024-03-21T11:20:48.154125+0000","last_active":"2024-03-21T11:23:40.756351+0000","last_peered":"2024-03-21T11:23:40.756351+0000","last_clean":"2024-03-21T11:23:40.756351+0000","last_became_active":"2024-03-21T11:20:48.153857+0000","last_became_peered":"2024-03-21T11:20:48.153857+0000","last_unstale":"2024-03-21T11:23:40.756351+0000","last_undegraded":"2024-03-21T11:23:40.756351+0000","last_fullsized":"2024-03-21T11:23:40.756351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:54:06.525335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":286,"num_read_kb":286,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129808+0000","last_change":"2024-03-21T11:20:48.650143+0000","last_active":"2024-03-21T11:22:51.129808+0000","last_peered":"2024-03-21T11:22:51.129808+0000","last_clean":"2024-03-21T11:22:51.129808+0000","last_became_active":"2024-03-21T11:20:48.649825+0000","last_became_peered":"2024-03-21T11:20:48.649825+0000","last_unstale":"2024-03-21T11:22:51.129808+0000","last_undegraded":"2024-03-21T11:22:51.129808+0000","last_fullsized":"2024-03-21T11:22:51.129808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:09:15.735280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.850023+0000","last_change":"2024-03-21T11:20:53.743006+0000","last_active":"2024-03-21T11:22:50.850023+0000","last_peered":"2024-03-21T11:22:50.850023+0000","last_clean":"2024-03-21T11:22:50.850023+0000","last_became_active":"2024-03-21T11:20:53.742587+0000","last_became_peered":"2024-03-21T11:20:53.742587+0000","last_unstale":"2024-03-21T11:22:50.850023+0000","last_undegraded":"2024-03-21T11:22:50.850023+0000","last_fullsized":"2024-03-21T11:22:50.850023+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:06:06.845791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806093+0000","last_change":"2024-03-21T11:20:48.137137+0000","last_active":"2024-03-21T11:22:51.806093+0000","last_peered":"2024-03-21T11:22:51.806093+0000","last_clean":"2024-03-21T11:22:51.806093+0000","last_became_active":"2024-03-21T11:20:48.136790+0000","last_became_peered":"2024-03-21T11:20:48.136790+0000","last_unstale":"2024-03-21T11:22:51.806093+0000","last_undegraded":"2024-03-21T11:22:51.806093+0000","last_fullsized":"2024-03-21T11:22:51.806093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:22:47.223969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855575+0000","last_change":"2024-03-21T11:20:48.673568+0000","last_active":"2024-03-21T11:22:50.855575+0000","last_peered":"2024-03-21T11:22:50.855575+0000","last_clean":"2024-03-21T11:22:50.855575+0000","last_became_active":"2024-03-21T11:20:48.672613+0000","last_became_peered":"2024-03-21T11:20:48.672613+0000","last_unstale":"2024-03-21T11:22:50.855575+0000","last_undegraded":"2024-03-21T11:22:50.855575+0000","last_fullsized":"2024-03-21T11:22:50.855575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:43:38.193802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846803+0000","last_change":"2024-03-21T11:20:53.739228+0000","last_active":"2024-03-21T11:22:50.846803+0000","last_peered":"2024-03-21T11:22:50.846803+0000","last_clean":"2024-03-21T11:22:50.846803+0000","last_became_active":"2024-03-21T11:20:53.738750+0000","last_became_peered":"2024-03-21T11:20:53.738750+0000","last_unstale":"2024-03-21T11:22:50.846803+0000","last_undegraded":"2024-03-21T11:22:50.846803+0000","last_fullsized":"2024-03-21T11:22:50.846803+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:58:33.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921548+0000","last_change":"2024-03-21T11:20:48.144451+0000","last_active":"2024-03-21T11:22:50.921548+0000","last_peered":"2024-03-21T11:22:50.921548+0000","last_clean":"2024-03-21T11:22:50.921548+0000","last_became_active":"2024-03-21T11:20:48.144074+0000","last_became_peered":"2024-03-21T11:20:48.144074+0000","last_unstale":"2024-03-21T11:22:50.921548+0000","last_undegraded":"2024-03-21T11:22:50.921548+0000","last_fullsized":"2024-03-21T11:22:50.921548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:33:17.032461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129797+0000","last_change":"2024-03-21T11:20:48.654472+0000","last_active":"2024-03-21T11:22:51.129797+0000","last_peered":"2024-03-21T11:22:51.129797+0000","last_clean":"2024-03-21T11:22:51.129797+0000","last_became_active":"2024-03-21T11:20:48.654129+0000","last_became_peered":"2024-03-21T11:20:48.654129+0000","last_unstale":"2024-03-21T11:22:51.129797+0000","last_undegraded":"2024-03-21T11:22:51.129797+0000","last_fullsized":"2024-03-21T11:22:51.129797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:28:31.224296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849451+0000","last_change":"2024-03-21T11:20:53.739994+0000","last_active":"2024-03-21T11:22:50.849451+0000","last_peered":"2024-03-21T11:22:50.849451+0000","last_clean":"2024-03-21T11:22:50.849451+0000","last_became_active":"2024-03-21T11:20:53.738319+0000","last_became_peered":"2024-03-21T11:20:53.738319+0000","last_unstale":"2024-03-21T11:22:50.849451+0000","last_undegraded":"2024-03-21T11:22:50.849451+0000","last_fullsized":"2024-03-21T11:22:50.849451+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:55.694986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":371,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.757016+0000","last_change":"2024-03-21T11:20:48.131461+0000","last_active":"2024-03-21T11:23:40.757016+0000","last_peered":"2024-03-21T11:23:40.757016+0000","last_clean":"2024-03-21T11:23:40.757016+0000","last_became_active":"2024-03-21T11:20:48.131079+0000","last_became_peered":"2024-03-21T11:20:48.131079+0000","last_unstale":"2024-03-21T11:23:40.757016+0000","last_undegraded":"2024-03-21T11:23:40.757016+0000","last_fullsized":"2024-03-21T11:23:40.757016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:48:57.528444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70525,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":296,"num_read_kb":217,"num_write":43,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846832+0000","last_change":"2024-03-21T11:20:48.653696+0000","last_active":"2024-03-21T11:22:50.846832+0000","last_peered":"2024-03-21T11:22:50.846832+0000","last_clean":"2024-03-21T11:22:50.846832+0000","last_became_active":"2024-03-21T11:20:48.653218+0000","last_became_peered":"2024-03-21T11:20:48.653218+0000","last_unstale":"2024-03-21T11:22:50.846832+0000","last_undegraded":"2024-03-21T11:22:50.846832+0000","last_fullsized":"2024-03-21T11:22:50.846832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:11:59.419040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921748+0000","last_change":"2024-03-21T11:20:53.754019+0000","last_active":"2024-03-21T11:22:50.921748+0000","last_peered":"2024-03-21T11:22:50.921748+0000","last_clean":"2024-03-21T11:22:50.921748+0000","last_became_active":"2024-03-21T11:20:53.753606+0000","last_became_peered":"2024-03-21T11:20:53.753606+0000","last_unstale":"2024-03-21T11:22:50.921748+0000","last_undegraded":"2024-03-21T11:22:50.921748+0000","last_fullsized":"2024-03-21T11:22:50.921748+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:23:27.898596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853065+0000","last_change":"2024-03-21T11:20:48.153544+0000","last_active":"2024-03-21T11:22:50.853065+0000","last_peered":"2024-03-21T11:22:50.853065+0000","last_clean":"2024-03-21T11:22:50.853065+0000","last_became_active":"2024-03-21T11:20:48.153279+0000","last_became_peered":"2024-03-21T11:20:48.153279+0000","last_unstale":"2024-03-21T11:22:50.853065+0000","last_undegraded":"2024-03-21T11:22:50.853065+0000","last_fullsized":"2024-03-21T11:22:50.853065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:31:31.911844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755158+0000","last_change":"2024-03-21T11:20:48.665140+0000","last_active":"2024-03-21T11:22:51.755158+0000","last_peered":"2024-03-21T11:22:51.755158+0000","last_clean":"2024-03-21T11:22:51.755158+0000","last_became_active":"2024-03-21T11:20:48.664226+0000","last_became_peered":"2024-03-21T11:20:48.664226+0000","last_unstale":"2024-03-21T11:22:51.755158+0000","last_undegraded":"2024-03-21T11:22:51.755158+0000","last_fullsized":"2024-03-21T11:22:51.755158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:47:38.951403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560135+0000","last_change":"2024-03-21T11:20:53.751887+0000","last_active":"2024-03-21T11:22:51.560135+0000","last_peered":"2024-03-21T11:22:51.560135+0000","last_clean":"2024-03-21T11:22:51.560135+0000","last_became_active":"2024-03-21T11:20:53.751353+0000","last_became_peered":"2024-03-21T11:20:53.751353+0000","last_unstale":"2024-03-21T11:22:51.560135+0000","last_undegraded":"2024-03-21T11:22:51.560135+0000","last_fullsized":"2024-03-21T11:22:51.560135+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:21:36.689786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921390+0000","last_change":"2024-03-21T11:20:48.162150+0000","last_active":"2024-03-21T11:22:50.921390+0000","last_peered":"2024-03-21T11:22:50.921390+0000","last_clean":"2024-03-21T11:22:50.921390+0000","last_became_active":"2024-03-21T11:20:48.161825+0000","last_became_peered":"2024-03-21T11:20:48.161825+0000","last_unstale":"2024-03-21T11:22:50.921390+0000","last_undegraded":"2024-03-21T11:22:50.921390+0000","last_fullsized":"2024-03-21T11:22:50.921390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:10:47.097540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805965+0000","last_change":"2024-03-21T11:20:48.663856+0000","last_active":"2024-03-21T11:22:51.805965+0000","last_peered":"2024-03-21T11:22:51.805965+0000","last_clean":"2024-03-21T11:22:51.805965+0000","last_became_active":"2024-03-21T11:20:48.662908+0000","last_became_peered":"2024-03-21T11:20:48.662908+0000","last_unstale":"2024-03-21T11:22:51.805965+0000","last_undegraded":"2024-03-21T11:22:51.805965+0000","last_fullsized":"2024-03-21T11:22:51.805965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:11:12.255265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.851969+0000","last_change":"2024-03-21T11:20:53.755942+0000","last_active":"2024-03-21T11:22:50.851969+0000","last_peered":"2024-03-21T11:22:50.851969+0000","last_clean":"2024-03-21T11:22:50.851969+0000","last_became_active":"2024-03-21T11:20:53.755141+0000","last_became_peered":"2024-03-21T11:20:53.755141+0000","last_unstale":"2024-03-21T11:22:50.851969+0000","last_undegraded":"2024-03-21T11:22:50.851969+0000","last_fullsized":"2024-03-21T11:22:50.851969+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:30.141117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":187,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.842723+0000","last_change":"2024-03-21T11:20:48.132460+0000","last_active":"2024-03-21T11:23:40.842723+0000","last_peered":"2024-03-21T11:23:40.842723+0000","last_clean":"2024-03-21T11:23:40.842723+0000","last_became_active":"2024-03-21T11:20:48.131573+0000","last_became_peered":"2024-03-21T11:20:48.131573+0000","last_unstale":"2024-03-21T11:23:40.842723+0000","last_undegraded":"2024-03-21T11:23:40.842723+0000","last_fullsized":"2024-03-21T11:23:40.842723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:40:02.821536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805992+0000","last_change":"2024-03-21T11:20:48.656262+0000","last_active":"2024-03-21T11:22:51.805992+0000","last_peered":"2024-03-21T11:22:51.805992+0000","last_clean":"2024-03-21T11:22:51.805992+0000","last_became_active":"2024-03-21T11:20:48.655924+0000","last_became_peered":"2024-03-21T11:20:48.655924+0000","last_unstale":"2024-03-21T11:22:51.805992+0000","last_undegraded":"2024-03-21T11:22:51.805992+0000","last_fullsized":"2024-03-21T11:22:51.805992+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:24:20.412185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755346+0000","last_change":"2024-03-21T11:20:53.758666+0000","last_active":"2024-03-21T11:22:51.755346+0000","last_peered":"2024-03-21T11:22:51.755346+0000","last_clean":"2024-03-21T11:22:51.755346+0000","last_became_active":"2024-03-21T11:20:53.757398+0000","last_became_peered":"2024-03-21T11:20:53.757398+0000","last_unstale":"2024-03-21T11:22:51.755346+0000","last_undegraded":"2024-03-21T11:22:51.755346+0000","last_fullsized":"2024-03-21T11:22:51.755346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:34.198388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846374+0000","last_change":"2024-03-21T11:20:48.132725+0000","last_active":"2024-03-21T11:22:50.846374+0000","last_peered":"2024-03-21T11:22:50.846374+0000","last_clean":"2024-03-21T11:22:50.846374+0000","last_became_active":"2024-03-21T11:20:48.132367+0000","last_became_peered":"2024-03-21T11:20:48.132367+0000","last_unstale":"2024-03-21T11:22:50.846374+0000","last_undegraded":"2024-03-21T11:22:50.846374+0000","last_fullsized":"2024-03-21T11:22:50.846374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:19:18.334962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921197+0000","last_change":"2024-03-21T11:20:48.656302+0000","last_active":"2024-03-21T11:22:50.921197+0000","last_peered":"2024-03-21T11:22:50.921197+0000","last_clean":"2024-03-21T11:22:50.921197+0000","last_became_active":"2024-03-21T11:20:48.654305+0000","last_became_peered":"2024-03-21T11:20:48.654305+0000","last_unstale":"2024-03-21T11:22:50.921197+0000","last_undegraded":"2024-03-21T11:22:50.921197+0000","last_fullsized":"2024-03-21T11:22:50.921197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:59:40.766829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921110+0000","last_change":"2024-03-21T11:20:53.750710+0000","last_active":"2024-03-21T11:22:50.921110+0000","last_peered":"2024-03-21T11:22:50.921110+0000","last_clean":"2024-03-21T11:22:50.921110+0000","last_became_active":"2024-03-21T11:20:53.750255+0000","last_became_peered":"2024-03-21T11:20:53.750255+0000","last_unstale":"2024-03-21T11:22:50.921110+0000","last_undegraded":"2024-03-21T11:22:50.921110+0000","last_fullsized":"2024-03-21T11:22:50.921110+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:36:28.611838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846888+0000","last_change":"2024-03-21T11:20:48.133524+0000","last_active":"2024-03-21T11:22:50.846888+0000","last_peered":"2024-03-21T11:22:50.846888+0000","last_clean":"2024-03-21T11:22:50.846888+0000","last_became_active":"2024-03-21T11:20:48.133186+0000","last_became_peered":"2024-03-21T11:20:48.133186+0000","last_unstale":"2024-03-21T11:22:50.846888+0000","last_undegraded":"2024-03-21T11:22:50.846888+0000","last_fullsized":"2024-03-21T11:22:50.846888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:02.018639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560425+0000","last_change":"2024-03-21T11:20:48.654126+0000","last_active":"2024-03-21T11:22:51.560425+0000","last_peered":"2024-03-21T11:22:51.560425+0000","last_clean":"2024-03-21T11:22:51.560425+0000","last_became_active":"2024-03-21T11:20:48.653835+0000","last_became_peered":"2024-03-21T11:20:48.653835+0000","last_unstale":"2024-03-21T11:22:51.560425+0000","last_undegraded":"2024-03-21T11:22:51.560425+0000","last_fullsized":"2024-03-21T11:22:51.560425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:12:03.332076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846394+0000","last_change":"2024-03-21T11:20:53.916645+0000","last_active":"2024-03-21T11:22:50.846394+0000","last_peered":"2024-03-21T11:22:50.846394+0000","last_clean":"2024-03-21T11:22:50.846394+0000","last_became_active":"2024-03-21T11:20:53.916178+0000","last_became_peered":"2024-03-21T11:20:53.916178+0000","last_unstale":"2024-03-21T11:22:50.846394+0000","last_undegraded":"2024-03-21T11:22:50.846394+0000","last_fullsized":"2024-03-21T11:22:50.846394+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:56:15.805231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.851892+0000","last_change":"2024-03-21T11:20:48.159877+0000","last_active":"2024-03-21T11:22:50.851892+0000","last_peered":"2024-03-21T11:22:50.851892+0000","last_clean":"2024-03-21T11:22:50.851892+0000","last_became_active":"2024-03-21T11:20:48.159529+0000","last_became_peered":"2024-03-21T11:20:48.159529+0000","last_unstale":"2024-03-21T11:22:50.851892+0000","last_undegraded":"2024-03-21T11:22:50.851892+0000","last_fullsized":"2024-03-21T11:22:50.851892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:17:09.662462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848812+0000","last_change":"2024-03-21T11:20:48.663311+0000","last_active":"2024-03-21T11:22:50.848812+0000","last_peered":"2024-03-21T11:22:50.848812+0000","last_clean":"2024-03-21T11:22:50.848812+0000","last_became_active":"2024-03-21T11:20:48.662961+0000","last_became_peered":"2024-03-21T11:20:48.662961+0000","last_unstale":"2024-03-21T11:22:50.848812+0000","last_undegraded":"2024-03-21T11:22:50.848812+0000","last_fullsized":"2024-03-21T11:22:50.848812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:14:46.616326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755080+0000","last_change":"2024-03-21T11:20:53.751804+0000","last_active":"2024-03-21T11:22:51.755080+0000","last_peered":"2024-03-21T11:22:51.755080+0000","last_clean":"2024-03-21T11:22:51.755080+0000","last_became_active":"2024-03-21T11:20:53.751323+0000","last_became_peered":"2024-03-21T11:20:53.751323+0000","last_unstale":"2024-03-21T11:22:51.755080+0000","last_undegraded":"2024-03-21T11:22:51.755080+0000","last_fullsized":"2024-03-21T11:22:51.755080+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:38:25.677345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":186,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.908004+0000","last_change":"2024-03-21T11:20:48.147580+0000","last_active":"2024-03-21T11:23:40.908004+0000","last_peered":"2024-03-21T11:23:40.908004+0000","last_clean":"2024-03-21T11:23:40.908004+0000","last_became_active":"2024-03-21T11:20:48.147222+0000","last_became_peered":"2024-03-21T11:20:48.147222+0000","last_unstale":"2024-03-21T11:23:40.908004+0000","last_undegraded":"2024-03-21T11:23:40.908004+0000","last_fullsized":"2024-03-21T11:23:40.908004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:49:04.204129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.856848+0000","last_change":"2024-03-21T11:22:50.856453+0000","last_active":"2024-03-21T11:22:50.856848+0000","last_peered":"2024-03-21T11:22:50.856848+0000","last_clean":"2024-03-21T11:22:50.856848+0000","last_became_active":"2024-03-21T11:22:50.855987+0000","last_became_peered":"2024-03-21T11:22:50.855987+0000","last_unstale":"2024-03-21T11:22:50.856848+0000","last_undegraded":"2024-03-21T11:22:50.856848+0000","last_fullsized":"2024-03-21T11:22:50.856848+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:32:05.460528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755558+0000","last_change":"2024-03-21T11:20:53.916931+0000","last_active":"2024-03-21T11:22:51.755558+0000","last_peered":"2024-03-21T11:22:51.755558+0000","last_clean":"2024-03-21T11:22:51.755558+0000","last_became_active":"2024-03-21T11:20:53.916495+0000","last_became_peered":"2024-03-21T11:20:53.916495+0000","last_unstale":"2024-03-21T11:22:51.755558+0000","last_undegraded":"2024-03-21T11:22:51.755558+0000","last_fullsized":"2024-03-21T11:22:51.755558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:18:33.046052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855560+0000","last_change":"2024-03-21T11:22:50.854745+0000","last_active":"2024-03-21T11:22:50.855560+0000","last_peered":"2024-03-21T11:22:50.855560+0000","last_clean":"2024-03-21T11:22:50.855560+0000","last_became_active":"2024-03-21T11:22:50.854201+0000","last_became_peered":"2024-03-21T11:22:50.854201+0000","last_unstale":"2024-03-21T11:22:50.855560+0000","last_undegraded":"2024-03-21T11:22:50.855560+0000","last_fullsized":"2024-03-21T11:22:50.855560+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:06:27.288363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":184,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.842225+0000","last_change":"2024-03-21T11:20:48.164607+0000","last_active":"2024-03-21T11:23:40.842225+0000","last_peered":"2024-03-21T11:23:40.842225+0000","last_clean":"2024-03-21T11:23:40.842225+0000","last_became_active":"2024-03-21T11:20:48.164222+0000","last_became_peered":"2024-03-21T11:20:48.164222+0000","last_unstale":"2024-03-21T11:23:40.842225+0000","last_undegraded":"2024-03-21T11:23:40.842225+0000","last_fullsized":"2024-03-21T11:23:40.842225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:52:24.359155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806069+0000","last_change":"2024-03-21T11:21:50.658412+0000","last_active":"2024-03-21T11:22:51.806069+0000","last_peered":"2024-03-21T11:22:51.806069+0000","last_clean":"2024-03-21T11:22:51.806069+0000","last_became_active":"2024-03-21T11:21:50.649270+0000","last_became_peered":"2024-03-21T11:21:50.649270+0000","last_unstale":"2024-03-21T11:22:51.806069+0000","last_undegraded":"2024-03-21T11:22:51.806069+0000","last_fullsized":"2024-03-21T11:22:51.806069+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:11:06.006302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848564+0000","last_change":"2024-03-21T11:20:48.667212+0000","last_active":"2024-03-21T11:22:50.848564+0000","last_peered":"2024-03-21T11:22:50.848564+0000","last_clean":"2024-03-21T11:22:50.848564+0000","last_became_active":"2024-03-21T11:20:48.662839+0000","last_became_peered":"2024-03-21T11:20:48.662839+0000","last_unstale":"2024-03-21T11:22:50.848564+0000","last_undegraded":"2024-03-21T11:22:50.848564+0000","last_fullsized":"2024-03-21T11:22:50.848564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:27:41.490922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846727+0000","last_change":"2024-03-21T11:20:48.151185+0000","last_active":"2024-03-21T11:22:50.846727+0000","last_peered":"2024-03-21T11:22:50.846727+0000","last_clean":"2024-03-21T11:22:50.846727+0000","last_became_active":"2024-03-21T11:20:48.150859+0000","last_became_peered":"2024-03-21T11:20:48.150859+0000","last_unstale":"2024-03-21T11:22:50.846727+0000","last_undegraded":"2024-03-21T11:22:50.846727+0000","last_fullsized":"2024-03-21T11:22:50.846727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:52:45.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":357,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805649+0000","last_change":"2024-03-21T11:19:28.916140+0000","last_active":"2024-03-21T11:22:51.805649+0000","last_peered":"2024-03-21T11:22:51.805649+0000","last_clean":"2024-03-21T11:22:51.805649+0000","last_became_active":"2024-03-21T11:19:28.915634+0000","last_became_peered":"2024-03-21T11:19:28.915634+0000","last_unstale":"2024-03-21T11:22:51.805649+0000","last_undegraded":"2024-03-21T11:22:51.805649+0000","last_fullsized":"2024-03-21T11:22:51.805649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:15:35.803515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:15:35.803515+0000","last_clean_scrub_stamp":"2024-03-21T11:15:35.803515+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:08:31.509894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921613+0000","last_change":"2024-03-21T11:21:50.656708+0000","last_active":"2024-03-21T11:22:50.921613+0000","last_peered":"2024-03-21T11:22:50.921613+0000","last_clean":"2024-03-21T11:22:50.921613+0000","last_became_active":"2024-03-21T11:21:50.656236+0000","last_became_peered":"2024-03-21T11:21:50.656236+0000","last_unstale":"2024-03-21T11:22:50.921613+0000","last_undegraded":"2024-03-21T11:22:50.921613+0000","last_fullsized":"2024-03-21T11:22:50.921613+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:17:44.114512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.754767+0000","last_change":"2024-03-21T11:20:48.657878+0000","last_active":"2024-03-21T11:22:51.754767+0000","last_peered":"2024-03-21T11:22:51.754767+0000","last_clean":"2024-03-21T11:22:51.754767+0000","last_became_active":"2024-03-21T11:20:48.657212+0000","last_became_peered":"2024-03-21T11:20:48.657212+0000","last_unstale":"2024-03-21T11:22:51.754767+0000","last_undegraded":"2024-03-21T11:22:51.754767+0000","last_fullsized":"2024-03-21T11:22:51.754767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:43:11.993577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805832+0000","last_change":"2024-03-21T11:20:48.135513+0000","last_active":"2024-03-21T11:22:51.805832+0000","last_peered":"2024-03-21T11:22:51.805832+0000","last_clean":"2024-03-21T11:22:51.805832+0000","last_became_active":"2024-03-21T11:20:48.135223+0000","last_became_peered":"2024-03-21T11:20:48.135223+0000","last_unstale":"2024-03-21T11:22:51.805832+0000","last_undegraded":"2024-03-21T11:22:51.805832+0000","last_fullsized":"2024-03-21T11:22:51.805832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:37:38.834284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560138+0000","last_change":"2024-03-21T11:20:53.751790+0000","last_active":"2024-03-21T11:22:51.560138+0000","last_peered":"2024-03-21T11:22:51.560138+0000","last_clean":"2024-03-21T11:22:51.560138+0000","last_became_active":"2024-03-21T11:20:53.751312+0000","last_became_peered":"2024-03-21T11:20:53.751312+0000","last_unstale":"2024-03-21T11:22:51.560138+0000","last_undegraded":"2024-03-21T11:22:51.560138+0000","last_fullsized":"2024-03-21T11:22:51.560138+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:21:41.599985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846835+0000","last_change":"2024-03-21T11:20:48.154119+0000","last_active":"2024-03-21T11:22:50.846835+0000","last_peered":"2024-03-21T11:22:50.846835+0000","last_clean":"2024-03-21T11:22:50.846835+0000","last_became_active":"2024-03-21T11:20:48.153781+0000","last_became_peered":"2024-03-21T11:20:48.153781+0000","last_unstale":"2024-03-21T11:22:50.846835+0000","last_undegraded":"2024-03-21T11:22:50.846835+0000","last_fullsized":"2024-03-21T11:22:50.846835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:44:13.937836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755223+0000","last_change":"2024-03-21T11:20:48.661282+0000","last_active":"2024-03-21T11:22:51.755223+0000","last_peered":"2024-03-21T11:22:51.755223+0000","last_clean":"2024-03-21T11:22:51.755223+0000","last_became_active":"2024-03-21T11:20:48.660569+0000","last_became_peered":"2024-03-21T11:20:48.660569+0000","last_unstale":"2024-03-21T11:22:51.755223+0000","last_undegraded":"2024-03-21T11:22:51.755223+0000","last_fullsized":"2024-03-21T11:22:51.755223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:14.875700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847881+0000","last_change":"2024-03-21T11:20:53.918016+0000","last_active":"2024-03-21T11:22:50.847881+0000","last_peered":"2024-03-21T11:22:50.847881+0000","last_clean":"2024-03-21T11:22:50.847881+0000","last_became_active":"2024-03-21T11:20:53.916891+0000","last_became_peered":"2024-03-21T11:20:53.916891+0000","last_unstale":"2024-03-21T11:22:50.847881+0000","last_undegraded":"2024-03-21T11:22:50.847881+0000","last_fullsized":"2024-03-21T11:22:50.847881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:40:25.206778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'15","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:37.609177+0000","last_change":"2024-03-21T11:20:48.167560+0000","last_active":"2024-03-21T11:23:37.609177+0000","last_peered":"2024-03-21T11:23:37.609177+0000","last_clean":"2024-03-21T11:23:37.609177+0000","last_became_active":"2024-03-21T11:20:48.164986+0000","last_became_peered":"2024-03-21T11:20:48.164986+0000","last_unstale":"2024-03-21T11:23:37.609177+0000","last_undegraded":"2024-03-21T11:23:37.609177+0000","last_fullsized":"2024-03-21T11:23:37.609177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:59.930626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849952+0000","last_change":"2024-03-21T11:20:48.675250+0000","last_active":"2024-03-21T11:22:50.849952+0000","last_peered":"2024-03-21T11:22:50.849952+0000","last_clean":"2024-03-21T11:22:50.849952+0000","last_became_active":"2024-03-21T11:20:48.674638+0000","last_became_peered":"2024-03-21T11:20:48.674638+0000","last_unstale":"2024-03-21T11:22:50.849952+0000","last_undegraded":"2024-03-21T11:22:50.849952+0000","last_fullsized":"2024-03-21T11:22:50.849952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:46:34.842363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852772+0000","last_change":"2024-03-21T11:20:53.733809+0000","last_active":"2024-03-21T11:22:50.852772+0000","last_peered":"2024-03-21T11:22:50.852772+0000","last_clean":"2024-03-21T11:22:50.852772+0000","last_became_active":"2024-03-21T11:20:53.733393+0000","last_became_peered":"2024-03-21T11:20:53.733393+0000","last_unstale":"2024-03-21T11:22:50.852772+0000","last_undegraded":"2024-03-21T11:22:50.852772+0000","last_fullsized":"2024-03-21T11:22:50.852772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:02:46.270716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847217+0000","last_change":"2024-03-21T11:20:48.139475+0000","last_active":"2024-03-21T11:22:50.847217+0000","last_peered":"2024-03-21T11:22:50.847217+0000","last_clean":"2024-03-21T11:22:50.847217+0000","last_became_active":"2024-03-21T11:20:48.139249+0000","last_became_peered":"2024-03-21T11:20:48.139249+0000","last_unstale":"2024-03-21T11:22:50.847217+0000","last_undegraded":"2024-03-21T11:22:50.847217+0000","last_fullsized":"2024-03-21T11:22:50.847217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:45:18.811829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855394+0000","last_change":"2024-03-21T11:20:48.673975+0000","last_active":"2024-03-21T11:22:50.855394+0000","last_peered":"2024-03-21T11:22:50.855394+0000","last_clean":"2024-03-21T11:22:50.855394+0000","last_became_active":"2024-03-21T11:20:48.673587+0000","last_became_peered":"2024-03-21T11:20:48.673587+0000","last_unstale":"2024-03-21T11:22:50.855394+0000","last_undegraded":"2024-03-21T11:22:50.855394+0000","last_fullsized":"2024-03-21T11:22:50.855394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:08:52.354780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847109+0000","last_change":"2024-03-21T11:20:53.917493+0000","last_active":"2024-03-21T11:22:50.847109+0000","last_peered":"2024-03-21T11:22:50.847109+0000","last_clean":"2024-03-21T11:22:50.847109+0000","last_became_active":"2024-03-21T11:20:53.917007+0000","last_became_peered":"2024-03-21T11:20:53.917007+0000","last_unstale":"2024-03-21T11:22:50.847109+0000","last_undegraded":"2024-03-21T11:22:50.847109+0000","last_fullsized":"2024-03-21T11:22:50.847109+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:39:20.280899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129330+0000","last_change":"2024-03-21T11:20:48.140914+0000","last_active":"2024-03-21T11:22:51.129330+0000","last_peered":"2024-03-21T11:22:51.129330+0000","last_clean":"2024-03-21T11:22:51.129330+0000","last_became_active":"2024-03-21T11:20:48.140558+0000","last_became_peered":"2024-03-21T11:20:48.140558+0000","last_unstale":"2024-03-21T11:22:51.129330+0000","last_undegraded":"2024-03-21T11:22:51.129330+0000","last_fullsized":"2024-03-21T11:22:51.129330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:59:52.065431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846373+0000","last_change":"2024-03-21T11:20:48.651190+0000","last_active":"2024-03-21T11:22:50.846373+0000","last_peered":"2024-03-21T11:22:50.846373+0000","last_clean":"2024-03-21T11:22:50.846373+0000","last_became_active":"2024-03-21T11:20:48.650818+0000","last_became_peered":"2024-03-21T11:20:48.650818+0000","last_unstale":"2024-03-21T11:22:50.846373+0000","last_undegraded":"2024-03-21T11:22:50.846373+0000","last_fullsized":"2024-03-21T11:22:50.846373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:47:17.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852214+0000","last_change":"2024-03-21T11:20:53.756844+0000","last_active":"2024-03-21T11:22:50.852214+0000","last_peered":"2024-03-21T11:22:50.852214+0000","last_clean":"2024-03-21T11:22:50.852214+0000","last_became_active":"2024-03-21T11:20:53.756380+0000","last_became_peered":"2024-03-21T11:20:53.756380+0000","last_unstale":"2024-03-21T11:22:50.852214+0000","last_undegraded":"2024-03-21T11:22:50.852214+0000","last_fullsized":"2024-03-21T11:22:50.852214+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:51:55.327798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848472+0000","last_change":"2024-03-21T11:20:48.160749+0000","last_active":"2024-03-21T11:22:50.848472+0000","last_peered":"2024-03-21T11:22:50.848472+0000","last_clean":"2024-03-21T11:22:50.848472+0000","last_became_active":"2024-03-21T11:20:48.158507+0000","last_became_peered":"2024-03-21T11:20:48.158507+0000","last_unstale":"2024-03-21T11:22:50.848472+0000","last_undegraded":"2024-03-21T11:22:50.848472+0000","last_fullsized":"2024-03-21T11:22:50.848472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:55.375529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560069+0000","last_change":"2024-03-21T11:20:48.667153+0000","last_active":"2024-03-21T11:22:51.560069+0000","last_peered":"2024-03-21T11:22:51.560069+0000","last_clean":"2024-03-21T11:22:51.560069+0000","last_became_active":"2024-03-21T11:20:48.666823+0000","last_became_peered":"2024-03-21T11:20:48.666823+0000","last_unstale":"2024-03-21T11:22:51.560069+0000","last_undegraded":"2024-03-21T11:22:51.560069+0000","last_fullsized":"2024-03-21T11:22:51.560069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:43:54.183372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846842+0000","last_change":"2024-03-21T11:20:53.739188+0000","last_active":"2024-03-21T11:22:50.846842+0000","last_peered":"2024-03-21T11:22:50.846842+0000","last_clean":"2024-03-21T11:22:50.846842+0000","last_became_active":"2024-03-21T11:20:53.738770+0000","last_became_peered":"2024-03-21T11:20:53.738770+0000","last_unstale":"2024-03-21T11:22:50.846842+0000","last_undegraded":"2024-03-21T11:22:50.846842+0000","last_fullsized":"2024-03-21T11:22:50.846842+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:53:48.897963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560058+0000","last_change":"2024-03-21T11:21:50.668860+0000","last_active":"2024-03-21T11:22:51.560058+0000","last_peered":"2024-03-21T11:22:51.560058+0000","last_clean":"2024-03-21T11:22:51.560058+0000","last_became_active":"2024-03-21T11:21:50.655255+0000","last_became_peered":"2024-03-21T11:21:50.655255+0000","last_unstale":"2024-03-21T11:22:51.560058+0000","last_undegraded":"2024-03-21T11:22:51.560058+0000","last_fullsized":"2024-03-21T11:22:51.560058+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:27.634508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.850136+0000","last_change":"2024-03-21T11:20:48.677303+0000","last_active":"2024-03-21T11:22:50.850136+0000","last_peered":"2024-03-21T11:22:50.850136+0000","last_clean":"2024-03-21T11:22:50.850136+0000","last_became_active":"2024-03-21T11:20:48.676903+0000","last_became_peered":"2024-03-21T11:20:48.676903+0000","last_unstale":"2024-03-21T11:22:50.850136+0000","last_undegraded":"2024-03-21T11:22:50.850136+0000","last_fullsized":"2024-03-21T11:22:50.850136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:06:59.922274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847033+0000","last_change":"2024-03-21T11:20:53.916691+0000","last_active":"2024-03-21T11:22:50.847033+0000","last_peered":"2024-03-21T11:22:50.847033+0000","last_clean":"2024-03-21T11:22:50.847033+0000","last_became_active":"2024-03-21T11:20:53.916275+0000","last_became_peered":"2024-03-21T11:20:53.916275+0000","last_unstale":"2024-03-21T11:22:50.847033+0000","last_undegraded":"2024-03-21T11:22:50.847033+0000","last_fullsized":"2024-03-21T11:22:50.847033+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:35.848589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846526+0000","last_change":"2024-03-21T11:21:50.668467+0000","last_active":"2024-03-21T11:22:50.846526+0000","last_peered":"2024-03-21T11:22:50.846526+0000","last_clean":"2024-03-21T11:22:50.846526+0000","last_became_active":"2024-03-21T11:21:50.649598+0000","last_became_peered":"2024-03-21T11:21:50.649598+0000","last_unstale":"2024-03-21T11:22:50.846526+0000","last_undegraded":"2024-03-21T11:22:50.846526+0000","last_fullsized":"2024-03-21T11:22:50.846526+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:01:48.186997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806108+0000","last_change":"2024-03-21T11:20:48.661486+0000","last_active":"2024-03-21T11:22:51.806108+0000","last_peered":"2024-03-21T11:22:51.806108+0000","last_clean":"2024-03-21T11:22:51.806108+0000","last_became_active":"2024-03-21T11:20:48.661109+0000","last_became_peered":"2024-03-21T11:20:48.661109+0000","last_unstale":"2024-03-21T11:22:51.806108+0000","last_undegraded":"2024-03-21T11:22:51.806108+0000","last_fullsized":"2024-03-21T11:22:51.806108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:15:27.093616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129398+0000","last_change":"2024-03-21T11:20:53.747223+0000","last_active":"2024-03-21T11:22:51.129398+0000","last_peered":"2024-03-21T11:22:51.129398+0000","last_clean":"2024-03-21T11:22:51.129398+0000","last_became_active":"2024-03-21T11:20:53.746805+0000","last_became_peered":"2024-03-21T11:20:53.746805+0000","last_unstale":"2024-03-21T11:22:51.129398+0000","last_undegraded":"2024-03-21T11:22:51.129398+0000","last_fullsized":"2024-03-21T11:22:51.129398+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:29.186745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"93'7","reported_seq":189,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:42.758848+0000","last_change":"2024-03-21T11:20:48.151172+0000","last_active":"2024-03-21T11:23:42.758848+0000","last_peered":"2024-03-21T11:23:42.758848+0000","last_clean":"2024-03-21T11:23:42.758848+0000","last_became_active":"2024-03-21T11:20:48.150832+0000","last_became_peered":"2024-03-21T11:20:48.150832+0000","last_unstale":"2024-03-21T11:23:42.758848+0000","last_undegraded":"2024-03-21T11:23:42.758848+0000","last_fullsized":"2024-03-21T11:23:42.758848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:34:46.367713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":149,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806211+0000","last_change":"2024-03-21T11:20:48.652275+0000","last_active":"2024-03-21T11:22:51.806211+0000","last_peered":"2024-03-21T11:22:51.806211+0000","last_clean":"2024-03-21T11:22:51.806211+0000","last_became_active":"2024-03-21T11:20:48.651965+0000","last_became_peered":"2024-03-21T11:20:48.651965+0000","last_unstale":"2024-03-21T11:22:51.806211+0000","last_undegraded":"2024-03-21T11:22:51.806211+0000","last_fullsized":"2024-03-21T11:22:51.806211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:56:18.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.854441+0000","last_change":"2024-03-21T11:20:53.748050+0000","last_active":"2024-03-21T11:22:50.854441+0000","last_peered":"2024-03-21T11:22:50.854441+0000","last_clean":"2024-03-21T11:22:50.854441+0000","last_became_active":"2024-03-21T11:20:53.747636+0000","last_became_peered":"2024-03-21T11:20:53.747636+0000","last_unstale":"2024-03-21T11:22:50.854441+0000","last_undegraded":"2024-03-21T11:22:50.854441+0000","last_fullsized":"2024-03-21T11:22:50.854441+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:09:15.202183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921814+0000","last_change":"2024-03-21T11:20:48.156928+0000","last_active":"2024-03-21T11:22:50.921814+0000","last_peered":"2024-03-21T11:22:50.921814+0000","last_clean":"2024-03-21T11:22:50.921814+0000","last_became_active":"2024-03-21T11:20:48.156625+0000","last_became_peered":"2024-03-21T11:20:48.156625+0000","last_unstale":"2024-03-21T11:22:50.921814+0000","last_undegraded":"2024-03-21T11:22:50.921814+0000","last_fullsized":"2024-03-21T11:22:50.921814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:19:42.035565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805952+0000","last_change":"2024-03-21T11:20:48.663655+0000","last_active":"2024-03-21T11:22:51.805952+0000","last_peered":"2024-03-21T11:22:51.805952+0000","last_clean":"2024-03-21T11:22:51.805952+0000","last_became_active":"2024-03-21T11:20:48.663336+0000","last_became_peered":"2024-03-21T11:20:48.663336+0000","last_unstale":"2024-03-21T11:22:51.805952+0000","last_undegraded":"2024-03-21T11:22:51.805952+0000","last_fullsized":"2024-03-21T11:22:51.805952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:29:05.119283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.920960+0000","last_change":"2024-03-21T11:20:53.732584+0000","last_active":"2024-03-21T11:22:50.920960+0000","last_peered":"2024-03-21T11:22:50.920960+0000","last_clean":"2024-03-21T11:22:50.920960+0000","last_became_active":"2024-03-21T11:20:53.731975+0000","last_became_peered":"2024-03-21T11:20:53.731975+0000","last_unstale":"2024-03-21T11:22:50.920960+0000","last_undegraded":"2024-03-21T11:22:50.920960+0000","last_fullsized":"2024-03-21T11:22:50.920960+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:51:06.336241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755235+0000","last_change":"2024-03-21T11:20:48.165594+0000","last_active":"2024-03-21T11:22:51.755235+0000","last_peered":"2024-03-21T11:22:51.755235+0000","last_clean":"2024-03-21T11:22:51.755235+0000","last_became_active":"2024-03-21T11:20:48.164394+0000","last_became_peered":"2024-03-21T11:20:48.164394+0000","last_unstale":"2024-03-21T11:22:51.755235+0000","last_undegraded":"2024-03-21T11:22:51.755235+0000","last_fullsized":"2024-03-21T11:22:51.755235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:13.317767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806214+0000","last_change":"2024-03-21T11:20:48.649359+0000","last_active":"2024-03-21T11:22:51.806214+0000","last_peered":"2024-03-21T11:22:51.806214+0000","last_clean":"2024-03-21T11:22:51.806214+0000","last_became_active":"2024-03-21T11:20:48.649031+0000","last_became_peered":"2024-03-21T11:20:48.649031+0000","last_unstale":"2024-03-21T11:22:51.806214+0000","last_undegraded":"2024-03-21T11:22:51.806214+0000","last_fullsized":"2024-03-21T11:22:51.806214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:17.055732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560206+0000","last_change":"2024-03-21T11:20:48.654999+0000","last_active":"2024-03-21T11:22:51.560206+0000","last_peered":"2024-03-21T11:22:51.560206+0000","last_clean":"2024-03-21T11:22:51.560206+0000","last_became_active":"2024-03-21T11:20:48.654638+0000","last_became_peered":"2024-03-21T11:20:48.654638+0000","last_unstale":"2024-03-21T11:22:51.560206+0000","last_undegraded":"2024-03-21T11:22:51.560206+0000","last_fullsized":"2024-03-21T11:22:51.560206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:49:48.711420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852998+0000","last_change":"2024-03-21T11:20:48.651864+0000","last_active":"2024-03-21T11:22:50.852998+0000","last_peered":"2024-03-21T11:22:50.852998+0000","last_clean":"2024-03-21T11:22:50.852998+0000","last_became_active":"2024-03-21T11:20:48.651565+0000","last_became_peered":"2024-03-21T11:20:48.651565+0000","last_unstale":"2024-03-21T11:22:50.852998+0000","last_undegraded":"2024-03-21T11:22:50.852998+0000","last_fullsized":"2024-03-21T11:22:50.852998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:58.659889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853152+0000","last_change":"2024-03-21T11:20:48.667038+0000","last_active":"2024-03-21T11:22:50.853152+0000","last_peered":"2024-03-21T11:22:50.853152+0000","last_clean":"2024-03-21T11:22:50.853152+0000","last_became_active":"2024-03-21T11:20:48.666570+0000","last_became_peered":"2024-03-21T11:20:48.666570+0000","last_unstale":"2024-03-21T11:22:50.853152+0000","last_undegraded":"2024-03-21T11:22:50.853152+0000","last_fullsized":"2024-03-21T11:22:50.853152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:36:39.043697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855033+0000","last_change":"2024-03-21T11:21:50.658024+0000","last_active":"2024-03-21T11:22:50.855033+0000","last_peered":"2024-03-21T11:22:50.855033+0000","last_clean":"2024-03-21T11:22:50.855033+0000","last_became_active":"2024-03-21T11:21:50.657390+0000","last_became_peered":"2024-03-21T11:21:50.657390+0000","last_unstale":"2024-03-21T11:22:50.855033+0000","last_undegraded":"2024-03-21T11:22:50.855033+0000","last_fullsized":"2024-03-21T11:22:50.855033+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:29.578366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755343+0000","last_change":"2024-03-21T11:20:48.670139+0000","last_active":"2024-03-21T11:22:51.755343+0000","last_peered":"2024-03-21T11:22:51.755343+0000","last_clean":"2024-03-21T11:22:51.755343+0000","last_became_active":"2024-03-21T11:20:48.669410+0000","last_became_peered":"2024-03-21T11:20:48.669410+0000","last_unstale":"2024-03-21T11:22:51.755343+0000","last_undegraded":"2024-03-21T11:22:51.755343+0000","last_fullsized":"2024-03-21T11:22:51.755343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:35:18.799840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560440+0000","last_change":"2024-03-21T11:20:48.655356+0000","last_active":"2024-03-21T11:22:51.560440+0000","last_peered":"2024-03-21T11:22:51.560440+0000","last_clean":"2024-03-21T11:22:51.560440+0000","last_became_active":"2024-03-21T11:20:48.655028+0000","last_became_peered":"2024-03-21T11:20:48.655028+0000","last_unstale":"2024-03-21T11:22:51.560440+0000","last_undegraded":"2024-03-21T11:22:51.560440+0000","last_fullsized":"2024-03-21T11:22:51.560440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:46:39.948982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853238+0000","last_change":"2024-03-21T11:20:48.648840+0000","last_active":"2024-03-21T11:22:50.853238+0000","last_peered":"2024-03-21T11:22:50.853238+0000","last_clean":"2024-03-21T11:22:50.853238+0000","last_became_active":"2024-03-21T11:20:48.648502+0000","last_became_peered":"2024-03-21T11:20:48.648502+0000","last_unstale":"2024-03-21T11:22:50.853238+0000","last_undegraded":"2024-03-21T11:22:50.853238+0000","last_fullsized":"2024-03-21T11:22:50.853238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:33:14.824880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.850295+0000","last_change":"2024-03-21T11:20:48.672015+0000","last_active":"2024-03-21T11:22:50.850295+0000","last_peered":"2024-03-21T11:22:50.850295+0000","last_clean":"2024-03-21T11:22:50.850295+0000","last_became_active":"2024-03-21T11:20:48.671663+0000","last_became_peered":"2024-03-21T11:20:48.671663+0000","last_unstale":"2024-03-21T11:22:50.850295+0000","last_undegraded":"2024-03-21T11:22:50.850295+0000","last_fullsized":"2024-03-21T11:22:50.850295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:21:25.328698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921220+0000","last_change":"2024-03-21T11:20:48.664114+0000","last_active":"2024-03-21T11:22:50.921220+0000","last_peered":"2024-03-21T11:22:50.921220+0000","last_clean":"2024-03-21T11:22:50.921220+0000","last_became_active":"2024-03-21T11:20:48.663819+0000","last_became_peered":"2024-03-21T11:20:48.663819+0000","last_unstale":"2024-03-21T11:22:50.921220+0000","last_undegraded":"2024-03-21T11:22:50.921220+0000","last_fullsized":"2024-03-21T11:22:50.921220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:08:08.201383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848874+0000","last_change":"2024-03-21T11:20:48.671336+0000","last_active":"2024-03-21T11:22:50.848874+0000","last_peered":"2024-03-21T11:22:50.848874+0000","last_clean":"2024-03-21T11:22:50.848874+0000","last_became_active":"2024-03-21T11:20:48.670918+0000","last_became_peered":"2024-03-21T11:20:48.670918+0000","last_unstale":"2024-03-21T11:22:50.848874+0000","last_undegraded":"2024-03-21T11:22:50.848874+0000","last_fullsized":"2024-03-21T11:22:50.848874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:01:44.764584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129593+0000","last_change":"2024-03-21T11:20:48.659339+0000","last_active":"2024-03-21T11:22:51.129593+0000","last_peered":"2024-03-21T11:22:51.129593+0000","last_clean":"2024-03-21T11:22:51.129593+0000","last_became_active":"2024-03-21T11:20:48.658989+0000","last_became_peered":"2024-03-21T11:20:48.658989+0000","last_unstale":"2024-03-21T11:22:51.129593+0000","last_undegraded":"2024-03-21T11:22:51.129593+0000","last_fullsized":"2024-03-21T11:22:51.129593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:44:09.599836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755291+0000","last_change":"2024-03-21T11:20:48.668679+0000","last_active":"2024-03-21T11:22:51.755291+0000","last_peered":"2024-03-21T11:22:51.755291+0000","last_clean":"2024-03-21T11:22:51.755291+0000","last_became_active":"2024-03-21T11:20:48.667956+0000","last_became_peered":"2024-03-21T11:20:48.667956+0000","last_unstale":"2024-03-21T11:22:51.755291+0000","last_undegraded":"2024-03-21T11:22:51.755291+0000","last_fullsized":"2024-03-21T11:22:51.755291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:04.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560270+0000","last_change":"2024-03-21T11:21:50.650600+0000","last_active":"2024-03-21T11:22:51.560270+0000","last_peered":"2024-03-21T11:22:51.560270+0000","last_clean":"2024-03-21T11:22:51.560270+0000","last_became_active":"2024-03-21T11:21:50.650221+0000","last_became_peered":"2024-03-21T11:21:50.650221+0000","last_unstale":"2024-03-21T11:22:51.560270+0000","last_undegraded":"2024-03-21T11:22:51.560270+0000","last_fullsized":"2024-03-21T11:22:51.560270+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:29:52.245281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247498,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79240,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1481,"num_read_kb":824,"num_write":267,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":240240,"data_compressed":5487,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351026,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":531977,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612663,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32356,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712796,"statfs":{"total":95995035648,"available":95961903104,"internally_reserved":0,"allocated":1208320,"data_stored":1193414,"data_compressed":64722,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776188,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":535471,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037827,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":531753,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299464,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":1129620,"data_compressed":62893,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462990,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":527270,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":598825,"data_compressed":28661,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":528867,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1142784,"data_stored":602544,"data_compressed":28661,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149790,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":526587,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888152,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":531485,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724627,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":1120183,"data_compressed":62893,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72668,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":76387,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":72793,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T11:23:45.615 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-21T11:23:45.615 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T11:23:45.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.877+0000 7f9ea2800640 1 -- 172.21.15.47:0/2820339500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e9c105490 msgr2=0x7f9e9c10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:45.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.877+0000 7f9ea2800640 1 --2- 172.21.15.47:0/2820339500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e9c105490 0x7f9e9c10bd20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009ca0 tx=0x7f9e9402c100 comp rx=0 tx=0).stop 2024-03-21T11:23:45.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.878+0000 7f9ea2800640 1 -- 172.21.15.47:0/2820339500 shutdown_connections 2024-03-21T11:23:45.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.878+0000 7f9ea2800640 1 --2- 172.21.15.47:0/2820339500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e9c105490 0x7f9e9c10bd20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.878+0000 7f9ea2800640 1 --2- 172.21.15.47:0/2820339500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e9c104ad0 0x7f9e9c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.878+0000 7f9ea2800640 1 --2- 172.21.15.47:0/2820339500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e9c1038e0 0x7f9e9c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.878+0000 7f9ea2800640 1 -- 172.21.15.47:0/2820339500 >> 172.21.15.47:0/2820339500 conn(0x7f9e9c0fee50 msgr2=0x7f9e9c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:45.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.878+0000 7f9ea2800640 1 -- 172.21.15.47:0/2820339500 shutdown_connections 2024-03-21T11:23:45.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.878+0000 7f9ea2800640 1 -- 172.21.15.47:0/2820339500 wait complete. 2024-03-21T11:23:45.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.879+0000 7f9ea2800640 1 Processor -- start 2024-03-21T11:23:45.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.879+0000 7f9ea2800640 1 -- start start 2024-03-21T11:23:45.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.879+0000 7f9ea2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e9c1038e0 0x7f9e9c199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9ea2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e9c104ad0 0x7f9e9c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9ea2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e9c105490 0x7f9e9c1a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9ea2800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c10f540 con 0x7f9e9c105490 2024-03-21T11:23:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9ea2800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c10f3c0 con 0x7f9e9c1038e0 2024-03-21T11:23:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9ea2800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c10f6c0 con 0x7f9e9c104ad0 2024-03-21T11:23:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9e9b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e9c1038e0 0x7f9e9c199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9e93e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e9c104ad0 0x7f9e9c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9e9b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e9c1038e0 0x7f9e9c199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38736/0 (socket says 172.21.15.47:38736) 2024-03-21T11:23:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9e9b400640 1 -- 172.21.15.47:0/1089190314 learned_addr learned my addr 172.21.15.47:0/1089190314 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9e9be00640 1 --2- 172.21.15.47:0/1089190314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e9c105490 0x7f9e9c1a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9e93e00640 1 -- 172.21.15.47:0/1089190314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e9c1038e0 msgr2=0x7f9e9c199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9e93e00640 1 --2- 172.21.15.47:0/1089190314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e9c1038e0 0x7f9e9c199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9e93e00640 1 -- 172.21.15.47:0/1089190314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e9c105490 msgr2=0x7f9e9c1a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9e93e00640 1 --2- 172.21.15.47:0/1089190314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e9c105490 0x7f9e9c1a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9e93e00640 1 -- 172.21.15.47:0/1089190314 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c1a17a0 con 0x7f9e9c104ad0 2024-03-21T11:23:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.880+0000 7f9e9be00640 1 --2- 172.21.15.47:0/1089190314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e9c105490 0x7f9e9c1a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.881+0000 7f9e93e00640 1 --2- 172.21.15.47:0/1089190314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e9c104ad0 0x7f9e9c19a0c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400e9f0 tx=0x7f9e8400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.881+0000 7f9e98c00640 1 -- 172.21.15.47:0/1089190314 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8400ccc0 con 0x7f9e9c104ad0 2024-03-21T11:23:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.881+0000 7f9e98c00640 1 -- 172.21.15.47:0/1089190314 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e84010910 con 0x7f9e9c104ad0 2024-03-21T11:23:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.881+0000 7f9ea2800640 1 -- 172.21.15.47:0/1089190314 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9c1a1a90 con 0x7f9e9c104ad0 2024-03-21T11:23:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.881+0000 7f9e98c00640 1 -- 172.21.15.47:0/1089190314 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e84018960 con 0x7f9e9c104ad0 2024-03-21T11:23:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.881+0000 7f9ea2800640 1 -- 172.21.15.47:0/1089190314 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9c1a1fe0 con 0x7f9e9c104ad0 2024-03-21T11:23:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.883+0000 7f9ea2800640 1 -- 172.21.15.47:0/1089190314 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e9c10aba0 con 0x7f9e9c104ad0 2024-03-21T11:23:45.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.884+0000 7f9e98c00640 1 -- 172.21.15.47:0/1089190314 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f9e84010ac0 con 0x7f9e9c104ad0 2024-03-21T11:23:45.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.885+0000 7f9e98c00640 1 --2- 172.21.15.47:0/1089190314 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9e800a2fb0 0x7f9e800a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:45.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.885+0000 7f9e98c00640 1 -- 172.21.15.47:0/1089190314 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9e840cfd50 con 0x7f9e9c104ad0 2024-03-21T11:23:45.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.885+0000 7f9e9b400640 1 --2- 172.21.15.47:0/1089190314 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9e800a2fb0 0x7f9e800a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:45.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.886+0000 7f9e9b400640 1 --2- 172.21.15.47:0/1089190314 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9e800a2fb0 0x7f9e800a5470 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c005fd0 tx=0x7f9e8c00b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:45.889+0000 7f9e98c00640 1 -- 172.21.15.47:0/1089190314 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9e8409c630 con 0x7f9e9c104ad0 2024-03-21T11:23:46.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.092+0000 7f9ea2800640 1 -- 172.21.15.47:0/1089190314 --> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e9c061ae0 con 0x7f9e800a2fb0 2024-03-21T11:23:46.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.121+0000 7f9e98c00640 1 -- 172.21.15.47:0/1089190314 <== mgr.14152 v2:172.21.15.47:6800/2057671763 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340844 (secure 0 0 0) 0x7f9e9c061ae0 con 0x7f9e800a2fb0 2024-03-21T11:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-21T11:23:46.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.130+0000 7f9ea2800640 1 -- 172.21.15.47:0/1089190314 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9e800a2fb0 msgr2=0x7f9e800a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:46.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.130+0000 7f9ea2800640 1 --2- 172.21.15.47:0/1089190314 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9e800a2fb0 0x7f9e800a5470 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c005fd0 tx=0x7f9e8c00b4a0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.130+0000 7f9ea2800640 1 -- 172.21.15.47:0/1089190314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e9c104ad0 msgr2=0x7f9e9c19a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:46.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.130+0000 7f9ea2800640 1 --2- 172.21.15.47:0/1089190314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e9c104ad0 0x7f9e9c19a0c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400e9f0 tx=0x7f9e8400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.130+0000 7f9ea2800640 1 -- 172.21.15.47:0/1089190314 shutdown_connections 2024-03-21T11:23:46.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.130+0000 7f9ea2800640 1 --2- 172.21.15.47:0/1089190314 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9e800a2fb0 0x7f9e800a5470 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.130+0000 7f9ea2800640 1 --2- 172.21.15.47:0/1089190314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e9c105490 0x7f9e9c1a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.130+0000 7f9ea2800640 1 --2- 172.21.15.47:0/1089190314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e9c104ad0 0x7f9e9c19a0c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.130+0000 7f9ea2800640 1 --2- 172.21.15.47:0/1089190314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e9c1038e0 0x7f9e9c199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.130+0000 7f9ea2800640 1 -- 172.21.15.47:0/1089190314 >> 172.21.15.47:0/1089190314 conn(0x7f9e9c0fee50 msgr2=0x7f9e9c107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:46.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.131+0000 7f9ea2800640 1 -- 172.21.15.47:0/1089190314 shutdown_connections 2024-03-21T11:23:46.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.131+0000 7f9ea2800640 1 -- 172.21.15.47:0/1089190314 wait complete. 2024-03-21T11:23:46.153 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-03-21T11:23:45.060482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669842,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1607,"num_read_kb":933,"num_write":450,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1180853,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378672,"kb_used_data":12848,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563152,"statfs":{"total":1151940427776,"available":1151552667648,"internally_reserved":0,"allocated":13156352,"data_stored":8357996,"data_compressed":435654,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6767,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":59,"num_write":13,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004592"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560653+0000","last_change":"2024-03-21T11:20:48.667185+0000","last_active":"2024-03-21T11:22:51.560653+0000","last_peered":"2024-03-21T11:22:51.560653+0000","last_clean":"2024-03-21T11:22:51.560653+0000","last_became_active":"2024-03-21T11:20:48.666855+0000","last_became_peered":"2024-03-21T11:20:48.666855+0000","last_unstale":"2024-03-21T11:22:51.560653+0000","last_undegraded":"2024-03-21T11:22:51.560653+0000","last_fullsized":"2024-03-21T11:22:51.560653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:30:37.393096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855276+0000","last_change":"2024-03-21T11:20:48.671613+0000","last_active":"2024-03-21T11:22:50.855276+0000","last_peered":"2024-03-21T11:22:50.855276+0000","last_clean":"2024-03-21T11:22:50.855276+0000","last_became_active":"2024-03-21T11:20:48.671211+0000","last_became_peered":"2024-03-21T11:20:48.671211+0000","last_unstale":"2024-03-21T11:22:50.855276+0000","last_undegraded":"2024-03-21T11:22:50.855276+0000","last_fullsized":"2024-03-21T11:22:50.855276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:20.167922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.854836+0000","last_change":"2024-03-21T11:20:48.655734+0000","last_active":"2024-03-21T11:22:50.854836+0000","last_peered":"2024-03-21T11:22:50.854836+0000","last_clean":"2024-03-21T11:22:50.854836+0000","last_became_active":"2024-03-21T11:20:48.653798+0000","last_became_peered":"2024-03-21T11:20:48.653798+0000","last_unstale":"2024-03-21T11:22:50.854836+0000","last_undegraded":"2024-03-21T11:22:50.854836+0000","last_fullsized":"2024-03-21T11:22:50.854836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:44.796405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847015+0000","last_change":"2024-03-21T11:20:48.648784+0000","last_active":"2024-03-21T11:22:50.847015+0000","last_peered":"2024-03-21T11:22:50.847015+0000","last_clean":"2024-03-21T11:22:50.847015+0000","last_became_active":"2024-03-21T11:20:48.648040+0000","last_became_peered":"2024-03-21T11:20:48.648040+0000","last_unstale":"2024-03-21T11:22:50.847015+0000","last_undegraded":"2024-03-21T11:22:50.847015+0000","last_fullsized":"2024-03-21T11:22:50.847015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:48:51.786346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846582+0000","last_change":"2024-03-21T11:20:48.672084+0000","last_active":"2024-03-21T11:22:50.846582+0000","last_peered":"2024-03-21T11:22:50.846582+0000","last_clean":"2024-03-21T11:22:50.846582+0000","last_became_active":"2024-03-21T11:20:48.671441+0000","last_became_peered":"2024-03-21T11:20:48.671441+0000","last_unstale":"2024-03-21T11:22:50.846582+0000","last_undegraded":"2024-03-21T11:22:50.846582+0000","last_fullsized":"2024-03-21T11:22:50.846582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:20:17.638270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848462+0000","last_change":"2024-03-21T11:20:48.659782+0000","last_active":"2024-03-21T11:22:50.848462+0000","last_peered":"2024-03-21T11:22:50.848462+0000","last_clean":"2024-03-21T11:22:50.848462+0000","last_became_active":"2024-03-21T11:20:48.659460+0000","last_became_peered":"2024-03-21T11:20:48.659460+0000","last_unstale":"2024-03-21T11:22:50.848462+0000","last_undegraded":"2024-03-21T11:22:50.848462+0000","last_fullsized":"2024-03-21T11:22:50.848462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:57.712654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805934+0000","last_change":"2024-03-21T11:20:48.659401+0000","last_active":"2024-03-21T11:22:51.805934+0000","last_peered":"2024-03-21T11:22:51.805934+0000","last_clean":"2024-03-21T11:22:51.805934+0000","last_became_active":"2024-03-21T11:20:48.659075+0000","last_became_peered":"2024-03-21T11:20:48.659075+0000","last_unstale":"2024-03-21T11:22:51.805934+0000","last_undegraded":"2024-03-21T11:22:51.805934+0000","last_fullsized":"2024-03-21T11:22:51.805934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:16:54.342236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847869+0000","last_change":"2024-03-21T11:20:48.667175+0000","last_active":"2024-03-21T11:22:50.847869+0000","last_peered":"2024-03-21T11:22:50.847869+0000","last_clean":"2024-03-21T11:22:50.847869+0000","last_became_active":"2024-03-21T11:20:48.666756+0000","last_became_peered":"2024-03-21T11:20:48.666756+0000","last_unstale":"2024-03-21T11:22:50.847869+0000","last_undegraded":"2024-03-21T11:22:50.847869+0000","last_fullsized":"2024-03-21T11:22:50.847869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:25:39.694699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129870+0000","last_change":"2024-03-21T11:20:48.650584+0000","last_active":"2024-03-21T11:22:51.129870+0000","last_peered":"2024-03-21T11:22:51.129870+0000","last_clean":"2024-03-21T11:22:51.129870+0000","last_became_active":"2024-03-21T11:20:48.650251+0000","last_became_peered":"2024-03-21T11:20:48.650251+0000","last_unstale":"2024-03-21T11:22:51.129870+0000","last_undegraded":"2024-03-21T11:22:51.129870+0000","last_fullsized":"2024-03-21T11:22:51.129870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:35:59.000522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848842+0000","last_change":"2024-03-21T11:20:48.652945+0000","last_active":"2024-03-21T11:22:50.848842+0000","last_peered":"2024-03-21T11:22:50.848842+0000","last_clean":"2024-03-21T11:22:50.848842+0000","last_became_active":"2024-03-21T11:20:48.652626+0000","last_became_peered":"2024-03-21T11:20:48.652626+0000","last_unstale":"2024-03-21T11:22:50.848842+0000","last_undegraded":"2024-03-21T11:22:50.848842+0000","last_fullsized":"2024-03-21T11:22:50.848842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:43.586907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806192+0000","last_change":"2024-03-21T11:20:48.651520+0000","last_active":"2024-03-21T11:22:51.806192+0000","last_peered":"2024-03-21T11:22:51.806192+0000","last_clean":"2024-03-21T11:22:51.806192+0000","last_became_active":"2024-03-21T11:20:48.651210+0000","last_became_peered":"2024-03-21T11:20:48.651210+0000","last_unstale":"2024-03-21T11:22:51.806192+0000","last_undegraded":"2024-03-21T11:22:51.806192+0000","last_fullsized":"2024-03-21T11:22:51.806192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:29.506732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846902+0000","last_change":"2024-03-21T11:20:48.649844+0000","last_active":"2024-03-21T11:22:50.846902+0000","last_peered":"2024-03-21T11:22:50.846902+0000","last_clean":"2024-03-21T11:22:50.846902+0000","last_became_active":"2024-03-21T11:20:48.649503+0000","last_became_peered":"2024-03-21T11:20:48.649503+0000","last_unstale":"2024-03-21T11:22:50.846902+0000","last_undegraded":"2024-03-21T11:22:50.846902+0000","last_fullsized":"2024-03-21T11:22:50.846902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:31:55.734583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755540+0000","last_change":"2024-03-21T11:20:48.662012+0000","last_active":"2024-03-21T11:22:51.755540+0000","last_peered":"2024-03-21T11:22:51.755540+0000","last_clean":"2024-03-21T11:22:51.755540+0000","last_became_active":"2024-03-21T11:20:48.661444+0000","last_became_peered":"2024-03-21T11:20:48.661444+0000","last_unstale":"2024-03-21T11:22:51.755540+0000","last_undegraded":"2024-03-21T11:22:51.755540+0000","last_fullsized":"2024-03-21T11:22:51.755540+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:52.493244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.854380+0000","last_change":"2024-03-21T11:20:48.663087+0000","last_active":"2024-03-21T11:22:50.854380+0000","last_peered":"2024-03-21T11:22:50.854380+0000","last_clean":"2024-03-21T11:22:50.854380+0000","last_became_active":"2024-03-21T11:20:48.662039+0000","last_became_peered":"2024-03-21T11:20:48.662039+0000","last_unstale":"2024-03-21T11:22:50.854380+0000","last_undegraded":"2024-03-21T11:22:50.854380+0000","last_fullsized":"2024-03-21T11:22:50.854380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:47:01.962314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.854313+0000","last_change":"2024-03-21T11:20:48.663478+0000","last_active":"2024-03-21T11:22:50.854313+0000","last_peered":"2024-03-21T11:22:50.854313+0000","last_clean":"2024-03-21T11:22:50.854313+0000","last_became_active":"2024-03-21T11:20:48.663135+0000","last_became_peered":"2024-03-21T11:20:48.663135+0000","last_unstale":"2024-03-21T11:22:50.854313+0000","last_undegraded":"2024-03-21T11:22:50.854313+0000","last_fullsized":"2024-03-21T11:22:50.854313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:55:29.139760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129288+0000","last_change":"2024-03-21T11:20:48.653893+0000","last_active":"2024-03-21T11:22:51.129288+0000","last_peered":"2024-03-21T11:22:51.129288+0000","last_clean":"2024-03-21T11:22:51.129288+0000","last_became_active":"2024-03-21T11:20:48.653578+0000","last_became_peered":"2024-03-21T11:20:48.653578+0000","last_unstale":"2024-03-21T11:22:51.129288+0000","last_undegraded":"2024-03-21T11:22:51.129288+0000","last_fullsized":"2024-03-21T11:22:51.129288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:54:31.254030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560545+0000","last_change":"2024-03-21T11:20:48.667388+0000","last_active":"2024-03-21T11:22:51.560545+0000","last_peered":"2024-03-21T11:22:51.560545+0000","last_clean":"2024-03-21T11:22:51.560545+0000","last_became_active":"2024-03-21T11:20:48.667077+0000","last_became_peered":"2024-03-21T11:20:48.667077+0000","last_unstale":"2024-03-21T11:22:51.560545+0000","last_undegraded":"2024-03-21T11:22:51.560545+0000","last_fullsized":"2024-03-21T11:22:51.560545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:13:32.093457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853273+0000","last_change":"2024-03-21T11:20:48.658479+0000","last_active":"2024-03-21T11:22:50.853273+0000","last_peered":"2024-03-21T11:22:50.853273+0000","last_clean":"2024-03-21T11:22:50.853273+0000","last_became_active":"2024-03-21T11:20:48.658011+0000","last_became_peered":"2024-03-21T11:20:48.658011+0000","last_unstale":"2024-03-21T11:22:50.853273+0000","last_undegraded":"2024-03-21T11:22:50.853273+0000","last_fullsized":"2024-03-21T11:22:50.853273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:33.313537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853403+0000","last_change":"2024-03-21T11:20:48.668648+0000","last_active":"2024-03-21T11:22:50.853403+0000","last_peered":"2024-03-21T11:22:50.853403+0000","last_clean":"2024-03-21T11:22:50.853403+0000","last_became_active":"2024-03-21T11:20:48.668376+0000","last_became_peered":"2024-03-21T11:20:48.668376+0000","last_unstale":"2024-03-21T11:22:50.853403+0000","last_undegraded":"2024-03-21T11:22:50.853403+0000","last_fullsized":"2024-03-21T11:22:50.853403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:49:47.601724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852874+0000","last_change":"2024-03-21T11:20:53.755600+0000","last_active":"2024-03-21T11:22:50.852874+0000","last_peered":"2024-03-21T11:22:50.852874+0000","last_clean":"2024-03-21T11:22:50.852874+0000","last_became_active":"2024-03-21T11:20:53.755321+0000","last_became_peered":"2024-03-21T11:20:53.755321+0000","last_unstale":"2024-03-21T11:22:50.852874+0000","last_undegraded":"2024-03-21T11:22:50.852874+0000","last_fullsized":"2024-03-21T11:22:50.852874+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:44:04.905910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":193,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.907144+0000","last_change":"2024-03-21T11:20:48.161001+0000","last_active":"2024-03-21T11:23:40.907144+0000","last_peered":"2024-03-21T11:23:40.907144+0000","last_clean":"2024-03-21T11:23:40.907144+0000","last_became_active":"2024-03-21T11:20:48.153053+0000","last_became_peered":"2024-03-21T11:20:48.153053+0000","last_unstale":"2024-03-21T11:23:40.907144+0000","last_undegraded":"2024-03-21T11:23:40.907144+0000","last_fullsized":"2024-03-21T11:23:40.907144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:26.010581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849345+0000","last_change":"2024-03-21T11:20:48.671430+0000","last_active":"2024-03-21T11:22:50.849345+0000","last_peered":"2024-03-21T11:22:50.849345+0000","last_clean":"2024-03-21T11:22:50.849345+0000","last_became_active":"2024-03-21T11:20:48.670192+0000","last_became_peered":"2024-03-21T11:20:48.670192+0000","last_unstale":"2024-03-21T11:22:50.849345+0000","last_undegraded":"2024-03-21T11:22:50.849345+0000","last_fullsized":"2024-03-21T11:22:50.849345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:00:53.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846554+0000","last_change":"2024-03-21T11:20:53.750313+0000","last_active":"2024-03-21T11:22:50.846554+0000","last_peered":"2024-03-21T11:22:50.846554+0000","last_clean":"2024-03-21T11:22:50.846554+0000","last_became_active":"2024-03-21T11:20:53.749913+0000","last_became_peered":"2024-03-21T11:20:53.749913+0000","last_unstale":"2024-03-21T11:22:50.846554+0000","last_undegraded":"2024-03-21T11:22:50.846554+0000","last_fullsized":"2024-03-21T11:22:50.846554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:39:09.007674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849723+0000","last_change":"2024-03-21T11:20:48.159934+0000","last_active":"2024-03-21T11:22:50.849723+0000","last_peered":"2024-03-21T11:22:50.849723+0000","last_clean":"2024-03-21T11:22:50.849723+0000","last_became_active":"2024-03-21T11:20:48.159650+0000","last_became_peered":"2024-03-21T11:20:48.159650+0000","last_unstale":"2024-03-21T11:22:50.849723+0000","last_undegraded":"2024-03-21T11:22:50.849723+0000","last_fullsized":"2024-03-21T11:22:50.849723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:49:56.891733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855277+0000","last_change":"2024-03-21T11:20:48.673012+0000","last_active":"2024-03-21T11:22:50.855277+0000","last_peered":"2024-03-21T11:22:50.855277+0000","last_clean":"2024-03-21T11:22:50.855277+0000","last_became_active":"2024-03-21T11:20:48.672584+0000","last_became_peered":"2024-03-21T11:20:48.672584+0000","last_unstale":"2024-03-21T11:22:50.855277+0000","last_undegraded":"2024-03-21T11:22:50.855277+0000","last_fullsized":"2024-03-21T11:22:50.855277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:48.213569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846699+0000","last_change":"2024-03-21T11:21:50.655957+0000","last_active":"2024-03-21T11:22:50.846699+0000","last_peered":"2024-03-21T11:22:50.846699+0000","last_clean":"2024-03-21T11:22:50.846699+0000","last_became_active":"2024-03-21T11:21:50.655477+0000","last_became_peered":"2024-03-21T11:21:50.655477+0000","last_unstale":"2024-03-21T11:22:50.846699+0000","last_undegraded":"2024-03-21T11:22:50.846699+0000","last_fullsized":"2024-03-21T11:22:50.846699+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:58:28.476186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'13","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560446+0000","last_change":"2024-03-21T11:21:50.664575+0000","last_active":"2024-03-21T11:22:51.560446+0000","last_peered":"2024-03-21T11:22:51.560446+0000","last_clean":"2024-03-21T11:22:51.560446+0000","last_became_active":"2024-03-21T11:21:50.647335+0000","last_became_peered":"2024-03-21T11:21:50.647335+0000","last_unstale":"2024-03-21T11:22:51.560446+0000","last_undegraded":"2024-03-21T11:22:51.560446+0000","last_fullsized":"2024-03-21T11:22:51.560446+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:50:28.567378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129559+0000","last_change":"2024-03-21T11:20:48.662392+0000","last_active":"2024-03-21T11:22:51.129559+0000","last_peered":"2024-03-21T11:22:51.129559+0000","last_clean":"2024-03-21T11:22:51.129559+0000","last_became_active":"2024-03-21T11:20:48.662042+0000","last_became_peered":"2024-03-21T11:20:48.662042+0000","last_unstale":"2024-03-21T11:22:51.129559+0000","last_undegraded":"2024-03-21T11:22:51.129559+0000","last_fullsized":"2024-03-21T11:22:51.129559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:59:34.456582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846900+0000","last_change":"2024-03-21T11:20:53.749693+0000","last_active":"2024-03-21T11:22:50.846900+0000","last_peered":"2024-03-21T11:22:50.846900+0000","last_clean":"2024-03-21T11:22:50.846900+0000","last_became_active":"2024-03-21T11:20:53.749438+0000","last_became_peered":"2024-03-21T11:20:53.749438+0000","last_unstale":"2024-03-21T11:22:50.846900+0000","last_undegraded":"2024-03-21T11:22:50.846900+0000","last_fullsized":"2024-03-21T11:22:50.846900+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:37.714471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846726+0000","last_change":"2024-03-21T11:20:48.132587+0000","last_active":"2024-03-21T11:22:50.846726+0000","last_peered":"2024-03-21T11:22:50.846726+0000","last_clean":"2024-03-21T11:22:50.846726+0000","last_became_active":"2024-03-21T11:20:48.132232+0000","last_became_peered":"2024-03-21T11:20:48.132232+0000","last_unstale":"2024-03-21T11:22:50.846726+0000","last_undegraded":"2024-03-21T11:22:50.846726+0000","last_fullsized":"2024-03-21T11:22:50.846726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:20:45.217545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921559+0000","last_change":"2024-03-21T11:20:48.656881+0000","last_active":"2024-03-21T11:22:50.921559+0000","last_peered":"2024-03-21T11:22:50.921559+0000","last_clean":"2024-03-21T11:22:50.921559+0000","last_became_active":"2024-03-21T11:20:48.656426+0000","last_became_peered":"2024-03-21T11:20:48.656426+0000","last_unstale":"2024-03-21T11:22:50.921559+0000","last_undegraded":"2024-03-21T11:22:50.921559+0000","last_fullsized":"2024-03-21T11:22:50.921559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:06:27.398062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852990+0000","last_change":"2024-03-21T11:20:53.733396+0000","last_active":"2024-03-21T11:22:50.852990+0000","last_peered":"2024-03-21T11:22:50.852990+0000","last_clean":"2024-03-21T11:22:50.852990+0000","last_became_active":"2024-03-21T11:20:53.732985+0000","last_became_peered":"2024-03-21T11:20:53.732985+0000","last_unstale":"2024-03-21T11:22:50.852990+0000","last_undegraded":"2024-03-21T11:22:50.852990+0000","last_fullsized":"2024-03-21T11:22:50.852990+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:54:14.064761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":189,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.854844+0000","last_change":"2024-03-21T11:20:48.133362+0000","last_active":"2024-03-21T11:23:40.854844+0000","last_peered":"2024-03-21T11:23:40.854844+0000","last_clean":"2024-03-21T11:23:40.854844+0000","last_became_active":"2024-03-21T11:20:48.132405+0000","last_became_peered":"2024-03-21T11:20:48.132405+0000","last_unstale":"2024-03-21T11:23:40.854844+0000","last_undegraded":"2024-03-21T11:23:40.854844+0000","last_fullsized":"2024-03-21T11:23:40.854844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:48:52.688948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1460,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":148,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846447+0000","last_change":"2024-03-21T11:20:48.659610+0000","last_active":"2024-03-21T11:22:50.846447+0000","last_peered":"2024-03-21T11:22:50.846447+0000","last_clean":"2024-03-21T11:22:50.846447+0000","last_became_active":"2024-03-21T11:20:48.659257+0000","last_became_peered":"2024-03-21T11:20:48.659257+0000","last_unstale":"2024-03-21T11:22:50.846447+0000","last_undegraded":"2024-03-21T11:22:50.846447+0000","last_fullsized":"2024-03-21T11:22:50.846447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:17:30.744434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852994+0000","last_change":"2024-03-21T11:21:50.656047+0000","last_active":"2024-03-21T11:22:50.852994+0000","last_peered":"2024-03-21T11:22:50.852994+0000","last_clean":"2024-03-21T11:22:50.852994+0000","last_became_active":"2024-03-21T11:21:50.655554+0000","last_became_peered":"2024-03-21T11:21:50.655554+0000","last_unstale":"2024-03-21T11:22:50.852994+0000","last_undegraded":"2024-03-21T11:22:50.852994+0000","last_fullsized":"2024-03-21T11:22:50.852994+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:35:15.858237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848356+0000","last_change":"2024-03-21T11:21:50.695331+0000","last_active":"2024-03-21T11:22:50.848356+0000","last_peered":"2024-03-21T11:22:50.848356+0000","last_clean":"2024-03-21T11:22:50.848356+0000","last_became_active":"2024-03-21T11:21:50.690368+0000","last_became_peered":"2024-03-21T11:21:50.690368+0000","last_unstale":"2024-03-21T11:22:50.848356+0000","last_undegraded":"2024-03-21T11:22:50.848356+0000","last_fullsized":"2024-03-21T11:22:50.848356+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:50:03.835364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921486+0000","last_change":"2024-03-21T11:20:48.671221+0000","last_active":"2024-03-21T11:22:50.921486+0000","last_peered":"2024-03-21T11:22:50.921486+0000","last_clean":"2024-03-21T11:22:50.921486+0000","last_became_active":"2024-03-21T11:20:48.670837+0000","last_became_peered":"2024-03-21T11:20:48.670837+0000","last_unstale":"2024-03-21T11:22:50.921486+0000","last_undegraded":"2024-03-21T11:22:50.921486+0000","last_fullsized":"2024-03-21T11:22:50.921486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:38.680409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560403+0000","last_change":"2024-03-21T11:20:53.747501+0000","last_active":"2024-03-21T11:22:51.560403+0000","last_peered":"2024-03-21T11:22:51.560403+0000","last_clean":"2024-03-21T11:22:51.560403+0000","last_became_active":"2024-03-21T11:20:53.747062+0000","last_became_peered":"2024-03-21T11:20:53.747062+0000","last_unstale":"2024-03-21T11:22:51.560403+0000","last_undegraded":"2024-03-21T11:22:51.560403+0000","last_fullsized":"2024-03-21T11:22:51.560403+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:58:36.904221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.754889+0000","last_change":"2024-03-21T11:20:48.156175+0000","last_active":"2024-03-21T11:22:51.754889+0000","last_peered":"2024-03-21T11:22:51.754889+0000","last_clean":"2024-03-21T11:22:51.754889+0000","last_became_active":"2024-03-21T11:20:48.154271+0000","last_became_peered":"2024-03-21T11:20:48.154271+0000","last_unstale":"2024-03-21T11:22:51.754889+0000","last_undegraded":"2024-03-21T11:22:51.754889+0000","last_fullsized":"2024-03-21T11:22:51.754889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:29:04.216391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848061+0000","last_change":"2024-03-21T11:20:48.670011+0000","last_active":"2024-03-21T11:22:50.848061+0000","last_peered":"2024-03-21T11:22:50.848061+0000","last_clean":"2024-03-21T11:22:50.848061+0000","last_became_active":"2024-03-21T11:20:48.669016+0000","last_became_peered":"2024-03-21T11:20:48.669016+0000","last_unstale":"2024-03-21T11:22:50.848061+0000","last_undegraded":"2024-03-21T11:22:50.848061+0000","last_fullsized":"2024-03-21T11:22:50.848061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:43:18.906490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855150+0000","last_change":"2024-03-21T11:20:53.751934+0000","last_active":"2024-03-21T11:22:50.855150+0000","last_peered":"2024-03-21T11:22:50.855150+0000","last_clean":"2024-03-21T11:22:50.855150+0000","last_became_active":"2024-03-21T11:20:53.751413+0000","last_became_peered":"2024-03-21T11:20:53.751413+0000","last_unstale":"2024-03-21T11:22:50.855150+0000","last_undegraded":"2024-03-21T11:22:50.855150+0000","last_fullsized":"2024-03-21T11:22:50.855150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:12:33.880376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921223+0000","last_change":"2024-03-21T11:20:48.131796+0000","last_active":"2024-03-21T11:22:50.921223+0000","last_peered":"2024-03-21T11:22:50.921223+0000","last_clean":"2024-03-21T11:22:50.921223+0000","last_became_active":"2024-03-21T11:20:48.131478+0000","last_became_peered":"2024-03-21T11:20:48.131478+0000","last_unstale":"2024-03-21T11:22:50.921223+0000","last_undegraded":"2024-03-21T11:22:50.921223+0000","last_fullsized":"2024-03-21T11:22:50.921223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:59:24.871300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849740+0000","last_change":"2024-03-21T11:20:48.660046+0000","last_active":"2024-03-21T11:22:50.849740+0000","last_peered":"2024-03-21T11:22:50.849740+0000","last_clean":"2024-03-21T11:22:50.849740+0000","last_became_active":"2024-03-21T11:20:48.659621+0000","last_became_peered":"2024-03-21T11:20:48.659621+0000","last_unstale":"2024-03-21T11:22:50.849740+0000","last_undegraded":"2024-03-21T11:22:50.849740+0000","last_fullsized":"2024-03-21T11:22:50.849740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:47:01.923285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848718+0000","last_change":"2024-03-21T11:20:53.754458+0000","last_active":"2024-03-21T11:22:50.848718+0000","last_peered":"2024-03-21T11:22:50.848718+0000","last_clean":"2024-03-21T11:22:50.848718+0000","last_became_active":"2024-03-21T11:20:53.753927+0000","last_became_peered":"2024-03-21T11:20:53.753927+0000","last_unstale":"2024-03-21T11:22:50.848718+0000","last_undegraded":"2024-03-21T11:22:50.848718+0000","last_fullsized":"2024-03-21T11:22:50.848718+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:00:47.392699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'7","reported_seq":327,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.756351+0000","last_change":"2024-03-21T11:20:48.154125+0000","last_active":"2024-03-21T11:23:40.756351+0000","last_peered":"2024-03-21T11:23:40.756351+0000","last_clean":"2024-03-21T11:23:40.756351+0000","last_became_active":"2024-03-21T11:20:48.153857+0000","last_became_peered":"2024-03-21T11:20:48.153857+0000","last_unstale":"2024-03-21T11:23:40.756351+0000","last_undegraded":"2024-03-21T11:23:40.756351+0000","last_fullsized":"2024-03-21T11:23:40.756351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:54:06.525335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":286,"num_read_kb":286,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129808+0000","last_change":"2024-03-21T11:20:48.650143+0000","last_active":"2024-03-21T11:22:51.129808+0000","last_peered":"2024-03-21T11:22:51.129808+0000","last_clean":"2024-03-21T11:22:51.129808+0000","last_became_active":"2024-03-21T11:20:48.649825+0000","last_became_peered":"2024-03-21T11:20:48.649825+0000","last_unstale":"2024-03-21T11:22:51.129808+0000","last_undegraded":"2024-03-21T11:22:51.129808+0000","last_fullsized":"2024-03-21T11:22:51.129808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:09:15.735280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.850023+0000","last_change":"2024-03-21T11:20:53.743006+0000","last_active":"2024-03-21T11:22:50.850023+0000","last_peered":"2024-03-21T11:22:50.850023+0000","last_clean":"2024-03-21T11:22:50.850023+0000","last_became_active":"2024-03-21T11:20:53.742587+0000","last_became_peered":"2024-03-21T11:20:53.742587+0000","last_unstale":"2024-03-21T11:22:50.850023+0000","last_undegraded":"2024-03-21T11:22:50.850023+0000","last_fullsized":"2024-03-21T11:22:50.850023+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:06:06.845791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806093+0000","last_change":"2024-03-21T11:20:48.137137+0000","last_active":"2024-03-21T11:22:51.806093+0000","last_peered":"2024-03-21T11:22:51.806093+0000","last_clean":"2024-03-21T11:22:51.806093+0000","last_became_active":"2024-03-21T11:20:48.136790+0000","last_became_peered":"2024-03-21T11:20:48.136790+0000","last_unstale":"2024-03-21T11:22:51.806093+0000","last_undegraded":"2024-03-21T11:22:51.806093+0000","last_fullsized":"2024-03-21T11:22:51.806093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:22:47.223969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855575+0000","last_change":"2024-03-21T11:20:48.673568+0000","last_active":"2024-03-21T11:22:50.855575+0000","last_peered":"2024-03-21T11:22:50.855575+0000","last_clean":"2024-03-21T11:22:50.855575+0000","last_became_active":"2024-03-21T11:20:48.672613+0000","last_became_peered":"2024-03-21T11:20:48.672613+0000","last_unstale":"2024-03-21T11:22:50.855575+0000","last_undegraded":"2024-03-21T11:22:50.855575+0000","last_fullsized":"2024-03-21T11:22:50.855575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:43:38.193802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846803+0000","last_change":"2024-03-21T11:20:53.739228+0000","last_active":"2024-03-21T11:22:50.846803+0000","last_peered":"2024-03-21T11:22:50.846803+0000","last_clean":"2024-03-21T11:22:50.846803+0000","last_became_active":"2024-03-21T11:20:53.738750+0000","last_became_peered":"2024-03-21T11:20:53.738750+0000","last_unstale":"2024-03-21T11:22:50.846803+0000","last_undegraded":"2024-03-21T11:22:50.846803+0000","last_fullsized":"2024-03-21T11:22:50.846803+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:58:33.912076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921548+0000","last_change":"2024-03-21T11:20:48.144451+0000","last_active":"2024-03-21T11:22:50.921548+0000","last_peered":"2024-03-21T11:22:50.921548+0000","last_clean":"2024-03-21T11:22:50.921548+0000","last_became_active":"2024-03-21T11:20:48.144074+0000","last_became_peered":"2024-03-21T11:20:48.144074+0000","last_unstale":"2024-03-21T11:22:50.921548+0000","last_undegraded":"2024-03-21T11:22:50.921548+0000","last_fullsized":"2024-03-21T11:22:50.921548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:33:17.032461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129797+0000","last_change":"2024-03-21T11:20:48.654472+0000","last_active":"2024-03-21T11:22:51.129797+0000","last_peered":"2024-03-21T11:22:51.129797+0000","last_clean":"2024-03-21T11:22:51.129797+0000","last_became_active":"2024-03-21T11:20:48.654129+0000","last_became_peered":"2024-03-21T11:20:48.654129+0000","last_unstale":"2024-03-21T11:22:51.129797+0000","last_undegraded":"2024-03-21T11:22:51.129797+0000","last_fullsized":"2024-03-21T11:22:51.129797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:28:31.224296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849451+0000","last_change":"2024-03-21T11:20:53.739994+0000","last_active":"2024-03-21T11:22:50.849451+0000","last_peered":"2024-03-21T11:22:50.849451+0000","last_clean":"2024-03-21T11:22:50.849451+0000","last_became_active":"2024-03-21T11:20:53.738319+0000","last_became_peered":"2024-03-21T11:20:53.738319+0000","last_unstale":"2024-03-21T11:22:50.849451+0000","last_undegraded":"2024-03-21T11:22:50.849451+0000","last_fullsized":"2024-03-21T11:22:50.849451+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:55.694986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":371,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.757016+0000","last_change":"2024-03-21T11:20:48.131461+0000","last_active":"2024-03-21T11:23:40.757016+0000","last_peered":"2024-03-21T11:23:40.757016+0000","last_clean":"2024-03-21T11:23:40.757016+0000","last_became_active":"2024-03-21T11:20:48.131079+0000","last_became_peered":"2024-03-21T11:20:48.131079+0000","last_unstale":"2024-03-21T11:23:40.757016+0000","last_undegraded":"2024-03-21T11:23:40.757016+0000","last_fullsized":"2024-03-21T11:23:40.757016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:48:57.528444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70525,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":296,"num_read_kb":217,"num_write":43,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846832+0000","last_change":"2024-03-21T11:20:48.653696+0000","last_active":"2024-03-21T11:22:50.846832+0000","last_peered":"2024-03-21T11:22:50.846832+0000","last_clean":"2024-03-21T11:22:50.846832+0000","last_became_active":"2024-03-21T11:20:48.653218+0000","last_became_peered":"2024-03-21T11:20:48.653218+0000","last_unstale":"2024-03-21T11:22:50.846832+0000","last_undegraded":"2024-03-21T11:22:50.846832+0000","last_fullsized":"2024-03-21T11:22:50.846832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:11:59.419040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921748+0000","last_change":"2024-03-21T11:20:53.754019+0000","last_active":"2024-03-21T11:22:50.921748+0000","last_peered":"2024-03-21T11:22:50.921748+0000","last_clean":"2024-03-21T11:22:50.921748+0000","last_became_active":"2024-03-21T11:20:53.753606+0000","last_became_peered":"2024-03-21T11:20:53.753606+0000","last_unstale":"2024-03-21T11:22:50.921748+0000","last_undegraded":"2024-03-21T11:22:50.921748+0000","last_fullsized":"2024-03-21T11:22:50.921748+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:23:27.898596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853065+0000","last_change":"2024-03-21T11:20:48.153544+0000","last_active":"2024-03-21T11:22:50.853065+0000","last_peered":"2024-03-21T11:22:50.853065+0000","last_clean":"2024-03-21T11:22:50.853065+0000","last_became_active":"2024-03-21T11:20:48.153279+0000","last_became_peered":"2024-03-21T11:20:48.153279+0000","last_unstale":"2024-03-21T11:22:50.853065+0000","last_undegraded":"2024-03-21T11:22:50.853065+0000","last_fullsized":"2024-03-21T11:22:50.853065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:31:31.911844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755158+0000","last_change":"2024-03-21T11:20:48.665140+0000","last_active":"2024-03-21T11:22:51.755158+0000","last_peered":"2024-03-21T11:22:51.755158+0000","last_clean":"2024-03-21T11:22:51.755158+0000","last_became_active":"2024-03-21T11:20:48.664226+0000","last_became_peered":"2024-03-21T11:20:48.664226+0000","last_unstale":"2024-03-21T11:22:51.755158+0000","last_undegraded":"2024-03-21T11:22:51.755158+0000","last_fullsized":"2024-03-21T11:22:51.755158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:47:38.951403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560135+0000","last_change":"2024-03-21T11:20:53.751887+0000","last_active":"2024-03-21T11:22:51.560135+0000","last_peered":"2024-03-21T11:22:51.560135+0000","last_clean":"2024-03-21T11:22:51.560135+0000","last_became_active":"2024-03-21T11:20:53.751353+0000","last_became_peered":"2024-03-21T11:20:53.751353+0000","last_unstale":"2024-03-21T11:22:51.560135+0000","last_undegraded":"2024-03-21T11:22:51.560135+0000","last_fullsized":"2024-03-21T11:22:51.560135+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:21:36.689786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921390+0000","last_change":"2024-03-21T11:20:48.162150+0000","last_active":"2024-03-21T11:22:50.921390+0000","last_peered":"2024-03-21T11:22:50.921390+0000","last_clean":"2024-03-21T11:22:50.921390+0000","last_became_active":"2024-03-21T11:20:48.161825+0000","last_became_peered":"2024-03-21T11:20:48.161825+0000","last_unstale":"2024-03-21T11:22:50.921390+0000","last_undegraded":"2024-03-21T11:22:50.921390+0000","last_fullsized":"2024-03-21T11:22:50.921390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:10:47.097540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805965+0000","last_change":"2024-03-21T11:20:48.663856+0000","last_active":"2024-03-21T11:22:51.805965+0000","last_peered":"2024-03-21T11:22:51.805965+0000","last_clean":"2024-03-21T11:22:51.805965+0000","last_became_active":"2024-03-21T11:20:48.662908+0000","last_became_peered":"2024-03-21T11:20:48.662908+0000","last_unstale":"2024-03-21T11:22:51.805965+0000","last_undegraded":"2024-03-21T11:22:51.805965+0000","last_fullsized":"2024-03-21T11:22:51.805965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:11:12.255265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.851969+0000","last_change":"2024-03-21T11:20:53.755942+0000","last_active":"2024-03-21T11:22:50.851969+0000","last_peered":"2024-03-21T11:22:50.851969+0000","last_clean":"2024-03-21T11:22:50.851969+0000","last_became_active":"2024-03-21T11:20:53.755141+0000","last_became_peered":"2024-03-21T11:20:53.755141+0000","last_unstale":"2024-03-21T11:22:50.851969+0000","last_undegraded":"2024-03-21T11:22:50.851969+0000","last_fullsized":"2024-03-21T11:22:50.851969+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:12:30.141117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":187,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.842723+0000","last_change":"2024-03-21T11:20:48.132460+0000","last_active":"2024-03-21T11:23:40.842723+0000","last_peered":"2024-03-21T11:23:40.842723+0000","last_clean":"2024-03-21T11:23:40.842723+0000","last_became_active":"2024-03-21T11:20:48.131573+0000","last_became_peered":"2024-03-21T11:20:48.131573+0000","last_unstale":"2024-03-21T11:23:40.842723+0000","last_undegraded":"2024-03-21T11:23:40.842723+0000","last_fullsized":"2024-03-21T11:23:40.842723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:40:02.821536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805992+0000","last_change":"2024-03-21T11:20:48.656262+0000","last_active":"2024-03-21T11:22:51.805992+0000","last_peered":"2024-03-21T11:22:51.805992+0000","last_clean":"2024-03-21T11:22:51.805992+0000","last_became_active":"2024-03-21T11:20:48.655924+0000","last_became_peered":"2024-03-21T11:20:48.655924+0000","last_unstale":"2024-03-21T11:22:51.805992+0000","last_undegraded":"2024-03-21T11:22:51.805992+0000","last_fullsized":"2024-03-21T11:22:51.805992+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:24:20.412185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755346+0000","last_change":"2024-03-21T11:20:53.758666+0000","last_active":"2024-03-21T11:22:51.755346+0000","last_peered":"2024-03-21T11:22:51.755346+0000","last_clean":"2024-03-21T11:22:51.755346+0000","last_became_active":"2024-03-21T11:20:53.757398+0000","last_became_peered":"2024-03-21T11:20:53.757398+0000","last_unstale":"2024-03-21T11:22:51.755346+0000","last_undegraded":"2024-03-21T11:22:51.755346+0000","last_fullsized":"2024-03-21T11:22:51.755346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:34.198388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846374+0000","last_change":"2024-03-21T11:20:48.132725+0000","last_active":"2024-03-21T11:22:50.846374+0000","last_peered":"2024-03-21T11:22:50.846374+0000","last_clean":"2024-03-21T11:22:50.846374+0000","last_became_active":"2024-03-21T11:20:48.132367+0000","last_became_peered":"2024-03-21T11:20:48.132367+0000","last_unstale":"2024-03-21T11:22:50.846374+0000","last_undegraded":"2024-03-21T11:22:50.846374+0000","last_fullsized":"2024-03-21T11:22:50.846374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:19:18.334962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921197+0000","last_change":"2024-03-21T11:20:48.656302+0000","last_active":"2024-03-21T11:22:50.921197+0000","last_peered":"2024-03-21T11:22:50.921197+0000","last_clean":"2024-03-21T11:22:50.921197+0000","last_became_active":"2024-03-21T11:20:48.654305+0000","last_became_peered":"2024-03-21T11:20:48.654305+0000","last_unstale":"2024-03-21T11:22:50.921197+0000","last_undegraded":"2024-03-21T11:22:50.921197+0000","last_fullsized":"2024-03-21T11:22:50.921197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:59:40.766829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921110+0000","last_change":"2024-03-21T11:20:53.750710+0000","last_active":"2024-03-21T11:22:50.921110+0000","last_peered":"2024-03-21T11:22:50.921110+0000","last_clean":"2024-03-21T11:22:50.921110+0000","last_became_active":"2024-03-21T11:20:53.750255+0000","last_became_peered":"2024-03-21T11:20:53.750255+0000","last_unstale":"2024-03-21T11:22:50.921110+0000","last_undegraded":"2024-03-21T11:22:50.921110+0000","last_fullsized":"2024-03-21T11:22:50.921110+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:36:28.611838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846888+0000","last_change":"2024-03-21T11:20:48.133524+0000","last_active":"2024-03-21T11:22:50.846888+0000","last_peered":"2024-03-21T11:22:50.846888+0000","last_clean":"2024-03-21T11:22:50.846888+0000","last_became_active":"2024-03-21T11:20:48.133186+0000","last_became_peered":"2024-03-21T11:20:48.133186+0000","last_unstale":"2024-03-21T11:22:50.846888+0000","last_undegraded":"2024-03-21T11:22:50.846888+0000","last_fullsized":"2024-03-21T11:22:50.846888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:02.018639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560425+0000","last_change":"2024-03-21T11:20:48.654126+0000","last_active":"2024-03-21T11:22:51.560425+0000","last_peered":"2024-03-21T11:22:51.560425+0000","last_clean":"2024-03-21T11:22:51.560425+0000","last_became_active":"2024-03-21T11:20:48.653835+0000","last_became_peered":"2024-03-21T11:20:48.653835+0000","last_unstale":"2024-03-21T11:22:51.560425+0000","last_undegraded":"2024-03-21T11:22:51.560425+0000","last_fullsized":"2024-03-21T11:22:51.560425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:12:03.332076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846394+0000","last_change":"2024-03-21T11:20:53.916645+0000","last_active":"2024-03-21T11:22:50.846394+0000","last_peered":"2024-03-21T11:22:50.846394+0000","last_clean":"2024-03-21T11:22:50.846394+0000","last_became_active":"2024-03-21T11:20:53.916178+0000","last_became_peered":"2024-03-21T11:20:53.916178+0000","last_unstale":"2024-03-21T11:22:50.846394+0000","last_undegraded":"2024-03-21T11:22:50.846394+0000","last_fullsized":"2024-03-21T11:22:50.846394+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:56:15.805231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.851892+0000","last_change":"2024-03-21T11:20:48.159877+0000","last_active":"2024-03-21T11:22:50.851892+0000","last_peered":"2024-03-21T11:22:50.851892+0000","last_clean":"2024-03-21T11:22:50.851892+0000","last_became_active":"2024-03-21T11:20:48.159529+0000","last_became_peered":"2024-03-21T11:20:48.159529+0000","last_unstale":"2024-03-21T11:22:50.851892+0000","last_undegraded":"2024-03-21T11:22:50.851892+0000","last_fullsized":"2024-03-21T11:22:50.851892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:17:09.662462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848812+0000","last_change":"2024-03-21T11:20:48.663311+0000","last_active":"2024-03-21T11:22:50.848812+0000","last_peered":"2024-03-21T11:22:50.848812+0000","last_clean":"2024-03-21T11:22:50.848812+0000","last_became_active":"2024-03-21T11:20:48.662961+0000","last_became_peered":"2024-03-21T11:20:48.662961+0000","last_unstale":"2024-03-21T11:22:50.848812+0000","last_undegraded":"2024-03-21T11:22:50.848812+0000","last_fullsized":"2024-03-21T11:22:50.848812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:14:46.616326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755080+0000","last_change":"2024-03-21T11:20:53.751804+0000","last_active":"2024-03-21T11:22:51.755080+0000","last_peered":"2024-03-21T11:22:51.755080+0000","last_clean":"2024-03-21T11:22:51.755080+0000","last_became_active":"2024-03-21T11:20:53.751323+0000","last_became_peered":"2024-03-21T11:20:53.751323+0000","last_unstale":"2024-03-21T11:22:51.755080+0000","last_undegraded":"2024-03-21T11:22:51.755080+0000","last_fullsized":"2024-03-21T11:22:51.755080+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:38:25.677345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":186,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.908004+0000","last_change":"2024-03-21T11:20:48.147580+0000","last_active":"2024-03-21T11:23:40.908004+0000","last_peered":"2024-03-21T11:23:40.908004+0000","last_clean":"2024-03-21T11:23:40.908004+0000","last_became_active":"2024-03-21T11:20:48.147222+0000","last_became_peered":"2024-03-21T11:20:48.147222+0000","last_unstale":"2024-03-21T11:23:40.908004+0000","last_undegraded":"2024-03-21T11:23:40.908004+0000","last_fullsized":"2024-03-21T11:23:40.908004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:49:04.204129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.856848+0000","last_change":"2024-03-21T11:22:50.856453+0000","last_active":"2024-03-21T11:22:50.856848+0000","last_peered":"2024-03-21T11:22:50.856848+0000","last_clean":"2024-03-21T11:22:50.856848+0000","last_became_active":"2024-03-21T11:22:50.855987+0000","last_became_peered":"2024-03-21T11:22:50.855987+0000","last_unstale":"2024-03-21T11:22:50.856848+0000","last_undegraded":"2024-03-21T11:22:50.856848+0000","last_fullsized":"2024-03-21T11:22:50.856848+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:32:05.460528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755558+0000","last_change":"2024-03-21T11:20:53.916931+0000","last_active":"2024-03-21T11:22:51.755558+0000","last_peered":"2024-03-21T11:22:51.755558+0000","last_clean":"2024-03-21T11:22:51.755558+0000","last_became_active":"2024-03-21T11:20:53.916495+0000","last_became_peered":"2024-03-21T11:20:53.916495+0000","last_unstale":"2024-03-21T11:22:51.755558+0000","last_undegraded":"2024-03-21T11:22:51.755558+0000","last_fullsized":"2024-03-21T11:22:51.755558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:18:33.046052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855560+0000","last_change":"2024-03-21T11:22:50.854745+0000","last_active":"2024-03-21T11:22:50.855560+0000","last_peered":"2024-03-21T11:22:50.855560+0000","last_clean":"2024-03-21T11:22:50.855560+0000","last_became_active":"2024-03-21T11:22:50.854201+0000","last_became_peered":"2024-03-21T11:22:50.854201+0000","last_unstale":"2024-03-21T11:22:50.855560+0000","last_undegraded":"2024-03-21T11:22:50.855560+0000","last_fullsized":"2024-03-21T11:22:50.855560+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:06:27.288363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":184,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:40.842225+0000","last_change":"2024-03-21T11:20:48.164607+0000","last_active":"2024-03-21T11:23:40.842225+0000","last_peered":"2024-03-21T11:23:40.842225+0000","last_clean":"2024-03-21T11:23:40.842225+0000","last_became_active":"2024-03-21T11:20:48.164222+0000","last_became_peered":"2024-03-21T11:20:48.164222+0000","last_unstale":"2024-03-21T11:23:40.842225+0000","last_undegraded":"2024-03-21T11:23:40.842225+0000","last_fullsized":"2024-03-21T11:23:40.842225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:52:24.359155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806069+0000","last_change":"2024-03-21T11:21:50.658412+0000","last_active":"2024-03-21T11:22:51.806069+0000","last_peered":"2024-03-21T11:22:51.806069+0000","last_clean":"2024-03-21T11:22:51.806069+0000","last_became_active":"2024-03-21T11:21:50.649270+0000","last_became_peered":"2024-03-21T11:21:50.649270+0000","last_unstale":"2024-03-21T11:22:51.806069+0000","last_undegraded":"2024-03-21T11:22:51.806069+0000","last_fullsized":"2024-03-21T11:22:51.806069+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:11:06.006302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848564+0000","last_change":"2024-03-21T11:20:48.667212+0000","last_active":"2024-03-21T11:22:50.848564+0000","last_peered":"2024-03-21T11:22:50.848564+0000","last_clean":"2024-03-21T11:22:50.848564+0000","last_became_active":"2024-03-21T11:20:48.662839+0000","last_became_peered":"2024-03-21T11:20:48.662839+0000","last_unstale":"2024-03-21T11:22:50.848564+0000","last_undegraded":"2024-03-21T11:22:50.848564+0000","last_fullsized":"2024-03-21T11:22:50.848564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:27:41.490922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846727+0000","last_change":"2024-03-21T11:20:48.151185+0000","last_active":"2024-03-21T11:22:50.846727+0000","last_peered":"2024-03-21T11:22:50.846727+0000","last_clean":"2024-03-21T11:22:50.846727+0000","last_became_active":"2024-03-21T11:20:48.150859+0000","last_became_peered":"2024-03-21T11:20:48.150859+0000","last_unstale":"2024-03-21T11:22:50.846727+0000","last_undegraded":"2024-03-21T11:22:50.846727+0000","last_fullsized":"2024-03-21T11:22:50.846727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:52:45.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":357,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805649+0000","last_change":"2024-03-21T11:19:28.916140+0000","last_active":"2024-03-21T11:22:51.805649+0000","last_peered":"2024-03-21T11:22:51.805649+0000","last_clean":"2024-03-21T11:22:51.805649+0000","last_became_active":"2024-03-21T11:19:28.915634+0000","last_became_peered":"2024-03-21T11:19:28.915634+0000","last_unstale":"2024-03-21T11:22:51.805649+0000","last_undegraded":"2024-03-21T11:22:51.805649+0000","last_fullsized":"2024-03-21T11:22:51.805649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:15:35.803515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:15:35.803515+0000","last_clean_scrub_stamp":"2024-03-21T11:15:35.803515+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:08:31.509894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921613+0000","last_change":"2024-03-21T11:21:50.656708+0000","last_active":"2024-03-21T11:22:50.921613+0000","last_peered":"2024-03-21T11:22:50.921613+0000","last_clean":"2024-03-21T11:22:50.921613+0000","last_became_active":"2024-03-21T11:21:50.656236+0000","last_became_peered":"2024-03-21T11:21:50.656236+0000","last_unstale":"2024-03-21T11:22:50.921613+0000","last_undegraded":"2024-03-21T11:22:50.921613+0000","last_fullsized":"2024-03-21T11:22:50.921613+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:17:44.114512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.754767+0000","last_change":"2024-03-21T11:20:48.657878+0000","last_active":"2024-03-21T11:22:51.754767+0000","last_peered":"2024-03-21T11:22:51.754767+0000","last_clean":"2024-03-21T11:22:51.754767+0000","last_became_active":"2024-03-21T11:20:48.657212+0000","last_became_peered":"2024-03-21T11:20:48.657212+0000","last_unstale":"2024-03-21T11:22:51.754767+0000","last_undegraded":"2024-03-21T11:22:51.754767+0000","last_fullsized":"2024-03-21T11:22:51.754767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:43:11.993577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805832+0000","last_change":"2024-03-21T11:20:48.135513+0000","last_active":"2024-03-21T11:22:51.805832+0000","last_peered":"2024-03-21T11:22:51.805832+0000","last_clean":"2024-03-21T11:22:51.805832+0000","last_became_active":"2024-03-21T11:20:48.135223+0000","last_became_peered":"2024-03-21T11:20:48.135223+0000","last_unstale":"2024-03-21T11:22:51.805832+0000","last_undegraded":"2024-03-21T11:22:51.805832+0000","last_fullsized":"2024-03-21T11:22:51.805832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:37:38.834284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560138+0000","last_change":"2024-03-21T11:20:53.751790+0000","last_active":"2024-03-21T11:22:51.560138+0000","last_peered":"2024-03-21T11:22:51.560138+0000","last_clean":"2024-03-21T11:22:51.560138+0000","last_became_active":"2024-03-21T11:20:53.751312+0000","last_became_peered":"2024-03-21T11:20:53.751312+0000","last_unstale":"2024-03-21T11:22:51.560138+0000","last_undegraded":"2024-03-21T11:22:51.560138+0000","last_fullsized":"2024-03-21T11:22:51.560138+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:21:41.599985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846835+0000","last_change":"2024-03-21T11:20:48.154119+0000","last_active":"2024-03-21T11:22:50.846835+0000","last_peered":"2024-03-21T11:22:50.846835+0000","last_clean":"2024-03-21T11:22:50.846835+0000","last_became_active":"2024-03-21T11:20:48.153781+0000","last_became_peered":"2024-03-21T11:20:48.153781+0000","last_unstale":"2024-03-21T11:22:50.846835+0000","last_undegraded":"2024-03-21T11:22:50.846835+0000","last_fullsized":"2024-03-21T11:22:50.846835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:44:13.937836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755223+0000","last_change":"2024-03-21T11:20:48.661282+0000","last_active":"2024-03-21T11:22:51.755223+0000","last_peered":"2024-03-21T11:22:51.755223+0000","last_clean":"2024-03-21T11:22:51.755223+0000","last_became_active":"2024-03-21T11:20:48.660569+0000","last_became_peered":"2024-03-21T11:20:48.660569+0000","last_unstale":"2024-03-21T11:22:51.755223+0000","last_undegraded":"2024-03-21T11:22:51.755223+0000","last_fullsized":"2024-03-21T11:22:51.755223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:14.875700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847881+0000","last_change":"2024-03-21T11:20:53.918016+0000","last_active":"2024-03-21T11:22:50.847881+0000","last_peered":"2024-03-21T11:22:50.847881+0000","last_clean":"2024-03-21T11:22:50.847881+0000","last_became_active":"2024-03-21T11:20:53.916891+0000","last_became_peered":"2024-03-21T11:20:53.916891+0000","last_unstale":"2024-03-21T11:22:50.847881+0000","last_undegraded":"2024-03-21T11:22:50.847881+0000","last_fullsized":"2024-03-21T11:22:50.847881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:40:25.206778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'15","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:37.609177+0000","last_change":"2024-03-21T11:20:48.167560+0000","last_active":"2024-03-21T11:23:37.609177+0000","last_peered":"2024-03-21T11:23:37.609177+0000","last_clean":"2024-03-21T11:23:37.609177+0000","last_became_active":"2024-03-21T11:20:48.164986+0000","last_became_peered":"2024-03-21T11:20:48.164986+0000","last_unstale":"2024-03-21T11:23:37.609177+0000","last_undegraded":"2024-03-21T11:23:37.609177+0000","last_fullsized":"2024-03-21T11:23:37.609177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:59.930626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.849952+0000","last_change":"2024-03-21T11:20:48.675250+0000","last_active":"2024-03-21T11:22:50.849952+0000","last_peered":"2024-03-21T11:22:50.849952+0000","last_clean":"2024-03-21T11:22:50.849952+0000","last_became_active":"2024-03-21T11:20:48.674638+0000","last_became_peered":"2024-03-21T11:20:48.674638+0000","last_unstale":"2024-03-21T11:22:50.849952+0000","last_undegraded":"2024-03-21T11:22:50.849952+0000","last_fullsized":"2024-03-21T11:22:50.849952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:46:34.842363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852772+0000","last_change":"2024-03-21T11:20:53.733809+0000","last_active":"2024-03-21T11:22:50.852772+0000","last_peered":"2024-03-21T11:22:50.852772+0000","last_clean":"2024-03-21T11:22:50.852772+0000","last_became_active":"2024-03-21T11:20:53.733393+0000","last_became_peered":"2024-03-21T11:20:53.733393+0000","last_unstale":"2024-03-21T11:22:50.852772+0000","last_undegraded":"2024-03-21T11:22:50.852772+0000","last_fullsized":"2024-03-21T11:22:50.852772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:02:46.270716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847217+0000","last_change":"2024-03-21T11:20:48.139475+0000","last_active":"2024-03-21T11:22:50.847217+0000","last_peered":"2024-03-21T11:22:50.847217+0000","last_clean":"2024-03-21T11:22:50.847217+0000","last_became_active":"2024-03-21T11:20:48.139249+0000","last_became_peered":"2024-03-21T11:20:48.139249+0000","last_unstale":"2024-03-21T11:22:50.847217+0000","last_undegraded":"2024-03-21T11:22:50.847217+0000","last_fullsized":"2024-03-21T11:22:50.847217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:45:18.811829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855394+0000","last_change":"2024-03-21T11:20:48.673975+0000","last_active":"2024-03-21T11:22:50.855394+0000","last_peered":"2024-03-21T11:22:50.855394+0000","last_clean":"2024-03-21T11:22:50.855394+0000","last_became_active":"2024-03-21T11:20:48.673587+0000","last_became_peered":"2024-03-21T11:20:48.673587+0000","last_unstale":"2024-03-21T11:22:50.855394+0000","last_undegraded":"2024-03-21T11:22:50.855394+0000","last_fullsized":"2024-03-21T11:22:50.855394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:08:52.354780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847109+0000","last_change":"2024-03-21T11:20:53.917493+0000","last_active":"2024-03-21T11:22:50.847109+0000","last_peered":"2024-03-21T11:22:50.847109+0000","last_clean":"2024-03-21T11:22:50.847109+0000","last_became_active":"2024-03-21T11:20:53.917007+0000","last_became_peered":"2024-03-21T11:20:53.917007+0000","last_unstale":"2024-03-21T11:22:50.847109+0000","last_undegraded":"2024-03-21T11:22:50.847109+0000","last_fullsized":"2024-03-21T11:22:50.847109+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:39:20.280899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129330+0000","last_change":"2024-03-21T11:20:48.140914+0000","last_active":"2024-03-21T11:22:51.129330+0000","last_peered":"2024-03-21T11:22:51.129330+0000","last_clean":"2024-03-21T11:22:51.129330+0000","last_became_active":"2024-03-21T11:20:48.140558+0000","last_became_peered":"2024-03-21T11:20:48.140558+0000","last_unstale":"2024-03-21T11:22:51.129330+0000","last_undegraded":"2024-03-21T11:22:51.129330+0000","last_fullsized":"2024-03-21T11:22:51.129330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:59:52.065431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846373+0000","last_change":"2024-03-21T11:20:48.651190+0000","last_active":"2024-03-21T11:22:50.846373+0000","last_peered":"2024-03-21T11:22:50.846373+0000","last_clean":"2024-03-21T11:22:50.846373+0000","last_became_active":"2024-03-21T11:20:48.650818+0000","last_became_peered":"2024-03-21T11:20:48.650818+0000","last_unstale":"2024-03-21T11:22:50.846373+0000","last_undegraded":"2024-03-21T11:22:50.846373+0000","last_fullsized":"2024-03-21T11:22:50.846373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:47:17.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852214+0000","last_change":"2024-03-21T11:20:53.756844+0000","last_active":"2024-03-21T11:22:50.852214+0000","last_peered":"2024-03-21T11:22:50.852214+0000","last_clean":"2024-03-21T11:22:50.852214+0000","last_became_active":"2024-03-21T11:20:53.756380+0000","last_became_peered":"2024-03-21T11:20:53.756380+0000","last_unstale":"2024-03-21T11:22:50.852214+0000","last_undegraded":"2024-03-21T11:22:50.852214+0000","last_fullsized":"2024-03-21T11:22:50.852214+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:51:55.327798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848472+0000","last_change":"2024-03-21T11:20:48.160749+0000","last_active":"2024-03-21T11:22:50.848472+0000","last_peered":"2024-03-21T11:22:50.848472+0000","last_clean":"2024-03-21T11:22:50.848472+0000","last_became_active":"2024-03-21T11:20:48.158507+0000","last_became_peered":"2024-03-21T11:20:48.158507+0000","last_unstale":"2024-03-21T11:22:50.848472+0000","last_undegraded":"2024-03-21T11:22:50.848472+0000","last_fullsized":"2024-03-21T11:22:50.848472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:33:55.375529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560069+0000","last_change":"2024-03-21T11:20:48.667153+0000","last_active":"2024-03-21T11:22:51.560069+0000","last_peered":"2024-03-21T11:22:51.560069+0000","last_clean":"2024-03-21T11:22:51.560069+0000","last_became_active":"2024-03-21T11:20:48.666823+0000","last_became_peered":"2024-03-21T11:20:48.666823+0000","last_unstale":"2024-03-21T11:22:51.560069+0000","last_undegraded":"2024-03-21T11:22:51.560069+0000","last_fullsized":"2024-03-21T11:22:51.560069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:43:54.183372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846842+0000","last_change":"2024-03-21T11:20:53.739188+0000","last_active":"2024-03-21T11:22:50.846842+0000","last_peered":"2024-03-21T11:22:50.846842+0000","last_clean":"2024-03-21T11:22:50.846842+0000","last_became_active":"2024-03-21T11:20:53.738770+0000","last_became_peered":"2024-03-21T11:20:53.738770+0000","last_unstale":"2024-03-21T11:22:50.846842+0000","last_undegraded":"2024-03-21T11:22:50.846842+0000","last_fullsized":"2024-03-21T11:22:50.846842+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:53:48.897963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560058+0000","last_change":"2024-03-21T11:21:50.668860+0000","last_active":"2024-03-21T11:22:51.560058+0000","last_peered":"2024-03-21T11:22:51.560058+0000","last_clean":"2024-03-21T11:22:51.560058+0000","last_became_active":"2024-03-21T11:21:50.655255+0000","last_became_peered":"2024-03-21T11:21:50.655255+0000","last_unstale":"2024-03-21T11:22:51.560058+0000","last_undegraded":"2024-03-21T11:22:51.560058+0000","last_fullsized":"2024-03-21T11:22:51.560058+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:27.634508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.850136+0000","last_change":"2024-03-21T11:20:48.677303+0000","last_active":"2024-03-21T11:22:50.850136+0000","last_peered":"2024-03-21T11:22:50.850136+0000","last_clean":"2024-03-21T11:22:50.850136+0000","last_became_active":"2024-03-21T11:20:48.676903+0000","last_became_peered":"2024-03-21T11:20:48.676903+0000","last_unstale":"2024-03-21T11:22:50.850136+0000","last_undegraded":"2024-03-21T11:22:50.850136+0000","last_fullsized":"2024-03-21T11:22:50.850136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:06:59.922274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.847033+0000","last_change":"2024-03-21T11:20:53.916691+0000","last_active":"2024-03-21T11:22:50.847033+0000","last_peered":"2024-03-21T11:22:50.847033+0000","last_clean":"2024-03-21T11:22:50.847033+0000","last_became_active":"2024-03-21T11:20:53.916275+0000","last_became_peered":"2024-03-21T11:20:53.916275+0000","last_unstale":"2024-03-21T11:22:50.847033+0000","last_undegraded":"2024-03-21T11:22:50.847033+0000","last_fullsized":"2024-03-21T11:22:50.847033+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:35.848589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.846526+0000","last_change":"2024-03-21T11:21:50.668467+0000","last_active":"2024-03-21T11:22:50.846526+0000","last_peered":"2024-03-21T11:22:50.846526+0000","last_clean":"2024-03-21T11:22:50.846526+0000","last_became_active":"2024-03-21T11:21:50.649598+0000","last_became_peered":"2024-03-21T11:21:50.649598+0000","last_unstale":"2024-03-21T11:22:50.846526+0000","last_undegraded":"2024-03-21T11:22:50.846526+0000","last_fullsized":"2024-03-21T11:22:50.846526+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:01:48.186997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806108+0000","last_change":"2024-03-21T11:20:48.661486+0000","last_active":"2024-03-21T11:22:51.806108+0000","last_peered":"2024-03-21T11:22:51.806108+0000","last_clean":"2024-03-21T11:22:51.806108+0000","last_became_active":"2024-03-21T11:20:48.661109+0000","last_became_peered":"2024-03-21T11:20:48.661109+0000","last_unstale":"2024-03-21T11:22:51.806108+0000","last_undegraded":"2024-03-21T11:22:51.806108+0000","last_fullsized":"2024-03-21T11:22:51.806108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:15:27.093616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129398+0000","last_change":"2024-03-21T11:20:53.747223+0000","last_active":"2024-03-21T11:22:51.129398+0000","last_peered":"2024-03-21T11:22:51.129398+0000","last_clean":"2024-03-21T11:22:51.129398+0000","last_became_active":"2024-03-21T11:20:53.746805+0000","last_became_peered":"2024-03-21T11:20:53.746805+0000","last_unstale":"2024-03-21T11:22:51.129398+0000","last_undegraded":"2024-03-21T11:22:51.129398+0000","last_fullsized":"2024-03-21T11:22:51.129398+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:29.186745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"93'7","reported_seq":189,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:23:42.758848+0000","last_change":"2024-03-21T11:20:48.151172+0000","last_active":"2024-03-21T11:23:42.758848+0000","last_peered":"2024-03-21T11:23:42.758848+0000","last_clean":"2024-03-21T11:23:42.758848+0000","last_became_active":"2024-03-21T11:20:48.150832+0000","last_became_peered":"2024-03-21T11:20:48.150832+0000","last_unstale":"2024-03-21T11:23:42.758848+0000","last_undegraded":"2024-03-21T11:23:42.758848+0000","last_fullsized":"2024-03-21T11:23:42.758848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:34:46.367713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":149,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806211+0000","last_change":"2024-03-21T11:20:48.652275+0000","last_active":"2024-03-21T11:22:51.806211+0000","last_peered":"2024-03-21T11:22:51.806211+0000","last_clean":"2024-03-21T11:22:51.806211+0000","last_became_active":"2024-03-21T11:20:48.651965+0000","last_became_peered":"2024-03-21T11:20:48.651965+0000","last_unstale":"2024-03-21T11:22:51.806211+0000","last_undegraded":"2024-03-21T11:22:51.806211+0000","last_fullsized":"2024-03-21T11:22:51.806211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:56:18.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.854441+0000","last_change":"2024-03-21T11:20:53.748050+0000","last_active":"2024-03-21T11:22:50.854441+0000","last_peered":"2024-03-21T11:22:50.854441+0000","last_clean":"2024-03-21T11:22:50.854441+0000","last_became_active":"2024-03-21T11:20:53.747636+0000","last_became_peered":"2024-03-21T11:20:53.747636+0000","last_unstale":"2024-03-21T11:22:50.854441+0000","last_undegraded":"2024-03-21T11:22:50.854441+0000","last_fullsized":"2024-03-21T11:22:50.854441+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:09:15.202183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921814+0000","last_change":"2024-03-21T11:20:48.156928+0000","last_active":"2024-03-21T11:22:50.921814+0000","last_peered":"2024-03-21T11:22:50.921814+0000","last_clean":"2024-03-21T11:22:50.921814+0000","last_became_active":"2024-03-21T11:20:48.156625+0000","last_became_peered":"2024-03-21T11:20:48.156625+0000","last_unstale":"2024-03-21T11:22:50.921814+0000","last_undegraded":"2024-03-21T11:22:50.921814+0000","last_fullsized":"2024-03-21T11:22:50.921814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:19:42.035565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.805952+0000","last_change":"2024-03-21T11:20:48.663655+0000","last_active":"2024-03-21T11:22:51.805952+0000","last_peered":"2024-03-21T11:22:51.805952+0000","last_clean":"2024-03-21T11:22:51.805952+0000","last_became_active":"2024-03-21T11:20:48.663336+0000","last_became_peered":"2024-03-21T11:20:48.663336+0000","last_unstale":"2024-03-21T11:22:51.805952+0000","last_undegraded":"2024-03-21T11:22:51.805952+0000","last_fullsized":"2024-03-21T11:22:51.805952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:29:05.119283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.920960+0000","last_change":"2024-03-21T11:20:53.732584+0000","last_active":"2024-03-21T11:22:50.920960+0000","last_peered":"2024-03-21T11:22:50.920960+0000","last_clean":"2024-03-21T11:22:50.920960+0000","last_became_active":"2024-03-21T11:20:53.731975+0000","last_became_peered":"2024-03-21T11:20:53.731975+0000","last_unstale":"2024-03-21T11:22:50.920960+0000","last_undegraded":"2024-03-21T11:22:50.920960+0000","last_fullsized":"2024-03-21T11:22:50.920960+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:52.695997+0000","last_clean_scrub_stamp":"2024-03-21T11:20:52.695997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:51:06.336241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755235+0000","last_change":"2024-03-21T11:20:48.165594+0000","last_active":"2024-03-21T11:22:51.755235+0000","last_peered":"2024-03-21T11:22:51.755235+0000","last_clean":"2024-03-21T11:22:51.755235+0000","last_became_active":"2024-03-21T11:20:48.164394+0000","last_became_peered":"2024-03-21T11:20:48.164394+0000","last_unstale":"2024-03-21T11:22:51.755235+0000","last_undegraded":"2024-03-21T11:22:51.755235+0000","last_fullsized":"2024-03-21T11:22:51.755235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:47.066435+0000","last_clean_scrub_stamp":"2024-03-21T11:20:47.066435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:41:13.317767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.806214+0000","last_change":"2024-03-21T11:20:48.649359+0000","last_active":"2024-03-21T11:22:51.806214+0000","last_peered":"2024-03-21T11:22:51.806214+0000","last_clean":"2024-03-21T11:22:51.806214+0000","last_became_active":"2024-03-21T11:20:48.649031+0000","last_became_peered":"2024-03-21T11:20:48.649031+0000","last_unstale":"2024-03-21T11:22:51.806214+0000","last_undegraded":"2024-03-21T11:22:51.806214+0000","last_fullsized":"2024-03-21T11:22:51.806214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:17.055732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560206+0000","last_change":"2024-03-21T11:20:48.654999+0000","last_active":"2024-03-21T11:22:51.560206+0000","last_peered":"2024-03-21T11:22:51.560206+0000","last_clean":"2024-03-21T11:22:51.560206+0000","last_became_active":"2024-03-21T11:20:48.654638+0000","last_became_peered":"2024-03-21T11:20:48.654638+0000","last_unstale":"2024-03-21T11:22:51.560206+0000","last_undegraded":"2024-03-21T11:22:51.560206+0000","last_fullsized":"2024-03-21T11:22:51.560206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:49:48.711420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.852998+0000","last_change":"2024-03-21T11:20:48.651864+0000","last_active":"2024-03-21T11:22:50.852998+0000","last_peered":"2024-03-21T11:22:50.852998+0000","last_clean":"2024-03-21T11:22:50.852998+0000","last_became_active":"2024-03-21T11:20:48.651565+0000","last_became_peered":"2024-03-21T11:20:48.651565+0000","last_unstale":"2024-03-21T11:22:50.852998+0000","last_undegraded":"2024-03-21T11:22:50.852998+0000","last_fullsized":"2024-03-21T11:22:50.852998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:58.659889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853152+0000","last_change":"2024-03-21T11:20:48.667038+0000","last_active":"2024-03-21T11:22:50.853152+0000","last_peered":"2024-03-21T11:22:50.853152+0000","last_clean":"2024-03-21T11:22:50.853152+0000","last_became_active":"2024-03-21T11:20:48.666570+0000","last_became_peered":"2024-03-21T11:20:48.666570+0000","last_unstale":"2024-03-21T11:22:50.853152+0000","last_undegraded":"2024-03-21T11:22:50.853152+0000","last_fullsized":"2024-03-21T11:22:50.853152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:36:39.043697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.855033+0000","last_change":"2024-03-21T11:21:50.658024+0000","last_active":"2024-03-21T11:22:50.855033+0000","last_peered":"2024-03-21T11:22:50.855033+0000","last_clean":"2024-03-21T11:22:50.855033+0000","last_became_active":"2024-03-21T11:21:50.657390+0000","last_became_peered":"2024-03-21T11:21:50.657390+0000","last_unstale":"2024-03-21T11:22:50.855033+0000","last_undegraded":"2024-03-21T11:22:50.855033+0000","last_fullsized":"2024-03-21T11:22:50.855033+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:06:29.578366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755343+0000","last_change":"2024-03-21T11:20:48.670139+0000","last_active":"2024-03-21T11:22:51.755343+0000","last_peered":"2024-03-21T11:22:51.755343+0000","last_clean":"2024-03-21T11:22:51.755343+0000","last_became_active":"2024-03-21T11:20:48.669410+0000","last_became_peered":"2024-03-21T11:20:48.669410+0000","last_unstale":"2024-03-21T11:22:51.755343+0000","last_undegraded":"2024-03-21T11:22:51.755343+0000","last_fullsized":"2024-03-21T11:22:51.755343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:35:18.799840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560440+0000","last_change":"2024-03-21T11:20:48.655356+0000","last_active":"2024-03-21T11:22:51.560440+0000","last_peered":"2024-03-21T11:22:51.560440+0000","last_clean":"2024-03-21T11:22:51.560440+0000","last_became_active":"2024-03-21T11:20:48.655028+0000","last_became_peered":"2024-03-21T11:20:48.655028+0000","last_unstale":"2024-03-21T11:22:51.560440+0000","last_undegraded":"2024-03-21T11:22:51.560440+0000","last_fullsized":"2024-03-21T11:22:51.560440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:46:39.948982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.853238+0000","last_change":"2024-03-21T11:20:48.648840+0000","last_active":"2024-03-21T11:22:50.853238+0000","last_peered":"2024-03-21T11:22:50.853238+0000","last_clean":"2024-03-21T11:22:50.853238+0000","last_became_active":"2024-03-21T11:20:48.648502+0000","last_became_peered":"2024-03-21T11:20:48.648502+0000","last_unstale":"2024-03-21T11:22:50.853238+0000","last_undegraded":"2024-03-21T11:22:50.853238+0000","last_fullsized":"2024-03-21T11:22:50.853238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:33:14.824880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.850295+0000","last_change":"2024-03-21T11:20:48.672015+0000","last_active":"2024-03-21T11:22:50.850295+0000","last_peered":"2024-03-21T11:22:50.850295+0000","last_clean":"2024-03-21T11:22:50.850295+0000","last_became_active":"2024-03-21T11:20:48.671663+0000","last_became_peered":"2024-03-21T11:20:48.671663+0000","last_unstale":"2024-03-21T11:22:50.850295+0000","last_undegraded":"2024-03-21T11:22:50.850295+0000","last_fullsized":"2024-03-21T11:22:50.850295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:21:25.328698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.921220+0000","last_change":"2024-03-21T11:20:48.664114+0000","last_active":"2024-03-21T11:22:50.921220+0000","last_peered":"2024-03-21T11:22:50.921220+0000","last_clean":"2024-03-21T11:22:50.921220+0000","last_became_active":"2024-03-21T11:20:48.663819+0000","last_became_peered":"2024-03-21T11:20:48.663819+0000","last_unstale":"2024-03-21T11:22:50.921220+0000","last_undegraded":"2024-03-21T11:22:50.921220+0000","last_fullsized":"2024-03-21T11:22:50.921220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:08:08.201383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:50.848874+0000","last_change":"2024-03-21T11:20:48.671336+0000","last_active":"2024-03-21T11:22:50.848874+0000","last_peered":"2024-03-21T11:22:50.848874+0000","last_clean":"2024-03-21T11:22:50.848874+0000","last_became_active":"2024-03-21T11:20:48.670918+0000","last_became_peered":"2024-03-21T11:20:48.670918+0000","last_unstale":"2024-03-21T11:22:50.848874+0000","last_undegraded":"2024-03-21T11:22:50.848874+0000","last_fullsized":"2024-03-21T11:22:50.848874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:01:44.764584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.129593+0000","last_change":"2024-03-21T11:20:48.659339+0000","last_active":"2024-03-21T11:22:51.129593+0000","last_peered":"2024-03-21T11:22:51.129593+0000","last_clean":"2024-03-21T11:22:51.129593+0000","last_became_active":"2024-03-21T11:20:48.658989+0000","last_became_peered":"2024-03-21T11:20:48.658989+0000","last_unstale":"2024-03-21T11:22:51.129593+0000","last_undegraded":"2024-03-21T11:22:51.129593+0000","last_fullsized":"2024-03-21T11:22:51.129593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:44:09.599836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.755291+0000","last_change":"2024-03-21T11:20:48.668679+0000","last_active":"2024-03-21T11:22:51.755291+0000","last_peered":"2024-03-21T11:22:51.755291+0000","last_clean":"2024-03-21T11:22:51.755291+0000","last_became_active":"2024-03-21T11:20:48.667956+0000","last_became_peered":"2024-03-21T11:20:48.667956+0000","last_unstale":"2024-03-21T11:22:51.755291+0000","last_undegraded":"2024-03-21T11:22:51.755291+0000","last_fullsized":"2024-03-21T11:22:51.755291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:04.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T11:22:51.560270+0000","last_change":"2024-03-21T11:21:50.650600+0000","last_active":"2024-03-21T11:22:51.560270+0000","last_peered":"2024-03-21T11:22:51.560270+0000","last_clean":"2024-03-21T11:22:51.560270+0000","last_became_active":"2024-03-21T11:21:50.650221+0000","last_became_peered":"2024-03-21T11:21:50.650221+0000","last_unstale":"2024-03-21T11:22:51.560270+0000","last_undegraded":"2024-03-21T11:22:51.560270+0000","last_fullsized":"2024-03-21T11:22:51.560270+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T11:20:48.109249+0000","last_clean_scrub_stamp":"2024-03-21T11:20:48.109249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:29:52.245281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247498,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79240,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1481,"num_read_kb":824,"num_write":267,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":240240,"data_compressed":5487,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351026,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":531977,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612663,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32356,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712796,"statfs":{"total":95995035648,"available":95961903104,"internally_reserved":0,"allocated":1208320,"data_stored":1193414,"data_compressed":64722,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776188,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":535471,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037827,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":531753,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299464,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":1129620,"data_compressed":62893,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462990,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":527270,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":598825,"data_compressed":28661,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":528867,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1142784,"data_stored":602544,"data_compressed":28661,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149790,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":526587,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888152,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":531485,"data_compressed":26832,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724627,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":1120183,"data_compressed":62893,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34418,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36061,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72668,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":76387,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":72793,"data_compressed":1829,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T11:23:46.159 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:23:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.423+0000 7f9749e00640 1 -- 172.21.15.47:0/2778294912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9744076060 msgr2=0x7f9744076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.423+0000 7f9749e00640 1 --2- 172.21.15.47:0/2778294912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9744076060 0x7f9744076480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9734009900 tx=0x7f973402bc40 comp rx=0 tx=0).stop 2024-03-21T11:23:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.424+0000 7f9749e00640 1 -- 172.21.15.47:0/2778294912 shutdown_connections 2024-03-21T11:23:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.424+0000 7f9749e00640 1 --2- 172.21.15.47:0/2778294912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9744077c40 0x7f97440784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.424+0000 7f9749e00640 1 --2- 172.21.15.47:0/2778294912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97440772a0 0x7f9744077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.424+0000 7f9749e00640 1 --2- 172.21.15.47:0/2778294912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9744076060 0x7f9744076480 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.424+0000 7f9749e00640 1 -- 172.21.15.47:0/2778294912 >> 172.21.15.47:0/2778294912 conn(0x7f97440fee50 msgr2=0x7f9744101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.424+0000 7f9749e00640 1 -- 172.21.15.47:0/2778294912 shutdown_connections 2024-03-21T11:23:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.424+0000 7f9749e00640 1 -- 172.21.15.47:0/2778294912 wait complete. 2024-03-21T11:23:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.425+0000 7f9749e00640 1 Processor -- start 2024-03-21T11:23:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.425+0000 7f9749e00640 1 -- start start 2024-03-21T11:23:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9749e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9744076060 0x7f974419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9749e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97440772a0 0x7f974419e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9749e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9744077c40 0x7f97441a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9749e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9744113990 con 0x7f9744077c40 2024-03-21T11:23:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9749e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9744113810 con 0x7f97440772a0 2024-03-21T11:23:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9749e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9744113b10 con 0x7f9744076060 2024-03-21T11:23:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9742000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97440772a0 0x7f974419e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9742a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9744076060 0x7f974419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9742000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97440772a0 0x7f974419e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38754/0 (socket says 172.21.15.47:38754) 2024-03-21T11:23:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9742000640 1 -- 172.21.15.47:0/3835239096 learned_addr learned my addr 172.21.15.47:0/3835239096 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9743400640 1 --2- 172.21.15.47:0/3835239096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9744077c40 0x7f97441a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9742000640 1 -- 172.21.15.47:0/3835239096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9744076060 msgr2=0x7f974419dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9742000640 1 --2- 172.21.15.47:0/3835239096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9744076060 0x7f974419dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9742000640 1 -- 172.21.15.47:0/3835239096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9744077c40 msgr2=0x7f97441a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9742000640 1 --2- 172.21.15.47:0/3835239096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9744077c40 0x7f97441a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9742000640 1 -- 172.21.15.47:0/3835239096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97441a5bd0 con 0x7f97440772a0 2024-03-21T11:23:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.426+0000 7f9743400640 1 --2- 172.21.15.47:0/3835239096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9744077c40 0x7f97441a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.427+0000 7f9742000640 1 --2- 172.21.15.47:0/3835239096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97440772a0 0x7f974419e4f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f972c00d950 tx=0x7f972c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.427+0000 7f9748a00640 1 -- 172.21.15.47:0/3835239096 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c017070 con 0x7f97440772a0 2024-03-21T11:23:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.427+0000 7f9748a00640 1 -- 172.21.15.47:0/3835239096 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f972c009c30 con 0x7f97440772a0 2024-03-21T11:23:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.427+0000 7f9749e00640 1 -- 172.21.15.47:0/3835239096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97441a5ec0 con 0x7f97440772a0 2024-03-21T11:23:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.427+0000 7f9748a00640 1 -- 172.21.15.47:0/3835239096 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c00f6b0 con 0x7f97440772a0 2024-03-21T11:23:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.427+0000 7f9749e00640 1 -- 172.21.15.47:0/3835239096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97441a63d0 con 0x7f97440772a0 2024-03-21T11:23:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.428+0000 7f9749e00640 1 -- 172.21.15.47:0/3835239096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9744110230 con 0x7f97440772a0 2024-03-21T11:23:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.430+0000 7f9748a00640 1 -- 172.21.15.47:0/3835239096 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f972c0109d0 con 0x7f97440772a0 2024-03-21T11:23:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.431+0000 7f9748a00640 1 --2- 172.21.15.47:0/3835239096 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97280a3030 0x7f97280a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.431+0000 7f9748a00640 1 -- 172.21.15.47:0/3835239096 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f972c0cf530 con 0x7f97440772a0 2024-03-21T11:23:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.432+0000 7f9742a00640 1 --2- 172.21.15.47:0/3835239096 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97280a3030 0x7f97280a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.432+0000 7f9742a00640 1 --2- 172.21.15.47:0/3835239096 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97280a3030 0x7f97280a54f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f973402c250 tx=0x7f973402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.437+0000 7f9748a00640 1 -- 172.21.15.47:0/3835239096 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f972c009de0 con 0x7f97440772a0 2024-03-21T11:23:46.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.642+0000 7f9749e00640 1 -- 172.21.15.47:0/3835239096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97440622a0 con 0x7f97440772a0 2024-03-21T11:23:46.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.645+0000 7f9748a00640 1 -- 172.21.15.47:0/3835239096 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24747 (secure 0 0 0) 0x7f972c09be10 con 0x7f97440772a0 2024-03-21T11:23:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:46.647 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:22:50.841892+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:23:46.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.648+0000 7f9749e00640 1 -- 172.21.15.47:0/3835239096 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97280a3030 msgr2=0x7f97280a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:46.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.648+0000 7f9749e00640 1 --2- 172.21.15.47:0/3835239096 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97280a3030 0x7f97280a54f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f973402c250 tx=0x7f973402cf90 comp rx=0 tx=0).stop 2024-03-21T11:23:46.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.648+0000 7f9749e00640 1 -- 172.21.15.47:0/3835239096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97440772a0 msgr2=0x7f974419e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:46.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.648+0000 7f9749e00640 1 --2- 172.21.15.47:0/3835239096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97440772a0 0x7f974419e4f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f972c00d950 tx=0x7f972c00de10 comp rx=0 tx=0).stop 2024-03-21T11:23:46.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.648+0000 7f9749e00640 1 -- 172.21.15.47:0/3835239096 shutdown_connections 2024-03-21T11:23:46.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.648+0000 7f9749e00640 1 --2- 172.21.15.47:0/3835239096 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97280a3030 0x7f97280a54f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.648+0000 7f9749e00640 1 --2- 172.21.15.47:0/3835239096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9744077c40 0x7f97441a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.648+0000 7f9749e00640 1 --2- 172.21.15.47:0/3835239096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97440772a0 0x7f974419e4f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.648+0000 7f9749e00640 1 --2- 172.21.15.47:0/3835239096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9744076060 0x7f974419dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.648+0000 7f9749e00640 1 -- 172.21.15.47:0/3835239096 >> 172.21.15.47:0/3835239096 conn(0x7f97440fee50 msgr2=0x7f9744100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:46.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.649+0000 7f9749e00640 1 -- 172.21.15.47:0/3835239096 shutdown_connections 2024-03-21T11:23:46.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.649+0000 7f9749e00640 1 -- 172.21.15.47:0/3835239096 wait complete. 2024-03-21T11:23:46.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T11:15:35.545245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T11:20:47.016174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T11:20:48.108506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T11:20:52.254154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T11:23:46.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T11:23:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.900+0000 7f7850a00640 1 -- 172.21.15.47:0/157937587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f784c076060 msgr2=0x7f784c076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.900+0000 7f7850a00640 1 --2- 172.21.15.47:0/157937587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f784c076060 0x7f784c076480 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f783c009900 tx=0x7f783c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.900+0000 7f7850a00640 1 -- 172.21.15.47:0/157937587 shutdown_connections 2024-03-21T11:23:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.900+0000 7f7850a00640 1 --2- 172.21.15.47:0/157937587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c077c40 0x7f784c0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.900+0000 7f7850a00640 1 --2- 172.21.15.47:0/157937587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f784c0772a0 0x7f784c077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.900+0000 7f7850a00640 1 --2- 172.21.15.47:0/157937587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f784c076060 0x7f784c076480 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.900+0000 7f7850a00640 1 -- 172.21.15.47:0/157937587 >> 172.21.15.47:0/157937587 conn(0x7f784c0fee50 msgr2=0x7f784c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.900+0000 7f7850a00640 1 -- 172.21.15.47:0/157937587 shutdown_connections 2024-03-21T11:23:46.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.900+0000 7f7850a00640 1 -- 172.21.15.47:0/157937587 wait complete. 2024-03-21T11:23:46.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.901+0000 7f7850a00640 1 Processor -- start 2024-03-21T11:23:46.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.901+0000 7f7850a00640 1 -- start start 2024-03-21T11:23:46.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.901+0000 7f7850a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f784c076060 0x7f784c19e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.901+0000 7f7850a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f784c0772a0 0x7f784c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f7850a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c077c40 0x7f784c1a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f7850a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c113930 con 0x7f784c0772a0 2024-03-21T11:23:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f7850a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c1137b0 con 0x7f784c077c40 2024-03-21T11:23:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f7850a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c113ab0 con 0x7f784c076060 2024-03-21T11:23:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f7849e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f784c076060 0x7f784c19e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f7849e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f784c076060 0x7f784c19e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52010/0 (socket says 172.21.15.47:52010) 2024-03-21T11:23:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f7849e00640 1 -- 172.21.15.47:0/1623042095 learned_addr learned my addr 172.21.15.47:0/1623042095 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f784a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c077c40 0x7f784c1a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f7841400640 1 --2- 172.21.15.47:0/1623042095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f784c0772a0 0x7f784c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f784a800640 1 -- 172.21.15.47:0/1623042095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f784c076060 msgr2=0x7f784c19e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f784a800640 1 --2- 172.21.15.47:0/1623042095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f784c076060 0x7f784c19e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f784a800640 1 -- 172.21.15.47:0/1623042095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f784c0772a0 msgr2=0x7f784c19e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f784a800640 1 --2- 172.21.15.47:0/1623042095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f784c0772a0 0x7f784c19e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f784a800640 1 -- 172.21.15.47:0/1623042095 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c1a5c80 con 0x7f784c077c40 2024-03-21T11:23:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.902+0000 7f7849e00640 1 --2- 172.21.15.47:0/1623042095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f784c076060 0x7f784c19e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.903+0000 7f784a800640 1 --2- 172.21.15.47:0/1623042095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c077c40 0x7f784c1a5610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f784400ec50 tx=0x7f784400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.903+0000 7f7843400640 1 -- 172.21.15.47:0/1623042095 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844003b70 con 0x7f784c077c40 2024-03-21T11:23:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.903+0000 7f7843400640 1 -- 172.21.15.47:0/1623042095 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7844003d10 con 0x7f784c077c40 2024-03-21T11:23:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.903+0000 7f7850a00640 1 -- 172.21.15.47:0/1623042095 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784c1a5f70 con 0x7f784c077c40 2024-03-21T11:23:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.903+0000 7f7843400640 1 -- 172.21.15.47:0/1623042095 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844010890 con 0x7f784c077c40 2024-03-21T11:23:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.903+0000 7f7850a00640 1 -- 172.21.15.47:0/1623042095 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784c1a63e0 con 0x7f784c077c40 2024-03-21T11:23:46.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.904+0000 7f7850a00640 1 -- 172.21.15.47:0/1623042095 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784c110250 con 0x7f784c077c40 2024-03-21T11:23:46.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.906+0000 7f7843400640 1 -- 172.21.15.47:0/1623042095 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f7844003fc0 con 0x7f784c077c40 2024-03-21T11:23:46.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.907+0000 7f7843400640 1 --2- 172.21.15.47:0/1623042095 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f78300a2fb0 0x7f78300a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:46.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.907+0000 7f7843400640 1 -- 172.21.15.47:0/1623042095 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f78440cff20 con 0x7f784c077c40 2024-03-21T11:23:46.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.907+0000 7f7849e00640 1 --2- 172.21.15.47:0/1623042095 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f78300a2fb0 0x7f78300a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:46.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.908+0000 7f7849e00640 1 --2- 172.21.15.47:0/1623042095 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f78300a2fb0 0x7f78300a5470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f783c000c00 tx=0x7f783c02c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:46.911+0000 7f7843400640 1 -- 172.21.15.47:0/1623042095 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f784409c800 con 0x7f784c077c40 2024-03-21T11:23:47.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:46 smithi047 ceph-mon[31351]: from='client.25532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:23:47.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:46 smithi047 ceph-mon[31351]: pgmap v413: 129 pgs: 129 active+clean; 654 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T11:23:47.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:46 smithi047 ceph-mon[31351]: from='client.25555 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:23:47.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:46 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3835239096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.115+0000 7f7850a00640 1 -- 172.21.15.47:0/1623042095 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f784c02d220 con 0x7f784c077c40 2024-03-21T11:23:47.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.115+0000 7f7843400640 1 -- 172.21.15.47:0/1623042095 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f784409c100 con 0x7f784c077c40 2024-03-21T11:23:47.117 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T11:23:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.119+0000 7f7850a00640 1 -- 172.21.15.47:0/1623042095 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f78300a2fb0 msgr2=0x7f78300a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.119+0000 7f7850a00640 1 --2- 172.21.15.47:0/1623042095 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f78300a2fb0 0x7f78300a5470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f783c000c00 tx=0x7f783c02c880 comp rx=0 tx=0).stop 2024-03-21T11:23:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.119+0000 7f7850a00640 1 -- 172.21.15.47:0/1623042095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c077c40 msgr2=0x7f784c1a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.119+0000 7f7850a00640 1 --2- 172.21.15.47:0/1623042095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c077c40 0x7f784c1a5610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f784400ec50 tx=0x7f784400c6a0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.119+0000 7f7850a00640 1 -- 172.21.15.47:0/1623042095 shutdown_connections 2024-03-21T11:23:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.119+0000 7f7850a00640 1 --2- 172.21.15.47:0/1623042095 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f78300a2fb0 0x7f78300a5470 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.119+0000 7f7850a00640 1 --2- 172.21.15.47:0/1623042095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f784c077c40 0x7f784c1a5610 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.119+0000 7f7850a00640 1 --2- 172.21.15.47:0/1623042095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f784c0772a0 0x7f784c19e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.119+0000 7f7850a00640 1 --2- 172.21.15.47:0/1623042095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f784c076060 0x7f784c19e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.119+0000 7f7850a00640 1 -- 172.21.15.47:0/1623042095 >> 172.21.15.47:0/1623042095 conn(0x7f784c0fee50 msgr2=0x7f784c100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.119+0000 7f7850a00640 1 -- 172.21.15.47:0/1623042095 shutdown_connections 2024-03-21T11:23:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.119+0000 7f7850a00640 1 -- 172.21.15.47:0/1623042095 wait complete. 2024-03-21T11:23:47.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:46 smithi100 ceph-mon[36245]: from='client.25532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:23:47.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:46 smithi100 ceph-mon[36245]: pgmap v413: 129 pgs: 129 active+clean; 654 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T11:23:47.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:46 smithi100 ceph-mon[36245]: from='client.25555 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:23:47.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:46 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3835239096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:47.150 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T11:23:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:46 smithi102 ceph-mon[36748]: from='client.25532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:23:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:46 smithi102 ceph-mon[36748]: pgmap v413: 129 pgs: 129 active+clean; 654 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T11:23:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:46 smithi102 ceph-mon[36748]: from='client.25555 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:23:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:46 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3835239096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.375+0000 7fe8f0600640 1 -- 172.21.15.47:0/98787890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e8105470 msgr2=0x7fe8e810bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.375+0000 7fe8f0600640 1 --2- 172.21.15.47:0/98787890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e8105470 0x7fe8e810bd00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009900 tx=0x7fe8e002bc40 comp rx=0 tx=0).stop 2024-03-21T11:23:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.376+0000 7fe8f0600640 1 -- 172.21.15.47:0/98787890 shutdown_connections 2024-03-21T11:23:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.376+0000 7fe8f0600640 1 --2- 172.21.15.47:0/98787890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e8105470 0x7fe8e810bd00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.376+0000 7fe8f0600640 1 --2- 172.21.15.47:0/98787890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8e8104ab0 0x7fe8e8104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.376+0000 7fe8f0600640 1 --2- 172.21.15.47:0/98787890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8e81038c0 0x7fe8e8103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.376+0000 7fe8f0600640 1 -- 172.21.15.47:0/98787890 >> 172.21.15.47:0/98787890 conn(0x7fe8e80fee50 msgr2=0x7fe8e8101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.376+0000 7fe8f0600640 1 -- 172.21.15.47:0/98787890 shutdown_connections 2024-03-21T11:23:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.376+0000 7fe8f0600640 1 -- 172.21.15.47:0/98787890 wait complete. 2024-03-21T11:23:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.377+0000 7fe8f0600640 1 Processor -- start 2024-03-21T11:23:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.377+0000 7fe8f0600640 1 -- start start 2024-03-21T11:23:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.377+0000 7fe8f0600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e81038c0 0x7fe8e8199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.377+0000 7fe8f0600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8e8104ab0 0x7fe8e819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.377+0000 7fe8f0600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8e8105470 0x7fe8e81a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.377+0000 7fe8f0600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e810f520 con 0x7fe8e8105470 2024-03-21T11:23:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.377+0000 7fe8f0600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e810f3a0 con 0x7fe8e8104ab0 2024-03-21T11:23:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.377+0000 7fe8f0600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e810f6a0 con 0x7fe8e81038c0 2024-03-21T11:23:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.377+0000 7fe8edc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e81038c0 0x7fe8e8199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.377+0000 7fe8edc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e81038c0 0x7fe8e8199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52040/0 (socket says 172.21.15.47:52040) 2024-03-21T11:23:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.377+0000 7fe8edc00640 1 -- 172.21.15.47:0/3595727062 learned_addr learned my addr 172.21.15.47:0/3595727062 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.377+0000 7fe8ed200640 1 --2- 172.21.15.47:0/3595727062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8e8104ab0 0x7fe8e819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.377+0000 7fe8ee600640 1 --2- 172.21.15.47:0/3595727062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8e8105470 0x7fe8e81a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.378+0000 7fe8edc00640 1 -- 172.21.15.47:0/3595727062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8e8104ab0 msgr2=0x7fe8e819a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.378+0000 7fe8edc00640 1 --2- 172.21.15.47:0/3595727062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8e8104ab0 0x7fe8e819a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.378+0000 7fe8edc00640 1 -- 172.21.15.47:0/3595727062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8e8105470 msgr2=0x7fe8e81a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.378+0000 7fe8edc00640 1 --2- 172.21.15.47:0/3595727062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8e8105470 0x7fe8e81a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.378+0000 7fe8edc00640 1 -- 172.21.15.47:0/3595727062 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e81a1720 con 0x7fe8e81038c0 2024-03-21T11:23:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.378+0000 7fe8edc00640 1 --2- 172.21.15.47:0/3595727062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e81038c0 0x7fe8e8199b00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00ea80 tx=0x7fe8dc00ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.378+0000 7fe8e6000640 1 -- 172.21.15.47:0/3595727062 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00ce40 con 0x7fe8e81038c0 2024-03-21T11:23:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.378+0000 7fe8e6000640 1 -- 172.21.15.47:0/3595727062 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8dc011950 con 0x7fe8e81038c0 2024-03-21T11:23:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.378+0000 7fe8f0600640 1 -- 172.21.15.47:0/3595727062 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e81a1a10 con 0x7fe8e81038c0 2024-03-21T11:23:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.378+0000 7fe8e6000640 1 -- 172.21.15.47:0/3595727062 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc01ab80 con 0x7fe8e81038c0 2024-03-21T11:23:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.379+0000 7fe8f0600640 1 -- 172.21.15.47:0/3595727062 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e81a1f60 con 0x7fe8e81038c0 2024-03-21T11:23:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.380+0000 7fe8f0600640 1 -- 172.21.15.47:0/3595727062 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e80688e0 con 0x7fe8e81038c0 2024-03-21T11:23:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.382+0000 7fe8e6000640 1 -- 172.21.15.47:0/3595727062 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fe8dc011b00 con 0x7fe8e81038c0 2024-03-21T11:23:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.383+0000 7fe8e6000640 1 --2- 172.21.15.47:0/3595727062 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe8d00a2fb0 0x7fe8d00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.383+0000 7fe8e6000640 1 -- 172.21.15.47:0/3595727062 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe8dc0d0170 con 0x7fe8e81038c0 2024-03-21T11:23:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.383+0000 7fe8ed200640 1 --2- 172.21.15.47:0/3595727062 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe8d00a2fb0 0x7fe8d00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.384+0000 7fe8ed200640 1 --2- 172.21.15.47:0/3595727062 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe8d00a2fb0 0x7fe8d00a5470 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe8e819b010 tx=0x7fe8d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.390+0000 7fe8e6000640 1 -- 172.21.15.47:0/3595727062 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe8dc016030 con 0x7fe8e81038c0 2024-03-21T11:23:47.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.590+0000 7fe8f0600640 1 -- 172.21.15.47:0/3595727062 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8e802d0f0 con 0x7fe8e81038c0 2024-03-21T11:23:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.591+0000 7fe8e6000640 1 -- 172.21.15.47:0/3595727062 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe8dc09ca50 con 0x7fe8e81038c0 2024-03-21T11:23:47.592 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:23:47.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.593+0000 7fe8f0600640 1 -- 172.21.15.47:0/3595727062 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe8d00a2fb0 msgr2=0x7fe8d00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:47.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.593+0000 7fe8f0600640 1 --2- 172.21.15.47:0/3595727062 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe8d00a2fb0 0x7fe8d00a5470 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe8e819b010 tx=0x7fe8d4005f90 comp rx=0 tx=0).stop 2024-03-21T11:23:47.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.593+0000 7fe8f0600640 1 -- 172.21.15.47:0/3595727062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e81038c0 msgr2=0x7fe8e8199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:47.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.594+0000 7fe8f0600640 1 --2- 172.21.15.47:0/3595727062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e81038c0 0x7fe8e8199b00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00ea80 tx=0x7fe8dc00ef40 comp rx=0 tx=0).stop 2024-03-21T11:23:47.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.594+0000 7fe8f0600640 1 -- 172.21.15.47:0/3595727062 shutdown_connections 2024-03-21T11:23:47.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.594+0000 7fe8f0600640 1 --2- 172.21.15.47:0/3595727062 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe8d00a2fb0 0x7fe8d00a5470 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.594+0000 7fe8f0600640 1 --2- 172.21.15.47:0/3595727062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8e8105470 0x7fe8e81a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.594+0000 7fe8f0600640 1 --2- 172.21.15.47:0/3595727062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8e8104ab0 0x7fe8e819a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.594+0000 7fe8f0600640 1 --2- 172.21.15.47:0/3595727062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e81038c0 0x7fe8e8199b00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.594+0000 7fe8f0600640 1 -- 172.21.15.47:0/3595727062 >> 172.21.15.47:0/3595727062 conn(0x7fe8e80fee50 msgr2=0x7fe8e8107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.594+0000 7fe8f0600640 1 -- 172.21.15.47:0/3595727062 shutdown_connections 2024-03-21T11:23:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.594+0000 7fe8f0600640 1 -- 172.21.15.47:0/3595727062 wait complete. 2024-03-21T11:23:47.614 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T11:23:47.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.851+0000 7f3ff6a00640 1 -- 172.21.15.47:0/3729122008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff0102130 msgr2=0x7f3ff0104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:47.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.851+0000 7f3ff6a00640 1 --2- 172.21.15.47:0/3729122008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff0102130 0x7f3ff0104520 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009be0 tx=0x7f3fe002bef0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.851+0000 7f3ff6a00640 1 -- 172.21.15.47:0/3729122008 shutdown_connections 2024-03-21T11:23:47.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.851+0000 7f3ff6a00640 1 --2- 172.21.15.47:0/3729122008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff0107390 0x7f3ff01097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.851+0000 7f3ff6a00640 1 --2- 172.21.15.47:0/3729122008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ff0104a60 0x7f3ff0106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.851+0000 7f3ff6a00640 1 --2- 172.21.15.47:0/3729122008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff0102130 0x7f3ff0104520 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.851+0000 7f3ff6a00640 1 -- 172.21.15.47:0/3729122008 >> 172.21.15.47:0/3729122008 conn(0x7f3ff00fbb40 msgr2=0x7f3ff00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:47.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.851+0000 7f3ff6a00640 1 -- 172.21.15.47:0/3729122008 shutdown_connections 2024-03-21T11:23:47.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.851+0000 7f3ff6a00640 1 -- 172.21.15.47:0/3729122008 wait complete. 2024-03-21T11:23:47.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.852+0000 7f3ff6a00640 1 Processor -- start 2024-03-21T11:23:47.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.853+0000 7f3ff6a00640 1 -- start start 2024-03-21T11:23:47.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.853+0000 7f3ff6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff0102130 0x7f3ff0101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:47.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.853+0000 7f3ff6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff0104a60 0x7f3ff01000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.853+0000 7f3ff6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ff0107390 0x7f3ff0100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.853+0000 7f3ff6a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff010d380 con 0x7f3ff0102130 2024-03-21T11:23:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.853+0000 7f3ff6a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff010d200 con 0x7f3ff0104a60 2024-03-21T11:23:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.853+0000 7f3ff6a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff010d500 con 0x7f3ff0107390 2024-03-21T11:23:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.853+0000 7f3fef400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff0104a60 0x7f3ff01000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.853+0000 7f3fefe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff0102130 0x7f3ff0101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.853+0000 7f3fef400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff0104a60 0x7f3ff01000e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38806/0 (socket says 172.21.15.47:38806) 2024-03-21T11:23:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.853+0000 7f3fef400640 1 -- 172.21.15.47:0/627420453 learned_addr learned my addr 172.21.15.47:0/627420453 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.853+0000 7f3ff4a00640 1 --2- 172.21.15.47:0/627420453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ff0107390 0x7f3ff0100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.854+0000 7f3fef400640 1 -- 172.21.15.47:0/627420453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ff0107390 msgr2=0x7f3ff0100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.854+0000 7f3fef400640 1 --2- 172.21.15.47:0/627420453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ff0107390 0x7f3ff0100620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.854+0000 7f3fef400640 1 -- 172.21.15.47:0/627420453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff0102130 msgr2=0x7f3ff0101aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:47.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.854+0000 7f3fef400640 1 --2- 172.21.15.47:0/627420453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff0102130 0x7f3ff0101aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:47.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.854+0000 7f3fef400640 1 -- 172.21.15.47:0/627420453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff0100eb0 con 0x7f3ff0104a60 2024-03-21T11:23:47.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.854+0000 7f3fef400640 1 --2- 172.21.15.47:0/627420453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff0104a60 0x7f3ff01000e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800e9a0 tx=0x7f3fd800eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:47.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.854+0000 7f3fecc00640 1 -- 172.21.15.47:0/627420453 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd800cd80 con 0x7f3ff0104a60 2024-03-21T11:23:47.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.855+0000 7f3fecc00640 1 -- 172.21.15.47:0/627420453 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fd80035f0 con 0x7f3ff0104a60 2024-03-21T11:23:47.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.855+0000 7f3fecc00640 1 -- 172.21.15.47:0/627420453 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8010690 con 0x7f3ff0104a60 2024-03-21T11:23:47.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.855+0000 7f3ff6a00640 1 -- 172.21.15.47:0/627420453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff01a1b40 con 0x7f3ff0104a60 2024-03-21T11:23:47.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.855+0000 7f3ff6a00640 1 -- 172.21.15.47:0/627420453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff01a20e0 con 0x7f3ff0104a60 2024-03-21T11:23:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.856+0000 7f3ff6a00640 1 -- 172.21.15.47:0/627420453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff00fd630 con 0x7f3ff0104a60 2024-03-21T11:23:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.858+0000 7f3fecc00640 1 -- 172.21.15.47:0/627420453 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f3fd8004020 con 0x7f3ff0104a60 2024-03-21T11:23:47.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.860+0000 7f3fecc00640 1 --2- 172.21.15.47:0/627420453 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3fd40a3030 0x7f3fd40a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:47.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.860+0000 7f3fecc00640 1 -- 172.21.15.47:0/627420453 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3fd80cfcb0 con 0x7f3ff0104a60 2024-03-21T11:23:47.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.860+0000 7f3fefe00640 1 --2- 172.21.15.47:0/627420453 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3fd40a3030 0x7f3fd40a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:47.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.861+0000 7f3fefe00640 1 --2- 172.21.15.47:0/627420453 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3fd40a3030 0x7f3fd40a54f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0006010 tx=0x7f3fe0000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:47.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:47.864+0000 7f3fecc00640 1 -- 172.21.15.47:0/627420453 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3fd809c5f0 con 0x7f3ff0104a60 2024-03-21T11:23:47.899 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:47 smithi047 ceph-mon[31351]: from='client.25561 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:23:47.899 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:47 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1623042095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:47.899 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:47 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3595727062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:48.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.072+0000 7f3ff6a00640 1 -- 172.21.15.47:0/627420453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ff002d220 con 0x7f3ff0104a60 2024-03-21T11:23:48.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.073+0000 7f3fecc00640 1 -- 172.21.15.47:0/627420453 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3fd809bef0 con 0x7f3ff0104a60 2024-03-21T11:23:48.074 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T11:23:48.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.076+0000 7f3ff6a00640 1 -- 172.21.15.47:0/627420453 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3fd40a3030 msgr2=0x7f3fd40a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:48.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.076+0000 7f3ff6a00640 1 --2- 172.21.15.47:0/627420453 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3fd40a3030 0x7f3fd40a54f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0006010 tx=0x7f3fe0000f40 comp rx=0 tx=0).stop 2024-03-21T11:23:48.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.076+0000 7f3ff6a00640 1 -- 172.21.15.47:0/627420453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff0104a60 msgr2=0x7f3ff01000e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:48.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.076+0000 7f3ff6a00640 1 --2- 172.21.15.47:0/627420453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff0104a60 0x7f3ff01000e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800e9a0 tx=0x7f3fd800eda0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.076+0000 7f3ff6a00640 1 -- 172.21.15.47:0/627420453 shutdown_connections 2024-03-21T11:23:48.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.076+0000 7f3ff6a00640 1 --2- 172.21.15.47:0/627420453 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3fd40a3030 0x7f3fd40a54f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.076+0000 7f3ff6a00640 1 --2- 172.21.15.47:0/627420453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ff0107390 0x7f3ff0100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.076+0000 7f3ff6a00640 1 --2- 172.21.15.47:0/627420453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ff0104a60 0x7f3ff01000e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.076+0000 7f3ff6a00640 1 --2- 172.21.15.47:0/627420453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ff0102130 0x7f3ff0101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.076+0000 7f3ff6a00640 1 -- 172.21.15.47:0/627420453 >> 172.21.15.47:0/627420453 conn(0x7f3ff00fbb40 msgr2=0x7f3ff01029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:48.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.077+0000 7f3ff6a00640 1 -- 172.21.15.47:0/627420453 shutdown_connections 2024-03-21T11:23:48.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.077+0000 7f3ff6a00640 1 -- 172.21.15.47:0/627420453 wait complete. 2024-03-21T11:23:48.095 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T11:23:48.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:47 smithi100 ceph-mon[36245]: from='client.25561 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:23:48.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:47 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1623042095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:48.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:47 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3595727062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:48.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:47 smithi102 ceph-mon[36748]: from='client.25561 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T11:23:48.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:47 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1623042095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:23:48.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:47 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3595727062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:23:48.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.320+0000 7fa8c3200640 1 -- 172.21.15.47:0/1024256219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8bc107260 msgr2=0x7fa8bc1096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:48.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.320+0000 7fa8c3200640 1 --2- 172.21.15.47:0/1024256219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8bc107260 0x7fa8bc1096b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009ca0 tx=0x7fa8b802c100 comp rx=0 tx=0).stop 2024-03-21T11:23:48.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.320+0000 7fa8c3200640 1 -- 172.21.15.47:0/1024256219 shutdown_connections 2024-03-21T11:23:48.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.320+0000 7fa8c3200640 1 --2- 172.21.15.47:0/1024256219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8bc107260 0x7fa8bc1096b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.320+0000 7fa8c3200640 1 --2- 172.21.15.47:0/1024256219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8bc104930 0x7fa8bc106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.320+0000 7fa8c3200640 1 --2- 172.21.15.47:0/1024256219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8bc102000 0x7fa8bc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.320+0000 7fa8c3200640 1 -- 172.21.15.47:0/1024256219 >> 172.21.15.47:0/1024256219 conn(0x7fa8bc0fba10 msgr2=0x7fa8bc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.321+0000 7fa8c3200640 1 -- 172.21.15.47:0/1024256219 shutdown_connections 2024-03-21T11:23:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.321+0000 7fa8c3200640 1 -- 172.21.15.47:0/1024256219 wait complete. 2024-03-21T11:23:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.322+0000 7fa8c3200640 1 Processor -- start 2024-03-21T11:23:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.322+0000 7fa8c3200640 1 -- start start 2024-03-21T11:23:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8bc102000 0x7fa8bc199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8bc104930 0x7fa8bc199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8bc107260 0x7fa8bc1a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc10d210 con 0x7fa8bc107260 2024-03-21T11:23:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc10d090 con 0x7fa8bc104930 2024-03-21T11:23:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc10d390 con 0x7fa8bc102000 2024-03-21T11:23:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8bc102000 0x7fa8bc199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8bc104930 0x7fa8bc199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8bc102000 0x7fa8bc199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52068/0 (socket says 172.21.15.47:52068) 2024-03-21T11:23:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c1e00640 1 -- 172.21.15.47:0/1170147371 learned_addr learned my addr 172.21.15.47:0/1170147371 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c2800640 1 --2- 172.21.15.47:0/1170147371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8bc107260 0x7fa8bc1a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c1e00640 1 -- 172.21.15.47:0/1170147371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8bc104930 msgr2=0x7fa8bc199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c1e00640 1 --2- 172.21.15.47:0/1170147371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8bc104930 0x7fa8bc199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c1e00640 1 -- 172.21.15.47:0/1170147371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8bc107260 msgr2=0x7fa8bc1a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c1e00640 1 --2- 172.21.15.47:0/1170147371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8bc107260 0x7fa8bc1a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c1e00640 1 -- 172.21.15.47:0/1170147371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc1a16a0 con 0x7fa8bc102000 2024-03-21T11:23:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.323+0000 7fa8c2800640 1 --2- 172.21.15.47:0/1170147371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8bc107260 0x7fa8bc1a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.324+0000 7fa8c1e00640 1 --2- 172.21.15.47:0/1170147371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8bc102000 0x7fa8bc199a80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000e9f0 tx=0x7fa8b000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.324+0000 7fa8b6a00640 1 -- 172.21.15.47:0/1170147371 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b000cd80 con 0x7fa8bc102000 2024-03-21T11:23:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.324+0000 7fa8c3200640 1 -- 172.21.15.47:0/1170147371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8bc1a1990 con 0x7fa8bc102000 2024-03-21T11:23:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.324+0000 7fa8b6a00640 1 -- 172.21.15.47:0/1170147371 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8b0005500 con 0x7fa8bc102000 2024-03-21T11:23:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.324+0000 7fa8b6a00640 1 -- 172.21.15.47:0/1170147371 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0010690 con 0x7fa8bc102000 2024-03-21T11:23:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.324+0000 7fa8c3200640 1 -- 172.21.15.47:0/1170147371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8bc1a1e90 con 0x7fa8bc102000 2024-03-21T11:23:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.325+0000 7fa8c3200640 1 -- 172.21.15.47:0/1170147371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8bc108d50 con 0x7fa8bc102000 2024-03-21T11:23:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.327+0000 7fa8b6a00640 1 -- 172.21.15.47:0/1170147371 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fa8b00046c0 con 0x7fa8bc102000 2024-03-21T11:23:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.329+0000 7fa8b6a00640 1 --2- 172.21.15.47:0/1170147371 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8a40a2fb0 0x7fa8a40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.329+0000 7fa8b6a00640 1 -- 172.21.15.47:0/1170147371 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa8b00cfe20 con 0x7fa8bc102000 2024-03-21T11:23:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.329+0000 7fa8c1400640 1 --2- 172.21.15.47:0/1170147371 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8a40a2fb0 0x7fa8a40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.330+0000 7fa8c1400640 1 --2- 172.21.15.47:0/1170147371 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8a40a2fb0 0x7fa8a40a5470 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc19af90 tx=0x7fa8ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.337+0000 7fa8b6a00640 1 -- 172.21.15.47:0/1170147371 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa8b009c700 con 0x7fa8bc102000 2024-03-21T11:23:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.540+0000 7fa8c3200640 1 -- 172.21.15.47:0/1170147371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa8bc02d300 con 0x7fa8bc102000 2024-03-21T11:23:48.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.541+0000 7fa8b6a00640 1 -- 172.21.15.47:0/1170147371 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa8b009c000 con 0x7fa8bc102000 2024-03-21T11:23:48.542 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:23:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.544+0000 7fa8c3200640 1 -- 172.21.15.47:0/1170147371 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8a40a2fb0 msgr2=0x7fa8a40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.544+0000 7fa8c3200640 1 --2- 172.21.15.47:0/1170147371 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8a40a2fb0 0x7fa8a40a5470 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc19af90 tx=0x7fa8ac00b410 comp rx=0 tx=0).stop 2024-03-21T11:23:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.544+0000 7fa8c3200640 1 -- 172.21.15.47:0/1170147371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8bc102000 msgr2=0x7fa8bc199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.544+0000 7fa8c3200640 1 --2- 172.21.15.47:0/1170147371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8bc102000 0x7fa8bc199a80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000e9f0 tx=0x7fa8b000edf0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.544+0000 7fa8c3200640 1 -- 172.21.15.47:0/1170147371 shutdown_connections 2024-03-21T11:23:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.544+0000 7fa8c3200640 1 --2- 172.21.15.47:0/1170147371 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8a40a2fb0 0x7fa8a40a5470 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.544+0000 7fa8c3200640 1 --2- 172.21.15.47:0/1170147371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8bc107260 0x7fa8bc1a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.544+0000 7fa8c3200640 1 --2- 172.21.15.47:0/1170147371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8bc104930 0x7fa8bc199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.544+0000 7fa8c3200640 1 --2- 172.21.15.47:0/1170147371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8bc102000 0x7fa8bc199a80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.544+0000 7fa8c3200640 1 -- 172.21.15.47:0/1170147371 >> 172.21.15.47:0/1170147371 conn(0x7fa8bc0fba10 msgr2=0x7fa8bc101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.544+0000 7fa8c3200640 1 -- 172.21.15.47:0/1170147371 shutdown_connections 2024-03-21T11:23:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.544+0000 7fa8c3200640 1 -- 172.21.15.47:0/1170147371 wait complete. 2024-03-21T11:23:48.563 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:23:48.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.786+0000 7ffa73200640 1 -- 172.21.15.47:0/1485063679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa741014e0 msgr2=0x7ffa74101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:48.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.786+0000 7ffa73200640 1 --2- 172.21.15.47:0/1485063679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa741014e0 0x7ffa74101900 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009900 tx=0x7ffa6402bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:48.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.786+0000 7ffa73200640 1 -- 172.21.15.47:0/1485063679 shutdown_connections 2024-03-21T11:23:48.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.786+0000 7ffa73200640 1 --2- 172.21.15.47:0/1485063679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa74103090 0x7ffa74109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.786+0000 7ffa73200640 1 --2- 172.21.15.47:0/1485063679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa741026d0 0x7ffa74102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.786+0000 7ffa73200640 1 --2- 172.21.15.47:0/1485063679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa741014e0 0x7ffa74101900 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.786+0000 7ffa73200640 1 -- 172.21.15.47:0/1485063679 >> 172.21.15.47:0/1485063679 conn(0x7ffa740fca30 msgr2=0x7ffa740fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:48.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.786+0000 7ffa73200640 1 -- 172.21.15.47:0/1485063679 shutdown_connections 2024-03-21T11:23:48.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.786+0000 7ffa73200640 1 -- 172.21.15.47:0/1485063679 wait complete. 2024-03-21T11:23:48.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.787+0000 7ffa73200640 1 Processor -- start 2024-03-21T11:23:48.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.787+0000 7ffa73200640 1 -- start start 2024-03-21T11:23:48.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa73200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa741014e0 0x7ffa74199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:48.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa73200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa741026d0 0x7ffa74199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:48.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa73200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa74103090 0x7ffa741a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:48.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa73200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7410d190 con 0x7ffa74103090 2024-03-21T11:23:48.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa73200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7410d010 con 0x7ffa741014e0 2024-03-21T11:23:48.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa73200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7410d310 con 0x7ffa741026d0 2024-03-21T11:23:48.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa71e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa741014e0 0x7ffa74199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:48.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa71e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa741014e0 0x7ffa74199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38846/0 (socket says 172.21.15.47:38846) 2024-03-21T11:23:48.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa71e00640 1 -- 172.21.15.47:0/203760074 learned_addr learned my addr 172.21.15.47:0/203760074 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:48.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa69400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa741026d0 0x7ffa74199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:48.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa72800640 1 --2- 172.21.15.47:0/203760074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa74103090 0x7ffa741a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:48.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa71e00640 1 -- 172.21.15.47:0/203760074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa741026d0 msgr2=0x7ffa74199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:48.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa71e00640 1 --2- 172.21.15.47:0/203760074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa741026d0 0x7ffa74199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa71e00640 1 -- 172.21.15.47:0/203760074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa74103090 msgr2=0x7ffa741a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:48.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa71e00640 1 --2- 172.21.15.47:0/203760074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa74103090 0x7ffa741a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:48.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.788+0000 7ffa71e00640 1 -- 172.21.15.47:0/203760074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa741a16d0 con 0x7ffa741014e0 2024-03-21T11:23:48.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.789+0000 7ffa71e00640 1 --2- 172.21.15.47:0/203760074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa741014e0 0x7ffa74199ab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffa64000c00 tx=0x7ffa6402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:48.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.789+0000 7ffa6b400640 1 -- 172.21.15.47:0/203760074 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64043020 con 0x7ffa741014e0 2024-03-21T11:23:48.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.789+0000 7ffa6b400640 1 -- 172.21.15.47:0/203760074 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa640054e0 con 0x7ffa741014e0 2024-03-21T11:23:48.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.789+0000 7ffa73200640 1 -- 172.21.15.47:0/203760074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa741a1960 con 0x7ffa741014e0 2024-03-21T11:23:48.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.789+0000 7ffa6b400640 1 -- 172.21.15.47:0/203760074 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64003c40 con 0x7ffa741014e0 2024-03-21T11:23:48.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.789+0000 7ffa73200640 1 -- 172.21.15.47:0/203760074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa741a1eb0 con 0x7ffa741014e0 2024-03-21T11:23:48.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.790+0000 7ffa73200640 1 -- 172.21.15.47:0/203760074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa74068800 con 0x7ffa741014e0 2024-03-21T11:23:48.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.792+0000 7ffa6b400640 1 -- 172.21.15.47:0/203760074 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7ffa64046020 con 0x7ffa741014e0 2024-03-21T11:23:48.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.793+0000 7ffa6b400640 1 --2- 172.21.15.47:0/203760074 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffa580a2fb0 0x7ffa580a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:48.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.794+0000 7ffa6b400640 1 -- 172.21.15.47:0/203760074 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffa640f18e0 con 0x7ffa741014e0 2024-03-21T11:23:48.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.794+0000 7ffa69400640 1 --2- 172.21.15.47:0/203760074 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffa580a2fb0 0x7ffa580a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:48.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.794+0000 7ffa69400640 1 --2- 172.21.15.47:0/203760074 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffa580a2fb0 0x7ffa580a5470 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffa7419afc0 tx=0x7ffa5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:48.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:48.797+0000 7ffa6b400640 1 -- 172.21.15.47:0/203760074 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffa6404d020 con 0x7ffa741014e0 2024-03-21T11:23:49.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.041+0000 7ffa73200640 1 -- 172.21.15.47:0/203760074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa741a2260 con 0x7ffa741014e0 2024-03-21T11:23:49.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.043+0000 7ffa6b400640 1 -- 172.21.15.47:0/203760074 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7ffa640be330 con 0x7ffa741014e0 2024-03-21T11:23:49.044 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:49.044 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:23:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:23:49.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.045+0000 7ffa73200640 1 -- 172.21.15.47:0/203760074 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffa580a2fb0 msgr2=0x7ffa580a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.045+0000 7ffa73200640 1 --2- 172.21.15.47:0/203760074 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffa580a2fb0 0x7ffa580a5470 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffa7419afc0 tx=0x7ffa5c005f90 comp rx=0 tx=0).stop 2024-03-21T11:23:49.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.045+0000 7ffa73200640 1 -- 172.21.15.47:0/203760074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa741014e0 msgr2=0x7ffa74199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.045+0000 7ffa73200640 1 --2- 172.21.15.47:0/203760074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa741014e0 0x7ffa74199ab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffa64000c00 tx=0x7ffa6402c650 comp rx=0 tx=0).stop 2024-03-21T11:23:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.046+0000 7ffa73200640 1 -- 172.21.15.47:0/203760074 shutdown_connections 2024-03-21T11:23:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.046+0000 7ffa73200640 1 --2- 172.21.15.47:0/203760074 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffa580a2fb0 0x7ffa580a5470 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.046+0000 7ffa73200640 1 --2- 172.21.15.47:0/203760074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa74103090 0x7ffa741a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.046+0000 7ffa73200640 1 --2- 172.21.15.47:0/203760074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa741026d0 0x7ffa74199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.046+0000 7ffa73200640 1 --2- 172.21.15.47:0/203760074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa741014e0 0x7ffa74199ab0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.046+0000 7ffa73200640 1 -- 172.21.15.47:0/203760074 >> 172.21.15.47:0/203760074 conn(0x7ffa740fca30 msgr2=0x7ffa740738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.046+0000 7ffa73200640 1 -- 172.21.15.47:0/203760074 shutdown_connections 2024-03-21T11:23:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.046+0000 7ffa73200640 1 -- 172.21.15.47:0/203760074 wait complete. 2024-03-21T11:23:49.066 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:23:49.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:48 smithi100 ceph-mon[36245]: pgmap v414: 129 pgs: 129 active+clean; 657 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-21T11:23:49.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:48 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/627420453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:49.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:48 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1170147371' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:49.150 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:48 smithi047 ceph-mon[31351]: pgmap v414: 129 pgs: 129 active+clean; 657 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-21T11:23:49.150 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:48 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/627420453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:49.150 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:48 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1170147371' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:48 smithi102 ceph-mon[36748]: pgmap v414: 129 pgs: 129 active+clean; 657 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-21T11:23:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:48 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/627420453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:23:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:48 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1170147371' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:23:49.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.287+0000 7f41e3e00640 1 -- 172.21.15.47:0/3502344520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41e41096b0 msgr2=0x7f41e410ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.287+0000 7f41e3e00640 1 --2- 172.21.15.47:0/3502344520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41e41096b0 0x7f41e410ff40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009860 tx=0x7f41dc02bb70 comp rx=0 tx=0).stop 2024-03-21T11:23:49.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.287+0000 7f41e3e00640 1 -- 172.21.15.47:0/3502344520 shutdown_connections 2024-03-21T11:23:49.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.287+0000 7f41e3e00640 1 --2- 172.21.15.47:0/3502344520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41e41096b0 0x7f41e410ff40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.287+0000 7f41e3e00640 1 --2- 172.21.15.47:0/3502344520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e4108cf0 0x7f41e4109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.287+0000 7f41e3e00640 1 --2- 172.21.15.47:0/3502344520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41e4107b00 0x7f41e4107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.287+0000 7f41e3e00640 1 -- 172.21.15.47:0/3502344520 >> 172.21.15.47:0/3502344520 conn(0x7f41e4076ab0 msgr2=0x7f41e4078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:49.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.287+0000 7f41e3e00640 1 -- 172.21.15.47:0/3502344520 shutdown_connections 2024-03-21T11:23:49.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.287+0000 7f41e3e00640 1 -- 172.21.15.47:0/3502344520 wait complete. 2024-03-21T11:23:49.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.288+0000 7f41e3e00640 1 Processor -- start 2024-03-21T11:23:49.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.288+0000 7f41e3e00640 1 -- start start 2024-03-21T11:23:49.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.288+0000 7f41e3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41e4107b00 0x7f41e419dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.288+0000 7f41e3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41e4108cf0 0x7f41e419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.288+0000 7f41e3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41096b0 0x7f41e41a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.288+0000 7f41e3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e4113760 con 0x7f41e4107b00 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.288+0000 7f41e3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e41135e0 con 0x7f41e41096b0 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.288+0000 7f41e3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e41138e0 con 0x7f41e4108cf0 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.289+0000 7f41e2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41e4108cf0 0x7f41e419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.289+0000 7f41e2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41e4107b00 0x7f41e419dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.289+0000 7f41e3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41096b0 0x7f41e41a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.289+0000 7f41e2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41e4108cf0 0x7f41e419e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36442/0 (socket says 172.21.15.47:36442) 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.289+0000 7f41e2000640 1 -- 172.21.15.47:0/4212070885 learned_addr learned my addr 172.21.15.47:0/4212070885 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.289+0000 7f41e2000640 1 -- 172.21.15.47:0/4212070885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41096b0 msgr2=0x7f41e41a5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.289+0000 7f41e2000640 1 --2- 172.21.15.47:0/4212070885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41096b0 0x7f41e41a5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.289+0000 7f41e2000640 1 -- 172.21.15.47:0/4212070885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41e4107b00 msgr2=0x7f41e419dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.289+0000 7f41e2000640 1 --2- 172.21.15.47:0/4212070885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41e4107b00 0x7f41e419dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.289+0000 7f41e2000640 1 -- 172.21.15.47:0/4212070885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e41a5970 con 0x7f41e4108cf0 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.289+0000 7f41e3400640 1 --2- 172.21.15.47:0/4212070885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41096b0 0x7f41e41a5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.289+0000 7f41e2000640 1 --2- 172.21.15.47:0/4212070885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41e4108cf0 0x7f41e419e290 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00e9c0 tx=0x7f41cc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:49.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.290+0000 7f41db400640 1 -- 172.21.15.47:0/4212070885 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc00cde0 con 0x7f41e4108cf0 2024-03-21T11:23:49.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.290+0000 7f41db400640 1 -- 172.21.15.47:0/4212070885 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41cc0035f0 con 0x7f41e4108cf0 2024-03-21T11:23:49.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.290+0000 7f41e3e00640 1 -- 172.21.15.47:0/4212070885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e41a5c60 con 0x7f41e4108cf0 2024-03-21T11:23:49.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.290+0000 7f41db400640 1 -- 172.21.15.47:0/4212070885 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc010650 con 0x7f41e4108cf0 2024-03-21T11:23:49.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.290+0000 7f41e3e00640 1 -- 172.21.15.47:0/4212070885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e41a61b0 con 0x7f41e4108cf0 2024-03-21T11:23:49.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.291+0000 7f41e3e00640 1 -- 172.21.15.47:0/4212070885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e410edc0 con 0x7f41e4108cf0 2024-03-21T11:23:49.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.293+0000 7f41db400640 1 -- 172.21.15.47:0/4212070885 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f41cc0037a0 con 0x7f41e4108cf0 2024-03-21T11:23:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.294+0000 7f41db400640 1 --2- 172.21.15.47:0/4212070885 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41c80a3030 0x7f41c80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.294+0000 7f41db400640 1 -- 172.21.15.47:0/4212070885 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f41cc0cfbd0 con 0x7f41e4108cf0 2024-03-21T11:23:49.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.295+0000 7f41e2a00640 1 --2- 172.21.15.47:0/4212070885 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41c80a3030 0x7f41c80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.295+0000 7f41e2a00640 1 --2- 172.21.15.47:0/4212070885 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41c80a3030 0x7f41c80a54f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f41e4108b50 tx=0x7f41d4009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:49.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.297+0000 7f41db400640 1 -- 172.21.15.47:0/4212070885 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f41cc09c4b0 con 0x7f41e4108cf0 2024-03-21T11:23:49.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.506+0000 7f41e3e00640 1 -- 172.21.15.47:0/4212070885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41e402d1f0 con 0x7f41e4108cf0 2024-03-21T11:23:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.508+0000 7f41db400640 1 -- 172.21.15.47:0/4212070885 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24747 (secure 0 0 0) 0x7f41cc09bdb0 con 0x7f41e4108cf0 2024-03-21T11:23:49.509 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:23:49.510 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:22:50.841892+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:23:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.511+0000 7f41e3e00640 1 -- 172.21.15.47:0/4212070885 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41c80a3030 msgr2=0x7f41c80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.511+0000 7f41e3e00640 1 --2- 172.21.15.47:0/4212070885 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41c80a3030 0x7f41c80a54f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f41e4108b50 tx=0x7f41d4009670 comp rx=0 tx=0).stop 2024-03-21T11:23:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.511+0000 7f41e3e00640 1 -- 172.21.15.47:0/4212070885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41e4108cf0 msgr2=0x7f41e419e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.511+0000 7f41e3e00640 1 --2- 172.21.15.47:0/4212070885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41e4108cf0 0x7f41e419e290 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00e9c0 tx=0x7f41cc00ee80 comp rx=0 tx=0).stop 2024-03-21T11:23:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.511+0000 7f41e3e00640 1 -- 172.21.15.47:0/4212070885 shutdown_connections 2024-03-21T11:23:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.511+0000 7f41e3e00640 1 --2- 172.21.15.47:0/4212070885 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f41c80a3030 0x7f41c80a54f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.511+0000 7f41e3e00640 1 --2- 172.21.15.47:0/4212070885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41e41096b0 0x7f41e41a5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.511+0000 7f41e3e00640 1 --2- 172.21.15.47:0/4212070885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41e4108cf0 0x7f41e419e290 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.511+0000 7f41e3e00640 1 --2- 172.21.15.47:0/4212070885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41e4107b00 0x7f41e419dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.511+0000 7f41e3e00640 1 -- 172.21.15.47:0/4212070885 >> 172.21.15.47:0/4212070885 conn(0x7f41e4076ab0 msgr2=0x7f41e410bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.512+0000 7f41e3e00640 1 -- 172.21.15.47:0/4212070885 shutdown_connections 2024-03-21T11:23:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.512+0000 7f41e3e00640 1 -- 172.21.15.47:0/4212070885 wait complete. 2024-03-21T11:23:49.531 DEBUG:tasks.fwd_scrub:Yielding 2024-03-21T11:23:49.532 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-21T11:23:49.542 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T11:23:49.542 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-21T11:23:49.542 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-21T11:23:49.542 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:23:49.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:23:49.543 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:23:49.547 INFO:teuthology.run_tasks:Running task workunit... 2024-03-21T11:23:49.573 INFO:tasks.workunit:Pulling workunits from ref 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T11:23:49.573 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-21T11:23:49.573 INFO:tasks.workunit:timeout=3h 2024-03-21T11:23:49.573 INFO:tasks.workunit:cleanup=True 2024-03-21T11:23:49.573 DEBUG:teuthology.orchestra.run.smithi047:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-21T11:23:49.613 INFO:teuthology.orchestra.run.smithi047.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-21T11:23:49.613 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-21T11:23:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 76h/118d Inode: 1099511628283 Links: 2 2024-03-21T11:23:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T11:23:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-21T11:23:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-21 11:21:19.443030693 +0000 2024-03-21T11:23:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-21 11:21:19.443030693 +0000 2024-03-21T11:23:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-21 11:23:37.378863309 +0000 2024-03-21T11:23:49.614 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-03-21 11:21:19.443030693 +0000 2024-03-21T11:23:49.614 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-21T11:23:49.614 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-21T11:23:49.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.735+0000 7f16ce800640 1 -- 172.21.15.47:0/524864182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16c80fffb0 msgr2=0x7f16c81003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.735+0000 7f16ce800640 1 --2- 172.21.15.47:0/524864182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16c80fffb0 0x7f16c81003d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009900 tx=0x7f16b802bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:49.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.736+0000 7f16ce800640 1 -- 172.21.15.47:0/524864182 shutdown_connections 2024-03-21T11:23:49.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.736+0000 7f16ce800640 1 --2- 172.21.15.47:0/524864182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16c81012b0 0x7f16c8101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.736+0000 7f16ce800640 1 --2- 172.21.15.47:0/524864182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16c8100910 0x7f16c8100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.736+0000 7f16ce800640 1 --2- 172.21.15.47:0/524864182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16c80fffb0 0x7f16c81003d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.736+0000 7f16ce800640 1 -- 172.21.15.47:0/524864182 >> 172.21.15.47:0/524864182 conn(0x7f16c80fbb50 msgr2=0x7f16c80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:49.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.736+0000 7f16ce800640 1 -- 172.21.15.47:0/524864182 shutdown_connections 2024-03-21T11:23:49.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.736+0000 7f16ce800640 1 -- 172.21.15.47:0/524864182 wait complete. 2024-03-21T11:23:49.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.737+0000 7f16ce800640 1 Processor -- start 2024-03-21T11:23:49.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.737+0000 7f16ce800640 1 -- start start 2024-03-21T11:23:49.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16ce800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16c80fffb0 0x7f16c8199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16ce800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16c8100910 0x7f16c819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16ce800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16c81012b0 0x7f16c81a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16ce800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c810f570 con 0x7f16c81012b0 2024-03-21T11:23:49.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16ce800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c810f3f0 con 0x7f16c8100910 2024-03-21T11:23:49.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16ce800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c810f6f0 con 0x7f16c80fffb0 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16c6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16c8100910 0x7f16c819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16c6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16c8100910 0x7f16c819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53692/0 (socket says 172.21.15.47:53692) 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16c6a00640 1 -- 172.21.15.47:0/1654138044 learned_addr learned my addr 172.21.15.47:0/1654138044 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16c7400640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16c80fffb0 0x7f16c8199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16c7e00640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16c81012b0 0x7f16c81a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16c6a00640 1 -- 172.21.15.47:0/1654138044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16c80fffb0 msgr2=0x7f16c8199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16c6a00640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16c80fffb0 0x7f16c8199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16c6a00640 1 -- 172.21.15.47:0/1654138044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16c81012b0 msgr2=0x7f16c81a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16c6a00640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16c81012b0 0x7f16c81a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16c6a00640 1 -- 172.21.15.47:0/1654138044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c81a1870 con 0x7f16c8100910 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.738+0000 7f16c7e00640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16c81012b0 0x7f16c81a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.739+0000 7f16c6a00640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16c8100910 0x7f16c819a1a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f16b400e9f0 tx=0x7f16b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.739+0000 7f16bfe00640 1 -- 172.21.15.47:0/1654138044 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b400cd80 con 0x7f16c8100910 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.739+0000 7f16bfe00640 1 -- 172.21.15.47:0/1654138044 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16b4010910 con 0x7f16c8100910 2024-03-21T11:23:49.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.739+0000 7f16ce800640 1 -- 172.21.15.47:0/1654138044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c81a1b60 con 0x7f16c8100910 2024-03-21T11:23:49.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.739+0000 7f16bfe00640 1 -- 172.21.15.47:0/1654138044 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4018990 con 0x7f16c8100910 2024-03-21T11:23:49.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.739+0000 7f16ce800640 1 -- 172.21.15.47:0/1654138044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c81a2080 con 0x7f16c8100910 2024-03-21T11:23:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.743+0000 7f16d160b740 1 Processor -- start 2024-03-21T11:23:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.743+0000 7f16d160b740 1 -- start start 2024-03-21T11:23:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.743+0000 7f16bfe00640 1 -- 172.21.15.47:0/1654138044 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f16b4010ac0 con 0x7f16c8100910 2024-03-21T11:23:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.743+0000 7f16d160b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238c0a0 0x562c2238c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.743+0000 7f16d160b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c2238ca90 0x562c22391590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.743+0000 7f16d160b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c22391c00 0x562c22394090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.743+0000 7f16d160b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c222f1eb0 con 0x562c2238c0a0 2024-03-21T11:23:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.743+0000 7f16d160b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2229aa60 con 0x562c2238ca90 2024-03-21T11:23:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.743+0000 7f16d160b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c222edf50 con 0x562c22391c00 2024-03-21T11:23:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.743+0000 7f16c7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c2238ca90 0x562c22391590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.743+0000 7f16c7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c2238ca90 0x562c22391590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53694/0 (socket says 172.21.15.47:53694) 2024-03-21T11:23:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.743+0000 7f16c7400640 1 -- 172.21.15.47:0/3906725081 learned_addr learned my addr 172.21.15.47:0/3906725081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:49.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.743+0000 7f16c6a00640 1 --2- 172.21.15.47:0/3906725081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c22391c00 0x562c22394090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.743+0000 7f16c7e00640 1 --2- 172.21.15.47:0/3906725081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238c0a0 0x562c2238c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.744+0000 7f16c7400640 1 -- 172.21.15.47:0/3906725081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c22391c00 msgr2=0x562c22394090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.744+0000 7f16c7400640 1 --2- 172.21.15.47:0/3906725081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c22391c00 0x562c22394090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.744+0000 7f16c7400640 1 -- 172.21.15.47:0/3906725081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238c0a0 msgr2=0x562c2238c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.744+0000 7f16c7400640 1 --2- 172.21.15.47:0/3906725081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238c0a0 0x562c2238c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.744+0000 7f16c7400640 1 -- 172.21.15.47:0/3906725081 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c220ce7e0 con 0x562c2238ca90 2024-03-21T11:23:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.744+0000 7f16c7400640 1 --2- 172.21.15.47:0/3906725081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c2238ca90 0x562c22391590 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009900 tx=0x7f16b8005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f14e79f6e92569c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.745+0000 7f16bfe00640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f16ac0a2ff0 0x7f16ac0a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.745+0000 7f16bcc00640 1 -- 172.21.15.47:0/3906725081 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8045020 con 0x562c2238ca90 2024-03-21T11:23:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.745+0000 7f16bcc00640 1 -- 172.21.15.47:0/3906725081 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16b8004040 con 0x562c2238ca90 2024-03-21T11:23:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.745+0000 7f16bcc00640 1 -- 172.21.15.47:0/3906725081 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8005540 con 0x562c2238ca90 2024-03-21T11:23:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.745+0000 7f16bfe00640 1 -- 172.21.15.47:0/1654138044 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f16b40cfd30 con 0x7f16c8100910 2024-03-21T11:23:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.745+0000 7f16c7e00640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f16ac0a2ff0 0x7f16ac0a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.746+0000 7f16d160b740 1 -- 172.21.15.47:0/3906725081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c2238ca90 msgr2=0x562c22391590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.746+0000 7f16d160b740 1 --2- 172.21.15.47:0/3906725081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c2238ca90 0x562c22391590 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009900 tx=0x7f16b8005b50 comp rx=0 tx=0).stop 2024-03-21T11:23:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.746+0000 7f16d160b740 1 -- 172.21.15.47:0/3906725081 shutdown_connections 2024-03-21T11:23:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.746+0000 7f16d160b740 1 --2- 172.21.15.47:0/3906725081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c22391c00 0x562c22394090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.746+0000 7f16d160b740 1 --2- 172.21.15.47:0/3906725081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c2238ca90 0x562c22391590 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.746+0000 7f16d160b740 1 --2- 172.21.15.47:0/3906725081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238c0a0 0x562c2238c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.746+0000 7f16d160b740 1 -- 172.21.15.47:0/3906725081 >> 172.21.15.47:0/3906725081 conn(0x562c22387610 msgr2=0x562c22387a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.746+0000 7f16c7e00640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f16ac0a2ff0 0x7f16ac0a54b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x562c2238d2f0 tx=0x7f16c0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.746+0000 7f16d160b740 1 -- 172.21.15.47:0/3906725081 shutdown_connections 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.746+0000 7f16d160b740 1 -- 172.21.15.47:0/3906725081 wait complete. 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.746+0000 7f16d160b740 1 -- start start 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.746+0000 7f16d160b740 1 Processor -- start 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16d160b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238c0a0 0x562c22396250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16d160b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c2238ca90 0x562c22396790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16d160b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c22391c00 0x562c2239d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16d160b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c22394b50 con 0x562c22391c00 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16d160b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c222adeb0 con 0x562c2238ca90 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16d160b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c22394cd0 con 0x562c2238c0a0 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16d160b740 10 client.0 _ll_register_callbacks cb 0x562c222a2150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16c6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c2238ca90 0x562c22396790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16c7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238c0a0 0x562c22396250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16c6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c2238ca90 0x562c22396790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53710/0 (socket says 172.21.15.47:53710) 2024-03-21T11:23:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16c7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238c0a0 0x562c22396250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36468/0 (socket says 172.21.15.47:36468) 2024-03-21T11:23:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16c6a00640 1 -- 172.21.15.47:0/448667812 learned_addr learned my addr 172.21.15.47:0/448667812 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16c7e00640 1 --2- 172.21.15.47:0/448667812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c22391c00 0x562c2239d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16c6a00640 1 -- 172.21.15.47:0/448667812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238c0a0 msgr2=0x562c22396250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16c6a00640 1 --2- 172.21.15.47:0/448667812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238c0a0 0x562c22396250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16c6a00640 1 -- 172.21.15.47:0/448667812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c22391c00 msgr2=0x562c2239d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16c6a00640 1 --2- 172.21.15.47:0/448667812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c22391c00 0x562c2239d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.747+0000 7f16c6a00640 1 -- 172.21.15.47:0/448667812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c2212ef70 con 0x562c2238ca90 2024-03-21T11:23:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.748+0000 7f16c6a00640 1 --2- 172.21.15.47:0/448667812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c2238ca90 0x562c22396790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f16b40a4450 tx=0x7f16b40cd7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcb22f1cb9a04421 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.748+0000 7f16b1600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:23:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.748+0000 7f16b1600640 1 -- 172.21.15.47:0/448667812 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b409b8d0 con 0x562c2238ca90 2024-03-21T11:23:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.748+0000 7f16b1600640 1 -- 172.21.15.47:0/448667812 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f16b4018d30 con 0x562c2238ca90 2024-03-21T11:23:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.748+0000 7f16d160b740 1 -- 172.21.15.47:0/448667812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16b8004040 con 0x562c2238ca90 2024-03-21T11:23:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.748+0000 7f16b1600640 1 -- 172.21.15.47:0/448667812 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16b409a0b0 con 0x562c2238ca90 2024-03-21T11:23:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.748+0000 7f16d160b740 10 client.25598 fetch_fsmap learned FSMap version 28 2024-03-21T11:23:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.748+0000 7f16d160b740 1 -- 172.21.15.47:0/448667812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562c21fc6170 con 0x562c2238ca90 2024-03-21T11:23:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.748+0000 7f16b1600640 1 -- 172.21.15.47:0/448667812 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16b8004040 con 0x562c2238ca90 2024-03-21T11:23:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.749+0000 7f16b1600640 1 -- 172.21.15.47:0/448667812 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f16b40bace0 con 0x562c2238ca90 2024-03-21T11:23:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.749+0000 7f16d160b740 10 client.25598 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:23:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.749+0000 7f16d160b740 10 client.25598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:23:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.749+0000 7f16d160b740 20 client.25598 populate_metadata read hostname 'smithi047' 2024-03-21T11:23:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.749+0000 7f16d160b740 1 --2- 172.21.15.47:0/448667812 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562c223a2080 0x562c223a4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.749+0000 7f16d160b740 4 client.25598 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:23:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.749+0000 7f16d160b740 1 -- 172.21.15.47:0/448667812 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c222ff3f0 con 0x562c223a2080 2024-03-21T11:23:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.749+0000 7f16c7400640 1 --2- 172.21.15.47:0/448667812 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562c223a2080 0x562c223a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.755 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T11:23:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.750+0000 7f16c7400640 1 --2- 172.21.15.47:0/448667812 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562c223a2080 0x562c223a4490 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f2695d93fd70592 server_cookie=6edb5251c6877ba9 in_seq=0 out_seq=0 2024-03-21T11:23:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.750+0000 7f16b1600640 10 client.25598 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:23:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.750+0000 7f16b1600640 1 -- 172.21.15.47:0/448667812 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562c222ff3f0 con 0x562c223a2080 2024-03-21T11:23:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.750+0000 7f16b1600640 10 client.25598 handle_command_reply: tid=0 2024-03-21T11:23:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.750+0000 7f16d160b740 1 client.25598 shutdown 2024-03-21T11:23:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.751+0000 7f16d160b740 1 -- 172.21.15.47:0/448667812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c2238ca90 msgr2=0x562c22396790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.751+0000 7f16d160b740 1 --2- 172.21.15.47:0/448667812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c2238ca90 0x562c22396790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f16b40a4450 tx=0x7f16b40cd7d0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.751+0000 7f16d160b740 1 -- 172.21.15.47:0/448667812 shutdown_connections 2024-03-21T11:23:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.751+0000 7f16d160b740 1 --2- 172.21.15.47:0/448667812 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562c223a2080 0x562c223a4490 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.751+0000 7f16d160b740 1 --2- 172.21.15.47:0/448667812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c22391c00 0x562c2239d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.751+0000 7f16d160b740 1 --2- 172.21.15.47:0/448667812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c2238ca90 0x562c22396790 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.751+0000 7f16d160b740 1 --2- 172.21.15.47:0/448667812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238c0a0 0x562c22396250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.751+0000 7f16b1600640 0 client.25598 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:23:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.751+0000 7f16d160b740 1 -- 172.21.15.47:0/448667812 >> 172.21.15.47:0/448667812 conn(0x562c22387610 msgr2=0x562c22386830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.751+0000 7f16d160b740 1 -- 172.21.15.47:0/448667812 shutdown_connections 2024-03-21T11:23:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.751+0000 7f16d160b740 1 -- 172.21.15.47:0/448667812 wait complete. 2024-03-21T11:23:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.751+0000 7f16d160b740 20 client.25598 trim_cache size 0 max 16384 2024-03-21T11:23:49.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.771+0000 7f16d160b740 1 Processor -- start 2024-03-21T11:23:49.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.771+0000 7f16d160b740 1 -- start start 2024-03-21T11:23:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.771+0000 7f16d160b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238c0a0 0x562c223866b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.771+0000 7f16d160b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238ca90 0x562c22386bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.771+0000 7f16d160b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c22391c00 0x562c2239d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.771+0000 7f16d160b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c81ac2c0 con 0x562c2238ca90 2024-03-21T11:23:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.771+0000 7f16d160b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c81adc40 con 0x562c22391c00 2024-03-21T11:23:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.771+0000 7f16d160b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c81ac920 con 0x562c2238c0a0 2024-03-21T11:23:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.772+0000 7f16c7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238c0a0 0x562c223866b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.772+0000 7f16c7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238c0a0 0x562c223866b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36484/0 (socket says 172.21.15.47:36484) 2024-03-21T11:23:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.772+0000 7f16c7400640 1 -- 172.21.15.47:0/2434365885 learned_addr learned my addr 172.21.15.47:0/2434365885 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.772+0000 7f16c7e00640 1 --2- 172.21.15.47:0/2434365885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c22391c00 0x562c2239d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.772+0000 7f16c6a00640 1 --2- 172.21.15.47:0/2434365885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238ca90 0x562c22386bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.772+0000 7f16c7400640 1 -- 172.21.15.47:0/2434365885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c22391c00 msgr2=0x562c2239d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.772+0000 7f16c7400640 1 --2- 172.21.15.47:0/2434365885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c22391c00 0x562c2239d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.772+0000 7f16c7400640 1 -- 172.21.15.47:0/2434365885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238ca90 msgr2=0x562c22386bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.772+0000 7f16c7400640 1 --2- 172.21.15.47:0/2434365885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238ca90 0x562c22386bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.772+0000 7f16c7400640 1 -- 172.21.15.47:0/2434365885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c22382c20 con 0x562c2238c0a0 2024-03-21T11:23:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.772+0000 7f16c7e00640 1 --2- 172.21.15.47:0/2434365885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c22391c00 0x562c2239d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.772+0000 7f16c7400640 1 --2- 172.21.15.47:0/2434365885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238c0a0 0x562c223866b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f16b8005ab0 tx=0x7f16b8005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bbc87b2ddf12707 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.773+0000 7f16b3e00640 1 -- 172.21.15.47:0/2434365885 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8045040 con 0x562c2238c0a0 2024-03-21T11:23:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.773+0000 7f16b3e00640 1 -- 172.21.15.47:0/2434365885 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16b804e070 con 0x562c2238c0a0 2024-03-21T11:23:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.773+0000 7f16b3e00640 1 -- 172.21.15.47:0/2434365885 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8005650 con 0x562c2238c0a0 2024-03-21T11:23:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.773+0000 7f16d160b740 1 -- 172.21.15.47:0/2434365885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238c0a0 msgr2=0x562c223866b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.773+0000 7f16d160b740 1 --2- 172.21.15.47:0/2434365885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238c0a0 0x562c223866b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f16b8005ab0 tx=0x7f16b8005b50 comp rx=0 tx=0).stop 2024-03-21T11:23:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.774+0000 7f16d160b740 1 -- 172.21.15.47:0/2434365885 shutdown_connections 2024-03-21T11:23:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.774+0000 7f16d160b740 1 --2- 172.21.15.47:0/2434365885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c22391c00 0x562c2239d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.774+0000 7f16d160b740 1 --2- 172.21.15.47:0/2434365885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238ca90 0x562c22386bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.774+0000 7f16d160b740 1 --2- 172.21.15.47:0/2434365885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238c0a0 0x562c223866b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.774+0000 7f16d160b740 1 -- 172.21.15.47:0/2434365885 >> 172.21.15.47:0/2434365885 conn(0x562c22387610 msgr2=0x562c22385fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.774+0000 7f16d160b740 1 -- 172.21.15.47:0/2434365885 shutdown_connections 2024-03-21T11:23:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.774+0000 7f16d160b740 1 -- 172.21.15.47:0/2434365885 wait complete. 2024-03-21T11:23:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.774+0000 7f16d160b740 1 -- start start 2024-03-21T11:23:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.774+0000 7f16d160b740 1 Processor -- start 2024-03-21T11:23:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16d160b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238c0a0 0x562c2239d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16d160b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238ca90 0x562c2239d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16d160b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c22391c00 0x562c2239b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16d160b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c81ad5e0 con 0x562c2238c0a0 2024-03-21T11:23:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16d160b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c81acf80 con 0x562c22391c00 2024-03-21T11:23:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16d160b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c81aa480 con 0x562c2238ca90 2024-03-21T11:23:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16d160b740 10 client.0 _ll_register_callbacks cb 0x7f16b40d3570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:23:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16c6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238ca90 0x562c2239d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16c6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238ca90 0x562c2239d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36494/0 (socket says 172.21.15.47:36494) 2024-03-21T11:23:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16c6a00640 1 -- 172.21.15.47:0/1308260444 learned_addr learned my addr 172.21.15.47:0/1308260444 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16c7e00640 1 --2- 172.21.15.47:0/1308260444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c22391c00 0x562c2239b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16c7400640 1 --2- 172.21.15.47:0/1308260444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238c0a0 0x562c2239d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16c6a00640 1 -- 172.21.15.47:0/1308260444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c22391c00 msgr2=0x562c2239b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16c6a00640 1 --2- 172.21.15.47:0/1308260444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c22391c00 0x562c2239b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16c6a00640 1 -- 172.21.15.47:0/1308260444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238c0a0 msgr2=0x562c2239d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16c6a00640 1 --2- 172.21.15.47:0/1308260444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238c0a0 0x562c2239d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16c6a00640 1 -- 172.21.15.47:0/1308260444 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c2239c170 con 0x562c2238ca90 2024-03-21T11:23:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.775+0000 7f16c7e00640 1 --2- 172.21.15.47:0/1308260444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c22391c00 0x562c2239b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:23:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.776+0000 7f16c6a00640 1 --2- 172.21.15.47:0/1308260444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238ca90 0x562c2239d780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f16b40d3620 tx=0x7f16b40cd3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bfb1935547c6aca server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.776+0000 7f16b1600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:23:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.776+0000 7f16b1600640 1 -- 172.21.15.47:0/1308260444 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b409b8d0 con 0x562c2238ca90 2024-03-21T11:23:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.776+0000 7f16b1600640 1 -- 172.21.15.47:0/1308260444 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f16b4097c30 con 0x562c2238ca90 2024-03-21T11:23:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.776+0000 7f16d160b740 1 -- 172.21.15.47:0/1308260444 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16b804e070 con 0x562c2238ca90 2024-03-21T11:23:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.776+0000 7f16b1600640 1 -- 172.21.15.47:0/1308260444 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16b40a4980 con 0x562c2238ca90 2024-03-21T11:23:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.776+0000 7f16d160b740 10 client.25621 fetch_fsmap learned FSMap version 28 2024-03-21T11:23:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.776+0000 7f16d160b740 1 -- 172.21.15.47:0/1308260444 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562c223fe580 con 0x562c2238ca90 2024-03-21T11:23:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.776+0000 7f16b1600640 1 -- 172.21.15.47:0/1308260444 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16b804e070 con 0x562c2238ca90 2024-03-21T11:23:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.777+0000 7f16b1600640 1 -- 172.21.15.47:0/1308260444 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f16b40bace0 con 0x562c2238ca90 2024-03-21T11:23:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.777+0000 7f16d160b740 10 client.25621 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:23:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.777+0000 7f16d160b740 10 client.25621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:23:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.777+0000 7f16d160b740 20 client.25621 populate_metadata read hostname 'smithi047' 2024-03-21T11:23:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.777+0000 7f16d160b740 1 --2- 172.21.15.47:0/1308260444 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562c223feb50 0x562c22400f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.777+0000 7f16d160b740 4 client.25621 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:23:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.777+0000 7f16d160b740 1 -- 172.21.15.47:0/1308260444 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562c222f9740 con 0x562c223feb50 2024-03-21T11:23:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.777+0000 7f16c7400640 1 --2- 172.21.15.47:0/1308260444 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562c223feb50 0x562c22400f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.777+0000 7f16c7400640 1 --2- 172.21.15.47:0/1308260444 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562c223feb50 0x562c22400f70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1ad735b3e2e4075 server_cookie=2764909bd913d6c in_seq=0 out_seq=0 2024-03-21T11:23:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.777+0000 7f16b1600640 10 client.25621 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:23:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.779+0000 7f16b1600640 1 -- 172.21.15.47:0/1308260444 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562c222f9740 con 0x562c223feb50 2024-03-21T11:23:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.779+0000 7f16b1600640 10 client.25621 handle_command_reply: tid=0 2024-03-21T11:23:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.779+0000 7f16d160b740 1 client.25621 shutdown 2024-03-21T11:23:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.779+0000 7f16d160b740 1 -- 172.21.15.47:0/1308260444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238ca90 msgr2=0x562c2239d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.779+0000 7f16d160b740 1 --2- 172.21.15.47:0/1308260444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238ca90 0x562c2239d780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f16b40d3620 tx=0x7f16b40cd3d0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.779+0000 7f16d160b740 1 -- 172.21.15.47:0/1308260444 shutdown_connections 2024-03-21T11:23:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.779+0000 7f16d160b740 1 --2- 172.21.15.47:0/1308260444 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562c223feb50 0x562c22400f70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.779+0000 7f16d160b740 1 --2- 172.21.15.47:0/1308260444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c22391c00 0x562c2239b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.779+0000 7f16d160b740 1 --2- 172.21.15.47:0/1308260444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c2238ca90 0x562c2239d780 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.779+0000 7f16d160b740 1 --2- 172.21.15.47:0/1308260444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c2238c0a0 0x562c2239d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.779+0000 7f16b1600640 0 client.25621 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:23:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.779+0000 7f16d160b740 1 -- 172.21.15.47:0/1308260444 >> 172.21.15.47:0/1308260444 conn(0x562c22387610 msgr2=0x562c22386830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.779+0000 7f16d160b740 1 -- 172.21.15.47:0/1308260444 shutdown_connections 2024-03-21T11:23:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.780+0000 7f16d160b740 1 -- 172.21.15.47:0/1308260444 wait complete. 2024-03-21T11:23:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.780+0000 7f16d160b740 20 client.25621 trim_cache size 0 max 16384 2024-03-21T11:23:49.783 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:23:49.783 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:23:49.783 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "51227099-0d5f-451b-a7f5-dad352ff9ab5", 2024-03-21T11:23:49.783 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:23:49.783 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:23:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.780+0000 7f16ce800640 1 -- 172.21.15.47:0/1654138044 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f16ac0a2ff0 msgr2=0x7f16ac0a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.780+0000 7f16ce800640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f16ac0a2ff0 0x7f16ac0a54b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x562c2238d2f0 tx=0x7f16c0009500 comp rx=0 tx=0).stop 2024-03-21T11:23:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.780+0000 7f16ce800640 1 -- 172.21.15.47:0/1654138044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16c8100910 msgr2=0x7f16c819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.780+0000 7f16ce800640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16c8100910 0x7f16c819a1a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f16b400e9f0 tx=0x7f16b400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.780+0000 7f16ce800640 1 -- 172.21.15.47:0/1654138044 shutdown_connections 2024-03-21T11:23:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.780+0000 7f16ce800640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f16ac0a2ff0 0x7f16ac0a54b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.780+0000 7f16ce800640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16c81012b0 0x7f16c81a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.780+0000 7f16ce800640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16c8100910 0x7f16c819a1a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.780+0000 7f16ce800640 1 --2- 172.21.15.47:0/1654138044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16c80fffb0 0x7f16c8199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.780+0000 7f16ce800640 1 -- 172.21.15.47:0/1654138044 >> 172.21.15.47:0/1654138044 conn(0x7f16c80fbb50 msgr2=0x7f16c80fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.781+0000 7f16ce800640 1 -- 172.21.15.47:0/1654138044 shutdown_connections 2024-03-21T11:23:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:49.781+0000 7f16ce800640 1 -- 172.21.15.47:0/1654138044 wait complete. 2024-03-21T11:23:49.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:23:49.799 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:23:49.801 INFO:tasks.workunit.client.0.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-21T11:23:50.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.007+0000 7f458e200640 1 -- 172.21.15.47:0/3845185524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4588102790 msgr2=0x7f4588102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:50.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.007+0000 7f458e200640 1 --2- 172.21.15.47:0/3845185524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4588102790 0x7f4588102c10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4574009900 tx=0x7f457402bc30 comp rx=0 tx=0).stop 2024-03-21T11:23:50.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.008+0000 7f458e200640 1 -- 172.21.15.47:0/3845185524 shutdown_connections 2024-03-21T11:23:50.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.008+0000 7f458e200640 1 --2- 172.21.15.47:0/3845185524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4588103150 0x7f45881099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.008+0000 7f458e200640 1 --2- 172.21.15.47:0/3845185524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4588102790 0x7f4588102c10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.008+0000 7f458e200640 1 --2- 172.21.15.47:0/3845185524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45881015a0 0x7f45881019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.008+0000 7f458e200640 1 -- 172.21.15.47:0/3845185524 >> 172.21.15.47:0/3845185524 conn(0x7f45880fcb10 msgr2=0x7f45880fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:50.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.008+0000 7f458e200640 1 -- 172.21.15.47:0/3845185524 shutdown_connections 2024-03-21T11:23:50.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.008+0000 7f458e200640 1 -- 172.21.15.47:0/3845185524 wait complete. 2024-03-21T11:23:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.009+0000 7f458e200640 1 Processor -- start 2024-03-21T11:23:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.009+0000 7f458e200640 1 -- start start 2024-03-21T11:23:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.009+0000 7f458e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45881015a0 0x7f4588199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.009+0000 7f458e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4588102790 0x7f4588199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.009+0000 7f458e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4588103150 0x7f45881a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.009+0000 7f458e200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458810d250 con 0x7f4588103150 2024-03-21T11:23:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.009+0000 7f458e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458810d0d0 con 0x7f4588102790 2024-03-21T11:23:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.009+0000 7f458e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458810d3d0 con 0x7f45881015a0 2024-03-21T11:23:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.010+0000 7f458ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45881015a0 0x7f4588199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.010+0000 7f458ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45881015a0 0x7f4588199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36514/0 (socket says 172.21.15.47:36514) 2024-03-21T11:23:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.010+0000 7f458ce00640 1 -- 172.21.15.47:0/3590619540 learned_addr learned my addr 172.21.15.47:0/3590619540 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.010+0000 7f4587e00640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4588102790 0x7f4588199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.010+0000 7f458d800640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4588103150 0x7f45881a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.010+0000 7f458ce00640 1 -- 172.21.15.47:0/3590619540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4588102790 msgr2=0x7f4588199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.010+0000 7f458ce00640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4588102790 0x7f4588199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.010+0000 7f458ce00640 1 -- 172.21.15.47:0/3590619540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4588103150 msgr2=0x7f45881a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.010+0000 7f458ce00640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4588103150 0x7f45881a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.010+0000 7f458ce00640 1 -- 172.21.15.47:0/3590619540 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45881a16a0 con 0x7f45881015a0 2024-03-21T11:23:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.010+0000 7f458d800640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4588103150 0x7f45881a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:23:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.010+0000 7f458ce00640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45881015a0 0x7f4588199a80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f457c00e9c0 tx=0x7f457c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.011+0000 7f4585600640 1 -- 172.21.15.47:0/3590619540 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c00cd80 con 0x7f45881015a0 2024-03-21T11:23:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.011+0000 7f4585600640 1 -- 172.21.15.47:0/3590619540 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f457c0035f0 con 0x7f45881015a0 2024-03-21T11:23:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.011+0000 7f4585600640 1 -- 172.21.15.47:0/3590619540 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c010690 con 0x7f45881015a0 2024-03-21T11:23:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.011+0000 7f458e200640 1 -- 172.21.15.47:0/3590619540 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45881a1990 con 0x7f45881015a0 2024-03-21T11:23:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.011+0000 7f458e200640 1 -- 172.21.15.47:0/3590619540 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45881a1f00 con 0x7f45881015a0 2024-03-21T11:23:50.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.014+0000 7f45925d2740 1 Processor -- start 2024-03-21T11:23:50.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.014+0000 7f4585600640 1 -- 172.21.15.47:0/3590619540 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f457c0037a0 con 0x7f45881015a0 2024-03-21T11:23:50.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.014+0000 7f45925d2740 1 -- start start 2024-03-21T11:23:50.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f45925d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf786470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f45925d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786a40 0x563edf78b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f45925d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf78bbb0 0x563edf78e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f45925d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edf6eb250 con 0x563edf78bbb0 2024-03-21T11:23:50.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f45925d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edf693a30 con 0x563edf786050 2024-03-21T11:23:50.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f45925d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edf6e72f0 con 0x563edf786a40 2024-03-21T11:23:50.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f458d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf786470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f458d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf786470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53760/0 (socket says 172.21.15.47:53760) 2024-03-21T11:23:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f458d800640 1 -- 172.21.15.47:0/1188557811 learned_addr learned my addr 172.21.15.47:0/1188557811 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f458ce00640 1 --2- 172.21.15.47:0/1188557811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786a40 0x563edf78b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f4587e00640 1 --2- 172.21.15.47:0/1188557811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf78bbb0 0x563edf78e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f458d800640 1 -- 172.21.15.47:0/1188557811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786a40 msgr2=0x563edf78b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f458d800640 1 --2- 172.21.15.47:0/1188557811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786a40 0x563edf78b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f458d800640 1 -- 172.21.15.47:0/1188557811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf78bbb0 msgr2=0x563edf78e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f458d800640 1 --2- 172.21.15.47:0/1188557811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf78bbb0 0x563edf78e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f458d800640 1 -- 172.21.15.47:0/1188557811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563edf67e690 con 0x563edf786050 2024-03-21T11:23:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.015+0000 7f4587e00640 1 --2- 172.21.15.47:0/1188557811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf78bbb0 0x563edf78e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:23:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.016+0000 7f458d800640 1 --2- 172.21.15.47:0/1188557811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf786470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4580009f90 tx=0x7f458000f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eceee2fcbb5d851 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.016+0000 7f457a000640 1 -- 172.21.15.47:0/1188557811 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580003b70 con 0x563edf786050 2024-03-21T11:23:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.016+0000 7f4585600640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f45700a3030 0x7f45700a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.016+0000 7f457a000640 1 -- 172.21.15.47:0/1188557811 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4580003d10 con 0x563edf786050 2024-03-21T11:23:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.016+0000 7f457a000640 1 -- 172.21.15.47:0/1188557811 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580011a30 con 0x563edf786050 2024-03-21T11:23:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.016+0000 7f4585600640 1 -- 172.21.15.47:0/3590619540 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f457c0cfd90 con 0x7f45881015a0 2024-03-21T11:23:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.016+0000 7f458ce00640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f45700a3030 0x7f45700a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.017+0000 7f45925d2740 1 -- 172.21.15.47:0/1188557811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 msgr2=0x563edf786470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.017+0000 7f45925d2740 1 --2- 172.21.15.47:0/1188557811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf786470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4580009f90 tx=0x7f458000f7f0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.017+0000 7f45925d2740 1 -- 172.21.15.47:0/1188557811 shutdown_connections 2024-03-21T11:23:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.017+0000 7f45925d2740 1 --2- 172.21.15.47:0/1188557811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf78bbb0 0x563edf78e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.017+0000 7f45925d2740 1 --2- 172.21.15.47:0/1188557811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786a40 0x563edf78b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.017+0000 7f45925d2740 1 --2- 172.21.15.47:0/1188557811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf786470 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.017+0000 7f45925d2740 1 -- 172.21.15.47:0/1188557811 >> 172.21.15.47:0/1188557811 conn(0x563edf74e200 msgr2=0x563edf7819c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.017+0000 7f458ce00640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f45700a3030 0x7f45700a54f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x563edf786e40 tx=0x7f457c0d3840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:50.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.017+0000 7f45925d2740 1 -- 172.21.15.47:0/1188557811 shutdown_connections 2024-03-21T11:23:50.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.017+0000 7f45925d2740 1 -- 172.21.15.47:0/1188557811 wait complete. 2024-03-21T11:23:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.018+0000 7f45925d2740 1 -- start start 2024-03-21T11:23:50.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.018+0000 7f45925d2740 1 Processor -- start 2024-03-21T11:23:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.018+0000 7f45925d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf790020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.018+0000 7f45925d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786a40 0x563edf790560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.018+0000 7f45925d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf78bbb0 0x563edf7975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.018+0000 7f45925d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edf78eb60 con 0x563edf78bbb0 2024-03-21T11:23:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.018+0000 7f45925d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edf6a6e80 con 0x563edf786050 2024-03-21T11:23:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.018+0000 7f45925d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edf78ece0 con 0x563edf786a40 2024-03-21T11:23:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.018+0000 7f45925d2740 10 client.0 _ll_register_callbacks cb 0x563edf6b9130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:23:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.019+0000 7f4587e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786a40 0x563edf790560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.019+0000 7f458d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf790020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.019+0000 7f458d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf790020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53770/0 (socket says 172.21.15.47:53770) 2024-03-21T11:23:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.019+0000 7f458d800640 1 -- 172.21.15.47:0/3318186656 learned_addr learned my addr 172.21.15.47:0/3318186656 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:50.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.019+0000 7f4587e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786a40 0x563edf790560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36538/0 (socket says 172.21.15.47:36538) 2024-03-21T11:23:50.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.019+0000 7f458d800640 1 --2- 172.21.15.47:0/3318186656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf78bbb0 0x563edf7975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.019+0000 7f4587e00640 1 -- 172.21.15.47:0/3318186656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 msgr2=0x563edf790020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:50.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.019+0000 7f4587e00640 1 --2- 172.21.15.47:0/3318186656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf790020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.019+0000 7f4587e00640 1 -- 172.21.15.47:0/3318186656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf78bbb0 msgr2=0x563edf7975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:50.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.019+0000 7f4587e00640 1 --2- 172.21.15.47:0/3318186656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf78bbb0 0x563edf7975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.019+0000 7f4587e00640 1 -- 172.21.15.47:0/3318186656 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563edf4c97d0 con 0x563edf786a40 2024-03-21T11:23:50.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.020+0000 7f4587e00640 1 --2- 172.21.15.47:0/3318186656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786a40 0x563edf790560 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4574036040 tx=0x7f4574004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aade3187179ee12 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:50.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.020+0000 7f456ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:23:50.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.020+0000 7f456ea00640 1 -- 172.21.15.47:0/3318186656 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574005d50 con 0x563edf786a40 2024-03-21T11:23:50.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.020+0000 7f45925d2740 1 -- 172.21.15.47:0/3318186656 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4580003d10 con 0x563edf786a40 2024-03-21T11:23:50.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.020+0000 7f456ea00640 1 -- 172.21.15.47:0/3318186656 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f45740046c0 con 0x563edf786a40 2024-03-21T11:23:50.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.020+0000 7f456ea00640 1 -- 172.21.15.47:0/3318186656 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4574052050 con 0x563edf786a40 2024-03-21T11:23:50.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.020+0000 7f45925d2740 10 client.25633 fetch_fsmap learned FSMap version 28 2024-03-21T11:23:50.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.020+0000 7f45925d2740 1 -- 172.21.15.47:0/3318186656 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563edf3ff3f0 con 0x563edf786a40 2024-03-21T11:23:50.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.020+0000 7f456ea00640 1 -- 172.21.15.47:0/3318186656 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4580003d10 con 0x563edf786a40 2024-03-21T11:23:50.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.021+0000 7f456ea00640 1 -- 172.21.15.47:0/3318186656 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4574005460 con 0x563edf786a40 2024-03-21T11:23:50.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.021+0000 7f45925d2740 10 client.25633 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:23:50.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.021+0000 7f45925d2740 10 client.25633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:23:50.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.021+0000 7f45925d2740 20 client.25633 populate_metadata read hostname 'smithi047' 2024-03-21T11:23:50.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.021+0000 7f45925d2740 1 --2- 172.21.15.47:0/3318186656 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563edf79bd90 0x563edf79e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.021+0000 7f45925d2740 4 client.25633 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:23:50.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.021+0000 7f45925d2740 1 -- 172.21.15.47:0/3318186656 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563edf6f8790 con 0x563edf79bd90 2024-03-21T11:23:50.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.021+0000 7f458d800640 1 --2- 172.21.15.47:0/3318186656 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563edf79bd90 0x563edf79e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.022+0000 7f458d800640 1 --2- 172.21.15.47:0/3318186656 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563edf79bd90 0x563edf79e1b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1a70b7c352c78d9 server_cookie=758e0b09aab8db5 in_seq=0 out_seq=0 2024-03-21T11:23:50.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.022+0000 7f456ea00640 10 client.25633 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:23:50.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.022+0000 7f456ea00640 1 -- 172.21.15.47:0/3318186656 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563edf6f8790 con 0x563edf79bd90 2024-03-21T11:23:50.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.022+0000 7f456ea00640 10 client.25633 handle_command_reply: tid=0 2024-03-21T11:23:50.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.022+0000 7f45925d2740 1 client.25633 shutdown 2024-03-21T11:23:50.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.023+0000 7f45925d2740 1 -- 172.21.15.47:0/3318186656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786a40 msgr2=0x563edf790560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:50.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.023+0000 7f45925d2740 1 --2- 172.21.15.47:0/3318186656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786a40 0x563edf790560 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4574036040 tx=0x7f4574004250 comp rx=0 tx=0).stop 2024-03-21T11:23:50.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.023+0000 7f45925d2740 1 -- 172.21.15.47:0/3318186656 shutdown_connections 2024-03-21T11:23:50.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.023+0000 7f45925d2740 1 --2- 172.21.15.47:0/3318186656 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563edf79bd90 0x563edf79e1b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.023+0000 7f45925d2740 1 --2- 172.21.15.47:0/3318186656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf78bbb0 0x563edf7975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.023+0000 7f45925d2740 1 --2- 172.21.15.47:0/3318186656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786a40 0x563edf790560 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.023+0000 7f45925d2740 1 --2- 172.21.15.47:0/3318186656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf790020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.023+0000 7f45925d2740 1 -- 172.21.15.47:0/3318186656 >> 172.21.15.47:0/3318186656 conn(0x563edf74e200 msgr2=0x563edf780bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:50.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.023+0000 7f456ea00640 0 client.25633 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:23:50.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.023+0000 7f45925d2740 1 -- 172.21.15.47:0/3318186656 shutdown_connections 2024-03-21T11:23:50.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.023+0000 7f45925d2740 1 -- 172.21.15.47:0/3318186656 wait complete. 2024-03-21T11:23:50.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.023+0000 7f45925d2740 20 client.25633 trim_cache size 0 max 16384 2024-03-21T11:23:50.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.042+0000 7f45925d2740 1 Processor -- start 2024-03-21T11:23:50.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.042+0000 7f45925d2740 1 -- start start 2024-03-21T11:23:50.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.042+0000 7f45925d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786050 0x563edf790020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.042+0000 7f45925d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf786a40 0x563edf797560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.042+0000 7f45925d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf78bbb0 0x563edf780a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.042+0000 7f45925d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45881ac0b0 con 0x563edf786a40 2024-03-21T11:23:50.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.042+0000 7f45925d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45881ada30 con 0x563edf78bbb0 2024-03-21T11:23:50.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.043+0000 7f45925d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45881ac710 con 0x563edf786050 2024-03-21T11:23:50.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.043+0000 7f4587e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf786a40 0x563edf797560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.043+0000 7f458d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786050 0x563edf790020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.043+0000 7f4587e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf786a40 0x563edf797560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45268/0 (socket says 172.21.15.47:45268) 2024-03-21T11:23:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.043+0000 7f4587e00640 1 -- 172.21.15.47:0/2192371703 learned_addr learned my addr 172.21.15.47:0/2192371703 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.043+0000 7f458d800640 1 --2- 172.21.15.47:0/2192371703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf78bbb0 0x563edf780a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.043+0000 7f4587e00640 1 -- 172.21.15.47:0/2192371703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786050 msgr2=0x563edf790020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.043+0000 7f4587e00640 1 --2- 172.21.15.47:0/2192371703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786050 0x563edf790020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.043+0000 7f4587e00640 1 -- 172.21.15.47:0/2192371703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf78bbb0 msgr2=0x563edf780a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.043+0000 7f4587e00640 1 --2- 172.21.15.47:0/2192371703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf78bbb0 0x563edf780a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.043+0000 7f4587e00640 1 -- 172.21.15.47:0/2192371703 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563edf3bf0f0 con 0x563edf786a40 2024-03-21T11:23:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.043+0000 7f458d800640 1 --2- 172.21.15.47:0/2192371703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786050 0x563edf790020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:23:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.043+0000 7f4587e00640 1 --2- 172.21.15.47:0/2192371703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf786a40 0x563edf797560 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4574003640 tx=0x7f457402cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34288d0dd3e7a8d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.044+0000 7f4579600640 1 -- 172.21.15.47:0/2192371703 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457404f040 con 0x563edf786a40 2024-03-21T11:23:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.044+0000 7f4579600640 1 -- 172.21.15.47:0/2192371703 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4574009780 con 0x563edf786a40 2024-03-21T11:23:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.044+0000 7f4579600640 1 -- 172.21.15.47:0/2192371703 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574004eb0 con 0x563edf786a40 2024-03-21T11:23:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.044+0000 7f45925d2740 1 -- 172.21.15.47:0/2192371703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf786a40 msgr2=0x563edf797560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.044+0000 7f45925d2740 1 --2- 172.21.15.47:0/2192371703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf786a40 0x563edf797560 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4574003640 tx=0x7f457402cdd0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.044+0000 7f45925d2740 1 -- 172.21.15.47:0/2192371703 shutdown_connections 2024-03-21T11:23:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.044+0000 7f45925d2740 1 --2- 172.21.15.47:0/2192371703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf78bbb0 0x563edf780a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.044+0000 7f45925d2740 1 --2- 172.21.15.47:0/2192371703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf786a40 0x563edf797560 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.044+0000 7f45925d2740 1 --2- 172.21.15.47:0/2192371703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf786050 0x563edf790020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.044+0000 7f45925d2740 1 -- 172.21.15.47:0/2192371703 >> 172.21.15.47:0/2192371703 conn(0x563edf74e200 msgr2=0x563edf780330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.045+0000 7f45925d2740 1 -- 172.21.15.47:0/2192371703 shutdown_connections 2024-03-21T11:23:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.045+0000 7f45925d2740 1 -- 172.21.15.47:0/2192371703 wait complete. 2024-03-21T11:23:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.045+0000 7f45925d2740 1 -- start start 2024-03-21T11:23:50.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.045+0000 7f45925d2740 1 Processor -- start 2024-03-21T11:23:50.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f45925d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf795400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f45925d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf786a40 0x563edf795940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f45925d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf78bbb0 0x563edf796950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f45925d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45881ad3d0 con 0x563edf786a40 2024-03-21T11:23:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f45925d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45881acd70 con 0x563edf786050 2024-03-21T11:23:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f45925d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45881aa270 con 0x563edf78bbb0 2024-03-21T11:23:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f45925d2740 10 client.0 _ll_register_callbacks cb 0x7f457c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:23:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f458d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf795400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f458d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf795400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53786/0 (socket says 172.21.15.47:53786) 2024-03-21T11:23:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f458d800640 1 -- 172.21.15.47:0/1610124674 learned_addr learned my addr 172.21.15.47:0/1610124674 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:23:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f458d800640 1 --2- 172.21.15.47:0/1610124674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf78bbb0 0x563edf796950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f4587e00640 1 --2- 172.21.15.47:0/1610124674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf786a40 0x563edf795940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f458d800640 1 -- 172.21.15.47:0/1610124674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf78bbb0 msgr2=0x563edf796950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f458d800640 1 --2- 172.21.15.47:0/1610124674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf78bbb0 0x563edf796950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f458d800640 1 -- 172.21.15.47:0/1610124674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf786a40 msgr2=0x563edf795940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:23:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f458d800640 1 --2- 172.21.15.47:0/1610124674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf786a40 0x563edf795940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.046+0000 7f458d800640 1 -- 172.21.15.47:0/1610124674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563edf796f40 con 0x563edf786050 2024-03-21T11:23:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.047+0000 7f458d800640 1 --2- 172.21.15.47:0/1610124674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf795400 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f458002e480 tx=0x7f457402cd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5014fccde3a73a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:23:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.047+0000 7f456ea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:23:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.047+0000 7f456ea00640 1 -- 172.21.15.47:0/1610124674 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580024040 con 0x563edf786050 2024-03-21T11:23:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.047+0000 7f45925d2740 1 -- 172.21.15.47:0/1610124674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4574009780 con 0x563edf786050 2024-03-21T11:23:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.047+0000 7f456ea00640 1 -- 172.21.15.47:0/1610124674 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4580004bf0 con 0x563edf786050 2024-03-21T11:23:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.047+0000 7f456ea00640 1 -- 172.21.15.47:0/1610124674 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4580042050 con 0x563edf786050 2024-03-21T11:23:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.047+0000 7f45925d2740 10 client.25622 fetch_fsmap learned FSMap version 28 2024-03-21T11:23:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.047+0000 7f45925d2740 1 -- 172.21.15.47:0/1610124674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563edf7f7df0 con 0x563edf786050 2024-03-21T11:23:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.048+0000 7f456ea00640 1 -- 172.21.15.47:0/1610124674 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4574009780 con 0x563edf786050 2024-03-21T11:23:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.048+0000 7f456ea00640 1 -- 172.21.15.47:0/1610124674 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f458002a040 con 0x563edf786050 2024-03-21T11:23:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.048+0000 7f45925d2740 10 client.25622 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:23:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.048+0000 7f45925d2740 10 client.25622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:23:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.048+0000 7f45925d2740 20 client.25622 populate_metadata read hostname 'smithi047' 2024-03-21T11:23:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.048+0000 7f45925d2740 1 --2- 172.21.15.47:0/1610124674 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563edf7f8590 0x563edf7fa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:23:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.048+0000 7f45925d2740 4 client.25622 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:23:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.048+0000 7f45925d2740 1 -- 172.21.15.47:0/1610124674 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563edf6f2ae0 con 0x563edf7f8590 2024-03-21T11:23:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.048+0000 7f4587e00640 1 --2- 172.21.15.47:0/1610124674 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563edf7f8590 0x563edf7fa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:23:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f4587e00640 1 --2- 172.21.15.47:0/1610124674 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563edf7f8590 0x563edf7fa9b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4592a39e261e73a2 server_cookie=f56195835dbb5059 in_seq=0 out_seq=0 2024-03-21T11:23:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f456ea00640 10 client.25622 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:23:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f456ea00640 1 -- 172.21.15.47:0/1610124674 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563edf6f2ae0 con 0x563edf7f8590 2024-03-21T11:23:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f456ea00640 10 client.25622 handle_command_reply: tid=0 2024-03-21T11:23:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f45925d2740 1 client.25622 shutdown 2024-03-21T11:23:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f45925d2740 1 -- 172.21.15.47:0/1610124674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 msgr2=0x563edf795400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f45925d2740 1 --2- 172.21.15.47:0/1610124674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf795400 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f458002e480 tx=0x7f457402cd60 comp rx=0 tx=0).stop 2024-03-21T11:23:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f45925d2740 1 -- 172.21.15.47:0/1610124674 shutdown_connections 2024-03-21T11:23:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f45925d2740 1 --2- 172.21.15.47:0/1610124674 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563edf7f8590 0x563edf7fa9b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f456ea00640 0 client.25622 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:23:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f45925d2740 1 --2- 172.21.15.47:0/1610124674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563edf78bbb0 0x563edf796950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f45925d2740 1 --2- 172.21.15.47:0/1610124674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563edf786a40 0x563edf795940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f45925d2740 1 --2- 172.21.15.47:0/1610124674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563edf786050 0x563edf795400 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f45925d2740 1 -- 172.21.15.47:0/1610124674 >> 172.21.15.47:0/1610124674 conn(0x563edf74e200 msgr2=0x563edf780330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f45925d2740 1 -- 172.21.15.47:0/1610124674 shutdown_connections 2024-03-21T11:23:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.049+0000 7f45925d2740 1 -- 172.21.15.47:0/1610124674 wait complete. 2024-03-21T11:23:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.050+0000 7f45925d2740 20 client.25622 trim_cache size 0 max 16384 2024-03-21T11:23:50.057 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:23:50.057 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:23:50.057 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:23:50.057 INFO:teuthology.orchestra.run.smithi047.stdout: "51227099-0d5f-451b-a7f5-dad352ff9ab5": { 2024-03-21T11:23:50.057 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:23:50.057 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "51227099-0d5f-451b-a7f5-dad352ff9ab5", 2024-03-21T11:23:50.057 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:23:50.057 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:23:50.057 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:23:50.057 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:23:50.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.050+0000 7f458e200640 1 -- 172.21.15.47:0/3590619540 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f45700a3030 msgr2=0x7f45700a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:50.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.050+0000 7f458e200640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f45700a3030 0x7f45700a54f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x563edf786e40 tx=0x7f457c0d3840 comp rx=0 tx=0).stop 2024-03-21T11:23:50.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.050+0000 7f458e200640 1 -- 172.21.15.47:0/3590619540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45881015a0 msgr2=0x7f4588199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:23:50.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.050+0000 7f458e200640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45881015a0 0x7f4588199a80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f457c00e9c0 tx=0x7f457c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:23:50.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.050+0000 7f458e200640 1 -- 172.21.15.47:0/3590619540 shutdown_connections 2024-03-21T11:23:50.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.050+0000 7f458e200640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f45700a3030 0x7f45700a54f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.050+0000 7f458e200640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4588103150 0x7f45881a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.050+0000 7f458e200640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4588102790 0x7f4588199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.050+0000 7f458e200640 1 --2- 172.21.15.47:0/3590619540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45881015a0 0x7f4588199a80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:23:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.050+0000 7f458e200640 1 -- 172.21.15.47:0/3590619540 >> 172.21.15.47:0/3590619540 conn(0x7f45880fcb10 msgr2=0x7f45880738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:23:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.051+0000 7f458e200640 1 -- 172.21.15.47:0/3590619540 shutdown_connections 2024-03-21T11:23:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:23:50.051+0000 7f458e200640 1 -- 172.21.15.47:0/3590619540 wait complete. 2024-03-21T11:23:50.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:49 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/203760074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:23:50.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:49 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/4212070885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:50.071 INFO:tasks.cephfs.filesystem:scrub status for tag:51227099-0d5f-451b-a7f5-dad352ff9ab5 - {'path': '/', 'tag': '51227099-0d5f-451b-a7f5-dad352ff9ab5', 'options': 'recursive,force'} 2024-03-21T11:23:50.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:49 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/203760074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:23:50.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:49 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/4212070885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:50.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:49 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/203760074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:23:50.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:49 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/4212070885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:23:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:50 smithi100 ceph-mon[36245]: pgmap v415: 129 pgs: 129 active+clean; 657 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-03-21T11:23:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:50 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:23:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:50 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:23:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:50 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:23:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:50 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:23:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:50 smithi102 ceph-mon[36748]: pgmap v415: 129 pgs: 129 active+clean; 657 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-03-21T11:23:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:50 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:23:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:50 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:23:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:50 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:23:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:50 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:23:51.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:50 smithi047 ceph-mon[31351]: pgmap v415: 129 pgs: 129 active+clean; 657 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-03-21T11:23:51.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:50 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:23:51.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:50 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:23:51.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:50 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:23:51.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:50 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:23:53.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:52 smithi100 ceph-mon[36245]: pgmap v416: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-03-21T11:23:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:52 smithi102 ceph-mon[36748]: pgmap v416: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-03-21T11:23:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:52 smithi047 ceph-mon[31351]: pgmap v416: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-03-21T11:23:55.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:54 smithi100 ceph-mon[36245]: pgmap v417: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-21T11:23:55.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:54 smithi102 ceph-mon[36748]: pgmap v417: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-21T11:23:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:54 smithi047 ceph-mon[31351]: pgmap v417: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-21T11:23:57.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:56 smithi100 ceph-mon[36245]: pgmap v418: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T11:23:57.240 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:56 smithi102 ceph-mon[36748]: pgmap v418: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T11:23:57.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:56 smithi047 ceph-mon[31351]: pgmap v418: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T11:23:59.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:23:58 smithi100 ceph-mon[36245]: pgmap v419: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-21T11:23:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:23:58 smithi102 ceph-mon[36748]: pgmap v419: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-21T11:23:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:23:58 smithi047 ceph-mon[31351]: pgmap v419: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-21T11:24:01.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:00 smithi100 ceph-mon[36245]: pgmap v420: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-21T11:24:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:00 smithi102 ceph-mon[36748]: pgmap v420: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-21T11:24:01.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:00 smithi047 ceph-mon[31351]: pgmap v420: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-21T11:24:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.791+0000 7f9052600640 1 -- 172.21.15.47:0/1380281311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c1021b0 msgr2=0x7f904c1045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.791+0000 7f9052600640 1 --2- 172.21.15.47:0/1380281311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c1021b0 0x7f904c1045a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f903c009ca0 tx=0x7f903c02c0f0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.791+0000 7f9052600640 1 -- 172.21.15.47:0/1380281311 shutdown_connections 2024-03-21T11:24:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.791+0000 7f9052600640 1 --2- 172.21.15.47:0/1380281311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f904c107410 0x7f904c109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.791+0000 7f9052600640 1 --2- 172.21.15.47:0/1380281311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f904c104ae0 0x7f904c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.791+0000 7f9052600640 1 --2- 172.21.15.47:0/1380281311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c1021b0 0x7f904c1045a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.791+0000 7f9052600640 1 -- 172.21.15.47:0/1380281311 >> 172.21.15.47:0/1380281311 conn(0x7f904c0fbba0 msgr2=0x7f904c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.791+0000 7f9052600640 1 -- 172.21.15.47:0/1380281311 shutdown_connections 2024-03-21T11:24:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.791+0000 7f9052600640 1 -- 172.21.15.47:0/1380281311 wait complete. 2024-03-21T11:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.792+0000 7f9052600640 1 Processor -- start 2024-03-21T11:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.793+0000 7f9052600640 1 -- start start 2024-03-21T11:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.793+0000 7f9052600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f904c1021b0 0x7f904c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.793+0000 7f9052600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c104ae0 0x7f904c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.793+0000 7f9052600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f904c107410 0x7f904c1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.793+0000 7f9052600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c10d360 con 0x7f904c107410 2024-03-21T11:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.793+0000 7f9052600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c10d1e0 con 0x7f904c104ae0 2024-03-21T11:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.793+0000 7f9052600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c10d4e0 con 0x7f904c1021b0 2024-03-21T11:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.793+0000 7f904b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f904c1021b0 0x7f904c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.793+0000 7f9042a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c104ae0 0x7f904c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.793+0000 7f904b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f904c1021b0 0x7f904c199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49044/0 (socket says 172.21.15.47:49044) 2024-03-21T11:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.793+0000 7f904b400640 1 -- 172.21.15.47:0/461087202 learned_addr learned my addr 172.21.15.47:0/461087202 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.793+0000 7f904be00640 1 --2- 172.21.15.47:0/461087202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f904c107410 0x7f904c1a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.794+0000 7f904b400640 1 -- 172.21.15.47:0/461087202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c104ae0 msgr2=0x7f904c19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.794+0000 7f904b400640 1 --2- 172.21.15.47:0/461087202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c104ae0 0x7f904c19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.794+0000 7f904b400640 1 -- 172.21.15.47:0/461087202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f904c107410 msgr2=0x7f904c1a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.794+0000 7f904b400640 1 --2- 172.21.15.47:0/461087202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f904c107410 0x7f904c1a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.794+0000 7f904b400640 1 -- 172.21.15.47:0/461087202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c1a1750 con 0x7f904c1021b0 2024-03-21T11:24:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.794+0000 7f904b400640 1 --2- 172.21.15.47:0/461087202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f904c1021b0 0x7f904c199b30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f903c0325f0 tx=0x7f903c032ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.794+0000 7f9048c00640 1 -- 172.21.15.47:0/461087202 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c041020 con 0x7f904c1021b0 2024-03-21T11:24:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.794+0000 7f9048c00640 1 -- 172.21.15.47:0/461087202 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f903c034cd0 con 0x7f904c1021b0 2024-03-21T11:24:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.794+0000 7f9048c00640 1 -- 172.21.15.47:0/461087202 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c039040 con 0x7f904c1021b0 2024-03-21T11:24:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.794+0000 7f9052600640 1 -- 172.21.15.47:0/461087202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f904c1a19e0 con 0x7f904c1021b0 2024-03-21T11:24:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.795+0000 7f9052600640 1 -- 172.21.15.47:0/461087202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904c1a1ef0 con 0x7f904c1021b0 2024-03-21T11:24:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.798+0000 7f9048c00640 1 -- 172.21.15.47:0/461087202 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f903c044020 con 0x7f904c1021b0 2024-03-21T11:24:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.798+0000 7f90553c6740 1 Processor -- start 2024-03-21T11:24:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.798+0000 7f90553c6740 1 -- start start 2024-03-21T11:24:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.798+0000 7f90553c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 0x56159907b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.798+0000 7f90553c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56159907bbd0 0x5615990806d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.798+0000 7f90553c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561599080d40 0x5615990831d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.798+0000 7f90553c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159903e3b0 con 0x56159907b1e0 2024-03-21T11:24:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.798+0000 7f90553c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561598fe0830 con 0x561599080d40 2024-03-21T11:24:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.798+0000 7f90553c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561598f895d0 con 0x56159907bbd0 2024-03-21T11:24:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.799+0000 7f904b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56159907bbd0 0x5615990806d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.799+0000 7f904be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 0x56159907b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.799+0000 7f904b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56159907bbd0 0x5615990806d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49052/0 (socket says 172.21.15.47:49052) 2024-03-21T11:24:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.799+0000 7f904be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 0x56159907b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32842/0 (socket says 172.21.15.47:32842) 2024-03-21T11:24:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.799+0000 7f9042a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561599080d40 0x5615990831d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.799+0000 7f904b400640 1 -- 172.21.15.47:0/1442097418 learned_addr learned my addr 172.21.15.47:0/1442097418 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.799+0000 7f904b400640 1 -- 172.21.15.47:0/1442097418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561599080d40 msgr2=0x5615990831d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.799+0000 7f904b400640 1 --2- 172.21.15.47:0/1442097418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561599080d40 0x5615990831d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.799+0000 7f904b400640 1 -- 172.21.15.47:0/1442097418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 msgr2=0x56159907b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.799+0000 7f904b400640 1 --2- 172.21.15.47:0/1442097418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 0x56159907b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.799+0000 7f904b400640 1 -- 172.21.15.47:0/1442097418 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561598fa12c0 con 0x56159907bbd0 2024-03-21T11:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.800+0000 7f904b400640 1 --2- 172.21.15.47:0/1442097418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56159907bbd0 0x5615990806d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f903c0eec60 tx=0x7f903c0f7b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3e49a27dafa81d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.800+0000 7f9040c00640 1 -- 172.21.15.47:0/1442097418 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c100070 con 0x56159907bbd0 2024-03-21T11:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.800+0000 7f9040c00640 1 -- 172.21.15.47:0/1442097418 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f903c0f9d60 con 0x56159907bbd0 2024-03-21T11:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.800+0000 7f9040c00640 1 -- 172.21.15.47:0/1442097418 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c0f3820 con 0x56159907bbd0 2024-03-21T11:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.800+0000 7f9048c00640 1 --2- 172.21.15.47:0/461087202 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f90300a30e0 0x7f90300a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.800+0000 7f9048c00640 1 -- 172.21.15.47:0/461087202 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f903c0f15f0 con 0x7f904c1021b0 2024-03-21T11:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.800+0000 7f904be00640 1 --2- 172.21.15.47:0/461087202 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f90300a30e0 0x7f90300a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.800+0000 7f90553c6740 1 -- 172.21.15.47:0/1442097418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56159907bbd0 msgr2=0x5615990806d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.800+0000 7f90553c6740 1 --2- 172.21.15.47:0/1442097418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56159907bbd0 0x5615990806d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f903c0eec60 tx=0x7f903c0f7b20 comp rx=0 tx=0).stop 2024-03-21T11:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.801+0000 7f90553c6740 1 -- 172.21.15.47:0/1442097418 shutdown_connections 2024-03-21T11:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.801+0000 7f90553c6740 1 --2- 172.21.15.47:0/1442097418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561599080d40 0x5615990831d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.801+0000 7f90553c6740 1 --2- 172.21.15.47:0/1442097418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56159907bbd0 0x5615990806d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.801+0000 7f90553c6740 1 --2- 172.21.15.47:0/1442097418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 0x56159907b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.801+0000 7f90553c6740 1 -- 172.21.15.47:0/1442097418 >> 172.21.15.47:0/1442097418 conn(0x561599076750 msgr2=0x561599076b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.801+0000 7f90553c6740 1 -- 172.21.15.47:0/1442097418 shutdown_connections 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.801+0000 7f90553c6740 1 -- 172.21.15.47:0/1442097418 wait complete. 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.801+0000 7f904be00640 1 --2- 172.21.15.47:0/461087202 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f90300a30e0 0x7f90300a55a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x56159907c430 tx=0x7f9044005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.801+0000 7f90553c6740 1 -- start start 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.802+0000 7f90553c6740 1 Processor -- start 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.802+0000 7f90553c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 0x5615990850e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.802+0000 7f90553c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907bbd0 0x561599085620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.802+0000 7f90553c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561599080d40 0x56159908c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.802+0000 7f90553c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561598f9ca20 con 0x56159907b1e0 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.802+0000 7f90553c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561598fdc8d0 con 0x56159907bbd0 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.802+0000 7f90553c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561599083ab0 con 0x561599080d40 2024-03-21T11:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.802+0000 7f90553c6740 10 client.0 _ll_register_callbacks cb 0x561598fd91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.802+0000 7f904be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561599080d40 0x56159908c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.802+0000 7f904b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 0x5615990850e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.802+0000 7f9042a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907bbd0 0x561599085620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.802+0000 7f904be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561599080d40 0x56159908c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49068/0 (socket says 172.21.15.47:49068) 2024-03-21T11:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.802+0000 7f904be00640 1 -- 172.21.15.47:0/3930830684 learned_addr learned my addr 172.21.15.47:0/3930830684 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.803+0000 7f904b400640 1 -- 172.21.15.47:0/3930830684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561599080d40 msgr2=0x56159908c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.803+0000 7f904b400640 1 --2- 172.21.15.47:0/3930830684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561599080d40 0x56159908c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.803+0000 7f904b400640 1 -- 172.21.15.47:0/3930830684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907bbd0 msgr2=0x561599085620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.803+0000 7f904b400640 1 --2- 172.21.15.47:0/3930830684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907bbd0 0x561599085620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.803+0000 7f904b400640 1 -- 172.21.15.47:0/3930830684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561598cf5210 con 0x56159907b1e0 2024-03-21T11:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.803+0000 7f904b400640 1 --2- 172.21.15.47:0/3930830684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 0x5615990850e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f903c0bd8d0 tx=0x7f903c003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d6fb1ba0a56470 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.803+0000 7f9039600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:24:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.803+0000 7f9039600640 1 -- 172.21.15.47:0/3930830684 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c100070 con 0x56159907b1e0 2024-03-21T11:24:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.803+0000 7f9039600640 1 -- 172.21.15.47:0/3930830684 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f903c0f1310 con 0x56159907b1e0 2024-03-21T11:24:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.803+0000 7f90553c6740 1 -- 172.21.15.47:0/3930830684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f903c0f9d60 con 0x56159907b1e0 2024-03-21T11:24:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.804+0000 7f9039600640 1 -- 172.21.15.47:0/3930830684 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f903c0b9be0 con 0x56159907b1e0 2024-03-21T11:24:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.804+0000 7f90553c6740 10 client.15489 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.804+0000 7f90553c6740 1 -- 172.21.15.47:0/3930830684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561598dc1620 con 0x56159907b1e0 2024-03-21T11:24:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.804+0000 7f9039600640 1 -- 172.21.15.47:0/3930830684 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f903c0f9d60 con 0x56159907b1e0 2024-03-21T11:24:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.804+0000 7f9039600640 1 -- 172.21.15.47:0/3930830684 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f903c03c030 con 0x56159907b1e0 2024-03-21T11:24:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.804+0000 7f90553c6740 10 client.15489 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.804+0000 7f90553c6740 10 client.15489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.804+0000 7f90553c6740 20 client.15489 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.804+0000 7f90553c6740 1 --2- 172.21.15.47:0/3930830684 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561599090c10 0x561599093000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.804+0000 7f90553c6740 4 client.15489 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.804+0000 7f90553c6740 1 -- 172.21.15.47:0/3930830684 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561598fedd70 con 0x561599090c10 2024-03-21T11:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.804+0000 7f9042a00640 1 --2- 172.21.15.47:0/3930830684 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561599090c10 0x561599093000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.805+0000 7f9042a00640 1 --2- 172.21.15.47:0/3930830684 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561599090c10 0x561599093000 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3051d517d7b395cd server_cookie=4e881a2558ba167e in_seq=0 out_seq=0 2024-03-21T11:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.805+0000 7f9039600640 10 client.15489 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f9039600640 1 -- 172.21.15.47:0/3930830684 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561598fedd70 con 0x561599090c10 2024-03-21T11:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f9039600640 10 client.15489 handle_command_reply: tid=0 2024-03-21T11:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f90553c6740 1 client.15489 shutdown 2024-03-21T11:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f90553c6740 1 -- 172.21.15.47:0/3930830684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 msgr2=0x5615990850e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f90553c6740 1 --2- 172.21.15.47:0/3930830684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 0x5615990850e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f903c0bd8d0 tx=0x7f903c003e60 comp rx=0 tx=0).stop 2024-03-21T11:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f90553c6740 1 -- 172.21.15.47:0/3930830684 shutdown_connections 2024-03-21T11:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f90553c6740 1 --2- 172.21.15.47:0/3930830684 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561599090c10 0x561599093000 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f90553c6740 1 --2- 172.21.15.47:0/3930830684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561599080d40 0x56159908c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f90553c6740 1 --2- 172.21.15.47:0/3930830684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907bbd0 0x561599085620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f90553c6740 1 --2- 172.21.15.47:0/3930830684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 0x5615990850e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f9039600640 0 client.15489 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f90553c6740 1 -- 172.21.15.47:0/3930830684 >> 172.21.15.47:0/3930830684 conn(0x561599076750 msgr2=0x561599075950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f90553c6740 1 -- 172.21.15.47:0/3930830684 shutdown_connections 2024-03-21T11:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f90553c6740 1 -- 172.21.15.47:0/3930830684 wait complete. 2024-03-21T11:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.806+0000 7f90553c6740 20 client.15489 trim_cache size 0 max 16384 2024-03-21T11:24:02.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.822+0000 7f90553c6740 1 Processor -- start 2024-03-21T11:24:02.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.822+0000 7f90553c6740 1 -- start start 2024-03-21T11:24:02.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.822+0000 7f90553c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907b1e0 0x5615990799b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.822+0000 7f90553c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56159907bbd0 0x56159908c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.822+0000 7f90553c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561599080d40 0x561599079ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.822+0000 7f90553c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c1ac1a0 con 0x561599080d40 2024-03-21T11:24:02.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.822+0000 7f90553c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c1adb20 con 0x56159907b1e0 2024-03-21T11:24:02.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.822+0000 7f90553c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c1ac800 con 0x56159907bbd0 2024-03-21T11:24:02.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.822+0000 7f904b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907b1e0 0x5615990799b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.822+0000 7f9042a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56159907bbd0 0x56159908c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.822+0000 7f904b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907b1e0 0x5615990799b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55532/0 (socket says 172.21.15.47:55532) 2024-03-21T11:24:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.822+0000 7f904be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561599080d40 0x561599079ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.822+0000 7f904b400640 1 -- 172.21.15.47:0/2542906377 learned_addr learned my addr 172.21.15.47:0/2542906377 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.823+0000 7f9042a00640 1 -- 172.21.15.47:0/2542906377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907b1e0 msgr2=0x5615990799b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.823+0000 7f9042a00640 1 --2- 172.21.15.47:0/2542906377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907b1e0 0x5615990799b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.823+0000 7f9042a00640 1 -- 172.21.15.47:0/2542906377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561599080d40 msgr2=0x561599079ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.823+0000 7f9042a00640 1 --2- 172.21.15.47:0/2542906377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561599080d40 0x561599079ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.823+0000 7f9042a00640 1 -- 172.21.15.47:0/2542906377 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561599083b70 con 0x56159907bbd0 2024-03-21T11:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.823+0000 7f904be00640 1 --2- 172.21.15.47:0/2542906377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561599080d40 0x561599079ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.823+0000 7f9042a00640 1 --2- 172.21.15.47:0/2542906377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56159907bbd0 0x56159908c580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x561599093840 tx=0x7f903400b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7490428f03c42a25 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.823+0000 7f903be00640 1 -- 172.21.15.47:0/2542906377 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903400bcc0 con 0x56159907bbd0 2024-03-21T11:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.823+0000 7f903be00640 1 -- 172.21.15.47:0/2542906377 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9034026070 con 0x56159907bbd0 2024-03-21T11:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.823+0000 7f903be00640 1 -- 172.21.15.47:0/2542906377 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034011830 con 0x56159907bbd0 2024-03-21T11:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.824+0000 7f90553c6740 1 -- 172.21.15.47:0/2542906377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56159907bbd0 msgr2=0x56159908c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.824+0000 7f90553c6740 1 --2- 172.21.15.47:0/2542906377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56159907bbd0 0x56159908c580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x561599093840 tx=0x7f903400b390 comp rx=0 tx=0).stop 2024-03-21T11:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.824+0000 7f90553c6740 1 -- 172.21.15.47:0/2542906377 shutdown_connections 2024-03-21T11:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.824+0000 7f90553c6740 1 --2- 172.21.15.47:0/2542906377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561599080d40 0x561599079ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.824+0000 7f90553c6740 1 --2- 172.21.15.47:0/2542906377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56159907bbd0 0x56159908c580 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.824+0000 7f90553c6740 1 --2- 172.21.15.47:0/2542906377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907b1e0 0x5615990799b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.824+0000 7f90553c6740 1 -- 172.21.15.47:0/2542906377 >> 172.21.15.47:0/2542906377 conn(0x561599076750 msgr2=0x561599076b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.824+0000 7f90553c6740 1 -- 172.21.15.47:0/2542906377 shutdown_connections 2024-03-21T11:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.824+0000 7f90553c6740 1 -- 172.21.15.47:0/2542906377 wait complete. 2024-03-21T11:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f90553c6740 1 -- start start 2024-03-21T11:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f90553c6740 1 Processor -- start 2024-03-21T11:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f90553c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 0x561599089f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f90553c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907bbd0 0x56159908a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f90553c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561599080d40 0x56159908b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f90553c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c1ad4c0 con 0x56159907b1e0 2024-03-21T11:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f90553c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c1ace60 con 0x56159907bbd0 2024-03-21T11:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f90553c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c1aa360 con 0x561599080d40 2024-03-21T11:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f90553c6740 10 client.0 _ll_register_callbacks cb 0x7f9044009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f9042a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907bbd0 0x56159908a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f904be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561599080d40 0x56159908b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f9042a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907bbd0 0x56159908a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55538/0 (socket says 172.21.15.47:55538) 2024-03-21T11:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f904b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 0x561599089f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f904be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561599080d40 0x56159908b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49082/0 (socket says 172.21.15.47:49082) 2024-03-21T11:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.825+0000 7f9042a00640 1 -- 172.21.15.47:0/2132947531 learned_addr learned my addr 172.21.15.47:0/2132947531 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.826+0000 7f9042a00640 1 -- 172.21.15.47:0/2132947531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561599080d40 msgr2=0x56159908b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.826+0000 7f9042a00640 1 --2- 172.21.15.47:0/2132947531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561599080d40 0x56159908b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.826+0000 7f9042a00640 1 -- 172.21.15.47:0/2132947531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 msgr2=0x561599089f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.826+0000 7f9042a00640 1 --2- 172.21.15.47:0/2132947531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 0x561599089f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.826+0000 7f9042a00640 1 -- 172.21.15.47:0/2132947531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56159908ba30 con 0x56159907bbd0 2024-03-21T11:24:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.826+0000 7f9042a00640 1 --2- 172.21.15.47:0/2132947531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907bbd0 0x56159908a480 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9034023040 tx=0x7f9034020d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5c229c26581615a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.826+0000 7f9039600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:24:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.826+0000 7f9039600640 1 -- 172.21.15.47:0/2132947531 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903402e020 con 0x56159907bbd0 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.826+0000 7f90553c6740 1 -- 172.21.15.47:0/2132947531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9034026070 con 0x56159907bbd0 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.826+0000 7f9039600640 1 -- 172.21.15.47:0/2132947531 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9034003aa0 con 0x56159907bbd0 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.827+0000 7f9039600640 1 -- 172.21.15.47:0/2132947531 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9034039050 con 0x56159907bbd0 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.827+0000 7f90553c6740 10 client.25640 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.827+0000 7f90553c6740 1 -- 172.21.15.47:0/2132947531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5615990ec750 con 0x56159907bbd0 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.827+0000 7f9039600640 1 -- 172.21.15.47:0/2132947531 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9034026070 con 0x56159907bbd0 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.827+0000 7f9039600640 1 -- 172.21.15.47:0/2132947531 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9034004830 con 0x56159907bbd0 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.827+0000 7f90553c6740 10 client.25640 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.827+0000 7f90553c6740 10 client.25640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.827+0000 7f90553c6740 20 client.25640 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.827+0000 7f90553c6740 1 --2- 172.21.15.47:0/2132947531 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615990ed090 0x5615990ef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.827+0000 7f90553c6740 4 client.25640 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.827+0000 7f90553c6740 1 -- 172.21.15.47:0/2132947531 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561598fe80c0 con 0x5615990ed090 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.828+0000 7f904b400640 1 --2- 172.21.15.47:0/2132947531 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615990ed090 0x5615990ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.828+0000 7f904b400640 1 --2- 172.21.15.47:0/2132947531 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615990ed090 0x5615990ef480 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=229921695c5d0a23 server_cookie=8083f9b6aef0e19f in_seq=0 out_seq=0 2024-03-21T11:24:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.828+0000 7f9039600640 10 client.25640 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f9039600640 1 -- 172.21.15.47:0/2132947531 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561598fe80c0 con 0x5615990ed090 2024-03-21T11:24:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f9039600640 10 client.25640 handle_command_reply: tid=0 2024-03-21T11:24:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f90553c6740 1 client.25640 shutdown 2024-03-21T11:24:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f90553c6740 1 -- 172.21.15.47:0/2132947531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907bbd0 msgr2=0x56159908a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f90553c6740 1 --2- 172.21.15.47:0/2132947531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907bbd0 0x56159908a480 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9034023040 tx=0x7f9034020d00 comp rx=0 tx=0).stop 2024-03-21T11:24:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f90553c6740 1 -- 172.21.15.47:0/2132947531 shutdown_connections 2024-03-21T11:24:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f90553c6740 1 --2- 172.21.15.47:0/2132947531 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615990ed090 0x5615990ef480 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f90553c6740 1 --2- 172.21.15.47:0/2132947531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561599080d40 0x56159908b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f90553c6740 1 --2- 172.21.15.47:0/2132947531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56159907bbd0 0x56159908a480 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f90553c6740 1 --2- 172.21.15.47:0/2132947531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56159907b1e0 0x561599089f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f9039600640 0 client.25640 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f90553c6740 1 -- 172.21.15.47:0/2132947531 >> 172.21.15.47:0/2132947531 conn(0x561599076750 msgr2=0x561599080110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f90553c6740 1 -- 172.21.15.47:0/2132947531 shutdown_connections 2024-03-21T11:24:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f90553c6740 1 -- 172.21.15.47:0/2132947531 wait complete. 2024-03-21T11:24:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.841+0000 7f90553c6740 20 client.25640 trim_cache size 0 max 16384 2024-03-21T11:24:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:24:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:24:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:24:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:24:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.842+0000 7f9052600640 1 -- 172.21.15.47:0/461087202 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f90300a30e0 msgr2=0x7f90300a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.842+0000 7f9052600640 1 --2- 172.21.15.47:0/461087202 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f90300a30e0 0x7f90300a55a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x56159907c430 tx=0x7f9044005f90 comp rx=0 tx=0).stop 2024-03-21T11:24:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.842+0000 7f9052600640 1 -- 172.21.15.47:0/461087202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f904c1021b0 msgr2=0x7f904c199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.842+0000 7f9052600640 1 --2- 172.21.15.47:0/461087202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f904c1021b0 0x7f904c199b30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f903c0325f0 tx=0x7f903c032ec0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.842+0000 7f9052600640 1 -- 172.21.15.47:0/461087202 shutdown_connections 2024-03-21T11:24:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.842+0000 7f9052600640 1 --2- 172.21.15.47:0/461087202 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f90300a30e0 0x7f90300a55a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.842+0000 7f9052600640 1 --2- 172.21.15.47:0/461087202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f904c107410 0x7f904c1a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.842+0000 7f9052600640 1 --2- 172.21.15.47:0/461087202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f904c104ae0 0x7f904c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.842+0000 7f9052600640 1 --2- 172.21.15.47:0/461087202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f904c1021b0 0x7f904c199b30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.842+0000 7f9052600640 1 -- 172.21.15.47:0/461087202 >> 172.21.15.47:0/461087202 conn(0x7f904c0fbba0 msgr2=0x7f904c101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.842+0000 7f9052600640 1 -- 172.21.15.47:0/461087202 shutdown_connections 2024-03-21T11:24:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:02.842+0000 7f9052600640 1 -- 172.21.15.47:0/461087202 wait complete. 2024-03-21T11:24:03.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:02 smithi100 ceph-mon[36245]: pgmap v421: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-21T11:24:03.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:02 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:24:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:02 smithi102 ceph-mon[36748]: pgmap v421: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-21T11:24:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:02 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:24:03.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:02 smithi047 ceph-mon[31351]: pgmap v421: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-21T11:24:03.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:02 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:24:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:04 smithi102 ceph-mon[36748]: pgmap v422: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T11:24:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:04 smithi047 ceph-mon[31351]: pgmap v422: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T11:24:05.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:04 smithi100 ceph-mon[36245]: pgmap v422: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T11:24:07.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:06 smithi102 ceph-mon[36748]: pgmap v423: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T11:24:07.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:06 smithi047 ceph-mon[31351]: pgmap v423: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T11:24:07.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:06 smithi100 ceph-mon[36245]: pgmap v423: 129 pgs: 129 active+clean; 663 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T11:24:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:08 smithi102 ceph-mon[36748]: pgmap v424: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 852 B/s wr, 12 op/s 2024-03-21T11:24:09.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:08 smithi047 ceph-mon[31351]: pgmap v424: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 852 B/s wr, 12 op/s 2024-03-21T11:24:09.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:08 smithi100 ceph-mon[36245]: pgmap v424: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 852 B/s wr, 12 op/s 2024-03-21T11:24:11.195 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:10 smithi100 ceph-mon[36245]: pgmap v425: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-21T11:24:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:10 smithi102 ceph-mon[36748]: pgmap v425: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-21T11:24:11.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:10 smithi047 ceph-mon[31351]: pgmap v425: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-21T11:24:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:12 smithi102 ceph-mon[36748]: pgmap v426: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-21T11:24:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:12 smithi047 ceph-mon[31351]: pgmap v426: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-21T11:24:13.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:12 smithi100 ceph-mon[36245]: pgmap v426: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-21T11:24:15.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:14 smithi102 ceph-mon[36748]: pgmap v427: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-03-21T11:24:15.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:14 smithi047 ceph-mon[31351]: pgmap v427: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-03-21T11:24:15.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:14 smithi100 ceph-mon[36245]: pgmap v427: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-03-21T11:24:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.991+0000 7f686f600640 1 -- 172.21.15.47:0/916241158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6868102140 msgr2=0x7f6868104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.991+0000 7f686f600640 1 --2- 172.21.15.47:0/916241158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6868102140 0x7f6868104530 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f685c009900 tx=0x7f685c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:24:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.992+0000 7f686f600640 1 -- 172.21.15.47:0/916241158 shutdown_connections 2024-03-21T11:24:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.992+0000 7f686f600640 1 --2- 172.21.15.47:0/916241158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68681073a0 0x7f68681097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.992+0000 7f686f600640 1 --2- 172.21.15.47:0/916241158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6868104a70 0x7f6868106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.992+0000 7f686f600640 1 --2- 172.21.15.47:0/916241158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6868102140 0x7f6868104530 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.992+0000 7f686f600640 1 -- 172.21.15.47:0/916241158 >> 172.21.15.47:0/916241158 conn(0x7f68680fbb30 msgr2=0x7f68680fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.992+0000 7f686f600640 1 -- 172.21.15.47:0/916241158 shutdown_connections 2024-03-21T11:24:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.992+0000 7f686f600640 1 -- 172.21.15.47:0/916241158 wait complete. 2024-03-21T11:24:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.993+0000 7f686f600640 1 Processor -- start 2024-03-21T11:24:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.993+0000 7f686f600640 1 -- start start 2024-03-21T11:24:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.993+0000 7f686f600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6868102140 0x7f6868199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.993+0000 7f686f600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6868104a70 0x7f686819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.993+0000 7f686f600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68681073a0 0x7f68681a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.993+0000 7f686f600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686810d3c0 con 0x7f6868102140 2024-03-21T11:24:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.993+0000 7f686f600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686810d240 con 0x7f6868104a70 2024-03-21T11:24:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.993+0000 7f686f600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686810d540 con 0x7f68681073a0 2024-03-21T11:24:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.994+0000 7f686cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6868102140 0x7f6868199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.994+0000 7f6867e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6868104a70 0x7f686819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.994+0000 7f686cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6868102140 0x7f6868199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39012/0 (socket says 172.21.15.47:39012) 2024-03-21T11:24:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.994+0000 7f686d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68681073a0 0x7f68681a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.994+0000 7f686cc00640 1 -- 172.21.15.47:0/1858235681 learned_addr learned my addr 172.21.15.47:0/1858235681 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.994+0000 7f686d600640 1 -- 172.21.15.47:0/1858235681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6868104a70 msgr2=0x7f686819a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.994+0000 7f686d600640 1 --2- 172.21.15.47:0/1858235681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6868104a70 0x7f686819a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.994+0000 7f686d600640 1 -- 172.21.15.47:0/1858235681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6868102140 msgr2=0x7f6868199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.994+0000 7f686d600640 1 --2- 172.21.15.47:0/1858235681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6868102140 0x7f6868199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.994+0000 7f686d600640 1 -- 172.21.15.47:0/1858235681 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68681a17b0 con 0x7f68681073a0 2024-03-21T11:24:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.994+0000 7f686cc00640 1 --2- 172.21.15.47:0/1858235681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6868102140 0x7f6868199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.995+0000 7f686d600640 1 --2- 172.21.15.47:0/1858235681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68681073a0 0x7f68681a1140 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6860009fd0 tx=0x7f686000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.995+0000 7f6865600640 1 -- 172.21.15.47:0/1858235681 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686000ce20 con 0x7f68681073a0 2024-03-21T11:24:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.995+0000 7f6865600640 1 -- 172.21.15.47:0/1858235681 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6860005020 con 0x7f68681073a0 2024-03-21T11:24:15.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.995+0000 7f6865600640 1 -- 172.21.15.47:0/1858235681 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860010720 con 0x7f68681073a0 2024-03-21T11:24:15.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.995+0000 7f686f600640 1 -- 172.21.15.47:0/1858235681 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68681a1aa0 con 0x7f68681073a0 2024-03-21T11:24:15.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.995+0000 7f686f600640 1 -- 172.21.15.47:0/1858235681 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68681a1fa0 con 0x7f68681073a0 2024-03-21T11:24:15.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.998+0000 7f6865600640 1 -- 172.21.15.47:0/1858235681 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f6860005540 con 0x7f68681073a0 2024-03-21T11:24:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.998+0000 7f6872208740 1 Processor -- start 2024-03-21T11:24:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.998+0000 7f6872208740 1 -- start start 2024-03-21T11:24:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.999+0000 7f6872208740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433e1e0 0x5601b433e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.999+0000 7f6872208740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433ebd0 0x5601b43436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.999+0000 7f6872208740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b4343d40 0x5601b43461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.999+0000 7f6872208740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b43013b0 con 0x5601b433e1e0 2024-03-21T11:24:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.999+0000 7f6872208740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b42a3830 con 0x5601b433ebd0 2024-03-21T11:24:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.999+0000 7f6872208740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b424c5d0 con 0x5601b4343d40 2024-03-21T11:24:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.999+0000 7f686cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433ebd0 0x5601b43436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.999+0000 7f686cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433ebd0 0x5601b43436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45494/0 (socket says 172.21.15.47:45494) 2024-03-21T11:24:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.999+0000 7f6867e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433e1e0 0x5601b433e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.999+0000 7f686cc00640 1 -- 172.21.15.47:0/1854854518 learned_addr learned my addr 172.21.15.47:0/1854854518 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.999+0000 7f6867e00640 1 --2- 172.21.15.47:0/1854854518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b4343d40 0x5601b43461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:15.999+0000 7f6865600640 1 --2- 172.21.15.47:0/1858235681 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f68500a30e0 0x7f68500a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.000+0000 7f686cc00640 1 -- 172.21.15.47:0/1854854518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b4343d40 msgr2=0x5601b43461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.000+0000 7f686cc00640 1 --2- 172.21.15.47:0/1854854518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b4343d40 0x5601b43461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.000+0000 7f686cc00640 1 -- 172.21.15.47:0/1854854518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433e1e0 msgr2=0x5601b433e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.000+0000 7f686cc00640 1 --2- 172.21.15.47:0/1854854518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433e1e0 0x5601b433e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.000+0000 7f686cc00640 1 -- 172.21.15.47:0/1854854518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601b42642c0 con 0x5601b433ebd0 2024-03-21T11:24:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.000+0000 7f6865600640 1 -- 172.21.15.47:0/1858235681 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f68600cfff0 con 0x7f68681073a0 2024-03-21T11:24:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.000+0000 7f686cc00640 1 --2- 172.21.15.47:0/1854854518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433ebd0 0x5601b43436d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f685c02c170 tx=0x7f685c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3398b84590718cd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.000+0000 7f686d600640 1 --2- 172.21.15.47:0/1858235681 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f68500a30e0 0x7f68500a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.000+0000 7f6859c00640 1 -- 172.21.15.47:0/1854854518 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c045070 con 0x5601b433ebd0 2024-03-21T11:24:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.000+0000 7f6859c00640 1 -- 172.21.15.47:0/1854854518 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f685c02cd30 con 0x5601b433ebd0 2024-03-21T11:24:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.000+0000 7f6859c00640 1 -- 172.21.15.47:0/1854854518 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c005130 con 0x5601b433ebd0 2024-03-21T11:24:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.001+0000 7f6872208740 1 -- 172.21.15.47:0/1854854518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433ebd0 msgr2=0x5601b43436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.001+0000 7f6872208740 1 --2- 172.21.15.47:0/1854854518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433ebd0 0x5601b43436d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f685c02c170 tx=0x7f685c009560 comp rx=0 tx=0).stop 2024-03-21T11:24:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.001+0000 7f686d600640 1 --2- 172.21.15.47:0/1858235681 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f68500a30e0 0x7f68500a55a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f68600058f0 tx=0x7f6860005b20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.001+0000 7f6872208740 1 -- 172.21.15.47:0/1854854518 shutdown_connections 2024-03-21T11:24:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.001+0000 7f6872208740 1 --2- 172.21.15.47:0/1854854518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b4343d40 0x5601b43461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.001+0000 7f6872208740 1 --2- 172.21.15.47:0/1854854518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433ebd0 0x5601b43436d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.001+0000 7f6872208740 1 --2- 172.21.15.47:0/1854854518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433e1e0 0x5601b433e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.001+0000 7f6872208740 1 -- 172.21.15.47:0/1854854518 >> 172.21.15.47:0/1854854518 conn(0x5601b4339750 msgr2=0x5601b4339b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.001+0000 7f6872208740 1 -- 172.21.15.47:0/1854854518 shutdown_connections 2024-03-21T11:24:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.001+0000 7f6872208740 1 -- 172.21.15.47:0/1854854518 wait complete. 2024-03-21T11:24:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.002+0000 7f6872208740 1 -- start start 2024-03-21T11:24:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.002+0000 7f6872208740 1 Processor -- start 2024-03-21T11:24:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.002+0000 7f6872208740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433e1e0 0x5601b4348320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.002+0000 7f6872208740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433ebd0 0x5601b4348860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.002+0000 7f6872208740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b4343d40 0x5601b434f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.002+0000 7f6872208740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b425fa20 con 0x5601b433ebd0 2024-03-21T11:24:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.002+0000 7f6872208740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b429f8d0 con 0x5601b433e1e0 2024-03-21T11:24:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.002+0000 7f6872208740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b4346c90 con 0x5601b4343d40 2024-03-21T11:24:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.003+0000 7f6872208740 10 client.0 _ll_register_callbacks cb 0x5601b429c1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.003+0000 7f6867e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433e1e0 0x5601b4348320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.003+0000 7f686cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433ebd0 0x5601b4348860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.003+0000 7f6867e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433e1e0 0x5601b4348320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45498/0 (socket says 172.21.15.47:45498) 2024-03-21T11:24:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.003+0000 7f6867e00640 1 -- 172.21.15.47:0/4171911635 learned_addr learned my addr 172.21.15.47:0/4171911635 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.003+0000 7f6867e00640 1 --2- 172.21.15.47:0/4171911635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b4343d40 0x5601b434f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.003+0000 7f6867e00640 1 -- 172.21.15.47:0/4171911635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b4343d40 msgr2=0x5601b434f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.003+0000 7f6867e00640 1 --2- 172.21.15.47:0/4171911635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b4343d40 0x5601b434f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.003+0000 7f6867e00640 1 -- 172.21.15.47:0/4171911635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433ebd0 msgr2=0x5601b4348860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.003+0000 7f6867e00640 1 --2- 172.21.15.47:0/4171911635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433ebd0 0x5601b4348860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.003+0000 7f6867e00640 1 -- 172.21.15.47:0/4171911635 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601b3fb8210 con 0x5601b433e1e0 2024-03-21T11:24:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.003+0000 7f6867e00640 1 --2- 172.21.15.47:0/4171911635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433e1e0 0x5601b4348320 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f685400f3c0 tx=0x7f6854015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ed4f285e1a90e03 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.003+0000 7f684e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:24:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.003+0000 7f684e000640 1 -- 172.21.15.47:0/4171911635 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685401c040 con 0x5601b433e1e0 2024-03-21T11:24:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.004+0000 7f684e000640 1 -- 172.21.15.47:0/4171911635 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6854005020 con 0x5601b433e1e0 2024-03-21T11:24:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.004+0000 7f6872208740 1 -- 172.21.15.47:0/4171911635 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f685c02cd30 con 0x5601b433e1e0 2024-03-21T11:24:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.004+0000 7f684e000640 1 -- 172.21.15.47:0/4171911635 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f685402a050 con 0x5601b433e1e0 2024-03-21T11:24:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.004+0000 7f6872208740 10 client.25652 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.004+0000 7f6872208740 1 -- 172.21.15.47:0/4171911635 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5601b4084620 con 0x5601b433e1e0 2024-03-21T11:24:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.004+0000 7f684e000640 1 -- 172.21.15.47:0/4171911635 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f685c02cd30 con 0x5601b433e1e0 2024-03-21T11:24:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.004+0000 7f684e000640 1 -- 172.21.15.47:0/4171911635 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6854021380 con 0x5601b433e1e0 2024-03-21T11:24:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.004+0000 7f6872208740 10 client.25652 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.004+0000 7f6872208740 10 client.25652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.004+0000 7f6872208740 20 client.25652 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.004+0000 7f6872208740 1 --2- 172.21.15.47:0/4171911635 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601b4354120 0x5601b4356580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.004+0000 7f6872208740 4 client.25652 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.004+0000 7f6872208740 1 -- 172.21.15.47:0/4171911635 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601b42b0d70 con 0x5601b4354120 2024-03-21T11:24:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.005+0000 7f686cc00640 1 --2- 172.21.15.47:0/4171911635 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601b4354120 0x5601b4356580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.005+0000 7f686cc00640 1 --2- 172.21.15.47:0/4171911635 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601b4354120 0x5601b4356580 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3918bf81ab279f72 server_cookie=40b90c2e66ba5848 in_seq=0 out_seq=0 2024-03-21T11:24:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.005+0000 7f684e000640 10 client.25652 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.006+0000 7f684e000640 1 -- 172.21.15.47:0/4171911635 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5601b42b0d70 con 0x5601b4354120 2024-03-21T11:24:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.006+0000 7f684e000640 10 client.25652 handle_command_reply: tid=0 2024-03-21T11:24:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.006+0000 7f6872208740 1 client.25652 shutdown 2024-03-21T11:24:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.006+0000 7f6872208740 1 -- 172.21.15.47:0/4171911635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433e1e0 msgr2=0x5601b4348320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.006+0000 7f6872208740 1 --2- 172.21.15.47:0/4171911635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433e1e0 0x5601b4348320 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f685400f3c0 tx=0x7f6854015040 comp rx=0 tx=0).stop 2024-03-21T11:24:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.006+0000 7f6872208740 1 -- 172.21.15.47:0/4171911635 shutdown_connections 2024-03-21T11:24:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.006+0000 7f6872208740 1 --2- 172.21.15.47:0/4171911635 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601b4354120 0x5601b4356580 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.006+0000 7f6872208740 1 --2- 172.21.15.47:0/4171911635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b4343d40 0x5601b434f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.006+0000 7f6872208740 1 --2- 172.21.15.47:0/4171911635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433ebd0 0x5601b4348860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.006+0000 7f6872208740 1 --2- 172.21.15.47:0/4171911635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433e1e0 0x5601b4348320 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.006+0000 7f6872208740 1 -- 172.21.15.47:0/4171911635 >> 172.21.15.47:0/4171911635 conn(0x5601b4339750 msgr2=0x5601b4338950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.006+0000 7f684e000640 0 client.25652 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.006+0000 7f6872208740 1 -- 172.21.15.47:0/4171911635 shutdown_connections 2024-03-21T11:24:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.007+0000 7f6872208740 1 -- 172.21.15.47:0/4171911635 wait complete. 2024-03-21T11:24:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.007+0000 7f6872208740 20 client.25652 trim_cache size 0 max 16384 2024-03-21T11:24:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.022+0000 7f6872208740 1 Processor -- start 2024-03-21T11:24:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.022+0000 7f6872208740 1 -- start start 2024-03-21T11:24:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.022+0000 7f6872208740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b433e1e0 0x5601b4348640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.022+0000 7f6872208740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433ebd0 0x5601b434f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.022+0000 7f6872208740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b4343d40 0x5601b43380c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.022+0000 7f6872208740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681ac150 con 0x5601b4343d40 2024-03-21T11:24:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.022+0000 7f6872208740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681adad0 con 0x5601b433ebd0 2024-03-21T11:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.022+0000 7f6872208740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681ac7b0 con 0x5601b433e1e0 2024-03-21T11:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.022+0000 7f686cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433ebd0 0x5601b434f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.022+0000 7f686cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433ebd0 0x5601b434f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45500/0 (socket says 172.21.15.47:45500) 2024-03-21T11:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.022+0000 7f686cc00640 1 -- 172.21.15.47:0/3253711538 learned_addr learned my addr 172.21.15.47:0/3253711538 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.023+0000 7f6867e00640 1 --2- 172.21.15.47:0/3253711538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b433e1e0 0x5601b4348640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.023+0000 7f6867e00640 1 --2- 172.21.15.47:0/3253711538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b4343d40 0x5601b43380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.023+0000 7f686cc00640 1 -- 172.21.15.47:0/3253711538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b433e1e0 msgr2=0x5601b4348640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.023+0000 7f686cc00640 1 --2- 172.21.15.47:0/3253711538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b433e1e0 0x5601b4348640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.023+0000 7f686cc00640 1 -- 172.21.15.47:0/3253711538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b4343d40 msgr2=0x5601b43380c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.023+0000 7f686cc00640 1 --2- 172.21.15.47:0/3253711538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b4343d40 0x5601b43380c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.023+0000 7f686cc00640 1 -- 172.21.15.47:0/3253711538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601b43343a0 con 0x5601b433ebd0 2024-03-21T11:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.023+0000 7f6867e00640 1 --2- 172.21.15.47:0/3253711538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b4343d40 0x5601b43380c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.023+0000 7f686cc00640 1 --2- 172.21.15.47:0/3253711538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433ebd0 0x5601b434f860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f685c000c00 tx=0x7f685c041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfda4fcc2bf3470 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.023+0000 7f6859200640 1 -- 172.21.15.47:0/3253711538 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c045040 con 0x5601b433ebd0 2024-03-21T11:24:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.023+0000 7f6859200640 1 -- 172.21.15.47:0/3253711538 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f685c004ed0 con 0x5601b433ebd0 2024-03-21T11:24:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.023+0000 7f6859200640 1 -- 172.21.15.47:0/3253711538 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c054800 con 0x5601b433ebd0 2024-03-21T11:24:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.024+0000 7f6872208740 1 -- 172.21.15.47:0/3253711538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433ebd0 msgr2=0x5601b434f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.024+0000 7f6872208740 1 --2- 172.21.15.47:0/3253711538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433ebd0 0x5601b434f860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f685c000c00 tx=0x7f685c041040 comp rx=0 tx=0).stop 2024-03-21T11:24:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.024+0000 7f6872208740 1 -- 172.21.15.47:0/3253711538 shutdown_connections 2024-03-21T11:24:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.024+0000 7f6872208740 1 --2- 172.21.15.47:0/3253711538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b4343d40 0x5601b43380c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.024+0000 7f6872208740 1 --2- 172.21.15.47:0/3253711538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b433ebd0 0x5601b434f860 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.024+0000 7f6872208740 1 --2- 172.21.15.47:0/3253711538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b433e1e0 0x5601b4348640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.024+0000 7f6872208740 1 -- 172.21.15.47:0/3253711538 >> 172.21.15.47:0/3253711538 conn(0x5601b4339750 msgr2=0x5601b4347f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.024+0000 7f6872208740 1 -- 172.21.15.47:0/3253711538 shutdown_connections 2024-03-21T11:24:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.024+0000 7f6872208740 1 -- 172.21.15.47:0/3253711538 wait complete. 2024-03-21T11:24:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.025+0000 7f6872208740 1 -- start start 2024-03-21T11:24:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.025+0000 7f6872208740 1 Processor -- start 2024-03-21T11:24:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.025+0000 7f6872208740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b433e1e0 0x5601b434d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.025+0000 7f6872208740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433ebd0 0x5601b434d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.025+0000 7f6872208740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b4343d40 0x5601b434e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.025+0000 7f6872208740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681ad470 con 0x5601b433ebd0 2024-03-21T11:24:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.025+0000 7f6872208740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681ace10 con 0x5601b4343d40 2024-03-21T11:24:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.025+0000 7f6872208740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681aa310 con 0x5601b433e1e0 2024-03-21T11:24:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.025+0000 7f6872208740 10 client.0 _ll_register_callbacks cb 0x7f6854009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.025+0000 7f6867e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b433e1e0 0x5601b434d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.026+0000 7f6867e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b433e1e0 0x5601b434d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56542/0 (socket says 172.21.15.47:56542) 2024-03-21T11:24:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.026+0000 7f686cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433ebd0 0x5601b434d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.026+0000 7f6867e00640 1 -- 172.21.15.47:0/1526753167 learned_addr learned my addr 172.21.15.47:0/1526753167 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.026+0000 7f6867e00640 1 --2- 172.21.15.47:0/1526753167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b4343d40 0x5601b434e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.026+0000 7f686cc00640 1 -- 172.21.15.47:0/1526753167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b433e1e0 msgr2=0x5601b434d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.026+0000 7f686cc00640 1 --2- 172.21.15.47:0/1526753167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b433e1e0 0x5601b434d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.026+0000 7f686cc00640 1 -- 172.21.15.47:0/1526753167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b4343d40 msgr2=0x5601b434e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.026+0000 7f686cc00640 1 --2- 172.21.15.47:0/1526753167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b4343d40 0x5601b434e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.026+0000 7f686cc00640 1 -- 172.21.15.47:0/1526753167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601b434eea0 con 0x5601b433ebd0 2024-03-21T11:24:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.026+0000 7f686cc00640 1 --2- 172.21.15.47:0/1526753167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433ebd0 0x5601b434d910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5601b4356a50 tx=0x7f685c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a27c0db872fc715 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.026+0000 7f684e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:24:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.027+0000 7f684e000640 1 -- 172.21.15.47:0/1526753167 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c063060 con 0x5601b433ebd0 2024-03-21T11:24:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.027+0000 7f684e000640 1 -- 172.21.15.47:0/1526753167 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f685c004040 con 0x5601b433ebd0 2024-03-21T11:24:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.027+0000 7f6872208740 1 -- 172.21.15.47:0/1526753167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f685c004ed0 con 0x5601b433ebd0 2024-03-21T11:24:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.027+0000 7f684e000640 1 -- 172.21.15.47:0/1526753167 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f685c045040 con 0x5601b433ebd0 2024-03-21T11:24:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.027+0000 7f6872208740 10 client.15510 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.027+0000 7f6872208740 1 -- 172.21.15.47:0/1526753167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5601b43afd40 con 0x5601b433ebd0 2024-03-21T11:24:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.027+0000 7f684e000640 1 -- 172.21.15.47:0/1526753167 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f685c004ed0 con 0x5601b433ebd0 2024-03-21T11:24:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.027+0000 7f684e000640 1 -- 172.21.15.47:0/1526753167 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f685c05d070 con 0x5601b433ebd0 2024-03-21T11:24:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.027+0000 7f6872208740 10 client.15510 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.027+0000 7f6872208740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.027+0000 7f6872208740 20 client.15510 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.028+0000 7f6872208740 1 --2- 172.21.15.47:0/1526753167 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601b43b0110 0x5601b43b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.028+0000 7f6872208740 4 client.15510 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:24:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.028+0000 7f6872208740 1 -- 172.21.15.47:0/1526753167 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5601b42ab0c0 con 0x5601b43b0110 2024-03-21T11:24:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.028+0000 7f6867e00640 1 --2- 172.21.15.47:0/1526753167 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601b43b0110 0x5601b43b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.028+0000 7f6867e00640 1 --2- 172.21.15.47:0/1526753167 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601b43b0110 0x5601b43b2500 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23df514ba527ca6b server_cookie=58433dc76c0e534d in_seq=0 out_seq=0 2024-03-21T11:24:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.028+0000 7f684e000640 10 client.15510 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.031+0000 7f684e000640 1 -- 172.21.15.47:0/1526753167 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5601b42ab0c0 con 0x5601b43b0110 2024-03-21T11:24:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.031+0000 7f684e000640 10 client.15510 handle_command_reply: tid=0 2024-03-21T11:24:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.031+0000 7f6872208740 1 client.15510 shutdown 2024-03-21T11:24:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.031+0000 7f6872208740 1 -- 172.21.15.47:0/1526753167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433ebd0 msgr2=0x5601b434d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.031+0000 7f6872208740 1 --2- 172.21.15.47:0/1526753167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433ebd0 0x5601b434d910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5601b4356a50 tx=0x7f685c005fd0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.031+0000 7f6872208740 1 -- 172.21.15.47:0/1526753167 shutdown_connections 2024-03-21T11:24:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.031+0000 7f6872208740 1 --2- 172.21.15.47:0/1526753167 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601b43b0110 0x5601b43b2500 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.031+0000 7f6872208740 1 --2- 172.21.15.47:0/1526753167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601b4343d40 0x5601b434e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.031+0000 7f6872208740 1 --2- 172.21.15.47:0/1526753167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601b433ebd0 0x5601b434d910 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.031+0000 7f6872208740 1 --2- 172.21.15.47:0/1526753167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601b433e1e0 0x5601b434d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.031+0000 7f6872208740 1 -- 172.21.15.47:0/1526753167 >> 172.21.15.47:0/1526753167 conn(0x5601b4339750 msgr2=0x5601b433c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.031+0000 7f684e000640 0 client.15510 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.032+0000 7f6872208740 1 -- 172.21.15.47:0/1526753167 shutdown_connections 2024-03-21T11:24:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.032+0000 7f6872208740 1 -- 172.21.15.47:0/1526753167 wait complete. 2024-03-21T11:24:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.032+0000 7f6872208740 20 client.15510 trim_cache size 0 max 16384 2024-03-21T11:24:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:24:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:24:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:24:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:24:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.032+0000 7f686f600640 1 -- 172.21.15.47:0/1858235681 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f68500a30e0 msgr2=0x7f68500a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.032+0000 7f686f600640 1 --2- 172.21.15.47:0/1858235681 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f68500a30e0 0x7f68500a55a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f68600058f0 tx=0x7f6860005b20 comp rx=0 tx=0).stop 2024-03-21T11:24:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.032+0000 7f686f600640 1 -- 172.21.15.47:0/1858235681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68681073a0 msgr2=0x7f68681a1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.032+0000 7f686f600640 1 --2- 172.21.15.47:0/1858235681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68681073a0 0x7f68681a1140 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6860009fd0 tx=0x7f686000ef10 comp rx=0 tx=0).stop 2024-03-21T11:24:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.033+0000 7f686f600640 1 -- 172.21.15.47:0/1858235681 shutdown_connections 2024-03-21T11:24:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.033+0000 7f686f600640 1 --2- 172.21.15.47:0/1858235681 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f68500a30e0 0x7f68500a55a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.033+0000 7f686f600640 1 --2- 172.21.15.47:0/1858235681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68681073a0 0x7f68681a1140 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.033+0000 7f686f600640 1 --2- 172.21.15.47:0/1858235681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6868104a70 0x7f686819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.033+0000 7f686f600640 1 --2- 172.21.15.47:0/1858235681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6868102140 0x7f6868199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.033+0000 7f686f600640 1 -- 172.21.15.47:0/1858235681 >> 172.21.15.47:0/1858235681 conn(0x7f68680fbb30 msgr2=0x7f6868101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.033+0000 7f686f600640 1 -- 172.21.15.47:0/1858235681 shutdown_connections 2024-03-21T11:24:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:16.033+0000 7f686f600640 1 -- 172.21.15.47:0/1858235681 wait complete. 2024-03-21T11:24:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:16 smithi102 ceph-mon[36748]: pgmap v428: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-03-21T11:24:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:16 smithi047 ceph-mon[31351]: pgmap v428: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-03-21T11:24:17.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:16 smithi100 ceph-mon[36245]: pgmap v428: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-03-21T11:24:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:18 smithi102 ceph-mon[36748]: pgmap v429: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-03-21T11:24:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:18 smithi047 ceph-mon[31351]: pgmap v429: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-03-21T11:24:19.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:18 smithi100 ceph-mon[36245]: pgmap v429: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-03-21T11:24:20.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:24:20.072 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:24:20.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.311+0000 7eff47200640 1 -- 172.21.15.47:0/1822308084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff48104930 msgr2=0x7eff48106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:20.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.311+0000 7eff47200640 1 --2- 172.21.15.47:0/1822308084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff48104930 0x7eff48106d20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7eff30009900 tx=0x7eff3002bc30 comp rx=0 tx=0).stop 2024-03-21T11:24:20.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.311+0000 7eff47200640 1 -- 172.21.15.47:0/1822308084 shutdown_connections 2024-03-21T11:24:20.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.311+0000 7eff47200640 1 --2- 172.21.15.47:0/1822308084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff48107260 0x7eff481096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.311+0000 7eff47200640 1 --2- 172.21.15.47:0/1822308084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff48104930 0x7eff48106d20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.311+0000 7eff47200640 1 --2- 172.21.15.47:0/1822308084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff48102000 0x7eff481043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.311+0000 7eff47200640 1 -- 172.21.15.47:0/1822308084 >> 172.21.15.47:0/1822308084 conn(0x7eff480fba10 msgr2=0x7eff480fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:20.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.311+0000 7eff47200640 1 -- 172.21.15.47:0/1822308084 shutdown_connections 2024-03-21T11:24:20.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.311+0000 7eff47200640 1 -- 172.21.15.47:0/1822308084 wait complete. 2024-03-21T11:24:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.312+0000 7eff47200640 1 Processor -- start 2024-03-21T11:24:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.313+0000 7eff47200640 1 -- start start 2024-03-21T11:24:20.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.313+0000 7eff47200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff48102000 0x7eff481978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.313+0000 7eff47200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff48104930 0x7eff48197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.313+0000 7eff47200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff48107260 0x7eff4819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.313+0000 7eff47200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4810d260 con 0x7eff48107260 2024-03-21T11:24:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.313+0000 7eff47200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4810d0e0 con 0x7eff48102000 2024-03-21T11:24:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.313+0000 7eff47200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4810d3e0 con 0x7eff48104930 2024-03-21T11:24:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.313+0000 7eff3fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff48104930 0x7eff48197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.313+0000 7eff3fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff48104930 0x7eff48197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60928/0 (socket says 172.21.15.47:60928) 2024-03-21T11:24:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.313+0000 7eff3fe00640 1 -- 172.21.15.47:0/3040411102 learned_addr learned my addr 172.21.15.47:0/3040411102 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.313+0000 7eff45e00640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff48102000 0x7eff481978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.313+0000 7eff46800640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff48107260 0x7eff4819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.314+0000 7eff3fe00640 1 -- 172.21.15.47:0/3040411102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff48102000 msgr2=0x7eff481978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.314+0000 7eff3fe00640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff48102000 0x7eff481978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.314+0000 7eff3fe00640 1 -- 172.21.15.47:0/3040411102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff48107260 msgr2=0x7eff4819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.314+0000 7eff3fe00640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff48107260 0x7eff4819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.314+0000 7eff3fe00640 1 -- 172.21.15.47:0/3040411102 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4819f4e0 con 0x7eff48104930 2024-03-21T11:24:20.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.314+0000 7eff45e00640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff48102000 0x7eff481978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:20.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.314+0000 7eff3fe00640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff48104930 0x7eff48197e00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7eff30000c00 tx=0x7eff3002c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:20.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.314+0000 7eff3ea00640 1 -- 172.21.15.47:0/3040411102 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30043020 con 0x7eff48104930 2024-03-21T11:24:20.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.314+0000 7eff3ea00640 1 -- 172.21.15.47:0/3040411102 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff300054e0 con 0x7eff48104930 2024-03-21T11:24:20.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.314+0000 7eff47200640 1 -- 172.21.15.47:0/3040411102 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4819f770 con 0x7eff48104930 2024-03-21T11:24:20.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.314+0000 7eff3ea00640 1 -- 172.21.15.47:0/3040411102 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30003ce0 con 0x7eff48104930 2024-03-21T11:24:20.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.315+0000 7eff47200640 1 -- 172.21.15.47:0/3040411102 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff4819fce0 con 0x7eff48104930 2024-03-21T11:24:20.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.318+0000 7eff4fba5740 1 Processor -- start 2024-03-21T11:24:20.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.318+0000 7eff3ea00640 1 -- 172.21.15.47:0/3040411102 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7eff30046020 con 0x7eff48104930 2024-03-21T11:24:20.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.318+0000 7eff4fba5740 1 -- start start 2024-03-21T11:24:20.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.318+0000 7eff4fba5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713460 0x55ed72713880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.318+0000 7eff4fba5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713e50 0x55ed72718950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.318+0000 7eff4fba5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed7271b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.318+0000 7eff4fba5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed72674250 con 0x55ed72713460 2024-03-21T11:24:20.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.318+0000 7eff4fba5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7261ca30 con 0x55ed72713e50 2024-03-21T11:24:20.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.318+0000 7eff4fba5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed726702f0 con 0x55ed72718fc0 2024-03-21T11:24:20.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.319+0000 7eff3fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed7271b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.319+0000 7eff46800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713460 0x55ed72713880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.319+0000 7eff45e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713e50 0x55ed72718950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.319+0000 7eff3fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed7271b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60944/0 (socket says 172.21.15.47:60944) 2024-03-21T11:24:20.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.319+0000 7eff46800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713460 0x55ed72713880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58814/0 (socket says 172.21.15.47:58814) 2024-03-21T11:24:20.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.319+0000 7eff3fe00640 1 -- 172.21.15.47:0/2086323878 learned_addr learned my addr 172.21.15.47:0/2086323878 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:20.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.319+0000 7eff45e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713e50 0x55ed72718950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51410/0 (socket says 172.21.15.47:51410) 2024-03-21T11:24:20.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.319+0000 7eff3fe00640 1 -- 172.21.15.47:0/2086323878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713e50 msgr2=0x55ed72718950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:20.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.319+0000 7eff3fe00640 1 --2- 172.21.15.47:0/2086323878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713e50 0x55ed72718950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.319+0000 7eff3fe00640 1 -- 172.21.15.47:0/2086323878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713460 msgr2=0x55ed72713880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.319+0000 7eff3fe00640 1 --2- 172.21.15.47:0/2086323878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713460 0x55ed72713880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.319+0000 7eff3fe00640 1 -- 172.21.15.47:0/2086323878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed72607690 con 0x55ed72718fc0 2024-03-21T11:24:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.319+0000 7eff46800640 1 --2- 172.21.15.47:0/2086323878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713460 0x55ed72713880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.319+0000 7eff3fe00640 1 --2- 172.21.15.47:0/2086323878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed7271b450 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7eff300f7a60 tx=0x7eff300f7b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=917b35a11a11444f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.320+0000 7eff3ea00640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff2c0a3030 0x7eff2c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.320+0000 7eff37400640 1 -- 172.21.15.47:0/2086323878 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30100070 con 0x55ed72718fc0 2024-03-21T11:24:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.320+0000 7eff37400640 1 -- 172.21.15.47:0/2086323878 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff300be280 con 0x55ed72718fc0 2024-03-21T11:24:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.320+0000 7eff37400640 1 -- 172.21.15.47:0/2086323878 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30087300 con 0x55ed72718fc0 2024-03-21T11:24:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.320+0000 7eff3ea00640 1 -- 172.21.15.47:0/3040411102 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7eff300f1890 con 0x7eff48104930 2024-03-21T11:24:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.320+0000 7eff46800640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff2c0a3030 0x7eff2c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.320+0000 7eff4fba5740 1 -- 172.21.15.47:0/2086323878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 msgr2=0x55ed7271b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.320+0000 7eff4fba5740 1 --2- 172.21.15.47:0/2086323878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed7271b450 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7eff300f7a60 tx=0x7eff300f7b50 comp rx=0 tx=0).stop 2024-03-21T11:24:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.321+0000 7eff4fba5740 1 -- 172.21.15.47:0/2086323878 shutdown_connections 2024-03-21T11:24:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.321+0000 7eff4fba5740 1 --2- 172.21.15.47:0/2086323878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed7271b450 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.321+0000 7eff4fba5740 1 --2- 172.21.15.47:0/2086323878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713e50 0x55ed72718950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.321+0000 7eff4fba5740 1 --2- 172.21.15.47:0/2086323878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713460 0x55ed72713880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.321+0000 7eff4fba5740 1 -- 172.21.15.47:0/2086323878 >> 172.21.15.47:0/2086323878 conn(0x55ed726d7200 msgr2=0x55ed7270a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.321+0000 7eff4fba5740 1 -- 172.21.15.47:0/2086323878 shutdown_connections 2024-03-21T11:24:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.321+0000 7eff46800640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff2c0a3030 0x7eff2c0a54f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55ed727146b0 tx=0x7eff40009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.321+0000 7eff4fba5740 1 -- 172.21.15.47:0/2086323878 wait complete. 2024-03-21T11:24:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.321+0000 7eff4fba5740 1 -- start start 2024-03-21T11:24:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.321+0000 7eff4fba5740 1 Processor -- start 2024-03-21T11:24:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.322+0000 7eff4fba5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713460 0x55ed7271d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.322+0000 7eff4fba5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713e50 0x55ed7271d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.322+0000 7eff4fba5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed72724670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.322+0000 7eff4fba5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7271bd30 con 0x55ed72713e50 2024-03-21T11:24:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.322+0000 7eff4fba5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7262fe80 con 0x55ed72713460 2024-03-21T11:24:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.322+0000 7eff4fba5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7271beb0 con 0x55ed72718fc0 2024-03-21T11:24:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.322+0000 7eff4fba5740 10 client.0 _ll_register_callbacks cb 0x55ed72642130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.322+0000 7eff3fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713e50 0x55ed7271d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.322+0000 7eff46800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed72724670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.322+0000 7eff45e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713460 0x55ed7271d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.322+0000 7eff3fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713e50 0x55ed7271d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58824/0 (socket says 172.21.15.47:58824) 2024-03-21T11:24:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.322+0000 7eff3fe00640 1 -- 172.21.15.47:0/3806554848 learned_addr learned my addr 172.21.15.47:0/3806554848 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.322+0000 7eff46800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed72724670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60956/0 (socket says 172.21.15.47:60956) 2024-03-21T11:24:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.323+0000 7eff45e00640 1 -- 172.21.15.47:0/3806554848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 msgr2=0x55ed72724670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:20.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.323+0000 7eff45e00640 1 --2- 172.21.15.47:0/3806554848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed72724670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.323+0000 7eff45e00640 1 -- 172.21.15.47:0/3806554848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713e50 msgr2=0x55ed7271d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.323+0000 7eff45e00640 1 --2- 172.21.15.47:0/3806554848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713e50 0x55ed7271d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.323+0000 7eff45e00640 1 -- 172.21.15.47:0/3806554848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed724527d0 con 0x55ed72713460 2024-03-21T11:24:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.323+0000 7eff3fe00640 1 --2- 172.21.15.47:0/3806554848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713e50 0x55ed7271d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.323+0000 7eff45e00640 1 --2- 172.21.15.47:0/3806554848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713460 0x55ed7271d0f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55ed72714250 tx=0x7eff3800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0f847f9349d0249 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.323+0000 7eff2be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:24:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.323+0000 7eff2be00640 1 -- 172.21.15.47:0/3806554848 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38012110 con 0x55ed72713460 2024-03-21T11:24:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.323+0000 7eff4fba5740 1 -- 172.21.15.47:0/3806554848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff300be280 con 0x55ed72713460 2024-03-21T11:24:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.323+0000 7eff2be00640 1 -- 172.21.15.47:0/3806554848 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7eff38004050 con 0x55ed72713460 2024-03-21T11:24:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.323+0000 7eff2be00640 1 -- 172.21.15.47:0/3806554848 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff38025050 con 0x55ed72713460 2024-03-21T11:24:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.323+0000 7eff4fba5740 10 client.25670 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.323+0000 7eff4fba5740 1 -- 172.21.15.47:0/3806554848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ed723883f0 con 0x55ed72713460 2024-03-21T11:24:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.324+0000 7eff2be00640 1 -- 172.21.15.47:0/3806554848 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff300be280 con 0x55ed72713460 2024-03-21T11:24:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.324+0000 7eff2be00640 1 -- 172.21.15.47:0/3806554848 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7eff38010430 con 0x55ed72713460 2024-03-21T11:24:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.324+0000 7eff4fba5740 10 client.25670 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.324+0000 7eff4fba5740 10 client.25670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.324+0000 7eff4fba5740 20 client.25670 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.324+0000 7eff4fba5740 1 --2- 172.21.15.47:0/3806554848 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed72728ea0 0x55ed7272b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.324+0000 7eff4fba5740 4 client.25670 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.324+0000 7eff4fba5740 1 -- 172.21.15.47:0/3806554848 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed72681790 con 0x55ed72728ea0 2024-03-21T11:24:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.324+0000 7eff3fe00640 1 --2- 172.21.15.47:0/3806554848 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed72728ea0 0x55ed7272b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.325+0000 7eff3fe00640 1 --2- 172.21.15.47:0/3806554848 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed72728ea0 0x55ed7272b340 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b9b8ac9f36844 server_cookie=bc3e5771d0c76b3c in_seq=0 out_seq=0 2024-03-21T11:24:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.325+0000 7eff2be00640 10 client.25670 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.325+0000 7eff2be00640 1 -- 172.21.15.47:0/3806554848 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ed72681790 con 0x55ed72728ea0 2024-03-21T11:24:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.326+0000 7eff2be00640 10 client.25670 handle_command_reply: tid=0 2024-03-21T11:24:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.326+0000 7eff4fba5740 1 client.25670 shutdown 2024-03-21T11:24:20.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.326+0000 7eff4fba5740 1 -- 172.21.15.47:0/3806554848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713460 msgr2=0x55ed7271d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.326+0000 7eff4fba5740 1 --2- 172.21.15.47:0/3806554848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713460 0x55ed7271d0f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55ed72714250 tx=0x7eff3800f790 comp rx=0 tx=0).stop 2024-03-21T11:24:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.326+0000 7eff4fba5740 1 -- 172.21.15.47:0/3806554848 shutdown_connections 2024-03-21T11:24:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.326+0000 7eff4fba5740 1 --2- 172.21.15.47:0/3806554848 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed72728ea0 0x55ed7272b340 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.326+0000 7eff4fba5740 1 --2- 172.21.15.47:0/3806554848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed72724670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.326+0000 7eff4fba5740 1 --2- 172.21.15.47:0/3806554848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713e50 0x55ed7271d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.326+0000 7eff4fba5740 1 --2- 172.21.15.47:0/3806554848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713460 0x55ed7271d0f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.326+0000 7eff2be00640 0 client.25670 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.326+0000 7eff4fba5740 1 -- 172.21.15.47:0/3806554848 >> 172.21.15.47:0/3806554848 conn(0x55ed726d7200 msgr2=0x55ed72709bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.326+0000 7eff4fba5740 1 -- 172.21.15.47:0/3806554848 shutdown_connections 2024-03-21T11:24:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.326+0000 7eff4fba5740 1 -- 172.21.15.47:0/3806554848 wait complete. 2024-03-21T11:24:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.326+0000 7eff4fba5740 20 client.25670 trim_cache size 0 max 16384 2024-03-21T11:24:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.344+0000 7eff4fba5740 1 Processor -- start 2024-03-21T11:24:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.344+0000 7eff4fba5740 1 -- start start 2024-03-21T11:24:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.344+0000 7eff4fba5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72713460 0x55ed72714870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.344+0000 7eff4fba5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713e50 0x55ed72724630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.344+0000 7eff4fba5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72718fc0 0x55ed7272fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.344+0000 7eff4fba5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff481a7d30 con 0x55ed72718fc0 2024-03-21T11:24:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.344+0000 7eff4fba5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff481a96b0 con 0x55ed72713e50 2024-03-21T11:24:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.344+0000 7eff4fba5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff481a8390 con 0x55ed72713460 2024-03-21T11:24:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.345+0000 7eff45e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72713460 0x55ed72714870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.345+0000 7eff45e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72713460 0x55ed72714870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60962/0 (socket says 172.21.15.47:60962) 2024-03-21T11:24:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.345+0000 7eff45e00640 1 -- 172.21.15.47:0/2334032288 learned_addr learned my addr 172.21.15.47:0/2334032288 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.345+0000 7eff3fe00640 1 --2- 172.21.15.47:0/2334032288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713e50 0x55ed72724630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.345+0000 7eff46800640 1 --2- 172.21.15.47:0/2334032288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72718fc0 0x55ed7272fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.345+0000 7eff45e00640 1 -- 172.21.15.47:0/2334032288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713e50 msgr2=0x55ed72724630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.345+0000 7eff45e00640 1 --2- 172.21.15.47:0/2334032288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713e50 0x55ed72724630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.345+0000 7eff45e00640 1 -- 172.21.15.47:0/2334032288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72718fc0 msgr2=0x55ed7272fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.345+0000 7eff45e00640 1 --2- 172.21.15.47:0/2334032288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72718fc0 0x55ed7272fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.345+0000 7eff45e00640 1 -- 172.21.15.47:0/2334032288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed723480f0 con 0x55ed72713460 2024-03-21T11:24:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.345+0000 7eff45e00640 1 --2- 172.21.15.47:0/2334032288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72713460 0x55ed72714870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7eff380107d0 tx=0x7eff38010c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9933aac5b97f67f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.346+0000 7eff36a00640 1 -- 172.21.15.47:0/2334032288 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38012110 con 0x55ed72713460 2024-03-21T11:24:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.346+0000 7eff36a00640 1 -- 172.21.15.47:0/2334032288 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff38010430 con 0x55ed72713460 2024-03-21T11:24:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.346+0000 7eff36a00640 1 -- 172.21.15.47:0/2334032288 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38003750 con 0x55ed72713460 2024-03-21T11:24:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.346+0000 7eff4fba5740 1 -- 172.21.15.47:0/2334032288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72713460 msgr2=0x55ed72714870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.346+0000 7eff4fba5740 1 --2- 172.21.15.47:0/2334032288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72713460 0x55ed72714870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7eff380107d0 tx=0x7eff38010c50 comp rx=0 tx=0).stop 2024-03-21T11:24:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.346+0000 7eff4fba5740 1 -- 172.21.15.47:0/2334032288 shutdown_connections 2024-03-21T11:24:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.346+0000 7eff4fba5740 1 --2- 172.21.15.47:0/2334032288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72718fc0 0x55ed7272fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.346+0000 7eff4fba5740 1 --2- 172.21.15.47:0/2334032288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713e50 0x55ed72724630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.346+0000 7eff4fba5740 1 --2- 172.21.15.47:0/2334032288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72713460 0x55ed72714870 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.346+0000 7eff4fba5740 1 -- 172.21.15.47:0/2334032288 >> 172.21.15.47:0/2334032288 conn(0x55ed726d7200 msgr2=0x55ed7270e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.346+0000 7eff4fba5740 1 -- 172.21.15.47:0/2334032288 shutdown_connections 2024-03-21T11:24:20.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.347+0000 7eff4fba5740 1 -- 172.21.15.47:0/2334032288 wait complete. 2024-03-21T11:24:20.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.347+0000 7eff4fba5740 1 -- start start 2024-03-21T11:24:20.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.347+0000 7eff4fba5740 1 Processor -- start 2024-03-21T11:24:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff4fba5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713460 0x55ed727220a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff4fba5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713e50 0x55ed727225e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff4fba5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed72723530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff4fba5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff481a9050 con 0x55ed72713e50 2024-03-21T11:24:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff4fba5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff481a89f0 con 0x55ed72713460 2024-03-21T11:24:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff4fba5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff481a5ef0 con 0x55ed72718fc0 2024-03-21T11:24:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff4fba5740 10 client.0 _ll_register_callbacks cb 0x7eff38009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff45e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713460 0x55ed727220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff46800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed72723530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff3fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713e50 0x55ed727225e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff46800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed72723530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60976/0 (socket says 172.21.15.47:60976) 2024-03-21T11:24:20.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff46800640 1 -- 172.21.15.47:0/2961796270 learned_addr learned my addr 172.21.15.47:0/2961796270 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff46800640 1 -- 172.21.15.47:0/2961796270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713460 msgr2=0x55ed727220a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff46800640 1 --2- 172.21.15.47:0/2961796270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713460 0x55ed727220a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff46800640 1 -- 172.21.15.47:0/2961796270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713e50 msgr2=0x55ed727225e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff46800640 1 --2- 172.21.15.47:0/2961796270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713e50 0x55ed727225e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff46800640 1 -- 172.21.15.47:0/2961796270 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed72723b20 con 0x55ed72718fc0 2024-03-21T11:24:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.348+0000 7eff45e00640 1 --2- 172.21.15.47:0/2961796270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713460 0x55ed727220a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.349+0000 7eff46800640 1 --2- 172.21.15.47:0/2961796270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed72723530 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7eff40011f30 tx=0x7eff40004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=471c6fa19b5938b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.349+0000 7eff2be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.349+0000 7eff2be00640 1 -- 172.21.15.47:0/2961796270 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4002e060 con 0x55ed72718fc0 2024-03-21T11:24:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.349+0000 7eff2be00640 1 -- 172.21.15.47:0/2961796270 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7eff40020b30 con 0x55ed72718fc0 2024-03-21T11:24:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.349+0000 7eff4fba5740 1 -- 172.21.15.47:0/2961796270 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff38010430 con 0x55ed72718fc0 2024-03-21T11:24:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.349+0000 7eff2be00640 1 -- 172.21.15.47:0/2961796270 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff40035050 con 0x55ed72718fc0 2024-03-21T11:24:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.349+0000 7eff4fba5740 10 client.25690 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.349+0000 7eff4fba5740 1 -- 172.21.15.47:0/2961796270 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ed72784b50 con 0x55ed72718fc0 2024-03-21T11:24:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.349+0000 7eff2be00640 1 -- 172.21.15.47:0/2961796270 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff38010430 con 0x55ed72718fc0 2024-03-21T11:24:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.350+0000 7eff2be00640 1 -- 172.21.15.47:0/2961796270 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7eff40028070 con 0x55ed72718fc0 2024-03-21T11:24:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.350+0000 7eff4fba5740 10 client.25690 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.350+0000 7eff4fba5740 10 client.25690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.350+0000 7eff4fba5740 20 client.25690 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.350+0000 7eff4fba5740 1 --2- 172.21.15.47:0/2961796270 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed72785530 0x55ed72787950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.350+0000 7eff4fba5740 4 client.25690 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:24:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.350+0000 7eff4fba5740 1 -- 172.21.15.47:0/2961796270 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ed7267bae0 con 0x55ed72785530 2024-03-21T11:24:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.350+0000 7eff45e00640 1 --2- 172.21.15.47:0/2961796270 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed72785530 0x55ed72787950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff45e00640 1 --2- 172.21.15.47:0/2961796270 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed72785530 0x55ed72787950 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e447075e5569d76 server_cookie=c7324bed51679652 in_seq=0 out_seq=0 2024-03-21T11:24:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff2be00640 10 client.25690 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff2be00640 1 -- 172.21.15.47:0/2961796270 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ed7267bae0 con 0x55ed72785530 2024-03-21T11:24:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff2be00640 10 client.25690 handle_command_reply: tid=0 2024-03-21T11:24:20.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff4fba5740 1 client.25690 shutdown 2024-03-21T11:24:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff4fba5740 1 -- 172.21.15.47:0/2961796270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 msgr2=0x55ed72723530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff4fba5740 1 --2- 172.21.15.47:0/2961796270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed72723530 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7eff40011f30 tx=0x7eff40004a40 comp rx=0 tx=0).stop 2024-03-21T11:24:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff4fba5740 1 -- 172.21.15.47:0/2961796270 shutdown_connections 2024-03-21T11:24:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff4fba5740 1 --2- 172.21.15.47:0/2961796270 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed72785530 0x55ed72787950 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff4fba5740 1 --2- 172.21.15.47:0/2961796270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed72718fc0 0x55ed72723530 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff4fba5740 1 --2- 172.21.15.47:0/2961796270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed72713e50 0x55ed727225e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff4fba5740 1 --2- 172.21.15.47:0/2961796270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed72713460 0x55ed727220a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff2be00640 0 client.25690 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff4fba5740 1 -- 172.21.15.47:0/2961796270 >> 172.21.15.47:0/2961796270 conn(0x55ed726d7200 msgr2=0x55ed7270e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff4fba5740 1 -- 172.21.15.47:0/2961796270 shutdown_connections 2024-03-21T11:24:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff4fba5740 1 -- 172.21.15.47:0/2961796270 wait complete. 2024-03-21T11:24:20.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.351+0000 7eff4fba5740 20 client.25690 trim_cache size 0 max 16384 2024-03-21T11:24:20.357 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:24:20.357 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:24:20.357 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:24:20.357 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:24:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.352+0000 7eff47200640 1 -- 172.21.15.47:0/3040411102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff2c0a3030 msgr2=0x7eff2c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.352+0000 7eff47200640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff2c0a3030 0x7eff2c0a54f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55ed727146b0 tx=0x7eff40009600 comp rx=0 tx=0).stop 2024-03-21T11:24:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.352+0000 7eff47200640 1 -- 172.21.15.47:0/3040411102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff48104930 msgr2=0x7eff48197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.352+0000 7eff47200640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff48104930 0x7eff48197e00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7eff30000c00 tx=0x7eff3002c6c0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.352+0000 7eff47200640 1 -- 172.21.15.47:0/3040411102 shutdown_connections 2024-03-21T11:24:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.352+0000 7eff47200640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff2c0a3030 0x7eff2c0a54f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.352+0000 7eff47200640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff48107260 0x7eff4819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.352+0000 7eff47200640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff48104930 0x7eff48197e00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.352+0000 7eff47200640 1 --2- 172.21.15.47:0/3040411102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff48102000 0x7eff481978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.352+0000 7eff47200640 1 -- 172.21.15.47:0/3040411102 >> 172.21.15.47:0/3040411102 conn(0x7eff480fba10 msgr2=0x7eff481028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.352+0000 7eff47200640 1 -- 172.21.15.47:0/3040411102 shutdown_connections 2024-03-21T11:24:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.352+0000 7eff47200640 1 -- 172.21.15.47:0/3040411102 wait complete. 2024-03-21T11:24:20.370 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:24:20.370 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:24:20.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.602+0000 7ff746a00640 1 -- 172.21.15.47:0/1594535110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7401093a0 msgr2=0x7ff74010b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:20.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.602+0000 7ff746a00640 1 --2- 172.21.15.47:0/1594535110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7401093a0 0x7ff74010b830 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff738009be0 tx=0x7ff73802bef0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.602+0000 7ff746a00640 1 -- 172.21.15.47:0/1594535110 shutdown_connections 2024-03-21T11:24:20.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.602+0000 7ff746a00640 1 --2- 172.21.15.47:0/1594535110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7401093a0 0x7ff74010b830 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.602+0000 7ff746a00640 1 --2- 172.21.15.47:0/1594535110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff740074100 0x7ff740074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.602+0000 7ff746a00640 1 --2- 172.21.15.47:0/1594535110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7400756e0 0x7ff740073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.602+0000 7ff746a00640 1 -- 172.21.15.47:0/1594535110 >> 172.21.15.47:0/1594535110 conn(0x7ff7400fca30 msgr2=0x7ff7400fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:20.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.602+0000 7ff746a00640 1 -- 172.21.15.47:0/1594535110 shutdown_connections 2024-03-21T11:24:20.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.602+0000 7ff746a00640 1 -- 172.21.15.47:0/1594535110 wait complete. 2024-03-21T11:24:20.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.603+0000 7ff746a00640 1 Processor -- start 2024-03-21T11:24:20.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.603+0000 7ff746a00640 1 -- start start 2024-03-21T11:24:20.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff746a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740074100 0x7ff740199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff746a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7400756e0 0x7ff740199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff746a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7401093a0 0x7ff7401a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff746a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74010f370 con 0x7ff7400756e0 2024-03-21T11:24:20.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff746a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74010f1f0 con 0x7ff740074100 2024-03-21T11:24:20.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff746a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74010f4f0 con 0x7ff7401093a0 2024-03-21T11:24:20.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff745600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740074100 0x7ff740199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff746000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7401093a0 0x7ff7401a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff744c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7400756e0 0x7ff740199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff745600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740074100 0x7ff740199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51464/0 (socket says 172.21.15.47:51464) 2024-03-21T11:24:20.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff745600640 1 -- 172.21.15.47:0/2167263737 learned_addr learned my addr 172.21.15.47:0/2167263737 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:20.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff746000640 1 -- 172.21.15.47:0/2167263737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740074100 msgr2=0x7ff740199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:20.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff746000640 1 --2- 172.21.15.47:0/2167263737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740074100 0x7ff740199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff746000640 1 -- 172.21.15.47:0/2167263737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7400756e0 msgr2=0x7ff740199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:20.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff746000640 1 --2- 172.21.15.47:0/2167263737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7400756e0 0x7ff740199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff746000640 1 -- 172.21.15.47:0/2167263737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7401a15b0 con 0x7ff7401093a0 2024-03-21T11:24:20.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff745600640 1 --2- 172.21.15.47:0/2167263737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740074100 0x7ff740199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:20.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.604+0000 7ff744c00640 1 --2- 172.21.15.47:0/2167263737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7400756e0 0x7ff740199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:20.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.605+0000 7ff746000640 1 --2- 172.21.15.47:0/2167263737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7401093a0 0x7ff7401a0f40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff73802c430 tx=0x7ff7380337b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:20.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.605+0000 7ff73e000640 1 -- 172.21.15.47:0/2167263737 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738043020 con 0x7ff7401093a0 2024-03-21T11:24:20.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.605+0000 7ff73e000640 1 -- 172.21.15.47:0/2167263737 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff73802ca60 con 0x7ff7401093a0 2024-03-21T11:24:20.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.605+0000 7ff746a00640 1 -- 172.21.15.47:0/2167263737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7401a1840 con 0x7ff7401093a0 2024-03-21T11:24:20.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.605+0000 7ff73e000640 1 -- 172.21.15.47:0/2167263737 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738034690 con 0x7ff7401093a0 2024-03-21T11:24:20.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.605+0000 7ff746a00640 1 -- 172.21.15.47:0/2167263737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7401a1de0 con 0x7ff7401093a0 2024-03-21T11:24:20.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.606+0000 7ff746a00640 1 -- 172.21.15.47:0/2167263737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7401080d0 con 0x7ff7401093a0 2024-03-21T11:24:20.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.608+0000 7ff73e000640 1 -- 172.21.15.47:0/2167263737 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7ff738046020 con 0x7ff7401093a0 2024-03-21T11:24:20.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.609+0000 7ff73e000640 1 --2- 172.21.15.47:0/2167263737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7280a3030 0x7ff7280a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:20.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.610+0000 7ff73e000640 1 -- 172.21.15.47:0/2167263737 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff7380f2280 con 0x7ff7401093a0 2024-03-21T11:24:20.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.610+0000 7ff745600640 1 --2- 172.21.15.47:0/2167263737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7280a3030 0x7ff7280a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:20.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.610+0000 7ff745600640 1 --2- 172.21.15.47:0/2167263737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7280a3030 0x7ff7280a54f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff7340058c0 tx=0x7ff734009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:20.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.616+0000 7ff73e000640 1 -- 172.21.15.47:0/2167263737 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff73804d020 con 0x7ff7401093a0 2024-03-21T11:24:20.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.864+0000 7ff746a00640 1 -- 172.21.15.47:0/2167263737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff74002d070 con 0x7ff7401093a0 2024-03-21T11:24:20.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.865+0000 7ff73e000640 1 -- 172.21.15.47:0/2167263737 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7ff7380bec70 con 0x7ff7401093a0 2024-03-21T11:24:20.866 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:24:20.866 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:24:20.866 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:24:20.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.868+0000 7ff746a00640 1 -- 172.21.15.47:0/2167263737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7280a3030 msgr2=0x7ff7280a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:20.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.868+0000 7ff746a00640 1 --2- 172.21.15.47:0/2167263737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7280a3030 0x7ff7280a54f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff7340058c0 tx=0x7ff734009260 comp rx=0 tx=0).stop 2024-03-21T11:24:20.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.868+0000 7ff746a00640 1 -- 172.21.15.47:0/2167263737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7401093a0 msgr2=0x7ff7401a0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:20.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.868+0000 7ff746a00640 1 --2- 172.21.15.47:0/2167263737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7401093a0 0x7ff7401a0f40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff73802c430 tx=0x7ff7380337b0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.868+0000 7ff746a00640 1 -- 172.21.15.47:0/2167263737 shutdown_connections 2024-03-21T11:24:20.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.868+0000 7ff746a00640 1 --2- 172.21.15.47:0/2167263737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7280a3030 0x7ff7280a54f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.868+0000 7ff746a00640 1 --2- 172.21.15.47:0/2167263737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7401093a0 0x7ff7401a0f40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.868+0000 7ff746a00640 1 --2- 172.21.15.47:0/2167263737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7400756e0 0x7ff740199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.868+0000 7ff746a00640 1 --2- 172.21.15.47:0/2167263737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740074100 0x7ff740199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:20.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.868+0000 7ff746a00640 1 -- 172.21.15.47:0/2167263737 >> 172.21.15.47:0/2167263737 conn(0x7ff7400fca30 msgr2=0x7ff7401052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:20.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.869+0000 7ff746a00640 1 -- 172.21.15.47:0/2167263737 shutdown_connections 2024-03-21T11:24:20.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:20.869+0000 7ff746a00640 1 -- 172.21.15.47:0/2167263737 wait complete. 2024-03-21T11:24:20.890 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:24:20.891 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:24:21.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.120+0000 7f398be00640 1 -- 172.21.15.47:0/1795221899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f398c109410 msgr2=0x7f398c10b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.120+0000 7f398be00640 1 --2- 172.21.15.47:0/1795221899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f398c109410 0x7f398c10b8b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3984009900 tx=0x7f398402bcc0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.120+0000 7f398be00640 1 -- 172.21.15.47:0/1795221899 shutdown_connections 2024-03-21T11:24:21.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.120+0000 7f398be00640 1 --2- 172.21.15.47:0/1795221899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f398c109410 0x7f398c10b8b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.120+0000 7f398be00640 1 --2- 172.21.15.47:0/1795221899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f398c0740a0 0x7f398c074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.120+0000 7f398be00640 1 --2- 172.21.15.47:0/1795221899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c075680 0x7f398c073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.120+0000 7f398be00640 1 -- 172.21.15.47:0/1795221899 >> 172.21.15.47:0/1795221899 conn(0x7f398c0fcb10 msgr2=0x7f398c0fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.120+0000 7f398be00640 1 -- 172.21.15.47:0/1795221899 shutdown_connections 2024-03-21T11:24:21.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.120+0000 7f398be00640 1 -- 172.21.15.47:0/1795221899 wait complete. 2024-03-21T11:24:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.121+0000 7f398be00640 1 Processor -- start 2024-03-21T11:24:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.121+0000 7f398be00640 1 -- start start 2024-03-21T11:24:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.122+0000 7f398be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f398c0740a0 0x7f398c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.122+0000 7f398be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c075680 0x7f398c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.122+0000 7f398be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f398c109410 0x7f398c1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.122+0000 7f398be00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c10f450 con 0x7f398c109410 2024-03-21T11:24:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.122+0000 7f398be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c10f2d0 con 0x7f398c075680 2024-03-21T11:24:21.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.122+0000 7f398be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c10f5d0 con 0x7f398c0740a0 2024-03-21T11:24:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.122+0000 7f3982000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c075680 0x7f398c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.122+0000 7f398aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f398c0740a0 0x7f398c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.122+0000 7f398aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f398c0740a0 0x7f398c199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60996/0 (socket says 172.21.15.47:60996) 2024-03-21T11:24:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.122+0000 7f398aa00640 1 -- 172.21.15.47:0/4067900439 learned_addr learned my addr 172.21.15.47:0/4067900439 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.122+0000 7f3982000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c075680 0x7f398c199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51480/0 (socket says 172.21.15.47:51480) 2024-03-21T11:24:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.122+0000 7f398b400640 1 --2- 172.21.15.47:0/4067900439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f398c109410 0x7f398c1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.123+0000 7f3982000640 1 -- 172.21.15.47:0/4067900439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f398c0740a0 msgr2=0x7f398c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.123+0000 7f3982000640 1 --2- 172.21.15.47:0/4067900439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f398c0740a0 0x7f398c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.123+0000 7f3982000640 1 -- 172.21.15.47:0/4067900439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f398c109410 msgr2=0x7f398c1a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.123+0000 7f3982000640 1 --2- 172.21.15.47:0/4067900439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f398c109410 0x7f398c1a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.123+0000 7f3982000640 1 -- 172.21.15.47:0/4067900439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c1a1640 con 0x7f398c075680 2024-03-21T11:24:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.123+0000 7f398aa00640 1 --2- 172.21.15.47:0/4067900439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f398c0740a0 0x7f398c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.123+0000 7f3982000640 1 --2- 172.21.15.47:0/4067900439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c075680 0x7f398c199f60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f397400d6d0 tx=0x7f397400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.123+0000 7f3983e00640 1 -- 172.21.15.47:0/4067900439 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974017070 con 0x7f398c075680 2024-03-21T11:24:21.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.123+0000 7f3983e00640 1 -- 172.21.15.47:0/4067900439 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3974009820 con 0x7f398c075680 2024-03-21T11:24:21.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.123+0000 7f3983e00640 1 -- 172.21.15.47:0/4067900439 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974004d00 con 0x7f398c075680 2024-03-21T11:24:21.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.123+0000 7f398be00640 1 -- 172.21.15.47:0/4067900439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c1a1930 con 0x7f398c075680 2024-03-21T11:24:21.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.123+0000 7f398be00640 1 -- 172.21.15.47:0/4067900439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c1a1e30 con 0x7f398c075680 2024-03-21T11:24:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.128+0000 7f3994623740 1 Processor -- start 2024-03-21T11:24:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.128+0000 7f3983e00640 1 -- 172.21.15.47:0/4067900439 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f39740099d0 con 0x7f398c075680 2024-03-21T11:24:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.128+0000 7f3994623740 1 -- start start 2024-03-21T11:24:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.128+0000 7f3994623740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 0x5634dc31be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.128+0000 7f3994623740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31cbc0 0x5634dc31d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.128+0000 7f3994623740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31d580 0x5634dc323e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.128+0000 7f3994623740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dc2802c0 con 0x5634dc31b9f0 2024-03-21T11:24:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.128+0000 7f3994623740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dc228c90 con 0x5634dc31d580 2024-03-21T11:24:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.128+0000 7f3994623740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dc27c360 con 0x5634dc31cbc0 2024-03-21T11:24:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.129+0000 7f398aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31cbc0 0x5634dc31d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.129+0000 7f398aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31cbc0 0x5634dc31d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32772/0 (socket says 172.21.15.47:32772) 2024-03-21T11:24:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.129+0000 7f3982000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31d580 0x5634dc323e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.129+0000 7f398b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 0x5634dc31be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.129+0000 7f398aa00640 1 -- 172.21.15.47:0/3022864246 learned_addr learned my addr 172.21.15.47:0/3022864246 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.129+0000 7f398aa00640 1 -- 172.21.15.47:0/3022864246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31d580 msgr2=0x5634dc323e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.129+0000 7f398aa00640 1 --2- 172.21.15.47:0/3022864246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31d580 0x5634dc323e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.129+0000 7f398aa00640 1 -- 172.21.15.47:0/3022864246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 msgr2=0x5634dc31be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.129+0000 7f398aa00640 1 --2- 172.21.15.47:0/3022864246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 0x5634dc31be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.129+0000 7f398aa00640 1 -- 172.21.15.47:0/3022864246 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634dc24d080 con 0x5634dc31cbc0 2024-03-21T11:24:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.129+0000 7f398aa00640 1 --2- 172.21.15.47:0/3022864246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31cbc0 0x5634dc31d040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f397c011930 tx=0x7f397c011df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57214e1fba9c5539 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.130+0000 7f3983e00640 1 --2- 172.21.15.47:0/4067900439 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39700a2fb0 0x7f39700a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.130+0000 7f397be00640 1 -- 172.21.15.47:0/3022864246 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c011330 con 0x5634dc31cbc0 2024-03-21T11:24:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.130+0000 7f397be00640 1 -- 172.21.15.47:0/3022864246 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f397c010430 con 0x5634dc31cbc0 2024-03-21T11:24:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.130+0000 7f397be00640 1 -- 172.21.15.47:0/3022864246 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c019600 con 0x5634dc31cbc0 2024-03-21T11:24:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.130+0000 7f3983e00640 1 -- 172.21.15.47:0/4067900439 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f39740cf370 con 0x7f398c075680 2024-03-21T11:24:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.130+0000 7f398b400640 1 --2- 172.21.15.47:0/4067900439 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39700a2fb0 0x7f39700a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.130+0000 7f3994623740 1 -- 172.21.15.47:0/3022864246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31cbc0 msgr2=0x5634dc31d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.130+0000 7f3994623740 1 --2- 172.21.15.47:0/3022864246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31cbc0 0x5634dc31d040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f397c011930 tx=0x7f397c011df0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.130+0000 7f3994623740 1 -- 172.21.15.47:0/3022864246 shutdown_connections 2024-03-21T11:24:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.130+0000 7f3994623740 1 --2- 172.21.15.47:0/3022864246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31d580 0x5634dc323e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.130+0000 7f3994623740 1 --2- 172.21.15.47:0/3022864246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31cbc0 0x5634dc31d040 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.130+0000 7f3994623740 1 --2- 172.21.15.47:0/3022864246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 0x5634dc31be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.130+0000 7f3994623740 1 -- 172.21.15.47:0/3022864246 >> 172.21.15.47:0/3022864246 conn(0x5634dc316de0 msgr2=0x5634dc3171e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.131+0000 7f3994623740 1 -- 172.21.15.47:0/3022864246 shutdown_connections 2024-03-21T11:24:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.131+0000 7f3994623740 1 -- 172.21.15.47:0/3022864246 wait complete. 2024-03-21T11:24:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.131+0000 7f398b400640 1 --2- 172.21.15.47:0/4067900439 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39700a2fb0 0x7f39700a5470 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3984036040 tx=0x7f398402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.131+0000 7f3994623740 1 -- start start 2024-03-21T11:24:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.131+0000 7f3994623740 1 Processor -- start 2024-03-21T11:24:21.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3994623740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 0x5634dc325bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3994623740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31cbc0 0x5634dc3260f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3994623740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31d580 0x5634dc32d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3994623740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dc324590 con 0x5634dc31b9f0 2024-03-21T11:24:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3994623740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dc23c0d0 con 0x5634dc31d580 2024-03-21T11:24:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3994623740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634dc324710 con 0x5634dc31cbc0 2024-03-21T11:24:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3994623740 10 client.0 _ll_register_callbacks cb 0x5634dc240e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3982000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31cbc0 0x5634dc3260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f398b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31d580 0x5634dc32d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3982000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31cbc0 0x5634dc3260f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32784/0 (socket says 172.21.15.47:32784) 2024-03-21T11:24:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3982000640 1 -- 172.21.15.47:0/1086269395 learned_addr learned my addr 172.21.15.47:0/1086269395 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f398b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31d580 0x5634dc32d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51506/0 (socket says 172.21.15.47:51506) 2024-03-21T11:24:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f398aa00640 1 --2- 172.21.15.47:0/1086269395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 0x5634dc325bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3982000640 1 -- 172.21.15.47:0/1086269395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31d580 msgr2=0x5634dc32d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3982000640 1 --2- 172.21.15.47:0/1086269395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31d580 0x5634dc32d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3982000640 1 -- 172.21.15.47:0/1086269395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 msgr2=0x5634dc325bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3982000640 1 --2- 172.21.15.47:0/1086269395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 0x5634dc325bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.132+0000 7f3982000640 1 -- 172.21.15.47:0/1086269395 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634dbf943f0 con 0x5634dc31cbc0 2024-03-21T11:24:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.133+0000 7f3982000640 1 --2- 172.21.15.47:0/1086269395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31cbc0 0x5634dc3260f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f39740980d0 tx=0x7f3974004ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8813e8bf897221 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.133+0000 7f3978c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.133+0000 7f3978c00640 1 -- 172.21.15.47:0/1086269395 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974020070 con 0x5634dc31cbc0 2024-03-21T11:24:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.133+0000 7f3978c00640 1 -- 172.21.15.47:0/1086269395 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f39740ccc80 con 0x5634dc31cbc0 2024-03-21T11:24:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.133+0000 7f3994623740 1 -- 172.21.15.47:0/1086269395 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f397c010430 con 0x5634dc31cbc0 2024-03-21T11:24:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.133+0000 7f3978c00640 1 -- 172.21.15.47:0/1086269395 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f397409b240 con 0x5634dc31cbc0 2024-03-21T11:24:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.133+0000 7f3994623740 10 client.25708 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.133+0000 7f3994623740 1 -- 172.21.15.47:0/1086269395 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5634dc05e7d0 con 0x5634dc31cbc0 2024-03-21T11:24:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.133+0000 7f3978c00640 1 -- 172.21.15.47:0/1086269395 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f397c010430 con 0x5634dc31cbc0 2024-03-21T11:24:21.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.134+0000 7f3978c00640 1 -- 172.21.15.47:0/1086269395 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3974097d70 con 0x5634dc31cbc0 2024-03-21T11:24:21.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.134+0000 7f3994623740 10 client.25708 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:21.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.134+0000 7f3994623740 10 client.25708 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:24:21.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.134+0000 7f3994623740 20 client.25708 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:21.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.134+0000 7f3994623740 1 --2- 172.21.15.47:0/1086269395 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5634dc3314a0 0x5634dc3338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.134+0000 7f3994623740 4 client.25708 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.134+0000 7f3994623740 1 -- 172.21.15.47:0/1086269395 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634dc28d800 con 0x5634dc3314a0 2024-03-21T11:24:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.134+0000 7f398aa00640 1 --2- 172.21.15.47:0/1086269395 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5634dc3314a0 0x5634dc3338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.135+0000 7f398aa00640 1 --2- 172.21.15.47:0/1086269395 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5634dc3314a0 0x5634dc3338f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a5b009b4f9f3324 server_cookie=4161b94ae811f8b2 in_seq=0 out_seq=0 2024-03-21T11:24:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.135+0000 7f3978c00640 10 client.25708 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:24:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.135+0000 7f3978c00640 1 -- 172.21.15.47:0/1086269395 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5634dc28d800 con 0x5634dc3314a0 2024-03-21T11:24:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.135+0000 7f3978c00640 10 client.25708 handle_command_reply: tid=0 2024-03-21T11:24:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.135+0000 7f3994623740 1 client.25708 shutdown 2024-03-21T11:24:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.136+0000 7f3994623740 1 -- 172.21.15.47:0/1086269395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31cbc0 msgr2=0x5634dc3260f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.136+0000 7f3994623740 1 --2- 172.21.15.47:0/1086269395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31cbc0 0x5634dc3260f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f39740980d0 tx=0x7f3974004ea0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.136+0000 7f3994623740 1 -- 172.21.15.47:0/1086269395 shutdown_connections 2024-03-21T11:24:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.136+0000 7f3994623740 1 --2- 172.21.15.47:0/1086269395 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5634dc3314a0 0x5634dc3338f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.136+0000 7f3994623740 1 --2- 172.21.15.47:0/1086269395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31d580 0x5634dc32d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.136+0000 7f3994623740 1 --2- 172.21.15.47:0/1086269395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31cbc0 0x5634dc3260f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.136+0000 7f3994623740 1 --2- 172.21.15.47:0/1086269395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 0x5634dc325bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.136+0000 7f3978c00640 0 client.25708 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:24:21.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.136+0000 7f3994623740 1 -- 172.21.15.47:0/1086269395 >> 172.21.15.47:0/1086269395 conn(0x5634dc316de0 msgr2=0x5634dc316160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.136+0000 7f3994623740 1 -- 172.21.15.47:0/1086269395 shutdown_connections 2024-03-21T11:24:21.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.136+0000 7f3994623740 1 -- 172.21.15.47:0/1086269395 wait complete. 2024-03-21T11:24:21.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.136+0000 7f3994623740 20 client.25708 trim_cache size 0 max 16384 2024-03-21T11:24:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.150+0000 7f3994623740 1 Processor -- start 2024-03-21T11:24:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.150+0000 7f3994623740 1 -- start start 2024-03-21T11:24:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.150+0000 7f3994623740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31b9f0 0x5634dc315fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.150+0000 7f3994623740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31cbc0 0x5634dc32d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.150+0000 7f3994623740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31d580 0x5634dc338180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.150+0000 7f3994623740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c1ac030 con 0x5634dc31cbc0 2024-03-21T11:24:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.150+0000 7f3994623740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c1ad9b0 con 0x5634dc31b9f0 2024-03-21T11:24:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.150+0000 7f3994623740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c1ac690 con 0x5634dc31d580 2024-03-21T11:24:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.150+0000 7f398aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31b9f0 0x5634dc315fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.150+0000 7f398b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31d580 0x5634dc338180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.150+0000 7f398b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31d580 0x5634dc338180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32792/0 (socket says 172.21.15.47:32792) 2024-03-21T11:24:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.150+0000 7f398aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31b9f0 0x5634dc315fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51520/0 (socket says 172.21.15.47:51520) 2024-03-21T11:24:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.150+0000 7f398b400640 1 -- 172.21.15.47:0/2003092972 learned_addr learned my addr 172.21.15.47:0/2003092972 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.150+0000 7f3982000640 1 --2- 172.21.15.47:0/2003092972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31cbc0 0x5634dc32d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.151+0000 7f398aa00640 1 -- 172.21.15.47:0/2003092972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31d580 msgr2=0x5634dc338180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.151+0000 7f398aa00640 1 --2- 172.21.15.47:0/2003092972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31d580 0x5634dc338180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.151+0000 7f398aa00640 1 -- 172.21.15.47:0/2003092972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31cbc0 msgr2=0x5634dc32d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.151+0000 7f398aa00640 1 --2- 172.21.15.47:0/2003092972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31cbc0 0x5634dc32d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.151+0000 7f398aa00640 1 -- 172.21.15.47:0/2003092972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634dc324810 con 0x5634dc31b9f0 2024-03-21T11:24:21.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.151+0000 7f398aa00640 1 --2- 172.21.15.47:0/2003092972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31b9f0 0x5634dc315fe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f397c011900 tx=0x7f397c009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f767194445bf2b36 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.151+0000 7f397b400640 1 -- 172.21.15.47:0/2003092972 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c01f040 con 0x5634dc31b9f0 2024-03-21T11:24:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.151+0000 7f397b400640 1 -- 172.21.15.47:0/2003092972 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f397c010430 con 0x5634dc31b9f0 2024-03-21T11:24:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.151+0000 7f397b400640 1 -- 172.21.15.47:0/2003092972 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c010860 con 0x5634dc31b9f0 2024-03-21T11:24:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.152+0000 7f3994623740 1 -- 172.21.15.47:0/2003092972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31b9f0 msgr2=0x5634dc315fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.152+0000 7f3994623740 1 --2- 172.21.15.47:0/2003092972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31b9f0 0x5634dc315fe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f397c011900 tx=0x7f397c009230 comp rx=0 tx=0).stop 2024-03-21T11:24:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.152+0000 7f3994623740 1 -- 172.21.15.47:0/2003092972 shutdown_connections 2024-03-21T11:24:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.152+0000 7f3994623740 1 --2- 172.21.15.47:0/2003092972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31d580 0x5634dc338180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.152+0000 7f3994623740 1 --2- 172.21.15.47:0/2003092972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31cbc0 0x5634dc32d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.152+0000 7f3994623740 1 --2- 172.21.15.47:0/2003092972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31b9f0 0x5634dc315fe0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.152+0000 7f3994623740 1 -- 172.21.15.47:0/2003092972 >> 172.21.15.47:0/2003092972 conn(0x5634dc316de0 msgr2=0x5634dc3158d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.152+0000 7f3994623740 1 -- 172.21.15.47:0/2003092972 shutdown_connections 2024-03-21T11:24:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.152+0000 7f3994623740 1 -- 172.21.15.47:0/2003092972 wait complete. 2024-03-21T11:24:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.152+0000 7f3994623740 1 -- start start 2024-03-21T11:24:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.153+0000 7f3994623740 1 Processor -- start 2024-03-21T11:24:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.153+0000 7f3994623740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 0x5634dc32ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.153+0000 7f3994623740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31cbc0 0x5634dc32cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.153+0000 7f3994623740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31d580 0x5634dc3295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.153+0000 7f3994623740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c1ad350 con 0x5634dc31b9f0 2024-03-21T11:24:21.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.153+0000 7f3994623740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c1accf0 con 0x5634dc31cbc0 2024-03-21T11:24:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.153+0000 7f3994623740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c1aa1f0 con 0x5634dc31d580 2024-03-21T11:24:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.153+0000 7f3994623740 10 client.0 _ll_register_callbacks cb 0x7f39740972e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.153+0000 7f3982000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31cbc0 0x5634dc32cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.153+0000 7f398b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31d580 0x5634dc3295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.153+0000 7f398b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31d580 0x5634dc3295d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32798/0 (socket says 172.21.15.47:32798) 2024-03-21T11:24:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.153+0000 7f3982000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31cbc0 0x5634dc32cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51530/0 (socket says 172.21.15.47:51530) 2024-03-21T11:24:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.153+0000 7f398b400640 1 -- 172.21.15.47:0/2588639964 learned_addr learned my addr 172.21.15.47:0/2588639964 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.153+0000 7f398aa00640 1 --2- 172.21.15.47:0/2588639964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 0x5634dc32ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.154+0000 7f398b400640 1 -- 172.21.15.47:0/2588639964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31cbc0 msgr2=0x5634dc32cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.154+0000 7f398b400640 1 --2- 172.21.15.47:0/2588639964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31cbc0 0x5634dc32cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.154+0000 7f398b400640 1 -- 172.21.15.47:0/2588639964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 msgr2=0x5634dc32ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.154+0000 7f398b400640 1 --2- 172.21.15.47:0/2588639964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 0x5634dc32ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.154+0000 7f398b400640 1 -- 172.21.15.47:0/2588639964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634dc329e00 con 0x5634dc31d580 2024-03-21T11:24:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.154+0000 7f398aa00640 1 --2- 172.21.15.47:0/2588639964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 0x5634dc32ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.154+0000 7f398b400640 1 --2- 172.21.15.47:0/2588639964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31d580 0x5634dc3295d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f398402c840 tx=0x7f398402bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f30ebe38cdc4abe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.154+0000 7f3978c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.154+0000 7f3978c00640 1 -- 172.21.15.47:0/2588639964 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984004f40 con 0x5634dc31d580 2024-03-21T11:24:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.154+0000 7f3994623740 1 -- 172.21.15.47:0/2588639964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f397c010430 con 0x5634dc31d580 2024-03-21T11:24:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.154+0000 7f3978c00640 1 -- 172.21.15.47:0/2588639964 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f398403a040 con 0x5634dc31d580 2024-03-21T11:24:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.154+0000 7f3978c00640 1 -- 172.21.15.47:0/2588639964 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39840050e0 con 0x5634dc31d580 2024-03-21T11:24:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.154+0000 7f3994623740 10 client.25714 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.154+0000 7f3994623740 1 -- 172.21.15.47:0/2588639964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5634dc3962b0 con 0x5634dc31d580 2024-03-21T11:24:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.155+0000 7f3978c00640 1 -- 172.21.15.47:0/2588639964 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f397c010430 con 0x5634dc31d580 2024-03-21T11:24:21.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.155+0000 7f3978c00640 1 -- 172.21.15.47:0/2588639964 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3984003c70 con 0x5634dc31d580 2024-03-21T11:24:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.155+0000 7f3994623740 10 client.25714 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.155+0000 7f3994623740 10 client.25714 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:24:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.155+0000 7f3994623740 20 client.25714 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.155+0000 7f3994623740 1 --2- 172.21.15.47:0/2588639964 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5634dc396bb0 0x5634dc398fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.155+0000 7f3994623740 4 client.25714 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:24:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.155+0000 7f3994623740 1 -- 172.21.15.47:0/2588639964 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5634dc287b50 con 0x5634dc396bb0 2024-03-21T11:24:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.155+0000 7f398aa00640 1 --2- 172.21.15.47:0/2588639964 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5634dc396bb0 0x5634dc398fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f398aa00640 1 --2- 172.21.15.47:0/2588639964 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5634dc396bb0 0x5634dc398fd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ca387e285ce69120 server_cookie=eb6c14cbdbe62a97 in_seq=0 out_seq=0 2024-03-21T11:24:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3978c00640 10 client.25714 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:24:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3978c00640 1 -- 172.21.15.47:0/2588639964 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5634dc287b50 con 0x5634dc396bb0 2024-03-21T11:24:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3978c00640 10 client.25714 handle_command_reply: tid=0 2024-03-21T11:24:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3994623740 1 client.25714 shutdown 2024-03-21T11:24:21.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3994623740 1 -- 172.21.15.47:0/2588639964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31d580 msgr2=0x5634dc3295d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3994623740 1 --2- 172.21.15.47:0/2588639964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31d580 0x5634dc3295d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f398402c840 tx=0x7f398402bcc0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3994623740 1 -- 172.21.15.47:0/2588639964 shutdown_connections 2024-03-21T11:24:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3994623740 1 --2- 172.21.15.47:0/2588639964 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5634dc396bb0 0x5634dc398fd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3994623740 1 --2- 172.21.15.47:0/2588639964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634dc31d580 0x5634dc3295d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3994623740 1 --2- 172.21.15.47:0/2588639964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634dc31cbc0 0x5634dc32cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3994623740 1 --2- 172.21.15.47:0/2588639964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634dc31b9f0 0x5634dc32ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3994623740 1 -- 172.21.15.47:0/2588639964 >> 172.21.15.47:0/2588639964 conn(0x5634dc316de0 msgr2=0x5634dc316160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3978c00640 0 client.25714 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:24:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3994623740 1 -- 172.21.15.47:0/2588639964 shutdown_connections 2024-03-21T11:24:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3994623740 1 -- 172.21.15.47:0/2588639964 wait complete. 2024-03-21T11:24:21.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.156+0000 7f3994623740 20 client.25714 trim_cache size 0 max 16384 2024-03-21T11:24:21.163 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:24:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.157+0000 7f398be00640 1 -- 172.21.15.47:0/4067900439 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39700a2fb0 msgr2=0x7f39700a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.157+0000 7f398be00640 1 --2- 172.21.15.47:0/4067900439 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39700a2fb0 0x7f39700a5470 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3984036040 tx=0x7f398402e040 comp rx=0 tx=0).stop 2024-03-21T11:24:21.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.157+0000 7f398be00640 1 -- 172.21.15.47:0/4067900439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c075680 msgr2=0x7f398c199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.157+0000 7f398be00640 1 --2- 172.21.15.47:0/4067900439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c075680 0x7f398c199f60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f397400d6d0 tx=0x7f397400db90 comp rx=0 tx=0).stop 2024-03-21T11:24:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.157+0000 7f398be00640 1 -- 172.21.15.47:0/4067900439 shutdown_connections 2024-03-21T11:24:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.157+0000 7f398be00640 1 --2- 172.21.15.47:0/4067900439 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39700a2fb0 0x7f39700a5470 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.157+0000 7f398be00640 1 --2- 172.21.15.47:0/4067900439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f398c109410 0x7f398c1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.157+0000 7f398be00640 1 --2- 172.21.15.47:0/4067900439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c075680 0x7f398c199f60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.157+0000 7f398be00640 1 --2- 172.21.15.47:0/4067900439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f398c0740a0 0x7f398c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.157+0000 7f398be00640 1 -- 172.21.15.47:0/4067900439 >> 172.21.15.47:0/4067900439 conn(0x7f398c0fcb10 msgr2=0x7f398c1053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.157+0000 7f398be00640 1 -- 172.21.15.47:0/4067900439 shutdown_connections 2024-03-21T11:24:21.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.157+0000 7f398be00640 1 -- 172.21.15.47:0/4067900439 wait complete. 2024-03-21T11:24:21.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:20 smithi047 ceph-mon[31351]: pgmap v430: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-21T11:24:21.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:20 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2167263737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:24:21.175 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:24:21.175 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:24:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:20 smithi102 ceph-mon[36748]: pgmap v430: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-21T11:24:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:20 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2167263737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:24:21.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.380+0000 7f3976400640 1 -- 172.21.15.47:0/2189486178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3970102000 msgr2=0x7f39701043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.380+0000 7f3976400640 1 --2- 172.21.15.47:0/2189486178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3970102000 0x7f39701043f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3960009ca0 tx=0x7f396002c0f0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.380+0000 7f3976400640 1 -- 172.21.15.47:0/2189486178 shutdown_connections 2024-03-21T11:24:21.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.380+0000 7f3976400640 1 --2- 172.21.15.47:0/2189486178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3970107260 0x7f39701096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.380+0000 7f3976400640 1 --2- 172.21.15.47:0/2189486178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3970104930 0x7f3970106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.380+0000 7f3976400640 1 --2- 172.21.15.47:0/2189486178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3970102000 0x7f39701043f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.380+0000 7f3976400640 1 -- 172.21.15.47:0/2189486178 >> 172.21.15.47:0/2189486178 conn(0x7f39700fba10 msgr2=0x7f39700fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.380+0000 7f3976400640 1 -- 172.21.15.47:0/2189486178 shutdown_connections 2024-03-21T11:24:21.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.380+0000 7f3976400640 1 -- 172.21.15.47:0/2189486178 wait complete. 2024-03-21T11:24:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.381+0000 7f3976400640 1 Processor -- start 2024-03-21T11:24:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.381+0000 7f3976400640 1 -- start start 2024-03-21T11:24:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3976400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3970102000 0x7f3970199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3976400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3970104930 0x7f397019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3976400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3970107260 0x7f39701a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3976400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397010d210 con 0x7f3970107260 2024-03-21T11:24:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3976400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397010d090 con 0x7f3970104930 2024-03-21T11:24:21.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3976400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397010d390 con 0x7f3970102000 2024-03-21T11:24:21.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3975000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3970102000 0x7f3970199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3975000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3970102000 0x7f3970199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32824/0 (socket says 172.21.15.47:32824) 2024-03-21T11:24:21.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3975000640 1 -- 172.21.15.47:0/3559676042 learned_addr learned my addr 172.21.15.47:0/3559676042 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f396fe00640 1 --2- 172.21.15.47:0/3559676042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3970104930 0x7f397019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3975a00640 1 --2- 172.21.15.47:0/3559676042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3970107260 0x7f39701a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3975000640 1 -- 172.21.15.47:0/3559676042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3970104930 msgr2=0x7f397019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3975000640 1 --2- 172.21.15.47:0/3559676042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3970104930 0x7f397019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3975000640 1 -- 172.21.15.47:0/3559676042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3970107260 msgr2=0x7f39701a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3975000640 1 --2- 172.21.15.47:0/3559676042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3970107260 0x7f39701a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.382+0000 7f3975000640 1 -- 172.21.15.47:0/3559676042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39701a1700 con 0x7f3970102000 2024-03-21T11:24:21.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.383+0000 7f3975000640 1 --2- 172.21.15.47:0/3559676042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3970102000 0x7f3970199ae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f39600325f0 tx=0x7f3960032ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.383+0000 7f396d600640 1 -- 172.21.15.47:0/3559676042 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960039040 con 0x7f3970102000 2024-03-21T11:24:21.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.383+0000 7f396d600640 1 -- 172.21.15.47:0/3559676042 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3960044070 con 0x7f3970102000 2024-03-21T11:24:21.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.383+0000 7f3976400640 1 -- 172.21.15.47:0/3559676042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39701a1990 con 0x7f3970102000 2024-03-21T11:24:21.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.383+0000 7f396d600640 1 -- 172.21.15.47:0/3559676042 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960004280 con 0x7f3970102000 2024-03-21T11:24:21.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.383+0000 7f3976400640 1 -- 172.21.15.47:0/3559676042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39701a1ea0 con 0x7f3970102000 2024-03-21T11:24:21.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.386+0000 7f397a79b740 1 Processor -- start 2024-03-21T11:24:21.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.387+0000 7f396d600640 1 -- 172.21.15.47:0/3559676042 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f3960003970 con 0x7f3970102000 2024-03-21T11:24:21.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.387+0000 7f397a79b740 1 -- start start 2024-03-21T11:24:21.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.387+0000 7f397a79b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d09f0 0x5641c98d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.387+0000 7f397a79b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d1bc0 0x5641c98d2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.387+0000 7f397a79b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d2580 0x5641c98d8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.387+0000 7f397a79b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c98352c0 con 0x5641c98d2580 2024-03-21T11:24:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.387+0000 7f397a79b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c97ddc90 con 0x5641c98d09f0 2024-03-21T11:24:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.387+0000 7f397a79b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c9831360 con 0x5641c98d1bc0 2024-03-21T11:24:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.387+0000 7f3975000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d1bc0 0x5641c98d2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.387+0000 7f3975a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d09f0 0x5641c98d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.387+0000 7f3975a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d09f0 0x5641c98d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51562/0 (socket says 172.21.15.47:51562) 2024-03-21T11:24:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.387+0000 7f3975a00640 1 -- 172.21.15.47:0/752734537 learned_addr learned my addr 172.21.15.47:0/752734537 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.387+0000 7f396fe00640 1 --2- 172.21.15.47:0/752734537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d2580 0x5641c98d8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.388+0000 7f3975a00640 1 -- 172.21.15.47:0/752734537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d1bc0 msgr2=0x5641c98d2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.388+0000 7f3975a00640 1 --2- 172.21.15.47:0/752734537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d1bc0 0x5641c98d2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.388+0000 7f3975a00640 1 -- 172.21.15.47:0/752734537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d2580 msgr2=0x5641c98d8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.388+0000 7f3975a00640 1 --2- 172.21.15.47:0/752734537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d2580 0x5641c98d8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.388+0000 7f3975a00640 1 -- 172.21.15.47:0/752734537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641c9802080 con 0x5641c98d09f0 2024-03-21T11:24:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.388+0000 7f396fe00640 1 --2- 172.21.15.47:0/752734537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d2580 0x5641c98d8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.388+0000 7f3975a00640 1 --2- 172.21.15.47:0/752734537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d09f0 0x5641c98d0e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f396800dc30 tx=0x7f396800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ddb57e85f63c2fc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.388+0000 7f396d600640 1 --2- 172.21.15.47:0/3559676042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39540a3030 0x7f39540a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.388+0000 7f3966000640 1 -- 172.21.15.47:0/752734537 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968004030 con 0x5641c98d09f0 2024-03-21T11:24:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.388+0000 7f3966000640 1 -- 172.21.15.47:0/752734537 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39680041d0 con 0x5641c98d09f0 2024-03-21T11:24:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.388+0000 7f3966000640 1 -- 172.21.15.47:0/752734537 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968011660 con 0x5641c98d09f0 2024-03-21T11:24:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.388+0000 7f396d600640 1 -- 172.21.15.47:0/3559676042 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f39600f1a50 con 0x7f3970102000 2024-03-21T11:24:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.389+0000 7f3975000640 1 --2- 172.21.15.47:0/3559676042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39540a3030 0x7f39540a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.389+0000 7f397a79b740 1 -- 172.21.15.47:0/752734537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d09f0 msgr2=0x5641c98d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.389+0000 7f397a79b740 1 --2- 172.21.15.47:0/752734537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d09f0 0x5641c98d0e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f396800dc30 tx=0x7f396800f730 comp rx=0 tx=0).stop 2024-03-21T11:24:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.389+0000 7f397a79b740 1 -- 172.21.15.47:0/752734537 shutdown_connections 2024-03-21T11:24:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.389+0000 7f397a79b740 1 --2- 172.21.15.47:0/752734537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d2580 0x5641c98d8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.389+0000 7f397a79b740 1 --2- 172.21.15.47:0/752734537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d1bc0 0x5641c98d2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.389+0000 7f397a79b740 1 --2- 172.21.15.47:0/752734537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d09f0 0x5641c98d0e10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.389+0000 7f397a79b740 1 -- 172.21.15.47:0/752734537 >> 172.21.15.47:0/752734537 conn(0x5641c98cbde0 msgr2=0x5641c98cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.389+0000 7f397a79b740 1 -- 172.21.15.47:0/752734537 shutdown_connections 2024-03-21T11:24:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.389+0000 7f397a79b740 1 -- 172.21.15.47:0/752734537 wait complete. 2024-03-21T11:24:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.389+0000 7f3975000640 1 --2- 172.21.15.47:0/3559676042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39540a3030 0x7f39540a54f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f39600ba1a0 tx=0x7f39600f54a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.390+0000 7f397a79b740 1 -- start start 2024-03-21T11:24:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.390+0000 7f397a79b740 1 Processor -- start 2024-03-21T11:24:21.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:20 smithi100 ceph-mon[36245]: pgmap v430: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-21T11:24:21.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:20 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2167263737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:24:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.390+0000 7f397a79b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d09f0 0x5641c98dac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.390+0000 7f397a79b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d1bc0 0x5641c98db1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.390+0000 7f397a79b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d2580 0x5641c98e2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.390+0000 7f397a79b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c98d9590 con 0x5641c98d09f0 2024-03-21T11:24:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.390+0000 7f397a79b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c97f10d0 con 0x5641c98d1bc0 2024-03-21T11:24:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.390+0000 7f397a79b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c98d9710 con 0x5641c98d2580 2024-03-21T11:24:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.391+0000 7f397a79b740 10 client.0 _ll_register_callbacks cb 0x5641c97f5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.391+0000 7f396fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d1bc0 0x5641c98db1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.391+0000 7f396fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d1bc0 0x5641c98db1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51572/0 (socket says 172.21.15.47:51572) 2024-03-21T11:24:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.391+0000 7f396fe00640 1 -- 172.21.15.47:0/3964262366 learned_addr learned my addr 172.21.15.47:0/3964262366 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.391+0000 7f3975a00640 1 --2- 172.21.15.47:0/3964262366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d09f0 0x5641c98dac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.391+0000 7f3975a00640 1 --2- 172.21.15.47:0/3964262366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d2580 0x5641c98e2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.391+0000 7f396fe00640 1 -- 172.21.15.47:0/3964262366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d2580 msgr2=0x5641c98e2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.391+0000 7f396fe00640 1 --2- 172.21.15.47:0/3964262366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d2580 0x5641c98e2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.391+0000 7f396fe00640 1 -- 172.21.15.47:0/3964262366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d09f0 msgr2=0x5641c98dac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.391+0000 7f396fe00640 1 --2- 172.21.15.47:0/3964262366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d09f0 0x5641c98dac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.391+0000 7f396fe00640 1 -- 172.21.15.47:0/3964262366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641c95493f0 con 0x5641c98d1bc0 2024-03-21T11:24:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.391+0000 7f3975a00640 1 --2- 172.21.15.47:0/3964262366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d2580 0x5641c98e2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.391+0000 7f396fe00640 1 --2- 172.21.15.47:0/3964262366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d1bc0 0x5641c98db1d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f395c00baa0 tx=0x7f395c00bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1104992045ec39c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.391+0000 7f395aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:24:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.392+0000 7f395aa00640 1 -- 172.21.15.47:0/3964262366 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c01b640 con 0x5641c98d1bc0 2024-03-21T11:24:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.392+0000 7f397a79b740 1 -- 172.21.15.47:0/3964262366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39680041d0 con 0x5641c98d1bc0 2024-03-21T11:24:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.392+0000 7f395aa00640 1 -- 172.21.15.47:0/3964262366 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f395c01b7e0 con 0x5641c98d1bc0 2024-03-21T11:24:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.392+0000 7f395aa00640 1 -- 172.21.15.47:0/3964262366 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f395c024050 con 0x5641c98d1bc0 2024-03-21T11:24:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.392+0000 7f397a79b740 10 client.25712 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.392+0000 7f397a79b740 1 -- 172.21.15.47:0/3964262366 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5641c96137d0 con 0x5641c98d1bc0 2024-03-21T11:24:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.392+0000 7f395aa00640 1 -- 172.21.15.47:0/3964262366 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39680041d0 con 0x5641c98d1bc0 2024-03-21T11:24:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.392+0000 7f395aa00640 1 -- 172.21.15.47:0/3964262366 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f395c019020 con 0x5641c98d1bc0 2024-03-21T11:24:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.392+0000 7f397a79b740 10 client.25712 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.392+0000 7f397a79b740 10 client.25712 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.392+0000 7f397a79b740 20 client.25712 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.393+0000 7f397a79b740 1 --2- 172.21.15.47:0/3964262366 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5641c98e6650 0x5641c98e8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.393+0000 7f397a79b740 4 client.25712 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.393+0000 7f397a79b740 1 -- 172.21.15.47:0/3964262366 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641c9842800 con 0x5641c98e6650 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.393+0000 7f3975a00640 1 --2- 172.21.15.47:0/3964262366 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5641c98e6650 0x5641c98e8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.393+0000 7f3975a00640 1 --2- 172.21.15.47:0/3964262366 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5641c98e6650 0x5641c98e8a90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=136a35fdf654eafd server_cookie=5da1bb5953a05b03 in_seq=0 out_seq=0 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.393+0000 7f395aa00640 10 client.25712 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.394+0000 7f395aa00640 1 -- 172.21.15.47:0/3964262366 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5641c9842800 con 0x5641c98e6650 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.394+0000 7f395aa00640 10 client.25712 handle_command_reply: tid=0 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.394+0000 7f397a79b740 1 client.25712 shutdown 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.394+0000 7f397a79b740 1 -- 172.21.15.47:0/3964262366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d1bc0 msgr2=0x5641c98db1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.394+0000 7f397a79b740 1 --2- 172.21.15.47:0/3964262366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d1bc0 0x5641c98db1d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f395c00baa0 tx=0x7f395c00bf60 comp rx=0 tx=0).stop 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.394+0000 7f397a79b740 1 -- 172.21.15.47:0/3964262366 shutdown_connections 2024-03-21T11:24:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.394+0000 7f397a79b740 1 --2- 172.21.15.47:0/3964262366 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5641c98e6650 0x5641c98e8a90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.394+0000 7f397a79b740 1 --2- 172.21.15.47:0/3964262366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d2580 0x5641c98e2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.394+0000 7f397a79b740 1 --2- 172.21.15.47:0/3964262366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d1bc0 0x5641c98db1d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.394+0000 7f395aa00640 0 client.25712 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:24:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.394+0000 7f397a79b740 1 --2- 172.21.15.47:0/3964262366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d09f0 0x5641c98dac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.394+0000 7f397a79b740 1 -- 172.21.15.47:0/3964262366 >> 172.21.15.47:0/3964262366 conn(0x5641c98cbde0 msgr2=0x5641c98cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.394+0000 7f397a79b740 1 -- 172.21.15.47:0/3964262366 shutdown_connections 2024-03-21T11:24:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.395+0000 7f397a79b740 1 -- 172.21.15.47:0/3964262366 wait complete. 2024-03-21T11:24:21.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.395+0000 7f397a79b740 20 client.25712 trim_cache size 0 max 16384 2024-03-21T11:24:21.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.408+0000 7f397a79b740 1 Processor -- start 2024-03-21T11:24:21.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.409+0000 7f397a79b740 1 -- start start 2024-03-21T11:24:21.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.409+0000 7f397a79b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d09f0 0x5641c98cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.409+0000 7f397a79b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d1bc0 0x5641c98e21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.409+0000 7f397a79b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d2580 0x5641c98d4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.409+0000 7f397a79b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39701ac170 con 0x5641c98d2580 2024-03-21T11:24:21.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.409+0000 7f397a79b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39701adaf0 con 0x5641c98d1bc0 2024-03-21T11:24:21.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.409+0000 7f397a79b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39701ac7d0 con 0x5641c98d09f0 2024-03-21T11:24:21.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.409+0000 7f396fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d1bc0 0x5641c98e21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.409+0000 7f3975a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d09f0 0x5641c98cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.409+0000 7f396fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d1bc0 0x5641c98e21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51574/0 (socket says 172.21.15.47:51574) 2024-03-21T11:24:21.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.409+0000 7f3975a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d09f0 0x5641c98cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32838/0 (socket says 172.21.15.47:32838) 2024-03-21T11:24:21.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.409+0000 7f396fe00640 1 -- 172.21.15.47:0/654238294 learned_addr learned my addr 172.21.15.47:0/654238294 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.409+0000 7f3975a00640 1 --2- 172.21.15.47:0/654238294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d2580 0x5641c98d4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.409+0000 7f396fe00640 1 -- 172.21.15.47:0/654238294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d09f0 msgr2=0x5641c98cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.410+0000 7f396fe00640 1 --2- 172.21.15.47:0/654238294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d09f0 0x5641c98cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.410+0000 7f396fe00640 1 -- 172.21.15.47:0/654238294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d2580 msgr2=0x5641c98d4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.410+0000 7f396fe00640 1 --2- 172.21.15.47:0/654238294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d2580 0x5641c98d4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.410+0000 7f396fe00640 1 -- 172.21.15.47:0/654238294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641c98ccd50 con 0x5641c98d1bc0 2024-03-21T11:24:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.410+0000 7f3975a00640 1 --2- 172.21.15.47:0/654238294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d2580 0x5641c98d4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.410+0000 7f396fe00640 1 --2- 172.21.15.47:0/654238294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d1bc0 0x5641c98e21d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f395c001eb0 tx=0x7f395c0043e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7c16d23d8c66635 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.410+0000 7f3965600640 1 -- 172.21.15.47:0/654238294 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c011440 con 0x5641c98d1bc0 2024-03-21T11:24:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.410+0000 7f3965600640 1 -- 172.21.15.47:0/654238294 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f395c005040 con 0x5641c98d1bc0 2024-03-21T11:24:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.410+0000 7f3965600640 1 -- 172.21.15.47:0/654238294 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c004b20 con 0x5641c98d1bc0 2024-03-21T11:24:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.411+0000 7f397a79b740 1 -- 172.21.15.47:0/654238294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d1bc0 msgr2=0x5641c98e21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.411+0000 7f397a79b740 1 --2- 172.21.15.47:0/654238294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d1bc0 0x5641c98e21d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f395c001eb0 tx=0x7f395c0043e0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.411+0000 7f397a79b740 1 -- 172.21.15.47:0/654238294 shutdown_connections 2024-03-21T11:24:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.411+0000 7f397a79b740 1 --2- 172.21.15.47:0/654238294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d2580 0x5641c98d4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.411+0000 7f397a79b740 1 --2- 172.21.15.47:0/654238294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d1bc0 0x5641c98e21d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.411+0000 7f397a79b740 1 --2- 172.21.15.47:0/654238294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d09f0 0x5641c98cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.411+0000 7f397a79b740 1 -- 172.21.15.47:0/654238294 >> 172.21.15.47:0/654238294 conn(0x5641c98cbde0 msgr2=0x5641c98cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.411+0000 7f397a79b740 1 -- 172.21.15.47:0/654238294 shutdown_connections 2024-03-21T11:24:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.411+0000 7f397a79b740 1 -- 172.21.15.47:0/654238294 wait complete. 2024-03-21T11:24:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.411+0000 7f397a79b740 1 -- start start 2024-03-21T11:24:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.411+0000 7f397a79b740 1 Processor -- start 2024-03-21T11:24:21.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f397a79b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d09f0 0x5641c98de270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f397a79b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d1bc0 0x5641c98de7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f397a79b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d2580 0x5641c98e1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f397a79b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39701ad490 con 0x5641c98d1bc0 2024-03-21T11:24:21.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f397a79b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39701ace30 con 0x5641c98d2580 2024-03-21T11:24:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f397a79b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39701aa330 con 0x5641c98d09f0 2024-03-21T11:24:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f397a79b740 10 client.0 _ll_register_callbacks cb 0x7f395c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f3975a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d09f0 0x5641c98de270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f3975a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d09f0 0x5641c98de270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32850/0 (socket says 172.21.15.47:32850) 2024-03-21T11:24:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f3975a00640 1 -- 172.21.15.47:0/2084913124 learned_addr learned my addr 172.21.15.47:0/2084913124 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f3975a00640 1 --2- 172.21.15.47:0/2084913124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d2580 0x5641c98e1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f396fe00640 1 --2- 172.21.15.47:0/2084913124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d1bc0 0x5641c98de7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f3975a00640 1 -- 172.21.15.47:0/2084913124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d2580 msgr2=0x5641c98e1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f3975a00640 1 --2- 172.21.15.47:0/2084913124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d2580 0x5641c98e1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f3975a00640 1 -- 172.21.15.47:0/2084913124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d1bc0 msgr2=0x5641c98de7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f3975a00640 1 --2- 172.21.15.47:0/2084913124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d1bc0 0x5641c98de7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.412+0000 7f3975a00640 1 -- 172.21.15.47:0/2084913124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641c98e1d70 con 0x5641c98d09f0 2024-03-21T11:24:21.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.413+0000 7f3975a00640 1 --2- 172.21.15.47:0/2084913124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d09f0 0x5641c98de270 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f396800fcb0 tx=0x7f396800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81fe36d0298a9902 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.413+0000 7f395aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:21.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.413+0000 7f395aa00640 1 -- 172.21.15.47:0/2084913124 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968025040 con 0x5641c98d09f0 2024-03-21T11:24:21.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.413+0000 7f397a79b740 1 -- 172.21.15.47:0/2084913124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f395c005040 con 0x5641c98d09f0 2024-03-21T11:24:21.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.413+0000 7f395aa00640 1 -- 172.21.15.47:0/2084913124 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f39680055f0 con 0x5641c98d09f0 2024-03-21T11:24:21.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.413+0000 7f395aa00640 1 -- 172.21.15.47:0/2084913124 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39680035f0 con 0x5641c98d09f0 2024-03-21T11:24:21.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.413+0000 7f397a79b740 10 client.25732 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:21.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.413+0000 7f397a79b740 1 -- 172.21.15.47:0/2084913124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5641c994b0c0 con 0x5641c98d09f0 2024-03-21T11:24:21.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.413+0000 7f395aa00640 1 -- 172.21.15.47:0/2084913124 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f395c005040 con 0x5641c98d09f0 2024-03-21T11:24:21.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.414+0000 7f395aa00640 1 -- 172.21.15.47:0/2084913124 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3968003820 con 0x5641c98d09f0 2024-03-21T11:24:21.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.414+0000 7f397a79b740 10 client.25732 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:21.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.414+0000 7f397a79b740 10 client.25732 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:24:21.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.414+0000 7f397a79b740 20 client.25732 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:21.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.414+0000 7f397a79b740 1 --2- 172.21.15.47:0/2084913124 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5641c994b870 0x5641c994dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.414+0000 7f397a79b740 4 client.25732 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:24:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.414+0000 7f397a79b740 1 -- 172.21.15.47:0/2084913124 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5641c983cb50 con 0x5641c994b870 2024-03-21T11:24:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.414+0000 7f396fe00640 1 --2- 172.21.15.47:0/2084913124 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5641c994b870 0x5641c994dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.414+0000 7f396fe00640 1 --2- 172.21.15.47:0/2084913124 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5641c994b870 0x5641c994dc90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a3a2bf7f950c4e65 server_cookie=cd2172b7e962b417 in_seq=0 out_seq=0 2024-03-21T11:24:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.414+0000 7f395aa00640 10 client.25732 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:24:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f395aa00640 1 -- 172.21.15.47:0/2084913124 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5641c983cb50 con 0x5641c994b870 2024-03-21T11:24:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f395aa00640 10 client.25732 handle_command_reply: tid=0 2024-03-21T11:24:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f397a79b740 1 client.25732 shutdown 2024-03-21T11:24:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f397a79b740 1 -- 172.21.15.47:0/2084913124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d09f0 msgr2=0x5641c98de270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f397a79b740 1 --2- 172.21.15.47:0/2084913124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d09f0 0x5641c98de270 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f396800fcb0 tx=0x7f396800f730 comp rx=0 tx=0).stop 2024-03-21T11:24:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f397a79b740 1 -- 172.21.15.47:0/2084913124 shutdown_connections 2024-03-21T11:24:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f397a79b740 1 --2- 172.21.15.47:0/2084913124 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5641c994b870 0x5641c994dc90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f397a79b740 1 --2- 172.21.15.47:0/2084913124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5641c98d2580 0x5641c98e1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f397a79b740 1 --2- 172.21.15.47:0/2084913124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641c98d1bc0 0x5641c98de7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f395aa00640 0 client.25732 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:24:21.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f397a79b740 1 --2- 172.21.15.47:0/2084913124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641c98d09f0 0x5641c98de270 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f397a79b740 1 -- 172.21.15.47:0/2084913124 >> 172.21.15.47:0/2084913124 conn(0x5641c98cbde0 msgr2=0x5641c98cf8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f397a79b740 1 -- 172.21.15.47:0/2084913124 shutdown_connections 2024-03-21T11:24:21.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f397a79b740 1 -- 172.21.15.47:0/2084913124 wait complete. 2024-03-21T11:24:21.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.415+0000 7f397a79b740 20 client.25732 trim_cache size 0 max 16384 2024-03-21T11:24:21.421 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:24:21.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.416+0000 7f3976400640 1 -- 172.21.15.47:0/3559676042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39540a3030 msgr2=0x7f39540a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.416+0000 7f3976400640 1 --2- 172.21.15.47:0/3559676042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39540a3030 0x7f39540a54f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f39600ba1a0 tx=0x7f39600f54a0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.416+0000 7f3976400640 1 -- 172.21.15.47:0/3559676042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3970102000 msgr2=0x7f3970199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.416+0000 7f3976400640 1 --2- 172.21.15.47:0/3559676042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3970102000 0x7f3970199ae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f39600325f0 tx=0x7f3960032ec0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.416+0000 7f3976400640 1 -- 172.21.15.47:0/3559676042 shutdown_connections 2024-03-21T11:24:21.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.416+0000 7f3976400640 1 --2- 172.21.15.47:0/3559676042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39540a3030 0x7f39540a54f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.416+0000 7f3976400640 1 --2- 172.21.15.47:0/3559676042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3970107260 0x7f39701a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.416+0000 7f3976400640 1 --2- 172.21.15.47:0/3559676042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3970104930 0x7f397019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.416+0000 7f3976400640 1 --2- 172.21.15.47:0/3559676042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3970102000 0x7f3970199ae0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.416+0000 7f3976400640 1 -- 172.21.15.47:0/3559676042 >> 172.21.15.47:0/3559676042 conn(0x7f39700fba10 msgr2=0x7f3970101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.416+0000 7f3976400640 1 -- 172.21.15.47:0/3559676042 shutdown_connections 2024-03-21T11:24:21.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.416+0000 7f3976400640 1 -- 172.21.15.47:0/3559676042 wait complete. 2024-03-21T11:24:21.435 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:24:21.435 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:24:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.656+0000 7f59b7000640 1 -- 172.21.15.47:0/3355664891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b0105350 msgr2=0x7f59b010bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.656+0000 7f59b7000640 1 --2- 172.21.15.47:0/3355664891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b0105350 0x7f59b010bbe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009ca0 tx=0x7f59a802c100 comp rx=0 tx=0).stop 2024-03-21T11:24:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.656+0000 7f59b7000640 1 -- 172.21.15.47:0/3355664891 shutdown_connections 2024-03-21T11:24:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.656+0000 7f59b7000640 1 --2- 172.21.15.47:0/3355664891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b0105350 0x7f59b010bbe0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.656+0000 7f59b7000640 1 --2- 172.21.15.47:0/3355664891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b0104990 0x7f59b0104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.656+0000 7f59b7000640 1 --2- 172.21.15.47:0/3355664891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b01037a0 0x7f59b0103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.656+0000 7f59b7000640 1 -- 172.21.15.47:0/3355664891 >> 172.21.15.47:0/3355664891 conn(0x7f59b00fed10 msgr2=0x7f59b0101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.656+0000 7f59b7000640 1 -- 172.21.15.47:0/3355664891 shutdown_connections 2024-03-21T11:24:21.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.656+0000 7f59b7000640 1 -- 172.21.15.47:0/3355664891 wait complete. 2024-03-21T11:24:21.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.657+0000 7f59b7000640 1 Processor -- start 2024-03-21T11:24:21.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.657+0000 7f59b7000640 1 -- start start 2024-03-21T11:24:21.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b7000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b01037a0 0x7f59b0199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b7000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b0104990 0x7f59b0199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b7000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b0105350 0x7f59b01a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b7000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b010f420 con 0x7f59b0105350 2024-03-21T11:24:21.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b7000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b010f2a0 con 0x7f59b0104990 2024-03-21T11:24:21.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b7000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b010f5a0 con 0x7f59b01037a0 2024-03-21T11:24:21.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b01037a0 0x7f59b0199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b01037a0 0x7f59b0199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32860/0 (socket says 172.21.15.47:32860) 2024-03-21T11:24:21.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b5c00640 1 -- 172.21.15.47:0/1433594347 learned_addr learned my addr 172.21.15.47:0/1433594347 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b5200640 1 --2- 172.21.15.47:0/1433594347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b0104990 0x7f59b0199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b6600640 1 --2- 172.21.15.47:0/1433594347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b0105350 0x7f59b01a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b5c00640 1 -- 172.21.15.47:0/1433594347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b0104990 msgr2=0x7f59b0199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b5c00640 1 --2- 172.21.15.47:0/1433594347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b0104990 0x7f59b0199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b5c00640 1 -- 172.21.15.47:0/1433594347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b0105350 msgr2=0x7f59b01a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b5c00640 1 --2- 172.21.15.47:0/1433594347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b0105350 0x7f59b01a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.658+0000 7f59b5c00640 1 -- 172.21.15.47:0/1433594347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b01a1660 con 0x7f59b01037a0 2024-03-21T11:24:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.659+0000 7f59b5c00640 1 --2- 172.21.15.47:0/1433594347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b01037a0 0x7f59b0199a40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f59a400e9c0 tx=0x7f59a400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.659+0000 7f59ae000640 1 -- 172.21.15.47:0/1433594347 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a400cd80 con 0x7f59b01037a0 2024-03-21T11:24:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.659+0000 7f59ae000640 1 -- 172.21.15.47:0/1433594347 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59a40035f0 con 0x7f59b01037a0 2024-03-21T11:24:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.659+0000 7f59ae000640 1 -- 172.21.15.47:0/1433594347 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4010690 con 0x7f59b01037a0 2024-03-21T11:24:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.659+0000 7f59b7000640 1 -- 172.21.15.47:0/1433594347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b01a1950 con 0x7f59b01037a0 2024-03-21T11:24:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.659+0000 7f59b7000640 1 -- 172.21.15.47:0/1433594347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b01a1ef0 con 0x7f59b01037a0 2024-03-21T11:24:21.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.662+0000 7f59ae000640 1 -- 172.21.15.47:0/1433594347 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f59a40037a0 con 0x7f59b01037a0 2024-03-21T11:24:21.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.662+0000 7f59bb3b6740 1 Processor -- start 2024-03-21T11:24:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.663+0000 7f59bb3b6740 1 -- start start 2024-03-21T11:24:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.663+0000 7f59bb3b6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e209f0 0x55e127e20e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.663+0000 7f59bb3b6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e21bc0 0x55e127e22040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.663+0000 7f59bb3b6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e22580 0x55e127e28e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.663+0000 7f59bb3b6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e127d852c0 con 0x55e127e21bc0 2024-03-21T11:24:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.663+0000 7f59bb3b6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e127d2dc90 con 0x55e127e22580 2024-03-21T11:24:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.663+0000 7f59bb3b6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e127d81360 con 0x55e127e209f0 2024-03-21T11:24:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.663+0000 7f59b6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e209f0 0x55e127e20e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.663+0000 7f59b6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e209f0 0x55e127e20e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32868/0 (socket says 172.21.15.47:32868) 2024-03-21T11:24:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.663+0000 7f59b6600640 1 -- 172.21.15.47:0/455113505 learned_addr learned my addr 172.21.15.47:0/455113505 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.663+0000 7f59b5200640 1 --2- 172.21.15.47:0/455113505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e22580 0x55e127e28e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.663+0000 7f59b5c00640 1 --2- 172.21.15.47:0/455113505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e21bc0 0x55e127e22040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.664+0000 7f59b6600640 1 -- 172.21.15.47:0/455113505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e22580 msgr2=0x55e127e28e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.664+0000 7f59b6600640 1 --2- 172.21.15.47:0/455113505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e22580 0x55e127e28e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.664+0000 7f59b6600640 1 -- 172.21.15.47:0/455113505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e21bc0 msgr2=0x55e127e22040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.664+0000 7f59b6600640 1 --2- 172.21.15.47:0/455113505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e21bc0 0x55e127e22040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.664+0000 7f59b6600640 1 -- 172.21.15.47:0/455113505 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e127d52080 con 0x55e127e209f0 2024-03-21T11:24:21.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.664+0000 7f59b5c00640 1 --2- 172.21.15.47:0/455113505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e21bc0 0x55e127e22040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:21.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.664+0000 7f59ae000640 1 --2- 172.21.15.47:0/1433594347 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f59980a3030 0x7f59980a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.664+0000 7f59ae000640 1 -- 172.21.15.47:0/1433594347 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f59a40cfd90 con 0x7f59b01037a0 2024-03-21T11:24:21.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.664+0000 7f59b6600640 1 --2- 172.21.15.47:0/455113505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e209f0 0x55e127e20e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59a8005e30 tx=0x7f59a80323d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=284910088853a894 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.664+0000 7f59b5c00640 1 --2- 172.21.15.47:0/1433594347 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f59980a3030 0x7f59980a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.664+0000 7f59a2a00640 1 -- 172.21.15.47:0/455113505 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8043020 con 0x55e127e209f0 2024-03-21T11:24:21.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.665+0000 7f59a2a00640 1 -- 172.21.15.47:0/455113505 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59a8032c90 con 0x55e127e209f0 2024-03-21T11:24:21.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.665+0000 7f59a2a00640 1 -- 172.21.15.47:0/455113505 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8039040 con 0x55e127e209f0 2024-03-21T11:24:21.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.665+0000 7f59bb3b6740 1 -- 172.21.15.47:0/455113505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e209f0 msgr2=0x55e127e20e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.665+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/455113505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e209f0 0x55e127e20e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59a8005e30 tx=0x7f59a80323d0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.665+0000 7f59b5c00640 1 --2- 172.21.15.47:0/1433594347 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f59980a3030 0x7f59980a54f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f59a409bf00 tx=0x7f59a4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.665+0000 7f59bb3b6740 1 -- 172.21.15.47:0/455113505 shutdown_connections 2024-03-21T11:24:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.665+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/455113505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e22580 0x55e127e28e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.665+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/455113505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e21bc0 0x55e127e22040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.665+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/455113505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e209f0 0x55e127e20e10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.665+0000 7f59bb3b6740 1 -- 172.21.15.47:0/455113505 >> 172.21.15.47:0/455113505 conn(0x55e127e1bde0 msgr2=0x55e127e1c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.665+0000 7f59bb3b6740 1 -- 172.21.15.47:0/455113505 shutdown_connections 2024-03-21T11:24:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.666+0000 7f59bb3b6740 1 -- 172.21.15.47:0/455113505 wait complete. 2024-03-21T11:24:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.666+0000 7f59bb3b6740 1 -- start start 2024-03-21T11:24:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.666+0000 7f59bb3b6740 1 Processor -- start 2024-03-21T11:24:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59bb3b6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e209f0 0x55e127e2ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59bb3b6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e2b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59bb3b6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e22580 0x55e127e32190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59bb3b6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e127e29530 con 0x55e127e22580 2024-03-21T11:24:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59bb3b6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e127d410d0 con 0x55e127e209f0 2024-03-21T11:24:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59bb3b6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e127e296b0 con 0x55e127e21bc0 2024-03-21T11:24:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59bb3b6740 10 client.0 _ll_register_callbacks cb 0x55e127d45e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59b5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e2b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59b5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e2b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32882/0 (socket says 172.21.15.47:32882) 2024-03-21T11:24:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59b5200640 1 -- 172.21.15.47:0/674352605 learned_addr learned my addr 172.21.15.47:0/674352605 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59b6600640 1 --2- 172.21.15.47:0/674352605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e209f0 0x55e127e2ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59b6600640 1 --2- 172.21.15.47:0/674352605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e22580 0x55e127e32190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59b5200640 1 -- 172.21.15.47:0/674352605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e209f0 msgr2=0x55e127e2ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59b5200640 1 --2- 172.21.15.47:0/674352605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e209f0 0x55e127e2ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59b5200640 1 -- 172.21.15.47:0/674352605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e22580 msgr2=0x55e127e32190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59b5200640 1 --2- 172.21.15.47:0/674352605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e22580 0x55e127e32190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.667+0000 7f59b5200640 1 -- 172.21.15.47:0/674352605 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e127a993f0 con 0x55e127e21bc0 2024-03-21T11:24:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.668+0000 7f59b5200640 1 --2- 172.21.15.47:0/674352605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e2b150 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f599c012700 tx=0x7f599c012b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10516f70e3915c0e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.668+0000 7f5997400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.668+0000 7f5997400640 1 -- 172.21.15.47:0/674352605 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c004020 con 0x55e127e21bc0 2024-03-21T11:24:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.668+0000 7f5997400640 1 -- 172.21.15.47:0/674352605 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f599c0041c0 con 0x55e127e21bc0 2024-03-21T11:24:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.668+0000 7f59bb3b6740 1 -- 172.21.15.47:0/674352605 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59a8032c90 con 0x55e127e21bc0 2024-03-21T11:24:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.668+0000 7f5997400640 1 -- 172.21.15.47:0/674352605 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f599c025050 con 0x55e127e21bc0 2024-03-21T11:24:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.668+0000 7f59bb3b6740 10 client.25744 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.668+0000 7f59bb3b6740 1 -- 172.21.15.47:0/674352605 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e127b637d0 con 0x55e127e21bc0 2024-03-21T11:24:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.669+0000 7f5997400640 1 -- 172.21.15.47:0/674352605 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59a8032c90 con 0x55e127e21bc0 2024-03-21T11:24:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.669+0000 7f5997400640 1 -- 172.21.15.47:0/674352605 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f599c003680 con 0x55e127e21bc0 2024-03-21T11:24:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.669+0000 7f59bb3b6740 10 client.25744 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.669+0000 7f59bb3b6740 10 client.25744 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:24:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.669+0000 7f59bb3b6740 20 client.25744 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:21.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.669+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/674352605 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e127e36790 0x55e127e38bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.669+0000 7f59bb3b6740 4 client.25744 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.669+0000 7f59bb3b6740 1 -- 172.21.15.47:0/674352605 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e127d92800 con 0x55e127e36790 2024-03-21T11:24:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.669+0000 7f59b6600640 1 --2- 172.21.15.47:0/674352605 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e127e36790 0x55e127e38bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.670+0000 7f59b6600640 1 --2- 172.21.15.47:0/674352605 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e127e36790 0x55e127e38bb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cd8e58ed42bee2c server_cookie=21b31152dfa0df36 in_seq=0 out_seq=0 2024-03-21T11:24:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.670+0000 7f5997400640 10 client.25744 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:24:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.670+0000 7f5997400640 1 -- 172.21.15.47:0/674352605 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e127d92800 con 0x55e127e36790 2024-03-21T11:24:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.671+0000 7f5997400640 10 client.25744 handle_command_reply: tid=0 2024-03-21T11:24:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.671+0000 7f59bb3b6740 1 client.25744 shutdown 2024-03-21T11:24:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.671+0000 7f59bb3b6740 1 -- 172.21.15.47:0/674352605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 msgr2=0x55e127e2b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.671+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/674352605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e2b150 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f599c012700 tx=0x7f599c012b00 comp rx=0 tx=0).stop 2024-03-21T11:24:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.671+0000 7f59bb3b6740 1 -- 172.21.15.47:0/674352605 shutdown_connections 2024-03-21T11:24:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.671+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/674352605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e22580 0x55e127e32190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.671+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/674352605 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e127e36790 0x55e127e38bb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.671+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/674352605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e2b150 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.671+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/674352605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e209f0 0x55e127e2ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.671+0000 7f5997400640 0 client.25744 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:24:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.671+0000 7f59bb3b6740 1 -- 172.21.15.47:0/674352605 >> 172.21.15.47:0/674352605 conn(0x55e127e1bde0 msgr2=0x55e127e1b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.671+0000 7f59bb3b6740 1 -- 172.21.15.47:0/674352605 shutdown_connections 2024-03-21T11:24:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.671+0000 7f59bb3b6740 1 -- 172.21.15.47:0/674352605 wait complete. 2024-03-21T11:24:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.671+0000 7f59bb3b6740 20 client.25744 trim_cache size 0 max 16384 2024-03-21T11:24:21.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.685+0000 7f59bb3b6740 1 Processor -- start 2024-03-21T11:24:21.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.685+0000 7f59bb3b6740 1 -- start start 2024-03-21T11:24:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.685+0000 7f59bb3b6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e209f0 0x55e127e1afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.685+0000 7f59bb3b6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e32150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.685+0000 7f59bb3b6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e22580 0x55e127e3d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.685+0000 7f59bb3b6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b01ac150 con 0x55e127e209f0 2024-03-21T11:24:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.685+0000 7f59bb3b6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b01adad0 con 0x55e127e22580 2024-03-21T11:24:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.685+0000 7f59bb3b6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b01ac7b0 con 0x55e127e21bc0 2024-03-21T11:24:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.686+0000 7f59b5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e32150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.686+0000 7f59b5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e32150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32892/0 (socket says 172.21.15.47:32892) 2024-03-21T11:24:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.686+0000 7f59b5200640 1 -- 172.21.15.47:0/2329692744 learned_addr learned my addr 172.21.15.47:0/2329692744 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.686+0000 7f59b6600640 1 --2- 172.21.15.47:0/2329692744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e209f0 0x55e127e1afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.686+0000 7f59b6600640 1 --2- 172.21.15.47:0/2329692744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e22580 0x55e127e3d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.686+0000 7f59b5200640 1 -- 172.21.15.47:0/2329692744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e22580 msgr2=0x55e127e3d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.686+0000 7f59b5200640 1 --2- 172.21.15.47:0/2329692744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e22580 0x55e127e3d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.686+0000 7f59b5200640 1 -- 172.21.15.47:0/2329692744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e209f0 msgr2=0x55e127e1afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.686+0000 7f59b5200640 1 --2- 172.21.15.47:0/2329692744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e209f0 0x55e127e1afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.686+0000 7f59b5200640 1 -- 172.21.15.47:0/2329692744 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e127e21640 con 0x55e127e21bc0 2024-03-21T11:24:21.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.686+0000 7f59b5200640 1 --2- 172.21.15.47:0/2329692744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e32150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f599c003a20 tx=0x7f599c005b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22c167c1040067bd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.687+0000 7f59a2000640 1 -- 172.21.15.47:0/2329692744 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c003830 con 0x55e127e21bc0 2024-03-21T11:24:21.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.687+0000 7f59a2000640 1 -- 172.21.15.47:0/2329692744 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f599c0055f0 con 0x55e127e21bc0 2024-03-21T11:24:21.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.687+0000 7f59a2000640 1 -- 172.21.15.47:0/2329692744 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c00fd40 con 0x55e127e21bc0 2024-03-21T11:24:21.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.687+0000 7f59bb3b6740 1 -- 172.21.15.47:0/2329692744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 msgr2=0x55e127e32150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.687+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/2329692744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e32150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f599c003a20 tx=0x7f599c005b70 comp rx=0 tx=0).stop 2024-03-21T11:24:21.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.687+0000 7f59bb3b6740 1 -- 172.21.15.47:0/2329692744 shutdown_connections 2024-03-21T11:24:21.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.687+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/2329692744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e22580 0x55e127e3d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.687+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/2329692744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e32150 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.687+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/2329692744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e209f0 0x55e127e1afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.687+0000 7f59bb3b6740 1 -- 172.21.15.47:0/2329692744 >> 172.21.15.47:0/2329692744 conn(0x55e127e1bde0 msgr2=0x55e127e1a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.687+0000 7f59bb3b6740 1 -- 172.21.15.47:0/2329692744 shutdown_connections 2024-03-21T11:24:21.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.687+0000 7f59bb3b6740 1 -- 172.21.15.47:0/2329692744 wait complete. 2024-03-21T11:24:21.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.688+0000 7f59bb3b6740 1 -- start start 2024-03-21T11:24:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.688+0000 7f59bb3b6740 1 Processor -- start 2024-03-21T11:24:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.688+0000 7f59bb3b6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e209f0 0x55e127e308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.688+0000 7f59bb3b6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e30df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.688+0000 7f59bb3b6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e22580 0x55e127e31330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.688+0000 7f59bb3b6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b01ad470 con 0x55e127e22580 2024-03-21T11:24:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.688+0000 7f59bb3b6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b01ace10 con 0x55e127e209f0 2024-03-21T11:24:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.688+0000 7f59bb3b6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b01aa310 con 0x55e127e21bc0 2024-03-21T11:24:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.688+0000 7f59bb3b6740 10 client.0 _ll_register_callbacks cb 0x7f599c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.689+0000 7f59b5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e30df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.689+0000 7f59b5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e30df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32896/0 (socket says 172.21.15.47:32896) 2024-03-21T11:24:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.689+0000 7f59b5200640 1 -- 172.21.15.47:0/335681345 learned_addr learned my addr 172.21.15.47:0/335681345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.689+0000 7f59b6600640 1 --2- 172.21.15.47:0/335681345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e209f0 0x55e127e308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.689+0000 7f59b6600640 1 --2- 172.21.15.47:0/335681345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e22580 0x55e127e31330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.689+0000 7f59b5200640 1 -- 172.21.15.47:0/335681345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e209f0 msgr2=0x55e127e308b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.689+0000 7f59b5200640 1 --2- 172.21.15.47:0/335681345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e209f0 0x55e127e308b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.689+0000 7f59b5200640 1 -- 172.21.15.47:0/335681345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e22580 msgr2=0x55e127e31330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.689+0000 7f59b5200640 1 --2- 172.21.15.47:0/335681345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e22580 0x55e127e31330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.689+0000 7f59b5200640 1 -- 172.21.15.47:0/335681345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e127e92040 con 0x55e127e21bc0 2024-03-21T11:24:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.689+0000 7f59b6600640 1 --2- 172.21.15.47:0/335681345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e22580 0x55e127e31330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.689+0000 7f59b5200640 1 --2- 172.21.15.47:0/335681345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e30df0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f599c003b70 tx=0x7f599c00f820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=879c6eb6bfe1bae6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.689+0000 7f5997400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.690+0000 7f5997400640 1 -- 172.21.15.47:0/335681345 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c003830 con 0x55e127e21bc0 2024-03-21T11:24:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.690+0000 7f59bb3b6740 1 -- 172.21.15.47:0/335681345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f599c0055f0 con 0x55e127e21bc0 2024-03-21T11:24:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.690+0000 7f5997400640 1 -- 172.21.15.47:0/335681345 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f599c005110 con 0x55e127e21bc0 2024-03-21T11:24:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.690+0000 7f5997400640 1 -- 172.21.15.47:0/335681345 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f599c03a050 con 0x55e127e21bc0 2024-03-21T11:24:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.690+0000 7f59bb3b6740 10 client.25750 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.690+0000 7f59bb3b6740 1 -- 172.21.15.47:0/335681345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e127e932e0 con 0x55e127e21bc0 2024-03-21T11:24:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.690+0000 7f5997400640 1 -- 172.21.15.47:0/335681345 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f599c0055f0 con 0x55e127e21bc0 2024-03-21T11:24:21.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.690+0000 7f5997400640 1 -- 172.21.15.47:0/335681345 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f599c030030 con 0x55e127e21bc0 2024-03-21T11:24:21.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.690+0000 7f59bb3b6740 10 client.25750 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:21.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.690+0000 7f59bb3b6740 10 client.25750 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:24:21.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.690+0000 7f59bb3b6740 20 client.25750 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:21.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.690+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/335681345 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e127e93b00 0x55e127e95f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.690+0000 7f59bb3b6740 4 client.25750 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:24:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.690+0000 7f59bb3b6740 1 -- 172.21.15.47:0/335681345 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e127d8cb50 con 0x55e127e93b00 2024-03-21T11:24:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.691+0000 7f59b6600640 1 --2- 172.21.15.47:0/335681345 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e127e93b00 0x55e127e95f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.691+0000 7f59b6600640 1 --2- 172.21.15.47:0/335681345 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e127e93b00 0x55e127e95f20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c9ed4394c848d32 server_cookie=2c94f2499b0a56c6 in_seq=0 out_seq=0 2024-03-21T11:24:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.691+0000 7f5997400640 10 client.25750 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:24:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.691+0000 7f5997400640 1 -- 172.21.15.47:0/335681345 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e127d8cb50 con 0x55e127e93b00 2024-03-21T11:24:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.691+0000 7f5997400640 10 client.25750 handle_command_reply: tid=0 2024-03-21T11:24:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.691+0000 7f59bb3b6740 1 client.25750 shutdown 2024-03-21T11:24:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.691+0000 7f59bb3b6740 1 -- 172.21.15.47:0/335681345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 msgr2=0x55e127e30df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/335681345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e30df0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f599c003b70 tx=0x7f599c00f820 comp rx=0 tx=0).stop 2024-03-21T11:24:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f59bb3b6740 1 -- 172.21.15.47:0/335681345 shutdown_connections 2024-03-21T11:24:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/335681345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e127e22580 0x55e127e31330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/335681345 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e127e93b00 0x55e127e95f20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/335681345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e127e21bc0 0x55e127e30df0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f59bb3b6740 1 --2- 172.21.15.47:0/335681345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e127e209f0 0x55e127e308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f5997400640 0 client.25750 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:24:21.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f59bb3b6740 1 -- 172.21.15.47:0/335681345 >> 172.21.15.47:0/335681345 conn(0x55e127e1bde0 msgr2=0x55e127e1b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f59bb3b6740 1 -- 172.21.15.47:0/335681345 shutdown_connections 2024-03-21T11:24:21.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f59bb3b6740 1 -- 172.21.15.47:0/335681345 wait complete. 2024-03-21T11:24:21.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f59bb3b6740 20 client.25750 trim_cache size 0 max 16384 2024-03-21T11:24:21.700 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:24:21.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f59b7000640 1 -- 172.21.15.47:0/1433594347 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f59980a3030 msgr2=0x7f59980a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f59b7000640 1 --2- 172.21.15.47:0/1433594347 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f59980a3030 0x7f59980a54f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f59a409bf00 tx=0x7f59a4000bc0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f59b7000640 1 -- 172.21.15.47:0/1433594347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b01037a0 msgr2=0x7f59b0199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.692+0000 7f59b7000640 1 --2- 172.21.15.47:0/1433594347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b01037a0 0x7f59b0199a40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f59a400e9c0 tx=0x7f59a400ee80 comp rx=0 tx=0).stop 2024-03-21T11:24:21.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.693+0000 7f59b7000640 1 -- 172.21.15.47:0/1433594347 shutdown_connections 2024-03-21T11:24:21.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.693+0000 7f59b7000640 1 --2- 172.21.15.47:0/1433594347 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f59980a3030 0x7f59980a54f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.693+0000 7f59b7000640 1 --2- 172.21.15.47:0/1433594347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59b0105350 0x7f59b01a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.693+0000 7f59b7000640 1 --2- 172.21.15.47:0/1433594347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59b0104990 0x7f59b0199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.693+0000 7f59b7000640 1 --2- 172.21.15.47:0/1433594347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b01037a0 0x7f59b0199a40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.693+0000 7f59b7000640 1 -- 172.21.15.47:0/1433594347 >> 172.21.15.47:0/1433594347 conn(0x7f59b00fed10 msgr2=0x7f59b0107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.693+0000 7f59b7000640 1 -- 172.21.15.47:0/1433594347 shutdown_connections 2024-03-21T11:24:21.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.693+0000 7f59b7000640 1 -- 172.21.15.47:0/1433594347 wait complete. 2024-03-21T11:24:21.711 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:24:21.711 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:24:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.921+0000 7f17a1000640 1 -- 172.21.15.47:0/1886013067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c104ab0 msgr2=0x7f179c106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.921+0000 7f17a1000640 1 --2- 172.21.15.47:0/1886013067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c104ab0 0x7f179c106ea0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1784009be0 tx=0x7f178402bef0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.922+0000 7f17a1000640 1 -- 172.21.15.47:0/1886013067 shutdown_connections 2024-03-21T11:24:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.922+0000 7f17a1000640 1 --2- 172.21.15.47:0/1886013067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c1073e0 0x7f179c1097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.922+0000 7f17a1000640 1 --2- 172.21.15.47:0/1886013067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c104ab0 0x7f179c106ea0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.922+0000 7f17a1000640 1 --2- 172.21.15.47:0/1886013067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c102180 0x7f179c104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.922+0000 7f17a1000640 1 -- 172.21.15.47:0/1886013067 >> 172.21.15.47:0/1886013067 conn(0x7f179c0fbb10 msgr2=0x7f179c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.922+0000 7f17a1000640 1 -- 172.21.15.47:0/1886013067 shutdown_connections 2024-03-21T11:24:21.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.922+0000 7f17a1000640 1 -- 172.21.15.47:0/1886013067 wait complete. 2024-03-21T11:24:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.923+0000 7f17a1000640 1 Processor -- start 2024-03-21T11:24:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.923+0000 7f17a1000640 1 -- start start 2024-03-21T11:24:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f17a1000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c102180 0x7f179c101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f17a1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c104ab0 0x7f179c100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f17a1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c1073e0 0x7f179c100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f17a1000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c10d340 con 0x7f179c102180 2024-03-21T11:24:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f17a1000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c10d1c0 con 0x7f179c104ab0 2024-03-21T11:24:21.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f17a1000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c10d4c0 con 0x7f179c1073e0 2024-03-21T11:24:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f179a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c1073e0 0x7f179c100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f1799400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c104ab0 0x7f179c100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f1799400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c104ab0 0x7f179c100130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51678/0 (socket says 172.21.15.47:51678) 2024-03-21T11:24:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f179a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c1073e0 0x7f179c100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32926/0 (socket says 172.21.15.47:32926) 2024-03-21T11:24:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f1799400640 1 -- 172.21.15.47:0/1810155296 learned_addr learned my addr 172.21.15.47:0/1810155296 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f1799e00640 1 --2- 172.21.15.47:0/1810155296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c102180 0x7f179c101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f1799400640 1 -- 172.21.15.47:0/1810155296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c1073e0 msgr2=0x7f179c100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f1799400640 1 --2- 172.21.15.47:0/1810155296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c1073e0 0x7f179c100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f1799400640 1 -- 172.21.15.47:0/1810155296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c102180 msgr2=0x7f179c101af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f1799400640 1 --2- 172.21.15.47:0/1810155296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c102180 0x7f179c101af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f1799400640 1 -- 172.21.15.47:0/1810155296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c100f00 con 0x7f179c104ab0 2024-03-21T11:24:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.924+0000 7f179a800640 1 --2- 172.21.15.47:0/1810155296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c1073e0 0x7f179c100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.925+0000 7f1799400640 1 --2- 172.21.15.47:0/1810155296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c104ab0 0x7f179c100130 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f178402c500 tx=0x7f1784033580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.925+0000 7f1792a00640 1 -- 172.21.15.47:0/1810155296 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784043020 con 0x7f179c104ab0 2024-03-21T11:24:21.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.925+0000 7f17a1000640 1 -- 172.21.15.47:0/1810155296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179c1a1b30 con 0x7f179c104ab0 2024-03-21T11:24:21.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.925+0000 7f1792a00640 1 -- 172.21.15.47:0/1810155296 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1784003e50 con 0x7f179c104ab0 2024-03-21T11:24:21.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.925+0000 7f1792a00640 1 -- 172.21.15.47:0/1810155296 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784034690 con 0x7f179c104ab0 2024-03-21T11:24:21.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.925+0000 7f17a1000640 1 -- 172.21.15.47:0/1810155296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179c1a2080 con 0x7f179c104ab0 2024-03-21T11:24:21.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.929+0000 7f1792a00640 1 -- 172.21.15.47:0/1810155296 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f1784046020 con 0x7f179c104ab0 2024-03-21T11:24:21.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.929+0000 7f17a3de2740 1 Processor -- start 2024-03-21T11:24:21.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.930+0000 7f17a3de2740 1 -- start start 2024-03-21T11:24:21.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.930+0000 7f17a3de2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b21810 0x55f7c2b21c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.930+0000 7f17a3de2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b229e0 0x55f7c2b22e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.930+0000 7f17a3de2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b233a0 0x55f7c2b29c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.930+0000 7f17a3de2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c2a862c0 con 0x55f7c2b233a0 2024-03-21T11:24:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.930+0000 7f17a3de2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c2a2ec90 con 0x55f7c2b21810 2024-03-21T11:24:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.930+0000 7f17a3de2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c2a82360 con 0x55f7c2b229e0 2024-03-21T11:24:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.930+0000 7f179a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b21810 0x55f7c2b21c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.930+0000 7f1799e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b229e0 0x55f7c2b22e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.930+0000 7f179a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b21810 0x55f7c2b21c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51688/0 (socket says 172.21.15.47:51688) 2024-03-21T11:24:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.930+0000 7f1799e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b229e0 0x55f7c2b22e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32934/0 (socket says 172.21.15.47:32934) 2024-03-21T11:24:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.930+0000 7f179a800640 1 -- 172.21.15.47:0/1132657860 learned_addr learned my addr 172.21.15.47:0/1132657860 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.931+0000 7f1799400640 1 --2- 172.21.15.47:0/1132657860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b233a0 0x55f7c2b29c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.931+0000 7f1799e00640 1 -- 172.21.15.47:0/1132657860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b21810 msgr2=0x55f7c2b21c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.931+0000 7f1799e00640 1 --2- 172.21.15.47:0/1132657860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b21810 0x55f7c2b21c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.931+0000 7f1799e00640 1 -- 172.21.15.47:0/1132657860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b233a0 msgr2=0x55f7c2b29c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.931+0000 7f1799e00640 1 --2- 172.21.15.47:0/1132657860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b233a0 0x55f7c2b29c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.931+0000 7f1799e00640 1 -- 172.21.15.47:0/1132657860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7c2a53080 con 0x55f7c2b229e0 2024-03-21T11:24:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.931+0000 7f1799e00640 1 --2- 172.21.15.47:0/1132657860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b229e0 0x55f7c2b22e60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f178c009c30 tx=0x7f178c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99a33166eb7665e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.932+0000 7f178b400640 1 -- 172.21.15.47:0/1132657860 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c01e070 con 0x55f7c2b229e0 2024-03-21T11:24:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.932+0000 7f178b400640 1 -- 172.21.15.47:0/1132657860 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f178c0035f0 con 0x55f7c2b229e0 2024-03-21T11:24:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.932+0000 7f178b400640 1 -- 172.21.15.47:0/1132657860 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c019690 con 0x55f7c2b229e0 2024-03-21T11:24:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.932+0000 7f1792a00640 1 --2- 172.21.15.47:0/1810155296 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f177c0a2fb0 0x7f177c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.932+0000 7f1792a00640 1 -- 172.21.15.47:0/1810155296 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f17840f1f20 con 0x7f179c104ab0 2024-03-21T11:24:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.932+0000 7f179a800640 1 --2- 172.21.15.47:0/1810155296 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f177c0a2fb0 0x7f177c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.933+0000 7f17a3de2740 1 -- 172.21.15.47:0/1132657860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b229e0 msgr2=0x55f7c2b22e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.933+0000 7f17a3de2740 1 --2- 172.21.15.47:0/1132657860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b229e0 0x55f7c2b22e60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f178c009c30 tx=0x7f178c011810 comp rx=0 tx=0).stop 2024-03-21T11:24:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.933+0000 7f17a3de2740 1 -- 172.21.15.47:0/1132657860 shutdown_connections 2024-03-21T11:24:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.933+0000 7f17a3de2740 1 --2- 172.21.15.47:0/1132657860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b233a0 0x55f7c2b29c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.933+0000 7f17a3de2740 1 --2- 172.21.15.47:0/1132657860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b229e0 0x55f7c2b22e60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.933+0000 7f17a3de2740 1 --2- 172.21.15.47:0/1132657860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b21810 0x55f7c2b21c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.933+0000 7f17a3de2740 1 -- 172.21.15.47:0/1132657860 >> 172.21.15.47:0/1132657860 conn(0x55f7c2b1cc00 msgr2=0x55f7c2b1d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.933+0000 7f17a3de2740 1 -- 172.21.15.47:0/1132657860 shutdown_connections 2024-03-21T11:24:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.933+0000 7f179a800640 1 --2- 172.21.15.47:0/1810155296 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f177c0a2fb0 0x7f177c0a5470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55f7c2b228e0 tx=0x7f1794005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.933+0000 7f17a3de2740 1 -- 172.21.15.47:0/1132657860 wait complete. 2024-03-21T11:24:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.933+0000 7f17a3de2740 1 -- start start 2024-03-21T11:24:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.934+0000 7f17a3de2740 1 Processor -- start 2024-03-21T11:24:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.934+0000 7f17a3de2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b21810 0x55f7c2b2ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.934+0000 7f17a3de2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b229e0 0x55f7c2b2bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.934+0000 7f17a3de2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b233a0 0x55f7c2b33000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.934+0000 7f17a3de2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c2b2a3b0 con 0x55f7c2b21810 2024-03-21T11:24:21.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.934+0000 7f17a3de2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c2a420d0 con 0x55f7c2b229e0 2024-03-21T11:24:21.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.934+0000 7f17a3de2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c2b2a530 con 0x55f7c2b233a0 2024-03-21T11:24:21.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.934+0000 7f17a3de2740 10 client.0 _ll_register_callbacks cb 0x55f7c2a46e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:21.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.935+0000 7f1799400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b229e0 0x55f7c2b2bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.935+0000 7f179a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b233a0 0x55f7c2b33000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.935+0000 7f1799e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b21810 0x55f7c2b2ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.935+0000 7f1799400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b229e0 0x55f7c2b2bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51690/0 (socket says 172.21.15.47:51690) 2024-03-21T11:24:21.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.935+0000 7f1799400640 1 -- 172.21.15.47:0/3047639322 learned_addr learned my addr 172.21.15.47:0/3047639322 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.935+0000 7f179a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b233a0 0x55f7c2b33000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32950/0 (socket says 172.21.15.47:32950) 2024-03-21T11:24:21.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.935+0000 7f179a800640 1 -- 172.21.15.47:0/3047639322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b229e0 msgr2=0x55f7c2b2bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.935+0000 7f179a800640 1 --2- 172.21.15.47:0/3047639322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b229e0 0x55f7c2b2bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.935+0000 7f179a800640 1 -- 172.21.15.47:0/3047639322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b21810 msgr2=0x55f7c2b2ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.935+0000 7f179a800640 1 --2- 172.21.15.47:0/3047639322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b21810 0x55f7c2b2ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.935+0000 7f179a800640 1 -- 172.21.15.47:0/3047639322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7c279a3f0 con 0x55f7c2b233a0 2024-03-21T11:24:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.935+0000 7f179a800640 1 --2- 172.21.15.47:0/3047639322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b233a0 0x55f7c2b33000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f179400f740 tx=0x7f1794004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e42502ef6336e23d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.935+0000 7f1783e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.936+0000 7f1783e00640 1 -- 172.21.15.47:0/3047639322 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794005020 con 0x55f7c2b233a0 2024-03-21T11:24:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.936+0000 7f1783e00640 1 -- 172.21.15.47:0/3047639322 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f17940051c0 con 0x55f7c2b233a0 2024-03-21T11:24:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.936+0000 7f17a3de2740 1 -- 172.21.15.47:0/3047639322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f178c0035f0 con 0x55f7c2b233a0 2024-03-21T11:24:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.936+0000 7f1783e00640 1 -- 172.21.15.47:0/3047639322 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17940053c0 con 0x55f7c2b233a0 2024-03-21T11:24:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.936+0000 7f17a3de2740 10 client.25762 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.936+0000 7f17a3de2740 1 -- 172.21.15.47:0/3047639322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f7c28647d0 con 0x55f7c2b233a0 2024-03-21T11:24:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.936+0000 7f1783e00640 1 -- 172.21.15.47:0/3047639322 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f178c0035f0 con 0x55f7c2b233a0 2024-03-21T11:24:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.936+0000 7f1783e00640 1 -- 172.21.15.47:0/3047639322 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1794027070 con 0x55f7c2b233a0 2024-03-21T11:24:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.936+0000 7f17a3de2740 10 client.25762 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.936+0000 7f17a3de2740 10 client.25762 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:24:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.937+0000 7f17a3de2740 20 client.25762 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.937+0000 7f17a3de2740 1 --2- 172.21.15.47:0/3047639322 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f7c2b37610 0x55f7c2b39a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.937+0000 7f17a3de2740 4 client.25762 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.937+0000 7f17a3de2740 1 -- 172.21.15.47:0/3047639322 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7c2a93800 con 0x55f7c2b37610 2024-03-21T11:24:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.937+0000 7f1799e00640 1 --2- 172.21.15.47:0/3047639322 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f7c2b37610 0x55f7c2b39a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.937+0000 7f1799e00640 1 --2- 172.21.15.47:0/3047639322 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f7c2b37610 0x55f7c2b39a70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9038d54f4597b6f6 server_cookie=5b247dec84fdcff3 in_seq=0 out_seq=0 2024-03-21T11:24:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.937+0000 7f1783e00640 10 client.25762 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:24:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.938+0000 7f1783e00640 1 -- 172.21.15.47:0/3047639322 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f7c2a93800 con 0x55f7c2b37610 2024-03-21T11:24:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.938+0000 7f1783e00640 10 client.25762 handle_command_reply: tid=0 2024-03-21T11:24:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.938+0000 7f17a3de2740 1 client.25762 shutdown 2024-03-21T11:24:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.938+0000 7f17a3de2740 1 -- 172.21.15.47:0/3047639322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b233a0 msgr2=0x55f7c2b33000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.938+0000 7f17a3de2740 1 --2- 172.21.15.47:0/3047639322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b233a0 0x55f7c2b33000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f179400f740 tx=0x7f1794004730 comp rx=0 tx=0).stop 2024-03-21T11:24:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.938+0000 7f17a3de2740 1 -- 172.21.15.47:0/3047639322 shutdown_connections 2024-03-21T11:24:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.938+0000 7f17a3de2740 1 --2- 172.21.15.47:0/3047639322 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f7c2b37610 0x55f7c2b39a70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.938+0000 7f17a3de2740 1 --2- 172.21.15.47:0/3047639322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b233a0 0x55f7c2b33000 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.938+0000 7f17a3de2740 1 --2- 172.21.15.47:0/3047639322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b229e0 0x55f7c2b2bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.938+0000 7f17a3de2740 1 --2- 172.21.15.47:0/3047639322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b21810 0x55f7c2b2ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.938+0000 7f1783e00640 0 client.25762 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:24:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.938+0000 7f17a3de2740 1 -- 172.21.15.47:0/3047639322 >> 172.21.15.47:0/3047639322 conn(0x55f7c2b1cc00 msgr2=0x55f7c2b1beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.939+0000 7f17a3de2740 1 -- 172.21.15.47:0/3047639322 shutdown_connections 2024-03-21T11:24:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.939+0000 7f17a3de2740 1 -- 172.21.15.47:0/3047639322 wait complete. 2024-03-21T11:24:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.939+0000 7f17a3de2740 20 client.25762 trim_cache size 0 max 16384 2024-03-21T11:24:21.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.952+0000 7f17a3de2740 1 Processor -- start 2024-03-21T11:24:21.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f17a3de2740 1 -- start start 2024-03-21T11:24:21.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f17a3de2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b21810 0x55f7c2b1bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f17a3de2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b229e0 0x55f7c2b32fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f17a3de2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b233a0 0x55f7c2b3e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f17a3de2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1ac380 con 0x55f7c2b233a0 2024-03-21T11:24:21.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f17a3de2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1add00 con 0x55f7c2b229e0 2024-03-21T11:24:21.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f17a3de2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1ac9e0 con 0x55f7c2b21810 2024-03-21T11:24:21.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f1799e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b21810 0x55f7c2b1bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f1799e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b21810 0x55f7c2b1bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32952/0 (socket says 172.21.15.47:32952) 2024-03-21T11:24:21.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f1799e00640 1 -- 172.21.15.47:0/3365636553 learned_addr learned my addr 172.21.15.47:0/3365636553 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f1799400640 1 --2- 172.21.15.47:0/3365636553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b229e0 0x55f7c2b32fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f179a800640 1 --2- 172.21.15.47:0/3365636553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b233a0 0x55f7c2b3e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f1799e00640 1 -- 172.21.15.47:0/3365636553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b229e0 msgr2=0x55f7c2b32fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f1799e00640 1 --2- 172.21.15.47:0/3365636553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b229e0 0x55f7c2b32fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f1799e00640 1 -- 172.21.15.47:0/3365636553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b233a0 msgr2=0x55f7c2b3e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f1799e00640 1 --2- 172.21.15.47:0/3365636553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b233a0 0x55f7c2b3e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f1799e00640 1 -- 172.21.15.47:0/3365636553 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7c2b22460 con 0x55f7c2b21810 2024-03-21T11:24:21.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.953+0000 7f1799400640 1 --2- 172.21.15.47:0/3365636553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b229e0 0x55f7c2b32fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:21.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.954+0000 7f1799e00640 1 --2- 172.21.15.47:0/3365636553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b21810 0x55f7c2b1bd30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f178c01c040 tx=0x7f178c005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c96719ee7a15e985 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.954+0000 7f178aa00640 1 -- 172.21.15.47:0/3365636553 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c030070 con 0x55f7c2b21810 2024-03-21T11:24:21.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.954+0000 7f178aa00640 1 -- 172.21.15.47:0/3365636553 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f178c029070 con 0x55f7c2b21810 2024-03-21T11:24:21.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.954+0000 7f178aa00640 1 -- 172.21.15.47:0/3365636553 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c01e040 con 0x55f7c2b21810 2024-03-21T11:24:21.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.955+0000 7f17a3de2740 1 -- 172.21.15.47:0/3365636553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b21810 msgr2=0x55f7c2b1bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.955+0000 7f17a3de2740 1 --2- 172.21.15.47:0/3365636553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b21810 0x55f7c2b1bd30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f178c01c040 tx=0x7f178c005df0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.955+0000 7f17a3de2740 1 -- 172.21.15.47:0/3365636553 shutdown_connections 2024-03-21T11:24:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.955+0000 7f17a3de2740 1 --2- 172.21.15.47:0/3365636553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b233a0 0x55f7c2b3e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.955+0000 7f17a3de2740 1 --2- 172.21.15.47:0/3365636553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b229e0 0x55f7c2b32fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.955+0000 7f17a3de2740 1 --2- 172.21.15.47:0/3365636553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b21810 0x55f7c2b1bd30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.955+0000 7f17a3de2740 1 -- 172.21.15.47:0/3365636553 >> 172.21.15.47:0/3365636553 conn(0x55f7c2b1cc00 msgr2=0x55f7c2b1b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.955+0000 7f17a3de2740 1 -- 172.21.15.47:0/3365636553 shutdown_connections 2024-03-21T11:24:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.955+0000 7f17a3de2740 1 -- 172.21.15.47:0/3365636553 wait complete. 2024-03-21T11:24:21.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.955+0000 7f17a3de2740 1 -- start start 2024-03-21T11:24:21.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.955+0000 7f17a3de2740 1 Processor -- start 2024-03-21T11:24:21.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.956+0000 7f17a3de2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b21810 0x55f7c2b328b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.956+0000 7f17a3de2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b229e0 0x55f7c2b32df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.956+0000 7f17a3de2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b233a0 0x55f7c2b2f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.956+0000 7f17a3de2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1ad6a0 con 0x55f7c2b21810 2024-03-21T11:24:21.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.956+0000 7f17a3de2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1ad040 con 0x55f7c2b233a0 2024-03-21T11:24:21.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.956+0000 7f17a3de2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1aa540 con 0x55f7c2b229e0 2024-03-21T11:24:21.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.956+0000 7f17a3de2740 10 client.0 _ll_register_callbacks cb 0x7f1794012050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:21.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.956+0000 7f179a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b233a0 0x55f7c2b2f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.956+0000 7f1799e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b21810 0x55f7c2b328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.956+0000 7f1799400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b229e0 0x55f7c2b32df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.956+0000 7f179a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b233a0 0x55f7c2b2f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51702/0 (socket says 172.21.15.47:51702) 2024-03-21T11:24:21.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.956+0000 7f179a800640 1 -- 172.21.15.47:0/1094648408 learned_addr learned my addr 172.21.15.47:0/1094648408 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:21.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.957+0000 7f1799e00640 1 -- 172.21.15.47:0/1094648408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b229e0 msgr2=0x55f7c2b32df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.957+0000 7f1799e00640 1 --2- 172.21.15.47:0/1094648408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b229e0 0x55f7c2b32df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.957+0000 7f1799e00640 1 -- 172.21.15.47:0/1094648408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b233a0 msgr2=0x55f7c2b2f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:21.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.957+0000 7f1799e00640 1 --2- 172.21.15.47:0/1094648408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b233a0 0x55f7c2b2f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.957+0000 7f1799e00640 1 -- 172.21.15.47:0/1094648408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7c2b2fa90 con 0x55f7c2b21810 2024-03-21T11:24:21.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.957+0000 7f179a800640 1 --2- 172.21.15.47:0/1094648408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b233a0 0x55f7c2b2f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:21.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.957+0000 7f1799400640 1 --2- 172.21.15.47:0/1094648408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b229e0 0x55f7c2b32df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:21.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.957+0000 7f1799e00640 1 --2- 172.21.15.47:0/1094648408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b21810 0x55f7c2b328b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f178c009230 tx=0x7f178c009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c076b2b461f6b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.957+0000 7f1783e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:24:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.957+0000 7f1783e00640 1 -- 172.21.15.47:0/1094648408 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c039060 con 0x55f7c2b21810 2024-03-21T11:24:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.957+0000 7f1783e00640 1 -- 172.21.15.47:0/1094648408 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f178c02a280 con 0x55f7c2b21810 2024-03-21T11:24:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.957+0000 7f17a3de2740 1 -- 172.21.15.47:0/1094648408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f178c029070 con 0x55f7c2b21810 2024-03-21T11:24:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.957+0000 7f1783e00640 1 -- 172.21.15.47:0/1094648408 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f178c040050 con 0x55f7c2b21810 2024-03-21T11:24:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.957+0000 7f17a3de2740 10 client.15603 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.958+0000 7f17a3de2740 1 -- 172.21.15.47:0/1094648408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f7c2b93b40 con 0x55f7c2b21810 2024-03-21T11:24:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.958+0000 7f1783e00640 1 -- 172.21.15.47:0/1094648408 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f178c029070 con 0x55f7c2b21810 2024-03-21T11:24:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.958+0000 7f1783e00640 1 -- 172.21.15.47:0/1094648408 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f178c004840 con 0x55f7c2b21810 2024-03-21T11:24:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.958+0000 7f17a3de2740 10 client.15603 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.958+0000 7f17a3de2740 10 client.15603 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:24:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.958+0000 7f17a3de2740 20 client.15603 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.958+0000 7f17a3de2740 1 --2- 172.21.15.47:0/1094648408 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f7c2b941f0 0x55f7c2b965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.958+0000 7f17a3de2740 4 client.15603 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:24:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.958+0000 7f17a3de2740 1 -- 172.21.15.47:0/1094648408 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f7c2a8db50 con 0x55f7c2b941f0 2024-03-21T11:24:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.958+0000 7f1799400640 1 --2- 172.21.15.47:0/1094648408 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f7c2b941f0 0x55f7c2b965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f1799400640 1 --2- 172.21.15.47:0/1094648408 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f7c2b941f0 0x55f7c2b965e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e28701a890fb066 server_cookie=e6a4dd49090175 in_seq=0 out_seq=0 2024-03-21T11:24:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f1783e00640 10 client.15603 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:24:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f1783e00640 1 -- 172.21.15.47:0/1094648408 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f7c2a8db50 con 0x55f7c2b941f0 2024-03-21T11:24:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f1783e00640 10 client.15603 handle_command_reply: tid=0 2024-03-21T11:24:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f17a3de2740 1 client.15603 shutdown 2024-03-21T11:24:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f17a3de2740 1 -- 172.21.15.47:0/1094648408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b21810 msgr2=0x55f7c2b328b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f17a3de2740 1 --2- 172.21.15.47:0/1094648408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b21810 0x55f7c2b328b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f178c009230 tx=0x7f178c009d20 comp rx=0 tx=0).stop 2024-03-21T11:24:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f17a3de2740 1 -- 172.21.15.47:0/1094648408 shutdown_connections 2024-03-21T11:24:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f17a3de2740 1 --2- 172.21.15.47:0/1094648408 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f7c2b941f0 0x55f7c2b965e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f17a3de2740 1 --2- 172.21.15.47:0/1094648408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7c2b233a0 0x55f7c2b2f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f17a3de2740 1 --2- 172.21.15.47:0/1094648408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7c2b229e0 0x55f7c2b32df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f17a3de2740 1 --2- 172.21.15.47:0/1094648408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7c2b21810 0x55f7c2b328b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f1783e00640 0 client.15603 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:24:21.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f17a3de2740 1 -- 172.21.15.47:0/1094648408 >> 172.21.15.47:0/1094648408 conn(0x55f7c2b1cc00 msgr2=0x55f7c2b1beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.959+0000 7f17a3de2740 1 -- 172.21.15.47:0/1094648408 shutdown_connections 2024-03-21T11:24:21.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.960+0000 7f17a3de2740 1 -- 172.21.15.47:0/1094648408 wait complete. 2024-03-21T11:24:21.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.960+0000 7f17a3de2740 20 client.15603 trim_cache size 0 max 16384 2024-03-21T11:24:21.966 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:24:21.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.960+0000 7f17a1000640 1 -- 172.21.15.47:0/1810155296 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f177c0a2fb0 msgr2=0x7f177c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.960+0000 7f17a1000640 1 --2- 172.21.15.47:0/1810155296 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f177c0a2fb0 0x7f177c0a5470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55f7c2b228e0 tx=0x7f1794005f90 comp rx=0 tx=0).stop 2024-03-21T11:24:21.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.960+0000 7f17a1000640 1 -- 172.21.15.47:0/1810155296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c104ab0 msgr2=0x7f179c100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:21.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.960+0000 7f17a1000640 1 --2- 172.21.15.47:0/1810155296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c104ab0 0x7f179c100130 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f178402c500 tx=0x7f1784033580 comp rx=0 tx=0).stop 2024-03-21T11:24:21.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.960+0000 7f17a1000640 1 -- 172.21.15.47:0/1810155296 shutdown_connections 2024-03-21T11:24:21.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.960+0000 7f17a1000640 1 --2- 172.21.15.47:0/1810155296 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f177c0a2fb0 0x7f177c0a5470 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.960+0000 7f17a1000640 1 --2- 172.21.15.47:0/1810155296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c1073e0 0x7f179c100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.960+0000 7f17a1000640 1 --2- 172.21.15.47:0/1810155296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c104ab0 0x7f179c100130 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.960+0000 7f17a1000640 1 --2- 172.21.15.47:0/1810155296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c102180 0x7f179c101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:21.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.960+0000 7f17a1000640 1 -- 172.21.15.47:0/1810155296 >> 172.21.15.47:0/1810155296 conn(0x7f179c0fbb10 msgr2=0x7f179c102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:21.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.961+0000 7f17a1000640 1 -- 172.21.15.47:0/1810155296 shutdown_connections 2024-03-21T11:24:21.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:21.961+0000 7f17a1000640 1 -- 172.21.15.47:0/1810155296 wait complete. 2024-03-21T11:24:21.980 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:24:21.980 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:24:22.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.198+0000 7f94fa400640 1 -- 172.21.15.47:0/730600714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f4109770 msgr2=0x7f94f4110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:22.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.198+0000 7f94fa400640 1 --2- 172.21.15.47:0/730600714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f4109770 0x7f94f4110000 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009ca0 tx=0x7f94ec02c100 comp rx=0 tx=0).stop 2024-03-21T11:24:22.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.199+0000 7f94fa400640 1 -- 172.21.15.47:0/730600714 shutdown_connections 2024-03-21T11:24:22.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.199+0000 7f94fa400640 1 --2- 172.21.15.47:0/730600714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f4109770 0x7f94f4110000 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.199+0000 7f94fa400640 1 --2- 172.21.15.47:0/730600714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f4108db0 0x7f94f4109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.199+0000 7f94fa400640 1 --2- 172.21.15.47:0/730600714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94f4107bc0 0x7f94f4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.199+0000 7f94fa400640 1 -- 172.21.15.47:0/730600714 >> 172.21.15.47:0/730600714 conn(0x7f94f4076b70 msgr2=0x7f94f4078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:22.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.199+0000 7f94fa400640 1 -- 172.21.15.47:0/730600714 shutdown_connections 2024-03-21T11:24:22.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.199+0000 7f94fa400640 1 -- 172.21.15.47:0/730600714 wait complete. 2024-03-21T11:24:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.200+0000 7f94fa400640 1 Processor -- start 2024-03-21T11:24:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.200+0000 7f94fa400640 1 -- start start 2024-03-21T11:24:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.200+0000 7f94fa400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f4107bc0 0x7f94f419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.200+0000 7f94fa400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f4108db0 0x7f94f419e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.200+0000 7f94fa400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94f4109770 0x7f94f41a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.200+0000 7f94fa400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f4113820 con 0x7f94f4109770 2024-03-21T11:24:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.200+0000 7f94fa400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f41136a0 con 0x7f94f4107bc0 2024-03-21T11:24:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.200+0000 7f94fa400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f41139a0 con 0x7f94f4108db0 2024-03-21T11:24:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.200+0000 7f94f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f4108db0 0x7f94f419e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.200+0000 7f94f9a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94f4109770 0x7f94f41a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.200+0000 7f94f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f4108db0 0x7f94f419e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32988/0 (socket says 172.21.15.47:32988) 2024-03-21T11:24:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.200+0000 7f94f3e00640 1 -- 172.21.15.47:0/782241094 learned_addr learned my addr 172.21.15.47:0/782241094 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.200+0000 7f94f9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f4107bc0 0x7f94f419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.201+0000 7f94f3e00640 1 -- 172.21.15.47:0/782241094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f4107bc0 msgr2=0x7f94f419de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.201+0000 7f94f3e00640 1 --2- 172.21.15.47:0/782241094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f4107bc0 0x7f94f419de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.201+0000 7f94f3e00640 1 -- 172.21.15.47:0/782241094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94f4109770 msgr2=0x7f94f41a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.201+0000 7f94f3e00640 1 --2- 172.21.15.47:0/782241094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94f4109770 0x7f94f41a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.201+0000 7f94f3e00640 1 -- 172.21.15.47:0/782241094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f41a5a20 con 0x7f94f4108db0 2024-03-21T11:24:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.201+0000 7f94f9a00640 1 --2- 172.21.15.47:0/782241094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94f4109770 0x7f94f41a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.201+0000 7f94f3e00640 1 --2- 172.21.15.47:0/782241094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f4108db0 0x7f94f419e340 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f94e400e9f0 tx=0x7f94e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.201+0000 7f94f1600640 1 -- 172.21.15.47:0/782241094 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e400cd80 con 0x7f94f4108db0 2024-03-21T11:24:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.201+0000 7f94f1600640 1 -- 172.21.15.47:0/782241094 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94e4010910 con 0x7f94f4108db0 2024-03-21T11:24:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.201+0000 7f94f1600640 1 -- 172.21.15.47:0/782241094 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4018990 con 0x7f94f4108db0 2024-03-21T11:24:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.202+0000 7f94fa400640 1 -- 172.21.15.47:0/782241094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f41a5d10 con 0x7f94f4108db0 2024-03-21T11:24:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.202+0000 7f94fa400640 1 -- 172.21.15.47:0/782241094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f41a61b0 con 0x7f94f4108db0 2024-03-21T11:24:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.205+0000 7f94fe5f1740 1 Processor -- start 2024-03-21T11:24:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.205+0000 7f94fe5f1740 1 -- start start 2024-03-21T11:24:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.205+0000 7f94f1600640 1 -- 172.21.15.47:0/782241094 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f94e4010ac0 con 0x7f94f4108db0 2024-03-21T11:24:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.205+0000 7f94fe5f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f49f0 0x55b7389f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.205+0000 7f94fe5f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 0x55b7389f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.205+0000 7f94fe5f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f6580 0x55b7389fce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.205+0000 7f94fe5f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7389592c0 con 0x55b7389f5bc0 2024-03-21T11:24:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.205+0000 7f94fe5f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b738901c90 con 0x55b7389f6580 2024-03-21T11:24:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.205+0000 7f94fe5f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b738955360 con 0x55b7389f49f0 2024-03-21T11:24:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.206+0000 7f94f9a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f49f0 0x55b7389f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.206+0000 7f94f9a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f49f0 0x55b7389f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:32996/0 (socket says 172.21.15.47:32996) 2024-03-21T11:24:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.206+0000 7f94f9a00640 1 -- 172.21.15.47:0/1869581221 learned_addr learned my addr 172.21.15.47:0/1869581221 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.206+0000 7f94f9000640 1 --2- 172.21.15.47:0/1869581221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 0x55b7389f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.206+0000 7f94f3e00640 1 --2- 172.21.15.47:0/1869581221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f6580 0x55b7389fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.206+0000 7f94f9a00640 1 -- 172.21.15.47:0/1869581221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f6580 msgr2=0x55b7389fce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.206+0000 7f94f9a00640 1 --2- 172.21.15.47:0/1869581221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f6580 0x55b7389fce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.206+0000 7f94f9a00640 1 -- 172.21.15.47:0/1869581221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 msgr2=0x55b7389f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.206+0000 7f94f9a00640 1 --2- 172.21.15.47:0/1869581221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 0x55b7389f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.206+0000 7f94f9a00640 1 -- 172.21.15.47:0/1869581221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b738926080 con 0x55b7389f49f0 2024-03-21T11:24:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.206+0000 7f94f9a00640 1 --2- 172.21.15.47:0/1869581221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f49f0 0x55b7389f4e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f94ec02c710 tx=0x7f94ec0323d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0291ace749d33ac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.207+0000 7f94e2000640 1 -- 172.21.15.47:0/1869581221 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec043070 con 0x55b7389f49f0 2024-03-21T11:24:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.207+0000 7f94e2000640 1 -- 172.21.15.47:0/1869581221 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94ec032c90 con 0x55b7389f49f0 2024-03-21T11:24:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.207+0000 7f94e2000640 1 -- 172.21.15.47:0/1869581221 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec039040 con 0x55b7389f49f0 2024-03-21T11:24:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.207+0000 7f94f1600640 1 --2- 172.21.15.47:0/782241094 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f94dc0a2fb0 0x7f94dc0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.207+0000 7f94f1600640 1 -- 172.21.15.47:0/782241094 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f94e40cfb90 con 0x7f94f4108db0 2024-03-21T11:24:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.207+0000 7f94f9000640 1 --2- 172.21.15.47:0/782241094 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f94dc0a2fb0 0x7f94dc0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.207+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1869581221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f49f0 msgr2=0x55b7389f4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.207+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1869581221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f49f0 0x55b7389f4e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f94ec02c710 tx=0x7f94ec0323d0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.208+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1869581221 shutdown_connections 2024-03-21T11:24:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.208+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1869581221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f6580 0x55b7389fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.208+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1869581221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 0x55b7389f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.208+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1869581221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f49f0 0x55b7389f4e10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.208+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1869581221 >> 172.21.15.47:0/1869581221 conn(0x55b7389efde0 msgr2=0x55b7389f01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.208+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1869581221 shutdown_connections 2024-03-21T11:24:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.208+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1869581221 wait complete. 2024-03-21T11:24:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.208+0000 7f94f9000640 1 --2- 172.21.15.47:0/782241094 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f94dc0a2fb0 0x7f94dc0a5470 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f94e8005fd0 tx=0x7f94e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.208+0000 7f94fe5f1740 1 -- start start 2024-03-21T11:24:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.209+0000 7f94fe5f1740 1 Processor -- start 2024-03-21T11:24:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.209+0000 7f94fe5f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f49f0 0x55b7389fec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.209+0000 7f94fe5f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 0x55b7389ff170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.209+0000 7f94fe5f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f6580 0x55b738a061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.209+0000 7f94fe5f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7389fd530 con 0x55b7389f5bc0 2024-03-21T11:24:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.209+0000 7f94fe5f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7389150d0 con 0x55b7389f49f0 2024-03-21T11:24:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.209+0000 7f94fe5f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7389fd6b0 con 0x55b7389f6580 2024-03-21T11:24:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.209+0000 7f94fe5f1740 10 client.0 _ll_register_callbacks cb 0x55b738919e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.209+0000 7f94f9a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f6580 0x55b738a061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.209+0000 7f94f3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 0x55b7389ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.209+0000 7f94f9a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f6580 0x55b738a061b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33010/0 (socket says 172.21.15.47:33010) 2024-03-21T11:24:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.209+0000 7f94f9a00640 1 -- 172.21.15.47:0/1215484397 learned_addr learned my addr 172.21.15.47:0/1215484397 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.210+0000 7f94f9a00640 1 -- 172.21.15.47:0/1215484397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f49f0 msgr2=0x55b7389fec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.210+0000 7f94f9a00640 1 --2- 172.21.15.47:0/1215484397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f49f0 0x55b7389fec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.210+0000 7f94f9a00640 1 -- 172.21.15.47:0/1215484397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 msgr2=0x55b7389ff170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.210+0000 7f94f9a00640 1 --2- 172.21.15.47:0/1215484397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 0x55b7389ff170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.210+0000 7f94f9a00640 1 -- 172.21.15.47:0/1215484397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b73866d3f0 con 0x55b7389f6580 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.210+0000 7f94f9a00640 1 --2- 172.21.15.47:0/1215484397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f6580 0x55b738a061b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0324c0 tx=0x7f94ec009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17fa933aa7c61a4e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.210+0000 7f94d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.210+0000 7f94d6a00640 1 -- 172.21.15.47:0/1215484397 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec04c030 con 0x55b7389f6580 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.210+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1215484397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94ec032c90 con 0x55b7389f6580 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.210+0000 7f94d6a00640 1 -- 172.21.15.47:0/1215484397 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f94ec003970 con 0x55b7389f6580 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.211+0000 7f94d6a00640 1 -- 172.21.15.47:0/1215484397 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94ec059050 con 0x55b7389f6580 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.211+0000 7f94fe5f1740 10 client.25780 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.211+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1215484397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b7387377d0 con 0x55b7389f6580 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.211+0000 7f94d6a00640 1 -- 172.21.15.47:0/1215484397 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94ec032c90 con 0x55b7389f6580 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.211+0000 7f94d6a00640 1 -- 172.21.15.47:0/1215484397 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f94ec004380 con 0x55b7389f6580 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.211+0000 7f94fe5f1740 10 client.25780 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.211+0000 7f94fe5f1740 10 client.25780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.211+0000 7f94fe5f1740 20 client.25780 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.211+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1215484397 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b738a0a310 0x55b738a0c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.211+0000 7f94fe5f1740 4 client.25780 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.211+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1215484397 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b738966800 con 0x55b738a0a310 2024-03-21T11:24:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.211+0000 7f94f9000640 1 --2- 172.21.15.47:0/1215484397 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b738a0a310 0x55b738a0c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.212+0000 7f94f9000640 1 --2- 172.21.15.47:0/1215484397 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b738a0a310 0x55b738a0c700 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa24277daed5ae8 server_cookie=852a87fa41e29067 in_seq=0 out_seq=0 2024-03-21T11:24:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.212+0000 7f94d6a00640 10 client.25780 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.213+0000 7f94d6a00640 1 -- 172.21.15.47:0/1215484397 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b738966800 con 0x55b738a0a310 2024-03-21T11:24:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.213+0000 7f94d6a00640 10 client.25780 handle_command_reply: tid=0 2024-03-21T11:24:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.213+0000 7f94fe5f1740 1 client.25780 shutdown 2024-03-21T11:24:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.213+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1215484397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f6580 msgr2=0x55b738a061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.213+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1215484397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f6580 0x55b738a061b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0324c0 tx=0x7f94ec009360 comp rx=0 tx=0).stop 2024-03-21T11:24:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.213+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1215484397 shutdown_connections 2024-03-21T11:24:22.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.213+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1215484397 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b738a0a310 0x55b738a0c700 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.213+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1215484397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f6580 0x55b738a061b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.213+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1215484397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 0x55b7389ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.213+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1215484397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f49f0 0x55b7389fec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.213+0000 7f94d6a00640 0 client.25780 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:22.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.213+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1215484397 >> 172.21.15.47:0/1215484397 conn(0x55b7389efde0 msgr2=0x55b7389ef160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:22.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.213+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1215484397 shutdown_connections 2024-03-21T11:24:22.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.213+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1215484397 wait complete. 2024-03-21T11:24:22.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.214+0000 7f94fe5f1740 20 client.25780 trim_cache size 0 max 16384 2024-03-21T11:24:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.227+0000 7f94fe5f1740 1 Processor -- start 2024-03-21T11:24:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.227+0000 7f94fe5f1740 1 -- start start 2024-03-21T11:24:22.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94fe5f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f49f0 0x55b7389eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94fe5f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f5bc0 0x55b738a06170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94fe5f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f6580 0x55b738a10e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94fe5f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f41b0470 con 0x55b7389f6580 2024-03-21T11:24:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94fe5f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f41b1df0 con 0x55b7389f5bc0 2024-03-21T11:24:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94fe5f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f41b0ad0 con 0x55b7389f49f0 2024-03-21T11:24:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94f9000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f49f0 0x55b7389eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94f9000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f49f0 0x55b7389eefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33022/0 (socket says 172.21.15.47:33022) 2024-03-21T11:24:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94f9000640 1 -- 172.21.15.47:0/3223619771 learned_addr learned my addr 172.21.15.47:0/3223619771 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94f3e00640 1 --2- 172.21.15.47:0/3223619771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f5bc0 0x55b738a06170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94f9a00640 1 --2- 172.21.15.47:0/3223619771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f6580 0x55b738a10e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94f9000640 1 -- 172.21.15.47:0/3223619771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f5bc0 msgr2=0x55b738a06170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94f9000640 1 --2- 172.21.15.47:0/3223619771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f5bc0 0x55b738a06170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94f9000640 1 -- 172.21.15.47:0/3223619771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f6580 msgr2=0x55b738a10e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94f9000640 1 --2- 172.21.15.47:0/3223619771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f6580 0x55b738a10e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.228+0000 7f94f9000640 1 -- 172.21.15.47:0/3223619771 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7389f0850 con 0x55b7389f49f0 2024-03-21T11:24:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.229+0000 7f94f9000640 1 --2- 172.21.15.47:0/3223619771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f49f0 0x55b7389eefe0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55b7389f0d50 tx=0x7f94e8016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffa62aa5eddf1e5a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.229+0000 7f94e1600640 1 -- 172.21.15.47:0/3223619771 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8027a60 con 0x55b7389f49f0 2024-03-21T11:24:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.229+0000 7f94e1600640 1 -- 172.21.15.47:0/3223619771 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94e802d070 con 0x55b7389f49f0 2024-03-21T11:24:22.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.229+0000 7f94e1600640 1 -- 172.21.15.47:0/3223619771 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e801d410 con 0x55b7389f49f0 2024-03-21T11:24:22.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.229+0000 7f94fe5f1740 1 -- 172.21.15.47:0/3223619771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f49f0 msgr2=0x55b7389eefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.229+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/3223619771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f49f0 0x55b7389eefe0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55b7389f0d50 tx=0x7f94e8016000 comp rx=0 tx=0).stop 2024-03-21T11:24:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.229+0000 7f94fe5f1740 1 -- 172.21.15.47:0/3223619771 shutdown_connections 2024-03-21T11:24:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.229+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/3223619771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f6580 0x55b738a10e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.229+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/3223619771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f5bc0 0x55b738a06170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.230+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/3223619771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f49f0 0x55b7389eefe0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.230+0000 7f94fe5f1740 1 -- 172.21.15.47:0/3223619771 >> 172.21.15.47:0/3223619771 conn(0x55b7389efde0 msgr2=0x55b7389ee8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.230+0000 7f94fe5f1740 1 -- 172.21.15.47:0/3223619771 shutdown_connections 2024-03-21T11:24:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.230+0000 7f94fe5f1740 1 -- 172.21.15.47:0/3223619771 wait complete. 2024-03-21T11:24:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.230+0000 7f94fe5f1740 1 -- start start 2024-03-21T11:24:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.230+0000 7f94fe5f1740 1 Processor -- start 2024-03-21T11:24:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94fe5f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f49f0 0x55b738a05a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94fe5f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 0x55b738a05f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94fe5f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f6580 0x55b738a024b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94fe5f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f41b1790 con 0x55b7389f5bc0 2024-03-21T11:24:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94fe5f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f41b1130 con 0x55b7389f49f0 2024-03-21T11:24:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94fe5f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f41ae630 con 0x55b7389f6580 2024-03-21T11:24:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94fe5f1740 10 client.0 _ll_register_callbacks cb 0x7f94e8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94f9a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f6580 0x55b738a024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94f9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f49f0 0x55b738a05a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94f9a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f6580 0x55b738a024b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33034/0 (socket says 172.21.15.47:33034) 2024-03-21T11:24:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94f9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f49f0 0x55b738a05a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51762/0 (socket says 172.21.15.47:51762) 2024-03-21T11:24:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94f9a00640 1 -- 172.21.15.47:0/1227861836 learned_addr learned my addr 172.21.15.47:0/1227861836 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94f3e00640 1 --2- 172.21.15.47:0/1227861836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 0x55b738a05f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94f9a00640 1 -- 172.21.15.47:0/1227861836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f49f0 msgr2=0x55b738a05a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94f9a00640 1 --2- 172.21.15.47:0/1227861836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f49f0 0x55b738a05a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94f9a00640 1 -- 172.21.15.47:0/1227861836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 msgr2=0x55b738a05f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:22.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94f9a00640 1 --2- 172.21.15.47:0/1227861836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 0x55b738a05f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94f9a00640 1 -- 172.21.15.47:0/1227861836 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b738a02b20 con 0x55b7389f6580 2024-03-21T11:24:22.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94f3e00640 1 --2- 172.21.15.47:0/1227861836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 0x55b738a05f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:22.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.231+0000 7f94f9000640 1 --2- 172.21.15.47:0/1227861836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f49f0 0x55b738a05a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:22.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.232+0000 7f94f9a00640 1 --2- 172.21.15.47:0/1227861836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f6580 0x55b738a024b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009620 tx=0x7f94ec003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57c8ca699f9fdfa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:22.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.232+0000 7f94d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:22.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.232+0000 7f94d6a00640 1 -- 172.21.15.47:0/1227861836 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec04c070 con 0x55b7389f6580 2024-03-21T11:24:22.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.232+0000 7f94d6a00640 1 -- 172.21.15.47:0/1227861836 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f94ec0349f0 con 0x55b7389f6580 2024-03-21T11:24:22.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.232+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1227861836 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94e802d070 con 0x55b7389f6580 2024-03-21T11:24:22.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.232+0000 7f94d6a00640 1 -- 172.21.15.47:0/1227861836 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94ec039070 con 0x55b7389f6580 2024-03-21T11:24:22.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.232+0000 7f94fe5f1740 10 client.25786 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:22.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.232+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1227861836 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b738a66610 con 0x55b7389f6580 2024-03-21T11:24:22.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.232+0000 7f94d6a00640 1 -- 172.21.15.47:0/1227861836 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94e802d070 con 0x55b7389f6580 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.232+0000 7f94d6a00640 1 -- 172.21.15.47:0/1227861836 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f94ec048450 con 0x55b7389f6580 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.232+0000 7f94fe5f1740 10 client.25786 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.233+0000 7f94fe5f1740 10 client.25786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.233+0000 7f94fe5f1740 20 client.25786 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.233+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1227861836 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b738a66f10 0x55b738a69330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.233+0000 7f94fe5f1740 4 client.25786 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.233+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1227861836 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b738960b50 con 0x55b738a66f10 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.233+0000 7f94f9000640 1 --2- 172.21.15.47:0/1227861836 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b738a66f10 0x55b738a69330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.233+0000 7f94f9000640 1 --2- 172.21.15.47:0/1227861836 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b738a66f10 0x55b738a69330 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4266b0cbbe55ebd server_cookie=5f41147ed64e5204 in_seq=0 out_seq=0 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.233+0000 7f94d6a00640 10 client.25786 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.233+0000 7f94d6a00640 1 -- 172.21.15.47:0/1227861836 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b738960b50 con 0x55b738a66f10 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.233+0000 7f94d6a00640 10 client.25786 handle_command_reply: tid=0 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.234+0000 7f94fe5f1740 1 client.25786 shutdown 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.234+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1227861836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f6580 msgr2=0x55b738a024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:22.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.234+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1227861836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f6580 0x55b738a024b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009620 tx=0x7f94ec003ba0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.234+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1227861836 shutdown_connections 2024-03-21T11:24:22.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.234+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1227861836 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b738a66f10 0x55b738a69330 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.234+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1227861836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7389f6580 0x55b738a024b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.234+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1227861836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7389f5bc0 0x55b738a05f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.234+0000 7f94fe5f1740 1 --2- 172.21.15.47:0/1227861836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7389f49f0 0x55b738a05a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.234+0000 7f94d6a00640 0 client.25786 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:22.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.234+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1227861836 >> 172.21.15.47:0/1227861836 conn(0x55b7389efde0 msgr2=0x55b7389ef160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:22.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.234+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1227861836 shutdown_connections 2024-03-21T11:24:22.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.234+0000 7f94fe5f1740 1 -- 172.21.15.47:0/1227861836 wait complete. 2024-03-21T11:24:22.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.234+0000 7f94fe5f1740 20 client.25786 trim_cache size 0 max 16384 2024-03-21T11:24:22.240 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:24:22.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.235+0000 7f94fa400640 1 -- 172.21.15.47:0/782241094 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f94dc0a2fb0 msgr2=0x7f94dc0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:22.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.235+0000 7f94fa400640 1 --2- 172.21.15.47:0/782241094 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f94dc0a2fb0 0x7f94dc0a5470 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f94e8005fd0 tx=0x7f94e800b410 comp rx=0 tx=0).stop 2024-03-21T11:24:22.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.235+0000 7f94fa400640 1 -- 172.21.15.47:0/782241094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f4108db0 msgr2=0x7f94f419e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:22.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.235+0000 7f94fa400640 1 --2- 172.21.15.47:0/782241094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f4108db0 0x7f94f419e340 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f94e400e9f0 tx=0x7f94e400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.235+0000 7f94fa400640 1 -- 172.21.15.47:0/782241094 shutdown_connections 2024-03-21T11:24:22.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.235+0000 7f94fa400640 1 --2- 172.21.15.47:0/782241094 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f94dc0a2fb0 0x7f94dc0a5470 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.235+0000 7f94fa400640 1 --2- 172.21.15.47:0/782241094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94f4109770 0x7f94f41a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.235+0000 7f94fa400640 1 --2- 172.21.15.47:0/782241094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94f4108db0 0x7f94f419e340 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.235+0000 7f94fa400640 1 --2- 172.21.15.47:0/782241094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f4107bc0 0x7f94f419de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:22.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.235+0000 7f94fa400640 1 -- 172.21.15.47:0/782241094 >> 172.21.15.47:0/782241094 conn(0x7f94f4076b70 msgr2=0x7f94f410c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:22.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.235+0000 7f94fa400640 1 -- 172.21.15.47:0/782241094 shutdown_connections 2024-03-21T11:24:22.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:22.235+0000 7f94fa400640 1 -- 172.21.15.47:0/782241094 wait complete. 2024-03-21T11:24:23.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:22 smithi102 ceph-mon[36748]: pgmap v431: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T11:24:23.255 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:24:23.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:24:23.256 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:24:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:22 smithi047 ceph-mon[31351]: pgmap v431: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T11:24:23.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:22 smithi100 ceph-mon[36245]: pgmap v431: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T11:24:23.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.490+0000 7f7d09e00640 1 -- 172.21.15.47:0/4083645307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d04105370 msgr2=0x7f7d0410bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.490+0000 7f7d09e00640 1 --2- 172.21.15.47:0/4083645307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d04105370 0x7f7d0410bc00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009be0 tx=0x7f7cfc02bef0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.490+0000 7f7d09e00640 1 -- 172.21.15.47:0/4083645307 shutdown_connections 2024-03-21T11:24:23.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.490+0000 7f7d09e00640 1 --2- 172.21.15.47:0/4083645307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d04105370 0x7f7d0410bc00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.490+0000 7f7d09e00640 1 --2- 172.21.15.47:0/4083645307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d041049b0 0x7f7d04104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.490+0000 7f7d09e00640 1 --2- 172.21.15.47:0/4083645307 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d041037c0 0x7f7d04103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.490+0000 7f7d09e00640 1 -- 172.21.15.47:0/4083645307 >> 172.21.15.47:0/4083645307 conn(0x7f7d040fed50 msgr2=0x7f7d04101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:23.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.490+0000 7f7d09e00640 1 -- 172.21.15.47:0/4083645307 shutdown_connections 2024-03-21T11:24:23.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.490+0000 7f7d09e00640 1 -- 172.21.15.47:0/4083645307 wait complete. 2024-03-21T11:24:23.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.491+0000 7f7d09e00640 1 Processor -- start 2024-03-21T11:24:23.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.491+0000 7f7d09e00640 1 -- start start 2024-03-21T11:24:23.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.492+0000 7f7d09e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d041037c0 0x7f7d04199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.492+0000 7f7d09e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d041049b0 0x7f7d04199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.492+0000 7f7d09e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d04105370 0x7f7d041a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.492+0000 7f7d09e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0410f470 con 0x7f7d041037c0 2024-03-21T11:24:23.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.492+0000 7f7d09e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0410f2f0 con 0x7f7d041049b0 2024-03-21T11:24:23.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.492+0000 7f7d09e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0410f5f0 con 0x7f7d04105370 2024-03-21T11:24:23.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.492+0000 7f7d08a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d041037c0 0x7f7d04199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.492+0000 7f7d03e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d041049b0 0x7f7d04199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.492+0000 7f7d03e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d041049b0 0x7f7d04199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51784/0 (socket says 172.21.15.47:51784) 2024-03-21T11:24:23.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.492+0000 7f7d03e00640 1 -- 172.21.15.47:0/1537652387 learned_addr learned my addr 172.21.15.47:0/1537652387 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:23.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.492+0000 7f7d09400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d04105370 0x7f7d041a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.493+0000 7f7d03e00640 1 -- 172.21.15.47:0/1537652387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d04105370 msgr2=0x7f7d041a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.493+0000 7f7d03e00640 1 --2- 172.21.15.47:0/1537652387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d04105370 0x7f7d041a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.493+0000 7f7d03e00640 1 -- 172.21.15.47:0/1537652387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d041037c0 msgr2=0x7f7d04199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.493+0000 7f7d03e00640 1 --2- 172.21.15.47:0/1537652387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d041037c0 0x7f7d04199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.493+0000 7f7d03e00640 1 -- 172.21.15.47:0/1537652387 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d041a1640 con 0x7f7d041049b0 2024-03-21T11:24:23.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.493+0000 7f7d03e00640 1 --2- 172.21.15.47:0/1537652387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d041049b0 0x7f7d04199f60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000e9f0 tx=0x7f7cf000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:23.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.493+0000 7f7d01600640 1 -- 172.21.15.47:0/1537652387 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf000ccc0 con 0x7f7d041049b0 2024-03-21T11:24:23.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.493+0000 7f7d01600640 1 -- 172.21.15.47:0/1537652387 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cf0010910 con 0x7f7d041049b0 2024-03-21T11:24:23.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.493+0000 7f7d09e00640 1 -- 172.21.15.47:0/1537652387 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d041a1930 con 0x7f7d041049b0 2024-03-21T11:24:23.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.493+0000 7f7d01600640 1 -- 172.21.15.47:0/1537652387 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf00188a0 con 0x7f7d041049b0 2024-03-21T11:24:23.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.493+0000 7f7d09e00640 1 -- 172.21.15.47:0/1537652387 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d041a1ed0 con 0x7f7d041049b0 2024-03-21T11:24:23.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.496+0000 7f7d01600640 1 -- 172.21.15.47:0/1537652387 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f7cf0010ac0 con 0x7f7d041049b0 2024-03-21T11:24:23.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.497+0000 7f7d0e1d7740 1 Processor -- start 2024-03-21T11:24:23.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.497+0000 7f7d0e1d7740 1 -- start start 2024-03-21T11:24:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.497+0000 7f7d0e1d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77dc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.497+0000 7f7d0e1d7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77e17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.497+0000 7f7d0e1d7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 0x564ed77e42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.497+0000 7f7d0e1d7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed7741eb0 con 0x564ed77e1e10 2024-03-21T11:24:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.497+0000 7f7d0e1d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed76eaa60 con 0x564ed77dc2b0 2024-03-21T11:24:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.497+0000 7f7d0e1d7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed773df50 con 0x564ed77dcca0 2024-03-21T11:24:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.497+0000 7f7d09400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77dc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.497+0000 7f7d08a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77e17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.497+0000 7f7d09400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77dc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51800/0 (socket says 172.21.15.47:51800) 2024-03-21T11:24:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.497+0000 7f7d09400640 1 -- 172.21.15.47:0/3374167123 learned_addr learned my addr 172.21.15.47:0/3374167123 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.497+0000 7f7d03e00640 1 --2- 172.21.15.47:0/3374167123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 0x564ed77e42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.498+0000 7f7d08a00640 1 -- 172.21.15.47:0/3374167123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 msgr2=0x564ed77dc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.498+0000 7f7d08a00640 1 --2- 172.21.15.47:0/3374167123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77dc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.498+0000 7f7d08a00640 1 -- 172.21.15.47:0/3374167123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 msgr2=0x564ed77e42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.498+0000 7f7d08a00640 1 --2- 172.21.15.47:0/3374167123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 0x564ed77e42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.498+0000 7f7d08a00640 1 -- 172.21.15.47:0/3374167123 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ed751e7e0 con 0x564ed77dcca0 2024-03-21T11:24:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.498+0000 7f7d08a00640 1 --2- 172.21.15.47:0/3374167123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77e17a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8005fd0 tx=0x7f7cf8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f052205e7e517f76 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.498+0000 7f7cf5c00640 1 -- 172.21.15.47:0/3374167123 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf801e070 con 0x564ed77dcca0 2024-03-21T11:24:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.498+0000 7f7cf5c00640 1 -- 172.21.15.47:0/3374167123 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cf8009db0 con 0x564ed77dcca0 2024-03-21T11:24:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.498+0000 7f7cf5c00640 1 -- 172.21.15.47:0/3374167123 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8019450 con 0x564ed77dcca0 2024-03-21T11:24:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.498+0000 7f7d01600640 1 --2- 172.21.15.47:0/1537652387 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7cec0a3030 0x7f7cec0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.499+0000 7f7d01600640 1 -- 172.21.15.47:0/1537652387 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7cf00cfd30 con 0x7f7d041049b0 2024-03-21T11:24:23.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.499+0000 7f7d09400640 1 --2- 172.21.15.47:0/1537652387 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7cec0a3030 0x7f7cec0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.499+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/3374167123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 msgr2=0x564ed77e17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.499+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/3374167123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77e17a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8005fd0 tx=0x7f7cf8011810 comp rx=0 tx=0).stop 2024-03-21T11:24:23.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.499+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/3374167123 shutdown_connections 2024-03-21T11:24:23.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.499+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/3374167123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 0x564ed77e42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.499+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/3374167123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77e17a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.499+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/3374167123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77dc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.499+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/3374167123 >> 172.21.15.47:0/3374167123 conn(0x564ed77d7820 msgr2=0x564ed77d7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.499+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/3374167123 shutdown_connections 2024-03-21T11:24:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.500+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/3374167123 wait complete. 2024-03-21T11:24:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.500+0000 7f7d09400640 1 --2- 172.21.15.47:0/1537652387 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7cec0a3030 0x7f7cec0a54f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0099a0 tx=0x7f7cfc004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.500+0000 7f7d0e1d7740 1 -- start start 2024-03-21T11:24:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.500+0000 7f7d0e1d7740 1 Processor -- start 2024-03-21T11:24:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d0e1d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77e64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d0e1d7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d0e1d7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 0x564ed77eda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d0e1d7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed77e4dc0 con 0x564ed77e1e10 2024-03-21T11:24:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d0e1d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed76fdeb0 con 0x564ed77dc2b0 2024-03-21T11:24:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d0e1d7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed77e4f40 con 0x564ed77dcca0 2024-03-21T11:24:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d0e1d7740 10 client.0 _ll_register_callbacks cb 0x564ed76f2150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d08a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77e64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d03e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d08a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77e64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51808/0 (socket says 172.21.15.47:51808) 2024-03-21T11:24:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d08a00640 1 -- 172.21.15.47:0/3298167232 learned_addr learned my addr 172.21.15.47:0/3298167232 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d09400640 1 --2- 172.21.15.47:0/3298167232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 0x564ed77eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d08a00640 1 -- 172.21.15.47:0/3298167232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 msgr2=0x564ed77e6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d08a00640 1 --2- 172.21.15.47:0/3298167232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77e6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d08a00640 1 -- 172.21.15.47:0/3298167232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 msgr2=0x564ed77eda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d08a00640 1 --2- 172.21.15.47:0/3298167232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 0x564ed77eda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d08a00640 1 -- 172.21.15.47:0/3298167232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ed757ef70 con 0x564ed77dc2b0 2024-03-21T11:24:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d03e00640 1 --2- 172.21.15.47:0/3298167232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.501+0000 7f7d08a00640 1 --2- 172.21.15.47:0/3298167232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77e64f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8011d10 tx=0x7f7cf80056a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16b54bd25634fb4b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.502+0000 7f7cea000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:24:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.502+0000 7f7cea000640 1 -- 172.21.15.47:0/3298167232 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf801e070 con 0x564ed77dc2b0 2024-03-21T11:24:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.502+0000 7f7cea000640 1 -- 172.21.15.47:0/3298167232 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7cf8003d60 con 0x564ed77dc2b0 2024-03-21T11:24:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.502+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/3298167232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7cf8009db0 con 0x564ed77dc2b0 2024-03-21T11:24:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.502+0000 7f7cea000640 1 -- 172.21.15.47:0/3298167232 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cf8031050 con 0x564ed77dc2b0 2024-03-21T11:24:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.502+0000 7f7d0e1d7740 10 client.25781 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.502+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/3298167232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564ed7416170 con 0x564ed77dc2b0 2024-03-21T11:24:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.502+0000 7f7cea000640 1 -- 172.21.15.47:0/3298167232 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cf8009db0 con 0x564ed77dc2b0 2024-03-21T11:24:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.502+0000 7f7cea000640 1 -- 172.21.15.47:0/3298167232 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7cf8004ba0 con 0x564ed77dc2b0 2024-03-21T11:24:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.503+0000 7f7d0e1d7740 10 client.25781 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.503+0000 7f7d0e1d7740 10 client.25781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.503+0000 7f7d0e1d7740 20 client.25781 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.503+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/3298167232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ed77f2010 0x564ed77f4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.503+0000 7f7d0e1d7740 4 client.25781 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.503+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/3298167232 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ed774f3f0 con 0x564ed77f2010 2024-03-21T11:24:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.503+0000 7f7d03e00640 1 --2- 172.21.15.47:0/3298167232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ed77f2010 0x564ed77f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.503+0000 7f7d03e00640 1 --2- 172.21.15.47:0/3298167232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ed77f2010 0x564ed77f4420 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84f97d481b3c62c1 server_cookie=c8218c60048b1186 in_seq=0 out_seq=0 2024-03-21T11:24:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.503+0000 7f7cea000640 10 client.25781 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.504+0000 7f7cea000640 1 -- 172.21.15.47:0/3298167232 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564ed774f3f0 con 0x564ed77f2010 2024-03-21T11:24:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.504+0000 7f7cea000640 10 client.25781 handle_command_reply: tid=0 2024-03-21T11:24:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.504+0000 7f7d0e1d7740 1 client.25781 shutdown 2024-03-21T11:24:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.504+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/3298167232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 msgr2=0x564ed77e64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.504+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/3298167232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77e64f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8011d10 tx=0x7f7cf80056a0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.504+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/3298167232 shutdown_connections 2024-03-21T11:24:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.504+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/3298167232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ed77f2010 0x564ed77f4420 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.504+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/3298167232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 0x564ed77eda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.504+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/3298167232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.504+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/3298167232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77e64f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.504+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/3298167232 >> 172.21.15.47:0/3298167232 conn(0x564ed77d7820 msgr2=0x564ed77d6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.504+0000 7f7cea000640 0 client.25781 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.505+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/3298167232 shutdown_connections 2024-03-21T11:24:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.505+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/3298167232 wait complete. 2024-03-21T11:24:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.505+0000 7f7d0e1d7740 20 client.25781 trim_cache size 0 max 16384 2024-03-21T11:24:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.522+0000 7f7d0e1d7740 1 Processor -- start 2024-03-21T11:24:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.522+0000 7f7d0e1d7740 1 -- start start 2024-03-21T11:24:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d0e1d7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dc2b0 0x564ed77dd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d0e1d7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77dcca0 0x564ed77ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d0e1d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77e1e10 0x564ed77de190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d0e1d7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d041ac190 con 0x564ed77dcca0 2024-03-21T11:24:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d0e1d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d041adb10 con 0x564ed77e1e10 2024-03-21T11:24:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d0e1d7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d041ac7f0 con 0x564ed77dc2b0 2024-03-21T11:24:23.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d08a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dc2b0 0x564ed77dd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d03e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77dcca0 0x564ed77ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d08a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dc2b0 0x564ed77dd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33078/0 (socket says 172.21.15.47:33078) 2024-03-21T11:24:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d08a00640 1 -- 172.21.15.47:0/2400198007 learned_addr learned my addr 172.21.15.47:0/2400198007 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d09400640 1 --2- 172.21.15.47:0/2400198007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77e1e10 0x564ed77de190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d08a00640 1 -- 172.21.15.47:0/2400198007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77e1e10 msgr2=0x564ed77de190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d08a00640 1 --2- 172.21.15.47:0/2400198007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77e1e10 0x564ed77de190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d08a00640 1 -- 172.21.15.47:0/2400198007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77dcca0 msgr2=0x564ed77ddc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d08a00640 1 --2- 172.21.15.47:0/2400198007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77dcca0 0x564ed77ddc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.523+0000 7f7d08a00640 1 -- 172.21.15.47:0/2400198007 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ed77e4660 con 0x564ed77dc2b0 2024-03-21T11:24:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.524+0000 7f7d08a00640 1 --2- 172.21.15.47:0/2400198007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dc2b0 0x564ed77dd710 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8004fd0 tx=0x7f7cf8005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecb517641754b3d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.524+0000 7f7cf5200640 1 -- 172.21.15.47:0/2400198007 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf801e040 con 0x564ed77dc2b0 2024-03-21T11:24:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.524+0000 7f7cf5200640 1 -- 172.21.15.47:0/2400198007 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cf8028070 con 0x564ed77dc2b0 2024-03-21T11:24:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.524+0000 7f7cf5200640 1 -- 172.21.15.47:0/2400198007 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8003dd0 con 0x564ed77dc2b0 2024-03-21T11:24:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.524+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/2400198007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dc2b0 msgr2=0x564ed77dd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.524+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/2400198007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dc2b0 0x564ed77dd710 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8004fd0 tx=0x7f7cf8005750 comp rx=0 tx=0).stop 2024-03-21T11:24:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.524+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/2400198007 shutdown_connections 2024-03-21T11:24:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.524+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/2400198007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77e1e10 0x564ed77de190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.524+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/2400198007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77dcca0 0x564ed77ddc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.524+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/2400198007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dc2b0 0x564ed77dd710 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.524+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/2400198007 >> 172.21.15.47:0/2400198007 conn(0x564ed77d7820 msgr2=0x564ed77db310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.525+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/2400198007 shutdown_connections 2024-03-21T11:24:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.525+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/2400198007 wait complete. 2024-03-21T11:24:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.525+0000 7f7d0e1d7740 1 -- start start 2024-03-21T11:24:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.525+0000 7f7d0e1d7740 1 Processor -- start 2024-03-21T11:24:23.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d0e1d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77ea9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d0e1d7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77eaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d0e1d7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 0x564ed77ebef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d0e1d7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d041ad4b0 con 0x564ed77e1e10 2024-03-21T11:24:23.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d0e1d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d041ace50 con 0x564ed77dc2b0 2024-03-21T11:24:23.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d0e1d7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d041aa350 con 0x564ed77dcca0 2024-03-21T11:24:23.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d0e1d7740 10 client.0 _ll_register_callbacks cb 0x7f7cf801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:23.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d08a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77ea9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d03e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77eaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d08a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77ea9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51818/0 (socket says 172.21.15.47:51818) 2024-03-21T11:24:23.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d08a00640 1 -- 172.21.15.47:0/2361057498 learned_addr learned my addr 172.21.15.47:0/2361057498 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:23.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d09400640 1 --2- 172.21.15.47:0/2361057498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 0x564ed77ebef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d08a00640 1 -- 172.21.15.47:0/2361057498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 msgr2=0x564ed77eaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d08a00640 1 --2- 172.21.15.47:0/2361057498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77eaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d08a00640 1 -- 172.21.15.47:0/2361057498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 msgr2=0x564ed77ebef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d08a00640 1 --2- 172.21.15.47:0/2361057498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 0x564ed77ebef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d08a00640 1 -- 172.21.15.47:0/2361057498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ed77ec4e0 con 0x564ed77dc2b0 2024-03-21T11:24:23.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.526+0000 7f7d03e00640 1 --2- 172.21.15.47:0/2361057498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77eaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:23.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.527+0000 7f7d08a00640 1 --2- 172.21.15.47:0/2361057498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77ea9f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8003840 tx=0x7f7cf8019860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb76b9793176644 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:23.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.527+0000 7f7cea000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:24:23.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.527+0000 7f7cea000640 1 -- 172.21.15.47:0/2361057498 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf801e070 con 0x564ed77dc2b0 2024-03-21T11:24:23.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.527+0000 7f7cea000640 1 -- 172.21.15.47:0/2361057498 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7cf8003b50 con 0x564ed77dc2b0 2024-03-21T11:24:23.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.527+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/2361057498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7cf8028070 con 0x564ed77dc2b0 2024-03-21T11:24:23.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.527+0000 7f7cea000640 1 -- 172.21.15.47:0/2361057498 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cf8003d70 con 0x564ed77dc2b0 2024-03-21T11:24:23.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.527+0000 7f7d0e1d7740 10 client.25787 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:23.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.527+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/2361057498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564ed78561c0 con 0x564ed77dc2b0 2024-03-21T11:24:23.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.527+0000 7f7cea000640 1 -- 172.21.15.47:0/2361057498 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cf8028070 con 0x564ed77dc2b0 2024-03-21T11:24:23.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.528+0000 7f7cea000640 1 -- 172.21.15.47:0/2361057498 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7cf8036a50 con 0x564ed77dc2b0 2024-03-21T11:24:23.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.528+0000 7f7d0e1d7740 10 client.25787 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:23.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.528+0000 7f7d0e1d7740 10 client.25787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:23.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.528+0000 7f7d0e1d7740 20 client.25787 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:23.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.528+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/2361057498 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ed78569a0 0x564ed7858dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.528+0000 7f7d0e1d7740 4 client.25787 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:24:23.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.528+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/2361057498 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564ed7749740 con 0x564ed78569a0 2024-03-21T11:24:23.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.528+0000 7f7d03e00640 1 --2- 172.21.15.47:0/2361057498 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ed78569a0 0x564ed7858dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.528+0000 7f7d03e00640 1 --2- 172.21.15.47:0/2361057498 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ed78569a0 0x564ed7858dc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=792ca0935d13a9ad server_cookie=22f3520c9d540c9 in_seq=0 out_seq=0 2024-03-21T11:24:23.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.528+0000 7f7cea000640 10 client.25787 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:23.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7cea000640 1 -- 172.21.15.47:0/2361057498 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564ed7749740 con 0x564ed78569a0 2024-03-21T11:24:23.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7cea000640 10 client.25787 handle_command_reply: tid=0 2024-03-21T11:24:23.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7d0e1d7740 1 client.25787 shutdown 2024-03-21T11:24:23.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/2361057498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 msgr2=0x564ed77ea9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/2361057498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77ea9f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8003840 tx=0x7f7cf8019860 comp rx=0 tx=0).stop 2024-03-21T11:24:23.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/2361057498 shutdown_connections 2024-03-21T11:24:23.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/2361057498 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ed78569a0 0x564ed7858dc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/2361057498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ed77e1e10 0x564ed77ebef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/2361057498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ed77dcca0 0x564ed77eaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7d0e1d7740 1 --2- 172.21.15.47:0/2361057498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ed77dc2b0 0x564ed77ea9f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7cea000640 0 client.25787 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:23.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/2361057498 >> 172.21.15.47:0/2361057498 conn(0x564ed77d7820 msgr2=0x564ed77d61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:23.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/2361057498 shutdown_connections 2024-03-21T11:24:23.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7d0e1d7740 1 -- 172.21.15.47:0/2361057498 wait complete. 2024-03-21T11:24:23.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.530+0000 7f7d0e1d7740 20 client.25787 trim_cache size 0 max 16384 2024-03-21T11:24:23.536 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:24:23.536 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:24:23.536 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "60490ad2-80f5-4219-9d1e-022b88eb8cbe", 2024-03-21T11:24:23.536 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:24:23.536 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:24:23.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.531+0000 7f7d09e00640 1 -- 172.21.15.47:0/1537652387 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7cec0a3030 msgr2=0x7f7cec0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.531+0000 7f7d09e00640 1 --2- 172.21.15.47:0/1537652387 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7cec0a3030 0x7f7cec0a54f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0099a0 tx=0x7f7cfc004a40 comp rx=0 tx=0).stop 2024-03-21T11:24:23.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.531+0000 7f7d09e00640 1 -- 172.21.15.47:0/1537652387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d041049b0 msgr2=0x7f7d04199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.531+0000 7f7d09e00640 1 --2- 172.21.15.47:0/1537652387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d041049b0 0x7f7d04199f60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000e9f0 tx=0x7f7cf000edf0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.531+0000 7f7d09e00640 1 -- 172.21.15.47:0/1537652387 shutdown_connections 2024-03-21T11:24:23.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.531+0000 7f7d09e00640 1 --2- 172.21.15.47:0/1537652387 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7cec0a3030 0x7f7cec0a54f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.531+0000 7f7d09e00640 1 --2- 172.21.15.47:0/1537652387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d04105370 0x7f7d041a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.531+0000 7f7d09e00640 1 --2- 172.21.15.47:0/1537652387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d041049b0 0x7f7d04199f60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.531+0000 7f7d09e00640 1 --2- 172.21.15.47:0/1537652387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d041037c0 0x7f7d04199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.531+0000 7f7d09e00640 1 -- 172.21.15.47:0/1537652387 >> 172.21.15.47:0/1537652387 conn(0x7f7d040fed50 msgr2=0x7f7d04107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:23.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.531+0000 7f7d09e00640 1 -- 172.21.15.47:0/1537652387 shutdown_connections 2024-03-21T11:24:23.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.531+0000 7f7d09e00640 1 -- 172.21.15.47:0/1537652387 wait complete. 2024-03-21T11:24:23.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:24:23.549 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:24:23.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.762+0000 7fb57ce00640 1 -- 172.21.15.47:0/572084243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb578104990 msgr2=0x7fb578104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.762+0000 7fb57ce00640 1 --2- 172.21.15.47:0/572084243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb578104990 0x7fb578104e10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb560009900 tx=0x7fb56002bc30 comp rx=0 tx=0).stop 2024-03-21T11:24:23.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.762+0000 7fb57ce00640 1 -- 172.21.15.47:0/572084243 shutdown_connections 2024-03-21T11:24:23.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.762+0000 7fb57ce00640 1 --2- 172.21.15.47:0/572084243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb578105350 0x7fb57810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.762+0000 7fb57ce00640 1 --2- 172.21.15.47:0/572084243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb578104990 0x7fb578104e10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.762+0000 7fb57ce00640 1 --2- 172.21.15.47:0/572084243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5781037a0 0x7fb578103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.762+0000 7fb57ce00640 1 -- 172.21.15.47:0/572084243 >> 172.21.15.47:0/572084243 conn(0x7fb5780fed10 msgr2=0x7fb578101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:23.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.762+0000 7fb57ce00640 1 -- 172.21.15.47:0/572084243 shutdown_connections 2024-03-21T11:24:23.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.762+0000 7fb57ce00640 1 -- 172.21.15.47:0/572084243 wait complete. 2024-03-21T11:24:23.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.764+0000 7fb57ce00640 1 Processor -- start 2024-03-21T11:24:23.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.764+0000 7fb57ce00640 1 -- start start 2024-03-21T11:24:23.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.764+0000 7fb57ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5781037a0 0x7fb578199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.764+0000 7fb57ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb578104990 0x7fb578199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.764+0000 7fb57ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb578105350 0x7fb5781a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.764+0000 7fb57ce00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57810f450 con 0x7fb5781037a0 2024-03-21T11:24:23.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.764+0000 7fb57ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57810f2d0 con 0x7fb578105350 2024-03-21T11:24:23.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.764+0000 7fb57ce00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57810f5d0 con 0x7fb578104990 2024-03-21T11:24:23.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.765+0000 7fb577400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5781037a0 0x7fb578199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.765+0000 7fb56fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb578104990 0x7fb578199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.765+0000 7fb577400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5781037a0 0x7fb578199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59178/0 (socket says 172.21.15.47:59178) 2024-03-21T11:24:23.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.765+0000 7fb577400640 1 -- 172.21.15.47:0/539879073 learned_addr learned my addr 172.21.15.47:0/539879073 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:23.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.765+0000 7fb577e00640 1 --2- 172.21.15.47:0/539879073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb578105350 0x7fb5781a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.765+0000 7fb56fe00640 1 -- 172.21.15.47:0/539879073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb578105350 msgr2=0x7fb5781a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.765+0000 7fb56fe00640 1 --2- 172.21.15.47:0/539879073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb578105350 0x7fb5781a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.765+0000 7fb56fe00640 1 -- 172.21.15.47:0/539879073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5781037a0 msgr2=0x7fb578199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.765+0000 7fb56fe00640 1 --2- 172.21.15.47:0/539879073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5781037a0 0x7fb578199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.765+0000 7fb56fe00640 1 -- 172.21.15.47:0/539879073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5781a16b0 con 0x7fb578104990 2024-03-21T11:24:23.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.765+0000 7fb577400640 1 --2- 172.21.15.47:0/539879073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5781037a0 0x7fb578199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:23.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.765+0000 7fb56fe00640 1 --2- 172.21.15.47:0/539879073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb578104990 0x7fb578199fd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb560036040 tx=0x7fb56002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:23.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.766+0000 7fb574c00640 1 -- 172.21.15.47:0/539879073 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560043020 con 0x7fb578104990 2024-03-21T11:24:23.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.766+0000 7fb574c00640 1 -- 172.21.15.47:0/539879073 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5600054e0 con 0x7fb578104990 2024-03-21T11:24:23.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.766+0000 7fb57ce00640 1 -- 172.21.15.47:0/539879073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5781a1940 con 0x7fb578104990 2024-03-21T11:24:23.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.766+0000 7fb574c00640 1 -- 172.21.15.47:0/539879073 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560003c10 con 0x7fb578104990 2024-03-21T11:24:23.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.766+0000 7fb57ce00640 1 -- 172.21.15.47:0/539879073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5781a1eb0 con 0x7fb578104990 2024-03-21T11:24:23.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.769+0000 7fb5811ba740 1 Processor -- start 2024-03-21T11:24:23.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.769+0000 7fb5811ba740 1 -- start start 2024-03-21T11:24:23.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.769+0000 7fb5811ba740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c6b050 0x556325c6b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.769+0000 7fb574c00640 1 -- 172.21.15.47:0/539879073 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fb560046020 con 0x7fb578104990 2024-03-21T11:24:23.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.769+0000 7fb5811ba740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6ba40 0x556325c70540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.769+0000 7fb5811ba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c70bb0 0x556325c73040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.769+0000 7fb5811ba740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556325bd0250 con 0x556325c6b050 2024-03-21T11:24:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.769+0000 7fb5811ba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556325b78a30 con 0x556325c70bb0 2024-03-21T11:24:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.769+0000 7fb5811ba740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556325bcc2f0 con 0x556325c6ba40 2024-03-21T11:24:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.770+0000 7fb56fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c70bb0 0x556325c73040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.770+0000 7fb577400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6ba40 0x556325c70540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.770+0000 7fb577e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c6b050 0x556325c6b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.770+0000 7fb577400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6ba40 0x556325c70540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33110/0 (socket says 172.21.15.47:33110) 2024-03-21T11:24:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.770+0000 7fb577400640 1 -- 172.21.15.47:0/3363078518 learned_addr learned my addr 172.21.15.47:0/3363078518 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.770+0000 7fb577400640 1 -- 172.21.15.47:0/3363078518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c70bb0 msgr2=0x556325c73040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.770+0000 7fb577400640 1 --2- 172.21.15.47:0/3363078518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c70bb0 0x556325c73040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.770+0000 7fb577400640 1 -- 172.21.15.47:0/3363078518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c6b050 msgr2=0x556325c6b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.770+0000 7fb577400640 1 --2- 172.21.15.47:0/3363078518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c6b050 0x556325c6b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.770+0000 7fb577400640 1 -- 172.21.15.47:0/3363078518 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556325b63690 con 0x556325c6ba40 2024-03-21T11:24:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.770+0000 7fb56fe00640 1 --2- 172.21.15.47:0/3363078518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c70bb0 0x556325c73040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.771+0000 7fb577400640 1 --2- 172.21.15.47:0/3363078518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6ba40 0x556325c70540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb5680115c0 tx=0x7fb568011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4e9967553876bd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.771+0000 7fb56cc00640 1 -- 172.21.15.47:0/3363078518 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568005020 con 0x556325c6ba40 2024-03-21T11:24:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.771+0000 7fb56cc00640 1 -- 172.21.15.47:0/3363078518 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5680047a0 con 0x556325c6ba40 2024-03-21T11:24:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.771+0000 7fb56cc00640 1 -- 172.21.15.47:0/3363078518 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56800fa00 con 0x556325c6ba40 2024-03-21T11:24:23.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.771+0000 7fb574c00640 1 --2- 172.21.15.47:0/539879073 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb55c0a3030 0x7fb55c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.771+0000 7fb574c00640 1 -- 172.21.15.47:0/539879073 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb5600f1790 con 0x7fb578104990 2024-03-21T11:24:23.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.772+0000 7fb577e00640 1 --2- 172.21.15.47:0/539879073 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb55c0a3030 0x7fb55c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.772+0000 7fb5811ba740 1 -- 172.21.15.47:0/3363078518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6ba40 msgr2=0x556325c70540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.772+0000 7fb5811ba740 1 --2- 172.21.15.47:0/3363078518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6ba40 0x556325c70540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb5680115c0 tx=0x7fb568011a80 comp rx=0 tx=0).stop 2024-03-21T11:24:23.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.772+0000 7fb5811ba740 1 -- 172.21.15.47:0/3363078518 shutdown_connections 2024-03-21T11:24:23.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.772+0000 7fb5811ba740 1 --2- 172.21.15.47:0/3363078518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c70bb0 0x556325c73040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.772+0000 7fb5811ba740 1 --2- 172.21.15.47:0/3363078518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6ba40 0x556325c70540 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.772+0000 7fb5811ba740 1 --2- 172.21.15.47:0/3363078518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c6b050 0x556325c6b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.772+0000 7fb5811ba740 1 -- 172.21.15.47:0/3363078518 >> 172.21.15.47:0/3363078518 conn(0x556325c33200 msgr2=0x556325c669c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:23.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.772+0000 7fb5811ba740 1 -- 172.21.15.47:0/3363078518 shutdown_connections 2024-03-21T11:24:23.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.772+0000 7fb5811ba740 1 -- 172.21.15.47:0/3363078518 wait complete. 2024-03-21T11:24:23.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.772+0000 7fb577e00640 1 --2- 172.21.15.47:0/539879073 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb55c0a3030 0x7fb55c0a54f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x556325c6c2a0 tx=0x7fb570009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:23.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.773+0000 7fb5811ba740 1 -- start start 2024-03-21T11:24:23.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.773+0000 7fb5811ba740 1 Processor -- start 2024-03-21T11:24:23.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.773+0000 7fb5811ba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6b050 0x556325c75090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.773+0000 7fb5811ba740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c6ba40 0x556325c755d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.773+0000 7fb5811ba740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c70bb0 0x556325c7c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.773+0000 7fb5811ba740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556325c73b60 con 0x556325c6ba40 2024-03-21T11:24:23.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.773+0000 7fb5811ba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556325b8be80 con 0x556325c6b050 2024-03-21T11:24:23.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.773+0000 7fb5811ba740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556325c73ce0 con 0x556325c70bb0 2024-03-21T11:24:23.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.773+0000 7fb5811ba740 10 client.0 _ll_register_callbacks cb 0x556325b9e130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:23.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.773+0000 7fb577e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c70bb0 0x556325c7c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.773+0000 7fb577400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6b050 0x556325c75090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.774+0000 7fb577e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c70bb0 0x556325c7c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33114/0 (socket says 172.21.15.47:33114) 2024-03-21T11:24:23.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.774+0000 7fb56fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c6ba40 0x556325c755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.774+0000 7fb577e00640 1 -- 172.21.15.47:0/1823055372 learned_addr learned my addr 172.21.15.47:0/1823055372 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:23.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.774+0000 7fb577e00640 1 -- 172.21.15.47:0/1823055372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6b050 msgr2=0x556325c75090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.774+0000 7fb577e00640 1 --2- 172.21.15.47:0/1823055372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6b050 0x556325c75090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.774+0000 7fb577e00640 1 -- 172.21.15.47:0/1823055372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c6ba40 msgr2=0x556325c755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.774+0000 7fb577e00640 1 --2- 172.21.15.47:0/1823055372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c6ba40 0x556325c755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.774+0000 7fb577e00640 1 -- 172.21.15.47:0/1823055372 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563259ae7d0 con 0x556325c70bb0 2024-03-21T11:24:23.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.774+0000 7fb577e00640 1 --2- 172.21.15.47:0/1823055372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c70bb0 0x556325c7c610 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb570009cd0 tx=0x7fb570004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7230d190fd5b844a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:23.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.774+0000 7fb565600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:23.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.774+0000 7fb565600640 1 -- 172.21.15.47:0/1823055372 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57000f830 con 0x556325c70bb0 2024-03-21T11:24:23.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.774+0000 7fb5811ba740 1 -- 172.21.15.47:0/1823055372 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5680047a0 con 0x556325c70bb0 2024-03-21T11:24:23.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.774+0000 7fb565600640 1 -- 172.21.15.47:0/1823055372 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb57000f9d0 con 0x556325c70bb0 2024-03-21T11:24:23.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.775+0000 7fb565600640 1 -- 172.21.15.47:0/1823055372 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb57000fbd0 con 0x556325c70bb0 2024-03-21T11:24:23.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.775+0000 7fb5811ba740 10 client.25816 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:23.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.775+0000 7fb5811ba740 1 -- 172.21.15.47:0/1823055372 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5563258e43f0 con 0x556325c70bb0 2024-03-21T11:24:23.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.775+0000 7fb565600640 1 -- 172.21.15.47:0/1823055372 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5680047a0 con 0x556325c70bb0 2024-03-21T11:24:23.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.775+0000 7fb565600640 1 -- 172.21.15.47:0/1823055372 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb570026070 con 0x556325c70bb0 2024-03-21T11:24:23.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.775+0000 7fb5811ba740 10 client.25816 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:23.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.775+0000 7fb5811ba740 10 client.25816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:23.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.775+0000 7fb5811ba740 20 client.25816 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:23.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.775+0000 7fb5811ba740 1 --2- 172.21.15.47:0/1823055372 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556325c80f40 0x556325c83360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.775+0000 7fb5811ba740 4 client.25816 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:23.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.775+0000 7fb5811ba740 1 -- 172.21.15.47:0/1823055372 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556325bdd790 con 0x556325c80f40 2024-03-21T11:24:23.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.776+0000 7fb577400640 1 --2- 172.21.15.47:0/1823055372 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556325c80f40 0x556325c83360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.776+0000 7fb577400640 1 --2- 172.21.15.47:0/1823055372 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556325c80f40 0x556325c83360 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7124848f64657c48 server_cookie=8b6e7f2ad9061dff in_seq=0 out_seq=0 2024-03-21T11:24:23.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.776+0000 7fb565600640 10 client.25816 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:23.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb565600640 1 -- 172.21.15.47:0/1823055372 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556325bdd790 con 0x556325c80f40 2024-03-21T11:24:23.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb565600640 10 client.25816 handle_command_reply: tid=0 2024-03-21T11:24:23.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb5811ba740 1 client.25816 shutdown 2024-03-21T11:24:23.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb5811ba740 1 -- 172.21.15.47:0/1823055372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c70bb0 msgr2=0x556325c7c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb5811ba740 1 --2- 172.21.15.47:0/1823055372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c70bb0 0x556325c7c610 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb570009cd0 tx=0x7fb570004ab0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb5811ba740 1 -- 172.21.15.47:0/1823055372 shutdown_connections 2024-03-21T11:24:23.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb5811ba740 1 --2- 172.21.15.47:0/1823055372 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556325c80f40 0x556325c83360 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb5811ba740 1 --2- 172.21.15.47:0/1823055372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c70bb0 0x556325c7c610 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb5811ba740 1 --2- 172.21.15.47:0/1823055372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c6ba40 0x556325c755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb5811ba740 1 --2- 172.21.15.47:0/1823055372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6b050 0x556325c75090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb5811ba740 1 -- 172.21.15.47:0/1823055372 >> 172.21.15.47:0/1823055372 conn(0x556325c33200 msgr2=0x556325c65bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:23.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb565600640 0 client.25816 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:23.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb5811ba740 1 -- 172.21.15.47:0/1823055372 shutdown_connections 2024-03-21T11:24:23.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb5811ba740 1 -- 172.21.15.47:0/1823055372 wait complete. 2024-03-21T11:24:23.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.777+0000 7fb5811ba740 20 client.25816 trim_cache size 0 max 16384 2024-03-21T11:24:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.795+0000 7fb5811ba740 1 Processor -- start 2024-03-21T11:24:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.795+0000 7fb5811ba740 1 -- start start 2024-03-21T11:24:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.795+0000 7fb5811ba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6b050 0x556325c69c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.795+0000 7fb5811ba740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6ba40 0x556325c7c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.795+0000 7fb5811ba740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c70bb0 0x556325c6a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.795+0000 7fb5811ba740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5781ac0e0 con 0x556325c70bb0 2024-03-21T11:24:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.795+0000 7fb5811ba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5781ada60 con 0x556325c6b050 2024-03-21T11:24:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.795+0000 7fb5811ba740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5781ac740 con 0x556325c6ba40 2024-03-21T11:24:23.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb577400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6b050 0x556325c69c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb56fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6ba40 0x556325c7c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb577400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6b050 0x556325c69c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51872/0 (socket says 172.21.15.47:51872) 2024-03-21T11:24:23.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb577400640 1 -- 172.21.15.47:0/3114299903 learned_addr learned my addr 172.21.15.47:0/3114299903 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:23.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb56fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6ba40 0x556325c7c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33118/0 (socket says 172.21.15.47:33118) 2024-03-21T11:24:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb577e00640 1 --2- 172.21.15.47:0/3114299903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c70bb0 0x556325c6a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb56fe00640 1 -- 172.21.15.47:0/3114299903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6b050 msgr2=0x556325c69c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb56fe00640 1 --2- 172.21.15.47:0/3114299903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6b050 0x556325c69c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb56fe00640 1 -- 172.21.15.47:0/3114299903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c70bb0 msgr2=0x556325c6a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb56fe00640 1 --2- 172.21.15.47:0/3114299903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c70bb0 0x556325c6a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb56fe00640 1 -- 172.21.15.47:0/3114299903 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563258a40f0 con 0x556325c6ba40 2024-03-21T11:24:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb577400640 1 --2- 172.21.15.47:0/3114299903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6b050 0x556325c69c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb56fe00640 1 --2- 172.21.15.47:0/3114299903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6ba40 0x556325c7c5d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb5600eee60 tx=0x7fb560004310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=196a21bea7f2d9ef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb567e00640 1 -- 172.21.15.47:0/3114299903 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5600bd5c0 con 0x556325c6ba40 2024-03-21T11:24:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.796+0000 7fb567e00640 1 -- 172.21.15.47:0/3114299903 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5600035f0 con 0x556325c6ba40 2024-03-21T11:24:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.797+0000 7fb567e00640 1 -- 172.21.15.47:0/3114299903 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5600bd3e0 con 0x556325c6ba40 2024-03-21T11:24:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.797+0000 7fb5811ba740 1 -- 172.21.15.47:0/3114299903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6ba40 msgr2=0x556325c7c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.797+0000 7fb5811ba740 1 --2- 172.21.15.47:0/3114299903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6ba40 0x556325c7c5d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb5600eee60 tx=0x7fb560004310 comp rx=0 tx=0).stop 2024-03-21T11:24:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.797+0000 7fb5811ba740 1 -- 172.21.15.47:0/3114299903 shutdown_connections 2024-03-21T11:24:23.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.797+0000 7fb5811ba740 1 --2- 172.21.15.47:0/3114299903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c70bb0 0x556325c6a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.797+0000 7fb5811ba740 1 --2- 172.21.15.47:0/3114299903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6ba40 0x556325c7c5d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.797+0000 7fb5811ba740 1 --2- 172.21.15.47:0/3114299903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6b050 0x556325c69c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.797+0000 7fb5811ba740 1 -- 172.21.15.47:0/3114299903 >> 172.21.15.47:0/3114299903 conn(0x556325c33200 msgr2=0x556325c6cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:23.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.797+0000 7fb5811ba740 1 -- 172.21.15.47:0/3114299903 shutdown_connections 2024-03-21T11:24:23.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.797+0000 7fb5811ba740 1 -- 172.21.15.47:0/3114299903 wait complete. 2024-03-21T11:24:23.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.798+0000 7fb5811ba740 1 -- start start 2024-03-21T11:24:23.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.798+0000 7fb5811ba740 1 Processor -- start 2024-03-21T11:24:23.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.798+0000 7fb5811ba740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6b050 0x556325c7a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.798+0000 7fb5811ba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6ba40 0x556325c7a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.798+0000 7fb5811ba740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c70bb0 0x556325c7b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.798+0000 7fb5811ba740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5781ad400 con 0x556325c70bb0 2024-03-21T11:24:23.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.798+0000 7fb5811ba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5781acda0 con 0x556325c6ba40 2024-03-21T11:24:23.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.799+0000 7fb5811ba740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5781aa2a0 con 0x556325c6b050 2024-03-21T11:24:23.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.799+0000 7fb5811ba740 10 client.0 _ll_register_callbacks cb 0x7fb570009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:23.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.799+0000 7fb577400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6b050 0x556325c7a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.799+0000 7fb577400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6b050 0x556325c7a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33126/0 (socket says 172.21.15.47:33126) 2024-03-21T11:24:23.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.799+0000 7fb577400640 1 -- 172.21.15.47:0/754754089 learned_addr learned my addr 172.21.15.47:0/754754089 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:23.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.799+0000 7fb56fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6ba40 0x556325c7a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.799+0000 7fb577e00640 1 --2- 172.21.15.47:0/754754089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c70bb0 0x556325c7b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.799+0000 7fb577400640 1 -- 172.21.15.47:0/754754089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6ba40 msgr2=0x556325c7a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.799+0000 7fb577400640 1 --2- 172.21.15.47:0/754754089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6ba40 0x556325c7a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.799+0000 7fb577400640 1 -- 172.21.15.47:0/754754089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c70bb0 msgr2=0x556325c7b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.799+0000 7fb577400640 1 --2- 172.21.15.47:0/754754089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c70bb0 0x556325c7b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.799+0000 7fb577400640 1 -- 172.21.15.47:0/754754089 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556325c7bf10 con 0x556325c6b050 2024-03-21T11:24:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.799+0000 7fb577400640 1 --2- 172.21.15.47:0/754754089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6b050 0x556325c7a420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb568009250 tx=0x7fb5680095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e555a95f20858b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.799+0000 7fb565600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.800+0000 7fb565600640 1 -- 172.21.15.47:0/754754089 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56801d040 con 0x556325c6b050 2024-03-21T11:24:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.800+0000 7fb565600640 1 -- 172.21.15.47:0/754754089 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb568009c10 con 0x556325c6b050 2024-03-21T11:24:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.800+0000 7fb5811ba740 1 -- 172.21.15.47:0/754754089 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556325bd7ae0 con 0x556325c6b050 2024-03-21T11:24:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.800+0000 7fb565600640 1 -- 172.21.15.47:0/754754089 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb56803c050 con 0x556325c6b050 2024-03-21T11:24:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.800+0000 7fb5811ba740 10 client.25822 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.800+0000 7fb5811ba740 1 -- 172.21.15.47:0/754754089 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556325cdced0 con 0x556325c6b050 2024-03-21T11:24:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.800+0000 7fb565600640 1 -- 172.21.15.47:0/754754089 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556325bd7ae0 con 0x556325c6b050 2024-03-21T11:24:23.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.800+0000 7fb565600640 1 -- 172.21.15.47:0/754754089 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb568037030 con 0x556325c6b050 2024-03-21T11:24:23.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.800+0000 7fb5811ba740 10 client.25822 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:23.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.800+0000 7fb5811ba740 10 client.25822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:23.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.800+0000 7fb5811ba740 20 client.25822 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:23.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.800+0000 7fb5811ba740 1 --2- 172.21.15.47:0/754754089 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556325cdd730 0x556325cdfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:23.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.800+0000 7fb5811ba740 4 client.25822 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:24:23.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.800+0000 7fb5811ba740 1 -- 172.21.15.47:0/754754089 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556325b553a0 con 0x556325cdd730 2024-03-21T11:24:23.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.801+0000 7fb56fe00640 1 --2- 172.21.15.47:0/754754089 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556325cdd730 0x556325cdfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:23.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.801+0000 7fb56fe00640 1 --2- 172.21.15.47:0/754754089 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556325cdd730 0x556325cdfb50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdb5046077947bc5 server_cookie=71b5800d3effa949 in_seq=0 out_seq=0 2024-03-21T11:24:23.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.801+0000 7fb565600640 10 client.25822 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:23.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.801+0000 7fb565600640 1 -- 172.21.15.47:0/754754089 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556325b553a0 con 0x556325cdd730 2024-03-21T11:24:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.801+0000 7fb565600640 10 client.25822 handle_command_reply: tid=0 2024-03-21T11:24:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.801+0000 7fb5811ba740 1 client.25822 shutdown 2024-03-21T11:24:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.802+0000 7fb5811ba740 1 -- 172.21.15.47:0/754754089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6b050 msgr2=0x556325c7a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.802+0000 7fb5811ba740 1 --2- 172.21.15.47:0/754754089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6b050 0x556325c7a420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb568009250 tx=0x7fb5680095c0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.802+0000 7fb5811ba740 1 -- 172.21.15.47:0/754754089 shutdown_connections 2024-03-21T11:24:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.802+0000 7fb5811ba740 1 --2- 172.21.15.47:0/754754089 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556325cdd730 0x556325cdfb50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.802+0000 7fb5811ba740 1 --2- 172.21.15.47:0/754754089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556325c70bb0 0x556325c7b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.802+0000 7fb5811ba740 1 --2- 172.21.15.47:0/754754089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556325c6ba40 0x556325c7a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.802+0000 7fb5811ba740 1 --2- 172.21.15.47:0/754754089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556325c6b050 0x556325c7a420 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.802+0000 7fb565600640 0 client.25822 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.802+0000 7fb5811ba740 1 -- 172.21.15.47:0/754754089 >> 172.21.15.47:0/754754089 conn(0x556325c33200 msgr2=0x556325c65330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.802+0000 7fb5811ba740 1 -- 172.21.15.47:0/754754089 shutdown_connections 2024-03-21T11:24:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.802+0000 7fb5811ba740 1 -- 172.21.15.47:0/754754089 wait complete. 2024-03-21T11:24:23.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.802+0000 7fb5811ba740 20 client.25822 trim_cache size 0 max 16384 2024-03-21T11:24:23.808 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:24:23.808 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:24:23.808 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:24:23.808 INFO:teuthology.orchestra.run.smithi047.stdout: "60490ad2-80f5-4219-9d1e-022b88eb8cbe": { 2024-03-21T11:24:23.808 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:24:23.808 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "60490ad2-80f5-4219-9d1e-022b88eb8cbe", 2024-03-21T11:24:23.808 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:24:23.808 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:24:23.809 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:24:23.809 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:24:23.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.803+0000 7fb57ce00640 1 -- 172.21.15.47:0/539879073 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb55c0a3030 msgr2=0x7fb55c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.803+0000 7fb57ce00640 1 --2- 172.21.15.47:0/539879073 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb55c0a3030 0x7fb55c0a54f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x556325c6c2a0 tx=0x7fb570009500 comp rx=0 tx=0).stop 2024-03-21T11:24:23.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.803+0000 7fb57ce00640 1 -- 172.21.15.47:0/539879073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb578104990 msgr2=0x7fb578199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:23.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.803+0000 7fb57ce00640 1 --2- 172.21.15.47:0/539879073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb578104990 0x7fb578199fd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb560036040 tx=0x7fb56002c650 comp rx=0 tx=0).stop 2024-03-21T11:24:23.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.803+0000 7fb57ce00640 1 -- 172.21.15.47:0/539879073 shutdown_connections 2024-03-21T11:24:23.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.803+0000 7fb57ce00640 1 --2- 172.21.15.47:0/539879073 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb55c0a3030 0x7fb55c0a54f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.803+0000 7fb57ce00640 1 --2- 172.21.15.47:0/539879073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb578105350 0x7fb5781a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.803+0000 7fb57ce00640 1 --2- 172.21.15.47:0/539879073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb578104990 0x7fb578199fd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.803+0000 7fb57ce00640 1 --2- 172.21.15.47:0/539879073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5781037a0 0x7fb578199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:23.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.803+0000 7fb57ce00640 1 -- 172.21.15.47:0/539879073 >> 172.21.15.47:0/539879073 conn(0x7fb5780fed10 msgr2=0x7fb578107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:23.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.803+0000 7fb57ce00640 1 -- 172.21.15.47:0/539879073 shutdown_connections 2024-03-21T11:24:23.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:23.803+0000 7fb57ce00640 1 -- 172.21.15.47:0/539879073 wait complete. 2024-03-21T11:24:23.823 INFO:tasks.cephfs.filesystem:scrub status for tag:60490ad2-80f5-4219-9d1e-022b88eb8cbe - {'path': '/', 'tag': '60490ad2-80f5-4219-9d1e-022b88eb8cbe', 'options': 'recursive,force'} 2024-03-21T11:24:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:24 smithi102 ceph-mon[36748]: pgmap v432: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-21T11:24:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:24 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:24:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:24 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:24:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:24 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:24:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:24 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:24:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:24 smithi047 ceph-mon[31351]: pgmap v432: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-21T11:24:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:24 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:24:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:24 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:24:25.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:24 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:24:25.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:24 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:24:25.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:24 smithi100 ceph-mon[36245]: pgmap v432: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-21T11:24:25.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:24 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:24:25.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:24 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:24:25.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:24 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:24:25.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:24 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:24:27.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:26 smithi102 ceph-mon[36748]: pgmap v433: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-21T11:24:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:26 smithi047 ceph-mon[31351]: pgmap v433: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-21T11:24:27.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:26 smithi100 ceph-mon[36245]: pgmap v433: 129 pgs: 129 active+clean; 664 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-21T11:24:28.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:27 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:24:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:27 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:24:28.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:27 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:24:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.177+0000 7ffbbcc00640 1 -- 172.21.15.47:0/3892944781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbb8101520 msgr2=0x7ffbb8101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.177+0000 7ffbbcc00640 1 --2- 172.21.15.47:0/3892944781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbb8101520 0x7ffbb8101940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009860 tx=0x7ffba802bb70 comp rx=0 tx=0).stop 2024-03-21T11:24:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.177+0000 7ffbbcc00640 1 -- 172.21.15.47:0/3892944781 shutdown_connections 2024-03-21T11:24:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.177+0000 7ffbbcc00640 1 --2- 172.21.15.47:0/3892944781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb81030d0 0x7ffbb8109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.177+0000 7ffbbcc00640 1 --2- 172.21.15.47:0/3892944781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb8102710 0x7ffbb8102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.177+0000 7ffbbcc00640 1 --2- 172.21.15.47:0/3892944781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbb8101520 0x7ffbb8101940 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.177+0000 7ffbbcc00640 1 -- 172.21.15.47:0/3892944781 >> 172.21.15.47:0/3892944781 conn(0x7ffbb80fcab0 msgr2=0x7ffbb80feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.177+0000 7ffbbcc00640 1 -- 172.21.15.47:0/3892944781 shutdown_connections 2024-03-21T11:24:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.177+0000 7ffbbcc00640 1 -- 172.21.15.47:0/3892944781 wait complete. 2024-03-21T11:24:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.179+0000 7ffbbcc00640 1 Processor -- start 2024-03-21T11:24:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.179+0000 7ffbbcc00640 1 -- start start 2024-03-21T11:24:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.179+0000 7ffbbcc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb8101520 0x7ffbb8197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.179+0000 7ffbbcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8102710 0x7ffbb8197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.179+0000 7ffbbcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbb81030d0 0x7ffbb819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.179+0000 7ffbbcc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb810d180 con 0x7ffbb8101520 2024-03-21T11:24:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.179+0000 7ffbbcc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb810d000 con 0x7ffbb81030d0 2024-03-21T11:24:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.179+0000 7ffbbcc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb810d300 con 0x7ffbb8102710 2024-03-21T11:24:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.179+0000 7ffbb6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8102710 0x7ffbb8197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.179+0000 7ffbb7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb8101520 0x7ffbb8197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.179+0000 7ffbb6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8102710 0x7ffbb8197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53712/0 (socket says 172.21.15.47:53712) 2024-03-21T11:24:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.180+0000 7ffbb7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbb81030d0 0x7ffbb819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.180+0000 7ffbb7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb8101520 0x7ffbb8197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51656/0 (socket says 172.21.15.47:51656) 2024-03-21T11:24:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.180+0000 7ffbb7400640 1 -- 172.21.15.47:0/1004408602 learned_addr learned my addr 172.21.15.47:0/1004408602 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.180+0000 7ffbb7400640 1 -- 172.21.15.47:0/1004408602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8102710 msgr2=0x7ffbb8197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.180+0000 7ffbb7400640 1 --2- 172.21.15.47:0/1004408602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8102710 0x7ffbb8197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.180+0000 7ffbb7400640 1 -- 172.21.15.47:0/1004408602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbb81030d0 msgr2=0x7ffbb819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.180+0000 7ffbb7400640 1 --2- 172.21.15.47:0/1004408602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbb81030d0 0x7ffbb819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.180+0000 7ffbb7400640 1 -- 172.21.15.47:0/1004408602 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb819f470 con 0x7ffbb8101520 2024-03-21T11:24:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.180+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/1004408602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbb81030d0 0x7ffbb819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.180+0000 7ffbb7400640 1 --2- 172.21.15.47:0/1004408602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb8101520 0x7ffbb8197850 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffba80349e0 tx=0x7ffba8032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.181+0000 7ffbafe00640 1 -- 172.21.15.47:0/1004408602 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8041020 con 0x7ffbb8101520 2024-03-21T11:24:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.181+0000 7ffbafe00640 1 -- 172.21.15.47:0/1004408602 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffba802c9f0 con 0x7ffbb8101520 2024-03-21T11:24:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.181+0000 7ffbbcc00640 1 -- 172.21.15.47:0/1004408602 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb819f700 con 0x7ffbb8101520 2024-03-21T11:24:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.181+0000 7ffbafe00640 1 -- 172.21.15.47:0/1004408602 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8039040 con 0x7ffbb8101520 2024-03-21T11:24:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.181+0000 7ffbbcc00640 1 -- 172.21.15.47:0/1004408602 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb819fbf0 con 0x7ffbb8101520 2024-03-21T11:24:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.184+0000 7ffbafe00640 1 -- 172.21.15.47:0/1004408602 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7ffba8044020 con 0x7ffbb8101520 2024-03-21T11:24:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.184+0000 7ffbc0fcc740 1 Processor -- start 2024-03-21T11:24:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.184+0000 7ffbc0fcc740 1 -- start start 2024-03-21T11:24:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbc0fcc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49f3f0 0x55cafb49f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbc0fcc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb49fde0 0x55cafb4a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbc0fcc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb4a4f50 0x55cafb4a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbc0fcc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cafb4623b0 con 0x55cafb4a4f50 2024-03-21T11:24:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbc0fcc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cafb404830 con 0x55cafb49fde0 2024-03-21T11:24:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbc0fcc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cafb3ad5d0 con 0x55cafb49f3f0 2024-03-21T11:24:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbb7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49f3f0 0x55cafb49f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbb7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb49fde0 0x55cafb4a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbb7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49f3f0 0x55cafb49f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53716/0 (socket says 172.21.15.47:53716) 2024-03-21T11:24:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbb7e00640 1 -- 172.21.15.47:0/3016063918 learned_addr learned my addr 172.21.15.47:0/3016063918 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbb6a00640 1 --2- 172.21.15.47:0/3016063918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb4a4f50 0x55cafb4a73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbb7e00640 1 -- 172.21.15.47:0/3016063918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb49fde0 msgr2=0x55cafb4a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/3016063918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb49fde0 0x55cafb4a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbb7e00640 1 -- 172.21.15.47:0/3016063918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb4a4f50 msgr2=0x55cafb4a73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/3016063918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb4a4f50 0x55cafb4a73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbb7e00640 1 -- 172.21.15.47:0/3016063918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cafb3c52c0 con 0x55cafb49f3f0 2024-03-21T11:24:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.185+0000 7ffbb6a00640 1 --2- 172.21.15.47:0/3016063918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb4a4f50 0x55cafb4a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.186+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/3016063918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49f3f0 0x55cafb49f810 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00117e0 tx=0x7ffbb0011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8f79c002c6b2b29 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.186+0000 7ffbafe00640 1 --2- 172.21.15.47:0/1004408602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb9c0a2fa0 0x7ffb9c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.186+0000 7ffba3e00640 1 -- 172.21.15.47:0/3016063918 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0011310 con 0x55cafb49f3f0 2024-03-21T11:24:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.186+0000 7ffba3e00640 1 -- 172.21.15.47:0/3016063918 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffbb0003e50 con 0x55cafb49f3f0 2024-03-21T11:24:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.186+0000 7ffba3e00640 1 -- 172.21.15.47:0/3016063918 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0004560 con 0x55cafb49f3f0 2024-03-21T11:24:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.186+0000 7ffbafe00640 1 -- 172.21.15.47:0/1004408602 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffba80f0ff0 con 0x7ffbb8101520 2024-03-21T11:24:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.186+0000 7ffbb7400640 1 --2- 172.21.15.47:0/1004408602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb9c0a2fa0 0x7ffb9c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.187+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/3016063918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49f3f0 msgr2=0x55cafb49f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.187+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/3016063918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49f3f0 0x55cafb49f810 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00117e0 tx=0x7ffbb0011ca0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.187+0000 7ffbb7400640 1 --2- 172.21.15.47:0/1004408602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb9c0a2fa0 0x7ffb9c0a5460 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55cafb4a01e0 tx=0x7ffba80f4ae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.187+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/3016063918 shutdown_connections 2024-03-21T11:24:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.187+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/3016063918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb4a4f50 0x55cafb4a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.187+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/3016063918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb49fde0 0x55cafb4a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.187+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/3016063918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49f3f0 0x55cafb49f810 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.187+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/3016063918 >> 172.21.15.47:0/3016063918 conn(0x55cafb49a960 msgr2=0x55cafb49ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.187+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/3016063918 shutdown_connections 2024-03-21T11:24:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.187+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/3016063918 wait complete. 2024-03-21T11:24:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.188+0000 7ffbc0fcc740 1 -- start start 2024-03-21T11:24:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.188+0000 7ffbc0fcc740 1 Processor -- start 2024-03-21T11:24:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.188+0000 7ffbc0fcc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb49f3f0 0x55cafb4a9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.188+0000 7ffbc0fcc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49fde0 0x55cafb4a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.188+0000 7ffbc0fcc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb4a4f50 0x55cafb4b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.188+0000 7ffbc0fcc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cafb3c0a20 con 0x55cafb49f3f0 2024-03-21T11:24:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.188+0000 7ffbc0fcc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cafb4008d0 con 0x55cafb4a4f50 2024-03-21T11:24:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.188+0000 7ffbc0fcc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cafb4a7f00 con 0x55cafb49fde0 2024-03-21T11:24:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.188+0000 7ffbc0fcc740 10 client.0 _ll_register_callbacks cb 0x55cafb3fd1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.189+0000 7ffbb6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49fde0 0x55cafb4a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.189+0000 7ffbb6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49fde0 0x55cafb4a9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53720/0 (socket says 172.21.15.47:53720) 2024-03-21T11:24:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.189+0000 7ffbb6a00640 1 -- 172.21.15.47:0/401846690 learned_addr learned my addr 172.21.15.47:0/401846690 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.189+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/401846690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb49f3f0 0x55cafb4a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.189+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/401846690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb4a4f50 0x55cafb4b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.189+0000 7ffbb6a00640 1 -- 172.21.15.47:0/401846690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb4a4f50 msgr2=0x55cafb4b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.189+0000 7ffbb6a00640 1 --2- 172.21.15.47:0/401846690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb4a4f50 0x55cafb4b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.189+0000 7ffbb6a00640 1 -- 172.21.15.47:0/401846690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb49f3f0 msgr2=0x55cafb4a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.189+0000 7ffbb6a00640 1 --2- 172.21.15.47:0/401846690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb49f3f0 0x55cafb4a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.189+0000 7ffbb6a00640 1 -- 172.21.15.47:0/401846690 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cafb119210 con 0x55cafb49fde0 2024-03-21T11:24:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.189+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/401846690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb4a4f50 0x55cafb4b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.189+0000 7ffbb6a00640 1 --2- 172.21.15.47:0/401846690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49fde0 0x55cafb4a9ba0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffba40116f0 tx=0x7ffba4011bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48ae6e6544eae958 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.189+0000 7ffba0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.189+0000 7ffba0c00640 1 -- 172.21.15.47:0/401846690 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba400fb50 con 0x55cafb49fde0 2024-03-21T11:24:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.190+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/401846690 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffbb0003e50 con 0x55cafb49fde0 2024-03-21T11:24:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.190+0000 7ffba0c00640 1 -- 172.21.15.47:0/401846690 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffba400fcf0 con 0x55cafb49fde0 2024-03-21T11:24:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.190+0000 7ffba0c00640 1 -- 172.21.15.47:0/401846690 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffba4027050 con 0x55cafb49fde0 2024-03-21T11:24:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.190+0000 7ffbc0fcc740 10 client.25834 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.190+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/401846690 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cafb1e5620 con 0x55cafb49fde0 2024-03-21T11:24:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.190+0000 7ffba0c00640 1 -- 172.21.15.47:0/401846690 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffbb0003e50 con 0x55cafb49fde0 2024-03-21T11:24:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.190+0000 7ffba0c00640 1 -- 172.21.15.47:0/401846690 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffba4022030 con 0x55cafb49fde0 2024-03-21T11:24:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.190+0000 7ffbc0fcc740 10 client.25834 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.190+0000 7ffbc0fcc740 10 client.25834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.190+0000 7ffbc0fcc740 20 client.25834 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.190+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/401846690 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cafb4b55a0 0x55cafb4b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.190+0000 7ffbc0fcc740 4 client.25834 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.190+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/401846690 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cafb411d70 con 0x55cafb4b55a0 2024-03-21T11:24:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.191+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/401846690 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cafb4b55a0 0x55cafb4b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.191+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/401846690 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cafb4b55a0 0x55cafb4b79c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb204048783b71c6 server_cookie=363f98a4b97f49a4 in_seq=0 out_seq=0 2024-03-21T11:24:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.191+0000 7ffba0c00640 10 client.25834 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffba0c00640 1 -- 172.21.15.47:0/401846690 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cafb411d70 con 0x55cafb4b55a0 2024-03-21T11:24:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffba0c00640 10 client.25834 handle_command_reply: tid=0 2024-03-21T11:24:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffbc0fcc740 1 client.25834 shutdown 2024-03-21T11:24:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/401846690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49fde0 msgr2=0x55cafb4a9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/401846690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49fde0 0x55cafb4a9ba0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffba40116f0 tx=0x7ffba4011bb0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/401846690 shutdown_connections 2024-03-21T11:24:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/401846690 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cafb4b55a0 0x55cafb4b79c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/401846690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb4a4f50 0x55cafb4b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/401846690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49fde0 0x55cafb4a9ba0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/401846690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb49f3f0 0x55cafb4a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/401846690 >> 172.21.15.47:0/401846690 conn(0x55cafb49a960 msgr2=0x55cafb499b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffba0c00640 0 client.25834 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/401846690 shutdown_connections 2024-03-21T11:24:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/401846690 wait complete. 2024-03-21T11:24:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.192+0000 7ffbc0fcc740 20 client.25834 trim_cache size 0 max 16384 2024-03-21T11:24:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.207+0000 7ffbc0fcc740 1 Processor -- start 2024-03-21T11:24:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.207+0000 7ffbc0fcc740 1 -- start start 2024-03-21T11:24:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbc0fcc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb49f3f0 0x55cafb4a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbc0fcc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb49fde0 0x55cafb4b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbc0fcc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb4a4f50 0x55cafb49e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbc0fcc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb81a9fa0 con 0x55cafb49fde0 2024-03-21T11:24:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbc0fcc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb81ab920 con 0x55cafb49f3f0 2024-03-21T11:24:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbc0fcc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb81aa600 con 0x55cafb4a4f50 2024-03-21T11:24:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbb7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb49f3f0 0x55cafb4a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbb6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb49fde0 0x55cafb4b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbb7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb49f3f0 0x55cafb4a8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55174/0 (socket says 172.21.15.47:55174) 2024-03-21T11:24:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbb7e00640 1 -- 172.21.15.47:0/2393165267 learned_addr learned my addr 172.21.15.47:0/2393165267 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/2393165267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb4a4f50 0x55cafb49e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbb6a00640 1 -- 172.21.15.47:0/2393165267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb4a4f50 msgr2=0x55cafb49e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbb6a00640 1 --2- 172.21.15.47:0/2393165267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb4a4f50 0x55cafb49e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbb6a00640 1 -- 172.21.15.47:0/2393165267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb49f3f0 msgr2=0x55cafb4a8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbb6a00640 1 --2- 172.21.15.47:0/2393165267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb49f3f0 0x55cafb4a8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbb6a00640 1 -- 172.21.15.47:0/2393165267 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cafb4a0bf0 con 0x55cafb49fde0 2024-03-21T11:24:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.208+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/2393165267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb49f3f0 0x55cafb4a8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.209+0000 7ffbb6a00640 1 --2- 172.21.15.47:0/2393165267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb49fde0 0x55cafb4b0ba0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffba40195a0 tx=0x7ffba4009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6d6917146a87d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.209+0000 7ffba3400640 1 -- 172.21.15.47:0/2393165267 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4019390 con 0x55cafb49fde0 2024-03-21T11:24:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.209+0000 7ffba3400640 1 -- 172.21.15.47:0/2393165267 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffba4019790 con 0x55cafb49fde0 2024-03-21T11:24:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.209+0000 7ffba3400640 1 -- 172.21.15.47:0/2393165267 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba400fcc0 con 0x55cafb49fde0 2024-03-21T11:24:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.210+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/2393165267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb49fde0 msgr2=0x55cafb4b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.210+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/2393165267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb49fde0 0x55cafb4b0ba0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffba40195a0 tx=0x7ffba4009e50 comp rx=0 tx=0).stop 2024-03-21T11:24:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.210+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/2393165267 shutdown_connections 2024-03-21T11:24:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.210+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/2393165267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb4a4f50 0x55cafb49e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.210+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/2393165267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb49fde0 0x55cafb4b0ba0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.210+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/2393165267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb49f3f0 0x55cafb4a8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.210+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/2393165267 >> 172.21.15.47:0/2393165267 conn(0x55cafb49a960 msgr2=0x55cafb49dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.210+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/2393165267 shutdown_connections 2024-03-21T11:24:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.210+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/2393165267 wait complete. 2024-03-21T11:24:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.210+0000 7ffbc0fcc740 1 -- start start 2024-03-21T11:24:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.210+0000 7ffbc0fcc740 1 Processor -- start 2024-03-21T11:24:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbc0fcc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49f3f0 0x55cafb4aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbc0fcc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb4a4f50 0x55cafb4af010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbc0fcc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb4b0020 0x55cafb4b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbc0fcc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb81ab2c0 con 0x55cafb4a4f50 2024-03-21T11:24:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbc0fcc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb81aac60 con 0x55cafb4b0020 2024-03-21T11:24:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbc0fcc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb81a8160 con 0x55cafb49f3f0 2024-03-21T11:24:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbc0fcc740 10 client.0 _ll_register_callbacks cb 0x7ffba4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbb7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49f3f0 0x55cafb4aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbb7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49f3f0 0x55cafb4aead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53738/0 (socket says 172.21.15.47:53738) 2024-03-21T11:24:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbb7e00640 1 -- 172.21.15.47:0/174900736 learned_addr learned my addr 172.21.15.47:0/174900736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbb6a00640 1 --2- 172.21.15.47:0/174900736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb4a4f50 0x55cafb4af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/174900736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb4b0020 0x55cafb4b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbb7e00640 1 -- 172.21.15.47:0/174900736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb4b0020 msgr2=0x55cafb4b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/174900736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb4b0020 0x55cafb4b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbb7e00640 1 -- 172.21.15.47:0/174900736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb4a4f50 msgr2=0x55cafb4af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.211+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/174900736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb4a4f50 0x55cafb4af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.212+0000 7ffbb7e00640 1 -- 172.21.15.47:0/174900736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cafb4b09e0 con 0x55cafb49f3f0 2024-03-21T11:24:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.212+0000 7ffbb7e00640 1 --2- 172.21.15.47:0/174900736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49f3f0 0x55cafb4aead0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00117b0 tx=0x7ffbb001c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18b52309554a583e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.212+0000 7ffba0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.212+0000 7ffba0c00640 1 -- 172.21.15.47:0/174900736 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0023040 con 0x55cafb49f3f0 2024-03-21T11:24:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.212+0000 7ffba0c00640 1 -- 172.21.15.47:0/174900736 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffbb0011350 con 0x55cafb49f3f0 2024-03-21T11:24:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.212+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/174900736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffba4019790 con 0x55cafb49f3f0 2024-03-21T11:24:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.212+0000 7ffba0c00640 1 -- 172.21.15.47:0/174900736 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbb000f9a0 con 0x55cafb49f3f0 2024-03-21T11:24:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.212+0000 7ffbc0fcc740 10 client.25840 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.212+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/174900736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cafb51a270 con 0x55cafb49f3f0 2024-03-21T11:24:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.212+0000 7ffba0c00640 1 -- 172.21.15.47:0/174900736 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffba4019790 con 0x55cafb49f3f0 2024-03-21T11:24:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.213+0000 7ffba0c00640 1 -- 172.21.15.47:0/174900736 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffbb0018020 con 0x55cafb49f3f0 2024-03-21T11:24:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.213+0000 7ffbc0fcc740 10 client.25840 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.213+0000 7ffbc0fcc740 10 client.25840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.213+0000 7ffbc0fcc740 20 client.25840 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.213+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/174900736 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cafb51aa40 0x55cafb51ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.213+0000 7ffbc0fcc740 4 client.25840 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:24:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.213+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/174900736 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cafb40c0c0 con 0x55cafb51aa40 2024-03-21T11:24:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.213+0000 7ffbb6a00640 1 --2- 172.21.15.47:0/174900736 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cafb51aa40 0x55cafb51ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.213+0000 7ffbb6a00640 1 --2- 172.21.15.47:0/174900736 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cafb51aa40 0x55cafb51ce60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61394140672c7eb server_cookie=2f1709816686b789 in_seq=0 out_seq=0 2024-03-21T11:24:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.213+0000 7ffba0c00640 10 client.25840 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.220+0000 7ffba0c00640 1 -- 172.21.15.47:0/174900736 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cafb40c0c0 con 0x55cafb51aa40 2024-03-21T11:24:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.220+0000 7ffba0c00640 10 client.25840 handle_command_reply: tid=0 2024-03-21T11:24:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.220+0000 7ffbc0fcc740 1 client.25840 shutdown 2024-03-21T11:24:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.220+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/174900736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49f3f0 msgr2=0x55cafb4aead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.220+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/174900736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49f3f0 0x55cafb4aead0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00117b0 tx=0x7ffbb001c040 comp rx=0 tx=0).stop 2024-03-21T11:24:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.220+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/174900736 shutdown_connections 2024-03-21T11:24:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.220+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/174900736 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cafb51aa40 0x55cafb51ce60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.220+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/174900736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cafb4b0020 0x55cafb4b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.220+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/174900736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cafb4a4f50 0x55cafb4af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.220+0000 7ffbc0fcc740 1 --2- 172.21.15.47:0/174900736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cafb49f3f0 0x55cafb4aead0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.220+0000 7ffba0c00640 0 client.25840 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.220+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/174900736 >> 172.21.15.47:0/174900736 conn(0x55cafb49a960 msgr2=0x55cafb4a8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.221+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/174900736 shutdown_connections 2024-03-21T11:24:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.221+0000 7ffbc0fcc740 1 -- 172.21.15.47:0/174900736 wait complete. 2024-03-21T11:24:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.221+0000 7ffbc0fcc740 20 client.25840 trim_cache size 0 max 16384 2024-03-21T11:24:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:24:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:24:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:24:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:24:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.221+0000 7ffbbcc00640 1 -- 172.21.15.47:0/1004408602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb9c0a2fa0 msgr2=0x7ffb9c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.221+0000 7ffbbcc00640 1 --2- 172.21.15.47:0/1004408602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb9c0a2fa0 0x7ffb9c0a5460 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55cafb4a01e0 tx=0x7ffba80f4ae0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.221+0000 7ffbbcc00640 1 -- 172.21.15.47:0/1004408602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb8101520 msgr2=0x7ffbb8197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.221+0000 7ffbbcc00640 1 --2- 172.21.15.47:0/1004408602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb8101520 0x7ffbb8197850 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffba80349e0 tx=0x7ffba8032430 comp rx=0 tx=0).stop 2024-03-21T11:24:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.222+0000 7ffbbcc00640 1 -- 172.21.15.47:0/1004408602 shutdown_connections 2024-03-21T11:24:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.222+0000 7ffbbcc00640 1 --2- 172.21.15.47:0/1004408602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb9c0a2fa0 0x7ffb9c0a5460 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.222+0000 7ffbbcc00640 1 --2- 172.21.15.47:0/1004408602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbb81030d0 0x7ffbb819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.222+0000 7ffbbcc00640 1 --2- 172.21.15.47:0/1004408602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffbb8102710 0x7ffbb8197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.222+0000 7ffbbcc00640 1 --2- 172.21.15.47:0/1004408602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbb8101520 0x7ffbb8197850 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.222+0000 7ffbbcc00640 1 -- 172.21.15.47:0/1004408602 >> 172.21.15.47:0/1004408602 conn(0x7ffbb80fcab0 msgr2=0x7ffbb8105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.222+0000 7ffbbcc00640 1 -- 172.21.15.47:0/1004408602 shutdown_connections 2024-03-21T11:24:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:29.222+0000 7ffbbcc00640 1 -- 172.21.15.47:0/1004408602 wait complete. 2024-03-21T11:24:29.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:28 smithi102 ceph-mon[36748]: pgmap v434: 129 pgs: 129 active+clean; 674 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-21T11:24:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:28 smithi047 ceph-mon[31351]: pgmap v434: 129 pgs: 129 active+clean; 674 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-21T11:24:29.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:28 smithi100 ceph-mon[36245]: pgmap v434: 129 pgs: 129 active+clean; 674 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-21T11:24:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:30 smithi102 ceph-mon[36748]: pgmap v435: 129 pgs: 129 active+clean; 674 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-21T11:24:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:30 smithi047 ceph-mon[31351]: pgmap v435: 129 pgs: 129 active+clean; 674 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-21T11:24:31.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:30 smithi100 ceph-mon[36245]: pgmap v435: 129 pgs: 129 active+clean; 674 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-21T11:24:32.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:32 smithi047 ceph-mon[31351]: pgmap v436: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-21T11:24:32.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:32 smithi100 ceph-mon[36245]: pgmap v436: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-21T11:24:32.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:32 smithi102 ceph-mon[36748]: pgmap v436: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-21T11:24:34.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:34 smithi100 ceph-mon[36245]: pgmap v437: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-21T11:24:34.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:34 smithi102 ceph-mon[36748]: pgmap v437: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-21T11:24:34.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:34 smithi047 ceph-mon[31351]: pgmap v437: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-21T11:24:36.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:36 smithi102 ceph-mon[36748]: pgmap v438: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-21T11:24:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:36 smithi047 ceph-mon[31351]: pgmap v438: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-21T11:24:36.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:36 smithi100 ceph-mon[36245]: pgmap v438: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-21T11:24:38.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:38 smithi102 ceph-mon[36748]: pgmap v439: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-21T11:24:38.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:38 smithi047 ceph-mon[31351]: pgmap v439: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-21T11:24:38.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:38 smithi100 ceph-mon[36245]: pgmap v439: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-03-21T11:24:40.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:40 smithi102 ceph-mon[36748]: pgmap v440: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-21T11:24:40.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:40 smithi047 ceph-mon[31351]: pgmap v440: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-21T11:24:40.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:40 smithi100 ceph-mon[36245]: pgmap v440: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-21T11:24:41.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:41 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:24:41.544 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:41 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:24:41.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:41 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:24:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.376+0000 7fd604e00640 1 -- 172.21.15.47:0/1772640252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd600075680 msgr2=0x7fd600073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.376+0000 7fd604e00640 1 --2- 172.21.15.47:0/1772640252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd600075680 0x7fd600073ad0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009860 tx=0x7fd5f002bc00 comp rx=0 tx=0).stop 2024-03-21T11:24:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.376+0000 7fd604e00640 1 -- 172.21.15.47:0/1772640252 shutdown_connections 2024-03-21T11:24:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.376+0000 7fd604e00640 1 --2- 172.21.15.47:0/1772640252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6001093b0 0x7fd60010b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.376+0000 7fd604e00640 1 --2- 172.21.15.47:0/1772640252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6000740a0 0x7fd600074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.376+0000 7fd604e00640 1 --2- 172.21.15.47:0/1772640252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd600075680 0x7fd600073ad0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.376+0000 7fd604e00640 1 -- 172.21.15.47:0/1772640252 >> 172.21.15.47:0/1772640252 conn(0x7fd6000fcab0 msgr2=0x7fd6000feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.376+0000 7fd604e00640 1 -- 172.21.15.47:0/1772640252 shutdown_connections 2024-03-21T11:24:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.376+0000 7fd604e00640 1 -- 172.21.15.47:0/1772640252 wait complete. 2024-03-21T11:24:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.377+0000 7fd604e00640 1 Processor -- start 2024-03-21T11:24:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.378+0000 7fd604e00640 1 -- start start 2024-03-21T11:24:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.378+0000 7fd604e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6000740a0 0x7fd600199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.378+0000 7fd604e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd600075680 0x7fd600199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.378+0000 7fd604e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6001093b0 0x7fd6001a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.378+0000 7fd604e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60010f370 con 0x7fd600075680 2024-03-21T11:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.378+0000 7fd604e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60010f1f0 con 0x7fd6000740a0 2024-03-21T11:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.378+0000 7fd604e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60010f4f0 con 0x7fd6001093b0 2024-03-21T11:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.378+0000 7fd5ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6000740a0 0x7fd600199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.378+0000 7fd5ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6000740a0 0x7fd600199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53972/0 (socket says 172.21.15.47:53972) 2024-03-21T11:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.378+0000 7fd5ff400640 1 -- 172.21.15.47:0/3037111092 learned_addr learned my addr 172.21.15.47:0/3037111092 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.378+0000 7fd5fea00640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd600075680 0x7fd600199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.379+0000 7fd5ffe00640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6001093b0 0x7fd6001a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.379+0000 7fd5ff400640 1 -- 172.21.15.47:0/3037111092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6001093b0 msgr2=0x7fd6001a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.379+0000 7fd5ff400640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6001093b0 0x7fd6001a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.379+0000 7fd5ff400640 1 -- 172.21.15.47:0/3037111092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd600075680 msgr2=0x7fd600199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.379+0000 7fd5ff400640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd600075680 0x7fd600199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.379+0000 7fd5ff400640 1 -- 172.21.15.47:0/3037111092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6001a1600 con 0x7fd6000740a0 2024-03-21T11:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.379+0000 7fd5fea00640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd600075680 0x7fd600199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.379+0000 7fd5ff400640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6000740a0 0x7fd600199a30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0005940 tx=0x7fd5f00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.380+0000 7fd5f7e00640 1 -- 172.21.15.47:0/3037111092 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0041020 con 0x7fd6000740a0 2024-03-21T11:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.380+0000 7fd5f7e00640 1 -- 172.21.15.47:0/3037111092 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5f00035f0 con 0x7fd6000740a0 2024-03-21T11:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.380+0000 7fd5f7e00640 1 -- 172.21.15.47:0/3037111092 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0039040 con 0x7fd6000740a0 2024-03-21T11:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.380+0000 7fd604e00640 1 -- 172.21.15.47:0/3037111092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6001a1890 con 0x7fd6000740a0 2024-03-21T11:24:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.380+0000 7fd604e00640 1 -- 172.21.15.47:0/3037111092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6001a1d30 con 0x7fd6000740a0 2024-03-21T11:24:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.384+0000 7fd6091b7740 1 Processor -- start 2024-03-21T11:24:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.384+0000 7fd6091b7740 1 -- start start 2024-03-21T11:24:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.384+0000 7fd6091b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174d3f0 0x564ff174d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.384+0000 7fd6091b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff174dde0 0x564ff17528e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.384+0000 7fd6091b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff1752f50 0x564ff17553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.384+0000 7fd6091b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ff17103b0 con 0x564ff174dde0 2024-03-21T11:24:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.385+0000 7fd6091b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ff16b2830 con 0x564ff174d3f0 2024-03-21T11:24:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.385+0000 7fd6091b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ff165b5d0 con 0x564ff1752f50 2024-03-21T11:24:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.385+0000 7fd5ffe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174d3f0 0x564ff174d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.385+0000 7fd5ffe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174d3f0 0x564ff174d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53984/0 (socket says 172.21.15.47:53984) 2024-03-21T11:24:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.385+0000 7fd5ffe00640 1 -- 172.21.15.47:0/3712636386 learned_addr learned my addr 172.21.15.47:0/3712636386 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.385+0000 7fd5fea00640 1 --2- 172.21.15.47:0/3712636386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff1752f50 0x564ff17553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.385+0000 7fd5ffe00640 1 -- 172.21.15.47:0/3712636386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff1752f50 msgr2=0x564ff17553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.385+0000 7fd5ffe00640 1 --2- 172.21.15.47:0/3712636386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff1752f50 0x564ff17553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.385+0000 7fd5ffe00640 1 -- 172.21.15.47:0/3712636386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff174dde0 msgr2=0x564ff17528e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:24:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.385+0000 7fd5ffe00640 1 --2- 172.21.15.47:0/3712636386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff174dde0 0x564ff17528e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.385+0000 7fd5ffe00640 1 -- 172.21.15.47:0/3712636386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ff16732c0 con 0x564ff174d3f0 2024-03-21T11:24:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.385+0000 7fd5ffe00640 1 --2- 172.21.15.47:0/3712636386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174d3f0 0x564ff174d810 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009fd0 tx=0x7fd5f80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a4dce5a6b185f7b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.386+0000 7fd5f4c00640 1 -- 172.21.15.47:0/3712636386 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8004d20 con 0x564ff174d3f0 2024-03-21T11:24:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.386+0000 7fd5f4c00640 1 -- 172.21.15.47:0/3712636386 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5f8004050 con 0x564ff174d3f0 2024-03-21T11:24:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.386+0000 7fd5f4c00640 1 -- 172.21.15.47:0/3712636386 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8005260 con 0x564ff174d3f0 2024-03-21T11:24:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.386+0000 7fd5f7e00640 1 -- 172.21.15.47:0/3037111092 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fd5f0044020 con 0x7fd6000740a0 2024-03-21T11:24:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.386+0000 7fd6091b7740 1 -- 172.21.15.47:0/3712636386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174d3f0 msgr2=0x564ff174d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.386+0000 7fd6091b7740 1 --2- 172.21.15.47:0/3712636386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174d3f0 0x564ff174d810 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009fd0 tx=0x7fd5f80118d0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.387+0000 7fd6091b7740 1 -- 172.21.15.47:0/3712636386 shutdown_connections 2024-03-21T11:24:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.387+0000 7fd6091b7740 1 --2- 172.21.15.47:0/3712636386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff1752f50 0x564ff17553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.387+0000 7fd6091b7740 1 --2- 172.21.15.47:0/3712636386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff174dde0 0x564ff17528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.387+0000 7fd6091b7740 1 --2- 172.21.15.47:0/3712636386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174d3f0 0x564ff174d810 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.387+0000 7fd6091b7740 1 -- 172.21.15.47:0/3712636386 >> 172.21.15.47:0/3712636386 conn(0x564ff1748960 msgr2=0x564ff1748d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.387+0000 7fd6091b7740 1 -- 172.21.15.47:0/3712636386 shutdown_connections 2024-03-21T11:24:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.387+0000 7fd6091b7740 1 -- 172.21.15.47:0/3712636386 wait complete. 2024-03-21T11:24:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.388+0000 7fd6091b7740 1 -- start start 2024-03-21T11:24:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.388+0000 7fd6091b7740 1 Processor -- start 2024-03-21T11:24:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.388+0000 7fd5f7e00640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5e40a2eb0 0x7fd5e40a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.388+0000 7fd5f7e00640 1 -- 172.21.15.47:0/3037111092 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd5f00f1100 con 0x7fd6000740a0 2024-03-21T11:24:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd6091b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff174d3f0 0x564ff1757600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd6091b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174dde0 0x564ff1757b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd6091b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff1752f50 0x564ff175eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd6091b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ff166ea20 con 0x564ff1752f50 2024-03-21T11:24:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd6091b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ff16ae8d0 con 0x564ff174dde0 2024-03-21T11:24:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd6091b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ff1755f00 con 0x564ff174d3f0 2024-03-21T11:24:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd6091b7740 10 client.0 _ll_register_callbacks cb 0x564ff16ab1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd5ffe00640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5e40a2eb0 0x7fd5e40a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd5ff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff174d3f0 0x564ff1757600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd5ff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff174d3f0 0x564ff1757600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53504/0 (socket says 172.21.15.47:53504) 2024-03-21T11:24:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd5ff400640 1 -- 172.21.15.47:0/745813540 learned_addr learned my addr 172.21.15.47:0/745813540 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd5fea00640 1 --2- 172.21.15.47:0/745813540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174dde0 0x564ff1757b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd5ffe00640 1 --2- 172.21.15.47:0/745813540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff1752f50 0x564ff175eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd5ff400640 1 -- 172.21.15.47:0/745813540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174dde0 msgr2=0x564ff1757b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd5ff400640 1 --2- 172.21.15.47:0/745813540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174dde0 0x564ff1757b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.389+0000 7fd5ff400640 1 -- 172.21.15.47:0/745813540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff1752f50 msgr2=0x564ff175eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.390+0000 7fd5ff400640 1 --2- 172.21.15.47:0/745813540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff1752f50 0x564ff175eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.390+0000 7fd5ff400640 1 -- 172.21.15.47:0/745813540 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ff13c7210 con 0x564ff174d3f0 2024-03-21T11:24:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.390+0000 7fd5ffe00640 1 --2- 172.21.15.47:0/745813540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff1752f50 0x564ff175eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.390+0000 7fd5ff400640 1 --2- 172.21.15.47:0/745813540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff174d3f0 0x564ff1757600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000b230 tx=0x7fd5f00bcd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e05bc981d2a93c34 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.390+0000 7fd5e9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.390+0000 7fd5e9600640 1 -- 172.21.15.47:0/745813540 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f00bd3f0 con 0x564ff174d3f0 2024-03-21T11:24:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.390+0000 7fd5e9600640 1 -- 172.21.15.47:0/745813540 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd5f00c1120 con 0x564ff174d3f0 2024-03-21T11:24:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.391+0000 7fd6091b7740 1 -- 172.21.15.47:0/745813540 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5f8004050 con 0x564ff174d3f0 2024-03-21T11:24:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.391+0000 7fd5ffe00640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5e40a2eb0 0x7fd5e40a5370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd5f801b040 tx=0x7fd5f8000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.391+0000 7fd5e9600640 1 -- 172.21.15.47:0/745813540 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5f00c1540 con 0x564ff174d3f0 2024-03-21T11:24:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.391+0000 7fd6091b7740 10 client.25852 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.391+0000 7fd6091b7740 1 -- 172.21.15.47:0/745813540 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564ff1493620 con 0x564ff174d3f0 2024-03-21T11:24:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.391+0000 7fd5e9600640 1 -- 172.21.15.47:0/745813540 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5f8004050 con 0x564ff174d3f0 2024-03-21T11:24:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.391+0000 7fd5e9600640 1 -- 172.21.15.47:0/745813540 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd5f003c030 con 0x564ff174d3f0 2024-03-21T11:24:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.391+0000 7fd6091b7740 10 client.25852 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.391+0000 7fd6091b7740 10 client.25852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.392+0000 7fd6091b7740 20 client.25852 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.392+0000 7fd6091b7740 1 --2- 172.21.15.47:0/745813540 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ff1763160 0x564ff17655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.392+0000 7fd6091b7740 4 client.25852 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.392+0000 7fd6091b7740 1 -- 172.21.15.47:0/745813540 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ff16bfd70 con 0x564ff1763160 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.392+0000 7fd5fea00640 1 --2- 172.21.15.47:0/745813540 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ff1763160 0x564ff17655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.392+0000 7fd5fea00640 1 --2- 172.21.15.47:0/745813540 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ff1763160 0x564ff17655a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a9d9cf5f8e1f01f server_cookie=c4c91740c90a147c in_seq=0 out_seq=0 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.393+0000 7fd5e9600640 10 client.25852 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.393+0000 7fd5e9600640 1 -- 172.21.15.47:0/745813540 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564ff16bfd70 con 0x564ff1763160 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.393+0000 7fd5e9600640 10 client.25852 handle_command_reply: tid=0 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.393+0000 7fd6091b7740 1 client.25852 shutdown 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.394+0000 7fd6091b7740 1 -- 172.21.15.47:0/745813540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff174d3f0 msgr2=0x564ff1757600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.394+0000 7fd6091b7740 1 --2- 172.21.15.47:0/745813540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff174d3f0 0x564ff1757600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000b230 tx=0x7fd5f00bcd50 comp rx=0 tx=0).stop 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.394+0000 7fd6091b7740 1 -- 172.21.15.47:0/745813540 shutdown_connections 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.394+0000 7fd6091b7740 1 --2- 172.21.15.47:0/745813540 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ff1763160 0x564ff17655a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.394+0000 7fd6091b7740 1 --2- 172.21.15.47:0/745813540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff1752f50 0x564ff175eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.394+0000 7fd6091b7740 1 --2- 172.21.15.47:0/745813540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174dde0 0x564ff1757b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.394+0000 7fd6091b7740 1 --2- 172.21.15.47:0/745813540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff174d3f0 0x564ff1757600 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.394+0000 7fd6091b7740 1 -- 172.21.15.47:0/745813540 >> 172.21.15.47:0/745813540 conn(0x564ff1748960 msgr2=0x564ff1747b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.394+0000 7fd5e9600640 0 client.25852 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.394+0000 7fd6091b7740 1 -- 172.21.15.47:0/745813540 shutdown_connections 2024-03-21T11:24:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.394+0000 7fd6091b7740 1 -- 172.21.15.47:0/745813540 wait complete. 2024-03-21T11:24:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.394+0000 7fd6091b7740 20 client.25852 trim_cache size 0 max 16384 2024-03-21T11:24:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.410+0000 7fd6091b7740 1 Processor -- start 2024-03-21T11:24:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.410+0000 7fd6091b7740 1 -- start start 2024-03-21T11:24:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.410+0000 7fd6091b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff174d3f0 0x564ff174bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.410+0000 7fd6091b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff174dde0 0x564ff175eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.410+0000 7fd6091b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff1752f50 0x564ff174c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.410+0000 7fd6091b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6001ac170 con 0x564ff174d3f0 2024-03-21T11:24:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.410+0000 7fd6091b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6001adaf0 con 0x564ff1752f50 2024-03-21T11:24:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.410+0000 7fd6091b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6001ac7d0 con 0x564ff174dde0 2024-03-21T11:24:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.410+0000 7fd5ffe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff1752f50 0x564ff174c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.410+0000 7fd5ffe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff1752f50 0x564ff174c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54014/0 (socket says 172.21.15.47:54014) 2024-03-21T11:24:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.410+0000 7fd5fea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff174dde0 0x564ff175eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.410+0000 7fd5ffe00640 1 -- 172.21.15.47:0/1455795615 learned_addr learned my addr 172.21.15.47:0/1455795615 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.410+0000 7fd5ff400640 1 --2- 172.21.15.47:0/1455795615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff174d3f0 0x564ff174bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.411+0000 7fd5ffe00640 1 -- 172.21.15.47:0/1455795615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff174dde0 msgr2=0x564ff175eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.411+0000 7fd5ffe00640 1 --2- 172.21.15.47:0/1455795615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff174dde0 0x564ff175eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.411+0000 7fd5ffe00640 1 -- 172.21.15.47:0/1455795615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff174d3f0 msgr2=0x564ff174bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.411+0000 7fd5ffe00640 1 --2- 172.21.15.47:0/1455795615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff174d3f0 0x564ff174bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.411+0000 7fd5ffe00640 1 -- 172.21.15.47:0/1455795615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ff1493620 con 0x564ff1752f50 2024-03-21T11:24:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.411+0000 7fd5fea00640 1 --2- 172.21.15.47:0/1455795615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff174dde0 0x564ff175eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.411+0000 7fd5ffe00640 1 --2- 172.21.15.47:0/1455795615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff1752f50 0x564ff174c100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x564ff174e640 tx=0x7fd5f8024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e828df780630094 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.411+0000 7fd5ebe00640 1 -- 172.21.15.47:0/1455795615 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8004c70 con 0x564ff1752f50 2024-03-21T11:24:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.412+0000 7fd5ebe00640 1 -- 172.21.15.47:0/1455795615 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5f8035070 con 0x564ff1752f50 2024-03-21T11:24:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.412+0000 7fd5ebe00640 1 -- 172.21.15.47:0/1455795615 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800f740 con 0x564ff1752f50 2024-03-21T11:24:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.412+0000 7fd6091b7740 1 -- 172.21.15.47:0/1455795615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff1752f50 msgr2=0x564ff174c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.412+0000 7fd6091b7740 1 --2- 172.21.15.47:0/1455795615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff1752f50 0x564ff174c100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x564ff174e640 tx=0x7fd5f8024000 comp rx=0 tx=0).stop 2024-03-21T11:24:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.412+0000 7fd6091b7740 1 -- 172.21.15.47:0/1455795615 shutdown_connections 2024-03-21T11:24:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.412+0000 7fd6091b7740 1 --2- 172.21.15.47:0/1455795615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff1752f50 0x564ff174c100 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.412+0000 7fd6091b7740 1 --2- 172.21.15.47:0/1455795615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff174dde0 0x564ff175eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.412+0000 7fd6091b7740 1 --2- 172.21.15.47:0/1455795615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff174d3f0 0x564ff174bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.412+0000 7fd6091b7740 1 -- 172.21.15.47:0/1455795615 >> 172.21.15.47:0/1455795615 conn(0x564ff1748960 msgr2=0x564ff1748d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.413+0000 7fd6091b7740 1 -- 172.21.15.47:0/1455795615 shutdown_connections 2024-03-21T11:24:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.413+0000 7fd6091b7740 1 -- 172.21.15.47:0/1455795615 wait complete. 2024-03-21T11:24:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.413+0000 7fd6091b7740 1 -- start start 2024-03-21T11:24:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.413+0000 7fd6091b7740 1 Processor -- start 2024-03-21T11:24:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.414+0000 7fd6091b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff174d3f0 0x564ff175c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.414+0000 7fd6091b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174dde0 0x564ff175ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.414+0000 7fd6091b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff1752f50 0x564ff175dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.414+0000 7fd6091b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6001ad490 con 0x564ff174d3f0 2024-03-21T11:24:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.414+0000 7fd6091b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6001ace30 con 0x564ff174dde0 2024-03-21T11:24:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.414+0000 7fd6091b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6001aa330 con 0x564ff1752f50 2024-03-21T11:24:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.414+0000 7fd6091b7740 10 client.0 _ll_register_callbacks cb 0x7fd5f00bd1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.414+0000 7fd5fea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174dde0 0x564ff175ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.414+0000 7fd5fea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174dde0 0x564ff175ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54022/0 (socket says 172.21.15.47:54022) 2024-03-21T11:24:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.414+0000 7fd5fea00640 1 -- 172.21.15.47:0/2815199101 learned_addr learned my addr 172.21.15.47:0/2815199101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.414+0000 7fd5ffe00640 1 --2- 172.21.15.47:0/2815199101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff1752f50 0x564ff175dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.414+0000 7fd5ff400640 1 --2- 172.21.15.47:0/2815199101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff174d3f0 0x564ff175c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.414+0000 7fd5fea00640 1 -- 172.21.15.47:0/2815199101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff1752f50 msgr2=0x564ff175dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.414+0000 7fd5fea00640 1 --2- 172.21.15.47:0/2815199101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff1752f50 0x564ff175dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.415+0000 7fd5fea00640 1 -- 172.21.15.47:0/2815199101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff174d3f0 msgr2=0x564ff175c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.415+0000 7fd5fea00640 1 --2- 172.21.15.47:0/2815199101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff174d3f0 0x564ff175c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.415+0000 7fd5fea00640 1 -- 172.21.15.47:0/2815199101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ff175e290 con 0x564ff174dde0 2024-03-21T11:24:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.415+0000 7fd5fea00640 1 --2- 172.21.15.47:0/2815199101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174dde0 0x564ff175ccc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009530 tx=0x7fd5ec010cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c794a0bebf542fb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.415+0000 7fd5e9600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:24:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.415+0000 7fd5e9600640 1 -- 172.21.15.47:0/2815199101 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec024070 con 0x564ff174dde0 2024-03-21T11:24:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.415+0000 7fd5e9600640 1 -- 172.21.15.47:0/2815199101 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd5ec00fc80 con 0x564ff174dde0 2024-03-21T11:24:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.415+0000 7fd6091b7740 1 -- 172.21.15.47:0/2815199101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5f8035070 con 0x564ff174dde0 2024-03-21T11:24:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.416+0000 7fd5e9600640 1 -- 172.21.15.47:0/2815199101 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5ec030050 con 0x564ff174dde0 2024-03-21T11:24:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.416+0000 7fd6091b7740 10 client.25838 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.416+0000 7fd6091b7740 1 -- 172.21.15.47:0/2815199101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564ff17c7840 con 0x564ff174dde0 2024-03-21T11:24:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.416+0000 7fd5e9600640 1 -- 172.21.15.47:0/2815199101 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5f8035070 con 0x564ff174dde0 2024-03-21T11:24:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.416+0000 7fd5e9600640 1 -- 172.21.15.47:0/2815199101 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd5ec00e7f0 con 0x564ff174dde0 2024-03-21T11:24:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.416+0000 7fd6091b7740 10 client.25838 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.416+0000 7fd6091b7740 10 client.25838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.416+0000 7fd6091b7740 20 client.25838 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.416+0000 7fd6091b7740 1 --2- 172.21.15.47:0/2815199101 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ff17c8170 0x564ff17ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.416+0000 7fd6091b7740 4 client.25838 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:24:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.416+0000 7fd6091b7740 1 -- 172.21.15.47:0/2815199101 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564ff16ba0c0 con 0x564ff17c8170 2024-03-21T11:24:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.416+0000 7fd5ff400640 1 --2- 172.21.15.47:0/2815199101 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ff17c8170 0x564ff17ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.417+0000 7fd5ff400640 1 --2- 172.21.15.47:0/2815199101 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ff17c8170 0x564ff17ca590 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f9273d1f19b48ec server_cookie=e5434293b0bb4b in_seq=0 out_seq=0 2024-03-21T11:24:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.417+0000 7fd5e9600640 10 client.25838 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.420+0000 7fd5e9600640 1 -- 172.21.15.47:0/2815199101 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564ff16ba0c0 con 0x564ff17c8170 2024-03-21T11:24:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.420+0000 7fd5e9600640 10 client.25838 handle_command_reply: tid=0 2024-03-21T11:24:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.420+0000 7fd6091b7740 1 client.25838 shutdown 2024-03-21T11:24:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.420+0000 7fd6091b7740 1 -- 172.21.15.47:0/2815199101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174dde0 msgr2=0x564ff175ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.420+0000 7fd6091b7740 1 --2- 172.21.15.47:0/2815199101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174dde0 0x564ff175ccc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009530 tx=0x7fd5ec010cc0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.421+0000 7fd6091b7740 1 -- 172.21.15.47:0/2815199101 shutdown_connections 2024-03-21T11:24:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.421+0000 7fd6091b7740 1 --2- 172.21.15.47:0/2815199101 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564ff17c8170 0x564ff17ca590 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.421+0000 7fd6091b7740 1 --2- 172.21.15.47:0/2815199101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ff1752f50 0x564ff175dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.421+0000 7fd6091b7740 1 --2- 172.21.15.47:0/2815199101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ff174dde0 0x564ff175ccc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.421+0000 7fd6091b7740 1 --2- 172.21.15.47:0/2815199101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564ff174d3f0 0x564ff175c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.421+0000 7fd6091b7740 1 -- 172.21.15.47:0/2815199101 >> 172.21.15.47:0/2815199101 conn(0x564ff1748960 msgr2=0x564ff174bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.421+0000 7fd5e9600640 0 client.25838 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.421+0000 7fd6091b7740 1 -- 172.21.15.47:0/2815199101 shutdown_connections 2024-03-21T11:24:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.421+0000 7fd6091b7740 1 -- 172.21.15.47:0/2815199101 wait complete. 2024-03-21T11:24:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.421+0000 7fd6091b7740 20 client.25838 trim_cache size 0 max 16384 2024-03-21T11:24:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:24:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:24:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:24:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:24:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.422+0000 7fd604e00640 1 -- 172.21.15.47:0/3037111092 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5e40a2eb0 msgr2=0x7fd5e40a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.422+0000 7fd604e00640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5e40a2eb0 0x7fd5e40a5370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd5f801b040 tx=0x7fd5f8000f40 comp rx=0 tx=0).stop 2024-03-21T11:24:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.422+0000 7fd604e00640 1 -- 172.21.15.47:0/3037111092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6000740a0 msgr2=0x7fd600199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.422+0000 7fd604e00640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6000740a0 0x7fd600199a30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0005940 tx=0x7fd5f00042f0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.422+0000 7fd604e00640 1 -- 172.21.15.47:0/3037111092 shutdown_connections 2024-03-21T11:24:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.422+0000 7fd604e00640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5e40a2eb0 0x7fd5e40a5370 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.422+0000 7fd604e00640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6001093b0 0x7fd6001a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.422+0000 7fd604e00640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd600075680 0x7fd600199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.422+0000 7fd604e00640 1 --2- 172.21.15.47:0/3037111092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6000740a0 0x7fd600199a30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.422+0000 7fd604e00640 1 -- 172.21.15.47:0/3037111092 >> 172.21.15.47:0/3037111092 conn(0x7fd6000fcab0 msgr2=0x7fd600105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.422+0000 7fd604e00640 1 -- 172.21.15.47:0/3037111092 shutdown_connections 2024-03-21T11:24:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:42.422+0000 7fd604e00640 1 -- 172.21.15.47:0/3037111092 wait complete. 2024-03-21T11:24:42.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:42 smithi102 ceph-mon[36748]: pgmap v441: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-21T11:24:42.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:24:42.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:24:42.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:24:42.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:42 smithi047 ceph-mon[31351]: pgmap v441: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-21T11:24:42.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:24:42.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:24:42.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:24:42.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:42 smithi100 ceph-mon[36245]: pgmap v441: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-21T11:24:42.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:24:42.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:24:42.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:24:44.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:44 smithi102 ceph-mon[36748]: pgmap v442: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T11:24:44.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:44 smithi047 ceph-mon[31351]: pgmap v442: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T11:24:44.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:44 smithi100 ceph-mon[36245]: pgmap v442: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T11:24:46.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:46 smithi102 ceph-mon[36748]: pgmap v443: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-21T11:24:46.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:46 smithi047 ceph-mon[31351]: pgmap v443: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-21T11:24:46.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:46 smithi100 ceph-mon[36245]: pgmap v443: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-21T11:24:48.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:48 smithi102 ceph-mon[36748]: pgmap v444: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-21T11:24:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:48 smithi047 ceph-mon[31351]: pgmap v444: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-21T11:24:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:48 smithi100 ceph-mon[36245]: pgmap v444: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-21T11:24:50.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:50 smithi102 ceph-mon[36748]: pgmap v445: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T11:24:50.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:50 smithi047 ceph-mon[31351]: pgmap v445: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T11:24:50.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:50 smithi100 ceph-mon[36245]: pgmap v445: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T11:24:51.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:51 smithi047 ceph-mon[31351]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-03-21T11:24:51.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:51 smithi102 ceph-mon[36748]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-03-21T11:24:51.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:51 smithi100 ceph-mon[36245]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-03-21T11:24:52.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:52 smithi102 ceph-mon[36748]: pgmap v446: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-21T11:24:52.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:52 smithi047 ceph-mon[31351]: pgmap v446: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-21T11:24:52.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:52 smithi100 ceph-mon[36245]: pgmap v446: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-21T11:24:53.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:24:53.824 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:24:54.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.048+0000 7f17d5200640 1 -- 172.21.15.47:0/1298793198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d0102000 msgr2=0x7f17d01043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.048+0000 7f17d5200640 1 --2- 172.21.15.47:0/1298793198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d0102000 0x7f17d01043f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009900 tx=0x7f17c002bc30 comp rx=0 tx=0).stop 2024-03-21T11:24:54.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.048+0000 7f17d5200640 1 -- 172.21.15.47:0/1298793198 shutdown_connections 2024-03-21T11:24:54.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.048+0000 7f17d5200640 1 --2- 172.21.15.47:0/1298793198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d0107260 0x7f17d01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.048+0000 7f17d5200640 1 --2- 172.21.15.47:0/1298793198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d0104930 0x7f17d0106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.048+0000 7f17d5200640 1 --2- 172.21.15.47:0/1298793198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d0102000 0x7f17d01043f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.048+0000 7f17d5200640 1 -- 172.21.15.47:0/1298793198 >> 172.21.15.47:0/1298793198 conn(0x7f17d00fba10 msgr2=0x7f17d00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:54.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.048+0000 7f17d5200640 1 -- 172.21.15.47:0/1298793198 shutdown_connections 2024-03-21T11:24:54.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.048+0000 7f17d5200640 1 -- 172.21.15.47:0/1298793198 wait complete. 2024-03-21T11:24:54.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.050+0000 7f17d5200640 1 Processor -- start 2024-03-21T11:24:54.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.050+0000 7f17d5200640 1 -- start start 2024-03-21T11:24:54.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.050+0000 7f17d5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d0102000 0x7f17d0199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.050+0000 7f17d5200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d0104930 0x7f17d019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.050+0000 7f17d5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d0107260 0x7f17d01a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.051+0000 7f17d5200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d010d260 con 0x7f17d0104930 2024-03-21T11:24:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.051+0000 7f17d5200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d010d0e0 con 0x7f17d0107260 2024-03-21T11:24:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.051+0000 7f17d5200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d010d3e0 con 0x7f17d0102000 2024-03-21T11:24:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.051+0000 7f17cf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d0102000 0x7f17d0199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.051+0000 7f17cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d0107260 0x7f17d01a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.051+0000 7f17cf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d0102000 0x7f17d0199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52374/0 (socket says 172.21.15.47:52374) 2024-03-21T11:24:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.051+0000 7f17cf400640 1 -- 172.21.15.47:0/2756197266 learned_addr learned my addr 172.21.15.47:0/2756197266 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:54.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.051+0000 7f17cea00640 1 --2- 172.21.15.47:0/2756197266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d0104930 0x7f17d019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.051+0000 7f17cf400640 1 -- 172.21.15.47:0/2756197266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d0107260 msgr2=0x7f17d01a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.051+0000 7f17cf400640 1 --2- 172.21.15.47:0/2756197266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d0107260 0x7f17d01a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.051+0000 7f17cf400640 1 -- 172.21.15.47:0/2756197266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d0104930 msgr2=0x7f17d019a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.051+0000 7f17cf400640 1 --2- 172.21.15.47:0/2756197266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d0104930 0x7f17d019a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.051+0000 7f17cf400640 1 -- 172.21.15.47:0/2756197266 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d01a1710 con 0x7f17d0102000 2024-03-21T11:24:54.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.051+0000 7f17cea00640 1 --2- 172.21.15.47:0/2756197266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d0104930 0x7f17d019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:54.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.052+0000 7f17cf400640 1 --2- 172.21.15.47:0/2756197266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d0102000 0x7f17d0199af0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f17c0005ab0 tx=0x7f17c002c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:54.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.052+0000 7f17c7e00640 1 -- 172.21.15.47:0/2756197266 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0043020 con 0x7f17d0102000 2024-03-21T11:24:54.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.052+0000 7f17c7e00640 1 -- 172.21.15.47:0/2756197266 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17c00054e0 con 0x7f17d0102000 2024-03-21T11:24:54.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.052+0000 7f17c7e00640 1 -- 172.21.15.47:0/2756197266 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0003ce0 con 0x7f17d0102000 2024-03-21T11:24:54.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.052+0000 7f17d5200640 1 -- 172.21.15.47:0/2756197266 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d01a19a0 con 0x7f17d0102000 2024-03-21T11:24:54.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.052+0000 7f17d5200640 1 -- 172.21.15.47:0/2756197266 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d01a1ea0 con 0x7f17d0102000 2024-03-21T11:24:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.056+0000 7f17c7e00640 1 -- 172.21.15.47:0/2756197266 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f17c0046020 con 0x7f17d0102000 2024-03-21T11:24:54.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.058+0000 7f17d957c740 1 Processor -- start 2024-03-21T11:24:54.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.058+0000 7f17d957c740 1 -- start start 2024-03-21T11:24:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.059+0000 7f17c7e00640 1 --2- 172.21.15.47:0/2756197266 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17b40a2fa0 0x7f17b40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.059+0000 7f17d957c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093050 0x56523b093470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.059+0000 7f17d957c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b093a40 0x56523b098540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.059+0000 7f17c7e00640 1 -- 172.21.15.47:0/2756197266 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f17c00f1270 con 0x7f17d0102000 2024-03-21T11:24:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.059+0000 7f17d957c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b098bb0 0x56523b09b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.059+0000 7f17d957c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523aff8250 con 0x56523b098bb0 2024-03-21T11:24:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.059+0000 7f17d957c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523afa0a30 con 0x56523b093050 2024-03-21T11:24:54.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.059+0000 7f17d957c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523aff42f0 con 0x56523b093a40 2024-03-21T11:24:54.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.060+0000 7f17cf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093050 0x56523b093470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.060+0000 7f17cfe00640 1 --2- 172.21.15.47:0/2756197266 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17b40a2fa0 0x7f17b40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.060+0000 7f17cf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093050 0x56523b093470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40272/0 (socket says 172.21.15.47:40272) 2024-03-21T11:24:54.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.060+0000 7f17cf400640 1 -- 172.21.15.47:0/2429860039 learned_addr learned my addr 172.21.15.47:0/2429860039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:54.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.060+0000 7f17cfe00640 1 --2- 172.21.15.47:0/2429860039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b098bb0 0x56523b09b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.060+0000 7f17cea00640 1 --2- 172.21.15.47:0/2429860039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b093a40 0x56523b098540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.060+0000 7f17cf400640 1 -- 172.21.15.47:0/2429860039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b093a40 msgr2=0x56523b098540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.060+0000 7f17cf400640 1 --2- 172.21.15.47:0/2429860039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b093a40 0x56523b098540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.060+0000 7f17cf400640 1 -- 172.21.15.47:0/2429860039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b098bb0 msgr2=0x56523b09b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.060+0000 7f17cf400640 1 --2- 172.21.15.47:0/2429860039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b098bb0 0x56523b09b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.060+0000 7f17cf400640 1 -- 172.21.15.47:0/2429860039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56523af8b690 con 0x56523b093050 2024-03-21T11:24:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.061+0000 7f17cf400640 1 --2- 172.21.15.47:0/2429860039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093050 0x56523b093470 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f17c00b9f80 tx=0x7f17c002c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55a520544147f460 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.061+0000 7f17c4c00640 1 -- 172.21.15.47:0/2429860039 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c00a12e0 con 0x56523b093050 2024-03-21T11:24:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.061+0000 7f17cfe00640 1 --2- 172.21.15.47:0/2756197266 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17b40a2fa0 0x7f17b40a5460 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f17c80098b0 tx=0x7f17c8009840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.061+0000 7f17c4c00640 1 -- 172.21.15.47:0/2429860039 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17c0004c80 con 0x56523b093050 2024-03-21T11:24:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.061+0000 7f17c4c00640 1 -- 172.21.15.47:0/2429860039 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c00c5fc0 con 0x56523b093050 2024-03-21T11:24:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.062+0000 7f17d957c740 1 -- 172.21.15.47:0/2429860039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093050 msgr2=0x56523b093470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.062+0000 7f17d957c740 1 --2- 172.21.15.47:0/2429860039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093050 0x56523b093470 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f17c00b9f80 tx=0x7f17c002c8a0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.062+0000 7f17d957c740 1 -- 172.21.15.47:0/2429860039 shutdown_connections 2024-03-21T11:24:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.062+0000 7f17d957c740 1 --2- 172.21.15.47:0/2429860039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b098bb0 0x56523b09b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.062+0000 7f17d957c740 1 --2- 172.21.15.47:0/2429860039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b093a40 0x56523b098540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.062+0000 7f17d957c740 1 --2- 172.21.15.47:0/2429860039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093050 0x56523b093470 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.062+0000 7f17d957c740 1 -- 172.21.15.47:0/2429860039 >> 172.21.15.47:0/2429860039 conn(0x56523b05b200 msgr2=0x56523b08e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.062+0000 7f17d957c740 1 -- 172.21.15.47:0/2429860039 shutdown_connections 2024-03-21T11:24:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.062+0000 7f17d957c740 1 -- 172.21.15.47:0/2429860039 wait complete. 2024-03-21T11:24:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.063+0000 7f17d957c740 1 -- start start 2024-03-21T11:24:54.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.063+0000 7f17d957c740 1 Processor -- start 2024-03-21T11:24:54.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.063+0000 7f17d957c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093050 0x56523b09cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.063+0000 7f17d957c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093a40 0x56523b09d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.063+0000 7f17d957c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b098bb0 0x56523b0a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.063+0000 7f17d957c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523b09b920 con 0x56523b093050 2024-03-21T11:24:54.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.063+0000 7f17d957c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523afb3e80 con 0x56523b093a40 2024-03-21T11:24:54.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.063+0000 7f17d957c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523b09baa0 con 0x56523b098bb0 2024-03-21T11:24:54.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.063+0000 7f17d957c740 10 client.0 _ll_register_callbacks cb 0x56523afc6130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:54.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.064+0000 7f17cfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b098bb0 0x56523b0a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.064+0000 7f17cfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b098bb0 0x56523b0a42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52396/0 (socket says 172.21.15.47:52396) 2024-03-21T11:24:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.064+0000 7f17cfe00640 1 -- 172.21.15.47:0/4184550962 learned_addr learned my addr 172.21.15.47:0/4184550962 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.064+0000 7f17cf400640 1 --2- 172.21.15.47:0/4184550962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093050 0x56523b09cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.064+0000 7f17cfe00640 1 -- 172.21.15.47:0/4184550962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093a40 msgr2=0x56523b09d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.064+0000 7f17cfe00640 1 --2- 172.21.15.47:0/4184550962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093a40 0x56523b09d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.064+0000 7f17cfe00640 1 -- 172.21.15.47:0/4184550962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093050 msgr2=0x56523b09cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.064+0000 7f17cfe00640 1 --2- 172.21.15.47:0/4184550962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093050 0x56523b09cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.064+0000 7f17cfe00640 1 -- 172.21.15.47:0/4184550962 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56523add67d0 con 0x56523b098bb0 2024-03-21T11:24:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.064+0000 7f17cf400640 1 --2- 172.21.15.47:0/4184550962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093050 0x56523b09cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.065+0000 7f17cfe00640 1 --2- 172.21.15.47:0/4184550962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b098bb0 0x56523b0a42b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f17c8012fd0 tx=0x7f17c8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26e0bc663d5f52a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.065+0000 7f17b9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.065+0000 7f17b9600640 1 -- 172.21.15.47:0/4184550962 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8025070 con 0x56523b098bb0 2024-03-21T11:24:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.065+0000 7f17b9600640 1 -- 172.21.15.47:0/4184550962 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f17c801edd0 con 0x56523b098bb0 2024-03-21T11:24:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.065+0000 7f17d957c740 1 -- 172.21.15.47:0/4184550962 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17c0004c80 con 0x56523b098bb0 2024-03-21T11:24:54.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.065+0000 7f17b9600640 1 -- 172.21.15.47:0/4184550962 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17c8034050 con 0x56523b098bb0 2024-03-21T11:24:54.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.065+0000 7f17d957c740 10 client.25870 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:54.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.065+0000 7f17d957c740 1 -- 172.21.15.47:0/4184550962 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56523ad0c3f0 con 0x56523b098bb0 2024-03-21T11:24:54.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.066+0000 7f17b9600640 1 -- 172.21.15.47:0/4184550962 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17c0004c80 con 0x56523b098bb0 2024-03-21T11:24:54.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.066+0000 7f17b9600640 1 -- 172.21.15.47:0/4184550962 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f17c800f040 con 0x56523b098bb0 2024-03-21T11:24:54.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.066+0000 7f17d957c740 10 client.25870 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:54.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.066+0000 7f17d957c740 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.066+0000 7f17d957c740 20 client.25870 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.066+0000 7f17d957c740 1 --2- 172.21.15.47:0/4184550962 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56523b0a8c00 0x56523b0ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.066+0000 7f17d957c740 4 client.25870 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.066+0000 7f17d957c740 1 -- 172.21.15.47:0/4184550962 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56523b005790 con 0x56523b0a8c00 2024-03-21T11:24:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.066+0000 7f17cf400640 1 --2- 172.21.15.47:0/4184550962 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56523b0a8c00 0x56523b0ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.067+0000 7f17cf400640 1 --2- 172.21.15.47:0/4184550962 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56523b0a8c00 0x56523b0ab0c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=193ec54e6b1c8fbd server_cookie=434048c8d5d4ef09 in_seq=0 out_seq=0 2024-03-21T11:24:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.067+0000 7f17b9600640 10 client.25870 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.067+0000 7f17b9600640 1 -- 172.21.15.47:0/4184550962 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56523b005790 con 0x56523b0a8c00 2024-03-21T11:24:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.067+0000 7f17b9600640 10 client.25870 handle_command_reply: tid=0 2024-03-21T11:24:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.068+0000 7f17d957c740 1 client.25870 shutdown 2024-03-21T11:24:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.068+0000 7f17d957c740 1 -- 172.21.15.47:0/4184550962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b098bb0 msgr2=0x56523b0a42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.068+0000 7f17d957c740 1 --2- 172.21.15.47:0/4184550962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b098bb0 0x56523b0a42b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f17c8012fd0 tx=0x7f17c8004ab0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.068+0000 7f17d957c740 1 -- 172.21.15.47:0/4184550962 shutdown_connections 2024-03-21T11:24:54.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.068+0000 7f17d957c740 1 --2- 172.21.15.47:0/4184550962 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56523b0a8c00 0x56523b0ab0c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.068+0000 7f17d957c740 1 --2- 172.21.15.47:0/4184550962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b098bb0 0x56523b0a42b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.068+0000 7f17d957c740 1 --2- 172.21.15.47:0/4184550962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093a40 0x56523b09d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.068+0000 7f17d957c740 1 --2- 172.21.15.47:0/4184550962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093050 0x56523b09cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.068+0000 7f17b9600640 0 client.25870 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:54.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.068+0000 7f17d957c740 1 -- 172.21.15.47:0/4184550962 >> 172.21.15.47:0/4184550962 conn(0x56523b05b200 msgr2=0x56523b08dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:54.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.068+0000 7f17d957c740 1 -- 172.21.15.47:0/4184550962 shutdown_connections 2024-03-21T11:24:54.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.068+0000 7f17d957c740 1 -- 172.21.15.47:0/4184550962 wait complete. 2024-03-21T11:24:54.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.068+0000 7f17d957c740 20 client.25870 trim_cache size 0 max 16384 2024-03-21T11:24:54.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.086+0000 7f17d957c740 1 Processor -- start 2024-03-21T11:24:54.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.086+0000 7f17d957c740 1 -- start start 2024-03-21T11:24:54.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.086+0000 7f17d957c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093050 0x56523b098080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.086+0000 7f17d957c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b093a40 0x56523b0a4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.086+0000 7f17d957c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b098bb0 0x56523b08d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.086+0000 7f17d957c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d01ac170 con 0x56523b093050 2024-03-21T11:24:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.086+0000 7f17d957c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d01adaf0 con 0x56523b098bb0 2024-03-21T11:24:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.086+0000 7f17d957c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d01ac7d0 con 0x56523b093a40 2024-03-21T11:24:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.086+0000 7f17cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b098bb0 0x56523b08d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.086+0000 7f17cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b098bb0 0x56523b08d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40296/0 (socket says 172.21.15.47:40296) 2024-03-21T11:24:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.086+0000 7f17cfe00640 1 -- 172.21.15.47:0/3471901840 learned_addr learned my addr 172.21.15.47:0/3471901840 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.086+0000 7f17cea00640 1 --2- 172.21.15.47:0/3471901840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b093a40 0x56523b0a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.086+0000 7f17cf400640 1 --2- 172.21.15.47:0/3471901840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093050 0x56523b098080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.087+0000 7f17cfe00640 1 -- 172.21.15.47:0/3471901840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b093a40 msgr2=0x56523b0a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.087+0000 7f17cfe00640 1 --2- 172.21.15.47:0/3471901840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b093a40 0x56523b0a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.087+0000 7f17cfe00640 1 -- 172.21.15.47:0/3471901840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093050 msgr2=0x56523b098080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.087+0000 7f17cfe00640 1 --2- 172.21.15.47:0/3471901840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093050 0x56523b098080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.087+0000 7f17cfe00640 1 -- 172.21.15.47:0/3471901840 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56523accc0f0 con 0x56523b098bb0 2024-03-21T11:24:54.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.087+0000 7f17cfe00640 1 --2- 172.21.15.47:0/3471901840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b098bb0 0x56523b08d330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f17c800a010 tx=0x7f17c8012ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76221cde1aa3b397 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:54.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.087+0000 7f17bbe00640 1 -- 172.21.15.47:0/3471901840 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8025040 con 0x56523b098bb0 2024-03-21T11:24:54.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.087+0000 7f17bbe00640 1 -- 172.21.15.47:0/3471901840 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17c801fd00 con 0x56523b098bb0 2024-03-21T11:24:54.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.087+0000 7f17bbe00640 1 -- 172.21.15.47:0/3471901840 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8029660 con 0x56523b098bb0 2024-03-21T11:24:54.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.088+0000 7f17d957c740 1 -- 172.21.15.47:0/3471901840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b098bb0 msgr2=0x56523b08d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.088+0000 7f17d957c740 1 --2- 172.21.15.47:0/3471901840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b098bb0 0x56523b08d330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f17c800a010 tx=0x7f17c8012ee0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.088+0000 7f17d957c740 1 -- 172.21.15.47:0/3471901840 shutdown_connections 2024-03-21T11:24:54.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.088+0000 7f17d957c740 1 --2- 172.21.15.47:0/3471901840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b098bb0 0x56523b08d330 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.088+0000 7f17d957c740 1 --2- 172.21.15.47:0/3471901840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b093a40 0x56523b0a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.088+0000 7f17d957c740 1 --2- 172.21.15.47:0/3471901840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093050 0x56523b098080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.088+0000 7f17d957c740 1 -- 172.21.15.47:0/3471901840 >> 172.21.15.47:0/3471901840 conn(0x56523b05b200 msgr2=0x56523b094d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:54.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.088+0000 7f17d957c740 1 -- 172.21.15.47:0/3471901840 shutdown_connections 2024-03-21T11:24:54.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.088+0000 7f17d957c740 1 -- 172.21.15.47:0/3471901840 wait complete. 2024-03-21T11:24:54.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.089+0000 7f17d957c740 1 -- start start 2024-03-21T11:24:54.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.089+0000 7f17d957c740 1 Processor -- start 2024-03-21T11:24:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.089+0000 7f17d957c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093050 0x56523b0a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.089+0000 7f17d957c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093a40 0x56523b0a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.089+0000 7f17d957c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b098bb0 0x56523b0a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.089+0000 7f17d957c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d01ad490 con 0x56523b093a40 2024-03-21T11:24:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.089+0000 7f17d957c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d01ace30 con 0x56523b093050 2024-03-21T11:24:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.089+0000 7f17d957c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d01aa330 con 0x56523b098bb0 2024-03-21T11:24:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.089+0000 7f17d957c740 10 client.0 _ll_register_callbacks cb 0x7f17c8012510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.090+0000 7f17cea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093a40 0x56523b0a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.090+0000 7f17cfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b098bb0 0x56523b0a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.090+0000 7f17cfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b098bb0 0x56523b0a33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52420/0 (socket says 172.21.15.47:52420) 2024-03-21T11:24:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.090+0000 7f17cfe00640 1 -- 172.21.15.47:0/281471026 learned_addr learned my addr 172.21.15.47:0/281471026 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.090+0000 7f17cf400640 1 --2- 172.21.15.47:0/281471026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093050 0x56523b0a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.090+0000 7f17cfe00640 1 -- 172.21.15.47:0/281471026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093050 msgr2=0x56523b0a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.090+0000 7f17cfe00640 1 --2- 172.21.15.47:0/281471026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093050 0x56523b0a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.090+0000 7f17cfe00640 1 -- 172.21.15.47:0/281471026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093a40 msgr2=0x56523b0a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.090+0000 7f17cfe00640 1 --2- 172.21.15.47:0/281471026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093a40 0x56523b0a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.090+0000 7f17cfe00640 1 -- 172.21.15.47:0/281471026 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56523b0a39e0 con 0x56523b098bb0 2024-03-21T11:24:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.090+0000 7f17cea00640 1 --2- 172.21.15.47:0/281471026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093a40 0x56523b0a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.090+0000 7f17cf400640 1 --2- 172.21.15.47:0/281471026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093050 0x56523b0a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.091+0000 7f17cfe00640 1 --2- 172.21.15.47:0/281471026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b098bb0 0x56523b0a33f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f17c801f630 tx=0x7f17c801ebd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c5e8638320224ff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.091+0000 7f17b9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.091+0000 7f17b9600640 1 -- 172.21.15.47:0/281471026 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8025040 con 0x56523b098bb0 2024-03-21T11:24:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.091+0000 7f17d957c740 1 -- 172.21.15.47:0/281471026 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17c801fd00 con 0x56523b098bb0 2024-03-21T11:24:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.091+0000 7f17b9600640 1 -- 172.21.15.47:0/281471026 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f17c801f9d0 con 0x56523b098bb0 2024-03-21T11:24:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.091+0000 7f17b9600640 1 -- 172.21.15.47:0/281471026 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17c8004a00 con 0x56523b098bb0 2024-03-21T11:24:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.091+0000 7f17d957c740 10 client.25876 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:54.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.091+0000 7f17d957c740 1 -- 172.21.15.47:0/281471026 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56523b104a40 con 0x56523b098bb0 2024-03-21T11:24:54.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.091+0000 7f17b9600640 1 -- 172.21.15.47:0/281471026 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17c801fd00 con 0x56523b098bb0 2024-03-21T11:24:54.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.092+0000 7f17b9600640 1 -- 172.21.15.47:0/281471026 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f17c8041400 con 0x56523b098bb0 2024-03-21T11:24:54.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.092+0000 7f17d957c740 10 client.25876 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:54.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.092+0000 7f17d957c740 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:54.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.092+0000 7f17d957c740 20 client.25876 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:54.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.092+0000 7f17d957c740 1 --2- 172.21.15.47:0/281471026 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56523b105280 0x56523b1076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.092+0000 7f17d957c740 4 client.25876 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:24:54.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.092+0000 7f17d957c740 1 -- 172.21.15.47:0/281471026 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56523afffae0 con 0x56523b105280 2024-03-21T11:24:54.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.093+0000 7f17cf400640 1 --2- 172.21.15.47:0/281471026 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56523b105280 0x56523b1076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.093+0000 7f17cf400640 1 --2- 172.21.15.47:0/281471026 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56523b105280 0x56523b1076a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9152a1c2caabebfa server_cookie=ddf7f8f00e147b32 in_seq=0 out_seq=0 2024-03-21T11:24:54.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.093+0000 7f17b9600640 10 client.25876 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:54.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.093+0000 7f17b9600640 1 -- 172.21.15.47:0/281471026 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56523afffae0 con 0x56523b105280 2024-03-21T11:24:54.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.093+0000 7f17b9600640 10 client.25876 handle_command_reply: tid=0 2024-03-21T11:24:54.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.093+0000 7f17d957c740 1 client.25876 shutdown 2024-03-21T11:24:54.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.093+0000 7f17d957c740 1 -- 172.21.15.47:0/281471026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b098bb0 msgr2=0x56523b0a33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.093+0000 7f17d957c740 1 --2- 172.21.15.47:0/281471026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b098bb0 0x56523b0a33f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f17c801f630 tx=0x7f17c801ebd0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.093+0000 7f17d957c740 1 -- 172.21.15.47:0/281471026 shutdown_connections 2024-03-21T11:24:54.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.093+0000 7f17d957c740 1 --2- 172.21.15.47:0/281471026 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56523b105280 0x56523b1076a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.094+0000 7f17d957c740 1 --2- 172.21.15.47:0/281471026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56523b098bb0 0x56523b0a33f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.094+0000 7f17d957c740 1 --2- 172.21.15.47:0/281471026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56523b093a40 0x56523b0a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.094+0000 7f17d957c740 1 --2- 172.21.15.47:0/281471026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56523b093050 0x56523b0a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.094+0000 7f17d957c740 1 -- 172.21.15.47:0/281471026 >> 172.21.15.47:0/281471026 conn(0x56523b05b200 msgr2=0x56523b092490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:54.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.094+0000 7f17b9600640 0 client.25876 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:54.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.094+0000 7f17d957c740 1 -- 172.21.15.47:0/281471026 shutdown_connections 2024-03-21T11:24:54.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.094+0000 7f17d957c740 1 -- 172.21.15.47:0/281471026 wait complete. 2024-03-21T11:24:54.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.094+0000 7f17d957c740 20 client.25876 trim_cache size 0 max 16384 2024-03-21T11:24:54.100 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:24:54.100 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:24:54.100 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:24:54.100 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:24:54.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.094+0000 7f17d5200640 1 -- 172.21.15.47:0/2756197266 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17b40a2fa0 msgr2=0x7f17b40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.095+0000 7f17d5200640 1 --2- 172.21.15.47:0/2756197266 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17b40a2fa0 0x7f17b40a5460 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f17c80098b0 tx=0x7f17c8009840 comp rx=0 tx=0).stop 2024-03-21T11:24:54.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.095+0000 7f17d5200640 1 -- 172.21.15.47:0/2756197266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d0102000 msgr2=0x7f17d0199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.095+0000 7f17d5200640 1 --2- 172.21.15.47:0/2756197266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d0102000 0x7f17d0199af0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f17c0005ab0 tx=0x7f17c002c6c0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.095+0000 7f17d5200640 1 -- 172.21.15.47:0/2756197266 shutdown_connections 2024-03-21T11:24:54.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.095+0000 7f17d5200640 1 --2- 172.21.15.47:0/2756197266 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17b40a2fa0 0x7f17b40a5460 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.095+0000 7f17d5200640 1 --2- 172.21.15.47:0/2756197266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d0107260 0x7f17d01a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.095+0000 7f17d5200640 1 --2- 172.21.15.47:0/2756197266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17d0104930 0x7f17d019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.095+0000 7f17d5200640 1 --2- 172.21.15.47:0/2756197266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17d0102000 0x7f17d0199af0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.095+0000 7f17d5200640 1 -- 172.21.15.47:0/2756197266 >> 172.21.15.47:0/2756197266 conn(0x7f17d00fba10 msgr2=0x7f17d0101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:54.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.095+0000 7f17d5200640 1 -- 172.21.15.47:0/2756197266 shutdown_connections 2024-03-21T11:24:54.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.095+0000 7f17d5200640 1 -- 172.21.15.47:0/2756197266 wait complete. 2024-03-21T11:24:54.113 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:24:54.113 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:24:54.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.291+0000 7fe81f200640 1 -- 172.21.15.47:0/1821185706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8201026d0 msgr2=0x7fe820104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.291+0000 7fe81f200640 1 --2- 172.21.15.47:0/1821185706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8201026d0 0x7fe820104b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009900 tx=0x7fe80c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:24:54.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.292+0000 7fe81f200640 1 -- 172.21.15.47:0/1821185706 shutdown_connections 2024-03-21T11:24:54.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.292+0000 7fe81f200640 1 --2- 172.21.15.47:0/1821185706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe820105080 0x7fe820105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.292+0000 7fe81f200640 1 --2- 172.21.15.47:0/1821185706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8201026d0 0x7fe820104b40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.292+0000 7fe81f200640 1 --2- 172.21.15.47:0/1821185706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8200ffd60 0x7fe820102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.292+0000 7fe81f200640 1 -- 172.21.15.47:0/1821185706 >> 172.21.15.47:0/1821185706 conn(0x7fe8200fb980 msgr2=0x7fe8200fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:54.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.292+0000 7fe81f200640 1 -- 172.21.15.47:0/1821185706 shutdown_connections 2024-03-21T11:24:54.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.292+0000 7fe81f200640 1 -- 172.21.15.47:0/1821185706 wait complete. 2024-03-21T11:24:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.293+0000 7fe81f200640 1 Processor -- start 2024-03-21T11:24:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.293+0000 7fe81f200640 1 -- start start 2024-03-21T11:24:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.293+0000 7fe81f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8200ffd60 0x7fe82019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.293+0000 7fe81f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8201026d0 0x7fe82019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.293+0000 7fe81f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe820105080 0x7fe8201a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.293+0000 7fe81f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe820113790 con 0x7fe8200ffd60 2024-03-21T11:24:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.293+0000 7fe81f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe820113610 con 0x7fe820105080 2024-03-21T11:24:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.293+0000 7fe81f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe820113910 con 0x7fe8201026d0 2024-03-21T11:24:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.294+0000 7fe81d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8201026d0 0x7fe82019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.294+0000 7fe81d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8201026d0 0x7fe82019e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52438/0 (socket says 172.21.15.47:52438) 2024-03-21T11:24:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.294+0000 7fe81d400640 1 -- 172.21.15.47:0/3919241894 learned_addr learned my addr 172.21.15.47:0/3919241894 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:54.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.294+0000 7fe81e800640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe820105080 0x7fe8201a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.294+0000 7fe81de00640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8200ffd60 0x7fe82019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.294+0000 7fe81d400640 1 -- 172.21.15.47:0/3919241894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe820105080 msgr2=0x7fe8201a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.294+0000 7fe81d400640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe820105080 0x7fe8201a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.294+0000 7fe81d400640 1 -- 172.21.15.47:0/3919241894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8200ffd60 msgr2=0x7fe82019ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.294+0000 7fe81d400640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8200ffd60 0x7fe82019ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.294+0000 7fe81d400640 1 -- 172.21.15.47:0/3919241894 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8201a59d0 con 0x7fe8201026d0 2024-03-21T11:24:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.294+0000 7fe81e800640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe820105080 0x7fe8201a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.294+0000 7fe81de00640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8200ffd60 0x7fe82019ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.294+0000 7fe81d400640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8201026d0 0x7fe82019e300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe80c005ab0 tx=0x7fe80c02c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.295+0000 7fe816a00640 1 -- 172.21.15.47:0/3919241894 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c043020 con 0x7fe8201026d0 2024-03-21T11:24:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.295+0000 7fe816a00640 1 -- 172.21.15.47:0/3919241894 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe80c0054e0 con 0x7fe8201026d0 2024-03-21T11:24:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.295+0000 7fe816a00640 1 -- 172.21.15.47:0/3919241894 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c003ce0 con 0x7fe8201026d0 2024-03-21T11:24:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.295+0000 7fe81f200640 1 -- 172.21.15.47:0/3919241894 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8201a5c60 con 0x7fe8201026d0 2024-03-21T11:24:54.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.295+0000 7fe81f200640 1 -- 172.21.15.47:0/3919241894 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8201a6160 con 0x7fe8201026d0 2024-03-21T11:24:54.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.296+0000 7fe81f200640 1 -- 172.21.15.47:0/3919241894 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8200fd470 con 0x7fe8201026d0 2024-03-21T11:24:54.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.299+0000 7fe816a00640 1 -- 172.21.15.47:0/3919241894 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe80c046020 con 0x7fe8201026d0 2024-03-21T11:24:54.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.300+0000 7fe816a00640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe8040a2f20 0x7fe8040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.301+0000 7fe816a00640 1 -- 172.21.15.47:0/3919241894 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe80c0f1a90 con 0x7fe8201026d0 2024-03-21T11:24:54.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.301+0000 7fe81de00640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe8040a2f20 0x7fe8040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.302+0000 7fe81de00640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe8040a2f20 0x7fe8040a53e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe82019f290 tx=0x7fe810005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.309+0000 7fe816a00640 1 -- 172.21.15.47:0/3919241894 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe80c04d020 con 0x7fe8201026d0 2024-03-21T11:24:54.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:54 smithi102 ceph-mon[36748]: pgmap v447: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-21T11:24:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.546+0000 7fe81f200640 1 -- 172.21.15.47:0/3919241894 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe82002d270 con 0x7fe8201026d0 2024-03-21T11:24:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.548+0000 7fe816a00640 1 -- 172.21.15.47:0/3919241894 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fe80c03c350 con 0x7fe8201026d0 2024-03-21T11:24:54.549 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:24:54.549 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:24:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:24:54.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.551+0000 7fe81f200640 1 -- 172.21.15.47:0/3919241894 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe8040a2f20 msgr2=0x7fe8040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.551+0000 7fe81f200640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe8040a2f20 0x7fe8040a53e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe82019f290 tx=0x7fe810005f90 comp rx=0 tx=0).stop 2024-03-21T11:24:54.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.551+0000 7fe81f200640 1 -- 172.21.15.47:0/3919241894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8201026d0 msgr2=0x7fe82019e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.551+0000 7fe81f200640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8201026d0 0x7fe82019e300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe80c005ab0 tx=0x7fe80c02c6c0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.552+0000 7fe81f200640 1 -- 172.21.15.47:0/3919241894 shutdown_connections 2024-03-21T11:24:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.552+0000 7fe81f200640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe8040a2f20 0x7fe8040a53e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.552+0000 7fe81f200640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe820105080 0x7fe8201a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.552+0000 7fe81f200640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8201026d0 0x7fe82019e300 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.552+0000 7fe81f200640 1 --2- 172.21.15.47:0/3919241894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8200ffd60 0x7fe82019ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.552+0000 7fe81f200640 1 -- 172.21.15.47:0/3919241894 >> 172.21.15.47:0/3919241894 conn(0x7fe8200fb980 msgr2=0x7fe820100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.552+0000 7fe81f200640 1 -- 172.21.15.47:0/3919241894 shutdown_connections 2024-03-21T11:24:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.552+0000 7fe81f200640 1 -- 172.21.15.47:0/3919241894 wait complete. 2024-03-21T11:24:54.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:54 smithi047 ceph-mon[31351]: pgmap v447: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-21T11:24:54.578 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:24:54.579 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:24:54.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:54 smithi100 ceph-mon[36245]: pgmap v447: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-21T11:24:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.762+0000 7f296fe00640 1 -- 172.21.15.47:0/3858598961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2970102790 msgr2=0x7f2970102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.762+0000 7f296fe00640 1 --2- 172.21.15.47:0/3858598961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2970102790 0x7f2970102c10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f295c009900 tx=0x7f295c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:24:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.762+0000 7f296fe00640 1 -- 172.21.15.47:0/3858598961 shutdown_connections 2024-03-21T11:24:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.762+0000 7f296fe00640 1 --2- 172.21.15.47:0/3858598961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2970103150 0x7f29701099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.762+0000 7f296fe00640 1 --2- 172.21.15.47:0/3858598961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2970102790 0x7f2970102c10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.762+0000 7f296fe00640 1 --2- 172.21.15.47:0/3858598961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29701015a0 0x7f29701019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.762+0000 7f296fe00640 1 -- 172.21.15.47:0/3858598961 >> 172.21.15.47:0/3858598961 conn(0x7f29700fcb10 msgr2=0x7f29700fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.762+0000 7f296fe00640 1 -- 172.21.15.47:0/3858598961 shutdown_connections 2024-03-21T11:24:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.762+0000 7f296fe00640 1 -- 172.21.15.47:0/3858598961 wait complete. 2024-03-21T11:24:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.763+0000 7f296fe00640 1 Processor -- start 2024-03-21T11:24:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.763+0000 7f296fe00640 1 -- start start 2024-03-21T11:24:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.763+0000 7f296fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29701015a0 0x7f2970199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2970102790 0x7f297019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2970103150 0x7f29701a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297010d270 con 0x7f2970103150 2024-03-21T11:24:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297010d0f0 con 0x7f2970102790 2024-03-21T11:24:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297010d3f0 con 0x7f29701015a0 2024-03-21T11:24:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2970102790 0x7f297019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29701015a0 0x7f2970199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2970102790 0x7f297019a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40352/0 (socket says 172.21.15.47:40352) 2024-03-21T11:24:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296e000640 1 -- 172.21.15.47:0/2416787039 learned_addr learned my addr 172.21.15.47:0/2416787039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29701015a0 0x7f2970199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52462/0 (socket says 172.21.15.47:52462) 2024-03-21T11:24:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296f400640 1 --2- 172.21.15.47:0/2416787039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2970103150 0x7f29701a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296ea00640 1 -- 172.21.15.47:0/2416787039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2970102790 msgr2=0x7f297019a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296ea00640 1 --2- 172.21.15.47:0/2416787039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2970102790 0x7f297019a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296ea00640 1 -- 172.21.15.47:0/2416787039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2970103150 msgr2=0x7f29701a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296ea00640 1 --2- 172.21.15.47:0/2416787039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2970103150 0x7f29701a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296ea00640 1 -- 172.21.15.47:0/2416787039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29701a1740 con 0x7f29701015a0 2024-03-21T11:24:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.764+0000 7f296e000640 1 --2- 172.21.15.47:0/2416787039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2970102790 0x7f297019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.765+0000 7f296ea00640 1 --2- 172.21.15.47:0/2416787039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29701015a0 0x7f2970199b20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f296000e9c0 tx=0x7f296000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.765+0000 7f2967400640 1 -- 172.21.15.47:0/2416787039 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296000cd80 con 0x7f29701015a0 2024-03-21T11:24:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.765+0000 7f2967400640 1 -- 172.21.15.47:0/2416787039 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2960010910 con 0x7f29701015a0 2024-03-21T11:24:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.765+0000 7f296fe00640 1 -- 172.21.15.47:0/2416787039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29701a1a30 con 0x7f29701015a0 2024-03-21T11:24:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.765+0000 7f2967400640 1 -- 172.21.15.47:0/2416787039 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960018980 con 0x7f29701015a0 2024-03-21T11:24:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.765+0000 7f296fe00640 1 -- 172.21.15.47:0/2416787039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29701a1fa0 con 0x7f29701015a0 2024-03-21T11:24:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.769+0000 7f297839b740 1 Processor -- start 2024-03-21T11:24:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.769+0000 7f297839b740 1 -- start start 2024-03-21T11:24:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.771+0000 7f297839b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6242299f0 0x55a624229e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.771+0000 7f297839b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a62422b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.771+0000 7f297839b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a624231e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.771+0000 7f297839b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62418e2c0 con 0x55a6242299f0 2024-03-21T11:24:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.771+0000 7f297839b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a624136c90 con 0x55a62422abc0 2024-03-21T11:24:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.771+0000 7f297839b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62418a360 con 0x55a62422b580 2024-03-21T11:24:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.771+0000 7f296f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6242299f0 0x55a624229e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.771+0000 7f296e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a624231e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.771+0000 7f296e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a624231e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52472/0 (socket says 172.21.15.47:52472) 2024-03-21T11:24:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.771+0000 7f296e000640 1 -- 172.21.15.47:0/4224024911 learned_addr learned my addr 172.21.15.47:0/4224024911 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.771+0000 7f2967400640 1 -- 172.21.15.47:0/2416787039 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f296001f020 con 0x7f29701015a0 2024-03-21T11:24:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.772+0000 7f296e000640 1 -- 172.21.15.47:0/4224024911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 msgr2=0x55a62422b040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:24:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.772+0000 7f296e000640 1 --2- 172.21.15.47:0/4224024911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a62422b040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.772+0000 7f296e000640 1 -- 172.21.15.47:0/4224024911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6242299f0 msgr2=0x55a624229e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.772+0000 7f296e000640 1 --2- 172.21.15.47:0/4224024911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6242299f0 0x55a624229e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.772+0000 7f296e000640 1 -- 172.21.15.47:0/4224024911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a62415b080 con 0x55a62422b580 2024-03-21T11:24:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.772+0000 7f296f400640 1 --2- 172.21.15.47:0/4224024911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6242299f0 0x55a624229e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.772+0000 7f296e000640 1 --2- 172.21.15.47:0/4224024911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a624231e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f295c02c180 tx=0x7f295c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce4f65d444124f64 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.772+0000 7f295b400640 1 -- 172.21.15.47:0/4224024911 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c042070 con 0x55a62422b580 2024-03-21T11:24:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.773+0000 7f295b400640 1 -- 172.21.15.47:0/4224024911 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f295c02cd40 con 0x55a62422b580 2024-03-21T11:24:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.773+0000 7f295b400640 1 -- 172.21.15.47:0/4224024911 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c005330 con 0x55a62422b580 2024-03-21T11:24:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.773+0000 7f2967400640 1 --2- 172.21.15.47:0/2416787039 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f29540a2fa0 0x7f29540a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.773+0000 7f2967400640 1 -- 172.21.15.47:0/2416787039 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f29600cf6a0 con 0x7f29701015a0 2024-03-21T11:24:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.774+0000 7f297839b740 1 -- 172.21.15.47:0/4224024911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 msgr2=0x55a624231e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.774+0000 7f297839b740 1 --2- 172.21.15.47:0/4224024911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a624231e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f295c02c180 tx=0x7f295c000bc0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.774+0000 7f296f400640 1 --2- 172.21.15.47:0/2416787039 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f29540a2fa0 0x7f29540a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.774+0000 7f297839b740 1 -- 172.21.15.47:0/4224024911 shutdown_connections 2024-03-21T11:24:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.774+0000 7f297839b740 1 --2- 172.21.15.47:0/4224024911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a624231e10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.774+0000 7f297839b740 1 --2- 172.21.15.47:0/4224024911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a62422b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.774+0000 7f297839b740 1 --2- 172.21.15.47:0/4224024911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6242299f0 0x55a624229e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.774+0000 7f297839b740 1 -- 172.21.15.47:0/4224024911 >> 172.21.15.47:0/4224024911 conn(0x55a624224de0 msgr2=0x55a6242251e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.774+0000 7f297839b740 1 -- 172.21.15.47:0/4224024911 shutdown_connections 2024-03-21T11:24:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.774+0000 7f297839b740 1 -- 172.21.15.47:0/4224024911 wait complete. 2024-03-21T11:24:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.775+0000 7f297839b740 1 -- start start 2024-03-21T11:24:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.775+0000 7f297839b740 1 Processor -- start 2024-03-21T11:24:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.776+0000 7f296f400640 1 --2- 172.21.15.47:0/2416787039 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f29540a2fa0 0x7f29540a5460 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55a62422aac0 tx=0x7f2968009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.776+0000 7f297839b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6242299f0 0x55a624233c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.776+0000 7f297839b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a624234190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.776+0000 7f297839b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a62422b580 0x55a62423b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.776+0000 7f297839b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a624232530 con 0x55a62422b580 2024-03-21T11:24:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.776+0000 7f297839b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62414a0d0 con 0x55a62422abc0 2024-03-21T11:24:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.776+0000 7f297839b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6242326b0 con 0x55a6242299f0 2024-03-21T11:24:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.776+0000 7f297839b740 10 client.0 _ll_register_callbacks cb 0x55a62414ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.776+0000 7f296e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a624234190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.776+0000 7f296e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a624234190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40364/0 (socket says 172.21.15.47:40364) 2024-03-21T11:24:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.776+0000 7f296e000640 1 -- 172.21.15.47:0/3321651043 learned_addr learned my addr 172.21.15.47:0/3321651043 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.776+0000 7f296f400640 1 --2- 172.21.15.47:0/3321651043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a62422b580 0x55a62423b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.777+0000 7f296e000640 1 -- 172.21.15.47:0/3321651043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6242299f0 msgr2=0x55a624233c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:24:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.777+0000 7f296e000640 1 --2- 172.21.15.47:0/3321651043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6242299f0 0x55a624233c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.777+0000 7f296e000640 1 -- 172.21.15.47:0/3321651043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a62422b580 msgr2=0x55a62423b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.777+0000 7f296e000640 1 --2- 172.21.15.47:0/3321651043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a62422b580 0x55a62423b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.777+0000 7f296e000640 1 -- 172.21.15.47:0/3321651043 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a623ea23f0 con 0x55a62422abc0 2024-03-21T11:24:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.777+0000 7f296f400640 1 --2- 172.21.15.47:0/3321651043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a62422b580 0x55a62423b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.777+0000 7f296e000640 1 --2- 172.21.15.47:0/3321651043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a624234190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f295c02c250 tx=0x7f295c0044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5704c62798c8ddc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.777+0000 7f294fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:24:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.777+0000 7f294fe00640 1 -- 172.21.15.47:0/3321651043 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c04d070 con 0x55a62422abc0 2024-03-21T11:24:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.777+0000 7f297839b740 1 -- 172.21.15.47:0/3321651043 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f295c02cd40 con 0x55a62422abc0 2024-03-21T11:24:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.777+0000 7f294fe00640 1 -- 172.21.15.47:0/3321651043 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f295c004c50 con 0x55a62422abc0 2024-03-21T11:24:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.778+0000 7f294fe00640 1 -- 172.21.15.47:0/3321651043 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f295c042050 con 0x55a62422abc0 2024-03-21T11:24:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.778+0000 7f297839b740 10 client.25865 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.778+0000 7f297839b740 1 -- 172.21.15.47:0/3321651043 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a623f6c7d0 con 0x55a62422abc0 2024-03-21T11:24:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.778+0000 7f294fe00640 1 -- 172.21.15.47:0/3321651043 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f295c02cd40 con 0x55a62422abc0 2024-03-21T11:24:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.779+0000 7f294fe00640 1 -- 172.21.15.47:0/3321651043 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f295c047690 con 0x55a62422abc0 2024-03-21T11:24:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.779+0000 7f297839b740 10 client.25865 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.779+0000 7f297839b740 10 client.25865 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:24:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.779+0000 7f297839b740 20 client.25865 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.779+0000 7f297839b740 1 --2- 172.21.15.47:0/3321651043 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a62423f2a0 0x55a624241690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.779+0000 7f297839b740 4 client.25865 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.779+0000 7f297839b740 1 -- 172.21.15.47:0/3321651043 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a62419b800 con 0x55a62423f2a0 2024-03-21T11:24:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.780+0000 7f296ea00640 1 --2- 172.21.15.47:0/3321651043 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a62423f2a0 0x55a624241690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.780+0000 7f296ea00640 1 --2- 172.21.15.47:0/3321651043 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a62423f2a0 0x55a624241690 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3aaa132efcfb580f server_cookie=97cc8b5f0c744db6 in_seq=0 out_seq=0 2024-03-21T11:24:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.780+0000 7f294fe00640 10 client.25865 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:24:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.781+0000 7f294fe00640 1 -- 172.21.15.47:0/3321651043 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a62419b800 con 0x55a62423f2a0 2024-03-21T11:24:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.781+0000 7f294fe00640 10 client.25865 handle_command_reply: tid=0 2024-03-21T11:24:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.781+0000 7f297839b740 1 client.25865 shutdown 2024-03-21T11:24:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.782+0000 7f297839b740 1 -- 172.21.15.47:0/3321651043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 msgr2=0x55a624234190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.782+0000 7f297839b740 1 --2- 172.21.15.47:0/3321651043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a624234190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f295c02c250 tx=0x7f295c0044b0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.782+0000 7f297839b740 1 -- 172.21.15.47:0/3321651043 shutdown_connections 2024-03-21T11:24:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.782+0000 7f297839b740 1 --2- 172.21.15.47:0/3321651043 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a62423f2a0 0x55a624241690 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.782+0000 7f297839b740 1 --2- 172.21.15.47:0/3321651043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a62422b580 0x55a62423b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.782+0000 7f297839b740 1 --2- 172.21.15.47:0/3321651043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a624234190 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.782+0000 7f297839b740 1 --2- 172.21.15.47:0/3321651043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6242299f0 0x55a624233c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.782+0000 7f297839b740 1 -- 172.21.15.47:0/3321651043 >> 172.21.15.47:0/3321651043 conn(0x55a624224de0 msgr2=0x55a624224160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.782+0000 7f294fe00640 0 client.25865 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:24:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.782+0000 7f297839b740 1 -- 172.21.15.47:0/3321651043 shutdown_connections 2024-03-21T11:24:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.782+0000 7f297839b740 1 -- 172.21.15.47:0/3321651043 wait complete. 2024-03-21T11:24:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.782+0000 7f297839b740 20 client.25865 trim_cache size 0 max 16384 2024-03-21T11:24:54.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.796+0000 7f297839b740 1 Processor -- start 2024-03-21T11:24:54.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.796+0000 7f297839b740 1 -- start start 2024-03-21T11:24:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.796+0000 7f297839b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6242299f0 0x55a62421fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.796+0000 7f297839b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a62423b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.796+0000 7f297839b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a62422dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.796+0000 7f297839b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29701ac090 con 0x55a6242299f0 2024-03-21T11:24:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.796+0000 7f297839b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29701ada10 con 0x55a62422abc0 2024-03-21T11:24:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.796+0000 7f297839b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29701ac6f0 con 0x55a62422b580 2024-03-21T11:24:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.797+0000 7f296e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a62423b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.797+0000 7f296f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a62422dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.797+0000 7f296f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a62422dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52482/0 (socket says 172.21.15.47:52482) 2024-03-21T11:24:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.797+0000 7f296e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a62423b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40370/0 (socket says 172.21.15.47:40370) 2024-03-21T11:24:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.797+0000 7f296f400640 1 -- 172.21.15.47:0/20250658 learned_addr learned my addr 172.21.15.47:0/20250658 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.797+0000 7f296e000640 1 -- 172.21.15.47:0/20250658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 msgr2=0x55a62422dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.797+0000 7f296e000640 1 --2- 172.21.15.47:0/20250658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a62422dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.797+0000 7f296e000640 1 -- 172.21.15.47:0/20250658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6242299f0 msgr2=0x55a62421fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:54.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.797+0000 7f296ea00640 1 --2- 172.21.15.47:0/20250658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6242299f0 0x55a62421fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.797+0000 7f296e000640 1 --2- 172.21.15.47:0/20250658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6242299f0 0x55a62421fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.797+0000 7f296e000640 1 -- 172.21.15.47:0/20250658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a624225d50 con 0x55a62422abc0 2024-03-21T11:24:54.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.797+0000 7f296ea00640 1 --2- 172.21.15.47:0/20250658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6242299f0 0x55a62421fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:54.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.798+0000 7f296e000640 1 --2- 172.21.15.47:0/20250658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a62423b190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f295c036040 tx=0x7f295c0045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0e67d1f94ab12d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:54.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.798+0000 7f295aa00640 1 -- 172.21.15.47:0/20250658 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c04d070 con 0x55a62422abc0 2024-03-21T11:24:54.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.798+0000 7f295aa00640 1 -- 172.21.15.47:0/20250658 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f295c02cd40 con 0x55a62422abc0 2024-03-21T11:24:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.798+0000 7f295aa00640 1 -- 172.21.15.47:0/20250658 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c052460 con 0x55a62422abc0 2024-03-21T11:24:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.799+0000 7f297839b740 1 -- 172.21.15.47:0/20250658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 msgr2=0x55a62423b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.799+0000 7f297839b740 1 --2- 172.21.15.47:0/20250658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a62423b190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f295c036040 tx=0x7f295c0045f0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.799+0000 7f297839b740 1 -- 172.21.15.47:0/20250658 shutdown_connections 2024-03-21T11:24:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.799+0000 7f297839b740 1 --2- 172.21.15.47:0/20250658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a62422dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.799+0000 7f297839b740 1 --2- 172.21.15.47:0/20250658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a62422abc0 0x55a62423b190 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.799+0000 7f297839b740 1 --2- 172.21.15.47:0/20250658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6242299f0 0x55a62421fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.799+0000 7f297839b740 1 -- 172.21.15.47:0/20250658 >> 172.21.15.47:0/20250658 conn(0x55a624224de0 msgr2=0x55a6242288d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.799+0000 7f297839b740 1 -- 172.21.15.47:0/20250658 shutdown_connections 2024-03-21T11:24:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.799+0000 7f297839b740 1 -- 172.21.15.47:0/20250658 wait complete. 2024-03-21T11:24:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.799+0000 7f297839b740 1 -- start start 2024-03-21T11:24:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.799+0000 7f297839b740 1 Processor -- start 2024-03-21T11:24:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.800+0000 7f297839b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6242299f0 0x55a624237340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.800+0000 7f297839b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a62422abc0 0x55a624237880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.800+0000 7f297839b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a62423a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.800+0000 7f297839b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29701ad3b0 con 0x55a62422abc0 2024-03-21T11:24:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.800+0000 7f297839b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29701acd50 con 0x55a6242299f0 2024-03-21T11:24:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.800+0000 7f297839b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29701aa250 con 0x55a62422b580 2024-03-21T11:24:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.800+0000 7f297839b740 10 client.0 _ll_register_callbacks cb 0x7f295c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:54.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.800+0000 7f296e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a62422abc0 0x55a624237880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.800+0000 7f296f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a62423a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.800+0000 7f296e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a62422abc0 0x55a624237880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52978/0 (socket says 172.21.15.47:52978) 2024-03-21T11:24:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.800+0000 7f296e000640 1 -- 172.21.15.47:0/969676452 learned_addr learned my addr 172.21.15.47:0/969676452 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.800+0000 7f296ea00640 1 --2- 172.21.15.47:0/969676452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6242299f0 0x55a624237340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.801+0000 7f296f400640 1 -- 172.21.15.47:0/969676452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6242299f0 msgr2=0x55a624237340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.801+0000 7f296f400640 1 --2- 172.21.15.47:0/969676452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6242299f0 0x55a624237340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.801+0000 7f296f400640 1 -- 172.21.15.47:0/969676452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a62422abc0 msgr2=0x55a624237880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.801+0000 7f296f400640 1 --2- 172.21.15.47:0/969676452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a62422abc0 0x55a624237880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.801+0000 7f296f400640 1 -- 172.21.15.47:0/969676452 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a62423ad60 con 0x55a62422b580 2024-03-21T11:24:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.801+0000 7f296e000640 1 --2- 172.21.15.47:0/969676452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a62422abc0 0x55a624237880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.801+0000 7f296f400640 1 --2- 172.21.15.47:0/969676452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a62423a770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2968009fd0 tx=0x7f2968004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fdb8dc2af033cb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.801+0000 7f294fe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.801+0000 7f294fe00640 1 -- 172.21.15.47:0/969676452 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296802d060 con 0x55a62422b580 2024-03-21T11:24:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.801+0000 7f297839b740 1 -- 172.21.15.47:0/969676452 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f295c02cd40 con 0x55a62422b580 2024-03-21T11:24:54.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.801+0000 7f294fe00640 1 -- 172.21.15.47:0/969676452 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2968003b70 con 0x55a62422b580 2024-03-21T11:24:54.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.802+0000 7f294fe00640 1 -- 172.21.15.47:0/969676452 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2968034050 con 0x55a62422b580 2024-03-21T11:24:54.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.802+0000 7f297839b740 10 client.25897 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:54.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.802+0000 7f297839b740 1 -- 172.21.15.47:0/969676452 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a62429b3a0 con 0x55a62422b580 2024-03-21T11:24:54.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.802+0000 7f294fe00640 1 -- 172.21.15.47:0/969676452 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f295c02cd40 con 0x55a62422b580 2024-03-21T11:24:54.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.803+0000 7f294fe00640 1 -- 172.21.15.47:0/969676452 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2968019040 con 0x55a62422b580 2024-03-21T11:24:54.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.803+0000 7f297839b740 10 client.25897 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:54.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.803+0000 7f297839b740 10 client.25897 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:24:54.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.803+0000 7f297839b740 20 client.25897 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:54.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.803+0000 7f297839b740 1 --2- 172.21.15.47:0/969676452 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a62429bd00 0x55a62429e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.803+0000 7f297839b740 4 client.25897 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:24:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.803+0000 7f297839b740 1 -- 172.21.15.47:0/969676452 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a624195b50 con 0x55a62429bd00 2024-03-21T11:24:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.804+0000 7f296ea00640 1 --2- 172.21.15.47:0/969676452 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a62429bd00 0x55a62429e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.804+0000 7f296ea00640 1 --2- 172.21.15.47:0/969676452 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a62429bd00 0x55a62429e120 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e92e35e98cf5d351 server_cookie=b63c517a8b240507 in_seq=0 out_seq=0 2024-03-21T11:24:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.804+0000 7f294fe00640 10 client.25897 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:24:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.804+0000 7f294fe00640 1 -- 172.21.15.47:0/969676452 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a624195b50 con 0x55a62429bd00 2024-03-21T11:24:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.804+0000 7f294fe00640 10 client.25897 handle_command_reply: tid=0 2024-03-21T11:24:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.804+0000 7f297839b740 1 client.25897 shutdown 2024-03-21T11:24:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.804+0000 7f297839b740 1 -- 172.21.15.47:0/969676452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 msgr2=0x55a62423a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.804+0000 7f297839b740 1 --2- 172.21.15.47:0/969676452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a62423a770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2968009fd0 tx=0x7f2968004a40 comp rx=0 tx=0).stop 2024-03-21T11:24:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.805+0000 7f297839b740 1 -- 172.21.15.47:0/969676452 shutdown_connections 2024-03-21T11:24:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.805+0000 7f297839b740 1 --2- 172.21.15.47:0/969676452 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a62429bd00 0x55a62429e120 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.805+0000 7f297839b740 1 --2- 172.21.15.47:0/969676452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a62422b580 0x55a62423a770 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.805+0000 7f297839b740 1 --2- 172.21.15.47:0/969676452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a62422abc0 0x55a624237880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.805+0000 7f297839b740 1 --2- 172.21.15.47:0/969676452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6242299f0 0x55a624237340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.805+0000 7f294fe00640 0 client.25897 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:24:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.805+0000 7f297839b740 1 -- 172.21.15.47:0/969676452 >> 172.21.15.47:0/969676452 conn(0x55a624224de0 msgr2=0x55a6242238d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.805+0000 7f297839b740 1 -- 172.21.15.47:0/969676452 shutdown_connections 2024-03-21T11:24:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.805+0000 7f297839b740 1 -- 172.21.15.47:0/969676452 wait complete. 2024-03-21T11:24:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.805+0000 7f297839b740 20 client.25897 trim_cache size 0 max 16384 2024-03-21T11:24:54.809 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:24:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.805+0000 7f296fe00640 1 -- 172.21.15.47:0/2416787039 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f29540a2fa0 msgr2=0x7f29540a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.805+0000 7f296fe00640 1 --2- 172.21.15.47:0/2416787039 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f29540a2fa0 0x7f29540a5460 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55a62422aac0 tx=0x7f2968009500 comp rx=0 tx=0).stop 2024-03-21T11:24:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.806+0000 7f296fe00640 1 -- 172.21.15.47:0/2416787039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29701015a0 msgr2=0x7f2970199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.806+0000 7f296fe00640 1 --2- 172.21.15.47:0/2416787039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29701015a0 0x7f2970199b20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f296000e9c0 tx=0x7f296000ee80 comp rx=0 tx=0).stop 2024-03-21T11:24:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.806+0000 7f296fe00640 1 -- 172.21.15.47:0/2416787039 shutdown_connections 2024-03-21T11:24:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.806+0000 7f296fe00640 1 --2- 172.21.15.47:0/2416787039 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f29540a2fa0 0x7f29540a5460 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.806+0000 7f296fe00640 1 --2- 172.21.15.47:0/2416787039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2970103150 0x7f29701a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.806+0000 7f296fe00640 1 --2- 172.21.15.47:0/2416787039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2970102790 0x7f297019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.806+0000 7f296fe00640 1 --2- 172.21.15.47:0/2416787039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29701015a0 0x7f2970199b20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.806+0000 7f296fe00640 1 -- 172.21.15.47:0/2416787039 >> 172.21.15.47:0/2416787039 conn(0x7f29700fcb10 msgr2=0x7f29700738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.806+0000 7f296fe00640 1 -- 172.21.15.47:0/2416787039 shutdown_connections 2024-03-21T11:24:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:54.806+0000 7f296fe00640 1 -- 172.21.15.47:0/2416787039 wait complete. 2024-03-21T11:24:54.823 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:24:54.823 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:24:55.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.003+0000 7efe6a200640 1 -- 172.21.15.47:0/3401312525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe641015a0 msgr2=0x7efe641019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.003+0000 7efe6a200640 1 --2- 172.21.15.47:0/3401312525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe641015a0 0x7efe641019c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efe58009900 tx=0x7efe5802bc30 comp rx=0 tx=0).stop 2024-03-21T11:24:55.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.003+0000 7efe6a200640 1 -- 172.21.15.47:0/3401312525 shutdown_connections 2024-03-21T11:24:55.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.003+0000 7efe6a200640 1 --2- 172.21.15.47:0/3401312525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe64103150 0x7efe641099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.003+0000 7efe6a200640 1 --2- 172.21.15.47:0/3401312525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe64102790 0x7efe64102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.003+0000 7efe6a200640 1 --2- 172.21.15.47:0/3401312525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe641015a0 0x7efe641019c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.003+0000 7efe6a200640 1 -- 172.21.15.47:0/3401312525 >> 172.21.15.47:0/3401312525 conn(0x7efe640fcb10 msgr2=0x7efe640fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.004+0000 7efe6a200640 1 -- 172.21.15.47:0/3401312525 shutdown_connections 2024-03-21T11:24:55.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.004+0000 7efe6a200640 1 -- 172.21.15.47:0/3401312525 wait complete. 2024-03-21T11:24:55.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.005+0000 7efe6a200640 1 Processor -- start 2024-03-21T11:24:55.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.005+0000 7efe6a200640 1 -- start start 2024-03-21T11:24:55.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.005+0000 7efe6a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe641015a0 0x7efe64199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.005+0000 7efe6a200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe64102790 0x7efe6419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.005+0000 7efe6a200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe64103150 0x7efe641a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.005+0000 7efe6a200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6410d230 con 0x7efe641015a0 2024-03-21T11:24:55.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.005+0000 7efe6a200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6410d0b0 con 0x7efe64103150 2024-03-21T11:24:55.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.005+0000 7efe6a200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6410d3b0 con 0x7efe64102790 2024-03-21T11:24:55.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.005+0000 7efe63e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe64102790 0x7efe6419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.005+0000 7efe69800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe64103150 0x7efe641a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.005+0000 7efe63e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe64102790 0x7efe6419a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52512/0 (socket says 172.21.15.47:52512) 2024-03-21T11:24:55.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.005+0000 7efe63e00640 1 -- 172.21.15.47:0/154213044 learned_addr learned my addr 172.21.15.47:0/154213044 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.006+0000 7efe68e00640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe641015a0 0x7efe64199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.006+0000 7efe63e00640 1 -- 172.21.15.47:0/154213044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe64103150 msgr2=0x7efe641a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.006+0000 7efe63e00640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe64103150 0x7efe641a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.006+0000 7efe63e00640 1 -- 172.21.15.47:0/154213044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe641015a0 msgr2=0x7efe64199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.006+0000 7efe63e00640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe641015a0 0x7efe64199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.006+0000 7efe63e00640 1 -- 172.21.15.47:0/154213044 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe641a16e0 con 0x7efe64102790 2024-03-21T11:24:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.006+0000 7efe69800640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe64103150 0x7efe641a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.006+0000 7efe68e00640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe641015a0 0x7efe64199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.006+0000 7efe63e00640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe64102790 0x7efe6419a000 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efe5000e9f0 tx=0x7efe5000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.007+0000 7efe61600640 1 -- 172.21.15.47:0/154213044 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5000cd80 con 0x7efe64102790 2024-03-21T11:24:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.007+0000 7efe61600640 1 -- 172.21.15.47:0/154213044 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe50005500 con 0x7efe64102790 2024-03-21T11:24:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.007+0000 7efe6a200640 1 -- 172.21.15.47:0/154213044 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe641a19d0 con 0x7efe64102790 2024-03-21T11:24:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.007+0000 7efe61600640 1 -- 172.21.15.47:0/154213044 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe50010690 con 0x7efe64102790 2024-03-21T11:24:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.007+0000 7efe6a200640 1 -- 172.21.15.47:0/154213044 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe641a1f20 con 0x7efe64102790 2024-03-21T11:24:55.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.012+0000 7efe6e3f2740 1 Processor -- start 2024-03-21T11:24:55.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.012+0000 7efe6e3f2740 1 -- start start 2024-03-21T11:24:55.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.013+0000 7efe6e3f2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8d9f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.013+0000 7efe6e3f2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da03d0 0x558dd8da0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.013+0000 7efe6e3f2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da0d90 0x558dd8da7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.013+0000 7efe6e3f2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd8cfb2c0 con 0x558dd8d9f200 2024-03-21T11:24:55.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.013+0000 7efe6e3f2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd8ca3c90 con 0x558dd8da0d90 2024-03-21T11:24:55.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.013+0000 7efe6e3f2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd8cf7360 con 0x558dd8da03d0 2024-03-21T11:24:55.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.013+0000 7efe61600640 1 -- 172.21.15.47:0/154213044 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7efe500046c0 con 0x7efe64102790 2024-03-21T11:24:55.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.013+0000 7efe68e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da03d0 0x558dd8da0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.013+0000 7efe68e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da03d0 0x558dd8da0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52518/0 (socket says 172.21.15.47:52518) 2024-03-21T11:24:55.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.013+0000 7efe63e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da0d90 0x558dd8da7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.013+0000 7efe68e00640 1 -- 172.21.15.47:0/1878011983 learned_addr learned my addr 172.21.15.47:0/1878011983 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.013+0000 7efe69800640 1 --2- 172.21.15.47:0/1878011983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8d9f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.014+0000 7efe63e00640 1 -- 172.21.15.47:0/1878011983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da03d0 msgr2=0x558dd8da0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.014+0000 7efe63e00640 1 --2- 172.21.15.47:0/1878011983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da03d0 0x558dd8da0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.014+0000 7efe63e00640 1 -- 172.21.15.47:0/1878011983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 msgr2=0x558dd8d9f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.014+0000 7efe63e00640 1 --2- 172.21.15.47:0/1878011983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8d9f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.014+0000 7efe63e00640 1 -- 172.21.15.47:0/1878011983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dd8cc8080 con 0x558dd8da0d90 2024-03-21T11:24:55.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.014+0000 7efe63e00640 1 --2- 172.21.15.47:0/1878011983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da0d90 0x558dd8da7620 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efe500cd2c0 tx=0x7efe500d3bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e0f2367cf80d153 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.015+0000 7efe56000640 1 -- 172.21.15.47:0/1878011983 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe500e0020 con 0x558dd8da0d90 2024-03-21T11:24:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.015+0000 7efe56000640 1 -- 172.21.15.47:0/1878011983 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe500d5050 con 0x558dd8da0d90 2024-03-21T11:24:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.015+0000 7efe56000640 1 -- 172.21.15.47:0/1878011983 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe500db3a0 con 0x558dd8da0d90 2024-03-21T11:24:55.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.015+0000 7efe61600640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe4c0a3020 0x7efe4c0a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.015+0000 7efe61600640 1 -- 172.21.15.47:0/154213044 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efe500cfc10 con 0x7efe64102790 2024-03-21T11:24:55.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.016+0000 7efe6e3f2740 1 -- 172.21.15.47:0/1878011983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da0d90 msgr2=0x558dd8da7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.016+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/1878011983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da0d90 0x558dd8da7620 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efe500cd2c0 tx=0x7efe500d3bd0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.016+0000 7efe69800640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe4c0a3020 0x7efe4c0a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.017+0000 7efe69800640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe4c0a3020 0x7efe4c0a54e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009fd0 tx=0x7efe5c0096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.017+0000 7efe6e3f2740 1 -- 172.21.15.47:0/1878011983 shutdown_connections 2024-03-21T11:24:55.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.017+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/1878011983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da0d90 0x558dd8da7620 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.017+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/1878011983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da03d0 0x558dd8da0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.017+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/1878011983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8d9f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.017+0000 7efe6e3f2740 1 -- 172.21.15.47:0/1878011983 >> 172.21.15.47:0/1878011983 conn(0x558dd8d9a5f0 msgr2=0x558dd8d9a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.018+0000 7efe6e3f2740 1 -- 172.21.15.47:0/1878011983 shutdown_connections 2024-03-21T11:24:55.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.018+0000 7efe6e3f2740 1 -- 172.21.15.47:0/1878011983 wait complete. 2024-03-21T11:24:55.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.019+0000 7efe6e3f2740 1 -- start start 2024-03-21T11:24:55.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.019+0000 7efe6e3f2740 1 Processor -- start 2024-03-21T11:24:55.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.020+0000 7efe6e3f2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8da94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.020+0000 7efe6e3f2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da03d0 0x558dd8da9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.020+0000 7efe6e3f2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da0d90 0x558dd8db09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.020+0000 7efe6e3f2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd8da7da0 con 0x558dd8d9f200 2024-03-21T11:24:55.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.020+0000 7efe6e3f2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd8cb70d0 con 0x558dd8da03d0 2024-03-21T11:24:55.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.020+0000 7efe6e3f2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd8da7f20 con 0x558dd8da0d90 2024-03-21T11:24:55.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.020+0000 7efe6e3f2740 10 client.0 _ll_register_callbacks cb 0x558dd8cbbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:55.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.020+0000 7efe68e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8da94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.020+0000 7efe68e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8da94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53012/0 (socket says 172.21.15.47:53012) 2024-03-21T11:24:55.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.020+0000 7efe68e00640 1 -- 172.21.15.47:0/2077081264 learned_addr learned my addr 172.21.15.47:0/2077081264 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.020+0000 7efe63e00640 1 --2- 172.21.15.47:0/2077081264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da03d0 0x558dd8da9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.020+0000 7efe69800640 1 --2- 172.21.15.47:0/2077081264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da0d90 0x558dd8db09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.021+0000 7efe69800640 1 -- 172.21.15.47:0/2077081264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da03d0 msgr2=0x558dd8da9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.021+0000 7efe69800640 1 --2- 172.21.15.47:0/2077081264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da03d0 0x558dd8da9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.021+0000 7efe69800640 1 -- 172.21.15.47:0/2077081264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 msgr2=0x558dd8da94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.021+0000 7efe69800640 1 --2- 172.21.15.47:0/2077081264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8da94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.021+0000 7efe69800640 1 -- 172.21.15.47:0/2077081264 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dd8a0f3f0 con 0x558dd8da0d90 2024-03-21T11:24:55.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.021+0000 7efe68e00640 1 --2- 172.21.15.47:0/2077081264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8da94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:55.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.021+0000 7efe69800640 1 --2- 172.21.15.47:0/2077081264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da0d90 0x558dd8db09e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efe5c005fd0 tx=0x7efe5c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=465a74401fa3a943 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.021+0000 7efe4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:55.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.021+0000 7efe4aa00640 1 -- 172.21.15.47:0/2077081264 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c01dd30 con 0x558dd8da0d90 2024-03-21T11:24:55.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.022+0000 7efe4aa00640 1 -- 172.21.15.47:0/2077081264 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efe5c029480 con 0x558dd8da0d90 2024-03-21T11:24:55.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.021+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2077081264 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe500d5050 con 0x558dd8da0d90 2024-03-21T11:24:55.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.022+0000 7efe4aa00640 1 -- 172.21.15.47:0/2077081264 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe5c029680 con 0x558dd8da0d90 2024-03-21T11:24:55.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.022+0000 7efe6e3f2740 10 client.25909 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:55.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.022+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2077081264 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558dd8ad97d0 con 0x558dd8da0d90 2024-03-21T11:24:55.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.022+0000 7efe4aa00640 1 -- 172.21.15.47:0/2077081264 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe500d5050 con 0x558dd8da0d90 2024-03-21T11:24:55.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.022+0000 7efe4aa00640 1 -- 172.21.15.47:0/2077081264 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efe5c01b5f0 con 0x558dd8da0d90 2024-03-21T11:24:55.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.023+0000 7efe6e3f2740 10 client.25909 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:55.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.023+0000 7efe6e3f2740 10 client.25909 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:24:55.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.023+0000 7efe6e3f2740 20 client.25909 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:55.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.023+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/2077081264 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558dd8db4fc0 0x558dd8db7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.023+0000 7efe6e3f2740 4 client.25909 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:55.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.023+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2077081264 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dd8d08800 con 0x558dd8db4fc0 2024-03-21T11:24:55.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.023+0000 7efe68e00640 1 --2- 172.21.15.47:0/2077081264 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558dd8db4fc0 0x558dd8db7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.023+0000 7efe68e00640 1 --2- 172.21.15.47:0/2077081264 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558dd8db4fc0 0x558dd8db7420 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b8d79c4dc351e357 server_cookie=1a8baf15a6f93b3e in_seq=0 out_seq=0 2024-03-21T11:24:55.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.023+0000 7efe4aa00640 10 client.25909 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:24:55.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.024+0000 7efe4aa00640 1 -- 172.21.15.47:0/2077081264 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558dd8d08800 con 0x558dd8db4fc0 2024-03-21T11:24:55.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.024+0000 7efe4aa00640 10 client.25909 handle_command_reply: tid=0 2024-03-21T11:24:55.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.024+0000 7efe6e3f2740 1 client.25909 shutdown 2024-03-21T11:24:55.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.025+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2077081264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da0d90 msgr2=0x558dd8db09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.025+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/2077081264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da0d90 0x558dd8db09e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efe5c005fd0 tx=0x7efe5c004a40 comp rx=0 tx=0).stop 2024-03-21T11:24:55.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.025+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2077081264 shutdown_connections 2024-03-21T11:24:55.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.025+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/2077081264 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558dd8db4fc0 0x558dd8db7420 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.025+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/2077081264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da0d90 0x558dd8db09e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.025+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/2077081264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da03d0 0x558dd8da9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.025+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/2077081264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8da94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.025+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2077081264 >> 172.21.15.47:0/2077081264 conn(0x558dd8d9a5f0 msgr2=0x558dd8d9d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.025+0000 7efe4aa00640 0 client.25909 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:24:55.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.025+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2077081264 shutdown_connections 2024-03-21T11:24:55.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.025+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2077081264 wait complete. 2024-03-21T11:24:55.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.025+0000 7efe6e3f2740 20 client.25909 trim_cache size 0 max 16384 2024-03-21T11:24:55.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.051+0000 7efe6e3f2740 1 Processor -- start 2024-03-21T11:24:55.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.051+0000 7efe6e3f2740 1 -- start start 2024-03-21T11:24:55.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.051+0000 7efe6e3f2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8d9f200 0x558dd8d91160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.051+0000 7efe6e3f2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da03d0 0x558dd8db09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe6e3f2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8da0d90 0x558dd8da89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe6e3f2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe641a7d30 con 0x558dd8da0d90 2024-03-21T11:24:55.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe6e3f2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe641a96b0 con 0x558dd8d9f200 2024-03-21T11:24:55.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe6e3f2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe641a8390 con 0x558dd8da03d0 2024-03-21T11:24:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe63e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da03d0 0x558dd8db09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe63e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da03d0 0x558dd8db09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52546/0 (socket says 172.21.15.47:52546) 2024-03-21T11:24:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe63e00640 1 -- 172.21.15.47:0/3002887217 learned_addr learned my addr 172.21.15.47:0/3002887217 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe68e00640 1 --2- 172.21.15.47:0/3002887217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8d9f200 0x558dd8d91160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe69800640 1 --2- 172.21.15.47:0/3002887217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8da0d90 0x558dd8da89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe63e00640 1 -- 172.21.15.47:0/3002887217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8d9f200 msgr2=0x558dd8d91160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe63e00640 1 --2- 172.21.15.47:0/3002887217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8d9f200 0x558dd8d91160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe63e00640 1 -- 172.21.15.47:0/3002887217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8da0d90 msgr2=0x558dd8da89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe63e00640 1 --2- 172.21.15.47:0/3002887217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8da0d90 0x558dd8da89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe63e00640 1 -- 172.21.15.47:0/3002887217 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dd8d9fe50 con 0x558dd8da03d0 2024-03-21T11:24:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.052+0000 7efe69800640 1 --2- 172.21.15.47:0/3002887217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8da0d90 0x558dd8da89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.053+0000 7efe63e00640 1 --2- 172.21.15.47:0/3002887217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da03d0 0x558dd8db09a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efe500049d0 tx=0x7efe500d34b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=559d223ea7f33ffb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.053+0000 7efe55600640 1 -- 172.21.15.47:0/3002887217 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe500e0070 con 0x558dd8da03d0 2024-03-21T11:24:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.053+0000 7efe55600640 1 -- 172.21.15.47:0/3002887217 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe5009c650 con 0x558dd8da03d0 2024-03-21T11:24:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.053+0000 7efe55600640 1 -- 172.21.15.47:0/3002887217 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5009b8b0 con 0x558dd8da03d0 2024-03-21T11:24:55.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.054+0000 7efe6e3f2740 1 -- 172.21.15.47:0/3002887217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da03d0 msgr2=0x558dd8db09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.054+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/3002887217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da03d0 0x558dd8db09a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efe500049d0 tx=0x7efe500d34b0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.054+0000 7efe6e3f2740 1 -- 172.21.15.47:0/3002887217 shutdown_connections 2024-03-21T11:24:55.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.054+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/3002887217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8da0d90 0x558dd8da89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.054+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/3002887217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8da03d0 0x558dd8db09a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.054+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/3002887217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8d9f200 0x558dd8d91160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.054+0000 7efe6e3f2740 1 -- 172.21.15.47:0/3002887217 >> 172.21.15.47:0/3002887217 conn(0x558dd8d9a5f0 msgr2=0x558dd8da82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.054+0000 7efe6e3f2740 1 -- 172.21.15.47:0/3002887217 shutdown_connections 2024-03-21T11:24:55.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.054+0000 7efe6e3f2740 1 -- 172.21.15.47:0/3002887217 wait complete. 2024-03-21T11:24:55.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.055+0000 7efe6e3f2740 1 -- start start 2024-03-21T11:24:55.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.055+0000 7efe6e3f2740 1 Processor -- start 2024-03-21T11:24:55.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.056+0000 7efe6e3f2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8dae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.056+0000 7efe6e3f2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da0d90 0x558dd8daedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.056+0000 7efe6e3f2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8dafe00 0x558dd8db0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.056+0000 7efe6e3f2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe641a9050 con 0x558dd8d9f200 2024-03-21T11:24:55.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.056+0000 7efe6e3f2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe641a89f0 con 0x558dd8da0d90 2024-03-21T11:24:55.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.056+0000 7efe6e3f2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe641a5f30 con 0x558dd8dafe00 2024-03-21T11:24:55.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.056+0000 7efe6e3f2740 10 client.0 _ll_register_callbacks cb 0x7efe5c010b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:55.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.056+0000 7efe63e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da0d90 0x558dd8daedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.056+0000 7efe63e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da0d90 0x558dd8daedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40436/0 (socket says 172.21.15.47:40436) 2024-03-21T11:24:55.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.056+0000 7efe63e00640 1 -- 172.21.15.47:0/2349798887 learned_addr learned my addr 172.21.15.47:0/2349798887 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.056+0000 7efe68e00640 1 --2- 172.21.15.47:0/2349798887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8dae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.056+0000 7efe69800640 1 --2- 172.21.15.47:0/2349798887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8dafe00 0x558dd8db0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.057+0000 7efe69800640 1 -- 172.21.15.47:0/2349798887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da0d90 msgr2=0x558dd8daedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.057+0000 7efe69800640 1 --2- 172.21.15.47:0/2349798887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da0d90 0x558dd8daedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.057+0000 7efe69800640 1 -- 172.21.15.47:0/2349798887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 msgr2=0x558dd8dae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.057+0000 7efe69800640 1 --2- 172.21.15.47:0/2349798887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8dae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.057+0000 7efe69800640 1 -- 172.21.15.47:0/2349798887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dd8db07c0 con 0x558dd8dafe00 2024-03-21T11:24:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.057+0000 7efe68e00640 1 --2- 172.21.15.47:0/2349798887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8dae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.057+0000 7efe69800640 1 --2- 172.21.15.47:0/2349798887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8dafe00 0x558dd8db0280 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efe5c017040 tx=0x7efe5c01dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e9e95784055a875 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.057+0000 7efe4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.058+0000 7efe4aa00640 1 -- 172.21.15.47:0/2349798887 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c03b020 con 0x558dd8dafe00 2024-03-21T11:24:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.058+0000 7efe4aa00640 1 -- 172.21.15.47:0/2349798887 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efe5c004cc0 con 0x558dd8dafe00 2024-03-21T11:24:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.058+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2349798887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe5009c650 con 0x558dd8dafe00 2024-03-21T11:24:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.058+0000 7efe4aa00640 1 -- 172.21.15.47:0/2349798887 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe5c046050 con 0x558dd8dafe00 2024-03-21T11:24:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.058+0000 7efe6e3f2740 10 client.25915 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.058+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2349798887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558dd8e11500 con 0x558dd8dafe00 2024-03-21T11:24:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.058+0000 7efe4aa00640 1 -- 172.21.15.47:0/2349798887 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe5009c650 con 0x558dd8dafe00 2024-03-21T11:24:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.059+0000 7efe4aa00640 1 -- 172.21.15.47:0/2349798887 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efe5c005040 con 0x558dd8dafe00 2024-03-21T11:24:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.059+0000 7efe6e3f2740 10 client.25915 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:55.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.059+0000 7efe6e3f2740 10 client.25915 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:24:55.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.059+0000 7efe6e3f2740 20 client.25915 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:55.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.059+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/2349798887 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558dd8e11eb0 0x558dd8e142d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.059+0000 7efe6e3f2740 4 client.25915 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:24:55.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.059+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2349798887 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558dd8d02b50 con 0x558dd8e11eb0 2024-03-21T11:24:55.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.059+0000 7efe68e00640 1 --2- 172.21.15.47:0/2349798887 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558dd8e11eb0 0x558dd8e142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe68e00640 1 --2- 172.21.15.47:0/2349798887 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558dd8e11eb0 0x558dd8e142d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6187020dc7152caf server_cookie=c72ee2ea11aeb271 in_seq=0 out_seq=0 2024-03-21T11:24:55.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe4aa00640 10 client.25915 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:24:55.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe4aa00640 1 -- 172.21.15.47:0/2349798887 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558dd8d02b50 con 0x558dd8e11eb0 2024-03-21T11:24:55.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe4aa00640 10 client.25915 handle_command_reply: tid=0 2024-03-21T11:24:55.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe6e3f2740 1 client.25915 shutdown 2024-03-21T11:24:55.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2349798887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8dafe00 msgr2=0x558dd8db0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/2349798887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8dafe00 0x558dd8db0280 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efe5c017040 tx=0x7efe5c01dd40 comp rx=0 tx=0).stop 2024-03-21T11:24:55.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2349798887 shutdown_connections 2024-03-21T11:24:55.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/2349798887 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558dd8e11eb0 0x558dd8e142d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe4aa00640 0 client.25915 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:24:55.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/2349798887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dd8dafe00 0x558dd8db0280 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/2349798887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558dd8da0d90 0x558dd8daedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe6e3f2740 1 --2- 172.21.15.47:0/2349798887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558dd8d9f200 0x558dd8dae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2349798887 >> 172.21.15.47:0/2349798887 conn(0x558dd8d9a5f0 msgr2=0x558dd8d90fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.060+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2349798887 shutdown_connections 2024-03-21T11:24:55.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.061+0000 7efe6e3f2740 1 -- 172.21.15.47:0/2349798887 wait complete. 2024-03-21T11:24:55.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.061+0000 7efe6e3f2740 20 client.25915 trim_cache size 0 max 16384 2024-03-21T11:24:55.066 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:24:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.061+0000 7efe6a200640 1 -- 172.21.15.47:0/154213044 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe4c0a3020 msgr2=0x7efe4c0a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.061+0000 7efe6a200640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe4c0a3020 0x7efe4c0a54e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009fd0 tx=0x7efe5c0096a0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.062+0000 7efe6a200640 1 -- 172.21.15.47:0/154213044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe64102790 msgr2=0x7efe6419a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.062+0000 7efe6a200640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe64102790 0x7efe6419a000 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efe5000e9f0 tx=0x7efe5000edf0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.063+0000 7efe6a200640 1 -- 172.21.15.47:0/154213044 shutdown_connections 2024-03-21T11:24:55.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.063+0000 7efe6a200640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efe4c0a3020 0x7efe4c0a54e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.063+0000 7efe6a200640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe64103150 0x7efe641a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.063+0000 7efe6a200640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe64102790 0x7efe6419a000 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.063+0000 7efe6a200640 1 --2- 172.21.15.47:0/154213044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe641015a0 0x7efe64199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.063+0000 7efe6a200640 1 -- 172.21.15.47:0/154213044 >> 172.21.15.47:0/154213044 conn(0x7efe640fcb10 msgr2=0x7efe640738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.063+0000 7efe6a200640 1 -- 172.21.15.47:0/154213044 shutdown_connections 2024-03-21T11:24:55.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.063+0000 7efe6a200640 1 -- 172.21.15.47:0/154213044 wait complete. 2024-03-21T11:24:55.088 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:24:55.088 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:24:55.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.303+0000 7f49f1200640 1 -- 172.21.15.47:0/428738280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49ec102000 msgr2=0x7f49ec1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.303+0000 7f49f1200640 1 --2- 172.21.15.47:0/428738280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49ec102000 0x7f49ec1043f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009900 tx=0x7f49dc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:24:55.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.303+0000 7f49f1200640 1 -- 172.21.15.47:0/428738280 shutdown_connections 2024-03-21T11:24:55.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.303+0000 7f49f1200640 1 --2- 172.21.15.47:0/428738280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49ec107260 0x7f49ec1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.303+0000 7f49f1200640 1 --2- 172.21.15.47:0/428738280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49ec104930 0x7f49ec106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.303+0000 7f49f1200640 1 --2- 172.21.15.47:0/428738280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49ec102000 0x7f49ec1043f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.303+0000 7f49f1200640 1 -- 172.21.15.47:0/428738280 >> 172.21.15.47:0/428738280 conn(0x7f49ec0fba10 msgr2=0x7f49ec0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.303+0000 7f49f1200640 1 -- 172.21.15.47:0/428738280 shutdown_connections 2024-03-21T11:24:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.303+0000 7f49f1200640 1 -- 172.21.15.47:0/428738280 wait complete. 2024-03-21T11:24:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.304+0000 7f49f1200640 1 Processor -- start 2024-03-21T11:24:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.304+0000 7f49f1200640 1 -- start start 2024-03-21T11:24:55.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.305+0000 7f49f1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49ec102000 0x7f49ec199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.305+0000 7f49f1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49ec104930 0x7f49ec19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.305+0000 7f49f1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49ec107260 0x7f49ec1a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.305+0000 7f49f1200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec10d260 con 0x7f49ec102000 2024-03-21T11:24:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.305+0000 7f49f1200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec10d0e0 con 0x7f49ec104930 2024-03-21T11:24:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.305+0000 7f49f1200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec10d3e0 con 0x7f49ec107260 2024-03-21T11:24:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.305+0000 7f49eb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49ec102000 0x7f49ec199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.305+0000 7f49eb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49ec102000 0x7f49ec199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53054/0 (socket says 172.21.15.47:53054) 2024-03-21T11:24:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.305+0000 7f49eb400640 1 -- 172.21.15.47:0/3571506770 learned_addr learned my addr 172.21.15.47:0/3571506770 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.305+0000 7f49ebe00640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49ec107260 0x7f49ec1a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.305+0000 7f49eaa00640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49ec104930 0x7f49ec19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.306+0000 7f49ebe00640 1 -- 172.21.15.47:0/3571506770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49ec104930 msgr2=0x7f49ec19a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.306+0000 7f49ebe00640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49ec104930 0x7f49ec19a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.306+0000 7f49ebe00640 1 -- 172.21.15.47:0/3571506770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49ec102000 msgr2=0x7f49ec199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.306+0000 7f49ebe00640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49ec102000 0x7f49ec199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.306+0000 7f49ebe00640 1 -- 172.21.15.47:0/3571506770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec1a16f0 con 0x7f49ec107260 2024-03-21T11:24:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.306+0000 7f49eb400640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49ec102000 0x7f49ec199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.306+0000 7f49ebe00640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49ec107260 0x7f49ec1a1080 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009fd0 tx=0x7f49e400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.306+0000 7f49e3e00640 1 -- 172.21.15.47:0/3571506770 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e400ce20 con 0x7f49ec107260 2024-03-21T11:24:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.306+0000 7f49e3e00640 1 -- 172.21.15.47:0/3571506770 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49e4005020 con 0x7f49ec107260 2024-03-21T11:24:55.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.306+0000 7f49f1200640 1 -- 172.21.15.47:0/3571506770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ec1a19e0 con 0x7f49ec107260 2024-03-21T11:24:55.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.306+0000 7f49e3e00640 1 -- 172.21.15.47:0/3571506770 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e4010720 con 0x7f49ec107260 2024-03-21T11:24:55.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.306+0000 7f49f1200640 1 -- 172.21.15.47:0/3571506770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ec1a1f30 con 0x7f49ec107260 2024-03-21T11:24:55.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.310+0000 7f49f5427740 1 Processor -- start 2024-03-21T11:24:55.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.311+0000 7f49e3e00640 1 -- 172.21.15.47:0/3571506770 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f49e4005540 con 0x7f49ec107260 2024-03-21T11:24:55.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.311+0000 7f49f5427740 1 -- start start 2024-03-21T11:24:55.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.311+0000 7f49f5427740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f79f0 0x5632126f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.311+0000 7f49f5427740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f8bc0 0x5632126f9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.311+0000 7f49f5427740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f9580 0x5632126ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.311+0000 7f49f5427740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56321265c2c0 con 0x5632126f8bc0 2024-03-21T11:24:55.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.311+0000 7f49f5427740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563212604c90 con 0x5632126f9580 2024-03-21T11:24:55.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.311+0000 7f49f5427740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563212658360 con 0x5632126f79f0 2024-03-21T11:24:55.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.312+0000 7f49eaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f79f0 0x5632126f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.312+0000 7f49eaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f79f0 0x5632126f7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52586/0 (socket says 172.21.15.47:52586) 2024-03-21T11:24:55.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.312+0000 7f49eaa00640 1 -- 172.21.15.47:0/1533983863 learned_addr learned my addr 172.21.15.47:0/1533983863 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.312+0000 7f49eb400640 1 --2- 172.21.15.47:0/1533983863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f8bc0 0x5632126f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.312+0000 7f49eaa00640 1 --2- 172.21.15.47:0/1533983863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f9580 0x5632126ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.312+0000 7f49eaa00640 1 -- 172.21.15.47:0/1533983863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f9580 msgr2=0x5632126ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.312+0000 7f49eaa00640 1 --2- 172.21.15.47:0/1533983863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f9580 0x5632126ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.312+0000 7f49eaa00640 1 -- 172.21.15.47:0/1533983863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f8bc0 msgr2=0x5632126f9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.312+0000 7f49eaa00640 1 --2- 172.21.15.47:0/1533983863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f8bc0 0x5632126f9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.312+0000 7f49eaa00640 1 -- 172.21.15.47:0/1533983863 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563212629080 con 0x5632126f79f0 2024-03-21T11:24:55.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.313+0000 7f49e3e00640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f49d00a2fa0 0x7f49d00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.313+0000 7f49e3e00640 1 -- 172.21.15.47:0/3571506770 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f49e40cfe50 con 0x7f49ec107260 2024-03-21T11:24:55.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.314+0000 7f49eaa00640 1 --2- 172.21.15.47:0/1533983863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f79f0 0x5632126f7e10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f49d80137e0 tx=0x7f49d8009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89b5f85decb43df9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.314+0000 7f49eb400640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f49d00a2fa0 0x7f49d00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.314+0000 7f49e0c00640 1 -- 172.21.15.47:0/1533983863 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8013e20 con 0x5632126f79f0 2024-03-21T11:24:55.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.315+0000 7f49e0c00640 1 -- 172.21.15.47:0/1533983863 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49d8013120 con 0x5632126f79f0 2024-03-21T11:24:55.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.315+0000 7f49e0c00640 1 -- 172.21.15.47:0/1533983863 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80035f0 con 0x5632126f79f0 2024-03-21T11:24:55.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.315+0000 7f49eb400640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f49d00a2fa0 0x7f49d00a5460 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f49dc02c8e0 tx=0x7f49dc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.316+0000 7f49f5427740 1 -- 172.21.15.47:0/1533983863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f79f0 msgr2=0x5632126f7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.316+0000 7f49f5427740 1 --2- 172.21.15.47:0/1533983863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f79f0 0x5632126f7e10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f49d80137e0 tx=0x7f49d8009770 comp rx=0 tx=0).stop 2024-03-21T11:24:55.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.317+0000 7f49f5427740 1 -- 172.21.15.47:0/1533983863 shutdown_connections 2024-03-21T11:24:55.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.317+0000 7f49f5427740 1 --2- 172.21.15.47:0/1533983863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f9580 0x5632126ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.317+0000 7f49f5427740 1 --2- 172.21.15.47:0/1533983863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f8bc0 0x5632126f9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.317+0000 7f49f5427740 1 --2- 172.21.15.47:0/1533983863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f79f0 0x5632126f7e10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.317+0000 7f49f5427740 1 -- 172.21.15.47:0/1533983863 >> 172.21.15.47:0/1533983863 conn(0x5632126f2de0 msgr2=0x5632126f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.317+0000 7f49f5427740 1 -- 172.21.15.47:0/1533983863 shutdown_connections 2024-03-21T11:24:55.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.317+0000 7f49f5427740 1 -- 172.21.15.47:0/1533983863 wait complete. 2024-03-21T11:24:55.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.317+0000 7f49f5427740 1 -- start start 2024-03-21T11:24:55.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.318+0000 7f49f5427740 1 Processor -- start 2024-03-21T11:24:55.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.318+0000 7f49f5427740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f79f0 0x563212701ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.318+0000 7f49f5427740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f8bc0 0x5632127021e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.318+0000 7f49f5427740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f9580 0x563212709220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.318+0000 7f49f5427740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563212700530 con 0x5632126f79f0 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.318+0000 7f49f5427740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632126180d0 con 0x5632126f8bc0 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.318+0000 7f49f5427740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632127006b0 con 0x5632126f9580 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.319+0000 7f49f5427740 10 client.0 _ll_register_callbacks cb 0x56321261ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.319+0000 7f49eaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f8bc0 0x5632127021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.319+0000 7f49eaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f8bc0 0x5632127021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40466/0 (socket says 172.21.15.47:40466) 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.319+0000 7f49eaa00640 1 -- 172.21.15.47:0/2568362102 learned_addr learned my addr 172.21.15.47:0/2568362102 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.319+0000 7f49eb400640 1 --2- 172.21.15.47:0/2568362102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f79f0 0x563212701ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.319+0000 7f49ebe00640 1 --2- 172.21.15.47:0/2568362102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f9580 0x563212709220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.319+0000 7f49eaa00640 1 -- 172.21.15.47:0/2568362102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f9580 msgr2=0x563212709220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.319+0000 7f49eaa00640 1 --2- 172.21.15.47:0/2568362102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f9580 0x563212709220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.319+0000 7f49eaa00640 1 -- 172.21.15.47:0/2568362102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f79f0 msgr2=0x563212701ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.319+0000 7f49eaa00640 1 --2- 172.21.15.47:0/2568362102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f79f0 0x563212701ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.319+0000 7f49eaa00640 1 -- 172.21.15.47:0/2568362102 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632123703f0 con 0x5632126f8bc0 2024-03-21T11:24:55.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.319+0000 7f49ebe00640 1 --2- 172.21.15.47:0/2568362102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f9580 0x563212709220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:55.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.319+0000 7f49eaa00640 1 --2- 172.21.15.47:0/2568362102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f8bc0 0x5632127021e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f49d80095d0 tx=0x7f49d800f830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15fc0b6cd4b46cb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.320+0000 7f49d5600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:24:55.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.320+0000 7f49d5600640 1 -- 172.21.15.47:0/2568362102 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8013e20 con 0x5632126f8bc0 2024-03-21T11:24:55.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.320+0000 7f49d5600640 1 -- 172.21.15.47:0/2568362102 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f49d80052c0 con 0x5632126f8bc0 2024-03-21T11:24:55.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.320+0000 7f49f5427740 1 -- 172.21.15.47:0/2568362102 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49d8013120 con 0x5632126f8bc0 2024-03-21T11:24:55.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.320+0000 7f49d5600640 1 -- 172.21.15.47:0/2568362102 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49d802d050 con 0x5632126f8bc0 2024-03-21T11:24:55.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.320+0000 7f49f5427740 10 client.25901 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:55.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.320+0000 7f49f5427740 1 -- 172.21.15.47:0/2568362102 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56321243a7d0 con 0x5632126f8bc0 2024-03-21T11:24:55.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.320+0000 7f49d5600640 1 -- 172.21.15.47:0/2568362102 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49d8013120 con 0x5632126f8bc0 2024-03-21T11:24:55.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.321+0000 7f49d5600640 1 -- 172.21.15.47:0/2568362102 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f49d80204f0 con 0x5632126f8bc0 2024-03-21T11:24:55.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.321+0000 7f49f5427740 10 client.25901 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:55.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.321+0000 7f49f5427740 10 client.25901 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:24:55.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.321+0000 7f49f5427740 20 client.25901 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:55.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.321+0000 7f49f5427740 1 --2- 172.21.15.47:0/2568362102 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56321270d3f0 0x56321270f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.321+0000 7f49f5427740 4 client.25901 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:55.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.321+0000 7f49f5427740 1 -- 172.21.15.47:0/2568362102 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563212669800 con 0x56321270d3f0 2024-03-21T11:24:55.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.321+0000 7f49eb400640 1 --2- 172.21.15.47:0/2568362102 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56321270d3f0 0x56321270f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.322+0000 7f49eb400640 1 --2- 172.21.15.47:0/2568362102 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56321270d3f0 0x56321270f7e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3622a62761421c server_cookie=e1fc9dfd97e21db8 in_seq=0 out_seq=0 2024-03-21T11:24:55.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.322+0000 7f49d5600640 10 client.25901 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:24:55.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.323+0000 7f49d5600640 1 -- 172.21.15.47:0/2568362102 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563212669800 con 0x56321270d3f0 2024-03-21T11:24:55.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.323+0000 7f49d5600640 10 client.25901 handle_command_reply: tid=0 2024-03-21T11:24:55.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.323+0000 7f49f5427740 1 client.25901 shutdown 2024-03-21T11:24:55.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.323+0000 7f49f5427740 1 -- 172.21.15.47:0/2568362102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f8bc0 msgr2=0x5632127021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.323+0000 7f49f5427740 1 --2- 172.21.15.47:0/2568362102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f8bc0 0x5632127021e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f49d80095d0 tx=0x7f49d800f830 comp rx=0 tx=0).stop 2024-03-21T11:24:55.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.323+0000 7f49f5427740 1 -- 172.21.15.47:0/2568362102 shutdown_connections 2024-03-21T11:24:55.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.323+0000 7f49f5427740 1 --2- 172.21.15.47:0/2568362102 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56321270d3f0 0x56321270f7e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.323+0000 7f49f5427740 1 --2- 172.21.15.47:0/2568362102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f9580 0x563212709220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.323+0000 7f49d5600640 0 client.25901 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:24:55.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.324+0000 7f49f5427740 1 --2- 172.21.15.47:0/2568362102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f8bc0 0x5632127021e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.324+0000 7f49f5427740 1 --2- 172.21.15.47:0/2568362102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f79f0 0x563212701ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.324+0000 7f49f5427740 1 -- 172.21.15.47:0/2568362102 >> 172.21.15.47:0/2568362102 conn(0x5632126f2de0 msgr2=0x5632126f2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.324+0000 7f49f5427740 1 -- 172.21.15.47:0/2568362102 shutdown_connections 2024-03-21T11:24:55.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.324+0000 7f49f5427740 1 -- 172.21.15.47:0/2568362102 wait complete. 2024-03-21T11:24:55.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.324+0000 7f49f5427740 20 client.25901 trim_cache size 0 max 16384 2024-03-21T11:24:55.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.338+0000 7f49f5427740 1 Processor -- start 2024-03-21T11:24:55.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.338+0000 7f49f5427740 1 -- start start 2024-03-21T11:24:55.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.338+0000 7f49f5427740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f79f0 0x5632126fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.338+0000 7f49f5427740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f8bc0 0x5632127091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.338+0000 7f49f5427740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f9580 0x5632126fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.338+0000 7f49f5427740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec1ac140 con 0x5632126f8bc0 2024-03-21T11:24:55.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.338+0000 7f49f5427740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec1adac0 con 0x5632126f79f0 2024-03-21T11:24:55.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.338+0000 7f49f5427740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec1ac7a0 con 0x5632126f9580 2024-03-21T11:24:55.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.338+0000 7f49eb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f79f0 0x5632126fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.338+0000 7f49eaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f8bc0 0x5632127091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.338+0000 7f49eb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f79f0 0x5632126fbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40474/0 (socket says 172.21.15.47:40474) 2024-03-21T11:24:55.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.338+0000 7f49eb400640 1 -- 172.21.15.47:0/180414722 learned_addr learned my addr 172.21.15.47:0/180414722 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.339+0000 7f49ebe00640 1 --2- 172.21.15.47:0/180414722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f9580 0x5632126fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.339+0000 7f49eb400640 1 -- 172.21.15.47:0/180414722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f9580 msgr2=0x5632126fc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.339+0000 7f49eb400640 1 --2- 172.21.15.47:0/180414722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f9580 0x5632126fc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.339+0000 7f49eb400640 1 -- 172.21.15.47:0/180414722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f8bc0 msgr2=0x5632127091e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.339+0000 7f49eb400640 1 --2- 172.21.15.47:0/180414722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f8bc0 0x5632127091e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.339+0000 7f49eb400640 1 -- 172.21.15.47:0/180414722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563212702dd0 con 0x5632126f79f0 2024-03-21T11:24:55.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.339+0000 7f49eaa00640 1 --2- 172.21.15.47:0/180414722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f8bc0 0x5632127091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:55.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.339+0000 7f49eb400640 1 --2- 172.21.15.47:0/180414722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f79f0 0x5632126fbc60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f49dc02c240 tx=0x7f49dc042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bf03ea583812aa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.339+0000 7f49d7e00640 1 -- 172.21.15.47:0/180414722 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc046070 con 0x5632126f79f0 2024-03-21T11:24:55.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.340+0000 7f49d7e00640 1 -- 172.21.15.47:0/180414722 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49dc05b070 con 0x5632126f79f0 2024-03-21T11:24:55.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.340+0000 7f49d7e00640 1 -- 172.21.15.47:0/180414722 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc005670 con 0x5632126f79f0 2024-03-21T11:24:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.340+0000 7f49f5427740 1 -- 172.21.15.47:0/180414722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f79f0 msgr2=0x5632126fbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.340+0000 7f49f5427740 1 --2- 172.21.15.47:0/180414722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f79f0 0x5632126fbc60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f49dc02c240 tx=0x7f49dc042000 comp rx=0 tx=0).stop 2024-03-21T11:24:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.340+0000 7f49f5427740 1 -- 172.21.15.47:0/180414722 shutdown_connections 2024-03-21T11:24:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.340+0000 7f49f5427740 1 --2- 172.21.15.47:0/180414722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f9580 0x5632126fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.340+0000 7f49f5427740 1 --2- 172.21.15.47:0/180414722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f8bc0 0x5632127091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.340+0000 7f49f5427740 1 --2- 172.21.15.47:0/180414722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f79f0 0x5632126fbc60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.340+0000 7f49f5427740 1 -- 172.21.15.47:0/180414722 >> 172.21.15.47:0/180414722 conn(0x5632126f2de0 msgr2=0x5632126f68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.340+0000 7f49f5427740 1 -- 172.21.15.47:0/180414722 shutdown_connections 2024-03-21T11:24:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.340+0000 7f49f5427740 1 -- 172.21.15.47:0/180414722 wait complete. 2024-03-21T11:24:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.341+0000 7f49f5427740 1 -- start start 2024-03-21T11:24:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.341+0000 7f49f5427740 1 Processor -- start 2024-03-21T11:24:55.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.341+0000 7f49f5427740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f79f0 0x563212704cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.341+0000 7f49f5427740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f8bc0 0x563212705200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.341+0000 7f49f5427740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f9580 0x5632127081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.341+0000 7f49f5427740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec1ad460 con 0x5632126f9580 2024-03-21T11:24:55.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.341+0000 7f49f5427740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec1ace00 con 0x5632126f79f0 2024-03-21T11:24:55.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.341+0000 7f49f5427740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec1aa300 con 0x5632126f8bc0 2024-03-21T11:24:55.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.342+0000 7f49f5427740 10 client.0 _ll_register_callbacks cb 0x7f49d8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:55.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.342+0000 7f49eaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f8bc0 0x563212705200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.342+0000 7f49eaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f8bc0 0x563212705200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52618/0 (socket says 172.21.15.47:52618) 2024-03-21T11:24:55.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.342+0000 7f49eaa00640 1 -- 172.21.15.47:0/3461540529 learned_addr learned my addr 172.21.15.47:0/3461540529 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.342+0000 7f49eb400640 1 --2- 172.21.15.47:0/3461540529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f79f0 0x563212704cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.342+0000 7f49ebe00640 1 --2- 172.21.15.47:0/3461540529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f9580 0x5632127081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.342+0000 7f49eaa00640 1 -- 172.21.15.47:0/3461540529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f79f0 msgr2=0x563212704cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.342+0000 7f49eaa00640 1 --2- 172.21.15.47:0/3461540529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f79f0 0x563212704cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.342+0000 7f49eaa00640 1 -- 172.21.15.47:0/3461540529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f9580 msgr2=0x5632127081d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.342+0000 7f49eaa00640 1 --2- 172.21.15.47:0/3461540529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f9580 0x5632127081d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.342+0000 7f49eaa00640 1 -- 172.21.15.47:0/3461540529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632127087c0 con 0x5632126f8bc0 2024-03-21T11:24:55.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.342+0000 7f49eaa00640 1 --2- 172.21.15.47:0/3461540529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f8bc0 0x563212705200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f49d8000c00 tx=0x7f49d8009a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adc72ae62688d7ed server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.342+0000 7f49d5600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:55.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.343+0000 7f49d5600640 1 -- 172.21.15.47:0/3461540529 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8020ce0 con 0x5632126f8bc0 2024-03-21T11:24:55.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.343+0000 7f49f5427740 1 -- 172.21.15.47:0/3461540529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49dc05b070 con 0x5632126f8bc0 2024-03-21T11:24:55.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.343+0000 7f49d5600640 1 -- 172.21.15.47:0/3461540529 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f49d8020460 con 0x5632126f8bc0 2024-03-21T11:24:55.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.343+0000 7f49d5600640 1 -- 172.21.15.47:0/3461540529 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49d803e050 con 0x5632126f8bc0 2024-03-21T11:24:55.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.343+0000 7f49f5427740 10 client.25930 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:55.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.343+0000 7f49f5427740 1 -- 172.21.15.47:0/3461540529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563212768fa0 con 0x5632126f8bc0 2024-03-21T11:24:55.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.343+0000 7f49d5600640 1 -- 172.21.15.47:0/3461540529 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49dc05b070 con 0x5632126f8bc0 2024-03-21T11:24:55.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.343+0000 7f49d5600640 1 -- 172.21.15.47:0/3461540529 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f49d8039030 con 0x5632126f8bc0 2024-03-21T11:24:55.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.344+0000 7f49f5427740 10 client.25930 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:55.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.344+0000 7f49f5427740 10 client.25930 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:24:55.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.344+0000 7f49f5427740 20 client.25930 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:55.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.344+0000 7f49f5427740 1 --2- 172.21.15.47:0/3461540529 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56321270d3f0 0x56321276b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.344+0000 7f49f5427740 4 client.25930 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:24:55.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.344+0000 7f49f5427740 1 -- 172.21.15.47:0/3461540529 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563212663b50 con 0x56321270d3f0 2024-03-21T11:24:55.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.345+0000 7f49eb400640 1 --2- 172.21.15.47:0/3461540529 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56321270d3f0 0x56321276b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.345+0000 7f49eb400640 1 --2- 172.21.15.47:0/3461540529 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56321270d3f0 0x56321276b800 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c85eb34ce1239947 server_cookie=8832b35ed0738c10 in_seq=0 out_seq=0 2024-03-21T11:24:55.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.345+0000 7f49d5600640 10 client.25930 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:24:55.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.345+0000 7f49d5600640 1 -- 172.21.15.47:0/3461540529 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563212663b50 con 0x56321270d3f0 2024-03-21T11:24:55.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.346+0000 7f49d5600640 10 client.25930 handle_command_reply: tid=0 2024-03-21T11:24:55.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.346+0000 7f49f5427740 1 client.25930 shutdown 2024-03-21T11:24:55.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.346+0000 7f49f5427740 1 -- 172.21.15.47:0/3461540529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f8bc0 msgr2=0x563212705200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.346+0000 7f49f5427740 1 --2- 172.21.15.47:0/3461540529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f8bc0 0x563212705200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f49d8000c00 tx=0x7f49d8009a50 comp rx=0 tx=0).stop 2024-03-21T11:24:55.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.346+0000 7f49f5427740 1 -- 172.21.15.47:0/3461540529 shutdown_connections 2024-03-21T11:24:55.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.346+0000 7f49f5427740 1 --2- 172.21.15.47:0/3461540529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5632126f9580 0x5632127081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.346+0000 7f49f5427740 1 --2- 172.21.15.47:0/3461540529 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56321270d3f0 0x56321276b800 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.346+0000 7f49f5427740 1 --2- 172.21.15.47:0/3461540529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5632126f8bc0 0x563212705200 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.346+0000 7f49f5427740 1 --2- 172.21.15.47:0/3461540529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5632126f79f0 0x563212704cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.346+0000 7f49f5427740 1 -- 172.21.15.47:0/3461540529 >> 172.21.15.47:0/3461540529 conn(0x5632126f2de0 msgr2=0x5632126f18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.346+0000 7f49d5600640 0 client.25930 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:24:55.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.346+0000 7f49f5427740 1 -- 172.21.15.47:0/3461540529 shutdown_connections 2024-03-21T11:24:55.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.346+0000 7f49f5427740 1 -- 172.21.15.47:0/3461540529 wait complete. 2024-03-21T11:24:55.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.346+0000 7f49f5427740 20 client.25930 trim_cache size 0 max 16384 2024-03-21T11:24:55.349 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:24:55.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.349+0000 7f49f1200640 1 -- 172.21.15.47:0/3571506770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f49d00a2fa0 msgr2=0x7f49d00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.349+0000 7f49f1200640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f49d00a2fa0 0x7f49d00a5460 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f49dc02c8e0 tx=0x7f49dc02df90 comp rx=0 tx=0).stop 2024-03-21T11:24:55.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.349+0000 7f49f1200640 1 -- 172.21.15.47:0/3571506770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49ec107260 msgr2=0x7f49ec1a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.349+0000 7f49f1200640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49ec107260 0x7f49ec1a1080 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009fd0 tx=0x7f49e400ef10 comp rx=0 tx=0).stop 2024-03-21T11:24:55.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.349+0000 7f49f1200640 1 -- 172.21.15.47:0/3571506770 shutdown_connections 2024-03-21T11:24:55.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.349+0000 7f49f1200640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f49d00a2fa0 0x7f49d00a5460 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.349+0000 7f49f1200640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49ec107260 0x7f49ec1a1080 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.349+0000 7f49f1200640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49ec104930 0x7f49ec19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.349+0000 7f49f1200640 1 --2- 172.21.15.47:0/3571506770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f49ec102000 0x7f49ec199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.349+0000 7f49f1200640 1 -- 172.21.15.47:0/3571506770 >> 172.21.15.47:0/3571506770 conn(0x7f49ec0fba10 msgr2=0x7f49ec1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.349+0000 7f49f1200640 1 -- 172.21.15.47:0/3571506770 shutdown_connections 2024-03-21T11:24:55.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.349+0000 7f49f1200640 1 -- 172.21.15.47:0/3571506770 wait complete. 2024-03-21T11:24:55.375 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:24:55.375 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:24:55.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:55 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3919241894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:24:55.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:55 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3919241894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:24:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.565+0000 7fc710e00640 1 -- 172.21.15.47:0/3954215960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc70c109450 msgr2=0x7fc70c10b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.565+0000 7fc710e00640 1 --2- 172.21.15.47:0/3954215960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc70c109450 0x7fc70c10b8e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc704009be0 tx=0x7fc70402bf80 comp rx=0 tx=0).stop 2024-03-21T11:24:55.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.566+0000 7fc710e00640 1 -- 172.21.15.47:0/3954215960 shutdown_connections 2024-03-21T11:24:55.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.566+0000 7fc710e00640 1 --2- 172.21.15.47:0/3954215960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc70c109450 0x7fc70c10b8e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.566+0000 7fc710e00640 1 --2- 172.21.15.47:0/3954215960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c074080 0x7fc70c0744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.566+0000 7fc710e00640 1 --2- 172.21.15.47:0/3954215960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc70c075660 0x7fc70c073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.566+0000 7fc710e00640 1 -- 172.21.15.47:0/3954215960 >> 172.21.15.47:0/3954215960 conn(0x7fc70c0fcb10 msgr2=0x7fc70c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.566+0000 7fc710e00640 1 -- 172.21.15.47:0/3954215960 shutdown_connections 2024-03-21T11:24:55.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.566+0000 7fc710e00640 1 -- 172.21.15.47:0/3954215960 wait complete. 2024-03-21T11:24:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.567+0000 7fc710e00640 1 Processor -- start 2024-03-21T11:24:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.567+0000 7fc710e00640 1 -- start start 2024-03-21T11:24:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.567+0000 7fc710e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc70c074080 0x7fc70c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.567+0000 7fc710e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c075660 0x7fc70c199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.567+0000 7fc710e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc70c109450 0x7fc70c1a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.567+0000 7fc710e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c10f480 con 0x7fc70c075660 2024-03-21T11:24:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.567+0000 7fc710e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c10f300 con 0x7fc70c109450 2024-03-21T11:24:55.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.567+0000 7fc710e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c10f600 con 0x7fc70c074080 2024-03-21T11:24:55.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.568+0000 7fc70be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc70c109450 0x7fc70c1a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.568+0000 7fc70be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc70c109450 0x7fc70c1a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40508/0 (socket says 172.21.15.47:40508) 2024-03-21T11:24:55.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.568+0000 7fc70be00640 1 -- 172.21.15.47:0/944248656 learned_addr learned my addr 172.21.15.47:0/944248656 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.568+0000 7fc70aa00640 1 --2- 172.21.15.47:0/944248656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c075660 0x7fc70c199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.568+0000 7fc70be00640 1 -- 172.21.15.47:0/944248656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc70c074080 msgr2=0x7fc70c1999e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T11:24:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.568+0000 7fc70be00640 1 --2- 172.21.15.47:0/944248656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc70c074080 0x7fc70c1999e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.568+0000 7fc70be00640 1 -- 172.21.15.47:0/944248656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c075660 msgr2=0x7fc70c199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.568+0000 7fc70be00640 1 --2- 172.21.15.47:0/944248656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c075660 0x7fc70c199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.568+0000 7fc70be00640 1 -- 172.21.15.47:0/944248656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c1a1600 con 0x7fc70c109450 2024-03-21T11:24:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.568+0000 7fc70aa00640 1 --2- 172.21.15.47:0/944248656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c075660 0x7fc70c199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.568+0000 7fc70be00640 1 --2- 172.21.15.47:0/944248656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc70c109450 0x7fc70c1a0f90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc704036040 tx=0x7fc704004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.568+0000 7fc703e00640 1 -- 172.21.15.47:0/944248656 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704043020 con 0x7fc70c109450 2024-03-21T11:24:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.569+0000 7fc703e00640 1 -- 172.21.15.47:0/944248656 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7040045f0 con 0x7fc70c109450 2024-03-21T11:24:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.569+0000 7fc710e00640 1 -- 172.21.15.47:0/944248656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70c1a1890 con 0x7fc70c109450 2024-03-21T11:24:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.569+0000 7fc703e00640 1 -- 172.21.15.47:0/944248656 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70403e870 con 0x7fc70c109450 2024-03-21T11:24:55.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.569+0000 7fc710e00640 1 -- 172.21.15.47:0/944248656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70c1a1d70 con 0x7fc70c109450 2024-03-21T11:24:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.572+0000 7fc714ffc740 1 Processor -- start 2024-03-21T11:24:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.573+0000 7fc703e00640 1 -- 172.21.15.47:0/944248656 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc704046020 con 0x7fc70c109450 2024-03-21T11:24:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.574+0000 7fc714ffc740 1 -- start start 2024-03-21T11:24:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.574+0000 7fc714ffc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7583200 0x5588c7583620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.574+0000 7fc714ffc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c7584850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.574+0000 7fc714ffc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7584d90 0x5588c758b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.574+0000 7fc714ffc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588c74df2c0 con 0x5588c7584d90 2024-03-21T11:24:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.574+0000 7fc714ffc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588c7487c90 con 0x5588c75843d0 2024-03-21T11:24:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.574+0000 7fc714ffc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588c74db360 con 0x5588c7583200 2024-03-21T11:24:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.575+0000 7fc70b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c7584850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.575+0000 7fc70b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c7584850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40510/0 (socket says 172.21.15.47:40510) 2024-03-21T11:24:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.575+0000 7fc70b400640 1 -- 172.21.15.47:0/1039037719 learned_addr learned my addr 172.21.15.47:0/1039037719 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.575+0000 7fc70b400640 1 -- 172.21.15.47:0/1039037719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7583200 msgr2=0x5588c7583620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:24:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.575+0000 7fc70b400640 1 --2- 172.21.15.47:0/1039037719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7583200 0x5588c7583620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.575+0000 7fc70b400640 1 -- 172.21.15.47:0/1039037719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7584d90 msgr2=0x5588c758b620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:24:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.575+0000 7fc70aa00640 1 --2- 172.21.15.47:0/1039037719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7584d90 0x5588c758b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.575+0000 7fc70b400640 1 --2- 172.21.15.47:0/1039037719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7584d90 0x5588c758b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.575+0000 7fc70b400640 1 -- 172.21.15.47:0/1039037719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588c74ac080 con 0x5588c75843d0 2024-03-21T11:24:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.575+0000 7fc70aa00640 1 --2- 172.21.15.47:0/1039037719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7584d90 0x5588c758b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.576+0000 7fc70b400640 1 --2- 172.21.15.47:0/1039037719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c7584850 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00ba80 tx=0x7fc6fc00bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d66f21f2daa889b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.576+0000 7fc703e00640 1 --2- 172.21.15.47:0/944248656 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc6f00a2fa0 0x7fc6f00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.576+0000 7fc700c00640 1 -- 172.21.15.47:0/1039037719 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc016070 con 0x5588c75843d0 2024-03-21T11:24:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.576+0000 7fc700c00640 1 -- 172.21.15.47:0/1039037719 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6fc005500 con 0x5588c75843d0 2024-03-21T11:24:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.576+0000 7fc700c00640 1 -- 172.21.15.47:0/1039037719 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc010460 con 0x5588c75843d0 2024-03-21T11:24:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.576+0000 7fc703e00640 1 -- 172.21.15.47:0/944248656 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc7040f21c0 con 0x7fc70c109450 2024-03-21T11:24:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.576+0000 7fc70aa00640 1 --2- 172.21.15.47:0/944248656 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc6f00a2fa0 0x7fc6f00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.577+0000 7fc714ffc740 1 -- 172.21.15.47:0/1039037719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 msgr2=0x5588c7584850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.577+0000 7fc714ffc740 1 --2- 172.21.15.47:0/1039037719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c7584850 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00ba80 tx=0x7fc6fc00bf40 comp rx=0 tx=0).stop 2024-03-21T11:24:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.577+0000 7fc714ffc740 1 -- 172.21.15.47:0/1039037719 shutdown_connections 2024-03-21T11:24:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.577+0000 7fc714ffc740 1 --2- 172.21.15.47:0/1039037719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7584d90 0x5588c758b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.577+0000 7fc714ffc740 1 --2- 172.21.15.47:0/1039037719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c7584850 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.577+0000 7fc714ffc740 1 --2- 172.21.15.47:0/1039037719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7583200 0x5588c7583620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.577+0000 7fc714ffc740 1 -- 172.21.15.47:0/1039037719 >> 172.21.15.47:0/1039037719 conn(0x5588c7575de0 msgr2=0x5588c75761e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.580+0000 7fc70aa00640 1 --2- 172.21.15.47:0/944248656 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc6f00a2fa0 0x7fc6f00a5460 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5588c75842d0 tx=0x7fc6f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.580+0000 7fc714ffc740 1 -- 172.21.15.47:0/1039037719 shutdown_connections 2024-03-21T11:24:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.580+0000 7fc714ffc740 1 -- 172.21.15.47:0/1039037719 wait complete. 2024-03-21T11:24:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.580+0000 7fc714ffc740 1 -- start start 2024-03-21T11:24:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.581+0000 7fc714ffc740 1 Processor -- start 2024-03-21T11:24:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.581+0000 7fc714ffc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7583200 0x5588c758d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.581+0000 7fc714ffc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c758d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.581+0000 7fc714ffc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 0x5588c75949b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.581+0000 7fc714ffc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588c758bda0 con 0x5588c7583200 2024-03-21T11:24:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.581+0000 7fc714ffc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588c749b0d0 con 0x5588c75843d0 2024-03-21T11:24:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.582+0000 7fc714ffc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588c758bf20 con 0x5588c7584d90 2024-03-21T11:24:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.582+0000 7fc70aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c758d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.582+0000 7fc70be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 0x5588c75949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.582+0000 7fc70aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c758d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40522/0 (socket says 172.21.15.47:40522) 2024-03-21T11:24:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.582+0000 7fc714ffc740 10 client.0 _ll_register_callbacks cb 0x5588c749fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.582+0000 7fc70aa00640 1 -- 172.21.15.47:0/4137292084 learned_addr learned my addr 172.21.15.47:0/4137292084 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.582+0000 7fc70b400640 1 --2- 172.21.15.47:0/4137292084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7583200 0x5588c758d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.582+0000 7fc70aa00640 1 -- 172.21.15.47:0/4137292084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 msgr2=0x5588c75949b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.582+0000 7fc70aa00640 1 --2- 172.21.15.47:0/4137292084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 0x5588c75949b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.582+0000 7fc70aa00640 1 -- 172.21.15.47:0/4137292084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7583200 msgr2=0x5588c758d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.583+0000 7fc70aa00640 1 --2- 172.21.15.47:0/4137292084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7583200 0x5588c758d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.583+0000 7fc70aa00640 1 -- 172.21.15.47:0/4137292084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588c71f33f0 con 0x5588c75843d0 2024-03-21T11:24:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.583+0000 7fc70be00640 1 --2- 172.21.15.47:0/4137292084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 0x5588c75949b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.583+0000 7fc70b400640 1 --2- 172.21.15.47:0/4137292084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7583200 0x5588c758d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.583+0000 7fc70aa00640 1 --2- 172.21.15.47:0/4137292084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c758d9e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8010c30 tx=0x7fc6f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb4d28990017cab9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.583+0000 7fc6f5600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:24:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.584+0000 7fc6f5600640 1 -- 172.21.15.47:0/4137292084 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f800fd50 con 0x5588c75843d0 2024-03-21T11:24:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.584+0000 7fc6f5600640 1 -- 172.21.15.47:0/4137292084 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc6f801f3f0 con 0x5588c75843d0 2024-03-21T11:24:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.584+0000 7fc714ffc740 1 -- 172.21.15.47:0/4137292084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6fc005500 con 0x5588c75843d0 2024-03-21T11:24:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.584+0000 7fc6f5600640 1 -- 172.21.15.47:0/4137292084 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6f801f5f0 con 0x5588c75843d0 2024-03-21T11:24:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.584+0000 7fc714ffc740 10 client.25919 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.584+0000 7fc714ffc740 1 -- 172.21.15.47:0/4137292084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5588c72bd7d0 con 0x5588c75843d0 2024-03-21T11:24:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.584+0000 7fc6f5600640 1 -- 172.21.15.47:0/4137292084 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6fc005500 con 0x5588c75843d0 2024-03-21T11:24:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.585+0000 7fc6f5600640 1 -- 172.21.15.47:0/4137292084 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc6f8027810 con 0x5588c75843d0 2024-03-21T11:24:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.585+0000 7fc714ffc740 10 client.25919 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.585+0000 7fc714ffc740 10 client.25919 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:24:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.585+0000 7fc714ffc740 20 client.25919 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.585+0000 7fc714ffc740 1 --2- 172.21.15.47:0/4137292084 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5588c757e570 0x5588c75809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.585+0000 7fc714ffc740 4 client.25919 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.585+0000 7fc714ffc740 1 -- 172.21.15.47:0/4137292084 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588c74ec800 con 0x5588c757e570 2024-03-21T11:24:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.585+0000 7fc70b400640 1 --2- 172.21.15.47:0/4137292084 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5588c757e570 0x5588c75809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.586+0000 7fc70b400640 1 --2- 172.21.15.47:0/4137292084 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5588c757e570 0x5588c75809f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa4ac6cf7d5c6259 server_cookie=254b585dd6000d14 in_seq=0 out_seq=0 2024-03-21T11:24:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.586+0000 7fc6f5600640 10 client.25919 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:24:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.587+0000 7fc6f5600640 1 -- 172.21.15.47:0/4137292084 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5588c74ec800 con 0x5588c757e570 2024-03-21T11:24:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.587+0000 7fc6f5600640 10 client.25919 handle_command_reply: tid=0 2024-03-21T11:24:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.587+0000 7fc714ffc740 1 client.25919 shutdown 2024-03-21T11:24:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.588+0000 7fc714ffc740 1 -- 172.21.15.47:0/4137292084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 msgr2=0x5588c758d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.588+0000 7fc714ffc740 1 --2- 172.21.15.47:0/4137292084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c758d9e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8010c30 tx=0x7fc6f80046c0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.588+0000 7fc714ffc740 1 -- 172.21.15.47:0/4137292084 shutdown_connections 2024-03-21T11:24:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.588+0000 7fc714ffc740 1 --2- 172.21.15.47:0/4137292084 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5588c757e570 0x5588c75809f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.588+0000 7fc714ffc740 1 --2- 172.21.15.47:0/4137292084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 0x5588c75949b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.588+0000 7fc714ffc740 1 --2- 172.21.15.47:0/4137292084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c758d9e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.588+0000 7fc714ffc740 1 --2- 172.21.15.47:0/4137292084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7583200 0x5588c758d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.588+0000 7fc714ffc740 1 -- 172.21.15.47:0/4137292084 >> 172.21.15.47:0/4137292084 conn(0x5588c7575de0 msgr2=0x5588c7575160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.588+0000 7fc6f5600640 0 client.25919 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:24:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.588+0000 7fc714ffc740 1 -- 172.21.15.47:0/4137292084 shutdown_connections 2024-03-21T11:24:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.588+0000 7fc714ffc740 1 -- 172.21.15.47:0/4137292084 wait complete. 2024-03-21T11:24:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.589+0000 7fc714ffc740 20 client.25919 trim_cache size 0 max 16384 2024-03-21T11:24:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.606+0000 7fd96b200640 1 -- 172.21.15.47:0/2179972601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9641054b0 msgr2=0x7fd96410bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.606+0000 7fd96b200640 1 --2- 172.21.15.47:0/2179972601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9641054b0 0x7fd96410bd40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd95c009be0 tx=0x7fd95c02bef0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.606+0000 7fd96b200640 1 -- 172.21.15.47:0/2179972601 shutdown_connections 2024-03-21T11:24:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.606+0000 7fd96b200640 1 --2- 172.21.15.47:0/2179972601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9641054b0 0x7fd96410bd40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.606+0000 7fd96b200640 1 --2- 172.21.15.47:0/2179972601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd964104af0 0x7fd964104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.606+0000 7fd96b200640 1 --2- 172.21.15.47:0/2179972601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd964103900 0x7fd964103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.606+0000 7fd96b200640 1 -- 172.21.15.47:0/2179972601 >> 172.21.15.47:0/2179972601 conn(0x7fd9640fee90 msgr2=0x7fd9641012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.607+0000 7fd96b200640 1 -- 172.21.15.47:0/2179972601 shutdown_connections 2024-03-21T11:24:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.607+0000 7fd96b200640 1 -- 172.21.15.47:0/2179972601 wait complete. 2024-03-21T11:24:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.608+0000 7fd96b200640 1 Processor -- start 2024-03-21T11:24:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.608+0000 7fd96b200640 1 -- start start 2024-03-21T11:24:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.609+0000 7fd96b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd964103900 0x7fd964079450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.609+0000 7fd96b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd964104af0 0x7fd964079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.609+0000 7fd96b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9641054b0 0x7fd964075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.609+0000 7fd96b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96410f5d0 con 0x7fd9641054b0 2024-03-21T11:24:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.609+0000 7fd96b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96410f450 con 0x7fd964104af0 2024-03-21T11:24:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.609+0000 7fd96b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96410f750 con 0x7fd964103900 2024-03-21T11:24:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.609+0000 7fd963e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd964103900 0x7fd964079450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.609+0000 7fd963e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd964103900 0x7fd964079450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52672/0 (socket says 172.21.15.47:52672) 2024-03-21T11:24:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.609+0000 7fd963e00640 1 -- 172.21.15.47:0/93928723 learned_addr learned my addr 172.21.15.47:0/93928723 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.609+0000 7fd963400640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd964104af0 0x7fd964079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.609+0000 7fd963e00640 1 -- 172.21.15.47:0/93928723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd964104af0 msgr2=0x7fd964079990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.609+0000 7fd963e00640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd964104af0 0x7fd964079990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.610+0000 7fd963e00640 1 -- 172.21.15.47:0/93928723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9641054b0 msgr2=0x7fd964075fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:24:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.610+0000 7fd969200640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9641054b0 0x7fd964075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.610+0000 7fd963e00640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9641054b0 0x7fd964075fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.610+0000 7fd963e00640 1 -- 172.21.15.47:0/93928723 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd964076880 con 0x7fd964103900 2024-03-21T11:24:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.610+0000 7fd969200640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9641054b0 0x7fd964075fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.610+0000 7fd963e00640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd964103900 0x7fd964079450 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd95400d950 tx=0x7fd95400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.611+0000 7fd960c00640 1 -- 172.21.15.47:0/93928723 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954017020 con 0x7fd964103900 2024-03-21T11:24:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.611+0000 7fd960c00640 1 -- 172.21.15.47:0/93928723 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd954009c30 con 0x7fd964103900 2024-03-21T11:24:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.611+0000 7fd960c00640 1 -- 172.21.15.47:0/93928723 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95400f720 con 0x7fd964103900 2024-03-21T11:24:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.611+0000 7fd96b200640 1 -- 172.21.15.47:0/93928723 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9641a5f40 con 0x7fd964103900 2024-03-21T11:24:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.611+0000 7fd96b200640 1 -- 172.21.15.47:0/93928723 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9641a62f0 con 0x7fd964103900 2024-03-21T11:24:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.613+0000 7fc714ffc740 1 Processor -- start 2024-03-21T11:24:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.613+0000 7fc714ffc740 1 -- start start 2024-03-21T11:24:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.613+0000 7fc714ffc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7583200 0x5588c7579750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.613+0000 7fc714ffc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c7594970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.613+0000 7fc714ffc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 0x5588c7587470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.613+0000 7fc714ffc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1a7c00 con 0x5588c7583200 2024-03-21T11:24:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.613+0000 7fc714ffc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1a9580 con 0x5588c75843d0 2024-03-21T11:24:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.613+0000 7fc714ffc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1a8260 con 0x5588c7584d90 2024-03-21T11:24:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.614+0000 7fc70aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c7594970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.614+0000 7fc70aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c7594970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40540/0 (socket says 172.21.15.47:40540) 2024-03-21T11:24:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.614+0000 7fc70aa00640 1 -- 172.21.15.47:0/168613146 learned_addr learned my addr 172.21.15.47:0/168613146 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.614+0000 7fc70aa00640 1 -- 172.21.15.47:0/168613146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 msgr2=0x5588c7587470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:24:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.614+0000 7fc70aa00640 1 --2- 172.21.15.47:0/168613146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 0x5588c7587470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.614+0000 7fc70aa00640 1 -- 172.21.15.47:0/168613146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7583200 msgr2=0x5588c7579750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.614+0000 7fc70aa00640 1 --2- 172.21.15.47:0/168613146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7583200 0x5588c7579750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.614+0000 7fc70aa00640 1 -- 172.21.15.47:0/168613146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588c7583e50 con 0x5588c75843d0 2024-03-21T11:24:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.615+0000 7fc70aa00640 1 --2- 172.21.15.47:0/168613146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c7594970 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc6f80057a0 tx=0x7fc6f8009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1814172d3dd3573 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.615+0000 7fc6f7e00640 1 -- 172.21.15.47:0/168613146 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f80107e0 con 0x5588c75843d0 2024-03-21T11:24:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.615+0000 7fc6f7e00640 1 -- 172.21.15.47:0/168613146 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6f800f790 con 0x5588c75843d0 2024-03-21T11:24:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.615+0000 7fc6f7e00640 1 -- 172.21.15.47:0/168613146 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f801d560 con 0x5588c75843d0 2024-03-21T11:24:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fd960c00640 1 -- 172.21.15.47:0/93928723 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd95401a020 con 0x7fd964103900 2024-03-21T11:24:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fd96df8d740 1 Processor -- start 2024-03-21T11:24:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fd96df8d740 1 -- start start 2024-03-21T11:24:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fc714ffc740 1 -- 172.21.15.47:0/168613146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 msgr2=0x5588c7594970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fc714ffc740 1 --2- 172.21.15.47:0/168613146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c7594970 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc6f80057a0 tx=0x7fc6f8009f20 comp rx=0 tx=0).stop 2024-03-21T11:24:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fc714ffc740 1 -- 172.21.15.47:0/168613146 shutdown_connections 2024-03-21T11:24:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fc714ffc740 1 --2- 172.21.15.47:0/168613146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 0x5588c7587470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fc714ffc740 1 --2- 172.21.15.47:0/168613146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c75843d0 0x5588c7594970 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fc714ffc740 1 --2- 172.21.15.47:0/168613146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c7583200 0x5588c7579750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fc714ffc740 1 -- 172.21.15.47:0/168613146 >> 172.21.15.47:0/168613146 conn(0x5588c7575de0 msgr2=0x5588c7579040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fd96df8d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798e1e0 0x562df798e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fd96df8d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df798ebd0 0x562df79936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fd96df8d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 0x562df79961d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fd96df8d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df79513b0 con 0x562df7993d40 2024-03-21T11:24:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fd96df8d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df78f3830 con 0x562df798ebd0 2024-03-21T11:24:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.616+0000 7fd96df8d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df789c5d0 con 0x562df798e1e0 2024-03-21T11:24:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.617+0000 7fc714ffc740 1 -- 172.21.15.47:0/168613146 shutdown_connections 2024-03-21T11:24:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.617+0000 7fc714ffc740 1 -- 172.21.15.47:0/168613146 wait complete. 2024-03-21T11:24:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.617+0000 7fd963400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 0x562df79961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.617+0000 7fd963400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 0x562df79961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53148/0 (socket says 172.21.15.47:53148) 2024-03-21T11:24:55.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.617+0000 7fd963400640 1 -- 172.21.15.47:0/2193199147 learned_addr learned my addr 172.21.15.47:0/2193199147 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.617+0000 7fc714ffc740 1 -- start start 2024-03-21T11:24:55.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.618+0000 7fc714ffc740 1 Processor -- start 2024-03-21T11:24:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.618+0000 7fd969200640 1 --2- 172.21.15.47:0/2193199147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798e1e0 0x562df798e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.618+0000 7fd963400640 1 -- 172.21.15.47:0/2193199147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798e1e0 msgr2=0x562df798e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.618+0000 7fd963400640 1 --2- 172.21.15.47:0/2193199147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798e1e0 0x562df798e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.618+0000 7fd963400640 1 -- 172.21.15.47:0/2193199147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df798ebd0 msgr2=0x562df79936d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:24:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.618+0000 7fd963400640 1 --2- 172.21.15.47:0/2193199147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df798ebd0 0x562df79936d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.618+0000 7fd963400640 1 -- 172.21.15.47:0/2193199147 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562df78b42c0 con 0x562df7993d40 2024-03-21T11:24:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.618+0000 7fd969200640 1 --2- 172.21.15.47:0/2193199147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798e1e0 0x562df798e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.618+0000 7fd960c00640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9480a3040 0x7fd9480a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.618+0000 7fd963400640 1 --2- 172.21.15.47:0/2193199147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 0x562df79961d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd950009d30 tx=0x7fd9500107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6c856a5da6e1d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fd960c00640 1 -- 172.21.15.47:0/93928723 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd9540cf140 con 0x7fd964103900 2024-03-21T11:24:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fc714ffc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c7583200 0x5588c758eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fc714ffc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c75843d0 0x5588c758f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fc714ffc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 0x5588c7593c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fc714ffc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1a8f20 con 0x5588c75843d0 2024-03-21T11:24:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fc714ffc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1a88c0 con 0x5588c7583200 2024-03-21T11:24:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fc714ffc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1a5dc0 con 0x5588c7584d90 2024-03-21T11:24:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fc714ffc740 10 client.0 _ll_register_callbacks cb 0x7fc6f8009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fc70aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c75843d0 0x5588c758f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fc70aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c75843d0 0x5588c758f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53152/0 (socket says 172.21.15.47:53152) 2024-03-21T11:24:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fc70aa00640 1 -- 172.21.15.47:0/1474846652 learned_addr learned my addr 172.21.15.47:0/1474846652 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fc70be00640 1 --2- 172.21.15.47:0/1474846652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 0x5588c7593c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fd959600640 1 -- 172.21.15.47:0/2193199147 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95001c680 con 0x562df7993d40 2024-03-21T11:24:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fd959600640 1 -- 172.21.15.47:0/2193199147 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd950009e20 con 0x562df7993d40 2024-03-21T11:24:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fd959600640 1 -- 172.21.15.47:0/2193199147 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950011ca0 con 0x562df7993d40 2024-03-21T11:24:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fd969200640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9480a3040 0x7fd9480a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.619+0000 7fc70b400640 1 --2- 172.21.15.47:0/1474846652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c7583200 0x5588c758eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fc70be00640 1 -- 172.21.15.47:0/1474846652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c7583200 msgr2=0x5588c758eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fc70be00640 1 --2- 172.21.15.47:0/1474846652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c7583200 0x5588c758eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fc70be00640 1 -- 172.21.15.47:0/1474846652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c75843d0 msgr2=0x5588c758f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fc70be00640 1 --2- 172.21.15.47:0/1474846652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c75843d0 0x5588c758f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fc70be00640 1 -- 172.21.15.47:0/1474846652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588c75941f0 con 0x5588c7584d90 2024-03-21T11:24:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fd96df8d740 1 -- 172.21.15.47:0/2193199147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 msgr2=0x562df79961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fd96df8d740 1 --2- 172.21.15.47:0/2193199147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 0x562df79961d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd950009d30 tx=0x7fd9500107a0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fd969200640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9480a3040 0x7fd9480a5500 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd95c004af0 tx=0x7fd95c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fc70be00640 1 --2- 172.21.15.47:0/1474846652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 0x5588c7593c00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc7040be490 tx=0x7fc704009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1512f6f3cbda95cd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fc6f5600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fd96df8d740 1 -- 172.21.15.47:0/2193199147 shutdown_connections 2024-03-21T11:24:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fd96df8d740 1 --2- 172.21.15.47:0/2193199147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 0x562df79961d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fd96df8d740 1 --2- 172.21.15.47:0/2193199147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df798ebd0 0x562df79936d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fd96df8d740 1 --2- 172.21.15.47:0/2193199147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798e1e0 0x562df798e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fd96df8d740 1 -- 172.21.15.47:0/2193199147 >> 172.21.15.47:0/2193199147 conn(0x562df7989750 msgr2=0x562df7989b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.620+0000 7fd96df8d740 1 -- 172.21.15.47:0/2193199147 shutdown_connections 2024-03-21T11:24:55.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.621+0000 7fd96df8d740 1 -- 172.21.15.47:0/2193199147 wait complete. 2024-03-21T11:24:55.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.621+0000 7fc6f5600640 1 -- 172.21.15.47:0/1474846652 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7040d6020 con 0x5588c7584d90 2024-03-21T11:24:55.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.621+0000 7fc6f5600640 1 -- 172.21.15.47:0/1474846652 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc7040c23b0 con 0x5588c7584d90 2024-03-21T11:24:55.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.621+0000 7fc714ffc740 1 -- 172.21.15.47:0/1474846652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6f800f790 con 0x5588c7584d90 2024-03-21T11:24:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.621+0000 7fd96df8d740 1 -- start start 2024-03-21T11:24:55.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.621+0000 7fd96df8d740 1 Processor -- start 2024-03-21T11:24:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.621+0000 7fc6f5600640 1 -- 172.21.15.47:0/1474846652 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7040dc050 con 0x5588c7584d90 2024-03-21T11:24:55.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.621+0000 7fc714ffc740 10 client.25945 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:55.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.621+0000 7fc714ffc740 1 -- 172.21.15.47:0/1474846652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5588c75fd1f0 con 0x5588c7584d90 2024-03-21T11:24:55.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fc6f5600640 1 -- 172.21.15.47:0/1474846652 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6f800f790 con 0x5588c7584d90 2024-03-21T11:24:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fd96df8d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df798e1e0 0x562df79984c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fd96df8d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df7998a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fd96df8d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df7993d40 0x562df799f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fd96df8d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df78afa20 con 0x562df798e1e0 2024-03-21T11:24:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fd96df8d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df78ef8d0 con 0x562df7993d40 2024-03-21T11:24:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fd96df8d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df7996cf0 con 0x562df798ebd0 2024-03-21T11:24:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fd96df8d740 10 client.0 _ll_register_callbacks cb 0x562df78ec1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fd969200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df7993d40 0x562df799f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fd969200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df7993d40 0x562df799f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40564/0 (socket says 172.21.15.47:40564) 2024-03-21T11:24:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fd969200640 1 -- 172.21.15.47:0/791912913 learned_addr learned my addr 172.21.15.47:0/791912913 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fc6f5600640 1 -- 172.21.15.47:0/1474846652 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc7040d2b50 con 0x5588c7584d90 2024-03-21T11:24:55.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fc714ffc740 10 client.25945 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:55.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fc714ffc740 10 client.25945 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:24:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fc714ffc740 20 client.25945 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fc714ffc740 1 --2- 172.21.15.47:0/1474846652 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5588c75fdba0 0x5588c75fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fc714ffc740 4 client.25945 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:24:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fc714ffc740 1 -- 172.21.15.47:0/1474846652 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5588c74e6b50 con 0x5588c75fdba0 2024-03-21T11:24:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.622+0000 7fd963e00640 1 --2- 172.21.15.47:0/791912913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df798e1e0 0x562df79984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.623+0000 7fd963400640 1 --2- 172.21.15.47:0/791912913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df7998a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.623+0000 7fd969200640 1 -- 172.21.15.47:0/791912913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 msgr2=0x562df7998a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.623+0000 7fd969200640 1 --2- 172.21.15.47:0/791912913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df7998a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.623+0000 7fd969200640 1 -- 172.21.15.47:0/791912913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df798e1e0 msgr2=0x562df79984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.623+0000 7fd969200640 1 --2- 172.21.15.47:0/791912913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df798e1e0 0x562df79984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.623+0000 7fd969200640 1 -- 172.21.15.47:0/791912913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562df7608210 con 0x562df7993d40 2024-03-21T11:24:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.623+0000 7fd963400640 1 --2- 172.21.15.47:0/791912913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df7998a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.623+0000 7fd969200640 1 --2- 172.21.15.47:0/791912913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df7993d40 0x562df799f9d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd95c035000 tx=0x7fd95c02bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16ec28dd792d81b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.623+0000 7fd94e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:24:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.623+0000 7fc70b400640 1 --2- 172.21.15.47:0/1474846652 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5588c75fdba0 0x5588c75fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.623+0000 7fd94e000640 1 -- 172.21.15.47:0/791912913 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c047050 con 0x562df7993d40 2024-03-21T11:24:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.623+0000 7fd94e000640 1 -- 172.21.15.47:0/791912913 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd95c004250 con 0x562df7993d40 2024-03-21T11:24:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.623+0000 7fd96df8d740 1 -- 172.21.15.47:0/791912913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd950009e20 con 0x562df7993d40 2024-03-21T11:24:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.624+0000 7fd94e000640 1 -- 172.21.15.47:0/791912913 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd95c05a050 con 0x562df7993d40 2024-03-21T11:24:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.624+0000 7fd96df8d740 10 client.25934 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.624+0000 7fd96df8d740 1 -- 172.21.15.47:0/791912913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562df76d4620 con 0x562df7993d40 2024-03-21T11:24:55.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.624+0000 7fc6f5600640 10 client.25945 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:24:55.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.624+0000 7fc70b400640 1 --2- 172.21.15.47:0/1474846652 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5588c75fdba0 0x5588c75fffc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27f9ea4bf1234a46 server_cookie=a46370b9e77037ee in_seq=0 out_seq=0 2024-03-21T11:24:55.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.624+0000 7fc6f5600640 1 -- 172.21.15.47:0/1474846652 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5588c74e6b50 con 0x5588c75fdba0 2024-03-21T11:24:55.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.624+0000 7fc6f5600640 10 client.25945 handle_command_reply: tid=0 2024-03-21T11:24:55.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.624+0000 7fc714ffc740 1 client.25945 shutdown 2024-03-21T11:24:55.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.624+0000 7fd94e000640 1 -- 172.21.15.47:0/791912913 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd950009e20 con 0x562df7993d40 2024-03-21T11:24:55.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.624+0000 7fc714ffc740 1 -- 172.21.15.47:0/1474846652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 msgr2=0x5588c7593c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.624+0000 7fc714ffc740 1 --2- 172.21.15.47:0/1474846652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 0x5588c7593c00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc7040be490 tx=0x7fc704009b60 comp rx=0 tx=0).stop 2024-03-21T11:24:55.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.625+0000 7fc714ffc740 1 -- 172.21.15.47:0/1474846652 shutdown_connections 2024-03-21T11:24:55.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.625+0000 7fc714ffc740 1 --2- 172.21.15.47:0/1474846652 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5588c75fdba0 0x5588c75fffc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.625+0000 7fc714ffc740 1 --2- 172.21.15.47:0/1474846652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5588c7584d90 0x5588c7593c00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.625+0000 7fc714ffc740 1 --2- 172.21.15.47:0/1474846652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588c75843d0 0x5588c758f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.625+0000 7fc714ffc740 1 --2- 172.21.15.47:0/1474846652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588c7583200 0x5588c758eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.625+0000 7fd94e000640 1 -- 172.21.15.47:0/791912913 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd95c034950 con 0x562df7993d40 2024-03-21T11:24:55.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.625+0000 7fc714ffc740 1 -- 172.21.15.47:0/1474846652 >> 172.21.15.47:0/1474846652 conn(0x5588c7575de0 msgr2=0x5588c75798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.625+0000 7fc6f5600640 0 client.25945 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:24:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.626+0000 7fd96df8d740 10 client.25934 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.626+0000 7fd96df8d740 10 client.25934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.626+0000 7fd96df8d740 20 client.25934 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:55.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.626+0000 7fc714ffc740 1 -- 172.21.15.47:0/1474846652 shutdown_connections 2024-03-21T11:24:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.626+0000 7fd96df8d740 1 --2- 172.21.15.47:0/791912913 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562df79a4230 0x562df79a6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.626+0000 7fd96df8d740 4 client.25934 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.626+0000 7fd96df8d740 1 -- 172.21.15.47:0/791912913 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562df7900d70 con 0x562df79a4230 2024-03-21T11:24:55.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.626+0000 7fc714ffc740 1 -- 172.21.15.47:0/1474846652 wait complete. 2024-03-21T11:24:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.626+0000 7fd963e00640 1 --2- 172.21.15.47:0/791912913 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562df79a4230 0x562df79a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.626+0000 7fd963e00640 1 --2- 172.21.15.47:0/791912913 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562df79a4230 0x562df79a6620 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66fe5fb27081b2fd server_cookie=4e235e932f9fe63d in_seq=0 out_seq=0 2024-03-21T11:24:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.626+0000 7fd94e000640 10 client.25934 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:55.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.627+0000 7fc714ffc740 20 client.25945 trim_cache size 0 max 16384 2024-03-21T11:24:55.639 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:24:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.627+0000 7fd94e000640 1 -- 172.21.15.47:0/791912913 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562df7900d70 con 0x562df79a4230 2024-03-21T11:24:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.627+0000 7fd94e000640 10 client.25934 handle_command_reply: tid=0 2024-03-21T11:24:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fd96df8d740 1 client.25934 shutdown 2024-03-21T11:24:55.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.627+0000 7fc710e00640 1 -- 172.21.15.47:0/944248656 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc6f00a2fa0 msgr2=0x7fc6f00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.627+0000 7fc710e00640 1 --2- 172.21.15.47:0/944248656 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc6f00a2fa0 0x7fc6f00a5460 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5588c75842d0 tx=0x7fc6f8009260 comp rx=0 tx=0).stop 2024-03-21T11:24:55.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fc710e00640 1 -- 172.21.15.47:0/944248656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc70c109450 msgr2=0x7fc70c1a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fc710e00640 1 --2- 172.21.15.47:0/944248656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc70c109450 0x7fc70c1a0f90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc704036040 tx=0x7fc704004270 comp rx=0 tx=0).stop 2024-03-21T11:24:55.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fc710e00640 1 -- 172.21.15.47:0/944248656 shutdown_connections 2024-03-21T11:24:55.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fc710e00640 1 --2- 172.21.15.47:0/944248656 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc6f00a2fa0 0x7fc6f00a5460 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fc710e00640 1 --2- 172.21.15.47:0/944248656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc70c109450 0x7fc70c1a0f90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fc710e00640 1 --2- 172.21.15.47:0/944248656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c075660 0x7fc70c199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fc710e00640 1 --2- 172.21.15.47:0/944248656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc70c074080 0x7fc70c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fc710e00640 1 -- 172.21.15.47:0/944248656 >> 172.21.15.47:0/944248656 conn(0x7fc70c0fcb10 msgr2=0x7fc70c1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fd96df8d740 1 -- 172.21.15.47:0/791912913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df7993d40 msgr2=0x562df799f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fd96df8d740 1 --2- 172.21.15.47:0/791912913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df7993d40 0x562df799f9d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd95c035000 tx=0x7fd95c02bef0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fd96df8d740 1 -- 172.21.15.47:0/791912913 shutdown_connections 2024-03-21T11:24:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fd96df8d740 1 --2- 172.21.15.47:0/791912913 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562df79a4230 0x562df79a6620 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fd96df8d740 1 --2- 172.21.15.47:0/791912913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df7993d40 0x562df799f9d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fd96df8d740 1 --2- 172.21.15.47:0/791912913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df7998a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fd96df8d740 1 --2- 172.21.15.47:0/791912913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df798e1e0 0x562df79984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fd96df8d740 1 -- 172.21.15.47:0/791912913 >> 172.21.15.47:0/791912913 conn(0x562df7989750 msgr2=0x562df7988950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fc710e00640 1 -- 172.21.15.47:0/944248656 shutdown_connections 2024-03-21T11:24:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fd94e000640 0 client.25934 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:55.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fc710e00640 1 -- 172.21.15.47:0/944248656 wait complete. 2024-03-21T11:24:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fd96df8d740 1 -- 172.21.15.47:0/791912913 shutdown_connections 2024-03-21T11:24:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.628+0000 7fd96df8d740 1 -- 172.21.15.47:0/791912913 wait complete. 2024-03-21T11:24:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.629+0000 7fd96df8d740 20 client.25934 trim_cache size 0 max 16384 2024-03-21T11:24:55.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:55 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3919241894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:24:55.644 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:24:55.644 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:24:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.645+0000 7fd96df8d740 1 Processor -- start 2024-03-21T11:24:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.645+0000 7fd96df8d740 1 -- start start 2024-03-21T11:24:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.645+0000 7fd96df8d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df798e1e0 0x562df79843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.645+0000 7fd96df8d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df799f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.645+0000 7fd96df8d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 0x562df798c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.645+0000 7fd96df8d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9641b0600 con 0x562df7993d40 2024-03-21T11:24:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.645+0000 7fd96df8d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9641b1f80 con 0x562df798e1e0 2024-03-21T11:24:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.645+0000 7fd96df8d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9641b0c60 con 0x562df798ebd0 2024-03-21T11:24:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.645+0000 7fd963400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df799f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.645+0000 7fd963400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df799f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52714/0 (socket says 172.21.15.47:52714) 2024-03-21T11:24:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.645+0000 7fd963400640 1 -- 172.21.15.47:0/952822365 learned_addr learned my addr 172.21.15.47:0/952822365 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.646+0000 7fd969200640 1 --2- 172.21.15.47:0/952822365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 0x562df798c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.646+0000 7fd963e00640 1 --2- 172.21.15.47:0/952822365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df798e1e0 0x562df79843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.646+0000 7fd963400640 1 -- 172.21.15.47:0/952822365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df798e1e0 msgr2=0x562df79843a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.646+0000 7fd963400640 1 --2- 172.21.15.47:0/952822365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df798e1e0 0x562df79843a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.646+0000 7fd963400640 1 -- 172.21.15.47:0/952822365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 msgr2=0x562df798c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.646+0000 7fd963400640 1 --2- 172.21.15.47:0/952822365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 0x562df798c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.646+0000 7fd963400640 1 -- 172.21.15.47:0/952822365 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562df798a4c0 con 0x562df798ebd0 2024-03-21T11:24:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.646+0000 7fd963400640 1 --2- 172.21.15.47:0/952822365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df799f990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd9500096b0 tx=0x7fd9500096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56e954c805d6398f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.647+0000 7fd958c00640 1 -- 172.21.15.47:0/952822365 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950009780 con 0x562df798ebd0 2024-03-21T11:24:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.647+0000 7fd958c00640 1 -- 172.21.15.47:0/952822365 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd950005520 con 0x562df798ebd0 2024-03-21T11:24:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.647+0000 7fd958c00640 1 -- 172.21.15.47:0/952822365 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95001c480 con 0x562df798ebd0 2024-03-21T11:24:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.648+0000 7fd96df8d740 1 -- 172.21.15.47:0/952822365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 msgr2=0x562df799f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.648+0000 7fd96df8d740 1 --2- 172.21.15.47:0/952822365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df799f990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd9500096b0 tx=0x7fd9500096e0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.648+0000 7fd96df8d740 1 -- 172.21.15.47:0/952822365 shutdown_connections 2024-03-21T11:24:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.648+0000 7fd96df8d740 1 --2- 172.21.15.47:0/952822365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 0x562df798c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.648+0000 7fd96df8d740 1 --2- 172.21.15.47:0/952822365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df799f990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.648+0000 7fd96df8d740 1 --2- 172.21.15.47:0/952822365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df798e1e0 0x562df79843a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.648+0000 7fd96df8d740 1 -- 172.21.15.47:0/952822365 >> 172.21.15.47:0/952822365 conn(0x562df7989750 msgr2=0x562df79984c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.648+0000 7fd96df8d740 1 -- 172.21.15.47:0/952822365 shutdown_connections 2024-03-21T11:24:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.648+0000 7fd96df8d740 1 -- 172.21.15.47:0/952822365 wait complete. 2024-03-21T11:24:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.648+0000 7fd96df8d740 1 -- start start 2024-03-21T11:24:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.649+0000 7fd96df8d740 1 Processor -- start 2024-03-21T11:24:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.649+0000 7fd96df8d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df798e1e0 0x562df799d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.649+0000 7fd96df8d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df799de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.649+0000 7fd96df8d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 0x562df799edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.649+0000 7fd96df8d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9641b1920 con 0x562df7993d40 2024-03-21T11:24:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.649+0000 7fd96df8d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9641b12c0 con 0x562df798e1e0 2024-03-21T11:24:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.649+0000 7fd96df8d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9641ae7c0 con 0x562df798ebd0 2024-03-21T11:24:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.649+0000 7fd96df8d740 10 client.0 _ll_register_callbacks cb 0x7fd950005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.649+0000 7fd963400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df799de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.649+0000 7fd963400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df799de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52724/0 (socket says 172.21.15.47:52724) 2024-03-21T11:24:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.649+0000 7fd963400640 1 -- 172.21.15.47:0/1693935126 learned_addr learned my addr 172.21.15.47:0/1693935126 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.650+0000 7fd963e00640 1 --2- 172.21.15.47:0/1693935126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df798e1e0 0x562df799d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.650+0000 7fd963400640 1 -- 172.21.15.47:0/1693935126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df798e1e0 msgr2=0x562df799d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.650+0000 7fd963400640 1 --2- 172.21.15.47:0/1693935126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df798e1e0 0x562df799d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.650+0000 7fd963400640 1 -- 172.21.15.47:0/1693935126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 msgr2=0x562df799edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.650+0000 7fd963400640 1 --2- 172.21.15.47:0/1693935126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 0x562df799edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.650+0000 7fd963400640 1 -- 172.21.15.47:0/1693935126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562df799f3c0 con 0x562df798ebd0 2024-03-21T11:24:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.650+0000 7fd963400640 1 --2- 172.21.15.47:0/1693935126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df799de80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd950009c00 tx=0x7fd950004870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df2f8397d416475c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.650+0000 7fd94e000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.650+0000 7fd94e000640 1 -- 172.21.15.47:0/1693935126 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950034060 con 0x562df798ebd0 2024-03-21T11:24:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.651+0000 7fd96df8d740 1 -- 172.21.15.47:0/1693935126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd950005520 con 0x562df798ebd0 2024-03-21T11:24:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.651+0000 7fd94e000640 1 -- 172.21.15.47:0/1693935126 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd950004f10 con 0x562df798ebd0 2024-03-21T11:24:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.651+0000 7fd94e000640 1 -- 172.21.15.47:0/1693935126 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd950005110 con 0x562df798ebd0 2024-03-21T11:24:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.651+0000 7fd96df8d740 10 client.25951 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.651+0000 7fd96df8d740 1 -- 172.21.15.47:0/1693935126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562df7a00300 con 0x562df798ebd0 2024-03-21T11:24:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.651+0000 7fd94e000640 1 -- 172.21.15.47:0/1693935126 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd950005520 con 0x562df798ebd0 2024-03-21T11:24:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.651+0000 7fd94e000640 1 -- 172.21.15.47:0/1693935126 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd95002f460 con 0x562df798ebd0 2024-03-21T11:24:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.651+0000 7fd96df8d740 10 client.25951 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.651+0000 7fd96df8d740 10 client.25951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.651+0000 7fd96df8d740 20 client.25951 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.651+0000 7fd96df8d740 1 --2- 172.21.15.47:0/1693935126 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562df7a00990 0x562df7a02db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.652+0000 7fd96df8d740 4 client.25951 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:24:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.652+0000 7fd96df8d740 1 -- 172.21.15.47:0/1693935126 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562df7984a80 con 0x562df7a00990 2024-03-21T11:24:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.652+0000 7fd963e00640 1 --2- 172.21.15.47:0/1693935126 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562df7a00990 0x562df7a02db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.652+0000 7fd963e00640 1 --2- 172.21.15.47:0/1693935126 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562df7a00990 0x562df7a02db0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92948600df83a130 server_cookie=bc22538572ac4342 in_seq=0 out_seq=0 2024-03-21T11:24:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.652+0000 7fd94e000640 10 client.25951 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.657+0000 7fd94e000640 1 -- 172.21.15.47:0/1693935126 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562df7984a80 con 0x562df7a00990 2024-03-21T11:24:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.657+0000 7fd94e000640 10 client.25951 handle_command_reply: tid=0 2024-03-21T11:24:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.657+0000 7fd96df8d740 1 client.25951 shutdown 2024-03-21T11:24:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.657+0000 7fd96df8d740 1 -- 172.21.15.47:0/1693935126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 msgr2=0x562df799de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.657+0000 7fd96df8d740 1 --2- 172.21.15.47:0/1693935126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df799de80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd950009c00 tx=0x7fd950004870 comp rx=0 tx=0).stop 2024-03-21T11:24:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.657+0000 7fd96df8d740 1 -- 172.21.15.47:0/1693935126 shutdown_connections 2024-03-21T11:24:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.657+0000 7fd96df8d740 1 --2- 172.21.15.47:0/1693935126 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562df7a00990 0x562df7a02db0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.657+0000 7fd96df8d740 1 --2- 172.21.15.47:0/1693935126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562df7993d40 0x562df799edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.657+0000 7fd96df8d740 1 --2- 172.21.15.47:0/1693935126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562df798ebd0 0x562df799de80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.657+0000 7fd96df8d740 1 --2- 172.21.15.47:0/1693935126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562df798e1e0 0x562df799d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.657+0000 7fd96df8d740 1 -- 172.21.15.47:0/1693935126 >> 172.21.15.47:0/1693935126 conn(0x562df7989750 msgr2=0x562df798c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.657+0000 7fd94e000640 0 client.25951 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.658+0000 7fd96df8d740 1 -- 172.21.15.47:0/1693935126 shutdown_connections 2024-03-21T11:24:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.658+0000 7fd96df8d740 1 -- 172.21.15.47:0/1693935126 wait complete. 2024-03-21T11:24:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.658+0000 7fd96df8d740 20 client.25951 trim_cache size 0 max 16384 2024-03-21T11:24:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:24:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:24:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:24:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:24:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.659+0000 7fd96b200640 1 -- 172.21.15.47:0/93928723 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9480a3040 msgr2=0x7fd9480a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.659+0000 7fd96b200640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9480a3040 0x7fd9480a5500 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd95c004af0 tx=0x7fd95c02e040 comp rx=0 tx=0).stop 2024-03-21T11:24:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.659+0000 7fd96b200640 1 -- 172.21.15.47:0/93928723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd964103900 msgr2=0x7fd964079450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.659+0000 7fd96b200640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd964103900 0x7fd964079450 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd95400d950 tx=0x7fd95400de10 comp rx=0 tx=0).stop 2024-03-21T11:24:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.659+0000 7fd96b200640 1 -- 172.21.15.47:0/93928723 shutdown_connections 2024-03-21T11:24:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.659+0000 7fd96b200640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd9480a3040 0x7fd9480a5500 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.659+0000 7fd96b200640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9641054b0 0x7fd964075fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.659+0000 7fd96b200640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd964104af0 0x7fd964079990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.659+0000 7fd96b200640 1 --2- 172.21.15.47:0/93928723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd964103900 0x7fd964079450 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.659+0000 7fd96b200640 1 -- 172.21.15.47:0/93928723 >> 172.21.15.47:0/93928723 conn(0x7fd9640fee90 msgr2=0x7fd964107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.659+0000 7fd96b200640 1 -- 172.21.15.47:0/93928723 shutdown_connections 2024-03-21T11:24:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:24:55.660+0000 7fd96b200640 1 -- 172.21.15.47:0/93928723 wait complete. 2024-03-21T11:24:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.881+0000 7f4364400640 1 -- 172.21.15.47:0/3967437280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f435c105490 msgr2=0x7f435c10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.881+0000 7f4364400640 1 --2- 172.21.15.47:0/3967437280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f435c105490 0x7f435c10bd20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4354009ca0 tx=0x7f435402c0f0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.881+0000 7f4364400640 1 -- 172.21.15.47:0/3967437280 shutdown_connections 2024-03-21T11:24:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.881+0000 7f4364400640 1 --2- 172.21.15.47:0/3967437280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f435c105490 0x7f435c10bd20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.881+0000 7f4364400640 1 --2- 172.21.15.47:0/3967437280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c104ad0 0x7f435c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.881+0000 7f4364400640 1 --2- 172.21.15.47:0/3967437280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f435c1038e0 0x7f435c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.881+0000 7f4364400640 1 -- 172.21.15.47:0/3967437280 >> 172.21.15.47:0/3967437280 conn(0x7f435c0fee50 msgr2=0x7f435c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.881+0000 7f4364400640 1 -- 172.21.15.47:0/3967437280 shutdown_connections 2024-03-21T11:24:55.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.881+0000 7f4364400640 1 -- 172.21.15.47:0/3967437280 wait complete. 2024-03-21T11:24:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.883+0000 7f4364400640 1 Processor -- start 2024-03-21T11:24:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.883+0000 7f4364400640 1 -- start start 2024-03-21T11:24:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.883+0000 7f4364400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c1038e0 0x7f435c199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.883+0000 7f4364400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f435c104ad0 0x7f435c19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.883+0000 7f4364400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f435c105490 0x7f435c1a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.883+0000 7f4364400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c10f560 con 0x7f435c1038e0 2024-03-21T11:24:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.883+0000 7f4364400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c10f3e0 con 0x7f435c104ad0 2024-03-21T11:24:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.883+0000 7f4364400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c10f6e0 con 0x7f435c105490 2024-03-21T11:24:55.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.884+0000 7f4362400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f435c105490 0x7f435c1a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.884+0000 7f4361000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f435c104ad0 0x7f435c19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.884+0000 7f4362400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f435c105490 0x7f435c1a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52730/0 (socket says 172.21.15.47:52730) 2024-03-21T11:24:55.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.884+0000 7f4361000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f435c104ad0 0x7f435c19a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40600/0 (socket says 172.21.15.47:40600) 2024-03-21T11:24:55.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.884+0000 7f4362400640 1 -- 172.21.15.47:0/2000011034 learned_addr learned my addr 172.21.15.47:0/2000011034 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.884+0000 7f4361a00640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c1038e0 0x7f435c199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.884+0000 7f4361000640 1 -- 172.21.15.47:0/2000011034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f435c105490 msgr2=0x7f435c1a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.884+0000 7f4361000640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f435c105490 0x7f435c1a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.884+0000 7f4361000640 1 -- 172.21.15.47:0/2000011034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c1038e0 msgr2=0x7f435c199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.884+0000 7f4361000640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c1038e0 0x7f435c199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.884+0000 7f4361000640 1 -- 172.21.15.47:0/2000011034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c1a17d0 con 0x7f435c104ad0 2024-03-21T11:24:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.884+0000 7f4362400640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f435c105490 0x7f435c1a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.884+0000 7f4361a00640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c1038e0 0x7f435c199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.885+0000 7f4361000640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f435c104ad0 0x7f435c19a0f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f434800e9f0 tx=0x7f434800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.885+0000 7f435a000640 1 -- 172.21.15.47:0/2000011034 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434800ccc0 con 0x7f435c104ad0 2024-03-21T11:24:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.885+0000 7f435a000640 1 -- 172.21.15.47:0/2000011034 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4348010910 con 0x7f435c104ad0 2024-03-21T11:24:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.885+0000 7f4364400640 1 -- 172.21.15.47:0/2000011034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435c1a1ac0 con 0x7f435c104ad0 2024-03-21T11:24:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.885+0000 7f435a000640 1 -- 172.21.15.47:0/2000011034 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348018960 con 0x7f435c104ad0 2024-03-21T11:24:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.885+0000 7f4364400640 1 -- 172.21.15.47:0/2000011034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435c1a2060 con 0x7f435c104ad0 2024-03-21T11:24:55.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.888+0000 7f435a000640 1 -- 172.21.15.47:0/2000011034 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4348010ac0 con 0x7f435c104ad0 2024-03-21T11:24:55.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.889+0000 7f435a000640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f43440a2fa0 0x7f43440a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.890+0000 7f435a000640 1 -- 172.21.15.47:0/2000011034 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f43480cfb40 con 0x7f435c104ad0 2024-03-21T11:24:55.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.890+0000 7f4367032740 1 Processor -- start 2024-03-21T11:24:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.890+0000 7f4361a00640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f43440a2fa0 0x7f43440a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.890+0000 7f4367032740 1 -- start start 2024-03-21T11:24:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4367032740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e231020 0x55848e231440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4367032740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e2321f0 0x55848e232670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4367032740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e232bb0 0x55848e239440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4367032740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848e18d2c0 con 0x55848e2321f0 2024-03-21T11:24:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4367032740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848e135c90 con 0x55848e231020 2024-03-21T11:24:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4367032740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848e189360 con 0x55848e232bb0 2024-03-21T11:24:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4361000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e2321f0 0x55848e232670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4361000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e2321f0 0x55848e232670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53222/0 (socket says 172.21.15.47:53222) 2024-03-21T11:24:55.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4361000640 1 -- 172.21.15.47:0/3558556117 learned_addr learned my addr 172.21.15.47:0/3558556117 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4362400640 1 --2- 172.21.15.47:0/3558556117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e232bb0 0x55848e239440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4361000640 1 -- 172.21.15.47:0/3558556117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e232bb0 msgr2=0x55848e239440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4361000640 1 --2- 172.21.15.47:0/3558556117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e232bb0 0x55848e239440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4361000640 1 -- 172.21.15.47:0/3558556117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e231020 msgr2=0x55848e231440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:24:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4361000640 1 --2- 172.21.15.47:0/3558556117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e231020 0x55848e231440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4361000640 1 -- 172.21.15.47:0/3558556117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55848e15a080 con 0x55848e2321f0 2024-03-21T11:24:55.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.891+0000 7f4361a00640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f43440a2fa0 0x7f43440a5460 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f435c104930 tx=0x7f4350005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.892+0000 7f4361000640 1 --2- 172.21.15.47:0/3558556117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e2321f0 0x55848e232670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4348098850 tx=0x7f434809c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d848024d5e16038 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.892+0000 7f434ea00640 1 -- 172.21.15.47:0/3558556117 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434809ffd0 con 0x55848e2321f0 2024-03-21T11:24:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.892+0000 7f434ea00640 1 -- 172.21.15.47:0/3558556117 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43480a4890 con 0x55848e2321f0 2024-03-21T11:24:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.892+0000 7f434ea00640 1 -- 172.21.15.47:0/3558556117 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348018b00 con 0x55848e2321f0 2024-03-21T11:24:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.893+0000 7f4367032740 1 -- 172.21.15.47:0/3558556117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e2321f0 msgr2=0x55848e232670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.893+0000 7f4367032740 1 --2- 172.21.15.47:0/3558556117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e2321f0 0x55848e232670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4348098850 tx=0x7f434809c580 comp rx=0 tx=0).stop 2024-03-21T11:24:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.893+0000 7f4367032740 1 -- 172.21.15.47:0/3558556117 shutdown_connections 2024-03-21T11:24:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.893+0000 7f4367032740 1 --2- 172.21.15.47:0/3558556117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e232bb0 0x55848e239440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.893+0000 7f4367032740 1 --2- 172.21.15.47:0/3558556117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e2321f0 0x55848e232670 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.893+0000 7f4367032740 1 --2- 172.21.15.47:0/3558556117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e231020 0x55848e231440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.893+0000 7f4367032740 1 -- 172.21.15.47:0/3558556117 >> 172.21.15.47:0/3558556117 conn(0x55848e223c00 msgr2=0x55848e224000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.893+0000 7f4367032740 1 -- 172.21.15.47:0/3558556117 shutdown_connections 2024-03-21T11:24:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.893+0000 7f4367032740 1 -- 172.21.15.47:0/3558556117 wait complete. 2024-03-21T11:24:55.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.894+0000 7f4367032740 1 -- start start 2024-03-21T11:24:55.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.894+0000 7f4367032740 1 Processor -- start 2024-03-21T11:24:55.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.895+0000 7f4367032740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e231020 0x55848e23b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.895+0000 7f4367032740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e232bb0 0x55848e23ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.895+0000 7f4367032740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e240a50 0x55848e242e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.895+0000 7f4367032740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848e239b60 con 0x55848e240a50 2024-03-21T11:24:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.895+0000 7f4367032740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848e1490d0 con 0x55848e232bb0 2024-03-21T11:24:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.895+0000 7f4367032740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848e239ce0 con 0x55848e231020 2024-03-21T11:24:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.895+0000 7f4367032740 10 client.0 _ll_register_callbacks cb 0x55848e14de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.895+0000 7f4361000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e232bb0 0x55848e23ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.895+0000 7f4361000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e232bb0 0x55848e23ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40624/0 (socket says 172.21.15.47:40624) 2024-03-21T11:24:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.895+0000 7f4361000640 1 -- 172.21.15.47:0/2770358670 learned_addr learned my addr 172.21.15.47:0/2770358670 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.895+0000 7f4361a00640 1 --2- 172.21.15.47:0/2770358670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e231020 0x55848e23b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.895+0000 7f4361000640 1 -- 172.21.15.47:0/2770358670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e231020 msgr2=0x55848e23b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.896+0000 7f4361000640 1 --2- 172.21.15.47:0/2770358670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e231020 0x55848e23b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.896+0000 7f4361000640 1 -- 172.21.15.47:0/2770358670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e240a50 msgr2=0x55848e242e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.896+0000 7f4361000640 1 --2- 172.21.15.47:0/2770358670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e240a50 0x55848e242e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.896+0000 7f4361000640 1 -- 172.21.15.47:0/2770358670 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55848dea13f0 con 0x55848e232bb0 2024-03-21T11:24:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.896+0000 7f4361a00640 1 --2- 172.21.15.47:0/2770358670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e231020 0x55848e23b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.896+0000 7f4367032740 1 -- 172.21.15.47:0/2770358670 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43480a4890 con 0x55848e232bb0 2024-03-21T11:24:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.896+0000 7f4361000640 1 --2- 172.21.15.47:0/2770358670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e232bb0 0x55848e23ba90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f434800c6a0 tx=0x7f43480bc050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e09528cb3f0c6b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.896+0000 7f4343400640 10 client.25949 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:24:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.896+0000 7f4343400640 1 -- 172.21.15.47:0/2770358670 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434809b4b0 con 0x55848e232bb0 2024-03-21T11:24:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.896+0000 7f4343400640 1 -- 172.21.15.47:0/2770358670 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f43480850a0 con 0x55848e232bb0 2024-03-21T11:24:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.896+0000 7f4343400640 1 -- 172.21.15.47:0/2770358670 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4348018b00 con 0x55848e232bb0 2024-03-21T11:24:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.896+0000 7f4367032740 10 client.25949 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.896+0000 7f4367032740 1 -- 172.21.15.47:0/2770358670 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=28}) v3 -- 0x55848df6b7d0 con 0x55848e232bb0 2024-03-21T11:24:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.897+0000 7f4343400640 1 -- 172.21.15.47:0/2770358670 <== mon.2 v2:172.21.15.100:3300/0 4 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4348082030 con 0x55848e232bb0 2024-03-21T11:24:55.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.897+0000 7f4367032740 10 client.25949 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:55.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.897+0000 7f4367032740 10 client.25949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:55.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.897+0000 7f4367032740 20 client.25949 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:55.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.897+0000 7f4367032740 1 --2- 172.21.15.47:0/2770358670 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55848e246ee0 0x55848e2492d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.897+0000 7f4367032740 4 client.25949 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:55.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.897+0000 7f4367032740 1 -- 172.21.15.47:0/2770358670 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55848e19a800 con 0x55848e246ee0 2024-03-21T11:24:55.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.897+0000 7f4361a00640 1 --2- 172.21.15.47:0/2770358670 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55848e246ee0 0x55848e2492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.898+0000 7f4361a00640 1 --2- 172.21.15.47:0/2770358670 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55848e246ee0 0x55848e2492d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea1f0ed391f743 server_cookie=deab674f136d0060 in_seq=0 out_seq=0 2024-03-21T11:24:55.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.898+0000 7f4343400640 10 client.25949 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:55.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.899+0000 7f4343400640 1 -- 172.21.15.47:0/2770358670 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55848e19a800 con 0x55848e246ee0 2024-03-21T11:24:55.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.899+0000 7f4343400640 10 client.25949 handle_command_reply: tid=0 2024-03-21T11:24:55.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.899+0000 7f4367032740 1 client.25949 shutdown 2024-03-21T11:24:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.899+0000 7f4367032740 1 -- 172.21.15.47:0/2770358670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e232bb0 msgr2=0x55848e23ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.899+0000 7f4367032740 1 --2- 172.21.15.47:0/2770358670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e232bb0 0x55848e23ba90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f434800c6a0 tx=0x7f43480bc050 comp rx=0 tx=0).stop 2024-03-21T11:24:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.899+0000 7f4367032740 1 -- 172.21.15.47:0/2770358670 shutdown_connections 2024-03-21T11:24:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.899+0000 7f4367032740 1 --2- 172.21.15.47:0/2770358670 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55848e246ee0 0x55848e2492d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.899+0000 7f4367032740 1 --2- 172.21.15.47:0/2770358670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e240a50 0x55848e242e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.899+0000 7f4367032740 1 --2- 172.21.15.47:0/2770358670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e232bb0 0x55848e23ba90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.899+0000 7f4343400640 0 client.25949 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.899+0000 7f4367032740 1 --2- 172.21.15.47:0/2770358670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e231020 0x55848e23b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.899+0000 7f4367032740 1 -- 172.21.15.47:0/2770358670 >> 172.21.15.47:0/2770358670 conn(0x55848e223c00 msgr2=0x55848e222eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.899+0000 7f4367032740 1 -- 172.21.15.47:0/2770358670 shutdown_connections 2024-03-21T11:24:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.900+0000 7f4367032740 1 -- 172.21.15.47:0/2770358670 wait complete. 2024-03-21T11:24:55.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.900+0000 7f4367032740 20 client.25949 trim_cache size 0 max 16384 2024-03-21T11:24:55.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.926+0000 7f4367032740 1 Processor -- start 2024-03-21T11:24:55.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.926+0000 7f4367032740 1 -- start start 2024-03-21T11:24:55.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.926+0000 7f4367032740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e231020 0x55848e23a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.926+0000 7f4367032740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e232bb0 0x55848e235470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.926+0000 7f4367032740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e240a50 0x55848e2359b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.926+0000 7f4367032740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1a7f70 con 0x55848e232bb0 2024-03-21T11:24:55.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.926+0000 7f4367032740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1a98f0 con 0x55848e231020 2024-03-21T11:24:55.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.926+0000 7f4367032740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1a85d0 con 0x55848e240a50 2024-03-21T11:24:55.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.926+0000 7f4361a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e231020 0x55848e23a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.926+0000 7f4361a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e231020 0x55848e23a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40630/0 (socket says 172.21.15.47:40630) 2024-03-21T11:24:55.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.926+0000 7f4361a00640 1 -- 172.21.15.47:0/2425572451 learned_addr learned my addr 172.21.15.47:0/2425572451 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.927+0000 7f4362400640 1 --2- 172.21.15.47:0/2425572451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e240a50 0x55848e2359b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.927+0000 7f4361000640 1 --2- 172.21.15.47:0/2425572451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e232bb0 0x55848e235470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.927+0000 7f4361a00640 1 -- 172.21.15.47:0/2425572451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e240a50 msgr2=0x55848e2359b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.927+0000 7f4361a00640 1 --2- 172.21.15.47:0/2425572451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e240a50 0x55848e2359b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.927+0000 7f4361a00640 1 -- 172.21.15.47:0/2425572451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e232bb0 msgr2=0x55848e235470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.927+0000 7f4361a00640 1 --2- 172.21.15.47:0/2425572451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e232bb0 0x55848e235470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.927+0000 7f4361a00640 1 -- 172.21.15.47:0/2425572451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55848e224970 con 0x55848e231020 2024-03-21T11:24:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.927+0000 7f4362400640 1 --2- 172.21.15.47:0/2425572451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e240a50 0x55848e2359b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.927+0000 7f4361000640 1 --2- 172.21.15.47:0/2425572451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e232bb0 0x55848e235470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.927+0000 7f4361a00640 1 --2- 172.21.15.47:0/2425572451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e231020 0x55848e23a0b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4350011ce0 tx=0x7f4350014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cc46ca7f46211c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.928+0000 7f434e000640 1 -- 172.21.15.47:0/2425572451 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435002d070 con 0x55848e231020 2024-03-21T11:24:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.928+0000 7f434e000640 1 -- 172.21.15.47:0/2425572451 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4350016070 con 0x55848e231020 2024-03-21T11:24:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.928+0000 7f434e000640 1 -- 172.21.15.47:0/2425572451 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350027de0 con 0x55848e231020 2024-03-21T11:24:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.929+0000 7f4367032740 1 -- 172.21.15.47:0/2425572451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e231020 msgr2=0x55848e23a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.929+0000 7f4367032740 1 --2- 172.21.15.47:0/2425572451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e231020 0x55848e23a0b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4350011ce0 tx=0x7f4350014000 comp rx=0 tx=0).stop 2024-03-21T11:24:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.929+0000 7f4367032740 1 -- 172.21.15.47:0/2425572451 shutdown_connections 2024-03-21T11:24:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.929+0000 7f4367032740 1 --2- 172.21.15.47:0/2425572451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e240a50 0x55848e2359b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.929+0000 7f4367032740 1 --2- 172.21.15.47:0/2425572451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e232bb0 0x55848e235470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.929+0000 7f4367032740 1 --2- 172.21.15.47:0/2425572451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e231020 0x55848e23a0b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.929+0000 7f4367032740 1 -- 172.21.15.47:0/2425572451 >> 172.21.15.47:0/2425572451 conn(0x55848e223c00 msgr2=0x55848e2276f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.929+0000 7f4367032740 1 -- 172.21.15.47:0/2425572451 shutdown_connections 2024-03-21T11:24:55.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.929+0000 7f4367032740 1 -- 172.21.15.47:0/2425572451 wait complete. 2024-03-21T11:24:55.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.929+0000 7f4367032740 1 -- start start 2024-03-21T11:24:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.930+0000 7f4367032740 1 Processor -- start 2024-03-21T11:24:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.930+0000 7f4367032740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e231020 0x55848e2564b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.930+0000 7f4367032740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e232bb0 0x55848e2569f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.930+0000 7f4367032740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e240a50 0x55848e2579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.930+0000 7f4367032740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1a9290 con 0x55848e240a50 2024-03-21T11:24:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.930+0000 7f4367032740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1a8c30 con 0x55848e232bb0 2024-03-21T11:24:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.930+0000 7f4367032740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1a6130 con 0x55848e231020 2024-03-21T11:24:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.930+0000 7f4367032740 10 client.0 _ll_register_callbacks cb 0x7f43480b3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.931+0000 7f4361a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e231020 0x55848e2564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.931+0000 7f4361000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e232bb0 0x55848e2569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.931+0000 7f4361a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e231020 0x55848e2564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52768/0 (socket says 172.21.15.47:52768) 2024-03-21T11:24:55.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.931+0000 7f4361a00640 1 -- 172.21.15.47:0/1742939636 learned_addr learned my addr 172.21.15.47:0/1742939636 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.931+0000 7f4362400640 1 --2- 172.21.15.47:0/1742939636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e240a50 0x55848e2579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.931+0000 7f4361a00640 1 -- 172.21.15.47:0/1742939636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e232bb0 msgr2=0x55848e2569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.931+0000 7f4361a00640 1 --2- 172.21.15.47:0/1742939636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e232bb0 0x55848e2569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.931+0000 7f4361a00640 1 -- 172.21.15.47:0/1742939636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e240a50 msgr2=0x55848e2579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.931+0000 7f4361a00640 1 --2- 172.21.15.47:0/1742939636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e240a50 0x55848e2579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.931+0000 7f4361a00640 1 -- 172.21.15.47:0/1742939636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55848e257fa0 con 0x55848e231020 2024-03-21T11:24:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.931+0000 7f4362400640 1 --2- 172.21.15.47:0/1742939636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e240a50 0x55848e2579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.931+0000 7f4361a00640 1 --2- 172.21.15.47:0/1742939636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e231020 0x55848e2564b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4350005760 tx=0x7f4350009d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfc02b7fe3d27897 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.931+0000 7f4343400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.932+0000 7f4343400640 1 -- 172.21.15.47:0/1742939636 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350036060 con 0x55848e231020 2024-03-21T11:24:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.932+0000 7f4367032740 1 -- 172.21.15.47:0/1742939636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4350016070 con 0x55848e231020 2024-03-21T11:24:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.932+0000 7f4343400640 1 -- 172.21.15.47:0/1742939636 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4350027b80 con 0x55848e231020 2024-03-21T11:24:55.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.932+0000 7f4343400640 1 -- 172.21.15.47:0/1742939636 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f435002d040 con 0x55848e231020 2024-03-21T11:24:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.932+0000 7f4367032740 10 client.25966 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.932+0000 7f4367032740 1 -- 172.21.15.47:0/1742939636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55848e259260 con 0x55848e231020 2024-03-21T11:24:55.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.932+0000 7f4343400640 1 -- 172.21.15.47:0/1742939636 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4350016070 con 0x55848e231020 2024-03-21T11:24:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.932+0000 7f4343400640 1 -- 172.21.15.47:0/1742939636 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4350012680 con 0x55848e231020 2024-03-21T11:24:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.933+0000 7f4367032740 10 client.25966 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.933+0000 7f4367032740 10 client.25966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.933+0000 7f4367032740 20 client.25966 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.933+0000 7f4367032740 1 --2- 172.21.15.47:0/1742939636 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55848e246ee0 0x55848e2a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:55.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.933+0000 7f4367032740 4 client.25966 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:24:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.933+0000 7f4367032740 1 -- 172.21.15.47:0/1742939636 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55848e194b50 con 0x55848e246ee0 2024-03-21T11:24:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.933+0000 7f4361000640 1 --2- 172.21.15.47:0/1742939636 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55848e246ee0 0x55848e2a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.933+0000 7f4361000640 1 --2- 172.21.15.47:0/1742939636 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55848e246ee0 0x55848e2a5d00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7ec774bcb7fcb5 server_cookie=c9b5ec20e66e83ef in_seq=0 out_seq=0 2024-03-21T11:24:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.933+0000 7f4343400640 10 client.25966 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.933+0000 7f4343400640 1 -- 172.21.15.47:0/1742939636 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55848e194b50 con 0x55848e246ee0 2024-03-21T11:24:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.933+0000 7f4343400640 10 client.25966 handle_command_reply: tid=0 2024-03-21T11:24:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.933+0000 7f4367032740 1 client.25966 shutdown 2024-03-21T11:24:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.934+0000 7f4367032740 1 -- 172.21.15.47:0/1742939636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e231020 msgr2=0x55848e2564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.934+0000 7f4367032740 1 --2- 172.21.15.47:0/1742939636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e231020 0x55848e2564b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4350005760 tx=0x7f4350009d30 comp rx=0 tx=0).stop 2024-03-21T11:24:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.934+0000 7f4367032740 1 -- 172.21.15.47:0/1742939636 shutdown_connections 2024-03-21T11:24:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.934+0000 7f4367032740 1 --2- 172.21.15.47:0/1742939636 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55848e246ee0 0x55848e2a5d00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.934+0000 7f4367032740 1 --2- 172.21.15.47:0/1742939636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55848e240a50 0x55848e2579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.934+0000 7f4367032740 1 --2- 172.21.15.47:0/1742939636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55848e232bb0 0x55848e2569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.934+0000 7f4367032740 1 --2- 172.21.15.47:0/1742939636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55848e231020 0x55848e2564b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.934+0000 7f4367032740 1 -- 172.21.15.47:0/1742939636 >> 172.21.15.47:0/1742939636 conn(0x55848e223c00 msgr2=0x55848e222620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.934+0000 7f4343400640 0 client.25966 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:55.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.934+0000 7f4367032740 1 -- 172.21.15.47:0/1742939636 shutdown_connections 2024-03-21T11:24:55.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.934+0000 7f4367032740 1 -- 172.21.15.47:0/1742939636 wait complete. 2024-03-21T11:24:55.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.934+0000 7f4367032740 20 client.25966 trim_cache size 0 max 16384 2024-03-21T11:24:55.938 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:24:55.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.935+0000 7f4364400640 1 -- 172.21.15.47:0/2000011034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f43440a2fa0 msgr2=0x7f43440a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.935+0000 7f4364400640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f43440a2fa0 0x7f43440a5460 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f435c104930 tx=0x7f4350005f90 comp rx=0 tx=0).stop 2024-03-21T11:24:55.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.935+0000 7f4364400640 1 -- 172.21.15.47:0/2000011034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f435c104ad0 msgr2=0x7f435c19a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:55.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.935+0000 7f4364400640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f435c104ad0 0x7f435c19a0f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f434800e9f0 tx=0x7f434800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.935+0000 7f4364400640 1 -- 172.21.15.47:0/2000011034 shutdown_connections 2024-03-21T11:24:55.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.935+0000 7f4364400640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f43440a2fa0 0x7f43440a5460 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.935+0000 7f4364400640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f435c105490 0x7f435c1a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.935+0000 7f4364400640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f435c104ad0 0x7f435c19a0f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.935+0000 7f4364400640 1 --2- 172.21.15.47:0/2000011034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f435c1038e0 0x7f435c199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:55.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.935+0000 7f4364400640 1 -- 172.21.15.47:0/2000011034 >> 172.21.15.47:0/2000011034 conn(0x7f435c0fee50 msgr2=0x7f435c107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:55.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.935+0000 7f4364400640 1 -- 172.21.15.47:0/2000011034 shutdown_connections 2024-03-21T11:24:55.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:55.935+0000 7f4364400640 1 -- 172.21.15.47:0/2000011034 wait complete. 2024-03-21T11:24:56.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:56 smithi102 ceph-mon[36748]: pgmap v448: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-21T11:24:56.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:56 smithi047 ceph-mon[31351]: pgmap v448: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-21T11:24:56.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:56 smithi100 ceph-mon[36245]: pgmap v448: 129 pgs: 129 active+clean; 676 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-21T11:24:56.956 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:24:56.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:24:56.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:24:57.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.184+0000 7fccf2c00640 1 -- 172.21.15.47:0/2373951919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccec1015a0 msgr2=0x7fccec1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.184+0000 7fccf2c00640 1 --2- 172.21.15.47:0/2373951919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccec1015a0 0x7fccec1019c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009900 tx=0x7fcce002bc40 comp rx=0 tx=0).stop 2024-03-21T11:24:57.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.184+0000 7fccf2c00640 1 -- 172.21.15.47:0/2373951919 shutdown_connections 2024-03-21T11:24:57.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.184+0000 7fccf2c00640 1 --2- 172.21.15.47:0/2373951919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccec103150 0x7fccec1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.184+0000 7fccf2c00640 1 --2- 172.21.15.47:0/2373951919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccec102790 0x7fccec102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.184+0000 7fccf2c00640 1 --2- 172.21.15.47:0/2373951919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccec1015a0 0x7fccec1019c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.184+0000 7fccf2c00640 1 -- 172.21.15.47:0/2373951919 >> 172.21.15.47:0/2373951919 conn(0x7fccec0fcb10 msgr2=0x7fccec0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:57.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.184+0000 7fccf2c00640 1 -- 172.21.15.47:0/2373951919 shutdown_connections 2024-03-21T11:24:57.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.184+0000 7fccf2c00640 1 -- 172.21.15.47:0/2373951919 wait complete. 2024-03-21T11:24:57.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.185+0000 7fccf2c00640 1 Processor -- start 2024-03-21T11:24:57.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.185+0000 7fccf2c00640 1 -- start start 2024-03-21T11:24:57.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.185+0000 7fccf2c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccec1015a0 0x7fccec1956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.185+0000 7fccf2c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccec102790 0x7fccec195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf2c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccec103150 0x7fccec19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf2c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec10b090 con 0x7fccec1015a0 2024-03-21T11:24:57.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf2c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec10af10 con 0x7fccec102790 2024-03-21T11:24:57.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf2c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec10b210 con 0x7fccec103150 2024-03-21T11:24:57.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccec102790 0x7fccec195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccec102790 0x7fccec195bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40648/0 (socket says 172.21.15.47:40648) 2024-03-21T11:24:57.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf0e00640 1 -- 172.21.15.47:0/258346101 learned_addr learned my addr 172.21.15.47:0/258346101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:57.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf1800640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccec1015a0 0x7fccec1956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf2200640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccec103150 0x7fccec19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf0e00640 1 -- 172.21.15.47:0/258346101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccec103150 msgr2=0x7fccec19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf0e00640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccec103150 0x7fccec19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf0e00640 1 -- 172.21.15.47:0/258346101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccec1015a0 msgr2=0x7fccec1956b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf0e00640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccec1015a0 0x7fccec1956b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf0e00640 1 -- 172.21.15.47:0/258346101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec19d2d0 con 0x7fccec102790 2024-03-21T11:24:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf2200640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccec103150 0x7fccec19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.186+0000 7fccf1800640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccec1015a0 0x7fccec1956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:24:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.187+0000 7fccf0e00640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccec102790 0x7fccec195bf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fccd800ea60 tx=0x7fccd800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.187+0000 7fccea000640 1 -- 172.21.15.47:0/258346101 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd800cde0 con 0x7fccec102790 2024-03-21T11:24:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.187+0000 7fccea000640 1 -- 172.21.15.47:0/258346101 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccd80035f0 con 0x7fccec102790 2024-03-21T11:24:57.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.187+0000 7fccea000640 1 -- 172.21.15.47:0/258346101 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8010670 con 0x7fccec102790 2024-03-21T11:24:57.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.187+0000 7fccf2c00640 1 -- 172.21.15.47:0/258346101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccec19d5c0 con 0x7fccec102790 2024-03-21T11:24:57.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.187+0000 7fccf2c00640 1 -- 172.21.15.47:0/258346101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec19dac0 con 0x7fccec102790 2024-03-21T11:24:57.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.192+0000 7fccf6ff8740 1 Processor -- start 2024-03-21T11:24:57.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.192+0000 7fccea000640 1 -- 172.21.15.47:0/258346101 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fccd80037a0 con 0x7fccec102790 2024-03-21T11:24:57.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.192+0000 7fccf6ff8740 1 -- start start 2024-03-21T11:24:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.192+0000 7fccf6ff8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe32b0 0x55bc5afe36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.192+0000 7fccf6ff8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5afe87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.192+0000 7fccf6ff8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe8e10 0x55bc5afeb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.192+0000 7fccf6ff8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5af48eb0 con 0x55bc5afe8e10 2024-03-21T11:24:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.192+0000 7fccf6ff8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5aef1a60 con 0x55bc5afe32b0 2024-03-21T11:24:57.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.192+0000 7fccf6ff8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5af44f50 con 0x55bc5afe3ca0 2024-03-21T11:24:57.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.193+0000 7fccf2200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe32b0 0x55bc5afe36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.193+0000 7fccf1800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5afe87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.193+0000 7fccf2200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe32b0 0x55bc5afe36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40654/0 (socket says 172.21.15.47:40654) 2024-03-21T11:24:57.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.193+0000 7fccf2200640 1 -- 172.21.15.47:0/3380101542 learned_addr learned my addr 172.21.15.47:0/3380101542 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.193+0000 7fccf2200640 1 -- 172.21.15.47:0/3380101542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 msgr2=0x55bc5afe87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.193+0000 7fccf2200640 1 --2- 172.21.15.47:0/3380101542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5afe87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.193+0000 7fccf2200640 1 -- 172.21.15.47:0/3380101542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe8e10 msgr2=0x55bc5afeb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.193+0000 7fccf2200640 1 --2- 172.21.15.47:0/3380101542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe8e10 0x55bc5afeb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.193+0000 7fccf2200640 1 -- 172.21.15.47:0/3380101542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc5ad257e0 con 0x55bc5afe32b0 2024-03-21T11:24:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.193+0000 7fccf2200640 1 --2- 172.21.15.47:0/3380101542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe32b0 0x55bc5afe36d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009f60 tx=0x7fcce4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bac41e1e583786e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.194+0000 7fccdea00640 1 -- 172.21.15.47:0/3380101542 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce4007440 con 0x55bc5afe32b0 2024-03-21T11:24:57.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.194+0000 7fccea000640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fccd40a2fa0 0x7fccd40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.194+0000 7fccdea00640 1 -- 172.21.15.47:0/3380101542 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcce4007ac0 con 0x55bc5afe32b0 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.194+0000 7fccdea00640 1 -- 172.21.15.47:0/3380101542 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce4019cc0 con 0x55bc5afe32b0 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.194+0000 7fccea000640 1 -- 172.21.15.47:0/258346101 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fccd80cfbb0 con 0x7fccec102790 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.194+0000 7fccf6ff8740 1 -- 172.21.15.47:0/3380101542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe32b0 msgr2=0x55bc5afe36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.194+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/3380101542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe32b0 0x55bc5afe36d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009f60 tx=0x7fcce4009f90 comp rx=0 tx=0).stop 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.194+0000 7fccf1800640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fccd40a2fa0 0x7fccd40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.195+0000 7fccf6ff8740 1 -- 172.21.15.47:0/3380101542 shutdown_connections 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.195+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/3380101542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe8e10 0x55bc5afeb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.195+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/3380101542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5afe87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.195+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/3380101542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe32b0 0x55bc5afe36d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.195+0000 7fccf6ff8740 1 -- 172.21.15.47:0/3380101542 >> 172.21.15.47:0/3380101542 conn(0x55bc5afde820 msgr2=0x55bc5afdec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.195+0000 7fccf6ff8740 1 -- 172.21.15.47:0/3380101542 shutdown_connections 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.195+0000 7fccf6ff8740 1 -- 172.21.15.47:0/3380101542 wait complete. 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.195+0000 7fccf1800640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fccd40a2fa0 0x7fccd40a5460 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcce002c180 tx=0x7fcce002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.196+0000 7fccf6ff8740 1 -- start start 2024-03-21T11:24:57.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.196+0000 7fccf6ff8740 1 Processor -- start 2024-03-21T11:24:57.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.196+0000 7fccf6ff8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5afed4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.197+0000 7fccf6ff8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5afeda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.197+0000 7fccf6ff8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe8e10 0x55bc5aff4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.197+0000 7fccf6ff8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5afebdc0 con 0x55bc5afe32b0 2024-03-21T11:24:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.197+0000 7fccf6ff8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5af04eb0 con 0x55bc5afe8e10 2024-03-21T11:24:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.197+0000 7fccf6ff8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc5afebf40 con 0x55bc5afe3ca0 2024-03-21T11:24:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.197+0000 7fccf6ff8740 10 client.0 _ll_register_callbacks cb 0x55bc5aef9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.197+0000 7fccf0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5afeda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.197+0000 7fccf1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5afed4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.197+0000 7fccf0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5afeda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52816/0 (socket says 172.21.15.47:52816) 2024-03-21T11:24:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.197+0000 7fccf0e00640 1 -- 172.21.15.47:0/1811431799 learned_addr learned my addr 172.21.15.47:0/1811431799 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.197+0000 7fccf2200640 1 --2- 172.21.15.47:0/1811431799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe8e10 0x55bc5aff4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.197+0000 7fccf0e00640 1 -- 172.21.15.47:0/1811431799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe8e10 msgr2=0x55bc5aff4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.197+0000 7fccf0e00640 1 --2- 172.21.15.47:0/1811431799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe8e10 0x55bc5aff4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.197+0000 7fccf0e00640 1 -- 172.21.15.47:0/1811431799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 msgr2=0x55bc5afed4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.198+0000 7fccf0e00640 1 --2- 172.21.15.47:0/1811431799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5afed4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.198+0000 7fccf0e00640 1 -- 172.21.15.47:0/1811431799 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc5ad85f70 con 0x55bc5afe3ca0 2024-03-21T11:24:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.198+0000 7fccf1800640 1 --2- 172.21.15.47:0/1811431799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5afed4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.198+0000 7fccf0e00640 1 --2- 172.21.15.47:0/1811431799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5afeda00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fccd80a43d0 tx=0x7fccd809bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25b1fc8ef9b0d84d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.198+0000 7fccd3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.198+0000 7fccd3400640 1 -- 172.21.15.47:0/1811431799 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd809bef0 con 0x55bc5afe3ca0 2024-03-21T11:24:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.198+0000 7fccd3400640 1 -- 172.21.15.47:0/1811431799 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fccd80cd3d0 con 0x55bc5afe3ca0 2024-03-21T11:24:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.198+0000 7fccf6ff8740 1 -- 172.21.15.47:0/1811431799 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcce4007ac0 con 0x55bc5afe3ca0 2024-03-21T11:24:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.199+0000 7fccd3400640 1 -- 172.21.15.47:0/1811431799 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccd80a0040 con 0x55bc5afe3ca0 2024-03-21T11:24:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.199+0000 7fccf6ff8740 10 client.25975 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.199+0000 7fccf6ff8740 1 -- 172.21.15.47:0/1811431799 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bc5ac1d170 con 0x55bc5afe3ca0 2024-03-21T11:24:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.199+0000 7fccd3400640 1 -- 172.21.15.47:0/1811431799 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcce4007ac0 con 0x55bc5afe3ca0 2024-03-21T11:24:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.199+0000 7fccd3400640 1 -- 172.21.15.47:0/1811431799 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fccd809b340 con 0x55bc5afe3ca0 2024-03-21T11:24:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.199+0000 7fccf6ff8740 10 client.25975 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.199+0000 7fccf6ff8740 10 client.25975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.199+0000 7fccf6ff8740 20 client.25975 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.199+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/1811431799 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bc5aff92b0 0x55bc5affb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.199+0000 7fccf6ff8740 4 client.25975 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.199+0000 7fccf6ff8740 1 -- 172.21.15.47:0/1811431799 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc5af563f0 con 0x55bc5aff92b0 2024-03-21T11:24:57.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.200+0000 7fccf1800640 1 --2- 172.21.15.47:0/1811431799 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bc5aff92b0 0x55bc5affb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.200+0000 7fccf1800640 1 --2- 172.21.15.47:0/1811431799 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bc5aff92b0 0x55bc5affb6a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e015597032f3e25d server_cookie=70ee1390f8d696ff in_seq=0 out_seq=0 2024-03-21T11:24:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.200+0000 7fccd3400640 10 client.25975 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.201+0000 7fccd3400640 1 -- 172.21.15.47:0/1811431799 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bc5af563f0 con 0x55bc5aff92b0 2024-03-21T11:24:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.201+0000 7fccd3400640 10 client.25975 handle_command_reply: tid=0 2024-03-21T11:24:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.201+0000 7fccf6ff8740 1 client.25975 shutdown 2024-03-21T11:24:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.202+0000 7fccf6ff8740 1 -- 172.21.15.47:0/1811431799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 msgr2=0x55bc5afeda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.202+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/1811431799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5afeda00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fccd80a43d0 tx=0x7fccd809bb60 comp rx=0 tx=0).stop 2024-03-21T11:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.202+0000 7fccf6ff8740 1 -- 172.21.15.47:0/1811431799 shutdown_connections 2024-03-21T11:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.202+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/1811431799 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bc5aff92b0 0x55bc5affb6a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.202+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/1811431799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe8e10 0x55bc5aff4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.202+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/1811431799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5afeda00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.202+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/1811431799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5afed4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.202+0000 7fccf6ff8740 1 -- 172.21.15.47:0/1811431799 >> 172.21.15.47:0/1811431799 conn(0x55bc5afde820 msgr2=0x55bc5afdda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.202+0000 7fccd3400640 0 client.25975 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.202+0000 7fccf6ff8740 1 -- 172.21.15.47:0/1811431799 shutdown_connections 2024-03-21T11:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.202+0000 7fccf6ff8740 1 -- 172.21.15.47:0/1811431799 wait complete. 2024-03-21T11:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.202+0000 7fccf6ff8740 20 client.25975 trim_cache size 0 max 16384 2024-03-21T11:24:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.229+0000 7fccf6ff8740 1 Processor -- start 2024-03-21T11:24:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.229+0000 7fccf6ff8740 1 -- start start 2024-03-21T11:24:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.229+0000 7fccf6ff8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5afe46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.229+0000 7fccf6ff8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe3ca0 0x55bc5aff4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.229+0000 7fccf6ff8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe8e10 0x55bc5afe4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.229+0000 7fccf6ff8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1a7e10 con 0x55bc5afe32b0 2024-03-21T11:24:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.229+0000 7fccf6ff8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1a9790 con 0x55bc5afe3ca0 2024-03-21T11:24:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.229+0000 7fccf6ff8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1a8470 con 0x55bc5afe8e10 2024-03-21T11:24:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.230+0000 7fccf0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe3ca0 0x55bc5aff4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.230+0000 7fccf0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe3ca0 0x55bc5aff4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40664/0 (socket says 172.21.15.47:40664) 2024-03-21T11:24:57.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.230+0000 7fccf0e00640 1 -- 172.21.15.47:0/3926644672 learned_addr learned my addr 172.21.15.47:0/3926644672 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.230+0000 7fccf2200640 1 --2- 172.21.15.47:0/3926644672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe8e10 0x55bc5afe4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.230+0000 7fccf1800640 1 --2- 172.21.15.47:0/3926644672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5afe46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.230+0000 7fccf0e00640 1 -- 172.21.15.47:0/3926644672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe8e10 msgr2=0x55bc5afe4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.230+0000 7fccf0e00640 1 --2- 172.21.15.47:0/3926644672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe8e10 0x55bc5afe4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.230+0000 7fccf0e00640 1 -- 172.21.15.47:0/3926644672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 msgr2=0x55bc5afe46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.230+0000 7fccf0e00640 1 --2- 172.21.15.47:0/3926644672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5afe46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.230+0000 7fccf0e00640 1 -- 172.21.15.47:0/3926644672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc5afdf270 con 0x55bc5afe3ca0 2024-03-21T11:24:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.230+0000 7fccf0e00640 1 --2- 172.21.15.47:0/3926644672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe3ca0 0x55bc5aff4a00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fccd8098350 tx=0x7fccd8003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ee0afed46fabaa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.231+0000 7fccde000640 1 -- 172.21.15.47:0/3926644672 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd809bef0 con 0x55bc5afe3ca0 2024-03-21T11:24:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.231+0000 7fccde000640 1 -- 172.21.15.47:0/3926644672 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccd807f020 con 0x55bc5afe3ca0 2024-03-21T11:24:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.231+0000 7fccde000640 1 -- 172.21.15.47:0/3926644672 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8003b30 con 0x55bc5afe3ca0 2024-03-21T11:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.231+0000 7fccf6ff8740 1 -- 172.21.15.47:0/3926644672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe3ca0 msgr2=0x55bc5aff4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.231+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/3926644672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe3ca0 0x55bc5aff4a00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fccd8098350 tx=0x7fccd8003f30 comp rx=0 tx=0).stop 2024-03-21T11:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.232+0000 7fccf6ff8740 1 -- 172.21.15.47:0/3926644672 shutdown_connections 2024-03-21T11:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.232+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/3926644672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe8e10 0x55bc5afe4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.232+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/3926644672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe3ca0 0x55bc5aff4a00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.232+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/3926644672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5afe46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.232+0000 7fccf6ff8740 1 -- 172.21.15.47:0/3926644672 >> 172.21.15.47:0/3926644672 conn(0x55bc5afde820 msgr2=0x55bc5afe2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.232+0000 7fccf6ff8740 1 -- 172.21.15.47:0/3926644672 shutdown_connections 2024-03-21T11:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.232+0000 7fccf6ff8740 1 -- 172.21.15.47:0/3926644672 wait complete. 2024-03-21T11:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.232+0000 7fccf6ff8740 1 -- start start 2024-03-21T11:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.232+0000 7fccf6ff8740 1 Processor -- start 2024-03-21T11:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf6ff8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5aff43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf6ff8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5aff4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf6ff8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe8e10 0x55bc5aff2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf6ff8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1a9130 con 0x55bc5afe32b0 2024-03-21T11:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf6ff8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1a8ad0 con 0x55bc5afe8e10 2024-03-21T11:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf6ff8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1a5fd0 con 0x55bc5afe3ca0 2024-03-21T11:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf6ff8740 10 client.0 _ll_register_callbacks cb 0x7fccd8003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5aff43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5aff4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5aff43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53316/0 (socket says 172.21.15.47:53316) 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf1800640 1 -- 172.21.15.47:0/569415180 learned_addr learned my addr 172.21.15.47:0/569415180 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5aff4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52838/0 (socket says 172.21.15.47:52838) 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf2200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe8e10 0x55bc5aff2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf1800640 1 -- 172.21.15.47:0/569415180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 msgr2=0x55bc5aff4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf1800640 1 --2- 172.21.15.47:0/569415180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5aff4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.233+0000 7fccf1800640 1 -- 172.21.15.47:0/569415180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe8e10 msgr2=0x55bc5aff2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.234+0000 7fccf1800640 1 --2- 172.21.15.47:0/569415180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe8e10 0x55bc5aff2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.234+0000 7fccf1800640 1 -- 172.21.15.47:0/569415180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc5aff3250 con 0x55bc5afe32b0 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.234+0000 7fccf0e00640 1 --2- 172.21.15.47:0/569415180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5aff4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.234+0000 7fccf1800640 1 --2- 172.21.15.47:0/569415180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5aff43f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcce002cae0 tx=0x7fcce002bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc38f9b43073654 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.234+0000 7fccd3400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.234+0000 7fccd3400640 1 -- 172.21.15.47:0/569415180 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0004ad0 con 0x55bc5afe32b0 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.234+0000 7fccd3400640 1 -- 172.21.15.47:0/569415180 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcce0004c70 con 0x55bc5afe32b0 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.234+0000 7fccf6ff8740 1 -- 172.21.15.47:0/569415180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccd807f020 con 0x55bc5afe32b0 2024-03-21T11:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.235+0000 7fccd3400640 1 -- 172.21.15.47:0/569415180 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcce0005610 con 0x55bc5afe32b0 2024-03-21T11:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.235+0000 7fccf6ff8740 10 client.15798 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.235+0000 7fccf6ff8740 1 -- 172.21.15.47:0/569415180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bc5b055790 con 0x55bc5afe32b0 2024-03-21T11:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.235+0000 7fccd3400640 1 -- 172.21.15.47:0/569415180 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccd807f020 con 0x55bc5afe32b0 2024-03-21T11:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.235+0000 7fccd3400640 1 -- 172.21.15.47:0/569415180 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcce0034660 con 0x55bc5afe32b0 2024-03-21T11:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.235+0000 7fccf6ff8740 10 client.15798 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.235+0000 7fccf6ff8740 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.235+0000 7fccf6ff8740 20 client.15798 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.235+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/569415180 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bc5b0560a0 0x55bc5b0584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.235+0000 7fccf6ff8740 4 client.15798 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.235+0000 7fccf6ff8740 1 -- 172.21.15.47:0/569415180 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bc5af50740 con 0x55bc5b0560a0 2024-03-21T11:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.236+0000 7fccf0e00640 1 --2- 172.21.15.47:0/569415180 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bc5b0560a0 0x55bc5b0584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.236+0000 7fccf0e00640 1 --2- 172.21.15.47:0/569415180 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bc5b0560a0 0x55bc5b0584c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20bb0cae2d8f592b server_cookie=92bacd918133429c in_seq=0 out_seq=0 2024-03-21T11:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.236+0000 7fccd3400640 10 client.15798 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.237+0000 7fccd3400640 1 -- 172.21.15.47:0/569415180 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bc5af50740 con 0x55bc5b0560a0 2024-03-21T11:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.237+0000 7fccd3400640 10 client.15798 handle_command_reply: tid=0 2024-03-21T11:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.237+0000 7fccf6ff8740 1 client.15798 shutdown 2024-03-21T11:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.238+0000 7fccf6ff8740 1 -- 172.21.15.47:0/569415180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 msgr2=0x55bc5aff43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.238+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/569415180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5aff43f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcce002cae0 tx=0x7fcce002bc40 comp rx=0 tx=0).stop 2024-03-21T11:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.238+0000 7fccf6ff8740 1 -- 172.21.15.47:0/569415180 shutdown_connections 2024-03-21T11:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.238+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/569415180 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bc5b0560a0 0x55bc5b0584c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.238+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/569415180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc5afe8e10 0x55bc5aff2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.238+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/569415180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bc5afe3ca0 0x55bc5aff4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.238+0000 7fccf6ff8740 1 --2- 172.21.15.47:0/569415180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc5afe32b0 0x55bc5aff43f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.238+0000 7fccf6ff8740 1 -- 172.21.15.47:0/569415180 >> 172.21.15.47:0/569415180 conn(0x55bc5afde820 msgr2=0x55bc5afdd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.238+0000 7fccd3400640 0 client.15798 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.238+0000 7fccf6ff8740 1 -- 172.21.15.47:0/569415180 shutdown_connections 2024-03-21T11:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.238+0000 7fccf6ff8740 1 -- 172.21.15.47:0/569415180 wait complete. 2024-03-21T11:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.238+0000 7fccf6ff8740 20 client.15798 trim_cache size 0 max 16384 2024-03-21T11:24:57.240 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:24:57.240 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:24:57.241 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "daaa94ea-3abb-44ad-9f7c-8094e082c9cb", 2024-03-21T11:24:57.241 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:24:57.241 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.239+0000 7fccf2c00640 1 -- 172.21.15.47:0/258346101 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fccd40a2fa0 msgr2=0x7fccd40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.239+0000 7fccf2c00640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fccd40a2fa0 0x7fccd40a5460 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcce002c180 tx=0x7fcce002cf90 comp rx=0 tx=0).stop 2024-03-21T11:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.239+0000 7fccf2c00640 1 -- 172.21.15.47:0/258346101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccec102790 msgr2=0x7fccec195bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.239+0000 7fccf2c00640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccec102790 0x7fccec195bf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fccd800ea60 tx=0x7fccd800ef20 comp rx=0 tx=0).stop 2024-03-21T11:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.239+0000 7fccf2c00640 1 -- 172.21.15.47:0/258346101 shutdown_connections 2024-03-21T11:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.239+0000 7fccf2c00640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fccd40a2fa0 0x7fccd40a5460 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.239+0000 7fccf2c00640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccec103150 0x7fccec19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.239+0000 7fccf2c00640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccec102790 0x7fccec195bf0 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fccd800ea60 tx=0x7fccd800ef20 comp rx=0 tx=0).stop 2024-03-21T11:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.239+0000 7fccf2c00640 1 --2- 172.21.15.47:0/258346101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccec1015a0 0x7fccec1956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.239+0000 7fccf2c00640 1 -- 172.21.15.47:0/258346101 >> 172.21.15.47:0/258346101 conn(0x7fccec0fcb10 msgr2=0x7fccec105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.240+0000 7fccf2c00640 1 -- 172.21.15.47:0/258346101 shutdown_connections 2024-03-21T11:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.240+0000 7fccf2c00640 1 -- 172.21.15.47:0/258346101 wait complete. 2024-03-21T11:24:57.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:24:57.256 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:24:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.433+0000 7fb562a00640 1 -- 172.21.15.47:0/807462074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb55c0740a0 msgr2=0x7fb55c074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.433+0000 7fb562a00640 1 --2- 172.21.15.47:0/807462074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb55c0740a0 0x7fb55c074500 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb548009900 tx=0x7fb54802bcd0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.433+0000 7fb562a00640 1 -- 172.21.15.47:0/807462074 shutdown_connections 2024-03-21T11:24:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.433+0000 7fb562a00640 1 --2- 172.21.15.47:0/807462074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb55c109430 0x7fb55c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.433+0000 7fb562a00640 1 --2- 172.21.15.47:0/807462074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb55c0740a0 0x7fb55c074500 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.433+0000 7fb562a00640 1 --2- 172.21.15.47:0/807462074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb55c075680 0x7fb55c073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.433+0000 7fb562a00640 1 -- 172.21.15.47:0/807462074 >> 172.21.15.47:0/807462074 conn(0x7fb55c0fcb10 msgr2=0x7fb55c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.433+0000 7fb562a00640 1 -- 172.21.15.47:0/807462074 shutdown_connections 2024-03-21T11:24:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.434+0000 7fb562a00640 1 -- 172.21.15.47:0/807462074 wait complete. 2024-03-21T11:24:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.434+0000 7fb562a00640 1 Processor -- start 2024-03-21T11:24:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.435+0000 7fb562a00640 1 -- start start 2024-03-21T11:24:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.435+0000 7fb562a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb55c0740a0 0x7fb55c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.435+0000 7fb562a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb55c075680 0x7fb55c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.435+0000 7fb562a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb55c109430 0x7fb55c1a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.435+0000 7fb562a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c10f450 con 0x7fb55c109430 2024-03-21T11:24:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.435+0000 7fb562a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c10f2d0 con 0x7fb55c0740a0 2024-03-21T11:24:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.435+0000 7fb562a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c10f5d0 con 0x7fb55c075680 2024-03-21T11:24:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.435+0000 7fb560c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb55c075680 0x7fb55c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.435+0000 7fb560c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb55c075680 0x7fb55c199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52854/0 (socket says 172.21.15.47:52854) 2024-03-21T11:24:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.435+0000 7fb560c00640 1 -- 172.21.15.47:0/110397481 learned_addr learned my addr 172.21.15.47:0/110397481 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.435+0000 7fb561600640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb55c0740a0 0x7fb55c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.435+0000 7fb562000640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb55c109430 0x7fb55c1a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.436+0000 7fb560c00640 1 -- 172.21.15.47:0/110397481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb55c0740a0 msgr2=0x7fb55c199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.436+0000 7fb560c00640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb55c0740a0 0x7fb55c199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.436+0000 7fb560c00640 1 -- 172.21.15.47:0/110397481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb55c109430 msgr2=0x7fb55c1a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.436+0000 7fb560c00640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb55c109430 0x7fb55c1a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.436+0000 7fb560c00640 1 -- 172.21.15.47:0/110397481 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c1a1630 con 0x7fb55c075680 2024-03-21T11:24:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.436+0000 7fb562000640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb55c109430 0x7fb55c1a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.436+0000 7fb560c00640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb55c075680 0x7fb55c199f50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb548005ab0 tx=0x7fb54802c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.436+0000 7fb55a000640 1 -- 172.21.15.47:0/110397481 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548043020 con 0x7fb55c075680 2024-03-21T11:24:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.436+0000 7fb55a000640 1 -- 172.21.15.47:0/110397481 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5480054e0 con 0x7fb55c075680 2024-03-21T11:24:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.436+0000 7fb55a000640 1 -- 172.21.15.47:0/110397481 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548003ca0 con 0x7fb55c075680 2024-03-21T11:24:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.436+0000 7fb562a00640 1 -- 172.21.15.47:0/110397481 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55c1a18c0 con 0x7fb55c075680 2024-03-21T11:24:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.436+0000 7fb562a00640 1 -- 172.21.15.47:0/110397481 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55c1a1dc0 con 0x7fb55c075680 2024-03-21T11:24:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.440+0000 7fb566c75740 1 Processor -- start 2024-03-21T11:24:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.440+0000 7fb566c75740 1 -- start start 2024-03-21T11:24:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb566c75740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50c5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb566c75740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50c5a40 0x55f2f50ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb566c75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50cabb0 0x55f2f50cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb566c75740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f502a250 con 0x55f2f50c5050 2024-03-21T11:24:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb566c75740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f4fd2a30 con 0x55f2f50c5a40 2024-03-21T11:24:57.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb566c75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f50262f0 con 0x55f2f50cabb0 2024-03-21T11:24:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb561600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50c5a40 0x55f2f50ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb562000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb561600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50c5a40 0x55f2f50ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40692/0 (socket says 172.21.15.47:40692) 2024-03-21T11:24:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb561600640 1 -- 172.21.15.47:0/3563070702 learned_addr learned my addr 172.21.15.47:0/3563070702 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb55a000640 1 -- 172.21.15.47:0/110397481 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb548046020 con 0x7fb55c075680 2024-03-21T11:24:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb561600640 1 -- 172.21.15.47:0/3563070702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50cabb0 msgr2=0x55f2f50cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb561600640 1 --2- 172.21.15.47:0/3563070702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50cabb0 0x55f2f50cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb561600640 1 -- 172.21.15.47:0/3563070702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 msgr2=0x55f2f50c5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb561600640 1 --2- 172.21.15.47:0/3563070702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50c5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb561600640 1 -- 172.21.15.47:0/3563070702 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2f4fbd690 con 0x55f2f50c5a40 2024-03-21T11:24:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.441+0000 7fb562000640 1 --2- 172.21.15.47:0/3563070702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.442+0000 7fb561600640 1 --2- 172.21.15.47:0/3563070702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50c5a40 0x55f2f50ca540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb550009cf0 tx=0x7fb550011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f31478a3c1a880 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.442+0000 7fb54e600640 1 -- 172.21.15.47:0/3563070702 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550004230 con 0x55f2f50c5a40 2024-03-21T11:24:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.442+0000 7fb54e600640 1 -- 172.21.15.47:0/3563070702 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5500043d0 con 0x55f2f50c5a40 2024-03-21T11:24:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.442+0000 7fb54e600640 1 -- 172.21.15.47:0/3563070702 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550019960 con 0x55f2f50c5a40 2024-03-21T11:24:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.443+0000 7fb566c75740 1 -- 172.21.15.47:0/3563070702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50c5a40 msgr2=0x55f2f50ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.443+0000 7fb566c75740 1 --2- 172.21.15.47:0/3563070702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50c5a40 0x55f2f50ca540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb550009cf0 tx=0x7fb550011810 comp rx=0 tx=0).stop 2024-03-21T11:24:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.443+0000 7fb55a000640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5440a2f20 0x7fb5440a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.443+0000 7fb55a000640 1 -- 172.21.15.47:0/110397481 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb5480f1a20 con 0x7fb55c075680 2024-03-21T11:24:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.443+0000 7fb566c75740 1 -- 172.21.15.47:0/3563070702 shutdown_connections 2024-03-21T11:24:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.443+0000 7fb566c75740 1 --2- 172.21.15.47:0/3563070702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50cabb0 0x55f2f50cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.443+0000 7fb566c75740 1 --2- 172.21.15.47:0/3563070702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50c5a40 0x55f2f50ca540 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.443+0000 7fb566c75740 1 --2- 172.21.15.47:0/3563070702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.443+0000 7fb566c75740 1 -- 172.21.15.47:0/3563070702 >> 172.21.15.47:0/3563070702 conn(0x55f2f508d200 msgr2=0x55f2f50c09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.444+0000 7fb562000640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5440a2f20 0x7fb5440a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.444+0000 7fb566c75740 1 -- 172.21.15.47:0/3563070702 shutdown_connections 2024-03-21T11:24:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.444+0000 7fb566c75740 1 -- 172.21.15.47:0/3563070702 wait complete. 2024-03-21T11:24:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.445+0000 7fb566c75740 1 -- start start 2024-03-21T11:24:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.445+0000 7fb566c75740 1 Processor -- start 2024-03-21T11:24:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.445+0000 7fb562000640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5440a2f20 0x7fb5440a53e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55f2f50c62a0 tx=0x7fb554009490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.445+0000 7fb566c75740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.445+0000 7fb566c75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5a40 0x55f2f50cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.445+0000 7fb566c75740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50cabb0 0x55f2f50d65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.445+0000 7fb566c75740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f50cdb60 con 0x55f2f50c5050 2024-03-21T11:24:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.445+0000 7fb566c75740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f4fe5e80 con 0x55f2f50cabb0 2024-03-21T11:24:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.445+0000 7fb566c75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2f50cdce0 con 0x55f2f50c5a40 2024-03-21T11:24:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.445+0000 7fb566c75740 10 client.0 _ll_register_callbacks cb 0x55f2f4ff8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.446+0000 7fb561600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.446+0000 7fb560c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5a40 0x55f2f50cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.446+0000 7fb561600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53352/0 (socket says 172.21.15.47:53352) 2024-03-21T11:24:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.446+0000 7fb560c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5a40 0x55f2f50cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52870/0 (socket says 172.21.15.47:52870) 2024-03-21T11:24:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.446+0000 7fb561600640 1 -- 172.21.15.47:0/3792395854 learned_addr learned my addr 172.21.15.47:0/3792395854 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.446+0000 7fb562000640 1 --2- 172.21.15.47:0/3792395854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50cabb0 0x55f2f50d65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.446+0000 7fb560c00640 1 -- 172.21.15.47:0/3792395854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50cabb0 msgr2=0x55f2f50d65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.446+0000 7fb560c00640 1 --2- 172.21.15.47:0/3792395854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50cabb0 0x55f2f50d65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.446+0000 7fb560c00640 1 -- 172.21.15.47:0/3792395854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 msgr2=0x55f2f50cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.446+0000 7fb560c00640 1 --2- 172.21.15.47:0/3792395854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.446+0000 7fb560c00640 1 -- 172.21.15.47:0/3792395854 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2f4e087d0 con 0x55f2f50c5a40 2024-03-21T11:24:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.446+0000 7fb560c00640 1 --2- 172.21.15.47:0/3792395854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5a40 0x55f2f50cf590 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb548006010 tx=0x7fb5480bd9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ddc21df66f8e20a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.446+0000 7fb542a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.447+0000 7fb542a00640 1 -- 172.21.15.47:0/3792395854 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548004250 con 0x55f2f50c5a40 2024-03-21T11:24:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.447+0000 7fb542a00640 1 -- 172.21.15.47:0/3792395854 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb5480c62e0 con 0x55f2f50c5a40 2024-03-21T11:24:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.447+0000 7fb566c75740 1 -- 172.21.15.47:0/3792395854 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5500043d0 con 0x55f2f50c5a40 2024-03-21T11:24:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.447+0000 7fb542a00640 1 -- 172.21.15.47:0/3792395854 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb548005690 con 0x55f2f50c5a40 2024-03-21T11:24:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.447+0000 7fb566c75740 10 client.25987 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.447+0000 7fb566c75740 1 -- 172.21.15.47:0/3792395854 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f2f4d3e3f0 con 0x55f2f50c5a40 2024-03-21T11:24:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.447+0000 7fb542a00640 1 -- 172.21.15.47:0/3792395854 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5500043d0 con 0x55f2f50c5a40 2024-03-21T11:24:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.448+0000 7fb542a00640 1 -- 172.21.15.47:0/3792395854 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb548085020 con 0x55f2f50c5a40 2024-03-21T11:24:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.448+0000 7fb566c75740 10 client.25987 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.448+0000 7fb566c75740 10 client.25987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.448+0000 7fb566c75740 20 client.25987 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.448+0000 7fb566c75740 1 --2- 172.21.15.47:0/3792395854 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f2f50daef0 0x55f2f50dd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.448+0000 7fb566c75740 4 client.25987 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:24:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.448+0000 7fb566c75740 1 -- 172.21.15.47:0/3792395854 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2f5037790 con 0x55f2f50daef0 2024-03-21T11:24:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.448+0000 7fb561600640 1 --2- 172.21.15.47:0/3792395854 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f2f50daef0 0x55f2f50dd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.449+0000 7fb561600640 1 --2- 172.21.15.47:0/3792395854 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f2f50daef0 0x55f2f50dd2e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88bdda2aad872797 server_cookie=e6da06ac4c73b0a6 in_seq=0 out_seq=0 2024-03-21T11:24:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.449+0000 7fb542a00640 10 client.25987 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.449+0000 7fb542a00640 1 -- 172.21.15.47:0/3792395854 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f2f5037790 con 0x55f2f50daef0 2024-03-21T11:24:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.449+0000 7fb542a00640 10 client.25987 handle_command_reply: tid=0 2024-03-21T11:24:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.450+0000 7fb566c75740 1 client.25987 shutdown 2024-03-21T11:24:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.450+0000 7fb566c75740 1 -- 172.21.15.47:0/3792395854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5a40 msgr2=0x55f2f50cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.450+0000 7fb566c75740 1 --2- 172.21.15.47:0/3792395854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5a40 0x55f2f50cf590 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb548006010 tx=0x7fb5480bd9d0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.450+0000 7fb566c75740 1 -- 172.21.15.47:0/3792395854 shutdown_connections 2024-03-21T11:24:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.450+0000 7fb566c75740 1 --2- 172.21.15.47:0/3792395854 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f2f50daef0 0x55f2f50dd2e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.450+0000 7fb566c75740 1 --2- 172.21.15.47:0/3792395854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50cabb0 0x55f2f50d65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.450+0000 7fb566c75740 1 --2- 172.21.15.47:0/3792395854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5a40 0x55f2f50cf590 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.450+0000 7fb566c75740 1 --2- 172.21.15.47:0/3792395854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.450+0000 7fb566c75740 1 -- 172.21.15.47:0/3792395854 >> 172.21.15.47:0/3792395854 conn(0x55f2f508d200 msgr2=0x55f2f50bfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.450+0000 7fb542a00640 0 client.25987 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.450+0000 7fb566c75740 1 -- 172.21.15.47:0/3792395854 shutdown_connections 2024-03-21T11:24:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.450+0000 7fb566c75740 1 -- 172.21.15.47:0/3792395854 wait complete. 2024-03-21T11:24:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.450+0000 7fb566c75740 20 client.25987 trim_cache size 0 max 16384 2024-03-21T11:24:57.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.467+0000 7fb566c75740 1 Processor -- start 2024-03-21T11:24:57.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.467+0000 7fb566c75740 1 -- start start 2024-03-21T11:24:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb566c75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5050 0x55f2f50c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb566c75740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50c5a40 0x55f2f50d6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb566c75740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50cabb0 0x55f2f50e1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb566c75740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c1ac020 con 0x55f2f50cabb0 2024-03-21T11:24:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb566c75740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c1ad9a0 con 0x55f2f50c5a40 2024-03-21T11:24:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb566c75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c1ac680 con 0x55f2f50c5050 2024-03-21T11:24:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb561600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5050 0x55f2f50c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb561600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5050 0x55f2f50c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52886/0 (socket says 172.21.15.47:52886) 2024-03-21T11:24:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb561600640 1 -- 172.21.15.47:0/1244624787 learned_addr learned my addr 172.21.15.47:0/1244624787 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:57.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb560c00640 1 --2- 172.21.15.47:0/1244624787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50c5a40 0x55f2f50d6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb562000640 1 --2- 172.21.15.47:0/1244624787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50cabb0 0x55f2f50e1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb561600640 1 -- 172.21.15.47:0/1244624787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50c5a40 msgr2=0x55f2f50d6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb561600640 1 --2- 172.21.15.47:0/1244624787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50c5a40 0x55f2f50d6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb561600640 1 -- 172.21.15.47:0/1244624787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50cabb0 msgr2=0x55f2f50e1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb561600640 1 --2- 172.21.15.47:0/1244624787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50cabb0 0x55f2f50e1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.468+0000 7fb561600640 1 -- 172.21.15.47:0/1244624787 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2f4cfe0f0 con 0x55f2f50c5050 2024-03-21T11:24:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.469+0000 7fb561600640 1 --2- 172.21.15.47:0/1244624787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5050 0x55f2f50c6bc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb550009c40 tx=0x7fb550005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3227a430c8af904 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.469+0000 7fb54dc00640 1 -- 172.21.15.47:0/1244624787 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550031070 con 0x55f2f50c5050 2024-03-21T11:24:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.469+0000 7fb54dc00640 1 -- 172.21.15.47:0/1244624787 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb550005040 con 0x55f2f50c5050 2024-03-21T11:24:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.469+0000 7fb54dc00640 1 -- 172.21.15.47:0/1244624787 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550019810 con 0x55f2f50c5050 2024-03-21T11:24:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.470+0000 7fb566c75740 1 -- 172.21.15.47:0/1244624787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5050 msgr2=0x55f2f50c6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.470+0000 7fb566c75740 1 --2- 172.21.15.47:0/1244624787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5050 0x55f2f50c6bc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb550009c40 tx=0x7fb550005f90 comp rx=0 tx=0).stop 2024-03-21T11:24:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.470+0000 7fb566c75740 1 -- 172.21.15.47:0/1244624787 shutdown_connections 2024-03-21T11:24:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.470+0000 7fb566c75740 1 --2- 172.21.15.47:0/1244624787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50cabb0 0x55f2f50e1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.470+0000 7fb566c75740 1 --2- 172.21.15.47:0/1244624787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50c5a40 0x55f2f50d6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.470+0000 7fb566c75740 1 --2- 172.21.15.47:0/1244624787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5050 0x55f2f50c6bc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.470+0000 7fb566c75740 1 -- 172.21.15.47:0/1244624787 >> 172.21.15.47:0/1244624787 conn(0x55f2f508d200 msgr2=0x55f2f50c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.470+0000 7fb566c75740 1 -- 172.21.15.47:0/1244624787 shutdown_connections 2024-03-21T11:24:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.470+0000 7fb566c75740 1 -- 172.21.15.47:0/1244624787 wait complete. 2024-03-21T11:24:57.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.471+0000 7fb566c75740 1 -- start start 2024-03-21T11:24:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.471+0000 7fb566c75740 1 Processor -- start 2024-03-21T11:24:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.471+0000 7fb566c75740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.471+0000 7fb566c75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5a40 0x55f2f50d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.471+0000 7fb566c75740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50cabb0 0x55f2f50d5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.471+0000 7fb566c75740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c1ad340 con 0x55f2f50c5050 2024-03-21T11:24:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.471+0000 7fb566c75740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c1acce0 con 0x55f2f50cabb0 2024-03-21T11:24:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.471+0000 7fb566c75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c1aa1e0 con 0x55f2f50c5a40 2024-03-21T11:24:57.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.471+0000 7fb566c75740 10 client.0 _ll_register_callbacks cb 0x7fb5480bdba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:24:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.471+0000 7fb561600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.472+0000 7fb560c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5a40 0x55f2f50d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.472+0000 7fb561600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50d3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53380/0 (socket says 172.21.15.47:53380) 2024-03-21T11:24:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.472+0000 7fb561600640 1 -- 172.21.15.47:0/1989093072 learned_addr learned my addr 172.21.15.47:0/1989093072 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:24:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.472+0000 7fb560c00640 1 -- 172.21.15.47:0/1989093072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50cabb0 msgr2=0x55f2f50d5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.472+0000 7fb562000640 1 --2- 172.21.15.47:0/1989093072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50cabb0 0x55f2f50d5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.472+0000 7fb560c00640 1 --2- 172.21.15.47:0/1989093072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50cabb0 0x55f2f50d5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.472+0000 7fb560c00640 1 -- 172.21.15.47:0/1989093072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 msgr2=0x55f2f50d3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:24:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.472+0000 7fb560c00640 1 --2- 172.21.15.47:0/1989093072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50d3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.472+0000 7fb560c00640 1 -- 172.21.15.47:0/1989093072 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2f50d5610 con 0x55f2f50c5a40 2024-03-21T11:24:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.472+0000 7fb561600640 1 --2- 172.21.15.47:0/1989093072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50d3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:24:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.472+0000 7fb562000640 1 --2- 172.21.15.47:0/1989093072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50cabb0 0x55f2f50d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:24:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.473+0000 7fb560c00640 1 --2- 172.21.15.47:0/1989093072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5a40 0x55f2f50d4080 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb5480ba6e0 tx=0x7fb548005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a42ef60c7d7e3ab7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:24:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.473+0000 7fb542a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:24:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.473+0000 7fb542a00640 1 -- 172.21.15.47:0/1989093072 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5480da030 con 0x55f2f50c5a40 2024-03-21T11:24:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.473+0000 7fb542a00640 1 -- 172.21.15.47:0/1989093072 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb5480ef240 con 0x55f2f50c5a40 2024-03-21T11:24:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.473+0000 7fb566c75740 1 -- 172.21.15.47:0/1989093072 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb550005040 con 0x55f2f50c5a40 2024-03-21T11:24:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.473+0000 7fb542a00640 1 -- 172.21.15.47:0/1989093072 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb548004250 con 0x55f2f50c5a40 2024-03-21T11:24:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.473+0000 7fb566c75740 10 client.25993 fetch_fsmap learned FSMap version 28 2024-03-21T11:24:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.473+0000 7fb566c75740 1 -- 172.21.15.47:0/1989093072 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f2f50d6990 con 0x55f2f50c5a40 2024-03-21T11:24:57.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.473+0000 7fb542a00640 1 -- 172.21.15.47:0/1989093072 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb550005040 con 0x55f2f50c5a40 2024-03-21T11:24:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.474+0000 7fb542a00640 1 -- 172.21.15.47:0/1989093072 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb548085040 con 0x55f2f50c5a40 2024-03-21T11:24:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.474+0000 7fb566c75740 10 client.25993 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:24:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.474+0000 7fb566c75740 10 client.25993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:24:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.474+0000 7fb566c75740 20 client.25993 populate_metadata read hostname 'smithi047' 2024-03-21T11:24:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.474+0000 7fb566c75740 1 --2- 172.21.15.47:0/1989093072 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f2f5136d20 0x55f2f5139140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:24:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.474+0000 7fb566c75740 4 client.25993 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:24:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.474+0000 7fb566c75740 1 -- 172.21.15.47:0/1989093072 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f2f5031ae0 con 0x55f2f5136d20 2024-03-21T11:24:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.474+0000 7fb561600640 1 --2- 172.21.15.47:0/1989093072 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f2f5136d20 0x55f2f5139140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:24:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.475+0000 7fb542a00640 10 client.25993 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:57.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.475+0000 7fb561600640 1 --2- 172.21.15.47:0/1989093072 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f2f5136d20 0x55f2f5139140 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77ad61de5170ea9 server_cookie=32a036ec9e6b9d17 in_seq=0 out_seq=0 2024-03-21T11:24:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.475+0000 7fb542a00640 1 -- 172.21.15.47:0/1989093072 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f2f5031ae0 con 0x55f2f5136d20 2024-03-21T11:24:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.475+0000 7fb542a00640 10 client.25993 handle_command_reply: tid=0 2024-03-21T11:24:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.475+0000 7fb566c75740 1 client.25993 shutdown 2024-03-21T11:24:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.475+0000 7fb566c75740 1 -- 172.21.15.47:0/1989093072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5a40 msgr2=0x55f2f50d4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.475+0000 7fb566c75740 1 --2- 172.21.15.47:0/1989093072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5a40 0x55f2f50d4080 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb5480ba6e0 tx=0x7fb548005230 comp rx=0 tx=0).stop 2024-03-21T11:24:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.475+0000 7fb566c75740 1 -- 172.21.15.47:0/1989093072 shutdown_connections 2024-03-21T11:24:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.475+0000 7fb566c75740 1 --2- 172.21.15.47:0/1989093072 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f2f5136d20 0x55f2f5139140 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.475+0000 7fb566c75740 1 --2- 172.21.15.47:0/1989093072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2f50cabb0 0x55f2f50d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.475+0000 7fb542a00640 0 client.25993 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:24:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.475+0000 7fb566c75740 1 --2- 172.21.15.47:0/1989093072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2f50c5a40 0x55f2f50d4080 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.475+0000 7fb566c75740 1 --2- 172.21.15.47:0/1989093072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2f50c5050 0x55f2f50d3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.476+0000 7fb566c75740 1 -- 172.21.15.47:0/1989093072 >> 172.21.15.47:0/1989093072 conn(0x55f2f508d200 msgr2=0x55f2f50c6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.476+0000 7fb566c75740 1 -- 172.21.15.47:0/1989093072 shutdown_connections 2024-03-21T11:24:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.476+0000 7fb566c75740 1 -- 172.21.15.47:0/1989093072 wait complete. 2024-03-21T11:24:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.476+0000 7fb566c75740 20 client.25993 trim_cache size 0 max 16384 2024-03-21T11:24:57.479 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:24:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:24:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:24:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: "daaa94ea-3abb-44ad-9f7c-8094e082c9cb": { 2024-03-21T11:24:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:24:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "daaa94ea-3abb-44ad-9f7c-8094e082c9cb", 2024-03-21T11:24:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:24:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:24:57.480 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:24:57.480 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:24:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.477+0000 7fb562a00640 1 -- 172.21.15.47:0/110397481 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5440a2f20 msgr2=0x7fb5440a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.477+0000 7fb562a00640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5440a2f20 0x7fb5440a53e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55f2f50c62a0 tx=0x7fb554009490 comp rx=0 tx=0).stop 2024-03-21T11:24:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.477+0000 7fb562a00640 1 -- 172.21.15.47:0/110397481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb55c075680 msgr2=0x7fb55c199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:24:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.477+0000 7fb562a00640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb55c075680 0x7fb55c199f50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb548005ab0 tx=0x7fb54802c760 comp rx=0 tx=0).stop 2024-03-21T11:24:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.477+0000 7fb562a00640 1 -- 172.21.15.47:0/110397481 shutdown_connections 2024-03-21T11:24:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.477+0000 7fb562a00640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5440a2f20 0x7fb5440a53e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.477+0000 7fb562a00640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb55c109430 0x7fb55c1a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.477+0000 7fb562a00640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb55c075680 0x7fb55c199f50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.477+0000 7fb562a00640 1 --2- 172.21.15.47:0/110397481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb55c0740a0 0x7fb55c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:24:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.477+0000 7fb562a00640 1 -- 172.21.15.47:0/110397481 >> 172.21.15.47:0/110397481 conn(0x7fb55c0fcb10 msgr2=0x7fb55c1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:24:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.477+0000 7fb562a00640 1 -- 172.21.15.47:0/110397481 shutdown_connections 2024-03-21T11:24:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:24:57.477+0000 7fb562a00640 1 -- 172.21.15.47:0/110397481 wait complete. 2024-03-21T11:24:57.499 INFO:tasks.cephfs.filesystem:scrub status for tag:daaa94ea-3abb-44ad-9f7c-8094e082c9cb - {'path': '/', 'tag': 'daaa94ea-3abb-44ad-9f7c-8094e082c9cb', 'options': 'recursive,force'} 2024-03-21T11:24:58.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:58 smithi102 ceph-mon[36748]: pgmap v449: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-21T11:24:58.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:58 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:24:58.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:58 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:24:58.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:58 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:24:58.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:58 smithi047 ceph-mon[31351]: pgmap v449: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-21T11:24:58.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:58 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:24:58.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:58 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:24:58.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:58 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:24:58.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:58 smithi100 ceph-mon[36245]: pgmap v449: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-21T11:24:58.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:58 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:24:58.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:58 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:24:58.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:58 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:24:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:24:59 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:24:59.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:24:59 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:24:59.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:24:59 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:25:00.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:00 smithi102 ceph-mon[36748]: pgmap v450: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-03-21T11:25:00.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:00 smithi047 ceph-mon[31351]: pgmap v450: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-03-21T11:25:00.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:00 smithi100 ceph-mon[36245]: pgmap v450: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-03-21T11:25:02.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:02 smithi102 ceph-mon[36748]: pgmap v451: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-21T11:25:02.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:02 smithi047 ceph-mon[31351]: pgmap v451: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-21T11:25:02.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:02 smithi100 ceph-mon[36245]: pgmap v451: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-21T11:25:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:03 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:25:03.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:03 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:25:03.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:03 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:25:04.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:04 smithi102 ceph-mon[36748]: pgmap v452: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-21T11:25:04.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:04 smithi047 ceph-mon[31351]: pgmap v452: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-21T11:25:04.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:04 smithi100 ceph-mon[36245]: pgmap v452: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-21T11:25:06.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:06 smithi102 ceph-mon[36748]: pgmap v453: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-21T11:25:06.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:06 smithi047 ceph-mon[31351]: pgmap v453: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-21T11:25:06.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:06 smithi100 ceph-mon[36245]: pgmap v453: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-21T11:25:08.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:08 smithi047 ceph-mon[31351]: pgmap v454: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-21T11:25:08.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:08 smithi100 ceph-mon[36245]: pgmap v454: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-21T11:25:08.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:08 smithi102 ceph-mon[36748]: pgmap v454: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-21T11:25:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.852+0000 7f727f000640 1 -- 172.21.15.47:0/415502236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72781052d0 msgr2=0x7f727810bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.852+0000 7f727f000640 1 --2- 172.21.15.47:0/415502236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72781052d0 0x7f727810bb60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7270009ca0 tx=0x7f727002c120 comp rx=0 tx=0).stop 2024-03-21T11:25:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.852+0000 7f727f000640 1 -- 172.21.15.47:0/415502236 shutdown_connections 2024-03-21T11:25:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.852+0000 7f727f000640 1 --2- 172.21.15.47:0/415502236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72781052d0 0x7f727810bb60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.852+0000 7f727f000640 1 --2- 172.21.15.47:0/415502236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7278104910 0x7f7278104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.852+0000 7f727f000640 1 --2- 172.21.15.47:0/415502236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7278103720 0x7f7278103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.852+0000 7f727f000640 1 -- 172.21.15.47:0/415502236 >> 172.21.15.47:0/415502236 conn(0x7f72780fecb0 msgr2=0x7f72781010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.852+0000 7f727f000640 1 -- 172.21.15.47:0/415502236 shutdown_connections 2024-03-21T11:25:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.852+0000 7f727f000640 1 -- 172.21.15.47:0/415502236 wait complete. 2024-03-21T11:25:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.853+0000 7f727f000640 1 Processor -- start 2024-03-21T11:25:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.853+0000 7f727f000640 1 -- start start 2024-03-21T11:25:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.853+0000 7f727f000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7278103720 0x7f72781999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7278104910 0x7f7278199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727f000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72781052d0 0x7f72781a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727f000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727810f410 con 0x7f7278103720 2024-03-21T11:25:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727810f290 con 0x7f7278104910 2024-03-21T11:25:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727f000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727810f590 con 0x7f72781052d0 2024-03-21T11:25:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7278103720 0x7f72781999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7278104910 0x7f7278199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7278104910 0x7f7278199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39758/0 (socket says 172.21.15.47:39758) 2024-03-21T11:25:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727d200640 1 -- 172.21.15.47:0/2994292480 learned_addr learned my addr 172.21.15.47:0/2994292480 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727e600640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72781052d0 0x7f72781a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727d200640 1 -- 172.21.15.47:0/2994292480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72781052d0 msgr2=0x7f72781a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727d200640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72781052d0 0x7f72781a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727d200640 1 -- 172.21.15.47:0/2994292480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7278103720 msgr2=0x7f72781999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727d200640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7278103720 0x7f72781999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727d200640 1 -- 172.21.15.47:0/2994292480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72781a15e0 con 0x7f7278104910 2024-03-21T11:25:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727dc00640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7278103720 0x7f72781999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.854+0000 7f727e600640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72781052d0 0x7f72781a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.855+0000 7f727d200640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7278104910 0x7f7278199f00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f726400e9f0 tx=0x7f726400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.855+0000 7f7276000640 1 -- 172.21.15.47:0/2994292480 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726400cd80 con 0x7f7278104910 2024-03-21T11:25:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.855+0000 7f7276000640 1 -- 172.21.15.47:0/2994292480 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7264010910 con 0x7f7278104910 2024-03-21T11:25:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.855+0000 7f727f000640 1 -- 172.21.15.47:0/2994292480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72781a18d0 con 0x7f7278104910 2024-03-21T11:25:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.855+0000 7f7276000640 1 -- 172.21.15.47:0/2994292480 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7264018990 con 0x7f7278104910 2024-03-21T11:25:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.855+0000 7f727f000640 1 -- 172.21.15.47:0/2994292480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72781a1d10 con 0x7f7278104910 2024-03-21T11:25:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.860+0000 7f72833f9740 1 Processor -- start 2024-03-21T11:25:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.860+0000 7f72833f9740 1 -- start start 2024-03-21T11:25:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.860+0000 7f7276000640 1 -- 172.21.15.47:0/2994292480 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7264010ac0 con 0x7f7278104910 2024-03-21T11:25:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.860+0000 7f72833f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c43f0 0x55c06e6c4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.860+0000 7f72833f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c4de0 0x55c06e6c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.860+0000 7f72833f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 0x55c06e6cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.861+0000 7f72833f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e6873b0 con 0x55c06e6c9f50 2024-03-21T11:25:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.861+0000 7f72833f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e629830 con 0x55c06e6c4de0 2024-03-21T11:25:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.861+0000 7f72833f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e5d25d0 con 0x55c06e6c43f0 2024-03-21T11:25:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.861+0000 7f727e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c43f0 0x55c06e6c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.861+0000 7f727e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c43f0 0x55c06e6c4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41168/0 (socket says 172.21.15.47:41168) 2024-03-21T11:25:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.861+0000 7f727e600640 1 -- 172.21.15.47:0/3293947137 learned_addr learned my addr 172.21.15.47:0/3293947137 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.861+0000 7f727dc00640 1 --2- 172.21.15.47:0/3293947137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c4de0 0x55c06e6c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.861+0000 7f727d200640 1 --2- 172.21.15.47:0/3293947137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 0x55c06e6cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.862+0000 7f727e600640 1 -- 172.21.15.47:0/3293947137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c4de0 msgr2=0x55c06e6c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.862+0000 7f727e600640 1 --2- 172.21.15.47:0/3293947137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c4de0 0x55c06e6c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.862+0000 7f727e600640 1 -- 172.21.15.47:0/3293947137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 msgr2=0x55c06e6cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.862+0000 7f727e600640 1 --2- 172.21.15.47:0/3293947137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 0x55c06e6cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.862+0000 7f727e600640 1 -- 172.21.15.47:0/3293947137 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c06e5ea2c0 con 0x55c06e6c43f0 2024-03-21T11:25:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.862+0000 7f727d200640 1 --2- 172.21.15.47:0/3293947137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 0x55c06e6cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.863+0000 7f7276000640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72600a2f10 0x7f72600a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.863+0000 7f7276000640 1 -- 172.21.15.47:0/2994292480 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f72640cfb20 con 0x7f7278104910 2024-03-21T11:25:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.863+0000 7f727e600640 1 --2- 172.21.15.47:0/3293947137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c43f0 0x55c06e6c4810 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f727002c730 tx=0x7f7270005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be8974b36a766705 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.863+0000 7f727dc00640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72600a2f10 0x7f72600a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.863+0000 7f726aa00640 1 -- 172.21.15.47:0/3293947137 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7270043070 con 0x55c06e6c43f0 2024-03-21T11:25:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.863+0000 7f726aa00640 1 -- 172.21.15.47:0/3293947137 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7270032d00 con 0x55c06e6c43f0 2024-03-21T11:25:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.863+0000 7f726aa00640 1 -- 172.21.15.47:0/3293947137 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7270039040 con 0x55c06e6c43f0 2024-03-21T11:25:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.864+0000 7f72833f9740 1 -- 172.21.15.47:0/3293947137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c43f0 msgr2=0x55c06e6c4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.864+0000 7f727dc00640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72600a2f10 0x7f72600a53d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55c06e6c51e0 tx=0x7f726c00b5a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.864+0000 7f72833f9740 1 --2- 172.21.15.47:0/3293947137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c43f0 0x55c06e6c4810 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f727002c730 tx=0x7f7270005f80 comp rx=0 tx=0).stop 2024-03-21T11:25:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.864+0000 7f72833f9740 1 -- 172.21.15.47:0/3293947137 shutdown_connections 2024-03-21T11:25:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.864+0000 7f72833f9740 1 --2- 172.21.15.47:0/3293947137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 0x55c06e6cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.864+0000 7f72833f9740 1 --2- 172.21.15.47:0/3293947137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c4de0 0x55c06e6c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.864+0000 7f72833f9740 1 --2- 172.21.15.47:0/3293947137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c43f0 0x55c06e6c4810 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.864+0000 7f72833f9740 1 -- 172.21.15.47:0/3293947137 >> 172.21.15.47:0/3293947137 conn(0x55c06e6bf960 msgr2=0x55c06e6bfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.865+0000 7f72833f9740 1 -- 172.21.15.47:0/3293947137 shutdown_connections 2024-03-21T11:25:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.865+0000 7f72833f9740 1 -- 172.21.15.47:0/3293947137 wait complete. 2024-03-21T11:25:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.865+0000 7f72833f9740 1 -- start start 2024-03-21T11:25:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.866+0000 7f72833f9740 1 Processor -- start 2024-03-21T11:25:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.866+0000 7f72833f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c43f0 0x55c06e6ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.866+0000 7f72833f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c4de0 0x55c06e6cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.866+0000 7f72833f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 0x55c06e6d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.866+0000 7f72833f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e5e5a20 con 0x55c06e6c9f50 2024-03-21T11:25:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.866+0000 7f72833f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e6258d0 con 0x55c06e6c43f0 2024-03-21T11:25:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.866+0000 7f72833f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e6ccd60 con 0x55c06e6c4de0 2024-03-21T11:25:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.867+0000 7f72833f9740 10 client.0 _ll_register_callbacks cb 0x55c06e6221c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.867+0000 7f727e600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 0x55c06e6d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.867+0000 7f727dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c43f0 0x55c06e6ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.867+0000 7f727e600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 0x55c06e6d3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39520/0 (socket says 172.21.15.47:39520) 2024-03-21T11:25:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.867+0000 7f727e600640 1 -- 172.21.15.47:0/1410053011 learned_addr learned my addr 172.21.15.47:0/1410053011 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.867+0000 7f727d200640 1 --2- 172.21.15.47:0/1410053011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c4de0 0x55c06e6cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.868+0000 7f727dc00640 1 -- 172.21.15.47:0/1410053011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c4de0 msgr2=0x55c06e6cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.868+0000 7f727dc00640 1 --2- 172.21.15.47:0/1410053011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c4de0 0x55c06e6cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.868+0000 7f727dc00640 1 -- 172.21.15.47:0/1410053011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 msgr2=0x55c06e6d3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.868+0000 7f727dc00640 1 --2- 172.21.15.47:0/1410053011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 0x55c06e6d3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.868+0000 7f727dc00640 1 -- 172.21.15.47:0/1410053011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c06e33e210 con 0x55c06e6c43f0 2024-03-21T11:25:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.868+0000 7f727d200640 1 --2- 172.21.15.47:0/1410053011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c4de0 0x55c06e6cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.868+0000 7f727dc00640 1 --2- 172.21.15.47:0/1410053011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c43f0 0x55c06e6ce4c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55c06e6c5850 tx=0x7f726c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5f2ac51a6563b78 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.868+0000 7f725f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:25:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.868+0000 7f725f400640 1 -- 172.21.15.47:0/1410053011 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c020d70 con 0x55c06e6c43f0 2024-03-21T11:25:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.869+0000 7f725f400640 1 -- 172.21.15.47:0/1410053011 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f726c038400 con 0x55c06e6c43f0 2024-03-21T11:25:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.869+0000 7f72833f9740 1 -- 172.21.15.47:0/1410053011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7270032d00 con 0x55c06e6c43f0 2024-03-21T11:25:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.869+0000 7f725f400640 1 -- 172.21.15.47:0/1410053011 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f726c041050 con 0x55c06e6c43f0 2024-03-21T11:25:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.869+0000 7f72833f9740 10 client.26000 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.869+0000 7f72833f9740 1 -- 172.21.15.47:0/1410053011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c06e40a620 con 0x55c06e6c43f0 2024-03-21T11:25:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.869+0000 7f725f400640 1 -- 172.21.15.47:0/1410053011 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7270032d00 con 0x55c06e6c43f0 2024-03-21T11:25:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.870+0000 7f725f400640 1 -- 172.21.15.47:0/1410053011 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f726c01f040 con 0x55c06e6c43f0 2024-03-21T11:25:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.870+0000 7f72833f9740 10 client.26000 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.870+0000 7f72833f9740 10 client.26000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.870+0000 7f72833f9740 20 client.26000 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.870+0000 7f72833f9740 1 --2- 172.21.15.47:0/1410053011 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c06e6da390 0x55c06e6dc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.870+0000 7f72833f9740 4 client.26000 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:25:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.870+0000 7f72833f9740 1 -- 172.21.15.47:0/1410053011 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c06e636d70 con 0x55c06e6da390 2024-03-21T11:25:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.870+0000 7f727d200640 1 --2- 172.21.15.47:0/1410053011 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c06e6da390 0x55c06e6dc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.871+0000 7f727d200640 1 --2- 172.21.15.47:0/1410053011 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c06e6da390 0x55c06e6dc780 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4333dd29772b7607 server_cookie=710df22f50f80c71 in_seq=0 out_seq=0 2024-03-21T11:25:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.871+0000 7f725f400640 10 client.26000 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.872+0000 7f725f400640 1 -- 172.21.15.47:0/1410053011 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c06e636d70 con 0x55c06e6da390 2024-03-21T11:25:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.872+0000 7f725f400640 10 client.26000 handle_command_reply: tid=0 2024-03-21T11:25:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.872+0000 7f72833f9740 1 client.26000 shutdown 2024-03-21T11:25:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.874+0000 7f72833f9740 1 -- 172.21.15.47:0/1410053011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c43f0 msgr2=0x55c06e6ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.874+0000 7f72833f9740 1 --2- 172.21.15.47:0/1410053011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c43f0 0x55c06e6ce4c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55c06e6c5850 tx=0x7f726c0046c0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.874+0000 7f72833f9740 1 -- 172.21.15.47:0/1410053011 shutdown_connections 2024-03-21T11:25:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.874+0000 7f72833f9740 1 --2- 172.21.15.47:0/1410053011 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c06e6da390 0x55c06e6dc780 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.874+0000 7f72833f9740 1 --2- 172.21.15.47:0/1410053011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 0x55c06e6d3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.874+0000 7f72833f9740 1 --2- 172.21.15.47:0/1410053011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c4de0 0x55c06e6cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.874+0000 7f72833f9740 1 --2- 172.21.15.47:0/1410053011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c43f0 0x55c06e6ce4c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.874+0000 7f725f400640 0 client.26000 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.874+0000 7f72833f9740 1 -- 172.21.15.47:0/1410053011 >> 172.21.15.47:0/1410053011 conn(0x55c06e6bf960 msgr2=0x55c06e6beb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.874+0000 7f72833f9740 1 -- 172.21.15.47:0/1410053011 shutdown_connections 2024-03-21T11:25:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.874+0000 7f72833f9740 1 -- 172.21.15.47:0/1410053011 wait complete. 2024-03-21T11:25:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.874+0000 7f72833f9740 20 client.26000 trim_cache size 0 max 16384 2024-03-21T11:25:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.889+0000 7f72833f9740 1 Processor -- start 2024-03-21T11:25:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.889+0000 7f72833f9740 1 -- start start 2024-03-21T11:25:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f72833f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c43f0 0x55c06e6ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f72833f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c4de0 0x55c06e6c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f72833f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c9f50 0x55c06e6be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f72833f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7278070e20 con 0x55c06e6c4de0 2024-03-21T11:25:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f72833f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72780727a0 con 0x55c06e6c43f0 2024-03-21T11:25:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f72833f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7278071480 con 0x55c06e6c9f50 2024-03-21T11:25:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f727dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c43f0 0x55c06e6ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f727dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c43f0 0x55c06e6ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52066/0 (socket says 172.21.15.47:52066) 2024-03-21T11:25:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f727dc00640 1 -- 172.21.15.47:0/3053540864 learned_addr learned my addr 172.21.15.47:0/3053540864 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f727e600640 1 --2- 172.21.15.47:0/3053540864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c9f50 0x55c06e6be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f727dc00640 1 -- 172.21.15.47:0/3053540864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c9f50 msgr2=0x55c06e6be2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f727dc00640 1 --2- 172.21.15.47:0/3053540864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c9f50 0x55c06e6be2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f727dc00640 1 -- 172.21.15.47:0/3053540864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c4de0 msgr2=0x55c06e6c9810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:25:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f727d200640 1 --2- 172.21.15.47:0/3053540864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c4de0 0x55c06e6c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f727dc00640 1 --2- 172.21.15.47:0/3053540864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c4de0 0x55c06e6c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.890+0000 7f727dc00640 1 -- 172.21.15.47:0/3053540864 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c06e6dcaf0 con 0x55c06e6c43f0 2024-03-21T11:25:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.891+0000 7f727e600640 1 --2- 172.21.15.47:0/3053540864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c9f50 0x55c06e6be2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.891+0000 7f727d200640 1 --2- 172.21.15.47:0/3053540864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c4de0 0x55c06e6c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:25:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.891+0000 7f727dc00640 1 --2- 172.21.15.47:0/3053540864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c43f0 0x55c06e6ce960 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f726c021800 tx=0x7f726c0207a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14ae6a82baec29bc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.891+0000 7f726a000640 1 -- 172.21.15.47:0/3053540864 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c049070 con 0x55c06e6c43f0 2024-03-21T11:25:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.891+0000 7f726a000640 1 -- 172.21.15.47:0/3053540864 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f726c0304b0 con 0x55c06e6c43f0 2024-03-21T11:25:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.891+0000 7f726a000640 1 -- 172.21.15.47:0/3053540864 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c020dd0 con 0x55c06e6c43f0 2024-03-21T11:25:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.892+0000 7f72833f9740 1 -- 172.21.15.47:0/3053540864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c43f0 msgr2=0x55c06e6ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.892+0000 7f72833f9740 1 --2- 172.21.15.47:0/3053540864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c43f0 0x55c06e6ce960 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f726c021800 tx=0x7f726c0207a0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.892+0000 7f72833f9740 1 -- 172.21.15.47:0/3053540864 shutdown_connections 2024-03-21T11:25:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.892+0000 7f72833f9740 1 --2- 172.21.15.47:0/3053540864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c9f50 0x55c06e6be2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.892+0000 7f72833f9740 1 --2- 172.21.15.47:0/3053540864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c4de0 0x55c06e6c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.892+0000 7f72833f9740 1 --2- 172.21.15.47:0/3053540864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c43f0 0x55c06e6ce960 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.892+0000 7f72833f9740 1 -- 172.21.15.47:0/3053540864 >> 172.21.15.47:0/3053540864 conn(0x55c06e6bf960 msgr2=0x55c06e6bfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.892+0000 7f72833f9740 1 -- 172.21.15.47:0/3053540864 shutdown_connections 2024-03-21T11:25:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.892+0000 7f72833f9740 1 -- 172.21.15.47:0/3053540864 wait complete. 2024-03-21T11:25:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.893+0000 7f72833f9740 1 -- start start 2024-03-21T11:25:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.893+0000 7f72833f9740 1 Processor -- start 2024-03-21T11:25:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.893+0000 7f72833f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c43f0 0x55c06e6ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.893+0000 7f72833f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c4de0 0x55c06e6eadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.893+0000 7f72833f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 0x55c06e6eb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.893+0000 7f72833f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7278072140 con 0x55c06e6c9f50 2024-03-21T11:25:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.893+0000 7f72833f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7278071ae0 con 0x55c06e6c4de0 2024-03-21T11:25:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.893+0000 7f72833f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727806efe0 con 0x55c06e6c43f0 2024-03-21T11:25:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.894+0000 7f72833f9740 10 client.0 _ll_register_callbacks cb 0x7f726c021850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.894+0000 7f727d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c4de0 0x55c06e6eadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.894+0000 7f727dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c43f0 0x55c06e6ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.894+0000 7f727d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c4de0 0x55c06e6eadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52078/0 (socket says 172.21.15.47:52078) 2024-03-21T11:25:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.894+0000 7f727d200640 1 -- 172.21.15.47:0/668203013 learned_addr learned my addr 172.21.15.47:0/668203013 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.894+0000 7f727dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c43f0 0x55c06e6ec7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52628/0 (socket says 172.21.15.47:52628) 2024-03-21T11:25:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.894+0000 7f727e600640 1 --2- 172.21.15.47:0/668203013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 0x55c06e6eb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.894+0000 7f727dc00640 1 -- 172.21.15.47:0/668203013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c4de0 msgr2=0x55c06e6eadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.894+0000 7f727dc00640 1 --2- 172.21.15.47:0/668203013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c4de0 0x55c06e6eadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.894+0000 7f727dc00640 1 -- 172.21.15.47:0/668203013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 msgr2=0x55c06e6eb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.894+0000 7f727dc00640 1 --2- 172.21.15.47:0/668203013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 0x55c06e6eb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.894+0000 7f727dc00640 1 -- 172.21.15.47:0/668203013 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c06e6ebb60 con 0x55c06e6c43f0 2024-03-21T11:25:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.895+0000 7f727d200640 1 --2- 172.21.15.47:0/668203013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c4de0 0x55c06e6eadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.895+0000 7f727dc00640 1 --2- 172.21.15.47:0/668203013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c43f0 0x55c06e6ec7e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f726c0307c0 tx=0x7f726c030b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1493748e6744f373 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.895+0000 7f725f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:25:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.895+0000 7f725f400640 1 -- 172.21.15.47:0/668203013 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c052030 con 0x55c06e6c43f0 2024-03-21T11:25:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.895+0000 7f72833f9740 1 -- 172.21.15.47:0/668203013 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f726c0304b0 con 0x55c06e6c43f0 2024-03-21T11:25:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.895+0000 7f725f400640 1 -- 172.21.15.47:0/668203013 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f726c03e070 con 0x55c06e6c43f0 2024-03-21T11:25:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.895+0000 7f725f400640 1 -- 172.21.15.47:0/668203013 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f726c05f050 con 0x55c06e6c43f0 2024-03-21T11:25:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.896+0000 7f72833f9740 10 client.26011 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.896+0000 7f72833f9740 1 -- 172.21.15.47:0/668203013 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c06e736040 con 0x55c06e6c43f0 2024-03-21T11:25:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.896+0000 7f725f400640 1 -- 172.21.15.47:0/668203013 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f726c0304b0 con 0x55c06e6c43f0 2024-03-21T11:25:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.896+0000 7f725f400640 1 -- 172.21.15.47:0/668203013 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f726c04e8c0 con 0x55c06e6c43f0 2024-03-21T11:25:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.896+0000 7f72833f9740 10 client.26011 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.896+0000 7f72833f9740 10 client.26011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.896+0000 7f72833f9740 20 client.26011 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.896+0000 7f72833f9740 1 --2- 172.21.15.47:0/668203013 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c06e736940 0x55c06e738d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.896+0000 7f72833f9740 4 client.26011 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:25:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.896+0000 7f72833f9740 1 -- 172.21.15.47:0/668203013 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c06e6310c0 con 0x55c06e736940 2024-03-21T11:25:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.896+0000 7f727d200640 1 --2- 172.21.15.47:0/668203013 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c06e736940 0x55c06e738d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.897+0000 7f727d200640 1 --2- 172.21.15.47:0/668203013 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c06e736940 0x55c06e738d60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62c49148232b845e server_cookie=cc67a639a008db47 in_seq=0 out_seq=0 2024-03-21T11:25:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.897+0000 7f725f400640 10 client.26011 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.900+0000 7f725f400640 1 -- 172.21.15.47:0/668203013 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c06e6310c0 con 0x55c06e736940 2024-03-21T11:25:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.900+0000 7f725f400640 10 client.26011 handle_command_reply: tid=0 2024-03-21T11:25:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.900+0000 7f72833f9740 1 client.26011 shutdown 2024-03-21T11:25:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.900+0000 7f72833f9740 1 -- 172.21.15.47:0/668203013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c43f0 msgr2=0x55c06e6ec7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.900+0000 7f72833f9740 1 --2- 172.21.15.47:0/668203013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c43f0 0x55c06e6ec7e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f726c0307c0 tx=0x7f726c030b70 comp rx=0 tx=0).stop 2024-03-21T11:25:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.901+0000 7f72833f9740 1 -- 172.21.15.47:0/668203013 shutdown_connections 2024-03-21T11:25:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.901+0000 7f72833f9740 1 --2- 172.21.15.47:0/668203013 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c06e736940 0x55c06e738d60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.901+0000 7f72833f9740 1 --2- 172.21.15.47:0/668203013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c06e6c9f50 0x55c06e6eb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.901+0000 7f72833f9740 1 --2- 172.21.15.47:0/668203013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c06e6c4de0 0x55c06e6eadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.901+0000 7f72833f9740 1 --2- 172.21.15.47:0/668203013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c06e6c43f0 0x55c06e6ec7e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.901+0000 7f72833f9740 1 -- 172.21.15.47:0/668203013 >> 172.21.15.47:0/668203013 conn(0x55c06e6bf960 msgr2=0x55c06e6ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.901+0000 7f725f400640 0 client.26011 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.901+0000 7f72833f9740 1 -- 172.21.15.47:0/668203013 shutdown_connections 2024-03-21T11:25:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.901+0000 7f72833f9740 1 -- 172.21.15.47:0/668203013 wait complete. 2024-03-21T11:25:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.901+0000 7f72833f9740 20 client.26011 trim_cache size 0 max 16384 2024-03-21T11:25:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:25:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:25:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:25:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:25:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.902+0000 7f727f000640 1 -- 172.21.15.47:0/2994292480 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72600a2f10 msgr2=0x7f72600a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.902+0000 7f727f000640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72600a2f10 0x7f72600a53d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55c06e6c51e0 tx=0x7f726c00b5a0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.902+0000 7f727f000640 1 -- 172.21.15.47:0/2994292480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7278104910 msgr2=0x7f7278199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.902+0000 7f727f000640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7278104910 0x7f7278199f00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f726400e9f0 tx=0x7f726400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.902+0000 7f727f000640 1 -- 172.21.15.47:0/2994292480 shutdown_connections 2024-03-21T11:25:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.902+0000 7f727f000640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72600a2f10 0x7f72600a53d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.902+0000 7f727f000640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72781052d0 0x7f72781a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.902+0000 7f727f000640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7278104910 0x7f7278199f00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.902+0000 7f727f000640 1 --2- 172.21.15.47:0/2994292480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7278103720 0x7f72781999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.902+0000 7f727f000640 1 -- 172.21.15.47:0/2994292480 >> 172.21.15.47:0/2994292480 conn(0x7f72780fecb0 msgr2=0x7f7278107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.903+0000 7f727f000640 1 -- 172.21.15.47:0/2994292480 shutdown_connections 2024-03-21T11:25:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:08.903+0000 7f727f000640 1 -- 172.21.15.47:0/2994292480 wait complete. 2024-03-21T11:25:10.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:10 smithi047 ceph-mon[31351]: pgmap v455: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-21T11:25:10.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:10 smithi100 ceph-mon[36245]: pgmap v455: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-21T11:25:10.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:10 smithi102 ceph-mon[36748]: pgmap v455: 129 pgs: 129 active+clean; 677 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-21T11:25:12.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:12 smithi047 ceph-mon[31351]: pgmap v456: 129 pgs: 129 active+clean; 678 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T11:25:12.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:12 smithi100 ceph-mon[36245]: pgmap v456: 129 pgs: 129 active+clean; 678 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T11:25:12.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:12 smithi102 ceph-mon[36748]: pgmap v456: 129 pgs: 129 active+clean; 678 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T11:25:14.350 INFO:tasks.workunit.client.0.smithi047.stderr:Updating files: 61% (7515/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-21T11:25:14.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:14 smithi047 ceph-mon[31351]: pgmap v457: 129 pgs: 129 active+clean; 678 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-21T11:25:14.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:14 smithi100 ceph-mon[36245]: pgmap v457: 129 pgs: 129 active+clean; 678 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-21T11:25:14.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:14 smithi102 ceph-mon[36748]: pgmap v457: 129 pgs: 129 active+clean; 678 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-21T11:25:14.855 INFO:tasks.workunit.client.0.smithi047.stderr:Note: switching to '2edfee68ad7243bed422a049b6f5f58619300f11'. 2024-03-21T11:25:14.856 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T11:25:14.856 INFO:tasks.workunit.client.0.smithi047.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-21T11:25:14.856 INFO:tasks.workunit.client.0.smithi047.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-21T11:25:14.856 INFO:tasks.workunit.client.0.smithi047.stderr:state without impacting any branches by switching back to a branch. 2024-03-21T11:25:14.856 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T11:25:14.856 INFO:tasks.workunit.client.0.smithi047.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-21T11:25:14.856 INFO:tasks.workunit.client.0.smithi047.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-21T11:25:14.856 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T11:25:14.856 INFO:tasks.workunit.client.0.smithi047.stderr: git switch -c 2024-03-21T11:25:14.856 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T11:25:14.856 INFO:tasks.workunit.client.0.smithi047.stderr:Or undo this operation with: 2024-03-21T11:25:14.856 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T11:25:14.856 INFO:tasks.workunit.client.0.smithi047.stderr: git switch - 2024-03-21T11:25:14.856 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T11:25:14.857 INFO:tasks.workunit.client.0.smithi047.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-21T11:25:14.857 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T11:25:14.857 INFO:tasks.workunit.client.0.smithi047.stderr:HEAD is now at 2edfee68ad7 Merge PR #55768 into wip-vshankar-testing-20240320.053002 2024-03-21T11:25:14.866 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-21T11:25:14.933 INFO:tasks.workunit.client.0.smithi047.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-21T11:25:14.940 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T11:25:14.940 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-21T11:25:15.874 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-21T11:25:16.039 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-21T11:25:16.130 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T11:25:16.136 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T11:25:16.136 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-21T11:25:16.221 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T11:25:16.233 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:25:16.233 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-21T11:25:16.286 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-03-21T11:25:16.288 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-03-21T11:25:16.288 DEBUG:teuthology.orchestra.run.smithi047:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2edfee68ad7243bed422a049b6f5f58619300f11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-03-21T11:25:16.358 INFO:tasks.workunit.client.0.smithi047.stderr:+ dbench 1 2024-03-21T11:25:16.369 INFO:tasks.workunit.client.0.smithi047.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-21T11:25:16.369 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-21T11:25:16.369 INFO:tasks.workunit.client.0.smithi047.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-21T11:25:16.384 INFO:tasks.workunit.client.0.smithi047.stdout:0 of 1 processes prepared for launch 0 sec 2024-03-21T11:25:16.484 INFO:tasks.workunit.client.0.smithi047.stdout:1 of 1 processes prepared for launch 0 sec 2024-03-21T11:25:16.485 INFO:tasks.workunit.client.0.smithi047.stdout:releasing clients 2024-03-21T11:25:16.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:16 smithi047 ceph-mon[31351]: pgmap v458: 129 pgs: 129 active+clean; 678 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T11:25:16.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:16 smithi100 ceph-mon[36245]: pgmap v458: 129 pgs: 129 active+clean; 678 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T11:25:16.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:16 smithi102 ceph-mon[36748]: pgmap v458: 129 pgs: 129 active+clean; 678 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T11:25:17.485 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1552 26.09 MB/sec warmup 1 sec latency 13.464 ms 2024-03-21T11:25:18.485 INFO:tasks.workunit.client.0.smithi047.stdout: 1 3860 19.88 MB/sec warmup 2 sec latency 14.700 ms 2024-03-21T11:25:18.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:18 smithi047 ceph-mon[31351]: pgmap v459: 129 pgs: 129 active+clean; 678 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-21T11:25:18.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:18 smithi100 ceph-mon[36245]: pgmap v459: 129 pgs: 129 active+clean; 678 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-21T11:25:18.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:18 smithi102 ceph-mon[36748]: pgmap v459: 129 pgs: 129 active+clean; 678 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-21T11:25:19.485 INFO:tasks.workunit.client.0.smithi047.stdout: 1 5451 15.16 MB/sec warmup 3 sec latency 10.333 ms 2024-03-21T11:25:20.485 INFO:tasks.workunit.client.0.smithi047.stdout: 1 7492 14.79 MB/sec warmup 4 sec latency 11.180 ms 2024-03-21T11:25:20.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:20 smithi047 ceph-mon[31351]: pgmap v460: 129 pgs: 129 active+clean; 845 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-03-21T11:25:20.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:20 smithi100 ceph-mon[36245]: pgmap v460: 129 pgs: 129 active+clean; 845 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-03-21T11:25:20.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:20 smithi102 ceph-mon[36748]: pgmap v460: 129 pgs: 129 active+clean; 845 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-03-21T11:25:21.485 INFO:tasks.workunit.client.0.smithi047.stdout: 1 9427 13.21 MB/sec warmup 5 sec latency 9.976 ms 2024-03-21T11:25:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.063+0000 7ffa0a800640 1 -- 172.21.15.47:0/1632302159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa04103900 msgr2=0x7ffa04103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.063+0000 7ffa0a800640 1 --2- 172.21.15.47:0/1632302159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa04103900 0x7ffa04103d20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009ca0 tx=0x7ff9f402c0f0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.063+0000 7ffa0a800640 1 -- 172.21.15.47:0/1632302159 shutdown_connections 2024-03-21T11:25:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.063+0000 7ffa0a800640 1 --2- 172.21.15.47:0/1632302159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa041054b0 0x7ffa0410bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.063+0000 7ffa0a800640 1 --2- 172.21.15.47:0/1632302159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04104af0 0x7ffa04104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.063+0000 7ffa0a800640 1 --2- 172.21.15.47:0/1632302159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa04103900 0x7ffa04103d20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.063+0000 7ffa0a800640 1 -- 172.21.15.47:0/1632302159 >> 172.21.15.47:0/1632302159 conn(0x7ffa040fee90 msgr2=0x7ffa041012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.063+0000 7ffa0a800640 1 -- 172.21.15.47:0/1632302159 shutdown_connections 2024-03-21T11:25:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.063+0000 7ffa0a800640 1 -- 172.21.15.47:0/1632302159 wait complete. 2024-03-21T11:25:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.064+0000 7ffa0a800640 1 Processor -- start 2024-03-21T11:25:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.064+0000 7ffa0a800640 1 -- start start 2024-03-21T11:25:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa0a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04103900 0x7ffa04199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa0a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04104af0 0x7ffa0419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa0a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa041054b0 0x7ffa041a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa0a800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0410f580 con 0x7ffa04104af0 2024-03-21T11:25:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa0a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0410f400 con 0x7ffa04103900 2024-03-21T11:25:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa0a800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0410f700 con 0x7ffa041054b0 2024-03-21T11:25:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04103900 0x7ffa04199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04103900 0x7ffa04199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41520/0 (socket says 172.21.15.47:41520) 2024-03-21T11:25:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa03400640 1 -- 172.21.15.47:0/413108964 learned_addr learned my addr 172.21.15.47:0/413108964 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa02a00640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04104af0 0x7ffa0419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa03e00640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa041054b0 0x7ffa041a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa03400640 1 -- 172.21.15.47:0/413108964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa041054b0 msgr2=0x7ffa041a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa03400640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa041054b0 0x7ffa041a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa03400640 1 -- 172.21.15.47:0/413108964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04104af0 msgr2=0x7ffa0419a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa03400640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04104af0 0x7ffa0419a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa03400640 1 -- 172.21.15.47:0/413108964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa041a1780 con 0x7ffa04103900 2024-03-21T11:25:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.065+0000 7ffa03e00640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa041054b0 0x7ffa041a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.066+0000 7ffa03400640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04103900 0x7ffa04199b60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4000f80 tx=0x7ff9f4032ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.066+0000 7ff9fbe00640 1 -- 172.21.15.47:0/413108964 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4041020 con 0x7ffa04103900 2024-03-21T11:25:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.066+0000 7ff9fbe00640 1 -- 172.21.15.47:0/413108964 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9f4034cd0 con 0x7ffa04103900 2024-03-21T11:25:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.066+0000 7ff9fbe00640 1 -- 172.21.15.47:0/413108964 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4039040 con 0x7ffa04103900 2024-03-21T11:25:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.066+0000 7ffa0a800640 1 -- 172.21.15.47:0/413108964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa041a1a10 con 0x7ffa04103900 2024-03-21T11:25:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.066+0000 7ffa0a800640 1 -- 172.21.15.47:0/413108964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa041a1e80 con 0x7ffa04103900 2024-03-21T11:25:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.069+0000 7ffa0d5ec740 1 Processor -- start 2024-03-21T11:25:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.069+0000 7ffa0d5ec740 1 -- start start 2024-03-21T11:25:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ff9fbe00640 1 -- 172.21.15.47:0/413108964 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff9f4044020 con 0x7ffa04103900 2024-03-21T11:25:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa0d5ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea701e0 0x559c0ea70600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa0d5ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea70bd0 0x559c0ea756d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa0d5ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea75d40 0x559c0ea781d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa0d5ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0ea333b0 con 0x559c0ea75d40 2024-03-21T11:25:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa0d5ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0e9d5830 con 0x559c0ea70bd0 2024-03-21T11:25:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa0d5ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0e97e5d0 con 0x559c0ea701e0 2024-03-21T11:25:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa03e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea701e0 0x559c0ea70600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea70bd0 0x559c0ea756d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa03e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea701e0 0x559c0ea70600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52514/0 (socket says 172.21.15.47:52514) 2024-03-21T11:25:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa03e00640 1 -- 172.21.15.47:0/3451611261 learned_addr learned my addr 172.21.15.47:0/3451611261 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa02a00640 1 --2- 172.21.15.47:0/3451611261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea75d40 0x559c0ea781d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa03e00640 1 -- 172.21.15.47:0/3451611261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea70bd0 msgr2=0x559c0ea756d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa03e00640 1 --2- 172.21.15.47:0/3451611261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea70bd0 0x559c0ea756d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa03e00640 1 -- 172.21.15.47:0/3451611261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea75d40 msgr2=0x559c0ea781d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa03e00640 1 --2- 172.21.15.47:0/3451611261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea75d40 0x559c0ea781d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa03e00640 1 -- 172.21.15.47:0/3451611261 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c0e9962c0 con 0x559c0ea701e0 2024-03-21T11:25:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.070+0000 7ffa02a00640 1 --2- 172.21.15.47:0/3451611261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea75d40 0x559c0ea781d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.071+0000 7ffa03e00640 1 --2- 172.21.15.47:0/3451611261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea701e0 0x559c0ea70600 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009cc0 tx=0x7ff9fc0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4dc590a4945875e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.071+0000 7ff9f8c00640 1 -- 172.21.15.47:0/3451611261 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc012450 con 0x559c0ea701e0 2024-03-21T11:25:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.071+0000 7ff9f8c00640 1 -- 172.21.15.47:0/3451611261 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9fc012ad0 con 0x559c0ea701e0 2024-03-21T11:25:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.071+0000 7ff9f8c00640 1 -- 172.21.15.47:0/3451611261 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00fc90 con 0x559c0ea701e0 2024-03-21T11:25:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.071+0000 7ff9fbe00640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff9e80a3040 0x7ff9e80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.072+0000 7ff9fbe00640 1 -- 172.21.15.47:0/413108964 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff9f40f16c0 con 0x7ffa04103900 2024-03-21T11:25:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.072+0000 7ffa03400640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff9e80a3040 0x7ff9e80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.072+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/3451611261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea701e0 msgr2=0x559c0ea70600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.072+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/3451611261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea701e0 0x559c0ea70600 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009cc0 tx=0x7ff9fc0117b0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.072+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/3451611261 shutdown_connections 2024-03-21T11:25:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.072+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/3451611261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea75d40 0x559c0ea781d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.072+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/3451611261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea70bd0 0x559c0ea756d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.072+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/3451611261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea701e0 0x559c0ea70600 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.072+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/3451611261 >> 172.21.15.47:0/3451611261 conn(0x559c0ea6b750 msgr2=0x559c0ea6bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.072+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/3451611261 shutdown_connections 2024-03-21T11:25:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.072+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/3451611261 wait complete. 2024-03-21T11:25:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.072+0000 7ffa03400640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff9e80a3040 0x7ff9e80a5500 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009bf0 tx=0x7ff9f40a6f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.073+0000 7ffa0d5ec740 1 -- start start 2024-03-21T11:25:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.073+0000 7ffa0d5ec740 1 Processor -- start 2024-03-21T11:25:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.073+0000 7ffa0d5ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea701e0 0x559c0ea7a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.073+0000 7ffa0d5ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea70bd0 0x559c0ea7a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.073+0000 7ffa0d5ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea75d40 0x559c0ea81970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.073+0000 7ffa0d5ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0e991a20 con 0x559c0ea70bd0 2024-03-21T11:25:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.073+0000 7ffa0d5ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0e9d18d0 con 0x559c0ea75d40 2024-03-21T11:25:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.073+0000 7ffa0d5ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0ea78cf0 con 0x559c0ea701e0 2024-03-21T11:25:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.073+0000 7ffa0d5ec740 10 client.0 _ll_register_callbacks cb 0x559c0e9ce1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.074+0000 7ffa02a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea70bd0 0x559c0ea7a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.074+0000 7ffa03e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea701e0 0x559c0ea7a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.074+0000 7ffa02a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea70bd0 0x559c0ea7a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34804/0 (socket says 172.21.15.47:34804) 2024-03-21T11:25:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.074+0000 7ffa02a00640 1 -- 172.21.15.47:0/963574378 learned_addr learned my addr 172.21.15.47:0/963574378 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.074+0000 7ffa03e00640 1 --2- 172.21.15.47:0/963574378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea75d40 0x559c0ea81970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.074+0000 7ffa02a00640 1 -- 172.21.15.47:0/963574378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea701e0 msgr2=0x559c0ea7a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.074+0000 7ffa02a00640 1 --2- 172.21.15.47:0/963574378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea701e0 0x559c0ea7a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.074+0000 7ffa02a00640 1 -- 172.21.15.47:0/963574378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea75d40 msgr2=0x559c0ea81970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.074+0000 7ffa02a00640 1 --2- 172.21.15.47:0/963574378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea75d40 0x559c0ea81970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.074+0000 7ffa02a00640 1 -- 172.21.15.47:0/963574378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c0e6ea210 con 0x559c0ea70bd0 2024-03-21T11:25:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.075+0000 7ffa02a00640 1 --2- 172.21.15.47:0/963574378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea70bd0 0x559c0ea7a930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000beb0 tx=0x7ff9f00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47bc6c1627ff34a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.075+0000 7ff9ed600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:25:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.075+0000 7ff9ed600640 1 -- 172.21.15.47:0/963574378 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0004230 con 0x559c0ea70bd0 2024-03-21T11:25:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.075+0000 7ff9ed600640 1 -- 172.21.15.47:0/963574378 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff9f00043d0 con 0x559c0ea70bd0 2024-03-21T11:25:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.075+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/963574378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9fc012ad0 con 0x559c0ea70bd0 2024-03-21T11:25:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.075+0000 7ff9ed600640 1 -- 172.21.15.47:0/963574378 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9f0024050 con 0x559c0ea70bd0 2024-03-21T11:25:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.075+0000 7ffa0d5ec740 10 client.15834 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.075+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/963574378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559c0e7b6620 con 0x559c0ea70bd0 2024-03-21T11:25:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.076+0000 7ff9ed600640 1 -- 172.21.15.47:0/963574378 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9fc012ad0 con 0x559c0ea70bd0 2024-03-21T11:25:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.076+0000 7ff9ed600640 1 -- 172.21.15.47:0/963574378 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff9f001b7d0 con 0x559c0ea70bd0 2024-03-21T11:25:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.076+0000 7ffa0d5ec740 10 client.15834 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.076+0000 7ffa0d5ec740 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.076+0000 7ffa0d5ec740 20 client.15834 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.076+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/963574378 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559c0ea86300 0x559c0ea88720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.076+0000 7ffa0d5ec740 4 client.15834 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:25:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.076+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/963574378 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c0e9e2d70 con 0x559c0ea86300 2024-03-21T11:25:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.076+0000 7ffa03e00640 1 --2- 172.21.15.47:0/963574378 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559c0ea86300 0x559c0ea88720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.077+0000 7ffa03e00640 1 --2- 172.21.15.47:0/963574378 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559c0ea86300 0x559c0ea88720 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=438dbd750eed721 server_cookie=88ae75dbc8bfbbb1 in_seq=0 out_seq=0 2024-03-21T11:25:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.077+0000 7ff9ed600640 10 client.15834 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.078+0000 7ff9ed600640 1 -- 172.21.15.47:0/963574378 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559c0e9e2d70 con 0x559c0ea86300 2024-03-21T11:25:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.078+0000 7ff9ed600640 10 client.15834 handle_command_reply: tid=0 2024-03-21T11:25:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.078+0000 7ffa0d5ec740 1 client.15834 shutdown 2024-03-21T11:25:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.078+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/963574378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea70bd0 msgr2=0x559c0ea7a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.078+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/963574378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea70bd0 0x559c0ea7a930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000beb0 tx=0x7ff9f00117b0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.078+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/963574378 shutdown_connections 2024-03-21T11:25:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.078+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/963574378 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559c0ea86300 0x559c0ea88720 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.078+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/963574378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea75d40 0x559c0ea81970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.078+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/963574378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea70bd0 0x559c0ea7a930 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.078+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/963574378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea701e0 0x559c0ea7a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.078+0000 7ff9ed600640 0 client.15834 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.078+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/963574378 >> 172.21.15.47:0/963574378 conn(0x559c0ea6b750 msgr2=0x559c0ea6a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.078+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/963574378 shutdown_connections 2024-03-21T11:25:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.078+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/963574378 wait complete. 2024-03-21T11:25:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.079+0000 7ffa0d5ec740 20 client.15834 trim_cache size 0 max 16384 2024-03-21T11:25:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.094+0000 7ffa0d5ec740 1 Processor -- start 2024-03-21T11:25:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.094+0000 7ffa0d5ec740 1 -- start start 2024-03-21T11:25:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.094+0000 7ffa0d5ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea701e0 0x559c0ea665d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.094+0000 7ffa0d5ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea70bd0 0x559c0ea81930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.094+0000 7ffa0d5ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea75d40 0x559c0ea6a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.094+0000 7ffa0d5ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa041ac280 con 0x559c0ea701e0 2024-03-21T11:25:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.094+0000 7ffa0d5ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa041adc00 con 0x559c0ea75d40 2024-03-21T11:25:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.094+0000 7ffa0d5ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa041ac8e0 con 0x559c0ea70bd0 2024-03-21T11:25:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.095+0000 7ffa02a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea70bd0 0x559c0ea81930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.095+0000 7ffa02a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea70bd0 0x559c0ea81930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52536/0 (socket says 172.21.15.47:52536) 2024-03-21T11:25:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.095+0000 7ffa02a00640 1 -- 172.21.15.47:0/1699766638 learned_addr learned my addr 172.21.15.47:0/1699766638 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.095+0000 7ffa03e00640 1 --2- 172.21.15.47:0/1699766638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea701e0 0x559c0ea665d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.095+0000 7ffa03e00640 1 --2- 172.21.15.47:0/1699766638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea75d40 0x559c0ea6a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.095+0000 7ffa02a00640 1 -- 172.21.15.47:0/1699766638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea75d40 msgr2=0x559c0ea6a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.095+0000 7ffa02a00640 1 --2- 172.21.15.47:0/1699766638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea75d40 0x559c0ea6a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.095+0000 7ffa02a00640 1 -- 172.21.15.47:0/1699766638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea701e0 msgr2=0x559c0ea665d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.095+0000 7ffa02a00640 1 --2- 172.21.15.47:0/1699766638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea701e0 0x559c0ea665d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.095+0000 7ffa02a00640 1 -- 172.21.15.47:0/1699766638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c0ea88d80 con 0x559c0ea70bd0 2024-03-21T11:25:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.095+0000 7ffa03e00640 1 --2- 172.21.15.47:0/1699766638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea701e0 0x559c0ea665d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.095+0000 7ffa02a00640 1 --2- 172.21.15.47:0/1699766638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea70bd0 0x559c0ea81930 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff9f001bb40 tx=0x7ff9f000baa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38a6a3e3260987ac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.096+0000 7ff9efe00640 1 -- 172.21.15.47:0/1699766638 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0011cb0 con 0x559c0ea70bd0 2024-03-21T11:25:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.096+0000 7ff9efe00640 1 -- 172.21.15.47:0/1699766638 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9f002c070 con 0x559c0ea70bd0 2024-03-21T11:25:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.096+0000 7ff9efe00640 1 -- 172.21.15.47:0/1699766638 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0005330 con 0x559c0ea70bd0 2024-03-21T11:25:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.096+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/1699766638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea70bd0 msgr2=0x559c0ea81930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.096+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/1699766638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea70bd0 0x559c0ea81930 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff9f001bb40 tx=0x7ff9f000baa0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.096+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/1699766638 shutdown_connections 2024-03-21T11:25:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.096+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/1699766638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea75d40 0x559c0ea6a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.096+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/1699766638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea70bd0 0x559c0ea81930 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.096+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/1699766638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea701e0 0x559c0ea665d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.096+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/1699766638 >> 172.21.15.47:0/1699766638 conn(0x559c0ea6b750 msgr2=0x559c0ea71ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.096+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/1699766638 shutdown_connections 2024-03-21T11:25:22.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.096+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/1699766638 wait complete. 2024-03-21T11:25:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.097+0000 7ffa0d5ec740 1 -- start start 2024-03-21T11:25:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.097+0000 7ffa0d5ec740 1 Processor -- start 2024-03-21T11:25:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa0d5ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea701e0 0x559c0ea7f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa0d5ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea70bd0 0x559c0ea7fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa0d5ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea75d40 0x559c0ea80b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa0d5ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa041ad5a0 con 0x559c0ea75d40 2024-03-21T11:25:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa0d5ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa041acf40 con 0x559c0ea701e0 2024-03-21T11:25:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa0d5ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa041aa440 con 0x559c0ea70bd0 2024-03-21T11:25:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa0d5ec740 10 client.0 _ll_register_callbacks cb 0x7ff9f000b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa02a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea70bd0 0x559c0ea7fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa02a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea70bd0 0x559c0ea7fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52538/0 (socket says 172.21.15.47:52538) 2024-03-21T11:25:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa02a00640 1 -- 172.21.15.47:0/3401364778 learned_addr learned my addr 172.21.15.47:0/3401364778 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa03e00640 1 --2- 172.21.15.47:0/3401364778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea701e0 0x559c0ea7f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa03e00640 1 --2- 172.21.15.47:0/3401364778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea75d40 0x559c0ea80b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa02a00640 1 -- 172.21.15.47:0/3401364778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea701e0 msgr2=0x559c0ea7f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa02a00640 1 --2- 172.21.15.47:0/3401364778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea701e0 0x559c0ea7f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa02a00640 1 -- 172.21.15.47:0/3401364778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea75d40 msgr2=0x559c0ea80b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa02a00640 1 --2- 172.21.15.47:0/3401364778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea75d40 0x559c0ea80b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa02a00640 1 -- 172.21.15.47:0/3401364778 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c0ea6c1c0 con 0x559c0ea70bd0 2024-03-21T11:25:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.098+0000 7ffa03e00640 1 --2- 172.21.15.47:0/3401364778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea75d40 0x559c0ea80b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:25:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.099+0000 7ffa02a00640 1 --2- 172.21.15.47:0/3401364778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea70bd0 0x559c0ea7fb50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000bdd0 tx=0x7ff9f001b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b831bff6b3b2a69 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.099+0000 7ff9ed600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:25:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.099+0000 7ff9ed600640 1 -- 172.21.15.47:0/3401364778 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0035070 con 0x559c0ea70bd0 2024-03-21T11:25:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.099+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/3401364778 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9f002c070 con 0x559c0ea70bd0 2024-03-21T11:25:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.099+0000 7ff9ed600640 1 -- 172.21.15.47:0/3401364778 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff9f00052c0 con 0x559c0ea70bd0 2024-03-21T11:25:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.099+0000 7ff9ed600640 1 -- 172.21.15.47:0/3401364778 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9f0011cb0 con 0x559c0ea70bd0 2024-03-21T11:25:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.099+0000 7ffa0d5ec740 10 client.26029 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.099+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/3401364778 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559c0ea6c4c0 con 0x559c0ea70bd0 2024-03-21T11:25:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.099+0000 7ff9ed600640 1 -- 172.21.15.47:0/3401364778 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9f002c070 con 0x559c0ea70bd0 2024-03-21T11:25:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.100+0000 7ff9ed600640 1 -- 172.21.15.47:0/3401364778 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff9f00314f0 con 0x559c0ea70bd0 2024-03-21T11:25:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.100+0000 7ffa0d5ec740 10 client.26029 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.100+0000 7ffa0d5ec740 10 client.26029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.100+0000 7ffa0d5ec740 20 client.26029 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.100+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/3401364778 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559c0eae2630 0x559c0eae4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.100+0000 7ffa0d5ec740 4 client.26029 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:25:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.100+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/3401364778 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559c0e9dd0c0 con 0x559c0eae2630 2024-03-21T11:25:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.100+0000 7ffa03e00640 1 --2- 172.21.15.47:0/3401364778 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559c0eae2630 0x559c0eae4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.100+0000 7ffa03e00640 1 --2- 172.21.15.47:0/3401364778 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559c0eae2630 0x559c0eae4a50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82245786b74d716a server_cookie=a65e4302538c1c6b in_seq=0 out_seq=0 2024-03-21T11:25:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.100+0000 7ff9ed600640 10 client.26029 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ff9ed600640 1 -- 172.21.15.47:0/3401364778 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559c0e9dd0c0 con 0x559c0eae2630 2024-03-21T11:25:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ff9ed600640 10 client.26029 handle_command_reply: tid=0 2024-03-21T11:25:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ffa0d5ec740 1 client.26029 shutdown 2024-03-21T11:25:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/3401364778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea70bd0 msgr2=0x559c0ea7fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/3401364778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea70bd0 0x559c0ea7fb50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000bdd0 tx=0x7ff9f001b720 comp rx=0 tx=0).stop 2024-03-21T11:25:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/3401364778 shutdown_connections 2024-03-21T11:25:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/3401364778 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559c0eae2630 0x559c0eae4a50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/3401364778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c0ea75d40 0x559c0ea80b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/3401364778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c0ea70bd0 0x559c0ea7fb50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ffa0d5ec740 1 --2- 172.21.15.47:0/3401364778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559c0ea701e0 0x559c0ea7f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ff9ed600640 0 client.26029 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/3401364778 >> 172.21.15.47:0/3401364778 conn(0x559c0ea6b750 msgr2=0x559c0ea6e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/3401364778 shutdown_connections 2024-03-21T11:25:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ffa0d5ec740 1 -- 172.21.15.47:0/3401364778 wait complete. 2024-03-21T11:25:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.306+0000 7ffa0d5ec740 20 client.26029 trim_cache size 0 max 16384 2024-03-21T11:25:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:25:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:25:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:25:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:25:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.307+0000 7ffa0a800640 1 -- 172.21.15.47:0/413108964 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff9e80a3040 msgr2=0x7ff9e80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.307+0000 7ffa0a800640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff9e80a3040 0x7ff9e80a5500 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009bf0 tx=0x7ff9f40a6f90 comp rx=0 tx=0).stop 2024-03-21T11:25:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.307+0000 7ffa0a800640 1 -- 172.21.15.47:0/413108964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04103900 msgr2=0x7ffa04199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.307+0000 7ffa0a800640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04103900 0x7ffa04199b60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4000f80 tx=0x7ff9f4032ec0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.307+0000 7ffa0a800640 1 -- 172.21.15.47:0/413108964 shutdown_connections 2024-03-21T11:25:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.307+0000 7ffa0a800640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff9e80a3040 0x7ff9e80a5500 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.307+0000 7ffa0a800640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa041054b0 0x7ffa041a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.307+0000 7ffa0a800640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04104af0 0x7ffa0419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.307+0000 7ffa0a800640 1 --2- 172.21.15.47:0/413108964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04103900 0x7ffa04199b60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.307+0000 7ffa0a800640 1 -- 172.21.15.47:0/413108964 >> 172.21.15.47:0/413108964 conn(0x7ffa040fee90 msgr2=0x7ffa04107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.307+0000 7ffa0a800640 1 -- 172.21.15.47:0/413108964 shutdown_connections 2024-03-21T11:25:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:22.307+0000 7ffa0a800640 1 -- 172.21.15.47:0/413108964 wait complete. 2024-03-21T11:25:22.486 INFO:tasks.workunit.client.0.smithi047.stdout: 1 11017 13.05 MB/sec warmup 6 sec latency 58.956 ms 2024-03-21T11:25:22.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:22 smithi047 ceph-mon[31351]: pgmap v461: 129 pgs: 129 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 274 KiB/s wr, 52 op/s 2024-03-21T11:25:22.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:22 smithi100 ceph-mon[36245]: pgmap v461: 129 pgs: 129 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 274 KiB/s wr, 52 op/s 2024-03-21T11:25:22.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:22 smithi102 ceph-mon[36748]: pgmap v461: 129 pgs: 129 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 274 KiB/s wr, 52 op/s 2024-03-21T11:25:23.486 INFO:tasks.workunit.client.0.smithi047.stdout: 1 12542 11.96 MB/sec warmup 7 sec latency 24.613 ms 2024-03-21T11:25:24.486 INFO:tasks.workunit.client.0.smithi047.stdout: 1 14583 12.18 MB/sec warmup 8 sec latency 9.160 ms 2024-03-21T11:25:24.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:24 smithi047 ceph-mon[31351]: pgmap v462: 129 pgs: 129 active+clean; 5.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 435 KiB/s wr, 106 op/s 2024-03-21T11:25:24.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:24 smithi100 ceph-mon[36245]: pgmap v462: 129 pgs: 129 active+clean; 5.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 435 KiB/s wr, 106 op/s 2024-03-21T11:25:24.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:24 smithi102 ceph-mon[36748]: pgmap v462: 129 pgs: 129 active+clean; 5.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 435 KiB/s wr, 106 op/s 2024-03-21T11:25:25.486 INFO:tasks.workunit.client.0.smithi047.stdout: 1 16395 11.47 MB/sec warmup 9 sec latency 12.051 ms 2024-03-21T11:25:26.486 INFO:tasks.workunit.client.0.smithi047.stdout: 1 18122 11.65 MB/sec warmup 10 sec latency 81.519 ms 2024-03-21T11:25:26.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:26 smithi047 ceph-mon[31351]: pgmap v463: 129 pgs: 129 active+clean; 5.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 460 KiB/s wr, 128 op/s 2024-03-21T11:25:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:26 smithi100 ceph-mon[36245]: pgmap v463: 129 pgs: 129 active+clean; 5.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 460 KiB/s wr, 128 op/s 2024-03-21T11:25:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:26 smithi102 ceph-mon[36748]: pgmap v463: 129 pgs: 129 active+clean; 5.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 460 KiB/s wr, 128 op/s 2024-03-21T11:25:27.486 INFO:tasks.workunit.client.0.smithi047.stdout: 1 19836 11.11 MB/sec warmup 11 sec latency 22.067 ms 2024-03-21T11:25:27.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:25:27.500 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:25:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.741+0000 7fa083000640 1 -- 172.21.15.47:0/2843800571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07c1038e0 msgr2=0x7fa07c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.741+0000 7fa083000640 1 --2- 172.21.15.47:0/2843800571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07c1038e0 0x7fa07c103d00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009900 tx=0x7fa06c02bc50 comp rx=0 tx=0).stop 2024-03-21T11:25:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.741+0000 7fa083000640 1 -- 172.21.15.47:0/2843800571 shutdown_connections 2024-03-21T11:25:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.741+0000 7fa083000640 1 --2- 172.21.15.47:0/2843800571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa07c105490 0x7fa07c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.741+0000 7fa083000640 1 --2- 172.21.15.47:0/2843800571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa07c104ad0 0x7fa07c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.741+0000 7fa083000640 1 --2- 172.21.15.47:0/2843800571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07c1038e0 0x7fa07c103d00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.741+0000 7fa083000640 1 -- 172.21.15.47:0/2843800571 >> 172.21.15.47:0/2843800571 conn(0x7fa07c0fee50 msgr2=0x7fa07c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.741+0000 7fa083000640 1 -- 172.21.15.47:0/2843800571 shutdown_connections 2024-03-21T11:25:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.741+0000 7fa083000640 1 -- 172.21.15.47:0/2843800571 wait complete. 2024-03-21T11:25:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.742+0000 7fa083000640 1 Processor -- start 2024-03-21T11:25:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.743+0000 7fa083000640 1 -- start start 2024-03-21T11:25:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.743+0000 7fa083000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa07c1038e0 0x7fa07c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.743+0000 7fa083000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07c104ad0 0x7fa07c19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.743+0000 7fa083000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa07c105490 0x7fa07c1a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.743+0000 7fa083000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c10f580 con 0x7fa07c105490 2024-03-21T11:25:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.743+0000 7fa083000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c10f400 con 0x7fa07c104ad0 2024-03-21T11:25:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.743+0000 7fa083000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c10f700 con 0x7fa07c1038e0 2024-03-21T11:25:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.743+0000 7fa07be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa07c1038e0 0x7fa07c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.743+0000 7fa07be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa07c1038e0 0x7fa07c199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52558/0 (socket says 172.21.15.47:52558) 2024-03-21T11:25:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.743+0000 7fa07be00640 1 -- 172.21.15.47:0/2952348993 learned_addr learned my addr 172.21.15.47:0/2952348993 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.743+0000 7fa081000640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa07c105490 0x7fa07c1a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.743+0000 7fa073e00640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07c104ad0 0x7fa07c19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.744+0000 7fa07be00640 1 -- 172.21.15.47:0/2952348993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07c104ad0 msgr2=0x7fa07c19a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.744+0000 7fa07be00640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07c104ad0 0x7fa07c19a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.744+0000 7fa07be00640 1 -- 172.21.15.47:0/2952348993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa07c105490 msgr2=0x7fa07c1a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.744+0000 7fa07be00640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa07c105490 0x7fa07c1a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.744+0000 7fa07be00640 1 -- 172.21.15.47:0/2952348993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c1a1880 con 0x7fa07c1038e0 2024-03-21T11:25:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.744+0000 7fa073e00640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07c104ad0 0x7fa07c19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.744+0000 7fa081000640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa07c105490 0x7fa07c1a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.744+0000 7fa07be00640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa07c1038e0 0x7fa07c199c70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa06c02c260 tx=0x7fa06c02c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.744+0000 7fa079600640 1 -- 172.21.15.47:0/2952348993 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c041070 con 0x7fa07c1038e0 2024-03-21T11:25:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.744+0000 7fa079600640 1 -- 172.21.15.47:0/2952348993 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa06c004450 con 0x7fa07c1038e0 2024-03-21T11:25:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.744+0000 7fa079600640 1 -- 172.21.15.47:0/2952348993 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c034440 con 0x7fa07c1038e0 2024-03-21T11:25:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.744+0000 7fa083000640 1 -- 172.21.15.47:0/2952348993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07c1a1b10 con 0x7fa07c1038e0 2024-03-21T11:25:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.745+0000 7fa083000640 1 -- 172.21.15.47:0/2952348993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07c1a2060 con 0x7fa07c1038e0 2024-03-21T11:25:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.748+0000 7fa079600640 1 -- 172.21.15.47:0/2952348993 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa06c044070 con 0x7fa07c1038e0 2024-03-21T11:25:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.748+0000 7fa085d90740 1 Processor -- start 2024-03-21T11:25:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.748+0000 7fa085d90740 1 -- start start 2024-03-21T11:25:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.748+0000 7fa085d90740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f121562e80 0x55f1215632a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.748+0000 7fa085d90740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f121568370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.748+0000 7fa085d90740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1215689e0 0x55f12156ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.748+0000 7fa085d90740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1214c8250 con 0x55f121563870 2024-03-21T11:25:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.748+0000 7fa085d90740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f121470a30 con 0x55f1215689e0 2024-03-21T11:25:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.748+0000 7fa085d90740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1214c42f0 con 0x55f121562e80 2024-03-21T11:25:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.749+0000 7fa081000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f121562e80 0x55f1215632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.749+0000 7fa081000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f121562e80 0x55f1215632a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52564/0 (socket says 172.21.15.47:52564) 2024-03-21T11:25:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.749+0000 7fa081000640 1 -- 172.21.15.47:0/2499578562 learned_addr learned my addr 172.21.15.47:0/2499578562 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.749+0000 7fa07be00640 1 --2- 172.21.15.47:0/2499578562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f121568370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.749+0000 7fa073e00640 1 --2- 172.21.15.47:0/2499578562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1215689e0 0x55f12156ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.749+0000 7fa081000640 1 -- 172.21.15.47:0/2499578562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1215689e0 msgr2=0x55f12156ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.749+0000 7fa081000640 1 --2- 172.21.15.47:0/2499578562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1215689e0 0x55f12156ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.749+0000 7fa081000640 1 -- 172.21.15.47:0/2499578562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 msgr2=0x55f121568370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.749+0000 7fa081000640 1 --2- 172.21.15.47:0/2499578562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f121568370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.749+0000 7fa081000640 1 -- 172.21.15.47:0/2499578562 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f12145b690 con 0x55f121562e80 2024-03-21T11:25:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.749+0000 7fa07be00640 1 --2- 172.21.15.47:0/2499578562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f121568370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.749+0000 7fa081000640 1 --2- 172.21.15.47:0/2499578562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f121562e80 0x55f1215632a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa074009fd0 tx=0x7fa0740108e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a22375648355100e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.750+0000 7fa071600640 1 -- 172.21.15.47:0/2499578562 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa074011760 con 0x55f121562e80 2024-03-21T11:25:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.750+0000 7fa079600640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa0600a2f20 0x7fa0600a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.750+0000 7fa071600640 1 -- 172.21.15.47:0/2499578562 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa074011de0 con 0x55f121562e80 2024-03-21T11:25:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.750+0000 7fa071600640 1 -- 172.21.15.47:0/2499578562 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07400e3c0 con 0x55f121562e80 2024-03-21T11:25:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.750+0000 7fa079600640 1 -- 172.21.15.47:0/2952348993 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa06c0f1420 con 0x7fa07c1038e0 2024-03-21T11:25:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.750+0000 7fa07be00640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa0600a2f20 0x7fa0600a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.750+0000 7fa085d90740 1 -- 172.21.15.47:0/2499578562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f121562e80 msgr2=0x55f1215632a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.750+0000 7fa085d90740 1 --2- 172.21.15.47:0/2499578562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f121562e80 0x55f1215632a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa074009fd0 tx=0x7fa0740108e0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.750+0000 7fa085d90740 1 -- 172.21.15.47:0/2499578562 shutdown_connections 2024-03-21T11:25:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.750+0000 7fa085d90740 1 --2- 172.21.15.47:0/2499578562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1215689e0 0x55f12156ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.750+0000 7fa085d90740 1 --2- 172.21.15.47:0/2499578562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f121568370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.750+0000 7fa085d90740 1 --2- 172.21.15.47:0/2499578562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f121562e80 0x55f1215632a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.750+0000 7fa085d90740 1 -- 172.21.15.47:0/2499578562 >> 172.21.15.47:0/2499578562 conn(0x55f12152b200 msgr2=0x55f12155e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.751+0000 7fa085d90740 1 -- 172.21.15.47:0/2499578562 shutdown_connections 2024-03-21T11:25:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.751+0000 7fa07be00640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa0600a2f20 0x7fa0600a53e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0c5c40 tx=0x7fa06c02bc50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.751+0000 7fa085d90740 1 -- 172.21.15.47:0/2499578562 wait complete. 2024-03-21T11:25:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.751+0000 7fa085d90740 1 -- start start 2024-03-21T11:25:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.751+0000 7fa085d90740 1 Processor -- start 2024-03-21T11:25:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.752+0000 7fa085d90740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121562e80 0x55f121575660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.752+0000 7fa085d90740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f121575ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.752+0000 7fa085d90740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f1215689e0 0x55f12157cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.752+0000 7fa085d90740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f12156b990 con 0x55f121563870 2024-03-21T11:25:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.752+0000 7fa085d90740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f121483e80 con 0x55f121562e80 2024-03-21T11:25:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.752+0000 7fa085d90740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f12156bb10 con 0x55f1215689e0 2024-03-21T11:25:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.752+0000 7fa085d90740 10 client.0 _ll_register_callbacks cb 0x55f121496130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.752+0000 7fa081000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121562e80 0x55f121575660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.752+0000 7fa073e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f121575ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.752+0000 7fa081000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121562e80 0x55f121575660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41602/0 (socket says 172.21.15.47:41602) 2024-03-21T11:25:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.752+0000 7fa081000640 1 -- 172.21.15.47:0/3060146742 learned_addr learned my addr 172.21.15.47:0/3060146742 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.752+0000 7fa081000640 1 --2- 172.21.15.47:0/3060146742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f1215689e0 0x55f12157cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.752+0000 7fa073e00640 1 -- 172.21.15.47:0/3060146742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f1215689e0 msgr2=0x55f12157cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.753+0000 7fa073e00640 1 --2- 172.21.15.47:0/3060146742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f1215689e0 0x55f12157cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.753+0000 7fa073e00640 1 -- 172.21.15.47:0/3060146742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121562e80 msgr2=0x55f121575660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.753+0000 7fa073e00640 1 --2- 172.21.15.47:0/3060146742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121562e80 0x55f121575660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.753+0000 7fa073e00640 1 -- 172.21.15.47:0/3060146742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1212a67d0 con 0x55f121563870 2024-03-21T11:25:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.753+0000 7fa073e00640 1 --2- 172.21.15.47:0/3060146742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f121575ba0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa0640126d0 tx=0x7fa064012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfd23fa2095f446 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.753+0000 7fa06a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:25:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.753+0000 7fa06a000640 1 -- 172.21.15.47:0/3060146742 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064004320 con 0x55f121563870 2024-03-21T11:25:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.753+0000 7fa06a000640 1 -- 172.21.15.47:0/3060146742 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa064011940 con 0x55f121563870 2024-03-21T11:25:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.753+0000 7fa085d90740 1 -- 172.21.15.47:0/3060146742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa074011de0 con 0x55f121563870 2024-03-21T11:25:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.753+0000 7fa06a000640 1 -- 172.21.15.47:0/3060146742 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0640044c0 con 0x55f121563870 2024-03-21T11:25:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.753+0000 7fa085d90740 10 client.15849 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.754+0000 7fa085d90740 1 -- 172.21.15.47:0/3060146742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f1211dc3f0 con 0x55f121563870 2024-03-21T11:25:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.754+0000 7fa06a000640 1 -- 172.21.15.47:0/3060146742 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa074011de0 con 0x55f121563870 2024-03-21T11:25:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.754+0000 7fa06a000640 1 -- 172.21.15.47:0/3060146742 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa06400f440 con 0x55f121563870 2024-03-21T11:25:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.754+0000 7fa085d90740 10 client.15849 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.754+0000 7fa085d90740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.754+0000 7fa085d90740 20 client.15849 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.754+0000 7fa085d90740 1 --2- 172.21.15.47:0/3060146742 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f1215814f0 0x55f121583950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.754+0000 7fa085d90740 4 client.15849 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:25:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.754+0000 7fa085d90740 1 -- 172.21.15.47:0/3060146742 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1214d5790 con 0x55f1215814f0 2024-03-21T11:25:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.754+0000 7fa081000640 1 --2- 172.21.15.47:0/3060146742 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f1215814f0 0x55f121583950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.755+0000 7fa081000640 1 --2- 172.21.15.47:0/3060146742 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f1215814f0 0x55f121583950 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed2b47e434503951 server_cookie=229005403051eb40 in_seq=0 out_seq=0 2024-03-21T11:25:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.755+0000 7fa06a000640 10 client.15849 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.755+0000 7fa06a000640 1 -- 172.21.15.47:0/3060146742 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f1214d5790 con 0x55f1215814f0 2024-03-21T11:25:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.755+0000 7fa06a000640 10 client.15849 handle_command_reply: tid=0 2024-03-21T11:25:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.756+0000 7fa085d90740 1 client.15849 shutdown 2024-03-21T11:25:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.756+0000 7fa085d90740 1 -- 172.21.15.47:0/3060146742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 msgr2=0x55f121575ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.756+0000 7fa085d90740 1 --2- 172.21.15.47:0/3060146742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f121575ba0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa0640126d0 tx=0x7fa064012b90 comp rx=0 tx=0).stop 2024-03-21T11:25:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.756+0000 7fa085d90740 1 -- 172.21.15.47:0/3060146742 shutdown_connections 2024-03-21T11:25:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.756+0000 7fa085d90740 1 --2- 172.21.15.47:0/3060146742 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f1215814f0 0x55f121583950 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.756+0000 7fa085d90740 1 --2- 172.21.15.47:0/3060146742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f1215689e0 0x55f12157cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.756+0000 7fa085d90740 1 --2- 172.21.15.47:0/3060146742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f121575ba0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.756+0000 7fa085d90740 1 --2- 172.21.15.47:0/3060146742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121562e80 0x55f121575660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.756+0000 7fa06a000640 0 client.15849 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.756+0000 7fa085d90740 1 -- 172.21.15.47:0/3060146742 >> 172.21.15.47:0/3060146742 conn(0x55f12152b200 msgr2=0x55f12155da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.756+0000 7fa085d90740 1 -- 172.21.15.47:0/3060146742 shutdown_connections 2024-03-21T11:25:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.756+0000 7fa085d90740 1 -- 172.21.15.47:0/3060146742 wait complete. 2024-03-21T11:25:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.756+0000 7fa085d90740 20 client.15849 trim_cache size 0 max 16384 2024-03-21T11:25:27.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.774+0000 7fa085d90740 1 Processor -- start 2024-03-21T11:25:27.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.774+0000 7fa085d90740 1 -- start start 2024-03-21T11:25:27.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa085d90740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f121562e80 0x55f121567e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa085d90740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121563870 0x55f12157cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa085d90740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1215689e0 0x55f12155d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa085d90740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c1a7ef0 con 0x55f1215689e0 2024-03-21T11:25:27.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa085d90740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c1a9870 con 0x55f121563870 2024-03-21T11:25:27.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa085d90740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c1a8550 con 0x55f121562e80 2024-03-21T11:25:27.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa073e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121563870 0x55f12157cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa081000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f121562e80 0x55f121567e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa073e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121563870 0x55f12157cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41616/0 (socket says 172.21.15.47:41616) 2024-03-21T11:25:27.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa073e00640 1 -- 172.21.15.47:0/948716569 learned_addr learned my addr 172.21.15.47:0/948716569 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:27.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa081000640 1 --2- 172.21.15.47:0/948716569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1215689e0 0x55f12155d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa073e00640 1 -- 172.21.15.47:0/948716569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f121562e80 msgr2=0x55f121567e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:27.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa073e00640 1 --2- 172.21.15.47:0/948716569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f121562e80 0x55f121567e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa073e00640 1 -- 172.21.15.47:0/948716569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1215689e0 msgr2=0x55f12155d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:27.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa073e00640 1 --2- 172.21.15.47:0/948716569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1215689e0 0x55f12155d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa073e00640 1 -- 172.21.15.47:0/948716569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f12119c0f0 con 0x55f121563870 2024-03-21T11:25:27.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.775+0000 7fa081000640 1 --2- 172.21.15.47:0/948716569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1215689e0 0x55f12155d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:27.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.776+0000 7fa073e00640 1 --2- 172.21.15.47:0/948716569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121563870 0x55f12157cba0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa0640092e0 tx=0x7fa064009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bff4af42b26cc475 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:27.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.776+0000 7fa070c00640 1 -- 172.21.15.47:0/948716569 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06402e020 con 0x55f121563870 2024-03-21T11:25:27.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.776+0000 7fa070c00640 1 -- 172.21.15.47:0/948716569 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa064009680 con 0x55f121563870 2024-03-21T11:25:27.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.776+0000 7fa070c00640 1 -- 172.21.15.47:0/948716569 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064011810 con 0x55f121563870 2024-03-21T11:25:27.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.777+0000 7fa085d90740 1 -- 172.21.15.47:0/948716569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121563870 msgr2=0x55f12157cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:27.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.777+0000 7fa085d90740 1 --2- 172.21.15.47:0/948716569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121563870 0x55f12157cba0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa0640092e0 tx=0x7fa064009e50 comp rx=0 tx=0).stop 2024-03-21T11:25:27.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.777+0000 7fa085d90740 1 -- 172.21.15.47:0/948716569 shutdown_connections 2024-03-21T11:25:27.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.777+0000 7fa085d90740 1 --2- 172.21.15.47:0/948716569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1215689e0 0x55f12155d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.777+0000 7fa085d90740 1 --2- 172.21.15.47:0/948716569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121563870 0x55f12157cba0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.777+0000 7fa085d90740 1 --2- 172.21.15.47:0/948716569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f121562e80 0x55f121567e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.777+0000 7fa085d90740 1 -- 172.21.15.47:0/948716569 >> 172.21.15.47:0/948716569 conn(0x55f12152b200 msgr2=0x55f121564ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:27.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.777+0000 7fa085d90740 1 -- 172.21.15.47:0/948716569 shutdown_connections 2024-03-21T11:25:27.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.777+0000 7fa085d90740 1 -- 172.21.15.47:0/948716569 wait complete. 2024-03-21T11:25:27.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.778+0000 7fa085d90740 1 -- start start 2024-03-21T11:25:27.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.778+0000 7fa085d90740 1 Processor -- start 2024-03-21T11:25:27.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.778+0000 7fa085d90740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121562e80 0x55f12157c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.778+0000 7fa085d90740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f12157af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.778+0000 7fa085d90740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f1215689e0 0x55f12157b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.778+0000 7fa085d90740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c1a9210 con 0x55f121563870 2024-03-21T11:25:27.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.778+0000 7fa085d90740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c1a8bb0 con 0x55f121562e80 2024-03-21T11:25:27.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.778+0000 7fa085d90740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c1a60b0 con 0x55f1215689e0 2024-03-21T11:25:27.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.778+0000 7fa085d90740 10 client.0 _ll_register_callbacks cb 0x7fa064009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:27.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.779+0000 7fa081000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121562e80 0x55f12157c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.779+0000 7fa081000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121562e80 0x55f12157c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41628/0 (socket says 172.21.15.47:41628) 2024-03-21T11:25:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.779+0000 7fa073e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f12157af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.779+0000 7fa081000640 1 -- 172.21.15.47:0/504569011 learned_addr learned my addr 172.21.15.47:0/504569011 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.779+0000 7fa081000640 1 --2- 172.21.15.47:0/504569011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f1215689e0 0x55f12157b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.779+0000 7fa081000640 1 -- 172.21.15.47:0/504569011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f1215689e0 msgr2=0x55f12157b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.779+0000 7fa081000640 1 --2- 172.21.15.47:0/504569011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f1215689e0 0x55f12157b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.779+0000 7fa081000640 1 -- 172.21.15.47:0/504569011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 msgr2=0x55f12157af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.779+0000 7fa081000640 1 --2- 172.21.15.47:0/504569011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f12157af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.779+0000 7fa081000640 1 -- 172.21.15.47:0/504569011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f12157baa0 con 0x55f121562e80 2024-03-21T11:25:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.779+0000 7fa073e00640 1 --2- 172.21.15.47:0/504569011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f12157af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.779+0000 7fa081000640 1 --2- 172.21.15.47:0/504569011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121562e80 0x55f12157c910 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa074011010 tx=0x7fa07401d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c3d3f8121340ef7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.779+0000 7fa06a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:25:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.780+0000 7fa06a000640 1 -- 172.21.15.47:0/504569011 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07402f0d0 con 0x55f121562e80 2024-03-21T11:25:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.780+0000 7fa085d90740 1 -- 172.21.15.47:0/504569011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa064009680 con 0x55f121562e80 2024-03-21T11:25:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.780+0000 7fa06a000640 1 -- 172.21.15.47:0/504569011 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa07402f270 con 0x55f121562e80 2024-03-21T11:25:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.780+0000 7fa06a000640 1 -- 172.21.15.47:0/504569011 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa07402f840 con 0x55f121562e80 2024-03-21T11:25:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.780+0000 7fa085d90740 10 client.26042 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.780+0000 7fa085d90740 1 -- 172.21.15.47:0/504569011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f1215ddd50 con 0x55f121562e80 2024-03-21T11:25:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.780+0000 7fa06a000640 1 -- 172.21.15.47:0/504569011 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa064009680 con 0x55f121562e80 2024-03-21T11:25:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.780+0000 7fa06a000640 1 -- 172.21.15.47:0/504569011 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa074029040 con 0x55f121562e80 2024-03-21T11:25:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.780+0000 7fa085d90740 10 client.26042 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.780+0000 7fa085d90740 10 client.26042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:27.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.781+0000 7fa085d90740 20 client.26042 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:27.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.781+0000 7fa085d90740 1 --2- 172.21.15.47:0/504569011 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f1215de570 0x55f1215e0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:27.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.781+0000 7fa085d90740 4 client.26042 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:25:27.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.781+0000 7fa085d90740 1 -- 172.21.15.47:0/504569011 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f1214cfae0 con 0x55f1215de570 2024-03-21T11:25:27.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.781+0000 7fa073e00640 1 --2- 172.21.15.47:0/504569011 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f1215de570 0x55f1215e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.781+0000 7fa073e00640 1 --2- 172.21.15.47:0/504569011 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f1215de570 0x55f1215e0990 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75c247f0f4fadfb3 server_cookie=8b30bc992c9b298b in_seq=0 out_seq=0 2024-03-21T11:25:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.781+0000 7fa06a000640 10 client.26042 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa06a000640 1 -- 172.21.15.47:0/504569011 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f1214cfae0 con 0x55f1215de570 2024-03-21T11:25:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa06a000640 10 client.26042 handle_command_reply: tid=0 2024-03-21T11:25:27.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa085d90740 1 client.26042 shutdown 2024-03-21T11:25:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa085d90740 1 -- 172.21.15.47:0/504569011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121562e80 msgr2=0x55f12157c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa085d90740 1 --2- 172.21.15.47:0/504569011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121562e80 0x55f12157c910 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa074011010 tx=0x7fa07401d040 comp rx=0 tx=0).stop 2024-03-21T11:25:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa085d90740 1 -- 172.21.15.47:0/504569011 shutdown_connections 2024-03-21T11:25:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa085d90740 1 --2- 172.21.15.47:0/504569011 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f1215de570 0x55f1215e0990 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa085d90740 1 --2- 172.21.15.47:0/504569011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f1215689e0 0x55f12157b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa085d90740 1 --2- 172.21.15.47:0/504569011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f121563870 0x55f12157af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa085d90740 1 --2- 172.21.15.47:0/504569011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f121562e80 0x55f12157c910 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa06a000640 0 client.26042 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa085d90740 1 -- 172.21.15.47:0/504569011 >> 172.21.15.47:0/504569011 conn(0x55f12152b200 msgr2=0x55f121561a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa085d90740 1 -- 172.21.15.47:0/504569011 shutdown_connections 2024-03-21T11:25:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa085d90740 1 -- 172.21.15.47:0/504569011 wait complete. 2024-03-21T11:25:27.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.782+0000 7fa085d90740 20 client.26042 trim_cache size 0 max 16384 2024-03-21T11:25:27.788 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:25:27.788 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:25:27.788 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:25:27.788 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:25:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.783+0000 7fa083000640 1 -- 172.21.15.47:0/2952348993 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa0600a2f20 msgr2=0x7fa0600a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.783+0000 7fa083000640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa0600a2f20 0x7fa0600a53e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0c5c40 tx=0x7fa06c02bc50 comp rx=0 tx=0).stop 2024-03-21T11:25:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.783+0000 7fa083000640 1 -- 172.21.15.47:0/2952348993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa07c1038e0 msgr2=0x7fa07c199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.783+0000 7fa083000640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa07c1038e0 0x7fa07c199c70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa06c02c260 tx=0x7fa06c02c670 comp rx=0 tx=0).stop 2024-03-21T11:25:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.783+0000 7fa083000640 1 -- 172.21.15.47:0/2952348993 shutdown_connections 2024-03-21T11:25:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.783+0000 7fa083000640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa0600a2f20 0x7fa0600a53e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.783+0000 7fa083000640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa07c105490 0x7fa07c1a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.783+0000 7fa083000640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa07c104ad0 0x7fa07c19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.783+0000 7fa083000640 1 --2- 172.21.15.47:0/2952348993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa07c1038e0 0x7fa07c199c70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.783+0000 7fa083000640 1 -- 172.21.15.47:0/2952348993 >> 172.21.15.47:0/2952348993 conn(0x7fa07c0fee50 msgr2=0x7fa07c107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.783+0000 7fa083000640 1 -- 172.21.15.47:0/2952348993 shutdown_connections 2024-03-21T11:25:27.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:27.783+0000 7fa083000640 1 -- 172.21.15.47:0/2952348993 wait complete. 2024-03-21T11:25:27.804 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:25:27.804 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:25:28.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.035+0000 7fcd0c600640 1 -- 172.21.15.47:0/3731675277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd04100910 msgr2=0x7fcd04100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.035+0000 7fcd0c600640 1 --2- 172.21.15.47:0/3731675277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd04100910 0x7fcd04100d70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009900 tx=0x7fccf002bc30 comp rx=0 tx=0).stop 2024-03-21T11:25:28.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.035+0000 7fcd0c600640 1 -- 172.21.15.47:0/3731675277 shutdown_connections 2024-03-21T11:25:28.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.035+0000 7fcd0c600640 1 --2- 172.21.15.47:0/3731675277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd041012b0 0x7fcd04101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.035+0000 7fcd0c600640 1 --2- 172.21.15.47:0/3731675277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd04100910 0x7fcd04100d70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.035+0000 7fcd0c600640 1 --2- 172.21.15.47:0/3731675277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd040fffb0 0x7fcd041003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.035+0000 7fcd0c600640 1 -- 172.21.15.47:0/3731675277 >> 172.21.15.47:0/3731675277 conn(0x7fcd040fbb50 msgr2=0x7fcd040fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:28.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.035+0000 7fcd0c600640 1 -- 172.21.15.47:0/3731675277 shutdown_connections 2024-03-21T11:25:28.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.035+0000 7fcd0c600640 1 -- 172.21.15.47:0/3731675277 wait complete. 2024-03-21T11:25:28.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.037+0000 7fcd0c600640 1 Processor -- start 2024-03-21T11:25:28.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.037+0000 7fcd0c600640 1 -- start start 2024-03-21T11:25:28.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.037+0000 7fcd0c600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd040fffb0 0x7fcd04199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.037+0000 7fcd0c600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd04100910 0x7fcd0419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.037+0000 7fcd0c600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd041012b0 0x7fcd041a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.037+0000 7fcd0c600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0410f570 con 0x7fcd041012b0 2024-03-21T11:25:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.037+0000 7fcd0c600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0410f3f0 con 0x7fcd040fffb0 2024-03-21T11:25:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.037+0000 7fcd0c600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0410f6f0 con 0x7fcd04100910 2024-03-21T11:25:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.037+0000 7fcd09200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd04100910 0x7fcd0419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.037+0000 7fcd09c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd040fffb0 0x7fcd04199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.037+0000 7fcd09200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd04100910 0x7fcd0419a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52606/0 (socket says 172.21.15.47:52606) 2024-03-21T11:25:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.037+0000 7fcd09200640 1 -- 172.21.15.47:0/963729770 learned_addr learned my addr 172.21.15.47:0/963729770 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.037+0000 7fcd0a600640 1 --2- 172.21.15.47:0/963729770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd041012b0 0x7fcd041a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.038+0000 7fcd09200640 1 -- 172.21.15.47:0/963729770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd040fffb0 msgr2=0x7fcd04199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.038+0000 7fcd09200640 1 --2- 172.21.15.47:0/963729770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd040fffb0 0x7fcd04199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.038+0000 7fcd09200640 1 -- 172.21.15.47:0/963729770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd041012b0 msgr2=0x7fcd041a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.038+0000 7fcd09200640 1 --2- 172.21.15.47:0/963729770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd041012b0 0x7fcd041a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.038+0000 7fcd09200640 1 -- 172.21.15.47:0/963729770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd041a1800 con 0x7fcd04100910 2024-03-21T11:25:28.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.038+0000 7fcd09200640 1 --2- 172.21.15.47:0/963729770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd04100910 0x7fcd0419a120 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fccf0000c00 tx=0x7fccf002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:28.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.038+0000 7fcd02000640 1 -- 172.21.15.47:0/963729770 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0043020 con 0x7fcd04100910 2024-03-21T11:25:28.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.038+0000 7fcd0c600640 1 -- 172.21.15.47:0/963729770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd041a1a90 con 0x7fcd04100910 2024-03-21T11:25:28.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.038+0000 7fcd02000640 1 -- 172.21.15.47:0/963729770 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccf00054e0 con 0x7fcd04100910 2024-03-21T11:25:28.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.038+0000 7fcd02000640 1 -- 172.21.15.47:0/963729770 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0003c40 con 0x7fcd04100910 2024-03-21T11:25:28.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.038+0000 7fcd0c600640 1 -- 172.21.15.47:0/963729770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd041a1fe0 con 0x7fcd04100910 2024-03-21T11:25:28.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.040+0000 7fcd0c600640 1 -- 172.21.15.47:0/963729770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0410ae90 con 0x7fcd04100910 2024-03-21T11:25:28.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.041+0000 7fcd02000640 1 -- 172.21.15.47:0/963729770 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fccf0046020 con 0x7fcd04100910 2024-03-21T11:25:28.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.043+0000 7fcd02000640 1 --2- 172.21.15.47:0/963729770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fccec0a2f20 0x7fccec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.043+0000 7fcd02000640 1 -- 172.21.15.47:0/963729770 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fccf00f18b0 con 0x7fcd04100910 2024-03-21T11:25:28.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.043+0000 7fcd09c00640 1 --2- 172.21.15.47:0/963729770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fccec0a2f20 0x7fccec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.044+0000 7fcd09c00640 1 --2- 172.21.15.47:0/963729770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fccec0a2f20 0x7fccec0a53e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009bd0 tx=0x7fccf8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:28.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.048+0000 7fcd02000640 1 -- 172.21.15.47:0/963729770 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fccf004d020 con 0x7fcd04100910 2024-03-21T11:25:28.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.301+0000 7fcd0c600640 1 -- 172.21.15.47:0/963729770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd0402d070 con 0x7fcd04100910 2024-03-21T11:25:28.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.303+0000 7fcd02000640 1 -- 172.21.15.47:0/963729770 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fccf00be2f0 con 0x7fcd04100910 2024-03-21T11:25:28.304 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:25:28.304 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:25:28.304 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:25:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.306+0000 7fcd0c600640 1 -- 172.21.15.47:0/963729770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fccec0a2f20 msgr2=0x7fccec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.306+0000 7fcd0c600640 1 --2- 172.21.15.47:0/963729770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fccec0a2f20 0x7fccec0a53e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009bd0 tx=0x7fccf8005f90 comp rx=0 tx=0).stop 2024-03-21T11:25:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.306+0000 7fcd0c600640 1 -- 172.21.15.47:0/963729770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd04100910 msgr2=0x7fcd0419a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.306+0000 7fcd0c600640 1 --2- 172.21.15.47:0/963729770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd04100910 0x7fcd0419a120 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fccf0000c00 tx=0x7fccf002c650 comp rx=0 tx=0).stop 2024-03-21T11:25:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.306+0000 7fcd0c600640 1 -- 172.21.15.47:0/963729770 shutdown_connections 2024-03-21T11:25:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.306+0000 7fcd0c600640 1 --2- 172.21.15.47:0/963729770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fccec0a2f20 0x7fccec0a53e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.306+0000 7fcd0c600640 1 --2- 172.21.15.47:0/963729770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd041012b0 0x7fcd041a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.306+0000 7fcd0c600640 1 --2- 172.21.15.47:0/963729770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd04100910 0x7fcd0419a120 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.306+0000 7fcd0c600640 1 --2- 172.21.15.47:0/963729770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd040fffb0 0x7fcd04199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.306+0000 7fcd0c600640 1 -- 172.21.15.47:0/963729770 >> 172.21.15.47:0/963729770 conn(0x7fcd040fbb50 msgr2=0x7fcd040fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.306+0000 7fcd0c600640 1 -- 172.21.15.47:0/963729770 shutdown_connections 2024-03-21T11:25:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.306+0000 7fcd0c600640 1 -- 172.21.15.47:0/963729770 wait complete. 2024-03-21T11:25:28.327 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:25:28.327 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:25:28.486 INFO:tasks.workunit.client.0.smithi047.stdout: 1 21795 11.31 MB/sec warmup 12 sec latency 9.289 ms 2024-03-21T11:25:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.556+0000 7fd1db200640 1 -- 172.21.15.47:0/963609856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1dc108db0 msgr2=0x7fd1dc109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.556+0000 7fd1db200640 1 --2- 172.21.15.47:0/963609856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1dc108db0 0x7fd1dc109230 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009900 tx=0x7fd1c402bc30 comp rx=0 tx=0).stop 2024-03-21T11:25:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.556+0000 7fd1db200640 1 -- 172.21.15.47:0/963609856 shutdown_connections 2024-03-21T11:25:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.556+0000 7fd1db200640 1 --2- 172.21.15.47:0/963609856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1dc109770 0x7fd1dc110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.556+0000 7fd1db200640 1 --2- 172.21.15.47:0/963609856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1dc108db0 0x7fd1dc109230 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.556+0000 7fd1db200640 1 --2- 172.21.15.47:0/963609856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1dc107bc0 0x7fd1dc107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.556+0000 7fd1db200640 1 -- 172.21.15.47:0/963609856 >> 172.21.15.47:0/963609856 conn(0x7fd1dc076b70 msgr2=0x7fd1dc078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:28.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.557+0000 7fd1db200640 1 -- 172.21.15.47:0/963609856 shutdown_connections 2024-03-21T11:25:28.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.557+0000 7fd1db200640 1 -- 172.21.15.47:0/963609856 wait complete. 2024-03-21T11:25:28.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.558+0000 7fd1db200640 1 Processor -- start 2024-03-21T11:25:28.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.558+0000 7fd1db200640 1 -- start start 2024-03-21T11:25:28.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.558+0000 7fd1db200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1dc107bc0 0x7fd1dc19dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.558+0000 7fd1db200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1dc108db0 0x7fd1dc19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.558+0000 7fd1db200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1dc109770 0x7fd1dc1a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.558+0000 7fd1db200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc113890 con 0x7fd1dc107bc0 2024-03-21T11:25:28.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.558+0000 7fd1db200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc113710 con 0x7fd1dc108db0 2024-03-21T11:25:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.558+0000 7fd1db200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc113a10 con 0x7fd1dc109770 2024-03-21T11:25:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.559+0000 7fd1d1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1dc108db0 0x7fd1dc19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.559+0000 7fd1da800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1dc109770 0x7fd1dc1a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.559+0000 7fd1d1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1dc108db0 0x7fd1dc19e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41664/0 (socket says 172.21.15.47:41664) 2024-03-21T11:25:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.559+0000 7fd1d1400640 1 -- 172.21.15.47:0/385597102 learned_addr learned my addr 172.21.15.47:0/385597102 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.559+0000 7fd1d9e00640 1 --2- 172.21.15.47:0/385597102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1dc107bc0 0x7fd1dc19dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.559+0000 7fd1da800640 1 -- 172.21.15.47:0/385597102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1dc108db0 msgr2=0x7fd1dc19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.559+0000 7fd1da800640 1 --2- 172.21.15.47:0/385597102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1dc108db0 0x7fd1dc19e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.559+0000 7fd1da800640 1 -- 172.21.15.47:0/385597102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1dc107bc0 msgr2=0x7fd1dc19dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.559+0000 7fd1da800640 1 --2- 172.21.15.47:0/385597102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1dc107bc0 0x7fd1dc19dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.559+0000 7fd1da800640 1 -- 172.21.15.47:0/385597102 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc1a5ac0 con 0x7fd1dc109770 2024-03-21T11:25:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.559+0000 7fd1d1400640 1 --2- 172.21.15.47:0/385597102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1dc108db0 0x7fd1dc19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.559+0000 7fd1da800640 1 --2- 172.21.15.47:0/385597102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1dc109770 0x7fd1dc1a5450 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009fd0 tx=0x7fd1d400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.560+0000 7fd1d3400640 1 -- 172.21.15.47:0/385597102 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400ce20 con 0x7fd1dc109770 2024-03-21T11:25:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.560+0000 7fd1d3400640 1 -- 172.21.15.47:0/385597102 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1d4005020 con 0x7fd1dc109770 2024-03-21T11:25:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.560+0000 7fd1d3400640 1 -- 172.21.15.47:0/385597102 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4010720 con 0x7fd1dc109770 2024-03-21T11:25:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.560+0000 7fd1db200640 1 -- 172.21.15.47:0/385597102 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc1a5db0 con 0x7fd1dc109770 2024-03-21T11:25:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.560+0000 7fd1db200640 1 -- 172.21.15.47:0/385597102 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1dc1a6220 con 0x7fd1dc109770 2024-03-21T11:25:28.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:28 smithi047 ceph-mon[31351]: pgmap v464: 129 pgs: 129 active+clean; 8.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 839 KiB/s wr, 243 op/s 2024-03-21T11:25:28.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.563+0000 7fd1e3e57740 1 Processor -- start 2024-03-21T11:25:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.564+0000 7fd1d3400640 1 -- 172.21.15.47:0/385597102 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd1d4005540 con 0x7fd1dc109770 2024-03-21T11:25:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.564+0000 7fd1e3e57740 1 -- start start 2024-03-21T11:25:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.564+0000 7fd1e3e57740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4a9f0 0x563f76e4ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.564+0000 7fd1e3e57740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4bbc0 0x563f76e4c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.564+0000 7fd1e3e57740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4c580 0x563f76e52e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.564+0000 7fd1e3e57740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f76daf2c0 con 0x563f76e4a9f0 2024-03-21T11:25:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.564+0000 7fd1e3e57740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f76d57c90 con 0x563f76e4c580 2024-03-21T11:25:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.564+0000 7fd1e3e57740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f76dab360 con 0x563f76e4bbc0 2024-03-21T11:25:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.564+0000 7fd1d9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4bbc0 0x563f76e4c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.565+0000 7fd1d9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4bbc0 0x563f76e4c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52648/0 (socket says 172.21.15.47:52648) 2024-03-21T11:25:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.565+0000 7fd1d1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4a9f0 0x563f76e4ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.565+0000 7fd1d9e00640 1 -- 172.21.15.47:0/3860094338 learned_addr learned my addr 172.21.15.47:0/3860094338 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.565+0000 7fd1d1400640 1 --2- 172.21.15.47:0/3860094338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4c580 0x563f76e52e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.565+0000 7fd1d9e00640 1 -- 172.21.15.47:0/3860094338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4c580 msgr2=0x563f76e52e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.565+0000 7fd1d9e00640 1 --2- 172.21.15.47:0/3860094338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4c580 0x563f76e52e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.565+0000 7fd1d9e00640 1 -- 172.21.15.47:0/3860094338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4a9f0 msgr2=0x563f76e4ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.565+0000 7fd1d9e00640 1 --2- 172.21.15.47:0/3860094338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4a9f0 0x563f76e4ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.565+0000 7fd1d9e00640 1 -- 172.21.15.47:0/3860094338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f76d7c080 con 0x563f76e4bbc0 2024-03-21T11:25:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.565+0000 7fd1d1400640 1 --2- 172.21.15.47:0/3860094338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4c580 0x563f76e52e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.565+0000 7fd1d3400640 1 --2- 172.21.15.47:0/385597102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1c00a2fa0 0x7fd1c00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.565+0000 7fd1d9e00640 1 --2- 172.21.15.47:0/3860094338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4bbc0 0x563f76e4c040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009ca0 tx=0x7fd1cc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d85db763fff29d55 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.566+0000 7fd1d3400640 1 -- 172.21.15.47:0/385597102 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd1d40cfe30 con 0x7fd1dc109770 2024-03-21T11:25:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.566+0000 7fd1d1400640 1 --2- 172.21.15.47:0/385597102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1c00a2fa0 0x7fd1c00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.566+0000 7fd1cb400640 1 -- 172.21.15.47:0/3860094338 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc004220 con 0x563f76e4bbc0 2024-03-21T11:25:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.566+0000 7fd1cb400640 1 -- 172.21.15.47:0/3860094338 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1cc0043c0 con 0x563f76e4bbc0 2024-03-21T11:25:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.566+0000 7fd1cb400640 1 -- 172.21.15.47:0/3860094338 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc019970 con 0x563f76e4bbc0 2024-03-21T11:25:28.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.567+0000 7fd1e3e57740 1 -- 172.21.15.47:0/3860094338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4bbc0 msgr2=0x563f76e4c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.567+0000 7fd1d1400640 1 --2- 172.21.15.47:0/385597102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1c00a2fa0 0x7fd1c00a5460 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd1c402c8e0 tx=0x7fd1c4005b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.567+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/3860094338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4bbc0 0x563f76e4c040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009ca0 tx=0x7fd1cc011810 comp rx=0 tx=0).stop 2024-03-21T11:25:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.567+0000 7fd1e3e57740 1 -- 172.21.15.47:0/3860094338 shutdown_connections 2024-03-21T11:25:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.567+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/3860094338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4c580 0x563f76e52e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.567+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/3860094338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4bbc0 0x563f76e4c040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.567+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/3860094338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4a9f0 0x563f76e4ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.567+0000 7fd1e3e57740 1 -- 172.21.15.47:0/3860094338 >> 172.21.15.47:0/3860094338 conn(0x563f76e45de0 msgr2=0x563f76e461e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.567+0000 7fd1e3e57740 1 -- 172.21.15.47:0/3860094338 shutdown_connections 2024-03-21T11:25:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.567+0000 7fd1e3e57740 1 -- 172.21.15.47:0/3860094338 wait complete. 2024-03-21T11:25:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.567+0000 7fd1e3e57740 1 -- start start 2024-03-21T11:25:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.568+0000 7fd1e3e57740 1 Processor -- start 2024-03-21T11:25:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.568+0000 7fd1e3e57740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4a9f0 0x563f76e54c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.568+0000 7fd1e3e57740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4bbc0 0x563f76e55150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.568+0000 7fd1e3e57740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4c580 0x563f76e5c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.568+0000 7fd1e3e57740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f76e53590 con 0x563f76e4bbc0 2024-03-21T11:25:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.568+0000 7fd1e3e57740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f76d6b0d0 con 0x563f76e4a9f0 2024-03-21T11:25:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.568+0000 7fd1e3e57740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f76e53710 con 0x563f76e4c580 2024-03-21T11:25:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.568+0000 7fd1d9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4a9f0 0x563f76e54c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.569+0000 7fd1d9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4a9f0 0x563f76e54c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41678/0 (socket says 172.21.15.47:41678) 2024-03-21T11:25:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.569+0000 7fd1d9e00640 1 -- 172.21.15.47:0/4081184508 learned_addr learned my addr 172.21.15.47:0/4081184508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.569+0000 7fd1da800640 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4c580 0x563f76e5c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.569+0000 7fd1e3e57740 10 client.0 _ll_register_callbacks cb 0x563f76d6fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.569+0000 7fd1d1400640 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4bbc0 0x563f76e55150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.569+0000 7fd1d9e00640 1 -- 172.21.15.47:0/4081184508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4c580 msgr2=0x563f76e5c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.569+0000 7fd1d9e00640 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4c580 0x563f76e5c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.569+0000 7fd1d9e00640 1 -- 172.21.15.47:0/4081184508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4bbc0 msgr2=0x563f76e55150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.569+0000 7fd1d9e00640 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4bbc0 0x563f76e55150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.569+0000 7fd1d9e00640 1 -- 172.21.15.47:0/4081184508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f76ac33f0 con 0x563f76e4a9f0 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.569+0000 7fd1da800640 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4c580 0x563f76e5c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.569+0000 7fd1d1400640 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4bbc0 0x563f76e55150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.569+0000 7fd1d9e00640 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4a9f0 0x563f76e54c10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009c70 tx=0x7fd1cc009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f3cca5b0320672e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.569+0000 7fd1bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.570+0000 7fd1bfe00640 1 -- 172.21.15.47:0/4081184508 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc028070 con 0x563f76e4a9f0 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.570+0000 7fd1bfe00640 1 -- 172.21.15.47:0/4081184508 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd1cc003800 con 0x563f76e4a9f0 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.570+0000 7fd1e3e57740 1 -- 172.21.15.47:0/4081184508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1cc0043c0 con 0x563f76e4a9f0 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.570+0000 7fd1bfe00640 1 -- 172.21.15.47:0/4081184508 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1cc034050 con 0x563f76e4a9f0 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.570+0000 7fd1e3e57740 10 client.26060 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.570+0000 7fd1e3e57740 1 -- 172.21.15.47:0/4081184508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563f76b8d7d0 con 0x563f76e4a9f0 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.570+0000 7fd1bfe00640 1 -- 172.21.15.47:0/4081184508 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1cc0043c0 con 0x563f76e4a9f0 2024-03-21T11:25:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.570+0000 7fd1bfe00640 1 -- 172.21.15.47:0/4081184508 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd1cc004960 con 0x563f76e4a9f0 2024-03-21T11:25:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.571+0000 7fd1e3e57740 10 client.26060 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.571+0000 7fd1e3e57740 10 client.26060 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:25:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.571+0000 7fd1e3e57740 20 client.26060 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.571+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x563f76e603f0 0x563f76e62890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.571+0000 7fd1e3e57740 4 client.26060 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:25:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.571+0000 7fd1e3e57740 1 -- 172.21.15.47:0/4081184508 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f76dbc800 con 0x563f76e603f0 2024-03-21T11:25:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.571+0000 7fd1d1400640 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x563f76e603f0 0x563f76e62890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.572+0000 7fd1d1400640 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x563f76e603f0 0x563f76e62890 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a76ad68f10afeb41 server_cookie=7b024fa93d7089eb in_seq=0 out_seq=0 2024-03-21T11:25:28.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.572+0000 7fd1bfe00640 10 client.26060 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:25:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.572+0000 7fd1bfe00640 1 -- 172.21.15.47:0/4081184508 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563f76dbc800 con 0x563f76e603f0 2024-03-21T11:25:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.572+0000 7fd1bfe00640 10 client.26060 handle_command_reply: tid=0 2024-03-21T11:25:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.572+0000 7fd1e3e57740 1 client.26060 shutdown 2024-03-21T11:25:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.572+0000 7fd1e3e57740 1 -- 172.21.15.47:0/4081184508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4a9f0 msgr2=0x563f76e54c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.572+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4a9f0 0x563f76e54c10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009c70 tx=0x7fd1cc009f80 comp rx=0 tx=0).stop 2024-03-21T11:25:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.573+0000 7fd1e3e57740 1 -- 172.21.15.47:0/4081184508 shutdown_connections 2024-03-21T11:25:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.573+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x563f76e603f0 0x563f76e62890 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.573+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4c580 0x563f76e5c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.573+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4bbc0 0x563f76e55150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.573+0000 7fd1bfe00640 0 client.26060 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:25:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.573+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/4081184508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4a9f0 0x563f76e54c10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.573+0000 7fd1e3e57740 1 -- 172.21.15.47:0/4081184508 >> 172.21.15.47:0/4081184508 conn(0x563f76e45de0 msgr2=0x563f76e45160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.573+0000 7fd1e3e57740 1 -- 172.21.15.47:0/4081184508 shutdown_connections 2024-03-21T11:25:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.573+0000 7fd1e3e57740 1 -- 172.21.15.47:0/4081184508 wait complete. 2024-03-21T11:25:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.573+0000 7fd1e3e57740 20 client.26060 trim_cache size 0 max 16384 2024-03-21T11:25:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.587+0000 7fd1e3e57740 1 Processor -- start 2024-03-21T11:25:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.587+0000 7fd1e3e57740 1 -- start start 2024-03-21T11:25:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.587+0000 7fd1e3e57740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4a9f0 0x563f76e4edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.587+0000 7fd1e3e57740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4bbc0 0x563f76e5c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.587+0000 7fd1e3e57740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4c580 0x563f76e4f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.587+0000 7fd1e3e57740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc1b03d0 con 0x563f76e4c580 2024-03-21T11:25:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.587+0000 7fd1e3e57740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc1b1d50 con 0x563f76e4bbc0 2024-03-21T11:25:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.587+0000 7fd1e3e57740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc1b0a30 con 0x563f76e4a9f0 2024-03-21T11:25:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.588+0000 7fd1d1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4bbc0 0x563f76e5c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.588+0000 7fd1da800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4c580 0x563f76e4f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.588+0000 7fd1d9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4a9f0 0x563f76e4edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.588+0000 7fd1d1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4bbc0 0x563f76e5c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41688/0 (socket says 172.21.15.47:41688) 2024-03-21T11:25:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.588+0000 7fd1d1400640 1 -- 172.21.15.47:0/1513964103 learned_addr learned my addr 172.21.15.47:0/1513964103 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.588+0000 7fd1d1400640 1 -- 172.21.15.47:0/1513964103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4a9f0 msgr2=0x563f76e4edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.588+0000 7fd1d1400640 1 --2- 172.21.15.47:0/1513964103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4a9f0 0x563f76e4edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.588+0000 7fd1d1400640 1 -- 172.21.15.47:0/1513964103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4c580 msgr2=0x563f76e4f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.588+0000 7fd1d1400640 1 --2- 172.21.15.47:0/1513964103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4c580 0x563f76e4f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.588+0000 7fd1d1400640 1 -- 172.21.15.47:0/1513964103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f76e53470 con 0x563f76e4bbc0 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.588+0000 7fd1d1400640 1 --2- 172.21.15.47:0/1513964103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4bbc0 0x563f76e5c150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd1c402c240 tx=0x7fd1c4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2b26b34547253b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.589+0000 7fd1caa00640 1 -- 172.21.15.47:0/1513964103 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4057230 con 0x563f76e4bbc0 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.589+0000 7fd1caa00640 1 -- 172.21.15.47:0/1513964103 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1c405d070 con 0x563f76e4bbc0 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.589+0000 7fd1caa00640 1 -- 172.21.15.47:0/1513964103 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40035f0 con 0x563f76e4bbc0 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.589+0000 7fd1e3e57740 1 -- 172.21.15.47:0/1513964103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4bbc0 msgr2=0x563f76e5c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.589+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/1513964103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4bbc0 0x563f76e5c150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd1c402c240 tx=0x7fd1c4009260 comp rx=0 tx=0).stop 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.589+0000 7fd1e3e57740 1 -- 172.21.15.47:0/1513964103 shutdown_connections 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.589+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/1513964103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4c580 0x563f76e4f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.589+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/1513964103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4bbc0 0x563f76e5c150 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.589+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/1513964103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4a9f0 0x563f76e4edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.589+0000 7fd1e3e57740 1 -- 172.21.15.47:0/1513964103 >> 172.21.15.47:0/1513964103 conn(0x563f76e45de0 msgr2=0x563f76e498d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.589+0000 7fd1e3e57740 1 -- 172.21.15.47:0/1513964103 shutdown_connections 2024-03-21T11:25:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.589+0000 7fd1e3e57740 1 -- 172.21.15.47:0/1513964103 wait complete. 2024-03-21T11:25:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.590+0000 7fd1e3e57740 1 -- start start 2024-03-21T11:25:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.590+0000 7fd1e3e57740 1 Processor -- start 2024-03-21T11:25:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.590+0000 7fd1e3e57740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4a9f0 0x563f76e597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.590+0000 7fd1e3e57740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4bbc0 0x563f76e59ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.590+0000 7fd1e3e57740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4c580 0x563f76e5acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.590+0000 7fd1e3e57740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc1b16f0 con 0x563f76e4bbc0 2024-03-21T11:25:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.590+0000 7fd1e3e57740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc1b1090 con 0x563f76e4c580 2024-03-21T11:25:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.590+0000 7fd1e3e57740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc1ae590 con 0x563f76e4a9f0 2024-03-21T11:25:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.590+0000 7fd1e3e57740 10 client.0 _ll_register_callbacks cb 0x7fd1cc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.591+0000 7fd1d9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4a9f0 0x563f76e597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.591+0000 7fd1d9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4a9f0 0x563f76e597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52666/0 (socket says 172.21.15.47:52666) 2024-03-21T11:25:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.591+0000 7fd1d9e00640 1 -- 172.21.15.47:0/3502477844 learned_addr learned my addr 172.21.15.47:0/3502477844 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.591+0000 7fd1da800640 1 --2- 172.21.15.47:0/3502477844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4c580 0x563f76e5acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.591+0000 7fd1d1400640 1 --2- 172.21.15.47:0/3502477844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4bbc0 0x563f76e59ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.591+0000 7fd1d9e00640 1 -- 172.21.15.47:0/3502477844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4c580 msgr2=0x563f76e5acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.591+0000 7fd1d9e00640 1 --2- 172.21.15.47:0/3502477844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4c580 0x563f76e5acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.591+0000 7fd1d9e00640 1 -- 172.21.15.47:0/3502477844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4bbc0 msgr2=0x563f76e59ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.591+0000 7fd1d9e00640 1 --2- 172.21.15.47:0/3502477844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4bbc0 0x563f76e59ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.591+0000 7fd1d9e00640 1 -- 172.21.15.47:0/3502477844 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f76e5b2e0 con 0x563f76e4a9f0 2024-03-21T11:25:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.591+0000 7fd1d9e00640 1 --2- 172.21.15.47:0/3502477844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4a9f0 0x563f76e597a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00a010 tx=0x7fd1cc005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3a0a1919f1ff498 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.591+0000 7fd1bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:25:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.591+0000 7fd1bfe00640 1 -- 172.21.15.47:0/3502477844 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc028040 con 0x563f76e4a9f0 2024-03-21T11:25:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.592+0000 7fd1e3e57740 1 -- 172.21.15.47:0/3502477844 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1c405d070 con 0x563f76e4a9f0 2024-03-21T11:25:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.592+0000 7fd1bfe00640 1 -- 172.21.15.47:0/3502477844 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd1cc003800 con 0x563f76e4a9f0 2024-03-21T11:25:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.592+0000 7fd1bfe00640 1 -- 172.21.15.47:0/3502477844 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1cc003e30 con 0x563f76e4a9f0 2024-03-21T11:25:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.592+0000 7fd1e3e57740 10 client.26071 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.592+0000 7fd1e3e57740 1 -- 172.21.15.47:0/3502477844 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563f76ebbd60 con 0x563f76e4a9f0 2024-03-21T11:25:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.592+0000 7fd1bfe00640 1 -- 172.21.15.47:0/3502477844 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1c405d070 con 0x563f76e4a9f0 2024-03-21T11:25:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.592+0000 7fd1bfe00640 1 -- 172.21.15.47:0/3502477844 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd1cc019b00 con 0x563f76e4a9f0 2024-03-21T11:25:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.592+0000 7fd1e3e57740 10 client.26071 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.592+0000 7fd1e3e57740 10 client.26071 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:25:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.592+0000 7fd1e3e57740 20 client.26071 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.592+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/3502477844 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x563f76ebc520 0x563f76ebe940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.593+0000 7fd1e3e57740 4 client.26071 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:25:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.593+0000 7fd1e3e57740 1 -- 172.21.15.47:0/3502477844 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f76db6b50 con 0x563f76ebc520 2024-03-21T11:25:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.593+0000 7fd1d1400640 1 --2- 172.21.15.47:0/3502477844 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x563f76ebc520 0x563f76ebe940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.593+0000 7fd1d1400640 1 --2- 172.21.15.47:0/3502477844 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x563f76ebc520 0x563f76ebe940 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c9aee731cce0882c server_cookie=4f956ae5f6e5f0b7 in_seq=0 out_seq=0 2024-03-21T11:25:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.593+0000 7fd1bfe00640 10 client.26071 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:25:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.593+0000 7fd1bfe00640 1 -- 172.21.15.47:0/3502477844 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f76db6b50 con 0x563f76ebc520 2024-03-21T11:25:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.593+0000 7fd1bfe00640 10 client.26071 handle_command_reply: tid=0 2024-03-21T11:25:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.593+0000 7fd1e3e57740 1 client.26071 shutdown 2024-03-21T11:25:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.594+0000 7fd1e3e57740 1 -- 172.21.15.47:0/3502477844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4a9f0 msgr2=0x563f76e597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.594+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/3502477844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4a9f0 0x563f76e597a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00a010 tx=0x7fd1cc005dc0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.594+0000 7fd1e3e57740 1 -- 172.21.15.47:0/3502477844 shutdown_connections 2024-03-21T11:25:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.594+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/3502477844 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x563f76ebc520 0x563f76ebe940 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.594+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/3502477844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f76e4c580 0x563f76e5acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.594+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/3502477844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f76e4bbc0 0x563f76e59ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.594+0000 7fd1e3e57740 1 --2- 172.21.15.47:0/3502477844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f76e4a9f0 0x563f76e597a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.594+0000 7fd1e3e57740 1 -- 172.21.15.47:0/3502477844 >> 172.21.15.47:0/3502477844 conn(0x563f76e45de0 msgr2=0x563f76e448d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.594+0000 7fd1bfe00640 0 client.26071 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:25:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.594+0000 7fd1e3e57740 1 -- 172.21.15.47:0/3502477844 shutdown_connections 2024-03-21T11:25:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.594+0000 7fd1e3e57740 1 -- 172.21.15.47:0/3502477844 wait complete. 2024-03-21T11:25:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.594+0000 7fd1e3e57740 20 client.26071 trim_cache size 0 max 16384 2024-03-21T11:25:28.599 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:25:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.595+0000 7fd1db200640 1 -- 172.21.15.47:0/385597102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1c00a2fa0 msgr2=0x7fd1c00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.595+0000 7fd1db200640 1 --2- 172.21.15.47:0/385597102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1c00a2fa0 0x7fd1c00a5460 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd1c402c8e0 tx=0x7fd1c4005b70 comp rx=0 tx=0).stop 2024-03-21T11:25:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.595+0000 7fd1db200640 1 -- 172.21.15.47:0/385597102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1dc109770 msgr2=0x7fd1dc1a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.595+0000 7fd1db200640 1 --2- 172.21.15.47:0/385597102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1dc109770 0x7fd1dc1a5450 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009fd0 tx=0x7fd1d400ef10 comp rx=0 tx=0).stop 2024-03-21T11:25:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.595+0000 7fd1db200640 1 -- 172.21.15.47:0/385597102 shutdown_connections 2024-03-21T11:25:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.595+0000 7fd1db200640 1 --2- 172.21.15.47:0/385597102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1c00a2fa0 0x7fd1c00a5460 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.595+0000 7fd1db200640 1 --2- 172.21.15.47:0/385597102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1dc109770 0x7fd1dc1a5450 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.595+0000 7fd1db200640 1 --2- 172.21.15.47:0/385597102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1dc108db0 0x7fd1dc19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.595+0000 7fd1db200640 1 --2- 172.21.15.47:0/385597102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1dc107bc0 0x7fd1dc19dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.595+0000 7fd1db200640 1 -- 172.21.15.47:0/385597102 >> 172.21.15.47:0/385597102 conn(0x7fd1dc076b70 msgr2=0x7fd1dc10c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.595+0000 7fd1db200640 1 -- 172.21.15.47:0/385597102 shutdown_connections 2024-03-21T11:25:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.595+0000 7fd1db200640 1 -- 172.21.15.47:0/385597102 wait complete. 2024-03-21T11:25:28.613 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:25:28.613 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:25:28.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:28 smithi100 ceph-mon[36245]: pgmap v464: 129 pgs: 129 active+clean; 8.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 839 KiB/s wr, 243 op/s 2024-03-21T11:25:28.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:28 smithi102 ceph-mon[36748]: pgmap v464: 129 pgs: 129 active+clean; 8.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 839 KiB/s wr, 243 op/s 2024-03-21T11:25:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.846+0000 7f2cb5000640 1 -- 172.21.15.47:0/1283384131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb0102790 msgr2=0x7f2cb0102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.846+0000 7f2cb5000640 1 --2- 172.21.15.47:0/1283384131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb0102790 0x7f2cb0102c10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009860 tx=0x7f2c9802bb70 comp rx=0 tx=0).stop 2024-03-21T11:25:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.846+0000 7f2cb5000640 1 -- 172.21.15.47:0/1283384131 shutdown_connections 2024-03-21T11:25:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.846+0000 7f2cb5000640 1 --2- 172.21.15.47:0/1283384131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb0103150 0x7f2cb01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.846+0000 7f2cb5000640 1 --2- 172.21.15.47:0/1283384131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb0102790 0x7f2cb0102c10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.846+0000 7f2cb5000640 1 --2- 172.21.15.47:0/1283384131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cb01015a0 0x7f2cb01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.846+0000 7f2cb5000640 1 -- 172.21.15.47:0/1283384131 >> 172.21.15.47:0/1283384131 conn(0x7f2cb00fcb10 msgr2=0x7f2cb00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.846+0000 7f2cb5000640 1 -- 172.21.15.47:0/1283384131 shutdown_connections 2024-03-21T11:25:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.846+0000 7f2cb5000640 1 -- 172.21.15.47:0/1283384131 wait complete. 2024-03-21T11:25:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.847+0000 7f2cb5000640 1 Processor -- start 2024-03-21T11:25:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.847+0000 7f2cb5000640 1 -- start start 2024-03-21T11:25:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2cb5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb01015a0 0x7f2cb0199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2cb5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cb0102790 0x7f2cb0199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2cb5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb0103150 0x7f2cb01a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2cb5000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb010d230 con 0x7f2cb0103150 2024-03-21T11:25:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2cb5000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb010d0b0 con 0x7f2cb01015a0 2024-03-21T11:25:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2cb5000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb010d3b0 con 0x7f2cb0102790 2024-03-21T11:25:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2caea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cb0102790 0x7f2cb0199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2caf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb01015a0 0x7f2cb0199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2caf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb01015a0 0x7f2cb0199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41720/0 (socket says 172.21.15.47:41720) 2024-03-21T11:25:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2caf400640 1 -- 172.21.15.47:0/1009677878 learned_addr learned my addr 172.21.15.47:0/1009677878 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2caea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cb0102790 0x7f2cb0199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52674/0 (socket says 172.21.15.47:52674) 2024-03-21T11:25:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2cafe00640 1 --2- 172.21.15.47:0/1009677878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb0103150 0x7f2cb01a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2caea00640 1 -- 172.21.15.47:0/1009677878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb01015a0 msgr2=0x7f2cb0199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2caea00640 1 --2- 172.21.15.47:0/1009677878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb01015a0 0x7f2cb0199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2caea00640 1 -- 172.21.15.47:0/1009677878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb0103150 msgr2=0x7f2cb01a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2caea00640 1 --2- 172.21.15.47:0/1009677878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb0103150 0x7f2cb01a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.848+0000 7f2caea00640 1 -- 172.21.15.47:0/1009677878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb01a1670 con 0x7f2cb0102790 2024-03-21T11:25:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.849+0000 7f2caea00640 1 --2- 172.21.15.47:0/1009677878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cb0102790 0x7f2cb0199f90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2c9802c0b0 tx=0x7f2c98032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.849+0000 7f2ca7e00640 1 -- 172.21.15.47:0/1009677878 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98041020 con 0x7f2cb0102790 2024-03-21T11:25:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.849+0000 7f2ca7e00640 1 -- 172.21.15.47:0/1009677878 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c9802c9f0 con 0x7f2cb0102790 2024-03-21T11:25:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.849+0000 7f2cb5000640 1 -- 172.21.15.47:0/1009677878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb01a1900 con 0x7f2cb0102790 2024-03-21T11:25:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.849+0000 7f2ca7e00640 1 -- 172.21.15.47:0/1009677878 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98039040 con 0x7f2cb0102790 2024-03-21T11:25:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.849+0000 7f2cb5000640 1 -- 172.21.15.47:0/1009677878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb01a1e10 con 0x7f2cb0102790 2024-03-21T11:25:28.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.852+0000 7f2ca7e00640 1 -- 172.21.15.47:0/1009677878 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2c98044020 con 0x7f2cb0102790 2024-03-21T11:25:28.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.852+0000 7f2cb943a740 1 Processor -- start 2024-03-21T11:25:28.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.853+0000 7f2cb943a740 1 -- start start 2024-03-21T11:25:28.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.853+0000 7f2cb943a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa58ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.853+0000 7f2cb943a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa58e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.853+0000 7f2cb943a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58e580 0x5609aa594e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.853+0000 7f2cb943a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa4f12c0 con 0x5609aa58e580 2024-03-21T11:25:28.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.853+0000 7f2cb943a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa499c90 con 0x5609aa58c9f0 2024-03-21T11:25:28.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.853+0000 7f2cb943a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa4ed360 con 0x5609aa58dbc0 2024-03-21T11:25:28.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.853+0000 7f2cafe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa58ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.853+0000 7f2caf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa58e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.853+0000 7f2cafe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa58ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41726/0 (socket says 172.21.15.47:41726) 2024-03-21T11:25:28.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.853+0000 7f2cafe00640 1 -- 172.21.15.47:0/505845215 learned_addr learned my addr 172.21.15.47:0/505845215 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:28.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.853+0000 7f2caea00640 1 --2- 172.21.15.47:0/505845215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58e580 0x5609aa594e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.854+0000 7f2cafe00640 1 -- 172.21.15.47:0/505845215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 msgr2=0x5609aa58e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.854+0000 7f2cafe00640 1 --2- 172.21.15.47:0/505845215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa58e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.854+0000 7f2cafe00640 1 -- 172.21.15.47:0/505845215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58e580 msgr2=0x5609aa594e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.854+0000 7f2cafe00640 1 --2- 172.21.15.47:0/505845215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58e580 0x5609aa594e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.854+0000 7f2cafe00640 1 -- 172.21.15.47:0/505845215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609aa4be080 con 0x5609aa58c9f0 2024-03-21T11:25:28.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.854+0000 7f2caea00640 1 --2- 172.21.15.47:0/505845215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58e580 0x5609aa594e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:28.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.854+0000 7f2ca7e00640 1 --2- 172.21.15.47:0/1009677878 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2c940a2fa0 0x7f2c940a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.854+0000 7f2cafe00640 1 --2- 172.21.15.47:0/505845215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa58ce10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009c10 tx=0x7f2ca00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5adbcbb4bcef630 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:28.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.854+0000 7f2ca7e00640 1 -- 172.21.15.47:0/1009677878 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2c980f0fe0 con 0x7f2cb0102790 2024-03-21T11:25:28.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.854+0000 7f2caf400640 1 --2- 172.21.15.47:0/1009677878 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2c940a2fa0 0x7f2c940a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.854+0000 7f2ca4c00640 1 -- 172.21.15.47:0/505845215 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0012450 con 0x5609aa58c9f0 2024-03-21T11:25:28.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.855+0000 7f2ca4c00640 1 -- 172.21.15.47:0/505845215 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ca0012ad0 con 0x5609aa58c9f0 2024-03-21T11:25:28.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.855+0000 7f2ca4c00640 1 -- 172.21.15.47:0/505845215 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000fbd0 con 0x5609aa58c9f0 2024-03-21T11:25:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.855+0000 7f2cb943a740 1 -- 172.21.15.47:0/505845215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 msgr2=0x5609aa58ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.855+0000 7f2cb943a740 1 --2- 172.21.15.47:0/505845215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa58ce10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009c10 tx=0x7f2ca00117b0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.855+0000 7f2caf400640 1 --2- 172.21.15.47:0/1009677878 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2c940a2fa0 0x7f2c940a5460 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009fd0 tx=0x7f2ca8010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.855+0000 7f2cb943a740 1 -- 172.21.15.47:0/505845215 shutdown_connections 2024-03-21T11:25:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.855+0000 7f2cb943a740 1 --2- 172.21.15.47:0/505845215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58e580 0x5609aa594e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.855+0000 7f2cb943a740 1 --2- 172.21.15.47:0/505845215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa58e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.855+0000 7f2cb943a740 1 --2- 172.21.15.47:0/505845215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa58ce10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.855+0000 7f2cb943a740 1 -- 172.21.15.47:0/505845215 >> 172.21.15.47:0/505845215 conn(0x5609aa587de0 msgr2=0x5609aa5881e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.855+0000 7f2cb943a740 1 -- 172.21.15.47:0/505845215 shutdown_connections 2024-03-21T11:25:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.856+0000 7f2cb943a740 1 -- 172.21.15.47:0/505845215 wait complete. 2024-03-21T11:25:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.856+0000 7f2cb943a740 1 -- start start 2024-03-21T11:25:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.856+0000 7f2cb943a740 1 Processor -- start 2024-03-21T11:25:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2cb943a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa596c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2cb943a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58dbc0 0x5609aa5971d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2cb943a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58e580 0x5609aa59e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2cb943a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa595590 con 0x5609aa58dbc0 2024-03-21T11:25:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2cb943a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa4ad0d0 con 0x5609aa58c9f0 2024-03-21T11:25:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2cb943a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa595710 con 0x5609aa58e580 2024-03-21T11:25:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2cb943a740 10 client.0 _ll_register_callbacks cb 0x5609aa4b1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2caf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa596c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2cafe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58e580 0x5609aa59e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2caf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa596c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41728/0 (socket says 172.21.15.47:41728) 2024-03-21T11:25:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2caf400640 1 -- 172.21.15.47:0/1799637834 learned_addr learned my addr 172.21.15.47:0/1799637834 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2caea00640 1 --2- 172.21.15.47:0/1799637834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58dbc0 0x5609aa5971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2cafe00640 1 -- 172.21.15.47:0/1799637834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 msgr2=0x5609aa596c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2cafe00640 1 --2- 172.21.15.47:0/1799637834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa596c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2cafe00640 1 -- 172.21.15.47:0/1799637834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58dbc0 msgr2=0x5609aa5971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2cafe00640 1 --2- 172.21.15.47:0/1799637834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58dbc0 0x5609aa5971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.857+0000 7f2cafe00640 1 -- 172.21.15.47:0/1799637834 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609aa2053f0 con 0x5609aa58e580 2024-03-21T11:25:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.858+0000 7f2cafe00640 1 --2- 172.21.15.47:0/1799637834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58e580 0x5609aa59e210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0011cb0 tx=0x7f2ca0012700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1736fbd4d8c32409 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.858+0000 7f2c9d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:25:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.858+0000 7f2c9d600640 1 -- 172.21.15.47:0/1799637834 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0029030 con 0x5609aa58e580 2024-03-21T11:25:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.858+0000 7f2cb943a740 1 -- 172.21.15.47:0/1799637834 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ca0012ad0 con 0x5609aa58e580 2024-03-21T11:25:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.858+0000 7f2c9d600640 1 -- 172.21.15.47:0/1799637834 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2ca0020040 con 0x5609aa58e580 2024-03-21T11:25:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.858+0000 7f2c9d600640 1 -- 172.21.15.47:0/1799637834 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ca0033050 con 0x5609aa58e580 2024-03-21T11:25:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.858+0000 7f2cb943a740 10 client.26083 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.858+0000 7f2cb943a740 1 -- 172.21.15.47:0/1799637834 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5609aa2cf7d0 con 0x5609aa58e580 2024-03-21T11:25:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.859+0000 7f2c9d600640 1 -- 172.21.15.47:0/1799637834 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ca0012ad0 con 0x5609aa58e580 2024-03-21T11:25:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.859+0000 7f2c9d600640 1 -- 172.21.15.47:0/1799637834 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2ca0005040 con 0x5609aa58e580 2024-03-21T11:25:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.859+0000 7f2cb943a740 10 client.26083 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.859+0000 7f2cb943a740 10 client.26083 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:25:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.859+0000 7f2cb943a740 20 client.26083 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.859+0000 7f2cb943a740 1 --2- 172.21.15.47:0/1799637834 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5609aa5a2420 0x5609aa5a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.859+0000 7f2cb943a740 4 client.26083 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:25:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.859+0000 7f2cb943a740 1 -- 172.21.15.47:0/1799637834 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609aa4fe800 con 0x5609aa5a2420 2024-03-21T11:25:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.859+0000 7f2caf400640 1 --2- 172.21.15.47:0/1799637834 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5609aa5a2420 0x5609aa5a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.860+0000 7f2caf400640 1 --2- 172.21.15.47:0/1799637834 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5609aa5a2420 0x5609aa5a4810 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=77333361a819c2d8 server_cookie=8690ecc80f111f63 in_seq=0 out_seq=0 2024-03-21T11:25:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.860+0000 7f2c9d600640 10 client.26083 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:25:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.860+0000 7f2c9d600640 1 -- 172.21.15.47:0/1799637834 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5609aa4fe800 con 0x5609aa5a2420 2024-03-21T11:25:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.860+0000 7f2c9d600640 10 client.26083 handle_command_reply: tid=0 2024-03-21T11:25:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.860+0000 7f2cb943a740 1 client.26083 shutdown 2024-03-21T11:25:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.861+0000 7f2cb943a740 1 -- 172.21.15.47:0/1799637834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58e580 msgr2=0x5609aa59e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.861+0000 7f2cb943a740 1 --2- 172.21.15.47:0/1799637834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58e580 0x5609aa59e210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0011cb0 tx=0x7f2ca0012700 comp rx=0 tx=0).stop 2024-03-21T11:25:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.861+0000 7f2cb943a740 1 -- 172.21.15.47:0/1799637834 shutdown_connections 2024-03-21T11:25:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.861+0000 7f2cb943a740 1 --2- 172.21.15.47:0/1799637834 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5609aa5a2420 0x5609aa5a4810 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.861+0000 7f2cb943a740 1 --2- 172.21.15.47:0/1799637834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58e580 0x5609aa59e210 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.861+0000 7f2cb943a740 1 --2- 172.21.15.47:0/1799637834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58dbc0 0x5609aa5971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.861+0000 7f2cb943a740 1 --2- 172.21.15.47:0/1799637834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa596c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.861+0000 7f2cb943a740 1 -- 172.21.15.47:0/1799637834 >> 172.21.15.47:0/1799637834 conn(0x5609aa587de0 msgr2=0x5609aa587160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:28.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.861+0000 7f2c9d600640 0 client.26083 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:25:28.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.861+0000 7f2cb943a740 1 -- 172.21.15.47:0/1799637834 shutdown_connections 2024-03-21T11:25:28.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.861+0000 7f2cb943a740 1 -- 172.21.15.47:0/1799637834 wait complete. 2024-03-21T11:25:28.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.861+0000 7f2cb943a740 20 client.26083 trim_cache size 0 max 16384 2024-03-21T11:25:28.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.876+0000 7f2cb943a740 1 Processor -- start 2024-03-21T11:25:28.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.876+0000 7f2cb943a740 1 -- start start 2024-03-21T11:25:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.876+0000 7f2cb943a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa5868d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.876+0000 7f2cb943a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa59e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.876+0000 7f2cb943a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58e580 0x5609aa586e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.876+0000 7f2cb943a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb01ac0c0 con 0x5609aa58e580 2024-03-21T11:25:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.876+0000 7f2cb943a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb01ada40 con 0x5609aa58c9f0 2024-03-21T11:25:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.876+0000 7f2cb943a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb01ac720 con 0x5609aa58dbc0 2024-03-21T11:25:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.876+0000 7f2caea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa59e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.876+0000 7f2cafe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58e580 0x5609aa586e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.876+0000 7f2caea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa59e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48536/0 (socket says 172.21.15.47:48536) 2024-03-21T11:25:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.876+0000 7f2caea00640 1 -- 172.21.15.47:0/1792993390 learned_addr learned my addr 172.21.15.47:0/1792993390 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.877+0000 7f2caf400640 1 --2- 172.21.15.47:0/1792993390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa5868d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.877+0000 7f2caea00640 1 -- 172.21.15.47:0/1792993390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 msgr2=0x5609aa5868d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.877+0000 7f2caea00640 1 --2- 172.21.15.47:0/1792993390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa5868d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.877+0000 7f2caea00640 1 -- 172.21.15.47:0/1792993390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58e580 msgr2=0x5609aa586e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.877+0000 7f2caea00640 1 --2- 172.21.15.47:0/1792993390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58e580 0x5609aa586e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.877+0000 7f2caea00640 1 -- 172.21.15.47:0/1792993390 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609aa5838b0 con 0x5609aa58dbc0 2024-03-21T11:25:28.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.877+0000 7f2caf400640 1 --2- 172.21.15.47:0/1792993390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa5868d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:28.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.877+0000 7f2caea00640 1 --2- 172.21.15.47:0/1792993390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa59e1d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2c9802cbc0 tx=0x7f2c9802bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adc6592b3af986de server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.877+0000 7f2c9fe00640 1 -- 172.21.15.47:0/1792993390 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980bce60 con 0x5609aa58dbc0 2024-03-21T11:25:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.877+0000 7f2c9fe00640 1 -- 172.21.15.47:0/1792993390 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c980bda20 con 0x5609aa58dbc0 2024-03-21T11:25:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.877+0000 7f2c9fe00640 1 -- 172.21.15.47:0/1792993390 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980bcc80 con 0x5609aa58dbc0 2024-03-21T11:25:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.878+0000 7f2cb943a740 1 -- 172.21.15.47:0/1792993390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 msgr2=0x5609aa59e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.878+0000 7f2cb943a740 1 --2- 172.21.15.47:0/1792993390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa59e1d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2c9802cbc0 tx=0x7f2c9802bb70 comp rx=0 tx=0).stop 2024-03-21T11:25:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.878+0000 7f2cb943a740 1 -- 172.21.15.47:0/1792993390 shutdown_connections 2024-03-21T11:25:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.878+0000 7f2cb943a740 1 --2- 172.21.15.47:0/1792993390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58e580 0x5609aa586e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.878+0000 7f2cb943a740 1 --2- 172.21.15.47:0/1792993390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa59e1d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.878+0000 7f2cb943a740 1 --2- 172.21.15.47:0/1792993390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58c9f0 0x5609aa5868d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.878+0000 7f2cb943a740 1 -- 172.21.15.47:0/1792993390 >> 172.21.15.47:0/1792993390 conn(0x5609aa587de0 msgr2=0x5609aa596c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.878+0000 7f2cb943a740 1 -- 172.21.15.47:0/1792993390 shutdown_connections 2024-03-21T11:25:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.878+0000 7f2cb943a740 1 -- 172.21.15.47:0/1792993390 wait complete. 2024-03-21T11:25:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.879+0000 7f2cb943a740 1 -- start start 2024-03-21T11:25:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.879+0000 7f2cb943a740 1 Processor -- start 2024-03-21T11:25:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.879+0000 7f2cb943a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58c9f0 0x5609aa599b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.879+0000 7f2cb943a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa59a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.879+0000 7f2cb943a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58e580 0x5609aa59d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.879+0000 7f2cb943a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb01ad3e0 con 0x5609aa58c9f0 2024-03-21T11:25:28.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.879+0000 7f2cb943a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb01acd80 con 0x5609aa58e580 2024-03-21T11:25:28.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.879+0000 7f2cb943a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb01aa280 con 0x5609aa58dbc0 2024-03-21T11:25:28.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.879+0000 7f2cb943a740 10 client.0 _ll_register_callbacks cb 0x7f2ca80099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:28.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.880+0000 7f2caf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58c9f0 0x5609aa599b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.880+0000 7f2caea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa59a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.880+0000 7f2caf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58c9f0 0x5609aa599b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54742/0 (socket says 172.21.15.47:54742) 2024-03-21T11:25:28.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.880+0000 7f2caf400640 1 -- 172.21.15.47:0/3984178335 learned_addr learned my addr 172.21.15.47:0/3984178335 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:28.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.880+0000 7f2cafe00640 1 --2- 172.21.15.47:0/3984178335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58e580 0x5609aa59d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.880+0000 7f2caf400640 1 -- 172.21.15.47:0/3984178335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 msgr2=0x5609aa59a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.880+0000 7f2caf400640 1 --2- 172.21.15.47:0/3984178335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa59a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.880+0000 7f2caf400640 1 -- 172.21.15.47:0/3984178335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58e580 msgr2=0x5609aa59d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.880+0000 7f2caf400640 1 --2- 172.21.15.47:0/3984178335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58e580 0x5609aa59d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.880+0000 7f2caf400640 1 -- 172.21.15.47:0/3984178335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609aa59d670 con 0x5609aa58c9f0 2024-03-21T11:25:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.880+0000 7f2caea00640 1 --2- 172.21.15.47:0/3984178335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa59a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.880+0000 7f2caf400640 1 --2- 172.21.15.47:0/3984178335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58c9f0 0x5609aa599b20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5609aa5a4f40 tx=0x7f2ca8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63850df9dad1aeb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.880+0000 7f2c9d600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:25:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.881+0000 7f2c9d600640 1 -- 172.21.15.47:0/3984178335 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8020040 con 0x5609aa58c9f0 2024-03-21T11:25:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.881+0000 7f2c9d600640 1 -- 172.21.15.47:0/3984178335 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2ca800da70 con 0x5609aa58c9f0 2024-03-21T11:25:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.881+0000 7f2cb943a740 1 -- 172.21.15.47:0/3984178335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c980bda20 con 0x5609aa58c9f0 2024-03-21T11:25:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.881+0000 7f2c9d600640 1 -- 172.21.15.47:0/3984178335 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ca800dcb0 con 0x5609aa58c9f0 2024-03-21T11:25:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.881+0000 7f2cb943a740 10 client.15897 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.881+0000 7f2cb943a740 1 -- 172.21.15.47:0/3984178335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5609aa5fde70 con 0x5609aa58c9f0 2024-03-21T11:25:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.881+0000 7f2c9d600640 1 -- 172.21.15.47:0/3984178335 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c980bda20 con 0x5609aa58c9f0 2024-03-21T11:25:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.882+0000 7f2c9d600640 1 -- 172.21.15.47:0/3984178335 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2ca8032070 con 0x5609aa58c9f0 2024-03-21T11:25:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.882+0000 7f2cb943a740 10 client.15897 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.882+0000 7f2cb943a740 10 client.15897 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:25:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.882+0000 7f2cb943a740 20 client.15897 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.882+0000 7f2cb943a740 1 --2- 172.21.15.47:0/3984178335 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5609aa5fe790 0x5609aa600c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.882+0000 7f2cb943a740 4 client.15897 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:25:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.882+0000 7f2cb943a740 1 -- 172.21.15.47:0/3984178335 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5609aa4f8b50 con 0x5609aa5fe790 2024-03-21T11:25:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.882+0000 7f2caea00640 1 --2- 172.21.15.47:0/3984178335 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5609aa5fe790 0x5609aa600c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:28.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.882+0000 7f2caea00640 1 --2- 172.21.15.47:0/3984178335 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5609aa5fe790 0x5609aa600c30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cc0e6485599cc0e4 server_cookie=97827a05902a7d0b in_seq=0 out_seq=0 2024-03-21T11:25:28.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.882+0000 7f2c9d600640 10 client.15897 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:25:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2c9d600640 1 -- 172.21.15.47:0/3984178335 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5609aa4f8b50 con 0x5609aa5fe790 2024-03-21T11:25:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2c9d600640 10 client.15897 handle_command_reply: tid=0 2024-03-21T11:25:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2cb943a740 1 client.15897 shutdown 2024-03-21T11:25:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2cb943a740 1 -- 172.21.15.47:0/3984178335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58c9f0 msgr2=0x5609aa599b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2cb943a740 1 --2- 172.21.15.47:0/3984178335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58c9f0 0x5609aa599b20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5609aa5a4f40 tx=0x7f2ca8004ab0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2cb943a740 1 -- 172.21.15.47:0/3984178335 shutdown_connections 2024-03-21T11:25:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2cb943a740 1 --2- 172.21.15.47:0/3984178335 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5609aa5fe790 0x5609aa600c30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2cb943a740 1 --2- 172.21.15.47:0/3984178335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5609aa58e580 0x5609aa59d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2cb943a740 1 --2- 172.21.15.47:0/3984178335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609aa58dbc0 0x5609aa59a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2cb943a740 1 --2- 172.21.15.47:0/3984178335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609aa58c9f0 0x5609aa599b20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2cb943a740 1 -- 172.21.15.47:0/3984178335 >> 172.21.15.47:0/3984178335 conn(0x5609aa587de0 msgr2=0x5609aa58b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2c9d600640 0 client.15897 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:25:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2cb943a740 1 -- 172.21.15.47:0/3984178335 shutdown_connections 2024-03-21T11:25:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2cb943a740 1 -- 172.21.15.47:0/3984178335 wait complete. 2024-03-21T11:25:28.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.883+0000 7f2cb943a740 20 client.15897 trim_cache size 0 max 16384 2024-03-21T11:25:28.889 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:25:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.884+0000 7f2cb5000640 1 -- 172.21.15.47:0/1009677878 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2c940a2fa0 msgr2=0x7f2c940a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.884+0000 7f2cb5000640 1 --2- 172.21.15.47:0/1009677878 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2c940a2fa0 0x7f2c940a5460 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009fd0 tx=0x7f2ca8010040 comp rx=0 tx=0).stop 2024-03-21T11:25:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.884+0000 7f2cb5000640 1 -- 172.21.15.47:0/1009677878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cb0102790 msgr2=0x7f2cb0199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.884+0000 7f2cb5000640 1 --2- 172.21.15.47:0/1009677878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cb0102790 0x7f2cb0199f90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2c9802c0b0 tx=0x7f2c98032430 comp rx=0 tx=0).stop 2024-03-21T11:25:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.884+0000 7f2cb5000640 1 -- 172.21.15.47:0/1009677878 shutdown_connections 2024-03-21T11:25:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.884+0000 7f2cb5000640 1 --2- 172.21.15.47:0/1009677878 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2c940a2fa0 0x7f2c940a5460 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.884+0000 7f2cb5000640 1 --2- 172.21.15.47:0/1009677878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cb0103150 0x7f2cb01a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.884+0000 7f2cb5000640 1 --2- 172.21.15.47:0/1009677878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cb0102790 0x7f2cb0199f90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.884+0000 7f2cb5000640 1 --2- 172.21.15.47:0/1009677878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cb01015a0 0x7f2cb0199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.884+0000 7f2cb5000640 1 -- 172.21.15.47:0/1009677878 >> 172.21.15.47:0/1009677878 conn(0x7f2cb00fcb10 msgr2=0x7f2cb00738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:28.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.885+0000 7f2cb5000640 1 -- 172.21.15.47:0/1009677878 shutdown_connections 2024-03-21T11:25:28.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:28.885+0000 7f2cb5000640 1 -- 172.21.15.47:0/1009677878 wait complete. 2024-03-21T11:25:28.904 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:25:28.904 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:25:29.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.118+0000 7fc6a3400640 1 -- 172.21.15.47:0/3131047431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc69c076060 msgr2=0x7fc69c076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.118+0000 7fc6a3400640 1 --2- 172.21.15.47:0/3131047431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc69c076060 0x7fc69c076480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc694009be0 tx=0x7fc69402bef0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.118+0000 7fc6a3400640 1 -- 172.21.15.47:0/3131047431 shutdown_connections 2024-03-21T11:25:29.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.118+0000 7fc6a3400640 1 --2- 172.21.15.47:0/3131047431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc69c077c40 0x7fc69c0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.118+0000 7fc6a3400640 1 --2- 172.21.15.47:0/3131047431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc69c0772a0 0x7fc69c077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.118+0000 7fc6a3400640 1 --2- 172.21.15.47:0/3131047431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc69c076060 0x7fc69c076480 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.118+0000 7fc6a3400640 1 -- 172.21.15.47:0/3131047431 >> 172.21.15.47:0/3131047431 conn(0x7fc69c0fee50 msgr2=0x7fc69c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.119+0000 7fc6a3400640 1 -- 172.21.15.47:0/3131047431 shutdown_connections 2024-03-21T11:25:29.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.119+0000 7fc6a3400640 1 -- 172.21.15.47:0/3131047431 wait complete. 2024-03-21T11:25:29.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.120+0000 7fc6a3400640 1 Processor -- start 2024-03-21T11:25:29.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.120+0000 7fc6a3400640 1 -- start start 2024-03-21T11:25:29.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.120+0000 7fc6a3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc69c076060 0x7fc69c19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.120+0000 7fc6a3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc69c0772a0 0x7fc69c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.120+0000 7fc6a3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc69c077c40 0x7fc69c1a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.120+0000 7fc6a3400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c1139b0 con 0x7fc69c077c40 2024-03-21T11:25:29.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.120+0000 7fc6a3400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c113830 con 0x7fc69c0772a0 2024-03-21T11:25:29.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.120+0000 7fc6a3400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c113b30 con 0x7fc69c076060 2024-03-21T11:25:29.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.120+0000 7fc6a0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc69c076060 0x7fc69c19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.120+0000 7fc6a0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc69c076060 0x7fc69c19e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48570/0 (socket says 172.21.15.47:48570) 2024-03-21T11:25:29.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.120+0000 7fc6a0a00640 1 -- 172.21.15.47:0/3477739847 learned_addr learned my addr 172.21.15.47:0/3477739847 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.121+0000 7fc69be00640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc69c0772a0 0x7fc69c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.121+0000 7fc6a1400640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc69c077c40 0x7fc69c1a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.121+0000 7fc6a0a00640 1 -- 172.21.15.47:0/3477739847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc69c0772a0 msgr2=0x7fc69c19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.121+0000 7fc6a0a00640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc69c0772a0 0x7fc69c19e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.121+0000 7fc6a0a00640 1 -- 172.21.15.47:0/3477739847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc69c077c40 msgr2=0x7fc69c1a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.121+0000 7fc6a0a00640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc69c077c40 0x7fc69c1a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.121+0000 7fc6a0a00640 1 -- 172.21.15.47:0/3477739847 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c1a5c90 con 0x7fc69c076060 2024-03-21T11:25:29.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.121+0000 7fc69be00640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc69c0772a0 0x7fc69c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.121+0000 7fc6a0a00640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc69c076060 0x7fc69c19e070 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc694000f80 tx=0x7fc6940337b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.121+0000 7fc699600640 1 -- 172.21.15.47:0/3477739847 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc694043020 con 0x7fc69c076060 2024-03-21T11:25:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.121+0000 7fc699600640 1 -- 172.21.15.47:0/3477739847 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc69402ca60 con 0x7fc69c076060 2024-03-21T11:25:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.121+0000 7fc6a3400640 1 -- 172.21.15.47:0/3477739847 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69c1a5f20 con 0x7fc69c076060 2024-03-21T11:25:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.122+0000 7fc699600640 1 -- 172.21.15.47:0/3477739847 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc694034690 con 0x7fc69c076060 2024-03-21T11:25:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.122+0000 7fc6a3400640 1 -- 172.21.15.47:0/3477739847 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c1a6390 con 0x7fc69c076060 2024-03-21T11:25:29.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.125+0000 7fc699600640 1 -- 172.21.15.47:0/3477739847 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc694046020 con 0x7fc69c076060 2024-03-21T11:25:29.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.125+0000 7fc6a616a740 1 Processor -- start 2024-03-21T11:25:29.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.125+0000 7fc6a616a740 1 -- start start 2024-03-21T11:25:29.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.125+0000 7fc6a616a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0d810 0x5596edc0dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.125+0000 7fc6a616a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0e9e0 0x5596edc0ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.125+0000 7fc6a616a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0f3a0 0x5596edc15c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.125+0000 7fc6a616a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596edb722c0 con 0x5596edc0e9e0 2024-03-21T11:25:29.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.125+0000 7fc6a616a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596edb1ac90 con 0x5596edc0d810 2024-03-21T11:25:29.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.126+0000 7fc6a616a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596edb6e360 con 0x5596edc0f3a0 2024-03-21T11:25:29.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.126+0000 7fc6a1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0d810 0x5596edc0dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.126+0000 7fc6a1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0d810 0x5596edc0dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42512/0 (socket says 172.21.15.47:42512) 2024-03-21T11:25:29.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.126+0000 7fc69be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0f3a0 0x5596edc15c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.126+0000 7fc6a1400640 1 -- 172.21.15.47:0/4242483184 learned_addr learned my addr 172.21.15.47:0/4242483184 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.126+0000 7fc6a0a00640 1 --2- 172.21.15.47:0/4242483184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0e9e0 0x5596edc0ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.126+0000 7fc6a1400640 1 -- 172.21.15.47:0/4242483184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0f3a0 msgr2=0x5596edc15c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.126+0000 7fc6a1400640 1 --2- 172.21.15.47:0/4242483184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0f3a0 0x5596edc15c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.126+0000 7fc6a1400640 1 -- 172.21.15.47:0/4242483184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0e9e0 msgr2=0x5596edc0ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.126+0000 7fc6a1400640 1 --2- 172.21.15.47:0/4242483184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0e9e0 0x5596edc0ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.126+0000 7fc6a1400640 1 -- 172.21.15.47:0/4242483184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596edb3f080 con 0x5596edc0d810 2024-03-21T11:25:29.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.126+0000 7fc69be00640 1 --2- 172.21.15.47:0/4242483184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0f3a0 0x5596edc15c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:29.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.126+0000 7fc699600640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc67c0a2f20 0x7fc67c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.127+0000 7fc6a1400640 1 --2- 172.21.15.47:0/4242483184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0d810 0x5596edc0dc30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009c00 tx=0x7fc68c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3074e57253349f9a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.127+0000 7fc699600640 1 -- 172.21.15.47:0/3477739847 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc6940f1fa0 con 0x7fc69c076060 2024-03-21T11:25:29.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.127+0000 7fc6a0a00640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc67c0a2f20 0x7fc67c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.127+0000 7fc691c00640 1 -- 172.21.15.47:0/4242483184 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c012450 con 0x5596edc0d810 2024-03-21T11:25:29.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.127+0000 7fc691c00640 1 -- 172.21.15.47:0/4242483184 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc68c012ad0 con 0x5596edc0d810 2024-03-21T11:25:29.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.127+0000 7fc691c00640 1 -- 172.21.15.47:0/4242483184 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c00fc00 con 0x5596edc0d810 2024-03-21T11:25:29.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.128+0000 7fc6a616a740 1 -- 172.21.15.47:0/4242483184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0d810 msgr2=0x5596edc0dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.128+0000 7fc6a0a00640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc67c0a2f20 0x7fc67c0a53e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc6940ba740 tx=0x7fc694009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.128+0000 7fc6a616a740 1 --2- 172.21.15.47:0/4242483184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0d810 0x5596edc0dc30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009c00 tx=0x7fc68c0117b0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.128+0000 7fc6a616a740 1 -- 172.21.15.47:0/4242483184 shutdown_connections 2024-03-21T11:25:29.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.128+0000 7fc6a616a740 1 --2- 172.21.15.47:0/4242483184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0f3a0 0x5596edc15c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.128+0000 7fc6a616a740 1 --2- 172.21.15.47:0/4242483184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0e9e0 0x5596edc0ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.128+0000 7fc6a616a740 1 --2- 172.21.15.47:0/4242483184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0d810 0x5596edc0dc30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.128+0000 7fc6a616a740 1 -- 172.21.15.47:0/4242483184 >> 172.21.15.47:0/4242483184 conn(0x5596edc08c00 msgr2=0x5596edc09000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.128+0000 7fc6a616a740 1 -- 172.21.15.47:0/4242483184 shutdown_connections 2024-03-21T11:25:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.128+0000 7fc6a616a740 1 -- 172.21.15.47:0/4242483184 wait complete. 2024-03-21T11:25:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.128+0000 7fc6a616a740 1 -- start start 2024-03-21T11:25:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.128+0000 7fc6a616a740 1 Processor -- start 2024-03-21T11:25:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.129+0000 7fc6a616a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0d810 0x5596edc17ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.129+0000 7fc6a616a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0e9e0 0x5596edc18000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.129+0000 7fc6a616a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0f3a0 0x5596edc1f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.129+0000 7fc6a616a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596edc163b0 con 0x5596edc0f3a0 2024-03-21T11:25:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.129+0000 7fc6a616a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596edb2e0d0 con 0x5596edc0d810 2024-03-21T11:25:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.129+0000 7fc6a616a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596edc16530 con 0x5596edc0e9e0 2024-03-21T11:25:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.129+0000 7fc6a616a740 10 client.0 _ll_register_callbacks cb 0x5596edb32e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.129+0000 7fc69be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0e9e0 0x5596edc18000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.129+0000 7fc6a1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0d810 0x5596edc17ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.129+0000 7fc69be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0e9e0 0x5596edc18000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48586/0 (socket says 172.21.15.47:48586) 2024-03-21T11:25:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.129+0000 7fc69be00640 1 -- 172.21.15.47:0/3906810933 learned_addr learned my addr 172.21.15.47:0/3906810933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.130+0000 7fc6a1400640 1 --2- 172.21.15.47:0/3906810933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0f3a0 0x5596edc1f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.130+0000 7fc69be00640 1 -- 172.21.15.47:0/3906810933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0d810 msgr2=0x5596edc17ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.130+0000 7fc69be00640 1 --2- 172.21.15.47:0/3906810933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0d810 0x5596edc17ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.130+0000 7fc69be00640 1 -- 172.21.15.47:0/3906810933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0f3a0 msgr2=0x5596edc1f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.130+0000 7fc69be00640 1 --2- 172.21.15.47:0/3906810933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0f3a0 0x5596edc1f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.130+0000 7fc69be00640 1 -- 172.21.15.47:0/3906810933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596ed8863f0 con 0x5596edc0e9e0 2024-03-21T11:25:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.130+0000 7fc69be00640 1 --2- 172.21.15.47:0/3906810933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0e9e0 0x5596edc18000 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc684013710 tx=0x7fc684013bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4d39d11ee8e5377 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.130+0000 7fc68a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:25:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.130+0000 7fc68a000640 1 -- 172.21.15.47:0/3906810933 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684011dd0 con 0x5596edc0e9e0 2024-03-21T11:25:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.130+0000 7fc6a616a740 1 -- 172.21.15.47:0/3906810933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc68c012ad0 con 0x5596edc0e9e0 2024-03-21T11:25:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.130+0000 7fc68a000640 1 -- 172.21.15.47:0/3906810933 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc684004030 con 0x5596edc0e9e0 2024-03-21T11:25:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.131+0000 7fc68a000640 1 -- 172.21.15.47:0/3906810933 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc684004230 con 0x5596edc0e9e0 2024-03-21T11:25:29.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.131+0000 7fc6a616a740 10 client.26101 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:29.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.131+0000 7fc6a616a740 1 -- 172.21.15.47:0/3906810933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5596ed9507d0 con 0x5596edc0e9e0 2024-03-21T11:25:29.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.131+0000 7fc68a000640 1 -- 172.21.15.47:0/3906810933 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc68c012ad0 con 0x5596edc0e9e0 2024-03-21T11:25:29.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.131+0000 7fc68a000640 1 -- 172.21.15.47:0/3906810933 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc68401a070 con 0x5596edc0e9e0 2024-03-21T11:25:29.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.131+0000 7fc6a616a740 10 client.26101 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:29.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.131+0000 7fc6a616a740 10 client.26101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:25:29.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.131+0000 7fc6a616a740 20 client.26101 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:29.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.131+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3906810933 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5596edc23630 0x5596edc25a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.131+0000 7fc6a616a740 4 client.26101 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:25:29.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.131+0000 7fc6a616a740 1 -- 172.21.15.47:0/3906810933 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596edb7f800 con 0x5596edc23630 2024-03-21T11:25:29.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.132+0000 7fc6a1400640 1 --2- 172.21.15.47:0/3906810933 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5596edc23630 0x5596edc25a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.132+0000 7fc6a1400640 1 --2- 172.21.15.47:0/3906810933 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5596edc23630 0x5596edc25a50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d76dcd86f5eceb7d server_cookie=a5775cfee919d3fb in_seq=0 out_seq=0 2024-03-21T11:25:29.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.132+0000 7fc68a000640 10 client.26101 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:25:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.133+0000 7fc68a000640 1 -- 172.21.15.47:0/3906810933 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5596edb7f800 con 0x5596edc23630 2024-03-21T11:25:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.133+0000 7fc68a000640 10 client.26101 handle_command_reply: tid=0 2024-03-21T11:25:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.133+0000 7fc6a616a740 1 client.26101 shutdown 2024-03-21T11:25:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.134+0000 7fc6a616a740 1 -- 172.21.15.47:0/3906810933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0e9e0 msgr2=0x5596edc18000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.134+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3906810933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0e9e0 0x5596edc18000 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc684013710 tx=0x7fc684013bd0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.134+0000 7fc6a616a740 1 -- 172.21.15.47:0/3906810933 shutdown_connections 2024-03-21T11:25:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.134+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3906810933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0f3a0 0x5596edc1f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.134+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3906810933 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5596edc23630 0x5596edc25a50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.134+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3906810933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0e9e0 0x5596edc18000 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.134+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3906810933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0d810 0x5596edc17ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.134+0000 7fc6a616a740 1 -- 172.21.15.47:0/3906810933 >> 172.21.15.47:0/3906810933 conn(0x5596edc08c00 msgr2=0x5596edc07eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.134+0000 7fc68a000640 0 client.26101 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:25:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.134+0000 7fc6a616a740 1 -- 172.21.15.47:0/3906810933 shutdown_connections 2024-03-21T11:25:29.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.134+0000 7fc6a616a740 1 -- 172.21.15.47:0/3906810933 wait complete. 2024-03-21T11:25:29.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.134+0000 7fc6a616a740 20 client.26101 trim_cache size 0 max 16384 2024-03-21T11:25:29.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.148+0000 7fc6a616a740 1 Processor -- start 2024-03-21T11:25:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.148+0000 7fc6a616a740 1 -- start start 2024-03-21T11:25:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a616a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0d810 0x5596edc07620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a616a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0e9e0 0x5596edc1f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a616a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0f3a0 0x5596edc07b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a616a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c1b0760 con 0x5596edc0e9e0 2024-03-21T11:25:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a616a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c1b20e0 con 0x5596edc0f3a0 2024-03-21T11:25:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a616a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c1b0dc0 con 0x5596edc0d810 2024-03-21T11:25:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0d810 0x5596edc07620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0d810 0x5596edc07620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48600/0 (socket says 172.21.15.47:48600) 2024-03-21T11:25:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a1400640 1 -- 172.21.15.47:0/3639680303 learned_addr learned my addr 172.21.15.47:0/3639680303 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a1400640 1 --2- 172.21.15.47:0/3639680303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0f3a0 0x5596edc07b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc69be00640 1 --2- 172.21.15.47:0/3639680303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0e9e0 0x5596edc1f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a1400640 1 -- 172.21.15.47:0/3639680303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0f3a0 msgr2=0x5596edc07b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a1400640 1 --2- 172.21.15.47:0/3639680303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0f3a0 0x5596edc07b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a1400640 1 -- 172.21.15.47:0/3639680303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0e9e0 msgr2=0x5596edc1f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a1400640 1 --2- 172.21.15.47:0/3639680303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0e9e0 0x5596edc1f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.149+0000 7fc6a1400640 1 -- 172.21.15.47:0/3639680303 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596edc09650 con 0x5596edc0d810 2024-03-21T11:25:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.150+0000 7fc6a1400640 1 --2- 172.21.15.47:0/3639680303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0d810 0x5596edc07620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009a10 tx=0x7fc68c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40b48fbc19c00950 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.150+0000 7fc691200640 1 -- 172.21.15.47:0/3639680303 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c02f040 con 0x5596edc0d810 2024-03-21T11:25:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.150+0000 7fc691200640 1 -- 172.21.15.47:0/3639680303 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc68c009e00 con 0x5596edc0d810 2024-03-21T11:25:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.150+0000 7fc691200640 1 -- 172.21.15.47:0/3639680303 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c005030 con 0x5596edc0d810 2024-03-21T11:25:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.151+0000 7fc6a616a740 1 -- 172.21.15.47:0/3639680303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0d810 msgr2=0x5596edc07620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.151+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3639680303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0d810 0x5596edc07620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009a10 tx=0x7fc68c005f90 comp rx=0 tx=0).stop 2024-03-21T11:25:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.151+0000 7fc6a616a740 1 -- 172.21.15.47:0/3639680303 shutdown_connections 2024-03-21T11:25:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.151+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3639680303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0f3a0 0x5596edc07b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.151+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3639680303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0e9e0 0x5596edc1f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.151+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3639680303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0d810 0x5596edc07620 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.151+0000 7fc6a616a740 1 -- 172.21.15.47:0/3639680303 >> 172.21.15.47:0/3639680303 conn(0x5596edc08c00 msgr2=0x5596edc17ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.151+0000 7fc6a616a740 1 -- 172.21.15.47:0/3639680303 shutdown_connections 2024-03-21T11:25:29.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.151+0000 7fc6a616a740 1 -- 172.21.15.47:0/3639680303 wait complete. 2024-03-21T11:25:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.151+0000 7fc6a616a740 1 -- start start 2024-03-21T11:25:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.152+0000 7fc6a616a740 1 Processor -- start 2024-03-21T11:25:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.152+0000 7fc6a616a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0d810 0x5596edc1abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.152+0000 7fc6a616a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0e9e0 0x5596edc1b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.152+0000 7fc6a616a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0f3a0 0x5596edc1e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.152+0000 7fc6a616a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c1b1a80 con 0x5596edc0d810 2024-03-21T11:25:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.152+0000 7fc6a616a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c1b1420 con 0x5596edc0f3a0 2024-03-21T11:25:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.152+0000 7fc6a616a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c1ae920 con 0x5596edc0e9e0 2024-03-21T11:25:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.152+0000 7fc6a616a740 10 client.0 _ll_register_callbacks cb 0x7fc6840097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.152+0000 7fc69be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0e9e0 0x5596edc1b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.152+0000 7fc69be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0e9e0 0x5596edc1b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48602/0 (socket says 172.21.15.47:48602) 2024-03-21T11:25:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.152+0000 7fc69be00640 1 -- 172.21.15.47:0/3272809141 learned_addr learned my addr 172.21.15.47:0/3272809141 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.152+0000 7fc6a1400640 1 --2- 172.21.15.47:0/3272809141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0d810 0x5596edc1abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.153+0000 7fc6a1400640 1 --2- 172.21.15.47:0/3272809141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0f3a0 0x5596edc1e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.153+0000 7fc69be00640 1 -- 172.21.15.47:0/3272809141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0f3a0 msgr2=0x5596edc1e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.153+0000 7fc69be00640 1 --2- 172.21.15.47:0/3272809141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0f3a0 0x5596edc1e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.153+0000 7fc69be00640 1 -- 172.21.15.47:0/3272809141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0d810 msgr2=0x5596edc1abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.153+0000 7fc69be00640 1 --2- 172.21.15.47:0/3272809141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0d810 0x5596edc1abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.153+0000 7fc69be00640 1 -- 172.21.15.47:0/3272809141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596edc1e6e0 con 0x5596edc0e9e0 2024-03-21T11:25:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.153+0000 7fc6a1400640 1 --2- 172.21.15.47:0/3272809141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0f3a0 0x5596edc1e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:25:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.153+0000 7fc69be00640 1 --2- 172.21.15.47:0/3272809141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0e9e0 0x5596edc1b120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc684000c00 tx=0x7fc684011920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=530ebed20de9fa7f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.153+0000 7fc68a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:25:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.153+0000 7fc68a000640 1 -- 172.21.15.47:0/3272809141 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68401cb80 con 0x5596edc0e9e0 2024-03-21T11:25:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.153+0000 7fc6a616a740 1 -- 172.21.15.47:0/3272809141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc68c009e00 con 0x5596edc0e9e0 2024-03-21T11:25:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.153+0000 7fc68a000640 1 -- 172.21.15.47:0/3272809141 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc68401cd20 con 0x5596edc0e9e0 2024-03-21T11:25:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.154+0000 7fc68a000640 1 -- 172.21.15.47:0/3272809141 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc684038050 con 0x5596edc0e9e0 2024-03-21T11:25:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.154+0000 7fc6a616a740 10 client.26107 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.154+0000 7fc6a616a740 1 -- 172.21.15.47:0/3272809141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5596edc7f390 con 0x5596edc0e9e0 2024-03-21T11:25:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.154+0000 7fc68a000640 1 -- 172.21.15.47:0/3272809141 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc68c009e00 con 0x5596edc0e9e0 2024-03-21T11:25:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.154+0000 7fc68a000640 1 -- 172.21.15.47:0/3272809141 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc68401a070 con 0x5596edc0e9e0 2024-03-21T11:25:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.154+0000 7fc6a616a740 10 client.26107 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.154+0000 7fc6a616a740 10 client.26107 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:25:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.154+0000 7fc6a616a740 20 client.26107 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.154+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3272809141 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5596edc7fd60 0x5596edc82180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.154+0000 7fc6a616a740 4 client.26107 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:25:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.154+0000 7fc6a616a740 1 -- 172.21.15.47:0/3272809141 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5596edb79b50 con 0x5596edc7fd60 2024-03-21T11:25:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.155+0000 7fc6a1400640 1 --2- 172.21.15.47:0/3272809141 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5596edc7fd60 0x5596edc82180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.155+0000 7fc6a1400640 1 --2- 172.21.15.47:0/3272809141 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5596edc7fd60 0x5596edc82180 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b57c55dd160920f server_cookie=5e7bfaf641441645 in_seq=0 out_seq=0 2024-03-21T11:25:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.155+0000 7fc68a000640 10 client.26107 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:25:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.155+0000 7fc68a000640 1 -- 172.21.15.47:0/3272809141 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5596edb79b50 con 0x5596edc7fd60 2024-03-21T11:25:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.155+0000 7fc68a000640 10 client.26107 handle_command_reply: tid=0 2024-03-21T11:25:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.155+0000 7fc6a616a740 1 client.26107 shutdown 2024-03-21T11:25:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.156+0000 7fc6a616a740 1 -- 172.21.15.47:0/3272809141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0e9e0 msgr2=0x5596edc1b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.156+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3272809141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0e9e0 0x5596edc1b120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc684000c00 tx=0x7fc684011920 comp rx=0 tx=0).stop 2024-03-21T11:25:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.156+0000 7fc6a616a740 1 -- 172.21.15.47:0/3272809141 shutdown_connections 2024-03-21T11:25:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.156+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3272809141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596edc0f3a0 0x5596edc1e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.156+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3272809141 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5596edc7fd60 0x5596edc82180 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.156+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3272809141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5596edc0e9e0 0x5596edc1b120 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.156+0000 7fc6a616a740 1 --2- 172.21.15.47:0/3272809141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5596edc0d810 0x5596edc1abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.156+0000 7fc68a000640 0 client.26107 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:25:29.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.156+0000 7fc6a616a740 1 -- 172.21.15.47:0/3272809141 >> 172.21.15.47:0/3272809141 conn(0x5596edc08c00 msgr2=0x5596edc07620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.156+0000 7fc6a616a740 1 -- 172.21.15.47:0/3272809141 shutdown_connections 2024-03-21T11:25:29.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.156+0000 7fc6a616a740 1 -- 172.21.15.47:0/3272809141 wait complete. 2024-03-21T11:25:29.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.156+0000 7fc6a616a740 20 client.26107 trim_cache size 0 max 16384 2024-03-21T11:25:29.162 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:25:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.157+0000 7fc6a3400640 1 -- 172.21.15.47:0/3477739847 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc67c0a2f20 msgr2=0x7fc67c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.157+0000 7fc6a3400640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc67c0a2f20 0x7fc67c0a53e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc6940ba740 tx=0x7fc694009500 comp rx=0 tx=0).stop 2024-03-21T11:25:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.157+0000 7fc6a3400640 1 -- 172.21.15.47:0/3477739847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc69c076060 msgr2=0x7fc69c19e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.157+0000 7fc6a3400640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc69c076060 0x7fc69c19e070 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc694000f80 tx=0x7fc6940337b0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.157+0000 7fc6a3400640 1 -- 172.21.15.47:0/3477739847 shutdown_connections 2024-03-21T11:25:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.157+0000 7fc6a3400640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc67c0a2f20 0x7fc67c0a53e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.157+0000 7fc6a3400640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc69c077c40 0x7fc69c1a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.157+0000 7fc6a3400640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc69c0772a0 0x7fc69c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.157+0000 7fc6a3400640 1 --2- 172.21.15.47:0/3477739847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc69c076060 0x7fc69c19e070 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.157+0000 7fc6a3400640 1 -- 172.21.15.47:0/3477739847 >> 172.21.15.47:0/3477739847 conn(0x7fc69c0fee50 msgr2=0x7fc69c100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.157+0000 7fc6a3400640 1 -- 172.21.15.47:0/3477739847 shutdown_connections 2024-03-21T11:25:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.157+0000 7fc6a3400640 1 -- 172.21.15.47:0/3477739847 wait complete. 2024-03-21T11:25:29.176 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:25:29.176 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:25:29.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.395+0000 7f507aa00640 1 -- 172.21.15.47:0/1757879044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074108eb0 msgr2=0x7f5074109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.395+0000 7f507aa00640 1 --2- 172.21.15.47:0/1757879044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074108eb0 0x7f5074109330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5060009900 tx=0x7f506002bc30 comp rx=0 tx=0).stop 2024-03-21T11:25:29.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.395+0000 7f507aa00640 1 -- 172.21.15.47:0/1757879044 shutdown_connections 2024-03-21T11:25:29.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.395+0000 7f507aa00640 1 --2- 172.21.15.47:0/1757879044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5074109870 0x7f5074110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.395+0000 7f507aa00640 1 --2- 172.21.15.47:0/1757879044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074108eb0 0x7f5074109330 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.395+0000 7f507aa00640 1 --2- 172.21.15.47:0/1757879044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074107cc0 0x7f50741080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.395+0000 7f507aa00640 1 -- 172.21.15.47:0/1757879044 >> 172.21.15.47:0/1757879044 conn(0x7f5074076c60 msgr2=0x7f50740790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.395+0000 7f507aa00640 1 -- 172.21.15.47:0/1757879044 shutdown_connections 2024-03-21T11:25:29.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.396+0000 7f507aa00640 1 -- 172.21.15.47:0/1757879044 wait complete. 2024-03-21T11:25:29.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.397+0000 7f507aa00640 1 Processor -- start 2024-03-21T11:25:29.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.397+0000 7f507aa00640 1 -- start start 2024-03-21T11:25:29.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.397+0000 7f507aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074107cc0 0x7f507419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.397+0000 7f507aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074108eb0 0x7f507419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.397+0000 7f507aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5074109870 0x7f50741a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.397+0000 7f507aa00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5074113920 con 0x7f5074108eb0 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.397+0000 7f507aa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50741137a0 con 0x7f5074107cc0 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.397+0000 7f507aa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5074113aa0 con 0x7f5074109870 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.397+0000 7f5073e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074107cc0 0x7f507419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.397+0000 7f5073e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074107cc0 0x7f507419dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42562/0 (socket says 172.21.15.47:42562) 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.397+0000 7f5073400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074108eb0 0x7f507419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.397+0000 7f5073e00640 1 -- 172.21.15.47:0/2383402876 learned_addr learned my addr 172.21.15.47:0/2383402876 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.397+0000 7f5078a00640 1 --2- 172.21.15.47:0/2383402876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5074109870 0x7f50741a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.398+0000 7f5073e00640 1 -- 172.21.15.47:0/2383402876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5074109870 msgr2=0x7f50741a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.398+0000 7f5073e00640 1 --2- 172.21.15.47:0/2383402876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5074109870 0x7f50741a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.398+0000 7f5073e00640 1 -- 172.21.15.47:0/2383402876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074108eb0 msgr2=0x7f507419e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.398+0000 7f5073e00640 1 --2- 172.21.15.47:0/2383402876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074108eb0 0x7f507419e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.398+0000 7f5073e00640 1 -- 172.21.15.47:0/2383402876 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50741a5be0 con 0x7f5074107cc0 2024-03-21T11:25:29.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.398+0000 7f5073e00640 1 --2- 172.21.15.47:0/2383402876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074107cc0 0x7f507419dfc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f506400e9c0 tx=0x7f506400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.398+0000 7f5070c00640 1 -- 172.21.15.47:0/2383402876 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506400cd80 con 0x7f5074107cc0 2024-03-21T11:25:29.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.398+0000 7f5070c00640 1 -- 172.21.15.47:0/2383402876 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50640035f0 con 0x7f5074107cc0 2024-03-21T11:25:29.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.398+0000 7f5070c00640 1 -- 172.21.15.47:0/2383402876 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064010690 con 0x7f5074107cc0 2024-03-21T11:25:29.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.399+0000 7f507aa00640 1 -- 172.21.15.47:0/2383402876 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50741a5ed0 con 0x7f5074107cc0 2024-03-21T11:25:29.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.399+0000 7f507aa00640 1 -- 172.21.15.47:0/2383402876 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50741a62e0 con 0x7f5074107cc0 2024-03-21T11:25:29.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.402+0000 7f507d679740 1 Processor -- start 2024-03-21T11:25:29.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.402+0000 7f507d679740 1 -- start start 2024-03-21T11:25:29.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.402+0000 7f507d679740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a416fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.402+0000 7f507d679740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fc9e0 0x561a416fce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.402+0000 7f507d679740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fd3a0 0x561a41703c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.402+0000 7f507d679740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a416602c0 con 0x561a416fc9e0 2024-03-21T11:25:29.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.402+0000 7f507d679740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a41608c90 con 0x561a416fd3a0 2024-03-21T11:25:29.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.402+0000 7f507d679740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a4165c360 con 0x561a416fb810 2024-03-21T11:25:29.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.402+0000 7f5070c00640 1 -- 172.21.15.47:0/2383402876 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f50640037a0 con 0x7f5074107cc0 2024-03-21T11:25:29.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.403+0000 7f5078a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a416fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.403+0000 7f5078a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a416fbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48634/0 (socket says 172.21.15.47:48634) 2024-03-21T11:25:29.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.403+0000 7f5078a00640 1 -- 172.21.15.47:0/1096463023 learned_addr learned my addr 172.21.15.47:0/1096463023 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.403+0000 7f5073400640 1 --2- 172.21.15.47:0/1096463023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fd3a0 0x561a41703c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.403+0000 7f5078a00640 1 -- 172.21.15.47:0/1096463023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fd3a0 msgr2=0x561a41703c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.403+0000 7f5078a00640 1 --2- 172.21.15.47:0/1096463023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fd3a0 0x561a41703c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.403+0000 7f5078a00640 1 -- 172.21.15.47:0/1096463023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fc9e0 msgr2=0x561a416fce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.403+0000 7f5078a00640 1 --2- 172.21.15.47:0/1096463023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fc9e0 0x561a416fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.403+0000 7f5078a00640 1 -- 172.21.15.47:0/1096463023 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a4162d080 con 0x561a416fb810 2024-03-21T11:25:29.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.403+0000 7f5078a00640 1 --2- 172.21.15.47:0/1096463023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a416fbc30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f506c009fd0 tx=0x7f506c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af0a8f690072f017 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.404+0000 7f5069200640 1 -- 172.21.15.47:0/1096463023 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c004d20 con 0x561a416fb810 2024-03-21T11:25:29.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.404+0000 7f5069200640 1 -- 172.21.15.47:0/1096463023 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f506c004050 con 0x561a416fb810 2024-03-21T11:25:29.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.404+0000 7f5069200640 1 -- 172.21.15.47:0/1096463023 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c005260 con 0x561a416fb810 2024-03-21T11:25:29.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.404+0000 7f5070c00640 1 --2- 172.21.15.47:0/2383402876 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50580a2f20 0x7f50580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.404+0000 7f507d679740 1 -- 172.21.15.47:0/1096463023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 msgr2=0x561a416fbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.404+0000 7f507d679740 1 --2- 172.21.15.47:0/1096463023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a416fbc30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f506c009fd0 tx=0x7f506c0118d0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.404+0000 7f5070c00640 1 -- 172.21.15.47:0/2383402876 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f50640cfb50 con 0x7f5074107cc0 2024-03-21T11:25:29.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.404+0000 7f5073e00640 1 --2- 172.21.15.47:0/2383402876 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50580a2f20 0x7f50580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.404+0000 7f507d679740 1 -- 172.21.15.47:0/1096463023 shutdown_connections 2024-03-21T11:25:29.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.404+0000 7f507d679740 1 --2- 172.21.15.47:0/1096463023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fd3a0 0x561a41703c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.404+0000 7f507d679740 1 --2- 172.21.15.47:0/1096463023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fc9e0 0x561a416fce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.404+0000 7f507d679740 1 --2- 172.21.15.47:0/1096463023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a416fbc30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.404+0000 7f507d679740 1 -- 172.21.15.47:0/1096463023 >> 172.21.15.47:0/1096463023 conn(0x561a416f6c00 msgr2=0x561a416f7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.405+0000 7f507d679740 1 -- 172.21.15.47:0/1096463023 shutdown_connections 2024-03-21T11:25:29.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.405+0000 7f507d679740 1 -- 172.21.15.47:0/1096463023 wait complete. 2024-03-21T11:25:29.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.405+0000 7f507d679740 1 -- start start 2024-03-21T11:25:29.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.405+0000 7f5073e00640 1 --2- 172.21.15.47:0/2383402876 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50580a2f20 0x7f50580a53e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f506409be80 tx=0x7f5064097ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.405+0000 7f507d679740 1 Processor -- start 2024-03-21T11:25:29.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.406+0000 7f507d679740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a417059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.406+0000 7f507d679740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fc9e0 0x561a41705f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.406+0000 7f507d679740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fd3a0 0x561a4170cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.406+0000 7f507d679740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a417043b0 con 0x561a416fd3a0 2024-03-21T11:25:29.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.406+0000 7f507d679740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a4161c0d0 con 0x561a416fc9e0 2024-03-21T11:25:29.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.406+0000 7f507d679740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a41704530 con 0x561a416fb810 2024-03-21T11:25:29.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.406+0000 7f507d679740 10 client.0 _ll_register_callbacks cb 0x561a41620e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:29.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.406+0000 7f5078a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a417059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.406+0000 7f5078a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a417059f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48642/0 (socket says 172.21.15.47:48642) 2024-03-21T11:25:29.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.406+0000 7f5078a00640 1 -- 172.21.15.47:0/3129305840 learned_addr learned my addr 172.21.15.47:0/3129305840 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.406+0000 7f5073400640 1 --2- 172.21.15.47:0/3129305840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fc9e0 0x561a41705f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.406+0000 7f5078a00640 1 --2- 172.21.15.47:0/3129305840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fd3a0 0x561a4170cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.407+0000 7f5078a00640 1 -- 172.21.15.47:0/3129305840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fc9e0 msgr2=0x561a41705f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.407+0000 7f5078a00640 1 --2- 172.21.15.47:0/3129305840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fc9e0 0x561a41705f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.407+0000 7f5078a00640 1 -- 172.21.15.47:0/3129305840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fd3a0 msgr2=0x561a4170cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.407+0000 7f5078a00640 1 --2- 172.21.15.47:0/3129305840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fd3a0 0x561a4170cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.407+0000 7f5078a00640 1 -- 172.21.15.47:0/3129305840 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a413743f0 con 0x561a416fb810 2024-03-21T11:25:29.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.407+0000 7f5073400640 1 --2- 172.21.15.47:0/3129305840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fc9e0 0x561a41705f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:29.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.407+0000 7f5078a00640 1 --2- 172.21.15.47:0/3129305840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a417059f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f506c009de0 tx=0x7f506c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26ecd21beff9bb52 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.407+0000 7f505d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:25:29.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.407+0000 7f505d600640 1 -- 172.21.15.47:0/3129305840 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c00f4e0 con 0x561a416fb810 2024-03-21T11:25:29.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.407+0000 7f505d600640 1 -- 172.21.15.47:0/3129305840 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f506c00f680 con 0x561a416fb810 2024-03-21T11:25:29.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.407+0000 7f507d679740 1 -- 172.21.15.47:0/3129305840 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f506c004050 con 0x561a416fb810 2024-03-21T11:25:29.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.408+0000 7f505d600640 1 -- 172.21.15.47:0/3129305840 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f506c036050 con 0x561a416fb810 2024-03-21T11:25:29.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.408+0000 7f507d679740 10 client.26119 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:29.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.408+0000 7f507d679740 1 -- 172.21.15.47:0/3129305840 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561a4143e7d0 con 0x561a416fb810 2024-03-21T11:25:29.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.408+0000 7f505d600640 1 -- 172.21.15.47:0/3129305840 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f506c004050 con 0x561a416fb810 2024-03-21T11:25:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.408+0000 7f505d600640 1 -- 172.21.15.47:0/3129305840 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f506c025040 con 0x561a416fb810 2024-03-21T11:25:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.408+0000 7f507d679740 10 client.26119 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.408+0000 7f507d679740 10 client.26119 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:25:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.408+0000 7f507d679740 20 client.26119 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.408+0000 7f507d679740 1 --2- 172.21.15.47:0/3129305840 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x561a41711280 0x561a41713670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.408+0000 7f507d679740 4 client.26119 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:25:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.408+0000 7f507d679740 1 -- 172.21.15.47:0/3129305840 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a4166d800 con 0x561a41711280 2024-03-21T11:25:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.409+0000 7f5073400640 1 --2- 172.21.15.47:0/3129305840 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x561a41711280 0x561a41713670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.409+0000 7f5073400640 1 --2- 172.21.15.47:0/3129305840 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x561a41711280 0x561a41713670 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f25b0721a2f9feea server_cookie=51f0e45bf47e713e in_seq=0 out_seq=0 2024-03-21T11:25:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.409+0000 7f505d600640 10 client.26119 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:25:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.410+0000 7f505d600640 1 -- 172.21.15.47:0/3129305840 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561a4166d800 con 0x561a41711280 2024-03-21T11:25:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.410+0000 7f505d600640 10 client.26119 handle_command_reply: tid=0 2024-03-21T11:25:29.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.410+0000 7f507d679740 1 client.26119 shutdown 2024-03-21T11:25:29.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.410+0000 7f507d679740 1 -- 172.21.15.47:0/3129305840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 msgr2=0x561a417059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.410+0000 7f507d679740 1 --2- 172.21.15.47:0/3129305840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a417059f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f506c009de0 tx=0x7f506c000f40 comp rx=0 tx=0).stop 2024-03-21T11:25:29.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.410+0000 7f507d679740 1 -- 172.21.15.47:0/3129305840 shutdown_connections 2024-03-21T11:25:29.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.410+0000 7f507d679740 1 --2- 172.21.15.47:0/3129305840 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x561a41711280 0x561a41713670 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.410+0000 7f507d679740 1 --2- 172.21.15.47:0/3129305840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fd3a0 0x561a4170cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.410+0000 7f507d679740 1 --2- 172.21.15.47:0/3129305840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fc9e0 0x561a41705f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.410+0000 7f507d679740 1 --2- 172.21.15.47:0/3129305840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a417059f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.410+0000 7f507d679740 1 -- 172.21.15.47:0/3129305840 >> 172.21.15.47:0/3129305840 conn(0x561a416f6c00 msgr2=0x561a416f5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.410+0000 7f505d600640 0 client.26119 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:25:29.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.410+0000 7f507d679740 1 -- 172.21.15.47:0/3129305840 shutdown_connections 2024-03-21T11:25:29.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.410+0000 7f507d679740 1 -- 172.21.15.47:0/3129305840 wait complete. 2024-03-21T11:25:29.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.411+0000 7f507d679740 20 client.26119 trim_cache size 0 max 16384 2024-03-21T11:25:29.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.424+0000 7f507d679740 1 Processor -- start 2024-03-21T11:25:29.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.424+0000 7f507d679740 1 -- start start 2024-03-21T11:25:29.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f507d679740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a41717f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f507d679740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fc9e0 0x561a4170cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f507d679740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fd3a0 0x561a41718450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f507d679740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50741b0790 con 0x561a416fd3a0 2024-03-21T11:25:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f507d679740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50741b2110 con 0x561a416fc9e0 2024-03-21T11:25:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f507d679740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50741b0df0 con 0x561a416fb810 2024-03-21T11:25:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f5073400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fc9e0 0x561a4170cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f5078a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a41717f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f5073400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fc9e0 0x561a4170cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42608/0 (socket says 172.21.15.47:42608) 2024-03-21T11:25:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f5073400640 1 -- 172.21.15.47:0/4179791667 learned_addr learned my addr 172.21.15.47:0/4179791667 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f5078a00640 1 --2- 172.21.15.47:0/4179791667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fd3a0 0x561a41718450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f5073400640 1 -- 172.21.15.47:0/4179791667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 msgr2=0x561a41717f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f5073400640 1 --2- 172.21.15.47:0/4179791667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a41717f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f5073400640 1 -- 172.21.15.47:0/4179791667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fd3a0 msgr2=0x561a41718450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f5073400640 1 --2- 172.21.15.47:0/4179791667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fd3a0 0x561a41718450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.425+0000 7f5073400640 1 -- 172.21.15.47:0/4179791667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a416f1d40 con 0x561a416fc9e0 2024-03-21T11:25:29.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.426+0000 7f5073400640 1 --2- 172.21.15.47:0/4179791667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fc9e0 0x561a4170cf30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5060009850 tx=0x7f5060005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf2fb0852b202864 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.426+0000 7f505fe00640 1 -- 172.21.15.47:0/4179791667 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060004240 con 0x561a416fc9e0 2024-03-21T11:25:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.426+0000 7f505fe00640 1 -- 172.21.15.47:0/4179791667 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f506004b070 con 0x561a416fc9e0 2024-03-21T11:25:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.426+0000 7f505fe00640 1 -- 172.21.15.47:0/4179791667 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060004920 con 0x561a416fc9e0 2024-03-21T11:25:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.427+0000 7f507d679740 1 -- 172.21.15.47:0/4179791667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fc9e0 msgr2=0x561a4170cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.427+0000 7f507d679740 1 --2- 172.21.15.47:0/4179791667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fc9e0 0x561a4170cf30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5060009850 tx=0x7f5060005fd0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.427+0000 7f507d679740 1 -- 172.21.15.47:0/4179791667 shutdown_connections 2024-03-21T11:25:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.427+0000 7f507d679740 1 --2- 172.21.15.47:0/4179791667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fd3a0 0x561a41718450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.427+0000 7f507d679740 1 --2- 172.21.15.47:0/4179791667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fc9e0 0x561a4170cf30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.427+0000 7f507d679740 1 --2- 172.21.15.47:0/4179791667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fb810 0x561a41717f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.427+0000 7f507d679740 1 -- 172.21.15.47:0/4179791667 >> 172.21.15.47:0/4179791667 conn(0x561a416f6c00 msgr2=0x561a416fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.427+0000 7f507d679740 1 -- 172.21.15.47:0/4179791667 shutdown_connections 2024-03-21T11:25:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.427+0000 7f507d679740 1 -- 172.21.15.47:0/4179791667 wait complete. 2024-03-21T11:25:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.427+0000 7f507d679740 1 -- start start 2024-03-21T11:25:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.427+0000 7f507d679740 1 Processor -- start 2024-03-21T11:25:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f507d679740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fb810 0x561a4170a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f507d679740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fc9e0 0x561a4170ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f507d679740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fd3a0 0x561a4170be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f507d679740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50741b1ab0 con 0x561a416fc9e0 2024-03-21T11:25:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f507d679740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50741b1450 con 0x561a416fb810 2024-03-21T11:25:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f507d679740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50741ae950 con 0x561a416fd3a0 2024-03-21T11:25:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f507d679740 10 client.0 _ll_register_callbacks cb 0x7f506c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f5078a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fb810 0x561a4170a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f5078a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fb810 0x561a4170a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42614/0 (socket says 172.21.15.47:42614) 2024-03-21T11:25:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f5078a00640 1 -- 172.21.15.47:0/395966284 learned_addr learned my addr 172.21.15.47:0/395966284 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f5078a00640 1 --2- 172.21.15.47:0/395966284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fd3a0 0x561a4170be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f5073400640 1 --2- 172.21.15.47:0/395966284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fc9e0 0x561a4170ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f5078a00640 1 -- 172.21.15.47:0/395966284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fd3a0 msgr2=0x561a4170be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f5078a00640 1 --2- 172.21.15.47:0/395966284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fd3a0 0x561a4170be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f5078a00640 1 -- 172.21.15.47:0/395966284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fc9e0 msgr2=0x561a4170ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.428+0000 7f5078a00640 1 --2- 172.21.15.47:0/395966284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fc9e0 0x561a4170ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.429+0000 7f5078a00640 1 -- 172.21.15.47:0/395966284 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a4170c420 con 0x561a416fb810 2024-03-21T11:25:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.429+0000 7f5078a00640 1 --2- 172.21.15.47:0/395966284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fb810 0x561a4170a8e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f506c002230 tx=0x7f506c004ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f7737b699fb2f04 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.429+0000 7f505d600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:25:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.429+0000 7f505d600640 1 -- 172.21.15.47:0/395966284 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c0047b0 con 0x561a416fb810 2024-03-21T11:25:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.429+0000 7f507d679740 1 -- 172.21.15.47:0/395966284 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f506004b070 con 0x561a416fb810 2024-03-21T11:25:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.429+0000 7f505d600640 1 -- 172.21.15.47:0/395966284 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f506c028500 con 0x561a416fb810 2024-03-21T11:25:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.429+0000 7f505d600640 1 -- 172.21.15.47:0/395966284 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f506c028700 con 0x561a416fb810 2024-03-21T11:25:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.429+0000 7f507d679740 10 client.26114 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.429+0000 7f507d679740 1 -- 172.21.15.47:0/395966284 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561a4176cd90 con 0x561a416fb810 2024-03-21T11:25:29.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.430+0000 7f505d600640 1 -- 172.21.15.47:0/395966284 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f506004b070 con 0x561a416fb810 2024-03-21T11:25:29.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.430+0000 7f505d600640 1 -- 172.21.15.47:0/395966284 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f506c03a530 con 0x561a416fb810 2024-03-21T11:25:29.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.430+0000 7f507d679740 10 client.26114 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:29.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.430+0000 7f507d679740 10 client.26114 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:25:29.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.430+0000 7f507d679740 20 client.26114 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:29.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.430+0000 7f507d679740 1 --2- 172.21.15.47:0/395966284 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x561a4176d480 0x561a4176f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.430+0000 7f507d679740 4 client.26114 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:25:29.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.430+0000 7f507d679740 1 -- 172.21.15.47:0/395966284 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561a41667b50 con 0x561a4176d480 2024-03-21T11:25:29.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.430+0000 7f5073400640 1 --2- 172.21.15.47:0/395966284 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x561a4176d480 0x561a4176f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f5073400640 1 --2- 172.21.15.47:0/395966284 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x561a4176d480 0x561a4176f8a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90acae0a2efbccd5 server_cookie=bbcfc019995eaa0 in_seq=0 out_seq=0 2024-03-21T11:25:29.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f505d600640 10 client.26114 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:25:29.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f505d600640 1 -- 172.21.15.47:0/395966284 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561a41667b50 con 0x561a4176d480 2024-03-21T11:25:29.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f505d600640 10 client.26114 handle_command_reply: tid=0 2024-03-21T11:25:29.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f507d679740 1 client.26114 shutdown 2024-03-21T11:25:29.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f507d679740 1 -- 172.21.15.47:0/395966284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fb810 msgr2=0x561a4170a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f507d679740 1 --2- 172.21.15.47:0/395966284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fb810 0x561a4170a8e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f506c002230 tx=0x7f506c004ac0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f507d679740 1 -- 172.21.15.47:0/395966284 shutdown_connections 2024-03-21T11:25:29.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f507d679740 1 --2- 172.21.15.47:0/395966284 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x561a4176d480 0x561a4176f8a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f507d679740 1 --2- 172.21.15.47:0/395966284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561a416fd3a0 0x561a4170be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f507d679740 1 --2- 172.21.15.47:0/395966284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a416fc9e0 0x561a4170ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f507d679740 1 --2- 172.21.15.47:0/395966284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a416fb810 0x561a4170a8e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f505d600640 0 client.26114 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:25:29.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f507d679740 1 -- 172.21.15.47:0/395966284 >> 172.21.15.47:0/395966284 conn(0x561a416f6c00 msgr2=0x561a416ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f507d679740 1 -- 172.21.15.47:0/395966284 shutdown_connections 2024-03-21T11:25:29.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.431+0000 7f507d679740 1 -- 172.21.15.47:0/395966284 wait complete. 2024-03-21T11:25:29.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.432+0000 7f507d679740 20 client.26114 trim_cache size 0 max 16384 2024-03-21T11:25:29.438 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:25:29.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.432+0000 7f507aa00640 1 -- 172.21.15.47:0/2383402876 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50580a2f20 msgr2=0x7f50580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.432+0000 7f507aa00640 1 --2- 172.21.15.47:0/2383402876 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50580a2f20 0x7f50580a53e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f506409be80 tx=0x7f5064097ab0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.432+0000 7f507aa00640 1 -- 172.21.15.47:0/2383402876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074107cc0 msgr2=0x7f507419dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.432+0000 7f507aa00640 1 --2- 172.21.15.47:0/2383402876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074107cc0 0x7f507419dfc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f506400e9c0 tx=0x7f506400ee80 comp rx=0 tx=0).stop 2024-03-21T11:25:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.432+0000 7f507aa00640 1 -- 172.21.15.47:0/2383402876 shutdown_connections 2024-03-21T11:25:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.432+0000 7f507aa00640 1 --2- 172.21.15.47:0/2383402876 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50580a2f20 0x7f50580a53e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.432+0000 7f507aa00640 1 --2- 172.21.15.47:0/2383402876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5074109870 0x7f50741a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.432+0000 7f507aa00640 1 --2- 172.21.15.47:0/2383402876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5074108eb0 0x7f507419e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.432+0000 7f507aa00640 1 --2- 172.21.15.47:0/2383402876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074107cc0 0x7f507419dfc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.432+0000 7f507aa00640 1 -- 172.21.15.47:0/2383402876 >> 172.21.15.47:0/2383402876 conn(0x7f5074076c60 msgr2=0x7f507410c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.432+0000 7f507aa00640 1 -- 172.21.15.47:0/2383402876 shutdown_connections 2024-03-21T11:25:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.432+0000 7f507aa00640 1 -- 172.21.15.47:0/2383402876 wait complete. 2024-03-21T11:25:29.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:29 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/963729770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:25:29.452 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:25:29.452 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:25:29.486 INFO:tasks.workunit.client.0.smithi047.stdout: 1 23907 11.05 MB/sec warmup 13 sec latency 8.703 ms 2024-03-21T11:25:29.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:29 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/963729770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:25:29.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.662+0000 7f32dea00640 1 -- 172.21.15.47:0/3631644611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d81015a0 msgr2=0x7f32d81019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.662+0000 7f32dea00640 1 --2- 172.21.15.47:0/3631644611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d81015a0 0x7f32d81019c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009900 tx=0x7f32cc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:25:29.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.662+0000 7f32dea00640 1 -- 172.21.15.47:0/3631644611 shutdown_connections 2024-03-21T11:25:29.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.662+0000 7f32dea00640 1 --2- 172.21.15.47:0/3631644611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32d8103150 0x7f32d81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.662+0000 7f32dea00640 1 --2- 172.21.15.47:0/3631644611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d8102790 0x7f32d8102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.662+0000 7f32dea00640 1 --2- 172.21.15.47:0/3631644611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d81015a0 0x7f32d81019c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.662+0000 7f32dea00640 1 -- 172.21.15.47:0/3631644611 >> 172.21.15.47:0/3631644611 conn(0x7f32d80fcb10 msgr2=0x7f32d80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.662+0000 7f32dea00640 1 -- 172.21.15.47:0/3631644611 shutdown_connections 2024-03-21T11:25:29.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.662+0000 7f32dea00640 1 -- 172.21.15.47:0/3631644611 wait complete. 2024-03-21T11:25:29.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.663+0000 7f32dea00640 1 Processor -- start 2024-03-21T11:25:29.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.663+0000 7f32dea00640 1 -- start start 2024-03-21T11:25:29.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32d81015a0 0x7f32d8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d8102790 0x7f32d819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d8103150 0x7f32d81a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dea00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d810d240 con 0x7f32d8103150 2024-03-21T11:25:29.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d810d0c0 con 0x7f32d8102790 2024-03-21T11:25:29.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d810d3c0 con 0x7f32d81015a0 2024-03-21T11:25:29.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dd600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32d81015a0 0x7f32d8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dd600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32d81015a0 0x7f32d8199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48680/0 (socket says 172.21.15.47:48680) 2024-03-21T11:25:29.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dd600640 1 -- 172.21.15.47:0/4040061997 learned_addr learned my addr 172.21.15.47:0/4040061997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dcc00640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d8102790 0x7f32d819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32de000640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d8103150 0x7f32d81a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dd600640 1 -- 172.21.15.47:0/4040061997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d8102790 msgr2=0x7f32d819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dd600640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d8102790 0x7f32d819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dd600640 1 -- 172.21.15.47:0/4040061997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d8103150 msgr2=0x7f32d81a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dd600640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d8103150 0x7f32d81a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32dd600640 1 -- 172.21.15.47:0/4040061997 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d81a1710 con 0x7f32d81015a0 2024-03-21T11:25:29.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.664+0000 7f32de000640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d8103150 0x7f32d81a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.665+0000 7f32dd600640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32d81015a0 0x7f32d8199af0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f32cc000c00 tx=0x7f32cc02c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.665+0000 7f32d6000640 1 -- 172.21.15.47:0/4040061997 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc043020 con 0x7f32d81015a0 2024-03-21T11:25:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.665+0000 7f32d6000640 1 -- 172.21.15.47:0/4040061997 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32cc0054e0 con 0x7f32d81015a0 2024-03-21T11:25:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.665+0000 7f32dea00640 1 -- 172.21.15.47:0/4040061997 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d81a19a0 con 0x7f32d81015a0 2024-03-21T11:25:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.665+0000 7f32d6000640 1 -- 172.21.15.47:0/4040061997 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc003ce0 con 0x7f32d81015a0 2024-03-21T11:25:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.666+0000 7f32dea00640 1 -- 172.21.15.47:0/4040061997 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d81a1ec0 con 0x7f32d81015a0 2024-03-21T11:25:29.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.669+0000 7f32d6000640 1 -- 172.21.15.47:0/4040061997 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f32cc046020 con 0x7f32d81015a0 2024-03-21T11:25:29.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.669+0000 7f32e2dba740 1 Processor -- start 2024-03-21T11:25:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.669+0000 7f32e2dba740 1 -- start start 2024-03-21T11:25:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32e2dba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b09f0 0x5635a67b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32e2dba740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32e2dba740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b2580 0x5635a67b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32e2dba740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a67152c0 con 0x5635a67b2580 2024-03-21T11:25:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32e2dba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a66bdc90 con 0x5635a67b09f0 2024-03-21T11:25:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32e2dba740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a6711360 con 0x5635a67b1bc0 2024-03-21T11:25:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32de000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b09f0 0x5635a67b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32dd600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32de000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b09f0 0x5635a67b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42638/0 (socket says 172.21.15.47:42638) 2024-03-21T11:25:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32de000640 1 -- 172.21.15.47:0/4205478310 learned_addr learned my addr 172.21.15.47:0/4205478310 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32dcc00640 1 --2- 172.21.15.47:0/4205478310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b2580 0x5635a67b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32de000640 1 -- 172.21.15.47:0/4205478310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 msgr2=0x5635a67b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32de000640 1 --2- 172.21.15.47:0/4205478310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32de000640 1 -- 172.21.15.47:0/4205478310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b2580 msgr2=0x5635a67b8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32de000640 1 --2- 172.21.15.47:0/4205478310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b2580 0x5635a67b8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.670+0000 7f32de000640 1 -- 172.21.15.47:0/4205478310 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635a66e2080 con 0x5635a67b09f0 2024-03-21T11:25:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.671+0000 7f32de000640 1 --2- 172.21.15.47:0/4205478310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b09f0 0x5635a67b0e10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009fb0 tx=0x7f32d0010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d997cf4a94f87ab8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.671+0000 7f32ca600640 1 -- 172.21.15.47:0/4205478310 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d0016070 con 0x5635a67b09f0 2024-03-21T11:25:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.671+0000 7f32ca600640 1 -- 172.21.15.47:0/4205478310 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32d0012bf0 con 0x5635a67b09f0 2024-03-21T11:25:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.671+0000 7f32ca600640 1 -- 172.21.15.47:0/4205478310 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d001bcc0 con 0x5635a67b09f0 2024-03-21T11:25:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.671+0000 7f32d6000640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32c00a2f20 0x7f32c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.671+0000 7f32d6000640 1 -- 172.21.15.47:0/4040061997 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f32cc0f1990 con 0x7f32d81015a0 2024-03-21T11:25:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.671+0000 7f32dd600640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32c00a2f20 0x7f32c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.672+0000 7f32e2dba740 1 -- 172.21.15.47:0/4205478310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b09f0 msgr2=0x5635a67b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.672+0000 7f32e2dba740 1 --2- 172.21.15.47:0/4205478310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b09f0 0x5635a67b0e10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009fb0 tx=0x7f32d0010800 comp rx=0 tx=0).stop 2024-03-21T11:25:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.672+0000 7f32e2dba740 1 -- 172.21.15.47:0/4205478310 shutdown_connections 2024-03-21T11:25:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.672+0000 7f32e2dba740 1 --2- 172.21.15.47:0/4205478310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b2580 0x5635a67b8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.672+0000 7f32e2dba740 1 --2- 172.21.15.47:0/4205478310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.672+0000 7f32e2dba740 1 --2- 172.21.15.47:0/4205478310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b09f0 0x5635a67b0e10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.672+0000 7f32e2dba740 1 -- 172.21.15.47:0/4205478310 >> 172.21.15.47:0/4205478310 conn(0x5635a67abde0 msgr2=0x5635a67ac1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.672+0000 7f32e2dba740 1 -- 172.21.15.47:0/4205478310 shutdown_connections 2024-03-21T11:25:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.672+0000 7f32dd600640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32c00a2f20 0x7f32c00a53e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0ba6a0 tx=0x7f32cc0f7530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.672+0000 7f32e2dba740 1 -- 172.21.15.47:0/4205478310 wait complete. 2024-03-21T11:25:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.673+0000 7f32e2dba740 1 -- start start 2024-03-21T11:25:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.673+0000 7f32e2dba740 1 Processor -- start 2024-03-21T11:25:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.673+0000 7f32e2dba740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b09f0 0x5635a67bac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.673+0000 7f32e2dba740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b1bc0 0x5635a67bb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.673+0000 7f32e2dba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b2580 0x5635a67c2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.673+0000 7f32e2dba740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a67b9590 con 0x5635a67b1bc0 2024-03-21T11:25:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.673+0000 7f32e2dba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a66d10d0 con 0x5635a67b2580 2024-03-21T11:25:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.673+0000 7f32e2dba740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a67b9710 con 0x5635a67b09f0 2024-03-21T11:25:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.673+0000 7f32e2dba740 10 client.0 _ll_register_callbacks cb 0x5635a66d5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.673+0000 7f32de000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b09f0 0x5635a67bac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.673+0000 7f32de000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b09f0 0x5635a67bac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48706/0 (socket says 172.21.15.47:48706) 2024-03-21T11:25:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.674+0000 7f32dcc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b1bc0 0x5635a67bb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.674+0000 7f32de000640 1 -- 172.21.15.47:0/166846594 learned_addr learned my addr 172.21.15.47:0/166846594 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.674+0000 7f32de000640 1 --2- 172.21.15.47:0/166846594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b2580 0x5635a67c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.674+0000 7f32dcc00640 1 -- 172.21.15.47:0/166846594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b09f0 msgr2=0x5635a67bac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.674+0000 7f32dcc00640 1 --2- 172.21.15.47:0/166846594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b09f0 0x5635a67bac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.674+0000 7f32dcc00640 1 -- 172.21.15.47:0/166846594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b2580 msgr2=0x5635a67c2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.674+0000 7f32dcc00640 1 --2- 172.21.15.47:0/166846594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b2580 0x5635a67c2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.674+0000 7f32dcc00640 1 -- 172.21.15.47:0/166846594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635a64293f0 con 0x5635a67b1bc0 2024-03-21T11:25:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.674+0000 7f32dcc00640 1 --2- 172.21.15.47:0/166846594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b1bc0 0x5635a67bb1d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f32d819b000 tx=0x7f32c4012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f7e9d58a54a21c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.674+0000 7f32bea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:25:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.675+0000 7f32bea00640 1 -- 172.21.15.47:0/166846594 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c4004050 con 0x5635a67b1bc0 2024-03-21T11:25:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.675+0000 7f32bea00640 1 -- 172.21.15.47:0/166846594 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f32c40041f0 con 0x5635a67b1bc0 2024-03-21T11:25:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.675+0000 7f32e2dba740 1 -- 172.21.15.47:0/166846594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32d0012bf0 con 0x5635a67b1bc0 2024-03-21T11:25:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.675+0000 7f32bea00640 1 -- 172.21.15.47:0/166846594 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32c4025050 con 0x5635a67b1bc0 2024-03-21T11:25:29.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.675+0000 7f32e2dba740 10 client.15942 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:29.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.675+0000 7f32e2dba740 1 -- 172.21.15.47:0/166846594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5635a64f37d0 con 0x5635a67b1bc0 2024-03-21T11:25:29.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.675+0000 7f32bea00640 1 -- 172.21.15.47:0/166846594 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32d0012bf0 con 0x5635a67b1bc0 2024-03-21T11:25:29.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.676+0000 7f32bea00640 1 -- 172.21.15.47:0/166846594 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f32c4003a60 con 0x5635a67b1bc0 2024-03-21T11:25:29.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.676+0000 7f32e2dba740 10 client.15942 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:29.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.676+0000 7f32e2dba740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:29.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.676+0000 7f32e2dba740 20 client.15942 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:29.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.676+0000 7f32e2dba740 1 --2- 172.21.15.47:0/166846594 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5635a67c67f0 0x5635a67c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.676+0000 7f32e2dba740 4 client.15942 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:25:29.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.676+0000 7f32e2dba740 1 -- 172.21.15.47:0/166846594 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635a6722800 con 0x5635a67c67f0 2024-03-21T11:25:29.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.676+0000 7f32de000640 1 --2- 172.21.15.47:0/166846594 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5635a67c67f0 0x5635a67c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.676+0000 7f32de000640 1 --2- 172.21.15.47:0/166846594 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5635a67c67f0 0x5635a67c8c10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b101d20a190497a5 server_cookie=6814cfbebc543b92 in_seq=0 out_seq=0 2024-03-21T11:25:29.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.676+0000 7f32bea00640 10 client.15942 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:29.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.677+0000 7f32bea00640 1 -- 172.21.15.47:0/166846594 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5635a6722800 con 0x5635a67c67f0 2024-03-21T11:25:29.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.677+0000 7f32bea00640 10 client.15942 handle_command_reply: tid=0 2024-03-21T11:25:29.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.677+0000 7f32e2dba740 1 client.15942 shutdown 2024-03-21T11:25:29.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.677+0000 7f32e2dba740 1 -- 172.21.15.47:0/166846594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b1bc0 msgr2=0x5635a67bb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.677+0000 7f32e2dba740 1 --2- 172.21.15.47:0/166846594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b1bc0 0x5635a67bb1d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f32d819b000 tx=0x7f32c4012b60 comp rx=0 tx=0).stop 2024-03-21T11:25:29.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.678+0000 7f32e2dba740 1 -- 172.21.15.47:0/166846594 shutdown_connections 2024-03-21T11:25:29.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.678+0000 7f32e2dba740 1 --2- 172.21.15.47:0/166846594 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5635a67c67f0 0x5635a67c8c10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.678+0000 7f32e2dba740 1 --2- 172.21.15.47:0/166846594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b2580 0x5635a67c2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.678+0000 7f32e2dba740 1 --2- 172.21.15.47:0/166846594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b1bc0 0x5635a67bb1d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.678+0000 7f32e2dba740 1 --2- 172.21.15.47:0/166846594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b09f0 0x5635a67bac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.678+0000 7f32bea00640 0 client.15942 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:29.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.678+0000 7f32e2dba740 1 -- 172.21.15.47:0/166846594 >> 172.21.15.47:0/166846594 conn(0x5635a67abde0 msgr2=0x5635a67ab160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.678+0000 7f32e2dba740 1 -- 172.21.15.47:0/166846594 shutdown_connections 2024-03-21T11:25:29.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.678+0000 7f32e2dba740 1 -- 172.21.15.47:0/166846594 wait complete. 2024-03-21T11:25:29.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.678+0000 7f32e2dba740 20 client.15942 trim_cache size 0 max 16384 2024-03-21T11:25:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.693+0000 7f32e2dba740 1 Processor -- start 2024-03-21T11:25:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.693+0000 7f32e2dba740 1 -- start start 2024-03-21T11:25:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.693+0000 7f32e2dba740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b09f0 0x5635a67a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.693+0000 7f32e2dba740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.693+0000 7f32e2dba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b2580 0x5635a67b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.693+0000 7f32e2dba740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d81ac180 con 0x5635a67b09f0 2024-03-21T11:25:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.693+0000 7f32e2dba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d81adb00 con 0x5635a67b2580 2024-03-21T11:25:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.693+0000 7f32e2dba740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d81ac7e0 con 0x5635a67b1bc0 2024-03-21T11:25:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.693+0000 7f32de000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b09f0 0x5635a67a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.693+0000 7f32dcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.694+0000 7f32de000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b09f0 0x5635a67a6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54898/0 (socket says 172.21.15.47:54898) 2024-03-21T11:25:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.694+0000 7f32de000640 1 -- 172.21.15.47:0/3422325955 learned_addr learned my addr 172.21.15.47:0/3422325955 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.694+0000 7f32de000640 1 --2- 172.21.15.47:0/3422325955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b2580 0x5635a67b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.694+0000 7f32dcc00640 1 -- 172.21.15.47:0/3422325955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b2580 msgr2=0x5635a67b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.694+0000 7f32dcc00640 1 --2- 172.21.15.47:0/3422325955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b2580 0x5635a67b4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.694+0000 7f32dcc00640 1 -- 172.21.15.47:0/3422325955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b09f0 msgr2=0x5635a67a6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.694+0000 7f32dcc00640 1 --2- 172.21.15.47:0/3422325955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b09f0 0x5635a67a6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.694+0000 7f32dcc00640 1 -- 172.21.15.47:0/3422325955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635a67abb50 con 0x5635a67b1bc0 2024-03-21T11:25:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.694+0000 7f32de000640 1 --2- 172.21.15.47:0/3422325955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b09f0 0x5635a67a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.694+0000 7f32dcc00640 1 --2- 172.21.15.47:0/3422325955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67c21d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f32c4003e50 tx=0x7f32c4010490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4432fe224eca2861 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.695+0000 7f32c9c00640 1 -- 172.21.15.47:0/3422325955 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c402d020 con 0x5635a67b1bc0 2024-03-21T11:25:29.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.695+0000 7f32c9c00640 1 -- 172.21.15.47:0/3422325955 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32c40055a0 con 0x5635a67b1bc0 2024-03-21T11:25:29.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.695+0000 7f32c9c00640 1 -- 172.21.15.47:0/3422325955 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c4010800 con 0x5635a67b1bc0 2024-03-21T11:25:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.695+0000 7f32e2dba740 1 -- 172.21.15.47:0/3422325955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 msgr2=0x5635a67c21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.695+0000 7f32e2dba740 1 --2- 172.21.15.47:0/3422325955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67c21d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f32c4003e50 tx=0x7f32c4010490 comp rx=0 tx=0).stop 2024-03-21T11:25:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.695+0000 7f32e2dba740 1 -- 172.21.15.47:0/3422325955 shutdown_connections 2024-03-21T11:25:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.695+0000 7f32e2dba740 1 --2- 172.21.15.47:0/3422325955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b2580 0x5635a67b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.695+0000 7f32e2dba740 1 --2- 172.21.15.47:0/3422325955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67c21d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.695+0000 7f32e2dba740 1 --2- 172.21.15.47:0/3422325955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b09f0 0x5635a67a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.695+0000 7f32e2dba740 1 -- 172.21.15.47:0/3422325955 >> 172.21.15.47:0/3422325955 conn(0x5635a67abde0 msgr2=0x5635a67af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.695+0000 7f32e2dba740 1 -- 172.21.15.47:0/3422325955 shutdown_connections 2024-03-21T11:25:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.695+0000 7f32e2dba740 1 -- 172.21.15.47:0/3422325955 wait complete. 2024-03-21T11:25:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.696+0000 7f32e2dba740 1 -- start start 2024-03-21T11:25:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.696+0000 7f32e2dba740 1 Processor -- start 2024-03-21T11:25:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.696+0000 7f32e2dba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b09f0 0x5635a67bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.696+0000 7f32e2dba740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.696+0000 7f32e2dba740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b2580 0x5635a67c13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.696+0000 7f32e2dba740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d81ad4a0 con 0x5635a67b2580 2024-03-21T11:25:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.696+0000 7f32e2dba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d81ace40 con 0x5635a67b09f0 2024-03-21T11:25:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.696+0000 7f32e2dba740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d81aa340 con 0x5635a67b1bc0 2024-03-21T11:25:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.696+0000 7f32e2dba740 10 client.0 _ll_register_callbacks cb 0x7f32c40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.697+0000 7f32de000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b09f0 0x5635a67bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.697+0000 7f32de000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b2580 0x5635a67c13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.697+0000 7f32dcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.697+0000 7f32dcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67c0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48730/0 (socket says 172.21.15.47:48730) 2024-03-21T11:25:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.697+0000 7f32dcc00640 1 -- 172.21.15.47:0/3473305317 learned_addr learned my addr 172.21.15.47:0/3473305317 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.697+0000 7f32de000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b09f0 0x5635a67bfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42676/0 (socket says 172.21.15.47:42676) 2024-03-21T11:25:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.697+0000 7f32dcc00640 1 -- 172.21.15.47:0/3473305317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b09f0 msgr2=0x5635a67bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.697+0000 7f32dcc00640 1 --2- 172.21.15.47:0/3473305317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b09f0 0x5635a67bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.697+0000 7f32dcc00640 1 -- 172.21.15.47:0/3473305317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b2580 msgr2=0x5635a67c13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.697+0000 7f32dcc00640 1 --2- 172.21.15.47:0/3473305317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b2580 0x5635a67c13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.697+0000 7f32dcc00640 1 -- 172.21.15.47:0/3473305317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635a67c1990 con 0x5635a67b1bc0 2024-03-21T11:25:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.697+0000 7f32dcc00640 1 --2- 172.21.15.47:0/3473305317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67c0400 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f32c4012420 tx=0x7f32c40052c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9207b701a8e30c44 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.697+0000 7f32bea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:25:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.698+0000 7f32bea00640 1 -- 172.21.15.47:0/3473305317 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c40035f0 con 0x5635a67b1bc0 2024-03-21T11:25:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.698+0000 7f32bea00640 1 -- 172.21.15.47:0/3473305317 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f32c4018070 con 0x5635a67b1bc0 2024-03-21T11:25:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.698+0000 7f32e2dba740 1 -- 172.21.15.47:0/3473305317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32c40055a0 con 0x5635a67b1bc0 2024-03-21T11:25:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.698+0000 7f32bea00640 1 -- 172.21.15.47:0/3473305317 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32c403f050 con 0x5635a67b1bc0 2024-03-21T11:25:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.698+0000 7f32e2dba740 10 client.26143 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.698+0000 7f32e2dba740 1 -- 172.21.15.47:0/3473305317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5635a6822670 con 0x5635a67b1bc0 2024-03-21T11:25:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.698+0000 7f32bea00640 1 -- 172.21.15.47:0/3473305317 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32c40055a0 con 0x5635a67b1bc0 2024-03-21T11:25:29.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.698+0000 7f32bea00640 1 -- 172.21.15.47:0/3473305317 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f32c400f800 con 0x5635a67b1bc0 2024-03-21T11:25:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.699+0000 7f32e2dba740 10 client.26143 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.699+0000 7f32e2dba740 10 client.26143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.699+0000 7f32e2dba740 20 client.26143 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.699+0000 7f32e2dba740 1 --2- 172.21.15.47:0/3473305317 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5635a6822f50 0x5635a6825370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.699+0000 7f32e2dba740 4 client.26143 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:25:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.699+0000 7f32e2dba740 1 -- 172.21.15.47:0/3473305317 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5635a671cb50 con 0x5635a6822f50 2024-03-21T11:25:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.699+0000 7f32de000640 1 --2- 172.21.15.47:0/3473305317 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5635a6822f50 0x5635a6825370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.699+0000 7f32de000640 1 --2- 172.21.15.47:0/3473305317 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5635a6822f50 0x5635a6825370 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6919f6feaae7551e server_cookie=ba78f28a4b15c800 in_seq=0 out_seq=0 2024-03-21T11:25:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.699+0000 7f32bea00640 10 client.26143 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.699+0000 7f32bea00640 1 -- 172.21.15.47:0/3473305317 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5635a671cb50 con 0x5635a6822f50 2024-03-21T11:25:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.700+0000 7f32bea00640 10 client.26143 handle_command_reply: tid=0 2024-03-21T11:25:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.700+0000 7f32e2dba740 1 client.26143 shutdown 2024-03-21T11:25:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.700+0000 7f32e2dba740 1 -- 172.21.15.47:0/3473305317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 msgr2=0x5635a67c0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.700+0000 7f32e2dba740 1 --2- 172.21.15.47:0/3473305317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67c0400 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f32c4012420 tx=0x7f32c40052c0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.700+0000 7f32e2dba740 1 -- 172.21.15.47:0/3473305317 shutdown_connections 2024-03-21T11:25:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.700+0000 7f32e2dba740 1 --2- 172.21.15.47:0/3473305317 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5635a6822f50 0x5635a6825370 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.700+0000 7f32e2dba740 1 --2- 172.21.15.47:0/3473305317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5635a67b2580 0x5635a67c13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.700+0000 7f32e2dba740 1 --2- 172.21.15.47:0/3473305317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635a67b1bc0 0x5635a67c0400 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.700+0000 7f32e2dba740 1 --2- 172.21.15.47:0/3473305317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635a67b09f0 0x5635a67bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.700+0000 7f32e2dba740 1 -- 172.21.15.47:0/3473305317 >> 172.21.15.47:0/3473305317 conn(0x5635a67abde0 msgr2=0x5635a67aa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.700+0000 7f32bea00640 0 client.26143 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.700+0000 7f32e2dba740 1 -- 172.21.15.47:0/3473305317 shutdown_connections 2024-03-21T11:25:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.700+0000 7f32e2dba740 1 -- 172.21.15.47:0/3473305317 wait complete. 2024-03-21T11:25:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.700+0000 7f32e2dba740 20 client.26143 trim_cache size 0 max 16384 2024-03-21T11:25:29.705 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:25:29.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.701+0000 7f32dea00640 1 -- 172.21.15.47:0/4040061997 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32c00a2f20 msgr2=0x7f32c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.701+0000 7f32dea00640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32c00a2f20 0x7f32c00a53e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0ba6a0 tx=0x7f32cc0f7530 comp rx=0 tx=0).stop 2024-03-21T11:25:29.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.701+0000 7f32dea00640 1 -- 172.21.15.47:0/4040061997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32d81015a0 msgr2=0x7f32d8199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:29.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.701+0000 7f32dea00640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32d81015a0 0x7f32d8199af0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f32cc000c00 tx=0x7f32cc02c6c0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.701+0000 7f32dea00640 1 -- 172.21.15.47:0/4040061997 shutdown_connections 2024-03-21T11:25:29.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.701+0000 7f32dea00640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32c00a2f20 0x7f32c00a53e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.701+0000 7f32dea00640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32d8103150 0x7f32d81a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.701+0000 7f32dea00640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d8102790 0x7f32d819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.701+0000 7f32dea00640 1 --2- 172.21.15.47:0/4040061997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32d81015a0 0x7f32d8199af0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:29.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.701+0000 7f32dea00640 1 -- 172.21.15.47:0/4040061997 >> 172.21.15.47:0/4040061997 conn(0x7f32d80fcb10 msgr2=0x7f32d80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:29.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.701+0000 7f32dea00640 1 -- 172.21.15.47:0/4040061997 shutdown_connections 2024-03-21T11:25:29.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:29.701+0000 7f32dea00640 1 -- 172.21.15.47:0/4040061997 wait complete. 2024-03-21T11:25:29.740 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:29 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/963729770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:25:30.487 INFO:tasks.workunit.client.0.smithi047.stdout: 1 25556 11.07 MB/sec warmup 14 sec latency 9.924 ms 2024-03-21T11:25:30.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:30 smithi047 ceph-mon[31351]: pgmap v465: 129 pgs: 129 active+clean; 9.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 968 KiB/s wr, 257 op/s 2024-03-21T11:25:30.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:30 smithi100 ceph-mon[36245]: pgmap v465: 129 pgs: 129 active+clean; 9.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 968 KiB/s wr, 257 op/s 2024-03-21T11:25:30.721 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:25:30.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:25:30.721 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:25:30.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:30 smithi102 ceph-mon[36748]: pgmap v465: 129 pgs: 129 active+clean; 9.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 968 KiB/s wr, 257 op/s 2024-03-21T11:25:30.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.954+0000 7f7a0b200640 1 -- 172.21.15.47:0/2255188947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a04101580 msgr2=0x7f7a041019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:30.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.954+0000 7f7a0b200640 1 --2- 172.21.15.47:0/2255188947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a04101580 0x7f7a041019a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009900 tx=0x7f79f802bc40 comp rx=0 tx=0).stop 2024-03-21T11:25:30.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.954+0000 7f7a0b200640 1 -- 172.21.15.47:0/2255188947 shutdown_connections 2024-03-21T11:25:30.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.954+0000 7f7a0b200640 1 --2- 172.21.15.47:0/2255188947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a04103130 0x7f7a041099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.954+0000 7f7a0b200640 1 --2- 172.21.15.47:0/2255188947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a04102770 0x7f7a04102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.954+0000 7f7a0b200640 1 --2- 172.21.15.47:0/2255188947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a04101580 0x7f7a041019a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.954+0000 7f7a0b200640 1 -- 172.21.15.47:0/2255188947 >> 172.21.15.47:0/2255188947 conn(0x7f7a040fcb10 msgr2=0x7f7a040fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:30.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.954+0000 7f7a0b200640 1 -- 172.21.15.47:0/2255188947 shutdown_connections 2024-03-21T11:25:30.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.954+0000 7f7a0b200640 1 -- 172.21.15.47:0/2255188947 wait complete. 2024-03-21T11:25:30.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.955+0000 7f7a0b200640 1 Processor -- start 2024-03-21T11:25:30.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.955+0000 7f7a0b200640 1 -- start start 2024-03-21T11:25:30.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.955+0000 7f7a0b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a04101580 0x7f7a04075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:30.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.955+0000 7f7a0b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a04102770 0x7f7a04073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:30.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.955+0000 7f7a0b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a04103130 0x7f7a04073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:30.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.955+0000 7f7a0b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0410d1d0 con 0x7f7a04101580 2024-03-21T11:25:30.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.955+0000 7f7a0b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0410d050 con 0x7f7a04103130 2024-03-21T11:25:30.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.955+0000 7f7a0b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0410d350 con 0x7f7a04102770 2024-03-21T11:25:30.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.956+0000 7f7a09400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a04102770 0x7f7a04073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:30.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.956+0000 7f7a09400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a04102770 0x7f7a04073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48754/0 (socket says 172.21.15.47:48754) 2024-03-21T11:25:30.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.956+0000 7f7a09400640 1 -- 172.21.15.47:0/757115753 learned_addr learned my addr 172.21.15.47:0/757115753 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:30.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.956+0000 7f7a0a800640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a04103130 0x7f7a04073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:30.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.956+0000 7f7a09e00640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a04101580 0x7f7a04075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:30.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.956+0000 7f7a09400640 1 -- 172.21.15.47:0/757115753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a04103130 msgr2=0x7f7a04073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:30.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.956+0000 7f7a09400640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a04103130 0x7f7a04073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.956+0000 7f7a09400640 1 -- 172.21.15.47:0/757115753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a04101580 msgr2=0x7f7a04075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:30.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.956+0000 7f7a09400640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a04101580 0x7f7a04075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.956+0000 7f7a09400640 1 -- 172.21.15.47:0/757115753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a04074860 con 0x7f7a04102770 2024-03-21T11:25:30.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.956+0000 7f7a09e00640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a04101580 0x7f7a04075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:30.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.956+0000 7f7a0a800640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a04103130 0x7f7a04073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:30.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.957+0000 7f7a09400640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a04102770 0x7f7a04073a90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f79f400d950 tx=0x7f79f400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:30.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.957+0000 7f79fea00640 1 -- 172.21.15.47:0/757115753 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f4017070 con 0x7f7a04102770 2024-03-21T11:25:30.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.957+0000 7f79fea00640 1 -- 172.21.15.47:0/757115753 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79f4009c30 con 0x7f7a04102770 2024-03-21T11:25:30.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.957+0000 7f7a0b200640 1 -- 172.21.15.47:0/757115753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a041a19c0 con 0x7f7a04102770 2024-03-21T11:25:30.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.957+0000 7f79fea00640 1 -- 172.21.15.47:0/757115753 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f400f6b0 con 0x7f7a04102770 2024-03-21T11:25:30.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.957+0000 7f7a0b200640 1 -- 172.21.15.47:0/757115753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a041a1f60 con 0x7f7a04102770 2024-03-21T11:25:30.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.960+0000 7f79fea00640 1 -- 172.21.15.47:0/757115753 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f79f40109d0 con 0x7f7a04102770 2024-03-21T11:25:30.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.961+0000 7f7a0f401740 1 Processor -- start 2024-03-21T11:25:30.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.961+0000 7f7a0f401740 1 -- start start 2024-03-21T11:25:30.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.961+0000 7f7a0f401740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e0835086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:30.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.961+0000 7f7a0f401740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e083508ca0 0x55e08350d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:30.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.961+0000 7f7a0f401740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08350de10 0x55e0835102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:30.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.961+0000 7f7a0f401740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08346deb0 con 0x55e083508ca0 2024-03-21T11:25:30.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.961+0000 7f7a0f401740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083416a60 con 0x55e0835082b0 2024-03-21T11:25:30.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.961+0000 7f7a0f401740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083469f50 con 0x55e08350de10 2024-03-21T11:25:30.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.962+0000 7f7a0a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e0835086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:30.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.962+0000 7f7a09400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08350de10 0x55e0835102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:30.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.962+0000 7f7a09e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e083508ca0 0x55e08350d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:30.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.962+0000 7f7a0a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e0835086d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42718/0 (socket says 172.21.15.47:42718) 2024-03-21T11:25:30.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.962+0000 7f7a0a800640 1 -- 172.21.15.47:0/1081689245 learned_addr learned my addr 172.21.15.47:0/1081689245 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:30.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.962+0000 7f79fea00640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f79ec0a2fa0 0x7f79ec0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:30.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.962+0000 7f79fea00640 1 -- 172.21.15.47:0/757115753 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f79f40cf390 con 0x7f7a04102770 2024-03-21T11:25:30.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.962+0000 7f7a0a800640 1 -- 172.21.15.47:0/1081689245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08350de10 msgr2=0x55e0835102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:30.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.962+0000 7f7a0a800640 1 --2- 172.21.15.47:0/1081689245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08350de10 0x55e0835102a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.962+0000 7f7a0a800640 1 -- 172.21.15.47:0/1081689245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e083508ca0 msgr2=0x55e08350d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:30.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.962+0000 7f7a0a800640 1 --2- 172.21.15.47:0/1081689245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e083508ca0 0x55e08350d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.962+0000 7f7a0a800640 1 -- 172.21.15.47:0/1081689245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e08324a7e0 con 0x55e0835082b0 2024-03-21T11:25:30.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.962+0000 7f7a09400640 1 --2- 172.21.15.47:0/1081689245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08350de10 0x55e0835102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:30.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.962+0000 7f7a0a800640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f79ec0a2fa0 0x7f79ec0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:30.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.963+0000 7f7a0a800640 1 --2- 172.21.15.47:0/1081689245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e0835086d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7a00011940 tx=0x7f7a00011e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6f66220d772bdf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:30.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.963+0000 7f79f3000640 1 -- 172.21.15.47:0/1081689245 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00017070 con 0x55e0835082b0 2024-03-21T11:25:30.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.963+0000 7f79f3000640 1 -- 172.21.15.47:0/1081689245 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a000046a0 con 0x55e0835082b0 2024-03-21T11:25:30.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.963+0000 7f79f3000640 1 -- 172.21.15.47:0/1081689245 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0001c690 con 0x55e0835082b0 2024-03-21T11:25:30.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.964+0000 7f7a0a800640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f79ec0a2fa0 0x7f79ec0a5460 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7a00021000 tx=0x7f7a00022000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:30.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.964+0000 7f7a0f401740 1 -- 172.21.15.47:0/1081689245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 msgr2=0x55e0835086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:30.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.964+0000 7f7a0f401740 1 --2- 172.21.15.47:0/1081689245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e0835086d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7a00011940 tx=0x7f7a00011e00 comp rx=0 tx=0).stop 2024-03-21T11:25:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.964+0000 7f7a0f401740 1 -- 172.21.15.47:0/1081689245 shutdown_connections 2024-03-21T11:25:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.964+0000 7f7a0f401740 1 --2- 172.21.15.47:0/1081689245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e08350de10 0x55e0835102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.964+0000 7f7a0f401740 1 --2- 172.21.15.47:0/1081689245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e083508ca0 0x55e08350d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.964+0000 7f7a0f401740 1 --2- 172.21.15.47:0/1081689245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e0835086d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.964+0000 7f7a0f401740 1 -- 172.21.15.47:0/1081689245 >> 172.21.15.47:0/1081689245 conn(0x55e083503820 msgr2=0x55e083503c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.964+0000 7f7a0f401740 1 -- 172.21.15.47:0/1081689245 shutdown_connections 2024-03-21T11:25:30.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.964+0000 7f7a0f401740 1 -- 172.21.15.47:0/1081689245 wait complete. 2024-03-21T11:25:30.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.965+0000 7f7a0f401740 1 -- start start 2024-03-21T11:25:30.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.965+0000 7f7a0f401740 1 Processor -- start 2024-03-21T11:25:30.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.965+0000 7f7a0f401740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e083512490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:30.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.965+0000 7f7a0f401740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e0835129d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:30.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a0f401740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e08350de10 0x55e083519a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:30.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a0f401740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083510dc0 con 0x55e08350de10 2024-03-21T11:25:30.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a0f401740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083429eb0 con 0x55e0835082b0 2024-03-21T11:25:30.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a0f401740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083510f40 con 0x55e083508ca0 2024-03-21T11:25:30.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a0f401740 10 client.0 _ll_register_callbacks cb 0x55e08341e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:30.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a09e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e083512490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:30.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a09e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e083512490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42730/0 (socket says 172.21.15.47:42730) 2024-03-21T11:25:30.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a09e00640 1 -- 172.21.15.47:0/1870739566 learned_addr learned my addr 172.21.15.47:0/1870739566 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:30.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a09400640 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e0835129d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:30.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a0a800640 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e08350de10 0x55e083519a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:30.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a09e00640 1 -- 172.21.15.47:0/1870739566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 msgr2=0x55e0835129d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:30.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a09e00640 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e0835129d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a09e00640 1 -- 172.21.15.47:0/1870739566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e08350de10 msgr2=0x55e083519a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:30.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a09e00640 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e08350de10 0x55e083519a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a09e00640 1 -- 172.21.15.47:0/1870739566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0832aaf70 con 0x55e0835082b0 2024-03-21T11:25:30.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a09400640 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e0835129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:30.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.966+0000 7f7a0a800640 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e08350de10 0x55e083519a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:30.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.967+0000 7f7a09e00640 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e083512490 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7a041025d0 tx=0x7f79f80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b152a0c80a2b94e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:30.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.967+0000 7f79e7400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:25:30.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.967+0000 7f79e7400640 1 -- 172.21.15.47:0/1870739566 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8005d50 con 0x55e0835082b0 2024-03-21T11:25:30.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.967+0000 7f7a0f401740 1 -- 172.21.15.47:0/1870739566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a000046a0 con 0x55e0835082b0 2024-03-21T11:25:30.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.967+0000 7f79e7400640 1 -- 172.21.15.47:0/1870739566 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f79f8005450 con 0x55e0835082b0 2024-03-21T11:25:30.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.967+0000 7f79e7400640 1 -- 172.21.15.47:0/1870739566 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79f80044e0 con 0x55e0835082b0 2024-03-21T11:25:30.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.967+0000 7f7a0f401740 10 client.26144 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:30.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.967+0000 7f7a0f401740 1 -- 172.21.15.47:0/1870739566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e083142170 con 0x55e0835082b0 2024-03-21T11:25:30.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.968+0000 7f79e7400640 1 -- 172.21.15.47:0/1870739566 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a000046a0 con 0x55e0835082b0 2024-03-21T11:25:30.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.968+0000 7f79e7400640 1 -- 172.21.15.47:0/1870739566 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f79f80036f0 con 0x55e0835082b0 2024-03-21T11:25:30.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.968+0000 7f7a0f401740 10 client.26144 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:30.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.968+0000 7f7a0f401740 10 client.26144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:30.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.968+0000 7f7a0f401740 20 client.26144 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:30.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.968+0000 7f7a0f401740 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e08351e230 0x55e083520680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:30.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.968+0000 7f7a0f401740 4 client.26144 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:25:30.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.968+0000 7f7a0f401740 1 -- 172.21.15.47:0/1870739566 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e08347b3f0 con 0x55e08351e230 2024-03-21T11:25:30.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.968+0000 7f7a09400640 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e08351e230 0x55e083520680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:30.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.969+0000 7f7a09400640 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e08351e230 0x55e083520680 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=643594396257b282 server_cookie=916e3857f0b3b228 in_seq=0 out_seq=0 2024-03-21T11:25:30.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.969+0000 7f79e7400640 10 client.26144 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:30.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f79e7400640 1 -- 172.21.15.47:0/1870739566 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e08347b3f0 con 0x55e08351e230 2024-03-21T11:25:30.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f79e7400640 10 client.26144 handle_command_reply: tid=0 2024-03-21T11:25:30.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f7a0f401740 1 client.26144 shutdown 2024-03-21T11:25:30.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f7a0f401740 1 -- 172.21.15.47:0/1870739566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 msgr2=0x55e083512490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:30.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f7a0f401740 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e083512490 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7a041025d0 tx=0x7f79f80042f0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f7a0f401740 1 -- 172.21.15.47:0/1870739566 shutdown_connections 2024-03-21T11:25:30.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f7a0f401740 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e08351e230 0x55e083520680 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f7a0f401740 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e08350de10 0x55e083519a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f7a0f401740 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e0835129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f79e7400640 0 client.26144 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:30.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f7a0f401740 1 --2- 172.21.15.47:0/1870739566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e083512490 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f7a0f401740 1 -- 172.21.15.47:0/1870739566 >> 172.21.15.47:0/1870739566 conn(0x55e083503820 msgr2=0x55e083502a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:30.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f7a0f401740 1 -- 172.21.15.47:0/1870739566 shutdown_connections 2024-03-21T11:25:30.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f7a0f401740 1 -- 172.21.15.47:0/1870739566 wait complete. 2024-03-21T11:25:30.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.970+0000 7f7a0f401740 20 client.26144 trim_cache size 0 max 16384 2024-03-21T11:25:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.990+0000 7f7a0f401740 1 Processor -- start 2024-03-21T11:25:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.991+0000 7f7a0f401740 1 -- start start 2024-03-21T11:25:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.991+0000 7f7a0f401740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0835082b0 0x55e083502900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.991+0000 7f7a0f401740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e083524f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.991+0000 7f7a0f401740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e08350de10 0x55e083525480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.991+0000 7f7a0f401740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a041ac180 con 0x55e0835082b0 2024-03-21T11:25:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.991+0000 7f7a0f401740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a041adb00 con 0x55e08350de10 2024-03-21T11:25:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.991+0000 7f7a0f401740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a041ac7e0 con 0x55e083508ca0 2024-03-21T11:25:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.991+0000 7f7a0a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e08350de10 0x55e083525480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.991+0000 7f7a09e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0835082b0 0x55e083502900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.991+0000 7f7a09400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e083524f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.991+0000 7f7a0a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e08350de10 0x55e083525480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42738/0 (socket says 172.21.15.47:42738) 2024-03-21T11:25:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.991+0000 7f7a0a800640 1 -- 172.21.15.47:0/1645165211 learned_addr learned my addr 172.21.15.47:0/1645165211 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:30.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.991+0000 7f7a09e00640 1 --2- 172.21.15.47:0/1645165211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0835082b0 0x55e083502900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54960/0 (socket says 172.21.15.47:54960) 2024-03-21T11:25:30.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.992+0000 7f7a0a800640 1 -- 172.21.15.47:0/1645165211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 msgr2=0x55e083524f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:30.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.992+0000 7f7a0a800640 1 --2- 172.21.15.47:0/1645165211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e083524f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.992+0000 7f7a0a800640 1 -- 172.21.15.47:0/1645165211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0835082b0 msgr2=0x55e083502900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:30.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.992+0000 7f7a0a800640 1 --2- 172.21.15.47:0/1645165211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0835082b0 0x55e083502900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.992+0000 7f7a0a800640 1 -- 172.21.15.47:0/1645165211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e083509080 con 0x55e08350de10 2024-03-21T11:25:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.992+0000 7f7a09400640 1 --2- 172.21.15.47:0/1645165211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e083524f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.992+0000 7f7a0a800640 1 --2- 172.21.15.47:0/1645165211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e08350de10 0x55e083525480 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7a00005c00 tx=0x7f7a00014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27874d78a9c1ba2d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.992+0000 7f79f2600640 1 -- 172.21.15.47:0/1645165211 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00036070 con 0x55e08350de10 2024-03-21T11:25:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.992+0000 7f79f2600640 1 -- 172.21.15.47:0/1645165211 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a0002c030 con 0x55e08350de10 2024-03-21T11:25:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.992+0000 7f79f2600640 1 -- 172.21.15.47:0/1645165211 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00017040 con 0x55e08350de10 2024-03-21T11:25:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.993+0000 7f7a0f401740 1 -- 172.21.15.47:0/1645165211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e08350de10 msgr2=0x55e083525480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.993+0000 7f7a0f401740 1 --2- 172.21.15.47:0/1645165211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e08350de10 0x55e083525480 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7a00005c00 tx=0x7f7a00014000 comp rx=0 tx=0).stop 2024-03-21T11:25:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.993+0000 7f7a0f401740 1 -- 172.21.15.47:0/1645165211 shutdown_connections 2024-03-21T11:25:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.993+0000 7f7a0f401740 1 --2- 172.21.15.47:0/1645165211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e08350de10 0x55e083525480 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.993+0000 7f7a0f401740 1 --2- 172.21.15.47:0/1645165211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e083524f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.993+0000 7f7a0f401740 1 --2- 172.21.15.47:0/1645165211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0835082b0 0x55e083502900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:30.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.993+0000 7f7a0f401740 1 -- 172.21.15.47:0/1645165211 >> 172.21.15.47:0/1645165211 conn(0x55e083503820 msgr2=0x55e0835021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:30.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.993+0000 7f7a0f401740 1 -- 172.21.15.47:0/1645165211 shutdown_connections 2024-03-21T11:25:30.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.993+0000 7f7a0f401740 1 -- 172.21.15.47:0/1645165211 wait complete. 2024-03-21T11:25:30.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.994+0000 7f7a0f401740 1 -- start start 2024-03-21T11:25:30.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.994+0000 7f7a0f401740 1 Processor -- start 2024-03-21T11:25:31.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.994+0000 7f7a0f401740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e083517080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.994+0000 7f7a0f401740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e0835175c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.994+0000 7f7a0f401740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e08350de10 0x55e083518530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.994+0000 7f7a0f401740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a041ad4a0 con 0x55e08350de10 2024-03-21T11:25:31.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.994+0000 7f7a0f401740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a041ace40 con 0x55e0835082b0 2024-03-21T11:25:31.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.994+0000 7f7a0f401740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a041aa340 con 0x55e083508ca0 2024-03-21T11:25:31.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.994+0000 7f7a0f401740 10 client.0 _ll_register_callbacks cb 0x7f79f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:31.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.994+0000 7f7a09e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e083517080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.994+0000 7f7a09400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e0835175c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.994+0000 7f7a09e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e083517080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42742/0 (socket says 172.21.15.47:42742) 2024-03-21T11:25:31.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.994+0000 7f7a09e00640 1 -- 172.21.15.47:0/117894400 learned_addr learned my addr 172.21.15.47:0/117894400 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:31.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.995+0000 7f7a0a800640 1 --2- 172.21.15.47:0/117894400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e08350de10 0x55e083518530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.995+0000 7f7a09e00640 1 -- 172.21.15.47:0/117894400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 msgr2=0x55e0835175c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:31.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.995+0000 7f7a09e00640 1 --2- 172.21.15.47:0/117894400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e0835175c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.995+0000 7f7a09e00640 1 -- 172.21.15.47:0/117894400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e08350de10 msgr2=0x55e083518530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:31.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.995+0000 7f7a09e00640 1 --2- 172.21.15.47:0/117894400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e08350de10 0x55e083518530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.995+0000 7f7a09e00640 1 -- 172.21.15.47:0/117894400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e083518b20 con 0x55e0835082b0 2024-03-21T11:25:31.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.995+0000 7f7a09400640 1 --2- 172.21.15.47:0/117894400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e0835175c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:31.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.995+0000 7f7a09e00640 1 --2- 172.21.15.47:0/117894400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e083517080 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f79f8006010 tx=0x7f79f8003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f63df1bce09f665a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:31.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.995+0000 7f79e7400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:25:31.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.995+0000 7f79e7400640 1 -- 172.21.15.47:0/117894400 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8059060 con 0x55e0835082b0 2024-03-21T11:25:31.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.996+0000 7f7a0f401740 1 -- 172.21.15.47:0/117894400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a0002c030 con 0x55e0835082b0 2024-03-21T11:25:31.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.996+0000 7f79e7400640 1 -- 172.21.15.47:0/117894400 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f79f8005660 con 0x55e0835082b0 2024-03-21T11:25:31.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.996+0000 7f79e7400640 1 -- 172.21.15.47:0/117894400 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79f8060050 con 0x55e0835082b0 2024-03-21T11:25:31.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.996+0000 7f7a0f401740 10 client.26150 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:31.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.996+0000 7f7a0f401740 1 -- 172.21.15.47:0/117894400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e08357a2c0 con 0x55e0835082b0 2024-03-21T11:25:31.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.996+0000 7f79e7400640 1 -- 172.21.15.47:0/117894400 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a0002c030 con 0x55e0835082b0 2024-03-21T11:25:31.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.996+0000 7f79e7400640 1 -- 172.21.15.47:0/117894400 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f79f8034480 con 0x55e0835082b0 2024-03-21T11:25:31.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.996+0000 7f7a0f401740 10 client.26150 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:31.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.996+0000 7f7a0f401740 10 client.26150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:31.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.996+0000 7f7a0f401740 20 client.26150 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:31.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.996+0000 7f7a0f401740 1 --2- 172.21.15.47:0/117894400 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e08357abb0 0x55e08357cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.996+0000 7f7a0f401740 4 client.26150 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:25:31.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.996+0000 7f7a0f401740 1 -- 172.21.15.47:0/117894400 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e083475740 con 0x55e08357abb0 2024-03-21T11:25:31.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.997+0000 7f7a09400640 1 --2- 172.21.15.47:0/117894400 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e08357abb0 0x55e08357cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.997+0000 7f7a09400640 1 --2- 172.21.15.47:0/117894400 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e08357abb0 0x55e08357cfd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5c99a5c56470df6 server_cookie=1b9719cb4be0f547 in_seq=0 out_seq=0 2024-03-21T11:25:31.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.997+0000 7f79e7400640 10 client.26150 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:31.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.998+0000 7f79e7400640 1 -- 172.21.15.47:0/117894400 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e083475740 con 0x55e08357abb0 2024-03-21T11:25:31.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.998+0000 7f79e7400640 10 client.26150 handle_command_reply: tid=0 2024-03-21T11:25:31.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.998+0000 7f7a0f401740 1 client.26150 shutdown 2024-03-21T11:25:31.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.999+0000 7f7a0f401740 1 -- 172.21.15.47:0/117894400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 msgr2=0x55e083517080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:31.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.999+0000 7f7a0f401740 1 --2- 172.21.15.47:0/117894400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e083517080 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f79f8006010 tx=0x7f79f8003820 comp rx=0 tx=0).stop 2024-03-21T11:25:31.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.999+0000 7f7a0f401740 1 -- 172.21.15.47:0/117894400 shutdown_connections 2024-03-21T11:25:31.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.999+0000 7f7a0f401740 1 --2- 172.21.15.47:0/117894400 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e08357abb0 0x55e08357cfd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.999+0000 7f7a0f401740 1 --2- 172.21.15.47:0/117894400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e08350de10 0x55e083518530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.999+0000 7f7a0f401740 1 --2- 172.21.15.47:0/117894400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e083508ca0 0x55e0835175c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.999+0000 7f7a0f401740 1 --2- 172.21.15.47:0/117894400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0835082b0 0x55e083517080 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.999+0000 7f7a0f401740 1 -- 172.21.15.47:0/117894400 >> 172.21.15.47:0/117894400 conn(0x55e083503820 msgr2=0x55e083502a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:31.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.999+0000 7f79e7400640 0 client.26150 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:31.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.999+0000 7f7a0f401740 1 -- 172.21.15.47:0/117894400 shutdown_connections 2024-03-21T11:25:31.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.999+0000 7f7a0f401740 1 -- 172.21.15.47:0/117894400 wait complete. 2024-03-21T11:25:31.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:30.999+0000 7f7a0f401740 20 client.26150 trim_cache size 0 max 16384 2024-03-21T11:25:31.005 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:25:31.005 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:25:31.005 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "4b02f0fc-c2cc-44f2-b948-0396154399a9", 2024-03-21T11:25:31.005 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:25:31.005 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:25:31.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.000+0000 7f7a0b200640 1 -- 172.21.15.47:0/757115753 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f79ec0a2fa0 msgr2=0x7f79ec0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:31.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.000+0000 7f7a0b200640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f79ec0a2fa0 0x7f79ec0a5460 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7a00021000 tx=0x7f7a00022000 comp rx=0 tx=0).stop 2024-03-21T11:25:31.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.000+0000 7f7a0b200640 1 -- 172.21.15.47:0/757115753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a04102770 msgr2=0x7f7a04073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:31.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.000+0000 7f7a0b200640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a04102770 0x7f7a04073a90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f79f400d950 tx=0x7f79f400de10 comp rx=0 tx=0).stop 2024-03-21T11:25:31.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.000+0000 7f7a0b200640 1 -- 172.21.15.47:0/757115753 shutdown_connections 2024-03-21T11:25:31.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.000+0000 7f7a0b200640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f79ec0a2fa0 0x7f79ec0a5460 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.000+0000 7f7a0b200640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a04103130 0x7f7a04073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.000+0000 7f7a0b200640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a04102770 0x7f7a04073a90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.000+0000 7f7a0b200640 1 --2- 172.21.15.47:0/757115753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a04101580 0x7f7a04075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.000+0000 7f7a0b200640 1 -- 172.21.15.47:0/757115753 >> 172.21.15.47:0/757115753 conn(0x7f7a040fcb10 msgr2=0x7f7a04105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:31.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.000+0000 7f7a0b200640 1 -- 172.21.15.47:0/757115753 shutdown_connections 2024-03-21T11:25:31.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.000+0000 7f7a0b200640 1 -- 172.21.15.47:0/757115753 wait complete. 2024-03-21T11:25:31.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:25:31.023 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:25:31.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.242+0000 7fe1aa200640 1 -- 172.21.15.47:0/173867148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a4074080 msgr2=0x7fe1a40744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.242+0000 7fe1aa200640 1 --2- 172.21.15.47:0/173867148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a4074080 0x7fe1a40744e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe194009900 tx=0x7fe19402bcc0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.242+0000 7fe1aa200640 1 -- 172.21.15.47:0/173867148 shutdown_connections 2024-03-21T11:25:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.242+0000 7fe1aa200640 1 --2- 172.21.15.47:0/173867148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a4109450 0x7fe1a410b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.242+0000 7fe1aa200640 1 --2- 172.21.15.47:0/173867148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a4074080 0x7fe1a40744e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.242+0000 7fe1aa200640 1 --2- 172.21.15.47:0/173867148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a4075660 0x7fe1a4073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.243+0000 7fe1aa200640 1 -- 172.21.15.47:0/173867148 >> 172.21.15.47:0/173867148 conn(0x7fe1a40fcb10 msgr2=0x7fe1a40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.243+0000 7fe1aa200640 1 -- 172.21.15.47:0/173867148 shutdown_connections 2024-03-21T11:25:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.243+0000 7fe1aa200640 1 -- 172.21.15.47:0/173867148 wait complete. 2024-03-21T11:25:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.244+0000 7fe1aa200640 1 Processor -- start 2024-03-21T11:25:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.244+0000 7fe1aa200640 1 -- start start 2024-03-21T11:25:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.244+0000 7fe1aa200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a4074080 0x7fe1a41999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.244+0000 7fe1aa200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a4075660 0x7fe1a4199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.244+0000 7fe1aa200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a4109450 0x7fe1a41a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.244+0000 7fe1aa200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a410f420 con 0x7fe1a4109450 2024-03-21T11:25:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.244+0000 7fe1aa200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a410f2a0 con 0x7fe1a4075660 2024-03-21T11:25:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.244+0000 7fe1aa200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a410f5a0 con 0x7fe1a4074080 2024-03-21T11:25:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.245+0000 7fe1a8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a4074080 0x7fe1a41999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.245+0000 7fe1a3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a4075660 0x7fe1a4199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.245+0000 7fe1a8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a4074080 0x7fe1a41999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48796/0 (socket says 172.21.15.47:48796) 2024-03-21T11:25:31.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.245+0000 7fe1a8e00640 1 -- 172.21.15.47:0/1269898638 learned_addr learned my addr 172.21.15.47:0/1269898638 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.245+0000 7fe1a9800640 1 --2- 172.21.15.47:0/1269898638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a4109450 0x7fe1a41a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.245+0000 7fe1a8e00640 1 -- 172.21.15.47:0/1269898638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a4075660 msgr2=0x7fe1a4199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.245+0000 7fe1a8e00640 1 --2- 172.21.15.47:0/1269898638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a4075660 0x7fe1a4199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.245+0000 7fe1a8e00640 1 -- 172.21.15.47:0/1269898638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a4109450 msgr2=0x7fe1a41a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.245+0000 7fe1a8e00640 1 --2- 172.21.15.47:0/1269898638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a4109450 0x7fe1a41a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.245+0000 7fe1a8e00640 1 -- 172.21.15.47:0/1269898638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a41a1610 con 0x7fe1a4074080 2024-03-21T11:25:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.246+0000 7fe1a8e00640 1 --2- 172.21.15.47:0/1269898638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a4074080 0x7fe1a41999f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe19800e9f0 tx=0x7fe19800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.246+0000 7fe1a1600640 1 -- 172.21.15.47:0/1269898638 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19800cd80 con 0x7fe1a4074080 2024-03-21T11:25:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.246+0000 7fe1a1600640 1 -- 172.21.15.47:0/1269898638 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe198005500 con 0x7fe1a4074080 2024-03-21T11:25:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.246+0000 7fe1aa200640 1 -- 172.21.15.47:0/1269898638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a41a1900 con 0x7fe1a4074080 2024-03-21T11:25:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.246+0000 7fe1a1600640 1 -- 172.21.15.47:0/1269898638 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198010690 con 0x7fe1a4074080 2024-03-21T11:25:31.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.246+0000 7fe1aa200640 1 -- 172.21.15.47:0/1269898638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a41a1dd0 con 0x7fe1a4074080 2024-03-21T11:25:31.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.250+0000 7fe1ae42f740 1 Processor -- start 2024-03-21T11:25:31.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.250+0000 7fe1ae42f740 1 -- start start 2024-03-21T11:25:31.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.251+0000 7fe1ae42f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013050 0x563347013470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.251+0000 7fe1ae42f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013a40 0x563347018540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.251+0000 7fe1ae42f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x56334701b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.251+0000 7fe1ae42f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563346f78250 con 0x563347018bb0 2024-03-21T11:25:31.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.251+0000 7fe1ae42f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563346f20a30 con 0x563347013a40 2024-03-21T11:25:31.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.251+0000 7fe1ae42f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563346f742f0 con 0x563347013050 2024-03-21T11:25:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.251+0000 7fe1a1600640 1 -- 172.21.15.47:0/1269898638 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe1980046c0 con 0x7fe1a4074080 2024-03-21T11:25:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.251+0000 7fe1a3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x56334701b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.251+0000 7fe1a3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x56334701b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55004/0 (socket says 172.21.15.47:55004) 2024-03-21T11:25:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.251+0000 7fe1a3e00640 1 -- 172.21.15.47:0/3326079826 learned_addr learned my addr 172.21.15.47:0/3326079826 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.251+0000 7fe1a9800640 1 --2- 172.21.15.47:0/3326079826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013050 0x563347013470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.251+0000 7fe1a8e00640 1 --2- 172.21.15.47:0/3326079826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013a40 0x563347018540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.252+0000 7fe1a3e00640 1 -- 172.21.15.47:0/3326079826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013050 msgr2=0x563347013470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.252+0000 7fe1a3e00640 1 --2- 172.21.15.47:0/3326079826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013050 0x563347013470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.252+0000 7fe1a3e00640 1 -- 172.21.15.47:0/3326079826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013a40 msgr2=0x563347018540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.252+0000 7fe1a3e00640 1 --2- 172.21.15.47:0/3326079826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013a40 0x563347018540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.252+0000 7fe1a3e00640 1 -- 172.21.15.47:0/3326079826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563346f0b690 con 0x563347018bb0 2024-03-21T11:25:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.252+0000 7fe1a9800640 1 --2- 172.21.15.47:0/3326079826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013050 0x563347013470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:31.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.252+0000 7fe1a8e00640 1 --2- 172.21.15.47:0/3326079826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013a40 0x563347018540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:31.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.252+0000 7fe1a3e00640 1 --2- 172.21.15.47:0/3326079826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x56334701b040 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe19402c970 tx=0x7fe194005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521790ce49eae1ff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:31.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.253+0000 7fe192000640 1 -- 172.21.15.47:0/3326079826 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194045070 con 0x563347018bb0 2024-03-21T11:25:31.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.253+0000 7fe192000640 1 -- 172.21.15.47:0/3326079826 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe19402ca90 con 0x563347018bb0 2024-03-21T11:25:31.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.253+0000 7fe192000640 1 -- 172.21.15.47:0/3326079826 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194004ef0 con 0x563347018bb0 2024-03-21T11:25:31.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.253+0000 7fe1a1600640 1 --2- 172.21.15.47:0/1269898638 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe18c0a2f20 0x7fe18c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.253+0000 7fe1a1600640 1 -- 172.21.15.47:0/1269898638 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe1980cfc20 con 0x7fe1a4074080 2024-03-21T11:25:31.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.253+0000 7fe1a9800640 1 --2- 172.21.15.47:0/1269898638 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe18c0a2f20 0x7fe18c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.253+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3326079826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 msgr2=0x56334701b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.253+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3326079826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x56334701b040 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe19402c970 tx=0x7fe194005d50 comp rx=0 tx=0).stop 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.254+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3326079826 shutdown_connections 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.254+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3326079826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x56334701b040 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.254+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3326079826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013a40 0x563347018540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.254+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3326079826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013050 0x563347013470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.254+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3326079826 >> 172.21.15.47:0/3326079826 conn(0x563346fdb200 msgr2=0x56334700e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.254+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3326079826 shutdown_connections 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.254+0000 7fe1a9800640 1 --2- 172.21.15.47:0/1269898638 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe18c0a2f20 0x7fe18c0a53e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5633470142a0 tx=0x7fe19c007680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.254+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3326079826 wait complete. 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.254+0000 7fe1ae42f740 1 -- start start 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.254+0000 7fe1ae42f740 1 Processor -- start 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.255+0000 7fe1ae42f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013050 0x56334701cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.255+0000 7fe1ae42f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013a40 0x56334701d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.255+0000 7fe1ae42f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x563347024550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.255+0000 7fe1ae42f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56334701bb00 con 0x563347018bb0 2024-03-21T11:25:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.255+0000 7fe1ae42f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563346f33e80 con 0x563347013050 2024-03-21T11:25:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.255+0000 7fe1ae42f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56334701bc80 con 0x563347013a40 2024-03-21T11:25:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.255+0000 7fe1ae42f740 10 client.0 _ll_register_callbacks cb 0x563346f46130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.255+0000 7fe1a8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013050 0x56334701cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.255+0000 7fe1a3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013a40 0x56334701d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.255+0000 7fe1a3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013a40 0x56334701d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48812/0 (socket says 172.21.15.47:48812) 2024-03-21T11:25:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.255+0000 7fe1a3e00640 1 -- 172.21.15.47:0/3229528058 learned_addr learned my addr 172.21.15.47:0/3229528058 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.255+0000 7fe1a9800640 1 --2- 172.21.15.47:0/3229528058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x563347024550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.256+0000 7fe1a3e00640 1 -- 172.21.15.47:0/3229528058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013050 msgr2=0x56334701cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.256+0000 7fe1a3e00640 1 --2- 172.21.15.47:0/3229528058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013050 0x56334701cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.256+0000 7fe1a3e00640 1 -- 172.21.15.47:0/3229528058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 msgr2=0x563347024550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:31.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.256+0000 7fe1a3e00640 1 --2- 172.21.15.47:0/3229528058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x563347024550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.256+0000 7fe1a3e00640 1 -- 172.21.15.47:0/3229528058 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563346d567d0 con 0x563347013a40 2024-03-21T11:25:31.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.256+0000 7fe1a8e00640 1 --2- 172.21.15.47:0/3229528058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013050 0x56334701cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.256+0000 7fe1a3e00640 1 --2- 172.21.15.47:0/3229528058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013a40 0x56334701d510 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe19402c200 tx=0x7fe19402ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78973cf93fed1090 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.256+0000 7fe186a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:25:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.256+0000 7fe186a00640 1 -- 172.21.15.47:0/3229528058 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19404e070 con 0x563347013a40 2024-03-21T11:25:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.256+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3229528058 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe19402ca90 con 0x563347013a40 2024-03-21T11:25:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.256+0000 7fe186a00640 1 -- 172.21.15.47:0/3229528058 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe194004450 con 0x563347013a40 2024-03-21T11:25:31.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.257+0000 7fe186a00640 1 -- 172.21.15.47:0/3229528058 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe19405a050 con 0x563347013a40 2024-03-21T11:25:31.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.257+0000 7fe1ae42f740 10 client.26173 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:31.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.257+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3229528058 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563346c8c3f0 con 0x563347013a40 2024-03-21T11:25:31.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.257+0000 7fe186a00640 1 -- 172.21.15.47:0/3229528058 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe19402ca90 con 0x563347013a40 2024-03-21T11:25:31.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.257+0000 7fe186a00640 1 -- 172.21.15.47:0/3229528058 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe194003b90 con 0x563347013a40 2024-03-21T11:25:31.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.257+0000 7fe1ae42f740 10 client.26173 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:31.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.257+0000 7fe1ae42f740 10 client.26173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:31.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.257+0000 7fe1ae42f740 20 client.26173 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:31.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.257+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3229528058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563347028a40 0x56334702ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.257+0000 7fe1ae42f740 4 client.26173 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:25:31.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.257+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3229528058 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563346f85790 con 0x563347028a40 2024-03-21T11:25:31.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.258+0000 7fe1a8e00640 1 --2- 172.21.15.47:0/3229528058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563347028a40 0x56334702ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.258+0000 7fe1a8e00640 1 --2- 172.21.15.47:0/3229528058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563347028a40 0x56334702ae30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5366d9e13d3abbd5 server_cookie=2e16d88c836b30b0 in_seq=0 out_seq=0 2024-03-21T11:25:31.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.258+0000 7fe186a00640 10 client.26173 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:31.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.259+0000 7fe186a00640 1 -- 172.21.15.47:0/3229528058 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563346f85790 con 0x563347028a40 2024-03-21T11:25:31.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.259+0000 7fe186a00640 10 client.26173 handle_command_reply: tid=0 2024-03-21T11:25:31.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.259+0000 7fe1ae42f740 1 client.26173 shutdown 2024-03-21T11:25:31.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.259+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3229528058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013a40 msgr2=0x56334701d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:31.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.259+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3229528058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013a40 0x56334701d510 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe19402c200 tx=0x7fe19402ccc0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.259+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3229528058 shutdown_connections 2024-03-21T11:25:31.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.259+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3229528058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563347028a40 0x56334702ae30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.259+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3229528058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x563347024550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.259+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3229528058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013a40 0x56334701d510 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.259+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3229528058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013050 0x56334701cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.259+0000 7fe186a00640 0 client.26173 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:31.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.259+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3229528058 >> 172.21.15.47:0/3229528058 conn(0x563346fdb200 msgr2=0x56334700dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:31.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.260+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3229528058 shutdown_connections 2024-03-21T11:25:31.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.260+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3229528058 wait complete. 2024-03-21T11:25:31.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.260+0000 7fe1ae42f740 20 client.26173 trim_cache size 0 max 16384 2024-03-21T11:25:31.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.278+0000 7fe1ae42f740 1 Processor -- start 2024-03-21T11:25:31.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.278+0000 7fe1ae42f740 1 -- start start 2024-03-21T11:25:31.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.278+0000 7fe1ae42f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347013050 0x563347011c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.278+0000 7fe1ae42f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013a40 0x563347024510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1ae42f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347018bb0 0x563347012140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1ae42f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a41abed0 con 0x563347013050 2024-03-21T11:25:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1ae42f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a41ad850 con 0x563347018bb0 2024-03-21T11:25:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1ae42f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a41ac530 con 0x563347013a40 2024-03-21T11:25:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1a3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013a40 0x563347024510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1a3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013a40 0x563347024510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48820/0 (socket says 172.21.15.47:48820) 2024-03-21T11:25:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1a3e00640 1 -- 172.21.15.47:0/1579959751 learned_addr learned my addr 172.21.15.47:0/1579959751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1a8e00640 1 --2- 172.21.15.47:0/1579959751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347013050 0x563347011c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1a9800640 1 --2- 172.21.15.47:0/1579959751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347018bb0 0x563347012140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1a3e00640 1 -- 172.21.15.47:0/1579959751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347018bb0 msgr2=0x563347012140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1a3e00640 1 --2- 172.21.15.47:0/1579959751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347018bb0 0x563347012140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1a3e00640 1 -- 172.21.15.47:0/1579959751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347013050 msgr2=0x563347011c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1a3e00640 1 --2- 172.21.15.47:0/1579959751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347013050 0x563347011c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1a3e00640 1 -- 172.21.15.47:0/1579959751 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563346c4c0f0 con 0x563347013a40 2024-03-21T11:25:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.279+0000 7fe1a3e00640 1 --2- 172.21.15.47:0/1579959751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013a40 0x563347024510 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe19402c970 tx=0x7fe1940095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e7cc2fd67d8d92c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.280+0000 7fe191600640 1 -- 172.21.15.47:0/1579959751 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19404e040 con 0x563347013a40 2024-03-21T11:25:31.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.280+0000 7fe191600640 1 -- 172.21.15.47:0/1579959751 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe194043070 con 0x563347013a40 2024-03-21T11:25:31.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.280+0000 7fe191600640 1 -- 172.21.15.47:0/1579959751 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194045050 con 0x563347013a40 2024-03-21T11:25:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.281+0000 7fe1ae42f740 1 -- 172.21.15.47:0/1579959751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013a40 msgr2=0x563347024510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.281+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/1579959751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013a40 0x563347024510 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe19402c970 tx=0x7fe1940095a0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.281+0000 7fe1ae42f740 1 -- 172.21.15.47:0/1579959751 shutdown_connections 2024-03-21T11:25:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.281+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/1579959751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347018bb0 0x563347012140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.281+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/1579959751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013a40 0x563347024510 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.281+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/1579959751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347013050 0x563347011c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.281+0000 7fe1ae42f740 1 -- 172.21.15.47:0/1579959751 >> 172.21.15.47:0/1579959751 conn(0x563346fdb200 msgr2=0x563347014d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.281+0000 7fe1ae42f740 1 -- 172.21.15.47:0/1579959751 shutdown_connections 2024-03-21T11:25:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.281+0000 7fe1ae42f740 1 -- 172.21.15.47:0/1579959751 wait complete. 2024-03-21T11:25:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.281+0000 7fe1ae42f740 1 -- start start 2024-03-21T11:25:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.281+0000 7fe1ae42f740 1 Processor -- start 2024-03-21T11:25:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.282+0000 7fe1ae42f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013050 0x563347021e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.282+0000 7fe1ae42f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013a40 0x5633470223c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.282+0000 7fe1ae42f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x5633470233d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.282+0000 7fe1ae42f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a41ad1f0 con 0x563347018bb0 2024-03-21T11:25:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.282+0000 7fe1ae42f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a41acb90 con 0x563347013a40 2024-03-21T11:25:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.282+0000 7fe1ae42f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a41aa090 con 0x563347013050 2024-03-21T11:25:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.282+0000 7fe1ae42f740 10 client.0 _ll_register_callbacks cb 0x7fe19c007520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.282+0000 7fe1a8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013050 0x563347021e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.282+0000 7fe1a9800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x5633470233d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.282+0000 7fe1a9800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x5633470233d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55014/0 (socket says 172.21.15.47:55014) 2024-03-21T11:25:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.282+0000 7fe1a8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013050 0x563347021e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48830/0 (socket says 172.21.15.47:48830) 2024-03-21T11:25:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.282+0000 7fe1a9800640 1 -- 172.21.15.47:0/3363142664 learned_addr learned my addr 172.21.15.47:0/3363142664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.282+0000 7fe1a3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013a40 0x5633470223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.283+0000 7fe1a8e00640 1 -- 172.21.15.47:0/3363142664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013a40 msgr2=0x5633470223c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:31.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.283+0000 7fe1a8e00640 1 --2- 172.21.15.47:0/3363142664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013a40 0x5633470223c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.283+0000 7fe1a8e00640 1 -- 172.21.15.47:0/3363142664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 msgr2=0x5633470233d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:31.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.283+0000 7fe1a8e00640 1 --2- 172.21.15.47:0/3363142664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x5633470233d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.283+0000 7fe1a8e00640 1 -- 172.21.15.47:0/3363142664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633470239c0 con 0x563347013050 2024-03-21T11:25:31.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.283+0000 7fe1a8e00640 1 --2- 172.21.15.47:0/3363142664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013050 0x563347021e80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe1980cd3e0 tx=0x7fe1980d3250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bd50621adb79a71 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:31.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.283+0000 7fe186a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:25:31.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.283+0000 7fe186a00640 1 -- 172.21.15.47:0/3363142664 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1980b5060 con 0x563347013050 2024-03-21T11:25:31.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.283+0000 7fe186a00640 1 -- 172.21.15.47:0/3363142664 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe1980bc0b0 con 0x563347013050 2024-03-21T11:25:31.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.283+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3363142664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe194043070 con 0x563347013050 2024-03-21T11:25:31.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.283+0000 7fe186a00640 1 -- 172.21.15.47:0/3363142664 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe19809c6c0 con 0x563347013050 2024-03-21T11:25:31.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.283+0000 7fe1ae42f740 10 client.26179 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:31.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.283+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3363142664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563347084530 con 0x563347013050 2024-03-21T11:25:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.284+0000 7fe186a00640 1 -- 172.21.15.47:0/3363142664 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe194043070 con 0x563347013050 2024-03-21T11:25:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.284+0000 7fe186a00640 1 -- 172.21.15.47:0/3363142664 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe1980b6020 con 0x563347013050 2024-03-21T11:25:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.284+0000 7fe1ae42f740 10 client.26179 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.284+0000 7fe1ae42f740 10 client.26179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.284+0000 7fe1ae42f740 20 client.26179 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.284+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3363142664 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563347084e30 0x563347087250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.284+0000 7fe1ae42f740 4 client.26179 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:25:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.284+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3363142664 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563346f7fae0 con 0x563347084e30 2024-03-21T11:25:31.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.284+0000 7fe1a3e00640 1 --2- 172.21.15.47:0/3363142664 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563347084e30 0x563347087250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:31.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe1a3e00640 1 --2- 172.21.15.47:0/3363142664 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563347084e30 0x563347087250 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3184afc7b64e9c42 server_cookie=9e0aa833a329a8b0 in_seq=0 out_seq=0 2024-03-21T11:25:31.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe186a00640 10 client.26179 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:31.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe186a00640 1 -- 172.21.15.47:0/3363142664 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563346f7fae0 con 0x563347084e30 2024-03-21T11:25:31.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe186a00640 10 client.26179 handle_command_reply: tid=0 2024-03-21T11:25:31.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe1ae42f740 1 client.26179 shutdown 2024-03-21T11:25:31.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3363142664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013050 msgr2=0x563347021e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:31.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3363142664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013050 0x563347021e80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe1980cd3e0 tx=0x7fe1980d3250 comp rx=0 tx=0).stop 2024-03-21T11:25:31.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3363142664 shutdown_connections 2024-03-21T11:25:31.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3363142664 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563347084e30 0x563347087250 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3363142664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563347018bb0 0x5633470233d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3363142664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563347013a40 0x5633470223c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe1ae42f740 1 --2- 172.21.15.47:0/3363142664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563347013050 0x563347021e80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe186a00640 0 client.26179 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:31.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3363142664 >> 172.21.15.47:0/3363142664 conn(0x563346fdb200 msgr2=0x5633470144e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:31.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.285+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3363142664 shutdown_connections 2024-03-21T11:25:31.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.286+0000 7fe1ae42f740 1 -- 172.21.15.47:0/3363142664 wait complete. 2024-03-21T11:25:31.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.286+0000 7fe1ae42f740 20 client.26179 trim_cache size 0 max 16384 2024-03-21T11:25:31.291 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:25:31.291 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:25:31.291 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:25:31.291 INFO:teuthology.orchestra.run.smithi047.stdout: "4b02f0fc-c2cc-44f2-b948-0396154399a9": { 2024-03-21T11:25:31.292 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:25:31.292 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "4b02f0fc-c2cc-44f2-b948-0396154399a9", 2024-03-21T11:25:31.292 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:25:31.292 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:25:31.292 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:25:31.292 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:25:31.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.286+0000 7fe1aa200640 1 -- 172.21.15.47:0/1269898638 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe18c0a2f20 msgr2=0x7fe18c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:31.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.286+0000 7fe1aa200640 1 --2- 172.21.15.47:0/1269898638 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe18c0a2f20 0x7fe18c0a53e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5633470142a0 tx=0x7fe19c007680 comp rx=0 tx=0).stop 2024-03-21T11:25:31.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.286+0000 7fe1aa200640 1 -- 172.21.15.47:0/1269898638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a4074080 msgr2=0x7fe1a41999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:31.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.286+0000 7fe1aa200640 1 --2- 172.21.15.47:0/1269898638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a4074080 0x7fe1a41999f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe19800e9f0 tx=0x7fe19800edf0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.286+0000 7fe1aa200640 1 -- 172.21.15.47:0/1269898638 shutdown_connections 2024-03-21T11:25:31.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.286+0000 7fe1aa200640 1 --2- 172.21.15.47:0/1269898638 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe18c0a2f20 0x7fe18c0a53e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.286+0000 7fe1aa200640 1 --2- 172.21.15.47:0/1269898638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a4109450 0x7fe1a41a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.286+0000 7fe1aa200640 1 --2- 172.21.15.47:0/1269898638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a4075660 0x7fe1a4199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.286+0000 7fe1aa200640 1 --2- 172.21.15.47:0/1269898638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a4074080 0x7fe1a41999f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:31.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.286+0000 7fe1aa200640 1 -- 172.21.15.47:0/1269898638 >> 172.21.15.47:0/1269898638 conn(0x7fe1a40fcb10 msgr2=0x7fe1a41053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:31.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.287+0000 7fe1aa200640 1 -- 172.21.15.47:0/1269898638 shutdown_connections 2024-03-21T11:25:31.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:25:31.287+0000 7fe1aa200640 1 -- 172.21.15.47:0/1269898638 wait complete. 2024-03-21T11:25:31.305 INFO:tasks.cephfs.filesystem:scrub status for tag:4b02f0fc-c2cc-44f2-b948-0396154399a9 - {'path': '/', 'tag': '4b02f0fc-c2cc-44f2-b948-0396154399a9', 'options': 'recursive,force'} 2024-03-21T11:25:31.487 INFO:tasks.workunit.client.0.smithi047.stdout: 1 27772 11.11 MB/sec warmup 15 sec latency 11.096 ms 2024-03-21T11:25:31.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:31 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:25:31.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:31 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:25:31.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:31 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:25:31.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:31 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:25:31.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:31 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:25:31.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:31 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:25:31.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:31 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:25:31.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:31 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:25:31.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:31 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:25:31.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:31 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:25:31.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:31 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:25:31.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:31 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:25:32.487 INFO:tasks.workunit.client.0.smithi047.stdout: 1 29197 10.90 MB/sec warmup 16 sec latency 9.508 ms 2024-03-21T11:25:32.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:32 smithi047 ceph-mon[31351]: pgmap v466: 129 pgs: 129 active+clean; 9.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 986 KiB/s wr, 259 op/s 2024-03-21T11:25:32.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:32 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:25:32.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:32 smithi100 ceph-mon[36245]: pgmap v466: 129 pgs: 129 active+clean; 9.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 986 KiB/s wr, 259 op/s 2024-03-21T11:25:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:32 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:25:32.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:32 smithi102 ceph-mon[36748]: pgmap v466: 129 pgs: 129 active+clean; 9.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 986 KiB/s wr, 259 op/s 2024-03-21T11:25:32.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:32 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:25:33.487 INFO:tasks.workunit.client.0.smithi047.stdout: 1 31415 10.97 MB/sec warmup 17 sec latency 18.793 ms 2024-03-21T11:25:34.488 INFO:tasks.workunit.client.0.smithi047.stdout: 1 32848 10.77 MB/sec warmup 18 sec latency 10.520 ms 2024-03-21T11:25:34.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:34 smithi047 ceph-mon[31351]: pgmap v467: 129 pgs: 129 active+clean; 14 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.1 MiB/s wr, 278 op/s 2024-03-21T11:25:34.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:34 smithi100 ceph-mon[36245]: pgmap v467: 129 pgs: 129 active+clean; 14 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.1 MiB/s wr, 278 op/s 2024-03-21T11:25:34.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:34 smithi102 ceph-mon[36748]: pgmap v467: 129 pgs: 129 active+clean; 14 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.1 MiB/s wr, 278 op/s 2024-03-21T11:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.456+0000 7f7b79400640 1 -- 172.21.15.47:0/660144616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b74108d10 msgr2=0x7f7b74109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.456+0000 7f7b79400640 1 --2- 172.21.15.47:0/660144616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b74108d10 0x7f7b74109190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009900 tx=0x7f7b6002bc40 comp rx=0 tx=0).stop 2024-03-21T11:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.457+0000 7f7b79400640 1 -- 172.21.15.47:0/660144616 shutdown_connections 2024-03-21T11:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.457+0000 7f7b79400640 1 --2- 172.21.15.47:0/660144616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b741096d0 0x7f7b7410ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.457+0000 7f7b79400640 1 --2- 172.21.15.47:0/660144616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b74108d10 0x7f7b74109190 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.457+0000 7f7b79400640 1 --2- 172.21.15.47:0/660144616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b74107b20 0x7f7b74107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.457+0000 7f7b79400640 1 -- 172.21.15.47:0/660144616 >> 172.21.15.47:0/660144616 conn(0x7f7b74076ae0 msgr2=0x7f7b74078f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.457+0000 7f7b79400640 1 -- 172.21.15.47:0/660144616 shutdown_connections 2024-03-21T11:25:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.457+0000 7f7b79400640 1 -- 172.21.15.47:0/660144616 wait complete. 2024-03-21T11:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.458+0000 7f7b79400640 1 Processor -- start 2024-03-21T11:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.458+0000 7f7b79400640 1 -- start start 2024-03-21T11:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.458+0000 7f7b79400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b74107b20 0x7f7b7419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.458+0000 7f7b79400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b74108d10 0x7f7b7419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.458+0000 7f7b79400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b741096d0 0x7f7b741a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.458+0000 7f7b79400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b74113780 con 0x7f7b74108d10 2024-03-21T11:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.458+0000 7f7b79400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b74113600 con 0x7f7b74107b20 2024-03-21T11:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.458+0000 7f7b79400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b74113900 con 0x7f7b741096d0 2024-03-21T11:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.459+0000 7f7b78a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b741096d0 0x7f7b741a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.459+0000 7f7b73400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b74108d10 0x7f7b7419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.459+0000 7f7b78a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b741096d0 0x7f7b741a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48850/0 (socket says 172.21.15.47:48850) 2024-03-21T11:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.459+0000 7f7b78a00640 1 -- 172.21.15.47:0/2535162 learned_addr learned my addr 172.21.15.47:0/2535162 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.459+0000 7f7b73e00640 1 --2- 172.21.15.47:0/2535162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b74107b20 0x7f7b7419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.459+0000 7f7b78a00640 1 -- 172.21.15.47:0/2535162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b74107b20 msgr2=0x7f7b7419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.459+0000 7f7b78a00640 1 --2- 172.21.15.47:0/2535162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b74107b20 0x7f7b7419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.459+0000 7f7b78a00640 1 -- 172.21.15.47:0/2535162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b74108d10 msgr2=0x7f7b7419e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.459+0000 7f7b78a00640 1 --2- 172.21.15.47:0/2535162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b74108d10 0x7f7b7419e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.459+0000 7f7b78a00640 1 -- 172.21.15.47:0/2535162 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b741a5a50 con 0x7f7b741096d0 2024-03-21T11:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.459+0000 7f7b78a00640 1 --2- 172.21.15.47:0/2535162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b741096d0 0x7f7b741a53e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00efd0 tx=0x7f7b6c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.460+0000 7f7b70c00640 1 -- 172.21.15.47:0/2535162 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c003b70 con 0x7f7b741096d0 2024-03-21T11:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.460+0000 7f7b79400640 1 -- 172.21.15.47:0/2535162 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b741a5d40 con 0x7f7b741096d0 2024-03-21T11:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.460+0000 7f7b70c00640 1 -- 172.21.15.47:0/2535162 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b6c003d10 con 0x7f7b741096d0 2024-03-21T11:25:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.460+0000 7f7b70c00640 1 -- 172.21.15.47:0/2535162 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0109f0 con 0x7f7b741096d0 2024-03-21T11:25:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.460+0000 7f7b79400640 1 -- 172.21.15.47:0/2535162 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b741a61b0 con 0x7f7b741096d0 2024-03-21T11:25:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.464+0000 7f7b7d76f740 1 Processor -- start 2024-03-21T11:25:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.465+0000 7f7b70c00640 1 -- 172.21.15.47:0/2535162 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7b6c004020 con 0x7f7b741096d0 2024-03-21T11:25:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.465+0000 7f7b7d76f740 1 -- start start 2024-03-21T11:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.465+0000 7f7b7d76f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702d3f0 0x55b24702d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.465+0000 7f7b7d76f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702dde0 0x55b2470328e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.465+0000 7f7b7d76f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b247032f50 0x55b2470353e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.465+0000 7f7b7d76f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b246ff03b0 con 0x55b24702dde0 2024-03-21T11:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.465+0000 7f7b7d76f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b246f92830 con 0x55b247032f50 2024-03-21T11:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.465+0000 7f7b7d76f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b246f3b5d0 con 0x55b24702d3f0 2024-03-21T11:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.465+0000 7f7b73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702d3f0 0x55b24702d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.465+0000 7f7b73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702d3f0 0x55b24702d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48852/0 (socket says 172.21.15.47:48852) 2024-03-21T11:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.465+0000 7f7b73400640 1 -- 172.21.15.47:0/3123481478 learned_addr learned my addr 172.21.15.47:0/3123481478 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.465+0000 7f7b73e00640 1 --2- 172.21.15.47:0/3123481478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702dde0 0x55b2470328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.466+0000 7f7b73400640 1 --2- 172.21.15.47:0/3123481478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b247032f50 0x55b2470353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.466+0000 7f7b73e00640 1 -- 172.21.15.47:0/3123481478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702d3f0 msgr2=0x55b24702d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.466+0000 7f7b73e00640 1 --2- 172.21.15.47:0/3123481478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702d3f0 0x55b24702d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.466+0000 7f7b73e00640 1 -- 172.21.15.47:0/3123481478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b247032f50 msgr2=0x55b2470353e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.466+0000 7f7b73e00640 1 --2- 172.21.15.47:0/3123481478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b247032f50 0x55b2470353e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.466+0000 7f7b73e00640 1 -- 172.21.15.47:0/3123481478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b246f532c0 con 0x55b24702dde0 2024-03-21T11:25:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.466+0000 7f7b73e00640 1 --2- 172.21.15.47:0/3123481478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702dde0 0x55b2470328e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009bf0 tx=0x7f7b64011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f69d5da7d076501 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.466+0000 7f7b70c00640 1 --2- 172.21.15.47:0/2535162 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7b580a2f10 0x7f7b580a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.467+0000 7f7b70c00640 1 -- 172.21.15.47:0/2535162 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7b6c0cfbe0 con 0x7f7b741096d0 2024-03-21T11:25:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.467+0000 7f7b69200640 1 -- 172.21.15.47:0/3123481478 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b64004050 con 0x55b24702dde0 2024-03-21T11:25:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.467+0000 7f7b73400640 1 --2- 172.21.15.47:0/2535162 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7b580a2f10 0x7f7b580a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.467+0000 7f7b69200640 1 -- 172.21.15.47:0/3123481478 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b64005500 con 0x55b24702dde0 2024-03-21T11:25:35.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.467+0000 7f7b69200640 1 -- 172.21.15.47:0/3123481478 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b64004d80 con 0x55b24702dde0 2024-03-21T11:25:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.468+0000 7f7b7d76f740 1 -- 172.21.15.47:0/3123481478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702dde0 msgr2=0x55b2470328e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.468+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/3123481478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702dde0 0x55b2470328e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009bf0 tx=0x7f7b64011810 comp rx=0 tx=0).stop 2024-03-21T11:25:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.468+0000 7f7b73400640 1 --2- 172.21.15.47:0/2535162 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7b580a2f10 0x7f7b580a53d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7b6002c720 tx=0x7f7b60044040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.468+0000 7f7b7d76f740 1 -- 172.21.15.47:0/3123481478 shutdown_connections 2024-03-21T11:25:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.468+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/3123481478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b247032f50 0x55b2470353e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.468+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/3123481478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702dde0 0x55b2470328e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.468+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/3123481478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702d3f0 0x55b24702d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.468+0000 7f7b7d76f740 1 -- 172.21.15.47:0/3123481478 >> 172.21.15.47:0/3123481478 conn(0x55b247028960 msgr2=0x55b247028d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.468+0000 7f7b7d76f740 1 -- 172.21.15.47:0/3123481478 shutdown_connections 2024-03-21T11:25:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.468+0000 7f7b7d76f740 1 -- 172.21.15.47:0/3123481478 wait complete. 2024-03-21T11:25:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.468+0000 7f7b7d76f740 1 -- start start 2024-03-21T11:25:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.469+0000 7f7b7d76f740 1 Processor -- start 2024-03-21T11:25:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.469+0000 7f7b7d76f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702d3f0 0x55b2470375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.469+0000 7f7b7d76f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702dde0 0x55b247037b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.469+0000 7f7b7d76f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b247032f50 0x55b24703eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.469+0000 7f7b7d76f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b246f4ea20 con 0x55b24702d3f0 2024-03-21T11:25:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.469+0000 7f7b7d76f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b246f8e8d0 con 0x55b247032f50 2024-03-21T11:25:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.469+0000 7f7b7d76f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b247035f00 con 0x55b24702dde0 2024-03-21T11:25:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.469+0000 7f7b7d76f740 10 client.0 _ll_register_callbacks cb 0x55b246f8b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.469+0000 7f7b73e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702d3f0 0x55b2470375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.469+0000 7f7b73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702dde0 0x55b247037b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.469+0000 7f7b78a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b247032f50 0x55b24703eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.469+0000 7f7b73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702dde0 0x55b247037b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48858/0 (socket says 172.21.15.47:48858) 2024-03-21T11:25:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.469+0000 7f7b73400640 1 -- 172.21.15.47:0/2620636053 learned_addr learned my addr 172.21.15.47:0/2620636053 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.470+0000 7f7b78a00640 1 -- 172.21.15.47:0/2620636053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702dde0 msgr2=0x55b247037b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.470+0000 7f7b78a00640 1 --2- 172.21.15.47:0/2620636053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702dde0 0x55b247037b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.470+0000 7f7b78a00640 1 -- 172.21.15.47:0/2620636053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702d3f0 msgr2=0x55b2470375d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.470+0000 7f7b78a00640 1 --2- 172.21.15.47:0/2620636053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702d3f0 0x55b2470375d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.470+0000 7f7b78a00640 1 -- 172.21.15.47:0/2620636053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b246ca7210 con 0x55b247032f50 2024-03-21T11:25:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.470+0000 7f7b73e00640 1 --2- 172.21.15.47:0/2620636053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702d3f0 0x55b2470375d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.470+0000 7f7b78a00640 1 --2- 172.21.15.47:0/2620636053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b247032f50 0x55b24703eb50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00cba0 tx=0x7f7b6c009820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1abe0f9afd167a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.470+0000 7f7b5d600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:25:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.470+0000 7f7b5d600640 1 -- 172.21.15.47:0/2620636053 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0a0070 con 0x55b247032f50 2024-03-21T11:25:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.470+0000 7f7b5d600640 1 -- 172.21.15.47:0/2620636053 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7b6c004620 con 0x55b247032f50 2024-03-21T11:25:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.470+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2620636053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b64005500 con 0x55b247032f50 2024-03-21T11:25:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.471+0000 7f7b5d600640 1 -- 172.21.15.47:0/2620636053 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b6c0a4930 con 0x55b247032f50 2024-03-21T11:25:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.471+0000 7f7b7d76f740 10 client.26180 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.471+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2620636053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b246d73620 con 0x55b247032f50 2024-03-21T11:25:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.471+0000 7f7b5d600640 1 -- 172.21.15.47:0/2620636053 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b64005500 con 0x55b247032f50 2024-03-21T11:25:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.471+0000 7f7b5d600640 1 -- 172.21.15.47:0/2620636053 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7b6c09f070 con 0x55b247032f50 2024-03-21T11:25:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.471+0000 7f7b7d76f740 10 client.26180 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.471+0000 7f7b7d76f740 10 client.26180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.471+0000 7f7b7d76f740 20 client.26180 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.471+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2620636053 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b247043450 0x55b2470458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.471+0000 7f7b7d76f740 4 client.26180 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.471+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2620636053 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b246f9fd70 con 0x55b247043450 2024-03-21T11:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.472+0000 7f7b73e00640 1 --2- 172.21.15.47:0/2620636053 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b247043450 0x55b2470458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.472+0000 7f7b73e00640 1 --2- 172.21.15.47:0/2620636053 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b247043450 0x55b2470458c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18ecfb43f914c8e0 server_cookie=a72527d7d423836e in_seq=0 out_seq=0 2024-03-21T11:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.472+0000 7f7b5d600640 10 client.26180 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.474+0000 7f7b5d600640 1 -- 172.21.15.47:0/2620636053 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b246f9fd70 con 0x55b247043450 2024-03-21T11:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.474+0000 7f7b5d600640 10 client.26180 handle_command_reply: tid=0 2024-03-21T11:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.474+0000 7f7b7d76f740 1 client.26180 shutdown 2024-03-21T11:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.474+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2620636053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b247032f50 msgr2=0x55b24703eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.474+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2620636053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b247032f50 0x55b24703eb50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00cba0 tx=0x7f7b6c009820 comp rx=0 tx=0).stop 2024-03-21T11:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.474+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2620636053 shutdown_connections 2024-03-21T11:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.474+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2620636053 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b247043450 0x55b2470458c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.474+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2620636053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b247032f50 0x55b24703eb50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.474+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2620636053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702dde0 0x55b247037b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.474+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2620636053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702d3f0 0x55b2470375d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.474+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2620636053 >> 172.21.15.47:0/2620636053 conn(0x55b247028960 msgr2=0x55b247027b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.474+0000 7f7b5d600640 0 client.26180 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.475+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2620636053 shutdown_connections 2024-03-21T11:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.475+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2620636053 wait complete. 2024-03-21T11:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.475+0000 7f7b7d76f740 20 client.26180 trim_cache size 0 max 16384 2024-03-21T11:25:35.487 INFO:tasks.workunit.client.0.smithi047.stdout: 1 35258 10.92 MB/sec warmup 19 sec latency 10.066 ms 2024-03-21T11:25:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.492+0000 7f7b7d76f740 1 Processor -- start 2024-03-21T11:25:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.493+0000 7f7b7d76f740 1 -- start start 2024-03-21T11:25:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.493+0000 7f7b7d76f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b247028960 0x55b24702f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.493+0000 7f7b7d76f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b24702d3f0 0x55b24703eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.493+0000 7f7b7d76f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702dde0 0x55b24702f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.493+0000 7f7b7d76f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741b03d0 con 0x55b24702dde0 2024-03-21T11:25:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.493+0000 7f7b7d76f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741b1d50 con 0x55b24702d3f0 2024-03-21T11:25:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.493+0000 7f7b7d76f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741b0a30 con 0x55b247028960 2024-03-21T11:25:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.493+0000 7f7b73e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b247028960 0x55b24702f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.493+0000 7f7b73e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b247028960 0x55b24702f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48866/0 (socket says 172.21.15.47:48866) 2024-03-21T11:25:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.493+0000 7f7b73e00640 1 -- 172.21.15.47:0/2196741899 learned_addr learned my addr 172.21.15.47:0/2196741899 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.493+0000 7f7b78a00640 1 --2- 172.21.15.47:0/2196741899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702dde0 0x55b24702f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.493+0000 7f7b73400640 1 --2- 172.21.15.47:0/2196741899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b24702d3f0 0x55b24703eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.494+0000 7f7b73e00640 1 -- 172.21.15.47:0/2196741899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b24702d3f0 msgr2=0x55b24703eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.494+0000 7f7b73e00640 1 --2- 172.21.15.47:0/2196741899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b24702d3f0 0x55b24703eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.494+0000 7f7b73e00640 1 -- 172.21.15.47:0/2196741899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702dde0 msgr2=0x55b24702f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.494+0000 7f7b73e00640 1 --2- 172.21.15.47:0/2196741899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702dde0 0x55b24702f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.494+0000 7f7b73e00640 1 -- 172.21.15.47:0/2196741899 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2470296d0 con 0x55b247028960 2024-03-21T11:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.494+0000 7f7b73400640 1 --2- 172.21.15.47:0/2196741899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b24702d3f0 0x55b24703eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.494+0000 7f7b78a00640 1 --2- 172.21.15.47:0/2196741899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702dde0 0x55b24702f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.494+0000 7f7b73e00640 1 --2- 172.21.15.47:0/2196741899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b247028960 0x55b24702f0a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009b40 tx=0x7f7b64005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99e2e7a8182457f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.494+0000 7f7b5fe00640 1 -- 172.21.15.47:0/2196741899 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b640053d0 con 0x55b247028960 2024-03-21T11:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.494+0000 7f7b5fe00640 1 -- 172.21.15.47:0/2196741899 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b64003ad0 con 0x55b247028960 2024-03-21T11:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.494+0000 7f7b5fe00640 1 -- 172.21.15.47:0/2196741899 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b64019820 con 0x55b247028960 2024-03-21T11:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.495+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2196741899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b247028960 msgr2=0x55b24702f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.495+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2196741899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b247028960 0x55b24702f0a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009b40 tx=0x7f7b64005f90 comp rx=0 tx=0).stop 2024-03-21T11:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.495+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2196741899 shutdown_connections 2024-03-21T11:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.495+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2196741899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b24702dde0 0x55b24702f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.495+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2196741899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b24702d3f0 0x55b24703eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.495+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2196741899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b247028960 0x55b24702f0a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.495+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2196741899 >> 172.21.15.47:0/2196741899 conn(0x55b247032a50 msgr2=0x55b247028d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.495+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2196741899 shutdown_connections 2024-03-21T11:25:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.496+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2196741899 wait complete. 2024-03-21T11:25:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.496+0000 7f7b7d76f740 1 -- start start 2024-03-21T11:25:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.496+0000 7f7b7d76f740 1 Processor -- start 2024-03-21T11:25:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b7d76f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b247028960 0x55b24703c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b7d76f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702d3f0 0x55b24703c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b7d76f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b24702dde0 0x55b24703d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b7d76f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741b16f0 con 0x55b247028960 2024-03-21T11:25:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b7d76f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741b1090 con 0x55b24702dde0 2024-03-21T11:25:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b7d76f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741ae590 con 0x55b24702d3f0 2024-03-21T11:25:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b7d76f740 10 client.0 _ll_register_callbacks cb 0x7f7b6c009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702d3f0 0x55b24703c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b73e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b247028960 0x55b24703c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702d3f0 0x55b24703c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48868/0 (socket says 172.21.15.47:48868) 2024-03-21T11:25:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b73400640 1 -- 172.21.15.47:0/2240353014 learned_addr learned my addr 172.21.15.47:0/2240353014 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b78a00640 1 --2- 172.21.15.47:0/2240353014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b24702dde0 0x55b24703d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b73400640 1 -- 172.21.15.47:0/2240353014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b24702dde0 msgr2=0x55b24703d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b73400640 1 --2- 172.21.15.47:0/2240353014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b24702dde0 0x55b24703d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b73400640 1 -- 172.21.15.47:0/2240353014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b247028960 msgr2=0x55b24703c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b73400640 1 --2- 172.21.15.47:0/2240353014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b247028960 0x55b24703c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b73400640 1 -- 172.21.15.47:0/2240353014 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b24703de40 con 0x55b24702d3f0 2024-03-21T11:25:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.497+0000 7f7b73e00640 1 --2- 172.21.15.47:0/2240353014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b247028960 0x55b24703c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.498+0000 7f7b73400640 1 --2- 172.21.15.47:0/2240353014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702d3f0 0x55b24703c840 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009230 tx=0x7f7b60009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76e7967de0f5bca2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.498+0000 7f7b5d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:25:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.498+0000 7f7b5d600640 1 -- 172.21.15.47:0/2240353014 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60053070 con 0x55b24702d3f0 2024-03-21T11:25:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.498+0000 7f7b5d600640 1 -- 172.21.15.47:0/2240353014 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7b600046c0 con 0x55b24702d3f0 2024-03-21T11:25:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.498+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2240353014 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b64003ad0 con 0x55b24702d3f0 2024-03-21T11:25:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.499+0000 7f7b5d600640 1 -- 172.21.15.47:0/2240353014 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b600048c0 con 0x55b24702d3f0 2024-03-21T11:25:35.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.499+0000 7f7b7d76f740 10 client.26197 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.499+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2240353014 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b24709eec0 con 0x55b24702d3f0 2024-03-21T11:25:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.499+0000 7f7b5d600640 1 -- 172.21.15.47:0/2240353014 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b64003ad0 con 0x55b24702d3f0 2024-03-21T11:25:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.499+0000 7f7b5d600640 1 -- 172.21.15.47:0/2240353014 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7b6004c040 con 0x55b24702d3f0 2024-03-21T11:25:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.499+0000 7f7b7d76f740 10 client.26197 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.499+0000 7f7b7d76f740 10 client.26197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.499+0000 7f7b7d76f740 20 client.26197 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.499+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2240353014 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b247032f50 0x55b2470a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.499+0000 7f7b7d76f740 4 client.26197 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:25:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.499+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2240353014 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b246f9a0c0 con 0x55b247032f50 2024-03-21T11:25:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.500+0000 7f7b73e00640 1 --2- 172.21.15.47:0/2240353014 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b247032f50 0x55b2470a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.500+0000 7f7b73e00640 1 --2- 172.21.15.47:0/2240353014 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b247032f50 0x55b2470a18a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b2833458bdc1228 server_cookie=b17a79c3f9b941e5 in_seq=0 out_seq=0 2024-03-21T11:25:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.500+0000 7f7b5d600640 10 client.26197 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.724+0000 7f7b5d600640 1 -- 172.21.15.47:0/2240353014 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b246f9a0c0 con 0x55b247032f50 2024-03-21T11:25:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.724+0000 7f7b5d600640 10 client.26197 handle_command_reply: tid=0 2024-03-21T11:25:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.724+0000 7f7b7d76f740 1 client.26197 shutdown 2024-03-21T11:25:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.724+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2240353014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702d3f0 msgr2=0x55b24703c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.724+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2240353014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702d3f0 0x55b24703c840 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009230 tx=0x7f7b60009260 comp rx=0 tx=0).stop 2024-03-21T11:25:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.724+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2240353014 shutdown_connections 2024-03-21T11:25:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.724+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2240353014 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b247032f50 0x55b2470a18a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.724+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2240353014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b24702dde0 0x55b24703d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.724+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2240353014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b24702d3f0 0x55b24703c840 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.724+0000 7f7b5d600640 0 client.26197 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.724+0000 7f7b7d76f740 1 --2- 172.21.15.47:0/2240353014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b247028960 0x55b24703c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.724+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2240353014 >> 172.21.15.47:0/2240353014 conn(0x55b247032a50 msgr2=0x55b2470322b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.725+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2240353014 shutdown_connections 2024-03-21T11:25:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.725+0000 7f7b7d76f740 1 -- 172.21.15.47:0/2240353014 wait complete. 2024-03-21T11:25:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.725+0000 7f7b7d76f740 20 client.26197 trim_cache size 0 max 16384 2024-03-21T11:25:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:25:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:25:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:25:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:25:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.725+0000 7f7b79400640 1 -- 172.21.15.47:0/2535162 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7b580a2f10 msgr2=0x7f7b580a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.725+0000 7f7b79400640 1 --2- 172.21.15.47:0/2535162 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7b580a2f10 0x7f7b580a53d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7b6002c720 tx=0x7f7b60044040 comp rx=0 tx=0).stop 2024-03-21T11:25:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.726+0000 7f7b79400640 1 -- 172.21.15.47:0/2535162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b741096d0 msgr2=0x7f7b741a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.726+0000 7f7b79400640 1 --2- 172.21.15.47:0/2535162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b741096d0 0x7f7b741a53e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00efd0 tx=0x7f7b6c00c6a0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.726+0000 7f7b79400640 1 -- 172.21.15.47:0/2535162 shutdown_connections 2024-03-21T11:25:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.726+0000 7f7b79400640 1 --2- 172.21.15.47:0/2535162 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7b580a2f10 0x7f7b580a53d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.726+0000 7f7b79400640 1 --2- 172.21.15.47:0/2535162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b741096d0 0x7f7b741a53e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.726+0000 7f7b79400640 1 --2- 172.21.15.47:0/2535162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b74108d10 0x7f7b7419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.726+0000 7f7b79400640 1 --2- 172.21.15.47:0/2535162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b74107b20 0x7f7b7419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.726+0000 7f7b79400640 1 -- 172.21.15.47:0/2535162 >> 172.21.15.47:0/2535162 conn(0x7f7b74076ae0 msgr2=0x7f7b7410c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.726+0000 7f7b79400640 1 -- 172.21.15.47:0/2535162 shutdown_connections 2024-03-21T11:25:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:35.726+0000 7f7b79400640 1 -- 172.21.15.47:0/2535162 wait complete. 2024-03-21T11:25:36.487 INFO:tasks.workunit.client.0.smithi047.stdout: 1 36292 10.63 MB/sec warmup 20 sec latency 106.840 ms 2024-03-21T11:25:36.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:36 smithi100 ceph-mon[36245]: pgmap v468: 129 pgs: 129 active+clean; 14 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 983 KiB/s wr, 225 op/s 2024-03-21T11:25:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:36 smithi102 ceph-mon[36748]: pgmap v468: 129 pgs: 129 active+clean; 14 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 983 KiB/s wr, 225 op/s 2024-03-21T11:25:36.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:36 smithi047 ceph-mon[31351]: pgmap v468: 129 pgs: 129 active+clean; 14 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 983 KiB/s wr, 225 op/s 2024-03-21T11:25:37.487 INFO:tasks.workunit.client.0.smithi047.stdout: 1 38515 10.70 MB/sec warmup 21 sec latency 10.343 ms 2024-03-21T11:25:38.488 INFO:tasks.workunit.client.0.smithi047.stdout: 1 39845 10.53 MB/sec warmup 22 sec latency 12.432 ms 2024-03-21T11:25:38.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:38 smithi100 ceph-mon[36245]: pgmap v469: 129 pgs: 129 active+clean; 7.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.1 MiB/s wr, 259 op/s 2024-03-21T11:25:38.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:38 smithi102 ceph-mon[36748]: pgmap v469: 129 pgs: 129 active+clean; 7.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.1 MiB/s wr, 259 op/s 2024-03-21T11:25:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:38 smithi047 ceph-mon[31351]: pgmap v469: 129 pgs: 129 active+clean; 7.1 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.1 MiB/s wr, 259 op/s 2024-03-21T11:25:39.488 INFO:tasks.workunit.client.0.smithi047.stdout: 1 42043 10.60 MB/sec warmup 23 sec latency 15.256 ms 2024-03-21T11:25:40.488 INFO:tasks.workunit.client.0.smithi047.stdout: 1 43481 10.46 MB/sec warmup 24 sec latency 8.642 ms 2024-03-21T11:25:40.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:40 smithi100 ceph-mon[36245]: pgmap v470: 129 pgs: 129 active+clean; 10 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.1 MiB/s wr, 255 op/s 2024-03-21T11:25:40.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:40 smithi102 ceph-mon[36748]: pgmap v470: 129 pgs: 129 active+clean; 10 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.1 MiB/s wr, 255 op/s 2024-03-21T11:25:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:40 smithi047 ceph-mon[31351]: pgmap v470: 129 pgs: 129 active+clean; 10 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.1 MiB/s wr, 255 op/s 2024-03-21T11:25:41.488 INFO:tasks.workunit.client.0.smithi047.stdout: 1 45568 10.52 MB/sec warmup 25 sec latency 12.587 ms 2024-03-21T11:25:42.488 INFO:tasks.workunit.client.0.smithi047.stdout: 1 46831 10.38 MB/sec warmup 26 sec latency 10.773 ms 2024-03-21T11:25:42.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:42 smithi100 ceph-mon[36245]: pgmap v471: 129 pgs: 129 active+clean; 10 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 997 KiB/s wr, 267 op/s 2024-03-21T11:25:42.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:42 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:25:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:42 smithi102 ceph-mon[36748]: pgmap v471: 129 pgs: 129 active+clean; 10 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 997 KiB/s wr, 267 op/s 2024-03-21T11:25:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:42 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:25:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:42 smithi047 ceph-mon[31351]: pgmap v471: 129 pgs: 129 active+clean; 10 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 997 KiB/s wr, 267 op/s 2024-03-21T11:25:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:42 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:25:43.488 INFO:tasks.workunit.client.0.smithi047.stdout: 1 48912 10.39 MB/sec warmup 27 sec latency 11.622 ms 2024-03-21T11:25:44.488 INFO:tasks.workunit.client.0.smithi047.stdout: 1 50243 10.31 MB/sec warmup 28 sec latency 16.844 ms 2024-03-21T11:25:44.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:44 smithi100 ceph-mon[36245]: pgmap v472: 129 pgs: 129 active+clean; 14 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.3 MiB/s wr, 323 op/s 2024-03-21T11:25:44.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:44 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:25:44.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:44 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:25:44.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:44 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:25:44.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:44 smithi102 ceph-mon[36748]: pgmap v472: 129 pgs: 129 active+clean; 14 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.3 MiB/s wr, 323 op/s 2024-03-21T11:25:44.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:44 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:25:44.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:44 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:25:44.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:44 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:25:44.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:44 smithi047 ceph-mon[31351]: pgmap v472: 129 pgs: 129 active+clean; 14 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.3 MiB/s wr, 323 op/s 2024-03-21T11:25:44.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:44 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:25:44.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:44 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:25:44.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:44 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:25:45.488 INFO:tasks.workunit.client.0.smithi047.stdout: 1 52098 10.20 MB/sec warmup 29 sec latency 20.547 ms 2024-03-21T11:25:46.488 INFO:tasks.workunit.client.0.smithi047.stdout: 1 53633 10.26 MB/sec warmup 30 sec latency 10.489 ms 2024-03-21T11:25:46.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:46 smithi100 ceph-mon[36245]: pgmap v473: 129 pgs: 129 active+clean; 14 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 959 KiB/s wr, 263 op/s 2024-03-21T11:25:46.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:46 smithi102 ceph-mon[36748]: pgmap v473: 129 pgs: 129 active+clean; 14 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 959 KiB/s wr, 263 op/s 2024-03-21T11:25:46.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:46 smithi047 ceph-mon[31351]: pgmap v473: 129 pgs: 129 active+clean; 14 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 959 KiB/s wr, 263 op/s 2024-03-21T11:25:47.490 INFO:tasks.workunit.client.0.smithi047.stdout: 1 54928 10.10 MB/sec warmup 31 sec latency 34.866 ms 2024-03-21T11:25:48.490 INFO:tasks.workunit.client.0.smithi047.stdout: 1 56870 10.17 MB/sec warmup 32 sec latency 58.845 ms 2024-03-21T11:25:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:48 smithi100 ceph-mon[36245]: pgmap v474: 129 pgs: 129 active+clean; 15 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.0 MiB/s wr, 269 op/s 2024-03-21T11:25:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:48 smithi102 ceph-mon[36748]: pgmap v474: 129 pgs: 129 active+clean; 15 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.0 MiB/s wr, 269 op/s 2024-03-21T11:25:48.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:48 smithi047 ceph-mon[31351]: pgmap v474: 129 pgs: 129 active+clean; 15 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.0 MiB/s wr, 269 op/s 2024-03-21T11:25:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.880+0000 7faf5b200640 1 -- 172.21.15.47:0/3618479563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf5c104910 msgr2=0x7faf5c104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.880+0000 7faf5b200640 1 --2- 172.21.15.47:0/3618479563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf5c104910 0x7faf5c104d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faf44009900 tx=0x7faf4402bc40 comp rx=0 tx=0).stop 2024-03-21T11:25:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.880+0000 7faf5b200640 1 -- 172.21.15.47:0/3618479563 shutdown_connections 2024-03-21T11:25:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.880+0000 7faf5b200640 1 --2- 172.21.15.47:0/3618479563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf5c1052d0 0x7faf5c10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.880+0000 7faf5b200640 1 --2- 172.21.15.47:0/3618479563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf5c104910 0x7faf5c104d90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.880+0000 7faf5b200640 1 --2- 172.21.15.47:0/3618479563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf5c103720 0x7faf5c103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.880+0000 7faf5b200640 1 -- 172.21.15.47:0/3618479563 >> 172.21.15.47:0/3618479563 conn(0x7faf5c0fecb0 msgr2=0x7faf5c1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.880+0000 7faf5b200640 1 -- 172.21.15.47:0/3618479563 shutdown_connections 2024-03-21T11:25:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.880+0000 7faf5b200640 1 -- 172.21.15.47:0/3618479563 wait complete. 2024-03-21T11:25:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.881+0000 7faf5b200640 1 Processor -- start 2024-03-21T11:25:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.881+0000 7faf5b200640 1 -- start start 2024-03-21T11:25:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.881+0000 7faf5b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf5c103720 0x7faf5c199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf5b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf5c104910 0x7faf5c199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf5b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf5c1052d0 0x7faf5c1a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf5b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c10f380 con 0x7faf5c103720 2024-03-21T11:25:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf5b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c10f200 con 0x7faf5c104910 2024-03-21T11:25:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf5b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c10f500 con 0x7faf5c1052d0 2024-03-21T11:25:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf59400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf5c104910 0x7faf5c199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf59e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf5c103720 0x7faf5c199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf5a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf5c1052d0 0x7faf5c1a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf59400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf5c104910 0x7faf5c199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53480/0 (socket says 172.21.15.47:53480) 2024-03-21T11:25:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf59400640 1 -- 172.21.15.47:0/1648238325 learned_addr learned my addr 172.21.15.47:0/1648238325 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf5a800640 1 -- 172.21.15.47:0/1648238325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf5c104910 msgr2=0x7faf5c199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf5a800640 1 --2- 172.21.15.47:0/1648238325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf5c104910 0x7faf5c199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf5a800640 1 -- 172.21.15.47:0/1648238325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf5c103720 msgr2=0x7faf5c199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf5a800640 1 --2- 172.21.15.47:0/1648238325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf5c103720 0x7faf5c199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.882+0000 7faf5a800640 1 -- 172.21.15.47:0/1648238325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c1a1650 con 0x7faf5c1052d0 2024-03-21T11:25:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.883+0000 7faf5a800640 1 --2- 172.21.15.47:0/1648238325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf5c1052d0 0x7faf5c1a0fe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00e970 tx=0x7faf4c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.883+0000 7faf52a00640 1 -- 172.21.15.47:0/1648238325 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c00ccc0 con 0x7faf5c1052d0 2024-03-21T11:25:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.883+0000 7faf52a00640 1 -- 172.21.15.47:0/1648238325 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf4c0035f0 con 0x7faf5c1052d0 2024-03-21T11:25:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.883+0000 7faf52a00640 1 -- 172.21.15.47:0/1648238325 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c00ce60 con 0x7faf5c1052d0 2024-03-21T11:25:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.883+0000 7faf5b200640 1 -- 172.21.15.47:0/1648238325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c1a1940 con 0x7faf5c1052d0 2024-03-21T11:25:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.883+0000 7faf5b200640 1 -- 172.21.15.47:0/1648238325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5c1a1e60 con 0x7faf5c1052d0 2024-03-21T11:25:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.887+0000 7faf63659740 1 Processor -- start 2024-03-21T11:25:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.887+0000 7faf52a00640 1 -- 172.21.15.47:0/1648238325 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7faf4c0037a0 con 0x7faf5c1052d0 2024-03-21T11:25:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.887+0000 7faf63659740 1 -- start start 2024-03-21T11:25:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.887+0000 7faf63659740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c873f0 0x562ee2c87810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.887+0000 7faf63659740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c87de0 0x562ee2c8c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.887+0000 7faf63659740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c8cf50 0x562ee2c8f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.887+0000 7faf63659740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee2c4a3b0 con 0x562ee2c87de0 2024-03-21T11:25:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.887+0000 7faf63659740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee2bec830 con 0x562ee2c8cf50 2024-03-21T11:25:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.887+0000 7faf63659740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee2b955d0 con 0x562ee2c873f0 2024-03-21T11:25:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.887+0000 7faf59400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c873f0 0x562ee2c87810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.888+0000 7faf59400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c873f0 0x562ee2c87810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53448/0 (socket says 172.21.15.47:53448) 2024-03-21T11:25:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.888+0000 7faf59400640 1 -- 172.21.15.47:0/2250820079 learned_addr learned my addr 172.21.15.47:0/2250820079 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.888+0000 7faf59400640 1 --2- 172.21.15.47:0/2250820079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c8cf50 0x562ee2c8f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.888+0000 7faf59e00640 1 --2- 172.21.15.47:0/2250820079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c87de0 0x562ee2c8c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.888+0000 7faf59400640 1 -- 172.21.15.47:0/2250820079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c8cf50 msgr2=0x562ee2c8f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.888+0000 7faf59400640 1 --2- 172.21.15.47:0/2250820079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c8cf50 0x562ee2c8f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.888+0000 7faf59400640 1 -- 172.21.15.47:0/2250820079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c87de0 msgr2=0x562ee2c8c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.888+0000 7faf59400640 1 --2- 172.21.15.47:0/2250820079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c87de0 0x562ee2c8c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.888+0000 7faf59400640 1 -- 172.21.15.47:0/2250820079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ee2bad2c0 con 0x562ee2c873f0 2024-03-21T11:25:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.888+0000 7faf52a00640 1 --2- 172.21.15.47:0/1648238325 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faf400a2f90 0x7faf400a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.888+0000 7faf52a00640 1 -- 172.21.15.47:0/1648238325 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7faf4c0cf910 con 0x7faf5c1052d0 2024-03-21T11:25:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.888+0000 7faf59400640 1 --2- 172.21.15.47:0/2250820079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c873f0 0x562ee2c87810 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faf4402c720 tx=0x7faf44005b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ec15dc676f934e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.889+0000 7faf59e00640 1 --2- 172.21.15.47:0/1648238325 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faf400a2f90 0x7faf400a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.889+0000 7faf4b000640 1 -- 172.21.15.47:0/2250820079 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44004230 con 0x562ee2c873f0 2024-03-21T11:25:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.889+0000 7faf4b000640 1 -- 172.21.15.47:0/2250820079 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf440043d0 con 0x562ee2c873f0 2024-03-21T11:25:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.889+0000 7faf4b000640 1 -- 172.21.15.47:0/2250820079 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44005330 con 0x562ee2c873f0 2024-03-21T11:25:48.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.890+0000 7faf63659740 1 -- 172.21.15.47:0/2250820079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c873f0 msgr2=0x562ee2c87810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.890+0000 7faf63659740 1 --2- 172.21.15.47:0/2250820079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c873f0 0x562ee2c87810 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faf4402c720 tx=0x7faf44005b70 comp rx=0 tx=0).stop 2024-03-21T11:25:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.890+0000 7faf59e00640 1 --2- 172.21.15.47:0/1648238325 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faf400a2f90 0x7faf400a5450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x562ee2c881e0 tx=0x7faf54009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.890+0000 7faf63659740 1 -- 172.21.15.47:0/2250820079 shutdown_connections 2024-03-21T11:25:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.890+0000 7faf63659740 1 --2- 172.21.15.47:0/2250820079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c8cf50 0x562ee2c8f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.890+0000 7faf63659740 1 --2- 172.21.15.47:0/2250820079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c87de0 0x562ee2c8c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.890+0000 7faf63659740 1 --2- 172.21.15.47:0/2250820079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c873f0 0x562ee2c87810 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.890+0000 7faf63659740 1 -- 172.21.15.47:0/2250820079 >> 172.21.15.47:0/2250820079 conn(0x562ee2c82960 msgr2=0x562ee2c82d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.890+0000 7faf63659740 1 -- 172.21.15.47:0/2250820079 shutdown_connections 2024-03-21T11:25:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.890+0000 7faf63659740 1 -- 172.21.15.47:0/2250820079 wait complete. 2024-03-21T11:25:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.890+0000 7faf63659740 1 -- start start 2024-03-21T11:25:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.891+0000 7faf63659740 1 Processor -- start 2024-03-21T11:25:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.891+0000 7faf63659740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c873f0 0x562ee2c915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.891+0000 7faf63659740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c87de0 0x562ee2c91b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.891+0000 7faf63659740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c8cf50 0x562ee2c98b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.891+0000 7faf63659740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee2ba8a20 con 0x562ee2c87de0 2024-03-21T11:25:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.891+0000 7faf63659740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee2be88d0 con 0x562ee2c873f0 2024-03-21T11:25:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.891+0000 7faf63659740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee2c8fea0 con 0x562ee2c8cf50 2024-03-21T11:25:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.891+0000 7faf63659740 10 client.0 _ll_register_callbacks cb 0x562ee2be51c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.891+0000 7faf59e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c873f0 0x562ee2c915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.891+0000 7faf59400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c87de0 0x562ee2c91b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.891+0000 7faf59e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c873f0 0x562ee2c915d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53494/0 (socket says 172.21.15.47:53494) 2024-03-21T11:25:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.891+0000 7faf59e00640 1 -- 172.21.15.47:0/4170261123 learned_addr learned my addr 172.21.15.47:0/4170261123 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.891+0000 7faf5a800640 1 --2- 172.21.15.47:0/4170261123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c8cf50 0x562ee2c98b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.892+0000 7faf5a800640 1 -- 172.21.15.47:0/4170261123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c873f0 msgr2=0x562ee2c915d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.892+0000 7faf5a800640 1 --2- 172.21.15.47:0/4170261123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c873f0 0x562ee2c915d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.892+0000 7faf5a800640 1 -- 172.21.15.47:0/4170261123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c87de0 msgr2=0x562ee2c91b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.892+0000 7faf5a800640 1 --2- 172.21.15.47:0/4170261123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c87de0 0x562ee2c91b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.892+0000 7faf5a800640 1 -- 172.21.15.47:0/4170261123 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ee2901210 con 0x562ee2c8cf50 2024-03-21T11:25:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.892+0000 7faf59e00640 1 --2- 172.21.15.47:0/4170261123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c873f0 0x562ee2c915d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.892+0000 7faf59400640 1 --2- 172.21.15.47:0/4170261123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c87de0 0x562ee2c91b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:25:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.892+0000 7faf5a800640 1 --2- 172.21.15.47:0/4170261123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c8cf50 0x562ee2c98b50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00c6a0 tx=0x7faf4c09c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ffe192646546849 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.892+0000 7faf3f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:25:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.892+0000 7faf3f400640 1 -- 172.21.15.47:0/4170261123 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c0a46c0 con 0x562ee2c8cf50 2024-03-21T11:25:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.892+0000 7faf3f400640 1 -- 172.21.15.47:0/4170261123 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7faf4c09b100 con 0x562ee2c8cf50 2024-03-21T11:25:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.892+0000 7faf63659740 1 -- 172.21.15.47:0/4170261123 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf440043d0 con 0x562ee2c8cf50 2024-03-21T11:25:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.893+0000 7faf3f400640 1 -- 172.21.15.47:0/4170261123 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf4c09b300 con 0x562ee2c8cf50 2024-03-21T11:25:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.893+0000 7faf63659740 10 client.26209 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.893+0000 7faf63659740 1 -- 172.21.15.47:0/4170261123 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562ee29cd620 con 0x562ee2c8cf50 2024-03-21T11:25:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.893+0000 7faf3f400640 1 -- 172.21.15.47:0/4170261123 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf440043d0 con 0x562ee2c8cf50 2024-03-21T11:25:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.893+0000 7faf3f400640 1 -- 172.21.15.47:0/4170261123 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7faf4c097c00 con 0x562ee2c8cf50 2024-03-21T11:25:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.893+0000 7faf63659740 10 client.26209 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.893+0000 7faf63659740 10 client.26209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.893+0000 7faf63659740 20 client.26209 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.893+0000 7faf63659740 1 --2- 172.21.15.47:0/4170261123 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ee2c9d480 0x562ee2c9f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.893+0000 7faf63659740 4 client.26209 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:25:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.893+0000 7faf63659740 1 -- 172.21.15.47:0/4170261123 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ee2bf9d70 con 0x562ee2c9d480 2024-03-21T11:25:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.894+0000 7faf59e00640 1 --2- 172.21.15.47:0/4170261123 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ee2c9d480 0x562ee2c9f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.894+0000 7faf59e00640 1 --2- 172.21.15.47:0/4170261123 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ee2c9d480 0x562ee2c9f8a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19b45da7ba106695 server_cookie=7b15cf933d8841a3 in_seq=0 out_seq=0 2024-03-21T11:25:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.894+0000 7faf3f400640 10 client.26209 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.895+0000 7faf3f400640 1 -- 172.21.15.47:0/4170261123 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562ee2bf9d70 con 0x562ee2c9d480 2024-03-21T11:25:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.895+0000 7faf3f400640 10 client.26209 handle_command_reply: tid=0 2024-03-21T11:25:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.895+0000 7faf63659740 1 client.26209 shutdown 2024-03-21T11:25:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.895+0000 7faf63659740 1 -- 172.21.15.47:0/4170261123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c8cf50 msgr2=0x562ee2c98b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.895+0000 7faf63659740 1 --2- 172.21.15.47:0/4170261123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c8cf50 0x562ee2c98b50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00c6a0 tx=0x7faf4c09c4f0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.895+0000 7faf63659740 1 -- 172.21.15.47:0/4170261123 shutdown_connections 2024-03-21T11:25:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.895+0000 7faf63659740 1 --2- 172.21.15.47:0/4170261123 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ee2c9d480 0x562ee2c9f8a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.895+0000 7faf63659740 1 --2- 172.21.15.47:0/4170261123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c8cf50 0x562ee2c98b50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.895+0000 7faf63659740 1 --2- 172.21.15.47:0/4170261123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c87de0 0x562ee2c91b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.895+0000 7faf63659740 1 --2- 172.21.15.47:0/4170261123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c873f0 0x562ee2c915d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.896+0000 7faf63659740 1 -- 172.21.15.47:0/4170261123 >> 172.21.15.47:0/4170261123 conn(0x562ee2c82960 msgr2=0x562ee2c81b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.896+0000 7faf3f400640 0 client.26209 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.896+0000 7faf63659740 1 -- 172.21.15.47:0/4170261123 shutdown_connections 2024-03-21T11:25:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.896+0000 7faf63659740 1 -- 172.21.15.47:0/4170261123 wait complete. 2024-03-21T11:25:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.896+0000 7faf63659740 20 client.26209 trim_cache size 0 max 16384 2024-03-21T11:25:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.912+0000 7faf63659740 1 Processor -- start 2024-03-21T11:25:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.912+0000 7faf63659740 1 -- start start 2024-03-21T11:25:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.912+0000 7faf63659740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c82960 0x562ee2c7d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.912+0000 7faf63659740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c873f0 0x562ee2c98b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.912+0000 7faf63659740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c87de0 0x562ee2c89100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.912+0000 7faf63659740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c1ac0f0 con 0x562ee2c873f0 2024-03-21T11:25:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.912+0000 7faf63659740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c1ada70 con 0x562ee2c82960 2024-03-21T11:25:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.912+0000 7faf63659740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c1ac750 con 0x562ee2c87de0 2024-03-21T11:25:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.913+0000 7faf5a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c87de0 0x562ee2c89100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.913+0000 7faf59e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c82960 0x562ee2c7d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.913+0000 7faf5a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c87de0 0x562ee2c89100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53466/0 (socket says 172.21.15.47:53466) 2024-03-21T11:25:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.913+0000 7faf5a800640 1 -- 172.21.15.47:0/2464787810 learned_addr learned my addr 172.21.15.47:0/2464787810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.913+0000 7faf59400640 1 --2- 172.21.15.47:0/2464787810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c873f0 0x562ee2c98b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.913+0000 7faf5a800640 1 -- 172.21.15.47:0/2464787810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c82960 msgr2=0x562ee2c7d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.913+0000 7faf5a800640 1 --2- 172.21.15.47:0/2464787810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c82960 0x562ee2c7d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.913+0000 7faf5a800640 1 -- 172.21.15.47:0/2464787810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c873f0 msgr2=0x562ee2c98b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.913+0000 7faf5a800640 1 --2- 172.21.15.47:0/2464787810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c873f0 0x562ee2c98b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.913+0000 7faf5a800640 1 -- 172.21.15.47:0/2464787810 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ee2c877d0 con 0x562ee2c87de0 2024-03-21T11:25:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.913+0000 7faf59400640 1 --2- 172.21.15.47:0/2464787810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c873f0 0x562ee2c98b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:25:48.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.913+0000 7faf5a800640 1 --2- 172.21.15.47:0/2464787810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c87de0 0x562ee2c89100 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0cd3f0 tx=0x7faf4c0058f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1f593aa1b907777 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:48.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.913+0000 7faf4a600640 1 -- 172.21.15.47:0/2464787810 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c0a46c0 con 0x562ee2c87de0 2024-03-21T11:25:48.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.914+0000 7faf4a600640 1 -- 172.21.15.47:0/2464787810 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf4c09af00 con 0x562ee2c87de0 2024-03-21T11:25:48.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.914+0000 7faf4a600640 1 -- 172.21.15.47:0/2464787810 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c097a00 con 0x562ee2c87de0 2024-03-21T11:25:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.914+0000 7faf63659740 1 -- 172.21.15.47:0/2464787810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c87de0 msgr2=0x562ee2c89100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.914+0000 7faf63659740 1 --2- 172.21.15.47:0/2464787810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c87de0 0x562ee2c89100 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0cd3f0 tx=0x7faf4c0058f0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.914+0000 7faf63659740 1 -- 172.21.15.47:0/2464787810 shutdown_connections 2024-03-21T11:25:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.914+0000 7faf63659740 1 --2- 172.21.15.47:0/2464787810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c87de0 0x562ee2c89100 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.914+0000 7faf63659740 1 --2- 172.21.15.47:0/2464787810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c873f0 0x562ee2c98b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.914+0000 7faf63659740 1 --2- 172.21.15.47:0/2464787810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c82960 0x562ee2c7d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.914+0000 7faf63659740 1 -- 172.21.15.47:0/2464787810 >> 172.21.15.47:0/2464787810 conn(0x562ee2c889e0 msgr2=0x562ee2c915d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.914+0000 7faf63659740 1 -- 172.21.15.47:0/2464787810 shutdown_connections 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.914+0000 7faf63659740 1 -- 172.21.15.47:0/2464787810 wait complete. 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.915+0000 7faf63659740 1 -- start start 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.915+0000 7faf63659740 1 Processor -- start 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.915+0000 7faf63659740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c82960 0x562ee2c961f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.915+0000 7faf63659740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c873f0 0x562ee2c96730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf63659740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c87de0 0x562ee2c97740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf63659740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c1ad410 con 0x562ee2c82960 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf63659740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c1acdb0 con 0x562ee2c87de0 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf63659740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c1aa2b0 con 0x562ee2c873f0 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf63659740 10 client.0 _ll_register_callbacks cb 0x7faf4c097f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf59400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c873f0 0x562ee2c96730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf5a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c87de0 0x562ee2c97740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf5a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c87de0 0x562ee2c97740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53510/0 (socket says 172.21.15.47:53510) 2024-03-21T11:25:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf59400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c873f0 0x562ee2c96730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53468/0 (socket says 172.21.15.47:53468) 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf5a800640 1 -- 172.21.15.47:0/4259234269 learned_addr learned my addr 172.21.15.47:0/4259234269 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf59e00640 1 --2- 172.21.15.47:0/4259234269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c82960 0x562ee2c961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf5a800640 1 -- 172.21.15.47:0/4259234269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c873f0 msgr2=0x562ee2c96730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf5a800640 1 --2- 172.21.15.47:0/4259234269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c873f0 0x562ee2c96730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf5a800640 1 -- 172.21.15.47:0/4259234269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c82960 msgr2=0x562ee2c961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf5a800640 1 --2- 172.21.15.47:0/4259234269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c82960 0x562ee2c961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf5a800640 1 -- 172.21.15.47:0/4259234269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ee2c97d30 con 0x562ee2c87de0 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf5a800640 1 --2- 172.21.15.47:0/4259234269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c87de0 0x562ee2c97740 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf4c098110 tx=0x7faf4c09b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9192712c0478415 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.916+0000 7faf3f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.917+0000 7faf3f400640 1 -- 172.21.15.47:0/4259234269 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c0a46c0 con 0x562ee2c87de0 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.917+0000 7faf63659740 1 -- 172.21.15.47:0/4259234269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ee2bf40c0 con 0x562ee2c87de0 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.917+0000 7faf3f400640 1 -- 172.21.15.47:0/4259234269 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7faf4c07ad30 con 0x562ee2c87de0 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.917+0000 7faf3f400640 1 -- 172.21.15.47:0/4259234269 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf4c097a00 con 0x562ee2c87de0 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.917+0000 7faf63659740 10 client.26201 fetch_fsmap learned FSMap version 28 2024-03-21T11:25:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.917+0000 7faf63659740 1 -- 172.21.15.47:0/4259234269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562ee2d01540 con 0x562ee2c87de0 2024-03-21T11:25:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.917+0000 7faf3f400640 1 -- 172.21.15.47:0/4259234269 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562ee2bf40c0 con 0x562ee2c87de0 2024-03-21T11:25:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.917+0000 7faf3f400640 1 -- 172.21.15.47:0/4259234269 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7faf4c0bac80 con 0x562ee2c87de0 2024-03-21T11:25:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.917+0000 7faf63659740 10 client.26201 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:25:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.917+0000 7faf63659740 10 client.26201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:25:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.917+0000 7faf63659740 20 client.26201 populate_metadata read hostname 'smithi047' 2024-03-21T11:25:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.917+0000 7faf63659740 1 --2- 172.21.15.47:0/4259234269 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ee2c8cf50 0x562ee2d03a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:25:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.917+0000 7faf63659740 4 client.26201 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:25:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.917+0000 7faf63659740 1 -- 172.21.15.47:0/4259234269 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ee2b725d0 con 0x562ee2c8cf50 2024-03-21T11:25:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.918+0000 7faf59e00640 1 --2- 172.21.15.47:0/4259234269 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ee2c8cf50 0x562ee2d03a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:25:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.918+0000 7faf59e00640 1 --2- 172.21.15.47:0/4259234269 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ee2c8cf50 0x562ee2d03a50 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddcb9f744f78292c server_cookie=3e30b93a68df3d80 in_seq=0 out_seq=0 2024-03-21T11:25:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:48.918+0000 7faf3f400640 10 client.26201 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.133+0000 7faf3f400640 1 -- 172.21.15.47:0/4259234269 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ee2b725d0 con 0x562ee2c8cf50 2024-03-21T11:25:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.133+0000 7faf3f400640 10 client.26201 handle_command_reply: tid=0 2024-03-21T11:25:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.134+0000 7faf63659740 1 client.26201 shutdown 2024-03-21T11:25:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.134+0000 7faf63659740 1 -- 172.21.15.47:0/4259234269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c87de0 msgr2=0x562ee2c97740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.134+0000 7faf63659740 1 --2- 172.21.15.47:0/4259234269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c87de0 0x562ee2c97740 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf4c098110 tx=0x7faf4c09b6e0 comp rx=0 tx=0).stop 2024-03-21T11:25:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.134+0000 7faf63659740 1 -- 172.21.15.47:0/4259234269 shutdown_connections 2024-03-21T11:25:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.134+0000 7faf63659740 1 --2- 172.21.15.47:0/4259234269 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ee2c8cf50 0x562ee2d03a50 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.134+0000 7faf63659740 1 --2- 172.21.15.47:0/4259234269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ee2c87de0 0x562ee2c97740 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.134+0000 7faf3f400640 0 client.26201 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:25:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.134+0000 7faf63659740 1 --2- 172.21.15.47:0/4259234269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ee2c873f0 0x562ee2c96730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.134+0000 7faf63659740 1 --2- 172.21.15.47:0/4259234269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ee2c82960 0x562ee2c961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.134+0000 7faf63659740 1 -- 172.21.15.47:0/4259234269 >> 172.21.15.47:0/4259234269 conn(0x562ee2c889e0 msgr2=0x562ee2c915d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.134+0000 7faf63659740 1 -- 172.21.15.47:0/4259234269 shutdown_connections 2024-03-21T11:25:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.134+0000 7faf63659740 1 -- 172.21.15.47:0/4259234269 wait complete. 2024-03-21T11:25:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.134+0000 7faf63659740 20 client.26201 trim_cache size 0 max 16384 2024-03-21T11:25:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:25:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:25:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:25:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:25:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.135+0000 7faf5b200640 1 -- 172.21.15.47:0/1648238325 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faf400a2f90 msgr2=0x7faf400a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.135+0000 7faf5b200640 1 --2- 172.21.15.47:0/1648238325 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faf400a2f90 0x7faf400a5450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x562ee2c881e0 tx=0x7faf54009500 comp rx=0 tx=0).stop 2024-03-21T11:25:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.135+0000 7faf5b200640 1 -- 172.21.15.47:0/1648238325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf5c1052d0 msgr2=0x7faf5c1a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:25:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.135+0000 7faf5b200640 1 --2- 172.21.15.47:0/1648238325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf5c1052d0 0x7faf5c1a0fe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00e970 tx=0x7faf4c00ee30 comp rx=0 tx=0).stop 2024-03-21T11:25:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.135+0000 7faf5b200640 1 -- 172.21.15.47:0/1648238325 shutdown_connections 2024-03-21T11:25:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.135+0000 7faf5b200640 1 --2- 172.21.15.47:0/1648238325 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faf400a2f90 0x7faf400a5450 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.135+0000 7faf5b200640 1 --2- 172.21.15.47:0/1648238325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf5c1052d0 0x7faf5c1a0fe0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.135+0000 7faf5b200640 1 --2- 172.21.15.47:0/1648238325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf5c104910 0x7faf5c199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.135+0000 7faf5b200640 1 --2- 172.21.15.47:0/1648238325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf5c103720 0x7faf5c199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:25:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.135+0000 7faf5b200640 1 -- 172.21.15.47:0/1648238325 >> 172.21.15.47:0/1648238325 conn(0x7faf5c0fecb0 msgr2=0x7faf5c107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:25:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.135+0000 7faf5b200640 1 -- 172.21.15.47:0/1648238325 shutdown_connections 2024-03-21T11:25:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:25:49.135+0000 7faf5b200640 1 -- 172.21.15.47:0/1648238325 wait complete. 2024-03-21T11:25:49.490 INFO:tasks.workunit.client.0.smithi047.stdout: 1 57607 9.92 MB/sec warmup 33 sec latency 104.256 ms 2024-03-21T11:25:50.490 INFO:tasks.workunit.client.0.smithi047.stdout: 1 59678 9.96 MB/sec warmup 34 sec latency 18.276 ms 2024-03-21T11:25:50.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:50 smithi100 ceph-mon[36245]: pgmap v475: 129 pgs: 129 active+clean; 26 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.9 MiB/s wr, 260 op/s 2024-03-21T11:25:50.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:50 smithi102 ceph-mon[36748]: pgmap v475: 129 pgs: 129 active+clean; 26 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.9 MiB/s wr, 260 op/s 2024-03-21T11:25:50.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:50 smithi047 ceph-mon[31351]: pgmap v475: 129 pgs: 129 active+clean; 26 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.9 MiB/s wr, 260 op/s 2024-03-21T11:25:51.490 INFO:tasks.workunit.client.0.smithi047.stdout: 1 60922 9.89 MB/sec warmup 35 sec latency 18.241 ms 2024-03-21T11:25:52.490 INFO:tasks.workunit.client.0.smithi047.stdout: 1 62725 9.81 MB/sec warmup 36 sec latency 14.284 ms 2024-03-21T11:25:52.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:52 smithi047 ceph-mon[31351]: pgmap v476: 129 pgs: 129 active+clean; 30 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-03-21T11:25:52.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:52 smithi100 ceph-mon[36245]: pgmap v476: 129 pgs: 129 active+clean; 30 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-03-21T11:25:52.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:52 smithi102 ceph-mon[36748]: pgmap v476: 129 pgs: 129 active+clean; 30 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-03-21T11:25:53.490 INFO:tasks.workunit.client.0.smithi047.stdout: 1 64090 9.86 MB/sec warmup 37 sec latency 62.077 ms 2024-03-21T11:25:54.490 INFO:tasks.workunit.client.0.smithi047.stdout: 1 64928 9.66 MB/sec warmup 38 sec latency 20.935 ms 2024-03-21T11:25:54.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:54 smithi100 ceph-mon[36245]: pgmap v477: 129 pgs: 129 active+clean; 35 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-03-21T11:25:54.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:54 smithi102 ceph-mon[36748]: pgmap v477: 129 pgs: 129 active+clean; 35 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-03-21T11:25:54.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:54 smithi047 ceph-mon[31351]: pgmap v477: 129 pgs: 129 active+clean; 35 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-03-21T11:25:55.491 INFO:tasks.workunit.client.0.smithi047.stdout: 1 67043 9.70 MB/sec warmup 39 sec latency 12.128 ms 2024-03-21T11:25:56.491 INFO:tasks.workunit.client.0.smithi047.stdout: 1 68131 9.61 MB/sec warmup 40 sec latency 29.075 ms 2024-03-21T11:25:56.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:56 smithi100 ceph-mon[36245]: pgmap v478: 129 pgs: 129 active+clean; 37 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-03-21T11:25:56.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:56 smithi102 ceph-mon[36748]: pgmap v478: 129 pgs: 129 active+clean; 37 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-03-21T11:25:56.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:56 smithi047 ceph-mon[31351]: pgmap v478: 129 pgs: 129 active+clean; 37 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-03-21T11:25:57.491 INFO:tasks.workunit.client.0.smithi047.stdout: 1 69878 9.55 MB/sec warmup 41 sec latency 12.282 ms 2024-03-21T11:25:58.491 INFO:tasks.workunit.client.0.smithi047.stdout: 1 71242 9.60 MB/sec warmup 42 sec latency 77.204 ms 2024-03-21T11:25:58.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:25:58 smithi100 ceph-mon[36245]: pgmap v479: 129 pgs: 129 active+clean; 39 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 3.0 MiB/s wr, 280 op/s 2024-03-21T11:25:58.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:25:58 smithi102 ceph-mon[36748]: pgmap v479: 129 pgs: 129 active+clean; 39 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 3.0 MiB/s wr, 280 op/s 2024-03-21T11:25:58.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:25:58 smithi047 ceph-mon[31351]: pgmap v479: 129 pgs: 129 active+clean; 39 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 3.0 MiB/s wr, 280 op/s 2024-03-21T11:25:59.491 INFO:tasks.workunit.client.0.smithi047.stdout: 1 72218 9.43 MB/sec warmup 43 sec latency 30.429 ms 2024-03-21T11:26:00.491 INFO:tasks.workunit.client.0.smithi047.stdout: 1 74299 9.50 MB/sec warmup 44 sec latency 11.870 ms 2024-03-21T11:26:00.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:00 smithi100 ceph-mon[36245]: pgmap v480: 129 pgs: 129 active+clean; 38 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 298 op/s 2024-03-21T11:26:00.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:00 smithi102 ceph-mon[36748]: pgmap v480: 129 pgs: 129 active+clean; 38 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 298 op/s 2024-03-21T11:26:00.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:00 smithi047 ceph-mon[31351]: pgmap v480: 129 pgs: 129 active+clean; 38 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 298 op/s 2024-03-21T11:26:01.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:26:01.306 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:26:01.492 INFO:tasks.workunit.client.0.smithi047.stdout: 1 75283 9.40 MB/sec warmup 45 sec latency 16.904 ms 2024-03-21T11:26:01.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.557+0000 7f863f200640 1 -- 172.21.15.47:0/1963471716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8640104990 msgr2=0x7f8640104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:01.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.557+0000 7f863f200640 1 --2- 172.21.15.47:0/1963471716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8640104990 0x7f8640104e10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8628009900 tx=0x7f862802bc30 comp rx=0 tx=0).stop 2024-03-21T11:26:01.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.558+0000 7f863f200640 1 -- 172.21.15.47:0/1963471716 shutdown_connections 2024-03-21T11:26:01.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.558+0000 7f863f200640 1 --2- 172.21.15.47:0/1963471716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8640105350 0x7f864010bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.558+0000 7f863f200640 1 --2- 172.21.15.47:0/1963471716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8640104990 0x7f8640104e10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.558+0000 7f863f200640 1 --2- 172.21.15.47:0/1963471716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86401037a0 0x7f8640103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.558+0000 7f863f200640 1 -- 172.21.15.47:0/1963471716 >> 172.21.15.47:0/1963471716 conn(0x7f86400fed10 msgr2=0x7f8640101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:01.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.558+0000 7f863f200640 1 -- 172.21.15.47:0/1963471716 shutdown_connections 2024-03-21T11:26:01.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.558+0000 7f863f200640 1 -- 172.21.15.47:0/1963471716 wait complete. 2024-03-21T11:26:01.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.559+0000 7f863f200640 1 Processor -- start 2024-03-21T11:26:01.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.559+0000 7f863f200640 1 -- start start 2024-03-21T11:26:01.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.559+0000 7f863f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86401037a0 0x7f864019ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.559+0000 7f863f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8640104990 0x7f864019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.559+0000 7f863f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8640105350 0x7f86401a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.559+0000 7f863f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864010f420 con 0x7f86401037a0 2024-03-21T11:26:01.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.559+0000 7f863f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864010f2a0 con 0x7f8640105350 2024-03-21T11:26:01.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.559+0000 7f863f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864010f5a0 con 0x7f8640104990 2024-03-21T11:26:01.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.560+0000 7f863d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8640104990 0x7f864019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.560+0000 7f863d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8640104990 0x7f864019e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34160/0 (socket says 172.21.15.47:34160) 2024-03-21T11:26:01.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.560+0000 7f863d400640 1 -- 172.21.15.47:0/2314566765 learned_addr learned my addr 172.21.15.47:0/2314566765 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.560+0000 7f863e800640 1 --2- 172.21.15.47:0/2314566765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8640105350 0x7f86401a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.560+0000 7f863de00640 1 --2- 172.21.15.47:0/2314566765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86401037a0 0x7f864019ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.560+0000 7f863d400640 1 -- 172.21.15.47:0/2314566765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8640105350 msgr2=0x7f86401a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.560+0000 7f863d400640 1 --2- 172.21.15.47:0/2314566765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8640105350 0x7f86401a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.560+0000 7f863d400640 1 -- 172.21.15.47:0/2314566765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86401037a0 msgr2=0x7f864019ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.560+0000 7f863d400640 1 --2- 172.21.15.47:0/2314566765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86401037a0 0x7f864019ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.560+0000 7f863d400640 1 -- 172.21.15.47:0/2314566765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86401a5af0 con 0x7f8640104990 2024-03-21T11:26:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.561+0000 7f863d400640 1 --2- 172.21.15.47:0/2314566765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8640104990 0x7f864019e410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8628009230 tx=0x7f8628005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.561+0000 7f8636a00640 1 -- 172.21.15.47:0/2314566765 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628041070 con 0x7f8640104990 2024-03-21T11:26:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.561+0000 7f8636a00640 1 -- 172.21.15.47:0/2314566765 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86280044d0 con 0x7f8640104990 2024-03-21T11:26:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.561+0000 7f863f200640 1 -- 172.21.15.47:0/2314566765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86401a5d80 con 0x7f8640104990 2024-03-21T11:26:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.561+0000 7f8636a00640 1 -- 172.21.15.47:0/2314566765 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628034440 con 0x7f8640104990 2024-03-21T11:26:01.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.561+0000 7f863f200640 1 -- 172.21.15.47:0/2314566765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86401a6260 con 0x7f8640104990 2024-03-21T11:26:01.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.565+0000 7f8648010740 1 Processor -- start 2024-03-21T11:26:01.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.565+0000 7f8648010740 1 -- start start 2024-03-21T11:26:01.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.565+0000 7f8636a00640 1 -- 172.21.15.47:0/2314566765 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8628044070 con 0x7f8640104990 2024-03-21T11:26:01.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.565+0000 7f8648010740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e71399050 0x563e71399470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.565+0000 7f8648010740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e7139e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.565+0000 7f8648010740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e7139ebb0 0x563e713a1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.565+0000 7f8648010740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e712fe250 con 0x563e71399050 2024-03-21T11:26:01.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.565+0000 7f8648010740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e712a6a30 con 0x563e7139ebb0 2024-03-21T11:26:01.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.565+0000 7f8648010740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e712fa2f0 con 0x563e71399a40 2024-03-21T11:26:01.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.566+0000 7f863de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e7139e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.566+0000 7f863d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e7139ebb0 0x563e713a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.566+0000 7f863de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e7139e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34174/0 (socket says 172.21.15.47:34174) 2024-03-21T11:26:01.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.566+0000 7f863de00640 1 -- 172.21.15.47:0/2656689376 learned_addr learned my addr 172.21.15.47:0/2656689376 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:01.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.566+0000 7f863e800640 1 --2- 172.21.15.47:0/2656689376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e71399050 0x563e71399470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.566+0000 7f863de00640 1 -- 172.21.15.47:0/2656689376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e7139ebb0 msgr2=0x563e713a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:01.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.566+0000 7f863de00640 1 --2- 172.21.15.47:0/2656689376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e7139ebb0 0x563e713a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.566+0000 7f863de00640 1 -- 172.21.15.47:0/2656689376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e71399050 msgr2=0x563e71399470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:01.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.566+0000 7f863de00640 1 --2- 172.21.15.47:0/2656689376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e71399050 0x563e71399470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.566+0000 7f863de00640 1 -- 172.21.15.47:0/2656689376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e71291690 con 0x563e71399a40 2024-03-21T11:26:01.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.566+0000 7f863e800640 1 --2- 172.21.15.47:0/2656689376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e71399050 0x563e71399470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:01.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.567+0000 7f863de00640 1 --2- 172.21.15.47:0/2656689376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e7139e540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8630009fd0 tx=0x7f8630010880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22da479cc5be6fc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:01.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.567+0000 7f862f400640 1 -- 172.21.15.47:0/2656689376 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630016070 con 0x563e71399a40 2024-03-21T11:26:01.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.567+0000 7f862f400640 1 -- 172.21.15.47:0/2656689376 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8630003b70 con 0x563e71399a40 2024-03-21T11:26:01.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.567+0000 7f862f400640 1 -- 172.21.15.47:0/2656689376 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86300129d0 con 0x563e71399a40 2024-03-21T11:26:01.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.568+0000 7f8648010740 1 -- 172.21.15.47:0/2656689376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 msgr2=0x563e7139e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.568+0000 7f8648010740 1 --2- 172.21.15.47:0/2656689376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e7139e540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8630009fd0 tx=0x7f8630010880 comp rx=0 tx=0).stop 2024-03-21T11:26:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.568+0000 7f8636a00640 1 --2- 172.21.15.47:0/2314566765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86240a2fa0 0x7f86240a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.568+0000 7f8648010740 1 -- 172.21.15.47:0/2656689376 shutdown_connections 2024-03-21T11:26:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.568+0000 7f8648010740 1 --2- 172.21.15.47:0/2656689376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e7139ebb0 0x563e713a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.568+0000 7f8648010740 1 --2- 172.21.15.47:0/2656689376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e7139e540 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.568+0000 7f8648010740 1 --2- 172.21.15.47:0/2656689376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e71399050 0x563e71399470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.568+0000 7f8648010740 1 -- 172.21.15.47:0/2656689376 >> 172.21.15.47:0/2656689376 conn(0x563e71361200 msgr2=0x563e713949c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.568+0000 7f8636a00640 1 -- 172.21.15.47:0/2314566765 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f86280f1300 con 0x7f8640104990 2024-03-21T11:26:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.568+0000 7f863e800640 1 --2- 172.21.15.47:0/2314566765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86240a2fa0 0x7f86240a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.568+0000 7f8648010740 1 -- 172.21.15.47:0/2656689376 shutdown_connections 2024-03-21T11:26:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.569+0000 7f8648010740 1 -- 172.21.15.47:0/2656689376 wait complete. 2024-03-21T11:26:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.569+0000 7f8648010740 1 -- start start 2024-03-21T11:26:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.569+0000 7f863e800640 1 --2- 172.21.15.47:0/2314566765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86240a2fa0 0x7f86240a5460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f86380095f0 tx=0x7f8638005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.569+0000 7f8648010740 1 Processor -- start 2024-03-21T11:26:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.570+0000 7f8648010740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e713a3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.570+0000 7f8648010740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e71399a40 0x563e713a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.570+0000 7f8648010740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7139ebb0 0x563e713aa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.570+0000 7f8648010740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e713a1b60 con 0x563e71399a40 2024-03-21T11:26:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.570+0000 7f8648010740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e712b9e80 con 0x563e71399050 2024-03-21T11:26:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.570+0000 7f8648010740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e713a1ce0 con 0x563e7139ebb0 2024-03-21T11:26:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.570+0000 7f8648010740 10 client.0 _ll_register_callbacks cb 0x563e712cc130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.570+0000 7f863e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7139ebb0 0x563e713aa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.570+0000 7f863de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e713a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.570+0000 7f863de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e713a3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34112/0 (socket says 172.21.15.47:34112) 2024-03-21T11:26:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.570+0000 7f863de00640 1 -- 172.21.15.47:0/743302641 learned_addr learned my addr 172.21.15.47:0/743302641 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.571+0000 7f863d400640 1 --2- 172.21.15.47:0/743302641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e71399a40 0x563e713a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.571+0000 7f863e800640 1 -- 172.21.15.47:0/743302641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 msgr2=0x563e713a3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.571+0000 7f863e800640 1 --2- 172.21.15.47:0/743302641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e713a3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.571+0000 7f863e800640 1 -- 172.21.15.47:0/743302641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e71399a40 msgr2=0x563e713a35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.571+0000 7f863e800640 1 --2- 172.21.15.47:0/743302641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e71399a40 0x563e713a35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.571+0000 7f863e800640 1 -- 172.21.15.47:0/743302641 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e710dc7d0 con 0x563e7139ebb0 2024-03-21T11:26:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.571+0000 7f863de00640 1 --2- 172.21.15.47:0/743302641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e713a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.571+0000 7f863e800640 1 --2- 172.21.15.47:0/743302641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7139ebb0 0x563e713aa610 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x563e7139a2a0 tx=0x7f8638004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2211fe7b0e6d26cd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.571+0000 7f8623e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.572+0000 7f8623e00640 1 -- 172.21.15.47:0/743302641 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638029440 con 0x563e7139ebb0 2024-03-21T11:26:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.572+0000 7f8648010740 1 -- 172.21.15.47:0/743302641 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8630003b70 con 0x563e7139ebb0 2024-03-21T11:26:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.572+0000 7f8623e00640 1 -- 172.21.15.47:0/743302641 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f863801f430 con 0x563e7139ebb0 2024-03-21T11:26:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.572+0000 7f8623e00640 1 -- 172.21.15.47:0/743302641 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8638005020 con 0x563e7139ebb0 2024-03-21T11:26:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.572+0000 7f8648010740 10 client.26227 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.572+0000 7f8648010740 1 -- 172.21.15.47:0/743302641 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563e710123f0 con 0x563e7139ebb0 2024-03-21T11:26:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.572+0000 7f8623e00640 1 -- 172.21.15.47:0/743302641 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8630003b70 con 0x563e7139ebb0 2024-03-21T11:26:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.573+0000 7f8623e00640 1 -- 172.21.15.47:0/743302641 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8638027070 con 0x563e7139ebb0 2024-03-21T11:26:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.573+0000 7f8648010740 10 client.26227 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.573+0000 7f8648010740 10 client.26227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.573+0000 7f8648010740 20 client.26227 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.573+0000 7f8648010740 1 --2- 172.21.15.47:0/743302641 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e713aeff0 0x563e713b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.573+0000 7f8648010740 4 client.26227 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:01.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.573+0000 7f8648010740 1 -- 172.21.15.47:0/743302641 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e7130b790 con 0x563e713aeff0 2024-03-21T11:26:01.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.573+0000 7f863de00640 1 --2- 172.21.15.47:0/743302641 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e713aeff0 0x563e713b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.573+0000 7f863de00640 1 --2- 172.21.15.47:0/743302641 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e713aeff0 0x563e713b1430 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51a41d1d14579bb9 server_cookie=dc4745f9256c5910 in_seq=0 out_seq=0 2024-03-21T11:26:01.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.573+0000 7f8623e00640 10 client.26227 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.576+0000 7f8623e00640 1 -- 172.21.15.47:0/743302641 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563e7130b790 con 0x563e713aeff0 2024-03-21T11:26:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.576+0000 7f8623e00640 10 client.26227 handle_command_reply: tid=0 2024-03-21T11:26:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.576+0000 7f8648010740 1 client.26227 shutdown 2024-03-21T11:26:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.576+0000 7f8648010740 1 -- 172.21.15.47:0/743302641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7139ebb0 msgr2=0x563e713aa610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.576+0000 7f8648010740 1 --2- 172.21.15.47:0/743302641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7139ebb0 0x563e713aa610 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x563e7139a2a0 tx=0x7f8638004730 comp rx=0 tx=0).stop 2024-03-21T11:26:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.576+0000 7f8648010740 1 -- 172.21.15.47:0/743302641 shutdown_connections 2024-03-21T11:26:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.576+0000 7f8648010740 1 --2- 172.21.15.47:0/743302641 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e713aeff0 0x563e713b1430 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.576+0000 7f8648010740 1 --2- 172.21.15.47:0/743302641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e7139ebb0 0x563e713aa610 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.576+0000 7f8648010740 1 --2- 172.21.15.47:0/743302641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e71399a40 0x563e713a35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.576+0000 7f8648010740 1 --2- 172.21.15.47:0/743302641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e713a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.576+0000 7f8648010740 1 -- 172.21.15.47:0/743302641 >> 172.21.15.47:0/743302641 conn(0x563e71361200 msgr2=0x563e71393bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:01.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.576+0000 7f8623e00640 0 client.26227 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:01.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.577+0000 7f8648010740 1 -- 172.21.15.47:0/743302641 shutdown_connections 2024-03-21T11:26:01.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.577+0000 7f8648010740 1 -- 172.21.15.47:0/743302641 wait complete. 2024-03-21T11:26:01.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.577+0000 7f8648010740 20 client.26227 trim_cache size 0 max 16384 2024-03-21T11:26:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.595+0000 7f8648010740 1 Processor -- start 2024-03-21T11:26:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.596+0000 7f8648010740 1 -- start start 2024-03-21T11:26:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.596+0000 7f8648010740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e71397c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.596+0000 7f8648010740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e713aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.596+0000 7f8648010740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e7139ebb0 0x563e71398140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.596+0000 7f8648010740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401b0430 con 0x563e7139ebb0 2024-03-21T11:26:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.596+0000 7f8648010740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401b1db0 con 0x563e71399050 2024-03-21T11:26:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.596+0000 7f8648010740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401b0a90 con 0x563e71399a40 2024-03-21T11:26:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.596+0000 7f863d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e713aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.596+0000 7f863e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e7139ebb0 0x563e71398140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.596+0000 7f863de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e71397c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.596+0000 7f863d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e713aa5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34186/0 (socket says 172.21.15.47:34186) 2024-03-21T11:26:01.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.596+0000 7f863d400640 1 -- 172.21.15.47:0/1162228709 learned_addr learned my addr 172.21.15.47:0/1162228709 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:01.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.597+0000 7f863d400640 1 -- 172.21.15.47:0/1162228709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 msgr2=0x563e71397c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:01.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.597+0000 7f863d400640 1 --2- 172.21.15.47:0/1162228709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e71397c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.597+0000 7f863d400640 1 -- 172.21.15.47:0/1162228709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e7139ebb0 msgr2=0x563e71398140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:01.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.597+0000 7f863d400640 1 --2- 172.21.15.47:0/1162228709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e7139ebb0 0x563e71398140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.597+0000 7f863d400640 1 -- 172.21.15.47:0/1162228709 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e70fd20f0 con 0x563e71399a40 2024-03-21T11:26:01.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.597+0000 7f863de00640 1 --2- 172.21.15.47:0/1162228709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e71397c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:01.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.597+0000 7f863d400640 1 --2- 172.21.15.47:0/1162228709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e713aa5d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f862802cfd0 tx=0x7f86280f4d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57743bbec22d6284 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:01.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.597+0000 7f862ea00640 1 -- 172.21.15.47:0/1162228709 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862804b040 con 0x563e71399a40 2024-03-21T11:26:01.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.597+0000 7f862ea00640 1 -- 172.21.15.47:0/1162228709 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86280bdd40 con 0x563e71399a40 2024-03-21T11:26:01.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.597+0000 7f862ea00640 1 -- 172.21.15.47:0/1162228709 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86280bcfa0 con 0x563e71399a40 2024-03-21T11:26:01.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.598+0000 7f8648010740 1 -- 172.21.15.47:0/1162228709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 msgr2=0x563e713aa5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:01.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.598+0000 7f8648010740 1 --2- 172.21.15.47:0/1162228709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e713aa5d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f862802cfd0 tx=0x7f86280f4d40 comp rx=0 tx=0).stop 2024-03-21T11:26:01.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.598+0000 7f8648010740 1 -- 172.21.15.47:0/1162228709 shutdown_connections 2024-03-21T11:26:01.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.598+0000 7f8648010740 1 --2- 172.21.15.47:0/1162228709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e7139ebb0 0x563e71398140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.598+0000 7f8648010740 1 --2- 172.21.15.47:0/1162228709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e713aa5d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.598+0000 7f8648010740 1 --2- 172.21.15.47:0/1162228709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e71397c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.598+0000 7f8648010740 1 -- 172.21.15.47:0/1162228709 >> 172.21.15.47:0/1162228709 conn(0x563e71361200 msgr2=0x563e7139ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:01.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.598+0000 7f8648010740 1 -- 172.21.15.47:0/1162228709 shutdown_connections 2024-03-21T11:26:01.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.598+0000 7f8648010740 1 -- 172.21.15.47:0/1162228709 wait complete. 2024-03-21T11:26:01.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.599+0000 7f8648010740 1 -- start start 2024-03-21T11:26:01.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.599+0000 7f8648010740 1 Processor -- start 2024-03-21T11:26:01.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.599+0000 7f8648010740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e713a80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.599+0000 7f8648010740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e713a85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.599+0000 7f8648010740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e7139ebb0 0x563e713a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.599+0000 7f8648010740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401b1750 con 0x563e7139ebb0 2024-03-21T11:26:01.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.599+0000 7f8648010740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401b10f0 con 0x563e71399050 2024-03-21T11:26:01.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.599+0000 7f8648010740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401ae5f0 con 0x563e71399a40 2024-03-21T11:26:01.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.599+0000 7f8648010740 10 client.0 _ll_register_callbacks cb 0x7f86380096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.600+0000 7f863de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e713a80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.600+0000 7f863e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e7139ebb0 0x563e713a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.600+0000 7f863de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e713a80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34126/0 (socket says 172.21.15.47:34126) 2024-03-21T11:26:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.600+0000 7f863de00640 1 -- 172.21.15.47:0/1228810104 learned_addr learned my addr 172.21.15.47:0/1228810104 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.600+0000 7f863d400640 1 --2- 172.21.15.47:0/1228810104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e713a85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.600+0000 7f863d400640 1 -- 172.21.15.47:0/1228810104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 msgr2=0x563e713a80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.600+0000 7f863d400640 1 --2- 172.21.15.47:0/1228810104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e713a80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.600+0000 7f863d400640 1 -- 172.21.15.47:0/1228810104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e7139ebb0 msgr2=0x563e713a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.600+0000 7f863d400640 1 --2- 172.21.15.47:0/1228810104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e7139ebb0 0x563e713a95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.600+0000 7f863d400640 1 -- 172.21.15.47:0/1228810104 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e713a9be0 con 0x563e71399a40 2024-03-21T11:26:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.600+0000 7f863e800640 1 --2- 172.21.15.47:0/1228810104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e7139ebb0 0x563e713a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.600+0000 7f863d400640 1 --2- 172.21.15.47:0/1228810104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e713a85e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8628033000 tx=0x7f862802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f471c768e1f918c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:01.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.600+0000 7f8623e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.601+0000 7f8623e00640 1 -- 172.21.15.47:0/1228810104 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862804b030 con 0x563e71399a40 2024-03-21T11:26:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.601+0000 7f8623e00640 1 -- 172.21.15.47:0/1228810104 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f86280035f0 con 0x563e71399a40 2024-03-21T11:26:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.601+0000 7f8648010740 1 -- 172.21.15.47:0/1228810104 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86280bdd40 con 0x563e71399a40 2024-03-21T11:26:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.601+0000 7f8623e00640 1 -- 172.21.15.47:0/1228810104 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86280bcfa0 con 0x563e71399a40 2024-03-21T11:26:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.601+0000 7f8648010740 10 client.26233 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.601+0000 7f8648010740 1 -- 172.21.15.47:0/1228810104 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563e7140abe0 con 0x563e71399a40 2024-03-21T11:26:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.601+0000 7f8623e00640 1 -- 172.21.15.47:0/1228810104 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86280bdd40 con 0x563e71399a40 2024-03-21T11:26:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.601+0000 7f8623e00640 1 -- 172.21.15.47:0/1228810104 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f86280a6bd0 con 0x563e71399a40 2024-03-21T11:26:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.601+0000 7f8648010740 10 client.26233 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.601+0000 7f8648010740 10 client.26233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.601+0000 7f8648010740 20 client.26233 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.601+0000 7f8648010740 1 --2- 172.21.15.47:0/1228810104 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e7140b480 0x563e7140d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.601+0000 7f8648010740 4 client.26233 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:26:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.601+0000 7f8648010740 1 -- 172.21.15.47:0/1228810104 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563e71305ae0 con 0x563e7140b480 2024-03-21T11:26:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.602+0000 7f863de00640 1 --2- 172.21.15.47:0/1228810104 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e7140b480 0x563e7140d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.602+0000 7f863de00640 1 --2- 172.21.15.47:0/1228810104 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e7140b480 0x563e7140d8a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5de6602107ca1374 server_cookie=3ad05e9420a2140e in_seq=0 out_seq=0 2024-03-21T11:26:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.602+0000 7f8623e00640 10 client.26233 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8623e00640 1 -- 172.21.15.47:0/1228810104 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563e71305ae0 con 0x563e7140b480 2024-03-21T11:26:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8623e00640 10 client.26233 handle_command_reply: tid=0 2024-03-21T11:26:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8648010740 1 client.26233 shutdown 2024-03-21T11:26:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8648010740 1 -- 172.21.15.47:0/1228810104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 msgr2=0x563e713a85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8648010740 1 --2- 172.21.15.47:0/1228810104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e713a85e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8628033000 tx=0x7f862802bc30 comp rx=0 tx=0).stop 2024-03-21T11:26:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8648010740 1 -- 172.21.15.47:0/1228810104 shutdown_connections 2024-03-21T11:26:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8648010740 1 --2- 172.21.15.47:0/1228810104 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e7140b480 0x563e7140d8a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8648010740 1 --2- 172.21.15.47:0/1228810104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e7139ebb0 0x563e713a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8648010740 1 --2- 172.21.15.47:0/1228810104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e71399a40 0x563e713a85e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8623e00640 0 client.26233 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8648010740 1 --2- 172.21.15.47:0/1228810104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e71399050 0x563e713a80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8648010740 1 -- 172.21.15.47:0/1228810104 >> 172.21.15.47:0/1228810104 conn(0x563e71361200 msgr2=0x563e71393330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8648010740 1 -- 172.21.15.47:0/1228810104 shutdown_connections 2024-03-21T11:26:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8648010740 1 -- 172.21.15.47:0/1228810104 wait complete. 2024-03-21T11:26:01.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.603+0000 7f8648010740 20 client.26233 trim_cache size 0 max 16384 2024-03-21T11:26:01.609 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:26:01.609 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:26:01.609 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:26:01.609 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:26:01.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.604+0000 7f863f200640 1 -- 172.21.15.47:0/2314566765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86240a2fa0 msgr2=0x7f86240a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:01.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.604+0000 7f863f200640 1 --2- 172.21.15.47:0/2314566765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86240a2fa0 0x7f86240a5460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f86380095f0 tx=0x7f8638005f90 comp rx=0 tx=0).stop 2024-03-21T11:26:01.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.604+0000 7f863f200640 1 -- 172.21.15.47:0/2314566765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8640104990 msgr2=0x7f864019e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:01.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.604+0000 7f863f200640 1 --2- 172.21.15.47:0/2314566765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8640104990 0x7f864019e410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8628009230 tx=0x7f8628005d50 comp rx=0 tx=0).stop 2024-03-21T11:26:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.605+0000 7f863f200640 1 -- 172.21.15.47:0/2314566765 shutdown_connections 2024-03-21T11:26:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.605+0000 7f863f200640 1 --2- 172.21.15.47:0/2314566765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f86240a2fa0 0x7f86240a5460 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.605+0000 7f863f200640 1 --2- 172.21.15.47:0/2314566765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8640105350 0x7f86401a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.605+0000 7f863f200640 1 --2- 172.21.15.47:0/2314566765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8640104990 0x7f864019e410 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.605+0000 7f863f200640 1 --2- 172.21.15.47:0/2314566765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86401037a0 0x7f864019ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.605+0000 7f863f200640 1 -- 172.21.15.47:0/2314566765 >> 172.21.15.47:0/2314566765 conn(0x7f86400fed10 msgr2=0x7f8640077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.605+0000 7f863f200640 1 -- 172.21.15.47:0/2314566765 shutdown_connections 2024-03-21T11:26:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.605+0000 7f863f200640 1 -- 172.21.15.47:0/2314566765 wait complete. 2024-03-21T11:26:01.625 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:26:01.625 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:26:01.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.871+0000 7f71f6000640 1 -- 172.21.15.47:0/2952744163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f00756a0 msgr2=0x7f71f0073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:01.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.871+0000 7f71f6000640 1 --2- 172.21.15.47:0/2952744163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f00756a0 0x7f71f0073af0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009900 tx=0x7f71e002bc40 comp rx=0 tx=0).stop 2024-03-21T11:26:01.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.871+0000 7f71f6000640 1 -- 172.21.15.47:0/2952744163 shutdown_connections 2024-03-21T11:26:01.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.871+0000 7f71f6000640 1 --2- 172.21.15.47:0/2952744163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71f0109570 0x7f71f010b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.871+0000 7f71f6000640 1 --2- 172.21.15.47:0/2952744163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f00740c0 0x7f71f0074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.871+0000 7f71f6000640 1 --2- 172.21.15.47:0/2952744163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f00756a0 0x7f71f0073af0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.871+0000 7f71f6000640 1 -- 172.21.15.47:0/2952744163 >> 172.21.15.47:0/2952744163 conn(0x7f71f00fcc50 msgr2=0x7f71f00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:01.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.872+0000 7f71f6000640 1 -- 172.21.15.47:0/2952744163 shutdown_connections 2024-03-21T11:26:01.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.872+0000 7f71f6000640 1 -- 172.21.15.47:0/2952744163 wait complete. 2024-03-21T11:26:01.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.873+0000 7f71f6000640 1 Processor -- start 2024-03-21T11:26:01.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.873+0000 7f71f6000640 1 -- start start 2024-03-21T11:26:01.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.873+0000 7f71f6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f00740c0 0x7f71f0199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.873+0000 7f71f6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f00756a0 0x7f71f019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.873+0000 7f71f6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71f0109570 0x7f71f01a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.873+0000 7f71f6000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f010f500 con 0x7f71f00740c0 2024-03-21T11:26:01.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.873+0000 7f71f6000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f010f380 con 0x7f71f00756a0 2024-03-21T11:26:01.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.873+0000 7f71f6000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f010f680 con 0x7f71f0109570 2024-03-21T11:26:01.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.874+0000 7f71efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71f0109570 0x7f71f01a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.874+0000 7f71ef400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f00740c0 0x7f71f0199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.874+0000 7f71efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71f0109570 0x7f71f01a1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34218/0 (socket says 172.21.15.47:34218) 2024-03-21T11:26:01.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.874+0000 7f71efe00640 1 -- 172.21.15.47:0/1332937571 learned_addr learned my addr 172.21.15.47:0/1332937571 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:01.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.874+0000 7f71efe00640 1 -- 172.21.15.47:0/1332937571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f00756a0 msgr2=0x7f71f019a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:01.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.874+0000 7f71eea00640 1 --2- 172.21.15.47:0/1332937571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f00756a0 0x7f71f019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.874+0000 7f71efe00640 1 --2- 172.21.15.47:0/1332937571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f00756a0 0x7f71f019a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.874+0000 7f71efe00640 1 -- 172.21.15.47:0/1332937571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f00740c0 msgr2=0x7f71f0199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:01.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.874+0000 7f71efe00640 1 --2- 172.21.15.47:0/1332937571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f00740c0 0x7f71f0199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:01.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.874+0000 7f71efe00640 1 -- 172.21.15.47:0/1332937571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f01a17b0 con 0x7f71f0109570 2024-03-21T11:26:01.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.874+0000 7f71eea00640 1 --2- 172.21.15.47:0/1332937571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f00756a0 0x7f71f019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:26:01.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.874+0000 7f71efe00640 1 --2- 172.21.15.47:0/1332937571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71f0109570 0x7f71f01a1140 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f71e800eb10 tx=0x7f71e800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:01.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.875+0000 7f71e7e00640 1 -- 172.21.15.47:0/1332937571 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8004b20 con 0x7f71f0109570 2024-03-21T11:26:01.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.875+0000 7f71f6000640 1 -- 172.21.15.47:0/1332937571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f01a1aa0 con 0x7f71f0109570 2024-03-21T11:26:01.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.875+0000 7f71e7e00640 1 -- 172.21.15.47:0/1332937571 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71e8004cc0 con 0x7f71f0109570 2024-03-21T11:26:01.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.875+0000 7f71e7e00640 1 -- 172.21.15.47:0/1332937571 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8010760 con 0x7f71f0109570 2024-03-21T11:26:01.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.875+0000 7f71f6000640 1 -- 172.21.15.47:0/1332937571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f01a1ff0 con 0x7f71f0109570 2024-03-21T11:26:01.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.876+0000 7f71f6000640 1 -- 172.21.15.47:0/1332937571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f01082f0 con 0x7f71f0109570 2024-03-21T11:26:01.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.878+0000 7f71e7e00640 1 -- 172.21.15.47:0/1332937571 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f71e8005020 con 0x7f71f0109570 2024-03-21T11:26:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.880+0000 7f71e7e00640 1 --2- 172.21.15.47:0/1332937571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f71d40a2fa0 0x7f71d40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.880+0000 7f71e7e00640 1 -- 172.21.15.47:0/1332937571 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f71e80cfd80 con 0x7f71f0109570 2024-03-21T11:26:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.881+0000 7f71ef400640 1 --2- 172.21.15.47:0/1332937571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f71d40a2fa0 0x7f71d40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.881+0000 7f71ef400640 1 --2- 172.21.15.47:0/1332937571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f71d40a2fa0 0x7f71d40a5460 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f71e002c7c0 tx=0x7f71e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:01.887+0000 7f71e7e00640 1 -- 172.21.15.47:0/1332937571 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f71e809c7c0 con 0x7f71f0109570 2024-03-21T11:26:02.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.162+0000 7f71f6000640 1 -- 172.21.15.47:0/1332937571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71f002d070 con 0x7f71f0109570 2024-03-21T11:26:02.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.163+0000 7f71e7e00640 1 -- 172.21.15.47:0/1332937571 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f71e809c0c0 con 0x7f71f0109570 2024-03-21T11:26:02.164 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:26:02.164 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:26:02.164 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:26:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.166+0000 7f71f6000640 1 -- 172.21.15.47:0/1332937571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f71d40a2fa0 msgr2=0x7f71d40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.166+0000 7f71f6000640 1 --2- 172.21.15.47:0/1332937571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f71d40a2fa0 0x7f71d40a5460 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f71e002c7c0 tx=0x7f71e0005fd0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.166+0000 7f71f6000640 1 -- 172.21.15.47:0/1332937571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71f0109570 msgr2=0x7f71f01a1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.166+0000 7f71f6000640 1 --2- 172.21.15.47:0/1332937571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71f0109570 0x7f71f01a1140 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f71e800eb10 tx=0x7f71e800c2d0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.166+0000 7f71f6000640 1 -- 172.21.15.47:0/1332937571 shutdown_connections 2024-03-21T11:26:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.166+0000 7f71f6000640 1 --2- 172.21.15.47:0/1332937571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f71d40a2fa0 0x7f71d40a5460 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.166+0000 7f71f6000640 1 --2- 172.21.15.47:0/1332937571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71f0109570 0x7f71f01a1140 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.166+0000 7f71f6000640 1 --2- 172.21.15.47:0/1332937571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f00756a0 0x7f71f019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.166+0000 7f71f6000640 1 --2- 172.21.15.47:0/1332937571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f00740c0 0x7f71f0199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.166+0000 7f71f6000640 1 -- 172.21.15.47:0/1332937571 >> 172.21.15.47:0/1332937571 conn(0x7f71f00fcc50 msgr2=0x7f71f01054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.166+0000 7f71f6000640 1 -- 172.21.15.47:0/1332937571 shutdown_connections 2024-03-21T11:26:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.166+0000 7f71f6000640 1 -- 172.21.15.47:0/1332937571 wait complete. 2024-03-21T11:26:02.188 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:26:02.188 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:26:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.286+0000 7f564aa00640 1 -- 172.21.15.47:0/3296396933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5644102160 msgr2=0x7f5644104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.286+0000 7f564aa00640 1 --2- 172.21.15.47:0/3296396933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5644102160 0x7f5644104550 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5634009900 tx=0x7f563402bc30 comp rx=0 tx=0).stop 2024-03-21T11:26:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.286+0000 7f564aa00640 1 -- 172.21.15.47:0/3296396933 shutdown_connections 2024-03-21T11:26:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.286+0000 7f564aa00640 1 --2- 172.21.15.47:0/3296396933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56441073c0 0x7f5644109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.286+0000 7f564aa00640 1 --2- 172.21.15.47:0/3296396933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5644104a90 0x7f5644106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.286+0000 7f564aa00640 1 --2- 172.21.15.47:0/3296396933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5644102160 0x7f5644104550 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.286+0000 7f564aa00640 1 -- 172.21.15.47:0/3296396933 >> 172.21.15.47:0/3296396933 conn(0x7f56440fbb50 msgr2=0x7f56440fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.286+0000 7f564aa00640 1 -- 172.21.15.47:0/3296396933 shutdown_connections 2024-03-21T11:26:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.286+0000 7f564aa00640 1 -- 172.21.15.47:0/3296396933 wait complete. 2024-03-21T11:26:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.287+0000 7f564aa00640 1 Processor -- start 2024-03-21T11:26:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.287+0000 7f564aa00640 1 -- start start 2024-03-21T11:26:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.288+0000 7f564aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5644102160 0x7f56441963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.288+0000 7f564aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5644104a90 0x7f5644196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.288+0000 7f564aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56441073c0 0x7f5644192f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.288+0000 7f564aa00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564410d370 con 0x7f56441073c0 2024-03-21T11:26:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.288+0000 7f564aa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564410d1f0 con 0x7f5644102160 2024-03-21T11:26:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.288+0000 7f564aa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564410d4f0 con 0x7f5644104a90 2024-03-21T11:26:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.288+0000 7f5643e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5644102160 0x7f56441963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.288+0000 7f5643400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5644104a90 0x7f5644196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.288+0000 7f5643e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5644102160 0x7f56441963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34162/0 (socket says 172.21.15.47:34162) 2024-03-21T11:26:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.288+0000 7f5643e00640 1 -- 172.21.15.47:0/3605596733 learned_addr learned my addr 172.21.15.47:0/3605596733 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.288+0000 7f5648a00640 1 --2- 172.21.15.47:0/3605596733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56441073c0 0x7f5644192f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.289+0000 7f5643400640 1 -- 172.21.15.47:0/3605596733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5644102160 msgr2=0x7f56441963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.289+0000 7f5643400640 1 --2- 172.21.15.47:0/3605596733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5644102160 0x7f56441963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.289+0000 7f5643400640 1 -- 172.21.15.47:0/3605596733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56441073c0 msgr2=0x7f5644192f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.289+0000 7f5643400640 1 --2- 172.21.15.47:0/3605596733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56441073c0 0x7f5644192f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.289+0000 7f5643400640 1 -- 172.21.15.47:0/3605596733 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56441937f0 con 0x7f5644104a90 2024-03-21T11:26:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.289+0000 7f5643400640 1 --2- 172.21.15.47:0/3605596733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5644104a90 0x7f5644196900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f563000e9c0 tx=0x7f563000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.289+0000 7f5640c00640 1 -- 172.21.15.47:0/3605596733 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563000ccc0 con 0x7f5644104a90 2024-03-21T11:26:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.289+0000 7f564aa00640 1 -- 172.21.15.47:0/3605596733 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56441997a0 con 0x7f5644104a90 2024-03-21T11:26:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.289+0000 7f5640c00640 1 -- 172.21.15.47:0/3605596733 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5630010910 con 0x7f5644104a90 2024-03-21T11:26:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.289+0000 7f5640c00640 1 -- 172.21.15.47:0/3605596733 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630018880 con 0x7f5644104a90 2024-03-21T11:26:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.289+0000 7f564aa00640 1 -- 172.21.15.47:0/3605596733 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5644199d40 con 0x7f5644104a90 2024-03-21T11:26:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.293+0000 7f564d7c1740 1 Processor -- start 2024-03-21T11:26:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.293+0000 7f564d7c1740 1 -- start start 2024-03-21T11:26:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.293+0000 7f564d7c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ab71e0 0x558260ab7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.293+0000 7f564d7c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab7bd0 0x558260abc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.293+0000 7f564d7c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260abf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.293+0000 7f564d7c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558260a7a3b0 con 0x558260ab71e0 2024-03-21T11:26:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.293+0000 7f564d7c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558260a1c830 con 0x558260abcd40 2024-03-21T11:26:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.293+0000 7f564d7c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582609c55d0 con 0x558260ab7bd0 2024-03-21T11:26:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.293+0000 7f5643e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab7bd0 0x558260abc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.293+0000 7f5648a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ab71e0 0x558260ab7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.293+0000 7f5643e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab7bd0 0x558260abc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34248/0 (socket says 172.21.15.47:34248) 2024-03-21T11:26:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.293+0000 7f5643e00640 1 -- 172.21.15.47:0/1053035859 learned_addr learned my addr 172.21.15.47:0/1053035859 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.294+0000 7f5643e00640 1 -- 172.21.15.47:0/1053035859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 msgr2=0x558260abf1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:26:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.294+0000 7f5643e00640 1 --2- 172.21.15.47:0/1053035859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260abf1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.294+0000 7f5643e00640 1 -- 172.21.15.47:0/1053035859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ab71e0 msgr2=0x558260ab7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.294+0000 7f5643e00640 1 --2- 172.21.15.47:0/1053035859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ab71e0 0x558260ab7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.294+0000 7f5643e00640 1 -- 172.21.15.47:0/1053035859 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582609dd2c0 con 0x558260ab7bd0 2024-03-21T11:26:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.295+0000 7f5640c00640 1 -- 172.21.15.47:0/3605596733 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5630010ac0 con 0x7f5644104a90 2024-03-21T11:26:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.296+0000 7f5640c00640 1 --2- 172.21.15.47:0/3605596733 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56280a3040 0x7f56280a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.296+0000 7f5643e00640 1 --2- 172.21.15.47:0/1053035859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab7bd0 0x558260abc6d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5634009230 tx=0x7f5634009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf16378d9b079f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.296+0000 7f5640c00640 1 -- 172.21.15.47:0/3605596733 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f56300cfb10 con 0x7f5644104a90 2024-03-21T11:26:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.297+0000 7f5639200640 1 -- 172.21.15.47:0/1053035859 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634042070 con 0x558260ab7bd0 2024-03-21T11:26:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.297+0000 7f5639200640 1 -- 172.21.15.47:0/1053035859 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f563402cb80 con 0x558260ab7bd0 2024-03-21T11:26:02.297 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.297+0000 7f5639200640 1 -- 172.21.15.47:0/1053035859 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634004ed0 con 0x558260ab7bd0 2024-03-21T11:26:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.297+0000 7f5648a00640 1 --2- 172.21.15.47:0/3605596733 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56280a3040 0x7f56280a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.297+0000 7f564d7c1740 1 -- 172.21.15.47:0/1053035859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab7bd0 msgr2=0x558260abc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.297+0000 7f564d7c1740 1 --2- 172.21.15.47:0/1053035859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab7bd0 0x558260abc6d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5634009230 tx=0x7f5634009560 comp rx=0 tx=0).stop 2024-03-21T11:26:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.298+0000 7f5648a00640 1 --2- 172.21.15.47:0/3605596733 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56280a3040 0x7f56280a5500 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x558260ab8430 tx=0x7f563c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.298+0000 7f564d7c1740 1 -- 172.21.15.47:0/1053035859 shutdown_connections 2024-03-21T11:26:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.298+0000 7f564d7c1740 1 --2- 172.21.15.47:0/1053035859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260abf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.298+0000 7f564d7c1740 1 --2- 172.21.15.47:0/1053035859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab7bd0 0x558260abc6d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.298+0000 7f564d7c1740 1 --2- 172.21.15.47:0/1053035859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ab71e0 0x558260ab7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.298+0000 7f564d7c1740 1 -- 172.21.15.47:0/1053035859 >> 172.21.15.47:0/1053035859 conn(0x558260ab2750 msgr2=0x558260ab2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.298+0000 7f564d7c1740 1 -- 172.21.15.47:0/1053035859 shutdown_connections 2024-03-21T11:26:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.298+0000 7f564d7c1740 1 -- 172.21.15.47:0/1053035859 wait complete. 2024-03-21T11:26:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.298+0000 7f564d7c1740 1 -- start start 2024-03-21T11:26:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.299+0000 7f564d7c1740 1 Processor -- start 2024-03-21T11:26:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.299+0000 7f564d7c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab71e0 0x558260ac1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.299+0000 7f564d7c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260ac1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.299+0000 7f564d7c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ac6af0 0x558260ac8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.299+0000 7f564d7c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582609d8a20 con 0x558260ac6af0 2024-03-21T11:26:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.299+0000 7f564d7c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558260a188d0 con 0x558260abcd40 2024-03-21T11:26:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.299+0000 7f564d7c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558260abfc90 con 0x558260ab71e0 2024-03-21T11:26:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.300+0000 7f564d7c1740 10 client.0 _ll_register_callbacks cb 0x558260a151c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.300+0000 7f5643e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab71e0 0x558260ac1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.300+0000 7f5643e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab71e0 0x558260ac1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34258/0 (socket says 172.21.15.47:34258) 2024-03-21T11:26:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.300+0000 7f5643400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260ac1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.300+0000 7f5643e00640 1 -- 172.21.15.47:0/3650184037 learned_addr learned my addr 172.21.15.47:0/3650184037 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.300+0000 7f5643e00640 1 -- 172.21.15.47:0/3650184037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 msgr2=0x558260ac1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.300+0000 7f5643e00640 1 --2- 172.21.15.47:0/3650184037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260ac1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.300+0000 7f5643e00640 1 -- 172.21.15.47:0/3650184037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ac6af0 msgr2=0x558260ac8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.300+0000 7f5643e00640 1 --2- 172.21.15.47:0/3650184037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ac6af0 0x558260ac8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.300+0000 7f5643e00640 1 -- 172.21.15.47:0/3650184037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558260731210 con 0x558260ab71e0 2024-03-21T11:26:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.301+0000 7f5643e00640 1 --2- 172.21.15.47:0/3650184037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab71e0 0x558260ac1580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f563402c7b0 tx=0x7f5634000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d6309f48cbb5abf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.301+0000 7f562d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.301+0000 7f562d600640 1 -- 172.21.15.47:0/3650184037 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634042070 con 0x558260ab71e0 2024-03-21T11:26:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.301+0000 7f562d600640 1 -- 172.21.15.47:0/3650184037 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5634004ed0 con 0x558260ab71e0 2024-03-21T11:26:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.301+0000 7f564d7c1740 1 -- 172.21.15.47:0/3650184037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f563402cb80 con 0x558260ab71e0 2024-03-21T11:26:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.302+0000 7f562d600640 1 -- 172.21.15.47:0/3650184037 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5634055050 con 0x558260ab71e0 2024-03-21T11:26:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.302+0000 7f564d7c1740 10 client.26251 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.302+0000 7f564d7c1740 1 -- 172.21.15.47:0/3650184037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5582607fd620 con 0x558260ab71e0 2024-03-21T11:26:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.302+0000 7f562d600640 1 -- 172.21.15.47:0/3650184037 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f563402cb80 con 0x558260ab71e0 2024-03-21T11:26:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.302+0000 7f562d600640 1 -- 172.21.15.47:0/3650184037 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f56340039d0 con 0x558260ab71e0 2024-03-21T11:26:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.302+0000 7f564d7c1740 10 client.26251 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.302+0000 7f564d7c1740 10 client.26251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.303+0000 7f564d7c1740 20 client.26251 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.303+0000 7f564d7c1740 1 --2- 172.21.15.47:0/3650184037 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558260acd3d0 0x558260acf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.303+0000 7f564d7c1740 4 client.26251 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.303+0000 7f564d7c1740 1 -- 172.21.15.47:0/3650184037 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558260a29d70 con 0x558260acd3d0 2024-03-21T11:26:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.303+0000 7f5643400640 1 --2- 172.21.15.47:0/3650184037 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558260acd3d0 0x558260acf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.304+0000 7f5643400640 1 --2- 172.21.15.47:0/3650184037 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558260acd3d0 0x558260acf7c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65a6225be59c7175 server_cookie=337bcafb3aa8174f in_seq=0 out_seq=0 2024-03-21T11:26:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.304+0000 7f562d600640 10 client.26251 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f562d600640 1 -- 172.21.15.47:0/3650184037 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558260a29d70 con 0x558260acd3d0 2024-03-21T11:26:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f562d600640 10 client.26251 handle_command_reply: tid=0 2024-03-21T11:26:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f564d7c1740 1 client.26251 shutdown 2024-03-21T11:26:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f564d7c1740 1 -- 172.21.15.47:0/3650184037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab71e0 msgr2=0x558260ac1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f564d7c1740 1 --2- 172.21.15.47:0/3650184037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab71e0 0x558260ac1580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f563402c7b0 tx=0x7f5634000bc0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f564d7c1740 1 -- 172.21.15.47:0/3650184037 shutdown_connections 2024-03-21T11:26:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f564d7c1740 1 --2- 172.21.15.47:0/3650184037 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558260acd3d0 0x558260acf7c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f564d7c1740 1 --2- 172.21.15.47:0/3650184037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ac6af0 0x558260ac8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f564d7c1740 1 --2- 172.21.15.47:0/3650184037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260ac1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f562d600640 0 client.26251 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f564d7c1740 1 --2- 172.21.15.47:0/3650184037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab71e0 0x558260ac1580 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f564d7c1740 1 -- 172.21.15.47:0/3650184037 >> 172.21.15.47:0/3650184037 conn(0x558260ab2750 msgr2=0x558260ab1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f564d7c1740 1 -- 172.21.15.47:0/3650184037 shutdown_connections 2024-03-21T11:26:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f564d7c1740 1 -- 172.21.15.47:0/3650184037 wait complete. 2024-03-21T11:26:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.305+0000 7f564d7c1740 20 client.26251 trim_cache size 0 max 16384 2024-03-21T11:26:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.323+0000 7f564d7c1740 1 Processor -- start 2024-03-21T11:26:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.323+0000 7f564d7c1740 1 -- start start 2024-03-21T11:26:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.323+0000 7f564d7c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab71e0 0x558260ab60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.323+0000 7f564d7c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260ac0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.323+0000 7f564d7c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ac6af0 0x558260ac10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.323+0000 7f564d7c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56441b8e40 con 0x558260ac6af0 2024-03-21T11:26:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.323+0000 7f564d7c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56441ba7c0 con 0x558260abcd40 2024-03-21T11:26:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.323+0000 7f564d7c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56441b94a0 con 0x558260ab71e0 2024-03-21T11:26:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.323+0000 7f5643e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab71e0 0x558260ab60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.323+0000 7f5643e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab71e0 0x558260ab60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34272/0 (socket says 172.21.15.47:34272) 2024-03-21T11:26:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.323+0000 7f5643e00640 1 -- 172.21.15.47:0/2178562586 learned_addr learned my addr 172.21.15.47:0/2178562586 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.323+0000 7f5643400640 1 --2- 172.21.15.47:0/2178562586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260ac0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.324+0000 7f5648a00640 1 --2- 172.21.15.47:0/2178562586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ac6af0 0x558260ac10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.324+0000 7f5643e00640 1 -- 172.21.15.47:0/2178562586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 msgr2=0x558260ac0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.324+0000 7f5643e00640 1 --2- 172.21.15.47:0/2178562586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260ac0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.324+0000 7f5643e00640 1 -- 172.21.15.47:0/2178562586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ac6af0 msgr2=0x558260ac10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.324+0000 7f5643e00640 1 --2- 172.21.15.47:0/2178562586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ac6af0 0x558260ac10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.325 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.324+0000 7f5643e00640 1 -- 172.21.15.47:0/2178562586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558260abc410 con 0x558260ab71e0 2024-03-21T11:26:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.324+0000 7f5643e00640 1 --2- 172.21.15.47:0/2178562586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab71e0 0x558260ab60c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f563402c8c0 tx=0x7f563402c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a23635eca4a9d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.324+0000 7f562fe00640 1 -- 172.21.15.47:0/2178562586 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563405f070 con 0x558260ab71e0 2024-03-21T11:26:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.324+0000 7f562fe00640 1 -- 172.21.15.47:0/2178562586 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f563402cb80 con 0x558260ab71e0 2024-03-21T11:26:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.324+0000 7f562fe00640 1 -- 172.21.15.47:0/2178562586 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634042070 con 0x558260ab71e0 2024-03-21T11:26:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.325+0000 7f564d7c1740 1 -- 172.21.15.47:0/2178562586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab71e0 msgr2=0x558260ab60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.325+0000 7f564d7c1740 1 --2- 172.21.15.47:0/2178562586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab71e0 0x558260ab60c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f563402c8c0 tx=0x7f563402c330 comp rx=0 tx=0).stop 2024-03-21T11:26:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.325+0000 7f564d7c1740 1 -- 172.21.15.47:0/2178562586 shutdown_connections 2024-03-21T11:26:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.325+0000 7f564d7c1740 1 --2- 172.21.15.47:0/2178562586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ac6af0 0x558260ac10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.325+0000 7f564d7c1740 1 --2- 172.21.15.47:0/2178562586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260ac0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.325+0000 7f564d7c1740 1 --2- 172.21.15.47:0/2178562586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ab71e0 0x558260ab60c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.325+0000 7f564d7c1740 1 -- 172.21.15.47:0/2178562586 >> 172.21.15.47:0/2178562586 conn(0x558260ab2750 msgr2=0x558260ab59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.325+0000 7f564d7c1740 1 -- 172.21.15.47:0/2178562586 shutdown_connections 2024-03-21T11:26:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.325+0000 7f564d7c1740 1 -- 172.21.15.47:0/2178562586 wait complete. 2024-03-21T11:26:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.326+0000 7f564d7c1740 1 -- start start 2024-03-21T11:26:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.326+0000 7f564d7c1740 1 Processor -- start 2024-03-21T11:26:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.326+0000 7f564d7c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ab71e0 0x558260ade200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.326+0000 7f564d7c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260ade740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.326+0000 7f564d7c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ac6af0 0x558260adec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.326+0000 7f564d7c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56441ba160 con 0x558260ab71e0 2024-03-21T11:26:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.326+0000 7f564d7c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56441b9b00 con 0x558260abcd40 2024-03-21T11:26:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.326+0000 7f564d7c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56441b7020 con 0x558260ac6af0 2024-03-21T11:26:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.327+0000 7f564d7c1740 10 client.0 _ll_register_callbacks cb 0x7f563c009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.327+0000 7f5643400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260ade740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.327+0000 7f5643400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260ade740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34194/0 (socket says 172.21.15.47:34194) 2024-03-21T11:26:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.327+0000 7f5643400640 1 -- 172.21.15.47:0/869551006 learned_addr learned my addr 172.21.15.47:0/869551006 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.327+0000 7f5643e00640 1 --2- 172.21.15.47:0/869551006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ab71e0 0x558260ade200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.327+0000 7f5648a00640 1 --2- 172.21.15.47:0/869551006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ac6af0 0x558260adec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.327+0000 7f5643400640 1 -- 172.21.15.47:0/869551006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ac6af0 msgr2=0x558260adec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.327+0000 7f5643400640 1 --2- 172.21.15.47:0/869551006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ac6af0 0x558260adec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.327+0000 7f5643400640 1 -- 172.21.15.47:0/869551006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ab71e0 msgr2=0x558260ade200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.327+0000 7f5643400640 1 --2- 172.21.15.47:0/869551006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ab71e0 0x558260ade200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.327+0000 7f5643400640 1 -- 172.21.15.47:0/869551006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558260b281c0 con 0x558260abcd40 2024-03-21T11:26:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.327+0000 7f5643400640 1 --2- 172.21.15.47:0/869551006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260ade740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f563009b900 tx=0x7f563000efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e2e54da6ed347d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.327+0000 7f562d600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.328+0000 7f562d600640 1 -- 172.21.15.47:0/869551006 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56300b4020 con 0x558260abcd40 2024-03-21T11:26:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.328+0000 7f562d600640 1 -- 172.21.15.47:0/869551006 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f563009a1b0 con 0x558260abcd40 2024-03-21T11:26:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.328+0000 7f564d7c1740 1 -- 172.21.15.47:0/869551006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f563402cb80 con 0x558260abcd40 2024-03-21T11:26:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.328+0000 7f562d600640 1 -- 172.21.15.47:0/869551006 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56300a0000 con 0x558260abcd40 2024-03-21T11:26:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.328+0000 7f564d7c1740 10 client.26237 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.328+0000 7f564d7c1740 1 -- 172.21.15.47:0/869551006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558260b29540 con 0x558260abcd40 2024-03-21T11:26:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.328+0000 7f562d600640 1 -- 172.21.15.47:0/869551006 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f563402cb80 con 0x558260abcd40 2024-03-21T11:26:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.328+0000 7f562d600640 1 -- 172.21.15.47:0/869551006 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5630097c00 con 0x558260abcd40 2024-03-21T11:26:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.329+0000 7f564d7c1740 10 client.26237 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.329+0000 7f564d7c1740 10 client.26237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.329+0000 7f564d7c1740 20 client.26237 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.329+0000 7f564d7c1740 1 --2- 172.21.15.47:0/869551006 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558260b29e40 0x558260b2c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.329+0000 7f564d7c1740 4 client.26237 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:26:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.329+0000 7f564d7c1740 1 -- 172.21.15.47:0/869551006 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558260a240c0 con 0x558260b29e40 2024-03-21T11:26:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.329+0000 7f5643e00640 1 --2- 172.21.15.47:0/869551006 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558260b29e40 0x558260b2c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.330+0000 7f5643e00640 1 --2- 172.21.15.47:0/869551006 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558260b29e40 0x558260b2c230 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35c8bb10c3c08062 server_cookie=1b6eec79b115f94 in_seq=0 out_seq=0 2024-03-21T11:26:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.330+0000 7f562d600640 10 client.26237 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:02.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.459+0000 7fae04000640 1 -- 172.21.15.47:0/2110243712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadfc104a50 msgr2=0x7fadfc106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.459+0000 7fae04000640 1 --2- 172.21.15.47:0/2110243712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadfc104a50 0x7fadfc106e40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009900 tx=0x7fadf002bc30 comp rx=0 tx=0).stop 2024-03-21T11:26:02.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.459+0000 7fae04000640 1 -- 172.21.15.47:0/2110243712 shutdown_connections 2024-03-21T11:26:02.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.459+0000 7fae04000640 1 --2- 172.21.15.47:0/2110243712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadfc107380 0x7fadfc1097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.459+0000 7fae04000640 1 --2- 172.21.15.47:0/2110243712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadfc104a50 0x7fadfc106e40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.459+0000 7fae04000640 1 --2- 172.21.15.47:0/2110243712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadfc102120 0x7fadfc104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.459+0000 7fae04000640 1 -- 172.21.15.47:0/2110243712 >> 172.21.15.47:0/2110243712 conn(0x7fadfc0fbb30 msgr2=0x7fadfc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.460+0000 7fae04000640 1 -- 172.21.15.47:0/2110243712 shutdown_connections 2024-03-21T11:26:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.460+0000 7fae04000640 1 -- 172.21.15.47:0/2110243712 wait complete. 2024-03-21T11:26:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.461+0000 7fae04000640 1 Processor -- start 2024-03-21T11:26:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.463+0000 7fae04000640 1 -- start start 2024-03-21T11:26:02.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.464+0000 7fae04000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadfc102120 0x7fadfc195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.464+0000 7fae04000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadfc104a50 0x7fadfc195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.464+0000 7fae04000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadfc107380 0x7fadfc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.464+0000 7fae04000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc10d360 con 0x7fadfc107380 2024-03-21T11:26:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.464+0000 7fae04000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc10d1e0 con 0x7fadfc102120 2024-03-21T11:26:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.464+0000 7fae04000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc10d4e0 con 0x7fadfc104a50 2024-03-21T11:26:02.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.464+0000 7fae01600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadfc102120 0x7fadfc195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.464+0000 7fae01600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadfc102120 0x7fadfc195780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34220/0 (socket says 172.21.15.47:34220) 2024-03-21T11:26:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.464+0000 7fae01600640 1 -- 172.21.15.47:0/868855277 learned_addr learned my addr 172.21.15.47:0/868855277 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.465+0000 7fae01600640 1 -- 172.21.15.47:0/868855277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadfc104a50 msgr2=0x7fadfc195cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T11:26:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.465+0000 7fae01600640 1 --2- 172.21.15.47:0/868855277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadfc104a50 0x7fadfc195cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.465+0000 7fae01600640 1 -- 172.21.15.47:0/868855277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadfc107380 msgr2=0x7fadfc19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.465+0000 7fae01600640 1 --2- 172.21.15.47:0/868855277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadfc107380 0x7fadfc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.465+0000 7fae01600640 1 -- 172.21.15.47:0/868855277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc19d3a0 con 0x7fadfc102120 2024-03-21T11:26:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.466+0000 7fae01600640 1 --2- 172.21.15.47:0/868855277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadfc102120 0x7fadfc195780 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fade800e9f0 tx=0x7fade800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.467+0000 7fadfa000640 1 -- 172.21.15.47:0/868855277 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade800cd80 con 0x7fadfc102120 2024-03-21T11:26:02.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.468+0000 7fadfa000640 1 -- 172.21.15.47:0/868855277 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fade8010910 con 0x7fadfc102120 2024-03-21T11:26:02.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.468+0000 7fadfa000640 1 -- 172.21.15.47:0/868855277 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade80188f0 con 0x7fadfc102120 2024-03-21T11:26:02.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.468+0000 7fae04000640 1 -- 172.21.15.47:0/868855277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc19d690 con 0x7fadfc102120 2024-03-21T11:26:02.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.469+0000 7fae04000640 1 -- 172.21.15.47:0/868855277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc19db90 con 0x7fadfc102120 2024-03-21T11:26:02.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.477+0000 7fadfa000640 1 -- 172.21.15.47:0/868855277 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fade8010ac0 con 0x7fadfc102120 2024-03-21T11:26:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.478+0000 7fadfa000640 1 --2- 172.21.15.47:0/868855277 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fade40a2f70 0x7fade40a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.478+0000 7fadfa000640 1 -- 172.21.15.47:0/868855277 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fade80cff10 con 0x7fadfc102120 2024-03-21T11:26:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.479+0000 7fae00c00640 1 --2- 172.21.15.47:0/868855277 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fade40a2f70 0x7fade40a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.480+0000 7fae00c00640 1 --2- 172.21.15.47:0/868855277 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fade40a2f70 0x7fade40a5430 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009230 tx=0x7fadf0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.484+0000 7fae06c4b740 1 Processor -- start 2024-03-21T11:26:02.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.484+0000 7fae06c4b740 1 -- start start 2024-03-21T11:26:02.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.484+0000 7fae06c4b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd5fec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.485+0000 7fae06c4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd5ff9e0 0x562fcd5ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.485+0000 7fae06c4b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd6003a0 0x562fcd606c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.485+0000 7fae06c4b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fcd5632c0 con 0x562fcd6003a0 2024-03-21T11:26:02.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.485+0000 7fae06c4b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fcd50bc90 con 0x562fcd5ff9e0 2024-03-21T11:26:02.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.485+0000 7fae06c4b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fcd55f360 con 0x562fcd5fe810 2024-03-21T11:26:02.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.485+0000 7fae00c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd5ff9e0 0x562fcd5ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.486+0000 7fae00c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd5ff9e0 0x562fcd5ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34236/0 (socket says 172.21.15.47:34236) 2024-03-21T11:26:02.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.486+0000 7fae00c00640 1 -- 172.21.15.47:0/1804183185 learned_addr learned my addr 172.21.15.47:0/1804183185 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.486+0000 7fae01600640 1 --2- 172.21.15.47:0/1804183185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd5fec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.486+0000 7fae02000640 1 --2- 172.21.15.47:0/1804183185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd6003a0 0x562fcd606c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.487+0000 7fae01600640 1 -- 172.21.15.47:0/1804183185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd5ff9e0 msgr2=0x562fcd5ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.487+0000 7fae01600640 1 --2- 172.21.15.47:0/1804183185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd5ff9e0 0x562fcd5ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.487+0000 7fae01600640 1 -- 172.21.15.47:0/1804183185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd6003a0 msgr2=0x562fcd606c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.487+0000 7fae01600640 1 --2- 172.21.15.47:0/1804183185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd6003a0 0x562fcd606c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.487+0000 7fae01600640 1 -- 172.21.15.47:0/1804183185 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fcd530080 con 0x562fcd5fe810 2024-03-21T11:26:02.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.487+0000 7fae02000640 1 --2- 172.21.15.47:0/1804183185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd6003a0 0x562fcd606c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:02.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.488+0000 7fae01600640 1 --2- 172.21.15.47:0/1804183185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd5fec30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fade800e9c0 tx=0x7fade809c150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d415f8628c779e02 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.489+0000 7fadee600640 1 -- 172.21.15.47:0/1804183185 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade80a0170 con 0x562fcd5fe810 2024-03-21T11:26:02.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.489+0000 7fadee600640 1 -- 172.21.15.47:0/1804183185 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fade80a4c60 con 0x562fcd5fe810 2024-03-21T11:26:02.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.489+0000 7fadee600640 1 -- 172.21.15.47:0/1804183185 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade809b6a0 con 0x562fcd5fe810 2024-03-21T11:26:02.492 INFO:tasks.workunit.client.0.smithi047.stdout: 1 77034 9.35 MB/sec warmup 46 sec latency 97.425 ms 2024-03-21T11:26:02.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.491+0000 7fae06c4b740 1 -- 172.21.15.47:0/1804183185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 msgr2=0x562fcd5fec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.491+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1804183185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd5fec30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fade800e9c0 tx=0x7fade809c150 comp rx=0 tx=0).stop 2024-03-21T11:26:02.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.492+0000 7fae06c4b740 1 -- 172.21.15.47:0/1804183185 shutdown_connections 2024-03-21T11:26:02.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.492+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1804183185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd6003a0 0x562fcd606c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.492+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1804183185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd5ff9e0 0x562fcd5ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.492+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1804183185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd5fec30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.492+0000 7fae06c4b740 1 -- 172.21.15.47:0/1804183185 >> 172.21.15.47:0/1804183185 conn(0x562fcd5f9c00 msgr2=0x562fcd5fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.493+0000 7fae06c4b740 1 -- 172.21.15.47:0/1804183185 shutdown_connections 2024-03-21T11:26:02.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.493+0000 7fae06c4b740 1 -- 172.21.15.47:0/1804183185 wait complete. 2024-03-21T11:26:02.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.493+0000 7fae06c4b740 1 -- start start 2024-03-21T11:26:02.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.494+0000 7fae06c4b740 1 Processor -- start 2024-03-21T11:26:02.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.495+0000 7fae06c4b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd608ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.495+0000 7fae06c4b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd608ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.495+0000 7fae06c4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 0x562fcd610030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.496+0000 7fae00c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd608ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.496+0000 7fae00c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd608ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53756/0 (socket says 172.21.15.47:53756) 2024-03-21T11:26:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.497+0000 7fae06c4b740 1 -- 172.21.15.47:0/1002244381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fcd607350 con 0x562fcd5ff9e0 2024-03-21T11:26:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.496+0000 7fae00c00640 1 -- 172.21.15.47:0/1002244381 learned_addr learned my addr 172.21.15.47:0/1002244381 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.497+0000 7fae02000640 1 --2- 172.21.15.47:0/1002244381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 0x562fcd610030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.497+0000 7fae01600640 1 --2- 172.21.15.47:0/1002244381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd608ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.497+0000 7fae06c4b740 1 -- 172.21.15.47:0/1002244381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fcd51f0d0 con 0x562fcd6003a0 2024-03-21T11:26:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.497+0000 7fae06c4b740 1 -- 172.21.15.47:0/1002244381 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fcd6074d0 con 0x562fcd5fe810 2024-03-21T11:26:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.497+0000 7fae06c4b740 10 client.0 _ll_register_callbacks cb 0x562fcd523e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:02.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.498+0000 7fae02000640 1 -- 172.21.15.47:0/1002244381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 msgr2=0x562fcd608ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.498+0000 7fae02000640 1 --2- 172.21.15.47:0/1002244381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd608ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.498+0000 7fae02000640 1 -- 172.21.15.47:0/1002244381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 msgr2=0x562fcd608ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.498+0000 7fae02000640 1 --2- 172.21.15.47:0/1002244381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd608ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.498+0000 7fae02000640 1 -- 172.21.15.47:0/1002244381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fcd2773f0 con 0x562fcd6003a0 2024-03-21T11:26:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.498+0000 7fae00c00640 1 --2- 172.21.15.47:0/1002244381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd608ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.498+0000 7fae02000640 1 --2- 172.21.15.47:0/1002244381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 0x562fcd610030 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fadf4005fd0 tx=0x7fadf400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29c5611c13c0143d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.499+0000 7fade2a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.499+0000 7fade2a00640 1 -- 172.21.15.47:0/1002244381 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf400cc40 con 0x562fcd6003a0 2024-03-21T11:26:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.499+0000 7fae06c4b740 1 -- 172.21.15.47:0/1002244381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fade80a4c60 con 0x562fcd6003a0 2024-03-21T11:26:02.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.499+0000 7fade2a00640 1 -- 172.21.15.47:0/1002244381 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fadf400cde0 con 0x562fcd6003a0 2024-03-21T11:26:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.500+0000 7fade2a00640 1 -- 172.21.15.47:0/1002244381 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadf4022050 con 0x562fcd6003a0 2024-03-21T11:26:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.500+0000 7fae06c4b740 10 client.26249 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.500+0000 7fae06c4b740 1 -- 172.21.15.47:0/1002244381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562fcd3417d0 con 0x562fcd6003a0 2024-03-21T11:26:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.500+0000 7fade2a00640 1 -- 172.21.15.47:0/1002244381 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fade80a4c60 con 0x562fcd6003a0 2024-03-21T11:26:02.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.502+0000 7fade2a00640 1 -- 172.21.15.47:0/1002244381 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fadf4010430 con 0x562fcd6003a0 2024-03-21T11:26:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.502+0000 7fae06c4b740 10 client.26249 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:02.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.502+0000 7fae06c4b740 10 client.26249 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:26:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.502+0000 7fae06c4b740 20 client.26249 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.503+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1002244381 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x562fcd614550 0x562fcd616960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.503+0000 7fae06c4b740 4 client.26249 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.503+0000 7fae06c4b740 1 -- 172.21.15.47:0/1002244381 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fcd570800 con 0x562fcd614550 2024-03-21T11:26:02.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.505+0000 7fae01600640 1 --2- 172.21.15.47:0/1002244381 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x562fcd614550 0x562fcd616960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.506+0000 7fae01600640 1 --2- 172.21.15.47:0/1002244381 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x562fcd614550 0x562fcd616960 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b98f5db600680423 server_cookie=c175c1e6ae46570f in_seq=0 out_seq=0 2024-03-21T11:26:02.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.506+0000 7fade2a00640 10 client.26249 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:26:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.507+0000 7fade2a00640 1 -- 172.21.15.47:0/1002244381 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562fcd570800 con 0x562fcd614550 2024-03-21T11:26:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.507+0000 7fade2a00640 10 client.26249 handle_command_reply: tid=0 2024-03-21T11:26:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.507+0000 7fae06c4b740 1 client.26249 shutdown 2024-03-21T11:26:02.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.509+0000 7fae06c4b740 1 -- 172.21.15.47:0/1002244381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 msgr2=0x562fcd610030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.509+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1002244381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 0x562fcd610030 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fadf4005fd0 tx=0x7fadf400ee50 comp rx=0 tx=0).stop 2024-03-21T11:26:02.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.509+0000 7fae06c4b740 1 -- 172.21.15.47:0/1002244381 shutdown_connections 2024-03-21T11:26:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.509+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1002244381 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x562fcd614550 0x562fcd616960 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.509+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1002244381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 0x562fcd610030 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.509+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1002244381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd608ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.509+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1002244381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd608ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.509+0000 7fae06c4b740 1 -- 172.21.15.47:0/1002244381 >> 172.21.15.47:0/1002244381 conn(0x562fcd5f9c00 msgr2=0x562fcd5f8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.509+0000 7fade2a00640 0 client.26249 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:26:02.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.509+0000 7fae06c4b740 1 -- 172.21.15.47:0/1002244381 shutdown_connections 2024-03-21T11:26:02.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.509+0000 7fae06c4b740 1 -- 172.21.15.47:0/1002244381 wait complete. 2024-03-21T11:26:02.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.510+0000 7fae06c4b740 20 client.26249 trim_cache size 0 max 16384 2024-03-21T11:26:02.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.535+0000 7fae06c4b740 1 Processor -- start 2024-03-21T11:26:02.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.536+0000 7fae06c4b740 1 -- start start 2024-03-21T11:26:02.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.536+0000 7fae06c4b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd5fd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.536+0000 7fae06c4b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd60fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.536+0000 7fae06c4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 0x562fcd602b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.536+0000 7fae06c4b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc1b8d80 con 0x562fcd5ff9e0 2024-03-21T11:26:02.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.536+0000 7fae06c4b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc1ba700 con 0x562fcd6003a0 2024-03-21T11:26:02.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.536+0000 7fae06c4b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc1b93e0 con 0x562fcd5fe810 2024-03-21T11:26:02.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.536+0000 7fae01600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd5fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.536+0000 7fae02000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 0x562fcd602b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.536+0000 7fae01600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd5fd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34318/0 (socket says 172.21.15.47:34318) 2024-03-21T11:26:02.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.536+0000 7fae01600640 1 -- 172.21.15.47:0/1911334984 learned_addr learned my addr 172.21.15.47:0/1911334984 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.536+0000 7fae00c00640 1 --2- 172.21.15.47:0/1911334984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd60fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.537+0000 7fae01600640 1 -- 172.21.15.47:0/1911334984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 msgr2=0x562fcd602b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.537+0000 7fae01600640 1 --2- 172.21.15.47:0/1911334984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 0x562fcd602b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.537+0000 7fae01600640 1 -- 172.21.15.47:0/1911334984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 msgr2=0x562fcd60fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.537+0000 7fae01600640 1 --2- 172.21.15.47:0/1911334984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd60fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.537+0000 7fae01600640 1 -- 172.21.15.47:0/1911334984 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fcd5fa970 con 0x562fcd5fe810 2024-03-21T11:26:02.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.537+0000 7fae02000640 1 --2- 172.21.15.47:0/1911334984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 0x562fcd602b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:02.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.537+0000 7fae00c00640 1 --2- 172.21.15.47:0/1911334984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd60fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:02.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.537+0000 7fae01600640 1 --2- 172.21.15.47:0/1911334984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd5fd570 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fade809c1e0 tx=0x7fade809c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77e4e47c491784b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.537+0000 7fadedc00640 1 -- 172.21.15.47:0/1911334984 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade809a2f0 con 0x562fcd5fe810 2024-03-21T11:26:02.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.537+0000 7fadedc00640 1 -- 172.21.15.47:0/1911334984 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fade80a0170 con 0x562fcd5fe810 2024-03-21T11:26:02.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.537+0000 7fadedc00640 1 -- 172.21.15.47:0/1911334984 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade80cdae0 con 0x562fcd5fe810 2024-03-21T11:26:02.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.538+0000 7fae06c4b740 1 -- 172.21.15.47:0/1911334984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 msgr2=0x562fcd5fd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.538+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1911334984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd5fd570 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fade809c1e0 tx=0x7fade809c950 comp rx=0 tx=0).stop 2024-03-21T11:26:02.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.538+0000 7fae06c4b740 1 -- 172.21.15.47:0/1911334984 shutdown_connections 2024-03-21T11:26:02.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.538+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1911334984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 0x562fcd602b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.538+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1911334984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd60fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.538+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1911334984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd5fd570 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.538+0000 7fae06c4b740 1 -- 172.21.15.47:0/1911334984 >> 172.21.15.47:0/1911334984 conn(0x562fcd5f9c00 msgr2=0x562fcd5fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.538+0000 7fae06c4b740 1 -- 172.21.15.47:0/1911334984 shutdown_connections 2024-03-21T11:26:02.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.538+0000 7fae06c4b740 1 -- 172.21.15.47:0/1911334984 wait complete. 2024-03-21T11:26:02.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.539+0000 7fae06c4b740 1 -- start start 2024-03-21T11:26:02.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.539+0000 7fae06c4b740 1 Processor -- start 2024-03-21T11:26:02.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.540+0000 7fae06c4b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd60b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.540+0000 7fae06c4b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd60bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.540+0000 7fae06c4b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 0x562fcd60ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.540+0000 7fae06c4b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc1ba0a0 con 0x562fcd5ff9e0 2024-03-21T11:26:02.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.540+0000 7fae06c4b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc1b9a40 con 0x562fcd6003a0 2024-03-21T11:26:02.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.540+0000 7fae06c4b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc1b6ff0 con 0x562fcd5fe810 2024-03-21T11:26:02.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.540+0000 7fae06c4b740 10 client.0 _ll_register_callbacks cb 0x7fadf40074c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:02.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.540+0000 7fae01600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd60b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.540+0000 7f562d600640 1 -- 172.21.15.47:0/869551006 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558260a240c0 con 0x558260b29e40 2024-03-21T11:26:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.540+0000 7f562d600640 10 client.26237 handle_command_reply: tid=0 2024-03-21T11:26:02.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.540+0000 7fae01600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd60b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34326/0 (socket says 172.21.15.47:34326) 2024-03-21T11:26:02.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.540+0000 7fae01600640 1 -- 172.21.15.47:0/1862384335 learned_addr learned my addr 172.21.15.47:0/1862384335 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.540+0000 7fae02000640 1 --2- 172.21.15.47:0/1862384335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 0x562fcd60ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.540+0000 7fae00c00640 1 --2- 172.21.15.47:0/1862384335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd60bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.540+0000 7f564d7c1740 1 client.26237 shutdown 2024-03-21T11:26:02.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.541+0000 7fae01600640 1 -- 172.21.15.47:0/1862384335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 msgr2=0x562fcd60ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.541+0000 7fae01600640 1 --2- 172.21.15.47:0/1862384335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 0x562fcd60ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.541+0000 7f564d7c1740 1 -- 172.21.15.47:0/869551006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 msgr2=0x558260ade740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.541+0000 7f564d7c1740 1 --2- 172.21.15.47:0/869551006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260ade740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f563009b900 tx=0x7f563000efb0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.541+0000 7fae01600640 1 -- 172.21.15.47:0/1862384335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 msgr2=0x562fcd60bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.541+0000 7fae01600640 1 --2- 172.21.15.47:0/1862384335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd60bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.541+0000 7fae01600640 1 -- 172.21.15.47:0/1862384335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fcd60f460 con 0x562fcd5fe810 2024-03-21T11:26:02.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.541+0000 7fae00c00640 1 --2- 172.21.15.47:0/1862384335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd60bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.541+0000 7f564d7c1740 1 -- 172.21.15.47:0/869551006 shutdown_connections 2024-03-21T11:26:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.541+0000 7f564d7c1740 1 --2- 172.21.15.47:0/869551006 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558260b29e40 0x558260b2c230 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.541+0000 7f564d7c1740 1 --2- 172.21.15.47:0/869551006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558260ac6af0 0x558260adec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.541+0000 7f564d7c1740 1 --2- 172.21.15.47:0/869551006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558260abcd40 0x558260ade740 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.541+0000 7f564d7c1740 1 --2- 172.21.15.47:0/869551006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558260ab71e0 0x558260ade200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.541+0000 7f564d7c1740 1 -- 172.21.15.47:0/869551006 >> 172.21.15.47:0/869551006 conn(0x558260ab2750 msgr2=0x558260ab6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.541+0000 7f562d600640 0 client.26237 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:02.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.541+0000 7fae01600640 1 --2- 172.21.15.47:0/1862384335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd60b980 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fade80a4730 tx=0x7fade809a370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62227f6185296397 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.541+0000 7fade2a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.542+0000 7f564d7c1740 1 -- 172.21.15.47:0/869551006 shutdown_connections 2024-03-21T11:26:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.542+0000 7f564d7c1740 1 -- 172.21.15.47:0/869551006 wait complete. 2024-03-21T11:26:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.542+0000 7f564d7c1740 20 client.26237 trim_cache size 0 max 16384 2024-03-21T11:26:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:26:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:26:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:26:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:26:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.541+0000 7fade2a00640 1 -- 172.21.15.47:0/1862384335 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade80a4c60 con 0x562fcd5fe810 2024-03-21T11:26:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.543+0000 7fade2a00640 1 -- 172.21.15.47:0/1862384335 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fade809b860 con 0x562fcd5fe810 2024-03-21T11:26:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.543+0000 7fae06c4b740 1 -- 172.21.15.47:0/1862384335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fade80a0170 con 0x562fcd5fe810 2024-03-21T11:26:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.543+0000 7fade2a00640 1 -- 172.21.15.47:0/1862384335 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fade80672b0 con 0x562fcd5fe810 2024-03-21T11:26:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.543+0000 7fae06c4b740 10 client.26272 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.543+0000 7fae06c4b740 1 -- 172.21.15.47:0/1862384335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562fcd670020 con 0x562fcd5fe810 2024-03-21T11:26:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.544+0000 7fade2a00640 1 -- 172.21.15.47:0/1862384335 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fade80a0170 con 0x562fcd5fe810 2024-03-21T11:26:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.543+0000 7f564aa00640 1 -- 172.21.15.47:0/3605596733 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56280a3040 msgr2=0x7f56280a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.543+0000 7f564aa00640 1 --2- 172.21.15.47:0/3605596733 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56280a3040 0x7f56280a5500 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x558260ab8430 tx=0x7f563c009500 comp rx=0 tx=0).stop 2024-03-21T11:26:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.543+0000 7f564aa00640 1 -- 172.21.15.47:0/3605596733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5644104a90 msgr2=0x7f5644196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.543+0000 7f564aa00640 1 --2- 172.21.15.47:0/3605596733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5644104a90 0x7f5644196900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f563000e9c0 tx=0x7f563000ee80 comp rx=0 tx=0).stop 2024-03-21T11:26:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.544+0000 7f564aa00640 1 -- 172.21.15.47:0/3605596733 shutdown_connections 2024-03-21T11:26:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.544+0000 7f564aa00640 1 --2- 172.21.15.47:0/3605596733 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56280a3040 0x7f56280a5500 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.544+0000 7f564aa00640 1 --2- 172.21.15.47:0/3605596733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56441073c0 0x7f5644192f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.544+0000 7f564aa00640 1 --2- 172.21.15.47:0/3605596733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5644104a90 0x7f5644196900 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.544+0000 7f564aa00640 1 --2- 172.21.15.47:0/3605596733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5644102160 0x7f56441963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.544+0000 7f564aa00640 1 -- 172.21.15.47:0/3605596733 >> 172.21.15.47:0/3605596733 conn(0x7f56440fbb50 msgr2=0x7f5644101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.544+0000 7f564aa00640 1 -- 172.21.15.47:0/3605596733 shutdown_connections 2024-03-21T11:26:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:02.544+0000 7f564aa00640 1 -- 172.21.15.47:0/3605596733 wait complete. 2024-03-21T11:26:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.545+0000 7fade2a00640 1 -- 172.21.15.47:0/1862384335 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fade8097fa0 con 0x562fcd5fe810 2024-03-21T11:26:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.545+0000 7fae06c4b740 10 client.26272 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.545+0000 7fae06c4b740 10 client.26272 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:26:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.545+0000 7fae06c4b740 20 client.26272 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.545+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1862384335 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x562fcd6703b0 0x562fcd6727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.545+0000 7fae06c4b740 4 client.26272 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:26:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.545+0000 7fae06c4b740 1 -- 172.21.15.47:0/1862384335 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562fcd56ab50 con 0x562fcd6703b0 2024-03-21T11:26:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.545+0000 7fae00c00640 1 --2- 172.21.15.47:0/1862384335 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x562fcd6703b0 0x562fcd6727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.546+0000 7fae00c00640 1 --2- 172.21.15.47:0/1862384335 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x562fcd6703b0 0x562fcd6727a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=302f8f8944338ef9 server_cookie=7e9105884bc23d07 in_seq=0 out_seq=0 2024-03-21T11:26:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.546+0000 7fade2a00640 10 client.26272 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:26:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.547+0000 7fade2a00640 1 -- 172.21.15.47:0/1862384335 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562fcd56ab50 con 0x562fcd6703b0 2024-03-21T11:26:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.547+0000 7fade2a00640 10 client.26272 handle_command_reply: tid=0 2024-03-21T11:26:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.547+0000 7fae06c4b740 1 client.26272 shutdown 2024-03-21T11:26:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.547+0000 7fae06c4b740 1 -- 172.21.15.47:0/1862384335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 msgr2=0x562fcd60b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.547+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1862384335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd60b980 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fade80a4730 tx=0x7fade809a370 comp rx=0 tx=0).stop 2024-03-21T11:26:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.547+0000 7fae06c4b740 1 -- 172.21.15.47:0/1862384335 shutdown_connections 2024-03-21T11:26:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.547+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1862384335 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x562fcd6703b0 0x562fcd6727a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.547+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1862384335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fcd6003a0 0x562fcd60ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.547+0000 7fade2a00640 0 client.26272 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:26:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.547+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1862384335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fcd5ff9e0 0x562fcd60bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.547+0000 7fae06c4b740 1 --2- 172.21.15.47:0/1862384335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fcd5fe810 0x562fcd60b980 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.547+0000 7fae06c4b740 1 -- 172.21.15.47:0/1862384335 >> 172.21.15.47:0/1862384335 conn(0x562fcd5f9c00 msgr2=0x562fcd5fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.548+0000 7fae06c4b740 1 -- 172.21.15.47:0/1862384335 shutdown_connections 2024-03-21T11:26:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.548+0000 7fae06c4b740 1 -- 172.21.15.47:0/1862384335 wait complete. 2024-03-21T11:26:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.548+0000 7fae06c4b740 20 client.26272 trim_cache size 0 max 16384 2024-03-21T11:26:02.553 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:26:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.549+0000 7fae04000640 1 -- 172.21.15.47:0/868855277 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fade40a2f70 msgr2=0x7fade40a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.549+0000 7fae04000640 1 --2- 172.21.15.47:0/868855277 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fade40a2f70 0x7fade40a5430 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009230 tx=0x7fadf0005fd0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.549+0000 7fae04000640 1 -- 172.21.15.47:0/868855277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadfc102120 msgr2=0x7fadfc195780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.549+0000 7fae04000640 1 --2- 172.21.15.47:0/868855277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadfc102120 0x7fadfc195780 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fade800e9f0 tx=0x7fade800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.549+0000 7fae04000640 1 -- 172.21.15.47:0/868855277 shutdown_connections 2024-03-21T11:26:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.549+0000 7fae04000640 1 --2- 172.21.15.47:0/868855277 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fade40a2f70 0x7fade40a5430 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.549+0000 7fae04000640 1 --2- 172.21.15.47:0/868855277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fadfc107380 0x7fadfc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.549+0000 7fae04000640 1 --2- 172.21.15.47:0/868855277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadfc104a50 0x7fadfc195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.549+0000 7fae04000640 1 --2- 172.21.15.47:0/868855277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadfc102120 0x7fadfc195780 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.549+0000 7fae04000640 1 -- 172.21.15.47:0/868855277 >> 172.21.15.47:0/868855277 conn(0x7fadfc0fbb30 msgr2=0x7fadfc101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.550+0000 7fae04000640 1 -- 172.21.15.47:0/868855277 shutdown_connections 2024-03-21T11:26:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.550+0000 7fae04000640 1 -- 172.21.15.47:0/868855277 wait complete. 2024-03-21T11:26:02.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:02 smithi047 ceph-mon[31351]: pgmap v481: 129 pgs: 129 active+clean; 39 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-03-21T11:26:02.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:02 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1332937571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:26:02.571 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:26:02.571 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:26:02.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:02 smithi100 ceph-mon[36245]: pgmap v481: 129 pgs: 129 active+clean; 39 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-03-21T11:26:02.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:02 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1332937571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:26:02.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:02 smithi102 ceph-mon[36748]: pgmap v481: 129 pgs: 129 active+clean; 39 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-03-21T11:26:02.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:02 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1332937571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:26:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.829+0000 7f4671400640 1 -- 172.21.15.47:0/894234609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f466c100740 msgr2=0x7f466c100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.829+0000 7f4671400640 1 --2- 172.21.15.47:0/894234609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f466c100740 0x7f466c100bc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4658009900 tx=0x7f465802bc40 comp rx=0 tx=0).stop 2024-03-21T11:26:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.829+0000 7f4671400640 1 -- 172.21.15.47:0/894234609 shutdown_connections 2024-03-21T11:26:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.829+0000 7f4671400640 1 --2- 172.21.15.47:0/894234609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c101100 0x7f466c1089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.829+0000 7f4671400640 1 --2- 172.21.15.47:0/894234609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f466c100740 0x7f466c100bc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.829+0000 7f4671400640 1 --2- 172.21.15.47:0/894234609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f466c101a10 0x7f466c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.829+0000 7f4671400640 1 -- 172.21.15.47:0/894234609 >> 172.21.15.47:0/894234609 conn(0x7f466c0fb190 msgr2=0x7f466c0fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.829+0000 7f4671400640 1 -- 172.21.15.47:0/894234609 shutdown_connections 2024-03-21T11:26:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.829+0000 7f4671400640 1 -- 172.21.15.47:0/894234609 wait complete. 2024-03-21T11:26:02.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.830+0000 7f4671400640 1 Processor -- start 2024-03-21T11:26:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.830+0000 7f4671400640 1 -- start start 2024-03-21T11:26:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.830+0000 7f4671400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f466c100740 0x7f466c1044f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.830+0000 7f4671400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f466c101100 0x7f466c102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.830+0000 7f4671400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c101a10 0x7f466c103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.830+0000 7f4671400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c10c170 con 0x7f466c100740 2024-03-21T11:26:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.830+0000 7f4671400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c10bff0 con 0x7f466c101100 2024-03-21T11:26:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.830+0000 7f4671400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c10c2f0 con 0x7f466c101a10 2024-03-21T11:26:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.831+0000 7f4670a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c101a10 0x7f466c103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.831+0000 7f466be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f466c100740 0x7f466c1044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.831+0000 7f4670a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c101a10 0x7f466c103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34342/0 (socket says 172.21.15.47:34342) 2024-03-21T11:26:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.831+0000 7f466b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f466c101100 0x7f466c102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.831+0000 7f4670a00640 1 -- 172.21.15.47:0/1159074274 learned_addr learned my addr 172.21.15.47:0/1159074274 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.831+0000 7f4670a00640 1 -- 172.21.15.47:0/1159074274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f466c101100 msgr2=0x7f466c102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.831+0000 7f4670a00640 1 --2- 172.21.15.47:0/1159074274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f466c101100 0x7f466c102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.831+0000 7f4670a00640 1 -- 172.21.15.47:0/1159074274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f466c100740 msgr2=0x7f466c1044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.831+0000 7f4670a00640 1 --2- 172.21.15.47:0/1159074274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f466c100740 0x7f466c1044f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.831+0000 7f4670a00640 1 -- 172.21.15.47:0/1159074274 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c103900 con 0x7f466c101a10 2024-03-21T11:26:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.831+0000 7f4670a00640 1 --2- 172.21.15.47:0/1159074274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c101a10 0x7f466c103070 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f466400efd0 tx=0x7f466400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.831+0000 7f4668c00640 1 -- 172.21.15.47:0/1159074274 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664003b70 con 0x7f466c101a10 2024-03-21T11:26:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.832+0000 7f4668c00640 1 -- 172.21.15.47:0/1159074274 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4664003d10 con 0x7f466c101a10 2024-03-21T11:26:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.832+0000 7f4668c00640 1 -- 172.21.15.47:0/1159074274 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46640109f0 con 0x7f466c101a10 2024-03-21T11:26:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.832+0000 7f4671400640 1 -- 172.21.15.47:0/1159074274 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466c1a19b0 con 0x7f466c101a10 2024-03-21T11:26:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.832+0000 7f4671400640 1 -- 172.21.15.47:0/1159074274 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466c1a1e80 con 0x7f466c101a10 2024-03-21T11:26:02.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.835+0000 7f467581b740 1 Processor -- start 2024-03-21T11:26:02.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.835+0000 7f4668c00640 1 -- 172.21.15.47:0/1159074274 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4664004020 con 0x7f466c101a10 2024-03-21T11:26:02.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.836+0000 7f467581b740 1 -- start start 2024-03-21T11:26:02.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.836+0000 7f467581b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17bf9f0 0x558fe17bfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.836+0000 7f467581b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17c0bc0 0x558fe17c1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.836+0000 7f467581b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 0x558fe17c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.836+0000 7f467581b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe17242c0 con 0x558fe17c0bc0 2024-03-21T11:26:02.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.836+0000 7f467581b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe16ccc90 con 0x558fe17bf9f0 2024-03-21T11:26:02.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.836+0000 7f467581b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe1720360 con 0x558fe17c1580 2024-03-21T11:26:02.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.836+0000 7f466b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17bf9f0 0x558fe17bfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.837+0000 7f466be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17c0bc0 0x558fe17c1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.837+0000 7f466be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17c0bc0 0x558fe17c1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53816/0 (socket says 172.21.15.47:53816) 2024-03-21T11:26:02.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.837+0000 7f466b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17bf9f0 0x558fe17bfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34284/0 (socket says 172.21.15.47:34284) 2024-03-21T11:26:02.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.837+0000 7f466be00640 1 -- 172.21.15.47:0/3160964600 learned_addr learned my addr 172.21.15.47:0/3160964600 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.837+0000 7f466b400640 1 --2- 172.21.15.47:0/3160964600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 0x558fe17c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.837+0000 7f466be00640 1 -- 172.21.15.47:0/3160964600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 msgr2=0x558fe17c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.837+0000 7f466be00640 1 --2- 172.21.15.47:0/3160964600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 0x558fe17c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.837+0000 7f466be00640 1 -- 172.21.15.47:0/3160964600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17bf9f0 msgr2=0x558fe17bfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.837+0000 7f466be00640 1 --2- 172.21.15.47:0/3160964600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17bf9f0 0x558fe17bfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.837+0000 7f466be00640 1 -- 172.21.15.47:0/3160964600 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fe16f1080 con 0x558fe17c0bc0 2024-03-21T11:26:02.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.837+0000 7f466b400640 1 --2- 172.21.15.47:0/3160964600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 0x558fe17c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:02.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.838+0000 7f4668c00640 1 --2- 172.21.15.47:0/1159074274 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46500a2fa0 0x7f46500a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.838+0000 7f466be00640 1 --2- 172.21.15.47:0/3160964600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17c0bc0 0x558fe17c1040 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f465c009ca0 tx=0x7f465c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1349941c3a0fbb2d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.838+0000 7f4668c00640 1 -- 172.21.15.47:0/1159074274 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f46640cfb90 con 0x7f466c101a10 2024-03-21T11:26:02.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.838+0000 7f4661200640 1 -- 172.21.15.47:0/3160964600 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c004220 con 0x558fe17c0bc0 2024-03-21T11:26:02.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.838+0000 7f4661200640 1 -- 172.21.15.47:0/3160964600 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f465c0043c0 con 0x558fe17c0bc0 2024-03-21T11:26:02.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.838+0000 7f4661200640 1 -- 172.21.15.47:0/3160964600 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c019930 con 0x558fe17c0bc0 2024-03-21T11:26:02.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.839+0000 7f466b400640 1 --2- 172.21.15.47:0/1159074274 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46500a2fa0 0x7f46500a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.839+0000 7f467581b740 1 -- 172.21.15.47:0/3160964600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17c0bc0 msgr2=0x558fe17c1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.839+0000 7f467581b740 1 --2- 172.21.15.47:0/3160964600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17c0bc0 0x558fe17c1040 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f465c009ca0 tx=0x7f465c011810 comp rx=0 tx=0).stop 2024-03-21T11:26:02.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.840+0000 7f467581b740 1 -- 172.21.15.47:0/3160964600 shutdown_connections 2024-03-21T11:26:02.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.840+0000 7f467581b740 1 --2- 172.21.15.47:0/3160964600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 0x558fe17c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.840+0000 7f467581b740 1 --2- 172.21.15.47:0/3160964600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17c0bc0 0x558fe17c1040 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.840+0000 7f467581b740 1 --2- 172.21.15.47:0/3160964600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17bf9f0 0x558fe17bfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.840+0000 7f466b400640 1 --2- 172.21.15.47:0/1159074274 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46500a2fa0 0x7f46500a5460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f465802c250 tx=0x7f4658000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.840+0000 7f467581b740 1 -- 172.21.15.47:0/3160964600 >> 172.21.15.47:0/3160964600 conn(0x558fe17bade0 msgr2=0x558fe17bb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.840+0000 7f467581b740 1 -- 172.21.15.47:0/3160964600 shutdown_connections 2024-03-21T11:26:02.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.840+0000 7f467581b740 1 -- 172.21.15.47:0/3160964600 wait complete. 2024-03-21T11:26:02.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.840+0000 7f467581b740 1 -- start start 2024-03-21T11:26:02.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.841+0000 7f467581b740 1 Processor -- start 2024-03-21T11:26:02.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.841+0000 7f467581b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17bf9f0 0x558fe17c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.841+0000 7f467581b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.841+0000 7f467581b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 0x558fe17d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.841+0000 7f467581b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe17c8590 con 0x558fe17bf9f0 2024-03-21T11:26:02.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.841+0000 7f467581b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe16e00d0 con 0x558fe17c0bc0 2024-03-21T11:26:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.841+0000 7f467581b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe17c8710 con 0x558fe17c1580 2024-03-21T11:26:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.841+0000 7f467581b740 10 client.0 _ll_register_callbacks cb 0x558fe16e4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.842+0000 7f466be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17bf9f0 0x558fe17c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.842+0000 7f466be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17bf9f0 0x558fe17c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53824/0 (socket says 172.21.15.47:53824) 2024-03-21T11:26:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.842+0000 7f466be00640 1 -- 172.21.15.47:0/3642163405 learned_addr learned my addr 172.21.15.47:0/3642163405 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.842+0000 7f4670a00640 1 --2- 172.21.15.47:0/3642163405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 0x558fe17d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.842+0000 7f466b400640 1 --2- 172.21.15.47:0/3642163405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.842+0000 7f466be00640 1 -- 172.21.15.47:0/3642163405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 msgr2=0x558fe17d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.842+0000 7f466be00640 1 --2- 172.21.15.47:0/3642163405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 0x558fe17d1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.842+0000 7f466be00640 1 -- 172.21.15.47:0/3642163405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 msgr2=0x558fe17ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.842+0000 7f466be00640 1 --2- 172.21.15.47:0/3642163405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.842+0000 7f466be00640 1 -- 172.21.15.47:0/3642163405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fe14383f0 con 0x558fe17bf9f0 2024-03-21T11:26:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.842+0000 7f466b400640 1 --2- 172.21.15.47:0/3642163405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.843+0000 7f466be00640 1 --2- 172.21.15.47:0/3642163405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17bf9f0 0x558fe17c9c60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f465c011d10 tx=0x7f465c004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23eec67d92ca427 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.843+0000 7f4655600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:26:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.843+0000 7f4655600640 1 -- 172.21.15.47:0/3642163405 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c028070 con 0x558fe17bf9f0 2024-03-21T11:26:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.843+0000 7f4655600640 1 -- 172.21.15.47:0/3642163405 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f465c005040 con 0x558fe17bf9f0 2024-03-21T11:26:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.843+0000 7f467581b740 1 -- 172.21.15.47:0/3642163405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f465c0043c0 con 0x558fe17bf9f0 2024-03-21T11:26:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.844+0000 7f4655600640 1 -- 172.21.15.47:0/3642163405 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f465c034050 con 0x558fe17bf9f0 2024-03-21T11:26:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.844+0000 7f467581b740 10 client.16071 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.844+0000 7f467581b740 1 -- 172.21.15.47:0/3642163405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558fe15027d0 con 0x558fe17bf9f0 2024-03-21T11:26:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.844+0000 7f4655600640 1 -- 172.21.15.47:0/3642163405 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f465c0043c0 con 0x558fe17bf9f0 2024-03-21T11:26:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.844+0000 7f4655600640 1 -- 172.21.15.47:0/3642163405 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f465c004a30 con 0x558fe17bf9f0 2024-03-21T11:26:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.844+0000 7f467581b740 10 client.16071 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.844+0000 7f467581b740 10 client.16071 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:26:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.844+0000 7f467581b740 20 client.16071 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.844+0000 7f467581b740 1 --2- 172.21.15.47:0/3642163405 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558fe17d5380 0x558fe17d77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.844+0000 7f467581b740 4 client.16071 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.845+0000 7f467581b740 1 -- 172.21.15.47:0/3642163405 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fe1731800 con 0x558fe17d5380 2024-03-21T11:26:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.845+0000 7f466b400640 1 --2- 172.21.15.47:0/3642163405 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558fe17d5380 0x558fe17d77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.846+0000 7f4655600640 10 client.16071 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:26:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.846+0000 7f466b400640 1 --2- 172.21.15.47:0/3642163405 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558fe17d5380 0x558fe17d77b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=69eced20dbd62370 server_cookie=6681c24807eeb234 in_seq=0 out_seq=0 2024-03-21T11:26:02.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.846+0000 7f4655600640 1 -- 172.21.15.47:0/3642163405 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558fe1731800 con 0x558fe17d5380 2024-03-21T11:26:02.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.846+0000 7f4655600640 10 client.16071 handle_command_reply: tid=0 2024-03-21T11:26:02.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.847+0000 7f467581b740 1 client.16071 shutdown 2024-03-21T11:26:02.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.847+0000 7f467581b740 1 -- 172.21.15.47:0/3642163405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17bf9f0 msgr2=0x558fe17c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.847+0000 7f467581b740 1 --2- 172.21.15.47:0/3642163405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17bf9f0 0x558fe17c9c60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f465c011d10 tx=0x7f465c004350 comp rx=0 tx=0).stop 2024-03-21T11:26:02.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.847+0000 7f467581b740 1 -- 172.21.15.47:0/3642163405 shutdown_connections 2024-03-21T11:26:02.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.847+0000 7f467581b740 1 --2- 172.21.15.47:0/3642163405 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558fe17d5380 0x558fe17d77b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.847+0000 7f467581b740 1 --2- 172.21.15.47:0/3642163405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 0x558fe17d1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.847+0000 7f467581b740 1 --2- 172.21.15.47:0/3642163405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.847+0000 7f467581b740 1 --2- 172.21.15.47:0/3642163405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17bf9f0 0x558fe17c9c60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.847+0000 7f467581b740 1 -- 172.21.15.47:0/3642163405 >> 172.21.15.47:0/3642163405 conn(0x558fe17bade0 msgr2=0x558fe17ba160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.847+0000 7f4655600640 0 client.16071 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:26:02.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.847+0000 7f467581b740 1 -- 172.21.15.47:0/3642163405 shutdown_connections 2024-03-21T11:26:02.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.848+0000 7f467581b740 1 -- 172.21.15.47:0/3642163405 wait complete. 2024-03-21T11:26:02.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.848+0000 7f467581b740 20 client.16071 trim_cache size 0 max 16384 2024-03-21T11:26:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.864+0000 7f467581b740 1 Processor -- start 2024-03-21T11:26:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.864+0000 7f467581b740 1 -- start start 2024-03-21T11:26:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.864+0000 7f467581b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17bf9f0 0x558fe17dc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.864+0000 7f467581b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.864+0000 7f467581b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17c1580 0x558fe17dc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.864+0000 7f467581b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c1ac150 con 0x558fe17c1580 2024-03-21T11:26:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.864+0000 7f467581b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c1adad0 con 0x558fe17c0bc0 2024-03-21T11:26:02.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.864+0000 7f467581b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c1ac7b0 con 0x558fe17bf9f0 2024-03-21T11:26:02.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.864+0000 7f466be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17bf9f0 0x558fe17dc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.864+0000 7f466b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.864+0000 7f466be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17bf9f0 0x558fe17dc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34374/0 (socket says 172.21.15.47:34374) 2024-03-21T11:26:02.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.864+0000 7f466be00640 1 -- 172.21.15.47:0/4123268192 learned_addr learned my addr 172.21.15.47:0/4123268192 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.864+0000 7f466b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34298/0 (socket says 172.21.15.47:34298) 2024-03-21T11:26:02.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.865+0000 7f4670a00640 1 --2- 172.21.15.47:0/4123268192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17c1580 0x558fe17dc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.865+0000 7f466be00640 1 -- 172.21.15.47:0/4123268192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 msgr2=0x558fe17d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.865+0000 7f466be00640 1 --2- 172.21.15.47:0/4123268192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.865+0000 7f466be00640 1 -- 172.21.15.47:0/4123268192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17c1580 msgr2=0x558fe17dc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.865+0000 7f466be00640 1 --2- 172.21.15.47:0/4123268192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17c1580 0x558fe17dc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.865+0000 7f466be00640 1 -- 172.21.15.47:0/4123268192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fe17c0640 con 0x558fe17bf9f0 2024-03-21T11:26:02.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.865+0000 7f4670a00640 1 --2- 172.21.15.47:0/4123268192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17c1580 0x558fe17dc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:02.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.865+0000 7f466be00640 1 --2- 172.21.15.47:0/4123268192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17bf9f0 0x558fe17dc070 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f465c005970 tx=0x7f465c0059a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6990446ed6ba07f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.865+0000 7f4657e00640 1 -- 172.21.15.47:0/4123268192 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c028040 con 0x558fe17bf9f0 2024-03-21T11:26:02.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.865+0000 7f4657e00640 1 -- 172.21.15.47:0/4123268192 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f465c0043c0 con 0x558fe17bf9f0 2024-03-21T11:26:02.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.865+0000 7f4657e00640 1 -- 172.21.15.47:0/4123268192 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c004680 con 0x558fe17bf9f0 2024-03-21T11:26:02.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.866+0000 7f467581b740 1 -- 172.21.15.47:0/4123268192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17bf9f0 msgr2=0x558fe17dc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.866+0000 7f467581b740 1 --2- 172.21.15.47:0/4123268192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17bf9f0 0x558fe17dc070 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f465c005970 tx=0x7f465c0059a0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.866+0000 7f467581b740 1 -- 172.21.15.47:0/4123268192 shutdown_connections 2024-03-21T11:26:02.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.866+0000 7f467581b740 1 --2- 172.21.15.47:0/4123268192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17c1580 0x558fe17dc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.866+0000 7f467581b740 1 --2- 172.21.15.47:0/4123268192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.866+0000 7f467581b740 1 --2- 172.21.15.47:0/4123268192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17bf9f0 0x558fe17dc070 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.866+0000 7f467581b740 1 -- 172.21.15.47:0/4123268192 >> 172.21.15.47:0/4123268192 conn(0x558fe17bade0 msgr2=0x558fe17ba160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.866+0000 7f467581b740 1 -- 172.21.15.47:0/4123268192 shutdown_connections 2024-03-21T11:26:02.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.866+0000 7f467581b740 1 -- 172.21.15.47:0/4123268192 wait complete. 2024-03-21T11:26:02.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.867+0000 7f467581b740 1 -- start start 2024-03-21T11:26:02.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.867+0000 7f467581b740 1 Processor -- start 2024-03-21T11:26:02.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.867+0000 7f467581b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17bf9f0 0x558fe17cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.867+0000 7f467581b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.867+0000 7f467581b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 0x558fe17cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.867+0000 7f467581b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c1ad470 con 0x558fe17bf9f0 2024-03-21T11:26:02.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.867+0000 7f467581b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c1ace10 con 0x558fe17c0bc0 2024-03-21T11:26:02.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.867+0000 7f467581b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c1aa310 con 0x558fe17c1580 2024-03-21T11:26:02.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.867+0000 7f467581b740 10 client.0 _ll_register_callbacks cb 0x7f46640093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:02.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.868+0000 7f466b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.868+0000 7f466be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17bf9f0 0x558fe17cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.868+0000 7f466b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34306/0 (socket says 172.21.15.47:34306) 2024-03-21T11:26:02.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.868+0000 7f466b400640 1 -- 172.21.15.47:0/68588651 learned_addr learned my addr 172.21.15.47:0/68588651 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:02.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.868+0000 7f4670a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 0x558fe17cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.868+0000 7f466b400640 1 -- 172.21.15.47:0/68588651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 msgr2=0x558fe17cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.868+0000 7f466b400640 1 --2- 172.21.15.47:0/68588651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 0x558fe17cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.868+0000 7f466b400640 1 -- 172.21.15.47:0/68588651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17bf9f0 msgr2=0x558fe17cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:02.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.868+0000 7f466b400640 1 --2- 172.21.15.47:0/68588651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17bf9f0 0x558fe17cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.868+0000 7f466b400640 1 -- 172.21.15.47:0/68588651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fe17d0340 con 0x558fe17c0bc0 2024-03-21T11:26:02.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.868+0000 7f4670a00640 1 --2- 172.21.15.47:0/68588651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 0x558fe17cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:02.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.868+0000 7f466b400640 1 --2- 172.21.15.47:0/68588651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17ccd30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f465802c7c0 tx=0x7f465802cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=632830523a5b674 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:02.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.868+0000 7f4655600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:02.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.869+0000 7f4655600640 1 -- 172.21.15.47:0/68588651 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46580046c0 con 0x558fe17c0bc0 2024-03-21T11:26:02.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.869+0000 7f4655600640 1 -- 172.21.15.47:0/68588651 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4658004860 con 0x558fe17c0bc0 2024-03-21T11:26:02.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.869+0000 7f467581b740 1 -- 172.21.15.47:0/68588651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f465c0043c0 con 0x558fe17c0bc0 2024-03-21T11:26:02.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.869+0000 7f4655600640 1 -- 172.21.15.47:0/68588651 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4658064050 con 0x558fe17c0bc0 2024-03-21T11:26:02.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.869+0000 7f467581b740 10 client.26273 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:02.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.869+0000 7f467581b740 1 -- 172.21.15.47:0/68588651 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558fe1830cb0 con 0x558fe17c0bc0 2024-03-21T11:26:02.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.869+0000 7f4655600640 1 -- 172.21.15.47:0/68588651 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f465c0043c0 con 0x558fe17c0bc0 2024-03-21T11:26:02.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.870+0000 7f4655600640 1 -- 172.21.15.47:0/68588651 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f465804d040 con 0x558fe17c0bc0 2024-03-21T11:26:02.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.870+0000 7f467581b740 10 client.26273 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:02.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.870+0000 7f467581b740 10 client.26273 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:26:02.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.870+0000 7f467581b740 20 client.26273 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:02.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.870+0000 7f467581b740 1 --2- 172.21.15.47:0/68588651 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558fe18315e0 0x558fe1833a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:02.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.870+0000 7f467581b740 4 client.26273 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:26:02.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.870+0000 7f467581b740 1 -- 172.21.15.47:0/68588651 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558fe172bb50 con 0x558fe18315e0 2024-03-21T11:26:02.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.870+0000 7f466be00640 1 --2- 172.21.15.47:0/68588651 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558fe18315e0 0x558fe1833a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:02.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f466be00640 1 --2- 172.21.15.47:0/68588651 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558fe18315e0 0x558fe1833a00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=60c0e86faeb5ea38 server_cookie=333da8cd5d34abc3 in_seq=0 out_seq=0 2024-03-21T11:26:02.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f4655600640 10 client.26273 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:26:02.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f4655600640 1 -- 172.21.15.47:0/68588651 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558fe172bb50 con 0x558fe18315e0 2024-03-21T11:26:02.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f4655600640 10 client.26273 handle_command_reply: tid=0 2024-03-21T11:26:02.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f467581b740 1 client.26273 shutdown 2024-03-21T11:26:02.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f467581b740 1 -- 172.21.15.47:0/68588651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 msgr2=0x558fe17ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f467581b740 1 --2- 172.21.15.47:0/68588651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17ccd30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f465802c7c0 tx=0x7f465802cf90 comp rx=0 tx=0).stop 2024-03-21T11:26:02.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f467581b740 1 -- 172.21.15.47:0/68588651 shutdown_connections 2024-03-21T11:26:02.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f467581b740 1 --2- 172.21.15.47:0/68588651 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558fe18315e0 0x558fe1833a00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f467581b740 1 --2- 172.21.15.47:0/68588651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558fe17c1580 0x558fe17cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f4655600640 0 client.26273 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:26:02.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f467581b740 1 --2- 172.21.15.47:0/68588651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558fe17c0bc0 0x558fe17ccd30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f467581b740 1 --2- 172.21.15.47:0/68588651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558fe17bf9f0 0x558fe17cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f467581b740 1 -- 172.21.15.47:0/68588651 >> 172.21.15.47:0/68588651 conn(0x558fe17bade0 msgr2=0x558fe17c3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f467581b740 1 -- 172.21.15.47:0/68588651 shutdown_connections 2024-03-21T11:26:02.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.871+0000 7f467581b740 1 -- 172.21.15.47:0/68588651 wait complete. 2024-03-21T11:26:02.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.872+0000 7f467581b740 20 client.26273 trim_cache size 0 max 16384 2024-03-21T11:26:02.877 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:26:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.872+0000 7f4671400640 1 -- 172.21.15.47:0/1159074274 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46500a2fa0 msgr2=0x7f46500a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.872+0000 7f4671400640 1 --2- 172.21.15.47:0/1159074274 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46500a2fa0 0x7f46500a5460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f465802c250 tx=0x7f4658000bc0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.872+0000 7f4671400640 1 -- 172.21.15.47:0/1159074274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c101a10 msgr2=0x7f466c103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.872+0000 7f4671400640 1 --2- 172.21.15.47:0/1159074274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c101a10 0x7f466c103070 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f466400efd0 tx=0x7f466400c6a0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.872+0000 7f4671400640 1 -- 172.21.15.47:0/1159074274 shutdown_connections 2024-03-21T11:26:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.872+0000 7f4671400640 1 --2- 172.21.15.47:0/1159074274 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46500a2fa0 0x7f46500a5460 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.872+0000 7f4671400640 1 --2- 172.21.15.47:0/1159074274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c101a10 0x7f466c103070 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.872+0000 7f4671400640 1 --2- 172.21.15.47:0/1159074274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f466c101100 0x7f466c102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.872+0000 7f4671400640 1 --2- 172.21.15.47:0/1159074274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f466c100740 0x7f466c1044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.872+0000 7f4671400640 1 -- 172.21.15.47:0/1159074274 >> 172.21.15.47:0/1159074274 conn(0x7f466c0fb190 msgr2=0x7f466c0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.872+0000 7f4671400640 1 -- 172.21.15.47:0/1159074274 shutdown_connections 2024-03-21T11:26:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:02.873+0000 7f4671400640 1 -- 172.21.15.47:0/1159074274 wait complete. 2024-03-21T11:26:02.894 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:26:02.894 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:26:03.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.115+0000 7f382f000640 1 -- 172.21.15.47:0/765480656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38281016b0 msgr2=0x7f3828101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.115+0000 7f382f000640 1 --2- 172.21.15.47:0/765480656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38281016b0 0x7f3828101ad0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3818009ca0 tx=0x7f381802c0f0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.115+0000 7f382f000640 1 -- 172.21.15.47:0/765480656 shutdown_connections 2024-03-21T11:26:03.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.115+0000 7f382f000640 1 --2- 172.21.15.47:0/765480656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3828103260 0x7f3828109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.115+0000 7f382f000640 1 --2- 172.21.15.47:0/765480656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38281028a0 0x7f3828102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.115+0000 7f382f000640 1 --2- 172.21.15.47:0/765480656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38281016b0 0x7f3828101ad0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.115+0000 7f382f000640 1 -- 172.21.15.47:0/765480656 >> 172.21.15.47:0/765480656 conn(0x7f38280fcc40 msgr2=0x7f38280ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.115+0000 7f382f000640 1 -- 172.21.15.47:0/765480656 shutdown_connections 2024-03-21T11:26:03.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.115+0000 7f382f000640 1 -- 172.21.15.47:0/765480656 wait complete. 2024-03-21T11:26:03.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.116+0000 7f382f000640 1 Processor -- start 2024-03-21T11:26:03.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.116+0000 7f382f000640 1 -- start start 2024-03-21T11:26:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.116+0000 7f382f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38281016b0 0x7f3828075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.116+0000 7f382f000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38281028a0 0x7f3828073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.116+0000 7f382f000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3828103260 0x7f3828073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.116+0000 7f382f000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382810d330 con 0x7f3828103260 2024-03-21T11:26:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.116+0000 7f382f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382810d1b0 con 0x7f38281016b0 2024-03-21T11:26:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.116+0000 7f382f000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382810d4b0 con 0x7f38281028a0 2024-03-21T11:26:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.117+0000 7f3827e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38281016b0 0x7f3828075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.117+0000 7f381f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38281028a0 0x7f3828073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.117+0000 7f381f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38281028a0 0x7f3828073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34404/0 (socket says 172.21.15.47:34404) 2024-03-21T11:26:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.117+0000 7f381f400640 1 -- 172.21.15.47:0/2051577197 learned_addr learned my addr 172.21.15.47:0/2051577197 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.117+0000 7f3827e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38281016b0 0x7f3828075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34332/0 (socket says 172.21.15.47:34332) 2024-03-21T11:26:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.117+0000 7f382d000640 1 --2- 172.21.15.47:0/2051577197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3828103260 0x7f3828073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.117+0000 7f381f400640 1 -- 172.21.15.47:0/2051577197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38281016b0 msgr2=0x7f3828075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.117+0000 7f381f400640 1 --2- 172.21.15.47:0/2051577197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38281016b0 0x7f3828075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.117+0000 7f381f400640 1 -- 172.21.15.47:0/2051577197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3828103260 msgr2=0x7f3828073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.117+0000 7f381f400640 1 --2- 172.21.15.47:0/2051577197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3828103260 0x7f3828073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.117+0000 7f381f400640 1 -- 172.21.15.47:0/2051577197 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3828074860 con 0x7f38281028a0 2024-03-21T11:26:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.117+0000 7f3827e00640 1 --2- 172.21.15.47:0/2051577197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38281016b0 0x7f3828075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.118+0000 7f381f400640 1 --2- 172.21.15.47:0/2051577197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38281028a0 0x7f3828073a90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f381000e9f0 tx=0x7f381000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.118+0000 7f3825600640 1 -- 172.21.15.47:0/2051577197 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381000cd80 con 0x7f38281028a0 2024-03-21T11:26:03.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.118+0000 7f3825600640 1 -- 172.21.15.47:0/2051577197 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3810005500 con 0x7f38281028a0 2024-03-21T11:26:03.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.118+0000 7f382f000640 1 -- 172.21.15.47:0/2051577197 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38281a1b50 con 0x7f38281028a0 2024-03-21T11:26:03.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.118+0000 7f3825600640 1 -- 172.21.15.47:0/2051577197 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810010690 con 0x7f38281028a0 2024-03-21T11:26:03.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.118+0000 7f382f000640 1 -- 172.21.15.47:0/2051577197 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38281a20a0 con 0x7f38281028a0 2024-03-21T11:26:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.121+0000 7f3825600640 1 -- 172.21.15.47:0/2051577197 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f38100046c0 con 0x7f38281028a0 2024-03-21T11:26:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.121+0000 7f3831c70740 1 Processor -- start 2024-03-21T11:26:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.122+0000 7f3831c70740 1 -- start start 2024-03-21T11:26:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.122+0000 7f3831c70740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce21810 0x5560dce21c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.122+0000 7f3831c70740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce22e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.122+0000 7f3831c70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce233a0 0x5560dce29c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.122+0000 7f3831c70740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560dcd862c0 con 0x5560dce21810 2024-03-21T11:26:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.122+0000 7f3831c70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560dcd2ec90 con 0x5560dce233a0 2024-03-21T11:26:03.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.122+0000 7f3831c70740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560dcd82360 con 0x5560dce229e0 2024-03-21T11:26:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.122+0000 7f382d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce21810 0x5560dce21c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.122+0000 7f382d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce21810 0x5560dce21c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53852/0 (socket says 172.21.15.47:53852) 2024-03-21T11:26:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.122+0000 7f382d000640 1 -- 172.21.15.47:0/1808429450 learned_addr learned my addr 172.21.15.47:0/1808429450 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.122+0000 7f3827e00640 1 --2- 172.21.15.47:0/1808429450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce22e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.122+0000 7f381f400640 1 --2- 172.21.15.47:0/1808429450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce233a0 0x5560dce29c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.123+0000 7f382d000640 1 -- 172.21.15.47:0/1808429450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 msgr2=0x5560dce22e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.123+0000 7f382d000640 1 --2- 172.21.15.47:0/1808429450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce22e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.123+0000 7f382d000640 1 -- 172.21.15.47:0/1808429450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce233a0 msgr2=0x5560dce29c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.123+0000 7f382d000640 1 --2- 172.21.15.47:0/1808429450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce233a0 0x5560dce29c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.123+0000 7f382d000640 1 -- 172.21.15.47:0/1808429450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560dcd53080 con 0x5560dce21810 2024-03-21T11:26:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.123+0000 7f382d000640 1 --2- 172.21.15.47:0/1808429450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce21810 0x5560dce21c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3820009ca0 tx=0x7f3820011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c088d903aea9e26 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.123+0000 7f3825600640 1 --2- 172.21.15.47:0/2051577197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f380c0a2fa0 0x7f380c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.123+0000 7f381d600640 1 -- 172.21.15.47:0/1808429450 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382001e070 con 0x5560dce21810 2024-03-21T11:26:03.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.123+0000 7f381d600640 1 -- 172.21.15.47:0/1808429450 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3820009e50 con 0x5560dce21810 2024-03-21T11:26:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.124+0000 7f381d600640 1 -- 172.21.15.47:0/1808429450 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382000ea80 con 0x5560dce21810 2024-03-21T11:26:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.124+0000 7f3825600640 1 -- 172.21.15.47:0/2051577197 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f38100cfb70 con 0x7f38281028a0 2024-03-21T11:26:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.124+0000 7f3827e00640 1 --2- 172.21.15.47:0/2051577197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f380c0a2fa0 0x7f380c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.124+0000 7f3831c70740 1 -- 172.21.15.47:0/1808429450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce21810 msgr2=0x5560dce21c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.124+0000 7f3831c70740 1 --2- 172.21.15.47:0/1808429450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce21810 0x5560dce21c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3820009ca0 tx=0x7f3820011810 comp rx=0 tx=0).stop 2024-03-21T11:26:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.124+0000 7f3831c70740 1 -- 172.21.15.47:0/1808429450 shutdown_connections 2024-03-21T11:26:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.124+0000 7f3831c70740 1 --2- 172.21.15.47:0/1808429450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce233a0 0x5560dce29c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.124+0000 7f3831c70740 1 --2- 172.21.15.47:0/1808429450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce22e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.124+0000 7f3831c70740 1 --2- 172.21.15.47:0/1808429450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce21810 0x5560dce21c30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.124+0000 7f3831c70740 1 -- 172.21.15.47:0/1808429450 >> 172.21.15.47:0/1808429450 conn(0x5560dce1cc00 msgr2=0x5560dce1d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.124+0000 7f3827e00640 1 --2- 172.21.15.47:0/2051577197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f380c0a2fa0 0x7f380c0a5460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f381802c630 tx=0x7f3818009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.125+0000 7f3831c70740 1 -- 172.21.15.47:0/1808429450 shutdown_connections 2024-03-21T11:26:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.125+0000 7f3831c70740 1 -- 172.21.15.47:0/1808429450 wait complete. 2024-03-21T11:26:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.125+0000 7f3831c70740 1 -- start start 2024-03-21T11:26:03.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.125+0000 7f3831c70740 1 Processor -- start 2024-03-21T11:26:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.126+0000 7f3831c70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce21810 0x5560dce2bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.126+0000 7f3831c70740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce2c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.126+0000 7f3831c70740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 0x5560dce33010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.126+0000 7f3831c70740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560dce2a3b0 con 0x5560dce233a0 2024-03-21T11:26:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.126+0000 7f3831c70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560dcd420d0 con 0x5560dce21810 2024-03-21T11:26:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.126+0000 7f3831c70740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560dce2a530 con 0x5560dce229e0 2024-03-21T11:26:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.126+0000 7f3831c70740 10 client.0 _ll_register_callbacks cb 0x5560dcd46e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.126+0000 7f381f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce2c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.126+0000 7f3827e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce21810 0x5560dce2bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.126+0000 7f381f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce2c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34418/0 (socket says 172.21.15.47:34418) 2024-03-21T11:26:03.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.126+0000 7f381f400640 1 -- 172.21.15.47:0/1504537349 learned_addr learned my addr 172.21.15.47:0/1504537349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.126+0000 7f382d000640 1 --2- 172.21.15.47:0/1504537349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 0x5560dce33010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.126+0000 7f381f400640 1 -- 172.21.15.47:0/1504537349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce21810 msgr2=0x5560dce2bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.126+0000 7f381f400640 1 --2- 172.21.15.47:0/1504537349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce21810 0x5560dce2bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.127+0000 7f381f400640 1 -- 172.21.15.47:0/1504537349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 msgr2=0x5560dce33010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.127+0000 7f381f400640 1 --2- 172.21.15.47:0/1504537349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 0x5560dce33010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.127+0000 7f381f400640 1 -- 172.21.15.47:0/1504537349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560dca9a3f0 con 0x5560dce229e0 2024-03-21T11:26:03.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.127+0000 7f382d000640 1 --2- 172.21.15.47:0/1504537349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 0x5560dce33010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:03.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.127+0000 7f381f400640 1 --2- 172.21.15.47:0/1504537349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce2c040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f38100a4390 tx=0x7f3810004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fa74d2482d7004a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.127+0000 7f3816000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:03.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.127+0000 7f3816000640 1 -- 172.21.15.47:0/1504537349 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38100d7040 con 0x5560dce229e0 2024-03-21T11:26:03.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.127+0000 7f3816000640 1 -- 172.21.15.47:0/1504537349 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f38100cd390 con 0x5560dce229e0 2024-03-21T11:26:03.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.127+0000 7f3831c70740 1 -- 172.21.15.47:0/1504537349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3820009e50 con 0x5560dce229e0 2024-03-21T11:26:03.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.127+0000 7f3816000640 1 -- 172.21.15.47:0/1504537349 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f381009b810 con 0x5560dce229e0 2024-03-21T11:26:03.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.127+0000 7f3831c70740 10 client.26302 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:03.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.127+0000 7f3831c70740 1 -- 172.21.15.47:0/1504537349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5560dcb647d0 con 0x5560dce229e0 2024-03-21T11:26:03.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.128+0000 7f3816000640 1 -- 172.21.15.47:0/1504537349 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3820009e50 con 0x5560dce229e0 2024-03-21T11:26:03.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.128+0000 7f3816000640 1 -- 172.21.15.47:0/1504537349 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f38100a4430 con 0x5560dce229e0 2024-03-21T11:26:03.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.128+0000 7f3831c70740 10 client.26302 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:03.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.128+0000 7f3831c70740 10 client.26302 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:26:03.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.128+0000 7f3831c70740 20 client.26302 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:03.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.128+0000 7f3831c70740 1 --2- 172.21.15.47:0/1504537349 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5560dce37600 0x5560dce39a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.128+0000 7f3831c70740 4 client.26302 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.128+0000 7f3831c70740 1 -- 172.21.15.47:0/1504537349 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560dcd93800 con 0x5560dce37600 2024-03-21T11:26:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.128+0000 7f3827e00640 1 --2- 172.21.15.47:0/1504537349 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5560dce37600 0x5560dce39a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.129+0000 7f3827e00640 1 --2- 172.21.15.47:0/1504537349 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5560dce37600 0x5560dce39a20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41e5aea194c26960 server_cookie=c2bec3b0e94b3ac4 in_seq=0 out_seq=0 2024-03-21T11:26:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.129+0000 7f3816000640 10 client.26302 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:26:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.130+0000 7f3816000640 1 -- 172.21.15.47:0/1504537349 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5560dcd93800 con 0x5560dce37600 2024-03-21T11:26:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.130+0000 7f3816000640 10 client.26302 handle_command_reply: tid=0 2024-03-21T11:26:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.130+0000 7f3831c70740 1 client.26302 shutdown 2024-03-21T11:26:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.130+0000 7f3831c70740 1 -- 172.21.15.47:0/1504537349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 msgr2=0x5560dce2c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.130+0000 7f3831c70740 1 --2- 172.21.15.47:0/1504537349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce2c040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f38100a4390 tx=0x7f3810004ba0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.130+0000 7f3831c70740 1 -- 172.21.15.47:0/1504537349 shutdown_connections 2024-03-21T11:26:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.130+0000 7f3831c70740 1 --2- 172.21.15.47:0/1504537349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 0x5560dce33010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.130+0000 7f3831c70740 1 --2- 172.21.15.47:0/1504537349 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5560dce37600 0x5560dce39a20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.130+0000 7f3831c70740 1 --2- 172.21.15.47:0/1504537349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce2c040 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.130+0000 7f3831c70740 1 --2- 172.21.15.47:0/1504537349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce21810 0x5560dce2bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.130+0000 7f3816000640 0 client.26302 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:26:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.130+0000 7f3831c70740 1 -- 172.21.15.47:0/1504537349 >> 172.21.15.47:0/1504537349 conn(0x5560dce1cc00 msgr2=0x5560dce1beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.130+0000 7f3831c70740 1 -- 172.21.15.47:0/1504537349 shutdown_connections 2024-03-21T11:26:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.131+0000 7f3831c70740 1 -- 172.21.15.47:0/1504537349 wait complete. 2024-03-21T11:26:03.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.131+0000 7f3831c70740 20 client.26302 trim_cache size 0 max 16384 2024-03-21T11:26:03.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.146+0000 7f3831c70740 1 Processor -- start 2024-03-21T11:26:03.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.146+0000 7f3831c70740 1 -- start start 2024-03-21T11:26:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f3831c70740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce21810 0x5560dce25b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f3831c70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce229e0 0x5560dce32fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f3831c70740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 0x5560dce26060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f3831c70740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38281ac290 con 0x5560dce233a0 2024-03-21T11:26:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f3831c70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38281adc10 con 0x5560dce229e0 2024-03-21T11:26:03.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f3831c70740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38281ac8f0 con 0x5560dce21810 2024-03-21T11:26:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f3827e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce21810 0x5560dce25b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f381f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce229e0 0x5560dce32fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f3827e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce21810 0x5560dce25b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34424/0 (socket says 172.21.15.47:34424) 2024-03-21T11:26:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f381f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce229e0 0x5560dce32fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34364/0 (socket says 172.21.15.47:34364) 2024-03-21T11:26:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f3827e00640 1 -- 172.21.15.47:0/2398783808 learned_addr learned my addr 172.21.15.47:0/2398783808 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f382d000640 1 --2- 172.21.15.47:0/2398783808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 0x5560dce26060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f3827e00640 1 -- 172.21.15.47:0/2398783808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce229e0 msgr2=0x5560dce32fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f3827e00640 1 --2- 172.21.15.47:0/2398783808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce229e0 0x5560dce32fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f3827e00640 1 -- 172.21.15.47:0/2398783808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 msgr2=0x5560dce26060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f3827e00640 1 --2- 172.21.15.47:0/2398783808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 0x5560dce26060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.147+0000 7f3827e00640 1 -- 172.21.15.47:0/2398783808 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560dce29640 con 0x5560dce21810 2024-03-21T11:26:03.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.148+0000 7f3827e00640 1 --2- 172.21.15.47:0/2398783808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce21810 0x5560dce25b20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3818000f80 tx=0x7f381802e050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a6a70511a4095ff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.148+0000 7f381cc00640 1 -- 172.21.15.47:0/2398783808 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3818039040 con 0x5560dce21810 2024-03-21T11:26:03.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.148+0000 7f381cc00640 1 -- 172.21.15.47:0/2398783808 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3818052070 con 0x5560dce21810 2024-03-21T11:26:03.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.148+0000 7f381cc00640 1 -- 172.21.15.47:0/2398783808 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3818053c40 con 0x5560dce21810 2024-03-21T11:26:03.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.149+0000 7f3831c70740 1 -- 172.21.15.47:0/2398783808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce21810 msgr2=0x5560dce25b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.149+0000 7f3831c70740 1 --2- 172.21.15.47:0/2398783808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce21810 0x5560dce25b20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3818000f80 tx=0x7f381802e050 comp rx=0 tx=0).stop 2024-03-21T11:26:03.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.149+0000 7f3831c70740 1 -- 172.21.15.47:0/2398783808 shutdown_connections 2024-03-21T11:26:03.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.149+0000 7f3831c70740 1 --2- 172.21.15.47:0/2398783808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 0x5560dce26060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.149+0000 7f3831c70740 1 --2- 172.21.15.47:0/2398783808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce229e0 0x5560dce32fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.149+0000 7f3831c70740 1 --2- 172.21.15.47:0/2398783808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce21810 0x5560dce25b20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.149+0000 7f3831c70740 1 -- 172.21.15.47:0/2398783808 >> 172.21.15.47:0/2398783808 conn(0x5560dce1cc00 msgr2=0x5560dce206f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.149+0000 7f3831c70740 1 -- 172.21.15.47:0/2398783808 shutdown_connections 2024-03-21T11:26:03.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.149+0000 7f3831c70740 1 -- 172.21.15.47:0/2398783808 wait complete. 2024-03-21T11:26:03.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.150+0000 7f3831c70740 1 -- start start 2024-03-21T11:26:03.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.150+0000 7f3831c70740 1 Processor -- start 2024-03-21T11:26:03.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.150+0000 7f3831c70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce21810 0x5560dce329f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.150+0000 7f3831c70740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce32f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.150+0000 7f3831c70740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 0x5560dce2f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.150+0000 7f3831c70740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38281ad5b0 con 0x5560dce233a0 2024-03-21T11:26:03.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.150+0000 7f3831c70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38281acf50 con 0x5560dce21810 2024-03-21T11:26:03.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.150+0000 7f3831c70740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38281aa450 con 0x5560dce229e0 2024-03-21T11:26:03.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.150+0000 7f3831c70740 10 client.0 _ll_register_callbacks cb 0x7f381000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:03.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.150+0000 7f381f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce32f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.150+0000 7f3827e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce21810 0x5560dce329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.151+0000 7f381f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce32f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34440/0 (socket says 172.21.15.47:34440) 2024-03-21T11:26:03.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.151+0000 7f381f400640 1 -- 172.21.15.47:0/3979829992 learned_addr learned my addr 172.21.15.47:0/3979829992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.151+0000 7f382d000640 1 --2- 172.21.15.47:0/3979829992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 0x5560dce2f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.151+0000 7f381f400640 1 -- 172.21.15.47:0/3979829992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce21810 msgr2=0x5560dce329f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.151+0000 7f381f400640 1 --2- 172.21.15.47:0/3979829992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce21810 0x5560dce329f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.151+0000 7f381f400640 1 -- 172.21.15.47:0/3979829992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 msgr2=0x5560dce2f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.151+0000 7f381f400640 1 --2- 172.21.15.47:0/3979829992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 0x5560dce2f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.151+0000 7f381f400640 1 -- 172.21.15.47:0/3979829992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560dce2fd70 con 0x5560dce229e0 2024-03-21T11:26:03.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.151+0000 7f3827e00640 1 --2- 172.21.15.47:0/3979829992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce21810 0x5560dce329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:03.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.151+0000 7f381f400640 1 --2- 172.21.15.47:0/3979829992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce32f30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f38100049d0 tx=0x7f38100cfdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60d1a6b944ec8808 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.151+0000 7f3816000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:03.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.151+0000 7f3816000640 1 -- 172.21.15.47:0/3979829992 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381009bc80 con 0x5560dce229e0 2024-03-21T11:26:03.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.151+0000 7f3816000640 1 -- 172.21.15.47:0/3979829992 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f38100cd300 con 0x5560dce229e0 2024-03-21T11:26:03.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.151+0000 7f3831c70740 1 -- 172.21.15.47:0/3979829992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3818052070 con 0x5560dce229e0 2024-03-21T11:26:03.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.152+0000 7f3816000640 1 -- 172.21.15.47:0/3979829992 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f381009b350 con 0x5560dce229e0 2024-03-21T11:26:03.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.152+0000 7f3831c70740 10 client.26308 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:03.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.152+0000 7f3831c70740 1 -- 172.21.15.47:0/3979829992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5560dce93cf0 con 0x5560dce229e0 2024-03-21T11:26:03.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.152+0000 7f3816000640 1 -- 172.21.15.47:0/3979829992 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38100a8020 con 0x5560dce229e0 2024-03-21T11:26:03.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.152+0000 7f3816000640 1 -- 172.21.15.47:0/3979829992 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3810097c00 con 0x5560dce229e0 2024-03-21T11:26:03.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.152+0000 7f3831c70740 10 client.26308 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:03.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.152+0000 7f3831c70740 10 client.26308 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:26:03.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.152+0000 7f3831c70740 20 client.26308 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:03.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.152+0000 7f3831c70740 1 --2- 172.21.15.47:0/3979829992 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5560dce945f0 0x5560dce96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.152+0000 7f3831c70740 4 client.26308 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:26:03.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.152+0000 7f3831c70740 1 -- 172.21.15.47:0/3979829992 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5560dcd8db50 con 0x5560dce945f0 2024-03-21T11:26:03.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.152+0000 7f3827e00640 1 --2- 172.21.15.47:0/3979829992 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5560dce945f0 0x5560dce96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.153+0000 7f3827e00640 1 --2- 172.21.15.47:0/3979829992 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5560dce945f0 0x5560dce96a10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d69db1dfbb9e7762 server_cookie=4916dd56d3e09012 in_seq=0 out_seq=0 2024-03-21T11:26:03.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.153+0000 7f3816000640 10 client.26308 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:26:03.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.153+0000 7f3816000640 1 -- 172.21.15.47:0/3979829992 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5560dcd8db50 con 0x5560dce945f0 2024-03-21T11:26:03.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.153+0000 7f3816000640 10 client.26308 handle_command_reply: tid=0 2024-03-21T11:26:03.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.153+0000 7f3831c70740 1 client.26308 shutdown 2024-03-21T11:26:03.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.153+0000 7f3831c70740 1 -- 172.21.15.47:0/3979829992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 msgr2=0x5560dce32f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.153+0000 7f3831c70740 1 --2- 172.21.15.47:0/3979829992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce32f30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f38100049d0 tx=0x7f38100cfdf0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.153+0000 7f3831c70740 1 -- 172.21.15.47:0/3979829992 shutdown_connections 2024-03-21T11:26:03.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.153+0000 7f3831c70740 1 --2- 172.21.15.47:0/3979829992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560dce233a0 0x5560dce2f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.153+0000 7f3831c70740 1 --2- 172.21.15.47:0/3979829992 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5560dce945f0 0x5560dce96a10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.153+0000 7f3831c70740 1 --2- 172.21.15.47:0/3979829992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560dce229e0 0x5560dce32f30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.153+0000 7f3831c70740 1 --2- 172.21.15.47:0/3979829992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560dce21810 0x5560dce329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.153+0000 7f3816000640 0 client.26308 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:26:03.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.153+0000 7f3831c70740 1 -- 172.21.15.47:0/3979829992 >> 172.21.15.47:0/3979829992 conn(0x5560dce1cc00 msgr2=0x5560dce1b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.154+0000 7f3831c70740 1 -- 172.21.15.47:0/3979829992 shutdown_connections 2024-03-21T11:26:03.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.154+0000 7f3831c70740 1 -- 172.21.15.47:0/3979829992 wait complete. 2024-03-21T11:26:03.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.154+0000 7f3831c70740 20 client.26308 trim_cache size 0 max 16384 2024-03-21T11:26:03.160 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:26:03.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.154+0000 7f382f000640 1 -- 172.21.15.47:0/2051577197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f380c0a2fa0 msgr2=0x7f380c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.154+0000 7f382f000640 1 --2- 172.21.15.47:0/2051577197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f380c0a2fa0 0x7f380c0a5460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f381802c630 tx=0x7f3818009360 comp rx=0 tx=0).stop 2024-03-21T11:26:03.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.154+0000 7f382f000640 1 -- 172.21.15.47:0/2051577197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38281028a0 msgr2=0x7f3828073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.154+0000 7f382f000640 1 --2- 172.21.15.47:0/2051577197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38281028a0 0x7f3828073a90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f381000e9f0 tx=0x7f381000edf0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.154+0000 7f382f000640 1 -- 172.21.15.47:0/2051577197 shutdown_connections 2024-03-21T11:26:03.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.154+0000 7f382f000640 1 --2- 172.21.15.47:0/2051577197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f380c0a2fa0 0x7f380c0a5460 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.154+0000 7f382f000640 1 --2- 172.21.15.47:0/2051577197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3828103260 0x7f3828073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.154+0000 7f382f000640 1 --2- 172.21.15.47:0/2051577197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38281028a0 0x7f3828073a90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.154+0000 7f382f000640 1 --2- 172.21.15.47:0/2051577197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38281016b0 0x7f3828075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.154+0000 7f382f000640 1 -- 172.21.15.47:0/2051577197 >> 172.21.15.47:0/2051577197 conn(0x7f38280fcc40 msgr2=0x7f3828105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.155+0000 7f382f000640 1 -- 172.21.15.47:0/2051577197 shutdown_connections 2024-03-21T11:26:03.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.155+0000 7f382f000640 1 -- 172.21.15.47:0/2051577197 wait complete. 2024-03-21T11:26:03.174 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:26:03.174 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:26:03.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.448+0000 7fb74a800640 1 -- 172.21.15.47:0/3296511155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7441016b0 msgr2=0x7fb744101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.448+0000 7fb74a800640 1 --2- 172.21.15.47:0/3296511155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7441016b0 0x7fb744101ad0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb734009900 tx=0x7fb73402bc40 comp rx=0 tx=0).stop 2024-03-21T11:26:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.449+0000 7fb74a800640 1 -- 172.21.15.47:0/3296511155 shutdown_connections 2024-03-21T11:26:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.449+0000 7fb74a800640 1 --2- 172.21.15.47:0/3296511155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb744103260 0x7fb744109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.449+0000 7fb74a800640 1 --2- 172.21.15.47:0/3296511155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7441028a0 0x7fb744102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.449+0000 7fb74a800640 1 --2- 172.21.15.47:0/3296511155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7441016b0 0x7fb744101ad0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.449+0000 7fb74a800640 1 -- 172.21.15.47:0/3296511155 >> 172.21.15.47:0/3296511155 conn(0x7fb7440fcc40 msgr2=0x7fb7440ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.449+0000 7fb74a800640 1 -- 172.21.15.47:0/3296511155 shutdown_connections 2024-03-21T11:26:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.449+0000 7fb74a800640 1 -- 172.21.15.47:0/3296511155 wait complete. 2024-03-21T11:26:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.450+0000 7fb74a800640 1 Processor -- start 2024-03-21T11:26:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.450+0000 7fb74a800640 1 -- start start 2024-03-21T11:26:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb74a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7441016b0 0x7fb744199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb74a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7441028a0 0x7fb74419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb74a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb744103260 0x7fb7441a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb74a800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74410d350 con 0x7fb7441028a0 2024-03-21T11:26:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb74a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74410d1d0 con 0x7fb7441016b0 2024-03-21T11:26:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb74a800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74410d4d0 con 0x7fb744103260 2024-03-21T11:26:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb743400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7441016b0 0x7fb744199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb743400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7441016b0 0x7fb744199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34390/0 (socket says 172.21.15.47:34390) 2024-03-21T11:26:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb743400640 1 -- 172.21.15.47:0/2322799019 learned_addr learned my addr 172.21.15.47:0/2322799019 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb743e00640 1 --2- 172.21.15.47:0/2322799019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb744103260 0x7fb7441a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb73be00640 1 --2- 172.21.15.47:0/2322799019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7441028a0 0x7fb74419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb743400640 1 -- 172.21.15.47:0/2322799019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb744103260 msgr2=0x7fb7441a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb743400640 1 --2- 172.21.15.47:0/2322799019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb744103260 0x7fb7441a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb743400640 1 -- 172.21.15.47:0/2322799019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7441028a0 msgr2=0x7fb74419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb743400640 1 --2- 172.21.15.47:0/2322799019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7441028a0 0x7fb74419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.451+0000 7fb743400640 1 -- 172.21.15.47:0/2322799019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7441a1890 con 0x7fb7441016b0 2024-03-21T11:26:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.452+0000 7fb743400640 1 --2- 172.21.15.47:0/2322799019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7441016b0 0x7fb744199c80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb73402c8f0 tx=0x7fb73402c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.452+0000 7fb740c00640 1 -- 172.21.15.47:0/2322799019 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734041070 con 0x7fb7441016b0 2024-03-21T11:26:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.452+0000 7fb74a800640 1 -- 172.21.15.47:0/2322799019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7441a1b20 con 0x7fb7441016b0 2024-03-21T11:26:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.452+0000 7fb740c00640 1 -- 172.21.15.47:0/2322799019 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb734004450 con 0x7fb7441016b0 2024-03-21T11:26:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.452+0000 7fb740c00640 1 -- 172.21.15.47:0/2322799019 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734034440 con 0x7fb7441016b0 2024-03-21T11:26:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.452+0000 7fb74a800640 1 -- 172.21.15.47:0/2322799019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7441a2010 con 0x7fb7441016b0 2024-03-21T11:26:03.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.457+0000 7fb74d449740 1 Processor -- start 2024-03-21T11:26:03.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.457+0000 7fb740c00640 1 -- 172.21.15.47:0/2322799019 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb734044070 con 0x7fb7441016b0 2024-03-21T11:26:03.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.457+0000 7fb74d449740 1 -- start start 2024-03-21T11:26:03.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb74d449740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab59691810 0x55ab59691c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb74d449740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596929e0 0x55ab59692e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb74d449740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596933a0 0x55ab59699c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb74d449740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab595f62c0 con 0x55ab596933a0 2024-03-21T11:26:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb74d449740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5959ec90 con 0x55ab596929e0 2024-03-21T11:26:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb74d449740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab595f2360 con 0x55ab59691810 2024-03-21T11:26:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb743400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596929e0 0x55ab59692e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb743e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab59691810 0x55ab59691c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb743e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab59691810 0x55ab59691c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34462/0 (socket says 172.21.15.47:34462) 2024-03-21T11:26:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb743e00640 1 -- 172.21.15.47:0/1470478675 learned_addr learned my addr 172.21.15.47:0/1470478675 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb743400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596929e0 0x55ab59692e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34398/0 (socket says 172.21.15.47:34398) 2024-03-21T11:26:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb73be00640 1 --2- 172.21.15.47:0/1470478675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596933a0 0x55ab59699c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb743e00640 1 -- 172.21.15.47:0/1470478675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596929e0 msgr2=0x55ab59692e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb743e00640 1 --2- 172.21.15.47:0/1470478675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596929e0 0x55ab59692e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb743e00640 1 -- 172.21.15.47:0/1470478675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596933a0 msgr2=0x55ab59699c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb743e00640 1 --2- 172.21.15.47:0/1470478675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596933a0 0x55ab59699c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.458+0000 7fb743e00640 1 -- 172.21.15.47:0/1470478675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab595c3080 con 0x55ab59691810 2024-03-21T11:26:03.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.459+0000 7fb740c00640 1 --2- 172.21.15.47:0/2322799019 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb7280a2f20 0x7fb7280a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.459+0000 7fb743e00640 1 --2- 172.21.15.47:0/1470478675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab59691810 0x55ab59691c30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009fd0 tx=0x7fb73c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8a51aa39c5ecc79 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.459+0000 7fb740c00640 1 -- 172.21.15.47:0/2322799019 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb7340f1360 con 0x7fb7441016b0 2024-03-21T11:26:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.459+0000 7fb743400640 1 --2- 172.21.15.47:0/2322799019 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb7280a2f20 0x7fb7280a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.459+0000 7fb738c00640 1 -- 172.21.15.47:0/1470478675 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c004d20 con 0x55ab59691810 2024-03-21T11:26:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.459+0000 7fb738c00640 1 -- 172.21.15.47:0/1470478675 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb73c004050 con 0x55ab59691810 2024-03-21T11:26:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.459+0000 7fb738c00640 1 -- 172.21.15.47:0/1470478675 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c005260 con 0x55ab59691810 2024-03-21T11:26:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.460+0000 7fb74d449740 1 -- 172.21.15.47:0/1470478675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab59691810 msgr2=0x55ab59691c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.460+0000 7fb74d449740 1 --2- 172.21.15.47:0/1470478675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab59691810 0x55ab59691c30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009fd0 tx=0x7fb73c0118d0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.460+0000 7fb743400640 1 --2- 172.21.15.47:0/2322799019 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb7280a2f20 0x7fb7280a53e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb7340b9b20 tx=0x7fb7340f4da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.460+0000 7fb74d449740 1 -- 172.21.15.47:0/1470478675 shutdown_connections 2024-03-21T11:26:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.460+0000 7fb74d449740 1 --2- 172.21.15.47:0/1470478675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596933a0 0x55ab59699c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.460+0000 7fb74d449740 1 --2- 172.21.15.47:0/1470478675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596929e0 0x55ab59692e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.460+0000 7fb74d449740 1 --2- 172.21.15.47:0/1470478675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab59691810 0x55ab59691c30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.460+0000 7fb74d449740 1 -- 172.21.15.47:0/1470478675 >> 172.21.15.47:0/1470478675 conn(0x55ab5968cc00 msgr2=0x55ab5968d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.460+0000 7fb74d449740 1 -- 172.21.15.47:0/1470478675 shutdown_connections 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.460+0000 7fb74d449740 1 -- 172.21.15.47:0/1470478675 wait complete. 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.461+0000 7fb74d449740 1 -- start start 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.461+0000 7fb74d449740 1 Processor -- start 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.461+0000 7fb74d449740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab59691810 0x55ab5969bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.461+0000 7fb74d449740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596929e0 0x55ab5969c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.461+0000 7fb74d449740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596933a0 0x55ab596a3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.461+0000 7fb74d449740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5969a3b0 con 0x55ab596933a0 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.461+0000 7fb74d449740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab595b20d0 con 0x55ab596929e0 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.462+0000 7fb74d449740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5969a530 con 0x55ab59691810 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.462+0000 7fb74d449740 10 client.0 _ll_register_callbacks cb 0x55ab595b6e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.462+0000 7fb73be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596929e0 0x55ab5969c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.462+0000 7fb743e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab59691810 0x55ab5969bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.462+0000 7fb73be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596929e0 0x55ab5969c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34408/0 (socket says 172.21.15.47:34408) 2024-03-21T11:26:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.462+0000 7fb73be00640 1 -- 172.21.15.47:0/3487269418 learned_addr learned my addr 172.21.15.47:0/3487269418 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.462+0000 7fb743e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab59691810 0x55ab5969bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34468/0 (socket says 172.21.15.47:34468) 2024-03-21T11:26:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.462+0000 7fb743e00640 1 --2- 172.21.15.47:0/3487269418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596933a0 0x55ab596a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.462+0000 7fb73be00640 1 -- 172.21.15.47:0/3487269418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab59691810 msgr2=0x55ab5969bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.462+0000 7fb73be00640 1 --2- 172.21.15.47:0/3487269418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab59691810 0x55ab5969bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.462+0000 7fb73be00640 1 -- 172.21.15.47:0/3487269418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596933a0 msgr2=0x55ab596a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.462+0000 7fb73be00640 1 --2- 172.21.15.47:0/3487269418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596933a0 0x55ab596a3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.462+0000 7fb73be00640 1 -- 172.21.15.47:0/3487269418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab5930a3f0 con 0x55ab596929e0 2024-03-21T11:26:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.463+0000 7fb73be00640 1 --2- 172.21.15.47:0/3487269418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596929e0 0x55ab5969c050 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0126d0 tx=0x7fb72c012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd293489c3cfa3b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.463+0000 7fb731600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.463+0000 7fb731600640 1 -- 172.21.15.47:0/3487269418 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c004050 con 0x55ab596929e0 2024-03-21T11:26:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.463+0000 7fb74d449740 1 -- 172.21.15.47:0/3487269418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb73c004050 con 0x55ab596929e0 2024-03-21T11:26:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.463+0000 7fb731600640 1 -- 172.21.15.47:0/3487269418 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb72c0041f0 con 0x55ab596929e0 2024-03-21T11:26:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.463+0000 7fb731600640 1 -- 172.21.15.47:0/3487269418 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb72c025050 con 0x55ab596929e0 2024-03-21T11:26:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.463+0000 7fb74d449740 10 client.26303 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.463+0000 7fb74d449740 1 -- 172.21.15.47:0/3487269418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ab593d47d0 con 0x55ab596929e0 2024-03-21T11:26:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.463+0000 7fb731600640 1 -- 172.21.15.47:0/3487269418 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb73c004050 con 0x55ab596929e0 2024-03-21T11:26:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.464+0000 7fb731600640 1 -- 172.21.15.47:0/3487269418 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb72c003a10 con 0x55ab596929e0 2024-03-21T11:26:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.464+0000 7fb74d449740 10 client.26303 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.464+0000 7fb74d449740 10 client.26303 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:26:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.464+0000 7fb74d449740 20 client.26303 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.464+0000 7fb74d449740 1 --2- 172.21.15.47:0/3487269418 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ab596a7560 0x55ab596a9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.464+0000 7fb74d449740 4 client.26303 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.464+0000 7fb74d449740 1 -- 172.21.15.47:0/3487269418 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab59603800 con 0x55ab596a7560 2024-03-21T11:26:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.464+0000 7fb743e00640 1 --2- 172.21.15.47:0/3487269418 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ab596a7560 0x55ab596a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.465+0000 7fb743e00640 1 --2- 172.21.15.47:0/3487269418 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ab596a7560 0x55ab596a9980 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f36f0f0e32dbcce server_cookie=dacf8ab6f3fd2ed5 in_seq=0 out_seq=0 2024-03-21T11:26:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.465+0000 7fb731600640 10 client.26303 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:26:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb731600640 1 -- 172.21.15.47:0/3487269418 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ab59603800 con 0x55ab596a7560 2024-03-21T11:26:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb731600640 10 client.26303 handle_command_reply: tid=0 2024-03-21T11:26:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb74d449740 1 client.26303 shutdown 2024-03-21T11:26:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb74d449740 1 -- 172.21.15.47:0/3487269418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596929e0 msgr2=0x55ab5969c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb74d449740 1 --2- 172.21.15.47:0/3487269418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596929e0 0x55ab5969c050 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0126d0 tx=0x7fb72c012b90 comp rx=0 tx=0).stop 2024-03-21T11:26:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb74d449740 1 -- 172.21.15.47:0/3487269418 shutdown_connections 2024-03-21T11:26:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb74d449740 1 --2- 172.21.15.47:0/3487269418 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ab596a7560 0x55ab596a9980 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb74d449740 1 --2- 172.21.15.47:0/3487269418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596933a0 0x55ab596a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb74d449740 1 --2- 172.21.15.47:0/3487269418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596929e0 0x55ab5969c050 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb731600640 0 client.26303 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:26:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb74d449740 1 --2- 172.21.15.47:0/3487269418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab59691810 0x55ab5969bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb74d449740 1 -- 172.21.15.47:0/3487269418 >> 172.21.15.47:0/3487269418 conn(0x55ab5968cc00 msgr2=0x55ab5968beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb74d449740 1 -- 172.21.15.47:0/3487269418 shutdown_connections 2024-03-21T11:26:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb74d449740 1 -- 172.21.15.47:0/3487269418 wait complete. 2024-03-21T11:26:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.466+0000 7fb74d449740 20 client.26303 trim_cache size 0 max 16384 2024-03-21T11:26:03.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.480+0000 7fb74d449740 1 Processor -- start 2024-03-21T11:26:03.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb74d449740 1 -- start start 2024-03-21T11:26:03.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb74d449740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab59691810 0x55ab5968b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb74d449740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab596929e0 0x55ab596a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb74d449740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596933a0 0x55ab5968bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb74d449740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7441ac3c0 con 0x55ab59691810 2024-03-21T11:26:03.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb74d449740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7441add40 con 0x55ab596933a0 2024-03-21T11:26:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb74d449740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7441aca20 con 0x55ab596929e0 2024-03-21T11:26:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb73be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab596929e0 0x55ab596a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb743e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab59691810 0x55ab5968b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb73be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab596929e0 0x55ab596a2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34470/0 (socket says 172.21.15.47:34470) 2024-03-21T11:26:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb73be00640 1 -- 172.21.15.47:0/3463295807 learned_addr learned my addr 172.21.15.47:0/3463295807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb743e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab59691810 0x55ab5968b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53922/0 (socket says 172.21.15.47:53922) 2024-03-21T11:26:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb743e00640 1 --2- 172.21.15.47:0/3463295807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596933a0 0x55ab5968bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb73be00640 1 -- 172.21.15.47:0/3463295807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596933a0 msgr2=0x55ab5968bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb73be00640 1 --2- 172.21.15.47:0/3463295807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596933a0 0x55ab5968bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb73be00640 1 -- 172.21.15.47:0/3463295807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab59691810 msgr2=0x55ab5968b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb73be00640 1 --2- 172.21.15.47:0/3463295807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab59691810 0x55ab5968b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.481+0000 7fb73be00640 1 -- 172.21.15.47:0/3463295807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab5968d650 con 0x55ab596929e0 2024-03-21T11:26:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.482+0000 7fb73be00640 1 --2- 172.21.15.47:0/3463295807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab596929e0 0x55ab596a2fe0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0058c0 tx=0x7fb72c010710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86f560a993670379 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.482+0000 7fb733e00640 1 -- 172.21.15.47:0/3463295807 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c00f920 con 0x55ab596929e0 2024-03-21T11:26:03.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.482+0000 7fb733e00640 1 -- 172.21.15.47:0/3463295807 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb72c02e070 con 0x55ab596929e0 2024-03-21T11:26:03.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.482+0000 7fb733e00640 1 -- 172.21.15.47:0/3463295807 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c0035f0 con 0x55ab596929e0 2024-03-21T11:26:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.483+0000 7fb74d449740 1 -- 172.21.15.47:0/3463295807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab596929e0 msgr2=0x55ab596a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.483+0000 7fb74d449740 1 --2- 172.21.15.47:0/3463295807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab596929e0 0x55ab596a2fe0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0058c0 tx=0x7fb72c010710 comp rx=0 tx=0).stop 2024-03-21T11:26:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.483+0000 7fb74d449740 1 -- 172.21.15.47:0/3463295807 shutdown_connections 2024-03-21T11:26:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.483+0000 7fb74d449740 1 --2- 172.21.15.47:0/3463295807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab596933a0 0x55ab5968bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.483+0000 7fb74d449740 1 --2- 172.21.15.47:0/3463295807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab596929e0 0x55ab596a2fe0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.483+0000 7fb74d449740 1 --2- 172.21.15.47:0/3463295807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab59691810 0x55ab5968b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.483+0000 7fb74d449740 1 -- 172.21.15.47:0/3463295807 >> 172.21.15.47:0/3463295807 conn(0x55ab5968cc00 msgr2=0x55ab5969bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.483+0000 7fb74d449740 1 -- 172.21.15.47:0/3463295807 shutdown_connections 2024-03-21T11:26:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.483+0000 7fb74d449740 1 -- 172.21.15.47:0/3463295807 wait complete. 2024-03-21T11:26:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.483+0000 7fb74d449740 1 -- start start 2024-03-21T11:26:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.484+0000 7fb74d449740 1 Processor -- start 2024-03-21T11:26:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.484+0000 7fb74d449740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab59691810 0x55ab5969e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.484+0000 7fb74d449740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596929e0 0x55ab5969ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.484+0000 7fb74d449740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab596933a0 0x55ab596a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.484+0000 7fb74d449740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7441ad6e0 con 0x55ab596929e0 2024-03-21T11:26:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.484+0000 7fb74d449740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7441ad080 con 0x55ab59691810 2024-03-21T11:26:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.484+0000 7fb74d449740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7441aa580 con 0x55ab596933a0 2024-03-21T11:26:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.484+0000 7fb74d449740 10 client.0 _ll_register_callbacks cb 0x7fb72c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.484+0000 7fb743e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab59691810 0x55ab5969e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.484+0000 7fb743e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab59691810 0x55ab5969e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34428/0 (socket says 172.21.15.47:34428) 2024-03-21T11:26:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.484+0000 7fb743e00640 1 -- 172.21.15.47:0/177213194 learned_addr learned my addr 172.21.15.47:0/177213194 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.484+0000 7fb743e00640 1 --2- 172.21.15.47:0/177213194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab596933a0 0x55ab596a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.484+0000 7fb73be00640 1 --2- 172.21.15.47:0/177213194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596929e0 0x55ab5969ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.485+0000 7fb743e00640 1 -- 172.21.15.47:0/177213194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab596933a0 msgr2=0x55ab596a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.485+0000 7fb743e00640 1 --2- 172.21.15.47:0/177213194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab596933a0 0x55ab596a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.485+0000 7fb743e00640 1 -- 172.21.15.47:0/177213194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596929e0 msgr2=0x55ab5969ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.485+0000 7fb743e00640 1 --2- 172.21.15.47:0/177213194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596929e0 0x55ab5969ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.485+0000 7fb743e00640 1 -- 172.21.15.47:0/177213194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab596a21f0 con 0x55ab59691810 2024-03-21T11:26:03.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.485+0000 7fb743e00640 1 --2- 172.21.15.47:0/177213194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab59691810 0x55ab5969e6a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb73c000f80 tx=0x7fb73c032040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17c4679b5fcadf7b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.485+0000 7fb731600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:03.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.485+0000 7fb731600640 1 -- 172.21.15.47:0/177213194 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c004090 con 0x55ab59691810 2024-03-21T11:26:03.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.485+0000 7fb731600640 1 -- 172.21.15.47:0/177213194 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb73c037040 con 0x55ab59691810 2024-03-21T11:26:03.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.485+0000 7fb74d449740 1 -- 172.21.15.47:0/177213194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb72c02e070 con 0x55ab59691810 2024-03-21T11:26:03.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.486+0000 7fb731600640 1 -- 172.21.15.47:0/177213194 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb73c042050 con 0x55ab59691810 2024-03-21T11:26:03.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.486+0000 7fb74d449740 10 client.26309 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:03.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.486+0000 7fb74d449740 1 -- 172.21.15.47:0/177213194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ab59702e10 con 0x55ab59691810 2024-03-21T11:26:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.486+0000 7fb731600640 1 -- 172.21.15.47:0/177213194 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb72c02e070 con 0x55ab59691810 2024-03-21T11:26:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.486+0000 7fb731600640 1 -- 172.21.15.47:0/177213194 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb73c025040 con 0x55ab59691810 2024-03-21T11:26:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.486+0000 7fb74d449740 10 client.26309 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.486+0000 7fb74d449740 10 client.26309 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:26:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.486+0000 7fb74d449740 20 client.26309 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.486+0000 7fb74d449740 1 --2- 172.21.15.47:0/177213194 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ab597034b0 0x55ab59705900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.486+0000 7fb74d449740 4 client.26309 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:26:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.486+0000 7fb74d449740 1 -- 172.21.15.47:0/177213194 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ab595fdb50 con 0x55ab597034b0 2024-03-21T11:26:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.487+0000 7fb73be00640 1 --2- 172.21.15.47:0/177213194 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ab597034b0 0x55ab59705900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.487+0000 7fb73be00640 1 --2- 172.21.15.47:0/177213194 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ab597034b0 0x55ab59705900 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1a1c0741e40c785 server_cookie=7cdc8d98f438018c in_seq=0 out_seq=0 2024-03-21T11:26:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.487+0000 7fb731600640 10 client.26309 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:26:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.487+0000 7fb731600640 1 -- 172.21.15.47:0/177213194 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ab595fdb50 con 0x55ab597034b0 2024-03-21T11:26:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.487+0000 7fb731600640 10 client.26309 handle_command_reply: tid=0 2024-03-21T11:26:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.488+0000 7fb74d449740 1 client.26309 shutdown 2024-03-21T11:26:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.488+0000 7fb74d449740 1 -- 172.21.15.47:0/177213194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab59691810 msgr2=0x55ab5969e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.488+0000 7fb74d449740 1 --2- 172.21.15.47:0/177213194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab59691810 0x55ab5969e6a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb73c000f80 tx=0x7fb73c032040 comp rx=0 tx=0).stop 2024-03-21T11:26:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.488+0000 7fb74d449740 1 -- 172.21.15.47:0/177213194 shutdown_connections 2024-03-21T11:26:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.488+0000 7fb74d449740 1 --2- 172.21.15.47:0/177213194 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ab597034b0 0x55ab59705900 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.488+0000 7fb74d449740 1 --2- 172.21.15.47:0/177213194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab596933a0 0x55ab596a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.488+0000 7fb731600640 0 client.26309 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:26:03.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.488+0000 7fb74d449740 1 --2- 172.21.15.47:0/177213194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab596929e0 0x55ab5969ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.488+0000 7fb74d449740 1 --2- 172.21.15.47:0/177213194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab59691810 0x55ab5969e6a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.488+0000 7fb74d449740 1 -- 172.21.15.47:0/177213194 >> 172.21.15.47:0/177213194 conn(0x55ab5968cc00 msgr2=0x55ab5968fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.488+0000 7fb74d449740 1 -- 172.21.15.47:0/177213194 shutdown_connections 2024-03-21T11:26:03.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.488+0000 7fb74d449740 1 -- 172.21.15.47:0/177213194 wait complete. 2024-03-21T11:26:03.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.488+0000 7fb74d449740 20 client.26309 trim_cache size 0 max 16384 2024-03-21T11:26:03.493 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:26:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.489+0000 7fb74a800640 1 -- 172.21.15.47:0/2322799019 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb7280a2f20 msgr2=0x7fb7280a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.489+0000 7fb74a800640 1 --2- 172.21.15.47:0/2322799019 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb7280a2f20 0x7fb7280a53e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb7340b9b20 tx=0x7fb7340f4da0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.489+0000 7fb74a800640 1 -- 172.21.15.47:0/2322799019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7441016b0 msgr2=0x7fb744199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.489+0000 7fb74a800640 1 --2- 172.21.15.47:0/2322799019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7441016b0 0x7fb744199c80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb73402c8f0 tx=0x7fb73402c660 comp rx=0 tx=0).stop 2024-03-21T11:26:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.489+0000 7fb74a800640 1 -- 172.21.15.47:0/2322799019 shutdown_connections 2024-03-21T11:26:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.489+0000 7fb74a800640 1 --2- 172.21.15.47:0/2322799019 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb7280a2f20 0x7fb7280a53e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.489+0000 7fb74a800640 1 --2- 172.21.15.47:0/2322799019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb744103260 0x7fb7441a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.489+0000 7fb74a800640 1 --2- 172.21.15.47:0/2322799019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7441028a0 0x7fb74419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.489+0000 7fb74a800640 1 --2- 172.21.15.47:0/2322799019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7441016b0 0x7fb744199c80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.489+0000 7fb74a800640 1 -- 172.21.15.47:0/2322799019 >> 172.21.15.47:0/2322799019 conn(0x7fb7440fcc40 msgr2=0x7fb7440738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.489+0000 7fb74a800640 1 -- 172.21.15.47:0/2322799019 shutdown_connections 2024-03-21T11:26:03.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.489+0000 7fb74a800640 1 -- 172.21.15.47:0/2322799019 wait complete. 2024-03-21T11:26:03.495 INFO:tasks.workunit.client.0.smithi047.stdout: 1 78482 9.40 MB/sec warmup 47 sec latency 88.003 ms 2024-03-21T11:26:03.508 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:26:03.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:26:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.710+0000 7f26da200640 1 -- 172.21.15.47:0/3151457286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d4104a50 msgr2=0x7f26d4106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.710+0000 7f26da200640 1 --2- 172.21.15.47:0/3151457286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d4104a50 0x7f26d4106e40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009900 tx=0x7f26bc02bc40 comp rx=0 tx=0).stop 2024-03-21T11:26:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.710+0000 7f26da200640 1 -- 172.21.15.47:0/3151457286 shutdown_connections 2024-03-21T11:26:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.710+0000 7f26da200640 1 --2- 172.21.15.47:0/3151457286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107380 0x7f26d41097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.710+0000 7f26da200640 1 --2- 172.21.15.47:0/3151457286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d4104a50 0x7f26d4106e40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.710+0000 7f26da200640 1 --2- 172.21.15.47:0/3151457286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d4102120 0x7f26d4104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.710+0000 7f26da200640 1 -- 172.21.15.47:0/3151457286 >> 172.21.15.47:0/3151457286 conn(0x7f26d40fbb10 msgr2=0x7f26d40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.710+0000 7f26da200640 1 -- 172.21.15.47:0/3151457286 shutdown_connections 2024-03-21T11:26:03.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.710+0000 7f26da200640 1 -- 172.21.15.47:0/3151457286 wait complete. 2024-03-21T11:26:03.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.712+0000 7f26da200640 1 Processor -- start 2024-03-21T11:26:03.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.712+0000 7f26da200640 1 -- start start 2024-03-21T11:26:03.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.712+0000 7f26da200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d4102120 0x7f26d4199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.712+0000 7f26da200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d4104a50 0x7f26d419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.712+0000 7f26da200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107380 0x7f26d41a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.712+0000 7f26da200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d410d360 con 0x7f26d4107380 2024-03-21T11:26:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.712+0000 7f26da200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d410d1e0 con 0x7f26d4104a50 2024-03-21T11:26:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.712+0000 7f26da200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d410d4e0 con 0x7f26d4102120 2024-03-21T11:26:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.712+0000 7f26cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d4104a50 0x7f26d419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.712+0000 7f26cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d4104a50 0x7f26d419a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34446/0 (socket says 172.21.15.47:34446) 2024-03-21T11:26:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.712+0000 7f26cbe00640 1 -- 172.21.15.47:0/1514188608 learned_addr learned my addr 172.21.15.47:0/1514188608 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.712+0000 7f26d3400640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d4102120 0x7f26d4199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.712+0000 7f26d3e00640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107380 0x7f26d41a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.713+0000 7f26cbe00640 1 -- 172.21.15.47:0/1514188608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d4102120 msgr2=0x7f26d4199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.713+0000 7f26cbe00640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d4102120 0x7f26d4199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.713+0000 7f26cbe00640 1 -- 172.21.15.47:0/1514188608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107380 msgr2=0x7f26d41a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.713+0000 7f26cbe00640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107380 0x7f26d41a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.713+0000 7f26cbe00640 1 -- 172.21.15.47:0/1514188608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d41a1880 con 0x7f26d4104a50 2024-03-21T11:26:03.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.713+0000 7f26d3400640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d4102120 0x7f26d4199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:03.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.713+0000 7f26d3e00640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107380 0x7f26d41a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:03.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.713+0000 7f26cbe00640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d4104a50 0x7f26d419a1b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f26bc02efd0 tx=0x7f26bc02c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.713+0000 7f26d0c00640 1 -- 172.21.15.47:0/1514188608 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc041070 con 0x7f26d4104a50 2024-03-21T11:26:03.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.714+0000 7f26d0c00640 1 -- 172.21.15.47:0/1514188608 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26bc004450 con 0x7f26d4104a50 2024-03-21T11:26:03.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.714+0000 7f26d0c00640 1 -- 172.21.15.47:0/1514188608 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc003e50 con 0x7f26d4104a50 2024-03-21T11:26:03.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.714+0000 7f26da200640 1 -- 172.21.15.47:0/1514188608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d41a1b10 con 0x7f26d4104a50 2024-03-21T11:26:03.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.714+0000 7f26da200640 1 -- 172.21.15.47:0/1514188608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d41a1f80 con 0x7f26d4104a50 2024-03-21T11:26:03.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.718+0000 7f26dce77740 1 Processor -- start 2024-03-21T11:26:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.718+0000 7f26dce77740 1 -- start start 2024-03-21T11:26:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.718+0000 7f26d0c00640 1 -- 172.21.15.47:0/1514188608 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f26bc044070 con 0x7f26d4104a50 2024-03-21T11:26:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.718+0000 7f26dce77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc201810 0x55a2bc201c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.718+0000 7f26dce77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc202e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.718+0000 7f26dce77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc2033a0 0x55a2bc209c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.718+0000 7f26dce77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2bc1662c0 con 0x55a2bc2029e0 2024-03-21T11:26:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.718+0000 7f26dce77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2bc10ec90 con 0x55a2bc2033a0 2024-03-21T11:26:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.718+0000 7f26dce77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2bc162360 con 0x55a2bc201810 2024-03-21T11:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.718+0000 7f26d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc201810 0x55a2bc201c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.718+0000 7f26d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc201810 0x55a2bc201c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34516/0 (socket says 172.21.15.47:34516) 2024-03-21T11:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.718+0000 7f26d3e00640 1 -- 172.21.15.47:0/3810277569 learned_addr learned my addr 172.21.15.47:0/3810277569 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.718+0000 7f26cbe00640 1 --2- 172.21.15.47:0/3810277569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc2033a0 0x55a2bc209c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.718+0000 7f26d3400640 1 --2- 172.21.15.47:0/3810277569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc202e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.719+0000 7f26d3e00640 1 -- 172.21.15.47:0/3810277569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc2033a0 msgr2=0x55a2bc209c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.719+0000 7f26d3e00640 1 --2- 172.21.15.47:0/3810277569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc2033a0 0x55a2bc209c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.719+0000 7f26d3e00640 1 -- 172.21.15.47:0/3810277569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 msgr2=0x55a2bc202e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.719+0000 7f26d3e00640 1 --2- 172.21.15.47:0/3810277569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc202e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.719+0000 7f26d3e00640 1 -- 172.21.15.47:0/3810277569 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2bc133080 con 0x55a2bc201810 2024-03-21T11:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.719+0000 7f26d3400640 1 --2- 172.21.15.47:0/3810277569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc202e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:03.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.719+0000 7f26d3e00640 1 --2- 172.21.15.47:0/3810277569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc201810 0x55a2bc201c30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009fd0 tx=0x7f26cc0108e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5541710b21c4ac3a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.719+0000 7f26c3e00640 1 -- 172.21.15.47:0/3810277569 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc011760 con 0x55a2bc201810 2024-03-21T11:26:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.719+0000 7f26c3e00640 1 -- 172.21.15.47:0/3810277569 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26cc011de0 con 0x55a2bc201810 2024-03-21T11:26:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.719+0000 7f26c3e00640 1 -- 172.21.15.47:0/3810277569 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00e3c0 con 0x55a2bc201810 2024-03-21T11:26:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.720+0000 7f26d0c00640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26b80a3140 0x7f26b80a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.720+0000 7f26d0c00640 1 -- 172.21.15.47:0/1514188608 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f26bc0f13e0 con 0x7f26d4104a50 2024-03-21T11:26:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.720+0000 7f26d3400640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26b80a3140 0x7f26b80a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.720+0000 7f26dce77740 1 -- 172.21.15.47:0/3810277569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc201810 msgr2=0x55a2bc201c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.720+0000 7f26dce77740 1 --2- 172.21.15.47:0/3810277569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc201810 0x55a2bc201c30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009fd0 tx=0x7f26cc0108e0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.721+0000 7f26dce77740 1 -- 172.21.15.47:0/3810277569 shutdown_connections 2024-03-21T11:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.721+0000 7f26dce77740 1 --2- 172.21.15.47:0/3810277569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc2033a0 0x55a2bc209c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.721+0000 7f26dce77740 1 --2- 172.21.15.47:0/3810277569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc202e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.721+0000 7f26dce77740 1 --2- 172.21.15.47:0/3810277569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc201810 0x55a2bc201c30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.721+0000 7f26dce77740 1 -- 172.21.15.47:0/3810277569 >> 172.21.15.47:0/3810277569 conn(0x55a2bc1fcc00 msgr2=0x55a2bc1fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.721+0000 7f26dce77740 1 -- 172.21.15.47:0/3810277569 shutdown_connections 2024-03-21T11:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.721+0000 7f26dce77740 1 -- 172.21.15.47:0/3810277569 wait complete. 2024-03-21T11:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.721+0000 7f26d3400640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26b80a3140 0x7f26b80a5600 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f26d419b140 tx=0x7f26c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.721+0000 7f26dce77740 1 -- start start 2024-03-21T11:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.722+0000 7f26dce77740 1 Processor -- start 2024-03-21T11:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.722+0000 7f26dce77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc201810 0x55a2bc20baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.722+0000 7f26dce77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc20c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.722+0000 7f26dce77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc2033a0 0x55a2bc213000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.722+0000 7f26dce77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2bc20a3b0 con 0x55a2bc2029e0 2024-03-21T11:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.722+0000 7f26dce77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2bc1220d0 con 0x55a2bc201810 2024-03-21T11:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.722+0000 7f26dce77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2bc20a530 con 0x55a2bc2033a0 2024-03-21T11:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.722+0000 7f26dce77740 10 client.0 _ll_register_callbacks cb 0x55a2bc126e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.722+0000 7f26d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc201810 0x55a2bc20baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.722+0000 7f26cbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc20c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.722+0000 7f26d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc201810 0x55a2bc20baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34470/0 (socket says 172.21.15.47:34470) 2024-03-21T11:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.722+0000 7f26d3400640 1 -- 172.21.15.47:0/1332942523 learned_addr learned my addr 172.21.15.47:0/1332942523 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.722+0000 7f26d3e00640 1 --2- 172.21.15.47:0/1332942523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc2033a0 0x55a2bc213000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.723+0000 7f26d3400640 1 -- 172.21.15.47:0/1332942523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc2033a0 msgr2=0x55a2bc213000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.723+0000 7f26d3400640 1 --2- 172.21.15.47:0/1332942523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc2033a0 0x55a2bc213000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.723+0000 7f26d3400640 1 -- 172.21.15.47:0/1332942523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 msgr2=0x55a2bc20c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.723+0000 7f26d3400640 1 --2- 172.21.15.47:0/1332942523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc20c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.723+0000 7f26d3400640 1 -- 172.21.15.47:0/1332942523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2bbe7a3f0 con 0x55a2bc201810 2024-03-21T11:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.723+0000 7f26d3400640 1 --2- 172.21.15.47:0/1332942523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc201810 0x55a2bc20baf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009c20 tx=0x7f26c4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a063960f529f7db server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.723+0000 7f26c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.723+0000 7f26c0c00640 1 -- 172.21.15.47:0/1332942523 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c4005020 con 0x55a2bc201810 2024-03-21T11:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.723+0000 7f26c0c00640 1 -- 172.21.15.47:0/1332942523 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f26c4018030 con 0x55a2bc201810 2024-03-21T11:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.723+0000 7f26dce77740 1 -- 172.21.15.47:0/1332942523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26cc011de0 con 0x55a2bc201810 2024-03-21T11:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.724+0000 7f26c0c00640 1 -- 172.21.15.47:0/1332942523 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26c4034050 con 0x55a2bc201810 2024-03-21T11:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.724+0000 7f26dce77740 10 client.26321 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.724+0000 7f26dce77740 1 -- 172.21.15.47:0/1332942523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a2bbf447d0 con 0x55a2bc201810 2024-03-21T11:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.724+0000 7f26c0c00640 1 -- 172.21.15.47:0/1332942523 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26cc011de0 con 0x55a2bc201810 2024-03-21T11:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.724+0000 7f26c0c00640 1 -- 172.21.15.47:0/1332942523 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f26c401e620 con 0x55a2bc201810 2024-03-21T11:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.724+0000 7f26dce77740 10 client.26321 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.724+0000 7f26dce77740 10 client.26321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.724+0000 7f26dce77740 20 client.26321 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.724+0000 7f26dce77740 1 --2- 172.21.15.47:0/1332942523 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a2bc217540 0x55a2bc2199b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.724+0000 7f26dce77740 4 client.26321 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:03.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.724+0000 7f26dce77740 1 -- 172.21.15.47:0/1332942523 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2bc173800 con 0x55a2bc217540 2024-03-21T11:26:03.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.725+0000 7f26cbe00640 1 --2- 172.21.15.47:0/1332942523 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a2bc217540 0x55a2bc2199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.725+0000 7f26cbe00640 1 --2- 172.21.15.47:0/1332942523 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a2bc217540 0x55a2bc2199b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31a1e2b0afcc8eba server_cookie=cec27071c3c1f9e1 in_seq=0 out_seq=0 2024-03-21T11:26:03.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.725+0000 7f26c0c00640 10 client.26321 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:03.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.729+0000 7f26c0c00640 1 -- 172.21.15.47:0/1332942523 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a2bc173800 con 0x55a2bc217540 2024-03-21T11:26:03.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.729+0000 7f26c0c00640 10 client.26321 handle_command_reply: tid=0 2024-03-21T11:26:03.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.729+0000 7f26dce77740 1 client.26321 shutdown 2024-03-21T11:26:03.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.729+0000 7f26dce77740 1 -- 172.21.15.47:0/1332942523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc201810 msgr2=0x55a2bc20baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.729+0000 7f26dce77740 1 --2- 172.21.15.47:0/1332942523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc201810 0x55a2bc20baf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009c20 tx=0x7f26c4004730 comp rx=0 tx=0).stop 2024-03-21T11:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.729+0000 7f26dce77740 1 -- 172.21.15.47:0/1332942523 shutdown_connections 2024-03-21T11:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.729+0000 7f26dce77740 1 --2- 172.21.15.47:0/1332942523 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a2bc217540 0x55a2bc2199b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.730+0000 7f26dce77740 1 --2- 172.21.15.47:0/1332942523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc2033a0 0x55a2bc213000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.730+0000 7f26dce77740 1 --2- 172.21.15.47:0/1332942523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc20c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.730+0000 7f26dce77740 1 --2- 172.21.15.47:0/1332942523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc201810 0x55a2bc20baf0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.730+0000 7f26c0c00640 0 client.26321 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.730+0000 7f26dce77740 1 -- 172.21.15.47:0/1332942523 >> 172.21.15.47:0/1332942523 conn(0x55a2bc1fcc00 msgr2=0x55a2bc1fbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.730+0000 7f26dce77740 1 -- 172.21.15.47:0/1332942523 shutdown_connections 2024-03-21T11:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.730+0000 7f26dce77740 1 -- 172.21.15.47:0/1332942523 wait complete. 2024-03-21T11:26:03.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.730+0000 7f26dce77740 20 client.26321 trim_cache size 0 max 16384 2024-03-21T11:26:03.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.745+0000 7f26dce77740 1 Processor -- start 2024-03-21T11:26:03.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.745+0000 7f26dce77740 1 -- start start 2024-03-21T11:26:03.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26dce77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc201810 0x55a2bc205a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26dce77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc212fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26dce77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc2033a0 0x55a2bc205fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26dce77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d41ac330 con 0x55a2bc2029e0 2024-03-21T11:26:03.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26dce77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d41adcb0 con 0x55a2bc201810 2024-03-21T11:26:03.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26dce77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d41ac990 con 0x55a2bc2033a0 2024-03-21T11:26:03.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc201810 0x55a2bc205a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc2033a0 0x55a2bc205fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc201810 0x55a2bc205a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34484/0 (socket says 172.21.15.47:34484) 2024-03-21T11:26:03.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26d3400640 1 -- 172.21.15.47:0/2153544428 learned_addr learned my addr 172.21.15.47:0/2153544428 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26cbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc212fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26d3400640 1 -- 172.21.15.47:0/2153544428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc2033a0 msgr2=0x55a2bc205fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26d3400640 1 --2- 172.21.15.47:0/2153544428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc2033a0 0x55a2bc205fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26d3400640 1 -- 172.21.15.47:0/2153544428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 msgr2=0x55a2bc212fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26d3400640 1 --2- 172.21.15.47:0/2153544428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc212fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.746+0000 7f26d3400640 1 -- 172.21.15.47:0/2153544428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2bc1fd970 con 0x55a2bc201810 2024-03-21T11:26:03.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.747+0000 7f26d3400640 1 --2- 172.21.15.47:0/2153544428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc201810 0x55a2bc205a80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009f80 tx=0x7f26c400fee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ebac091b2882a71 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.747+0000 7f26c3400640 1 -- 172.21.15.47:0/2153544428 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c403d070 con 0x55a2bc201810 2024-03-21T11:26:03.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.747+0000 7f26c3400640 1 -- 172.21.15.47:0/2153544428 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26c401edc0 con 0x55a2bc201810 2024-03-21T11:26:03.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.747+0000 7f26c3400640 1 -- 172.21.15.47:0/2153544428 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c4029c30 con 0x55a2bc201810 2024-03-21T11:26:03.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.748+0000 7f26dce77740 1 -- 172.21.15.47:0/2153544428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc201810 msgr2=0x55a2bc205a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.748+0000 7f26dce77740 1 --2- 172.21.15.47:0/2153544428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc201810 0x55a2bc205a80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009f80 tx=0x7f26c400fee0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.748+0000 7f26dce77740 1 -- 172.21.15.47:0/2153544428 shutdown_connections 2024-03-21T11:26:03.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.748+0000 7f26dce77740 1 --2- 172.21.15.47:0/2153544428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc2033a0 0x55a2bc205fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.748+0000 7f26dce77740 1 --2- 172.21.15.47:0/2153544428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc212fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.748+0000 7f26dce77740 1 --2- 172.21.15.47:0/2153544428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc201810 0x55a2bc205a80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.748+0000 7f26dce77740 1 -- 172.21.15.47:0/2153544428 >> 172.21.15.47:0/2153544428 conn(0x55a2bc1fcc00 msgr2=0x55a2bc2006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.748+0000 7f26dce77740 1 -- 172.21.15.47:0/2153544428 shutdown_connections 2024-03-21T11:26:03.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.748+0000 7f26dce77740 1 -- 172.21.15.47:0/2153544428 wait complete. 2024-03-21T11:26:03.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.748+0000 7f26dce77740 1 -- start start 2024-03-21T11:26:03.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.749+0000 7f26dce77740 1 Processor -- start 2024-03-21T11:26:03.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.749+0000 7f26dce77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc201810 0x55a2bc20f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.749+0000 7f26dce77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc20f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.749+0000 7f26dce77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc2033a0 0x55a2bc2126e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.749+0000 7f26dce77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d41ad650 con 0x55a2bc2029e0 2024-03-21T11:26:03.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.749+0000 7f26dce77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d41acff0 con 0x55a2bc2033a0 2024-03-21T11:26:03.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.749+0000 7f26dce77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d41aa4f0 con 0x55a2bc201810 2024-03-21T11:26:03.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.749+0000 7f26dce77740 10 client.0 _ll_register_callbacks cb 0x7f26c4009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:03.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.749+0000 7f26d3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc201810 0x55a2bc20f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.749+0000 7f26d3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc201810 0x55a2bc20f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34538/0 (socket says 172.21.15.47:34538) 2024-03-21T11:26:03.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.749+0000 7f26d3400640 1 -- 172.21.15.47:0/147317369 learned_addr learned my addr 172.21.15.47:0/147317369 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:03.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.749+0000 7f26d3e00640 1 --2- 172.21.15.47:0/147317369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc2033a0 0x55a2bc2126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.749+0000 7f26cbe00640 1 --2- 172.21.15.47:0/147317369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc20f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.750+0000 7f26d3400640 1 -- 172.21.15.47:0/147317369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc2033a0 msgr2=0x55a2bc2126e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.750+0000 7f26d3400640 1 --2- 172.21.15.47:0/147317369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc2033a0 0x55a2bc2126e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.750+0000 7f26d3400640 1 -- 172.21.15.47:0/147317369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 msgr2=0x55a2bc20f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:03.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.750+0000 7f26d3400640 1 --2- 172.21.15.47:0/147317369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc20f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.750+0000 7f26d3400640 1 -- 172.21.15.47:0/147317369 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2bc212cd0 con 0x55a2bc201810 2024-03-21T11:26:03.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.750+0000 7f26d3400640 1 --2- 172.21.15.47:0/147317369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc201810 0x55a2bc20f1f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f26c4012f70 tx=0x7f26c401e8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e71f8c7a74050a1f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:03.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.750+0000 7f26c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.751+0000 7f26c0c00640 1 -- 172.21.15.47:0/147317369 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c403d070 con 0x55a2bc201810 2024-03-21T11:26:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.751+0000 7f26c0c00640 1 -- 172.21.15.47:0/147317369 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f26c40296f0 con 0x55a2bc201810 2024-03-21T11:26:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.751+0000 7f26dce77740 1 -- 172.21.15.47:0/147317369 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26c401edc0 con 0x55a2bc201810 2024-03-21T11:26:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.751+0000 7f26c0c00640 1 -- 172.21.15.47:0/147317369 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26c4029970 con 0x55a2bc201810 2024-03-21T11:26:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.751+0000 7f26dce77740 10 client.26344 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.751+0000 7f26dce77740 1 -- 172.21.15.47:0/147317369 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a2bc273860 con 0x55a2bc201810 2024-03-21T11:26:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.751+0000 7f26c0c00640 1 -- 172.21.15.47:0/147317369 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26c401edc0 con 0x55a2bc201810 2024-03-21T11:26:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.752+0000 7f26c0c00640 1 -- 172.21.15.47:0/147317369 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f26c4004870 con 0x55a2bc201810 2024-03-21T11:26:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.752+0000 7f26dce77740 10 client.26344 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.752+0000 7f26dce77740 10 client.26344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.752+0000 7f26dce77740 20 client.26344 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:03.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.752+0000 7f26dce77740 1 --2- 172.21.15.47:0/147317369 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a2bc274070 0x55a2bc276490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:03.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.752+0000 7f26dce77740 4 client.26344 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:26:03.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.752+0000 7f26dce77740 1 -- 172.21.15.47:0/147317369 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a2bc16db50 con 0x55a2bc274070 2024-03-21T11:26:03.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.752+0000 7f26cbe00640 1 --2- 172.21.15.47:0/147317369 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a2bc274070 0x55a2bc276490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:03.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.753+0000 7f26cbe00640 1 --2- 172.21.15.47:0/147317369 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a2bc274070 0x55a2bc276490 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e81b92533402379 server_cookie=2bf7ec4693931047 in_seq=0 out_seq=0 2024-03-21T11:26:03.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.753+0000 7f26c0c00640 10 client.26344 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:03.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.753+0000 7f26c0c00640 1 -- 172.21.15.47:0/147317369 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a2bc16db50 con 0x55a2bc274070 2024-03-21T11:26:03.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.753+0000 7f26c0c00640 10 client.26344 handle_command_reply: tid=0 2024-03-21T11:26:03.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.753+0000 7f26dce77740 1 client.26344 shutdown 2024-03-21T11:26:03.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.753+0000 7f26dce77740 1 -- 172.21.15.47:0/147317369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc201810 msgr2=0x55a2bc20f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.753+0000 7f26dce77740 1 --2- 172.21.15.47:0/147317369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc201810 0x55a2bc20f1f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f26c4012f70 tx=0x7f26c401e8d0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.753+0000 7f26dce77740 1 -- 172.21.15.47:0/147317369 shutdown_connections 2024-03-21T11:26:03.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.753+0000 7f26dce77740 1 --2- 172.21.15.47:0/147317369 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a2bc274070 0x55a2bc276490 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.753+0000 7f26dce77740 1 --2- 172.21.15.47:0/147317369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bc2033a0 0x55a2bc2126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.753+0000 7f26dce77740 1 --2- 172.21.15.47:0/147317369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bc2029e0 0x55a2bc20f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.753+0000 7f26dce77740 1 --2- 172.21.15.47:0/147317369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2bc201810 0x55a2bc20f1f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.753+0000 7f26c0c00640 0 client.26344 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:03.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.753+0000 7f26dce77740 1 -- 172.21.15.47:0/147317369 >> 172.21.15.47:0/147317369 conn(0x55a2bc1fcc00 msgr2=0x55a2bc2006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.754+0000 7f26dce77740 1 -- 172.21.15.47:0/147317369 shutdown_connections 2024-03-21T11:26:03.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.754+0000 7f26dce77740 1 -- 172.21.15.47:0/147317369 wait complete. 2024-03-21T11:26:03.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.754+0000 7f26dce77740 20 client.26344 trim_cache size 0 max 16384 2024-03-21T11:26:03.760 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:26:03.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.755+0000 7f26da200640 1 -- 172.21.15.47:0/1514188608 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26b80a3140 msgr2=0x7f26b80a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.755+0000 7f26da200640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26b80a3140 0x7f26b80a5600 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f26d419b140 tx=0x7f26c4005f90 comp rx=0 tx=0).stop 2024-03-21T11:26:03.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.755+0000 7f26da200640 1 -- 172.21.15.47:0/1514188608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d4104a50 msgr2=0x7f26d419a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:03.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.755+0000 7f26da200640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d4104a50 0x7f26d419a1b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f26bc02efd0 tx=0x7f26bc02c660 comp rx=0 tx=0).stop 2024-03-21T11:26:03.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.755+0000 7f26da200640 1 -- 172.21.15.47:0/1514188608 shutdown_connections 2024-03-21T11:26:03.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.755+0000 7f26da200640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26b80a3140 0x7f26b80a5600 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.755+0000 7f26da200640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d4107380 0x7f26d41a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.755+0000 7f26da200640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d4104a50 0x7f26d419a1b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.755+0000 7f26da200640 1 --2- 172.21.15.47:0/1514188608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d4102120 0x7f26d4199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:03.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.755+0000 7f26da200640 1 -- 172.21.15.47:0/1514188608 >> 172.21.15.47:0/1514188608 conn(0x7f26d40fbb10 msgr2=0x7f26d41018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:03.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.755+0000 7f26da200640 1 -- 172.21.15.47:0/1514188608 shutdown_connections 2024-03-21T11:26:03.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:03.755+0000 7f26da200640 1 -- 172.21.15.47:0/1514188608 wait complete. 2024-03-21T11:26:04.492 INFO:tasks.workunit.client.0.smithi047.stdout: 1 79774 9.31 MB/sec warmup 48 sec latency 20.377 ms 2024-03-21T11:26:04.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:04 smithi100 ceph-mon[36245]: pgmap v482: 129 pgs: 129 active+clean; 37 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 333 op/s 2024-03-21T11:26:04.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:04 smithi102 ceph-mon[36748]: pgmap v482: 129 pgs: 129 active+clean; 37 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 333 op/s 2024-03-21T11:26:04.780 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:26:04.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:26:04.781 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:26:04.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:04 smithi047 ceph-mon[31351]: pgmap v482: 129 pgs: 129 active+clean; 37 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 333 op/s 2024-03-21T11:26:05.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.021+0000 7f3cc5600640 1 -- 172.21.15.47:0/1729020679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cc0104930 msgr2=0x7f3cc0106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.021+0000 7f3cc5600640 1 --2- 172.21.15.47:0/1729020679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cc0104930 0x7f3cc0106d20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009900 tx=0x7f3cac02bc40 comp rx=0 tx=0).stop 2024-03-21T11:26:05.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.021+0000 7f3cc5600640 1 -- 172.21.15.47:0/1729020679 shutdown_connections 2024-03-21T11:26:05.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.021+0000 7f3cc5600640 1 --2- 172.21.15.47:0/1729020679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cc0107260 0x7f3cc01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.021+0000 7f3cc5600640 1 --2- 172.21.15.47:0/1729020679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cc0104930 0x7f3cc0106d20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.021+0000 7f3cc5600640 1 --2- 172.21.15.47:0/1729020679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cc0102000 0x7f3cc01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.021+0000 7f3cc5600640 1 -- 172.21.15.47:0/1729020679 >> 172.21.15.47:0/1729020679 conn(0x7f3cc00fba10 msgr2=0x7f3cc00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:05.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.021+0000 7f3cc5600640 1 -- 172.21.15.47:0/1729020679 shutdown_connections 2024-03-21T11:26:05.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.021+0000 7f3cc5600640 1 -- 172.21.15.47:0/1729020679 wait complete. 2024-03-21T11:26:05.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.022+0000 7f3cc5600640 1 Processor -- start 2024-03-21T11:26:05.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.022+0000 7f3cc5600640 1 -- start start 2024-03-21T11:26:05.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cc5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cc0102000 0x7f3cc0199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cc5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cc0104930 0x7f3cc019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cc5600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cc0107260 0x7f3cc01a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cc5600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc010d210 con 0x7f3cc0107260 2024-03-21T11:26:05.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cc5600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc010d090 con 0x7f3cc0104930 2024-03-21T11:26:05.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cc5600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc010d390 con 0x7f3cc0102000 2024-03-21T11:26:05.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cbfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cc0102000 0x7f3cc0199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cbfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cc0102000 0x7f3cc0199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34546/0 (socket says 172.21.15.47:34546) 2024-03-21T11:26:05.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cbfe00640 1 -- 172.21.15.47:0/1940445373 learned_addr learned my addr 172.21.15.47:0/1940445373 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:05.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cbf400640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cc0104930 0x7f3cc019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cc0107260 0x7f3cc01a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cbfe00640 1 -- 172.21.15.47:0/1940445373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cc0104930 msgr2=0x7f3cc019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cbfe00640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cc0104930 0x7f3cc019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cbfe00640 1 -- 172.21.15.47:0/1940445373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cc0107260 msgr2=0x7f3cc01a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cbfe00640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cc0107260 0x7f3cc01a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cbfe00640 1 -- 172.21.15.47:0/1940445373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc01a16f0 con 0x7f3cc0102000 2024-03-21T11:26:05.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.023+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cc0107260 0x7f3cc01a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:05.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.024+0000 7f3cbfe00640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cc0102000 0x7f3cc0199ad0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000e9f0 tx=0x7f3cb000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:05.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.024+0000 7f3cbcc00640 1 -- 172.21.15.47:0/1940445373 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb000cd80 con 0x7f3cc0102000 2024-03-21T11:26:05.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.024+0000 7f3cbcc00640 1 -- 172.21.15.47:0/1940445373 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cb0010910 con 0x7f3cc0102000 2024-03-21T11:26:05.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.024+0000 7f3cc5600640 1 -- 172.21.15.47:0/1940445373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc01a19e0 con 0x7f3cc0102000 2024-03-21T11:26:05.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.024+0000 7f3cbcc00640 1 -- 172.21.15.47:0/1940445373 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0018990 con 0x7f3cc0102000 2024-03-21T11:26:05.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.024+0000 7f3cc5600640 1 -- 172.21.15.47:0/1940445373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc01a1f50 con 0x7f3cc0102000 2024-03-21T11:26:05.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.028+0000 7f3cc9a32740 1 Processor -- start 2024-03-21T11:26:05.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.028+0000 7f3cbcc00640 1 -- 172.21.15.47:0/1940445373 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3cb0010ac0 con 0x7f3cc0102000 2024-03-21T11:26:05.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.028+0000 7f3cc9a32740 1 -- start start 2024-03-21T11:26:05.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.028+0000 7f3cc9a32740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba92b0 0x5653d0ba96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.028+0000 7f3cc9a32740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba9ca0 0x5653d0bae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.028+0000 7f3cc9a32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 0x5653d0bb12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.028+0000 7f3cc9a32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d0b0eeb0 con 0x5653d0baee10 2024-03-21T11:26:05.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.028+0000 7f3cc9a32740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d0ab7a60 con 0x5653d0ba9ca0 2024-03-21T11:26:05.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.028+0000 7f3cc9a32740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d0b0af50 con 0x5653d0ba92b0 2024-03-21T11:26:05.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.029+0000 7f3cbfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba9ca0 0x5653d0bae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.029+0000 7f3cc4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba92b0 0x5653d0ba96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.029+0000 7f3cc4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba92b0 0x5653d0ba96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34562/0 (socket says 172.21.15.47:34562) 2024-03-21T11:26:05.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.029+0000 7f3cc4c00640 1 -- 172.21.15.47:0/4075594551 learned_addr learned my addr 172.21.15.47:0/4075594551 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:05.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.029+0000 7f3cbf400640 1 --2- 172.21.15.47:0/4075594551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 0x5653d0bb12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.029+0000 7f3cc4c00640 1 -- 172.21.15.47:0/4075594551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba9ca0 msgr2=0x5653d0bae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.029+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/4075594551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba9ca0 0x5653d0bae7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.029+0000 7f3cc4c00640 1 -- 172.21.15.47:0/4075594551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 msgr2=0x5653d0bb12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.029+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/4075594551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 0x5653d0bb12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.029+0000 7f3cc4c00640 1 -- 172.21.15.47:0/4075594551 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d08eb7e0 con 0x5653d0ba92b0 2024-03-21T11:26:05.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.029+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/4075594551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba92b0 0x5653d0ba96d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009fb0 tx=0x7f3cb8010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb68b409e0688a07 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:05.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.030+0000 7f3cb5200640 1 -- 172.21.15.47:0/4075594551 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8016070 con 0x5653d0ba92b0 2024-03-21T11:26:05.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.030+0000 7f3cb5200640 1 -- 172.21.15.47:0/4075594551 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cb8012bf0 con 0x5653d0ba92b0 2024-03-21T11:26:05.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.030+0000 7f3cb5200640 1 -- 172.21.15.47:0/4075594551 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb801bd20 con 0x5653d0ba92b0 2024-03-21T11:26:05.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.030+0000 7f3cbcc00640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ca40a2fa0 0x7f3ca40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.030+0000 7f3cbcc00640 1 -- 172.21.15.47:0/1940445373 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3cb00cfaf0 con 0x7f3cc0102000 2024-03-21T11:26:05.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.030+0000 7f3cbfe00640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ca40a2fa0 0x7f3ca40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.031+0000 7f3cc9a32740 1 -- 172.21.15.47:0/4075594551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba92b0 msgr2=0x5653d0ba96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.031+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/4075594551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba92b0 0x5653d0ba96d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009fb0 tx=0x7f3cb8010800 comp rx=0 tx=0).stop 2024-03-21T11:26:05.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.031+0000 7f3cc9a32740 1 -- 172.21.15.47:0/4075594551 shutdown_connections 2024-03-21T11:26:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.031+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/4075594551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 0x5653d0bb12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.031+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/4075594551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba9ca0 0x5653d0bae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.031+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/4075594551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba92b0 0x5653d0ba96d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.031+0000 7f3cc9a32740 1 -- 172.21.15.47:0/4075594551 >> 172.21.15.47:0/4075594551 conn(0x5653d0ba4820 msgr2=0x5653d0ba4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.031+0000 7f3cc9a32740 1 -- 172.21.15.47:0/4075594551 shutdown_connections 2024-03-21T11:26:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.031+0000 7f3cc9a32740 1 -- 172.21.15.47:0/4075594551 wait complete. 2024-03-21T11:26:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.031+0000 7f3cbfe00640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ca40a2fa0 0x7f3ca40a5460 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3cb009bdc0 tx=0x7f3cb00d3530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.031+0000 7f3cc9a32740 1 -- start start 2024-03-21T11:26:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.032+0000 7f3cc9a32740 1 Processor -- start 2024-03-21T11:26:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.032+0000 7f3cc9a32740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba92b0 0x5653d0bbbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.032+0000 7f3cc9a32740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba9ca0 0x5653d0bbc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.032+0000 7f3cc9a32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 0x5653d0bc3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.032+0000 7f3cc9a32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d0bb1dc0 con 0x5653d0baee10 2024-03-21T11:26:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.032+0000 7f3cc9a32740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d0acaeb0 con 0x5653d0ba92b0 2024-03-21T11:26:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.032+0000 7f3cc9a32740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d0bb1f40 con 0x5653d0ba9ca0 2024-03-21T11:26:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.033+0000 7f3cbf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba9ca0 0x5653d0bbc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.033+0000 7f3cbf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba9ca0 0x5653d0bbc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34574/0 (socket says 172.21.15.47:34574) 2024-03-21T11:26:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.033+0000 7f3cbf400640 1 -- 172.21.15.47:0/3388430305 learned_addr learned my addr 172.21.15.47:0/3388430305 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.033+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/3388430305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba92b0 0x5653d0bbbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.033+0000 7f3cc9a32740 10 client.0 _ll_register_callbacks cb 0x5653d0abf150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.033+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/3388430305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 0x5653d0bc3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.033+0000 7f3cbf400640 1 -- 172.21.15.47:0/3388430305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba92b0 msgr2=0x5653d0bbbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.033+0000 7f3cbf400640 1 --2- 172.21.15.47:0/3388430305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba92b0 0x5653d0bbbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.033+0000 7f3cbf400640 1 -- 172.21.15.47:0/3388430305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 msgr2=0x5653d0bc3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.033+0000 7f3cbf400640 1 --2- 172.21.15.47:0/3388430305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 0x5653d0bc3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.033+0000 7f3cbf400640 1 -- 172.21.15.47:0/3388430305 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653d094bf70 con 0x5653d0ba9ca0 2024-03-21T11:26:05.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.033+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/3388430305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 0x5653d0bc3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.034+0000 7f3cbf400640 1 --2- 172.21.15.47:0/3388430305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba9ca0 0x5653d0bbc240 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3cac005c40 tx=0x7f3cac009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d293b95df69217a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.034+0000 7f3ca9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.034+0000 7f3ca9600640 1 -- 172.21.15.47:0/3388430305 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac005d50 con 0x5653d0ba9ca0 2024-03-21T11:26:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.034+0000 7f3cc9a32740 1 -- 172.21.15.47:0/3388430305 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cb8012bf0 con 0x5653d0ba9ca0 2024-03-21T11:26:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.034+0000 7f3ca9600640 1 -- 172.21.15.47:0/3388430305 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3cac005230 con 0x5653d0ba9ca0 2024-03-21T11:26:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.034+0000 7f3ca9600640 1 -- 172.21.15.47:0/3388430305 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cac04f050 con 0x5653d0ba9ca0 2024-03-21T11:26:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.034+0000 7f3cc9a32740 10 client.26356 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.034+0000 7f3cc9a32740 1 -- 172.21.15.47:0/3388430305 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5653d07e3170 con 0x5653d0ba9ca0 2024-03-21T11:26:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.034+0000 7f3ca9600640 1 -- 172.21.15.47:0/3388430305 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cb8012bf0 con 0x5653d0ba9ca0 2024-03-21T11:26:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.035+0000 7f3ca9600640 1 -- 172.21.15.47:0/3388430305 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3cac003730 con 0x5653d0ba9ca0 2024-03-21T11:26:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.035+0000 7f3cc9a32740 10 client.26356 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.035+0000 7f3cc9a32740 10 client.26356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.035+0000 7f3cc9a32740 20 client.26356 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:05.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.035+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/3388430305 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653d0bb7510 0x5653d0bb9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.035+0000 7f3cc9a32740 4 client.26356 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.035+0000 7f3cc9a32740 1 -- 172.21.15.47:0/3388430305 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653d0b1c3f0 con 0x5653d0bb7510 2024-03-21T11:26:05.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.035+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/3388430305 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653d0bb7510 0x5653d0bb9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.036+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/3388430305 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653d0bb7510 0x5653d0bb9940 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c053daf481501a0 server_cookie=703cdba195fcdc8e in_seq=0 out_seq=0 2024-03-21T11:26:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.036+0000 7f3ca9600640 10 client.26356 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.037+0000 7f3ca9600640 1 -- 172.21.15.47:0/3388430305 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5653d0b1c3f0 con 0x5653d0bb7510 2024-03-21T11:26:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.037+0000 7f3ca9600640 10 client.26356 handle_command_reply: tid=0 2024-03-21T11:26:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.037+0000 7f3cc9a32740 1 client.26356 shutdown 2024-03-21T11:26:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.037+0000 7f3cc9a32740 1 -- 172.21.15.47:0/3388430305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba9ca0 msgr2=0x5653d0bbc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.037+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/3388430305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba9ca0 0x5653d0bbc240 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3cac005c40 tx=0x7f3cac009680 comp rx=0 tx=0).stop 2024-03-21T11:26:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.037+0000 7f3cc9a32740 1 -- 172.21.15.47:0/3388430305 shutdown_connections 2024-03-21T11:26:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.037+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/3388430305 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653d0bb7510 0x5653d0bb9940 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.037+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/3388430305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 0x5653d0bc3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.037+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/3388430305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba9ca0 0x5653d0bbc240 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.037+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/3388430305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba92b0 0x5653d0bbbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.037+0000 7f3ca9600640 0 client.26356 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.037+0000 7f3cc9a32740 1 -- 172.21.15.47:0/3388430305 >> 172.21.15.47:0/3388430305 conn(0x5653d0ba4820 msgr2=0x5653d0ba3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.038+0000 7f3cc9a32740 1 -- 172.21.15.47:0/3388430305 shutdown_connections 2024-03-21T11:26:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.038+0000 7f3cc9a32740 1 -- 172.21.15.47:0/3388430305 wait complete. 2024-03-21T11:26:05.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.038+0000 7f3cc9a32740 20 client.26356 trim_cache size 0 max 16384 2024-03-21T11:26:05.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.057+0000 7f3cc9a32740 1 Processor -- start 2024-03-21T11:26:05.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.057+0000 7f3cc9a32740 1 -- start start 2024-03-21T11:26:05.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.057+0000 7f3cc9a32740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba92b0 0x5653d0ba80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.057+0000 7f3cc9a32740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba9ca0 0x5653d0bc3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cc9a32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 0x5653d0bd87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cc9a32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc01a7cf0 con 0x5653d0baee10 2024-03-21T11:26:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cc9a32740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc01a9670 con 0x5653d0ba9ca0 2024-03-21T11:26:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cc9a32740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc01a8350 con 0x5653d0ba92b0 2024-03-21T11:26:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cbf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba9ca0 0x5653d0bc3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cbf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba9ca0 0x5653d0bc3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34528/0 (socket says 172.21.15.47:34528) 2024-03-21T11:26:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cbf400640 1 -- 172.21.15.47:0/3759746501 learned_addr learned my addr 172.21.15.47:0/3759746501 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/3759746501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba92b0 0x5653d0ba80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/3759746501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 0x5653d0bd87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cbf400640 1 -- 172.21.15.47:0/3759746501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba92b0 msgr2=0x5653d0ba80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cbf400640 1 --2- 172.21.15.47:0/3759746501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba92b0 0x5653d0ba80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cbf400640 1 -- 172.21.15.47:0/3759746501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 msgr2=0x5653d0bd87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cbf400640 1 --2- 172.21.15.47:0/3759746501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 0x5653d0bd87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cbf400640 1 -- 172.21.15.47:0/3759746501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d0baa530 con 0x5653d0ba9ca0 2024-03-21T11:26:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.058+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/3759746501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 0x5653d0bd87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:26:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.059+0000 7f3cbf400640 1 --2- 172.21.15.47:0/3759746501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba9ca0 0x5653d0bc3240 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3cac003b50 tx=0x7f3cac02bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46997880f108274e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.059+0000 7f3cabe00640 1 -- 172.21.15.47:0/3759746501 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac005d50 con 0x5653d0ba9ca0 2024-03-21T11:26:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.059+0000 7f3cabe00640 1 -- 172.21.15.47:0/3759746501 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cac005550 con 0x5653d0ba9ca0 2024-03-21T11:26:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.059+0000 7f3cabe00640 1 -- 172.21.15.47:0/3759746501 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac004250 con 0x5653d0ba9ca0 2024-03-21T11:26:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.060+0000 7f3cc9a32740 1 -- 172.21.15.47:0/3759746501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba9ca0 msgr2=0x5653d0bc3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.060+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/3759746501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba9ca0 0x5653d0bc3240 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3cac003b50 tx=0x7f3cac02bc40 comp rx=0 tx=0).stop 2024-03-21T11:26:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.060+0000 7f3cc9a32740 1 -- 172.21.15.47:0/3759746501 shutdown_connections 2024-03-21T11:26:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.060+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/3759746501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0baee10 0x5653d0bd87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.060+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/3759746501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba9ca0 0x5653d0bc3240 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.060+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/3759746501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0ba92b0 0x5653d0ba80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.060+0000 7f3cc9a32740 1 -- 172.21.15.47:0/3759746501 >> 172.21.15.47:0/3759746501 conn(0x5653d0ba4820 msgr2=0x5653d0ba3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.060+0000 7f3cc9a32740 1 -- 172.21.15.47:0/3759746501 shutdown_connections 2024-03-21T11:26:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.060+0000 7f3cc9a32740 1 -- 172.21.15.47:0/3759746501 wait complete. 2024-03-21T11:26:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.060+0000 7f3cc9a32740 1 -- start start 2024-03-21T11:26:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.061+0000 7f3cc9a32740 1 Processor -- start 2024-03-21T11:26:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.061+0000 7f3cc9a32740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba92b0 0x5653d0bc0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.061+0000 7f3cc9a32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0ba9ca0 0x5653d0bc09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.061+0000 7f3cc9a32740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0baee10 0x5653d0bc19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.061+0000 7f3cc9a32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc01a9010 con 0x5653d0ba9ca0 2024-03-21T11:26:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.061+0000 7f3cc9a32740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc01a89b0 con 0x5653d0ba92b0 2024-03-21T11:26:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.061+0000 7f3cc9a32740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc01a5eb0 con 0x5653d0baee10 2024-03-21T11:26:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.061+0000 7f3cc9a32740 10 client.0 _ll_register_callbacks cb 0x7f3cac005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.062+0000 7f3cbf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0ba9ca0 0x5653d0bc09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.062+0000 7f3cc4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba92b0 0x5653d0bc0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.062+0000 7f3cbf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0ba9ca0 0x5653d0bc09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54046/0 (socket says 172.21.15.47:54046) 2024-03-21T11:26:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.062+0000 7f3cbf400640 1 -- 172.21.15.47:0/413369027 learned_addr learned my addr 172.21.15.47:0/413369027 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.062+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/413369027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0baee10 0x5653d0bc19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.062+0000 7f3cbf400640 1 -- 172.21.15.47:0/413369027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0baee10 msgr2=0x5653d0bc19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.062+0000 7f3cbf400640 1 --2- 172.21.15.47:0/413369027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0baee10 0x5653d0bc19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.062+0000 7f3cbf400640 1 -- 172.21.15.47:0/413369027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba92b0 msgr2=0x5653d0bc0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.062+0000 7f3cbf400640 1 --2- 172.21.15.47:0/413369027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba92b0 0x5653d0bc0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.062+0000 7f3cbf400640 1 -- 172.21.15.47:0/413369027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653d0bc1fd0 con 0x5653d0ba9ca0 2024-03-21T11:26:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.062+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/413369027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba92b0 0x5653d0bc0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.062+0000 7f3cbf400640 1 --2- 172.21.15.47:0/413369027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0ba9ca0 0x5653d0bc09d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3cac02c7c0 tx=0x7f3cac004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0d0e8f883afbaa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.062+0000 7f3ca9600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:26:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.063+0000 7f3ca9600640 1 -- 172.21.15.47:0/413369027 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac046680 con 0x5653d0ba9ca0 2024-03-21T11:26:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.063+0000 7f3cc9a32740 1 -- 172.21.15.47:0/413369027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cac005550 con 0x5653d0ba9ca0 2024-03-21T11:26:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.063+0000 7f3ca9600640 1 -- 172.21.15.47:0/413369027 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3cac046820 con 0x5653d0ba9ca0 2024-03-21T11:26:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.063+0000 7f3ca9600640 1 -- 172.21.15.47:0/413369027 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cac0036b0 con 0x5653d0ba9ca0 2024-03-21T11:26:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.063+0000 7f3cc9a32740 10 client.16146 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.063+0000 7f3cc9a32740 1 -- 172.21.15.47:0/413369027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5653d0bc3280 con 0x5653d0ba9ca0 2024-03-21T11:26:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.063+0000 7f3ca9600640 1 -- 172.21.15.47:0/413369027 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cac005550 con 0x5653d0ba9ca0 2024-03-21T11:26:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.064+0000 7f3ca9600640 1 -- 172.21.15.47:0/413369027 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3cac054c20 con 0x5653d0ba9ca0 2024-03-21T11:26:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.064+0000 7f3cc9a32740 10 client.16146 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.064+0000 7f3cc9a32740 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.064+0000 7f3cc9a32740 20 client.16146 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.064+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/413369027 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653d0c2d7d0 0x5653d0c2fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.064+0000 7f3cc9a32740 4 client.16146 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:26:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.064+0000 7f3cc9a32740 1 -- 172.21.15.47:0/413369027 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5653d0b16740 con 0x5653d0c2d7d0 2024-03-21T11:26:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.064+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/413369027 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653d0c2d7d0 0x5653d0c2fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.064+0000 7f3cc4c00640 1 --2- 172.21.15.47:0/413369027 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653d0c2d7d0 0x5653d0c2fbf0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e2b71f793f6939 server_cookie=9e33114cdc9fd513 in_seq=0 out_seq=0 2024-03-21T11:26:05.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.064+0000 7f3ca9600640 10 client.16146 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:05.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.065+0000 7f3ca9600640 1 -- 172.21.15.47:0/413369027 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5653d0b16740 con 0x5653d0c2d7d0 2024-03-21T11:26:05.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.065+0000 7f3ca9600640 10 client.16146 handle_command_reply: tid=0 2024-03-21T11:26:05.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.066+0000 7f3cc9a32740 1 client.16146 shutdown 2024-03-21T11:26:05.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.066+0000 7f3cc9a32740 1 -- 172.21.15.47:0/413369027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0ba9ca0 msgr2=0x5653d0bc09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.066+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/413369027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0ba9ca0 0x5653d0bc09d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3cac02c7c0 tx=0x7f3cac004370 comp rx=0 tx=0).stop 2024-03-21T11:26:05.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.066+0000 7f3cc9a32740 1 -- 172.21.15.47:0/413369027 shutdown_connections 2024-03-21T11:26:05.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.066+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/413369027 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653d0c2d7d0 0x5653d0c2fbf0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.066+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/413369027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653d0baee10 0x5653d0bc19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.066+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/413369027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653d0ba9ca0 0x5653d0bc09d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.066+0000 7f3cc9a32740 1 --2- 172.21.15.47:0/413369027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653d0ba92b0 0x5653d0bc0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.066+0000 7f3cc9a32740 1 -- 172.21.15.47:0/413369027 >> 172.21.15.47:0/413369027 conn(0x5653d0ba4820 msgr2=0x5653d0baa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.066+0000 7f3ca9600640 0 client.16146 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.066+0000 7f3cc9a32740 1 -- 172.21.15.47:0/413369027 shutdown_connections 2024-03-21T11:26:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.066+0000 7f3cc9a32740 1 -- 172.21.15.47:0/413369027 wait complete. 2024-03-21T11:26:05.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.067+0000 7f3cc9a32740 20 client.16146 trim_cache size 0 max 16384 2024-03-21T11:26:05.073 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:26:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:26:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "52171052-e3c4-4c71-ac4e-f76bec62c9d0", 2024-03-21T11:26:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:26:05.074 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:26:05.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.067+0000 7f3cc5600640 1 -- 172.21.15.47:0/1940445373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ca40a2fa0 msgr2=0x7f3ca40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.067+0000 7f3cc5600640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ca40a2fa0 0x7f3ca40a5460 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3cb009bdc0 tx=0x7f3cb00d3530 comp rx=0 tx=0).stop 2024-03-21T11:26:05.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.067+0000 7f3cc5600640 1 -- 172.21.15.47:0/1940445373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cc0102000 msgr2=0x7f3cc0199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.067+0000 7f3cc5600640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cc0102000 0x7f3cc0199ad0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000e9f0 tx=0x7f3cb000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.067+0000 7f3cc5600640 1 -- 172.21.15.47:0/1940445373 shutdown_connections 2024-03-21T11:26:05.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.067+0000 7f3cc5600640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ca40a2fa0 0x7f3ca40a5460 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.067+0000 7f3cc5600640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cc0107260 0x7f3cc01a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.067+0000 7f3cc5600640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cc0104930 0x7f3cc019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.067+0000 7f3cc5600640 1 --2- 172.21.15.47:0/1940445373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cc0102000 0x7f3cc0199ad0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.067+0000 7f3cc5600640 1 -- 172.21.15.47:0/1940445373 >> 172.21.15.47:0/1940445373 conn(0x7f3cc00fba10 msgr2=0x7f3cc01017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:05.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.068+0000 7f3cc5600640 1 -- 172.21.15.47:0/1940445373 shutdown_connections 2024-03-21T11:26:05.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.068+0000 7f3cc5600640 1 -- 172.21.15.47:0/1940445373 wait complete. 2024-03-21T11:26:05.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:26:05.088 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:26:05.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.355+0000 7f789be00640 1 -- 172.21.15.47:0/1946337163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78941005c0 msgr2=0x7f78941009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.355+0000 7f789be00640 1 --2- 172.21.15.47:0/1946337163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78941005c0 0x7f78941009e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7888009900 tx=0x7f788802bc50 comp rx=0 tx=0).stop 2024-03-21T11:26:05.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.355+0000 7f789be00640 1 -- 172.21.15.47:0/1946337163 shutdown_connections 2024-03-21T11:26:05.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.355+0000 7f789be00640 1 --2- 172.21.15.47:0/1946337163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7894102170 0x7f7894108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.355+0000 7f789be00640 1 --2- 172.21.15.47:0/1946337163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78941017b0 0x7f7894101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.355+0000 7f789be00640 1 --2- 172.21.15.47:0/1946337163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78941005c0 0x7f78941009e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.355+0000 7f789be00640 1 -- 172.21.15.47:0/1946337163 >> 172.21.15.47:0/1946337163 conn(0x7f78940fbb50 msgr2=0x7f78940fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:05.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.356+0000 7f789be00640 1 -- 172.21.15.47:0/1946337163 shutdown_connections 2024-03-21T11:26:05.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.356+0000 7f789be00640 1 -- 172.21.15.47:0/1946337163 wait complete. 2024-03-21T11:26:05.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.357+0000 7f789be00640 1 Processor -- start 2024-03-21T11:26:05.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.357+0000 7f789be00640 1 -- start start 2024-03-21T11:26:05.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.357+0000 7f789be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78941005c0 0x7f78941044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.357+0000 7f789be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78941017b0 0x7f7894102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.357+0000 7f789be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7894102170 0x7f78941030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f789be00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789410c280 con 0x7f78941005c0 2024-03-21T11:26:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f789be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789410c100 con 0x7f7894102170 2024-03-21T11:26:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f789be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789410c400 con 0x7f78941017b0 2024-03-21T11:26:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f7899400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78941005c0 0x7f78941044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f7899e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7894102170 0x7f78941030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f7898a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78941017b0 0x7f7894102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f7899400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78941005c0 0x7f78941044e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54060/0 (socket says 172.21.15.47:54060) 2024-03-21T11:26:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f7899e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7894102170 0x7f78941030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34562/0 (socket says 172.21.15.47:34562) 2024-03-21T11:26:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f7899400640 1 -- 172.21.15.47:0/1167606814 learned_addr learned my addr 172.21.15.47:0/1167606814 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f7898a00640 1 -- 172.21.15.47:0/1167606814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7894102170 msgr2=0x7f78941030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f7898a00640 1 --2- 172.21.15.47:0/1167606814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7894102170 0x7f78941030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f7898a00640 1 -- 172.21.15.47:0/1167606814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78941005c0 msgr2=0x7f78941044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f7898a00640 1 --2- 172.21.15.47:0/1167606814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78941005c0 0x7f78941044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f7898a00640 1 -- 172.21.15.47:0/1167606814 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78941038f0 con 0x7f78941017b0 2024-03-21T11:26:05.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.358+0000 7f7898a00640 1 --2- 172.21.15.47:0/1167606814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78941017b0 0x7f7894102b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f788000e9f0 tx=0x7f788000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:05.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.359+0000 7f7892000640 1 -- 172.21.15.47:0/1167606814 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788000ccc0 con 0x7f78941017b0 2024-03-21T11:26:05.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.359+0000 7f789be00640 1 -- 172.21.15.47:0/1167606814 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78941a1b30 con 0x7f78941017b0 2024-03-21T11:26:05.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.359+0000 7f7892000640 1 -- 172.21.15.47:0/1167606814 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7880010910 con 0x7f78941017b0 2024-03-21T11:26:05.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.359+0000 7f7892000640 1 -- 172.21.15.47:0/1167606814 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880018960 con 0x7f78941017b0 2024-03-21T11:26:05.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.359+0000 7f789be00640 1 -- 172.21.15.47:0/1167606814 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78941a20d0 con 0x7f78941017b0 2024-03-21T11:26:05.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.364+0000 7f789ebb2740 1 Processor -- start 2024-03-21T11:26:05.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.364+0000 7f789ebb2740 1 -- start start 2024-03-21T11:26:05.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.364+0000 7f789ebb2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f342e80 0x555a3f3432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.364+0000 7f789ebb2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f343870 0x555a3f348370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.364+0000 7f789ebb2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f3489e0 0x555a3f34ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.364+0000 7f789ebb2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3f2a8250 con 0x555a3f3489e0 2024-03-21T11:26:05.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.364+0000 7f789ebb2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3f250a30 con 0x555a3f342e80 2024-03-21T11:26:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.364+0000 7f789ebb2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3f2a42f0 con 0x555a3f343870 2024-03-21T11:26:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.365+0000 7f7899e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f342e80 0x555a3f3432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.365+0000 7f7899e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f342e80 0x555a3f3432a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34574/0 (socket says 172.21.15.47:34574) 2024-03-21T11:26:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.365+0000 7f7899e00640 1 -- 172.21.15.47:0/3508465277 learned_addr learned my addr 172.21.15.47:0/3508465277 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.365+0000 7f7899400640 1 --2- 172.21.15.47:0/3508465277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f343870 0x555a3f348370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.365+0000 7f7892000640 1 -- 172.21.15.47:0/1167606814 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7880010ac0 con 0x7f78941017b0 2024-03-21T11:26:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.365+0000 7f7899e00640 1 -- 172.21.15.47:0/3508465277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f343870 msgr2=0x555a3f348370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.365+0000 7f7899e00640 1 --2- 172.21.15.47:0/3508465277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f343870 0x555a3f348370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.365+0000 7f7899e00640 1 -- 172.21.15.47:0/3508465277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f3489e0 msgr2=0x555a3f34ae70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:26:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.365+0000 7f7899e00640 1 --2- 172.21.15.47:0/3508465277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f3489e0 0x555a3f34ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.365+0000 7f7899e00640 1 -- 172.21.15.47:0/3508465277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a3f23b690 con 0x555a3f342e80 2024-03-21T11:26:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.365+0000 7f7899e00640 1 --2- 172.21.15.47:0/3508465277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f342e80 0x555a3f3432a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f788c009fd0 tx=0x7f788c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54c575098bf90ecc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.366+0000 7f7886600640 1 -- 172.21.15.47:0/3508465277 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c004d20 con 0x555a3f342e80 2024-03-21T11:26:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.366+0000 7f7886600640 1 -- 172.21.15.47:0/3508465277 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f788c004050 con 0x555a3f342e80 2024-03-21T11:26:05.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.366+0000 7f7886600640 1 -- 172.21.15.47:0/3508465277 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c005260 con 0x555a3f342e80 2024-03-21T11:26:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.366+0000 7f7892000640 1 --2- 172.21.15.47:0/1167606814 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f787c0a2fa0 0x7f787c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.366+0000 7f7892000640 1 -- 172.21.15.47:0/1167606814 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f78800cfa90 con 0x7f78941017b0 2024-03-21T11:26:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.367+0000 7f789ebb2740 1 -- 172.21.15.47:0/3508465277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f342e80 msgr2=0x555a3f3432a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.367+0000 7f789ebb2740 1 --2- 172.21.15.47:0/3508465277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f342e80 0x555a3f3432a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f788c009fd0 tx=0x7f788c0118d0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.367+0000 7f7899400640 1 --2- 172.21.15.47:0/1167606814 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f787c0a2fa0 0x7f787c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.367+0000 7f789ebb2740 1 -- 172.21.15.47:0/3508465277 shutdown_connections 2024-03-21T11:26:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.367+0000 7f789ebb2740 1 --2- 172.21.15.47:0/3508465277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f3489e0 0x555a3f34ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.367+0000 7f789ebb2740 1 --2- 172.21.15.47:0/3508465277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f343870 0x555a3f348370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.367+0000 7f789ebb2740 1 --2- 172.21.15.47:0/3508465277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f342e80 0x555a3f3432a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.367+0000 7f789ebb2740 1 -- 172.21.15.47:0/3508465277 >> 172.21.15.47:0/3508465277 conn(0x555a3f30b200 msgr2=0x555a3f33e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.367+0000 7f789ebb2740 1 -- 172.21.15.47:0/3508465277 shutdown_connections 2024-03-21T11:26:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.367+0000 7f789ebb2740 1 -- 172.21.15.47:0/3508465277 wait complete. 2024-03-21T11:26:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.367+0000 7f789ebb2740 1 -- start start 2024-03-21T11:26:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.368+0000 7f7899400640 1 --2- 172.21.15.47:0/1167606814 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f787c0a2fa0 0x7f787c0a5460 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f788802c900 tx=0x7f7888000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.368+0000 7f789ebb2740 1 Processor -- start 2024-03-21T11:26:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.368+0000 7f789ebb2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f342e80 0x555a3f34ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.368+0000 7f789ebb2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f343870 0x555a3f34d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.368+0000 7f789ebb2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f3489e0 0x555a3f3543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.368+0000 7f789ebb2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3f34b990 con 0x555a3f3489e0 2024-03-21T11:26:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.368+0000 7f789ebb2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3f263e80 con 0x555a3f343870 2024-03-21T11:26:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.368+0000 7f789ebb2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3f34bb10 con 0x555a3f342e80 2024-03-21T11:26:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.368+0000 7f789ebb2740 10 client.0 _ll_register_callbacks cb 0x555a3f276130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.368+0000 7f7899400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f342e80 0x555a3f34ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.368+0000 7f7898a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f343870 0x555a3f34d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.368+0000 7f7899400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f342e80 0x555a3f34ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34624/0 (socket says 172.21.15.47:34624) 2024-03-21T11:26:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.368+0000 7f7899400640 1 -- 172.21.15.47:0/2132083889 learned_addr learned my addr 172.21.15.47:0/2132083889 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.368+0000 7f7898a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f343870 0x555a3f34d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34580/0 (socket says 172.21.15.47:34580) 2024-03-21T11:26:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.369+0000 7f7899e00640 1 --2- 172.21.15.47:0/2132083889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f3489e0 0x555a3f3543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.369+0000 7f7899400640 1 -- 172.21.15.47:0/2132083889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f343870 msgr2=0x555a3f34d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.369+0000 7f7899400640 1 --2- 172.21.15.47:0/2132083889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f343870 0x555a3f34d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.369+0000 7f7899400640 1 -- 172.21.15.47:0/2132083889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f3489e0 msgr2=0x555a3f3543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.369+0000 7f7899400640 1 --2- 172.21.15.47:0/2132083889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f3489e0 0x555a3f3543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.369+0000 7f7899400640 1 -- 172.21.15.47:0/2132083889 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a3f0867d0 con 0x555a3f342e80 2024-03-21T11:26:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.369+0000 7f7899400640 1 --2- 172.21.15.47:0/2132083889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f342e80 0x555a3f34ce50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7888036040 tx=0x7f7888009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c5bc50868855f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.369+0000 7f787aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.369+0000 7f787aa00640 1 -- 172.21.15.47:0/2132083889 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78880050d0 con 0x555a3f342e80 2024-03-21T11:26:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.369+0000 7f787aa00640 1 -- 172.21.15.47:0/2132083889 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7888005270 con 0x555a3f342e80 2024-03-21T11:26:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.370+0000 7f789ebb2740 1 -- 172.21.15.47:0/2132083889 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f788c004050 con 0x555a3f342e80 2024-03-21T11:26:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.370+0000 7f787aa00640 1 -- 172.21.15.47:0/2132083889 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7888005470 con 0x555a3f342e80 2024-03-21T11:26:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.370+0000 7f789ebb2740 10 client.26374 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.370+0000 7f789ebb2740 1 -- 172.21.15.47:0/2132083889 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555a3efbc3f0 con 0x555a3f342e80 2024-03-21T11:26:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.370+0000 7f787aa00640 1 -- 172.21.15.47:0/2132083889 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f788c004050 con 0x555a3f342e80 2024-03-21T11:26:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.370+0000 7f787aa00640 1 -- 172.21.15.47:0/2132083889 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7888051070 con 0x555a3f342e80 2024-03-21T11:26:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.370+0000 7f789ebb2740 10 client.26374 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.370+0000 7f789ebb2740 10 client.26374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.370+0000 7f789ebb2740 20 client.26374 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.370+0000 7f789ebb2740 1 --2- 172.21.15.47:0/2132083889 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555a3f358c50 0x555a3f35b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.370+0000 7f789ebb2740 4 client.26374 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.370+0000 7f789ebb2740 1 -- 172.21.15.47:0/2132083889 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a3f2b5790 con 0x555a3f358c50 2024-03-21T11:26:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.371+0000 7f7898a00640 1 --2- 172.21.15.47:0/2132083889 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555a3f358c50 0x555a3f35b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.371+0000 7f7898a00640 1 --2- 172.21.15.47:0/2132083889 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555a3f358c50 0x555a3f35b090 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f223ac923fb9357 server_cookie=710092d9f551bd2c in_seq=0 out_seq=0 2024-03-21T11:26:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.371+0000 7f787aa00640 10 client.26374 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.372+0000 7f787aa00640 1 -- 172.21.15.47:0/2132083889 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555a3f2b5790 con 0x555a3f358c50 2024-03-21T11:26:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.372+0000 7f787aa00640 10 client.26374 handle_command_reply: tid=0 2024-03-21T11:26:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.372+0000 7f789ebb2740 1 client.26374 shutdown 2024-03-21T11:26:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.372+0000 7f789ebb2740 1 -- 172.21.15.47:0/2132083889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f342e80 msgr2=0x555a3f34ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.372+0000 7f789ebb2740 1 --2- 172.21.15.47:0/2132083889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f342e80 0x555a3f34ce50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7888036040 tx=0x7f7888009260 comp rx=0 tx=0).stop 2024-03-21T11:26:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.372+0000 7f789ebb2740 1 -- 172.21.15.47:0/2132083889 shutdown_connections 2024-03-21T11:26:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.372+0000 7f789ebb2740 1 --2- 172.21.15.47:0/2132083889 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555a3f358c50 0x555a3f35b090 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.372+0000 7f789ebb2740 1 --2- 172.21.15.47:0/2132083889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f3489e0 0x555a3f3543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.372+0000 7f789ebb2740 1 --2- 172.21.15.47:0/2132083889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f343870 0x555a3f34d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.372+0000 7f789ebb2740 1 --2- 172.21.15.47:0/2132083889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f342e80 0x555a3f34ce50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.372+0000 7f787aa00640 0 client.26374 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.372+0000 7f789ebb2740 1 -- 172.21.15.47:0/2132083889 >> 172.21.15.47:0/2132083889 conn(0x555a3f30b200 msgr2=0x555a3f33da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.373+0000 7f789ebb2740 1 -- 172.21.15.47:0/2132083889 shutdown_connections 2024-03-21T11:26:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.373+0000 7f789ebb2740 1 -- 172.21.15.47:0/2132083889 wait complete. 2024-03-21T11:26:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.373+0000 7f789ebb2740 20 client.26374 trim_cache size 0 max 16384 2024-03-21T11:26:05.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.394+0000 7f789ebb2740 1 Processor -- start 2024-03-21T11:26:05.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.394+0000 7f789ebb2740 1 -- start start 2024-03-21T11:26:05.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.394+0000 7f789ebb2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f342e80 0x555a3f34d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.394+0000 7f789ebb2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f343870 0x555a3f354390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.394+0000 7f789ebb2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f3489e0 0x555a3f344980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.394+0000 7f789ebb2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78941ac250 con 0x555a3f342e80 2024-03-21T11:26:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.394+0000 7f789ebb2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78941adbd0 con 0x555a3f343870 2024-03-21T11:26:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.394+0000 7f789ebb2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78941ac8b0 con 0x555a3f3489e0 2024-03-21T11:26:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.394+0000 7f7898a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f343870 0x555a3f354390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.394+0000 7f7899400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f342e80 0x555a3f34d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.394+0000 7f7899e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f3489e0 0x555a3f344980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.394+0000 7f7898a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f343870 0x555a3f354390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34586/0 (socket says 172.21.15.47:34586) 2024-03-21T11:26:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.394+0000 7f7898a00640 1 -- 172.21.15.47:0/1203889279 learned_addr learned my addr 172.21.15.47:0/1203889279 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.395+0000 7f7898a00640 1 -- 172.21.15.47:0/1203889279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f3489e0 msgr2=0x555a3f344980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.395+0000 7f7898a00640 1 --2- 172.21.15.47:0/1203889279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f3489e0 0x555a3f344980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.395+0000 7f7898a00640 1 -- 172.21.15.47:0/1203889279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f342e80 msgr2=0x555a3f34d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.395+0000 7f7898a00640 1 --2- 172.21.15.47:0/1203889279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f342e80 0x555a3f34d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.395+0000 7f7898a00640 1 -- 172.21.15.47:0/1203889279 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a3ef7c0f0 con 0x555a3f343870 2024-03-21T11:26:05.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.395+0000 7f7899400640 1 --2- 172.21.15.47:0/1203889279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f342e80 0x555a3f34d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:05.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.395+0000 7f7898a00640 1 --2- 172.21.15.47:0/1203889279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f343870 0x555a3f354390 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f788009bb80 tx=0x7f788000cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3e9aa6013be2c14 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:05.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.395+0000 7f7885c00640 1 -- 172.21.15.47:0/1203889279 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788009a180 con 0x555a3f343870 2024-03-21T11:26:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.395+0000 7f7885c00640 1 -- 172.21.15.47:0/1203889279 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78800cd2b0 con 0x555a3f343870 2024-03-21T11:26:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.395+0000 7f7885c00640 1 -- 172.21.15.47:0/1203889279 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788009b2d0 con 0x555a3f343870 2024-03-21T11:26:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.396+0000 7f789ebb2740 1 -- 172.21.15.47:0/1203889279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f343870 msgr2=0x555a3f354390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.396+0000 7f789ebb2740 1 --2- 172.21.15.47:0/1203889279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f343870 0x555a3f354390 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f788009bb80 tx=0x7f788000cf60 comp rx=0 tx=0).stop 2024-03-21T11:26:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.396+0000 7f789ebb2740 1 -- 172.21.15.47:0/1203889279 shutdown_connections 2024-03-21T11:26:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.396+0000 7f789ebb2740 1 --2- 172.21.15.47:0/1203889279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f3489e0 0x555a3f344980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.396+0000 7f789ebb2740 1 --2- 172.21.15.47:0/1203889279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f343870 0x555a3f354390 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.396+0000 7f789ebb2740 1 --2- 172.21.15.47:0/1203889279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f342e80 0x555a3f34d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.396+0000 7f789ebb2740 1 -- 172.21.15.47:0/1203889279 >> 172.21.15.47:0/1203889279 conn(0x555a3f30b200 msgr2=0x555a3f344270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.397+0000 7f789ebb2740 1 -- 172.21.15.47:0/1203889279 shutdown_connections 2024-03-21T11:26:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.397+0000 7f789ebb2740 1 -- 172.21.15.47:0/1203889279 wait complete. 2024-03-21T11:26:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.397+0000 7f789ebb2740 1 -- start start 2024-03-21T11:26:05.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.397+0000 7f789ebb2740 1 Processor -- start 2024-03-21T11:26:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f789ebb2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f342e80 0x555a3f351ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f789ebb2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f343870 0x555a3f352010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f789ebb2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f3489e0 0x555a3f353020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f789ebb2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78941ad570 con 0x555a3f3489e0 2024-03-21T11:26:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f789ebb2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78941acf10 con 0x555a3f342e80 2024-03-21T11:26:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f789ebb2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78941aa410 con 0x555a3f343870 2024-03-21T11:26:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f789ebb2740 10 client.0 _ll_register_callbacks cb 0x7f7888009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f7898a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f343870 0x555a3f352010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f7898a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f343870 0x555a3f352010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34640/0 (socket says 172.21.15.47:34640) 2024-03-21T11:26:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f7898a00640 1 -- 172.21.15.47:0/993251177 learned_addr learned my addr 172.21.15.47:0/993251177 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f7899400640 1 --2- 172.21.15.47:0/993251177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f342e80 0x555a3f351ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f7899e00640 1 --2- 172.21.15.47:0/993251177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f3489e0 0x555a3f353020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f7898a00640 1 -- 172.21.15.47:0/993251177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f342e80 msgr2=0x555a3f351ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f7898a00640 1 --2- 172.21.15.47:0/993251177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f342e80 0x555a3f351ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f7898a00640 1 -- 172.21.15.47:0/993251177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f3489e0 msgr2=0x555a3f353020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:05.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f7898a00640 1 --2- 172.21.15.47:0/993251177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f3489e0 0x555a3f353020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f7898a00640 1 -- 172.21.15.47:0/993251177 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a3f353610 con 0x555a3f343870 2024-03-21T11:26:05.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.398+0000 7f7899400640 1 --2- 172.21.15.47:0/993251177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f342e80 0x555a3f351ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:05.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.399+0000 7f7898a00640 1 --2- 172.21.15.47:0/993251177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f343870 0x555a3f352010 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f78800a42b0 tx=0x7f78800cd140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e43593c0e21bf35d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:05.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.399+0000 7f787aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:05.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.399+0000 7f787aa00640 1 -- 172.21.15.47:0/993251177 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788007e020 con 0x555a3f343870 2024-03-21T11:26:05.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.399+0000 7f787aa00640 1 -- 172.21.15.47:0/993251177 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f788009b2d0 con 0x555a3f343870 2024-03-21T11:26:05.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.399+0000 7f789ebb2740 1 -- 172.21.15.47:0/993251177 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78800cd2b0 con 0x555a3f343870 2024-03-21T11:26:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.399+0000 7f787aa00640 1 -- 172.21.15.47:0/993251177 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f788009b4d0 con 0x555a3f343870 2024-03-21T11:26:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.399+0000 7f789ebb2740 10 client.26380 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.399+0000 7f789ebb2740 1 -- 172.21.15.47:0/993251177 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555a3f3b4560 con 0x555a3f343870 2024-03-21T11:26:05.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.400+0000 7f787aa00640 1 -- 172.21.15.47:0/993251177 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78800cd2b0 con 0x555a3f343870 2024-03-21T11:26:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.400+0000 7f787aa00640 1 -- 172.21.15.47:0/993251177 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7880097bd0 con 0x555a3f343870 2024-03-21T11:26:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.400+0000 7f789ebb2740 10 client.26380 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.400+0000 7f789ebb2740 10 client.26380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.400+0000 7f789ebb2740 20 client.26380 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.400+0000 7f789ebb2740 1 --2- 172.21.15.47:0/993251177 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555a3f3b4d30 0x555a3f3b7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.400+0000 7f789ebb2740 4 client.26380 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:26:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.400+0000 7f789ebb2740 1 -- 172.21.15.47:0/993251177 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555a3f2afae0 con 0x555a3f3b4d30 2024-03-21T11:26:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.401+0000 7f7899400640 1 --2- 172.21.15.47:0/993251177 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555a3f3b4d30 0x555a3f3b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.401+0000 7f7899400640 1 --2- 172.21.15.47:0/993251177 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555a3f3b4d30 0x555a3f3b7150 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfcb9978a06619e7 server_cookie=ad2ab8736abf2ab4 in_seq=0 out_seq=0 2024-03-21T11:26:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.401+0000 7f787aa00640 10 client.26380 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.403+0000 7f787aa00640 1 -- 172.21.15.47:0/993251177 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555a3f2afae0 con 0x555a3f3b4d30 2024-03-21T11:26:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.403+0000 7f787aa00640 10 client.26380 handle_command_reply: tid=0 2024-03-21T11:26:05.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.403+0000 7f789ebb2740 1 client.26380 shutdown 2024-03-21T11:26:05.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.403+0000 7f789ebb2740 1 -- 172.21.15.47:0/993251177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f343870 msgr2=0x555a3f352010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.403+0000 7f789ebb2740 1 --2- 172.21.15.47:0/993251177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f343870 0x555a3f352010 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f78800a42b0 tx=0x7f78800cd140 comp rx=0 tx=0).stop 2024-03-21T11:26:05.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.403+0000 7f789ebb2740 1 -- 172.21.15.47:0/993251177 shutdown_connections 2024-03-21T11:26:05.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.403+0000 7f789ebb2740 1 --2- 172.21.15.47:0/993251177 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555a3f3b4d30 0x555a3f3b7150 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.403+0000 7f789ebb2740 1 --2- 172.21.15.47:0/993251177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a3f3489e0 0x555a3f353020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.403+0000 7f789ebb2740 1 --2- 172.21.15.47:0/993251177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a3f343870 0x555a3f352010 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.403+0000 7f789ebb2740 1 --2- 172.21.15.47:0/993251177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a3f342e80 0x555a3f351ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.403+0000 7f787aa00640 0 client.26380 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:05.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.403+0000 7f789ebb2740 1 -- 172.21.15.47:0/993251177 >> 172.21.15.47:0/993251177 conn(0x555a3f30b200 msgr2=0x555a3f34d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:05.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.403+0000 7f789ebb2740 1 -- 172.21.15.47:0/993251177 shutdown_connections 2024-03-21T11:26:05.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.403+0000 7f789ebb2740 1 -- 172.21.15.47:0/993251177 wait complete. 2024-03-21T11:26:05.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.404+0000 7f789ebb2740 20 client.26380 trim_cache size 0 max 16384 2024-03-21T11:26:05.407 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:26:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:26:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:26:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: "52171052-e3c4-4c71-ac4e-f76bec62c9d0": { 2024-03-21T11:26:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:26:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "52171052-e3c4-4c71-ac4e-f76bec62c9d0", 2024-03-21T11:26:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:26:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:26:05.408 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:26:05.408 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:26:05.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.404+0000 7f789be00640 1 -- 172.21.15.47:0/1167606814 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f787c0a2fa0 msgr2=0x7f787c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.404+0000 7f789be00640 1 --2- 172.21.15.47:0/1167606814 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f787c0a2fa0 0x7f787c0a5460 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f788802c900 tx=0x7f7888000bc0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.404+0000 7f789be00640 1 -- 172.21.15.47:0/1167606814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78941017b0 msgr2=0x7f7894102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:05.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.404+0000 7f789be00640 1 --2- 172.21.15.47:0/1167606814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78941017b0 0x7f7894102b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f788000e9f0 tx=0x7f788000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.404+0000 7f789be00640 1 -- 172.21.15.47:0/1167606814 shutdown_connections 2024-03-21T11:26:05.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.404+0000 7f789be00640 1 --2- 172.21.15.47:0/1167606814 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f787c0a2fa0 0x7f787c0a5460 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.404+0000 7f789be00640 1 --2- 172.21.15.47:0/1167606814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7894102170 0x7f78941030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.404+0000 7f789be00640 1 --2- 172.21.15.47:0/1167606814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78941017b0 0x7f7894102b90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.404+0000 7f789be00640 1 --2- 172.21.15.47:0/1167606814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78941005c0 0x7f78941044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:05.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.404+0000 7f789be00640 1 -- 172.21.15.47:0/1167606814 >> 172.21.15.47:0/1167606814 conn(0x7f78940fbb50 msgr2=0x7f78940fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:05.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.405+0000 7f789be00640 1 -- 172.21.15.47:0/1167606814 shutdown_connections 2024-03-21T11:26:05.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:05.405+0000 7f789be00640 1 -- 172.21.15.47:0/1167606814 wait complete. 2024-03-21T11:26:05.424 INFO:tasks.cephfs.filesystem:scrub status for tag:52171052-e3c4-4c71-ac4e-f76bec62c9d0 - {'path': '/', 'tag': '52171052-e3c4-4c71-ac4e-f76bec62c9d0', 'options': 'recursive,force'} 2024-03-21T11:26:05.492 INFO:tasks.workunit.client.0.smithi047.stdout: 1 81842 9.40 MB/sec warmup 49 sec latency 13.495 ms 2024-03-21T11:26:05.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:05 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:26:05.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:05 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:26:05.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:05 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:26:05.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:05 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:26:05.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:05 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:26:05.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:05 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:26:05.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:05 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:26:05.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:05 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:26:05.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:05 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:26:05.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:05 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:26:05.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:05 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:26:05.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:05 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:26:06.492 INFO:tasks.workunit.client.0.smithi047.stdout: 1 82743 9.26 MB/sec warmup 50 sec latency 17.959 ms 2024-03-21T11:26:06.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:06 smithi102 ceph-mon[36748]: pgmap v483: 129 pgs: 129 active+clean; 37 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 253 op/s 2024-03-21T11:26:06.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:06 smithi047 ceph-mon[31351]: pgmap v483: 129 pgs: 129 active+clean; 37 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 253 op/s 2024-03-21T11:26:06.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:06 smithi100 ceph-mon[36245]: pgmap v483: 129 pgs: 129 active+clean; 37 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 253 op/s 2024-03-21T11:26:07.492 INFO:tasks.workunit.client.0.smithi047.stdout: 1 85184 9.38 MB/sec warmup 51 sec latency 10.874 ms 2024-03-21T11:26:07.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:07 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:26:07.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:07 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:26:07.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:07 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:26:08.493 INFO:tasks.workunit.client.0.smithi047.stdout: 1 86156 9.25 MB/sec warmup 52 sec latency 34.202 ms 2024-03-21T11:26:08.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:08 smithi102 ceph-mon[36748]: pgmap v484: 129 pgs: 129 active+clean; 35 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 335 op/s 2024-03-21T11:26:08.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:08 smithi047 ceph-mon[31351]: pgmap v484: 129 pgs: 129 active+clean; 35 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 335 op/s 2024-03-21T11:26:08.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:08 smithi100 ceph-mon[36245]: pgmap v484: 129 pgs: 129 active+clean; 35 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 335 op/s 2024-03-21T11:26:09.493 INFO:tasks.workunit.client.0.smithi047.stdout: 1 88372 9.31 MB/sec warmup 53 sec latency 64.013 ms 2024-03-21T11:26:10.493 INFO:tasks.workunit.client.0.smithi047.stdout: 1 89609 9.25 MB/sec warmup 54 sec latency 9.244 ms 2024-03-21T11:26:10.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:10 smithi102 ceph-mon[36748]: pgmap v485: 129 pgs: 129 active+clean; 34 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 305 op/s 2024-03-21T11:26:10.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:10 smithi047 ceph-mon[31351]: pgmap v485: 129 pgs: 129 active+clean; 34 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 305 op/s 2024-03-21T11:26:10.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:10 smithi100 ceph-mon[36245]: pgmap v485: 129 pgs: 129 active+clean; 34 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 305 op/s 2024-03-21T11:26:11.493 INFO:tasks.workunit.client.0.smithi047.stdout: 1 91927 9.32 MB/sec warmup 55 sec latency 10.638 ms 2024-03-21T11:26:12.494 INFO:tasks.workunit.client.0.smithi047.stdout: 1 93089 9.26 MB/sec warmup 56 sec latency 20.910 ms 2024-03-21T11:26:12.694 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:12 smithi100 ceph-mon[36245]: pgmap v486: 129 pgs: 129 active+clean; 37 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-03-21T11:26:12.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:12 smithi102 ceph-mon[36748]: pgmap v486: 129 pgs: 129 active+clean; 37 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-03-21T11:26:12.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:12 smithi047 ceph-mon[31351]: pgmap v486: 129 pgs: 129 active+clean; 37 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-03-21T11:26:13.494 INFO:tasks.workunit.client.0.smithi047.stdout: 1 95267 9.31 MB/sec warmup 57 sec latency 17.742 ms 2024-03-21T11:26:14.494 INFO:tasks.workunit.client.0.smithi047.stdout: 1 96405 9.26 MB/sec warmup 58 sec latency 62.398 ms 2024-03-21T11:26:14.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:14 smithi102 ceph-mon[36748]: pgmap v487: 129 pgs: 129 active+clean; 37 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 325 op/s 2024-03-21T11:26:14.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:14 smithi047 ceph-mon[31351]: pgmap v487: 129 pgs: 129 active+clean; 37 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 325 op/s 2024-03-21T11:26:14.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:14 smithi100 ceph-mon[36245]: pgmap v487: 129 pgs: 129 active+clean; 37 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 325 op/s 2024-03-21T11:26:15.494 INFO:tasks.workunit.client.0.smithi047.stdout: 1 98295 9.24 MB/sec warmup 59 sec latency 10.616 ms 2024-03-21T11:26:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.698+0000 7ff20bc00640 1 -- 172.21.15.47:0/1850679716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204102150 msgr2=0x7ff204104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.698+0000 7ff20bc00640 1 --2- 172.21.15.47:0/1850679716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204102150 0x7ff204104540 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009ca0 tx=0x7ff1f402c100 comp rx=0 tx=0).stop 2024-03-21T11:26:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.698+0000 7ff20bc00640 1 -- 172.21.15.47:0/1850679716 shutdown_connections 2024-03-21T11:26:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.698+0000 7ff20bc00640 1 --2- 172.21.15.47:0/1850679716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2041073b0 0x7ff204109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.698+0000 7ff20bc00640 1 --2- 172.21.15.47:0/1850679716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff204104a80 0x7ff204106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.698+0000 7ff20bc00640 1 --2- 172.21.15.47:0/1850679716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204102150 0x7ff204104540 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.698+0000 7ff20bc00640 1 -- 172.21.15.47:0/1850679716 >> 172.21.15.47:0/1850679716 conn(0x7ff2040fbb60 msgr2=0x7ff2040fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.698+0000 7ff20bc00640 1 -- 172.21.15.47:0/1850679716 shutdown_connections 2024-03-21T11:26:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.698+0000 7ff20bc00640 1 -- 172.21.15.47:0/1850679716 wait complete. 2024-03-21T11:26:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.699+0000 7ff20bc00640 1 Processor -- start 2024-03-21T11:26:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.699+0000 7ff20bc00640 1 -- start start 2024-03-21T11:26:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.699+0000 7ff20bc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204102150 0x7ff204101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.699+0000 7ff20bc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff204104a80 0x7ff204100100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.699+0000 7ff20bc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2041073b0 0x7ff204100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.699+0000 7ff20bc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20410d380 con 0x7ff2041073b0 2024-03-21T11:26:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.699+0000 7ff20bc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20410d200 con 0x7ff204104a80 2024-03-21T11:26:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.699+0000 7ff20bc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20410d500 con 0x7ff204102150 2024-03-21T11:26:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.700+0000 7ff209200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204102150 0x7ff204101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.700+0000 7ff1fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff204104a80 0x7ff204100100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.700+0000 7ff209200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204102150 0x7ff204101ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50654/0 (socket says 172.21.15.47:50654) 2024-03-21T11:26:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.700+0000 7ff209200640 1 -- 172.21.15.47:0/1619015319 learned_addr learned my addr 172.21.15.47:0/1619015319 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.700+0000 7ff209c00640 1 --2- 172.21.15.47:0/1619015319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2041073b0 0x7ff204100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.700+0000 7ff209200640 1 -- 172.21.15.47:0/1619015319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff204104a80 msgr2=0x7ff204100100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.700+0000 7ff209200640 1 --2- 172.21.15.47:0/1619015319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff204104a80 0x7ff204100100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.700+0000 7ff209200640 1 -- 172.21.15.47:0/1619015319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2041073b0 msgr2=0x7ff204100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.700+0000 7ff209200640 1 --2- 172.21.15.47:0/1619015319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2041073b0 0x7ff204100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.700+0000 7ff209200640 1 -- 172.21.15.47:0/1619015319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff204100b80 con 0x7ff204102150 2024-03-21T11:26:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.700+0000 7ff209200640 1 --2- 172.21.15.47:0/1619015319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204102150 0x7ff204101ac0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009dc0 tx=0x7ff1f4035d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.701+0000 7ff202000640 1 -- 172.21.15.47:0/1619015319 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4039040 con 0x7ff204102150 2024-03-21T11:26:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.701+0000 7ff202000640 1 -- 172.21.15.47:0/1619015319 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1f4044070 con 0x7ff204102150 2024-03-21T11:26:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.701+0000 7ff202000640 1 -- 172.21.15.47:0/1619015319 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f40044a0 con 0x7ff204102150 2024-03-21T11:26:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.701+0000 7ff20bc00640 1 -- 172.21.15.47:0/1619015319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff204100e10 con 0x7ff204102150 2024-03-21T11:26:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.701+0000 7ff20bc00640 1 -- 172.21.15.47:0/1619015319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2041a6370 con 0x7ff204102150 2024-03-21T11:26:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff20e9c7740 1 Processor -- start 2024-03-21T11:26:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff20e9c7740 1 -- start start 2024-03-21T11:26:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff202000640 1 -- 172.21.15.47:0/1619015319 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff1f4004640 con 0x7ff204102150 2024-03-21T11:26:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff20e9c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e61e0 0x55c83b3e6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff20e9c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e6bd0 0x55c83b3eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff20e9c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3ebd40 0x55c83b3ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff20e9c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c83b3a93b0 con 0x55c83b3e6bd0 2024-03-21T11:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff20e9c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c83b34b830 con 0x55c83b3e61e0 2024-03-21T11:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff20e9c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c83b2f45d0 con 0x55c83b3ebd40 2024-03-21T11:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff209c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e61e0 0x55c83b3e6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff1fbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3ebd40 0x55c83b3ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff209200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e6bd0 0x55c83b3eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff1fbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3ebd40 0x55c83b3ee1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50666/0 (socket says 172.21.15.47:50666) 2024-03-21T11:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff1fbe00640 1 -- 172.21.15.47:0/1737915724 learned_addr learned my addr 172.21.15.47:0/1737915724 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.705+0000 7ff209c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e61e0 0x55c83b3e6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36880/0 (socket says 172.21.15.47:36880) 2024-03-21T11:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.706+0000 7ff1fbe00640 1 -- 172.21.15.47:0/1737915724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e61e0 msgr2=0x55c83b3e6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.706+0000 7ff1fbe00640 1 --2- 172.21.15.47:0/1737915724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e61e0 0x55c83b3e6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.706+0000 7ff1fbe00640 1 -- 172.21.15.47:0/1737915724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e6bd0 msgr2=0x55c83b3eb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.706+0000 7ff1fbe00640 1 --2- 172.21.15.47:0/1737915724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e6bd0 0x55c83b3eb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.706+0000 7ff1fbe00640 1 -- 172.21.15.47:0/1737915724 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c83b30c2c0 con 0x55c83b3ebd40 2024-03-21T11:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.706+0000 7ff209c00640 1 --2- 172.21.15.47:0/1737915724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e61e0 0x55c83b3e6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.706+0000 7ff1fbe00640 1 --2- 172.21.15.47:0/1737915724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3ebd40 0x55c83b3ee1d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009c60 tx=0x7ff1ec011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f7cdc077eb0d936 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.706+0000 7ff1fa000640 1 -- 172.21.15.47:0/1737915724 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec004230 con 0x55c83b3ebd40 2024-03-21T11:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.707+0000 7ff1fa000640 1 -- 172.21.15.47:0/1737915724 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1ec0043d0 con 0x55c83b3ebd40 2024-03-21T11:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.707+0000 7ff1fa000640 1 -- 172.21.15.47:0/1737915724 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec004dc0 con 0x55c83b3ebd40 2024-03-21T11:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.707+0000 7ff202000640 1 --2- 172.21.15.47:0/1619015319 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff1e80a3040 0x7ff1e80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.707+0000 7ff202000640 1 -- 172.21.15.47:0/1619015319 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff1f40f1bd0 con 0x7ff204102150 2024-03-21T11:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.707+0000 7ff209c00640 1 --2- 172.21.15.47:0/1619015319 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff1e80a3040 0x7ff1e80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.707+0000 7ff20e9c7740 1 -- 172.21.15.47:0/1737915724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3ebd40 msgr2=0x55c83b3ee1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.707+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/1737915724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3ebd40 0x55c83b3ee1d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009c60 tx=0x7ff1ec011810 comp rx=0 tx=0).stop 2024-03-21T11:26:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.708+0000 7ff20e9c7740 1 -- 172.21.15.47:0/1737915724 shutdown_connections 2024-03-21T11:26:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.708+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/1737915724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3ebd40 0x55c83b3ee1d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.708+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/1737915724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e6bd0 0x55c83b3eb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.708+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/1737915724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e61e0 0x55c83b3e6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.708+0000 7ff20e9c7740 1 -- 172.21.15.47:0/1737915724 >> 172.21.15.47:0/1737915724 conn(0x55c83b3e1750 msgr2=0x55c83b3e1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.708+0000 7ff20e9c7740 1 -- 172.21.15.47:0/1737915724 shutdown_connections 2024-03-21T11:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.708+0000 7ff209c00640 1 --2- 172.21.15.47:0/1619015319 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff1e80a3040 0x7ff1e80a5500 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55c83b3e7430 tx=0x7ff1fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.708+0000 7ff20e9c7740 1 -- 172.21.15.47:0/1737915724 wait complete. 2024-03-21T11:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.708+0000 7ff20e9c7740 1 -- start start 2024-03-21T11:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.708+0000 7ff20e9c7740 1 Processor -- start 2024-03-21T11:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.709+0000 7ff20e9c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e61e0 0x55c83b3f0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.709+0000 7ff20e9c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.709+0000 7ff20e9c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3ebd40 0x55c83b3f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.709+0000 7ff20e9c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c83b307a20 con 0x55c83b3e61e0 2024-03-21T11:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.709+0000 7ff20e9c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c83b3478d0 con 0x55c83b3e6bd0 2024-03-21T11:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.709+0000 7ff20e9c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c83b3eecf0 con 0x55c83b3ebd40 2024-03-21T11:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.709+0000 7ff20e9c7740 10 client.0 _ll_register_callbacks cb 0x55c83b3441c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.709+0000 7ff1fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.709+0000 7ff1fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36884/0 (socket says 172.21.15.47:36884) 2024-03-21T11:26:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.709+0000 7ff209200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e61e0 0x55c83b3f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.709+0000 7ff1fbe00640 1 -- 172.21.15.47:0/578050838 learned_addr learned my addr 172.21.15.47:0/578050838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.709+0000 7ff209c00640 1 --2- 172.21.15.47:0/578050838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3ebd40 0x55c83b3f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.710+0000 7ff1fbe00640 1 -- 172.21.15.47:0/578050838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3ebd40 msgr2=0x55c83b3f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.710+0000 7ff1fbe00640 1 --2- 172.21.15.47:0/578050838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3ebd40 0x55c83b3f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.710+0000 7ff1fbe00640 1 -- 172.21.15.47:0/578050838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e61e0 msgr2=0x55c83b3f0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.710+0000 7ff1fbe00640 1 --2- 172.21.15.47:0/578050838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e61e0 0x55c83b3f0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.710+0000 7ff1fbe00640 1 -- 172.21.15.47:0/578050838 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c83b060210 con 0x55c83b3e6bd0 2024-03-21T11:26:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.710+0000 7ff209c00640 1 --2- 172.21.15.47:0/578050838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3ebd40 0x55c83b3f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.710+0000 7ff209200640 1 --2- 172.21.15.47:0/578050838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e61e0 0x55c83b3f0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.710+0000 7ff1fbe00640 1 --2- 172.21.15.47:0/578050838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f08a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec000c00 tx=0x7ff1ec0051c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=740d6b9ae9f32b17 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.710+0000 7ff1f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.710+0000 7ff1f2a00640 1 -- 172.21.15.47:0/578050838 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec028020 con 0x55c83b3e6bd0 2024-03-21T11:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.710+0000 7ff1f2a00640 1 -- 172.21.15.47:0/578050838 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff1ec024420 con 0x55c83b3e6bd0 2024-03-21T11:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.710+0000 7ff20e9c7740 1 -- 172.21.15.47:0/578050838 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1ec0043d0 con 0x55c83b3e6bd0 2024-03-21T11:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.711+0000 7ff1f2a00640 1 -- 172.21.15.47:0/578050838 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1ec033050 con 0x55c83b3e6bd0 2024-03-21T11:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.711+0000 7ff20e9c7740 10 client.26375 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.711+0000 7ff20e9c7740 1 -- 172.21.15.47:0/578050838 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c83b12c620 con 0x55c83b3e6bd0 2024-03-21T11:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.711+0000 7ff1f2a00640 1 -- 172.21.15.47:0/578050838 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1ec0043d0 con 0x55c83b3e6bd0 2024-03-21T11:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.711+0000 7ff1f2a00640 1 -- 172.21.15.47:0/578050838 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff1ec018070 con 0x55c83b3e6bd0 2024-03-21T11:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.711+0000 7ff20e9c7740 10 client.26375 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.711+0000 7ff20e9c7740 10 client.26375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.711+0000 7ff20e9c7740 20 client.26375 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.711+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/578050838 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c83b3fbf70 0x55c83b3fe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.711+0000 7ff20e9c7740 4 client.26375 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.711+0000 7ff20e9c7740 1 -- 172.21.15.47:0/578050838 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c83b358d70 con 0x55c83b3fbf70 2024-03-21T11:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.712+0000 7ff209200640 1 --2- 172.21.15.47:0/578050838 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c83b3fbf70 0x55c83b3fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.712+0000 7ff209200640 1 --2- 172.21.15.47:0/578050838 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c83b3fbf70 0x55c83b3fe380 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=252dca34adb5fa1c server_cookie=de1c6abb3ed84105 in_seq=0 out_seq=0 2024-03-21T11:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.712+0000 7ff1f2a00640 10 client.26375 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff1f2a00640 1 -- 172.21.15.47:0/578050838 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c83b358d70 con 0x55c83b3fbf70 2024-03-21T11:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff1f2a00640 10 client.26375 handle_command_reply: tid=0 2024-03-21T11:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff20e9c7740 1 client.26375 shutdown 2024-03-21T11:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff20e9c7740 1 -- 172.21.15.47:0/578050838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e6bd0 msgr2=0x55c83b3f08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/578050838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f08a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec000c00 tx=0x7ff1ec0051c0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff20e9c7740 1 -- 172.21.15.47:0/578050838 shutdown_connections 2024-03-21T11:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/578050838 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c83b3fbf70 0x55c83b3fe380 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/578050838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3ebd40 0x55c83b3f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/578050838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f08a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff1f2a00640 0 client.26375 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/578050838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e61e0 0x55c83b3f0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff20e9c7740 1 -- 172.21.15.47:0/578050838 >> 172.21.15.47:0/578050838 conn(0x55c83b3e1750 msgr2=0x55c83b3e0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff20e9c7740 1 -- 172.21.15.47:0/578050838 shutdown_connections 2024-03-21T11:26:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff20e9c7740 1 -- 172.21.15.47:0/578050838 wait complete. 2024-03-21T11:26:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.713+0000 7ff20e9c7740 20 client.26375 trim_cache size 0 max 16384 2024-03-21T11:26:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.729+0000 7ff20e9c7740 1 Processor -- start 2024-03-21T11:26:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.729+0000 7ff20e9c7740 1 -- start start 2024-03-21T11:26:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff20e9c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3e61e0 0x55c83b3ee130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff20e9c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff20e9c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3ebd40 0x55c83b3e1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff20e9c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2041b0590 con 0x55c83b3ebd40 2024-03-21T11:26:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff20e9c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2041b1f10 con 0x55c83b3e6bd0 2024-03-21T11:26:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff20e9c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2041b0bf0 con 0x55c83b3e61e0 2024-03-21T11:26:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff209200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3e61e0 0x55c83b3ee130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff209200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3e61e0 0x55c83b3ee130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50684/0 (socket says 172.21.15.47:50684) 2024-03-21T11:26:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff209200640 1 -- 172.21.15.47:0/3060771920 learned_addr learned my addr 172.21.15.47:0/3060771920 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff1fbe00640 1 --2- 172.21.15.47:0/3060771920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff209c00640 1 --2- 172.21.15.47:0/3060771920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3ebd40 0x55c83b3e1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff209200640 1 -- 172.21.15.47:0/3060771920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e6bd0 msgr2=0x55c83b3f78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff209200640 1 --2- 172.21.15.47:0/3060771920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff209200640 1 -- 172.21.15.47:0/3060771920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3ebd40 msgr2=0x55c83b3e1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff209200640 1 --2- 172.21.15.47:0/3060771920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3ebd40 0x55c83b3e1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff209200640 1 -- 172.21.15.47:0/3060771920 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c83b3ef420 con 0x55c83b3e61e0 2024-03-21T11:26:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.730+0000 7ff209c00640 1 --2- 172.21.15.47:0/3060771920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3ebd40 0x55c83b3e1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.731+0000 7ff209200640 1 --2- 172.21.15.47:0/3060771920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3e61e0 0x55c83b3ee130 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4004af0 tx=0x7ff1f40ef1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a95c8ab44fbceef1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.731+0000 7ff1f9600640 1 -- 172.21.15.47:0/3060771920 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f40bdce0 con 0x55c83b3e61e0 2024-03-21T11:26:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.731+0000 7ff1f9600640 1 -- 172.21.15.47:0/3060771920 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1f40be610 con 0x55c83b3e61e0 2024-03-21T11:26:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.731+0000 7ff1f9600640 1 -- 172.21.15.47:0/3060771920 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f40bd870 con 0x55c83b3e61e0 2024-03-21T11:26:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.732+0000 7ff20e9c7740 1 -- 172.21.15.47:0/3060771920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3e61e0 msgr2=0x55c83b3ee130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.732+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/3060771920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3e61e0 0x55c83b3ee130 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4004af0 tx=0x7ff1f40ef1b0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.732+0000 7ff20e9c7740 1 -- 172.21.15.47:0/3060771920 shutdown_connections 2024-03-21T11:26:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.732+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/3060771920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3ebd40 0x55c83b3e1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.732+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/3060771920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.732+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/3060771920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3e61e0 0x55c83b3ee130 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.732+0000 7ff20e9c7740 1 -- 172.21.15.47:0/3060771920 >> 172.21.15.47:0/3060771920 conn(0x55c83b3e1750 msgr2=0x55c83b3eb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.732+0000 7ff20e9c7740 1 -- 172.21.15.47:0/3060771920 shutdown_connections 2024-03-21T11:26:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.732+0000 7ff20e9c7740 1 -- 172.21.15.47:0/3060771920 wait complete. 2024-03-21T11:26:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.732+0000 7ff20e9c7740 1 -- start start 2024-03-21T11:26:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.733+0000 7ff20e9c7740 1 Processor -- start 2024-03-21T11:26:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.733+0000 7ff20e9c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e61e0 0x55c83b3f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.733+0000 7ff20e9c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.733+0000 7ff20e9c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3ebd40 0x55c83b455840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.733+0000 7ff20e9c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2041b18b0 con 0x55c83b3e61e0 2024-03-21T11:26:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.733+0000 7ff20e9c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2041b1250 con 0x55c83b3ebd40 2024-03-21T11:26:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.733+0000 7ff20e9c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2041ae750 con 0x55c83b3e6bd0 2024-03-21T11:26:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.733+0000 7ff20e9c7740 10 client.0 _ll_register_callbacks cb 0x7ff1f40f54e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.733+0000 7ff1fbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.733+0000 7ff209c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3ebd40 0x55c83b455840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.733+0000 7ff1fbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50696/0 (socket says 172.21.15.47:50696) 2024-03-21T11:26:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.733+0000 7ff1fbe00640 1 -- 172.21.15.47:0/3195183402 learned_addr learned my addr 172.21.15.47:0/3195183402 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.733+0000 7ff209c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3ebd40 0x55c83b455840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36902/0 (socket says 172.21.15.47:36902) 2024-03-21T11:26:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.733+0000 7ff209200640 1 --2- 172.21.15.47:0/3195183402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e61e0 0x55c83b3f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.734+0000 7ff209c00640 1 -- 172.21.15.47:0/3195183402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3e6bd0 msgr2=0x55c83b3f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.734+0000 7ff209c00640 1 --2- 172.21.15.47:0/3195183402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.734+0000 7ff209c00640 1 -- 172.21.15.47:0/3195183402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e61e0 msgr2=0x55c83b3f47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.734+0000 7ff209c00640 1 --2- 172.21.15.47:0/3195183402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e61e0 0x55c83b3f47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.734+0000 7ff209c00640 1 -- 172.21.15.47:0/3195183402 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c83b455d80 con 0x55c83b3ebd40 2024-03-21T11:26:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.734+0000 7ff209c00640 1 --2- 172.21.15.47:0/3195183402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3ebd40 0x55c83b455840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009ca0 tx=0x7ff1fc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60348c7d33163e0f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.734+0000 7ff1f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.734+0000 7ff1f2a00640 1 -- 172.21.15.47:0/3195183402 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc004320 con 0x55c83b3ebd40 2024-03-21T11:26:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.734+0000 7ff20e9c7740 1 -- 172.21.15.47:0/3195183402 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1f40be610 con 0x55c83b3ebd40 2024-03-21T11:26:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.734+0000 7ff1f2a00640 1 -- 172.21.15.47:0/3195183402 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff1fc0035f0 con 0x55c83b3ebd40 2024-03-21T11:26:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.735+0000 7ff1f2a00640 1 -- 172.21.15.47:0/3195183402 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1fc0044c0 con 0x55c83b3ebd40 2024-03-21T11:26:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.735+0000 7ff20e9c7740 10 client.26381 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.735+0000 7ff20e9c7740 1 -- 172.21.15.47:0/3195183402 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c83b4570f0 con 0x55c83b3ebd40 2024-03-21T11:26:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.735+0000 7ff1f2a00640 1 -- 172.21.15.47:0/3195183402 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1f40be610 con 0x55c83b3ebd40 2024-03-21T11:26:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.735+0000 7ff1f2a00640 1 -- 172.21.15.47:0/3195183402 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff1fc01ebd0 con 0x55c83b3ebd40 2024-03-21T11:26:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.735+0000 7ff20e9c7740 10 client.26381 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.735+0000 7ff20e9c7740 10 client.26381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.735+0000 7ff20e9c7740 20 client.26381 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.735+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/3195183402 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c83b457ad0 0x55c83b459ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.735+0000 7ff20e9c7740 4 client.26381 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:26:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.735+0000 7ff20e9c7740 1 -- 172.21.15.47:0/3195183402 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c83b3530c0 con 0x55c83b457ad0 2024-03-21T11:26:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.736+0000 7ff209200640 1 --2- 172.21.15.47:0/3195183402 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c83b457ad0 0x55c83b459ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.736+0000 7ff209200640 1 --2- 172.21.15.47:0/3195183402 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c83b457ad0 0x55c83b459ef0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2443548befbabcf7 server_cookie=c53f5bc6f04ab757 in_seq=0 out_seq=0 2024-03-21T11:26:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.736+0000 7ff1f2a00640 10 client.26381 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.937+0000 7ff1f2a00640 1 -- 172.21.15.47:0/3195183402 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c83b3530c0 con 0x55c83b457ad0 2024-03-21T11:26:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.937+0000 7ff1f2a00640 10 client.26381 handle_command_reply: tid=0 2024-03-21T11:26:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.937+0000 7ff20e9c7740 1 client.26381 shutdown 2024-03-21T11:26:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.939+0000 7ff20e9c7740 1 -- 172.21.15.47:0/3195183402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3ebd40 msgr2=0x55c83b455840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.939+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/3195183402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3ebd40 0x55c83b455840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009ca0 tx=0x7ff1fc0046c0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.939+0000 7ff20e9c7740 1 -- 172.21.15.47:0/3195183402 shutdown_connections 2024-03-21T11:26:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.939+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/3195183402 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c83b457ad0 0x55c83b459ef0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.939+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/3195183402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83b3ebd40 0x55c83b455840 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.939+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/3195183402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c83b3e6bd0 0x55c83b3f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.939+0000 7ff20e9c7740 1 --2- 172.21.15.47:0/3195183402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c83b3e61e0 0x55c83b3f47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.939+0000 7ff20e9c7740 1 -- 172.21.15.47:0/3195183402 >> 172.21.15.47:0/3195183402 conn(0x55c83b3e1750 msgr2=0x55c83b3eb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.939+0000 7ff1f2a00640 0 client.26381 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.939+0000 7ff20e9c7740 1 -- 172.21.15.47:0/3195183402 shutdown_connections 2024-03-21T11:26:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.939+0000 7ff20e9c7740 1 -- 172.21.15.47:0/3195183402 wait complete. 2024-03-21T11:26:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.939+0000 7ff20e9c7740 20 client.26381 trim_cache size 0 max 16384 2024-03-21T11:26:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:26:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:26:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:26:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:26:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.940+0000 7ff20bc00640 1 -- 172.21.15.47:0/1619015319 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff1e80a3040 msgr2=0x7ff1e80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.940+0000 7ff20bc00640 1 --2- 172.21.15.47:0/1619015319 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff1e80a3040 0x7ff1e80a5500 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55c83b3e7430 tx=0x7ff1fc005f90 comp rx=0 tx=0).stop 2024-03-21T11:26:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.940+0000 7ff20bc00640 1 -- 172.21.15.47:0/1619015319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204102150 msgr2=0x7ff204101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.940+0000 7ff20bc00640 1 --2- 172.21.15.47:0/1619015319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204102150 0x7ff204101ac0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009dc0 tx=0x7ff1f4035d90 comp rx=0 tx=0).stop 2024-03-21T11:26:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.940+0000 7ff20bc00640 1 -- 172.21.15.47:0/1619015319 shutdown_connections 2024-03-21T11:26:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.940+0000 7ff20bc00640 1 --2- 172.21.15.47:0/1619015319 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff1e80a3040 0x7ff1e80a5500 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.940+0000 7ff20bc00640 1 --2- 172.21.15.47:0/1619015319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2041073b0 0x7ff204100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.940+0000 7ff20bc00640 1 --2- 172.21.15.47:0/1619015319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff204104a80 0x7ff204100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.940+0000 7ff20bc00640 1 --2- 172.21.15.47:0/1619015319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff204102150 0x7ff204101ac0 secure :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009dc0 tx=0x7ff1f4035d90 comp rx=0 tx=0).stop 2024-03-21T11:26:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.940+0000 7ff20bc00640 1 -- 172.21.15.47:0/1619015319 >> 172.21.15.47:0/1619015319 conn(0x7ff2040fbb60 msgr2=0x7ff204102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.941+0000 7ff20bc00640 1 -- 172.21.15.47:0/1619015319 shutdown_connections 2024-03-21T11:26:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:15.941+0000 7ff20bc00640 1 -- 172.21.15.47:0/1619015319 wait complete. 2024-03-21T11:26:16.494 INFO:tasks.workunit.client.0.smithi047.stdout: 1 99811 9.27 MB/sec warmup 60 sec latency 75.167 ms 2024-03-21T11:26:16.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:16 smithi102 ceph-mon[36748]: pgmap v488: 129 pgs: 129 active+clean; 37 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-03-21T11:26:16.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:16 smithi047 ceph-mon[31351]: pgmap v488: 129 pgs: 129 active+clean; 37 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-03-21T11:26:16.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:16 smithi100 ceph-mon[36245]: pgmap v488: 129 pgs: 129 active+clean; 37 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-03-21T11:26:17.494 INFO:tasks.workunit.client.0.smithi047.stdout: 1 101340 9.21 MB/sec warmup 61 sec latency 17.092 ms 2024-03-21T11:26:18.494 INFO:tasks.workunit.client.0.smithi047.stdout: 1 103216 9.28 MB/sec warmup 62 sec latency 15.091 ms 2024-03-21T11:26:18.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:18 smithi102 ceph-mon[36748]: pgmap v489: 129 pgs: 129 active+clean; 35 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-03-21T11:26:18.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:18 smithi047 ceph-mon[31351]: pgmap v489: 129 pgs: 129 active+clean; 35 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-03-21T11:26:18.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:18 smithi100 ceph-mon[36245]: pgmap v489: 129 pgs: 129 active+clean; 35 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-03-21T11:26:19.494 INFO:tasks.workunit.client.0.smithi047.stdout: 1 104335 9.22 MB/sec warmup 63 sec latency 33.399 ms 2024-03-21T11:26:20.495 INFO:tasks.workunit.client.0.smithi047.stdout: 1 106560 9.27 MB/sec warmup 64 sec latency 15.591 ms 2024-03-21T11:26:20.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:20 smithi102 ceph-mon[36748]: pgmap v490: 129 pgs: 129 active+clean; 35 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 293 op/s 2024-03-21T11:26:20.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:20 smithi047 ceph-mon[31351]: pgmap v490: 129 pgs: 129 active+clean; 35 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 293 op/s 2024-03-21T11:26:20.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:20 smithi100 ceph-mon[36245]: pgmap v490: 129 pgs: 129 active+clean; 35 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 293 op/s 2024-03-21T11:26:21.495 INFO:tasks.workunit.client.0.smithi047.stdout: 1 107518 9.18 MB/sec warmup 65 sec latency 80.109 ms 2024-03-21T11:26:22.495 INFO:tasks.workunit.client.0.smithi047.stdout: 1 109991 9.27 MB/sec warmup 66 sec latency 13.442 ms 2024-03-21T11:26:22.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:22 smithi102 ceph-mon[36748]: pgmap v491: 129 pgs: 129 active+clean; 34 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 349 op/s 2024-03-21T11:26:22.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:22 smithi047 ceph-mon[31351]: pgmap v491: 129 pgs: 129 active+clean; 34 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 349 op/s 2024-03-21T11:26:22.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:22 smithi100 ceph-mon[36245]: pgmap v491: 129 pgs: 129 active+clean; 34 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 349 op/s 2024-03-21T11:26:23.495 INFO:tasks.workunit.client.0.smithi047.stdout: 1 111126 9.19 MB/sec warmup 67 sec latency 10.419 ms 2024-03-21T11:26:24.495 INFO:tasks.workunit.client.0.smithi047.stdout: 1 113265 9.24 MB/sec warmup 68 sec latency 50.892 ms 2024-03-21T11:26:24.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:24 smithi102 ceph-mon[36748]: pgmap v492: 129 pgs: 129 active+clean; 35 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 349 op/s 2024-03-21T11:26:24.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:24 smithi047 ceph-mon[31351]: pgmap v492: 129 pgs: 129 active+clean; 35 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 349 op/s 2024-03-21T11:26:24.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:24 smithi100 ceph-mon[36245]: pgmap v492: 129 pgs: 129 active+clean; 35 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 349 op/s 2024-03-21T11:26:25.495 INFO:tasks.workunit.client.0.smithi047.stdout: 1 114418 9.18 MB/sec warmup 69 sec latency 17.045 ms 2024-03-21T11:26:26.495 INFO:tasks.workunit.client.0.smithi047.stdout: 1 116716 9.23 MB/sec warmup 70 sec latency 10.335 ms 2024-03-21T11:26:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:26 smithi102 ceph-mon[36748]: pgmap v493: 129 pgs: 129 active+clean; 34 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 323 op/s 2024-03-21T11:26:26.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:26 smithi047 ceph-mon[31351]: pgmap v493: 129 pgs: 129 active+clean; 34 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 323 op/s 2024-03-21T11:26:26.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:26 smithi100 ceph-mon[36245]: pgmap v493: 129 pgs: 129 active+clean; 34 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 323 op/s 2024-03-21T11:26:27.495 INFO:tasks.workunit.client.0.smithi047.stdout: 1 117897 9.19 MB/sec warmup 71 sec latency 9.690 ms 2024-03-21T11:26:28.496 INFO:tasks.workunit.client.0.smithi047.stdout: 1 120082 9.23 MB/sec warmup 72 sec latency 11.541 ms 2024-03-21T11:26:28.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:28 smithi102 ceph-mon[36748]: pgmap v494: 129 pgs: 129 active+clean; 35 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 337 op/s 2024-03-21T11:26:28.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:28 smithi047 ceph-mon[31351]: pgmap v494: 129 pgs: 129 active+clean; 35 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 337 op/s 2024-03-21T11:26:28.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:28 smithi100 ceph-mon[36245]: pgmap v494: 129 pgs: 129 active+clean; 35 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 337 op/s 2024-03-21T11:26:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.102+0000 7f6be4800640 1 -- 172.21.15.47:0/473297510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc1054b0 msgr2=0x7f6bdc10bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.102+0000 7f6be4800640 1 --2- 172.21.15.47:0/473297510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc1054b0 0x7f6bdc10bd40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009ca0 tx=0x7f6bd802c100 comp rx=0 tx=0).stop 2024-03-21T11:26:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.102+0000 7f6be4800640 1 -- 172.21.15.47:0/473297510 shutdown_connections 2024-03-21T11:26:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.102+0000 7f6be4800640 1 --2- 172.21.15.47:0/473297510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc1054b0 0x7f6bdc10bd40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.102+0000 7f6be4800640 1 --2- 172.21.15.47:0/473297510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc104af0 0x7f6bdc104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.102+0000 7f6be4800640 1 --2- 172.21.15.47:0/473297510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc103900 0x7f6bdc103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.102+0000 7f6be4800640 1 -- 172.21.15.47:0/473297510 >> 172.21.15.47:0/473297510 conn(0x7f6bdc0fee90 msgr2=0x7f6bdc1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.102+0000 7f6be4800640 1 -- 172.21.15.47:0/473297510 shutdown_connections 2024-03-21T11:26:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.102+0000 7f6be4800640 1 -- 172.21.15.47:0/473297510 wait complete. 2024-03-21T11:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.103+0000 7f6be4800640 1 Processor -- start 2024-03-21T11:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.103+0000 7f6be4800640 1 -- start start 2024-03-21T11:26:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.103+0000 7f6be4800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc103900 0x7f6bdc199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.103+0000 7f6be4800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc104af0 0x7f6bdc199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be4800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc1054b0 0x7f6bdc1a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be4800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc10f560 con 0x7f6bdc104af0 2024-03-21T11:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be4800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc10f3e0 con 0x7f6bdc1054b0 2024-03-21T11:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be4800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc10f6e0 con 0x7f6bdc103900 2024-03-21T11:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc103900 0x7f6bdc199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc103900 0x7f6bdc199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53838/0 (socket says 172.21.15.47:53838) 2024-03-21T11:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be1e00640 1 -- 172.21.15.47:0/3778777416 learned_addr learned my addr 172.21.15.47:0/3778777416 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be2800640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc1054b0 0x7f6bdc1a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be1400640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc104af0 0x7f6bdc199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be1e00640 1 -- 172.21.15.47:0/3778777416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc1054b0 msgr2=0x7f6bdc1a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be1e00640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc1054b0 0x7f6bdc1a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be1e00640 1 -- 172.21.15.47:0/3778777416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc104af0 msgr2=0x7f6bdc199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be1e00640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc104af0 0x7f6bdc199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be1e00640 1 -- 172.21.15.47:0/3778777416 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc1a16d0 con 0x7f6bdc103900 2024-03-21T11:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.104+0000 7f6be2800640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc1054b0 0x7f6bdc1a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.105+0000 7f6be1e00640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc103900 0x7f6bdc199ab0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000e9c0 tx=0x7f6bd000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.105+0000 7f6bd6a00640 1 -- 172.21.15.47:0/3778777416 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd000cd80 con 0x7f6bdc103900 2024-03-21T11:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.105+0000 7f6bd6a00640 1 -- 172.21.15.47:0/3778777416 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bd00035f0 con 0x7f6bdc103900 2024-03-21T11:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.105+0000 7f6be4800640 1 -- 172.21.15.47:0/3778777416 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bdc1a19c0 con 0x7f6bdc103900 2024-03-21T11:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.105+0000 7f6bd6a00640 1 -- 172.21.15.47:0/3778777416 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0010690 con 0x7f6bdc103900 2024-03-21T11:26:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.105+0000 7f6be4800640 1 -- 172.21.15.47:0/3778777416 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bdc1a1f10 con 0x7f6bdc103900 2024-03-21T11:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.109+0000 7f6bd6a00640 1 -- 172.21.15.47:0/3778777416 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6bd00037a0 con 0x7f6bdc103900 2024-03-21T11:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.109+0000 7f6be7576740 1 Processor -- start 2024-03-21T11:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.109+0000 7f6be7576740 1 -- start start 2024-03-21T11:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.109+0000 7f6be7576740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05f1e0 0x56382c05f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.109+0000 7f6be7576740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c05fbd0 0x56382c0646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.109+0000 7f6be7576740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c064d40 0x56382c0671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.109+0000 7f6be7576740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56382c0223b0 con 0x56382c064d40 2024-03-21T11:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.109+0000 7f6be7576740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56382bfc4830 con 0x56382c05fbd0 2024-03-21T11:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.109+0000 7f6be7576740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56382bf6d5d0 con 0x56382c05f1e0 2024-03-21T11:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.109+0000 7f6be2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05f1e0 0x56382c05f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.109+0000 7f6be1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c05fbd0 0x56382c0646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.109+0000 7f6be2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05f1e0 0x56382c05f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53846/0 (socket says 172.21.15.47:53846) 2024-03-21T11:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.109+0000 7f6be2800640 1 -- 172.21.15.47:0/58914398 learned_addr learned my addr 172.21.15.47:0/58914398 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.110+0000 7f6be1400640 1 --2- 172.21.15.47:0/58914398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c064d40 0x56382c0671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.110+0000 7f6be1e00640 1 -- 172.21.15.47:0/58914398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05f1e0 msgr2=0x56382c05f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.110+0000 7f6be1e00640 1 --2- 172.21.15.47:0/58914398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05f1e0 0x56382c05f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.110+0000 7f6be1e00640 1 -- 172.21.15.47:0/58914398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c064d40 msgr2=0x56382c0671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.110+0000 7f6be1e00640 1 --2- 172.21.15.47:0/58914398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c064d40 0x56382c0671d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.110+0000 7f6be1e00640 1 -- 172.21.15.47:0/58914398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56382bf852c0 con 0x56382c05fbd0 2024-03-21T11:26:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.110+0000 7f6be2800640 1 --2- 172.21.15.47:0/58914398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05f1e0 0x56382c05f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.110+0000 7f6be1e00640 1 --2- 172.21.15.47:0/58914398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c05fbd0 0x56382c0646d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00cd260 tx=0x7f6bd00d8230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cff2059cf4fc430 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.110+0000 7f6bd6a00640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bc40a2fc0 0x7f6bc40a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.111+0000 7f6bcb000640 1 -- 172.21.15.47:0/58914398 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0098190 con 0x56382c05fbd0 2024-03-21T11:26:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.111+0000 7f6be2800640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bc40a2fc0 0x7f6bc40a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.111+0000 7f6bd6a00640 1 -- 172.21.15.47:0/3778777416 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6bd00cfbb0 con 0x7f6bdc103900 2024-03-21T11:26:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.111+0000 7f6bcb000640 1 -- 172.21.15.47:0/58914398 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bd00e1070 con 0x56382c05fbd0 2024-03-21T11:26:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.111+0000 7f6bcb000640 1 -- 172.21.15.47:0/58914398 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd009a4b0 con 0x56382c05fbd0 2024-03-21T11:26:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.112+0000 7f6be7576740 1 -- 172.21.15.47:0/58914398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c05fbd0 msgr2=0x56382c0646d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.112+0000 7f6be7576740 1 --2- 172.21.15.47:0/58914398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c05fbd0 0x56382c0646d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00cd260 tx=0x7f6bd00d8230 comp rx=0 tx=0).stop 2024-03-21T11:26:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.112+0000 7f6be2800640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bc40a2fc0 0x7f6bc40a5480 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8006010 tx=0x7f6bd8032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.112+0000 7f6be7576740 1 -- 172.21.15.47:0/58914398 shutdown_connections 2024-03-21T11:26:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.112+0000 7f6be7576740 1 --2- 172.21.15.47:0/58914398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c064d40 0x56382c0671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.112+0000 7f6be7576740 1 --2- 172.21.15.47:0/58914398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c05fbd0 0x56382c0646d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.112+0000 7f6be7576740 1 --2- 172.21.15.47:0/58914398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05f1e0 0x56382c05f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.112+0000 7f6be7576740 1 -- 172.21.15.47:0/58914398 >> 172.21.15.47:0/58914398 conn(0x56382c05a750 msgr2=0x56382c05ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.112+0000 7f6be7576740 1 -- 172.21.15.47:0/58914398 shutdown_connections 2024-03-21T11:26:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.112+0000 7f6be7576740 1 -- 172.21.15.47:0/58914398 wait complete. 2024-03-21T11:26:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.112+0000 7f6be7576740 1 -- start start 2024-03-21T11:26:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.113+0000 7f6be7576740 1 Processor -- start 2024-03-21T11:26:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.113+0000 7f6be7576740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c05f1e0 0x56382c069300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.113+0000 7f6be7576740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05fbd0 0x56382c069840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.113+0000 7f6be7576740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c064d40 0x56382c070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.113+0000 7f6be7576740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56382bf80a20 con 0x56382c064d40 2024-03-21T11:26:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.113+0000 7f6be7576740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56382bfc08d0 con 0x56382c05f1e0 2024-03-21T11:26:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.113+0000 7f6be7576740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56382c067c90 con 0x56382c05fbd0 2024-03-21T11:26:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.113+0000 7f6be7576740 10 client.0 _ll_register_callbacks cb 0x56382bfbd1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.113+0000 7f6be1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c05f1e0 0x56382c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.113+0000 7f6be1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05fbd0 0x56382c069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.113+0000 7f6be1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c05f1e0 0x56382c069300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46162/0 (socket says 172.21.15.47:46162) 2024-03-21T11:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.113+0000 7f6be1e00640 1 -- 172.21.15.47:0/1942806575 learned_addr learned my addr 172.21.15.47:0/1942806575 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.113+0000 7f6be2800640 1 --2- 172.21.15.47:0/1942806575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c064d40 0x56382c070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.114+0000 7f6be1e00640 1 -- 172.21.15.47:0/1942806575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05fbd0 msgr2=0x56382c069840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.114+0000 7f6be1e00640 1 --2- 172.21.15.47:0/1942806575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05fbd0 0x56382c069840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.114+0000 7f6be1e00640 1 -- 172.21.15.47:0/1942806575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c064d40 msgr2=0x56382c070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.114+0000 7f6be1e00640 1 --2- 172.21.15.47:0/1942806575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c064d40 0x56382c070880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.114+0000 7f6be1e00640 1 -- 172.21.15.47:0/1942806575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56382bcd9210 con 0x56382c05f1e0 2024-03-21T11:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.114+0000 7f6be1400640 1 --2- 172.21.15.47:0/1942806575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05fbd0 0x56382c069840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.114+0000 7f6be1e00640 1 --2- 172.21.15.47:0/1942806575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c05f1e0 0x56382c069300 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00cd230 tx=0x7f6bd0003ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f144149994711f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.114+0000 7f6bbf400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.114+0000 7f6bbf400640 1 -- 172.21.15.47:0/1942806575 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd009a6a0 con 0x56382c05f1e0 2024-03-21T11:26:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.114+0000 7f6bbf400640 1 -- 172.21.15.47:0/1942806575 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6bd00cd340 con 0x56382c05f1e0 2024-03-21T11:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.114+0000 7f6be7576740 1 -- 172.21.15.47:0/1942806575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bd00e1070 con 0x56382c05f1e0 2024-03-21T11:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.115+0000 7f6bbf400640 1 -- 172.21.15.47:0/1942806575 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bd009aa20 con 0x56382c05f1e0 2024-03-21T11:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.115+0000 7f6be7576740 10 client.26393 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.115+0000 7f6be7576740 1 -- 172.21.15.47:0/1942806575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56382bda5620 con 0x56382c05f1e0 2024-03-21T11:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.115+0000 7f6bbf400640 1 -- 172.21.15.47:0/1942806575 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bd00e1070 con 0x56382c05f1e0 2024-03-21T11:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.115+0000 7f6bbf400640 1 -- 172.21.15.47:0/1942806575 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6bd00d8c40 con 0x56382c05f1e0 2024-03-21T11:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.115+0000 7f6be7576740 10 client.26393 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.115+0000 7f6be7576740 10 client.26393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.115+0000 7f6be7576740 20 client.26393 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.115+0000 7f6be7576740 1 --2- 172.21.15.47:0/1942806575 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56382c074d70 0x56382c077160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.115+0000 7f6be7576740 4 client.26393 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.115+0000 7f6be7576740 1 -- 172.21.15.47:0/1942806575 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56382bfd1d70 con 0x56382c074d70 2024-03-21T11:26:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.115+0000 7f6be1400640 1 --2- 172.21.15.47:0/1942806575 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56382c074d70 0x56382c077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.116+0000 7f6be1400640 1 --2- 172.21.15.47:0/1942806575 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56382c074d70 0x56382c077160 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dbda446a3e43edf server_cookie=b8406fb13bcae7ff in_seq=0 out_seq=0 2024-03-21T11:26:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.116+0000 7f6bbf400640 10 client.26393 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.118+0000 7f6bbf400640 1 -- 172.21.15.47:0/1942806575 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56382bfd1d70 con 0x56382c074d70 2024-03-21T11:26:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.118+0000 7f6bbf400640 10 client.26393 handle_command_reply: tid=0 2024-03-21T11:26:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.118+0000 7f6be7576740 1 client.26393 shutdown 2024-03-21T11:26:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.119+0000 7f6be7576740 1 -- 172.21.15.47:0/1942806575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c05f1e0 msgr2=0x56382c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.119+0000 7f6be7576740 1 --2- 172.21.15.47:0/1942806575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c05f1e0 0x56382c069300 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00cd230 tx=0x7f6bd0003ea0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.119+0000 7f6be7576740 1 -- 172.21.15.47:0/1942806575 shutdown_connections 2024-03-21T11:26:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.119+0000 7f6be7576740 1 --2- 172.21.15.47:0/1942806575 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56382c074d70 0x56382c077160 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.119+0000 7f6be7576740 1 --2- 172.21.15.47:0/1942806575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c064d40 0x56382c070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.119+0000 7f6bbf400640 0 client.26393 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.119+0000 7f6be7576740 1 --2- 172.21.15.47:0/1942806575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05fbd0 0x56382c069840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.119+0000 7f6be7576740 1 --2- 172.21.15.47:0/1942806575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c05f1e0 0x56382c069300 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.119+0000 7f6be7576740 1 -- 172.21.15.47:0/1942806575 >> 172.21.15.47:0/1942806575 conn(0x56382c05a750 msgr2=0x56382c059950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.119+0000 7f6be7576740 1 -- 172.21.15.47:0/1942806575 shutdown_connections 2024-03-21T11:26:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.119+0000 7f6be7576740 1 -- 172.21.15.47:0/1942806575 wait complete. 2024-03-21T11:26:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.119+0000 7f6be7576740 20 client.26393 trim_cache size 0 max 16384 2024-03-21T11:26:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.136+0000 7f6be7576740 1 Processor -- start 2024-03-21T11:26:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.136+0000 7f6be7576740 1 -- start start 2024-03-21T11:26:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.136+0000 7f6be7576740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c05f1e0 0x56382c068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.136+0000 7f6be7576740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05fbd0 0x56382c070840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.136+0000 7f6be7576740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c064d40 0x56382c069450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.136+0000 7f6be7576740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc1ac270 con 0x56382c05f1e0 2024-03-21T11:26:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.136+0000 7f6be7576740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc1adbf0 con 0x56382c064d40 2024-03-21T11:26:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.136+0000 7f6be7576740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc1ac8d0 con 0x56382c05fbd0 2024-03-21T11:26:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.137+0000 7f6be1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05fbd0 0x56382c070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.137+0000 7f6be2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c064d40 0x56382c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.137+0000 7f6be1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c05f1e0 0x56382c068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.137+0000 7f6be2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c064d40 0x56382c069450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46170/0 (socket says 172.21.15.47:46170) 2024-03-21T11:26:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.137+0000 7f6be1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05fbd0 0x56382c070840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53866/0 (socket says 172.21.15.47:53866) 2024-03-21T11:26:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.137+0000 7f6be2800640 1 -- 172.21.15.47:0/1190335892 learned_addr learned my addr 172.21.15.47:0/1190335892 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.137+0000 7f6be1400640 1 -- 172.21.15.47:0/1190335892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c064d40 msgr2=0x56382c069450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.137+0000 7f6be1400640 1 --2- 172.21.15.47:0/1190335892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c064d40 0x56382c069450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.137+0000 7f6be1400640 1 -- 172.21.15.47:0/1190335892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c05f1e0 msgr2=0x56382c068f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.137+0000 7f6be1400640 1 --2- 172.21.15.47:0/1190335892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c05f1e0 0x56382c068f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.137+0000 7f6be1400640 1 -- 172.21.15.47:0/1190335892 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56382c058d40 con 0x56382c05fbd0 2024-03-21T11:26:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.137+0000 7f6be1e00640 1 --2- 172.21.15.47:0/1190335892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c05f1e0 0x56382c068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.137+0000 7f6be1400640 1 --2- 172.21.15.47:0/1190335892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05fbd0 0x56382c070840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x56382c0778a0 tx=0x7f6bcc005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15a9cc65a6ad5db6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.138+0000 7f6bca600640 1 -- 172.21.15.47:0/1190335892 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc020310 con 0x56382c05fbd0 2024-03-21T11:26:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.138+0000 7f6bca600640 1 -- 172.21.15.47:0/1190335892 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bcc012710 con 0x56382c05fbd0 2024-03-21T11:26:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.138+0000 7f6bca600640 1 -- 172.21.15.47:0/1190335892 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc003970 con 0x56382c05fbd0 2024-03-21T11:26:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.138+0000 7f6be7576740 1 -- 172.21.15.47:0/1190335892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05fbd0 msgr2=0x56382c070840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.138+0000 7f6be7576740 1 --2- 172.21.15.47:0/1190335892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05fbd0 0x56382c070840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x56382c0778a0 tx=0x7f6bcc005f90 comp rx=0 tx=0).stop 2024-03-21T11:26:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.138+0000 7f6be7576740 1 -- 172.21.15.47:0/1190335892 shutdown_connections 2024-03-21T11:26:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.138+0000 7f6be7576740 1 --2- 172.21.15.47:0/1190335892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c064d40 0x56382c069450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.138+0000 7f6be7576740 1 --2- 172.21.15.47:0/1190335892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05fbd0 0x56382c070840 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.138+0000 7f6be7576740 1 --2- 172.21.15.47:0/1190335892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c05f1e0 0x56382c068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.138+0000 7f6be7576740 1 -- 172.21.15.47:0/1190335892 >> 172.21.15.47:0/1190335892 conn(0x56382c05a750 msgr2=0x56382c05d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.138+0000 7f6be7576740 1 -- 172.21.15.47:0/1190335892 shutdown_connections 2024-03-21T11:26:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.139+0000 7f6be7576740 1 -- 172.21.15.47:0/1190335892 wait complete. 2024-03-21T11:26:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.139+0000 7f6be7576740 1 -- start start 2024-03-21T11:26:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.139+0000 7f6be7576740 1 Processor -- start 2024-03-21T11:26:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.139+0000 7f6be7576740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05f1e0 0x56382c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.139+0000 7f6be7576740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c05fbd0 0x56382c06ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be7576740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c064d40 0x56382c06fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be7576740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc1ad590 con 0x56382c05fbd0 2024-03-21T11:26:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be7576740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc1acf30 con 0x56382c064d40 2024-03-21T11:26:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be7576740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc1aa430 con 0x56382c05f1e0 2024-03-21T11:26:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be7576740 10 client.0 _ll_register_callbacks cb 0x7f6bcc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05f1e0 0x56382c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05f1e0 0x56382c06e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53872/0 (socket says 172.21.15.47:53872) 2024-03-21T11:26:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be1e00640 1 -- 172.21.15.47:0/3042867652 learned_addr learned my addr 172.21.15.47:0/3042867652 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be2800640 1 --2- 172.21.15.47:0/3042867652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c064d40 0x56382c06fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be1400640 1 --2- 172.21.15.47:0/3042867652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c05fbd0 0x56382c06ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be1e00640 1 -- 172.21.15.47:0/3042867652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c064d40 msgr2=0x56382c06fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be1e00640 1 --2- 172.21.15.47:0/3042867652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c064d40 0x56382c06fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be1e00640 1 -- 172.21.15.47:0/3042867652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c05fbd0 msgr2=0x56382c06ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be1e00640 1 --2- 172.21.15.47:0/3042867652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c05fbd0 0x56382c06ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be1e00640 1 -- 172.21.15.47:0/3042867652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56382c070260 con 0x56382c05f1e0 2024-03-21T11:26:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6be1e00640 1 --2- 172.21.15.47:0/3042867652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05f1e0 0x56382c06e770 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6bd009bbf0 tx=0x7f6bd0003d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=844143b4a08542e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.140+0000 7f6bbf400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.141+0000 7f6bbf400640 1 -- 172.21.15.47:0/3042867652 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd009abf0 con 0x56382c05f1e0 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.141+0000 7f6bbf400640 1 -- 172.21.15.47:0/3042867652 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6bd00a4670 con 0x56382c05f1e0 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.141+0000 7f6be7576740 1 -- 172.21.15.47:0/3042867652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bcc012710 con 0x56382c05f1e0 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.141+0000 7f6bbf400640 1 -- 172.21.15.47:0/3042867652 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bd009b850 con 0x56382c05f1e0 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.141+0000 7f6be7576740 10 client.26416 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.141+0000 7f6be7576740 1 -- 172.21.15.47:0/3042867652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56382c0d0ce0 con 0x56382c05f1e0 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.141+0000 7f6bbf400640 1 -- 172.21.15.47:0/3042867652 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bcc012710 con 0x56382c05f1e0 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.141+0000 7f6bbf400640 1 -- 172.21.15.47:0/3042867652 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6bd009c150 con 0x56382c05f1e0 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.141+0000 7f6be7576740 10 client.26416 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.141+0000 7f6be7576740 10 client.26416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.141+0000 7f6be7576740 20 client.26416 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.141+0000 7f6be7576740 1 --2- 172.21.15.47:0/3042867652 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56382c0d14a0 0x56382c0d38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.141+0000 7f6be7576740 4 client.26416 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.141+0000 7f6be7576740 1 -- 172.21.15.47:0/3042867652 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56382c064a70 con 0x56382c0d14a0 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.142+0000 7f6be1400640 1 --2- 172.21.15.47:0/3042867652 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56382c0d14a0 0x56382c0d38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.142+0000 7f6be1400640 1 --2- 172.21.15.47:0/3042867652 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56382c0d14a0 0x56382c0d38c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f8b39c0dccee3fc server_cookie=afa3db8c1eb8dbd2 in_seq=0 out_seq=0 2024-03-21T11:26:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.142+0000 7f6bbf400640 10 client.26416 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.398+0000 7f6bbf400640 1 -- 172.21.15.47:0/3042867652 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56382c064a70 con 0x56382c0d14a0 2024-03-21T11:26:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.398+0000 7f6bbf400640 10 client.26416 handle_command_reply: tid=0 2024-03-21T11:26:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.398+0000 7f6be7576740 1 client.26416 shutdown 2024-03-21T11:26:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.398+0000 7f6be7576740 1 -- 172.21.15.47:0/3042867652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05f1e0 msgr2=0x56382c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.398+0000 7f6be7576740 1 --2- 172.21.15.47:0/3042867652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05f1e0 0x56382c06e770 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6bd009bbf0 tx=0x7f6bd0003d60 comp rx=0 tx=0).stop 2024-03-21T11:26:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.398+0000 7f6be7576740 1 -- 172.21.15.47:0/3042867652 shutdown_connections 2024-03-21T11:26:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.399+0000 7f6be7576740 1 --2- 172.21.15.47:0/3042867652 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56382c0d14a0 0x56382c0d38c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.399+0000 7f6be7576740 1 --2- 172.21.15.47:0/3042867652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56382c064d40 0x56382c06fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.399+0000 7f6be7576740 1 --2- 172.21.15.47:0/3042867652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56382c05fbd0 0x56382c06ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.399+0000 7f6be7576740 1 --2- 172.21.15.47:0/3042867652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56382c05f1e0 0x56382c06e770 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.399+0000 7f6bbf400640 0 client.26416 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.399+0000 7f6be7576740 1 -- 172.21.15.47:0/3042867652 >> 172.21.15.47:0/3042867652 conn(0x56382c05a750 msgr2=0x56382c068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.399+0000 7f6be7576740 1 -- 172.21.15.47:0/3042867652 shutdown_connections 2024-03-21T11:26:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.399+0000 7f6be7576740 1 -- 172.21.15.47:0/3042867652 wait complete. 2024-03-21T11:26:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.399+0000 7f6be7576740 20 client.26416 trim_cache size 0 max 16384 2024-03-21T11:26:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:26:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:26:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:26:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:26:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.400+0000 7f6be4800640 1 -- 172.21.15.47:0/3778777416 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bc40a2fc0 msgr2=0x7f6bc40a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.400+0000 7f6be4800640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bc40a2fc0 0x7f6bc40a5480 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8006010 tx=0x7f6bd8032590 comp rx=0 tx=0).stop 2024-03-21T11:26:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.400+0000 7f6be4800640 1 -- 172.21.15.47:0/3778777416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc103900 msgr2=0x7f6bdc199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.400+0000 7f6be4800640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc103900 0x7f6bdc199ab0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000e9c0 tx=0x7f6bd000ee80 comp rx=0 tx=0).stop 2024-03-21T11:26:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.400+0000 7f6be4800640 1 -- 172.21.15.47:0/3778777416 shutdown_connections 2024-03-21T11:26:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.400+0000 7f6be4800640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bc40a2fc0 0x7f6bc40a5480 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.400+0000 7f6be4800640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc1054b0 0x7f6bdc1a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.400+0000 7f6be4800640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc104af0 0x7f6bdc199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.400+0000 7f6be4800640 1 --2- 172.21.15.47:0/3778777416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc103900 0x7f6bdc199ab0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.400+0000 7f6be4800640 1 -- 172.21.15.47:0/3778777416 >> 172.21.15.47:0/3778777416 conn(0x7f6bdc0fee90 msgr2=0x7f6bdc107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.400+0000 7f6be4800640 1 -- 172.21.15.47:0/3778777416 shutdown_connections 2024-03-21T11:26:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:29.400+0000 7f6be4800640 1 -- 172.21.15.47:0/3778777416 wait complete. 2024-03-21T11:26:29.496 INFO:tasks.workunit.client.0.smithi047.stdout: 1 121105 9.19 MB/sec warmup 73 sec latency 93.187 ms 2024-03-21T11:26:30.496 INFO:tasks.workunit.client.0.smithi047.stdout: 1 122724 9.15 MB/sec warmup 74 sec latency 25.817 ms 2024-03-21T11:26:30.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:30 smithi102 ceph-mon[36748]: pgmap v495: 129 pgs: 129 active+clean; 38 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 299 op/s 2024-03-21T11:26:30.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:30 smithi047 ceph-mon[31351]: pgmap v495: 129 pgs: 129 active+clean; 38 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 299 op/s 2024-03-21T11:26:30.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:30 smithi100 ceph-mon[36245]: pgmap v495: 129 pgs: 129 active+clean; 38 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 299 op/s 2024-03-21T11:26:31.496 INFO:tasks.workunit.client.0.smithi047.stdout: 1 124437 9.20 MB/sec warmup 75 sec latency 39.807 ms 2024-03-21T11:26:32.496 INFO:tasks.workunit.client.0.smithi047.stdout: 1 125434 9.11 MB/sec warmup 76 sec latency 20.587 ms 2024-03-21T11:26:32.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:32 smithi102 ceph-mon[36748]: pgmap v496: 129 pgs: 129 active+clean; 34 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-03-21T11:26:32.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:32 smithi047 ceph-mon[31351]: pgmap v496: 129 pgs: 129 active+clean; 34 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-03-21T11:26:32.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:32 smithi100 ceph-mon[36245]: pgmap v496: 129 pgs: 129 active+clean; 34 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-03-21T11:26:33.496 INFO:tasks.workunit.client.0.smithi047.stdout: 1 127663 9.19 MB/sec warmup 77 sec latency 11.799 ms 2024-03-21T11:26:34.496 INFO:tasks.workunit.client.0.smithi047.stdout: 1 128459 9.10 MB/sec warmup 78 sec latency 42.745 ms 2024-03-21T11:26:34.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:34 smithi102 ceph-mon[36748]: pgmap v497: 129 pgs: 129 active+clean; 30 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 293 op/s 2024-03-21T11:26:34.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:34 smithi047 ceph-mon[31351]: pgmap v497: 129 pgs: 129 active+clean; 30 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 293 op/s 2024-03-21T11:26:34.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:34 smithi100 ceph-mon[36245]: pgmap v497: 129 pgs: 129 active+clean; 30 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 293 op/s 2024-03-21T11:26:35.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:26:35.425 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:26:35.496 INFO:tasks.workunit.client.0.smithi047.stdout: 1 130298 9.08 MB/sec warmup 79 sec latency 28.317 ms 2024-03-21T11:26:35.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.667+0000 7f9779400640 1 -- 172.21.15.47:0/3566425692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97741038c0 msgr2=0x7f9774103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:35.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.667+0000 7f9779400640 1 --2- 172.21.15.47:0/3566425692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97741038c0 0x7f9774103ce0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9764009900 tx=0x7f976402bc30 comp rx=0 tx=0).stop 2024-03-21T11:26:35.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.667+0000 7f9779400640 1 -- 172.21.15.47:0/3566425692 shutdown_connections 2024-03-21T11:26:35.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.667+0000 7f9779400640 1 --2- 172.21.15.47:0/3566425692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9774105470 0x7f977410bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.667+0000 7f9779400640 1 --2- 172.21.15.47:0/3566425692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9774104ab0 0x7f9774104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.667+0000 7f9779400640 1 --2- 172.21.15.47:0/3566425692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97741038c0 0x7f9774103ce0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.667+0000 7f9779400640 1 -- 172.21.15.47:0/3566425692 >> 172.21.15.47:0/3566425692 conn(0x7f97740fee50 msgr2=0x7f9774101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:35.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.667+0000 7f9779400640 1 -- 172.21.15.47:0/3566425692 shutdown_connections 2024-03-21T11:26:35.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.667+0000 7f9779400640 1 -- 172.21.15.47:0/3566425692 wait complete. 2024-03-21T11:26:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.668+0000 7f9779400640 1 Processor -- start 2024-03-21T11:26:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.668+0000 7f9779400640 1 -- start start 2024-03-21T11:26:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.668+0000 7f9779400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97741038c0 0x7f9774199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.668+0000 7f9779400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9774104ab0 0x7f977419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.668+0000 7f9779400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9774105470 0x7f97741a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.668+0000 7f9779400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977410f560 con 0x7f97741038c0 2024-03-21T11:26:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.668+0000 7f9779400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977410f3e0 con 0x7f9774104ab0 2024-03-21T11:26:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.668+0000 7f9779400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977410f6e0 con 0x7f9774105470 2024-03-21T11:26:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.669+0000 7f9769e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9774104ab0 0x7f977419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.669+0000 7f9773200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9774105470 0x7f97741a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.669+0000 7f9769e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9774104ab0 0x7f977419a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46190/0 (socket says 172.21.15.47:46190) 2024-03-21T11:26:35.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.669+0000 7f9769e00640 1 -- 172.21.15.47:0/1143000577 learned_addr learned my addr 172.21.15.47:0/1143000577 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:35.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.669+0000 7f9772800640 1 --2- 172.21.15.47:0/1143000577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97741038c0 0x7f9774199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.669+0000 7f9773200640 1 -- 172.21.15.47:0/1143000577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9774104ab0 msgr2=0x7f977419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:35.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.669+0000 7f9773200640 1 --2- 172.21.15.47:0/1143000577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9774104ab0 0x7f977419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.669+0000 7f9773200640 1 -- 172.21.15.47:0/1143000577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97741038c0 msgr2=0x7f9774199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:35.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.669+0000 7f9773200640 1 --2- 172.21.15.47:0/1143000577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97741038c0 0x7f9774199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.669+0000 7f9773200640 1 -- 172.21.15.47:0/1143000577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97741a17f0 con 0x7f9774105470 2024-03-21T11:26:35.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.669+0000 7f9769e00640 1 --2- 172.21.15.47:0/1143000577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9774104ab0 0x7f977419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:35.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.670+0000 7f9773200640 1 --2- 172.21.15.47:0/1143000577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9774105470 0x7f97741a1180 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f976c009fd0 tx=0x7f976c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:35.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.670+0000 7f976be00640 1 -- 172.21.15.47:0/1143000577 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c00ce20 con 0x7f9774105470 2024-03-21T11:26:35.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.670+0000 7f9779400640 1 -- 172.21.15.47:0/1143000577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97741a1ae0 con 0x7f9774105470 2024-03-21T11:26:35.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.670+0000 7f976be00640 1 -- 172.21.15.47:0/1143000577 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f976c005020 con 0x7f9774105470 2024-03-21T11:26:35.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.670+0000 7f976be00640 1 -- 172.21.15.47:0/1143000577 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c010720 con 0x7f9774105470 2024-03-21T11:26:35.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.670+0000 7f9779400640 1 -- 172.21.15.47:0/1143000577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97741a2030 con 0x7f9774105470 2024-03-21T11:26:35.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.674+0000 7f977c028740 1 Processor -- start 2024-03-21T11:26:35.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.674+0000 7f976be00640 1 -- 172.21.15.47:0/1143000577 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f976c005540 con 0x7f9774105470 2024-03-21T11:26:35.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.675+0000 7f977c028740 1 -- start start 2024-03-21T11:26:35.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.675+0000 7f977c028740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acb52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.675+0000 7f977c028740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acb5870 0x55a03acba370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.675+0000 7f977c028740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acba9e0 0x55a03acbce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.675+0000 7f977c028740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a03ac1a250 con 0x55a03acb5870 2024-03-21T11:26:35.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.675+0000 7f977c028740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a03abc2a30 con 0x55a03acba9e0 2024-03-21T11:26:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.675+0000 7f977c028740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a03ac162f0 con 0x55a03acb4e80 2024-03-21T11:26:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.675+0000 7f9769e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acb52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.675+0000 7f9769e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acb52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53888/0 (socket says 172.21.15.47:53888) 2024-03-21T11:26:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.675+0000 7f9769e00640 1 -- 172.21.15.47:0/1168102069 learned_addr learned my addr 172.21.15.47:0/1168102069 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.675+0000 7f9769e00640 1 --2- 172.21.15.47:0/1168102069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acba9e0 0x55a03acbce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.676+0000 7f9772800640 1 --2- 172.21.15.47:0/1168102069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acb5870 0x55a03acba370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.676+0000 7f9769e00640 1 -- 172.21.15.47:0/1168102069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acba9e0 msgr2=0x55a03acbce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.676+0000 7f9769e00640 1 --2- 172.21.15.47:0/1168102069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acba9e0 0x55a03acbce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.676+0000 7f9769e00640 1 -- 172.21.15.47:0/1168102069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acb5870 msgr2=0x55a03acba370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.676+0000 7f9769e00640 1 --2- 172.21.15.47:0/1168102069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acb5870 0x55a03acba370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.676+0000 7f9769e00640 1 -- 172.21.15.47:0/1168102069 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a03abad690 con 0x55a03acb4e80 2024-03-21T11:26:35.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.676+0000 7f9769e00640 1 --2- 172.21.15.47:0/1168102069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acb52a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f975c011d00 tx=0x7f975c005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c6c864c401a639c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.676+0000 7f9763400640 1 -- 172.21.15.47:0/1168102069 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c00fc50 con 0x55a03acb4e80 2024-03-21T11:26:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.676+0000 7f9763400640 1 -- 172.21.15.47:0/1168102069 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f975c00fdf0 con 0x55a03acb4e80 2024-03-21T11:26:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.676+0000 7f9763400640 1 -- 172.21.15.47:0/1168102069 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c003d90 con 0x55a03acb4e80 2024-03-21T11:26:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.677+0000 7f976be00640 1 --2- 172.21.15.47:0/1143000577 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97540a2f20 0x7f97540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.677+0000 7f976be00640 1 -- 172.21.15.47:0/1143000577 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f976c0cfe40 con 0x7f9774105470 2024-03-21T11:26:35.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.677+0000 7f9772800640 1 --2- 172.21.15.47:0/1143000577 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97540a2f20 0x7f97540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.677+0000 7f977c028740 1 -- 172.21.15.47:0/1168102069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 msgr2=0x55a03acb52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:35.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.677+0000 7f977c028740 1 --2- 172.21.15.47:0/1168102069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acb52a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f975c011d00 tx=0x7f975c005c90 comp rx=0 tx=0).stop 2024-03-21T11:26:35.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.678+0000 7f977c028740 1 -- 172.21.15.47:0/1168102069 shutdown_connections 2024-03-21T11:26:35.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.678+0000 7f977c028740 1 --2- 172.21.15.47:0/1168102069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acba9e0 0x55a03acbce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.678+0000 7f977c028740 1 --2- 172.21.15.47:0/1168102069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acb5870 0x55a03acba370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.678+0000 7f977c028740 1 --2- 172.21.15.47:0/1168102069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acb52a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.678+0000 7f977c028740 1 -- 172.21.15.47:0/1168102069 >> 172.21.15.47:0/1168102069 conn(0x55a03ac7d200 msgr2=0x55a03acb07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:35.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.678+0000 7f977c028740 1 -- 172.21.15.47:0/1168102069 shutdown_connections 2024-03-21T11:26:35.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.678+0000 7f9772800640 1 --2- 172.21.15.47:0/1143000577 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97540a2f20 0x7f97540a53e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f976402c170 tx=0x7f976402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:35.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.678+0000 7f977c028740 1 -- 172.21.15.47:0/1168102069 wait complete. 2024-03-21T11:26:35.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.678+0000 7f977c028740 1 -- start start 2024-03-21T11:26:35.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.679+0000 7f977c028740 1 Processor -- start 2024-03-21T11:26:35.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.679+0000 7f977c028740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acbedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.679+0000 7f977c028740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acb5870 0x55a03acbf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.679+0000 7f977c028740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acba9e0 0x55a03acc6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.679+0000 7f977c028740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a03acbd990 con 0x55a03acba9e0 2024-03-21T11:26:35.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.679+0000 7f977c028740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a03abd5e80 con 0x55a03acb5870 2024-03-21T11:26:35.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.679+0000 7f977c028740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a03acbdb10 con 0x55a03acb4e80 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.679+0000 7f977c028740 10 client.0 _ll_register_callbacks cb 0x55a03abe8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.679+0000 7f9772800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acbedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.679+0000 7f9769e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acb5870 0x55a03acbf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.680+0000 7f9772800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acbedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53904/0 (socket says 172.21.15.47:53904) 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.680+0000 7f9772800640 1 -- 172.21.15.47:0/1438380259 learned_addr learned my addr 172.21.15.47:0/1438380259 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.680+0000 7f9769e00640 1 --2- 172.21.15.47:0/1438380259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acb5870 0x55a03acbf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46212/0 (socket says 172.21.15.47:46212) 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.680+0000 7f9772800640 1 -- 172.21.15.47:0/1438380259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acb5870 msgr2=0x55a03acbf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.680+0000 7f9772800640 1 --2- 172.21.15.47:0/1438380259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acb5870 0x55a03acbf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.680+0000 7f9773200640 1 --2- 172.21.15.47:0/1438380259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acba9e0 0x55a03acc6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.680+0000 7f9772800640 1 -- 172.21.15.47:0/1438380259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acba9e0 msgr2=0x55a03acc6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.680+0000 7f9772800640 1 --2- 172.21.15.47:0/1438380259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acba9e0 0x55a03acc6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.680+0000 7f9772800640 1 -- 172.21.15.47:0/1438380259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a03a9f87d0 con 0x55a03acb4e80 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.680+0000 7f9773200640 1 --2- 172.21.15.47:0/1438380259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acba9e0 0x55a03acc6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.680+0000 7f9772800640 1 --2- 172.21.15.47:0/1438380259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acbedf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f97640096c0 tx=0x7f9764009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3c675e405b58c18 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:35.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.680+0000 7f975be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.681+0000 7f975be00640 1 -- 172.21.15.47:0/1438380259 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764044070 con 0x55a03acb4e80 2024-03-21T11:26:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.681+0000 7f975be00640 1 -- 172.21.15.47:0/1438380259 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9764003b40 con 0x55a03acb4e80 2024-03-21T11:26:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.681+0000 7f977c028740 1 -- 172.21.15.47:0/1438380259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f975c00fdf0 con 0x55a03acb4e80 2024-03-21T11:26:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.681+0000 7f975be00640 1 -- 172.21.15.47:0/1438380259 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9764003d40 con 0x55a03acb4e80 2024-03-21T11:26:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.681+0000 7f977c028740 10 client.26428 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.681+0000 7f977c028740 1 -- 172.21.15.47:0/1438380259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a03a92e3f0 con 0x55a03acb4e80 2024-03-21T11:26:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.681+0000 7f975be00640 1 -- 172.21.15.47:0/1438380259 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f975c00fdf0 con 0x55a03acb4e80 2024-03-21T11:26:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.681+0000 7f975be00640 1 -- 172.21.15.47:0/1438380259 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9764053430 con 0x55a03acb4e80 2024-03-21T11:26:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.681+0000 7f977c028740 10 client.26428 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.681+0000 7f977c028740 10 client.26428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.681+0000 7f977c028740 20 client.26428 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.681+0000 7f977c028740 1 --2- 172.21.15.47:0/1438380259 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a03accac80 0x55a03accd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.682+0000 7f977c028740 4 client.26428 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:35.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.682+0000 7f977c028740 1 -- 172.21.15.47:0/1438380259 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a03ac27790 con 0x55a03accac80 2024-03-21T11:26:35.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.682+0000 7f9769e00640 1 --2- 172.21.15.47:0/1438380259 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a03accac80 0x55a03accd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.682+0000 7f9769e00640 1 --2- 172.21.15.47:0/1438380259 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a03accac80 0x55a03accd100 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c32fb389d4ce32f server_cookie=b210cb2990d3ba56 in_seq=0 out_seq=0 2024-03-21T11:26:35.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.682+0000 7f975be00640 10 client.26428 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:35.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.683+0000 7f975be00640 1 -- 172.21.15.47:0/1438380259 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a03ac27790 con 0x55a03accac80 2024-03-21T11:26:35.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.683+0000 7f975be00640 10 client.26428 handle_command_reply: tid=0 2024-03-21T11:26:35.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.683+0000 7f977c028740 1 client.26428 shutdown 2024-03-21T11:26:35.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.683+0000 7f977c028740 1 -- 172.21.15.47:0/1438380259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 msgr2=0x55a03acbedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:35.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.683+0000 7f977c028740 1 --2- 172.21.15.47:0/1438380259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acbedf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f97640096c0 tx=0x7f9764009260 comp rx=0 tx=0).stop 2024-03-21T11:26:35.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.684+0000 7f977c028740 1 -- 172.21.15.47:0/1438380259 shutdown_connections 2024-03-21T11:26:35.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.684+0000 7f977c028740 1 --2- 172.21.15.47:0/1438380259 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a03accac80 0x55a03accd100 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.684+0000 7f977c028740 1 --2- 172.21.15.47:0/1438380259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acba9e0 0x55a03acc6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.684+0000 7f977c028740 1 --2- 172.21.15.47:0/1438380259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acb5870 0x55a03acbf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.684+0000 7f977c028740 1 --2- 172.21.15.47:0/1438380259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acbedf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.684+0000 7f975be00640 0 client.26428 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:35.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.684+0000 7f977c028740 1 -- 172.21.15.47:0/1438380259 >> 172.21.15.47:0/1438380259 conn(0x55a03ac7d200 msgr2=0x55a03acafa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:35.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.684+0000 7f977c028740 1 -- 172.21.15.47:0/1438380259 shutdown_connections 2024-03-21T11:26:35.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.684+0000 7f977c028740 1 -- 172.21.15.47:0/1438380259 wait complete. 2024-03-21T11:26:35.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.684+0000 7f977c028740 20 client.26428 trim_cache size 0 max 16384 2024-03-21T11:26:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.701+0000 7f977c028740 1 Processor -- start 2024-03-21T11:26:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.702+0000 7f977c028740 1 -- start start 2024-03-21T11:26:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.702+0000 7f977c028740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acbf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.702+0000 7f977c028740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acb5870 0x55a03acc6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.702+0000 7f977c028740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acba9e0 0x55a03acaf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.702+0000 7f977c028740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97741ac2a0 con 0x55a03acb5870 2024-03-21T11:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.702+0000 7f977c028740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97741adc20 con 0x55a03acba9e0 2024-03-21T11:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.702+0000 7f977c028740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97741ac900 con 0x55a03acb4e80 2024-03-21T11:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.702+0000 7f9772800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acbf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.702+0000 7f9773200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acba9e0 0x55a03acaf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.702+0000 7f9772800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acbf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53920/0 (socket says 172.21.15.47:53920) 2024-03-21T11:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.702+0000 7f9772800640 1 -- 172.21.15.47:0/564023118 learned_addr learned my addr 172.21.15.47:0/564023118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.702+0000 7f9769e00640 1 --2- 172.21.15.47:0/564023118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acb5870 0x55a03acc6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.703+0000 7f9772800640 1 -- 172.21.15.47:0/564023118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acba9e0 msgr2=0x55a03acaf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.703+0000 7f9772800640 1 --2- 172.21.15.47:0/564023118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acba9e0 0x55a03acaf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.703+0000 7f9772800640 1 -- 172.21.15.47:0/564023118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acb5870 msgr2=0x55a03acc6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.703+0000 7f9772800640 1 --2- 172.21.15.47:0/564023118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acb5870 0x55a03acc6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.703+0000 7f9772800640 1 -- 172.21.15.47:0/564023118 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a03acba730 con 0x55a03acb4e80 2024-03-21T11:26:35.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.703+0000 7f9769e00640 1 --2- 172.21.15.47:0/564023118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acb5870 0x55a03acc6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:35.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.703+0000 7f9773200640 1 --2- 172.21.15.47:0/564023118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acba9e0 0x55a03acaf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:35.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.703+0000 7f9772800640 1 --2- 172.21.15.47:0/564023118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acbf290 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9764009230 tx=0x7f97640097c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=696b3e7ad9fa7349 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:35.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.703+0000 7f9762a00640 1 -- 172.21.15.47:0/564023118 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764005d50 con 0x55a03acb4e80 2024-03-21T11:26:35.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.703+0000 7f9762a00640 1 -- 172.21.15.47:0/564023118 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9764066070 con 0x55a03acb4e80 2024-03-21T11:26:35.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.703+0000 7f9762a00640 1 -- 172.21.15.47:0/564023118 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764061960 con 0x55a03acb4e80 2024-03-21T11:26:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.704+0000 7f977c028740 1 -- 172.21.15.47:0/564023118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 msgr2=0x55a03acbf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.704+0000 7f977c028740 1 --2- 172.21.15.47:0/564023118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acbf290 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9764009230 tx=0x7f97640097c0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.704+0000 7f977c028740 1 -- 172.21.15.47:0/564023118 shutdown_connections 2024-03-21T11:26:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.704+0000 7f977c028740 1 --2- 172.21.15.47:0/564023118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acba9e0 0x55a03acaf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.704+0000 7f977c028740 1 --2- 172.21.15.47:0/564023118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acb5870 0x55a03acc6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.704+0000 7f977c028740 1 --2- 172.21.15.47:0/564023118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acbf290 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.704+0000 7f977c028740 1 -- 172.21.15.47:0/564023118 >> 172.21.15.47:0/564023118 conn(0x55a03ac7d200 msgr2=0x55a03acaf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.704+0000 7f977c028740 1 -- 172.21.15.47:0/564023118 shutdown_connections 2024-03-21T11:26:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.704+0000 7f977c028740 1 -- 172.21.15.47:0/564023118 wait complete. 2024-03-21T11:26:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.705+0000 7f977c028740 1 -- start start 2024-03-21T11:26:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.705+0000 7f977c028740 1 Processor -- start 2024-03-21T11:26:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.705+0000 7f977c028740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acc4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.705+0000 7f977c028740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acb5870 0x55a03acc4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.705+0000 7f977c028740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acba9e0 0x55a03acc5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f977c028740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97741ad5c0 con 0x55a03acba9e0 2024-03-21T11:26:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f977c028740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97741acf60 con 0x55a03acb5870 2024-03-21T11:26:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f977c028740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97741aa460 con 0x55a03acb4e80 2024-03-21T11:26:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f977c028740 10 client.0 _ll_register_callbacks cb 0x7f975c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f9769e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acb5870 0x55a03acc4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f9772800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acc4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f9769e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acb5870 0x55a03acc4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46228/0 (socket says 172.21.15.47:46228) 2024-03-21T11:26:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f9772800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acc4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53936/0 (socket says 172.21.15.47:53936) 2024-03-21T11:26:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f9769e00640 1 -- 172.21.15.47:0/99282304 learned_addr learned my addr 172.21.15.47:0/99282304 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f9773200640 1 --2- 172.21.15.47:0/99282304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acba9e0 0x55a03acc5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f9769e00640 1 -- 172.21.15.47:0/99282304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 msgr2=0x55a03acc4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f9769e00640 1 --2- 172.21.15.47:0/99282304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acc4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f9769e00640 1 -- 172.21.15.47:0/99282304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acba9e0 msgr2=0x55a03acc5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f9769e00640 1 --2- 172.21.15.47:0/99282304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acba9e0 0x55a03acc5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f9769e00640 1 -- 172.21.15.47:0/99282304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a03a8ee0f0 con 0x55a03acb5870 2024-03-21T11:26:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.706+0000 7f9772800640 1 --2- 172.21.15.47:0/99282304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acc4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.707+0000 7f9769e00640 1 --2- 172.21.15.47:0/99282304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acb5870 0x55a03acc4660 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f975c011cd0 tx=0x7f975c009fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9438074f842c23a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.707+0000 7f975be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.707+0000 7f975be00640 1 -- 172.21.15.47:0/99282304 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c011040 con 0x55a03acb5870 2024-03-21T11:26:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.707+0000 7f977c028740 1 -- 172.21.15.47:0/99282304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9764066070 con 0x55a03acb5870 2024-03-21T11:26:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.707+0000 7f975be00640 1 -- 172.21.15.47:0/99282304 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f975c0111e0 con 0x55a03acb5870 2024-03-21T11:26:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.707+0000 7f975be00640 1 -- 172.21.15.47:0/99282304 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f975c039050 con 0x55a03acb5870 2024-03-21T11:26:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.707+0000 7f977c028740 10 client.26417 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.707+0000 7f977c028740 1 -- 172.21.15.47:0/99282304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a03ad269e0 con 0x55a03acb5870 2024-03-21T11:26:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.708+0000 7f975be00640 1 -- 172.21.15.47:0/99282304 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9764066070 con 0x55a03acb5870 2024-03-21T11:26:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.708+0000 7f975be00640 1 -- 172.21.15.47:0/99282304 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f975c003910 con 0x55a03acb5870 2024-03-21T11:26:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.708+0000 7f977c028740 10 client.26417 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.708+0000 7f977c028740 10 client.26417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.708+0000 7f977c028740 20 client.26417 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.708+0000 7f977c028740 1 --2- 172.21.15.47:0/99282304 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a03ad27190 0x55a03ad295b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.708+0000 7f977c028740 4 client.26417 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:26:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.708+0000 7f977c028740 1 -- 172.21.15.47:0/99282304 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a03ac21ae0 con 0x55a03ad27190 2024-03-21T11:26:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.708+0000 7f9772800640 1 --2- 172.21.15.47:0/99282304 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a03ad27190 0x55a03ad295b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f9772800640 1 --2- 172.21.15.47:0/99282304 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a03ad27190 0x55a03ad295b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69025e86910fda56 server_cookie=658494203023b3d in_seq=0 out_seq=0 2024-03-21T11:26:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f975be00640 10 client.26417 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f975be00640 1 -- 172.21.15.47:0/99282304 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a03ac21ae0 con 0x55a03ad27190 2024-03-21T11:26:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f975be00640 10 client.26417 handle_command_reply: tid=0 2024-03-21T11:26:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f977c028740 1 client.26417 shutdown 2024-03-21T11:26:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f977c028740 1 -- 172.21.15.47:0/99282304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acb5870 msgr2=0x55a03acc4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f977c028740 1 --2- 172.21.15.47:0/99282304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acb5870 0x55a03acc4660 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f975c011cd0 tx=0x7f975c009fa0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f977c028740 1 -- 172.21.15.47:0/99282304 shutdown_connections 2024-03-21T11:26:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f977c028740 1 --2- 172.21.15.47:0/99282304 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a03ad27190 0x55a03ad295b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f977c028740 1 --2- 172.21.15.47:0/99282304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a03acba9e0 0x55a03acc5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f977c028740 1 --2- 172.21.15.47:0/99282304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a03acb5870 0x55a03acc4660 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f977c028740 1 --2- 172.21.15.47:0/99282304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a03acb4e80 0x55a03acc4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f975be00640 0 client.26417 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f977c028740 1 -- 172.21.15.47:0/99282304 >> 172.21.15.47:0/99282304 conn(0x55a03ac7d200 msgr2=0x55a03acbf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.709+0000 7f977c028740 1 -- 172.21.15.47:0/99282304 shutdown_connections 2024-03-21T11:26:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.710+0000 7f977c028740 1 -- 172.21.15.47:0/99282304 wait complete. 2024-03-21T11:26:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.710+0000 7f977c028740 20 client.26417 trim_cache size 0 max 16384 2024-03-21T11:26:35.717 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:26:35.717 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:26:35.717 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:26:35.717 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:26:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.710+0000 7f9779400640 1 -- 172.21.15.47:0/1143000577 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97540a2f20 msgr2=0x7f97540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.710+0000 7f9779400640 1 --2- 172.21.15.47:0/1143000577 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97540a2f20 0x7f97540a53e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f976402c170 tx=0x7f976402df90 comp rx=0 tx=0).stop 2024-03-21T11:26:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.710+0000 7f9779400640 1 -- 172.21.15.47:0/1143000577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9774105470 msgr2=0x7f97741a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.710+0000 7f9779400640 1 --2- 172.21.15.47:0/1143000577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9774105470 0x7f97741a1180 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f976c009fd0 tx=0x7f976c00ef10 comp rx=0 tx=0).stop 2024-03-21T11:26:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.710+0000 7f9779400640 1 -- 172.21.15.47:0/1143000577 shutdown_connections 2024-03-21T11:26:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.710+0000 7f9779400640 1 --2- 172.21.15.47:0/1143000577 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f97540a2f20 0x7f97540a53e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.710+0000 7f9779400640 1 --2- 172.21.15.47:0/1143000577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9774105470 0x7f97741a1180 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.710+0000 7f9779400640 1 --2- 172.21.15.47:0/1143000577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9774104ab0 0x7f977419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.710+0000 7f9779400640 1 --2- 172.21.15.47:0/1143000577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97741038c0 0x7f9774199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.710+0000 7f9779400640 1 -- 172.21.15.47:0/1143000577 >> 172.21.15.47:0/1143000577 conn(0x7f97740fee50 msgr2=0x7f9774107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.711+0000 7f9779400640 1 -- 172.21.15.47:0/1143000577 shutdown_connections 2024-03-21T11:26:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.711+0000 7f9779400640 1 -- 172.21.15.47:0/1143000577 wait complete. 2024-03-21T11:26:35.733 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:26:35.733 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:26:35.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.954+0000 7fa875000640 1 -- 172.21.15.47:0/1047188567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa870109890 msgr2=0x7fa870110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:35.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.954+0000 7fa875000640 1 --2- 172.21.15.47:0/1047188567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa870109890 0x7fa870110120 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa868009ca0 tx=0x7fa86802c0f0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.954+0000 7fa875000640 1 -- 172.21.15.47:0/1047188567 shutdown_connections 2024-03-21T11:26:35.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.954+0000 7fa875000640 1 --2- 172.21.15.47:0/1047188567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa870109890 0x7fa870110120 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.954+0000 7fa875000640 1 --2- 172.21.15.47:0/1047188567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa870108ed0 0x7fa870109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.954+0000 7fa875000640 1 --2- 172.21.15.47:0/1047188567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa870107ce0 0x7fa870108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.954+0000 7fa875000640 1 -- 172.21.15.47:0/1047188567 >> 172.21.15.47:0/1047188567 conn(0x7fa870076cb0 msgr2=0x7fa8700790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:35.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.954+0000 7fa875000640 1 -- 172.21.15.47:0/1047188567 shutdown_connections 2024-03-21T11:26:35.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.954+0000 7fa875000640 1 -- 172.21.15.47:0/1047188567 wait complete. 2024-03-21T11:26:35.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.955+0000 7fa875000640 1 Processor -- start 2024-03-21T11:26:35.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.956+0000 7fa875000640 1 -- start start 2024-03-21T11:26:35.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.956+0000 7fa875000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa870107ce0 0x7fa87019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.956+0000 7fa875000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa870108ed0 0x7fa87019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.956+0000 7fa875000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa870109890 0x7fa8701a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.956+0000 7fa875000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa870113940 con 0x7fa870109890 2024-03-21T11:26:35.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.956+0000 7fa875000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8701137c0 con 0x7fa870108ed0 2024-03-21T11:26:35.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.956+0000 7fa875000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa870113ac0 con 0x7fa870107ce0 2024-03-21T11:26:35.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.956+0000 7fa86de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa870107ce0 0x7fa87019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.956+0000 7fa86de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa870107ce0 0x7fa87019df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53952/0 (socket says 172.21.15.47:53952) 2024-03-21T11:26:35.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.956+0000 7fa86de00640 1 -- 172.21.15.47:0/2170970509 learned_addr learned my addr 172.21.15.47:0/2170970509 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:35.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.956+0000 7fa867e00640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa870108ed0 0x7fa87019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.956+0000 7fa86e800640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa870109890 0x7fa8701a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.957+0000 7fa86de00640 1 -- 172.21.15.47:0/2170970509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa870108ed0 msgr2=0x7fa87019e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.957+0000 7fa86de00640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa870108ed0 0x7fa87019e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.957+0000 7fa86de00640 1 -- 172.21.15.47:0/2170970509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa870109890 msgr2=0x7fa8701a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.957+0000 7fa86de00640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa870109890 0x7fa8701a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.957+0000 7fa86de00640 1 -- 172.21.15.47:0/2170970509 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8701a5b40 con 0x7fa870107ce0 2024-03-21T11:26:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.957+0000 7fa86e800640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa870109890 0x7fa8701a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.957+0000 7fa86de00640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa870107ce0 0x7fa87019df20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa86000e9c0 tx=0x7fa86000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.957+0000 7fa866a00640 1 -- 172.21.15.47:0/2170970509 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000cd80 con 0x7fa870107ce0 2024-03-21T11:26:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.957+0000 7fa866a00640 1 -- 172.21.15.47:0/2170970509 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8600035f0 con 0x7fa870107ce0 2024-03-21T11:26:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.957+0000 7fa875000640 1 -- 172.21.15.47:0/2170970509 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8701a5e30 con 0x7fa870107ce0 2024-03-21T11:26:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.957+0000 7fa866a00640 1 -- 172.21.15.47:0/2170970509 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860010690 con 0x7fa870107ce0 2024-03-21T11:26:35.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.957+0000 7fa875000640 1 -- 172.21.15.47:0/2170970509 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8701a62f0 con 0x7fa870107ce0 2024-03-21T11:26:35.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.959+0000 7fa875000640 1 -- 172.21.15.47:0/2170970509 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87010f000 con 0x7fa870107ce0 2024-03-21T11:26:35.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.961+0000 7fa866a00640 1 -- 172.21.15.47:0/2170970509 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa8600037a0 con 0x7fa870107ce0 2024-03-21T11:26:35.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.962+0000 7fa866a00640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8540a2fa0 0x7fa8540a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:35.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.962+0000 7fa866a00640 1 -- 172.21.15.47:0/2170970509 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa8600cfb80 con 0x7fa870107ce0 2024-03-21T11:26:35.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.962+0000 7fa867e00640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8540a2fa0 0x7fa8540a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:35.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.963+0000 7fa867e00640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8540a2fa0 0x7fa8540a5460 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa87019f430 tx=0x7fa858005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:35.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:35.967+0000 7fa866a00640 1 -- 172.21.15.47:0/2170970509 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa86009c5c0 con 0x7fa870107ce0 2024-03-21T11:26:36.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.228+0000 7fa875000640 1 -- 172.21.15.47:0/2170970509 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa87002d070 con 0x7fa870107ce0 2024-03-21T11:26:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.230+0000 7fa866a00640 1 -- 172.21.15.47:0/2170970509 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fa86009bec0 con 0x7fa870107ce0 2024-03-21T11:26:36.231 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:26:36.231 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:26:36.231 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:26:36.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.233+0000 7fa875000640 1 -- 172.21.15.47:0/2170970509 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8540a2fa0 msgr2=0x7fa8540a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.233+0000 7fa875000640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8540a2fa0 0x7fa8540a5460 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa87019f430 tx=0x7fa858005f90 comp rx=0 tx=0).stop 2024-03-21T11:26:36.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.233+0000 7fa875000640 1 -- 172.21.15.47:0/2170970509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa870107ce0 msgr2=0x7fa87019df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.233+0000 7fa875000640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa870107ce0 0x7fa87019df20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa86000e9c0 tx=0x7fa86000ee80 comp rx=0 tx=0).stop 2024-03-21T11:26:36.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.233+0000 7fa875000640 1 -- 172.21.15.47:0/2170970509 shutdown_connections 2024-03-21T11:26:36.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.233+0000 7fa875000640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8540a2fa0 0x7fa8540a5460 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.233+0000 7fa875000640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa870109890 0x7fa8701a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.233+0000 7fa875000640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa870108ed0 0x7fa87019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.233+0000 7fa875000640 1 --2- 172.21.15.47:0/2170970509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa870107ce0 0x7fa87019df20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.233+0000 7fa875000640 1 -- 172.21.15.47:0/2170970509 >> 172.21.15.47:0/2170970509 conn(0x7fa870076cb0 msgr2=0x7fa87010c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:36.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.233+0000 7fa875000640 1 -- 172.21.15.47:0/2170970509 shutdown_connections 2024-03-21T11:26:36.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.233+0000 7fa875000640 1 -- 172.21.15.47:0/2170970509 wait complete. 2024-03-21T11:26:36.254 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:26:36.254 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:26:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.485+0000 7f15d6200640 1 -- 172.21.15.47:0/1959473881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0102120 msgr2=0x7f15d0104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.485+0000 7f15d6200640 1 --2- 172.21.15.47:0/1959473881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0102120 0x7f15d0104510 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009900 tx=0x7f15c002bc30 comp rx=0 tx=0).stop 2024-03-21T11:26:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.485+0000 7f15d6200640 1 -- 172.21.15.47:0/1959473881 shutdown_connections 2024-03-21T11:26:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.485+0000 7f15d6200640 1 --2- 172.21.15.47:0/1959473881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15d0107380 0x7f15d01097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.485+0000 7f15d6200640 1 --2- 172.21.15.47:0/1959473881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15d0104a50 0x7f15d0106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.485+0000 7f15d6200640 1 --2- 172.21.15.47:0/1959473881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0102120 0x7f15d0104510 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.485+0000 7f15d6200640 1 -- 172.21.15.47:0/1959473881 >> 172.21.15.47:0/1959473881 conn(0x7f15d00fbb30 msgr2=0x7f15d00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:36.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.485+0000 7f15d6200640 1 -- 172.21.15.47:0/1959473881 shutdown_connections 2024-03-21T11:26:36.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.485+0000 7f15d6200640 1 -- 172.21.15.47:0/1959473881 wait complete. 2024-03-21T11:26:36.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.486+0000 7f15d6200640 1 Processor -- start 2024-03-21T11:26:36.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.486+0000 7f15d6200640 1 -- start start 2024-03-21T11:26:36.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15d6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0102120 0x7f15d0199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15d6200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15d0104a50 0x7f15d019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15d6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15d0107380 0x7f15d01a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15d6200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d010d380 con 0x7f15d0104a50 2024-03-21T11:26:36.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15d6200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d010d200 con 0x7f15d0102120 2024-03-21T11:26:36.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15d6200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d010d500 con 0x7f15d0107380 2024-03-21T11:26:36.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15cf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0102120 0x7f15d0199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15cf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0102120 0x7f15d0199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46252/0 (socket says 172.21.15.47:46252) 2024-03-21T11:26:36.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15cf400640 1 -- 172.21.15.47:0/1400428344 learned_addr learned my addr 172.21.15.47:0/1400428344 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:36.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15cfe00640 1 --2- 172.21.15.47:0/1400428344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15d0107380 0x7f15d01a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15cea00640 1 --2- 172.21.15.47:0/1400428344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15d0104a50 0x7f15d019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15cf400640 1 -- 172.21.15.47:0/1400428344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15d0107380 msgr2=0x7f15d01a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15cf400640 1 --2- 172.21.15.47:0/1400428344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15d0107380 0x7f15d01a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15cf400640 1 -- 172.21.15.47:0/1400428344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15d0104a50 msgr2=0x7f15d019a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15cf400640 1 --2- 172.21.15.47:0/1400428344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15d0104a50 0x7f15d019a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.487+0000 7f15cf400640 1 -- 172.21.15.47:0/1400428344 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d01a1840 con 0x7f15d0102120 2024-03-21T11:26:36.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.488+0000 7f15cf400640 1 --2- 172.21.15.47:0/1400428344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0102120 0x7f15d0199c30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f15c0000c00 tx=0x7f15c002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:36.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.488+0000 7f15c7e00640 1 -- 172.21.15.47:0/1400428344 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c0043020 con 0x7f15d0102120 2024-03-21T11:26:36.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.488+0000 7f15c7e00640 1 -- 172.21.15.47:0/1400428344 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15c00054e0 con 0x7f15d0102120 2024-03-21T11:26:36.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.488+0000 7f15d6200640 1 -- 172.21.15.47:0/1400428344 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d01a1ad0 con 0x7f15d0102120 2024-03-21T11:26:36.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.488+0000 7f15c7e00640 1 -- 172.21.15.47:0/1400428344 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c0003c40 con 0x7f15d0102120 2024-03-21T11:26:36.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.488+0000 7f15d6200640 1 -- 172.21.15.47:0/1400428344 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d01a2020 con 0x7f15d0102120 2024-03-21T11:26:36.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.492+0000 7f15c7e00640 1 -- 172.21.15.47:0/1400428344 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f15c0046020 con 0x7f15d0102120 2024-03-21T11:26:36.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.492+0000 7f15d8f9f740 1 Processor -- start 2024-03-21T11:26:36.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.492+0000 7f15d8f9f740 1 -- start start 2024-03-21T11:26:36.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.492+0000 7f15d8f9f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1d810 0x55dc3fa1dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15d8f9f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa1ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15d8f9f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa25c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15d8f9f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3f9822c0 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15d8f9f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3f92ac90 con 0x55dc3fa1f3a0 2024-03-21T11:26:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15d8f9f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3f97e360 con 0x55dc3fa1d810 2024-03-21T11:26:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15cfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1d810 0x55dc3fa1dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15cf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa1ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15cfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1d810 0x55dc3fa1dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53982/0 (socket says 172.21.15.47:53982) 2024-03-21T11:26:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15cfe00640 1 -- 172.21.15.47:0/1659957862 learned_addr learned my addr 172.21.15.47:0/1659957862 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15cea00640 1 --2- 172.21.15.47:0/1659957862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa25c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15cfe00640 1 -- 172.21.15.47:0/1659957862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1f3a0 msgr2=0x55dc3fa25c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15cfe00640 1 --2- 172.21.15.47:0/1659957862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa25c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15cfe00640 1 -- 172.21.15.47:0/1659957862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1e9e0 msgr2=0x55dc3fa1ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15cfe00640 1 --2- 172.21.15.47:0/1659957862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa1ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.493+0000 7f15cfe00640 1 -- 172.21.15.47:0/1659957862 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc3f94f080 con 0x55dc3fa1d810 2024-03-21T11:26:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.494+0000 7f15c7e00640 1 --2- 172.21.15.47:0/1400428344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15b40a2f20 0x7f15b40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.494+0000 7f15cfe00640 1 --2- 172.21.15.47:0/1659957862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1d810 0x55dc3fa1dc30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009fd0 tx=0x7f15c80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bad42d0494611da1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.494+0000 7f15c7e00640 1 -- 172.21.15.47:0/1400428344 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f15c00f1850 con 0x7f15d0102120 2024-03-21T11:26:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.494+0000 7f15cf400640 1 --2- 172.21.15.47:0/1400428344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15b40a2f20 0x7f15b40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.494+0000 7f15bbe00640 1 -- 172.21.15.47:0/1659957862 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8004d20 con 0x55dc3fa1d810 2024-03-21T11:26:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.494+0000 7f15bbe00640 1 -- 172.21.15.47:0/1659957862 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15c8004050 con 0x55dc3fa1d810 2024-03-21T11:26:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.494+0000 7f15bbe00640 1 -- 172.21.15.47:0/1659957862 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8005260 con 0x55dc3fa1d810 2024-03-21T11:26:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.495+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1659957862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1d810 msgr2=0x55dc3fa1dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.495+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/1659957862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1d810 0x55dc3fa1dc30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009fd0 tx=0x7f15c80118d0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.495+0000 7f15cf400640 1 --2- 172.21.15.47:0/1400428344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15b40a2f20 0x7f15b40a53e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f15c00b9fc0 tx=0x7f15c00f7550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.495+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1659957862 shutdown_connections 2024-03-21T11:26:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.495+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/1659957862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa25c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.495+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/1659957862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa1ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.495+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/1659957862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1d810 0x55dc3fa1dc30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.495+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1659957862 >> 172.21.15.47:0/1659957862 conn(0x55dc3fa18c00 msgr2=0x55dc3fa19000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.495+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1659957862 shutdown_connections 2024-03-21T11:26:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.495+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1659957862 wait complete. 2024-03-21T11:26:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.496+0000 7f15d8f9f740 1 -- start start 2024-03-21T11:26:36.499 INFO:tasks.workunit.client.0.smithi047.stdout: 1 131833 9.11 MB/sec warmup 80 sec latency 9.213 ms 2024-03-21T11:26:36.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.496+0000 7f15d8f9f740 1 Processor -- start 2024-03-21T11:26:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.496+0000 7f15d8f9f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1d810 0x55dc3fa27a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.496+0000 7f15d8f9f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa27fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.496+0000 7f15d8f9f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa2f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.496+0000 7f15d8f9f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3fa263b0 con 0x55dc3fa1d810 2024-03-21T11:26:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.496+0000 7f15d8f9f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3f93e0d0 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.496+0000 7f15d8f9f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3fa26530 con 0x55dc3fa1f3a0 2024-03-21T11:26:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.496+0000 7f15d8f9f740 10 client.0 _ll_register_callbacks cb 0x55dc3f942e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.497+0000 7f15cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa27fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.497+0000 7f15cfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1d810 0x55dc3fa27a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.497+0000 7f15cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa27fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46274/0 (socket says 172.21.15.47:46274) 2024-03-21T11:26:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.497+0000 7f15cea00640 1 -- 172.21.15.47:0/1069256211 learned_addr learned my addr 172.21.15.47:0/1069256211 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.497+0000 7f15cfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1d810 0x55dc3fa27a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37072/0 (socket says 172.21.15.47:37072) 2024-03-21T11:26:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.497+0000 7f15cfe00640 1 --2- 172.21.15.47:0/1069256211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa2f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.497+0000 7f15cea00640 1 -- 172.21.15.47:0/1069256211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1f3a0 msgr2=0x55dc3fa2f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.497+0000 7f15cea00640 1 --2- 172.21.15.47:0/1069256211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa2f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.497+0000 7f15cea00640 1 -- 172.21.15.47:0/1069256211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1d810 msgr2=0x55dc3fa27a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.497+0000 7f15cea00640 1 --2- 172.21.15.47:0/1069256211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1d810 0x55dc3fa27a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.497+0000 7f15cea00640 1 -- 172.21.15.47:0/1069256211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc3f6963f0 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.497+0000 7f15cfe00640 1 --2- 172.21.15.47:0/1069256211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1d810 0x55dc3fa27a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.497+0000 7f15cea00640 1 --2- 172.21.15.47:0/1069256211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa27fc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f15bc012700 tx=0x7f15bc012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef6d4c8623c47876 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.497+0000 7f15b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.498+0000 7f15b8c00640 1 -- 172.21.15.47:0/1069256211 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc00f8c0 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.498+0000 7f15b8c00640 1 -- 172.21.15.47:0/1069256211 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f15bc00fa60 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.498+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1069256211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15c8004050 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.498+0000 7f15b8c00640 1 -- 172.21.15.47:0/1069256211 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15bc025050 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.498+0000 7f15d8f9f740 10 client.26435 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.498+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1069256211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dc3f7607d0 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.498+0000 7f15b8c00640 1 -- 172.21.15.47:0/1069256211 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15c8004050 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.499+0000 7f15b8c00640 1 -- 172.21.15.47:0/1069256211 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f15bc010430 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.499+0000 7f15d8f9f740 10 client.26435 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.499+0000 7f15d8f9f740 10 client.26435 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:26:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.499+0000 7f15d8f9f740 20 client.26435 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.499+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/1069256211 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dc3fa33480 0x55dc3fa358e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.499+0000 7f15d8f9f740 4 client.26435 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.499+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1069256211 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc3f98f800 con 0x55dc3fa33480 2024-03-21T11:26:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.499+0000 7f15cfe00640 1 --2- 172.21.15.47:0/1069256211 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dc3fa33480 0x55dc3fa358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.499+0000 7f15cfe00640 1 --2- 172.21.15.47:0/1069256211 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dc3fa33480 0x55dc3fa358e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=93894828659df816 server_cookie=aea679362c286bf2 in_seq=0 out_seq=0 2024-03-21T11:26:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.499+0000 7f15b8c00640 10 client.26435 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:26:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.500+0000 7f15b8c00640 1 -- 172.21.15.47:0/1069256211 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dc3f98f800 con 0x55dc3fa33480 2024-03-21T11:26:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.500+0000 7f15b8c00640 10 client.26435 handle_command_reply: tid=0 2024-03-21T11:26:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.501+0000 7f15d8f9f740 1 client.26435 shutdown 2024-03-21T11:26:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.501+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1069256211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1e9e0 msgr2=0x55dc3fa27fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.501+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/1069256211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa27fc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f15bc012700 tx=0x7f15bc012bc0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.501+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1069256211 shutdown_connections 2024-03-21T11:26:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.501+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/1069256211 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dc3fa33480 0x55dc3fa358e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.501+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/1069256211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa2f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.501+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/1069256211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa27fc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.501+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/1069256211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1d810 0x55dc3fa27a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.501+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1069256211 >> 172.21.15.47:0/1069256211 conn(0x55dc3fa18c00 msgr2=0x55dc3fa17eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.501+0000 7f15b8c00640 0 client.26435 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:26:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.501+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1069256211 shutdown_connections 2024-03-21T11:26:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.501+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1069256211 wait complete. 2024-03-21T11:26:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.501+0000 7f15d8f9f740 20 client.26435 trim_cache size 0 max 16384 2024-03-21T11:26:36.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.515+0000 7f15d8f9f740 1 Processor -- start 2024-03-21T11:26:36.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.515+0000 7f15d8f9f740 1 -- start start 2024-03-21T11:26:36.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15d8f9f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1d810 0x55dc3fa21a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15d8f9f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa2efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15d8f9f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa21fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15d8f9f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d01ac360 con 0x55dc3fa1f3a0 2024-03-21T11:26:36.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15d8f9f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d01adce0 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15d8f9f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d01ac9c0 con 0x55dc3fa1d810 2024-03-21T11:26:36.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa2efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa2efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46282/0 (socket says 172.21.15.47:46282) 2024-03-21T11:26:36.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15cea00640 1 -- 172.21.15.47:0/1143445228 learned_addr learned my addr 172.21.15.47:0/1143445228 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:36.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15cfe00640 1 --2- 172.21.15.47:0/1143445228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1d810 0x55dc3fa21a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15cfe00640 1 --2- 172.21.15.47:0/1143445228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa21fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15cea00640 1 -- 172.21.15.47:0/1143445228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1d810 msgr2=0x55dc3fa21a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15cea00640 1 --2- 172.21.15.47:0/1143445228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1d810 0x55dc3fa21a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15cea00640 1 -- 172.21.15.47:0/1143445228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1f3a0 msgr2=0x55dc3fa21fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15cea00640 1 --2- 172.21.15.47:0/1143445228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa21fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15cea00640 1 -- 172.21.15.47:0/1143445228 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc3fa28a70 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.516+0000 7f15cfe00640 1 --2- 172.21.15.47:0/1143445228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa21fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:36.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.517+0000 7f15cea00640 1 --2- 172.21.15.47:0/1143445228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa2efc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f15bc001eb0 tx=0x7f15bc011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baba9d30f8644fe4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:36.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.517+0000 7f15bb400640 1 -- 172.21.15.47:0/1143445228 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc010430 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.517+0000 7f15bb400640 1 -- 172.21.15.47:0/1143445228 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15bc0055b0 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.517+0000 7f15bb400640 1 -- 172.21.15.47:0/1143445228 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc00fbc0 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.518+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1143445228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1e9e0 msgr2=0x55dc3fa2efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.518+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/1143445228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa2efc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f15bc001eb0 tx=0x7f15bc011f90 comp rx=0 tx=0).stop 2024-03-21T11:26:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.518+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1143445228 shutdown_connections 2024-03-21T11:26:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.518+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/1143445228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa21fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.518+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/1143445228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa2efc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.518+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/1143445228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1d810 0x55dc3fa21a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.518+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1143445228 >> 172.21.15.47:0/1143445228 conn(0x55dc3fa18c00 msgr2=0x55dc3fa1c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.518+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1143445228 shutdown_connections 2024-03-21T11:26:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.518+0000 7f15d8f9f740 1 -- 172.21.15.47:0/1143445228 wait complete. 2024-03-21T11:26:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.518+0000 7f15d8f9f740 1 -- start start 2024-03-21T11:26:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.518+0000 7f15d8f9f740 1 Processor -- start 2024-03-21T11:26:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.519+0000 7f15d8f9f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1d810 0x55dc3fa2b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.519+0000 7f15d8f9f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa2b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.519+0000 7f15d8f9f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa2e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.519+0000 7f15d8f9f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d01ad680 con 0x55dc3fa1d810 2024-03-21T11:26:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.519+0000 7f15d8f9f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d01ad020 con 0x55dc3fa1f3a0 2024-03-21T11:26:36.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.519+0000 7f15d8f9f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d01aa520 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.519+0000 7f15d8f9f740 10 client.0 _ll_register_callbacks cb 0x7f15bc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:36.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.519+0000 7f15cea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa2b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.519+0000 7f15cea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa2b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54004/0 (socket says 172.21.15.47:54004) 2024-03-21T11:26:36.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.519+0000 7f15cea00640 1 -- 172.21.15.47:0/867803705 learned_addr learned my addr 172.21.15.47:0/867803705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:36.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.519+0000 7f15cfe00640 1 --2- 172.21.15.47:0/867803705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1d810 0x55dc3fa2b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.519+0000 7f15cfe00640 1 --2- 172.21.15.47:0/867803705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa2e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.520+0000 7f15cea00640 1 -- 172.21.15.47:0/867803705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1f3a0 msgr2=0x55dc3fa2e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.520+0000 7f15cea00640 1 --2- 172.21.15.47:0/867803705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa2e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.520+0000 7f15cea00640 1 -- 172.21.15.47:0/867803705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1d810 msgr2=0x55dc3fa2b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.520+0000 7f15cea00640 1 --2- 172.21.15.47:0/867803705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1d810 0x55dc3fa2b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.520+0000 7f15cea00640 1 -- 172.21.15.47:0/867803705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc3fa2eb70 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.520+0000 7f15cea00640 1 --2- 172.21.15.47:0/867803705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa2b5d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f15bc010820 tx=0x7f15bc010bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fcac5b393c8b4b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.520+0000 7f15b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.520+0000 7f15b8c00640 1 -- 172.21.15.47:0/867803705 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc0050d0 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.520+0000 7f15d8f9f740 1 -- 172.21.15.47:0/867803705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15bc0055b0 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.520+0000 7f15b8c00640 1 -- 172.21.15.47:0/867803705 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f15bc022040 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.520+0000 7f15b8c00640 1 -- 172.21.15.47:0/867803705 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15bc03a050 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.520+0000 7f15d8f9f740 10 client.26458 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.521+0000 7f15d8f9f740 1 -- 172.21.15.47:0/867803705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dc3fa8f730 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.521+0000 7f15b8c00640 1 -- 172.21.15.47:0/867803705 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15bc0055b0 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.521+0000 7f15b8c00640 1 -- 172.21.15.47:0/867803705 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f15bc003850 con 0x55dc3fa1e9e0 2024-03-21T11:26:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.521+0000 7f15d8f9f740 10 client.26458 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.521+0000 7f15d8f9f740 10 client.26458 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:26:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.521+0000 7f15d8f9f740 20 client.26458 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.521+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/867803705 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dc3fa90090 0x55dc3fa924b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.521+0000 7f15d8f9f740 4 client.26458 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:26:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.521+0000 7f15d8f9f740 1 -- 172.21.15.47:0/867803705 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dc3f989b50 con 0x55dc3fa90090 2024-03-21T11:26:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.521+0000 7f15cfe00640 1 --2- 172.21.15.47:0/867803705 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dc3fa90090 0x55dc3fa924b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15cfe00640 1 --2- 172.21.15.47:0/867803705 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dc3fa90090 0x55dc3fa924b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=17991e5eaccdbe52 server_cookie=d0fd32e8fa415936 in_seq=0 out_seq=0 2024-03-21T11:26:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15b8c00640 10 client.26458 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:26:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15b8c00640 1 -- 172.21.15.47:0/867803705 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dc3f989b50 con 0x55dc3fa90090 2024-03-21T11:26:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15b8c00640 10 client.26458 handle_command_reply: tid=0 2024-03-21T11:26:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15d8f9f740 1 client.26458 shutdown 2024-03-21T11:26:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15d8f9f740 1 -- 172.21.15.47:0/867803705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1e9e0 msgr2=0x55dc3fa2b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/867803705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa2b5d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f15bc010820 tx=0x7f15bc010bf0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15d8f9f740 1 -- 172.21.15.47:0/867803705 shutdown_connections 2024-03-21T11:26:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/867803705 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dc3fa90090 0x55dc3fa924b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/867803705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc3fa1f3a0 0x55dc3fa2e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/867803705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc3fa1e9e0 0x55dc3fa2b5d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15d8f9f740 1 --2- 172.21.15.47:0/867803705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc3fa1d810 0x55dc3fa2b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15d8f9f740 1 -- 172.21.15.47:0/867803705 >> 172.21.15.47:0/867803705 conn(0x55dc3fa18c00 msgr2=0x55dc3fa17620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15b8c00640 0 client.26458 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:26:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.522+0000 7f15d8f9f740 1 -- 172.21.15.47:0/867803705 shutdown_connections 2024-03-21T11:26:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.523+0000 7f15d8f9f740 1 -- 172.21.15.47:0/867803705 wait complete. 2024-03-21T11:26:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.523+0000 7f15d8f9f740 20 client.26458 trim_cache size 0 max 16384 2024-03-21T11:26:36.534 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:26:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.523+0000 7f15d6200640 1 -- 172.21.15.47:0/1400428344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15b40a2f20 msgr2=0x7f15b40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.523+0000 7f15d6200640 1 --2- 172.21.15.47:0/1400428344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15b40a2f20 0x7f15b40a53e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f15c00b9fc0 tx=0x7f15c00f7550 comp rx=0 tx=0).stop 2024-03-21T11:26:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.523+0000 7f15d6200640 1 -- 172.21.15.47:0/1400428344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0102120 msgr2=0x7f15d0199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.523+0000 7f15d6200640 1 --2- 172.21.15.47:0/1400428344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0102120 0x7f15d0199c30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f15c0000c00 tx=0x7f15c002c650 comp rx=0 tx=0).stop 2024-03-21T11:26:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.523+0000 7f15d6200640 1 -- 172.21.15.47:0/1400428344 shutdown_connections 2024-03-21T11:26:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.523+0000 7f15d6200640 1 --2- 172.21.15.47:0/1400428344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15b40a2f20 0x7f15b40a53e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.523+0000 7f15d6200640 1 --2- 172.21.15.47:0/1400428344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15d0107380 0x7f15d01a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.523+0000 7f15d6200640 1 --2- 172.21.15.47:0/1400428344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15d0104a50 0x7f15d019a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.523+0000 7f15d6200640 1 --2- 172.21.15.47:0/1400428344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0102120 0x7f15d0199c30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.523+0000 7f15d6200640 1 -- 172.21.15.47:0/1400428344 >> 172.21.15.47:0/1400428344 conn(0x7f15d00fbb30 msgr2=0x7f15d01018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.524+0000 7f15d6200640 1 -- 172.21.15.47:0/1400428344 shutdown_connections 2024-03-21T11:26:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.524+0000 7f15d6200640 1 -- 172.21.15.47:0/1400428344 wait complete. 2024-03-21T11:26:36.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:36 smithi047 ceph-mon[31351]: pgmap v498: 129 pgs: 129 active+clean; 34 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 304 op/s 2024-03-21T11:26:36.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:36 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2170970509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:26:36.542 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:26:36.542 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:26:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:36 smithi102 ceph-mon[36748]: pgmap v498: 129 pgs: 129 active+clean; 34 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 304 op/s 2024-03-21T11:26:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:36 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2170970509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:26:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.772+0000 7fda6d000640 1 -- 172.21.15.47:0/1241930169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda68075660 msgr2=0x7fda68073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.772+0000 7fda6d000640 1 --2- 172.21.15.47:0/1241930169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda68075660 0x7fda68073ab0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fda58009900 tx=0x7fda5802bcd0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.772+0000 7fda6d000640 1 -- 172.21.15.47:0/1241930169 shutdown_connections 2024-03-21T11:26:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.772+0000 7fda6d000640 1 --2- 172.21.15.47:0/1241930169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda68109560 0x7fda6810b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.772+0000 7fda6d000640 1 --2- 172.21.15.47:0/1241930169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda68074080 0x7fda680744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.772+0000 7fda6d000640 1 --2- 172.21.15.47:0/1241930169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda68075660 0x7fda68073ab0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.772+0000 7fda6d000640 1 -- 172.21.15.47:0/1241930169 >> 172.21.15.47:0/1241930169 conn(0x7fda680fcc00 msgr2=0x7fda680ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.772+0000 7fda6d000640 1 -- 172.21.15.47:0/1241930169 shutdown_connections 2024-03-21T11:26:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.773+0000 7fda6d000640 1 -- 172.21.15.47:0/1241930169 wait complete. 2024-03-21T11:26:36.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.774+0000 7fda6d000640 1 Processor -- start 2024-03-21T11:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.774+0000 7fda6d000640 1 -- start start 2024-03-21T11:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.774+0000 7fda6d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda68074080 0x7fda68199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.774+0000 7fda6d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda68075660 0x7fda6819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.774+0000 7fda6d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda68109560 0x7fda681a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.774+0000 7fda6d000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6810f560 con 0x7fda68109560 2024-03-21T11:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.774+0000 7fda6d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6810f3e0 con 0x7fda68074080 2024-03-21T11:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.774+0000 7fda6d000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6810f6e0 con 0x7fda68075660 2024-03-21T11:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.774+0000 7fda65e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda68074080 0x7fda68199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.774+0000 7fda65e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda68074080 0x7fda68199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46310/0 (socket says 172.21.15.47:46310) 2024-03-21T11:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.774+0000 7fda65e00640 1 -- 172.21.15.47:0/1531049446 learned_addr learned my addr 172.21.15.47:0/1531049446 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.774+0000 7fda65400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda68075660 0x7fda6819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.774+0000 7fda66800640 1 --2- 172.21.15.47:0/1531049446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda68109560 0x7fda681a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.775+0000 7fda65e00640 1 -- 172.21.15.47:0/1531049446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda68075660 msgr2=0x7fda6819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.775+0000 7fda65e00640 1 --2- 172.21.15.47:0/1531049446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda68075660 0x7fda6819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.775+0000 7fda65e00640 1 -- 172.21.15.47:0/1531049446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda68109560 msgr2=0x7fda681a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.775+0000 7fda65e00640 1 --2- 172.21.15.47:0/1531049446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda68109560 0x7fda681a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.775+0000 7fda65e00640 1 -- 172.21.15.47:0/1531049446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda681a1830 con 0x7fda68074080 2024-03-21T11:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.775+0000 7fda65400640 1 --2- 172.21.15.47:0/1531049446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda68075660 0x7fda6819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.775+0000 7fda65e00640 1 --2- 172.21.15.47:0/1531049446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda68074080 0x7fda68199c20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fda58000c00 tx=0x7fda5802c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.775+0000 7fda5ea00640 1 -- 172.21.15.47:0/1531049446 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58043020 con 0x7fda68074080 2024-03-21T11:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.775+0000 7fda5ea00640 1 -- 172.21.15.47:0/1531049446 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda580054e0 con 0x7fda68074080 2024-03-21T11:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.775+0000 7fda5ea00640 1 -- 172.21.15.47:0/1531049446 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58003c20 con 0x7fda68074080 2024-03-21T11:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.776+0000 7fda6d000640 1 -- 172.21.15.47:0/1531049446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda681a1ac0 con 0x7fda68074080 2024-03-21T11:26:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.776+0000 7fda6d000640 1 -- 172.21.15.47:0/1531049446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda681a1fc0 con 0x7fda68074080 2024-03-21T11:26:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.780+0000 7fda6fc16740 1 Processor -- start 2024-03-21T11:26:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.780+0000 7fda6fc16740 1 -- start start 2024-03-21T11:26:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.781+0000 7fda5ea00640 1 -- 172.21.15.47:0/1531049446 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fda58046020 con 0x7fda68074080 2024-03-21T11:26:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.781+0000 7fda6fc16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e3429c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.781+0000 7fda6fc16740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342a9e0 0x55f6e342ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.781+0000 7fda6fc16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342b3a0 0x55f6e3431c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.781+0000 7fda6fc16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e338e2c0 con 0x55f6e3429810 2024-03-21T11:26:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.781+0000 7fda6fc16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e3336c90 con 0x55f6e342b3a0 2024-03-21T11:26:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.781+0000 7fda6fc16740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e338a360 con 0x55f6e342a9e0 2024-03-21T11:26:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.781+0000 7fda65e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342a9e0 0x55f6e342ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.781+0000 7fda65400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342b3a0 0x55f6e3431c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.781+0000 7fda65400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342b3a0 0x55f6e3431c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46326/0 (socket says 172.21.15.47:46326) 2024-03-21T11:26:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.781+0000 7fda65400640 1 -- 172.21.15.47:0/2807025704 learned_addr learned my addr 172.21.15.47:0/2807025704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.781+0000 7fda65e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342a9e0 0x55f6e342ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54032/0 (socket says 172.21.15.47:54032) 2024-03-21T11:26:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.781+0000 7fda66800640 1 --2- 172.21.15.47:0/2807025704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e3429c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.782+0000 7fda65400640 1 -- 172.21.15.47:0/2807025704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342a9e0 msgr2=0x55f6e342ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.782+0000 7fda65400640 1 --2- 172.21.15.47:0/2807025704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342a9e0 0x55f6e342ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.782+0000 7fda65400640 1 -- 172.21.15.47:0/2807025704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 msgr2=0x55f6e3429c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.782+0000 7fda65400640 1 --2- 172.21.15.47:0/2807025704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e3429c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.782+0000 7fda65400640 1 -- 172.21.15.47:0/2807025704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6e335b080 con 0x55f6e342b3a0 2024-03-21T11:26:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.782+0000 7fda65e00640 1 --2- 172.21.15.47:0/2807025704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342a9e0 0x55f6e342ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:36.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.782+0000 7fda65400640 1 --2- 172.21.15.47:0/2807025704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342b3a0 0x55f6e3431c30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fda500115f0 tx=0x7fda50011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf8aa1536ea3c203 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.782+0000 7fda57400640 1 -- 172.21.15.47:0/2807025704 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5001e070 con 0x55f6e342b3a0 2024-03-21T11:26:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.782+0000 7fda57400640 1 -- 172.21.15.47:0/2807025704 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda5000fb20 con 0x55f6e342b3a0 2024-03-21T11:26:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.782+0000 7fda57400640 1 -- 172.21.15.47:0/2807025704 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50019c40 con 0x55f6e342b3a0 2024-03-21T11:26:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.783+0000 7fda5ea00640 1 --2- 172.21.15.47:0/1531049446 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda4c0a2fa0 0x7fda4c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.783+0000 7fda5ea00640 1 -- 172.21.15.47:0/1531049446 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fda580f1800 con 0x7fda68074080 2024-03-21T11:26:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.783+0000 7fda66800640 1 --2- 172.21.15.47:0/1531049446 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda4c0a2fa0 0x7fda4c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.783+0000 7fda6fc16740 1 -- 172.21.15.47:0/2807025704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342b3a0 msgr2=0x55f6e3431c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.783+0000 7fda6fc16740 1 --2- 172.21.15.47:0/2807025704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342b3a0 0x55f6e3431c30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fda500115f0 tx=0x7fda50011ab0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.783+0000 7fda6fc16740 1 -- 172.21.15.47:0/2807025704 shutdown_connections 2024-03-21T11:26:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.784+0000 7fda6fc16740 1 --2- 172.21.15.47:0/2807025704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342b3a0 0x55f6e3431c30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.784+0000 7fda6fc16740 1 --2- 172.21.15.47:0/2807025704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342a9e0 0x55f6e342ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.784+0000 7fda6fc16740 1 --2- 172.21.15.47:0/2807025704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e3429c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.784+0000 7fda6fc16740 1 -- 172.21.15.47:0/2807025704 >> 172.21.15.47:0/2807025704 conn(0x55f6e3424c00 msgr2=0x55f6e3425000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:36.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.784+0000 7fda6fc16740 1 -- 172.21.15.47:0/2807025704 shutdown_connections 2024-03-21T11:26:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.784+0000 7fda66800640 1 --2- 172.21.15.47:0/1531049446 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda4c0a2fa0 0x7fda4c0a5460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55f6e342a8e0 tx=0x7fda60009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:36.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.784+0000 7fda6fc16740 1 -- 172.21.15.47:0/2807025704 wait complete. 2024-03-21T11:26:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.784+0000 7fda6fc16740 1 -- start start 2024-03-21T11:26:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.784+0000 7fda6fc16740 1 Processor -- start 2024-03-21T11:26:36.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda6fc16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e3433ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda6fc16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e3434020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda6fc16740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 0x55f6e343b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda6fc16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e34323b0 con 0x55f6e3429810 2024-03-21T11:26:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda6fc16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e334a0d0 con 0x55f6e342a9e0 2024-03-21T11:26:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda6fc16740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e3432530 con 0x55f6e342b3a0 2024-03-21T11:26:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda6fc16740 10 client.0 _ll_register_callbacks cb 0x55f6e334ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda65400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e3434020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda65e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e3433ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda66800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 0x55f6e343b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda65400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e3434020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46340/0 (socket says 172.21.15.47:46340) 2024-03-21T11:26:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda65400640 1 -- 172.21.15.47:0/4181210885 learned_addr learned my addr 172.21.15.47:0/4181210885 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda65400640 1 -- 172.21.15.47:0/4181210885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 msgr2=0x55f6e343b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda65400640 1 --2- 172.21.15.47:0/4181210885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 0x55f6e343b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda65400640 1 -- 172.21.15.47:0/4181210885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 msgr2=0x55f6e3433ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda65400640 1 --2- 172.21.15.47:0/4181210885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e3433ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda65400640 1 -- 172.21.15.47:0/4181210885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6e30a23f0 con 0x55f6e342a9e0 2024-03-21T11:26:36.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.785+0000 7fda65e00640 1 --2- 172.21.15.47:0/4181210885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e3433ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:36.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.786+0000 7fda65400640 1 --2- 172.21.15.47:0/4181210885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e3434020 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fda50005890 tx=0x7fda5000f870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5537daa8b70b0ddd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:36.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.786+0000 7fda4be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:36.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.786+0000 7fda4be00640 1 -- 172.21.15.47:0/4181210885 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50027060 con 0x55f6e342a9e0 2024-03-21T11:26:36.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.786+0000 7fda6fc16740 1 -- 172.21.15.47:0/4181210885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda5000fb20 con 0x55f6e342a9e0 2024-03-21T11:26:36.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.786+0000 7fda4be00640 1 -- 172.21.15.47:0/4181210885 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fda500199c0 con 0x55f6e342a9e0 2024-03-21T11:26:36.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.786+0000 7fda4be00640 1 -- 172.21.15.47:0/4181210885 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda5002e050 con 0x55f6e342a9e0 2024-03-21T11:26:36.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.786+0000 7fda6fc16740 10 client.26453 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:36.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.786+0000 7fda6fc16740 1 -- 172.21.15.47:0/4181210885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f6e316c7d0 con 0x55f6e342a9e0 2024-03-21T11:26:36.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.787+0000 7fda4be00640 1 -- 172.21.15.47:0/4181210885 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda5000fb20 con 0x55f6e342a9e0 2024-03-21T11:26:36.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.787+0000 7fda4be00640 1 -- 172.21.15.47:0/4181210885 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fda50021070 con 0x55f6e342a9e0 2024-03-21T11:26:36.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.787+0000 7fda6fc16740 10 client.26453 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:36.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.787+0000 7fda6fc16740 10 client.26453 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:26:36.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.787+0000 7fda6fc16740 20 client.26453 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:36.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.787+0000 7fda6fc16740 1 --2- 172.21.15.47:0/4181210885 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55f6e343f1c0 0x55f6e34415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.787+0000 7fda6fc16740 4 client.26453 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:36.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.787+0000 7fda6fc16740 1 -- 172.21.15.47:0/4181210885 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6e339b800 con 0x55f6e343f1c0 2024-03-21T11:26:36.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.787+0000 7fda65e00640 1 --2- 172.21.15.47:0/4181210885 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55f6e343f1c0 0x55f6e34415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.788+0000 7fda65e00640 1 --2- 172.21.15.47:0/4181210885 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55f6e343f1c0 0x55f6e34415d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=515d4ca9c3348133 server_cookie=1f041afd910028e4 in_seq=0 out_seq=0 2024-03-21T11:26:36.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.788+0000 7fda4be00640 10 client.26453 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:26:36.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.788+0000 7fda4be00640 1 -- 172.21.15.47:0/4181210885 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f6e339b800 con 0x55f6e343f1c0 2024-03-21T11:26:36.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.788+0000 7fda4be00640 10 client.26453 handle_command_reply: tid=0 2024-03-21T11:26:36.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.789+0000 7fda6fc16740 1 client.26453 shutdown 2024-03-21T11:26:36.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.789+0000 7fda6fc16740 1 -- 172.21.15.47:0/4181210885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 msgr2=0x55f6e3434020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.789+0000 7fda6fc16740 1 --2- 172.21.15.47:0/4181210885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e3434020 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fda50005890 tx=0x7fda5000f870 comp rx=0 tx=0).stop 2024-03-21T11:26:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.789+0000 7fda6fc16740 1 -- 172.21.15.47:0/4181210885 shutdown_connections 2024-03-21T11:26:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.789+0000 7fda6fc16740 1 --2- 172.21.15.47:0/4181210885 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55f6e343f1c0 0x55f6e34415d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.789+0000 7fda6fc16740 1 --2- 172.21.15.47:0/4181210885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 0x55f6e343b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.789+0000 7fda6fc16740 1 --2- 172.21.15.47:0/4181210885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e3434020 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.789+0000 7fda6fc16740 1 --2- 172.21.15.47:0/4181210885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e3433ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.789+0000 7fda6fc16740 1 -- 172.21.15.47:0/4181210885 >> 172.21.15.47:0/4181210885 conn(0x55f6e3424c00 msgr2=0x55f6e3423eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.789+0000 7fda4be00640 0 client.26453 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:26:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.789+0000 7fda6fc16740 1 -- 172.21.15.47:0/4181210885 shutdown_connections 2024-03-21T11:26:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.789+0000 7fda6fc16740 1 -- 172.21.15.47:0/4181210885 wait complete. 2024-03-21T11:26:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.789+0000 7fda6fc16740 20 client.26453 trim_cache size 0 max 16384 2024-03-21T11:26:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.804+0000 7fda6fc16740 1 Processor -- start 2024-03-21T11:26:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.804+0000 7fda6fc16740 1 -- start start 2024-03-21T11:26:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.804+0000 7fda6fc16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e34415b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.804+0000 7fda6fc16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e343b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.804+0000 7fda6fc16740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 0x55f6e3424fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.804+0000 7fda6fc16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda681ac310 con 0x55f6e3429810 2024-03-21T11:26:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.804+0000 7fda6fc16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda681adc90 con 0x55f6e342a9e0 2024-03-21T11:26:36.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.804+0000 7fda6fc16740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda681ac970 con 0x55f6e342b3a0 2024-03-21T11:26:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.804+0000 7fda65400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e343b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.804+0000 7fda65400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e343b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46350/0 (socket says 172.21.15.47:46350) 2024-03-21T11:26:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.804+0000 7fda65400640 1 -- 172.21.15.47:0/3943037396 learned_addr learned my addr 172.21.15.47:0/3943037396 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.804+0000 7fda66800640 1 --2- 172.21.15.47:0/3943037396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 0x55f6e3424fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.804+0000 7fda65e00640 1 --2- 172.21.15.47:0/3943037396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e34415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.805+0000 7fda65400640 1 -- 172.21.15.47:0/3943037396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 msgr2=0x55f6e3424fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.805+0000 7fda65400640 1 --2- 172.21.15.47:0/3943037396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 0x55f6e3424fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.805+0000 7fda65400640 1 -- 172.21.15.47:0/3943037396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 msgr2=0x55f6e34415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.805+0000 7fda65400640 1 --2- 172.21.15.47:0/3943037396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e34415b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.805+0000 7fda65400640 1 -- 172.21.15.47:0/3943037396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6e342a460 con 0x55f6e342a9e0 2024-03-21T11:26:36.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.805+0000 7fda65e00640 1 --2- 172.21.15.47:0/3943037396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e34415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:36.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.805+0000 7fda65400640 1 --2- 172.21.15.47:0/3943037396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e343b020 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fda50000c00 tx=0x7fda50004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca833badb17c68de server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:36.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.805+0000 7fda56a00640 1 -- 172.21.15.47:0/3943037396 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50027070 con 0x55f6e342a9e0 2024-03-21T11:26:36.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.805+0000 7fda56a00640 1 -- 172.21.15.47:0/3943037396 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda50005520 con 0x55f6e342a9e0 2024-03-21T11:26:36.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.805+0000 7fda56a00640 1 -- 172.21.15.47:0/3943037396 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5001e040 con 0x55f6e342a9e0 2024-03-21T11:26:36.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.806+0000 7fda6fc16740 1 -- 172.21.15.47:0/3943037396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 msgr2=0x55f6e343b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.806+0000 7fda6fc16740 1 --2- 172.21.15.47:0/3943037396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e343b020 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fda50000c00 tx=0x7fda50004a40 comp rx=0 tx=0).stop 2024-03-21T11:26:36.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.806+0000 7fda6fc16740 1 -- 172.21.15.47:0/3943037396 shutdown_connections 2024-03-21T11:26:36.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.806+0000 7fda6fc16740 1 --2- 172.21.15.47:0/3943037396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 0x55f6e3424fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.806+0000 7fda6fc16740 1 --2- 172.21.15.47:0/3943037396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e343b020 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.806+0000 7fda6fc16740 1 --2- 172.21.15.47:0/3943037396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e34415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.806+0000 7fda6fc16740 1 -- 172.21.15.47:0/3943037396 >> 172.21.15.47:0/3943037396 conn(0x55f6e3424c00 msgr2=0x55f6e34286f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.806+0000 7fda6fc16740 1 -- 172.21.15.47:0/3943037396 shutdown_connections 2024-03-21T11:26:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.806+0000 7fda6fc16740 1 -- 172.21.15.47:0/3943037396 wait complete. 2024-03-21T11:26:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.807+0000 7fda6fc16740 1 -- start start 2024-03-21T11:26:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.807+0000 7fda6fc16740 1 Processor -- start 2024-03-21T11:26:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.807+0000 7fda6fc16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e3436dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.807+0000 7fda6fc16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e3437310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.807+0000 7fda6fc16740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 0x55f6e343a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.807+0000 7fda6fc16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda681ad630 con 0x55f6e3429810 2024-03-21T11:26:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.807+0000 7fda6fc16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda681acfd0 con 0x55f6e342a9e0 2024-03-21T11:26:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.807+0000 7fda6fc16740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda681aa4d0 con 0x55f6e342b3a0 2024-03-21T11:26:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.808+0000 7fda6fc16740 10 client.0 _ll_register_callbacks cb 0x7fda5001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.808+0000 7fda65400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e3437310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.808+0000 7fda65400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e3437310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46358/0 (socket says 172.21.15.47:46358) 2024-03-21T11:26:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.808+0000 7fda65400640 1 -- 172.21.15.47:0/1594308108 learned_addr learned my addr 172.21.15.47:0/1594308108 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.808+0000 7fda66800640 1 --2- 172.21.15.47:0/1594308108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 0x55f6e343a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.808+0000 7fda65e00640 1 --2- 172.21.15.47:0/1594308108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e3436dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.808+0000 7fda65400640 1 -- 172.21.15.47:0/1594308108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 msgr2=0x55f6e343a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.808+0000 7fda65400640 1 --2- 172.21.15.47:0/1594308108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 0x55f6e343a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.808+0000 7fda65400640 1 -- 172.21.15.47:0/1594308108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 msgr2=0x55f6e3436dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.808+0000 7fda65400640 1 --2- 172.21.15.47:0/1594308108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e3436dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.808+0000 7fda65400640 1 -- 172.21.15.47:0/1594308108 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6e343a880 con 0x55f6e342a9e0 2024-03-21T11:26:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.808+0000 7fda66800640 1 --2- 172.21.15.47:0/1594308108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 0x55f6e343a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.809+0000 7fda65400640 1 --2- 172.21.15.47:0/1594308108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e3437310 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fda50011510 tx=0x7fda50009d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfe331c75a28dfe6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.809+0000 7fda4be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.809+0000 7fda4be00640 1 -- 172.21.15.47:0/1594308108 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50027070 con 0x55f6e342a9e0 2024-03-21T11:26:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.809+0000 7fda6fc16740 1 -- 172.21.15.47:0/1594308108 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda50005520 con 0x55f6e342a9e0 2024-03-21T11:26:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.809+0000 7fda4be00640 1 -- 172.21.15.47:0/1594308108 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fda50004d00 con 0x55f6e342a9e0 2024-03-21T11:26:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.809+0000 7fda4be00640 1 -- 172.21.15.47:0/1594308108 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda50044050 con 0x55f6e342a9e0 2024-03-21T11:26:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.809+0000 7fda6fc16740 10 client.26459 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:36.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.809+0000 7fda6fc16740 1 -- 172.21.15.47:0/1594308108 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f6e34a3840 con 0x55f6e342a9e0 2024-03-21T11:26:36.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.810+0000 7fda4be00640 1 -- 172.21.15.47:0/1594308108 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda50005520 con 0x55f6e342a9e0 2024-03-21T11:26:36.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.810+0000 7fda4be00640 1 -- 172.21.15.47:0/1594308108 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fda50021040 con 0x55f6e342a9e0 2024-03-21T11:26:36.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.810+0000 7fda6fc16740 10 client.26459 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:36.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.810+0000 7fda6fc16740 10 client.26459 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:26:36.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.810+0000 7fda6fc16740 20 client.26459 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:36.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.810+0000 7fda6fc16740 1 --2- 172.21.15.47:0/1594308108 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55f6e34a4100 0x55f6e34a6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:36.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.810+0000 7fda6fc16740 4 client.26459 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:26:36.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.810+0000 7fda6fc16740 1 -- 172.21.15.47:0/1594308108 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f6e3395b50 con 0x55f6e34a4100 2024-03-21T11:26:36.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.810+0000 7fda65e00640 1 --2- 172.21.15.47:0/1594308108 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55f6e34a4100 0x55f6e34a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:36.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda65e00640 1 --2- 172.21.15.47:0/1594308108 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55f6e34a4100 0x55f6e34a6520 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=320722961cc377f9 server_cookie=9ffaf5b52a49569c in_seq=0 out_seq=0 2024-03-21T11:26:36.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda4be00640 10 client.26459 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:26:36.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda4be00640 1 -- 172.21.15.47:0/1594308108 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f6e3395b50 con 0x55f6e34a4100 2024-03-21T11:26:36.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda4be00640 10 client.26459 handle_command_reply: tid=0 2024-03-21T11:26:36.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda6fc16740 1 client.26459 shutdown 2024-03-21T11:26:36.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda6fc16740 1 -- 172.21.15.47:0/1594308108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 msgr2=0x55f6e3437310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda6fc16740 1 --2- 172.21.15.47:0/1594308108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e3437310 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fda50011510 tx=0x7fda50009d80 comp rx=0 tx=0).stop 2024-03-21T11:26:36.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda6fc16740 1 -- 172.21.15.47:0/1594308108 shutdown_connections 2024-03-21T11:26:36.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda6fc16740 1 --2- 172.21.15.47:0/1594308108 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55f6e34a4100 0x55f6e34a6520 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda6fc16740 1 --2- 172.21.15.47:0/1594308108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6e342b3a0 0x55f6e343a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda6fc16740 1 --2- 172.21.15.47:0/1594308108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f6e342a9e0 0x55f6e3437310 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda6fc16740 1 --2- 172.21.15.47:0/1594308108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f6e3429810 0x55f6e3436dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda4be00640 0 client.26459 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:26:36.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda6fc16740 1 -- 172.21.15.47:0/1594308108 >> 172.21.15.47:0/1594308108 conn(0x55f6e3424c00 msgr2=0x55f6e34286f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:36.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda6fc16740 1 -- 172.21.15.47:0/1594308108 shutdown_connections 2024-03-21T11:26:36.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda6fc16740 1 -- 172.21.15.47:0/1594308108 wait complete. 2024-03-21T11:26:36.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.811+0000 7fda6fc16740 20 client.26459 trim_cache size 0 max 16384 2024-03-21T11:26:36.820 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:26:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.812+0000 7fda6d000640 1 -- 172.21.15.47:0/1531049446 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda4c0a2fa0 msgr2=0x7fda4c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.812+0000 7fda6d000640 1 --2- 172.21.15.47:0/1531049446 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda4c0a2fa0 0x7fda4c0a5460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55f6e342a8e0 tx=0x7fda60009500 comp rx=0 tx=0).stop 2024-03-21T11:26:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.812+0000 7fda6d000640 1 -- 172.21.15.47:0/1531049446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda68074080 msgr2=0x7fda68199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.812+0000 7fda6d000640 1 --2- 172.21.15.47:0/1531049446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda68074080 0x7fda68199c20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fda58000c00 tx=0x7fda5802c6f0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.812+0000 7fda6d000640 1 -- 172.21.15.47:0/1531049446 shutdown_connections 2024-03-21T11:26:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.812+0000 7fda6d000640 1 --2- 172.21.15.47:0/1531049446 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda4c0a2fa0 0x7fda4c0a5460 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.812+0000 7fda6d000640 1 --2- 172.21.15.47:0/1531049446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda68109560 0x7fda681a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.812+0000 7fda6d000640 1 --2- 172.21.15.47:0/1531049446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda68075660 0x7fda6819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.812+0000 7fda6d000640 1 --2- 172.21.15.47:0/1531049446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda68074080 0x7fda68199c20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.812+0000 7fda6d000640 1 -- 172.21.15.47:0/1531049446 >> 172.21.15.47:0/1531049446 conn(0x7fda680fcc00 msgr2=0x7fda681054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.812+0000 7fda6d000640 1 -- 172.21.15.47:0/1531049446 shutdown_connections 2024-03-21T11:26:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:36.812+0000 7fda6d000640 1 -- 172.21.15.47:0/1531049446 wait complete. 2024-03-21T11:26:36.836 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:26:36.837 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:26:36.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:36 smithi100 ceph-mon[36245]: pgmap v498: 129 pgs: 129 active+clean; 34 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 304 op/s 2024-03-21T11:26:36.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:36 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2170970509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:26:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.058+0000 7fed5c200640 1 -- 172.21.15.47:0/2170858298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed541021c0 msgr2=0x7fed541045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.058+0000 7fed5c200640 1 --2- 172.21.15.47:0/2170858298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed541021c0 0x7fed541045b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fed48009900 tx=0x7fed4802bc30 comp rx=0 tx=0).stop 2024-03-21T11:26:37.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.058+0000 7fed5c200640 1 -- 172.21.15.47:0/2170858298 shutdown_connections 2024-03-21T11:26:37.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.058+0000 7fed5c200640 1 --2- 172.21.15.47:0/2170858298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed54107420 0x7fed54109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.058+0000 7fed5c200640 1 --2- 172.21.15.47:0/2170858298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed54104af0 0x7fed54106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.058+0000 7fed5c200640 1 --2- 172.21.15.47:0/2170858298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed541021c0 0x7fed541045b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.058+0000 7fed5c200640 1 -- 172.21.15.47:0/2170858298 >> 172.21.15.47:0/2170858298 conn(0x7fed540fbb30 msgr2=0x7fed540fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.058+0000 7fed5c200640 1 -- 172.21.15.47:0/2170858298 shutdown_connections 2024-03-21T11:26:37.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.058+0000 7fed5c200640 1 -- 172.21.15.47:0/2170858298 wait complete. 2024-03-21T11:26:37.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.059+0000 7fed5c200640 1 Processor -- start 2024-03-21T11:26:37.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.059+0000 7fed5c200640 1 -- start start 2024-03-21T11:26:37.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed5c200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed541021c0 0x7fed54197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed5c200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed54104af0 0x7fed54197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed5c200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed54107420 0x7fed5419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed5c200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5410d360 con 0x7fed54107420 2024-03-21T11:26:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed5c200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5410d1e0 con 0x7fed54104af0 2024-03-21T11:26:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed5c200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5410d4e0 con 0x7fed541021c0 2024-03-21T11:26:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed59800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed541021c0 0x7fed54197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed58e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed54104af0 0x7fed54197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed59800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed541021c0 0x7fed54197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54092/0 (socket says 172.21.15.47:54092) 2024-03-21T11:26:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed59800640 1 -- 172.21.15.47:0/882216778 learned_addr learned my addr 172.21.15.47:0/882216778 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed5a200640 1 --2- 172.21.15.47:0/882216778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed54107420 0x7fed5419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed59800640 1 -- 172.21.15.47:0/882216778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed54104af0 msgr2=0x7fed54197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed59800640 1 --2- 172.21.15.47:0/882216778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed54104af0 0x7fed54197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed59800640 1 -- 172.21.15.47:0/882216778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed54107420 msgr2=0x7fed5419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed59800640 1 --2- 172.21.15.47:0/882216778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed54107420 0x7fed5419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.060+0000 7fed59800640 1 -- 172.21.15.47:0/882216778 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5419f630 con 0x7fed541021c0 2024-03-21T11:26:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.061+0000 7fed58e00640 1 --2- 172.21.15.47:0/882216778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed54104af0 0x7fed54197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.061+0000 7fed59800640 1 --2- 172.21.15.47:0/882216778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed541021c0 0x7fed54197a10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fed48000c00 tx=0x7fed4802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.061+0000 7fed52000640 1 -- 172.21.15.47:0/882216778 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48043020 con 0x7fed541021c0 2024-03-21T11:26:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.061+0000 7fed52000640 1 -- 172.21.15.47:0/882216778 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed480054e0 con 0x7fed541021c0 2024-03-21T11:26:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.061+0000 7fed5c200640 1 -- 172.21.15.47:0/882216778 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5419f8c0 con 0x7fed541021c0 2024-03-21T11:26:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.061+0000 7fed52000640 1 -- 172.21.15.47:0/882216778 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48003c40 con 0x7fed541021c0 2024-03-21T11:26:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.061+0000 7fed5c200640 1 -- 172.21.15.47:0/882216778 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5419fe10 con 0x7fed541021c0 2024-03-21T11:26:37.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.065+0000 7fed5ee7c740 1 Processor -- start 2024-03-21T11:26:37.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.065+0000 7fed52000640 1 -- 172.21.15.47:0/882216778 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fed48046020 con 0x7fed541021c0 2024-03-21T11:26:37.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.065+0000 7fed5ee7c740 1 -- start start 2024-03-21T11:26:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.065+0000 7fed5ee7c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2c0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed5ee7c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed5ee7c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed5ee7c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1c2252c0 con 0x55dd1c2c23a0 2024-03-21T11:26:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed5ee7c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1c1cdc90 con 0x55dd1c2c19e0 2024-03-21T11:26:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed5ee7c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1c221360 con 0x55dd1c2c0810 2024-03-21T11:26:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed5a200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2c0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed5a200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2c0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54106/0 (socket says 172.21.15.47:54106) 2024-03-21T11:26:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed5a200640 1 -- 172.21.15.47:0/2911935157 learned_addr learned my addr 172.21.15.47:0/2911935157 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed59800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed58e00640 1 --2- 172.21.15.47:0/2911935157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed5a200640 1 -- 172.21.15.47:0/2911935157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c19e0 msgr2=0x55dd1c2c1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed5a200640 1 --2- 172.21.15.47:0/2911935157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2c1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed5a200640 1 -- 172.21.15.47:0/2911935157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c23a0 msgr2=0x55dd1c2c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed5a200640 1 --2- 172.21.15.47:0/2911935157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.066+0000 7fed5a200640 1 -- 172.21.15.47:0/2911935157 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd1c1f2080 con 0x55dd1c2c0810 2024-03-21T11:26:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.067+0000 7fed5a200640 1 --2- 172.21.15.47:0/2911935157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2c0c30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009fc0 tx=0x7fed4c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e571eb459e59e73 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.067+0000 7fed46a00640 1 -- 172.21.15.47:0/2911935157 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c003b70 con 0x55dd1c2c0810 2024-03-21T11:26:37.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.067+0000 7fed46a00640 1 -- 172.21.15.47:0/2911935157 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed4c003d10 con 0x55dd1c2c0810 2024-03-21T11:26:37.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.067+0000 7fed46a00640 1 -- 172.21.15.47:0/2911935157 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c011aa0 con 0x55dd1c2c0810 2024-03-21T11:26:37.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.067+0000 7fed52000640 1 --2- 172.21.15.47:0/882216778 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed3c0a2f20 0x7fed3c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.068+0000 7fed52000640 1 -- 172.21.15.47:0/882216778 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fed480f18e0 con 0x7fed541021c0 2024-03-21T11:26:37.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.068+0000 7fed59800640 1 --2- 172.21.15.47:0/882216778 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed3c0a2f20 0x7fed3c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.068+0000 7fed5ee7c740 1 -- 172.21.15.47:0/2911935157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 msgr2=0x55dd1c2c0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.068+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/2911935157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2c0c30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009fc0 tx=0x7fed4c010860 comp rx=0 tx=0).stop 2024-03-21T11:26:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.068+0000 7fed5ee7c740 1 -- 172.21.15.47:0/2911935157 shutdown_connections 2024-03-21T11:26:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.068+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/2911935157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.068+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/2911935157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2c1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.068+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/2911935157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2c0c30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.068+0000 7fed5ee7c740 1 -- 172.21.15.47:0/2911935157 >> 172.21.15.47:0/2911935157 conn(0x55dd1c2bbc00 msgr2=0x55dd1c2bc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.068+0000 7fed5ee7c740 1 -- 172.21.15.47:0/2911935157 shutdown_connections 2024-03-21T11:26:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.068+0000 7fed5ee7c740 1 -- 172.21.15.47:0/2911935157 wait complete. 2024-03-21T11:26:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.068+0000 7fed59800640 1 --2- 172.21.15.47:0/882216778 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed3c0a2f20 0x7fed3c0a53e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fed480f1b80 tx=0x7fed480f7530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.069+0000 7fed5ee7c740 1 -- start start 2024-03-21T11:26:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.069+0000 7fed5ee7c740 1 Processor -- start 2024-03-21T11:26:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.069+0000 7fed5ee7c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.069+0000 7fed5ee7c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.069+0000 7fed5ee7c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2d6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.069+0000 7fed5ee7c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1c2c93b0 con 0x55dd1c2c23a0 2024-03-21T11:26:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.069+0000 7fed5ee7c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1c1e10d0 con 0x55dd1c2c19e0 2024-03-21T11:26:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.069+0000 7fed5ee7c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1c2c9530 con 0x55dd1c2c0810 2024-03-21T11:26:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.069+0000 7fed5ee7c740 10 client.0 _ll_register_callbacks cb 0x55dd1c1e5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.070+0000 7fed58e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.070+0000 7fed58e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2cf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46406/0 (socket says 172.21.15.47:46406) 2024-03-21T11:26:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.070+0000 7fed58e00640 1 -- 172.21.15.47:0/3063051521 learned_addr learned my addr 172.21.15.47:0/3063051521 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.070+0000 7fed5a200640 1 --2- 172.21.15.47:0/3063051521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.070+0000 7fed5a200640 1 --2- 172.21.15.47:0/3063051521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2d6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.070+0000 7fed58e00640 1 -- 172.21.15.47:0/3063051521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 msgr2=0x55dd1c2ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.070+0000 7fed58e00640 1 --2- 172.21.15.47:0/3063051521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.070+0000 7fed58e00640 1 -- 172.21.15.47:0/3063051521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c23a0 msgr2=0x55dd1c2d6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.070+0000 7fed58e00640 1 --2- 172.21.15.47:0/3063051521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2d6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.070+0000 7fed58e00640 1 -- 172.21.15.47:0/3063051521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd1bf393f0 con 0x55dd1c2c19e0 2024-03-21T11:26:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.070+0000 7fed5a200640 1 --2- 172.21.15.47:0/3063051521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.070+0000 7fed58e00640 1 --2- 172.21.15.47:0/3063051521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2cf400 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fed40012d20 tx=0x7fed4000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b698128c99760a99 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.070+0000 7fed3b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.071+0000 7fed3b400640 1 -- 172.21.15.47:0/3063051521 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40012110 con 0x55dd1c2c19e0 2024-03-21T11:26:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.071+0000 7fed5ee7c740 1 -- 172.21.15.47:0/3063051521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed4c003d10 con 0x55dd1c2c19e0 2024-03-21T11:26:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.071+0000 7fed3b400640 1 -- 172.21.15.47:0/3063051521 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fed40004050 con 0x55dd1c2c19e0 2024-03-21T11:26:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.071+0000 7fed3b400640 1 -- 172.21.15.47:0/3063051521 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed400122b0 con 0x55dd1c2c19e0 2024-03-21T11:26:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.071+0000 7fed5ee7c740 10 client.26471 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.071+0000 7fed5ee7c740 1 -- 172.21.15.47:0/3063051521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dd1c0037d0 con 0x55dd1c2c19e0 2024-03-21T11:26:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.071+0000 7fed3b400640 1 -- 172.21.15.47:0/3063051521 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed4c003d10 con 0x55dd1c2c19e0 2024-03-21T11:26:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.071+0000 7fed3b400640 1 -- 172.21.15.47:0/3063051521 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fed40010430 con 0x55dd1c2c19e0 2024-03-21T11:26:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.072+0000 7fed5ee7c740 10 client.26471 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.072+0000 7fed5ee7c740 10 client.26471 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:26:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.072+0000 7fed5ee7c740 20 client.26471 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.072+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/3063051521 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55dd1c2da980 0x55dd1c2dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.072+0000 7fed5ee7c740 4 client.26471 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.072+0000 7fed5ee7c740 1 -- 172.21.15.47:0/3063051521 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd1c232800 con 0x55dd1c2da980 2024-03-21T11:26:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.072+0000 7fed5a200640 1 --2- 172.21.15.47:0/3063051521 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55dd1c2da980 0x55dd1c2dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.072+0000 7fed5a200640 1 --2- 172.21.15.47:0/3063051521 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55dd1c2da980 0x55dd1c2dcda0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=506a772db9c4567 server_cookie=72969d673216f28e in_seq=0 out_seq=0 2024-03-21T11:26:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.072+0000 7fed3b400640 10 client.26471 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:26:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.073+0000 7fed3b400640 1 -- 172.21.15.47:0/3063051521 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dd1c232800 con 0x55dd1c2da980 2024-03-21T11:26:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.073+0000 7fed3b400640 10 client.26471 handle_command_reply: tid=0 2024-03-21T11:26:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.073+0000 7fed5ee7c740 1 client.26471 shutdown 2024-03-21T11:26:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.074+0000 7fed5ee7c740 1 -- 172.21.15.47:0/3063051521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c19e0 msgr2=0x55dd1c2cf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.074+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/3063051521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2cf400 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fed40012d20 tx=0x7fed4000f790 comp rx=0 tx=0).stop 2024-03-21T11:26:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.074+0000 7fed5ee7c740 1 -- 172.21.15.47:0/3063051521 shutdown_connections 2024-03-21T11:26:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.074+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/3063051521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2d6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.074+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/3063051521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2cf400 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.074+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/3063051521 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55dd1c2da980 0x55dd1c2dcda0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.074+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/3063051521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.074+0000 7fed5ee7c740 1 -- 172.21.15.47:0/3063051521 >> 172.21.15.47:0/3063051521 conn(0x55dd1c2bbc00 msgr2=0x55dd1c2baeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.074+0000 7fed3b400640 0 client.26471 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:26:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.074+0000 7fed5ee7c740 1 -- 172.21.15.47:0/3063051521 shutdown_connections 2024-03-21T11:26:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.074+0000 7fed5ee7c740 1 -- 172.21.15.47:0/3063051521 wait complete. 2024-03-21T11:26:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.074+0000 7fed5ee7c740 20 client.26471 trim_cache size 0 max 16384 2024-03-21T11:26:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.090+0000 7fed5ee7c740 1 Processor -- start 2024-03-21T11:26:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.090+0000 7fed5ee7c740 1 -- start start 2024-03-21T11:26:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.090+0000 7fed5ee7c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c0810 0x55dd1c2bf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.090+0000 7fed5ee7c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.090+0000 7fed5ee7c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.090+0000 7fed5ee7c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed541a7f50 con 0x55dd1c2c0810 2024-03-21T11:26:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.090+0000 7fed5ee7c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed541a98d0 con 0x55dd1c2c23a0 2024-03-21T11:26:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.090+0000 7fed5ee7c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed541a85b0 con 0x55dd1c2c19e0 2024-03-21T11:26:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.091+0000 7fed58e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.091+0000 7fed58e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2c4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54124/0 (socket says 172.21.15.47:54124) 2024-03-21T11:26:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.091+0000 7fed58e00640 1 -- 172.21.15.47:0/72742637 learned_addr learned my addr 172.21.15.47:0/72742637 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.091+0000 7fed5a200640 1 --2- 172.21.15.47:0/72742637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c0810 0x55dd1c2bf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.091+0000 7fed5a200640 1 --2- 172.21.15.47:0/72742637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.091+0000 7fed58e00640 1 -- 172.21.15.47:0/72742637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c23a0 msgr2=0x55dd1c2c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.091+0000 7fed58e00640 1 --2- 172.21.15.47:0/72742637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.091+0000 7fed58e00640 1 -- 172.21.15.47:0/72742637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c0810 msgr2=0x55dd1c2bf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.091+0000 7fed58e00640 1 --2- 172.21.15.47:0/72742637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c0810 0x55dd1c2bf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.091+0000 7fed58e00640 1 -- 172.21.15.47:0/72742637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd1c2b7700 con 0x55dd1c2c19e0 2024-03-21T11:26:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.091+0000 7fed5a200640 1 --2- 172.21.15.47:0/72742637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:26:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.091+0000 7fed58e00640 1 --2- 172.21.15.47:0/72742637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2c4a80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fed40010820 tx=0x7fed40001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5c0e6a41fcc5536 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.092+0000 7fed46000640 1 -- 172.21.15.47:0/72742637 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40010a90 con 0x55dd1c2c19e0 2024-03-21T11:26:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.092+0000 7fed46000640 1 -- 172.21.15.47:0/72742637 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed4002e070 con 0x55dd1c2c19e0 2024-03-21T11:26:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.092+0000 7fed46000640 1 -- 172.21.15.47:0/72742637 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40003850 con 0x55dd1c2c19e0 2024-03-21T11:26:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.092+0000 7fed5ee7c740 1 -- 172.21.15.47:0/72742637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c19e0 msgr2=0x55dd1c2c4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.092+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/72742637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2c4a80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fed40010820 tx=0x7fed40001e70 comp rx=0 tx=0).stop 2024-03-21T11:26:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.092+0000 7fed5ee7c740 1 -- 172.21.15.47:0/72742637 shutdown_connections 2024-03-21T11:26:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.092+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/72742637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.092+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/72742637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2c4a80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.092+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/72742637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c0810 0x55dd1c2bf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.092+0000 7fed5ee7c740 1 -- 172.21.15.47:0/72742637 >> 172.21.15.47:0/72742637 conn(0x55dd1c2bbc00 msgr2=0x55dd1c2bee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.093+0000 7fed5ee7c740 1 -- 172.21.15.47:0/72742637 shutdown_connections 2024-03-21T11:26:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.093+0000 7fed5ee7c740 1 -- 172.21.15.47:0/72742637 wait complete. 2024-03-21T11:26:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.093+0000 7fed5ee7c740 1 -- start start 2024-03-21T11:26:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.093+0000 7fed5ee7c740 1 Processor -- start 2024-03-21T11:26:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5ee7c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5ee7c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5ee7c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2d6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5ee7c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed541a9270 con 0x55dd1c2c19e0 2024-03-21T11:26:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5ee7c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed541a8c10 con 0x55dd1c2c23a0 2024-03-21T11:26:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5ee7c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed541a6110 con 0x55dd1c2c0810 2024-03-21T11:26:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5ee7c740 10 client.0 _ll_register_callbacks cb 0x7fed40009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5a200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5a200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54130/0 (socket says 172.21.15.47:54130) 2024-03-21T11:26:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5a200640 1 -- 172.21.15.47:0/191613170 learned_addr learned my addr 172.21.15.47:0/191613170 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed58e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5a200640 1 --2- 172.21.15.47:0/191613170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2d6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5a200640 1 -- 172.21.15.47:0/191613170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c23a0 msgr2=0x55dd1c2d6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5a200640 1 --2- 172.21.15.47:0/191613170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2d6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5a200640 1 -- 172.21.15.47:0/191613170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c19e0 msgr2=0x55dd1c2ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5a200640 1 --2- 172.21.15.47:0/191613170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.094+0000 7fed5a200640 1 -- 172.21.15.47:0/191613170 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd1c2cc600 con 0x55dd1c2c0810 2024-03-21T11:26:37.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.095+0000 7fed5a200640 1 --2- 172.21.15.47:0/191613170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2cdf40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009ba0 tx=0x7fed4c01d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e10ef51035bdbf9d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.095+0000 7fed3b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:37.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.095+0000 7fed3b400640 1 -- 172.21.15.47:0/191613170 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c02dd70 con 0x55dd1c2c0810 2024-03-21T11:26:37.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.095+0000 7fed5ee7c740 1 -- 172.21.15.47:0/191613170 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed4002e070 con 0x55dd1c2c0810 2024-03-21T11:26:37.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.095+0000 7fed3b400640 1 -- 172.21.15.47:0/191613170 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fed4c023040 con 0x55dd1c2c0810 2024-03-21T11:26:37.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.095+0000 7fed3b400640 1 -- 172.21.15.47:0/191613170 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed4c02d0d0 con 0x55dd1c2c0810 2024-03-21T11:26:37.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.095+0000 7fed5ee7c740 10 client.26494 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:37.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.095+0000 7fed5ee7c740 1 -- 172.21.15.47:0/191613170 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dd1c33ad80 con 0x55dd1c2c0810 2024-03-21T11:26:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.095+0000 7fed3b400640 1 -- 172.21.15.47:0/191613170 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed4002e070 con 0x55dd1c2c0810 2024-03-21T11:26:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.096+0000 7fed3b400640 1 -- 172.21.15.47:0/191613170 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fed4c005680 con 0x55dd1c2c0810 2024-03-21T11:26:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.096+0000 7fed5ee7c740 10 client.26494 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.096+0000 7fed5ee7c740 10 client.26494 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:26:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.096+0000 7fed5ee7c740 20 client.26494 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.096+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/191613170 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55dd1c33b400 0x55dd1c33d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.096+0000 7fed5ee7c740 4 client.26494 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:26:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.096+0000 7fed5ee7c740 1 -- 172.21.15.47:0/191613170 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dd1c22cb50 con 0x55dd1c33b400 2024-03-21T11:26:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.096+0000 7fed58e00640 1 --2- 172.21.15.47:0/191613170 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55dd1c33b400 0x55dd1c33d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.096+0000 7fed58e00640 1 --2- 172.21.15.47:0/191613170 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55dd1c33b400 0x55dd1c33d7f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a01909af17340aff server_cookie=3f405fa1b98c4f02 in_seq=0 out_seq=0 2024-03-21T11:26:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.096+0000 7fed3b400640 10 client.26494 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:26:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed3b400640 1 -- 172.21.15.47:0/191613170 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dd1c22cb50 con 0x55dd1c33b400 2024-03-21T11:26:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed3b400640 10 client.26494 handle_command_reply: tid=0 2024-03-21T11:26:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed5ee7c740 1 client.26494 shutdown 2024-03-21T11:26:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed5ee7c740 1 -- 172.21.15.47:0/191613170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 msgr2=0x55dd1c2cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/191613170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2cdf40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009ba0 tx=0x7fed4c01d040 comp rx=0 tx=0).stop 2024-03-21T11:26:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed5ee7c740 1 -- 172.21.15.47:0/191613170 shutdown_connections 2024-03-21T11:26:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/191613170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd1c2c23a0 0x55dd1c2d6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/191613170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd1c2c19e0 0x55dd1c2ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/191613170 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55dd1c33b400 0x55dd1c33d7f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed5ee7c740 1 --2- 172.21.15.47:0/191613170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd1c2c0810 0x55dd1c2cdf40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed3b400640 0 client.26494 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:26:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed5ee7c740 1 -- 172.21.15.47:0/191613170 >> 172.21.15.47:0/191613170 conn(0x55dd1c2bbc00 msgr2=0x55dd1c2bf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed5ee7c740 1 -- 172.21.15.47:0/191613170 shutdown_connections 2024-03-21T11:26:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed5ee7c740 1 -- 172.21.15.47:0/191613170 wait complete. 2024-03-21T11:26:37.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.097+0000 7fed5ee7c740 20 client.26494 trim_cache size 0 max 16384 2024-03-21T11:26:37.104 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:26:37.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.098+0000 7fed5c200640 1 -- 172.21.15.47:0/882216778 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed3c0a2f20 msgr2=0x7fed3c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.098+0000 7fed5c200640 1 --2- 172.21.15.47:0/882216778 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed3c0a2f20 0x7fed3c0a53e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fed480f1b80 tx=0x7fed480f7530 comp rx=0 tx=0).stop 2024-03-21T11:26:37.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.098+0000 7fed5c200640 1 -- 172.21.15.47:0/882216778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed541021c0 msgr2=0x7fed54197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.098+0000 7fed5c200640 1 --2- 172.21.15.47:0/882216778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed541021c0 0x7fed54197a10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fed48000c00 tx=0x7fed4802c650 comp rx=0 tx=0).stop 2024-03-21T11:26:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.098+0000 7fed5c200640 1 -- 172.21.15.47:0/882216778 shutdown_connections 2024-03-21T11:26:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.098+0000 7fed5c200640 1 --2- 172.21.15.47:0/882216778 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed3c0a2f20 0x7fed3c0a53e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.098+0000 7fed5c200640 1 --2- 172.21.15.47:0/882216778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed54107420 0x7fed5419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.098+0000 7fed5c200640 1 --2- 172.21.15.47:0/882216778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed54104af0 0x7fed54197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.098+0000 7fed5c200640 1 --2- 172.21.15.47:0/882216778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed541021c0 0x7fed54197a10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.098+0000 7fed5c200640 1 -- 172.21.15.47:0/882216778 >> 172.21.15.47:0/882216778 conn(0x7fed540fbb30 msgr2=0x7fed54102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.098+0000 7fed5c200640 1 -- 172.21.15.47:0/882216778 shutdown_connections 2024-03-21T11:26:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.098+0000 7fed5c200640 1 -- 172.21.15.47:0/882216778 wait complete. 2024-03-21T11:26:37.119 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:26:37.119 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:26:37.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.344+0000 7fa62da00640 1 -- 172.21.15.47:0/2241479475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa628103260 msgr2=0x7fa628109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.344+0000 7fa62da00640 1 --2- 172.21.15.47:0/2241479475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa628103260 0x7fa628109af0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa620009be0 tx=0x7fa62002bef0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.344+0000 7fa62da00640 1 -- 172.21.15.47:0/2241479475 shutdown_connections 2024-03-21T11:26:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.344+0000 7fa62da00640 1 --2- 172.21.15.47:0/2241479475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa628103260 0x7fa628109af0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.344+0000 7fa62da00640 1 --2- 172.21.15.47:0/2241479475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6281028a0 0x7fa628102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.344+0000 7fa62da00640 1 --2- 172.21.15.47:0/2241479475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6281016b0 0x7fa628101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.344+0000 7fa62da00640 1 -- 172.21.15.47:0/2241479475 >> 172.21.15.47:0/2241479475 conn(0x7fa6280fcc40 msgr2=0x7fa6280ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.344+0000 7fa62da00640 1 -- 172.21.15.47:0/2241479475 shutdown_connections 2024-03-21T11:26:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.344+0000 7fa62da00640 1 -- 172.21.15.47:0/2241479475 wait complete. 2024-03-21T11:26:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.345+0000 7fa62da00640 1 Processor -- start 2024-03-21T11:26:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.346+0000 7fa62da00640 1 -- start start 2024-03-21T11:26:37.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.346+0000 7fa62da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6281016b0 0x7fa628075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.346+0000 7fa62da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6281028a0 0x7fa628073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.346+0000 7fa62da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa628103260 0x7fa628073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.346+0000 7fa62da00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62810d360 con 0x7fa6281028a0 2024-03-21T11:26:37.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.346+0000 7fa62da00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62810d1e0 con 0x7fa628103260 2024-03-21T11:26:37.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.346+0000 7fa62da00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62810d4e0 con 0x7fa6281016b0 2024-03-21T11:26:37.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.346+0000 7fa626a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6281016b0 0x7fa628075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.346+0000 7fa626a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6281016b0 0x7fa628075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54152/0 (socket says 172.21.15.47:54152) 2024-03-21T11:26:37.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.346+0000 7fa626a00640 1 -- 172.21.15.47:0/2936247943 learned_addr learned my addr 172.21.15.47:0/2936247943 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.346+0000 7fa61e000640 1 --2- 172.21.15.47:0/2936247943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6281028a0 0x7fa628073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.347+0000 7fa626a00640 1 -- 172.21.15.47:0/2936247943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa628103260 msgr2=0x7fa628073fd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T11:26:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.347+0000 7fa626a00640 1 --2- 172.21.15.47:0/2936247943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa628103260 0x7fa628073fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.347+0000 7fa626a00640 1 -- 172.21.15.47:0/2936247943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6281028a0 msgr2=0x7fa628073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.347+0000 7fa626a00640 1 --2- 172.21.15.47:0/2936247943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6281028a0 0x7fa628073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.347+0000 7fa626a00640 1 -- 172.21.15.47:0/2936247943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa628074860 con 0x7fa6281016b0 2024-03-21T11:26:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.347+0000 7fa626a00640 1 --2- 172.21.15.47:0/2936247943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6281016b0 0x7fa628075450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa61800e9f0 tx=0x7fa61800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.348+0000 7fa61fe00640 1 -- 172.21.15.47:0/2936247943 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61800ccc0 con 0x7fa6281016b0 2024-03-21T11:26:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.348+0000 7fa61fe00640 1 -- 172.21.15.47:0/2936247943 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6180035f0 con 0x7fa6281016b0 2024-03-21T11:26:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.348+0000 7fa61fe00640 1 -- 172.21.15.47:0/2936247943 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61800ce60 con 0x7fa6281016b0 2024-03-21T11:26:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.348+0000 7fa62da00640 1 -- 172.21.15.47:0/2936247943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6281a1a60 con 0x7fa6281016b0 2024-03-21T11:26:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.348+0000 7fa62da00640 1 -- 172.21.15.47:0/2936247943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6281a1f60 con 0x7fa6281016b0 2024-03-21T11:26:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.351+0000 7fa61fe00640 1 -- 172.21.15.47:0/2936247943 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa6180037a0 con 0x7fa6281016b0 2024-03-21T11:26:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.352+0000 7fa63079c740 1 Processor -- start 2024-03-21T11:26:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.352+0000 7fa63079c740 1 -- start start 2024-03-21T11:26:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.352+0000 7fa63079c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6bf810 0x55c20f6bfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.352+0000 7fa63079c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6c09e0 0x55c20f6c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.352+0000 7fa63079c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c13a0 0x55c20f6c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.352+0000 7fa63079c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20f6242c0 con 0x55c20f6c13a0 2024-03-21T11:26:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.352+0000 7fa63079c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20f5ccc90 con 0x55c20f6bf810 2024-03-21T11:26:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.352+0000 7fa63079c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20f620360 con 0x55c20f6c09e0 2024-03-21T11:26:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.352+0000 7fa626a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6c09e0 0x55c20f6c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.352+0000 7fa627400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6bf810 0x55c20f6bfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.353+0000 7fa627400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6bf810 0x55c20f6bfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46468/0 (socket says 172.21.15.47:46468) 2024-03-21T11:26:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.353+0000 7fa627400640 1 -- 172.21.15.47:0/3606926714 learned_addr learned my addr 172.21.15.47:0/3606926714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.353+0000 7fa61e000640 1 --2- 172.21.15.47:0/3606926714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c13a0 0x55c20f6c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.353+0000 7fa627400640 1 -- 172.21.15.47:0/3606926714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6c09e0 msgr2=0x55c20f6c0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.353+0000 7fa627400640 1 --2- 172.21.15.47:0/3606926714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6c09e0 0x55c20f6c0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.353+0000 7fa627400640 1 -- 172.21.15.47:0/3606926714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c13a0 msgr2=0x55c20f6c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.353+0000 7fa627400640 1 --2- 172.21.15.47:0/3606926714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c13a0 0x55c20f6c7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.353+0000 7fa627400640 1 -- 172.21.15.47:0/3606926714 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c20f5f1080 con 0x55c20f6bf810 2024-03-21T11:26:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.353+0000 7fa61e000640 1 --2- 172.21.15.47:0/3606926714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c13a0 0x55c20f6c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:26:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.353+0000 7fa61fe00640 1 --2- 172.21.15.47:0/2936247943 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa60c0a2fa0 0x7fa60c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.353+0000 7fa627400640 1 --2- 172.21.15.47:0/3606926714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6bf810 0x55c20f6bfc30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa620036040 tx=0x7fa620009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec8052620743bb0e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.353+0000 7fa61fe00640 1 -- 172.21.15.47:0/2936247943 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa6180cfa50 con 0x7fa6281016b0 2024-03-21T11:26:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.353+0000 7fa626a00640 1 --2- 172.21.15.47:0/2936247943 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa60c0a2fa0 0x7fa60c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.354+0000 7fa617e00640 1 -- 172.21.15.47:0/3606926714 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620042020 con 0x55c20f6bf810 2024-03-21T11:26:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.354+0000 7fa617e00640 1 -- 172.21.15.47:0/3606926714 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa620033580 con 0x55c20f6bf810 2024-03-21T11:26:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.354+0000 7fa617e00640 1 -- 172.21.15.47:0/3606926714 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62002caf0 con 0x55c20f6bf810 2024-03-21T11:26:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.355+0000 7fa63079c740 1 -- 172.21.15.47:0/3606926714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6bf810 msgr2=0x55c20f6bfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.355+0000 7fa63079c740 1 --2- 172.21.15.47:0/3606926714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6bf810 0x55c20f6bfc30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa620036040 tx=0x7fa620009310 comp rx=0 tx=0).stop 2024-03-21T11:26:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.355+0000 7fa626a00640 1 --2- 172.21.15.47:0/2936247943 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa60c0a2fa0 0x7fa60c0a5460 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa6180cfcf0 tx=0x7fa6180d5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.355+0000 7fa63079c740 1 -- 172.21.15.47:0/3606926714 shutdown_connections 2024-03-21T11:26:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.355+0000 7fa63079c740 1 --2- 172.21.15.47:0/3606926714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c13a0 0x55c20f6c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.355+0000 7fa63079c740 1 --2- 172.21.15.47:0/3606926714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6c09e0 0x55c20f6c0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.355+0000 7fa63079c740 1 --2- 172.21.15.47:0/3606926714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6bf810 0x55c20f6bfc30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.355+0000 7fa63079c740 1 -- 172.21.15.47:0/3606926714 >> 172.21.15.47:0/3606926714 conn(0x55c20f6bac00 msgr2=0x55c20f6bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.355+0000 7fa63079c740 1 -- 172.21.15.47:0/3606926714 shutdown_connections 2024-03-21T11:26:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.355+0000 7fa63079c740 1 -- 172.21.15.47:0/3606926714 wait complete. 2024-03-21T11:26:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.355+0000 7fa63079c740 1 -- start start 2024-03-21T11:26:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.356+0000 7fa63079c740 1 Processor -- start 2024-03-21T11:26:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.356+0000 7fa63079c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6bf810 0x55c20f6c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.356+0000 7fa63079c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c09e0 0x55c20f6c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.356+0000 7fa63079c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c13a0 0x55c20f6d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.356+0000 7fa63079c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20f6c8350 con 0x55c20f6c09e0 2024-03-21T11:26:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.356+0000 7fa63079c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20f5e00d0 con 0x55c20f6c13a0 2024-03-21T11:26:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.356+0000 7fa63079c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20f6c84d0 con 0x55c20f6bf810 2024-03-21T11:26:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.356+0000 7fa63079c740 10 client.0 _ll_register_callbacks cb 0x55c20f5e4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.356+0000 7fa627400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6bf810 0x55c20f6c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.356+0000 7fa627400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6bf810 0x55c20f6c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54170/0 (socket says 172.21.15.47:54170) 2024-03-21T11:26:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.356+0000 7fa627400640 1 -- 172.21.15.47:0/3433883264 learned_addr learned my addr 172.21.15.47:0/3433883264 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.356+0000 7fa627400640 1 --2- 172.21.15.47:0/3433883264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c13a0 0x55c20f6d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.356+0000 7fa61e000640 1 --2- 172.21.15.47:0/3433883264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c09e0 0x55c20f6c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.357+0000 7fa627400640 1 -- 172.21.15.47:0/3433883264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c13a0 msgr2=0x55c20f6d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.357+0000 7fa627400640 1 --2- 172.21.15.47:0/3433883264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c13a0 0x55c20f6d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.357+0000 7fa627400640 1 -- 172.21.15.47:0/3433883264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c09e0 msgr2=0x55c20f6c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.357+0000 7fa627400640 1 --2- 172.21.15.47:0/3433883264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c09e0 0x55c20f6c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.357+0000 7fa627400640 1 -- 172.21.15.47:0/3433883264 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c20f3383f0 con 0x55c20f6bf810 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.357+0000 7fa627400640 1 --2- 172.21.15.47:0/3433883264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6bf810 0x55c20f6c9a50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa620003a90 tx=0x7fa62002bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=129c9be4050b10a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.357+0000 7fa614c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.357+0000 7fa614c00640 1 -- 172.21.15.47:0/3433883264 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620042070 con 0x55c20f6bf810 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.357+0000 7fa63079c740 1 -- 172.21.15.47:0/3433883264 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa620033580 con 0x55c20f6bf810 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.357+0000 7fa614c00640 1 -- 172.21.15.47:0/3433883264 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa62002c890 con 0x55c20f6bf810 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.358+0000 7fa614c00640 1 -- 172.21.15.47:0/3433883264 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa62002ca90 con 0x55c20f6bf810 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.358+0000 7fa63079c740 10 client.26506 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.358+0000 7fa63079c740 1 -- 172.21.15.47:0/3433883264 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c20f4027d0 con 0x55c20f6bf810 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.358+0000 7fa614c00640 1 -- 172.21.15.47:0/3433883264 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa620033580 con 0x55c20f6bf810 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.358+0000 7fa614c00640 1 -- 172.21.15.47:0/3433883264 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa620034430 con 0x55c20f6bf810 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.358+0000 7fa63079c740 10 client.26506 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.358+0000 7fa63079c740 10 client.26506 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:26:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.358+0000 7fa63079c740 20 client.26506 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.358+0000 7fa63079c740 1 --2- 172.21.15.47:0/3433883264 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c20f6d50a0 0x55c20f6d7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.358+0000 7fa63079c740 4 client.26506 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.358+0000 7fa63079c740 1 -- 172.21.15.47:0/3433883264 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c20f631800 con 0x55c20f6d50a0 2024-03-21T11:26:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.358+0000 7fa61e000640 1 --2- 172.21.15.47:0/3433883264 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c20f6d50a0 0x55c20f6d7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.359+0000 7fa61e000640 1 --2- 172.21.15.47:0/3433883264 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c20f6d50a0 0x55c20f6d7490 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2feb00fb9b6fafa6 server_cookie=ee9998ec11aabf19 in_seq=0 out_seq=0 2024-03-21T11:26:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.359+0000 7fa614c00640 10 client.26506 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:26:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa614c00640 1 -- 172.21.15.47:0/3433883264 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c20f631800 con 0x55c20f6d50a0 2024-03-21T11:26:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa614c00640 10 client.26506 handle_command_reply: tid=0 2024-03-21T11:26:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa63079c740 1 client.26506 shutdown 2024-03-21T11:26:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa63079c740 1 -- 172.21.15.47:0/3433883264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6bf810 msgr2=0x55c20f6c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa63079c740 1 --2- 172.21.15.47:0/3433883264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6bf810 0x55c20f6c9a50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa620003a90 tx=0x7fa62002bef0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa63079c740 1 -- 172.21.15.47:0/3433883264 shutdown_connections 2024-03-21T11:26:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa63079c740 1 --2- 172.21.15.47:0/3433883264 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c20f6d50a0 0x55c20f6d7490 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa63079c740 1 --2- 172.21.15.47:0/3433883264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c13a0 0x55c20f6d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa63079c740 1 --2- 172.21.15.47:0/3433883264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c09e0 0x55c20f6c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa614c00640 0 client.26506 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:26:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa63079c740 1 --2- 172.21.15.47:0/3433883264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6bf810 0x55c20f6c9a50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa63079c740 1 -- 172.21.15.47:0/3433883264 >> 172.21.15.47:0/3433883264 conn(0x55c20f6bac00 msgr2=0x55c20f6b9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa63079c740 1 -- 172.21.15.47:0/3433883264 shutdown_connections 2024-03-21T11:26:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa63079c740 1 -- 172.21.15.47:0/3433883264 wait complete. 2024-03-21T11:26:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.360+0000 7fa63079c740 20 client.26506 trim_cache size 0 max 16384 2024-03-21T11:26:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.376+0000 7fa63079c740 1 Processor -- start 2024-03-21T11:26:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.377+0000 7fa63079c740 1 -- start start 2024-03-21T11:26:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.377+0000 7fa63079c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6bf810 0x55c20f6c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.377+0000 7fa63079c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c09e0 0x55c20f6be570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.377+0000 7fa63079c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c13a0 0x55c20f6dbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.377+0000 7fa63079c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6281ac2e0 con 0x55c20f6c13a0 2024-03-21T11:26:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.377+0000 7fa63079c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6281adc60 con 0x55c20f6c09e0 2024-03-21T11:26:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.377+0000 7fa63079c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6281ac940 con 0x55c20f6bf810 2024-03-21T11:26:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.377+0000 7fa61e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c09e0 0x55c20f6be570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.377+0000 7fa61e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c09e0 0x55c20f6be570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46488/0 (socket says 172.21.15.47:46488) 2024-03-21T11:26:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.377+0000 7fa61e000640 1 -- 172.21.15.47:0/1687095560 learned_addr learned my addr 172.21.15.47:0/1687095560 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.377+0000 7fa627400640 1 --2- 172.21.15.47:0/1687095560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6bf810 0x55c20f6c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.377+0000 7fa627400640 1 --2- 172.21.15.47:0/1687095560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c13a0 0x55c20f6dbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.378+0000 7fa61e000640 1 -- 172.21.15.47:0/1687095560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6bf810 msgr2=0x55c20f6c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.378+0000 7fa61e000640 1 --2- 172.21.15.47:0/1687095560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6bf810 0x55c20f6c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.378+0000 7fa61e000640 1 -- 172.21.15.47:0/1687095560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c13a0 msgr2=0x55c20f6dbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.378+0000 7fa61e000640 1 --2- 172.21.15.47:0/1687095560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c13a0 0x55c20f6dbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.378+0000 7fa61e000640 1 -- 172.21.15.47:0/1687095560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c20f6bb970 con 0x55c20f6c09e0 2024-03-21T11:26:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.378+0000 7fa61e000640 1 --2- 172.21.15.47:0/1687095560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c09e0 0x55c20f6be570 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55c20f6d7b70 tx=0x7fa61001ea50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43bf1f90e2675d5b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.378+0000 7fa617400640 1 -- 172.21.15.47:0/1687095560 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610004230 con 0x55c20f6c09e0 2024-03-21T11:26:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.378+0000 7fa617400640 1 -- 172.21.15.47:0/1687095560 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6100043d0 con 0x55c20f6c09e0 2024-03-21T11:26:37.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.378+0000 7fa617400640 1 -- 172.21.15.47:0/1687095560 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610012950 con 0x55c20f6c09e0 2024-03-21T11:26:37.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.379+0000 7fa63079c740 1 -- 172.21.15.47:0/1687095560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c09e0 msgr2=0x55c20f6be570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.379+0000 7fa63079c740 1 --2- 172.21.15.47:0/1687095560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c09e0 0x55c20f6be570 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55c20f6d7b70 tx=0x7fa61001ea50 comp rx=0 tx=0).stop 2024-03-21T11:26:37.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.379+0000 7fa63079c740 1 -- 172.21.15.47:0/1687095560 shutdown_connections 2024-03-21T11:26:37.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.379+0000 7fa63079c740 1 --2- 172.21.15.47:0/1687095560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6c13a0 0x55c20f6dbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.379+0000 7fa63079c740 1 --2- 172.21.15.47:0/1687095560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c09e0 0x55c20f6be570 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.379+0000 7fa63079c740 1 --2- 172.21.15.47:0/1687095560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6bf810 0x55c20f6c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.379+0000 7fa63079c740 1 -- 172.21.15.47:0/1687095560 >> 172.21.15.47:0/1687095560 conn(0x55c20f6bac00 msgr2=0x55c20f6bde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.379+0000 7fa63079c740 1 -- 172.21.15.47:0/1687095560 shutdown_connections 2024-03-21T11:26:37.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.379+0000 7fa63079c740 1 -- 172.21.15.47:0/1687095560 wait complete. 2024-03-21T11:26:37.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.380+0000 7fa63079c740 1 -- start start 2024-03-21T11:26:37.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.380+0000 7fa63079c740 1 Processor -- start 2024-03-21T11:26:37.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.380+0000 7fa63079c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6bf810 0x55c20f6cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.380+0000 7fa63079c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c09e0 0x55c20f6cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.380+0000 7fa63079c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6c13a0 0x55c20f6cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.380+0000 7fa63079c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6281ad600 con 0x55c20f6bf810 2024-03-21T11:26:37.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.380+0000 7fa63079c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6281acfa0 con 0x55c20f6c09e0 2024-03-21T11:26:37.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.380+0000 7fa63079c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6281aa4a0 con 0x55c20f6c13a0 2024-03-21T11:26:37.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.380+0000 7fa63079c740 10 client.0 _ll_register_callbacks cb 0x7fa6180d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:37.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.380+0000 7fa61e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c09e0 0x55c20f6cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.381+0000 7fa61e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c09e0 0x55c20f6cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46502/0 (socket says 172.21.15.47:46502) 2024-03-21T11:26:37.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.381+0000 7fa61e000640 1 -- 172.21.15.47:0/4076724461 learned_addr learned my addr 172.21.15.47:0/4076724461 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.381+0000 7fa627400640 1 --2- 172.21.15.47:0/4076724461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6bf810 0x55c20f6cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.381+0000 7fa627400640 1 --2- 172.21.15.47:0/4076724461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6c13a0 0x55c20f6cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.381+0000 7fa61e000640 1 -- 172.21.15.47:0/4076724461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6c13a0 msgr2=0x55c20f6cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.381+0000 7fa61e000640 1 --2- 172.21.15.47:0/4076724461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6c13a0 0x55c20f6cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.381+0000 7fa61e000640 1 -- 172.21.15.47:0/4076724461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6bf810 msgr2=0x55c20f6cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.381+0000 7fa61e000640 1 --2- 172.21.15.47:0/4076724461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6bf810 0x55c20f6cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.381+0000 7fa61e000640 1 -- 172.21.15.47:0/4076724461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c20f6cfce0 con 0x55c20f6c09e0 2024-03-21T11:26:37.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.381+0000 7fa627400640 1 --2- 172.21.15.47:0/4076724461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6bf810 0x55c20f6cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:37.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.381+0000 7fa61e000640 1 --2- 172.21.15.47:0/4076724461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c09e0 0x55c20f6cc6d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa610009bc0 tx=0x7fa610009d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8077db18f7c52de6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.381+0000 7fa614c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:37.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.381+0000 7fa614c00640 1 -- 172.21.15.47:0/4076724461 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610005440 con 0x55c20f6c09e0 2024-03-21T11:26:37.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.382+0000 7fa63079c740 1 -- 172.21.15.47:0/4076724461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6100043d0 con 0x55c20f6c09e0 2024-03-21T11:26:37.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.382+0000 7fa614c00640 1 -- 172.21.15.47:0/4076724461 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa610012750 con 0x55c20f6c09e0 2024-03-21T11:26:37.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.382+0000 7fa614c00640 1 -- 172.21.15.47:0/4076724461 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa610037050 con 0x55c20f6c09e0 2024-03-21T11:26:37.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.382+0000 7fa63079c740 10 client.26489 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:37.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.382+0000 7fa63079c740 1 -- 172.21.15.47:0/4076724461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c20f730b30 con 0x55c20f6c09e0 2024-03-21T11:26:37.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.382+0000 7fa614c00640 1 -- 172.21.15.47:0/4076724461 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6100043d0 con 0x55c20f6c09e0 2024-03-21T11:26:37.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.382+0000 7fa614c00640 1 -- 172.21.15.47:0/4076724461 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa610010430 con 0x55c20f6c09e0 2024-03-21T11:26:37.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.382+0000 7fa63079c740 10 client.26489 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:37.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.382+0000 7fa63079c740 10 client.26489 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:26:37.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.382+0000 7fa63079c740 20 client.26489 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:37.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.382+0000 7fa63079c740 1 --2- 172.21.15.47:0/4076724461 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c20f731410 0x55c20f733830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.382+0000 7fa63079c740 4 client.26489 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:26:37.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.382+0000 7fa63079c740 1 -- 172.21.15.47:0/4076724461 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c20f62bb50 con 0x55c20f731410 2024-03-21T11:26:37.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.383+0000 7fa627400640 1 --2- 172.21.15.47:0/4076724461 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c20f731410 0x55c20f733830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.383+0000 7fa627400640 1 --2- 172.21.15.47:0/4076724461 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c20f731410 0x55c20f733830 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ade5549f9e9ad42b server_cookie=a1ac133dc998bb4d in_seq=0 out_seq=0 2024-03-21T11:26:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.383+0000 7fa614c00640 10 client.26489 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:26:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.383+0000 7fa614c00640 1 -- 172.21.15.47:0/4076724461 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c20f62bb50 con 0x55c20f731410 2024-03-21T11:26:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.383+0000 7fa614c00640 10 client.26489 handle_command_reply: tid=0 2024-03-21T11:26:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.383+0000 7fa63079c740 1 client.26489 shutdown 2024-03-21T11:26:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.384+0000 7fa63079c740 1 -- 172.21.15.47:0/4076724461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c09e0 msgr2=0x55c20f6cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.384+0000 7fa63079c740 1 --2- 172.21.15.47:0/4076724461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c09e0 0x55c20f6cc6d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa610009bc0 tx=0x7fa610009d60 comp rx=0 tx=0).stop 2024-03-21T11:26:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.384+0000 7fa63079c740 1 -- 172.21.15.47:0/4076724461 shutdown_connections 2024-03-21T11:26:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.384+0000 7fa63079c740 1 --2- 172.21.15.47:0/4076724461 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c20f731410 0x55c20f733830 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.384+0000 7fa63079c740 1 --2- 172.21.15.47:0/4076724461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c20f6c13a0 0x55c20f6cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.384+0000 7fa63079c740 1 --2- 172.21.15.47:0/4076724461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c20f6c09e0 0x55c20f6cc6d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.384+0000 7fa63079c740 1 --2- 172.21.15.47:0/4076724461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c20f6bf810 0x55c20f6cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.384+0000 7fa63079c740 1 -- 172.21.15.47:0/4076724461 >> 172.21.15.47:0/4076724461 conn(0x55c20f6bac00 msgr2=0x55c20f6c9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.384+0000 7fa614c00640 0 client.26489 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:26:37.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.384+0000 7fa63079c740 1 -- 172.21.15.47:0/4076724461 shutdown_connections 2024-03-21T11:26:37.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.384+0000 7fa63079c740 1 -- 172.21.15.47:0/4076724461 wait complete. 2024-03-21T11:26:37.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.384+0000 7fa63079c740 20 client.26489 trim_cache size 0 max 16384 2024-03-21T11:26:37.391 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:26:37.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.385+0000 7fa62da00640 1 -- 172.21.15.47:0/2936247943 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa60c0a2fa0 msgr2=0x7fa60c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.385+0000 7fa62da00640 1 --2- 172.21.15.47:0/2936247943 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa60c0a2fa0 0x7fa60c0a5460 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa6180cfcf0 tx=0x7fa6180d5530 comp rx=0 tx=0).stop 2024-03-21T11:26:37.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.385+0000 7fa62da00640 1 -- 172.21.15.47:0/2936247943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6281016b0 msgr2=0x7fa628075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.385+0000 7fa62da00640 1 --2- 172.21.15.47:0/2936247943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6281016b0 0x7fa628075450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa61800e9f0 tx=0x7fa61800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.385+0000 7fa62da00640 1 -- 172.21.15.47:0/2936247943 shutdown_connections 2024-03-21T11:26:37.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.385+0000 7fa62da00640 1 --2- 172.21.15.47:0/2936247943 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa60c0a2fa0 0x7fa60c0a5460 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.385+0000 7fa62da00640 1 --2- 172.21.15.47:0/2936247943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa628103260 0x7fa628073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.385+0000 7fa62da00640 1 --2- 172.21.15.47:0/2936247943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6281028a0 0x7fa628073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.385+0000 7fa62da00640 1 --2- 172.21.15.47:0/2936247943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6281016b0 0x7fa628075450 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.385+0000 7fa62da00640 1 -- 172.21.15.47:0/2936247943 >> 172.21.15.47:0/2936247943 conn(0x7fa6280fcc40 msgr2=0x7fa628105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.385+0000 7fa62da00640 1 -- 172.21.15.47:0/2936247943 shutdown_connections 2024-03-21T11:26:37.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.385+0000 7fa62da00640 1 -- 172.21.15.47:0/2936247943 wait complete. 2024-03-21T11:26:37.404 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:26:37.404 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:26:37.496 INFO:tasks.workunit.client.0.smithi047.stdout: 1 133836 9.09 MB/sec warmup 81 sec latency 10.120 ms 2024-03-21T11:26:37.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.622+0000 7f3615c00640 1 -- 172.21.15.47:0/759110408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36100740a0 msgr2=0x7f3610074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.622+0000 7f3615c00640 1 --2- 172.21.15.47:0/759110408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36100740a0 0x7f3610074500 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009900 tx=0x7f35f802bcd0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.622+0000 7f3615c00640 1 -- 172.21.15.47:0/759110408 shutdown_connections 2024-03-21T11:26:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.622+0000 7f3615c00640 1 --2- 172.21.15.47:0/759110408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3610109540 0x7f361010b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.622+0000 7f3615c00640 1 --2- 172.21.15.47:0/759110408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36100740a0 0x7f3610074500 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.622+0000 7f3615c00640 1 --2- 172.21.15.47:0/759110408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3610075680 0x7f3610073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.622+0000 7f3615c00640 1 -- 172.21.15.47:0/759110408 >> 172.21.15.47:0/759110408 conn(0x7f36100fcc40 msgr2=0x7f36100ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.622+0000 7f3615c00640 1 -- 172.21.15.47:0/759110408 shutdown_connections 2024-03-21T11:26:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.622+0000 7f3615c00640 1 -- 172.21.15.47:0/759110408 wait complete. 2024-03-21T11:26:37.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.623+0000 7f3615c00640 1 Processor -- start 2024-03-21T11:26:37.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.624+0000 7f3615c00640 1 -- start start 2024-03-21T11:26:37.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.624+0000 7f3615c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36100740a0 0x7f3610199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.624+0000 7f3615c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3610075680 0x7f361019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.624+0000 7f3615c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3610109540 0x7f36101a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.624+0000 7f3615c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361010f580 con 0x7f3610109540 2024-03-21T11:26:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.624+0000 7f3615c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361010f400 con 0x7f3610075680 2024-03-21T11:26:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.624+0000 7f3615c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361010f700 con 0x7f36100740a0 2024-03-21T11:26:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.624+0000 7f360ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36100740a0 0x7f3610199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.624+0000 7f360ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36100740a0 0x7f3610199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54208/0 (socket says 172.21.15.47:54208) 2024-03-21T11:26:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.624+0000 7f360ea00640 1 -- 172.21.15.47:0/3522767661 learned_addr learned my addr 172.21.15.47:0/3522767661 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.624+0000 7f3606000640 1 --2- 172.21.15.47:0/3522767661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3610075680 0x7f361019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.624+0000 7f360f400640 1 --2- 172.21.15.47:0/3522767661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3610109540 0x7f36101a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.625+0000 7f360ea00640 1 -- 172.21.15.47:0/3522767661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3610075680 msgr2=0x7f361019a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.625+0000 7f360ea00640 1 --2- 172.21.15.47:0/3522767661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3610075680 0x7f361019a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.625+0000 7f360ea00640 1 -- 172.21.15.47:0/3522767661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3610109540 msgr2=0x7f36101a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.625+0000 7f360ea00640 1 --2- 172.21.15.47:0/3522767661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3610109540 0x7f36101a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.625+0000 7f360ea00640 1 -- 172.21.15.47:0/3522767661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36101a17f0 con 0x7f36100740a0 2024-03-21T11:26:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.625+0000 7f360ea00640 1 --2- 172.21.15.47:0/3522767661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36100740a0 0x7f3610199bd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f360000d950 tx=0x7f360000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.625+0000 7f3607e00640 1 -- 172.21.15.47:0/3522767661 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600017030 con 0x7f36100740a0 2024-03-21T11:26:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.625+0000 7f3607e00640 1 -- 172.21.15.47:0/3522767661 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3600004020 con 0x7f36100740a0 2024-03-21T11:26:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.625+0000 7f3607e00640 1 -- 172.21.15.47:0/3522767661 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36000047a0 con 0x7f36100740a0 2024-03-21T11:26:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.625+0000 7f3615c00640 1 -- 172.21.15.47:0/3522767661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36101a1ae0 con 0x7f36100740a0 2024-03-21T11:26:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.625+0000 7f3615c00640 1 -- 172.21.15.47:0/3522767661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36101a2050 con 0x7f36100740a0 2024-03-21T11:26:37.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.629+0000 7f3607e00640 1 -- 172.21.15.47:0/3522767661 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f36000041d0 con 0x7f36100740a0 2024-03-21T11:26:37.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.629+0000 7f36189c4740 1 Processor -- start 2024-03-21T11:26:37.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.629+0000 7f36189c4740 1 -- start start 2024-03-21T11:26:37.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f36189c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed01810 0x55dabed01c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f36189c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed029e0 0x55dabed02e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f36189c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed033a0 0x55dabed09c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f36189c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabec662c0 con 0x55dabed033a0 2024-03-21T11:26:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f36189c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabec0ec90 con 0x55dabed029e0 2024-03-21T11:26:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f36189c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabec62360 con 0x55dabed01810 2024-03-21T11:26:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f360ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed029e0 0x55dabed02e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f360f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed01810 0x55dabed01c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f360f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed01810 0x55dabed01c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54210/0 (socket says 172.21.15.47:54210) 2024-03-21T11:26:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f360f400640 1 -- 172.21.15.47:0/2524439320 learned_addr learned my addr 172.21.15.47:0/2524439320 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f360f400640 1 -- 172.21.15.47:0/2524439320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed029e0 msgr2=0x55dabed02e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f360f400640 1 --2- 172.21.15.47:0/2524439320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed029e0 0x55dabed02e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f360f400640 1 -- 172.21.15.47:0/2524439320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed033a0 msgr2=0x55dabed09c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f360f400640 1 --2- 172.21.15.47:0/2524439320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed033a0 0x55dabed09c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.630+0000 7f360f400640 1 -- 172.21.15.47:0/2524439320 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dabec33080 con 0x55dabed01810 2024-03-21T11:26:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.631+0000 7f360f400640 1 --2- 172.21.15.47:0/2524439320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed01810 0x55dabed01c30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3608009f40 tx=0x7f3608010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91abf42699a6a798 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.631+0000 7f35ffe00640 1 -- 172.21.15.47:0/2524439320 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608003b70 con 0x55dabed01810 2024-03-21T11:26:37.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.631+0000 7f35ffe00640 1 -- 172.21.15.47:0/2524439320 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3608003d10 con 0x55dabed01810 2024-03-21T11:26:37.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.631+0000 7f35ffe00640 1 -- 172.21.15.47:0/2524439320 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608011a40 con 0x55dabed01810 2024-03-21T11:26:37.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.631+0000 7f3607e00640 1 --2- 172.21.15.47:0/3522767661 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35f40a2fa0 0x7f35f40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.632+0000 7f3607e00640 1 -- 172.21.15.47:0/3522767661 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f36000cf190 con 0x7f36100740a0 2024-03-21T11:26:37.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.632+0000 7f360ea00640 1 --2- 172.21.15.47:0/3522767661 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35f40a2fa0 0x7f35f40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.632+0000 7f36189c4740 1 -- 172.21.15.47:0/2524439320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed01810 msgr2=0x55dabed01c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.632+0000 7f36189c4740 1 --2- 172.21.15.47:0/2524439320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed01810 0x55dabed01c30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3608009f40 tx=0x7f3608010860 comp rx=0 tx=0).stop 2024-03-21T11:26:37.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.632+0000 7f36189c4740 1 -- 172.21.15.47:0/2524439320 shutdown_connections 2024-03-21T11:26:37.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.632+0000 7f36189c4740 1 --2- 172.21.15.47:0/2524439320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed033a0 0x55dabed09c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.632+0000 7f36189c4740 1 --2- 172.21.15.47:0/2524439320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed029e0 0x55dabed02e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.632+0000 7f36189c4740 1 --2- 172.21.15.47:0/2524439320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed01810 0x55dabed01c30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.632+0000 7f36189c4740 1 -- 172.21.15.47:0/2524439320 >> 172.21.15.47:0/2524439320 conn(0x55dabecfcc00 msgr2=0x55dabecfd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.632+0000 7f36189c4740 1 -- 172.21.15.47:0/2524439320 shutdown_connections 2024-03-21T11:26:37.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.632+0000 7f36189c4740 1 -- 172.21.15.47:0/2524439320 wait complete. 2024-03-21T11:26:37.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.632+0000 7f360ea00640 1 --2- 172.21.15.47:0/3522767661 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35f40a2fa0 0x7f35f40a5460 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f36000cc8a0 tx=0x7f3600097040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.633+0000 7f36189c4740 1 -- start start 2024-03-21T11:26:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.633+0000 7f36189c4740 1 Processor -- start 2024-03-21T11:26:37.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f36189c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed01810 0x55dabed0baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f36189c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed029e0 0x55dabed0c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f36189c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed033a0 0x55dabed13000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f36189c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabed0a3b0 con 0x55dabed029e0 2024-03-21T11:26:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f36189c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabec220d0 con 0x55dabed01810 2024-03-21T11:26:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f36189c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabed0a530 con 0x55dabed033a0 2024-03-21T11:26:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f36189c4740 10 client.0 _ll_register_callbacks cb 0x55dabec26e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f3606000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed029e0 0x55dabed0c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f3606000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed029e0 0x55dabed0c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37278/0 (socket says 172.21.15.47:37278) 2024-03-21T11:26:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f3606000640 1 -- 172.21.15.47:0/718229349 learned_addr learned my addr 172.21.15.47:0/718229349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f360f400640 1 --2- 172.21.15.47:0/718229349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed01810 0x55dabed0baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f3606000640 1 -- 172.21.15.47:0/718229349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed033a0 msgr2=0x55dabed13000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:26:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f3606000640 1 --2- 172.21.15.47:0/718229349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed033a0 0x55dabed13000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f3606000640 1 -- 172.21.15.47:0/718229349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed01810 msgr2=0x55dabed0baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f3606000640 1 --2- 172.21.15.47:0/718229349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed01810 0x55dabed0baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.634+0000 7f3606000640 1 -- 172.21.15.47:0/718229349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dabe97a3f0 con 0x55dabed029e0 2024-03-21T11:26:37.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.635+0000 7f3606000640 1 --2- 172.21.15.47:0/718229349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed029e0 0x55dabed0c030 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005ab0 tx=0x7f35f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a8ec0a4ba05df6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.635+0000 7f35fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:26:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.635+0000 7f35fcc00640 1 -- 172.21.15.47:0/718229349 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8043030 con 0x55dabed029e0 2024-03-21T11:26:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.635+0000 7f35fcc00640 1 -- 172.21.15.47:0/718229349 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f35f803a040 con 0x55dabed029e0 2024-03-21T11:26:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.635+0000 7f36189c4740 1 -- 172.21.15.47:0/718229349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3608003d10 con 0x55dabed029e0 2024-03-21T11:26:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.636+0000 7f35fcc00640 1 -- 172.21.15.47:0/718229349 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35f8005d50 con 0x55dabed029e0 2024-03-21T11:26:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.636+0000 7f36189c4740 10 client.16296 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.636+0000 7f36189c4740 1 -- 172.21.15.47:0/718229349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dabea447d0 con 0x55dabed029e0 2024-03-21T11:26:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.636+0000 7f35fcc00640 1 -- 172.21.15.47:0/718229349 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3608003d10 con 0x55dabed029e0 2024-03-21T11:26:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.637+0000 7f35fcc00640 1 -- 172.21.15.47:0/718229349 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f35f8005340 con 0x55dabed029e0 2024-03-21T11:26:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.637+0000 7f36189c4740 10 client.16296 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.637+0000 7f36189c4740 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.637+0000 7f36189c4740 20 client.16296 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.637+0000 7f36189c4740 1 --2- 172.21.15.47:0/718229349 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dabed175a0 0x55dabed199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.637+0000 7f36189c4740 4 client.16296 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.637+0000 7f36189c4740 1 -- 172.21.15.47:0/718229349 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dabec73800 con 0x55dabed175a0 2024-03-21T11:26:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.637+0000 7f360f400640 1 --2- 172.21.15.47:0/718229349 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dabed175a0 0x55dabed199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.638+0000 7f360f400640 1 --2- 172.21.15.47:0/718229349 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dabed175a0 0x55dabed199e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fae62f3fd0bb3bff server_cookie=856399160622d6ba in_seq=0 out_seq=0 2024-03-21T11:26:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.638+0000 7f35fcc00640 10 client.16296 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.639+0000 7f35fcc00640 1 -- 172.21.15.47:0/718229349 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dabec73800 con 0x55dabed175a0 2024-03-21T11:26:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.639+0000 7f35fcc00640 10 client.16296 handle_command_reply: tid=0 2024-03-21T11:26:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.639+0000 7f36189c4740 1 client.16296 shutdown 2024-03-21T11:26:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.639+0000 7f36189c4740 1 -- 172.21.15.47:0/718229349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed029e0 msgr2=0x55dabed0c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.639+0000 7f36189c4740 1 --2- 172.21.15.47:0/718229349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed029e0 0x55dabed0c030 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005ab0 tx=0x7f35f8004270 comp rx=0 tx=0).stop 2024-03-21T11:26:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.639+0000 7f36189c4740 1 -- 172.21.15.47:0/718229349 shutdown_connections 2024-03-21T11:26:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.639+0000 7f36189c4740 1 --2- 172.21.15.47:0/718229349 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dabed175a0 0x55dabed199e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.639+0000 7f36189c4740 1 --2- 172.21.15.47:0/718229349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed033a0 0x55dabed13000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.639+0000 7f36189c4740 1 --2- 172.21.15.47:0/718229349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed029e0 0x55dabed0c030 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.639+0000 7f36189c4740 1 --2- 172.21.15.47:0/718229349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed01810 0x55dabed0baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.639+0000 7f35fcc00640 0 client.16296 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.639+0000 7f36189c4740 1 -- 172.21.15.47:0/718229349 >> 172.21.15.47:0/718229349 conn(0x55dabecfcc00 msgr2=0x55dabecfbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.640+0000 7f36189c4740 1 -- 172.21.15.47:0/718229349 shutdown_connections 2024-03-21T11:26:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.640+0000 7f36189c4740 1 -- 172.21.15.47:0/718229349 wait complete. 2024-03-21T11:26:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.640+0000 7f36189c4740 20 client.16296 trim_cache size 0 max 16384 2024-03-21T11:26:37.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.654+0000 7f36189c4740 1 Processor -- start 2024-03-21T11:26:37.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.655+0000 7f36189c4740 1 -- start start 2024-03-21T11:26:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.655+0000 7f36189c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed01810 0x55dabecfcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.655+0000 7f36189c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed029e0 0x55dabed12fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.655+0000 7f36189c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed033a0 0x55dabed05a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.655+0000 7f36189c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36101ac200 con 0x55dabed033a0 2024-03-21T11:26:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.655+0000 7f36189c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36101adb80 con 0x55dabed01810 2024-03-21T11:26:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.655+0000 7f36189c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36101ac860 con 0x55dabed029e0 2024-03-21T11:26:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.655+0000 7f360f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed01810 0x55dabecfcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.655+0000 7f3606000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed029e0 0x55dabed12fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.655+0000 7f3606000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed029e0 0x55dabed12fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54220/0 (socket says 172.21.15.47:54220) 2024-03-21T11:26:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.655+0000 7f3606000640 1 -- 172.21.15.47:0/321829034 learned_addr learned my addr 172.21.15.47:0/321829034 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.655+0000 7f360f400640 1 --2- 172.21.15.47:0/321829034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed033a0 0x55dabed05a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.656+0000 7f3606000640 1 -- 172.21.15.47:0/321829034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed01810 msgr2=0x55dabecfcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.656+0000 7f3606000640 1 --2- 172.21.15.47:0/321829034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed01810 0x55dabecfcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.656+0000 7f3606000640 1 -- 172.21.15.47:0/321829034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed033a0 msgr2=0x55dabed05a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.656+0000 7f3606000640 1 --2- 172.21.15.47:0/321829034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed033a0 0x55dabed05a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.656+0000 7f3606000640 1 -- 172.21.15.47:0/321829034 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dabed095c0 con 0x55dabed029e0 2024-03-21T11:26:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.656+0000 7f360f400640 1 --2- 172.21.15.47:0/321829034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed033a0 0x55dabed05a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.656+0000 7f3606000640 1 --2- 172.21.15.47:0/321829034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed029e0 0x55dabed12fc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009230 tx=0x7f35f802cdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa3a3b86814dc42e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.656+0000 7f35ff400640 1 -- 172.21.15.47:0/321829034 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8056070 con 0x55dabed029e0 2024-03-21T11:26:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.656+0000 7f35ff400640 1 -- 172.21.15.47:0/321829034 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35f8005830 con 0x55dabed029e0 2024-03-21T11:26:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.656+0000 7f35ff400640 1 -- 172.21.15.47:0/321829034 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8043070 con 0x55dabed029e0 2024-03-21T11:26:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.657+0000 7f36189c4740 1 -- 172.21.15.47:0/321829034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed029e0 msgr2=0x55dabed12fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.657+0000 7f36189c4740 1 --2- 172.21.15.47:0/321829034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed029e0 0x55dabed12fc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009230 tx=0x7f35f802cdc0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.657+0000 7f36189c4740 1 -- 172.21.15.47:0/321829034 shutdown_connections 2024-03-21T11:26:37.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.657+0000 7f36189c4740 1 --2- 172.21.15.47:0/321829034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed033a0 0x55dabed05a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.657+0000 7f36189c4740 1 --2- 172.21.15.47:0/321829034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed029e0 0x55dabed12fc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.657+0000 7f36189c4740 1 --2- 172.21.15.47:0/321829034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed01810 0x55dabecfcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.657+0000 7f36189c4740 1 -- 172.21.15.47:0/321829034 >> 172.21.15.47:0/321829034 conn(0x55dabecfcc00 msgr2=0x55dabed006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.657+0000 7f36189c4740 1 -- 172.21.15.47:0/321829034 shutdown_connections 2024-03-21T11:26:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.657+0000 7f36189c4740 1 -- 172.21.15.47:0/321829034 wait complete. 2024-03-21T11:26:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.658+0000 7f36189c4740 1 -- start start 2024-03-21T11:26:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.658+0000 7f36189c4740 1 Processor -- start 2024-03-21T11:26:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.658+0000 7f36189c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed01810 0x55dabed0f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.658+0000 7f36189c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed029e0 0x55dabed0f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.658+0000 7f36189c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed033a0 0x55dabed126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.658+0000 7f36189c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36101ad520 con 0x55dabed029e0 2024-03-21T11:26:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.658+0000 7f36189c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36101acec0 con 0x55dabed033a0 2024-03-21T11:26:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.658+0000 7f36189c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36101aa3c0 con 0x55dabed01810 2024-03-21T11:26:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.658+0000 7f36189c4740 10 client.0 _ll_register_callbacks cb 0x7f35f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.658+0000 7f360f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed01810 0x55dabed0f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.659+0000 7f3606000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed029e0 0x55dabed0f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.659+0000 7f360f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed01810 0x55dabed0f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54232/0 (socket says 172.21.15.47:54232) 2024-03-21T11:26:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.659+0000 7f360f400640 1 -- 172.21.15.47:0/639247058 learned_addr learned my addr 172.21.15.47:0/639247058 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.659+0000 7f360f400640 1 --2- 172.21.15.47:0/639247058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed033a0 0x55dabed126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.659+0000 7f3606000640 1 -- 172.21.15.47:0/639247058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed01810 msgr2=0x55dabed0f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.659+0000 7f3606000640 1 --2- 172.21.15.47:0/639247058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed01810 0x55dabed0f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.659+0000 7f3606000640 1 -- 172.21.15.47:0/639247058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed033a0 msgr2=0x55dabed126b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.659+0000 7f3606000640 1 --2- 172.21.15.47:0/639247058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed033a0 0x55dabed126b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.659+0000 7f3606000640 1 -- 172.21.15.47:0/639247058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dabed12ca0 con 0x55dabed029e0 2024-03-21T11:26:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.659+0000 7f360f400640 1 --2- 172.21.15.47:0/639247058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed033a0 0x55dabed126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.659+0000 7f3606000640 1 --2- 172.21.15.47:0/639247058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed029e0 0x55dabed0f700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005730 tx=0x7f35f802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af532b603d3f6ac7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.659+0000 7f35fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:26:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.660+0000 7f35fcc00640 1 -- 172.21.15.47:0/639247058 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8056030 con 0x55dabed029e0 2024-03-21T11:26:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.660+0000 7f35fcc00640 1 -- 172.21.15.47:0/639247058 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f35f80051e0 con 0x55dabed029e0 2024-03-21T11:26:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.660+0000 7f36189c4740 1 -- 172.21.15.47:0/639247058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35f8005830 con 0x55dabed029e0 2024-03-21T11:26:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.660+0000 7f35fcc00640 1 -- 172.21.15.47:0/639247058 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35f8063050 con 0x55dabed029e0 2024-03-21T11:26:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.660+0000 7f36189c4740 10 client.16302 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.660+0000 7f36189c4740 1 -- 172.21.15.47:0/639247058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dabed738d0 con 0x55dabed029e0 2024-03-21T11:26:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.660+0000 7f35fcc00640 1 -- 172.21.15.47:0/639247058 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35f8005830 con 0x55dabed029e0 2024-03-21T11:26:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.660+0000 7f35fcc00640 1 -- 172.21.15.47:0/639247058 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f35f803f680 con 0x55dabed029e0 2024-03-21T11:26:37.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.660+0000 7f36189c4740 10 client.16302 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:37.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.660+0000 7f36189c4740 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:37.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.660+0000 7f36189c4740 20 client.16302 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:37.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.661+0000 7f36189c4740 1 --2- 172.21.15.47:0/639247058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dabed741c0 0x55dabed765e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:37.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.661+0000 7f36189c4740 4 client.16302 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:26:37.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.661+0000 7f36189c4740 1 -- 172.21.15.47:0/639247058 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dabec6db50 con 0x55dabed741c0 2024-03-21T11:26:37.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.661+0000 7f360f400640 1 --2- 172.21.15.47:0/639247058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dabed741c0 0x55dabed765e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:37.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.661+0000 7f360f400640 1 --2- 172.21.15.47:0/639247058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dabed741c0 0x55dabed765e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68f14bbf4f845619 server_cookie=224e83f518392a4f in_seq=0 out_seq=0 2024-03-21T11:26:37.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.661+0000 7f35fcc00640 10 client.16302 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:37.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.662+0000 7f35fcc00640 1 -- 172.21.15.47:0/639247058 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dabec6db50 con 0x55dabed741c0 2024-03-21T11:26:37.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.662+0000 7f35fcc00640 10 client.16302 handle_command_reply: tid=0 2024-03-21T11:26:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.662+0000 7f36189c4740 1 client.16302 shutdown 2024-03-21T11:26:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.662+0000 7f36189c4740 1 -- 172.21.15.47:0/639247058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed029e0 msgr2=0x55dabed0f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.662+0000 7f36189c4740 1 --2- 172.21.15.47:0/639247058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed029e0 0x55dabed0f700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005730 tx=0x7f35f802c920 comp rx=0 tx=0).stop 2024-03-21T11:26:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.662+0000 7f36189c4740 1 -- 172.21.15.47:0/639247058 shutdown_connections 2024-03-21T11:26:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.662+0000 7f36189c4740 1 --2- 172.21.15.47:0/639247058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dabed741c0 0x55dabed765e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.662+0000 7f36189c4740 1 --2- 172.21.15.47:0/639247058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dabed033a0 0x55dabed126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.662+0000 7f35fcc00640 0 client.16302 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.662+0000 7f36189c4740 1 --2- 172.21.15.47:0/639247058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dabed029e0 0x55dabed0f700 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.662+0000 7f36189c4740 1 --2- 172.21.15.47:0/639247058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dabed01810 0x55dabed0f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.662+0000 7f36189c4740 1 -- 172.21.15.47:0/639247058 >> 172.21.15.47:0/639247058 conn(0x55dabecfcc00 msgr2=0x55dabed006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.662+0000 7f36189c4740 1 -- 172.21.15.47:0/639247058 shutdown_connections 2024-03-21T11:26:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.662+0000 7f36189c4740 1 -- 172.21.15.47:0/639247058 wait complete. 2024-03-21T11:26:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.663+0000 7f36189c4740 20 client.16302 trim_cache size 0 max 16384 2024-03-21T11:26:37.669 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:26:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.663+0000 7f3615c00640 1 -- 172.21.15.47:0/3522767661 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35f40a2fa0 msgr2=0x7f35f40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.663+0000 7f3615c00640 1 --2- 172.21.15.47:0/3522767661 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35f40a2fa0 0x7f35f40a5460 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f36000cc8a0 tx=0x7f3600097040 comp rx=0 tx=0).stop 2024-03-21T11:26:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.663+0000 7f3615c00640 1 -- 172.21.15.47:0/3522767661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36100740a0 msgr2=0x7f3610199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.663+0000 7f3615c00640 1 --2- 172.21.15.47:0/3522767661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36100740a0 0x7f3610199bd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f360000d950 tx=0x7f360000de10 comp rx=0 tx=0).stop 2024-03-21T11:26:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.663+0000 7f3615c00640 1 -- 172.21.15.47:0/3522767661 shutdown_connections 2024-03-21T11:26:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.663+0000 7f3615c00640 1 --2- 172.21.15.47:0/3522767661 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35f40a2fa0 0x7f35f40a5460 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.663+0000 7f3615c00640 1 --2- 172.21.15.47:0/3522767661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3610109540 0x7f36101a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.663+0000 7f3615c00640 1 --2- 172.21.15.47:0/3522767661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3610075680 0x7f361019a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.663+0000 7f3615c00640 1 --2- 172.21.15.47:0/3522767661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36100740a0 0x7f3610199bd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.663+0000 7f3615c00640 1 -- 172.21.15.47:0/3522767661 >> 172.21.15.47:0/3522767661 conn(0x7f36100fcc40 msgr2=0x7f36101054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.664+0000 7f3615c00640 1 -- 172.21.15.47:0/3522767661 shutdown_connections 2024-03-21T11:26:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:37.664+0000 7f3615c00640 1 -- 172.21.15.47:0/3522767661 wait complete. 2024-03-21T11:26:38.497 INFO:tasks.workunit.client.0.smithi047.stdout: 1 135261 9.12 MB/sec warmup 82 sec latency 15.593 ms 2024-03-21T11:26:38.683 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:26:38.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:26:38.684 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:26:38.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:38 smithi102 ceph-mon[36748]: pgmap v499: 129 pgs: 129 active+clean; 34 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 321 op/s 2024-03-21T11:26:38.771 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:38 smithi047 ceph-mon[31351]: pgmap v499: 129 pgs: 129 active+clean; 34 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 321 op/s 2024-03-21T11:26:38.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:38 smithi100 ceph-mon[36245]: pgmap v499: 129 pgs: 129 active+clean; 34 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 321 op/s 2024-03-21T11:26:38.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.948+0000 7fe209800640 1 -- 172.21.15.47:0/2386363972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2040ffe50 msgr2=0x7fe204100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:38.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.948+0000 7fe209800640 1 --2- 172.21.15.47:0/2386363972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2040ffe50 0x7fe204100270 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009900 tx=0x7fe1f402bc30 comp rx=0 tx=0).stop 2024-03-21T11:26:38.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.948+0000 7fe209800640 1 -- 172.21.15.47:0/2386363972 shutdown_connections 2024-03-21T11:26:38.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.948+0000 7fe209800640 1 --2- 172.21.15.47:0/2386363972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe204101150 0x7fe2041015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.948+0000 7fe209800640 1 --2- 172.21.15.47:0/2386363972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2041007b0 0x7fe204100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.948+0000 7fe209800640 1 --2- 172.21.15.47:0/2386363972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2040ffe50 0x7fe204100270 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.948+0000 7fe209800640 1 -- 172.21.15.47:0/2386363972 >> 172.21.15.47:0/2386363972 conn(0x7fe2040fba10 msgr2=0x7fe2040fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:38.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.948+0000 7fe209800640 1 -- 172.21.15.47:0/2386363972 shutdown_connections 2024-03-21T11:26:38.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.948+0000 7fe209800640 1 -- 172.21.15.47:0/2386363972 wait complete. 2024-03-21T11:26:38.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.949+0000 7fe209800640 1 Processor -- start 2024-03-21T11:26:38.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.949+0000 7fe209800640 1 -- start start 2024-03-21T11:26:38.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.950+0000 7fe209800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2040ffe50 0x7fe204199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.950+0000 7fe209800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2041007b0 0x7fe204199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.950+0000 7fe209800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe204101150 0x7fe2041a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.950+0000 7fe209800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20410f410 con 0x7fe2040ffe50 2024-03-21T11:26:38.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.950+0000 7fe209800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20410f290 con 0x7fe2041007b0 2024-03-21T11:26:38.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.950+0000 7fe209800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20410f590 con 0x7fe204101150 2024-03-21T11:26:38.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.950+0000 7fe203e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2040ffe50 0x7fe204199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.950+0000 7fe203e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2040ffe50 0x7fe204199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33032/0 (socket says 172.21.15.47:33032) 2024-03-21T11:26:38.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.950+0000 7fe203e00640 1 -- 172.21.15.47:0/1296219889 learned_addr learned my addr 172.21.15.47:0/1296219889 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:38.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.950+0000 7fe203400640 1 --2- 172.21.15.47:0/1296219889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2041007b0 0x7fe204199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.950+0000 7fe208e00640 1 --2- 172.21.15.47:0/1296219889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe204101150 0x7fe2041a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.950+0000 7fe203e00640 1 -- 172.21.15.47:0/1296219889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe204101150 msgr2=0x7fe2041a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:38.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.950+0000 7fe203e00640 1 --2- 172.21.15.47:0/1296219889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe204101150 0x7fe2041a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.950+0000 7fe203e00640 1 -- 172.21.15.47:0/1296219889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2041007b0 msgr2=0x7fe204199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:38.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.951+0000 7fe203e00640 1 --2- 172.21.15.47:0/1296219889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2041007b0 0x7fe204199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.951+0000 7fe203e00640 1 -- 172.21.15.47:0/1296219889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2041a16a0 con 0x7fe2040ffe50 2024-03-21T11:26:38.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.951+0000 7fe203e00640 1 --2- 172.21.15.47:0/1296219889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2040ffe50 0x7fe204199a80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4000c00 tx=0x7fe1f402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:38.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.951+0000 7fe200c00640 1 -- 172.21.15.47:0/1296219889 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4043020 con 0x7fe2040ffe50 2024-03-21T11:26:38.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.951+0000 7fe200c00640 1 -- 172.21.15.47:0/1296219889 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1f40054e0 con 0x7fe2040ffe50 2024-03-21T11:26:38.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.951+0000 7fe209800640 1 -- 172.21.15.47:0/1296219889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2041a1930 con 0x7fe2040ffe50 2024-03-21T11:26:38.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.951+0000 7fe200c00640 1 -- 172.21.15.47:0/1296219889 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4003c40 con 0x7fe2040ffe50 2024-03-21T11:26:38.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.951+0000 7fe209800640 1 -- 172.21.15.47:0/1296219889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2041a1e80 con 0x7fe2040ffe50 2024-03-21T11:26:38.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.956+0000 7fe20db8e740 1 Processor -- start 2024-03-21T11:26:38.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.956+0000 7fe20db8e740 1 -- start start 2024-03-21T11:26:38.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe200c00640 1 -- 172.21.15.47:0/1296219889 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe1f4046020 con 0x7fe2040ffe50 2024-03-21T11:26:38.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe20db8e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e8b2b0 0x5570c0e8b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe20db8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8bca0 0x5570c0e907a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe20db8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e90e10 0x5570c0e932a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe20db8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c0df0eb0 con 0x5570c0e90e10 2024-03-21T11:26:38.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe20db8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c0d99a60 con 0x5570c0e8bca0 2024-03-21T11:26:38.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe20db8e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c0decf50 con 0x5570c0e8b2b0 2024-03-21T11:26:38.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe208e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e8b2b0 0x5570c0e8b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe203e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8bca0 0x5570c0e907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe208e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e8b2b0 0x5570c0e8b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47280/0 (socket says 172.21.15.47:47280) 2024-03-21T11:26:38.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe208e00640 1 -- 172.21.15.47:0/1453885585 learned_addr learned my addr 172.21.15.47:0/1453885585 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:38.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe203400640 1 --2- 172.21.15.47:0/1453885585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e90e10 0x5570c0e932a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe208e00640 1 -- 172.21.15.47:0/1453885585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8bca0 msgr2=0x5570c0e907a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:38.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe208e00640 1 --2- 172.21.15.47:0/1453885585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8bca0 0x5570c0e907a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe208e00640 1 -- 172.21.15.47:0/1453885585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e90e10 msgr2=0x5570c0e932a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:38.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe208e00640 1 --2- 172.21.15.47:0/1453885585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e90e10 0x5570c0e932a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.957+0000 7fe208e00640 1 -- 172.21.15.47:0/1453885585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570c0bcd7e0 con 0x5570c0e8b2b0 2024-03-21T11:26:38.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.958+0000 7fe208e00640 1 --2- 172.21.15.47:0/1453885585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e8b2b0 0x5570c0e8b6d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009fd0 tx=0x7fe1fc010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e068078299edab8b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:38.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.958+0000 7fe1f9600640 1 -- 172.21.15.47:0/1453885585 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc011700 con 0x5570c0e8b2b0 2024-03-21T11:26:38.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.958+0000 7fe1f9600640 1 -- 172.21.15.47:0/1453885585 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1fc011d80 con 0x5570c0e8b2b0 2024-03-21T11:26:38.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.958+0000 7fe1f9600640 1 -- 172.21.15.47:0/1453885585 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc01a380 con 0x5570c0e8b2b0 2024-03-21T11:26:38.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.958+0000 7fe200c00640 1 --2- 172.21.15.47:0/1296219889 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe1e80a2f20 0x7fe1e80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.958+0000 7fe200c00640 1 -- 172.21.15.47:0/1296219889 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe1f40f1890 con 0x7fe2040ffe50 2024-03-21T11:26:38.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.959+0000 7fe203e00640 1 --2- 172.21.15.47:0/1296219889 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe1e80a2f20 0x7fe1e80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.959+0000 7fe20db8e740 1 -- 172.21.15.47:0/1453885585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e8b2b0 msgr2=0x5570c0e8b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:38.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.959+0000 7fe20db8e740 1 --2- 172.21.15.47:0/1453885585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e8b2b0 0x5570c0e8b6d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009fd0 tx=0x7fe1fc010860 comp rx=0 tx=0).stop 2024-03-21T11:26:38.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.959+0000 7fe20db8e740 1 -- 172.21.15.47:0/1453885585 shutdown_connections 2024-03-21T11:26:38.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.959+0000 7fe20db8e740 1 --2- 172.21.15.47:0/1453885585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e90e10 0x5570c0e932a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.959+0000 7fe20db8e740 1 --2- 172.21.15.47:0/1453885585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8bca0 0x5570c0e907a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.959+0000 7fe20db8e740 1 --2- 172.21.15.47:0/1453885585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e8b2b0 0x5570c0e8b6d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.959+0000 7fe20db8e740 1 -- 172.21.15.47:0/1453885585 >> 172.21.15.47:0/1453885585 conn(0x5570c0e86820 msgr2=0x5570c0e86c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:38.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.959+0000 7fe20db8e740 1 -- 172.21.15.47:0/1453885585 shutdown_connections 2024-03-21T11:26:38.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.959+0000 7fe20db8e740 1 -- 172.21.15.47:0/1453885585 wait complete. 2024-03-21T11:26:38.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.960+0000 7fe203e00640 1 --2- 172.21.15.47:0/1296219889 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe1e80a2f20 0x7fe1e80a53e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40ba070 tx=0x7fe1f40f7530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:38.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.960+0000 7fe20db8e740 1 -- start start 2024-03-21T11:26:38.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.960+0000 7fe20db8e740 1 Processor -- start 2024-03-21T11:26:38.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe20db8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e8b2b0 0x5570c0e955b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe20db8e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e8bca0 0x5570c0e95af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe20db8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e90e10 0x5570c0e9cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe20db8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c0e93dc0 con 0x5570c0e8b2b0 2024-03-21T11:26:38.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe20db8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c0daceb0 con 0x5570c0e90e10 2024-03-21T11:26:38.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe20db8e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c0e93f40 con 0x5570c0e8bca0 2024-03-21T11:26:38.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe20db8e740 10 client.0 _ll_register_callbacks cb 0x5570c0da1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:38.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe203400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e8bca0 0x5570c0e95af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe208e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e8b2b0 0x5570c0e955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe203400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e8bca0 0x5570c0e95af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47282/0 (socket says 172.21.15.47:47282) 2024-03-21T11:26:38.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe203400640 1 -- 172.21.15.47:0/3482839244 learned_addr learned my addr 172.21.15.47:0/3482839244 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:38.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe208e00640 1 --2- 172.21.15.47:0/3482839244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e90e10 0x5570c0e9cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe203400640 1 -- 172.21.15.47:0/3482839244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e90e10 msgr2=0x5570c0e9cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:38.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe203400640 1 --2- 172.21.15.47:0/3482839244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e90e10 0x5570c0e9cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe203400640 1 -- 172.21.15.47:0/3482839244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e8b2b0 msgr2=0x5570c0e955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:38.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe203400640 1 --2- 172.21.15.47:0/3482839244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e8b2b0 0x5570c0e955b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe203400640 1 -- 172.21.15.47:0/3482839244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570c0c2df70 con 0x5570c0e8bca0 2024-03-21T11:26:38.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.961+0000 7fe208e00640 1 --2- 172.21.15.47:0/3482839244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e8b2b0 0x5570c0e955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:38.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.962+0000 7fe203400640 1 --2- 172.21.15.47:0/3482839244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e8bca0 0x5570c0e95af0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0012720 tx=0x7fe1f0012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3363358c9f4cbde server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:38.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.962+0000 7fe1ee000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:38.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.962+0000 7fe1ee000640 1 -- 172.21.15.47:0/3482839244 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f000f8c0 con 0x5570c0e8bca0 2024-03-21T11:26:38.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.962+0000 7fe1ee000640 1 -- 172.21.15.47:0/3482839244 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe1f000fa60 con 0x5570c0e8bca0 2024-03-21T11:26:38.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.962+0000 7fe20db8e740 1 -- 172.21.15.47:0/3482839244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1fc011d80 con 0x5570c0e8bca0 2024-03-21T11:26:38.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.962+0000 7fe1ee000640 1 -- 172.21.15.47:0/3482839244 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1f000fca0 con 0x5570c0e8bca0 2024-03-21T11:26:38.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.962+0000 7fe20db8e740 10 client.26539 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:38.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.962+0000 7fe20db8e740 1 -- 172.21.15.47:0/3482839244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5570c0ac5170 con 0x5570c0e8bca0 2024-03-21T11:26:38.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.963+0000 7fe1ee000640 1 -- 172.21.15.47:0/3482839244 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1fc011d80 con 0x5570c0e8bca0 2024-03-21T11:26:38.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.963+0000 7fe1ee000640 1 -- 172.21.15.47:0/3482839244 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe1f0010550 con 0x5570c0e8bca0 2024-03-21T11:26:38.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.963+0000 7fe20db8e740 10 client.26539 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.963+0000 7fe20db8e740 10 client.26539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.963+0000 7fe20db8e740 20 client.26539 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.963+0000 7fe20db8e740 1 --2- 172.21.15.47:0/3482839244 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5570c0ea1480 0x5570c0ea38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.963+0000 7fe20db8e740 4 client.26539 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.963+0000 7fe20db8e740 1 -- 172.21.15.47:0/3482839244 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570c0dfe3f0 con 0x5570c0ea1480 2024-03-21T11:26:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.963+0000 7fe208e00640 1 --2- 172.21.15.47:0/3482839244 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5570c0ea1480 0x5570c0ea38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.964+0000 7fe208e00640 1 --2- 172.21.15.47:0/3482839244 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5570c0ea1480 0x5570c0ea38e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2a6993784bbced server_cookie=3fd4112bf818c97 in_seq=0 out_seq=0 2024-03-21T11:26:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.964+0000 7fe1ee000640 10 client.26539 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.965+0000 7fe1ee000640 1 -- 172.21.15.47:0/3482839244 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5570c0dfe3f0 con 0x5570c0ea1480 2024-03-21T11:26:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.965+0000 7fe1ee000640 10 client.26539 handle_command_reply: tid=0 2024-03-21T11:26:38.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.965+0000 7fe20db8e740 1 client.26539 shutdown 2024-03-21T11:26:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.965+0000 7fe20db8e740 1 -- 172.21.15.47:0/3482839244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e8bca0 msgr2=0x5570c0e95af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.965+0000 7fe20db8e740 1 --2- 172.21.15.47:0/3482839244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e8bca0 0x5570c0e95af0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0012720 tx=0x7fe1f0012be0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.965+0000 7fe20db8e740 1 -- 172.21.15.47:0/3482839244 shutdown_connections 2024-03-21T11:26:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.965+0000 7fe20db8e740 1 --2- 172.21.15.47:0/3482839244 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5570c0ea1480 0x5570c0ea38e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.965+0000 7fe20db8e740 1 --2- 172.21.15.47:0/3482839244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e90e10 0x5570c0e9cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.965+0000 7fe20db8e740 1 --2- 172.21.15.47:0/3482839244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e8bca0 0x5570c0e95af0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.965+0000 7fe20db8e740 1 --2- 172.21.15.47:0/3482839244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e8b2b0 0x5570c0e955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.965+0000 7fe1ee000640 0 client.26539 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.965+0000 7fe20db8e740 1 -- 172.21.15.47:0/3482839244 >> 172.21.15.47:0/3482839244 conn(0x5570c0e86820 msgr2=0x5570c0e85a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:38.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.966+0000 7fe20db8e740 1 -- 172.21.15.47:0/3482839244 shutdown_connections 2024-03-21T11:26:38.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.966+0000 7fe20db8e740 1 -- 172.21.15.47:0/3482839244 wait complete. 2024-03-21T11:26:38.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.966+0000 7fe20db8e740 20 client.26539 trim_cache size 0 max 16384 2024-03-21T11:26:38.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.988+0000 7fe20db8e740 1 Processor -- start 2024-03-21T11:26:38.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.988+0000 7fe20db8e740 1 -- start start 2024-03-21T11:26:38.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.988+0000 7fe20db8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e86820 0x5570c0e8c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.988+0000 7fe20db8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8b2b0 0x5570c0e9caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.988+0000 7fe20db8e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e90e10 0x5570c0e8cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.988+0000 7fe20db8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2041ac0c0 con 0x5570c0e86820 2024-03-21T11:26:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.988+0000 7fe20db8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2041ada40 con 0x5570c0e8b2b0 2024-03-21T11:26:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.988+0000 7fe20db8e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2041ac720 con 0x5570c0e90e10 2024-03-21T11:26:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.988+0000 7fe203400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8b2b0 0x5570c0e9caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.988+0000 7fe208e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e86820 0x5570c0e8c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.988+0000 7fe203400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8b2b0 0x5570c0e9caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38586/0 (socket says 172.21.15.47:38586) 2024-03-21T11:26:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.988+0000 7fe203400640 1 -- 172.21.15.47:0/2252414174 learned_addr learned my addr 172.21.15.47:0/2252414174 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:38.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.988+0000 7fe208e00640 1 --2- 172.21.15.47:0/2252414174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e90e10 0x5570c0e8cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.989+0000 7fe203400640 1 -- 172.21.15.47:0/2252414174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e90e10 msgr2=0x5570c0e8cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:38.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.989+0000 7fe203400640 1 --2- 172.21.15.47:0/2252414174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e90e10 0x5570c0e8cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.989+0000 7fe203400640 1 -- 172.21.15.47:0/2252414174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e86820 msgr2=0x5570c0e8c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:38.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.989+0000 7fe203400640 1 --2- 172.21.15.47:0/2252414174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e86820 0x5570c0e8c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.989+0000 7fe203400640 1 -- 172.21.15.47:0/2252414174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570c0ea1260 con 0x5570c0e8b2b0 2024-03-21T11:26:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.989+0000 7fe203400640 1 --2- 172.21.15.47:0/2252414174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8b2b0 0x5570c0e9caf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0010a10 tx=0x7fe1f0009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d9f3c886c4d0e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.989+0000 7fe1f8c00640 1 -- 172.21.15.47:0/2252414174 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0010b80 con 0x5570c0e8b2b0 2024-03-21T11:26:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.989+0000 7fe1f8c00640 1 -- 172.21.15.47:0/2252414174 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1f0010d20 con 0x5570c0e8b2b0 2024-03-21T11:26:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.989+0000 7fe1f8c00640 1 -- 172.21.15.47:0/2252414174 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0005610 con 0x5570c0e8b2b0 2024-03-21T11:26:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.990+0000 7fe20db8e740 1 -- 172.21.15.47:0/2252414174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8b2b0 msgr2=0x5570c0e9caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.990+0000 7fe20db8e740 1 --2- 172.21.15.47:0/2252414174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8b2b0 0x5570c0e9caf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0010a10 tx=0x7fe1f0009640 comp rx=0 tx=0).stop 2024-03-21T11:26:38.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.991+0000 7fe20db8e740 1 -- 172.21.15.47:0/2252414174 shutdown_connections 2024-03-21T11:26:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.991+0000 7fe20db8e740 1 --2- 172.21.15.47:0/2252414174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e90e10 0x5570c0e8cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.991+0000 7fe20db8e740 1 --2- 172.21.15.47:0/2252414174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8b2b0 0x5570c0e9caf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.991+0000 7fe20db8e740 1 --2- 172.21.15.47:0/2252414174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e86820 0x5570c0e8c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.991+0000 7fe20db8e740 1 -- 172.21.15.47:0/2252414174 >> 172.21.15.47:0/2252414174 conn(0x5570c0e90910 msgr2=0x5570c0e8a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.991+0000 7fe20db8e740 1 -- 172.21.15.47:0/2252414174 shutdown_connections 2024-03-21T11:26:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.991+0000 7fe20db8e740 1 -- 172.21.15.47:0/2252414174 wait complete. 2024-03-21T11:26:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.991+0000 7fe20db8e740 1 -- start start 2024-03-21T11:26:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.991+0000 7fe20db8e740 1 Processor -- start 2024-03-21T11:26:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe20db8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e86820 0x5570c0e9c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe20db8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8b2b0 0x5570c0e9b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe20db8e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e90e10 0x5570c0e9b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe20db8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2041ad3e0 con 0x5570c0e86820 2024-03-21T11:26:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe20db8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2041acd80 con 0x5570c0e8b2b0 2024-03-21T11:26:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe20db8e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2041aa280 con 0x5570c0e90e10 2024-03-21T11:26:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe20db8e740 10 client.0 _ll_register_callbacks cb 0x7fe1f0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe203400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8b2b0 0x5570c0e9b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe203400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8b2b0 0x5570c0e9b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38598/0 (socket says 172.21.15.47:38598) 2024-03-21T11:26:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe203400640 1 -- 172.21.15.47:0/2899685098 learned_addr learned my addr 172.21.15.47:0/2899685098 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe208e00640 1 --2- 172.21.15.47:0/2899685098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e86820 0x5570c0e9c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe208e00640 1 --2- 172.21.15.47:0/2899685098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e90e10 0x5570c0e9b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe203400640 1 -- 172.21.15.47:0/2899685098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e90e10 msgr2=0x5570c0e9b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:38.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe203400640 1 --2- 172.21.15.47:0/2899685098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e90e10 0x5570c0e9b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe203400640 1 -- 172.21.15.47:0/2899685098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e86820 msgr2=0x5570c0e9c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:38.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe203400640 1 --2- 172.21.15.47:0/2899685098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e86820 0x5570c0e9c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:38.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe203400640 1 -- 172.21.15.47:0/2899685098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570c0e9bc40 con 0x5570c0e8b2b0 2024-03-21T11:26:38.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.992+0000 7fe208e00640 1 --2- 172.21.15.47:0/2899685098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e86820 0x5570c0e9c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:38.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.993+0000 7fe203400640 1 --2- 172.21.15.47:0/2899685098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8b2b0 0x5570c0e9b010 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0010990 tx=0x7fe1f000f9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bcc854bb156d753 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:38.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.993+0000 7fe1ee000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:38.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.993+0000 7fe1ee000640 1 -- 172.21.15.47:0/2899685098 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0010b30 con 0x5570c0e8b2b0 2024-03-21T11:26:38.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.993+0000 7fe20db8e740 1 -- 172.21.15.47:0/2899685098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1f0010d20 con 0x5570c0e8b2b0 2024-03-21T11:26:38.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.993+0000 7fe1ee000640 1 -- 172.21.15.47:0/2899685098 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe1f0005560 con 0x5570c0e8b2b0 2024-03-21T11:26:38.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.993+0000 7fe1ee000640 1 -- 172.21.15.47:0/2899685098 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1f0039050 con 0x5570c0e8b2b0 2024-03-21T11:26:38.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.993+0000 7fe20db8e740 10 client.26525 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:38.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.993+0000 7fe20db8e740 1 -- 172.21.15.47:0/2899685098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5570c0efdcd0 con 0x5570c0e8b2b0 2024-03-21T11:26:38.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.994+0000 7fe1ee000640 1 -- 172.21.15.47:0/2899685098 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1f0010d20 con 0x5570c0e8b2b0 2024-03-21T11:26:38.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.994+0000 7fe1ee000640 1 -- 172.21.15.47:0/2899685098 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe1f001c470 con 0x5570c0e8b2b0 2024-03-21T11:26:38.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.994+0000 7fe20db8e740 10 client.26525 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:38.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.994+0000 7fe20db8e740 10 client.26525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:38.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.994+0000 7fe20db8e740 20 client.26525 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:38.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.994+0000 7fe20db8e740 1 --2- 172.21.15.47:0/2899685098 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5570c0efe570 0x5570c0f00990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:38.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.994+0000 7fe20db8e740 4 client.26525 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:26:38.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.994+0000 7fe20db8e740 1 -- 172.21.15.47:0/2899685098 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5570c0df8740 con 0x5570c0efe570 2024-03-21T11:26:39.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.994+0000 7fe208e00640 1 --2- 172.21.15.47:0/2899685098 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5570c0efe570 0x5570c0f00990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.995+0000 7fe208e00640 1 --2- 172.21.15.47:0/2899685098 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5570c0efe570 0x5570c0f00990 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=504d00b51da02c49 server_cookie=52e2b7e079a26403 in_seq=0 out_seq=0 2024-03-21T11:26:39.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.995+0000 7fe1ee000640 10 client.26525 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:39.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.996+0000 7fe1ee000640 1 -- 172.21.15.47:0/2899685098 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5570c0df8740 con 0x5570c0efe570 2024-03-21T11:26:39.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.996+0000 7fe1ee000640 10 client.26525 handle_command_reply: tid=0 2024-03-21T11:26:39.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.997+0000 7fe20db8e740 1 client.26525 shutdown 2024-03-21T11:26:39.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.997+0000 7fe20db8e740 1 -- 172.21.15.47:0/2899685098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8b2b0 msgr2=0x5570c0e9b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.997+0000 7fe20db8e740 1 --2- 172.21.15.47:0/2899685098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8b2b0 0x5570c0e9b010 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0010990 tx=0x7fe1f000f9a0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.997+0000 7fe20db8e740 1 -- 172.21.15.47:0/2899685098 shutdown_connections 2024-03-21T11:26:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.997+0000 7fe20db8e740 1 --2- 172.21.15.47:0/2899685098 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5570c0efe570 0x5570c0f00990 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.997+0000 7fe20db8e740 1 --2- 172.21.15.47:0/2899685098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5570c0e90e10 0x5570c0e9b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.997+0000 7fe20db8e740 1 --2- 172.21.15.47:0/2899685098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570c0e8b2b0 0x5570c0e9b010 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.997+0000 7fe20db8e740 1 --2- 172.21.15.47:0/2899685098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5570c0e86820 0x5570c0e9c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.997+0000 7fe20db8e740 1 -- 172.21.15.47:0/2899685098 >> 172.21.15.47:0/2899685098 conn(0x5570c0e90910 msgr2=0x5570c0e851f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.997+0000 7fe1ee000640 0 client.26525 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.997+0000 7fe20db8e740 1 -- 172.21.15.47:0/2899685098 shutdown_connections 2024-03-21T11:26:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.997+0000 7fe20db8e740 1 -- 172.21.15.47:0/2899685098 wait complete. 2024-03-21T11:26:39.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.997+0000 7fe20db8e740 20 client.26525 trim_cache size 0 max 16384 2024-03-21T11:26:39.003 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:26:39.003 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:26:39.003 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "e1e8e7e0-091c-43de-bd0d-a44f57540792", 2024-03-21T11:26:39.003 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:26:39.003 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:26:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.998+0000 7fe209800640 1 -- 172.21.15.47:0/1296219889 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe1e80a2f20 msgr2=0x7fe1e80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.998+0000 7fe209800640 1 --2- 172.21.15.47:0/1296219889 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe1e80a2f20 0x7fe1e80a53e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40ba070 tx=0x7fe1f40f7530 comp rx=0 tx=0).stop 2024-03-21T11:26:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.998+0000 7fe209800640 1 -- 172.21.15.47:0/1296219889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2040ffe50 msgr2=0x7fe204199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.998+0000 7fe209800640 1 --2- 172.21.15.47:0/1296219889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2040ffe50 0x7fe204199a80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4000c00 tx=0x7fe1f402c650 comp rx=0 tx=0).stop 2024-03-21T11:26:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.998+0000 7fe209800640 1 -- 172.21.15.47:0/1296219889 shutdown_connections 2024-03-21T11:26:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.998+0000 7fe209800640 1 --2- 172.21.15.47:0/1296219889 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe1e80a2f20 0x7fe1e80a53e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.998+0000 7fe209800640 1 --2- 172.21.15.47:0/1296219889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe204101150 0x7fe2041a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.998+0000 7fe209800640 1 --2- 172.21.15.47:0/1296219889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2041007b0 0x7fe204199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.998+0000 7fe209800640 1 --2- 172.21.15.47:0/1296219889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2040ffe50 0x7fe204199a80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.998+0000 7fe209800640 1 -- 172.21.15.47:0/1296219889 >> 172.21.15.47:0/1296219889 conn(0x7fe2040fba10 msgr2=0x7fe2040fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:39.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.998+0000 7fe209800640 1 -- 172.21.15.47:0/1296219889 shutdown_connections 2024-03-21T11:26:39.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:38.998+0000 7fe209800640 1 -- 172.21.15.47:0/1296219889 wait complete. 2024-03-21T11:26:39.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:26:39.018 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:26:39.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.270+0000 7f05e1c00640 1 -- 172.21.15.47:0/653218797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05dc103150 msgr2=0x7f05dc1099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:39.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.270+0000 7f05e1c00640 1 --2- 172.21.15.47:0/653218797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05dc103150 0x7f05dc1099e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009ca0 tx=0x7f05d402c100 comp rx=0 tx=0).stop 2024-03-21T11:26:39.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.270+0000 7f05e1c00640 1 -- 172.21.15.47:0/653218797 shutdown_connections 2024-03-21T11:26:39.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.270+0000 7f05e1c00640 1 --2- 172.21.15.47:0/653218797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05dc103150 0x7f05dc1099e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.270+0000 7f05e1c00640 1 --2- 172.21.15.47:0/653218797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05dc102790 0x7f05dc102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.270+0000 7f05e1c00640 1 --2- 172.21.15.47:0/653218797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05dc1015a0 0x7f05dc1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.270+0000 7f05e1c00640 1 -- 172.21.15.47:0/653218797 >> 172.21.15.47:0/653218797 conn(0x7f05dc0fcb10 msgr2=0x7f05dc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:39.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.270+0000 7f05e1c00640 1 -- 172.21.15.47:0/653218797 shutdown_connections 2024-03-21T11:26:39.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.271+0000 7f05e1c00640 1 -- 172.21.15.47:0/653218797 wait complete. 2024-03-21T11:26:39.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.272+0000 7f05e1c00640 1 Processor -- start 2024-03-21T11:26:39.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.272+0000 7f05e1c00640 1 -- start start 2024-03-21T11:26:39.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.272+0000 7f05e1c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05dc1015a0 0x7f05dc199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.272+0000 7f05e1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05dc102790 0x7f05dc199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.272+0000 7f05e1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05dc103150 0x7f05dc1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.272+0000 7f05e1c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc10d200 con 0x7f05dc103150 2024-03-21T11:26:39.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.272+0000 7f05e1c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc10d080 con 0x7f05dc102790 2024-03-21T11:26:39.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.272+0000 7f05e1c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc10d380 con 0x7f05dc1015a0 2024-03-21T11:26:39.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.272+0000 7f05d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05dc102790 0x7f05dc199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.272+0000 7f05d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05dc102790 0x7f05dc199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38618/0 (socket says 172.21.15.47:38618) 2024-03-21T11:26:39.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.272+0000 7f05e1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05dc103150 0x7f05dc1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.273+0000 7f05d3e00640 1 -- 172.21.15.47:0/3754161811 learned_addr learned my addr 172.21.15.47:0/3754161811 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:39.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.273+0000 7f05dbe00640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05dc1015a0 0x7f05dc199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.273+0000 7f05d3e00640 1 -- 172.21.15.47:0/3754161811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05dc1015a0 msgr2=0x7f05dc199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.273+0000 7f05d3e00640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05dc1015a0 0x7f05dc199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.273+0000 7f05d3e00640 1 -- 172.21.15.47:0/3754161811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05dc103150 msgr2=0x7f05dc1a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.273+0000 7f05d3e00640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05dc103150 0x7f05dc1a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.273+0000 7f05d3e00640 1 -- 172.21.15.47:0/3754161811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc1a16b0 con 0x7f05dc102790 2024-03-21T11:26:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.273+0000 7f05dbe00640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05dc1015a0 0x7f05dc199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.273+0000 7f05e1200640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05dc103150 0x7f05dc1a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:39.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.273+0000 7f05d3e00640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05dc102790 0x7f05dc199fd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f05c400e9f0 tx=0x7f05c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.274+0000 7f05d9600640 1 -- 172.21.15.47:0/3754161811 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c400cd80 con 0x7f05dc102790 2024-03-21T11:26:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.274+0000 7f05e1c00640 1 -- 172.21.15.47:0/3754161811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05dc1a19a0 con 0x7f05dc102790 2024-03-21T11:26:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.274+0000 7f05d9600640 1 -- 172.21.15.47:0/3754161811 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05c4010910 con 0x7f05dc102790 2024-03-21T11:26:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.274+0000 7f05d9600640 1 -- 172.21.15.47:0/3754161811 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c40188f0 con 0x7f05dc102790 2024-03-21T11:26:39.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.274+0000 7f05e1c00640 1 -- 172.21.15.47:0/3754161811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05dc1a1f40 con 0x7f05dc102790 2024-03-21T11:26:39.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.277+0000 7f05e5f8b740 1 Processor -- start 2024-03-21T11:26:39.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.277+0000 7f05d9600640 1 -- 172.21.15.47:0/3754161811 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f05c4010ac0 con 0x7f05dc102790 2024-03-21T11:26:39.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.277+0000 7f05e5f8b740 1 -- start start 2024-03-21T11:26:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.278+0000 7f05e5f8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f8ff050 0x56040f8ff470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.278+0000 7f05e5f8b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ffa40 0x56040f904540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.278+0000 7f05e5f8b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f904bb0 0x56040f907040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.278+0000 7f05e5f8b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040f864250 con 0x56040f904bb0 2024-03-21T11:26:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.278+0000 7f05e5f8b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040f80ca30 con 0x56040f8ffa40 2024-03-21T11:26:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.278+0000 7f05e5f8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040f8602f0 con 0x56040f8ff050 2024-03-21T11:26:39.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.278+0000 7f05dbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ffa40 0x56040f904540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.278+0000 7f05e1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f8ff050 0x56040f8ff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.278+0000 7f05dbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ffa40 0x56040f904540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38634/0 (socket says 172.21.15.47:38634) 2024-03-21T11:26:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.278+0000 7f05dbe00640 1 -- 172.21.15.47:0/3270580652 learned_addr learned my addr 172.21.15.47:0/3270580652 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.278+0000 7f05d3e00640 1 --2- 172.21.15.47:0/3270580652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f904bb0 0x56040f907040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.278+0000 7f05dbe00640 1 -- 172.21.15.47:0/3270580652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f8ff050 msgr2=0x56040f8ff470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.278+0000 7f05dbe00640 1 --2- 172.21.15.47:0/3270580652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f8ff050 0x56040f8ff470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.278+0000 7f05dbe00640 1 -- 172.21.15.47:0/3270580652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f904bb0 msgr2=0x56040f907040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.279+0000 7f05dbe00640 1 --2- 172.21.15.47:0/3270580652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f904bb0 0x56040f907040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.279+0000 7f05dbe00640 1 -- 172.21.15.47:0/3270580652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56040f7f7690 con 0x56040f8ffa40 2024-03-21T11:26:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.279+0000 7f05e1200640 1 --2- 172.21.15.47:0/3270580652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f8ff050 0x56040f8ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.279+0000 7f05dbe00640 1 --2- 172.21.15.47:0/3270580652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ffa40 0x56040f904540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f05cc010450 tx=0x7f05cc010910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cdc0c7071f98946 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.279+0000 7f05d9600640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05c00a3020 0x7f05c00a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.279+0000 7f05d1600640 1 -- 172.21.15.47:0/3270580652 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc011760 con 0x56040f8ffa40 2024-03-21T11:26:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.279+0000 7f05d9600640 1 -- 172.21.15.47:0/3754161811 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f05c40cfb30 con 0x7f05dc102790 2024-03-21T11:26:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.279+0000 7f05d1600640 1 -- 172.21.15.47:0/3270580652 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05cc011de0 con 0x56040f8ffa40 2024-03-21T11:26:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.279+0000 7f05d1600640 1 -- 172.21.15.47:0/3270580652 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc01a380 con 0x56040f8ffa40 2024-03-21T11:26:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.279+0000 7f05e1200640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05c00a3020 0x7f05c00a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.280+0000 7f05e5f8b740 1 -- 172.21.15.47:0/3270580652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ffa40 msgr2=0x56040f904540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:39.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.280+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/3270580652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ffa40 0x56040f904540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f05cc010450 tx=0x7f05cc010910 comp rx=0 tx=0).stop 2024-03-21T11:26:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.280+0000 7f05e5f8b740 1 -- 172.21.15.47:0/3270580652 shutdown_connections 2024-03-21T11:26:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.280+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/3270580652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f904bb0 0x56040f907040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.280+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/3270580652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ffa40 0x56040f904540 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.280+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/3270580652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f8ff050 0x56040f8ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.280+0000 7f05e5f8b740 1 -- 172.21.15.47:0/3270580652 >> 172.21.15.47:0/3270580652 conn(0x56040f8c7200 msgr2=0x56040f8fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.280+0000 7f05e1200640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05c00a3020 0x7f05c00a54e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f05d402c640 tx=0x7f05d4032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:39.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.281+0000 7f05e5f8b740 1 -- 172.21.15.47:0/3270580652 shutdown_connections 2024-03-21T11:26:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.281+0000 7f05e5f8b740 1 -- 172.21.15.47:0/3270580652 wait complete. 2024-03-21T11:26:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.281+0000 7f05e5f8b740 1 -- start start 2024-03-21T11:26:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.282+0000 7f05e5f8b740 1 Processor -- start 2024-03-21T11:26:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05e5f8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f8ff050 0x56040f911830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05e5f8b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ffa40 0x56040f911d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05e5f8b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f904bb0 0x56040f918d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05e5f8b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040f907b60 con 0x56040f8ffa40 2024-03-21T11:26:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05e5f8b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040f81fe80 con 0x56040f904bb0 2024-03-21T11:26:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05e5f8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040f907ce0 con 0x56040f8ff050 2024-03-21T11:26:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05dbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f8ff050 0x56040f911830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05dbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f8ff050 0x56040f911830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47334/0 (socket says 172.21.15.47:47334) 2024-03-21T11:26:39.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05dbe00640 1 -- 172.21.15.47:0/2365301820 learned_addr learned my addr 172.21.15.47:0/2365301820 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05e1200640 1 --2- 172.21.15.47:0/2365301820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f904bb0 0x56040f918d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05d3e00640 1 --2- 172.21.15.47:0/2365301820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ffa40 0x56040f911d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05e5f8b740 10 client.0 _ll_register_callbacks cb 0x56040f832130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05dbe00640 1 -- 172.21.15.47:0/2365301820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f904bb0 msgr2=0x56040f918d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05dbe00640 1 --2- 172.21.15.47:0/2365301820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f904bb0 0x56040f918d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05dbe00640 1 -- 172.21.15.47:0/2365301820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ffa40 msgr2=0x56040f911d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05dbe00640 1 --2- 172.21.15.47:0/2365301820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ffa40 0x56040f911d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05dbe00640 1 -- 172.21.15.47:0/2365301820 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56040f6427d0 con 0x56040f8ff050 2024-03-21T11:26:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.283+0000 7f05d3e00640 1 --2- 172.21.15.47:0/2365301820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ffa40 0x56040f911d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.284+0000 7f05dbe00640 1 --2- 172.21.15.47:0/2365301820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f8ff050 0x56040f911830 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f05cc000c00 tx=0x7f05cc01c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db79a09e2e5cfaef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.284+0000 7f05ca000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:39.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.284+0000 7f05ca000640 1 -- 172.21.15.47:0/2365301820 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc004450 con 0x56040f8ff050 2024-03-21T11:26:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.284+0000 7f05ca000640 1 -- 172.21.15.47:0/2365301820 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f05cc005050 con 0x56040f8ff050 2024-03-21T11:26:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.284+0000 7f05e5f8b740 1 -- 172.21.15.47:0/2365301820 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05cc011de0 con 0x56040f8ff050 2024-03-21T11:26:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.284+0000 7f05ca000640 1 -- 172.21.15.47:0/2365301820 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05cc032050 con 0x56040f8ff050 2024-03-21T11:26:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.284+0000 7f05e5f8b740 10 client.26557 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.284+0000 7f05e5f8b740 1 -- 172.21.15.47:0/2365301820 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56040f5783f0 con 0x56040f8ff050 2024-03-21T11:26:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.285+0000 7f05ca000640 1 -- 172.21.15.47:0/2365301820 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05cc011de0 con 0x56040f8ff050 2024-03-21T11:26:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.285+0000 7f05ca000640 1 -- 172.21.15.47:0/2365301820 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f05cc02d030 con 0x56040f8ff050 2024-03-21T11:26:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.285+0000 7f05e5f8b740 10 client.26557 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.285+0000 7f05e5f8b740 10 client.26557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.285+0000 7f05e5f8b740 20 client.26557 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.285+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/2365301820 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56040f91d2c0 0x56040f91f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.285+0000 7f05e5f8b740 4 client.26557 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.286+0000 7f05e5f8b740 1 -- 172.21.15.47:0/2365301820 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56040f871790 con 0x56040f91d2c0 2024-03-21T11:26:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.286+0000 7f05d3e00640 1 --2- 172.21.15.47:0/2365301820 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56040f91d2c0 0x56040f91f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.286+0000 7f05d3e00640 1 --2- 172.21.15.47:0/2365301820 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56040f91d2c0 0x56040f91f6b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ef6d4e7824cc54d server_cookie=774486f58ae0660c in_seq=0 out_seq=0 2024-03-21T11:26:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.286+0000 7f05ca000640 10 client.26557 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.287+0000 7f05ca000640 1 -- 172.21.15.47:0/2365301820 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56040f871790 con 0x56040f91d2c0 2024-03-21T11:26:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.287+0000 7f05ca000640 10 client.26557 handle_command_reply: tid=0 2024-03-21T11:26:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.287+0000 7f05e5f8b740 1 client.26557 shutdown 2024-03-21T11:26:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.288+0000 7f05e5f8b740 1 -- 172.21.15.47:0/2365301820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f8ff050 msgr2=0x56040f911830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.288+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/2365301820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f8ff050 0x56040f911830 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f05cc000c00 tx=0x7f05cc01c8f0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.288+0000 7f05e5f8b740 1 -- 172.21.15.47:0/2365301820 shutdown_connections 2024-03-21T11:26:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.288+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/2365301820 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56040f91d2c0 0x56040f91f6b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.288+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/2365301820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f904bb0 0x56040f918d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.288+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/2365301820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ffa40 0x56040f911d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.288+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/2365301820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f8ff050 0x56040f911830 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.288+0000 7f05e5f8b740 1 -- 172.21.15.47:0/2365301820 >> 172.21.15.47:0/2365301820 conn(0x56040f8c7200 msgr2=0x56040f8f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:39.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.288+0000 7f05ca000640 0 client.26557 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:39.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.288+0000 7f05e5f8b740 1 -- 172.21.15.47:0/2365301820 shutdown_connections 2024-03-21T11:26:39.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.288+0000 7f05e5f8b740 1 -- 172.21.15.47:0/2365301820 wait complete. 2024-03-21T11:26:39.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.288+0000 7f05e5f8b740 20 client.26557 trim_cache size 0 max 16384 2024-03-21T11:26:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.306+0000 7f05e5f8b740 1 Processor -- start 2024-03-21T11:26:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.306+0000 7f05e5f8b740 1 -- start start 2024-03-21T11:26:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.306+0000 7f05e5f8b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ff050 0x56040f9004b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.306+0000 7f05e5f8b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ffa40 0x56040f918d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.306+0000 7f05e5f8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f904bb0 0x56040f923ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.306+0000 7f05e5f8b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc1a7df0 con 0x56040f8ffa40 2024-03-21T11:26:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.306+0000 7f05e5f8b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc1a9770 con 0x56040f8ff050 2024-03-21T11:26:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.306+0000 7f05e5f8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc1a8450 con 0x56040f904bb0 2024-03-21T11:26:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.307+0000 7f05dbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ff050 0x56040f9004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.307+0000 7f05e1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f904bb0 0x56040f923ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.307+0000 7f05dbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ff050 0x56040f9004b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38644/0 (socket says 172.21.15.47:38644) 2024-03-21T11:26:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.307+0000 7f05dbe00640 1 -- 172.21.15.47:0/597833353 learned_addr learned my addr 172.21.15.47:0/597833353 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.307+0000 7f05d3e00640 1 --2- 172.21.15.47:0/597833353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ffa40 0x56040f918d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.307+0000 7f05dbe00640 1 -- 172.21.15.47:0/597833353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f904bb0 msgr2=0x56040f923ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.307+0000 7f05dbe00640 1 --2- 172.21.15.47:0/597833353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f904bb0 0x56040f923ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.307+0000 7f05dbe00640 1 -- 172.21.15.47:0/597833353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ffa40 msgr2=0x56040f918d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:39.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.307+0000 7f05dbe00640 1 --2- 172.21.15.47:0/597833353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ffa40 0x56040f918d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.307+0000 7f05dbe00640 1 -- 172.21.15.47:0/597833353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56040f5380f0 con 0x56040f8ff050 2024-03-21T11:26:39.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.307+0000 7f05e1200640 1 --2- 172.21.15.47:0/597833353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f904bb0 0x56040f923ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:39.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.307+0000 7f05d3e00640 1 --2- 172.21.15.47:0/597833353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ffa40 0x56040f918d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:39.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.307+0000 7f05dbe00640 1 --2- 172.21.15.47:0/597833353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ff050 0x56040f9004b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0116f0 tx=0x7f05cc0103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb416b4d1ad917e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:39.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.308+0000 7f05d0c00640 1 -- 172.21.15.47:0/597833353 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc004250 con 0x56040f8ff050 2024-03-21T11:26:39.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.308+0000 7f05d0c00640 1 -- 172.21.15.47:0/597833353 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05cc01ce40 con 0x56040f8ff050 2024-03-21T11:26:39.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.308+0000 7f05d0c00640 1 -- 172.21.15.47:0/597833353 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc011b50 con 0x56040f8ff050 2024-03-21T11:26:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.308+0000 7f05e5f8b740 1 -- 172.21.15.47:0/597833353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ff050 msgr2=0x56040f9004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.308+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/597833353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ff050 0x56040f9004b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0116f0 tx=0x7f05cc0103d0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.308+0000 7f05e5f8b740 1 -- 172.21.15.47:0/597833353 shutdown_connections 2024-03-21T11:26:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.308+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/597833353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f904bb0 0x56040f923ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.308+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/597833353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ffa40 0x56040f918d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.308+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/597833353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ff050 0x56040f9004b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.309+0000 7f05e5f8b740 1 -- 172.21.15.47:0/597833353 >> 172.21.15.47:0/597833353 conn(0x56040f8c7200 msgr2=0x56040f8f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.309+0000 7f05e5f8b740 1 -- 172.21.15.47:0/597833353 shutdown_connections 2024-03-21T11:26:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.309+0000 7f05e5f8b740 1 -- 172.21.15.47:0/597833353 wait complete. 2024-03-21T11:26:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.309+0000 7f05e5f8b740 1 -- start start 2024-03-21T11:26:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.309+0000 7f05e5f8b740 1 Processor -- start 2024-03-21T11:26:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05e5f8b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ff050 0x56040f916340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05e5f8b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ffa40 0x56040f916880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05e5f8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f904bb0 0x56040f917890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05e5f8b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc1a9110 con 0x56040f8ff050 2024-03-21T11:26:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05e5f8b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc1a8ab0 con 0x56040f8ffa40 2024-03-21T11:26:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05e5f8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc1a5fd0 con 0x56040f904bb0 2024-03-21T11:26:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05e5f8b740 10 client.0 _ll_register_callbacks cb 0x7f05c4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ffa40 0x56040f916880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05dbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ff050 0x56040f916340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05e1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f904bb0 0x56040f917890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ffa40 0x56040f916880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38654/0 (socket says 172.21.15.47:38654) 2024-03-21T11:26:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05d3e00640 1 -- 172.21.15.47:0/1051478539 learned_addr learned my addr 172.21.15.47:0/1051478539 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05e1200640 1 -- 172.21.15.47:0/1051478539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ffa40 msgr2=0x56040f916880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05e1200640 1 --2- 172.21.15.47:0/1051478539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ffa40 0x56040f916880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05e1200640 1 -- 172.21.15.47:0/1051478539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ff050 msgr2=0x56040f916340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05e1200640 1 --2- 172.21.15.47:0/1051478539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ff050 0x56040f916340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05e1200640 1 -- 172.21.15.47:0/1051478539 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56040f917e80 con 0x56040f904bb0 2024-03-21T11:26:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.310+0000 7f05d3e00640 1 --2- 172.21.15.47:0/1051478539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ffa40 0x56040f916880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.311+0000 7f05e1200640 1 --2- 172.21.15.47:0/1051478539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f904bb0 0x56040f917890 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f05d4037040 tx=0x7f05d40095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac30f95ed5c900d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.311+0000 7f05ca000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.311+0000 7f05ca000640 1 -- 172.21.15.47:0/1051478539 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4053030 con 0x56040f904bb0 2024-03-21T11:26:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.311+0000 7f05e5f8b740 1 -- 172.21.15.47:0/1051478539 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56040f86bae0 con 0x56040f904bb0 2024-03-21T11:26:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.311+0000 7f05ca000640 1 -- 172.21.15.47:0/1051478539 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f05d4034ac0 con 0x56040f904bb0 2024-03-21T11:26:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.311+0000 7f05ca000640 1 -- 172.21.15.47:0/1051478539 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05d4039040 con 0x56040f904bb0 2024-03-21T11:26:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.311+0000 7f05e5f8b740 10 client.26563 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.311+0000 7f05e5f8b740 1 -- 172.21.15.47:0/1051478539 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56040f978aa0 con 0x56040f904bb0 2024-03-21T11:26:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.312+0000 7f05ca000640 1 -- 172.21.15.47:0/1051478539 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56040f86bae0 con 0x56040f904bb0 2024-03-21T11:26:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.312+0000 7f05ca000640 1 -- 172.21.15.47:0/1051478539 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f05d404f440 con 0x56040f904bb0 2024-03-21T11:26:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.312+0000 7f05e5f8b740 10 client.26563 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.312+0000 7f05e5f8b740 10 client.26563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.312+0000 7f05e5f8b740 20 client.26563 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.312+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/1051478539 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56040f9793e0 0x56040f97b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.312+0000 7f05e5f8b740 4 client.26563 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:26:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.312+0000 7f05e5f8b740 1 -- 172.21.15.47:0/1051478539 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56040f7e93a0 con 0x56040f9793e0 2024-03-21T11:26:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.312+0000 7f05dbe00640 1 --2- 172.21.15.47:0/1051478539 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56040f9793e0 0x56040f97b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.313+0000 7f05dbe00640 1 --2- 172.21.15.47:0/1051478539 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56040f9793e0 0x56040f97b800 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=685c237e50cd6fb6 server_cookie=f92b9272528cbe52 in_seq=0 out_seq=0 2024-03-21T11:26:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.313+0000 7f05ca000640 10 client.26563 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.314+0000 7f05ca000640 1 -- 172.21.15.47:0/1051478539 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56040f7e93a0 con 0x56040f9793e0 2024-03-21T11:26:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.314+0000 7f05ca000640 10 client.26563 handle_command_reply: tid=0 2024-03-21T11:26:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.314+0000 7f05e5f8b740 1 client.26563 shutdown 2024-03-21T11:26:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.315+0000 7f05e5f8b740 1 -- 172.21.15.47:0/1051478539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f904bb0 msgr2=0x56040f917890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.315+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/1051478539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f904bb0 0x56040f917890 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f05d4037040 tx=0x7f05d40095c0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.315+0000 7f05e5f8b740 1 -- 172.21.15.47:0/1051478539 shutdown_connections 2024-03-21T11:26:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.315+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/1051478539 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56040f9793e0 0x56040f97b800 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.315+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/1051478539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56040f904bb0 0x56040f917890 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.315+0000 7f05ca000640 0 client.26563 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.315+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/1051478539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56040f8ffa40 0x56040f916880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.315+0000 7f05e5f8b740 1 --2- 172.21.15.47:0/1051478539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56040f8ff050 0x56040f916340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.315+0000 7f05e5f8b740 1 -- 172.21.15.47:0/1051478539 >> 172.21.15.47:0/1051478539 conn(0x56040f8c7200 msgr2=0x56040f8f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.315+0000 7f05e5f8b740 1 -- 172.21.15.47:0/1051478539 shutdown_connections 2024-03-21T11:26:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.315+0000 7f05e5f8b740 1 -- 172.21.15.47:0/1051478539 wait complete. 2024-03-21T11:26:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.315+0000 7f05e5f8b740 20 client.26563 trim_cache size 0 max 16384 2024-03-21T11:26:39.321 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:26:39.321 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:26:39.321 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:26:39.321 INFO:teuthology.orchestra.run.smithi047.stdout: "e1e8e7e0-091c-43de-bd0d-a44f57540792": { 2024-03-21T11:26:39.321 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:26:39.322 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "e1e8e7e0-091c-43de-bd0d-a44f57540792", 2024-03-21T11:26:39.322 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:26:39.322 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:26:39.322 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:26:39.322 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:26:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.316+0000 7f05e1c00640 1 -- 172.21.15.47:0/3754161811 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05c00a3020 msgr2=0x7f05c00a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.316+0000 7f05e1c00640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05c00a3020 0x7f05c00a54e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f05d402c640 tx=0x7f05d4032590 comp rx=0 tx=0).stop 2024-03-21T11:26:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.316+0000 7f05e1c00640 1 -- 172.21.15.47:0/3754161811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05dc102790 msgr2=0x7f05dc199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.316+0000 7f05e1c00640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05dc102790 0x7f05dc199fd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f05c400e9f0 tx=0x7f05c400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.316+0000 7f05e1c00640 1 -- 172.21.15.47:0/3754161811 shutdown_connections 2024-03-21T11:26:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.316+0000 7f05e1c00640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05c00a3020 0x7f05c00a54e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.316+0000 7f05e1c00640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05dc103150 0x7f05dc1a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.316+0000 7f05e1c00640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05dc102790 0x7f05dc199fd0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.316+0000 7f05e1c00640 1 --2- 172.21.15.47:0/3754161811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05dc1015a0 0x7f05dc199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.316+0000 7f05e1c00640 1 -- 172.21.15.47:0/3754161811 >> 172.21.15.47:0/3754161811 conn(0x7f05dc0fcb10 msgr2=0x7f05dc0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.316+0000 7f05e1c00640 1 -- 172.21.15.47:0/3754161811 shutdown_connections 2024-03-21T11:26:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:26:39.316+0000 7f05e1c00640 1 -- 172.21.15.47:0/3754161811 wait complete. 2024-03-21T11:26:39.334 INFO:tasks.cephfs.filesystem:scrub status for tag:e1e8e7e0-091c-43de-bd0d-a44f57540792 - {'path': '/', 'tag': 'e1e8e7e0-091c-43de-bd0d-a44f57540792', 'options': 'recursive,force'} 2024-03-21T11:26:39.497 INFO:tasks.workunit.client.0.smithi047.stdout: 1 136681 9.07 MB/sec warmup 83 sec latency 23.959 ms 2024-03-21T11:26:39.743 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:39 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:26:39.743 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:39 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:26:39.743 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:39 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:26:39.743 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:39 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:26:39.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:39 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:26:39.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:39 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:26:39.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:39 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:26:39.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:39 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:26:39.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:39 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:26:39.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:39 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:26:39.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:39 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:26:39.897 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:39 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:26:40.497 INFO:tasks.workunit.client.0.smithi047.stdout: 1 138613 9.12 MB/sec warmup 84 sec latency 33.633 ms 2024-03-21T11:26:40.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:40 smithi102 ceph-mon[36748]: pgmap v500: 129 pgs: 129 active+clean; 36 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 345 op/s 2024-03-21T11:26:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:40 smithi047 ceph-mon[31351]: pgmap v500: 129 pgs: 129 active+clean; 36 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 345 op/s 2024-03-21T11:26:40.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:40 smithi100 ceph-mon[36245]: pgmap v500: 129 pgs: 129 active+clean; 36 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 345 op/s 2024-03-21T11:26:41.497 INFO:tasks.workunit.client.0.smithi047.stdout: 1 139625 9.05 MB/sec warmup 85 sec latency 10.836 ms 2024-03-21T11:26:42.497 INFO:tasks.workunit.client.0.smithi047.stdout: 1 141757 9.08 MB/sec warmup 86 sec latency 10.249 ms 2024-03-21T11:26:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.553+0000 7f7c5b200640 1 -- 172.21.15.47:0/2957518014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c5c075680 msgr2=0x7f7c5c073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.553+0000 7f7c5b200640 1 --2- 172.21.15.47:0/2957518014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c5c075680 0x7f7c5c073ad0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009900 tx=0x7f7c4c02bcd0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.553+0000 7f7c5b200640 1 -- 172.21.15.47:0/2957518014 shutdown_connections 2024-03-21T11:26:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.553+0000 7f7c5b200640 1 --2- 172.21.15.47:0/2957518014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c5c109380 0x7f7c5c10b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.553+0000 7f7c5b200640 1 --2- 172.21.15.47:0/2957518014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c5c0740a0 0x7f7c5c074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.553+0000 7f7c5b200640 1 --2- 172.21.15.47:0/2957518014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c5c075680 0x7f7c5c073ad0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.553+0000 7f7c5b200640 1 -- 172.21.15.47:0/2957518014 >> 172.21.15.47:0/2957518014 conn(0x7f7c5c0fca80 msgr2=0x7f7c5c0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.553+0000 7f7c5b200640 1 -- 172.21.15.47:0/2957518014 shutdown_connections 2024-03-21T11:26:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.553+0000 7f7c5b200640 1 -- 172.21.15.47:0/2957518014 wait complete. 2024-03-21T11:26:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.554+0000 7f7c5b200640 1 Processor -- start 2024-03-21T11:26:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.554+0000 7f7c5b200640 1 -- start start 2024-03-21T11:26:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c5b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c5c0740a0 0x7f7c5c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c5b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c5c075680 0x7f7c5c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c5b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c5c109380 0x7f7c5c1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c5b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c10f390 con 0x7f7c5c109380 2024-03-21T11:26:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c5b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c10f210 con 0x7f7c5c0740a0 2024-03-21T11:26:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c5b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c10f510 con 0x7f7c5c075680 2024-03-21T11:26:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c5c0740a0 0x7f7c5c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c59400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c5c075680 0x7f7c5c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c5c0740a0 0x7f7c5c199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38674/0 (socket says 172.21.15.47:38674) 2024-03-21T11:26:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c59e00640 1 -- 172.21.15.47:0/3862938523 learned_addr learned my addr 172.21.15.47:0/3862938523 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c5a800640 1 --2- 172.21.15.47:0/3862938523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c5c109380 0x7f7c5c1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c59e00640 1 -- 172.21.15.47:0/3862938523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c5c075680 msgr2=0x7f7c5c199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c59e00640 1 --2- 172.21.15.47:0/3862938523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c5c075680 0x7f7c5c199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c59e00640 1 -- 172.21.15.47:0/3862938523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c5c109380 msgr2=0x7f7c5c1a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c59e00640 1 --2- 172.21.15.47:0/3862938523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c5c109380 0x7f7c5c1a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.555+0000 7f7c59e00640 1 -- 172.21.15.47:0/3862938523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c1a1640 con 0x7f7c5c0740a0 2024-03-21T11:26:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.556+0000 7f7c59e00640 1 --2- 172.21.15.47:0/3862938523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c5c0740a0 0x7f7c5c199a20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c000c00 tx=0x7f7c4c02c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.556+0000 7f7c52a00640 1 -- 172.21.15.47:0/3862938523 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c043020 con 0x7f7c5c0740a0 2024-03-21T11:26:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.556+0000 7f7c52a00640 1 -- 172.21.15.47:0/3862938523 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c4c0054e0 con 0x7f7c5c0740a0 2024-03-21T11:26:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.556+0000 7f7c52a00640 1 -- 172.21.15.47:0/3862938523 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c003c20 con 0x7f7c5c0740a0 2024-03-21T11:26:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.556+0000 7f7c5b200640 1 -- 172.21.15.47:0/3862938523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5c1a18d0 con 0x7f7c5c0740a0 2024-03-21T11:26:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.556+0000 7f7c5b200640 1 -- 172.21.15.47:0/3862938523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5c1a1dd0 con 0x7f7c5c0740a0 2024-03-21T11:26:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.559+0000 7f7c52a00640 1 -- 172.21.15.47:0/3862938523 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7c4c046020 con 0x7f7c5c0740a0 2024-03-21T11:26:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.560+0000 7f7c63e1c740 1 Processor -- start 2024-03-21T11:26:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.560+0000 7f7c63e1c740 1 -- start start 2024-03-21T11:26:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.560+0000 7f7c63e1c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f2463f0 0x55835f246810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.560+0000 7f7c63e1c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f246de0 0x55835f24b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.560+0000 7f7c63e1c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f24bf50 0x55835f24e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.560+0000 7f7c63e1c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55835f2093b0 con 0x55835f246de0 2024-03-21T11:26:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.560+0000 7f7c63e1c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55835f1ab830 con 0x55835f24bf50 2024-03-21T11:26:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.560+0000 7f7c63e1c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55835f1545d0 con 0x55835f2463f0 2024-03-21T11:26:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.561+0000 7f7c5a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f2463f0 0x55835f246810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.561+0000 7f7c5a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f2463f0 0x55835f246810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47382/0 (socket says 172.21.15.47:47382) 2024-03-21T11:26:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.561+0000 7f7c5a800640 1 -- 172.21.15.47:0/194438231 learned_addr learned my addr 172.21.15.47:0/194438231 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.561+0000 7f7c59e00640 1 --2- 172.21.15.47:0/194438231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f246de0 0x55835f24b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.561+0000 7f7c59400640 1 --2- 172.21.15.47:0/194438231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f24bf50 0x55835f24e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.561+0000 7f7c5a800640 1 -- 172.21.15.47:0/194438231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f24bf50 msgr2=0x55835f24e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.561+0000 7f7c5a800640 1 --2- 172.21.15.47:0/194438231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f24bf50 0x55835f24e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.561+0000 7f7c5a800640 1 -- 172.21.15.47:0/194438231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f246de0 msgr2=0x55835f24b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.561+0000 7f7c5a800640 1 --2- 172.21.15.47:0/194438231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f246de0 0x55835f24b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.561+0000 7f7c5a800640 1 -- 172.21.15.47:0/194438231 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55835f16c2c0 con 0x55835f2463f0 2024-03-21T11:26:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.561+0000 7f7c59400640 1 --2- 172.21.15.47:0/194438231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f24bf50 0x55835f24e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.561+0000 7f7c59e00640 1 --2- 172.21.15.47:0/194438231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f246de0 0x55835f24b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.561+0000 7f7c52a00640 1 --2- 172.21.15.47:0/3862938523 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7c400a2f90 0x7f7c400a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.562+0000 7f7c52a00640 1 -- 172.21.15.47:0/3862938523 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7c4c0f1870 con 0x7f7c5c0740a0 2024-03-21T11:26:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.562+0000 7f7c5a800640 1 --2- 172.21.15.47:0/194438231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f2463f0 0x55835f246810 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009fc0 tx=0x7f7c54010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff1a18611808c87e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.562+0000 7f7c59e00640 1 --2- 172.21.15.47:0/3862938523 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7c400a2f90 0x7f7c400a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.562+0000 7f7c47400640 1 -- 172.21.15.47:0/194438231 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c54003b70 con 0x55835f2463f0 2024-03-21T11:26:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.562+0000 7f7c47400640 1 -- 172.21.15.47:0/194438231 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c54003d10 con 0x55835f2463f0 2024-03-21T11:26:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.562+0000 7f7c47400640 1 -- 172.21.15.47:0/194438231 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c54011aa0 con 0x55835f2463f0 2024-03-21T11:26:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.563+0000 7f7c63e1c740 1 -- 172.21.15.47:0/194438231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f2463f0 msgr2=0x55835f246810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.563+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/194438231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f2463f0 0x55835f246810 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009fc0 tx=0x7f7c54010860 comp rx=0 tx=0).stop 2024-03-21T11:26:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.563+0000 7f7c59e00640 1 --2- 172.21.15.47:0/3862938523 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7c400a2f90 0x7f7c400a5450 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0bd8b0 tx=0x7f7c4c02bcd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.563+0000 7f7c63e1c740 1 -- 172.21.15.47:0/194438231 shutdown_connections 2024-03-21T11:26:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.563+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/194438231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f24bf50 0x55835f24e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.563+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/194438231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f246de0 0x55835f24b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.563+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/194438231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f2463f0 0x55835f246810 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.563+0000 7f7c63e1c740 1 -- 172.21.15.47:0/194438231 >> 172.21.15.47:0/194438231 conn(0x55835f241960 msgr2=0x55835f241d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.563+0000 7f7c63e1c740 1 -- 172.21.15.47:0/194438231 shutdown_connections 2024-03-21T11:26:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.563+0000 7f7c63e1c740 1 -- 172.21.15.47:0/194438231 wait complete. 2024-03-21T11:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.563+0000 7f7c63e1c740 1 -- start start 2024-03-21T11:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.563+0000 7f7c63e1c740 1 Processor -- start 2024-03-21T11:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.564+0000 7f7c63e1c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f2463f0 0x55835f250600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.564+0000 7f7c63e1c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f250b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.564+0000 7f7c63e1c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f24bf50 0x55835f257b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.564+0000 7f7c63e1c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55835f167a20 con 0x55835f24bf50 2024-03-21T11:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.564+0000 7f7c63e1c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55835f1a78d0 con 0x55835f2463f0 2024-03-21T11:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.564+0000 7f7c63e1c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55835f24ef00 con 0x55835f246de0 2024-03-21T11:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.564+0000 7f7c63e1c740 10 client.0 _ll_register_callbacks cb 0x55835f1a41c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.564+0000 7f7c59400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f250b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.564+0000 7f7c59400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f250b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47390/0 (socket says 172.21.15.47:47390) 2024-03-21T11:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.564+0000 7f7c59400640 1 -- 172.21.15.47:0/915218825 learned_addr learned my addr 172.21.15.47:0/915218825 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.564+0000 7f7c5a800640 1 --2- 172.21.15.47:0/915218825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f2463f0 0x55835f250600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.564+0000 7f7c5a800640 1 --2- 172.21.15.47:0/915218825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f24bf50 0x55835f257b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.565+0000 7f7c59400640 1 -- 172.21.15.47:0/915218825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f2463f0 msgr2=0x55835f250600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.565+0000 7f7c59400640 1 --2- 172.21.15.47:0/915218825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f2463f0 0x55835f250600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.565+0000 7f7c59400640 1 -- 172.21.15.47:0/915218825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f24bf50 msgr2=0x55835f257b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.565+0000 7f7c59400640 1 --2- 172.21.15.47:0/915218825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f24bf50 0x55835f257b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.565+0000 7f7c59400640 1 -- 172.21.15.47:0/915218825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55835eec0210 con 0x55835f246de0 2024-03-21T11:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.565+0000 7f7c5a800640 1 --2- 172.21.15.47:0/915218825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f24bf50 0x55835f257b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.565+0000 7f7c59400640 1 --2- 172.21.15.47:0/915218825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f250b40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7c48012700 tx=0x7f7c48012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b014d4fc8f05884b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.565+0000 7f7c3be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.565+0000 7f7c3be00640 1 -- 172.21.15.47:0/915218825 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48004050 con 0x55835f246de0 2024-03-21T11:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.565+0000 7f7c3be00640 1 -- 172.21.15.47:0/915218825 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7c480041f0 con 0x55835f246de0 2024-03-21T11:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.565+0000 7f7c63e1c740 1 -- 172.21.15.47:0/915218825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c54003d10 con 0x55835f246de0 2024-03-21T11:26:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.566+0000 7f7c3be00640 1 -- 172.21.15.47:0/915218825 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c48025050 con 0x55835f246de0 2024-03-21T11:26:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.566+0000 7f7c63e1c740 10 client.26575 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.566+0000 7f7c63e1c740 1 -- 172.21.15.47:0/915218825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55835ef8c620 con 0x55835f246de0 2024-03-21T11:26:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.566+0000 7f7c3be00640 1 -- 172.21.15.47:0/915218825 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c54003d10 con 0x55835f246de0 2024-03-21T11:26:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.566+0000 7f7c3be00640 1 -- 172.21.15.47:0/915218825 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7c480039c0 con 0x55835f246de0 2024-03-21T11:26:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.566+0000 7f7c63e1c740 10 client.26575 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.566+0000 7f7c63e1c740 10 client.26575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.566+0000 7f7c63e1c740 20 client.26575 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.566+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/915218825 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835f25c590 0x55835f25e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.566+0000 7f7c63e1c740 4 client.26575 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.566+0000 7f7c63e1c740 1 -- 172.21.15.47:0/915218825 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55835f1b8d70 con 0x55835f25c590 2024-03-21T11:26:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.567+0000 7f7c5a800640 1 --2- 172.21.15.47:0/915218825 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835f25c590 0x55835f25e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.567+0000 7f7c5a800640 1 --2- 172.21.15.47:0/915218825 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835f25c590 0x55835f25e9b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1370250e81617748 server_cookie=2c66025688afa158 in_seq=0 out_seq=0 2024-03-21T11:26:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.567+0000 7f7c3be00640 10 client.26575 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.569+0000 7f7c3be00640 1 -- 172.21.15.47:0/915218825 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55835f1b8d70 con 0x55835f25c590 2024-03-21T11:26:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.569+0000 7f7c3be00640 10 client.26575 handle_command_reply: tid=0 2024-03-21T11:26:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.569+0000 7f7c63e1c740 1 client.26575 shutdown 2024-03-21T11:26:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.569+0000 7f7c63e1c740 1 -- 172.21.15.47:0/915218825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 msgr2=0x55835f250b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.569+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/915218825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f250b40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7c48012700 tx=0x7f7c48012bc0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.569+0000 7f7c63e1c740 1 -- 172.21.15.47:0/915218825 shutdown_connections 2024-03-21T11:26:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.569+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/915218825 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835f25c590 0x55835f25e9b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.569+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/915218825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f24bf50 0x55835f257b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.569+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/915218825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f250b40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.569+0000 7f7c3be00640 0 client.26575 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.569+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/915218825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f2463f0 0x55835f250600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.569+0000 7f7c63e1c740 1 -- 172.21.15.47:0/915218825 >> 172.21.15.47:0/915218825 conn(0x55835f241960 msgr2=0x55835f240b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.569+0000 7f7c63e1c740 1 -- 172.21.15.47:0/915218825 shutdown_connections 2024-03-21T11:26:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.569+0000 7f7c63e1c740 1 -- 172.21.15.47:0/915218825 wait complete. 2024-03-21T11:26:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.570+0000 7f7c63e1c740 20 client.26575 trim_cache size 0 max 16384 2024-03-21T11:26:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.586+0000 7f7c63e1c740 1 Processor -- start 2024-03-21T11:26:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.586+0000 7f7c63e1c740 1 -- start start 2024-03-21T11:26:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.587+0000 7f7c63e1c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f2463f0 0x55835f2452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.587+0000 7f7c63e1c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f257b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.587+0000 7f7c63e1c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f24bf50 0x55835f263230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.587+0000 7f7c63e1c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1ac190 con 0x55835f24bf50 2024-03-21T11:26:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.587+0000 7f7c63e1c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1adb10 con 0x55835f2463f0 2024-03-21T11:26:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.587+0000 7f7c63e1c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1ac7f0 con 0x55835f246de0 2024-03-21T11:26:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.587+0000 7f7c5a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f2463f0 0x55835f2452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.587+0000 7f7c5a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f2463f0 0x55835f2452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38704/0 (socket says 172.21.15.47:38704) 2024-03-21T11:26:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.587+0000 7f7c59400640 1 --2- 172.21.15.47:0/1356939283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f257b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.587+0000 7f7c5a800640 1 -- 172.21.15.47:0/1356939283 learned_addr learned my addr 172.21.15.47:0/1356939283 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.587+0000 7f7c5a800640 1 --2- 172.21.15.47:0/1356939283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f24bf50 0x55835f263230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.588+0000 7f7c5a800640 1 -- 172.21.15.47:0/1356939283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 msgr2=0x55835f257b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.588+0000 7f7c5a800640 1 --2- 172.21.15.47:0/1356939283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f257b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.588+0000 7f7c5a800640 1 -- 172.21.15.47:0/1356939283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f24bf50 msgr2=0x55835f263230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.588+0000 7f7c5a800640 1 --2- 172.21.15.47:0/1356939283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f24bf50 0x55835f263230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.588+0000 7f7c5a800640 1 -- 172.21.15.47:0/1356939283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55835f25ed20 con 0x55835f2463f0 2024-03-21T11:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.588+0000 7f7c5a800640 1 --2- 172.21.15.47:0/1356939283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f2463f0 0x55835f2452d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7c54002230 tx=0x7f7c54011500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6554584d73befaa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.588+0000 7f7c46a00640 1 -- 172.21.15.47:0/1356939283 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c54012a10 con 0x55835f2463f0 2024-03-21T11:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.588+0000 7f7c46a00640 1 -- 172.21.15.47:0/1356939283 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c54012510 con 0x55835f2463f0 2024-03-21T11:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.589+0000 7f7c46a00640 1 -- 172.21.15.47:0/1356939283 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c54011930 con 0x55835f2463f0 2024-03-21T11:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.589+0000 7f7c63e1c740 1 -- 172.21.15.47:0/1356939283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f2463f0 msgr2=0x55835f2452d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.589+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/1356939283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f2463f0 0x55835f2452d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7c54002230 tx=0x7f7c54011500 comp rx=0 tx=0).stop 2024-03-21T11:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.589+0000 7f7c63e1c740 1 -- 172.21.15.47:0/1356939283 shutdown_connections 2024-03-21T11:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.589+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/1356939283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f24bf50 0x55835f263230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.589+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/1356939283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f257b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.589+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/1356939283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f2463f0 0x55835f2452d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.589+0000 7f7c63e1c740 1 -- 172.21.15.47:0/1356939283 >> 172.21.15.47:0/1356939283 conn(0x55835f241960 msgr2=0x55835f244bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.590+0000 7f7c63e1c740 1 -- 172.21.15.47:0/1356939283 shutdown_connections 2024-03-21T11:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.590+0000 7f7c63e1c740 1 -- 172.21.15.47:0/1356939283 wait complete. 2024-03-21T11:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.590+0000 7f7c63e1c740 1 -- start start 2024-03-21T11:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.590+0000 7f7c63e1c740 1 Processor -- start 2024-03-21T11:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c63e1c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f2463f0 0x55835f255b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c63e1c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f2560b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c63e1c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f24bf50 0x55835f257050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c63e1c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1ad4b0 con 0x55835f2463f0 2024-03-21T11:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c63e1c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1ace50 con 0x55835f24bf50 2024-03-21T11:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c63e1c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c1aa350 con 0x55835f246de0 2024-03-21T11:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c63e1c740 10 client.0 _ll_register_callbacks cb 0x7f7c48009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c5a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f2463f0 0x55835f255b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c59400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f2560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c59400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f2560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47404/0 (socket says 172.21.15.47:47404) 2024-03-21T11:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c59400640 1 -- 172.21.15.47:0/318796347 learned_addr learned my addr 172.21.15.47:0/318796347 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c5a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f2463f0 0x55835f255b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33190/0 (socket says 172.21.15.47:33190) 2024-03-21T11:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c5a800640 1 --2- 172.21.15.47:0/318796347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f24bf50 0x55835f257050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c59400640 1 -- 172.21.15.47:0/318796347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f24bf50 msgr2=0x55835f257050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c59400640 1 --2- 172.21.15.47:0/318796347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f24bf50 0x55835f257050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c59400640 1 -- 172.21.15.47:0/318796347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f2463f0 msgr2=0x55835f255b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c59400640 1 --2- 172.21.15.47:0/318796347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f2463f0 0x55835f255b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c59400640 1 -- 172.21.15.47:0/318796347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55835f257640 con 0x55835f246de0 2024-03-21T11:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.591+0000 7f7c5a800640 1 --2- 172.21.15.47:0/318796347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f24bf50 0x55835f257050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.592+0000 7f7c59400640 1 --2- 172.21.15.47:0/318796347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f2560b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55835f240040 tx=0x7f7c48003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baa4de49516aaa63 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.592+0000 7f7c3be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.592+0000 7f7c3be00640 1 -- 172.21.15.47:0/318796347 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48009e90 con 0x55835f246de0 2024-03-21T11:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.592+0000 7f7c63e1c740 1 -- 172.21.15.47:0/318796347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c54012510 con 0x55835f246de0 2024-03-21T11:26:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.592+0000 7f7c3be00640 1 -- 172.21.15.47:0/318796347 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7c48022070 con 0x55835f246de0 2024-03-21T11:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.592+0000 7f7c3be00640 1 -- 172.21.15.47:0/318796347 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c4803a050 con 0x55835f246de0 2024-03-21T11:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.592+0000 7f7c63e1c740 10 client.26581 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.592+0000 7f7c63e1c740 1 -- 172.21.15.47:0/318796347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55835f2b86e0 con 0x55835f246de0 2024-03-21T11:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.593+0000 7f7c3be00640 1 -- 172.21.15.47:0/318796347 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c54012510 con 0x55835f246de0 2024-03-21T11:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.593+0000 7f7c3be00640 1 -- 172.21.15.47:0/318796347 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7c4800fd20 con 0x55835f246de0 2024-03-21T11:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.593+0000 7f7c63e1c740 10 client.26581 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.593+0000 7f7c63e1c740 10 client.26581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.593+0000 7f7c63e1c740 20 client.26581 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.593+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/318796347 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835f25c590 0x55835f2bb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.593+0000 7f7c63e1c740 4 client.26581 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:26:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.593+0000 7f7c63e1c740 1 -- 172.21.15.47:0/318796347 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55835f2503f0 con 0x55835f25c590 2024-03-21T11:26:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.593+0000 7f7c5a800640 1 --2- 172.21.15.47:0/318796347 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835f25c590 0x55835f2bb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.594+0000 7f7c5a800640 1 --2- 172.21.15.47:0/318796347 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835f25c590 0x55835f2bb040 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f767df29ad87ce server_cookie=8a237db497980cea in_seq=0 out_seq=0 2024-03-21T11:26:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.594+0000 7f7c3be00640 10 client.26581 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:42.813 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:42 smithi047 ceph-mon[31351]: pgmap v501: 129 pgs: 129 active+clean; 34 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.9 MiB/s wr, 298 op/s 2024-03-21T11:26:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:42 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:26:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.813+0000 7f7c3be00640 1 -- 172.21.15.47:0/318796347 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55835f2503f0 con 0x55835f25c590 2024-03-21T11:26:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.813+0000 7f7c3be00640 10 client.26581 handle_command_reply: tid=0 2024-03-21T11:26:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.813+0000 7f7c63e1c740 1 client.26581 shutdown 2024-03-21T11:26:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.813+0000 7f7c63e1c740 1 -- 172.21.15.47:0/318796347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 msgr2=0x55835f2560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.813+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/318796347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f2560b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55835f240040 tx=0x7f7c48003820 comp rx=0 tx=0).stop 2024-03-21T11:26:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.813+0000 7f7c63e1c740 1 -- 172.21.15.47:0/318796347 shutdown_connections 2024-03-21T11:26:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.813+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/318796347 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835f25c590 0x55835f2bb040 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.813+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/318796347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835f24bf50 0x55835f257050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.813+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/318796347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835f246de0 0x55835f2560b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.813+0000 7f7c63e1c740 1 --2- 172.21.15.47:0/318796347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835f2463f0 0x55835f255b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.813+0000 7f7c3be00640 0 client.26581 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.813+0000 7f7c63e1c740 1 -- 172.21.15.47:0/318796347 >> 172.21.15.47:0/318796347 conn(0x55835f241960 msgr2=0x55835f245450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.814+0000 7f7c63e1c740 1 -- 172.21.15.47:0/318796347 shutdown_connections 2024-03-21T11:26:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.814+0000 7f7c63e1c740 1 -- 172.21.15.47:0/318796347 wait complete. 2024-03-21T11:26:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.814+0000 7f7c63e1c740 20 client.26581 trim_cache size 0 max 16384 2024-03-21T11:26:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:26:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:26:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:26:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:26:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.815+0000 7f7c5b200640 1 -- 172.21.15.47:0/3862938523 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7c400a2f90 msgr2=0x7f7c400a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.815+0000 7f7c5b200640 1 --2- 172.21.15.47:0/3862938523 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7c400a2f90 0x7f7c400a5450 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0bd8b0 tx=0x7f7c4c02bcd0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.815+0000 7f7c5b200640 1 -- 172.21.15.47:0/3862938523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c5c0740a0 msgr2=0x7f7c5c199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.815+0000 7f7c5b200640 1 --2- 172.21.15.47:0/3862938523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c5c0740a0 0x7f7c5c199a20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c000c00 tx=0x7f7c4c02c6f0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.815+0000 7f7c5b200640 1 -- 172.21.15.47:0/3862938523 shutdown_connections 2024-03-21T11:26:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.815+0000 7f7c5b200640 1 --2- 172.21.15.47:0/3862938523 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7c400a2f90 0x7f7c400a5450 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.815+0000 7f7c5b200640 1 --2- 172.21.15.47:0/3862938523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c5c109380 0x7f7c5c1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.815+0000 7f7c5b200640 1 --2- 172.21.15.47:0/3862938523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c5c075680 0x7f7c5c199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.815+0000 7f7c5b200640 1 --2- 172.21.15.47:0/3862938523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c5c0740a0 0x7f7c5c199a20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.815+0000 7f7c5b200640 1 -- 172.21.15.47:0/3862938523 >> 172.21.15.47:0/3862938523 conn(0x7f7c5c0fca80 msgr2=0x7f7c5c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.815+0000 7f7c5b200640 1 -- 172.21.15.47:0/3862938523 shutdown_connections 2024-03-21T11:26:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:42.815+0000 7f7c5b200640 1 -- 172.21.15.47:0/3862938523 wait complete. 2024-03-21T11:26:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:42 smithi100 ceph-mon[36245]: pgmap v501: 129 pgs: 129 active+clean; 34 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.9 MiB/s wr, 298 op/s 2024-03-21T11:26:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:42 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:26:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:42 smithi102 ceph-mon[36748]: pgmap v501: 129 pgs: 129 active+clean; 34 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.9 MiB/s wr, 298 op/s 2024-03-21T11:26:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:42 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:26:43.497 INFO:tasks.workunit.client.0.smithi047.stdout: 1 142632 9.04 MB/sec warmup 87 sec latency 114.420 ms 2024-03-21T11:26:44.497 INFO:tasks.workunit.client.0.smithi047.stdout: 1 144288 9.01 MB/sec warmup 88 sec latency 44.190 ms 2024-03-21T11:26:44.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:44 smithi047 ceph-mon[31351]: pgmap v502: 129 pgs: 129 active+clean; 37 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 357 op/s 2024-03-21T11:26:44.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:44 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:26:44.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:44 smithi100 ceph-mon[36245]: pgmap v502: 129 pgs: 129 active+clean; 37 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 357 op/s 2024-03-21T11:26:44.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:44 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:26:44.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:44 smithi102 ceph-mon[36748]: pgmap v502: 129 pgs: 129 active+clean; 37 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 357 op/s 2024-03-21T11:26:44.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:44 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:26:45.497 INFO:tasks.workunit.client.0.smithi047.stdout: 1 145837 9.04 MB/sec warmup 89 sec latency 12.924 ms 2024-03-21T11:26:45.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:45 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:26:45.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:45 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:26:45.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:45 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:26:45.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:45 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:26:45.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:45 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:26:45.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:45 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:26:45.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:45 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:26:45.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:45 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:26:45.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:45 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:26:46.498 INFO:tasks.workunit.client.0.smithi047.stdout: 1 147350 9.00 MB/sec warmup 90 sec latency 11.904 ms 2024-03-21T11:26:46.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:46 smithi047 ceph-mon[31351]: pgmap v503: 129 pgs: 129 active+clean; 33 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2024-03-21T11:26:46.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:46 smithi100 ceph-mon[36245]: pgmap v503: 129 pgs: 129 active+clean; 33 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2024-03-21T11:26:46.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:46 smithi102 ceph-mon[36748]: pgmap v503: 129 pgs: 129 active+clean; 33 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 273 op/s 2024-03-21T11:26:47.498 INFO:tasks.workunit.client.0.smithi047.stdout: 1 149364 9.06 MB/sec warmup 91 sec latency 10.261 ms 2024-03-21T11:26:48.498 INFO:tasks.workunit.client.0.smithi047.stdout: 1 150764 9.02 MB/sec warmup 92 sec latency 15.908 ms 2024-03-21T11:26:48.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:48 smithi047 ceph-mon[31351]: pgmap v504: 129 pgs: 129 active+clean; 33 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 295 op/s 2024-03-21T11:26:48.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:48 smithi100 ceph-mon[36245]: pgmap v504: 129 pgs: 129 active+clean; 33 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 295 op/s 2024-03-21T11:26:48.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:48 smithi102 ceph-mon[36748]: pgmap v504: 129 pgs: 129 active+clean; 33 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 295 op/s 2024-03-21T11:26:49.498 INFO:tasks.workunit.client.0.smithi047.stdout: 1 152771 9.06 MB/sec warmup 93 sec latency 48.515 ms 2024-03-21T11:26:50.498 INFO:tasks.workunit.client.0.smithi047.stdout: 1 153733 8.99 MB/sec warmup 94 sec latency 86.609 ms 2024-03-21T11:26:50.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:50 smithi047 ceph-mon[31351]: pgmap v505: 129 pgs: 129 active+clean; 36 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 357 op/s 2024-03-21T11:26:50.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:50 smithi100 ceph-mon[36245]: pgmap v505: 129 pgs: 129 active+clean; 36 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 357 op/s 2024-03-21T11:26:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:50 smithi102 ceph-mon[36748]: pgmap v505: 129 pgs: 129 active+clean; 36 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 357 op/s 2024-03-21T11:26:51.498 INFO:tasks.workunit.client.0.smithi047.stdout: 1 156097 9.06 MB/sec warmup 95 sec latency 12.684 ms 2024-03-21T11:26:52.498 INFO:tasks.workunit.client.0.smithi047.stdout: 1 157006 8.99 MB/sec warmup 96 sec latency 11.665 ms 2024-03-21T11:26:52.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:52 smithi047 ceph-mon[31351]: pgmap v506: 129 pgs: 129 active+clean; 36 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 320 op/s 2024-03-21T11:26:52.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:52 smithi100 ceph-mon[36245]: pgmap v506: 129 pgs: 129 active+clean; 36 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 320 op/s 2024-03-21T11:26:52.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:52 smithi102 ceph-mon[36748]: pgmap v506: 129 pgs: 129 active+clean; 36 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 320 op/s 2024-03-21T11:26:53.498 INFO:tasks.workunit.client.0.smithi047.stdout: 1 159065 9.02 MB/sec warmup 97 sec latency 21.060 ms 2024-03-21T11:26:54.498 INFO:tasks.workunit.client.0.smithi047.stdout: 1 160316 9.00 MB/sec warmup 98 sec latency 10.818 ms 2024-03-21T11:26:54.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:54 smithi047 ceph-mon[31351]: pgmap v507: 129 pgs: 129 active+clean; 39 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 376 op/s 2024-03-21T11:26:54.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:54 smithi100 ceph-mon[36245]: pgmap v507: 129 pgs: 129 active+clean; 39 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 376 op/s 2024-03-21T11:26:54.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:54 smithi102 ceph-mon[36748]: pgmap v507: 129 pgs: 129 active+clean; 39 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 376 op/s 2024-03-21T11:26:55.499 INFO:tasks.workunit.client.0.smithi047.stdout: 1 161935 8.97 MB/sec warmup 99 sec latency 81.193 ms 2024-03-21T11:26:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.968+0000 7f9856000640 1 -- 172.21.15.47:0/3095007155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501028f0 msgr2=0x7f9850102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.968+0000 7f9856000640 1 --2- 172.21.15.47:0/3095007155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501028f0 0x7f9850102d70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9838009900 tx=0x7f983802bc30 comp rx=0 tx=0).stop 2024-03-21T11:26:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.968+0000 7f9856000640 1 -- 172.21.15.47:0/3095007155 shutdown_connections 2024-03-21T11:26:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.968+0000 7f9856000640 1 --2- 172.21.15.47:0/3095007155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98501032b0 0x7f9850109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.968+0000 7f9856000640 1 --2- 172.21.15.47:0/3095007155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501028f0 0x7f9850102d70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.968+0000 7f9856000640 1 --2- 172.21.15.47:0/3095007155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9850101700 0x7f9850101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.968+0000 7f9856000640 1 -- 172.21.15.47:0/3095007155 >> 172.21.15.47:0/3095007155 conn(0x7f98500fcc70 msgr2=0x7f98500ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:55.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.968+0000 7f9856000640 1 -- 172.21.15.47:0/3095007155 shutdown_connections 2024-03-21T11:26:55.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.968+0000 7f9856000640 1 -- 172.21.15.47:0/3095007155 wait complete. 2024-03-21T11:26:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.969+0000 7f9856000640 1 Processor -- start 2024-03-21T11:26:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.969+0000 7f9856000640 1 -- start start 2024-03-21T11:26:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f9856000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9850101700 0x7f9850199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f9856000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98501028f0 0x7f985019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f9856000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501032b0 0x7f98501a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f9856000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985010d380 con 0x7f98501028f0 2024-03-21T11:26:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f9856000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985010d200 con 0x7f9850101700 2024-03-21T11:26:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f9856000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985010d500 con 0x7f98501032b0 2024-03-21T11:26:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f984f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9850101700 0x7f9850199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f984fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501032b0 0x7f98501a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f984f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9850101700 0x7f9850199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38808/0 (socket says 172.21.15.47:38808) 2024-03-21T11:26:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f984f400640 1 -- 172.21.15.47:0/2085708345 learned_addr learned my addr 172.21.15.47:0/2085708345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f9847e00640 1 --2- 172.21.15.47:0/2085708345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98501028f0 0x7f985019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f984fe00640 1 -- 172.21.15.47:0/2085708345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9850101700 msgr2=0x7f9850199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f984fe00640 1 --2- 172.21.15.47:0/2085708345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9850101700 0x7f9850199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f984fe00640 1 -- 172.21.15.47:0/2085708345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98501028f0 msgr2=0x7f985019a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.970+0000 7f984fe00640 1 --2- 172.21.15.47:0/2085708345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98501028f0 0x7f985019a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.971+0000 7f984fe00640 1 -- 172.21.15.47:0/2085708345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98501a17d0 con 0x7f98501032b0 2024-03-21T11:26:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.971+0000 7f9847e00640 1 --2- 172.21.15.47:0/2085708345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98501028f0 0x7f985019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.971+0000 7f984fe00640 1 --2- 172.21.15.47:0/2085708345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501032b0 0x7f98501a1160 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9848009fd0 tx=0x7f984800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.971+0000 7f984cc00640 1 -- 172.21.15.47:0/2085708345 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984800ce20 con 0x7f98501032b0 2024-03-21T11:26:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.971+0000 7f984cc00640 1 -- 172.21.15.47:0/2085708345 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9848005020 con 0x7f98501032b0 2024-03-21T11:26:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.971+0000 7f984cc00640 1 -- 172.21.15.47:0/2085708345 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848010720 con 0x7f98501032b0 2024-03-21T11:26:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.971+0000 7f9856000640 1 -- 172.21.15.47:0/2085708345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98501a1ac0 con 0x7f98501032b0 2024-03-21T11:26:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.971+0000 7f9856000640 1 -- 172.21.15.47:0/2085708345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98501a1f90 con 0x7f98501032b0 2024-03-21T11:26:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.974+0000 7f9858dcf740 1 Processor -- start 2024-03-21T11:26:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.975+0000 7f984cc00640 1 -- 172.21.15.47:0/2085708345 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9848005540 con 0x7f98501032b0 2024-03-21T11:26:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.975+0000 7f9858dcf740 1 -- start start 2024-03-21T11:26:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.975+0000 7f9858dcf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x55877403e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.975+0000 7f9858dcf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403ebd0 0x5587740436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.975+0000 7f9858dcf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558774043d40 0x5587740461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.975+0000 7f9858dcf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587740013b0 con 0x558774043d40 2024-03-21T11:26:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.975+0000 7f9858dcf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558773fa3830 con 0x55877403ebd0 2024-03-21T11:26:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.975+0000 7f9858dcf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558773f4c5d0 con 0x55877403e1e0 2024-03-21T11:26:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.975+0000 7f9847e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x55877403e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.976+0000 7f984f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403ebd0 0x5587740436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.976+0000 7f9847e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x55877403e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53028/0 (socket says 172.21.15.47:53028) 2024-03-21T11:26:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.976+0000 7f984f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403ebd0 0x5587740436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38824/0 (socket says 172.21.15.47:38824) 2024-03-21T11:26:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.976+0000 7f9847e00640 1 -- 172.21.15.47:0/2162293029 learned_addr learned my addr 172.21.15.47:0/2162293029 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.976+0000 7f9847e00640 1 --2- 172.21.15.47:0/2162293029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558774043d40 0x5587740461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.976+0000 7f984f400640 1 -- 172.21.15.47:0/2162293029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 msgr2=0x55877403e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.976+0000 7f984f400640 1 --2- 172.21.15.47:0/2162293029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x55877403e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.976+0000 7f984f400640 1 -- 172.21.15.47:0/2162293029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558774043d40 msgr2=0x5587740461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.976+0000 7f984f400640 1 --2- 172.21.15.47:0/2162293029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558774043d40 0x5587740461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.976+0000 7f984f400640 1 -- 172.21.15.47:0/2162293029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558773f642c0 con 0x55877403ebd0 2024-03-21T11:26:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.976+0000 7f9847e00640 1 --2- 172.21.15.47:0/2162293029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558774043d40 0x5587740461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.976+0000 7f984f400640 1 --2- 172.21.15.47:0/2162293029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403ebd0 0x5587740436d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f98400058c0 tx=0x7f9840011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e174c22ceb209fc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.976+0000 7f984cc00640 1 --2- 172.21.15.47:0/2085708345 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98340a3040 0x7f98340a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.977+0000 7f983fe00640 1 -- 172.21.15.47:0/2162293029 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840004220 con 0x55877403ebd0 2024-03-21T11:26:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.977+0000 7f984cc00640 1 -- 172.21.15.47:0/2085708345 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f98480cfd80 con 0x7f98501032b0 2024-03-21T11:26:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.977+0000 7f983fe00640 1 -- 172.21.15.47:0/2162293029 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98400043c0 con 0x55877403ebd0 2024-03-21T11:26:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.977+0000 7f983fe00640 1 -- 172.21.15.47:0/2162293029 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840004db0 con 0x55877403ebd0 2024-03-21T11:26:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.977+0000 7f9847e00640 1 --2- 172.21.15.47:0/2085708345 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98340a3040 0x7f98340a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.978+0000 7f9858dcf740 1 -- 172.21.15.47:0/2162293029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403ebd0 msgr2=0x5587740436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.978+0000 7f9847e00640 1 --2- 172.21.15.47:0/2085708345 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98340a3040 0x7f98340a5500 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9838005ec0 tx=0x7f983802bc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.978+0000 7f9858dcf740 1 --2- 172.21.15.47:0/2162293029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403ebd0 0x5587740436d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f98400058c0 tx=0x7f9840011810 comp rx=0 tx=0).stop 2024-03-21T11:26:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.978+0000 7f9858dcf740 1 -- 172.21.15.47:0/2162293029 shutdown_connections 2024-03-21T11:26:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.978+0000 7f9858dcf740 1 --2- 172.21.15.47:0/2162293029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558774043d40 0x5587740461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.978+0000 7f9858dcf740 1 --2- 172.21.15.47:0/2162293029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403ebd0 0x5587740436d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.978+0000 7f9858dcf740 1 --2- 172.21.15.47:0/2162293029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x55877403e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.978+0000 7f9858dcf740 1 -- 172.21.15.47:0/2162293029 >> 172.21.15.47:0/2162293029 conn(0x558774039750 msgr2=0x558774039b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.978+0000 7f9858dcf740 1 -- 172.21.15.47:0/2162293029 shutdown_connections 2024-03-21T11:26:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.978+0000 7f9858dcf740 1 -- 172.21.15.47:0/2162293029 wait complete. 2024-03-21T11:26:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.978+0000 7f9858dcf740 1 -- start start 2024-03-21T11:26:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.979+0000 7f9858dcf740 1 Processor -- start 2024-03-21T11:26:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.979+0000 7f9858dcf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403e1e0 0x5587740483f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.979+0000 7f9858dcf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55877403ebd0 0x558774048930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.979+0000 7f9858dcf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558774043d40 0x55877404f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.979+0000 7f9858dcf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558773f5fa20 con 0x55877403ebd0 2024-03-21T11:26:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.979+0000 7f9858dcf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558773f9f8d0 con 0x55877403e1e0 2024-03-21T11:26:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.979+0000 7f9858dcf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558774046cf0 con 0x558774043d40 2024-03-21T11:26:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.979+0000 7f9858dcf740 10 client.0 _ll_register_callbacks cb 0x558773f9c1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.979+0000 7f984f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403e1e0 0x5587740483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.979+0000 7f9847e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55877403ebd0 0x558774048930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.979+0000 7f984f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403e1e0 0x5587740483f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38840/0 (socket says 172.21.15.47:38840) 2024-03-21T11:26:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.979+0000 7f984f400640 1 -- 172.21.15.47:0/3316601557 learned_addr learned my addr 172.21.15.47:0/3316601557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.979+0000 7f984fe00640 1 --2- 172.21.15.47:0/3316601557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558774043d40 0x55877404f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.980+0000 7f984f400640 1 -- 172.21.15.47:0/3316601557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558774043d40 msgr2=0x55877404f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.980+0000 7f984f400640 1 --2- 172.21.15.47:0/3316601557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558774043d40 0x55877404f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.980+0000 7f984f400640 1 -- 172.21.15.47:0/3316601557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55877403ebd0 msgr2=0x558774048930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.980+0000 7f984f400640 1 --2- 172.21.15.47:0/3316601557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55877403ebd0 0x558774048930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.980+0000 7f984f400640 1 -- 172.21.15.47:0/3316601557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558773cb8210 con 0x55877403e1e0 2024-03-21T11:26:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.980+0000 7f984fe00640 1 --2- 172.21.15.47:0/3316601557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558774043d40 0x55877404f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.980+0000 7f984f400640 1 --2- 172.21.15.47:0/3316601557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403e1e0 0x5587740483f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f984001b040 tx=0x7f9840005240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=871fe85b63221173 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.980+0000 7f983cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:26:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.980+0000 7f983cc00640 1 -- 172.21.15.47:0/3316601557 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840027030 con 0x55877403e1e0 2024-03-21T11:26:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.980+0000 7f983cc00640 1 -- 172.21.15.47:0/3316601557 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f98400046c0 con 0x55877403e1e0 2024-03-21T11:26:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.980+0000 7f9858dcf740 1 -- 172.21.15.47:0/3316601557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98400043c0 con 0x55877403e1e0 2024-03-21T11:26:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.981+0000 7f983cc00640 1 -- 172.21.15.47:0/3316601557 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9840034050 con 0x55877403e1e0 2024-03-21T11:26:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.981+0000 7f9858dcf740 10 client.26573 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.981+0000 7f9858dcf740 1 -- 172.21.15.47:0/3316601557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558773d84620 con 0x55877403e1e0 2024-03-21T11:26:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.981+0000 7f983cc00640 1 -- 172.21.15.47:0/3316601557 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98400043c0 con 0x55877403e1e0 2024-03-21T11:26:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.981+0000 7f983cc00640 1 -- 172.21.15.47:0/3316601557 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9840019ce0 con 0x55877403e1e0 2024-03-21T11:26:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.981+0000 7f9858dcf740 10 client.26573 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.981+0000 7f9858dcf740 10 client.26573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.981+0000 7f9858dcf740 20 client.26573 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.981+0000 7f9858dcf740 1 --2- 172.21.15.47:0/3316601557 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558774053f10 0x558774056320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.981+0000 7f9858dcf740 4 client.26573 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:26:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.981+0000 7f9858dcf740 1 -- 172.21.15.47:0/3316601557 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558773fb0d70 con 0x558774053f10 2024-03-21T11:26:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.982+0000 7f9847e00640 1 --2- 172.21.15.47:0/3316601557 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558774053f10 0x558774056320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.982+0000 7f9847e00640 1 --2- 172.21.15.47:0/3316601557 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558774053f10 0x558774056320 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f637ca6038b3b017 server_cookie=beb4e1d5afdc9aab in_seq=0 out_seq=0 2024-03-21T11:26:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.982+0000 7f983cc00640 10 client.26573 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f983cc00640 1 -- 172.21.15.47:0/3316601557 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558773fb0d70 con 0x558774053f10 2024-03-21T11:26:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f983cc00640 10 client.26573 handle_command_reply: tid=0 2024-03-21T11:26:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f9858dcf740 1 client.26573 shutdown 2024-03-21T11:26:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f9858dcf740 1 -- 172.21.15.47:0/3316601557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403e1e0 msgr2=0x5587740483f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f9858dcf740 1 --2- 172.21.15.47:0/3316601557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403e1e0 0x5587740483f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f984001b040 tx=0x7f9840005240 comp rx=0 tx=0).stop 2024-03-21T11:26:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f9858dcf740 1 -- 172.21.15.47:0/3316601557 shutdown_connections 2024-03-21T11:26:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f9858dcf740 1 --2- 172.21.15.47:0/3316601557 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558774053f10 0x558774056320 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f9858dcf740 1 --2- 172.21.15.47:0/3316601557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558774043d40 0x55877404f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f9858dcf740 1 --2- 172.21.15.47:0/3316601557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55877403ebd0 0x558774048930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f9858dcf740 1 --2- 172.21.15.47:0/3316601557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403e1e0 0x5587740483f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f983cc00640 0 client.26573 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f9858dcf740 1 -- 172.21.15.47:0/3316601557 >> 172.21.15.47:0/3316601557 conn(0x558774039750 msgr2=0x558774038950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f9858dcf740 1 -- 172.21.15.47:0/3316601557 shutdown_connections 2024-03-21T11:26:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f9858dcf740 1 -- 172.21.15.47:0/3316601557 wait complete. 2024-03-21T11:26:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:55.984+0000 7f9858dcf740 20 client.26573 trim_cache size 0 max 16384 2024-03-21T11:26:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.002+0000 7f9858dcf740 1 Processor -- start 2024-03-21T11:26:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.002+0000 7f9858dcf740 1 -- start start 2024-03-21T11:26:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.002+0000 7f9858dcf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x5587740380c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f9858dcf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403ebd0 0x558774038600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f9858dcf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558774043d40 0x558774038b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f9858dcf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98501ac2b0 con 0x558774043d40 2024-03-21T11:26:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f9858dcf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98501adc30 con 0x55877403ebd0 2024-03-21T11:26:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f9858dcf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98501ac910 con 0x55877403e1e0 2024-03-21T11:26:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f9847e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403ebd0 0x558774038600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f9847e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403ebd0 0x558774038600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38842/0 (socket says 172.21.15.47:38842) 2024-03-21T11:26:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f9847e00640 1 -- 172.21.15.47:0/275480338 learned_addr learned my addr 172.21.15.47:0/275480338 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f984f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x5587740380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f984fe00640 1 --2- 172.21.15.47:0/275480338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558774043d40 0x558774038b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f9847e00640 1 -- 172.21.15.47:0/275480338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 msgr2=0x5587740380c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f9847e00640 1 --2- 172.21.15.47:0/275480338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x5587740380c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f9847e00640 1 -- 172.21.15.47:0/275480338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558774043d40 msgr2=0x558774038b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f9847e00640 1 --2- 172.21.15.47:0/275480338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558774043d40 0x558774038b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f9847e00640 1 -- 172.21.15.47:0/275480338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55877403a4c0 con 0x55877403ebd0 2024-03-21T11:26:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.003+0000 7f984fe00640 1 --2- 172.21.15.47:0/275480338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558774043d40 0x558774038b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:26:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.004+0000 7f9847e00640 1 --2- 172.21.15.47:0/275480338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403ebd0 0x558774038600 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9838009230 tx=0x7f983802e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a3e5f6a40dbd354 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.004+0000 7f983f400640 1 -- 172.21.15.47:0/275480338 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98380548b0 con 0x55877403ebd0 2024-03-21T11:26:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.004+0000 7f983f400640 1 -- 172.21.15.47:0/275480338 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9838059070 con 0x55877403ebd0 2024-03-21T11:26:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.004+0000 7f983f400640 1 -- 172.21.15.47:0/275480338 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838042440 con 0x55877403ebd0 2024-03-21T11:26:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.005+0000 7f9858dcf740 1 -- 172.21.15.47:0/275480338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403ebd0 msgr2=0x558774038600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.005+0000 7f9858dcf740 1 --2- 172.21.15.47:0/275480338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403ebd0 0x558774038600 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9838009230 tx=0x7f983802e040 comp rx=0 tx=0).stop 2024-03-21T11:26:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.005+0000 7f9858dcf740 1 -- 172.21.15.47:0/275480338 shutdown_connections 2024-03-21T11:26:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.005+0000 7f9858dcf740 1 --2- 172.21.15.47:0/275480338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558774043d40 0x558774038b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.005+0000 7f9858dcf740 1 --2- 172.21.15.47:0/275480338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55877403ebd0 0x558774038600 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.005+0000 7f9858dcf740 1 --2- 172.21.15.47:0/275480338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x5587740380c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.005+0000 7f9858dcf740 1 -- 172.21.15.47:0/275480338 >> 172.21.15.47:0/275480338 conn(0x558774039750 msgr2=0x55877403fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.005+0000 7f9858dcf740 1 -- 172.21.15.47:0/275480338 shutdown_connections 2024-03-21T11:26:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.005+0000 7f9858dcf740 1 -- 172.21.15.47:0/275480338 wait complete. 2024-03-21T11:26:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.005+0000 7f9858dcf740 1 -- start start 2024-03-21T11:26:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.005+0000 7f9858dcf740 1 Processor -- start 2024-03-21T11:26:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.006+0000 7f9858dcf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x55877404c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.006+0000 7f9858dcf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55877403ebd0 0x55877404c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.006+0000 7f9858dcf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558774043d40 0x55877404d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.006+0000 7f9858dcf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98501ad5d0 con 0x55877403ebd0 2024-03-21T11:26:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.006+0000 7f9858dcf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98501acf70 con 0x558774043d40 2024-03-21T11:26:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.006+0000 7f9858dcf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98501aa450 con 0x55877403e1e0 2024-03-21T11:26:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.006+0000 7f9858dcf740 10 client.0 _ll_register_callbacks cb 0x7f9840005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:26:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.006+0000 7f984fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558774043d40 0x55877404d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.006+0000 7f984f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x55877404c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.006+0000 7f984fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558774043d40 0x55877404d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38848/0 (socket says 172.21.15.47:38848) 2024-03-21T11:26:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.006+0000 7f984f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x55877404c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53062/0 (socket says 172.21.15.47:53062) 2024-03-21T11:26:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.006+0000 7f984fe00640 1 -- 172.21.15.47:0/3811482925 learned_addr learned my addr 172.21.15.47:0/3811482925 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:26:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.006+0000 7f9847e00640 1 --2- 172.21.15.47:0/3811482925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55877403ebd0 0x55877404c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.007+0000 7f984f400640 1 -- 172.21.15.47:0/3811482925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558774043d40 msgr2=0x55877404d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.007+0000 7f984f400640 1 --2- 172.21.15.47:0/3811482925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558774043d40 0x55877404d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.007+0000 7f984f400640 1 -- 172.21.15.47:0/3811482925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55877403ebd0 msgr2=0x55877404c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:26:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.007+0000 7f984f400640 1 --2- 172.21.15.47:0/3811482925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55877403ebd0 0x55877404c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.007+0000 7f984f400640 1 -- 172.21.15.47:0/3811482925 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55877404dec0 con 0x55877403e1e0 2024-03-21T11:26:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.007+0000 7f984fe00640 1 --2- 172.21.15.47:0/3811482925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558774043d40 0x55877404d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:26:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.007+0000 7f984f400640 1 --2- 172.21.15.47:0/3811482925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x55877404c380 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9840004b00 tx=0x7f9840005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e580d2cc431f1084 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:26:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.007+0000 7f983cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:26:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.007+0000 7f983cc00640 1 -- 172.21.15.47:0/3811482925 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840027040 con 0x55877403e1e0 2024-03-21T11:26:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.007+0000 7f983cc00640 1 -- 172.21.15.47:0/3811482925 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f98400052b0 con 0x55877403e1e0 2024-03-21T11:26:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.007+0000 7f9858dcf740 1 -- 172.21.15.47:0/3811482925 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9838059070 con 0x55877403e1e0 2024-03-21T11:26:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.008+0000 7f983cc00640 1 -- 172.21.15.47:0/3811482925 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f984001d040 con 0x55877403e1e0 2024-03-21T11:26:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.008+0000 7f9858dcf740 10 client.26599 fetch_fsmap learned FSMap version 28 2024-03-21T11:26:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.008+0000 7f9858dcf740 1 -- 172.21.15.47:0/3811482925 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55877404f1d0 con 0x55877403e1e0 2024-03-21T11:26:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.008+0000 7f983cc00640 1 -- 172.21.15.47:0/3811482925 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9838059070 con 0x55877403e1e0 2024-03-21T11:26:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.008+0000 7f983cc00640 1 -- 172.21.15.47:0/3811482925 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9840023a90 con 0x55877403e1e0 2024-03-21T11:26:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.008+0000 7f9858dcf740 10 client.26599 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:26:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.008+0000 7f9858dcf740 10 client.26599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:26:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.008+0000 7f9858dcf740 20 client.26599 populate_metadata read hostname 'smithi047' 2024-03-21T11:26:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.008+0000 7f9858dcf740 1 --2- 172.21.15.47:0/3811482925 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5587740afc10 0x5587740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:26:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.008+0000 7f9858dcf740 4 client.26599 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:26:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.008+0000 7f9858dcf740 1 -- 172.21.15.47:0/3811482925 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558773fab0c0 con 0x5587740afc10 2024-03-21T11:26:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.008+0000 7f9847e00640 1 --2- 172.21.15.47:0/3811482925 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5587740afc10 0x5587740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:26:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.009+0000 7f9847e00640 1 --2- 172.21.15.47:0/3811482925 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5587740afc10 0x5587740b2000 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af11463272139bd server_cookie=32013b4e95b506c8 in_seq=0 out_seq=0 2024-03-21T11:26:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.009+0000 7f983cc00640 10 client.26599 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.190+0000 7f983cc00640 1 -- 172.21.15.47:0/3811482925 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558773fab0c0 con 0x5587740afc10 2024-03-21T11:26:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.190+0000 7f983cc00640 10 client.26599 handle_command_reply: tid=0 2024-03-21T11:26:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.190+0000 7f9858dcf740 1 client.26599 shutdown 2024-03-21T11:26:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.191+0000 7f9858dcf740 1 -- 172.21.15.47:0/3811482925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 msgr2=0x55877404c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.191+0000 7f9858dcf740 1 --2- 172.21.15.47:0/3811482925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x55877404c380 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9840004b00 tx=0x7f9840005d30 comp rx=0 tx=0).stop 2024-03-21T11:26:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.191+0000 7f9858dcf740 1 -- 172.21.15.47:0/3811482925 shutdown_connections 2024-03-21T11:26:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.191+0000 7f9858dcf740 1 --2- 172.21.15.47:0/3811482925 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5587740afc10 0x5587740b2000 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.191+0000 7f9858dcf740 1 --2- 172.21.15.47:0/3811482925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558774043d40 0x55877404d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.191+0000 7f9858dcf740 1 --2- 172.21.15.47:0/3811482925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55877403ebd0 0x55877404c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.191+0000 7f983cc00640 0 client.26599 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:26:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.191+0000 7f9858dcf740 1 --2- 172.21.15.47:0/3811482925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55877403e1e0 0x55877404c380 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.191+0000 7f9858dcf740 1 -- 172.21.15.47:0/3811482925 >> 172.21.15.47:0/3811482925 conn(0x558774039750 msgr2=0x55877403c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.191+0000 7f9858dcf740 1 -- 172.21.15.47:0/3811482925 shutdown_connections 2024-03-21T11:26:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.191+0000 7f9858dcf740 1 -- 172.21.15.47:0/3811482925 wait complete. 2024-03-21T11:26:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.191+0000 7f9858dcf740 20 client.26599 trim_cache size 0 max 16384 2024-03-21T11:26:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:26:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:26:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:26:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:26:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.192+0000 7f9856000640 1 -- 172.21.15.47:0/2085708345 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98340a3040 msgr2=0x7f98340a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.192+0000 7f9856000640 1 --2- 172.21.15.47:0/2085708345 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98340a3040 0x7f98340a5500 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9838005ec0 tx=0x7f983802bc30 comp rx=0 tx=0).stop 2024-03-21T11:26:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.192+0000 7f9856000640 1 -- 172.21.15.47:0/2085708345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501032b0 msgr2=0x7f98501a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:26:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.192+0000 7f9856000640 1 --2- 172.21.15.47:0/2085708345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501032b0 0x7f98501a1160 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9848009fd0 tx=0x7f984800ef10 comp rx=0 tx=0).stop 2024-03-21T11:26:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.192+0000 7f9856000640 1 -- 172.21.15.47:0/2085708345 shutdown_connections 2024-03-21T11:26:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.192+0000 7f9856000640 1 --2- 172.21.15.47:0/2085708345 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98340a3040 0x7f98340a5500 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.192+0000 7f9856000640 1 --2- 172.21.15.47:0/2085708345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98501032b0 0x7f98501a1160 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.192+0000 7f9856000640 1 --2- 172.21.15.47:0/2085708345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98501028f0 0x7f985019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.192+0000 7f9856000640 1 --2- 172.21.15.47:0/2085708345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9850101700 0x7f9850199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:26:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.192+0000 7f9856000640 1 -- 172.21.15.47:0/2085708345 >> 172.21.15.47:0/2085708345 conn(0x7f98500fcc70 msgr2=0x7f98500738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:26:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.192+0000 7f9856000640 1 -- 172.21.15.47:0/2085708345 shutdown_connections 2024-03-21T11:26:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:26:56.193+0000 7f9856000640 1 -- 172.21.15.47:0/2085708345 wait complete. 2024-03-21T11:26:56.499 INFO:tasks.workunit.client.0.smithi047.stdout: 1 163395 9.00 MB/sec warmup 100 sec latency 76.877 ms 2024-03-21T11:26:56.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:56 smithi047 ceph-mon[31351]: pgmap v508: 129 pgs: 129 active+clean; 39 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 291 op/s 2024-03-21T11:26:56.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:56 smithi100 ceph-mon[36245]: pgmap v508: 129 pgs: 129 active+clean; 39 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 291 op/s 2024-03-21T11:26:56.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:56 smithi102 ceph-mon[36748]: pgmap v508: 129 pgs: 129 active+clean; 39 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 291 op/s 2024-03-21T11:26:57.499 INFO:tasks.workunit.client.0.smithi047.stdout: 1 164163 8.92 MB/sec warmup 101 sec latency 12.907 ms 2024-03-21T11:26:58.499 INFO:tasks.workunit.client.0.smithi047.stdout: 1 166191 8.95 MB/sec warmup 102 sec latency 12.660 ms 2024-03-21T11:26:58.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:26:58 smithi047 ceph-mon[31351]: pgmap v509: 129 pgs: 129 active+clean; 38 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 313 op/s 2024-03-21T11:26:58.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:26:58 smithi100 ceph-mon[36245]: pgmap v509: 129 pgs: 129 active+clean; 38 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 313 op/s 2024-03-21T11:26:58.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:26:58 smithi102 ceph-mon[36748]: pgmap v509: 129 pgs: 129 active+clean; 38 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 313 op/s 2024-03-21T11:26:59.499 INFO:tasks.workunit.client.0.smithi047.stdout: 1 167343 8.93 MB/sec warmup 103 sec latency 36.770 ms 2024-03-21T11:27:00.499 INFO:tasks.workunit.client.0.smithi047.stdout: 1 169031 8.91 MB/sec warmup 104 sec latency 32.444 ms 2024-03-21T11:27:00.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:00 smithi047 ceph-mon[31351]: pgmap v510: 129 pgs: 129 active+clean; 33 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 340 op/s 2024-03-21T11:27:00.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:00 smithi100 ceph-mon[36245]: pgmap v510: 129 pgs: 129 active+clean; 33 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 340 op/s 2024-03-21T11:27:00.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:00 smithi102 ceph-mon[36748]: pgmap v510: 129 pgs: 129 active+clean; 33 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 340 op/s 2024-03-21T11:27:01.499 INFO:tasks.workunit.client.0.smithi047.stdout: 1 170648 8.94 MB/sec warmup 105 sec latency 12.310 ms 2024-03-21T11:27:02.500 INFO:tasks.workunit.client.0.smithi047.stdout: 1 172144 8.91 MB/sec warmup 106 sec latency 19.524 ms 2024-03-21T11:27:02.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:02 smithi047 ceph-mon[31351]: pgmap v511: 129 pgs: 129 active+clean; 33 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 292 op/s 2024-03-21T11:27:02.895 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:02 smithi100 ceph-mon[36245]: pgmap v511: 129 pgs: 129 active+clean; 33 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 292 op/s 2024-03-21T11:27:02.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:02 smithi102 ceph-mon[36748]: pgmap v511: 129 pgs: 129 active+clean; 33 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 292 op/s 2024-03-21T11:27:03.500 INFO:tasks.workunit.client.0.smithi047.stdout: 1 174171 8.95 MB/sec warmup 107 sec latency 10.894 ms 2024-03-21T11:27:04.500 INFO:tasks.workunit.client.0.smithi047.stdout: 1 175648 8.92 MB/sec warmup 108 sec latency 14.840 ms 2024-03-21T11:27:04.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:04 smithi047 ceph-mon[31351]: pgmap v512: 129 pgs: 129 active+clean; 33 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 321 op/s 2024-03-21T11:27:04.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:04 smithi100 ceph-mon[36245]: pgmap v512: 129 pgs: 129 active+clean; 33 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 321 op/s 2024-03-21T11:27:04.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:04 smithi102 ceph-mon[36748]: pgmap v512: 129 pgs: 129 active+clean; 33 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 321 op/s 2024-03-21T11:27:05.500 INFO:tasks.workunit.client.0.smithi047.stdout: 1 177607 8.96 MB/sec warmup 109 sec latency 45.775 ms 2024-03-21T11:27:06.500 INFO:tasks.workunit.client.0.smithi047.stdout: 1 178853 8.93 MB/sec warmup 110 sec latency 9.490 ms 2024-03-21T11:27:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:06 smithi100 ceph-mon[36245]: pgmap v513: 129 pgs: 129 active+clean; 34 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-03-21T11:27:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:06 smithi102 ceph-mon[36748]: pgmap v513: 129 pgs: 129 active+clean; 34 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-03-21T11:27:07.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:06 smithi047 ceph-mon[31351]: pgmap v513: 129 pgs: 129 active+clean; 34 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-03-21T11:27:07.500 INFO:tasks.workunit.client.0.smithi047.stdout: 1 181139 8.97 MB/sec warmup 111 sec latency 8.978 ms 2024-03-21T11:27:08.501 INFO:tasks.workunit.client.0.smithi047.stdout: 1 182337 8.94 MB/sec warmup 112 sec latency 10.746 ms 2024-03-21T11:27:08.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:08 smithi100 ceph-mon[36245]: pgmap v514: 129 pgs: 129 active+clean; 37 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-03-21T11:27:08.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:08 smithi102 ceph-mon[36748]: pgmap v514: 129 pgs: 129 active+clean; 37 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-03-21T11:27:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:08 smithi047 ceph-mon[31351]: pgmap v514: 129 pgs: 129 active+clean; 37 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-03-21T11:27:09.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:27:09.335 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:27:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.347+0000 7ff18b400640 1 -- 172.21.15.47:0/657392287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff184104af0 msgr2=0x7ff184104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.347+0000 7ff18b400640 1 --2- 172.21.15.47:0/657392287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff184104af0 0x7ff184104f70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff170009900 tx=0x7ff17002bc30 comp rx=0 tx=0).stop 2024-03-21T11:27:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.347+0000 7ff18b400640 1 -- 172.21.15.47:0/657392287 shutdown_connections 2024-03-21T11:27:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.347+0000 7ff18b400640 1 --2- 172.21.15.47:0/657392287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1841054b0 0x7ff18410bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.347+0000 7ff18b400640 1 --2- 172.21.15.47:0/657392287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff184104af0 0x7ff184104f70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.347+0000 7ff18b400640 1 --2- 172.21.15.47:0/657392287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff184103900 0x7ff184103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.347+0000 7ff18b400640 1 -- 172.21.15.47:0/657392287 >> 172.21.15.47:0/657392287 conn(0x7ff1840fee90 msgr2=0x7ff1841012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:09.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.347+0000 7ff18b400640 1 -- 172.21.15.47:0/657392287 shutdown_connections 2024-03-21T11:27:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.347+0000 7ff18b400640 1 -- 172.21.15.47:0/657392287 wait complete. 2024-03-21T11:27:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.348+0000 7ff18b400640 1 Processor -- start 2024-03-21T11:27:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.348+0000 7ff18b400640 1 -- start start 2024-03-21T11:27:09.350 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.349+0000 7ff18b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff184103900 0x7ff184199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.349+0000 7ff18b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff184104af0 0x7ff18419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.349+0000 7ff18b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1841054b0 0x7ff1841a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.349+0000 7ff18b400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18410f5d0 con 0x7ff184103900 2024-03-21T11:27:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.349+0000 7ff18b400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18410f450 con 0x7ff1841054b0 2024-03-21T11:27:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.349+0000 7ff18b400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18410f750 con 0x7ff184104af0 2024-03-21T11:27:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.349+0000 7ff183e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff184104af0 0x7ff18419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.349+0000 7ff183e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff184104af0 0x7ff18419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43874/0 (socket says 172.21.15.47:43874) 2024-03-21T11:27:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.349+0000 7ff183e00640 1 -- 172.21.15.47:0/1817099205 learned_addr learned my addr 172.21.15.47:0/1817099205 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.349+0000 7ff189400640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1841054b0 0x7ff1841a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.349+0000 7ff188a00640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff184103900 0x7ff184199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.350+0000 7ff183e00640 1 -- 172.21.15.47:0/1817099205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1841054b0 msgr2=0x7ff1841a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.350+0000 7ff183e00640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1841054b0 0x7ff1841a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.350+0000 7ff183e00640 1 -- 172.21.15.47:0/1817099205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff184103900 msgr2=0x7ff184199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.350+0000 7ff183e00640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff184103900 0x7ff184199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.350+0000 7ff183e00640 1 -- 172.21.15.47:0/1817099205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1841a1750 con 0x7ff184104af0 2024-03-21T11:27:09.352 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.350+0000 7ff188a00640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff184103900 0x7ff184199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.350+0000 7ff183e00640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff184104af0 0x7ff18419a070 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff170000c00 tx=0x7ff17002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.350+0000 7ff181600640 1 -- 172.21.15.47:0/1817099205 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170043020 con 0x7ff184104af0 2024-03-21T11:27:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.350+0000 7ff181600640 1 -- 172.21.15.47:0/1817099205 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1700054e0 con 0x7ff184104af0 2024-03-21T11:27:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.351+0000 7ff18b400640 1 -- 172.21.15.47:0/1817099205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1841a19e0 con 0x7ff184104af0 2024-03-21T11:27:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.351+0000 7ff181600640 1 -- 172.21.15.47:0/1817099205 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170003c40 con 0x7ff184104af0 2024-03-21T11:27:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.351+0000 7ff18b400640 1 -- 172.21.15.47:0/1817099205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1841a1f80 con 0x7ff184104af0 2024-03-21T11:27:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.354+0000 7ff18e033740 1 Processor -- start 2024-03-21T11:27:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.355+0000 7ff18e033740 1 -- start start 2024-03-21T11:27:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.355+0000 7ff18e033740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e93061e0 0x5619e9306600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.355+0000 7ff18e033740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e9306bd0 0x5619e930b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.355+0000 7ff18e033740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e930bd40 0x5619e930e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.355+0000 7ff18e033740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e92c93b0 con 0x5619e93061e0 2024-03-21T11:27:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.355+0000 7ff18e033740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e926b830 con 0x5619e930bd40 2024-03-21T11:27:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.355+0000 7ff18e033740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e92145d0 con 0x5619e9306bd0 2024-03-21T11:27:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.355+0000 7ff188a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e9306bd0 0x5619e930b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.355+0000 7ff188a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e9306bd0 0x5619e930b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43888/0 (socket says 172.21.15.47:43888) 2024-03-21T11:27:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.355+0000 7ff188a00640 1 -- 172.21.15.47:0/49643012 learned_addr learned my addr 172.21.15.47:0/49643012 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.355+0000 7ff181600640 1 -- 172.21.15.47:0/1817099205 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff170046020 con 0x7ff184104af0 2024-03-21T11:27:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.355+0000 7ff189400640 1 --2- 172.21.15.47:0/49643012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e93061e0 0x5619e9306600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.356+0000 7ff188a00640 1 -- 172.21.15.47:0/49643012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e930bd40 msgr2=0x5619e930e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.356+0000 7ff188a00640 1 --2- 172.21.15.47:0/49643012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e930bd40 0x5619e930e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.356+0000 7ff188a00640 1 -- 172.21.15.47:0/49643012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e93061e0 msgr2=0x5619e9306600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.356+0000 7ff188a00640 1 --2- 172.21.15.47:0/49643012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e93061e0 0x5619e9306600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.356+0000 7ff188a00640 1 -- 172.21.15.47:0/49643012 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e922c2c0 con 0x5619e9306bd0 2024-03-21T11:27:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.356+0000 7ff189400640 1 --2- 172.21.15.47:0/49643012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e93061e0 0x5619e9306600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.356+0000 7ff188a00640 1 --2- 172.21.15.47:0/49643012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e9306bd0 0x5619e930b6d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff178009cd0 tx=0x7ff1780117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e6dc9d3fa064155 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.356+0000 7ff175c00640 1 -- 172.21.15.47:0/49643012 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178012450 con 0x5619e9306bd0 2024-03-21T11:27:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.356+0000 7ff175c00640 1 -- 172.21.15.47:0/49643012 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff178012ad0 con 0x5619e9306bd0 2024-03-21T11:27:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.356+0000 7ff175c00640 1 -- 172.21.15.47:0/49643012 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17800fca0 con 0x5619e9306bd0 2024-03-21T11:27:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.357+0000 7ff18e033740 1 -- 172.21.15.47:0/49643012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e9306bd0 msgr2=0x5619e930b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.357+0000 7ff18e033740 1 --2- 172.21.15.47:0/49643012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e9306bd0 0x5619e930b6d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff178009cd0 tx=0x7ff1780117b0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.358+0000 7ff18e033740 1 -- 172.21.15.47:0/49643012 shutdown_connections 2024-03-21T11:27:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.358+0000 7ff18e033740 1 --2- 172.21.15.47:0/49643012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e930bd40 0x5619e930e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.358+0000 7ff18e033740 1 --2- 172.21.15.47:0/49643012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e9306bd0 0x5619e930b6d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.358+0000 7ff18e033740 1 --2- 172.21.15.47:0/49643012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e93061e0 0x5619e9306600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.358+0000 7ff18e033740 1 -- 172.21.15.47:0/49643012 >> 172.21.15.47:0/49643012 conn(0x5619e9301750 msgr2=0x5619e9301b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.358+0000 7ff181600640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff16c0a3040 0x7ff16c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.358+0000 7ff181600640 1 -- 172.21.15.47:0/1817099205 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff1700f1870 con 0x7ff184104af0 2024-03-21T11:27:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.358+0000 7ff18e033740 1 -- 172.21.15.47:0/49643012 shutdown_connections 2024-03-21T11:27:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.358+0000 7ff18e033740 1 -- 172.21.15.47:0/49643012 wait complete. 2024-03-21T11:27:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.358+0000 7ff188a00640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff16c0a3040 0x7ff16c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.358+0000 7ff18e033740 1 -- start start 2024-03-21T11:27:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.359+0000 7ff18e033740 1 Processor -- start 2024-03-21T11:27:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.359+0000 7ff188a00640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff16c0a3040 0x7ff16c0a5500 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff178000c00 tx=0x7ff178014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff18e033740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e93061e0 0x5619e9310400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff18e033740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e9306bd0 0x5619e9310940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff18e033740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e930bd40 0x5619e9317980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff18e033740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e9227a20 con 0x5619e930bd40 2024-03-21T11:27:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff18e033740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e92678d0 con 0x5619e9306bd0 2024-03-21T11:27:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff18e033740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e930ecf0 con 0x5619e93061e0 2024-03-21T11:27:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff18e033740 10 client.0 _ll_register_callbacks cb 0x5619e92641c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff188a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e93061e0 0x5619e9310400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff183e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e9306bd0 0x5619e9310940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff188a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e93061e0 0x5619e9310400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43900/0 (socket says 172.21.15.47:43900) 2024-03-21T11:27:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff188a00640 1 -- 172.21.15.47:0/2693208026 learned_addr learned my addr 172.21.15.47:0/2693208026 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff183e00640 1 -- 172.21.15.47:0/2693208026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e93061e0 msgr2=0x5619e9310400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff183e00640 1 --2- 172.21.15.47:0/2693208026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e93061e0 0x5619e9310400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff183e00640 1 -- 172.21.15.47:0/2693208026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e930bd40 msgr2=0x5619e9317980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff183e00640 1 --2- 172.21.15.47:0/2693208026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e930bd40 0x5619e9317980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.360+0000 7ff183e00640 1 -- 172.21.15.47:0/2693208026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619e8f80210 con 0x5619e9306bd0 2024-03-21T11:27:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.361+0000 7ff183e00640 1 --2- 172.21.15.47:0/2693208026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e9306bd0 0x5619e9310940 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff1700eef80 tx=0x7ff1700bd5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=431133bbb0733217 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.361+0000 7ff16a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.361+0000 7ff16a000640 1 -- 172.21.15.47:0/2693208026 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170005690 con 0x5619e9306bd0 2024-03-21T11:27:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.361+0000 7ff16a000640 1 -- 172.21.15.47:0/2693208026 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff1700ef090 con 0x5619e9306bd0 2024-03-21T11:27:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.361+0000 7ff18e033740 1 -- 172.21.15.47:0/2693208026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff178012ad0 con 0x5619e9306bd0 2024-03-21T11:27:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.362+0000 7ff16a000640 1 -- 172.21.15.47:0/2693208026 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1700db050 con 0x5619e9306bd0 2024-03-21T11:27:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.362+0000 7ff18e033740 10 client.26588 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.362+0000 7ff18e033740 1 -- 172.21.15.47:0/2693208026 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5619e904c620 con 0x5619e9306bd0 2024-03-21T11:27:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.362+0000 7ff16a000640 1 -- 172.21.15.47:0/2693208026 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff178012ad0 con 0x5619e9306bd0 2024-03-21T11:27:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.362+0000 7ff16a000640 1 -- 172.21.15.47:0/2693208026 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff1700c6130 con 0x5619e9306bd0 2024-03-21T11:27:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.362+0000 7ff18e033740 10 client.26588 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.362+0000 7ff18e033740 10 client.26588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.362+0000 7ff18e033740 20 client.26588 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.362+0000 7ff18e033740 1 --2- 172.21.15.47:0/2693208026 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5619e9324a10 0x5619e9326e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.362+0000 7ff18e033740 4 client.26588 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.362+0000 7ff18e033740 1 -- 172.21.15.47:0/2693208026 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619e9278d70 con 0x5619e9324a10 2024-03-21T11:27:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.363+0000 7ff188a00640 1 --2- 172.21.15.47:0/2693208026 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5619e9324a10 0x5619e9326e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.363+0000 7ff188a00640 1 --2- 172.21.15.47:0/2693208026 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5619e9324a10 0x5619e9326e50 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b364b36de2efed00 server_cookie=d281de21ce39043f in_seq=0 out_seq=0 2024-03-21T11:27:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.363+0000 7ff16a000640 10 client.26588 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.364+0000 7ff16a000640 1 -- 172.21.15.47:0/2693208026 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5619e9278d70 con 0x5619e9324a10 2024-03-21T11:27:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.364+0000 7ff16a000640 10 client.26588 handle_command_reply: tid=0 2024-03-21T11:27:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.364+0000 7ff18e033740 1 client.26588 shutdown 2024-03-21T11:27:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.365+0000 7ff18e033740 1 -- 172.21.15.47:0/2693208026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e9306bd0 msgr2=0x5619e9310940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.365+0000 7ff18e033740 1 --2- 172.21.15.47:0/2693208026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e9306bd0 0x5619e9310940 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff1700eef80 tx=0x7ff1700bd5d0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.365+0000 7ff18e033740 1 -- 172.21.15.47:0/2693208026 shutdown_connections 2024-03-21T11:27:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.365+0000 7ff18e033740 1 --2- 172.21.15.47:0/2693208026 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5619e9324a10 0x5619e9326e50 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.365+0000 7ff18e033740 1 --2- 172.21.15.47:0/2693208026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e930bd40 0x5619e9317980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.365+0000 7ff18e033740 1 --2- 172.21.15.47:0/2693208026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e9306bd0 0x5619e9310940 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.365+0000 7ff16a000640 0 client.26588 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.365+0000 7ff18e033740 1 --2- 172.21.15.47:0/2693208026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e93061e0 0x5619e9310400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.365+0000 7ff18e033740 1 -- 172.21.15.47:0/2693208026 >> 172.21.15.47:0/2693208026 conn(0x5619e9301750 msgr2=0x5619e9300950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.365+0000 7ff18e033740 1 -- 172.21.15.47:0/2693208026 shutdown_connections 2024-03-21T11:27:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.365+0000 7ff18e033740 1 -- 172.21.15.47:0/2693208026 wait complete. 2024-03-21T11:27:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.365+0000 7ff18e033740 20 client.26588 trim_cache size 0 max 16384 2024-03-21T11:27:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.381+0000 7ff18e033740 1 Processor -- start 2024-03-21T11:27:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.381+0000 7ff18e033740 1 -- start start 2024-03-21T11:27:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.381+0000 7ff18e033740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e93061e0 0x5619e93000c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.381+0000 7ff18e033740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e9306bd0 0x5619e9317940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff18e033740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e930bd40 0x5619e9300600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff18e033740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1841a7df0 con 0x5619e93061e0 2024-03-21T11:27:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff18e033740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1841a9770 con 0x5619e930bd40 2024-03-21T11:27:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff18e033740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1841a8450 con 0x5619e9306bd0 2024-03-21T11:27:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff188a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e93061e0 0x5619e93000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff189400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e930bd40 0x5619e9300600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff188a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e93061e0 0x5619e93000c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38426/0 (socket says 172.21.15.47:38426) 2024-03-21T11:27:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff188a00640 1 -- 172.21.15.47:0/3581844261 learned_addr learned my addr 172.21.15.47:0/3581844261 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff183e00640 1 --2- 172.21.15.47:0/3581844261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e9306bd0 0x5619e9317940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff188a00640 1 -- 172.21.15.47:0/3581844261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e9306bd0 msgr2=0x5619e9317940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff188a00640 1 --2- 172.21.15.47:0/3581844261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e9306bd0 0x5619e9317940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff188a00640 1 -- 172.21.15.47:0/3581844261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e930bd40 msgr2=0x5619e9300600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff188a00640 1 --2- 172.21.15.47:0/3581844261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e930bd40 0x5619e9300600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff188a00640 1 -- 172.21.15.47:0/3581844261 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e930e1b0 con 0x5619e93061e0 2024-03-21T11:27:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.382+0000 7ff189400640 1 --2- 172.21.15.47:0/3581844261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e930bd40 0x5619e9300600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.383+0000 7ff188a00640 1 --2- 172.21.15.47:0/3581844261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e93061e0 0x5619e93000c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff178009cd0 tx=0x7ff178022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6ee7a5ee9aaed7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.383+0000 7ff175200640 1 -- 172.21.15.47:0/3581844261 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178009570 con 0x5619e93061e0 2024-03-21T11:27:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.383+0000 7ff175200640 1 -- 172.21.15.47:0/3581844261 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff17802d040 con 0x5619e93061e0 2024-03-21T11:27:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.383+0000 7ff175200640 1 -- 172.21.15.47:0/3581844261 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178005410 con 0x5619e93061e0 2024-03-21T11:27:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.384+0000 7ff18e033740 1 -- 172.21.15.47:0/3581844261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e93061e0 msgr2=0x5619e93000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.384+0000 7ff18e033740 1 --2- 172.21.15.47:0/3581844261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e93061e0 0x5619e93000c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff178009cd0 tx=0x7ff178022000 comp rx=0 tx=0).stop 2024-03-21T11:27:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.384+0000 7ff18e033740 1 -- 172.21.15.47:0/3581844261 shutdown_connections 2024-03-21T11:27:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.384+0000 7ff18e033740 1 --2- 172.21.15.47:0/3581844261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e930bd40 0x5619e9300600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.384+0000 7ff18e033740 1 --2- 172.21.15.47:0/3581844261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e9306bd0 0x5619e9317940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.384+0000 7ff18e033740 1 --2- 172.21.15.47:0/3581844261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e93061e0 0x5619e93000c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.384+0000 7ff18e033740 1 -- 172.21.15.47:0/3581844261 >> 172.21.15.47:0/3581844261 conn(0x5619e9301750 msgr2=0x5619e9307f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.384+0000 7ff18e033740 1 -- 172.21.15.47:0/3581844261 shutdown_connections 2024-03-21T11:27:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.384+0000 7ff18e033740 1 -- 172.21.15.47:0/3581844261 wait complete. 2024-03-21T11:27:09.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.384+0000 7ff18e033740 1 -- start start 2024-03-21T11:27:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.385+0000 7ff18e033740 1 Processor -- start 2024-03-21T11:27:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.385+0000 7ff18e033740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e93061e0 0x5619e93155c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.385+0000 7ff18e033740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e9306bd0 0x5619e9315b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.385+0000 7ff18e033740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e930bd40 0x5619e9316aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.385+0000 7ff18e033740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1841a9110 con 0x5619e930bd40 2024-03-21T11:27:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.385+0000 7ff18e033740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1841a8ab0 con 0x5619e9306bd0 2024-03-21T11:27:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.385+0000 7ff18e033740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1841a5fd0 con 0x5619e93061e0 2024-03-21T11:27:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.385+0000 7ff18e033740 10 client.0 _ll_register_callbacks cb 0x7ff1700bd980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.386+0000 7ff188a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e93061e0 0x5619e93155c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.386+0000 7ff188a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e93061e0 0x5619e93155c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43912/0 (socket says 172.21.15.47:43912) 2024-03-21T11:27:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.386+0000 7ff188a00640 1 -- 172.21.15.47:0/3856130161 learned_addr learned my addr 172.21.15.47:0/3856130161 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.386+0000 7ff183e00640 1 --2- 172.21.15.47:0/3856130161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e9306bd0 0x5619e9315b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.386+0000 7ff189400640 1 --2- 172.21.15.47:0/3856130161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e930bd40 0x5619e9316aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.386+0000 7ff188a00640 1 -- 172.21.15.47:0/3856130161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e9306bd0 msgr2=0x5619e9315b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.386+0000 7ff188a00640 1 --2- 172.21.15.47:0/3856130161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e9306bd0 0x5619e9315b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.386+0000 7ff188a00640 1 -- 172.21.15.47:0/3856130161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e930bd40 msgr2=0x5619e9316aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.386+0000 7ff188a00640 1 --2- 172.21.15.47:0/3856130161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e930bd40 0x5619e9316aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.386+0000 7ff188a00640 1 -- 172.21.15.47:0/3856130161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619e9317090 con 0x5619e93061e0 2024-03-21T11:27:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.386+0000 7ff188a00640 1 --2- 172.21.15.47:0/3856130161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e93061e0 0x5619e93155c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff17800f520 tx=0x7ff178009dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56e4577af6c69a48 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.386+0000 7ff16a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.387+0000 7ff16a000640 1 -- 172.21.15.47:0/3856130161 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17803d070 con 0x5619e93061e0 2024-03-21T11:27:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.387+0000 7ff18e033740 1 -- 172.21.15.47:0/3856130161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff17802d040 con 0x5619e93061e0 2024-03-21T11:27:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.387+0000 7ff16a000640 1 -- 172.21.15.47:0/3856130161 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff1780051b0 con 0x5619e93061e0 2024-03-21T11:27:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.387+0000 7ff16a000640 1 -- 172.21.15.47:0/3856130161 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1780053b0 con 0x5619e93061e0 2024-03-21T11:27:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.387+0000 7ff18e033740 10 client.26617 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.387+0000 7ff18e033740 1 -- 172.21.15.47:0/3856130161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5619e9380840 con 0x5619e93061e0 2024-03-21T11:27:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.387+0000 7ff16a000640 1 -- 172.21.15.47:0/3856130161 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff17802d040 con 0x5619e93061e0 2024-03-21T11:27:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.387+0000 7ff16a000640 1 -- 172.21.15.47:0/3856130161 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff17800fb50 con 0x5619e93061e0 2024-03-21T11:27:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.387+0000 7ff18e033740 10 client.26617 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.387+0000 7ff18e033740 10 client.26617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.387+0000 7ff18e033740 20 client.26617 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.387+0000 7ff18e033740 1 --2- 172.21.15.47:0/3856130161 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5619e9380f80 0x5619e93833a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.387+0000 7ff18e033740 4 client.26617 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:27:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.388+0000 7ff18e033740 1 -- 172.21.15.47:0/3856130161 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5619e92730c0 con 0x5619e9380f80 2024-03-21T11:27:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.388+0000 7ff183e00640 1 --2- 172.21.15.47:0/3856130161 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5619e9380f80 0x5619e93833a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.388+0000 7ff183e00640 1 --2- 172.21.15.47:0/3856130161 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5619e9380f80 0x5619e93833a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da70a934cb2a78e0 server_cookie=7e47fb98d68fcf01 in_seq=0 out_seq=0 2024-03-21T11:27:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.388+0000 7ff16a000640 10 client.26617 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:09.501 INFO:tasks.workunit.client.0.smithi047.stdout: 1 184287 8.94 MB/sec warmup 113 sec latency 72.889 ms 2024-03-21T11:27:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.610+0000 7ff16a000640 1 -- 172.21.15.47:0/3856130161 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5619e92730c0 con 0x5619e9380f80 2024-03-21T11:27:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.610+0000 7ff16a000640 10 client.26617 handle_command_reply: tid=0 2024-03-21T11:27:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.610+0000 7ff18e033740 1 client.26617 shutdown 2024-03-21T11:27:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.610+0000 7ff18e033740 1 -- 172.21.15.47:0/3856130161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e93061e0 msgr2=0x5619e93155c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.610+0000 7ff18e033740 1 --2- 172.21.15.47:0/3856130161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e93061e0 0x5619e93155c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff17800f520 tx=0x7ff178009dc0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.610+0000 7ff18e033740 1 -- 172.21.15.47:0/3856130161 shutdown_connections 2024-03-21T11:27:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.610+0000 7ff18e033740 1 --2- 172.21.15.47:0/3856130161 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5619e9380f80 0x5619e93833a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.610+0000 7ff18e033740 1 --2- 172.21.15.47:0/3856130161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619e930bd40 0x5619e9316aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.610+0000 7ff18e033740 1 --2- 172.21.15.47:0/3856130161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619e9306bd0 0x5619e9315b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.610+0000 7ff16a000640 0 client.26617 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.610+0000 7ff18e033740 1 --2- 172.21.15.47:0/3856130161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619e93061e0 0x5619e93155c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.610+0000 7ff18e033740 1 -- 172.21.15.47:0/3856130161 >> 172.21.15.47:0/3856130161 conn(0x5619e9301750 msgr2=0x5619e93049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.611+0000 7ff18e033740 1 -- 172.21.15.47:0/3856130161 shutdown_connections 2024-03-21T11:27:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.611+0000 7ff18e033740 1 -- 172.21.15.47:0/3856130161 wait complete. 2024-03-21T11:27:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.611+0000 7ff18e033740 20 client.26617 trim_cache size 0 max 16384 2024-03-21T11:27:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:27:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:27:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:27:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:27:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.612+0000 7ff18b400640 1 -- 172.21.15.47:0/1817099205 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff16c0a3040 msgr2=0x7ff16c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.612+0000 7ff18b400640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff16c0a3040 0x7ff16c0a5500 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff178000c00 tx=0x7ff178014040 comp rx=0 tx=0).stop 2024-03-21T11:27:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.612+0000 7ff18b400640 1 -- 172.21.15.47:0/1817099205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff184104af0 msgr2=0x7ff18419a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.612+0000 7ff18b400640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff184104af0 0x7ff18419a070 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff170000c00 tx=0x7ff17002c650 comp rx=0 tx=0).stop 2024-03-21T11:27:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.612+0000 7ff18b400640 1 -- 172.21.15.47:0/1817099205 shutdown_connections 2024-03-21T11:27:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.612+0000 7ff18b400640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff16c0a3040 0x7ff16c0a5500 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.612+0000 7ff18b400640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1841054b0 0x7ff1841a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.612+0000 7ff18b400640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff184104af0 0x7ff18419a070 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.612+0000 7ff18b400640 1 --2- 172.21.15.47:0/1817099205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff184103900 0x7ff184199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.612+0000 7ff18b400640 1 -- 172.21.15.47:0/1817099205 >> 172.21.15.47:0/1817099205 conn(0x7ff1840fee90 msgr2=0x7ff184107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.612+0000 7ff18b400640 1 -- 172.21.15.47:0/1817099205 shutdown_connections 2024-03-21T11:27:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:09.612+0000 7ff18b400640 1 -- 172.21.15.47:0/1817099205 wait complete. 2024-03-21T11:27:09.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.614+0000 7fd49d600640 1 -- 172.21.15.47:0/4079165019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4980772a0 msgr2=0x7fd498077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.614+0000 7fd49d600640 1 --2- 172.21.15.47:0/4079165019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4980772a0 0x7fd498077700 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd484009900 tx=0x7fd48402bc30 comp rx=0 tx=0).stop 2024-03-21T11:27:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.614+0000 7fd49d600640 1 -- 172.21.15.47:0/4079165019 shutdown_connections 2024-03-21T11:27:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.614+0000 7fd49d600640 1 --2- 172.21.15.47:0/4079165019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd498077c40 0x7fd4980784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.614+0000 7fd49d600640 1 --2- 172.21.15.47:0/4079165019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4980772a0 0x7fd498077700 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.614+0000 7fd49d600640 1 --2- 172.21.15.47:0/4079165019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd498076060 0x7fd498076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.614+0000 7fd49d600640 1 -- 172.21.15.47:0/4079165019 >> 172.21.15.47:0/4079165019 conn(0x7fd4980fee50 msgr2=0x7fd498101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.615+0000 7fd49d600640 1 -- 172.21.15.47:0/4079165019 shutdown_connections 2024-03-21T11:27:09.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.615+0000 7fd49d600640 1 -- 172.21.15.47:0/4079165019 wait complete. 2024-03-21T11:27:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.616+0000 7fd49d600640 1 Processor -- start 2024-03-21T11:27:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.616+0000 7fd49d600640 1 -- start start 2024-03-21T11:27:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.616+0000 7fd49d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd498076060 0x7fd49819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.616+0000 7fd49d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4980772a0 0x7fd49819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.616+0000 7fd49d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd498077c40 0x7fd4981a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.616+0000 7fd49d600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4981139b0 con 0x7fd4980772a0 2024-03-21T11:27:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.616+0000 7fd49d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd498113830 con 0x7fd498076060 2024-03-21T11:27:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.616+0000 7fd49d600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd498113b30 con 0x7fd498077c40 2024-03-21T11:27:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.616+0000 7fd496000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4980772a0 0x7fd49819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.616+0000 7fd497400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd498077c40 0x7fd4981a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.617+0000 7fd496a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd498076060 0x7fd49819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.617+0000 7fd497400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd498077c40 0x7fd4981a55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43924/0 (socket says 172.21.15.47:43924) 2024-03-21T11:27:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.617+0000 7fd497400640 1 -- 172.21.15.47:0/432477166 learned_addr learned my addr 172.21.15.47:0/432477166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.617+0000 7fd497400640 1 -- 172.21.15.47:0/432477166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd498076060 msgr2=0x7fd49819e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.617+0000 7fd497400640 1 --2- 172.21.15.47:0/432477166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd498076060 0x7fd49819e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.617+0000 7fd497400640 1 -- 172.21.15.47:0/432477166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4980772a0 msgr2=0x7fd49819e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.617+0000 7fd497400640 1 --2- 172.21.15.47:0/432477166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4980772a0 0x7fd49819e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.617+0000 7fd497400640 1 -- 172.21.15.47:0/432477166 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4981a5c20 con 0x7fd498077c40 2024-03-21T11:27:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.617+0000 7fd496000640 1 --2- 172.21.15.47:0/432477166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4980772a0 0x7fd49819e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:09.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.617+0000 7fd497400640 1 --2- 172.21.15.47:0/432477166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd498077c40 0x7fd4981a55b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd49000eb10 tx=0x7fd49000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:09.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.618+0000 7fd48f400640 1 -- 172.21.15.47:0/432477166 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd490004b20 con 0x7fd498077c40 2024-03-21T11:27:09.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.618+0000 7fd48f400640 1 -- 172.21.15.47:0/432477166 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd490004cc0 con 0x7fd498077c40 2024-03-21T11:27:09.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.618+0000 7fd49d600640 1 -- 172.21.15.47:0/432477166 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4981a5f10 con 0x7fd498077c40 2024-03-21T11:27:09.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.618+0000 7fd49d600640 1 -- 172.21.15.47:0/432477166 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4981a63d0 con 0x7fd498077c40 2024-03-21T11:27:09.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.618+0000 7fd48f400640 1 -- 172.21.15.47:0/432477166 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd490010760 con 0x7fd498077c40 2024-03-21T11:27:09.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.622+0000 7fd4a03ef740 1 Processor -- start 2024-03-21T11:27:09.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.623+0000 7fd48f400640 1 -- 172.21.15.47:0/432477166 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd490005020 con 0x7fd498077c40 2024-03-21T11:27:09.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.623+0000 7fd4a03ef740 1 -- start start 2024-03-21T11:27:09.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.624+0000 7fd4a03ef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a6970ee80 0x564a6970f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.624+0000 7fd4a03ef740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970f870 0x564a69714370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.624+0000 7fd4a03ef740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a697149e0 0x564a69716e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.624+0000 7fd4a03ef740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a69674250 con 0x564a697149e0 2024-03-21T11:27:09.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.624+0000 7fd4a03ef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a6961ca30 con 0x564a6970ee80 2024-03-21T11:27:09.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.624+0000 7fd4a03ef740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a696702f0 con 0x564a6970f870 2024-03-21T11:27:09.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.624+0000 7fd496a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970f870 0x564a69714370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.624+0000 7fd496000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a6970ee80 0x564a6970f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.624+0000 7fd496a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970f870 0x564a69714370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43932/0 (socket says 172.21.15.47:43932) 2024-03-21T11:27:09.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.624+0000 7fd496a00640 1 -- 172.21.15.47:0/1568928901 learned_addr learned my addr 172.21.15.47:0/1568928901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:09.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.624+0000 7fd496000640 1 --2- 172.21.15.47:0/1568928901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a697149e0 0x564a69716e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.625+0000 7fd496a00640 1 -- 172.21.15.47:0/1568928901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a6970ee80 msgr2=0x564a6970f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.625+0000 7fd496a00640 1 --2- 172.21.15.47:0/1568928901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a6970ee80 0x564a6970f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.625+0000 7fd496a00640 1 -- 172.21.15.47:0/1568928901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a697149e0 msgr2=0x564a69716e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.625+0000 7fd496a00640 1 --2- 172.21.15.47:0/1568928901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a697149e0 0x564a69716e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.625+0000 7fd496a00640 1 -- 172.21.15.47:0/1568928901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a69607690 con 0x564a6970f870 2024-03-21T11:27:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.625+0000 7fd496a00640 1 --2- 172.21.15.47:0/1568928901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970f870 0x564a69714370 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd488009d20 tx=0x7fd488011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0e98ee487a5f464 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.625+0000 7fd48f400640 1 --2- 172.21.15.47:0/432477166 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd47c0a2fa0 0x7fd47c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.625+0000 7fd48f400640 1 -- 172.21.15.47:0/432477166 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd4900cfc60 con 0x7fd498077c40 2024-03-21T11:27:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.625+0000 7fd496000640 1 --2- 172.21.15.47:0/432477166 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd47c0a2fa0 0x7fd47c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.625+0000 7fd483400640 1 -- 172.21.15.47:0/1568928901 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48801e070 con 0x564a6970f870 2024-03-21T11:27:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.625+0000 7fd483400640 1 -- 172.21.15.47:0/1568928901 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd488019b10 con 0x564a6970f870 2024-03-21T11:27:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.625+0000 7fd483400640 1 -- 172.21.15.47:0/1568928901 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800ebf0 con 0x564a6970f870 2024-03-21T11:27:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.626+0000 7fd4a03ef740 1 -- 172.21.15.47:0/1568928901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970f870 msgr2=0x564a69714370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.627+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/1568928901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970f870 0x564a69714370 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd488009d20 tx=0x7fd488011810 comp rx=0 tx=0).stop 2024-03-21T11:27:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.626+0000 7fd496000640 1 --2- 172.21.15.47:0/432477166 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd47c0a2fa0 0x7fd47c0a5460 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd48402c240 tx=0x7fd48402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.627+0000 7fd4a03ef740 1 -- 172.21.15.47:0/1568928901 shutdown_connections 2024-03-21T11:27:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.627+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/1568928901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a697149e0 0x564a69716e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.627+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/1568928901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970f870 0x564a69714370 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.627+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/1568928901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a6970ee80 0x564a6970f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.627+0000 7fd4a03ef740 1 -- 172.21.15.47:0/1568928901 >> 172.21.15.47:0/1568928901 conn(0x564a696d7200 msgr2=0x564a6970a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:09.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.627+0000 7fd4a03ef740 1 -- 172.21.15.47:0/1568928901 shutdown_connections 2024-03-21T11:27:09.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.627+0000 7fd4a03ef740 1 -- 172.21.15.47:0/1568928901 wait complete. 2024-03-21T11:27:09.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.627+0000 7fd4a03ef740 1 -- start start 2024-03-21T11:27:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.628+0000 7fd4a03ef740 1 Processor -- start 2024-03-21T11:27:09.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.628+0000 7fd4a03ef740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970ee80 0x564a69718e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.628+0000 7fd4a03ef740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 0x564a697193a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.628+0000 7fd4a03ef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a697149e0 0x564a697203e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.628+0000 7fd4a03ef740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a69717990 con 0x564a6970f870 2024-03-21T11:27:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.628+0000 7fd4a03ef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a6962fe80 con 0x564a697149e0 2024-03-21T11:27:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.628+0000 7fd4a03ef740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a69717b10 con 0x564a6970ee80 2024-03-21T11:27:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.628+0000 7fd4a03ef740 10 client.0 _ll_register_callbacks cb 0x564a69642130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.628+0000 7fd496a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970ee80 0x564a69718e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.628+0000 7fd497400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a697149e0 0x564a697203e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.628+0000 7fd496a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970ee80 0x564a69718e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43934/0 (socket says 172.21.15.47:43934) 2024-03-21T11:27:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.628+0000 7fd497400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a697149e0 0x564a697203e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59262/0 (socket says 172.21.15.47:59262) 2024-03-21T11:27:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.628+0000 7fd496a00640 1 -- 172.21.15.47:0/3021776983 learned_addr learned my addr 172.21.15.47:0/3021776983 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.628+0000 7fd496000640 1 --2- 172.21.15.47:0/3021776983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 0x564a697193a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.629+0000 7fd497400640 1 -- 172.21.15.47:0/3021776983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970ee80 msgr2=0x564a69718e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.629+0000 7fd497400640 1 --2- 172.21.15.47:0/3021776983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970ee80 0x564a69718e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.629+0000 7fd497400640 1 -- 172.21.15.47:0/3021776983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 msgr2=0x564a697193a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.629+0000 7fd497400640 1 --2- 172.21.15.47:0/3021776983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 0x564a697193a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.629+0000 7fd497400640 1 -- 172.21.15.47:0/3021776983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a694527d0 con 0x564a697149e0 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.629+0000 7fd496000640 1 --2- 172.21.15.47:0/3021776983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 0x564a697193a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.629+0000 7fd496a00640 1 --2- 172.21.15.47:0/3021776983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970ee80 0x564a69718e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.629+0000 7fd497400640 1 --2- 172.21.15.47:0/3021776983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a697149e0 0x564a697203e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd49009bd00 tx=0x7fd49009ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1968758bcc9229d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.629+0000 7fd477e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.629+0000 7fd477e00640 1 -- 172.21.15.47:0/3021776983 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4900a4a10 con 0x564a697149e0 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.629+0000 7fd477e00640 1 -- 172.21.15.47:0/3021776983 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd4900a4580 con 0x564a697149e0 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.629+0000 7fd4a03ef740 1 -- 172.21.15.47:0/3021776983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd488019b10 con 0x564a697149e0 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.630+0000 7fd477e00640 1 -- 172.21.15.47:0/3021776983 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4900a4800 con 0x564a697149e0 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.630+0000 7fd4a03ef740 10 client.26606 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.630+0000 7fd4a03ef740 1 -- 172.21.15.47:0/3021776983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564a693883f0 con 0x564a697149e0 2024-03-21T11:27:09.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.630+0000 7fd477e00640 1 -- 172.21.15.47:0/3021776983 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd488019b10 con 0x564a697149e0 2024-03-21T11:27:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.630+0000 7fd477e00640 1 -- 172.21.15.47:0/3021776983 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd49009b450 con 0x564a697149e0 2024-03-21T11:27:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.631+0000 7fd4a03ef740 10 client.26606 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.631+0000 7fd4a03ef740 10 client.26606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.631+0000 7fd4a03ef740 20 client.26606 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.631+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/3021776983 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564a69724bb0 0x564a69727050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.631+0000 7fd4a03ef740 4 client.26606 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.631+0000 7fd4a03ef740 1 -- 172.21.15.47:0/3021776983 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a69681790 con 0x564a69724bb0 2024-03-21T11:27:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.631+0000 7fd496a00640 1 --2- 172.21.15.47:0/3021776983 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564a69724bb0 0x564a69727050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.632+0000 7fd496a00640 1 --2- 172.21.15.47:0/3021776983 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564a69724bb0 0x564a69727050 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce60d6dd361fac0 server_cookie=73a472f2145cced1 in_seq=0 out_seq=0 2024-03-21T11:27:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.632+0000 7fd477e00640 10 client.26606 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.632+0000 7fd477e00640 1 -- 172.21.15.47:0/3021776983 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564a69681790 con 0x564a69724bb0 2024-03-21T11:27:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.632+0000 7fd477e00640 10 client.26606 handle_command_reply: tid=0 2024-03-21T11:27:09.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.632+0000 7fd4a03ef740 1 client.26606 shutdown 2024-03-21T11:27:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.633+0000 7fd4a03ef740 1 -- 172.21.15.47:0/3021776983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a697149e0 msgr2=0x564a697203e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.633+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/3021776983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a697149e0 0x564a697203e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd49009bd00 tx=0x7fd49009ba20 comp rx=0 tx=0).stop 2024-03-21T11:27:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.633+0000 7fd4a03ef740 1 -- 172.21.15.47:0/3021776983 shutdown_connections 2024-03-21T11:27:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.633+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/3021776983 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564a69724bb0 0x564a69727050 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.633+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/3021776983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a697149e0 0x564a697203e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.633+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/3021776983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 0x564a697193a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.633+0000 7fd477e00640 0 client.26606 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.633+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/3021776983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970ee80 0x564a69718e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.633+0000 7fd4a03ef740 1 -- 172.21.15.47:0/3021776983 >> 172.21.15.47:0/3021776983 conn(0x564a696d7200 msgr2=0x564a69709a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.633+0000 7fd4a03ef740 1 -- 172.21.15.47:0/3021776983 shutdown_connections 2024-03-21T11:27:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.633+0000 7fd4a03ef740 1 -- 172.21.15.47:0/3021776983 wait complete. 2024-03-21T11:27:09.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.633+0000 7fd4a03ef740 20 client.26606 trim_cache size 0 max 16384 2024-03-21T11:27:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.652+0000 7fd4a03ef740 1 Processor -- start 2024-03-21T11:27:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.652+0000 7fd4a03ef740 1 -- start start 2024-03-21T11:27:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.652+0000 7fd4a03ef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a6970ee80 0x564a69718150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.652+0000 7fd4a03ef740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 0x564a697203a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.652+0000 7fd4a03ef740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a697149e0 0x564a697109f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.652+0000 7fd4a03ef740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47800f930 con 0x564a6970f870 2024-03-21T11:27:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.652+0000 7fd4a03ef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4780112b0 con 0x564a6970ee80 2024-03-21T11:27:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.652+0000 7fd4a03ef740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47800ff90 con 0x564a697149e0 2024-03-21T11:27:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.653+0000 7fd496a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a6970ee80 0x564a69718150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.653+0000 7fd496a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a6970ee80 0x564a69718150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59270/0 (socket says 172.21.15.47:59270) 2024-03-21T11:27:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.653+0000 7fd496a00640 1 -- 172.21.15.47:0/612456377 learned_addr learned my addr 172.21.15.47:0/612456377 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.653+0000 7fd497400640 1 --2- 172.21.15.47:0/612456377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a697149e0 0x564a697109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.653+0000 7fd496000640 1 --2- 172.21.15.47:0/612456377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 0x564a697203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.653+0000 7fd496a00640 1 -- 172.21.15.47:0/612456377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a697149e0 msgr2=0x564a697109f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.653+0000 7fd496a00640 1 --2- 172.21.15.47:0/612456377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a697149e0 0x564a697109f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.653+0000 7fd496a00640 1 -- 172.21.15.47:0/612456377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 msgr2=0x564a697203a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.653+0000 7fd496a00640 1 --2- 172.21.15.47:0/612456377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 0x564a697203a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.653+0000 7fd496a00640 1 -- 172.21.15.47:0/612456377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a693480f0 con 0x564a6970ee80 2024-03-21T11:27:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.653+0000 7fd496000640 1 --2- 172.21.15.47:0/612456377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 0x564a697203a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.653+0000 7fd496a00640 1 --2- 172.21.15.47:0/612456377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a6970ee80 0x564a69718150 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd48801c040 tx=0x7fd488009910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad3605a3b6830a9f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.654+0000 7fd482a00640 1 -- 172.21.15.47:0/612456377 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48801e040 con 0x564a6970ee80 2024-03-21T11:27:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.654+0000 7fd482a00640 1 -- 172.21.15.47:0/612456377 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4880035f0 con 0x564a6970ee80 2024-03-21T11:27:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.654+0000 7fd482a00640 1 -- 172.21.15.47:0/612456377 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4880197b0 con 0x564a6970ee80 2024-03-21T11:27:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.654+0000 7fd4a03ef740 1 -- 172.21.15.47:0/612456377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a6970ee80 msgr2=0x564a69718150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.654+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/612456377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a6970ee80 0x564a69718150 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd48801c040 tx=0x7fd488009910 comp rx=0 tx=0).stop 2024-03-21T11:27:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.655+0000 7fd4a03ef740 1 -- 172.21.15.47:0/612456377 shutdown_connections 2024-03-21T11:27:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.655+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/612456377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a697149e0 0x564a697109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.655+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/612456377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 0x564a697203a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.655+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/612456377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a6970ee80 0x564a69718150 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.655+0000 7fd4a03ef740 1 -- 172.21.15.47:0/612456377 >> 172.21.15.47:0/612456377 conn(0x564a696d7200 msgr2=0x564a697102e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.655+0000 7fd4a03ef740 1 -- 172.21.15.47:0/612456377 shutdown_connections 2024-03-21T11:27:09.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.655+0000 7fd4a03ef740 1 -- 172.21.15.47:0/612456377 wait complete. 2024-03-21T11:27:09.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.655+0000 7fd4a03ef740 1 -- start start 2024-03-21T11:27:09.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.655+0000 7fd4a03ef740 1 Processor -- start 2024-03-21T11:27:09.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd4a03ef740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970ee80 0x564a6971da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd4a03ef740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 0x564a6971df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd4a03ef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a697149e0 0x564a6971ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd4a03ef740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd478010c50 con 0x564a6970f870 2024-03-21T11:27:09.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd4a03ef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4780105f0 con 0x564a697149e0 2024-03-21T11:27:09.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd4a03ef740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47800daf0 con 0x564a6970ee80 2024-03-21T11:27:09.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd4a03ef740 10 client.0 _ll_register_callbacks cb 0x7fd49009bec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:09.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd496000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 0x564a6971df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd496a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970ee80 0x564a6971da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd496a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970ee80 0x564a6971da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43940/0 (socket says 172.21.15.47:43940) 2024-03-21T11:27:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd496a00640 1 -- 172.21.15.47:0/2733832236 learned_addr learned my addr 172.21.15.47:0/2733832236 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd497400640 1 --2- 172.21.15.47:0/2733832236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a697149e0 0x564a6971ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd496a00640 1 -- 172.21.15.47:0/2733832236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a697149e0 msgr2=0x564a6971ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd496a00640 1 --2- 172.21.15.47:0/2733832236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a697149e0 0x564a6971ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd496a00640 1 -- 172.21.15.47:0/2733832236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 msgr2=0x564a6971df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd496a00640 1 --2- 172.21.15.47:0/2733832236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 0x564a6971df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.656+0000 7fd496a00640 1 -- 172.21.15.47:0/2733832236 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a6971f570 con 0x564a6970ee80 2024-03-21T11:27:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.657+0000 7fd496a00640 1 --2- 172.21.15.47:0/2733832236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970ee80 0x564a6971da30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd488009250 tx=0x7fd488009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cdf25aea8c9e144 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.657+0000 7fd477e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.657+0000 7fd477e00640 1 -- 172.21.15.47:0/2733832236 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488032030 con 0x564a6970ee80 2024-03-21T11:27:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.657+0000 7fd4a03ef740 1 -- 172.21.15.47:0/2733832236 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4880035f0 con 0x564a6970ee80 2024-03-21T11:27:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.657+0000 7fd477e00640 1 -- 172.21.15.47:0/2733832236 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd4880051d0 con 0x564a6970ee80 2024-03-21T11:27:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.657+0000 7fd477e00640 1 -- 172.21.15.47:0/2733832236 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd48803e050 con 0x564a6970ee80 2024-03-21T11:27:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.657+0000 7fd4a03ef740 10 client.26635 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.657+0000 7fd4a03ef740 1 -- 172.21.15.47:0/2733832236 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564a697804e0 con 0x564a6970ee80 2024-03-21T11:27:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.658+0000 7fd477e00640 1 -- 172.21.15.47:0/2733832236 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4880035f0 con 0x564a6970ee80 2024-03-21T11:27:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.658+0000 7fd477e00640 1 -- 172.21.15.47:0/2733832236 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd488019910 con 0x564a6970ee80 2024-03-21T11:27:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.658+0000 7fd4a03ef740 10 client.26635 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.658+0000 7fd4a03ef740 10 client.26635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.658+0000 7fd4a03ef740 20 client.26635 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.658+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/2733832236 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564a69780be0 0x564a69783000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.658+0000 7fd4a03ef740 4 client.26635 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:27:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.658+0000 7fd4a03ef740 1 -- 172.21.15.47:0/2733832236 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564a6967bae0 con 0x564a69780be0 2024-03-21T11:27:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.658+0000 7fd496000640 1 --2- 172.21.15.47:0/2733832236 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564a69780be0 0x564a69783000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.659+0000 7fd496000640 1 --2- 172.21.15.47:0/2733832236 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564a69780be0 0x564a69783000 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ab2896993ee18a0 server_cookie=77ca7eba3f63d488 in_seq=0 out_seq=0 2024-03-21T11:27:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.659+0000 7fd477e00640 10 client.26635 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.661+0000 7fd477e00640 1 -- 172.21.15.47:0/2733832236 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564a6967bae0 con 0x564a69780be0 2024-03-21T11:27:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.661+0000 7fd477e00640 10 client.26635 handle_command_reply: tid=0 2024-03-21T11:27:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.661+0000 7fd4a03ef740 1 client.26635 shutdown 2024-03-21T11:27:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.661+0000 7fd4a03ef740 1 -- 172.21.15.47:0/2733832236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970ee80 msgr2=0x564a6971da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.661+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/2733832236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970ee80 0x564a6971da30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd488009250 tx=0x7fd488009830 comp rx=0 tx=0).stop 2024-03-21T11:27:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.661+0000 7fd4a03ef740 1 -- 172.21.15.47:0/2733832236 shutdown_connections 2024-03-21T11:27:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.661+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/2733832236 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564a69780be0 0x564a69783000 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.661+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/2733832236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a697149e0 0x564a6971ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.661+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/2733832236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a6970f870 0x564a6971df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.661+0000 7fd4a03ef740 1 --2- 172.21.15.47:0/2733832236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a6970ee80 0x564a6971da30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.661+0000 7fd477e00640 0 client.26635 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.661+0000 7fd4a03ef740 1 -- 172.21.15.47:0/2733832236 >> 172.21.15.47:0/2733832236 conn(0x564a696d7200 msgr2=0x564a69710b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.661+0000 7fd4a03ef740 1 -- 172.21.15.47:0/2733832236 shutdown_connections 2024-03-21T11:27:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.661+0000 7fd4a03ef740 1 -- 172.21.15.47:0/2733832236 wait complete. 2024-03-21T11:27:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.662+0000 7fd4a03ef740 20 client.26635 trim_cache size 0 max 16384 2024-03-21T11:27:09.667 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:27:09.667 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:27:09.667 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:27:09.667 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:27:09.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.662+0000 7fd49d600640 1 -- 172.21.15.47:0/432477166 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd47c0a2fa0 msgr2=0x7fd47c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.662+0000 7fd49d600640 1 --2- 172.21.15.47:0/432477166 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd47c0a2fa0 0x7fd47c0a5460 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd48402c240 tx=0x7fd48402df90 comp rx=0 tx=0).stop 2024-03-21T11:27:09.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.662+0000 7fd49d600640 1 -- 172.21.15.47:0/432477166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd498077c40 msgr2=0x7fd4981a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.662+0000 7fd49d600640 1 --2- 172.21.15.47:0/432477166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd498077c40 0x7fd4981a55b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd49000eb10 tx=0x7fd49000c2d0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.662+0000 7fd49d600640 1 -- 172.21.15.47:0/432477166 shutdown_connections 2024-03-21T11:27:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.662+0000 7fd49d600640 1 --2- 172.21.15.47:0/432477166 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd47c0a2fa0 0x7fd47c0a5460 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.662+0000 7fd49d600640 1 --2- 172.21.15.47:0/432477166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd498077c40 0x7fd4981a55b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.662+0000 7fd49d600640 1 --2- 172.21.15.47:0/432477166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4980772a0 0x7fd49819e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.662+0000 7fd49d600640 1 --2- 172.21.15.47:0/432477166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd498076060 0x7fd49819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.662+0000 7fd49d600640 1 -- 172.21.15.47:0/432477166 >> 172.21.15.47:0/432477166 conn(0x7fd4980fee50 msgr2=0x7fd498100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.663+0000 7fd49d600640 1 -- 172.21.15.47:0/432477166 shutdown_connections 2024-03-21T11:27:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.663+0000 7fd49d600640 1 -- 172.21.15.47:0/432477166 wait complete. 2024-03-21T11:27:09.680 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:27:09.681 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:27:09.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.956+0000 7f5657e00640 1 -- 172.21.15.47:0/989067807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56501017b0 msgr2=0x7f5650101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.956+0000 7f5657e00640 1 --2- 172.21.15.47:0/989067807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56501017b0 0x7f5650101c30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f563c009860 tx=0x7f563c02bb90 comp rx=0 tx=0).stop 2024-03-21T11:27:09.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.957+0000 7f5657e00640 1 -- 172.21.15.47:0/989067807 shutdown_connections 2024-03-21T11:27:09.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.957+0000 7f5657e00640 1 --2- 172.21.15.47:0/989067807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5650102170 0x7f5650068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.957+0000 7f5657e00640 1 --2- 172.21.15.47:0/989067807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56501017b0 0x7f5650101c30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.957+0000 7f5657e00640 1 --2- 172.21.15.47:0/989067807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56501005c0 0x7f56501009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.957+0000 7f5657e00640 1 -- 172.21.15.47:0/989067807 >> 172.21.15.47:0/989067807 conn(0x7f56500fbb30 msgr2=0x7f56500fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:09.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.958+0000 7f5657e00640 1 -- 172.21.15.47:0/989067807 shutdown_connections 2024-03-21T11:27:09.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.958+0000 7f5657e00640 1 -- 172.21.15.47:0/989067807 wait complete. 2024-03-21T11:27:09.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.960+0000 7f5657e00640 1 Processor -- start 2024-03-21T11:27:09.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.962+0000 7f5657e00640 1 -- start start 2024-03-21T11:27:09.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.962+0000 7f5657e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56501005c0 0x7f5650195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.962+0000 7f5657e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56501017b0 0x7f5650195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.962+0000 7f5657e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650102170 0x7f56501ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.962+0000 7f5657e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565010d380 con 0x7f56501017b0 2024-03-21T11:27:09.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.962+0000 7f5657e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565010d200 con 0x7f56501005c0 2024-03-21T11:27:09.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.962+0000 7f5657e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565010d500 con 0x7f5650102170 2024-03-21T11:27:09.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.963+0000 7f5655e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650102170 0x7f56501ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.963+0000 7f5655e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650102170 0x7f56501ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43952/0 (socket says 172.21.15.47:43952) 2024-03-21T11:27:09.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.963+0000 7f5655e00640 1 -- 172.21.15.47:0/3059985367 learned_addr learned my addr 172.21.15.47:0/3059985367 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:09.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.963+0000 7f5655400640 1 --2- 172.21.15.47:0/3059985367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56501005c0 0x7f5650195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.963+0000 7f5655e00640 1 -- 172.21.15.47:0/3059985367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56501005c0 msgr2=0x7f5650195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:09.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.963+0000 7f5655e00640 1 --2- 172.21.15.47:0/3059985367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56501005c0 0x7f5650195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.963+0000 7f5655e00640 1 -- 172.21.15.47:0/3059985367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56501017b0 msgr2=0x7f5650195dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:27:09.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.965+0000 7f5654a00640 1 --2- 172.21.15.47:0/3059985367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56501017b0 0x7f5650195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.965+0000 7f5655e00640 1 --2- 172.21.15.47:0/3059985367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56501017b0 0x7f5650195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:09.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.965+0000 7f5655e00640 1 -- 172.21.15.47:0/3059985367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565019af60 con 0x7f5650102170 2024-03-21T11:27:09.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.965+0000 7f5655e00640 1 --2- 172.21.15.47:0/3059985367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650102170 0x7f56501ae290 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f564400c970 tx=0x7f564400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:09.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.966+0000 7f564e000640 1 -- 172.21.15.47:0/3059985367 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644007bb0 con 0x7f5650102170 2024-03-21T11:27:09.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.966+0000 7f564e000640 1 -- 172.21.15.47:0/3059985367 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5644007d50 con 0x7f5650102170 2024-03-21T11:27:09.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.966+0000 7f564e000640 1 -- 172.21.15.47:0/3059985367 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564400f450 con 0x7f5650102170 2024-03-21T11:27:09.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.967+0000 7f5657e00640 1 -- 172.21.15.47:0/3059985367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56501ae7d0 con 0x7f5650102170 2024-03-21T11:27:09.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.967+0000 7f5657e00640 1 -- 172.21.15.47:0/3059985367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56501aecd0 con 0x7f5650102170 2024-03-21T11:27:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.972+0000 7f5657e00640 1 -- 172.21.15.47:0/3059985367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5650107340 con 0x7f5650102170 2024-03-21T11:27:09.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.978+0000 7f564e000640 1 -- 172.21.15.47:0/3059985367 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5644004f40 con 0x7f5650102170 2024-03-21T11:27:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.980+0000 7f564e000640 1 --2- 172.21.15.47:0/3059985367 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56380a3120 0x7f56380a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.981+0000 7f564e000640 1 -- 172.21.15.47:0/3059985367 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f56440cf480 con 0x7f5650102170 2024-03-21T11:27:09.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.981+0000 7f5655400640 1 --2- 172.21.15.47:0/3059985367 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56380a3120 0x7f56380a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:09.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.982+0000 7f564e000640 1 -- 172.21.15.47:0/3059985367 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5644014030 con 0x7f5650102170 2024-03-21T11:27:09.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:09.983+0000 7f5655400640 1 --2- 172.21.15.47:0/3059985367 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56380a3120 0x7f56380a55e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5650101610 tx=0x7f56480095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:10.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.293+0000 7f5657e00640 1 -- 172.21.15.47:0/3059985367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f565002d070 con 0x7f5650102170 2024-03-21T11:27:10.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.294+0000 7f564e000640 1 -- 172.21.15.47:0/3059985367 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f564409bfc0 con 0x7f5650102170 2024-03-21T11:27:10.295 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:27:10.295 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:27:10.296 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:27:10.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.297+0000 7f5657e00640 1 -- 172.21.15.47:0/3059985367 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56380a3120 msgr2=0x7f56380a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.297+0000 7f5657e00640 1 --2- 172.21.15.47:0/3059985367 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56380a3120 0x7f56380a55e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5650101610 tx=0x7f56480095c0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.297+0000 7f5657e00640 1 -- 172.21.15.47:0/3059985367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650102170 msgr2=0x7f56501ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.297+0000 7f5657e00640 1 --2- 172.21.15.47:0/3059985367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650102170 0x7f56501ae290 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f564400c970 tx=0x7f564400ce30 comp rx=0 tx=0).stop 2024-03-21T11:27:10.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.297+0000 7f5657e00640 1 -- 172.21.15.47:0/3059985367 shutdown_connections 2024-03-21T11:27:10.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.297+0000 7f5657e00640 1 --2- 172.21.15.47:0/3059985367 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f56380a3120 0x7f56380a55e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.297+0000 7f5657e00640 1 --2- 172.21.15.47:0/3059985367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650102170 0x7f56501ae290 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.297+0000 7f5657e00640 1 --2- 172.21.15.47:0/3059985367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56501017b0 0x7f5650195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.297+0000 7f5657e00640 1 --2- 172.21.15.47:0/3059985367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56501005c0 0x7f5650195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.297+0000 7f5657e00640 1 -- 172.21.15.47:0/3059985367 >> 172.21.15.47:0/3059985367 conn(0x7f56500fbb30 msgr2=0x7f56500fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.298+0000 7f5657e00640 1 -- 172.21.15.47:0/3059985367 shutdown_connections 2024-03-21T11:27:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.298+0000 7f5657e00640 1 -- 172.21.15.47:0/3059985367 wait complete. 2024-03-21T11:27:10.317 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:10.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:27:10.501 INFO:tasks.workunit.client.0.smithi047.stdout: 1 185113 8.91 MB/sec warmup 114 sec latency 63.207 ms 2024-03-21T11:27:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.528+0000 7f5504400640 1 -- 172.21.15.47:0/2332099387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54fc0772a0 msgr2=0x7f54fc077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.528+0000 7f5504400640 1 --2- 172.21.15.47:0/2332099387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54fc0772a0 0x7f54fc077700 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009900 tx=0x7f54f002bc50 comp rx=0 tx=0).stop 2024-03-21T11:27:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.528+0000 7f5504400640 1 -- 172.21.15.47:0/2332099387 shutdown_connections 2024-03-21T11:27:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.528+0000 7f5504400640 1 --2- 172.21.15.47:0/2332099387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54fc077c40 0x7f54fc0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.528+0000 7f5504400640 1 --2- 172.21.15.47:0/2332099387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54fc0772a0 0x7f54fc077700 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.528+0000 7f5504400640 1 --2- 172.21.15.47:0/2332099387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc076060 0x7f54fc076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.528+0000 7f5504400640 1 -- 172.21.15.47:0/2332099387 >> 172.21.15.47:0/2332099387 conn(0x7f54fc0fee50 msgr2=0x7f54fc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.528+0000 7f5504400640 1 -- 172.21.15.47:0/2332099387 shutdown_connections 2024-03-21T11:27:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.528+0000 7f5504400640 1 -- 172.21.15.47:0/2332099387 wait complete. 2024-03-21T11:27:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.529+0000 7f5504400640 1 Processor -- start 2024-03-21T11:27:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.530+0000 7f5504400640 1 -- start start 2024-03-21T11:27:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.530+0000 7f5504400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54fc076060 0x7f54fc19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.530+0000 7f5504400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc0772a0 0x7f54fc19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.530+0000 7f5504400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54fc077c40 0x7f54fc1a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.530+0000 7f5504400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc1139b0 con 0x7f54fc076060 2024-03-21T11:27:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.530+0000 7f5504400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc113830 con 0x7f54fc077c40 2024-03-21T11:27:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.530+0000 7f5504400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc113b30 con 0x7f54fc0772a0 2024-03-21T11:27:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.530+0000 7f5501000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc0772a0 0x7f54fc19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.530+0000 7f5501a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54fc076060 0x7f54fc19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.530+0000 7f5501000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc0772a0 0x7f54fc19e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43964/0 (socket says 172.21.15.47:43964) 2024-03-21T11:27:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.530+0000 7f5501000640 1 -- 172.21.15.47:0/3040272554 learned_addr learned my addr 172.21.15.47:0/3040272554 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.530+0000 7f5502400640 1 --2- 172.21.15.47:0/3040272554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54fc077c40 0x7f54fc1a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.531+0000 7f5501000640 1 -- 172.21.15.47:0/3040272554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54fc077c40 msgr2=0x7f54fc1a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.531+0000 7f5501000640 1 --2- 172.21.15.47:0/3040272554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54fc077c40 0x7f54fc1a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.531+0000 7f5501000640 1 -- 172.21.15.47:0/3040272554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54fc076060 msgr2=0x7f54fc19dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.531+0000 7f5501000640 1 --2- 172.21.15.47:0/3040272554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54fc076060 0x7f54fc19dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.531+0000 7f5501000640 1 -- 172.21.15.47:0/3040272554 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc1a5c10 con 0x7f54fc0772a0 2024-03-21T11:27:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.531+0000 7f5501a00640 1 --2- 172.21.15.47:0/3040272554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54fc076060 0x7f54fc19dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.531+0000 7f5501000640 1 --2- 172.21.15.47:0/3040272554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc0772a0 0x7f54fc19e530 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f54f002c260 tx=0x7f54f002c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.532+0000 7f54fa000640 1 -- 172.21.15.47:0/3040272554 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0041070 con 0x7f54fc0772a0 2024-03-21T11:27:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.532+0000 7f54fa000640 1 -- 172.21.15.47:0/3040272554 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54f0004450 con 0x7f54fc0772a0 2024-03-21T11:27:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.532+0000 7f5504400640 1 -- 172.21.15.47:0/3040272554 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc1a5ea0 con 0x7f54fc0772a0 2024-03-21T11:27:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.532+0000 7f54fa000640 1 -- 172.21.15.47:0/3040272554 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0034440 con 0x7f54fc0772a0 2024-03-21T11:27:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.532+0000 7f5504400640 1 -- 172.21.15.47:0/3040272554 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc1a63b0 con 0x7f54fc0772a0 2024-03-21T11:27:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.535+0000 7f55071b8740 1 Processor -- start 2024-03-21T11:27:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.535+0000 7f55071b8740 1 -- start start 2024-03-21T11:27:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.536+0000 7f54fa000640 1 -- 172.21.15.47:0/3040272554 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f54f0044070 con 0x7f54fc0772a0 2024-03-21T11:27:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.536+0000 7f55071b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cbac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.536+0000 7f55071b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 0x561624cbbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.536+0000 7f55071b8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbc3a0 0x561624cc2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.536+0000 7f55071b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561624c1f2c0 con 0x561624cbb9e0 2024-03-21T11:27:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.536+0000 7f55071b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561624bc7c90 con 0x561624cba810 2024-03-21T11:27:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.536+0000 7f55071b8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561624c1b360 con 0x561624cbc3a0 2024-03-21T11:27:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.536+0000 7f5502400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cbac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.536+0000 7f5501a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 0x561624cbbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.536+0000 7f5502400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cbac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59328/0 (socket says 172.21.15.47:59328) 2024-03-21T11:27:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.536+0000 7f5502400640 1 -- 172.21.15.47:0/3936851459 learned_addr learned my addr 172.21.15.47:0/3936851459 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.536+0000 7f5501000640 1 --2- 172.21.15.47:0/3936851459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbc3a0 0x561624cc2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.537+0000 7f5502400640 1 -- 172.21.15.47:0/3936851459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbc3a0 msgr2=0x561624cc2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.537+0000 7f5502400640 1 --2- 172.21.15.47:0/3936851459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbc3a0 0x561624cc2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.537+0000 7f5502400640 1 -- 172.21.15.47:0/3936851459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 msgr2=0x561624cbbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.537+0000 7f5502400640 1 --2- 172.21.15.47:0/3936851459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 0x561624cbbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.537+0000 7f5502400640 1 -- 172.21.15.47:0/3936851459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561624bec080 con 0x561624cba810 2024-03-21T11:27:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.537+0000 7f5502400640 1 --2- 172.21.15.47:0/3936851459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cbac30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009fd0 tx=0x7f54f40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bae23559c6da418 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.537+0000 7f54fa000640 1 --2- 172.21.15.47:0/3040272554 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54e40a2fa0 0x7f54e40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.537+0000 7f54eaa00640 1 -- 172.21.15.47:0/3936851459 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4004d20 con 0x561624cba810 2024-03-21T11:27:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.538+0000 7f54eaa00640 1 -- 172.21.15.47:0/3936851459 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54f4004050 con 0x561624cba810 2024-03-21T11:27:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.538+0000 7f54eaa00640 1 -- 172.21.15.47:0/3936851459 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4005260 con 0x561624cba810 2024-03-21T11:27:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.538+0000 7f54fa000640 1 -- 172.21.15.47:0/3040272554 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f54f00f1220 con 0x7f54fc0772a0 2024-03-21T11:27:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.538+0000 7f5501a00640 1 --2- 172.21.15.47:0/3040272554 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54e40a2fa0 0x7f54e40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.538+0000 7f55071b8740 1 -- 172.21.15.47:0/3936851459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 msgr2=0x561624cbac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.538+0000 7f55071b8740 1 --2- 172.21.15.47:0/3936851459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cbac30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009fd0 tx=0x7f54f40118d0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.539+0000 7f5501a00640 1 --2- 172.21.15.47:0/3040272554 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54e40a2fa0 0x7f54e40a5460 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f54ec012640 tx=0x7f54ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.539+0000 7f55071b8740 1 -- 172.21.15.47:0/3936851459 shutdown_connections 2024-03-21T11:27:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.539+0000 7f55071b8740 1 --2- 172.21.15.47:0/3936851459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbc3a0 0x561624cc2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.539+0000 7f55071b8740 1 --2- 172.21.15.47:0/3936851459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 0x561624cbbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.539+0000 7f55071b8740 1 --2- 172.21.15.47:0/3936851459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cbac30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.539+0000 7f55071b8740 1 -- 172.21.15.47:0/3936851459 >> 172.21.15.47:0/3936851459 conn(0x561624cb5c00 msgr2=0x561624cb6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.539+0000 7f55071b8740 1 -- 172.21.15.47:0/3936851459 shutdown_connections 2024-03-21T11:27:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.539+0000 7f55071b8740 1 -- 172.21.15.47:0/3936851459 wait complete. 2024-03-21T11:27:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.539+0000 7f55071b8740 1 -- start start 2024-03-21T11:27:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.539+0000 7f55071b8740 1 Processor -- start 2024-03-21T11:27:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.540+0000 7f55071b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cc4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.540+0000 7f55071b8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbb9e0 0x561624cc5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.540+0000 7f55071b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbc3a0 0x561624ccc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.540+0000 7f55071b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561624cc33b0 con 0x561624cbc3a0 2024-03-21T11:27:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.540+0000 7f55071b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561624bdb0d0 con 0x561624cba810 2024-03-21T11:27:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.540+0000 7f55071b8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561624cc3530 con 0x561624cbb9e0 2024-03-21T11:27:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.540+0000 7f55071b8740 10 client.0 _ll_register_callbacks cb 0x561624bdfe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.540+0000 7f5501a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cc4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.540+0000 7f5501a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cc4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59330/0 (socket says 172.21.15.47:59330) 2024-03-21T11:27:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.540+0000 7f5501a00640 1 -- 172.21.15.47:0/676878415 learned_addr learned my addr 172.21.15.47:0/676878415 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.540+0000 7f5501000640 1 --2- 172.21.15.47:0/676878415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbb9e0 0x561624cc5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.541+0000 7f5501a00640 1 -- 172.21.15.47:0/676878415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbb9e0 msgr2=0x561624cc5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.541+0000 7f5501a00640 1 --2- 172.21.15.47:0/676878415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbb9e0 0x561624cc5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.541+0000 7f5501a00640 1 -- 172.21.15.47:0/676878415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbc3a0 msgr2=0x561624ccc010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:27:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.541+0000 7f5501a00640 1 --2- 172.21.15.47:0/676878415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbc3a0 0x561624ccc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.541+0000 7f5501a00640 1 -- 172.21.15.47:0/676878415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616249333f0 con 0x561624cba810 2024-03-21T11:27:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.541+0000 7f5501a00640 1 --2- 172.21.15.47:0/676878415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cc4b00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f54ec003640 tx=0x7f54ec004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=395c3dd94850eb82 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.541+0000 7f54df400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.541+0000 7f54df400640 1 -- 172.21.15.47:0/676878415 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec005020 con 0x561624cba810 2024-03-21T11:27:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.541+0000 7f54df400640 1 -- 172.21.15.47:0/676878415 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f54ec0051c0 con 0x561624cba810 2024-03-21T11:27:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.541+0000 7f55071b8740 1 -- 172.21.15.47:0/676878415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54f4004050 con 0x561624cba810 2024-03-21T11:27:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.542+0000 7f54df400640 1 -- 172.21.15.47:0/676878415 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54ec0053c0 con 0x561624cba810 2024-03-21T11:27:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.542+0000 7f55071b8740 10 client.26630 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.542+0000 7f55071b8740 1 -- 172.21.15.47:0/676878415 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5616249fd7d0 con 0x561624cba810 2024-03-21T11:27:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.542+0000 7f54df400640 1 -- 172.21.15.47:0/676878415 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54f4004050 con 0x561624cba810 2024-03-21T11:27:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.542+0000 7f54df400640 1 -- 172.21.15.47:0/676878415 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f54ec027070 con 0x561624cba810 2024-03-21T11:27:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.542+0000 7f55071b8740 10 client.26630 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.542+0000 7f55071b8740 10 client.26630 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:27:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.543+0000 7f55071b8740 20 client.26630 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.543+0000 7f55071b8740 1 --2- 172.21.15.47:0/676878415 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561624cd05b0 0x561624cd2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.543+0000 7f55071b8740 4 client.26630 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.543+0000 7f55071b8740 1 -- 172.21.15.47:0/676878415 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561624c2c800 con 0x561624cd05b0 2024-03-21T11:27:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.543+0000 7f5501000640 1 --2- 172.21.15.47:0/676878415 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561624cd05b0 0x561624cd2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.543+0000 7f5501000640 1 --2- 172.21.15.47:0/676878415 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561624cd05b0 0x561624cd2a50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=729217e9e0796568 server_cookie=cae3de20c08c377f in_seq=0 out_seq=0 2024-03-21T11:27:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.543+0000 7f54df400640 10 client.26630 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:27:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.544+0000 7f54df400640 1 -- 172.21.15.47:0/676878415 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561624c2c800 con 0x561624cd05b0 2024-03-21T11:27:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.544+0000 7f54df400640 10 client.26630 handle_command_reply: tid=0 2024-03-21T11:27:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.544+0000 7f55071b8740 1 client.26630 shutdown 2024-03-21T11:27:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.544+0000 7f55071b8740 1 -- 172.21.15.47:0/676878415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 msgr2=0x561624cc4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.544+0000 7f55071b8740 1 --2- 172.21.15.47:0/676878415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cc4b00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f54ec003640 tx=0x7f54ec004730 comp rx=0 tx=0).stop 2024-03-21T11:27:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.544+0000 7f55071b8740 1 -- 172.21.15.47:0/676878415 shutdown_connections 2024-03-21T11:27:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.544+0000 7f55071b8740 1 --2- 172.21.15.47:0/676878415 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561624cd05b0 0x561624cd2a50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.544+0000 7f55071b8740 1 --2- 172.21.15.47:0/676878415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbc3a0 0x561624ccc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.544+0000 7f55071b8740 1 --2- 172.21.15.47:0/676878415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbb9e0 0x561624cc5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.544+0000 7f55071b8740 1 --2- 172.21.15.47:0/676878415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cc4b00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.544+0000 7f54df400640 0 client.26630 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:27:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.544+0000 7f55071b8740 1 -- 172.21.15.47:0/676878415 >> 172.21.15.47:0/676878415 conn(0x561624cb5c00 msgr2=0x561624cb4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.545+0000 7f55071b8740 1 -- 172.21.15.47:0/676878415 shutdown_connections 2024-03-21T11:27:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.545+0000 7f55071b8740 1 -- 172.21.15.47:0/676878415 wait complete. 2024-03-21T11:27:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.545+0000 7f55071b8740 20 client.26630 trim_cache size 0 max 16384 2024-03-21T11:27:10.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.560+0000 7f55071b8740 1 Processor -- start 2024-03-21T11:27:10.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.560+0000 7f55071b8740 1 -- start start 2024-03-21T11:27:10.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.560+0000 7f55071b8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cba810 0x561624cb5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.560+0000 7f55071b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 0x561624ccbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f55071b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cbc3a0 0x561624cbeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f55071b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc1b06e0 con 0x561624cbb9e0 2024-03-21T11:27:10.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f55071b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc1b2060 con 0x561624cbc3a0 2024-03-21T11:27:10.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f55071b8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc1b0d40 con 0x561624cba810 2024-03-21T11:27:10.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f5502400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cbc3a0 0x561624cbeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f5501a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cba810 0x561624cb5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f5502400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cbc3a0 0x561624cbeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59342/0 (socket says 172.21.15.47:59342) 2024-03-21T11:27:10.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f5501a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cba810 0x561624cb5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44002/0 (socket says 172.21.15.47:44002) 2024-03-21T11:27:10.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f5502400640 1 -- 172.21.15.47:0/64573175 learned_addr learned my addr 172.21.15.47:0/64573175 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:10.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f5501000640 1 --2- 172.21.15.47:0/64573175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 0x561624ccbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f5502400640 1 -- 172.21.15.47:0/64573175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cba810 msgr2=0x561624cb5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f5502400640 1 --2- 172.21.15.47:0/64573175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cba810 0x561624cb5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f5502400640 1 -- 172.21.15.47:0/64573175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 msgr2=0x561624ccbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f5502400640 1 --2- 172.21.15.47:0/64573175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 0x561624ccbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f5502400640 1 -- 172.21.15.47:0/64573175 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561624cb02c0 con 0x561624cbc3a0 2024-03-21T11:27:10.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.561+0000 7f5501a00640 1 --2- 172.21.15.47:0/64573175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cba810 0x561624cb5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:10.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.562+0000 7f5502400640 1 --2- 172.21.15.47:0/64573175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cbc3a0 0x561624cbeb20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009be0 tx=0x7f54f4009c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fc449bf407ac281 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:10.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.562+0000 7f54ea000640 1 -- 172.21.15.47:0/64573175 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4003970 con 0x561624cbc3a0 2024-03-21T11:27:10.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.562+0000 7f54ea000640 1 -- 172.21.15.47:0/64573175 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54f4003b10 con 0x561624cbc3a0 2024-03-21T11:27:10.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.562+0000 7f54ea000640 1 -- 172.21.15.47:0/64573175 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f400f740 con 0x561624cbc3a0 2024-03-21T11:27:10.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.563+0000 7f55071b8740 1 -- 172.21.15.47:0/64573175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cbc3a0 msgr2=0x561624cbeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.563+0000 7f55071b8740 1 --2- 172.21.15.47:0/64573175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cbc3a0 0x561624cbeb20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009be0 tx=0x7f54f4009c10 comp rx=0 tx=0).stop 2024-03-21T11:27:10.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.563+0000 7f55071b8740 1 -- 172.21.15.47:0/64573175 shutdown_connections 2024-03-21T11:27:10.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.563+0000 7f55071b8740 1 --2- 172.21.15.47:0/64573175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cbc3a0 0x561624cbeb20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.563+0000 7f55071b8740 1 --2- 172.21.15.47:0/64573175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 0x561624ccbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.563+0000 7f55071b8740 1 --2- 172.21.15.47:0/64573175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cba810 0x561624cb5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.563+0000 7f55071b8740 1 -- 172.21.15.47:0/64573175 >> 172.21.15.47:0/64573175 conn(0x561624cb5c00 msgr2=0x561624cb96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:10.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.563+0000 7f55071b8740 1 -- 172.21.15.47:0/64573175 shutdown_connections 2024-03-21T11:27:10.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.563+0000 7f55071b8740 1 -- 172.21.15.47:0/64573175 wait complete. 2024-03-21T11:27:10.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.563+0000 7f55071b8740 1 -- start start 2024-03-21T11:27:10.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.564+0000 7f55071b8740 1 Processor -- start 2024-03-21T11:27:10.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.564+0000 7f55071b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cc7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.564+0000 7f55071b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 0x561624cc8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.564+0000 7f55071b8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbc3a0 0x561624ccb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.564+0000 7f55071b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc1b1a00 con 0x561624cbb9e0 2024-03-21T11:27:10.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.564+0000 7f55071b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc1b13a0 con 0x561624cba810 2024-03-21T11:27:10.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.564+0000 7f55071b8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc1ae8a0 con 0x561624cbc3a0 2024-03-21T11:27:10.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.564+0000 7f55071b8740 10 client.0 _ll_register_callbacks cb 0x7f54ec009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:10.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.564+0000 7f5501a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cc7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.564+0000 7f5502400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbc3a0 0x561624ccb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.564+0000 7f5501000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 0x561624cc8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.564+0000 7f5501a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cc7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59352/0 (socket says 172.21.15.47:59352) 2024-03-21T11:27:10.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.564+0000 7f5502400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbc3a0 0x561624ccb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44016/0 (socket says 172.21.15.47:44016) 2024-03-21T11:27:10.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.564+0000 7f5501a00640 1 -- 172.21.15.47:0/3710873599 learned_addr learned my addr 172.21.15.47:0/3710873599 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:10.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.565+0000 7f5501a00640 1 -- 172.21.15.47:0/3710873599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbc3a0 msgr2=0x561624ccb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.565+0000 7f5501a00640 1 --2- 172.21.15.47:0/3710873599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbc3a0 0x561624ccb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.565+0000 7f5501a00640 1 -- 172.21.15.47:0/3710873599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 msgr2=0x561624cc8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.565+0000 7f5501a00640 1 --2- 172.21.15.47:0/3710873599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 0x561624cc8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.565+0000 7f5501a00640 1 -- 172.21.15.47:0/3710873599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561624ccb910 con 0x561624cba810 2024-03-21T11:27:10.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.565+0000 7f5501a00640 1 --2- 172.21.15.47:0/3710873599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cc7e10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f54ec004ca0 tx=0x7f54ec004bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36ddc18856238689 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:10.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.565+0000 7f54df400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:10.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.565+0000 7f54df400640 1 -- 172.21.15.47:0/3710873599 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec03b060 con 0x561624cba810 2024-03-21T11:27:10.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.565+0000 7f54df400640 1 -- 172.21.15.47:0/3710873599 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f54ec012040 con 0x561624cba810 2024-03-21T11:27:10.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.565+0000 7f55071b8740 1 -- 172.21.15.47:0/3710873599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54f4003b10 con 0x561624cba810 2024-03-21T11:27:10.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.566+0000 7f54df400640 1 -- 172.21.15.47:0/3710873599 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54ec042050 con 0x561624cba810 2024-03-21T11:27:10.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.566+0000 7f55071b8740 10 client.26636 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:10.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.566+0000 7f55071b8740 1 -- 172.21.15.47:0/3710873599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561624d2c610 con 0x561624cba810 2024-03-21T11:27:10.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.566+0000 7f54df400640 1 -- 172.21.15.47:0/3710873599 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54f4003b10 con 0x561624cba810 2024-03-21T11:27:10.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.566+0000 7f54df400640 1 -- 172.21.15.47:0/3710873599 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f54ec027040 con 0x561624cba810 2024-03-21T11:27:10.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.566+0000 7f55071b8740 10 client.26636 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:10.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.566+0000 7f55071b8740 10 client.26636 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:27:10.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.566+0000 7f55071b8740 20 client.26636 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:10.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.566+0000 7f55071b8740 1 --2- 172.21.15.47:0/3710873599 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561624d2cf30 0x561624d2f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.566+0000 7f55071b8740 4 client.26636 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:10.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.567+0000 7f55071b8740 1 -- 172.21.15.47:0/3710873599 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561624c26b50 con 0x561624d2cf30 2024-03-21T11:27:10.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.567+0000 7f5501000640 1 --2- 172.21.15.47:0/3710873599 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561624d2cf30 0x561624d2f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.567+0000 7f5501000640 1 --2- 172.21.15.47:0/3710873599 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561624d2cf30 0x561624d2f350 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c3b5c7401784b1bd server_cookie=a850a281ea7b401 in_seq=0 out_seq=0 2024-03-21T11:27:10.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.567+0000 7f54df400640 10 client.26636 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:27:10.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.567+0000 7f54df400640 1 -- 172.21.15.47:0/3710873599 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561624c26b50 con 0x561624d2cf30 2024-03-21T11:27:10.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.567+0000 7f54df400640 10 client.26636 handle_command_reply: tid=0 2024-03-21T11:27:10.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.567+0000 7f55071b8740 1 client.26636 shutdown 2024-03-21T11:27:10.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.568+0000 7f55071b8740 1 -- 172.21.15.47:0/3710873599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 msgr2=0x561624cc7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.568+0000 7f55071b8740 1 --2- 172.21.15.47:0/3710873599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cc7e10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f54ec004ca0 tx=0x7f54ec004bd0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.568+0000 7f55071b8740 1 -- 172.21.15.47:0/3710873599 shutdown_connections 2024-03-21T11:27:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.568+0000 7f55071b8740 1 --2- 172.21.15.47:0/3710873599 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561624d2cf30 0x561624d2f350 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.568+0000 7f55071b8740 1 --2- 172.21.15.47:0/3710873599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561624cbc3a0 0x561624ccb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.568+0000 7f55071b8740 1 --2- 172.21.15.47:0/3710873599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561624cbb9e0 0x561624cc8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.568+0000 7f55071b8740 1 --2- 172.21.15.47:0/3710873599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561624cba810 0x561624cc7e10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.568+0000 7f55071b8740 1 -- 172.21.15.47:0/3710873599 >> 172.21.15.47:0/3710873599 conn(0x561624cb5c00 msgr2=0x561624cb96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.568+0000 7f54df400640 0 client.26636 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:27:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.568+0000 7f55071b8740 1 -- 172.21.15.47:0/3710873599 shutdown_connections 2024-03-21T11:27:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.568+0000 7f55071b8740 1 -- 172.21.15.47:0/3710873599 wait complete. 2024-03-21T11:27:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.568+0000 7f55071b8740 20 client.26636 trim_cache size 0 max 16384 2024-03-21T11:27:10.577 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.569+0000 7f5504400640 1 -- 172.21.15.47:0/3040272554 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54e40a2fa0 msgr2=0x7f54e40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.569+0000 7f5504400640 1 --2- 172.21.15.47:0/3040272554 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54e40a2fa0 0x7f54e40a5460 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f54ec012640 tx=0x7f54ec005f90 comp rx=0 tx=0).stop 2024-03-21T11:27:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.569+0000 7f5504400640 1 -- 172.21.15.47:0/3040272554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc0772a0 msgr2=0x7f54fc19e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.569+0000 7f5504400640 1 --2- 172.21.15.47:0/3040272554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc0772a0 0x7f54fc19e530 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f54f002c260 tx=0x7f54f002c670 comp rx=0 tx=0).stop 2024-03-21T11:27:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.569+0000 7f5504400640 1 -- 172.21.15.47:0/3040272554 shutdown_connections 2024-03-21T11:27:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.569+0000 7f5504400640 1 --2- 172.21.15.47:0/3040272554 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54e40a2fa0 0x7f54e40a5460 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.569+0000 7f5504400640 1 --2- 172.21.15.47:0/3040272554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54fc077c40 0x7f54fc1a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.569+0000 7f5504400640 1 --2- 172.21.15.47:0/3040272554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc0772a0 0x7f54fc19e530 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.569+0000 7f5504400640 1 --2- 172.21.15.47:0/3040272554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54fc076060 0x7f54fc19dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.569+0000 7f5504400640 1 -- 172.21.15.47:0/3040272554 >> 172.21.15.47:0/3040272554 conn(0x7f54fc0fee50 msgr2=0x7f54fc100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:10.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.569+0000 7f5504400640 1 -- 172.21.15.47:0/3040272554 shutdown_connections 2024-03-21T11:27:10.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.569+0000 7f5504400640 1 -- 172.21.15.47:0/3040272554 wait complete. 2024-03-21T11:27:10.591 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:10.591 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:27:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.805+0000 7f60df200640 1 -- 172.21.15.47:0/1225843147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60e0103150 msgr2=0x7f60e01099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.805+0000 7f60df200640 1 --2- 172.21.15.47:0/1225843147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60e0103150 0x7f60e01099e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009be0 tx=0x7f60d802bef0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.806+0000 7f60df200640 1 -- 172.21.15.47:0/1225843147 shutdown_connections 2024-03-21T11:27:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.806+0000 7f60df200640 1 --2- 172.21.15.47:0/1225843147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60e0103150 0x7f60e01099e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.806+0000 7f60df200640 1 --2- 172.21.15.47:0/1225843147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60e0102790 0x7f60e0102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.806+0000 7f60df200640 1 --2- 172.21.15.47:0/1225843147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60e01015a0 0x7f60e01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.806+0000 7f60df200640 1 -- 172.21.15.47:0/1225843147 >> 172.21.15.47:0/1225843147 conn(0x7f60e00fcb10 msgr2=0x7f60e00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.806+0000 7f60df200640 1 -- 172.21.15.47:0/1225843147 shutdown_connections 2024-03-21T11:27:10.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.806+0000 7f60df200640 1 -- 172.21.15.47:0/1225843147 wait complete. 2024-03-21T11:27:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.807+0000 7f60df200640 1 Processor -- start 2024-03-21T11:27:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.807+0000 7f60df200640 1 -- start start 2024-03-21T11:27:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.807+0000 7f60df200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60e01015a0 0x7f60e0075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.807+0000 7f60df200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60e0102790 0x7f60e0073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.807+0000 7f60df200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60e0103150 0x7f60e0073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.807+0000 7f60df200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e010d270 con 0x7f60e0102790 2024-03-21T11:27:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.807+0000 7f60df200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e010d0f0 con 0x7f60e0103150 2024-03-21T11:27:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.807+0000 7f60df200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e010d3f0 con 0x7f60e01015a0 2024-03-21T11:27:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.808+0000 7f60dde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60e01015a0 0x7f60e0075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.808+0000 7f60de800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60e0103150 0x7f60e0073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.808+0000 7f60dde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60e01015a0 0x7f60e0075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44034/0 (socket says 172.21.15.47:44034) 2024-03-21T11:27:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.808+0000 7f60dde00640 1 -- 172.21.15.47:0/2573671088 learned_addr learned my addr 172.21.15.47:0/2573671088 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:10.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.808+0000 7f60dd400640 1 --2- 172.21.15.47:0/2573671088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60e0102790 0x7f60e0073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.808+0000 7f60de800640 1 -- 172.21.15.47:0/2573671088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60e01015a0 msgr2=0x7f60e0075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.808+0000 7f60de800640 1 --2- 172.21.15.47:0/2573671088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60e01015a0 0x7f60e0075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.808+0000 7f60de800640 1 -- 172.21.15.47:0/2573671088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60e0102790 msgr2=0x7f60e0073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.808+0000 7f60de800640 1 --2- 172.21.15.47:0/2573671088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60e0102790 0x7f60e0073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.808+0000 7f60de800640 1 -- 172.21.15.47:0/2573671088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e0074860 con 0x7f60e0103150 2024-03-21T11:27:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.808+0000 7f60de800640 1 --2- 172.21.15.47:0/2573671088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60e0103150 0x7f60e0073fd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f60d802c430 tx=0x7f60d80337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.809+0000 7f60d6a00640 1 -- 172.21.15.47:0/2573671088 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8043020 con 0x7f60e0103150 2024-03-21T11:27:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.809+0000 7f60d6a00640 1 -- 172.21.15.47:0/2573671088 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60d802ca60 con 0x7f60e0103150 2024-03-21T11:27:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.809+0000 7f60df200640 1 -- 172.21.15.47:0/2573671088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e01a1950 con 0x7f60e0103150 2024-03-21T11:27:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.809+0000 7f60d6a00640 1 -- 172.21.15.47:0/2573671088 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8034690 con 0x7f60e0103150 2024-03-21T11:27:10.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.809+0000 7f60df200640 1 -- 172.21.15.47:0/2573671088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e01a1ea0 con 0x7f60e0103150 2024-03-21T11:27:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.812+0000 7f60e7982740 1 Processor -- start 2024-03-21T11:27:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.812+0000 7f60d6a00640 1 -- 172.21.15.47:0/2573671088 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f60d8046020 con 0x7f60e0103150 2024-03-21T11:27:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.813+0000 7f60e7982740 1 -- start start 2024-03-21T11:27:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.813+0000 7f60e7982740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab05e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.813+0000 7f60e7982740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab06bc0 0x55aefab07040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.813+0000 7f60e7982740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab07580 0x55aefab0de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.813+0000 7f60e7982740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefaa6a2c0 con 0x55aefab06bc0 2024-03-21T11:27:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.813+0000 7f60e7982740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefaa12c90 con 0x55aefab07580 2024-03-21T11:27:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.813+0000 7f60e7982740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefaa66360 con 0x55aefab059f0 2024-03-21T11:27:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.813+0000 7f60dde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab06bc0 0x55aefab07040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.813+0000 7f60dd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab05e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.813+0000 7f60dde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab06bc0 0x55aefab07040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38574/0 (socket says 172.21.15.47:38574) 2024-03-21T11:27:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.813+0000 7f60dde00640 1 -- 172.21.15.47:0/319083685 learned_addr learned my addr 172.21.15.47:0/319083685 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.814+0000 7f60dd400640 1 --2- 172.21.15.47:0/319083685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab07580 0x55aefab0de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.814+0000 7f60dde00640 1 -- 172.21.15.47:0/319083685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 msgr2=0x55aefab05e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.814+0000 7f60dde00640 1 --2- 172.21.15.47:0/319083685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab05e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.814+0000 7f60dde00640 1 -- 172.21.15.47:0/319083685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab07580 msgr2=0x55aefab0de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.814+0000 7f60dde00640 1 --2- 172.21.15.47:0/319083685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab07580 0x55aefab0de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.814+0000 7f60dde00640 1 -- 172.21.15.47:0/319083685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aefaa37080 con 0x55aefab06bc0 2024-03-21T11:27:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.814+0000 7f60d6a00640 1 --2- 172.21.15.47:0/2573671088 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60c40a2f20 0x7f60c40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.814+0000 7f60d6a00640 1 -- 172.21.15.47:0/2573671088 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f60d80f1da0 con 0x7f60e0103150 2024-03-21T11:27:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.814+0000 7f60dde00640 1 --2- 172.21.15.47:0/319083685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab06bc0 0x55aefab07040 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009d70 tx=0x7f60d0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9684e0cf81b08b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.815+0000 7f60dd400640 1 --2- 172.21.15.47:0/2573671088 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60c40a2f20 0x7f60c40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.815+0000 7f60cb000640 1 -- 172.21.15.47:0/319083685 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0004220 con 0x55aefab06bc0 2024-03-21T11:27:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.815+0000 7f60cb000640 1 -- 172.21.15.47:0/319083685 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60d00043c0 con 0x55aefab06bc0 2024-03-21T11:27:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.815+0000 7f60cb000640 1 -- 172.21.15.47:0/319083685 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d00198f0 con 0x55aefab06bc0 2024-03-21T11:27:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.815+0000 7f60e7982740 1 -- 172.21.15.47:0/319083685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab06bc0 msgr2=0x55aefab07040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.815+0000 7f60e7982740 1 --2- 172.21.15.47:0/319083685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab06bc0 0x55aefab07040 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009d70 tx=0x7f60d0011810 comp rx=0 tx=0).stop 2024-03-21T11:27:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.815+0000 7f60dd400640 1 --2- 172.21.15.47:0/2573671088 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60c40a2f20 0x7f60c40a53e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009fd0 tx=0x7f60cc004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.816+0000 7f60e7982740 1 -- 172.21.15.47:0/319083685 shutdown_connections 2024-03-21T11:27:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.816+0000 7f60e7982740 1 --2- 172.21.15.47:0/319083685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab07580 0x55aefab0de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.816+0000 7f60e7982740 1 --2- 172.21.15.47:0/319083685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab06bc0 0x55aefab07040 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.816+0000 7f60e7982740 1 --2- 172.21.15.47:0/319083685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab05e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.816+0000 7f60e7982740 1 -- 172.21.15.47:0/319083685 >> 172.21.15.47:0/319083685 conn(0x55aefab00de0 msgr2=0x55aefab011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.816+0000 7f60e7982740 1 -- 172.21.15.47:0/319083685 shutdown_connections 2024-03-21T11:27:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.816+0000 7f60e7982740 1 -- 172.21.15.47:0/319083685 wait complete. 2024-03-21T11:27:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.816+0000 7f60e7982740 1 -- start start 2024-03-21T11:27:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.816+0000 7f60e7982740 1 Processor -- start 2024-03-21T11:27:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.817+0000 7f60e7982740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab0fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.817+0000 7f60e7982740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 0x55aefab10050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.817+0000 7f60e7982740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab07580 0x55aefab17090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.817+0000 7f60e7982740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefab0e590 con 0x55aefab07580 2024-03-21T11:27:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.817+0000 7f60e7982740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefaa260d0 con 0x55aefab06bc0 2024-03-21T11:27:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.817+0000 7f60e7982740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aefab0e710 con 0x55aefab059f0 2024-03-21T11:27:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.817+0000 7f60e7982740 10 client.0 _ll_register_callbacks cb 0x55aefaa2ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.817+0000 7f60dde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab0fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.817+0000 7f60dde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab0fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44058/0 (socket says 172.21.15.47:44058) 2024-03-21T11:27:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.817+0000 7f60dde00640 1 -- 172.21.15.47:0/3925551975 learned_addr learned my addr 172.21.15.47:0/3925551975 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.817+0000 7f60dd400640 1 --2- 172.21.15.47:0/3925551975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 0x55aefab10050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.817+0000 7f60de800640 1 --2- 172.21.15.47:0/3925551975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab07580 0x55aefab17090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.818+0000 7f60dde00640 1 -- 172.21.15.47:0/3925551975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 msgr2=0x55aefab10050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.818+0000 7f60dde00640 1 --2- 172.21.15.47:0/3925551975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 0x55aefab10050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.818+0000 7f60dde00640 1 -- 172.21.15.47:0/3925551975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab07580 msgr2=0x55aefab17090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.818+0000 7f60dde00640 1 --2- 172.21.15.47:0/3925551975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab07580 0x55aefab17090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.818+0000 7f60dde00640 1 -- 172.21.15.47:0/3925551975 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aefa77e3f0 con 0x55aefab059f0 2024-03-21T11:27:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.818+0000 7f60dde00640 1 --2- 172.21.15.47:0/3925551975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab0fb10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f60d001b040 tx=0x7f60d0003880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1718f963f4d69c40 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.818+0000 7f60bf400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.818+0000 7f60bf400640 1 -- 172.21.15.47:0/3925551975 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0028070 con 0x55aefab059f0 2024-03-21T11:27:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.818+0000 7f60e7982740 1 -- 172.21.15.47:0/3925551975 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60d00043c0 con 0x55aefab059f0 2024-03-21T11:27:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.818+0000 7f60bf400640 1 -- 172.21.15.47:0/3925551975 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f60d0005040 con 0x55aefab059f0 2024-03-21T11:27:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.819+0000 7f60bf400640 1 -- 172.21.15.47:0/3925551975 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60d0034050 con 0x55aefab059f0 2024-03-21T11:27:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.819+0000 7f60e7982740 10 client.26671 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.819+0000 7f60e7982740 1 -- 172.21.15.47:0/3925551975 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55aefa8487d0 con 0x55aefab059f0 2024-03-21T11:27:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.819+0000 7f60bf400640 1 -- 172.21.15.47:0/3925551975 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60d00043c0 con 0x55aefab059f0 2024-03-21T11:27:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.819+0000 7f60bf400640 1 -- 172.21.15.47:0/3925551975 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f60d0004bb0 con 0x55aefab059f0 2024-03-21T11:27:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.819+0000 7f60e7982740 10 client.26671 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.819+0000 7f60e7982740 10 client.26671 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:27:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.819+0000 7f60e7982740 20 client.26671 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.819+0000 7f60e7982740 1 --2- 172.21.15.47:0/3925551975 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aefab1b310 0x55aefab1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.819+0000 7f60e7982740 4 client.26671 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.819+0000 7f60e7982740 1 -- 172.21.15.47:0/3925551975 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aefaa77800 con 0x55aefab1b310 2024-03-21T11:27:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.820+0000 7f60dd400640 1 --2- 172.21.15.47:0/3925551975 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aefab1b310 0x55aefab1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.820+0000 7f60dd400640 1 --2- 172.21.15.47:0/3925551975 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aefab1b310 0x55aefab1d760 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=76391311e1437ab7 server_cookie=38f35eeeffee7bcd in_seq=0 out_seq=0 2024-03-21T11:27:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.820+0000 7f60bf400640 10 client.26671 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:27:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.821+0000 7f60bf400640 1 -- 172.21.15.47:0/3925551975 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55aefaa77800 con 0x55aefab1b310 2024-03-21T11:27:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.821+0000 7f60bf400640 10 client.26671 handle_command_reply: tid=0 2024-03-21T11:27:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.821+0000 7f60e7982740 1 client.26671 shutdown 2024-03-21T11:27:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.822+0000 7f60e7982740 1 -- 172.21.15.47:0/3925551975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 msgr2=0x55aefab0fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.822+0000 7f60e7982740 1 --2- 172.21.15.47:0/3925551975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab0fb10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f60d001b040 tx=0x7f60d0003880 comp rx=0 tx=0).stop 2024-03-21T11:27:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.822+0000 7f60e7982740 1 -- 172.21.15.47:0/3925551975 shutdown_connections 2024-03-21T11:27:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.822+0000 7f60e7982740 1 --2- 172.21.15.47:0/3925551975 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aefab1b310 0x55aefab1d760 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.822+0000 7f60e7982740 1 --2- 172.21.15.47:0/3925551975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab07580 0x55aefab17090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.822+0000 7f60e7982740 1 --2- 172.21.15.47:0/3925551975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 0x55aefab10050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.822+0000 7f60e7982740 1 --2- 172.21.15.47:0/3925551975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab0fb10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.822+0000 7f60bf400640 0 client.26671 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:27:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.822+0000 7f60e7982740 1 -- 172.21.15.47:0/3925551975 >> 172.21.15.47:0/3925551975 conn(0x55aefab00de0 msgr2=0x55aefab00160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.822+0000 7f60e7982740 1 -- 172.21.15.47:0/3925551975 shutdown_connections 2024-03-21T11:27:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.822+0000 7f60e7982740 1 -- 172.21.15.47:0/3925551975 wait complete. 2024-03-21T11:27:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.822+0000 7f60e7982740 20 client.26671 trim_cache size 0 max 16384 2024-03-21T11:27:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.838+0000 7f60e7982740 1 Processor -- start 2024-03-21T11:27:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.838+0000 7f60e7982740 1 -- start start 2024-03-21T11:27:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.838+0000 7f60e7982740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab09df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.838+0000 7f60e7982740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 0x55aefab17020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.838+0000 7f60e7982740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab07580 0x55aefab0a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.838+0000 7f60e7982740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e01ac030 con 0x55aefab07580 2024-03-21T11:27:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.839+0000 7f60e7982740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e01ad9b0 con 0x55aefab06bc0 2024-03-21T11:27:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.839+0000 7f60e7982740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e01ac690 con 0x55aefab059f0 2024-03-21T11:27:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.839+0000 7f60dde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab09df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.839+0000 7f60dd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 0x55aefab17020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.839+0000 7f60de800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab07580 0x55aefab0a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.839+0000 7f60dde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab09df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44074/0 (socket says 172.21.15.47:44074) 2024-03-21T11:27:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.839+0000 7f60de800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab07580 0x55aefab0a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38578/0 (socket says 172.21.15.47:38578) 2024-03-21T11:27:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.839+0000 7f60dde00640 1 -- 172.21.15.47:0/3856795067 learned_addr learned my addr 172.21.15.47:0/3856795067 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.839+0000 7f60dd400640 1 -- 172.21.15.47:0/3856795067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 msgr2=0x55aefab09df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.839+0000 7f60dd400640 1 --2- 172.21.15.47:0/3856795067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab09df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.839+0000 7f60dd400640 1 -- 172.21.15.47:0/3856795067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab07580 msgr2=0x55aefab0a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.839+0000 7f60dd400640 1 --2- 172.21.15.47:0/3856795067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab07580 0x55aefab0a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.839+0000 7f60dd400640 1 -- 172.21.15.47:0/3856795067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aefab06640 con 0x55aefab06bc0 2024-03-21T11:27:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.840+0000 7f60dd400640 1 --2- 172.21.15.47:0/3856795067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 0x55aefab17020 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f60cc011010 tx=0x7f60cc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1342de90c96d705 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:10.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.840+0000 7f60ca600640 1 -- 172.21.15.47:0/3856795067 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc02e130 con 0x55aefab06bc0 2024-03-21T11:27:10.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.840+0000 7f60ca600640 1 -- 172.21.15.47:0/3856795067 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60cc02e2d0 con 0x55aefab06bc0 2024-03-21T11:27:10.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.840+0000 7f60ca600640 1 -- 172.21.15.47:0/3856795067 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0035f0 con 0x55aefab06bc0 2024-03-21T11:27:10.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.841+0000 7f60e7982740 1 -- 172.21.15.47:0/3856795067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 msgr2=0x55aefab17020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.841+0000 7f60e7982740 1 --2- 172.21.15.47:0/3856795067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 0x55aefab17020 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f60cc011010 tx=0x7f60cc009260 comp rx=0 tx=0).stop 2024-03-21T11:27:10.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.841+0000 7f60e7982740 1 -- 172.21.15.47:0/3856795067 shutdown_connections 2024-03-21T11:27:10.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.841+0000 7f60e7982740 1 --2- 172.21.15.47:0/3856795067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab07580 0x55aefab0a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.841+0000 7f60e7982740 1 --2- 172.21.15.47:0/3856795067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 0x55aefab17020 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.841+0000 7f60e7982740 1 --2- 172.21.15.47:0/3856795067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab059f0 0x55aefab09df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.841+0000 7f60e7982740 1 -- 172.21.15.47:0/3856795067 >> 172.21.15.47:0/3856795067 conn(0x55aefab00de0 msgr2=0x55aefab048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:10.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.841+0000 7f60e7982740 1 -- 172.21.15.47:0/3856795067 shutdown_connections 2024-03-21T11:27:10.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.841+0000 7f60e7982740 1 -- 172.21.15.47:0/3856795067 wait complete. 2024-03-21T11:27:10.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.841+0000 7f60e7982740 1 -- start start 2024-03-21T11:27:10.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.842+0000 7f60e7982740 1 Processor -- start 2024-03-21T11:27:10.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.842+0000 7f60e7982740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab059f0 0x55aefab14ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.842+0000 7f60e7982740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 0x55aefab15410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.842+0000 7f60e7982740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab07580 0x55aefab16420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.842+0000 7f60e7982740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e01ad350 con 0x55aefab059f0 2024-03-21T11:27:10.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.842+0000 7f60e7982740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e01accf0 con 0x55aefab06bc0 2024-03-21T11:27:10.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.842+0000 7f60e7982740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e01aa1f0 con 0x55aefab07580 2024-03-21T11:27:10.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.842+0000 7f60e7982740 10 client.0 _ll_register_callbacks cb 0x7f60d0004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:10.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.842+0000 7f60de800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab07580 0x55aefab16420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.842+0000 7f60dd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 0x55aefab15410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.842+0000 7f60de800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab07580 0x55aefab16420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44076/0 (socket says 172.21.15.47:44076) 2024-03-21T11:27:10.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.842+0000 7f60de800640 1 -- 172.21.15.47:0/3666007629 learned_addr learned my addr 172.21.15.47:0/3666007629 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:10.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.842+0000 7f60dde00640 1 --2- 172.21.15.47:0/3666007629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab059f0 0x55aefab14ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.843+0000 7f60de800640 1 -- 172.21.15.47:0/3666007629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 msgr2=0x55aefab15410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.843+0000 7f60de800640 1 --2- 172.21.15.47:0/3666007629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 0x55aefab15410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.843+0000 7f60de800640 1 -- 172.21.15.47:0/3666007629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab059f0 msgr2=0x55aefab14ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.843+0000 7f60de800640 1 --2- 172.21.15.47:0/3666007629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab059f0 0x55aefab14ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.843+0000 7f60de800640 1 -- 172.21.15.47:0/3666007629 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aefab16a10 con 0x55aefab07580 2024-03-21T11:27:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.843+0000 7f60dde00640 1 --2- 172.21.15.47:0/3666007629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab059f0 0x55aefab14ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.843+0000 7f60dd400640 1 --2- 172.21.15.47:0/3666007629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 0x55aefab15410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.843+0000 7f60de800640 1 --2- 172.21.15.47:0/3666007629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab07580 0x55aefab16420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f60d8033040 tx=0x7f60d8009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f6f56d8afa19ea1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.843+0000 7f60bf400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.843+0000 7f60bf400640 1 -- 172.21.15.47:0/3666007629 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8004820 con 0x55aefab07580 2024-03-21T11:27:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.843+0000 7f60bf400640 1 -- 172.21.15.47:0/3666007629 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f60d80de0c0 con 0x55aefab07580 2024-03-21T11:27:10.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.843+0000 7f60e7982740 1 -- 172.21.15.47:0/3666007629 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60cc02e2d0 con 0x55aefab07580 2024-03-21T11:27:10.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.844+0000 7f60bf400640 1 -- 172.21.15.47:0/3666007629 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60d80d7200 con 0x55aefab07580 2024-03-21T11:27:10.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.844+0000 7f60e7982740 10 client.26677 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:10.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.844+0000 7f60e7982740 1 -- 172.21.15.47:0/3666007629 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55aefab773e0 con 0x55aefab07580 2024-03-21T11:27:10.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.844+0000 7f60bf400640 1 -- 172.21.15.47:0/3666007629 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60cc02e2d0 con 0x55aefab07580 2024-03-21T11:27:10.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.844+0000 7f60bf400640 1 -- 172.21.15.47:0/3666007629 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f60d8004b40 con 0x55aefab07580 2024-03-21T11:27:10.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.844+0000 7f60e7982740 10 client.26677 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:10.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.844+0000 7f60e7982740 10 client.26677 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:27:10.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.844+0000 7f60e7982740 20 client.26677 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:10.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.844+0000 7f60e7982740 1 --2- 172.21.15.47:0/3666007629 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aefab77d30 0x55aefab7a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:10.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.844+0000 7f60e7982740 4 client.26677 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:10.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.844+0000 7f60e7982740 1 -- 172.21.15.47:0/3666007629 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aefaa71b50 con 0x55aefab77d30 2024-03-21T11:27:10.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.845+0000 7f60dde00640 1 --2- 172.21.15.47:0/3666007629 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aefab77d30 0x55aefab7a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:10.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.845+0000 7f60dde00640 1 --2- 172.21.15.47:0/3666007629 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aefab77d30 0x55aefab7a120 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b4c1eb4f7b935a0 server_cookie=92acf978e9ab69f9 in_seq=0 out_seq=0 2024-03-21T11:27:10.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.845+0000 7f60bf400640 10 client.26677 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:27:10.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.845+0000 7f60bf400640 1 -- 172.21.15.47:0/3666007629 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aefaa71b50 con 0x55aefab77d30 2024-03-21T11:27:10.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.845+0000 7f60bf400640 10 client.26677 handle_command_reply: tid=0 2024-03-21T11:27:10.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.845+0000 7f60e7982740 1 client.26677 shutdown 2024-03-21T11:27:10.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.845+0000 7f60e7982740 1 -- 172.21.15.47:0/3666007629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab07580 msgr2=0x55aefab16420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.845+0000 7f60e7982740 1 --2- 172.21.15.47:0/3666007629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab07580 0x55aefab16420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f60d8033040 tx=0x7f60d8009500 comp rx=0 tx=0).stop 2024-03-21T11:27:10.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.846+0000 7f60e7982740 1 -- 172.21.15.47:0/3666007629 shutdown_connections 2024-03-21T11:27:10.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.846+0000 7f60e7982740 1 --2- 172.21.15.47:0/3666007629 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aefab77d30 0x55aefab7a120 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.846+0000 7f60e7982740 1 --2- 172.21.15.47:0/3666007629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aefab07580 0x55aefab16420 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.846+0000 7f60e7982740 1 --2- 172.21.15.47:0/3666007629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aefab06bc0 0x55aefab15410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.846+0000 7f60e7982740 1 --2- 172.21.15.47:0/3666007629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aefab059f0 0x55aefab14ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.846+0000 7f60bf400640 0 client.26677 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:27:10.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.846+0000 7f60e7982740 1 -- 172.21.15.47:0/3666007629 >> 172.21.15.47:0/3666007629 conn(0x55aefab00de0 msgr2=0x55aefaaff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:10.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.846+0000 7f60e7982740 1 -- 172.21.15.47:0/3666007629 shutdown_connections 2024-03-21T11:27:10.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.846+0000 7f60e7982740 1 -- 172.21.15.47:0/3666007629 wait complete. 2024-03-21T11:27:10.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.846+0000 7f60e7982740 20 client.26677 trim_cache size 0 max 16384 2024-03-21T11:27:10.854 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:10.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.847+0000 7f60df200640 1 -- 172.21.15.47:0/2573671088 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60c40a2f20 msgr2=0x7f60c40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.847+0000 7f60df200640 1 --2- 172.21.15.47:0/2573671088 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60c40a2f20 0x7f60c40a53e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009fd0 tx=0x7f60cc004730 comp rx=0 tx=0).stop 2024-03-21T11:27:10.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.847+0000 7f60df200640 1 -- 172.21.15.47:0/2573671088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60e0103150 msgr2=0x7f60e0073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:10.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.847+0000 7f60df200640 1 --2- 172.21.15.47:0/2573671088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60e0103150 0x7f60e0073fd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f60d802c430 tx=0x7f60d80337b0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.847+0000 7f60df200640 1 -- 172.21.15.47:0/2573671088 shutdown_connections 2024-03-21T11:27:10.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.847+0000 7f60df200640 1 --2- 172.21.15.47:0/2573671088 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60c40a2f20 0x7f60c40a53e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.847+0000 7f60df200640 1 --2- 172.21.15.47:0/2573671088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60e0103150 0x7f60e0073fd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.847+0000 7f60df200640 1 --2- 172.21.15.47:0/2573671088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60e0102790 0x7f60e0073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.847+0000 7f60df200640 1 --2- 172.21.15.47:0/2573671088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60e01015a0 0x7f60e0075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:10.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.847+0000 7f60df200640 1 -- 172.21.15.47:0/2573671088 >> 172.21.15.47:0/2573671088 conn(0x7f60e00fcb10 msgr2=0x7f60e0105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:10.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.847+0000 7f60df200640 1 -- 172.21.15.47:0/2573671088 shutdown_connections 2024-03-21T11:27:10.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:10.847+0000 7f60df200640 1 -- 172.21.15.47:0/2573671088 wait complete. 2024-03-21T11:27:10.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:10 smithi047 ceph-mon[31351]: pgmap v515: 129 pgs: 129 active+clean; 37 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-03-21T11:27:10.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:10 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3059985367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:27:10.865 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:10.866 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:27:10.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:10 smithi100 ceph-mon[36245]: pgmap v515: 129 pgs: 129 active+clean; 37 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-03-21T11:27:10.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:10 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3059985367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:27:10.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:10 smithi102 ceph-mon[36748]: pgmap v515: 129 pgs: 129 active+clean; 37 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-03-21T11:27:10.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:10 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3059985367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:27:11.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.131+0000 7fb9c7600640 1 -- 172.21.15.47:0/32024729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9c0107380 msgr2=0x7fb9c01097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.131+0000 7fb9c7600640 1 --2- 172.21.15.47:0/32024729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9c0107380 0x7fb9c01097d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009be0 tx=0x7fb9b802bef0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.132+0000 7fb9c7600640 1 -- 172.21.15.47:0/32024729 shutdown_connections 2024-03-21T11:27:11.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.132+0000 7fb9c7600640 1 --2- 172.21.15.47:0/32024729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9c0107380 0x7fb9c01097d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.132+0000 7fb9c7600640 1 --2- 172.21.15.47:0/32024729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c0104a50 0x7fb9c0106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.132+0000 7fb9c7600640 1 --2- 172.21.15.47:0/32024729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9c0102120 0x7fb9c0104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.132+0000 7fb9c7600640 1 -- 172.21.15.47:0/32024729 >> 172.21.15.47:0/32024729 conn(0x7fb9c00fbb30 msgr2=0x7fb9c00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.132+0000 7fb9c7600640 1 -- 172.21.15.47:0/32024729 shutdown_connections 2024-03-21T11:27:11.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.132+0000 7fb9c7600640 1 -- 172.21.15.47:0/32024729 wait complete. 2024-03-21T11:27:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.133+0000 7fb9c7600640 1 Processor -- start 2024-03-21T11:27:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.133+0000 7fb9c7600640 1 -- start start 2024-03-21T11:27:11.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.134+0000 7fb9c7600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c0102120 0x7fb9c0110a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.134+0000 7fb9c7600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9c0104a50 0x7fb9c0110fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.134+0000 7fb9c7600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9c0107380 0x7fb9c01ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.134+0000 7fb9c7600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0101a30 con 0x7fb9c0107380 2024-03-21T11:27:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.134+0000 7fb9c7600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c01018b0 con 0x7fb9c0102120 2024-03-21T11:27:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.134+0000 7fb9c7600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0101bb0 con 0x7fb9c0104a50 2024-03-21T11:27:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.135+0000 7fb9c4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c0102120 0x7fb9c0110a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.135+0000 7fb9bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9c0104a50 0x7fb9c0110fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.135+0000 7fb9bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9c0104a50 0x7fb9c0110fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44106/0 (socket says 172.21.15.47:44106) 2024-03-21T11:27:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.135+0000 7fb9bfe00640 1 -- 172.21.15.47:0/4072116518 learned_addr learned my addr 172.21.15.47:0/4072116518 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.135+0000 7fb9bfe00640 1 -- 172.21.15.47:0/4072116518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c0102120 msgr2=0x7fb9c0110a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.135+0000 7fb9c5600640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9c0107380 0x7fb9c01ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.135+0000 7fb9bfe00640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c0102120 0x7fb9c0110a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.135+0000 7fb9bfe00640 1 -- 172.21.15.47:0/4072116518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9c0107380 msgr2=0x7fb9c01ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.135+0000 7fb9bfe00640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9c0107380 0x7fb9c01ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.135+0000 7fb9bfe00640 1 -- 172.21.15.47:0/4072116518 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c0118150 con 0x7fb9c0104a50 2024-03-21T11:27:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.135+0000 7fb9c4c00640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c0102120 0x7fb9c0110a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.136+0000 7fb9c5600640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9c0107380 0x7fb9c01ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:27:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.136+0000 7fb9bfe00640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9c0104a50 0x7fb9c0110fb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac005fd0 tx=0x7fb9ac00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.136+0000 7fb9bd600640 1 -- 172.21.15.47:0/4072116518 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac017070 con 0x7fb9c0104a50 2024-03-21T11:27:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.136+0000 7fb9bd600640 1 -- 172.21.15.47:0/4072116518 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9ac009c30 con 0x7fb9c0104a50 2024-03-21T11:27:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.136+0000 7fb9bd600640 1 -- 172.21.15.47:0/4072116518 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac010460 con 0x7fb9c0104a50 2024-03-21T11:27:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.136+0000 7fb9c7600640 1 -- 172.21.15.47:0/4072116518 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c01ae6d0 con 0x7fb9c0104a50 2024-03-21T11:27:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.136+0000 7fb9c7600640 1 -- 172.21.15.47:0/4072116518 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c01aeba0 con 0x7fb9c0104a50 2024-03-21T11:27:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.141+0000 7fb9ca3cc740 1 Processor -- start 2024-03-21T11:27:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.141+0000 7fb9bd600640 1 -- 172.21.15.47:0/4072116518 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb9ac003e70 con 0x7fb9c0104a50 2024-03-21T11:27:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.141+0000 7fb9ca3cc740 1 -- start start 2024-03-21T11:27:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.141+0000 7fb9ca3cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3b810 0x556408f3bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9ca3cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3c9e0 0x556408f3ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9ca3cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f43c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9ca3cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556408ea02c0 con 0x556408f3c9e0 2024-03-21T11:27:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9ca3cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556408e48c90 con 0x556408f3d3a0 2024-03-21T11:27:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9ca3cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556408e9c360 con 0x556408f3b810 2024-03-21T11:27:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9c5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3b810 0x556408f3bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9c5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3b810 0x556408f3bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44112/0 (socket says 172.21.15.47:44112) 2024-03-21T11:27:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9c5600640 1 -- 172.21.15.47:0/4087885505 learned_addr learned my addr 172.21.15.47:0/4087885505 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9bfe00640 1 --2- 172.21.15.47:0/4087885505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f43c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9c4c00640 1 --2- 172.21.15.47:0/4087885505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3c9e0 0x556408f3ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9c5600640 1 -- 172.21.15.47:0/4087885505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 msgr2=0x556408f43c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9c5600640 1 --2- 172.21.15.47:0/4087885505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f43c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9c5600640 1 -- 172.21.15.47:0/4087885505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3c9e0 msgr2=0x556408f3ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9c5600640 1 --2- 172.21.15.47:0/4087885505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3c9e0 0x556408f3ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9c5600640 1 -- 172.21.15.47:0/4087885505 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556408e6d080 con 0x556408f3b810 2024-03-21T11:27:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9c4c00640 1 --2- 172.21.15.47:0/4087885505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3c9e0 0x556408f3ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.142+0000 7fb9bd600640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9a80a2fa0 0x7fb9a80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.143+0000 7fb9bd600640 1 -- 172.21.15.47:0/4072116518 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb9ac0cf470 con 0x7fb9c0104a50 2024-03-21T11:27:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.143+0000 7fb9c5600640 1 --2- 172.21.15.47:0/4087885505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3b810 0x556408f3bc30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8004af0 tx=0x7fb9b8033b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a44e8ec0b1c47adf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.143+0000 7fb9b1c00640 1 -- 172.21.15.47:0/4087885505 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8042070 con 0x556408f3b810 2024-03-21T11:27:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.143+0000 7fb9b1c00640 1 -- 172.21.15.47:0/4087885505 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9b8003970 con 0x556408f3b810 2024-03-21T11:27:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.143+0000 7fb9b1c00640 1 -- 172.21.15.47:0/4087885505 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b802ce30 con 0x556408f3b810 2024-03-21T11:27:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.143+0000 7fb9c4c00640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9a80a2fa0 0x7fb9a80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.144+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/4087885505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3b810 msgr2=0x556408f3bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.144+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/4087885505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3b810 0x556408f3bc30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8004af0 tx=0x7fb9b8033b40 comp rx=0 tx=0).stop 2024-03-21T11:27:11.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.144+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/4087885505 shutdown_connections 2024-03-21T11:27:11.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.144+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/4087885505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f43c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.144+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/4087885505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3c9e0 0x556408f3ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.144+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/4087885505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3b810 0x556408f3bc30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.144+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/4087885505 >> 172.21.15.47:0/4087885505 conn(0x556408f36c00 msgr2=0x556408f37000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.144+0000 7fb9c4c00640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9a80a2fa0 0x7fb9a80a5460 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40058c0 tx=0x7fb9b4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.144+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/4087885505 shutdown_connections 2024-03-21T11:27:11.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.144+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/4087885505 wait complete. 2024-03-21T11:27:11.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.145+0000 7fb9ca3cc740 1 -- start start 2024-03-21T11:27:11.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.145+0000 7fb9ca3cc740 1 Processor -- start 2024-03-21T11:27:11.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9ca3cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3b810 0x556408f45ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9ca3cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3c9e0 0x556408f45ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9ca3cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f4cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9ca3cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556408f44350 con 0x556408f3c9e0 2024-03-21T11:27:11.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9ca3cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556408e5c0d0 con 0x556408f3d3a0 2024-03-21T11:27:11.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9ca3cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556408f444d0 con 0x556408f3b810 2024-03-21T11:27:11.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9ca3cc740 10 client.0 _ll_register_callbacks cb 0x556408e60e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:11.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9bfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3c9e0 0x556408f45ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9bfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3c9e0 0x556408f45ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38618/0 (socket says 172.21.15.47:38618) 2024-03-21T11:27:11.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9bfe00640 1 -- 172.21.15.47:0/3796431729 learned_addr learned my addr 172.21.15.47:0/3796431729 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9c5600640 1 --2- 172.21.15.47:0/3796431729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f4cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9c4c00640 1 --2- 172.21.15.47:0/3796431729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3b810 0x556408f45ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9bfe00640 1 -- 172.21.15.47:0/3796431729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3b810 msgr2=0x556408f45ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9bfe00640 1 --2- 172.21.15.47:0/3796431729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3b810 0x556408f45ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9bfe00640 1 -- 172.21.15.47:0/3796431729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 msgr2=0x556408f4cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9bfe00640 1 --2- 172.21.15.47:0/3796431729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f4cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9bfe00640 1 -- 172.21.15.47:0/3796431729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556408bb43f0 con 0x556408f3c9e0 2024-03-21T11:27:11.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.146+0000 7fb9c5600640 1 --2- 172.21.15.47:0/3796431729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f4cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:11.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.147+0000 7fb9bfe00640 1 --2- 172.21.15.47:0/3796431729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3c9e0 0x556408f45ff0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0a3c90 tx=0x7fb9ac09b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237fadd618dd6992 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.147+0000 7fb9a6000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:27:11.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.147+0000 7fb9a6000640 1 -- 172.21.15.47:0/3796431729 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac09beb0 con 0x556408f3c9e0 2024-03-21T11:27:11.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.147+0000 7fb9a6000640 1 -- 172.21.15.47:0/3796431729 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb9ac097e20 con 0x556408f3c9e0 2024-03-21T11:27:11.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.147+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/3796431729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9b8003970 con 0x556408f3c9e0 2024-03-21T11:27:11.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.147+0000 7fb9a6000640 1 -- 172.21.15.47:0/3796431729 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9ac009de0 con 0x556408f3c9e0 2024-03-21T11:27:11.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.148+0000 7fb9ca3cc740 10 client.16449 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:11.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.148+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/3796431729 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556408c7e7d0 con 0x556408f3c9e0 2024-03-21T11:27:11.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.148+0000 7fb9a6000640 1 -- 172.21.15.47:0/3796431729 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9b8003970 con 0x556408f3c9e0 2024-03-21T11:27:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.148+0000 7fb9a6000640 1 -- 172.21.15.47:0/3796431729 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb9ac0cf0c0 con 0x556408f3c9e0 2024-03-21T11:27:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.148+0000 7fb9ca3cc740 10 client.16449 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.148+0000 7fb9ca3cc740 10 client.16449 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:27:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.148+0000 7fb9ca3cc740 20 client.16449 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.148+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/3796431729 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x556408f514d0 0x556408f538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.148+0000 7fb9ca3cc740 4 client.16449 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.148+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/3796431729 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556408ead800 con 0x556408f514d0 2024-03-21T11:27:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.149+0000 7fb9c4c00640 1 --2- 172.21.15.47:0/3796431729 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x556408f514d0 0x556408f538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.149+0000 7fb9c4c00640 1 --2- 172.21.15.47:0/3796431729 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x556408f514d0 0x556408f538e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=286a4b8893615bb6 server_cookie=b22a057700f6cde in_seq=0 out_seq=0 2024-03-21T11:27:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.149+0000 7fb9a6000640 10 client.16449 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:27:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.150+0000 7fb9a6000640 1 -- 172.21.15.47:0/3796431729 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556408ead800 con 0x556408f514d0 2024-03-21T11:27:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.150+0000 7fb9a6000640 10 client.16449 handle_command_reply: tid=0 2024-03-21T11:27:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.151+0000 7fb9ca3cc740 1 client.16449 shutdown 2024-03-21T11:27:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.151+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/3796431729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3c9e0 msgr2=0x556408f45ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.151+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/3796431729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3c9e0 0x556408f45ff0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0a3c90 tx=0x7fb9ac09b580 comp rx=0 tx=0).stop 2024-03-21T11:27:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.151+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/3796431729 shutdown_connections 2024-03-21T11:27:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.151+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/3796431729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f4cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.151+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/3796431729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3c9e0 0x556408f45ff0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.151+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/3796431729 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x556408f514d0 0x556408f538e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.151+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/3796431729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3b810 0x556408f45ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.151+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/3796431729 >> 172.21.15.47:0/3796431729 conn(0x556408f36c00 msgr2=0x556408f35eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.151+0000 7fb9a6000640 0 client.16449 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:27:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.151+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/3796431729 shutdown_connections 2024-03-21T11:27:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.151+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/3796431729 wait complete. 2024-03-21T11:27:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.151+0000 7fb9ca3cc740 20 client.16449 trim_cache size 0 max 16384 2024-03-21T11:27:11.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.166+0000 7fb9ca3cc740 1 Processor -- start 2024-03-21T11:27:11.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.166+0000 7fb9ca3cc740 1 -- start start 2024-03-21T11:27:11.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.166+0000 7fb9ca3cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3b810 0x556408f3a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.166+0000 7fb9ca3cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3c9e0 0x556408f4cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.166+0000 7fb9ca3cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f3fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.166+0000 7fb9ca3cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c01b8f00 con 0x556408f3b810 2024-03-21T11:27:11.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.166+0000 7fb9ca3cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c01ba880 con 0x556408f3d3a0 2024-03-21T11:27:11.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.166+0000 7fb9ca3cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c01b9560 con 0x556408f3c9e0 2024-03-21T11:27:11.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.167+0000 7fb9c5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f3fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.167+0000 7fb9c4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3b810 0x556408f3a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.167+0000 7fb9bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3c9e0 0x556408f4cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.167+0000 7fb9c5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f3fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59480/0 (socket says 172.21.15.47:59480) 2024-03-21T11:27:11.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.167+0000 7fb9bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3c9e0 0x556408f4cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44138/0 (socket says 172.21.15.47:44138) 2024-03-21T11:27:11.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.167+0000 7fb9c5600640 1 -- 172.21.15.47:0/1487791530 learned_addr learned my addr 172.21.15.47:0/1487791530 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.167+0000 7fb9c5600640 1 -- 172.21.15.47:0/1487791530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3c9e0 msgr2=0x556408f4cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.167+0000 7fb9c5600640 1 --2- 172.21.15.47:0/1487791530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3c9e0 0x556408f4cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.167+0000 7fb9c5600640 1 -- 172.21.15.47:0/1487791530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3b810 msgr2=0x556408f3a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.167+0000 7fb9c5600640 1 --2- 172.21.15.47:0/1487791530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3b810 0x556408f3a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.167+0000 7fb9c5600640 1 -- 172.21.15.47:0/1487791530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556408f37970 con 0x556408f3d3a0 2024-03-21T11:27:11.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.167+0000 7fb9bfe00640 1 --2- 172.21.15.47:0/1487791530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3c9e0 0x556408f4cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:11.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.168+0000 7fb9c5600640 1 --2- 172.21.15.47:0/1487791530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f3fac0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009d00 tx=0x7fb9b802bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53cf84b1eae4ce05 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.168+0000 7fb9b1200640 1 -- 172.21.15.47:0/1487791530 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8042040 con 0x556408f3d3a0 2024-03-21T11:27:11.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.168+0000 7fb9b1200640 1 -- 172.21.15.47:0/1487791530 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9b804b070 con 0x556408f3d3a0 2024-03-21T11:27:11.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.168+0000 7fb9b1200640 1 -- 172.21.15.47:0/1487791530 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8003e00 con 0x556408f3d3a0 2024-03-21T11:27:11.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.169+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/1487791530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 msgr2=0x556408f3fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.169+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/1487791530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f3fac0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009d00 tx=0x7fb9b802bef0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.169+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/1487791530 shutdown_connections 2024-03-21T11:27:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.169+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/1487791530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3d3a0 0x556408f3fac0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.169+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/1487791530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3c9e0 0x556408f4cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.169+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/1487791530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3b810 0x556408f3a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.169+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/1487791530 >> 172.21.15.47:0/1487791530 conn(0x556408f36c00 msgr2=0x556408f39e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.169+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/1487791530 shutdown_connections 2024-03-21T11:27:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.169+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/1487791530 wait complete. 2024-03-21T11:27:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.170+0000 7fb9ca3cc740 1 -- start start 2024-03-21T11:27:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.170+0000 7fb9ca3cc740 1 Processor -- start 2024-03-21T11:27:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.170+0000 7fb9ca3cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3b810 0x556408f48d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.170+0000 7fb9ca3cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3c9e0 0x556408f492a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.170+0000 7fb9ca3cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3d3a0 0x556408f4c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.170+0000 7fb9ca3cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c01ba220 con 0x556408f3b810 2024-03-21T11:27:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.170+0000 7fb9ca3cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c01b9bc0 con 0x556408f3c9e0 2024-03-21T11:27:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.170+0000 7fb9ca3cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c01b70c0 con 0x556408f3d3a0 2024-03-21T11:27:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.170+0000 7fb9ca3cc740 10 client.0 _ll_register_callbacks cb 0x7fb9ac00df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.170+0000 7fb9c5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3d3a0 0x556408f4c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.171+0000 7fb9c5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3d3a0 0x556408f4c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44142/0 (socket says 172.21.15.47:44142) 2024-03-21T11:27:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.171+0000 7fb9c5600640 1 -- 172.21.15.47:0/1779487036 learned_addr learned my addr 172.21.15.47:0/1779487036 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.171+0000 7fb9bfe00640 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3c9e0 0x556408f492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.171+0000 7fb9c4c00640 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3b810 0x556408f48d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.171+0000 7fb9c5600640 1 -- 172.21.15.47:0/1779487036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3c9e0 msgr2=0x556408f492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.171+0000 7fb9c5600640 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3c9e0 0x556408f492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.171+0000 7fb9c5600640 1 -- 172.21.15.47:0/1779487036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3b810 msgr2=0x556408f48d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.171+0000 7fb9c5600640 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3b810 0x556408f48d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.171+0000 7fb9c5600640 1 -- 172.21.15.47:0/1779487036 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556408f4c8b0 con 0x556408f3d3a0 2024-03-21T11:27:11.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.171+0000 7fb9bfe00640 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3c9e0 0x556408f492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.171+0000 7fb9c4c00640 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3b810 0x556408f48d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:27:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.171+0000 7fb9c5600640 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3d3a0 0x556408f4c2c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80099a0 tx=0x7fb9b8033b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb2d6a18d8700f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.171+0000 7fb9a6000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.172+0000 7fb9a6000640 1 -- 172.21.15.47:0/1779487036 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8042030 con 0x556408f3d3a0 2024-03-21T11:27:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.172+0000 7fb9a6000640 1 -- 172.21.15.47:0/1779487036 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb9b8034430 con 0x556408f3d3a0 2024-03-21T11:27:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.172+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/1779487036 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9b804b070 con 0x556408f3d3a0 2024-03-21T11:27:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.172+0000 7fb9a6000640 1 -- 172.21.15.47:0/1779487036 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9b8060050 con 0x556408f3d3a0 2024-03-21T11:27:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.172+0000 7fb9ca3cc740 10 client.26695 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.172+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/1779487036 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556408fad3d0 con 0x556408f3d3a0 2024-03-21T11:27:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.172+0000 7fb9a6000640 1 -- 172.21.15.47:0/1779487036 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9b804b070 con 0x556408f3d3a0 2024-03-21T11:27:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.172+0000 7fb9a6000640 1 -- 172.21.15.47:0/1779487036 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb9b8047860 con 0x556408f3d3a0 2024-03-21T11:27:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.172+0000 7fb9ca3cc740 10 client.26695 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:11.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.172+0000 7fb9ca3cc740 10 client.26695 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:27:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.172+0000 7fb9ca3cc740 20 client.26695 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.172+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x556408fad8c0 0x556408fafce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.172+0000 7fb9ca3cc740 4 client.26695 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.172+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/1779487036 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556408ea7b50 con 0x556408fad8c0 2024-03-21T11:27:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.173+0000 7fb9c4c00640 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x556408fad8c0 0x556408fafce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.173+0000 7fb9c4c00640 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x556408fad8c0 0x556408fafce0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f7084fe4a96679f server_cookie=9b66fd5f9f6fc8b3 in_seq=0 out_seq=0 2024-03-21T11:27:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.173+0000 7fb9a6000640 10 client.26695 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:27:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.173+0000 7fb9a6000640 1 -- 172.21.15.47:0/1779487036 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556408ea7b50 con 0x556408fad8c0 2024-03-21T11:27:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.173+0000 7fb9a6000640 10 client.26695 handle_command_reply: tid=0 2024-03-21T11:27:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.173+0000 7fb9ca3cc740 1 client.26695 shutdown 2024-03-21T11:27:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.174+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/1779487036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3d3a0 msgr2=0x556408f4c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.174+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3d3a0 0x556408f4c2c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80099a0 tx=0x7fb9b8033b40 comp rx=0 tx=0).stop 2024-03-21T11:27:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.174+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/1779487036 shutdown_connections 2024-03-21T11:27:11.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.174+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x556408fad8c0 0x556408fafce0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.174+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556408f3d3a0 0x556408f4c2c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.174+0000 7fb9a6000640 0 client.26695 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:27:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.174+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556408f3c9e0 0x556408f492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.174+0000 7fb9ca3cc740 1 --2- 172.21.15.47:0/1779487036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556408f3b810 0x556408f48d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.174+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/1779487036 >> 172.21.15.47:0/1779487036 conn(0x556408f36c00 msgr2=0x556408f3a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.174+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/1779487036 shutdown_connections 2024-03-21T11:27:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.174+0000 7fb9ca3cc740 1 -- 172.21.15.47:0/1779487036 wait complete. 2024-03-21T11:27:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.174+0000 7fb9ca3cc740 20 client.26695 trim_cache size 0 max 16384 2024-03-21T11:27:11.179 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.175+0000 7fb9c7600640 1 -- 172.21.15.47:0/4072116518 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9a80a2fa0 msgr2=0x7fb9a80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.175+0000 7fb9c7600640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9a80a2fa0 0x7fb9a80a5460 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40058c0 tx=0x7fb9b4005850 comp rx=0 tx=0).stop 2024-03-21T11:27:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.175+0000 7fb9c7600640 1 -- 172.21.15.47:0/4072116518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9c0104a50 msgr2=0x7fb9c0110fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.175+0000 7fb9c7600640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9c0104a50 0x7fb9c0110fb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac005fd0 tx=0x7fb9ac00dde0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.175+0000 7fb9c7600640 1 -- 172.21.15.47:0/4072116518 shutdown_connections 2024-03-21T11:27:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.175+0000 7fb9c7600640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9a80a2fa0 0x7fb9a80a5460 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.175+0000 7fb9c7600640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9c0107380 0x7fb9c01ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.175+0000 7fb9c7600640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9c0104a50 0x7fb9c0110fb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.175+0000 7fb9c7600640 1 --2- 172.21.15.47:0/4072116518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c0102120 0x7fb9c0110a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.175+0000 7fb9c7600640 1 -- 172.21.15.47:0/4072116518 >> 172.21.15.47:0/4072116518 conn(0x7fb9c00fbb30 msgr2=0x7fb9c0102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.175+0000 7fb9c7600640 1 -- 172.21.15.47:0/4072116518 shutdown_connections 2024-03-21T11:27:11.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.175+0000 7fb9c7600640 1 -- 172.21.15.47:0/4072116518 wait complete. 2024-03-21T11:27:11.194 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:11.194 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:27:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.398+0000 7f6663a00640 1 -- 172.21.15.47:0/1063635979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f665c1038c0 msgr2=0x7f665c103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.398+0000 7f6663a00640 1 --2- 172.21.15.47:0/1063635979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f665c1038c0 0x7f665c103ce0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6650009900 tx=0x7f665002bc50 comp rx=0 tx=0).stop 2024-03-21T11:27:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.398+0000 7f6663a00640 1 -- 172.21.15.47:0/1063635979 shutdown_connections 2024-03-21T11:27:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.398+0000 7f6663a00640 1 --2- 172.21.15.47:0/1063635979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f665c105470 0x7f665c10bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.398+0000 7f6663a00640 1 --2- 172.21.15.47:0/1063635979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c104ab0 0x7f665c104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.398+0000 7f6663a00640 1 --2- 172.21.15.47:0/1063635979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f665c1038c0 0x7f665c103ce0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.398+0000 7f6663a00640 1 -- 172.21.15.47:0/1063635979 >> 172.21.15.47:0/1063635979 conn(0x7f665c0fee50 msgr2=0x7f665c101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.398+0000 7f6663a00640 1 -- 172.21.15.47:0/1063635979 shutdown_connections 2024-03-21T11:27:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.398+0000 7f6663a00640 1 -- 172.21.15.47:0/1063635979 wait complete. 2024-03-21T11:27:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.399+0000 7f6663a00640 1 Processor -- start 2024-03-21T11:27:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.399+0000 7f6663a00640 1 -- start start 2024-03-21T11:27:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f6663a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f665c1038c0 0x7f665c199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f6663a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f665c104ab0 0x7f665c19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f6663a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c105470 0x7f665c1a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f6663a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c10f560 con 0x7f665c1038c0 2024-03-21T11:27:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f6663a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c10f3e0 con 0x7f665c105470 2024-03-21T11:27:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f6663a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c10f6e0 con 0x7f665c104ab0 2024-03-21T11:27:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f665be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f665c104ab0 0x7f665c19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f6661000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f665c1038c0 0x7f665c199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f6661a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c105470 0x7f665c1a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f665be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f665c104ab0 0x7f665c19a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44158/0 (socket says 172.21.15.47:44158) 2024-03-21T11:27:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f665be00640 1 -- 172.21.15.47:0/165125737 learned_addr learned my addr 172.21.15.47:0/165125737 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f665be00640 1 -- 172.21.15.47:0/165125737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c105470 msgr2=0x7f665c1a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f665be00640 1 --2- 172.21.15.47:0/165125737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c105470 0x7f665c1a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f665be00640 1 -- 172.21.15.47:0/165125737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f665c1038c0 msgr2=0x7f665c199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f665be00640 1 --2- 172.21.15.47:0/165125737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f665c1038c0 0x7f665c199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f665be00640 1 -- 172.21.15.47:0/165125737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c1a17f0 con 0x7f665c104ab0 2024-03-21T11:27:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.400+0000 7f6661a00640 1 --2- 172.21.15.47:0/165125737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c105470 0x7f665c1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:11.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.401+0000 7f665be00640 1 --2- 172.21.15.47:0/165125737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f665c104ab0 0x7f665c19a110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f664800e9f0 tx=0x7f664800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.401+0000 7f6659600640 1 -- 172.21.15.47:0/165125737 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664800cd80 con 0x7f665c104ab0 2024-03-21T11:27:11.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.401+0000 7f6659600640 1 -- 172.21.15.47:0/165125737 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6648010910 con 0x7f665c104ab0 2024-03-21T11:27:11.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.401+0000 7f6663a00640 1 -- 172.21.15.47:0/165125737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c1a1ae0 con 0x7f665c104ab0 2024-03-21T11:27:11.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.401+0000 7f6659600640 1 -- 172.21.15.47:0/165125737 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648018990 con 0x7f665c104ab0 2024-03-21T11:27:11.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.401+0000 7f6663a00640 1 -- 172.21.15.47:0/165125737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665c1a2080 con 0x7f665c104ab0 2024-03-21T11:27:11.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.405+0000 7f6666766740 1 Processor -- start 2024-03-21T11:27:11.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.405+0000 7f6659600640 1 -- 172.21.15.47:0/165125737 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6648010ac0 con 0x7f665c104ab0 2024-03-21T11:27:11.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.405+0000 7f6666766740 1 -- start start 2024-03-21T11:27:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.405+0000 7f6666766740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735d6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.405+0000 7f6666766740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d79e0 0x55ed735d7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.405+0000 7f6666766740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d83a0 0x55ed735dec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.405+0000 7f6666766740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed7353b2c0 con 0x55ed735d83a0 2024-03-21T11:27:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.405+0000 7f6666766740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed734e3c90 con 0x55ed735d79e0 2024-03-21T11:27:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.405+0000 7f6666766740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed73537360 con 0x55ed735d6810 2024-03-21T11:27:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.406+0000 7f6661a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735d6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.406+0000 7f6661a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735d6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44174/0 (socket says 172.21.15.47:44174) 2024-03-21T11:27:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.406+0000 7f6661000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d79e0 0x55ed735d7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.406+0000 7f6661a00640 1 -- 172.21.15.47:0/3507970610 learned_addr learned my addr 172.21.15.47:0/3507970610 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.406+0000 7f665be00640 1 --2- 172.21.15.47:0/3507970610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d83a0 0x55ed735dec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.406+0000 7f6661a00640 1 -- 172.21.15.47:0/3507970610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d79e0 msgr2=0x55ed735d7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.406+0000 7f6661a00640 1 --2- 172.21.15.47:0/3507970610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d79e0 0x55ed735d7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.406+0000 7f6661a00640 1 -- 172.21.15.47:0/3507970610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d83a0 msgr2=0x55ed735dec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.406+0000 7f6661a00640 1 --2- 172.21.15.47:0/3507970610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d83a0 0x55ed735dec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.406+0000 7f6661a00640 1 -- 172.21.15.47:0/3507970610 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed73508080 con 0x55ed735d6810 2024-03-21T11:27:11.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.406+0000 7f6661a00640 1 --2- 172.21.15.47:0/3507970610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735d6c30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f66540117e0 tx=0x7f6654011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=585128bc0f4f5b88 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.407+0000 7f664e000640 1 -- 172.21.15.47:0/3507970610 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654011310 con 0x55ed735d6810 2024-03-21T11:27:11.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.407+0000 7f664e000640 1 -- 172.21.15.47:0/3507970610 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6654003e50 con 0x55ed735d6810 2024-03-21T11:27:11.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.407+0000 7f664e000640 1 -- 172.21.15.47:0/3507970610 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654004620 con 0x55ed735d6810 2024-03-21T11:27:11.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.407+0000 7f6659600640 1 --2- 172.21.15.47:0/165125737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f66440a2fa0 0x7f66440a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.407+0000 7f6661000640 1 --2- 172.21.15.47:0/165125737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f66440a2fa0 0x7f66440a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.407+0000 7f6659600640 1 -- 172.21.15.47:0/165125737 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f66480cf9e0 con 0x7f665c104ab0 2024-03-21T11:27:11.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.407+0000 7f6666766740 1 -- 172.21.15.47:0/3507970610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 msgr2=0x55ed735d6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.407+0000 7f6666766740 1 --2- 172.21.15.47:0/3507970610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735d6c30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f66540117e0 tx=0x7f6654011ca0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.408+0000 7f6666766740 1 -- 172.21.15.47:0/3507970610 shutdown_connections 2024-03-21T11:27:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.408+0000 7f6666766740 1 --2- 172.21.15.47:0/3507970610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d83a0 0x55ed735dec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.408+0000 7f6666766740 1 --2- 172.21.15.47:0/3507970610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d79e0 0x55ed735d7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.408+0000 7f6666766740 1 --2- 172.21.15.47:0/3507970610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735d6c30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.408+0000 7f6666766740 1 -- 172.21.15.47:0/3507970610 >> 172.21.15.47:0/3507970610 conn(0x55ed735d1c00 msgr2=0x55ed735d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.408+0000 7f6666766740 1 -- 172.21.15.47:0/3507970610 shutdown_connections 2024-03-21T11:27:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.408+0000 7f6666766740 1 -- 172.21.15.47:0/3507970610 wait complete. 2024-03-21T11:27:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.408+0000 7f6661000640 1 --2- 172.21.15.47:0/165125737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f66440a2fa0 0x7f66440a5460 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f665002c190 tx=0x7f6650000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.408+0000 7f6666766740 1 -- start start 2024-03-21T11:27:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.408+0000 7f6666766740 1 Processor -- start 2024-03-21T11:27:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.409+0000 7f6666766740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.409+0000 7f6666766740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 0x55ed735f2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.409+0000 7f6666766740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d83a0 0x55ed735f9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.409+0000 7f6666766740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed735f03c0 con 0x55ed735d79e0 2024-03-21T11:27:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.409+0000 7f6666766740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed734f70d0 con 0x55ed735d83a0 2024-03-21T11:27:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.409+0000 7f6666766740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed735f0540 con 0x55ed735d6810 2024-03-21T11:27:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.409+0000 7f6666766740 10 client.0 _ll_register_callbacks cb 0x55ed734fbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.409+0000 7f6661000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.409+0000 7f6661a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d83a0 0x55ed735f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.409+0000 7f6661000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735f1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44178/0 (socket says 172.21.15.47:44178) 2024-03-21T11:27:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.409+0000 7f6661a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d83a0 0x55ed735f9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59532/0 (socket says 172.21.15.47:59532) 2024-03-21T11:27:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.409+0000 7f6661000640 1 -- 172.21.15.47:0/4289246319 learned_addr learned my addr 172.21.15.47:0/4289246319 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.409+0000 7f665be00640 1 --2- 172.21.15.47:0/4289246319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 0x55ed735f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.410+0000 7f6661000640 1 -- 172.21.15.47:0/4289246319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d83a0 msgr2=0x55ed735f9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.410+0000 7f6661000640 1 --2- 172.21.15.47:0/4289246319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d83a0 0x55ed735f9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.410+0000 7f6661000640 1 -- 172.21.15.47:0/4289246319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 msgr2=0x55ed735f2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.410+0000 7f6661000640 1 --2- 172.21.15.47:0/4289246319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 0x55ed735f2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.410+0000 7f6661000640 1 -- 172.21.15.47:0/4289246319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed7324f3f0 con 0x55ed735d6810 2024-03-21T11:27:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.410+0000 7f6661000640 1 --2- 172.21.15.47:0/4289246319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735f1ad0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f665c104910 tx=0x7f6650009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57c59f557faa2fd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.410+0000 7f6642a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.410+0000 7f6642a00640 1 -- 172.21.15.47:0/4289246319 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66500050b0 con 0x55ed735d6810 2024-03-21T11:27:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.410+0000 7f6642a00640 1 -- 172.21.15.47:0/4289246319 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6650005250 con 0x55ed735d6810 2024-03-21T11:27:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.410+0000 7f6666766740 1 -- 172.21.15.47:0/4289246319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6654003e50 con 0x55ed735d6810 2024-03-21T11:27:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.411+0000 7f6642a00640 1 -- 172.21.15.47:0/4289246319 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6650005450 con 0x55ed735d6810 2024-03-21T11:27:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.411+0000 7f6666766740 10 client.26707 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.411+0000 7f6666766740 1 -- 172.21.15.47:0/4289246319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ed733197d0 con 0x55ed735d6810 2024-03-21T11:27:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.411+0000 7f6642a00640 1 -- 172.21.15.47:0/4289246319 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6654003e50 con 0x55ed735d6810 2024-03-21T11:27:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.411+0000 7f6642a00640 1 -- 172.21.15.47:0/4289246319 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6650051070 con 0x55ed735d6810 2024-03-21T11:27:11.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.411+0000 7f6666766740 10 client.26707 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.411+0000 7f6666766740 10 client.26707 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:27:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.411+0000 7f6666766740 20 client.26707 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.411+0000 7f6666766740 1 --2- 172.21.15.47:0/4289246319 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ed735fd540 0x55ed735ff990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.411+0000 7f6666766740 4 client.26707 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.411+0000 7f6666766740 1 -- 172.21.15.47:0/4289246319 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed73548800 con 0x55ed735fd540 2024-03-21T11:27:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.412+0000 7f665be00640 1 --2- 172.21.15.47:0/4289246319 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ed735fd540 0x55ed735ff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.412+0000 7f665be00640 1 --2- 172.21.15.47:0/4289246319 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ed735fd540 0x55ed735ff990 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=92d123ca75c75a2d server_cookie=46d14fa39442ff00 in_seq=0 out_seq=0 2024-03-21T11:27:11.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.412+0000 7f6642a00640 10 client.26707 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:27:11.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.413+0000 7f6642a00640 1 -- 172.21.15.47:0/4289246319 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ed73548800 con 0x55ed735fd540 2024-03-21T11:27:11.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.413+0000 7f6642a00640 10 client.26707 handle_command_reply: tid=0 2024-03-21T11:27:11.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.413+0000 7f6666766740 1 client.26707 shutdown 2024-03-21T11:27:11.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.413+0000 7f6666766740 1 -- 172.21.15.47:0/4289246319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 msgr2=0x55ed735f1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.413+0000 7f6666766740 1 --2- 172.21.15.47:0/4289246319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735f1ad0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f665c104910 tx=0x7f6650009260 comp rx=0 tx=0).stop 2024-03-21T11:27:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.413+0000 7f6666766740 1 -- 172.21.15.47:0/4289246319 shutdown_connections 2024-03-21T11:27:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.413+0000 7f6666766740 1 --2- 172.21.15.47:0/4289246319 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ed735fd540 0x55ed735ff990 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.413+0000 7f6666766740 1 --2- 172.21.15.47:0/4289246319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d83a0 0x55ed735f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.413+0000 7f6666766740 1 --2- 172.21.15.47:0/4289246319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 0x55ed735f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.413+0000 7f6642a00640 0 client.26707 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:27:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.413+0000 7f6666766740 1 --2- 172.21.15.47:0/4289246319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735f1ad0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.413+0000 7f6666766740 1 -- 172.21.15.47:0/4289246319 >> 172.21.15.47:0/4289246319 conn(0x55ed735d1c00 msgr2=0x55ed735d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.413+0000 7f6666766740 1 -- 172.21.15.47:0/4289246319 shutdown_connections 2024-03-21T11:27:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.413+0000 7f6666766740 1 -- 172.21.15.47:0/4289246319 wait complete. 2024-03-21T11:27:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.414+0000 7f6666766740 20 client.26707 trim_cache size 0 max 16384 2024-03-21T11:27:11.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.428+0000 7f6666766740 1 Processor -- start 2024-03-21T11:27:11.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.428+0000 7f6666766740 1 -- start start 2024-03-21T11:27:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.428+0000 7f6666766740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d6810 0x55ed735f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.428+0000 7f6666766740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 0x55ed735f9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.428+0000 7f6666766740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d83a0 0x55ed735d5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.428+0000 7f6666766740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c1a39f0 con 0x55ed735d79e0 2024-03-21T11:27:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.428+0000 7f6666766740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c1a5370 con 0x55ed735d6810 2024-03-21T11:27:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.428+0000 7f6666766740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c1a4050 con 0x55ed735d83a0 2024-03-21T11:27:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.428+0000 7f6661000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d6810 0x55ed735f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.428+0000 7f6661a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d83a0 0x55ed735d5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.428+0000 7f6661a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d83a0 0x55ed735d5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44184/0 (socket says 172.21.15.47:44184) 2024-03-21T11:27:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.428+0000 7f6661a00640 1 -- 172.21.15.47:0/3334221807 learned_addr learned my addr 172.21.15.47:0/3334221807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.428+0000 7f6661000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d6810 0x55ed735f1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59534/0 (socket says 172.21.15.47:59534) 2024-03-21T11:27:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.428+0000 7f665be00640 1 --2- 172.21.15.47:0/3334221807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 0x55ed735f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.429+0000 7f6661a00640 1 -- 172.21.15.47:0/3334221807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d6810 msgr2=0x55ed735f1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.429+0000 7f6661a00640 1 --2- 172.21.15.47:0/3334221807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d6810 0x55ed735f1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.429+0000 7f6661a00640 1 -- 172.21.15.47:0/3334221807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 msgr2=0x55ed735f9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.429+0000 7f6661a00640 1 --2- 172.21.15.47:0/3334221807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 0x55ed735f9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.429+0000 7f6661a00640 1 -- 172.21.15.47:0/3334221807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed735f0410 con 0x55ed735d83a0 2024-03-21T11:27:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.429+0000 7f6661000640 1 --2- 172.21.15.47:0/3334221807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d6810 0x55ed735f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.429+0000 7f665be00640 1 --2- 172.21.15.47:0/3334221807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 0x55ed735f9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.429+0000 7f6661a00640 1 --2- 172.21.15.47:0/3334221807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d83a0 0x55ed735d5570 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6654005c40 tx=0x7f66540111e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6093a9a27a7e9a22 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.429+0000 7f664d600640 1 -- 172.21.15.47:0/3334221807 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665400f9a0 con 0x55ed735d83a0 2024-03-21T11:27:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.429+0000 7f664d600640 1 -- 172.21.15.47:0/3334221807 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6654003e50 con 0x55ed735d83a0 2024-03-21T11:27:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.429+0000 7f664d600640 1 -- 172.21.15.47:0/3334221807 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654005560 con 0x55ed735d83a0 2024-03-21T11:27:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.430+0000 7f6666766740 1 -- 172.21.15.47:0/3334221807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d83a0 msgr2=0x55ed735d5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.430+0000 7f6666766740 1 --2- 172.21.15.47:0/3334221807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d83a0 0x55ed735d5570 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6654005c40 tx=0x7f66540111e0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.430+0000 7f6666766740 1 -- 172.21.15.47:0/3334221807 shutdown_connections 2024-03-21T11:27:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.430+0000 7f6666766740 1 --2- 172.21.15.47:0/3334221807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d83a0 0x55ed735d5570 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.430+0000 7f6666766740 1 --2- 172.21.15.47:0/3334221807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 0x55ed735f9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.430+0000 7f6666766740 1 --2- 172.21.15.47:0/3334221807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d6810 0x55ed735f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.430+0000 7f6666766740 1 -- 172.21.15.47:0/3334221807 >> 172.21.15.47:0/3334221807 conn(0x55ed735d1c00 msgr2=0x55ed735d4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.430+0000 7f6666766740 1 -- 172.21.15.47:0/3334221807 shutdown_connections 2024-03-21T11:27:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.430+0000 7f6666766740 1 -- 172.21.15.47:0/3334221807 wait complete. 2024-03-21T11:27:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.431+0000 7f6666766740 1 -- start start 2024-03-21T11:27:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.431+0000 7f6666766740 1 Processor -- start 2024-03-21T11:27:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.431+0000 7f6666766740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.431+0000 7f6666766740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 0x55ed735f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.431+0000 7f6666766740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d83a0 0x55ed735f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.431+0000 7f6666766740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c1a4d10 con 0x55ed735d79e0 2024-03-21T11:27:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.431+0000 7f6666766740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c1a46b0 con 0x55ed735d83a0 2024-03-21T11:27:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.431+0000 7f6666766740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c073220 con 0x55ed735d6810 2024-03-21T11:27:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.431+0000 7f6666766740 10 client.0 _ll_register_callbacks cb 0x7f6650009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.432+0000 7f6661a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d83a0 0x55ed735f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.432+0000 7f6661000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.432+0000 7f6661a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d83a0 0x55ed735f82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59544/0 (socket says 172.21.15.47:59544) 2024-03-21T11:27:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.432+0000 7f6661a00640 1 -- 172.21.15.47:0/3084392865 learned_addr learned my addr 172.21.15.47:0/3084392865 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.432+0000 7f665be00640 1 --2- 172.21.15.47:0/3084392865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 0x55ed735f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.432+0000 7f6661a00640 1 -- 172.21.15.47:0/3084392865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 msgr2=0x55ed735f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.432+0000 7f6661a00640 1 --2- 172.21.15.47:0/3084392865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.432+0000 7f6661a00640 1 -- 172.21.15.47:0/3084392865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 msgr2=0x55ed735f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.432+0000 7f6661a00640 1 --2- 172.21.15.47:0/3084392865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 0x55ed735f52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.432+0000 7f6661a00640 1 -- 172.21.15.47:0/3084392865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed735f88c0 con 0x55ed735d83a0 2024-03-21T11:27:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.432+0000 7f6661a00640 1 --2- 172.21.15.47:0/3084392865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d83a0 0x55ed735f82d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6654010010 tx=0x7f6654003cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=482f9f673c57196a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.432+0000 7f6642a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.433+0000 7f6642a00640 1 -- 172.21.15.47:0/3084392865 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654034060 con 0x55ed735d83a0 2024-03-21T11:27:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.433+0000 7f6642a00640 1 -- 172.21.15.47:0/3084392865 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f665400f9a0 con 0x55ed735d83a0 2024-03-21T11:27:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.433+0000 7f6666766740 1 -- 172.21.15.47:0/3084392865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6654003e50 con 0x55ed735d83a0 2024-03-21T11:27:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.433+0000 7f6642a00640 1 -- 172.21.15.47:0/3084392865 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f665403b050 con 0x55ed735d83a0 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.433+0000 7f6666766740 10 client.26690 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.433+0000 7f6666766740 1 -- 172.21.15.47:0/3084392865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ed73659460 con 0x55ed735d83a0 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.433+0000 7f6642a00640 1 -- 172.21.15.47:0/3084392865 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6654003e50 con 0x55ed735d83a0 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.433+0000 7f6642a00640 1 -- 172.21.15.47:0/3084392865 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6654018020 con 0x55ed735d83a0 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.434+0000 7f6666766740 10 client.26690 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.434+0000 7f6666766740 10 client.26690 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.434+0000 7f6666766740 20 client.26690 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.434+0000 7f6666766740 1 --2- 172.21.15.47:0/3084392865 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ed73659b10 0x55ed7365bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.434+0000 7f6666766740 4 client.26690 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.434+0000 7f6666766740 1 -- 172.21.15.47:0/3084392865 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ed73542b50 con 0x55ed73659b10 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.434+0000 7f6661000640 1 --2- 172.21.15.47:0/3084392865 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ed73659b10 0x55ed7365bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.434+0000 7f6661000640 1 --2- 172.21.15.47:0/3084392865 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ed73659b10 0x55ed7365bf30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1a7cd03986dc71d server_cookie=1dc0cfe6441f6c4d in_seq=0 out_seq=0 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.434+0000 7f6642a00640 10 client.26690 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6642a00640 1 -- 172.21.15.47:0/3084392865 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ed73542b50 con 0x55ed73659b10 2024-03-21T11:27:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6642a00640 10 client.26690 handle_command_reply: tid=0 2024-03-21T11:27:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6666766740 1 client.26690 shutdown 2024-03-21T11:27:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6666766740 1 -- 172.21.15.47:0/3084392865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d83a0 msgr2=0x55ed735f82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6666766740 1 --2- 172.21.15.47:0/3084392865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d83a0 0x55ed735f82d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6654010010 tx=0x7f6654003cc0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6666766740 1 -- 172.21.15.47:0/3084392865 shutdown_connections 2024-03-21T11:27:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6666766740 1 --2- 172.21.15.47:0/3084392865 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ed73659b10 0x55ed7365bf30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6666766740 1 --2- 172.21.15.47:0/3084392865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed735d83a0 0x55ed735f82d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6666766740 1 --2- 172.21.15.47:0/3084392865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed735d79e0 0x55ed735f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6666766740 1 --2- 172.21.15.47:0/3084392865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed735d6810 0x55ed735f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6642a00640 0 client.26690 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:27:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6666766740 1 -- 172.21.15.47:0/3084392865 >> 172.21.15.47:0/3084392865 conn(0x55ed735d1c00 msgr2=0x55ed735f1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6666766740 1 -- 172.21.15.47:0/3084392865 shutdown_connections 2024-03-21T11:27:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6666766740 1 -- 172.21.15.47:0/3084392865 wait complete. 2024-03-21T11:27:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.435+0000 7f6666766740 20 client.26690 trim_cache size 0 max 16384 2024-03-21T11:27:11.441 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:11.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.436+0000 7f6663a00640 1 -- 172.21.15.47:0/165125737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f66440a2fa0 msgr2=0x7f66440a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.436+0000 7f6663a00640 1 --2- 172.21.15.47:0/165125737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f66440a2fa0 0x7f66440a5460 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f665002c190 tx=0x7f6650000bc0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.436+0000 7f6663a00640 1 -- 172.21.15.47:0/165125737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f665c104ab0 msgr2=0x7f665c19a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.436+0000 7f6663a00640 1 --2- 172.21.15.47:0/165125737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f665c104ab0 0x7f665c19a110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f664800e9f0 tx=0x7f664800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.436+0000 7f6663a00640 1 -- 172.21.15.47:0/165125737 shutdown_connections 2024-03-21T11:27:11.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.436+0000 7f6663a00640 1 --2- 172.21.15.47:0/165125737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f66440a2fa0 0x7f66440a5460 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.436+0000 7f6663a00640 1 --2- 172.21.15.47:0/165125737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f665c105470 0x7f665c1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.436+0000 7f6663a00640 1 --2- 172.21.15.47:0/165125737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f665c104ab0 0x7f665c19a110 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.436+0000 7f6663a00640 1 --2- 172.21.15.47:0/165125737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f665c1038c0 0x7f665c199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.436+0000 7f6663a00640 1 -- 172.21.15.47:0/165125737 >> 172.21.15.47:0/165125737 conn(0x7f665c0fee50 msgr2=0x7f665c107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.436+0000 7f6663a00640 1 -- 172.21.15.47:0/165125737 shutdown_connections 2024-03-21T11:27:11.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.436+0000 7f6663a00640 1 -- 172.21.15.47:0/165125737 wait complete. 2024-03-21T11:27:11.456 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:11.456 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:27:11.501 INFO:tasks.workunit.client.0.smithi047.stdout: 1 187038 8.90 MB/sec warmup 115 sec latency 26.365 ms 2024-03-21T11:27:11.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.681+0000 7f821b200640 1 -- 172.21.15.47:0/23375793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f821c103150 msgr2=0x7f821c1099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.681+0000 7f821b200640 1 --2- 172.21.15.47:0/23375793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f821c103150 0x7f821c1099e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8214009ca0 tx=0x7f821402c0f0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.681+0000 7f821b200640 1 -- 172.21.15.47:0/23375793 shutdown_connections 2024-03-21T11:27:11.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.681+0000 7f821b200640 1 --2- 172.21.15.47:0/23375793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f821c103150 0x7f821c1099e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.681+0000 7f821b200640 1 --2- 172.21.15.47:0/23375793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c102790 0x7f821c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.681+0000 7f821b200640 1 --2- 172.21.15.47:0/23375793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f821c1015a0 0x7f821c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.681+0000 7f821b200640 1 -- 172.21.15.47:0/23375793 >> 172.21.15.47:0/23375793 conn(0x7f821c0fcb10 msgr2=0x7f821c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.682+0000 7f821b200640 1 -- 172.21.15.47:0/23375793 shutdown_connections 2024-03-21T11:27:11.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.682+0000 7f821b200640 1 -- 172.21.15.47:0/23375793 wait complete. 2024-03-21T11:27:11.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.683+0000 7f821b200640 1 Processor -- start 2024-03-21T11:27:11.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.683+0000 7f821b200640 1 -- start start 2024-03-21T11:27:11.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.683+0000 7f821b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f821c1015a0 0x7f821c1a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.683+0000 7f821b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c102790 0x7f821c1a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.683+0000 7f821b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f821c103150 0x7f821c1a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.683+0000 7f821b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c06bad0 con 0x7f821c1015a0 2024-03-21T11:27:11.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.683+0000 7f821b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c06b950 con 0x7f821c102790 2024-03-21T11:27:11.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.683+0000 7f821b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c06bc50 con 0x7f821c103150 2024-03-21T11:27:11.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.684+0000 7f8213e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c102790 0x7f821c1a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.684+0000 7f8213e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c102790 0x7f821c1a2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59570/0 (socket says 172.21.15.47:59570) 2024-03-21T11:27:11.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.684+0000 7f821a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f821c103150 0x7f821c1a97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.684+0000 7f8213e00640 1 -- 172.21.15.47:0/1671730581 learned_addr learned my addr 172.21.15.47:0/1671730581 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.684+0000 7f8219e00640 1 --2- 172.21.15.47:0/1671730581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f821c1015a0 0x7f821c1a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.684+0000 7f821a800640 1 -- 172.21.15.47:0/1671730581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c102790 msgr2=0x7f821c1a2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.684+0000 7f821a800640 1 --2- 172.21.15.47:0/1671730581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c102790 0x7f821c1a2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.684+0000 7f821a800640 1 -- 172.21.15.47:0/1671730581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f821c1015a0 msgr2=0x7f821c1a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.684+0000 7f821a800640 1 --2- 172.21.15.47:0/1671730581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f821c1015a0 0x7f821c1a2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.684+0000 7f821a800640 1 -- 172.21.15.47:0/1671730581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c1a9e30 con 0x7f821c103150 2024-03-21T11:27:11.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.684+0000 7f8219e00640 1 --2- 172.21.15.47:0/1671730581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f821c1015a0 0x7f821c1a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:27:11.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.684+0000 7f821a800640 1 --2- 172.21.15.47:0/1671730581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f821c103150 0x7f821c1a97c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8214037040 tx=0x7f8214032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.685+0000 7f8212a00640 1 -- 172.21.15.47:0/1671730581 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214041020 con 0x7f821c103150 2024-03-21T11:27:11.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.685+0000 7f8212a00640 1 -- 172.21.15.47:0/1671730581 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8214034cd0 con 0x7f821c103150 2024-03-21T11:27:11.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.685+0000 7f8212a00640 1 -- 172.21.15.47:0/1671730581 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214039040 con 0x7f821c103150 2024-03-21T11:27:11.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.685+0000 7f821b200640 1 -- 172.21.15.47:0/1671730581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821c1aa0c0 con 0x7f821c103150 2024-03-21T11:27:11.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.685+0000 7f821b200640 1 -- 172.21.15.47:0/1671730581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821c1aa580 con 0x7f821c103150 2024-03-21T11:27:11.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.688+0000 7f8212a00640 1 -- 172.21.15.47:0/1671730581 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8214044020 con 0x7f821c103150 2024-03-21T11:27:11.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.689+0000 7f82237be740 1 Processor -- start 2024-03-21T11:27:11.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.689+0000 7f82237be740 1 -- start start 2024-03-21T11:27:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.689+0000 7f82237be740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539de5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.689+0000 7f82237be740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539de7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.689+0000 7f82237be740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 0x561539dede10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.689+0000 7f82237be740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561539d4a2c0 con 0x561539de7580 2024-03-21T11:27:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.689+0000 7f82237be740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561539cf2c90 con 0x561539de59f0 2024-03-21T11:27:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.689+0000 7f82237be740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561539d46360 con 0x561539de6bc0 2024-03-21T11:27:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.689+0000 7f8219e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539de7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.690+0000 7f8219e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539de7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44218/0 (socket says 172.21.15.47:44218) 2024-03-21T11:27:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.690+0000 7f8219e00640 1 -- 172.21.15.47:0/816557293 learned_addr learned my addr 172.21.15.47:0/816557293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.690+0000 7f8213e00640 1 --2- 172.21.15.47:0/816557293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539de5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.690+0000 7f8213e00640 1 --2- 172.21.15.47:0/816557293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 0x561539dede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.690+0000 7f8212a00640 1 --2- 172.21.15.47:0/1671730581 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f82000a2f20 0x7f82000a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.690+0000 7f8219e00640 1 -- 172.21.15.47:0/816557293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 msgr2=0x561539de5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.690+0000 7f8219e00640 1 --2- 172.21.15.47:0/816557293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539de5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.690+0000 7f8219e00640 1 -- 172.21.15.47:0/816557293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 msgr2=0x561539dede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.690+0000 7f8219e00640 1 --2- 172.21.15.47:0/816557293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 0x561539dede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.690+0000 7f8219e00640 1 -- 172.21.15.47:0/816557293 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561539d17080 con 0x561539de6bc0 2024-03-21T11:27:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.690+0000 7f8213e00640 1 --2- 172.21.15.47:0/816557293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539de5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.690+0000 7f8212a00640 1 -- 172.21.15.47:0/1671730581 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f82140f1610 con 0x7f821c103150 2024-03-21T11:27:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.690+0000 7f821a800640 1 --2- 172.21.15.47:0/1671730581 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f82000a2f20 0x7f82000a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.691+0000 7f8219e00640 1 --2- 172.21.15.47:0/816557293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539de7040 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f820c009c20 tx=0x7f820c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29fc3587b24c7e08 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.691+0000 7f821a800640 1 --2- 172.21.15.47:0/1671730581 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f82000a2f20 0x7f82000a53e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8214006010 tx=0x7f82140095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.691+0000 7f820b000640 1 -- 172.21.15.47:0/816557293 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c005020 con 0x561539de6bc0 2024-03-21T11:27:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.691+0000 7f820b000640 1 -- 172.21.15.47:0/816557293 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f820c0047a0 con 0x561539de6bc0 2024-03-21T11:27:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.691+0000 7f820b000640 1 -- 172.21.15.47:0/816557293 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c0126f0 con 0x561539de6bc0 2024-03-21T11:27:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.692+0000 7f82237be740 1 -- 172.21.15.47:0/816557293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 msgr2=0x561539de7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.692+0000 7f82237be740 1 --2- 172.21.15.47:0/816557293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539de7040 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f820c009c20 tx=0x7f820c010740 comp rx=0 tx=0).stop 2024-03-21T11:27:11.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.692+0000 7f82237be740 1 -- 172.21.15.47:0/816557293 shutdown_connections 2024-03-21T11:27:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.692+0000 7f82237be740 1 --2- 172.21.15.47:0/816557293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 0x561539dede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.692+0000 7f82237be740 1 --2- 172.21.15.47:0/816557293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539de7040 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.692+0000 7f82237be740 1 --2- 172.21.15.47:0/816557293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539de5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.692+0000 7f82237be740 1 -- 172.21.15.47:0/816557293 >> 172.21.15.47:0/816557293 conn(0x561539de0de0 msgr2=0x561539de11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.692+0000 7f82237be740 1 -- 172.21.15.47:0/816557293 shutdown_connections 2024-03-21T11:27:11.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.692+0000 7f82237be740 1 -- 172.21.15.47:0/816557293 wait complete. 2024-03-21T11:27:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.692+0000 7f82237be740 1 -- start start 2024-03-21T11:27:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.693+0000 7f82237be740 1 Processor -- start 2024-03-21T11:27:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.693+0000 7f82237be740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539defcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.693+0000 7f82237be740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de6bc0 0x561539df0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.693+0000 7f82237be740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de7580 0x561539df7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.693+0000 7f82237be740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561539dee590 con 0x561539de6bc0 2024-03-21T11:27:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.693+0000 7f82237be740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561539d060d0 con 0x561539de59f0 2024-03-21T11:27:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.693+0000 7f82237be740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561539dee710 con 0x561539de7580 2024-03-21T11:27:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.693+0000 7f82237be740 10 client.0 _ll_register_callbacks cb 0x561539d0ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.693+0000 7f8219e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de6bc0 0x561539df0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.693+0000 7f8219e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de6bc0 0x561539df0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38706/0 (socket says 172.21.15.47:38706) 2024-03-21T11:27:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.693+0000 7f8219e00640 1 -- 172.21.15.47:0/1385341633 learned_addr learned my addr 172.21.15.47:0/1385341633 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.694+0000 7f8213e00640 1 --2- 172.21.15.47:0/1385341633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539defcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.694+0000 7f8213e00640 1 --2- 172.21.15.47:0/1385341633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de7580 0x561539df7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.694+0000 7f8219e00640 1 -- 172.21.15.47:0/1385341633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de7580 msgr2=0x561539df7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.694+0000 7f8219e00640 1 --2- 172.21.15.47:0/1385341633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de7580 0x561539df7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.694+0000 7f8219e00640 1 -- 172.21.15.47:0/1385341633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 msgr2=0x561539defcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.694+0000 7f8219e00640 1 --2- 172.21.15.47:0/1385341633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539defcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.694+0000 7f8219e00640 1 -- 172.21.15.47:0/1385341633 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561539a5e3f0 con 0x561539de6bc0 2024-03-21T11:27:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.694+0000 7f8219e00640 1 --2- 172.21.15.47:0/1385341633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de6bc0 0x561539df0230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f820c009bf0 tx=0x7f820c005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b9fe62a2f19b6f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.694+0000 7f81ff400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:27:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.695+0000 7f81ff400640 1 -- 172.21.15.47:0/1385341633 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c02a060 con 0x561539de6bc0 2024-03-21T11:27:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.695+0000 7f81ff400640 1 -- 172.21.15.47:0/1385341633 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f820c009d10 con 0x561539de6bc0 2024-03-21T11:27:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.695+0000 7f82237be740 1 -- 172.21.15.47:0/1385341633 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f820c0047a0 con 0x561539de6bc0 2024-03-21T11:27:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.695+0000 7f81ff400640 1 -- 172.21.15.47:0/1385341633 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f820c031050 con 0x561539de6bc0 2024-03-21T11:27:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.695+0000 7f82237be740 10 client.16479 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.695+0000 7f82237be740 1 -- 172.21.15.47:0/1385341633 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561539b287d0 con 0x561539de6bc0 2024-03-21T11:27:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.695+0000 7f81ff400640 1 -- 172.21.15.47:0/1385341633 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f820c0047a0 con 0x561539de6bc0 2024-03-21T11:27:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.696+0000 7f81ff400640 1 -- 172.21.15.47:0/1385341633 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f820c00f040 con 0x561539de6bc0 2024-03-21T11:27:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.696+0000 7f82237be740 10 client.16479 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.696+0000 7f82237be740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.696+0000 7f82237be740 20 client.16479 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.696+0000 7f82237be740 1 --2- 172.21.15.47:0/1385341633 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561539dfb470 0x561539dfd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.696+0000 7f82237be740 4 client.16479 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.696+0000 7f82237be740 1 -- 172.21.15.47:0/1385341633 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561539d57800 con 0x561539dfb470 2024-03-21T11:27:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.696+0000 7f8213e00640 1 --2- 172.21.15.47:0/1385341633 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561539dfb470 0x561539dfd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.697+0000 7f8213e00640 1 --2- 172.21.15.47:0/1385341633 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561539dfb470 0x561539dfd860 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46205ac4d3ce18d4 server_cookie=b6c918fdbdf10ff in_seq=0 out_seq=0 2024-03-21T11:27:11.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.697+0000 7f81ff400640 10 client.16479 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.698+0000 7f81ff400640 1 -- 172.21.15.47:0/1385341633 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561539d57800 con 0x561539dfb470 2024-03-21T11:27:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.699+0000 7f81ff400640 10 client.16479 handle_command_reply: tid=0 2024-03-21T11:27:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.699+0000 7f82237be740 1 client.16479 shutdown 2024-03-21T11:27:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.699+0000 7f82237be740 1 -- 172.21.15.47:0/1385341633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de6bc0 msgr2=0x561539df0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.699+0000 7f82237be740 1 --2- 172.21.15.47:0/1385341633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de6bc0 0x561539df0230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f820c009bf0 tx=0x7f820c005210 comp rx=0 tx=0).stop 2024-03-21T11:27:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.699+0000 7f82237be740 1 -- 172.21.15.47:0/1385341633 shutdown_connections 2024-03-21T11:27:11.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.699+0000 7f82237be740 1 --2- 172.21.15.47:0/1385341633 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561539dfb470 0x561539dfd860 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.699+0000 7f82237be740 1 --2- 172.21.15.47:0/1385341633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de7580 0x561539df7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.699+0000 7f82237be740 1 --2- 172.21.15.47:0/1385341633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de6bc0 0x561539df0230 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.699+0000 7f82237be740 1 --2- 172.21.15.47:0/1385341633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539defcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.699+0000 7f81ff400640 0 client.16479 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.699+0000 7f82237be740 1 -- 172.21.15.47:0/1385341633 >> 172.21.15.47:0/1385341633 conn(0x561539de0de0 msgr2=0x561539de0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.699+0000 7f82237be740 1 -- 172.21.15.47:0/1385341633 shutdown_connections 2024-03-21T11:27:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.699+0000 7f82237be740 1 -- 172.21.15.47:0/1385341633 wait complete. 2024-03-21T11:27:11.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.699+0000 7f82237be740 20 client.16479 trim_cache size 0 max 16384 2024-03-21T11:27:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.715+0000 7f82237be740 1 Processor -- start 2024-03-21T11:27:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.715+0000 7f82237be740 1 -- start start 2024-03-21T11:27:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.715+0000 7f82237be740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539de9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.715+0000 7f82237be740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539df71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.715+0000 7f82237be740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 0x561539dea240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.715+0000 7f82237be740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c1b4880 con 0x561539de7580 2024-03-21T11:27:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.715+0000 7f82237be740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c1b6200 con 0x561539de59f0 2024-03-21T11:27:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.715+0000 7f82237be740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c1b4ee0 con 0x561539de6bc0 2024-03-21T11:27:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.715+0000 7f8219e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539df71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.715+0000 7f8219e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539df71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44232/0 (socket says 172.21.15.47:44232) 2024-03-21T11:27:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.715+0000 7f8219e00640 1 -- 172.21.15.47:0/3476944919 learned_addr learned my addr 172.21.15.47:0/3476944919 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.715+0000 7f8213e00640 1 --2- 172.21.15.47:0/3476944919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539de9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.716+0000 7f8213e00640 1 --2- 172.21.15.47:0/3476944919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 0x561539dea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.716+0000 7f8219e00640 1 -- 172.21.15.47:0/3476944919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 msgr2=0x561539de9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.716+0000 7f8219e00640 1 --2- 172.21.15.47:0/3476944919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539de9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.716+0000 7f8219e00640 1 -- 172.21.15.47:0/3476944919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 msgr2=0x561539dea240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.716+0000 7f8219e00640 1 --2- 172.21.15.47:0/3476944919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 0x561539dea240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.716+0000 7f8219e00640 1 -- 172.21.15.47:0/3476944919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561539dee470 con 0x561539de6bc0 2024-03-21T11:27:11.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.716+0000 7f8213e00640 1 --2- 172.21.15.47:0/3476944919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 0x561539dea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:27:11.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.716+0000 7f8219e00640 1 --2- 172.21.15.47:0/3476944919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539df71c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f820c0046b0 tx=0x7f820c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=827f006c35efb49e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.716+0000 7f820a600640 1 -- 172.21.15.47:0/3476944919 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c02a040 con 0x561539de6bc0 2024-03-21T11:27:11.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.716+0000 7f820a600640 1 -- 172.21.15.47:0/3476944919 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f820c01f070 con 0x561539de6bc0 2024-03-21T11:27:11.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.716+0000 7f820a600640 1 -- 172.21.15.47:0/3476944919 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c009d10 con 0x561539de6bc0 2024-03-21T11:27:11.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.717+0000 7f82237be740 1 -- 172.21.15.47:0/3476944919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 msgr2=0x561539df71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.717+0000 7f82237be740 1 --2- 172.21.15.47:0/3476944919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539df71c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f820c0046b0 tx=0x7f820c009b20 comp rx=0 tx=0).stop 2024-03-21T11:27:11.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.717+0000 7f82237be740 1 -- 172.21.15.47:0/3476944919 shutdown_connections 2024-03-21T11:27:11.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.717+0000 7f82237be740 1 --2- 172.21.15.47:0/3476944919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 0x561539dea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.717+0000 7f82237be740 1 --2- 172.21.15.47:0/3476944919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539df71c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.717+0000 7f82237be740 1 --2- 172.21.15.47:0/3476944919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539de9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.717+0000 7f82237be740 1 -- 172.21.15.47:0/3476944919 >> 172.21.15.47:0/3476944919 conn(0x561539de0de0 msgr2=0x561539de48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.717+0000 7f82237be740 1 -- 172.21.15.47:0/3476944919 shutdown_connections 2024-03-21T11:27:11.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.717+0000 7f82237be740 1 -- 172.21.15.47:0/3476944919 wait complete. 2024-03-21T11:27:11.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.718+0000 7f82237be740 1 -- start start 2024-03-21T11:27:11.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.718+0000 7f82237be740 1 Processor -- start 2024-03-21T11:27:11.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.718+0000 7f82237be740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539df3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.718+0000 7f82237be740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539df3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.718+0000 7f82237be740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 0x561539df6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.718+0000 7f82237be740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c1b5ba0 con 0x561539de7580 2024-03-21T11:27:11.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.718+0000 7f82237be740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c1b5540 con 0x561539de59f0 2024-03-21T11:27:11.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.718+0000 7f82237be740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c1b2a40 con 0x561539de6bc0 2024-03-21T11:27:11.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.718+0000 7f82237be740 10 client.0 _ll_register_callbacks cb 0x7f820c009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:11.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.718+0000 7f8219e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539df3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.718+0000 7f8219e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539df3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44234/0 (socket says 172.21.15.47:44234) 2024-03-21T11:27:11.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.718+0000 7f8219e00640 1 -- 172.21.15.47:0/995831467 learned_addr learned my addr 172.21.15.47:0/995831467 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:11.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.719+0000 7f8213e00640 1 --2- 172.21.15.47:0/995831467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539df3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.719+0000 7f8213e00640 1 --2- 172.21.15.47:0/995831467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 0x561539df6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.719+0000 7f8219e00640 1 -- 172.21.15.47:0/995831467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 msgr2=0x561539df3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.719+0000 7f8219e00640 1 --2- 172.21.15.47:0/995831467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539df3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.719+0000 7f8219e00640 1 -- 172.21.15.47:0/995831467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 msgr2=0x561539df6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:11.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.719+0000 7f8219e00640 1 --2- 172.21.15.47:0/995831467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 0x561539df6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.719+0000 7f8219e00640 1 -- 172.21.15.47:0/995831467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561539df6e50 con 0x561539de6bc0 2024-03-21T11:27:11.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.719+0000 7f8219e00640 1 --2- 172.21.15.47:0/995831467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539df3840 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f820c019000 tx=0x7f820c005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ac70f33ba526e90 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:11.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.719+0000 7f81ff400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:11.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.719+0000 7f81ff400640 1 -- 172.21.15.47:0/995831467 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c02a040 con 0x561539de6bc0 2024-03-21T11:27:11.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.720+0000 7f81ff400640 1 -- 172.21.15.47:0/995831467 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f820c009530 con 0x561539de6bc0 2024-03-21T11:27:11.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.720+0000 7f82237be740 1 -- 172.21.15.47:0/995831467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f820c01f070 con 0x561539de6bc0 2024-03-21T11:27:11.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.720+0000 7f81ff400640 1 -- 172.21.15.47:0/995831467 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f820c0047a0 con 0x561539de6bc0 2024-03-21T11:27:11.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.720+0000 7f82237be740 10 client.26731 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:11.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.720+0000 7f82237be740 1 -- 172.21.15.47:0/995831467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561539e57560 con 0x561539de6bc0 2024-03-21T11:27:11.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.720+0000 7f81ff400640 1 -- 172.21.15.47:0/995831467 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f820c01f070 con 0x561539de6bc0 2024-03-21T11:27:11.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.720+0000 7f81ff400640 1 -- 172.21.15.47:0/995831467 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f820c03d070 con 0x561539de6bc0 2024-03-21T11:27:11.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.720+0000 7f82237be740 10 client.26731 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:11.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.720+0000 7f82237be740 10 client.26731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:11.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.720+0000 7f82237be740 20 client.26731 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:11.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.720+0000 7f82237be740 1 --2- 172.21.15.47:0/995831467 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561539e57e30 0x561539e5a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:11.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.720+0000 7f82237be740 4 client.26731 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:11.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.720+0000 7f82237be740 1 -- 172.21.15.47:0/995831467 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561539d51b50 con 0x561539e57e30 2024-03-21T11:27:11.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.721+0000 7f8213e00640 1 --2- 172.21.15.47:0/995831467 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561539e57e30 0x561539e5a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:11.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.721+0000 7f8213e00640 1 --2- 172.21.15.47:0/995831467 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561539e57e30 0x561539e5a250 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74cb5443cf2431b server_cookie=67c1cb4b290dd75a in_seq=0 out_seq=0 2024-03-21T11:27:11.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.721+0000 7f81ff400640 10 client.26731 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:11.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.721+0000 7f81ff400640 1 -- 172.21.15.47:0/995831467 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561539d51b50 con 0x561539e57e30 2024-03-21T11:27:11.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.721+0000 7f81ff400640 10 client.26731 handle_command_reply: tid=0 2024-03-21T11:27:11.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.721+0000 7f82237be740 1 client.26731 shutdown 2024-03-21T11:27:11.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.722+0000 7f82237be740 1 -- 172.21.15.47:0/995831467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 msgr2=0x561539df3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.722+0000 7f82237be740 1 --2- 172.21.15.47:0/995831467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539df3840 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f820c019000 tx=0x7f820c005ce0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.722+0000 7f82237be740 1 -- 172.21.15.47:0/995831467 shutdown_connections 2024-03-21T11:27:11.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.722+0000 7f82237be740 1 --2- 172.21.15.47:0/995831467 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561539e57e30 0x561539e5a250 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.722+0000 7f82237be740 1 --2- 172.21.15.47:0/995831467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561539de7580 0x561539df6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.722+0000 7f82237be740 1 --2- 172.21.15.47:0/995831467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561539de6bc0 0x561539df3840 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.722+0000 7f82237be740 1 --2- 172.21.15.47:0/995831467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561539de59f0 0x561539df3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.722+0000 7f82237be740 1 -- 172.21.15.47:0/995831467 >> 172.21.15.47:0/995831467 conn(0x561539de0de0 msgr2=0x561539de48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.722+0000 7f81ff400640 0 client.26731 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:11.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.722+0000 7f82237be740 1 -- 172.21.15.47:0/995831467 shutdown_connections 2024-03-21T11:27:11.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.722+0000 7f82237be740 1 -- 172.21.15.47:0/995831467 wait complete. 2024-03-21T11:27:11.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.722+0000 7f82237be740 20 client.26731 trim_cache size 0 max 16384 2024-03-21T11:27:11.733 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:11.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.723+0000 7f821b200640 1 -- 172.21.15.47:0/1671730581 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f82000a2f20 msgr2=0x7f82000a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.723+0000 7f821b200640 1 --2- 172.21.15.47:0/1671730581 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f82000a2f20 0x7f82000a53e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8214006010 tx=0x7f82140095c0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.723+0000 7f821b200640 1 -- 172.21.15.47:0/1671730581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f821c103150 msgr2=0x7f821c1a97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:11.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.723+0000 7f821b200640 1 --2- 172.21.15.47:0/1671730581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f821c103150 0x7f821c1a97c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8214037040 tx=0x7f8214032ea0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.723+0000 7f821b200640 1 -- 172.21.15.47:0/1671730581 shutdown_connections 2024-03-21T11:27:11.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.723+0000 7f821b200640 1 --2- 172.21.15.47:0/1671730581 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f82000a2f20 0x7f82000a53e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.723+0000 7f821b200640 1 --2- 172.21.15.47:0/1671730581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f821c103150 0x7f821c1a97c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.723+0000 7f821b200640 1 --2- 172.21.15.47:0/1671730581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c102790 0x7f821c1a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.723+0000 7f821b200640 1 --2- 172.21.15.47:0/1671730581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f821c1015a0 0x7f821c1a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:11.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.723+0000 7f821b200640 1 -- 172.21.15.47:0/1671730581 >> 172.21.15.47:0/1671730581 conn(0x7f821c0fcb10 msgr2=0x7f821c06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:11.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.723+0000 7f821b200640 1 -- 172.21.15.47:0/1671730581 shutdown_connections 2024-03-21T11:27:11.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:11.723+0000 7f821b200640 1 -- 172.21.15.47:0/1671730581 wait complete. 2024-03-21T11:27:12.502 INFO:tasks.workunit.client.0.smithi047.stdout: 1 188561 8.92 MB/sec warmup 116 sec latency 9.162 ms 2024-03-21T11:27:12.742 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:27:12.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:27:12.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:27:12.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:12 smithi100 ceph-mon[36245]: pgmap v516: 129 pgs: 129 active+clean; 40 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 230 op/s 2024-03-21T11:27:12.927 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:12 smithi047 ceph-mon[31351]: pgmap v516: 129 pgs: 129 active+clean; 40 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 230 op/s 2024-03-21T11:27:12.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:12 smithi102 ceph-mon[36748]: pgmap v516: 129 pgs: 129 active+clean; 40 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 230 op/s 2024-03-21T11:27:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.108+0000 7f14b2600640 1 -- 172.21.15.47:0/3740895341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14ac0ff2f0 msgr2=0x7f14ac0ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.108+0000 7f14b2600640 1 --2- 172.21.15.47:0/3740895341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14ac0ff2f0 0x7f14ac0ff710 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009900 tx=0x7f14a002bc30 comp rx=0 tx=0).stop 2024-03-21T11:27:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.109+0000 7f14b2600640 1 -- 172.21.15.47:0/3740895341 shutdown_connections 2024-03-21T11:27:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.109+0000 7f14b2600640 1 --2- 172.21.15.47:0/3740895341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14ac1071e0 0x7f14ac1096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.109+0000 7f14b2600640 1 --2- 172.21.15.47:0/3740895341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14ac0ffc50 0x7f14ac106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.109+0000 7f14b2600640 1 --2- 172.21.15.47:0/3740895341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14ac0ff2f0 0x7f14ac0ff710 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.109+0000 7f14b2600640 1 -- 172.21.15.47:0/3740895341 >> 172.21.15.47:0/3740895341 conn(0x7f14ac0fad20 msgr2=0x7f14ac0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.109+0000 7f14b2600640 1 -- 172.21.15.47:0/3740895341 shutdown_connections 2024-03-21T11:27:13.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.109+0000 7f14b2600640 1 -- 172.21.15.47:0/3740895341 wait complete. 2024-03-21T11:27:13.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.110+0000 7f14b2600640 1 Processor -- start 2024-03-21T11:27:13.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.110+0000 7f14b2600640 1 -- start start 2024-03-21T11:27:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.110+0000 7f14b2600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14ac0ff2f0 0x7f14ac199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.110+0000 7f14b2600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14ac0ffc50 0x7f14ac199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.110+0000 7f14b2600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14ac1071e0 0x7f14ac1a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.110+0000 7f14b2600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac10d250 con 0x7f14ac1071e0 2024-03-21T11:27:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.110+0000 7f14b2600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac10d0d0 con 0x7f14ac0ffc50 2024-03-21T11:27:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.110+0000 7f14b2600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac10d3d0 con 0x7f14ac0ff2f0 2024-03-21T11:27:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.111+0000 7f14b1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14ac0ff2f0 0x7f14ac199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.111+0000 7f14abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14ac0ffc50 0x7f14ac199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.111+0000 7f14b1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14ac0ff2f0 0x7f14ac199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44248/0 (socket says 172.21.15.47:44248) 2024-03-21T11:27:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.111+0000 7f14b1200640 1 -- 172.21.15.47:0/739847939 learned_addr learned my addr 172.21.15.47:0/739847939 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.111+0000 7f14abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14ac0ffc50 0x7f14ac199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59626/0 (socket says 172.21.15.47:59626) 2024-03-21T11:27:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.111+0000 7f14b1c00640 1 --2- 172.21.15.47:0/739847939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14ac1071e0 0x7f14ac1a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.111+0000 7f14b1200640 1 -- 172.21.15.47:0/739847939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14ac0ffc50 msgr2=0x7f14ac199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.111+0000 7f14b1200640 1 --2- 172.21.15.47:0/739847939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14ac0ffc50 0x7f14ac199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.111+0000 7f14b1200640 1 -- 172.21.15.47:0/739847939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14ac1071e0 msgr2=0x7f14ac1a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.111+0000 7f14b1200640 1 --2- 172.21.15.47:0/739847939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14ac1071e0 0x7f14ac1a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.111+0000 7f14b1200640 1 -- 172.21.15.47:0/739847939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac1a1690 con 0x7f14ac0ff2f0 2024-03-21T11:27:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.111+0000 7f14b1c00640 1 --2- 172.21.15.47:0/739847939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14ac1071e0 0x7f14ac1a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.111+0000 7f14b1200640 1 --2- 172.21.15.47:0/739847939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14ac0ff2f0 0x7f14ac199a70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f14a0000c00 tx=0x7f14a002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.112+0000 7f14a9600640 1 -- 172.21.15.47:0/739847939 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0043020 con 0x7f14ac0ff2f0 2024-03-21T11:27:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.112+0000 7f14a9600640 1 -- 172.21.15.47:0/739847939 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14a00054e0 con 0x7f14ac0ff2f0 2024-03-21T11:27:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.112+0000 7f14b2600640 1 -- 172.21.15.47:0/739847939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ac1a1920 con 0x7f14ac0ff2f0 2024-03-21T11:27:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.112+0000 7f14a9600640 1 -- 172.21.15.47:0/739847939 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0003c40 con 0x7f14ac0ff2f0 2024-03-21T11:27:13.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.112+0000 7f14b2600640 1 -- 172.21.15.47:0/739847939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac1a1ec0 con 0x7f14ac0ff2f0 2024-03-21T11:27:13.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.115+0000 7f14a9600640 1 -- 172.21.15.47:0/739847939 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f14a0046020 con 0x7f14ac0ff2f0 2024-03-21T11:27:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.115+0000 7f14b69c3740 1 Processor -- start 2024-03-21T11:27:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.115+0000 7f14b69c3740 1 -- start start 2024-03-21T11:27:13.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b69c3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553e2b0 0x55c3b553e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b69c3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553eca0 0x55c3b55437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b69c3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 0x55c3b55462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b69c3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b54a3eb0 con 0x55c3b553e2b0 2024-03-21T11:27:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b69c3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b544ca60 con 0x55c3b553eca0 2024-03-21T11:27:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b69c3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b549ff50 con 0x55c3b5543e10 2024-03-21T11:27:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553eca0 0x55c3b55437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553eca0 0x55c3b55437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59632/0 (socket says 172.21.15.47:59632) 2024-03-21T11:27:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b1200640 1 -- 172.21.15.47:0/668863265 learned_addr learned my addr 172.21.15.47:0/668863265 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553e2b0 0x55c3b553e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14abe00640 1 --2- 172.21.15.47:0/668863265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 0x55c3b55462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14a9600640 1 --2- 172.21.15.47:0/739847939 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14940a2fa0 0x7f14940a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14a9600640 1 -- 172.21.15.47:0/739847939 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f14a00f17d0 con 0x7f14ac0ff2f0 2024-03-21T11:27:13.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b1200640 1 -- 172.21.15.47:0/668863265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 msgr2=0x55c3b55462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b1200640 1 --2- 172.21.15.47:0/668863265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 0x55c3b55462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b1200640 1 -- 172.21.15.47:0/668863265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553e2b0 msgr2=0x55c3b553e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b1200640 1 --2- 172.21.15.47:0/668863265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553e2b0 0x55c3b553e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.116+0000 7f14b1200640 1 -- 172.21.15.47:0/668863265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3b52807e0 con 0x55c3b553eca0 2024-03-21T11:27:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.117+0000 7f14abe00640 1 --2- 172.21.15.47:0/668863265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 0x55c3b55462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.117+0000 7f14b1c00640 1 --2- 172.21.15.47:0/739847939 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14940a2fa0 0x7f14940a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.117+0000 7f14b1200640 1 --2- 172.21.15.47:0/668863265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553eca0 0x55c3b55437a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f14a00eef40 tx=0x7f14a0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd8d2c6308884d3d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.117+0000 7f149e000640 1 -- 172.21.15.47:0/668863265 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a00b9e10 con 0x55c3b553eca0 2024-03-21T11:27:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.117+0000 7f149e000640 1 -- 172.21.15.47:0/668863265 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14a0004250 con 0x55c3b553eca0 2024-03-21T11:27:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.117+0000 7f149e000640 1 -- 172.21.15.47:0/668863265 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a00bd4d0 con 0x55c3b553eca0 2024-03-21T11:27:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.118+0000 7f14b69c3740 1 -- 172.21.15.47:0/668863265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553eca0 msgr2=0x55c3b55437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.118+0000 7f14b69c3740 1 --2- 172.21.15.47:0/668863265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553eca0 0x55c3b55437a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f14a00eef40 tx=0x7f14a0009260 comp rx=0 tx=0).stop 2024-03-21T11:27:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.118+0000 7f14b1c00640 1 --2- 172.21.15.47:0/739847939 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14940a2fa0 0x7f14940a5460 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55c3b553f500 tx=0x7f14a4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.118+0000 7f14b69c3740 1 -- 172.21.15.47:0/668863265 shutdown_connections 2024-03-21T11:27:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.118+0000 7f14b69c3740 1 --2- 172.21.15.47:0/668863265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 0x55c3b55462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.118+0000 7f14b69c3740 1 --2- 172.21.15.47:0/668863265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553eca0 0x55c3b55437a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.118+0000 7f14b69c3740 1 --2- 172.21.15.47:0/668863265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553e2b0 0x55c3b553e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.118+0000 7f14b69c3740 1 -- 172.21.15.47:0/668863265 >> 172.21.15.47:0/668863265 conn(0x55c3b5539820 msgr2=0x55c3b5539c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:13.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.118+0000 7f14b69c3740 1 -- 172.21.15.47:0/668863265 shutdown_connections 2024-03-21T11:27:13.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.118+0000 7f14b69c3740 1 -- 172.21.15.47:0/668863265 wait complete. 2024-03-21T11:27:13.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.118+0000 7f14b69c3740 1 -- start start 2024-03-21T11:27:13.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.119+0000 7f14b69c3740 1 Processor -- start 2024-03-21T11:27:13.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.119+0000 7f14b69c3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b553e2b0 0x55c3b5548170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.119+0000 7f14b69c3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553eca0 0x55c3b55486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.119+0000 7f14b69c3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b5543e10 0x55c3b554f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.119+0000 7f14b69c3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b544ca60 con 0x55c3b553eca0 2024-03-21T11:27:13.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.119+0000 7f14b69c3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b545feb0 con 0x55c3b5543e10 2024-03-21T11:27:13.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.119+0000 7f14b69c3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b5546b80 con 0x55c3b553e2b0 2024-03-21T11:27:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.119+0000 7f14b69c3740 10 client.0 _ll_register_callbacks cb 0x55c3b5454150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f14b1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b553e2b0 0x55c3b5548170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f14abe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553eca0 0x55c3b55486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f14b1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b5543e10 0x55c3b554f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f14b1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b553e2b0 0x55c3b5548170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44272/0 (socket says 172.21.15.47:44272) 2024-03-21T11:27:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f14b1200640 1 -- 172.21.15.47:0/2515053380 learned_addr learned my addr 172.21.15.47:0/2515053380 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f14b1200640 1 -- 172.21.15.47:0/2515053380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b5543e10 msgr2=0x55c3b554f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f14b1200640 1 --2- 172.21.15.47:0/2515053380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b5543e10 0x55c3b554f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f14b1200640 1 -- 172.21.15.47:0/2515053380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553eca0 msgr2=0x55c3b55486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f14b1200640 1 --2- 172.21.15.47:0/2515053380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553eca0 0x55c3b55486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f14b1200640 1 -- 172.21.15.47:0/2515053380 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3b52e0f70 con 0x55c3b553e2b0 2024-03-21T11:27:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f14b1200640 1 --2- 172.21.15.47:0/2515053380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b553e2b0 0x55c3b5548170 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f14a00ef2b0 tx=0x7f14a00b9f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6056ec3054328fa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f1492a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f1492a00640 1 -- 172.21.15.47:0/2515053380 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a00bd6b0 con 0x55c3b553e2b0 2024-03-21T11:27:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f1492a00640 1 -- 172.21.15.47:0/2515053380 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f14a00bbe70 con 0x55c3b553e2b0 2024-03-21T11:27:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.120+0000 7f14b69c3740 1 -- 172.21.15.47:0/2515053380 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3b54b13f0 con 0x55c3b553e2b0 2024-03-21T11:27:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.121+0000 7f1492a00640 1 -- 172.21.15.47:0/2515053380 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14a00bd940 con 0x55c3b553e2b0 2024-03-21T11:27:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.121+0000 7f14b69c3740 10 client.26743 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.121+0000 7f14b69c3740 1 -- 172.21.15.47:0/2515053380 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c3b5178170 con 0x55c3b553e2b0 2024-03-21T11:27:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.121+0000 7f1492a00640 1 -- 172.21.15.47:0/2515053380 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c3b54b13f0 con 0x55c3b553e2b0 2024-03-21T11:27:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.121+0000 7f1492a00640 1 -- 172.21.15.47:0/2515053380 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f14a00dc3e0 con 0x55c3b553e2b0 2024-03-21T11:27:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.121+0000 7f14b69c3740 10 client.26743 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.121+0000 7f14b69c3740 10 client.26743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.121+0000 7f14b69c3740 20 client.26743 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.121+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2515053380 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c3b5553e20 0x55c3b5556230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.121+0000 7f14b69c3740 4 client.26743 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:13.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.121+0000 7f14b69c3740 1 -- 172.21.15.47:0/2515053380 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3b54ab740 con 0x55c3b5553e20 2024-03-21T11:27:13.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.122+0000 7f14abe00640 1 --2- 172.21.15.47:0/2515053380 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c3b5553e20 0x55c3b5556230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.122+0000 7f14abe00640 1 --2- 172.21.15.47:0/2515053380 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c3b5553e20 0x55c3b5556230 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ff9cb2ff159b2e server_cookie=daaa225244c19422 in_seq=0 out_seq=0 2024-03-21T11:27:13.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.122+0000 7f1492a00640 10 client.26743 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f1492a00640 1 -- 172.21.15.47:0/2515053380 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c3b54ab740 con 0x55c3b5553e20 2024-03-21T11:27:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f1492a00640 10 client.26743 handle_command_reply: tid=0 2024-03-21T11:27:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f14b69c3740 1 client.26743 shutdown 2024-03-21T11:27:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f14b69c3740 1 -- 172.21.15.47:0/2515053380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b553e2b0 msgr2=0x55c3b5548170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2515053380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b553e2b0 0x55c3b5548170 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f14a00ef2b0 tx=0x7f14a00b9f00 comp rx=0 tx=0).stop 2024-03-21T11:27:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f14b69c3740 1 -- 172.21.15.47:0/2515053380 shutdown_connections 2024-03-21T11:27:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2515053380 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c3b5553e20 0x55c3b5556230 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2515053380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b5543e10 0x55c3b554f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2515053380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553eca0 0x55c3b55486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2515053380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b553e2b0 0x55c3b5548170 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f1492a00640 0 client.26743 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f14b69c3740 1 -- 172.21.15.47:0/2515053380 >> 172.21.15.47:0/2515053380 conn(0x55c3b5539820 msgr2=0x55c3b5538a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f14b69c3740 1 -- 172.21.15.47:0/2515053380 shutdown_connections 2024-03-21T11:27:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f14b69c3740 1 -- 172.21.15.47:0/2515053380 wait complete. 2024-03-21T11:27:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.124+0000 7f14b69c3740 20 client.26743 trim_cache size 0 max 16384 2024-03-21T11:27:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.144+0000 7f14b69c3740 1 Processor -- start 2024-03-21T11:27:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.144+0000 7f14b69c3740 1 -- start start 2024-03-21T11:27:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b69c3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553e2b0 0x55c3b553f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b69c3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553eca0 0x55c3b554f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b69c3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 0x55c3b553fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b69c3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac1ac060 con 0x55c3b553eca0 2024-03-21T11:27:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b69c3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac1ad9e0 con 0x55c3b553e2b0 2024-03-21T11:27:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b69c3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac1ac6c0 con 0x55c3b5543e10 2024-03-21T11:27:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553e2b0 0x55c3b553f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14abe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553eca0 0x55c3b554f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553e2b0 0x55c3b553f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59658/0 (socket says 172.21.15.47:59658) 2024-03-21T11:27:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b1200640 1 -- 172.21.15.47:0/2718266279 learned_addr learned my addr 172.21.15.47:0/2718266279 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b1c00640 1 --2- 172.21.15.47:0/2718266279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 0x55c3b553fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b1200640 1 -- 172.21.15.47:0/2718266279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 msgr2=0x55c3b553fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b1200640 1 --2- 172.21.15.47:0/2718266279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 0x55c3b553fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b1200640 1 -- 172.21.15.47:0/2718266279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553eca0 msgr2=0x55c3b554f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b1200640 1 --2- 172.21.15.47:0/2718266279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553eca0 0x55c3b554f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b1200640 1 -- 172.21.15.47:0/2718266279 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3b553a270 con 0x55c3b553e2b0 2024-03-21T11:27:13.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.145+0000 7f14b1200640 1 --2- 172.21.15.47:0/2718266279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553e2b0 0x55c3b553f710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f14a00bd870 tx=0x7f14a0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8a0a72c93ffe987 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:13.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.146+0000 7f149d600640 1 -- 172.21.15.47:0/2718266279 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a00bd6b0 con 0x55c3b553e2b0 2024-03-21T11:27:13.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.146+0000 7f149d600640 1 -- 172.21.15.47:0/2718266279 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14a00be270 con 0x55c3b553e2b0 2024-03-21T11:27:13.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.146+0000 7f149d600640 1 -- 172.21.15.47:0/2718266279 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0004430 con 0x55c3b553e2b0 2024-03-21T11:27:13.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.146+0000 7f14b69c3740 1 -- 172.21.15.47:0/2718266279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553e2b0 msgr2=0x55c3b553f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.146+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2718266279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553e2b0 0x55c3b553f710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f14a00bd870 tx=0x7f14a0009260 comp rx=0 tx=0).stop 2024-03-21T11:27:13.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.147+0000 7f14b69c3740 1 -- 172.21.15.47:0/2718266279 shutdown_connections 2024-03-21T11:27:13.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.147+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2718266279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 0x55c3b553fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.147+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2718266279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553eca0 0x55c3b554f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.147+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2718266279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553e2b0 0x55c3b553f710 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.147+0000 7f14b69c3740 1 -- 172.21.15.47:0/2718266279 >> 172.21.15.47:0/2718266279 conn(0x55c3b5539820 msgr2=0x55c3b553d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:13.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.147+0000 7f14b69c3740 1 -- 172.21.15.47:0/2718266279 shutdown_connections 2024-03-21T11:27:13.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.147+0000 7f14b69c3740 1 -- 172.21.15.47:0/2718266279 wait complete. 2024-03-21T11:27:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.147+0000 7f14b69c3740 1 -- start start 2024-03-21T11:27:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.147+0000 7f14b69c3740 1 Processor -- start 2024-03-21T11:27:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14b69c3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553e2b0 0x55c3b554f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14b69c3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553eca0 0x55c3b554f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14b69c3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 0x55c3b554d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14b69c3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac1ad380 con 0x55c3b553e2b0 2024-03-21T11:27:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14b69c3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac1acd20 con 0x55c3b553eca0 2024-03-21T11:27:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14b69c3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac1aa220 con 0x55c3b5543e10 2024-03-21T11:27:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14b69c3740 10 client.0 _ll_register_callbacks cb 0x7f14980097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553eca0 0x55c3b554f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14b1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553e2b0 0x55c3b554f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553eca0 0x55c3b554f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59662/0 (socket says 172.21.15.47:59662) 2024-03-21T11:27:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14abe00640 1 -- 172.21.15.47:0/2146701667 learned_addr learned my addr 172.21.15.47:0/2146701667 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14b1c00640 1 --2- 172.21.15.47:0/2146701667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 0x55c3b554d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14abe00640 1 -- 172.21.15.47:0/2146701667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 msgr2=0x55c3b554d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14abe00640 1 --2- 172.21.15.47:0/2146701667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 0x55c3b554d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14abe00640 1 -- 172.21.15.47:0/2146701667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553e2b0 msgr2=0x55c3b554f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14abe00640 1 --2- 172.21.15.47:0/2146701667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553e2b0 0x55c3b554f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.148+0000 7f14abe00640 1 -- 172.21.15.47:0/2146701667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3b554e0b0 con 0x55c3b553eca0 2024-03-21T11:27:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.149+0000 7f14abe00640 1 --2- 172.21.15.47:0/2146701667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553eca0 0x55c3b554f6e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f14ac19af80 tx=0x7f1498020f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad2af41509900da server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.149+0000 7f1492a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.149+0000 7f1492a00640 1 -- 172.21.15.47:0/2146701667 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498010ba0 con 0x55c3b553eca0 2024-03-21T11:27:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.149+0000 7f14b69c3740 1 -- 172.21.15.47:0/2146701667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14a00be270 con 0x55c3b553eca0 2024-03-21T11:27:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.149+0000 7f1492a00640 1 -- 172.21.15.47:0/2146701667 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1498010d40 con 0x55c3b553eca0 2024-03-21T11:27:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.149+0000 7f1492a00640 1 -- 172.21.15.47:0/2146701667 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1498030050 con 0x55c3b553eca0 2024-03-21T11:27:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.149+0000 7f14b69c3740 10 client.26726 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.149+0000 7f14b69c3740 1 -- 172.21.15.47:0/2146701667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c3b55b03e0 con 0x55c3b553eca0 2024-03-21T11:27:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.150+0000 7f1492a00640 1 -- 172.21.15.47:0/2146701667 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14a00be270 con 0x55c3b553eca0 2024-03-21T11:27:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.150+0000 7f1492a00640 1 -- 172.21.15.47:0/2146701667 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f14980283c0 con 0x55c3b553eca0 2024-03-21T11:27:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.150+0000 7f14b69c3740 10 client.26726 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.150+0000 7f14b69c3740 10 client.26726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.150+0000 7f14b69c3740 20 client.26726 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.150+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2146701667 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c3b55b0d20 0x55c3b55b3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.150+0000 7f14b69c3740 4 client.26726 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:27:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.150+0000 7f14b69c3740 1 -- 172.21.15.47:0/2146701667 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c3b5429a40 con 0x55c3b55b0d20 2024-03-21T11:27:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.150+0000 7f14b1200640 1 --2- 172.21.15.47:0/2146701667 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c3b55b0d20 0x55c3b55b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.151+0000 7f14b1200640 1 --2- 172.21.15.47:0/2146701667 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c3b55b0d20 0x55c3b55b3140 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99c978c20d0a975e server_cookie=eb0fc66b851c20d3 in_seq=0 out_seq=0 2024-03-21T11:27:13.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.151+0000 7f1492a00640 10 client.26726 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:13.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.152+0000 7f1492a00640 1 -- 172.21.15.47:0/2146701667 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c3b5429a40 con 0x55c3b55b0d20 2024-03-21T11:27:13.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.152+0000 7f1492a00640 10 client.26726 handle_command_reply: tid=0 2024-03-21T11:27:13.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.152+0000 7f14b69c3740 1 client.26726 shutdown 2024-03-21T11:27:13.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.152+0000 7f14b69c3740 1 -- 172.21.15.47:0/2146701667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553eca0 msgr2=0x55c3b554f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.152+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2146701667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553eca0 0x55c3b554f6e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f14ac19af80 tx=0x7f1498020f00 comp rx=0 tx=0).stop 2024-03-21T11:27:13.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.152+0000 7f14b69c3740 1 -- 172.21.15.47:0/2146701667 shutdown_connections 2024-03-21T11:27:13.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.152+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2146701667 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c3b55b0d20 0x55c3b55b3140 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.152+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2146701667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b5543e10 0x55c3b554d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.152+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2146701667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b553eca0 0x55c3b554f6e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.152+0000 7f14b69c3740 1 --2- 172.21.15.47:0/2146701667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b553e2b0 0x55c3b554f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.152+0000 7f14b69c3740 1 -- 172.21.15.47:0/2146701667 >> 172.21.15.47:0/2146701667 conn(0x55c3b5539820 msgr2=0x55c3b55381f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:13.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.152+0000 7f1492a00640 0 client.26726 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:13.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.152+0000 7f14b69c3740 1 -- 172.21.15.47:0/2146701667 shutdown_connections 2024-03-21T11:27:13.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.152+0000 7f14b69c3740 1 -- 172.21.15.47:0/2146701667 wait complete. 2024-03-21T11:27:13.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.153+0000 7f14b69c3740 20 client.26726 trim_cache size 0 max 16384 2024-03-21T11:27:13.156 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:27:13.156 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:27:13.156 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "a00aa6fb-22bd-4505-b6da-651d2d8df88f", 2024-03-21T11:27:13.156 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:27:13.157 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:27:13.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.153+0000 7f14b2600640 1 -- 172.21.15.47:0/739847939 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14940a2fa0 msgr2=0x7f14940a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.153+0000 7f14b2600640 1 --2- 172.21.15.47:0/739847939 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14940a2fa0 0x7f14940a5460 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55c3b553f500 tx=0x7f14a4004a40 comp rx=0 tx=0).stop 2024-03-21T11:27:13.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.153+0000 7f14b2600640 1 -- 172.21.15.47:0/739847939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14ac0ff2f0 msgr2=0x7f14ac199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.153+0000 7f14b2600640 1 --2- 172.21.15.47:0/739847939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14ac0ff2f0 0x7f14ac199a70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f14a0000c00 tx=0x7f14a002c650 comp rx=0 tx=0).stop 2024-03-21T11:27:13.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.153+0000 7f14b2600640 1 -- 172.21.15.47:0/739847939 shutdown_connections 2024-03-21T11:27:13.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.153+0000 7f14b2600640 1 --2- 172.21.15.47:0/739847939 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14940a2fa0 0x7f14940a5460 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.153+0000 7f14b2600640 1 --2- 172.21.15.47:0/739847939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14ac1071e0 0x7f14ac1a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.153+0000 7f14b2600640 1 --2- 172.21.15.47:0/739847939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14ac0ffc50 0x7f14ac199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.153+0000 7f14b2600640 1 --2- 172.21.15.47:0/739847939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14ac0ff2f0 0x7f14ac199a70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.153+0000 7f14b2600640 1 -- 172.21.15.47:0/739847939 >> 172.21.15.47:0/739847939 conn(0x7f14ac0fad20 msgr2=0x7f14ac0fb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:13.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.153+0000 7f14b2600640 1 -- 172.21.15.47:0/739847939 shutdown_connections 2024-03-21T11:27:13.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.154+0000 7f14b2600640 1 -- 172.21.15.47:0/739847939 wait complete. 2024-03-21T11:27:13.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:27:13.174 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:27:13.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.389+0000 7f1284c00640 1 -- 172.21.15.47:0/3158951796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1280076060 msgr2=0x7f1280076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.389+0000 7f1284c00640 1 --2- 172.21.15.47:0/3158951796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1280076060 0x7f1280076480 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1270009ca0 tx=0x7f127002c100 comp rx=0 tx=0).stop 2024-03-21T11:27:13.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.389+0000 7f1284c00640 1 -- 172.21.15.47:0/3158951796 shutdown_connections 2024-03-21T11:27:13.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.389+0000 7f1284c00640 1 --2- 172.21.15.47:0/3158951796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1280077c40 0x7f12800784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.389+0000 7f1284c00640 1 --2- 172.21.15.47:0/3158951796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12800772a0 0x7f1280077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.389+0000 7f1284c00640 1 --2- 172.21.15.47:0/3158951796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1280076060 0x7f1280076480 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.389+0000 7f1284c00640 1 -- 172.21.15.47:0/3158951796 >> 172.21.15.47:0/3158951796 conn(0x7f12800fee50 msgr2=0x7f1280101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:13.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.389+0000 7f1284c00640 1 -- 172.21.15.47:0/3158951796 shutdown_connections 2024-03-21T11:27:13.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.389+0000 7f1284c00640 1 -- 172.21.15.47:0/3158951796 wait complete. 2024-03-21T11:27:13.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.390+0000 7f1284c00640 1 Processor -- start 2024-03-21T11:27:13.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.390+0000 7f1284c00640 1 -- start start 2024-03-21T11:27:13.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f1284c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1280076060 0x7f128019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f1284c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12800772a0 0x7f128019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f1284c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1280077c40 0x7f12801a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f1284c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1280113970 con 0x7f12800772a0 2024-03-21T11:27:13.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f1284c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12801137f0 con 0x7f1280076060 2024-03-21T11:27:13.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f1284c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1280113af0 con 0x7f1280077c40 2024-03-21T11:27:13.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f127de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1280076060 0x7f128019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f127de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1280076060 0x7f128019e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59680/0 (socket says 172.21.15.47:59680) 2024-03-21T11:27:13.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f127de00640 1 -- 172.21.15.47:0/1176595577 learned_addr learned my addr 172.21.15.47:0/1176595577 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:13.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f127d400640 1 --2- 172.21.15.47:0/1176595577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12800772a0 0x7f128019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f127e800640 1 --2- 172.21.15.47:0/1176595577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1280077c40 0x7f12801a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f127de00640 1 -- 172.21.15.47:0/1176595577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1280077c40 msgr2=0x7f12801a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f127de00640 1 --2- 172.21.15.47:0/1176595577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1280077c40 0x7f12801a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f127de00640 1 -- 172.21.15.47:0/1176595577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12800772a0 msgr2=0x7f128019e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.391+0000 7f127de00640 1 --2- 172.21.15.47:0/1176595577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12800772a0 0x7f128019e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.392+0000 7f127de00640 1 -- 172.21.15.47:0/1176595577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12801a5c40 con 0x7f1280076060 2024-03-21T11:27:13.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.392+0000 7f127de00640 1 --2- 172.21.15.47:0/1176595577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1280076060 0x7f128019e020 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f12700325f0 tx=0x7f1270035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:13.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.392+0000 7f1276a00640 1 -- 172.21.15.47:0/1176595577 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1270039040 con 0x7f1280076060 2024-03-21T11:27:13.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.392+0000 7f1284c00640 1 -- 172.21.15.47:0/1176595577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12801a5ed0 con 0x7f1280076060 2024-03-21T11:27:13.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.392+0000 7f1276a00640 1 -- 172.21.15.47:0/1176595577 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1270044070 con 0x7f1280076060 2024-03-21T11:27:13.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.392+0000 7f1276a00640 1 -- 172.21.15.47:0/1176595577 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1270004460 con 0x7f1280076060 2024-03-21T11:27:13.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.392+0000 7f1284c00640 1 -- 172.21.15.47:0/1176595577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12801a62d0 con 0x7f1280076060 2024-03-21T11:27:13.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.396+0000 7f12879f9740 1 Processor -- start 2024-03-21T11:27:13.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.396+0000 7f1276a00640 1 -- 172.21.15.47:0/1176595577 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1270004600 con 0x7f1280076060 2024-03-21T11:27:13.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.396+0000 7f12879f9740 1 -- start start 2024-03-21T11:27:13.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.396+0000 7f12879f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b1ae80 0x562ea4b1b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.396+0000 7f12879f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1b870 0x562ea4b20370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.396+0000 7f12879f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b209e0 0x562ea4b22e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.396+0000 7f12879f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4a80250 con 0x562ea4b209e0 2024-03-21T11:27:13.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.396+0000 7f12879f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4a28a30 con 0x562ea4b1b870 2024-03-21T11:27:13.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.396+0000 7f12879f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4a7c2f0 con 0x562ea4b1ae80 2024-03-21T11:27:13.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.397+0000 7f127e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b1ae80 0x562ea4b1b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.397+0000 7f127e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b1ae80 0x562ea4b1b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44302/0 (socket says 172.21.15.47:44302) 2024-03-21T11:27:13.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.397+0000 7f127e800640 1 -- 172.21.15.47:0/2215284967 learned_addr learned my addr 172.21.15.47:0/2215284967 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:13.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.397+0000 7f127de00640 1 --2- 172.21.15.47:0/2215284967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1b870 0x562ea4b20370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.397+0000 7f127d400640 1 --2- 172.21.15.47:0/2215284967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b209e0 0x562ea4b22e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.397+0000 7f127e800640 1 -- 172.21.15.47:0/2215284967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1b870 msgr2=0x562ea4b20370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.397+0000 7f127e800640 1 --2- 172.21.15.47:0/2215284967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1b870 0x562ea4b20370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.397+0000 7f127e800640 1 -- 172.21.15.47:0/2215284967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b209e0 msgr2=0x562ea4b22e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.397+0000 7f127e800640 1 --2- 172.21.15.47:0/2215284967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b209e0 0x562ea4b22e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.397+0000 7f127e800640 1 -- 172.21.15.47:0/2215284967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ea4a13690 con 0x562ea4b1ae80 2024-03-21T11:27:13.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.398+0000 7f127e800640 1 --2- 172.21.15.47:0/2215284967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b1ae80 0x562ea4b1b2a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1278009c90 tx=0x7f1278011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b8d1e1ced35525c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:13.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.398+0000 7f126f000640 1 -- 172.21.15.47:0/2215284967 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127801e070 con 0x562ea4b1ae80 2024-03-21T11:27:13.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.398+0000 7f126f000640 1 -- 172.21.15.47:0/2215284967 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1278009e40 con 0x562ea4b1ae80 2024-03-21T11:27:13.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.398+0000 7f126f000640 1 -- 172.21.15.47:0/2215284967 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127800eab0 con 0x562ea4b1ae80 2024-03-21T11:27:13.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.398+0000 7f1276a00640 1 --2- 172.21.15.47:0/1176595577 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12640a2f20 0x7f12640a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.398+0000 7f127de00640 1 --2- 172.21.15.47:0/1176595577 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12640a2f20 0x7f12640a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.399+0000 7f1276a00640 1 -- 172.21.15.47:0/1176595577 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f12700f1cd0 con 0x7f1280076060 2024-03-21T11:27:13.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.399+0000 7f12879f9740 1 -- 172.21.15.47:0/2215284967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b1ae80 msgr2=0x562ea4b1b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.399+0000 7f12879f9740 1 --2- 172.21.15.47:0/2215284967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b1ae80 0x562ea4b1b2a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1278009c90 tx=0x7f1278011810 comp rx=0 tx=0).stop 2024-03-21T11:27:13.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.399+0000 7f12879f9740 1 -- 172.21.15.47:0/2215284967 shutdown_connections 2024-03-21T11:27:13.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.399+0000 7f12879f9740 1 --2- 172.21.15.47:0/2215284967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b209e0 0x562ea4b22e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.399+0000 7f12879f9740 1 --2- 172.21.15.47:0/2215284967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1b870 0x562ea4b20370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.399+0000 7f12879f9740 1 --2- 172.21.15.47:0/2215284967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b1ae80 0x562ea4b1b2a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.399+0000 7f12879f9740 1 -- 172.21.15.47:0/2215284967 >> 172.21.15.47:0/2215284967 conn(0x562ea4ae3200 msgr2=0x562ea4b167f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.399+0000 7f12879f9740 1 -- 172.21.15.47:0/2215284967 shutdown_connections 2024-03-21T11:27:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.399+0000 7f127de00640 1 --2- 172.21.15.47:0/1176595577 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12640a2f20 0x7f12640a53e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f12700bdfa0 tx=0x7f12700f5710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.399+0000 7f12879f9740 1 -- 172.21.15.47:0/2215284967 wait complete. 2024-03-21T11:27:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.400+0000 7f12879f9740 1 -- start start 2024-03-21T11:27:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.400+0000 7f12879f9740 1 Processor -- start 2024-03-21T11:27:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.400+0000 7f12879f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1ae80 0x562ea4b24e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.400+0000 7f12879f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1b870 0x562ea4b25390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.400+0000 7f12879f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b209e0 0x562ea4b2c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.400+0000 7f12879f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4b23990 con 0x562ea4b1b870 2024-03-21T11:27:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.400+0000 7f12879f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4a3be80 con 0x562ea4b1ae80 2024-03-21T11:27:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.401+0000 7f12879f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea4b23b10 con 0x562ea4b209e0 2024-03-21T11:27:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.401+0000 7f12879f9740 10 client.0 _ll_register_callbacks cb 0x562ea4a4e130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.401+0000 7f127d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1b870 0x562ea4b25390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.401+0000 7f127e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1ae80 0x562ea4b24e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.401+0000 7f127d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1b870 0x562ea4b25390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38818/0 (socket says 172.21.15.47:38818) 2024-03-21T11:27:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.401+0000 7f127d400640 1 -- 172.21.15.47:0/3672662232 learned_addr learned my addr 172.21.15.47:0/3672662232 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.401+0000 7f127e800640 1 --2- 172.21.15.47:0/3672662232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b209e0 0x562ea4b2c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.401+0000 7f127d400640 1 -- 172.21.15.47:0/3672662232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b209e0 msgr2=0x562ea4b2c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.401+0000 7f127d400640 1 --2- 172.21.15.47:0/3672662232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b209e0 0x562ea4b2c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.401+0000 7f127d400640 1 -- 172.21.15.47:0/3672662232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1ae80 msgr2=0x562ea4b24e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.401+0000 7f127d400640 1 --2- 172.21.15.47:0/3672662232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1ae80 0x562ea4b24e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.401+0000 7f127d400640 1 -- 172.21.15.47:0/3672662232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ea485e7d0 con 0x562ea4b1b870 2024-03-21T11:27:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.402+0000 7f127d400640 1 --2- 172.21.15.47:0/3672662232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1b870 0x562ea4b25390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1268013710 tx=0x7f1268013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ed570ae82687df server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.402+0000 7f1263400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:27:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.402+0000 7f1263400640 1 -- 172.21.15.47:0/3672662232 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12680118e0 con 0x562ea4b1b870 2024-03-21T11:27:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.402+0000 7f1263400640 1 -- 172.21.15.47:0/3672662232 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1268011a80 con 0x562ea4b1b870 2024-03-21T11:27:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.402+0000 7f12879f9740 1 -- 172.21.15.47:0/3672662232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ea4a8d790 con 0x562ea4b1b870 2024-03-21T11:27:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.402+0000 7f1263400640 1 -- 172.21.15.47:0/3672662232 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1268011c80 con 0x562ea4b1b870 2024-03-21T11:27:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.402+0000 7f12879f9740 10 client.16509 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.402+0000 7f12879f9740 1 -- 172.21.15.47:0/3672662232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562ea47943f0 con 0x562ea4b1b870 2024-03-21T11:27:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.403+0000 7f1263400640 1 -- 172.21.15.47:0/3672662232 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562ea4a8d790 con 0x562ea4b1b870 2024-03-21T11:27:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.403+0000 7f1263400640 1 -- 172.21.15.47:0/3672662232 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f126801a070 con 0x562ea4b1b870 2024-03-21T11:27:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.403+0000 7f12879f9740 10 client.16509 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.403+0000 7f12879f9740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.403+0000 7f12879f9740 20 client.16509 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.403+0000 7f12879f9740 1 --2- 172.21.15.47:0/3672662232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ea4b30c60 0x562ea4b330a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.403+0000 7f12879f9740 4 client.16509 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.403+0000 7f12879f9740 1 -- 172.21.15.47:0/3672662232 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ea4a87ae0 con 0x562ea4b30c60 2024-03-21T11:27:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.403+0000 7f127e800640 1 --2- 172.21.15.47:0/3672662232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ea4b30c60 0x562ea4b330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.404+0000 7f127e800640 1 --2- 172.21.15.47:0/3672662232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ea4b30c60 0x562ea4b330a0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33436bc651a472c3 server_cookie=c61744922ad3e3e2 in_seq=0 out_seq=0 2024-03-21T11:27:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.404+0000 7f1263400640 10 client.16509 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.405+0000 7f1263400640 1 -- 172.21.15.47:0/3672662232 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562ea4a87ae0 con 0x562ea4b30c60 2024-03-21T11:27:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.406+0000 7f1263400640 10 client.16509 handle_command_reply: tid=0 2024-03-21T11:27:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.406+0000 7f12879f9740 1 client.16509 shutdown 2024-03-21T11:27:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.406+0000 7f12879f9740 1 -- 172.21.15.47:0/3672662232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1b870 msgr2=0x562ea4b25390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.406+0000 7f12879f9740 1 --2- 172.21.15.47:0/3672662232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1b870 0x562ea4b25390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1268013710 tx=0x7f1268013bd0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.406+0000 7f12879f9740 1 -- 172.21.15.47:0/3672662232 shutdown_connections 2024-03-21T11:27:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.406+0000 7f12879f9740 1 --2- 172.21.15.47:0/3672662232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ea4b30c60 0x562ea4b330a0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.406+0000 7f12879f9740 1 --2- 172.21.15.47:0/3672662232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b209e0 0x562ea4b2c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.406+0000 7f12879f9740 1 --2- 172.21.15.47:0/3672662232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1b870 0x562ea4b25390 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.406+0000 7f12879f9740 1 --2- 172.21.15.47:0/3672662232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1ae80 0x562ea4b24e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.406+0000 7f1263400640 0 client.16509 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.406+0000 7f12879f9740 1 -- 172.21.15.47:0/3672662232 >> 172.21.15.47:0/3672662232 conn(0x562ea4ae3200 msgr2=0x562ea4b15a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.406+0000 7f12879f9740 1 -- 172.21.15.47:0/3672662232 shutdown_connections 2024-03-21T11:27:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.406+0000 7f12879f9740 1 -- 172.21.15.47:0/3672662232 wait complete. 2024-03-21T11:27:13.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.406+0000 7f12879f9740 20 client.16509 trim_cache size 0 max 16384 2024-03-21T11:27:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.427+0000 7f12879f9740 1 Processor -- start 2024-03-21T11:27:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.428+0000 7f12879f9740 1 -- start start 2024-03-21T11:27:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.428+0000 7f12879f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1ae80 0x562ea4b15200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.428+0000 7f12879f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b1b870 0x562ea4b2c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.428+0000 7f12879f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b209e0 0x562ea4b15740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.428+0000 7f12879f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12801b0700 con 0x562ea4b1ae80 2024-03-21T11:27:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.428+0000 7f12879f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12801b2080 con 0x562ea4b209e0 2024-03-21T11:27:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.428+0000 7f12879f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12801b0d60 con 0x562ea4b1b870 2024-03-21T11:27:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.428+0000 7f127d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b1b870 0x562ea4b2c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.428+0000 7f127d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b1b870 0x562ea4b2c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44328/0 (socket says 172.21.15.47:44328) 2024-03-21T11:27:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.428+0000 7f127d400640 1 -- 172.21.15.47:0/3318754222 learned_addr learned my addr 172.21.15.47:0/3318754222 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.428+0000 7f127e800640 1 --2- 172.21.15.47:0/3318754222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1ae80 0x562ea4b15200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.429+0000 7f127e800640 1 --2- 172.21.15.47:0/3318754222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b209e0 0x562ea4b15740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.429+0000 7f127d400640 1 -- 172.21.15.47:0/3318754222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b209e0 msgr2=0x562ea4b15740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.429+0000 7f127d400640 1 --2- 172.21.15.47:0/3318754222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b209e0 0x562ea4b15740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.429+0000 7f127d400640 1 -- 172.21.15.47:0/3318754222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1ae80 msgr2=0x562ea4b15200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.429+0000 7f127d400640 1 --2- 172.21.15.47:0/3318754222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1ae80 0x562ea4b15200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.429+0000 7f127d400640 1 -- 172.21.15.47:0/3318754222 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ea47540f0 con 0x562ea4b1b870 2024-03-21T11:27:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.429+0000 7f127e800640 1 --2- 172.21.15.47:0/3318754222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b209e0 0x562ea4b15740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:27:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.429+0000 7f127d400640 1 --2- 172.21.15.47:0/3318754222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b1b870 0x562ea4b2c390 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1268004cb0 tx=0x7f1268009d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c4b30d667baa2a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.429+0000 7f126e600640 1 -- 172.21.15.47:0/3318754222 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268033070 con 0x562ea4b1b870 2024-03-21T11:27:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.430+0000 7f126e600640 1 -- 172.21.15.47:0/3318754222 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12680119b0 con 0x562ea4b1b870 2024-03-21T11:27:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.430+0000 7f126e600640 1 -- 172.21.15.47:0/3318754222 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126802a040 con 0x562ea4b1b870 2024-03-21T11:27:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.430+0000 7f12879f9740 1 -- 172.21.15.47:0/3318754222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b1b870 msgr2=0x562ea4b2c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.430+0000 7f12879f9740 1 --2- 172.21.15.47:0/3318754222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b1b870 0x562ea4b2c390 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1268004cb0 tx=0x7f1268009d90 comp rx=0 tx=0).stop 2024-03-21T11:27:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.430+0000 7f12879f9740 1 -- 172.21.15.47:0/3318754222 shutdown_connections 2024-03-21T11:27:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.430+0000 7f12879f9740 1 --2- 172.21.15.47:0/3318754222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b209e0 0x562ea4b15740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.430+0000 7f12879f9740 1 --2- 172.21.15.47:0/3318754222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b1b870 0x562ea4b2c390 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.430+0000 7f12879f9740 1 --2- 172.21.15.47:0/3318754222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1ae80 0x562ea4b15200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.430+0000 7f12879f9740 1 -- 172.21.15.47:0/3318754222 >> 172.21.15.47:0/3318754222 conn(0x562ea4ae3200 msgr2=0x562ea4b1cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.431+0000 7f12879f9740 1 -- 172.21.15.47:0/3318754222 shutdown_connections 2024-03-21T11:27:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.431+0000 7f12879f9740 1 -- 172.21.15.47:0/3318754222 wait complete. 2024-03-21T11:27:13.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.431+0000 7f12879f9740 1 -- start start 2024-03-21T11:27:13.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.431+0000 7f12879f9740 1 Processor -- start 2024-03-21T11:27:13.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f12879f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1ae80 0x562ea4b29bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f12879f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1b870 0x562ea4b2a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f12879f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b209e0 0x562ea4b2b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f12879f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12801b1a20 con 0x562ea4b1ae80 2024-03-21T11:27:13.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f12879f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12801b13c0 con 0x562ea4b1b870 2024-03-21T11:27:13.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f12879f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12801ae8c0 con 0x562ea4b209e0 2024-03-21T11:27:13.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f12879f9740 10 client.0 _ll_register_callbacks cb 0x7f12680097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:13.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f127d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1b870 0x562ea4b2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f127e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1ae80 0x562ea4b29bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f127d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1b870 0x562ea4b2a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59718/0 (socket says 172.21.15.47:59718) 2024-03-21T11:27:13.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f127d400640 1 -- 172.21.15.47:0/3812948046 learned_addr learned my addr 172.21.15.47:0/3812948046 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:13.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f127e800640 1 --2- 172.21.15.47:0/3812948046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b209e0 0x562ea4b2b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f127d400640 1 -- 172.21.15.47:0/3812948046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b209e0 msgr2=0x562ea4b2b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f127d400640 1 --2- 172.21.15.47:0/3812948046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b209e0 0x562ea4b2b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f127d400640 1 -- 172.21.15.47:0/3812948046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1ae80 msgr2=0x562ea4b29bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:13.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f127d400640 1 --2- 172.21.15.47:0/3812948046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1ae80 0x562ea4b29bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.432+0000 7f127d400640 1 -- 172.21.15.47:0/3812948046 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ea4b2b700 con 0x562ea4b1b870 2024-03-21T11:27:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.433+0000 7f127d400640 1 --2- 172.21.15.47:0/3812948046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1b870 0x562ea4b2a100 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1268000c00 tx=0x7f1268004720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6696589c96cf6436 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.433+0000 7f1263400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.433+0000 7f1263400640 1 -- 172.21.15.47:0/3812948046 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268033040 con 0x562ea4b1b870 2024-03-21T11:27:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.433+0000 7f12879f9740 1 -- 172.21.15.47:0/3812948046 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12680119b0 con 0x562ea4b1b870 2024-03-21T11:27:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.433+0000 7f1263400640 1 -- 172.21.15.47:0/3812948046 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1268005060 con 0x562ea4b1b870 2024-03-21T11:27:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.433+0000 7f1263400640 1 -- 172.21.15.47:0/3812948046 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f126802a040 con 0x562ea4b1b870 2024-03-21T11:27:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.433+0000 7f12879f9740 10 client.26741 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.433+0000 7f12879f9740 1 -- 172.21.15.47:0/3812948046 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562ea4b8c5f0 con 0x562ea4b1b870 2024-03-21T11:27:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.434+0000 7f1263400640 1 -- 172.21.15.47:0/3812948046 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12680119b0 con 0x562ea4b1b870 2024-03-21T11:27:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.434+0000 7f1263400640 1 -- 172.21.15.47:0/3812948046 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f126801a050 con 0x562ea4b1b870 2024-03-21T11:27:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.434+0000 7f12879f9740 10 client.26741 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.434+0000 7f12879f9740 10 client.26741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.434+0000 7f12879f9740 20 client.26741 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.434+0000 7f12879f9740 1 --2- 172.21.15.47:0/3812948046 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ea4b8ce50 0x562ea4b8f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.434+0000 7f12879f9740 4 client.26741 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:27:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.434+0000 7f12879f9740 1 -- 172.21.15.47:0/3812948046 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562ea4b24c40 con 0x562ea4b8ce50 2024-03-21T11:27:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.434+0000 7f127e800640 1 --2- 172.21.15.47:0/3812948046 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ea4b8ce50 0x562ea4b8f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.434+0000 7f127e800640 1 --2- 172.21.15.47:0/3812948046 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ea4b8ce50 0x562ea4b8f270 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe544e62287ee6fd server_cookie=896d4c302796ee62 in_seq=0 out_seq=0 2024-03-21T11:27:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.434+0000 7f1263400640 10 client.26741 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f1263400640 1 -- 172.21.15.47:0/3812948046 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562ea4b24c40 con 0x562ea4b8ce50 2024-03-21T11:27:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f1263400640 10 client.26741 handle_command_reply: tid=0 2024-03-21T11:27:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f12879f9740 1 client.26741 shutdown 2024-03-21T11:27:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f12879f9740 1 -- 172.21.15.47:0/3812948046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1b870 msgr2=0x562ea4b2a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f12879f9740 1 --2- 172.21.15.47:0/3812948046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1b870 0x562ea4b2a100 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1268000c00 tx=0x7f1268004720 comp rx=0 tx=0).stop 2024-03-21T11:27:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f12879f9740 1 -- 172.21.15.47:0/3812948046 shutdown_connections 2024-03-21T11:27:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f12879f9740 1 --2- 172.21.15.47:0/3812948046 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ea4b8ce50 0x562ea4b8f270 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f12879f9740 1 --2- 172.21.15.47:0/3812948046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ea4b209e0 0x562ea4b2b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f12879f9740 1 --2- 172.21.15.47:0/3812948046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ea4b1b870 0x562ea4b2a100 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f1263400640 0 client.26741 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f12879f9740 1 --2- 172.21.15.47:0/3812948046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ea4b1ae80 0x562ea4b29bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f12879f9740 1 -- 172.21.15.47:0/3812948046 >> 172.21.15.47:0/3812948046 conn(0x562ea4ae3200 msgr2=0x562ea4b1a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f12879f9740 1 -- 172.21.15.47:0/3812948046 shutdown_connections 2024-03-21T11:27:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f12879f9740 1 -- 172.21.15.47:0/3812948046 wait complete. 2024-03-21T11:27:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.435+0000 7f12879f9740 20 client.26741 trim_cache size 0 max 16384 2024-03-21T11:27:13.442 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:27:13.442 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:27:13.442 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:27:13.442 INFO:teuthology.orchestra.run.smithi047.stdout: "a00aa6fb-22bd-4505-b6da-651d2d8df88f": { 2024-03-21T11:27:13.442 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:27:13.442 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "a00aa6fb-22bd-4505-b6da-651d2d8df88f", 2024-03-21T11:27:13.442 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:27:13.442 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:27:13.442 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:27:13.442 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:27:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.436+0000 7f1284c00640 1 -- 172.21.15.47:0/1176595577 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12640a2f20 msgr2=0x7f12640a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.436+0000 7f1284c00640 1 --2- 172.21.15.47:0/1176595577 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12640a2f20 0x7f12640a53e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f12700bdfa0 tx=0x7f12700f5710 comp rx=0 tx=0).stop 2024-03-21T11:27:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.436+0000 7f1284c00640 1 -- 172.21.15.47:0/1176595577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1280076060 msgr2=0x7f128019e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.436+0000 7f1284c00640 1 --2- 172.21.15.47:0/1176595577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1280076060 0x7f128019e020 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f12700325f0 tx=0x7f1270035d10 comp rx=0 tx=0).stop 2024-03-21T11:27:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.436+0000 7f1284c00640 1 -- 172.21.15.47:0/1176595577 shutdown_connections 2024-03-21T11:27:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.436+0000 7f1284c00640 1 --2- 172.21.15.47:0/1176595577 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12640a2f20 0x7f12640a53e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.436+0000 7f1284c00640 1 --2- 172.21.15.47:0/1176595577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1280077c40 0x7f12801a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.436+0000 7f1284c00640 1 --2- 172.21.15.47:0/1176595577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12800772a0 0x7f128019e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.436+0000 7f1284c00640 1 --2- 172.21.15.47:0/1176595577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1280076060 0x7f128019e020 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:13.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.436+0000 7f1284c00640 1 -- 172.21.15.47:0/1176595577 >> 172.21.15.47:0/1176595577 conn(0x7f12800fee50 msgr2=0x7f1280100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:13.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.437+0000 7f1284c00640 1 -- 172.21.15.47:0/1176595577 shutdown_connections 2024-03-21T11:27:13.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:13.437+0000 7f1284c00640 1 -- 172.21.15.47:0/1176595577 wait complete. 2024-03-21T11:27:13.455 INFO:tasks.cephfs.filesystem:scrub status for tag:a00aa6fb-22bd-4505-b6da-651d2d8df88f - {'path': '/', 'tag': 'a00aa6fb-22bd-4505-b6da-651d2d8df88f', 'options': 'recursive,force'} 2024-03-21T11:27:13.501 INFO:tasks.workunit.client.0.smithi047.stdout: 1 190442 8.90 MB/sec warmup 117 sec latency 16.745 ms 2024-03-21T11:27:13.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:13 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:27:13.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:13 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:13.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:13 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:27:13.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:13 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:13.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:13 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:27:13.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:13 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:13.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:13 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:27:13.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:13 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:14.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:13 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:27:14.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:13 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:14.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:13 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:27:14.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:13 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:14.502 INFO:tasks.workunit.client.0.smithi047.stdout: 1 191945 8.93 MB/sec warmup 118 sec latency 12.075 ms 2024-03-21T11:27:14.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:14 smithi100 ceph-mon[36245]: pgmap v517: 129 pgs: 129 active+clean; 37 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 313 op/s 2024-03-21T11:27:14.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:14 smithi102 ceph-mon[36748]: pgmap v517: 129 pgs: 129 active+clean; 37 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 313 op/s 2024-03-21T11:27:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:14 smithi047 ceph-mon[31351]: pgmap v517: 129 pgs: 129 active+clean; 37 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 313 op/s 2024-03-21T11:27:15.502 INFO:tasks.workunit.client.0.smithi047.stdout: 1 193035 8.90 MB/sec warmup 119 sec latency 57.811 ms 2024-03-21T11:27:16.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:16 smithi100 ceph-mon[36245]: pgmap v518: 129 pgs: 129 active+clean; 33 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 297 op/s 2024-03-21T11:27:16.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:16 smithi102 ceph-mon[36748]: pgmap v518: 129 pgs: 129 active+clean; 33 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 297 op/s 2024-03-21T11:27:17.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:16 smithi047 ceph-mon[31351]: pgmap v518: 129 pgs: 129 active+clean; 33 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 297 op/s 2024-03-21T11:27:17.504 INFO:tasks.workunit.client.0.smithi047.stdout: 1 196075 2.51 MB/sec execute 1 sec latency 16.251 ms 2024-03-21T11:27:17.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:17 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:27:17.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:17 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:27:18.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:17 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:27:18.502 INFO:tasks.workunit.client.0.smithi047.stdout: 1 198111 7.12 MB/sec execute 2 sec latency 19.469 ms 2024-03-21T11:27:18.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:18 smithi100 ceph-mon[36245]: pgmap v519: 129 pgs: 129 active+clean; 36 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 348 op/s 2024-03-21T11:27:18.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:18 smithi102 ceph-mon[36748]: pgmap v519: 129 pgs: 129 active+clean; 36 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 348 op/s 2024-03-21T11:27:19.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:18 smithi047 ceph-mon[31351]: pgmap v519: 129 pgs: 129 active+clean; 36 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 348 op/s 2024-03-21T11:27:19.502 INFO:tasks.workunit.client.0.smithi047.stdout: 1 199371 7.04 MB/sec execute 3 sec latency 9.964 ms 2024-03-21T11:27:20.502 INFO:tasks.workunit.client.0.smithi047.stdout: 1 201057 6.98 MB/sec execute 4 sec latency 48.144 ms 2024-03-21T11:27:20.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:20 smithi100 ceph-mon[36245]: pgmap v520: 129 pgs: 129 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 309 op/s 2024-03-21T11:27:20.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:20 smithi102 ceph-mon[36748]: pgmap v520: 129 pgs: 129 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 309 op/s 2024-03-21T11:27:21.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:20 smithi047 ceph-mon[31351]: pgmap v520: 129 pgs: 129 active+clean; 36 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 309 op/s 2024-03-21T11:27:21.502 INFO:tasks.workunit.client.0.smithi047.stdout: 1 202497 7.92 MB/sec execute 5 sec latency 29.620 ms 2024-03-21T11:27:22.503 INFO:tasks.workunit.client.0.smithi047.stdout: 1 203716 7.50 MB/sec execute 6 sec latency 23.921 ms 2024-03-21T11:27:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.768+0000 7fe9f5800640 1 -- 172.21.15.47:0/2068361497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9f0101520 msgr2=0x7fe9f0101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.768+0000 7fe9f5800640 1 --2- 172.21.15.47:0/2068361497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9f0101520 0x7fe9f0101940 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009900 tx=0x7fe9e002bc40 comp rx=0 tx=0).stop 2024-03-21T11:27:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.768+0000 7fe9f5800640 1 -- 172.21.15.47:0/2068361497 shutdown_connections 2024-03-21T11:27:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.768+0000 7fe9f5800640 1 --2- 172.21.15.47:0/2068361497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9f01030d0 0x7fe9f0109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.768+0000 7fe9f5800640 1 --2- 172.21.15.47:0/2068361497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f0102710 0x7fe9f0102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.768+0000 7fe9f5800640 1 --2- 172.21.15.47:0/2068361497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9f0101520 0x7fe9f0101940 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.768+0000 7fe9f5800640 1 -- 172.21.15.47:0/2068361497 >> 172.21.15.47:0/2068361497 conn(0x7fe9f00fcab0 msgr2=0x7fe9f00feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.768+0000 7fe9f5800640 1 -- 172.21.15.47:0/2068361497 shutdown_connections 2024-03-21T11:27:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.768+0000 7fe9f5800640 1 -- 172.21.15.47:0/2068361497 wait complete. 2024-03-21T11:27:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.769+0000 7fe9f5800640 1 Processor -- start 2024-03-21T11:27:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.769+0000 7fe9f5800640 1 -- start start 2024-03-21T11:27:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9f5800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9f0101520 0x7fe9f0199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9f5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9f0102710 0x7fe9f0199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9f5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f01030d0 0x7fe9f01a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9f5800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f010d200 con 0x7fe9f0101520 2024-03-21T11:27:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9f5800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f010d080 con 0x7fe9f01030d0 2024-03-21T11:27:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9f5800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f010d380 con 0x7fe9f0102710 2024-03-21T11:27:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9efe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9f0101520 0x7fe9f0199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9f4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f01030d0 0x7fe9f01a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9e7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9f0102710 0x7fe9f0199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9f4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f01030d0 0x7fe9f01a0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55950/0 (socket says 172.21.15.47:55950) 2024-03-21T11:27:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9f4e00640 1 -- 172.21.15.47:0/48117423 learned_addr learned my addr 172.21.15.47:0/48117423 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9efe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9f0101520 0x7fe9f0199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48556/0 (socket says 172.21.15.47:48556) 2024-03-21T11:27:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9f4e00640 1 -- 172.21.15.47:0/48117423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9f0102710 msgr2=0x7fe9f0199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9f4e00640 1 --2- 172.21.15.47:0/48117423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9f0102710 0x7fe9f0199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9f4e00640 1 -- 172.21.15.47:0/48117423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9f0101520 msgr2=0x7fe9f0199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9f4e00640 1 --2- 172.21.15.47:0/48117423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9f0101520 0x7fe9f0199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.770+0000 7fe9f4e00640 1 -- 172.21.15.47:0/48117423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f01a1660 con 0x7fe9f01030d0 2024-03-21T11:27:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.771+0000 7fe9f4e00640 1 --2- 172.21.15.47:0/48117423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f01030d0 0x7fe9f01a0ff0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800eaa0 tx=0x7fe9e800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.771+0000 7fe9ed600640 1 -- 172.21.15.47:0/48117423 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e800cdb0 con 0x7fe9f01030d0 2024-03-21T11:27:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.771+0000 7fe9ed600640 1 -- 172.21.15.47:0/48117423 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9e8005020 con 0x7fe9f01030d0 2024-03-21T11:27:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.771+0000 7fe9ed600640 1 -- 172.21.15.47:0/48117423 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80116b0 con 0x7fe9f01030d0 2024-03-21T11:27:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.771+0000 7fe9f5800640 1 -- 172.21.15.47:0/48117423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f01a1950 con 0x7fe9f01030d0 2024-03-21T11:27:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.771+0000 7fe9f5800640 1 -- 172.21.15.47:0/48117423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f01a1e50 con 0x7fe9f01030d0 2024-03-21T11:27:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.774+0000 7fe9ed600640 1 -- 172.21.15.47:0/48117423 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe9e8005560 con 0x7fe9f01030d0 2024-03-21T11:27:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.775+0000 7fe9f9bff740 1 Processor -- start 2024-03-21T11:27:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.775+0000 7fe9f9bff740 1 -- start start 2024-03-21T11:27:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.775+0000 7fe9f9bff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024a3f0 0x561ab024a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.775+0000 7fe9f9bff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab024f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.775+0000 7fe9f9bff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024ff50 0x561ab02523e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.775+0000 7fe9f9bff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab020d3b0 con 0x561ab024a3f0 2024-03-21T11:27:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.775+0000 7fe9f9bff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab01af830 con 0x561ab024ade0 2024-03-21T11:27:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.775+0000 7fe9f9bff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab01585d0 con 0x561ab024ff50 2024-03-21T11:27:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.775+0000 7fe9efe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab024f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.775+0000 7fe9efe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab024f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55960/0 (socket says 172.21.15.47:55960) 2024-03-21T11:27:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.775+0000 7fe9efe00640 1 -- 172.21.15.47:0/4050877965 learned_addr learned my addr 172.21.15.47:0/4050877965 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.776+0000 7fe9e7e00640 1 --2- 172.21.15.47:0/4050877965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024a3f0 0x561ab024a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.776+0000 7fe9e7e00640 1 --2- 172.21.15.47:0/4050877965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024ff50 0x561ab02523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.776+0000 7fe9efe00640 1 -- 172.21.15.47:0/4050877965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024ff50 msgr2=0x561ab02523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.776+0000 7fe9efe00640 1 --2- 172.21.15.47:0/4050877965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024ff50 0x561ab02523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.776+0000 7fe9efe00640 1 -- 172.21.15.47:0/4050877965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024a3f0 msgr2=0x561ab024a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.776+0000 7fe9efe00640 1 --2- 172.21.15.47:0/4050877965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024a3f0 0x561ab024a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.776+0000 7fe9efe00640 1 -- 172.21.15.47:0/4050877965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ab01702c0 con 0x561ab024ade0 2024-03-21T11:27:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.776+0000 7fe9e7e00640 1 --2- 172.21.15.47:0/4050877965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024ff50 0x561ab02523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:27:22.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.776+0000 7fe9ed600640 1 --2- 172.21.15.47:0/48117423 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9d40a2f10 0x7fe9d40a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.776+0000 7fe9efe00640 1 --2- 172.21.15.47:0/4050877965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab024f8e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe9e002efd0 tx=0x7fe9e0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f0550574fa7b9eb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.776+0000 7fe9ed600640 1 -- 172.21.15.47:0/48117423 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe9e80cf980 con 0x7fe9f01030d0 2024-03-21T11:27:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.776+0000 7fe9e7e00640 1 --2- 172.21.15.47:0/48117423 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9d40a2f10 0x7fe9d40a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.777+0000 7fe9e5600640 1 -- 172.21.15.47:0/4050877965 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0042070 con 0x561ab024ade0 2024-03-21T11:27:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.777+0000 7fe9e5600640 1 -- 172.21.15.47:0/4050877965 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9e002cb90 con 0x561ab024ade0 2024-03-21T11:27:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.777+0000 7fe9e5600640 1 -- 172.21.15.47:0/4050877965 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0004920 con 0x561ab024ade0 2024-03-21T11:27:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.777+0000 7fe9f9bff740 1 -- 172.21.15.47:0/4050877965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 msgr2=0x561ab024f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.777+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/4050877965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab024f8e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe9e002efd0 tx=0x7fe9e0000bc0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.777+0000 7fe9e7e00640 1 --2- 172.21.15.47:0/48117423 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9d40a2f10 0x7fe9d40a53d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009710 tx=0x7fe9d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.778+0000 7fe9f9bff740 1 -- 172.21.15.47:0/4050877965 shutdown_connections 2024-03-21T11:27:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.778+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/4050877965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024ff50 0x561ab02523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.778+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/4050877965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab024f8e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.778+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/4050877965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024a3f0 0x561ab024a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.778+0000 7fe9f9bff740 1 -- 172.21.15.47:0/4050877965 >> 172.21.15.47:0/4050877965 conn(0x561ab0245960 msgr2=0x561ab0245d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.778+0000 7fe9f9bff740 1 -- 172.21.15.47:0/4050877965 shutdown_connections 2024-03-21T11:27:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.778+0000 7fe9f9bff740 1 -- 172.21.15.47:0/4050877965 wait complete. 2024-03-21T11:27:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.778+0000 7fe9f9bff740 1 -- start start 2024-03-21T11:27:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.778+0000 7fe9f9bff740 1 Processor -- start 2024-03-21T11:27:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9f9bff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024a3f0 0x561ab0254580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9f9bff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024ade0 0x561ab0254ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9f9bff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 0x561ab025ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9f9bff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab016ba20 con 0x561ab024ff50 2024-03-21T11:27:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9f9bff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab01ab8d0 con 0x561ab024a3f0 2024-03-21T11:27:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9f9bff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ab0252ea0 con 0x561ab024ade0 2024-03-21T11:27:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9f9bff740 10 client.0 _ll_register_callbacks cb 0x561ab01a81c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9e7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024ade0 0x561ab0254ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9efe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024a3f0 0x561ab0254580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9e7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024ade0 0x561ab0254ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53926/0 (socket says 172.21.15.47:53926) 2024-03-21T11:27:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9e7e00640 1 -- 172.21.15.47:0/1250921474 learned_addr learned my addr 172.21.15.47:0/1250921474 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9f4e00640 1 --2- 172.21.15.47:0/1250921474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 0x561ab025ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9e7e00640 1 -- 172.21.15.47:0/1250921474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024a3f0 msgr2=0x561ab0254580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9e7e00640 1 --2- 172.21.15.47:0/1250921474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024a3f0 0x561ab0254580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9e7e00640 1 -- 172.21.15.47:0/1250921474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 msgr2=0x561ab025ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9e7e00640 1 --2- 172.21.15.47:0/1250921474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 0x561ab025ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.779+0000 7fe9e7e00640 1 -- 172.21.15.47:0/1250921474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561aafec4210 con 0x561ab024ade0 2024-03-21T11:27:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.780+0000 7fe9e7e00640 1 --2- 172.21.15.47:0/1250921474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024ade0 0x561ab0254ac0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8020a00 tx=0x7fe9d8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8943c5dedbdaa7ce server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.780+0000 7fe9de000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.780+0000 7fe9efe00640 1 --2- 172.21.15.47:0/1250921474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024a3f0 0x561ab0254580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.780+0000 7fe9de000640 1 -- 172.21.15.47:0/1250921474 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8005020 con 0x561ab024ade0 2024-03-21T11:27:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.780+0000 7fe9f4e00640 1 --2- 172.21.15.47:0/1250921474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 0x561ab025ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.780+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1250921474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9e002cb90 con 0x561ab024ade0 2024-03-21T11:27:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.780+0000 7fe9de000640 1 -- 172.21.15.47:0/1250921474 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe9d80051c0 con 0x561ab024ade0 2024-03-21T11:27:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.781+0000 7fe9de000640 1 -- 172.21.15.47:0/1250921474 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9d8035050 con 0x561ab024ade0 2024-03-21T11:27:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.781+0000 7fe9f9bff740 10 client.26776 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.781+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1250921474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561aaff90620 con 0x561ab024ade0 2024-03-21T11:27:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.781+0000 7fe9de000640 1 -- 172.21.15.47:0/1250921474 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9e002cb90 con 0x561ab024ade0 2024-03-21T11:27:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.781+0000 7fe9de000640 1 -- 172.21.15.47:0/1250921474 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe9d801e650 con 0x561ab024ade0 2024-03-21T11:27:22.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.781+0000 7fe9f9bff740 10 client.26776 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.781+0000 7fe9f9bff740 10 client.26776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.782+0000 7fe9f9bff740 20 client.26776 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.782+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/1250921474 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561ab02603f0 0x561ab0262840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.782+0000 7fe9f9bff740 4 client.26776 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.782+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1250921474 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ab01bcd70 con 0x561ab02603f0 2024-03-21T11:27:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.782+0000 7fe9efe00640 1 --2- 172.21.15.47:0/1250921474 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561ab02603f0 0x561ab0262840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.782+0000 7fe9efe00640 1 --2- 172.21.15.47:0/1250921474 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561ab02603f0 0x561ab0262840 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4fe85df50177c13 server_cookie=9d7fdea0f5e52aa3 in_seq=0 out_seq=0 2024-03-21T11:27:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.782+0000 7fe9de000640 10 client.26776 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.783+0000 7fe9de000640 1 -- 172.21.15.47:0/1250921474 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561ab01bcd70 con 0x561ab02603f0 2024-03-21T11:27:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.783+0000 7fe9de000640 10 client.26776 handle_command_reply: tid=0 2024-03-21T11:27:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.783+0000 7fe9f9bff740 1 client.26776 shutdown 2024-03-21T11:27:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.784+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1250921474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024ade0 msgr2=0x561ab0254ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.784+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/1250921474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024ade0 0x561ab0254ac0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8020a00 tx=0x7fe9d8004730 comp rx=0 tx=0).stop 2024-03-21T11:27:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.784+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1250921474 shutdown_connections 2024-03-21T11:27:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.784+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/1250921474 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561ab02603f0 0x561ab0262840 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.784+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/1250921474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 0x561ab025ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.784+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/1250921474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024ade0 0x561ab0254ac0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.784+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/1250921474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024a3f0 0x561ab0254580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.784+0000 7fe9de000640 0 client.26776 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.784+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1250921474 >> 172.21.15.47:0/1250921474 conn(0x561ab0245960 msgr2=0x561ab0244b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.784+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1250921474 shutdown_connections 2024-03-21T11:27:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.784+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1250921474 wait complete. 2024-03-21T11:27:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.784+0000 7fe9f9bff740 20 client.26776 trim_cache size 0 max 16384 2024-03-21T11:27:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.803+0000 7fe9f9bff740 1 Processor -- start 2024-03-21T11:27:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.803+0000 7fe9f9bff740 1 -- start start 2024-03-21T11:27:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.803+0000 7fe9f9bff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024a3f0 0x561ab024f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.803+0000 7fe9f9bff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab025ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.803+0000 7fe9f9bff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 0x561ab0245d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.803+0000 7fe9f9bff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01ac0e0 con 0x561ab024ff50 2024-03-21T11:27:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.803+0000 7fe9f9bff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01ada60 con 0x561ab024ade0 2024-03-21T11:27:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.803+0000 7fe9f9bff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01ac740 con 0x561ab024a3f0 2024-03-21T11:27:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.804+0000 7fe9efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024a3f0 0x561ab024f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.804+0000 7fe9efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024a3f0 0x561ab024f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53928/0 (socket says 172.21.15.47:53928) 2024-03-21T11:27:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.804+0000 7fe9efe00640 1 -- 172.21.15.47:0/3424949294 learned_addr learned my addr 172.21.15.47:0/3424949294 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.804+0000 7fe9f4e00640 1 --2- 172.21.15.47:0/3424949294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 0x561ab0245d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.804+0000 7fe9e7e00640 1 --2- 172.21.15.47:0/3424949294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab025ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.804+0000 7fe9efe00640 1 -- 172.21.15.47:0/3424949294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 msgr2=0x561ab025ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.804+0000 7fe9efe00640 1 --2- 172.21.15.47:0/3424949294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab025ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.804+0000 7fe9efe00640 1 -- 172.21.15.47:0/3424949294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 msgr2=0x561ab0245d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.804+0000 7fe9efe00640 1 --2- 172.21.15.47:0/3424949294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 0x561ab0245d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.804+0000 7fe9efe00640 1 -- 172.21.15.47:0/3424949294 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ab024f410 con 0x561ab024a3f0 2024-03-21T11:27:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.804+0000 7fe9efe00640 1 --2- 172.21.15.47:0/3424949294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024a3f0 0x561ab024f790 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe9e002c250 tx=0x7fe9e002cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e03c8322c89c479e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.805+0000 7fe9e4c00640 1 -- 172.21.15.47:0/3424949294 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0042040 con 0x561ab024a3f0 2024-03-21T11:27:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.805+0000 7fe9e4c00640 1 -- 172.21.15.47:0/3424949294 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9e002cb90 con 0x561ab024a3f0 2024-03-21T11:27:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.805+0000 7fe9e4c00640 1 -- 172.21.15.47:0/3424949294 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0053700 con 0x561ab024a3f0 2024-03-21T11:27:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.805+0000 7fe9f9bff740 1 -- 172.21.15.47:0/3424949294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024a3f0 msgr2=0x561ab024f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.805+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/3424949294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024a3f0 0x561ab024f790 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe9e002c250 tx=0x7fe9e002cf90 comp rx=0 tx=0).stop 2024-03-21T11:27:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.806+0000 7fe9f9bff740 1 -- 172.21.15.47:0/3424949294 shutdown_connections 2024-03-21T11:27:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.806+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/3424949294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 0x561ab0245d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.806+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/3424949294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab025ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.806+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/3424949294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024a3f0 0x561ab024f790 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.806+0000 7fe9f9bff740 1 -- 172.21.15.47:0/3424949294 >> 172.21.15.47:0/3424949294 conn(0x561ab0245960 msgr2=0x561ab02406a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.806+0000 7fe9f9bff740 1 -- 172.21.15.47:0/3424949294 shutdown_connections 2024-03-21T11:27:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.806+0000 7fe9f9bff740 1 -- 172.21.15.47:0/3424949294 wait complete. 2024-03-21T11:27:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.806+0000 7fe9f9bff740 1 -- start start 2024-03-21T11:27:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.806+0000 7fe9f9bff740 1 Processor -- start 2024-03-21T11:27:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.807+0000 7fe9f9bff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024a3f0 0x561ab0259460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.807+0000 7fe9f9bff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab02599a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.807+0000 7fe9f9bff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 0x561ab025a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.807+0000 7fe9f9bff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01ad400 con 0x561ab024ff50 2024-03-21T11:27:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.807+0000 7fe9f9bff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01acda0 con 0x561ab024ade0 2024-03-21T11:27:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.807+0000 7fe9f9bff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01aa2a0 con 0x561ab024a3f0 2024-03-21T11:27:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.807+0000 7fe9f9bff740 10 client.0 _ll_register_callbacks cb 0x7fe9d8020890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.807+0000 7fe9e7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab02599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.807+0000 7fe9e7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab02599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55990/0 (socket says 172.21.15.47:55990) 2024-03-21T11:27:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.807+0000 7fe9e7e00640 1 -- 172.21.15.47:0/1036940479 learned_addr learned my addr 172.21.15.47:0/1036940479 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.807+0000 7fe9efe00640 1 --2- 172.21.15.47:0/1036940479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024a3f0 0x561ab0259460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.807+0000 7fe9f4e00640 1 --2- 172.21.15.47:0/1036940479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 0x561ab025a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.808+0000 7fe9e7e00640 1 -- 172.21.15.47:0/1036940479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024a3f0 msgr2=0x561ab0259460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.808+0000 7fe9e7e00640 1 --2- 172.21.15.47:0/1036940479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024a3f0 0x561ab0259460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.808+0000 7fe9e7e00640 1 -- 172.21.15.47:0/1036940479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 msgr2=0x561ab025a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.808+0000 7fe9e7e00640 1 --2- 172.21.15.47:0/1036940479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 0x561ab025a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.808+0000 7fe9e7e00640 1 -- 172.21.15.47:0/1036940479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ab025afa0 con 0x561ab024ade0 2024-03-21T11:27:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.808+0000 7fe9f4e00640 1 --2- 172.21.15.47:0/1036940479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 0x561ab025a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.808+0000 7fe9e7e00640 1 --2- 172.21.15.47:0/1036940479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab02599a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe9d801eac0 tx=0x7fe9d801ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce5727803e6b4f24 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.808+0000 7fe9de000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.808+0000 7fe9de000640 1 -- 172.21.15.47:0/1036940479 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d803d020 con 0x561ab024ade0 2024-03-21T11:27:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.808+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1036940479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9e002cb90 con 0x561ab024ade0 2024-03-21T11:27:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.808+0000 7fe9de000640 1 -- 172.21.15.47:0/1036940479 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe9d8005240 con 0x561ab024ade0 2024-03-21T11:27:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.808+0000 7fe9de000640 1 -- 172.21.15.47:0/1036940479 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9d8045050 con 0x561ab024ade0 2024-03-21T11:27:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.808+0000 7fe9f9bff740 10 client.26759 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.808+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1036940479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561ab02bbfd0 con 0x561ab024ade0 2024-03-21T11:27:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.809+0000 7fe9de000640 1 -- 172.21.15.47:0/1036940479 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9e002cb90 con 0x561ab024ade0 2024-03-21T11:27:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.809+0000 7fe9de000640 1 -- 172.21.15.47:0/1036940479 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe9d802a420 con 0x561ab024ade0 2024-03-21T11:27:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.809+0000 7fe9f9bff740 10 client.26759 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.809+0000 7fe9f9bff740 10 client.26759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.809+0000 7fe9f9bff740 20 client.26759 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.809+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/1036940479 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561ab02bc820 0x561ab02beca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.809+0000 7fe9f9bff740 4 client.26759 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:27:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.809+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1036940479 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561ab01b70c0 con 0x561ab02bc820 2024-03-21T11:27:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.809+0000 7fe9efe00640 1 --2- 172.21.15.47:0/1036940479 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561ab02bc820 0x561ab02beca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.810+0000 7fe9efe00640 1 --2- 172.21.15.47:0/1036940479 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561ab02bc820 0x561ab02beca0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5affc609b047c0d5 server_cookie=467c420e75b9d297 in_seq=0 out_seq=0 2024-03-21T11:27:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:22.810+0000 7fe9de000640 10 client.26759 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:22.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:22 smithi100 ceph-mon[36245]: pgmap v521: 129 pgs: 129 active+clean; 35 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 300 op/s 2024-03-21T11:27:22.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:22 smithi102 ceph-mon[36748]: pgmap v521: 129 pgs: 129 active+clean; 35 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 300 op/s 2024-03-21T11:27:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.023+0000 7fe9de000640 1 -- 172.21.15.47:0/1036940479 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561ab01b70c0 con 0x561ab02bc820 2024-03-21T11:27:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.023+0000 7fe9de000640 10 client.26759 handle_command_reply: tid=0 2024-03-21T11:27:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.023+0000 7fe9f9bff740 1 client.26759 shutdown 2024-03-21T11:27:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.023+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1036940479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 msgr2=0x561ab02599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.023+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/1036940479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab02599a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe9d801eac0 tx=0x7fe9d801ef40 comp rx=0 tx=0).stop 2024-03-21T11:27:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.024+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1036940479 shutdown_connections 2024-03-21T11:27:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.024+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/1036940479 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561ab02bc820 0x561ab02beca0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.024+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/1036940479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ab024ff50 0x561ab025a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.024+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/1036940479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ab024ade0 0x561ab02599a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.024+0000 7fe9f9bff740 1 --2- 172.21.15.47:0/1036940479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ab024a3f0 0x561ab0259460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.024+0000 7fe9de000640 0 client.26759 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.024+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1036940479 >> 172.21.15.47:0/1036940479 conn(0x561ab0245960 msgr2=0x561ab024f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.024+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1036940479 shutdown_connections 2024-03-21T11:27:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.024+0000 7fe9f9bff740 1 -- 172.21.15.47:0/1036940479 wait complete. 2024-03-21T11:27:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.024+0000 7fe9f9bff740 20 client.26759 trim_cache size 0 max 16384 2024-03-21T11:27:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:27:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:27:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:27:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:27:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.025+0000 7fe9f5800640 1 -- 172.21.15.47:0/48117423 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9d40a2f10 msgr2=0x7fe9d40a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.025+0000 7fe9f5800640 1 --2- 172.21.15.47:0/48117423 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9d40a2f10 0x7fe9d40a53d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009710 tx=0x7fe9d8005f90 comp rx=0 tx=0).stop 2024-03-21T11:27:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.025+0000 7fe9f5800640 1 -- 172.21.15.47:0/48117423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f01030d0 msgr2=0x7fe9f01a0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.025+0000 7fe9f5800640 1 --2- 172.21.15.47:0/48117423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f01030d0 0x7fe9f01a0ff0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800eaa0 tx=0x7fe9e800ef60 comp rx=0 tx=0).stop 2024-03-21T11:27:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.025+0000 7fe9f5800640 1 -- 172.21.15.47:0/48117423 shutdown_connections 2024-03-21T11:27:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.025+0000 7fe9f5800640 1 --2- 172.21.15.47:0/48117423 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9d40a2f10 0x7fe9d40a53d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.025+0000 7fe9f5800640 1 --2- 172.21.15.47:0/48117423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f01030d0 0x7fe9f01a0ff0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.025+0000 7fe9f5800640 1 --2- 172.21.15.47:0/48117423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9f0102710 0x7fe9f0199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.025+0000 7fe9f5800640 1 --2- 172.21.15.47:0/48117423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9f0101520 0x7fe9f0199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.025+0000 7fe9f5800640 1 -- 172.21.15.47:0/48117423 >> 172.21.15.47:0/48117423 conn(0x7fe9f00fcab0 msgr2=0x7fe9f00738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.026+0000 7fe9f5800640 1 -- 172.21.15.47:0/48117423 shutdown_connections 2024-03-21T11:27:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:23.026+0000 7fe9f5800640 1 -- 172.21.15.47:0/48117423 wait complete. 2024-03-21T11:27:23.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:22 smithi047 ceph-mon[31351]: pgmap v521: 129 pgs: 129 active+clean; 35 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 300 op/s 2024-03-21T11:27:23.503 INFO:tasks.workunit.client.0.smithi047.stdout: 1 205543 7.77 MB/sec execute 7 sec latency 130.183 ms 2024-03-21T11:27:24.503 INFO:tasks.workunit.client.0.smithi047.stdout: 1 206462 7.42 MB/sec execute 8 sec latency 30.174 ms 2024-03-21T11:27:24.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:24 smithi100 ceph-mon[36245]: pgmap v522: 129 pgs: 129 active+clean; 37 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 313 op/s 2024-03-21T11:27:24.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:24 smithi102 ceph-mon[36748]: pgmap v522: 129 pgs: 129 active+clean; 37 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 313 op/s 2024-03-21T11:27:25.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:24 smithi047 ceph-mon[31351]: pgmap v522: 129 pgs: 129 active+clean; 37 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 313 op/s 2024-03-21T11:27:25.503 INFO:tasks.workunit.client.0.smithi047.stdout: 1 208137 7.34 MB/sec execute 9 sec latency 40.752 ms 2024-03-21T11:27:26.503 INFO:tasks.workunit.client.0.smithi047.stdout: 1 209638 7.81 MB/sec execute 10 sec latency 23.662 ms 2024-03-21T11:27:26.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:26 smithi100 ceph-mon[36245]: pgmap v523: 129 pgs: 129 active+clean; 37 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-03-21T11:27:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:26 smithi102 ceph-mon[36748]: pgmap v523: 129 pgs: 129 active+clean; 37 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-03-21T11:27:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:26 smithi047 ceph-mon[31351]: pgmap v523: 129 pgs: 129 active+clean; 37 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-03-21T11:27:27.503 INFO:tasks.workunit.client.0.smithi047.stdout: 1 210812 7.57 MB/sec execute 11 sec latency 13.810 ms 2024-03-21T11:27:28.503 INFO:tasks.workunit.client.0.smithi047.stdout: 1 212894 7.98 MB/sec execute 12 sec latency 11.884 ms 2024-03-21T11:27:28.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:28 smithi102 ceph-mon[36748]: pgmap v524: 129 pgs: 129 active+clean; 34 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 282 op/s 2024-03-21T11:27:29.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:28 smithi047 ceph-mon[31351]: pgmap v524: 129 pgs: 129 active+clean; 34 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 282 op/s 2024-03-21T11:27:29.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:28 smithi100 ceph-mon[36245]: pgmap v524: 129 pgs: 129 active+clean; 34 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 282 op/s 2024-03-21T11:27:29.503 INFO:tasks.workunit.client.0.smithi047.stdout: 1 213742 7.54 MB/sec execute 13 sec latency 11.713 ms 2024-03-21T11:27:30.504 INFO:tasks.workunit.client.0.smithi047.stdout: 1 215465 7.53 MB/sec execute 14 sec latency 47.855 ms 2024-03-21T11:27:30.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:30 smithi102 ceph-mon[36748]: pgmap v525: 129 pgs: 129 active+clean; 35 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 292 op/s 2024-03-21T11:27:31.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:30 smithi047 ceph-mon[31351]: pgmap v525: 129 pgs: 129 active+clean; 35 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 292 op/s 2024-03-21T11:27:31.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:30 smithi100 ceph-mon[36245]: pgmap v525: 129 pgs: 129 active+clean; 35 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 292 op/s 2024-03-21T11:27:31.504 INFO:tasks.workunit.client.0.smithi047.stdout: 1 216885 7.76 MB/sec execute 15 sec latency 14.298 ms 2024-03-21T11:27:32.504 INFO:tasks.workunit.client.0.smithi047.stdout: 1 218417 7.62 MB/sec execute 16 sec latency 17.391 ms 2024-03-21T11:27:32.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:32 smithi102 ceph-mon[36748]: pgmap v526: 129 pgs: 129 active+clean; 35 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 280 op/s 2024-03-21T11:27:33.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:32 smithi047 ceph-mon[31351]: pgmap v526: 129 pgs: 129 active+clean; 35 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 280 op/s 2024-03-21T11:27:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:32 smithi100 ceph-mon[36245]: pgmap v526: 129 pgs: 129 active+clean; 35 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 280 op/s 2024-03-21T11:27:33.504 INFO:tasks.workunit.client.0.smithi047.stdout: 1 220261 7.95 MB/sec execute 17 sec latency 11.912 ms 2024-03-21T11:27:34.504 INFO:tasks.workunit.client.0.smithi047.stdout: 1 221418 7.81 MB/sec execute 18 sec latency 97.833 ms 2024-03-21T11:27:34.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:34 smithi102 ceph-mon[36748]: pgmap v527: 129 pgs: 129 active+clean; 37 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 329 op/s 2024-03-21T11:27:35.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:34 smithi100 ceph-mon[36245]: pgmap v527: 129 pgs: 129 active+clean; 37 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 329 op/s 2024-03-21T11:27:35.249 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:34 smithi047 ceph-mon[31351]: pgmap v527: 129 pgs: 129 active+clean; 37 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 329 op/s 2024-03-21T11:27:35.504 INFO:tasks.workunit.client.0.smithi047.stdout: 1 223618 8.07 MB/sec execute 19 sec latency 29.937 ms 2024-03-21T11:27:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.205+0000 7f32ff600640 1 -- 172.21.15.47:0/2803666861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32f80fff30 msgr2=0x7f32f8102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.205+0000 7f32ff600640 1 --2- 172.21.15.47:0/2803666861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32f80fff30 0x7f32f8102360 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009900 tx=0x7f32ec02bc40 comp rx=0 tx=0).stop 2024-03-21T11:27:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.205+0000 7f32ff600640 1 -- 172.21.15.47:0/2803666861 shutdown_connections 2024-03-21T11:27:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.205+0000 7f32ff600640 1 --2- 172.21.15.47:0/2803666861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32f8105250 0x7f32f81056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.205+0000 7f32ff600640 1 --2- 172.21.15.47:0/2803666861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32f81028a0 0x7f32f8104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.205+0000 7f32ff600640 1 --2- 172.21.15.47:0/2803666861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32f80fff30 0x7f32f8102360 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.205+0000 7f32ff600640 1 -- 172.21.15.47:0/2803666861 >> 172.21.15.47:0/2803666861 conn(0x7f32f80fbb30 msgr2=0x7f32f80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:36.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.205+0000 7f32ff600640 1 -- 172.21.15.47:0/2803666861 shutdown_connections 2024-03-21T11:27:36.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.205+0000 7f32ff600640 1 -- 172.21.15.47:0/2803666861 wait complete. 2024-03-21T11:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.206+0000 7f32ff600640 1 Processor -- start 2024-03-21T11:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.206+0000 7f32ff600640 1 -- start start 2024-03-21T11:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.206+0000 7f32ff600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32f80fff30 0x7f32f819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.207+0000 7f32ff600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32f81028a0 0x7f32f819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.207+0000 7f32ff600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32f8105250 0x7f32f81a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.207+0000 7f32ff600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f8113960 con 0x7f32f8105250 2024-03-21T11:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.207+0000 7f32ff600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f81137e0 con 0x7f32f81028a0 2024-03-21T11:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.207+0000 7f32ff600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f8113ae0 con 0x7f32f80fff30 2024-03-21T11:27:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.207+0000 7f32fcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32f80fff30 0x7f32f819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.207+0000 7f32fcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32f80fff30 0x7f32f819ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36310/0 (socket says 172.21.15.47:36310) 2024-03-21T11:27:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.207+0000 7f32fcc00640 1 -- 172.21.15.47:0/2743172462 learned_addr learned my addr 172.21.15.47:0/2743172462 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.207+0000 7f32f7e00640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32f81028a0 0x7f32f819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.207+0000 7f32fd600640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32f8105250 0x7f32f81a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.208+0000 7f32fcc00640 1 -- 172.21.15.47:0/2743172462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32f81028a0 msgr2=0x7f32f819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.208+0000 7f32fcc00640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32f81028a0 0x7f32f819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.208+0000 7f32fcc00640 1 -- 172.21.15.47:0/2743172462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32f8105250 msgr2=0x7f32f81a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.208+0000 7f32fcc00640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32f8105250 0x7f32f81a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.208+0000 7f32fcc00640 1 -- 172.21.15.47:0/2743172462 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f81a5af0 con 0x7f32f80fff30 2024-03-21T11:27:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.208+0000 7f32fd600640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32f8105250 0x7f32f81a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.209+0000 7f32fcc00640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32f80fff30 0x7f32f819ded0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f32ec02c250 tx=0x7f32ec004e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.210+0000 7f32f5600640 1 -- 172.21.15.47:0/2743172462 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec041070 con 0x7f32f80fff30 2024-03-21T11:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.210+0000 7f32f5600640 1 -- 172.21.15.47:0/2743172462 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32ec0054e0 con 0x7f32f80fff30 2024-03-21T11:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.210+0000 7f32f5600640 1 -- 172.21.15.47:0/2743172462 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec034440 con 0x7f32f80fff30 2024-03-21T11:27:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.211+0000 7f32ff600640 1 -- 172.21.15.47:0/2743172462 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f81a5d80 con 0x7f32f80fff30 2024-03-21T11:27:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.212+0000 7f32ff600640 1 -- 172.21.15.47:0/2743172462 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f81a61c0 con 0x7f32f80fff30 2024-03-21T11:27:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.217+0000 7f33023a3740 1 Processor -- start 2024-03-21T11:27:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.218+0000 7f33023a3740 1 -- start start 2024-03-21T11:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.218+0000 7f33023a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddba9d1e0 0x562ddba9d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.218+0000 7f33023a3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbaa26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.218+0000 7f33023a3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddbaa2d40 0x562ddbaa51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.218+0000 7f33023a3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddba603b0 con 0x562ddbaa2d40 2024-03-21T11:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.218+0000 7f33023a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddba02830 con 0x562ddba9d1e0 2024-03-21T11:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.218+0000 7f33023a3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddb9ab5d0 con 0x562ddba9dbd0 2024-03-21T11:27:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.219+0000 7f32fd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddba9d1e0 0x562ddba9d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.219+0000 7f32fd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddba9d1e0 0x562ddba9d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49228/0 (socket says 172.21.15.47:49228) 2024-03-21T11:27:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.219+0000 7f32fd600640 1 -- 172.21.15.47:0/1219979093 learned_addr learned my addr 172.21.15.47:0/1219979093 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.220+0000 7f32f7e00640 1 --2- 172.21.15.47:0/1219979093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddbaa2d40 0x562ddbaa51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.221+0000 7f32f5600640 1 -- 172.21.15.47:0/2743172462 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f32ec044070 con 0x7f32f80fff30 2024-03-21T11:27:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.221+0000 7f32fcc00640 1 --2- 172.21.15.47:0/1219979093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbaa26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.221+0000 7f32fd600640 1 -- 172.21.15.47:0/1219979093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 msgr2=0x562ddbaa26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.221+0000 7f32fd600640 1 --2- 172.21.15.47:0/1219979093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbaa26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.221+0000 7f32fd600640 1 -- 172.21.15.47:0/1219979093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddbaa2d40 msgr2=0x562ddbaa51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.221+0000 7f32fd600640 1 --2- 172.21.15.47:0/1219979093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddbaa2d40 0x562ddbaa51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.221+0000 7f32fd600640 1 -- 172.21.15.47:0/1219979093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ddb9c32c0 con 0x562ddba9d1e0 2024-03-21T11:27:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.221+0000 7f32fcc00640 1 --2- 172.21.15.47:0/1219979093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbaa26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:27:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.221+0000 7f32f7e00640 1 --2- 172.21.15.47:0/1219979093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddbaa2d40 0x562ddbaa51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.222+0000 7f32fd600640 1 --2- 172.21.15.47:0/1219979093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddba9d1e0 0x562ddba9d600 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009fd0 tx=0x7f32f00108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=988ac6801bb34903 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.223+0000 7f32f5600640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32e00a3200 0x7f32e00a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.223+0000 7f32f5600640 1 -- 172.21.15.47:0/2743172462 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f32ec0f0da0 con 0x7f32f80fff30 2024-03-21T11:27:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.223+0000 7f32fcc00640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32e00a3200 0x7f32e00a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.224+0000 7f32e9c00640 1 -- 172.21.15.47:0/1219979093 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0011760 con 0x562ddba9d1e0 2024-03-21T11:27:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.224+0000 7f32e9c00640 1 -- 172.21.15.47:0/1219979093 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32f0011de0 con 0x562ddba9d1e0 2024-03-21T11:27:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.224+0000 7f32e9c00640 1 -- 172.21.15.47:0/1219979093 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f000e3c0 con 0x562ddba9d1e0 2024-03-21T11:27:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.225+0000 7f33023a3740 1 -- 172.21.15.47:0/1219979093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddba9d1e0 msgr2=0x562ddba9d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.225+0000 7f33023a3740 1 --2- 172.21.15.47:0/1219979093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddba9d1e0 0x562ddba9d600 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009fd0 tx=0x7f32f00108e0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.225+0000 7f32fcc00640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32e00a3200 0x7f32e00a56c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009850 tx=0x7f32ec0dd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.226+0000 7f33023a3740 1 -- 172.21.15.47:0/1219979093 shutdown_connections 2024-03-21T11:27:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.226+0000 7f33023a3740 1 --2- 172.21.15.47:0/1219979093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddbaa2d40 0x562ddbaa51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.226+0000 7f33023a3740 1 --2- 172.21.15.47:0/1219979093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbaa26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.226+0000 7f33023a3740 1 --2- 172.21.15.47:0/1219979093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddba9d1e0 0x562ddba9d600 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.226+0000 7f33023a3740 1 -- 172.21.15.47:0/1219979093 >> 172.21.15.47:0/1219979093 conn(0x562ddba98750 msgr2=0x562ddba98b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.227+0000 7f33023a3740 1 -- 172.21.15.47:0/1219979093 shutdown_connections 2024-03-21T11:27:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.227+0000 7f33023a3740 1 -- 172.21.15.47:0/1219979093 wait complete. 2024-03-21T11:27:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.229+0000 7f33023a3740 1 -- start start 2024-03-21T11:27:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.229+0000 7f33023a3740 1 Processor -- start 2024-03-21T11:27:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.231+0000 7f33023a3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9d1e0 0x562ddbab8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.231+0000 7f33023a3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddba9dbd0 0x562ddbab8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.231+0000 7f33023a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddbaa2d40 0x562ddbabf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.231+0000 7f33023a3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddb9bea20 con 0x562ddba9dbd0 2024-03-21T11:27:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.231+0000 7f33023a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddb9fe8d0 con 0x562ddbaa2d40 2024-03-21T11:27:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.231+0000 7f33023a3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddbab6d00 con 0x562ddba9d1e0 2024-03-21T11:27:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.231+0000 7f33023a3740 10 client.0 _ll_register_callbacks cb 0x562ddb9fb1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.232+0000 7f32fd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddbaa2d40 0x562ddbabf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.232+0000 7f32fd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddbaa2d40 0x562ddbabf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49242/0 (socket says 172.21.15.47:49242) 2024-03-21T11:27:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.232+0000 7f32fd600640 1 -- 172.21.15.47:0/2274174234 learned_addr learned my addr 172.21.15.47:0/2274174234 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.232+0000 7f32fd600640 1 --2- 172.21.15.47:0/2274174234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9d1e0 0x562ddbab8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.233+0000 7f32f7e00640 1 --2- 172.21.15.47:0/2274174234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddba9dbd0 0x562ddbab8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.233+0000 7f32fd600640 1 -- 172.21.15.47:0/2274174234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9d1e0 msgr2=0x562ddbab8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.233+0000 7f32fd600640 1 --2- 172.21.15.47:0/2274174234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9d1e0 0x562ddbab8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.233+0000 7f32fd600640 1 -- 172.21.15.47:0/2274174234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddba9dbd0 msgr2=0x562ddbab8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.233+0000 7f32fd600640 1 --2- 172.21.15.47:0/2274174234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddba9dbd0 0x562ddbab8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.233+0000 7f32fd600640 1 -- 172.21.15.47:0/2274174234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ddb717210 con 0x562ddbaa2d40 2024-03-21T11:27:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.234+0000 7f32fd600640 1 --2- 172.21.15.47:0/2274174234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddbaa2d40 0x562ddbabf9b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f32f001ba90 tx=0x7f32f0013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa35a60e277de5d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.234+0000 7f32de000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.235+0000 7f32de000640 1 -- 172.21.15.47:0/2274174234 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0011a40 con 0x562ddbaa2d40 2024-03-21T11:27:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.235+0000 7f32de000640 1 -- 172.21.15.47:0/2274174234 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f32f0011be0 con 0x562ddbaa2d40 2024-03-21T11:27:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.236+0000 7f33023a3740 1 -- 172.21.15.47:0/2274174234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32f0011de0 con 0x562ddbaa2d40 2024-03-21T11:27:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.236+0000 7f32de000640 1 -- 172.21.15.47:0/2274174234 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32f002b460 con 0x562ddbaa2d40 2024-03-21T11:27:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.236+0000 7f33023a3740 10 client.26771 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.236+0000 7f33023a3740 1 -- 172.21.15.47:0/2274174234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562ddb7e3620 con 0x562ddbaa2d40 2024-03-21T11:27:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.238+0000 7f32de000640 1 -- 172.21.15.47:0/2274174234 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32f0011de0 con 0x562ddbaa2d40 2024-03-21T11:27:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.238+0000 7f32de000640 1 -- 172.21.15.47:0/2274174234 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f32f0029040 con 0x562ddbaa2d40 2024-03-21T11:27:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.238+0000 7f33023a3740 10 client.26771 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.238+0000 7f33023a3740 10 client.26771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.238+0000 7f33023a3740 20 client.26771 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.238+0000 7f33023a3740 1 --2- 172.21.15.47:0/2274174234 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ddbad4f30 0x562ddbad7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.238+0000 7f33023a3740 4 client.26771 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.238+0000 7f33023a3740 1 -- 172.21.15.47:0/2274174234 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ddba0fd70 con 0x562ddbad4f30 2024-03-21T11:27:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.239+0000 7f32f7e00640 1 --2- 172.21.15.47:0/2274174234 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ddbad4f30 0x562ddbad7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.241+0000 7f32f7e00640 1 --2- 172.21.15.47:0/2274174234 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ddbad4f30 0x562ddbad7320 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=587012a169ef687f server_cookie=5ac4ec5b81e088a7 in_seq=0 out_seq=0 2024-03-21T11:27:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.242+0000 7f32de000640 10 client.26771 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.243+0000 7f32de000640 1 -- 172.21.15.47:0/2274174234 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562ddba0fd70 con 0x562ddbad4f30 2024-03-21T11:27:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.243+0000 7f32de000640 10 client.26771 handle_command_reply: tid=0 2024-03-21T11:27:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.243+0000 7f33023a3740 1 client.26771 shutdown 2024-03-21T11:27:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.245+0000 7f33023a3740 1 -- 172.21.15.47:0/2274174234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddbaa2d40 msgr2=0x562ddbabf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.245+0000 7f33023a3740 1 --2- 172.21.15.47:0/2274174234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddbaa2d40 0x562ddbabf9b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f32f001ba90 tx=0x7f32f0013040 comp rx=0 tx=0).stop 2024-03-21T11:27:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.245+0000 7f33023a3740 1 -- 172.21.15.47:0/2274174234 shutdown_connections 2024-03-21T11:27:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.245+0000 7f33023a3740 1 --2- 172.21.15.47:0/2274174234 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ddbad4f30 0x562ddbad7320 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.245+0000 7f33023a3740 1 --2- 172.21.15.47:0/2274174234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddbaa2d40 0x562ddbabf9b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.245+0000 7f33023a3740 1 --2- 172.21.15.47:0/2274174234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddba9dbd0 0x562ddbab8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.245+0000 7f33023a3740 1 --2- 172.21.15.47:0/2274174234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9d1e0 0x562ddbab8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.245+0000 7f33023a3740 1 -- 172.21.15.47:0/2274174234 >> 172.21.15.47:0/2274174234 conn(0x562ddba98750 msgr2=0x562ddba97950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.245+0000 7f32de000640 0 client.26771 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.246+0000 7f33023a3740 1 -- 172.21.15.47:0/2274174234 shutdown_connections 2024-03-21T11:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.246+0000 7f33023a3740 1 -- 172.21.15.47:0/2274174234 wait complete. 2024-03-21T11:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.246+0000 7f33023a3740 20 client.26771 trim_cache size 0 max 16384 2024-03-21T11:27:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.277+0000 7f33023a3740 1 Processor -- start 2024-03-21T11:27:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.277+0000 7f33023a3740 1 -- start start 2024-03-21T11:27:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.278+0000 7f33023a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddba9d1e0 0x562ddbac0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.278+0000 7f33023a3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbabf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.278+0000 7f33023a3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddbaa2d40 0x562ddbac0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.278+0000 7f33023a3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f8108090 con 0x562ddbaa2d40 2024-03-21T11:27:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.278+0000 7f33023a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f8109a10 con 0x562ddba9d1e0 2024-03-21T11:27:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.278+0000 7f33023a3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f81086f0 con 0x562ddba9dbd0 2024-03-21T11:27:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.279+0000 7f32f7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbabf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.279+0000 7f32f7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbabf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36332/0 (socket says 172.21.15.47:36332) 2024-03-21T11:27:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.279+0000 7f32f7e00640 1 -- 172.21.15.47:0/1020340469 learned_addr learned my addr 172.21.15.47:0/1020340469 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.279+0000 7f32fd600640 1 --2- 172.21.15.47:0/1020340469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddbaa2d40 0x562ddbac0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.280+0000 7f32fd600640 1 --2- 172.21.15.47:0/1020340469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddba9d1e0 0x562ddbac0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.280+0000 7f32f7e00640 1 -- 172.21.15.47:0/1020340469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddba9d1e0 msgr2=0x562ddbac0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.280+0000 7f32f7e00640 1 --2- 172.21.15.47:0/1020340469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddba9d1e0 0x562ddbac0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.280+0000 7f32f7e00640 1 -- 172.21.15.47:0/1020340469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddbaa2d40 msgr2=0x562ddbac0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.281+0000 7f32f7e00640 1 --2- 172.21.15.47:0/1020340469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddbaa2d40 0x562ddbac0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.281+0000 7f32f7e00640 1 -- 172.21.15.47:0/1020340469 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ddbad7690 con 0x562ddba9dbd0 2024-03-21T11:27:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.281+0000 7f32fd600640 1 --2- 172.21.15.47:0/1020340469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddbaa2d40 0x562ddbac0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.282+0000 7f32f7e00640 1 --2- 172.21.15.47:0/1020340469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbabf970 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009b30 tx=0x7f32e401eac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5de7ad0fb2889d8c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.282+0000 7f32e9200640 1 -- 172.21.15.47:0/1020340469 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e401e050 con 0x562ddba9dbd0 2024-03-21T11:27:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.282+0000 7f32e9200640 1 -- 172.21.15.47:0/1020340469 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32e401e1f0 con 0x562ddba9dbd0 2024-03-21T11:27:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.282+0000 7f32e9200640 1 -- 172.21.15.47:0/1020340469 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4003e50 con 0x562ddba9dbd0 2024-03-21T11:27:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.284+0000 7f33023a3740 1 -- 172.21.15.47:0/1020340469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 msgr2=0x562ddbabf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.284+0000 7f33023a3740 1 --2- 172.21.15.47:0/1020340469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbabf970 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009b30 tx=0x7f32e401eac0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.284+0000 7f33023a3740 1 -- 172.21.15.47:0/1020340469 shutdown_connections 2024-03-21T11:27:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.284+0000 7f33023a3740 1 --2- 172.21.15.47:0/1020340469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddbaa2d40 0x562ddbac0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.284+0000 7f33023a3740 1 --2- 172.21.15.47:0/1020340469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbabf970 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.284+0000 7f33023a3740 1 --2- 172.21.15.47:0/1020340469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddba9d1e0 0x562ddbac0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.284+0000 7f33023a3740 1 -- 172.21.15.47:0/1020340469 >> 172.21.15.47:0/1020340469 conn(0x562ddba98750 msgr2=0x562ddba9ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.285+0000 7f33023a3740 1 -- 172.21.15.47:0/1020340469 shutdown_connections 2024-03-21T11:27:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.285+0000 7f33023a3740 1 -- 172.21.15.47:0/1020340469 wait complete. 2024-03-21T11:27:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.285+0000 7f33023a3740 1 -- start start 2024-03-21T11:27:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.286+0000 7f33023a3740 1 Processor -- start 2024-03-21T11:27:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.287+0000 7f33023a3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddba9d1e0 0x562ddbabce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.288+0000 7f33023a3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbabd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.288+0000 7f33023a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddbaa2d40 0x562ddbabe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.288+0000 7f32fd600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddba9d1e0 0x562ddbabce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.288+0000 7f32fd600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddba9d1e0 0x562ddbabce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44976/0 (socket says 172.21.15.47:44976) 2024-03-21T11:27:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.288+0000 7f32fd600640 1 -- 172.21.15.47:0/1203794182 learned_addr learned my addr 172.21.15.47:0/1203794182 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.288+0000 7f33023a3740 1 -- 172.21.15.47:0/1203794182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f81093b0 con 0x562ddba9d1e0 2024-03-21T11:27:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.288+0000 7f33023a3740 1 -- 172.21.15.47:0/1203794182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f8108d50 con 0x562ddbaa2d40 2024-03-21T11:27:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.288+0000 7f33023a3740 1 -- 172.21.15.47:0/1203794182 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f8073630 con 0x562ddba9dbd0 2024-03-21T11:27:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.288+0000 7f33023a3740 10 client.0 _ll_register_callbacks cb 0x7f32f0009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.288+0000 7f32f7e00640 1 --2- 172.21.15.47:0/1203794182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbabd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.289+0000 7f32f7e00640 1 -- 172.21.15.47:0/1203794182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddbaa2d40 msgr2=0x562ddbabe300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:27:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.289+0000 7f32f7e00640 1 --2- 172.21.15.47:0/1203794182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddbaa2d40 0x562ddbabe300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.289+0000 7f32f7e00640 1 -- 172.21.15.47:0/1203794182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddba9d1e0 msgr2=0x562ddbabce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.289+0000 7f32f7e00640 1 --2- 172.21.15.47:0/1203794182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddba9d1e0 0x562ddbabce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.289+0000 7f32f7e00640 1 -- 172.21.15.47:0/1203794182 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ddbabe8f0 con 0x562ddba9dbd0 2024-03-21T11:27:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.289+0000 7f32f7e00640 1 --2- 172.21.15.47:0/1203794182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbabd340 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000c00 tx=0x7f32e4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dcd09f1a95d2d2e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.289+0000 7f32de000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.290+0000 7f32de000640 1 -- 172.21.15.47:0/1203794182 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4009370 con 0x562ddba9dbd0 2024-03-21T11:27:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.290+0000 7f32de000640 1 -- 172.21.15.47:0/1203794182 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f32e40035f0 con 0x562ddba9dbd0 2024-03-21T11:27:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.290+0000 7f33023a3740 1 -- 172.21.15.47:0/1203794182 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32e401e1f0 con 0x562ddba9dbd0 2024-03-21T11:27:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.290+0000 7f32de000640 1 -- 172.21.15.47:0/1203794182 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32e4038050 con 0x562ddba9dbd0 2024-03-21T11:27:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.290+0000 7f33023a3740 10 client.26797 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.290+0000 7f33023a3740 1 -- 172.21.15.47:0/1203794182 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562ddbabfc30 con 0x562ddba9dbd0 2024-03-21T11:27:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.291+0000 7f32de000640 1 -- 172.21.15.47:0/1203794182 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32e401e1f0 con 0x562ddba9dbd0 2024-03-21T11:27:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.292+0000 7f32de000640 1 -- 172.21.15.47:0/1203794182 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f32e40146a0 con 0x562ddba9dbd0 2024-03-21T11:27:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.292+0000 7f33023a3740 10 client.26797 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.292+0000 7f33023a3740 10 client.26797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.292+0000 7f33023a3740 20 client.26797 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.292+0000 7f33023a3740 1 --2- 172.21.15.47:0/1203794182 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ddbb30e20 0x562ddbb33240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.292+0000 7f33023a3740 4 client.26797 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:27:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.292+0000 7f33023a3740 1 -- 172.21.15.47:0/1203794182 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ddba0a0c0 con 0x562ddbb30e20 2024-03-21T11:27:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.292+0000 7f32fd600640 1 --2- 172.21.15.47:0/1203794182 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ddbb30e20 0x562ddbb33240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.294+0000 7f32fd600640 1 --2- 172.21.15.47:0/1203794182 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ddbb30e20 0x562ddbb33240 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e347e05e7a9af216 server_cookie=b279ea3ffa21480 in_seq=0 out_seq=0 2024-03-21T11:27:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.294+0000 7f32de000640 10 client.26797 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.489+0000 7f32de000640 1 -- 172.21.15.47:0/1203794182 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ddba0a0c0 con 0x562ddbb30e20 2024-03-21T11:27:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.489+0000 7f32de000640 10 client.26797 handle_command_reply: tid=0 2024-03-21T11:27:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.489+0000 7f33023a3740 1 client.26797 shutdown 2024-03-21T11:27:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.490+0000 7f33023a3740 1 -- 172.21.15.47:0/1203794182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 msgr2=0x562ddbabd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.490+0000 7f33023a3740 1 --2- 172.21.15.47:0/1203794182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbabd340 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000c00 tx=0x7f32e4005880 comp rx=0 tx=0).stop 2024-03-21T11:27:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.490+0000 7f33023a3740 1 -- 172.21.15.47:0/1203794182 shutdown_connections 2024-03-21T11:27:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.490+0000 7f33023a3740 1 --2- 172.21.15.47:0/1203794182 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ddbb30e20 0x562ddbb33240 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.490+0000 7f33023a3740 1 --2- 172.21.15.47:0/1203794182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ddbaa2d40 0x562ddbabe300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.490+0000 7f33023a3740 1 --2- 172.21.15.47:0/1203794182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ddba9dbd0 0x562ddbabd340 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.490+0000 7f33023a3740 1 --2- 172.21.15.47:0/1203794182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ddba9d1e0 0x562ddbabce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.490+0000 7f32de000640 0 client.26797 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.490+0000 7f33023a3740 1 -- 172.21.15.47:0/1203794182 >> 172.21.15.47:0/1203794182 conn(0x562ddba98750 msgr2=0x562ddbac0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.490+0000 7f33023a3740 1 -- 172.21.15.47:0/1203794182 shutdown_connections 2024-03-21T11:27:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.491+0000 7f33023a3740 1 -- 172.21.15.47:0/1203794182 wait complete. 2024-03-21T11:27:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.491+0000 7f33023a3740 20 client.26797 trim_cache size 0 max 16384 2024-03-21T11:27:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:27:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:27:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:27:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:27:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.492+0000 7f32ff600640 1 -- 172.21.15.47:0/2743172462 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32e00a3200 msgr2=0x7f32e00a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.492+0000 7f32ff600640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32e00a3200 0x7f32e00a56c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009850 tx=0x7f32ec0dd040 comp rx=0 tx=0).stop 2024-03-21T11:27:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.492+0000 7f32ff600640 1 -- 172.21.15.47:0/2743172462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32f80fff30 msgr2=0x7f32f819ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.492+0000 7f32ff600640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32f80fff30 0x7f32f819ded0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f32ec02c250 tx=0x7f32ec004e30 comp rx=0 tx=0).stop 2024-03-21T11:27:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.492+0000 7f32ff600640 1 -- 172.21.15.47:0/2743172462 shutdown_connections 2024-03-21T11:27:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.492+0000 7f32ff600640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32e00a3200 0x7f32e00a56c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.492+0000 7f32ff600640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32f8105250 0x7f32f81a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.492+0000 7f32ff600640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32f81028a0 0x7f32f819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.492+0000 7f32ff600640 1 --2- 172.21.15.47:0/2743172462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32f80fff30 0x7f32f819ded0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.492+0000 7f32ff600640 1 -- 172.21.15.47:0/2743172462 >> 172.21.15.47:0/2743172462 conn(0x7f32f80fbb30 msgr2=0x7f32f81031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.492+0000 7f32ff600640 1 -- 172.21.15.47:0/2743172462 shutdown_connections 2024-03-21T11:27:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:36.492+0000 7f32ff600640 1 -- 172.21.15.47:0/2743172462 wait complete. 2024-03-21T11:27:36.504 INFO:tasks.workunit.client.0.smithi047.stdout: 1 224255 7.75 MB/sec execute 20 sec latency 103.517 ms 2024-03-21T11:27:36.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:36 smithi102 ceph-mon[36748]: pgmap v528: 129 pgs: 129 active+clean; 37 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2024-03-21T11:27:37.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:36 smithi047 ceph-mon[31351]: pgmap v528: 129 pgs: 129 active+clean; 37 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2024-03-21T11:27:37.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:36 smithi100 ceph-mon[36245]: pgmap v528: 129 pgs: 129 active+clean; 37 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2024-03-21T11:27:37.505 INFO:tasks.workunit.client.0.smithi047.stdout: 1 226428 7.96 MB/sec execute 21 sec latency 11.796 ms 2024-03-21T11:27:38.504 INFO:tasks.workunit.client.0.smithi047.stdout: 1 227748 7.91 MB/sec execute 22 sec latency 10.089 ms 2024-03-21T11:27:38.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:38 smithi102 ceph-mon[36748]: pgmap v529: 129 pgs: 129 active+clean; 36 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 323 op/s 2024-03-21T11:27:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:38 smithi047 ceph-mon[31351]: pgmap v529: 129 pgs: 129 active+clean; 36 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 323 op/s 2024-03-21T11:27:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:38 smithi100 ceph-mon[36245]: pgmap v529: 129 pgs: 129 active+clean; 36 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 323 op/s 2024-03-21T11:27:39.506 INFO:tasks.workunit.client.0.smithi047.stdout: 1 229657 7.91 MB/sec execute 23 sec latency 21.296 ms 2024-03-21T11:27:40.505 INFO:tasks.workunit.client.0.smithi047.stdout: 1 231201 8.04 MB/sec execute 24 sec latency 12.135 ms 2024-03-21T11:27:40.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:40 smithi102 ceph-mon[36748]: pgmap v530: 129 pgs: 129 active+clean; 33 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 310 op/s 2024-03-21T11:27:41.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:40 smithi047 ceph-mon[31351]: pgmap v530: 129 pgs: 129 active+clean; 33 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 310 op/s 2024-03-21T11:27:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:40 smithi100 ceph-mon[36245]: pgmap v530: 129 pgs: 129 active+clean; 33 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 310 op/s 2024-03-21T11:27:41.505 INFO:tasks.workunit.client.0.smithi047.stdout: 1 232497 7.93 MB/sec execute 25 sec latency 80.669 ms 2024-03-21T11:27:42.505 INFO:tasks.workunit.client.0.smithi047.stdout: 1 234362 8.14 MB/sec execute 26 sec latency 12.893 ms 2024-03-21T11:27:42.944 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:42 smithi100 ceph-mon[36245]: pgmap v531: 129 pgs: 129 active+clean; 32 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 2.0 MiB/s wr, 265 op/s 2024-03-21T11:27:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:42 smithi102 ceph-mon[36748]: pgmap v531: 129 pgs: 129 active+clean; 32 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 2.0 MiB/s wr, 265 op/s 2024-03-21T11:27:43.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:42 smithi047 ceph-mon[31351]: pgmap v531: 129 pgs: 129 active+clean; 32 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 2.0 MiB/s wr, 265 op/s 2024-03-21T11:27:43.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:27:43.456 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:27:43.505 INFO:tasks.workunit.client.0.smithi047.stdout: 1 235206 7.92 MB/sec execute 27 sec latency 12.814 ms 2024-03-21T11:27:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.697+0000 7f5541200640 1 -- 172.21.15.47:0/1041134385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553c077c40 msgr2=0x7f553c0784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.697+0000 7f5541200640 1 --2- 172.21.15.47:0/1041134385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553c077c40 0x7f553c0784e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5534009ca0 tx=0x7f553402c0f0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.697+0000 7f5541200640 1 -- 172.21.15.47:0/1041134385 shutdown_connections 2024-03-21T11:27:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.697+0000 7f5541200640 1 --2- 172.21.15.47:0/1041134385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553c077c40 0x7f553c0784e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.697+0000 7f5541200640 1 --2- 172.21.15.47:0/1041134385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c0772a0 0x7f553c077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.697+0000 7f5541200640 1 --2- 172.21.15.47:0/1041134385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c076060 0x7f553c076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.697+0000 7f5541200640 1 -- 172.21.15.47:0/1041134385 >> 172.21.15.47:0/1041134385 conn(0x7f553c0fee50 msgr2=0x7f553c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.697+0000 7f5541200640 1 -- 172.21.15.47:0/1041134385 shutdown_connections 2024-03-21T11:27:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.697+0000 7f5541200640 1 -- 172.21.15.47:0/1041134385 wait complete. 2024-03-21T11:27:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.698+0000 7f5541200640 1 Processor -- start 2024-03-21T11:27:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.698+0000 7f5541200640 1 -- start start 2024-03-21T11:27:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.698+0000 7f5541200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553c076060 0x7f553c19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.698+0000 7f5541200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c0772a0 0x7f553c19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.698+0000 7f5541200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c077c40 0x7f553c1a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.699+0000 7f5541200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c113970 con 0x7f553c076060 2024-03-21T11:27:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.699+0000 7f5541200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c1137f0 con 0x7f553c077c40 2024-03-21T11:27:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.699+0000 7f5541200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c113af0 con 0x7f553c0772a0 2024-03-21T11:27:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.699+0000 7f553a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c077c40 0x7f553c1a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.699+0000 7f5539400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c0772a0 0x7f553c19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.699+0000 7f553a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c077c40 0x7f553c1a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57746/0 (socket says 172.21.15.47:57746) 2024-03-21T11:27:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.699+0000 7f5539400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c0772a0 0x7f553c19e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40132/0 (socket says 172.21.15.47:40132) 2024-03-21T11:27:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.699+0000 7f553a800640 1 -- 172.21.15.47:0/4016222956 learned_addr learned my addr 172.21.15.47:0/4016222956 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.699+0000 7f5539e00640 1 --2- 172.21.15.47:0/4016222956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553c076060 0x7f553c19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.699+0000 7f553a800640 1 -- 172.21.15.47:0/4016222956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c0772a0 msgr2=0x7f553c19e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.699+0000 7f553a800640 1 --2- 172.21.15.47:0/4016222956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c0772a0 0x7f553c19e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.699+0000 7f553a800640 1 -- 172.21.15.47:0/4016222956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553c076060 msgr2=0x7f553c19dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.699+0000 7f553a800640 1 --2- 172.21.15.47:0/4016222956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553c076060 0x7f553c19dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.699+0000 7f553a800640 1 -- 172.21.15.47:0/4016222956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553c1a5be0 con 0x7f553c077c40 2024-03-21T11:27:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.700+0000 7f553a800640 1 --2- 172.21.15.47:0/4016222956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c077c40 0x7f553c1a5570 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f553402c630 tx=0x7f5534032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.700+0000 7f5532a00640 1 -- 172.21.15.47:0/4016222956 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534041020 con 0x7f553c077c40 2024-03-21T11:27:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.700+0000 7f5532a00640 1 -- 172.21.15.47:0/4016222956 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5534034cd0 con 0x7f553c077c40 2024-03-21T11:27:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.700+0000 7f5532a00640 1 -- 172.21.15.47:0/4016222956 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534039040 con 0x7f553c077c40 2024-03-21T11:27:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.700+0000 7f5541200640 1 -- 172.21.15.47:0/4016222956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553c1a5e70 con 0x7f553c077c40 2024-03-21T11:27:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.700+0000 7f5541200640 1 -- 172.21.15.47:0/4016222956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553c1a6280 con 0x7f553c077c40 2024-03-21T11:27:43.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.703+0000 7f5543fe8740 1 Processor -- start 2024-03-21T11:27:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.704+0000 7f5532a00640 1 -- 172.21.15.47:0/4016222956 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5534044020 con 0x7f553c077c40 2024-03-21T11:27:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.705+0000 7f5543fe8740 1 -- start start 2024-03-21T11:27:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.705+0000 7f5543fe8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 0x5613cd8d32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.705+0000 7f5543fe8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d3870 0x5613cd8d8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.705+0000 7f5543fe8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d89e0 0x5613cd8dae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.705+0000 7f5543fe8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cd838250 con 0x5613cd8d2e80 2024-03-21T11:27:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.705+0000 7f5543fe8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cd7e0a30 con 0x5613cd8d3870 2024-03-21T11:27:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.705+0000 7f5543fe8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cd8342f0 con 0x5613cd8d89e0 2024-03-21T11:27:43.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.705+0000 7f5539e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d3870 0x5613cd8d8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.705+0000 7f5539e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d3870 0x5613cd8d8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57754/0 (socket says 172.21.15.47:57754) 2024-03-21T11:27:43.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.705+0000 7f5539e00640 1 -- 172.21.15.47:0/2757198382 learned_addr learned my addr 172.21.15.47:0/2757198382 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.705+0000 7f5539400640 1 --2- 172.21.15.47:0/2757198382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 0x5613cd8d32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.706+0000 7f5539400640 1 --2- 172.21.15.47:0/2757198382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d89e0 0x5613cd8dae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.706+0000 7f5539e00640 1 -- 172.21.15.47:0/2757198382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d89e0 msgr2=0x5613cd8dae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.706+0000 7f5539e00640 1 --2- 172.21.15.47:0/2757198382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d89e0 0x5613cd8dae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.706+0000 7f5539e00640 1 -- 172.21.15.47:0/2757198382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 msgr2=0x5613cd8d32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.706+0000 7f5539e00640 1 --2- 172.21.15.47:0/2757198382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 0x5613cd8d32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.706+0000 7f5539e00640 1 -- 172.21.15.47:0/2757198382 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613cd7cb690 con 0x5613cd8d3870 2024-03-21T11:27:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.706+0000 7f5539400640 1 --2- 172.21.15.47:0/2757198382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d89e0 0x5613cd8dae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:27:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.706+0000 7f5532a00640 1 --2- 172.21.15.47:0/4016222956 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f55200a2fa0 0x7f55200a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.706+0000 7f5539e00640 1 --2- 172.21.15.47:0/2757198382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d3870 0x5613cd8d8370 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f552c009cf0 tx=0x7f552c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9b08d6ae5695d2e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.706+0000 7f5527400640 1 -- 172.21.15.47:0/2757198382 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c004230 con 0x5613cd8d3870 2024-03-21T11:27:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.706+0000 7f5532a00640 1 -- 172.21.15.47:0/4016222956 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f55340f0f10 con 0x7f553c077c40 2024-03-21T11:27:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.707+0000 7f5527400640 1 -- 172.21.15.47:0/2757198382 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f552c0043d0 con 0x5613cd8d3870 2024-03-21T11:27:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.707+0000 7f5527400640 1 -- 172.21.15.47:0/2757198382 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c019920 con 0x5613cd8d3870 2024-03-21T11:27:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.707+0000 7f5539400640 1 --2- 172.21.15.47:0/4016222956 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f55200a2fa0 0x7f55200a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.707+0000 7f5543fe8740 1 -- 172.21.15.47:0/2757198382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d3870 msgr2=0x5613cd8d8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:43.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.707+0000 7f5543fe8740 1 --2- 172.21.15.47:0/2757198382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d3870 0x5613cd8d8370 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f552c009cf0 tx=0x7f552c011810 comp rx=0 tx=0).stop 2024-03-21T11:27:43.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.707+0000 7f5539400640 1 --2- 172.21.15.47:0/4016222956 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f55200a2fa0 0x7f55200a5460 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f55280058c0 tx=0x7f5528015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.708+0000 7f5543fe8740 1 -- 172.21.15.47:0/2757198382 shutdown_connections 2024-03-21T11:27:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.708+0000 7f5543fe8740 1 --2- 172.21.15.47:0/2757198382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d89e0 0x5613cd8dae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.708+0000 7f5543fe8740 1 --2- 172.21.15.47:0/2757198382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d3870 0x5613cd8d8370 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.708+0000 7f5543fe8740 1 --2- 172.21.15.47:0/2757198382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 0x5613cd8d32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.708+0000 7f5543fe8740 1 -- 172.21.15.47:0/2757198382 >> 172.21.15.47:0/2757198382 conn(0x5613cd89b200 msgr2=0x5613cd8ce7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.708+0000 7f5543fe8740 1 -- 172.21.15.47:0/2757198382 shutdown_connections 2024-03-21T11:27:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.708+0000 7f5543fe8740 1 -- 172.21.15.47:0/2757198382 wait complete. 2024-03-21T11:27:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.708+0000 7f5543fe8740 1 -- start start 2024-03-21T11:27:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.708+0000 7f5543fe8740 1 Processor -- start 2024-03-21T11:27:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.709+0000 7f5543fe8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 0x5613cd8dcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.709+0000 7f5543fe8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d3870 0x5613cd8dd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.709+0000 7f5543fe8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d89e0 0x5613cd8e4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.709+0000 7f5543fe8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cd8db990 con 0x5613cd8d2e80 2024-03-21T11:27:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.709+0000 7f5543fe8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cd7f3e80 con 0x5613cd8d3870 2024-03-21T11:27:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.709+0000 7f5543fe8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cd8dbb10 con 0x5613cd8d89e0 2024-03-21T11:27:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.709+0000 7f5543fe8740 10 client.0 _ll_register_callbacks cb 0x5613cd806130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.709+0000 7f5539400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d3870 0x5613cd8dd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.709+0000 7f5539400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d3870 0x5613cd8dd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57758/0 (socket says 172.21.15.47:57758) 2024-03-21T11:27:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.709+0000 7f5539400640 1 -- 172.21.15.47:0/2594718155 learned_addr learned my addr 172.21.15.47:0/2594718155 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.709+0000 7f553a800640 1 --2- 172.21.15.47:0/2594718155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d89e0 0x5613cd8e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.709+0000 7f5539400640 1 -- 172.21.15.47:0/2594718155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d89e0 msgr2=0x5613cd8e4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.709+0000 7f5539400640 1 --2- 172.21.15.47:0/2594718155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d89e0 0x5613cd8e4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.709+0000 7f5539400640 1 -- 172.21.15.47:0/2594718155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 msgr2=0x5613cd8dcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.710+0000 7f5539e00640 1 --2- 172.21.15.47:0/2594718155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 0x5613cd8dcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.710+0000 7f5539400640 1 --2- 172.21.15.47:0/2594718155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 0x5613cd8dcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.710+0000 7f5539400640 1 -- 172.21.15.47:0/2594718155 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613cd6167d0 con 0x5613cd8d3870 2024-03-21T11:27:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.710+0000 7f5539e00640 1 --2- 172.21.15.47:0/2594718155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 0x5613cd8dcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:27:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.710+0000 7f5539400640 1 --2- 172.21.15.47:0/2594718155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d3870 0x5613cd8dd310 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f552801fc00 tx=0x7f5528004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fc99248b620bae5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.710+0000 7f551be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.711+0000 7f551be00640 1 -- 172.21.15.47:0/2594718155 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528005020 con 0x5613cd8d3870 2024-03-21T11:27:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.711+0000 7f551be00640 1 -- 172.21.15.47:0/2594718155 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f55280051c0 con 0x5613cd8d3870 2024-03-21T11:27:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.711+0000 7f5543fe8740 1 -- 172.21.15.47:0/2594718155 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f552c0043d0 con 0x5613cd8d3870 2024-03-21T11:27:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.711+0000 7f551be00640 1 -- 172.21.15.47:0/2594718155 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5528032050 con 0x5613cd8d3870 2024-03-21T11:27:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.711+0000 7f5543fe8740 10 client.26786 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.711+0000 7f5543fe8740 1 -- 172.21.15.47:0/2594718155 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5613cd54c3f0 con 0x5613cd8d3870 2024-03-21T11:27:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.711+0000 7f551be00640 1 -- 172.21.15.47:0/2594718155 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f552c0043d0 con 0x5613cd8d3870 2024-03-21T11:27:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.712+0000 7f551be00640 1 -- 172.21.15.47:0/2594718155 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f552801c070 con 0x5613cd8d3870 2024-03-21T11:27:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.712+0000 7f5543fe8740 10 client.26786 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.712+0000 7f5543fe8740 10 client.26786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.712+0000 7f5543fe8740 20 client.26786 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.712+0000 7f5543fe8740 1 --2- 172.21.15.47:0/2594718155 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5613cd8e8b80 0x5613cd8eb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.712+0000 7f5543fe8740 4 client.26786 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:43.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.712+0000 7f5543fe8740 1 -- 172.21.15.47:0/2594718155 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613cd845790 con 0x5613cd8e8b80 2024-03-21T11:27:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.712+0000 7f5539e00640 1 --2- 172.21.15.47:0/2594718155 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5613cd8e8b80 0x5613cd8eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.713+0000 7f5539e00640 1 --2- 172.21.15.47:0/2594718155 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5613cd8e8b80 0x5613cd8eb040 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303f60b8e0d15544 server_cookie=b331381b7bf30f72 in_seq=0 out_seq=0 2024-03-21T11:27:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.713+0000 7f551be00640 10 client.26786 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f551be00640 1 -- 172.21.15.47:0/2594718155 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5613cd845790 con 0x5613cd8e8b80 2024-03-21T11:27:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f551be00640 10 client.26786 handle_command_reply: tid=0 2024-03-21T11:27:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f5543fe8740 1 client.26786 shutdown 2024-03-21T11:27:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f5543fe8740 1 -- 172.21.15.47:0/2594718155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d3870 msgr2=0x5613cd8dd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f5543fe8740 1 --2- 172.21.15.47:0/2594718155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d3870 0x5613cd8dd310 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f552801fc00 tx=0x7f5528004730 comp rx=0 tx=0).stop 2024-03-21T11:27:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f5543fe8740 1 -- 172.21.15.47:0/2594718155 shutdown_connections 2024-03-21T11:27:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f5543fe8740 1 --2- 172.21.15.47:0/2594718155 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5613cd8e8b80 0x5613cd8eb040 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f5543fe8740 1 --2- 172.21.15.47:0/2594718155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d89e0 0x5613cd8e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f5543fe8740 1 --2- 172.21.15.47:0/2594718155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d3870 0x5613cd8dd310 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f5543fe8740 1 --2- 172.21.15.47:0/2594718155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 0x5613cd8dcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f5543fe8740 1 -- 172.21.15.47:0/2594718155 >> 172.21.15.47:0/2594718155 conn(0x5613cd89b200 msgr2=0x5613cd8cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f551be00640 0 client.26786 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f5543fe8740 1 -- 172.21.15.47:0/2594718155 shutdown_connections 2024-03-21T11:27:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f5543fe8740 1 -- 172.21.15.47:0/2594718155 wait complete. 2024-03-21T11:27:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.714+0000 7f5543fe8740 20 client.26786 trim_cache size 0 max 16384 2024-03-21T11:27:43.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.734+0000 7f5543fe8740 1 Processor -- start 2024-03-21T11:27:43.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.734+0000 7f5543fe8740 1 -- start start 2024-03-21T11:27:43.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.734+0000 7f5543fe8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 0x5613cd8d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.734+0000 7f5543fe8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d3870 0x5613cd8e4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.734+0000 7f5543fe8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d89e0 0x5613cd8d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.734+0000 7f5543fe8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c1b06d0 con 0x5613cd8d2e80 2024-03-21T11:27:43.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.734+0000 7f5543fe8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c1b2050 con 0x5613cd8d89e0 2024-03-21T11:27:43.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.734+0000 7f5543fe8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c1b0d30 con 0x5613cd8d3870 2024-03-21T11:27:43.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.734+0000 7f553a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d89e0 0x5613cd8d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.734+0000 7f553a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d89e0 0x5613cd8d1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57764/0 (socket says 172.21.15.47:57764) 2024-03-21T11:27:43.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.734+0000 7f553a800640 1 -- 172.21.15.47:0/2582174329 learned_addr learned my addr 172.21.15.47:0/2582174329 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:43.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.734+0000 7f5539e00640 1 --2- 172.21.15.47:0/2582174329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 0x5613cd8d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.734+0000 7f5539400640 1 --2- 172.21.15.47:0/2582174329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d3870 0x5613cd8e4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.735+0000 7f553a800640 1 -- 172.21.15.47:0/2582174329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d3870 msgr2=0x5613cd8e4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:43.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.735+0000 7f553a800640 1 --2- 172.21.15.47:0/2582174329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d3870 0x5613cd8e4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.735+0000 7f553a800640 1 -- 172.21.15.47:0/2582174329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 msgr2=0x5613cd8d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:43.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.735+0000 7f553a800640 1 --2- 172.21.15.47:0/2582174329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 0x5613cd8d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.735+0000 7f553a800640 1 -- 172.21.15.47:0/2582174329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613cd50c0f0 con 0x5613cd8d89e0 2024-03-21T11:27:43.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.735+0000 7f553a800640 1 --2- 172.21.15.47:0/2582174329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d89e0 0x5613cd8d1f70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f55340bcfe0 tx=0x7f55340bcee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fff5691711bb3ffb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:43.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.735+0000 7f5526a00640 1 -- 172.21.15.47:0/2582174329 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55340bd2e0 con 0x5613cd8d89e0 2024-03-21T11:27:43.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.735+0000 7f5526a00640 1 -- 172.21.15.47:0/2582174329 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55340b7e50 con 0x5613cd8d89e0 2024-03-21T11:27:43.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.735+0000 7f5526a00640 1 -- 172.21.15.47:0/2582174329 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55340b9950 con 0x5613cd8d89e0 2024-03-21T11:27:43.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.736+0000 7f5543fe8740 1 -- 172.21.15.47:0/2582174329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d89e0 msgr2=0x5613cd8d1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:43.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.736+0000 7f5543fe8740 1 --2- 172.21.15.47:0/2582174329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d89e0 0x5613cd8d1f70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f55340bcfe0 tx=0x7f55340bcee0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.736+0000 7f5543fe8740 1 -- 172.21.15.47:0/2582174329 shutdown_connections 2024-03-21T11:27:43.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.736+0000 7f5543fe8740 1 --2- 172.21.15.47:0/2582174329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d89e0 0x5613cd8d1f70 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.736+0000 7f5543fe8740 1 --2- 172.21.15.47:0/2582174329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d3870 0x5613cd8e4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.736+0000 7f5543fe8740 1 --2- 172.21.15.47:0/2582174329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d2e80 0x5613cd8d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.736+0000 7f5543fe8740 1 -- 172.21.15.47:0/2582174329 >> 172.21.15.47:0/2582174329 conn(0x5613cd89b200 msgr2=0x5613cd8ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:43.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.736+0000 7f5543fe8740 1 -- 172.21.15.47:0/2582174329 shutdown_connections 2024-03-21T11:27:43.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.736+0000 7f5543fe8740 1 -- 172.21.15.47:0/2582174329 wait complete. 2024-03-21T11:27:43.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.737+0000 7f5543fe8740 1 -- start start 2024-03-21T11:27:43.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.737+0000 7f5543fe8740 1 Processor -- start 2024-03-21T11:27:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5543fe8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d2e80 0x5613cd8e3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5543fe8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d3870 0x5613cd8e4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5543fe8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d89e0 0x5613cd8e2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5543fe8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c1b19f0 con 0x5613cd8d89e0 2024-03-21T11:27:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5543fe8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c1b1390 con 0x5613cd8d2e80 2024-03-21T11:27:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5543fe8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c1ae890 con 0x5613cd8d3870 2024-03-21T11:27:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5543fe8740 10 client.0 _ll_register_callbacks cb 0x7f5528009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5539e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d2e80 0x5613cd8e3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5539e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d2e80 0x5613cd8e3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57778/0 (socket says 172.21.15.47:57778) 2024-03-21T11:27:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5539e00640 1 -- 172.21.15.47:0/1204679851 learned_addr learned my addr 172.21.15.47:0/1204679851 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5539400640 1 --2- 172.21.15.47:0/1204679851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d3870 0x5613cd8e4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f553a800640 1 --2- 172.21.15.47:0/1204679851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d89e0 0x5613cd8e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5539e00640 1 -- 172.21.15.47:0/1204679851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d3870 msgr2=0x5613cd8e4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:43.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5539e00640 1 --2- 172.21.15.47:0/1204679851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d3870 0x5613cd8e4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5539e00640 1 -- 172.21.15.47:0/1204679851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d89e0 msgr2=0x5613cd8e2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5539e00640 1 --2- 172.21.15.47:0/1204679851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d89e0 0x5613cd8e2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.738+0000 7f5539e00640 1 -- 172.21.15.47:0/1204679851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613cd8e2860 con 0x5613cd8d2e80 2024-03-21T11:27:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.739+0000 7f5539e00640 1 --2- 172.21.15.47:0/1204679851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d2e80 0x5613cd8e3b00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f552c01b040 tx=0x7f552c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50ce67e5fad613f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.739+0000 7f551be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.739+0000 7f551be00640 1 -- 172.21.15.47:0/1204679851 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c031060 con 0x5613cd8d2e80 2024-03-21T11:27:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.739+0000 7f5543fe8740 1 -- 172.21.15.47:0/1204679851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55340b7e50 con 0x5613cd8d2e80 2024-03-21T11:27:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.739+0000 7f551be00640 1 -- 172.21.15.47:0/1204679851 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f552c005040 con 0x5613cd8d2e80 2024-03-21T11:27:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.739+0000 7f551be00640 1 -- 172.21.15.47:0/1204679851 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f552c038050 con 0x5613cd8d2e80 2024-03-21T11:27:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.739+0000 7f5543fe8740 10 client.26792 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:43.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.739+0000 7f5543fe8740 1 -- 172.21.15.47:0/1204679851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5613cd944de0 con 0x5613cd8d2e80 2024-03-21T11:27:43.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.740+0000 7f551be00640 1 -- 172.21.15.47:0/1204679851 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55340b7e50 con 0x5613cd8d2e80 2024-03-21T11:27:43.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.740+0000 7f551be00640 1 -- 172.21.15.47:0/1204679851 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f552c025420 con 0x5613cd8d2e80 2024-03-21T11:27:43.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.740+0000 7f5543fe8740 10 client.26792 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:43.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.740+0000 7f5543fe8740 10 client.26792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:43.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.740+0000 7f5543fe8740 20 client.26792 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:43.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.740+0000 7f5543fe8740 1 --2- 172.21.15.47:0/1204679851 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5613cd9455b0 0x5613cd9479d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:43.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.740+0000 7f5543fe8740 4 client.26792 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:27:43.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.740+0000 7f5543fe8740 1 -- 172.21.15.47:0/1204679851 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5613cd83fae0 con 0x5613cd9455b0 2024-03-21T11:27:43.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.740+0000 7f5539400640 1 --2- 172.21.15.47:0/1204679851 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5613cd9455b0 0x5613cd9479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:43.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f5539400640 1 --2- 172.21.15.47:0/1204679851 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5613cd9455b0 0x5613cd9479d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6145190479c7f32c server_cookie=6c584920fef2bdeb in_seq=0 out_seq=0 2024-03-21T11:27:43.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f551be00640 10 client.26792 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:43.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f551be00640 1 -- 172.21.15.47:0/1204679851 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5613cd83fae0 con 0x5613cd9455b0 2024-03-21T11:27:43.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f551be00640 10 client.26792 handle_command_reply: tid=0 2024-03-21T11:27:43.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f5543fe8740 1 client.26792 shutdown 2024-03-21T11:27:43.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f5543fe8740 1 -- 172.21.15.47:0/1204679851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d2e80 msgr2=0x5613cd8e3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:43.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f5543fe8740 1 --2- 172.21.15.47:0/1204679851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d2e80 0x5613cd8e3b00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f552c01b040 tx=0x7f552c005d30 comp rx=0 tx=0).stop 2024-03-21T11:27:43.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f5543fe8740 1 -- 172.21.15.47:0/1204679851 shutdown_connections 2024-03-21T11:27:43.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f5543fe8740 1 --2- 172.21.15.47:0/1204679851 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5613cd9455b0 0x5613cd9479d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f5543fe8740 1 --2- 172.21.15.47:0/1204679851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5613cd8d89e0 0x5613cd8e2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f5543fe8740 1 --2- 172.21.15.47:0/1204679851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613cd8d3870 0x5613cd8e4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f551be00640 0 client.26792 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:43.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f5543fe8740 1 --2- 172.21.15.47:0/1204679851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5613cd8d2e80 0x5613cd8e3b00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f5543fe8740 1 -- 172.21.15.47:0/1204679851 >> 172.21.15.47:0/1204679851 conn(0x5613cd89b200 msgr2=0x5613cd8cd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:43.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.741+0000 7f5543fe8740 1 -- 172.21.15.47:0/1204679851 shutdown_connections 2024-03-21T11:27:43.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.742+0000 7f5543fe8740 1 -- 172.21.15.47:0/1204679851 wait complete. 2024-03-21T11:27:43.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.742+0000 7f5543fe8740 20 client.26792 trim_cache size 0 max 16384 2024-03-21T11:27:43.746 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:27:43.746 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:27:43.747 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:27:43.747 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:27:43.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.742+0000 7f5541200640 1 -- 172.21.15.47:0/4016222956 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f55200a2fa0 msgr2=0x7f55200a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:43.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.742+0000 7f5541200640 1 --2- 172.21.15.47:0/4016222956 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f55200a2fa0 0x7f55200a5460 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f55280058c0 tx=0x7f5528015040 comp rx=0 tx=0).stop 2024-03-21T11:27:43.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.742+0000 7f5541200640 1 -- 172.21.15.47:0/4016222956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c077c40 msgr2=0x7f553c1a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:43.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.742+0000 7f5541200640 1 --2- 172.21.15.47:0/4016222956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c077c40 0x7f553c1a5570 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f553402c630 tx=0x7f5534032ea0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.742+0000 7f5541200640 1 -- 172.21.15.47:0/4016222956 shutdown_connections 2024-03-21T11:27:43.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.742+0000 7f5541200640 1 --2- 172.21.15.47:0/4016222956 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f55200a2fa0 0x7f55200a5460 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.742+0000 7f5541200640 1 --2- 172.21.15.47:0/4016222956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f553c077c40 0x7f553c1a5570 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.742+0000 7f5541200640 1 --2- 172.21.15.47:0/4016222956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f553c0772a0 0x7f553c19e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.742+0000 7f5541200640 1 --2- 172.21.15.47:0/4016222956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f553c076060 0x7f553c19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:43.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.742+0000 7f5541200640 1 -- 172.21.15.47:0/4016222956 >> 172.21.15.47:0/4016222956 conn(0x7f553c0fee50 msgr2=0x7f553c1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:43.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.743+0000 7f5541200640 1 -- 172.21.15.47:0/4016222956 shutdown_connections 2024-03-21T11:27:43.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:43.743+0000 7f5541200640 1 -- 172.21.15.47:0/4016222956 wait complete. 2024-03-21T11:27:43.764 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:27:43.764 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:27:44.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.030+0000 7f7376200640 1 -- 172.21.15.47:0/2903879980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7370103700 msgr2=0x7f7370103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.030+0000 7f7376200640 1 --2- 172.21.15.47:0/2903879980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7370103700 0x7f7370103b20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7360009900 tx=0x7f736002bc30 comp rx=0 tx=0).stop 2024-03-21T11:27:44.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.031+0000 7f7376200640 1 -- 172.21.15.47:0/2903879980 shutdown_connections 2024-03-21T11:27:44.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.031+0000 7f7376200640 1 --2- 172.21.15.47:0/2903879980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73701052b0 0x7f737010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.031+0000 7f7376200640 1 --2- 172.21.15.47:0/2903879980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73701048f0 0x7f7370104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.031+0000 7f7376200640 1 --2- 172.21.15.47:0/2903879980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7370103700 0x7f7370103b20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.031+0000 7f7376200640 1 -- 172.21.15.47:0/2903879980 >> 172.21.15.47:0/2903879980 conn(0x7f73700fec70 msgr2=0x7f73701010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:44.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.031+0000 7f7376200640 1 -- 172.21.15.47:0/2903879980 shutdown_connections 2024-03-21T11:27:44.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.031+0000 7f7376200640 1 -- 172.21.15.47:0/2903879980 wait complete. 2024-03-21T11:27:44.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.032+0000 7f7376200640 1 Processor -- start 2024-03-21T11:27:44.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.032+0000 7f7376200640 1 -- start start 2024-03-21T11:27:44.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.032+0000 7f7376200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7370103700 0x7f7370199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.032+0000 7f7376200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73701048f0 0x7f7370199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.032+0000 7f7376200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73701052b0 0x7f73701a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.032+0000 7f7376200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737010f3c0 con 0x7f73701048f0 2024-03-21T11:27:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.032+0000 7f7376200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737010f240 con 0x7f73701052b0 2024-03-21T11:27:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.032+0000 7f7376200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737010f540 con 0x7f7370103700 2024-03-21T11:27:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.033+0000 7f7374e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7370103700 0x7f7370199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.033+0000 7f7374e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7370103700 0x7f7370199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40176/0 (socket says 172.21.15.47:40176) 2024-03-21T11:27:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.033+0000 7f7374e00640 1 -- 172.21.15.47:0/1319348225 learned_addr learned my addr 172.21.15.47:0/1319348225 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.033+0000 7f736fe00640 1 --2- 172.21.15.47:0/1319348225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73701048f0 0x7f7370199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.033+0000 7f7375800640 1 --2- 172.21.15.47:0/1319348225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73701052b0 0x7f73701a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.033+0000 7f7374e00640 1 -- 172.21.15.47:0/1319348225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73701052b0 msgr2=0x7f73701a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.033+0000 7f7374e00640 1 --2- 172.21.15.47:0/1319348225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73701052b0 0x7f73701a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.033+0000 7f7374e00640 1 -- 172.21.15.47:0/1319348225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73701048f0 msgr2=0x7f7370199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.033+0000 7f7374e00640 1 --2- 172.21.15.47:0/1319348225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73701048f0 0x7f7370199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.033+0000 7f7374e00640 1 -- 172.21.15.47:0/1319348225 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73701a16b0 con 0x7f7370103700 2024-03-21T11:27:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.034+0000 7f7374e00640 1 --2- 172.21.15.47:0/1319348225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7370103700 0x7f7370199a90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7360000c00 tx=0x7f736002c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.034+0000 7f736d600640 1 -- 172.21.15.47:0/1319348225 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360043020 con 0x7f7370103700 2024-03-21T11:27:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.034+0000 7f736d600640 1 -- 172.21.15.47:0/1319348225 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f73600054e0 con 0x7f7370103700 2024-03-21T11:27:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.034+0000 7f736d600640 1 -- 172.21.15.47:0/1319348225 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360003ce0 con 0x7f7370103700 2024-03-21T11:27:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.034+0000 7f7376200640 1 -- 172.21.15.47:0/1319348225 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73701a1940 con 0x7f7370103700 2024-03-21T11:27:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.034+0000 7f7376200640 1 -- 172.21.15.47:0/1319348225 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73701a1eb0 con 0x7f7370103700 2024-03-21T11:27:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.035+0000 7f7376200640 1 -- 172.21.15.47:0/1319348225 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f737010aac0 con 0x7f7370103700 2024-03-21T11:27:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.037+0000 7f736d600640 1 -- 172.21.15.47:0/1319348225 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7360046020 con 0x7f7370103700 2024-03-21T11:27:44.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.039+0000 7f736d600640 1 --2- 172.21.15.47:0/1319348225 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73500a2fa0 0x7f73500a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.039+0000 7f736d600640 1 -- 172.21.15.47:0/1319348225 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f73600f1840 con 0x7f7370103700 2024-03-21T11:27:44.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.039+0000 7f736fe00640 1 --2- 172.21.15.47:0/1319348225 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73500a2fa0 0x7f73500a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.040+0000 7f736fe00640 1 --2- 172.21.15.47:0/1319348225 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73500a2fa0 0x7f73500a5460 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f737019afa0 tx=0x7f7358009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.044+0000 7f736d600640 1 -- 172.21.15.47:0/1319348225 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f736004d020 con 0x7f7370103700 2024-03-21T11:27:44.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.360+0000 7f7376200640 1 -- 172.21.15.47:0/1319348225 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73700622a0 con 0x7f7370103700 2024-03-21T11:27:44.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.361+0000 7f736d600640 1 -- 172.21.15.47:0/1319348225 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f73600be280 con 0x7f7370103700 2024-03-21T11:27:44.362 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:27:44.363 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:27:44.363 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:27:44.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.364+0000 7f7376200640 1 -- 172.21.15.47:0/1319348225 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73500a2fa0 msgr2=0x7f73500a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.364+0000 7f7376200640 1 --2- 172.21.15.47:0/1319348225 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73500a2fa0 0x7f73500a5460 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f737019afa0 tx=0x7f7358009500 comp rx=0 tx=0).stop 2024-03-21T11:27:44.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.364+0000 7f7376200640 1 -- 172.21.15.47:0/1319348225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7370103700 msgr2=0x7f7370199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.364+0000 7f7376200640 1 --2- 172.21.15.47:0/1319348225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7370103700 0x7f7370199a90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7360000c00 tx=0x7f736002c6c0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.364+0000 7f7376200640 1 -- 172.21.15.47:0/1319348225 shutdown_connections 2024-03-21T11:27:44.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.364+0000 7f7376200640 1 --2- 172.21.15.47:0/1319348225 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73500a2fa0 0x7f73500a5460 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.364+0000 7f7376200640 1 --2- 172.21.15.47:0/1319348225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73701052b0 0x7f73701a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.364+0000 7f7376200640 1 --2- 172.21.15.47:0/1319348225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73701048f0 0x7f7370199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.364+0000 7f7376200640 1 --2- 172.21.15.47:0/1319348225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7370103700 0x7f7370199a90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.364+0000 7f7376200640 1 -- 172.21.15.47:0/1319348225 >> 172.21.15.47:0/1319348225 conn(0x7f73700fec70 msgr2=0x7f7370107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:44.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.365+0000 7f7376200640 1 -- 172.21.15.47:0/1319348225 shutdown_connections 2024-03-21T11:27:44.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.365+0000 7f7376200640 1 -- 172.21.15.47:0/1319348225 wait complete. 2024-03-21T11:27:44.388 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:44.388 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:27:44.505 INFO:tasks.workunit.client.0.smithi047.stdout: 1 237511 8.09 MB/sec execute 28 sec latency 24.420 ms 2024-03-21T11:27:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.607+0000 7f875a600640 1 -- 172.21.15.47:0/2513504372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8754077c40 msgr2=0x7f87540784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.607+0000 7f875a600640 1 --2- 172.21.15.47:0/2513504372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8754077c40 0x7f87540784e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f874c009ca0 tx=0x7f874c02c100 comp rx=0 tx=0).stop 2024-03-21T11:27:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.608+0000 7f875a600640 1 -- 172.21.15.47:0/2513504372 shutdown_connections 2024-03-21T11:27:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.608+0000 7f875a600640 1 --2- 172.21.15.47:0/2513504372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8754077c40 0x7f87540784e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.608+0000 7f875a600640 1 --2- 172.21.15.47:0/2513504372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87540772a0 0x7f8754077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.608+0000 7f875a600640 1 --2- 172.21.15.47:0/2513504372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8754076060 0x7f8754076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.608+0000 7f875a600640 1 -- 172.21.15.47:0/2513504372 >> 172.21.15.47:0/2513504372 conn(0x7f87540fed10 msgr2=0x7f8754101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.608+0000 7f875a600640 1 -- 172.21.15.47:0/2513504372 shutdown_connections 2024-03-21T11:27:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.608+0000 7f875a600640 1 -- 172.21.15.47:0/2513504372 wait complete. 2024-03-21T11:27:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.609+0000 7f875a600640 1 Processor -- start 2024-03-21T11:27:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.609+0000 7f875a600640 1 -- start start 2024-03-21T11:27:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.609+0000 7f875a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8754076060 0x7f875419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.609+0000 7f875a600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87540772a0 0x7f875419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.609+0000 7f875a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8754077c40 0x7f87541a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.609+0000 7f875a600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8754113830 con 0x7f87540772a0 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.609+0000 7f875a600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87541136b0 con 0x7f8754077c40 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.609+0000 7f875a600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87541139b0 con 0x7f8754076060 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.610+0000 7f8759200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8754076060 0x7f875419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.610+0000 7f8759c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8754077c40 0x7f87541a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.610+0000 7f8753e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87540772a0 0x7f875419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.610+0000 7f8759200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8754076060 0x7f875419de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40208/0 (socket says 172.21.15.47:40208) 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.610+0000 7f8759200640 1 -- 172.21.15.47:0/1190666730 learned_addr learned my addr 172.21.15.47:0/1190666730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.610+0000 7f8759c00640 1 -- 172.21.15.47:0/1190666730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8754076060 msgr2=0x7f875419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.610+0000 7f8759c00640 1 --2- 172.21.15.47:0/1190666730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8754076060 0x7f875419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.610+0000 7f8759c00640 1 -- 172.21.15.47:0/1190666730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87540772a0 msgr2=0x7f875419e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.610+0000 7f8759c00640 1 --2- 172.21.15.47:0/1190666730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87540772a0 0x7f875419e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.610+0000 7f8759c00640 1 -- 172.21.15.47:0/1190666730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87541a5a50 con 0x7f8754077c40 2024-03-21T11:27:44.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.610+0000 7f8759c00640 1 --2- 172.21.15.47:0/1190666730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8754077c40 0x7f87541a53e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f874c037040 tx=0x7f874c035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:44.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.611+0000 7f8751600640 1 -- 172.21.15.47:0/1190666730 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c039040 con 0x7f8754077c40 2024-03-21T11:27:44.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.611+0000 7f8751600640 1 -- 172.21.15.47:0/1190666730 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f874c044070 con 0x7f8754077c40 2024-03-21T11:27:44.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.611+0000 7f875a600640 1 -- 172.21.15.47:0/1190666730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87541a5ce0 con 0x7f8754077c40 2024-03-21T11:27:44.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.611+0000 7f8751600640 1 -- 172.21.15.47:0/1190666730 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c0044e0 con 0x7f8754077c40 2024-03-21T11:27:44.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.611+0000 7f875a600640 1 -- 172.21.15.47:0/1190666730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87541a61a0 con 0x7f8754077c40 2024-03-21T11:27:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.614+0000 7f8751600640 1 -- 172.21.15.47:0/1190666730 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f874c032890 con 0x7f8754077c40 2024-03-21T11:27:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.614+0000 7f875e966740 1 Processor -- start 2024-03-21T11:27:44.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.615+0000 7f875e966740 1 -- start start 2024-03-21T11:27:44.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.615+0000 7f875e966740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989c9f0 0x55f76989ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.615+0000 7f875e966740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989dbc0 0x55f76989e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.615+0000 7f875e966740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 0x55f7698a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.615+0000 7f875e966740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7698012c0 con 0x55f76989dbc0 2024-03-21T11:27:44.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.615+0000 7f875e966740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7697a9c90 con 0x55f76989c9f0 2024-03-21T11:27:44.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.615+0000 7f875e966740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7697fd360 con 0x55f76989e580 2024-03-21T11:27:44.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.615+0000 7f8753e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989c9f0 0x55f76989ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.616+0000 7f8753e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989c9f0 0x55f76989ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57816/0 (socket says 172.21.15.47:57816) 2024-03-21T11:27:44.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.616+0000 7f8753e00640 1 -- 172.21.15.47:0/3814042883 learned_addr learned my addr 172.21.15.47:0/3814042883 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:44.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.616+0000 7f8759200640 1 --2- 172.21.15.47:0/3814042883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989dbc0 0x55f76989e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.616+0000 7f8753e00640 1 --2- 172.21.15.47:0/3814042883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 0x55f7698a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.616+0000 7f8753e00640 1 -- 172.21.15.47:0/3814042883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 msgr2=0x55f7698a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.616+0000 7f8753e00640 1 --2- 172.21.15.47:0/3814042883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 0x55f7698a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.616+0000 7f8753e00640 1 -- 172.21.15.47:0/3814042883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989dbc0 msgr2=0x55f76989e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.616+0000 7f8753e00640 1 --2- 172.21.15.47:0/3814042883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989dbc0 0x55f76989e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.616+0000 7f8753e00640 1 -- 172.21.15.47:0/3814042883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7697ce080 con 0x55f76989c9f0 2024-03-21T11:27:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.616+0000 7f8751600640 1 --2- 172.21.15.47:0/1190666730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f873c0a2f20 0x7f873c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.616+0000 7f8753e00640 1 --2- 172.21.15.47:0/3814042883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989c9f0 0x55f76989ce10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f874000b6c0 tx=0x7f874000b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90330ad94d50ced4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.617+0000 7f8751600640 1 -- 172.21.15.47:0/1190666730 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f874c0f1e10 con 0x7f8754077c40 2024-03-21T11:27:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.617+0000 7f8759200640 1 --2- 172.21.15.47:0/1190666730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f873c0a2f20 0x7f873c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.617+0000 7f8746000640 1 -- 172.21.15.47:0/3814042883 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740012d00 con 0x55f76989c9f0 2024-03-21T11:27:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.617+0000 7f8746000640 1 -- 172.21.15.47:0/3814042883 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8740010770 con 0x55f76989c9f0 2024-03-21T11:27:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.617+0000 7f8746000640 1 -- 172.21.15.47:0/3814042883 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740003830 con 0x55f76989c9f0 2024-03-21T11:27:44.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.618+0000 7f875e966740 1 -- 172.21.15.47:0/3814042883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989c9f0 msgr2=0x55f76989ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.618+0000 7f8759200640 1 --2- 172.21.15.47:0/1190666730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f873c0a2f20 0x7f873c0a53e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8748005fd0 tx=0x7f874800b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:44.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.618+0000 7f875e966740 1 --2- 172.21.15.47:0/3814042883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989c9f0 0x55f76989ce10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f874000b6c0 tx=0x7f874000b650 comp rx=0 tx=0).stop 2024-03-21T11:27:44.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.618+0000 7f875e966740 1 -- 172.21.15.47:0/3814042883 shutdown_connections 2024-03-21T11:27:44.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.618+0000 7f875e966740 1 --2- 172.21.15.47:0/3814042883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 0x55f7698a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.618+0000 7f875e966740 1 --2- 172.21.15.47:0/3814042883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989dbc0 0x55f76989e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.618+0000 7f875e966740 1 --2- 172.21.15.47:0/3814042883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989c9f0 0x55f76989ce10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.618+0000 7f875e966740 1 -- 172.21.15.47:0/3814042883 >> 172.21.15.47:0/3814042883 conn(0x55f769897de0 msgr2=0x55f7698981e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.618+0000 7f875e966740 1 -- 172.21.15.47:0/3814042883 shutdown_connections 2024-03-21T11:27:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.618+0000 7f875e966740 1 -- 172.21.15.47:0/3814042883 wait complete. 2024-03-21T11:27:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.618+0000 7f875e966740 1 -- start start 2024-03-21T11:27:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.619+0000 7f875e966740 1 Processor -- start 2024-03-21T11:27:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.619+0000 7f875e966740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989c9f0 0x55f7698a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.619+0000 7f875e966740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989dbc0 0x55f7698a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.619+0000 7f875e966740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 0x55f7698ae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.620+0000 7f875e966740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7698a5590 con 0x55f76989dbc0 2024-03-21T11:27:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.620+0000 7f875e966740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7697bd0d0 con 0x55f76989c9f0 2024-03-21T11:27:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.620+0000 7f875e966740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7698a5710 con 0x55f76989e580 2024-03-21T11:27:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.620+0000 7f875e966740 10 client.0 _ll_register_callbacks cb 0x55f7697c1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.620+0000 7f8759200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989c9f0 0x55f7698a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.620+0000 7f8753e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989dbc0 0x55f7698a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.620+0000 7f8759200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989c9f0 0x55f7698a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57830/0 (socket says 172.21.15.47:57830) 2024-03-21T11:27:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.620+0000 7f8759200640 1 -- 172.21.15.47:0/398682839 learned_addr learned my addr 172.21.15.47:0/398682839 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.620+0000 7f8759c00640 1 --2- 172.21.15.47:0/398682839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 0x55f7698ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.620+0000 7f8759200640 1 -- 172.21.15.47:0/398682839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 msgr2=0x55f7698ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.620+0000 7f8759200640 1 --2- 172.21.15.47:0/398682839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 0x55f7698ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.620+0000 7f8759200640 1 -- 172.21.15.47:0/398682839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989dbc0 msgr2=0x55f7698a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.620+0000 7f8759200640 1 --2- 172.21.15.47:0/398682839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989dbc0 0x55f7698a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.620+0000 7f8759200640 1 -- 172.21.15.47:0/398682839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7695153f0 con 0x55f76989c9f0 2024-03-21T11:27:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.621+0000 7f8759c00640 1 --2- 172.21.15.47:0/398682839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 0x55f7698ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.621+0000 7f8753e00640 1 --2- 172.21.15.47:0/398682839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989dbc0 0x55f7698a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.621+0000 7f8759200640 1 --2- 172.21.15.47:0/398682839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989c9f0 0x55f7698a6cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8748003df0 tx=0x7f8748004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9301350c1cb25a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.621+0000 7f873aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.621+0000 7f873aa00640 1 -- 172.21.15.47:0/398682839 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748005020 con 0x55f76989c9f0 2024-03-21T11:27:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.621+0000 7f873aa00640 1 -- 172.21.15.47:0/398682839 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8748017030 con 0x55f76989c9f0 2024-03-21T11:27:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.621+0000 7f875e966740 1 -- 172.21.15.47:0/398682839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8740010770 con 0x55f76989c9f0 2024-03-21T11:27:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.621+0000 7f873aa00640 1 -- 172.21.15.47:0/398682839 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8748032050 con 0x55f76989c9f0 2024-03-21T11:27:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.621+0000 7f875e966740 10 client.26810 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.621+0000 7f875e966740 1 -- 172.21.15.47:0/398682839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f7695df7d0 con 0x55f76989c9f0 2024-03-21T11:27:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.622+0000 7f873aa00640 1 -- 172.21.15.47:0/398682839 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8740010770 con 0x55f76989c9f0 2024-03-21T11:27:44.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.622+0000 7f873aa00640 1 -- 172.21.15.47:0/398682839 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8748027410 con 0x55f76989c9f0 2024-03-21T11:27:44.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.622+0000 7f875e966740 10 client.26810 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.622+0000 7f875e966740 10 client.26810 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:27:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.622+0000 7f875e966740 20 client.26810 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.622+0000 7f875e966740 1 --2- 172.21.15.47:0/398682839 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f7698b2820 0x55f7698b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.622+0000 7f875e966740 4 client.26810 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.622+0000 7f875e966740 1 -- 172.21.15.47:0/398682839 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f76980e800 con 0x55f7698b2820 2024-03-21T11:27:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.622+0000 7f8753e00640 1 --2- 172.21.15.47:0/398682839 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f7698b2820 0x55f7698b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.623+0000 7f8753e00640 1 --2- 172.21.15.47:0/398682839 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f7698b2820 0x55f7698b4c80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2be191e41cdc8837 server_cookie=487f32598b89ab55 in_seq=0 out_seq=0 2024-03-21T11:27:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.623+0000 7f873aa00640 10 client.26810 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:27:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f873aa00640 1 -- 172.21.15.47:0/398682839 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f76980e800 con 0x55f7698b2820 2024-03-21T11:27:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f873aa00640 10 client.26810 handle_command_reply: tid=0 2024-03-21T11:27:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f875e966740 1 client.26810 shutdown 2024-03-21T11:27:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f875e966740 1 -- 172.21.15.47:0/398682839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989c9f0 msgr2=0x55f7698a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f875e966740 1 --2- 172.21.15.47:0/398682839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989c9f0 0x55f7698a6cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8748003df0 tx=0x7f8748004730 comp rx=0 tx=0).stop 2024-03-21T11:27:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f875e966740 1 -- 172.21.15.47:0/398682839 shutdown_connections 2024-03-21T11:27:44.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f875e966740 1 --2- 172.21.15.47:0/398682839 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f7698b2820 0x55f7698b4c80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f875e966740 1 --2- 172.21.15.47:0/398682839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 0x55f7698ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f875e966740 1 --2- 172.21.15.47:0/398682839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989dbc0 0x55f7698a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f875e966740 1 --2- 172.21.15.47:0/398682839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989c9f0 0x55f7698a6cc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f873aa00640 0 client.26810 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:27:44.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f875e966740 1 -- 172.21.15.47:0/398682839 >> 172.21.15.47:0/398682839 conn(0x55f769897de0 msgr2=0x55f769897160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:44.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f875e966740 1 -- 172.21.15.47:0/398682839 shutdown_connections 2024-03-21T11:27:44.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f875e966740 1 -- 172.21.15.47:0/398682839 wait complete. 2024-03-21T11:27:44.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.624+0000 7f875e966740 20 client.26810 trim_cache size 0 max 16384 2024-03-21T11:27:44.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.639+0000 7f875e966740 1 Processor -- start 2024-03-21T11:27:44.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.640+0000 7f875e966740 1 -- start start 2024-03-21T11:27:44.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.640+0000 7f875e966740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989c9f0 0x55f7698a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.640+0000 7f875e966740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989dbc0 0x55f7698ae200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.640+0000 7f875e966740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989e580 0x55f7698a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.640+0000 7f875e966740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87541b03d0 con 0x55f76989c9f0 2024-03-21T11:27:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.640+0000 7f875e966740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87541b1d50 con 0x55f76989e580 2024-03-21T11:27:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.640+0000 7f875e966740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87541b0a30 con 0x55f76989dbc0 2024-03-21T11:27:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.640+0000 7f8753e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989dbc0 0x55f7698ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.640+0000 7f8759c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989e580 0x55f7698a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.640+0000 7f8753e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989dbc0 0x55f7698ae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40232/0 (socket says 172.21.15.47:40232) 2024-03-21T11:27:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.640+0000 7f8753e00640 1 -- 172.21.15.47:0/3756828498 learned_addr learned my addr 172.21.15.47:0/3756828498 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.640+0000 7f8759200640 1 --2- 172.21.15.47:0/3756828498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989c9f0 0x55f7698a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.641+0000 7f8753e00640 1 -- 172.21.15.47:0/3756828498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989e580 msgr2=0x55f7698a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.641+0000 7f8753e00640 1 --2- 172.21.15.47:0/3756828498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989e580 0x55f7698a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.641+0000 7f8753e00640 1 -- 172.21.15.47:0/3756828498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989c9f0 msgr2=0x55f7698a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.641+0000 7f8753e00640 1 --2- 172.21.15.47:0/3756828498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989c9f0 0x55f7698a7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.641+0000 7f8753e00640 1 -- 172.21.15.47:0/3756828498 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7698a4750 con 0x55f76989dbc0 2024-03-21T11:27:44.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.641+0000 7f8759200640 1 --2- 172.21.15.47:0/3756828498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989c9f0 0x55f7698a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:44.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.641+0000 7f8759c00640 1 --2- 172.21.15.47:0/3756828498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989e580 0x55f7698a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:44.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.641+0000 7f8753e00640 1 --2- 172.21.15.47:0/3756828498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989dbc0 0x55f7698ae200 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f874000b6c0 tx=0x7f874001b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8592e133c5b4c7f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.641+0000 7f8745600640 1 -- 172.21.15.47:0/3756828498 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740005040 con 0x55f76989dbc0 2024-03-21T11:27:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.641+0000 7f8745600640 1 -- 172.21.15.47:0/3756828498 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f874002e070 con 0x55f76989dbc0 2024-03-21T11:27:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.641+0000 7f8745600640 1 -- 172.21.15.47:0/3756828498 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740010d30 con 0x55f76989dbc0 2024-03-21T11:27:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.642+0000 7f875e966740 1 -- 172.21.15.47:0/3756828498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989dbc0 msgr2=0x55f7698ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.642+0000 7f875e966740 1 --2- 172.21.15.47:0/3756828498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989dbc0 0x55f7698ae200 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f874000b6c0 tx=0x7f874001b040 comp rx=0 tx=0).stop 2024-03-21T11:27:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.642+0000 7f875e966740 1 -- 172.21.15.47:0/3756828498 shutdown_connections 2024-03-21T11:27:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.642+0000 7f875e966740 1 --2- 172.21.15.47:0/3756828498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989e580 0x55f7698a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.642+0000 7f875e966740 1 --2- 172.21.15.47:0/3756828498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989dbc0 0x55f7698ae200 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.642+0000 7f875e966740 1 --2- 172.21.15.47:0/3756828498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989c9f0 0x55f7698a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.642+0000 7f875e966740 1 -- 172.21.15.47:0/3756828498 >> 172.21.15.47:0/3756828498 conn(0x55f769897de0 msgr2=0x55f76989b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.642+0000 7f875e966740 1 -- 172.21.15.47:0/3756828498 shutdown_connections 2024-03-21T11:27:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.642+0000 7f875e966740 1 -- 172.21.15.47:0/3756828498 wait complete. 2024-03-21T11:27:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.643+0000 7f875e966740 1 -- start start 2024-03-21T11:27:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.643+0000 7f875e966740 1 Processor -- start 2024-03-21T11:27:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.643+0000 7f875e966740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989c9f0 0x55f7698a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.643+0000 7f875e966740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989dbc0 0x55f7698aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.643+0000 7f875e966740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 0x55f7698ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.643+0000 7f875e966740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87541b16f0 con 0x55f76989c9f0 2024-03-21T11:27:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.643+0000 7f875e966740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87541b1090 con 0x55f76989dbc0 2024-03-21T11:27:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.643+0000 7f875e966740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87541ae590 con 0x55f76989e580 2024-03-21T11:27:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.643+0000 7f875e966740 10 client.0 _ll_register_callbacks cb 0x7f8748004280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.643+0000 7f8753e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989dbc0 0x55f7698aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.644+0000 7f8759c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 0x55f7698ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.644+0000 7f8753e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989dbc0 0x55f7698aa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57844/0 (socket says 172.21.15.47:57844) 2024-03-21T11:27:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.644+0000 7f8753e00640 1 -- 172.21.15.47:0/2949645006 learned_addr learned my addr 172.21.15.47:0/2949645006 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.644+0000 7f8759200640 1 --2- 172.21.15.47:0/2949645006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989c9f0 0x55f7698a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.644+0000 7f8753e00640 1 -- 172.21.15.47:0/2949645006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 msgr2=0x55f7698ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.644+0000 7f8753e00640 1 --2- 172.21.15.47:0/2949645006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 0x55f7698ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.644+0000 7f8753e00640 1 -- 172.21.15.47:0/2949645006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989c9f0 msgr2=0x55f7698a9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.644+0000 7f8753e00640 1 --2- 172.21.15.47:0/2949645006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989c9f0 0x55f7698a9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.644+0000 7f8753e00640 1 -- 172.21.15.47:0/2949645006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7698ad6d0 con 0x55f76989dbc0 2024-03-21T11:27:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.644+0000 7f8759200640 1 --2- 172.21.15.47:0/2949645006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989c9f0 0x55f7698a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.644+0000 7f8759c00640 1 --2- 172.21.15.47:0/2949645006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 0x55f7698ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.644+0000 7f8753e00640 1 --2- 172.21.15.47:0/2949645006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989dbc0 0x55f7698aa0c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8740005d00 tx=0x7f8740029330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5712116aea2429d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:44.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.644+0000 7f873aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:44.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.645+0000 7f873aa00640 1 -- 172.21.15.47:0/2949645006 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740012050 con 0x55f76989dbc0 2024-03-21T11:27:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.645+0000 7f873aa00640 1 -- 172.21.15.47:0/2949645006 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f87400121f0 con 0x55f76989dbc0 2024-03-21T11:27:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.645+0000 7f875e966740 1 -- 172.21.15.47:0/2949645006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f874002e070 con 0x55f76989dbc0 2024-03-21T11:27:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.645+0000 7f873aa00640 1 -- 172.21.15.47:0/2949645006 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8740010ab0 con 0x55f76989dbc0 2024-03-21T11:27:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.645+0000 7f875e966740 10 client.26816 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.645+0000 7f875e966740 1 -- 172.21.15.47:0/2949645006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f76990e2f0 con 0x55f76989dbc0 2024-03-21T11:27:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.646+0000 7f873aa00640 1 -- 172.21.15.47:0/2949645006 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f874002e070 con 0x55f76989dbc0 2024-03-21T11:27:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.646+0000 7f873aa00640 1 -- 172.21.15.47:0/2949645006 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8740004930 con 0x55f76989dbc0 2024-03-21T11:27:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.646+0000 7f875e966740 10 client.26816 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.646+0000 7f875e966740 10 client.26816 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:27:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.646+0000 7f875e966740 20 client.26816 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.646+0000 7f875e966740 1 --2- 172.21.15.47:0/2949645006 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f76990e9d0 0x55f769910df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.646+0000 7f875e966740 4 client.26816 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.646+0000 7f875e966740 1 -- 172.21.15.47:0/2949645006 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f769808b50 con 0x55f76990e9d0 2024-03-21T11:27:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.646+0000 7f8759200640 1 --2- 172.21.15.47:0/2949645006 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f76990e9d0 0x55f769910df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.647+0000 7f8759200640 1 --2- 172.21.15.47:0/2949645006 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f76990e9d0 0x55f769910df0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d0e1165d8f85b646 server_cookie=efd7170ce4fc18e9 in_seq=0 out_seq=0 2024-03-21T11:27:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.647+0000 7f873aa00640 10 client.26816 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:27:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.647+0000 7f873aa00640 1 -- 172.21.15.47:0/2949645006 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f769808b50 con 0x55f76990e9d0 2024-03-21T11:27:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.647+0000 7f873aa00640 10 client.26816 handle_command_reply: tid=0 2024-03-21T11:27:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.647+0000 7f875e966740 1 client.26816 shutdown 2024-03-21T11:27:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.647+0000 7f875e966740 1 -- 172.21.15.47:0/2949645006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989dbc0 msgr2=0x55f7698aa0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.647+0000 7f875e966740 1 --2- 172.21.15.47:0/2949645006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989dbc0 0x55f7698aa0c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8740005d00 tx=0x7f8740029330 comp rx=0 tx=0).stop 2024-03-21T11:27:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.647+0000 7f875e966740 1 -- 172.21.15.47:0/2949645006 shutdown_connections 2024-03-21T11:27:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.647+0000 7f875e966740 1 --2- 172.21.15.47:0/2949645006 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f76990e9d0 0x55f769910df0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.647+0000 7f875e966740 1 --2- 172.21.15.47:0/2949645006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f76989e580 0x55f7698ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.647+0000 7f875e966740 1 --2- 172.21.15.47:0/2949645006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f76989dbc0 0x55f7698aa0c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.647+0000 7f875e966740 1 --2- 172.21.15.47:0/2949645006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f76989c9f0 0x55f7698a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.647+0000 7f875e966740 1 -- 172.21.15.47:0/2949645006 >> 172.21.15.47:0/2949645006 conn(0x55f769897de0 msgr2=0x55f76989b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:44.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.647+0000 7f873aa00640 0 client.26816 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:27:44.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.648+0000 7f875e966740 1 -- 172.21.15.47:0/2949645006 shutdown_connections 2024-03-21T11:27:44.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.648+0000 7f875e966740 1 -- 172.21.15.47:0/2949645006 wait complete. 2024-03-21T11:27:44.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.648+0000 7f875e966740 20 client.26816 trim_cache size 0 max 16384 2024-03-21T11:27:44.652 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:44.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.648+0000 7f875a600640 1 -- 172.21.15.47:0/1190666730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f873c0a2f20 msgr2=0x7f873c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.648+0000 7f875a600640 1 --2- 172.21.15.47:0/1190666730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f873c0a2f20 0x7f873c0a53e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8748005fd0 tx=0x7f874800b480 comp rx=0 tx=0).stop 2024-03-21T11:27:44.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.648+0000 7f875a600640 1 -- 172.21.15.47:0/1190666730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8754077c40 msgr2=0x7f87541a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.648+0000 7f875a600640 1 --2- 172.21.15.47:0/1190666730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8754077c40 0x7f87541a53e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f874c037040 tx=0x7f874c035d10 comp rx=0 tx=0).stop 2024-03-21T11:27:44.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.648+0000 7f875a600640 1 -- 172.21.15.47:0/1190666730 shutdown_connections 2024-03-21T11:27:44.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.648+0000 7f875a600640 1 --2- 172.21.15.47:0/1190666730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f873c0a2f20 0x7f873c0a53e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.648+0000 7f875a600640 1 --2- 172.21.15.47:0/1190666730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8754077c40 0x7f87541a53e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.648+0000 7f875a600640 1 --2- 172.21.15.47:0/1190666730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87540772a0 0x7f875419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.648+0000 7f875a600640 1 --2- 172.21.15.47:0/1190666730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8754076060 0x7f875419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.648+0000 7f875a600640 1 -- 172.21.15.47:0/1190666730 >> 172.21.15.47:0/1190666730 conn(0x7f87540fed10 msgr2=0x7f8754100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:44.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.649+0000 7f875a600640 1 -- 172.21.15.47:0/1190666730 shutdown_connections 2024-03-21T11:27:44.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.649+0000 7f875a600640 1 -- 172.21.15.47:0/1190666730 wait complete. 2024-03-21T11:27:44.668 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:44.668 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:27:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.883+0000 7f7b03200640 1 -- 172.21.15.47:0/797644846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7afc0740a0 msgr2=0x7f7afc074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.883+0000 7f7b03200640 1 --2- 172.21.15.47:0/797644846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7afc0740a0 0x7f7afc074500 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009860 tx=0x7f7ae402bc10 comp rx=0 tx=0).stop 2024-03-21T11:27:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.883+0000 7f7b03200640 1 -- 172.21.15.47:0/797644846 shutdown_connections 2024-03-21T11:27:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.883+0000 7f7b03200640 1 --2- 172.21.15.47:0/797644846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7afc109540 0x7f7afc10b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.883+0000 7f7b03200640 1 --2- 172.21.15.47:0/797644846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7afc0740a0 0x7f7afc074500 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.883+0000 7f7b03200640 1 --2- 172.21.15.47:0/797644846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7afc075680 0x7f7afc073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.883+0000 7f7b03200640 1 -- 172.21.15.47:0/797644846 >> 172.21.15.47:0/797644846 conn(0x7f7afc0fcc40 msgr2=0x7f7afc0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.883+0000 7f7b03200640 1 -- 172.21.15.47:0/797644846 shutdown_connections 2024-03-21T11:27:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.883+0000 7f7b03200640 1 -- 172.21.15.47:0/797644846 wait complete. 2024-03-21T11:27:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.885+0000 7f7b03200640 1 Processor -- start 2024-03-21T11:27:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.885+0000 7f7b03200640 1 -- start start 2024-03-21T11:27:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.885+0000 7f7b03200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7afc0740a0 0x7f7afc199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.885+0000 7f7b03200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7afc075680 0x7f7afc19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.885+0000 7f7b03200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7afc109540 0x7f7afc1a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.885+0000 7f7b03200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc10f530 con 0x7f7afc075680 2024-03-21T11:27:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.885+0000 7f7b03200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc10f3b0 con 0x7f7afc109540 2024-03-21T11:27:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.885+0000 7f7b03200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc10f6b0 con 0x7f7afc0740a0 2024-03-21T11:27:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.885+0000 7f7afbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7afc0740a0 0x7f7afc199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.885+0000 7f7b01200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7afc109540 0x7f7afc1a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.886+0000 7f7afbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7afc0740a0 0x7f7afc199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40252/0 (socket says 172.21.15.47:40252) 2024-03-21T11:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.886+0000 7f7b01200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7afc109540 0x7f7afc1a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57862/0 (socket says 172.21.15.47:57862) 2024-03-21T11:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.886+0000 7f7afbe00640 1 -- 172.21.15.47:0/2225154851 learned_addr learned my addr 172.21.15.47:0/2225154851 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.886+0000 7f7af3400640 1 --2- 172.21.15.47:0/2225154851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7afc075680 0x7f7afc19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.886+0000 7f7afbe00640 1 -- 172.21.15.47:0/2225154851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7afc109540 msgr2=0x7f7afc1a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.886+0000 7f7afbe00640 1 --2- 172.21.15.47:0/2225154851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7afc109540 0x7f7afc1a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.886+0000 7f7afbe00640 1 -- 172.21.15.47:0/2225154851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7afc075680 msgr2=0x7f7afc19a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.886+0000 7f7afbe00640 1 --2- 172.21.15.47:0/2225154851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7afc075680 0x7f7afc19a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.886+0000 7f7afbe00640 1 -- 172.21.15.47:0/2225154851 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc1a17f0 con 0x7f7afc0740a0 2024-03-21T11:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.886+0000 7f7b01200640 1 --2- 172.21.15.47:0/2225154851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7afc109540 0x7f7afc1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.886+0000 7f7afbe00640 1 --2- 172.21.15.47:0/2225154851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7afc0740a0 0x7f7afc199bd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00c420 tx=0x7f7aec009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.887+0000 7f7af9600640 1 -- 172.21.15.47:0/2225154851 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec003970 con 0x7f7afc0740a0 2024-03-21T11:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.887+0000 7f7af9600640 1 -- 172.21.15.47:0/2225154851 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aec004020 con 0x7f7afc0740a0 2024-03-21T11:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.887+0000 7f7af9600640 1 -- 172.21.15.47:0/2225154851 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec00bad0 con 0x7f7afc0740a0 2024-03-21T11:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.887+0000 7f7b03200640 1 -- 172.21.15.47:0/2225154851 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc1a1ae0 con 0x7f7afc0740a0 2024-03-21T11:27:44.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.887+0000 7f7b03200640 1 -- 172.21.15.47:0/2225154851 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc1a1fb0 con 0x7f7afc0740a0 2024-03-21T11:27:44.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.890+0000 7f7af9600640 1 -- 172.21.15.47:0/2225154851 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7aec01d020 con 0x7f7afc0740a0 2024-03-21T11:27:44.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.891+0000 7f7b05fde740 1 Processor -- start 2024-03-21T11:27:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.891+0000 7f7b05fde740 1 -- start start 2024-03-21T11:27:44.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.891+0000 7f7b05fde740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.891+0000 7f7b05fde740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5b9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.891+0000 7f7b05fde740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.891+0000 7f7b05fde740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbc51d2c0 con 0x55ccbc5b8810 2024-03-21T11:27:44.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.891+0000 7f7b05fde740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbc4c5c90 con 0x55ccbc5ba3a0 2024-03-21T11:27:44.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.891+0000 7f7b05fde740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbc519360 con 0x55ccbc5b99e0 2024-03-21T11:27:44.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.892+0000 7f7af3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.892+0000 7f7afbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5b9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.892+0000 7f7b01200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.892+0000 7f7af3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57870/0 (socket says 172.21.15.47:57870) 2024-03-21T11:27:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.892+0000 7f7af3400640 1 -- 172.21.15.47:0/975575625 learned_addr learned my addr 172.21.15.47:0/975575625 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.892+0000 7f7afbe00640 1 -- 172.21.15.47:0/975575625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5ba3a0 msgr2=0x55ccbc5c0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.892+0000 7f7afbe00640 1 --2- 172.21.15.47:0/975575625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.892+0000 7f7afbe00640 1 -- 172.21.15.47:0/975575625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 msgr2=0x55ccbc5b8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.892+0000 7f7afbe00640 1 --2- 172.21.15.47:0/975575625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5b8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.892+0000 7f7afbe00640 1 -- 172.21.15.47:0/975575625 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccbc4ea080 con 0x55ccbc5b99e0 2024-03-21T11:27:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.892+0000 7f7af3400640 1 --2- 172.21.15.47:0/975575625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.892+0000 7f7afbe00640 1 --2- 172.21.15.47:0/975575625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5b9e60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0ccc80 tx=0x7f7aec0d5760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=594ca9830ae86b10 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.893+0000 7f7af9600640 1 --2- 172.21.15.47:0/2225154851 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ae00a2fa0 0x7f7ae00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.893+0000 7f7af1600640 1 -- 172.21.15.47:0/975575625 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec0e3070 con 0x55ccbc5b99e0 2024-03-21T11:27:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.893+0000 7f7af1600640 1 -- 172.21.15.47:0/975575625 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aec0703e0 con 0x55ccbc5b99e0 2024-03-21T11:27:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.893+0000 7f7af1600640 1 -- 172.21.15.47:0/975575625 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec09b270 con 0x55ccbc5b99e0 2024-03-21T11:27:44.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.893+0000 7f7af9600640 1 -- 172.21.15.47:0/2225154851 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7aec0cf5d0 con 0x7f7afc0740a0 2024-03-21T11:27:44.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.894+0000 7f7b01200640 1 --2- 172.21.15.47:0/2225154851 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ae00a2fa0 0x7f7ae00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.894+0000 7f7b05fde740 1 -- 172.21.15.47:0/975575625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b99e0 msgr2=0x55ccbc5b9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.894+0000 7f7b05fde740 1 --2- 172.21.15.47:0/975575625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5b9e60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0ccc80 tx=0x7f7aec0d5760 comp rx=0 tx=0).stop 2024-03-21T11:27:44.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.895+0000 7f7b05fde740 1 -- 172.21.15.47:0/975575625 shutdown_connections 2024-03-21T11:27:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.895+0000 7f7b05fde740 1 --2- 172.21.15.47:0/975575625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.895+0000 7f7b05fde740 1 --2- 172.21.15.47:0/975575625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5b9e60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.895+0000 7f7b05fde740 1 --2- 172.21.15.47:0/975575625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5b8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.895+0000 7f7b05fde740 1 -- 172.21.15.47:0/975575625 >> 172.21.15.47:0/975575625 conn(0x55ccbc5b3c00 msgr2=0x55ccbc5b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.895+0000 7f7b01200640 1 --2- 172.21.15.47:0/2225154851 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ae00a2fa0 0x7f7ae00a5460 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55ccbc5b98e0 tx=0x7f7af4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.895+0000 7f7b05fde740 1 -- 172.21.15.47:0/975575625 shutdown_connections 2024-03-21T11:27:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.895+0000 7f7b05fde740 1 -- 172.21.15.47:0/975575625 wait complete. 2024-03-21T11:27:44.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.895+0000 7f7b05fde740 1 -- start start 2024-03-21T11:27:44.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.896+0000 7f7b05fde740 1 Processor -- start 2024-03-21T11:27:44.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.896+0000 7f7b05fde740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5c2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.896+0000 7f7b05fde740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.896+0000 7f7b05fde740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.896+0000 7f7b05fde740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbc5c1230 con 0x55ccbc5b8810 2024-03-21T11:27:44.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.896+0000 7f7b05fde740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbc4d90d0 con 0x55ccbc5b99e0 2024-03-21T11:27:44.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.896+0000 7f7b05fde740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccbc5c13b0 con 0x55ccbc5ba3a0 2024-03-21T11:27:44.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.896+0000 7f7b05fde740 10 client.0 _ll_register_callbacks cb 0x55ccbc4dde80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:44.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.896+0000 7f7af3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.896+0000 7f7afbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.897+0000 7f7af3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57878/0 (socket says 172.21.15.47:57878) 2024-03-21T11:27:44.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.897+0000 7f7af3400640 1 -- 172.21.15.47:0/2024079090 learned_addr learned my addr 172.21.15.47:0/2024079090 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:44.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.897+0000 7f7b01200640 1 --2- 172.21.15.47:0/2024079090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.897+0000 7f7af3400640 1 -- 172.21.15.47:0/2024079090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5ba3a0 msgr2=0x55ccbc5c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.897+0000 7f7af3400640 1 --2- 172.21.15.47:0/2024079090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.897+0000 7f7af3400640 1 -- 172.21.15.47:0/2024079090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 msgr2=0x55ccbc5c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.897+0000 7f7af3400640 1 --2- 172.21.15.47:0/2024079090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5c2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.897+0000 7f7af3400640 1 -- 172.21.15.47:0/2024079090 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ccbc2313f0 con 0x55ccbc5b99e0 2024-03-21T11:27:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.897+0000 7f7b01200640 1 --2- 172.21.15.47:0/2024079090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.897+0000 7f7afbe00640 1 --2- 172.21.15.47:0/2024079090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.897+0000 7f7af3400640 1 --2- 172.21.15.47:0/2024079090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c2ea0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009780 tx=0x7f7ae4005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4e617a748a2c36 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.897+0000 7f7aea000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.898+0000 7f7aea000640 1 -- 172.21.15.47:0/2024079090 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4043070 con 0x55ccbc5b99e0 2024-03-21T11:27:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.898+0000 7f7aea000640 1 -- 172.21.15.47:0/2024079090 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7ae402c5b0 con 0x55ccbc5b99e0 2024-03-21T11:27:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.898+0000 7f7b05fde740 1 -- 172.21.15.47:0/2024079090 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7aec0703e0 con 0x55ccbc5b99e0 2024-03-21T11:27:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.898+0000 7f7aea000640 1 -- 172.21.15.47:0/2024079090 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae404f050 con 0x55ccbc5b99e0 2024-03-21T11:27:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.898+0000 7f7b05fde740 10 client.26828 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.898+0000 7f7b05fde740 1 -- 172.21.15.47:0/2024079090 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ccbc2fb7d0 con 0x55ccbc5b99e0 2024-03-21T11:27:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.898+0000 7f7aea000640 1 -- 172.21.15.47:0/2024079090 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aec0703e0 con 0x55ccbc5b99e0 2024-03-21T11:27:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.899+0000 7f7aea000640 1 -- 172.21.15.47:0/2024079090 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7ae4003ba0 con 0x55ccbc5b99e0 2024-03-21T11:27:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.899+0000 7f7b05fde740 10 client.26828 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.899+0000 7f7b05fde740 10 client.26828 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:27:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.899+0000 7f7b05fde740 20 client.26828 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.899+0000 7f7b05fde740 1 --2- 172.21.15.47:0/2024079090 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ccbc5ce2b0 0x55ccbc5d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.899+0000 7f7b05fde740 4 client.26828 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.899+0000 7f7b05fde740 1 -- 172.21.15.47:0/2024079090 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccbc52a800 con 0x55ccbc5ce2b0 2024-03-21T11:27:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.899+0000 7f7afbe00640 1 --2- 172.21.15.47:0/2024079090 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ccbc5ce2b0 0x55ccbc5d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.900+0000 7f7afbe00640 1 --2- 172.21.15.47:0/2024079090 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ccbc5ce2b0 0x55ccbc5d0700 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a4eb3f320d5990cf server_cookie=c312d4fe0672e402 in_seq=0 out_seq=0 2024-03-21T11:27:44.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.900+0000 7f7aea000640 10 client.26828 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:27:44.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.901+0000 7f7aea000640 1 -- 172.21.15.47:0/2024079090 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ccbc52a800 con 0x55ccbc5ce2b0 2024-03-21T11:27:44.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.901+0000 7f7aea000640 10 client.26828 handle_command_reply: tid=0 2024-03-21T11:27:44.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.901+0000 7f7b05fde740 1 client.26828 shutdown 2024-03-21T11:27:44.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.901+0000 7f7b05fde740 1 -- 172.21.15.47:0/2024079090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5b99e0 msgr2=0x55ccbc5c2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.901+0000 7f7b05fde740 1 --2- 172.21.15.47:0/2024079090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c2ea0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009780 tx=0x7f7ae4005a90 comp rx=0 tx=0).stop 2024-03-21T11:27:44.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.901+0000 7f7b05fde740 1 -- 172.21.15.47:0/2024079090 shutdown_connections 2024-03-21T11:27:44.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.901+0000 7f7b05fde740 1 --2- 172.21.15.47:0/2024079090 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ccbc5ce2b0 0x55ccbc5d0700 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.901+0000 7f7b05fde740 1 --2- 172.21.15.47:0/2024079090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.901+0000 7f7b05fde740 1 --2- 172.21.15.47:0/2024079090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c2ea0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.901+0000 7f7b05fde740 1 --2- 172.21.15.47:0/2024079090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.901+0000 7f7b05fde740 1 -- 172.21.15.47:0/2024079090 >> 172.21.15.47:0/2024079090 conn(0x55ccbc5b3c00 msgr2=0x55ccbc5b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:44.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.901+0000 7f7aea000640 0 client.26828 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:27:44.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.901+0000 7f7b05fde740 1 -- 172.21.15.47:0/2024079090 shutdown_connections 2024-03-21T11:27:44.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.901+0000 7f7b05fde740 1 -- 172.21.15.47:0/2024079090 wait complete. 2024-03-21T11:27:44.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.902+0000 7f7b05fde740 20 client.26828 trim_cache size 0 max 16384 2024-03-21T11:27:44.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.916+0000 7f7b05fde740 1 Processor -- start 2024-03-21T11:27:44.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.916+0000 7f7b05fde740 1 -- start start 2024-03-21T11:27:44.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.916+0000 7f7b05fde740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b8810 0x55ccbc5b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7b05fde740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7b05fde740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5bca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7b05fde740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1ac310 con 0x55ccbc5ba3a0 2024-03-21T11:27:44.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7b05fde740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1adc90 con 0x55ccbc5b99e0 2024-03-21T11:27:44.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7b05fde740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1ac970 con 0x55ccbc5b8810 2024-03-21T11:27:44.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7af3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7afbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b8810 0x55ccbc5b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7b01200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5bca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7af3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57888/0 (socket says 172.21.15.47:57888) 2024-03-21T11:27:44.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7af3400640 1 -- 172.21.15.47:0/1317695594 learned_addr learned my addr 172.21.15.47:0/1317695594 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:44.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7afbe00640 1 --2- 172.21.15.47:0/1317695594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b8810 0x55ccbc5b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40288/0 (socket says 172.21.15.47:40288) 2024-03-21T11:27:44.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7af3400640 1 -- 172.21.15.47:0/1317695594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b8810 msgr2=0x55ccbc5b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7af3400640 1 --2- 172.21.15.47:0/1317695594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b8810 0x55ccbc5b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7af3400640 1 -- 172.21.15.47:0/1317695594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5ba3a0 msgr2=0x55ccbc5bca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7af3400640 1 --2- 172.21.15.47:0/1317695594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5bca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.917+0000 7f7af3400640 1 -- 172.21.15.47:0/1317695594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccbc5b9460 con 0x55ccbc5b99e0 2024-03-21T11:27:44.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.918+0000 7f7af3400640 1 --2- 172.21.15.47:0/1317695594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c9e30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7ae402c150 tx=0x7f7ae4001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2655297da8d9e165 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:44.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.918+0000 7f7af0c00640 1 -- 172.21.15.47:0/1317695594 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4058070 con 0x55ccbc5b99e0 2024-03-21T11:27:44.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.918+0000 7f7af0c00640 1 -- 172.21.15.47:0/1317695594 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ae4009450 con 0x55ccbc5b99e0 2024-03-21T11:27:44.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.918+0000 7f7af0c00640 1 -- 172.21.15.47:0/1317695594 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4043070 con 0x55ccbc5b99e0 2024-03-21T11:27:44.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.919+0000 7f7b05fde740 1 -- 172.21.15.47:0/1317695594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5b99e0 msgr2=0x55ccbc5c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.919+0000 7f7b05fde740 1 --2- 172.21.15.47:0/1317695594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c9e30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7ae402c150 tx=0x7f7ae4001e70 comp rx=0 tx=0).stop 2024-03-21T11:27:44.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.919+0000 7f7b05fde740 1 -- 172.21.15.47:0/1317695594 shutdown_connections 2024-03-21T11:27:44.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.919+0000 7f7b05fde740 1 --2- 172.21.15.47:0/1317695594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5bca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.919+0000 7f7b05fde740 1 --2- 172.21.15.47:0/1317695594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c9e30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.919+0000 7f7b05fde740 1 --2- 172.21.15.47:0/1317695594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b8810 0x55ccbc5b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.919+0000 7f7b05fde740 1 -- 172.21.15.47:0/1317695594 >> 172.21.15.47:0/1317695594 conn(0x55ccbc5b3c00 msgr2=0x55ccbc5b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.919+0000 7f7b05fde740 1 -- 172.21.15.47:0/1317695594 shutdown_connections 2024-03-21T11:27:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.919+0000 7f7b05fde740 1 -- 172.21.15.47:0/1317695594 wait complete. 2024-03-21T11:27:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7b05fde740 1 -- start start 2024-03-21T11:27:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7b05fde740 1 Processor -- start 2024-03-21T11:27:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7b05fde740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7b05fde740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7b05fde740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7b05fde740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1ad630 con 0x55ccbc5b8810 2024-03-21T11:27:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7b05fde740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1acfd0 con 0x55ccbc5ba3a0 2024-03-21T11:27:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7b05fde740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1aa4d0 con 0x55ccbc5b99e0 2024-03-21T11:27:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7b05fde740 10 client.0 _ll_register_callbacks cb 0x7f7af4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7afbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7af3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7b01200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7afbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40698/0 (socket says 172.21.15.47:40698) 2024-03-21T11:27:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7afbe00640 1 -- 172.21.15.47:0/699464972 learned_addr learned my addr 172.21.15.47:0/699464972 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.920+0000 7f7af3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40300/0 (socket says 172.21.15.47:40300) 2024-03-21T11:27:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.921+0000 7f7b01200640 1 -- 172.21.15.47:0/699464972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b99e0 msgr2=0x55ccbc5c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.921+0000 7f7b01200640 1 --2- 172.21.15.47:0/699464972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.921+0000 7f7b01200640 1 -- 172.21.15.47:0/699464972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 msgr2=0x55ccbc5c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.921+0000 7f7b01200640 1 --2- 172.21.15.47:0/699464972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.921+0000 7f7b01200640 1 -- 172.21.15.47:0/699464972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ccbc5c94d0 con 0x55ccbc5ba3a0 2024-03-21T11:27:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.921+0000 7f7af3400640 1 --2- 172.21.15.47:0/699464972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:44.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.921+0000 7f7afbe00640 1 --2- 172.21.15.47:0/699464972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:44.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.921+0000 7f7b01200640 1 --2- 172.21.15.47:0/699464972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c8ee0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55ccbc5ca430 tx=0x7f7af40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78ead69b1fb6666 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:44.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.921+0000 7f7aea000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:44.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.921+0000 7f7aea000640 1 -- 172.21.15.47:0/699464972 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af402f060 con 0x55ccbc5ba3a0 2024-03-21T11:27:44.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.921+0000 7f7b05fde740 1 -- 172.21.15.47:0/699464972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ae4009450 con 0x55ccbc5ba3a0 2024-03-21T11:27:44.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.921+0000 7f7aea000640 1 -- 172.21.15.47:0/699464972 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7af400fd80 con 0x55ccbc5ba3a0 2024-03-21T11:27:44.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.922+0000 7f7aea000640 1 -- 172.21.15.47:0/699464972 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7af4036050 con 0x55ccbc5ba3a0 2024-03-21T11:27:44.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.922+0000 7f7b05fde740 10 client.26834 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:44.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.922+0000 7f7b05fde740 1 -- 172.21.15.47:0/699464972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ccbc629fc0 con 0x55ccbc5ba3a0 2024-03-21T11:27:44.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.922+0000 7f7aea000640 1 -- 172.21.15.47:0/699464972 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ae4009450 con 0x55ccbc5ba3a0 2024-03-21T11:27:44.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.922+0000 7f7aea000640 1 -- 172.21.15.47:0/699464972 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7af401e540 con 0x55ccbc5ba3a0 2024-03-21T11:27:44.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.922+0000 7f7b05fde740 10 client.26834 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:44.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.922+0000 7f7b05fde740 10 client.26834 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:27:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.922+0000 7f7b05fde740 20 client.26834 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.922+0000 7f7b05fde740 1 --2- 172.21.15.47:0/699464972 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ccbc62a940 0x55ccbc62cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.922+0000 7f7b05fde740 4 client.26834 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.922+0000 7f7b05fde740 1 -- 172.21.15.47:0/699464972 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ccbc524b50 con 0x55ccbc62a940 2024-03-21T11:27:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.923+0000 7f7afbe00640 1 --2- 172.21.15.47:0/699464972 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ccbc62a940 0x55ccbc62cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.923+0000 7f7afbe00640 1 --2- 172.21.15.47:0/699464972 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ccbc62a940 0x55ccbc62cdc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e75d718bb226989 server_cookie=3a9f9fa520d695c6 in_seq=0 out_seq=0 2024-03-21T11:27:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.923+0000 7f7aea000640 10 client.26834 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:27:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.923+0000 7f7aea000640 1 -- 172.21.15.47:0/699464972 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ccbc524b50 con 0x55ccbc62a940 2024-03-21T11:27:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.923+0000 7f7aea000640 10 client.26834 handle_command_reply: tid=0 2024-03-21T11:27:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.923+0000 7f7b05fde740 1 client.26834 shutdown 2024-03-21T11:27:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.923+0000 7f7b05fde740 1 -- 172.21.15.47:0/699464972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5ba3a0 msgr2=0x55ccbc5c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.923+0000 7f7b05fde740 1 --2- 172.21.15.47:0/699464972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c8ee0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55ccbc5ca430 tx=0x7f7af40046c0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.924+0000 7f7b05fde740 1 -- 172.21.15.47:0/699464972 shutdown_connections 2024-03-21T11:27:44.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.924+0000 7f7b05fde740 1 --2- 172.21.15.47:0/699464972 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ccbc62a940 0x55ccbc62cdc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.924+0000 7f7b05fde740 1 --2- 172.21.15.47:0/699464972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ccbc5ba3a0 0x55ccbc5c8ee0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.924+0000 7f7b05fde740 1 --2- 172.21.15.47:0/699464972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccbc5b99e0 0x55ccbc5c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.924+0000 7f7aea000640 0 client.26834 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:27:44.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.924+0000 7f7b05fde740 1 --2- 172.21.15.47:0/699464972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ccbc5b8810 0x55ccbc5c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.924+0000 7f7b05fde740 1 -- 172.21.15.47:0/699464972 >> 172.21.15.47:0/699464972 conn(0x55ccbc5b3c00 msgr2=0x55ccbc5b76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:44.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.924+0000 7f7b05fde740 1 -- 172.21.15.47:0/699464972 shutdown_connections 2024-03-21T11:27:44.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.924+0000 7f7b05fde740 1 -- 172.21.15.47:0/699464972 wait complete. 2024-03-21T11:27:44.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.924+0000 7f7b05fde740 20 client.26834 trim_cache size 0 max 16384 2024-03-21T11:27:44.932 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.925+0000 7f7b03200640 1 -- 172.21.15.47:0/2225154851 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ae00a2fa0 msgr2=0x7f7ae00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.925+0000 7f7b03200640 1 --2- 172.21.15.47:0/2225154851 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ae00a2fa0 0x7f7ae00a5460 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55ccbc5b98e0 tx=0x7f7af4005f90 comp rx=0 tx=0).stop 2024-03-21T11:27:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.925+0000 7f7b03200640 1 -- 172.21.15.47:0/2225154851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7afc0740a0 msgr2=0x7f7afc199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.925+0000 7f7b03200640 1 --2- 172.21.15.47:0/2225154851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7afc0740a0 0x7f7afc199bd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00c420 tx=0x7f7aec009f90 comp rx=0 tx=0).stop 2024-03-21T11:27:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.925+0000 7f7b03200640 1 -- 172.21.15.47:0/2225154851 shutdown_connections 2024-03-21T11:27:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.925+0000 7f7b03200640 1 --2- 172.21.15.47:0/2225154851 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ae00a2fa0 0x7f7ae00a5460 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.925+0000 7f7b03200640 1 --2- 172.21.15.47:0/2225154851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7afc109540 0x7f7afc1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.925+0000 7f7b03200640 1 --2- 172.21.15.47:0/2225154851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7afc075680 0x7f7afc19a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.925+0000 7f7b03200640 1 --2- 172.21.15.47:0/2225154851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7afc0740a0 0x7f7afc199bd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.925+0000 7f7b03200640 1 -- 172.21.15.47:0/2225154851 >> 172.21.15.47:0/2225154851 conn(0x7f7afc0fcc40 msgr2=0x7f7afc1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.925+0000 7f7b03200640 1 -- 172.21.15.47:0/2225154851 shutdown_connections 2024-03-21T11:27:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:44.925+0000 7f7b03200640 1 -- 172.21.15.47:0/2225154851 wait complete. 2024-03-21T11:27:44.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:44 smithi047 ceph-mon[31351]: pgmap v532: 129 pgs: 129 active+clean; 34 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 326 op/s 2024-03-21T11:27:44.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:44 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1319348225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:27:44.943 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:44.943 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:27:45.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:44 smithi100 ceph-mon[36245]: pgmap v532: 129 pgs: 129 active+clean; 34 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 326 op/s 2024-03-21T11:27:45.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:44 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1319348225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:27:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.163+0000 7fe9eb200640 1 -- 172.21.15.47:0/663826191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec0740a0 msgr2=0x7fe9ec074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.163+0000 7fe9eb200640 1 --2- 172.21.15.47:0/663826191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec0740a0 0x7fe9ec074500 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009900 tx=0x7fe9d802bcd0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.163+0000 7fe9eb200640 1 -- 172.21.15.47:0/663826191 shutdown_connections 2024-03-21T11:27:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.163+0000 7fe9eb200640 1 --2- 172.21.15.47:0/663826191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9ec109410 0x7fe9ec10b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.163+0000 7fe9eb200640 1 --2- 172.21.15.47:0/663826191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec0740a0 0x7fe9ec074500 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.163+0000 7fe9eb200640 1 --2- 172.21.15.47:0/663826191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ec075680 0x7fe9ec073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.163+0000 7fe9eb200640 1 -- 172.21.15.47:0/663826191 >> 172.21.15.47:0/663826191 conn(0x7fe9ec0fcb10 msgr2=0x7fe9ec0fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.163+0000 7fe9eb200640 1 -- 172.21.15.47:0/663826191 shutdown_connections 2024-03-21T11:27:45.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.163+0000 7fe9eb200640 1 -- 172.21.15.47:0/663826191 wait complete. 2024-03-21T11:27:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.164+0000 7fe9eb200640 1 Processor -- start 2024-03-21T11:27:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.164+0000 7fe9eb200640 1 -- start start 2024-03-21T11:27:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.164+0000 7fe9eb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9ec0740a0 0x7fe9ec199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.164+0000 7fe9eb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec075680 0x7fe9ec199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.164+0000 7fe9eb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ec109410 0x7fe9ec1a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.164+0000 7fe9eb200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec10f430 con 0x7fe9ec0740a0 2024-03-21T11:27:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.164+0000 7fe9eb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec10f2b0 con 0x7fe9ec075680 2024-03-21T11:27:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.164+0000 7fe9eb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec10f5b0 con 0x7fe9ec109410 2024-03-21T11:27:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.165+0000 7fe9ea800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ec109410 0x7fe9ec1a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.165+0000 7fe9ea800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ec109410 0x7fe9ec1a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40322/0 (socket says 172.21.15.47:40322) 2024-03-21T11:27:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.165+0000 7fe9e9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9ec0740a0 0x7fe9ec199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.165+0000 7fe9ea800640 1 -- 172.21.15.47:0/2381961977 learned_addr learned my addr 172.21.15.47:0/2381961977 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.165+0000 7fe9e9400640 1 --2- 172.21.15.47:0/2381961977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec075680 0x7fe9ec199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.165+0000 7fe9e9400640 1 -- 172.21.15.47:0/2381961977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ec109410 msgr2=0x7fe9ec1a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.165+0000 7fe9e9400640 1 --2- 172.21.15.47:0/2381961977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ec109410 0x7fe9ec1a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.165+0000 7fe9e9400640 1 -- 172.21.15.47:0/2381961977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9ec0740a0 msgr2=0x7fe9ec199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.165+0000 7fe9e9400640 1 --2- 172.21.15.47:0/2381961977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9ec0740a0 0x7fe9ec199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.165+0000 7fe9e9400640 1 -- 172.21.15.47:0/2381961977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec1a1680 con 0x7fe9ec075680 2024-03-21T11:27:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.165+0000 7fe9ea800640 1 --2- 172.21.15.47:0/2381961977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ec109410 0x7fe9ec1a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.165+0000 7fe9e9400640 1 --2- 172.21.15.47:0/2381961977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec075680 0x7fe9ec199fa0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8036040 tx=0x7fe9d8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.166+0000 7fe9e2a00640 1 -- 172.21.15.47:0/2381961977 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8043020 con 0x7fe9ec075680 2024-03-21T11:27:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.166+0000 7fe9e2a00640 1 -- 172.21.15.47:0/2381961977 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9d80054e0 con 0x7fe9ec075680 2024-03-21T11:27:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.166+0000 7fe9e2a00640 1 -- 172.21.15.47:0/2381961977 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8003c70 con 0x7fe9ec075680 2024-03-21T11:27:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.166+0000 7fe9eb200640 1 -- 172.21.15.47:0/2381961977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ec1a1910 con 0x7fe9ec075680 2024-03-21T11:27:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.166+0000 7fe9eb200640 1 -- 172.21.15.47:0/2381961977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ec1a1de0 con 0x7fe9ec075680 2024-03-21T11:27:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.170+0000 7fe9e2a00640 1 -- 172.21.15.47:0/2381961977 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe9d8046020 con 0x7fe9ec075680 2024-03-21T11:27:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.170+0000 7fe9f3e30740 1 Processor -- start 2024-03-21T11:27:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.170+0000 7fe9f3e30740 1 -- start start 2024-03-21T11:27:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.170+0000 7fe9f3e30740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da8ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.170+0000 7fe9f3e30740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8bbc0 0x55f83da8c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.170+0000 7fe9f3e30740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8c580 0x55f83da92e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.170+0000 7fe9f3e30740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83d9ef2c0 con 0x55f83da8c580 2024-03-21T11:27:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.170+0000 7fe9f3e30740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83d997c90 con 0x55f83da8a9f0 2024-03-21T11:27:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.170+0000 7fe9f3e30740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83d9eb360 con 0x55f83da8bbc0 2024-03-21T11:27:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.170+0000 7fe9ea800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da8ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.170+0000 7fe9ea800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da8ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57942/0 (socket says 172.21.15.47:57942) 2024-03-21T11:27:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.170+0000 7fe9ea800640 1 -- 172.21.15.47:0/4273797356 learned_addr learned my addr 172.21.15.47:0/4273797356 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.170+0000 7fe9e9e00640 1 --2- 172.21.15.47:0/4273797356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8bbc0 0x55f83da8c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.170+0000 7fe9e9400640 1 --2- 172.21.15.47:0/4273797356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8c580 0x55f83da92e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.171+0000 7fe9ea800640 1 -- 172.21.15.47:0/4273797356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8bbc0 msgr2=0x55f83da8c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.171+0000 7fe9ea800640 1 --2- 172.21.15.47:0/4273797356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8bbc0 0x55f83da8c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.171+0000 7fe9ea800640 1 -- 172.21.15.47:0/4273797356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8c580 msgr2=0x55f83da92e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.171+0000 7fe9ea800640 1 --2- 172.21.15.47:0/4273797356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8c580 0x55f83da92e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.171+0000 7fe9ea800640 1 -- 172.21.15.47:0/4273797356 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f83d9bc080 con 0x55f83da8a9f0 2024-03-21T11:27:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.171+0000 7fe9ea800640 1 --2- 172.21.15.47:0/4273797356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da8ae10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40117e0 tx=0x7fe9e4011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=487a3605fcfa00ad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.171+0000 7fe9d7400640 1 -- 172.21.15.47:0/4273797356 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e400fb40 con 0x55f83da8a9f0 2024-03-21T11:27:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.171+0000 7fe9d7400640 1 -- 172.21.15.47:0/4273797356 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9e400fce0 con 0x55f83da8a9f0 2024-03-21T11:27:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.171+0000 7fe9d7400640 1 -- 172.21.15.47:0/4273797356 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40106a0 con 0x55f83da8a9f0 2024-03-21T11:27:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.171+0000 7fe9e2a00640 1 --2- 172.21.15.47:0/2381961977 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9d00a2f20 0x7fe9d00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.172+0000 7fe9e2a00640 1 -- 172.21.15.47:0/2381961977 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe9d80f1860 con 0x7fe9ec075680 2024-03-21T11:27:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.172+0000 7fe9e9e00640 1 --2- 172.21.15.47:0/2381961977 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9d00a2f20 0x7fe9d00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.172+0000 7fe9f3e30740 1 -- 172.21.15.47:0/4273797356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 msgr2=0x55f83da8ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.172+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/4273797356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da8ae10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40117e0 tx=0x7fe9e4011ca0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.172+0000 7fe9f3e30740 1 -- 172.21.15.47:0/4273797356 shutdown_connections 2024-03-21T11:27:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.172+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/4273797356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8c580 0x55f83da92e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.172+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/4273797356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8bbc0 0x55f83da8c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.172+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/4273797356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da8ae10 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.172+0000 7fe9f3e30740 1 -- 172.21.15.47:0/4273797356 >> 172.21.15.47:0/4273797356 conn(0x55f83da85de0 msgr2=0x55f83da861e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.172+0000 7fe9f3e30740 1 -- 172.21.15.47:0/4273797356 shutdown_connections 2024-03-21T11:27:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.173+0000 7fe9f3e30740 1 -- 172.21.15.47:0/4273797356 wait complete. 2024-03-21T11:27:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.173+0000 7fe9e9e00640 1 --2- 172.21.15.47:0/2381961977 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9d00a2f20 0x7fe9d00a53e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009e20 tx=0x7fe9dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.173+0000 7fe9f3e30740 1 -- start start 2024-03-21T11:27:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.173+0000 7fe9f3e30740 1 Processor -- start 2024-03-21T11:27:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9f3e30740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da94c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9f3e30740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8bbc0 0x55f83da951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9f3e30740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 0x55f83da9c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9f3e30740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83da93590 con 0x55f83da8bbc0 2024-03-21T11:27:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9f3e30740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83d9ab0d0 con 0x55f83da8a9f0 2024-03-21T11:27:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9f3e30740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83da93710 con 0x55f83da8c580 2024-03-21T11:27:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9f3e30740 10 client.0 _ll_register_callbacks cb 0x55f83d9afe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da94c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da94c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57952/0 (socket says 172.21.15.47:57952) 2024-03-21T11:27:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9e9e00640 1 -- 172.21.15.47:0/1953877600 learned_addr learned my addr 172.21.15.47:0/1953877600 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9e9400640 1 --2- 172.21.15.47:0/1953877600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8bbc0 0x55f83da951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9ea800640 1 --2- 172.21.15.47:0/1953877600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 0x55f83da9c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9e9e00640 1 -- 172.21.15.47:0/1953877600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 msgr2=0x55f83da9c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9e9e00640 1 --2- 172.21.15.47:0/1953877600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 0x55f83da9c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9e9e00640 1 -- 172.21.15.47:0/1953877600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8bbc0 msgr2=0x55f83da951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9e9e00640 1 --2- 172.21.15.47:0/1953877600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8bbc0 0x55f83da951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.174+0000 7fe9e9e00640 1 -- 172.21.15.47:0/1953877600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f83d7033f0 con 0x55f83da8a9f0 2024-03-21T11:27:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.175+0000 7fe9e9e00640 1 --2- 172.21.15.47:0/1953877600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da94c90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc004550 tx=0x7fe9dc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=544d167dd0cc577e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.175+0000 7fe9cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.175+0000 7fe9cbe00640 1 -- 172.21.15.47:0/1953877600 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc02c060 con 0x55f83da8a9f0 2024-03-21T11:27:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.175+0000 7fe9cbe00640 1 -- 172.21.15.47:0/1953877600 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe9dc01d620 con 0x55f83da8a9f0 2024-03-21T11:27:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.175+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1953877600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9e400fce0 con 0x55f83da8a9f0 2024-03-21T11:27:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.175+0000 7fe9cbe00640 1 -- 172.21.15.47:0/1953877600 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9dc033050 con 0x55f83da8a9f0 2024-03-21T11:27:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.175+0000 7fe9f3e30740 10 client.26846 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.175+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1953877600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f83d7cd7d0 con 0x55f83da8a9f0 2024-03-21T11:27:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.175+0000 7fe9cbe00640 1 -- 172.21.15.47:0/1953877600 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9e400fce0 con 0x55f83da8a9f0 2024-03-21T11:27:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.176+0000 7fe9cbe00640 1 -- 172.21.15.47:0/1953877600 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe9dc026030 con 0x55f83da8a9f0 2024-03-21T11:27:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.176+0000 7fe9f3e30740 10 client.26846 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.176+0000 7fe9f3e30740 10 client.26846 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:27:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.176+0000 7fe9f3e30740 20 client.26846 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.176+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/1953877600 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f83daa0740 0x55f83daa2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.176+0000 7fe9f3e30740 4 client.26846 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.176+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1953877600 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f83d9fc800 con 0x55f83daa0740 2024-03-21T11:27:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.176+0000 7fe9e9400640 1 --2- 172.21.15.47:0/1953877600 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f83daa0740 0x55f83daa2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.177+0000 7fe9e9400640 1 --2- 172.21.15.47:0/1953877600 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f83daa0740 0x55f83daa2be0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52e86738e4015d1e server_cookie=a455fdbf81b95c8 in_seq=0 out_seq=0 2024-03-21T11:27:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.177+0000 7fe9cbe00640 10 client.26846 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:27:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.177+0000 7fe9cbe00640 1 -- 172.21.15.47:0/1953877600 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f83d9fc800 con 0x55f83daa0740 2024-03-21T11:27:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.177+0000 7fe9cbe00640 10 client.26846 handle_command_reply: tid=0 2024-03-21T11:27:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.177+0000 7fe9f3e30740 1 client.26846 shutdown 2024-03-21T11:27:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.178+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1953877600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 msgr2=0x55f83da94c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.178+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/1953877600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da94c90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc004550 tx=0x7fe9dc004730 comp rx=0 tx=0).stop 2024-03-21T11:27:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.178+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1953877600 shutdown_connections 2024-03-21T11:27:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.178+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/1953877600 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f83daa0740 0x55f83daa2be0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.178+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/1953877600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 0x55f83da9c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.178+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/1953877600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8bbc0 0x55f83da951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.178+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/1953877600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da94c90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.178+0000 7fe9cbe00640 0 client.26846 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:27:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.178+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1953877600 >> 172.21.15.47:0/1953877600 conn(0x55f83da85de0 msgr2=0x55f83da85160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.178+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1953877600 shutdown_connections 2024-03-21T11:27:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.178+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1953877600 wait complete. 2024-03-21T11:27:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.178+0000 7fe9f3e30740 20 client.26846 trim_cache size 0 max 16384 2024-03-21T11:27:45.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.192+0000 7fe9f3e30740 1 Processor -- start 2024-03-21T11:27:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.192+0000 7fe9f3e30740 1 -- start start 2024-03-21T11:27:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.192+0000 7fe9f3e30740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8a9f0 0x55f83da8ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.192+0000 7fe9f3e30740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8bbc0 0x55f83da9c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.192+0000 7fe9f3e30740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 0x55f83da8f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.192+0000 7fe9f3e30740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec1ac0e0 con 0x55f83da8a9f0 2024-03-21T11:27:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.192+0000 7fe9f3e30740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec1ada60 con 0x55f83da8bbc0 2024-03-21T11:27:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.192+0000 7fe9f3e30740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec1ac740 con 0x55f83da8c580 2024-03-21T11:27:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.192+0000 7fe9e9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8bbc0 0x55f83da9c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.192+0000 7fe9e9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8a9f0 0x55f83da8ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.192+0000 7fe9ea800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 0x55f83da8f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.192+0000 7fe9e9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8bbc0 0x55f83da9c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57968/0 (socket says 172.21.15.47:57968) 2024-03-21T11:27:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.192+0000 7fe9ea800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 0x55f83da8f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40360/0 (socket says 172.21.15.47:40360) 2024-03-21T11:27:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.192+0000 7fe9e9400640 1 -- 172.21.15.47:0/4288344426 learned_addr learned my addr 172.21.15.47:0/4288344426 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.193+0000 7fe9e9400640 1 -- 172.21.15.47:0/4288344426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 msgr2=0x55f83da8f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.193+0000 7fe9e9400640 1 --2- 172.21.15.47:0/4288344426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 0x55f83da8f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.193+0000 7fe9e9400640 1 -- 172.21.15.47:0/4288344426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8a9f0 msgr2=0x55f83da8ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.193+0000 7fe9e9400640 1 --2- 172.21.15.47:0/4288344426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8a9f0 0x55f83da8ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.193+0000 7fe9e9400640 1 -- 172.21.15.47:0/4288344426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f83da818b0 con 0x55f83da8bbc0 2024-03-21T11:27:45.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.193+0000 7fe9e9400640 1 --2- 172.21.15.47:0/4288344426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8bbc0 0x55f83da9c1d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8006010 tx=0x7fe9d80f3040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adce45404b12c59e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.193+0000 7fe9d6a00640 1 -- 172.21.15.47:0/4288344426 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8004c80 con 0x55f83da8bbc0 2024-03-21T11:27:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.193+0000 7fe9d6a00640 1 -- 172.21.15.47:0/4288344426 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9d8005690 con 0x55f83da8bbc0 2024-03-21T11:27:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.193+0000 7fe9d6a00640 1 -- 172.21.15.47:0/4288344426 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d80ef080 con 0x55f83da8bbc0 2024-03-21T11:27:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.194+0000 7fe9f3e30740 1 -- 172.21.15.47:0/4288344426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8bbc0 msgr2=0x55f83da9c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.194+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/4288344426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8bbc0 0x55f83da9c1d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8006010 tx=0x7fe9d80f3040 comp rx=0 tx=0).stop 2024-03-21T11:27:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.194+0000 7fe9f3e30740 1 -- 172.21.15.47:0/4288344426 shutdown_connections 2024-03-21T11:27:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.194+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/4288344426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 0x55f83da8f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.194+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/4288344426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8bbc0 0x55f83da9c1d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.194+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/4288344426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8a9f0 0x55f83da8ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.194+0000 7fe9f3e30740 1 -- 172.21.15.47:0/4288344426 >> 172.21.15.47:0/4288344426 conn(0x55f83da85de0 msgr2=0x55f83da898d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.194+0000 7fe9f3e30740 1 -- 172.21.15.47:0/4288344426 shutdown_connections 2024-03-21T11:27:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.194+0000 7fe9f3e30740 1 -- 172.21.15.47:0/4288344426 wait complete. 2024-03-21T11:27:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.195+0000 7fe9f3e30740 1 -- start start 2024-03-21T11:27:45.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.195+0000 7fe9f3e30740 1 Processor -- start 2024-03-21T11:27:45.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.195+0000 7fe9f3e30740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da982a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.195+0000 7fe9f3e30740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8bbc0 0x55f83da987e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.195+0000 7fe9f3e30740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 0x55f83da9b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.195+0000 7fe9f3e30740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec1ad400 con 0x55f83da8bbc0 2024-03-21T11:27:45.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.195+0000 7fe9f3e30740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec1acda0 con 0x55f83da8a9f0 2024-03-21T11:27:45.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.195+0000 7fe9f3e30740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec1aa2a0 con 0x55f83da8c580 2024-03-21T11:27:45.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.195+0000 7fe9f3e30740 10 client.0 _ll_register_callbacks cb 0x7fe9dc009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:45.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.196+0000 7fe9e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.196+0000 7fe9e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57976/0 (socket says 172.21.15.47:57976) 2024-03-21T11:27:45.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.196+0000 7fe9e9e00640 1 -- 172.21.15.47:0/1387622862 learned_addr learned my addr 172.21.15.47:0/1387622862 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.196+0000 7fe9ea800640 1 --2- 172.21.15.47:0/1387622862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 0x55f83da9b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.196+0000 7fe9e9400640 1 --2- 172.21.15.47:0/1387622862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8bbc0 0x55f83da987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.196+0000 7fe9e9e00640 1 -- 172.21.15.47:0/1387622862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 msgr2=0x55f83da9b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.196+0000 7fe9e9e00640 1 --2- 172.21.15.47:0/1387622862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 0x55f83da9b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.196+0000 7fe9e9e00640 1 -- 172.21.15.47:0/1387622862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8bbc0 msgr2=0x55f83da987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.196+0000 7fe9e9e00640 1 --2- 172.21.15.47:0/1387622862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8bbc0 0x55f83da987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.196+0000 7fe9e9e00640 1 -- 172.21.15.47:0/1387622862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f83da9bdf0 con 0x55f83da8a9f0 2024-03-21T11:27:45.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.196+0000 7fe9e9e00640 1 --2- 172.21.15.47:0/1387622862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da982a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc012d50 tx=0x7fe9dc01e6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2275c05bb7791398 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.197+0000 7fe9cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:45.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.197+0000 7fe9cbe00640 1 -- 172.21.15.47:0/1387622862 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc03b020 con 0x55f83da8a9f0 2024-03-21T11:27:45.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.197+0000 7fe9cbe00640 1 -- 172.21.15.47:0/1387622862 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe9dc005020 con 0x55f83da8a9f0 2024-03-21T11:27:45.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.197+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1387622862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f83d9f6b50 con 0x55f83da8a9f0 2024-03-21T11:27:45.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.197+0000 7fe9cbe00640 1 -- 172.21.15.47:0/1387622862 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9dc004340 con 0x55f83da8a9f0 2024-03-21T11:27:45.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.197+0000 7fe9f3e30740 10 client.26852 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:45.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.197+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1387622862 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f83dafcac0 con 0x55f83da8a9f0 2024-03-21T11:27:45.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.198+0000 7fe9cbe00640 1 -- 172.21.15.47:0/1387622862 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f83d9f6b50 con 0x55f83da8a9f0 2024-03-21T11:27:45.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.198+0000 7fe9cbe00640 1 -- 172.21.15.47:0/1387622862 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe9dc026040 con 0x55f83da8a9f0 2024-03-21T11:27:45.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.198+0000 7fe9f3e30740 10 client.26852 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:45.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.198+0000 7fe9f3e30740 10 client.26852 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:27:45.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.198+0000 7fe9f3e30740 20 client.26852 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:45.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.198+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/1387622862 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f83dafd350 0x55f83daff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.198+0000 7fe9f3e30740 4 client.26852 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:45.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.198+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1387622862 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f83d974500 con 0x55f83dafd350 2024-03-21T11:27:45.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.198+0000 7fe9e9400640 1 --2- 172.21.15.47:0/1387622862 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f83dafd350 0x55f83daff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9cbe00640 10 client.26852 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:27:45.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9e9400640 1 --2- 172.21.15.47:0/1387622862 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f83dafd350 0x55f83daff770 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1ae589af3bf5f31 server_cookie=4f98acd9229659f8 in_seq=0 out_seq=0 2024-03-21T11:27:45.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9cbe00640 1 -- 172.21.15.47:0/1387622862 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f83d974500 con 0x55f83dafd350 2024-03-21T11:27:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9cbe00640 10 client.26852 handle_command_reply: tid=0 2024-03-21T11:27:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9f3e30740 1 client.26852 shutdown 2024-03-21T11:27:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1387622862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 msgr2=0x55f83da982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/1387622862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da982a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc012d50 tx=0x7fe9dc01e6a0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1387622862 shutdown_connections 2024-03-21T11:27:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/1387622862 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f83dafd350 0x55f83daff770 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/1387622862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f83da8c580 0x55f83da9b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/1387622862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f83da8bbc0 0x55f83da987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9f3e30740 1 --2- 172.21.15.47:0/1387622862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83da8a9f0 0x55f83da982a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9cbe00640 0 client.26852 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:27:45.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1387622862 >> 172.21.15.47:0/1387622862 conn(0x55f83da85de0 msgr2=0x55f83da848d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1387622862 shutdown_connections 2024-03-21T11:27:45.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.199+0000 7fe9f3e30740 1 -- 172.21.15.47:0/1387622862 wait complete. 2024-03-21T11:27:45.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.200+0000 7fe9f3e30740 20 client.26852 trim_cache size 0 max 16384 2024-03-21T11:27:45.209 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:45.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.200+0000 7fe9eb200640 1 -- 172.21.15.47:0/2381961977 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9d00a2f20 msgr2=0x7fe9d00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.200+0000 7fe9eb200640 1 --2- 172.21.15.47:0/2381961977 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9d00a2f20 0x7fe9d00a53e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009e20 tx=0x7fe9dc005f90 comp rx=0 tx=0).stop 2024-03-21T11:27:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.200+0000 7fe9eb200640 1 -- 172.21.15.47:0/2381961977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec075680 msgr2=0x7fe9ec199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.200+0000 7fe9eb200640 1 --2- 172.21.15.47:0/2381961977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec075680 0x7fe9ec199fa0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8036040 tx=0x7fe9d8005d50 comp rx=0 tx=0).stop 2024-03-21T11:27:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.200+0000 7fe9eb200640 1 -- 172.21.15.47:0/2381961977 shutdown_connections 2024-03-21T11:27:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.200+0000 7fe9eb200640 1 --2- 172.21.15.47:0/2381961977 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9d00a2f20 0x7fe9d00a53e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.200+0000 7fe9eb200640 1 --2- 172.21.15.47:0/2381961977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9ec109410 0x7fe9ec1a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.200+0000 7fe9eb200640 1 --2- 172.21.15.47:0/2381961977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9ec075680 0x7fe9ec199fa0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.200+0000 7fe9eb200640 1 --2- 172.21.15.47:0/2381961977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9ec0740a0 0x7fe9ec199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.200+0000 7fe9eb200640 1 -- 172.21.15.47:0/2381961977 >> 172.21.15.47:0/2381961977 conn(0x7fe9ec0fcb10 msgr2=0x7fe9ec1053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.201+0000 7fe9eb200640 1 -- 172.21.15.47:0/2381961977 shutdown_connections 2024-03-21T11:27:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.201+0000 7fe9eb200640 1 -- 172.21.15.47:0/2381961977 wait complete. 2024-03-21T11:27:45.218 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:45.219 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:27:45.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:44 smithi102 ceph-mon[36748]: pgmap v532: 129 pgs: 129 active+clean; 34 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 326 op/s 2024-03-21T11:27:45.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:44 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1319348225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:27:45.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.450+0000 7efe01c00640 1 -- 172.21.15.47:0/3721303306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdfc109870 msgr2=0x7efdfc110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.450+0000 7efe01c00640 1 --2- 172.21.15.47:0/3721303306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdfc109870 0x7efdfc110100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009900 tx=0x7efdf402bc30 comp rx=0 tx=0).stop 2024-03-21T11:27:45.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.450+0000 7efe01c00640 1 -- 172.21.15.47:0/3721303306 shutdown_connections 2024-03-21T11:27:45.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.450+0000 7efe01c00640 1 --2- 172.21.15.47:0/3721303306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdfc109870 0x7efdfc110100 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.450+0000 7efe01c00640 1 --2- 172.21.15.47:0/3721303306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdfc108eb0 0x7efdfc109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.450+0000 7efe01c00640 1 --2- 172.21.15.47:0/3721303306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdfc107cc0 0x7efdfc1080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.450+0000 7efe01c00640 1 -- 172.21.15.47:0/3721303306 >> 172.21.15.47:0/3721303306 conn(0x7efdfc076c60 msgr2=0x7efdfc0790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.450+0000 7efe01c00640 1 -- 172.21.15.47:0/3721303306 shutdown_connections 2024-03-21T11:27:45.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.450+0000 7efe01c00640 1 -- 172.21.15.47:0/3721303306 wait complete. 2024-03-21T11:27:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.451+0000 7efe01c00640 1 Processor -- start 2024-03-21T11:27:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.451+0000 7efe01c00640 1 -- start start 2024-03-21T11:27:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.451+0000 7efe01c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdfc107cc0 0x7efdfc19df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efe01c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdfc108eb0 0x7efdfc19e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efe01c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdfc109870 0x7efdfc1a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efe01c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc113990 con 0x7efdfc107cc0 2024-03-21T11:27:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efe01c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc113810 con 0x7efdfc108eb0 2024-03-21T11:27:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efe01c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc113b10 con 0x7efdfc109870 2024-03-21T11:27:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efdfb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdfc109870 0x7efdfc1a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efdfb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdfc109870 0x7efdfc1a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40384/0 (socket says 172.21.15.47:40384) 2024-03-21T11:27:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efdfb400640 1 -- 172.21.15.47:0/2585385163 learned_addr learned my addr 172.21.15.47:0/2585385163 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efdfaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdfc107cc0 0x7efdfc19df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efdf2000640 1 --2- 172.21.15.47:0/2585385163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdfc108eb0 0x7efdfc19e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efdfb400640 1 -- 172.21.15.47:0/2585385163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdfc108eb0 msgr2=0x7efdfc19e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efdfb400640 1 --2- 172.21.15.47:0/2585385163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdfc108eb0 0x7efdfc19e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efdfb400640 1 -- 172.21.15.47:0/2585385163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdfc107cc0 msgr2=0x7efdfc19df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efdfb400640 1 --2- 172.21.15.47:0/2585385163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdfc107cc0 0x7efdfc19df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.452+0000 7efdfb400640 1 -- 172.21.15.47:0/2585385163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc1a5b70 con 0x7efdfc109870 2024-03-21T11:27:45.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.453+0000 7efdfb400640 1 --2- 172.21.15.47:0/2585385163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdfc109870 0x7efdfc1a5500 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efdf402c8e0 tx=0x7efdf402c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.453+0000 7efdf3e00640 1 -- 172.21.15.47:0/2585385163 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4043020 con 0x7efdfc109870 2024-03-21T11:27:45.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.453+0000 7efdf3e00640 1 -- 172.21.15.47:0/2585385163 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efdf40054e0 con 0x7efdfc109870 2024-03-21T11:27:45.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.453+0000 7efdf3e00640 1 -- 172.21.15.47:0/2585385163 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4003d00 con 0x7efdfc109870 2024-03-21T11:27:45.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.453+0000 7efe01c00640 1 -- 172.21.15.47:0/2585385163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc1a5e00 con 0x7efdfc109870 2024-03-21T11:27:45.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.453+0000 7efe01c00640 1 -- 172.21.15.47:0/2585385163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc1a6270 con 0x7efdfc109870 2024-03-21T11:27:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.457+0000 7efdf3e00640 1 -- 172.21.15.47:0/2585385163 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7efdf4046020 con 0x7efdfc109870 2024-03-21T11:27:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.457+0000 7efe04802740 1 Processor -- start 2024-03-21T11:27:45.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.457+0000 7efe04802740 1 -- start start 2024-03-21T11:27:45.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.458+0000 7efe04802740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe96810 0x55e3dfe96c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.458+0000 7efe04802740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe979e0 0x55e3dfe97e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.458+0000 7efe04802740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe983a0 0x55e3dfe9ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.458+0000 7efe04802740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dfdfb2c0 con 0x55e3dfe96810 2024-03-21T11:27:45.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.458+0000 7efe04802740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dfda3c90 con 0x55e3dfe983a0 2024-03-21T11:27:45.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.458+0000 7efe04802740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dfdf7360 con 0x55e3dfe979e0 2024-03-21T11:27:45.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.458+0000 7efdfaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe979e0 0x55e3dfe97e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.458+0000 7efdf2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe96810 0x55e3dfe96c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.458+0000 7efdfaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe979e0 0x55e3dfe97e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40394/0 (socket says 172.21.15.47:40394) 2024-03-21T11:27:45.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.458+0000 7efdfaa00640 1 -- 172.21.15.47:0/433764859 learned_addr learned my addr 172.21.15.47:0/433764859 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.458+0000 7efdf2000640 1 --2- 172.21.15.47:0/433764859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe983a0 0x55e3dfe9ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.459+0000 7efdfaa00640 1 -- 172.21.15.47:0/433764859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe983a0 msgr2=0x55e3dfe9ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.459+0000 7efdfaa00640 1 --2- 172.21.15.47:0/433764859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe983a0 0x55e3dfe9ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.459+0000 7efdfaa00640 1 -- 172.21.15.47:0/433764859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe96810 msgr2=0x55e3dfe96c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.459+0000 7efdfaa00640 1 --2- 172.21.15.47:0/433764859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe96810 0x55e3dfe96c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.459+0000 7efdfaa00640 1 -- 172.21.15.47:0/433764859 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3dfdc8080 con 0x55e3dfe979e0 2024-03-21T11:27:45.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.459+0000 7efdf2000640 1 --2- 172.21.15.47:0/433764859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe96810 0x55e3dfe96c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:45.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.459+0000 7efdf3e00640 1 --2- 172.21.15.47:0/2585385163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efde00a2fa0 0x7efde00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.459+0000 7efdf3e00640 1 -- 172.21.15.47:0/2585385163 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efdf40f1760 con 0x7efdfc109870 2024-03-21T11:27:45.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.459+0000 7efdfaa00640 1 --2- 172.21.15.47:0/433764859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe979e0 0x55e3dfe97e60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efdec009ca0 tx=0x7efdec011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e16c0ffff57017b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.459+0000 7efdf2000640 1 --2- 172.21.15.47:0/2585385163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efde00a2fa0 0x7efde00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.459+0000 7efdebe00640 1 -- 172.21.15.47:0/433764859 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec004220 con 0x55e3dfe979e0 2024-03-21T11:27:45.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.459+0000 7efdebe00640 1 -- 172.21.15.47:0/433764859 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efdec0043c0 con 0x55e3dfe979e0 2024-03-21T11:27:45.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.459+0000 7efdebe00640 1 -- 172.21.15.47:0/433764859 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec019830 con 0x55e3dfe979e0 2024-03-21T11:27:45.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.460+0000 7efe04802740 1 -- 172.21.15.47:0/433764859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe979e0 msgr2=0x55e3dfe97e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.460+0000 7efdf2000640 1 --2- 172.21.15.47:0/2585385163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efde00a2fa0 0x7efde00a5460 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55e3dfe978e0 tx=0x7efde4009960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.460+0000 7efe04802740 1 --2- 172.21.15.47:0/433764859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe979e0 0x55e3dfe97e60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efdec009ca0 tx=0x7efdec011810 comp rx=0 tx=0).stop 2024-03-21T11:27:45.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.460+0000 7efe04802740 1 -- 172.21.15.47:0/433764859 shutdown_connections 2024-03-21T11:27:45.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.460+0000 7efe04802740 1 --2- 172.21.15.47:0/433764859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe983a0 0x55e3dfe9ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.460+0000 7efe04802740 1 --2- 172.21.15.47:0/433764859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe979e0 0x55e3dfe97e60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.460+0000 7efe04802740 1 --2- 172.21.15.47:0/433764859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe96810 0x55e3dfe96c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.460+0000 7efe04802740 1 -- 172.21.15.47:0/433764859 >> 172.21.15.47:0/433764859 conn(0x55e3dfe91c00 msgr2=0x55e3dfe92000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.461+0000 7efe04802740 1 -- 172.21.15.47:0/433764859 shutdown_connections 2024-03-21T11:27:45.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.461+0000 7efe04802740 1 -- 172.21.15.47:0/433764859 wait complete. 2024-03-21T11:27:45.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.461+0000 7efe04802740 1 -- start start 2024-03-21T11:27:45.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.461+0000 7efe04802740 1 Processor -- start 2024-03-21T11:27:45.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efe04802740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe96810 0x55e3dfea0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efe04802740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe979e0 0x55e3dfea0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efe04802740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe983a0 0x55e3dfea7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efe04802740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dfe9f3b0 con 0x55e3dfe979e0 2024-03-21T11:27:45.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efe04802740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dfdb70d0 con 0x55e3dfe96810 2024-03-21T11:27:45.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efe04802740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3dfe9f530 con 0x55e3dfe983a0 2024-03-21T11:27:45.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efe04802740 10 client.0 _ll_register_callbacks cb 0x55e3dfdbbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:45.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efdfaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe96810 0x55e3dfea0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efdfb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe983a0 0x55e3dfea7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efdf2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe979e0 0x55e3dfea0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efdfaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe96810 0x55e3dfea0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58020/0 (socket says 172.21.15.47:58020) 2024-03-21T11:27:45.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efdfaa00640 1 -- 172.21.15.47:0/1138955564 learned_addr learned my addr 172.21.15.47:0/1138955564 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efdfb400640 1 -- 172.21.15.47:0/1138955564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe96810 msgr2=0x55e3dfea0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efdfb400640 1 --2- 172.21.15.47:0/1138955564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe96810 0x55e3dfea0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.462+0000 7efdfb400640 1 -- 172.21.15.47:0/1138955564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe979e0 msgr2=0x55e3dfea0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.463+0000 7efdfb400640 1 --2- 172.21.15.47:0/1138955564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe979e0 0x55e3dfea0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.463+0000 7efdfb400640 1 -- 172.21.15.47:0/1138955564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3dfb0f3f0 con 0x55e3dfe983a0 2024-03-21T11:27:45.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.463+0000 7efdfaa00640 1 --2- 172.21.15.47:0/1138955564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe96810 0x55e3dfea0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:45.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.463+0000 7efdf2000640 1 --2- 172.21.15.47:0/1138955564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe979e0 0x55e3dfea0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:45.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.463+0000 7efdfb400640 1 --2- 172.21.15.47:0/1138955564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe983a0 0x55e3dfea7f60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efdf40eee70 tx=0x7efdf40bd790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=872edd4b0ecbaee3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.463+0000 7efde8c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:45.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.463+0000 7efde8c00640 1 -- 172.21.15.47:0/1138955564 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4005690 con 0x55e3dfe983a0 2024-03-21T11:27:45.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.463+0000 7efde8c00640 1 -- 172.21.15.47:0/1138955564 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efdf40f1450 con 0x55e3dfe983a0 2024-03-21T11:27:45.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.463+0000 7efe04802740 1 -- 172.21.15.47:0/1138955564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdec0043c0 con 0x55e3dfe983a0 2024-03-21T11:27:45.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.463+0000 7efde8c00640 1 -- 172.21.15.47:0/1138955564 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdf40be350 con 0x55e3dfe983a0 2024-03-21T11:27:45.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.464+0000 7efe04802740 10 client.26884 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:45.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.464+0000 7efe04802740 1 -- 172.21.15.47:0/1138955564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e3dfbd97d0 con 0x55e3dfe983a0 2024-03-21T11:27:45.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.464+0000 7efde8c00640 1 -- 172.21.15.47:0/1138955564 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efdec0043c0 con 0x55e3dfe983a0 2024-03-21T11:27:45.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.464+0000 7efde8c00640 1 -- 172.21.15.47:0/1138955564 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efdf40c40e0 con 0x55e3dfe983a0 2024-03-21T11:27:45.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.464+0000 7efe04802740 10 client.26884 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:45.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.464+0000 7efe04802740 10 client.26884 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:27:45.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.464+0000 7efe04802740 20 client.26884 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:45.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.464+0000 7efe04802740 1 --2- 172.21.15.47:0/1138955564 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e3dfeac4a0 0x55e3dfeae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.464+0000 7efe04802740 4 client.26884 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:45.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.464+0000 7efe04802740 1 -- 172.21.15.47:0/1138955564 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3dfe08800 con 0x55e3dfeac4a0 2024-03-21T11:27:45.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.465+0000 7efdfaa00640 1 --2- 172.21.15.47:0/1138955564 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e3dfeac4a0 0x55e3dfeae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.466+0000 7efde8c00640 10 client.26884 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:27:45.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.466+0000 7efdfaa00640 1 --2- 172.21.15.47:0/1138955564 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e3dfeac4a0 0x55e3dfeae8d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d4475271dbd8401 server_cookie=d8bbbc8c37c3d4eb in_seq=0 out_seq=0 2024-03-21T11:27:45.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.466+0000 7efde8c00640 1 -- 172.21.15.47:0/1138955564 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e3dfe08800 con 0x55e3dfeac4a0 2024-03-21T11:27:45.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.466+0000 7efde8c00640 10 client.26884 handle_command_reply: tid=0 2024-03-21T11:27:45.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.466+0000 7efe04802740 1 client.26884 shutdown 2024-03-21T11:27:45.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.467+0000 7efe04802740 1 -- 172.21.15.47:0/1138955564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe983a0 msgr2=0x55e3dfea7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.467+0000 7efe04802740 1 --2- 172.21.15.47:0/1138955564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe983a0 0x55e3dfea7f60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efdf40eee70 tx=0x7efdf40bd790 comp rx=0 tx=0).stop 2024-03-21T11:27:45.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.467+0000 7efe04802740 1 -- 172.21.15.47:0/1138955564 shutdown_connections 2024-03-21T11:27:45.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.467+0000 7efe04802740 1 --2- 172.21.15.47:0/1138955564 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e3dfeac4a0 0x55e3dfeae8d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.467+0000 7efe04802740 1 --2- 172.21.15.47:0/1138955564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe983a0 0x55e3dfea7f60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.467+0000 7efe04802740 1 --2- 172.21.15.47:0/1138955564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe979e0 0x55e3dfea0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.467+0000 7efde8c00640 0 client.26884 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:27:45.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.467+0000 7efe04802740 1 --2- 172.21.15.47:0/1138955564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe96810 0x55e3dfea0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.467+0000 7efe04802740 1 -- 172.21.15.47:0/1138955564 >> 172.21.15.47:0/1138955564 conn(0x55e3dfe91c00 msgr2=0x55e3dfe90eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.467+0000 7efe04802740 1 -- 172.21.15.47:0/1138955564 shutdown_connections 2024-03-21T11:27:45.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.467+0000 7efe04802740 1 -- 172.21.15.47:0/1138955564 wait complete. 2024-03-21T11:27:45.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.467+0000 7efe04802740 20 client.26884 trim_cache size 0 max 16384 2024-03-21T11:27:45.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.482+0000 7efe04802740 1 Processor -- start 2024-03-21T11:27:45.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.483+0000 7efe04802740 1 -- start start 2024-03-21T11:27:45.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.483+0000 7efe04802740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe96810 0x55e3dfea0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.483+0000 7efe04802740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe979e0 0x55e3dfea7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.483+0000 7efe04802740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe983a0 0x55e3dfe9a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.483+0000 7efe04802740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1b0660 con 0x55e3dfe96810 2024-03-21T11:27:45.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.483+0000 7efe04802740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1b1fe0 con 0x55e3dfe983a0 2024-03-21T11:27:45.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.483+0000 7efe04802740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1b0cc0 con 0x55e3dfe979e0 2024-03-21T11:27:45.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.483+0000 7efdfb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe983a0 0x55e3dfe9a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.483+0000 7efdf2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe979e0 0x55e3dfea7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.483+0000 7efdfb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe983a0 0x55e3dfe9a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58034/0 (socket says 172.21.15.47:58034) 2024-03-21T11:27:45.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.483+0000 7efdf2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe979e0 0x55e3dfea7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40416/0 (socket says 172.21.15.47:40416) 2024-03-21T11:27:45.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.483+0000 7efdfb400640 1 -- 172.21.15.47:0/314478509 learned_addr learned my addr 172.21.15.47:0/314478509 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.483+0000 7efdfaa00640 1 --2- 172.21.15.47:0/314478509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe96810 0x55e3dfea0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.484+0000 7efdfb400640 1 -- 172.21.15.47:0/314478509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe979e0 msgr2=0x55e3dfea7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.484+0000 7efdfb400640 1 --2- 172.21.15.47:0/314478509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe979e0 0x55e3dfea7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.484+0000 7efdfb400640 1 -- 172.21.15.47:0/314478509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe96810 msgr2=0x55e3dfea0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.484+0000 7efdfb400640 1 --2- 172.21.15.47:0/314478509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe96810 0x55e3dfea0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.484+0000 7efdfb400640 1 -- 172.21.15.47:0/314478509 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3dfea0840 con 0x55e3dfe983a0 2024-03-21T11:27:45.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.484+0000 7efdfaa00640 1 --2- 172.21.15.47:0/314478509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe96810 0x55e3dfea0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:45.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.484+0000 7efdfb400640 1 --2- 172.21.15.47:0/314478509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe983a0 0x55e3dfe9a9e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efdf4004650 tx=0x7efdf40bc160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8823bf9da58bca8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.484+0000 7efdeb400640 1 -- 172.21.15.47:0/314478509 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf40bd5b0 con 0x55e3dfe983a0 2024-03-21T11:27:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.484+0000 7efdeb400640 1 -- 172.21.15.47:0/314478509 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efdf40a0070 con 0x55e3dfe983a0 2024-03-21T11:27:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.484+0000 7efdeb400640 1 -- 172.21.15.47:0/314478509 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf40f1520 con 0x55e3dfe983a0 2024-03-21T11:27:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.485+0000 7efe04802740 1 -- 172.21.15.47:0/314478509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe983a0 msgr2=0x55e3dfe9a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.485+0000 7efe04802740 1 --2- 172.21.15.47:0/314478509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe983a0 0x55e3dfe9a9e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efdf4004650 tx=0x7efdf40bc160 comp rx=0 tx=0).stop 2024-03-21T11:27:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.485+0000 7efe04802740 1 -- 172.21.15.47:0/314478509 shutdown_connections 2024-03-21T11:27:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.485+0000 7efe04802740 1 --2- 172.21.15.47:0/314478509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe983a0 0x55e3dfe9a9e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.485+0000 7efe04802740 1 --2- 172.21.15.47:0/314478509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe979e0 0x55e3dfea7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.485+0000 7efe04802740 1 --2- 172.21.15.47:0/314478509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe96810 0x55e3dfea0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.485+0000 7efe04802740 1 -- 172.21.15.47:0/314478509 >> 172.21.15.47:0/314478509 conn(0x55e3dfe91c00 msgr2=0x55e3dfe956f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.486+0000 7efe04802740 1 -- 172.21.15.47:0/314478509 shutdown_connections 2024-03-21T11:27:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.486+0000 7efe04802740 1 -- 172.21.15.47:0/314478509 wait complete. 2024-03-21T11:27:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.486+0000 7efe04802740 1 -- start start 2024-03-21T11:27:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.486+0000 7efe04802740 1 Processor -- start 2024-03-21T11:27:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.487+0000 7efe04802740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe96810 0x55e3dfea7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.487+0000 7efe04802740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe979e0 0x55e3dfea7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.487+0000 7efe04802740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe983a0 0x55e3dfea41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.487+0000 7efe04802740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1b1980 con 0x55e3dfe983a0 2024-03-21T11:27:45.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.487+0000 7efe04802740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1b1320 con 0x55e3dfe979e0 2024-03-21T11:27:45.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.487+0000 7efe04802740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1ae820 con 0x55e3dfe96810 2024-03-21T11:27:45.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.487+0000 7efe04802740 10 client.0 _ll_register_callbacks cb 0x7efdf4009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:45.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.487+0000 7efdf2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe979e0 0x55e3dfea7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.487+0000 7efdf2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe979e0 0x55e3dfea7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58048/0 (socket says 172.21.15.47:58048) 2024-03-21T11:27:45.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.487+0000 7efdf2000640 1 -- 172.21.15.47:0/726562536 learned_addr learned my addr 172.21.15.47:0/726562536 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.487+0000 7efdfaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe96810 0x55e3dfea7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.488+0000 7efdfb400640 1 --2- 172.21.15.47:0/726562536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe983a0 0x55e3dfea41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.488+0000 7efdf2000640 1 -- 172.21.15.47:0/726562536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe96810 msgr2=0x55e3dfea7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.488+0000 7efdf2000640 1 --2- 172.21.15.47:0/726562536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe96810 0x55e3dfea7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.488+0000 7efdf2000640 1 -- 172.21.15.47:0/726562536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe983a0 msgr2=0x55e3dfea41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.488+0000 7efdf2000640 1 --2- 172.21.15.47:0/726562536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe983a0 0x55e3dfea41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.488+0000 7efdf2000640 1 -- 172.21.15.47:0/726562536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3dfea4790 con 0x55e3dfe979e0 2024-03-21T11:27:45.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.488+0000 7efdf2000640 1 --2- 172.21.15.47:0/726562536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe979e0 0x55e3dfea7c40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efdfc19f460 tx=0x7efde4009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=469787797221a4a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.488+0000 7efdfaa00640 1 --2- 172.21.15.47:0/726562536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe96810 0x55e3dfea7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:27:45.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.488+0000 7efde8c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:45.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.488+0000 7efde8c00640 1 -- 172.21.15.47:0/726562536 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde4032060 con 0x55e3dfe979e0 2024-03-21T11:27:45.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.488+0000 7efe04802740 1 -- 172.21.15.47:0/726562536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdf40a0070 con 0x55e3dfe979e0 2024-03-21T11:27:45.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.489+0000 7efde8c00640 1 -- 172.21.15.47:0/726562536 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efde401c030 con 0x55e3dfe979e0 2024-03-21T11:27:45.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.489+0000 7efde8c00640 1 -- 172.21.15.47:0/726562536 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efde4039050 con 0x55e3dfe979e0 2024-03-21T11:27:45.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.489+0000 7efe04802740 10 client.26870 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:45.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.489+0000 7efe04802740 1 -- 172.21.15.47:0/726562536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e3dff089a0 con 0x55e3dfe979e0 2024-03-21T11:27:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.489+0000 7efde8c00640 1 -- 172.21.15.47:0/726562536 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efdf40a0070 con 0x55e3dfe979e0 2024-03-21T11:27:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.489+0000 7efde8c00640 1 -- 172.21.15.47:0/726562536 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efde402da60 con 0x55e3dfe979e0 2024-03-21T11:27:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.489+0000 7efe04802740 10 client.26870 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.489+0000 7efe04802740 10 client.26870 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:27:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.490+0000 7efe04802740 20 client.26870 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.490+0000 7efe04802740 1 --2- 172.21.15.47:0/726562536 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e3dff09310 0x55e3dff0b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.490+0000 7efe04802740 4 client.26870 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.490+0000 7efe04802740 1 -- 172.21.15.47:0/726562536 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e3dfe02b50 con 0x55e3dff09310 2024-03-21T11:27:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.490+0000 7efdfaa00640 1 --2- 172.21.15.47:0/726562536 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e3dff09310 0x55e3dff0b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.490+0000 7efdfaa00640 1 --2- 172.21.15.47:0/726562536 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e3dff09310 0x55e3dff0b730 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f8a7bc2bb5e5a70 server_cookie=d99ae7d02b6a2378 in_seq=0 out_seq=0 2024-03-21T11:27:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.490+0000 7efde8c00640 10 client.26870 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:27:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.491+0000 7efde8c00640 1 -- 172.21.15.47:0/726562536 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e3dfe02b50 con 0x55e3dff09310 2024-03-21T11:27:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.491+0000 7efde8c00640 10 client.26870 handle_command_reply: tid=0 2024-03-21T11:27:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.491+0000 7efe04802740 1 client.26870 shutdown 2024-03-21T11:27:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.491+0000 7efe04802740 1 -- 172.21.15.47:0/726562536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe979e0 msgr2=0x55e3dfea7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.491+0000 7efe04802740 1 --2- 172.21.15.47:0/726562536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe979e0 0x55e3dfea7c40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efdfc19f460 tx=0x7efde4009500 comp rx=0 tx=0).stop 2024-03-21T11:27:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.492+0000 7efe04802740 1 -- 172.21.15.47:0/726562536 shutdown_connections 2024-03-21T11:27:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.492+0000 7efe04802740 1 --2- 172.21.15.47:0/726562536 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e3dff09310 0x55e3dff0b730 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.492+0000 7efe04802740 1 --2- 172.21.15.47:0/726562536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3dfe983a0 0x55e3dfea41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.492+0000 7efe04802740 1 --2- 172.21.15.47:0/726562536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3dfe979e0 0x55e3dfea7c40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.492+0000 7efe04802740 1 --2- 172.21.15.47:0/726562536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3dfe96810 0x55e3dfea7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.492+0000 7efe04802740 1 -- 172.21.15.47:0/726562536 >> 172.21.15.47:0/726562536 conn(0x55e3dfe91c00 msgr2=0x55e3dfe956f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.492+0000 7efde8c00640 0 client.26870 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:27:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.492+0000 7efe04802740 1 -- 172.21.15.47:0/726562536 shutdown_connections 2024-03-21T11:27:45.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.492+0000 7efe04802740 1 -- 172.21.15.47:0/726562536 wait complete. 2024-03-21T11:27:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.492+0000 7efe04802740 20 client.26870 trim_cache size 0 max 16384 2024-03-21T11:27:45.497 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.493+0000 7efe01c00640 1 -- 172.21.15.47:0/2585385163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efde00a2fa0 msgr2=0x7efde00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.493+0000 7efe01c00640 1 --2- 172.21.15.47:0/2585385163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efde00a2fa0 0x7efde00a5460 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55e3dfe978e0 tx=0x7efde4009960 comp rx=0 tx=0).stop 2024-03-21T11:27:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.493+0000 7efe01c00640 1 -- 172.21.15.47:0/2585385163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdfc109870 msgr2=0x7efdfc1a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.493+0000 7efe01c00640 1 --2- 172.21.15.47:0/2585385163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdfc109870 0x7efdfc1a5500 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efdf402c8e0 tx=0x7efdf402c640 comp rx=0 tx=0).stop 2024-03-21T11:27:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.493+0000 7efe01c00640 1 -- 172.21.15.47:0/2585385163 shutdown_connections 2024-03-21T11:27:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.493+0000 7efe01c00640 1 --2- 172.21.15.47:0/2585385163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efde00a2fa0 0x7efde00a5460 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.493+0000 7efe01c00640 1 --2- 172.21.15.47:0/2585385163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdfc109870 0x7efdfc1a5500 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.493+0000 7efe01c00640 1 --2- 172.21.15.47:0/2585385163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdfc108eb0 0x7efdfc19e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.493+0000 7efe01c00640 1 --2- 172.21.15.47:0/2585385163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdfc107cc0 0x7efdfc19df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.493+0000 7efe01c00640 1 -- 172.21.15.47:0/2585385163 >> 172.21.15.47:0/2585385163 conn(0x7efdfc076c60 msgr2=0x7efdfc10c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.493+0000 7efe01c00640 1 -- 172.21.15.47:0/2585385163 shutdown_connections 2024-03-21T11:27:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.493+0000 7efe01c00640 1 -- 172.21.15.47:0/2585385163 wait complete. 2024-03-21T11:27:45.505 INFO:tasks.workunit.client.0.smithi047.stdout: 1 238665 8.00 MB/sec execute 29 sec latency 12.712 ms 2024-03-21T11:27:45.515 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:45.515 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:27:45.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.771+0000 7f517fe00640 1 -- 172.21.15.47:0/767749023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5180102760 msgr2=0x7f5180104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.771+0000 7f517fe00640 1 --2- 172.21.15.47:0/767749023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5180102760 0x7f5180104bd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5168009860 tx=0x7f516802bb70 comp rx=0 tx=0).stop 2024-03-21T11:27:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.771+0000 7f517fe00640 1 -- 172.21.15.47:0/767749023 shutdown_connections 2024-03-21T11:27:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.771+0000 7f517fe00640 1 --2- 172.21.15.47:0/767749023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5180105110 0x7f5180105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.771+0000 7f517fe00640 1 --2- 172.21.15.47:0/767749023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5180102760 0x7f5180104bd0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.771+0000 7f517fe00640 1 --2- 172.21.15.47:0/767749023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51800ffdf0 0x7f5180102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.771+0000 7f517fe00640 1 -- 172.21.15.47:0/767749023 >> 172.21.15.47:0/767749023 conn(0x7f51800fba10 msgr2=0x7f51800fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.773+0000 7f517fe00640 1 -- 172.21.15.47:0/767749023 shutdown_connections 2024-03-21T11:27:45.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.773+0000 7f517fe00640 1 -- 172.21.15.47:0/767749023 wait complete. 2024-03-21T11:27:45.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.774+0000 7f517fe00640 1 Processor -- start 2024-03-21T11:27:45.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.775+0000 7f517fe00640 1 -- start start 2024-03-21T11:27:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.776+0000 7f517fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51800ffdf0 0x7f5180071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.776+0000 7f517fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5180102760 0x7f5180072380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.776+0000 7f517fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5180105110 0x7f51801a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.776+0000 7f517fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5180109000 con 0x7f51800ffdf0 2024-03-21T11:27:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.776+0000 7f517fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5180108e80 con 0x7f5180102760 2024-03-21T11:27:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.776+0000 7f517fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5180109180 con 0x7f5180105110 2024-03-21T11:27:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.776+0000 7f517ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51800ffdf0 0x7f5180071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.776+0000 7f517ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51800ffdf0 0x7f5180071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40802/0 (socket says 172.21.15.47:40802) 2024-03-21T11:27:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.776+0000 7f517ea00640 1 -- 172.21.15.47:0/320323734 learned_addr learned my addr 172.21.15.47:0/320323734 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.777+0000 7f517ea00640 1 -- 172.21.15.47:0/320323734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5180105110 msgr2=0x7f51801a56e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T11:27:45.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.777+0000 7f517ea00640 1 --2- 172.21.15.47:0/320323734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5180105110 0x7f51801a56e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.777+0000 7f517ea00640 1 -- 172.21.15.47:0/320323734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5180102760 msgr2=0x7f5180072380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:27:45.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.777+0000 7f517ea00640 1 --2- 172.21.15.47:0/320323734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5180102760 0x7f5180072380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.777+0000 7f517ea00640 1 -- 172.21.15.47:0/320323734 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51801a5c20 con 0x7f51800ffdf0 2024-03-21T11:27:45.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.778+0000 7f517ea00640 1 --2- 172.21.15.47:0/320323734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51800ffdf0 0x7f5180071e40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f517000e9f0 tx=0x7f517000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.779+0000 7f5177400640 1 -- 172.21.15.47:0/320323734 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517000cd80 con 0x7f51800ffdf0 2024-03-21T11:27:45.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.779+0000 7f5177400640 1 -- 172.21.15.47:0/320323734 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5170005500 con 0x7f51800ffdf0 2024-03-21T11:27:45.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.779+0000 7f5177400640 1 -- 172.21.15.47:0/320323734 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170010690 con 0x7f51800ffdf0 2024-03-21T11:27:45.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.780+0000 7f517fe00640 1 -- 172.21.15.47:0/320323734 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51801a5df0 con 0x7f51800ffdf0 2024-03-21T11:27:45.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.780+0000 7f517fe00640 1 -- 172.21.15.47:0/320323734 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51801a61a0 con 0x7f51800ffdf0 2024-03-21T11:27:45.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.788+0000 7f5188372740 1 Processor -- start 2024-03-21T11:27:45.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.788+0000 7f5188372740 1 -- start start 2024-03-21T11:27:45.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.788+0000 7f5188372740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.788+0000 7f5188372740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d7bc0 0x5583511d8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.788+0000 7f5188372740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d8580 0x5583511dee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.788+0000 7f5188372740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55835113b2c0 con 0x5583511d8580 2024-03-21T11:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.788+0000 7f5188372740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583510e3c90 con 0x5583511d69f0 2024-03-21T11:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.788+0000 7f5188372740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558351137360 con 0x5583511d7bc0 2024-03-21T11:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.789+0000 7f517f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.789+0000 7f5177e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d8580 0x5583511dee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.789+0000 7f517f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511d6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58060/0 (socket says 172.21.15.47:58060) 2024-03-21T11:27:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.789+0000 7f5177e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d8580 0x5583511dee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40808/0 (socket says 172.21.15.47:40808) 2024-03-21T11:27:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.789+0000 7f517f400640 1 -- 172.21.15.47:0/22835052 learned_addr learned my addr 172.21.15.47:0/22835052 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.790+0000 7f517f400640 1 -- 172.21.15.47:0/22835052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d7bc0 msgr2=0x5583511d8040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:27:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.791+0000 7f5177400640 1 -- 172.21.15.47:0/320323734 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f51700046c0 con 0x7f51800ffdf0 2024-03-21T11:27:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.791+0000 7f517ea00640 1 --2- 172.21.15.47:0/22835052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d7bc0 0x5583511d8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.791+0000 7f517f400640 1 --2- 172.21.15.47:0/22835052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d7bc0 0x5583511d8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.791+0000 7f517f400640 1 -- 172.21.15.47:0/22835052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d8580 msgr2=0x5583511dee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.791+0000 7f517f400640 1 --2- 172.21.15.47:0/22835052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d8580 0x5583511dee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.791+0000 7f517f400640 1 -- 172.21.15.47:0/22835052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558351108080 con 0x5583511d69f0 2024-03-21T11:27:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.791+0000 7f5177e00640 1 --2- 172.21.15.47:0/22835052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d8580 0x5583511dee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.791+0000 7f517ea00640 1 --2- 172.21.15.47:0/22835052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d7bc0 0x5583511d8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:27:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.793+0000 7f517f400640 1 --2- 172.21.15.47:0/22835052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511d6e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f517801f400 tx=0x7f517801f8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9e247fd2642a5b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.793+0000 7f5177400640 1 --2- 172.21.15.47:0/320323734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f51640a30f0 0x7f51640a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.793+0000 7f5177400640 1 -- 172.21.15.47:0/320323734 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f51700cfc80 con 0x7f51800ffdf0 2024-03-21T11:27:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.794+0000 7f517ea00640 1 --2- 172.21.15.47:0/320323734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f51640a30f0 0x7f51640a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.794+0000 7f516f400640 1 -- 172.21.15.47:0/22835052 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178003970 con 0x5583511d69f0 2024-03-21T11:27:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.794+0000 7f516f400640 1 -- 172.21.15.47:0/22835052 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5178004020 con 0x5583511d69f0 2024-03-21T11:27:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.794+0000 7f516f400640 1 -- 172.21.15.47:0/22835052 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517801d860 con 0x5583511d69f0 2024-03-21T11:27:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.798+0000 7f517ea00640 1 --2- 172.21.15.47:0/320323734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f51640a30f0 0x7f51640a55b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5170098420 tx=0x7f5170004b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.799+0000 7f5188372740 1 -- 172.21.15.47:0/22835052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 msgr2=0x5583511d6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.799+0000 7f5188372740 1 --2- 172.21.15.47:0/22835052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511d6e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f517801f400 tx=0x7f517801f8c0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.799+0000 7f5188372740 1 -- 172.21.15.47:0/22835052 shutdown_connections 2024-03-21T11:27:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.799+0000 7f5188372740 1 --2- 172.21.15.47:0/22835052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d8580 0x5583511dee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.799+0000 7f5188372740 1 --2- 172.21.15.47:0/22835052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d7bc0 0x5583511d8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.799+0000 7f5188372740 1 --2- 172.21.15.47:0/22835052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511d6e10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.799+0000 7f5188372740 1 -- 172.21.15.47:0/22835052 >> 172.21.15.47:0/22835052 conn(0x5583511d1de0 msgr2=0x5583511d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.801+0000 7f5188372740 1 -- 172.21.15.47:0/22835052 shutdown_connections 2024-03-21T11:27:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.801+0000 7f5188372740 1 -- 172.21.15.47:0/22835052 wait complete. 2024-03-21T11:27:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.801+0000 7f5188372740 1 -- start start 2024-03-21T11:27:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.803+0000 7f5188372740 1 Processor -- start 2024-03-21T11:27:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.806+0000 7f5188372740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d69f0 0x5583511e0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.808+0000 7f5188372740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d7bc0 0x5583511e1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.808+0000 7f5188372740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d8580 0x5583511e8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.808+0000 7f5188372740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583511df590 con 0x5583511d7bc0 2024-03-21T11:27:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.808+0000 7f5188372740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583510f70d0 con 0x5583511d8580 2024-03-21T11:27:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.808+0000 7f5188372740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583511df710 con 0x5583511d69f0 2024-03-21T11:27:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.808+0000 7f5188372740 10 client.0 _ll_register_callbacks cb 0x5583510fbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.808+0000 7f517f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d69f0 0x5583511e0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.808+0000 7f517f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d69f0 0x5583511e0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40458/0 (socket says 172.21.15.47:40458) 2024-03-21T11:27:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.808+0000 7f517f400640 1 -- 172.21.15.47:0/623131918 learned_addr learned my addr 172.21.15.47:0/623131918 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.808+0000 7f517f400640 1 --2- 172.21.15.47:0/623131918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d8580 0x5583511e8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.809+0000 7f5177e00640 1 --2- 172.21.15.47:0/623131918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d7bc0 0x5583511e1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.809+0000 7f517f400640 1 -- 172.21.15.47:0/623131918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d8580 msgr2=0x5583511e8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.809+0000 7f517f400640 1 --2- 172.21.15.47:0/623131918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d8580 0x5583511e8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.809+0000 7f517f400640 1 -- 172.21.15.47:0/623131918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d7bc0 msgr2=0x5583511e1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.809+0000 7f517f400640 1 --2- 172.21.15.47:0/623131918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d7bc0 0x5583511e1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.809+0000 7f517f400640 1 -- 172.21.15.47:0/623131918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558350e4f3f0 con 0x5583511d69f0 2024-03-21T11:27:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.811+0000 7f517f400640 1 --2- 172.21.15.47:0/623131918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d69f0 0x5583511e0cc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f51780047e0 tx=0x7f5178004840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78a4350266995e37 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.811+0000 7f5163e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.813+0000 7f5163e00640 1 -- 172.21.15.47:0/623131918 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517802b030 con 0x5583511d69f0 2024-03-21T11:27:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.813+0000 7f5163e00640 1 -- 172.21.15.47:0/623131918 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5178024030 con 0x5583511d69f0 2024-03-21T11:27:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.813+0000 7f5188372740 1 -- 172.21.15.47:0/623131918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5178004020 con 0x5583511d69f0 2024-03-21T11:27:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.813+0000 7f5163e00640 1 -- 172.21.15.47:0/623131918 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5178004550 con 0x5583511d69f0 2024-03-21T11:27:45.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.814+0000 7f5188372740 10 client.26893 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:45.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.814+0000 7f5188372740 1 -- 172.21.15.47:0/623131918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558350f197d0 con 0x5583511d69f0 2024-03-21T11:27:45.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.815+0000 7f5163e00640 1 -- 172.21.15.47:0/623131918 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5178004020 con 0x5583511d69f0 2024-03-21T11:27:45.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.815+0000 7f5163e00640 1 -- 172.21.15.47:0/623131918 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5178036560 con 0x5583511d69f0 2024-03-21T11:27:45.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.815+0000 7f5188372740 10 client.26893 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:45.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.815+0000 7f5188372740 10 client.26893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:45.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.816+0000 7f5188372740 20 client.26893 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:45.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.816+0000 7f5188372740 1 --2- 172.21.15.47:0/623131918 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583511ec370 0x5583511ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.816+0000 7f5188372740 4 client.26893 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:45.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.816+0000 7f5188372740 1 -- 172.21.15.47:0/623131918 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558351148800 con 0x5583511ec370 2024-03-21T11:27:45.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.816+0000 7f5177e00640 1 --2- 172.21.15.47:0/623131918 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583511ec370 0x5583511ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.817+0000 7f5177e00640 1 --2- 172.21.15.47:0/623131918 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583511ec370 0x5583511ee760 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7222a46cb8290bd4 server_cookie=d2380872f088b25d in_seq=0 out_seq=0 2024-03-21T11:27:45.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.817+0000 7f5163e00640 10 client.26893 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.820+0000 7f5163e00640 1 -- 172.21.15.47:0/623131918 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558351148800 con 0x5583511ec370 2024-03-21T11:27:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.820+0000 7f5163e00640 10 client.26893 handle_command_reply: tid=0 2024-03-21T11:27:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.820+0000 7f5188372740 1 client.26893 shutdown 2024-03-21T11:27:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.821+0000 7f5188372740 1 -- 172.21.15.47:0/623131918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d69f0 msgr2=0x5583511e0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.821+0000 7f5188372740 1 --2- 172.21.15.47:0/623131918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d69f0 0x5583511e0cc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f51780047e0 tx=0x7f5178004840 comp rx=0 tx=0).stop 2024-03-21T11:27:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.821+0000 7f5188372740 1 -- 172.21.15.47:0/623131918 shutdown_connections 2024-03-21T11:27:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.821+0000 7f5188372740 1 --2- 172.21.15.47:0/623131918 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583511ec370 0x5583511ee760 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.821+0000 7f5188372740 1 --2- 172.21.15.47:0/623131918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d8580 0x5583511e8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.821+0000 7f5188372740 1 --2- 172.21.15.47:0/623131918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d7bc0 0x5583511e1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.821+0000 7f5188372740 1 --2- 172.21.15.47:0/623131918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d69f0 0x5583511e0cc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.821+0000 7f5163e00640 0 client.26893 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.821+0000 7f5188372740 1 -- 172.21.15.47:0/623131918 >> 172.21.15.47:0/623131918 conn(0x5583511d1de0 msgr2=0x5583511d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.822+0000 7f5188372740 1 -- 172.21.15.47:0/623131918 shutdown_connections 2024-03-21T11:27:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.822+0000 7f5188372740 1 -- 172.21.15.47:0/623131918 wait complete. 2024-03-21T11:27:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.823+0000 7f5188372740 20 client.26893 trim_cache size 0 max 16384 2024-03-21T11:27:45.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.841+0000 7f5188372740 1 Processor -- start 2024-03-21T11:27:45.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.842+0000 7f5188372740 1 -- start start 2024-03-21T11:27:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.842+0000 7f5188372740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.842+0000 7f5188372740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d7bc0 0x5583511e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.842+0000 7f5188372740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d8580 0x5583511d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.842+0000 7f5188372740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51801b0600 con 0x5583511d7bc0 2024-03-21T11:27:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.842+0000 7f5188372740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51801b1f80 con 0x5583511d69f0 2024-03-21T11:27:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.842+0000 7f5188372740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51801b0c60 con 0x5583511d8580 2024-03-21T11:27:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.842+0000 7f517f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.843+0000 7f517f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58086/0 (socket says 172.21.15.47:58086) 2024-03-21T11:27:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.843+0000 7f517f400640 1 -- 172.21.15.47:0/197406393 learned_addr learned my addr 172.21.15.47:0/197406393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.843+0000 7f517f400640 1 --2- 172.21.15.47:0/197406393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d8580 0x5583511d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.843+0000 7f5177e00640 1 --2- 172.21.15.47:0/197406393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d7bc0 0x5583511e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.843+0000 7f517f400640 1 -- 172.21.15.47:0/197406393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d8580 msgr2=0x5583511d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.843+0000 7f517f400640 1 --2- 172.21.15.47:0/197406393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d8580 0x5583511d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.843+0000 7f517f400640 1 -- 172.21.15.47:0/197406393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d7bc0 msgr2=0x5583511e8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.843+0000 7f517f400640 1 --2- 172.21.15.47:0/197406393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d7bc0 0x5583511e8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.843+0000 7f517f400640 1 -- 172.21.15.47:0/197406393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583511e1a80 con 0x5583511d69f0 2024-03-21T11:27:45.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.843+0000 7f5177e00640 1 --2- 172.21.15.47:0/197406393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d7bc0 0x5583511e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:27:45.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.843+0000 7f517f400640 1 --2- 172.21.15.47:0/197406393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511d08d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5178004f30 tx=0x7f5178004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52ac9b4cb95ea6a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.844+0000 7f516ea00640 1 -- 172.21.15.47:0/197406393 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178004410 con 0x5583511d69f0 2024-03-21T11:27:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.844+0000 7f516ea00640 1 -- 172.21.15.47:0/197406393 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5178023030 con 0x5583511d69f0 2024-03-21T11:27:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.844+0000 7f516ea00640 1 -- 172.21.15.47:0/197406393 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178036700 con 0x5583511d69f0 2024-03-21T11:27:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.844+0000 7f5188372740 1 -- 172.21.15.47:0/197406393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 msgr2=0x5583511d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.844+0000 7f5188372740 1 --2- 172.21.15.47:0/197406393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511d08d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5178004f30 tx=0x7f5178004770 comp rx=0 tx=0).stop 2024-03-21T11:27:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.845+0000 7f5188372740 1 -- 172.21.15.47:0/197406393 shutdown_connections 2024-03-21T11:27:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.845+0000 7f5188372740 1 --2- 172.21.15.47:0/197406393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d8580 0x5583511d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.845+0000 7f5188372740 1 --2- 172.21.15.47:0/197406393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d7bc0 0x5583511e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.845+0000 7f5188372740 1 --2- 172.21.15.47:0/197406393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511d08d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.845+0000 7f5188372740 1 -- 172.21.15.47:0/197406393 >> 172.21.15.47:0/197406393 conn(0x5583511d1de0 msgr2=0x5583511d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.845+0000 7f5188372740 1 -- 172.21.15.47:0/197406393 shutdown_connections 2024-03-21T11:27:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.845+0000 7f5188372740 1 -- 172.21.15.47:0/197406393 wait complete. 2024-03-21T11:27:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.845+0000 7f5188372740 1 -- start start 2024-03-21T11:27:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.845+0000 7f5188372740 1 Processor -- start 2024-03-21T11:27:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.846+0000 7f5188372740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511e37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.846+0000 7f5188372740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d7bc0 0x5583511e3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.846+0000 7f5188372740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d8580 0x5583511e6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.846+0000 7f5188372740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51801b1920 con 0x5583511d8580 2024-03-21T11:27:45.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.846+0000 7f5188372740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51801b12c0 con 0x5583511d69f0 2024-03-21T11:27:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.846+0000 7f5188372740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51801ae7c0 con 0x5583511d7bc0 2024-03-21T11:27:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.846+0000 7f5188372740 10 client.0 _ll_register_callbacks cb 0x7f5168037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.846+0000 7f517f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511e37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.846+0000 7f5177e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d7bc0 0x5583511e3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.846+0000 7f517f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511e37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58090/0 (socket says 172.21.15.47:58090) 2024-03-21T11:27:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.846+0000 7f517f400640 1 -- 172.21.15.47:0/3596849800 learned_addr learned my addr 172.21.15.47:0/3596849800 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.846+0000 7f517f400640 1 --2- 172.21.15.47:0/3596849800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d8580 0x5583511e6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.847+0000 7f517f400640 1 -- 172.21.15.47:0/3596849800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d7bc0 msgr2=0x5583511e3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.847+0000 7f517f400640 1 --2- 172.21.15.47:0/3596849800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d7bc0 0x5583511e3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.847+0000 7f517f400640 1 -- 172.21.15.47:0/3596849800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d8580 msgr2=0x5583511e6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.847+0000 7f517f400640 1 --2- 172.21.15.47:0/3596849800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d8580 0x5583511e6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.847+0000 7f517f400640 1 -- 172.21.15.47:0/3596849800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583511e7330 con 0x5583511d69f0 2024-03-21T11:27:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.847+0000 7f5177e00640 1 --2- 172.21.15.47:0/3596849800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d7bc0 0x5583511e3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.847+0000 7f517f400640 1 --2- 172.21.15.47:0/3596849800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511e37e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f517801a730 tx=0x7f5178036c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a31ca932d0f7caa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.847+0000 7f5163e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.847+0000 7f5163e00640 1 -- 172.21.15.47:0/3596849800 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178059060 con 0x5583511d69f0 2024-03-21T11:27:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.847+0000 7f5188372740 1 -- 172.21.15.47:0/3596849800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5178023030 con 0x5583511d69f0 2024-03-21T11:27:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.847+0000 7f5163e00640 1 -- 172.21.15.47:0/3596849800 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5178002030 con 0x5583511d69f0 2024-03-21T11:27:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.847+0000 7f5163e00640 1 -- 172.21.15.47:0/3596849800 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f517801abc0 con 0x5583511d69f0 2024-03-21T11:27:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.847+0000 7f5188372740 10 client.26885 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.847+0000 7f5188372740 1 -- 172.21.15.47:0/3596849800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558351247b30 con 0x5583511d69f0 2024-03-21T11:27:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.848+0000 7f5163e00640 1 -- 172.21.15.47:0/3596849800 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5178023030 con 0x5583511d69f0 2024-03-21T11:27:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.848+0000 7f5163e00640 1 -- 172.21.15.47:0/3596849800 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5178054440 con 0x5583511d69f0 2024-03-21T11:27:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.848+0000 7f5188372740 10 client.26885 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.848+0000 7f5188372740 10 client.26885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.848+0000 7f5188372740 20 client.26885 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.848+0000 7f5188372740 1 --2- 172.21.15.47:0/3596849800 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558351248150 0x55835124a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.848+0000 7f5188372740 4 client.26885 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.848+0000 7f5188372740 1 -- 172.21.15.47:0/3596849800 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558351142b50 con 0x558351248150 2024-03-21T11:27:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.848+0000 7f5177e00640 1 --2- 172.21.15.47:0/3596849800 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558351248150 0x55835124a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:45.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.849+0000 7f5177e00640 1 --2- 172.21.15.47:0/3596849800 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558351248150 0x55835124a540 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b21ac6f730ebe5 server_cookie=3cf7c0ec6758d40 in_seq=0 out_seq=0 2024-03-21T11:27:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.849+0000 7f5163e00640 10 client.26885 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.849+0000 7f5163e00640 1 -- 172.21.15.47:0/3596849800 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558351142b50 con 0x558351248150 2024-03-21T11:27:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.849+0000 7f5163e00640 10 client.26885 handle_command_reply: tid=0 2024-03-21T11:27:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.850+0000 7f5188372740 1 client.26885 shutdown 2024-03-21T11:27:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.850+0000 7f5188372740 1 -- 172.21.15.47:0/3596849800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 msgr2=0x5583511e37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.850+0000 7f5188372740 1 --2- 172.21.15.47:0/3596849800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511e37e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f517801a730 tx=0x7f5178036c50 comp rx=0 tx=0).stop 2024-03-21T11:27:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.850+0000 7f5188372740 1 -- 172.21.15.47:0/3596849800 shutdown_connections 2024-03-21T11:27:45.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.850+0000 7f5188372740 1 --2- 172.21.15.47:0/3596849800 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558351248150 0x55835124a540 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.850+0000 7f5188372740 1 --2- 172.21.15.47:0/3596849800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583511d8580 0x5583511e6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.850+0000 7f5188372740 1 --2- 172.21.15.47:0/3596849800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583511d7bc0 0x5583511e3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.850+0000 7f5188372740 1 --2- 172.21.15.47:0/3596849800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583511d69f0 0x5583511e37e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.850+0000 7f5163e00640 0 client.26885 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:45.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.850+0000 7f5188372740 1 -- 172.21.15.47:0/3596849800 >> 172.21.15.47:0/3596849800 conn(0x5583511d1de0 msgr2=0x5583511d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.850+0000 7f5188372740 1 -- 172.21.15.47:0/3596849800 shutdown_connections 2024-03-21T11:27:45.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.850+0000 7f5188372740 1 -- 172.21.15.47:0/3596849800 wait complete. 2024-03-21T11:27:45.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.850+0000 7f5188372740 20 client.26885 trim_cache size 0 max 16384 2024-03-21T11:27:45.858 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:45.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.851+0000 7f517fe00640 1 -- 172.21.15.47:0/320323734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f51640a30f0 msgr2=0x7f51640a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.851+0000 7f517fe00640 1 --2- 172.21.15.47:0/320323734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f51640a30f0 0x7f51640a55b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5170098420 tx=0x7f5170004b70 comp rx=0 tx=0).stop 2024-03-21T11:27:45.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.851+0000 7f517fe00640 1 -- 172.21.15.47:0/320323734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51800ffdf0 msgr2=0x7f5180071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:45.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.851+0000 7f517fe00640 1 --2- 172.21.15.47:0/320323734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51800ffdf0 0x7f5180071e40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f517000e9f0 tx=0x7f517000edf0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.851+0000 7f517fe00640 1 -- 172.21.15.47:0/320323734 shutdown_connections 2024-03-21T11:27:45.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.851+0000 7f517fe00640 1 --2- 172.21.15.47:0/320323734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f51640a30f0 0x7f51640a55b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.851+0000 7f517fe00640 1 --2- 172.21.15.47:0/320323734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5180105110 0x7f51801a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.851+0000 7f517fe00640 1 --2- 172.21.15.47:0/320323734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5180102760 0x7f5180072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.851+0000 7f517fe00640 1 --2- 172.21.15.47:0/320323734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51800ffdf0 0x7f5180071e40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:45.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.851+0000 7f517fe00640 1 -- 172.21.15.47:0/320323734 >> 172.21.15.47:0/320323734 conn(0x7f51800fba10 msgr2=0x7f51800fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:45.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.851+0000 7f517fe00640 1 -- 172.21.15.47:0/320323734 shutdown_connections 2024-03-21T11:27:45.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:45.852+0000 7f517fe00640 1 -- 172.21.15.47:0/320323734 wait complete. 2024-03-21T11:27:46.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:45 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:27:46.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:45 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:27:46.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:45 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:27:46.505 INFO:tasks.workunit.client.0.smithi047.stdout: 1 240480 8.07 MB/sec execute 30 sec latency 66.976 ms 2024-03-21T11:27:46.870 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:27:46.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:27:46.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:27:47.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:46 smithi047 ceph-mon[31351]: pgmap v533: 129 pgs: 129 active+clean; 34 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.0 MiB/s wr, 271 op/s 2024-03-21T11:27:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.102+0000 7f486ca00640 1 -- 172.21.15.47:0/2175960465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4868101150 msgr2=0x7f48681015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.102+0000 7f486ca00640 1 --2- 172.21.15.47:0/2175960465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4868101150 0x7f48681015d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4860009900 tx=0x7f486002bc40 comp rx=0 tx=0).stop 2024-03-21T11:27:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.102+0000 7f486ca00640 1 -- 172.21.15.47:0/2175960465 shutdown_connections 2024-03-21T11:27:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.102+0000 7f486ca00640 1 --2- 172.21.15.47:0/2175960465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4868101150 0x7f48681015d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.102+0000 7f486ca00640 1 --2- 172.21.15.47:0/2175960465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48681007b0 0x7f4868100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.102+0000 7f486ca00640 1 --2- 172.21.15.47:0/2175960465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48680ffe50 0x7f4868100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.102+0000 7f486ca00640 1 -- 172.21.15.47:0/2175960465 >> 172.21.15.47:0/2175960465 conn(0x7f48680fba10 msgr2=0x7f48680fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.102+0000 7f486ca00640 1 -- 172.21.15.47:0/2175960465 shutdown_connections 2024-03-21T11:27:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.102+0000 7f486ca00640 1 -- 172.21.15.47:0/2175960465 wait complete. 2024-03-21T11:27:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.103+0000 7f486ca00640 1 Processor -- start 2024-03-21T11:27:47.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.103+0000 7f486ca00640 1 -- start start 2024-03-21T11:27:47.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.103+0000 7f486ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48680ffe50 0x7f4868199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.103+0000 7f486ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48681007b0 0x7f4868199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.103+0000 7f486ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4868101150 0x7f48681a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.103+0000 7f486ca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486810f420 con 0x7f48681007b0 2024-03-21T11:27:47.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.103+0000 7f486ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486810f2a0 con 0x7f48680ffe50 2024-03-21T11:27:47.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.103+0000 7f486ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486810f5a0 con 0x7f4868101150 2024-03-21T11:27:47.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.104+0000 7f4867400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48680ffe50 0x7f4868199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.104+0000 7f4867e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4868101150 0x7f48681a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.104+0000 7f4867e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4868101150 0x7f48681a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40482/0 (socket says 172.21.15.47:40482) 2024-03-21T11:27:47.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.104+0000 7f4866a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48681007b0 0x7f4868199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.104+0000 7f4867e00640 1 -- 172.21.15.47:0/3616203770 learned_addr learned my addr 172.21.15.47:0/3616203770 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.104+0000 7f4867400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48680ffe50 0x7f4868199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58108/0 (socket says 172.21.15.47:58108) 2024-03-21T11:27:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.104+0000 7f4867e00640 1 -- 172.21.15.47:0/3616203770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48680ffe50 msgr2=0x7f4868199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.104+0000 7f4867e00640 1 --2- 172.21.15.47:0/3616203770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48680ffe50 0x7f4868199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.104+0000 7f4867e00640 1 -- 172.21.15.47:0/3616203770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48681007b0 msgr2=0x7f4868199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.104+0000 7f4867e00640 1 --2- 172.21.15.47:0/3616203770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48681007b0 0x7f4868199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.104+0000 7f4867e00640 1 -- 172.21.15.47:0/3616203770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48681a1620 con 0x7f4868101150 2024-03-21T11:27:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.104+0000 7f4867400640 1 --2- 172.21.15.47:0/3616203770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48680ffe50 0x7f4868199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.104+0000 7f4867e00640 1 --2- 172.21.15.47:0/3616203770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4868101150 0x7f48681a0fb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f486002c180 tx=0x7f4860005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.105+0000 7f485fe00640 1 -- 172.21.15.47:0/3616203770 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860041070 con 0x7f4868101150 2024-03-21T11:27:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.105+0000 7f486ca00640 1 -- 172.21.15.47:0/3616203770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48681a18b0 con 0x7f4868101150 2024-03-21T11:27:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.105+0000 7f485fe00640 1 -- 172.21.15.47:0/3616203770 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48600054e0 con 0x7f4868101150 2024-03-21T11:27:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.105+0000 7f485fe00640 1 -- 172.21.15.47:0/3616203770 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860034440 con 0x7f4868101150 2024-03-21T11:27:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.105+0000 7f486ca00640 1 -- 172.21.15.47:0/3616203770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48681a1da0 con 0x7f4868101150 2024-03-21T11:27:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.108+0000 7f4870c2a740 1 Processor -- start 2024-03-21T11:27:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.109+0000 7f485fe00640 1 -- 172.21.15.47:0/3616203770 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4860044070 con 0x7f4868101150 2024-03-21T11:27:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.109+0000 7f4870c2a740 1 -- start start 2024-03-21T11:27:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.109+0000 7f4870c2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c1a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.109+0000 7f4870c2a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c1f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.109+0000 7f4870c2a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c222a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.109+0000 7f4870c2a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a1b7feb0 con 0x55e9a1c1aca0 2024-03-21T11:27:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.109+0000 7f4870c2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a1b28a60 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.109+0000 7f4870c2a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a1b7bf50 con 0x55e9a1c1fe10 2024-03-21T11:27:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.110+0000 7f4867400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c1f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.110+0000 7f4866a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c222a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.110+0000 7f4867400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c1f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40862/0 (socket says 172.21.15.47:40862) 2024-03-21T11:27:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.110+0000 7f4867400640 1 -- 172.21.15.47:0/1700010586 learned_addr learned my addr 172.21.15.47:0/1700010586 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.110+0000 7f4866a00640 1 --2- 172.21.15.47:0/1700010586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c1a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.110+0000 7f4867400640 1 -- 172.21.15.47:0/1700010586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1fe10 msgr2=0x55e9a1c222a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.110+0000 7f4867400640 1 --2- 172.21.15.47:0/1700010586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c222a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.110+0000 7f4867400640 1 -- 172.21.15.47:0/1700010586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1a2b0 msgr2=0x55e9a1c1a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.110+0000 7f4867400640 1 --2- 172.21.15.47:0/1700010586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c1a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.110+0000 7f4867400640 1 -- 172.21.15.47:0/1700010586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9a195c7e0 con 0x55e9a1c1aca0 2024-03-21T11:27:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.111+0000 7f4866a00640 1 --2- 172.21.15.47:0/1700010586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.111+0000 7f485fe00640 1 --2- 172.21.15.47:0/3616203770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f484c0a2f20 0x7f484c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.111+0000 7f4867400640 1 --2- 172.21.15.47:0/1700010586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c1f7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f48580108c0 tx=0x7f4858010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f348d62718a7a7f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:47.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.111+0000 7f485fe00640 1 -- 172.21.15.47:0/3616203770 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f48600f1430 con 0x7f4868101150 2024-03-21T11:27:47.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.111+0000 7f4857e00640 1 -- 172.21.15.47:0/1700010586 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858004d20 con 0x55e9a1c1aca0 2024-03-21T11:27:47.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.111+0000 7f4866a00640 1 --2- 172.21.15.47:0/3616203770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f484c0a2f20 0x7f484c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.111+0000 7f4857e00640 1 -- 172.21.15.47:0/1700010586 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4858015070 con 0x55e9a1c1aca0 2024-03-21T11:27:47.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.111+0000 7f4857e00640 1 -- 172.21.15.47:0/1700010586 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858005870 con 0x55e9a1c1aca0 2024-03-21T11:27:47.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.112+0000 7f4870c2a740 1 -- 172.21.15.47:0/1700010586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1aca0 msgr2=0x55e9a1c1f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.112+0000 7f4870c2a740 1 --2- 172.21.15.47:0/1700010586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c1f7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f48580108c0 tx=0x7f4858010d80 comp rx=0 tx=0).stop 2024-03-21T11:27:47.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.112+0000 7f4866a00640 1 --2- 172.21.15.47:0/3616203770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f484c0a2f20 0x7f484c0a53e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55e9a1c1b500 tx=0x7f4850004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:47.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.112+0000 7f4870c2a740 1 -- 172.21.15.47:0/1700010586 shutdown_connections 2024-03-21T11:27:47.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.112+0000 7f4870c2a740 1 --2- 172.21.15.47:0/1700010586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.112+0000 7f4870c2a740 1 --2- 172.21.15.47:0/1700010586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c1f7a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.112+0000 7f4870c2a740 1 --2- 172.21.15.47:0/1700010586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c1a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.112+0000 7f4870c2a740 1 -- 172.21.15.47:0/1700010586 >> 172.21.15.47:0/1700010586 conn(0x55e9a1c15820 msgr2=0x55e9a1c15c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.113+0000 7f4870c2a740 1 -- 172.21.15.47:0/1700010586 shutdown_connections 2024-03-21T11:27:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.113+0000 7f4870c2a740 1 -- 172.21.15.47:0/1700010586 wait complete. 2024-03-21T11:27:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.113+0000 7f4870c2a740 1 -- start start 2024-03-21T11:27:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.113+0000 7f4870c2a740 1 Processor -- start 2024-03-21T11:27:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.114+0000 7f4870c2a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c24490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.114+0000 7f4870c2a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c249d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.114+0000 7f4870c2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c2ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.114+0000 7f4870c2a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a1c22dc0 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.114+0000 7f4870c2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a1b3beb0 con 0x55e9a1c1fe10 2024-03-21T11:27:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.114+0000 7f4870c2a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a1c22f40 con 0x55e9a1c1aca0 2024-03-21T11:27:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.114+0000 7f4870c2a740 10 client.0 _ll_register_callbacks cb 0x55e9a1b30150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.114+0000 7f4866a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c249d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.114+0000 7f4867400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c24490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.114+0000 7f4867e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c2ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.114+0000 7f4866a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c249d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40496/0 (socket says 172.21.15.47:40496) 2024-03-21T11:27:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.114+0000 7f4866a00640 1 -- 172.21.15.47:0/3750855200 learned_addr learned my addr 172.21.15.47:0/3750855200 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.114+0000 7f4867e00640 1 -- 172.21.15.47:0/3750855200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1aca0 msgr2=0x55e9a1c249d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.114+0000 7f4867e00640 1 --2- 172.21.15.47:0/3750855200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c249d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.115+0000 7f4867e00640 1 -- 172.21.15.47:0/3750855200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1a2b0 msgr2=0x55e9a1c24490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.115+0000 7f4867e00640 1 --2- 172.21.15.47:0/3750855200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c24490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.115+0000 7f4867e00640 1 -- 172.21.15.47:0/3750855200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9a19bcf70 con 0x55e9a1c1fe10 2024-03-21T11:27:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.115+0000 7f4867e00640 1 --2- 172.21.15.47:0/3750855200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c2ba10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f48600eebf0 tx=0x7f48600bd1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2588409b92ac18e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.115+0000 7f4854c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.115+0000 7f4854c00640 1 -- 172.21.15.47:0/3750855200 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486004b040 con 0x55e9a1c1fe10 2024-03-21T11:27:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.115+0000 7f4854c00640 1 -- 172.21.15.47:0/3750855200 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4860003680 con 0x55e9a1c1fe10 2024-03-21T11:27:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.115+0000 7f4870c2a740 1 -- 172.21.15.47:0/3750855200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4858015070 con 0x55e9a1c1fe10 2024-03-21T11:27:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.115+0000 7f4854c00640 1 -- 172.21.15.47:0/3750855200 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48600c1920 con 0x55e9a1c1fe10 2024-03-21T11:27:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.115+0000 7f4870c2a740 10 client.26897 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.115+0000 7f4870c2a740 1 -- 172.21.15.47:0/3750855200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e9a1854170 con 0x55e9a1c1fe10 2024-03-21T11:27:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.116+0000 7f4854c00640 1 -- 172.21.15.47:0/3750855200 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4858015070 con 0x55e9a1c1fe10 2024-03-21T11:27:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.116+0000 7f4854c00640 1 -- 172.21.15.47:0/3750855200 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f48600c5de0 con 0x55e9a1c1fe10 2024-03-21T11:27:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.116+0000 7f4870c2a740 10 client.26897 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.116+0000 7f4870c2a740 10 client.26897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.116+0000 7f4870c2a740 20 client.26897 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.116+0000 7f4870c2a740 1 --2- 172.21.15.47:0/3750855200 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9a1c301d0 0x55e9a1c32670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.116+0000 7f4870c2a740 4 client.26897 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.116+0000 7f4870c2a740 1 -- 172.21.15.47:0/3750855200 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9a1b8d3f0 con 0x55e9a1c301d0 2024-03-21T11:27:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.116+0000 7f4867400640 1 --2- 172.21.15.47:0/3750855200 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9a1c301d0 0x55e9a1c32670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.117+0000 7f4867400640 1 --2- 172.21.15.47:0/3750855200 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9a1c301d0 0x55e9a1c32670 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfc2d3d460ba6ef0 server_cookie=65e7dd919b693155 in_seq=0 out_seq=0 2024-03-21T11:27:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.117+0000 7f4854c00640 10 client.26897 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.118+0000 7f4854c00640 1 -- 172.21.15.47:0/3750855200 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e9a1b8d3f0 con 0x55e9a1c301d0 2024-03-21T11:27:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.118+0000 7f4854c00640 10 client.26897 handle_command_reply: tid=0 2024-03-21T11:27:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.118+0000 7f4870c2a740 1 client.26897 shutdown 2024-03-21T11:27:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.118+0000 7f4870c2a740 1 -- 172.21.15.47:0/3750855200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1fe10 msgr2=0x55e9a1c2ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.118+0000 7f4870c2a740 1 --2- 172.21.15.47:0/3750855200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c2ba10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f48600eebf0 tx=0x7f48600bd1e0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.118+0000 7f4870c2a740 1 -- 172.21.15.47:0/3750855200 shutdown_connections 2024-03-21T11:27:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.118+0000 7f4870c2a740 1 --2- 172.21.15.47:0/3750855200 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9a1c301d0 0x55e9a1c32670 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.118+0000 7f4870c2a740 1 --2- 172.21.15.47:0/3750855200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c2ba10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.118+0000 7f4870c2a740 1 --2- 172.21.15.47:0/3750855200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c249d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.118+0000 7f4870c2a740 1 --2- 172.21.15.47:0/3750855200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c24490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.118+0000 7f4870c2a740 1 -- 172.21.15.47:0/3750855200 >> 172.21.15.47:0/3750855200 conn(0x55e9a1c15820 msgr2=0x55e9a1c14a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.118+0000 7f4854c00640 0 client.26897 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.118+0000 7f4870c2a740 1 -- 172.21.15.47:0/3750855200 shutdown_connections 2024-03-21T11:27:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.118+0000 7f4870c2a740 1 -- 172.21.15.47:0/3750855200 wait complete. 2024-03-21T11:27:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.119+0000 7f4870c2a740 20 client.26897 trim_cache size 0 max 16384 2024-03-21T11:27:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.138+0000 7f4870c2a740 1 Processor -- start 2024-03-21T11:27:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.138+0000 7f4870c2a740 1 -- start start 2024-03-21T11:27:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4870c2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c14900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4870c2a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c2b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4870c2a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c36f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4870c2a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48681abfe0 con 0x55e9a1c1fe10 2024-03-21T11:27:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4870c2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48681ad960 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4870c2a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48681ac640 con 0x55e9a1c1aca0 2024-03-21T11:27:47.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4867400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c14900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4867400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c14900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58132/0 (socket says 172.21.15.47:58132) 2024-03-21T11:27:47.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4866a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c2b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4867400640 1 -- 172.21.15.47:0/671719296 learned_addr learned my addr 172.21.15.47:0/671719296 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:47.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4867e00640 1 --2- 172.21.15.47:0/671719296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c36f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4867400640 1 -- 172.21.15.47:0/671719296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1aca0 msgr2=0x55e9a1c2b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4867400640 1 --2- 172.21.15.47:0/671719296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c2b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4867400640 1 -- 172.21.15.47:0/671719296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1fe10 msgr2=0x55e9a1c36f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4867400640 1 --2- 172.21.15.47:0/671719296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c36f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.139+0000 7f4867400640 1 -- 172.21.15.47:0/671719296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9a1c24d40 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.140+0000 7f4867400640 1 --2- 172.21.15.47:0/671719296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c14900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f48580107e0 tx=0x7f4858010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eab452d41e86302a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:47.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.140+0000 7f4857400640 1 -- 172.21.15.47:0/671719296 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48580050d0 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.140+0000 7f4857400640 1 -- 172.21.15.47:0/671719296 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4858015040 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.140+0000 7f4857400640 1 -- 172.21.15.47:0/671719296 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485800ed10 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:46 smithi100 ceph-mon[36245]: pgmap v533: 129 pgs: 129 active+clean; 34 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.0 MiB/s wr, 271 op/s 2024-03-21T11:27:47.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.141+0000 7f4870c2a740 1 -- 172.21.15.47:0/671719296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1a2b0 msgr2=0x55e9a1c14900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.141+0000 7f4870c2a740 1 --2- 172.21.15.47:0/671719296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c14900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f48580107e0 tx=0x7f4858010fd0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.141+0000 7f4870c2a740 1 -- 172.21.15.47:0/671719296 shutdown_connections 2024-03-21T11:27:47.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.141+0000 7f4870c2a740 1 --2- 172.21.15.47:0/671719296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c36f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.141+0000 7f4870c2a740 1 --2- 172.21.15.47:0/671719296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c2b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.141+0000 7f4870c2a740 1 --2- 172.21.15.47:0/671719296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c14900 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.141+0000 7f4870c2a740 1 -- 172.21.15.47:0/671719296 >> 172.21.15.47:0/671719296 conn(0x55e9a1c15820 msgr2=0x55e9a1c141f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:47.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.141+0000 7f4870c2a740 1 -- 172.21.15.47:0/671719296 shutdown_connections 2024-03-21T11:27:47.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.141+0000 7f4870c2a740 1 -- 172.21.15.47:0/671719296 wait complete. 2024-03-21T11:27:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.141+0000 7f4870c2a740 1 -- start start 2024-03-21T11:27:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.142+0000 7f4870c2a740 1 Processor -- start 2024-03-21T11:27:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.142+0000 7f4870c2a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c2a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.142+0000 7f4870c2a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c2a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.142+0000 7f4870c2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c2aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.142+0000 7f4870c2a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48681ad300 con 0x55e9a1c1aca0 2024-03-21T11:27:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.142+0000 7f4870c2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48681acca0 con 0x55e9a1c1fe10 2024-03-21T11:27:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.142+0000 7f4870c2a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48681aa1a0 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.142+0000 7f4870c2a740 10 client.0 _ll_register_callbacks cb 0x7f4860003f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.142+0000 7f4867400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c2a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.142+0000 7f4867400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c2a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40512/0 (socket says 172.21.15.47:40512) 2024-03-21T11:27:47.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.142+0000 7f4867400640 1 -- 172.21.15.47:0/3221711639 learned_addr learned my addr 172.21.15.47:0/3221711639 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.142+0000 7f4866a00640 1 --2- 172.21.15.47:0/3221711639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c2a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.142+0000 7f4867e00640 1 --2- 172.21.15.47:0/3221711639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c2aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.143+0000 7f4867400640 1 -- 172.21.15.47:0/3221711639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1fe10 msgr2=0x55e9a1c2aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.143+0000 7f4867400640 1 --2- 172.21.15.47:0/3221711639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c2aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.143+0000 7f4867400640 1 -- 172.21.15.47:0/3221711639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1aca0 msgr2=0x55e9a1c2a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.143+0000 7f4867400640 1 --2- 172.21.15.47:0/3221711639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c2a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.143+0000 7f4867400640 1 -- 172.21.15.47:0/3221711639 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9a1c8bbf0 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.143+0000 7f4867400640 1 --2- 172.21.15.47:0/3221711639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c2a220 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4858027040 tx=0x7f4858010810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42b879e4afb47496 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.143+0000 7f4854c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.143+0000 7f4854c00640 1 -- 172.21.15.47:0/3221711639 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48580050d0 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.144+0000 7f4870c2a740 1 -- 172.21.15.47:0/3221711639 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4858015040 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.144+0000 7f4854c00640 1 -- 172.21.15.47:0/3221711639 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4858005270 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.144+0000 7f4854c00640 1 -- 172.21.15.47:0/3221711639 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f485800ea90 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.144+0000 7f4870c2a740 10 client.26917 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.144+0000 7f4870c2a740 1 -- 172.21.15.47:0/3221711639 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e9a1c8cf10 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.144+0000 7f4854c00640 1 -- 172.21.15.47:0/3221711639 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4858015040 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.144+0000 7f4854c00640 1 -- 172.21.15.47:0/3221711639 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f485802a8f0 con 0x55e9a1c1a2b0 2024-03-21T11:27:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.144+0000 7f4870c2a740 10 client.26917 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.144+0000 7f4870c2a740 10 client.26917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.144+0000 7f4870c2a740 20 client.26917 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.144+0000 7f4870c2a740 1 --2- 172.21.15.47:0/3221711639 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9a1c8d6b0 0x55e9a1c8fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.144+0000 7f4870c2a740 4 client.26917 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:27:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.144+0000 7f4870c2a740 1 -- 172.21.15.47:0/3221711639 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e9a1b87740 con 0x55e9a1c8d6b0 2024-03-21T11:27:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.145+0000 7f4866a00640 1 --2- 172.21.15.47:0/3221711639 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9a1c8d6b0 0x55e9a1c8fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.145+0000 7f4866a00640 1 --2- 172.21.15.47:0/3221711639 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9a1c8d6b0 0x55e9a1c8fad0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320ba690ed9b0f8c server_cookie=293a5809c5eafcd5 in_seq=0 out_seq=0 2024-03-21T11:27:47.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.145+0000 7f4854c00640 10 client.26917 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:47.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.147+0000 7f4854c00640 1 -- 172.21.15.47:0/3221711639 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e9a1b87740 con 0x55e9a1c8d6b0 2024-03-21T11:27:47.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.147+0000 7f4854c00640 10 client.26917 handle_command_reply: tid=0 2024-03-21T11:27:47.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.147+0000 7f4870c2a740 1 client.26917 shutdown 2024-03-21T11:27:47.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.147+0000 7f4870c2a740 1 -- 172.21.15.47:0/3221711639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1a2b0 msgr2=0x55e9a1c2a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.147+0000 7f4870c2a740 1 --2- 172.21.15.47:0/3221711639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c2a220 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4858027040 tx=0x7f4858010810 comp rx=0 tx=0).stop 2024-03-21T11:27:47.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.147+0000 7f4870c2a740 1 -- 172.21.15.47:0/3221711639 shutdown_connections 2024-03-21T11:27:47.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.147+0000 7f4870c2a740 1 --2- 172.21.15.47:0/3221711639 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9a1c8d6b0 0x55e9a1c8fad0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.147+0000 7f4870c2a740 1 --2- 172.21.15.47:0/3221711639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9a1c1fe10 0x55e9a1c2aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.147+0000 7f4870c2a740 1 --2- 172.21.15.47:0/3221711639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9a1c1aca0 0x55e9a1c2a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.147+0000 7f4854c00640 0 client.26917 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:47.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.147+0000 7f4870c2a740 1 --2- 172.21.15.47:0/3221711639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9a1c1a2b0 0x55e9a1c2a220 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.147+0000 7f4870c2a740 1 -- 172.21.15.47:0/3221711639 >> 172.21.15.47:0/3221711639 conn(0x55e9a1c15820 msgr2=0x55e9a1c14a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:47.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.148+0000 7f4870c2a740 1 -- 172.21.15.47:0/3221711639 shutdown_connections 2024-03-21T11:27:47.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.148+0000 7f4870c2a740 1 -- 172.21.15.47:0/3221711639 wait complete. 2024-03-21T11:27:47.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.148+0000 7f4870c2a740 20 client.26917 trim_cache size 0 max 16384 2024-03-21T11:27:47.152 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:27:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:27:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "013ff486-dfb8-4fff-8af4-14f8cf626e3d", 2024-03-21T11:27:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:27:47.153 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:27:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.148+0000 7f486ca00640 1 -- 172.21.15.47:0/3616203770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f484c0a2f20 msgr2=0x7f484c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.148+0000 7f486ca00640 1 --2- 172.21.15.47:0/3616203770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f484c0a2f20 0x7f484c0a53e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55e9a1c1b500 tx=0x7f4850004730 comp rx=0 tx=0).stop 2024-03-21T11:27:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.148+0000 7f486ca00640 1 -- 172.21.15.47:0/3616203770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4868101150 msgr2=0x7f48681a0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.148+0000 7f486ca00640 1 --2- 172.21.15.47:0/3616203770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4868101150 0x7f48681a0fb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f486002c180 tx=0x7f4860005d50 comp rx=0 tx=0).stop 2024-03-21T11:27:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.149+0000 7f486ca00640 1 -- 172.21.15.47:0/3616203770 shutdown_connections 2024-03-21T11:27:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.149+0000 7f486ca00640 1 --2- 172.21.15.47:0/3616203770 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f484c0a2f20 0x7f484c0a53e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.149+0000 7f486ca00640 1 --2- 172.21.15.47:0/3616203770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4868101150 0x7f48681a0fb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.149+0000 7f486ca00640 1 --2- 172.21.15.47:0/3616203770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48681007b0 0x7f4868199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.149+0000 7f486ca00640 1 --2- 172.21.15.47:0/3616203770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48680ffe50 0x7f4868199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.149+0000 7f486ca00640 1 -- 172.21.15.47:0/3616203770 >> 172.21.15.47:0/3616203770 conn(0x7f48680fba10 msgr2=0x7f48680fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:47.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.149+0000 7f486ca00640 1 -- 172.21.15.47:0/3616203770 shutdown_connections 2024-03-21T11:27:47.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.149+0000 7f486ca00640 1 -- 172.21.15.47:0/3616203770 wait complete. 2024-03-21T11:27:47.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:27:47.171 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:27:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:46 smithi102 ceph-mon[36748]: pgmap v533: 129 pgs: 129 active+clean; 34 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.0 MiB/s wr, 271 op/s 2024-03-21T11:27:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.381+0000 7fc1ae800640 1 -- 172.21.15.47:0/157969424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1a81021e0 msgr2=0x7fc1a81045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.381+0000 7fc1ae800640 1 --2- 172.21.15.47:0/157969424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1a81021e0 0x7fc1a81045d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc198009900 tx=0x7fc19802bc30 comp rx=0 tx=0).stop 2024-03-21T11:27:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.381+0000 7fc1ae800640 1 -- 172.21.15.47:0/157969424 shutdown_connections 2024-03-21T11:27:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.381+0000 7fc1ae800640 1 --2- 172.21.15.47:0/157969424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1a8107440 0x7fc1a8109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.381+0000 7fc1ae800640 1 --2- 172.21.15.47:0/157969424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a8104b10 0x7fc1a8106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.381+0000 7fc1ae800640 1 --2- 172.21.15.47:0/157969424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1a81021e0 0x7fc1a81045d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.381+0000 7fc1ae800640 1 -- 172.21.15.47:0/157969424 >> 172.21.15.47:0/157969424 conn(0x7fc1a80fbb30 msgr2=0x7fc1a80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.381+0000 7fc1ae800640 1 -- 172.21.15.47:0/157969424 shutdown_connections 2024-03-21T11:27:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.381+0000 7fc1ae800640 1 -- 172.21.15.47:0/157969424 wait complete. 2024-03-21T11:27:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.382+0000 7fc1ae800640 1 Processor -- start 2024-03-21T11:27:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.382+0000 7fc1ae800640 1 -- start start 2024-03-21T11:27:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.382+0000 7fc1ae800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a81021e0 0x7fc1a8199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1ae800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1a8104b10 0x7fc1a819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1ae800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1a8107440 0x7fc1a81a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1ae800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a810d330 con 0x7fc1a81021e0 2024-03-21T11:27:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1ae800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a810d1b0 con 0x7fc1a8107440 2024-03-21T11:27:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1ae800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a810d4b0 con 0x7fc1a8104b10 2024-03-21T11:27:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a81021e0 0x7fc1a8199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1a8104b10 0x7fc1a819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1a8104b10 0x7fc1a819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40534/0 (socket says 172.21.15.47:40534) 2024-03-21T11:27:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1a6a00640 1 -- 172.21.15.47:0/1638269093 learned_addr learned my addr 172.21.15.47:0/1638269093 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1a7e00640 1 --2- 172.21.15.47:0/1638269093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1a8107440 0x7fc1a81a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1a6a00640 1 -- 172.21.15.47:0/1638269093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1a8107440 msgr2=0x7fc1a81a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1a6a00640 1 --2- 172.21.15.47:0/1638269093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1a8107440 0x7fc1a81a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1a6a00640 1 -- 172.21.15.47:0/1638269093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a81021e0 msgr2=0x7fc1a8199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1a6a00640 1 --2- 172.21.15.47:0/1638269093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a81021e0 0x7fc1a8199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.383+0000 7fc1a6a00640 1 -- 172.21.15.47:0/1638269093 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a81a1880 con 0x7fc1a8104b10 2024-03-21T11:27:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.384+0000 7fc1a6a00640 1 --2- 172.21.15.47:0/1638269093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1a8104b10 0x7fc1a819a1b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc19400e9c0 tx=0x7fc19400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.384+0000 7fc19fe00640 1 -- 172.21.15.47:0/1638269093 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19400cd80 con 0x7fc1a8104b10 2024-03-21T11:27:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.384+0000 7fc19fe00640 1 -- 172.21.15.47:0/1638269093 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc194005500 con 0x7fc1a8104b10 2024-03-21T11:27:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.384+0000 7fc1ae800640 1 -- 172.21.15.47:0/1638269093 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a81a1b70 con 0x7fc1a8104b10 2024-03-21T11:27:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.384+0000 7fc19fe00640 1 -- 172.21.15.47:0/1638269093 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194010690 con 0x7fc1a8104b10 2024-03-21T11:27:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.384+0000 7fc1ae800640 1 -- 172.21.15.47:0/1638269093 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a81a2110 con 0x7fc1a8104b10 2024-03-21T11:27:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.387+0000 7fc1b1475740 1 Processor -- start 2024-03-21T11:27:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.388+0000 7fc1b1475740 1 -- start start 2024-03-21T11:27:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.388+0000 7fc1b1475740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c3738f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.388+0000 7fc1b1475740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c37394370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.388+0000 7fc1b1475740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c37396e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.388+0000 7fc1b1475740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c372f4250 con 0x560c373949e0 2024-03-21T11:27:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.388+0000 7fc1b1475740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3729ca30 con 0x560c3738ee80 2024-03-21T11:27:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.388+0000 7fc1b1475740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c372f02f0 con 0x560c3738f870 2024-03-21T11:27:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.388+0000 7fc1a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c3738f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.388+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c37394370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.388+0000 7fc19fe00640 1 -- 172.21.15.47:0/1638269093 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc1940046c0 con 0x7fc1a8104b10 2024-03-21T11:27:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.388+0000 7fc1a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c3738f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58170/0 (socket says 172.21.15.47:58170) 2024-03-21T11:27:47.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.388+0000 7fc1a7e00640 1 -- 172.21.15.47:0/3104747926 learned_addr learned my addr 172.21.15.47:0/3104747926 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.389+0000 7fc1a7e00640 1 -- 172.21.15.47:0/3104747926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 msgr2=0x560c37394370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.389+0000 7fc1a7e00640 1 --2- 172.21.15.47:0/3104747926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c37394370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.389+0000 7fc1a7e00640 1 -- 172.21.15.47:0/3104747926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 msgr2=0x560c37396e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.389+0000 7fc1a7e00640 1 --2- 172.21.15.47:0/3104747926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c37396e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.389+0000 7fc1a7e00640 1 -- 172.21.15.47:0/3104747926 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c37287690 con 0x560c3738ee80 2024-03-21T11:27:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.389+0000 7fc1a7e00640 1 --2- 172.21.15.47:0/3104747926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c3738f2a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009fd0 tx=0x7fc1a00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ce9a1328907280f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:47.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.389+0000 7fc19cc00640 1 -- 172.21.15.47:0/3104747926 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0004d20 con 0x560c3738ee80 2024-03-21T11:27:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.389+0000 7fc19cc00640 1 -- 172.21.15.47:0/3104747926 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1a0004050 con 0x560c3738ee80 2024-03-21T11:27:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.389+0000 7fc19cc00640 1 -- 172.21.15.47:0/3104747926 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0005240 con 0x560c3738ee80 2024-03-21T11:27:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.390+0000 7fc1b1475740 1 -- 172.21.15.47:0/3104747926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 msgr2=0x560c3738f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.390+0000 7fc1b1475740 1 --2- 172.21.15.47:0/3104747926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c3738f2a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009fd0 tx=0x7fc1a00118d0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.390+0000 7fc19fe00640 1 --2- 172.21.15.47:0/1638269093 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc18c0a2fa0 0x7fc18c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.390+0000 7fc19fe00640 1 -- 172.21.15.47:0/1638269093 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc1940cfb30 con 0x7fc1a8104b10 2024-03-21T11:27:47.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.391+0000 7fc1a7e00640 1 --2- 172.21.15.47:0/1638269093 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc18c0a2fa0 0x7fc18c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.391+0000 7fc1b1475740 1 -- 172.21.15.47:0/3104747926 shutdown_connections 2024-03-21T11:27:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.391+0000 7fc1b1475740 1 --2- 172.21.15.47:0/3104747926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c37396e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.391+0000 7fc1b1475740 1 --2- 172.21.15.47:0/3104747926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c37394370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.391+0000 7fc1b1475740 1 --2- 172.21.15.47:0/3104747926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c3738f2a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.391+0000 7fc1b1475740 1 -- 172.21.15.47:0/3104747926 >> 172.21.15.47:0/3104747926 conn(0x560c37357200 msgr2=0x560c3738a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.391+0000 7fc1b1475740 1 -- 172.21.15.47:0/3104747926 shutdown_connections 2024-03-21T11:27:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.391+0000 7fc1b1475740 1 -- 172.21.15.47:0/3104747926 wait complete. 2024-03-21T11:27:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.392+0000 7fc1a7e00640 1 --2- 172.21.15.47:0/1638269093 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc18c0a2fa0 0x7fc18c0a5460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009de0 tx=0x7fc1a0000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.392+0000 7fc1b1475740 1 -- start start 2024-03-21T11:27:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.392+0000 7fc1b1475740 1 Processor -- start 2024-03-21T11:27:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1b1475740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c37398e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1b1475740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c373993a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1b1475740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c373a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1b1475740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c37397990 con 0x560c373949e0 2024-03-21T11:27:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1b1475740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c372afe80 con 0x560c3738ee80 2024-03-21T11:27:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1b1475740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c37397b10 con 0x560c3738f870 2024-03-21T11:27:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1b1475740 10 client.0 _ll_register_callbacks cb 0x560c372c2130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c373993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c37398e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c373993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40548/0 (socket says 172.21.15.47:40548) 2024-03-21T11:27:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1a6a00640 1 -- 172.21.15.47:0/1343275401 learned_addr learned my addr 172.21.15.47:0/1343275401 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1a7e00640 1 --2- 172.21.15.47:0/1343275401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c373a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1a6a00640 1 -- 172.21.15.47:0/1343275401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 msgr2=0x560c37398e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1a6a00640 1 --2- 172.21.15.47:0/1343275401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c37398e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1a6a00640 1 -- 172.21.15.47:0/1343275401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 msgr2=0x560c373a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1a6a00640 1 --2- 172.21.15.47:0/1343275401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c373a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1a6a00640 1 -- 172.21.15.47:0/1343275401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c370d27d0 con 0x560c3738f870 2024-03-21T11:27:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.393+0000 7fc1a7400640 1 --2- 172.21.15.47:0/1343275401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c37398e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.394+0000 7fc1a6a00640 1 --2- 172.21.15.47:0/1343275401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c373993a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc1940a4350 tx=0x7fc19409ba70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55f71e4a4df03fd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.394+0000 7fc191600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.394+0000 7fc191600640 1 -- 172.21.15.47:0/1343275401 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940bc070 con 0x560c3738f870 2024-03-21T11:27:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.394+0000 7fc191600640 1 -- 172.21.15.47:0/1343275401 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc1940a43f0 con 0x560c3738f870 2024-03-21T11:27:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.394+0000 7fc1b1475740 1 -- 172.21.15.47:0/1343275401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1a0004050 con 0x560c3738f870 2024-03-21T11:27:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.394+0000 7fc191600640 1 -- 172.21.15.47:0/1343275401 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc19409be70 con 0x560c3738f870 2024-03-21T11:27:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.394+0000 7fc1b1475740 10 client.26929 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.394+0000 7fc1b1475740 1 -- 172.21.15.47:0/1343275401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560c370083f0 con 0x560c3738f870 2024-03-21T11:27:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.394+0000 7fc191600640 1 -- 172.21.15.47:0/1343275401 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1a0004050 con 0x560c3738f870 2024-03-21T11:27:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.395+0000 7fc191600640 1 -- 172.21.15.47:0/1343275401 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc19409ad30 con 0x560c3738f870 2024-03-21T11:27:47.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.395+0000 7fc1b1475740 10 client.26929 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:47.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.395+0000 7fc1b1475740 10 client.26929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:47.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.395+0000 7fc1b1475740 20 client.26929 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:47.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.395+0000 7fc1b1475740 1 --2- 172.21.15.47:0/1343275401 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c373a4c80 0x560c373a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.395+0000 7fc1b1475740 4 client.26929 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:47.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.395+0000 7fc1b1475740 1 -- 172.21.15.47:0/1343275401 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c37301790 con 0x560c373a4c80 2024-03-21T11:27:47.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.395+0000 7fc1a7400640 1 --2- 172.21.15.47:0/1343275401 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c373a4c80 0x560c373a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.395+0000 7fc1a7400640 1 --2- 172.21.15.47:0/1343275401 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c373a4c80 0x560c373a7070 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7ae43aa45029863 server_cookie=64441a01d4fff806 in_seq=0 out_seq=0 2024-03-21T11:27:47.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.395+0000 7fc191600640 10 client.26929 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:47.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc191600640 1 -- 172.21.15.47:0/1343275401 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560c37301790 con 0x560c373a4c80 2024-03-21T11:27:47.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc191600640 10 client.26929 handle_command_reply: tid=0 2024-03-21T11:27:47.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc1b1475740 1 client.26929 shutdown 2024-03-21T11:27:47.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc1b1475740 1 -- 172.21.15.47:0/1343275401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 msgr2=0x560c373993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc1b1475740 1 --2- 172.21.15.47:0/1343275401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c373993a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc1940a4350 tx=0x7fc19409ba70 comp rx=0 tx=0).stop 2024-03-21T11:27:47.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc1b1475740 1 -- 172.21.15.47:0/1343275401 shutdown_connections 2024-03-21T11:27:47.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc1b1475740 1 --2- 172.21.15.47:0/1343275401 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c373a4c80 0x560c373a7070 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc1b1475740 1 --2- 172.21.15.47:0/1343275401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c373a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc1b1475740 1 --2- 172.21.15.47:0/1343275401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c373993a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc1b1475740 1 --2- 172.21.15.47:0/1343275401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c37398e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc191600640 0 client.26929 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:47.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc1b1475740 1 -- 172.21.15.47:0/1343275401 >> 172.21.15.47:0/1343275401 conn(0x560c37357200 msgr2=0x560c37389a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:47.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc1b1475740 1 -- 172.21.15.47:0/1343275401 shutdown_connections 2024-03-21T11:27:47.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc1b1475740 1 -- 172.21.15.47:0/1343275401 wait complete. 2024-03-21T11:27:47.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.397+0000 7fc1b1475740 20 client.26929 trim_cache size 0 max 16384 2024-03-21T11:27:47.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.417+0000 7fc1b1475740 1 Processor -- start 2024-03-21T11:27:47.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.417+0000 7fc1b1475740 1 -- start start 2024-03-21T11:27:47.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.417+0000 7fc1b1475740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738ee80 0x560c3738da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.417+0000 7fc1b1475740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738f870 0x560c373a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.417+0000 7fc1b1475740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c3738df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.417+0000 7fc1b1475740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a81ac310 con 0x560c373949e0 2024-03-21T11:27:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.417+0000 7fc1b1475740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a81adc90 con 0x560c3738f870 2024-03-21T11:27:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.417+0000 7fc1b1475740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a81ac970 con 0x560c3738ee80 2024-03-21T11:27:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.417+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738ee80 0x560c3738da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.417+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738ee80 0x560c3738da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40556/0 (socket says 172.21.15.47:40556) 2024-03-21T11:27:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.417+0000 7fc1a7400640 1 -- 172.21.15.47:0/3325420800 learned_addr learned my addr 172.21.15.47:0/3325420800 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.417+0000 7fc1a6a00640 1 --2- 172.21.15.47:0/3325420800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738f870 0x560c373a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.418+0000 7fc1a7e00640 1 --2- 172.21.15.47:0/3325420800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c3738df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.418+0000 7fc1a7400640 1 -- 172.21.15.47:0/3325420800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738f870 msgr2=0x560c373a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.418+0000 7fc1a7400640 1 --2- 172.21.15.47:0/3325420800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738f870 0x560c373a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.418+0000 7fc1a7400640 1 -- 172.21.15.47:0/3325420800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 msgr2=0x560c3738df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.418+0000 7fc1a7400640 1 --2- 172.21.15.47:0/3325420800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c3738df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.418+0000 7fc1a7400640 1 -- 172.21.15.47:0/3325420800 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c37398c50 con 0x560c3738ee80 2024-03-21T11:27:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.418+0000 7fc1a7e00640 1 --2- 172.21.15.47:0/3325420800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c3738df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:27:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.418+0000 7fc1a6a00640 1 --2- 172.21.15.47:0/3325420800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738f870 0x560c373a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.418+0000 7fc1a7400640 1 --2- 172.21.15.47:0/3325420800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738ee80 0x560c3738da30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc19802cfd0 tx=0x7fc198005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46128b36be9a6fba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.418+0000 7fc193e00640 1 -- 172.21.15.47:0/3325420800 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198004240 con 0x560c3738ee80 2024-03-21T11:27:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.418+0000 7fc193e00640 1 -- 172.21.15.47:0/3325420800 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc19804b070 con 0x560c3738ee80 2024-03-21T11:27:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.418+0000 7fc193e00640 1 -- 172.21.15.47:0/3325420800 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198004b80 con 0x560c3738ee80 2024-03-21T11:27:47.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.419+0000 7fc1b1475740 1 -- 172.21.15.47:0/3325420800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738ee80 msgr2=0x560c3738da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.419+0000 7fc1b1475740 1 --2- 172.21.15.47:0/3325420800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738ee80 0x560c3738da30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc19802cfd0 tx=0x7fc198005fd0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.419+0000 7fc1b1475740 1 -- 172.21.15.47:0/3325420800 shutdown_connections 2024-03-21T11:27:47.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.419+0000 7fc1b1475740 1 --2- 172.21.15.47:0/3325420800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c3738df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.419+0000 7fc1b1475740 1 --2- 172.21.15.47:0/3325420800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738f870 0x560c373a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.419+0000 7fc1b1475740 1 --2- 172.21.15.47:0/3325420800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738ee80 0x560c3738da30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.419+0000 7fc1b1475740 1 -- 172.21.15.47:0/3325420800 >> 172.21.15.47:0/3325420800 conn(0x560c37357200 msgr2=0x560c37389a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.420+0000 7fc1b1475740 1 -- 172.21.15.47:0/3325420800 shutdown_connections 2024-03-21T11:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.420+0000 7fc1b1475740 1 -- 172.21.15.47:0/3325420800 wait complete. 2024-03-21T11:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.420+0000 7fc1b1475740 1 -- start start 2024-03-21T11:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.420+0000 7fc1b1475740 1 Processor -- start 2024-03-21T11:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1b1475740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c3739e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1b1475740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c3739e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1b1475740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c3739f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1b1475740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a81ad630 con 0x560c373949e0 2024-03-21T11:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1b1475740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a81acfd0 con 0x560c3738ee80 2024-03-21T11:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1b1475740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a81aa4d0 con 0x560c3738f870 2024-03-21T11:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1b1475740 10 client.0 _ll_register_callbacks cb 0x7fc194009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c3739e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c3739e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c3739e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58196/0 (socket says 172.21.15.47:58196) 2024-03-21T11:27:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1a7400640 1 -- 172.21.15.47:0/392462647 learned_addr learned my addr 172.21.15.47:0/392462647 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1a7e00640 1 --2- 172.21.15.47:0/392462647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c3739f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1a6a00640 1 -- 172.21.15.47:0/392462647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 msgr2=0x560c3739e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1a6a00640 1 --2- 172.21.15.47:0/392462647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c3739e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1a6a00640 1 -- 172.21.15.47:0/392462647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 msgr2=0x560c3739f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1a6a00640 1 --2- 172.21.15.47:0/392462647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c3739f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.421+0000 7fc1a6a00640 1 -- 172.21.15.47:0/392462647 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c36fc80f0 con 0x560c3738f870 2024-03-21T11:27:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.422+0000 7fc1a6a00640 1 --2- 172.21.15.47:0/392462647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c3739e570 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc19409be00 tx=0x7fc194099e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf7a79e385c9b982 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.422+0000 7fc191600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.422+0000 7fc191600640 1 -- 172.21.15.47:0/392462647 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194099fb0 con 0x560c3738f870 2024-03-21T11:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.422+0000 7fc1b1475740 1 -- 172.21.15.47:0/392462647 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc19804b070 con 0x560c3738f870 2024-03-21T11:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.422+0000 7fc191600640 1 -- 172.21.15.47:0/392462647 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc1940a45f0 con 0x560c3738f870 2024-03-21T11:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.422+0000 7fc191600640 1 -- 172.21.15.47:0/392462647 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc19409bc40 con 0x560c3738f870 2024-03-21T11:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.422+0000 7fc1b1475740 10 client.26935 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.422+0000 7fc1b1475740 1 -- 172.21.15.47:0/392462647 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560c36fc80f0 con 0x560c3738f870 2024-03-21T11:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.422+0000 7fc191600640 1 -- 172.21.15.47:0/392462647 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc19804b070 con 0x560c3738f870 2024-03-21T11:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.422+0000 7fc191600640 1 -- 172.21.15.47:0/392462647 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc19409b2c0 con 0x560c3738f870 2024-03-21T11:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.422+0000 7fc1b1475740 10 client.26935 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.422+0000 7fc1b1475740 10 client.26935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.422+0000 7fc1b1475740 20 client.26935 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.422+0000 7fc1b1475740 1 --2- 172.21.15.47:0/392462647 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c37400d50 0x560c37403170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.423+0000 7fc1b1475740 4 client.26935 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.423+0000 7fc1b1475740 1 -- 172.21.15.47:0/392462647 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560c372fbae0 con 0x560c37400d50 2024-03-21T11:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.423+0000 7fc1a7400640 1 --2- 172.21.15.47:0/392462647 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c37400d50 0x560c37403170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.423+0000 7fc1a7400640 1 --2- 172.21.15.47:0/392462647 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c37400d50 0x560c37403170 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89b5b037c7b3af57 server_cookie=a3d3b1a52f1b7dee in_seq=0 out_seq=0 2024-03-21T11:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.423+0000 7fc191600640 10 client.26935 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.423+0000 7fc191600640 1 -- 172.21.15.47:0/392462647 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560c372fbae0 con 0x560c37400d50 2024-03-21T11:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.423+0000 7fc191600640 10 client.26935 handle_command_reply: tid=0 2024-03-21T11:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.423+0000 7fc1b1475740 1 client.26935 shutdown 2024-03-21T11:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.424+0000 7fc1b1475740 1 -- 172.21.15.47:0/392462647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 msgr2=0x560c3739e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.424+0000 7fc1b1475740 1 --2- 172.21.15.47:0/392462647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c3739e570 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc19409be00 tx=0x7fc194099e80 comp rx=0 tx=0).stop 2024-03-21T11:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.424+0000 7fc1b1475740 1 -- 172.21.15.47:0/392462647 shutdown_connections 2024-03-21T11:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.424+0000 7fc1b1475740 1 --2- 172.21.15.47:0/392462647 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c37400d50 0x560c37403170 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.424+0000 7fc1b1475740 1 --2- 172.21.15.47:0/392462647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c373949e0 0x560c3739f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.424+0000 7fc1b1475740 1 --2- 172.21.15.47:0/392462647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c3738f870 0x560c3739e570 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.424+0000 7fc1b1475740 1 --2- 172.21.15.47:0/392462647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c3738ee80 0x560c3739e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.424+0000 7fc191600640 0 client.26935 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.424+0000 7fc1b1475740 1 -- 172.21.15.47:0/392462647 >> 172.21.15.47:0/392462647 conn(0x560c37357200 msgr2=0x560c37389a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.424+0000 7fc1b1475740 1 -- 172.21.15.47:0/392462647 shutdown_connections 2024-03-21T11:27:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.424+0000 7fc1b1475740 1 -- 172.21.15.47:0/392462647 wait complete. 2024-03-21T11:27:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.424+0000 7fc1b1475740 20 client.26935 trim_cache size 0 max 16384 2024-03-21T11:27:47.430 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:27:47.430 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:27:47.430 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:27:47.430 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:27:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.425+0000 7fc1ae800640 1 -- 172.21.15.47:0/1638269093 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc18c0a2fa0 msgr2=0x7fc18c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.425+0000 7fc1ae800640 1 --2- 172.21.15.47:0/1638269093 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc18c0a2fa0 0x7fc18c0a5460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009de0 tx=0x7fc1a0000f40 comp rx=0 tx=0).stop 2024-03-21T11:27:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.425+0000 7fc1ae800640 1 -- 172.21.15.47:0/1638269093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1a8104b10 msgr2=0x7fc1a819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.425+0000 7fc1ae800640 1 --2- 172.21.15.47:0/1638269093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1a8104b10 0x7fc1a819a1b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc19400e9c0 tx=0x7fc19400ee80 comp rx=0 tx=0).stop 2024-03-21T11:27:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.425+0000 7fc1ae800640 1 -- 172.21.15.47:0/1638269093 shutdown_connections 2024-03-21T11:27:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.425+0000 7fc1ae800640 1 --2- 172.21.15.47:0/1638269093 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc18c0a2fa0 0x7fc18c0a5460 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.425+0000 7fc1ae800640 1 --2- 172.21.15.47:0/1638269093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1a8107440 0x7fc1a81a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.425+0000 7fc1ae800640 1 --2- 172.21.15.47:0/1638269093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1a8104b10 0x7fc1a819a1b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.425+0000 7fc1ae800640 1 --2- 172.21.15.47:0/1638269093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1a81021e0 0x7fc1a8199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.425+0000 7fc1ae800640 1 -- 172.21.15.47:0/1638269093 >> 172.21.15.47:0/1638269093 conn(0x7fc1a80fbb30 msgr2=0x7fc1a8101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:47.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.425+0000 7fc1ae800640 1 -- 172.21.15.47:0/1638269093 shutdown_connections 2024-03-21T11:27:47.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.425+0000 7fc1ae800640 1 -- 172.21.15.47:0/1638269093 wait complete. 2024-03-21T11:27:47.444 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:27:47.444 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:27:47.505 INFO:tasks.workunit.client.0.smithi047.stdout: 1 241832 8.07 MB/sec execute 31 sec latency 10.061 ms 2024-03-21T11:27:47.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.670+0000 7f02f9a00640 1 -- 172.21.15.47:0/1053666558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f4104a60 msgr2=0x7f02f4106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.670+0000 7f02f9a00640 1 --2- 172.21.15.47:0/1053666558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f4104a60 0x7f02f4106e50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009860 tx=0x7f02e002bb70 comp rx=0 tx=0).stop 2024-03-21T11:27:47.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.670+0000 7f02f9a00640 1 -- 172.21.15.47:0/1053666558 shutdown_connections 2024-03-21T11:27:47.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.670+0000 7f02f9a00640 1 --2- 172.21.15.47:0/1053666558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02f4107390 0x7f02f41097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.670+0000 7f02f9a00640 1 --2- 172.21.15.47:0/1053666558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f4104a60 0x7f02f4106e50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.670+0000 7f02f9a00640 1 --2- 172.21.15.47:0/1053666558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4102130 0x7f02f4104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.670+0000 7f02f9a00640 1 -- 172.21.15.47:0/1053666558 >> 172.21.15.47:0/1053666558 conn(0x7f02f40fbb20 msgr2=0x7f02f40fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:47.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.670+0000 7f02f9a00640 1 -- 172.21.15.47:0/1053666558 shutdown_connections 2024-03-21T11:27:47.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.670+0000 7f02f9a00640 1 -- 172.21.15.47:0/1053666558 wait complete. 2024-03-21T11:27:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.671+0000 7f02f9a00640 1 Processor -- start 2024-03-21T11:27:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.672+0000 7f02f9a00640 1 -- start start 2024-03-21T11:27:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.672+0000 7f02f9a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f4102130 0x7f02f41979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.672+0000 7f02f9a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02f4104a60 0x7f02f4197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.672+0000 7f02f9a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4107390 0x7f02f419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.672+0000 7f02f9a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f410d320 con 0x7f02f4104a60 2024-03-21T11:27:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.672+0000 7f02f9a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f410d1a0 con 0x7f02f4102130 2024-03-21T11:27:47.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.672+0000 7f02f9a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f410d4a0 con 0x7f02f4107390 2024-03-21T11:27:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.672+0000 7f02f2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f4102130 0x7f02f41979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.672+0000 7f02f2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f4102130 0x7f02f41979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58204/0 (socket says 172.21.15.47:58204) 2024-03-21T11:27:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.672+0000 7f02f2a00640 1 -- 172.21.15.47:0/1095409361 learned_addr learned my addr 172.21.15.47:0/1095409361 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.672+0000 7f02f3400640 1 --2- 172.21.15.47:0/1095409361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4107390 0x7f02f419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.672+0000 7f02f2000640 1 --2- 172.21.15.47:0/1095409361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02f4104a60 0x7f02f4197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.673+0000 7f02f2a00640 1 -- 172.21.15.47:0/1095409361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4107390 msgr2=0x7f02f419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.673+0000 7f02f2a00640 1 --2- 172.21.15.47:0/1095409361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4107390 0x7f02f419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.673+0000 7f02f2a00640 1 -- 172.21.15.47:0/1095409361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02f4104a60 msgr2=0x7f02f4197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.673+0000 7f02f2a00640 1 --2- 172.21.15.47:0/1095409361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02f4104a60 0x7f02f4197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.673+0000 7f02f2a00640 1 -- 172.21.15.47:0/1095409361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f419f5e0 con 0x7f02f4102130 2024-03-21T11:27:47.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.673+0000 7f02f2a00640 1 --2- 172.21.15.47:0/1095409361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f4102130 0x7f02f41979c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e9c0 tx=0x7f02e400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:47.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.673+0000 7f02eb400640 1 -- 172.21.15.47:0/1095409361 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e400ccc0 con 0x7f02f4102130 2024-03-21T11:27:47.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.673+0000 7f02eb400640 1 -- 172.21.15.47:0/1095409361 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02e4005500 con 0x7f02f4102130 2024-03-21T11:27:47.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.673+0000 7f02f9a00640 1 -- 172.21.15.47:0/1095409361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f419f8d0 con 0x7f02f4102130 2024-03-21T11:27:47.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.674+0000 7f02eb400640 1 -- 172.21.15.47:0/1095409361 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e4010430 con 0x7f02f4102130 2024-03-21T11:27:47.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.674+0000 7f02f9a00640 1 -- 172.21.15.47:0/1095409361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f419fe40 con 0x7f02f4102130 2024-03-21T11:27:47.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.675+0000 7f02f9a00640 1 -- 172.21.15.47:0/1095409361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f40fd610 con 0x7f02f4102130 2024-03-21T11:27:47.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.677+0000 7f02eb400640 1 -- 172.21.15.47:0/1095409361 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f02e40046c0 con 0x7f02f4102130 2024-03-21T11:27:47.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.679+0000 7f02eb400640 1 --2- 172.21.15.47:0/1095409361 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f02d80a2fa0 0x7f02d80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:47.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.679+0000 7f02eb400640 1 -- 172.21.15.47:0/1095409361 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f02e40cfa30 con 0x7f02f4102130 2024-03-21T11:27:47.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.679+0000 7f02f2000640 1 --2- 172.21.15.47:0/1095409361 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f02d80a2fa0 0x7f02d80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:47.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.680+0000 7f02f2000640 1 --2- 172.21.15.47:0/1095409361 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f02d80a2fa0 0x7f02d80a5460 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f02e00349e0 tx=0x7f02e0034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:47.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.684+0000 7f02eb400640 1 -- 172.21.15.47:0/1095409361 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f02e409c470 con 0x7f02f4102130 2024-03-21T11:27:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.983+0000 7f02f9a00640 1 -- 172.21.15.47:0/1095409361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02f402d070 con 0x7f02f4102130 2024-03-21T11:27:47.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.984+0000 7f02eb400640 1 -- 172.21.15.47:0/1095409361 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f02e409bd70 con 0x7f02f4102130 2024-03-21T11:27:47.985 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:27:47.985 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:27:47.986 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:27:47.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.987+0000 7f02f9a00640 1 -- 172.21.15.47:0/1095409361 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f02d80a2fa0 msgr2=0x7f02d80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.987+0000 7f02f9a00640 1 --2- 172.21.15.47:0/1095409361 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f02d80a2fa0 0x7f02d80a5460 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f02e00349e0 tx=0x7f02e0034ab0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.987+0000 7f02f9a00640 1 -- 172.21.15.47:0/1095409361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f4102130 msgr2=0x7f02f41979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:47.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.987+0000 7f02f9a00640 1 --2- 172.21.15.47:0/1095409361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f4102130 0x7f02f41979c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e9c0 tx=0x7f02e400ee80 comp rx=0 tx=0).stop 2024-03-21T11:27:47.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.987+0000 7f02f9a00640 1 -- 172.21.15.47:0/1095409361 shutdown_connections 2024-03-21T11:27:47.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.987+0000 7f02f9a00640 1 --2- 172.21.15.47:0/1095409361 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f02d80a2fa0 0x7f02d80a5460 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.987+0000 7f02f9a00640 1 --2- 172.21.15.47:0/1095409361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02f4107390 0x7f02f419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.987+0000 7f02f9a00640 1 --2- 172.21.15.47:0/1095409361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02f4104a60 0x7f02f4197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.987+0000 7f02f9a00640 1 --2- 172.21.15.47:0/1095409361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02f4102130 0x7f02f41979c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:47.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.987+0000 7f02f9a00640 1 -- 172.21.15.47:0/1095409361 >> 172.21.15.47:0/1095409361 conn(0x7f02f40fbb20 msgr2=0x7f02f4102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:47.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.987+0000 7f02f9a00640 1 -- 172.21.15.47:0/1095409361 shutdown_connections 2024-03-21T11:27:47.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:47.987+0000 7f02f9a00640 1 -- 172.21.15.47:0/1095409361 wait complete. 2024-03-21T11:27:48.008 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:48.008 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:27:48.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:27:48.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:27:48.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:27:48.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:47 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:27:48.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:47 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:48.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:47 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:27:48.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:47 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:48.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:47 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:27:48.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:27:48.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:27:48.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:27:48.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:47 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:27:48.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:47 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:48.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:47 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:27:48.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:47 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:48.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:47 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:27:48.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:27:48.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:27:48.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:27:48.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:47 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:27:48.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:47 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:48.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:47 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:27:48.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:47 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:48.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:47 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:27:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.252+0000 7fad5ba00640 1 -- 172.21.15.47:0/2620108914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54104ad0 msgr2=0x7fad54104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.252+0000 7fad5ba00640 1 --2- 172.21.15.47:0/2620108914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54104ad0 0x7fad54104f50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fad40009900 tx=0x7fad4002bc40 comp rx=0 tx=0).stop 2024-03-21T11:27:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.252+0000 7fad5ba00640 1 -- 172.21.15.47:0/2620108914 shutdown_connections 2024-03-21T11:27:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.252+0000 7fad5ba00640 1 --2- 172.21.15.47:0/2620108914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad54105490 0x7fad5410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.252+0000 7fad5ba00640 1 --2- 172.21.15.47:0/2620108914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54104ad0 0x7fad54104f50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.252+0000 7fad5ba00640 1 --2- 172.21.15.47:0/2620108914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad541038e0 0x7fad54103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.252+0000 7fad5ba00640 1 -- 172.21.15.47:0/2620108914 >> 172.21.15.47:0/2620108914 conn(0x7fad540fee50 msgr2=0x7fad54101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.252+0000 7fad5ba00640 1 -- 172.21.15.47:0/2620108914 shutdown_connections 2024-03-21T11:27:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.252+0000 7fad5ba00640 1 -- 172.21.15.47:0/2620108914 wait complete. 2024-03-21T11:27:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.253+0000 7fad5ba00640 1 Processor -- start 2024-03-21T11:27:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.254+0000 7fad5ba00640 1 -- start start 2024-03-21T11:27:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.254+0000 7fad5ba00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad541038e0 0x7fad54199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.254+0000 7fad5ba00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad54104ad0 0x7fad5419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.254+0000 7fad5ba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54105490 0x7fad541a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.254+0000 7fad5ba00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5410f5b0 con 0x7fad541038e0 2024-03-21T11:27:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.254+0000 7fad5ba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5410f430 con 0x7fad54105490 2024-03-21T11:27:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.254+0000 7fad5ba00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5410f730 con 0x7fad54104ad0 2024-03-21T11:27:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.254+0000 7fad59000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad541038e0 0x7fad54199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.254+0000 7fad53e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad54104ad0 0x7fad5419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.254+0000 7fad59000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad541038e0 0x7fad54199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40996/0 (socket says 172.21.15.47:40996) 2024-03-21T11:27:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.254+0000 7fad59000640 1 -- 172.21.15.47:0/2323115042 learned_addr learned my addr 172.21.15.47:0/2323115042 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.254+0000 7fad59a00640 1 --2- 172.21.15.47:0/2323115042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54105490 0x7fad541a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.255+0000 7fad59000640 1 -- 172.21.15.47:0/2323115042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad54104ad0 msgr2=0x7fad5419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.255+0000 7fad59000640 1 --2- 172.21.15.47:0/2323115042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad54104ad0 0x7fad5419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.255+0000 7fad59000640 1 -- 172.21.15.47:0/2323115042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54105490 msgr2=0x7fad541a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.255+0000 7fad59000640 1 --2- 172.21.15.47:0/2323115042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54105490 0x7fad541a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.255+0000 7fad59000640 1 -- 172.21.15.47:0/2323115042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad541a17f0 con 0x7fad541038e0 2024-03-21T11:27:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.255+0000 7fad53e00640 1 --2- 172.21.15.47:0/2323115042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad54104ad0 0x7fad5419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.255+0000 7fad59000640 1 --2- 172.21.15.47:0/2323115042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad541038e0 0x7fad54199bd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fad4800d770 tx=0x7fad4800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.255+0000 7fad51600640 1 -- 172.21.15.47:0/2323115042 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48017070 con 0x7fad541038e0 2024-03-21T11:27:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.255+0000 7fad5ba00640 1 -- 172.21.15.47:0/2323115042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad541a1ae0 con 0x7fad541038e0 2024-03-21T11:27:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.256+0000 7fad51600640 1 -- 172.21.15.47:0/2323115042 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad48009c30 con 0x7fad541038e0 2024-03-21T11:27:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.256+0000 7fad51600640 1 -- 172.21.15.47:0/2323115042 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48003dd0 con 0x7fad541038e0 2024-03-21T11:27:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.256+0000 7fad5ba00640 1 -- 172.21.15.47:0/2323115042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad541a2050 con 0x7fad541038e0 2024-03-21T11:27:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.259+0000 7fad5e79a740 1 Processor -- start 2024-03-21T11:27:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.259+0000 7fad51600640 1 -- 172.21.15.47:0/2323115042 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fad48004000 con 0x7fad541038e0 2024-03-21T11:27:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.259+0000 7fad5e79a740 1 -- start start 2024-03-21T11:27:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad5e79a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fb810 0x5614067fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad5e79a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fc9e0 0x5614067fce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad5e79a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fd3a0 0x561406803c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad5e79a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614067602c0 con 0x5614067fc9e0 2024-03-21T11:27:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad5e79a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561406708c90 con 0x5614067fb810 2024-03-21T11:27:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad5e79a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140675c360 con 0x5614067fd3a0 2024-03-21T11:27:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad59a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fb810 0x5614067fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad53e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fd3a0 0x561406803c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad59a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fb810 0x5614067fbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58236/0 (socket says 172.21.15.47:58236) 2024-03-21T11:27:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad59a00640 1 -- 172.21.15.47:0/2302756217 learned_addr learned my addr 172.21.15.47:0/2302756217 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad59000640 1 --2- 172.21.15.47:0/2302756217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fc9e0 0x5614067fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad59a00640 1 -- 172.21.15.47:0/2302756217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fd3a0 msgr2=0x561406803c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad59a00640 1 --2- 172.21.15.47:0/2302756217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fd3a0 0x561406803c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad59a00640 1 -- 172.21.15.47:0/2302756217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fc9e0 msgr2=0x5614067fce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad59a00640 1 --2- 172.21.15.47:0/2302756217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fc9e0 0x5614067fce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.260+0000 7fad59a00640 1 -- 172.21.15.47:0/2302756217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56140672d080 con 0x5614067fb810 2024-03-21T11:27:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.261+0000 7fad59000640 1 --2- 172.21.15.47:0/2302756217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fc9e0 0x5614067fce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:27:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.261+0000 7fad59a00640 1 --2- 172.21.15.47:0/2302756217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fb810 0x5614067fbc30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009fd0 tx=0x7fad4c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad2489e53f22c9fc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.261+0000 7fad51600640 1 --2- 172.21.15.47:0/2323115042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad3c0a2fa0 0x7fad3c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.261+0000 7fad46000640 1 -- 172.21.15.47:0/2302756217 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c004d20 con 0x5614067fb810 2024-03-21T11:27:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.261+0000 7fad46000640 1 -- 172.21.15.47:0/2302756217 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad4c004050 con 0x5614067fb810 2024-03-21T11:27:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.261+0000 7fad46000640 1 -- 172.21.15.47:0/2302756217 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c005260 con 0x5614067fb810 2024-03-21T11:27:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.261+0000 7fad51600640 1 -- 172.21.15.47:0/2323115042 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fad480cec40 con 0x7fad541038e0 2024-03-21T11:27:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.261+0000 7fad59000640 1 --2- 172.21.15.47:0/2323115042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad3c0a2fa0 0x7fad3c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.262+0000 7fad5e79a740 1 -- 172.21.15.47:0/2302756217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fb810 msgr2=0x5614067fbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.262+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2302756217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fb810 0x5614067fbc30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009fd0 tx=0x7fad4c0118d0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.262+0000 7fad59000640 1 --2- 172.21.15.47:0/2323115042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad3c0a2fa0 0x7fad3c0a5460 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fad480cc3b0 tx=0x7fad48004590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.262+0000 7fad5e79a740 1 -- 172.21.15.47:0/2302756217 shutdown_connections 2024-03-21T11:27:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.262+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2302756217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fd3a0 0x561406803c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.262+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2302756217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fc9e0 0x5614067fce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.262+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2302756217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fb810 0x5614067fbc30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.262+0000 7fad5e79a740 1 -- 172.21.15.47:0/2302756217 >> 172.21.15.47:0/2302756217 conn(0x5614067f6c00 msgr2=0x5614067f7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.262+0000 7fad5e79a740 1 -- 172.21.15.47:0/2302756217 shutdown_connections 2024-03-21T11:27:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.262+0000 7fad5e79a740 1 -- 172.21.15.47:0/2302756217 wait complete. 2024-03-21T11:27:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.263+0000 7fad5e79a740 1 -- start start 2024-03-21T11:27:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.263+0000 7fad5e79a740 1 Processor -- start 2024-03-21T11:27:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.263+0000 7fad5e79a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fb810 0x561406805ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad5e79a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x561406806020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad5e79a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fd3a0 0x56140680cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad5e79a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614068043b0 con 0x5614067fd3a0 2024-03-21T11:27:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad5e79a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140671c0d0 con 0x5614067fc9e0 2024-03-21T11:27:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad5e79a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561406804530 con 0x5614067fb810 2024-03-21T11:27:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad5e79a740 10 client.0 _ll_register_callbacks cb 0x561406720e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad59a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fb810 0x561406805ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x561406806020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad59a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fb810 0x561406805ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40612/0 (socket says 172.21.15.47:40612) 2024-03-21T11:27:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad59a00640 1 -- 172.21.15.47:0/2085726985 learned_addr learned my addr 172.21.15.47:0/2085726985 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad59a00640 1 --2- 172.21.15.47:0/2085726985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fd3a0 0x56140680cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad53e00640 1 -- 172.21.15.47:0/2085726985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fb810 msgr2=0x561406805ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad53e00640 1 --2- 172.21.15.47:0/2085726985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fb810 0x561406805ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad53e00640 1 -- 172.21.15.47:0/2085726985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fd3a0 msgr2=0x56140680cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad53e00640 1 --2- 172.21.15.47:0/2085726985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fd3a0 0x56140680cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.264+0000 7fad53e00640 1 -- 172.21.15.47:0/2085726985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614064743f0 con 0x5614067fc9e0 2024-03-21T11:27:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.265+0000 7fad53e00640 1 --2- 172.21.15.47:0/2085726985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x561406806020 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fad40036040 tx=0x7fad400048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=451666026eae11b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.265+0000 7fad3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.265+0000 7fad3aa00640 1 -- 172.21.15.47:0/2085726985 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40005d50 con 0x5614067fc9e0 2024-03-21T11:27:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.265+0000 7fad3aa00640 1 -- 172.21.15.47:0/2085726985 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fad400050f0 con 0x5614067fc9e0 2024-03-21T11:27:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.265+0000 7fad5e79a740 1 -- 172.21.15.47:0/2085726985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad4c004050 con 0x5614067fc9e0 2024-03-21T11:27:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.266+0000 7fad3aa00640 1 -- 172.21.15.47:0/2085726985 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad4002cd40 con 0x5614067fc9e0 2024-03-21T11:27:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.266+0000 7fad5e79a740 10 client.26939 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.266+0000 7fad5e79a740 1 -- 172.21.15.47:0/2085726985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56140653e7d0 con 0x5614067fc9e0 2024-03-21T11:27:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.266+0000 7fad3aa00640 1 -- 172.21.15.47:0/2085726985 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad4c004050 con 0x5614067fc9e0 2024-03-21T11:27:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.266+0000 7fad3aa00640 1 -- 172.21.15.47:0/2085726985 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fad40046070 con 0x5614067fc9e0 2024-03-21T11:27:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.266+0000 7fad5e79a740 10 client.26939 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.266+0000 7fad5e79a740 10 client.26939 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:27:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.266+0000 7fad5e79a740 20 client.26939 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.266+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2085726985 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5614068113f0 0x561406813810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.266+0000 7fad5e79a740 4 client.26939 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.266+0000 7fad5e79a740 1 -- 172.21.15.47:0/2085726985 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56140676d800 con 0x5614068113f0 2024-03-21T11:27:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.266+0000 7fad59a00640 1 --2- 172.21.15.47:0/2085726985 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5614068113f0 0x561406813810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.267+0000 7fad59a00640 1 --2- 172.21.15.47:0/2085726985 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5614068113f0 0x561406813810 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b484773e775239ec server_cookie=7445f75ce9cfd765 in_seq=0 out_seq=0 2024-03-21T11:27:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.267+0000 7fad3aa00640 10 client.26939 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:27:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.268+0000 7fad3aa00640 1 -- 172.21.15.47:0/2085726985 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56140676d800 con 0x5614068113f0 2024-03-21T11:27:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.268+0000 7fad3aa00640 10 client.26939 handle_command_reply: tid=0 2024-03-21T11:27:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.268+0000 7fad5e79a740 1 client.26939 shutdown 2024-03-21T11:27:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.268+0000 7fad5e79a740 1 -- 172.21.15.47:0/2085726985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 msgr2=0x561406806020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.268+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2085726985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x561406806020 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fad40036040 tx=0x7fad400048f0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.268+0000 7fad5e79a740 1 -- 172.21.15.47:0/2085726985 shutdown_connections 2024-03-21T11:27:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.268+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2085726985 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5614068113f0 0x561406813810 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.268+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2085726985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fd3a0 0x56140680cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.268+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2085726985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x561406806020 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.268+0000 7fad3aa00640 0 client.26939 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:27:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.268+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2085726985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fb810 0x561406805ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.268+0000 7fad5e79a740 1 -- 172.21.15.47:0/2085726985 >> 172.21.15.47:0/2085726985 conn(0x5614067f6c00 msgr2=0x5614067f5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.269+0000 7fad5e79a740 1 -- 172.21.15.47:0/2085726985 shutdown_connections 2024-03-21T11:27:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.269+0000 7fad5e79a740 1 -- 172.21.15.47:0/2085726985 wait complete. 2024-03-21T11:27:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.269+0000 7fad5e79a740 20 client.26939 trim_cache size 0 max 16384 2024-03-21T11:27:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.285+0000 7fad5e79a740 1 Processor -- start 2024-03-21T11:27:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.285+0000 7fad5e79a740 1 -- start start 2024-03-21T11:27:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.285+0000 7fad5e79a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fb810 0x5614067ffc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.285+0000 7fad5e79a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x56140680cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.285+0000 7fad5e79a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fd3a0 0x561406800150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.285+0000 7fad5e79a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad541ac360 con 0x5614067fb810 2024-03-21T11:27:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.285+0000 7fad5e79a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad541adce0 con 0x5614067fc9e0 2024-03-21T11:27:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.285+0000 7fad5e79a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad541ac9c0 con 0x5614067fd3a0 2024-03-21T11:27:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.286+0000 7fad59a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fb810 0x5614067ffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.286+0000 7fad53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x56140680cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.286+0000 7fad59a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fb810 0x5614067ffc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41030/0 (socket says 172.21.15.47:41030) 2024-03-21T11:27:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.286+0000 7fad53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x56140680cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58250/0 (socket says 172.21.15.47:58250) 2024-03-21T11:27:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.286+0000 7fad59a00640 1 -- 172.21.15.47:0/2529135319 learned_addr learned my addr 172.21.15.47:0/2529135319 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.286+0000 7fad59a00640 1 --2- 172.21.15.47:0/2529135319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fd3a0 0x561406800150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.286+0000 7fad53e00640 1 -- 172.21.15.47:0/2529135319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fd3a0 msgr2=0x561406800150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.286+0000 7fad53e00640 1 --2- 172.21.15.47:0/2529135319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fd3a0 0x561406800150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.286+0000 7fad53e00640 1 -- 172.21.15.47:0/2529135319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fb810 msgr2=0x5614067ffc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.286+0000 7fad53e00640 1 --2- 172.21.15.47:0/2529135319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fb810 0x5614067ffc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.286+0000 7fad53e00640 1 -- 172.21.15.47:0/2529135319 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614068046b0 con 0x5614067fc9e0 2024-03-21T11:27:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.286+0000 7fad59a00640 1 --2- 172.21.15.47:0/2529135319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fb810 0x5614067ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.286+0000 7fad53e00640 1 --2- 172.21.15.47:0/2529135319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x56140680cfb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fad4002c8f0 tx=0x7fad400058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49bc8cf74ca476a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.287+0000 7fad45600640 1 -- 172.21.15.47:0/2529135319 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4002cd40 con 0x5614067fc9e0 2024-03-21T11:27:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.287+0000 7fad45600640 1 -- 172.21.15.47:0/2529135319 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad40004cf0 con 0x5614067fc9e0 2024-03-21T11:27:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.287+0000 7fad45600640 1 -- 172.21.15.47:0/2529135319 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40005390 con 0x5614067fc9e0 2024-03-21T11:27:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.288+0000 7fad5e79a740 1 -- 172.21.15.47:0/2529135319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 msgr2=0x56140680cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.288+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2529135319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x56140680cfb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fad4002c8f0 tx=0x7fad400058d0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.288+0000 7fad5e79a740 1 -- 172.21.15.47:0/2529135319 shutdown_connections 2024-03-21T11:27:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.288+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2529135319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fd3a0 0x561406800150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.288+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2529135319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x56140680cfb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.288+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2529135319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fb810 0x5614067ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.288+0000 7fad5e79a740 1 -- 172.21.15.47:0/2529135319 >> 172.21.15.47:0/2529135319 conn(0x5614067f6c00 msgr2=0x5614067fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.288+0000 7fad5e79a740 1 -- 172.21.15.47:0/2529135319 shutdown_connections 2024-03-21T11:27:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.288+0000 7fad5e79a740 1 -- 172.21.15.47:0/2529135319 wait complete. 2024-03-21T11:27:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.288+0000 7fad5e79a740 1 -- start start 2024-03-21T11:27:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.288+0000 7fad5e79a740 1 Processor -- start 2024-03-21T11:27:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.289+0000 7fad5e79a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fb810 0x56140680b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.289+0000 7fad5e79a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x56140680b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.289+0000 7fad5e79a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fd3a0 0x56140680c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.289+0000 7fad5e79a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad541ad680 con 0x5614067fd3a0 2024-03-21T11:27:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.289+0000 7fad5e79a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad541ad020 con 0x5614067fc9e0 2024-03-21T11:27:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.289+0000 7fad5e79a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad541aa520 con 0x5614067fb810 2024-03-21T11:27:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.289+0000 7fad5e79a740 10 client.0 _ll_register_callbacks cb 0x7fad40009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.289+0000 7fad53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x56140680b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.289+0000 7fad59a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fb810 0x56140680b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.289+0000 7fad53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x56140680b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58260/0 (socket says 172.21.15.47:58260) 2024-03-21T11:27:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.289+0000 7fad53e00640 1 -- 172.21.15.47:0/2047983818 learned_addr learned my addr 172.21.15.47:0/2047983818 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.289+0000 7fad59a00640 1 --2- 172.21.15.47:0/2047983818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fd3a0 0x56140680c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.289+0000 7fad53e00640 1 -- 172.21.15.47:0/2047983818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fb810 msgr2=0x56140680b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.289+0000 7fad53e00640 1 --2- 172.21.15.47:0/2047983818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fb810 0x56140680b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.290+0000 7fad53e00640 1 -- 172.21.15.47:0/2047983818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fd3a0 msgr2=0x56140680c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.290+0000 7fad53e00640 1 --2- 172.21.15.47:0/2047983818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fd3a0 0x56140680c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.290+0000 7fad53e00640 1 -- 172.21.15.47:0/2047983818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56140680cb60 con 0x5614067fc9e0 2024-03-21T11:27:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.290+0000 7fad53e00640 1 --2- 172.21.15.47:0/2047983818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x56140680b560 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fad4002c7c0 tx=0x7fad4002bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5555e3d39e6bbc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.290+0000 7fad3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.290+0000 7fad3aa00640 1 -- 172.21.15.47:0/2047983818 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad400052d0 con 0x5614067fc9e0 2024-03-21T11:27:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.290+0000 7fad3aa00640 1 -- 172.21.15.47:0/2047983818 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fad40005470 con 0x5614067fc9e0 2024-03-21T11:27:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.290+0000 7fad5e79a740 1 -- 172.21.15.47:0/2047983818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad40004cf0 con 0x5614067fc9e0 2024-03-21T11:27:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.290+0000 7fad3aa00640 1 -- 172.21.15.47:0/2047983818 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad40063050 con 0x5614067fc9e0 2024-03-21T11:27:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.290+0000 7fad5e79a740 10 client.26945 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.291+0000 7fad5e79a740 1 -- 172.21.15.47:0/2047983818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561406875d90 con 0x5614067fc9e0 2024-03-21T11:27:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.291+0000 7fad3aa00640 1 -- 172.21.15.47:0/2047983818 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad40004cf0 con 0x5614067fc9e0 2024-03-21T11:27:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.291+0000 7fad3aa00640 1 -- 172.21.15.47:0/2047983818 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fad40046040 con 0x5614067fc9e0 2024-03-21T11:27:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.291+0000 7fad5e79a740 10 client.26945 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.291+0000 7fad5e79a740 10 client.26945 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:27:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.291+0000 7fad5e79a740 20 client.26945 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.291+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2047983818 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561406876620 0x561406878a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.291+0000 7fad5e79a740 4 client.26945 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.291+0000 7fad5e79a740 1 -- 172.21.15.47:0/2047983818 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561406767b50 con 0x561406876620 2024-03-21T11:27:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.291+0000 7fad59a00640 1 --2- 172.21.15.47:0/2047983818 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561406876620 0x561406878a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad59a00640 1 --2- 172.21.15.47:0/2047983818 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561406876620 0x561406878a40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cdbb9d7479d184b2 server_cookie=8ea2ac81f94ec16e in_seq=0 out_seq=0 2024-03-21T11:27:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad3aa00640 10 client.26945 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:27:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad3aa00640 1 -- 172.21.15.47:0/2047983818 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561406767b50 con 0x561406876620 2024-03-21T11:27:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad3aa00640 10 client.26945 handle_command_reply: tid=0 2024-03-21T11:27:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad5e79a740 1 client.26945 shutdown 2024-03-21T11:27:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad5e79a740 1 -- 172.21.15.47:0/2047983818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 msgr2=0x56140680b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2047983818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x56140680b560 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fad4002c7c0 tx=0x7fad4002bc40 comp rx=0 tx=0).stop 2024-03-21T11:27:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad5e79a740 1 -- 172.21.15.47:0/2047983818 shutdown_connections 2024-03-21T11:27:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2047983818 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561406876620 0x561406878a40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2047983818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614067fd3a0 0x56140680c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2047983818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614067fc9e0 0x56140680b560 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad3aa00640 0 client.26945 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:27:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad5e79a740 1 --2- 172.21.15.47:0/2047983818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5614067fb810 0x56140680b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad5e79a740 1 -- 172.21.15.47:0/2047983818 >> 172.21.15.47:0/2047983818 conn(0x5614067f6c00 msgr2=0x5614067f5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad5e79a740 1 -- 172.21.15.47:0/2047983818 shutdown_connections 2024-03-21T11:27:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.292+0000 7fad5e79a740 1 -- 172.21.15.47:0/2047983818 wait complete. 2024-03-21T11:27:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.293+0000 7fad5e79a740 20 client.26945 trim_cache size 0 max 16384 2024-03-21T11:27:48.299 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.293+0000 7fad5ba00640 1 -- 172.21.15.47:0/2323115042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad3c0a2fa0 msgr2=0x7fad3c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.293+0000 7fad5ba00640 1 --2- 172.21.15.47:0/2323115042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad3c0a2fa0 0x7fad3c0a5460 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fad480cc3b0 tx=0x7fad48004590 comp rx=0 tx=0).stop 2024-03-21T11:27:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.293+0000 7fad5ba00640 1 -- 172.21.15.47:0/2323115042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad541038e0 msgr2=0x7fad54199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.293+0000 7fad5ba00640 1 --2- 172.21.15.47:0/2323115042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad541038e0 0x7fad54199bd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fad4800d770 tx=0x7fad4800dc30 comp rx=0 tx=0).stop 2024-03-21T11:27:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.293+0000 7fad5ba00640 1 -- 172.21.15.47:0/2323115042 shutdown_connections 2024-03-21T11:27:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.294+0000 7fad5ba00640 1 --2- 172.21.15.47:0/2323115042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad3c0a2fa0 0x7fad3c0a5460 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.294+0000 7fad5ba00640 1 --2- 172.21.15.47:0/2323115042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad54105490 0x7fad541a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.294+0000 7fad5ba00640 1 --2- 172.21.15.47:0/2323115042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad54104ad0 0x7fad5419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.294+0000 7fad5ba00640 1 --2- 172.21.15.47:0/2323115042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad541038e0 0x7fad54199bd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.294+0000 7fad5ba00640 1 -- 172.21.15.47:0/2323115042 >> 172.21.15.47:0/2323115042 conn(0x7fad540fee50 msgr2=0x7fad54107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.294+0000 7fad5ba00640 1 -- 172.21.15.47:0/2323115042 shutdown_connections 2024-03-21T11:27:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.294+0000 7fad5ba00640 1 -- 172.21.15.47:0/2323115042 wait complete. 2024-03-21T11:27:48.316 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:48.316 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:27:48.505 INFO:tasks.workunit.client.0.smithi047.stdout: 1 243675 8.04 MB/sec execute 32 sec latency 22.087 ms 2024-03-21T11:27:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.526+0000 7fed34800640 1 -- 172.21.15.47:0/2160639068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed2c107cc0 msgr2=0x7fed2c1080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.526+0000 7fed34800640 1 --2- 172.21.15.47:0/2160639068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed2c107cc0 0x7fed2c1080e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fed20009900 tx=0x7fed2002bc40 comp rx=0 tx=0).stop 2024-03-21T11:27:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.526+0000 7fed34800640 1 -- 172.21.15.47:0/2160639068 shutdown_connections 2024-03-21T11:27:48.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.526+0000 7fed34800640 1 --2- 172.21.15.47:0/2160639068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed2c109870 0x7fed2c110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.526+0000 7fed34800640 1 --2- 172.21.15.47:0/2160639068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed2c108eb0 0x7fed2c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.526+0000 7fed34800640 1 --2- 172.21.15.47:0/2160639068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed2c107cc0 0x7fed2c1080e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.526+0000 7fed34800640 1 -- 172.21.15.47:0/2160639068 >> 172.21.15.47:0/2160639068 conn(0x7fed2c076c60 msgr2=0x7fed2c0790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.526+0000 7fed34800640 1 -- 172.21.15.47:0/2160639068 shutdown_connections 2024-03-21T11:27:48.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.526+0000 7fed34800640 1 -- 172.21.15.47:0/2160639068 wait complete. 2024-03-21T11:27:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.527+0000 7fed34800640 1 Processor -- start 2024-03-21T11:27:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.527+0000 7fed34800640 1 -- start start 2024-03-21T11:27:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed34800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed2c107cc0 0x7fed2c19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed34800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed2c108eb0 0x7fed2c19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed34800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed2c109870 0x7fed2c1a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed34800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c113990 con 0x7fed2c107cc0 2024-03-21T11:27:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed34800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c113810 con 0x7fed2c109870 2024-03-21T11:27:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed34800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c113b10 con 0x7fed2c108eb0 2024-03-21T11:27:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed31e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed2c107cc0 0x7fed2c19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed31400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed2c108eb0 0x7fed2c19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed32800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed2c109870 0x7fed2c1a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed31e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed2c107cc0 0x7fed2c19dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41072/0 (socket says 172.21.15.47:41072) 2024-03-21T11:27:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed31e00640 1 -- 172.21.15.47:0/3461970561 learned_addr learned my addr 172.21.15.47:0/3461970561 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed32800640 1 -- 172.21.15.47:0/3461970561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed2c108eb0 msgr2=0x7fed2c19e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed32800640 1 --2- 172.21.15.47:0/3461970561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed2c108eb0 0x7fed2c19e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed32800640 1 -- 172.21.15.47:0/3461970561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed2c107cc0 msgr2=0x7fed2c19dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed32800640 1 --2- 172.21.15.47:0/3461970561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed2c107cc0 0x7fed2c19dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed32800640 1 -- 172.21.15.47:0/3461970561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c1a5bc0 con 0x7fed2c109870 2024-03-21T11:27:48.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.528+0000 7fed31e00640 1 --2- 172.21.15.47:0/3461970561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed2c107cc0 0x7fed2c19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.529+0000 7fed32800640 1 --2- 172.21.15.47:0/3461970561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed2c109870 0x7fed2c1a5550 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fed28009fd0 tx=0x7fed2800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.529+0000 7fed26a00640 1 -- 172.21.15.47:0/3461970561 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2800ce20 con 0x7fed2c109870 2024-03-21T11:27:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.529+0000 7fed26a00640 1 -- 172.21.15.47:0/3461970561 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed28005020 con 0x7fed2c109870 2024-03-21T11:27:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.529+0000 7fed34800640 1 -- 172.21.15.47:0/3461970561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2c1a5eb0 con 0x7fed2c109870 2024-03-21T11:27:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.529+0000 7fed26a00640 1 -- 172.21.15.47:0/3461970561 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28010720 con 0x7fed2c109870 2024-03-21T11:27:48.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.529+0000 7fed34800640 1 -- 172.21.15.47:0/3461970561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2c1a63c0 con 0x7fed2c109870 2024-03-21T11:27:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.533+0000 7fed37468740 1 Processor -- start 2024-03-21T11:27:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.533+0000 7fed26a00640 1 -- 172.21.15.47:0/3461970561 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fed28005540 con 0x7fed2c109870 2024-03-21T11:27:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.533+0000 7fed37468740 1 -- start start 2024-03-21T11:27:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed37468740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb52984810 0x55fb52984c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed37468740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529859e0 0x55fb52985e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed37468740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb529863a0 0x55fb5298cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed37468740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb528e92c0 con 0x55fb52984810 2024-03-21T11:27:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed37468740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb52891c90 con 0x55fb529859e0 2024-03-21T11:27:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed37468740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb528e5360 con 0x55fb529863a0 2024-03-21T11:27:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed31e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529859e0 0x55fb52985e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed31400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb52984810 0x55fb52984c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed31e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529859e0 0x55fb52985e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58276/0 (socket says 172.21.15.47:58276) 2024-03-21T11:27:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed31e00640 1 -- 172.21.15.47:0/4152353877 learned_addr learned my addr 172.21.15.47:0/4152353877 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed31400640 1 --2- 172.21.15.47:0/4152353877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb529863a0 0x55fb5298cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed31e00640 1 -- 172.21.15.47:0/4152353877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb529863a0 msgr2=0x55fb5298cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed31e00640 1 --2- 172.21.15.47:0/4152353877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb529863a0 0x55fb5298cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed31e00640 1 -- 172.21.15.47:0/4152353877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb52984810 msgr2=0x55fb52984c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed31e00640 1 --2- 172.21.15.47:0/4152353877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb52984810 0x55fb52984c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.534+0000 7fed31e00640 1 -- 172.21.15.47:0/4152353877 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb528b6080 con 0x55fb529859e0 2024-03-21T11:27:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.535+0000 7fed31e00640 1 --2- 172.21.15.47:0/4152353877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529859e0 0x55fb52985e60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fed20036040 tx=0x7fed20000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce0b691b661896b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.535+0000 7fed26a00640 1 --2- 172.21.15.47:0/3461970561 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed140a2fa0 0x7fed140a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.535+0000 7fed1b000640 1 -- 172.21.15.47:0/4152353877 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20042070 con 0x55fb529859e0 2024-03-21T11:27:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.535+0000 7fed1b000640 1 -- 172.21.15.47:0/4152353877 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed2002cd40 con 0x55fb529859e0 2024-03-21T11:27:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.535+0000 7fed1b000640 1 -- 172.21.15.47:0/4152353877 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20005230 con 0x55fb529859e0 2024-03-21T11:27:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.535+0000 7fed26a00640 1 -- 172.21.15.47:0/3461970561 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fed280cfd50 con 0x7fed2c109870 2024-03-21T11:27:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.535+0000 7fed31400640 1 --2- 172.21.15.47:0/3461970561 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed140a2fa0 0x7fed140a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.536+0000 7fed37468740 1 -- 172.21.15.47:0/4152353877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529859e0 msgr2=0x55fb52985e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.536+0000 7fed37468740 1 --2- 172.21.15.47:0/4152353877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529859e0 0x55fb52985e60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fed20036040 tx=0x7fed20000bc0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.536+0000 7fed37468740 1 -- 172.21.15.47:0/4152353877 shutdown_connections 2024-03-21T11:27:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.536+0000 7fed37468740 1 --2- 172.21.15.47:0/4152353877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb529863a0 0x55fb5298cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.536+0000 7fed37468740 1 --2- 172.21.15.47:0/4152353877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529859e0 0x55fb52985e60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.536+0000 7fed37468740 1 --2- 172.21.15.47:0/4152353877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb52984810 0x55fb52984c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.536+0000 7fed37468740 1 -- 172.21.15.47:0/4152353877 >> 172.21.15.47:0/4152353877 conn(0x55fb5297fc00 msgr2=0x55fb52980000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.536+0000 7fed31400640 1 --2- 172.21.15.47:0/3461970561 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed140a2fa0 0x7fed140a5460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55fb529858e0 tx=0x7fed1c015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.536+0000 7fed37468740 1 -- 172.21.15.47:0/4152353877 shutdown_connections 2024-03-21T11:27:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.536+0000 7fed37468740 1 -- 172.21.15.47:0/4152353877 wait complete. 2024-03-21T11:27:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.537+0000 7fed37468740 1 -- start start 2024-03-21T11:27:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.537+0000 7fed37468740 1 Processor -- start 2024-03-21T11:27:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.538+0000 7fed37468740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb52984810 0x55fb5298e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.538+0000 7fed37468740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529859e0 0x55fb5298ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.538+0000 7fed37468740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb529863a0 0x55fb52995f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.538+0000 7fed37468740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5298d350 con 0x55fb529863a0 2024-03-21T11:27:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.538+0000 7fed37468740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb528a50d0 con 0x55fb529859e0 2024-03-21T11:27:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.538+0000 7fed37468740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5298d4d0 con 0x55fb52984810 2024-03-21T11:27:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.538+0000 7fed37468740 10 client.0 _ll_register_callbacks cb 0x55fb528a9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.538+0000 7fed31e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb52984810 0x55fb5298e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.538+0000 7fed31e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb52984810 0x55fb5298e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40678/0 (socket says 172.21.15.47:40678) 2024-03-21T11:27:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.538+0000 7fed31e00640 1 -- 172.21.15.47:0/1737621615 learned_addr learned my addr 172.21.15.47:0/1737621615 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.538+0000 7fed31400640 1 --2- 172.21.15.47:0/1737621615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529859e0 0x55fb5298ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.539+0000 7fed31e00640 1 -- 172.21.15.47:0/1737621615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529859e0 msgr2=0x55fb5298ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.539+0000 7fed31e00640 1 --2- 172.21.15.47:0/1737621615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529859e0 0x55fb5298ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.539+0000 7fed31e00640 1 -- 172.21.15.47:0/1737621615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb529863a0 msgr2=0x55fb52995f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.539+0000 7fed31e00640 1 --2- 172.21.15.47:0/1737621615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb529863a0 0x55fb52995f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.539+0000 7fed31e00640 1 -- 172.21.15.47:0/1737621615 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb525fd3f0 con 0x55fb52984810 2024-03-21T11:27:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.539+0000 7fed31e00640 1 --2- 172.21.15.47:0/1737621615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb52984810 0x55fb5298e9f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fed2002c8f0 tx=0x7fed20005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed44ce83cb9503ac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.539+0000 7fed0f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.539+0000 7fed0f400640 1 -- 172.21.15.47:0/1737621615 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2004e020 con 0x55fb52984810 2024-03-21T11:27:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.539+0000 7fed0f400640 1 -- 172.21.15.47:0/1737621615 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fed20004b50 con 0x55fb52984810 2024-03-21T11:27:48.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.539+0000 7fed37468740 1 -- 172.21.15.47:0/1737621615 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed2002cd40 con 0x55fb52984810 2024-03-21T11:27:48.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.540+0000 7fed0f400640 1 -- 172.21.15.47:0/1737621615 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed20059050 con 0x55fb52984810 2024-03-21T11:27:48.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.540+0000 7fed37468740 10 client.26971 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:48.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.540+0000 7fed37468740 1 -- 172.21.15.47:0/1737621615 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55fb526c77d0 con 0x55fb52984810 2024-03-21T11:27:48.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.540+0000 7fed0f400640 1 -- 172.21.15.47:0/1737621615 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed2002cd40 con 0x55fb52984810 2024-03-21T11:27:48.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.540+0000 7fed0f400640 1 -- 172.21.15.47:0/1737621615 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fed20034560 con 0x55fb52984810 2024-03-21T11:27:48.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.540+0000 7fed37468740 10 client.26971 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:48.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.540+0000 7fed37468740 10 client.26971 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:27:48.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.540+0000 7fed37468740 20 client.26971 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:48.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.540+0000 7fed37468740 1 --2- 172.21.15.47:0/1737621615 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55fb5299a0d0 0x55fb5299c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.540+0000 7fed37468740 4 client.26971 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:48.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.540+0000 7fed37468740 1 -- 172.21.15.47:0/1737621615 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb528f6800 con 0x55fb5299a0d0 2024-03-21T11:27:48.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.541+0000 7fed31400640 1 --2- 172.21.15.47:0/1737621615 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55fb5299a0d0 0x55fb5299c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.541+0000 7fed31400640 1 --2- 172.21.15.47:0/1737621615 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55fb5299a0d0 0x55fb5299c4c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46ae30fe116d7285 server_cookie=48492a0255a67b52 in_seq=0 out_seq=0 2024-03-21T11:27:48.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.541+0000 7fed0f400640 10 client.26971 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:27:48.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.542+0000 7fed0f400640 1 -- 172.21.15.47:0/1737621615 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fb528f6800 con 0x55fb5299a0d0 2024-03-21T11:27:48.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.542+0000 7fed0f400640 10 client.26971 handle_command_reply: tid=0 2024-03-21T11:27:48.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.542+0000 7fed37468740 1 client.26971 shutdown 2024-03-21T11:27:48.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.542+0000 7fed37468740 1 -- 172.21.15.47:0/1737621615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb52984810 msgr2=0x55fb5298e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.542+0000 7fed37468740 1 --2- 172.21.15.47:0/1737621615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb52984810 0x55fb5298e9f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fed2002c8f0 tx=0x7fed20005770 comp rx=0 tx=0).stop 2024-03-21T11:27:48.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.542+0000 7fed37468740 1 -- 172.21.15.47:0/1737621615 shutdown_connections 2024-03-21T11:27:48.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.542+0000 7fed37468740 1 --2- 172.21.15.47:0/1737621615 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55fb5299a0d0 0x55fb5299c4c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.542+0000 7fed37468740 1 --2- 172.21.15.47:0/1737621615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb529863a0 0x55fb52995f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.542+0000 7fed37468740 1 --2- 172.21.15.47:0/1737621615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529859e0 0x55fb5298ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.542+0000 7fed37468740 1 --2- 172.21.15.47:0/1737621615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb52984810 0x55fb5298e9f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.542+0000 7fed0f400640 0 client.26971 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:27:48.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.542+0000 7fed37468740 1 -- 172.21.15.47:0/1737621615 >> 172.21.15.47:0/1737621615 conn(0x55fb5297fc00 msgr2=0x55fb5297eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.543+0000 7fed37468740 1 -- 172.21.15.47:0/1737621615 shutdown_connections 2024-03-21T11:27:48.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.543+0000 7fed37468740 1 -- 172.21.15.47:0/1737621615 wait complete. 2024-03-21T11:27:48.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.543+0000 7fed37468740 20 client.26971 trim_cache size 0 max 16384 2024-03-21T11:27:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.558+0000 7fed37468740 1 Processor -- start 2024-03-21T11:27:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.558+0000 7fed37468740 1 -- start start 2024-03-21T11:27:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed37468740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb52984810 0x55fb529a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed37468740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb529859e0 0x55fb52995f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed37468740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529863a0 0x55fb529a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed37468740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c1b05f0 con 0x55fb529859e0 2024-03-21T11:27:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed37468740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c1b1f70 con 0x55fb529863a0 2024-03-21T11:27:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed37468740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c1b0c50 con 0x55fb52984810 2024-03-21T11:27:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed32800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529863a0 0x55fb529a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed31e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb52984810 0x55fb529a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed32800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529863a0 0x55fb529a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58286/0 (socket says 172.21.15.47:58286) 2024-03-21T11:27:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed32800640 1 -- 172.21.15.47:0/3696886774 learned_addr learned my addr 172.21.15.47:0/3696886774 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed31400640 1 --2- 172.21.15.47:0/3696886774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb529859e0 0x55fb52995f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed32800640 1 -- 172.21.15.47:0/3696886774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb52984810 msgr2=0x55fb529a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed32800640 1 --2- 172.21.15.47:0/3696886774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb52984810 0x55fb529a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed32800640 1 -- 172.21.15.47:0/3696886774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb529859e0 msgr2=0x55fb52995f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed32800640 1 --2- 172.21.15.47:0/3696886774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb529859e0 0x55fb52995f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed32800640 1 -- 172.21.15.47:0/3696886774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb52980670 con 0x55fb529863a0 2024-03-21T11:27:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed31400640 1 --2- 172.21.15.47:0/3696886774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb529859e0 0x55fb52995f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.559+0000 7fed31e00640 1 --2- 172.21.15.47:0/3696886774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb52984810 0x55fb529a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.560+0000 7fed32800640 1 --2- 172.21.15.47:0/3696886774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529863a0 0x55fb529a12c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fed280984f0 tx=0x7fed2809bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c15a7cc13203304 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.560+0000 7fed1a600640 1 -- 172.21.15.47:0/3696886774 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280a4aa0 con 0x55fb529863a0 2024-03-21T11:27:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.560+0000 7fed1a600640 1 -- 172.21.15.47:0/3696886774 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed2809a130 con 0x55fb529863a0 2024-03-21T11:27:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.560+0000 7fed1a600640 1 -- 172.21.15.47:0/3696886774 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280bc0f0 con 0x55fb529863a0 2024-03-21T11:27:48.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.561+0000 7fed37468740 1 -- 172.21.15.47:0/3696886774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529863a0 msgr2=0x55fb529a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.561+0000 7fed37468740 1 --2- 172.21.15.47:0/3696886774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529863a0 0x55fb529a12c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fed280984f0 tx=0x7fed2809bab0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.561+0000 7fed37468740 1 -- 172.21.15.47:0/3696886774 shutdown_connections 2024-03-21T11:27:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.561+0000 7fed37468740 1 --2- 172.21.15.47:0/3696886774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529863a0 0x55fb529a12c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.561+0000 7fed37468740 1 --2- 172.21.15.47:0/3696886774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb529859e0 0x55fb52995f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.561+0000 7fed37468740 1 --2- 172.21.15.47:0/3696886774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb52984810 0x55fb529a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.561+0000 7fed37468740 1 -- 172.21.15.47:0/3696886774 >> 172.21.15.47:0/3696886774 conn(0x55fb5297fc00 msgr2=0x55fb5297eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.561+0000 7fed37468740 1 -- 172.21.15.47:0/3696886774 shutdown_connections 2024-03-21T11:27:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.561+0000 7fed37468740 1 -- 172.21.15.47:0/3696886774 wait complete. 2024-03-21T11:27:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.561+0000 7fed37468740 1 -- start start 2024-03-21T11:27:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.562+0000 7fed37468740 1 Processor -- start 2024-03-21T11:27:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.562+0000 7fed37468740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb52984810 0x55fb52991ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.562+0000 7fed37468740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb529859e0 0x55fb52992420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.562+0000 7fed37468740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529863a0 0x55fb52995440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.562+0000 7fed37468740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c1b1910 con 0x55fb52984810 2024-03-21T11:27:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.562+0000 7fed37468740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c1b12b0 con 0x55fb529863a0 2024-03-21T11:27:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.562+0000 7fed37468740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c1ae7b0 con 0x55fb529859e0 2024-03-21T11:27:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.562+0000 7fed37468740 10 client.0 _ll_register_callbacks cb 0x7fed1c009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.562+0000 7fed32800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529863a0 0x55fb52995440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.562+0000 7fed32800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529863a0 0x55fb52995440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58288/0 (socket says 172.21.15.47:58288) 2024-03-21T11:27:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.562+0000 7fed32800640 1 -- 172.21.15.47:0/3549247495 learned_addr learned my addr 172.21.15.47:0/3549247495 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.562+0000 7fed31e00640 1 --2- 172.21.15.47:0/3549247495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb52984810 0x55fb52991ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.563+0000 7fed31400640 1 --2- 172.21.15.47:0/3549247495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb529859e0 0x55fb52992420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.563+0000 7fed32800640 1 -- 172.21.15.47:0/3549247495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb529859e0 msgr2=0x55fb52992420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.563+0000 7fed32800640 1 --2- 172.21.15.47:0/3549247495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb529859e0 0x55fb52992420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.563+0000 7fed32800640 1 -- 172.21.15.47:0/3549247495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb52984810 msgr2=0x55fb52991ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.563+0000 7fed32800640 1 --2- 172.21.15.47:0/3549247495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb52984810 0x55fb52991ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.563+0000 7fed32800640 1 -- 172.21.15.47:0/3549247495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb52995a30 con 0x55fb529863a0 2024-03-21T11:27:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.563+0000 7fed32800640 1 --2- 172.21.15.47:0/3549247495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529863a0 0x55fb52995440 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fed2800c760 tx=0x7fed2809bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=713f9f20cad39ef3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.563+0000 7fed0f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.563+0000 7fed0f400640 1 -- 172.21.15.47:0/3549247495 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280a4aa0 con 0x55fb529863a0 2024-03-21T11:27:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.564+0000 7fed0f400640 1 -- 172.21.15.47:0/3549247495 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fed2809a390 con 0x55fb529863a0 2024-03-21T11:27:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.564+0000 7fed37468740 1 -- 172.21.15.47:0/3549247495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed2809a130 con 0x55fb529863a0 2024-03-21T11:27:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.564+0000 7fed0f400640 1 -- 172.21.15.47:0/3549247495 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed28084050 con 0x55fb529863a0 2024-03-21T11:27:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.564+0000 7fed37468740 10 client.26963 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.564+0000 7fed37468740 1 -- 172.21.15.47:0/3549247495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55fb529f62c0 con 0x55fb529863a0 2024-03-21T11:27:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.564+0000 7fed0f400640 1 -- 172.21.15.47:0/3549247495 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed2809a130 con 0x55fb529863a0 2024-03-21T11:27:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.564+0000 7fed0f400640 1 -- 172.21.15.47:0/3549247495 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fed2809b4e0 con 0x55fb529863a0 2024-03-21T11:27:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.564+0000 7fed37468740 10 client.26963 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.564+0000 7fed37468740 10 client.26963 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:27:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.564+0000 7fed37468740 20 client.26963 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.564+0000 7fed37468740 1 --2- 172.21.15.47:0/3549247495 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55fb529f6be0 0x55fb529f9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.564+0000 7fed37468740 4 client.26963 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:48.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.564+0000 7fed37468740 1 -- 172.21.15.47:0/3549247495 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fb528f0b50 con 0x55fb529f6be0 2024-03-21T11:27:48.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.565+0000 7fed31e00640 1 --2- 172.21.15.47:0/3549247495 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55fb529f6be0 0x55fb529f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.565+0000 7fed31e00640 1 --2- 172.21.15.47:0/3549247495 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55fb529f6be0 0x55fb529f9000 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=27cb7f63230327bc server_cookie=2db9e71cd9015796 in_seq=0 out_seq=0 2024-03-21T11:27:48.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.565+0000 7fed0f400640 10 client.26963 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:27:48.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.565+0000 7fed0f400640 1 -- 172.21.15.47:0/3549247495 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fb528f0b50 con 0x55fb529f6be0 2024-03-21T11:27:48.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.565+0000 7fed0f400640 10 client.26963 handle_command_reply: tid=0 2024-03-21T11:27:48.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.565+0000 7fed37468740 1 client.26963 shutdown 2024-03-21T11:27:48.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.566+0000 7fed37468740 1 -- 172.21.15.47:0/3549247495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529863a0 msgr2=0x55fb52995440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.566+0000 7fed37468740 1 --2- 172.21.15.47:0/3549247495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529863a0 0x55fb52995440 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fed2800c760 tx=0x7fed2809bb20 comp rx=0 tx=0).stop 2024-03-21T11:27:48.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.566+0000 7fed37468740 1 -- 172.21.15.47:0/3549247495 shutdown_connections 2024-03-21T11:27:48.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.566+0000 7fed37468740 1 --2- 172.21.15.47:0/3549247495 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55fb529f6be0 0x55fb529f9000 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.566+0000 7fed37468740 1 --2- 172.21.15.47:0/3549247495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb529863a0 0x55fb52995440 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.566+0000 7fed37468740 1 --2- 172.21.15.47:0/3549247495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb529859e0 0x55fb52992420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.566+0000 7fed0f400640 0 client.26963 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:27:48.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.566+0000 7fed37468740 1 --2- 172.21.15.47:0/3549247495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb52984810 0x55fb52991ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.566+0000 7fed37468740 1 -- 172.21.15.47:0/3549247495 >> 172.21.15.47:0/3549247495 conn(0x55fb5297fc00 msgr2=0x55fb529a0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.566+0000 7fed37468740 1 -- 172.21.15.47:0/3549247495 shutdown_connections 2024-03-21T11:27:48.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.566+0000 7fed37468740 1 -- 172.21.15.47:0/3549247495 wait complete. 2024-03-21T11:27:48.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.566+0000 7fed37468740 20 client.26963 trim_cache size 0 max 16384 2024-03-21T11:27:48.573 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:48.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.567+0000 7fed34800640 1 -- 172.21.15.47:0/3461970561 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed140a2fa0 msgr2=0x7fed140a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.567+0000 7fed34800640 1 --2- 172.21.15.47:0/3461970561 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed140a2fa0 0x7fed140a5460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55fb529858e0 tx=0x7fed1c015040 comp rx=0 tx=0).stop 2024-03-21T11:27:48.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.567+0000 7fed34800640 1 -- 172.21.15.47:0/3461970561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed2c109870 msgr2=0x7fed2c1a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.567+0000 7fed34800640 1 --2- 172.21.15.47:0/3461970561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed2c109870 0x7fed2c1a5550 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fed28009fd0 tx=0x7fed2800ef10 comp rx=0 tx=0).stop 2024-03-21T11:27:48.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.567+0000 7fed34800640 1 -- 172.21.15.47:0/3461970561 shutdown_connections 2024-03-21T11:27:48.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.567+0000 7fed34800640 1 --2- 172.21.15.47:0/3461970561 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed140a2fa0 0x7fed140a5460 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.567+0000 7fed34800640 1 --2- 172.21.15.47:0/3461970561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed2c109870 0x7fed2c1a5550 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.567+0000 7fed34800640 1 --2- 172.21.15.47:0/3461970561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed2c108eb0 0x7fed2c19e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.567+0000 7fed34800640 1 --2- 172.21.15.47:0/3461970561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed2c107cc0 0x7fed2c19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.567+0000 7fed34800640 1 -- 172.21.15.47:0/3461970561 >> 172.21.15.47:0/3461970561 conn(0x7fed2c076c60 msgr2=0x7fed2c10c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.567+0000 7fed34800640 1 -- 172.21.15.47:0/3461970561 shutdown_connections 2024-03-21T11:27:48.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.567+0000 7fed34800640 1 -- 172.21.15.47:0/3461970561 wait complete. 2024-03-21T11:27:48.585 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:48.585 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:27:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.810+0000 7f6be3800640 1 -- 172.21.15.47:0/2483345666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc104ad0 msgr2=0x7f6bdc104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.810+0000 7f6be3800640 1 --2- 172.21.15.47:0/2483345666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc104ad0 0x7f6bdc104f50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009900 tx=0x7f6bcc02bc40 comp rx=0 tx=0).stop 2024-03-21T11:27:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.810+0000 7f6be3800640 1 -- 172.21.15.47:0/2483345666 shutdown_connections 2024-03-21T11:27:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.810+0000 7f6be3800640 1 --2- 172.21.15.47:0/2483345666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc105490 0x7f6bdc10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.810+0000 7f6be3800640 1 --2- 172.21.15.47:0/2483345666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc104ad0 0x7f6bdc104f50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.810+0000 7f6be3800640 1 --2- 172.21.15.47:0/2483345666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc1038e0 0x7f6bdc103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.810+0000 7f6be3800640 1 -- 172.21.15.47:0/2483345666 >> 172.21.15.47:0/2483345666 conn(0x7f6bdc0fee50 msgr2=0x7f6bdc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.810+0000 7f6be3800640 1 -- 172.21.15.47:0/2483345666 shutdown_connections 2024-03-21T11:27:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.810+0000 7f6be3800640 1 -- 172.21.15.47:0/2483345666 wait complete. 2024-03-21T11:27:48.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.811+0000 7f6be3800640 1 Processor -- start 2024-03-21T11:27:48.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.812+0000 7f6be3800640 1 -- start start 2024-03-21T11:27:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.812+0000 7f6be3800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc1038e0 0x7f6bdc199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.812+0000 7f6be3800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc104ad0 0x7f6bdc19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.812+0000 7f6be3800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc105490 0x7f6bdc1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.812+0000 7f6be3800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc10f560 con 0x7f6bdc105490 2024-03-21T11:27:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.812+0000 7f6be3800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc10f3e0 con 0x7f6bdc104ad0 2024-03-21T11:27:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.812+0000 7f6be3800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc10f6e0 con 0x7f6bdc1038e0 2024-03-21T11:27:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.812+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc104ad0 0x7f6bdc19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.812+0000 7f6be0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc1038e0 0x7f6bdc199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.812+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc104ad0 0x7f6bdc19a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58312/0 (socket says 172.21.15.47:58312) 2024-03-21T11:27:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.812+0000 7f6bd3e00640 1 -- 172.21.15.47:0/1220541078 learned_addr learned my addr 172.21.15.47:0/1220541078 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.812+0000 7f6be1800640 1 --2- 172.21.15.47:0/1220541078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc105490 0x7f6bdc1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.813+0000 7f6bd3e00640 1 -- 172.21.15.47:0/1220541078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc1038e0 msgr2=0x7f6bdc199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.813+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/1220541078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc1038e0 0x7f6bdc199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.813+0000 7f6bd3e00640 1 -- 172.21.15.47:0/1220541078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc105490 msgr2=0x7f6bdc1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.813+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/1220541078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc105490 0x7f6bdc1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.813+0000 7f6bd3e00640 1 -- 172.21.15.47:0/1220541078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc1a1810 con 0x7f6bdc104ad0 2024-03-21T11:27:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.813+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/1220541078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc104ad0 0x7f6bdc19a130 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009230 tx=0x7f6bcc005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.813+0000 7f6bda000640 1 -- 172.21.15.47:0/1220541078 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc041070 con 0x7f6bdc104ad0 2024-03-21T11:27:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.813+0000 7f6bda000640 1 -- 172.21.15.47:0/1220541078 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bcc0054e0 con 0x7f6bdc104ad0 2024-03-21T11:27:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.813+0000 7f6bda000640 1 -- 172.21.15.47:0/1220541078 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc034440 con 0x7f6bdc104ad0 2024-03-21T11:27:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.813+0000 7f6be3800640 1 -- 172.21.15.47:0/1220541078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bdc1a1aa0 con 0x7f6bdc104ad0 2024-03-21T11:27:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.814+0000 7f6be3800640 1 -- 172.21.15.47:0/1220541078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bdc1a1f40 con 0x7f6bdc104ad0 2024-03-21T11:27:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.817+0000 7f6bda000640 1 -- 172.21.15.47:0/1220541078 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6bcc044070 con 0x7f6bdc104ad0 2024-03-21T11:27:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.817+0000 7f6be6429740 1 Processor -- start 2024-03-21T11:27:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.817+0000 7f6be6429740 1 -- start start 2024-03-21T11:27:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be6429740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb72810 0x56110eb72c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be6429740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb73e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be6429740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb743a0 0x56110eb7ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be6429740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110ead72c0 con 0x56110eb72810 2024-03-21T11:27:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be6429740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110ea7fc90 con 0x56110eb743a0 2024-03-21T11:27:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be6429740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110ead3360 con 0x56110eb739e0 2024-03-21T11:27:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb73e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb743a0 0x56110eb7ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb73e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40732/0 (socket says 172.21.15.47:40732) 2024-03-21T11:27:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be0e00640 1 -- 172.21.15.47:0/695810388 learned_addr learned my addr 172.21.15.47:0/695810388 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be1800640 1 --2- 172.21.15.47:0/695810388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb72810 0x56110eb72c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be0e00640 1 -- 172.21.15.47:0/695810388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb743a0 msgr2=0x56110eb7ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be0e00640 1 --2- 172.21.15.47:0/695810388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb743a0 0x56110eb7ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be0e00640 1 -- 172.21.15.47:0/695810388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb72810 msgr2=0x56110eb72c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be0e00640 1 --2- 172.21.15.47:0/695810388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb72810 0x56110eb72c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.818+0000 7f6be0e00640 1 -- 172.21.15.47:0/695810388 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56110eaa4080 con 0x56110eb739e0 2024-03-21T11:27:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.819+0000 7f6be0e00640 1 --2- 172.21.15.47:0/695810388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb73e60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400bcb0 tx=0x7f6bc40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab1af348405d5e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.819+0000 7f6bda000640 1 --2- 172.21.15.47:0/1220541078 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bbc0a2fa0 0x7f6bbc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.819+0000 7f6bd2000640 1 -- 172.21.15.47:0/695810388 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc401c480 con 0x56110eb739e0 2024-03-21T11:27:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.819+0000 7f6bd2000640 1 -- 172.21.15.47:0/695810388 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bc401cb00 con 0x56110eb739e0 2024-03-21T11:27:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.819+0000 7f6bd2000640 1 -- 172.21.15.47:0/695810388 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4011ca0 con 0x56110eb739e0 2024-03-21T11:27:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.819+0000 7f6bda000640 1 -- 172.21.15.47:0/1220541078 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6bcc0f1400 con 0x7f6bdc104ad0 2024-03-21T11:27:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.819+0000 7f6be1800640 1 --2- 172.21.15.47:0/1220541078 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bbc0a2fa0 0x7f6bbc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.820+0000 7f6be6429740 1 -- 172.21.15.47:0/695810388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 msgr2=0x56110eb73e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.820+0000 7f6be6429740 1 --2- 172.21.15.47:0/695810388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb73e60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400bcb0 tx=0x7f6bc40107a0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.820+0000 7f6be1800640 1 --2- 172.21.15.47:0/1220541078 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bbc0a2fa0 0x7f6bbc0a5460 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x56110eb738e0 tx=0x7f6bd4009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.820+0000 7f6be6429740 1 -- 172.21.15.47:0/695810388 shutdown_connections 2024-03-21T11:27:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.820+0000 7f6be6429740 1 --2- 172.21.15.47:0/695810388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb743a0 0x56110eb7ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.820+0000 7f6be6429740 1 --2- 172.21.15.47:0/695810388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb73e60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.820+0000 7f6be6429740 1 --2- 172.21.15.47:0/695810388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb72810 0x56110eb72c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.820+0000 7f6be6429740 1 -- 172.21.15.47:0/695810388 >> 172.21.15.47:0/695810388 conn(0x56110eb6dc00 msgr2=0x56110eb6e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.820+0000 7f6be6429740 1 -- 172.21.15.47:0/695810388 shutdown_connections 2024-03-21T11:27:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.820+0000 7f6be6429740 1 -- 172.21.15.47:0/695810388 wait complete. 2024-03-21T11:27:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.821+0000 7f6be6429740 1 -- start start 2024-03-21T11:27:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.821+0000 7f6be6429740 1 Processor -- start 2024-03-21T11:27:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.822+0000 7f6be6429740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb72810 0x56110eb7ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.822+0000 7f6be6429740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb7cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.822+0000 7f6be6429740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb743a0 0x56110eb83fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.822+0000 7f6be6429740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110eb7b3b0 con 0x56110eb743a0 2024-03-21T11:27:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.822+0000 7f6be6429740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110ea930d0 con 0x56110eb72810 2024-03-21T11:27:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.822+0000 7f6be6429740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110eb7b530 con 0x56110eb739e0 2024-03-21T11:27:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.822+0000 7f6be6429740 10 client.0 _ll_register_callbacks cb 0x56110ea97e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.822+0000 7f6be0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb72810 0x56110eb7ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.822+0000 7f6be0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb72810 0x56110eb7ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58336/0 (socket says 172.21.15.47:58336) 2024-03-21T11:27:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.822+0000 7f6be0e00640 1 -- 172.21.15.47:0/1408289209 learned_addr learned my addr 172.21.15.47:0/1408289209 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.822+0000 7f6be1800640 1 --2- 172.21.15.47:0/1408289209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb743a0 0x56110eb83fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.822+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/1408289209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb7cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.823+0000 7f6be1800640 1 -- 172.21.15.47:0/1408289209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 msgr2=0x56110eb7cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.823+0000 7f6be1800640 1 --2- 172.21.15.47:0/1408289209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb7cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.823+0000 7f6be1800640 1 -- 172.21.15.47:0/1408289209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb72810 msgr2=0x56110eb7ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.823+0000 7f6be1800640 1 --2- 172.21.15.47:0/1408289209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb72810 0x56110eb7ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.823+0000 7f6be1800640 1 -- 172.21.15.47:0/1408289209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56110e7eb3f0 con 0x56110eb743a0 2024-03-21T11:27:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.823+0000 7f6be0e00640 1 --2- 172.21.15.47:0/1408289209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb72810 0x56110eb7ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.823+0000 7f6be1800640 1 --2- 172.21.15.47:0/1408289209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb743a0 0x56110eb83fd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4010850 tx=0x7f6bd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f73a078628e577 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.823+0000 7f6bcaa00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:27:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.823+0000 7f6bcaa00640 1 -- 172.21.15.47:0/1408289209 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4027630 con 0x56110eb743a0 2024-03-21T11:27:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.823+0000 7f6bcaa00640 1 -- 172.21.15.47:0/1408289209 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6bd401d410 con 0x56110eb743a0 2024-03-21T11:27:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.824+0000 7f6be6429740 1 -- 172.21.15.47:0/1408289209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bc401cb00 con 0x56110eb743a0 2024-03-21T11:27:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.824+0000 7f6bcaa00640 1 -- 172.21.15.47:0/1408289209 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bd4003970 con 0x56110eb743a0 2024-03-21T11:27:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.824+0000 7f6be6429740 10 client.16728 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.824+0000 7f6be6429740 1 -- 172.21.15.47:0/1408289209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56110e8b57d0 con 0x56110eb743a0 2024-03-21T11:27:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.824+0000 7f6bcaa00640 1 -- 172.21.15.47:0/1408289209 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bc401cb00 con 0x56110eb743a0 2024-03-21T11:27:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.824+0000 7f6bcaa00640 1 -- 172.21.15.47:0/1408289209 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6bd400f040 con 0x56110eb743a0 2024-03-21T11:27:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.825+0000 7f6be6429740 10 client.16728 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.825+0000 7f6be6429740 10 client.16728 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:27:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.825+0000 7f6be6429740 20 client.16728 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.825+0000 7f6be6429740 1 --2- 172.21.15.47:0/1408289209 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56110eb88620 0x56110eb8aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.825+0000 7f6be6429740 4 client.16728 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.825+0000 7f6be6429740 1 -- 172.21.15.47:0/1408289209 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56110eae4800 con 0x56110eb88620 2024-03-21T11:27:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.825+0000 7f6be0e00640 1 --2- 172.21.15.47:0/1408289209 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56110eb88620 0x56110eb8aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.826+0000 7f6be0e00640 1 --2- 172.21.15.47:0/1408289209 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56110eb88620 0x56110eb8aac0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18ea4c0398a01eb8 server_cookie=fb6b5a921c64f9e1 in_seq=0 out_seq=0 2024-03-21T11:27:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.826+0000 7f6bcaa00640 10 client.16728 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:27:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6bcaa00640 1 -- 172.21.15.47:0/1408289209 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56110eae4800 con 0x56110eb88620 2024-03-21T11:27:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6bcaa00640 10 client.16728 handle_command_reply: tid=0 2024-03-21T11:27:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6be6429740 1 client.16728 shutdown 2024-03-21T11:27:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6be6429740 1 -- 172.21.15.47:0/1408289209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb743a0 msgr2=0x56110eb83fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6be6429740 1 --2- 172.21.15.47:0/1408289209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb743a0 0x56110eb83fd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4010850 tx=0x7f6bd4004a40 comp rx=0 tx=0).stop 2024-03-21T11:27:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6be6429740 1 -- 172.21.15.47:0/1408289209 shutdown_connections 2024-03-21T11:27:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6be6429740 1 --2- 172.21.15.47:0/1408289209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb743a0 0x56110eb83fd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6be6429740 1 --2- 172.21.15.47:0/1408289209 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56110eb88620 0x56110eb8aac0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6be6429740 1 --2- 172.21.15.47:0/1408289209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb7cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6be6429740 1 --2- 172.21.15.47:0/1408289209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb72810 0x56110eb7ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6be6429740 1 -- 172.21.15.47:0/1408289209 >> 172.21.15.47:0/1408289209 conn(0x56110eb6dc00 msgr2=0x56110eb6ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6bcaa00640 0 client.16728 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:27:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6be6429740 1 -- 172.21.15.47:0/1408289209 shutdown_connections 2024-03-21T11:27:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6be6429740 1 -- 172.21.15.47:0/1408289209 wait complete. 2024-03-21T11:27:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.827+0000 7f6be6429740 20 client.16728 trim_cache size 0 max 16384 2024-03-21T11:27:48.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.843+0000 7f6be6429740 1 Processor -- start 2024-03-21T11:27:48.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.843+0000 7f6be6429740 1 -- start start 2024-03-21T11:27:48.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.843+0000 7f6be6429740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb72810 0x56110eb769e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.843+0000 7f6be6429740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb83f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.843+0000 7f6be6429740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb743a0 0x56110eb76f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.844+0000 7f6be6429740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc1ac260 con 0x56110eb743a0 2024-03-21T11:27:48.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.844+0000 7f6be6429740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc1adbe0 con 0x56110eb72810 2024-03-21T11:27:48.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.844+0000 7f6be6429740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc1ac8c0 con 0x56110eb739e0 2024-03-21T11:27:48.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.844+0000 7f6be0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb72810 0x56110eb769e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.844+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb83f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.844+0000 7f6be0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb72810 0x56110eb769e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58342/0 (socket says 172.21.15.47:58342) 2024-03-21T11:27:48.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.844+0000 7f6be0e00640 1 -- 172.21.15.47:0/2525770742 learned_addr learned my addr 172.21.15.47:0/2525770742 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.844+0000 7f6be1800640 1 --2- 172.21.15.47:0/2525770742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb743a0 0x56110eb76f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.844+0000 7f6be0e00640 1 -- 172.21.15.47:0/2525770742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 msgr2=0x56110eb83f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.844+0000 7f6be0e00640 1 --2- 172.21.15.47:0/2525770742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb83f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.844+0000 7f6be0e00640 1 -- 172.21.15.47:0/2525770742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb743a0 msgr2=0x56110eb76f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.844+0000 7f6be0e00640 1 --2- 172.21.15.47:0/2525770742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb743a0 0x56110eb76f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.844+0000 7f6be0e00640 1 -- 172.21.15.47:0/2525770742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56110eb7a4f0 con 0x56110eb72810 2024-03-21T11:27:48.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.844+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/2525770742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb83f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:48.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.845+0000 7f6be0e00640 1 --2- 172.21.15.47:0/2525770742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb72810 0x56110eb769e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400bbd0 tx=0x7f6bc400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7953614ebce75288 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.845+0000 7f6bd1600640 1 -- 172.21.15.47:0/2525770742 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4005040 con 0x56110eb72810 2024-03-21T11:27:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.845+0000 7f6bd1600640 1 -- 172.21.15.47:0/2525770742 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bc4031070 con 0x56110eb72810 2024-03-21T11:27:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.845+0000 7f6bd1600640 1 -- 172.21.15.47:0/2525770742 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc402c790 con 0x56110eb72810 2024-03-21T11:27:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.846+0000 7f6be6429740 1 -- 172.21.15.47:0/2525770742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb72810 msgr2=0x56110eb769e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.846+0000 7f6be6429740 1 --2- 172.21.15.47:0/2525770742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb72810 0x56110eb769e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400bbd0 tx=0x7f6bc400b8b0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.846+0000 7f6be6429740 1 -- 172.21.15.47:0/2525770742 shutdown_connections 2024-03-21T11:27:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.846+0000 7f6be6429740 1 --2- 172.21.15.47:0/2525770742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb743a0 0x56110eb76f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.846+0000 7f6be6429740 1 --2- 172.21.15.47:0/2525770742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb83f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.846+0000 7f6be6429740 1 --2- 172.21.15.47:0/2525770742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb72810 0x56110eb769e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.846+0000 7f6be6429740 1 -- 172.21.15.47:0/2525770742 >> 172.21.15.47:0/2525770742 conn(0x56110eb6dc00 msgr2=0x56110eb716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.846+0000 7f6be6429740 1 -- 172.21.15.47:0/2525770742 shutdown_connections 2024-03-21T11:27:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.846+0000 7f6be6429740 1 -- 172.21.15.47:0/2525770742 wait complete. 2024-03-21T11:27:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.846+0000 7f6be6429740 1 -- start start 2024-03-21T11:27:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.847+0000 7f6be6429740 1 Processor -- start 2024-03-21T11:27:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.847+0000 7f6be6429740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb72810 0x56110eb7fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.847+0000 7f6be6429740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb800a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.847+0000 7f6be6429740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb743a0 0x56110eb830c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.847+0000 7f6be6429740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc1ad580 con 0x56110eb72810 2024-03-21T11:27:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.847+0000 7f6be6429740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc1acf20 con 0x56110eb743a0 2024-03-21T11:27:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.847+0000 7f6be6429740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc1aa420 con 0x56110eb739e0 2024-03-21T11:27:48.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.847+0000 7f6be6429740 10 client.0 _ll_register_callbacks cb 0x7f6bd4009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:48.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.847+0000 7f6be0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb72810 0x56110eb7fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.847+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb800a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.847+0000 7f6be0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb72810 0x56110eb7fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41180/0 (socket says 172.21.15.47:41180) 2024-03-21T11:27:48.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.847+0000 7f6be0e00640 1 -- 172.21.15.47:0/2218097590 learned_addr learned my addr 172.21.15.47:0/2218097590 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:48.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.847+0000 7f6be1800640 1 --2- 172.21.15.47:0/2218097590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb743a0 0x56110eb830c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.848+0000 7f6be0e00640 1 -- 172.21.15.47:0/2218097590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 msgr2=0x56110eb800a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.848+0000 7f6be0e00640 1 --2- 172.21.15.47:0/2218097590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb800a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.848+0000 7f6be0e00640 1 -- 172.21.15.47:0/2218097590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb743a0 msgr2=0x56110eb830c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:48.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.848+0000 7f6be0e00640 1 --2- 172.21.15.47:0/2218097590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb743a0 0x56110eb830c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.848+0000 7f6be0e00640 1 -- 172.21.15.47:0/2218097590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56110eb836b0 con 0x56110eb72810 2024-03-21T11:27:48.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.848+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/2218097590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb800a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:48.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.848+0000 7f6be0e00640 1 --2- 172.21.15.47:0/2218097590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb72810 0x56110eb7fb60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4005fd0 tx=0x7f6bc4010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b686e101cae9ac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:48.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.848+0000 7f6bcaa00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:27:48.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.848+0000 7f6bcaa00640 1 -- 172.21.15.47:0/2218097590 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4039070 con 0x56110eb72810 2024-03-21T11:27:48.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.849+0000 7f6bcaa00640 1 -- 172.21.15.47:0/2218097590 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6bc401c900 con 0x56110eb72810 2024-03-21T11:27:48.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.849+0000 7f6be6429740 1 -- 172.21.15.47:0/2218097590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bc4031070 con 0x56110eb72810 2024-03-21T11:27:48.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.849+0000 7f6bcaa00640 1 -- 172.21.15.47:0/2218097590 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bc4045050 con 0x56110eb72810 2024-03-21T11:27:48.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.849+0000 7f6be6429740 10 client.16734 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.849+0000 7f6be6429740 1 -- 172.21.15.47:0/2218097590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56110ebe4430 con 0x56110eb72810 2024-03-21T11:27:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.849+0000 7f6bcaa00640 1 -- 172.21.15.47:0/2218097590 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bc4031070 con 0x56110eb72810 2024-03-21T11:27:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.850+0000 7f6bcaa00640 1 -- 172.21.15.47:0/2218097590 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6bc402cab0 con 0x56110eb72810 2024-03-21T11:27:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.850+0000 7f6be6429740 10 client.16734 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.850+0000 7f6be6429740 10 client.16734 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:27:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.850+0000 7f6be6429740 20 client.16734 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.850+0000 7f6be6429740 1 --2- 172.21.15.47:0/2218097590 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56110ebe4ae0 0x56110ebe6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.850+0000 7f6be6429740 4 client.16734 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:48.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.850+0000 7f6be6429740 1 -- 172.21.15.47:0/2218097590 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56110eadeb50 con 0x56110ebe4ae0 2024-03-21T11:27:48.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.850+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/2218097590 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56110ebe4ae0 0x56110ebe6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:48.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.850+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/2218097590 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56110ebe4ae0 0x56110ebe6ed0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e246e625b07c46 server_cookie=bf7f35c3f34ba846 in_seq=0 out_seq=0 2024-03-21T11:27:48.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.850+0000 7f6bcaa00640 10 client.16734 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:27:48.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6bcaa00640 1 -- 172.21.15.47:0/2218097590 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56110eadeb50 con 0x56110ebe4ae0 2024-03-21T11:27:48.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6bcaa00640 10 client.16734 handle_command_reply: tid=0 2024-03-21T11:27:48.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6be6429740 1 client.16734 shutdown 2024-03-21T11:27:48.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6be6429740 1 -- 172.21.15.47:0/2218097590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb72810 msgr2=0x56110eb7fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6be6429740 1 --2- 172.21.15.47:0/2218097590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb72810 0x56110eb7fb60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4005fd0 tx=0x7f6bc4010d50 comp rx=0 tx=0).stop 2024-03-21T11:27:48.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6be6429740 1 -- 172.21.15.47:0/2218097590 shutdown_connections 2024-03-21T11:27:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6be6429740 1 --2- 172.21.15.47:0/2218097590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56110eb743a0 0x56110eb830c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6be6429740 1 --2- 172.21.15.47:0/2218097590 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56110ebe4ae0 0x56110ebe6ed0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6be6429740 1 --2- 172.21.15.47:0/2218097590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56110eb739e0 0x56110eb800a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6be6429740 1 --2- 172.21.15.47:0/2218097590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56110eb72810 0x56110eb7fb60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6bcaa00640 0 client.16734 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:27:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6be6429740 1 -- 172.21.15.47:0/2218097590 >> 172.21.15.47:0/2218097590 conn(0x56110eb6dc00 msgr2=0x56110eb6c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6be6429740 1 -- 172.21.15.47:0/2218097590 shutdown_connections 2024-03-21T11:27:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6be6429740 1 -- 172.21.15.47:0/2218097590 wait complete. 2024-03-21T11:27:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.851+0000 7f6be6429740 20 client.16734 trim_cache size 0 max 16384 2024-03-21T11:27:48.861 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.852+0000 7f6be3800640 1 -- 172.21.15.47:0/1220541078 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bbc0a2fa0 msgr2=0x7f6bbc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.852+0000 7f6be3800640 1 --2- 172.21.15.47:0/1220541078 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bbc0a2fa0 0x7f6bbc0a5460 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x56110eb738e0 tx=0x7f6bd4009600 comp rx=0 tx=0).stop 2024-03-21T11:27:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.852+0000 7f6be3800640 1 -- 172.21.15.47:0/1220541078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc104ad0 msgr2=0x7f6bdc19a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.852+0000 7f6be3800640 1 --2- 172.21.15.47:0/1220541078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc104ad0 0x7f6bdc19a130 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009230 tx=0x7f6bcc005dd0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.852+0000 7f6be3800640 1 -- 172.21.15.47:0/1220541078 shutdown_connections 2024-03-21T11:27:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.852+0000 7f6be3800640 1 --2- 172.21.15.47:0/1220541078 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bbc0a2fa0 0x7f6bbc0a5460 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.852+0000 7f6be3800640 1 --2- 172.21.15.47:0/1220541078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bdc105490 0x7f6bdc1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.852+0000 7f6be3800640 1 --2- 172.21.15.47:0/1220541078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc104ad0 0x7f6bdc19a130 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.852+0000 7f6be3800640 1 --2- 172.21.15.47:0/1220541078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bdc1038e0 0x7f6bdc199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.852+0000 7f6be3800640 1 -- 172.21.15.47:0/1220541078 >> 172.21.15.47:0/1220541078 conn(0x7f6bdc0fee50 msgr2=0x7f6bdc107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.852+0000 7f6be3800640 1 -- 172.21.15.47:0/1220541078 shutdown_connections 2024-03-21T11:27:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:48.852+0000 7f6be3800640 1 -- 172.21.15.47:0/1220541078 wait complete. 2024-03-21T11:27:48.874 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:48.874 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:27:49.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.088+0000 7fd1c4400640 1 -- 172.21.15.47:0/3092071918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1bc076060 msgr2=0x7fd1bc076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.088+0000 7fd1c4400640 1 --2- 172.21.15.47:0/3092071918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1bc076060 0x7fd1bc076480 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009900 tx=0x7fd1b002bc40 comp rx=0 tx=0).stop 2024-03-21T11:27:49.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.088+0000 7fd1c4400640 1 -- 172.21.15.47:0/3092071918 shutdown_connections 2024-03-21T11:27:49.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.088+0000 7fd1c4400640 1 --2- 172.21.15.47:0/3092071918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1bc077c40 0x7fd1bc0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.088+0000 7fd1c4400640 1 --2- 172.21.15.47:0/3092071918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1bc0772a0 0x7fd1bc077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.088+0000 7fd1c4400640 1 --2- 172.21.15.47:0/3092071918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1bc076060 0x7fd1bc076480 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.088+0000 7fd1c4400640 1 -- 172.21.15.47:0/3092071918 >> 172.21.15.47:0/3092071918 conn(0x7fd1bc0fee50 msgr2=0x7fd1bc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.088+0000 7fd1c4400640 1 -- 172.21.15.47:0/3092071918 shutdown_connections 2024-03-21T11:27:49.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.089+0000 7fd1c4400640 1 -- 172.21.15.47:0/3092071918 wait complete. 2024-03-21T11:27:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.090+0000 7fd1c4400640 1 Processor -- start 2024-03-21T11:27:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.090+0000 7fd1c4400640 1 -- start start 2024-03-21T11:27:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.090+0000 7fd1c4400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1bc076060 0x7fd1bc19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.090+0000 7fd1c4400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1bc0772a0 0x7fd1bc19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.090+0000 7fd1c4400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1bc077c40 0x7fd1bc1a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.090+0000 7fd1c4400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc113970 con 0x7fd1bc076060 2024-03-21T11:27:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.090+0000 7fd1c4400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc1137f0 con 0x7fd1bc0772a0 2024-03-21T11:27:49.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.090+0000 7fd1c4400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc113af0 con 0x7fd1bc077c40 2024-03-21T11:27:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.091+0000 7fd1c1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1bc076060 0x7fd1bc19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.091+0000 7fd1c2400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1bc077c40 0x7fd1bc1a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.091+0000 7fd1c1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1bc076060 0x7fd1bc19dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50018/0 (socket says 172.21.15.47:50018) 2024-03-21T11:27:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.091+0000 7fd1c1a00640 1 -- 172.21.15.47:0/1682210912 learned_addr learned my addr 172.21.15.47:0/1682210912 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.091+0000 7fd1c1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1bc0772a0 0x7fd1bc19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.091+0000 7fd1c1a00640 1 -- 172.21.15.47:0/1682210912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1bc077c40 msgr2=0x7fd1bc1a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.091+0000 7fd1c1a00640 1 --2- 172.21.15.47:0/1682210912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1bc077c40 0x7fd1bc1a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.091+0000 7fd1c1a00640 1 -- 172.21.15.47:0/1682210912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1bc0772a0 msgr2=0x7fd1bc19e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.091+0000 7fd1c1a00640 1 --2- 172.21.15.47:0/1682210912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1bc0772a0 0x7fd1bc19e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.091+0000 7fd1c1a00640 1 -- 172.21.15.47:0/1682210912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc1a5c10 con 0x7fd1bc076060 2024-03-21T11:27:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.091+0000 7fd1c1000640 1 --2- 172.21.15.47:0/1682210912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1bc0772a0 0x7fd1bc19e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.092+0000 7fd1c1a00640 1 --2- 172.21.15.47:0/1682210912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1bc076060 0x7fd1bc19dff0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009230 tx=0x7fd1b0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.092+0000 7fd1ba000640 1 -- 172.21.15.47:0/1682210912 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0041070 con 0x7fd1bc076060 2024-03-21T11:27:49.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.092+0000 7fd1ba000640 1 -- 172.21.15.47:0/1682210912 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1b00044d0 con 0x7fd1bc076060 2024-03-21T11:27:49.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.092+0000 7fd1ba000640 1 -- 172.21.15.47:0/1682210912 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0034440 con 0x7fd1bc076060 2024-03-21T11:27:49.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.092+0000 7fd1c4400640 1 -- 172.21.15.47:0/1682210912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1bc1a5ea0 con 0x7fd1bc076060 2024-03-21T11:27:49.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.092+0000 7fd1c4400640 1 -- 172.21.15.47:0/1682210912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1bc1a62a0 con 0x7fd1bc076060 2024-03-21T11:27:49.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.096+0000 7fd1ba000640 1 -- 172.21.15.47:0/1682210912 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd1b0044070 con 0x7fd1bc076060 2024-03-21T11:27:49.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.098+0000 7fd1ba000640 1 --2- 172.21.15.47:0/1682210912 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1a40a2f20 0x7fd1a40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.098+0000 7fd1ba000640 1 -- 172.21.15.47:0/1682210912 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd1b00f1390 con 0x7fd1bc076060 2024-03-21T11:27:49.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.098+0000 7fd1c7071740 1 Processor -- start 2024-03-21T11:27:49.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.098+0000 7fd1c1000640 1 --2- 172.21.15.47:0/1682210912 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1a40a2f20 0x7fd1a40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.098+0000 7fd1c7071740 1 -- start start 2024-03-21T11:27:49.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c7071740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564763142810 0x564763142c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c7071740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5647631439e0 0x564763143e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c7071740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631443a0 0x56476314ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c1000640 1 --2- 172.21.15.47:0/1682210912 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1a40a2f20 0x7fd1a40a53e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc19f500 tx=0x7fd1a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c7071740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647630a72c0 con 0x5647631439e0 2024-03-21T11:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c7071740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56476304fc90 con 0x564763142810 2024-03-21T11:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c7071740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647630a3360 con 0x5647631443a0 2024-03-21T11:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564763142810 0x564763142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564763142810 0x564763142c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36220/0 (socket says 172.21.15.47:36220) 2024-03-21T11:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c1a00640 1 -- 172.21.15.47:0/2404857532 learned_addr learned my addr 172.21.15.47:0/2404857532 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c2400640 1 --2- 172.21.15.47:0/2404857532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631443a0 0x56476314ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c1000640 1 --2- 172.21.15.47:0/2404857532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5647631439e0 0x564763143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c1a00640 1 -- 172.21.15.47:0/2404857532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631443a0 msgr2=0x56476314ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c1a00640 1 --2- 172.21.15.47:0/2404857532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631443a0 0x56476314ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c1a00640 1 -- 172.21.15.47:0/2404857532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5647631439e0 msgr2=0x564763143e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c1a00640 1 --2- 172.21.15.47:0/2404857532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5647631439e0 0x564763143e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.099+0000 7fd1c1a00640 1 -- 172.21.15.47:0/2404857532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564763074080 con 0x564763142810 2024-03-21T11:27:49.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.100+0000 7fd1c1a00640 1 --2- 172.21.15.47:0/2404857532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564763142810 0x564763142c30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00c5bb0 tx=0x7fd1b002bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=415da464d0323c32 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.100+0000 7fd1aea00640 1 -- 172.21.15.47:0/2404857532 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b004b040 con 0x564763142810 2024-03-21T11:27:49.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.100+0000 7fd1aea00640 1 -- 172.21.15.47:0/2404857532 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1b00f1170 con 0x564763142810 2024-03-21T11:27:49.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.100+0000 7fd1aea00640 1 -- 172.21.15.47:0/2404857532 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b00c5c50 con 0x564763142810 2024-03-21T11:27:49.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.101+0000 7fd1c7071740 1 -- 172.21.15.47:0/2404857532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564763142810 msgr2=0x564763142c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.101+0000 7fd1c7071740 1 --2- 172.21.15.47:0/2404857532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564763142810 0x564763142c30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00c5bb0 tx=0x7fd1b002bc40 comp rx=0 tx=0).stop 2024-03-21T11:27:49.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.101+0000 7fd1c7071740 1 -- 172.21.15.47:0/2404857532 shutdown_connections 2024-03-21T11:27:49.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.101+0000 7fd1c7071740 1 --2- 172.21.15.47:0/2404857532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631443a0 0x56476314ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.101+0000 7fd1c7071740 1 --2- 172.21.15.47:0/2404857532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5647631439e0 0x564763143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.101+0000 7fd1c7071740 1 --2- 172.21.15.47:0/2404857532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564763142810 0x564763142c30 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.101+0000 7fd1c7071740 1 -- 172.21.15.47:0/2404857532 >> 172.21.15.47:0/2404857532 conn(0x56476313dc00 msgr2=0x56476313e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.101+0000 7fd1c7071740 1 -- 172.21.15.47:0/2404857532 shutdown_connections 2024-03-21T11:27:49.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.101+0000 7fd1c7071740 1 -- 172.21.15.47:0/2404857532 wait complete. 2024-03-21T11:27:49.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.101+0000 7fd1c7071740 1 -- start start 2024-03-21T11:27:49.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.102+0000 7fd1c7071740 1 Processor -- start 2024-03-21T11:27:49.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.102+0000 7fd1c7071740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564763142810 0x56476314c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.102+0000 7fd1c7071740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631439e0 0x56476314ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.102+0000 7fd1c7071740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647631443a0 0x564763153d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.102+0000 7fd1c7071740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56476314b170 con 0x564763142810 2024-03-21T11:27:49.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.102+0000 7fd1c7071740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647630630d0 con 0x5647631443a0 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.102+0000 7fd1c7071740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56476314b2f0 con 0x5647631439e0 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.102+0000 7fd1c7071740 10 client.0 _ll_register_callbacks cb 0x564763067e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.103+0000 7fd1c2400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647631443a0 0x564763153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.103+0000 7fd1c1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564763142810 0x56476314c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.103+0000 7fd1c2400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647631443a0 0x564763153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36226/0 (socket says 172.21.15.47:36226) 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.103+0000 7fd1c2400640 1 -- 172.21.15.47:0/2122473821 learned_addr learned my addr 172.21.15.47:0/2122473821 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.103+0000 7fd1c1000640 1 --2- 172.21.15.47:0/2122473821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631439e0 0x56476314ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.103+0000 7fd1c2400640 1 -- 172.21.15.47:0/2122473821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631439e0 msgr2=0x56476314ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.103+0000 7fd1c2400640 1 --2- 172.21.15.47:0/2122473821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631439e0 0x56476314ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.103+0000 7fd1c2400640 1 -- 172.21.15.47:0/2122473821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564763142810 msgr2=0x56476314c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.103+0000 7fd1c2400640 1 --2- 172.21.15.47:0/2122473821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564763142810 0x56476314c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.103+0000 7fd1c2400640 1 -- 172.21.15.47:0/2122473821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564762dbb3f0 con 0x5647631443a0 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.103+0000 7fd1c2400640 1 --2- 172.21.15.47:0/2122473821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647631443a0 0x564763153d30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4013d30 tx=0x7fd1b40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d049f71e0ed92ed0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.103+0000 7fd1a3400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.103+0000 7fd1a3400640 1 -- 172.21.15.47:0/2122473821 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4004d20 con 0x5647631443a0 2024-03-21T11:27:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.104+0000 7fd1a3400640 1 -- 172.21.15.47:0/2122473821 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd1b4003970 con 0x5647631443a0 2024-03-21T11:27:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.104+0000 7fd1c7071740 1 -- 172.21.15.47:0/2122473821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647630b4800 con 0x5647631443a0 2024-03-21T11:27:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.104+0000 7fd1a3400640 1 -- 172.21.15.47:0/2122473821 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1b4003bb0 con 0x5647631443a0 2024-03-21T11:27:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.104+0000 7fd1c7071740 10 client.26993 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.104+0000 7fd1c7071740 1 -- 172.21.15.47:0/2122473821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564762e857d0 con 0x5647631443a0 2024-03-21T11:27:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.104+0000 7fd1a3400640 1 -- 172.21.15.47:0/2122473821 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5647630b4800 con 0x5647631443a0 2024-03-21T11:27:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.104+0000 7fd1a3400640 1 -- 172.21.15.47:0/2122473821 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd1b401a030 con 0x5647631443a0 2024-03-21T11:27:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.104+0000 7fd1c7071740 10 client.26993 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.104+0000 7fd1c7071740 10 client.26993 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:27:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.105+0000 7fd1c7071740 20 client.26993 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.105+0000 7fd1c7071740 1 --2- 172.21.15.47:0/2122473821 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564763158260 0x56476315a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.105+0000 7fd1c7071740 4 client.26993 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.105+0000 7fd1c7071740 1 -- 172.21.15.47:0/2122473821 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647630aeb50 con 0x564763158260 2024-03-21T11:27:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.105+0000 7fd1c1a00640 1 --2- 172.21.15.47:0/2122473821 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564763158260 0x56476315a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.105+0000 7fd1c1a00640 1 --2- 172.21.15.47:0/2122473821 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564763158260 0x56476315a6c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=173956538dc0081c server_cookie=862fa9562d704a0b in_seq=0 out_seq=0 2024-03-21T11:27:49.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.105+0000 7fd1a3400640 10 client.26993 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:27:49.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.106+0000 7fd1a3400640 1 -- 172.21.15.47:0/2122473821 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5647630aeb50 con 0x564763158260 2024-03-21T11:27:49.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.106+0000 7fd1a3400640 10 client.26993 handle_command_reply: tid=0 2024-03-21T11:27:49.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.106+0000 7fd1c7071740 1 client.26993 shutdown 2024-03-21T11:27:49.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.107+0000 7fd1c7071740 1 -- 172.21.15.47:0/2122473821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647631443a0 msgr2=0x564763153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.107+0000 7fd1c7071740 1 --2- 172.21.15.47:0/2122473821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647631443a0 0x564763153d30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4013d30 tx=0x7fd1b40117b0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.107+0000 7fd1c7071740 1 -- 172.21.15.47:0/2122473821 shutdown_connections 2024-03-21T11:27:49.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.107+0000 7fd1c7071740 1 --2- 172.21.15.47:0/2122473821 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564763158260 0x56476315a6c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.107+0000 7fd1c7071740 1 --2- 172.21.15.47:0/2122473821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647631443a0 0x564763153d30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.107+0000 7fd1c7071740 1 --2- 172.21.15.47:0/2122473821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631439e0 0x56476314ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.107+0000 7fd1c7071740 1 --2- 172.21.15.47:0/2122473821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564763142810 0x56476314c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.107+0000 7fd1c7071740 1 -- 172.21.15.47:0/2122473821 >> 172.21.15.47:0/2122473821 conn(0x56476313dc00 msgr2=0x56476313ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.107+0000 7fd1a3400640 0 client.26993 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:27:49.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.107+0000 7fd1c7071740 1 -- 172.21.15.47:0/2122473821 shutdown_connections 2024-03-21T11:27:49.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.107+0000 7fd1c7071740 1 -- 172.21.15.47:0/2122473821 wait complete. 2024-03-21T11:27:49.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.107+0000 7fd1c7071740 20 client.26993 trim_cache size 0 max 16384 2024-03-21T11:27:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.124+0000 7fd1c7071740 1 Processor -- start 2024-03-21T11:27:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.124+0000 7fd1c7071740 1 -- start start 2024-03-21T11:27:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.124+0000 7fd1c7071740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564763142810 0x564763146b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.124+0000 7fd1c7071740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631439e0 0x564763153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.124+0000 7fd1c7071740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647631443a0 0x564763147060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.124+0000 7fd1c7071740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc1b0770 con 0x564763142810 2024-03-21T11:27:49.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.124+0000 7fd1c7071740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc1b20f0 con 0x5647631443a0 2024-03-21T11:27:49.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.124+0000 7fd1c7071740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc1b0dd0 con 0x5647631439e0 2024-03-21T11:27:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.125+0000 7fd1c2400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647631443a0 0x564763147060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.125+0000 7fd1c1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631439e0 0x564763153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.125+0000 7fd1c2400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647631443a0 0x564763147060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36232/0 (socket says 172.21.15.47:36232) 2024-03-21T11:27:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.125+0000 7fd1c2400640 1 -- 172.21.15.47:0/71185760 learned_addr learned my addr 172.21.15.47:0/71185760 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.125+0000 7fd1c1a00640 1 --2- 172.21.15.47:0/71185760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564763142810 0x564763146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.125+0000 7fd1c2400640 1 -- 172.21.15.47:0/71185760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631439e0 msgr2=0x564763153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.125+0000 7fd1c2400640 1 --2- 172.21.15.47:0/71185760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631439e0 0x564763153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.125+0000 7fd1c2400640 1 -- 172.21.15.47:0/71185760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564763142810 msgr2=0x564763146b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.125+0000 7fd1c2400640 1 --2- 172.21.15.47:0/71185760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564763142810 0x564763146b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.125+0000 7fd1c2400640 1 -- 172.21.15.47:0/71185760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56476314a5e0 con 0x5647631443a0 2024-03-21T11:27:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.125+0000 7fd1c2400640 1 --2- 172.21.15.47:0/71185760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647631443a0 0x564763147060 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4005200 tx=0x7fd1b40139a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=659f8fdf06c063f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.126+0000 7fd1ae000640 1 -- 172.21.15.47:0/71185760 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b402c070 con 0x5647631443a0 2024-03-21T11:27:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.126+0000 7fd1ae000640 1 -- 172.21.15.47:0/71185760 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1b4011cb0 con 0x5647631443a0 2024-03-21T11:27:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.126+0000 7fd1ae000640 1 -- 172.21.15.47:0/71185760 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4028ba0 con 0x5647631443a0 2024-03-21T11:27:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.127+0000 7fd1c7071740 1 -- 172.21.15.47:0/71185760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647631443a0 msgr2=0x564763147060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.127+0000 7fd1c7071740 1 --2- 172.21.15.47:0/71185760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647631443a0 0x564763147060 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4005200 tx=0x7fd1b40139a0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.127+0000 7fd1c7071740 1 -- 172.21.15.47:0/71185760 shutdown_connections 2024-03-21T11:27:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.127+0000 7fd1c7071740 1 --2- 172.21.15.47:0/71185760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647631443a0 0x564763147060 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.127+0000 7fd1c7071740 1 --2- 172.21.15.47:0/71185760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631439e0 0x564763153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.127+0000 7fd1c7071740 1 --2- 172.21.15.47:0/71185760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564763142810 0x564763146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.127+0000 7fd1c7071740 1 -- 172.21.15.47:0/71185760 >> 172.21.15.47:0/71185760 conn(0x56476313dc00 msgr2=0x5647631416f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.127+0000 7fd1c7071740 1 -- 172.21.15.47:0/71185760 shutdown_connections 2024-03-21T11:27:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.127+0000 7fd1c7071740 1 -- 172.21.15.47:0/71185760 wait complete. 2024-03-21T11:27:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.127+0000 7fd1c7071740 1 -- start start 2024-03-21T11:27:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.127+0000 7fd1c7071740 1 Processor -- start 2024-03-21T11:27:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c7071740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564763142810 0x56476314f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c7071740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5647631439e0 0x56476314fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c7071740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631443a0 0x564763152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c7071740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc1b1a90 con 0x5647631439e0 2024-03-21T11:27:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c7071740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc1b1430 con 0x564763142810 2024-03-21T11:27:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c7071740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc1ae930 con 0x5647631443a0 2024-03-21T11:27:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c7071740 10 client.0 _ll_register_callbacks cb 0x7fd1b4009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564763142810 0x56476314f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c2400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631443a0 0x564763152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564763142810 0x56476314f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36238/0 (socket says 172.21.15.47:36238) 2024-03-21T11:27:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c2400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631443a0 0x564763152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48986/0 (socket says 172.21.15.47:48986) 2024-03-21T11:27:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c1a00640 1 -- 172.21.15.47:0/1212136595 learned_addr learned my addr 172.21.15.47:0/1212136595 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c1000640 1 --2- 172.21.15.47:0/1212136595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5647631439e0 0x56476314fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c2400640 1 -- 172.21.15.47:0/1212136595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564763142810 msgr2=0x56476314f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c2400640 1 --2- 172.21.15.47:0/1212136595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564763142810 0x56476314f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c2400640 1 -- 172.21.15.47:0/1212136595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5647631439e0 msgr2=0x56476314fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c2400640 1 --2- 172.21.15.47:0/1212136595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5647631439e0 0x56476314fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c2400640 1 -- 172.21.15.47:0/1212136595 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564763153360 con 0x5647631443a0 2024-03-21T11:27:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.128+0000 7fd1c1000640 1 --2- 172.21.15.47:0/1212136595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5647631439e0 0x56476314fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.129+0000 7fd1c2400640 1 --2- 172.21.15.47:0/1212136595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631443a0 0x564763152d70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4006010 tx=0x7fd1b4009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab143b4a4e32be10 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.129+0000 7fd1a3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.129+0000 7fd1a3400640 1 -- 172.21.15.47:0/1212136595 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b402c040 con 0x5647631443a0 2024-03-21T11:27:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.129+0000 7fd1c7071740 1 -- 172.21.15.47:0/1212136595 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1b4011cb0 con 0x5647631443a0 2024-03-21T11:27:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.129+0000 7fd1a3400640 1 -- 172.21.15.47:0/1212136595 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd1b4028d30 con 0x5647631443a0 2024-03-21T11:27:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.129+0000 7fd1a3400640 1 -- 172.21.15.47:0/1212136595 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1b4005320 con 0x5647631443a0 2024-03-21T11:27:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.129+0000 7fd1c7071740 10 client.27013 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.129+0000 7fd1c7071740 1 -- 172.21.15.47:0/1212136595 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5647631b3f40 con 0x5647631443a0 2024-03-21T11:27:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.130+0000 7fd1a3400640 1 -- 172.21.15.47:0/1212136595 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1b4011cb0 con 0x5647631443a0 2024-03-21T11:27:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.130+0000 7fd1a3400640 1 -- 172.21.15.47:0/1212136595 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd1b401a040 con 0x5647631443a0 2024-03-21T11:27:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.130+0000 7fd1c7071740 10 client.27013 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.130+0000 7fd1c7071740 10 client.27013 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:27:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.130+0000 7fd1c7071740 20 client.27013 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.130+0000 7fd1c7071740 1 --2- 172.21.15.47:0/1212136595 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5647631b4760 0x5647631b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.130+0000 7fd1c7071740 4 client.27013 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.130+0000 7fd1c7071740 1 -- 172.21.15.47:0/1212136595 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56476302c500 con 0x5647631b4760 2024-03-21T11:27:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.130+0000 7fd1c1a00640 1 --2- 172.21.15.47:0/1212136595 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5647631b4760 0x5647631b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1c1a00640 1 --2- 172.21.15.47:0/1212136595 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5647631b4760 0x5647631b6ba0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=545da966d0c409e9 server_cookie=888d7b345ef3d24a in_seq=0 out_seq=0 2024-03-21T11:27:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1a3400640 10 client.27013 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:27:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1a3400640 1 -- 172.21.15.47:0/1212136595 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56476302c500 con 0x5647631b4760 2024-03-21T11:27:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1a3400640 10 client.27013 handle_command_reply: tid=0 2024-03-21T11:27:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1c7071740 1 client.27013 shutdown 2024-03-21T11:27:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1c7071740 1 -- 172.21.15.47:0/1212136595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631443a0 msgr2=0x564763152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1c7071740 1 --2- 172.21.15.47:0/1212136595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631443a0 0x564763152d70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4006010 tx=0x7fd1b4009640 comp rx=0 tx=0).stop 2024-03-21T11:27:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1c7071740 1 -- 172.21.15.47:0/1212136595 shutdown_connections 2024-03-21T11:27:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1c7071740 1 --2- 172.21.15.47:0/1212136595 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5647631b4760 0x5647631b6ba0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1c7071740 1 --2- 172.21.15.47:0/1212136595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5647631443a0 0x564763152d70 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1c7071740 1 --2- 172.21.15.47:0/1212136595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5647631439e0 0x56476314fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1c7071740 1 --2- 172.21.15.47:0/1212136595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564763142810 0x56476314f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1c7071740 1 -- 172.21.15.47:0/1212136595 >> 172.21.15.47:0/1212136595 conn(0x56476313dc00 msgr2=0x56476313c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1a3400640 0 client.27013 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:27:49.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1c7071740 1 -- 172.21.15.47:0/1212136595 shutdown_connections 2024-03-21T11:27:49.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.131+0000 7fd1c7071740 1 -- 172.21.15.47:0/1212136595 wait complete. 2024-03-21T11:27:49.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.132+0000 7fd1c7071740 20 client.27013 trim_cache size 0 max 16384 2024-03-21T11:27:49.143 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:49.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.132+0000 7fd1c4400640 1 -- 172.21.15.47:0/1682210912 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1a40a2f20 msgr2=0x7fd1a40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.132+0000 7fd1c4400640 1 --2- 172.21.15.47:0/1682210912 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1a40a2f20 0x7fd1a40a53e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc19f500 tx=0x7fd1a8005f90 comp rx=0 tx=0).stop 2024-03-21T11:27:49.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.132+0000 7fd1c4400640 1 -- 172.21.15.47:0/1682210912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1bc076060 msgr2=0x7fd1bc19dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.132+0000 7fd1c4400640 1 --2- 172.21.15.47:0/1682210912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1bc076060 0x7fd1bc19dff0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009230 tx=0x7fd1b0005d50 comp rx=0 tx=0).stop 2024-03-21T11:27:49.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.132+0000 7fd1c4400640 1 -- 172.21.15.47:0/1682210912 shutdown_connections 2024-03-21T11:27:49.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.132+0000 7fd1c4400640 1 --2- 172.21.15.47:0/1682210912 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1a40a2f20 0x7fd1a40a53e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.132+0000 7fd1c4400640 1 --2- 172.21.15.47:0/1682210912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1bc077c40 0x7fd1bc1a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.132+0000 7fd1c4400640 1 --2- 172.21.15.47:0/1682210912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1bc0772a0 0x7fd1bc19e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.132+0000 7fd1c4400640 1 --2- 172.21.15.47:0/1682210912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1bc076060 0x7fd1bc19dff0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.132+0000 7fd1c4400640 1 -- 172.21.15.47:0/1682210912 >> 172.21.15.47:0/1682210912 conn(0x7fd1bc0fee50 msgr2=0x7fd1bc1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.133+0000 7fd1c4400640 1 -- 172.21.15.47:0/1682210912 shutdown_connections 2024-03-21T11:27:49.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.133+0000 7fd1c4400640 1 -- 172.21.15.47:0/1682210912 wait complete. 2024-03-21T11:27:49.151 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:27:49.151 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:27:49.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:48 smithi047 ceph-mon[31351]: pgmap v534: 129 pgs: 129 active+clean; 37 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 322 op/s 2024-03-21T11:27:49.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:48 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1095409361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:27:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:48 smithi102 ceph-mon[36748]: pgmap v534: 129 pgs: 129 active+clean; 37 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 322 op/s 2024-03-21T11:27:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:48 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1095409361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:27:49.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.355+0000 7f6504200640 1 -- 172.21.15.47:0/1808495839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc10dc80 msgr2=0x7f64fc110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.355+0000 7f6504200640 1 --2- 172.21.15.47:0/1808495839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc10dc80 0x7f64fc110070 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009930 tx=0x7f64f402bc30 comp rx=0 tx=0).stop 2024-03-21T11:27:49.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.355+0000 7f6504200640 1 -- 172.21.15.47:0/1808495839 shutdown_connections 2024-03-21T11:27:49.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.355+0000 7f6504200640 1 --2- 172.21.15.47:0/1808495839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc10dc80 0x7f64fc110070 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.355+0000 7f6504200640 1 --2- 172.21.15.47:0/1808495839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64fc074100 0x7f64fc074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.355+0000 7f6504200640 1 --2- 172.21.15.47:0/1808495839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64fc0756e0 0x7f64fc073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.355+0000 7f6504200640 1 -- 172.21.15.47:0/1808495839 >> 172.21.15.47:0/1808495839 conn(0x7f64fc0fcc40 msgr2=0x7f64fc0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.356+0000 7f6504200640 1 -- 172.21.15.47:0/1808495839 shutdown_connections 2024-03-21T11:27:49.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.356+0000 7f6504200640 1 -- 172.21.15.47:0/1808495839 wait complete. 2024-03-21T11:27:49.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.357+0000 7f6504200640 1 Processor -- start 2024-03-21T11:27:49.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.357+0000 7f6504200640 1 -- start start 2024-03-21T11:27:49.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.357+0000 7f6504200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64fc074100 0x7f64fc19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.357+0000 7f6504200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0756e0 0x7f64fc19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.357+0000 7f6504200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64fc10dc80 0x7f64fc1a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.357+0000 7f6504200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc113950 con 0x7f64fc10dc80 2024-03-21T11:27:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.357+0000 7f6504200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1137d0 con 0x7f64fc0756e0 2024-03-21T11:27:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.357+0000 7f6504200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc113ad0 con 0x7f64fc074100 2024-03-21T11:27:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.357+0000 7f6501800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64fc074100 0x7f64fc19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.357+0000 7f6500e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0756e0 0x7f64fc19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.357+0000 7f6501800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64fc074100 0x7f64fc19dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49002/0 (socket says 172.21.15.47:49002) 2024-03-21T11:27:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.357+0000 7f6501800640 1 -- 172.21.15.47:0/3922882874 learned_addr learned my addr 172.21.15.47:0/3922882874 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.357+0000 7f6502200640 1 --2- 172.21.15.47:0/3922882874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64fc10dc80 0x7f64fc1a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.358+0000 7f6500e00640 1 -- 172.21.15.47:0/3922882874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64fc074100 msgr2=0x7f64fc19dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.358+0000 7f6500e00640 1 --2- 172.21.15.47:0/3922882874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64fc074100 0x7f64fc19dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.358+0000 7f6500e00640 1 -- 172.21.15.47:0/3922882874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64fc10dc80 msgr2=0x7f64fc1a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.358+0000 7f6500e00640 1 --2- 172.21.15.47:0/3922882874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64fc10dc80 0x7f64fc1a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.358+0000 7f6500e00640 1 -- 172.21.15.47:0/3922882874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc1a5bc0 con 0x7f64fc0756e0 2024-03-21T11:27:49.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.358+0000 7f6501800640 1 --2- 172.21.15.47:0/3922882874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64fc074100 0x7f64fc19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:49.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.358+0000 7f6500e00640 1 --2- 172.21.15.47:0/3922882874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0756e0 0x7f64fc19e4e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f64e800e9f0 tx=0x7f64e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.359+0000 7f64fa000640 1 -- 172.21.15.47:0/3922882874 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e800ccc0 con 0x7f64fc0756e0 2024-03-21T11:27:49.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.359+0000 7f64fa000640 1 -- 172.21.15.47:0/3922882874 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64e8010910 con 0x7f64fc0756e0 2024-03-21T11:27:49.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.359+0000 7f6504200640 1 -- 172.21.15.47:0/3922882874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc1a5eb0 con 0x7f64fc0756e0 2024-03-21T11:27:49.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.359+0000 7f64fa000640 1 -- 172.21.15.47:0/3922882874 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e8018960 con 0x7f64fc0756e0 2024-03-21T11:27:49.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.359+0000 7f6504200640 1 -- 172.21.15.47:0/3922882874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc1a63c0 con 0x7f64fc0756e0 2024-03-21T11:27:49.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.362+0000 7f64fa000640 1 -- 172.21.15.47:0/3922882874 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f64e8010ac0 con 0x7f64fc0756e0 2024-03-21T11:27:49.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.362+0000 7f6506e48740 1 Processor -- start 2024-03-21T11:27:49.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.362+0000 7f6506e48740 1 -- start start 2024-03-21T11:27:49.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.362+0000 7f6506e48740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb61810 0x55c6efb61c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.362+0000 7f6506e48740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb629e0 0x55c6efb62e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.362+0000 7f6506e48740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb633a0 0x55c6efb69c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.363+0000 7f6506e48740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6efac62c0 con 0x55c6efb61810 2024-03-21T11:27:49.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.363+0000 7f6506e48740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6efa6ec90 con 0x55c6efb633a0 2024-03-21T11:27:49.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.363+0000 7f6506e48740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6efac2360 con 0x55c6efb629e0 2024-03-21T11:27:49.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.363+0000 7f6501800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb629e0 0x55c6efb62e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.363+0000 7f6500e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb633a0 0x55c6efb69c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.363+0000 7f6501800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb629e0 0x55c6efb62e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49014/0 (socket says 172.21.15.47:49014) 2024-03-21T11:27:49.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.363+0000 7f6501800640 1 -- 172.21.15.47:0/2847592157 learned_addr learned my addr 172.21.15.47:0/2847592157 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.363+0000 7f6502200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb61810 0x55c6efb61c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.363+0000 7f6501800640 1 -- 172.21.15.47:0/2847592157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb633a0 msgr2=0x55c6efb69c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.363+0000 7f6501800640 1 --2- 172.21.15.47:0/2847592157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb633a0 0x55c6efb69c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.363+0000 7f6501800640 1 -- 172.21.15.47:0/2847592157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb61810 msgr2=0x55c6efb61c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.363+0000 7f6501800640 1 --2- 172.21.15.47:0/2847592157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb61810 0x55c6efb61c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.363+0000 7f6501800640 1 -- 172.21.15.47:0/2847592157 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6efa93080 con 0x55c6efb629e0 2024-03-21T11:27:49.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.364+0000 7f6501800640 1 --2- 172.21.15.47:0/2847592157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb629e0 0x55c6efb62e60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f64f00118b0 tx=0x7f64f0011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b2f80b1ca97bdff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.364+0000 7f64fa000640 1 --2- 172.21.15.47:0/3922882874 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f64e40a2fa0 0x7f64e40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.364+0000 7f64fa000640 1 -- 172.21.15.47:0/3922882874 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f64e80cfa30 con 0x7f64fc0756e0 2024-03-21T11:27:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.364+0000 7f64eea00640 1 -- 172.21.15.47:0/2847592157 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f000fc00 con 0x55c6efb629e0 2024-03-21T11:27:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.364+0000 7f6502200640 1 --2- 172.21.15.47:0/3922882874 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f64e40a2fa0 0x7f64e40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.364+0000 7f64eea00640 1 -- 172.21.15.47:0/2847592157 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64f0016070 con 0x55c6efb629e0 2024-03-21T11:27:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.364+0000 7f64eea00640 1 -- 172.21.15.47:0/2847592157 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f00126b0 con 0x55c6efb629e0 2024-03-21T11:27:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.365+0000 7f6506e48740 1 -- 172.21.15.47:0/2847592157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb629e0 msgr2=0x55c6efb62e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.365+0000 7f6506e48740 1 --2- 172.21.15.47:0/2847592157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb629e0 0x55c6efb62e60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f64f00118b0 tx=0x7f64f0011d70 comp rx=0 tx=0).stop 2024-03-21T11:27:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.365+0000 7f6502200640 1 --2- 172.21.15.47:0/3922882874 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f64e40a2fa0 0x7f64e40a5460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f64f402c7b0 tx=0x7f64f402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.365+0000 7f6506e48740 1 -- 172.21.15.47:0/2847592157 shutdown_connections 2024-03-21T11:27:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.365+0000 7f6506e48740 1 --2- 172.21.15.47:0/2847592157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb633a0 0x55c6efb69c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.365+0000 7f6506e48740 1 --2- 172.21.15.47:0/2847592157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb629e0 0x55c6efb62e60 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.365+0000 7f6506e48740 1 --2- 172.21.15.47:0/2847592157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb61810 0x55c6efb61c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.365+0000 7f6506e48740 1 -- 172.21.15.47:0/2847592157 >> 172.21.15.47:0/2847592157 conn(0x55c6efb5cc00 msgr2=0x55c6efb5d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.365+0000 7f6506e48740 1 -- 172.21.15.47:0/2847592157 shutdown_connections 2024-03-21T11:27:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.365+0000 7f6506e48740 1 -- 172.21.15.47:0/2847592157 wait complete. 2024-03-21T11:27:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.365+0000 7f6506e48740 1 -- start start 2024-03-21T11:27:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.366+0000 7f6506e48740 1 Processor -- start 2024-03-21T11:27:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.366+0000 7f6506e48740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb61810 0x55c6efb6bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.366+0000 7f6506e48740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 0x55c6efb6bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.366+0000 7f6506e48740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb633a0 0x55c6efb72fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.366+0000 7f6506e48740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6efb6a3b0 con 0x55c6efb629e0 2024-03-21T11:27:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.366+0000 7f6506e48740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6efa820d0 con 0x55c6efb61810 2024-03-21T11:27:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.366+0000 7f6506e48740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6efb6a530 con 0x55c6efb633a0 2024-03-21T11:27:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.366+0000 7f6506e48740 10 client.0 _ll_register_callbacks cb 0x55c6efa86e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.366+0000 7f6502200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb633a0 0x55c6efb72fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.366+0000 7f6501800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb61810 0x55c6efb6bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.366+0000 7f6500e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 0x55c6efb6bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.366+0000 7f6502200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb633a0 0x55c6efb72fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49026/0 (socket says 172.21.15.47:49026) 2024-03-21T11:27:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.366+0000 7f6502200640 1 -- 172.21.15.47:0/483596286 learned_addr learned my addr 172.21.15.47:0/483596286 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.366+0000 7f6501800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb61810 0x55c6efb6bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36300/0 (socket says 172.21.15.47:36300) 2024-03-21T11:27:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.367+0000 7f6501800640 1 -- 172.21.15.47:0/483596286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb633a0 msgr2=0x55c6efb72fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.367+0000 7f6501800640 1 --2- 172.21.15.47:0/483596286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb633a0 0x55c6efb72fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.367+0000 7f6501800640 1 -- 172.21.15.47:0/483596286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 msgr2=0x55c6efb6bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.367+0000 7f6501800640 1 --2- 172.21.15.47:0/483596286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 0x55c6efb6bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.367+0000 7f6501800640 1 -- 172.21.15.47:0/483596286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6ef7da3f0 con 0x55c6efb61810 2024-03-21T11:27:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.367+0000 7f6500e00640 1 --2- 172.21.15.47:0/483596286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 0x55c6efb6bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.367+0000 7f6502200640 1 --2- 172.21.15.47:0/483596286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb633a0 0x55c6efb72fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.367+0000 7f6501800640 1 --2- 172.21.15.47:0/483596286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb61810 0x55c6efb6bab0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f64f0011880 tx=0x7f64f0009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c65f6322276c9f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.367+0000 7f64e3400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.368+0000 7f64e3400640 1 -- 172.21.15.47:0/483596286 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f000fa00 con 0x55c6efb61810 2024-03-21T11:27:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.368+0000 7f64e3400640 1 -- 172.21.15.47:0/483596286 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f64f000fba0 con 0x55c6efb61810 2024-03-21T11:27:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.368+0000 7f6506e48740 1 -- 172.21.15.47:0/483596286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64f0016070 con 0x55c6efb61810 2024-03-21T11:27:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.368+0000 7f64e3400640 1 -- 172.21.15.47:0/483596286 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64f000fda0 con 0x55c6efb61810 2024-03-21T11:27:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.368+0000 7f6506e48740 10 client.27011 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.368+0000 7f6506e48740 1 -- 172.21.15.47:0/483596286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c6ef8a47d0 con 0x55c6efb61810 2024-03-21T11:27:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.368+0000 7f64e3400640 1 -- 172.21.15.47:0/483596286 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64f0016070 con 0x55c6efb61810 2024-03-21T11:27:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.369+0000 7f64e3400640 1 -- 172.21.15.47:0/483596286 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f64f000f040 con 0x55c6efb61810 2024-03-21T11:27:49.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.369+0000 7f6506e48740 10 client.27011 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:49.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.369+0000 7f6506e48740 10 client.27011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:49.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.369+0000 7f6506e48740 20 client.27011 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:49.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.369+0000 7f6506e48740 1 --2- 172.21.15.47:0/483596286 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c6efb77120 0x55c6efb79510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.369+0000 7f6506e48740 4 client.27011 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:49.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.369+0000 7f6506e48740 1 -- 172.21.15.47:0/483596286 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6efad3800 con 0x55c6efb77120 2024-03-21T11:27:49.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.369+0000 7f6500e00640 1 --2- 172.21.15.47:0/483596286 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c6efb77120 0x55c6efb79510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.370+0000 7f6500e00640 1 --2- 172.21.15.47:0/483596286 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c6efb77120 0x55c6efb79510 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fefb2b543bcdaf7 server_cookie=7567ff5fa7bf70c7 in_seq=0 out_seq=0 2024-03-21T11:27:49.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.370+0000 7f64e3400640 10 client.27011 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:49.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.370+0000 7f64e3400640 1 -- 172.21.15.47:0/483596286 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c6efad3800 con 0x55c6efb77120 2024-03-21T11:27:49.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.370+0000 7f64e3400640 10 client.27011 handle_command_reply: tid=0 2024-03-21T11:27:49.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.370+0000 7f6506e48740 1 client.27011 shutdown 2024-03-21T11:27:49.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.371+0000 7f6506e48740 1 -- 172.21.15.47:0/483596286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb61810 msgr2=0x55c6efb6bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.371+0000 7f6506e48740 1 --2- 172.21.15.47:0/483596286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb61810 0x55c6efb6bab0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f64f0011880 tx=0x7f64f0009e80 comp rx=0 tx=0).stop 2024-03-21T11:27:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.371+0000 7f6506e48740 1 -- 172.21.15.47:0/483596286 shutdown_connections 2024-03-21T11:27:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.371+0000 7f6506e48740 1 --2- 172.21.15.47:0/483596286 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c6efb77120 0x55c6efb79510 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.371+0000 7f6506e48740 1 --2- 172.21.15.47:0/483596286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb633a0 0x55c6efb72fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.371+0000 7f6506e48740 1 --2- 172.21.15.47:0/483596286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 0x55c6efb6bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.371+0000 7f6506e48740 1 --2- 172.21.15.47:0/483596286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb61810 0x55c6efb6bab0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.371+0000 7f64e3400640 0 client.27011 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.371+0000 7f6506e48740 1 -- 172.21.15.47:0/483596286 >> 172.21.15.47:0/483596286 conn(0x55c6efb5cc00 msgr2=0x55c6efb5beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.371+0000 7f6506e48740 1 -- 172.21.15.47:0/483596286 shutdown_connections 2024-03-21T11:27:49.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.371+0000 7f6506e48740 1 -- 172.21.15.47:0/483596286 wait complete. 2024-03-21T11:27:49.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.371+0000 7f6506e48740 20 client.27011 trim_cache size 0 max 16384 2024-03-21T11:27:49.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.385+0000 7f6506e48740 1 Processor -- start 2024-03-21T11:27:49.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.385+0000 7f6506e48740 1 -- start start 2024-03-21T11:27:49.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6506e48740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb61810 0x55c6efb6a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6506e48740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 0x55c6efb72f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6506e48740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb633a0 0x55c6efb5cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6506e48740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1b0820 con 0x55c6efb629e0 2024-03-21T11:27:49.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6506e48740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1b21a0 con 0x55c6efb61810 2024-03-21T11:27:49.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6506e48740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1b0e80 con 0x55c6efb633a0 2024-03-21T11:27:49.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6501800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb61810 0x55c6efb6a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6502200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb633a0 0x55c6efb5cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6501800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb61810 0x55c6efb6a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36312/0 (socket says 172.21.15.47:36312) 2024-03-21T11:27:49.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6502200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb633a0 0x55c6efb5cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49034/0 (socket says 172.21.15.47:49034) 2024-03-21T11:27:49.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6501800640 1 -- 172.21.15.47:0/4164611213 learned_addr learned my addr 172.21.15.47:0/4164611213 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6500e00640 1 --2- 172.21.15.47:0/4164611213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 0x55c6efb72f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6502200640 1 -- 172.21.15.47:0/4164611213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb61810 msgr2=0x55c6efb6a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6502200640 1 --2- 172.21.15.47:0/4164611213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb61810 0x55c6efb6a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6502200640 1 -- 172.21.15.47:0/4164611213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 msgr2=0x55c6efb72f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6502200640 1 --2- 172.21.15.47:0/4164611213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 0x55c6efb72f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.386+0000 7f6502200640 1 -- 172.21.15.47:0/4164611213 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6efb79880 con 0x55c6efb633a0 2024-03-21T11:27:49.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.387+0000 7f6502200640 1 --2- 172.21.15.47:0/4164611213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb633a0 0x55c6efb5cfe0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f64f402c8e0 tx=0x7f64f4042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f8a0a6b013d8eff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.387+0000 7f64ee000640 1 -- 172.21.15.47:0/4164611213 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4046070 con 0x55c6efb633a0 2024-03-21T11:27:49.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.387+0000 7f64ee000640 1 -- 172.21.15.47:0/4164611213 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64f4004ba0 con 0x55c6efb633a0 2024-03-21T11:27:49.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.387+0000 7f64ee000640 1 -- 172.21.15.47:0/4164611213 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f40035f0 con 0x55c6efb633a0 2024-03-21T11:27:49.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.388+0000 7f6506e48740 1 -- 172.21.15.47:0/4164611213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb633a0 msgr2=0x55c6efb5cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.388+0000 7f6506e48740 1 --2- 172.21.15.47:0/4164611213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb633a0 0x55c6efb5cfe0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f64f402c8e0 tx=0x7f64f4042000 comp rx=0 tx=0).stop 2024-03-21T11:27:49.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.388+0000 7f6506e48740 1 -- 172.21.15.47:0/4164611213 shutdown_connections 2024-03-21T11:27:49.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.388+0000 7f6506e48740 1 --2- 172.21.15.47:0/4164611213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb633a0 0x55c6efb5cfe0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.388+0000 7f6506e48740 1 --2- 172.21.15.47:0/4164611213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 0x55c6efb72f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.388+0000 7f6506e48740 1 --2- 172.21.15.47:0/4164611213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb61810 0x55c6efb6a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.388+0000 7f6506e48740 1 -- 172.21.15.47:0/4164611213 >> 172.21.15.47:0/4164611213 conn(0x55c6efb5cc00 msgr2=0x55c6efb606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.388+0000 7f6506e48740 1 -- 172.21.15.47:0/4164611213 shutdown_connections 2024-03-21T11:27:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.388+0000 7f6506e48740 1 -- 172.21.15.47:0/4164611213 wait complete. 2024-03-21T11:27:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.388+0000 7f6506e48740 1 -- start start 2024-03-21T11:27:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.389+0000 7f6506e48740 1 Processor -- start 2024-03-21T11:27:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.389+0000 7f6506e48740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb61810 0x55c6efb6eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.389+0000 7f6506e48740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 0x55c6efb6efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.389+0000 7f6506e48740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb633a0 0x55c6efb72000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.389+0000 7f6506e48740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1b1b40 con 0x55c6efb629e0 2024-03-21T11:27:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.389+0000 7f6506e48740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1b14e0 con 0x55c6efb633a0 2024-03-21T11:27:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.389+0000 7f6506e48740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1ae9e0 con 0x55c6efb61810 2024-03-21T11:27:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.389+0000 7f6506e48740 10 client.0 _ll_register_callbacks cb 0x7f64f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.389+0000 7f6501800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb61810 0x55c6efb6eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.389+0000 7f6501800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb61810 0x55c6efb6eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49038/0 (socket says 172.21.15.47:49038) 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.389+0000 7f6501800640 1 -- 172.21.15.47:0/2371368712 learned_addr learned my addr 172.21.15.47:0/2371368712 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.389+0000 7f6502200640 1 --2- 172.21.15.47:0/2371368712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb633a0 0x55c6efb72000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.389+0000 7f6500e00640 1 --2- 172.21.15.47:0/2371368712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 0x55c6efb6efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.390+0000 7f6501800640 1 -- 172.21.15.47:0/2371368712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb633a0 msgr2=0x55c6efb72000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.390+0000 7f6501800640 1 --2- 172.21.15.47:0/2371368712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb633a0 0x55c6efb72000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.390+0000 7f6501800640 1 -- 172.21.15.47:0/2371368712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 msgr2=0x55c6efb6efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.390+0000 7f6501800640 1 --2- 172.21.15.47:0/2371368712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 0x55c6efb6efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.390+0000 7f6501800640 1 -- 172.21.15.47:0/2371368712 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6efb725f0 con 0x55c6efb61810 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.390+0000 7f6501800640 1 --2- 172.21.15.47:0/2371368712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb61810 0x55c6efb6eaa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f64f001ba60 tx=0x7f64f0011700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9db303c630b0182f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.390+0000 7f64e3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.390+0000 7f64e3400640 1 -- 172.21.15.47:0/2371368712 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f003e060 con 0x55c6efb61810 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.390+0000 7f64e3400640 1 -- 172.21.15.47:0/2371368712 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f64f001bb60 con 0x55c6efb61810 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.390+0000 7f6506e48740 1 -- 172.21.15.47:0/2371368712 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64f4004ba0 con 0x55c6efb61810 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.390+0000 7f64e3400640 1 -- 172.21.15.47:0/2371368712 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64f0045050 con 0x55c6efb61810 2024-03-21T11:27:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.390+0000 7f6506e48740 10 client.27031 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.390+0000 7f6506e48740 1 -- 172.21.15.47:0/2371368712 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c6efbd2ce0 con 0x55c6efb61810 2024-03-21T11:27:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.391+0000 7f64e3400640 1 -- 172.21.15.47:0/2371368712 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64f4004ba0 con 0x55c6efb61810 2024-03-21T11:27:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.391+0000 7f64e3400640 1 -- 172.21.15.47:0/2371368712 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f64f0003970 con 0x55c6efb61810 2024-03-21T11:27:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.391+0000 7f6506e48740 10 client.27031 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.391+0000 7f6506e48740 10 client.27031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.391+0000 7f6506e48740 20 client.27031 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.391+0000 7f6506e48740 1 --2- 172.21.15.47:0/2371368712 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c6efbd3520 0x55c6efbd5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.391+0000 7f6506e48740 4 client.27031 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:27:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.391+0000 7f6506e48740 1 -- 172.21.15.47:0/2371368712 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6efacdb50 con 0x55c6efbd3520 2024-03-21T11:27:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.391+0000 7f6500e00640 1 --2- 172.21.15.47:0/2371368712 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c6efbd3520 0x55c6efbd5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f6500e00640 1 --2- 172.21.15.47:0/2371368712 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c6efbd3520 0x55c6efbd5960 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d33e3c01e8bc72a server_cookie=c81cad5885030b2 in_seq=0 out_seq=0 2024-03-21T11:27:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f64e3400640 10 client.27031 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f64e3400640 1 -- 172.21.15.47:0/2371368712 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6efacdb50 con 0x55c6efbd3520 2024-03-21T11:27:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f64e3400640 10 client.27031 handle_command_reply: tid=0 2024-03-21T11:27:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f6506e48740 1 client.27031 shutdown 2024-03-21T11:27:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f6506e48740 1 -- 172.21.15.47:0/2371368712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb61810 msgr2=0x55c6efb6eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f6506e48740 1 --2- 172.21.15.47:0/2371368712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb61810 0x55c6efb6eaa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f64f001ba60 tx=0x7f64f0011700 comp rx=0 tx=0).stop 2024-03-21T11:27:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f6506e48740 1 -- 172.21.15.47:0/2371368712 shutdown_connections 2024-03-21T11:27:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f6506e48740 1 --2- 172.21.15.47:0/2371368712 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c6efbd3520 0x55c6efbd5960 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f6506e48740 1 --2- 172.21.15.47:0/2371368712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6efb633a0 0x55c6efb72000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f6506e48740 1 --2- 172.21.15.47:0/2371368712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6efb629e0 0x55c6efb6efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f6506e48740 1 --2- 172.21.15.47:0/2371368712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6efb61810 0x55c6efb6eaa0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f64e3400640 0 client.27031 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f6506e48740 1 -- 172.21.15.47:0/2371368712 >> 172.21.15.47:0/2371368712 conn(0x55c6efb5cc00 msgr2=0x55c6efb606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f6506e48740 1 -- 172.21.15.47:0/2371368712 shutdown_connections 2024-03-21T11:27:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.392+0000 7f6506e48740 1 -- 172.21.15.47:0/2371368712 wait complete. 2024-03-21T11:27:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.393+0000 7f6506e48740 20 client.27031 trim_cache size 0 max 16384 2024-03-21T11:27:49.399 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:27:49.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.393+0000 7f6504200640 1 -- 172.21.15.47:0/3922882874 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f64e40a2fa0 msgr2=0x7f64e40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.393+0000 7f6504200640 1 --2- 172.21.15.47:0/3922882874 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f64e40a2fa0 0x7f64e40a5460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f64f402c7b0 tx=0x7f64f402df90 comp rx=0 tx=0).stop 2024-03-21T11:27:49.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.393+0000 7f6504200640 1 -- 172.21.15.47:0/3922882874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0756e0 msgr2=0x7f64fc19e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.393+0000 7f6504200640 1 --2- 172.21.15.47:0/3922882874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0756e0 0x7f64fc19e4e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f64e800e9f0 tx=0x7f64e800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.393+0000 7f6504200640 1 -- 172.21.15.47:0/3922882874 shutdown_connections 2024-03-21T11:27:49.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.393+0000 7f6504200640 1 --2- 172.21.15.47:0/3922882874 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f64e40a2fa0 0x7f64e40a5460 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.393+0000 7f6504200640 1 --2- 172.21.15.47:0/3922882874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64fc10dc80 0x7f64fc1a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.393+0000 7f6504200640 1 --2- 172.21.15.47:0/3922882874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64fc0756e0 0x7f64fc19e4e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.393+0000 7f6504200640 1 --2- 172.21.15.47:0/3922882874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64fc074100 0x7f64fc19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.393+0000 7f6504200640 1 -- 172.21.15.47:0/3922882874 >> 172.21.15.47:0/3922882874 conn(0x7f64fc0fcc40 msgr2=0x7f64fc101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.393+0000 7f6504200640 1 -- 172.21.15.47:0/3922882874 shutdown_connections 2024-03-21T11:27:49.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:49.393+0000 7f6504200640 1 -- 172.21.15.47:0/3922882874 wait complete. 2024-03-21T11:27:49.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:48 smithi100 ceph-mon[36245]: pgmap v534: 129 pgs: 129 active+clean; 37 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 322 op/s 2024-03-21T11:27:49.401 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:48 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1095409361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:27:49.506 INFO:tasks.workunit.client.0.smithi047.stdout: 1 245339 8.16 MB/sec execute 33 sec latency 9.832 ms 2024-03-21T11:27:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.645+0000 7f57f5400640 1 -- 172.21.15.47:0/2406935943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57f0103720 msgr2=0x7f57f0103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.645+0000 7f57f5400640 1 --2- 172.21.15.47:0/2406935943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57f0103720 0x7f57f0103b40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009900 tx=0x7f57e002bc40 comp rx=0 tx=0).stop 2024-03-21T11:27:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.645+0000 7f57f5400640 1 -- 172.21.15.47:0/2406935943 shutdown_connections 2024-03-21T11:27:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.645+0000 7f57f5400640 1 --2- 172.21.15.47:0/2406935943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f01052d0 0x7f57f010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.645+0000 7f57f5400640 1 --2- 172.21.15.47:0/2406935943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57f0104910 0x7f57f0104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.645+0000 7f57f5400640 1 --2- 172.21.15.47:0/2406935943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57f0103720 0x7f57f0103b40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.645+0000 7f57f5400640 1 -- 172.21.15.47:0/2406935943 >> 172.21.15.47:0/2406935943 conn(0x7f57f00fecb0 msgr2=0x7f57f01010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.645+0000 7f57f5400640 1 -- 172.21.15.47:0/2406935943 shutdown_connections 2024-03-21T11:27:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.645+0000 7f57f5400640 1 -- 172.21.15.47:0/2406935943 wait complete. 2024-03-21T11:27:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.647+0000 7f57f5400640 1 Processor -- start 2024-03-21T11:27:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.647+0000 7f57f5400640 1 -- start start 2024-03-21T11:27:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.648+0000 7f57f5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f0103720 0x7f57f0199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.648+0000 7f57f5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57f0104910 0x7f57f0199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.648+0000 7f57f5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57f01052d0 0x7f57f01a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.648+0000 7f57f5400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f010f3e0 con 0x7f57f0103720 2024-03-21T11:27:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.648+0000 7f57f5400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f010f260 con 0x7f57f01052d0 2024-03-21T11:27:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.648+0000 7f57f5400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f010f560 con 0x7f57f0104910 2024-03-21T11:27:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.648+0000 7f57ef400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57f0104910 0x7f57f0199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.648+0000 7f57f4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57f01052d0 0x7f57f01a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.648+0000 7f57ef400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57f0104910 0x7f57f0199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49056/0 (socket says 172.21.15.47:49056) 2024-03-21T11:27:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.648+0000 7f57efe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f0103720 0x7f57f0199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.648+0000 7f57ef400640 1 -- 172.21.15.47:0/2967527545 learned_addr learned my addr 172.21.15.47:0/2967527545 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.649+0000 7f57f4a00640 1 -- 172.21.15.47:0/2967527545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57f0104910 msgr2=0x7f57f0199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.649+0000 7f57f4a00640 1 --2- 172.21.15.47:0/2967527545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57f0104910 0x7f57f0199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.649+0000 7f57f4a00640 1 -- 172.21.15.47:0/2967527545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f0103720 msgr2=0x7f57f0199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.649+0000 7f57f4a00640 1 --2- 172.21.15.47:0/2967527545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f0103720 0x7f57f0199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.649+0000 7f57f4a00640 1 -- 172.21.15.47:0/2967527545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f01a1650 con 0x7f57f01052d0 2024-03-21T11:27:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.649+0000 7f57f4a00640 1 --2- 172.21.15.47:0/2967527545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57f01052d0 0x7f57f01a0fe0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f57e800e940 tx=0x7f57e800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.649+0000 7f57ecc00640 1 -- 172.21.15.47:0/2967527545 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8004d20 con 0x7f57f01052d0 2024-03-21T11:27:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.649+0000 7f57ecc00640 1 -- 172.21.15.47:0/2967527545 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57e8005020 con 0x7f57f01052d0 2024-03-21T11:27:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.649+0000 7f57ecc00640 1 -- 172.21.15.47:0/2967527545 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e80116b0 con 0x7f57f01052d0 2024-03-21T11:27:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.649+0000 7f57f5400640 1 -- 172.21.15.47:0/2967527545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f01a1940 con 0x7f57f01052d0 2024-03-21T11:27:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.650+0000 7f57f5400640 1 -- 172.21.15.47:0/2967527545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f01a1e90 con 0x7f57f01052d0 2024-03-21T11:27:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.653+0000 7f57f97f4740 1 Processor -- start 2024-03-21T11:27:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.653+0000 7f57ecc00640 1 -- 172.21.15.47:0/2967527545 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f57e80054d0 con 0x7f57f01052d0 2024-03-21T11:27:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.653+0000 7f57f97f4740 1 -- start start 2024-03-21T11:27:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.653+0000 7f57f97f4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.653+0000 7f57f97f4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.653+0000 7f57f97f4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7d83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.653+0000 7f57f97f4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ae7933b0 con 0x55a8ae7d0de0 2024-03-21T11:27:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.653+0000 7f57f97f4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ae735830 con 0x55a8ae7d5f50 2024-03-21T11:27:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.653+0000 7f57f97f4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ae6de5d0 con 0x55a8ae7d03f0 2024-03-21T11:27:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.654+0000 7f57ef400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.654+0000 7f57ef400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49064/0 (socket says 172.21.15.47:49064) 2024-03-21T11:27:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.654+0000 7f57ef400640 1 -- 172.21.15.47:0/1881995072 learned_addr learned my addr 172.21.15.47:0/1881995072 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.654+0000 7f57efe00640 1 --2- 172.21.15.47:0/1881995072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7d58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.654+0000 7f57ef400640 1 --2- 172.21.15.47:0/1881995072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7d83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.654+0000 7f57ef400640 1 -- 172.21.15.47:0/1881995072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d5f50 msgr2=0x55a8ae7d83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.654+0000 7f57ef400640 1 --2- 172.21.15.47:0/1881995072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7d83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.654+0000 7f57ef400640 1 -- 172.21.15.47:0/1881995072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d0de0 msgr2=0x55a8ae7d58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.654+0000 7f57ef400640 1 --2- 172.21.15.47:0/1881995072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7d58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.654+0000 7f57ef400640 1 -- 172.21.15.47:0/1881995072 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8ae6f62c0 con 0x55a8ae7d03f0 2024-03-21T11:27:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.655+0000 7f57ef400640 1 --2- 172.21.15.47:0/1881995072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7d0810 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009810 tx=0x7f57dc009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dbc999cfb221b50 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.655+0000 7f57e5200640 1 -- 172.21.15.47:0/1881995072 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc013170 con 0x55a8ae7d03f0 2024-03-21T11:27:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.655+0000 7f57e5200640 1 -- 172.21.15.47:0/1881995072 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57dc00fca0 con 0x55a8ae7d03f0 2024-03-21T11:27:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.655+0000 7f57e5200640 1 -- 172.21.15.47:0/1881995072 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0109c0 con 0x55a8ae7d03f0 2024-03-21T11:27:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.655+0000 7f57ecc00640 1 --2- 172.21.15.47:0/2967527545 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f57d40a2f90 0x7f57d40a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.655+0000 7f57ecc00640 1 -- 172.21.15.47:0/2967527545 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f57e80cf780 con 0x7f57f01052d0 2024-03-21T11:27:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.656+0000 7f57efe00640 1 --2- 172.21.15.47:0/2967527545 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f57d40a2f90 0x7f57d40a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.656+0000 7f57f97f4740 1 -- 172.21.15.47:0/1881995072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d03f0 msgr2=0x55a8ae7d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.656+0000 7f57f97f4740 1 --2- 172.21.15.47:0/1881995072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7d0810 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009810 tx=0x7f57dc009880 comp rx=0 tx=0).stop 2024-03-21T11:27:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.656+0000 7f57f97f4740 1 -- 172.21.15.47:0/1881995072 shutdown_connections 2024-03-21T11:27:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.656+0000 7f57f97f4740 1 --2- 172.21.15.47:0/1881995072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7d83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.656+0000 7f57f97f4740 1 --2- 172.21.15.47:0/1881995072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7d58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.656+0000 7f57f97f4740 1 --2- 172.21.15.47:0/1881995072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7d0810 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.656+0000 7f57f97f4740 1 -- 172.21.15.47:0/1881995072 >> 172.21.15.47:0/1881995072 conn(0x55a8ae7cb960 msgr2=0x55a8ae7cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.656+0000 7f57f97f4740 1 -- 172.21.15.47:0/1881995072 shutdown_connections 2024-03-21T11:27:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.657+0000 7f57f97f4740 1 -- 172.21.15.47:0/1881995072 wait complete. 2024-03-21T11:27:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.657+0000 7f57efe00640 1 --2- 172.21.15.47:0/2967527545 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f57d40a2f90 0x7f57d40a5450 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f57e002c180 tx=0x7f57e002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.657+0000 7f57f97f4740 1 -- start start 2024-03-21T11:27:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.657+0000 7f57f97f4740 1 Processor -- start 2024-03-21T11:27:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.658+0000 7f57f97f4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7da5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.658+0000 7f57f97f4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7dab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.658+0000 7f57f97f4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.658+0000 7f57f97f4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ae6f1a20 con 0x55a8ae7d03f0 2024-03-21T11:27:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.658+0000 7f57f97f4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ae7318d0 con 0x55a8ae7d0de0 2024-03-21T11:27:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.658+0000 7f57f97f4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ae7d8f00 con 0x55a8ae7d5f50 2024-03-21T11:27:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.658+0000 7f57f97f4740 10 client.0 _ll_register_callbacks cb 0x55a8ae72e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:49.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.658+0000 7f57efe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7da5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.658+0000 7f57ef400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7dab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.658+0000 7f57efe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7da5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50142/0 (socket says 172.21.15.47:50142) 2024-03-21T11:27:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.658+0000 7f57ef400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7dab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36344/0 (socket says 172.21.15.47:36344) 2024-03-21T11:27:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.658+0000 7f57efe00640 1 -- 172.21.15.47:0/1033388017 learned_addr learned my addr 172.21.15.47:0/1033388017 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.658+0000 7f57f4a00640 1 --2- 172.21.15.47:0/1033388017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.659+0000 7f57ef400640 1 -- 172.21.15.47:0/1033388017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d5f50 msgr2=0x55a8ae7e1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.659+0000 7f57ef400640 1 --2- 172.21.15.47:0/1033388017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7e1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.659+0000 7f57ef400640 1 -- 172.21.15.47:0/1033388017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d03f0 msgr2=0x55a8ae7da5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.659+0000 7f57ef400640 1 --2- 172.21.15.47:0/1033388017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7da5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.659+0000 7f57ef400640 1 -- 172.21.15.47:0/1033388017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8ae44a210 con 0x55a8ae7d0de0 2024-03-21T11:27:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.659+0000 7f57efe00640 1 --2- 172.21.15.47:0/1033388017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7da5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.659+0000 7f57f4a00640 1 --2- 172.21.15.47:0/1033388017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7e1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.659+0000 7f57ef400640 1 --2- 172.21.15.47:0/1033388017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7dab10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009810 tx=0x7f57dc00fc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cf68b1488721df9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.659+0000 7f57d9600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.659+0000 7f57d9600640 1 -- 172.21.15.47:0/1033388017 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00fe50 con 0x55a8ae7d0de0 2024-03-21T11:27:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.659+0000 7f57d9600640 1 -- 172.21.15.47:0/1033388017 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f57dc005250 con 0x55a8ae7d0de0 2024-03-21T11:27:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.659+0000 7f57f97f4740 1 -- 172.21.15.47:0/1033388017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57dc00fca0 con 0x55a8ae7d0de0 2024-03-21T11:27:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.660+0000 7f57d9600640 1 -- 172.21.15.47:0/1033388017 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57dc02f050 con 0x55a8ae7d0de0 2024-03-21T11:27:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.660+0000 7f57f97f4740 10 client.27029 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.660+0000 7f57f97f4740 1 -- 172.21.15.47:0/1033388017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a8ae516620 con 0x55a8ae7d0de0 2024-03-21T11:27:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.660+0000 7f57d9600640 1 -- 172.21.15.47:0/1033388017 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57dc00fca0 con 0x55a8ae7d0de0 2024-03-21T11:27:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.660+0000 7f57d9600640 1 -- 172.21.15.47:0/1033388017 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f57dc004680 con 0x55a8ae7d0de0 2024-03-21T11:27:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.660+0000 7f57f97f4740 10 client.27029 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.660+0000 7f57f97f4740 10 client.27029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.660+0000 7f57f97f4740 20 client.27029 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.660+0000 7f57f97f4740 1 --2- 172.21.15.47:0/1033388017 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a8ae7e6100 0x55a8ae7e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.660+0000 7f57f97f4740 4 client.27029 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.660+0000 7f57f97f4740 1 -- 172.21.15.47:0/1033388017 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8ae742d70 con 0x55a8ae7e6100 2024-03-21T11:27:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.661+0000 7f57efe00640 1 --2- 172.21.15.47:0/1033388017 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a8ae7e6100 0x55a8ae7e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.661+0000 7f57efe00640 1 --2- 172.21.15.47:0/1033388017 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a8ae7e6100 0x55a8ae7e84f0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=979ba5a278c26f87 server_cookie=96ba01f540c7979e in_seq=0 out_seq=0 2024-03-21T11:27:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.661+0000 7f57d9600640 10 client.27029 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57d9600640 1 -- 172.21.15.47:0/1033388017 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a8ae742d70 con 0x55a8ae7e6100 2024-03-21T11:27:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57d9600640 10 client.27029 handle_command_reply: tid=0 2024-03-21T11:27:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57f97f4740 1 client.27029 shutdown 2024-03-21T11:27:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57f97f4740 1 -- 172.21.15.47:0/1033388017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d0de0 msgr2=0x55a8ae7dab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57f97f4740 1 --2- 172.21.15.47:0/1033388017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7dab10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009810 tx=0x7f57dc00fc30 comp rx=0 tx=0).stop 2024-03-21T11:27:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57f97f4740 1 -- 172.21.15.47:0/1033388017 shutdown_connections 2024-03-21T11:27:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57f97f4740 1 --2- 172.21.15.47:0/1033388017 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a8ae7e6100 0x55a8ae7e84f0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57f97f4740 1 --2- 172.21.15.47:0/1033388017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7e1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57f97f4740 1 --2- 172.21.15.47:0/1033388017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7dab10 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57f97f4740 1 --2- 172.21.15.47:0/1033388017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7da5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57f97f4740 1 -- 172.21.15.47:0/1033388017 >> 172.21.15.47:0/1033388017 conn(0x55a8ae7cb960 msgr2=0x55a8ae7cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57d9600640 0 client.27029 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57f97f4740 1 -- 172.21.15.47:0/1033388017 shutdown_connections 2024-03-21T11:27:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57f97f4740 1 -- 172.21.15.47:0/1033388017 wait complete. 2024-03-21T11:27:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.663+0000 7f57f97f4740 20 client.27029 trim_cache size 0 max 16384 2024-03-21T11:27:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.683+0000 7f57f97f4740 1 Processor -- start 2024-03-21T11:27:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.683+0000 7f57f97f4740 1 -- start start 2024-03-21T11:27:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57f97f4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7d2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57f97f4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7e1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57f97f4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57f97f4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f01ac130 con 0x55a8ae7d0de0 2024-03-21T11:27:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57f97f4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f01adab0 con 0x55a8ae7d03f0 2024-03-21T11:27:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57f97f4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f01ac790 con 0x55a8ae7d5f50 2024-03-21T11:27:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57f4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57ef400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7e1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57efe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57f4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49086/0 (socket says 172.21.15.47:49086) 2024-03-21T11:27:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57f4a00640 1 -- 172.21.15.47:0/4255707551 learned_addr learned my addr 172.21.15.47:0/4255707551 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57efe00640 1 -- 172.21.15.47:0/4255707551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d5f50 msgr2=0x55a8ae7d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57efe00640 1 --2- 172.21.15.47:0/4255707551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57efe00640 1 -- 172.21.15.47:0/4255707551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d0de0 msgr2=0x55a8ae7e1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57efe00640 1 --2- 172.21.15.47:0/4255707551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7e1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57efe00640 1 -- 172.21.15.47:0/4255707551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8ae7e8860 con 0x55a8ae7d03f0 2024-03-21T11:27:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57ef400640 1 --2- 172.21.15.47:0/4255707551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7e1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.684+0000 7f57f4a00640 1 --2- 172.21.15.47:0/4255707551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.685+0000 7f57efe00640 1 --2- 172.21.15.47:0/4255707551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7d2140 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f57e0006010 tx=0x7f57e0040040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7bef055322fb0d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.685+0000 7f57dbe00640 1 -- 172.21.15.47:0/4255707551 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0058070 con 0x55a8ae7d03f0 2024-03-21T11:27:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.685+0000 7f57dbe00640 1 -- 172.21.15.47:0/4255707551 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57e0053610 con 0x55a8ae7d03f0 2024-03-21T11:27:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.685+0000 7f57dbe00640 1 -- 172.21.15.47:0/4255707551 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0034680 con 0x55a8ae7d03f0 2024-03-21T11:27:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.686+0000 7f57f97f4740 1 -- 172.21.15.47:0/4255707551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d03f0 msgr2=0x55a8ae7d2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.686+0000 7f57f97f4740 1 --2- 172.21.15.47:0/4255707551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7d2140 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f57e0006010 tx=0x7f57e0040040 comp rx=0 tx=0).stop 2024-03-21T11:27:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.686+0000 7f57f97f4740 1 -- 172.21.15.47:0/4255707551 shutdown_connections 2024-03-21T11:27:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.686+0000 7f57f97f4740 1 --2- 172.21.15.47:0/4255707551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.686+0000 7f57f97f4740 1 --2- 172.21.15.47:0/4255707551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7e1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.686+0000 7f57f97f4740 1 --2- 172.21.15.47:0/4255707551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7d2140 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.686+0000 7f57f97f4740 1 -- 172.21.15.47:0/4255707551 >> 172.21.15.47:0/4255707551 conn(0x55a8ae7cb960 msgr2=0x55a8ae7da5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.686+0000 7f57f97f4740 1 -- 172.21.15.47:0/4255707551 shutdown_connections 2024-03-21T11:27:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.686+0000 7f57f97f4740 1 -- 172.21.15.47:0/4255707551 wait complete. 2024-03-21T11:27:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.686+0000 7f57f97f4740 1 -- start start 2024-03-21T11:27:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.687+0000 7f57f97f4740 1 Processor -- start 2024-03-21T11:27:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.687+0000 7f57f97f4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7dee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.687+0000 7f57f97f4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7df3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.687+0000 7f57f97f4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7e0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.687+0000 7f57f97f4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f01ad450 con 0x55a8ae7d03f0 2024-03-21T11:27:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.687+0000 7f57f97f4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f01acdf0 con 0x55a8ae7d5f50 2024-03-21T11:27:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.687+0000 7f57f97f4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f01aa2f0 con 0x55a8ae7d0de0 2024-03-21T11:27:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.687+0000 7f57f97f4740 10 client.0 _ll_register_callbacks cb 0x7f57e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.687+0000 7f57ef400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7df3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.687+0000 7f57efe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7dee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.687+0000 7f57ef400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7df3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49098/0 (socket says 172.21.15.47:49098) 2024-03-21T11:27:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.687+0000 7f57ef400640 1 -- 172.21.15.47:0/2520988895 learned_addr learned my addr 172.21.15.47:0/2520988895 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.687+0000 7f57f4a00640 1 --2- 172.21.15.47:0/2520988895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7e0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.688+0000 7f57ef400640 1 -- 172.21.15.47:0/2520988895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d5f50 msgr2=0x55a8ae7e0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.688+0000 7f57ef400640 1 --2- 172.21.15.47:0/2520988895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7e0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.688+0000 7f57ef400640 1 -- 172.21.15.47:0/2520988895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d03f0 msgr2=0x55a8ae7dee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.688+0000 7f57ef400640 1 --2- 172.21.15.47:0/2520988895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7dee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.688+0000 7f57ef400640 1 -- 172.21.15.47:0/2520988895 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8ae7e0960 con 0x55a8ae7d0de0 2024-03-21T11:27:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.688+0000 7f57efe00640 1 --2- 172.21.15.47:0/2520988895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7dee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.688+0000 7f57ef400640 1 --2- 172.21.15.47:0/2520988895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7df3b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x55a8ae7dbab0 tx=0x7f57dc005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd3aecf79a4edb1b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.688+0000 7f57d9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.688+0000 7f57d9600640 1 -- 172.21.15.47:0/2520988895 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00fca0 con 0x55a8ae7d0de0 2024-03-21T11:27:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.688+0000 7f57f97f4740 1 -- 172.21.15.47:0/2520988895 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57e0053610 con 0x55a8ae7d0de0 2024-03-21T11:27:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.688+0000 7f57d9600640 1 -- 172.21.15.47:0/2520988895 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f57dc00fe40 con 0x55a8ae7d0de0 2024-03-21T11:27:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.688+0000 7f57d9600640 1 -- 172.21.15.47:0/2520988895 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57dc005250 con 0x55a8ae7d0de0 2024-03-21T11:27:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.688+0000 7f57f97f4740 10 client.27049 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.688+0000 7f57f97f4740 1 -- 172.21.15.47:0/2520988895 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a8ae7e1c70 con 0x55a8ae7d0de0 2024-03-21T11:27:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.689+0000 7f57d9600640 1 -- 172.21.15.47:0/2520988895 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57e0053610 con 0x55a8ae7d0de0 2024-03-21T11:27:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.689+0000 7f57d9600640 1 -- 172.21.15.47:0/2520988895 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f57dc034920 con 0x55a8ae7d0de0 2024-03-21T11:27:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.689+0000 7f57f97f4740 10 client.27049 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.689+0000 7f57f97f4740 10 client.27049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.689+0000 7f57f97f4740 20 client.27049 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.689+0000 7f57f97f4740 1 --2- 172.21.15.47:0/2520988895 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a8ae841f10 0x55a8ae844300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.689+0000 7f57f97f4740 4 client.27049 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:27:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.689+0000 7f57f97f4740 1 -- 172.21.15.47:0/2520988895 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a8ae73d0c0 con 0x55a8ae841f10 2024-03-21T11:27:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.689+0000 7f57efe00640 1 --2- 172.21.15.47:0/2520988895 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a8ae841f10 0x55a8ae844300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.690+0000 7f57efe00640 1 --2- 172.21.15.47:0/2520988895 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a8ae841f10 0x55a8ae844300 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d630b7b6d824677 server_cookie=77e1c3012e0de626 in_seq=0 out_seq=0 2024-03-21T11:27:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.690+0000 7f57d9600640 10 client.27049 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.898+0000 7f57d9600640 1 -- 172.21.15.47:0/2520988895 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a8ae73d0c0 con 0x55a8ae841f10 2024-03-21T11:27:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.898+0000 7f57d9600640 10 client.27049 handle_command_reply: tid=0 2024-03-21T11:27:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.898+0000 7f57f97f4740 1 client.27049 shutdown 2024-03-21T11:27:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.898+0000 7f57f97f4740 1 -- 172.21.15.47:0/2520988895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d0de0 msgr2=0x55a8ae7df3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.898+0000 7f57f97f4740 1 --2- 172.21.15.47:0/2520988895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7df3b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x55a8ae7dbab0 tx=0x7f57dc005d30 comp rx=0 tx=0).stop 2024-03-21T11:27:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.898+0000 7f57f97f4740 1 -- 172.21.15.47:0/2520988895 shutdown_connections 2024-03-21T11:27:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.898+0000 7f57f97f4740 1 --2- 172.21.15.47:0/2520988895 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a8ae841f10 0x55a8ae844300 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.898+0000 7f57f97f4740 1 --2- 172.21.15.47:0/2520988895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a8ae7d5f50 0x55a8ae7e0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.898+0000 7f57f97f4740 1 --2- 172.21.15.47:0/2520988895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a8ae7d0de0 0x55a8ae7df3b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.898+0000 7f57f97f4740 1 --2- 172.21.15.47:0/2520988895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a8ae7d03f0 0x55a8ae7dee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.898+0000 7f57f97f4740 1 -- 172.21.15.47:0/2520988895 >> 172.21.15.47:0/2520988895 conn(0x55a8ae7cb960 msgr2=0x55a8ae7da5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.898+0000 7f57d9600640 0 client.27049 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.899+0000 7f57f97f4740 1 -- 172.21.15.47:0/2520988895 shutdown_connections 2024-03-21T11:27:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.899+0000 7f57f97f4740 1 -- 172.21.15.47:0/2520988895 wait complete. 2024-03-21T11:27:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.902+0000 7f57f97f4740 20 client.27049 trim_cache size 0 max 16384 2024-03-21T11:27:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:27:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:27:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:27:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:27:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.902+0000 7f57f5400640 1 -- 172.21.15.47:0/2967527545 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f57d40a2f90 msgr2=0x7f57d40a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.902+0000 7f57f5400640 1 --2- 172.21.15.47:0/2967527545 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f57d40a2f90 0x7f57d40a5450 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f57e002c180 tx=0x7f57e002cf90 comp rx=0 tx=0).stop 2024-03-21T11:27:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.902+0000 7f57f5400640 1 -- 172.21.15.47:0/2967527545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57f01052d0 msgr2=0x7f57f01a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.902+0000 7f57f5400640 1 --2- 172.21.15.47:0/2967527545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57f01052d0 0x7f57f01a0fe0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f57e800e940 tx=0x7f57e800ee00 comp rx=0 tx=0).stop 2024-03-21T11:27:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.903+0000 7f57f5400640 1 -- 172.21.15.47:0/2967527545 shutdown_connections 2024-03-21T11:27:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.903+0000 7f57f5400640 1 --2- 172.21.15.47:0/2967527545 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f57d40a2f90 0x7f57d40a5450 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.903+0000 7f57f5400640 1 --2- 172.21.15.47:0/2967527545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57f01052d0 0x7f57f01a0fe0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.903+0000 7f57f5400640 1 --2- 172.21.15.47:0/2967527545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57f0104910 0x7f57f0199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.903+0000 7f57f5400640 1 --2- 172.21.15.47:0/2967527545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f0103720 0x7f57f0199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.903+0000 7f57f5400640 1 -- 172.21.15.47:0/2967527545 >> 172.21.15.47:0/2967527545 conn(0x7f57f00fecb0 msgr2=0x7f57f0107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.903+0000 7f57f5400640 1 -- 172.21.15.47:0/2967527545 shutdown_connections 2024-03-21T11:27:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:27:49.903+0000 7f57f5400640 1 -- 172.21.15.47:0/2967527545 wait complete. 2024-03-21T11:27:50.412 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:27:50.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:27:50.412 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:27:50.506 INFO:tasks.workunit.client.0.smithi047.stdout: 1 246629 8.08 MB/sec execute 34 sec latency 93.843 ms 2024-03-21T11:27:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.651+0000 7f2a7f000640 1 -- 172.21.15.47:0/1967977988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a78104a60 msgr2=0x7f2a78106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.651+0000 7f2a7f000640 1 --2- 172.21.15.47:0/1967977988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a78104a60 0x7f2a78106e50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009900 tx=0x7f2a6002bc40 comp rx=0 tx=0).stop 2024-03-21T11:27:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.652+0000 7f2a7f000640 1 -- 172.21.15.47:0/1967977988 shutdown_connections 2024-03-21T11:27:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.652+0000 7f2a7f000640 1 --2- 172.21.15.47:0/1967977988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a78107390 0x7f2a781097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.652+0000 7f2a7f000640 1 --2- 172.21.15.47:0/1967977988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a78104a60 0x7f2a78106e50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.652+0000 7f2a7f000640 1 --2- 172.21.15.47:0/1967977988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a78102130 0x7f2a78104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.652+0000 7f2a7f000640 1 -- 172.21.15.47:0/1967977988 >> 172.21.15.47:0/1967977988 conn(0x7f2a780fbb40 msgr2=0x7f2a780fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.652+0000 7f2a7f000640 1 -- 172.21.15.47:0/1967977988 shutdown_connections 2024-03-21T11:27:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.652+0000 7f2a7f000640 1 -- 172.21.15.47:0/1967977988 wait complete. 2024-03-21T11:27:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.653+0000 7f2a7f000640 1 Processor -- start 2024-03-21T11:27:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.653+0000 7f2a7f000640 1 -- start start 2024-03-21T11:27:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.653+0000 7f2a7f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a78102130 0x7f2a78197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.653+0000 7f2a7f000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a78104a60 0x7f2a78197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.653+0000 7f2a7f000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a78107390 0x7f2a7819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.653+0000 7f2a7f000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7810d320 con 0x7f2a78104a60 2024-03-21T11:27:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.653+0000 7f2a7f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7810d1a0 con 0x7f2a78102130 2024-03-21T11:27:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.653+0000 7f2a7f000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7810d4a0 con 0x7f2a78107390 2024-03-21T11:27:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.654+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a78102130 0x7f2a78197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.654+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a78102130 0x7f2a78197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36386/0 (socket says 172.21.15.47:36386) 2024-03-21T11:27:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.654+0000 7f2a77e00640 1 -- 172.21.15.47:0/2971719420 learned_addr learned my addr 172.21.15.47:0/2971719420 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.654+0000 7f2a7d000640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a78107390 0x7f2a7819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.654+0000 7f2a77400640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a78104a60 0x7f2a78197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.654+0000 7f2a77e00640 1 -- 172.21.15.47:0/2971719420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a78107390 msgr2=0x7f2a7819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.654+0000 7f2a77e00640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a78107390 0x7f2a7819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.654+0000 7f2a77e00640 1 -- 172.21.15.47:0/2971719420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a78104a60 msgr2=0x7f2a78197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.654+0000 7f2a77e00640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a78104a60 0x7f2a78197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.654+0000 7f2a77e00640 1 -- 172.21.15.47:0/2971719420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7819f640 con 0x7f2a78102130 2024-03-21T11:27:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.654+0000 7f2a77400640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a78104a60 0x7f2a78197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.654+0000 7f2a77e00640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a78102130 0x7f2a78197a20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009d50 tx=0x7f2a6800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.655+0000 7f2a74c00640 1 -- 172.21.15.47:0/2971719420 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68003b70 con 0x7f2a78102130 2024-03-21T11:27:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.655+0000 7f2a74c00640 1 -- 172.21.15.47:0/2971719420 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a68003d10 con 0x7f2a78102130 2024-03-21T11:27:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.655+0000 7f2a74c00640 1 -- 172.21.15.47:0/2971719420 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68010a20 con 0x7f2a78102130 2024-03-21T11:27:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.655+0000 7f2a7f000640 1 -- 172.21.15.47:0/2971719420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7819f930 con 0x7f2a78102130 2024-03-21T11:27:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.655+0000 7f2a7f000640 1 -- 172.21.15.47:0/2971719420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7819fea0 con 0x7f2a78102130 2024-03-21T11:27:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.658+0000 7f2a74c00640 1 -- 172.21.15.47:0/2971719420 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2a68003ec0 con 0x7f2a78102130 2024-03-21T11:27:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.658+0000 7f2a81e0e740 1 Processor -- start 2024-03-21T11:27:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.659+0000 7f2a81e0e740 1 -- start start 2024-03-21T11:27:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.659+0000 7f2a81e0e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e678840a0 0x559e678844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.659+0000 7f2a81e0e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e67884a90 0x559e67889590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.659+0000 7f2a81e0e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e6788c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.659+0000 7f2a81e0e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e677e9eb0 con 0x559e678840a0 2024-03-21T11:27:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.659+0000 7f2a81e0e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e67792a60 con 0x559e67889c00 2024-03-21T11:27:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.659+0000 7f2a81e0e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e677e5f50 con 0x559e67884a90 2024-03-21T11:27:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.659+0000 7f2a7d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e678840a0 0x559e678844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.659+0000 7f2a77400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e6788c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.659+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e67884a90 0x559e67889590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.659+0000 7f2a77400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e6788c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36396/0 (socket says 172.21.15.47:36396) 2024-03-21T11:27:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.659+0000 7f2a77400640 1 -- 172.21.15.47:0/3279609631 learned_addr learned my addr 172.21.15.47:0/3279609631 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.659+0000 7f2a7d000640 1 --2- 172.21.15.47:0/3279609631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e678840a0 0x559e678844c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50196/0 (socket says 172.21.15.47:50196) 2024-03-21T11:27:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.660+0000 7f2a77400640 1 -- 172.21.15.47:0/3279609631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e67884a90 msgr2=0x559e67889590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.660+0000 7f2a77400640 1 --2- 172.21.15.47:0/3279609631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e67884a90 0x559e67889590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.660+0000 7f2a77400640 1 -- 172.21.15.47:0/3279609631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e678840a0 msgr2=0x559e678844c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.660+0000 7f2a77400640 1 --2- 172.21.15.47:0/3279609631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e678840a0 0x559e678844c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.660+0000 7f2a77400640 1 -- 172.21.15.47:0/3279609631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e675c67e0 con 0x559e67889c00 2024-03-21T11:27:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.660+0000 7f2a77400640 1 --2- 172.21.15.47:0/3279609631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e6788c090 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2a7806a7d0 tx=0x7f2a6002cd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=771bdd21b0381440 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.660+0000 7f2a74c00640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2a5c0a2fa0 0x7f2a5c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.661+0000 7f2a6d600640 1 -- 172.21.15.47:0/3279609631 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a60005d50 con 0x559e67889c00 2024-03-21T11:27:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.661+0000 7f2a74c00640 1 -- 172.21.15.47:0/2971719420 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2a680cfe20 con 0x7f2a78102130 2024-03-21T11:27:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.661+0000 7f2a6d600640 1 -- 172.21.15.47:0/3279609631 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a60005520 con 0x559e67889c00 2024-03-21T11:27:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.661+0000 7f2a6d600640 1 -- 172.21.15.47:0/3279609631 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a60004ca0 con 0x559e67889c00 2024-03-21T11:27:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.661+0000 7f2a7d000640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2a5c0a2fa0 0x7f2a5c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.661+0000 7f2a81e0e740 1 -- 172.21.15.47:0/3279609631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 msgr2=0x559e6788c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.661+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/3279609631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e6788c090 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2a7806a7d0 tx=0x7f2a6002cd40 comp rx=0 tx=0).stop 2024-03-21T11:27:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.662+0000 7f2a7d000640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2a5c0a2fa0 0x7f2a5c0a5460 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x559e678852f0 tx=0x7f2a70015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.662+0000 7f2a81e0e740 1 -- 172.21.15.47:0/3279609631 shutdown_connections 2024-03-21T11:27:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.662+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/3279609631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e6788c090 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.662+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/3279609631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e67884a90 0x559e67889590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.662+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/3279609631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e678840a0 0x559e678844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.662+0000 7f2a81e0e740 1 -- 172.21.15.47:0/3279609631 >> 172.21.15.47:0/3279609631 conn(0x559e6787f610 msgr2=0x559e6787fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.662+0000 7f2a81e0e740 1 -- 172.21.15.47:0/3279609631 shutdown_connections 2024-03-21T11:27:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.662+0000 7f2a81e0e740 1 -- 172.21.15.47:0/3279609631 wait complete. 2024-03-21T11:27:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.662+0000 7f2a81e0e740 1 -- start start 2024-03-21T11:27:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.662+0000 7f2a81e0e740 1 Processor -- start 2024-03-21T11:27:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.663+0000 7f2a81e0e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e678840a0 0x559e6788e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.663+0000 7f2a81e0e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e67884a90 0x559e6788e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.663+0000 7f2a81e0e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e67895790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.663+0000 7f2a81e0e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e6788cb50 con 0x559e67884a90 2024-03-21T11:27:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.663+0000 7f2a81e0e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e677a5eb0 con 0x559e67889c00 2024-03-21T11:27:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.663+0000 7f2a81e0e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e6788ccd0 con 0x559e678840a0 2024-03-21T11:27:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.663+0000 7f2a81e0e740 10 client.0 _ll_register_callbacks cb 0x559e6779a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.663+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e678840a0 0x559e6788e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.663+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e678840a0 0x559e6788e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49136/0 (socket says 172.21.15.47:49136) 2024-03-21T11:27:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.663+0000 7f2a77e00640 1 -- 172.21.15.47:0/1371969466 learned_addr learned my addr 172.21.15.47:0/1371969466 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.663+0000 7f2a7d000640 1 --2- 172.21.15.47:0/1371969466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e67895790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.663+0000 7f2a77400640 1 --2- 172.21.15.47:0/1371969466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e67884a90 0x559e6788e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.664+0000 7f2a77e00640 1 -- 172.21.15.47:0/1371969466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 msgr2=0x559e67895790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.664+0000 7f2a77e00640 1 --2- 172.21.15.47:0/1371969466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e67895790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.664+0000 7f2a77e00640 1 -- 172.21.15.47:0/1371969466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e67884a90 msgr2=0x559e6788e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.664+0000 7f2a77e00640 1 --2- 172.21.15.47:0/1371969466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e67884a90 0x559e6788e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.664+0000 7f2a77e00640 1 -- 172.21.15.47:0/1371969466 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e67626f70 con 0x559e678840a0 2024-03-21T11:27:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.664+0000 7f2a77400640 1 --2- 172.21.15.47:0/1371969466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e67884a90 0x559e6788e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.664+0000 7f2a77e00640 1 --2- 172.21.15.47:0/1371969466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e678840a0 0x559e6788e280 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2a68098b30 tx=0x7f2a68009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f44d533c300a55a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.664+0000 7f2a66000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.664+0000 7f2a66000640 1 -- 172.21.15.47:0/1371969466 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6809bac0 con 0x559e678840a0 2024-03-21T11:27:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.664+0000 7f2a66000640 1 -- 172.21.15.47:0/1371969466 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2a680987d0 con 0x559e678840a0 2024-03-21T11:27:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.664+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1371969466 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a60005520 con 0x559e678840a0 2024-03-21T11:27:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.665+0000 7f2a66000640 1 -- 172.21.15.47:0/1371969466 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a680a00b0 con 0x559e678840a0 2024-03-21T11:27:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.665+0000 7f2a81e0e740 10 client.27061 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.665+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1371969466 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559e674be170 con 0x559e678840a0 2024-03-21T11:27:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.665+0000 7f2a66000640 1 -- 172.21.15.47:0/1371969466 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a60005520 con 0x559e678840a0 2024-03-21T11:27:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.665+0000 7f2a66000640 1 -- 172.21.15.47:0/1371969466 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2a680b5100 con 0x559e678840a0 2024-03-21T11:27:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.665+0000 7f2a81e0e740 10 client.27061 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.665+0000 7f2a81e0e740 10 client.27061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.665+0000 7f2a81e0e740 20 client.27061 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.665+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/1371969466 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e6789a010 0x559e6789c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.665+0000 7f2a81e0e740 4 client.27061 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.665+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1371969466 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e677f73f0 con 0x559e6789a010 2024-03-21T11:27:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.666+0000 7f2a77400640 1 --2- 172.21.15.47:0/1371969466 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e6789a010 0x559e6789c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.666+0000 7f2a77400640 1 --2- 172.21.15.47:0/1371969466 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e6789a010 0x559e6789c4d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e4a4a75a59603f6 server_cookie=4fd6574c42a2ba1b in_seq=0 out_seq=0 2024-03-21T11:27:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.666+0000 7f2a66000640 10 client.27061 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a66000640 1 -- 172.21.15.47:0/1371969466 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559e677f73f0 con 0x559e6789a010 2024-03-21T11:27:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a66000640 10 client.27061 handle_command_reply: tid=0 2024-03-21T11:27:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a81e0e740 1 client.27061 shutdown 2024-03-21T11:27:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1371969466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e678840a0 msgr2=0x559e6788e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/1371969466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e678840a0 0x559e6788e280 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2a68098b30 tx=0x7f2a68009f60 comp rx=0 tx=0).stop 2024-03-21T11:27:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1371969466 shutdown_connections 2024-03-21T11:27:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/1371969466 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e6789a010 0x559e6789c4d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/1371969466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e67895790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/1371969466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e67884a90 0x559e6788e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/1371969466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e678840a0 0x559e6788e280 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a66000640 0 client.27061 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1371969466 >> 172.21.15.47:0/1371969466 conn(0x559e6787f610 msgr2=0x559e6787e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1371969466 shutdown_connections 2024-03-21T11:27:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1371969466 wait complete. 2024-03-21T11:27:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.668+0000 7f2a81e0e740 20 client.27061 trim_cache size 0 max 16384 2024-03-21T11:27:50.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.688+0000 7f2a81e0e740 1 Processor -- start 2024-03-21T11:27:50.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.688+0000 7f2a81e0e740 1 -- start start 2024-03-21T11:27:50.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.688+0000 7f2a81e0e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e678840a0 0x559e67882ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.688+0000 7f2a81e0e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67884a90 0x559e67895750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.688+0000 7f2a81e0e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e67889c00 0x559e67885320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.688+0000 7f2a81e0e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781a7e90 con 0x559e678840a0 2024-03-21T11:27:50.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.688+0000 7f2a81e0e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781a9810 con 0x559e67884a90 2024-03-21T11:27:50.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.688+0000 7f2a81e0e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781a84f0 con 0x559e67889c00 2024-03-21T11:27:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.688+0000 7f2a77400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67884a90 0x559e67895750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.689+0000 7f2a77400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67884a90 0x559e67895750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36414/0 (socket says 172.21.15.47:36414) 2024-03-21T11:27:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.689+0000 7f2a77400640 1 -- 172.21.15.47:0/3480321681 learned_addr learned my addr 172.21.15.47:0/3480321681 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.689+0000 7f2a7d000640 1 --2- 172.21.15.47:0/3480321681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e67889c00 0x559e67885320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.689+0000 7f2a77e00640 1 --2- 172.21.15.47:0/3480321681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e678840a0 0x559e67882ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.689+0000 7f2a77400640 1 -- 172.21.15.47:0/3480321681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e67889c00 msgr2=0x559e67885320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:50.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.689+0000 7f2a77400640 1 --2- 172.21.15.47:0/3480321681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e67889c00 0x559e67885320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.689+0000 7f2a77400640 1 -- 172.21.15.47:0/3480321681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e678840a0 msgr2=0x559e67882ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.689+0000 7f2a77400640 1 --2- 172.21.15.47:0/3480321681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e678840a0 0x559e67882ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.689+0000 7f2a77400640 1 -- 172.21.15.47:0/3480321681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e6787ac20 con 0x559e67884a90 2024-03-21T11:27:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.689+0000 7f2a77e00640 1 --2- 172.21.15.47:0/3480321681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e678840a0 0x559e67882ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:27:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.689+0000 7f2a77400640 1 --2- 172.21.15.47:0/3480321681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67884a90 0x559e67895750 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2a60004650 tx=0x7f2a600045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=477317808c4cdc88 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.690+0000 7f2a6cc00640 1 -- 172.21.15.47:0/3480321681 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a60005040 con 0x559e67884a90 2024-03-21T11:27:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.690+0000 7f2a6cc00640 1 -- 172.21.15.47:0/3480321681 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a600046c0 con 0x559e67884a90 2024-03-21T11:27:50.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.690+0000 7f2a6cc00640 1 -- 172.21.15.47:0/3480321681 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a600346a0 con 0x559e67884a90 2024-03-21T11:27:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.691+0000 7f2a81e0e740 1 -- 172.21.15.47:0/3480321681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67884a90 msgr2=0x559e67895750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.691+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/3480321681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67884a90 0x559e67895750 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2a60004650 tx=0x7f2a600045e0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.691+0000 7f2a81e0e740 1 -- 172.21.15.47:0/3480321681 shutdown_connections 2024-03-21T11:27:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.691+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/3480321681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e67889c00 0x559e67885320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.691+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/3480321681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67884a90 0x559e67895750 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.691+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/3480321681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e678840a0 0x559e67882ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.691+0000 7f2a81e0e740 1 -- 172.21.15.47:0/3480321681 >> 172.21.15.47:0/3480321681 conn(0x559e6787f610 msgr2=0x559e6787e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.691+0000 7f2a81e0e740 1 -- 172.21.15.47:0/3480321681 shutdown_connections 2024-03-21T11:27:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.691+0000 7f2a81e0e740 1 -- 172.21.15.47:0/3480321681 wait complete. 2024-03-21T11:27:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.692+0000 7f2a81e0e740 1 -- start start 2024-03-21T11:27:50.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.692+0000 7f2a81e0e740 1 Processor -- start 2024-03-21T11:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.692+0000 7f2a81e0e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e678840a0 0x559e67895370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.692+0000 7f2a81e0e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e67884a90 0x559e678939d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.692+0000 7f2a81e0e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e67893f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.692+0000 7f2a81e0e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781a91b0 con 0x559e67884a90 2024-03-21T11:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.692+0000 7f2a81e0e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781a8b50 con 0x559e67889c00 2024-03-21T11:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.692+0000 7f2a81e0e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781a6050 con 0x559e678840a0 2024-03-21T11:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.693+0000 7f2a81e0e740 10 client.0 _ll_register_callbacks cb 0x7f2a680048f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.693+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e678840a0 0x559e67895370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.693+0000 7f2a7d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e67893f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.693+0000 7f2a77e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e678840a0 0x559e67895370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49158/0 (socket says 172.21.15.47:49158) 2024-03-21T11:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.693+0000 7f2a77e00640 1 -- 172.21.15.47:0/1820096943 learned_addr learned my addr 172.21.15.47:0/1820096943 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.693+0000 7f2a77400640 1 --2- 172.21.15.47:0/1820096943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e67884a90 0x559e678939d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.693+0000 7f2a7d000640 1 -- 172.21.15.47:0/1820096943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e678840a0 msgr2=0x559e67895370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.693+0000 7f2a7d000640 1 --2- 172.21.15.47:0/1820096943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e678840a0 0x559e67895370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.693+0000 7f2a7d000640 1 -- 172.21.15.47:0/1820096943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e67884a90 msgr2=0x559e678939d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.693+0000 7f2a7d000640 1 --2- 172.21.15.47:0/1820096943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e67884a90 0x559e678939d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.693+0000 7f2a7d000640 1 -- 172.21.15.47:0/1820096943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e6788ce70 con 0x559e67889c00 2024-03-21T11:27:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.693+0000 7f2a77400640 1 --2- 172.21.15.47:0/1820096943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e67884a90 0x559e678939d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.693+0000 7f2a7d000640 1 --2- 172.21.15.47:0/1820096943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e67893f10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2a70013df0 tx=0x7f2a70004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bded70ef6782bc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.693+0000 7f2a66000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:27:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.694+0000 7f2a66000640 1 -- 172.21.15.47:0/1820096943 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70005020 con 0x559e67889c00 2024-03-21T11:27:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.694+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1820096943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a600046c0 con 0x559e67889c00 2024-03-21T11:27:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.694+0000 7f2a66000640 1 -- 172.21.15.47:0/1820096943 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2a700051c0 con 0x559e67889c00 2024-03-21T11:27:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.694+0000 7f2a66000640 1 -- 172.21.15.47:0/1820096943 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a70037050 con 0x559e67889c00 2024-03-21T11:27:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.694+0000 7f2a81e0e740 10 client.27053 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.694+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1820096943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559e678feeb0 con 0x559e67889c00 2024-03-21T11:27:50.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.695+0000 7f2a66000640 1 -- 172.21.15.47:0/1820096943 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a600046c0 con 0x559e67889c00 2024-03-21T11:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.695+0000 7f2a66000640 1 -- 172.21.15.47:0/1820096943 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2a70011440 con 0x559e67889c00 2024-03-21T11:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.695+0000 7f2a81e0e740 10 client.27053 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.695+0000 7f2a81e0e740 10 client.27053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.695+0000 7f2a81e0e740 20 client.27053 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.695+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/1820096943 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e678ff870 0x559e67901c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.695+0000 7f2a81e0e740 4 client.27053 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.695+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1820096943 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559e677f1740 con 0x559e678ff870 2024-03-21T11:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.695+0000 7f2a77e00640 1 --2- 172.21.15.47:0/1820096943 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e678ff870 0x559e67901c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.696+0000 7f2a77e00640 1 --2- 172.21.15.47:0/1820096943 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e678ff870 0x559e67901c90 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5921f639a4d23ccb server_cookie=c4c9913f879ad533 in_seq=0 out_seq=0 2024-03-21T11:27:50.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.696+0000 7f2a66000640 10 client.27053 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:50.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a66000640 1 -- 172.21.15.47:0/1820096943 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559e677f1740 con 0x559e678ff870 2024-03-21T11:27:50.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a66000640 10 client.27053 handle_command_reply: tid=0 2024-03-21T11:27:50.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a81e0e740 1 client.27053 shutdown 2024-03-21T11:27:50.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1820096943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 msgr2=0x559e67893f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:50.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/1820096943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e67893f10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2a70013df0 tx=0x7f2a70004730 comp rx=0 tx=0).stop 2024-03-21T11:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1820096943 shutdown_connections 2024-03-21T11:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/1820096943 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e678ff870 0x559e67901c90 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/1820096943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e67889c00 0x559e67893f10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/1820096943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e67884a90 0x559e678939d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a81e0e740 1 --2- 172.21.15.47:0/1820096943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e678840a0 0x559e67895370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1820096943 >> 172.21.15.47:0/1820096943 conn(0x559e6787f610 msgr2=0x559e678a0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a66000640 0 client.27053 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1820096943 shutdown_connections 2024-03-21T11:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a81e0e740 1 -- 172.21.15.47:0/1820096943 wait complete. 2024-03-21T11:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.700+0000 7f2a81e0e740 20 client.27053 trim_cache size 0 max 16384 2024-03-21T11:27:50.703 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:27:50.703 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:27:50.703 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "308e4830-bc4c-46a3-90f4-23130b732cb1", 2024-03-21T11:27:50.703 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:27:50.703 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:27:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.701+0000 7f2a7f000640 1 -- 172.21.15.47:0/2971719420 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2a5c0a2fa0 msgr2=0x7f2a5c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.701+0000 7f2a7f000640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2a5c0a2fa0 0x7f2a5c0a5460 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x559e678852f0 tx=0x7f2a70015040 comp rx=0 tx=0).stop 2024-03-21T11:27:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.701+0000 7f2a7f000640 1 -- 172.21.15.47:0/2971719420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a78102130 msgr2=0x7f2a78197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.701+0000 7f2a7f000640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a78102130 0x7f2a78197a20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009d50 tx=0x7f2a6800c6a0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.701+0000 7f2a7f000640 1 -- 172.21.15.47:0/2971719420 shutdown_connections 2024-03-21T11:27:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.701+0000 7f2a7f000640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2a5c0a2fa0 0x7f2a5c0a5460 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.701+0000 7f2a7f000640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a78107390 0x7f2a7819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.701+0000 7f2a7f000640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a78104a60 0x7f2a78197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.701+0000 7f2a7f000640 1 --2- 172.21.15.47:0/2971719420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a78102130 0x7f2a78197a20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.701+0000 7f2a7f000640 1 -- 172.21.15.47:0/2971719420 >> 172.21.15.47:0/2971719420 conn(0x7f2a780fbb40 msgr2=0x7f2a78102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.701+0000 7f2a7f000640 1 -- 172.21.15.47:0/2971719420 shutdown_connections 2024-03-21T11:27:50.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.702+0000 7f2a7f000640 1 -- 172.21.15.47:0/2971719420 wait complete. 2024-03-21T11:27:50.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:27:50.723 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:27:50.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.978+0000 7f8e7cc00640 1 -- 172.21.15.47:0/2075072223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e78104930 msgr2=0x7f8e78106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:50.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.978+0000 7f8e7cc00640 1 --2- 172.21.15.47:0/2075072223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e78104930 0x7f8e78106d20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009900 tx=0x7f8e6402bc40 comp rx=0 tx=0).stop 2024-03-21T11:27:50.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.978+0000 7f8e7cc00640 1 -- 172.21.15.47:0/2075072223 shutdown_connections 2024-03-21T11:27:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.978+0000 7f8e7cc00640 1 --2- 172.21.15.47:0/2075072223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e78107260 0x7f8e781096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.978+0000 7f8e7cc00640 1 --2- 172.21.15.47:0/2075072223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e78104930 0x7f8e78106d20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.978+0000 7f8e7cc00640 1 --2- 172.21.15.47:0/2075072223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e78102000 0x7f8e781043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.978+0000 7f8e7cc00640 1 -- 172.21.15.47:0/2075072223 >> 172.21.15.47:0/2075072223 conn(0x7f8e780fba10 msgr2=0x7f8e780fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.978+0000 7f8e7cc00640 1 -- 172.21.15.47:0/2075072223 shutdown_connections 2024-03-21T11:27:50.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.978+0000 7f8e7cc00640 1 -- 172.21.15.47:0/2075072223 wait complete. 2024-03-21T11:27:50.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.979+0000 7f8e7cc00640 1 Processor -- start 2024-03-21T11:27:50.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.979+0000 7f8e7cc00640 1 -- start start 2024-03-21T11:27:50.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e7cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e78102000 0x7f8e78199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e7cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e78104930 0x7f8e78199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e7cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e78107260 0x7f8e781a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e7cc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7810d210 con 0x7f8e78104930 2024-03-21T11:27:50.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e7cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7810d090 con 0x7f8e78107260 2024-03-21T11:27:50.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e7cc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7810d390 con 0x7f8e78102000 2024-03-21T11:27:50.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e77400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e78102000 0x7f8e78199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e77400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e78102000 0x7f8e78199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49166/0 (socket says 172.21.15.47:49166) 2024-03-21T11:27:50.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e77400640 1 -- 172.21.15.47:0/652403411 learned_addr learned my addr 172.21.15.47:0/652403411 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:50.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e76a00640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e78104930 0x7f8e78199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e77e00640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e78107260 0x7f8e781a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e77400640 1 -- 172.21.15.47:0/652403411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e78107260 msgr2=0x7f8e781a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:50.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e77400640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e78107260 0x7f8e781a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e77400640 1 -- 172.21.15.47:0/652403411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e78104930 msgr2=0x7f8e78199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:50.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e77400640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e78104930 0x7f8e78199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e77400640 1 -- 172.21.15.47:0/652403411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e781a16c0 con 0x7f8e78102000 2024-03-21T11:27:50.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e77e00640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e78107260 0x7f8e781a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:50.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.980+0000 7f8e76a00640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e78104930 0x7f8e78199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:50.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.981+0000 7f8e77400640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e78102000 0x7f8e78199aa0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800ea60 tx=0x7f8e6800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:50.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.981+0000 7f8e6fe00640 1 -- 172.21.15.47:0/652403411 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6800cde0 con 0x7f8e78102000 2024-03-21T11:27:50.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.981+0000 7f8e7cc00640 1 -- 172.21.15.47:0/652403411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e781a19b0 con 0x7f8e78102000 2024-03-21T11:27:50.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.981+0000 7f8e6fe00640 1 -- 172.21.15.47:0/652403411 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e680035f0 con 0x7f8e78102000 2024-03-21T11:27:50.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.981+0000 7f8e6fe00640 1 -- 172.21.15.47:0/652403411 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68010670 con 0x7f8e78102000 2024-03-21T11:27:50.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.981+0000 7f8e7cc00640 1 -- 172.21.15.47:0/652403411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e781a1f20 con 0x7f8e78102000 2024-03-21T11:27:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.984+0000 7f8e6fe00640 1 -- 172.21.15.47:0/652403411 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8e680037a0 con 0x7f8e78102000 2024-03-21T11:27:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.984+0000 7f8e80f73740 1 Processor -- start 2024-03-21T11:27:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.985+0000 7f8e80f73740 1 -- start start 2024-03-21T11:27:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.985+0000 7f8e80f73740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e62470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.985+0000 7f8e80f73740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62a40 0x556008e67540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.985+0000 7f8e80f73740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 0x556008e6a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.985+0000 7f8e80f73740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556008dc7250 con 0x556008e62050 2024-03-21T11:27:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.985+0000 7f8e80f73740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556008d6fa30 con 0x556008e67bb0 2024-03-21T11:27:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.985+0000 7f8e80f73740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556008dc32f0 con 0x556008e62a40 2024-03-21T11:27:50.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.985+0000 7f8e77e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e62470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.985+0000 7f8e77e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e62470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50250/0 (socket says 172.21.15.47:50250) 2024-03-21T11:27:50.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.985+0000 7f8e77400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62a40 0x556008e67540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.985+0000 7f8e77e00640 1 -- 172.21.15.47:0/2779197883 learned_addr learned my addr 172.21.15.47:0/2779197883 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:50.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.985+0000 7f8e76a00640 1 --2- 172.21.15.47:0/2779197883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 0x556008e6a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.986+0000 7f8e77e00640 1 -- 172.21.15.47:0/2779197883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62a40 msgr2=0x556008e67540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:50.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.986+0000 7f8e77e00640 1 --2- 172.21.15.47:0/2779197883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62a40 0x556008e67540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.986+0000 7f8e77e00640 1 -- 172.21.15.47:0/2779197883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 msgr2=0x556008e6a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:50.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.986+0000 7f8e77e00640 1 --2- 172.21.15.47:0/2779197883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 0x556008e6a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.986+0000 7f8e77e00640 1 -- 172.21.15.47:0/2779197883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556008d5a690 con 0x556008e62050 2024-03-21T11:27:50.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.986+0000 7f8e77e00640 1 --2- 172.21.15.47:0/2779197883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e62470 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009fd0 tx=0x7f8e700108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cda0833493ad3c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:50.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.986+0000 7f8e6fe00640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.986+0000 7f8e63e00640 1 -- 172.21.15.47:0/2779197883 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70011760 con 0x556008e62050 2024-03-21T11:27:50.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.986+0000 7f8e63e00640 1 -- 172.21.15.47:0/2779197883 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e70011de0 con 0x556008e62050 2024-03-21T11:27:50.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.986+0000 7f8e63e00640 1 -- 172.21.15.47:0/2779197883 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7000e3c0 con 0x556008e62050 2024-03-21T11:27:50.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.986+0000 7f8e6fe00640 1 -- 172.21.15.47:0/652403411 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8e680cfc30 con 0x7f8e78102000 2024-03-21T11:27:50.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.987+0000 7f8e77400640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.987+0000 7f8e77400640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8e680983d0 tx=0x7f8e680d3670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:50.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.987+0000 7f8e80f73740 1 -- 172.21.15.47:0/2779197883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 msgr2=0x556008e62470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:50.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.987+0000 7f8e80f73740 1 --2- 172.21.15.47:0/2779197883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e62470 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009fd0 tx=0x7f8e700108e0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.987+0000 7f8e80f73740 1 -- 172.21.15.47:0/2779197883 shutdown_connections 2024-03-21T11:27:50.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.987+0000 7f8e80f73740 1 --2- 172.21.15.47:0/2779197883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 0x556008e6a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.987+0000 7f8e80f73740 1 --2- 172.21.15.47:0/2779197883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62a40 0x556008e67540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.987+0000 7f8e80f73740 1 --2- 172.21.15.47:0/2779197883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e62470 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.987+0000 7f8e80f73740 1 -- 172.21.15.47:0/2779197883 >> 172.21.15.47:0/2779197883 conn(0x556008e2a200 msgr2=0x556008e5d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:50.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.987+0000 7f8e80f73740 1 -- 172.21.15.47:0/2779197883 shutdown_connections 2024-03-21T11:27:50.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.987+0000 7f8e80f73740 1 -- 172.21.15.47:0/2779197883 wait complete. 2024-03-21T11:27:50.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.988+0000 7f8e80f73740 1 -- start start 2024-03-21T11:27:50.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.988+0000 7f8e80f73740 1 Processor -- start 2024-03-21T11:27:50.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e80f73740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62050 0x556008e6c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e80f73740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62a40 0x556008e6c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e80f73740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 0x556008e735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e80f73740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556008e6ab60 con 0x556008e62a40 2024-03-21T11:27:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e80f73740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556008d82e80 con 0x556008e67bb0 2024-03-21T11:27:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e80f73740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556008e6ace0 con 0x556008e62050 2024-03-21T11:27:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e80f73740 10 client.0 _ll_register_callbacks cb 0x556008d95130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e77e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62050 0x556008e6c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e77e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62050 0x556008e6c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49170/0 (socket says 172.21.15.47:49170) 2024-03-21T11:27:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e77e00640 1 -- 172.21.15.47:0/281798583 learned_addr learned my addr 172.21.15.47:0/281798583 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e77e00640 1 --2- 172.21.15.47:0/281798583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 0x556008e735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e76a00640 1 --2- 172.21.15.47:0/281798583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62a40 0x556008e6c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e77e00640 1 -- 172.21.15.47:0/281798583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 msgr2=0x556008e735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e77e00640 1 --2- 172.21.15.47:0/281798583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 0x556008e735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e77e00640 1 -- 172.21.15.47:0/281798583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62a40 msgr2=0x556008e6c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e77e00640 1 --2- 172.21.15.47:0/281798583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62a40 0x556008e6c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.989+0000 7f8e77e00640 1 -- 172.21.15.47:0/281798583 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556008ba57d0 con 0x556008e62050 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.990+0000 7f8e77e00640 1 --2- 172.21.15.47:0/281798583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62050 0x556008e6c040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8e7001b7d0 tx=0x7f8e70011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dda7cc67f01dc1ea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.990+0000 7f8e60c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.990+0000 7f8e60c00640 1 -- 172.21.15.47:0/281798583 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70024040 con 0x556008e62050 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.990+0000 7f8e60c00640 1 -- 172.21.15.47:0/281798583 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8e70004c40 con 0x556008e62050 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.990+0000 7f8e80f73740 1 -- 172.21.15.47:0/281798583 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e70011de0 con 0x556008e62050 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.990+0000 7f8e60c00640 1 -- 172.21.15.47:0/281798583 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e70032050 con 0x556008e62050 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.990+0000 7f8e80f73740 10 client.27079 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.990+0000 7f8e80f73740 1 -- 172.21.15.47:0/281798583 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556008adb3f0 con 0x556008e62050 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.991+0000 7f8e60c00640 1 -- 172.21.15.47:0/281798583 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e70011de0 con 0x556008e62050 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.991+0000 7f8e60c00640 1 -- 172.21.15.47:0/281798583 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8e700053c0 con 0x556008e62050 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.991+0000 7f8e80f73740 10 client.27079 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.991+0000 7f8e80f73740 10 client.27079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.991+0000 7f8e80f73740 20 client.27079 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.991+0000 7f8e80f73740 1 --2- 172.21.15.47:0/281798583 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556008e77bd0 0x556008e79fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.991+0000 7f8e80f73740 4 client.27079 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:27:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.991+0000 7f8e80f73740 1 -- 172.21.15.47:0/281798583 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556008dd4790 con 0x556008e77bd0 2024-03-21T11:27:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.991+0000 7f8e76a00640 1 --2- 172.21.15.47:0/281798583 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556008e77bd0 0x556008e79fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.992+0000 7f8e76a00640 1 --2- 172.21.15.47:0/281798583 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556008e77bd0 0x556008e79fc0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f20e299337722fd server_cookie=6e27c0bcaf1fe539 in_seq=0 out_seq=0 2024-03-21T11:27:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.992+0000 7f8e60c00640 10 client.27079 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.994+0000 7f8e60c00640 1 -- 172.21.15.47:0/281798583 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556008dd4790 con 0x556008e77bd0 2024-03-21T11:27:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.994+0000 7f8e60c00640 10 client.27079 handle_command_reply: tid=0 2024-03-21T11:27:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.994+0000 7f8e80f73740 1 client.27079 shutdown 2024-03-21T11:27:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.995+0000 7f8e80f73740 1 -- 172.21.15.47:0/281798583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62050 msgr2=0x556008e6c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.995+0000 7f8e80f73740 1 --2- 172.21.15.47:0/281798583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62050 0x556008e6c040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8e7001b7d0 tx=0x7f8e70011f90 comp rx=0 tx=0).stop 2024-03-21T11:27:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.995+0000 7f8e80f73740 1 -- 172.21.15.47:0/281798583 shutdown_connections 2024-03-21T11:27:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.995+0000 7f8e80f73740 1 --2- 172.21.15.47:0/281798583 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556008e77bd0 0x556008e79fc0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.995+0000 7f8e80f73740 1 --2- 172.21.15.47:0/281798583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 0x556008e735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.995+0000 7f8e80f73740 1 --2- 172.21.15.47:0/281798583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62a40 0x556008e6c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.995+0000 7f8e80f73740 1 --2- 172.21.15.47:0/281798583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62050 0x556008e6c040 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.995+0000 7f8e60c00640 0 client.27079 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.995+0000 7f8e80f73740 1 -- 172.21.15.47:0/281798583 >> 172.21.15.47:0/281798583 conn(0x556008e2a200 msgr2=0x556008e5cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.995+0000 7f8e80f73740 1 -- 172.21.15.47:0/281798583 shutdown_connections 2024-03-21T11:27:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.995+0000 7f8e80f73740 1 -- 172.21.15.47:0/281798583 wait complete. 2024-03-21T11:27:50.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:50.995+0000 7f8e80f73740 20 client.27079 trim_cache size 0 max 16384 2024-03-21T11:27:51.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.014+0000 7f8e80f73740 1 Processor -- start 2024-03-21T11:27:51.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.014+0000 7f8e80f73740 1 -- start start 2024-03-21T11:27:51.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.014+0000 7f8e80f73740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e7e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:51.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.014+0000 7f8e80f73740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e62a40 0x556008e73580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.014+0000 7f8e80f73740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e67bb0 0x556008e7edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.014+0000 7f8e80f73740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e781ac100 con 0x556008e62050 2024-03-21T11:27:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.014+0000 7f8e80f73740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e781ada80 con 0x556008e62a40 2024-03-21T11:27:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.014+0000 7f8e80f73740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e781ac760 con 0x556008e67bb0 2024-03-21T11:27:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.014+0000 7f8e76a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e62a40 0x556008e73580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.014+0000 7f8e76a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e62a40 0x556008e73580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36440/0 (socket says 172.21.15.47:36440) 2024-03-21T11:27:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.014+0000 7f8e76a00640 1 -- 172.21.15.47:0/3367655684 learned_addr learned my addr 172.21.15.47:0/3367655684 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.014+0000 7f8e77e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e7e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.015+0000 7f8e77e00640 1 --2- 172.21.15.47:0/3367655684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e67bb0 0x556008e7edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.015+0000 7f8e76a00640 1 -- 172.21.15.47:0/3367655684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e67bb0 msgr2=0x556008e7edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.015+0000 7f8e76a00640 1 --2- 172.21.15.47:0/3367655684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e67bb0 0x556008e7edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.015+0000 7f8e76a00640 1 -- 172.21.15.47:0/3367655684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 msgr2=0x556008e7e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:51.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.015+0000 7f8e76a00640 1 --2- 172.21.15.47:0/3367655684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e7e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.015+0000 7f8e76a00640 1 -- 172.21.15.47:0/3367655684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556008e7a330 con 0x556008e62a40 2024-03-21T11:27:51.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.015+0000 7f8e76a00640 1 --2- 172.21.15.47:0/3367655684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e62a40 0x556008e73580 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8e6402c7c0 tx=0x7f8e64005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79e756d09e8783a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:51.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.015+0000 7f8e63400640 1 -- 172.21.15.47:0/3367655684 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64004040 con 0x556008e62a40 2024-03-21T11:27:51.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.015+0000 7f8e63400640 1 -- 172.21.15.47:0/3367655684 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e6404b070 con 0x556008e62a40 2024-03-21T11:27:51.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.015+0000 7f8e63400640 1 -- 172.21.15.47:0/3367655684 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e640046c0 con 0x556008e62a40 2024-03-21T11:27:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.016+0000 7f8e80f73740 1 -- 172.21.15.47:0/3367655684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e62a40 msgr2=0x556008e73580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.016+0000 7f8e80f73740 1 --2- 172.21.15.47:0/3367655684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e62a40 0x556008e73580 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8e6402c7c0 tx=0x7f8e64005a20 comp rx=0 tx=0).stop 2024-03-21T11:27:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.016+0000 7f8e80f73740 1 -- 172.21.15.47:0/3367655684 shutdown_connections 2024-03-21T11:27:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.016+0000 7f8e80f73740 1 --2- 172.21.15.47:0/3367655684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e67bb0 0x556008e7edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.016+0000 7f8e80f73740 1 --2- 172.21.15.47:0/3367655684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e62a40 0x556008e73580 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.016+0000 7f8e80f73740 1 --2- 172.21.15.47:0/3367655684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e7e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.016+0000 7f8e80f73740 1 -- 172.21.15.47:0/3367655684 >> 172.21.15.47:0/3367655684 conn(0x556008e2a200 msgr2=0x556008e63d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.016+0000 7f8e80f73740 1 -- 172.21.15.47:0/3367655684 shutdown_connections 2024-03-21T11:27:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.016+0000 7f8e80f73740 1 -- 172.21.15.47:0/3367655684 wait complete. 2024-03-21T11:27:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.017+0000 7f8e80f73740 1 -- start start 2024-03-21T11:27:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.017+0000 7f8e80f73740 1 Processor -- start 2024-03-21T11:27:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.017+0000 7f8e80f73740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e71040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.017+0000 7f8e80f73740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62a40 0x556008e71580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:51.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.017+0000 7f8e80f73740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 0x556008e724f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:51.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.017+0000 7f8e80f73740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e781ad420 con 0x556008e62050 2024-03-21T11:27:51.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.017+0000 7f8e80f73740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e781acdc0 con 0x556008e67bb0 2024-03-21T11:27:51.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.017+0000 7f8e80f73740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e781aa2c0 con 0x556008e62a40 2024-03-21T11:27:51.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.018+0000 7f8e80f73740 10 client.0 _ll_register_callbacks cb 0x7f8e700093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:27:51.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.018+0000 7f8e76a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62a40 0x556008e71580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:51.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.018+0000 7f8e76a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62a40 0x556008e71580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49178/0 (socket says 172.21.15.47:49178) 2024-03-21T11:27:51.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.018+0000 7f8e76a00640 1 -- 172.21.15.47:0/2423078205 learned_addr learned my addr 172.21.15.47:0/2423078205 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:27:51.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.018+0000 7f8e77e00640 1 --2- 172.21.15.47:0/2423078205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e71040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:51.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.018+0000 7f8e77e00640 1 --2- 172.21.15.47:0/2423078205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 0x556008e724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:51.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.018+0000 7f8e76a00640 1 -- 172.21.15.47:0/2423078205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 msgr2=0x556008e724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:51.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.018+0000 7f8e76a00640 1 --2- 172.21.15.47:0/2423078205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 0x556008e724f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.018+0000 7f8e76a00640 1 -- 172.21.15.47:0/2423078205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 msgr2=0x556008e71040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:27:51.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.018+0000 7f8e76a00640 1 --2- 172.21.15.47:0/2423078205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e71040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.018+0000 7f8e76a00640 1 -- 172.21.15.47:0/2423078205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556008a9b0f0 con 0x556008e62a40 2024-03-21T11:27:51.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.018+0000 7f8e77e00640 1 --2- 172.21.15.47:0/2423078205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e71040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:27:51.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.018+0000 7f8e76a00640 1 --2- 172.21.15.47:0/2423078205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62a40 0x556008e71580 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009850 tx=0x7f8e64004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d355375bb1fe1b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:27:51.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.018+0000 7f8e60c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:27:51.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.019+0000 7f8e60c00640 1 -- 172.21.15.47:0/2423078205 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64053020 con 0x556008e62a40 2024-03-21T11:27:51.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.019+0000 7f8e80f73740 1 -- 172.21.15.47:0/2423078205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e6404b070 con 0x556008e62a40 2024-03-21T11:27:51.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.019+0000 7f8e60c00640 1 -- 172.21.15.47:0/2423078205 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8e64005240 con 0x556008e62a40 2024-03-21T11:27:51.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.019+0000 7f8e60c00640 1 -- 172.21.15.47:0/2423078205 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e640044b0 con 0x556008e62a40 2024-03-21T11:27:51.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.019+0000 7f8e80f73740 10 client.27085 fetch_fsmap learned FSMap version 28 2024-03-21T11:27:51.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.019+0000 7f8e80f73740 1 -- 172.21.15.47:0/2423078205 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556008ed35c0 con 0x556008e62a40 2024-03-21T11:27:51.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.019+0000 7f8e60c00640 1 -- 172.21.15.47:0/2423078205 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e6404b070 con 0x556008e62a40 2024-03-21T11:27:51.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.019+0000 7f8e60c00640 1 -- 172.21.15.47:0/2423078205 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8e64046600 con 0x556008e62a40 2024-03-21T11:27:51.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.020+0000 7f8e80f73740 10 client.27085 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:27:51.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.020+0000 7f8e80f73740 10 client.27085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:27:51.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.020+0000 7f8e80f73740 20 client.27085 populate_metadata read hostname 'smithi047' 2024-03-21T11:27:51.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.020+0000 7f8e80f73740 1 --2- 172.21.15.47:0/2423078205 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556008ed3fa0 0x556008ed63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:27:51.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.020+0000 7f8e80f73740 4 client.27085 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:27:51.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.020+0000 7f8e80f73740 1 -- 172.21.15.47:0/2423078205 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556008dceae0 con 0x556008ed3fa0 2024-03-21T11:27:51.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.020+0000 7f8e77e00640 1 --2- 172.21.15.47:0/2423078205 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556008ed3fa0 0x556008ed63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:27:51.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.020+0000 7f8e77e00640 1 --2- 172.21.15.47:0/2423078205 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556008ed3fa0 0x556008ed63c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4a705fa905aa696 server_cookie=50f7980a1a606a8a in_seq=0 out_seq=0 2024-03-21T11:27:51.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.020+0000 7f8e60c00640 10 client.27085 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:51.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e60c00640 1 -- 172.21.15.47:0/2423078205 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556008dceae0 con 0x556008ed3fa0 2024-03-21T11:27:51.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e60c00640 10 client.27085 handle_command_reply: tid=0 2024-03-21T11:27:51.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e80f73740 1 client.27085 shutdown 2024-03-21T11:27:51.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e80f73740 1 -- 172.21.15.47:0/2423078205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62a40 msgr2=0x556008e71580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:51.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e80f73740 1 --2- 172.21.15.47:0/2423078205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62a40 0x556008e71580 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009850 tx=0x7f8e64004040 comp rx=0 tx=0).stop 2024-03-21T11:27:51.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e80f73740 1 -- 172.21.15.47:0/2423078205 shutdown_connections 2024-03-21T11:27:51.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e80f73740 1 --2- 172.21.15.47:0/2423078205 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556008ed3fa0 0x556008ed63c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e80f73740 1 --2- 172.21.15.47:0/2423078205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556008e67bb0 0x556008e724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e80f73740 1 --2- 172.21.15.47:0/2423078205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556008e62a40 0x556008e71580 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e60c00640 0 client.27085 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:27:51.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e80f73740 1 --2- 172.21.15.47:0/2423078205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556008e62050 0x556008e71040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e80f73740 1 -- 172.21.15.47:0/2423078205 >> 172.21.15.47:0/2423078205 conn(0x556008e2a200 msgr2=0x556008e60c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:51.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e80f73740 1 -- 172.21.15.47:0/2423078205 shutdown_connections 2024-03-21T11:27:51.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e80f73740 1 -- 172.21.15.47:0/2423078205 wait complete. 2024-03-21T11:27:51.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.021+0000 7f8e80f73740 20 client.27085 trim_cache size 0 max 16384 2024-03-21T11:27:51.027 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:27:51.027 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:27:51.027 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:27:51.027 INFO:teuthology.orchestra.run.smithi047.stdout: "308e4830-bc4c-46a3-90f4-23130b732cb1": { 2024-03-21T11:27:51.027 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:27:51.027 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "308e4830-bc4c-46a3-90f4-23130b732cb1", 2024-03-21T11:27:51.027 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:27:51.027 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:27:51.027 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:27:51.027 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:27:51.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.022+0000 7f8e7cc00640 1 -- 172.21.15.47:0/652403411 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8e5c0a2f20 msgr2=0x7f8e5c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:51.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.022+0000 7f8e7cc00640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8e680983d0 tx=0x7f8e680d3670 comp rx=0 tx=0).stop 2024-03-21T11:27:51.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.022+0000 7f8e7cc00640 1 -- 172.21.15.47:0/652403411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e78102000 msgr2=0x7f8e78199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:27:51.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.022+0000 7f8e7cc00640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e78102000 0x7f8e78199aa0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800ea60 tx=0x7f8e6800ef20 comp rx=0 tx=0).stop 2024-03-21T11:27:51.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.022+0000 7f8e7cc00640 1 -- 172.21.15.47:0/652403411 shutdown_connections 2024-03-21T11:27:51.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.022+0000 7f8e7cc00640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.022+0000 7f8e7cc00640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e78107260 0x7f8e781a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.022+0000 7f8e7cc00640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e78104930 0x7f8e78199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.022+0000 7f8e7cc00640 1 --2- 172.21.15.47:0/652403411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e78102000 0x7f8e78199aa0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:27:51.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.022+0000 7f8e7cc00640 1 -- 172.21.15.47:0/652403411 >> 172.21.15.47:0/652403411 conn(0x7f8e780fba10 msgr2=0x7f8e78101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:27:51.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.022+0000 7f8e7cc00640 1 -- 172.21.15.47:0/652403411 shutdown_connections 2024-03-21T11:27:51.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:27:51.022+0000 7f8e7cc00640 1 -- 172.21.15.47:0/652403411 wait complete. 2024-03-21T11:27:51.040 INFO:tasks.cephfs.filesystem:scrub status for tag:308e4830-bc4c-46a3-90f4-23130b732cb1 - {'path': '/', 'tag': '308e4830-bc4c-46a3-90f4-23130b732cb1', 'options': 'recursive,force'} 2024-03-21T11:27:51.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:50 smithi047 ceph-mon[31351]: pgmap v535: 129 pgs: 129 active+clean; 37 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 272 op/s 2024-03-21T11:27:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:50 smithi102 ceph-mon[36748]: pgmap v535: 129 pgs: 129 active+clean; 37 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 272 op/s 2024-03-21T11:27:51.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:50 smithi100 ceph-mon[36245]: pgmap v535: 129 pgs: 129 active+clean; 37 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 272 op/s 2024-03-21T11:27:51.506 INFO:tasks.workunit.client.0.smithi047.stdout: 1 248478 8.21 MB/sec execute 35 sec latency 39.737 ms 2024-03-21T11:27:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:51 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:27:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:51 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:51 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:27:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:51 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:52.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:51 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:27:52.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:51 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:52.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:51 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:27:52.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:51 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:52.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:51 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:27:52.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:51 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:52.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:51 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:27:52.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:51 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:27:52.506 INFO:tasks.workunit.client.0.smithi047.stdout: 1 249586 8.07 MB/sec execute 36 sec latency 9.242 ms 2024-03-21T11:27:53.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:52 smithi047 ceph-mon[31351]: pgmap v536: 129 pgs: 129 active+clean; 37 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-03-21T11:27:53.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:52 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:27:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:52 smithi102 ceph-mon[36748]: pgmap v536: 129 pgs: 129 active+clean; 37 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-03-21T11:27:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:52 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:27:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:52 smithi100 ceph-mon[36245]: pgmap v536: 129 pgs: 129 active+clean; 37 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-03-21T11:27:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:52 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:27:53.506 INFO:tasks.workunit.client.0.smithi047.stdout: 1 251963 8.28 MB/sec execute 37 sec latency 11.894 ms 2024-03-21T11:27:54.506 INFO:tasks.workunit.client.0.smithi047.stdout: 1 253043 8.15 MB/sec execute 38 sec latency 12.160 ms 2024-03-21T11:27:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:54 smithi102 ceph-mon[36748]: pgmap v537: 129 pgs: 129 active+clean; 32 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-03-21T11:27:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:54 smithi047 ceph-mon[31351]: pgmap v537: 129 pgs: 129 active+clean; 32 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-03-21T11:27:55.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:54 smithi100 ceph-mon[36245]: pgmap v537: 129 pgs: 129 active+clean; 32 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-03-21T11:27:55.506 INFO:tasks.workunit.client.0.smithi047.stdout: 1 255409 8.33 MB/sec execute 39 sec latency 9.221 ms 2024-03-21T11:27:56.507 INFO:tasks.workunit.client.0.smithi047.stdout: 1 256162 8.18 MB/sec execute 40 sec latency 54.758 ms 2024-03-21T11:27:57.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:56 smithi102 ceph-mon[36748]: pgmap v538: 129 pgs: 129 active+clean; 32 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 320 op/s 2024-03-21T11:27:57.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:56 smithi047 ceph-mon[31351]: pgmap v538: 129 pgs: 129 active+clean; 32 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 320 op/s 2024-03-21T11:27:57.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:56 smithi100 ceph-mon[36245]: pgmap v538: 129 pgs: 129 active+clean; 32 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 320 op/s 2024-03-21T11:27:57.507 INFO:tasks.workunit.client.0.smithi047.stdout: 1 258188 8.24 MB/sec execute 41 sec latency 11.141 ms 2024-03-21T11:27:58.506 INFO:tasks.workunit.client.0.smithi047.stdout: 1 259581 8.23 MB/sec execute 42 sec latency 9.720 ms 2024-03-21T11:27:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:27:58 smithi102 ceph-mon[36748]: pgmap v539: 129 pgs: 129 active+clean; 33 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 329 op/s 2024-03-21T11:27:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:27:58 smithi047 ceph-mon[31351]: pgmap v539: 129 pgs: 129 active+clean; 33 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 329 op/s 2024-03-21T11:27:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:27:58 smithi100 ceph-mon[36245]: pgmap v539: 129 pgs: 129 active+clean; 33 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 329 op/s 2024-03-21T11:27:59.507 INFO:tasks.workunit.client.0.smithi047.stdout: 1 261462 8.22 MB/sec execute 43 sec latency 14.898 ms 2024-03-21T11:28:00.507 INFO:tasks.workunit.client.0.smithi047.stdout: 1 263070 8.30 MB/sec execute 44 sec latency 8.639 ms 2024-03-21T11:28:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:00 smithi102 ceph-mon[36748]: pgmap v540: 129 pgs: 129 active+clean; 36 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 321 op/s 2024-03-21T11:28:01.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:00 smithi047 ceph-mon[31351]: pgmap v540: 129 pgs: 129 active+clean; 36 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 321 op/s 2024-03-21T11:28:01.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:00 smithi100 ceph-mon[36245]: pgmap v540: 129 pgs: 129 active+clean; 36 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 321 op/s 2024-03-21T11:28:01.507 INFO:tasks.workunit.client.0.smithi047.stdout: 1 264654 8.24 MB/sec execute 45 sec latency 73.469 ms 2024-03-21T11:28:02.507 INFO:tasks.workunit.client.0.smithi047.stdout: 1 266368 8.34 MB/sec execute 46 sec latency 13.670 ms 2024-03-21T11:28:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.056+0000 7fa6d1400640 1 -- 172.21.15.47:0/1097174897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6cc103900 msgr2=0x7fa6cc103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.056+0000 7fa6d1400640 1 --2- 172.21.15.47:0/1097174897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6cc103900 0x7fa6cc103d20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009900 tx=0x7fa6bc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:28:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.057+0000 7fa6d1400640 1 -- 172.21.15.47:0/1097174897 shutdown_connections 2024-03-21T11:28:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.057+0000 7fa6d1400640 1 --2- 172.21.15.47:0/1097174897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6cc1054b0 0x7fa6cc10bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.057+0000 7fa6d1400640 1 --2- 172.21.15.47:0/1097174897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6cc104af0 0x7fa6cc104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.057+0000 7fa6d1400640 1 --2- 172.21.15.47:0/1097174897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6cc103900 0x7fa6cc103d20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.057+0000 7fa6d1400640 1 -- 172.21.15.47:0/1097174897 >> 172.21.15.47:0/1097174897 conn(0x7fa6cc0fee90 msgr2=0x7fa6cc1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.057+0000 7fa6d1400640 1 -- 172.21.15.47:0/1097174897 shutdown_connections 2024-03-21T11:28:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.057+0000 7fa6d1400640 1 -- 172.21.15.47:0/1097174897 wait complete. 2024-03-21T11:28:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.058+0000 7fa6d1400640 1 Processor -- start 2024-03-21T11:28:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.058+0000 7fa6d1400640 1 -- start start 2024-03-21T11:28:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.058+0000 7fa6d1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6cc103900 0x7fa6cc199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.058+0000 7fa6d1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6cc104af0 0x7fa6cc19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.058+0000 7fa6d1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6cc1054b0 0x7fa6cc1a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.058+0000 7fa6d1400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc10f5c0 con 0x7fa6cc1054b0 2024-03-21T11:28:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.058+0000 7fa6d1400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc10f440 con 0x7fa6cc104af0 2024-03-21T11:28:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.058+0000 7fa6d1400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc10f740 con 0x7fa6cc103900 2024-03-21T11:28:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.058+0000 7fa6ca800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6cc103900 0x7fa6cc199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.059+0000 7fa6c9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6cc104af0 0x7fa6cc19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.059+0000 7fa6ca800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6cc103900 0x7fa6cc199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41432/0 (socket says 172.21.15.47:41432) 2024-03-21T11:28:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.059+0000 7fa6ca800640 1 -- 172.21.15.47:0/3513429043 learned_addr learned my addr 172.21.15.47:0/3513429043 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.059+0000 7fa6cb200640 1 --2- 172.21.15.47:0/3513429043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6cc1054b0 0x7fa6cc1a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.059+0000 7fa6ca800640 1 -- 172.21.15.47:0/3513429043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6cc104af0 msgr2=0x7fa6cc19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.059+0000 7fa6ca800640 1 --2- 172.21.15.47:0/3513429043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6cc104af0 0x7fa6cc19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.059+0000 7fa6ca800640 1 -- 172.21.15.47:0/3513429043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6cc1054b0 msgr2=0x7fa6cc1a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.059+0000 7fa6ca800640 1 --2- 172.21.15.47:0/3513429043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6cc1054b0 0x7fa6cc1a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.059+0000 7fa6ca800640 1 -- 172.21.15.47:0/3513429043 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc1a17c0 con 0x7fa6cc103900 2024-03-21T11:28:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.059+0000 7fa6ca800640 1 --2- 172.21.15.47:0/3513429043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6cc103900 0x7fa6cc199ba0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc000c00 tx=0x7fa6bc02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.060+0000 7fa6c3400640 1 -- 172.21.15.47:0/3513429043 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc043020 con 0x7fa6cc103900 2024-03-21T11:28:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.060+0000 7fa6c3400640 1 -- 172.21.15.47:0/3513429043 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6bc0054e0 con 0x7fa6cc103900 2024-03-21T11:28:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.060+0000 7fa6c3400640 1 -- 172.21.15.47:0/3513429043 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc003bd0 con 0x7fa6cc103900 2024-03-21T11:28:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.060+0000 7fa6d1400640 1 -- 172.21.15.47:0/3513429043 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6cc1a1a50 con 0x7fa6cc103900 2024-03-21T11:28:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.060+0000 7fa6d1400640 1 -- 172.21.15.47:0/3513429043 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6cc1a1f50 con 0x7fa6cc103900 2024-03-21T11:28:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.063+0000 7fa6d4198740 1 Processor -- start 2024-03-21T11:28:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.063+0000 7fa6c3400640 1 -- 172.21.15.47:0/3513429043 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa6bc046020 con 0x7fa6cc103900 2024-03-21T11:28:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.063+0000 7fa6d4198740 1 -- start start 2024-03-21T11:28:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6d4198740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed14921e0 0x562ed1492600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6d4198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed1492bd0 0x562ed14976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6d4198740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1497d40 0x562ed149a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6d4198740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed14553b0 con 0x562ed1497d40 2024-03-21T11:28:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6d4198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed13f7830 con 0x562ed1492bd0 2024-03-21T11:28:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6d4198740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed13a05d0 con 0x562ed14921e0 2024-03-21T11:28:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6cb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed14921e0 0x562ed1492600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6cb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed14921e0 0x562ed1492600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41444/0 (socket says 172.21.15.47:41444) 2024-03-21T11:28:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6cb200640 1 -- 172.21.15.47:0/2608275412 learned_addr learned my addr 172.21.15.47:0/2608275412 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6ca800640 1 --2- 172.21.15.47:0/2608275412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed1492bd0 0x562ed14976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6c9e00640 1 --2- 172.21.15.47:0/2608275412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1497d40 0x562ed149a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6cb200640 1 -- 172.21.15.47:0/2608275412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed1492bd0 msgr2=0x562ed14976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6cb200640 1 --2- 172.21.15.47:0/2608275412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed1492bd0 0x562ed14976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6cb200640 1 -- 172.21.15.47:0/2608275412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1497d40 msgr2=0x562ed149a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6cb200640 1 --2- 172.21.15.47:0/2608275412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1497d40 0x562ed149a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.064+0000 7fa6cb200640 1 -- 172.21.15.47:0/2608275412 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ed13b82c0 con 0x562ed14921e0 2024-03-21T11:28:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.065+0000 7fa6cb200640 1 --2- 172.21.15.47:0/2608275412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed14921e0 0x562ed1492600 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009c90 tx=0x7fa6c4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4696720063d44300 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.065+0000 7fa6c3400640 1 --2- 172.21.15.47:0/3513429043 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6ac0a3040 0x7fa6ac0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.065+0000 7fa6bb400640 1 -- 172.21.15.47:0/2608275412 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4004230 con 0x562ed14921e0 2024-03-21T11:28:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.065+0000 7fa6bb400640 1 -- 172.21.15.47:0/2608275412 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6c4009e40 con 0x562ed14921e0 2024-03-21T11:28:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.065+0000 7fa6bb400640 1 -- 172.21.15.47:0/2608275412 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40197e0 con 0x562ed14921e0 2024-03-21T11:28:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.065+0000 7fa6c3400640 1 -- 172.21.15.47:0/3513429043 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa6bc0f1780 con 0x7fa6cc103900 2024-03-21T11:28:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.065+0000 7fa6ca800640 1 --2- 172.21.15.47:0/3513429043 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6ac0a3040 0x7fa6ac0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.066+0000 7fa6d4198740 1 -- 172.21.15.47:0/2608275412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed14921e0 msgr2=0x562ed1492600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.066+0000 7fa6d4198740 1 --2- 172.21.15.47:0/2608275412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed14921e0 0x562ed1492600 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009c90 tx=0x7fa6c4011810 comp rx=0 tx=0).stop 2024-03-21T11:28:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.066+0000 7fa6d4198740 1 -- 172.21.15.47:0/2608275412 shutdown_connections 2024-03-21T11:28:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.066+0000 7fa6d4198740 1 --2- 172.21.15.47:0/2608275412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1497d40 0x562ed149a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.066+0000 7fa6d4198740 1 --2- 172.21.15.47:0/2608275412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed1492bd0 0x562ed14976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.066+0000 7fa6d4198740 1 --2- 172.21.15.47:0/2608275412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed14921e0 0x562ed1492600 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.066+0000 7fa6d4198740 1 -- 172.21.15.47:0/2608275412 >> 172.21.15.47:0/2608275412 conn(0x562ed148d750 msgr2=0x562ed148db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.066+0000 7fa6d4198740 1 -- 172.21.15.47:0/2608275412 shutdown_connections 2024-03-21T11:28:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.066+0000 7fa6ca800640 1 --2- 172.21.15.47:0/3513429043 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6ac0a3040 0x7fa6ac0a5500 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0ba490 tx=0x7fa6bc0f7530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.066+0000 7fa6d4198740 1 -- 172.21.15.47:0/2608275412 wait complete. 2024-03-21T11:28:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.066+0000 7fa6d4198740 1 -- start start 2024-03-21T11:28:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.067+0000 7fa6d4198740 1 Processor -- start 2024-03-21T11:28:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.067+0000 7fa6d4198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed14921e0 0x562ed149c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.067+0000 7fa6d4198740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1492bd0 0x562ed149c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.067+0000 7fa6d4198740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed1497d40 0x562ed14a3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.067+0000 7fa6d4198740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed13b3a20 con 0x562ed1492bd0 2024-03-21T11:28:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.067+0000 7fa6d4198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed13f38d0 con 0x562ed14921e0 2024-03-21T11:28:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.067+0000 7fa6d4198740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed149acf0 con 0x562ed1497d40 2024-03-21T11:28:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.067+0000 7fa6cb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed14921e0 0x562ed149c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.067+0000 7fa6cb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed14921e0 0x562ed149c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44120/0 (socket says 172.21.15.47:44120) 2024-03-21T11:28:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.067+0000 7fa6cb200640 1 -- 172.21.15.47:0/1698288207 learned_addr learned my addr 172.21.15.47:0/1698288207 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.067+0000 7fa6c9e00640 1 --2- 172.21.15.47:0/1698288207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1492bd0 0x562ed149c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.067+0000 7fa6d4198740 10 client.0 _ll_register_callbacks cb 0x562ed13f01c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.068+0000 7fa6cb200640 1 --2- 172.21.15.47:0/1698288207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed1497d40 0x562ed14a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.068+0000 7fa6c9e00640 1 -- 172.21.15.47:0/1698288207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed1497d40 msgr2=0x562ed14a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.068+0000 7fa6c9e00640 1 --2- 172.21.15.47:0/1698288207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed1497d40 0x562ed14a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.068+0000 7fa6c9e00640 1 -- 172.21.15.47:0/1698288207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed14921e0 msgr2=0x562ed149c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.068+0000 7fa6c9e00640 1 --2- 172.21.15.47:0/1698288207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed14921e0 0x562ed149c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.068+0000 7fa6c9e00640 1 -- 172.21.15.47:0/1698288207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ed110c210 con 0x562ed1492bd0 2024-03-21T11:28:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.068+0000 7fa6c9e00640 1 --2- 172.21.15.47:0/1698288207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1492bd0 0x562ed149c900 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc19b0b0 tx=0x7fa6b4012e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05b8d8033e31f45 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.068+0000 7fa6b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:28:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.069+0000 7fa6b3e00640 1 -- 172.21.15.47:0/1698288207 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4003970 con 0x562ed1492bd0 2024-03-21T11:28:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.069+0000 7fa6d4198740 1 -- 172.21.15.47:0/1698288207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ed1404d70 con 0x562ed1492bd0 2024-03-21T11:28:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.069+0000 7fa6b3e00640 1 -- 172.21.15.47:0/1698288207 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa6b4003b10 con 0x562ed1492bd0 2024-03-21T11:28:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.069+0000 7fa6b3e00640 1 -- 172.21.15.47:0/1698288207 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6b4025050 con 0x562ed1492bd0 2024-03-21T11:28:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.069+0000 7fa6d4198740 10 client.16836 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.069+0000 7fa6d4198740 1 -- 172.21.15.47:0/1698288207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562ed11d8620 con 0x562ed1492bd0 2024-03-21T11:28:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.069+0000 7fa6b3e00640 1 -- 172.21.15.47:0/1698288207 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562ed1404d70 con 0x562ed1492bd0 2024-03-21T11:28:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.069+0000 7fa6b3e00640 1 -- 172.21.15.47:0/1698288207 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa6b40044c0 con 0x562ed1492bd0 2024-03-21T11:28:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.069+0000 7fa6d4198740 10 client.16836 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.069+0000 7fa6d4198740 10 client.16836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.070+0000 7fa6d4198740 20 client.16836 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.070+0000 7fa6d4198740 1 --2- 172.21.15.47:0/1698288207 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ed14a8210 0x562ed14aa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.070+0000 7fa6d4198740 4 client.16836 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.070+0000 7fa6d4198740 1 -- 172.21.15.47:0/1698288207 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ed13ff0c0 con 0x562ed14a8210 2024-03-21T11:28:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.070+0000 7fa6cb200640 1 --2- 172.21.15.47:0/1698288207 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ed14a8210 0x562ed14aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.070+0000 7fa6cb200640 1 --2- 172.21.15.47:0/1698288207 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ed14a8210 0x562ed14aa660 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec2764c4ebf7b451 server_cookie=10dac5ba526bea5a in_seq=0 out_seq=0 2024-03-21T11:28:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.070+0000 7fa6b3e00640 10 client.16836 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.071+0000 7fa6b3e00640 1 -- 172.21.15.47:0/1698288207 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562ed13ff0c0 con 0x562ed14a8210 2024-03-21T11:28:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.071+0000 7fa6b3e00640 10 client.16836 handle_command_reply: tid=0 2024-03-21T11:28:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.071+0000 7fa6d4198740 1 client.16836 shutdown 2024-03-21T11:28:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.071+0000 7fa6d4198740 1 -- 172.21.15.47:0/1698288207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1492bd0 msgr2=0x562ed149c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.071+0000 7fa6d4198740 1 --2- 172.21.15.47:0/1698288207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1492bd0 0x562ed149c900 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc19b0b0 tx=0x7fa6b4012e50 comp rx=0 tx=0).stop 2024-03-21T11:28:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.071+0000 7fa6d4198740 1 -- 172.21.15.47:0/1698288207 shutdown_connections 2024-03-21T11:28:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.071+0000 7fa6d4198740 1 --2- 172.21.15.47:0/1698288207 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ed14a8210 0x562ed14aa660 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.071+0000 7fa6b3e00640 0 client.16836 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.071+0000 7fa6d4198740 1 --2- 172.21.15.47:0/1698288207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed1497d40 0x562ed14a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.071+0000 7fa6d4198740 1 --2- 172.21.15.47:0/1698288207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1492bd0 0x562ed149c900 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.071+0000 7fa6d4198740 1 --2- 172.21.15.47:0/1698288207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed14921e0 0x562ed149c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.071+0000 7fa6d4198740 1 -- 172.21.15.47:0/1698288207 >> 172.21.15.47:0/1698288207 conn(0x562ed148d750 msgr2=0x562ed148c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.071+0000 7fa6d4198740 1 -- 172.21.15.47:0/1698288207 shutdown_connections 2024-03-21T11:28:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.072+0000 7fa6d4198740 1 -- 172.21.15.47:0/1698288207 wait complete. 2024-03-21T11:28:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.072+0000 7fa6d4198740 20 client.16836 trim_cache size 0 max 16384 2024-03-21T11:28:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.087+0000 7fa6d4198740 1 Processor -- start 2024-03-21T11:28:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.087+0000 7fa6d4198740 1 -- start start 2024-03-21T11:28:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.087+0000 7fa6d4198740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed148d750 0x562ed14909b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.087+0000 7fa6d4198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed14921e0 0x562ed14a3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.087+0000 7fa6d4198740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed1492bd0 0x562ed1490ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.087+0000 7fa6d4198740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc1ac270 con 0x562ed148d750 2024-03-21T11:28:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.087+0000 7fa6d4198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc1adbf0 con 0x562ed14921e0 2024-03-21T11:28:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.087+0000 7fa6d4198740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc1ac8d0 con 0x562ed1492bd0 2024-03-21T11:28:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.087+0000 7fa6cb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed148d750 0x562ed14909b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.087+0000 7fa6cb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed148d750 0x562ed14909b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34174/0 (socket says 172.21.15.47:34174) 2024-03-21T11:28:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.087+0000 7fa6cb200640 1 -- 172.21.15.47:0/588855190 learned_addr learned my addr 172.21.15.47:0/588855190 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.088+0000 7fa6cb200640 1 --2- 172.21.15.47:0/588855190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed1492bd0 0x562ed1490ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.088+0000 7fa6c9e00640 1 --2- 172.21.15.47:0/588855190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed14921e0 0x562ed14a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.088+0000 7fa6c9e00640 1 -- 172.21.15.47:0/588855190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed1492bd0 msgr2=0x562ed1490ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.088+0000 7fa6c9e00640 1 --2- 172.21.15.47:0/588855190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed1492bd0 0x562ed1490ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.088+0000 7fa6c9e00640 1 -- 172.21.15.47:0/588855190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed148d750 msgr2=0x562ed14909b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.088+0000 7fa6c9e00640 1 --2- 172.21.15.47:0/588855190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed148d750 0x562ed14909b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.088+0000 7fa6c9e00640 1 -- 172.21.15.47:0/588855190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ed14939e0 con 0x562ed14921e0 2024-03-21T11:28:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.088+0000 7fa6cb200640 1 --2- 172.21.15.47:0/588855190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed148d750 0x562ed14909b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.088+0000 7fa6c9e00640 1 --2- 172.21.15.47:0/588855190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed14921e0 0x562ed14a3900 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4004960 tx=0x7fa6b4012600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c8a9459a9f94ef2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.089+0000 7fa6baa00640 1 -- 172.21.15.47:0/588855190 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40049e0 con 0x562ed14921e0 2024-03-21T11:28:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.089+0000 7fa6baa00640 1 -- 172.21.15.47:0/588855190 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6b402e070 con 0x562ed14921e0 2024-03-21T11:28:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.089+0000 7fa6baa00640 1 -- 172.21.15.47:0/588855190 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b400fc30 con 0x562ed14921e0 2024-03-21T11:28:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.089+0000 7fa6d4198740 1 -- 172.21.15.47:0/588855190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed14921e0 msgr2=0x562ed14a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.089+0000 7fa6d4198740 1 --2- 172.21.15.47:0/588855190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed14921e0 0x562ed14a3900 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4004960 tx=0x7fa6b4012600 comp rx=0 tx=0).stop 2024-03-21T11:28:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.090+0000 7fa6d4198740 1 -- 172.21.15.47:0/588855190 shutdown_connections 2024-03-21T11:28:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.090+0000 7fa6d4198740 1 --2- 172.21.15.47:0/588855190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed1492bd0 0x562ed1490ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.090+0000 7fa6d4198740 1 --2- 172.21.15.47:0/588855190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed14921e0 0x562ed14a3900 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.090+0000 7fa6d4198740 1 --2- 172.21.15.47:0/588855190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed148d750 0x562ed14909b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.090+0000 7fa6d4198740 1 -- 172.21.15.47:0/588855190 >> 172.21.15.47:0/588855190 conn(0x562ed1497880 msgr2=0x562ed149c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.090+0000 7fa6d4198740 1 -- 172.21.15.47:0/588855190 shutdown_connections 2024-03-21T11:28:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.090+0000 7fa6d4198740 1 -- 172.21.15.47:0/588855190 wait complete. 2024-03-21T11:28:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.090+0000 7fa6d4198740 1 -- start start 2024-03-21T11:28:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.090+0000 7fa6d4198740 1 Processor -- start 2024-03-21T11:28:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.091+0000 7fa6d4198740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed148d750 0x562ed14a1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.091+0000 7fa6d4198740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1492bd0 0x562ed14a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.091+0000 7fa6d4198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed1497d40 0x562ed14a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.091+0000 7fa6d4198740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc1ad590 con 0x562ed1492bd0 2024-03-21T11:28:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.091+0000 7fa6d4198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc1acf30 con 0x562ed1497d40 2024-03-21T11:28:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.091+0000 7fa6d4198740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc1aa430 con 0x562ed148d750 2024-03-21T11:28:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.091+0000 7fa6d4198740 10 client.0 _ll_register_callbacks cb 0x7fa6b4009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.091+0000 7fa6cb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed148d750 0x562ed14a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.091+0000 7fa6cb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed148d750 0x562ed14a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41466/0 (socket says 172.21.15.47:41466) 2024-03-21T11:28:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.091+0000 7fa6cb200640 1 -- 172.21.15.47:0/1644370344 learned_addr learned my addr 172.21.15.47:0/1644370344 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.091+0000 7fa6cb200640 1 --2- 172.21.15.47:0/1644370344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed1497d40 0x562ed14a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.091+0000 7fa6c9e00640 1 --2- 172.21.15.47:0/1644370344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1492bd0 0x562ed14a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.092+0000 7fa6cb200640 1 -- 172.21.15.47:0/1644370344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed1497d40 msgr2=0x562ed14a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.092+0000 7fa6cb200640 1 --2- 172.21.15.47:0/1644370344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed1497d40 0x562ed14a2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.092+0000 7fa6cb200640 1 -- 172.21.15.47:0/1644370344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1492bd0 msgr2=0x562ed14a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.092+0000 7fa6cb200640 1 --2- 172.21.15.47:0/1644370344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1492bd0 0x562ed14a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.092+0000 7fa6cb200640 1 -- 172.21.15.47:0/1644370344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ed14a33b0 con 0x562ed148d750 2024-03-21T11:28:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.092+0000 7fa6cb200640 1 --2- 172.21.15.47:0/1644370344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed148d750 0x562ed14a1870 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4011d10 tx=0x7fa6c4009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0b4b6e28dd5876b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.092+0000 7fa6b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.092+0000 7fa6b3e00640 1 -- 172.21.15.47:0/1644370344 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4032040 con 0x562ed148d750 2024-03-21T11:28:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.092+0000 7fa6d4198740 1 -- 172.21.15.47:0/1644370344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6b402e070 con 0x562ed148d750 2024-03-21T11:28:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.092+0000 7fa6b3e00640 1 -- 172.21.15.47:0/1644370344 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa6c40037e0 con 0x562ed148d750 2024-03-21T11:28:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.092+0000 7fa6b3e00640 1 -- 172.21.15.47:0/1644370344 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6c40039e0 con 0x562ed148d750 2024-03-21T11:28:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.092+0000 7fa6d4198740 10 client.27103 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.092+0000 7fa6d4198740 1 -- 172.21.15.47:0/1644370344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562ed1504350 con 0x562ed148d750 2024-03-21T11:28:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.093+0000 7fa6b3e00640 1 -- 172.21.15.47:0/1644370344 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6b402e070 con 0x562ed148d750 2024-03-21T11:28:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.093+0000 7fa6b3e00640 1 -- 172.21.15.47:0/1644370344 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa6c4025040 con 0x562ed148d750 2024-03-21T11:28:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.093+0000 7fa6d4198740 10 client.27103 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.093+0000 7fa6d4198740 10 client.27103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.093+0000 7fa6d4198740 20 client.27103 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.093+0000 7fa6d4198740 1 --2- 172.21.15.47:0/1644370344 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ed1504ba0 0x562ed1506fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.093+0000 7fa6d4198740 4 client.27103 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:28:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.093+0000 7fa6d4198740 1 -- 172.21.15.47:0/1644370344 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ed137d5d0 con 0x562ed1504ba0 2024-03-21T11:28:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.093+0000 7fa6c9e00640 1 --2- 172.21.15.47:0/1644370344 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ed1504ba0 0x562ed1506fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.094+0000 7fa6b3e00640 10 client.27103 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.094+0000 7fa6c9e00640 1 --2- 172.21.15.47:0/1644370344 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ed1504ba0 0x562ed1506fc0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7471a03b4dd88363 server_cookie=cbb4835241644d24 in_seq=0 out_seq=0 2024-03-21T11:28:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:02 smithi102 ceph-mon[36748]: pgmap v541: 129 pgs: 129 active+clean; 38 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 338 op/s 2024-03-21T11:28:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.295+0000 7fa6b3e00640 1 -- 172.21.15.47:0/1644370344 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ed137d5d0 con 0x562ed1504ba0 2024-03-21T11:28:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.295+0000 7fa6b3e00640 10 client.27103 handle_command_reply: tid=0 2024-03-21T11:28:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.295+0000 7fa6d4198740 1 client.27103 shutdown 2024-03-21T11:28:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.295+0000 7fa6d4198740 1 -- 172.21.15.47:0/1644370344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed148d750 msgr2=0x562ed14a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.295+0000 7fa6d4198740 1 --2- 172.21.15.47:0/1644370344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed148d750 0x562ed14a1870 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4011d10 tx=0x7fa6c4009760 comp rx=0 tx=0).stop 2024-03-21T11:28:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.295+0000 7fa6d4198740 1 -- 172.21.15.47:0/1644370344 shutdown_connections 2024-03-21T11:28:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.295+0000 7fa6d4198740 1 --2- 172.21.15.47:0/1644370344 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ed1504ba0 0x562ed1506fc0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.295+0000 7fa6d4198740 1 --2- 172.21.15.47:0/1644370344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ed1497d40 0x562ed14a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.295+0000 7fa6d4198740 1 --2- 172.21.15.47:0/1644370344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ed1492bd0 0x562ed14a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.295+0000 7fa6d4198740 1 --2- 172.21.15.47:0/1644370344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ed148d750 0x562ed14a1870 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.295+0000 7fa6b3e00640 0 client.27103 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.295+0000 7fa6d4198740 1 -- 172.21.15.47:0/1644370344 >> 172.21.15.47:0/1644370344 conn(0x562ed1497880 msgr2=0x562ed149c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.296+0000 7fa6d4198740 1 -- 172.21.15.47:0/1644370344 shutdown_connections 2024-03-21T11:28:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.296+0000 7fa6d4198740 1 -- 172.21.15.47:0/1644370344 wait complete. 2024-03-21T11:28:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.296+0000 7fa6d4198740 20 client.27103 trim_cache size 0 max 16384 2024-03-21T11:28:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:28:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:28:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:28:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:28:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.297+0000 7fa6d1400640 1 -- 172.21.15.47:0/3513429043 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6ac0a3040 msgr2=0x7fa6ac0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.297+0000 7fa6d1400640 1 --2- 172.21.15.47:0/3513429043 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6ac0a3040 0x7fa6ac0a5500 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0ba490 tx=0x7fa6bc0f7530 comp rx=0 tx=0).stop 2024-03-21T11:28:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.297+0000 7fa6d1400640 1 -- 172.21.15.47:0/3513429043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6cc103900 msgr2=0x7fa6cc199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.297+0000 7fa6d1400640 1 --2- 172.21.15.47:0/3513429043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6cc103900 0x7fa6cc199ba0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc000c00 tx=0x7fa6bc02c650 comp rx=0 tx=0).stop 2024-03-21T11:28:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.297+0000 7fa6d1400640 1 -- 172.21.15.47:0/3513429043 shutdown_connections 2024-03-21T11:28:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.297+0000 7fa6d1400640 1 --2- 172.21.15.47:0/3513429043 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6ac0a3040 0x7fa6ac0a5500 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.297+0000 7fa6d1400640 1 --2- 172.21.15.47:0/3513429043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6cc1054b0 0x7fa6cc1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.297+0000 7fa6d1400640 1 --2- 172.21.15.47:0/3513429043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6cc104af0 0x7fa6cc19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.297+0000 7fa6d1400640 1 --2- 172.21.15.47:0/3513429043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6cc103900 0x7fa6cc199ba0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.297+0000 7fa6d1400640 1 -- 172.21.15.47:0/3513429043 >> 172.21.15.47:0/3513429043 conn(0x7fa6cc0fee90 msgr2=0x7fa6cc107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.297+0000 7fa6d1400640 1 -- 172.21.15.47:0/3513429043 shutdown_connections 2024-03-21T11:28:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:03.297+0000 7fa6d1400640 1 -- 172.21.15.47:0/3513429043 wait complete. 2024-03-21T11:28:03.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:02 smithi047 ceph-mon[31351]: pgmap v541: 129 pgs: 129 active+clean; 38 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 338 op/s 2024-03-21T11:28:03.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:02 smithi100 ceph-mon[36245]: pgmap v541: 129 pgs: 129 active+clean; 38 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 338 op/s 2024-03-21T11:28:03.507 INFO:tasks.workunit.client.0.smithi047.stdout: 1 267450 8.28 MB/sec execute 47 sec latency 116.037 ms 2024-03-21T11:28:04.507 INFO:tasks.workunit.client.0.smithi047.stdout: 1 269677 8.37 MB/sec execute 48 sec latency 10.582 ms 2024-03-21T11:28:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:04 smithi102 ceph-mon[36748]: pgmap v542: 129 pgs: 129 active+clean; 37 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 329 op/s 2024-03-21T11:28:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:04 smithi047 ceph-mon[31351]: pgmap v542: 129 pgs: 129 active+clean; 37 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 329 op/s 2024-03-21T11:28:05.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:04 smithi100 ceph-mon[36245]: pgmap v542: 129 pgs: 129 active+clean; 37 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 329 op/s 2024-03-21T11:28:05.508 INFO:tasks.workunit.client.0.smithi047.stdout: 1 270738 8.27 MB/sec execute 49 sec latency 15.493 ms 2024-03-21T11:28:06.508 INFO:tasks.workunit.client.0.smithi047.stdout: 1 272916 8.35 MB/sec execute 50 sec latency 31.452 ms 2024-03-21T11:28:07.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:06 smithi102 ceph-mon[36748]: pgmap v543: 129 pgs: 129 active+clean; 37 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 189 op/s 2024-03-21T11:28:07.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:06 smithi047 ceph-mon[31351]: pgmap v543: 129 pgs: 129 active+clean; 37 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 189 op/s 2024-03-21T11:28:07.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:06 smithi100 ceph-mon[36245]: pgmap v543: 129 pgs: 129 active+clean; 37 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 189 op/s 2024-03-21T11:28:07.508 INFO:tasks.workunit.client.0.smithi047.stdout: 1 274064 8.30 MB/sec execute 51 sec latency 25.573 ms 2024-03-21T11:28:08.508 INFO:tasks.workunit.client.0.smithi047.stdout: 1 276418 8.40 MB/sec execute 52 sec latency 10.463 ms 2024-03-21T11:28:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:08 smithi102 ceph-mon[36748]: pgmap v544: 129 pgs: 129 active+clean; 39 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-03-21T11:28:09.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:08 smithi047 ceph-mon[31351]: pgmap v544: 129 pgs: 129 active+clean; 39 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-03-21T11:28:09.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:08 smithi100 ceph-mon[36245]: pgmap v544: 129 pgs: 129 active+clean; 39 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-03-21T11:28:09.510 INFO:tasks.workunit.client.0.smithi047.stdout: 1 277516 8.34 MB/sec execute 53 sec latency 19.270 ms 2024-03-21T11:28:10.508 INFO:tasks.workunit.client.0.smithi047.stdout: 1 279703 8.41 MB/sec execute 54 sec latency 12.226 ms 2024-03-21T11:28:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:10 smithi102 ceph-mon[36748]: pgmap v545: 129 pgs: 129 active+clean; 37 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 331 op/s 2024-03-21T11:28:11.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:10 smithi047 ceph-mon[31351]: pgmap v545: 129 pgs: 129 active+clean; 37 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 331 op/s 2024-03-21T11:28:11.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:10 smithi100 ceph-mon[36245]: pgmap v545: 129 pgs: 129 active+clean; 37 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 331 op/s 2024-03-21T11:28:11.508 INFO:tasks.workunit.client.0.smithi047.stdout: 1 280980 8.38 MB/sec execute 55 sec latency 62.127 ms 2024-03-21T11:28:12.508 INFO:tasks.workunit.client.0.smithi047.stdout: 1 282849 8.37 MB/sec execute 56 sec latency 28.826 ms 2024-03-21T11:28:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:12 smithi102 ceph-mon[36748]: pgmap v546: 129 pgs: 129 active+clean; 37 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 293 op/s 2024-03-21T11:28:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:12 smithi047 ceph-mon[31351]: pgmap v546: 129 pgs: 129 active+clean; 37 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 293 op/s 2024-03-21T11:28:13.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:12 smithi100 ceph-mon[36245]: pgmap v546: 129 pgs: 129 active+clean; 37 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 293 op/s 2024-03-21T11:28:13.508 INFO:tasks.workunit.client.0.smithi047.stdout: 1 284337 8.42 MB/sec execute 57 sec latency 10.740 ms 2024-03-21T11:28:14.509 INFO:tasks.workunit.client.0.smithi047.stdout: 1 286132 8.39 MB/sec execute 58 sec latency 23.054 ms 2024-03-21T11:28:15.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:14 smithi102 ceph-mon[36748]: pgmap v547: 129 pgs: 129 active+clean; 39 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 315 op/s 2024-03-21T11:28:15.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:14 smithi047 ceph-mon[31351]: pgmap v547: 129 pgs: 129 active+clean; 39 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 315 op/s 2024-03-21T11:28:15.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:14 smithi100 ceph-mon[36245]: pgmap v547: 129 pgs: 129 active+clean; 39 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 315 op/s 2024-03-21T11:28:15.509 INFO:tasks.workunit.client.0.smithi047.stdout: 1 287674 8.45 MB/sec execute 59 sec latency 11.700 ms 2024-03-21T11:28:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.438+0000 7f7289800640 1 -- 172.21.15.47:0/1847889582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72841097b0 msgr2=0x7f728410bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.438+0000 7f7289800640 1 --2- 172.21.15.47:0/1847889582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72841097b0 0x7f728410bba0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7274009900 tx=0x7f727402bc30 comp rx=0 tx=0).stop 2024-03-21T11:28:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.438+0000 7f7289800640 1 -- 172.21.15.47:0/1847889582 shutdown_connections 2024-03-21T11:28:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.438+0000 7f7289800640 1 --2- 172.21.15.47:0/1847889582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f728410ea60 0x7f7284110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.438+0000 7f7289800640 1 --2- 172.21.15.47:0/1847889582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f728410c0e0 0x7f728410e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.438+0000 7f7289800640 1 --2- 172.21.15.47:0/1847889582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72841097b0 0x7f728410bba0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.438+0000 7f7289800640 1 -- 172.21.15.47:0/1847889582 >> 172.21.15.47:0/1847889582 conn(0x7f728406e8a0 msgr2=0x7f7284070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.439+0000 7f7289800640 1 -- 172.21.15.47:0/1847889582 shutdown_connections 2024-03-21T11:28:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.439+0000 7f7289800640 1 -- 172.21.15.47:0/1847889582 wait complete. 2024-03-21T11:28:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.441+0000 7f7289800640 1 Processor -- start 2024-03-21T11:28:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.441+0000 7f7289800640 1 -- start start 2024-03-21T11:28:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.441+0000 7f7289800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72841097b0 0x7f72841a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.442+0000 7f7289800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f728410c0e0 0x7f72841a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.442+0000 7f7289800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f728410ea60 0x7f72841a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.442+0000 7f7289800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72841139a0 con 0x7f72841097b0 2024-03-21T11:28:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.442+0000 7f7289800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7284113820 con 0x7f728410ea60 2024-03-21T11:28:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.442+0000 7f7289800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7284113b20 con 0x7f728410c0e0 2024-03-21T11:28:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.442+0000 7f7283400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f728410ea60 0x7f72841a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.442+0000 7f7283400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f728410ea60 0x7f72841a9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49660/0 (socket says 172.21.15.47:49660) 2024-03-21T11:28:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.442+0000 7f7283400640 1 -- 172.21.15.47:0/1548634874 learned_addr learned my addr 172.21.15.47:0/1548634874 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.443+0000 7f727be00640 1 --2- 172.21.15.47:0/1548634874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f728410c0e0 0x7f72841a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.443+0000 7f7283400640 1 -- 172.21.15.47:0/1548634874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f728410c0e0 msgr2=0x7f72841a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.443+0000 7f7283400640 1 --2- 172.21.15.47:0/1548634874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f728410c0e0 0x7f72841a28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.443+0000 7f7283400640 1 -- 172.21.15.47:0/1548634874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72841097b0 msgr2=0x7f72841a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.443+0000 7f7283400640 1 --2- 172.21.15.47:0/1548634874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72841097b0 0x7f72841a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.443+0000 7f7283400640 1 -- 172.21.15.47:0/1548634874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72841a9f80 con 0x7f728410ea60 2024-03-21T11:28:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.444+0000 7f7283400640 1 --2- 172.21.15.47:0/1548634874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f728410ea60 0x7f72841a9910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f727c00eb10 tx=0x7f727c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.445+0000 7f727b400640 1 -- 172.21.15.47:0/1548634874 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c004b20 con 0x7f728410ea60 2024-03-21T11:28:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.445+0000 7f727b400640 1 -- 172.21.15.47:0/1548634874 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f727c004cc0 con 0x7f728410ea60 2024-03-21T11:28:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.445+0000 7f727b400640 1 -- 172.21.15.47:0/1548634874 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c010760 con 0x7f728410ea60 2024-03-21T11:28:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.445+0000 7f7289800640 1 -- 172.21.15.47:0/1548634874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72841aa270 con 0x7f728410ea60 2024-03-21T11:28:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.446+0000 7f7289800640 1 -- 172.21.15.47:0/1548634874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7284199040 con 0x7f728410ea60 2024-03-21T11:28:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.453+0000 7f728c469740 1 Processor -- start 2024-03-21T11:28:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.454+0000 7f727b400640 1 -- 172.21.15.47:0/1548634874 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f727c010900 con 0x7f728410ea60 2024-03-21T11:28:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.456+0000 7f728c469740 1 -- start start 2024-03-21T11:28:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.456+0000 7f728c469740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872c1e0 0x55749872c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.456+0000 7f728c469740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55749872cbd0 0x5574987316d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.457+0000 7f727be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872c1e0 0x55749872c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.457+0000 7f727be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872c1e0 0x55749872c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37312/0 (socket says 172.21.15.47:37312) 2024-03-21T11:28:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.457+0000 7f728c469740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557498731d40 0x5574987341d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.457+0000 7f728c469740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574986ef3b0 con 0x557498731d40 2024-03-21T11:28:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.457+0000 7f728c469740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557498691830 con 0x55749872cbd0 2024-03-21T11:28:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.457+0000 7f728c469740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55749863a5d0 con 0x55749872c1e0 2024-03-21T11:28:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.457+0000 7f727be00640 1 -- 172.21.15.47:0/889384597 learned_addr learned my addr 172.21.15.47:0/889384597 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:16.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.458+0000 7f727be00640 1 --2- 172.21.15.47:0/889384597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557498731d40 0x5574987341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.459+0000 7f727be00640 1 -- 172.21.15.47:0/889384597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55749872cbd0 msgr2=0x5574987316d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.459+0000 7f727be00640 1 --2- 172.21.15.47:0/889384597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55749872cbd0 0x5574987316d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.459+0000 7f727be00640 1 -- 172.21.15.47:0/889384597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557498731d40 msgr2=0x5574987341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.459+0000 7f727be00640 1 --2- 172.21.15.47:0/889384597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557498731d40 0x5574987341d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.459+0000 7f727be00640 1 -- 172.21.15.47:0/889384597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574986522c0 con 0x55749872c1e0 2024-03-21T11:28:16.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.459+0000 7f727be00640 1 --2- 172.21.15.47:0/889384597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872c1e0 0x55749872c600 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f726c004650 tx=0x7f726c0096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24a3e2c7746c4d0d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.462+0000 7f7273e00640 1 -- 172.21.15.47:0/889384597 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c0130d0 con 0x55749872c1e0 2024-03-21T11:28:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.462+0000 7f7273e00640 1 -- 172.21.15.47:0/889384597 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f726c013cb0 con 0x55749872c1e0 2024-03-21T11:28:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.462+0000 7f7273e00640 1 -- 172.21.15.47:0/889384597 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c0116b0 con 0x55749872c1e0 2024-03-21T11:28:16.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.462+0000 7f727b400640 1 --2- 172.21.15.47:0/1548634874 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72680a3190 0x7f72680a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.462+0000 7f727b400640 1 -- 172.21.15.47:0/1548634874 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f727c0d0240 con 0x7f728410ea60 2024-03-21T11:28:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.465+0000 7f728c469740 1 -- 172.21.15.47:0/889384597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872c1e0 msgr2=0x55749872c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.465+0000 7f728c469740 1 --2- 172.21.15.47:0/889384597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872c1e0 0x55749872c600 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f726c004650 tx=0x7f726c0096a0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.466+0000 7f7282a00640 1 --2- 172.21.15.47:0/1548634874 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72680a3190 0x7f72680a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.467+0000 7f728c469740 1 -- 172.21.15.47:0/889384597 shutdown_connections 2024-03-21T11:28:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.467+0000 7f728c469740 1 --2- 172.21.15.47:0/889384597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557498731d40 0x5574987341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.467+0000 7f728c469740 1 --2- 172.21.15.47:0/889384597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55749872cbd0 0x5574987316d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.467+0000 7f728c469740 1 --2- 172.21.15.47:0/889384597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872c1e0 0x55749872c600 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.467+0000 7f728c469740 1 -- 172.21.15.47:0/889384597 >> 172.21.15.47:0/889384597 conn(0x557498727750 msgr2=0x557498727b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.469+0000 7f728c469740 1 -- 172.21.15.47:0/889384597 shutdown_connections 2024-03-21T11:28:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.470+0000 7f7282a00640 1 --2- 172.21.15.47:0/1548634874 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72680a3190 0x7f72680a5650 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f727402c7b0 tx=0x7f727402c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:16.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.471+0000 7f728c469740 1 -- 172.21.15.47:0/889384597 wait complete. 2024-03-21T11:28:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.473+0000 7f728c469740 1 -- start start 2024-03-21T11:28:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.473+0000 7f728c469740 1 Processor -- start 2024-03-21T11:28:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.474+0000 7f728c469740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55749872c1e0 0x5574987363f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.474+0000 7f728c469740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872cbd0 0x557498736930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.474+0000 7f728c469740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557498731d40 0x55749873d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.474+0000 7f728c469740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55749864da20 con 0x55749872c1e0 2024-03-21T11:28:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.474+0000 7f728c469740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55749868d8d0 con 0x557498731d40 2024-03-21T11:28:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.474+0000 7f728c469740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557498734c90 con 0x55749872cbd0 2024-03-21T11:28:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.475+0000 7f728c469740 10 client.0 _ll_register_callbacks cb 0x55749868a1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.475+0000 7f727be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872cbd0 0x557498736930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.475+0000 7f7283400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557498731d40 0x55749873d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.475+0000 7f7283400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557498731d40 0x55749873d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49676/0 (socket says 172.21.15.47:49676) 2024-03-21T11:28:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.475+0000 7f7283400640 1 -- 172.21.15.47:0/940281834 learned_addr learned my addr 172.21.15.47:0/940281834 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.476+0000 7f7283400640 1 -- 172.21.15.47:0/940281834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872cbd0 msgr2=0x557498736930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.476+0000 7f7283400640 1 --2- 172.21.15.47:0/940281834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872cbd0 0x557498736930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.476+0000 7f7283400640 1 -- 172.21.15.47:0/940281834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55749872c1e0 msgr2=0x5574987363f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.476+0000 7f7283400640 1 --2- 172.21.15.47:0/940281834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55749872c1e0 0x5574987363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.476+0000 7f7283400640 1 -- 172.21.15.47:0/940281834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574983a6210 con 0x557498731d40 2024-03-21T11:28:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.476+0000 7f727be00640 1 --2- 172.21.15.47:0/940281834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872cbd0 0x557498736930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.478+0000 7f7283400640 1 --2- 172.21.15.47:0/940281834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557498731d40 0x55749873d970 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f727c099b70 tx=0x7f727c09cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e17bdf9b915f3ea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.478+0000 7f7270c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.479+0000 7f7270c00640 1 -- 172.21.15.47:0/940281834 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c0b6060 con 0x557498731d40 2024-03-21T11:28:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.479+0000 7f7270c00640 1 -- 172.21.15.47:0/940281834 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f727c0a5720 con 0x557498731d40 2024-03-21T11:28:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.480+0000 7f728c469740 1 -- 172.21.15.47:0/940281834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f726c013cb0 con 0x557498731d40 2024-03-21T11:28:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.480+0000 7f7270c00640 1 -- 172.21.15.47:0/940281834 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f727c0bb050 con 0x557498731d40 2024-03-21T11:28:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.481+0000 7f728c469740 10 client.27095 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.481+0000 7f728c469740 1 -- 172.21.15.47:0/940281834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557498472620 con 0x557498731d40 2024-03-21T11:28:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.482+0000 7f7270c00640 1 -- 172.21.15.47:0/940281834 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f727c09d1a0 con 0x557498731d40 2024-03-21T11:28:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.482+0000 7f7270c00640 1 -- 172.21.15.47:0/940281834 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f727c0bd0a0 con 0x557498731d40 2024-03-21T11:28:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.482+0000 7f728c469740 10 client.27095 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.482+0000 7f728c469740 10 client.27095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.483+0000 7f728c469740 20 client.27095 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.483+0000 7f728c469740 1 --2- 172.21.15.47:0/940281834 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557498742290 0x5574987446a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.483+0000 7f728c469740 4 client.27095 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.483+0000 7f728c469740 1 -- 172.21.15.47:0/940281834 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f726c013cb0 con 0x557498742290 2024-03-21T11:28:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.484+0000 7f7282a00640 1 --2- 172.21.15.47:0/940281834 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557498742290 0x5574987446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.485+0000 7f7282a00640 1 --2- 172.21.15.47:0/940281834 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557498742290 0x5574987446a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8e69c129489f369 server_cookie=115e7a685df32f2d in_seq=0 out_seq=0 2024-03-21T11:28:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.486+0000 7f7270c00640 10 client.27095 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.491+0000 7f7270c00640 1 -- 172.21.15.47:0/940281834 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f726c013cb0 con 0x557498742290 2024-03-21T11:28:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.491+0000 7f7270c00640 10 client.27095 handle_command_reply: tid=0 2024-03-21T11:28:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.491+0000 7f728c469740 1 client.27095 shutdown 2024-03-21T11:28:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.493+0000 7f728c469740 1 -- 172.21.15.47:0/940281834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557498731d40 msgr2=0x55749873d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.493+0000 7f728c469740 1 --2- 172.21.15.47:0/940281834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557498731d40 0x55749873d970 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f727c099b70 tx=0x7f727c09cbc0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.493+0000 7f728c469740 1 -- 172.21.15.47:0/940281834 shutdown_connections 2024-03-21T11:28:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.493+0000 7f728c469740 1 --2- 172.21.15.47:0/940281834 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557498742290 0x5574987446a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.493+0000 7f728c469740 1 --2- 172.21.15.47:0/940281834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557498731d40 0x55749873d970 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.493+0000 7f728c469740 1 --2- 172.21.15.47:0/940281834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872cbd0 0x557498736930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.493+0000 7f728c469740 1 --2- 172.21.15.47:0/940281834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55749872c1e0 0x5574987363f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.493+0000 7f7270c00640 0 client.27095 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:16.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.493+0000 7f728c469740 1 -- 172.21.15.47:0/940281834 >> 172.21.15.47:0/940281834 conn(0x557498727750 msgr2=0x557498726950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:16.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.494+0000 7f728c469740 1 -- 172.21.15.47:0/940281834 shutdown_connections 2024-03-21T11:28:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.495+0000 7f728c469740 1 -- 172.21.15.47:0/940281834 wait complete. 2024-03-21T11:28:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.495+0000 7f728c469740 20 client.27095 trim_cache size 0 max 16384 2024-03-21T11:28:16.510 INFO:tasks.workunit.client.0.smithi047.stdout: 1 288784 8.40 MB/sec execute 60 sec latency 42.789 ms 2024-03-21T11:28:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.522+0000 7f728c469740 1 Processor -- start 2024-03-21T11:28:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.522+0000 7f728c469740 1 -- start start 2024-03-21T11:28:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.522+0000 7f728c469740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55749872c1e0 0x55749872def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.522+0000 7f728c469740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872cbd0 0x55749873d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.522+0000 7f728c469740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557498731d40 0x55749872e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.522+0000 7f728c469740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726000f940 con 0x557498731d40 2024-03-21T11:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.522+0000 7f728c469740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72600112c0 con 0x55749872c1e0 2024-03-21T11:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.522+0000 7f728c469740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726000ffa0 con 0x55749872cbd0 2024-03-21T11:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.523+0000 7f727be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872cbd0 0x55749873d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.523+0000 7f727be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872cbd0 0x55749873d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37316/0 (socket says 172.21.15.47:37316) 2024-03-21T11:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.523+0000 7f727be00640 1 -- 172.21.15.47:0/2959450280 learned_addr learned my addr 172.21.15.47:0/2959450280 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.523+0000 7f7282a00640 1 --2- 172.21.15.47:0/2959450280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55749872c1e0 0x55749872def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.523+0000 7f7283400640 1 --2- 172.21.15.47:0/2959450280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557498731d40 0x55749872e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.523+0000 7f727be00640 1 -- 172.21.15.47:0/2959450280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55749872c1e0 msgr2=0x55749872def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.523+0000 7f727be00640 1 --2- 172.21.15.47:0/2959450280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55749872c1e0 0x55749872def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.523+0000 7f727be00640 1 -- 172.21.15.47:0/2959450280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557498731d40 msgr2=0x55749872e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.523+0000 7f727be00640 1 --2- 172.21.15.47:0/2959450280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557498731d40 0x55749872e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.523+0000 7f727be00640 1 -- 172.21.15.47:0/2959450280 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55749872d9e0 con 0x55749872cbd0 2024-03-21T11:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.523+0000 7f7283400640 1 --2- 172.21.15.47:0/2959450280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557498731d40 0x55749872e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.524+0000 7f727be00640 1 --2- 172.21.15.47:0/2959450280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872cbd0 0x55749873d930 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f726c013730 tx=0x7f726c009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35e0b98d91e9c5f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.524+0000 7f7273400640 1 -- 172.21.15.47:0/2959450280 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c013e60 con 0x55749872cbd0 2024-03-21T11:28:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.524+0000 7f7273400640 1 -- 172.21.15.47:0/2959450280 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f726c013cb0 con 0x55749872cbd0 2024-03-21T11:28:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.524+0000 7f7273400640 1 -- 172.21.15.47:0/2959450280 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c0116d0 con 0x55749872cbd0 2024-03-21T11:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.525+0000 7f728c469740 1 -- 172.21.15.47:0/2959450280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872cbd0 msgr2=0x55749873d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.525+0000 7f728c469740 1 --2- 172.21.15.47:0/2959450280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872cbd0 0x55749873d930 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f726c013730 tx=0x7f726c009550 comp rx=0 tx=0).stop 2024-03-21T11:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.525+0000 7f728c469740 1 -- 172.21.15.47:0/2959450280 shutdown_connections 2024-03-21T11:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.525+0000 7f728c469740 1 --2- 172.21.15.47:0/2959450280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557498731d40 0x55749872e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.525+0000 7f728c469740 1 --2- 172.21.15.47:0/2959450280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872cbd0 0x55749873d930 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.525+0000 7f728c469740 1 --2- 172.21.15.47:0/2959450280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55749872c1e0 0x55749872def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.525+0000 7f728c469740 1 -- 172.21.15.47:0/2959450280 >> 172.21.15.47:0/2959450280 conn(0x557498727750 msgr2=0x5574987363f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.525+0000 7f728c469740 1 -- 172.21.15.47:0/2959450280 shutdown_connections 2024-03-21T11:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.525+0000 7f728c469740 1 -- 172.21.15.47:0/2959450280 wait complete. 2024-03-21T11:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.526+0000 7f728c469740 1 -- start start 2024-03-21T11:28:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.526+0000 7f728c469740 1 Processor -- start 2024-03-21T11:28:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.526+0000 7f728c469740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872c1e0 0x55749873aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.526+0000 7f728c469740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55749872cbd0 0x55749873b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.526+0000 7f728c469740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557498731d40 0x55749873c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.526+0000 7f728c469740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7260010c60 con 0x55749872cbd0 2024-03-21T11:28:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f728c469740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7260010600 con 0x557498731d40 2024-03-21T11:28:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f728c469740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726000db00 con 0x55749872c1e0 2024-03-21T11:28:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f728c469740 10 client.0 _ll_register_callbacks cb 0x7f727c09cfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f7282a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872c1e0 0x55749873aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f7282a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872c1e0 0x55749873aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37322/0 (socket says 172.21.15.47:37322) 2024-03-21T11:28:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f7282a00640 1 -- 172.21.15.47:0/156262363 learned_addr learned my addr 172.21.15.47:0/156262363 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f7283400640 1 --2- 172.21.15.47:0/156262363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557498731d40 0x55749873c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f7282a00640 1 -- 172.21.15.47:0/156262363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557498731d40 msgr2=0x55749873c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f7282a00640 1 --2- 172.21.15.47:0/156262363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557498731d40 0x55749873c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f7282a00640 1 -- 172.21.15.47:0/156262363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55749872cbd0 msgr2=0x55749873b3f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:28:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f727be00640 1 --2- 172.21.15.47:0/156262363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55749872cbd0 0x55749873b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f7282a00640 1 --2- 172.21.15.47:0/156262363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55749872cbd0 0x55749873b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f7282a00640 1 -- 172.21.15.47:0/156262363 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55749873c9f0 con 0x55749872c1e0 2024-03-21T11:28:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f7282a00640 1 --2- 172.21.15.47:0/156262363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872c1e0 0x55749873aeb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7274009850 tx=0x7f7274005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec2b60d17718f572 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.527+0000 7f7270c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.528+0000 7f7270c00640 1 -- 172.21.15.47:0/156262363 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274044040 con 0x55749872c1e0 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.528+0000 7f7270c00640 1 -- 172.21.15.47:0/156262363 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7274004240 con 0x55749872c1e0 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.528+0000 7f728c469740 1 -- 172.21.15.47:0/156262363 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f726c013cb0 con 0x55749872c1e0 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.528+0000 7f7270c00640 1 -- 172.21.15.47:0/156262363 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f727404e2e0 con 0x55749872c1e0 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.528+0000 7f728c469740 10 client.27121 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.528+0000 7f728c469740 1 -- 172.21.15.47:0/156262363 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55749879db70 con 0x55749872c1e0 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.528+0000 7f7270c00640 1 -- 172.21.15.47:0/156262363 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f726c013cb0 con 0x55749872c1e0 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.528+0000 7f7270c00640 1 -- 172.21.15.47:0/156262363 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f727404ec60 con 0x55749872c1e0 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.528+0000 7f728c469740 10 client.27121 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.528+0000 7f728c469740 10 client.27121 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.528+0000 7f728c469740 20 client.27121 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.528+0000 7f728c469740 1 --2- 172.21.15.47:0/156262363 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55749879e410 0x5574987a0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.528+0000 7f728c469740 4 client.27121 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.528+0000 7f728c469740 1 -- 172.21.15.47:0/156262363 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55749869ed70 con 0x55749879e410 2024-03-21T11:28:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.529+0000 7f727be00640 1 --2- 172.21.15.47:0/156262363 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55749879e410 0x5574987a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.529+0000 7f727be00640 1 --2- 172.21.15.47:0/156262363 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55749879e410 0x5574987a0850 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5bc62f94f1dc4c3 server_cookie=e183175044d66694 in_seq=0 out_seq=0 2024-03-21T11:28:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.529+0000 7f7270c00640 10 client.27121 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.774+0000 7f7270c00640 1 -- 172.21.15.47:0/156262363 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55749869ed70 con 0x55749879e410 2024-03-21T11:28:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.774+0000 7f7270c00640 10 client.27121 handle_command_reply: tid=0 2024-03-21T11:28:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.774+0000 7f728c469740 1 client.27121 shutdown 2024-03-21T11:28:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.774+0000 7f728c469740 1 -- 172.21.15.47:0/156262363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872c1e0 msgr2=0x55749873aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.774+0000 7f728c469740 1 --2- 172.21.15.47:0/156262363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872c1e0 0x55749873aeb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7274009850 tx=0x7f7274005fd0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.774+0000 7f728c469740 1 -- 172.21.15.47:0/156262363 shutdown_connections 2024-03-21T11:28:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.774+0000 7f728c469740 1 --2- 172.21.15.47:0/156262363 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55749879e410 0x5574987a0850 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.774+0000 7f728c469740 1 --2- 172.21.15.47:0/156262363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557498731d40 0x55749873c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.774+0000 7f728c469740 1 --2- 172.21.15.47:0/156262363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55749872cbd0 0x55749873b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.774+0000 7f728c469740 1 --2- 172.21.15.47:0/156262363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55749872c1e0 0x55749873aeb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.774+0000 7f7270c00640 0 client.27121 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.774+0000 7f728c469740 1 -- 172.21.15.47:0/156262363 >> 172.21.15.47:0/156262363 conn(0x557498727750 msgr2=0x5574987363f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.775+0000 7f728c469740 1 -- 172.21.15.47:0/156262363 shutdown_connections 2024-03-21T11:28:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.775+0000 7f728c469740 1 -- 172.21.15.47:0/156262363 wait complete. 2024-03-21T11:28:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.775+0000 7f728c469740 20 client.27121 trim_cache size 0 max 16384 2024-03-21T11:28:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:28:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:28:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:28:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:28:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.775+0000 7f7289800640 1 -- 172.21.15.47:0/1548634874 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72680a3190 msgr2=0x7f72680a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.775+0000 7f7289800640 1 --2- 172.21.15.47:0/1548634874 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72680a3190 0x7f72680a5650 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f727402c7b0 tx=0x7f727402c880 comp rx=0 tx=0).stop 2024-03-21T11:28:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.775+0000 7f7289800640 1 -- 172.21.15.47:0/1548634874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f728410ea60 msgr2=0x7f72841a9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.775+0000 7f7289800640 1 --2- 172.21.15.47:0/1548634874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f728410ea60 0x7f72841a9910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f727c00eb10 tx=0x7f727c00c2d0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.776+0000 7f7289800640 1 -- 172.21.15.47:0/1548634874 shutdown_connections 2024-03-21T11:28:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.776+0000 7f7289800640 1 --2- 172.21.15.47:0/1548634874 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72680a3190 0x7f72680a5650 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.776+0000 7f7289800640 1 --2- 172.21.15.47:0/1548634874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f728410ea60 0x7f72841a9910 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.776+0000 7f7289800640 1 --2- 172.21.15.47:0/1548634874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f728410c0e0 0x7f72841a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.776+0000 7f7289800640 1 --2- 172.21.15.47:0/1548634874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72841097b0 0x7f72841a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.776+0000 7f7289800640 1 -- 172.21.15.47:0/1548634874 >> 172.21.15.47:0/1548634874 conn(0x7f728406e8a0 msgr2=0x7f728410a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.776+0000 7f7289800640 1 -- 172.21.15.47:0/1548634874 shutdown_connections 2024-03-21T11:28:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:16.776+0000 7f7289800640 1 -- 172.21.15.47:0/1548634874 wait complete. 2024-03-21T11:28:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:16 smithi102 ceph-mon[36748]: pgmap v548: 129 pgs: 129 active+clean; 39 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2024-03-21T11:28:17.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:16 smithi047 ceph-mon[31351]: pgmap v548: 129 pgs: 129 active+clean; 39 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2024-03-21T11:28:17.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:16 smithi100 ceph-mon[36245]: pgmap v548: 129 pgs: 129 active+clean; 39 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 295 op/s 2024-03-21T11:28:17.510 INFO:tasks.workunit.client.0.smithi047.stdout: 1 290528 8.40 MB/sec execute 61 sec latency 124.098 ms 2024-03-21T11:28:18.510 INFO:tasks.workunit.client.0.smithi047.stdout: 1 291665 8.36 MB/sec execute 62 sec latency 13.486 ms 2024-03-21T11:28:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:18 smithi102 ceph-mon[36748]: pgmap v549: 129 pgs: 129 active+clean; 42 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 346 op/s 2024-03-21T11:28:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:18 smithi047 ceph-mon[31351]: pgmap v549: 129 pgs: 129 active+clean; 42 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 346 op/s 2024-03-21T11:28:19.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:18 smithi100 ceph-mon[36245]: pgmap v549: 129 pgs: 129 active+clean; 42 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 346 op/s 2024-03-21T11:28:19.510 INFO:tasks.workunit.client.0.smithi047.stdout: 1 293316 8.33 MB/sec execute 63 sec latency 20.367 ms 2024-03-21T11:28:20.510 INFO:tasks.workunit.client.0.smithi047.stdout: 1 294783 8.39 MB/sec execute 64 sec latency 12.910 ms 2024-03-21T11:28:21.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:28:21.041 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:28:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.291+0000 7f9f69800640 1 -- 172.21.15.47:0/2860714747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f640740c0 msgr2=0x7f9f64074540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.291+0000 7f9f69800640 1 --2- 172.21.15.47:0/2860714747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f640740c0 0x7f9f64074540 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009860 tx=0x7f9f5002bb70 comp rx=0 tx=0).stop 2024-03-21T11:28:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.292+0000 7f9f69800640 1 -- 172.21.15.47:0/2860714747 shutdown_connections 2024-03-21T11:28:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.292+0000 7f9f69800640 1 --2- 172.21.15.47:0/2860714747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f64109560 0x7f9f6410b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.292+0000 7f9f69800640 1 --2- 172.21.15.47:0/2860714747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f640740c0 0x7f9f64074540 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.292+0000 7f9f69800640 1 --2- 172.21.15.47:0/2860714747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f640756a0 0x7f9f64073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.292+0000 7f9f69800640 1 -- 172.21.15.47:0/2860714747 >> 172.21.15.47:0/2860714747 conn(0x7f9f640fcc40 msgr2=0x7f9f640ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.292+0000 7f9f69800640 1 -- 172.21.15.47:0/2860714747 shutdown_connections 2024-03-21T11:28:21.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.292+0000 7f9f69800640 1 -- 172.21.15.47:0/2860714747 wait complete. 2024-03-21T11:28:21.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.293+0000 7f9f69800640 1 Processor -- start 2024-03-21T11:28:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.293+0000 7f9f69800640 1 -- start start 2024-03-21T11:28:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.293+0000 7f9f69800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f640740c0 0x7f9f64199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.293+0000 7f9f69800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f640756a0 0x7f9f6419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.293+0000 7f9f69800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f64109560 0x7f9f641a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.293+0000 7f9f69800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6410f4f0 con 0x7f9f640740c0 2024-03-21T11:28:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.293+0000 7f9f69800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6410f370 con 0x7f9f640756a0 2024-03-21T11:28:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.293+0000 7f9f69800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6410f670 con 0x7f9f64109560 2024-03-21T11:28:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.294+0000 7f9f62000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f640756a0 0x7f9f6419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.294+0000 7f9f62a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f640740c0 0x7f9f64199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.294+0000 7f9f63400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f64109560 0x7f9f641a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.294+0000 7f9f62000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f640756a0 0x7f9f6419a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44512/0 (socket says 172.21.15.47:44512) 2024-03-21T11:28:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.294+0000 7f9f62000640 1 -- 172.21.15.47:0/1677051879 learned_addr learned my addr 172.21.15.47:0/1677051879 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.294+0000 7f9f62000640 1 -- 172.21.15.47:0/1677051879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f64109560 msgr2=0x7f9f641a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.294+0000 7f9f62000640 1 --2- 172.21.15.47:0/1677051879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f64109560 0x7f9f641a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.294+0000 7f9f62000640 1 -- 172.21.15.47:0/1677051879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f640740c0 msgr2=0x7f9f64199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.294+0000 7f9f62000640 1 --2- 172.21.15.47:0/1677051879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f640740c0 0x7f9f64199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.294+0000 7f9f62000640 1 -- 172.21.15.47:0/1677051879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f641a1800 con 0x7f9f640756a0 2024-03-21T11:28:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.294+0000 7f9f62000640 1 --2- 172.21.15.47:0/1677051879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f640756a0 0x7f9f6419a120 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9f500349e0 tx=0x7f9f50032450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.294+0000 7f9f5b400640 1 -- 172.21.15.47:0/1677051879 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50041030 con 0x7f9f640756a0 2024-03-21T11:28:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.295+0000 7f9f5b400640 1 -- 172.21.15.47:0/1677051879 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f5002c510 con 0x7f9f640756a0 2024-03-21T11:28:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.295+0000 7f9f5b400640 1 -- 172.21.15.47:0/1677051879 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50039040 con 0x7f9f640756a0 2024-03-21T11:28:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.295+0000 7f9f69800640 1 -- 172.21.15.47:0/1677051879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f641a1a90 con 0x7f9f640756a0 2024-03-21T11:28:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.295+0000 7f9f69800640 1 -- 172.21.15.47:0/1677051879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f641a1f30 con 0x7f9f640756a0 2024-03-21T11:28:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.298+0000 7f9f6c594740 1 Processor -- start 2024-03-21T11:28:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.298+0000 7f9f6c594740 1 -- start start 2024-03-21T11:28:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.298+0000 7f9f5b400640 1 -- 172.21.15.47:0/1677051879 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9f50044070 con 0x7f9f640756a0 2024-03-21T11:28:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f6c594740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc4e80 0x55776efc52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f6c594740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc5870 0x55776efca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f6c594740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 0x55776efcce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f6c594740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55776ef2a250 con 0x55776efc5870 2024-03-21T11:28:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f6c594740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55776eed2a30 con 0x55776efca9e0 2024-03-21T11:28:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f6c594740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55776ef262f0 con 0x55776efc4e80 2024-03-21T11:28:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f63400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc4e80 0x55776efc52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f63400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc4e80 0x55776efc52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36696/0 (socket says 172.21.15.47:36696) 2024-03-21T11:28:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f63400640 1 -- 172.21.15.47:0/4016776368 learned_addr learned my addr 172.21.15.47:0/4016776368 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f62a00640 1 --2- 172.21.15.47:0/4016776368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc5870 0x55776efca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f62000640 1 --2- 172.21.15.47:0/4016776368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 0x55776efcce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f63400640 1 -- 172.21.15.47:0/4016776368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 msgr2=0x55776efcce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f63400640 1 --2- 172.21.15.47:0/4016776368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 0x55776efcce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f63400640 1 -- 172.21.15.47:0/4016776368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc5870 msgr2=0x55776efca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f63400640 1 --2- 172.21.15.47:0/4016776368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc5870 0x55776efca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.299+0000 7f9f63400640 1 -- 172.21.15.47:0/4016776368 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55776eebd690 con 0x55776efc4e80 2024-03-21T11:28:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.300+0000 7f9f63400640 1 --2- 172.21.15.47:0/4016776368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc4e80 0x55776efc52a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00e580 tx=0x7f9f5c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78601f257bbaf50f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.300+0000 7f9f4fe00640 1 -- 172.21.15.47:0/4016776368 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c003970 con 0x55776efc4e80 2024-03-21T11:28:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.300+0000 7f9f4fe00640 1 -- 172.21.15.47:0/4016776368 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f5c00d830 con 0x55776efc4e80 2024-03-21T11:28:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.300+0000 7f9f4fe00640 1 -- 172.21.15.47:0/4016776368 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0045a0 con 0x55776efc4e80 2024-03-21T11:28:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.300+0000 7f9f5b400640 1 --2- 172.21.15.47:0/1677051879 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9f480a2fa0 0x7f9f480a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.301+0000 7f9f5b400640 1 -- 172.21.15.47:0/1677051879 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9f500f0c10 con 0x7f9f640756a0 2024-03-21T11:28:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.301+0000 7f9f62a00640 1 --2- 172.21.15.47:0/1677051879 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9f480a2fa0 0x7f9f480a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.301+0000 7f9f6c594740 1 -- 172.21.15.47:0/4016776368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc4e80 msgr2=0x55776efc52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.301+0000 7f9f6c594740 1 --2- 172.21.15.47:0/4016776368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc4e80 0x55776efc52a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00e580 tx=0x7f9f5c0095c0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.301+0000 7f9f6c594740 1 -- 172.21.15.47:0/4016776368 shutdown_connections 2024-03-21T11:28:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.301+0000 7f9f6c594740 1 --2- 172.21.15.47:0/4016776368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 0x55776efcce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.301+0000 7f9f6c594740 1 --2- 172.21.15.47:0/4016776368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc5870 0x55776efca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.301+0000 7f9f6c594740 1 --2- 172.21.15.47:0/4016776368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc4e80 0x55776efc52a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.301+0000 7f9f6c594740 1 -- 172.21.15.47:0/4016776368 >> 172.21.15.47:0/4016776368 conn(0x55776ef8d200 msgr2=0x55776efc07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.301+0000 7f9f6c594740 1 -- 172.21.15.47:0/4016776368 shutdown_connections 2024-03-21T11:28:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.301+0000 7f9f6c594740 1 -- 172.21.15.47:0/4016776368 wait complete. 2024-03-21T11:28:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.301+0000 7f9f62a00640 1 --2- 172.21.15.47:0/1677051879 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9f480a2fa0 0x7f9f480a5460 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55776efc5c70 tx=0x7f9f54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.302+0000 7f9f6c594740 1 -- start start 2024-03-21T11:28:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.302+0000 7f9f6c594740 1 Processor -- start 2024-03-21T11:28:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.302+0000 7f9f6c594740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 0x55776efceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f6c594740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc5870 0x55776efcf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f6c594740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 0x55776efd6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f6c594740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55776efcd990 con 0x55776efc4e80 2024-03-21T11:28:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f6c594740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55776eee5e80 con 0x55776efca9e0 2024-03-21T11:28:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f6c594740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55776efcdb10 con 0x55776efc5870 2024-03-21T11:28:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f6c594740 10 client.0 _ll_register_callbacks cb 0x55776eef8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f62000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc5870 0x55776efcf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f63400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 0x55776efd6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f62000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc5870 0x55776efcf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36710/0 (socket says 172.21.15.47:36710) 2024-03-21T11:28:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f62000640 1 -- 172.21.15.47:0/894162405 learned_addr learned my addr 172.21.15.47:0/894162405 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f62a00640 1 --2- 172.21.15.47:0/894162405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 0x55776efceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f62000640 1 -- 172.21.15.47:0/894162405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 msgr2=0x55776efd6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f62000640 1 --2- 172.21.15.47:0/894162405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 0x55776efd6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f62000640 1 -- 172.21.15.47:0/894162405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 msgr2=0x55776efceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f62000640 1 --2- 172.21.15.47:0/894162405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 0x55776efceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f62000640 1 -- 172.21.15.47:0/894162405 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55776ed087d0 con 0x55776efc5870 2024-03-21T11:28:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.303+0000 7f9f62a00640 1 --2- 172.21.15.47:0/894162405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 0x55776efceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.304+0000 7f9f62000640 1 --2- 172.21.15.47:0/894162405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc5870 0x55776efcf410 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9f500bcf40 tx=0x7f9f50009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e50eaf893a1595b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.304+0000 7f9f4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.304+0000 7f9f4cc00640 1 -- 172.21.15.47:0/894162405 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500bcaf0 con 0x55776efc5870 2024-03-21T11:28:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.304+0000 7f9f4cc00640 1 -- 172.21.15.47:0/894162405 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9f5002c6c0 con 0x55776efc5870 2024-03-21T11:28:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.304+0000 7f9f6c594740 1 -- 172.21.15.47:0/894162405 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f5c00d830 con 0x55776efc5870 2024-03-21T11:28:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.304+0000 7f9f4cc00640 1 -- 172.21.15.47:0/894162405 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f500bc910 con 0x55776efc5870 2024-03-21T11:28:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.304+0000 7f9f6c594740 10 client.27133 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.304+0000 7f9f6c594740 1 -- 172.21.15.47:0/894162405 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55776ec3e3f0 con 0x55776efc5870 2024-03-21T11:28:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.304+0000 7f9f4cc00640 1 -- 172.21.15.47:0/894162405 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f5c00d830 con 0x55776efc5870 2024-03-21T11:28:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.305+0000 7f9f4cc00640 1 -- 172.21.15.47:0/894162405 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9f5004e060 con 0x55776efc5870 2024-03-21T11:28:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.305+0000 7f9f6c594740 10 client.27133 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.305+0000 7f9f6c594740 10 client.27133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.305+0000 7f9f6c594740 20 client.27133 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.305+0000 7f9f6c594740 1 --2- 172.21.15.47:0/894162405 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55776efdac10 0x55776efdd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.305+0000 7f9f6c594740 4 client.27133 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.305+0000 7f9f6c594740 1 -- 172.21.15.47:0/894162405 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55776ef37790 con 0x55776efdac10 2024-03-21T11:28:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.305+0000 7f9f62a00640 1 --2- 172.21.15.47:0/894162405 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55776efdac10 0x55776efdd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.305+0000 7f9f62a00640 1 --2- 172.21.15.47:0/894162405 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55776efdac10 0x55776efdd060 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ee22926a86f7d5 server_cookie=dfc2b3c7d04ac5d3 in_seq=0 out_seq=0 2024-03-21T11:28:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.305+0000 7f9f4cc00640 10 client.27133 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.307+0000 7f9f4cc00640 1 -- 172.21.15.47:0/894162405 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55776ef37790 con 0x55776efdac10 2024-03-21T11:28:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.307+0000 7f9f4cc00640 10 client.27133 handle_command_reply: tid=0 2024-03-21T11:28:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.307+0000 7f9f6c594740 1 client.27133 shutdown 2024-03-21T11:28:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.307+0000 7f9f6c594740 1 -- 172.21.15.47:0/894162405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc5870 msgr2=0x55776efcf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.307+0000 7f9f6c594740 1 --2- 172.21.15.47:0/894162405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc5870 0x55776efcf410 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9f500bcf40 tx=0x7f9f50009490 comp rx=0 tx=0).stop 2024-03-21T11:28:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.307+0000 7f9f6c594740 1 -- 172.21.15.47:0/894162405 shutdown_connections 2024-03-21T11:28:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.307+0000 7f9f6c594740 1 --2- 172.21.15.47:0/894162405 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55776efdac10 0x55776efdd060 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.307+0000 7f9f6c594740 1 --2- 172.21.15.47:0/894162405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 0x55776efd6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.307+0000 7f9f6c594740 1 --2- 172.21.15.47:0/894162405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc5870 0x55776efcf410 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.307+0000 7f9f6c594740 1 --2- 172.21.15.47:0/894162405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 0x55776efceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.307+0000 7f9f6c594740 1 -- 172.21.15.47:0/894162405 >> 172.21.15.47:0/894162405 conn(0x55776ef8d200 msgr2=0x55776efbfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:21.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.307+0000 7f9f4cc00640 0 client.27133 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:21.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.307+0000 7f9f6c594740 1 -- 172.21.15.47:0/894162405 shutdown_connections 2024-03-21T11:28:21.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.307+0000 7f9f6c594740 1 -- 172.21.15.47:0/894162405 wait complete. 2024-03-21T11:28:21.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.308+0000 7f9f6c594740 20 client.27133 trim_cache size 0 max 16384 2024-03-21T11:28:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:20 smithi047 ceph-mon[31351]: pgmap v550: 129 pgs: 129 active+clean; 38 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 317 op/s 2024-03-21T11:28:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.327+0000 7f9f6c594740 1 Processor -- start 2024-03-21T11:28:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.327+0000 7f9f6c594740 1 -- start start 2024-03-21T11:28:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.327+0000 7f9f6c594740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 0x55776efc3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.327+0000 7f9f6c594740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efc5870 0x55776efd6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.327+0000 7f9f6c594740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efca9e0 0x55776efc3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.327+0000 7f9f6c594740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f641ac2f0 con 0x55776efc4e80 2024-03-21T11:28:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.327+0000 7f9f6c594740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f641adc70 con 0x55776efc5870 2024-03-21T11:28:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.327+0000 7f9f6c594740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f641ac950 con 0x55776efca9e0 2024-03-21T11:28:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.328+0000 7f9f63400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efca9e0 0x55776efc3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.328+0000 7f9f63400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efca9e0 0x55776efc3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36726/0 (socket says 172.21.15.47:36726) 2024-03-21T11:28:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.328+0000 7f9f63400640 1 -- 172.21.15.47:0/1697867057 learned_addr learned my addr 172.21.15.47:0/1697867057 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.328+0000 7f9f62000640 1 --2- 172.21.15.47:0/1697867057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efc5870 0x55776efd6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.328+0000 7f9f62a00640 1 --2- 172.21.15.47:0/1697867057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 0x55776efc3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.328+0000 7f9f63400640 1 -- 172.21.15.47:0/1697867057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efc5870 msgr2=0x55776efd6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.328+0000 7f9f63400640 1 --2- 172.21.15.47:0/1697867057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efc5870 0x55776efd6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.328+0000 7f9f63400640 1 -- 172.21.15.47:0/1697867057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 msgr2=0x55776efc3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.328+0000 7f9f63400640 1 --2- 172.21.15.47:0/1697867057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 0x55776efc3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.328+0000 7f9f63400640 1 -- 172.21.15.47:0/1697867057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55776ebfe0f0 con 0x55776efca9e0 2024-03-21T11:28:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.328+0000 7f9f62a00640 1 --2- 172.21.15.47:0/1697867057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 0x55776efc3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.328+0000 7f9f63400640 1 --2- 172.21.15.47:0/1697867057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efca9e0 0x55776efc3f70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c01a000 tx=0x7f9f5c009e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=260a63e8d0218506 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.329+0000 7f9f4f400640 1 -- 172.21.15.47:0/1697867057 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c003970 con 0x55776efca9e0 2024-03-21T11:28:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.329+0000 7f9f4f400640 1 -- 172.21.15.47:0/1697867057 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f5c00d830 con 0x55776efca9e0 2024-03-21T11:28:21.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.329+0000 7f9f4f400640 1 -- 172.21.15.47:0/1697867057 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c005550 con 0x55776efca9e0 2024-03-21T11:28:21.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.329+0000 7f9f6c594740 1 -- 172.21.15.47:0/1697867057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efca9e0 msgr2=0x55776efc3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:21.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.329+0000 7f9f6c594740 1 --2- 172.21.15.47:0/1697867057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efca9e0 0x55776efc3f70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c01a000 tx=0x7f9f5c009e10 comp rx=0 tx=0).stop 2024-03-21T11:28:21.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.330+0000 7f9f6c594740 1 -- 172.21.15.47:0/1697867057 shutdown_connections 2024-03-21T11:28:21.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.330+0000 7f9f6c594740 1 --2- 172.21.15.47:0/1697867057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efca9e0 0x55776efc3f70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.330+0000 7f9f6c594740 1 --2- 172.21.15.47:0/1697867057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efc5870 0x55776efd6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.330+0000 7f9f6c594740 1 --2- 172.21.15.47:0/1697867057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 0x55776efc3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.330+0000 7f9f6c594740 1 -- 172.21.15.47:0/1697867057 >> 172.21.15.47:0/1697867057 conn(0x55776ef8d200 msgr2=0x55776efc6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.330+0000 7f9f6c594740 1 -- 172.21.15.47:0/1697867057 shutdown_connections 2024-03-21T11:28:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.330+0000 7f9f6c594740 1 -- 172.21.15.47:0/1697867057 wait complete. 2024-03-21T11:28:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.330+0000 7f9f6c594740 1 -- start start 2024-03-21T11:28:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.330+0000 7f9f6c594740 1 Processor -- start 2024-03-21T11:28:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f6c594740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 0x55776efd3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f6c594740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc5870 0x55776efd4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f6c594740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 0x55776efd5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f6c594740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f641ad610 con 0x55776efc4e80 2024-03-21T11:28:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f6c594740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f641acfb0 con 0x55776efca9e0 2024-03-21T11:28:21.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f6c594740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f641aa4b0 con 0x55776efc5870 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f6c594740 10 client.0 _ll_register_callbacks cb 0x7f9f500057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f62000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc5870 0x55776efd4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f62000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc5870 0x55776efd4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36740/0 (socket says 172.21.15.47:36740) 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f62000640 1 -- 172.21.15.47:0/1611625765 learned_addr learned my addr 172.21.15.47:0/1611625765 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f63400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 0x55776efd5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f62a00640 1 --2- 172.21.15.47:0/1611625765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 0x55776efd3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f62000640 1 -- 172.21.15.47:0/1611625765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 msgr2=0x55776efd5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f62000640 1 --2- 172.21.15.47:0/1611625765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 0x55776efd5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f62000640 1 -- 172.21.15.47:0/1611625765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 msgr2=0x55776efd3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f62000640 1 --2- 172.21.15.47:0/1611625765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 0x55776efd3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f62000640 1 -- 172.21.15.47:0/1611625765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55776efd5a80 con 0x55776efc5870 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f62000640 1 --2- 172.21.15.47:0/1611625765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc5870 0x55776efd4480 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9f500ee6f0 tx=0x7f9f500dd2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66932e26772abc17 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.331+0000 7f9f4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.332+0000 7f9f4cc00640 1 -- 172.21.15.47:0/1611625765 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500bd6b0 con 0x55776efc5870 2024-03-21T11:28:21.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.332+0000 7f9f4cc00640 1 -- 172.21.15.47:0/1611625765 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9f500c5600 con 0x55776efc5870 2024-03-21T11:28:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.332+0000 7f9f6c594740 1 -- 172.21.15.47:0/1611625765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f5c00d830 con 0x55776efc5870 2024-03-21T11:28:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.332+0000 7f9f4cc00640 1 -- 172.21.15.47:0/1611625765 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f500c1100 con 0x55776efc5870 2024-03-21T11:28:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.332+0000 7f9f6c594740 10 client.27139 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.332+0000 7f9f6c594740 1 -- 172.21.15.47:0/1611625765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55776f036920 con 0x55776efc5870 2024-03-21T11:28:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.332+0000 7f9f4cc00640 1 -- 172.21.15.47:0/1611625765 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f5c00d830 con 0x55776efc5870 2024-03-21T11:28:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.332+0000 7f9f4cc00640 1 -- 172.21.15.47:0/1611625765 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9f500a1050 con 0x55776efc5870 2024-03-21T11:28:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.332+0000 7f9f6c594740 10 client.27139 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.332+0000 7f9f6c594740 10 client.27139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.332+0000 7f9f6c594740 20 client.27139 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.332+0000 7f9f6c594740 1 --2- 172.21.15.47:0/1611625765 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55776f036ee0 0x55776f039300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.332+0000 7f9f6c594740 4 client.27139 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:28:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.332+0000 7f9f6c594740 1 -- 172.21.15.47:0/1611625765 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55776ef31ae0 con 0x55776f036ee0 2024-03-21T11:28:21.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.333+0000 7f9f62a00640 1 --2- 172.21.15.47:0/1611625765 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55776f036ee0 0x55776f039300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.333+0000 7f9f62a00640 1 --2- 172.21.15.47:0/1611625765 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55776f036ee0 0x55776f039300 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a449b2a933cec90 server_cookie=64da31f14ea2725b in_seq=0 out_seq=0 2024-03-21T11:28:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.333+0000 7f9f4cc00640 10 client.27139 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.333+0000 7f9f4cc00640 1 -- 172.21.15.47:0/1611625765 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55776ef31ae0 con 0x55776f036ee0 2024-03-21T11:28:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.333+0000 7f9f4cc00640 10 client.27139 handle_command_reply: tid=0 2024-03-21T11:28:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.333+0000 7f9f6c594740 1 client.27139 shutdown 2024-03-21T11:28:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.334+0000 7f9f6c594740 1 -- 172.21.15.47:0/1611625765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc5870 msgr2=0x55776efd4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.334+0000 7f9f6c594740 1 --2- 172.21.15.47:0/1611625765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc5870 0x55776efd4480 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9f500ee6f0 tx=0x7f9f500dd2e0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.334+0000 7f9f6c594740 1 -- 172.21.15.47:0/1611625765 shutdown_connections 2024-03-21T11:28:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.334+0000 7f9f6c594740 1 --2- 172.21.15.47:0/1611625765 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55776f036ee0 0x55776f039300 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.334+0000 7f9f6c594740 1 --2- 172.21.15.47:0/1611625765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55776efca9e0 0x55776efd5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.334+0000 7f9f6c594740 1 --2- 172.21.15.47:0/1611625765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55776efc5870 0x55776efd4480 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.334+0000 7f9f6c594740 1 --2- 172.21.15.47:0/1611625765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55776efc4e80 0x55776efd3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.334+0000 7f9f4cc00640 0 client.27139 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:21.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.334+0000 7f9f6c594740 1 -- 172.21.15.47:0/1611625765 >> 172.21.15.47:0/1611625765 conn(0x55776ef8d200 msgr2=0x55776efc6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:21.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.334+0000 7f9f6c594740 1 -- 172.21.15.47:0/1611625765 shutdown_connections 2024-03-21T11:28:21.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.334+0000 7f9f6c594740 1 -- 172.21.15.47:0/1611625765 wait complete. 2024-03-21T11:28:21.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.334+0000 7f9f6c594740 20 client.27139 trim_cache size 0 max 16384 2024-03-21T11:28:21.339 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:28:21.339 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:28:21.339 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:28:21.339 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:28:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.335+0000 7f9f69800640 1 -- 172.21.15.47:0/1677051879 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9f480a2fa0 msgr2=0x7f9f480a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.335+0000 7f9f69800640 1 --2- 172.21.15.47:0/1677051879 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9f480a2fa0 0x7f9f480a5460 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55776efc5c70 tx=0x7f9f54005f90 comp rx=0 tx=0).stop 2024-03-21T11:28:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.335+0000 7f9f69800640 1 -- 172.21.15.47:0/1677051879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f640756a0 msgr2=0x7f9f6419a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.335+0000 7f9f69800640 1 --2- 172.21.15.47:0/1677051879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f640756a0 0x7f9f6419a120 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9f500349e0 tx=0x7f9f50032450 comp rx=0 tx=0).stop 2024-03-21T11:28:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.335+0000 7f9f69800640 1 -- 172.21.15.47:0/1677051879 shutdown_connections 2024-03-21T11:28:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.335+0000 7f9f69800640 1 --2- 172.21.15.47:0/1677051879 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9f480a2fa0 0x7f9f480a5460 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.335+0000 7f9f69800640 1 --2- 172.21.15.47:0/1677051879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f64109560 0x7f9f641a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.335+0000 7f9f69800640 1 --2- 172.21.15.47:0/1677051879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f640756a0 0x7f9f6419a120 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.335+0000 7f9f69800640 1 --2- 172.21.15.47:0/1677051879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f640740c0 0x7f9f64199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.335+0000 7f9f69800640 1 -- 172.21.15.47:0/1677051879 >> 172.21.15.47:0/1677051879 conn(0x7f9f640fcc40 msgr2=0x7f9f641054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:21.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.335+0000 7f9f69800640 1 -- 172.21.15.47:0/1677051879 shutdown_connections 2024-03-21T11:28:21.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.335+0000 7f9f69800640 1 -- 172.21.15.47:0/1677051879 wait complete. 2024-03-21T11:28:21.353 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:28:21.353 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:28:21.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:20 smithi100 ceph-mon[36245]: pgmap v550: 129 pgs: 129 active+clean; 38 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 317 op/s 2024-03-21T11:28:21.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:20 smithi102 ceph-mon[36748]: pgmap v550: 129 pgs: 129 active+clean; 38 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 317 op/s 2024-03-21T11:28:21.510 INFO:tasks.workunit.client.0.smithi047.stdout: 1 296117 8.34 MB/sec execute 65 sec latency 14.835 ms 2024-03-21T11:28:21.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.583+0000 7f10d9400640 1 -- 172.21.15.47:0/1674096013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d410b490 msgr2=0x7f10d4114330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:21.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.583+0000 7f10d9400640 1 --2- 172.21.15.47:0/1674096013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d410b490 0x7f10d4114330 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009900 tx=0x7f10c002bc50 comp rx=0 tx=0).stop 2024-03-21T11:28:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.584+0000 7f10d9400640 1 -- 172.21.15.47:0/1674096013 shutdown_connections 2024-03-21T11:28:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.584+0000 7f10d9400640 1 --2- 172.21.15.47:0/1674096013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d410b490 0x7f10d4114330 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.584+0000 7f10d9400640 1 --2- 172.21.15.47:0/1674096013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d410b010 0x7f10d4111de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.584+0000 7f10d9400640 1 --2- 172.21.15.47:0/1674096013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d41086a0 0x7f10d410aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.584+0000 7f10d9400640 1 -- 172.21.15.47:0/1674096013 >> 172.21.15.47:0/1674096013 conn(0x7f10d406e510 msgr2=0x7f10d4070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:21.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.585+0000 7f10d9400640 1 -- 172.21.15.47:0/1674096013 shutdown_connections 2024-03-21T11:28:21.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.585+0000 7f10d9400640 1 -- 172.21.15.47:0/1674096013 wait complete. 2024-03-21T11:28:21.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.586+0000 7f10d9400640 1 Processor -- start 2024-03-21T11:28:21.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.587+0000 7f10d9400640 1 -- start start 2024-03-21T11:28:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.589+0000 7f10d9400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d41086a0 0x7f10d41a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.589+0000 7f10d9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d410b010 0x7f10d41a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.589+0000 7f10d9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d410b490 0x7f10d41a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.589+0000 7f10d9400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d4117e50 con 0x7f10d41086a0 2024-03-21T11:28:21.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.589+0000 7f10d9400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d4117cd0 con 0x7f10d410b490 2024-03-21T11:28:21.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.589+0000 7f10d9400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d4117fd0 con 0x7f10d410b010 2024-03-21T11:28:21.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.590+0000 7f10d3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d410b490 0x7f10d41a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.590+0000 7f10d1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d410b010 0x7f10d41a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.590+0000 7f10d1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d410b010 0x7f10d41a2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36758/0 (socket says 172.21.15.47:36758) 2024-03-21T11:28:21.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.590+0000 7f10d1e00640 1 -- 172.21.15.47:0/2097290497 learned_addr learned my addr 172.21.15.47:0/2097290497 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:21.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.592+0000 7f10d3200640 1 -- 172.21.15.47:0/2097290497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d410b010 msgr2=0x7f10d41a2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:21.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.592+0000 7f10d3200640 1 --2- 172.21.15.47:0/2097290497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d410b010 0x7f10d41a2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.592+0000 7f10d3200640 1 -- 172.21.15.47:0/2097290497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d41086a0 msgr2=0x7f10d41a2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:21.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.592+0000 7f10d3200640 1 --2- 172.21.15.47:0/2097290497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d41086a0 0x7f10d41a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.592+0000 7f10d3200640 1 -- 172.21.15.47:0/2097290497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d41aa0d0 con 0x7f10d410b490 2024-03-21T11:28:21.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.592+0000 7f10d1e00640 1 --2- 172.21.15.47:0/2097290497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d410b010 0x7f10d41a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:21.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.593+0000 7f10d3200640 1 --2- 172.21.15.47:0/2097290497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d410b490 0x7f10d41a9a60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f10c002c7b0 tx=0x7f10c0005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:21.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.596+0000 7f10cb400640 1 -- 172.21.15.47:0/2097290497 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0043020 con 0x7f10d410b490 2024-03-21T11:28:21.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.596+0000 7f10cb400640 1 -- 172.21.15.47:0/2097290497 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10c0004410 con 0x7f10d410b490 2024-03-21T11:28:21.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.596+0000 7f10cb400640 1 -- 172.21.15.47:0/2097290497 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0003d50 con 0x7f10d410b490 2024-03-21T11:28:21.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.596+0000 7f10d9400640 1 -- 172.21.15.47:0/2097290497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d41aa360 con 0x7f10d410b490 2024-03-21T11:28:21.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.597+0000 7f10d9400640 1 -- 172.21.15.47:0/2097290497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d41aa830 con 0x7f10d410b490 2024-03-21T11:28:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.600+0000 7f10d9400640 1 -- 172.21.15.47:0/2097290497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d4068490 con 0x7f10d410b490 2024-03-21T11:28:21.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.609+0000 7f10cb400640 1 -- 172.21.15.47:0/2097290497 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f10c0046020 con 0x7f10d410b490 2024-03-21T11:28:21.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.611+0000 7f10cb400640 1 --2- 172.21.15.47:0/2097290497 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f10b80a30a0 0x7f10b80a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:21.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.612+0000 7f10cb400640 1 -- 172.21.15.47:0/2097290497 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f10c00f1ae0 con 0x7f10d410b490 2024-03-21T11:28:21.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.612+0000 7f10d2800640 1 --2- 172.21.15.47:0/2097290497 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f10b80a30a0 0x7f10b80a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:21.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.614+0000 7f10d2800640 1 --2- 172.21.15.47:0/2097290497 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f10b80a30a0 0x7f10b80a5560 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f10d4072790 tx=0x7f10c4006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.620+0000 7f10cb400640 1 -- 172.21.15.47:0/2097290497 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f10c004d020 con 0x7f10d410b490 2024-03-21T11:28:21.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.908+0000 7f10d9400640 1 -- 172.21.15.47:0/2097290497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10d40622a0 con 0x7f10d410b490 2024-03-21T11:28:21.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.909+0000 7f10cb400640 1 -- 172.21.15.47:0/2097290497 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f10c00be580 con 0x7f10d410b490 2024-03-21T11:28:21.911 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:28:21.911 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:28:21.911 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:28:21.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.912+0000 7f10d9400640 1 -- 172.21.15.47:0/2097290497 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f10b80a30a0 msgr2=0x7f10b80a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:21.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.913+0000 7f10d9400640 1 --2- 172.21.15.47:0/2097290497 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f10b80a30a0 0x7f10b80a5560 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f10d4072790 tx=0x7f10c4006c90 comp rx=0 tx=0).stop 2024-03-21T11:28:21.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.913+0000 7f10d9400640 1 -- 172.21.15.47:0/2097290497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d410b490 msgr2=0x7f10d41a9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:21.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.913+0000 7f10d9400640 1 --2- 172.21.15.47:0/2097290497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d410b490 0x7f10d41a9a60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f10c002c7b0 tx=0x7f10c0005a40 comp rx=0 tx=0).stop 2024-03-21T11:28:21.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.913+0000 7f10d9400640 1 -- 172.21.15.47:0/2097290497 shutdown_connections 2024-03-21T11:28:21.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.913+0000 7f10d9400640 1 --2- 172.21.15.47:0/2097290497 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f10b80a30a0 0x7f10b80a5560 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.913+0000 7f10d9400640 1 --2- 172.21.15.47:0/2097290497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d410b490 0x7f10d41a9a60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.913+0000 7f10d9400640 1 --2- 172.21.15.47:0/2097290497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d410b010 0x7f10d41a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.913+0000 7f10d9400640 1 --2- 172.21.15.47:0/2097290497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10d41086a0 0x7f10d41a2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:21.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.913+0000 7f10d9400640 1 -- 172.21.15.47:0/2097290497 >> 172.21.15.47:0/2097290497 conn(0x7f10d406e510 msgr2=0x7f10d4070650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:21.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.913+0000 7f10d9400640 1 -- 172.21.15.47:0/2097290497 shutdown_connections 2024-03-21T11:28:21.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:21.913+0000 7f10d9400640 1 -- 172.21.15.47:0/2097290497 wait complete. 2024-03-21T11:28:21.936 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:28:21.936 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:28:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.149+0000 7fc27fa00640 1 -- 172.21.15.47:0/1750725205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2781016b0 msgr2=0x7fc278101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.149+0000 7fc27fa00640 1 --2- 172.21.15.47:0/1750725205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2781016b0 0x7fc278101ad0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc268009900 tx=0x7fc26802bc40 comp rx=0 tx=0).stop 2024-03-21T11:28:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.150+0000 7fc27fa00640 1 -- 172.21.15.47:0/1750725205 shutdown_connections 2024-03-21T11:28:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.150+0000 7fc27fa00640 1 --2- 172.21.15.47:0/1750725205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc278103260 0x7fc278109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.150+0000 7fc27fa00640 1 --2- 172.21.15.47:0/1750725205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2781028a0 0x7fc278102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.150+0000 7fc27fa00640 1 --2- 172.21.15.47:0/1750725205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2781016b0 0x7fc278101ad0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.150+0000 7fc27fa00640 1 -- 172.21.15.47:0/1750725205 >> 172.21.15.47:0/1750725205 conn(0x7fc2780fcc40 msgr2=0x7fc2780ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.150+0000 7fc27fa00640 1 -- 172.21.15.47:0/1750725205 shutdown_connections 2024-03-21T11:28:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.150+0000 7fc27fa00640 1 -- 172.21.15.47:0/1750725205 wait complete. 2024-03-21T11:28:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.151+0000 7fc27fa00640 1 Processor -- start 2024-03-21T11:28:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.151+0000 7fc27fa00640 1 -- start start 2024-03-21T11:28:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.151+0000 7fc27fa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2781016b0 0x7fc278197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.151+0000 7fc27fa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2781028a0 0x7fc278197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.151+0000 7fc27fa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc278103260 0x7fc27819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.151+0000 7fc27fa00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27810d300 con 0x7fc2781028a0 2024-03-21T11:28:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.151+0000 7fc27fa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27810d180 con 0x7fc278103260 2024-03-21T11:28:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.151+0000 7fc27fa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27810d480 con 0x7fc2781016b0 2024-03-21T11:28:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.152+0000 7fc27d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2781016b0 0x7fc278197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.152+0000 7fc27d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2781016b0 0x7fc278197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36774/0 (socket says 172.21.15.47:36774) 2024-03-21T11:28:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.152+0000 7fc27d000640 1 -- 172.21.15.47:0/2812673884 learned_addr learned my addr 172.21.15.47:0/2812673884 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.152+0000 7fc277e00640 1 --2- 172.21.15.47:0/2812673884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2781028a0 0x7fc278197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.152+0000 7fc27da00640 1 --2- 172.21.15.47:0/2812673884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc278103260 0x7fc27819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.152+0000 7fc27d000640 1 -- 172.21.15.47:0/2812673884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc278103260 msgr2=0x7fc27819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.152+0000 7fc27d000640 1 --2- 172.21.15.47:0/2812673884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc278103260 0x7fc27819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.152+0000 7fc27d000640 1 -- 172.21.15.47:0/2812673884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2781028a0 msgr2=0x7fc278197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.152+0000 7fc27d000640 1 --2- 172.21.15.47:0/2812673884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2781028a0 0x7fc278197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.152+0000 7fc27d000640 1 -- 172.21.15.47:0/2812673884 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27819f580 con 0x7fc2781016b0 2024-03-21T11:28:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.152+0000 7fc27d000640 1 --2- 172.21.15.47:0/2812673884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2781016b0 0x7fc278197960 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc268036040 tx=0x7fc268005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.153+0000 7fc275600640 1 -- 172.21.15.47:0/2812673884 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268041070 con 0x7fc2781016b0 2024-03-21T11:28:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.153+0000 7fc275600640 1 -- 172.21.15.47:0/2812673884 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2680054e0 con 0x7fc2781016b0 2024-03-21T11:28:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.153+0000 7fc27fa00640 1 -- 172.21.15.47:0/2812673884 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27819f810 con 0x7fc2781016b0 2024-03-21T11:28:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.153+0000 7fc275600640 1 -- 172.21.15.47:0/2812673884 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268034440 con 0x7fc2781016b0 2024-03-21T11:28:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.153+0000 7fc27fa00640 1 -- 172.21.15.47:0/2812673884 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27819fd00 con 0x7fc2781016b0 2024-03-21T11:28:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.156+0000 7fc275600640 1 -- 172.21.15.47:0/2812673884 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc268044070 con 0x7fc2781016b0 2024-03-21T11:28:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.156+0000 7fc2825df740 1 Processor -- start 2024-03-21T11:28:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.157+0000 7fc2825df740 1 -- start start 2024-03-21T11:28:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.157+0000 7fc2825df740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5710c20 0x5564a5711040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.157+0000 7fc2825df740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5711df0 0x5564a5712270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.157+0000 7fc2825df740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a5719040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.157+0000 7fc2825df740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a56712c0 con 0x5564a5711df0 2024-03-21T11:28:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.157+0000 7fc2825df740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a5619c90 con 0x5564a5710c20 2024-03-21T11:28:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.157+0000 7fc2825df740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a566d360 con 0x5564a57127b0 2024-03-21T11:28:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.157+0000 7fc27da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5710c20 0x5564a5711040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.158+0000 7fc277e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a5719040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.158+0000 7fc27da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5710c20 0x5564a5711040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44604/0 (socket says 172.21.15.47:44604) 2024-03-21T11:28:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.158+0000 7fc27da00640 1 -- 172.21.15.47:0/3397844028 learned_addr learned my addr 172.21.15.47:0/3397844028 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.158+0000 7fc27d000640 1 --2- 172.21.15.47:0/3397844028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5711df0 0x5564a5712270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.158+0000 7fc27da00640 1 -- 172.21.15.47:0/3397844028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 msgr2=0x5564a5719040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.158+0000 7fc27da00640 1 --2- 172.21.15.47:0/3397844028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a5719040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.158+0000 7fc27da00640 1 -- 172.21.15.47:0/3397844028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5711df0 msgr2=0x5564a5712270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.158+0000 7fc27da00640 1 --2- 172.21.15.47:0/3397844028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5711df0 0x5564a5712270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.158+0000 7fc27da00640 1 -- 172.21.15.47:0/3397844028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564a563e080 con 0x5564a5710c20 2024-03-21T11:28:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.158+0000 7fc277e00640 1 --2- 172.21.15.47:0/3397844028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a5719040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.158+0000 7fc275600640 1 --2- 172.21.15.47:0/2812673884 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc25c0a2f20 0x7fc25c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.158+0000 7fc275600640 1 -- 172.21.15.47:0/2812673884 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc2680f13f0 con 0x7fc2781016b0 2024-03-21T11:28:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.158+0000 7fc27da00640 1 --2- 172.21.15.47:0/3397844028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5710c20 0x5564a5711040 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc270007f40 tx=0x7fc27000f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adeaf4978116f9e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.158+0000 7fc277e00640 1 --2- 172.21.15.47:0/2812673884 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc25c0a2f20 0x7fc25c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.159+0000 7fc26e000640 1 -- 172.21.15.47:0/3397844028 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270010900 con 0x5564a5710c20 2024-03-21T11:28:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.159+0000 7fc26e000640 1 -- 172.21.15.47:0/3397844028 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2700183f0 con 0x5564a5710c20 2024-03-21T11:28:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.159+0000 7fc26e000640 1 -- 172.21.15.47:0/3397844028 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2700175a0 con 0x5564a5710c20 2024-03-21T11:28:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.159+0000 7fc2825df740 1 -- 172.21.15.47:0/3397844028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5710c20 msgr2=0x5564a5711040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.159+0000 7fc277e00640 1 --2- 172.21.15.47:0/2812673884 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc25c0a2f20 0x7fc25c0a53e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc278198e70 tx=0x7fc26000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.159+0000 7fc2825df740 1 --2- 172.21.15.47:0/3397844028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5710c20 0x5564a5711040 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc270007f40 tx=0x7fc27000f850 comp rx=0 tx=0).stop 2024-03-21T11:28:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.160+0000 7fc2825df740 1 -- 172.21.15.47:0/3397844028 shutdown_connections 2024-03-21T11:28:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.160+0000 7fc2825df740 1 --2- 172.21.15.47:0/3397844028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a5719040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.160+0000 7fc2825df740 1 --2- 172.21.15.47:0/3397844028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5711df0 0x5564a5712270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.160+0000 7fc2825df740 1 --2- 172.21.15.47:0/3397844028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5710c20 0x5564a5711040 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.160+0000 7fc2825df740 1 -- 172.21.15.47:0/3397844028 >> 172.21.15.47:0/3397844028 conn(0x5564a5707c00 msgr2=0x5564a5708000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.160+0000 7fc2825df740 1 -- 172.21.15.47:0/3397844028 shutdown_connections 2024-03-21T11:28:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.160+0000 7fc2825df740 1 -- 172.21.15.47:0/3397844028 wait complete. 2024-03-21T11:28:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.160+0000 7fc2825df740 1 -- start start 2024-03-21T11:28:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.160+0000 7fc2825df740 1 Processor -- start 2024-03-21T11:28:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.161+0000 7fc2825df740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5710c20 0x5564a571aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.161+0000 7fc2825df740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5711df0 0x5564a571b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.161+0000 7fc2825df740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a57223d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.161+0000 7fc2825df740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a57197c0 con 0x5564a5710c20 2024-03-21T11:28:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.161+0000 7fc2825df740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a562d0d0 con 0x5564a5711df0 2024-03-21T11:28:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.161+0000 7fc2825df740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a5719940 con 0x5564a57127b0 2024-03-21T11:28:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.161+0000 7fc2825df740 10 client.0 _ll_register_callbacks cb 0x5564a5631e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.161+0000 7fc277e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5711df0 0x5564a571b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.161+0000 7fc27da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a57223d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.161+0000 7fc27d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5710c20 0x5564a571aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.161+0000 7fc27da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a57223d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36802/0 (socket says 172.21.15.47:36802) 2024-03-21T11:28:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.161+0000 7fc27da00640 1 -- 172.21.15.47:0/598275067 learned_addr learned my addr 172.21.15.47:0/598275067 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.161+0000 7fc277e00640 1 --2- 172.21.15.47:0/598275067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5711df0 0x5564a571b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44618/0 (socket says 172.21.15.47:44618) 2024-03-21T11:28:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.162+0000 7fc27da00640 1 -- 172.21.15.47:0/598275067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5711df0 msgr2=0x5564a571b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.162+0000 7fc27da00640 1 --2- 172.21.15.47:0/598275067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5711df0 0x5564a571b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.162+0000 7fc27da00640 1 -- 172.21.15.47:0/598275067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5710c20 msgr2=0x5564a571aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.162+0000 7fc27da00640 1 --2- 172.21.15.47:0/598275067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5710c20 0x5564a571aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.162+0000 7fc27da00640 1 -- 172.21.15.47:0/598275067 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564a53853f0 con 0x5564a57127b0 2024-03-21T11:28:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.162+0000 7fc27da00640 1 --2- 172.21.15.47:0/598275067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a57223d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc27001a040 tx=0x7fc270007c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13e2ba0ec4bfcb63 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.162+0000 7fc266a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.162+0000 7fc266a00640 1 -- 172.21.15.47:0/598275067 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270025020 con 0x5564a57127b0 2024-03-21T11:28:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.162+0000 7fc2825df740 1 -- 172.21.15.47:0/598275067 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2700183f0 con 0x5564a57127b0 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.162+0000 7fc266a00640 1 -- 172.21.15.47:0/598275067 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc270004c50 con 0x5564a57127b0 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.163+0000 7fc266a00640 1 -- 172.21.15.47:0/598275067 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc270030050 con 0x5564a57127b0 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.163+0000 7fc2825df740 10 client.27154 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.163+0000 7fc2825df740 1 -- 172.21.15.47:0/598275067 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5564a544f7d0 con 0x5564a57127b0 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.163+0000 7fc266a00640 1 -- 172.21.15.47:0/598275067 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2700183f0 con 0x5564a57127b0 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.163+0000 7fc266a00640 1 -- 172.21.15.47:0/598275067 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc270010b40 con 0x5564a57127b0 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.163+0000 7fc2825df740 10 client.27154 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.163+0000 7fc2825df740 10 client.27154 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.163+0000 7fc2825df740 20 client.27154 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.163+0000 7fc2825df740 1 --2- 172.21.15.47:0/598275067 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564a57265e0 0x5564a57289d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.163+0000 7fc2825df740 4 client.27154 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.163+0000 7fc2825df740 1 -- 172.21.15.47:0/598275067 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564a567e800 con 0x5564a57265e0 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.163+0000 7fc27d000640 1 --2- 172.21.15.47:0/598275067 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564a57265e0 0x5564a57289d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.164+0000 7fc27d000640 1 --2- 172.21.15.47:0/598275067 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564a57265e0 0x5564a57289d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4a89658cdaafd4d0 server_cookie=3031dbcf71e9460 in_seq=0 out_seq=0 2024-03-21T11:28:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.164+0000 7fc266a00640 10 client.27154 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:28:22.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.165+0000 7fc266a00640 1 -- 172.21.15.47:0/598275067 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5564a567e800 con 0x5564a57265e0 2024-03-21T11:28:22.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.165+0000 7fc266a00640 10 client.27154 handle_command_reply: tid=0 2024-03-21T11:28:22.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.165+0000 7fc2825df740 1 client.27154 shutdown 2024-03-21T11:28:22.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.165+0000 7fc2825df740 1 -- 172.21.15.47:0/598275067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 msgr2=0x5564a57223d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.165+0000 7fc2825df740 1 --2- 172.21.15.47:0/598275067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a57223d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc27001a040 tx=0x7fc270007c40 comp rx=0 tx=0).stop 2024-03-21T11:28:22.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.165+0000 7fc2825df740 1 -- 172.21.15.47:0/598275067 shutdown_connections 2024-03-21T11:28:22.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.165+0000 7fc2825df740 1 --2- 172.21.15.47:0/598275067 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564a57265e0 0x5564a57289d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.165+0000 7fc2825df740 1 --2- 172.21.15.47:0/598275067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a57223d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.165+0000 7fc2825df740 1 --2- 172.21.15.47:0/598275067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5711df0 0x5564a571b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.165+0000 7fc266a00640 0 client.27154 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:28:22.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.165+0000 7fc2825df740 1 --2- 172.21.15.47:0/598275067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5710c20 0x5564a571aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.165+0000 7fc2825df740 1 -- 172.21.15.47:0/598275067 >> 172.21.15.47:0/598275067 conn(0x5564a5707c00 msgr2=0x5564a5706eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.165+0000 7fc2825df740 1 -- 172.21.15.47:0/598275067 shutdown_connections 2024-03-21T11:28:22.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.166+0000 7fc2825df740 1 -- 172.21.15.47:0/598275067 wait complete. 2024-03-21T11:28:22.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.166+0000 7fc2825df740 20 client.27154 trim_cache size 0 max 16384 2024-03-21T11:28:22.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.181+0000 7fc2825df740 1 Processor -- start 2024-03-21T11:28:22.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.182+0000 7fc2825df740 1 -- start start 2024-03-21T11:28:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.182+0000 7fc2825df740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5710c20 0x5564a57155a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.182+0000 7fc2825df740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5711df0 0x5564a5715ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.182+0000 7fc2825df740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a5706620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.182+0000 7fc2825df740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781a7e80 con 0x5564a5711df0 2024-03-21T11:28:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.182+0000 7fc2825df740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781a9800 con 0x5564a5710c20 2024-03-21T11:28:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.182+0000 7fc2825df740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781a84e0 con 0x5564a57127b0 2024-03-21T11:28:22.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.182+0000 7fc27d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5710c20 0x5564a57155a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.182+0000 7fc277e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5711df0 0x5564a5715ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.182+0000 7fc27d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5710c20 0x5564a57155a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44620/0 (socket says 172.21.15.47:44620) 2024-03-21T11:28:22.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.182+0000 7fc27d000640 1 -- 172.21.15.47:0/4260099644 learned_addr learned my addr 172.21.15.47:0/4260099644 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.182+0000 7fc27da00640 1 --2- 172.21.15.47:0/4260099644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a5706620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.183+0000 7fc27d000640 1 -- 172.21.15.47:0/4260099644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 msgr2=0x5564a5706620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.183+0000 7fc27d000640 1 --2- 172.21.15.47:0/4260099644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a5706620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.183+0000 7fc27d000640 1 -- 172.21.15.47:0/4260099644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5711df0 msgr2=0x5564a5715ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.183+0000 7fc27d000640 1 --2- 172.21.15.47:0/4260099644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5711df0 0x5564a5715ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.183+0000 7fc27d000640 1 -- 172.21.15.47:0/4260099644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564a5711610 con 0x5564a5710c20 2024-03-21T11:28:22.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.183+0000 7fc27d000640 1 --2- 172.21.15.47:0/4260099644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5710c20 0x5564a57155a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc2680b9b90 tx=0x7fc2680045d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=917f29c22122b8dd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.183+0000 7fc26d600640 1 -- 172.21.15.47:0/4260099644 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26804b040 con 0x5564a5710c20 2024-03-21T11:28:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.183+0000 7fc26d600640 1 -- 172.21.15.47:0/4260099644 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc268005690 con 0x5564a5710c20 2024-03-21T11:28:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.183+0000 7fc26d600640 1 -- 172.21.15.47:0/4260099644 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2680a80b0 con 0x5564a5710c20 2024-03-21T11:28:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.184+0000 7fc2825df740 1 -- 172.21.15.47:0/4260099644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5710c20 msgr2=0x5564a57155a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.184+0000 7fc2825df740 1 --2- 172.21.15.47:0/4260099644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5710c20 0x5564a57155a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc2680b9b90 tx=0x7fc2680045d0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.184+0000 7fc2825df740 1 -- 172.21.15.47:0/4260099644 shutdown_connections 2024-03-21T11:28:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.184+0000 7fc2825df740 1 --2- 172.21.15.47:0/4260099644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a57127b0 0x5564a5706620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.184+0000 7fc2825df740 1 --2- 172.21.15.47:0/4260099644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5711df0 0x5564a5715ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.184+0000 7fc2825df740 1 --2- 172.21.15.47:0/4260099644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a5710c20 0x5564a57155a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.184+0000 7fc2825df740 1 -- 172.21.15.47:0/4260099644 >> 172.21.15.47:0/4260099644 conn(0x5564a5707c00 msgr2=0x5564a5714e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.184+0000 7fc2825df740 1 -- 172.21.15.47:0/4260099644 shutdown_connections 2024-03-21T11:28:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.184+0000 7fc2825df740 1 -- 172.21.15.47:0/4260099644 wait complete. 2024-03-21T11:28:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.185+0000 7fc2825df740 1 -- start start 2024-03-21T11:28:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.185+0000 7fc2825df740 1 Processor -- start 2024-03-21T11:28:22.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc2825df740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5710c20 0x5564a570e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc2825df740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a5711df0 0x5564a570e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc2825df740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a57127b0 0x5564a5722390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc2825df740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781a91a0 con 0x5564a5710c20 2024-03-21T11:28:22.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc2825df740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781a8b40 con 0x5564a57127b0 2024-03-21T11:28:22.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc2825df740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781a6040 con 0x5564a5711df0 2024-03-21T11:28:22.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc2825df740 10 client.0 _ll_register_callbacks cb 0x7fc26000a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:22.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc27da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a57127b0 0x5564a5722390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc27da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a57127b0 0x5564a5722390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44634/0 (socket says 172.21.15.47:44634) 2024-03-21T11:28:22.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc27d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5710c20 0x5564a570e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc27da00640 1 -- 172.21.15.47:0/3380152080 learned_addr learned my addr 172.21.15.47:0/3380152080 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc277e00640 1 --2- 172.21.15.47:0/3380152080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a5711df0 0x5564a570e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc27da00640 1 -- 172.21.15.47:0/3380152080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a5711df0 msgr2=0x5564a570e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc27da00640 1 --2- 172.21.15.47:0/3380152080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a5711df0 0x5564a570e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc27da00640 1 -- 172.21.15.47:0/3380152080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5710c20 msgr2=0x5564a570e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.186+0000 7fc27da00640 1 --2- 172.21.15.47:0/3380152080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5710c20 0x5564a570e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.187+0000 7fc27da00640 1 -- 172.21.15.47:0/3380152080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564a570f800 con 0x5564a57127b0 2024-03-21T11:28:22.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.187+0000 7fc27da00640 1 --2- 172.21.15.47:0/3380152080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a57127b0 0x5564a5722390 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc27000f890 tx=0x7fc270004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee4a3865dc54460c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.187+0000 7fc266a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:22.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.187+0000 7fc266a00640 1 -- 172.21.15.47:0/3380152080 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270038020 con 0x5564a57127b0 2024-03-21T11:28:22.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.187+0000 7fc2825df740 1 -- 172.21.15.47:0/3380152080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564a5678b50 con 0x5564a57127b0 2024-03-21T11:28:22.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.187+0000 7fc266a00640 1 -- 172.21.15.47:0/3380152080 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc270003970 con 0x5564a57127b0 2024-03-21T11:28:22.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.187+0000 7fc266a00640 1 -- 172.21.15.47:0/3380152080 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc270043050 con 0x5564a57127b0 2024-03-21T11:28:22.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.187+0000 7fc2825df740 10 client.27140 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:22.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.187+0000 7fc2825df740 1 -- 172.21.15.47:0/3380152080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5564a57869b0 con 0x5564a57127b0 2024-03-21T11:28:22.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.188+0000 7fc266a00640 1 -- 172.21.15.47:0/3380152080 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5564a5678b50 con 0x5564a57127b0 2024-03-21T11:28:22.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.188+0000 7fc266a00640 1 -- 172.21.15.47:0/3380152080 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc270018ac0 con 0x5564a57127b0 2024-03-21T11:28:22.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.188+0000 7fc2825df740 10 client.27140 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:22.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.188+0000 7fc2825df740 10 client.27140 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:28:22.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.188+0000 7fc2825df740 20 client.27140 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:22.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.188+0000 7fc2825df740 1 --2- 172.21.15.47:0/3380152080 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564a57872d0 0x5564a57896f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.188+0000 7fc2825df740 4 client.27140 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:28:22.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.188+0000 7fc2825df740 1 -- 172.21.15.47:0/3380152080 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5564a55f6500 con 0x5564a57872d0 2024-03-21T11:28:22.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.188+0000 7fc27d000640 1 --2- 172.21.15.47:0/3380152080 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564a57872d0 0x5564a57896f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc27d000640 1 --2- 172.21.15.47:0/3380152080 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564a57872d0 0x5564a57896f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=29676d13e4b7718d server_cookie=8ab7e40632d95591 in_seq=0 out_seq=0 2024-03-21T11:28:22.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc266a00640 10 client.27140 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:28:22.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc266a00640 1 -- 172.21.15.47:0/3380152080 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5564a55f6500 con 0x5564a57872d0 2024-03-21T11:28:22.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc266a00640 10 client.27140 handle_command_reply: tid=0 2024-03-21T11:28:22.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc2825df740 1 client.27140 shutdown 2024-03-21T11:28:22.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc2825df740 1 -- 172.21.15.47:0/3380152080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a57127b0 msgr2=0x5564a5722390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc2825df740 1 --2- 172.21.15.47:0/3380152080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a57127b0 0x5564a5722390 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc27000f890 tx=0x7fc270004360 comp rx=0 tx=0).stop 2024-03-21T11:28:22.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc2825df740 1 -- 172.21.15.47:0/3380152080 shutdown_connections 2024-03-21T11:28:22.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc2825df740 1 --2- 172.21.15.47:0/3380152080 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5564a57872d0 0x5564a57896f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc2825df740 1 --2- 172.21.15.47:0/3380152080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564a57127b0 0x5564a5722390 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc2825df740 1 --2- 172.21.15.47:0/3380152080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564a5711df0 0x5564a570e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc2825df740 1 --2- 172.21.15.47:0/3380152080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5564a5710c20 0x5564a570e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc2825df740 1 -- 172.21.15.47:0/3380152080 >> 172.21.15.47:0/3380152080 conn(0x5564a5707c00 msgr2=0x5564a572db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc266a00640 0 client.27140 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:28:22.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.189+0000 7fc2825df740 1 -- 172.21.15.47:0/3380152080 shutdown_connections 2024-03-21T11:28:22.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.190+0000 7fc2825df740 1 -- 172.21.15.47:0/3380152080 wait complete. 2024-03-21T11:28:22.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.190+0000 7fc2825df740 20 client.27140 trim_cache size 0 max 16384 2024-03-21T11:28:22.194 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:28:22.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.190+0000 7fc27fa00640 1 -- 172.21.15.47:0/2812673884 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc25c0a2f20 msgr2=0x7fc25c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.190+0000 7fc27fa00640 1 --2- 172.21.15.47:0/2812673884 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc25c0a2f20 0x7fc25c0a53e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc278198e70 tx=0x7fc26000a2d0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.190+0000 7fc27fa00640 1 -- 172.21.15.47:0/2812673884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2781016b0 msgr2=0x7fc278197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.190+0000 7fc27fa00640 1 --2- 172.21.15.47:0/2812673884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2781016b0 0x7fc278197960 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc268036040 tx=0x7fc268005e50 comp rx=0 tx=0).stop 2024-03-21T11:28:22.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.190+0000 7fc27fa00640 1 -- 172.21.15.47:0/2812673884 shutdown_connections 2024-03-21T11:28:22.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.190+0000 7fc27fa00640 1 --2- 172.21.15.47:0/2812673884 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc25c0a2f20 0x7fc25c0a53e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.190+0000 7fc27fa00640 1 --2- 172.21.15.47:0/2812673884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc278103260 0x7fc27819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.190+0000 7fc27fa00640 1 --2- 172.21.15.47:0/2812673884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2781028a0 0x7fc278197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.190+0000 7fc27fa00640 1 --2- 172.21.15.47:0/2812673884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2781016b0 0x7fc278197960 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.190+0000 7fc27fa00640 1 -- 172.21.15.47:0/2812673884 >> 172.21.15.47:0/2812673884 conn(0x7fc2780fcc40 msgr2=0x7fc278105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.191+0000 7fc27fa00640 1 -- 172.21.15.47:0/2812673884 shutdown_connections 2024-03-21T11:28:22.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.191+0000 7fc27fa00640 1 -- 172.21.15.47:0/2812673884 wait complete. 2024-03-21T11:28:22.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:21 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2097290497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:28:22.215 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:28:22.215 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:28:22.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:21 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2097290497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:28:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.454+0000 7fa4a8800640 1 -- 172.21.15.47:0/2621821875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4a01038c0 msgr2=0x7fa4a0103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.454+0000 7fa4a8800640 1 --2- 172.21.15.47:0/2621821875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4a01038c0 0x7fa4a0103ce0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009860 tx=0x7fa48c02bb70 comp rx=0 tx=0).stop 2024-03-21T11:28:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.454+0000 7fa4a8800640 1 -- 172.21.15.47:0/2621821875 shutdown_connections 2024-03-21T11:28:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.454+0000 7fa4a8800640 1 --2- 172.21.15.47:0/2621821875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4a0105470 0x7fa4a010bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.454+0000 7fa4a8800640 1 --2- 172.21.15.47:0/2621821875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4a0104ab0 0x7fa4a0104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.454+0000 7fa4a8800640 1 --2- 172.21.15.47:0/2621821875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4a01038c0 0x7fa4a0103ce0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.454+0000 7fa4a8800640 1 -- 172.21.15.47:0/2621821875 >> 172.21.15.47:0/2621821875 conn(0x7fa4a00fee50 msgr2=0x7fa4a0101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.454+0000 7fa4a8800640 1 -- 172.21.15.47:0/2621821875 shutdown_connections 2024-03-21T11:28:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.454+0000 7fa4a8800640 1 -- 172.21.15.47:0/2621821875 wait complete. 2024-03-21T11:28:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.455+0000 7fa4a8800640 1 Processor -- start 2024-03-21T11:28:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.455+0000 7fa4a8800640 1 -- start start 2024-03-21T11:28:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.456+0000 7fa4a8800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4a01038c0 0x7fa4a0199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.456+0000 7fa4a8800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4a0104ab0 0x7fa4a019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.456+0000 7fa4a8800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4a0105470 0x7fa4a01a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.456+0000 7fa4a8800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a010f510 con 0x7fa4a01038c0 2024-03-21T11:28:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.456+0000 7fa4a8800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a010f390 con 0x7fa4a0105470 2024-03-21T11:28:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.456+0000 7fa4a8800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a010f690 con 0x7fa4a0104ab0 2024-03-21T11:28:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.456+0000 7fa4a5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4a01038c0 0x7fa4a0199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.456+0000 7fa4a5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4a0104ab0 0x7fa4a019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.456+0000 7fa4a5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4a0104ab0 0x7fa4a019a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36840/0 (socket says 172.21.15.47:36840) 2024-03-21T11:28:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.456+0000 7fa4a5400640 1 -- 172.21.15.47:0/2716586527 learned_addr learned my addr 172.21.15.47:0/2716586527 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.456+0000 7fa4a6800640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4a0105470 0x7fa4a01a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.456+0000 7fa4a5400640 1 -- 172.21.15.47:0/2716586527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4a0105470 msgr2=0x7fa4a01a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.456+0000 7fa4a5400640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4a0105470 0x7fa4a01a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.457+0000 7fa4a5400640 1 -- 172.21.15.47:0/2716586527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4a01038c0 msgr2=0x7fa4a0199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.457+0000 7fa4a5400640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4a01038c0 0x7fa4a0199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.457+0000 7fa4a5400640 1 -- 172.21.15.47:0/2716586527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a01a17d0 con 0x7fa4a0104ab0 2024-03-21T11:28:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.457+0000 7fa4a5e00640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4a01038c0 0x7fa4a0199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.457+0000 7fa4a6800640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4a0105470 0x7fa4a01a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.457+0000 7fa4a5400640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4a0104ab0 0x7fa4a019a0f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa49400e9c0 tx=0x7fa49400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.457+0000 7fa49aa00640 1 -- 172.21.15.47:0/2716586527 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49400ccc0 con 0x7fa4a0104ab0 2024-03-21T11:28:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.457+0000 7fa4a8800640 1 -- 172.21.15.47:0/2716586527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a01a1ac0 con 0x7fa4a0104ab0 2024-03-21T11:28:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.457+0000 7fa49aa00640 1 -- 172.21.15.47:0/2716586527 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa494005500 con 0x7fa4a0104ab0 2024-03-21T11:28:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.457+0000 7fa49aa00640 1 -- 172.21.15.47:0/2716586527 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494010430 con 0x7fa4a0104ab0 2024-03-21T11:28:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.457+0000 7fa4a8800640 1 -- 172.21.15.47:0/2716586527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a01a2010 con 0x7fa4a0104ab0 2024-03-21T11:28:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.461+0000 7fa4ab5c2740 1 Processor -- start 2024-03-21T11:28:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.461+0000 7fa49aa00640 1 -- 172.21.15.47:0/2716586527 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa4940046c0 con 0x7fa4a0104ab0 2024-03-21T11:28:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.461+0000 7fa4ab5c2740 1 -- start start 2024-03-21T11:28:22.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.461+0000 7fa4ab5c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373d810 0x563fc373dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.461+0000 7fa4ab5c2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 0x563fc373ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.461+0000 7fa4ab5c2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373f3a0 0x563fc3745c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.461+0000 7fa4ab5c2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc36a22c0 con 0x563fc373e9e0 2024-03-21T11:28:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.461+0000 7fa4ab5c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc364ac90 con 0x563fc373d810 2024-03-21T11:28:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.461+0000 7fa4ab5c2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc369e360 con 0x563fc373f3a0 2024-03-21T11:28:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.462+0000 7fa4a6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373d810 0x563fc373dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.462+0000 7fa4a5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 0x563fc373ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.462+0000 7fa4a5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373f3a0 0x563fc3745c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.462+0000 7fa4a5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 0x563fc373ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51598/0 (socket says 172.21.15.47:51598) 2024-03-21T11:28:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.462+0000 7fa4a5e00640 1 -- 172.21.15.47:0/1154110042 learned_addr learned my addr 172.21.15.47:0/1154110042 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.462+0000 7fa4a6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373d810 0x563fc373dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44664/0 (socket says 172.21.15.47:44664) 2024-03-21T11:28:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.462+0000 7fa4a5400640 1 -- 172.21.15.47:0/1154110042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373d810 msgr2=0x563fc373dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.462+0000 7fa4a5400640 1 --2- 172.21.15.47:0/1154110042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373d810 0x563fc373dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.462+0000 7fa4a5400640 1 -- 172.21.15.47:0/1154110042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 msgr2=0x563fc373ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.462+0000 7fa4a5400640 1 --2- 172.21.15.47:0/1154110042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 0x563fc373ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.462+0000 7fa4a5400640 1 -- 172.21.15.47:0/1154110042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fc366f080 con 0x563fc373f3a0 2024-03-21T11:28:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.463+0000 7fa4a5400640 1 --2- 172.21.15.47:0/1154110042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373f3a0 0x563fc3745c30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa4940cd180 tx=0x7fa4940d3ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd4a1f10455717a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.463+0000 7fa493000640 1 -- 172.21.15.47:0/1154110042 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4940e3070 con 0x563fc373f3a0 2024-03-21T11:28:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.463+0000 7fa493000640 1 -- 172.21.15.47:0/1154110042 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4940d8d30 con 0x563fc373f3a0 2024-03-21T11:28:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.463+0000 7fa49aa00640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa4880a2f20 0x7fa4880a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.463+0000 7fa493000640 1 -- 172.21.15.47:0/1154110042 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4940dd3c0 con 0x563fc373f3a0 2024-03-21T11:28:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.463+0000 7fa49aa00640 1 -- 172.21.15.47:0/2716586527 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa4940cfa40 con 0x7fa4a0104ab0 2024-03-21T11:28:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.463+0000 7fa4a6800640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa4880a2f20 0x7fa4880a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.464+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/1154110042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373f3a0 msgr2=0x563fc3745c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.464+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/1154110042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373f3a0 0x563fc3745c30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa4940cd180 tx=0x7fa4940d3ab0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.464+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/1154110042 shutdown_connections 2024-03-21T11:28:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.464+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/1154110042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373f3a0 0x563fc3745c30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.464+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/1154110042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 0x563fc373ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.464+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/1154110042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373d810 0x563fc373dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.464+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/1154110042 >> 172.21.15.47:0/1154110042 conn(0x563fc3738c00 msgr2=0x563fc3739000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.464+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/1154110042 shutdown_connections 2024-03-21T11:28:22.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.464+0000 7fa4a6800640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa4880a2f20 0x7fa4880a53e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x563fc373e8e0 tx=0x7fa49c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.464+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/1154110042 wait complete. 2024-03-21T11:28:22.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.465+0000 7fa4ab5c2740 1 -- start start 2024-03-21T11:28:22.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.465+0000 7fa4ab5c2740 1 Processor -- start 2024-03-21T11:28:22.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.465+0000 7fa4ab5c2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373d810 0x563fc3747a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.465+0000 7fa4ab5c2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 0x563fc3747f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.465+0000 7fa4ab5c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373f3a0 0x563fc374efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.465+0000 7fa4ab5c2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc37463b0 con 0x563fc373e9e0 2024-03-21T11:28:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.465+0000 7fa4ab5c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc365e0d0 con 0x563fc373f3a0 2024-03-21T11:28:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.465+0000 7fa4ab5c2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc3746530 con 0x563fc373d810 2024-03-21T11:28:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.465+0000 7fa4ab5c2740 10 client.0 _ll_register_callbacks cb 0x563fc3662e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.466+0000 7fa4a6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373f3a0 0x563fc374efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.466+0000 7fa4a5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373d810 0x563fc3747a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.466+0000 7fa4a5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 0x563fc3747f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.466+0000 7fa4a5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373d810 0x563fc3747a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36858/0 (socket says 172.21.15.47:36858) 2024-03-21T11:28:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.466+0000 7fa4a5e00640 1 -- 172.21.15.47:0/3388471733 learned_addr learned my addr 172.21.15.47:0/3388471733 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.466+0000 7fa4a5e00640 1 -- 172.21.15.47:0/3388471733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373f3a0 msgr2=0x563fc374efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.466+0000 7fa4a5e00640 1 --2- 172.21.15.47:0/3388471733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373f3a0 0x563fc374efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.466+0000 7fa4a5e00640 1 -- 172.21.15.47:0/3388471733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 msgr2=0x563fc3747f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.466+0000 7fa4a5e00640 1 --2- 172.21.15.47:0/3388471733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 0x563fc3747f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.466+0000 7fa4a5e00640 1 -- 172.21.15.47:0/3388471733 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fc33b63f0 con 0x563fc373d810 2024-03-21T11:28:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.466+0000 7fa4a5400640 1 --2- 172.21.15.47:0/3388471733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 0x563fc3747f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.466+0000 7fa4a5e00640 1 --2- 172.21.15.47:0/3388471733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373d810 0x563fc3747a50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0349e0 tx=0x7fa48c005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa7a061c3868d426 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.466+0000 7fa487400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.467+0000 7fa487400640 1 -- 172.21.15.47:0/3388471733 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c045070 con 0x563fc373d810 2024-03-21T11:28:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.467+0000 7fa487400640 1 -- 172.21.15.47:0/3388471733 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa48c004450 con 0x563fc373d810 2024-03-21T11:28:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.467+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/3388471733 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4940d8d30 con 0x563fc373d810 2024-03-21T11:28:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.467+0000 7fa487400640 1 -- 172.21.15.47:0/3388471733 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa48c051050 con 0x563fc373d810 2024-03-21T11:28:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.467+0000 7fa4ab5c2740 10 client.27172 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.467+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/3388471733 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563fc34807d0 con 0x563fc373d810 2024-03-21T11:28:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.467+0000 7fa487400640 1 -- 172.21.15.47:0/3388471733 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4940d8d30 con 0x563fc373d810 2024-03-21T11:28:22.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.467+0000 7fa487400640 1 -- 172.21.15.47:0/3388471733 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa48c032a20 con 0x563fc373d810 2024-03-21T11:28:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.467+0000 7fa4ab5c2740 10 client.27172 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.467+0000 7fa4ab5c2740 10 client.27172 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:28:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.468+0000 7fa4ab5c2740 20 client.27172 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.468+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/3388471733 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563fc3753470 0x563fc37558c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.468+0000 7fa4ab5c2740 4 client.27172 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.468+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/3388471733 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fc36af800 con 0x563fc3753470 2024-03-21T11:28:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.468+0000 7fa4a5400640 1 --2- 172.21.15.47:0/3388471733 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563fc3753470 0x563fc37558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.468+0000 7fa4a5400640 1 --2- 172.21.15.47:0/3388471733 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563fc3753470 0x563fc37558c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b677d51a10cf688 server_cookie=2dc1b4bf5980c8e in_seq=0 out_seq=0 2024-03-21T11:28:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.468+0000 7fa487400640 10 client.27172 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:28:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.469+0000 7fa487400640 1 -- 172.21.15.47:0/3388471733 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563fc36af800 con 0x563fc3753470 2024-03-21T11:28:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.469+0000 7fa487400640 10 client.27172 handle_command_reply: tid=0 2024-03-21T11:28:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.469+0000 7fa4ab5c2740 1 client.27172 shutdown 2024-03-21T11:28:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.470+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/3388471733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373d810 msgr2=0x563fc3747a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.470+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/3388471733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373d810 0x563fc3747a50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0349e0 tx=0x7fa48c005a90 comp rx=0 tx=0).stop 2024-03-21T11:28:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.470+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/3388471733 shutdown_connections 2024-03-21T11:28:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.470+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/3388471733 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563fc3753470 0x563fc37558c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.470+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/3388471733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373f3a0 0x563fc374efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.470+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/3388471733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 0x563fc3747f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.470+0000 7fa487400640 0 client.27172 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:28:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.470+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/3388471733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373d810 0x563fc3747a50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.470+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/3388471733 >> 172.21.15.47:0/3388471733 conn(0x563fc3738c00 msgr2=0x563fc3737eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.470+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/3388471733 shutdown_connections 2024-03-21T11:28:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.470+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/3388471733 wait complete. 2024-03-21T11:28:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.470+0000 7fa4ab5c2740 20 client.27172 trim_cache size 0 max 16384 2024-03-21T11:28:22.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.486+0000 7fa4ab5c2740 1 Processor -- start 2024-03-21T11:28:22.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.486+0000 7fa4ab5c2740 1 -- start start 2024-03-21T11:28:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.486+0000 7fa4ab5c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373d810 0x563fc373c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.486+0000 7fa4ab5c2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373e9e0 0x563fc374ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.486+0000 7fa4ab5c2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373f3a0 0x563fc375a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.486+0000 7fa4ab5c2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01ac300 con 0x563fc373f3a0 2024-03-21T11:28:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.486+0000 7fa4ab5c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01adc80 con 0x563fc373d810 2024-03-21T11:28:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.486+0000 7fa4ab5c2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01ac960 con 0x563fc373e9e0 2024-03-21T11:28:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.487+0000 7fa4a5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373e9e0 0x563fc374ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.487+0000 7fa4a5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373e9e0 0x563fc374ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36864/0 (socket says 172.21.15.47:36864) 2024-03-21T11:28:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.487+0000 7fa4a5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373d810 0x563fc373c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.487+0000 7fa4a5400640 1 -- 172.21.15.47:0/3306708890 learned_addr learned my addr 172.21.15.47:0/3306708890 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.487+0000 7fa4a6800640 1 --2- 172.21.15.47:0/3306708890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373f3a0 0x563fc375a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.487+0000 7fa4a5400640 1 -- 172.21.15.47:0/3306708890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373d810 msgr2=0x563fc373c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.487+0000 7fa4a5400640 1 --2- 172.21.15.47:0/3306708890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373d810 0x563fc373c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.487+0000 7fa4a5400640 1 -- 172.21.15.47:0/3306708890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373f3a0 msgr2=0x563fc375a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.487+0000 7fa4a5400640 1 --2- 172.21.15.47:0/3306708890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373f3a0 0x563fc375a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.487+0000 7fa4a5400640 1 -- 172.21.15.47:0/3306708890 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fc3739670 con 0x563fc373e9e0 2024-03-21T11:28:22.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.487+0000 7fa4a5400640 1 --2- 172.21.15.47:0/3306708890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373e9e0 0x563fc374ef90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa494098760 tx=0x7fa494009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78c38e16fa3e9bc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.487+0000 7fa492600640 1 -- 172.21.15.47:0/3306708890 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49409bbb0 con 0x563fc373e9e0 2024-03-21T11:28:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.488+0000 7fa492600640 1 -- 172.21.15.47:0/3306708890 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa49409c4e0 con 0x563fc373e9e0 2024-03-21T11:28:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.488+0000 7fa492600640 1 -- 172.21.15.47:0/3306708890 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4940056b0 con 0x563fc373e9e0 2024-03-21T11:28:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.488+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/3306708890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373e9e0 msgr2=0x563fc374ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.488+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/3306708890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373e9e0 0x563fc374ef90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa494098760 tx=0x7fa494009f90 comp rx=0 tx=0).stop 2024-03-21T11:28:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.488+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/3306708890 shutdown_connections 2024-03-21T11:28:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.488+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/3306708890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373f3a0 0x563fc375a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.488+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/3306708890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373e9e0 0x563fc374ef90 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.488+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/3306708890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373d810 0x563fc373c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.488+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/3306708890 >> 172.21.15.47:0/3306708890 conn(0x563fc3738c00 msgr2=0x563fc3737eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.489+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/3306708890 shutdown_connections 2024-03-21T11:28:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.489+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/3306708890 wait complete. 2024-03-21T11:28:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.489+0000 7fa4ab5c2740 1 -- start start 2024-03-21T11:28:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.489+0000 7fa4ab5c2740 1 Processor -- start 2024-03-21T11:28:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4ab5c2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373d810 0x563fc374c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4ab5c2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 0x563fc374cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4ab5c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373f3a0 0x563fc374dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4ab5c2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01ad620 con 0x563fc373e9e0 2024-03-21T11:28:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4ab5c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01acfc0 con 0x563fc373f3a0 2024-03-21T11:28:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4ab5c2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01aa4c0 con 0x563fc373d810 2024-03-21T11:28:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4ab5c2740 10 client.0 _ll_register_callbacks cb 0x7fa494005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4a5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373d810 0x563fc374c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4a5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373d810 0x563fc374c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36874/0 (socket says 172.21.15.47:36874) 2024-03-21T11:28:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4a5e00640 1 -- 172.21.15.47:0/1455086533 learned_addr learned my addr 172.21.15.47:0/1455086533 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4a5400640 1 --2- 172.21.15.47:0/1455086533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 0x563fc374cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4a6800640 1 --2- 172.21.15.47:0/1455086533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373f3a0 0x563fc374dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4a5e00640 1 -- 172.21.15.47:0/1455086533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373f3a0 msgr2=0x563fc374dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4a5e00640 1 --2- 172.21.15.47:0/1455086533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373f3a0 0x563fc374dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4a5e00640 1 -- 172.21.15.47:0/1455086533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 msgr2=0x563fc374cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4a5e00640 1 --2- 172.21.15.47:0/1455086533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 0x563fc374cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4a5e00640 1 -- 172.21.15.47:0/1455086533 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fc374e240 con 0x563fc373d810 2024-03-21T11:28:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.490+0000 7fa4a6800640 1 --2- 172.21.15.47:0/1455086533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373f3a0 0x563fc374dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.491+0000 7fa4a5e00640 1 --2- 172.21.15.47:0/1455086533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373d810 0x563fc374c700 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa48c02e010 tx=0x7fa48c02c1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=893d591390531274 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.491+0000 7fa487400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.491+0000 7fa487400640 1 -- 172.21.15.47:0/1455086533 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c045040 con 0x563fc373d810 2024-03-21T11:28:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.491+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/1455086533 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa49409c4e0 con 0x563fc373d810 2024-03-21T11:28:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.491+0000 7fa487400640 1 -- 172.21.15.47:0/1455086533 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa48c032640 con 0x563fc373d810 2024-03-21T11:28:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.491+0000 7fa487400640 1 -- 172.21.15.47:0/1455086533 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa48c039040 con 0x563fc373d810 2024-03-21T11:28:22.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.491+0000 7fa4ab5c2740 10 client.27178 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.491+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/1455086533 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563fc37aee10 con 0x563fc373d810 2024-03-21T11:28:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.491+0000 7fa487400640 1 -- 172.21.15.47:0/1455086533 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa49409c4e0 con 0x563fc373d810 2024-03-21T11:28:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.492+0000 7fa487400640 1 -- 172.21.15.47:0/1455086533 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa48c003b50 con 0x563fc373d810 2024-03-21T11:28:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.492+0000 7fa4ab5c2740 10 client.27178 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.492+0000 7fa4ab5c2740 10 client.27178 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:28:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.492+0000 7fa4ab5c2740 20 client.27178 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.492+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/1455086533 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563fc37af550 0x563fc37b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.492+0000 7fa4ab5c2740 4 client.27178 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:28:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.492+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/1455086533 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563fc36a9b50 con 0x563fc37af550 2024-03-21T11:28:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.492+0000 7fa4a5400640 1 --2- 172.21.15.47:0/1455086533 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563fc37af550 0x563fc37b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.497 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:21 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2097290497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:28:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa4a5400640 1 --2- 172.21.15.47:0/1455086533 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563fc37af550 0x563fc37b1a10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=58d2f99322b400d9 server_cookie=c4c338d74687cd77 in_seq=0 out_seq=0 2024-03-21T11:28:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa487400640 10 client.27178 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:28:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa487400640 1 -- 172.21.15.47:0/1455086533 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563fc36a9b50 con 0x563fc37af550 2024-03-21T11:28:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa487400640 10 client.27178 handle_command_reply: tid=0 2024-03-21T11:28:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa4ab5c2740 1 client.27178 shutdown 2024-03-21T11:28:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/1455086533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373d810 msgr2=0x563fc374c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/1455086533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373d810 0x563fc374c700 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa48c02e010 tx=0x7fa48c02c1f0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/1455086533 shutdown_connections 2024-03-21T11:28:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/1455086533 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563fc37af550 0x563fc37b1a10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/1455086533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563fc373f3a0 0x563fc374dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/1455086533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563fc373e9e0 0x563fc374cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa4ab5c2740 1 --2- 172.21.15.47:0/1455086533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc373d810 0x563fc374c700 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa487400640 0 client.27178 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:28:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/1455086533 >> 172.21.15.47:0/1455086533 conn(0x563fc3738c00 msgr2=0x563fc3737620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/1455086533 shutdown_connections 2024-03-21T11:28:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.493+0000 7fa4ab5c2740 1 -- 172.21.15.47:0/1455086533 wait complete. 2024-03-21T11:28:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.494+0000 7fa4ab5c2740 20 client.27178 trim_cache size 0 max 16384 2024-03-21T11:28:22.500 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:28:22.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.494+0000 7fa4a8800640 1 -- 172.21.15.47:0/2716586527 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa4880a2f20 msgr2=0x7fa4880a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.494+0000 7fa4a8800640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa4880a2f20 0x7fa4880a53e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x563fc373e8e0 tx=0x7fa49c009500 comp rx=0 tx=0).stop 2024-03-21T11:28:22.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.494+0000 7fa4a8800640 1 -- 172.21.15.47:0/2716586527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4a0104ab0 msgr2=0x7fa4a019a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.494+0000 7fa4a8800640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4a0104ab0 0x7fa4a019a0f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa49400e9c0 tx=0x7fa49400ee80 comp rx=0 tx=0).stop 2024-03-21T11:28:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.494+0000 7fa4a8800640 1 -- 172.21.15.47:0/2716586527 shutdown_connections 2024-03-21T11:28:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.494+0000 7fa4a8800640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa4880a2f20 0x7fa4880a53e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.494+0000 7fa4a8800640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4a0105470 0x7fa4a01a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.494+0000 7fa4a8800640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4a0104ab0 0x7fa4a019a0f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.494+0000 7fa4a8800640 1 --2- 172.21.15.47:0/2716586527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4a01038c0 0x7fa4a0199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.494+0000 7fa4a8800640 1 -- 172.21.15.47:0/2716586527 >> 172.21.15.47:0/2716586527 conn(0x7fa4a00fee50 msgr2=0x7fa4a0107e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.495+0000 7fa4a8800640 1 -- 172.21.15.47:0/2716586527 shutdown_connections 2024-03-21T11:28:22.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.495+0000 7fa4a8800640 1 -- 172.21.15.47:0/2716586527 wait complete. 2024-03-21T11:28:22.510 INFO:tasks.workunit.client.0.smithi047.stdout: 1 298093 8.41 MB/sec execute 66 sec latency 44.987 ms 2024-03-21T11:28:22.513 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:28:22.513 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:28:22.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.728+0000 7f2b06000640 1 -- 172.21.15.47:0/1767243806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b00109450 msgr2=0x7f2b0010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.728+0000 7f2b06000640 1 --2- 172.21.15.47:0/1767243806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b00109450 0x7f2b0010b8e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009ca0 tx=0x7f2af802c180 comp rx=0 tx=0).stop 2024-03-21T11:28:22.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.728+0000 7f2b06000640 1 -- 172.21.15.47:0/1767243806 shutdown_connections 2024-03-21T11:28:22.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.728+0000 7f2b06000640 1 --2- 172.21.15.47:0/1767243806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b00109450 0x7f2b0010b8e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.728+0000 7f2b06000640 1 --2- 172.21.15.47:0/1767243806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b00074080 0x7f2b000744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.728+0000 7f2b06000640 1 --2- 172.21.15.47:0/1767243806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00075660 0x7f2b00073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.728+0000 7f2b06000640 1 -- 172.21.15.47:0/1767243806 >> 172.21.15.47:0/1767243806 conn(0x7f2b000fcb10 msgr2=0x7f2b000fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.728+0000 7f2b06000640 1 -- 172.21.15.47:0/1767243806 shutdown_connections 2024-03-21T11:28:22.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.728+0000 7f2b06000640 1 -- 172.21.15.47:0/1767243806 wait complete. 2024-03-21T11:28:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.730+0000 7f2b06000640 1 Processor -- start 2024-03-21T11:28:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.730+0000 7f2b06000640 1 -- start start 2024-03-21T11:28:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.730+0000 7f2b06000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b00074080 0x7f2b00199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.730+0000 7f2b06000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00075660 0x7f2b00199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.730+0000 7f2b06000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b00109450 0x7f2b001a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.730+0000 7f2b06000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0010f410 con 0x7f2b00075660 2024-03-21T11:28:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.730+0000 7f2b06000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0010f290 con 0x7f2b00109450 2024-03-21T11:28:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.730+0000 7f2b06000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0010f590 con 0x7f2b00074080 2024-03-21T11:28:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.730+0000 7f2b04c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b00074080 0x7f2b00199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.730+0000 7f2b04c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b00074080 0x7f2b00199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36898/0 (socket says 172.21.15.47:36898) 2024-03-21T11:28:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.730+0000 7f2b04c00640 1 -- 172.21.15.47:0/1828222168 learned_addr learned my addr 172.21.15.47:0/1828222168 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.730+0000 7f2b05600640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b00109450 0x7f2b001a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.730+0000 7f2affe00640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00075660 0x7f2b00199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.731+0000 7f2b04c00640 1 -- 172.21.15.47:0/1828222168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b00109450 msgr2=0x7f2b001a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.731+0000 7f2b04c00640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b00109450 0x7f2b001a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.731+0000 7f2b04c00640 1 -- 172.21.15.47:0/1828222168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00075660 msgr2=0x7f2b00199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.731+0000 7f2b04c00640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00075660 0x7f2b00199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.731+0000 7f2b04c00640 1 -- 172.21.15.47:0/1828222168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b001a1620 con 0x7f2b00074080 2024-03-21T11:28:22.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.731+0000 7f2b05600640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b00109450 0x7f2b001a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:22.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.731+0000 7f2b04c00640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b00074080 0x7f2b00199a00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2af400e9f0 tx=0x7f2af400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.732+0000 7f2afd600640 1 -- 172.21.15.47:0/1828222168 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af400cd80 con 0x7f2b00074080 2024-03-21T11:28:22.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.732+0000 7f2afd600640 1 -- 172.21.15.47:0/1828222168 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2af4010910 con 0x7f2b00074080 2024-03-21T11:28:22.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.732+0000 7f2afd600640 1 -- 172.21.15.47:0/1828222168 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af4018990 con 0x7f2b00074080 2024-03-21T11:28:22.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.732+0000 7f2b06000640 1 -- 172.21.15.47:0/1828222168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b001a1910 con 0x7f2b00074080 2024-03-21T11:28:22.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.732+0000 7f2b06000640 1 -- 172.21.15.47:0/1828222168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b001a1eb0 con 0x7f2b00074080 2024-03-21T11:28:22.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.735+0000 7f2b0a3ae740 1 Processor -- start 2024-03-21T11:28:22.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.735+0000 7f2afd600640 1 -- 172.21.15.47:0/1828222168 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2af4010ac0 con 0x7f2b00074080 2024-03-21T11:28:22.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.735+0000 7f2b0a3ae740 1 -- start start 2024-03-21T11:28:22.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.735+0000 7f2b0a3ae740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550dc9f0 0x555f550dce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.735+0000 7f2b0a3ae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550ddbc0 0x555f550de040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.736+0000 7f2b0a3ae740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550de580 0x555f550e4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.736+0000 7f2b0a3ae740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f550412c0 con 0x555f550dc9f0 2024-03-21T11:28:22.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.736+0000 7f2b0a3ae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f54fe9c90 con 0x555f550ddbc0 2024-03-21T11:28:22.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.736+0000 7f2b0a3ae740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5503d360 con 0x555f550de580 2024-03-21T11:28:22.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.736+0000 7f2b04c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550ddbc0 0x555f550de040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.736+0000 7f2affe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550de580 0x555f550e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.736+0000 7f2b04c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550ddbc0 0x555f550de040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44726/0 (socket says 172.21.15.47:44726) 2024-03-21T11:28:22.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.736+0000 7f2b04c00640 1 -- 172.21.15.47:0/3791935254 learned_addr learned my addr 172.21.15.47:0/3791935254 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.736+0000 7f2b05600640 1 --2- 172.21.15.47:0/3791935254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550dc9f0 0x555f550dce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.736+0000 7f2b04c00640 1 -- 172.21.15.47:0/3791935254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550de580 msgr2=0x555f550e4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.736+0000 7f2b04c00640 1 --2- 172.21.15.47:0/3791935254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550de580 0x555f550e4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.736+0000 7f2b04c00640 1 -- 172.21.15.47:0/3791935254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550dc9f0 msgr2=0x555f550dce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.736+0000 7f2b04c00640 1 --2- 172.21.15.47:0/3791935254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550dc9f0 0x555f550dce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.736+0000 7f2b04c00640 1 -- 172.21.15.47:0/3791935254 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f5500e080 con 0x555f550ddbc0 2024-03-21T11:28:22.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.737+0000 7f2b04c00640 1 --2- 172.21.15.47:0/3791935254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550ddbc0 0x555f550de040 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2af40d5bd0 tx=0x7f2af40d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae5f15e50fb3433f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.737+0000 7f2afd600640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2ae80a2fa0 0x7f2ae80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.737+0000 7f2af1c00640 1 -- 172.21.15.47:0/3791935254 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af40de070 con 0x555f550ddbc0 2024-03-21T11:28:22.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.737+0000 7f2b05600640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2ae80a2fa0 0x7f2ae80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.737+0000 7f2afd600640 1 -- 172.21.15.47:0/1828222168 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2af40cfa40 con 0x7f2b00074080 2024-03-21T11:28:22.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.737+0000 7f2af1c00640 1 -- 172.21.15.47:0/3791935254 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2af409be30 con 0x555f550ddbc0 2024-03-21T11:28:22.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.737+0000 7f2af1c00640 1 -- 172.21.15.47:0/3791935254 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af409b970 con 0x555f550ddbc0 2024-03-21T11:28:22.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.738+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/3791935254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550ddbc0 msgr2=0x555f550de040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.738+0000 7f2b05600640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2ae80a2fa0 0x7f2ae80a5460 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2af802c6c0 tx=0x7f2af8032670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.738+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/3791935254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550ddbc0 0x555f550de040 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2af40d5bd0 tx=0x7f2af40d7670 comp rx=0 tx=0).stop 2024-03-21T11:28:22.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.738+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/3791935254 shutdown_connections 2024-03-21T11:28:22.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.738+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/3791935254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550de580 0x555f550e4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.738+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/3791935254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550ddbc0 0x555f550de040 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.738+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/3791935254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550dc9f0 0x555f550dce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.738+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/3791935254 >> 172.21.15.47:0/3791935254 conn(0x555f550d7de0 msgr2=0x555f550d81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.738+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/3791935254 shutdown_connections 2024-03-21T11:28:22.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.738+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/3791935254 wait complete. 2024-03-21T11:28:22.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.739+0000 7f2b0a3ae740 1 -- start start 2024-03-21T11:28:22.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.739+0000 7f2b0a3ae740 1 Processor -- start 2024-03-21T11:28:22.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b0a3ae740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550dc9f0 0x555f550e6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b0a3ae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550ddbc0 0x555f550e7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b0a3ae740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550de580 0x555f550ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b0a3ae740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f550e5590 con 0x555f550de580 2024-03-21T11:28:22.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b0a3ae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f54ffd0d0 con 0x555f550ddbc0 2024-03-21T11:28:22.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b0a3ae740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f550e5710 con 0x555f550dc9f0 2024-03-21T11:28:22.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b0a3ae740 10 client.0 _ll_register_callbacks cb 0x555f55001e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:22.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b04c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550dc9f0 0x555f550e6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2affe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550ddbc0 0x555f550e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b04c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550dc9f0 0x555f550e6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36916/0 (socket says 172.21.15.47:36916) 2024-03-21T11:28:22.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b04c00640 1 -- 172.21.15.47:0/3129429776 learned_addr learned my addr 172.21.15.47:0/3129429776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2affe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550ddbc0 0x555f550e7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44740/0 (socket says 172.21.15.47:44740) 2024-03-21T11:28:22.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b05600640 1 --2- 172.21.15.47:0/3129429776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550de580 0x555f550ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b04c00640 1 -- 172.21.15.47:0/3129429776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550ddbc0 msgr2=0x555f550e7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b04c00640 1 --2- 172.21.15.47:0/3129429776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550ddbc0 0x555f550e7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b04c00640 1 -- 172.21.15.47:0/3129429776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550de580 msgr2=0x555f550ee1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b04c00640 1 --2- 172.21.15.47:0/3129429776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550de580 0x555f550ee1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.740+0000 7f2b04c00640 1 -- 172.21.15.47:0/3129429776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f54d553f0 con 0x555f550dc9f0 2024-03-21T11:28:22.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.741+0000 7f2b04c00640 1 --2- 172.21.15.47:0/3129429776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550dc9f0 0x555f550e6c50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2af4098240 tx=0x7f2af4018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fe9d4a001660ab2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.741+0000 7f2ae6000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:22.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.741+0000 7f2ae6000640 1 -- 172.21.15.47:0/3129429776 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af40de030 con 0x555f550dc9f0 2024-03-21T11:28:22.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.741+0000 7f2ae6000640 1 -- 172.21.15.47:0/3129429776 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2af409c090 con 0x555f550dc9f0 2024-03-21T11:28:22.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.741+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/3129429776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2af409be30 con 0x555f550dc9f0 2024-03-21T11:28:22.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.741+0000 7f2ae6000640 1 -- 172.21.15.47:0/3129429776 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2af409b790 con 0x555f550dc9f0 2024-03-21T11:28:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.741+0000 7f2b0a3ae740 10 client.27190 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.741+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/3129429776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555f54e1f7d0 con 0x555f550dc9f0 2024-03-21T11:28:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.741+0000 7f2ae6000640 1 -- 172.21.15.47:0/3129429776 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2af409be30 con 0x555f550dc9f0 2024-03-21T11:28:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.741+0000 7f2ae6000640 1 -- 172.21.15.47:0/3129429776 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2af40a4300 con 0x555f550dc9f0 2024-03-21T11:28:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.742+0000 7f2b0a3ae740 10 client.27190 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.742+0000 7f2b0a3ae740 10 client.27190 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:28:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.742+0000 7f2b0a3ae740 20 client.27190 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.742+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/3129429776 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x555f550f2330 0x555f550f4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.742+0000 7f2b0a3ae740 4 client.27190 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.742+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/3129429776 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f5504e800 con 0x555f550f2330 2024-03-21T11:28:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.742+0000 7f2affe00640 1 --2- 172.21.15.47:0/3129429776 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x555f550f2330 0x555f550f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.743+0000 7f2affe00640 1 --2- 172.21.15.47:0/3129429776 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x555f550f2330 0x555f550f4720 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14cd87cbafe6fbf6 server_cookie=cda1294ee8bcc140 in_seq=0 out_seq=0 2024-03-21T11:28:22.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.743+0000 7f2ae6000640 10 client.27190 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:28:22.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2ae6000640 1 -- 172.21.15.47:0/3129429776 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555f5504e800 con 0x555f550f2330 2024-03-21T11:28:22.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2ae6000640 10 client.27190 handle_command_reply: tid=0 2024-03-21T11:28:22.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2b0a3ae740 1 client.27190 shutdown 2024-03-21T11:28:22.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/3129429776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550dc9f0 msgr2=0x555f550e6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/3129429776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550dc9f0 0x555f550e6c50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2af4098240 tx=0x7f2af4018420 comp rx=0 tx=0).stop 2024-03-21T11:28:22.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/3129429776 shutdown_connections 2024-03-21T11:28:22.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/3129429776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550de580 0x555f550ee1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/3129429776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550ddbc0 0x555f550e7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/3129429776 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x555f550f2330 0x555f550f4720 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/3129429776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550dc9f0 0x555f550e6c50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/3129429776 >> 172.21.15.47:0/3129429776 conn(0x555f550d7de0 msgr2=0x555f550d7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2ae6000640 0 client.27190 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:28:22.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/3129429776 shutdown_connections 2024-03-21T11:28:22.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/3129429776 wait complete. 2024-03-21T11:28:22.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.744+0000 7f2b0a3ae740 20 client.27190 trim_cache size 0 max 16384 2024-03-21T11:28:22.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.758+0000 7f2b0a3ae740 1 Processor -- start 2024-03-21T11:28:22.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.758+0000 7f2b0a3ae740 1 -- start start 2024-03-21T11:28:22.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2b0a3ae740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550dc9f0 0x555f550e70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2b0a3ae740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550ddbc0 0x555f550db750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2b0a3ae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550de580 0x555f550f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2b0a3ae740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b001ac0a0 con 0x555f550dc9f0 2024-03-21T11:28:22.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2b0a3ae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b001ada20 con 0x555f550de580 2024-03-21T11:28:22.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2b0a3ae740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b001ac700 con 0x555f550ddbc0 2024-03-21T11:28:22.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2b04c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550dc9f0 0x555f550e70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2affe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550ddbc0 0x555f550db750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2affe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550ddbc0 0x555f550db750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36918/0 (socket says 172.21.15.47:36918) 2024-03-21T11:28:22.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2b04c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550dc9f0 0x555f550e70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51666/0 (socket says 172.21.15.47:51666) 2024-03-21T11:28:22.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2affe00640 1 -- 172.21.15.47:0/2488481237 learned_addr learned my addr 172.21.15.47:0/2488481237 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2b05600640 1 --2- 172.21.15.47:0/2488481237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550de580 0x555f550f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2affe00640 1 -- 172.21.15.47:0/2488481237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550de580 msgr2=0x555f550f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2affe00640 1 --2- 172.21.15.47:0/2488481237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550de580 0x555f550f8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2affe00640 1 -- 172.21.15.47:0/2488481237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550dc9f0 msgr2=0x555f550e70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2affe00640 1 --2- 172.21.15.47:0/2488481237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550dc9f0 0x555f550e70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.759+0000 7f2affe00640 1 -- 172.21.15.47:0/2488481237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f550d38b0 con 0x555f550ddbc0 2024-03-21T11:28:22.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.760+0000 7f2affe00640 1 --2- 172.21.15.47:0/2488481237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550ddbc0 0x555f550db750 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009d70 tx=0x7f2aec005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18c0bdf848ed6e60 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.760+0000 7f2af1200640 1 -- 172.21.15.47:0/2488481237 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec024070 con 0x555f550ddbc0 2024-03-21T11:28:22.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.760+0000 7f2af1200640 1 -- 172.21.15.47:0/2488481237 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2aec01f230 con 0x555f550ddbc0 2024-03-21T11:28:22.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.760+0000 7f2af1200640 1 -- 172.21.15.47:0/2488481237 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0035f0 con 0x555f550ddbc0 2024-03-21T11:28:22.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.761+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/2488481237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550ddbc0 msgr2=0x555f550db750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.761+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/2488481237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550ddbc0 0x555f550db750 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009d70 tx=0x7f2aec005f90 comp rx=0 tx=0).stop 2024-03-21T11:28:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.761+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/2488481237 shutdown_connections 2024-03-21T11:28:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.761+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/2488481237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550de580 0x555f550f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.761+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/2488481237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550ddbc0 0x555f550db750 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.761+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/2488481237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550dc9f0 0x555f550e70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.761+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/2488481237 >> 172.21.15.47:0/2488481237 conn(0x555f550d7de0 msgr2=0x555f550db040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.761+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/2488481237 shutdown_connections 2024-03-21T11:28:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.761+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/2488481237 wait complete. 2024-03-21T11:28:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.761+0000 7f2b0a3ae740 1 -- start start 2024-03-21T11:28:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.761+0000 7f2b0a3ae740 1 Processor -- start 2024-03-21T11:28:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.762+0000 7f2b0a3ae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550dc9f0 0x555f550e9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.762+0000 7f2b0a3ae740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550ddbc0 0x555f550e9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.762+0000 7f2b0a3ae740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550de580 0x555f550ecad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.762+0000 7f2b0a3ae740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b001ad3c0 con 0x555f550ddbc0 2024-03-21T11:28:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.762+0000 7f2b0a3ae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b001acd60 con 0x555f550dc9f0 2024-03-21T11:28:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.762+0000 7f2b0a3ae740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b001aa260 con 0x555f550de580 2024-03-21T11:28:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.762+0000 7f2b0a3ae740 10 client.0 _ll_register_callbacks cb 0x7f2af4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.762+0000 7f2b04c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550dc9f0 0x555f550e9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.762+0000 7f2b05600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550de580 0x555f550ecad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.762+0000 7f2b04c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550dc9f0 0x555f550e9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44758/0 (socket says 172.21.15.47:44758) 2024-03-21T11:28:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.762+0000 7f2b05600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550de580 0x555f550ecad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36924/0 (socket says 172.21.15.47:36924) 2024-03-21T11:28:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.762+0000 7f2b04c00640 1 -- 172.21.15.47:0/1176866774 learned_addr learned my addr 172.21.15.47:0/1176866774 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.762+0000 7f2affe00640 1 --2- 172.21.15.47:0/1176866774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550ddbc0 0x555f550e9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.763+0000 7f2b04c00640 1 -- 172.21.15.47:0/1176866774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550de580 msgr2=0x555f550ecad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.763+0000 7f2b04c00640 1 --2- 172.21.15.47:0/1176866774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550de580 0x555f550ecad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.763+0000 7f2b04c00640 1 -- 172.21.15.47:0/1176866774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550ddbc0 msgr2=0x555f550e9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.763+0000 7f2b04c00640 1 --2- 172.21.15.47:0/1176866774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550ddbc0 0x555f550e9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.763+0000 7f2b04c00640 1 -- 172.21.15.47:0/1176866774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f550ed0c0 con 0x555f550dc9f0 2024-03-21T11:28:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.763+0000 7f2b05600640 1 --2- 172.21.15.47:0/1176866774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550de580 0x555f550ecad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.763+0000 7f2b04c00640 1 --2- 172.21.15.47:0/1176866774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550dc9f0 0x555f550e9570 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2af4098290 tx=0x7f2af40a45b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c39b579388bb2445 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.763+0000 7f2ae6000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.763+0000 7f2ae6000640 1 -- 172.21.15.47:0/1176866774 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af40de070 con 0x555f550dc9f0 2024-03-21T11:28:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.763+0000 7f2ae6000640 1 -- 172.21.15.47:0/1176866774 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2af40d5430 con 0x555f550dc9f0 2024-03-21T11:28:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.763+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/1176866774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2aec01f230 con 0x555f550dc9f0 2024-03-21T11:28:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.764+0000 7f2ae6000640 1 -- 172.21.15.47:0/1176866774 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2af409b970 con 0x555f550dc9f0 2024-03-21T11:28:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.764+0000 7f2b0a3ae740 10 client.27176 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.764+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/1176866774 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555f550ed0c0 con 0x555f550dc9f0 2024-03-21T11:28:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.764+0000 7f2ae6000640 1 -- 172.21.15.47:0/1176866774 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2aec01f230 con 0x555f550dc9f0 2024-03-21T11:28:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.764+0000 7f2ae6000640 1 -- 172.21.15.47:0/1176866774 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2af409b280 con 0x555f550dc9f0 2024-03-21T11:28:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.764+0000 7f2b0a3ae740 10 client.27176 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.764+0000 7f2b0a3ae740 10 client.27176 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:28:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.764+0000 7f2b0a3ae740 20 client.27176 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.764+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/1176866774 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x555f5514e2f0 0x555f55150710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.765+0000 7f2b0a3ae740 4 client.27176 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:28:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.765+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/1176866774 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555f55048b50 con 0x555f5514e2f0 2024-03-21T11:28:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.765+0000 7f2affe00640 1 --2- 172.21.15.47:0/1176866774 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x555f5514e2f0 0x555f55150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.765+0000 7f2affe00640 1 --2- 172.21.15.47:0/1176866774 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x555f5514e2f0 0x555f55150710 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed9bf166fb9f42a3 server_cookie=c7038516542216fc in_seq=0 out_seq=0 2024-03-21T11:28:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.765+0000 7f2ae6000640 10 client.27176 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:28:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.765+0000 7f2ae6000640 1 -- 172.21.15.47:0/1176866774 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555f55048b50 con 0x555f5514e2f0 2024-03-21T11:28:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.765+0000 7f2ae6000640 10 client.27176 handle_command_reply: tid=0 2024-03-21T11:28:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.765+0000 7f2b0a3ae740 1 client.27176 shutdown 2024-03-21T11:28:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.766+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/1176866774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550dc9f0 msgr2=0x555f550e9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.766+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/1176866774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550dc9f0 0x555f550e9570 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2af4098290 tx=0x7f2af40a45b0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.766+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/1176866774 shutdown_connections 2024-03-21T11:28:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.766+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/1176866774 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x555f5514e2f0 0x555f55150710 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.766+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/1176866774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f550de580 0x555f550ecad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.766+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/1176866774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f550ddbc0 0x555f550e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.766+0000 7f2ae6000640 0 client.27176 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:28:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.766+0000 7f2b0a3ae740 1 --2- 172.21.15.47:0/1176866774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f550dc9f0 0x555f550e9570 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.766+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/1176866774 >> 172.21.15.47:0/1176866774 conn(0x555f550d7de0 msgr2=0x555f550e6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.766+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/1176866774 shutdown_connections 2024-03-21T11:28:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.766+0000 7f2b0a3ae740 1 -- 172.21.15.47:0/1176866774 wait complete. 2024-03-21T11:28:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.766+0000 7f2b0a3ae740 20 client.27176 trim_cache size 0 max 16384 2024-03-21T11:28:22.771 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:28:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.767+0000 7f2b06000640 1 -- 172.21.15.47:0/1828222168 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2ae80a2fa0 msgr2=0x7f2ae80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.767+0000 7f2b06000640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2ae80a2fa0 0x7f2ae80a5460 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2af802c6c0 tx=0x7f2af8032670 comp rx=0 tx=0).stop 2024-03-21T11:28:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.767+0000 7f2b06000640 1 -- 172.21.15.47:0/1828222168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b00074080 msgr2=0x7f2b00199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.767+0000 7f2b06000640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b00074080 0x7f2b00199a00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2af400e9f0 tx=0x7f2af400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.767+0000 7f2b06000640 1 -- 172.21.15.47:0/1828222168 shutdown_connections 2024-03-21T11:28:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.767+0000 7f2b06000640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2ae80a2fa0 0x7f2ae80a5460 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.767+0000 7f2b06000640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b00109450 0x7f2b001a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.767+0000 7f2b06000640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2b00075660 0x7f2b00199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.767+0000 7f2b06000640 1 --2- 172.21.15.47:0/1828222168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b00074080 0x7f2b00199a00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.767+0000 7f2b06000640 1 -- 172.21.15.47:0/1828222168 >> 172.21.15.47:0/1828222168 conn(0x7f2b000fcb10 msgr2=0x7f2b001053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.767+0000 7f2b06000640 1 -- 172.21.15.47:0/1828222168 shutdown_connections 2024-03-21T11:28:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:22.767+0000 7f2b06000640 1 -- 172.21.15.47:0/1828222168 wait complete. 2024-03-21T11:28:22.788 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:28:22.789 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:28:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.014+0000 7f737a200640 1 -- 172.21.15.47:0/1211561347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7374107cc0 msgr2=0x7f73741080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.014+0000 7f737a200640 1 --2- 172.21.15.47:0/1211561347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7374107cc0 0x7f73741080e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7364009900 tx=0x7f736402bc40 comp rx=0 tx=0).stop 2024-03-21T11:28:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.014+0000 7f737a200640 1 -- 172.21.15.47:0/1211561347 shutdown_connections 2024-03-21T11:28:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.014+0000 7f737a200640 1 --2- 172.21.15.47:0/1211561347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7374109870 0x7f7374110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.014+0000 7f737a200640 1 --2- 172.21.15.47:0/1211561347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7374108eb0 0x7f7374109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.014+0000 7f737a200640 1 --2- 172.21.15.47:0/1211561347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7374107cc0 0x7f73741080e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.014+0000 7f737a200640 1 -- 172.21.15.47:0/1211561347 >> 172.21.15.47:0/1211561347 conn(0x7f7374076c60 msgr2=0x7f73740790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.015+0000 7f737a200640 1 -- 172.21.15.47:0/1211561347 shutdown_connections 2024-03-21T11:28:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.015+0000 7f737a200640 1 -- 172.21.15.47:0/1211561347 wait complete. 2024-03-21T11:28:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.016+0000 7f737a200640 1 Processor -- start 2024-03-21T11:28:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.016+0000 7f737a200640 1 -- start start 2024-03-21T11:28:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.016+0000 7f737a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7374108eb0 0x7f737419e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.016+0000 7f737a200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7374109870 0x7f737419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.016+0000 7f737a200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73741a3770 0x7f73741a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.016+0000 7f737a200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7374113910 con 0x7f7374108eb0 2024-03-21T11:28:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.016+0000 7f737a200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7374113790 con 0x7f7374109870 2024-03-21T11:28:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.016+0000 7f737a200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7374113a90 con 0x7f73741a3770 2024-03-21T11:28:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.016+0000 7f7373400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7374108eb0 0x7f737419e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.016+0000 7f7373e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73741a3770 0x7f73741a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.016+0000 7f7373400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7374108eb0 0x7f737419e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51710/0 (socket says 172.21.15.47:51710) 2024-03-21T11:28:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.016+0000 7f736aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7374109870 0x7f737419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.016+0000 7f7373400640 1 -- 172.21.15.47:0/2808163781 learned_addr learned my addr 172.21.15.47:0/2808163781 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.017+0000 7f7373e00640 1 -- 172.21.15.47:0/2808163781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7374109870 msgr2=0x7f737419e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.017+0000 7f7373e00640 1 --2- 172.21.15.47:0/2808163781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7374109870 0x7f737419e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.017+0000 7f7373e00640 1 -- 172.21.15.47:0/2808163781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7374108eb0 msgr2=0x7f737419e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.017+0000 7f7373e00640 1 --2- 172.21.15.47:0/2808163781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7374108eb0 0x7f737419e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.017+0000 7f7373e00640 1 -- 172.21.15.47:0/2808163781 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73741a61d0 con 0x7f73741a3770 2024-03-21T11:28:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.017+0000 7f7373400640 1 --2- 172.21.15.47:0/2808163781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7374108eb0 0x7f737419e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.017+0000 7f7373e00640 1 --2- 172.21.15.47:0/2808163781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73741a3770 0x7f73741a5b60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f736c00e970 tx=0x7f736c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.017+0000 7f7370c00640 1 -- 172.21.15.47:0/2808163781 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c00ccc0 con 0x7f73741a3770 2024-03-21T11:28:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.018+0000 7f7370c00640 1 -- 172.21.15.47:0/2808163781 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f736c005020 con 0x7f73741a3770 2024-03-21T11:28:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.018+0000 7f737a200640 1 -- 172.21.15.47:0/2808163781 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73741a64c0 con 0x7f73741a3770 2024-03-21T11:28:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.018+0000 7f7370c00640 1 -- 172.21.15.47:0/2808163781 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c004a00 con 0x7f73741a3770 2024-03-21T11:28:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.018+0000 7f737a200640 1 -- 172.21.15.47:0/2808163781 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73741a6a60 con 0x7f73741a3770 2024-03-21T11:28:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.021+0000 7f7370c00640 1 -- 172.21.15.47:0/2808163781 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f736c0051d0 con 0x7f73741a3770 2024-03-21T11:28:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.021+0000 7f737cfb3740 1 Processor -- start 2024-03-21T11:28:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.021+0000 7f737cfb3740 1 -- start start 2024-03-21T11:28:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.021+0000 7f737cfb3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b00810 0x55a2b7b00c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.021+0000 7f737cfb3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b019e0 0x55a2b7b01e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.021+0000 7f737cfb3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b023a0 0x55a2b7b08c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.021+0000 7f737cfb3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b7a652c0 con 0x55a2b7b023a0 2024-03-21T11:28:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.021+0000 7f737cfb3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b7a0dc90 con 0x55a2b7b00810 2024-03-21T11:28:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.021+0000 7f737cfb3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b7a61360 con 0x55a2b7b019e0 2024-03-21T11:28:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.022+0000 7f736aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b00810 0x55a2b7b00c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.022+0000 7f7373400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b019e0 0x55a2b7b01e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.022+0000 7f7373400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b019e0 0x55a2b7b01e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36942/0 (socket says 172.21.15.47:36942) 2024-03-21T11:28:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.022+0000 7f7373400640 1 -- 172.21.15.47:0/108923828 learned_addr learned my addr 172.21.15.47:0/108923828 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.022+0000 7f736aa00640 1 --2- 172.21.15.47:0/108923828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b023a0 0x55a2b7b08c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.022+0000 7f7373400640 1 -- 172.21.15.47:0/108923828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b00810 msgr2=0x55a2b7b00c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.022+0000 7f7373400640 1 --2- 172.21.15.47:0/108923828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b00810 0x55a2b7b00c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.022+0000 7f7373400640 1 -- 172.21.15.47:0/108923828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b023a0 msgr2=0x55a2b7b08c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.022+0000 7f7373400640 1 --2- 172.21.15.47:0/108923828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b023a0 0x55a2b7b08c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.022+0000 7f7373400640 1 -- 172.21.15.47:0/108923828 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2b7a32080 con 0x55a2b7b019e0 2024-03-21T11:28:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.022+0000 7f7370c00640 1 --2- 172.21.15.47:0/2808163781 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73580a2fa0 0x7f73580a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.023+0000 7f7373400640 1 --2- 172.21.15.47:0/108923828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b019e0 0x55a2b7b01e60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f736402c7c0 tx=0x7f736402cd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b5e421f25159955 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.023+0000 7f7370c00640 1 -- 172.21.15.47:0/2808163781 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f736c0cf670 con 0x7f73741a3770 2024-03-21T11:28:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.023+0000 7f736aa00640 1 --2- 172.21.15.47:0/2808163781 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73580a2fa0 0x7f73580a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.023+0000 7f7363e00640 1 -- 172.21.15.47:0/108923828 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364042070 con 0x55a2b7b019e0 2024-03-21T11:28:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.023+0000 7f7363e00640 1 -- 172.21.15.47:0/108923828 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7364004ba0 con 0x55a2b7b019e0 2024-03-21T11:28:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.023+0000 7f7363e00640 1 -- 172.21.15.47:0/108923828 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364005150 con 0x55a2b7b019e0 2024-03-21T11:28:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.024+0000 7f737cfb3740 1 -- 172.21.15.47:0/108923828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b019e0 msgr2=0x55a2b7b01e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.024+0000 7f737cfb3740 1 --2- 172.21.15.47:0/108923828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b019e0 0x55a2b7b01e60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f736402c7c0 tx=0x7f736402cd40 comp rx=0 tx=0).stop 2024-03-21T11:28:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.024+0000 7f736aa00640 1 --2- 172.21.15.47:0/2808163781 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73580a2fa0 0x7f73580a5460 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x55a2b7b018e0 tx=0x7f735c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.024+0000 7f737cfb3740 1 -- 172.21.15.47:0/108923828 shutdown_connections 2024-03-21T11:28:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.024+0000 7f737cfb3740 1 --2- 172.21.15.47:0/108923828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b023a0 0x55a2b7b08c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.024+0000 7f737cfb3740 1 --2- 172.21.15.47:0/108923828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b019e0 0x55a2b7b01e60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.024+0000 7f737cfb3740 1 --2- 172.21.15.47:0/108923828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b00810 0x55a2b7b00c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.024+0000 7f737cfb3740 1 -- 172.21.15.47:0/108923828 >> 172.21.15.47:0/108923828 conn(0x55a2b7afbc00 msgr2=0x55a2b7afc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.024+0000 7f737cfb3740 1 -- 172.21.15.47:0/108923828 shutdown_connections 2024-03-21T11:28:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.024+0000 7f737cfb3740 1 -- 172.21.15.47:0/108923828 wait complete. 2024-03-21T11:28:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.024+0000 7f737cfb3740 1 -- start start 2024-03-21T11:28:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.025+0000 7f737cfb3740 1 Processor -- start 2024-03-21T11:28:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.025+0000 7f737cfb3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b00810 0x55a2b7b0a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.025+0000 7f737cfb3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b019e0 0x55a2b7b0aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.025+0000 7f737cfb3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b023a0 0x55a2b7b11f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.025+0000 7f737cfb3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b7b09350 con 0x55a2b7b00810 2024-03-21T11:28:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.025+0000 7f737cfb3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b7a210d0 con 0x55a2b7b023a0 2024-03-21T11:28:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.025+0000 7f737cfb3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b7b094d0 con 0x55a2b7b019e0 2024-03-21T11:28:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.025+0000 7f737cfb3740 10 client.0 _ll_register_callbacks cb 0x55a2b7a25e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.025+0000 7f736aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b019e0 0x55a2b7b0aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.025+0000 7f7373400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b00810 0x55a2b7b0a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.025+0000 7f736aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b019e0 0x55a2b7b0aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36958/0 (socket says 172.21.15.47:36958) 2024-03-21T11:28:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.025+0000 7f736aa00640 1 -- 172.21.15.47:0/294233502 learned_addr learned my addr 172.21.15.47:0/294233502 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.025+0000 7f7373e00640 1 --2- 172.21.15.47:0/294233502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b023a0 0x55a2b7b11f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.026+0000 7f736aa00640 1 -- 172.21.15.47:0/294233502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b023a0 msgr2=0x55a2b7b11f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.026+0000 7f736aa00640 1 --2- 172.21.15.47:0/294233502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b023a0 0x55a2b7b11f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.026+0000 7f736aa00640 1 -- 172.21.15.47:0/294233502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b00810 msgr2=0x55a2b7b0a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.026+0000 7f736aa00640 1 --2- 172.21.15.47:0/294233502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b00810 0x55a2b7b0a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.026+0000 7f736aa00640 1 -- 172.21.15.47:0/294233502 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2b77793f0 con 0x55a2b7b019e0 2024-03-21T11:28:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.026+0000 7f7373e00640 1 --2- 172.21.15.47:0/294233502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b023a0 0x55a2b7b11f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.026+0000 7f7373400640 1 --2- 172.21.15.47:0/294233502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b00810 0x55a2b7b0a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.026+0000 7f736aa00640 1 --2- 172.21.15.47:0/294233502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b019e0 0x55a2b7b0aee0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f735c005c40 tx=0x7f735c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a1e8ee7cff77a83 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.026+0000 7f7360c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:23.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.027+0000 7f7360c00640 1 -- 172.21.15.47:0/294233502 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c023c90 con 0x55a2b7b019e0 2024-03-21T11:28:23.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.027+0000 7f7360c00640 1 -- 172.21.15.47:0/294233502 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f735c01a050 con 0x55a2b7b019e0 2024-03-21T11:28:23.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.027+0000 7f737cfb3740 1 -- 172.21.15.47:0/294233502 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7364004ba0 con 0x55a2b7b019e0 2024-03-21T11:28:23.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.027+0000 7f7360c00640 1 -- 172.21.15.47:0/294233502 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f735c023e30 con 0x55a2b7b019e0 2024-03-21T11:28:23.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.027+0000 7f737cfb3740 10 client.27208 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:23.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.027+0000 7f737cfb3740 1 -- 172.21.15.47:0/294233502 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a2b78437d0 con 0x55a2b7b019e0 2024-03-21T11:28:23.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.027+0000 7f7360c00640 1 -- 172.21.15.47:0/294233502 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7364004ba0 con 0x55a2b7b019e0 2024-03-21T11:28:23.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.027+0000 7f7360c00640 1 -- 172.21.15.47:0/294233502 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f735c02b410 con 0x55a2b7b019e0 2024-03-21T11:28:23.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.028+0000 7f737cfb3740 10 client.27208 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:23.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.028+0000 7f737cfb3740 10 client.27208 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:28:23.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.028+0000 7f737cfb3740 20 client.27208 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.028+0000 7f737cfb3740 1 --2- 172.21.15.47:0/294233502 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55a2b7b16450 0x55a2b7b188f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.028+0000 7f737cfb3740 4 client.27208 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.028+0000 7f737cfb3740 1 -- 172.21.15.47:0/294233502 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2b7a72800 con 0x55a2b7b16450 2024-03-21T11:28:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.028+0000 7f7373400640 1 --2- 172.21.15.47:0/294233502 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55a2b7b16450 0x55a2b7b188f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.028+0000 7f7373400640 1 --2- 172.21.15.47:0/294233502 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55a2b7b16450 0x55a2b7b188f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f10d08eb5958dff8 server_cookie=caed218800b7a9f4 in_seq=0 out_seq=0 2024-03-21T11:28:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.028+0000 7f7360c00640 10 client.27208 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:28:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.029+0000 7f7360c00640 1 -- 172.21.15.47:0/294233502 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a2b7a72800 con 0x55a2b7b16450 2024-03-21T11:28:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.029+0000 7f7360c00640 10 client.27208 handle_command_reply: tid=0 2024-03-21T11:28:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.029+0000 7f737cfb3740 1 client.27208 shutdown 2024-03-21T11:28:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.029+0000 7f737cfb3740 1 -- 172.21.15.47:0/294233502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b019e0 msgr2=0x55a2b7b0aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.029+0000 7f737cfb3740 1 --2- 172.21.15.47:0/294233502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b019e0 0x55a2b7b0aee0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f735c005c40 tx=0x7f735c004ab0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.029+0000 7f737cfb3740 1 -- 172.21.15.47:0/294233502 shutdown_connections 2024-03-21T11:28:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.029+0000 7f737cfb3740 1 --2- 172.21.15.47:0/294233502 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55a2b7b16450 0x55a2b7b188f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.029+0000 7f737cfb3740 1 --2- 172.21.15.47:0/294233502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b023a0 0x55a2b7b11f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.029+0000 7f737cfb3740 1 --2- 172.21.15.47:0/294233502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b019e0 0x55a2b7b0aee0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.029+0000 7f737cfb3740 1 --2- 172.21.15.47:0/294233502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b00810 0x55a2b7b0a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.029+0000 7f7360c00640 0 client.27208 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:28:23.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.029+0000 7f737cfb3740 1 -- 172.21.15.47:0/294233502 >> 172.21.15.47:0/294233502 conn(0x55a2b7afbc00 msgr2=0x55a2b7afaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:23.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.029+0000 7f737cfb3740 1 -- 172.21.15.47:0/294233502 shutdown_connections 2024-03-21T11:28:23.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.030+0000 7f737cfb3740 1 -- 172.21.15.47:0/294233502 wait complete. 2024-03-21T11:28:23.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.030+0000 7f737cfb3740 20 client.27208 trim_cache size 0 max 16384 2024-03-21T11:28:23.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.045+0000 7f737cfb3740 1 Processor -- start 2024-03-21T11:28:23.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.045+0000 7f737cfb3740 1 -- start start 2024-03-21T11:28:23.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.045+0000 7f737cfb3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b00810 0x55a2b7aff570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.045+0000 7f737cfb3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b019e0 0x55a2b7b11ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.045+0000 7f737cfb3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b023a0 0x55a2b7b049e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.045+0000 7f737cfb3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73741b0c90 con 0x55a2b7b019e0 2024-03-21T11:28:23.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.045+0000 7f737cfb3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73741b2610 con 0x55a2b7b00810 2024-03-21T11:28:23.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.045+0000 7f737cfb3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73741b12f0 con 0x55a2b7b023a0 2024-03-21T11:28:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.046+0000 7f736aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b019e0 0x55a2b7b11ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.046+0000 7f7373400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b00810 0x55a2b7aff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.046+0000 7f7373400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b00810 0x55a2b7aff570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44796/0 (socket says 172.21.15.47:44796) 2024-03-21T11:28:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.046+0000 7f7373400640 1 -- 172.21.15.47:0/3319260293 learned_addr learned my addr 172.21.15.47:0/3319260293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.046+0000 7f7373e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b023a0 0x55a2b7b049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.046+0000 7f7373400640 1 -- 172.21.15.47:0/3319260293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b023a0 msgr2=0x55a2b7b049e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.046+0000 7f7373400640 1 --2- 172.21.15.47:0/3319260293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b023a0 0x55a2b7b049e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.046+0000 7f7373400640 1 -- 172.21.15.47:0/3319260293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b019e0 msgr2=0x55a2b7b11ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.046+0000 7f7373400640 1 --2- 172.21.15.47:0/3319260293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b019e0 0x55a2b7b11ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.046+0000 7f7373400640 1 -- 172.21.15.47:0/3319260293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2b7b08500 con 0x55a2b7b00810 2024-03-21T11:28:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.046+0000 7f7373e00640 1 --2- 172.21.15.47:0/3319260293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b023a0 0x55a2b7b049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.046+0000 7f7373400640 1 --2- 172.21.15.47:0/3319260293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b00810 0x55a2b7aff570 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55a2b7afcb70 tx=0x7f736402bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57c27398df5e15d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.047+0000 7f7363400640 1 -- 172.21.15.47:0/3319260293 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364042070 con 0x55a2b7b00810 2024-03-21T11:28:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.047+0000 7f7363400640 1 -- 172.21.15.47:0/3319260293 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7364009520 con 0x55a2b7b00810 2024-03-21T11:28:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.047+0000 7f7363400640 1 -- 172.21.15.47:0/3319260293 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364005650 con 0x55a2b7b00810 2024-03-21T11:28:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.047+0000 7f737cfb3740 1 -- 172.21.15.47:0/3319260293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b00810 msgr2=0x55a2b7aff570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.047+0000 7f737cfb3740 1 --2- 172.21.15.47:0/3319260293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b00810 0x55a2b7aff570 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55a2b7afcb70 tx=0x7f736402bc40 comp rx=0 tx=0).stop 2024-03-21T11:28:23.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.047+0000 7f737cfb3740 1 -- 172.21.15.47:0/3319260293 shutdown_connections 2024-03-21T11:28:23.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.047+0000 7f737cfb3740 1 --2- 172.21.15.47:0/3319260293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b023a0 0x55a2b7b049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.047+0000 7f737cfb3740 1 --2- 172.21.15.47:0/3319260293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b019e0 0x55a2b7b11ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.047+0000 7f737cfb3740 1 --2- 172.21.15.47:0/3319260293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b00810 0x55a2b7aff570 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.047+0000 7f737cfb3740 1 -- 172.21.15.47:0/3319260293 >> 172.21.15.47:0/3319260293 conn(0x55a2b7afbc00 msgr2=0x55a2b7afee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:23.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.048+0000 7f737cfb3740 1 -- 172.21.15.47:0/3319260293 shutdown_connections 2024-03-21T11:28:23.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.048+0000 7f737cfb3740 1 -- 172.21.15.47:0/3319260293 wait complete. 2024-03-21T11:28:23.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.048+0000 7f737cfb3740 1 -- start start 2024-03-21T11:28:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.048+0000 7f737cfb3740 1 Processor -- start 2024-03-21T11:28:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f737cfb3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b00810 0x55a2b7b0da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f737cfb3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b019e0 0x55a2b7b0dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f737cfb3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b023a0 0x55a2b7b10ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f737cfb3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73741b1fb0 con 0x55a2b7b00810 2024-03-21T11:28:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f737cfb3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73741b1950 con 0x55a2b7b019e0 2024-03-21T11:28:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f737cfb3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73741aee50 con 0x55a2b7b023a0 2024-03-21T11:28:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f737cfb3740 10 client.0 _ll_register_callbacks cb 0x7f735c005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f7373400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b00810 0x55a2b7b0da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f736aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b019e0 0x55a2b7b0dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f7373e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b023a0 0x55a2b7b10ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f736aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b019e0 0x55a2b7b0dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44804/0 (socket says 172.21.15.47:44804) 2024-03-21T11:28:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f7373400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b00810 0x55a2b7b0da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51748/0 (socket says 172.21.15.47:51748) 2024-03-21T11:28:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f7373400640 1 -- 172.21.15.47:0/4025736044 learned_addr learned my addr 172.21.15.47:0/4025736044 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f7373400640 1 -- 172.21.15.47:0/4025736044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b023a0 msgr2=0x55a2b7b10ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f7373400640 1 --2- 172.21.15.47:0/4025736044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b023a0 0x55a2b7b10ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f7373400640 1 -- 172.21.15.47:0/4025736044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b019e0 msgr2=0x55a2b7b0dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f7373400640 1 --2- 172.21.15.47:0/4025736044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b019e0 0x55a2b7b0dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f7373400640 1 -- 172.21.15.47:0/4025736044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2b7b114b0 con 0x55a2b7b00810 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.049+0000 7f736aa00640 1 --2- 172.21.15.47:0/4025736044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b019e0 0x55a2b7b0dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.050+0000 7f7373400640 1 --2- 172.21.15.47:0/4025736044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b00810 0x55a2b7b0da70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7364005c40 tx=0x7f736402cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63358b7c93dcb57f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.050+0000 7f7360c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.050+0000 7f7360c00640 1 -- 172.21.15.47:0/4025736044 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364042030 con 0x55a2b7b00810 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.050+0000 7f7360c00640 1 -- 172.21.15.47:0/4025736044 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f73640055d0 con 0x55a2b7b00810 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.050+0000 7f737cfb3740 1 -- 172.21.15.47:0/4025736044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7364009520 con 0x55a2b7b00810 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.050+0000 7f7360c00640 1 -- 172.21.15.47:0/4025736044 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f736405d050 con 0x55a2b7b00810 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.050+0000 7f737cfb3740 10 client.16938 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.050+0000 7f737cfb3740 1 -- 172.21.15.47:0/4025736044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a2b7b72160 con 0x55a2b7b00810 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.051+0000 7f7360c00640 1 -- 172.21.15.47:0/4025736044 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7364009520 con 0x55a2b7b00810 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.051+0000 7f7360c00640 1 -- 172.21.15.47:0/4025736044 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f736404fb90 con 0x55a2b7b00810 2024-03-21T11:28:23.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.051+0000 7f737cfb3740 10 client.16938 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.051+0000 7f737cfb3740 10 client.16938 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:28:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.051+0000 7f737cfb3740 20 client.16938 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.051+0000 7f737cfb3740 1 --2- 172.21.15.47:0/4025736044 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55a2b7b72630 0x55a2b7b74a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.051+0000 7f737cfb3740 4 client.16938 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:28:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.051+0000 7f737cfb3740 1 -- 172.21.15.47:0/4025736044 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a2b7a6cb50 con 0x55a2b7b72630 2024-03-21T11:28:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.051+0000 7f736aa00640 1 --2- 172.21.15.47:0/4025736044 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55a2b7b72630 0x55a2b7b74a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f736aa00640 1 --2- 172.21.15.47:0/4025736044 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55a2b7b72630 0x55a2b7b74a50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1ce3963ccfd346c server_cookie=db80a01ac51f8a73 in_seq=0 out_seq=0 2024-03-21T11:28:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f7360c00640 10 client.16938 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:28:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f7360c00640 1 -- 172.21.15.47:0/4025736044 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a2b7a6cb50 con 0x55a2b7b72630 2024-03-21T11:28:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f7360c00640 10 client.16938 handle_command_reply: tid=0 2024-03-21T11:28:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f737cfb3740 1 client.16938 shutdown 2024-03-21T11:28:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f737cfb3740 1 -- 172.21.15.47:0/4025736044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b00810 msgr2=0x55a2b7b0da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f737cfb3740 1 --2- 172.21.15.47:0/4025736044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b00810 0x55a2b7b0da70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7364005c40 tx=0x7f736402cdc0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f737cfb3740 1 -- 172.21.15.47:0/4025736044 shutdown_connections 2024-03-21T11:28:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f737cfb3740 1 --2- 172.21.15.47:0/4025736044 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55a2b7b72630 0x55a2b7b74a50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f737cfb3740 1 --2- 172.21.15.47:0/4025736044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2b7b023a0 0x55a2b7b10ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f737cfb3740 1 --2- 172.21.15.47:0/4025736044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2b7b019e0 0x55a2b7b0dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f737cfb3740 1 --2- 172.21.15.47:0/4025736044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2b7b00810 0x55a2b7b0da70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f7360c00640 0 client.16938 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:28:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f737cfb3740 1 -- 172.21.15.47:0/4025736044 >> 172.21.15.47:0/4025736044 conn(0x55a2b7afbc00 msgr2=0x55a2b7aff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f737cfb3740 1 -- 172.21.15.47:0/4025736044 shutdown_connections 2024-03-21T11:28:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.052+0000 7f737cfb3740 1 -- 172.21.15.47:0/4025736044 wait complete. 2024-03-21T11:28:23.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.053+0000 7f737cfb3740 20 client.16938 trim_cache size 0 max 16384 2024-03-21T11:28:23.057 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:28:23.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.053+0000 7f737a200640 1 -- 172.21.15.47:0/2808163781 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73580a2fa0 msgr2=0x7f73580a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.053+0000 7f737a200640 1 --2- 172.21.15.47:0/2808163781 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73580a2fa0 0x7f73580a5460 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x55a2b7b018e0 tx=0x7f735c009500 comp rx=0 tx=0).stop 2024-03-21T11:28:23.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.053+0000 7f737a200640 1 -- 172.21.15.47:0/2808163781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73741a3770 msgr2=0x7f73741a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.053+0000 7f737a200640 1 --2- 172.21.15.47:0/2808163781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73741a3770 0x7f73741a5b60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f736c00e970 tx=0x7f736c00ee30 comp rx=0 tx=0).stop 2024-03-21T11:28:23.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.053+0000 7f737a200640 1 -- 172.21.15.47:0/2808163781 shutdown_connections 2024-03-21T11:28:23.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.053+0000 7f737a200640 1 --2- 172.21.15.47:0/2808163781 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73580a2fa0 0x7f73580a5460 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.053+0000 7f737a200640 1 --2- 172.21.15.47:0/2808163781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73741a3770 0x7f73741a5b60 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.053+0000 7f737a200640 1 --2- 172.21.15.47:0/2808163781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7374109870 0x7f737419e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.053+0000 7f737a200640 1 --2- 172.21.15.47:0/2808163781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7374108eb0 0x7f737419e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.053+0000 7f737a200640 1 -- 172.21.15.47:0/2808163781 >> 172.21.15.47:0/2808163781 conn(0x7f7374076c60 msgr2=0x7f737410c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:23.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.053+0000 7f737a200640 1 -- 172.21.15.47:0/2808163781 shutdown_connections 2024-03-21T11:28:23.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.053+0000 7f737a200640 1 -- 172.21.15.47:0/2808163781 wait complete. 2024-03-21T11:28:23.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:22 smithi047 ceph-mon[31351]: pgmap v551: 129 pgs: 129 active+clean; 37 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 298 op/s 2024-03-21T11:28:23.073 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:28:23.073 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:28:23.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.327+0000 7fec20c00640 1 -- 172.21.15.47:0/1492923973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec1c107410 msgr2=0x7fec1c109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.327+0000 7fec20c00640 1 --2- 172.21.15.47:0/1492923973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec1c107410 0x7fec1c109800 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009900 tx=0x7fec0c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:28:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.327+0000 7fec20c00640 1 -- 172.21.15.47:0/1492923973 shutdown_connections 2024-03-21T11:28:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.327+0000 7fec20c00640 1 --2- 172.21.15.47:0/1492923973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec1c107410 0x7fec1c109800 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.327+0000 7fec20c00640 1 --2- 172.21.15.47:0/1492923973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c0ffcd0 0x7fec1c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.327+0000 7fec20c00640 1 --2- 172.21.15.47:0/1492923973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec1c0ff300 0x7fec1c0ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.327+0000 7fec20c00640 1 -- 172.21.15.47:0/1492923973 >> 172.21.15.47:0/1492923973 conn(0x7fec1c0fae60 msgr2=0x7fec1c0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.328+0000 7fec20c00640 1 -- 172.21.15.47:0/1492923973 shutdown_connections 2024-03-21T11:28:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.328+0000 7fec20c00640 1 -- 172.21.15.47:0/1492923973 wait complete. 2024-03-21T11:28:23.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.329+0000 7fec20c00640 1 Processor -- start 2024-03-21T11:28:23.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.329+0000 7fec20c00640 1 -- start start 2024-03-21T11:28:23.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.329+0000 7fec20c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c0ff300 0x7fec1c102820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.329+0000 7fec20c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec1c0ffcd0 0x7fec1c100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.330+0000 7fec20c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec1c107410 0x7fec1c1013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.330+0000 7fec20c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c10d340 con 0x7fec1c0ffcd0 2024-03-21T11:28:23.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.330+0000 7fec20c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c10d1c0 con 0x7fec1c0ff300 2024-03-21T11:28:23.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.330+0000 7fec20c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c10d4c0 con 0x7fec1c107410 2024-03-21T11:28:23.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.330+0000 7fec19e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c0ff300 0x7fec1c102820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.330+0000 7fec19e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c0ff300 0x7fec1c102820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44812/0 (socket says 172.21.15.47:44812) 2024-03-21T11:28:23.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.330+0000 7fec19e00640 1 -- 172.21.15.47:0/1832506810 learned_addr learned my addr 172.21.15.47:0/1832506810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:23.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.330+0000 7fec19400640 1 --2- 172.21.15.47:0/1832506810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec1c0ffcd0 0x7fec1c100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.330+0000 7fec1a800640 1 --2- 172.21.15.47:0/1832506810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec1c107410 0x7fec1c1013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.330+0000 7fec19e00640 1 -- 172.21.15.47:0/1832506810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec1c107410 msgr2=0x7fec1c1013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.330+0000 7fec19e00640 1 --2- 172.21.15.47:0/1832506810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec1c107410 0x7fec1c1013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.330+0000 7fec19e00640 1 -- 172.21.15.47:0/1832506810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec1c0ffcd0 msgr2=0x7fec1c100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.330+0000 7fec19e00640 1 --2- 172.21.15.47:0/1832506810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec1c0ffcd0 0x7fec1c100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.330+0000 7fec19e00640 1 -- 172.21.15.47:0/1832506810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c101c30 con 0x7fec1c0ff300 2024-03-21T11:28:23.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.331+0000 7fec19e00640 1 --2- 172.21.15.47:0/1832506810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c0ff300 0x7fec1c102820 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fec1400ebc0 tx=0x7fec1400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:23.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.331+0000 7fec12a00640 1 -- 172.21.15.47:0/1832506810 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14010630 con 0x7fec1c0ff300 2024-03-21T11:28:23.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.331+0000 7fec12a00640 1 -- 172.21.15.47:0/1832506810 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec14010cb0 con 0x7fec1c0ff300 2024-03-21T11:28:23.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.331+0000 7fec20c00640 1 -- 172.21.15.47:0/1832506810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c1a1ad0 con 0x7fec1c0ff300 2024-03-21T11:28:23.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.331+0000 7fec12a00640 1 -- 172.21.15.47:0/1832506810 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14018bd0 con 0x7fec1c0ff300 2024-03-21T11:28:23.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.331+0000 7fec20c00640 1 -- 172.21.15.47:0/1832506810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c1a2070 con 0x7fec1c0ff300 2024-03-21T11:28:23.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.335+0000 7fec2381b740 1 Processor -- start 2024-03-21T11:28:23.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.335+0000 7fec2381b740 1 -- start start 2024-03-21T11:28:23.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.335+0000 7fec2381b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921d020 0x55bad921d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.335+0000 7fec12a00640 1 -- 172.21.15.47:0/1832506810 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fec140107d0 con 0x7fec1c0ff300 2024-03-21T11:28:23.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.335+0000 7fec2381b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921e1f0 0x55bad921e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.335+0000 7fec2381b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921ebb0 0x55bad9225440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.335+0000 7fec2381b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad91792c0 con 0x55bad921e1f0 2024-03-21T11:28:23.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.335+0000 7fec2381b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad9121c90 con 0x55bad921d020 2024-03-21T11:28:23.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.335+0000 7fec2381b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad9175360 con 0x55bad921ebb0 2024-03-21T11:28:23.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.336+0000 7fec19e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921e1f0 0x55bad921e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.336+0000 7fec1a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921d020 0x55bad921d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.336+0000 7fec19e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921e1f0 0x55bad921e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51778/0 (socket says 172.21.15.47:51778) 2024-03-21T11:28:23.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.336+0000 7fec19e00640 1 -- 172.21.15.47:0/687708945 learned_addr learned my addr 172.21.15.47:0/687708945 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:23.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.336+0000 7fec19400640 1 --2- 172.21.15.47:0/687708945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921ebb0 0x55bad9225440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.337+0000 7fec19400640 1 -- 172.21.15.47:0/687708945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921d020 msgr2=0x55bad921d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.337+0000 7fec19400640 1 --2- 172.21.15.47:0/687708945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921d020 0x55bad921d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.337+0000 7fec19400640 1 -- 172.21.15.47:0/687708945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921e1f0 msgr2=0x55bad921e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.337+0000 7fec19400640 1 --2- 172.21.15.47:0/687708945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921e1f0 0x55bad921e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.337+0000 7fec19400640 1 -- 172.21.15.47:0/687708945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bad9146080 con 0x55bad921ebb0 2024-03-21T11:28:23.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.337+0000 7fec19e00640 1 --2- 172.21.15.47:0/687708945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921e1f0 0x55bad921e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:23.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.337+0000 7fec19400640 1 --2- 172.21.15.47:0/687708945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921ebb0 0x55bad9225440 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fec1c1009a0 tx=0x7fec08011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e930b11952cd0d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:23.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.337+0000 7fec12a00640 1 --2- 172.21.15.47:0/1832506810 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fec000a2fa0 0x7fec000a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.337+0000 7fec07000640 1 -- 172.21.15.47:0/687708945 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec08004230 con 0x55bad921ebb0 2024-03-21T11:28:23.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.337+0000 7fec07000640 1 -- 172.21.15.47:0/687708945 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec08009e10 con 0x55bad921ebb0 2024-03-21T11:28:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.337+0000 7fec07000640 1 -- 172.21.15.47:0/687708945 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec080197e0 con 0x55bad921ebb0 2024-03-21T11:28:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.337+0000 7fec12a00640 1 -- 172.21.15.47:0/1832506810 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fec140cfcf0 con 0x7fec1c0ff300 2024-03-21T11:28:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.338+0000 7fec1a800640 1 --2- 172.21.15.47:0/1832506810 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fec000a2fa0 0x7fec000a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.338+0000 7fec2381b740 1 -- 172.21.15.47:0/687708945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921ebb0 msgr2=0x55bad9225440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.338+0000 7fec2381b740 1 --2- 172.21.15.47:0/687708945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921ebb0 0x55bad9225440 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fec1c1009a0 tx=0x7fec08011810 comp rx=0 tx=0).stop 2024-03-21T11:28:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.338+0000 7fec1a800640 1 --2- 172.21.15.47:0/1832506810 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fec000a2fa0 0x7fec000a5460 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fec0c005ab0 tx=0x7fec0c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.338+0000 7fec2381b740 1 -- 172.21.15.47:0/687708945 shutdown_connections 2024-03-21T11:28:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.338+0000 7fec2381b740 1 --2- 172.21.15.47:0/687708945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921ebb0 0x55bad9225440 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.339+0000 7fec2381b740 1 --2- 172.21.15.47:0/687708945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921e1f0 0x55bad921e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.339+0000 7fec2381b740 1 --2- 172.21.15.47:0/687708945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921d020 0x55bad921d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.339+0000 7fec2381b740 1 -- 172.21.15.47:0/687708945 >> 172.21.15.47:0/687708945 conn(0x55bad920fc00 msgr2=0x55bad9210000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.339+0000 7fec2381b740 1 -- 172.21.15.47:0/687708945 shutdown_connections 2024-03-21T11:28:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.339+0000 7fec2381b740 1 -- 172.21.15.47:0/687708945 wait complete. 2024-03-21T11:28:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.339+0000 7fec2381b740 1 -- start start 2024-03-21T11:28:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.339+0000 7fec2381b740 1 Processor -- start 2024-03-21T11:28:23.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec2381b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921d020 0x55bad9215cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec2381b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921e1f0 0x55bad9216200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec2381b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921ebb0 0x55bad921b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec2381b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad92145d0 con 0x55bad921d020 2024-03-21T11:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec2381b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad91350d0 con 0x55bad921ebb0 2024-03-21T11:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec2381b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad9214750 con 0x55bad921e1f0 2024-03-21T11:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec2381b740 10 client.0 _ll_register_callbacks cb 0x55bad9139e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec1a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921ebb0 0x55bad921b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec19400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921e1f0 0x55bad9216200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec1a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921ebb0 0x55bad921b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44832/0 (socket says 172.21.15.47:44832) 2024-03-21T11:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec1a800640 1 -- 172.21.15.47:0/3594660281 learned_addr learned my addr 172.21.15.47:0/3594660281 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec19e00640 1 --2- 172.21.15.47:0/3594660281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921d020 0x55bad9215cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec19400640 1 -- 172.21.15.47:0/3594660281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921ebb0 msgr2=0x55bad921b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec19400640 1 --2- 172.21.15.47:0/3594660281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921ebb0 0x55bad921b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec19400640 1 -- 172.21.15.47:0/3594660281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921d020 msgr2=0x55bad9215cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec19400640 1 --2- 172.21.15.47:0/3594660281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921d020 0x55bad9215cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.340+0000 7fec19400640 1 -- 172.21.15.47:0/3594660281 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bad8e8d3f0 con 0x55bad921e1f0 2024-03-21T11:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.341+0000 7fec19400640 1 --2- 172.21.15.47:0/3594660281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921e1f0 0x55bad9216200 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fec0801b040 tx=0x7fec08003d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=217cea961a273702 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.341+0000 7febfb400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.341+0000 7febfb400640 1 -- 172.21.15.47:0/3594660281 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec08027070 con 0x55bad921e1f0 2024-03-21T11:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.341+0000 7fec2381b740 1 -- 172.21.15.47:0/3594660281 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec08009e10 con 0x55bad921e1f0 2024-03-21T11:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.341+0000 7febfb400640 1 -- 172.21.15.47:0/3594660281 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fec080054d0 con 0x55bad921e1f0 2024-03-21T11:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.341+0000 7febfb400640 1 -- 172.21.15.47:0/3594660281 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec08033050 con 0x55bad921e1f0 2024-03-21T11:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.341+0000 7fec2381b740 10 client.27226 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.341+0000 7fec2381b740 1 -- 172.21.15.47:0/3594660281 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bad8f577d0 con 0x55bad921e1f0 2024-03-21T11:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.342+0000 7febfb400640 1 -- 172.21.15.47:0/3594660281 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec08009e10 con 0x55bad921e1f0 2024-03-21T11:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.342+0000 7febfb400640 1 -- 172.21.15.47:0/3594660281 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fec08023420 con 0x55bad921e1f0 2024-03-21T11:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.342+0000 7fec2381b740 10 client.27226 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.342+0000 7fec2381b740 10 client.27226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.342+0000 7fec2381b740 20 client.27226 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.342+0000 7fec2381b740 1 --2- 172.21.15.47:0/3594660281 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bad923b220 0x55bad923d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.342+0000 7fec2381b740 4 client.27226 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.342+0000 7fec2381b740 1 -- 172.21.15.47:0/3594660281 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bad9186800 con 0x55bad923b220 2024-03-21T11:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.342+0000 7fec19e00640 1 --2- 172.21.15.47:0/3594660281 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bad923b220 0x55bad923d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.343+0000 7fec19e00640 1 --2- 172.21.15.47:0/3594660281 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bad923b220 0x55bad923d670 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8339b24c7505a1 server_cookie=e7c8cefb41fa0571 in_seq=0 out_seq=0 2024-03-21T11:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.343+0000 7febfb400640 10 client.27226 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.344+0000 7febfb400640 1 -- 172.21.15.47:0/3594660281 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bad9186800 con 0x55bad923b220 2024-03-21T11:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.344+0000 7febfb400640 10 client.27226 handle_command_reply: tid=0 2024-03-21T11:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.344+0000 7fec2381b740 1 client.27226 shutdown 2024-03-21T11:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.344+0000 7fec2381b740 1 -- 172.21.15.47:0/3594660281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921e1f0 msgr2=0x55bad9216200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.344+0000 7fec2381b740 1 --2- 172.21.15.47:0/3594660281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921e1f0 0x55bad9216200 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fec0801b040 tx=0x7fec08003d90 comp rx=0 tx=0).stop 2024-03-21T11:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.344+0000 7fec2381b740 1 -- 172.21.15.47:0/3594660281 shutdown_connections 2024-03-21T11:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.344+0000 7fec2381b740 1 --2- 172.21.15.47:0/3594660281 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bad923b220 0x55bad923d670 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.344+0000 7fec2381b740 1 --2- 172.21.15.47:0/3594660281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921ebb0 0x55bad921b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.344+0000 7fec2381b740 1 --2- 172.21.15.47:0/3594660281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921e1f0 0x55bad9216200 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.344+0000 7fec2381b740 1 --2- 172.21.15.47:0/3594660281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921d020 0x55bad9215cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.344+0000 7fec2381b740 1 -- 172.21.15.47:0/3594660281 >> 172.21.15.47:0/3594660281 conn(0x55bad920fc00 msgr2=0x55bad920eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.344+0000 7febfb400640 0 client.27226 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.344+0000 7fec2381b740 1 -- 172.21.15.47:0/3594660281 shutdown_connections 2024-03-21T11:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.345+0000 7fec2381b740 1 -- 172.21.15.47:0/3594660281 wait complete. 2024-03-21T11:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.345+0000 7fec2381b740 20 client.27226 trim_cache size 0 max 16384 2024-03-21T11:28:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.359+0000 7fec2381b740 1 Processor -- start 2024-03-21T11:28:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.360+0000 7fec2381b740 1 -- start start 2024-03-21T11:28:23.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.360+0000 7fec2381b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921d020 0x55bad9221330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.360+0000 7fec2381b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921e1f0 0x55bad9221870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.360+0000 7fec2381b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921ebb0 0x55bad9221db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.360+0000 7fec2381b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c1a7f50 con 0x55bad921ebb0 2024-03-21T11:28:23.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.360+0000 7fec2381b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c1a98d0 con 0x55bad921e1f0 2024-03-21T11:28:23.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.360+0000 7fec2381b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c1a85b0 con 0x55bad921d020 2024-03-21T11:28:23.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.360+0000 7fec19e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921d020 0x55bad9221330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.360+0000 7fec19400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921e1f0 0x55bad9221870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.360+0000 7fec19e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921d020 0x55bad9221330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36998/0 (socket says 172.21.15.47:36998) 2024-03-21T11:28:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.360+0000 7fec19e00640 1 -- 172.21.15.47:0/3210563503 learned_addr learned my addr 172.21.15.47:0/3210563503 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.360+0000 7fec1a800640 1 --2- 172.21.15.47:0/3210563503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921ebb0 0x55bad9221db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.361+0000 7fec19e00640 1 -- 172.21.15.47:0/3210563503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921e1f0 msgr2=0x55bad9221870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.361+0000 7fec19e00640 1 --2- 172.21.15.47:0/3210563503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921e1f0 0x55bad9221870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.361+0000 7fec19e00640 1 -- 172.21.15.47:0/3210563503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921ebb0 msgr2=0x55bad9221db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.361+0000 7fec19e00640 1 --2- 172.21.15.47:0/3210563503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921ebb0 0x55bad9221db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.361+0000 7fec19e00640 1 -- 172.21.15.47:0/3210563503 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bad9216570 con 0x55bad921d020 2024-03-21T11:28:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.361+0000 7fec19400640 1 --2- 172.21.15.47:0/3210563503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921e1f0 0x55bad9221870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:23.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.361+0000 7fec19e00640 1 --2- 172.21.15.47:0/3210563503 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921d020 0x55bad9221330 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fec14098b10 tx=0x7fec140d37d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f2d01978ad7ba0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.361+0000 7fec06600640 1 -- 172.21.15.47:0/3210563503 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14010e60 con 0x55bad921d020 2024-03-21T11:28:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.361+0000 7fec06600640 1 -- 172.21.15.47:0/3210563503 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec1409c140 con 0x55bad921d020 2024-03-21T11:28:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.361+0000 7fec06600640 1 -- 172.21.15.47:0/3210563503 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14018420 con 0x55bad921d020 2024-03-21T11:28:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.362+0000 7fec2381b740 1 -- 172.21.15.47:0/3210563503 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921d020 msgr2=0x55bad9221330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.362+0000 7fec2381b740 1 --2- 172.21.15.47:0/3210563503 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921d020 0x55bad9221330 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fec14098b10 tx=0x7fec140d37d0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.362+0000 7fec2381b740 1 -- 172.21.15.47:0/3210563503 shutdown_connections 2024-03-21T11:28:23.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.362+0000 7fec2381b740 1 --2- 172.21.15.47:0/3210563503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921ebb0 0x55bad9221db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.362+0000 7fec2381b740 1 --2- 172.21.15.47:0/3210563503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921e1f0 0x55bad9221870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.362+0000 7fec2381b740 1 --2- 172.21.15.47:0/3210563503 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921d020 0x55bad9221330 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.362+0000 7fec2381b740 1 -- 172.21.15.47:0/3210563503 >> 172.21.15.47:0/3210563503 conn(0x55bad920fc00 msgr2=0x55bad92136f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.362+0000 7fec2381b740 1 -- 172.21.15.47:0/3210563503 shutdown_connections 2024-03-21T11:28:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.362+0000 7fec2381b740 1 -- 172.21.15.47:0/3210563503 wait complete. 2024-03-21T11:28:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.363+0000 7fec2381b740 1 -- start start 2024-03-21T11:28:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.363+0000 7fec2381b740 1 Processor -- start 2024-03-21T11:28:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.363+0000 7fec2381b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921d020 0x55bad924a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.363+0000 7fec2381b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921e1f0 0x55bad924ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.363+0000 7fec2381b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921ebb0 0x55bad9295fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.363+0000 7fec2381b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c1a9270 con 0x55bad921ebb0 2024-03-21T11:28:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.363+0000 7fec2381b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c1a8c10 con 0x55bad921e1f0 2024-03-21T11:28:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.363+0000 7fec2381b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c1a6110 con 0x55bad921d020 2024-03-21T11:28:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.363+0000 7fec2381b740 10 client.0 _ll_register_callbacks cb 0x7fec140d36a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.364+0000 7fec19400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921e1f0 0x55bad924ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.364+0000 7fec19400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921e1f0 0x55bad924ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44848/0 (socket says 172.21.15.47:44848) 2024-03-21T11:28:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.364+0000 7fec19400640 1 -- 172.21.15.47:0/3962351214 learned_addr learned my addr 172.21.15.47:0/3962351214 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.364+0000 7fec19e00640 1 --2- 172.21.15.47:0/3962351214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921d020 0x55bad924a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.364+0000 7fec1a800640 1 --2- 172.21.15.47:0/3962351214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921ebb0 0x55bad9295fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.364+0000 7fec19400640 1 -- 172.21.15.47:0/3962351214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921d020 msgr2=0x55bad924a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.364+0000 7fec19400640 1 --2- 172.21.15.47:0/3962351214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921d020 0x55bad924a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.364+0000 7fec19400640 1 -- 172.21.15.47:0/3962351214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921ebb0 msgr2=0x55bad9295fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.364+0000 7fec19400640 1 --2- 172.21.15.47:0/3962351214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921ebb0 0x55bad9295fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.364+0000 7fec19400640 1 -- 172.21.15.47:0/3962351214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bad921b650 con 0x55bad921e1f0 2024-03-21T11:28:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.364+0000 7fec19e00640 1 --2- 172.21.15.47:0/3962351214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921d020 0x55bad924a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.364+0000 7fec19400640 1 --2- 172.21.15.47:0/3962351214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921e1f0 0x55bad924ad00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fec08011d90 tx=0x7fec08005d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30d7aba3fb2d3ece server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.364+0000 7febfb400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.365+0000 7febfb400640 1 -- 172.21.15.47:0/3962351214 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0803c060 con 0x55bad921e1f0 2024-03-21T11:28:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.365+0000 7febfb400640 1 -- 172.21.15.47:0/3962351214 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fec08023420 con 0x55bad921e1f0 2024-03-21T11:28:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.365+0000 7fec2381b740 1 -- 172.21.15.47:0/3962351214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec1409c140 con 0x55bad921e1f0 2024-03-21T11:28:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.365+0000 7febfb400640 1 -- 172.21.15.47:0/3962351214 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec08043050 con 0x55bad921e1f0 2024-03-21T11:28:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.365+0000 7fec2381b740 10 client.27212 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.365+0000 7fec2381b740 1 -- 172.21.15.47:0/3962351214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bad92976d0 con 0x55bad921e1f0 2024-03-21T11:28:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.365+0000 7febfb400640 1 -- 172.21.15.47:0/3962351214 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec1409c140 con 0x55bad921e1f0 2024-03-21T11:28:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.366+0000 7febfb400640 1 -- 172.21.15.47:0/3962351214 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fec0802c740 con 0x55bad921e1f0 2024-03-21T11:28:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.366+0000 7fec2381b740 10 client.27212 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:23.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.366+0000 7fec2381b740 10 client.27212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:23.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.366+0000 7fec2381b740 20 client.27212 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:23.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.366+0000 7fec2381b740 1 --2- 172.21.15.47:0/3962351214 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bad9297ff0 0x55bad929a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:23.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.366+0000 7fec2381b740 4 client.27212 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:28:23.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.366+0000 7fec2381b740 1 -- 172.21.15.47:0/3962351214 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bad9180b50 con 0x55bad9297ff0 2024-03-21T11:28:23.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.366+0000 7fec19e00640 1 --2- 172.21.15.47:0/3962351214 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bad9297ff0 0x55bad929a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:23.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.367+0000 7fec19e00640 1 --2- 172.21.15.47:0/3962351214 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bad9297ff0 0x55bad929a410 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97872c77f8d6a9a9 server_cookie=bcca82bf337498b3 in_seq=0 out_seq=0 2024-03-21T11:28:23.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.367+0000 7febfb400640 10 client.27212 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.368+0000 7febfb400640 1 -- 172.21.15.47:0/3962351214 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bad9180b50 con 0x55bad9297ff0 2024-03-21T11:28:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.368+0000 7febfb400640 10 client.27212 handle_command_reply: tid=0 2024-03-21T11:28:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.368+0000 7fec2381b740 1 client.27212 shutdown 2024-03-21T11:28:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.369+0000 7fec2381b740 1 -- 172.21.15.47:0/3962351214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921e1f0 msgr2=0x55bad924ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.369+0000 7fec2381b740 1 --2- 172.21.15.47:0/3962351214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921e1f0 0x55bad924ad00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fec08011d90 tx=0x7fec08005d70 comp rx=0 tx=0).stop 2024-03-21T11:28:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.369+0000 7fec2381b740 1 -- 172.21.15.47:0/3962351214 shutdown_connections 2024-03-21T11:28:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.369+0000 7fec2381b740 1 --2- 172.21.15.47:0/3962351214 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bad9297ff0 0x55bad929a410 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.369+0000 7fec2381b740 1 --2- 172.21.15.47:0/3962351214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bad921ebb0 0x55bad9295fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.369+0000 7fec2381b740 1 --2- 172.21.15.47:0/3962351214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bad921e1f0 0x55bad924ad00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.369+0000 7fec2381b740 1 --2- 172.21.15.47:0/3962351214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bad921d020 0x55bad924a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.369+0000 7fec2381b740 1 -- 172.21.15.47:0/3962351214 >> 172.21.15.47:0/3962351214 conn(0x55bad920fc00 msgr2=0x55bad92136f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:23.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.369+0000 7febfb400640 0 client.27212 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:23.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.369+0000 7fec2381b740 1 -- 172.21.15.47:0/3962351214 shutdown_connections 2024-03-21T11:28:23.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.369+0000 7fec2381b740 1 -- 172.21.15.47:0/3962351214 wait complete. 2024-03-21T11:28:23.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.369+0000 7fec2381b740 20 client.27212 trim_cache size 0 max 16384 2024-03-21T11:28:23.374 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:28:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.370+0000 7fec20c00640 1 -- 172.21.15.47:0/1832506810 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fec000a2fa0 msgr2=0x7fec000a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.370+0000 7fec20c00640 1 --2- 172.21.15.47:0/1832506810 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fec000a2fa0 0x7fec000a5460 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fec0c005ab0 tx=0x7fec0c02e040 comp rx=0 tx=0).stop 2024-03-21T11:28:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.370+0000 7fec20c00640 1 -- 172.21.15.47:0/1832506810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c0ff300 msgr2=0x7fec1c102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.370+0000 7fec20c00640 1 --2- 172.21.15.47:0/1832506810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c0ff300 0x7fec1c102820 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fec1400ebc0 tx=0x7fec1400c6a0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.370+0000 7fec20c00640 1 -- 172.21.15.47:0/1832506810 shutdown_connections 2024-03-21T11:28:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.370+0000 7fec20c00640 1 --2- 172.21.15.47:0/1832506810 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fec000a2fa0 0x7fec000a5460 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.370+0000 7fec20c00640 1 --2- 172.21.15.47:0/1832506810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec1c107410 0x7fec1c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.370+0000 7fec20c00640 1 --2- 172.21.15.47:0/1832506810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec1c0ffcd0 0x7fec1c100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.370+0000 7fec20c00640 1 --2- 172.21.15.47:0/1832506810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c0ff300 0x7fec1c102820 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.370+0000 7fec20c00640 1 -- 172.21.15.47:0/1832506810 >> 172.21.15.47:0/1832506810 conn(0x7fec1c0fae60 msgr2=0x7fec1c102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.370+0000 7fec20c00640 1 -- 172.21.15.47:0/1832506810 shutdown_connections 2024-03-21T11:28:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:23.370+0000 7fec20c00640 1 -- 172.21.15.47:0/1832506810 wait complete. 2024-03-21T11:28:23.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:22 smithi100 ceph-mon[36245]: pgmap v551: 129 pgs: 129 active+clean; 37 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 298 op/s 2024-03-21T11:28:23.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:22 smithi102 ceph-mon[36748]: pgmap v551: 129 pgs: 129 active+clean; 37 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 298 op/s 2024-03-21T11:28:23.511 INFO:tasks.workunit.client.0.smithi047.stdout: 1 299201 8.33 MB/sec execute 67 sec latency 12.656 ms 2024-03-21T11:28:24.392 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:28:24.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:28:24.393 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:28:24.511 INFO:tasks.workunit.client.0.smithi047.stdout: 1 301324 8.39 MB/sec execute 68 sec latency 12.384 ms 2024-03-21T11:28:24.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.635+0000 7f0388a00640 1 -- 172.21.15.47:0/2370945724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0384109770 msgr2=0x7f0384110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.635+0000 7f0388a00640 1 --2- 172.21.15.47:0/2370945724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0384109770 0x7f0384110000 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f037c009860 tx=0x7f037c02bb70 comp rx=0 tx=0).stop 2024-03-21T11:28:24.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.635+0000 7f0388a00640 1 -- 172.21.15.47:0/2370945724 shutdown_connections 2024-03-21T11:28:24.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.635+0000 7f0388a00640 1 --2- 172.21.15.47:0/2370945724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0384109770 0x7f0384110000 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.635+0000 7f0388a00640 1 --2- 172.21.15.47:0/2370945724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0384108db0 0x7f0384109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.635+0000 7f0388a00640 1 --2- 172.21.15.47:0/2370945724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0384107bc0 0x7f0384107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.635+0000 7f0388a00640 1 -- 172.21.15.47:0/2370945724 >> 172.21.15.47:0/2370945724 conn(0x7f0384076b70 msgr2=0x7f0384078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:24.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.635+0000 7f0388a00640 1 -- 172.21.15.47:0/2370945724 shutdown_connections 2024-03-21T11:28:24.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.635+0000 7f0388a00640 1 -- 172.21.15.47:0/2370945724 wait complete. 2024-03-21T11:28:24.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.636+0000 7f0388a00640 1 Processor -- start 2024-03-21T11:28:24.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.636+0000 7f0388a00640 1 -- start start 2024-03-21T11:28:24.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.636+0000 7f0388a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0384107bc0 0x7f038419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.636+0000 7f0388a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0384108db0 0x7f038419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0388a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0384109770 0x7f03841a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0388a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0384113820 con 0x7f0384107bc0 2024-03-21T11:28:24.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0388a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03841136a0 con 0x7f0384109770 2024-03-21T11:28:24.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0388a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03841139a0 con 0x7f0384108db0 2024-03-21T11:28:24.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0381e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0384108db0 0x7f038419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0382800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0384107bc0 0x7f038419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0382800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0384107bc0 0x7f038419de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51828/0 (socket says 172.21.15.47:51828) 2024-03-21T11:28:24.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0381e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0384108db0 0x7f038419e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37026/0 (socket says 172.21.15.47:37026) 2024-03-21T11:28:24.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0382800640 1 -- 172.21.15.47:0/2085065150 learned_addr learned my addr 172.21.15.47:0/2085065150 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:24.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0383200640 1 --2- 172.21.15.47:0/2085065150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0384109770 0x7f03841a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0382800640 1 -- 172.21.15.47:0/2085065150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0384108db0 msgr2=0x7f038419e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0382800640 1 --2- 172.21.15.47:0/2085065150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0384108db0 0x7f038419e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0382800640 1 -- 172.21.15.47:0/2085065150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0384109770 msgr2=0x7f03841a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0382800640 1 --2- 172.21.15.47:0/2085065150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0384109770 0x7f03841a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0382800640 1 -- 172.21.15.47:0/2085065150 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03841a5a50 con 0x7f0384107bc0 2024-03-21T11:28:24.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.637+0000 7f0381e00640 1 --2- 172.21.15.47:0/2085065150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0384108db0 0x7f038419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:24.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.638+0000 7f0382800640 1 --2- 172.21.15.47:0/2085065150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0384107bc0 0x7f038419de30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f037400c3a0 tx=0x7f037400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:24.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.638+0000 7f037b400640 1 -- 172.21.15.47:0/2085065150 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0374003970 con 0x7f0384107bc0 2024-03-21T11:28:24.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.638+0000 7f037b400640 1 -- 172.21.15.47:0/2085065150 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0374004020 con 0x7f0384107bc0 2024-03-21T11:28:24.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.638+0000 7f037b400640 1 -- 172.21.15.47:0/2085065150 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037400b9f0 con 0x7f0384107bc0 2024-03-21T11:28:24.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.638+0000 7f0388a00640 1 -- 172.21.15.47:0/2085065150 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03841a5d40 con 0x7f0384107bc0 2024-03-21T11:28:24.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.638+0000 7f0388a00640 1 -- 172.21.15.47:0/2085065150 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03841a61d0 con 0x7f0384107bc0 2024-03-21T11:28:24.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.641+0000 7f037b400640 1 -- 172.21.15.47:0/2085065150 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f037401d020 con 0x7f0384107bc0 2024-03-21T11:28:24.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.641+0000 7f038c01b740 1 Processor -- start 2024-03-21T11:28:24.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.642+0000 7f038c01b740 1 -- start start 2024-03-21T11:28:24.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.642+0000 7f038c01b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffe2b0 0x557627ffe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.642+0000 7f038c01b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557627ffeca0 0x5576280037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.642+0000 7f038c01b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557628003e10 0x5576280062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.642+0000 7f038c01b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557627f63eb0 con 0x557628003e10 2024-03-21T11:28:24.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.642+0000 7f038c01b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557627f0ca60 con 0x557627ffeca0 2024-03-21T11:28:24.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.642+0000 7f038c01b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557627f5ff50 con 0x557627ffe2b0 2024-03-21T11:28:24.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.642+0000 7f0383200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffe2b0 0x557627ffe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.642+0000 7f0382800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557627ffeca0 0x5576280037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.642+0000 7f0383200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffe2b0 0x557627ffe6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37032/0 (socket says 172.21.15.47:37032) 2024-03-21T11:28:24.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.642+0000 7f0381e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557628003e10 0x5576280062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.642+0000 7f0383200640 1 -- 172.21.15.47:0/2710697345 learned_addr learned my addr 172.21.15.47:0/2710697345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:24.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.643+0000 7f0382800640 1 -- 172.21.15.47:0/2710697345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffe2b0 msgr2=0x557627ffe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.643+0000 7f0382800640 1 --2- 172.21.15.47:0/2710697345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffe2b0 0x557627ffe6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.643+0000 7f0382800640 1 -- 172.21.15.47:0/2710697345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557628003e10 msgr2=0x5576280062a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.643+0000 7f0382800640 1 --2- 172.21.15.47:0/2710697345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557628003e10 0x5576280062a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.643+0000 7f0382800640 1 -- 172.21.15.47:0/2710697345 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557627d407e0 con 0x557627ffeca0 2024-03-21T11:28:24.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.643+0000 7f0383200640 1 --2- 172.21.15.47:0/2710697345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffe2b0 0x557627ffe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:24.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.643+0000 7f0382800640 1 --2- 172.21.15.47:0/2710697345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557627ffeca0 0x5576280037a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f03740cced0 tx=0x7f03740d5cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efe10f3401d9c939 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:24.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.643+0000 7f037b400640 1 --2- 172.21.15.47:0/2085065150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f03680a2f20 0x7f03680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.644+0000 7f0373400640 1 -- 172.21.15.47:0/2710697345 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03740de070 con 0x557627ffeca0 2024-03-21T11:28:24.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.644+0000 7f0373400640 1 -- 172.21.15.47:0/2710697345 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03740528e0 con 0x557627ffeca0 2024-03-21T11:28:24.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.644+0000 7f0373400640 1 -- 172.21.15.47:0/2710697345 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037409b6a0 con 0x557627ffeca0 2024-03-21T11:28:24.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.644+0000 7f037b400640 1 -- 172.21.15.47:0/2085065150 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f03740cf820 con 0x7f0384107bc0 2024-03-21T11:28:24.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.644+0000 7f0383200640 1 --2- 172.21.15.47:0/2085065150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f03680a2f20 0x7f03680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.645+0000 7f038c01b740 1 -- 172.21.15.47:0/2710697345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557627ffeca0 msgr2=0x5576280037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.645+0000 7f038c01b740 1 --2- 172.21.15.47:0/2710697345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557627ffeca0 0x5576280037a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f03740cced0 tx=0x7f03740d5cb0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.645+0000 7f0383200640 1 --2- 172.21.15.47:0/2085065150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f03680a2f20 0x7f03680a53e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f037c0057c0 tx=0x7f037c0056d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:24.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.645+0000 7f038c01b740 1 -- 172.21.15.47:0/2710697345 shutdown_connections 2024-03-21T11:28:24.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.645+0000 7f038c01b740 1 --2- 172.21.15.47:0/2710697345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557628003e10 0x5576280062a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.645+0000 7f038c01b740 1 --2- 172.21.15.47:0/2710697345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557627ffeca0 0x5576280037a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.645+0000 7f038c01b740 1 --2- 172.21.15.47:0/2710697345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffe2b0 0x557627ffe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.645+0000 7f038c01b740 1 -- 172.21.15.47:0/2710697345 >> 172.21.15.47:0/2710697345 conn(0x557627ff9820 msgr2=0x557627ff9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:24.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.645+0000 7f038c01b740 1 -- 172.21.15.47:0/2710697345 shutdown_connections 2024-03-21T11:28:24.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.645+0000 7f038c01b740 1 -- 172.21.15.47:0/2710697345 wait complete. 2024-03-21T11:28:24.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.645+0000 7f038c01b740 1 -- start start 2024-03-21T11:28:24.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.646+0000 7f038c01b740 1 Processor -- start 2024-03-21T11:28:24.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.646+0000 7f038c01b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557627ffe2b0 0x5576280084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.646+0000 7f038c01b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x557628008a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.646+0000 7f038c01b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557628003e10 0x55762800fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.646+0000 7f038c01b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557628006dc0 con 0x557627ffe2b0 2024-03-21T11:28:24.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.646+0000 7f038c01b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557627f1feb0 con 0x557628003e10 2024-03-21T11:28:24.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.646+0000 7f038c01b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557628006f40 con 0x557627ffeca0 2024-03-21T11:28:24.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.646+0000 7f038c01b740 10 client.0 _ll_register_callbacks cb 0x557627f14150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:24.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.646+0000 7f0383200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557628003e10 0x55762800fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.646+0000 7f0381e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x557628008a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.646+0000 7f0382800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557627ffe2b0 0x5576280084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.646+0000 7f0383200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557628003e10 0x55762800fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44878/0 (socket says 172.21.15.47:44878) 2024-03-21T11:28:24.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.646+0000 7f0381e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x557628008a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37034/0 (socket says 172.21.15.47:37034) 2024-03-21T11:28:24.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.646+0000 7f0383200640 1 -- 172.21.15.47:0/875886241 learned_addr learned my addr 172.21.15.47:0/875886241 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:24.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.647+0000 7f0383200640 1 -- 172.21.15.47:0/875886241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 msgr2=0x557628008a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.647+0000 7f0383200640 1 --2- 172.21.15.47:0/875886241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x557628008a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.647+0000 7f0383200640 1 -- 172.21.15.47:0/875886241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557627ffe2b0 msgr2=0x5576280084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.647+0000 7f0383200640 1 --2- 172.21.15.47:0/875886241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557627ffe2b0 0x5576280084f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.647+0000 7f0383200640 1 -- 172.21.15.47:0/875886241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557627da0f70 con 0x557628003e10 2024-03-21T11:28:24.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.647+0000 7f0381e00640 1 --2- 172.21.15.47:0/875886241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x557628008a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:24.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.647+0000 7f0382800640 1 --2- 172.21.15.47:0/875886241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557627ffe2b0 0x5576280084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:24.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.647+0000 7f0383200640 1 --2- 172.21.15.47:0/875886241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557628003e10 0x55762800fa70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f037c009860 tx=0x7f037c02bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7355dbbc4671576 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:24.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.647+0000 7f0367e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:24.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.647+0000 7f0367e00640 1 -- 172.21.15.47:0/875886241 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c039040 con 0x557628003e10 2024-03-21T11:28:24.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.647+0000 7f038c01b740 1 -- 172.21.15.47:0/875886241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03740528e0 con 0x557628003e10 2024-03-21T11:28:24.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.647+0000 7f0367e00640 1 -- 172.21.15.47:0/875886241 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f037c02c9e0 con 0x557628003e10 2024-03-21T11:28:24.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.648+0000 7f0367e00640 1 -- 172.21.15.47:0/875886241 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f037c058050 con 0x557628003e10 2024-03-21T11:28:24.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.648+0000 7f038c01b740 10 client.27224 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:24.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.648+0000 7f038c01b740 1 -- 172.21.15.47:0/875886241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557627c38170 con 0x557628003e10 2024-03-21T11:28:24.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.648+0000 7f0367e00640 1 -- 172.21.15.47:0/875886241 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03740528e0 con 0x557628003e10 2024-03-21T11:28:24.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.648+0000 7f0367e00640 1 -- 172.21.15.47:0/875886241 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f037c0036f0 con 0x557628003e10 2024-03-21T11:28:24.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.648+0000 7f038c01b740 10 client.27224 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:24.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.648+0000 7f038c01b740 10 client.27224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:24.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.648+0000 7f038c01b740 20 client.27224 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:24.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.648+0000 7f038c01b740 1 --2- 172.21.15.47:0/875886241 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576280142f0 0x557628016770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.648+0000 7f038c01b740 4 client.27224 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:24.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.648+0000 7f038c01b740 1 -- 172.21.15.47:0/875886241 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557627f713f0 con 0x5576280142f0 2024-03-21T11:28:24.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.649+0000 7f0382800640 1 --2- 172.21.15.47:0/875886241 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576280142f0 0x557628016770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.649+0000 7f0382800640 1 --2- 172.21.15.47:0/875886241 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576280142f0 0x557628016770 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a9efcab6d6aa3b server_cookie=7c50916b29b5d356 in_seq=0 out_seq=0 2024-03-21T11:28:24.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.649+0000 7f0367e00640 10 client.27224 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:24.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.650+0000 7f0367e00640 1 -- 172.21.15.47:0/875886241 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557627f713f0 con 0x5576280142f0 2024-03-21T11:28:24.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.650+0000 7f0367e00640 10 client.27224 handle_command_reply: tid=0 2024-03-21T11:28:24.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.650+0000 7f038c01b740 1 client.27224 shutdown 2024-03-21T11:28:24.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.650+0000 7f038c01b740 1 -- 172.21.15.47:0/875886241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557628003e10 msgr2=0x55762800fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.650+0000 7f038c01b740 1 --2- 172.21.15.47:0/875886241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557628003e10 0x55762800fa70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f037c009860 tx=0x7f037c02bb70 comp rx=0 tx=0).stop 2024-03-21T11:28:24.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.650+0000 7f038c01b740 1 -- 172.21.15.47:0/875886241 shutdown_connections 2024-03-21T11:28:24.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.650+0000 7f038c01b740 1 --2- 172.21.15.47:0/875886241 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576280142f0 0x557628016770 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.650+0000 7f038c01b740 1 --2- 172.21.15.47:0/875886241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557628003e10 0x55762800fa70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.650+0000 7f038c01b740 1 --2- 172.21.15.47:0/875886241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x557628008a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.650+0000 7f038c01b740 1 --2- 172.21.15.47:0/875886241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557627ffe2b0 0x5576280084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.650+0000 7f0367e00640 0 client.27224 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:24.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.650+0000 7f038c01b740 1 -- 172.21.15.47:0/875886241 >> 172.21.15.47:0/875886241 conn(0x557627ff9820 msgr2=0x557627ff8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:24.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.651+0000 7f038c01b740 1 -- 172.21.15.47:0/875886241 shutdown_connections 2024-03-21T11:28:24.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.651+0000 7f038c01b740 1 -- 172.21.15.47:0/875886241 wait complete. 2024-03-21T11:28:24.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.651+0000 7f038c01b740 20 client.27224 trim_cache size 0 max 16384 2024-03-21T11:28:24.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.672+0000 7f038c01b740 1 Processor -- start 2024-03-21T11:28:24.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.673+0000 7f038c01b740 1 -- start start 2024-03-21T11:28:24.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.673+0000 7f038c01b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557627ffe2b0 0x557627ff8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.673+0000 7f038c01b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x55762800fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.673+0000 7f038c01b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557628003e10 0x55762801b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.673+0000 7f038c01b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03841b0530 con 0x557628003e10 2024-03-21T11:28:24.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.673+0000 7f038c01b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03841b1eb0 con 0x557627ffe2b0 2024-03-21T11:28:24.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.673+0000 7f038c01b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03841b0b90 con 0x557627ffeca0 2024-03-21T11:28:24.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.673+0000 7f0381e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x55762800fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.673+0000 7f0381e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x55762800fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37046/0 (socket says 172.21.15.47:37046) 2024-03-21T11:28:24.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.673+0000 7f0381e00640 1 -- 172.21.15.47:0/3776408012 learned_addr learned my addr 172.21.15.47:0/3776408012 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:24.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.673+0000 7f0382800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557627ffe2b0 0x557627ff8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.673+0000 7f0383200640 1 --2- 172.21.15.47:0/3776408012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557628003e10 0x55762801b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.674+0000 7f0381e00640 1 -- 172.21.15.47:0/3776408012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557627ffe2b0 msgr2=0x557627ff8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.674+0000 7f0381e00640 1 --2- 172.21.15.47:0/3776408012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557627ffe2b0 0x557627ff8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.674+0000 7f0381e00640 1 -- 172.21.15.47:0/3776408012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557628003e10 msgr2=0x55762801b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.674+0000 7f0381e00640 1 --2- 172.21.15.47:0/3776408012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557628003e10 0x55762801b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.674+0000 7f0381e00640 1 -- 172.21.15.47:0/3776408012 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557627ffa590 con 0x557627ffeca0 2024-03-21T11:28:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.674+0000 7f0383200640 1 --2- 172.21.15.47:0/3776408012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557628003e10 0x55762801b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.674+0000 7f0381e00640 1 --2- 172.21.15.47:0/3776408012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x55762800fa30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f036c012ab0 tx=0x7f036c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e7704bf35d6c55a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.674+0000 7f0372a00640 1 -- 172.21.15.47:0/3776408012 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c022070 con 0x557627ffeca0 2024-03-21T11:28:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.674+0000 7f0372a00640 1 -- 172.21.15.47:0/3776408012 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f036c00f9e0 con 0x557627ffeca0 2024-03-21T11:28:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.674+0000 7f0372a00640 1 -- 172.21.15.47:0/3776408012 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c003af0 con 0x557627ffeca0 2024-03-21T11:28:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.675+0000 7f038c01b740 1 -- 172.21.15.47:0/3776408012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 msgr2=0x55762800fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.675+0000 7f038c01b740 1 --2- 172.21.15.47:0/3776408012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x55762800fa30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f036c012ab0 tx=0x7f036c005f90 comp rx=0 tx=0).stop 2024-03-21T11:28:24.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.675+0000 7f038c01b740 1 -- 172.21.15.47:0/3776408012 shutdown_connections 2024-03-21T11:28:24.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.675+0000 7f038c01b740 1 --2- 172.21.15.47:0/3776408012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557628003e10 0x55762801b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.675+0000 7f038c01b740 1 --2- 172.21.15.47:0/3776408012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x55762800fa30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.675+0000 7f038c01b740 1 --2- 172.21.15.47:0/3776408012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557627ffe2b0 0x557627ff8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.675+0000 7f038c01b740 1 -- 172.21.15.47:0/3776408012 >> 172.21.15.47:0/3776408012 conn(0x557627ff9820 msgr2=0x557627ff81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:24.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.675+0000 7f038c01b740 1 -- 172.21.15.47:0/3776408012 shutdown_connections 2024-03-21T11:28:24.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.675+0000 7f038c01b740 1 -- 172.21.15.47:0/3776408012 wait complete. 2024-03-21T11:28:24.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.676+0000 7f038c01b740 1 -- start start 2024-03-21T11:28:24.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.676+0000 7f038c01b740 1 Processor -- start 2024-03-21T11:28:24.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.676+0000 7f038c01b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557627ffe2b0 0x55762800f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f038c01b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x55762800f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f038c01b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557628003e10 0x55762800d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f038c01b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03841b1850 con 0x557627ffe2b0 2024-03-21T11:28:24.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f038c01b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03841b11f0 con 0x557628003e10 2024-03-21T11:28:24.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f038c01b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03841ae6f0 con 0x557627ffeca0 2024-03-21T11:28:24.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f038c01b740 10 client.0 _ll_register_callbacks cb 0x7f0374009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:24.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f0383200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557628003e10 0x55762800d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f0381e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x55762800f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f0383200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557628003e10 0x55762800d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44896/0 (socket says 172.21.15.47:44896) 2024-03-21T11:28:24.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f0383200640 1 -- 172.21.15.47:0/3392811652 learned_addr learned my addr 172.21.15.47:0/3392811652 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:24.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f0382800640 1 --2- 172.21.15.47:0/3392811652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557627ffe2b0 0x55762800f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f0383200640 1 -- 172.21.15.47:0/3392811652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 msgr2=0x55762800f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f0383200640 1 --2- 172.21.15.47:0/3392811652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x55762800f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f0383200640 1 -- 172.21.15.47:0/3392811652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557627ffe2b0 msgr2=0x55762800f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f0383200640 1 --2- 172.21.15.47:0/3392811652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557627ffe2b0 0x55762800f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.677+0000 7f0383200640 1 -- 172.21.15.47:0/3392811652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55762800e150 con 0x557628003e10 2024-03-21T11:28:24.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.678+0000 7f0383200640 1 --2- 172.21.15.47:0/3392811652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557628003e10 0x55762800d9e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f037c02c0b0 tx=0x7f037c005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e243a26603b3542 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:24.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.678+0000 7f0367e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:24.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.678+0000 7f0367e00640 1 -- 172.21.15.47:0/3392811652 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c039040 con 0x557628003e10 2024-03-21T11:28:24.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.678+0000 7f0367e00640 1 -- 172.21.15.47:0/3392811652 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f037c02c7e0 con 0x557628003e10 2024-03-21T11:28:24.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.678+0000 7f038c01b740 1 -- 172.21.15.47:0/3392811652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f036c00f9e0 con 0x557628003e10 2024-03-21T11:28:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.679+0000 7f0367e00640 1 -- 172.21.15.47:0/3392811652 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f037c06a050 con 0x557628003e10 2024-03-21T11:28:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.679+0000 7f038c01b740 10 client.27230 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.679+0000 7f038c01b740 1 -- 172.21.15.47:0/3392811652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557628070740 con 0x557628003e10 2024-03-21T11:28:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.679+0000 7f0367e00640 1 -- 172.21.15.47:0/3392811652 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f036c00f9e0 con 0x557628003e10 2024-03-21T11:28:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.679+0000 7f0367e00640 1 -- 172.21.15.47:0/3392811652 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f037c04a450 con 0x557628003e10 2024-03-21T11:28:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.679+0000 7f038c01b740 10 client.27230 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.679+0000 7f038c01b740 10 client.27230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.679+0000 7f038c01b740 20 client.27230 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.679+0000 7f038c01b740 1 --2- 172.21.15.47:0/3392811652 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576280710c0 0x5576280734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.679+0000 7f038c01b740 4 client.27230 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:28:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.679+0000 7f038c01b740 1 -- 172.21.15.47:0/3392811652 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557627f6b740 con 0x5576280710c0 2024-03-21T11:28:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.679+0000 7f0382800640 1 --2- 172.21.15.47:0/3392811652 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576280710c0 0x5576280734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.680+0000 7f0382800640 1 --2- 172.21.15.47:0/3392811652 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576280710c0 0x5576280734e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a742abaef722e658 server_cookie=1591fe5f715ebde2 in_seq=0 out_seq=0 2024-03-21T11:28:24.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.680+0000 7f0367e00640 10 client.27230 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:24.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.681+0000 7f0367e00640 1 -- 172.21.15.47:0/3392811652 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557627f6b740 con 0x5576280710c0 2024-03-21T11:28:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.681+0000 7f0367e00640 10 client.27230 handle_command_reply: tid=0 2024-03-21T11:28:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.681+0000 7f038c01b740 1 client.27230 shutdown 2024-03-21T11:28:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.682+0000 7f038c01b740 1 -- 172.21.15.47:0/3392811652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557628003e10 msgr2=0x55762800d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.682+0000 7f038c01b740 1 --2- 172.21.15.47:0/3392811652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557628003e10 0x55762800d9e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f037c02c0b0 tx=0x7f037c005970 comp rx=0 tx=0).stop 2024-03-21T11:28:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.682+0000 7f038c01b740 1 -- 172.21.15.47:0/3392811652 shutdown_connections 2024-03-21T11:28:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.682+0000 7f038c01b740 1 --2- 172.21.15.47:0/3392811652 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576280710c0 0x5576280734e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.682+0000 7f038c01b740 1 --2- 172.21.15.47:0/3392811652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557628003e10 0x55762800d9e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.682+0000 7f038c01b740 1 --2- 172.21.15.47:0/3392811652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557627ffeca0 0x55762800f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.682+0000 7f038c01b740 1 --2- 172.21.15.47:0/3392811652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557627ffe2b0 0x55762800f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.682+0000 7f0367e00640 0 client.27230 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.682+0000 7f038c01b740 1 -- 172.21.15.47:0/3392811652 >> 172.21.15.47:0/3392811652 conn(0x557627ff9820 msgr2=0x557627ff8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.682+0000 7f038c01b740 1 -- 172.21.15.47:0/3392811652 shutdown_connections 2024-03-21T11:28:24.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.682+0000 7f038c01b740 1 -- 172.21.15.47:0/3392811652 wait complete. 2024-03-21T11:28:24.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.682+0000 7f038c01b740 20 client.27230 trim_cache size 0 max 16384 2024-03-21T11:28:24.686 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:28:24.686 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:28:24.686 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "4ff702c3-106e-44c7-8a26-aeb50f6ba8a1", 2024-03-21T11:28:24.686 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:28:24.686 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:28:24.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.683+0000 7f0388a00640 1 -- 172.21.15.47:0/2085065150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f03680a2f20 msgr2=0x7f03680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.683+0000 7f0388a00640 1 --2- 172.21.15.47:0/2085065150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f03680a2f20 0x7f03680a53e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f037c0057c0 tx=0x7f037c0056d0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.683+0000 7f0388a00640 1 -- 172.21.15.47:0/2085065150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0384107bc0 msgr2=0x7f038419de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.683+0000 7f0388a00640 1 --2- 172.21.15.47:0/2085065150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0384107bc0 0x7f038419de30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f037400c3a0 tx=0x7f037400c860 comp rx=0 tx=0).stop 2024-03-21T11:28:24.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.683+0000 7f0388a00640 1 -- 172.21.15.47:0/2085065150 shutdown_connections 2024-03-21T11:28:24.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.683+0000 7f0388a00640 1 --2- 172.21.15.47:0/2085065150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f03680a2f20 0x7f03680a53e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.683+0000 7f0388a00640 1 --2- 172.21.15.47:0/2085065150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0384109770 0x7f03841a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.683+0000 7f0388a00640 1 --2- 172.21.15.47:0/2085065150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0384108db0 0x7f038419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.683+0000 7f0388a00640 1 --2- 172.21.15.47:0/2085065150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0384107bc0 0x7f038419de30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.683+0000 7f0388a00640 1 -- 172.21.15.47:0/2085065150 >> 172.21.15.47:0/2085065150 conn(0x7f0384076b70 msgr2=0x7f038410c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:24.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.683+0000 7f0388a00640 1 -- 172.21.15.47:0/2085065150 shutdown_connections 2024-03-21T11:28:24.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.683+0000 7f0388a00640 1 -- 172.21.15.47:0/2085065150 wait complete. 2024-03-21T11:28:24.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:28:24.704 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:28:24.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.914+0000 7fe3ac000640 1 -- 172.21.15.47:0/632899467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a41028c0 msgr2=0x7fe3a4102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.914+0000 7fe3ac000640 1 --2- 172.21.15.47:0/632899467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a41028c0 0x7fe3a4102d40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe390009900 tx=0x7fe39002bc40 comp rx=0 tx=0).stop 2024-03-21T11:28:24.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.914+0000 7fe3ac000640 1 -- 172.21.15.47:0/632899467 shutdown_connections 2024-03-21T11:28:24.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.914+0000 7fe3ac000640 1 --2- 172.21.15.47:0/632899467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a4103280 0x7fe3a4109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.914+0000 7fe3ac000640 1 --2- 172.21.15.47:0/632899467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a41028c0 0x7fe3a4102d40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.914+0000 7fe3ac000640 1 --2- 172.21.15.47:0/632899467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a41016d0 0x7fe3a4101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.914+0000 7fe3ac000640 1 -- 172.21.15.47:0/632899467 >> 172.21.15.47:0/632899467 conn(0x7fe3a40fcc40 msgr2=0x7fe3a40ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:24.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.914+0000 7fe3ac000640 1 -- 172.21.15.47:0/632899467 shutdown_connections 2024-03-21T11:28:24.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.914+0000 7fe3ac000640 1 -- 172.21.15.47:0/632899467 wait complete. 2024-03-21T11:28:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.915+0000 7fe3ac000640 1 Processor -- start 2024-03-21T11:28:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.916+0000 7fe3ac000640 1 -- start start 2024-03-21T11:28:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.916+0000 7fe3ac000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a41016d0 0x7fe3a4197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.916+0000 7fe3ac000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a41028c0 0x7fe3a4197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.916+0000 7fe3ac000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a4103280 0x7fe3a419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.916+0000 7fe3ac000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a410d330 con 0x7fe3a41028c0 2024-03-21T11:28:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.916+0000 7fe3ac000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a410d1b0 con 0x7fe3a41016d0 2024-03-21T11:28:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.916+0000 7fe3ac000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a410d4b0 con 0x7fe3a4103280 2024-03-21T11:28:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.916+0000 7fe3a9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a41016d0 0x7fe3a4197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.916+0000 7fe3aa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a4103280 0x7fe3a419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.916+0000 7fe3a9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a41016d0 0x7fe3a4197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44914/0 (socket says 172.21.15.47:44914) 2024-03-21T11:28:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.916+0000 7fe3a9600640 1 -- 172.21.15.47:0/3077942355 learned_addr learned my addr 172.21.15.47:0/3077942355 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.916+0000 7fe3a8c00640 1 --2- 172.21.15.47:0/3077942355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a41028c0 0x7fe3a4197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.917+0000 7fe3aa000640 1 -- 172.21.15.47:0/3077942355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a41016d0 msgr2=0x7fe3a4197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.917+0000 7fe3aa000640 1 --2- 172.21.15.47:0/3077942355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a41016d0 0x7fe3a4197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.917+0000 7fe3aa000640 1 -- 172.21.15.47:0/3077942355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a41028c0 msgr2=0x7fe3a4197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.917+0000 7fe3aa000640 1 --2- 172.21.15.47:0/3077942355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a41028c0 0x7fe3a4197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.917+0000 7fe3aa000640 1 -- 172.21.15.47:0/3077942355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a419f630 con 0x7fe3a4103280 2024-03-21T11:28:24.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.917+0000 7fe3aa000640 1 --2- 172.21.15.47:0/3077942355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a4103280 0x7fe3a419efc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00e970 tx=0x7fe39c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:24.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.917+0000 7fe3a2000640 1 -- 172.21.15.47:0/3077942355 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00ccc0 con 0x7fe3a4103280 2024-03-21T11:28:24.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.917+0000 7fe3a2000640 1 -- 172.21.15.47:0/3077942355 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe39c0035f0 con 0x7fe3a4103280 2024-03-21T11:28:24.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.917+0000 7fe3a2000640 1 -- 172.21.15.47:0/3077942355 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00ce60 con 0x7fe3a4103280 2024-03-21T11:28:24.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.917+0000 7fe3ac000640 1 -- 172.21.15.47:0/3077942355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a4073930 con 0x7fe3a4103280 2024-03-21T11:28:24.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.917+0000 7fe3ac000640 1 -- 172.21.15.47:0/3077942355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a4073e50 con 0x7fe3a4103280 2024-03-21T11:28:24.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.921+0000 7fe3aee2f740 1 Processor -- start 2024-03-21T11:28:24.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.921+0000 7fe3a2000640 1 -- 172.21.15.47:0/3077942355 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe39c0037a0 con 0x7fe3a4103280 2024-03-21T11:28:24.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.921+0000 7fe3aee2f740 1 -- start start 2024-03-21T11:28:24.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.922+0000 7fe3aee2f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2c4e80 0x5573ad2c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.922+0000 7fe3aee2f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.922+0000 7fe3aee2f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2ca9e0 0x5573ad2cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.922+0000 7fe3aee2f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad22a250 con 0x5573ad2c5870 2024-03-21T11:28:24.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.922+0000 7fe3aee2f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad1d2a30 con 0x5573ad2ca9e0 2024-03-21T11:28:24.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.922+0000 7fe3aee2f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad2262f0 con 0x5573ad2c4e80 2024-03-21T11:28:24.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.922+0000 7fe3a8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2c4e80 0x5573ad2c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.922+0000 7fe3a9600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.922+0000 7fe3a9600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2ca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51862/0 (socket says 172.21.15.47:51862) 2024-03-21T11:28:24.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.922+0000 7fe3a9600640 1 -- 172.21.15.47:0/3824046766 learned_addr learned my addr 172.21.15.47:0/3824046766 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:24.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.922+0000 7fe3a8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2c4e80 0x5573ad2c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37098/0 (socket says 172.21.15.47:37098) 2024-03-21T11:28:24.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.922+0000 7fe3a8c00640 1 --2- 172.21.15.47:0/3824046766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2ca9e0 0x5573ad2cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.923+0000 7fe3a9600640 1 -- 172.21.15.47:0/3824046766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2c4e80 msgr2=0x5573ad2c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.923+0000 7fe3a9600640 1 --2- 172.21.15.47:0/3824046766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2c4e80 0x5573ad2c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.923+0000 7fe3a9600640 1 -- 172.21.15.47:0/3824046766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2ca9e0 msgr2=0x5573ad2cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.923+0000 7fe3a9600640 1 --2- 172.21.15.47:0/3824046766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2ca9e0 0x5573ad2cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.923+0000 7fe3a9600640 1 -- 172.21.15.47:0/3824046766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573ad1bd690 con 0x5573ad2c5870 2024-03-21T11:28:24.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.923+0000 7fe3a8c00640 1 --2- 172.21.15.47:0/3824046766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2ca9e0 0x5573ad2cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:24.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.923+0000 7fe3a2000640 1 --2- 172.21.15.47:0/3077942355 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe38c0a2fa0 0x7fe38c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.923+0000 7fe3a2000640 1 -- 172.21.15.47:0/3077942355 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe39c0cf9e0 con 0x7fe3a4103280 2024-03-21T11:28:24.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.923+0000 7fe3a8c00640 1 --2- 172.21.15.47:0/3077942355 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe38c0a2fa0 0x7fe38c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.923+0000 7fe3a9600640 1 --2- 172.21.15.47:0/3824046766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2ca370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe3980115f0 tx=0x7fe398011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e8862211ddfa34 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:24.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.924+0000 7fe396600640 1 -- 172.21.15.47:0/3824046766 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398004b20 con 0x5573ad2c5870 2024-03-21T11:28:24.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.924+0000 7fe396600640 1 -- 172.21.15.47:0/3824046766 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe398004cc0 con 0x5573ad2c5870 2024-03-21T11:28:24.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.924+0000 7fe396600640 1 -- 172.21.15.47:0/3824046766 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398005460 con 0x5573ad2c5870 2024-03-21T11:28:24.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.925+0000 7fe3aee2f740 1 -- 172.21.15.47:0/3824046766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 msgr2=0x5573ad2ca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.925+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/3824046766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2ca370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe3980115f0 tx=0x7fe398011ab0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.925+0000 7fe3a8c00640 1 --2- 172.21.15.47:0/3077942355 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe38c0a2fa0 0x7fe38c0a5460 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe390006010 tx=0x7fe39002bc40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:24.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.925+0000 7fe3aee2f740 1 -- 172.21.15.47:0/3824046766 shutdown_connections 2024-03-21T11:28:24.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.925+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/3824046766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2ca9e0 0x5573ad2cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.925+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/3824046766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2ca370 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.925+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/3824046766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2c4e80 0x5573ad2c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.925+0000 7fe3aee2f740 1 -- 172.21.15.47:0/3824046766 >> 172.21.15.47:0/3824046766 conn(0x5573ad28d200 msgr2=0x5573ad2c07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:24.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.925+0000 7fe3aee2f740 1 -- 172.21.15.47:0/3824046766 shutdown_connections 2024-03-21T11:28:24.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.925+0000 7fe3aee2f740 1 -- 172.21.15.47:0/3824046766 wait complete. 2024-03-21T11:28:24.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.925+0000 7fe3aee2f740 1 -- start start 2024-03-21T11:28:24.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.925+0000 7fe3aee2f740 1 Processor -- start 2024-03-21T11:28:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.926+0000 7fe3aee2f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2c4e80 0x5573ad2ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.926+0000 7fe3aee2f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.926+0000 7fe3aee2f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2ca9e0 0x5573ad2d62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.926+0000 7fe3aee2f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad2cd990 con 0x5573ad2c5870 2024-03-21T11:28:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.926+0000 7fe3aee2f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad1e5e80 con 0x5573ad2c4e80 2024-03-21T11:28:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.926+0000 7fe3aee2f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad2cdb10 con 0x5573ad2ca9e0 2024-03-21T11:28:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.926+0000 7fe3aee2f740 10 client.0 _ll_register_callbacks cb 0x5573ad1f8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.926+0000 7fe3a9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2c4e80 0x5573ad2ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.926+0000 7fe3aa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2ca9e0 0x5573ad2d62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.926+0000 7fe3a8c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.926+0000 7fe3a9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2c4e80 0x5573ad2ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44928/0 (socket says 172.21.15.47:44928) 2024-03-21T11:28:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.926+0000 7fe3a9600640 1 -- 172.21.15.47:0/908530713 learned_addr learned my addr 172.21.15.47:0/908530713 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.927+0000 7fe3a9600640 1 -- 172.21.15.47:0/908530713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2ca9e0 msgr2=0x5573ad2d62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.927+0000 7fe3a9600640 1 --2- 172.21.15.47:0/908530713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2ca9e0 0x5573ad2d62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.927+0000 7fe3a9600640 1 -- 172.21.15.47:0/908530713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 msgr2=0x5573ad2cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.927+0000 7fe3a9600640 1 --2- 172.21.15.47:0/908530713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.927+0000 7fe3a9600640 1 -- 172.21.15.47:0/908530713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573ad0087d0 con 0x5573ad2c4e80 2024-03-21T11:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.927+0000 7fe3a9600640 1 --2- 172.21.15.47:0/908530713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2c4e80 0x5573ad2ced30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe398005c40 tx=0x7fe39800fe20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cab7a6ae9367294 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.927+0000 7fe38aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.927+0000 7fe38aa00640 1 -- 172.21.15.47:0/908530713 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398028070 con 0x5573ad2c4e80 2024-03-21T11:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.927+0000 7fe3aee2f740 1 -- 172.21.15.47:0/908530713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe398004cc0 con 0x5573ad2c4e80 2024-03-21T11:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.927+0000 7fe38aa00640 1 -- 172.21.15.47:0/908530713 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe398003970 con 0x5573ad2c4e80 2024-03-21T11:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.928+0000 7fe38aa00640 1 -- 172.21.15.47:0/908530713 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe398032050 con 0x5573ad2c4e80 2024-03-21T11:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.928+0000 7fe3aee2f740 10 client.27242 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.928+0000 7fe3aee2f740 1 -- 172.21.15.47:0/908530713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5573acf3e3f0 con 0x5573ad2c4e80 2024-03-21T11:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.928+0000 7fe38aa00640 1 -- 172.21.15.47:0/908530713 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe398004cc0 con 0x5573ad2c4e80 2024-03-21T11:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.928+0000 7fe38aa00640 1 -- 172.21.15.47:0/908530713 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe3980051a0 con 0x5573ad2c4e80 2024-03-21T11:28:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.928+0000 7fe3aee2f740 10 client.27242 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.928+0000 7fe3aee2f740 10 client.27242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.928+0000 7fe3aee2f740 20 client.27242 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.928+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/908530713 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5573ad2da900 0x5573ad2dcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.928+0000 7fe3aee2f740 4 client.27242 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.928+0000 7fe3aee2f740 1 -- 172.21.15.47:0/908530713 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573ad237790 con 0x5573ad2da900 2024-03-21T11:28:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.929+0000 7fe3a8c00640 1 --2- 172.21.15.47:0/908530713 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5573ad2da900 0x5573ad2dcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.929+0000 7fe3a8c00640 1 --2- 172.21.15.47:0/908530713 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5573ad2da900 0x5573ad2dcd30 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ebc141620a7c05f server_cookie=6959201f0f382fc in_seq=0 out_seq=0 2024-03-21T11:28:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.929+0000 7fe38aa00640 10 client.27242 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.930+0000 7fe38aa00640 1 -- 172.21.15.47:0/908530713 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5573ad237790 con 0x5573ad2da900 2024-03-21T11:28:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.930+0000 7fe38aa00640 10 client.27242 handle_command_reply: tid=0 2024-03-21T11:28:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.930+0000 7fe3aee2f740 1 client.27242 shutdown 2024-03-21T11:28:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.931+0000 7fe3aee2f740 1 -- 172.21.15.47:0/908530713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2c4e80 msgr2=0x5573ad2ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.931+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/908530713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2c4e80 0x5573ad2ced30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe398005c40 tx=0x7fe39800fe20 comp rx=0 tx=0).stop 2024-03-21T11:28:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.931+0000 7fe3aee2f740 1 -- 172.21.15.47:0/908530713 shutdown_connections 2024-03-21T11:28:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.931+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/908530713 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5573ad2da900 0x5573ad2dcd30 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.931+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/908530713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2ca9e0 0x5573ad2d62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.931+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/908530713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.931+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/908530713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2c4e80 0x5573ad2ced30 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.931+0000 7fe38aa00640 0 client.27242 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.931+0000 7fe3aee2f740 1 -- 172.21.15.47:0/908530713 >> 172.21.15.47:0/908530713 conn(0x5573ad28d200 msgr2=0x5573ad2bfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.931+0000 7fe3aee2f740 1 -- 172.21.15.47:0/908530713 shutdown_connections 2024-03-21T11:28:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.931+0000 7fe3aee2f740 1 -- 172.21.15.47:0/908530713 wait complete. 2024-03-21T11:28:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.931+0000 7fe3aee2f740 20 client.27242 trim_cache size 0 max 16384 2024-03-21T11:28:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.949+0000 7fe3aee2f740 1 Processor -- start 2024-03-21T11:28:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.949+0000 7fe3aee2f740 1 -- start start 2024-03-21T11:28:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3aee2f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2c4e80 0x5573ad2c4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3aee2f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2d6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3aee2f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2ca9e0 0x5573ad2e1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3aee2f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41ac350 con 0x5573ad2c5870 2024-03-21T11:28:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3aee2f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41adcd0 con 0x5573ad2ca9e0 2024-03-21T11:28:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3aee2f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41ac9b0 con 0x5573ad2c4e80 2024-03-21T11:28:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3a9600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2c4e80 0x5573ad2c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3a9600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2c4e80 0x5573ad2c4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37112/0 (socket says 172.21.15.47:37112) 2024-03-21T11:28:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3a9600640 1 -- 172.21.15.47:0/729602668 learned_addr learned my addr 172.21.15.47:0/729602668 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3aa000640 1 --2- 172.21.15.47:0/729602668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2ca9e0 0x5573ad2e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3a8c00640 1 --2- 172.21.15.47:0/729602668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2d6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3a9600640 1 -- 172.21.15.47:0/729602668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2ca9e0 msgr2=0x5573ad2e1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3a9600640 1 --2- 172.21.15.47:0/729602668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2ca9e0 0x5573ad2e1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3a9600640 1 -- 172.21.15.47:0/729602668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 msgr2=0x5573ad2d6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3a9600640 1 --2- 172.21.15.47:0/729602668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2d6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3a9600640 1 -- 172.21.15.47:0/729602668 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573acefe0f0 con 0x5573ad2c4e80 2024-03-21T11:28:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3a8c00640 1 --2- 172.21.15.47:0/729602668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2d6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.950+0000 7fe3aa000640 1 --2- 172.21.15.47:0/729602668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2ca9e0 0x5573ad2e1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.951+0000 7fe3a9600640 1 --2- 172.21.15.47:0/729602668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2c4e80 0x5573ad2c4500 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe398011af0 tx=0x7fe398005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f825f74b28886bc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.951+0000 7fe395c00640 1 -- 172.21.15.47:0/729602668 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398028040 con 0x5573ad2c4e80 2024-03-21T11:28:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.951+0000 7fe395c00640 1 -- 172.21.15.47:0/729602668 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe398005730 con 0x5573ad2c4e80 2024-03-21T11:28:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.951+0000 7fe395c00640 1 -- 172.21.15.47:0/729602668 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398004eb0 con 0x5573ad2c4e80 2024-03-21T11:28:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.952+0000 7fe3aee2f740 1 -- 172.21.15.47:0/729602668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2c4e80 msgr2=0x5573ad2c4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.952+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/729602668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2c4e80 0x5573ad2c4500 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe398011af0 tx=0x7fe398005d30 comp rx=0 tx=0).stop 2024-03-21T11:28:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.952+0000 7fe3aee2f740 1 -- 172.21.15.47:0/729602668 shutdown_connections 2024-03-21T11:28:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.952+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/729602668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2ca9e0 0x5573ad2e1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.952+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/729602668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2d6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.952+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/729602668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2c4e80 0x5573ad2c4500 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.952+0000 7fe3aee2f740 1 -- 172.21.15.47:0/729602668 >> 172.21.15.47:0/729602668 conn(0x5573ad28d200 msgr2=0x5573ad2bfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.952+0000 7fe3aee2f740 1 -- 172.21.15.47:0/729602668 shutdown_connections 2024-03-21T11:28:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.952+0000 7fe3aee2f740 1 -- 172.21.15.47:0/729602668 wait complete. 2024-03-21T11:28:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.953+0000 7fe3aee2f740 1 -- start start 2024-03-21T11:28:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.953+0000 7fe3aee2f740 1 Processor -- start 2024-03-21T11:28:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.953+0000 7fe3aee2f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2c4e80 0x5573ad2d3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.953+0000 7fe3aee2f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2d3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.953+0000 7fe3aee2f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2ca9e0 0x5573ad2d4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.953+0000 7fe3aee2f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41ad670 con 0x5573ad2c5870 2024-03-21T11:28:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.953+0000 7fe3aee2f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41ad010 con 0x5573ad2c4e80 2024-03-21T11:28:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.953+0000 7fe3aee2f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a41aa510 con 0x5573ad2ca9e0 2024-03-21T11:28:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.953+0000 7fe3aee2f740 10 client.0 _ll_register_callbacks cb 0x7fe39800fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.953+0000 7fe3a9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2c4e80 0x5573ad2d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.953+0000 7fe3a9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2c4e80 0x5573ad2d3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44948/0 (socket says 172.21.15.47:44948) 2024-03-21T11:28:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.954+0000 7fe3a9600640 1 -- 172.21.15.47:0/1062712190 learned_addr learned my addr 172.21.15.47:0/1062712190 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.954+0000 7fe3a8c00640 1 --2- 172.21.15.47:0/1062712190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2d3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.954+0000 7fe3aa000640 1 --2- 172.21.15.47:0/1062712190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2ca9e0 0x5573ad2d4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.954+0000 7fe3a9600640 1 -- 172.21.15.47:0/1062712190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2ca9e0 msgr2=0x5573ad2d4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.954+0000 7fe3a9600640 1 --2- 172.21.15.47:0/1062712190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2ca9e0 0x5573ad2d4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.954+0000 7fe3a9600640 1 -- 172.21.15.47:0/1062712190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 msgr2=0x5573ad2d3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.954+0000 7fe3a9600640 1 --2- 172.21.15.47:0/1062712190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2d3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.954+0000 7fe3a9600640 1 -- 172.21.15.47:0/1062712190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573ad2d5070 con 0x5573ad2c4e80 2024-03-21T11:28:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.954+0000 7fe3aa000640 1 --2- 172.21.15.47:0/1062712190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2ca9e0 0x5573ad2d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.954+0000 7fe3a9600640 1 --2- 172.21.15.47:0/1062712190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2c4e80 0x5573ad2d3530 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe398002230 tx=0x7fe398009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34e62bac12e25d47 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.954+0000 7fe38aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.954+0000 7fe38aa00640 1 -- 172.21.15.47:0/1062712190 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398028040 con 0x5573ad2c4e80 2024-03-21T11:28:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.955+0000 7fe3aee2f740 1 -- 172.21.15.47:0/1062712190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe398005730 con 0x5573ad2c4e80 2024-03-21T11:28:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.955+0000 7fe38aa00640 1 -- 172.21.15.47:0/1062712190 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe398003b80 con 0x5573ad2c4e80 2024-03-21T11:28:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.955+0000 7fe38aa00640 1 -- 172.21.15.47:0/1062712190 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe398042050 con 0x5573ad2c4e80 2024-03-21T11:28:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.955+0000 7fe3aee2f740 10 client.27248 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.955+0000 7fe3aee2f740 1 -- 172.21.15.47:0/1062712190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5573ad2d6380 con 0x5573ad2c4e80 2024-03-21T11:28:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.955+0000 7fe38aa00640 1 -- 172.21.15.47:0/1062712190 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe398005730 con 0x5573ad2c4e80 2024-03-21T11:28:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.955+0000 7fe38aa00640 1 -- 172.21.15.47:0/1062712190 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe398004190 con 0x5573ad2c4e80 2024-03-21T11:28:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.955+0000 7fe3aee2f740 10 client.27248 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.955+0000 7fe3aee2f740 10 client.27248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.956+0000 7fe3aee2f740 20 client.27248 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.956+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/1062712190 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5573ad336610 0x5573ad338a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.956+0000 7fe3aee2f740 4 client.27248 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:28:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.956+0000 7fe3aee2f740 1 -- 172.21.15.47:0/1062712190 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5573ad231ae0 con 0x5573ad336610 2024-03-21T11:28:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.956+0000 7fe3a8c00640 1 --2- 172.21.15.47:0/1062712190 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5573ad336610 0x5573ad338a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.956+0000 7fe3a8c00640 1 --2- 172.21.15.47:0/1062712190 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5573ad336610 0x5573ad338a30 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f6b7cd215f43b6b server_cookie=67a592582fc107ff in_seq=0 out_seq=0 2024-03-21T11:28:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.956+0000 7fe38aa00640 10 client.27248 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.957+0000 7fe38aa00640 1 -- 172.21.15.47:0/1062712190 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5573ad231ae0 con 0x5573ad336610 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.957+0000 7fe38aa00640 10 client.27248 handle_command_reply: tid=0 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.957+0000 7fe3aee2f740 1 client.27248 shutdown 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.957+0000 7fe3aee2f740 1 -- 172.21.15.47:0/1062712190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2c4e80 msgr2=0x5573ad2d3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.957+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/1062712190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2c4e80 0x5573ad2d3530 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe398002230 tx=0x7fe398009ca0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.957+0000 7fe3aee2f740 1 -- 172.21.15.47:0/1062712190 shutdown_connections 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.957+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/1062712190 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5573ad336610 0x5573ad338a30 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.957+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/1062712190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573ad2ca9e0 0x5573ad2d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.957+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/1062712190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573ad2c5870 0x5573ad2d3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.957+0000 7fe38aa00640 0 client.27248 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.957+0000 7fe3aee2f740 1 --2- 172.21.15.47:0/1062712190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573ad2c4e80 0x5573ad2d3530 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.957+0000 7fe3aee2f740 1 -- 172.21.15.47:0/1062712190 >> 172.21.15.47:0/1062712190 conn(0x5573ad28d200 msgr2=0x5573ad2c5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.957+0000 7fe3aee2f740 1 -- 172.21.15.47:0/1062712190 shutdown_connections 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.957+0000 7fe3aee2f740 1 -- 172.21.15.47:0/1062712190 wait complete. 2024-03-21T11:28:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.958+0000 7fe3aee2f740 20 client.27248 trim_cache size 0 max 16384 2024-03-21T11:28:24.962 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:28:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:28:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:28:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "4ff702c3-106e-44c7-8a26-aeb50f6ba8a1": { 2024-03-21T11:28:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:28:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "4ff702c3-106e-44c7-8a26-aeb50f6ba8a1", 2024-03-21T11:28:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:28:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:28:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:28:24.962 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:28:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.958+0000 7fe3ac000640 1 -- 172.21.15.47:0/3077942355 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe38c0a2fa0 msgr2=0x7fe38c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.958+0000 7fe3ac000640 1 --2- 172.21.15.47:0/3077942355 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe38c0a2fa0 0x7fe38c0a5460 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe390006010 tx=0x7fe39002bc40 comp rx=0 tx=0).stop 2024-03-21T11:28:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.958+0000 7fe3ac000640 1 -- 172.21.15.47:0/3077942355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a4103280 msgr2=0x7fe3a419efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.958+0000 7fe3ac000640 1 --2- 172.21.15.47:0/3077942355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a4103280 0x7fe3a419efc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00e970 tx=0x7fe39c00ee30 comp rx=0 tx=0).stop 2024-03-21T11:28:24.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.958+0000 7fe3ac000640 1 -- 172.21.15.47:0/3077942355 shutdown_connections 2024-03-21T11:28:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.958+0000 7fe3ac000640 1 --2- 172.21.15.47:0/3077942355 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe38c0a2fa0 0x7fe38c0a5460 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.958+0000 7fe3ac000640 1 --2- 172.21.15.47:0/3077942355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3a4103280 0x7fe3a419efc0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.958+0000 7fe3ac000640 1 --2- 172.21.15.47:0/3077942355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3a41028c0 0x7fe3a4197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.958+0000 7fe3ac000640 1 --2- 172.21.15.47:0/3077942355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a41016d0 0x7fe3a4197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.958+0000 7fe3ac000640 1 -- 172.21.15.47:0/3077942355 >> 172.21.15.47:0/3077942355 conn(0x7fe3a40fcc40 msgr2=0x7fe3a4105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.958+0000 7fe3ac000640 1 -- 172.21.15.47:0/3077942355 shutdown_connections 2024-03-21T11:28:24.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:24.958+0000 7fe3ac000640 1 -- 172.21.15.47:0/3077942355 wait complete. 2024-03-21T11:28:24.976 INFO:tasks.cephfs.filesystem:scrub status for tag:4ff702c3-106e-44c7-8a26-aeb50f6ba8a1 - {'path': '/', 'tag': '4ff702c3-106e-44c7-8a26-aeb50f6ba8a1', 'options': 'recursive,force'} 2024-03-21T11:28:25.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:25 smithi047 ceph-mon[31351]: pgmap v552: 129 pgs: 129 active+clean; 36 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 315 op/s 2024-03-21T11:28:25.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:25 smithi100 ceph-mon[36245]: pgmap v552: 129 pgs: 129 active+clean; 36 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 315 op/s 2024-03-21T11:28:25.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:25 smithi102 ceph-mon[36748]: pgmap v552: 129 pgs: 129 active+clean; 36 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 315 op/s 2024-03-21T11:28:25.511 INFO:tasks.workunit.client.0.smithi047.stdout: 1 302349 8.34 MB/sec execute 69 sec latency 11.073 ms 2024-03-21T11:28:26.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:26 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:28:26.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:28:26.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:28:26.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:28:26.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:26 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:28:26.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:28:26.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:28:26.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:28:26.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:26 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:28:26.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:28:26.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:28:26.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:28:26.511 INFO:tasks.workunit.client.0.smithi047.stdout: 1 304299 8.38 MB/sec execute 70 sec latency 28.882 ms 2024-03-21T11:28:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:27 smithi047 ceph-mon[31351]: pgmap v553: 129 pgs: 129 active+clean; 36 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 2.0 MiB/s wr, 257 op/s 2024-03-21T11:28:27.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:27 smithi100 ceph-mon[36245]: pgmap v553: 129 pgs: 129 active+clean; 36 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 2.0 MiB/s wr, 257 op/s 2024-03-21T11:28:27.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:27 smithi102 ceph-mon[36748]: pgmap v553: 129 pgs: 129 active+clean; 36 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 2.0 MiB/s wr, 257 op/s 2024-03-21T11:28:27.511 INFO:tasks.workunit.client.0.smithi047.stdout: 1 305456 8.37 MB/sec execute 71 sec latency 55.579 ms 2024-03-21T11:28:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:28 smithi047 ceph-mon[31351]: pgmap v554: 129 pgs: 129 active+clean; 41 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-03-21T11:28:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:28 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:28:28.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:28 smithi100 ceph-mon[36245]: pgmap v554: 129 pgs: 129 active+clean; 41 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-03-21T11:28:28.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:28 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:28:28.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:28 smithi102 ceph-mon[36748]: pgmap v554: 129 pgs: 129 active+clean; 41 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-03-21T11:28:28.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:28 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:28:28.511 INFO:tasks.workunit.client.0.smithi047.stdout: 1 306760 8.33 MB/sec execute 72 sec latency 13.065 ms 2024-03-21T11:28:29.511 INFO:tasks.workunit.client.0.smithi047.stdout: 1 308901 8.40 MB/sec execute 73 sec latency 9.482 ms 2024-03-21T11:28:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.924+0000 7f6f58000640 1 -- 172.21.15.47:0/2357692880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50104af0 msgr2=0x7f6f50104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.924+0000 7f6f58000640 1 --2- 172.21.15.47:0/2357692880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50104af0 0x7f6f50104f70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009900 tx=0x7f6f3c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:28:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.925+0000 7f6f58000640 1 -- 172.21.15.47:0/2357692880 shutdown_connections 2024-03-21T11:28:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.925+0000 7f6f58000640 1 --2- 172.21.15.47:0/2357692880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f501054b0 0x7f6f5010bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.925+0000 7f6f58000640 1 --2- 172.21.15.47:0/2357692880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50104af0 0x7f6f50104f70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.925+0000 7f6f58000640 1 --2- 172.21.15.47:0/2357692880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f50103900 0x7f6f50103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.925+0000 7f6f58000640 1 -- 172.21.15.47:0/2357692880 >> 172.21.15.47:0/2357692880 conn(0x7f6f500fee90 msgr2=0x7f6f501012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.925+0000 7f6f58000640 1 -- 172.21.15.47:0/2357692880 shutdown_connections 2024-03-21T11:28:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.925+0000 7f6f58000640 1 -- 172.21.15.47:0/2357692880 wait complete. 2024-03-21T11:28:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.926+0000 7f6f58000640 1 Processor -- start 2024-03-21T11:28:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.926+0000 7f6f58000640 1 -- start start 2024-03-21T11:28:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.926+0000 7f6f58000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f50103900 0x7f6f50199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.926+0000 7f6f58000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50104af0 0x7f6f5019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.926+0000 7f6f58000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f501054b0 0x7f6f501a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.926+0000 7f6f58000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5010f580 con 0x7f6f50103900 2024-03-21T11:28:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.926+0000 7f6f58000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5010f400 con 0x7f6f50104af0 2024-03-21T11:28:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.926+0000 7f6f58000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5010f700 con 0x7f6f501054b0 2024-03-21T11:28:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.927+0000 7f6f54c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50104af0 0x7f6f5019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.927+0000 7f6f54c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50104af0 0x7f6f5019a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36758/0 (socket says 172.21.15.47:36758) 2024-03-21T11:28:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.927+0000 7f6f54c00640 1 -- 172.21.15.47:0/1445863364 learned_addr learned my addr 172.21.15.47:0/1445863364 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.927+0000 7f6f56000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f501054b0 0x7f6f501a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.927+0000 7f6f55600640 1 --2- 172.21.15.47:0/1445863364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f50103900 0x7f6f50199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.927+0000 7f6f54c00640 1 -- 172.21.15.47:0/1445863364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f501054b0 msgr2=0x7f6f501a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.927+0000 7f6f54c00640 1 --2- 172.21.15.47:0/1445863364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f501054b0 0x7f6f501a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.927+0000 7f6f54c00640 1 -- 172.21.15.47:0/1445863364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f50103900 msgr2=0x7f6f50199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.927+0000 7f6f54c00640 1 --2- 172.21.15.47:0/1445863364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f50103900 0x7f6f50199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.927+0000 7f6f54c00640 1 -- 172.21.15.47:0/1445863364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f501a1720 con 0x7f6f50104af0 2024-03-21T11:28:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.928+0000 7f6f54c00640 1 --2- 172.21.15.47:0/1445863364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50104af0 0x7f6f5019a040 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009230 tx=0x7f6f3c02c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.928+0000 7f6f4e000640 1 -- 172.21.15.47:0/1445863364 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c041070 con 0x7f6f50104af0 2024-03-21T11:28:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.928+0000 7f6f4e000640 1 -- 172.21.15.47:0/1445863364 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f3c004450 con 0x7f6f50104af0 2024-03-21T11:28:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.928+0000 7f6f58000640 1 -- 172.21.15.47:0/1445863364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f501a19b0 con 0x7f6f50104af0 2024-03-21T11:28:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.928+0000 7f6f4e000640 1 -- 172.21.15.47:0/1445863364 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c003e00 con 0x7f6f50104af0 2024-03-21T11:28:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.928+0000 7f6f58000640 1 -- 172.21.15.47:0/1445863364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f501a1ef0 con 0x7f6f50104af0 2024-03-21T11:28:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.931+0000 7f6f5ada9740 1 Processor -- start 2024-03-21T11:28:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.932+0000 7f6f4e000640 1 -- 172.21.15.47:0/1445863364 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6f3c044070 con 0x7f6f50104af0 2024-03-21T11:28:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.932+0000 7f6f5ada9740 1 -- start start 2024-03-21T11:28:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.932+0000 7f6f5ada9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3679111e0 0x55b367911600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.932+0000 7f6f5ada9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367911bd0 0x55b3679166d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.932+0000 7f6f5ada9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b367916d40 0x55b3679191d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.932+0000 7f6f5ada9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3678d43b0 con 0x55b3679111e0 2024-03-21T11:28:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.932+0000 7f6f5ada9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b367876830 con 0x55b367911bd0 2024-03-21T11:28:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.932+0000 7f6f5ada9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36781f5d0 con 0x55b367916d40 2024-03-21T11:28:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.932+0000 7f6f54c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b367916d40 0x55b3679191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.932+0000 7f6f55600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367911bd0 0x55b3679166d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.932+0000 7f6f56000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3679111e0 0x55b367911600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.932+0000 7f6f55600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367911bd0 0x55b3679166d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36772/0 (socket says 172.21.15.47:36772) 2024-03-21T11:28:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.932+0000 7f6f55600640 1 -- 172.21.15.47:0/265201764 learned_addr learned my addr 172.21.15.47:0/265201764 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.933+0000 7f6f55600640 1 -- 172.21.15.47:0/265201764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b367916d40 msgr2=0x55b3679191d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.933+0000 7f6f55600640 1 --2- 172.21.15.47:0/265201764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b367916d40 0x55b3679191d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.933+0000 7f6f55600640 1 -- 172.21.15.47:0/265201764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3679111e0 msgr2=0x55b367911600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.933+0000 7f6f55600640 1 --2- 172.21.15.47:0/265201764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3679111e0 0x55b367911600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.933+0000 7f6f55600640 1 -- 172.21.15.47:0/265201764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3678372c0 con 0x55b367911bd0 2024-03-21T11:28:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.933+0000 7f6f55600640 1 --2- 172.21.15.47:0/265201764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367911bd0 0x55b3679166d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009ca0 tx=0x7f6f44011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d43754d1a8700fc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.933+0000 7f6f42600640 1 -- 172.21.15.47:0/265201764 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44004220 con 0x55b367911bd0 2024-03-21T11:28:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.933+0000 7f6f42600640 1 -- 172.21.15.47:0/265201764 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f440043c0 con 0x55b367911bd0 2024-03-21T11:28:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.933+0000 7f6f42600640 1 -- 172.21.15.47:0/265201764 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f440198d0 con 0x55b367911bd0 2024-03-21T11:28:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.933+0000 7f6f4e000640 1 --2- 172.21.15.47:0/1445863364 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6f380a3040 0x7f6f380a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.934+0000 7f6f4e000640 1 -- 172.21.15.47:0/1445863364 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6f3c0f1330 con 0x7f6f50104af0 2024-03-21T11:28:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.934+0000 7f6f56000640 1 --2- 172.21.15.47:0/1445863364 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6f380a3040 0x7f6f380a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.934+0000 7f6f5ada9740 1 -- 172.21.15.47:0/265201764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367911bd0 msgr2=0x55b3679166d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.934+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/265201764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367911bd0 0x55b3679166d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009ca0 tx=0x7f6f44011810 comp rx=0 tx=0).stop 2024-03-21T11:28:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.934+0000 7f6f5ada9740 1 -- 172.21.15.47:0/265201764 shutdown_connections 2024-03-21T11:28:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.934+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/265201764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b367916d40 0x55b3679191d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.934+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/265201764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367911bd0 0x55b3679166d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.934+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/265201764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3679111e0 0x55b367911600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.934+0000 7f6f5ada9740 1 -- 172.21.15.47:0/265201764 >> 172.21.15.47:0/265201764 conn(0x55b36790c750 msgr2=0x55b36790cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.934+0000 7f6f5ada9740 1 -- 172.21.15.47:0/265201764 shutdown_connections 2024-03-21T11:28:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.935+0000 7f6f5ada9740 1 -- 172.21.15.47:0/265201764 wait complete. 2024-03-21T11:28:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.935+0000 7f6f56000640 1 --2- 172.21.15.47:0/1445863364 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6f380a3040 0x7f6f380a5500 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55b367912430 tx=0x7f6f48009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.935+0000 7f6f5ada9740 1 -- start start 2024-03-21T11:28:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.935+0000 7f6f5ada9740 1 Processor -- start 2024-03-21T11:28:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f5ada9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3679111e0 0x55b36791b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f5ada9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 0x55b36791b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f5ada9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367916d40 0x55b3679228e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f5ada9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b367832a20 con 0x55b367911bd0 2024-03-21T11:28:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f5ada9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3678728d0 con 0x55b367916d40 2024-03-21T11:28:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f5ada9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b367919cf0 con 0x55b3679111e0 2024-03-21T11:28:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f5ada9740 10 client.0 _ll_register_callbacks cb 0x55b36786f1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f55600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3679111e0 0x55b36791b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f56000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367916d40 0x55b3679228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f55600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3679111e0 0x55b36791b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35088/0 (socket says 172.21.15.47:35088) 2024-03-21T11:28:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f56000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367916d40 0x55b3679228e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36786/0 (socket says 172.21.15.47:36786) 2024-03-21T11:28:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f55600640 1 -- 172.21.15.47:0/1193753000 learned_addr learned my addr 172.21.15.47:0/1193753000 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f54c00640 1 --2- 172.21.15.47:0/1193753000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 0x55b36791b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f56000640 1 -- 172.21.15.47:0/1193753000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3679111e0 msgr2=0x55b36791b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f56000640 1 --2- 172.21.15.47:0/1193753000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3679111e0 0x55b36791b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f56000640 1 -- 172.21.15.47:0/1193753000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 msgr2=0x55b36791b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f56000640 1 --2- 172.21.15.47:0/1193753000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 0x55b36791b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f56000640 1 -- 172.21.15.47:0/1193753000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b36758b210 con 0x55b367916d40 2024-03-21T11:28:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.936+0000 7f6f55600640 1 --2- 172.21.15.47:0/1193753000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3679111e0 0x55b36791b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.937+0000 7f6f56000640 1 --2- 172.21.15.47:0/1193753000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367916d40 0x55b3679228e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6f48012010 tx=0x7f6f48004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e184879b01b73d44 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.937+0000 7f6f36a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.937+0000 7f6f36a00640 1 -- 172.21.15.47:0/1193753000 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4800f7b0 con 0x55b367916d40 2024-03-21T11:28:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.937+0000 7f6f36a00640 1 -- 172.21.15.47:0/1193753000 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6f4800f950 con 0x55b367916d40 2024-03-21T11:28:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.937+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1193753000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f440043c0 con 0x55b367916d40 2024-03-21T11:28:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.937+0000 7f6f36a00640 1 -- 172.21.15.47:0/1193753000 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f4800fb50 con 0x55b367916d40 2024-03-21T11:28:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.937+0000 7f6f5ada9740 10 client.27260 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.937+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1193753000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b367657620 con 0x55b367916d40 2024-03-21T11:28:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.938+0000 7f6f36a00640 1 -- 172.21.15.47:0/1193753000 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f440043c0 con 0x55b367916d40 2024-03-21T11:28:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.938+0000 7f6f36a00640 1 -- 172.21.15.47:0/1193753000 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6f48026070 con 0x55b367916d40 2024-03-21T11:28:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.938+0000 7f6f5ada9740 10 client.27260 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.938+0000 7f6f5ada9740 10 client.27260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.938+0000 7f6f5ada9740 20 client.27260 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.938+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/1193753000 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b367927280 0x55b3679296b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.938+0000 7f6f5ada9740 4 client.27260 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.938+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1193753000 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b367883d70 con 0x55b367927280 2024-03-21T11:28:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.939+0000 7f6f55600640 1 --2- 172.21.15.47:0/1193753000 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b367927280 0x55b3679296b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.939+0000 7f6f55600640 1 --2- 172.21.15.47:0/1193753000 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b367927280 0x55b3679296b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd6637708deeafba server_cookie=b6b045230257adb3 in_seq=0 out_seq=0 2024-03-21T11:28:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.939+0000 7f6f36a00640 10 client.27260 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.940+0000 7f6f36a00640 1 -- 172.21.15.47:0/1193753000 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b367883d70 con 0x55b367927280 2024-03-21T11:28:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.940+0000 7f6f36a00640 10 client.27260 handle_command_reply: tid=0 2024-03-21T11:28:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.940+0000 7f6f5ada9740 1 client.27260 shutdown 2024-03-21T11:28:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.941+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1193753000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367916d40 msgr2=0x55b3679228e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.941+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/1193753000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367916d40 0x55b3679228e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6f48012010 tx=0x7f6f48004ab0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.941+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1193753000 shutdown_connections 2024-03-21T11:28:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.941+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/1193753000 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b367927280 0x55b3679296b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.941+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/1193753000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367916d40 0x55b3679228e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.941+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/1193753000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 0x55b36791b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.941+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/1193753000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3679111e0 0x55b36791b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.941+0000 7f6f36a00640 0 client.27260 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.941+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1193753000 >> 172.21.15.47:0/1193753000 conn(0x55b36790c750 msgr2=0x55b36790b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.941+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1193753000 shutdown_connections 2024-03-21T11:28:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.941+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1193753000 wait complete. 2024-03-21T11:28:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.941+0000 7f6f5ada9740 20 client.27260 trim_cache size 0 max 16384 2024-03-21T11:28:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.960+0000 7f6f5ada9740 1 Processor -- start 2024-03-21T11:28:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.960+0000 7f6f5ada9740 1 -- start start 2024-03-21T11:28:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.960+0000 7f6f5ada9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3679111e0 0x55b367912e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.960+0000 7f6f5ada9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 0x55b367913390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.960+0000 7f6f5ada9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367916d40 0x55b367922870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.960+0000 7f6f5ada9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501ac230 con 0x55b367911bd0 2024-03-21T11:28:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.960+0000 7f6f5ada9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501adbb0 con 0x55b367916d40 2024-03-21T11:28:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.960+0000 7f6f5ada9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501ac890 con 0x55b3679111e0 2024-03-21T11:28:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.960+0000 7f6f55600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3679111e0 0x55b367912e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.960+0000 7f6f56000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367916d40 0x55b367922870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.960+0000 7f6f55600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3679111e0 0x55b367912e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35100/0 (socket says 172.21.15.47:35100) 2024-03-21T11:28:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.960+0000 7f6f55600640 1 -- 172.21.15.47:0/2927918539 learned_addr learned my addr 172.21.15.47:0/2927918539 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.961+0000 7f6f54c00640 1 --2- 172.21.15.47:0/2927918539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 0x55b367913390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.961+0000 7f6f55600640 1 -- 172.21.15.47:0/2927918539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367916d40 msgr2=0x55b367922870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.961+0000 7f6f55600640 1 --2- 172.21.15.47:0/2927918539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367916d40 0x55b367922870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.961+0000 7f6f55600640 1 -- 172.21.15.47:0/2927918539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 msgr2=0x55b367913390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.961+0000 7f6f55600640 1 --2- 172.21.15.47:0/2927918539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 0x55b367913390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.961+0000 7f6f55600640 1 -- 172.21.15.47:0/2927918539 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b36791a420 con 0x55b3679111e0 2024-03-21T11:28:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.961+0000 7f6f54c00640 1 --2- 172.21.15.47:0/2927918539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 0x55b367913390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.961+0000 7f6f55600640 1 --2- 172.21.15.47:0/2927918539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3679111e0 0x55b367912e50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6f44013010 tx=0x7f6f44003dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd592b995d18f649 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.961+0000 7f6f41c00640 1 -- 172.21.15.47:0/2927918539 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4402c310 con 0x55b3679111e0 2024-03-21T11:28:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.962+0000 7f6f41c00640 1 -- 172.21.15.47:0/2927918539 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f44003b50 con 0x55b3679111e0 2024-03-21T11:28:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.962+0000 7f6f41c00640 1 -- 172.21.15.47:0/2927918539 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4402c610 con 0x55b3679111e0 2024-03-21T11:28:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.962+0000 7f6f5ada9740 1 -- 172.21.15.47:0/2927918539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3679111e0 msgr2=0x55b367912e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.962+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/2927918539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3679111e0 0x55b367912e50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6f44013010 tx=0x7f6f44003dd0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.962+0000 7f6f5ada9740 1 -- 172.21.15.47:0/2927918539 shutdown_connections 2024-03-21T11:28:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.962+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/2927918539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b367916d40 0x55b367922870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.962+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/2927918539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 0x55b367913390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.962+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/2927918539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3679111e0 0x55b367912e50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.962+0000 7f6f5ada9740 1 -- 172.21.15.47:0/2927918539 >> 172.21.15.47:0/2927918539 conn(0x55b36790c750 msgr2=0x55b36790cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.963+0000 7f6f5ada9740 1 -- 172.21.15.47:0/2927918539 shutdown_connections 2024-03-21T11:28:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.963+0000 7f6f5ada9740 1 -- 172.21.15.47:0/2927918539 wait complete. 2024-03-21T11:28:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.963+0000 7f6f5ada9740 1 -- start start 2024-03-21T11:28:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.963+0000 7f6f5ada9740 1 Processor -- start 2024-03-21T11:28:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.964+0000 7f6f5ada9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3679111e0 0x55b367920230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.964+0000 7f6f5ada9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 0x55b367920770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.964+0000 7f6f5ada9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b367916d40 0x55b367921780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.964+0000 7f6f5ada9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501ad550 con 0x55b367911bd0 2024-03-21T11:28:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.964+0000 7f6f5ada9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501acef0 con 0x55b3679111e0 2024-03-21T11:28:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.964+0000 7f6f5ada9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501aa3f0 con 0x55b367916d40 2024-03-21T11:28:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.964+0000 7f6f5ada9740 10 client.0 _ll_register_callbacks cb 0x7f6f48009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.964+0000 7f6f55600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3679111e0 0x55b367920230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.964+0000 7f6f55600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3679111e0 0x55b367920230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36814/0 (socket says 172.21.15.47:36814) 2024-03-21T11:28:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.964+0000 7f6f56000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b367916d40 0x55b367921780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.964+0000 7f6f55600640 1 -- 172.21.15.47:0/1525244717 learned_addr learned my addr 172.21.15.47:0/1525244717 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.965+0000 7f6f55600640 1 -- 172.21.15.47:0/1525244717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b367916d40 msgr2=0x55b367921780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.965+0000 7f6f54c00640 1 --2- 172.21.15.47:0/1525244717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 0x55b367920770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.965+0000 7f6f55600640 1 --2- 172.21.15.47:0/1525244717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b367916d40 0x55b367921780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.965+0000 7f6f55600640 1 -- 172.21.15.47:0/1525244717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 msgr2=0x55b367920770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.965+0000 7f6f55600640 1 --2- 172.21.15.47:0/1525244717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 0x55b367920770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.965+0000 7f6f55600640 1 -- 172.21.15.47:0/1525244717 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b367921d70 con 0x55b3679111e0 2024-03-21T11:28:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.965+0000 7f6f54c00640 1 --2- 172.21.15.47:0/1525244717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 0x55b367920770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:28:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.965+0000 7f6f55600640 1 --2- 172.21.15.47:0/1525244717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3679111e0 0x55b367920230 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6f44000c00 tx=0x7f6f44005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e700960c130d0eb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.965+0000 7f6f36a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.965+0000 7f6f36a00640 1 -- 172.21.15.47:0/1525244717 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f440053d0 con 0x55b3679111e0 2024-03-21T11:28:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.965+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1525244717 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f44003b50 con 0x55b3679111e0 2024-03-21T11:28:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.965+0000 7f6f36a00640 1 -- 172.21.15.47:0/1525244717 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6f44003670 con 0x55b3679111e0 2024-03-21T11:28:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.966+0000 7f6f36a00640 1 -- 172.21.15.47:0/1525244717 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f44043050 con 0x55b3679111e0 2024-03-21T11:28:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.966+0000 7f6f5ada9740 10 client.27266 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.966+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1525244717 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b367982df0 con 0x55b3679111e0 2024-03-21T11:28:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.966+0000 7f6f36a00640 1 -- 172.21.15.47:0/1525244717 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f44003b50 con 0x55b3679111e0 2024-03-21T11:28:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.966+0000 7f6f36a00640 1 -- 172.21.15.47:0/1525244717 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6f44035460 con 0x55b3679111e0 2024-03-21T11:28:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.966+0000 7f6f5ada9740 10 client.27266 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.966+0000 7f6f5ada9740 10 client.27266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.966+0000 7f6f5ada9740 20 client.27266 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.966+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/1525244717 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b367983470 0x55b367985910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.966+0000 7f6f5ada9740 4 client.27266 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:28:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.966+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1525244717 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b36787e0c0 con 0x55b367983470 2024-03-21T11:28:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.967+0000 7f6f54c00640 1 --2- 172.21.15.47:0/1525244717 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b367983470 0x55b367985910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.967+0000 7f6f54c00640 1 --2- 172.21.15.47:0/1525244717 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b367983470 0x55b367985910 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c2432ebe89e583 server_cookie=22cc2ca189dc322d in_seq=0 out_seq=0 2024-03-21T11:28:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:29.967+0000 7f6f36a00640 10 client.27266 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.192+0000 7f6f36a00640 1 -- 172.21.15.47:0/1525244717 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b36787e0c0 con 0x55b367983470 2024-03-21T11:28:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.192+0000 7f6f36a00640 10 client.27266 handle_command_reply: tid=0 2024-03-21T11:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.192+0000 7f6f5ada9740 1 client.27266 shutdown 2024-03-21T11:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.192+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1525244717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3679111e0 msgr2=0x55b367920230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.192+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/1525244717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3679111e0 0x55b367920230 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6f44000c00 tx=0x7f6f44005f10 comp rx=0 tx=0).stop 2024-03-21T11:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.192+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1525244717 shutdown_connections 2024-03-21T11:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.192+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/1525244717 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b367983470 0x55b367985910 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.192+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/1525244717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b367916d40 0x55b367921780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.192+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/1525244717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b367911bd0 0x55b367920770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.192+0000 7f6f5ada9740 1 --2- 172.21.15.47:0/1525244717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3679111e0 0x55b367920230 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.192+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1525244717 >> 172.21.15.47:0/1525244717 conn(0x55b36790c750 msgr2=0x55b367912e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.192+0000 7f6f36a00640 0 client.27266 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.192+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1525244717 shutdown_connections 2024-03-21T11:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.192+0000 7f6f5ada9740 1 -- 172.21.15.47:0/1525244717 wait complete. 2024-03-21T11:28:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.193+0000 7f6f5ada9740 20 client.27266 trim_cache size 0 max 16384 2024-03-21T11:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:28:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:28:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.193+0000 7f6f58000640 1 -- 172.21.15.47:0/1445863364 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6f380a3040 msgr2=0x7f6f380a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.193+0000 7f6f58000640 1 --2- 172.21.15.47:0/1445863364 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6f380a3040 0x7f6f380a5500 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55b367912430 tx=0x7f6f48009500 comp rx=0 tx=0).stop 2024-03-21T11:28:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.193+0000 7f6f58000640 1 -- 172.21.15.47:0/1445863364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50104af0 msgr2=0x7f6f5019a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.193+0000 7f6f58000640 1 --2- 172.21.15.47:0/1445863364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50104af0 0x7f6f5019a040 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009230 tx=0x7f6f3c02c660 comp rx=0 tx=0).stop 2024-03-21T11:28:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.194+0000 7f6f58000640 1 -- 172.21.15.47:0/1445863364 shutdown_connections 2024-03-21T11:28:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.194+0000 7f6f58000640 1 --2- 172.21.15.47:0/1445863364 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6f380a3040 0x7f6f380a5500 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.194+0000 7f6f58000640 1 --2- 172.21.15.47:0/1445863364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f501054b0 0x7f6f501a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.194+0000 7f6f58000640 1 --2- 172.21.15.47:0/1445863364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50104af0 0x7f6f5019a040 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.194+0000 7f6f58000640 1 --2- 172.21.15.47:0/1445863364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f50103900 0x7f6f50199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.194+0000 7f6f58000640 1 -- 172.21.15.47:0/1445863364 >> 172.21.15.47:0/1445863364 conn(0x7f6f500fee90 msgr2=0x7f6f50107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.194+0000 7f6f58000640 1 -- 172.21.15.47:0/1445863364 shutdown_connections 2024-03-21T11:28:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:30.194+0000 7f6f58000640 1 -- 172.21.15.47:0/1445863364 wait complete. 2024-03-21T11:28:30.512 INFO:tasks.workunit.client.0.smithi047.stdout: 1 309800 8.32 MB/sec execute 74 sec latency 114.304 ms 2024-03-21T11:28:30.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:30 smithi047 ceph-mon[31351]: pgmap v555: 129 pgs: 129 active+clean; 40 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 263 op/s 2024-03-21T11:28:30.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:30 smithi100 ceph-mon[36245]: pgmap v555: 129 pgs: 129 active+clean; 40 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 263 op/s 2024-03-21T11:28:30.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:30 smithi102 ceph-mon[36748]: pgmap v555: 129 pgs: 129 active+clean; 40 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 263 op/s 2024-03-21T11:28:31.512 INFO:tasks.workunit.client.0.smithi047.stdout: 1 311962 8.37 MB/sec execute 75 sec latency 23.896 ms 2024-03-21T11:28:32.512 INFO:tasks.workunit.client.0.smithi047.stdout: 1 312879 8.33 MB/sec execute 76 sec latency 63.630 ms 2024-03-21T11:28:32.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:32 smithi047 ceph-mon[31351]: pgmap v556: 129 pgs: 129 active+clean; 33 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 238 op/s 2024-03-21T11:28:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:32 smithi100 ceph-mon[36245]: pgmap v556: 129 pgs: 129 active+clean; 33 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 238 op/s 2024-03-21T11:28:32.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:32 smithi102 ceph-mon[36748]: pgmap v556: 129 pgs: 129 active+clean; 33 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 238 op/s 2024-03-21T11:28:33.512 INFO:tasks.workunit.client.0.smithi047.stdout: 1 314922 8.36 MB/sec execute 77 sec latency 10.167 ms 2024-03-21T11:28:34.512 INFO:tasks.workunit.client.0.smithi047.stdout: 1 316383 8.36 MB/sec execute 78 sec latency 11.669 ms 2024-03-21T11:28:34.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:34 smithi047 ceph-mon[31351]: pgmap v557: 129 pgs: 129 active+clean; 36 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 311 op/s 2024-03-21T11:28:34.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:34 smithi100 ceph-mon[36245]: pgmap v557: 129 pgs: 129 active+clean; 36 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 311 op/s 2024-03-21T11:28:34.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:34 smithi102 ceph-mon[36748]: pgmap v557: 129 pgs: 129 active+clean; 36 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 311 op/s 2024-03-21T11:28:35.515 INFO:tasks.workunit.client.0.smithi047.stdout: 1 318266 8.35 MB/sec execute 79 sec latency 20.347 ms 2024-03-21T11:28:36.512 INFO:tasks.workunit.client.0.smithi047.stdout: 1 319846 8.39 MB/sec execute 80 sec latency 10.424 ms 2024-03-21T11:28:36.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:36 smithi047 ceph-mon[31351]: pgmap v558: 129 pgs: 129 active+clean; 36 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 312 op/s 2024-03-21T11:28:36.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:36 smithi100 ceph-mon[36245]: pgmap v558: 129 pgs: 129 active+clean; 36 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 312 op/s 2024-03-21T11:28:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:36 smithi102 ceph-mon[36748]: pgmap v558: 129 pgs: 129 active+clean; 36 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 312 op/s 2024-03-21T11:28:37.512 INFO:tasks.workunit.client.0.smithi047.stdout: 1 321593 8.37 MB/sec execute 81 sec latency 52.770 ms 2024-03-21T11:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout: 1 323210 8.41 MB/sec execute 82 sec latency 10.283 ms 2024-03-21T11:28:38.541 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:38 smithi102 ceph-mon[36748]: pgmap v559: 129 pgs: 129 active+clean; 34 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 327 op/s 2024-03-21T11:28:38.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:38 smithi047 ceph-mon[31351]: pgmap v559: 129 pgs: 129 active+clean; 34 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 327 op/s 2024-03-21T11:28:38.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:38 smithi100 ceph-mon[36245]: pgmap v559: 129 pgs: 129 active+clean; 34 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 327 op/s 2024-03-21T11:28:39.513 INFO:tasks.workunit.client.0.smithi047.stdout: 1 324790 8.38 MB/sec execute 83 sec latency 11.226 ms 2024-03-21T11:28:40.513 INFO:tasks.workunit.client.0.smithi047.stdout: 1 326695 8.44 MB/sec execute 84 sec latency 10.775 ms 2024-03-21T11:28:40.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:40 smithi047 ceph-mon[31351]: pgmap v560: 129 pgs: 129 active+clean; 35 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 312 op/s 2024-03-21T11:28:40.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:40 smithi100 ceph-mon[36245]: pgmap v560: 129 pgs: 129 active+clean; 35 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 312 op/s 2024-03-21T11:28:40.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:40 smithi102 ceph-mon[36748]: pgmap v560: 129 pgs: 129 active+clean; 35 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 312 op/s 2024-03-21T11:28:41.513 INFO:tasks.workunit.client.0.smithi047.stdout: 1 327996 8.40 MB/sec execute 85 sec latency 14.891 ms 2024-03-21T11:28:42.513 INFO:tasks.workunit.client.0.smithi047.stdout: 1 329945 8.45 MB/sec execute 86 sec latency 62.285 ms 2024-03-21T11:28:42.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:42 smithi047 ceph-mon[31351]: pgmap v561: 129 pgs: 129 active+clean; 38 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 337 op/s 2024-03-21T11:28:42.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:42 smithi100 ceph-mon[36245]: pgmap v561: 129 pgs: 129 active+clean; 38 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 337 op/s 2024-03-21T11:28:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:42 smithi102 ceph-mon[36748]: pgmap v561: 129 pgs: 129 active+clean; 38 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 337 op/s 2024-03-21T11:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.362+0000 7f0f65800640 1 -- 172.21.15.47:0/265771067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f60108ef0 msgr2=0x7f0f60109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.362+0000 7f0f65800640 1 --2- 172.21.15.47:0/265771067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f60108ef0 0x7f0f60109370 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009900 tx=0x7f0f4802bc40 comp rx=0 tx=0).stop 2024-03-21T11:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.362+0000 7f0f65800640 1 -- 172.21.15.47:0/265771067 shutdown_connections 2024-03-21T11:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.362+0000 7f0f65800640 1 --2- 172.21.15.47:0/265771067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f601098b0 0x7f0f60110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.362+0000 7f0f65800640 1 --2- 172.21.15.47:0/265771067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f60108ef0 0x7f0f60109370 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.362+0000 7f0f65800640 1 --2- 172.21.15.47:0/265771067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f60107d00 0x7f0f60108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.362+0000 7f0f65800640 1 -- 172.21.15.47:0/265771067 >> 172.21.15.47:0/265771067 conn(0x7f0f60076cb0 msgr2=0x7f0f600790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.363+0000 7f0f65800640 1 -- 172.21.15.47:0/265771067 shutdown_connections 2024-03-21T11:28:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.363+0000 7f0f65800640 1 -- 172.21.15.47:0/265771067 wait complete. 2024-03-21T11:28:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.364+0000 7f0f65800640 1 Processor -- start 2024-03-21T11:28:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.364+0000 7f0f65800640 1 -- start start 2024-03-21T11:28:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.364+0000 7f0f65800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f60107d00 0x7f0f6019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.364+0000 7f0f65800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f60108ef0 0x7f0f6019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.364+0000 7f0f65800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f601098b0 0x7f0f601a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.364+0000 7f0f65800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f601139d0 con 0x7f0f60108ef0 2024-03-21T11:28:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.364+0000 7f0f65800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f60113850 con 0x7f0f601098b0 2024-03-21T11:28:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.364+0000 7f0f65800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f60113b50 con 0x7f0f60107d00 2024-03-21T11:28:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.364+0000 7f0f5ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f60107d00 0x7f0f6019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.364+0000 7f0f5f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f601098b0 0x7f0f601a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.364+0000 7f0f5ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f60107d00 0x7f0f6019df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37890/0 (socket says 172.21.15.47:37890) 2024-03-21T11:28:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.364+0000 7f0f5ea00640 1 -- 172.21.15.47:0/4053984359 learned_addr learned my addr 172.21.15.47:0/4053984359 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.364+0000 7f0f5f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f601098b0 0x7f0f601a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34326/0 (socket says 172.21.15.47:34326) 2024-03-21T11:28:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.365+0000 7f0f57e00640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f60108ef0 0x7f0f6019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.365+0000 7f0f5f400640 1 -- 172.21.15.47:0/4053984359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f60107d00 msgr2=0x7f0f6019df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.365+0000 7f0f5f400640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f60107d00 0x7f0f6019df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.365+0000 7f0f5f400640 1 -- 172.21.15.47:0/4053984359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f60108ef0 msgr2=0x7f0f6019e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.365+0000 7f0f5f400640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f60108ef0 0x7f0f6019e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.365+0000 7f0f5f400640 1 -- 172.21.15.47:0/4053984359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f601a5b40 con 0x7f0f601098b0 2024-03-21T11:28:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.365+0000 7f0f57e00640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f60108ef0 0x7f0f6019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.365+0000 7f0f5ea00640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f60107d00 0x7f0f6019df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.365+0000 7f0f5f400640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f601098b0 0x7f0f601a54d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0f58009fd0 tx=0x7f0f5800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.365+0000 7f0f57400640 1 -- 172.21.15.47:0/4053984359 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5800ce20 con 0x7f0f601098b0 2024-03-21T11:28:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.366+0000 7f0f57400640 1 -- 172.21.15.47:0/4053984359 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f58005020 con 0x7f0f601098b0 2024-03-21T11:28:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.366+0000 7f0f65800640 1 -- 172.21.15.47:0/4053984359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f601a5e30 con 0x7f0f601098b0 2024-03-21T11:28:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.366+0000 7f0f57400640 1 -- 172.21.15.47:0/4053984359 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f58010720 con 0x7f0f601098b0 2024-03-21T11:28:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.366+0000 7f0f65800640 1 -- 172.21.15.47:0/4053984359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f601a62f0 con 0x7f0f601098b0 2024-03-21T11:28:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.369+0000 7f0f685ef740 1 Processor -- start 2024-03-21T11:28:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.369+0000 7f0f57400640 1 -- 172.21.15.47:0/4053984359 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0f58005540 con 0x7f0f601098b0 2024-03-21T11:28:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.369+0000 7f0f685ef740 1 -- start start 2024-03-21T11:28:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.370+0000 7f0f685ef740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e487600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.370+0000 7f0f685ef740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e48c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.370+0000 7f0f685ef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 0x55dc2e48f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.370+0000 7f0f685ef740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2e44a3b0 con 0x55dc2e487bd0 2024-03-21T11:28:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.370+0000 7f0f685ef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2e3ec830 con 0x55dc2e48cd40 2024-03-21T11:28:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.370+0000 7f0f685ef740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2e3955d0 con 0x55dc2e4871e0 2024-03-21T11:28:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.370+0000 7f0f5ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e48c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.370+0000 7f0f5ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e48c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51826/0 (socket says 172.21.15.47:51826) 2024-03-21T11:28:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.370+0000 7f0f5ea00640 1 -- 172.21.15.47:0/4109764844 learned_addr learned my addr 172.21.15.47:0/4109764844 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.370+0000 7f0f57e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e487600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.370+0000 7f0f57e00640 1 --2- 172.21.15.47:0/4109764844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 0x55dc2e48f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.371+0000 7f0f5ea00640 1 -- 172.21.15.47:0/4109764844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 msgr2=0x55dc2e487600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.371+0000 7f0f5ea00640 1 --2- 172.21.15.47:0/4109764844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e487600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.371+0000 7f0f5ea00640 1 -- 172.21.15.47:0/4109764844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 msgr2=0x55dc2e48f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.371+0000 7f0f5ea00640 1 --2- 172.21.15.47:0/4109764844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 0x55dc2e48f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.371+0000 7f0f5ea00640 1 -- 172.21.15.47:0/4109764844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc2e3ad2c0 con 0x55dc2e487bd0 2024-03-21T11:28:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.371+0000 7f0f57e00640 1 --2- 172.21.15.47:0/4109764844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 0x55dc2e48f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.371+0000 7f0f57400640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f440a2fc0 0x7f0f440a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.371+0000 7f0f5ea00640 1 --2- 172.21.15.47:0/4109764844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e48c6d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0f50010580 tx=0x7f0f5000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99199b59dbe85e9a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.371+0000 7f0f57400640 1 -- 172.21.15.47:0/4053984359 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0f580cfec0 con 0x7f0f601098b0 2024-03-21T11:28:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.372+0000 7f0f57e00640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f440a2fc0 0x7f0f440a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.372+0000 7f0f4fe00640 1 -- 172.21.15.47:0/4109764844 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50011760 con 0x55dc2e487bd0 2024-03-21T11:28:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.372+0000 7f0f4fe00640 1 -- 172.21.15.47:0/4109764844 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f50011de0 con 0x55dc2e487bd0 2024-03-21T11:28:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.372+0000 7f0f4fe00640 1 -- 172.21.15.47:0/4109764844 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5000e430 con 0x55dc2e487bd0 2024-03-21T11:28:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.373+0000 7f0f685ef740 1 -- 172.21.15.47:0/4109764844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 msgr2=0x55dc2e48c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.373+0000 7f0f685ef740 1 --2- 172.21.15.47:0/4109764844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e48c6d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0f50010580 tx=0x7f0f5000bf90 comp rx=0 tx=0).stop 2024-03-21T11:28:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.373+0000 7f0f57e00640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f440a2fc0 0x7f0f440a5480 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0f4802c250 tx=0x7f0f48000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.373+0000 7f0f685ef740 1 -- 172.21.15.47:0/4109764844 shutdown_connections 2024-03-21T11:28:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.373+0000 7f0f685ef740 1 --2- 172.21.15.47:0/4109764844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 0x55dc2e48f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.373+0000 7f0f685ef740 1 --2- 172.21.15.47:0/4109764844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e48c6d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.373+0000 7f0f685ef740 1 --2- 172.21.15.47:0/4109764844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e487600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.373+0000 7f0f685ef740 1 -- 172.21.15.47:0/4109764844 >> 172.21.15.47:0/4109764844 conn(0x55dc2e482750 msgr2=0x55dc2e482b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.373+0000 7f0f685ef740 1 -- 172.21.15.47:0/4109764844 shutdown_connections 2024-03-21T11:28:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.373+0000 7f0f685ef740 1 -- 172.21.15.47:0/4109764844 wait complete. 2024-03-21T11:28:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.374+0000 7f0f685ef740 1 -- start start 2024-03-21T11:28:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.374+0000 7f0f685ef740 1 Processor -- start 2024-03-21T11:28:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.374+0000 7f0f685ef740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e4913f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.374+0000 7f0f685ef740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e491930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.374+0000 7f0f685ef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 0x55dc2e498900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.374+0000 7f0f685ef740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2e3a8a20 con 0x55dc2e487bd0 2024-03-21T11:28:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.374+0000 7f0f685ef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2e3e88d0 con 0x55dc2e48cd40 2024-03-21T11:28:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.374+0000 7f0f685ef740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2e48fcf0 con 0x55dc2e4871e0 2024-03-21T11:28:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.374+0000 7f0f685ef740 10 client.0 _ll_register_callbacks cb 0x55dc2e3e51c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.375+0000 7f0f5ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e4913f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.375+0000 7f0f5ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e4913f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37910/0 (socket says 172.21.15.47:37910) 2024-03-21T11:28:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.375+0000 7f0f5ea00640 1 -- 172.21.15.47:0/2306438639 learned_addr learned my addr 172.21.15.47:0/2306438639 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.375+0000 7f0f57e00640 1 --2- 172.21.15.47:0/2306438639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e491930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.375+0000 7f0f5f400640 1 --2- 172.21.15.47:0/2306438639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 0x55dc2e498900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.375+0000 7f0f5ea00640 1 -- 172.21.15.47:0/2306438639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 msgr2=0x55dc2e498900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.375+0000 7f0f5ea00640 1 --2- 172.21.15.47:0/2306438639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 0x55dc2e498900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.375+0000 7f0f5ea00640 1 -- 172.21.15.47:0/2306438639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 msgr2=0x55dc2e491930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.375+0000 7f0f5ea00640 1 --2- 172.21.15.47:0/2306438639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e491930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.375+0000 7f0f5ea00640 1 -- 172.21.15.47:0/2306438639 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc2e101210 con 0x55dc2e4871e0 2024-03-21T11:28:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.375+0000 7f0f5ea00640 1 --2- 172.21.15.47:0/2306438639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e4913f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0f50010550 tx=0x7f0f5001c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=997118673a306d5b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.375+0000 7f0f4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.376+0000 7f0f4cc00640 1 -- 172.21.15.47:0/2306438639 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5002a060 con 0x55dc2e4871e0 2024-03-21T11:28:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.376+0000 7f0f4cc00640 1 -- 172.21.15.47:0/2306438639 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0f50005260 con 0x55dc2e4871e0 2024-03-21T11:28:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.376+0000 7f0f685ef740 1 -- 172.21.15.47:0/2306438639 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f50011de0 con 0x55dc2e4871e0 2024-03-21T11:28:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.376+0000 7f0f4cc00640 1 -- 172.21.15.47:0/2306438639 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f50031050 con 0x55dc2e4871e0 2024-03-21T11:28:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.376+0000 7f0f685ef740 10 client.27298 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.376+0000 7f0f685ef740 1 -- 172.21.15.47:0/2306438639 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dc2e1cd620 con 0x55dc2e4871e0 2024-03-21T11:28:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.377+0000 7f0f4cc00640 1 -- 172.21.15.47:0/2306438639 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f50011de0 con 0x55dc2e4871e0 2024-03-21T11:28:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.377+0000 7f0f4cc00640 1 -- 172.21.15.47:0/2306438639 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0f50004920 con 0x55dc2e4871e0 2024-03-21T11:28:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.377+0000 7f0f685ef740 10 client.27298 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.377+0000 7f0f685ef740 10 client.27298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.377+0000 7f0f685ef740 20 client.27298 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.377+0000 7f0f685ef740 1 --2- 172.21.15.47:0/2306438639 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc2e49cea0 0x55dc2e49f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.377+0000 7f0f685ef740 4 client.27298 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.377+0000 7f0f685ef740 1 -- 172.21.15.47:0/2306438639 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc2e3f9d70 con 0x55dc2e49cea0 2024-03-21T11:28:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.377+0000 7f0f57e00640 1 --2- 172.21.15.47:0/2306438639 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc2e49cea0 0x55dc2e49f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.378+0000 7f0f57e00640 1 --2- 172.21.15.47:0/2306438639 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc2e49cea0 0x55dc2e49f290 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b35d2e5d87ac04b3 server_cookie=1f83941cf0d4095c in_seq=0 out_seq=0 2024-03-21T11:28:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.378+0000 7f0f4cc00640 10 client.27298 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.379+0000 7f0f4cc00640 1 -- 172.21.15.47:0/2306438639 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dc2e3f9d70 con 0x55dc2e49cea0 2024-03-21T11:28:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.379+0000 7f0f4cc00640 10 client.27298 handle_command_reply: tid=0 2024-03-21T11:28:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.379+0000 7f0f685ef740 1 client.27298 shutdown 2024-03-21T11:28:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.379+0000 7f0f685ef740 1 -- 172.21.15.47:0/2306438639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 msgr2=0x55dc2e4913f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.379+0000 7f0f685ef740 1 --2- 172.21.15.47:0/2306438639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e4913f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0f50010550 tx=0x7f0f5001c6b0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.379+0000 7f0f685ef740 1 -- 172.21.15.47:0/2306438639 shutdown_connections 2024-03-21T11:28:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.379+0000 7f0f685ef740 1 --2- 172.21.15.47:0/2306438639 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc2e49cea0 0x55dc2e49f290 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.379+0000 7f0f685ef740 1 --2- 172.21.15.47:0/2306438639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 0x55dc2e498900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.379+0000 7f0f685ef740 1 --2- 172.21.15.47:0/2306438639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e491930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.379+0000 7f0f685ef740 1 --2- 172.21.15.47:0/2306438639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e4913f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.379+0000 7f0f685ef740 1 -- 172.21.15.47:0/2306438639 >> 172.21.15.47:0/2306438639 conn(0x55dc2e482750 msgr2=0x55dc2e481950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.379+0000 7f0f4cc00640 0 client.27298 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.379+0000 7f0f685ef740 1 -- 172.21.15.47:0/2306438639 shutdown_connections 2024-03-21T11:28:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.379+0000 7f0f685ef740 1 -- 172.21.15.47:0/2306438639 wait complete. 2024-03-21T11:28:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.380+0000 7f0f685ef740 20 client.27298 trim_cache size 0 max 16384 2024-03-21T11:28:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.396+0000 7f0f685ef740 1 Processor -- start 2024-03-21T11:28:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.396+0000 7f0f685ef740 1 -- start start 2024-03-21T11:28:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f685ef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e4871e0 0x55dc2e47d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f685ef740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e4988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f685ef740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e48cd40 0x55dc2e4860c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f685ef740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f601b0690 con 0x55dc2e487bd0 2024-03-21T11:28:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f685ef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f601b2010 con 0x55dc2e4871e0 2024-03-21T11:28:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f685ef740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f601b0cf0 con 0x55dc2e48cd40 2024-03-21T11:28:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f5ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e4871e0 0x55dc2e47d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f57e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e4988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f5ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e4871e0 0x55dc2e47d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34358/0 (socket says 172.21.15.47:34358) 2024-03-21T11:28:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f5ea00640 1 -- 172.21.15.47:0/786396534 learned_addr learned my addr 172.21.15.47:0/786396534 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f5f400640 1 --2- 172.21.15.47:0/786396534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e48cd40 0x55dc2e4860c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f5ea00640 1 -- 172.21.15.47:0/786396534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e48cd40 msgr2=0x55dc2e4860c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f5ea00640 1 --2- 172.21.15.47:0/786396534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e48cd40 0x55dc2e4860c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f5ea00640 1 -- 172.21.15.47:0/786396534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 msgr2=0x55dc2e4988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.397+0000 7f0f5ea00640 1 --2- 172.21.15.47:0/786396534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e4988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.398+0000 7f0f5ea00640 1 -- 172.21.15.47:0/786396534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc2e4831c0 con 0x55dc2e4871e0 2024-03-21T11:28:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.398+0000 7f0f57e00640 1 --2- 172.21.15.47:0/786396534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e4988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.398+0000 7f0f5ea00640 1 --2- 172.21.15.47:0/786396534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e4871e0 0x55dc2e47d5d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0f50000c00 tx=0x7f0f50010450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d741267a8f893c32 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.398+0000 7f0f4f400640 1 -- 172.21.15.47:0/786396534 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5002a030 con 0x55dc2e4871e0 2024-03-21T11:28:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.398+0000 7f0f4f400640 1 -- 172.21.15.47:0/786396534 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f5000bbc0 con 0x55dc2e4871e0 2024-03-21T11:28:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.398+0000 7f0f4f400640 1 -- 172.21.15.47:0/786396534 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f500115c0 con 0x55dc2e4871e0 2024-03-21T11:28:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.399+0000 7f0f685ef740 1 -- 172.21.15.47:0/786396534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e4871e0 msgr2=0x55dc2e47d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.399+0000 7f0f685ef740 1 --2- 172.21.15.47:0/786396534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e4871e0 0x55dc2e47d5d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0f50000c00 tx=0x7f0f50010450 comp rx=0 tx=0).stop 2024-03-21T11:28:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.399+0000 7f0f685ef740 1 -- 172.21.15.47:0/786396534 shutdown_connections 2024-03-21T11:28:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.399+0000 7f0f685ef740 1 --2- 172.21.15.47:0/786396534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e48cd40 0x55dc2e4860c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.399+0000 7f0f685ef740 1 --2- 172.21.15.47:0/786396534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e4988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.399+0000 7f0f685ef740 1 --2- 172.21.15.47:0/786396534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e4871e0 0x55dc2e47d5d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.399+0000 7f0f685ef740 1 -- 172.21.15.47:0/786396534 >> 172.21.15.47:0/786396534 conn(0x55dc2e482750 msgr2=0x55dc2e4859b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.399+0000 7f0f685ef740 1 -- 172.21.15.47:0/786396534 shutdown_connections 2024-03-21T11:28:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.400+0000 7f0f685ef740 1 -- 172.21.15.47:0/786396534 wait complete. 2024-03-21T11:28:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.400+0000 7f0f685ef740 1 -- start start 2024-03-21T11:28:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.400+0000 7f0f685ef740 1 Processor -- start 2024-03-21T11:28:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f685ef740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e4955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f685ef740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e495b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f685ef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 0x55dc2e4f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f685ef740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f601b19b0 con 0x55dc2e487bd0 2024-03-21T11:28:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f685ef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f601b1350 con 0x55dc2e48cd40 2024-03-21T11:28:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f685ef740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f601ae850 con 0x55dc2e4871e0 2024-03-21T11:28:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f685ef740 10 client.0 _ll_register_callbacks cb 0x7f0f580093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f5ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e4955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f57e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e495b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f5ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e4955e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37934/0 (socket says 172.21.15.47:37934) 2024-03-21T11:28:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f5ea00640 1 -- 172.21.15.47:0/1386586187 learned_addr learned my addr 172.21.15.47:0/1386586187 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f5f400640 1 --2- 172.21.15.47:0/1386586187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 0x55dc2e4f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f5ea00640 1 -- 172.21.15.47:0/1386586187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 msgr2=0x55dc2e4f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f5ea00640 1 --2- 172.21.15.47:0/1386586187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 0x55dc2e4f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f5ea00640 1 -- 172.21.15.47:0/1386586187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 msgr2=0x55dc2e495b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.401+0000 7f0f5ea00640 1 --2- 172.21.15.47:0/1386586187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e495b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.402+0000 7f0f5ea00640 1 -- 172.21.15.47:0/1386586187 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc2e498b00 con 0x55dc2e4871e0 2024-03-21T11:28:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.402+0000 7f0f5f400640 1 --2- 172.21.15.47:0/1386586187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 0x55dc2e4f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.402+0000 7f0f5ea00640 1 --2- 172.21.15.47:0/1386586187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e4955e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0f50004e60 tx=0x7f0f5001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dcd6dfddcac7e4c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.402+0000 7f0f4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.402+0000 7f0f4cc00640 1 -- 172.21.15.47:0/1386586187 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5002a040 con 0x55dc2e4871e0 2024-03-21T11:28:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.402+0000 7f0f4cc00640 1 -- 172.21.15.47:0/1386586187 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0f50004020 con 0x55dc2e4871e0 2024-03-21T11:28:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.402+0000 7f0f685ef740 1 -- 172.21.15.47:0/1386586187 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f5000bbc0 con 0x55dc2e4871e0 2024-03-21T11:28:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.403+0000 7f0f4cc00640 1 -- 172.21.15.47:0/1386586187 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f5000e430 con 0x55dc2e4871e0 2024-03-21T11:28:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.403+0000 7f0f685ef740 10 client.27304 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.403+0000 7f0f685ef740 1 -- 172.21.15.47:0/1386586187 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dc2e4f7b20 con 0x55dc2e4871e0 2024-03-21T11:28:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.403+0000 7f0f4cc00640 1 -- 172.21.15.47:0/1386586187 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f5000bbc0 con 0x55dc2e4871e0 2024-03-21T11:28:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.403+0000 7f0f4cc00640 1 -- 172.21.15.47:0/1386586187 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0f50011890 con 0x55dc2e4871e0 2024-03-21T11:28:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.403+0000 7f0f685ef740 10 client.27304 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.403+0000 7f0f685ef740 10 client.27304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.403+0000 7f0f685ef740 20 client.27304 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.403+0000 7f0f685ef740 1 --2- 172.21.15.47:0/1386586187 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc2e4f8300 0x55dc2e4fa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.403+0000 7f0f685ef740 4 client.27304 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:28:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.403+0000 7f0f685ef740 1 -- 172.21.15.47:0/1386586187 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dc2e3f40c0 con 0x55dc2e4f8300 2024-03-21T11:28:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.404+0000 7f0f57e00640 1 --2- 172.21.15.47:0/1386586187 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc2e4f8300 0x55dc2e4fa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.404+0000 7f0f57e00640 1 --2- 172.21.15.47:0/1386586187 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc2e4f8300 0x55dc2e4fa720 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d87c379c247b846 server_cookie=e6f2c6bc7de3a705 in_seq=0 out_seq=0 2024-03-21T11:28:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.404+0000 7f0f4cc00640 10 client.27304 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:43.513 INFO:tasks.workunit.client.0.smithi047.stdout: 1 330829 8.38 MB/sec execute 87 sec latency 59.151 ms 2024-03-21T11:28:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.612+0000 7f0f4cc00640 1 -- 172.21.15.47:0/1386586187 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dc2e3f40c0 con 0x55dc2e4f8300 2024-03-21T11:28:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.612+0000 7f0f4cc00640 10 client.27304 handle_command_reply: tid=0 2024-03-21T11:28:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.613+0000 7f0f685ef740 1 client.27304 shutdown 2024-03-21T11:28:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.613+0000 7f0f685ef740 1 -- 172.21.15.47:0/1386586187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 msgr2=0x55dc2e4955e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.613+0000 7f0f685ef740 1 --2- 172.21.15.47:0/1386586187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e4955e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0f50004e60 tx=0x7f0f5001c950 comp rx=0 tx=0).stop 2024-03-21T11:28:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.613+0000 7f0f685ef740 1 -- 172.21.15.47:0/1386586187 shutdown_connections 2024-03-21T11:28:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.613+0000 7f0f685ef740 1 --2- 172.21.15.47:0/1386586187 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc2e4f8300 0x55dc2e4fa720 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.613+0000 7f0f685ef740 1 --2- 172.21.15.47:0/1386586187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc2e48cd40 0x55dc2e4f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.613+0000 7f0f685ef740 1 --2- 172.21.15.47:0/1386586187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc2e487bd0 0x55dc2e495b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.613+0000 7f0f685ef740 1 --2- 172.21.15.47:0/1386586187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc2e4871e0 0x55dc2e4955e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.613+0000 7f0f685ef740 1 -- 172.21.15.47:0/1386586187 >> 172.21.15.47:0/1386586187 conn(0x55dc2e482750 msgr2=0x55dc2e486240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.613+0000 7f0f4cc00640 0 client.27304 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.613+0000 7f0f685ef740 1 -- 172.21.15.47:0/1386586187 shutdown_connections 2024-03-21T11:28:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.613+0000 7f0f685ef740 1 -- 172.21.15.47:0/1386586187 wait complete. 2024-03-21T11:28:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.613+0000 7f0f685ef740 20 client.27304 trim_cache size 0 max 16384 2024-03-21T11:28:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:28:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:28:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:28:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:28:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.614+0000 7f0f65800640 1 -- 172.21.15.47:0/4053984359 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f440a2fc0 msgr2=0x7f0f440a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.614+0000 7f0f65800640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f440a2fc0 0x7f0f440a5480 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0f4802c250 tx=0x7f0f48000bc0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.614+0000 7f0f65800640 1 -- 172.21.15.47:0/4053984359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f601098b0 msgr2=0x7f0f601a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.614+0000 7f0f65800640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f601098b0 0x7f0f601a54d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0f58009fd0 tx=0x7f0f5800ef10 comp rx=0 tx=0).stop 2024-03-21T11:28:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.614+0000 7f0f65800640 1 -- 172.21.15.47:0/4053984359 shutdown_connections 2024-03-21T11:28:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.614+0000 7f0f65800640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f440a2fc0 0x7f0f440a5480 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.614+0000 7f0f65800640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f601098b0 0x7f0f601a54d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.614+0000 7f0f65800640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f60108ef0 0x7f0f6019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.614+0000 7f0f65800640 1 --2- 172.21.15.47:0/4053984359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f60107d00 0x7f0f6019df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.614+0000 7f0f65800640 1 -- 172.21.15.47:0/4053984359 >> 172.21.15.47:0/4053984359 conn(0x7f0f60076cb0 msgr2=0x7f0f6010c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.614+0000 7f0f65800640 1 -- 172.21.15.47:0/4053984359 shutdown_connections 2024-03-21T11:28:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:43.615+0000 7f0f65800640 1 -- 172.21.15.47:0/4053984359 wait complete. 2024-03-21T11:28:44.513 INFO:tasks.workunit.client.0.smithi047.stdout: 1 332652 8.40 MB/sec execute 88 sec latency 105.889 ms 2024-03-21T11:28:44.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:44 smithi047 ceph-mon[31351]: pgmap v562: 129 pgs: 129 active+clean; 37 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 325 op/s 2024-03-21T11:28:44.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:44 smithi100 ceph-mon[36245]: pgmap v562: 129 pgs: 129 active+clean; 37 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 325 op/s 2024-03-21T11:28:44.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:44 smithi102 ceph-mon[36748]: pgmap v562: 129 pgs: 129 active+clean; 37 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 325 op/s 2024-03-21T11:28:45.513 INFO:tasks.workunit.client.0.smithi047.stdout: 1 333855 8.40 MB/sec execute 89 sec latency 17.217 ms 2024-03-21T11:28:46.513 INFO:tasks.workunit.client.0.smithi047.stdout: 1 335096 8.36 MB/sec execute 90 sec latency 17.772 ms 2024-03-21T11:28:46.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:46 smithi047 ceph-mon[31351]: pgmap v563: 129 pgs: 129 active+clean; 37 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-03-21T11:28:46.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:46 smithi100 ceph-mon[36245]: pgmap v563: 129 pgs: 129 active+clean; 37 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-03-21T11:28:46.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:46 smithi102 ceph-mon[36748]: pgmap v563: 129 pgs: 129 active+clean; 37 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-03-21T11:28:47.514 INFO:tasks.workunit.client.0.smithi047.stdout: 1 336799 8.37 MB/sec execute 91 sec latency 58.515 ms 2024-03-21T11:28:47.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:47 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:28:47.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:47 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:28:47.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:47 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:28:48.514 INFO:tasks.workunit.client.0.smithi047.stdout: 1 337782 8.34 MB/sec execute 92 sec latency 31.086 ms 2024-03-21T11:28:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:48 smithi100 ceph-mon[36245]: pgmap v564: 129 pgs: 129 active+clean; 39 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 288 op/s 2024-03-21T11:28:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:48 smithi102 ceph-mon[36748]: pgmap v564: 129 pgs: 129 active+clean; 39 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 288 op/s 2024-03-21T11:28:48.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:48 smithi047 ceph-mon[31351]: pgmap v564: 129 pgs: 129 active+clean; 39 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 288 op/s 2024-03-21T11:28:49.514 INFO:tasks.workunit.client.0.smithi047.stdout: 1 339691 8.36 MB/sec execute 93 sec latency 17.213 ms 2024-03-21T11:28:49.989 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:28:49.989 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:28:49.989 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:28:49.989 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:28:49.989 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:28:49.990 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:28:49.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:28:49.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:28:49.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:28:50.515 INFO:tasks.workunit.client.0.smithi047.stdout: 1 340961 8.36 MB/sec execute 94 sec latency 14.291 ms 2024-03-21T11:28:50.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:50 smithi100 ceph-mon[36245]: pgmap v565: 129 pgs: 129 active+clean; 39 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 328 op/s 2024-03-21T11:28:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:50 smithi102 ceph-mon[36748]: pgmap v565: 129 pgs: 129 active+clean; 39 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 328 op/s 2024-03-21T11:28:51.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:50 smithi047 ceph-mon[31351]: pgmap v565: 129 pgs: 129 active+clean; 39 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 328 op/s 2024-03-21T11:28:51.514 INFO:tasks.workunit.client.0.smithi047.stdout: 1 342170 8.32 MB/sec execute 95 sec latency 14.369 ms 2024-03-21T11:28:52.514 INFO:tasks.workunit.client.0.smithi047.stdout: 1 343859 8.33 MB/sec execute 96 sec latency 103.153 ms 2024-03-21T11:28:52.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:52 smithi100 ceph-mon[36245]: pgmap v566: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 288 op/s 2024-03-21T11:28:52.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:52 smithi102 ceph-mon[36748]: pgmap v566: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 288 op/s 2024-03-21T11:28:53.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:52 smithi047 ceph-mon[31351]: pgmap v566: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 288 op/s 2024-03-21T11:28:53.514 INFO:tasks.workunit.client.0.smithi047.stdout: 1 344857 8.30 MB/sec execute 97 sec latency 20.325 ms 2024-03-21T11:28:54.514 INFO:tasks.workunit.client.0.smithi047.stdout: 1 347060 8.34 MB/sec execute 98 sec latency 11.347 ms 2024-03-21T11:28:54.866 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:54 smithi047 ceph-mon[31351]: pgmap v567: 129 pgs: 129 active+clean; 40 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 302 op/s 2024-03-21T11:28:54.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:54 smithi100 ceph-mon[36245]: pgmap v567: 129 pgs: 129 active+clean; 40 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 302 op/s 2024-03-21T11:28:54.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:28:54.977 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:28:54.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:54 smithi102 ceph-mon[36748]: pgmap v567: 129 pgs: 129 active+clean; 40 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 302 op/s 2024-03-21T11:28:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.209+0000 7fac33200640 1 -- 172.21.15.47:0/3823981456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac34104930 msgr2=0x7fac34106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.209+0000 7fac33200640 1 --2- 172.21.15.47:0/3823981456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac34104930 0x7fac34106d20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009900 tx=0x7fac1c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:28:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.210+0000 7fac33200640 1 -- 172.21.15.47:0/3823981456 shutdown_connections 2024-03-21T11:28:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.210+0000 7fac33200640 1 --2- 172.21.15.47:0/3823981456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac34107260 0x7fac341096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.210+0000 7fac33200640 1 --2- 172.21.15.47:0/3823981456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac34104930 0x7fac34106d20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.210+0000 7fac33200640 1 --2- 172.21.15.47:0/3823981456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac34102000 0x7fac341043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.210+0000 7fac33200640 1 -- 172.21.15.47:0/3823981456 >> 172.21.15.47:0/3823981456 conn(0x7fac340fba10 msgr2=0x7fac340fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.210+0000 7fac33200640 1 -- 172.21.15.47:0/3823981456 shutdown_connections 2024-03-21T11:28:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.210+0000 7fac33200640 1 -- 172.21.15.47:0/3823981456 wait complete. 2024-03-21T11:28:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.211+0000 7fac33200640 1 Processor -- start 2024-03-21T11:28:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.211+0000 7fac33200640 1 -- start start 2024-03-21T11:28:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.211+0000 7fac33200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac34102000 0x7fac341978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.211+0000 7fac33200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac34104930 0x7fac34197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.211+0000 7fac33200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac34107260 0x7fac3419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.211+0000 7fac33200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3410d220 con 0x7fac34102000 2024-03-21T11:28:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.211+0000 7fac33200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3410d0a0 con 0x7fac34107260 2024-03-21T11:28:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.211+0000 7fac33200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3410d3a0 con 0x7fac34104930 2024-03-21T11:28:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.212+0000 7fac31e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac34102000 0x7fac341978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.212+0000 7fac32800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac34107260 0x7fac3419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.212+0000 7fac32800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac34107260 0x7fac3419ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58200/0 (socket says 172.21.15.47:58200) 2024-03-21T11:28:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.212+0000 7fac32800640 1 -- 172.21.15.47:0/228258163 learned_addr learned my addr 172.21.15.47:0/228258163 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.212+0000 7fac2be00640 1 --2- 172.21.15.47:0/228258163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac34104930 0x7fac34197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.212+0000 7fac32800640 1 -- 172.21.15.47:0/228258163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac34104930 msgr2=0x7fac34197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.212+0000 7fac32800640 1 --2- 172.21.15.47:0/228258163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac34104930 0x7fac34197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.212+0000 7fac32800640 1 -- 172.21.15.47:0/228258163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac34102000 msgr2=0x7fac341978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.212+0000 7fac32800640 1 --2- 172.21.15.47:0/228258163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac34102000 0x7fac341978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.212+0000 7fac32800640 1 -- 172.21.15.47:0/228258163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3419f4f0 con 0x7fac34107260 2024-03-21T11:28:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.212+0000 7fac32800640 1 --2- 172.21.15.47:0/228258163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac34107260 0x7fac3419ee80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00eb10 tx=0x7fac2c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.213+0000 7fac2aa00640 1 -- 172.21.15.47:0/228258163 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c004b20 con 0x7fac34107260 2024-03-21T11:28:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.213+0000 7fac33200640 1 -- 172.21.15.47:0/228258163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac3419f7e0 con 0x7fac34107260 2024-03-21T11:28:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.213+0000 7fac2aa00640 1 -- 172.21.15.47:0/228258163 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac2c004cc0 con 0x7fac34107260 2024-03-21T11:28:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.213+0000 7fac2aa00640 1 -- 172.21.15.47:0/228258163 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c010760 con 0x7fac34107260 2024-03-21T11:28:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.213+0000 7fac33200640 1 -- 172.21.15.47:0/228258163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3419fcb0 con 0x7fac34107260 2024-03-21T11:28:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.216+0000 7fac3bd83740 1 Processor -- start 2024-03-21T11:28:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.216+0000 7fac2aa00640 1 -- 172.21.15.47:0/228258163 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fac2c005020 con 0x7fac34107260 2024-03-21T11:28:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.216+0000 7fac3bd83740 1 -- start start 2024-03-21T11:28:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.216+0000 7fac3bd83740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfe050 0x55fe76dfe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.216+0000 7fac3bd83740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76dfea40 0x55fe76e03540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.216+0000 7fac3bd83740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76e03bb0 0x55fe76e06040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.217+0000 7fac3bd83740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe76d63250 con 0x55fe76e03bb0 2024-03-21T11:28:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.217+0000 7fac3bd83740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe76d0ba30 con 0x55fe76dfea40 2024-03-21T11:28:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.217+0000 7fac3bd83740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe76d5f2f0 con 0x55fe76dfe050 2024-03-21T11:28:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.217+0000 7fac31e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76dfea40 0x55fe76e03540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.217+0000 7fac2be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfe050 0x55fe76dfe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.217+0000 7fac31e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76dfea40 0x55fe76e03540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58206/0 (socket says 172.21.15.47:58206) 2024-03-21T11:28:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.217+0000 7fac31e00640 1 -- 172.21.15.47:0/3838806411 learned_addr learned my addr 172.21.15.47:0/3838806411 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.217+0000 7fac2be00640 1 --2- 172.21.15.47:0/3838806411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76e03bb0 0x55fe76e06040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.217+0000 7fac31e00640 1 -- 172.21.15.47:0/3838806411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfe050 msgr2=0x55fe76dfe470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.217+0000 7fac31e00640 1 --2- 172.21.15.47:0/3838806411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfe050 0x55fe76dfe470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.217+0000 7fac31e00640 1 -- 172.21.15.47:0/3838806411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76e03bb0 msgr2=0x55fe76e06040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.217+0000 7fac31e00640 1 --2- 172.21.15.47:0/3838806411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76e03bb0 0x55fe76e06040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.217+0000 7fac31e00640 1 -- 172.21.15.47:0/3838806411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe76cf6690 con 0x55fe76dfea40 2024-03-21T11:28:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.217+0000 7fac2be00640 1 --2- 172.21.15.47:0/3838806411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfe050 0x55fe76dfe470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.218+0000 7fac31e00640 1 --2- 172.21.15.47:0/3838806411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76dfea40 0x55fe76e03540 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fac24009cf0 tx=0x7fac24011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f31b4d553f9894 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.218+0000 7fac2aa00640 1 --2- 172.21.15.47:0/228258163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac180a2f20 0x7fac180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.218+0000 7fac23400640 1 -- 172.21.15.47:0/3838806411 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac24004230 con 0x55fe76dfea40 2024-03-21T11:28:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.218+0000 7fac23400640 1 -- 172.21.15.47:0/3838806411 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac240043d0 con 0x55fe76dfea40 2024-03-21T11:28:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.218+0000 7fac23400640 1 -- 172.21.15.47:0/3838806411 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac24019940 con 0x55fe76dfea40 2024-03-21T11:28:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.218+0000 7fac2aa00640 1 -- 172.21.15.47:0/228258163 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fac2c0cfd30 con 0x7fac34107260 2024-03-21T11:28:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.218+0000 7fac2be00640 1 --2- 172.21.15.47:0/228258163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac180a2f20 0x7fac180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.219+0000 7fac3bd83740 1 -- 172.21.15.47:0/3838806411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76dfea40 msgr2=0x55fe76e03540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.219+0000 7fac3bd83740 1 --2- 172.21.15.47:0/3838806411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76dfea40 0x55fe76e03540 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fac24009cf0 tx=0x7fac24011810 comp rx=0 tx=0).stop 2024-03-21T11:28:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.219+0000 7fac2be00640 1 --2- 172.21.15.47:0/228258163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac180a2f20 0x7fac180a53e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fac1c02c180 tx=0x7fac1c02cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.219+0000 7fac3bd83740 1 -- 172.21.15.47:0/3838806411 shutdown_connections 2024-03-21T11:28:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.219+0000 7fac3bd83740 1 --2- 172.21.15.47:0/3838806411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76e03bb0 0x55fe76e06040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.219+0000 7fac3bd83740 1 --2- 172.21.15.47:0/3838806411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76dfea40 0x55fe76e03540 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.219+0000 7fac3bd83740 1 --2- 172.21.15.47:0/3838806411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfe050 0x55fe76dfe470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.219+0000 7fac3bd83740 1 -- 172.21.15.47:0/3838806411 >> 172.21.15.47:0/3838806411 conn(0x55fe76dc6200 msgr2=0x55fe76df99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.219+0000 7fac3bd83740 1 -- 172.21.15.47:0/3838806411 shutdown_connections 2024-03-21T11:28:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.219+0000 7fac3bd83740 1 -- 172.21.15.47:0/3838806411 wait complete. 2024-03-21T11:28:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.219+0000 7fac3bd83740 1 -- start start 2024-03-21T11:28:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.220+0000 7fac3bd83740 1 Processor -- start 2024-03-21T11:28:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.220+0000 7fac3bd83740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfe050 0x55fe76e0c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.220+0000 7fac3bd83740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76dfea40 0x55fe76e0c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.220+0000 7fac3bd83740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76e03bb0 0x55fe76e13990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.220+0000 7fac3bd83740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe76e06b60 con 0x55fe76dfea40 2024-03-21T11:28:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.220+0000 7fac3bd83740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe76d1ee80 con 0x55fe76e03bb0 2024-03-21T11:28:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.220+0000 7fac3bd83740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe76e06ce0 con 0x55fe76dfe050 2024-03-21T11:28:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.220+0000 7fac3bd83740 10 client.0 _ll_register_callbacks cb 0x55fe76d31130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.220+0000 7fac32800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76e03bb0 0x55fe76e13990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.220+0000 7fac31e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfe050 0x55fe76e0c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.220+0000 7fac32800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76e03bb0 0x55fe76e13990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58210/0 (socket says 172.21.15.47:58210) 2024-03-21T11:28:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.220+0000 7fac32800640 1 -- 172.21.15.47:0/3782144302 learned_addr learned my addr 172.21.15.47:0/3782144302 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.220+0000 7fac2be00640 1 --2- 172.21.15.47:0/3782144302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76dfea40 0x55fe76e0c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.221+0000 7fac32800640 1 -- 172.21.15.47:0/3782144302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfe050 msgr2=0x55fe76e0c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.221+0000 7fac32800640 1 --2- 172.21.15.47:0/3782144302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfe050 0x55fe76e0c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.221+0000 7fac32800640 1 -- 172.21.15.47:0/3782144302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76dfea40 msgr2=0x55fe76e0c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.221+0000 7fac32800640 1 --2- 172.21.15.47:0/3782144302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76dfea40 0x55fe76e0c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.221+0000 7fac32800640 1 -- 172.21.15.47:0/3782144302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe76b417d0 con 0x55fe76e03bb0 2024-03-21T11:28:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.221+0000 7fac31e00640 1 --2- 172.21.15.47:0/3782144302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfe050 0x55fe76e0c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.221+0000 7fac32800640 1 --2- 172.21.15.47:0/3782144302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76e03bb0 0x55fe76e13990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0a4550 tx=0x7fac2c09b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f59d3a2a7450eef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.221+0000 7fac17e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.221+0000 7fac17e00640 1 -- 172.21.15.47:0/3782144302 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0bc270 con 0x55fe76e03bb0 2024-03-21T11:28:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.221+0000 7fac17e00640 1 -- 172.21.15.47:0/3782144302 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fac2c09c2d0 con 0x55fe76e03bb0 2024-03-21T11:28:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.221+0000 7fac3bd83740 1 -- 172.21.15.47:0/3782144302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac240043d0 con 0x55fe76e03bb0 2024-03-21T11:28:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.222+0000 7fac17e00640 1 -- 172.21.15.47:0/3782144302 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac2c0a01c0 con 0x55fe76e03bb0 2024-03-21T11:28:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.222+0000 7fac3bd83740 10 client.27296 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.222+0000 7fac3bd83740 1 -- 172.21.15.47:0/3782144302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55fe76a773f0 con 0x55fe76e03bb0 2024-03-21T11:28:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.222+0000 7fac17e00640 1 -- 172.21.15.47:0/3782144302 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac240043d0 con 0x55fe76e03bb0 2024-03-21T11:28:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.222+0000 7fac17e00640 1 -- 172.21.15.47:0/3782144302 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fac2c005310 con 0x55fe76e03bb0 2024-03-21T11:28:55.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.222+0000 7fac3bd83740 10 client.27296 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.222+0000 7fac3bd83740 10 client.27296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.222+0000 7fac3bd83740 20 client.27296 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.222+0000 7fac3bd83740 1 --2- 172.21.15.47:0/3782144302 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe76e18160 0x55fe76e1a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.222+0000 7fac3bd83740 4 client.27296 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.222+0000 7fac3bd83740 1 -- 172.21.15.47:0/3782144302 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe76d70790 con 0x55fe76e18160 2024-03-21T11:28:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.223+0000 7fac31e00640 1 --2- 172.21.15.47:0/3782144302 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe76e18160 0x55fe76e1a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.223+0000 7fac31e00640 1 --2- 172.21.15.47:0/3782144302 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe76e18160 0x55fe76e1a5d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa217ae8ef96950f server_cookie=64844e77c2678464 in_seq=0 out_seq=0 2024-03-21T11:28:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.223+0000 7fac17e00640 10 client.27296 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:55.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.224+0000 7fac17e00640 1 -- 172.21.15.47:0/3782144302 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fe76d70790 con 0x55fe76e18160 2024-03-21T11:28:55.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.224+0000 7fac17e00640 10 client.27296 handle_command_reply: tid=0 2024-03-21T11:28:55.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.224+0000 7fac3bd83740 1 client.27296 shutdown 2024-03-21T11:28:55.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.224+0000 7fac3bd83740 1 -- 172.21.15.47:0/3782144302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76e03bb0 msgr2=0x55fe76e13990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:55.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.224+0000 7fac3bd83740 1 --2- 172.21.15.47:0/3782144302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76e03bb0 0x55fe76e13990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0a4550 tx=0x7fac2c09b9d0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.225+0000 7fac3bd83740 1 -- 172.21.15.47:0/3782144302 shutdown_connections 2024-03-21T11:28:55.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.225+0000 7fac3bd83740 1 --2- 172.21.15.47:0/3782144302 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe76e18160 0x55fe76e1a5d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.225+0000 7fac3bd83740 1 --2- 172.21.15.47:0/3782144302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76e03bb0 0x55fe76e13990 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.225+0000 7fac3bd83740 1 --2- 172.21.15.47:0/3782144302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76dfea40 0x55fe76e0c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.225+0000 7fac3bd83740 1 --2- 172.21.15.47:0/3782144302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfe050 0x55fe76e0c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.225+0000 7fac17e00640 0 client.27296 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:55.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.225+0000 7fac3bd83740 1 -- 172.21.15.47:0/3782144302 >> 172.21.15.47:0/3782144302 conn(0x55fe76dc6200 msgr2=0x55fe76df8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:55.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.225+0000 7fac3bd83740 1 -- 172.21.15.47:0/3782144302 shutdown_connections 2024-03-21T11:28:55.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.225+0000 7fac3bd83740 1 -- 172.21.15.47:0/3782144302 wait complete. 2024-03-21T11:28:55.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.225+0000 7fac3bd83740 20 client.27296 trim_cache size 0 max 16384 2024-03-21T11:28:55.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.243+0000 7fac3bd83740 1 Processor -- start 2024-03-21T11:28:55.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.243+0000 7fac3bd83740 1 -- start start 2024-03-21T11:28:55.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.243+0000 7fac3bd83740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76dfe050 0x55fe76df8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.243+0000 7fac3bd83740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfea40 0x55fe76df8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.243+0000 7fac3bd83740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76e03bb0 0x55fe76e138b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.243+0000 7fac3bd83740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac341a7cb0 con 0x55fe76dfe050 2024-03-21T11:28:55.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.243+0000 7fac3bd83740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac341a9630 con 0x55fe76e03bb0 2024-03-21T11:28:55.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.243+0000 7fac3bd83740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac341a8310 con 0x55fe76dfea40 2024-03-21T11:28:55.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.243+0000 7fac2be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfea40 0x55fe76df8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.243+0000 7fac31e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76dfe050 0x55fe76df8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.243+0000 7fac2be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfea40 0x55fe76df8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45936/0 (socket says 172.21.15.47:45936) 2024-03-21T11:28:55.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.243+0000 7fac2be00640 1 -- 172.21.15.47:0/3364546861 learned_addr learned my addr 172.21.15.47:0/3364546861 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:55.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.243+0000 7fac32800640 1 --2- 172.21.15.47:0/3364546861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76e03bb0 0x55fe76e138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.244+0000 7fac2be00640 1 -- 172.21.15.47:0/3364546861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76e03bb0 msgr2=0x55fe76e138b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.244+0000 7fac2be00640 1 --2- 172.21.15.47:0/3364546861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76e03bb0 0x55fe76e138b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.244+0000 7fac2be00640 1 -- 172.21.15.47:0/3364546861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76dfe050 msgr2=0x55fe76df8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.244+0000 7fac2be00640 1 --2- 172.21.15.47:0/3364546861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76dfe050 0x55fe76df8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.244+0000 7fac2be00640 1 -- 172.21.15.47:0/3364546861 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe76a370f0 con 0x55fe76dfea40 2024-03-21T11:28:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.244+0000 7fac32800640 1 --2- 172.21.15.47:0/3364546861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76e03bb0 0x55fe76e138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.244+0000 7fac2be00640 1 --2- 172.21.15.47:0/3364546861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfea40 0x55fe76df8f80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fac1c02c250 tx=0x7fac1c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86d4f88374bd765c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.244+0000 7fac22a00640 1 -- 172.21.15.47:0/3364546861 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c004380 con 0x55fe76dfea40 2024-03-21T11:28:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.244+0000 7fac22a00640 1 -- 172.21.15.47:0/3364546861 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac1c005530 con 0x55fe76dfea40 2024-03-21T11:28:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.244+0000 7fac22a00640 1 -- 172.21.15.47:0/3364546861 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0035f0 con 0x55fe76dfea40 2024-03-21T11:28:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.245+0000 7fac3bd83740 1 -- 172.21.15.47:0/3364546861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfea40 msgr2=0x55fe76df8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.245+0000 7fac3bd83740 1 --2- 172.21.15.47:0/3364546861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfea40 0x55fe76df8f80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fac1c02c250 tx=0x7fac1c009260 comp rx=0 tx=0).stop 2024-03-21T11:28:55.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.245+0000 7fac3bd83740 1 -- 172.21.15.47:0/3364546861 shutdown_connections 2024-03-21T11:28:55.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.245+0000 7fac3bd83740 1 --2- 172.21.15.47:0/3364546861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76e03bb0 0x55fe76e138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.245+0000 7fac3bd83740 1 --2- 172.21.15.47:0/3364546861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfea40 0x55fe76df8f80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.245+0000 7fac3bd83740 1 --2- 172.21.15.47:0/3364546861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76dfe050 0x55fe76df8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.245+0000 7fac3bd83740 1 -- 172.21.15.47:0/3364546861 >> 172.21.15.47:0/3364546861 conn(0x55fe76dc6200 msgr2=0x55fe76df8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:55.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.245+0000 7fac3bd83740 1 -- 172.21.15.47:0/3364546861 shutdown_connections 2024-03-21T11:28:55.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.245+0000 7fac3bd83740 1 -- 172.21.15.47:0/3364546861 wait complete. 2024-03-21T11:28:55.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.246+0000 7fac3bd83740 1 -- start start 2024-03-21T11:28:55.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.246+0000 7fac3bd83740 1 Processor -- start 2024-03-21T11:28:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.246+0000 7fac3bd83740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76dfe050 0x55fe76e114b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.246+0000 7fac3bd83740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfea40 0x55fe76e119f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.246+0000 7fac3bd83740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76e03bb0 0x55fe76e12a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.246+0000 7fac3bd83740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac341a8fd0 con 0x55fe76e03bb0 2024-03-21T11:28:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.246+0000 7fac3bd83740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac341a8970 con 0x55fe76dfe050 2024-03-21T11:28:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.246+0000 7fac3bd83740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac341a5e70 con 0x55fe76dfea40 2024-03-21T11:28:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.246+0000 7fac3bd83740 10 client.0 _ll_register_callbacks cb 0x7fac2c09be40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.247+0000 7fac2be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfea40 0x55fe76e119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.247+0000 7fac31e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76dfe050 0x55fe76e114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.247+0000 7fac2be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfea40 0x55fe76e119f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45950/0 (socket says 172.21.15.47:45950) 2024-03-21T11:28:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.247+0000 7fac2be00640 1 -- 172.21.15.47:0/1035273278 learned_addr learned my addr 172.21.15.47:0/1035273278 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.247+0000 7fac31e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76dfe050 0x55fe76e114b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58220/0 (socket says 172.21.15.47:58220) 2024-03-21T11:28:55.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.247+0000 7fac32800640 1 --2- 172.21.15.47:0/1035273278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76e03bb0 0x55fe76e12a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.247+0000 7fac2be00640 1 -- 172.21.15.47:0/1035273278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76dfe050 msgr2=0x55fe76e114b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.247+0000 7fac2be00640 1 --2- 172.21.15.47:0/1035273278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76dfe050 0x55fe76e114b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.247+0000 7fac2be00640 1 -- 172.21.15.47:0/1035273278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76e03bb0 msgr2=0x55fe76e12a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.247+0000 7fac2be00640 1 --2- 172.21.15.47:0/1035273278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76e03bb0 0x55fe76e12a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.247+0000 7fac2be00640 1 -- 172.21.15.47:0/1035273278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe76e12ff0 con 0x55fe76dfea40 2024-03-21T11:28:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.247+0000 7fac32800640 1 --2- 172.21.15.47:0/1035273278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76e03bb0 0x55fe76e12a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.247+0000 7fac2be00640 1 --2- 172.21.15.47:0/1035273278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfea40 0x55fe76e119f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fac1c02c8f0 tx=0x7fac1c005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6b4ee3fc63b1ba3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.247+0000 7fac17e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.248+0000 7fac17e00640 1 -- 172.21.15.47:0/1035273278 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c063060 con 0x55fe76dfea40 2024-03-21T11:28:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.248+0000 7fac17e00640 1 -- 172.21.15.47:0/1035273278 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fac1c004390 con 0x55fe76dfea40 2024-03-21T11:28:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.248+0000 7fac3bd83740 1 -- 172.21.15.47:0/1035273278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac1c005530 con 0x55fe76dfea40 2024-03-21T11:28:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.248+0000 7fac17e00640 1 -- 172.21.15.47:0/1035273278 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac1c06a050 con 0x55fe76dfea40 2024-03-21T11:28:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.248+0000 7fac3bd83740 10 client.27322 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:55.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.248+0000 7fac3bd83740 1 -- 172.21.15.47:0/1035273278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55fe76e73f80 con 0x55fe76dfea40 2024-03-21T11:28:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.248+0000 7fac17e00640 1 -- 172.21.15.47:0/1035273278 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac1c005530 con 0x55fe76dfea40 2024-03-21T11:28:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.248+0000 7fac17e00640 1 -- 172.21.15.47:0/1035273278 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fac1c0039e0 con 0x55fe76dfea40 2024-03-21T11:28:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.248+0000 7fac3bd83740 10 client.27322 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.248+0000 7fac3bd83740 10 client.27322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.248+0000 7fac3bd83740 20 client.27322 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.248+0000 7fac3bd83740 1 --2- 172.21.15.47:0/1035273278 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe76e74890 0x55fe76e76cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.248+0000 7fac3bd83740 4 client.27322 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:28:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.248+0000 7fac3bd83740 1 -- 172.21.15.47:0/1035273278 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fe76d6aae0 con 0x55fe76e74890 2024-03-21T11:28:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.249+0000 7fac31e00640 1 --2- 172.21.15.47:0/1035273278 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe76e74890 0x55fe76e76cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.249+0000 7fac31e00640 1 --2- 172.21.15.47:0/1035273278 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe76e74890 0x55fe76e76cb0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95a22c6abd9dfcf6 server_cookie=be09f52507559bd1 in_seq=0 out_seq=0 2024-03-21T11:28:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.249+0000 7fac17e00640 10 client.27322 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac17e00640 1 -- 172.21.15.47:0/1035273278 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fe76d6aae0 con 0x55fe76e74890 2024-03-21T11:28:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac17e00640 10 client.27322 handle_command_reply: tid=0 2024-03-21T11:28:55.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac3bd83740 1 client.27322 shutdown 2024-03-21T11:28:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac3bd83740 1 -- 172.21.15.47:0/1035273278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfea40 msgr2=0x55fe76e119f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac3bd83740 1 --2- 172.21.15.47:0/1035273278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfea40 0x55fe76e119f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fac1c02c8f0 tx=0x7fac1c005700 comp rx=0 tx=0).stop 2024-03-21T11:28:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac3bd83740 1 -- 172.21.15.47:0/1035273278 shutdown_connections 2024-03-21T11:28:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac3bd83740 1 --2- 172.21.15.47:0/1035273278 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe76e74890 0x55fe76e76cb0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac3bd83740 1 --2- 172.21.15.47:0/1035273278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe76e03bb0 0x55fe76e12a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac3bd83740 1 --2- 172.21.15.47:0/1035273278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe76dfea40 0x55fe76e119f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac3bd83740 1 --2- 172.21.15.47:0/1035273278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe76dfe050 0x55fe76e114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac3bd83740 1 -- 172.21.15.47:0/1035273278 >> 172.21.15.47:0/1035273278 conn(0x55fe76dc6200 msgr2=0x55fe76df8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac17e00640 0 client.27322 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac3bd83740 1 -- 172.21.15.47:0/1035273278 shutdown_connections 2024-03-21T11:28:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac3bd83740 1 -- 172.21.15.47:0/1035273278 wait complete. 2024-03-21T11:28:55.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.250+0000 7fac3bd83740 20 client.27322 trim_cache size 0 max 16384 2024-03-21T11:28:55.256 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:28:55.256 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:28:55.256 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:28:55.256 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:28:55.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.251+0000 7fac33200640 1 -- 172.21.15.47:0/228258163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac180a2f20 msgr2=0x7fac180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.251+0000 7fac33200640 1 --2- 172.21.15.47:0/228258163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac180a2f20 0x7fac180a53e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fac1c02c180 tx=0x7fac1c02cf90 comp rx=0 tx=0).stop 2024-03-21T11:28:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.251+0000 7fac33200640 1 -- 172.21.15.47:0/228258163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac34107260 msgr2=0x7fac3419ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.251+0000 7fac33200640 1 --2- 172.21.15.47:0/228258163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac34107260 0x7fac3419ee80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00eb10 tx=0x7fac2c00c2d0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.251+0000 7fac33200640 1 -- 172.21.15.47:0/228258163 shutdown_connections 2024-03-21T11:28:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.251+0000 7fac33200640 1 --2- 172.21.15.47:0/228258163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac180a2f20 0x7fac180a53e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.251+0000 7fac33200640 1 --2- 172.21.15.47:0/228258163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac34107260 0x7fac3419ee80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.251+0000 7fac33200640 1 --2- 172.21.15.47:0/228258163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac34104930 0x7fac34197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.251+0000 7fac33200640 1 --2- 172.21.15.47:0/228258163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac34102000 0x7fac341978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.251+0000 7fac33200640 1 -- 172.21.15.47:0/228258163 >> 172.21.15.47:0/228258163 conn(0x7fac340fba10 msgr2=0x7fac341028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.251+0000 7fac33200640 1 -- 172.21.15.47:0/228258163 shutdown_connections 2024-03-21T11:28:55.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.251+0000 7fac33200640 1 -- 172.21.15.47:0/228258163 wait complete. 2024-03-21T11:28:55.270 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:28:55.270 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:28:55.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.502+0000 7f0ea3000640 1 -- 172.21.15.47:0/379633047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e9c1038e0 msgr2=0x7f0e9c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:55.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.502+0000 7f0ea3000640 1 --2- 172.21.15.47:0/379633047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e9c1038e0 0x7f0e9c103d00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009860 tx=0x7f0e8c02bb70 comp rx=0 tx=0).stop 2024-03-21T11:28:55.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.502+0000 7f0ea3000640 1 -- 172.21.15.47:0/379633047 shutdown_connections 2024-03-21T11:28:55.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.502+0000 7f0ea3000640 1 --2- 172.21.15.47:0/379633047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c105490 0x7f0e9c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.502+0000 7f0ea3000640 1 --2- 172.21.15.47:0/379633047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e9c104ad0 0x7f0e9c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.502+0000 7f0ea3000640 1 --2- 172.21.15.47:0/379633047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e9c1038e0 0x7f0e9c103d00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.502+0000 7f0ea3000640 1 -- 172.21.15.47:0/379633047 >> 172.21.15.47:0/379633047 conn(0x7f0e9c0fee90 msgr2=0x7f0e9c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:55.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.502+0000 7f0ea3000640 1 -- 172.21.15.47:0/379633047 shutdown_connections 2024-03-21T11:28:55.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.502+0000 7f0ea3000640 1 -- 172.21.15.47:0/379633047 wait complete. 2024-03-21T11:28:55.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.503+0000 7f0ea3000640 1 Processor -- start 2024-03-21T11:28:55.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.503+0000 7f0ea3000640 1 -- start start 2024-03-21T11:28:55.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0ea3000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c1038e0 0x7f0e9c199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0ea3000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e9c104ad0 0x7f0e9c19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0ea3000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e9c105490 0x7f0e9c1a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0ea3000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c10f530 con 0x7f0e9c1038e0 2024-03-21T11:28:55.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0ea3000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c10f3b0 con 0x7f0e9c105490 2024-03-21T11:28:55.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0ea3000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c10f6b0 con 0x7f0e9c104ad0 2024-03-21T11:28:55.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0e9b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e9c104ad0 0x7f0e9c19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0e9be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c1038e0 0x7f0e9c199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0e9b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e9c104ad0 0x7f0e9c19a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45956/0 (socket says 172.21.15.47:45956) 2024-03-21T11:28:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0e9b400640 1 -- 172.21.15.47:0/2659151637 learned_addr learned my addr 172.21.15.47:0/2659151637 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0ea1000640 1 --2- 172.21.15.47:0/2659151637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e9c105490 0x7f0e9c1a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0e9b400640 1 -- 172.21.15.47:0/2659151637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e9c105490 msgr2=0x7f0e9c1a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0e9b400640 1 --2- 172.21.15.47:0/2659151637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e9c105490 0x7f0e9c1a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0e9b400640 1 -- 172.21.15.47:0/2659151637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c1038e0 msgr2=0x7f0e9c199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0e9b400640 1 --2- 172.21.15.47:0/2659151637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c1038e0 0x7f0e9c199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0e9b400640 1 -- 172.21.15.47:0/2659151637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c1a17f0 con 0x7f0e9c104ad0 2024-03-21T11:28:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.504+0000 7f0e9be00640 1 --2- 172.21.15.47:0/2659151637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c1038e0 0x7f0e9c199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.505+0000 7f0e9b400640 1 --2- 172.21.15.47:0/2659151637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e9c104ad0 0x7f0e9c19a110 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800e9c0 tx=0x7f0e8800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.505+0000 7f0e98c00640 1 -- 172.21.15.47:0/2659151637 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8800ccc0 con 0x7f0e9c104ad0 2024-03-21T11:28:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.505+0000 7f0e98c00640 1 -- 172.21.15.47:0/2659151637 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e88005500 con 0x7f0e9c104ad0 2024-03-21T11:28:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.505+0000 7f0ea3000640 1 -- 172.21.15.47:0/2659151637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9c1a1ae0 con 0x7f0e9c104ad0 2024-03-21T11:28:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.505+0000 7f0e98c00640 1 -- 172.21.15.47:0/2659151637 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e88010430 con 0x7f0e9c104ad0 2024-03-21T11:28:55.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.505+0000 7f0ea3000640 1 -- 172.21.15.47:0/2659151637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9c1a2030 con 0x7f0e9c104ad0 2024-03-21T11:28:55.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.506+0000 7f0ea3000640 1 -- 172.21.15.47:0/2659151637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9c10ac80 con 0x7f0e9c104ad0 2024-03-21T11:28:55.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.509+0000 7f0e98c00640 1 -- 172.21.15.47:0/2659151637 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0e880046c0 con 0x7f0e9c104ad0 2024-03-21T11:28:55.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.510+0000 7f0e98c00640 1 --2- 172.21.15.47:0/2659151637 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0e800a2f20 0x7f0e800a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:55.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.510+0000 7f0e98c00640 1 -- 172.21.15.47:0/2659151637 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0e880cfaa0 con 0x7f0e9c104ad0 2024-03-21T11:28:55.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.511+0000 7f0e9be00640 1 --2- 172.21.15.47:0/2659151637 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0e800a2f20 0x7f0e800a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:55.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.511+0000 7f0e9be00640 1 --2- 172.21.15.47:0/2659151637 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0e800a2f20 0x7f0e800a53e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009830 tx=0x7f0e8c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:55.514 INFO:tasks.workunit.client.0.smithi047.stdout: 1 348332 8.32 MB/sec execute 99 sec latency 15.927 ms 2024-03-21T11:28:55.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.515+0000 7f0e98c00640 1 -- 172.21.15.47:0/2659151637 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0e8809c4e0 con 0x7f0e9c104ad0 2024-03-21T11:28:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.776+0000 7f0ea3000640 1 -- 172.21.15.47:0/2659151637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e9c0622d0 con 0x7f0e9c104ad0 2024-03-21T11:28:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.777+0000 7f0e98c00640 1 -- 172.21.15.47:0/2659151637 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f0e8809bde0 con 0x7f0e9c104ad0 2024-03-21T11:28:55.778 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:28:55.778 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:28:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:28:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.780+0000 7f0ea3000640 1 -- 172.21.15.47:0/2659151637 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0e800a2f20 msgr2=0x7f0e800a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.780+0000 7f0ea3000640 1 --2- 172.21.15.47:0/2659151637 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0e800a2f20 0x7f0e800a53e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009830 tx=0x7f0e8c02e040 comp rx=0 tx=0).stop 2024-03-21T11:28:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.780+0000 7f0ea3000640 1 -- 172.21.15.47:0/2659151637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e9c104ad0 msgr2=0x7f0e9c19a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.780+0000 7f0ea3000640 1 --2- 172.21.15.47:0/2659151637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e9c104ad0 0x7f0e9c19a110 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800e9c0 tx=0x7f0e8800ee80 comp rx=0 tx=0).stop 2024-03-21T11:28:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.780+0000 7f0ea3000640 1 -- 172.21.15.47:0/2659151637 shutdown_connections 2024-03-21T11:28:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.780+0000 7f0ea3000640 1 --2- 172.21.15.47:0/2659151637 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0e800a2f20 0x7f0e800a53e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.780+0000 7f0ea3000640 1 --2- 172.21.15.47:0/2659151637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e9c105490 0x7f0e9c1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.780+0000 7f0ea3000640 1 --2- 172.21.15.47:0/2659151637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e9c104ad0 0x7f0e9c19a110 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.780+0000 7f0ea3000640 1 --2- 172.21.15.47:0/2659151637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e9c1038e0 0x7f0e9c199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.780+0000 7f0ea3000640 1 -- 172.21.15.47:0/2659151637 >> 172.21.15.47:0/2659151637 conn(0x7f0e9c0fee90 msgr2=0x7f0e9c107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.780+0000 7f0ea3000640 1 -- 172.21.15.47:0/2659151637 shutdown_connections 2024-03-21T11:28:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:55.780+0000 7f0ea3000640 1 -- 172.21.15.47:0/2659151637 wait complete. 2024-03-21T11:28:55.801 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:28:55.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:28:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.028+0000 7f0be7a00640 1 -- 172.21.15.47:0/2614867583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0be01038e0 msgr2=0x7f0be0103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.028+0000 7f0be7a00640 1 --2- 172.21.15.47:0/2614867583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0be01038e0 0x7f0be0103d00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009ca0 tx=0x7f0bd002c0f0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.029+0000 7f0be7a00640 1 -- 172.21.15.47:0/2614867583 shutdown_connections 2024-03-21T11:28:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.029+0000 7f0be7a00640 1 --2- 172.21.15.47:0/2614867583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be0105490 0x7f0be010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.029+0000 7f0be7a00640 1 --2- 172.21.15.47:0/2614867583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be0104ad0 0x7f0be0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.029+0000 7f0be7a00640 1 --2- 172.21.15.47:0/2614867583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0be01038e0 0x7f0be0103d00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.029+0000 7f0be7a00640 1 -- 172.21.15.47:0/2614867583 >> 172.21.15.47:0/2614867583 conn(0x7f0be00fee50 msgr2=0x7f0be0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.029+0000 7f0be7a00640 1 -- 172.21.15.47:0/2614867583 shutdown_connections 2024-03-21T11:28:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.029+0000 7f0be7a00640 1 -- 172.21.15.47:0/2614867583 wait complete. 2024-03-21T11:28:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.030+0000 7f0be7a00640 1 Processor -- start 2024-03-21T11:28:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.030+0000 7f0be7a00640 1 -- start start 2024-03-21T11:28:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be7a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be01038e0 0x7f0be0199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be7a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0be0104ad0 0x7f0be019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be7a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be0105490 0x7f0be01a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be7a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be010f530 con 0x7f0be0104ad0 2024-03-21T11:28:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be7a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be010f3b0 con 0x7f0be01038e0 2024-03-21T11:28:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be7a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be010f6b0 con 0x7f0be0105490 2024-03-21T11:28:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be01038e0 0x7f0be0199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be5a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be0105490 0x7f0be01a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be01038e0 0x7f0be0199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58266/0 (socket says 172.21.15.47:58266) 2024-03-21T11:28:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be5000640 1 -- 172.21.15.47:0/4282102953 learned_addr learned my addr 172.21.15.47:0/4282102953 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0bdfe00640 1 --2- 172.21.15.47:0/4282102953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0be0104ad0 0x7f0be019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be5000640 1 -- 172.21.15.47:0/4282102953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be0105490 msgr2=0x7f0be01a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be5000640 1 --2- 172.21.15.47:0/4282102953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be0105490 0x7f0be01a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be5000640 1 -- 172.21.15.47:0/4282102953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0be0104ad0 msgr2=0x7f0be019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be5000640 1 --2- 172.21.15.47:0/4282102953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0be0104ad0 0x7f0be019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be5000640 1 -- 172.21.15.47:0/4282102953 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be01a1880 con 0x7f0be01038e0 2024-03-21T11:28:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.031+0000 7f0be5a00640 1 --2- 172.21.15.47:0/4282102953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be0105490 0x7f0be01a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.032+0000 7f0be5000640 1 --2- 172.21.15.47:0/4282102953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be01038e0 0x7f0be0199c70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0000f80 tx=0x7f0bd0032ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.032+0000 7f0bdd600640 1 -- 172.21.15.47:0/4282102953 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0041020 con 0x7f0be01038e0 2024-03-21T11:28:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.032+0000 7f0be7a00640 1 -- 172.21.15.47:0/4282102953 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be01a1b10 con 0x7f0be01038e0 2024-03-21T11:28:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.032+0000 7f0bdd600640 1 -- 172.21.15.47:0/4282102953 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bd0034cd0 con 0x7f0be01038e0 2024-03-21T11:28:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.032+0000 7f0bdd600640 1 -- 172.21.15.47:0/4282102953 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0039040 con 0x7f0be01038e0 2024-03-21T11:28:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.032+0000 7f0be7a00640 1 -- 172.21.15.47:0/4282102953 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be01a2020 con 0x7f0be01038e0 2024-03-21T11:28:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.035+0000 7f0bdd600640 1 -- 172.21.15.47:0/4282102953 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0bd0044020 con 0x7f0be01038e0 2024-03-21T11:28:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.036+0000 7f0bea7f9740 1 Processor -- start 2024-03-21T11:28:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.036+0000 7f0bea7f9740 1 -- start start 2024-03-21T11:28:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.036+0000 7f0bea7f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df3810 0x55c167df3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.036+0000 7f0bea7f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167df4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.036+0000 7f0bea7f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df53a0 0x55c167dfbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.036+0000 7f0bea7f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c167d582c0 con 0x55c167df53a0 2024-03-21T11:28:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.036+0000 7f0bea7f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c167d00c90 con 0x55c167df3810 2024-03-21T11:28:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.036+0000 7f0bea7f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c167d54360 con 0x55c167df49e0 2024-03-21T11:28:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.036+0000 7f0be5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167df4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.036+0000 7f0be5a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df3810 0x55c167df3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.036+0000 7f0be5a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df3810 0x55c167df3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58280/0 (socket says 172.21.15.47:58280) 2024-03-21T11:28:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.036+0000 7f0be5a00640 1 -- 172.21.15.47:0/4013084893 learned_addr learned my addr 172.21.15.47:0/4013084893 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.037+0000 7f0bdfe00640 1 --2- 172.21.15.47:0/4013084893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df53a0 0x55c167dfbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.037+0000 7f0be5000640 1 -- 172.21.15.47:0/4013084893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df3810 msgr2=0x55c167df3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.037+0000 7f0be5000640 1 --2- 172.21.15.47:0/4013084893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df3810 0x55c167df3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.037+0000 7f0be5000640 1 -- 172.21.15.47:0/4013084893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df53a0 msgr2=0x55c167dfbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.037+0000 7f0be5000640 1 --2- 172.21.15.47:0/4013084893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df53a0 0x55c167dfbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.037+0000 7f0be5000640 1 -- 172.21.15.47:0/4013084893 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c167d25080 con 0x55c167df49e0 2024-03-21T11:28:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.037+0000 7f0bdd600640 1 --2- 172.21.15.47:0/4282102953 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0bc40a2f20 0x7f0bc40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.037+0000 7f0be5000640 1 --2- 172.21.15.47:0/4013084893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167df4e60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00eecd0 tx=0x7f0bd00f7600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=860240d763d71826 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.037+0000 7f0bdd600640 1 -- 172.21.15.47:0/4282102953 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0bd00f15f0 con 0x7f0be01038e0 2024-03-21T11:28:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.037+0000 7f0be5a00640 1 --2- 172.21.15.47:0/4282102953 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0bc40a2f20 0x7f0bc40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.037+0000 7f0bd6000640 1 -- 172.21.15.47:0/4013084893 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00b9bd0 con 0x55c167df49e0 2024-03-21T11:28:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.038+0000 7f0bd6000640 1 -- 172.21.15.47:0/4013084893 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bd00be030 con 0x55c167df49e0 2024-03-21T11:28:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.038+0000 7f0bd6000640 1 -- 172.21.15.47:0/4013084893 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00bd290 con 0x55c167df49e0 2024-03-21T11:28:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.038+0000 7f0bea7f9740 1 -- 172.21.15.47:0/4013084893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 msgr2=0x55c167df4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.038+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/4013084893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167df4e60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00eecd0 tx=0x7f0bd00f7600 comp rx=0 tx=0).stop 2024-03-21T11:28:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.038+0000 7f0be5a00640 1 --2- 172.21.15.47:0/4282102953 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0bc40a2f20 0x7f0bc40a53e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55c167df48e0 tx=0x7f0bd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.038+0000 7f0bea7f9740 1 -- 172.21.15.47:0/4013084893 shutdown_connections 2024-03-21T11:28:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.038+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/4013084893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df53a0 0x55c167dfbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.038+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/4013084893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167df4e60 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.038+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/4013084893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df3810 0x55c167df3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.038+0000 7f0bea7f9740 1 -- 172.21.15.47:0/4013084893 >> 172.21.15.47:0/4013084893 conn(0x55c167deec00 msgr2=0x55c167def000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.038+0000 7f0bea7f9740 1 -- 172.21.15.47:0/4013084893 shutdown_connections 2024-03-21T11:28:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.038+0000 7f0bea7f9740 1 -- 172.21.15.47:0/4013084893 wait complete. 2024-03-21T11:28:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.039+0000 7f0bea7f9740 1 -- start start 2024-03-21T11:28:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.039+0000 7f0bea7f9740 1 Processor -- start 2024-03-21T11:28:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bea7f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167dfd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bea7f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167dfdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bea7f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df53a0 0x55c167e04d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bea7f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c167dfc170 con 0x55c167df3810 2024-03-21T11:28:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bea7f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c167d140d0 con 0x55c167df53a0 2024-03-21T11:28:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bea7f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c167dfc2f0 con 0x55c167df49e0 2024-03-21T11:28:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bea7f9740 10 client.0 _ll_register_callbacks cb 0x55c167d18e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bdfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167dfdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0be5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167dfd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0be5a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df53a0 0x55c167e04d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bdfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167dfdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45984/0 (socket says 172.21.15.47:45984) 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bdfe00640 1 -- 172.21.15.47:0/2801326882 learned_addr learned my addr 172.21.15.47:0/2801326882 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bdfe00640 1 -- 172.21.15.47:0/2801326882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df53a0 msgr2=0x55c167e04d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bdfe00640 1 --2- 172.21.15.47:0/2801326882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df53a0 0x55c167e04d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bdfe00640 1 -- 172.21.15.47:0/2801326882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 msgr2=0x55c167dfd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bdfe00640 1 --2- 172.21.15.47:0/2801326882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167dfd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0bdfe00640 1 -- 172.21.15.47:0/2801326882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c167a6c3f0 con 0x55c167df49e0 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.040+0000 7f0be5000640 1 --2- 172.21.15.47:0/2801326882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167dfd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.041+0000 7f0bdfe00640 1 --2- 172.21.15.47:0/2801326882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167dfdce0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8012700 tx=0x7f0bc8012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50ca5135620b336f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.041+0000 7f0bcea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.041+0000 7f0bcea00640 1 -- 172.21.15.47:0/2801326882 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc800f8c0 con 0x55c167df49e0 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.041+0000 7f0bea7f9740 1 -- 172.21.15.47:0/2801326882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bd00be030 con 0x55c167df49e0 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.041+0000 7f0bcea00640 1 -- 172.21.15.47:0/2801326882 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0bc800fa60 con 0x55c167df49e0 2024-03-21T11:28:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.041+0000 7f0bcea00640 1 -- 172.21.15.47:0/2801326882 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bc800fc60 con 0x55c167df49e0 2024-03-21T11:28:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.041+0000 7f0bea7f9740 10 client.27340 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.041+0000 7f0bea7f9740 1 -- 172.21.15.47:0/2801326882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c167b367d0 con 0x55c167df49e0 2024-03-21T11:28:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.042+0000 7f0bcea00640 1 -- 172.21.15.47:0/2801326882 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bd00be030 con 0x55c167df49e0 2024-03-21T11:28:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.042+0000 7f0bcea00640 1 -- 172.21.15.47:0/2801326882 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0bc801c420 con 0x55c167df49e0 2024-03-21T11:28:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.042+0000 7f0bea7f9740 10 client.27340 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.042+0000 7f0bea7f9740 10 client.27340 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:28:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.042+0000 7f0bea7f9740 20 client.27340 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.042+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/2801326882 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c167e09250 0x55c167e0b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.042+0000 7f0bea7f9740 4 client.27340 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.042+0000 7f0bea7f9740 1 -- 172.21.15.47:0/2801326882 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c167d65800 con 0x55c167e09250 2024-03-21T11:28:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.042+0000 7f0be5000640 1 --2- 172.21.15.47:0/2801326882 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c167e09250 0x55c167e0b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.043+0000 7f0be5000640 1 --2- 172.21.15.47:0/2801326882 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c167e09250 0x55c167e0b6b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5736707611713ec server_cookie=1eb2274e899d2afa in_seq=0 out_seq=0 2024-03-21T11:28:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.043+0000 7f0bcea00640 10 client.27340 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:28:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.044+0000 7f0bcea00640 1 -- 172.21.15.47:0/2801326882 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c167d65800 con 0x55c167e09250 2024-03-21T11:28:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.044+0000 7f0bcea00640 10 client.27340 handle_command_reply: tid=0 2024-03-21T11:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.044+0000 7f0bea7f9740 1 client.27340 shutdown 2024-03-21T11:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.044+0000 7f0bea7f9740 1 -- 172.21.15.47:0/2801326882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 msgr2=0x55c167dfdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.044+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/2801326882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167dfdce0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8012700 tx=0x7f0bc8012bc0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.044+0000 7f0bea7f9740 1 -- 172.21.15.47:0/2801326882 shutdown_connections 2024-03-21T11:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.044+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/2801326882 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c167e09250 0x55c167e0b6b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.044+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/2801326882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df53a0 0x55c167e04d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.044+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/2801326882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167dfdce0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.044+0000 7f0bcea00640 0 client.27340 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.044+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/2801326882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167dfd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.044+0000 7f0bea7f9740 1 -- 172.21.15.47:0/2801326882 >> 172.21.15.47:0/2801326882 conn(0x55c167deec00 msgr2=0x55c167dedeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.045+0000 7f0bea7f9740 1 -- 172.21.15.47:0/2801326882 shutdown_connections 2024-03-21T11:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.045+0000 7f0bea7f9740 1 -- 172.21.15.47:0/2801326882 wait complete. 2024-03-21T11:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.045+0000 7f0bea7f9740 20 client.27340 trim_cache size 0 max 16384 2024-03-21T11:28:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.058+0000 7f0bea7f9740 1 Processor -- start 2024-03-21T11:28:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0bea7f9740 1 -- start start 2024-03-21T11:28:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0bea7f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167df7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0bea7f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df49e0 0x55c167e04ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0bea7f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df53a0 0x55c167df8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0bea7f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be01ac3a0 con 0x55c167df3810 2024-03-21T11:28:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0bea7f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be01add20 con 0x55c167df49e0 2024-03-21T11:28:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0bea7f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be01aca00 con 0x55c167df53a0 2024-03-21T11:28:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0be5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167df7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0be5a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df53a0 0x55c167df8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0be5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167df7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47176/0 (socket says 172.21.15.47:47176) 2024-03-21T11:28:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0be5000640 1 -- 172.21.15.47:0/2533931848 learned_addr learned my addr 172.21.15.47:0/2533931848 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0bdfe00640 1 --2- 172.21.15.47:0/2533931848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df49e0 0x55c167e04ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0be5000640 1 -- 172.21.15.47:0/2533931848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df53a0 msgr2=0x55c167df8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0be5000640 1 --2- 172.21.15.47:0/2533931848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df53a0 0x55c167df8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0be5000640 1 -- 172.21.15.47:0/2533931848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df49e0 msgr2=0x55c167e04ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.059+0000 7f0be5000640 1 --2- 172.21.15.47:0/2533931848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df49e0 0x55c167e04ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.060+0000 7f0be5000640 1 -- 172.21.15.47:0/2533931848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c167df4460 con 0x55c167df3810 2024-03-21T11:28:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.060+0000 7f0be5a00640 1 --2- 172.21.15.47:0/2533931848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df53a0 0x55c167df8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:56.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.060+0000 7f0be5000640 1 --2- 172.21.15.47:0/2533931848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167df7c10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00bd630 tx=0x7f0bd0003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63f956297d612b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.060+0000 7f0bd5600640 1 -- 172.21.15.47:0/2533931848 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00bd470 con 0x55c167df3810 2024-03-21T11:28:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.060+0000 7f0bd5600640 1 -- 172.21.15.47:0/2533931848 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bd00be030 con 0x55c167df3810 2024-03-21T11:28:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.060+0000 7f0bd5600640 1 -- 172.21.15.47:0/2533931848 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00bd700 con 0x55c167df3810 2024-03-21T11:28:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.061+0000 7f0bea7f9740 1 -- 172.21.15.47:0/2533931848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 msgr2=0x55c167df7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.061+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/2533931848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167df7c10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00bd630 tx=0x7f0bd0003d80 comp rx=0 tx=0).stop 2024-03-21T11:28:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.061+0000 7f0bea7f9740 1 -- 172.21.15.47:0/2533931848 shutdown_connections 2024-03-21T11:28:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.061+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/2533931848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df53a0 0x55c167df8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.061+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/2533931848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df49e0 0x55c167e04ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.061+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/2533931848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167df7c10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.061+0000 7f0bea7f9740 1 -- 172.21.15.47:0/2533931848 >> 172.21.15.47:0/2533931848 conn(0x55c167deec00 msgr2=0x55c167df26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.061+0000 7f0bea7f9740 1 -- 172.21.15.47:0/2533931848 shutdown_connections 2024-03-21T11:28:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.061+0000 7f0bea7f9740 1 -- 172.21.15.47:0/2533931848 wait complete. 2024-03-21T11:28:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.061+0000 7f0bea7f9740 1 -- start start 2024-03-21T11:28:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.062+0000 7f0bea7f9740 1 Processor -- start 2024-03-21T11:28:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.062+0000 7f0bea7f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167e02ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.062+0000 7f0bea7f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167e03400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.062+0000 7f0bea7f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df53a0 0x55c167e04230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.062+0000 7f0bea7f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be01ad6c0 con 0x55c167df3810 2024-03-21T11:28:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.062+0000 7f0bea7f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be01ad060 con 0x55c167df53a0 2024-03-21T11:28:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.062+0000 7f0bea7f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be01aa560 con 0x55c167df49e0 2024-03-21T11:28:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.062+0000 7f0bea7f9740 10 client.0 _ll_register_callbacks cb 0x7f0bc8009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.062+0000 7f0be5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167e02ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.062+0000 7f0bdfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167e03400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.062+0000 7f0be5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167e02ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47190/0 (socket says 172.21.15.47:47190) 2024-03-21T11:28:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.062+0000 7f0be5000640 1 -- 172.21.15.47:0/3131383639 learned_addr learned my addr 172.21.15.47:0/3131383639 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.062+0000 7f0be5a00640 1 --2- 172.21.15.47:0/3131383639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df53a0 0x55c167e04230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.063+0000 7f0bdfe00640 1 -- 172.21.15.47:0/3131383639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df53a0 msgr2=0x55c167e04230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.063+0000 7f0bdfe00640 1 --2- 172.21.15.47:0/3131383639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df53a0 0x55c167e04230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.063+0000 7f0bdfe00640 1 -- 172.21.15.47:0/3131383639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 msgr2=0x55c167e02ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.063+0000 7f0bdfe00640 1 --2- 172.21.15.47:0/3131383639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167e02ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.063+0000 7f0bdfe00640 1 -- 172.21.15.47:0/3131383639 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c167e04920 con 0x55c167df49e0 2024-03-21T11:28:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.063+0000 7f0bdfe00640 1 --2- 172.21.15.47:0/3131383639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167e03400 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009460 tx=0x7f0bc8005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d6c5e8135b0583d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.063+0000 7f0bcea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.063+0000 7f0bcea00640 1 -- 172.21.15.47:0/3131383639 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc801cc90 con 0x55c167df49e0 2024-03-21T11:28:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.063+0000 7f0bcea00640 1 -- 172.21.15.47:0/3131383639 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0bc800f790 con 0x55c167df49e0 2024-03-21T11:28:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.063+0000 7f0bea7f9740 1 -- 172.21.15.47:0/3131383639 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bd00be030 con 0x55c167df49e0 2024-03-21T11:28:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.064+0000 7f0bcea00640 1 -- 172.21.15.47:0/3131383639 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bc801c630 con 0x55c167df49e0 2024-03-21T11:28:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.064+0000 7f0bea7f9740 10 client.27346 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.064+0000 7f0bea7f9740 1 -- 172.21.15.47:0/3131383639 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c167e65530 con 0x55c167df49e0 2024-03-21T11:28:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.064+0000 7f0bcea00640 1 -- 172.21.15.47:0/3131383639 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bd00be030 con 0x55c167df49e0 2024-03-21T11:28:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.064+0000 7f0bcea00640 1 -- 172.21.15.47:0/3131383639 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0bc8003b40 con 0x55c167df49e0 2024-03-21T11:28:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.064+0000 7f0bea7f9740 10 client.27346 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.064+0000 7f0bea7f9740 10 client.27346 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:28:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.064+0000 7f0bea7f9740 20 client.27346 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.064+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/3131383639 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c167e65d50 0x55c167e68170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.064+0000 7f0bea7f9740 4 client.27346 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:28:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.064+0000 7f0bea7f9740 1 -- 172.21.15.47:0/3131383639 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c167d5fb50 con 0x55c167e65d50 2024-03-21T11:28:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.064+0000 7f0be5000640 1 --2- 172.21.15.47:0/3131383639 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c167e65d50 0x55c167e68170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.065+0000 7f0be5000640 1 --2- 172.21.15.47:0/3131383639 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c167e65d50 0x55c167e68170 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b363b17f5ea8528e server_cookie=b872e6019cec0b52 in_seq=0 out_seq=0 2024-03-21T11:28:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.065+0000 7f0bcea00640 10 client.27346 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:28:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.065+0000 7f0bcea00640 1 -- 172.21.15.47:0/3131383639 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c167d5fb50 con 0x55c167e65d50 2024-03-21T11:28:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.065+0000 7f0bcea00640 10 client.27346 handle_command_reply: tid=0 2024-03-21T11:28:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.065+0000 7f0bea7f9740 1 client.27346 shutdown 2024-03-21T11:28:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.065+0000 7f0bea7f9740 1 -- 172.21.15.47:0/3131383639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 msgr2=0x55c167e03400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.065+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/3131383639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167e03400 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009460 tx=0x7f0bc8005840 comp rx=0 tx=0).stop 2024-03-21T11:28:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.065+0000 7f0bea7f9740 1 -- 172.21.15.47:0/3131383639 shutdown_connections 2024-03-21T11:28:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.065+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/3131383639 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c167e65d50 0x55c167e68170 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.065+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/3131383639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167df53a0 0x55c167e04230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.065+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/3131383639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167df49e0 0x55c167e03400 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.065+0000 7f0bcea00640 0 client.27346 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:28:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.065+0000 7f0bea7f9740 1 --2- 172.21.15.47:0/3131383639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167df3810 0x55c167e02ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.065+0000 7f0bea7f9740 1 -- 172.21.15.47:0/3131383639 >> 172.21.15.47:0/3131383639 conn(0x55c167deec00 msgr2=0x55c167ded620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.066+0000 7f0bea7f9740 1 -- 172.21.15.47:0/3131383639 shutdown_connections 2024-03-21T11:28:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.066+0000 7f0bea7f9740 1 -- 172.21.15.47:0/3131383639 wait complete. 2024-03-21T11:28:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.066+0000 7f0bea7f9740 20 client.27346 trim_cache size 0 max 16384 2024-03-21T11:28:56.072 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:28:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.066+0000 7f0be7a00640 1 -- 172.21.15.47:0/4282102953 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0bc40a2f20 msgr2=0x7f0bc40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.066+0000 7f0be7a00640 1 --2- 172.21.15.47:0/4282102953 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0bc40a2f20 0x7f0bc40a53e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55c167df48e0 tx=0x7f0bd8005f90 comp rx=0 tx=0).stop 2024-03-21T11:28:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.066+0000 7f0be7a00640 1 -- 172.21.15.47:0/4282102953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be01038e0 msgr2=0x7f0be0199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.066+0000 7f0be7a00640 1 --2- 172.21.15.47:0/4282102953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be01038e0 0x7f0be0199c70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0000f80 tx=0x7f0bd0032ec0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.066+0000 7f0be7a00640 1 -- 172.21.15.47:0/4282102953 shutdown_connections 2024-03-21T11:28:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.066+0000 7f0be7a00640 1 --2- 172.21.15.47:0/4282102953 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0bc40a2f20 0x7f0bc40a53e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.066+0000 7f0be7a00640 1 --2- 172.21.15.47:0/4282102953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be0105490 0x7f0be01a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.066+0000 7f0be7a00640 1 --2- 172.21.15.47:0/4282102953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0be0104ad0 0x7f0be019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.066+0000 7f0be7a00640 1 --2- 172.21.15.47:0/4282102953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0be01038e0 0x7f0be0199c70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.066+0000 7f0be7a00640 1 -- 172.21.15.47:0/4282102953 >> 172.21.15.47:0/4282102953 conn(0x7f0be00fee50 msgr2=0x7f0be0107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.067+0000 7f0be7a00640 1 -- 172.21.15.47:0/4282102953 shutdown_connections 2024-03-21T11:28:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.067+0000 7f0be7a00640 1 -- 172.21.15.47:0/4282102953 wait complete. 2024-03-21T11:28:56.085 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:28:56.085 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:28:56.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.318+0000 7f48b9e00640 1 -- 172.21.15.47:0/3582089014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48b4109000 msgr2=0x7f48b410f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.318+0000 7f48b9e00640 1 --2- 172.21.15.47:0/3582089014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48b4109000 0x7f48b410f9e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f489c009900 tx=0x7f489c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:28:56.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.318+0000 7f48b9e00640 1 -- 172.21.15.47:0/3582089014 shutdown_connections 2024-03-21T11:28:56.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.318+0000 7f48b9e00640 1 --2- 172.21.15.47:0/3582089014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b410ff20 0x7f48b4112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.318+0000 7f48b9e00640 1 --2- 172.21.15.47:0/3582089014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48b4109000 0x7f48b410f9e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.318+0000 7f48b9e00640 1 --2- 172.21.15.47:0/3582089014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48b41086a0 0x7f48b4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.318+0000 7f48b9e00640 1 -- 172.21.15.47:0/3582089014 >> 172.21.15.47:0/3582089014 conn(0x7f48b406e500 msgr2=0x7f48b4070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.319+0000 7f48b9e00640 1 -- 172.21.15.47:0/3582089014 shutdown_connections 2024-03-21T11:28:56.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.319+0000 7f48b9e00640 1 -- 172.21.15.47:0/3582089014 wait complete. 2024-03-21T11:28:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.321+0000 7f48b9e00640 1 Processor -- start 2024-03-21T11:28:56.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.321+0000 7f48b9e00640 1 -- start start 2024-03-21T11:28:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.322+0000 7f48b9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48b41086a0 0x7f48b410b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.322+0000 7f48b9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b4109000 0x7f48b4109970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.322+0000 7f48b9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48b410ff20 0x7f48b4109eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.322+0000 7f48b9e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4115b80 con 0x7f48b41086a0 2024-03-21T11:28:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.322+0000 7f48b9e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4115a00 con 0x7f48b4109000 2024-03-21T11:28:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.322+0000 7f48b9e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4115d00 con 0x7f48b410ff20 2024-03-21T11:28:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.322+0000 7f48b3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48b410ff20 0x7f48b4109eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.322+0000 7f48b3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48b410ff20 0x7f48b4109eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46032/0 (socket says 172.21.15.47:46032) 2024-03-21T11:28:56.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.322+0000 7f48b3400640 1 -- 172.21.15.47:0/3242520600 learned_addr learned my addr 172.21.15.47:0/3242520600 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.322+0000 7f48b2000640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b4109000 0x7f48b4109970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.322+0000 7f48b2a00640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48b41086a0 0x7f48b410b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.323+0000 7f48b3400640 1 -- 172.21.15.47:0/3242520600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b4109000 msgr2=0x7f48b4109970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.323+0000 7f48b3400640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b4109000 0x7f48b4109970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.323+0000 7f48b3400640 1 -- 172.21.15.47:0/3242520600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48b41086a0 msgr2=0x7f48b410b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.323+0000 7f48b3400640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48b41086a0 0x7f48b410b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.323+0000 7f48b3400640 1 -- 172.21.15.47:0/3242520600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b410a740 con 0x7f48b410ff20 2024-03-21T11:28:56.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.323+0000 7f48b2a00640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48b41086a0 0x7f48b410b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:56.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.323+0000 7f48b3400640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48b410ff20 0x7f48b4109eb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f48a400d6a0 tx=0x7f48a400db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.324+0000 7f48b8a00640 1 -- 172.21.15.47:0/3242520600 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4017070 con 0x7f48b410ff20 2024-03-21T11:28:56.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.324+0000 7f48b8a00640 1 -- 172.21.15.47:0/3242520600 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48a4009820 con 0x7f48b410ff20 2024-03-21T11:28:56.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.324+0000 7f48b9e00640 1 -- 172.21.15.47:0/3242520600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b41aa360 con 0x7f48b410ff20 2024-03-21T11:28:56.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.325+0000 7f48b9e00640 1 -- 172.21.15.47:0/3242520600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b41aa8b0 con 0x7f48b410ff20 2024-03-21T11:28:56.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.326+0000 7f48b8a00640 1 -- 172.21.15.47:0/3242520600 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4004ce0 con 0x7f48b410ff20 2024-03-21T11:28:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.331+0000 7f48b8a00640 1 -- 172.21.15.47:0/3242520600 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f48a4010460 con 0x7f48b410ff20 2024-03-21T11:28:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.331+0000 7f48bc9f2740 1 Processor -- start 2024-03-21T11:28:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.331+0000 7f48bc9f2740 1 -- start start 2024-03-21T11:28:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.332+0000 7f48bc9f2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92ed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.332+0000 7f48bc9f2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 0x555ba92ee670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.332+0000 7f48bc9f2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 0x555ba92f5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.332+0000 7f48bc9f2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba92492c0 con 0x555ba92eebb0 2024-03-21T11:28:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.332+0000 7f48bc9f2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba91f1c90 con 0x555ba92ee1f0 2024-03-21T11:28:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.332+0000 7f48bc9f2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba9245360 con 0x555ba92ed020 2024-03-21T11:28:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.332+0000 7f48b2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92ed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.332+0000 7f48b2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 0x555ba92f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.332+0000 7f48b2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92ed440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46034/0 (socket says 172.21.15.47:46034) 2024-03-21T11:28:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.332+0000 7f48b2000640 1 -- 172.21.15.47:0/4169540535 learned_addr learned my addr 172.21.15.47:0/4169540535 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.333+0000 7f48b2a00640 1 --2- 172.21.15.47:0/4169540535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 0x555ba92ee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.333+0000 7f48b2a00640 1 -- 172.21.15.47:0/4169540535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 msgr2=0x555ba92ed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.334+0000 7f48b2a00640 1 --2- 172.21.15.47:0/4169540535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92ed440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.334+0000 7f48b2a00640 1 -- 172.21.15.47:0/4169540535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 msgr2=0x555ba92f5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.334+0000 7f48b2a00640 1 --2- 172.21.15.47:0/4169540535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 0x555ba92f5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.334+0000 7f48b2a00640 1 -- 172.21.15.47:0/4169540535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ba9216080 con 0x555ba92ee1f0 2024-03-21T11:28:56.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.334+0000 7f48b2000640 1 --2- 172.21.15.47:0/4169540535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 0x555ba92f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:28:56.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.335+0000 7f48b2a00640 1 --2- 172.21.15.47:0/4169540535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 0x555ba92ee670 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f48ac010450 tx=0x7f48ac010910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d1224138fb6c462 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.335+0000 7f48a3e00640 1 -- 172.21.15.47:0/4169540535 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac011760 con 0x555ba92ee1f0 2024-03-21T11:28:56.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.335+0000 7f48a3e00640 1 -- 172.21.15.47:0/4169540535 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48ac015070 con 0x555ba92ee1f0 2024-03-21T11:28:56.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.335+0000 7f48a3e00640 1 -- 172.21.15.47:0/4169540535 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac01bde0 con 0x555ba92ee1f0 2024-03-21T11:28:56.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.336+0000 7f48bc9f2740 1 -- 172.21.15.47:0/4169540535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 msgr2=0x555ba92ee670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.336+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/4169540535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 0x555ba92ee670 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f48ac010450 tx=0x7f48ac010910 comp rx=0 tx=0).stop 2024-03-21T11:28:56.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.336+0000 7f48bc9f2740 1 -- 172.21.15.47:0/4169540535 shutdown_connections 2024-03-21T11:28:56.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.336+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/4169540535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 0x555ba92f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.336+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/4169540535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 0x555ba92ee670 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.337+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/4169540535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92ed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.337+0000 7f48bc9f2740 1 -- 172.21.15.47:0/4169540535 >> 172.21.15.47:0/4169540535 conn(0x555ba92e8410 msgr2=0x555ba92e8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.337+0000 7f48bc9f2740 1 -- 172.21.15.47:0/4169540535 shutdown_connections 2024-03-21T11:28:56.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.337+0000 7f48bc9f2740 1 -- 172.21.15.47:0/4169540535 wait complete. 2024-03-21T11:28:56.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.338+0000 7f48bc9f2740 1 -- start start 2024-03-21T11:28:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.338+0000 7f48bc9f2740 1 Processor -- start 2024-03-21T11:28:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.339+0000 7f48b8a00640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48980a2eb0 0x7f48980a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.339+0000 7f48b8a00640 1 -- 172.21.15.47:0/3242520600 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f48a40cf210 con 0x7f48b410ff20 2024-03-21T11:28:56.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.339+0000 7f48b2000640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48980a2eb0 0x7f48980a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.340+0000 7f48bc9f2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92f7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.340+0000 7f48bc9f2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 0x555ba92f7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.340+0000 7f48bc9f2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 0x555ba92fe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.340+0000 7f48b2000640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48980a2eb0 0x7f48980a5370 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f489c009230 tx=0x7f489c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.340+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.340+0000 7f48bc9f2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba92f5bc0 con 0x555ba92eebb0 2024-03-21T11:28:56.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.340+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92f7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46038/0 (socket says 172.21.15.47:46038) 2024-03-21T11:28:56.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.340+0000 7f48b2a00640 1 -- 172.21.15.47:0/567453360 learned_addr learned my addr 172.21.15.47:0/567453360 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.340+0000 7f48b2000640 1 --2- 172.21.15.47:0/567453360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 0x555ba92f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.341+0000 7f48bc9f2740 1 -- 172.21.15.47:0/567453360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba92050d0 con 0x555ba92ee1f0 2024-03-21T11:28:56.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.341+0000 7f48bc9f2740 1 -- 172.21.15.47:0/567453360 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba92f5d40 con 0x555ba92ed020 2024-03-21T11:28:56.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.341+0000 7f48bc9f2740 10 client.0 _ll_register_callbacks cb 0x555ba9209e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:56.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.341+0000 7f48b3400640 1 --2- 172.21.15.47:0/567453360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 0x555ba92fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.341+0000 7f48b2a00640 1 -- 172.21.15.47:0/567453360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 msgr2=0x555ba92f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.341+0000 7f48b2a00640 1 --2- 172.21.15.47:0/567453360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 0x555ba92f7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.341+0000 7f48b2a00640 1 -- 172.21.15.47:0/567453360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 msgr2=0x555ba92fe880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.341+0000 7f48b2a00640 1 --2- 172.21.15.47:0/567453360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 0x555ba92fe880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.341+0000 7f48b2a00640 1 -- 172.21.15.47:0/567453360 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ba8f5d3f0 con 0x555ba92ed020 2024-03-21T11:28:56.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.341+0000 7f48b2000640 1 --2- 172.21.15.47:0/567453360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 0x555ba92f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:56.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.342+0000 7f48b2a00640 1 --2- 172.21.15.47:0/567453360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92f7300 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f48ac000f80 tx=0x7f48ac01b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4554dcf277d272f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.342+0000 7f48a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:56.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.342+0000 7f48a0c00640 1 -- 172.21.15.47:0/567453360 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac02a060 con 0x555ba92ed020 2024-03-21T11:28:56.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.342+0000 7f48bc9f2740 1 -- 172.21.15.47:0/567453360 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48ac015070 con 0x555ba92ed020 2024-03-21T11:28:56.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.342+0000 7f48a0c00640 1 -- 172.21.15.47:0/567453360 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f48ac004a40 con 0x555ba92ed020 2024-03-21T11:28:56.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.342+0000 7f48a0c00640 1 -- 172.21.15.47:0/567453360 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48ac031050 con 0x555ba92ed020 2024-03-21T11:28:56.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.342+0000 7f48bc9f2740 10 client.27358 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:56.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.342+0000 7f48bc9f2740 1 -- 172.21.15.47:0/567453360 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555ba90277d0 con 0x555ba92ed020 2024-03-21T11:28:56.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.343+0000 7f48a0c00640 1 -- 172.21.15.47:0/567453360 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48ac015070 con 0x555ba92ed020 2024-03-21T11:28:56.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.343+0000 7f48a0c00640 1 -- 172.21.15.47:0/567453360 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f48ac005040 con 0x555ba92ed020 2024-03-21T11:28:56.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.343+0000 7f48bc9f2740 10 client.27358 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:56.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.343+0000 7f48bc9f2740 10 client.27358 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:28:56.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.343+0000 7f48bc9f2740 20 client.27358 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:56.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.343+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/567453360 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x555ba9302930 0x555ba9304d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.343+0000 7f48bc9f2740 4 client.27358 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:56.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.343+0000 7f48bc9f2740 1 -- 172.21.15.47:0/567453360 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ba9256800 con 0x555ba9302930 2024-03-21T11:28:56.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.343+0000 7f48b2000640 1 --2- 172.21.15.47:0/567453360 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x555ba9302930 0x555ba9304d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.344+0000 7f48b2000640 1 --2- 172.21.15.47:0/567453360 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x555ba9302930 0x555ba9304d20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f755e580f3492a21 server_cookie=6d9fc5edf279b97d in_seq=0 out_seq=0 2024-03-21T11:28:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.344+0000 7f48a0c00640 10 client.27358 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:28:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.345+0000 7f48a0c00640 1 -- 172.21.15.47:0/567453360 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555ba9256800 con 0x555ba9302930 2024-03-21T11:28:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.345+0000 7f48a0c00640 10 client.27358 handle_command_reply: tid=0 2024-03-21T11:28:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.345+0000 7f48bc9f2740 1 client.27358 shutdown 2024-03-21T11:28:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.345+0000 7f48bc9f2740 1 -- 172.21.15.47:0/567453360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 msgr2=0x555ba92f7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.345+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/567453360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92f7300 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f48ac000f80 tx=0x7f48ac01b610 comp rx=0 tx=0).stop 2024-03-21T11:28:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.345+0000 7f48bc9f2740 1 -- 172.21.15.47:0/567453360 shutdown_connections 2024-03-21T11:28:56.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.345+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/567453360 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x555ba9302930 0x555ba9304d20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.345+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/567453360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 0x555ba92fe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.345+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/567453360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 0x555ba92f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.345+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/567453360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92f7300 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.345+0000 7f48a0c00640 0 client.27358 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:28:56.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.346+0000 7f48bc9f2740 1 -- 172.21.15.47:0/567453360 >> 172.21.15.47:0/567453360 conn(0x555ba92e8410 msgr2=0x555ba92deeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.346+0000 7f48bc9f2740 1 -- 172.21.15.47:0/567453360 shutdown_connections 2024-03-21T11:28:56.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.346+0000 7f48bc9f2740 1 -- 172.21.15.47:0/567453360 wait complete. 2024-03-21T11:28:56.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.346+0000 7f48bc9f2740 20 client.27358 trim_cache size 0 max 16384 2024-03-21T11:28:56.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.363+0000 7f48bc9f2740 1 Processor -- start 2024-03-21T11:28:56.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.363+0000 7f48bc9f2740 1 -- start start 2024-03-21T11:28:56.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.363+0000 7f48bc9f2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92e87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.363+0000 7f48bc9f2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 0x555ba92f1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.363+0000 7f48bc9f2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 0x555ba92f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.363+0000 7f48bc9f2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b41b0860 con 0x555ba92eebb0 2024-03-21T11:28:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.363+0000 7f48bc9f2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b41b21e0 con 0x555ba92ee1f0 2024-03-21T11:28:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.363+0000 7f48bc9f2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b41b0ec0 con 0x555ba92ed020 2024-03-21T11:28:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.363+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92e87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.363+0000 7f48b3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 0x555ba92f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.363+0000 7f48b2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 0x555ba92f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.363+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92e87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46048/0 (socket says 172.21.15.47:46048) 2024-03-21T11:28:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.363+0000 7f48b2a00640 1 -- 172.21.15.47:0/4084339220 learned_addr learned my addr 172.21.15.47:0/4084339220 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.364+0000 7f48b2a00640 1 -- 172.21.15.47:0/4084339220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 msgr2=0x555ba92f1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.364+0000 7f48b2a00640 1 --2- 172.21.15.47:0/4084339220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 0x555ba92f1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.364+0000 7f48b2a00640 1 -- 172.21.15.47:0/4084339220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 msgr2=0x555ba92f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.364+0000 7f48b2a00640 1 --2- 172.21.15.47:0/4084339220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 0x555ba92f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.364+0000 7f48b2a00640 1 -- 172.21.15.47:0/4084339220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ba92edaa0 con 0x555ba92ed020 2024-03-21T11:28:56.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.364+0000 7f48b2a00640 1 --2- 172.21.15.47:0/4084339220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92e87f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f48ac010950 tx=0x7f48ac009bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29781411eefbb10a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.364+0000 7f48a3400640 1 -- 172.21.15.47:0/4084339220 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac02a050 con 0x555ba92ed020 2024-03-21T11:28:56.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.364+0000 7f48a3400640 1 -- 172.21.15.47:0/4084339220 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48ac015070 con 0x555ba92ed020 2024-03-21T11:28:56.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.364+0000 7f48a3400640 1 -- 172.21.15.47:0/4084339220 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac004e40 con 0x555ba92ed020 2024-03-21T11:28:56.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.365+0000 7f48bc9f2740 1 -- 172.21.15.47:0/4084339220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 msgr2=0x555ba92e87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.365+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/4084339220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92e87f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f48ac010950 tx=0x7f48ac009bc0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.365+0000 7f48bc9f2740 1 -- 172.21.15.47:0/4084339220 shutdown_connections 2024-03-21T11:28:56.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.365+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/4084339220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92eebb0 0x555ba92f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.365+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/4084339220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92ee1f0 0x555ba92f1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.365+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/4084339220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92e87f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.365+0000 7f48bc9f2740 1 -- 172.21.15.47:0/4084339220 >> 172.21.15.47:0/4084339220 conn(0x555ba92e8410 msgr2=0x555ba92ebf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.365+0000 7f48bc9f2740 1 -- 172.21.15.47:0/4084339220 shutdown_connections 2024-03-21T11:28:56.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.365+0000 7f48bc9f2740 1 -- 172.21.15.47:0/4084339220 wait complete. 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.366+0000 7f48bc9f2740 1 -- start start 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.366+0000 7f48bc9f2740 1 Processor -- start 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.366+0000 7f48bc9f2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92f9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.366+0000 7f48bc9f2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92ee1f0 0x555ba92f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.366+0000 7f48bc9f2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92eebb0 0x555ba92fc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.366+0000 7f48bc9f2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b41b1b80 con 0x555ba92ee1f0 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.366+0000 7f48bc9f2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b41b1520 con 0x555ba92eebb0 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.366+0000 7f48bc9f2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b41aea20 con 0x555ba92ed020 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.366+0000 7f48bc9f2740 10 client.0 _ll_register_callbacks cb 0x7f48ac009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.366+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.367+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92f9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46054/0 (socket says 172.21.15.47:46054) 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.367+0000 7f48b2a00640 1 -- 172.21.15.47:0/3222014042 learned_addr learned my addr 172.21.15.47:0/3222014042 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.367+0000 7f48b2000640 1 --2- 172.21.15.47:0/3222014042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92ee1f0 0x555ba92f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.367+0000 7f48b3400640 1 --2- 172.21.15.47:0/3222014042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92eebb0 0x555ba92fc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.367+0000 7f48b2a00640 1 -- 172.21.15.47:0/3222014042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92eebb0 msgr2=0x555ba92fc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.367+0000 7f48b2a00640 1 --2- 172.21.15.47:0/3222014042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92eebb0 0x555ba92fc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.367+0000 7f48b2a00640 1 -- 172.21.15.47:0/3222014042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92ee1f0 msgr2=0x555ba92f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.367+0000 7f48b2a00640 1 --2- 172.21.15.47:0/3222014042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92ee1f0 0x555ba92f98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.367+0000 7f48b2a00640 1 -- 172.21.15.47:0/3222014042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ba92fcec0 con 0x555ba92ed020 2024-03-21T11:28:56.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.367+0000 7f48b2a00640 1 --2- 172.21.15.47:0/3222014042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92f9370 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f48ac000f80 tx=0x7f48ac009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0fe776120a88be7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.367+0000 7f48a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.367+0000 7f48a0c00640 1 -- 172.21.15.47:0/3222014042 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac02a050 con 0x555ba92ed020 2024-03-21T11:28:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.367+0000 7f48a0c00640 1 -- 172.21.15.47:0/3222014042 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f48ac01b7f0 con 0x555ba92ed020 2024-03-21T11:28:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.368+0000 7f48bc9f2740 1 -- 172.21.15.47:0/3222014042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48ac015070 con 0x555ba92ed020 2024-03-21T11:28:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.368+0000 7f48a0c00640 1 -- 172.21.15.47:0/3222014042 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48ac01b9f0 con 0x555ba92ed020 2024-03-21T11:28:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.368+0000 7f48bc9f2740 10 client.27364 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.368+0000 7f48bc9f2740 1 -- 172.21.15.47:0/3222014042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555ba92fe1c0 con 0x555ba92ed020 2024-03-21T11:28:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.368+0000 7f48a0c00640 1 -- 172.21.15.47:0/3222014042 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48ac015070 con 0x555ba92ed020 2024-03-21T11:28:56.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.368+0000 7f48a0c00640 1 -- 172.21.15.47:0/3222014042 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f48ac03c070 con 0x555ba92ed020 2024-03-21T11:28:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.368+0000 7f48bc9f2740 10 client.27364 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.368+0000 7f48bc9f2740 10 client.27364 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:28:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.368+0000 7f48bc9f2740 20 client.27364 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.368+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/3222014042 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x555ba935e6e0 0x555ba9360b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.368+0000 7f48bc9f2740 4 client.27364 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:28:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.368+0000 7f48bc9f2740 1 -- 172.21.15.47:0/3222014042 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555ba9250b50 con 0x555ba935e6e0 2024-03-21T11:28:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.369+0000 7f48b2000640 1 --2- 172.21.15.47:0/3222014042 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x555ba935e6e0 0x555ba9360b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.369+0000 7f48b2000640 1 --2- 172.21.15.47:0/3222014042 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x555ba935e6e0 0x555ba9360b00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ad8074dce2f19e0b server_cookie=a1116517f5c4f1b2 in_seq=0 out_seq=0 2024-03-21T11:28:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.369+0000 7f48a0c00640 10 client.27364 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:28:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.369+0000 7f48a0c00640 1 -- 172.21.15.47:0/3222014042 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555ba9250b50 con 0x555ba935e6e0 2024-03-21T11:28:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.369+0000 7f48a0c00640 10 client.27364 handle_command_reply: tid=0 2024-03-21T11:28:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.369+0000 7f48bc9f2740 1 client.27364 shutdown 2024-03-21T11:28:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.370+0000 7f48bc9f2740 1 -- 172.21.15.47:0/3222014042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 msgr2=0x555ba92f9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.370+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/3222014042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92f9370 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f48ac000f80 tx=0x7f48ac009c60 comp rx=0 tx=0).stop 2024-03-21T11:28:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.370+0000 7f48bc9f2740 1 -- 172.21.15.47:0/3222014042 shutdown_connections 2024-03-21T11:28:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.370+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/3222014042 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x555ba935e6e0 0x555ba9360b00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.370+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/3222014042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ba92eebb0 0x555ba92fc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.370+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/3222014042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ba92ee1f0 0x555ba92f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.370+0000 7f48bc9f2740 1 --2- 172.21.15.47:0/3222014042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ba92ed020 0x555ba92f9370 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.370+0000 7f48bc9f2740 1 -- 172.21.15.47:0/3222014042 >> 172.21.15.47:0/3222014042 conn(0x555ba92e8410 msgr2=0x555ba92ebf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.370+0000 7f48a0c00640 0 client.27364 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:28:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.370+0000 7f48bc9f2740 1 -- 172.21.15.47:0/3222014042 shutdown_connections 2024-03-21T11:28:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.370+0000 7f48bc9f2740 1 -- 172.21.15.47:0/3222014042 wait complete. 2024-03-21T11:28:56.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.370+0000 7f48bc9f2740 20 client.27364 trim_cache size 0 max 16384 2024-03-21T11:28:56.374 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:28:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.371+0000 7f48b9e00640 1 -- 172.21.15.47:0/3242520600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48980a2eb0 msgr2=0x7f48980a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.371+0000 7f48b9e00640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48980a2eb0 0x7f48980a5370 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f489c009230 tx=0x7f489c02e040 comp rx=0 tx=0).stop 2024-03-21T11:28:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.371+0000 7f48b9e00640 1 -- 172.21.15.47:0/3242520600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48b410ff20 msgr2=0x7f48b4109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.371+0000 7f48b9e00640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48b410ff20 0x7f48b4109eb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f48a400d6a0 tx=0x7f48a400db60 comp rx=0 tx=0).stop 2024-03-21T11:28:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.371+0000 7f48b9e00640 1 -- 172.21.15.47:0/3242520600 shutdown_connections 2024-03-21T11:28:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.371+0000 7f48b9e00640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48980a2eb0 0x7f48980a5370 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.371+0000 7f48b9e00640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48b410ff20 0x7f48b4109eb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.371+0000 7f48b9e00640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b4109000 0x7f48b4109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.371+0000 7f48b9e00640 1 --2- 172.21.15.47:0/3242520600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48b41086a0 0x7f48b410b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.371+0000 7f48b9e00640 1 -- 172.21.15.47:0/3242520600 >> 172.21.15.47:0/3242520600 conn(0x7f48b406e500 msgr2=0x7f48b4070210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.371+0000 7f48b9e00640 1 -- 172.21.15.47:0/3242520600 shutdown_connections 2024-03-21T11:28:56.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.371+0000 7f48b9e00640 1 -- 172.21.15.47:0/3242520600 wait complete. 2024-03-21T11:28:56.389 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:28:56.389 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:28:56.514 INFO:tasks.workunit.client.0.smithi047.stdout: 1 350525 8.36 MB/sec execute 100 sec latency 10.631 ms 2024-03-21T11:28:56.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.590+0000 7fee0de00640 1 -- 172.21.15.47:0/2071867985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee08107cc0 msgr2=0x7fee081080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.590+0000 7fee0de00640 1 --2- 172.21.15.47:0/2071867985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee08107cc0 0x7fee081080e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009ca0 tx=0x7fedf802c100 comp rx=0 tx=0).stop 2024-03-21T11:28:56.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.590+0000 7fee0de00640 1 -- 172.21.15.47:0/2071867985 shutdown_connections 2024-03-21T11:28:56.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.590+0000 7fee0de00640 1 --2- 172.21.15.47:0/2071867985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee08109870 0x7fee08110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.590+0000 7fee0de00640 1 --2- 172.21.15.47:0/2071867985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee08108eb0 0x7fee08109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.590+0000 7fee0de00640 1 --2- 172.21.15.47:0/2071867985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee08107cc0 0x7fee081080e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.590+0000 7fee0de00640 1 -- 172.21.15.47:0/2071867985 >> 172.21.15.47:0/2071867985 conn(0x7fee08076c60 msgr2=0x7fee080790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.590+0000 7fee0de00640 1 -- 172.21.15.47:0/2071867985 shutdown_connections 2024-03-21T11:28:56.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.590+0000 7fee0de00640 1 -- 172.21.15.47:0/2071867985 wait complete. 2024-03-21T11:28:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.591+0000 7fee0de00640 1 Processor -- start 2024-03-21T11:28:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.591+0000 7fee0de00640 1 -- start start 2024-03-21T11:28:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.592+0000 7fee0de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee08107cc0 0x7fee0819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.592+0000 7fee0de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee08108eb0 0x7fee0819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.592+0000 7fee0de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee08109870 0x7fee081a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.592+0000 7fee0de00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee08113910 con 0x7fee08107cc0 2024-03-21T11:28:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.592+0000 7fee0de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee08113790 con 0x7fee08108eb0 2024-03-21T11:28:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.592+0000 7fee0de00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee08113a90 con 0x7fee08109870 2024-03-21T11:28:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.592+0000 7fedfe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee08108eb0 0x7fee0819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.592+0000 7fee07400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee08109870 0x7fee081a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.592+0000 7fee06a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee08107cc0 0x7fee0819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.592+0000 7fedfe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee08108eb0 0x7fee0819e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58402/0 (socket says 172.21.15.47:58402) 2024-03-21T11:28:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.592+0000 7fedfe000640 1 -- 172.21.15.47:0/1708207381 learned_addr learned my addr 172.21.15.47:0/1708207381 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.592+0000 7fedfe000640 1 -- 172.21.15.47:0/1708207381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee08109870 msgr2=0x7fee081a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.592+0000 7fedfe000640 1 --2- 172.21.15.47:0/1708207381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee08109870 0x7fee081a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.593+0000 7fedfe000640 1 -- 172.21.15.47:0/1708207381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee08107cc0 msgr2=0x7fee0819e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.593+0000 7fedfe000640 1 --2- 172.21.15.47:0/1708207381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee08107cc0 0x7fee0819e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.593+0000 7fedfe000640 1 -- 172.21.15.47:0/1708207381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee081a5c30 con 0x7fee08108eb0 2024-03-21T11:28:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.593+0000 7fee06a00640 1 --2- 172.21.15.47:0/1708207381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee08107cc0 0x7fee0819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.593+0000 7fedfe000640 1 --2- 172.21.15.47:0/1708207381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee08108eb0 0x7fee0819e550 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fedf000e9f0 tx=0x7fedf000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.593+0000 7fee0ca00640 1 -- 172.21.15.47:0/1708207381 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf000ccc0 con 0x7fee08108eb0 2024-03-21T11:28:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.593+0000 7fee0ca00640 1 -- 172.21.15.47:0/1708207381 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedf0010910 con 0x7fee08108eb0 2024-03-21T11:28:56.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.593+0000 7fee0ca00640 1 -- 172.21.15.47:0/1708207381 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf00188a0 con 0x7fee08108eb0 2024-03-21T11:28:56.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.593+0000 7fee0de00640 1 -- 172.21.15.47:0/1708207381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee081a5f20 con 0x7fee08108eb0 2024-03-21T11:28:56.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.593+0000 7fee0de00640 1 -- 172.21.15.47:0/1708207381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee081a63e0 con 0x7fee08108eb0 2024-03-21T11:28:56.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.596+0000 7fee0ca00640 1 -- 172.21.15.47:0/1708207381 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fedf0010ac0 con 0x7fee08108eb0 2024-03-21T11:28:56.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.597+0000 7fee10a24740 1 Processor -- start 2024-03-21T11:28:56.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.597+0000 7fee10a24740 1 -- start start 2024-03-21T11:28:56.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.597+0000 7fee10a24740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067e9f810 0x55b067e9fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.597+0000 7fee10a24740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea09e0 0x55b067ea0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.597+0000 7fee10a24740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067ea13a0 0x55b067ea7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.597+0000 7fee10a24740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067e042c0 con 0x55b067e9f810 2024-03-21T11:28:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.597+0000 7fee10a24740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067dacc90 con 0x55b067ea13a0 2024-03-21T11:28:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.597+0000 7fee10a24740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067e00360 con 0x55b067ea09e0 2024-03-21T11:28:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.597+0000 7fee06a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea09e0 0x55b067ea0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.597+0000 7fedfe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067ea13a0 0x55b067ea7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.597+0000 7fee06a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea09e0 0x55b067ea0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46102/0 (socket says 172.21.15.47:46102) 2024-03-21T11:28:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.597+0000 7fee06a00640 1 -- 172.21.15.47:0/2369827088 learned_addr learned my addr 172.21.15.47:0/2369827088 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.598+0000 7fee07400640 1 --2- 172.21.15.47:0/2369827088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067e9f810 0x55b067e9fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.598+0000 7fee06a00640 1 -- 172.21.15.47:0/2369827088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067ea13a0 msgr2=0x55b067ea7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.598+0000 7fee06a00640 1 --2- 172.21.15.47:0/2369827088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067ea13a0 0x55b067ea7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.598+0000 7fee06a00640 1 -- 172.21.15.47:0/2369827088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067e9f810 msgr2=0x55b067e9fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.598+0000 7fee06a00640 1 --2- 172.21.15.47:0/2369827088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067e9f810 0x55b067e9fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.598+0000 7fee06a00640 1 -- 172.21.15.47:0/2369827088 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b067dd1080 con 0x55b067ea09e0 2024-03-21T11:28:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.598+0000 7fee07400640 1 --2- 172.21.15.47:0/2369827088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067e9f810 0x55b067e9fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.598+0000 7fee0ca00640 1 --2- 172.21.15.47:0/1708207381 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fedec0a2f20 0x7fedec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.598+0000 7fee06a00640 1 --2- 172.21.15.47:0/2369827088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea09e0 0x55b067ea0e60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fedf802c710 tx=0x7fedf8005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbc0b6a6e32f0f0d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.599+0000 7fee0ca00640 1 -- 172.21.15.47:0/1708207381 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fedf00cfb90 con 0x7fee08108eb0 2024-03-21T11:28:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.599+0000 7fee07400640 1 --2- 172.21.15.47:0/1708207381 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fedec0a2f20 0x7fedec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.600+0000 7fee07400640 1 --2- 172.21.15.47:0/1708207381 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fedec0a2f20 0x7fedec0a53e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55b067ea08e0 tx=0x7fee00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.602+0000 7fedf7e00640 1 -- 172.21.15.47:0/2369827088 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8043070 con 0x55b067ea09e0 2024-03-21T11:28:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.602+0000 7fedf7e00640 1 -- 172.21.15.47:0/2369827088 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedf8003970 con 0x55b067ea09e0 2024-03-21T11:28:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.602+0000 7fedf7e00640 1 -- 172.21.15.47:0/2369827088 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8039040 con 0x55b067ea09e0 2024-03-21T11:28:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.603+0000 7fee10a24740 1 -- 172.21.15.47:0/2369827088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea09e0 msgr2=0x55b067ea0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.603+0000 7fee10a24740 1 --2- 172.21.15.47:0/2369827088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea09e0 0x55b067ea0e60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fedf802c710 tx=0x7fedf8005ef0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.603+0000 7fee10a24740 1 -- 172.21.15.47:0/2369827088 shutdown_connections 2024-03-21T11:28:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.603+0000 7fee10a24740 1 --2- 172.21.15.47:0/2369827088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067ea13a0 0x55b067ea7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.603+0000 7fee10a24740 1 --2- 172.21.15.47:0/2369827088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea09e0 0x55b067ea0e60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.603+0000 7fee10a24740 1 --2- 172.21.15.47:0/2369827088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067e9f810 0x55b067e9fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.603+0000 7fee10a24740 1 -- 172.21.15.47:0/2369827088 >> 172.21.15.47:0/2369827088 conn(0x55b067e9ac00 msgr2=0x55b067e9b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.603+0000 7fee10a24740 1 -- 172.21.15.47:0/2369827088 shutdown_connections 2024-03-21T11:28:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.603+0000 7fee10a24740 1 -- 172.21.15.47:0/2369827088 wait complete. 2024-03-21T11:28:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.604+0000 7fee10a24740 1 -- start start 2024-03-21T11:28:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.604+0000 7fee10a24740 1 Processor -- start 2024-03-21T11:28:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.604+0000 7fee10a24740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067e9f810 0x55b067ea9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.604+0000 7fee10a24740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea09e0 0x55b067ea9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.604+0000 7fee10a24740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067ea13a0 0x55b067eb0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.604+0000 7fee10a24740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067ea8350 con 0x55b067ea13a0 2024-03-21T11:28:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.604+0000 7fee10a24740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067dc00d0 con 0x55b067e9f810 2024-03-21T11:28:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.604+0000 7fee10a24740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b067ea84d0 con 0x55b067ea09e0 2024-03-21T11:28:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.604+0000 7fee10a24740 10 client.0 _ll_register_callbacks cb 0x55b067dc4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.605+0000 7fedfe000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea09e0 0x55b067ea9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.605+0000 7fedfe000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea09e0 0x55b067ea9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46108/0 (socket says 172.21.15.47:46108) 2024-03-21T11:28:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.605+0000 7fedfe000640 1 -- 172.21.15.47:0/2331136691 learned_addr learned my addr 172.21.15.47:0/2331136691 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.605+0000 7fee06a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067e9f810 0x55b067ea9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.605+0000 7fee07400640 1 --2- 172.21.15.47:0/2331136691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067ea13a0 0x55b067eb0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.605+0000 7fee06a00640 1 -- 172.21.15.47:0/2331136691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea09e0 msgr2=0x55b067ea9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.605+0000 7fee06a00640 1 --2- 172.21.15.47:0/2331136691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea09e0 0x55b067ea9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.605+0000 7fee06a00640 1 -- 172.21.15.47:0/2331136691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067ea13a0 msgr2=0x55b067eb0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.605+0000 7fee06a00640 1 --2- 172.21.15.47:0/2331136691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067ea13a0 0x55b067eb0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.605+0000 7fee06a00640 1 -- 172.21.15.47:0/2331136691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b067b183f0 con 0x55b067e9f810 2024-03-21T11:28:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.605+0000 7fedfe000640 1 --2- 172.21.15.47:0/2331136691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea09e0 0x55b067ea9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.605+0000 7fee06a00640 1 --2- 172.21.15.47:0/2331136691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067e9f810 0x55b067ea9a90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009ca0 tx=0x7fedf80099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af3a897a9b482dd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.605+0000 7fedf4c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.606+0000 7fedf4c00640 1 -- 172.21.15.47:0/2331136691 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf804c030 con 0x55b067e9f810 2024-03-21T11:28:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.606+0000 7fedf4c00640 1 -- 172.21.15.47:0/2331136691 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fedf8032690 con 0x55b067e9f810 2024-03-21T11:28:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.606+0000 7fee10a24740 1 -- 172.21.15.47:0/2331136691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedf8003970 con 0x55b067e9f810 2024-03-21T11:28:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.606+0000 7fedf4c00640 1 -- 172.21.15.47:0/2331136691 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedf8059050 con 0x55b067e9f810 2024-03-21T11:28:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.606+0000 7fee10a24740 10 client.27353 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.606+0000 7fee10a24740 1 -- 172.21.15.47:0/2331136691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b067be27d0 con 0x55b067e9f810 2024-03-21T11:28:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.606+0000 7fedf4c00640 1 -- 172.21.15.47:0/2331136691 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedf8003970 con 0x55b067e9f810 2024-03-21T11:28:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.607+0000 7fedf4c00640 1 -- 172.21.15.47:0/2331136691 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fedf80044f0 con 0x55b067e9f810 2024-03-21T11:28:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.607+0000 7fee10a24740 10 client.27353 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.607+0000 7fee10a24740 10 client.27353 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:28:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.607+0000 7fee10a24740 20 client.27353 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.607+0000 7fee10a24740 1 --2- 172.21.15.47:0/2331136691 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b067eb5100 0x55b067eb74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.607+0000 7fee10a24740 4 client.27353 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.607+0000 7fee10a24740 1 -- 172.21.15.47:0/2331136691 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b067e11800 con 0x55b067eb5100 2024-03-21T11:28:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.607+0000 7fedfe000640 1 --2- 172.21.15.47:0/2331136691 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b067eb5100 0x55b067eb74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.607+0000 7fedfe000640 1 --2- 172.21.15.47:0/2331136691 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b067eb5100 0x55b067eb74f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac0909f8daba0aa2 server_cookie=a4221012601490c0 in_seq=0 out_seq=0 2024-03-21T11:28:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.607+0000 7fedf4c00640 10 client.27353 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:28:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.608+0000 7fedf4c00640 1 -- 172.21.15.47:0/2331136691 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b067e11800 con 0x55b067eb5100 2024-03-21T11:28:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.608+0000 7fedf4c00640 10 client.27353 handle_command_reply: tid=0 2024-03-21T11:28:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.608+0000 7fee10a24740 1 client.27353 shutdown 2024-03-21T11:28:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.608+0000 7fee10a24740 1 -- 172.21.15.47:0/2331136691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067e9f810 msgr2=0x55b067ea9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.608+0000 7fee10a24740 1 --2- 172.21.15.47:0/2331136691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067e9f810 0x55b067ea9a90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009ca0 tx=0x7fedf80099c0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.608+0000 7fee10a24740 1 -- 172.21.15.47:0/2331136691 shutdown_connections 2024-03-21T11:28:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.608+0000 7fee10a24740 1 --2- 172.21.15.47:0/2331136691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067ea13a0 0x55b067eb0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.608+0000 7fee10a24740 1 --2- 172.21.15.47:0/2331136691 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b067eb5100 0x55b067eb74f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.609+0000 7fee10a24740 1 --2- 172.21.15.47:0/2331136691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea09e0 0x55b067ea9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.609+0000 7fee10a24740 1 --2- 172.21.15.47:0/2331136691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067e9f810 0x55b067ea9a90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.609+0000 7fedf4c00640 0 client.27353 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:28:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.609+0000 7fee10a24740 1 -- 172.21.15.47:0/2331136691 >> 172.21.15.47:0/2331136691 conn(0x55b067e9ac00 msgr2=0x55b067e99eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.609+0000 7fee10a24740 1 -- 172.21.15.47:0/2331136691 shutdown_connections 2024-03-21T11:28:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.609+0000 7fee10a24740 1 -- 172.21.15.47:0/2331136691 wait complete. 2024-03-21T11:28:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.609+0000 7fee10a24740 20 client.27353 trim_cache size 0 max 16384 2024-03-21T11:28:56.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.624+0000 7fee10a24740 1 Processor -- start 2024-03-21T11:28:56.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.624+0000 7fee10a24740 1 -- start start 2024-03-21T11:28:56.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee10a24740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067e9f810 0x55b067e99eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee10a24740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067ea09e0 0x55b067eb0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee10a24740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea13a0 0x55b067ebbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee10a24740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee081b07b0 con 0x55b067e9f810 2024-03-21T11:28:56.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee10a24740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee081b2130 con 0x55b067ea09e0 2024-03-21T11:28:56.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee10a24740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee081b0e10 con 0x55b067ea13a0 2024-03-21T11:28:56.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fedfe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067ea09e0 0x55b067eb0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee07400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea13a0 0x55b067ebbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee07400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea13a0 0x55b067ebbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46118/0 (socket says 172.21.15.47:46118) 2024-03-21T11:28:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fedfe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067ea09e0 0x55b067eb0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58430/0 (socket says 172.21.15.47:58430) 2024-03-21T11:28:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee07400640 1 -- 172.21.15.47:0/846180420 learned_addr learned my addr 172.21.15.47:0/846180420 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee06a00640 1 --2- 172.21.15.47:0/846180420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067e9f810 0x55b067e99eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee07400640 1 -- 172.21.15.47:0/846180420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067ea09e0 msgr2=0x55b067eb0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee07400640 1 --2- 172.21.15.47:0/846180420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067ea09e0 0x55b067eb0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee07400640 1 -- 172.21.15.47:0/846180420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067e9f810 msgr2=0x55b067e99eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee07400640 1 --2- 172.21.15.47:0/846180420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067e9f810 0x55b067e99eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fee07400640 1 -- 172.21.15.47:0/846180420 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b067e9a3f0 con 0x55b067ea13a0 2024-03-21T11:28:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.625+0000 7fedfe000640 1 --2- 172.21.15.47:0/846180420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067ea09e0 0x55b067eb0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.626+0000 7fee07400640 1 --2- 172.21.15.47:0/846180420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea13a0 0x55b067ebbba0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fee00009fc0 tx=0x7fee00017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2743e8d319c7fed server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.626+0000 7fedf7400640 1 -- 172.21.15.47:0/846180420 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0001da90 con 0x55b067ea13a0 2024-03-21T11:28:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.626+0000 7fedf7400640 1 -- 172.21.15.47:0/846180420 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee0002e070 con 0x55b067ea13a0 2024-03-21T11:28:56.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.626+0000 7fedf7400640 1 -- 172.21.15.47:0/846180420 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0000f620 con 0x55b067ea13a0 2024-03-21T11:28:56.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.627+0000 7fee10a24740 1 -- 172.21.15.47:0/846180420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea13a0 msgr2=0x55b067ebbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.627+0000 7fee10a24740 1 --2- 172.21.15.47:0/846180420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea13a0 0x55b067ebbba0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fee00009fc0 tx=0x7fee00017000 comp rx=0 tx=0).stop 2024-03-21T11:28:56.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.627+0000 7fee10a24740 1 -- 172.21.15.47:0/846180420 shutdown_connections 2024-03-21T11:28:56.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.627+0000 7fee10a24740 1 --2- 172.21.15.47:0/846180420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea13a0 0x55b067ebbba0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.627+0000 7fee10a24740 1 --2- 172.21.15.47:0/846180420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067ea09e0 0x55b067eb0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.627+0000 7fee10a24740 1 --2- 172.21.15.47:0/846180420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067e9f810 0x55b067e99eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.627+0000 7fee10a24740 1 -- 172.21.15.47:0/846180420 >> 172.21.15.47:0/846180420 conn(0x55b067e9ac00 msgr2=0x55b067e9afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.627+0000 7fee10a24740 1 -- 172.21.15.47:0/846180420 shutdown_connections 2024-03-21T11:28:56.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.627+0000 7fee10a24740 1 -- 172.21.15.47:0/846180420 wait complete. 2024-03-21T11:28:56.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.628+0000 7fee10a24740 1 -- start start 2024-03-21T11:28:56.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.628+0000 7fee10a24740 1 Processor -- start 2024-03-21T11:28:56.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.628+0000 7fee10a24740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067e9f810 0x55b067eace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.628+0000 7fee10a24740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067ea09e0 0x55b067ead390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.628+0000 7fee10a24740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea13a0 0x55b067eb03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.628+0000 7fee10a24740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee081b1ad0 con 0x55b067ea09e0 2024-03-21T11:28:56.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.628+0000 7fee10a24740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee081b1470 con 0x55b067e9f810 2024-03-21T11:28:56.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.628+0000 7fee10a24740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee081ae970 con 0x55b067ea13a0 2024-03-21T11:28:56.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.628+0000 7fee10a24740 10 client.0 _ll_register_callbacks cb 0x7fedf00d3540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:56.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.629+0000 7fee06a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067e9f810 0x55b067eace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.629+0000 7fee06a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067e9f810 0x55b067eace50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58444/0 (socket says 172.21.15.47:58444) 2024-03-21T11:28:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.629+0000 7fee06a00640 1 -- 172.21.15.47:0/1240322119 learned_addr learned my addr 172.21.15.47:0/1240322119 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.629+0000 7fedfe000640 1 --2- 172.21.15.47:0/1240322119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067ea09e0 0x55b067ead390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.629+0000 7fee07400640 1 --2- 172.21.15.47:0/1240322119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea13a0 0x55b067eb03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.629+0000 7fee06a00640 1 -- 172.21.15.47:0/1240322119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea13a0 msgr2=0x55b067eb03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.629+0000 7fee06a00640 1 --2- 172.21.15.47:0/1240322119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea13a0 0x55b067eb03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.629+0000 7fee06a00640 1 -- 172.21.15.47:0/1240322119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067ea09e0 msgr2=0x55b067ead390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.629+0000 7fee06a00640 1 --2- 172.21.15.47:0/1240322119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067ea09e0 0x55b067ead390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.629+0000 7fee06a00640 1 -- 172.21.15.47:0/1240322119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b067eb09a0 con 0x55b067e9f810 2024-03-21T11:28:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.629+0000 7fee06a00640 1 --2- 172.21.15.47:0/1240322119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067e9f810 0x55b067eace50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fedf80048c0 tx=0x7fedf8005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94abf3ccf778c44d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.629+0000 7fedf4c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.629+0000 7fedf4c00640 1 -- 172.21.15.47:0/1240322119 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8055070 con 0x55b067e9f810 2024-03-21T11:28:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.630+0000 7fee10a24740 1 -- 172.21.15.47:0/1240322119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee0002e070 con 0x55b067e9f810 2024-03-21T11:28:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.630+0000 7fedf4c00640 1 -- 172.21.15.47:0/1240322119 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fedf80347f0 con 0x55b067e9f810 2024-03-21T11:28:56.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.630+0000 7fedf4c00640 1 -- 172.21.15.47:0/1240322119 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedf8065050 con 0x55b067e9f810 2024-03-21T11:28:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.630+0000 7fee10a24740 10 client.27359 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.630+0000 7fee10a24740 1 -- 172.21.15.47:0/1240322119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b067f10f70 con 0x55b067e9f810 2024-03-21T11:28:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.630+0000 7fedf4c00640 1 -- 172.21.15.47:0/1240322119 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee0002e070 con 0x55b067e9f810 2024-03-21T11:28:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.630+0000 7fedf4c00640 1 -- 172.21.15.47:0/1240322119 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fedf8004090 con 0x55b067e9f810 2024-03-21T11:28:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.630+0000 7fee10a24740 10 client.27359 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.630+0000 7fee10a24740 10 client.27359 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:28:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.630+0000 7fee10a24740 20 client.27359 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.630+0000 7fee10a24740 1 --2- 172.21.15.47:0/1240322119 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b067f11740 0x55b067f13be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.630+0000 7fee10a24740 4 client.27359 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:28:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.630+0000 7fee10a24740 1 -- 172.21.15.47:0/1240322119 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b067e0bb50 con 0x55b067f11740 2024-03-21T11:28:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.631+0000 7fedfe000640 1 --2- 172.21.15.47:0/1240322119 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b067f11740 0x55b067f13be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.631+0000 7fedfe000640 1 --2- 172.21.15.47:0/1240322119 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b067f11740 0x55b067f13be0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d36fb9714a5ad1a server_cookie=bf6d7d08dca49126 in_seq=0 out_seq=0 2024-03-21T11:28:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.631+0000 7fedf4c00640 10 client.27359 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:28:56.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.631+0000 7fedf4c00640 1 -- 172.21.15.47:0/1240322119 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b067e0bb50 con 0x55b067f11740 2024-03-21T11:28:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.631+0000 7fedf4c00640 10 client.27359 handle_command_reply: tid=0 2024-03-21T11:28:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.632+0000 7fee10a24740 1 client.27359 shutdown 2024-03-21T11:28:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.632+0000 7fee10a24740 1 -- 172.21.15.47:0/1240322119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067e9f810 msgr2=0x55b067eace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.632+0000 7fee10a24740 1 --2- 172.21.15.47:0/1240322119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067e9f810 0x55b067eace50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fedf80048c0 tx=0x7fedf8005ef0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.632+0000 7fee10a24740 1 -- 172.21.15.47:0/1240322119 shutdown_connections 2024-03-21T11:28:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.632+0000 7fee10a24740 1 --2- 172.21.15.47:0/1240322119 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b067f11740 0x55b067f13be0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.632+0000 7fee10a24740 1 --2- 172.21.15.47:0/1240322119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b067ea13a0 0x55b067eb03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.632+0000 7fedf4c00640 0 client.27359 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:28:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.632+0000 7fee10a24740 1 --2- 172.21.15.47:0/1240322119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b067ea09e0 0x55b067ead390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.632+0000 7fee10a24740 1 --2- 172.21.15.47:0/1240322119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b067e9f810 0x55b067eace50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.632+0000 7fee10a24740 1 -- 172.21.15.47:0/1240322119 >> 172.21.15.47:0/1240322119 conn(0x55b067e9ac00 msgr2=0x55b067ebbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.632+0000 7fee10a24740 1 -- 172.21.15.47:0/1240322119 shutdown_connections 2024-03-21T11:28:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.632+0000 7fee10a24740 1 -- 172.21.15.47:0/1240322119 wait complete. 2024-03-21T11:28:56.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.632+0000 7fee10a24740 20 client.27359 trim_cache size 0 max 16384 2024-03-21T11:28:56.638 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:28:56.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.633+0000 7fee0de00640 1 -- 172.21.15.47:0/1708207381 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fedec0a2f20 msgr2=0x7fedec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.633+0000 7fee0de00640 1 --2- 172.21.15.47:0/1708207381 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fedec0a2f20 0x7fedec0a53e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55b067ea08e0 tx=0x7fee00005f90 comp rx=0 tx=0).stop 2024-03-21T11:28:56.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.633+0000 7fee0de00640 1 -- 172.21.15.47:0/1708207381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee08108eb0 msgr2=0x7fee0819e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.633+0000 7fee0de00640 1 --2- 172.21.15.47:0/1708207381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee08108eb0 0x7fee0819e550 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fedf000e9f0 tx=0x7fedf000edf0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.633+0000 7fee0de00640 1 -- 172.21.15.47:0/1708207381 shutdown_connections 2024-03-21T11:28:56.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.633+0000 7fee0de00640 1 --2- 172.21.15.47:0/1708207381 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fedec0a2f20 0x7fedec0a53e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.633+0000 7fee0de00640 1 --2- 172.21.15.47:0/1708207381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee08109870 0x7fee081a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.633+0000 7fee0de00640 1 --2- 172.21.15.47:0/1708207381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee08108eb0 0x7fee0819e550 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.633+0000 7fee0de00640 1 --2- 172.21.15.47:0/1708207381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee08107cc0 0x7fee0819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.633+0000 7fee0de00640 1 -- 172.21.15.47:0/1708207381 >> 172.21.15.47:0/1708207381 conn(0x7fee08076c60 msgr2=0x7fee0810c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.633+0000 7fee0de00640 1 -- 172.21.15.47:0/1708207381 shutdown_connections 2024-03-21T11:28:56.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.633+0000 7fee0de00640 1 -- 172.21.15.47:0/1708207381 wait complete. 2024-03-21T11:28:56.649 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:56 smithi047 ceph-mon[31351]: pgmap v568: 129 pgs: 129 active+clean; 40 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.9 MiB/s wr, 280 op/s 2024-03-21T11:28:56.649 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:56 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2659151637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:28:56.652 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:28:56.652 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:28:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.752+0000 7fcf9bc00640 1 -- 172.21.15.47:0/1170035984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf940756e0 msgr2=0x7fcf94073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.752+0000 7fcf9bc00640 1 --2- 172.21.15.47:0/1170035984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf940756e0 0x7fcf94073b30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009930 tx=0x7fcf8802bc30 comp rx=0 tx=0).stop 2024-03-21T11:28:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.752+0000 7fcf9bc00640 1 -- 172.21.15.47:0/1170035984 shutdown_connections 2024-03-21T11:28:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.752+0000 7fcf9bc00640 1 --2- 172.21.15.47:0/1170035984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf9410dca0 0x7fcf94110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.752+0000 7fcf9bc00640 1 --2- 172.21.15.47:0/1170035984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf94074100 0x7fcf94074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.752+0000 7fcf9bc00640 1 --2- 172.21.15.47:0/1170035984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf940756e0 0x7fcf94073b30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.752+0000 7fcf9bc00640 1 -- 172.21.15.47:0/1170035984 >> 172.21.15.47:0/1170035984 conn(0x7fcf940fcc60 msgr2=0x7fcf940ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.752+0000 7fcf9bc00640 1 -- 172.21.15.47:0/1170035984 shutdown_connections 2024-03-21T11:28:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.752+0000 7fcf9bc00640 1 -- 172.21.15.47:0/1170035984 wait complete. 2024-03-21T11:28:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.753+0000 7fcf9bc00640 1 Processor -- start 2024-03-21T11:28:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.753+0000 7fcf9bc00640 1 -- start start 2024-03-21T11:28:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.753+0000 7fcf9bc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf94074100 0x7fcf9419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.753+0000 7fcf9bc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf940756e0 0x7fcf9419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.753+0000 7fcf9bc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf9410dca0 0x7fcf941a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.753+0000 7fcf9bc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf94113940 con 0x7fcf9410dca0 2024-03-21T11:28:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.753+0000 7fcf9bc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf941137c0 con 0x7fcf94074100 2024-03-21T11:28:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.753+0000 7fcf9bc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf94113ac0 con 0x7fcf940756e0 2024-03-21T11:28:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.754+0000 7fcf93e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf940756e0 0x7fcf9419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.754+0000 7fcf99200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf94074100 0x7fcf9419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.754+0000 7fcf93e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf940756e0 0x7fcf9419e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46132/0 (socket says 172.21.15.47:46132) 2024-03-21T11:28:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.754+0000 7fcf93e00640 1 -- 172.21.15.47:0/2392403128 learned_addr learned my addr 172.21.15.47:0/2392403128 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.754+0000 7fcf99c00640 1 --2- 172.21.15.47:0/2392403128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf9410dca0 0x7fcf941a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.754+0000 7fcf93e00640 1 -- 172.21.15.47:0/2392403128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf94074100 msgr2=0x7fcf9419df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.754+0000 7fcf93e00640 1 --2- 172.21.15.47:0/2392403128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf94074100 0x7fcf9419df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.754+0000 7fcf93e00640 1 -- 172.21.15.47:0/2392403128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf9410dca0 msgr2=0x7fcf941a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.754+0000 7fcf93e00640 1 --2- 172.21.15.47:0/2392403128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf9410dca0 0x7fcf941a54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.754+0000 7fcf93e00640 1 -- 172.21.15.47:0/2392403128 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf941a5b50 con 0x7fcf940756e0 2024-03-21T11:28:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.754+0000 7fcf93e00640 1 --2- 172.21.15.47:0/2392403128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf940756e0 0x7fcf9419e470 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000d950 tx=0x7fcf8000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.755+0000 7fcf91600640 1 -- 172.21.15.47:0/2392403128 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80017030 con 0x7fcf940756e0 2024-03-21T11:28:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.755+0000 7fcf91600640 1 -- 172.21.15.47:0/2392403128 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf80004020 con 0x7fcf940756e0 2024-03-21T11:28:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.755+0000 7fcf9bc00640 1 -- 172.21.15.47:0/2392403128 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf941a5e40 con 0x7fcf940756e0 2024-03-21T11:28:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.755+0000 7fcf91600640 1 -- 172.21.15.47:0/2392403128 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf800047a0 con 0x7fcf940756e0 2024-03-21T11:28:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.755+0000 7fcf9bc00640 1 -- 172.21.15.47:0/2392403128 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf941a6300 con 0x7fcf940756e0 2024-03-21T11:28:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.760+0000 7fcf91600640 1 -- 172.21.15.47:0/2392403128 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcf800041d0 con 0x7fcf940756e0 2024-03-21T11:28:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.760+0000 7fcf9e983740 1 Processor -- start 2024-03-21T11:28:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.760+0000 7fcf9e983740 1 -- start start 2024-03-21T11:28:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf9e983740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4ce1e0 0x5620bd4ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf9e983740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4cebd0 0x5620bd4d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf9e983740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4d3d40 0x5620bd4d61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf9e983740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd4913b0 con 0x5620bd4ce1e0 2024-03-21T11:28:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf9e983740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd433830 con 0x5620bd4cebd0 2024-03-21T11:28:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf9e983740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd3dc5d0 con 0x5620bd4d3d40 2024-03-21T11:28:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf99200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4cebd0 0x5620bd4d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf99200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4cebd0 0x5620bd4d36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58474/0 (socket says 172.21.15.47:58474) 2024-03-21T11:28:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf99200640 1 -- 172.21.15.47:0/1305170744 learned_addr learned my addr 172.21.15.47:0/1305170744 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf99c00640 1 --2- 172.21.15.47:0/1305170744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4ce1e0 0x5620bd4ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf93e00640 1 --2- 172.21.15.47:0/1305170744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4d3d40 0x5620bd4d61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf99200640 1 -- 172.21.15.47:0/1305170744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4d3d40 msgr2=0x5620bd4d61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf99200640 1 --2- 172.21.15.47:0/1305170744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4d3d40 0x5620bd4d61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf99200640 1 -- 172.21.15.47:0/1305170744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4ce1e0 msgr2=0x5620bd4ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf99200640 1 --2- 172.21.15.47:0/1305170744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4ce1e0 0x5620bd4ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.761+0000 7fcf99200640 1 -- 172.21.15.47:0/1305170744 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620bd3f42c0 con 0x5620bd4cebd0 2024-03-21T11:28:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.762+0000 7fcf99c00640 1 --2- 172.21.15.47:0/1305170744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4ce1e0 0x5620bd4ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.762+0000 7fcf91600640 1 --2- 172.21.15.47:0/2392403128 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcf7c0a2fc0 0x7fcf7c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.762+0000 7fcf99200640 1 --2- 172.21.15.47:0/1305170744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4cebd0 0x5620bd4d36d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009230 tx=0x7fcf880095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d1a7bbc3737c94 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.762+0000 7fcf91600640 1 -- 172.21.15.47:0/2392403128 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcf800cf1e0 con 0x7fcf940756e0 2024-03-21T11:28:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.762+0000 7fcf99c00640 1 --2- 172.21.15.47:0/2392403128 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcf7c0a2fc0 0x7fcf7c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.762+0000 7fcf86000640 1 -- 172.21.15.47:0/1305170744 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8802cb80 con 0x5620bd4cebd0 2024-03-21T11:28:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.762+0000 7fcf86000640 1 -- 172.21.15.47:0/1305170744 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf88005040 con 0x5620bd4cebd0 2024-03-21T11:28:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.762+0000 7fcf86000640 1 -- 172.21.15.47:0/1305170744 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf88004920 con 0x5620bd4cebd0 2024-03-21T11:28:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.763+0000 7fcf99c00640 1 --2- 172.21.15.47:0/2392403128 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcf7c0a2fc0 0x7fcf7c0a5480 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x5620bd4cf430 tx=0x7fcf8c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.763+0000 7fcf9e983740 1 -- 172.21.15.47:0/1305170744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4cebd0 msgr2=0x5620bd4d36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.763+0000 7fcf9e983740 1 --2- 172.21.15.47:0/1305170744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4cebd0 0x5620bd4d36d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009230 tx=0x7fcf880095e0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.763+0000 7fcf9e983740 1 -- 172.21.15.47:0/1305170744 shutdown_connections 2024-03-21T11:28:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.763+0000 7fcf9e983740 1 --2- 172.21.15.47:0/1305170744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4d3d40 0x5620bd4d61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.763+0000 7fcf9e983740 1 --2- 172.21.15.47:0/1305170744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4cebd0 0x5620bd4d36d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.763+0000 7fcf9e983740 1 --2- 172.21.15.47:0/1305170744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4ce1e0 0x5620bd4ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.763+0000 7fcf9e983740 1 -- 172.21.15.47:0/1305170744 >> 172.21.15.47:0/1305170744 conn(0x5620bd4c9750 msgr2=0x5620bd4c9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.763+0000 7fcf9e983740 1 -- 172.21.15.47:0/1305170744 shutdown_connections 2024-03-21T11:28:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.763+0000 7fcf9e983740 1 -- 172.21.15.47:0/1305170744 wait complete. 2024-03-21T11:28:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.764+0000 7fcf9e983740 1 -- start start 2024-03-21T11:28:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.764+0000 7fcf9e983740 1 Processor -- start 2024-03-21T11:28:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.764+0000 7fcf9e983740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4ce1e0 0x5620bd4d83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.764+0000 7fcf9e983740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4cebd0 0x5620bd4d8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.764+0000 7fcf9e983740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4d3d40 0x5620bd4df8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.764+0000 7fcf9e983740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd3efa20 con 0x5620bd4cebd0 2024-03-21T11:28:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.765+0000 7fcf9e983740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd42f8d0 con 0x5620bd4ce1e0 2024-03-21T11:28:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.765+0000 7fcf9e983740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620bd4d6c90 con 0x5620bd4d3d40 2024-03-21T11:28:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.765+0000 7fcf9e983740 10 client.0 _ll_register_callbacks cb 0x5620bd42c1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.765+0000 7fcf99200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4ce1e0 0x5620bd4d83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.765+0000 7fcf99200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4ce1e0 0x5620bd4d83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58486/0 (socket says 172.21.15.47:58486) 2024-03-21T11:28:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.765+0000 7fcf99200640 1 -- 172.21.15.47:0/4086006132 learned_addr learned my addr 172.21.15.47:0/4086006132 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.765+0000 7fcf99c00640 1 --2- 172.21.15.47:0/4086006132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4d3d40 0x5620bd4df8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.765+0000 7fcf93e00640 1 --2- 172.21.15.47:0/4086006132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4cebd0 0x5620bd4d8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.765+0000 7fcf99200640 1 -- 172.21.15.47:0/4086006132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4d3d40 msgr2=0x5620bd4df8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.765+0000 7fcf99200640 1 --2- 172.21.15.47:0/4086006132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4d3d40 0x5620bd4df8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.765+0000 7fcf99200640 1 -- 172.21.15.47:0/4086006132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4cebd0 msgr2=0x5620bd4d8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.765+0000 7fcf99200640 1 --2- 172.21.15.47:0/4086006132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4cebd0 0x5620bd4d8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.765+0000 7fcf99200640 1 -- 172.21.15.47:0/4086006132 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620bd148210 con 0x5620bd4ce1e0 2024-03-21T11:28:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.765+0000 7fcf99c00640 1 --2- 172.21.15.47:0/4086006132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4d3d40 0x5620bd4df8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.766+0000 7fcf99200640 1 --2- 172.21.15.47:0/4086006132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4ce1e0 0x5620bd4d83c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fcf8802c170 tx=0x7fcf880054a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb09d1f3f74a4fc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.766+0000 7fcf7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.766+0000 7fcf7aa00640 1 -- 172.21.15.47:0/4086006132 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf88004ad0 con 0x5620bd4ce1e0 2024-03-21T11:28:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.766+0000 7fcf7aa00640 1 -- 172.21.15.47:0/4086006132 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcf88004c70 con 0x5620bd4ce1e0 2024-03-21T11:28:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.766+0000 7fcf9e983740 1 -- 172.21.15.47:0/4086006132 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf88005040 con 0x5620bd4ce1e0 2024-03-21T11:28:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.766+0000 7fcf7aa00640 1 -- 172.21.15.47:0/4086006132 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf88004e70 con 0x5620bd4ce1e0 2024-03-21T11:28:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.766+0000 7fcf9e983740 10 client.27371 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.766+0000 7fcf9e983740 1 -- 172.21.15.47:0/4086006132 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5620bd214620 con 0x5620bd4ce1e0 2024-03-21T11:28:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.766+0000 7fcf7aa00640 1 -- 172.21.15.47:0/4086006132 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf88005040 con 0x5620bd4ce1e0 2024-03-21T11:28:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.767+0000 7fcf7aa00640 1 -- 172.21.15.47:0/4086006132 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcf8804e4c0 con 0x5620bd4ce1e0 2024-03-21T11:28:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.767+0000 7fcf9e983740 10 client.27371 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.767+0000 7fcf9e983740 10 client.27371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.767+0000 7fcf9e983740 20 client.27371 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.767+0000 7fcf9e983740 1 --2- 172.21.15.47:0/4086006132 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620bd4e3dc0 0x5620bd4e61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.767+0000 7fcf9e983740 4 client.27371 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.767+0000 7fcf9e983740 1 -- 172.21.15.47:0/4086006132 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620bd440d70 con 0x5620bd4e3dc0 2024-03-21T11:28:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.767+0000 7fcf93e00640 1 --2- 172.21.15.47:0/4086006132 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620bd4e3dc0 0x5620bd4e61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.768+0000 7fcf93e00640 1 --2- 172.21.15.47:0/4086006132 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620bd4e3dc0 0x5620bd4e61b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bff968af9ead24fc server_cookie=370f867b5c4ecdfa in_seq=0 out_seq=0 2024-03-21T11:28:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.768+0000 7fcf7aa00640 10 client.27371 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.769+0000 7fcf7aa00640 1 -- 172.21.15.47:0/4086006132 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5620bd440d70 con 0x5620bd4e3dc0 2024-03-21T11:28:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.769+0000 7fcf7aa00640 10 client.27371 handle_command_reply: tid=0 2024-03-21T11:28:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.769+0000 7fcf9e983740 1 client.27371 shutdown 2024-03-21T11:28:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.769+0000 7fcf9e983740 1 -- 172.21.15.47:0/4086006132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4ce1e0 msgr2=0x5620bd4d83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.769+0000 7fcf9e983740 1 --2- 172.21.15.47:0/4086006132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4ce1e0 0x5620bd4d83c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fcf8802c170 tx=0x7fcf880054a0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.769+0000 7fcf9e983740 1 -- 172.21.15.47:0/4086006132 shutdown_connections 2024-03-21T11:28:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.769+0000 7fcf9e983740 1 --2- 172.21.15.47:0/4086006132 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620bd4e3dc0 0x5620bd4e61b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.769+0000 7fcf9e983740 1 --2- 172.21.15.47:0/4086006132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4d3d40 0x5620bd4df8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.769+0000 7fcf9e983740 1 --2- 172.21.15.47:0/4086006132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4cebd0 0x5620bd4d8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.769+0000 7fcf9e983740 1 --2- 172.21.15.47:0/4086006132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4ce1e0 0x5620bd4d83c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.769+0000 7fcf9e983740 1 -- 172.21.15.47:0/4086006132 >> 172.21.15.47:0/4086006132 conn(0x5620bd4c9750 msgr2=0x5620bd4c8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.769+0000 7fcf7aa00640 0 client.27371 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.769+0000 7fcf9e983740 1 -- 172.21.15.47:0/4086006132 shutdown_connections 2024-03-21T11:28:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.769+0000 7fcf9e983740 1 -- 172.21.15.47:0/4086006132 wait complete. 2024-03-21T11:28:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.770+0000 7fcf9e983740 20 client.27371 trim_cache size 0 max 16384 2024-03-21T11:28:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.785+0000 7fcf9e983740 1 Processor -- start 2024-03-21T11:28:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.786+0000 7fcf9e983740 1 -- start start 2024-03-21T11:28:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.786+0000 7fcf9e983740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4ce1e0 0x5620bd4c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.786+0000 7fcf9e983740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4cebd0 0x5620bd4df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.786+0000 7fcf9e983740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4d3d40 0x5620bd4cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.786+0000 7fcf9e983740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf941b0590 con 0x5620bd4ce1e0 2024-03-21T11:28:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.786+0000 7fcf9e983740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf941b1f10 con 0x5620bd4d3d40 2024-03-21T11:28:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.786+0000 7fcf9e983740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf941b0bf0 con 0x5620bd4cebd0 2024-03-21T11:28:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.786+0000 7fcf99c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4d3d40 0x5620bd4cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.786+0000 7fcf99200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4ce1e0 0x5620bd4c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.786+0000 7fcf99c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4d3d40 0x5620bd4cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58500/0 (socket says 172.21.15.47:58500) 2024-03-21T11:28:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.786+0000 7fcf99c00640 1 -- 172.21.15.47:0/1800278877 learned_addr learned my addr 172.21.15.47:0/1800278877 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.786+0000 7fcf93e00640 1 --2- 172.21.15.47:0/1800278877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4cebd0 0x5620bd4df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.787+0000 7fcf99200640 1 -- 172.21.15.47:0/1800278877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4cebd0 msgr2=0x5620bd4df7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.787+0000 7fcf99200640 1 --2- 172.21.15.47:0/1800278877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4cebd0 0x5620bd4df7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.787+0000 7fcf99200640 1 -- 172.21.15.47:0/1800278877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4d3d40 msgr2=0x5620bd4cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.787+0000 7fcf99200640 1 --2- 172.21.15.47:0/1800278877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4d3d40 0x5620bd4cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.787+0000 7fcf99200640 1 -- 172.21.15.47:0/1800278877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620bd4d9180 con 0x5620bd4ce1e0 2024-03-21T11:28:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.787+0000 7fcf93e00640 1 --2- 172.21.15.47:0/1800278877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4cebd0 0x5620bd4df7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.787+0000 7fcf99200640 1 --2- 172.21.15.47:0/1800278877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4ce1e0 0x5620bd4c45d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009230 tx=0x7fcf8802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dec60bdfbdb66d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.787+0000 7fcf85600640 1 -- 172.21.15.47:0/1800278877 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf88005010 con 0x5620bd4ce1e0 2024-03-21T11:28:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.787+0000 7fcf85600640 1 -- 172.21.15.47:0/1800278877 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf8804ea50 con 0x5620bd4ce1e0 2024-03-21T11:28:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.787+0000 7fcf85600640 1 -- 172.21.15.47:0/1800278877 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf88004310 con 0x5620bd4ce1e0 2024-03-21T11:28:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.788+0000 7fcf9e983740 1 -- 172.21.15.47:0/1800278877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4ce1e0 msgr2=0x5620bd4c45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.788+0000 7fcf9e983740 1 --2- 172.21.15.47:0/1800278877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4ce1e0 0x5620bd4c45d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009230 tx=0x7fcf8802bc30 comp rx=0 tx=0).stop 2024-03-21T11:28:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.788+0000 7fcf9e983740 1 -- 172.21.15.47:0/1800278877 shutdown_connections 2024-03-21T11:28:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.788+0000 7fcf9e983740 1 --2- 172.21.15.47:0/1800278877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4d3d40 0x5620bd4cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.788+0000 7fcf9e983740 1 --2- 172.21.15.47:0/1800278877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4cebd0 0x5620bd4df7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.788+0000 7fcf9e983740 1 --2- 172.21.15.47:0/1800278877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4ce1e0 0x5620bd4c45d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.788+0000 7fcf9e983740 1 -- 172.21.15.47:0/1800278877 >> 172.21.15.47:0/1800278877 conn(0x5620bd4c9750 msgr2=0x5620bd4cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.788+0000 7fcf9e983740 1 -- 172.21.15.47:0/1800278877 shutdown_connections 2024-03-21T11:28:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.788+0000 7fcf9e983740 1 -- 172.21.15.47:0/1800278877 wait complete. 2024-03-21T11:28:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.789+0000 7fcf9e983740 1 -- start start 2024-03-21T11:28:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.789+0000 7fcf9e983740 1 Processor -- start 2024-03-21T11:28:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.789+0000 7fcf9e983740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4ce1e0 0x5620bd4dcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.789+0000 7fcf9e983740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4cebd0 0x5620bd4dd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.789+0000 7fcf9e983740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4d3d40 0x5620bd4de350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.789+0000 7fcf9e983740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf941b18b0 con 0x5620bd4cebd0 2024-03-21T11:28:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.789+0000 7fcf9e983740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf941b1250 con 0x5620bd4d3d40 2024-03-21T11:28:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.789+0000 7fcf9e983740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf941ae750 con 0x5620bd4ce1e0 2024-03-21T11:28:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.789+0000 7fcf9e983740 10 client.0 _ll_register_callbacks cb 0x7fcf800d2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.790+0000 7fcf99200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4ce1e0 0x5620bd4dcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.790+0000 7fcf99200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4ce1e0 0x5620bd4dcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46172/0 (socket says 172.21.15.47:46172) 2024-03-21T11:28:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.790+0000 7fcf99200640 1 -- 172.21.15.47:0/2401898151 learned_addr learned my addr 172.21.15.47:0/2401898151 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.790+0000 7fcf99c00640 1 --2- 172.21.15.47:0/2401898151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4d3d40 0x5620bd4de350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.790+0000 7fcf93e00640 1 --2- 172.21.15.47:0/2401898151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4cebd0 0x5620bd4dd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.790+0000 7fcf99200640 1 -- 172.21.15.47:0/2401898151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4d3d40 msgr2=0x5620bd4de350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.790+0000 7fcf99200640 1 --2- 172.21.15.47:0/2401898151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4d3d40 0x5620bd4de350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.790+0000 7fcf99200640 1 -- 172.21.15.47:0/2401898151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4cebd0 msgr2=0x5620bd4dd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.790+0000 7fcf99200640 1 --2- 172.21.15.47:0/2401898151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4cebd0 0x5620bd4dd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.790+0000 7fcf99200640 1 -- 172.21.15.47:0/2401898151 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620bd4de940 con 0x5620bd4ce1e0 2024-03-21T11:28:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.790+0000 7fcf99200640 1 --2- 172.21.15.47:0/2401898151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4ce1e0 0x5620bd4dcee0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcf88005a50 tx=0x7fcf88009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d607ddebe4e93c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.790+0000 7fcf7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.791+0000 7fcf7aa00640 1 -- 172.21.15.47:0/2401898151 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8805c070 con 0x5620bd4ce1e0 2024-03-21T11:28:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.791+0000 7fcf7aa00640 1 -- 172.21.15.47:0/2401898151 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcf88004ad0 con 0x5620bd4ce1e0 2024-03-21T11:28:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.791+0000 7fcf9e983740 1 -- 172.21.15.47:0/2401898151 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf8804ea50 con 0x5620bd4ce1e0 2024-03-21T11:28:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.791+0000 7fcf7aa00640 1 -- 172.21.15.47:0/2401898151 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf88069050 con 0x5620bd4ce1e0 2024-03-21T11:28:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.791+0000 7fcf9e983740 10 client.27400 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.791+0000 7fcf9e983740 1 -- 172.21.15.47:0/2401898151 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5620bd53f4e0 con 0x5620bd4ce1e0 2024-03-21T11:28:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.791+0000 7fcf7aa00640 1 -- 172.21.15.47:0/2401898151 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf8804ea50 con 0x5620bd4ce1e0 2024-03-21T11:28:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.792+0000 7fcf7aa00640 1 -- 172.21.15.47:0/2401898151 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcf88003830 con 0x5620bd4ce1e0 2024-03-21T11:28:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.792+0000 7fcf9e983740 10 client.27400 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.792+0000 7fcf9e983740 10 client.27400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.792+0000 7fcf9e983740 20 client.27400 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.792+0000 7fcf9e983740 1 --2- 172.21.15.47:0/2401898151 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620bd53fd50 0x5620bd542140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.792+0000 7fcf9e983740 4 client.27400 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:28:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.792+0000 7fcf9e983740 1 -- 172.21.15.47:0/2401898151 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5620bd43b0c0 con 0x5620bd53fd50 2024-03-21T11:28:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.792+0000 7fcf93e00640 1 --2- 172.21.15.47:0/2401898151 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620bd53fd50 0x5620bd542140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.793+0000 7fcf93e00640 1 --2- 172.21.15.47:0/2401898151 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620bd53fd50 0x5620bd542140 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c43eb580d10c024 server_cookie=571ecceff1c76d56 in_seq=0 out_seq=0 2024-03-21T11:28:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:56.793+0000 7fcf7aa00640 10 client.27400 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:56.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:56 smithi100 ceph-mon[36245]: pgmap v568: 129 pgs: 129 active+clean; 40 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.9 MiB/s wr, 280 op/s 2024-03-21T11:28:56.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:56 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2659151637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:28:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.906+0000 7f0f25c00640 1 -- 172.21.15.47:0/2603679563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f20111de0 msgr2=0x7f0f201141f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.906+0000 7f0f25c00640 1 --2- 172.21.15.47:0/2603679563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f20111de0 0x7f0f201141f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009ca0 tx=0x7f0f1802c0f0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.906+0000 7f0f25c00640 1 -- 172.21.15.47:0/2603679563 shutdown_connections 2024-03-21T11:28:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.906+0000 7f0f25c00640 1 --2- 172.21.15.47:0/2603679563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f20111de0 0x7f0f201141f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.906+0000 7f0f25c00640 1 --2- 172.21.15.47:0/2603679563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f2010b010 0x7f0f2010b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.906+0000 7f0f25c00640 1 --2- 172.21.15.47:0/2603679563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f201086a0 0x7f0f2010aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.906+0000 7f0f25c00640 1 -- 172.21.15.47:0/2603679563 >> 172.21.15.47:0/2603679563 conn(0x7f0f2006e500 msgr2=0x7f0f20070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.907+0000 7f0f25c00640 1 -- 172.21.15.47:0/2603679563 shutdown_connections 2024-03-21T11:28:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.908+0000 7f0f25c00640 1 -- 172.21.15.47:0/2603679563 wait complete. 2024-03-21T11:28:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.909+0000 7f0f25c00640 1 Processor -- start 2024-03-21T11:28:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.911+0000 7f0f25c00640 1 -- start start 2024-03-21T11:28:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.912+0000 7f0f25c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f201086a0 0x7f0f201a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.912+0000 7f0f25c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2010b010 0x7f0f201a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.912+0000 7f0f25c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f20111de0 0x7f0f201a9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.912+0000 7f0f25c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f20117d50 con 0x7f0f201086a0 2024-03-21T11:28:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.912+0000 7f0f25c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f20117bd0 con 0x7f0f2010b010 2024-03-21T11:28:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.912+0000 7f0f25c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f20117ed0 con 0x7f0f20111de0 2024-03-21T11:28:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.912+0000 7f0f1e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2010b010 0x7f0f201a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.913+0000 7f0f1e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2010b010 0x7f0f201a2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58526/0 (socket says 172.21.15.47:58526) 2024-03-21T11:28:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.913+0000 7f0f1e000640 1 -- 172.21.15.47:0/1249894912 learned_addr learned my addr 172.21.15.47:0/1249894912 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.913+0000 7f0f1f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f20111de0 0x7f0f201a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.913+0000 7f0f1ea00640 1 --2- 172.21.15.47:0/1249894912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f201086a0 0x7f0f201a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.913+0000 7f0f1e000640 1 -- 172.21.15.47:0/1249894912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f20111de0 msgr2=0x7f0f201a9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.913+0000 7f0f1e000640 1 --2- 172.21.15.47:0/1249894912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f20111de0 0x7f0f201a9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.913+0000 7f0f1e000640 1 -- 172.21.15.47:0/1249894912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f201086a0 msgr2=0x7f0f201a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.913+0000 7f0f1e000640 1 --2- 172.21.15.47:0/1249894912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f201086a0 0x7f0f201a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.913+0000 7f0f1e000640 1 -- 172.21.15.47:0/1249894912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f201a9fe0 con 0x7f0f2010b010 2024-03-21T11:28:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.914+0000 7f0f1e000640 1 --2- 172.21.15.47:0/1249894912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2010b010 0x7f0f201a2900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00e9f0 tx=0x7f0f0c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.915+0000 7f0f17400640 1 -- 172.21.15.47:0/1249894912 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c00cd80 con 0x7f0f2010b010 2024-03-21T11:28:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.915+0000 7f0f17400640 1 -- 172.21.15.47:0/1249894912 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f0c010910 con 0x7f0f2010b010 2024-03-21T11:28:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.915+0000 7f0f17400640 1 -- 172.21.15.47:0/1249894912 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c018990 con 0x7f0f2010b010 2024-03-21T11:28:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.915+0000 7f0f25c00640 1 -- 172.21.15.47:0/1249894912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f201aa2d0 con 0x7f0f2010b010 2024-03-21T11:28:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.916+0000 7f0f25c00640 1 -- 172.21.15.47:0/1249894912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f201aa7d0 con 0x7f0f2010b010 2024-03-21T11:28:56.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.924+0000 7f0f2897c740 1 Processor -- start 2024-03-21T11:28:56.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.924+0000 7f0f2897c740 1 -- start start 2024-03-21T11:28:56.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.925+0000 7f0f2897c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 0x55cb188fac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.925+0000 7f0f2897c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fb9e0 0x55cb188fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.925+0000 7f0f2897c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 0x55cb18902c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.925+0000 7f0f2897c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1885f2c0 con 0x55cb188fb9e0 2024-03-21T11:28:56.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.925+0000 7f0f2897c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb18807c90 con 0x55cb188fa810 2024-03-21T11:28:56.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.925+0000 7f0f2897c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1885b360 con 0x55cb188fc3a0 2024-03-21T11:28:56.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.925+0000 7f0f17400640 1 -- 172.21.15.47:0/1249894912 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0f0c018b30 con 0x7f0f2010b010 2024-03-21T11:28:56.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.926+0000 7f0f1ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fb9e0 0x55cb188fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.926+0000 7f0f1ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fb9e0 0x55cb188fbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47380/0 (socket says 172.21.15.47:47380) 2024-03-21T11:28:56.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.926+0000 7f0f1ea00640 1 -- 172.21.15.47:0/2172659632 learned_addr learned my addr 172.21.15.47:0/2172659632 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.926+0000 7f0f1f400640 1 --2- 172.21.15.47:0/2172659632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 0x55cb188fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.926+0000 7f0f1e000640 1 --2- 172.21.15.47:0/2172659632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 0x55cb18902c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.927+0000 7f0f1ea00640 1 -- 172.21.15.47:0/2172659632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 msgr2=0x55cb18902c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.927+0000 7f0f1ea00640 1 --2- 172.21.15.47:0/2172659632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 0x55cb18902c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.927+0000 7f0f1ea00640 1 -- 172.21.15.47:0/2172659632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 msgr2=0x55cb188fac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.927+0000 7f0f1ea00640 1 --2- 172.21.15.47:0/2172659632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 0x55cb188fac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.927+0000 7f0f1ea00640 1 -- 172.21.15.47:0/2172659632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb1882c080 con 0x55cb188fb9e0 2024-03-21T11:28:56.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.927+0000 7f0f1e000640 1 --2- 172.21.15.47:0/2172659632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 0x55cb18902c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:28:56.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.928+0000 7f0f1ea00640 1 --2- 172.21.15.47:0/2172659632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fb9e0 0x55cb188fbe60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009c50 tx=0x7f0f10010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb9477b3b26b4a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.929+0000 7f0f17400640 1 --2- 172.21.15.47:0/1249894912 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f040a2fa0 0x7f0f040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.930+0000 7f0f17400640 1 -- 172.21.15.47:0/1249894912 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0f0c0cfc30 con 0x7f0f2010b010 2024-03-21T11:28:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.931+0000 7f0f1f400640 1 --2- 172.21.15.47:0/1249894912 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f040a2fa0 0x7f0f040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.932+0000 7f0f0be00640 1 -- 172.21.15.47:0/2172659632 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10004020 con 0x55cb188fb9e0 2024-03-21T11:28:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.932+0000 7f0f0be00640 1 -- 172.21.15.47:0/2172659632 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f100041c0 con 0x55cb188fb9e0 2024-03-21T11:28:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.932+0000 7f0f0be00640 1 -- 172.21.15.47:0/2172659632 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10004020 con 0x55cb188fb9e0 2024-03-21T11:28:56.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.933+0000 7f0f2897c740 1 -- 172.21.15.47:0/2172659632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fb9e0 msgr2=0x55cb188fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.933+0000 7f0f1f400640 1 --2- 172.21.15.47:0/1249894912 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f040a2fa0 0x7f0f040a5460 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0f18037040 tx=0x7f0f18005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.933+0000 7f0f2897c740 1 --2- 172.21.15.47:0/2172659632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fb9e0 0x55cb188fbe60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009c50 tx=0x7f0f10010740 comp rx=0 tx=0).stop 2024-03-21T11:28:56.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.933+0000 7f0f2897c740 1 -- 172.21.15.47:0/2172659632 shutdown_connections 2024-03-21T11:28:56.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.933+0000 7f0f2897c740 1 --2- 172.21.15.47:0/2172659632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 0x55cb18902c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.933+0000 7f0f2897c740 1 --2- 172.21.15.47:0/2172659632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fb9e0 0x55cb188fbe60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.933+0000 7f0f2897c740 1 --2- 172.21.15.47:0/2172659632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 0x55cb188fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.933+0000 7f0f2897c740 1 -- 172.21.15.47:0/2172659632 >> 172.21.15.47:0/2172659632 conn(0x55cb188f5c00 msgr2=0x55cb188f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.934+0000 7f0f2897c740 1 -- 172.21.15.47:0/2172659632 shutdown_connections 2024-03-21T11:28:56.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.934+0000 7f0f2897c740 1 -- 172.21.15.47:0/2172659632 wait complete. 2024-03-21T11:28:56.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.935+0000 7f0f2897c740 1 -- start start 2024-03-21T11:28:56.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.935+0000 7f0f2897c740 1 Processor -- start 2024-03-21T11:28:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.937+0000 7f0f2897c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 0x55cb18904a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.937+0000 7f0f2897c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fb9e0 0x55cb18904f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.937+0000 7f0f2897c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 0x55cb1890bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.937+0000 7f0f2897c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb189033b0 con 0x55cb188fb9e0 2024-03-21T11:28:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.937+0000 7f0f2897c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1881b0d0 con 0x55cb188fa810 2024-03-21T11:28:56.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.937+0000 7f0f2897c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb18903530 con 0x55cb188fc3a0 2024-03-21T11:28:56.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.938+0000 7f0f1f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 0x55cb1890bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.937+0000 7f0f2897c740 10 client.0 _ll_register_callbacks cb 0x55cb1881fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.939+0000 7f0f1e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fb9e0 0x55cb18904f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.939+0000 7f0f1e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fb9e0 0x55cb18904f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47384/0 (socket says 172.21.15.47:47384) 2024-03-21T11:28:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.939+0000 7f0f1e000640 1 -- 172.21.15.47:0/1602649186 learned_addr learned my addr 172.21.15.47:0/1602649186 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.938+0000 7f0f1f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 0x55cb1890bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46202/0 (socket says 172.21.15.47:46202) 2024-03-21T11:28:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.938+0000 7f0f1ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 0x55cb18904a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.939+0000 7f0f1e000640 1 -- 172.21.15.47:0/1602649186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 msgr2=0x55cb1890bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.939+0000 7f0f1e000640 1 --2- 172.21.15.47:0/1602649186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 0x55cb1890bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.940+0000 7f0f1e000640 1 -- 172.21.15.47:0/1602649186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 msgr2=0x55cb18904a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.940+0000 7f0f1e000640 1 --2- 172.21.15.47:0/1602649186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 0x55cb18904a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.940+0000 7f0f1e000640 1 -- 172.21.15.47:0/1602649186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb185733f0 con 0x55cb188fb9e0 2024-03-21T11:28:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.940+0000 7f0f1e000640 1 --2- 172.21.15.47:0/1602649186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fb9e0 0x55cb18904f60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c09cc60 tx=0x7f0f0c09d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2cc74918ecf9cc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.940+0000 7f0f08c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:28:56.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.941+0000 7f0f08c00640 1 -- 172.21.15.47:0/1602649186 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c0bc0b0 con 0x55cb188fb9e0 2024-03-21T11:28:56.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.941+0000 7f0f2897c740 1 -- 172.21.15.47:0/1602649186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f100041c0 con 0x55cb188fb9e0 2024-03-21T11:28:56.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.941+0000 7f0f08c00640 1 -- 172.21.15.47:0/1602649186 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0f0c0995d0 con 0x55cb188fb9e0 2024-03-21T11:28:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.942+0000 7f0f08c00640 1 -- 172.21.15.47:0/1602649186 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f0c09c3b0 con 0x55cb188fb9e0 2024-03-21T11:28:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.942+0000 7f0f2897c740 10 client.17127 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.942+0000 7f0f2897c740 1 -- 172.21.15.47:0/1602649186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cb1863d7d0 con 0x55cb188fb9e0 2024-03-21T11:28:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.943+0000 7f0f08c00640 1 -- 172.21.15.47:0/1602649186 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f100041c0 con 0x55cb188fb9e0 2024-03-21T11:28:56.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.944+0000 7f0f08c00640 1 -- 172.21.15.47:0/1602649186 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0f0c0bace0 con 0x55cb188fb9e0 2024-03-21T11:28:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.945+0000 7f0f2897c740 10 client.17127 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.945+0000 7f0f2897c740 10 client.17127 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:28:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.945+0000 7f0f2897c740 20 client.17127 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.945+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1602649186 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55cb18910420 0x55cb18912830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.945+0000 7f0f2897c740 4 client.17127 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.945+0000 7f0f2897c740 1 -- 172.21.15.47:0/1602649186 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb1886c800 con 0x55cb18910420 2024-03-21T11:28:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.946+0000 7f0f1ea00640 1 --2- 172.21.15.47:0/1602649186 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55cb18910420 0x55cb18912830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.948+0000 7f0f08c00640 10 client.17127 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:28:56.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.948+0000 7f0f1ea00640 1 --2- 172.21.15.47:0/1602649186 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55cb18910420 0x55cb18912830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe89322a48581703 server_cookie=84d9c73bd7e95d83 in_seq=0 out_seq=0 2024-03-21T11:28:56.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.950+0000 7f0f08c00640 1 -- 172.21.15.47:0/1602649186 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cb1886c800 con 0x55cb18910420 2024-03-21T11:28:56.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.950+0000 7f0f08c00640 10 client.17127 handle_command_reply: tid=0 2024-03-21T11:28:56.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.951+0000 7f0f2897c740 1 client.17127 shutdown 2024-03-21T11:28:56.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.953+0000 7f0f2897c740 1 -- 172.21.15.47:0/1602649186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fb9e0 msgr2=0x55cb18904f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.953+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1602649186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fb9e0 0x55cb18904f60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c09cc60 tx=0x7f0f0c09d660 comp rx=0 tx=0).stop 2024-03-21T11:28:56.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.953+0000 7f0f2897c740 1 -- 172.21.15.47:0/1602649186 shutdown_connections 2024-03-21T11:28:56.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.953+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1602649186 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55cb18910420 0x55cb18912830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.953+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1602649186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 0x55cb1890bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.953+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1602649186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fb9e0 0x55cb18904f60 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c09cc60 tx=0x7f0f0c09d660 comp rx=0 tx=0).stop 2024-03-21T11:28:56.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.953+0000 7f0f08c00640 0 client.17127 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:28:56.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.953+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1602649186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 0x55cb18904a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.953+0000 7f0f2897c740 1 -- 172.21.15.47:0/1602649186 >> 172.21.15.47:0/1602649186 conn(0x55cb188f5c00 msgr2=0x55cb188f4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.953+0000 7f0f2897c740 1 -- 172.21.15.47:0/1602649186 shutdown_connections 2024-03-21T11:28:56.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.953+0000 7f0f2897c740 1 -- 172.21.15.47:0/1602649186 wait complete. 2024-03-21T11:28:56.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.955+0000 7f0f2897c740 20 client.17127 trim_cache size 0 max 16384 2024-03-21T11:28:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.985+0000 7f0f2897c740 1 Processor -- start 2024-03-21T11:28:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.986+0000 7f0f2897c740 1 -- start start 2024-03-21T11:28:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.986+0000 7f0f2897c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 0x55cb18904040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.986+0000 7f0f2897c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 0x55cb1890bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.986+0000 7f0f2897c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fe9e0 0x55cb188fee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.986+0000 7f0f2897c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201b8fc0 con 0x55cb188fe9e0 2024-03-21T11:28:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.986+0000 7f0f2897c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201ba940 con 0x55cb188fa810 2024-03-21T11:28:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.986+0000 7f0f2897c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201b9620 con 0x55cb188fc3a0 2024-03-21T11:28:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.986+0000 7f0f1ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 0x55cb18904040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.986+0000 7f0f1ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 0x55cb18904040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58540/0 (socket says 172.21.15.47:58540) 2024-03-21T11:28:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.986+0000 7f0f1ea00640 1 -- 172.21.15.47:0/1616389123 learned_addr learned my addr 172.21.15.47:0/1616389123 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.986+0000 7f0f1e000640 1 --2- 172.21.15.47:0/1616389123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 0x55cb1890bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.986+0000 7f0f1f400640 1 --2- 172.21.15.47:0/1616389123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fe9e0 0x55cb188fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.987+0000 7f0f1ea00640 1 -- 172.21.15.47:0/1616389123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 msgr2=0x55cb1890bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.987+0000 7f0f1ea00640 1 --2- 172.21.15.47:0/1616389123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 0x55cb1890bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.987+0000 7f0f1ea00640 1 -- 172.21.15.47:0/1616389123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fe9e0 msgr2=0x55cb188fee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.988+0000 7f0f1ea00640 1 --2- 172.21.15.47:0/1616389123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fe9e0 0x55cb188fee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.988+0000 7f0f1ea00640 1 -- 172.21.15.47:0/1616389123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb188fb460 con 0x55cb188fa810 2024-03-21T11:28:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.988+0000 7f0f1f400640 1 --2- 172.21.15.47:0/1616389123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fe9e0 0x55cb188fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.989+0000 7f0f1ea00640 1 --2- 172.21.15.47:0/1616389123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 0x55cb18904040 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55cb188f6b70 tx=0x7f0f10005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=109f1eabbab0f2c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.989+0000 7f0f0b400640 1 -- 172.21.15.47:0/1616389123 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10028120 con 0x55cb188fa810 2024-03-21T11:28:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.989+0000 7f0f0b400640 1 -- 172.21.15.47:0/1616389123 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f100282c0 con 0x55cb188fa810 2024-03-21T11:28:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.989+0000 7f0f0b400640 1 -- 172.21.15.47:0/1616389123 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10028780 con 0x55cb188fa810 2024-03-21T11:28:56.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:56 smithi102 ceph-mon[36748]: pgmap v568: 129 pgs: 129 active+clean; 40 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.9 MiB/s wr, 280 op/s 2024-03-21T11:28:56.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:56 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2659151637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:28:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.991+0000 7f0f2897c740 1 -- 172.21.15.47:0/1616389123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 msgr2=0x55cb18904040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.991+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1616389123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 0x55cb18904040 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55cb188f6b70 tx=0x7f0f10005ab0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.992+0000 7f0f2897c740 1 -- 172.21.15.47:0/1616389123 shutdown_connections 2024-03-21T11:28:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.992+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1616389123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fe9e0 0x55cb188fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.992+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1616389123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb188fc3a0 0x55cb1890bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.992+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1616389123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fa810 0x55cb18904040 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.992+0000 7f0f2897c740 1 -- 172.21.15.47:0/1616389123 >> 172.21.15.47:0/1616389123 conn(0x55cb188f5c00 msgr2=0x55cb188f96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.992+0000 7f0f2897c740 1 -- 172.21.15.47:0/1616389123 shutdown_connections 2024-03-21T11:28:56.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.992+0000 7f0f2897c740 1 -- 172.21.15.47:0/1616389123 wait complete. 2024-03-21T11:28:56.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.993+0000 7f0f2897c740 1 -- start start 2024-03-21T11:28:56.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.993+0000 7f0f2897c740 1 Processor -- start 2024-03-21T11:28:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.994+0000 7f0f2897c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fc3a0 0x55cb18908f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.994+0000 7f0f2897c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fe9e0 0x55cb18909480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.994+0000 7f0f2897c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb18969cb0 0x55cb1896c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.994+0000 7f0f2897c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201ba2e0 con 0x55cb188fe9e0 2024-03-21T11:28:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.994+0000 7f0f2897c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201b9c80 con 0x55cb188fc3a0 2024-03-21T11:28:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.994+0000 7f0f2897c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201b7180 con 0x55cb18969cb0 2024-03-21T11:28:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.994+0000 7f0f1f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb18969cb0 0x55cb1896c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.994+0000 7f0f1f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb18969cb0 0x55cb1896c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46214/0 (socket says 172.21.15.47:46214) 2024-03-21T11:28:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.994+0000 7f0f1f400640 1 -- 172.21.15.47:0/1643477082 learned_addr learned my addr 172.21.15.47:0/1643477082 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.995+0000 7f0f2897c740 10 client.0 _ll_register_callbacks cb 0x7f0f0c099200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.995+0000 7f0f1f400640 1 -- 172.21.15.47:0/1643477082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fc3a0 msgr2=0x55cb18908f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.995+0000 7f0f1ea00640 1 --2- 172.21.15.47:0/1643477082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fc3a0 0x55cb18908f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.995+0000 7f0f1f400640 1 --2- 172.21.15.47:0/1643477082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fc3a0 0x55cb18908f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.995+0000 7f0f1f400640 1 -- 172.21.15.47:0/1643477082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fe9e0 msgr2=0x55cb18909480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.995+0000 7f0f1e000640 1 --2- 172.21.15.47:0/1643477082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fe9e0 0x55cb18909480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.995+0000 7f0f1f400640 1 --2- 172.21.15.47:0/1643477082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fe9e0 0x55cb18909480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.995+0000 7f0f1f400640 1 -- 172.21.15.47:0/1643477082 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb1897d5f0 con 0x55cb18969cb0 2024-03-21T11:28:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.996+0000 7f0f1ea00640 1 --2- 172.21.15.47:0/1643477082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fc3a0 0x55cb18908f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:28:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.996+0000 7f0f1f400640 1 --2- 172.21.15.47:0/1643477082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb18969cb0 0x55cb1896c0a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009a60 tx=0x7f0f180095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24ba6c4f8350a674 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.996+0000 7f0f08c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.997+0000 7f0f08c00640 1 -- 172.21.15.47:0/1643477082 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f18051070 con 0x55cb18969cb0 2024-03-21T11:28:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.997+0000 7f0f08c00640 1 -- 172.21.15.47:0/1643477082 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0f18040660 con 0x55cb18969cb0 2024-03-21T11:28:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.997+0000 7f0f2897c740 1 -- 172.21.15.47:0/1643477082 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f100282c0 con 0x55cb18969cb0 2024-03-21T11:28:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.998+0000 7f0f08c00640 1 -- 172.21.15.47:0/1643477082 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f1805d050 con 0x55cb18969cb0 2024-03-21T11:28:57.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.998+0000 7f0f2897c740 10 client.27415 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:57.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.998+0000 7f0f2897c740 1 -- 172.21.15.47:0/1643477082 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cb1897e940 con 0x55cb18969cb0 2024-03-21T11:28:57.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.998+0000 7f0f08c00640 1 -- 172.21.15.47:0/1643477082 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f100282c0 con 0x55cb18969cb0 2024-03-21T11:28:57.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.999+0000 7f0f08c00640 1 -- 172.21.15.47:0/1643477082 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0f18034a10 con 0x55cb18969cb0 2024-03-21T11:28:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.999+0000 7f0f2897c740 10 client.27415 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.999+0000 7f0f2897c740 10 client.27415 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:28:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.999+0000 7f0f2897c740 20 client.27415 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.999+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1643477082 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55cb1897f280 0x55cb189816a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.999+0000 7f0f2897c740 4 client.27415 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:28:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.999+0000 7f0f2897c740 1 -- 172.21.15.47:0/1643477082 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb18904810 con 0x55cb1897f280 2024-03-21T11:28:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:56.999+0000 7f0f1ea00640 1 --2- 172.21.15.47:0/1643477082 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55cb1897f280 0x55cb189816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.000+0000 7f0f1ea00640 1 --2- 172.21.15.47:0/1643477082 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55cb1897f280 0x55cb189816a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78ffc0b9598e667b server_cookie=89db82a82d521583 in_seq=0 out_seq=0 2024-03-21T11:28:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.000+0000 7f0f08c00640 10 client.27415 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:28:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.001+0000 7f0f08c00640 1 -- 172.21.15.47:0/1643477082 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb18904810 con 0x55cb1897f280 2024-03-21T11:28:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.001+0000 7f0f08c00640 10 client.27415 handle_command_reply: tid=0 2024-03-21T11:28:57.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.001+0000 7f0f2897c740 1 client.27415 shutdown 2024-03-21T11:28:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.002+0000 7f0f2897c740 1 -- 172.21.15.47:0/1643477082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb18969cb0 msgr2=0x55cb1896c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.002+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1643477082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb18969cb0 0x55cb1896c0a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009a60 tx=0x7f0f180095c0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.002+0000 7f0f2897c740 1 -- 172.21.15.47:0/1643477082 shutdown_connections 2024-03-21T11:28:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.002+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1643477082 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55cb1897f280 0x55cb189816a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.002+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1643477082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb18969cb0 0x55cb1896c0a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.002+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1643477082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb188fe9e0 0x55cb18909480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.002+0000 7f0f2897c740 1 --2- 172.21.15.47:0/1643477082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb188fc3a0 0x55cb18908f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.002+0000 7f0f2897c740 1 -- 172.21.15.47:0/1643477082 >> 172.21.15.47:0/1643477082 conn(0x55cb188f5c00 msgr2=0x55cb188f96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.002+0000 7f0f08c00640 0 client.27415 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:28:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.003+0000 7f0f2897c740 1 -- 172.21.15.47:0/1643477082 shutdown_connections 2024-03-21T11:28:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.003+0000 7f0f2897c740 1 -- 172.21.15.47:0/1643477082 wait complete. 2024-03-21T11:28:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.003+0000 7f0f2897c740 20 client.27415 trim_cache size 0 max 16384 2024-03-21T11:28:57.005 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:28:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.005+0000 7f0f25c00640 1 -- 172.21.15.47:0/1249894912 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f040a2fa0 msgr2=0x7f0f040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.005+0000 7f0f25c00640 1 --2- 172.21.15.47:0/1249894912 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f040a2fa0 0x7f0f040a5460 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0f18037040 tx=0x7f0f18005d80 comp rx=0 tx=0).stop 2024-03-21T11:28:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.005+0000 7f0f25c00640 1 -- 172.21.15.47:0/1249894912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2010b010 msgr2=0x7f0f201a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.005+0000 7f0f25c00640 1 --2- 172.21.15.47:0/1249894912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2010b010 0x7f0f201a2900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00e9f0 tx=0x7f0f0c00eeb0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.005+0000 7f0f25c00640 1 -- 172.21.15.47:0/1249894912 shutdown_connections 2024-03-21T11:28:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.005+0000 7f0f25c00640 1 --2- 172.21.15.47:0/1249894912 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0f040a2fa0 0x7f0f040a5460 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.005+0000 7f0f25c00640 1 --2- 172.21.15.47:0/1249894912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f20111de0 0x7f0f201a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.005+0000 7f0f25c00640 1 --2- 172.21.15.47:0/1249894912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f2010b010 0x7f0f201a2900 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.005+0000 7f0f25c00640 1 --2- 172.21.15.47:0/1249894912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f201086a0 0x7f0f201a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.005+0000 7f0f25c00640 1 -- 172.21.15.47:0/1249894912 >> 172.21.15.47:0/1249894912 conn(0x7f0f2006e500 msgr2=0x7f0f20109c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.006+0000 7f0f25c00640 1 -- 172.21.15.47:0/1249894912 shutdown_connections 2024-03-21T11:28:57.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.006+0000 7f0f25c00640 1 -- 172.21.15.47:0/1249894912 wait complete. 2024-03-21T11:28:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.017+0000 7fcf7aa00640 1 -- 172.21.15.47:0/2401898151 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5620bd43b0c0 con 0x5620bd53fd50 2024-03-21T11:28:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.017+0000 7fcf7aa00640 10 client.27400 handle_command_reply: tid=0 2024-03-21T11:28:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.017+0000 7fcf9e983740 1 client.27400 shutdown 2024-03-21T11:28:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.018+0000 7fcf9e983740 1 -- 172.21.15.47:0/2401898151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4ce1e0 msgr2=0x5620bd4dcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.018+0000 7fcf9e983740 1 --2- 172.21.15.47:0/2401898151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4ce1e0 0x5620bd4dcee0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcf88005a50 tx=0x7fcf88009260 comp rx=0 tx=0).stop 2024-03-21T11:28:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.018+0000 7fcf9e983740 1 -- 172.21.15.47:0/2401898151 shutdown_connections 2024-03-21T11:28:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.018+0000 7fcf9e983740 1 --2- 172.21.15.47:0/2401898151 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620bd53fd50 0x5620bd542140 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.018+0000 7fcf9e983740 1 --2- 172.21.15.47:0/2401898151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620bd4d3d40 0x5620bd4de350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.018+0000 7fcf9e983740 1 --2- 172.21.15.47:0/2401898151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620bd4cebd0 0x5620bd4dd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.018+0000 7fcf9e983740 1 --2- 172.21.15.47:0/2401898151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620bd4ce1e0 0x5620bd4dcee0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.018+0000 7fcf7aa00640 0 client.27400 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.018+0000 7fcf9e983740 1 -- 172.21.15.47:0/2401898151 >> 172.21.15.47:0/2401898151 conn(0x5620bd4c9750 msgr2=0x5620bd4cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.018+0000 7fcf9e983740 1 -- 172.21.15.47:0/2401898151 shutdown_connections 2024-03-21T11:28:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.018+0000 7fcf9e983740 1 -- 172.21.15.47:0/2401898151 wait complete. 2024-03-21T11:28:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.018+0000 7fcf9e983740 20 client.27400 trim_cache size 0 max 16384 2024-03-21T11:28:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:28:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:28:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:28:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:28:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.019+0000 7fcf9bc00640 1 -- 172.21.15.47:0/2392403128 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcf7c0a2fc0 msgr2=0x7fcf7c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.019+0000 7fcf9bc00640 1 --2- 172.21.15.47:0/2392403128 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcf7c0a2fc0 0x7fcf7c0a5480 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x5620bd4cf430 tx=0x7fcf8c009500 comp rx=0 tx=0).stop 2024-03-21T11:28:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.019+0000 7fcf9bc00640 1 -- 172.21.15.47:0/2392403128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf940756e0 msgr2=0x7fcf9419e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.019+0000 7fcf9bc00640 1 --2- 172.21.15.47:0/2392403128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf940756e0 0x7fcf9419e470 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000d950 tx=0x7fcf8000de10 comp rx=0 tx=0).stop 2024-03-21T11:28:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.019+0000 7fcf9bc00640 1 -- 172.21.15.47:0/2392403128 shutdown_connections 2024-03-21T11:28:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.019+0000 7fcf9bc00640 1 --2- 172.21.15.47:0/2392403128 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcf7c0a2fc0 0x7fcf7c0a5480 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.019+0000 7fcf9bc00640 1 --2- 172.21.15.47:0/2392403128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf9410dca0 0x7fcf941a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.019+0000 7fcf9bc00640 1 --2- 172.21.15.47:0/2392403128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf940756e0 0x7fcf9419e470 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.019+0000 7fcf9bc00640 1 --2- 172.21.15.47:0/2392403128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf94074100 0x7fcf9419df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.019+0000 7fcf9bc00640 1 -- 172.21.15.47:0/2392403128 >> 172.21.15.47:0/2392403128 conn(0x7fcf940fcc60 msgr2=0x7fcf94101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.020+0000 7fcf9bc00640 1 -- 172.21.15.47:0/2392403128 shutdown_connections 2024-03-21T11:28:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:28:57.020+0000 7fcf9bc00640 1 -- 172.21.15.47:0/2392403128 wait complete. 2024-03-21T11:28:57.027 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:28:57.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:28:57.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.219+0000 7f3a06200640 1 -- 172.21.15.47:0/304609982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a001005a0 msgr2=0x7f3a001009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.219+0000 7f3a06200640 1 --2- 172.21.15.47:0/304609982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a001005a0 0x7f3a001009c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009900 tx=0x7f39f002bc40 comp rx=0 tx=0).stop 2024-03-21T11:28:57.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.220+0000 7f3a06200640 1 -- 172.21.15.47:0/304609982 shutdown_connections 2024-03-21T11:28:57.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.220+0000 7f3a06200640 1 --2- 172.21.15.47:0/304609982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a00102150 0x7f3a001089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.220+0000 7f3a06200640 1 --2- 172.21.15.47:0/304609982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a00101790 0x7f3a00101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.220+0000 7f3a06200640 1 --2- 172.21.15.47:0/304609982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a001005a0 0x7f3a001009c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.220+0000 7f3a06200640 1 -- 172.21.15.47:0/304609982 >> 172.21.15.47:0/304609982 conn(0x7f3a000fbb30 msgr2=0x7f3a000fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:57.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.220+0000 7f3a06200640 1 -- 172.21.15.47:0/304609982 shutdown_connections 2024-03-21T11:28:57.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.220+0000 7f3a06200640 1 -- 172.21.15.47:0/304609982 wait complete. 2024-03-21T11:28:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.221+0000 7f3a06200640 1 Processor -- start 2024-03-21T11:28:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.221+0000 7f3a06200640 1 -- start start 2024-03-21T11:28:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.221+0000 7f3a06200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a001005a0 0x7f3a00199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.221+0000 7f3a06200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a00101790 0x7f3a0019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.221+0000 7f3a06200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a00102150 0x7f3a001a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.221+0000 7f3a06200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0010c240 con 0x7f3a001005a0 2024-03-21T11:28:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.221+0000 7f3a06200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0010c0c0 con 0x7f3a00101790 2024-03-21T11:28:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.221+0000 7f3a06200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0010c3c0 con 0x7f3a00102150 2024-03-21T11:28:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.222+0000 7f39ffe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a00102150 0x7f3a001a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.222+0000 7f39fea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a00101790 0x7f3a0019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.222+0000 7f39ff400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a001005a0 0x7f3a00199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.222+0000 7f39ffe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a00102150 0x7f3a001a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46234/0 (socket says 172.21.15.47:46234) 2024-03-21T11:28:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.222+0000 7f39ffe00640 1 -- 172.21.15.47:0/175498007 learned_addr learned my addr 172.21.15.47:0/175498007 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.222+0000 7f39ffe00640 1 -- 172.21.15.47:0/175498007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a00101790 msgr2=0x7f3a0019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.222+0000 7f39ffe00640 1 --2- 172.21.15.47:0/175498007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a00101790 0x7f3a0019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.222+0000 7f39ffe00640 1 -- 172.21.15.47:0/175498007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a001005a0 msgr2=0x7f3a00199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.222+0000 7f39ffe00640 1 --2- 172.21.15.47:0/175498007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a001005a0 0x7f3a00199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.222+0000 7f39ffe00640 1 -- 172.21.15.47:0/175498007 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a001a18b0 con 0x7f3a00102150 2024-03-21T11:28:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.222+0000 7f39fea00640 1 --2- 172.21.15.47:0/175498007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a00101790 0x7f3a0019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.222+0000 7f39ffe00640 1 --2- 172.21.15.47:0/175498007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a00102150 0x7f3a001a1210 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009fd0 tx=0x7f39f800b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.223+0000 7f39f7e00640 1 -- 172.21.15.47:0/175498007 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8017070 con 0x7f3a00102150 2024-03-21T11:28:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.223+0000 7f39f7e00640 1 -- 172.21.15.47:0/175498007 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39f8003e50 con 0x7f3a00102150 2024-03-21T11:28:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.223+0000 7f3a06200640 1 -- 172.21.15.47:0/175498007 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a001a1ba0 con 0x7f3a00102150 2024-03-21T11:28:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.223+0000 7f39f7e00640 1 -- 172.21.15.47:0/175498007 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80106c0 con 0x7f3a00102150 2024-03-21T11:28:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.223+0000 7f3a06200640 1 -- 172.21.15.47:0/175498007 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a001a2070 con 0x7f3a00102150 2024-03-21T11:28:57.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.226+0000 7f3a09027740 1 Processor -- start 2024-03-21T11:28:57.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.226+0000 7f39f7e00640 1 -- 172.21.15.47:0/175498007 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f39f8003970 con 0x7f3a00102150 2024-03-21T11:28:57.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.226+0000 7f3a09027740 1 -- start start 2024-03-21T11:28:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.227+0000 7f3a09027740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c99810 0x55b256c99c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.227+0000 7f3a09027740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 0x55b256c9ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.227+0000 7f3a09027740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 0x55b256ca1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.227+0000 7f3a09027740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256bfe2c0 con 0x55b256c9a9e0 2024-03-21T11:28:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.227+0000 7f3a09027740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256ba6c90 con 0x55b256c99810 2024-03-21T11:28:57.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.227+0000 7f3a09027740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256bfa360 con 0x55b256c9b3a0 2024-03-21T11:28:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.227+0000 7f39fea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c99810 0x55b256c99c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.227+0000 7f39fea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c99810 0x55b256c99c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58576/0 (socket says 172.21.15.47:58576) 2024-03-21T11:28:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.227+0000 7f39fea00640 1 -- 172.21.15.47:0/1758185421 learned_addr learned my addr 172.21.15.47:0/1758185421 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.227+0000 7f39ff400640 1 --2- 172.21.15.47:0/1758185421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 0x55b256c9ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.227+0000 7f39fea00640 1 --2- 172.21.15.47:0/1758185421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 0x55b256ca1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.228+0000 7f39ff400640 1 -- 172.21.15.47:0/1758185421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 msgr2=0x55b256ca1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.228+0000 7f39ff400640 1 --2- 172.21.15.47:0/1758185421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 0x55b256ca1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.228+0000 7f39ff400640 1 -- 172.21.15.47:0/1758185421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c99810 msgr2=0x55b256c99c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.228+0000 7f39ff400640 1 --2- 172.21.15.47:0/1758185421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c99810 0x55b256c99c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.228+0000 7f39ff400640 1 -- 172.21.15.47:0/1758185421 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b256bcb080 con 0x55b256c9a9e0 2024-03-21T11:28:57.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.228+0000 7f39fea00640 1 --2- 172.21.15.47:0/1758185421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c99810 0x55b256c99c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.228+0000 7f39f7e00640 1 --2- 172.21.15.47:0/175498007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39e40a2f20 0x7f39e40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.228+0000 7f39ff400640 1 --2- 172.21.15.47:0/1758185421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 0x55b256c9ae60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009230 tx=0x7f39f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f493a8ea4222a04c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.228+0000 7f39f7e00640 1 -- 172.21.15.47:0/175498007 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f39f80cf4f0 con 0x7f3a00102150 2024-03-21T11:28:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.228+0000 7f39fea00640 1 --2- 172.21.15.47:0/175498007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39e40a2f20 0x7f39e40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.228+0000 7f39ebe00640 1 -- 172.21.15.47:0/1758185421 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0042070 con 0x55b256c9a9e0 2024-03-21T11:28:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.229+0000 7f39ebe00640 1 -- 172.21.15.47:0/1758185421 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39f002cb90 con 0x55b256c9a9e0 2024-03-21T11:28:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.229+0000 7f39ebe00640 1 -- 172.21.15.47:0/1758185421 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0004920 con 0x55b256c9a9e0 2024-03-21T11:28:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.229+0000 7f3a09027740 1 -- 172.21.15.47:0/1758185421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 msgr2=0x55b256c9ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.229+0000 7f3a09027740 1 --2- 172.21.15.47:0/1758185421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 0x55b256c9ae60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009230 tx=0x7f39f0000bc0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.229+0000 7f39fea00640 1 --2- 172.21.15.47:0/175498007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39e40a2f20 0x7f39e40a53e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55b256c9a8e0 tx=0x7f39ec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.229+0000 7f3a09027740 1 -- 172.21.15.47:0/1758185421 shutdown_connections 2024-03-21T11:28:57.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.229+0000 7f3a09027740 1 --2- 172.21.15.47:0/1758185421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 0x55b256ca1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.229+0000 7f3a09027740 1 --2- 172.21.15.47:0/1758185421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 0x55b256c9ae60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.229+0000 7f3a09027740 1 --2- 172.21.15.47:0/1758185421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c99810 0x55b256c99c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.229+0000 7f3a09027740 1 -- 172.21.15.47:0/1758185421 >> 172.21.15.47:0/1758185421 conn(0x55b256c94c00 msgr2=0x55b256c95000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.230+0000 7f3a09027740 1 -- 172.21.15.47:0/1758185421 shutdown_connections 2024-03-21T11:28:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.230+0000 7f3a09027740 1 -- 172.21.15.47:0/1758185421 wait complete. 2024-03-21T11:28:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.230+0000 7f3a09027740 1 -- start start 2024-03-21T11:28:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.230+0000 7f3a09027740 1 Processor -- start 2024-03-21T11:28:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f3a09027740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c99810 0x55b256ca3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f3a09027740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 0x55b256ca3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f3a09027740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c9b3a0 0x55b256caafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f3a09027740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256ca2350 con 0x55b256c9a9e0 2024-03-21T11:28:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f3a09027740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256bba0d0 con 0x55b256c9b3a0 2024-03-21T11:28:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f3a09027740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b256ca24d0 con 0x55b256c99810 2024-03-21T11:28:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f3a09027740 10 client.0 _ll_register_callbacks cb 0x55b256bbee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f39ffe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c9b3a0 0x55b256caafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f39ff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c99810 0x55b256ca3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f39ffe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c9b3a0 0x55b256caafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58590/0 (socket says 172.21.15.47:58590) 2024-03-21T11:28:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f39ff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c99810 0x55b256ca3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46246/0 (socket says 172.21.15.47:46246) 2024-03-21T11:28:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f39ffe00640 1 -- 172.21.15.47:0/3463919331 learned_addr learned my addr 172.21.15.47:0/3463919331 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f39fea00640 1 --2- 172.21.15.47:0/3463919331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 0x55b256ca3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f39ff400640 1 -- 172.21.15.47:0/3463919331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c9b3a0 msgr2=0x55b256caafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f39ff400640 1 --2- 172.21.15.47:0/3463919331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c9b3a0 0x55b256caafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f39ff400640 1 -- 172.21.15.47:0/3463919331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 msgr2=0x55b256ca3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f39ff400640 1 --2- 172.21.15.47:0/3463919331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 0x55b256ca3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f39ff400640 1 -- 172.21.15.47:0/3463919331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2569123f0 con 0x55b256c99810 2024-03-21T11:28:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.231+0000 7f39ffe00640 1 --2- 172.21.15.47:0/3463919331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c9b3a0 0x55b256caafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.232+0000 7f39ff400640 1 --2- 172.21.15.47:0/3463919331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c99810 0x55b256ca3a30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f39f002c180 tx=0x7f39f00045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c65c1670763649e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.232+0000 7f39e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.232+0000 7f39e8c00640 1 -- 172.21.15.47:0/3463919331 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f004a070 con 0x55b256c99810 2024-03-21T11:28:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.232+0000 7f39e8c00640 1 -- 172.21.15.47:0/3463919331 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f39f0003800 con 0x55b256c99810 2024-03-21T11:28:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.232+0000 7f3a09027740 1 -- 172.21.15.47:0/3463919331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39f002cb90 con 0x55b256c99810 2024-03-21T11:28:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.232+0000 7f39e8c00640 1 -- 172.21.15.47:0/3463919331 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39f0058050 con 0x55b256c99810 2024-03-21T11:28:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.232+0000 7f3a09027740 10 client.27427 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.232+0000 7f3a09027740 1 -- 172.21.15.47:0/3463919331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b2569dc7d0 con 0x55b256c99810 2024-03-21T11:28:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.233+0000 7f39e8c00640 1 -- 172.21.15.47:0/3463919331 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39f002cb90 con 0x55b256c99810 2024-03-21T11:28:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.233+0000 7f39e8c00640 1 -- 172.21.15.47:0/3463919331 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f39f004f9b0 con 0x55b256c99810 2024-03-21T11:28:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.233+0000 7f3a09027740 10 client.27427 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.233+0000 7f3a09027740 10 client.27427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.233+0000 7f3a09027740 20 client.27427 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.233+0000 7f3a09027740 1 --2- 172.21.15.47:0/3463919331 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b256caf110 0x55b256cb1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.233+0000 7f3a09027740 4 client.27427 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.233+0000 7f3a09027740 1 -- 172.21.15.47:0/3463919331 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b256c0b800 con 0x55b256caf110 2024-03-21T11:28:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.233+0000 7f39fea00640 1 --2- 172.21.15.47:0/3463919331 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b256caf110 0x55b256cb1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.234+0000 7f39fea00640 1 --2- 172.21.15.47:0/3463919331 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b256caf110 0x55b256cb1500 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ed82fc18aef7bf7 server_cookie=b2e144707b261eb2 in_seq=0 out_seq=0 2024-03-21T11:28:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.234+0000 7f39e8c00640 10 client.27427 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.236+0000 7f39e8c00640 1 -- 172.21.15.47:0/3463919331 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b256c0b800 con 0x55b256caf110 2024-03-21T11:28:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.237+0000 7f39e8c00640 10 client.27427 handle_command_reply: tid=0 2024-03-21T11:28:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.237+0000 7f3a09027740 1 client.27427 shutdown 2024-03-21T11:28:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.237+0000 7f3a09027740 1 -- 172.21.15.47:0/3463919331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c99810 msgr2=0x55b256ca3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.237+0000 7f3a09027740 1 --2- 172.21.15.47:0/3463919331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c99810 0x55b256ca3a30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f39f002c180 tx=0x7f39f00045e0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.237+0000 7f3a09027740 1 -- 172.21.15.47:0/3463919331 shutdown_connections 2024-03-21T11:28:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.237+0000 7f3a09027740 1 --2- 172.21.15.47:0/3463919331 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b256caf110 0x55b256cb1500 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.237+0000 7f3a09027740 1 --2- 172.21.15.47:0/3463919331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c9b3a0 0x55b256caafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.237+0000 7f3a09027740 1 --2- 172.21.15.47:0/3463919331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 0x55b256ca3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.237+0000 7f3a09027740 1 --2- 172.21.15.47:0/3463919331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c99810 0x55b256ca3a30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.237+0000 7f39e8c00640 0 client.27427 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.237+0000 7f3a09027740 1 -- 172.21.15.47:0/3463919331 >> 172.21.15.47:0/3463919331 conn(0x55b256c94c00 msgr2=0x55b256c93eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.237+0000 7f3a09027740 1 -- 172.21.15.47:0/3463919331 shutdown_connections 2024-03-21T11:28:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.237+0000 7f3a09027740 1 -- 172.21.15.47:0/3463919331 wait complete. 2024-03-21T11:28:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.237+0000 7f3a09027740 20 client.27427 trim_cache size 0 max 16384 2024-03-21T11:28:57.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.251+0000 7f3a09027740 1 Processor -- start 2024-03-21T11:28:57.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.251+0000 7f3a09027740 1 -- start start 2024-03-21T11:28:57.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f3a09027740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c99810 0x55b256c9da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f3a09027740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 0x55b256c9dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f3a09027740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 0x55b256c9e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f3a09027740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a001ac250 con 0x55b256c9a9e0 2024-03-21T11:28:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f3a09027740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a001adbd0 con 0x55b256c99810 2024-03-21T11:28:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f3a09027740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a001ac8b0 con 0x55b256c9b3a0 2024-03-21T11:28:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f39ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c99810 0x55b256c9da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f39ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c99810 0x55b256c9da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58594/0 (socket says 172.21.15.47:58594) 2024-03-21T11:28:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f39ff400640 1 -- 172.21.15.47:0/1817201535 learned_addr learned my addr 172.21.15.47:0/1817201535 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f39ffe00640 1 --2- 172.21.15.47:0/1817201535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 0x55b256c9e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f39fea00640 1 --2- 172.21.15.47:0/1817201535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 0x55b256c9dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f39ff400640 1 -- 172.21.15.47:0/1817201535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 msgr2=0x55b256c9e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:57.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f39ff400640 1 --2- 172.21.15.47:0/1817201535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 0x55b256c9e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f39ff400640 1 -- 172.21.15.47:0/1817201535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 msgr2=0x55b256c9dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:57.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f39ff400640 1 --2- 172.21.15.47:0/1817201535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 0x55b256c9dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f39ff400640 1 -- 172.21.15.47:0/1817201535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b256c95970 con 0x55b256c99810 2024-03-21T11:28:57.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f39ffe00640 1 --2- 172.21.15.47:0/1817201535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 0x55b256c9e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:57.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.252+0000 7f39fea00640 1 --2- 172.21.15.47:0/1817201535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 0x55b256c9dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:57.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.253+0000 7f39ff400640 1 --2- 172.21.15.47:0/1817201535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c99810 0x55b256c9da80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f39f0004650 tx=0x7f39f002c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c99511dc0d3e1cdb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:57.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.253+0000 7f39eb400640 1 -- 172.21.15.47:0/1817201535 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f004a040 con 0x55b256c99810 2024-03-21T11:28:57.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.253+0000 7f39eb400640 1 -- 172.21.15.47:0/1817201535 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39f0009520 con 0x55b256c99810 2024-03-21T11:28:57.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.253+0000 7f39eb400640 1 -- 172.21.15.47:0/1817201535 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0042040 con 0x55b256c99810 2024-03-21T11:28:57.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.254+0000 7f3a09027740 1 -- 172.21.15.47:0/1817201535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c99810 msgr2=0x55b256c9da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.254+0000 7f3a09027740 1 --2- 172.21.15.47:0/1817201535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c99810 0x55b256c9da80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f39f0004650 tx=0x7f39f002c890 comp rx=0 tx=0).stop 2024-03-21T11:28:57.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.254+0000 7f3a09027740 1 -- 172.21.15.47:0/1817201535 shutdown_connections 2024-03-21T11:28:57.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.254+0000 7f3a09027740 1 --2- 172.21.15.47:0/1817201535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 0x55b256c9e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.254+0000 7f3a09027740 1 --2- 172.21.15.47:0/1817201535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c9a9e0 0x55b256c9dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.254+0000 7f3a09027740 1 --2- 172.21.15.47:0/1817201535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c99810 0x55b256c9da80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.254+0000 7f3a09027740 1 -- 172.21.15.47:0/1817201535 >> 172.21.15.47:0/1817201535 conn(0x55b256c94c00 msgr2=0x55b256c986f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.254+0000 7f3a09027740 1 -- 172.21.15.47:0/1817201535 shutdown_connections 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.254+0000 7f3a09027740 1 -- 172.21.15.47:0/1817201535 wait complete. 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.254+0000 7f3a09027740 1 -- start start 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.254+0000 7f3a09027740 1 Processor -- start 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.255+0000 7f3a09027740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c99810 0x55b256ca63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.255+0000 7f3a09027740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c9a9e0 0x55b256ca68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.255+0000 7f3a09027740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 0x55b256ca9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.255+0000 7f3a09027740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a001ad570 con 0x55b256c99810 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.255+0000 7f3a09027740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a001acf10 con 0x55b256c9a9e0 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.255+0000 7f3a09027740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a001aa410 con 0x55b256c9b3a0 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.255+0000 7f3a09027740 10 client.0 _ll_register_callbacks cb 0x7f39ec009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.255+0000 7f39fea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c9a9e0 0x55b256ca68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.255+0000 7f39ffe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 0x55b256ca9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.255+0000 7f39fea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c9a9e0 0x55b256ca68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58602/0 (socket says 172.21.15.47:58602) 2024-03-21T11:28:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.255+0000 7f39fea00640 1 -- 172.21.15.47:0/3894143805 learned_addr learned my addr 172.21.15.47:0/3894143805 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.255+0000 7f39ff400640 1 --2- 172.21.15.47:0/3894143805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c99810 0x55b256ca63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.256+0000 7f39fea00640 1 -- 172.21.15.47:0/3894143805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 msgr2=0x55b256ca9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.256+0000 7f39fea00640 1 --2- 172.21.15.47:0/3894143805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 0x55b256ca9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.256+0000 7f39fea00640 1 -- 172.21.15.47:0/3894143805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c99810 msgr2=0x55b256ca63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.256+0000 7f39fea00640 1 --2- 172.21.15.47:0/3894143805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c99810 0x55b256ca63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.256+0000 7f39fea00640 1 -- 172.21.15.47:0/3894143805 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b256ca2610 con 0x55b256c9a9e0 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.256+0000 7f39ffe00640 1 --2- 172.21.15.47:0/3894143805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 0x55b256ca9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.256+0000 7f39fea00640 1 --2- 172.21.15.47:0/3894143805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c9a9e0 0x55b256ca68f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f39ec009e90 tx=0x7f39ec005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5fbf28287f0afad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.256+0000 7f39e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.256+0000 7f39e8c00640 1 -- 172.21.15.47:0/3894143805 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec032060 con 0x55b256c9a9e0 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.256+0000 7f3a09027740 1 -- 172.21.15.47:0/3894143805 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39f0009520 con 0x55b256c9a9e0 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.256+0000 7f39e8c00640 1 -- 172.21.15.47:0/3894143805 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f39ec009040 con 0x55b256c9a9e0 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.257+0000 7f39e8c00640 1 -- 172.21.15.47:0/3894143805 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39ec039050 con 0x55b256c9a9e0 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.257+0000 7f3a09027740 10 client.27410 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.257+0000 7f3a09027740 1 -- 172.21.15.47:0/3894143805 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b256c9a200 con 0x55b256c9a9e0 2024-03-21T11:28:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.257+0000 7f39e8c00640 1 -- 172.21.15.47:0/3894143805 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39f0009520 con 0x55b256c9a9e0 2024-03-21T11:28:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.257+0000 7f39e8c00640 1 -- 172.21.15.47:0/3894143805 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f39ec021400 con 0x55b256c9a9e0 2024-03-21T11:28:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.257+0000 7f3a09027740 10 client.27410 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.257+0000 7f3a09027740 10 client.27410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.257+0000 7f3a09027740 20 client.27410 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.257+0000 7f3a09027740 1 --2- 172.21.15.47:0/3894143805 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b256d0b520 0x55b256d0d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.257+0000 7f3a09027740 4 client.27410 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:28:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.257+0000 7f3a09027740 1 -- 172.21.15.47:0/3894143805 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b256c05b50 con 0x55b256d0b520 2024-03-21T11:28:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.257+0000 7f39ff400640 1 --2- 172.21.15.47:0/3894143805 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b256d0b520 0x55b256d0d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:57.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.258+0000 7f39ff400640 1 --2- 172.21.15.47:0/3894143805 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b256d0b520 0x55b256d0d940 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2118f93a58c15e03 server_cookie=21626fb5859ed082 in_seq=0 out_seq=0 2024-03-21T11:28:57.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.258+0000 7f39e8c00640 10 client.27410 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:57.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.258+0000 7f39e8c00640 1 -- 172.21.15.47:0/3894143805 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b256c05b50 con 0x55b256d0b520 2024-03-21T11:28:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.258+0000 7f39e8c00640 10 client.27410 handle_command_reply: tid=0 2024-03-21T11:28:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.258+0000 7f3a09027740 1 client.27410 shutdown 2024-03-21T11:28:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.258+0000 7f3a09027740 1 -- 172.21.15.47:0/3894143805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c9a9e0 msgr2=0x55b256ca68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.258+0000 7f3a09027740 1 --2- 172.21.15.47:0/3894143805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c9a9e0 0x55b256ca68f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f39ec009e90 tx=0x7f39ec005f90 comp rx=0 tx=0).stop 2024-03-21T11:28:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.259+0000 7f3a09027740 1 -- 172.21.15.47:0/3894143805 shutdown_connections 2024-03-21T11:28:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.259+0000 7f3a09027740 1 --2- 172.21.15.47:0/3894143805 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b256d0b520 0x55b256d0d940 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.259+0000 7f3a09027740 1 --2- 172.21.15.47:0/3894143805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b256c9b3a0 0x55b256ca9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.259+0000 7f3a09027740 1 --2- 172.21.15.47:0/3894143805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b256c9a9e0 0x55b256ca68f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.259+0000 7f3a09027740 1 --2- 172.21.15.47:0/3894143805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b256c99810 0x55b256ca63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.259+0000 7f39e8c00640 0 client.27410 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.259+0000 7f3a09027740 1 -- 172.21.15.47:0/3894143805 >> 172.21.15.47:0/3894143805 conn(0x55b256c94c00 msgr2=0x55b256c93620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.259+0000 7f3a09027740 1 -- 172.21.15.47:0/3894143805 shutdown_connections 2024-03-21T11:28:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.259+0000 7f3a09027740 1 -- 172.21.15.47:0/3894143805 wait complete. 2024-03-21T11:28:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.259+0000 7f3a09027740 20 client.27410 trim_cache size 0 max 16384 2024-03-21T11:28:57.264 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:28:57.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.260+0000 7f3a06200640 1 -- 172.21.15.47:0/175498007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39e40a2f20 msgr2=0x7f39e40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.260+0000 7f3a06200640 1 --2- 172.21.15.47:0/175498007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39e40a2f20 0x7f39e40a53e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55b256c9a8e0 tx=0x7f39ec005850 comp rx=0 tx=0).stop 2024-03-21T11:28:57.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.260+0000 7f3a06200640 1 -- 172.21.15.47:0/175498007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a00102150 msgr2=0x7f3a001a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:57.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.260+0000 7f3a06200640 1 --2- 172.21.15.47:0/175498007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a00102150 0x7f3a001a1210 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009fd0 tx=0x7f39f800b260 comp rx=0 tx=0).stop 2024-03-21T11:28:57.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.260+0000 7f3a06200640 1 -- 172.21.15.47:0/175498007 shutdown_connections 2024-03-21T11:28:57.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.260+0000 7f3a06200640 1 --2- 172.21.15.47:0/175498007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39e40a2f20 0x7f39e40a53e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.260+0000 7f3a06200640 1 --2- 172.21.15.47:0/175498007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a00102150 0x7f3a001a1210 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.260+0000 7f3a06200640 1 --2- 172.21.15.47:0/175498007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a00101790 0x7f3a0019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.260+0000 7f3a06200640 1 --2- 172.21.15.47:0/175498007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a001005a0 0x7f3a00199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:57.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.260+0000 7f3a06200640 1 -- 172.21.15.47:0/175498007 >> 172.21.15.47:0/175498007 conn(0x7f3a000fbb30 msgr2=0x7f3a00068800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:57.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.260+0000 7f3a06200640 1 -- 172.21.15.47:0/175498007 shutdown_connections 2024-03-21T11:28:57.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:57.260+0000 7f3a06200640 1 -- 172.21.15.47:0/175498007 wait complete. 2024-03-21T11:28:57.515 INFO:tasks.workunit.client.0.smithi047.stdout: 1 351620 8.35 MB/sec execute 101 sec latency 84.324 ms 2024-03-21T11:28:58.280 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:28:58.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:28:58.281 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:28:58.514 INFO:tasks.workunit.client.0.smithi047.stdout: 1 353299 8.32 MB/sec execute 102 sec latency 54.330 ms 2024-03-21T11:28:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.569+0000 7f36a8200640 1 -- 172.21.15.47:0/1347861473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a01016b0 msgr2=0x7f36a0101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.569+0000 7f36a8200640 1 --2- 172.21.15.47:0/1347861473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a01016b0 0x7f36a0101ad0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3694009900 tx=0x7f369402bc30 comp rx=0 tx=0).stop 2024-03-21T11:28:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.569+0000 7f36a8200640 1 -- 172.21.15.47:0/1347861473 shutdown_connections 2024-03-21T11:28:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.569+0000 7f36a8200640 1 --2- 172.21.15.47:0/1347861473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36a0103260 0x7f36a0109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.569+0000 7f36a8200640 1 --2- 172.21.15.47:0/1347861473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36a01028a0 0x7f36a0102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.569+0000 7f36a8200640 1 --2- 172.21.15.47:0/1347861473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a01016b0 0x7f36a0101ad0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.569+0000 7f36a8200640 1 -- 172.21.15.47:0/1347861473 >> 172.21.15.47:0/1347861473 conn(0x7f36a00fcc40 msgr2=0x7f36a00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.569+0000 7f36a8200640 1 -- 172.21.15.47:0/1347861473 shutdown_connections 2024-03-21T11:28:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.569+0000 7f36a8200640 1 -- 172.21.15.47:0/1347861473 wait complete. 2024-03-21T11:28:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.570+0000 7f36a8200640 1 Processor -- start 2024-03-21T11:28:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.570+0000 7f36a8200640 1 -- start start 2024-03-21T11:28:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.570+0000 7f36a8200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36a01016b0 0x7f36a0199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.570+0000 7f36a8200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36a01028a0 0x7f36a019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.570+0000 7f36a8200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a0103260 0x7f36a01a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.570+0000 7f36a8200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a010d330 con 0x7f36a01016b0 2024-03-21T11:28:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.570+0000 7f36a8200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a010d1b0 con 0x7f36a0103260 2024-03-21T11:28:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.570+0000 7f36a8200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a010d4b0 con 0x7f36a01028a0 2024-03-21T11:28:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.571+0000 7f36a4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36a01028a0 0x7f36a019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.571+0000 7f36a4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36a01028a0 0x7f36a019a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46288/0 (socket says 172.21.15.47:46288) 2024-03-21T11:28:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.571+0000 7f36a4e00640 1 -- 172.21.15.47:0/148662646 learned_addr learned my addr 172.21.15.47:0/148662646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.571+0000 7f36a5800640 1 --2- 172.21.15.47:0/148662646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36a01016b0 0x7f36a0199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.571+0000 7f36a6200640 1 --2- 172.21.15.47:0/148662646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a0103260 0x7f36a01a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.571+0000 7f36a4e00640 1 -- 172.21.15.47:0/148662646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a0103260 msgr2=0x7f36a01a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.571+0000 7f36a4e00640 1 --2- 172.21.15.47:0/148662646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a0103260 0x7f36a01a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.571+0000 7f36a4e00640 1 -- 172.21.15.47:0/148662646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36a01016b0 msgr2=0x7f36a0199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.571+0000 7f36a4e00640 1 --2- 172.21.15.47:0/148662646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36a01016b0 0x7f36a0199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.571+0000 7f36a4e00640 1 -- 172.21.15.47:0/148662646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a01a1890 con 0x7f36a01028a0 2024-03-21T11:28:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.571+0000 7f36a4e00640 1 --2- 172.21.15.47:0/148662646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36a01028a0 0x7f36a019a210 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f368c00e9f0 tx=0x7f368c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.572+0000 7f369e000640 1 -- 172.21.15.47:0/148662646 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c00cd80 con 0x7f36a01028a0 2024-03-21T11:28:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.572+0000 7f369e000640 1 -- 172.21.15.47:0/148662646 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f368c010910 con 0x7f36a01028a0 2024-03-21T11:28:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.572+0000 7f36a8200640 1 -- 172.21.15.47:0/148662646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a01a1b80 con 0x7f36a01028a0 2024-03-21T11:28:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.572+0000 7f369e000640 1 -- 172.21.15.47:0/148662646 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c0188f0 con 0x7f36a01028a0 2024-03-21T11:28:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.572+0000 7f36a8200640 1 -- 172.21.15.47:0/148662646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a01a20f0 con 0x7f36a01028a0 2024-03-21T11:28:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.576+0000 7f369e000640 1 -- 172.21.15.47:0/148662646 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f368c010ac0 con 0x7f36a01028a0 2024-03-21T11:28:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.576+0000 7f36aae6b740 1 Processor -- start 2024-03-21T11:28:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.576+0000 7f36aae6b740 1 -- start start 2024-03-21T11:28:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.576+0000 7f36aae6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae3910a0 0x5559ae3914c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.576+0000 7f36aae6b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae391a90 0x5559ae396590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.576+0000 7f36aae6b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae399090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.576+0000 7f36aae6b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559ae2f6eb0 con 0x5559ae391a90 2024-03-21T11:28:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.576+0000 7f36aae6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559ae29fa60 con 0x5559ae3910a0 2024-03-21T11:28:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.576+0000 7f36aae6b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559ae2f2f50 con 0x5559ae396c00 2024-03-21T11:28:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.577+0000 7f36a6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae3910a0 0x5559ae3914c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.577+0000 7f36a6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae3910a0 0x5559ae3914c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58618/0 (socket says 172.21.15.47:58618) 2024-03-21T11:28:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.577+0000 7f36a6200640 1 -- 172.21.15.47:0/3643123034 learned_addr learned my addr 172.21.15.47:0/3643123034 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.577+0000 7f36a4e00640 1 --2- 172.21.15.47:0/3643123034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae399090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.577+0000 7f36a5800640 1 --2- 172.21.15.47:0/3643123034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae391a90 0x5559ae396590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.577+0000 7f36a6200640 1 -- 172.21.15.47:0/3643123034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 msgr2=0x5559ae399090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.577+0000 7f36a6200640 1 --2- 172.21.15.47:0/3643123034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae399090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.577+0000 7f36a6200640 1 -- 172.21.15.47:0/3643123034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae391a90 msgr2=0x5559ae396590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.577+0000 7f36a6200640 1 --2- 172.21.15.47:0/3643123034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae391a90 0x5559ae396590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.577+0000 7f36a6200640 1 -- 172.21.15.47:0/3643123034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559ae0d37e0 con 0x5559ae3910a0 2024-03-21T11:28:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.577+0000 7f36a6200640 1 --2- 172.21.15.47:0/3643123034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae3910a0 0x5559ae3914c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3698009fd0 tx=0x7f36980118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b68779f296932004 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.578+0000 7f369e000640 1 --2- 172.21.15.47:0/148662646 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36880a2fa0 0x7f36880a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.578+0000 7f3692a00640 1 -- 172.21.15.47:0/3643123034 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3698004d20 con 0x5559ae3910a0 2024-03-21T11:28:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.578+0000 7f3692a00640 1 -- 172.21.15.47:0/3643123034 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3698004050 con 0x5559ae3910a0 2024-03-21T11:28:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.578+0000 7f3692a00640 1 -- 172.21.15.47:0/3643123034 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3698005260 con 0x5559ae3910a0 2024-03-21T11:28:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.578+0000 7f369e000640 1 -- 172.21.15.47:0/148662646 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f368c0cfb30 con 0x7f36a01028a0 2024-03-21T11:28:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.578+0000 7f36a5800640 1 --2- 172.21.15.47:0/148662646 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36880a2fa0 0x7f36880a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.579+0000 7f36aae6b740 1 -- 172.21.15.47:0/3643123034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae3910a0 msgr2=0x5559ae3914c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.579+0000 7f36a5800640 1 --2- 172.21.15.47:0/148662646 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36880a2fa0 0x7f36880a5460 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3694000c00 tx=0x7f3694005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.579+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3643123034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae3910a0 0x5559ae3914c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3698009fd0 tx=0x7f36980118d0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.579+0000 7f36aae6b740 1 -- 172.21.15.47:0/3643123034 shutdown_connections 2024-03-21T11:28:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.579+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3643123034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae399090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.579+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3643123034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae391a90 0x5559ae396590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.579+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3643123034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae3910a0 0x5559ae3914c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.579+0000 7f36aae6b740 1 -- 172.21.15.47:0/3643123034 >> 172.21.15.47:0/3643123034 conn(0x5559ae38c610 msgr2=0x5559ae38ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.579+0000 7f36aae6b740 1 -- 172.21.15.47:0/3643123034 shutdown_connections 2024-03-21T11:28:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.579+0000 7f36aae6b740 1 -- 172.21.15.47:0/3643123034 wait complete. 2024-03-21T11:28:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.579+0000 7f36aae6b740 1 -- start start 2024-03-21T11:28:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.580+0000 7f36aae6b740 1 Processor -- start 2024-03-21T11:28:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.580+0000 7f36aae6b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae3910a0 0x5559ae39b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.580+0000 7f36aae6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae391a90 0x5559ae39b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.580+0000 7f36aae6b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae396c00 0x5559ae3a2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.580+0000 7f36aae6b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559ae399bb0 con 0x5559ae396c00 2024-03-21T11:28:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.580+0000 7f36aae6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559ae2b2eb0 con 0x5559ae391a90 2024-03-21T11:28:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.580+0000 7f36aae6b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559ae399d30 con 0x5559ae3910a0 2024-03-21T11:28:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.580+0000 7f36aae6b740 10 client.0 _ll_register_callbacks cb 0x5559ae2a7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.580+0000 7f36a5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae3910a0 0x5559ae39b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.580+0000 7f36a4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae391a90 0x5559ae39b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.580+0000 7f36a5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae3910a0 0x5559ae39b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46320/0 (socket says 172.21.15.47:46320) 2024-03-21T11:28:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.580+0000 7f36a5800640 1 -- 172.21.15.47:0/3987607440 learned_addr learned my addr 172.21.15.47:0/3987607440 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.580+0000 7f36a6200640 1 --2- 172.21.15.47:0/3987607440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae396c00 0x5559ae3a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.581+0000 7f36a5800640 1 -- 172.21.15.47:0/3987607440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae391a90 msgr2=0x5559ae39b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.581+0000 7f36a5800640 1 --2- 172.21.15.47:0/3987607440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae391a90 0x5559ae39b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.581+0000 7f36a5800640 1 -- 172.21.15.47:0/3987607440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae396c00 msgr2=0x5559ae3a2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.581+0000 7f36a5800640 1 --2- 172.21.15.47:0/3987607440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae396c00 0x5559ae3a2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.581+0000 7f36a5800640 1 -- 172.21.15.47:0/3987607440 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559ae133f70 con 0x5559ae3910a0 2024-03-21T11:28:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.581+0000 7f36a4e00640 1 --2- 172.21.15.47:0/3987607440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae391a90 0x5559ae39b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.581+0000 7f36a5800640 1 --2- 172.21.15.47:0/3987607440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae3910a0 0x5559ae39b2b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f36a0102700 tx=0x7f3694009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c61f5965cfd2238d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.581+0000 7f3687400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.581+0000 7f3687400640 1 -- 172.21.15.47:0/3987607440 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369404c070 con 0x5559ae3910a0 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.581+0000 7f36aae6b740 1 -- 172.21.15.47:0/3987607440 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3698004050 con 0x5559ae3910a0 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.581+0000 7f3687400640 1 -- 172.21.15.47:0/3987607440 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3694005050 con 0x5559ae3910a0 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.582+0000 7f3687400640 1 -- 172.21.15.47:0/3987607440 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f369405a050 con 0x5559ae3910a0 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.582+0000 7f36aae6b740 10 client.27445 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.582+0000 7f36aae6b740 1 -- 172.21.15.47:0/3987607440 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5559adfcb170 con 0x5559ae3910a0 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.582+0000 7f3687400640 1 -- 172.21.15.47:0/3987607440 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3698004050 con 0x5559ae3910a0 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.582+0000 7f3687400640 1 -- 172.21.15.47:0/3987607440 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3694051460 con 0x5559ae3910a0 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.582+0000 7f36aae6b740 10 client.27445 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.582+0000 7f36aae6b740 10 client.27445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.582+0000 7f36aae6b740 20 client.27445 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.582+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3987607440 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5559ae3a7140 0x5559ae3a9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.582+0000 7f36aae6b740 4 client.27445 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.583+0000 7f36aae6b740 1 -- 172.21.15.47:0/3987607440 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559ae3043f0 con 0x5559ae3a7140 2024-03-21T11:28:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.583+0000 7f36a4e00640 1 --2- 172.21.15.47:0/3987607440 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5559ae3a7140 0x5559ae3a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.583+0000 7f36a4e00640 1 --2- 172.21.15.47:0/3987607440 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5559ae3a7140 0x5559ae3a9570 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f43bcc27b2c54839 server_cookie=142e8b4ba6bc3cad in_seq=0 out_seq=0 2024-03-21T11:28:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.583+0000 7f3687400640 10 client.27445 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.584+0000 7f3687400640 1 -- 172.21.15.47:0/3987607440 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5559ae3043f0 con 0x5559ae3a7140 2024-03-21T11:28:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.584+0000 7f3687400640 10 client.27445 handle_command_reply: tid=0 2024-03-21T11:28:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.584+0000 7f36aae6b740 1 client.27445 shutdown 2024-03-21T11:28:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.585+0000 7f36aae6b740 1 -- 172.21.15.47:0/3987607440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae3910a0 msgr2=0x5559ae39b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.585+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3987607440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae3910a0 0x5559ae39b2b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f36a0102700 tx=0x7f3694009260 comp rx=0 tx=0).stop 2024-03-21T11:28:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.585+0000 7f36aae6b740 1 -- 172.21.15.47:0/3987607440 shutdown_connections 2024-03-21T11:28:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.585+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3987607440 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5559ae3a7140 0x5559ae3a9570 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.585+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3987607440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae396c00 0x5559ae3a2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.585+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3987607440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae391a90 0x5559ae39b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.585+0000 7f3687400640 0 client.27445 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.585+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3987607440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae3910a0 0x5559ae39b2b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.585+0000 7f36aae6b740 1 -- 172.21.15.47:0/3987607440 >> 172.21.15.47:0/3987607440 conn(0x5559ae38c610 msgr2=0x5559ae38b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.585+0000 7f36aae6b740 1 -- 172.21.15.47:0/3987607440 shutdown_connections 2024-03-21T11:28:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.585+0000 7f36aae6b740 1 -- 172.21.15.47:0/3987607440 wait complete. 2024-03-21T11:28:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.585+0000 7f36aae6b740 20 client.27445 trim_cache size 0 max 16384 2024-03-21T11:28:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.604+0000 7f36aae6b740 1 Processor -- start 2024-03-21T11:28:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.604+0000 7f36aae6b740 1 -- start start 2024-03-21T11:28:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.604+0000 7f36aae6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae3910a0 0x5559ae39b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.604+0000 7f36aae6b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae391a90 0x5559ae3a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.604+0000 7f36aae6b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae38ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.604+0000 7f36aae6b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a01ac2b0 con 0x5559ae391a90 2024-03-21T11:28:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.604+0000 7f36aae6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a01adc30 con 0x5559ae3910a0 2024-03-21T11:28:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.604+0000 7f36aae6b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a01ac910 con 0x5559ae396c00 2024-03-21T11:28:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.605+0000 7f36a5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae3910a0 0x5559ae39b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.605+0000 7f36a6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae38ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.605+0000 7f36a5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae3910a0 0x5559ae39b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58640/0 (socket says 172.21.15.47:58640) 2024-03-21T11:28:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.605+0000 7f36a5800640 1 -- 172.21.15.47:0/1000921383 learned_addr learned my addr 172.21.15.47:0/1000921383 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.605+0000 7f36a4e00640 1 --2- 172.21.15.47:0/1000921383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae391a90 0x5559ae3a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.605+0000 7f36a5800640 1 -- 172.21.15.47:0/1000921383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 msgr2=0x5559ae38ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.605+0000 7f36a5800640 1 --2- 172.21.15.47:0/1000921383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae38ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.605+0000 7f36a5800640 1 -- 172.21.15.47:0/1000921383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae391a90 msgr2=0x5559ae3a27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.605+0000 7f36a5800640 1 --2- 172.21.15.47:0/1000921383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae391a90 0x5559ae3a27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.605+0000 7f36a5800640 1 -- 172.21.15.47:0/1000921383 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559ae390c60 con 0x5559ae3910a0 2024-03-21T11:28:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.605+0000 7f36a6200640 1 --2- 172.21.15.47:0/1000921383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae38ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.606+0000 7f36a5800640 1 --2- 172.21.15.47:0/1000921383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae3910a0 0x5559ae39b750 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3694009230 tx=0x7f3694005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=648303db98da1f4b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.607+0000 7f3692000640 1 -- 172.21.15.47:0/1000921383 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369404c030 con 0x5559ae3910a0 2024-03-21T11:28:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.608+0000 7f3692000640 1 -- 172.21.15.47:0/1000921383 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3694051460 con 0x5559ae3910a0 2024-03-21T11:28:58.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.608+0000 7f3692000640 1 -- 172.21.15.47:0/1000921383 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36940055d0 con 0x5559ae3910a0 2024-03-21T11:28:58.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.608+0000 7f36aae6b740 1 -- 172.21.15.47:0/1000921383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae3910a0 msgr2=0x5559ae39b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.608+0000 7f36aae6b740 1 --2- 172.21.15.47:0/1000921383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae3910a0 0x5559ae39b750 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3694009230 tx=0x7f3694005990 comp rx=0 tx=0).stop 2024-03-21T11:28:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.608+0000 7f36aae6b740 1 -- 172.21.15.47:0/1000921383 shutdown_connections 2024-03-21T11:28:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.608+0000 7f36aae6b740 1 --2- 172.21.15.47:0/1000921383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae38ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.608+0000 7f36aae6b740 1 --2- 172.21.15.47:0/1000921383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae391a90 0x5559ae3a27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.608+0000 7f36aae6b740 1 --2- 172.21.15.47:0/1000921383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae3910a0 0x5559ae39b750 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.608+0000 7f36aae6b740 1 -- 172.21.15.47:0/1000921383 >> 172.21.15.47:0/1000921383 conn(0x5559ae38c610 msgr2=0x5559ae38f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.608+0000 7f36aae6b740 1 -- 172.21.15.47:0/1000921383 shutdown_connections 2024-03-21T11:28:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.609+0000 7f36aae6b740 1 -- 172.21.15.47:0/1000921383 wait complete. 2024-03-21T11:28:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.609+0000 7f36aae6b740 1 -- start start 2024-03-21T11:28:58.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.609+0000 7f36aae6b740 1 Processor -- start 2024-03-21T11:28:58.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.609+0000 7f36aae6b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae3910a0 0x5559ae3a0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.609+0000 7f36aae6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae391a90 0x5559ae3a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.609+0000 7f36aae6b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae3a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36aae6b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a01ad5d0 con 0x5559ae3910a0 2024-03-21T11:28:58.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36aae6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a01acf70 con 0x5559ae391a90 2024-03-21T11:28:58.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36aae6b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a01aa470 con 0x5559ae396c00 2024-03-21T11:28:58.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36aae6b740 10 client.0 _ll_register_callbacks cb 0x7f3694009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36a4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae391a90 0x5559ae3a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36a6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae3a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36a6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae3a1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46334/0 (socket says 172.21.15.47:46334) 2024-03-21T11:28:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36a4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae391a90 0x5559ae3a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58648/0 (socket says 172.21.15.47:58648) 2024-03-21T11:28:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36a6200640 1 -- 172.21.15.47:0/3398183729 learned_addr learned my addr 172.21.15.47:0/3398183729 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36a5800640 1 --2- 172.21.15.47:0/3398183729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae3910a0 0x5559ae3a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36a6200640 1 -- 172.21.15.47:0/3398183729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae391a90 msgr2=0x5559ae3a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36a6200640 1 --2- 172.21.15.47:0/3398183729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae391a90 0x5559ae3a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36a6200640 1 -- 172.21.15.47:0/3398183729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae3910a0 msgr2=0x5559ae3a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36a6200640 1 --2- 172.21.15.47:0/3398183729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae3910a0 0x5559ae3a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36a6200640 1 -- 172.21.15.47:0/3398183729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559ae3a21c0 con 0x5559ae396c00 2024-03-21T11:28:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.610+0000 7f36a4e00640 1 --2- 172.21.15.47:0/3398183729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae391a90 0x5559ae3a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.611+0000 7f36a6200640 1 --2- 172.21.15.47:0/3398183729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae3a1bd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3698014040 tx=0x7f3698004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5046758d42bd5b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.611+0000 7f3687400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:28:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.611+0000 7f3687400640 1 -- 172.21.15.47:0/3398183729 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3698004050 con 0x5559ae396c00 2024-03-21T11:28:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.611+0000 7f36aae6b740 1 -- 172.21.15.47:0/3398183729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3694051460 con 0x5559ae396c00 2024-03-21T11:28:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.611+0000 7f3687400640 1 -- 172.21.15.47:0/3398183729 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f36980041f0 con 0x5559ae396c00 2024-03-21T11:28:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.611+0000 7f3687400640 1 -- 172.21.15.47:0/3398183729 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3698035050 con 0x5559ae396c00 2024-03-21T11:28:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.611+0000 7f36aae6b740 10 client.27451 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.611+0000 7f36aae6b740 1 -- 172.21.15.47:0/3398183729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5559ae403130 con 0x5559ae396c00 2024-03-21T11:28:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.612+0000 7f3687400640 1 -- 172.21.15.47:0/3398183729 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3694051460 con 0x5559ae396c00 2024-03-21T11:28:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.612+0000 7f3687400640 1 -- 172.21.15.47:0/3398183729 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3698005640 con 0x5559ae396c00 2024-03-21T11:28:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.612+0000 7f36aae6b740 10 client.27451 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.612+0000 7f36aae6b740 10 client.27451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.612+0000 7f36aae6b740 20 client.27451 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.612+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3398183729 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5559ae4038f0 0x5559ae405d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.612+0000 7f36aae6b740 4 client.27451 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:28:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.612+0000 7f36aae6b740 1 -- 172.21.15.47:0/3398183729 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5559ae2fe740 con 0x5559ae4038f0 2024-03-21T11:28:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.612+0000 7f36a5800640 1 --2- 172.21.15.47:0/3398183729 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5559ae4038f0 0x5559ae405d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.613+0000 7f36a5800640 1 --2- 172.21.15.47:0/3398183729 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5559ae4038f0 0x5559ae405d10 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22f2851a34376ea9 server_cookie=a29b064655085637 in_seq=0 out_seq=0 2024-03-21T11:28:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.613+0000 7f3687400640 10 client.27451 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.614+0000 7f3687400640 1 -- 172.21.15.47:0/3398183729 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5559ae2fe740 con 0x5559ae4038f0 2024-03-21T11:28:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.614+0000 7f3687400640 10 client.27451 handle_command_reply: tid=0 2024-03-21T11:28:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.614+0000 7f36aae6b740 1 client.27451 shutdown 2024-03-21T11:28:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.614+0000 7f36aae6b740 1 -- 172.21.15.47:0/3398183729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 msgr2=0x5559ae3a1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.614+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3398183729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae3a1bd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3698014040 tx=0x7f3698004ba0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.614+0000 7f36aae6b740 1 -- 172.21.15.47:0/3398183729 shutdown_connections 2024-03-21T11:28:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.614+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3398183729 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5559ae4038f0 0x5559ae405d10 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.614+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3398183729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5559ae396c00 0x5559ae3a1bd0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.614+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3398183729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559ae391a90 0x5559ae3a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.614+0000 7f36aae6b740 1 --2- 172.21.15.47:0/3398183729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559ae3910a0 0x5559ae3a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.614+0000 7f36aae6b740 1 -- 172.21.15.47:0/3398183729 >> 172.21.15.47:0/3398183729 conn(0x5559ae38c610 msgr2=0x5559ae39b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.614+0000 7f3687400640 0 client.27451 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.614+0000 7f36aae6b740 1 -- 172.21.15.47:0/3398183729 shutdown_connections 2024-03-21T11:28:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.614+0000 7f36aae6b740 1 -- 172.21.15.47:0/3398183729 wait complete. 2024-03-21T11:28:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.615+0000 7f36aae6b740 20 client.27451 trim_cache size 0 max 16384 2024-03-21T11:28:58.619 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:28:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:28:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "612e1030-4305-460a-8fd1-45c03320a4fa", 2024-03-21T11:28:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:28:58.619 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:28:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.615+0000 7f36a8200640 1 -- 172.21.15.47:0/148662646 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36880a2fa0 msgr2=0x7f36880a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.616+0000 7f36a8200640 1 --2- 172.21.15.47:0/148662646 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36880a2fa0 0x7f36880a5460 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3694000c00 tx=0x7f3694005fd0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.616+0000 7f36a8200640 1 -- 172.21.15.47:0/148662646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36a01028a0 msgr2=0x7f36a019a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.616+0000 7f36a8200640 1 --2- 172.21.15.47:0/148662646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36a01028a0 0x7f36a019a210 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f368c00e9f0 tx=0x7f368c00eeb0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.616+0000 7f36a8200640 1 -- 172.21.15.47:0/148662646 shutdown_connections 2024-03-21T11:28:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.616+0000 7f36a8200640 1 --2- 172.21.15.47:0/148662646 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36880a2fa0 0x7f36880a5460 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.616+0000 7f36a8200640 1 --2- 172.21.15.47:0/148662646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a0103260 0x7f36a01a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.616+0000 7f36a8200640 1 --2- 172.21.15.47:0/148662646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36a01028a0 0x7f36a019a210 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.616+0000 7f36a8200640 1 --2- 172.21.15.47:0/148662646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36a01016b0 0x7f36a0199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.616+0000 7f36a8200640 1 -- 172.21.15.47:0/148662646 >> 172.21.15.47:0/148662646 conn(0x7f36a00fcc40 msgr2=0x7f36a00738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.616+0000 7f36a8200640 1 -- 172.21.15.47:0/148662646 shutdown_connections 2024-03-21T11:28:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.616+0000 7f36a8200640 1 -- 172.21.15.47:0/148662646 wait complete. 2024-03-21T11:28:58.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:28:58.636 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:28:58.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:58 smithi047 ceph-mon[31351]: pgmap v569: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 304 op/s 2024-03-21T11:28:58.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.852+0000 7fdf5ea00640 1 -- 172.21.15.47:0/2500882414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf580756e0 msgr2=0x7fdf58073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.852+0000 7fdf5ea00640 1 --2- 172.21.15.47:0/2500882414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf580756e0 0x7fdf58073b30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009900 tx=0x7fdf4c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:28:58.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.852+0000 7fdf5ea00640 1 -- 172.21.15.47:0/2500882414 shutdown_connections 2024-03-21T11:28:58.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.852+0000 7fdf5ea00640 1 --2- 172.21.15.47:0/2500882414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf58109460 0x7fdf5810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.852+0000 7fdf5ea00640 1 --2- 172.21.15.47:0/2500882414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf58074100 0x7fdf58074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.852+0000 7fdf5ea00640 1 --2- 172.21.15.47:0/2500882414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf580756e0 0x7fdf58073b30 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.852+0000 7fdf5ea00640 1 -- 172.21.15.47:0/2500882414 >> 172.21.15.47:0/2500882414 conn(0x7fdf580fcb10 msgr2=0x7fdf580fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:58.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.852+0000 7fdf5ea00640 1 -- 172.21.15.47:0/2500882414 shutdown_connections 2024-03-21T11:28:58.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.852+0000 7fdf5ea00640 1 -- 172.21.15.47:0/2500882414 wait complete. 2024-03-21T11:28:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.853+0000 7fdf5ea00640 1 Processor -- start 2024-03-21T11:28:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.854+0000 7fdf5ea00640 1 -- start start 2024-03-21T11:28:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.854+0000 7fdf5ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf58074100 0x7fdf58199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.854+0000 7fdf5ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf580756e0 0x7fdf5819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.854+0000 7fdf5ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf58109460 0x7fdf581a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.854+0000 7fdf5ea00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5810f400 con 0x7fdf58109460 2024-03-21T11:28:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.854+0000 7fdf5ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5810f280 con 0x7fdf580756e0 2024-03-21T11:28:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.854+0000 7fdf5ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5810f580 con 0x7fdf58074100 2024-03-21T11:28:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.854+0000 7fdf5cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf580756e0 0x7fdf5819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.854+0000 7fdf5d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf58074100 0x7fdf58199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.854+0000 7fdf5cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf580756e0 0x7fdf5819a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58660/0 (socket says 172.21.15.47:58660) 2024-03-21T11:28:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.854+0000 7fdf5cc00640 1 -- 172.21.15.47:0/1484237270 learned_addr learned my addr 172.21.15.47:0/1484237270 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.854+0000 7fdf5d600640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf58074100 0x7fdf58199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46348/0 (socket says 172.21.15.47:46348) 2024-03-21T11:28:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.854+0000 7fdf5e000640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf58109460 0x7fdf581a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.855+0000 7fdf5d600640 1 -- 172.21.15.47:0/1484237270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf580756e0 msgr2=0x7fdf5819a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.855+0000 7fdf5d600640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf580756e0 0x7fdf5819a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.855+0000 7fdf5d600640 1 -- 172.21.15.47:0/1484237270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf58109460 msgr2=0x7fdf581a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.855+0000 7fdf5d600640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf58109460 0x7fdf581a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.855+0000 7fdf5d600640 1 -- 172.21.15.47:0/1484237270 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf581a16f0 con 0x7fdf58074100 2024-03-21T11:28:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.855+0000 7fdf5e000640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf58109460 0x7fdf581a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.855+0000 7fdf5d600640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf58074100 0x7fdf58199ad0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009230 tx=0x7fdf4c02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.855+0000 7fdf56000640 1 -- 172.21.15.47:0/1484237270 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c041070 con 0x7fdf58074100 2024-03-21T11:28:58.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.855+0000 7fdf56000640 1 -- 172.21.15.47:0/1484237270 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf4c004450 con 0x7fdf58074100 2024-03-21T11:28:58.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.855+0000 7fdf56000640 1 -- 172.21.15.47:0/1484237270 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c003df0 con 0x7fdf58074100 2024-03-21T11:28:58.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.855+0000 7fdf5ea00640 1 -- 172.21.15.47:0/1484237270 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf581a1980 con 0x7fdf58074100 2024-03-21T11:28:58.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.855+0000 7fdf5ea00640 1 -- 172.21.15.47:0/1484237270 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf581a1e20 con 0x7fdf58074100 2024-03-21T11:28:58.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.860+0000 7fdf62d94740 1 Processor -- start 2024-03-21T11:28:58.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.860+0000 7fdf62d94740 1 -- start start 2024-03-21T11:28:58.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.860+0000 7fdf56000640 1 -- 172.21.15.47:0/1484237270 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdf4c044070 con 0x7fdf58074100 2024-03-21T11:28:58.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.860+0000 7fdf62d94740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1c050 0x55dd37f1c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.860+0000 7fdf62d94740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f1ca40 0x55dd37f21540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.860+0000 7fdf62d94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f21bb0 0x55dd37f24040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.860+0000 7fdf62d94740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd37e81250 con 0x55dd37f1c050 2024-03-21T11:28:58.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.860+0000 7fdf62d94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd37e29a30 con 0x55dd37f21bb0 2024-03-21T11:28:58.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.860+0000 7fdf62d94740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd37e7d2f0 con 0x55dd37f1ca40 2024-03-21T11:28:58.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.861+0000 7fdf5d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f1ca40 0x55dd37f21540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.861+0000 7fdf5d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f1ca40 0x55dd37f21540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46360/0 (socket says 172.21.15.47:46360) 2024-03-21T11:28:58.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.861+0000 7fdf5d600640 1 -- 172.21.15.47:0/1508386328 learned_addr learned my addr 172.21.15.47:0/1508386328 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:58.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.861+0000 7fdf5cc00640 1 --2- 172.21.15.47:0/1508386328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f21bb0 0x55dd37f24040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.861+0000 7fdf5e000640 1 --2- 172.21.15.47:0/1508386328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1c050 0x55dd37f1c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.861+0000 7fdf5d600640 1 -- 172.21.15.47:0/1508386328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f21bb0 msgr2=0x55dd37f24040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.861+0000 7fdf5d600640 1 --2- 172.21.15.47:0/1508386328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f21bb0 0x55dd37f24040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.861+0000 7fdf5d600640 1 -- 172.21.15.47:0/1508386328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1c050 msgr2=0x55dd37f1c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.861+0000 7fdf5d600640 1 --2- 172.21.15.47:0/1508386328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1c050 0x55dd37f1c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.861+0000 7fdf5d600640 1 -- 172.21.15.47:0/1508386328 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd37e14690 con 0x55dd37f1ca40 2024-03-21T11:28:58.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.861+0000 7fdf5e000640 1 --2- 172.21.15.47:0/1508386328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1c050 0x55dd37f1c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:28:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.862+0000 7fdf5d600640 1 --2- 172.21.15.47:0/1508386328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f1ca40 0x55dd37f21540 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0f9260 tx=0x7fdf4c0f9350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75b8e83aa64ad168 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.862+0000 7fdf4a600640 1 -- 172.21.15.47:0/1508386328 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0f30d0 con 0x55dd37f1ca40 2024-03-21T11:28:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.862+0000 7fdf4a600640 1 -- 172.21.15.47:0/1508386328 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf4c0fe030 con 0x55dd37f1ca40 2024-03-21T11:28:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.862+0000 7fdf4a600640 1 -- 172.21.15.47:0/1508386328 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c1025d0 con 0x55dd37f1ca40 2024-03-21T11:28:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.862+0000 7fdf56000640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdf400a2fa0 0x7fdf400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.862+0000 7fdf56000640 1 -- 172.21.15.47:0/1484237270 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdf4c0f11a0 con 0x7fdf58074100 2024-03-21T11:28:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.862+0000 7fdf5e000640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdf400a2fa0 0x7fdf400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.863+0000 7fdf62d94740 1 -- 172.21.15.47:0/1508386328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f1ca40 msgr2=0x55dd37f21540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.863+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1508386328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f1ca40 0x55dd37f21540 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0f9260 tx=0x7fdf4c0f9350 comp rx=0 tx=0).stop 2024-03-21T11:28:58.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.863+0000 7fdf62d94740 1 -- 172.21.15.47:0/1508386328 shutdown_connections 2024-03-21T11:28:58.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.863+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1508386328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f21bb0 0x55dd37f24040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.863+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1508386328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f1ca40 0x55dd37f21540 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.863+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1508386328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1c050 0x55dd37f1c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.863+0000 7fdf62d94740 1 -- 172.21.15.47:0/1508386328 >> 172.21.15.47:0/1508386328 conn(0x55dd37ee4200 msgr2=0x55dd37f179c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:58.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.863+0000 7fdf62d94740 1 -- 172.21.15.47:0/1508386328 shutdown_connections 2024-03-21T11:28:58.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.863+0000 7fdf62d94740 1 -- 172.21.15.47:0/1508386328 wait complete. 2024-03-21T11:28:58.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.864+0000 7fdf5e000640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdf400a2fa0 0x7fdf400a5460 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55dd37f1d2a0 tx=0x7fdf50009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:58.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.864+0000 7fdf62d94740 1 -- start start 2024-03-21T11:28:58.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.864+0000 7fdf62d94740 1 Processor -- start 2024-03-21T11:28:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf62d94740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1c050 0x55dd37f25d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf62d94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1ca40 0x55dd37f262c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf62d94740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f21bb0 0x55dd37f2d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf62d94740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd37f24920 con 0x55dd37f1c050 2024-03-21T11:28:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf62d94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd37e3ce80 con 0x55dd37f1ca40 2024-03-21T11:28:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf62d94740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd37f24aa0 con 0x55dd37f21bb0 2024-03-21T11:28:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf62d94740 10 client.0 _ll_register_callbacks cb 0x55dd37e4f130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf5d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1c050 0x55dd37f25d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf5e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f21bb0 0x55dd37f2d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf5cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1ca40 0x55dd37f262c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf5d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1c050 0x55dd37f25d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47528/0 (socket says 172.21.15.47:47528) 2024-03-21T11:28:58.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf5d600640 1 -- 172.21.15.47:0/1684805206 learned_addr learned my addr 172.21.15.47:0/1684805206 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:58.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf5d600640 1 -- 172.21.15.47:0/1684805206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f21bb0 msgr2=0x55dd37f2d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf5d600640 1 --2- 172.21.15.47:0/1684805206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f21bb0 0x55dd37f2d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf5d600640 1 -- 172.21.15.47:0/1684805206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1ca40 msgr2=0x55dd37f262c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf5d600640 1 --2- 172.21.15.47:0/1684805206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1ca40 0x55dd37f262c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf5d600640 1 -- 172.21.15.47:0/1684805206 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd37c5f7d0 con 0x55dd37f1c050 2024-03-21T11:28:58.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf5cc00640 1 --2- 172.21.15.47:0/1684805206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1ca40 0x55dd37f262c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:58.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.865+0000 7fdf5e000640 1 --2- 172.21.15.47:0/1684805206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f21bb0 0x55dd37f2d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:28:58.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.866+0000 7fdf5d600640 1 --2- 172.21.15.47:0/1684805206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1c050 0x55dd37f25d80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0b9f10 tx=0x7fdf4c005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5b8f200cb51dfd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:58.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.866+0000 7fdf3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:28:58.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.866+0000 7fdf3ea00640 1 -- 172.21.15.47:0/1684805206 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c04b030 con 0x55dd37f1c050 2024-03-21T11:28:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.866+0000 7fdf3ea00640 1 -- 172.21.15.47:0/1684805206 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdf4c0bb840 con 0x55dd37f1c050 2024-03-21T11:28:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.866+0000 7fdf62d94740 1 -- 172.21.15.47:0/1684805206 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf4c0fe030 con 0x55dd37f1c050 2024-03-21T11:28:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.866+0000 7fdf3ea00640 1 -- 172.21.15.47:0/1684805206 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf4c0b97e0 con 0x55dd37f1c050 2024-03-21T11:28:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.866+0000 7fdf62d94740 10 client.17178 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.866+0000 7fdf62d94740 1 -- 172.21.15.47:0/1684805206 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dd37b953f0 con 0x55dd37f1c050 2024-03-21T11:28:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.867+0000 7fdf3ea00640 1 -- 172.21.15.47:0/1684805206 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf4c0fe030 con 0x55dd37f1c050 2024-03-21T11:28:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.867+0000 7fdf3ea00640 1 -- 172.21.15.47:0/1684805206 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdf4c0b9bb0 con 0x55dd37f1c050 2024-03-21T11:28:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.867+0000 7fdf62d94740 10 client.17178 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.867+0000 7fdf62d94740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.867+0000 7fdf62d94740 20 client.17178 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.867+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1684805206 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dd37f31820 0x55dd37f33c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.867+0000 7fdf62d94740 4 client.17178 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:28:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.867+0000 7fdf62d94740 1 -- 172.21.15.47:0/1684805206 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd37e8e790 con 0x55dd37f31820 2024-03-21T11:28:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.867+0000 7fdf5cc00640 1 --2- 172.21.15.47:0/1684805206 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dd37f31820 0x55dd37f33c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.868+0000 7fdf5cc00640 1 --2- 172.21.15.47:0/1684805206 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dd37f31820 0x55dd37f33c10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73dfc59b80cf708d server_cookie=f765a6a207aae0bb in_seq=0 out_seq=0 2024-03-21T11:28:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.868+0000 7fdf3ea00640 10 client.17178 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.869+0000 7fdf3ea00640 1 -- 172.21.15.47:0/1684805206 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dd37e8e790 con 0x55dd37f31820 2024-03-21T11:28:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.869+0000 7fdf3ea00640 10 client.17178 handle_command_reply: tid=0 2024-03-21T11:28:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.869+0000 7fdf62d94740 1 client.17178 shutdown 2024-03-21T11:28:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.869+0000 7fdf62d94740 1 -- 172.21.15.47:0/1684805206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1c050 msgr2=0x55dd37f25d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.869+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1684805206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1c050 0x55dd37f25d80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0b9f10 tx=0x7fdf4c005670 comp rx=0 tx=0).stop 2024-03-21T11:28:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.869+0000 7fdf62d94740 1 -- 172.21.15.47:0/1684805206 shutdown_connections 2024-03-21T11:28:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.869+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1684805206 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dd37f31820 0x55dd37f33c10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.869+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1684805206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f21bb0 0x55dd37f2d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.869+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1684805206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1ca40 0x55dd37f262c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.869+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1684805206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1c050 0x55dd37f25d80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.869+0000 7fdf62d94740 1 -- 172.21.15.47:0/1684805206 >> 172.21.15.47:0/1684805206 conn(0x55dd37ee4200 msgr2=0x55dd37f16bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.869+0000 7fdf3ea00640 0 client.17178 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.869+0000 7fdf62d94740 1 -- 172.21.15.47:0/1684805206 shutdown_connections 2024-03-21T11:28:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.869+0000 7fdf62d94740 1 -- 172.21.15.47:0/1684805206 wait complete. 2024-03-21T11:28:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.870+0000 7fdf62d94740 20 client.17178 trim_cache size 0 max 16384 2024-03-21T11:28:58.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.889+0000 7fdf62d94740 1 Processor -- start 2024-03-21T11:28:58.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.889+0000 7fdf62d94740 1 -- start start 2024-03-21T11:28:58.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.889+0000 7fdf62d94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1c050 0x55dd37f26220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.889+0000 7fdf62d94740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f1ca40 0x55dd37f2d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.889+0000 7fdf62d94740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f21bb0 0x55dd37f1dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.889+0000 7fdf62d94740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf581ac140 con 0x55dd37f21bb0 2024-03-21T11:28:58.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.889+0000 7fdf62d94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf581adac0 con 0x55dd37f1c050 2024-03-21T11:28:58.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.889+0000 7fdf62d94740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf581ac7a0 con 0x55dd37f1ca40 2024-03-21T11:28:58.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.889+0000 7fdf5e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f21bb0 0x55dd37f1dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.889+0000 7fdf5d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1c050 0x55dd37f26220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.890+0000 7fdf5e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f21bb0 0x55dd37f1dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33140/0 (socket says 172.21.15.47:33140) 2024-03-21T11:28:58.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.890+0000 7fdf5e000640 1 -- 172.21.15.47:0/2108544181 learned_addr learned my addr 172.21.15.47:0/2108544181 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:58.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.890+0000 7fdf5cc00640 1 --2- 172.21.15.47:0/2108544181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f1ca40 0x55dd37f2d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.890+0000 7fdf5e000640 1 -- 172.21.15.47:0/2108544181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f1ca40 msgr2=0x55dd37f2d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.890+0000 7fdf5e000640 1 --2- 172.21.15.47:0/2108544181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f1ca40 0x55dd37f2d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.890+0000 7fdf5e000640 1 -- 172.21.15.47:0/2108544181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1c050 msgr2=0x55dd37f26220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.890+0000 7fdf5e000640 1 --2- 172.21.15.47:0/2108544181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1c050 0x55dd37f26220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.890+0000 7fdf5e000640 1 -- 172.21.15.47:0/2108544181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd37b550f0 con 0x55dd37f21bb0 2024-03-21T11:28:58.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.890+0000 7fdf5e000640 1 --2- 172.21.15.47:0/2108544181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f21bb0 0x55dd37f1dbc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdf50010b10 tx=0x7fdf50019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939915ff80847d4c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:58.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.891+0000 7fdf49c00640 1 -- 172.21.15.47:0/2108544181 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50004d20 con 0x55dd37f21bb0 2024-03-21T11:28:58.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.891+0000 7fdf49c00640 1 -- 172.21.15.47:0/2108544181 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf50005020 con 0x55dd37f21bb0 2024-03-21T11:28:58.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.891+0000 7fdf49c00640 1 -- 172.21.15.47:0/2108544181 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50028670 con 0x55dd37f21bb0 2024-03-21T11:28:58.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.891+0000 7fdf62d94740 1 -- 172.21.15.47:0/2108544181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f21bb0 msgr2=0x55dd37f1dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.891+0000 7fdf62d94740 1 --2- 172.21.15.47:0/2108544181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f21bb0 0x55dd37f1dbc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdf50010b10 tx=0x7fdf50019000 comp rx=0 tx=0).stop 2024-03-21T11:28:58.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.892+0000 7fdf62d94740 1 -- 172.21.15.47:0/2108544181 shutdown_connections 2024-03-21T11:28:58.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.892+0000 7fdf62d94740 1 --2- 172.21.15.47:0/2108544181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f21bb0 0x55dd37f1dbc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.892+0000 7fdf62d94740 1 --2- 172.21.15.47:0/2108544181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f1ca40 0x55dd37f2d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.892+0000 7fdf62d94740 1 --2- 172.21.15.47:0/2108544181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1c050 0x55dd37f26220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.892+0000 7fdf62d94740 1 -- 172.21.15.47:0/2108544181 >> 172.21.15.47:0/2108544181 conn(0x55dd37ee4200 msgr2=0x55dd37f1d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:58.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.892+0000 7fdf62d94740 1 -- 172.21.15.47:0/2108544181 shutdown_connections 2024-03-21T11:28:58.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.892+0000 7fdf62d94740 1 -- 172.21.15.47:0/2108544181 wait complete. 2024-03-21T11:28:58.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:58 smithi100 ceph-mon[36245]: pgmap v569: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 304 op/s 2024-03-21T11:28:58.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.892+0000 7fdf62d94740 1 -- start start 2024-03-21T11:28:58.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.892+0000 7fdf62d94740 1 Processor -- start 2024-03-21T11:28:58.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.893+0000 7fdf62d94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1c050 0x55dd37f2a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.893+0000 7fdf62d94740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1ca40 0x55dd37f2af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.893+0000 7fdf62d94740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f21bb0 0x55dd37f2bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.893+0000 7fdf62d94740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf581ad460 con 0x55dd37f1ca40 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.893+0000 7fdf62d94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf581ace00 con 0x55dd37f1c050 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.893+0000 7fdf62d94740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf581aa2e0 con 0x55dd37f21bb0 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.893+0000 7fdf62d94740 10 client.0 _ll_register_callbacks cb 0x7fdf44009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.893+0000 7fdf5d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1c050 0x55dd37f2a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.893+0000 7fdf5d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1c050 0x55dd37f2a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43606/0 (socket says 172.21.15.47:43606) 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.893+0000 7fdf5d600640 1 -- 172.21.15.47:0/1531876384 learned_addr learned my addr 172.21.15.47:0/1531876384 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.893+0000 7fdf5cc00640 1 --2- 172.21.15.47:0/1531876384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1ca40 0x55dd37f2af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.894+0000 7fdf5e000640 1 --2- 172.21.15.47:0/1531876384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f21bb0 0x55dd37f2bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.894+0000 7fdf5d600640 1 -- 172.21.15.47:0/1531876384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f21bb0 msgr2=0x55dd37f2bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.894+0000 7fdf5d600640 1 --2- 172.21.15.47:0/1531876384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f21bb0 0x55dd37f2bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.894+0000 7fdf5d600640 1 -- 172.21.15.47:0/1531876384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1ca40 msgr2=0x55dd37f2af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.894+0000 7fdf5d600640 1 --2- 172.21.15.47:0/1531876384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1ca40 0x55dd37f2af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.894+0000 7fdf5d600640 1 -- 172.21.15.47:0/1531876384 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd37f2c530 con 0x55dd37f1c050 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.894+0000 7fdf5d600640 1 --2- 172.21.15.47:0/1531876384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1c050 0x55dd37f2a9f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c02c7b0 tx=0x7fdf4c0057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=955a5170e3cac268 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:28:58.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.894+0000 7fdf3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:28:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.894+0000 7fdf3ea00640 1 -- 172.21.15.47:0/1531876384 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c04b040 con 0x55dd37f1c050 2024-03-21T11:28:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.894+0000 7fdf62d94740 1 -- 172.21.15.47:0/1531876384 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf50005020 con 0x55dd37f1c050 2024-03-21T11:28:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.895+0000 7fdf3ea00640 1 -- 172.21.15.47:0/1531876384 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdf4c0c5ac0 con 0x55dd37f1c050 2024-03-21T11:28:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.895+0000 7fdf3ea00640 1 -- 172.21.15.47:0/1531876384 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf4c0bcea0 con 0x55dd37f1c050 2024-03-21T11:28:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.895+0000 7fdf62d94740 10 client.27446 fetch_fsmap learned FSMap version 28 2024-03-21T11:28:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.895+0000 7fdf62d94740 1 -- 172.21.15.47:0/1531876384 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dd37f2c530 con 0x55dd37f1c050 2024-03-21T11:28:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.895+0000 7fdf3ea00640 1 -- 172.21.15.47:0/1531876384 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf50005020 con 0x55dd37f1c050 2024-03-21T11:28:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.895+0000 7fdf3ea00640 1 -- 172.21.15.47:0/1531876384 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdf4c0bd7a0 con 0x55dd37f1c050 2024-03-21T11:28:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.895+0000 7fdf62d94740 10 client.27446 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:28:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.895+0000 7fdf62d94740 10 client.27446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:28:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.896+0000 7fdf62d94740 20 client.27446 populate_metadata read hostname 'smithi047' 2024-03-21T11:28:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.896+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1531876384 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dd37f8d6b0 0x55dd37f8fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:28:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.896+0000 7fdf62d94740 4 client.27446 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:28:58.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.896+0000 7fdf62d94740 1 -- 172.21.15.47:0/1531876384 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dd37e88ae0 con 0x55dd37f8d6b0 2024-03-21T11:28:58.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.896+0000 7fdf5cc00640 1 --2- 172.21.15.47:0/1531876384 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dd37f8d6b0 0x55dd37f8fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:28:58.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.896+0000 7fdf5cc00640 1 --2- 172.21.15.47:0/1531876384 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dd37f8d6b0 0x55dd37f8fad0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5037440d77cded48 server_cookie=6dce8c5f6f8b2afc in_seq=0 out_seq=0 2024-03-21T11:28:58.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.896+0000 7fdf3ea00640 10 client.27446 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:58.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf3ea00640 1 -- 172.21.15.47:0/1531876384 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dd37e88ae0 con 0x55dd37f8d6b0 2024-03-21T11:28:58.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf3ea00640 10 client.27446 handle_command_reply: tid=0 2024-03-21T11:28:58.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf62d94740 1 client.27446 shutdown 2024-03-21T11:28:58.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf62d94740 1 -- 172.21.15.47:0/1531876384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1c050 msgr2=0x55dd37f2a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1531876384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1c050 0x55dd37f2a9f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c02c7b0 tx=0x7fdf4c0057e0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf62d94740 1 -- 172.21.15.47:0/1531876384 shutdown_connections 2024-03-21T11:28:58.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1531876384 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dd37f8d6b0 0x55dd37f8fad0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1531876384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd37f21bb0 0x55dd37f2bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1531876384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd37f1ca40 0x55dd37f2af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf3ea00640 0 client.27446 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:28:58.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf62d94740 1 --2- 172.21.15.47:0/1531876384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd37f1c050 0x55dd37f2a9f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf62d94740 1 -- 172.21.15.47:0/1531876384 >> 172.21.15.47:0/1531876384 conn(0x55dd37ee4200 msgr2=0x55dd37f1b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:58.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf62d94740 1 -- 172.21.15.47:0/1531876384 shutdown_connections 2024-03-21T11:28:58.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf62d94740 1 -- 172.21.15.47:0/1531876384 wait complete. 2024-03-21T11:28:58.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.897+0000 7fdf62d94740 20 client.27446 trim_cache size 0 max 16384 2024-03-21T11:28:58.901 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:28:58.902 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:28:58.902 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:28:58.902 INFO:teuthology.orchestra.run.smithi047.stdout: "612e1030-4305-460a-8fd1-45c03320a4fa": { 2024-03-21T11:28:58.902 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:28:58.902 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "612e1030-4305-460a-8fd1-45c03320a4fa", 2024-03-21T11:28:58.902 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:28:58.902 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:28:58.902 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:28:58.902 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:28:58.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.898+0000 7fdf5ea00640 1 -- 172.21.15.47:0/1484237270 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdf400a2fa0 msgr2=0x7fdf400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.898+0000 7fdf5ea00640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdf400a2fa0 0x7fdf400a5460 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55dd37f1d2a0 tx=0x7fdf50009500 comp rx=0 tx=0).stop 2024-03-21T11:28:58.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.898+0000 7fdf5ea00640 1 -- 172.21.15.47:0/1484237270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf58074100 msgr2=0x7fdf58199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:28:58.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.898+0000 7fdf5ea00640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf58074100 0x7fdf58199ad0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009230 tx=0x7fdf4c02c650 comp rx=0 tx=0).stop 2024-03-21T11:28:58.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.898+0000 7fdf5ea00640 1 -- 172.21.15.47:0/1484237270 shutdown_connections 2024-03-21T11:28:58.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.898+0000 7fdf5ea00640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdf400a2fa0 0x7fdf400a5460 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.898+0000 7fdf5ea00640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf58109460 0x7fdf581a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.898+0000 7fdf5ea00640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf580756e0 0x7fdf5819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.898+0000 7fdf5ea00640 1 --2- 172.21.15.47:0/1484237270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf58074100 0x7fdf58199ad0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:28:58.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.898+0000 7fdf5ea00640 1 -- 172.21.15.47:0/1484237270 >> 172.21.15.47:0/1484237270 conn(0x7fdf580fcb10 msgr2=0x7fdf58105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:28:58.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.898+0000 7fdf5ea00640 1 -- 172.21.15.47:0/1484237270 shutdown_connections 2024-03-21T11:28:58.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:28:58.898+0000 7fdf5ea00640 1 -- 172.21.15.47:0/1484237270 wait complete. 2024-03-21T11:28:58.919 INFO:tasks.cephfs.filesystem:scrub status for tag:612e1030-4305-460a-8fd1-45c03320a4fa - {'path': '/', 'tag': '612e1030-4305-460a-8fd1-45c03320a4fa', 'options': 'recursive,force'} 2024-03-21T11:28:58.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:58 smithi102 ceph-mon[36748]: pgmap v569: 129 pgs: 129 active+clean; 41 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 304 op/s 2024-03-21T11:28:59.515 INFO:tasks.workunit.client.0.smithi047.stdout: 1 355111 8.37 MB/sec execute 103 sec latency 9.919 ms 2024-03-21T11:28:59.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:59 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:28:59.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:59 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:28:59.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:59 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:28:59.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:28:59 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:28:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:59 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:28:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:59 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:28:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:59 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:28:59.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:28:59 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:29:00.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:59 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:29:00.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:59 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:29:00.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:59 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:29:00.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:28:59 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:29:00.515 INFO:tasks.workunit.client.0.smithi047.stdout: 1 356783 8.34 MB/sec execute 104 sec latency 13.516 ms 2024-03-21T11:29:00.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:00 smithi100 ceph-mon[36245]: pgmap v570: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 289 op/s 2024-03-21T11:29:00.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:00 smithi102 ceph-mon[36748]: pgmap v570: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 289 op/s 2024-03-21T11:29:01.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:00 smithi047 ceph-mon[31351]: pgmap v570: 129 pgs: 129 active+clean; 42 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 289 op/s 2024-03-21T11:29:01.515 INFO:tasks.workunit.client.0.smithi047.stdout: 1 358501 8.39 MB/sec execute 105 sec latency 11.680 ms 2024-03-21T11:29:02.515 INFO:tasks.workunit.client.0.smithi047.stdout: 1 359447 8.33 MB/sec execute 106 sec latency 20.281 ms 2024-03-21T11:29:02.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:02 smithi100 ceph-mon[36245]: pgmap v571: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-03-21T11:29:02.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:02 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:29:02.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:02 smithi102 ceph-mon[36748]: pgmap v571: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-03-21T11:29:02.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:02 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:29:03.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:02 smithi047 ceph-mon[31351]: pgmap v571: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-03-21T11:29:03.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:02 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:29:03.516 INFO:tasks.workunit.client.0.smithi047.stdout: 1 361258 8.35 MB/sec execute 107 sec latency 46.458 ms 2024-03-21T11:29:04.516 INFO:tasks.workunit.client.0.smithi047.stdout: 1 362490 8.34 MB/sec execute 108 sec latency 12.205 ms 2024-03-21T11:29:04.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:04 smithi100 ceph-mon[36245]: pgmap v572: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 333 op/s 2024-03-21T11:29:04.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:04 smithi102 ceph-mon[36748]: pgmap v572: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 333 op/s 2024-03-21T11:29:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:04 smithi047 ceph-mon[31351]: pgmap v572: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 333 op/s 2024-03-21T11:29:05.515 INFO:tasks.workunit.client.0.smithi047.stdout: 1 364232 8.32 MB/sec execute 109 sec latency 12.055 ms 2024-03-21T11:29:06.516 INFO:tasks.workunit.client.0.smithi047.stdout: 1 365719 8.36 MB/sec execute 110 sec latency 11.743 ms 2024-03-21T11:29:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:06 smithi100 ceph-mon[36245]: pgmap v573: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-03-21T11:29:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:06 smithi102 ceph-mon[36748]: pgmap v573: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-03-21T11:29:07.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:06 smithi047 ceph-mon[31351]: pgmap v573: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-03-21T11:29:07.516 INFO:tasks.workunit.client.0.smithi047.stdout: 1 366899 8.33 MB/sec execute 111 sec latency 23.396 ms 2024-03-21T11:29:08.517 INFO:tasks.workunit.client.0.smithi047.stdout: 1 368612 8.34 MB/sec execute 112 sec latency 44.218 ms 2024-03-21T11:29:08.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:08 smithi100 ceph-mon[36245]: pgmap v574: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 291 op/s 2024-03-21T11:29:08.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:08 smithi102 ceph-mon[36748]: pgmap v574: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 291 op/s 2024-03-21T11:29:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:08 smithi047 ceph-mon[31351]: pgmap v574: 129 pgs: 129 active+clean; 39 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 291 op/s 2024-03-21T11:29:09.517 INFO:tasks.workunit.client.0.smithi047.stdout: 1 369674 8.31 MB/sec execute 113 sec latency 15.276 ms 2024-03-21T11:29:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.174+0000 7f6f46e00640 1 -- 172.21.15.47:0/1330017162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f40103900 msgr2=0x7f6f40103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.174+0000 7f6f46e00640 1 --2- 172.21.15.47:0/1330017162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f40103900 0x7f6f40103d20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009900 tx=0x7f6f3002bc30 comp rx=0 tx=0).stop 2024-03-21T11:29:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.174+0000 7f6f46e00640 1 -- 172.21.15.47:0/1330017162 shutdown_connections 2024-03-21T11:29:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.174+0000 7f6f46e00640 1 --2- 172.21.15.47:0/1330017162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f401054b0 0x7f6f4010bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.174+0000 7f6f46e00640 1 --2- 172.21.15.47:0/1330017162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f40104af0 0x7f6f40104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.174+0000 7f6f46e00640 1 --2- 172.21.15.47:0/1330017162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f40103900 0x7f6f40103d20 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.174+0000 7f6f46e00640 1 -- 172.21.15.47:0/1330017162 >> 172.21.15.47:0/1330017162 conn(0x7f6f400fee90 msgr2=0x7f6f401012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.174+0000 7f6f46e00640 1 -- 172.21.15.47:0/1330017162 shutdown_connections 2024-03-21T11:29:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.174+0000 7f6f46e00640 1 -- 172.21.15.47:0/1330017162 wait complete. 2024-03-21T11:29:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.175+0000 7f6f46e00640 1 Processor -- start 2024-03-21T11:29:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.175+0000 7f6f46e00640 1 -- start start 2024-03-21T11:29:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.175+0000 7f6f46e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f40103900 0x7f6f40199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f46e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f40104af0 0x7f6f4019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f46e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f401054b0 0x7f6f401a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f46e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4010f5a0 con 0x7f6f401054b0 2024-03-21T11:29:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f46e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4010f420 con 0x7f6f40103900 2024-03-21T11:29:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f46e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4010f720 con 0x7f6f40104af0 2024-03-21T11:29:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f3f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f40104af0 0x7f6f4019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f3f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f40104af0 0x7f6f4019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46942/0 (socket says 172.21.15.47:46942) 2024-03-21T11:29:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f3f400640 1 -- 172.21.15.47:0/1469526875 learned_addr learned my addr 172.21.15.47:0/1469526875 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f3fe00640 1 --2- 172.21.15.47:0/1469526875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f40103900 0x7f6f40199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f44e00640 1 --2- 172.21.15.47:0/1469526875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f401054b0 0x7f6f401a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f3f400640 1 -- 172.21.15.47:0/1469526875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f40103900 msgr2=0x7f6f40199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f3f400640 1 --2- 172.21.15.47:0/1469526875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f40103900 0x7f6f40199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f3f400640 1 -- 172.21.15.47:0/1469526875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f401054b0 msgr2=0x7f6f401a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f3f400640 1 --2- 172.21.15.47:0/1469526875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f401054b0 0x7f6f401a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.176+0000 7f6f3f400640 1 -- 172.21.15.47:0/1469526875 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f401a17a0 con 0x7f6f40104af0 2024-03-21T11:29:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.177+0000 7f6f3f400640 1 --2- 172.21.15.47:0/1469526875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f40104af0 0x7f6f4019a0c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00e9f0 tx=0x7f6f2c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.177+0000 7f6f3cc00640 1 -- 172.21.15.47:0/1469526875 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c00cd80 con 0x7f6f40104af0 2024-03-21T11:29:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.177+0000 7f6f3cc00640 1 -- 172.21.15.47:0/1469526875 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f2c010910 con 0x7f6f40104af0 2024-03-21T11:29:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.177+0000 7f6f3cc00640 1 -- 172.21.15.47:0/1469526875 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c018990 con 0x7f6f40104af0 2024-03-21T11:29:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.177+0000 7f6f46e00640 1 -- 172.21.15.47:0/1469526875 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f401a1a90 con 0x7f6f40104af0 2024-03-21T11:29:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.177+0000 7f6f46e00640 1 -- 172.21.15.47:0/1469526875 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f401a1f90 con 0x7f6f40104af0 2024-03-21T11:29:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.181+0000 7f6f49bab740 1 Processor -- start 2024-03-21T11:29:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.181+0000 7f6f49bab740 1 -- start start 2024-03-21T11:29:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.182+0000 7f6f49bab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e61e0 0x56034a8e6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.182+0000 7f6f49bab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e6bd0 0x56034a8eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.182+0000 7f6f3cc00640 1 -- 172.21.15.47:0/1469526875 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6f2c010ac0 con 0x7f6f40104af0 2024-03-21T11:29:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.182+0000 7f6f49bab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 0x56034a8ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.182+0000 7f6f49bab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034a8a93b0 con 0x56034a8e6bd0 2024-03-21T11:29:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.182+0000 7f6f49bab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034a84b830 con 0x56034a8ebd40 2024-03-21T11:29:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.182+0000 7f6f49bab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034a7f45d0 con 0x56034a8e61e0 2024-03-21T11:29:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.182+0000 7f6f44e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e61e0 0x56034a8e6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.182+0000 7f6f44e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e61e0 0x56034a8e6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46956/0 (socket says 172.21.15.47:46956) 2024-03-21T11:29:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.182+0000 7f6f44e00640 1 -- 172.21.15.47:0/2815440447 learned_addr learned my addr 172.21.15.47:0/2815440447 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.182+0000 7f6f3f400640 1 --2- 172.21.15.47:0/2815440447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 0x56034a8ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.182+0000 7f6f3fe00640 1 --2- 172.21.15.47:0/2815440447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e6bd0 0x56034a8eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.183+0000 7f6f44e00640 1 -- 172.21.15.47:0/2815440447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 msgr2=0x56034a8ee1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.183+0000 7f6f44e00640 1 --2- 172.21.15.47:0/2815440447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 0x56034a8ee1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.183+0000 7f6f44e00640 1 -- 172.21.15.47:0/2815440447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e6bd0 msgr2=0x56034a8eb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.183+0000 7f6f44e00640 1 --2- 172.21.15.47:0/2815440447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e6bd0 0x56034a8eb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.183+0000 7f6f44e00640 1 -- 172.21.15.47:0/2815440447 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56034a80c2c0 con 0x56034a8e61e0 2024-03-21T11:29:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.183+0000 7f6f44e00640 1 --2- 172.21.15.47:0/2815440447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e61e0 0x56034a8e6600 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009fd0 tx=0x7f6f380118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6512ea78e5752040 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.183+0000 7f6f35600640 1 -- 172.21.15.47:0/2815440447 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38004d20 con 0x56034a8e61e0 2024-03-21T11:29:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.183+0000 7f6f35600640 1 -- 172.21.15.47:0/2815440447 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f38004050 con 0x56034a8e61e0 2024-03-21T11:29:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.183+0000 7f6f35600640 1 -- 172.21.15.47:0/2815440447 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38005220 con 0x56034a8e61e0 2024-03-21T11:29:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.184+0000 7f6f3cc00640 1 --2- 172.21.15.47:0/1469526875 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6f240a3040 0x7f6f240a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.184+0000 7f6f3cc00640 1 -- 172.21.15.47:0/1469526875 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6f2c0cfa40 con 0x7f6f40104af0 2024-03-21T11:29:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.184+0000 7f6f3fe00640 1 --2- 172.21.15.47:0/1469526875 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6f240a3040 0x7f6f240a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.184+0000 7f6f49bab740 1 -- 172.21.15.47:0/2815440447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e61e0 msgr2=0x56034a8e6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.184+0000 7f6f49bab740 1 --2- 172.21.15.47:0/2815440447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e61e0 0x56034a8e6600 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009fd0 tx=0x7f6f380118d0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.184+0000 7f6f49bab740 1 -- 172.21.15.47:0/2815440447 shutdown_connections 2024-03-21T11:29:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.184+0000 7f6f49bab740 1 --2- 172.21.15.47:0/2815440447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 0x56034a8ee1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.184+0000 7f6f49bab740 1 --2- 172.21.15.47:0/2815440447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e6bd0 0x56034a8eb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.184+0000 7f6f49bab740 1 --2- 172.21.15.47:0/2815440447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e61e0 0x56034a8e6600 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.184+0000 7f6f49bab740 1 -- 172.21.15.47:0/2815440447 >> 172.21.15.47:0/2815440447 conn(0x56034a8e1750 msgr2=0x56034a8e1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.184+0000 7f6f49bab740 1 -- 172.21.15.47:0/2815440447 shutdown_connections 2024-03-21T11:29:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.184+0000 7f6f49bab740 1 -- 172.21.15.47:0/2815440447 wait complete. 2024-03-21T11:29:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.185+0000 7f6f3fe00640 1 --2- 172.21.15.47:0/1469526875 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6f240a3040 0x7f6f240a5500 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6f3002c170 tx=0x7f6f3002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.185+0000 7f6f49bab740 1 -- start start 2024-03-21T11:29:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.185+0000 7f6f49bab740 1 Processor -- start 2024-03-21T11:29:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f49bab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e61e0 0x56034a8f0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f49bab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e6bd0 0x56034a8f0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f49bab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 0x56034a8f7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f49bab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034a807a20 con 0x56034a8e61e0 2024-03-21T11:29:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f49bab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034a8478d0 con 0x56034a8ebd40 2024-03-21T11:29:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f49bab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034a8eecf0 con 0x56034a8e6bd0 2024-03-21T11:29:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f49bab740 10 client.0 _ll_register_callbacks cb 0x56034a8441c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f44e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 0x56034a8f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f3fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e61e0 0x56034a8f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f3f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e6bd0 0x56034a8f0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f44e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 0x56034a8f7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55326/0 (socket says 172.21.15.47:55326) 2024-03-21T11:29:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f44e00640 1 -- 172.21.15.47:0/2547416452 learned_addr learned my addr 172.21.15.47:0/2547416452 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f44e00640 1 -- 172.21.15.47:0/2547416452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e6bd0 msgr2=0x56034a8f0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f44e00640 1 --2- 172.21.15.47:0/2547416452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e6bd0 0x56034a8f0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f44e00640 1 -- 172.21.15.47:0/2547416452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e61e0 msgr2=0x56034a8f0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f44e00640 1 --2- 172.21.15.47:0/2547416452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e61e0 0x56034a8f0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.186+0000 7f6f44e00640 1 -- 172.21.15.47:0/2547416452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56034a560210 con 0x56034a8ebd40 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.187+0000 7f6f44e00640 1 --2- 172.21.15.47:0/2547416452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 0x56034a8f7890 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6f3801b040 tx=0x7f6f38004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b777b03147ec487 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.187+0000 7f6f2a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.187+0000 7f6f2a000640 1 -- 172.21.15.47:0/2547416452 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38028020 con 0x56034a8ebd40 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.187+0000 7f6f49bab740 1 -- 172.21.15.47:0/2547416452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f38004050 con 0x56034a8ebd40 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.187+0000 7f6f2a000640 1 -- 172.21.15.47:0/2547416452 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6f38004410 con 0x56034a8ebd40 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.187+0000 7f6f2a000640 1 -- 172.21.15.47:0/2547416452 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f38033050 con 0x56034a8ebd40 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.187+0000 7f6f49bab740 10 client.27458 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.187+0000 7f6f49bab740 1 -- 172.21.15.47:0/2547416452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56034a62c620 con 0x56034a8ebd40 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.187+0000 7f6f2a000640 1 -- 172.21.15.47:0/2547416452 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f38004050 con 0x56034a8ebd40 2024-03-21T11:29:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.188+0000 7f6f2a000640 1 -- 172.21.15.47:0/2547416452 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6f3800f4e0 con 0x56034a8ebd40 2024-03-21T11:29:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.188+0000 7f6f49bab740 10 client.27458 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.188+0000 7f6f49bab740 10 client.27458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.188+0000 7f6f49bab740 20 client.27458 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.188+0000 7f6f49bab740 1 --2- 172.21.15.47:0/2547416452 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56034a8fc010 0x56034a8fe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.188+0000 7f6f49bab740 4 client.27458 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:29:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.188+0000 7f6f49bab740 1 -- 172.21.15.47:0/2547416452 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56034a858d70 con 0x56034a8fc010 2024-03-21T11:29:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.188+0000 7f6f3fe00640 1 --2- 172.21.15.47:0/2547416452 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56034a8fc010 0x56034a8fe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.189+0000 7f6f3fe00640 1 --2- 172.21.15.47:0/2547416452 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56034a8fc010 0x56034a8fe400 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb3be9a4a7e47a7a server_cookie=8a7fb8d264817df5 in_seq=0 out_seq=0 2024-03-21T11:29:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.189+0000 7f6f2a000640 10 client.27458 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.190+0000 7f6f2a000640 1 -- 172.21.15.47:0/2547416452 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56034a858d70 con 0x56034a8fc010 2024-03-21T11:29:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.190+0000 7f6f2a000640 10 client.27458 handle_command_reply: tid=0 2024-03-21T11:29:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.190+0000 7f6f49bab740 1 client.27458 shutdown 2024-03-21T11:29:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.190+0000 7f6f49bab740 1 -- 172.21.15.47:0/2547416452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 msgr2=0x56034a8f7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.190+0000 7f6f49bab740 1 --2- 172.21.15.47:0/2547416452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 0x56034a8f7890 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6f3801b040 tx=0x7f6f38004cf0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.190+0000 7f6f49bab740 1 -- 172.21.15.47:0/2547416452 shutdown_connections 2024-03-21T11:29:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.190+0000 7f6f49bab740 1 --2- 172.21.15.47:0/2547416452 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56034a8fc010 0x56034a8fe400 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.190+0000 7f6f49bab740 1 --2- 172.21.15.47:0/2547416452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 0x56034a8f7890 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.190+0000 7f6f2a000640 0 client.27458 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.190+0000 7f6f49bab740 1 --2- 172.21.15.47:0/2547416452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e6bd0 0x56034a8f0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.190+0000 7f6f49bab740 1 --2- 172.21.15.47:0/2547416452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e61e0 0x56034a8f0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.190+0000 7f6f49bab740 1 -- 172.21.15.47:0/2547416452 >> 172.21.15.47:0/2547416452 conn(0x56034a8e1750 msgr2=0x56034a8e0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.190+0000 7f6f49bab740 1 -- 172.21.15.47:0/2547416452 shutdown_connections 2024-03-21T11:29:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.190+0000 7f6f49bab740 1 -- 172.21.15.47:0/2547416452 wait complete. 2024-03-21T11:29:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.191+0000 7f6f49bab740 20 client.27458 trim_cache size 0 max 16384 2024-03-21T11:29:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.207+0000 7f6f49bab740 1 Processor -- start 2024-03-21T11:29:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.207+0000 7f6f49bab740 1 -- start start 2024-03-21T11:29:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f49bab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e61e0 0x56034a8eb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f49bab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e6bd0 0x56034a8f7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f49bab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 0x56034a8e00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f49bab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f401ac370 con 0x56034a8e6bd0 2024-03-21T11:29:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f49bab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f401adcf0 con 0x56034a8ebd40 2024-03-21T11:29:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f49bab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f401ac9d0 con 0x56034a8e61e0 2024-03-21T11:29:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f3fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e61e0 0x56034a8eb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f3fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e61e0 0x56034a8eb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46976/0 (socket says 172.21.15.47:46976) 2024-03-21T11:29:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f3fe00640 1 -- 172.21.15.47:0/955107231 learned_addr learned my addr 172.21.15.47:0/955107231 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f44e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 0x56034a8e00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f3f400640 1 --2- 172.21.15.47:0/955107231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e6bd0 0x56034a8f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f3fe00640 1 -- 172.21.15.47:0/955107231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 msgr2=0x56034a8e00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f3fe00640 1 --2- 172.21.15.47:0/955107231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 0x56034a8e00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f3fe00640 1 -- 172.21.15.47:0/955107231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e6bd0 msgr2=0x56034a8f7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f3fe00640 1 --2- 172.21.15.47:0/955107231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e6bd0 0x56034a8f7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f3fe00640 1 -- 172.21.15.47:0/955107231 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56034a8fea30 con 0x56034a8e61e0 2024-03-21T11:29:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.208+0000 7f6f3f400640 1 --2- 172.21.15.47:0/955107231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e6bd0 0x56034a8f7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:29:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.209+0000 7f6f3fe00640 1 --2- 172.21.15.47:0/955107231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e61e0 0x56034a8eb4c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6f30000c00 tx=0x7f6f30041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d14704c755fcdf53 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.209+0000 7f6f34c00640 1 -- 172.21.15.47:0/955107231 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f30045070 con 0x56034a8e61e0 2024-03-21T11:29:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.209+0000 7f6f34c00640 1 -- 172.21.15.47:0/955107231 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f30057110 con 0x56034a8e61e0 2024-03-21T11:29:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.209+0000 7f6f34c00640 1 -- 172.21.15.47:0/955107231 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f300035f0 con 0x56034a8e61e0 2024-03-21T11:29:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.210+0000 7f6f49bab740 1 -- 172.21.15.47:0/955107231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e61e0 msgr2=0x56034a8eb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.210+0000 7f6f49bab740 1 --2- 172.21.15.47:0/955107231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e61e0 0x56034a8eb4c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6f30000c00 tx=0x7f6f30041000 comp rx=0 tx=0).stop 2024-03-21T11:29:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.210+0000 7f6f49bab740 1 -- 172.21.15.47:0/955107231 shutdown_connections 2024-03-21T11:29:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.210+0000 7f6f49bab740 1 --2- 172.21.15.47:0/955107231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8ebd40 0x56034a8e00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.210+0000 7f6f49bab740 1 --2- 172.21.15.47:0/955107231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e6bd0 0x56034a8f7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.210+0000 7f6f49bab740 1 --2- 172.21.15.47:0/955107231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8e61e0 0x56034a8eb4c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.210+0000 7f6f49bab740 1 -- 172.21.15.47:0/955107231 >> 172.21.15.47:0/955107231 conn(0x56034a8e1750 msgr2=0x56034a8f0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.210+0000 7f6f49bab740 1 -- 172.21.15.47:0/955107231 shutdown_connections 2024-03-21T11:29:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.210+0000 7f6f49bab740 1 -- 172.21.15.47:0/955107231 wait complete. 2024-03-21T11:29:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.210+0000 7f6f49bab740 1 -- start start 2024-03-21T11:29:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.210+0000 7f6f49bab740 1 Processor -- start 2024-03-21T11:29:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f49bab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e61e0 0x56034a8f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f49bab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8e6bd0 0x56034a8f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f49bab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8ebd40 0x56034a8f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f49bab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f401ad690 con 0x56034a8e61e0 2024-03-21T11:29:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f49bab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f401ad030 con 0x56034a8e6bd0 2024-03-21T11:29:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f49bab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f401aa530 con 0x56034a8ebd40 2024-03-21T11:29:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f49bab740 10 client.0 _ll_register_callbacks cb 0x7f6f2c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f3f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8e6bd0 0x56034a8f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f3f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8e6bd0 0x56034a8f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55338/0 (socket says 172.21.15.47:55338) 2024-03-21T11:29:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f44e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8ebd40 0x56034a8f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f3f400640 1 -- 172.21.15.47:0/547513210 learned_addr learned my addr 172.21.15.47:0/547513210 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f3fe00640 1 --2- 172.21.15.47:0/547513210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e61e0 0x56034a8f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f3f400640 1 -- 172.21.15.47:0/547513210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8ebd40 msgr2=0x56034a8f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f3f400640 1 --2- 172.21.15.47:0/547513210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8ebd40 0x56034a8f6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f3f400640 1 -- 172.21.15.47:0/547513210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e61e0 msgr2=0x56034a8f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f3f400640 1 --2- 172.21.15.47:0/547513210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e61e0 0x56034a8f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f3f400640 1 -- 172.21.15.47:0/547513210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56034a8e10a0 con 0x56034a8e6bd0 2024-03-21T11:29:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.211+0000 7f6f44e00640 1 --2- 172.21.15.47:0/547513210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8ebd40 0x56034a8f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.212+0000 7f6f3f400640 1 --2- 172.21.15.47:0/547513210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8e6bd0 0x56034a8f5a10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c09bb30 tx=0x7f6f2c09baa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=153beed8ce8c6d29 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.212+0000 7f6f2a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:29:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.212+0000 7f6f2a000640 1 -- 172.21.15.47:0/547513210 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0a4790 con 0x56034a8e6bd0 2024-03-21T11:29:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.212+0000 7f6f2a000640 1 -- 172.21.15.47:0/547513210 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6f2c09b280 con 0x56034a8e6bd0 2024-03-21T11:29:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.212+0000 7f6f49bab740 1 -- 172.21.15.47:0/547513210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f30057110 con 0x56034a8e6bd0 2024-03-21T11:29:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.212+0000 7f6f2a000640 1 -- 172.21.15.47:0/547513210 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f2c09b480 con 0x56034a8e6bd0 2024-03-21T11:29:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.212+0000 7f6f49bab740 10 client.27464 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.212+0000 7f6f49bab740 1 -- 172.21.15.47:0/547513210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56034a957bf0 con 0x56034a8e6bd0 2024-03-21T11:29:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.212+0000 7f6f2a000640 1 -- 172.21.15.47:0/547513210 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f30057110 con 0x56034a8e6bd0 2024-03-21T11:29:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.213+0000 7f6f2a000640 1 -- 172.21.15.47:0/547513210 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6f2c0baaf0 con 0x56034a8e6bd0 2024-03-21T11:29:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.213+0000 7f6f49bab740 10 client.27464 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.213+0000 7f6f49bab740 10 client.27464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.213+0000 7f6f49bab740 20 client.27464 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.213+0000 7f6f49bab740 1 --2- 172.21.15.47:0/547513210 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56034a958570 0x56034a95a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.213+0000 7f6f49bab740 4 client.27464 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:29:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.213+0000 7f6f49bab740 1 -- 172.21.15.47:0/547513210 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56034a8dca80 con 0x56034a958570 2024-03-21T11:29:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.213+0000 7f6f3fe00640 1 --2- 172.21.15.47:0/547513210 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56034a958570 0x56034a95a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.214+0000 7f6f3fe00640 1 --2- 172.21.15.47:0/547513210 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56034a958570 0x56034a95a990 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afa2a400fef2aca3 server_cookie=13fda7c0944aa6 in_seq=0 out_seq=0 2024-03-21T11:29:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.214+0000 7f6f2a000640 10 client.27464 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.429+0000 7f6f2a000640 1 -- 172.21.15.47:0/547513210 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56034a8dca80 con 0x56034a958570 2024-03-21T11:29:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.429+0000 7f6f2a000640 10 client.27464 handle_command_reply: tid=0 2024-03-21T11:29:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.429+0000 7f6f49bab740 1 client.27464 shutdown 2024-03-21T11:29:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.430+0000 7f6f49bab740 1 -- 172.21.15.47:0/547513210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8e6bd0 msgr2=0x56034a8f5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.430+0000 7f6f49bab740 1 --2- 172.21.15.47:0/547513210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8e6bd0 0x56034a8f5a10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c09bb30 tx=0x7f6f2c09baa0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.430+0000 7f6f49bab740 1 -- 172.21.15.47:0/547513210 shutdown_connections 2024-03-21T11:29:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.430+0000 7f6f49bab740 1 --2- 172.21.15.47:0/547513210 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56034a958570 0x56034a95a990 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.430+0000 7f6f49bab740 1 --2- 172.21.15.47:0/547513210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56034a8ebd40 0x56034a8f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.430+0000 7f6f49bab740 1 --2- 172.21.15.47:0/547513210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56034a8e6bd0 0x56034a8f5a10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.430+0000 7f6f49bab740 1 --2- 172.21.15.47:0/547513210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56034a8e61e0 0x56034a8f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.430+0000 7f6f49bab740 1 -- 172.21.15.47:0/547513210 >> 172.21.15.47:0/547513210 conn(0x56034a8e1750 msgr2=0x56034a8f01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.430+0000 7f6f2a000640 0 client.27464 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.430+0000 7f6f49bab740 1 -- 172.21.15.47:0/547513210 shutdown_connections 2024-03-21T11:29:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.430+0000 7f6f49bab740 1 -- 172.21.15.47:0/547513210 wait complete. 2024-03-21T11:29:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.430+0000 7f6f49bab740 20 client.27464 trim_cache size 0 max 16384 2024-03-21T11:29:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:29:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:29:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:29:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:29:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.431+0000 7f6f46e00640 1 -- 172.21.15.47:0/1469526875 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6f240a3040 msgr2=0x7f6f240a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.431+0000 7f6f46e00640 1 --2- 172.21.15.47:0/1469526875 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6f240a3040 0x7f6f240a5500 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6f3002c170 tx=0x7f6f3002df90 comp rx=0 tx=0).stop 2024-03-21T11:29:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.431+0000 7f6f46e00640 1 -- 172.21.15.47:0/1469526875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f40104af0 msgr2=0x7f6f4019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.431+0000 7f6f46e00640 1 --2- 172.21.15.47:0/1469526875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f40104af0 0x7f6f4019a0c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00e9f0 tx=0x7f6f2c00eeb0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.432+0000 7f6f46e00640 1 -- 172.21.15.47:0/1469526875 shutdown_connections 2024-03-21T11:29:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.432+0000 7f6f46e00640 1 --2- 172.21.15.47:0/1469526875 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6f240a3040 0x7f6f240a5500 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.432+0000 7f6f46e00640 1 --2- 172.21.15.47:0/1469526875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f401054b0 0x7f6f401a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.432+0000 7f6f46e00640 1 --2- 172.21.15.47:0/1469526875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f40104af0 0x7f6f4019a0c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.432+0000 7f6f46e00640 1 --2- 172.21.15.47:0/1469526875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f40103900 0x7f6f40199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.432+0000 7f6f46e00640 1 -- 172.21.15.47:0/1469526875 >> 172.21.15.47:0/1469526875 conn(0x7f6f400fee90 msgr2=0x7f6f40107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.432+0000 7f6f46e00640 1 -- 172.21.15.47:0/1469526875 shutdown_connections 2024-03-21T11:29:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:10.432+0000 7f6f46e00640 1 -- 172.21.15.47:0/1469526875 wait complete. 2024-03-21T11:29:10.517 INFO:tasks.workunit.client.0.smithi047.stdout: 1 371326 8.30 MB/sec execute 114 sec latency 113.912 ms 2024-03-21T11:29:10.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:10 smithi100 ceph-mon[36245]: pgmap v575: 129 pgs: 129 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 299 op/s 2024-03-21T11:29:10.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:10 smithi102 ceph-mon[36748]: pgmap v575: 129 pgs: 129 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 299 op/s 2024-03-21T11:29:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:10 smithi047 ceph-mon[31351]: pgmap v575: 129 pgs: 129 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 299 op/s 2024-03-21T11:29:11.517 INFO:tasks.workunit.client.0.smithi047.stdout: 1 372811 8.33 MB/sec execute 115 sec latency 12.404 ms 2024-03-21T11:29:12.517 INFO:tasks.workunit.client.0.smithi047.stdout: 1 373903 8.30 MB/sec execute 116 sec latency 20.534 ms 2024-03-21T11:29:12.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:12 smithi102 ceph-mon[36748]: pgmap v576: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 239 op/s 2024-03-21T11:29:13.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:12 smithi047 ceph-mon[31351]: pgmap v576: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 239 op/s 2024-03-21T11:29:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:12 smithi100 ceph-mon[36245]: pgmap v576: 129 pgs: 129 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 239 op/s 2024-03-21T11:29:13.517 INFO:tasks.workunit.client.0.smithi047.stdout: 1 375626 8.30 MB/sec execute 117 sec latency 44.519 ms 2024-03-21T11:29:14.517 INFO:tasks.workunit.client.0.smithi047.stdout: 1 376714 8.28 MB/sec execute 118 sec latency 13.670 ms 2024-03-21T11:29:14.945 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:14 smithi100 ceph-mon[36245]: pgmap v577: 129 pgs: 129 active+clean; 33 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 313 op/s 2024-03-21T11:29:14.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:14 smithi102 ceph-mon[36748]: pgmap v577: 129 pgs: 129 active+clean; 33 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 313 op/s 2024-03-21T11:29:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:14 smithi047 ceph-mon[31351]: pgmap v577: 129 pgs: 129 active+clean; 33 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 313 op/s 2024-03-21T11:29:15.517 INFO:tasks.workunit.client.0.smithi047.stdout: 1 378481 8.27 MB/sec execute 119 sec latency 13.971 ms 2024-03-21T11:29:16.518 INFO:tasks.workunit.client.0.smithi047.stdout: 1 379946 8.30 MB/sec execute 120 sec latency 30.380 ms 2024-03-21T11:29:16.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:16 smithi102 ceph-mon[36748]: pgmap v578: 129 pgs: 129 active+clean; 33 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.8 MiB/s wr, 257 op/s 2024-03-21T11:29:17.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:16 smithi047 ceph-mon[31351]: pgmap v578: 129 pgs: 129 active+clean; 33 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.8 MiB/s wr, 257 op/s 2024-03-21T11:29:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:16 smithi100 ceph-mon[36245]: pgmap v578: 129 pgs: 129 active+clean; 33 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.8 MiB/s wr, 257 op/s 2024-03-21T11:29:17.518 INFO:tasks.workunit.client.0.smithi047.stdout: 1 381015 8.27 MB/sec execute 121 sec latency 66.524 ms 2024-03-21T11:29:18.518 INFO:tasks.workunit.client.0.smithi047.stdout: 1 382920 8.28 MB/sec execute 122 sec latency 61.337 ms 2024-03-21T11:29:18.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:18 smithi102 ceph-mon[36748]: pgmap v579: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 318 op/s 2024-03-21T11:29:19.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:18 smithi047 ceph-mon[31351]: pgmap v579: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 318 op/s 2024-03-21T11:29:19.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:18 smithi100 ceph-mon[36245]: pgmap v579: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 318 op/s 2024-03-21T11:29:19.518 INFO:tasks.workunit.client.0.smithi047.stdout: 1 383988 8.26 MB/sec execute 123 sec latency 10.752 ms 2024-03-21T11:29:20.518 INFO:tasks.workunit.client.0.smithi047.stdout: 1 386101 8.29 MB/sec execute 124 sec latency 13.098 ms 2024-03-21T11:29:20.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:20 smithi102 ceph-mon[36748]: pgmap v580: 129 pgs: 129 active+clean; 33 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2024-03-21T11:29:21.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:20 smithi047 ceph-mon[31351]: pgmap v580: 129 pgs: 129 active+clean; 33 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2024-03-21T11:29:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:20 smithi100 ceph-mon[36245]: pgmap v580: 129 pgs: 129 active+clean; 33 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 316 op/s 2024-03-21T11:29:21.518 INFO:tasks.workunit.client.0.smithi047.stdout: 1 387358 8.28 MB/sec execute 125 sec latency 10.038 ms 2024-03-21T11:29:22.518 INFO:tasks.workunit.client.0.smithi047.stdout: 1 389154 8.27 MB/sec execute 126 sec latency 22.315 ms 2024-03-21T11:29:22.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:22 smithi102 ceph-mon[36748]: pgmap v581: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-03-21T11:29:23.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:22 smithi047 ceph-mon[31351]: pgmap v581: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-03-21T11:29:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:22 smithi100 ceph-mon[36245]: pgmap v581: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-03-21T11:29:23.519 INFO:tasks.workunit.client.0.smithi047.stdout: 1 390610 8.29 MB/sec execute 127 sec latency 44.510 ms 2024-03-21T11:29:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.615+0000 7f35b6e00640 1 -- 172.21.15.47:0/2834634291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b01028c0 msgr2=0x7f35b0102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.615+0000 7f35b6e00640 1 --2- 172.21.15.47:0/2834634291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b01028c0 0x7f35b0102d40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f359c009900 tx=0x7f359c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:29:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.615+0000 7f35b6e00640 1 -- 172.21.15.47:0/2834634291 shutdown_connections 2024-03-21T11:29:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.615+0000 7f35b6e00640 1 --2- 172.21.15.47:0/2834634291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b0103280 0x7f35b0109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.615+0000 7f35b6e00640 1 --2- 172.21.15.47:0/2834634291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b01028c0 0x7f35b0102d40 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.615+0000 7f35b6e00640 1 --2- 172.21.15.47:0/2834634291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35b01016d0 0x7f35b0101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.615+0000 7f35b6e00640 1 -- 172.21.15.47:0/2834634291 >> 172.21.15.47:0/2834634291 conn(0x7f35b00fcc60 msgr2=0x7f35b00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.615+0000 7f35b6e00640 1 -- 172.21.15.47:0/2834634291 shutdown_connections 2024-03-21T11:29:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.615+0000 7f35b6e00640 1 -- 172.21.15.47:0/2834634291 wait complete. 2024-03-21T11:29:23.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.616+0000 7f35b6e00640 1 Processor -- start 2024-03-21T11:29:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.617+0000 7f35b6e00640 1 -- start start 2024-03-21T11:29:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.617+0000 7f35b6e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35b01016d0 0x7f35b01978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.617+0000 7f35b6e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b01028c0 0x7f35b0197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.617+0000 7f35b6e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b0103280 0x7f35b019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.617+0000 7f35b6e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b010d3a0 con 0x7f35b01016d0 2024-03-21T11:29:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.617+0000 7f35b6e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b010d220 con 0x7f35b0103280 2024-03-21T11:29:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.617+0000 7f35b6e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b010d520 con 0x7f35b01028c0 2024-03-21T11:29:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.617+0000 7f35afe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35b01016d0 0x7f35b01978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.617+0000 7f35b4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b0103280 0x7f35b019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.617+0000 7f35afe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35b01016d0 0x7f35b01978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60678/0 (socket says 172.21.15.47:60678) 2024-03-21T11:29:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.617+0000 7f35af400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b01028c0 0x7f35b0197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.617+0000 7f35afe00640 1 -- 172.21.15.47:0/3632621629 learned_addr learned my addr 172.21.15.47:0/3632621629 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.618+0000 7f35afe00640 1 -- 172.21.15.47:0/3632621629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b01028c0 msgr2=0x7f35b0197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.618+0000 7f35afe00640 1 --2- 172.21.15.47:0/3632621629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b01028c0 0x7f35b0197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.618+0000 7f35afe00640 1 -- 172.21.15.47:0/3632621629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b0103280 msgr2=0x7f35b019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.618+0000 7f35afe00640 1 --2- 172.21.15.47:0/3632621629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b0103280 0x7f35b019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.618+0000 7f35afe00640 1 -- 172.21.15.47:0/3632621629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b019f510 con 0x7f35b01016d0 2024-03-21T11:29:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.618+0000 7f35afe00640 1 --2- 172.21.15.47:0/3632621629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35b01016d0 0x7f35b01978f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f35a000e9c0 tx=0x7f35a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.618+0000 7f35acc00640 1 -- 172.21.15.47:0/3632621629 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a000cd80 con 0x7f35b01016d0 2024-03-21T11:29:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.618+0000 7f35b6e00640 1 -- 172.21.15.47:0/3632621629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b0073930 con 0x7f35b01016d0 2024-03-21T11:29:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.619+0000 7f35acc00640 1 -- 172.21.15.47:0/3632621629 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35a00035f0 con 0x7f35b01016d0 2024-03-21T11:29:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.619+0000 7f35acc00640 1 -- 172.21.15.47:0/3632621629 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0010690 con 0x7f35b01016d0 2024-03-21T11:29:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.619+0000 7f35b6e00640 1 -- 172.21.15.47:0/3632621629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b0073ea0 con 0x7f35b01016d0 2024-03-21T11:29:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.622+0000 7f35b9bf9740 1 Processor -- start 2024-03-21T11:29:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.622+0000 7f35acc00640 1 -- 172.21.15.47:0/3632621629 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f35a00037a0 con 0x7f35b01016d0 2024-03-21T11:29:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.622+0000 7f35b9bf9740 1 -- start start 2024-03-21T11:29:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.622+0000 7f35b9bf9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711ab8a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.622+0000 7f35b9bf9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8abd0 0x55711ab8f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.622+0000 7f35b9bf9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711ab921d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.622+0000 7f35b9bf9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711ab4d3b0 con 0x55711ab8a1e0 2024-03-21T11:29:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.622+0000 7f35b9bf9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711aaef830 con 0x55711ab8abd0 2024-03-21T11:29:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.622+0000 7f35b9bf9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711aa985d0 con 0x55711ab8fd40 2024-03-21T11:29:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.623+0000 7f35af400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711ab921d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.623+0000 7f35b4e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711ab8a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.623+0000 7f35afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8abd0 0x55711ab8f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.623+0000 7f35af400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711ab921d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49228/0 (socket says 172.21.15.47:49228) 2024-03-21T11:29:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.623+0000 7f35af400640 1 -- 172.21.15.47:0/2020010846 learned_addr learned my addr 172.21.15.47:0/2020010846 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.623+0000 7f35af400640 1 -- 172.21.15.47:0/2020010846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8abd0 msgr2=0x55711ab8f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.623+0000 7f35af400640 1 --2- 172.21.15.47:0/2020010846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8abd0 0x55711ab8f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.623+0000 7f35af400640 1 -- 172.21.15.47:0/2020010846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 msgr2=0x55711ab8a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.623+0000 7f35af400640 1 --2- 172.21.15.47:0/2020010846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711ab8a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.623+0000 7f35af400640 1 -- 172.21.15.47:0/2020010846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55711aab02c0 con 0x55711ab8fd40 2024-03-21T11:29:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.623+0000 7f35af400640 1 --2- 172.21.15.47:0/2020010846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711ab921d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f359c000c00 tx=0x7f359c02df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38f298f09a6404bb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.624+0000 7f35a5600640 1 -- 172.21.15.47:0/2020010846 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c02cb80 con 0x55711ab8fd40 2024-03-21T11:29:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.624+0000 7f35acc00640 1 --2- 172.21.15.47:0/3632621629 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35940a2fc0 0x7f35940a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.624+0000 7f35a5600640 1 -- 172.21.15.47:0/2020010846 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f359c02cd20 con 0x55711ab8fd40 2024-03-21T11:29:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.624+0000 7f35a5600640 1 -- 172.21.15.47:0/2020010846 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c005110 con 0x55711ab8fd40 2024-03-21T11:29:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.624+0000 7f35acc00640 1 -- 172.21.15.47:0/3632621629 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f35a00cfb90 con 0x7f35b01016d0 2024-03-21T11:29:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.624+0000 7f35b4e00640 1 --2- 172.21.15.47:0/3632621629 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35940a2fc0 0x7f35940a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.625+0000 7f35b9bf9740 1 -- 172.21.15.47:0/2020010846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 msgr2=0x55711ab921d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.625+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/2020010846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711ab921d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f359c000c00 tx=0x7f359c02df90 comp rx=0 tx=0).stop 2024-03-21T11:29:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.625+0000 7f35b4e00640 1 --2- 172.21.15.47:0/3632621629 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35940a2fc0 0x7f35940a5480 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55711ab8b430 tx=0x7f35a8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.625+0000 7f35b9bf9740 1 -- 172.21.15.47:0/2020010846 shutdown_connections 2024-03-21T11:29:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.625+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/2020010846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711ab921d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.625+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/2020010846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8abd0 0x55711ab8f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.625+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/2020010846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711ab8a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.625+0000 7f35b9bf9740 1 -- 172.21.15.47:0/2020010846 >> 172.21.15.47:0/2020010846 conn(0x55711ab85750 msgr2=0x55711ab85b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.625+0000 7f35b9bf9740 1 -- 172.21.15.47:0/2020010846 shutdown_connections 2024-03-21T11:29:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.625+0000 7f35b9bf9740 1 -- 172.21.15.47:0/2020010846 wait complete. 2024-03-21T11:29:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.625+0000 7f35b9bf9740 1 -- start start 2024-03-21T11:29:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.626+0000 7f35b9bf9740 1 Processor -- start 2024-03-21T11:29:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.626+0000 7f35b9bf9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8a1e0 0x55711aba5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.626+0000 7f35b9bf9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8abd0 0x55711aba5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.626+0000 7f35b9bf9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711abac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.626+0000 7f35b9bf9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711aaaba20 con 0x55711ab8abd0 2024-03-21T11:29:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.626+0000 7f35b9bf9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711aaeb8d0 con 0x55711ab8a1e0 2024-03-21T11:29:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.626+0000 7f35b9bf9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711aba3ca0 con 0x55711ab8fd40 2024-03-21T11:29:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.626+0000 7f35b9bf9740 10 client.0 _ll_register_callbacks cb 0x55711aae81c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.626+0000 7f35b4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711abac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.626+0000 7f35b4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711abac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49230/0 (socket says 172.21.15.47:49230) 2024-03-21T11:29:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.626+0000 7f35b4e00640 1 -- 172.21.15.47:0/1804003081 learned_addr learned my addr 172.21.15.47:0/1804003081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.627+0000 7f35afe00640 1 --2- 172.21.15.47:0/1804003081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8a1e0 0x55711aba5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.627+0000 7f35b4e00640 1 -- 172.21.15.47:0/1804003081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8a1e0 msgr2=0x55711aba5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.627+0000 7f35b4e00640 1 --2- 172.21.15.47:0/1804003081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8a1e0 0x55711aba5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.627+0000 7f35af400640 1 --2- 172.21.15.47:0/1804003081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8abd0 0x55711aba5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.627+0000 7f35b4e00640 1 -- 172.21.15.47:0/1804003081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8abd0 msgr2=0x55711aba5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.627+0000 7f35b4e00640 1 --2- 172.21.15.47:0/1804003081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8abd0 0x55711aba5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.627+0000 7f35b4e00640 1 -- 172.21.15.47:0/1804003081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55711a804210 con 0x55711ab8fd40 2024-03-21T11:29:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.627+0000 7f35af400640 1 --2- 172.21.15.47:0/1804003081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8abd0 0x55711aba5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:29:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.627+0000 7f35b4e00640 1 --2- 172.21.15.47:0/1804003081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711abac8a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009cd0 tx=0x7f35a8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36dfeca172d1c6c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.627+0000 7f359a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:29:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.627+0000 7f359a000640 1 -- 172.21.15.47:0/1804003081 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a800f830 con 0x55711ab8fd40 2024-03-21T11:29:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.628+0000 7f359a000640 1 -- 172.21.15.47:0/1804003081 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f35a800f9d0 con 0x55711ab8fd40 2024-03-21T11:29:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.628+0000 7f35b9bf9740 1 -- 172.21.15.47:0/1804003081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55711aafcd70 con 0x55711ab8fd40 2024-03-21T11:29:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.628+0000 7f359a000640 1 -- 172.21.15.47:0/1804003081 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a800fbd0 con 0x55711ab8fd40 2024-03-21T11:29:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.628+0000 7f35b9bf9740 10 client.27499 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.628+0000 7f35b9bf9740 1 -- 172.21.15.47:0/1804003081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55711a8d0620 con 0x55711ab8fd40 2024-03-21T11:29:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.628+0000 7f359a000640 1 -- 172.21.15.47:0/1804003081 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55711aafcd70 con 0x55711ab8fd40 2024-03-21T11:29:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.628+0000 7f359a000640 1 -- 172.21.15.47:0/1804003081 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f35a8026070 con 0x55711ab8fd40 2024-03-21T11:29:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.628+0000 7f35b9bf9740 10 client.27499 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.628+0000 7f35b9bf9740 10 client.27499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.628+0000 7f35b9bf9740 20 client.27499 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.628+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/1804003081 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55711abb1270 0x55711abb36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.628+0000 7f35b9bf9740 4 client.27499 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:29:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.628+0000 7f35b9bf9740 1 -- 172.21.15.47:0/1804003081 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55711aaf70c0 con 0x55711abb1270 2024-03-21T11:29:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.629+0000 7f35afe00640 1 --2- 172.21.15.47:0/1804003081 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55711abb1270 0x55711abb36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.629+0000 7f35afe00640 1 --2- 172.21.15.47:0/1804003081 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55711abb1270 0x55711abb36c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d44ecbe3b14576e server_cookie=9285fe50ebe15841 in_seq=0 out_seq=0 2024-03-21T11:29:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.629+0000 7f359a000640 10 client.27499 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.631+0000 7f359a000640 1 -- 172.21.15.47:0/1804003081 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55711aaf70c0 con 0x55711abb1270 2024-03-21T11:29:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.631+0000 7f359a000640 10 client.27499 handle_command_reply: tid=0 2024-03-21T11:29:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.631+0000 7f35b9bf9740 1 client.27499 shutdown 2024-03-21T11:29:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.631+0000 7f35b9bf9740 1 -- 172.21.15.47:0/1804003081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 msgr2=0x55711abac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.631+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/1804003081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711abac8a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009cd0 tx=0x7f35a8004ab0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.631+0000 7f35b9bf9740 1 -- 172.21.15.47:0/1804003081 shutdown_connections 2024-03-21T11:29:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.631+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/1804003081 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55711abb1270 0x55711abb36c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.631+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/1804003081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711abac8a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.631+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/1804003081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8abd0 0x55711aba5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.631+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/1804003081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8a1e0 0x55711aba5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.631+0000 7f35b9bf9740 1 -- 172.21.15.47:0/1804003081 >> 172.21.15.47:0/1804003081 conn(0x55711ab85750 msgr2=0x55711ab84950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.631+0000 7f359a000640 0 client.27499 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.632+0000 7f35b9bf9740 1 -- 172.21.15.47:0/1804003081 shutdown_connections 2024-03-21T11:29:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.632+0000 7f35b9bf9740 1 -- 172.21.15.47:0/1804003081 wait complete. 2024-03-21T11:29:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.632+0000 7f35b9bf9740 20 client.27499 trim_cache size 0 max 16384 2024-03-21T11:29:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.647+0000 7f35b9bf9740 1 Processor -- start 2024-03-21T11:29:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.647+0000 7f35b9bf9740 1 -- start start 2024-03-21T11:29:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.647+0000 7f35b9bf9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711abb8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.647+0000 7f35b9bf9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8abd0 0x55711abac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.647+0000 7f35b9bf9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711abb8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.647+0000 7f35b9bf9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b019f6e0 con 0x55711ab8a1e0 2024-03-21T11:29:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.647+0000 7f35b9bf9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b01a0f70 con 0x55711ab8abd0 2024-03-21T11:29:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.647+0000 7f35b9bf9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b019fc50 con 0x55711ab8fd40 2024-03-21T11:29:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.647+0000 7f35afe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711abb8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.648+0000 7f35afe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711abb8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60704/0 (socket says 172.21.15.47:60704) 2024-03-21T11:29:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.648+0000 7f35afe00640 1 -- 172.21.15.47:0/1463216160 learned_addr learned my addr 172.21.15.47:0/1463216160 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.648+0000 7f35af400640 1 --2- 172.21.15.47:0/1463216160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8abd0 0x55711abac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.648+0000 7f35b4e00640 1 --2- 172.21.15.47:0/1463216160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711abb8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.648+0000 7f35afe00640 1 -- 172.21.15.47:0/1463216160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 msgr2=0x55711abb8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.648+0000 7f35afe00640 1 --2- 172.21.15.47:0/1463216160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711abb8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.648+0000 7f35afe00640 1 -- 172.21.15.47:0/1463216160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8abd0 msgr2=0x55711abac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.648+0000 7f35afe00640 1 --2- 172.21.15.47:0/1463216160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8abd0 0x55711abac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.648+0000 7f35afe00640 1 -- 172.21.15.47:0/1463216160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55711ab8b460 con 0x55711ab8a1e0 2024-03-21T11:29:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.648+0000 7f35afe00640 1 --2- 172.21.15.47:0/1463216160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711abb8010 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f35a00988a0 tx=0x7f35a00097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7671ab616790306c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.648+0000 7f35a4c00640 1 -- 172.21.15.47:0/1463216160 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a00d5020 con 0x55711ab8a1e0 2024-03-21T11:29:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.649+0000 7f35a4c00640 1 -- 172.21.15.47:0/1463216160 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35a00bc0d0 con 0x55711ab8a1e0 2024-03-21T11:29:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.649+0000 7f35a4c00640 1 -- 172.21.15.47:0/1463216160 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a00cd3b0 con 0x55711ab8a1e0 2024-03-21T11:29:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.649+0000 7f35b9bf9740 1 -- 172.21.15.47:0/1463216160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 msgr2=0x55711abb8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.649+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/1463216160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711abb8010 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f35a00988a0 tx=0x7f35a00097e0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.649+0000 7f35b9bf9740 1 -- 172.21.15.47:0/1463216160 shutdown_connections 2024-03-21T11:29:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.649+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/1463216160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8fd40 0x55711abb8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.649+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/1463216160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8abd0 0x55711abac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.649+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/1463216160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711abb8010 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.649+0000 7f35b9bf9740 1 -- 172.21.15.47:0/1463216160 >> 172.21.15.47:0/1463216160 conn(0x55711ab85750 msgr2=0x55711ab889b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.649+0000 7f35b9bf9740 1 -- 172.21.15.47:0/1463216160 shutdown_connections 2024-03-21T11:29:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.649+0000 7f35b9bf9740 1 -- 172.21.15.47:0/1463216160 wait complete. 2024-03-21T11:29:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.650+0000 7f35b9bf9740 1 -- start start 2024-03-21T11:29:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.650+0000 7f35b9bf9740 1 Processor -- start 2024-03-21T11:29:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.650+0000 7f35b9bf9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711abaa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.650+0000 7f35b9bf9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8abd0 0x55711abaacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.650+0000 7f35b9bf9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8fd40 0x55711ababcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.650+0000 7f35b9bf9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b01a0910 con 0x55711ab8a1e0 2024-03-21T11:29:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.650+0000 7f35b9bf9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b01a02b0 con 0x55711ab8fd40 2024-03-21T11:29:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.650+0000 7f35b9bf9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b0071a00 con 0x55711ab8abd0 2024-03-21T11:29:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f35b9bf9740 10 client.0 _ll_register_callbacks cb 0x7f35a8009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f35afe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711abaa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f35afe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711abaa7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60712/0 (socket says 172.21.15.47:60712) 2024-03-21T11:29:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f35afe00640 1 -- 172.21.15.47:0/3615717058 learned_addr learned my addr 172.21.15.47:0/3615717058 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f35b4e00640 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8fd40 0x55711ababcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f35afe00640 1 -- 172.21.15.47:0/3615717058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8abd0 msgr2=0x55711abaacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f35af400640 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8abd0 0x55711abaacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f35afe00640 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8abd0 0x55711abaacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f35afe00640 1 -- 172.21.15.47:0/3615717058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8fd40 msgr2=0x55711ababcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f35afe00640 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8fd40 0x55711ababcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f35afe00640 1 -- 172.21.15.47:0/3615717058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55711abac2a0 con 0x55711ab8a1e0 2024-03-21T11:29:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f35af400640 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8abd0 0x55711abaacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:29:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f35b4e00640 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8fd40 0x55711ababcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:29:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f35afe00640 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711abaa7b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f35a000c6a0 tx=0x7f35a000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23084363dae00c7e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.651+0000 7f359a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:29:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.652+0000 7f359a000640 1 -- 172.21.15.47:0/3615717058 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a00b4030 con 0x55711ab8a1e0 2024-03-21T11:29:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.652+0000 7f359a000640 1 -- 172.21.15.47:0/3615717058 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f35a0003b30 con 0x55711ab8a1e0 2024-03-21T11:29:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.652+0000 7f35b9bf9740 1 -- 172.21.15.47:0/3615717058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35a00bc0d0 con 0x55711ab8a1e0 2024-03-21T11:29:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.652+0000 7f359a000640 1 -- 172.21.15.47:0/3615717058 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a00d5070 con 0x55711ab8a1e0 2024-03-21T11:29:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.652+0000 7f35b9bf9740 10 client.17217 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.652+0000 7f35b9bf9740 1 -- 172.21.15.47:0/3615717058 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55711ac0d3d0 con 0x55711ab8a1e0 2024-03-21T11:29:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.652+0000 7f359a000640 1 -- 172.21.15.47:0/3615717058 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35a00bc0d0 con 0x55711ab8a1e0 2024-03-21T11:29:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.652+0000 7f359a000640 1 -- 172.21.15.47:0/3615717058 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f35a009b320 con 0x55711ab8a1e0 2024-03-21T11:29:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.653+0000 7f35b9bf9740 10 client.17217 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.653+0000 7f35b9bf9740 10 client.17217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.653+0000 7f35b9bf9740 20 client.17217 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.653+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55711ac0dd60 0x55711ac10180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.653+0000 7f35b9bf9740 4 client.17217 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:29:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.653+0000 7f35b9bf9740 1 -- 172.21.15.47:0/3615717058 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55711aa755d0 con 0x55711ac0dd60 2024-03-21T11:29:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.653+0000 7f35af400640 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55711ac0dd60 0x55711ac10180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.653+0000 7f35af400640 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55711ac0dd60 0x55711ac10180 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe641d10c3faa9f server_cookie=27939bcd7965ddcc in_seq=0 out_seq=0 2024-03-21T11:29:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.653+0000 7f359a000640 10 client.17217 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f359a000640 1 -- 172.21.15.47:0/3615717058 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55711aa755d0 con 0x55711ac0dd60 2024-03-21T11:29:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f359a000640 10 client.17217 handle_command_reply: tid=0 2024-03-21T11:29:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f35b9bf9740 1 client.17217 shutdown 2024-03-21T11:29:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f35b9bf9740 1 -- 172.21.15.47:0/3615717058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 msgr2=0x55711abaa7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711abaa7b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f35a000c6a0 tx=0x7f35a000c840 comp rx=0 tx=0).stop 2024-03-21T11:29:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f35b9bf9740 1 -- 172.21.15.47:0/3615717058 shutdown_connections 2024-03-21T11:29:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55711ac0dd60 0x55711ac10180 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55711ab8fd40 0x55711ababcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55711ab8abd0 0x55711abaacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f35b9bf9740 1 --2- 172.21.15.47:0/3615717058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55711ab8a1e0 0x55711abaa7b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f35b9bf9740 1 -- 172.21.15.47:0/3615717058 >> 172.21.15.47:0/3615717058 conn(0x55711ab85750 msgr2=0x55711ab840c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f359a000640 0 client.17217 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f35b9bf9740 1 -- 172.21.15.47:0/3615717058 shutdown_connections 2024-03-21T11:29:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f35b9bf9740 1 -- 172.21.15.47:0/3615717058 wait complete. 2024-03-21T11:29:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.876+0000 7f35b9bf9740 20 client.17217 trim_cache size 0 max 16384 2024-03-21T11:29:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:29:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:29:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:29:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:29:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.877+0000 7f35b6e00640 1 -- 172.21.15.47:0/3632621629 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35940a2fc0 msgr2=0x7f35940a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.877+0000 7f35b6e00640 1 --2- 172.21.15.47:0/3632621629 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35940a2fc0 0x7f35940a5480 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55711ab8b430 tx=0x7f35a8009500 comp rx=0 tx=0).stop 2024-03-21T11:29:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.877+0000 7f35b6e00640 1 -- 172.21.15.47:0/3632621629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35b01016d0 msgr2=0x7f35b01978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.877+0000 7f35b6e00640 1 --2- 172.21.15.47:0/3632621629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35b01016d0 0x7f35b01978f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f35a000e9c0 tx=0x7f35a000ee80 comp rx=0 tx=0).stop 2024-03-21T11:29:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.877+0000 7f35b6e00640 1 -- 172.21.15.47:0/3632621629 shutdown_connections 2024-03-21T11:29:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.877+0000 7f35b6e00640 1 --2- 172.21.15.47:0/3632621629 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35940a2fc0 0x7f35940a5480 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.877+0000 7f35b6e00640 1 --2- 172.21.15.47:0/3632621629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35b0103280 0x7f35b019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.877+0000 7f35b6e00640 1 --2- 172.21.15.47:0/3632621629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b01028c0 0x7f35b0197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.877+0000 7f35b6e00640 1 --2- 172.21.15.47:0/3632621629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35b01016d0 0x7f35b01978f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.877+0000 7f35b6e00640 1 -- 172.21.15.47:0/3632621629 >> 172.21.15.47:0/3632621629 conn(0x7f35b00fcc60 msgr2=0x7f35b0105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.878+0000 7f35b6e00640 1 -- 172.21.15.47:0/3632621629 shutdown_connections 2024-03-21T11:29:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:23.878+0000 7f35b6e00640 1 -- 172.21.15.47:0/3632621629 wait complete. 2024-03-21T11:29:24.519 INFO:tasks.workunit.client.0.smithi047.stdout: 1 391749 8.27 MB/sec execute 128 sec latency 31.073 ms 2024-03-21T11:29:24.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:24 smithi102 ceph-mon[36748]: pgmap v582: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 297 op/s 2024-03-21T11:29:25.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:24 smithi047 ceph-mon[31351]: pgmap v582: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 297 op/s 2024-03-21T11:29:25.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:24 smithi100 ceph-mon[36245]: pgmap v582: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 297 op/s 2024-03-21T11:29:25.519 INFO:tasks.workunit.client.0.smithi047.stdout: 1 393883 8.30 MB/sec execute 129 sec latency 12.377 ms 2024-03-21T11:29:26.519 INFO:tasks.workunit.client.0.smithi047.stdout: 1 394891 8.26 MB/sec execute 130 sec latency 21.281 ms 2024-03-21T11:29:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:26 smithi102 ceph-mon[36748]: pgmap v583: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-03-21T11:29:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:26 smithi047 ceph-mon[31351]: pgmap v583: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-03-21T11:29:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:26 smithi100 ceph-mon[36245]: pgmap v583: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-03-21T11:29:27.519 INFO:tasks.workunit.client.0.smithi047.stdout: 1 397085 8.30 MB/sec execute 131 sec latency 11.119 ms 2024-03-21T11:29:28.520 INFO:tasks.workunit.client.0.smithi047.stdout: 1 397976 8.27 MB/sec execute 132 sec latency 35.086 ms 2024-03-21T11:29:28.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:29:28.920 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:29:28.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:28 smithi102 ceph-mon[36748]: pgmap v584: 129 pgs: 129 active+clean; 34 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 285 op/s 2024-03-21T11:29:29.009 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:28 smithi047 ceph-mon[31351]: pgmap v584: 129 pgs: 129 active+clean; 34 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 285 op/s 2024-03-21T11:29:29.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:28 smithi100 ceph-mon[36245]: pgmap v584: 129 pgs: 129 active+clean; 34 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 285 op/s 2024-03-21T11:29:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.159+0000 7f7554a00640 1 -- 172.21.15.47:0/1950336563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7550102060 msgr2=0x7f7550104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.159+0000 7f7554a00640 1 --2- 172.21.15.47:0/1950336563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7550102060 0x7f7550104450 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7540009900 tx=0x7f754002bc30 comp rx=0 tx=0).stop 2024-03-21T11:29:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.159+0000 7f7554a00640 1 -- 172.21.15.47:0/1950336563 shutdown_connections 2024-03-21T11:29:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.160+0000 7f7554a00640 1 --2- 172.21.15.47:0/1950336563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75501072c0 0x7f75501096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.160+0000 7f7554a00640 1 --2- 172.21.15.47:0/1950336563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7550104990 0x7f7550106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.160+0000 7f7554a00640 1 --2- 172.21.15.47:0/1950336563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7550102060 0x7f7550104450 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.160+0000 7f7554a00640 1 -- 172.21.15.47:0/1950336563 >> 172.21.15.47:0/1950336563 conn(0x7f75500fba10 msgr2=0x7f75500fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:29.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.160+0000 7f7554a00640 1 -- 172.21.15.47:0/1950336563 shutdown_connections 2024-03-21T11:29:29.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.160+0000 7f7554a00640 1 -- 172.21.15.47:0/1950336563 wait complete. 2024-03-21T11:29:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.161+0000 7f7554a00640 1 Processor -- start 2024-03-21T11:29:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.161+0000 7f7554a00640 1 -- start start 2024-03-21T11:29:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.161+0000 7f7554a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550102060 0x7f75501019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.161+0000 7f7554a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7550104990 0x7f7550100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.161+0000 7f7554a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75501072c0 0x7f7550100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.161+0000 7f7554a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755010d260 con 0x7f75501072c0 2024-03-21T11:29:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.161+0000 7f7554a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755010d0e0 con 0x7f7550102060 2024-03-21T11:29:29.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.161+0000 7f7554a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755010d3e0 con 0x7f7550104990 2024-03-21T11:29:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.161+0000 7f754ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7550104990 0x7f7550100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.161+0000 7f754f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550102060 0x7f75501019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.161+0000 7f754ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7550104990 0x7f7550100010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45312/0 (socket says 172.21.15.47:45312) 2024-03-21T11:29:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.161+0000 7f754ea00640 1 -- 172.21.15.47:0/1209372760 learned_addr learned my addr 172.21.15.47:0/1209372760 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.162+0000 7f754fe00640 1 --2- 172.21.15.47:0/1209372760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75501072c0 0x7f7550100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.162+0000 7f754f400640 1 -- 172.21.15.47:0/1209372760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7550104990 msgr2=0x7f7550100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.162+0000 7f754f400640 1 --2- 172.21.15.47:0/1209372760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7550104990 0x7f7550100010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.162+0000 7f754f400640 1 -- 172.21.15.47:0/1209372760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75501072c0 msgr2=0x7f7550100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.162+0000 7f754f400640 1 --2- 172.21.15.47:0/1209372760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75501072c0 0x7f7550100550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.162+0000 7f754f400640 1 -- 172.21.15.47:0/1209372760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7550100de0 con 0x7f7550102060 2024-03-21T11:29:29.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.162+0000 7f754ea00640 1 --2- 172.21.15.47:0/1209372760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7550104990 0x7f7550100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.162+0000 7f754f400640 1 --2- 172.21.15.47:0/1209372760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550102060 0x7f75501019d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7540036040 tx=0x7f754002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.163+0000 7f7547e00640 1 -- 172.21.15.47:0/1209372760 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540043020 con 0x7f7550102060 2024-03-21T11:29:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.163+0000 7f7547e00640 1 -- 172.21.15.47:0/1209372760 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75400054e0 con 0x7f7550102060 2024-03-21T11:29:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.163+0000 7f7547e00640 1 -- 172.21.15.47:0/1209372760 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540003ca0 con 0x7f7550102060 2024-03-21T11:29:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.163+0000 7f7554a00640 1 -- 172.21.15.47:0/1209372760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75501a19d0 con 0x7f7550102060 2024-03-21T11:29:29.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.163+0000 7f7554a00640 1 -- 172.21.15.47:0/1209372760 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75501a1ea0 con 0x7f7550102060 2024-03-21T11:29:29.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.166+0000 7f7547e00640 1 -- 172.21.15.47:0/1209372760 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7540046020 con 0x7f7550102060 2024-03-21T11:29:29.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.166+0000 7f7558d66740 1 Processor -- start 2024-03-21T11:29:29.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.167+0000 7f7558d66740 1 -- start start 2024-03-21T11:29:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.167+0000 7f7558d66740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218a9050 0x5598218a9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.167+0000 7f7558d66740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 0x5598218ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.167+0000 7f7558d66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218aebb0 0x5598218b1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.167+0000 7f7558d66740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982180e250 con 0x5598218a9a40 2024-03-21T11:29:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.167+0000 7f7558d66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598217b6a30 con 0x5598218aebb0 2024-03-21T11:29:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.167+0000 7f7558d66740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982180a2f0 con 0x5598218a9050 2024-03-21T11:29:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.167+0000 7f754fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218a9050 0x5598218a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.167+0000 7f754f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 0x5598218ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.167+0000 7f754fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218a9050 0x5598218a9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45320/0 (socket says 172.21.15.47:45320) 2024-03-21T11:29:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.167+0000 7f754fe00640 1 -- 172.21.15.47:0/3525908007 learned_addr learned my addr 172.21.15.47:0/3525908007 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.167+0000 7f754ea00640 1 --2- 172.21.15.47:0/3525908007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218aebb0 0x5598218b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.168+0000 7f754fe00640 1 -- 172.21.15.47:0/3525908007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218aebb0 msgr2=0x5598218b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.168+0000 7f754fe00640 1 --2- 172.21.15.47:0/3525908007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218aebb0 0x5598218b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.168+0000 7f754fe00640 1 -- 172.21.15.47:0/3525908007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 msgr2=0x5598218ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.168+0000 7f754fe00640 1 --2- 172.21.15.47:0/3525908007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 0x5598218ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.168+0000 7f754fe00640 1 -- 172.21.15.47:0/3525908007 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598217a1690 con 0x5598218a9050 2024-03-21T11:29:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.168+0000 7f7547e00640 1 --2- 172.21.15.47:0/1209372760 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f75340a2fa0 0x7f75340a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.168+0000 7f754f400640 1 --2- 172.21.15.47:0/3525908007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 0x5598218ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:29:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.168+0000 7f7547e00640 1 -- 172.21.15.47:0/1209372760 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f75400f1870 con 0x7f7550102060 2024-03-21T11:29:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.168+0000 7f754ea00640 1 --2- 172.21.15.47:0/1209372760 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f75340a2fa0 0x7f75340a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.168+0000 7f754fe00640 1 --2- 172.21.15.47:0/3525908007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218a9050 0x5598218a9470 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7548009f40 tx=0x7f75480107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3b3dacb5af5d4c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:29.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.168+0000 7f753be00640 1 -- 172.21.15.47:0/3525908007 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548003b70 con 0x5598218a9050 2024-03-21T11:29:29.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.169+0000 7f753be00640 1 -- 172.21.15.47:0/3525908007 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7548003d10 con 0x5598218a9050 2024-03-21T11:29:29.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.169+0000 7f753be00640 1 -- 172.21.15.47:0/3525908007 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548011980 con 0x5598218a9050 2024-03-21T11:29:29.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.169+0000 7f7558d66740 1 -- 172.21.15.47:0/3525908007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218a9050 msgr2=0x5598218a9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:29.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.169+0000 7f754ea00640 1 --2- 172.21.15.47:0/1209372760 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f75340a2fa0 0x7f75340a5460 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7550101740 tx=0x7f753c00b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:29.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.169+0000 7f7558d66740 1 --2- 172.21.15.47:0/3525908007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218a9050 0x5598218a9470 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7548009f40 tx=0x7f75480107a0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.169+0000 7f7558d66740 1 -- 172.21.15.47:0/3525908007 shutdown_connections 2024-03-21T11:29:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.169+0000 7f7558d66740 1 --2- 172.21.15.47:0/3525908007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218aebb0 0x5598218b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.169+0000 7f7558d66740 1 --2- 172.21.15.47:0/3525908007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 0x5598218ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.169+0000 7f7558d66740 1 --2- 172.21.15.47:0/3525908007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218a9050 0x5598218a9470 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.169+0000 7f7558d66740 1 -- 172.21.15.47:0/3525908007 >> 172.21.15.47:0/3525908007 conn(0x559821871200 msgr2=0x5598218a49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.170+0000 7f7558d66740 1 -- 172.21.15.47:0/3525908007 shutdown_connections 2024-03-21T11:29:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.170+0000 7f7558d66740 1 -- 172.21.15.47:0/3525908007 wait complete. 2024-03-21T11:29:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.170+0000 7f7558d66740 1 -- start start 2024-03-21T11:29:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.170+0000 7f7558d66740 1 Processor -- start 2024-03-21T11:29:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f7558d66740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9050 0x5598218b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f7558d66740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218a9a40 0x5598218b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f7558d66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218aebb0 0x5598218ba610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f7558d66740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598218b1b60 con 0x5598218a9050 2024-03-21T11:29:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f7558d66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598217c9e80 con 0x5598218aebb0 2024-03-21T11:29:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f7558d66740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598218b1ce0 con 0x5598218a9a40 2024-03-21T11:29:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f7558d66740 10 client.0 _ll_register_callbacks cb 0x5598217dc130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f754ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218a9a40 0x5598218b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f754fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218aebb0 0x5598218ba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f754ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218a9a40 0x5598218b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45336/0 (socket says 172.21.15.47:45336) 2024-03-21T11:29:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f754ea00640 1 -- 172.21.15.47:0/2990299063 learned_addr learned my addr 172.21.15.47:0/2990299063 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f754f400640 1 --2- 172.21.15.47:0/2990299063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9050 0x5598218b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f754ea00640 1 -- 172.21.15.47:0/2990299063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218aebb0 msgr2=0x5598218ba610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:29.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f754ea00640 1 --2- 172.21.15.47:0/2990299063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218aebb0 0x5598218ba610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f754ea00640 1 -- 172.21.15.47:0/2990299063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9050 msgr2=0x5598218b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f754ea00640 1 --2- 172.21.15.47:0/2990299063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9050 0x5598218b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.171+0000 7f754ea00640 1 -- 172.21.15.47:0/2990299063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598215ec7d0 con 0x5598218a9a40 2024-03-21T11:29:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.172+0000 7f754ea00640 1 --2- 172.21.15.47:0/2990299063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218a9a40 0x5598218b35d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f753c010a00 tx=0x7f753c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1daf4662c5af61b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.172+0000 7f7538c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:29:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.172+0000 7f7538c00640 1 -- 172.21.15.47:0/2990299063 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c004260 con 0x5598218a9a40 2024-03-21T11:29:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.172+0000 7f7558d66740 1 -- 172.21.15.47:0/2990299063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7548003d10 con 0x5598218a9a40 2024-03-21T11:29:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.172+0000 7f7538c00640 1 -- 172.21.15.47:0/2990299063 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f753c004400 con 0x5598218a9a40 2024-03-21T11:29:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.172+0000 7f7538c00640 1 -- 172.21.15.47:0/2990299063 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f753c01f930 con 0x5598218a9a40 2024-03-21T11:29:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.172+0000 7f7558d66740 10 client.27514 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.172+0000 7f7558d66740 1 -- 172.21.15.47:0/2990299063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5598215223f0 con 0x5598218a9a40 2024-03-21T11:29:29.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.173+0000 7f7538c00640 1 -- 172.21.15.47:0/2990299063 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7548003d10 con 0x5598218a9a40 2024-03-21T11:29:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.173+0000 7f7538c00640 1 -- 172.21.15.47:0/2990299063 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f753c026070 con 0x5598218a9a40 2024-03-21T11:29:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.173+0000 7f7558d66740 10 client.27514 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.173+0000 7f7558d66740 10 client.27514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.173+0000 7f7558d66740 20 client.27514 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.173+0000 7f7558d66740 1 --2- 172.21.15.47:0/2990299063 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5598218befb0 0x5598218c1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.173+0000 7f7558d66740 4 client.27514 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:29:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.173+0000 7f7558d66740 1 -- 172.21.15.47:0/2990299063 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55982181b790 con 0x5598218befb0 2024-03-21T11:29:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.173+0000 7f754f400640 1 --2- 172.21.15.47:0/2990299063 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5598218befb0 0x5598218c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.174+0000 7f754f400640 1 --2- 172.21.15.47:0/2990299063 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5598218befb0 0x5598218c1430 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adca0bc2206e6b88 server_cookie=a752cf5abe0df87b in_seq=0 out_seq=0 2024-03-21T11:29:29.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.174+0000 7f7538c00640 10 client.27514 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:29.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.176+0000 7f7538c00640 1 -- 172.21.15.47:0/2990299063 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55982181b790 con 0x5598218befb0 2024-03-21T11:29:29.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.176+0000 7f7538c00640 10 client.27514 handle_command_reply: tid=0 2024-03-21T11:29:29.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.176+0000 7f7558d66740 1 client.27514 shutdown 2024-03-21T11:29:29.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.176+0000 7f7558d66740 1 -- 172.21.15.47:0/2990299063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218a9a40 msgr2=0x5598218b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:29.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.176+0000 7f7558d66740 1 --2- 172.21.15.47:0/2990299063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218a9a40 0x5598218b35d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f753c010a00 tx=0x7f753c004730 comp rx=0 tx=0).stop 2024-03-21T11:29:29.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.176+0000 7f7558d66740 1 -- 172.21.15.47:0/2990299063 shutdown_connections 2024-03-21T11:29:29.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.176+0000 7f7558d66740 1 --2- 172.21.15.47:0/2990299063 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5598218befb0 0x5598218c1430 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.176+0000 7f7558d66740 1 --2- 172.21.15.47:0/2990299063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218aebb0 0x5598218ba610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.176+0000 7f7558d66740 1 --2- 172.21.15.47:0/2990299063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218a9a40 0x5598218b35d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.176+0000 7f7558d66740 1 --2- 172.21.15.47:0/2990299063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9050 0x5598218b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.176+0000 7f7558d66740 1 -- 172.21.15.47:0/2990299063 >> 172.21.15.47:0/2990299063 conn(0x559821871200 msgr2=0x5598218a3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:29.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.176+0000 7f7538c00640 0 client.27514 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:29.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.177+0000 7f7558d66740 1 -- 172.21.15.47:0/2990299063 shutdown_connections 2024-03-21T11:29:29.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.177+0000 7f7558d66740 1 -- 172.21.15.47:0/2990299063 wait complete. 2024-03-21T11:29:29.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.177+0000 7f7558d66740 20 client.27514 trim_cache size 0 max 16384 2024-03-21T11:29:29.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.195+0000 7f7558d66740 1 Processor -- start 2024-03-21T11:29:29.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.195+0000 7f7558d66740 1 -- start start 2024-03-21T11:29:29.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.195+0000 7f7558d66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218a9050 0x5598218a8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.195+0000 7f7558d66740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 0x5598218ba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.195+0000 7f7558d66740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218aebb0 0x5598218c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.195+0000 7f7558d66740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75501ac060 con 0x5598218a9a40 2024-03-21T11:29:29.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.195+0000 7f7558d66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75501ad9e0 con 0x5598218a9050 2024-03-21T11:29:29.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.195+0000 7f7558d66740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75501ac6c0 con 0x5598218aebb0 2024-03-21T11:29:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.196+0000 7f754fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218aebb0 0x5598218c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.196+0000 7f754ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 0x5598218ba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.196+0000 7f754fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218aebb0 0x5598218c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45344/0 (socket says 172.21.15.47:45344) 2024-03-21T11:29:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.196+0000 7f754fe00640 1 -- 172.21.15.47:0/3486528033 learned_addr learned my addr 172.21.15.47:0/3486528033 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:29.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.196+0000 7f754f400640 1 --2- 172.21.15.47:0/3486528033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218a9050 0x5598218a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.196+0000 7f754fe00640 1 -- 172.21.15.47:0/3486528033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218a9050 msgr2=0x5598218a8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.196+0000 7f754fe00640 1 --2- 172.21.15.47:0/3486528033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218a9050 0x5598218a8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.196+0000 7f754fe00640 1 -- 172.21.15.47:0/3486528033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 msgr2=0x5598218ba5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.196+0000 7f754fe00640 1 --2- 172.21.15.47:0/3486528033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 0x5598218ba5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.196+0000 7f754fe00640 1 -- 172.21.15.47:0/3486528033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598214e20f0 con 0x5598218aebb0 2024-03-21T11:29:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.196+0000 7f754f400640 1 --2- 172.21.15.47:0/3486528033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218a9050 0x5598218a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:29:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.196+0000 7f754fe00640 1 --2- 172.21.15.47:0/3486528033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218aebb0 0x5598218c5d30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7548010ca0 tx=0x7f7548003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=284692bc417d4af4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.197+0000 7f753b400640 1 -- 172.21.15.47:0/3486528033 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548005240 con 0x5598218aebb0 2024-03-21T11:29:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.197+0000 7f753b400640 1 -- 172.21.15.47:0/3486528033 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7548003ce0 con 0x5598218aebb0 2024-03-21T11:29:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.197+0000 7f753b400640 1 -- 172.21.15.47:0/3486528033 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548011b80 con 0x5598218aebb0 2024-03-21T11:29:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.197+0000 7f7558d66740 1 -- 172.21.15.47:0/3486528033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218aebb0 msgr2=0x5598218c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.197+0000 7f7558d66740 1 --2- 172.21.15.47:0/3486528033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218aebb0 0x5598218c5d30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7548010ca0 tx=0x7f7548003970 comp rx=0 tx=0).stop 2024-03-21T11:29:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.197+0000 7f7558d66740 1 -- 172.21.15.47:0/3486528033 shutdown_connections 2024-03-21T11:29:29.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.197+0000 7f7558d66740 1 --2- 172.21.15.47:0/3486528033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218aebb0 0x5598218c5d30 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.197+0000 7f7558d66740 1 --2- 172.21.15.47:0/3486528033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 0x5598218ba5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.197+0000 7f7558d66740 1 --2- 172.21.15.47:0/3486528033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218a9050 0x5598218a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.197+0000 7f7558d66740 1 -- 172.21.15.47:0/3486528033 >> 172.21.15.47:0/3486528033 conn(0x559821871200 msgr2=0x5598218a7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:29.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.198+0000 7f7558d66740 1 -- 172.21.15.47:0/3486528033 shutdown_connections 2024-03-21T11:29:29.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.198+0000 7f7558d66740 1 -- 172.21.15.47:0/3486528033 wait complete. 2024-03-21T11:29:29.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.198+0000 7f7558d66740 1 -- start start 2024-03-21T11:29:29.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.198+0000 7f7558d66740 1 Processor -- start 2024-03-21T11:29:29.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f7558d66740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218a9050 0x5598218ba4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f7558d66740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 0x5598218b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f7558d66740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218aebb0 0x5598218b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f7558d66740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75501ad380 con 0x5598218a9a40 2024-03-21T11:29:29.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f7558d66740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75501acd20 con 0x5598218a9050 2024-03-21T11:29:29.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f7558d66740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75501aa220 con 0x5598218aebb0 2024-03-21T11:29:29.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f7558d66740 10 client.0 _ll_register_callbacks cb 0x7f753c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:29.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f754f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218a9050 0x5598218ba4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f754fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218aebb0 0x5598218b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f754ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 0x5598218b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f754fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218aebb0 0x5598218b9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45352/0 (socket says 172.21.15.47:45352) 2024-03-21T11:29:29.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f754fe00640 1 -- 172.21.15.47:0/1228626299 learned_addr learned my addr 172.21.15.47:0/1228626299 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:29.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f754f400640 1 -- 172.21.15.47:0/1228626299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218aebb0 msgr2=0x5598218b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:29.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f754f400640 1 --2- 172.21.15.47:0/1228626299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218aebb0 0x5598218b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f754f400640 1 -- 172.21.15.47:0/1228626299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 msgr2=0x5598218b8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:29.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f754f400640 1 --2- 172.21.15.47:0/1228626299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 0x5598218b8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f754f400640 1 -- 172.21.15.47:0/1228626299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598218b9890 con 0x5598218a9050 2024-03-21T11:29:29.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.199+0000 7f754fe00640 1 --2- 172.21.15.47:0/1228626299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218aebb0 0x5598218b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:29.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.200+0000 7f754f400640 1 --2- 172.21.15.47:0/1228626299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218a9050 0x5598218ba4c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x5598218aa4b0 tx=0x7f7540000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8914fc295d44fd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:29.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.200+0000 7f7538c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:29:29.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.200+0000 7f7538c00640 1 -- 172.21.15.47:0/1228626299 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540004480 con 0x5598218a9050 2024-03-21T11:29:29.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.200+0000 7f7538c00640 1 -- 172.21.15.47:0/1228626299 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f75400ef090 con 0x5598218a9050 2024-03-21T11:29:29.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.200+0000 7f7558d66740 1 -- 172.21.15.47:0/1228626299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7548003ce0 con 0x5598218a9050 2024-03-21T11:29:29.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.200+0000 7f7538c00640 1 -- 172.21.15.47:0/1228626299 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7540005690 con 0x5598218a9050 2024-03-21T11:29:29.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.200+0000 7f7558d66740 10 client.27500 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:29.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.200+0000 7f7558d66740 1 -- 172.21.15.47:0/1228626299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55982191b9d0 con 0x5598218a9050 2024-03-21T11:29:29.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.201+0000 7f7538c00640 1 -- 172.21.15.47:0/1228626299 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7548003ce0 con 0x5598218a9050 2024-03-21T11:29:29.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.201+0000 7f7538c00640 1 -- 172.21.15.47:0/1228626299 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f75400ba220 con 0x5598218a9050 2024-03-21T11:29:29.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.201+0000 7f7558d66740 10 client.27500 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:29.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.201+0000 7f7558d66740 10 client.27500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:29.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.201+0000 7f7558d66740 20 client.27500 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:29.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.201+0000 7f7558d66740 1 --2- 172.21.15.47:0/1228626299 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55982191c230 0x55982191e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.201+0000 7f7558d66740 4 client.27500 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:29:29.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.201+0000 7f7558d66740 1 -- 172.21.15.47:0/1228626299 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559821815ae0 con 0x55982191c230 2024-03-21T11:29:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.201+0000 7f754ea00640 1 --2- 172.21.15.47:0/1228626299 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55982191c230 0x55982191e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.202+0000 7f754ea00640 1 --2- 172.21.15.47:0/1228626299 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55982191c230 0x55982191e650 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbff62944747ee94 server_cookie=a380faf4c3b65d14 in_seq=0 out_seq=0 2024-03-21T11:29:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.202+0000 7f7538c00640 10 client.27500 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:29.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.203+0000 7f7538c00640 1 -- 172.21.15.47:0/1228626299 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559821815ae0 con 0x55982191c230 2024-03-21T11:29:29.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.203+0000 7f7538c00640 10 client.27500 handle_command_reply: tid=0 2024-03-21T11:29:29.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.204+0000 7f7558d66740 1 client.27500 shutdown 2024-03-21T11:29:29.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.204+0000 7f7558d66740 1 -- 172.21.15.47:0/1228626299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218a9050 msgr2=0x5598218ba4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:29.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.204+0000 7f7558d66740 1 --2- 172.21.15.47:0/1228626299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218a9050 0x5598218ba4c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x5598218aa4b0 tx=0x7f7540000bc0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.204+0000 7f7558d66740 1 -- 172.21.15.47:0/1228626299 shutdown_connections 2024-03-21T11:29:29.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.204+0000 7f7558d66740 1 --2- 172.21.15.47:0/1228626299 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55982191c230 0x55982191e650 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.204+0000 7f7558d66740 1 --2- 172.21.15.47:0/1228626299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5598218aebb0 0x5598218b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.204+0000 7f7558d66740 1 --2- 172.21.15.47:0/1228626299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5598218a9a40 0x5598218b8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.204+0000 7f7558d66740 1 --2- 172.21.15.47:0/1228626299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5598218a9050 0x5598218ba4c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.204+0000 7f7538c00640 0 client.27500 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:29.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.204+0000 7f7558d66740 1 -- 172.21.15.47:0/1228626299 >> 172.21.15.47:0/1228626299 conn(0x559821871200 msgr2=0x5598218a8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:29.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.204+0000 7f7558d66740 1 -- 172.21.15.47:0/1228626299 shutdown_connections 2024-03-21T11:29:29.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.204+0000 7f7558d66740 1 -- 172.21.15.47:0/1228626299 wait complete. 2024-03-21T11:29:29.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.204+0000 7f7558d66740 20 client.27500 trim_cache size 0 max 16384 2024-03-21T11:29:29.207 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:29:29.207 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:29:29.208 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:29:29.208 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:29:29.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.205+0000 7f7554a00640 1 -- 172.21.15.47:0/1209372760 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f75340a2fa0 msgr2=0x7f75340a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:29.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.205+0000 7f7554a00640 1 --2- 172.21.15.47:0/1209372760 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f75340a2fa0 0x7f75340a5460 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7550101740 tx=0x7f753c00b480 comp rx=0 tx=0).stop 2024-03-21T11:29:29.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.205+0000 7f7554a00640 1 -- 172.21.15.47:0/1209372760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550102060 msgr2=0x7f75501019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:29.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.205+0000 7f7554a00640 1 --2- 172.21.15.47:0/1209372760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550102060 0x7f75501019d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7540036040 tx=0x7f754002c6c0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.205+0000 7f7554a00640 1 -- 172.21.15.47:0/1209372760 shutdown_connections 2024-03-21T11:29:29.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.205+0000 7f7554a00640 1 --2- 172.21.15.47:0/1209372760 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f75340a2fa0 0x7f75340a5460 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.205+0000 7f7554a00640 1 --2- 172.21.15.47:0/1209372760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75501072c0 0x7f7550100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.205+0000 7f7554a00640 1 --2- 172.21.15.47:0/1209372760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7550104990 0x7f7550100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.205+0000 7f7554a00640 1 --2- 172.21.15.47:0/1209372760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7550102060 0x7f75501019d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.205+0000 7f7554a00640 1 -- 172.21.15.47:0/1209372760 >> 172.21.15.47:0/1209372760 conn(0x7f75500fba10 msgr2=0x7f7550102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:29.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.205+0000 7f7554a00640 1 -- 172.21.15.47:0/1209372760 shutdown_connections 2024-03-21T11:29:29.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.205+0000 7f7554a00640 1 -- 172.21.15.47:0/1209372760 wait complete. 2024-03-21T11:29:29.223 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:29:29.223 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:29:29.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.449+0000 7f2e7dc00640 1 -- 172.21.15.47:0/66121145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e78101500 msgr2=0x7f2e78101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:29.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.449+0000 7f2e7dc00640 1 --2- 172.21.15.47:0/66121145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e78101500 0x7f2e78101920 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009900 tx=0x7f2e6002bc40 comp rx=0 tx=0).stop 2024-03-21T11:29:29.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.449+0000 7f2e7dc00640 1 -- 172.21.15.47:0/66121145 shutdown_connections 2024-03-21T11:29:29.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.449+0000 7f2e7dc00640 1 --2- 172.21.15.47:0/66121145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e781030b0 0x7f2e78109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.449+0000 7f2e7dc00640 1 --2- 172.21.15.47:0/66121145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e781026f0 0x7f2e78102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.449+0000 7f2e7dc00640 1 --2- 172.21.15.47:0/66121145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e78101500 0x7f2e78101920 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.449+0000 7f2e7dc00640 1 -- 172.21.15.47:0/66121145 >> 172.21.15.47:0/66121145 conn(0x7f2e780fca70 msgr2=0x7f2e780feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.449+0000 7f2e7dc00640 1 -- 172.21.15.47:0/66121145 shutdown_connections 2024-03-21T11:29:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.449+0000 7f2e7dc00640 1 -- 172.21.15.47:0/66121145 wait complete. 2024-03-21T11:29:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.450+0000 7f2e7dc00640 1 Processor -- start 2024-03-21T11:29:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.450+0000 7f2e7dc00640 1 -- start start 2024-03-21T11:29:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.451+0000 7f2e7dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e78101500 0x7f2e78075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.451+0000 7f2e7dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e781026f0 0x7f2e78073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.451+0000 7f2e7dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e781030b0 0x7f2e78073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.451+0000 7f2e7dc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7810d1b0 con 0x7f2e78101500 2024-03-21T11:29:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.451+0000 7f2e7dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7810d030 con 0x7f2e781030b0 2024-03-21T11:29:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.451+0000 7f2e7dc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7810d330 con 0x7f2e781026f0 2024-03-21T11:29:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.451+0000 7f2e77e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e78101500 0x7f2e78075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.451+0000 7f2e77e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e78101500 0x7f2e78075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34328/0 (socket says 172.21.15.47:34328) 2024-03-21T11:29:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.451+0000 7f2e77e00640 1 -- 172.21.15.47:0/3350974012 learned_addr learned my addr 172.21.15.47:0/3350974012 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.451+0000 7f2e77400640 1 --2- 172.21.15.47:0/3350974012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e781026f0 0x7f2e78073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.451+0000 7f2e7d200640 1 --2- 172.21.15.47:0/3350974012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e781030b0 0x7f2e78073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.452+0000 7f2e77e00640 1 -- 172.21.15.47:0/3350974012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e781026f0 msgr2=0x7f2e78073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.452+0000 7f2e77e00640 1 --2- 172.21.15.47:0/3350974012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e781026f0 0x7f2e78073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.452+0000 7f2e77e00640 1 -- 172.21.15.47:0/3350974012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e781030b0 msgr2=0x7f2e78073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.452+0000 7f2e77e00640 1 --2- 172.21.15.47:0/3350974012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e781030b0 0x7f2e78073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.452+0000 7f2e77e00640 1 -- 172.21.15.47:0/3350974012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e78074860 con 0x7f2e78101500 2024-03-21T11:29:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.452+0000 7f2e77e00640 1 --2- 172.21.15.47:0/3350974012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e78101500 0x7f2e78075450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e6002efd0 tx=0x7f2e60005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.452+0000 7f2e74c00640 1 -- 172.21.15.47:0/3350974012 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60041070 con 0x7f2e78101500 2024-03-21T11:29:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.452+0000 7f2e74c00640 1 -- 172.21.15.47:0/3350974012 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e600044d0 con 0x7f2e78101500 2024-03-21T11:29:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.452+0000 7f2e7dc00640 1 -- 172.21.15.47:0/3350974012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e781a18e0 con 0x7f2e78101500 2024-03-21T11:29:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.453+0000 7f2e74c00640 1 -- 172.21.15.47:0/3350974012 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60034440 con 0x7f2e78101500 2024-03-21T11:29:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.453+0000 7f2e7dc00640 1 -- 172.21.15.47:0/3350974012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e781a1e50 con 0x7f2e78101500 2024-03-21T11:29:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.454+0000 7f2e7dc00640 1 -- 172.21.15.47:0/3350974012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e78108820 con 0x7f2e78101500 2024-03-21T11:29:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.456+0000 7f2e74c00640 1 -- 172.21.15.47:0/3350974012 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2e60044070 con 0x7f2e78101500 2024-03-21T11:29:29.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.458+0000 7f2e74c00640 1 --2- 172.21.15.47:0/3350974012 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2e5c0a2fa0 0x7f2e5c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:29.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.458+0000 7f2e74c00640 1 -- 172.21.15.47:0/3350974012 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2e600f12c0 con 0x7f2e78101500 2024-03-21T11:29:29.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.459+0000 7f2e77400640 1 --2- 172.21.15.47:0/3350974012 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2e5c0a2fa0 0x7f2e5c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:29.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.459+0000 7f2e77400640 1 --2- 172.21.15.47:0/3350974012 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2e5c0a2fa0 0x7f2e5c0a5460 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2e680095f0 tx=0x7f2e68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:29.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.464+0000 7f2e74c00640 1 -- 172.21.15.47:0/3350974012 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2e600bd6b0 con 0x7f2e78101500 2024-03-21T11:29:29.520 INFO:tasks.workunit.client.0.smithi047.stdout: 1 399899 8.27 MB/sec execute 133 sec latency 20.796 ms 2024-03-21T11:29:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.748+0000 7f2e7dc00640 1 -- 172.21.15.47:0/3350974012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e780622a0 con 0x7f2e78101500 2024-03-21T11:29:29.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.749+0000 7f2e74c00640 1 -- 172.21.15.47:0/3350974012 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f2e600a80c0 con 0x7f2e78101500 2024-03-21T11:29:29.749 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:29:29.750 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:29:29.750 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:29:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.752+0000 7f2e7dc00640 1 -- 172.21.15.47:0/3350974012 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2e5c0a2fa0 msgr2=0x7f2e5c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.752+0000 7f2e7dc00640 1 --2- 172.21.15.47:0/3350974012 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2e5c0a2fa0 0x7f2e5c0a5460 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2e680095f0 tx=0x7f2e68005f90 comp rx=0 tx=0).stop 2024-03-21T11:29:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.752+0000 7f2e7dc00640 1 -- 172.21.15.47:0/3350974012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e78101500 msgr2=0x7f2e78075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.752+0000 7f2e7dc00640 1 --2- 172.21.15.47:0/3350974012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e78101500 0x7f2e78075450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e6002efd0 tx=0x7f2e60005d50 comp rx=0 tx=0).stop 2024-03-21T11:29:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.752+0000 7f2e7dc00640 1 -- 172.21.15.47:0/3350974012 shutdown_connections 2024-03-21T11:29:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.752+0000 7f2e7dc00640 1 --2- 172.21.15.47:0/3350974012 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2e5c0a2fa0 0x7f2e5c0a5460 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.752+0000 7f2e7dc00640 1 --2- 172.21.15.47:0/3350974012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e781030b0 0x7f2e78073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.752+0000 7f2e7dc00640 1 --2- 172.21.15.47:0/3350974012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e781026f0 0x7f2e78073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.752+0000 7f2e7dc00640 1 --2- 172.21.15.47:0/3350974012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e78101500 0x7f2e78075450 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:29.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.752+0000 7f2e7dc00640 1 -- 172.21.15.47:0/3350974012 >> 172.21.15.47:0/3350974012 conn(0x7f2e780fca70 msgr2=0x7f2e781059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:29.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.752+0000 7f2e7dc00640 1 -- 172.21.15.47:0/3350974012 shutdown_connections 2024-03-21T11:29:29.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:29.752+0000 7f2e7dc00640 1 -- 172.21.15.47:0/3350974012 wait complete. 2024-03-21T11:29:29.774 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:29:29.774 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:29:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.000+0000 7f20fec00640 1 -- 172.21.15.47:0/3807045472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20f81015a0 msgr2=0x7f20f81019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.000+0000 7f20fec00640 1 --2- 172.21.15.47:0/3807045472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20f81015a0 0x7f20f81019c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009900 tx=0x7f20ec02bc30 comp rx=0 tx=0).stop 2024-03-21T11:29:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.000+0000 7f20fec00640 1 -- 172.21.15.47:0/3807045472 shutdown_connections 2024-03-21T11:29:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.000+0000 7f20fec00640 1 --2- 172.21.15.47:0/3807045472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20f8103150 0x7f20f81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.000+0000 7f20fec00640 1 --2- 172.21.15.47:0/3807045472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20f8102790 0x7f20f8102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.000+0000 7f20fec00640 1 --2- 172.21.15.47:0/3807045472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20f81015a0 0x7f20f81019c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.000+0000 7f20fec00640 1 -- 172.21.15.47:0/3807045472 >> 172.21.15.47:0/3807045472 conn(0x7f20f80fcb10 msgr2=0x7f20f80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.000+0000 7f20fec00640 1 -- 172.21.15.47:0/3807045472 shutdown_connections 2024-03-21T11:29:30.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.000+0000 7f20fec00640 1 -- 172.21.15.47:0/3807045472 wait complete. 2024-03-21T11:29:30.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.001+0000 7f20fec00640 1 Processor -- start 2024-03-21T11:29:30.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.001+0000 7f20fec00640 1 -- start start 2024-03-21T11:29:30.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fec00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20f81015a0 0x7f20f8075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20f8102790 0x7f20f8073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20f8103150 0x7f20f8073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fec00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f810d240 con 0x7f20f81015a0 2024-03-21T11:29:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f810d0c0 con 0x7f20f8103150 2024-03-21T11:29:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f810d3c0 con 0x7f20f8102790 2024-03-21T11:29:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20f8102790 0x7f20f8073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fd800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20f81015a0 0x7f20f8075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20f8102790 0x7f20f8073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45378/0 (socket says 172.21.15.47:45378) 2024-03-21T11:29:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fce00640 1 -- 172.21.15.47:0/3037483520 learned_addr learned my addr 172.21.15.47:0/3037483520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fe200640 1 --2- 172.21.15.47:0/3037483520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20f8103150 0x7f20f8073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fe200640 1 -- 172.21.15.47:0/3037483520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20f8102790 msgr2=0x7f20f8073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fe200640 1 --2- 172.21.15.47:0/3037483520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20f8102790 0x7f20f8073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fe200640 1 -- 172.21.15.47:0/3037483520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20f81015a0 msgr2=0x7f20f8075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fe200640 1 --2- 172.21.15.47:0/3037483520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20f81015a0 0x7f20f8075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fe200640 1 -- 172.21.15.47:0/3037483520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f8074860 con 0x7f20f8103150 2024-03-21T11:29:30.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.002+0000 7f20fce00640 1 --2- 172.21.15.47:0/3037483520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20f8102790 0x7f20f8073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:30.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.003+0000 7f20fe200640 1 --2- 172.21.15.47:0/3037483520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20f8103150 0x7f20f8073fd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009fd0 tx=0x7f20f000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.003+0000 7f20f6000640 1 -- 172.21.15.47:0/3037483520 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f000ce20 con 0x7f20f8103150 2024-03-21T11:29:30.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.003+0000 7f20f6000640 1 -- 172.21.15.47:0/3037483520 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20f0005020 con 0x7f20f8103150 2024-03-21T11:29:30.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.003+0000 7f20f6000640 1 -- 172.21.15.47:0/3037483520 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0010720 con 0x7f20f8103150 2024-03-21T11:29:30.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.003+0000 7f20fec00640 1 -- 172.21.15.47:0/3037483520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f81a19a0 con 0x7f20f8103150 2024-03-21T11:29:30.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.003+0000 7f20fec00640 1 -- 172.21.15.47:0/3037483520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f81a1ea0 con 0x7f20f8103150 2024-03-21T11:29:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.006+0000 7f2102fa0740 1 Processor -- start 2024-03-21T11:29:30.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.006+0000 7f20f6000640 1 -- 172.21.15.47:0/3037483520 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f20f0005540 con 0x7f20f8103150 2024-03-21T11:29:30.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.007+0000 7f2102fa0740 1 -- start start 2024-03-21T11:29:30.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.007+0000 7f2102fa0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 0x55bd58b10e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.007+0000 7f2102fa0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b12040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.007+0000 7f2102fa0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b18e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.007+0000 7f2102fa0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd58a752c0 con 0x55bd58b109f0 2024-03-21T11:29:30.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.007+0000 7f2102fa0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd58a1dc90 con 0x55bd58b11bc0 2024-03-21T11:29:30.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.007+0000 7f2102fa0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd58a71360 con 0x55bd58b12580 2024-03-21T11:29:30.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.007+0000 7f20fd800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b12040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.007+0000 7f20fce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 0x55bd58b10e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.007+0000 7f20fd800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b12040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43322/0 (socket says 172.21.15.47:43322) 2024-03-21T11:29:30.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.007+0000 7f20fd800640 1 -- 172.21.15.47:0/1513571641 learned_addr learned my addr 172.21.15.47:0/1513571641 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.008+0000 7f20fce00640 1 --2- 172.21.15.47:0/1513571641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b18e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.008+0000 7f20fd800640 1 -- 172.21.15.47:0/1513571641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 msgr2=0x55bd58b18e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.008+0000 7f20fd800640 1 --2- 172.21.15.47:0/1513571641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b18e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.008+0000 7f20fd800640 1 -- 172.21.15.47:0/1513571641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 msgr2=0x55bd58b10e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.008+0000 7f20fd800640 1 --2- 172.21.15.47:0/1513571641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 0x55bd58b10e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.008+0000 7f20fd800640 1 -- 172.21.15.47:0/1513571641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd58a42080 con 0x55bd58b11bc0 2024-03-21T11:29:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.008+0000 7f20fce00640 1 --2- 172.21.15.47:0/1513571641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 0x55bd58b10e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:29:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.008+0000 7f20fd800640 1 --2- 172.21.15.47:0/1513571641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b12040 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f20ec000c00 tx=0x7f20ec009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5439035a8533a4a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.008+0000 7f20f6000640 1 --2- 172.21.15.47:0/3037483520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f20e00a2fa0 0x7f20e00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.008+0000 7f20f6000640 1 -- 172.21.15.47:0/3037483520 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f20f00cfe00 con 0x7f20f8103150 2024-03-21T11:29:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.009+0000 7f20eaa00640 1 -- 172.21.15.47:0/1513571641 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec02c980 con 0x55bd58b11bc0 2024-03-21T11:29:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.009+0000 7f20fce00640 1 --2- 172.21.15.47:0/3037483520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f20e00a2fa0 0x7f20e00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.009+0000 7f20eaa00640 1 -- 172.21.15.47:0/1513571641 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20ec02cb20 con 0x55bd58b11bc0 2024-03-21T11:29:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.009+0000 7f20eaa00640 1 -- 172.21.15.47:0/1513571641 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0046c0 con 0x55bd58b11bc0 2024-03-21T11:29:30.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.009+0000 7f2102fa0740 1 -- 172.21.15.47:0/1513571641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 msgr2=0x55bd58b12040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.009+0000 7f2102fa0740 1 --2- 172.21.15.47:0/1513571641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b12040 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f20ec000c00 tx=0x7f20ec009560 comp rx=0 tx=0).stop 2024-03-21T11:29:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.009+0000 7f20fce00640 1 --2- 172.21.15.47:0/3037483520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f20e00a2fa0 0x7f20e00a5460 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55bd58b11ac0 tx=0x7f20e4009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.010+0000 7f2102fa0740 1 -- 172.21.15.47:0/1513571641 shutdown_connections 2024-03-21T11:29:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.010+0000 7f2102fa0740 1 --2- 172.21.15.47:0/1513571641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b18e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.010+0000 7f2102fa0740 1 --2- 172.21.15.47:0/1513571641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b12040 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.010+0000 7f2102fa0740 1 --2- 172.21.15.47:0/1513571641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 0x55bd58b10e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.010+0000 7f2102fa0740 1 -- 172.21.15.47:0/1513571641 >> 172.21.15.47:0/1513571641 conn(0x55bd58b0bde0 msgr2=0x55bd58b0c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.010+0000 7f2102fa0740 1 -- 172.21.15.47:0/1513571641 shutdown_connections 2024-03-21T11:29:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.010+0000 7f2102fa0740 1 -- 172.21.15.47:0/1513571641 wait complete. 2024-03-21T11:29:30.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.010+0000 7f2102fa0740 1 -- start start 2024-03-21T11:29:30.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.011+0000 7f2102fa0740 1 Processor -- start 2024-03-21T11:29:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.011+0000 7f2102fa0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 0x55bd58b1abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.011+0000 7f2102fa0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b1b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.011+0000 7f2102fa0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b22150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.011+0000 7f2102fa0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd58b19530 con 0x55bd58b109f0 2024-03-21T11:29:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.011+0000 7f2102fa0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd58a310d0 con 0x55bd58b11bc0 2024-03-21T11:29:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.011+0000 7f2102fa0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd58b196b0 con 0x55bd58b12580 2024-03-21T11:29:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.011+0000 7f2102fa0740 10 client.0 _ll_register_callbacks cb 0x55bd58a35e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.011+0000 7f20fce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b1b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.011+0000 7f20fce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b1b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43334/0 (socket says 172.21.15.47:43334) 2024-03-21T11:29:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.011+0000 7f20fce00640 1 -- 172.21.15.47:0/1899654713 learned_addr learned my addr 172.21.15.47:0/1899654713 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.011+0000 7f20fd800640 1 --2- 172.21.15.47:0/1899654713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 0x55bd58b1abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.011+0000 7f20fe200640 1 --2- 172.21.15.47:0/1899654713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b22150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.012+0000 7f20fe200640 1 -- 172.21.15.47:0/1899654713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 msgr2=0x55bd58b1b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.012+0000 7f20fe200640 1 --2- 172.21.15.47:0/1899654713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b1b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.012+0000 7f20fe200640 1 -- 172.21.15.47:0/1899654713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 msgr2=0x55bd58b1abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.012+0000 7f20fe200640 1 --2- 172.21.15.47:0/1899654713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 0x55bd58b1abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.012+0000 7f20fe200640 1 -- 172.21.15.47:0/1899654713 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd587893f0 con 0x55bd58b12580 2024-03-21T11:29:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.012+0000 7f20fce00640 1 --2- 172.21.15.47:0/1899654713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b1b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.012+0000 7f20fe200640 1 --2- 172.21.15.47:0/1899654713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b22150 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f20f001b000 tx=0x7f20f009c260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c5487b791e2d250 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.012+0000 7f20df400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:29:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.012+0000 7f20df400640 1 -- 172.21.15.47:0/1899654713 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f00a4720 con 0x55bd58b12580 2024-03-21T11:29:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.012+0000 7f2102fa0740 1 -- 172.21.15.47:0/1899654713 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20ec02cb20 con 0x55bd58b12580 2024-03-21T11:29:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.012+0000 7f20df400640 1 -- 172.21.15.47:0/1899654713 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f20f00a48c0 con 0x55bd58b12580 2024-03-21T11:29:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.013+0000 7f20df400640 1 -- 172.21.15.47:0/1899654713 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20f00b5240 con 0x55bd58b12580 2024-03-21T11:29:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.013+0000 7f2102fa0740 10 client.27538 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:30.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.013+0000 7f2102fa0740 1 -- 172.21.15.47:0/1899654713 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bd588537d0 con 0x55bd58b12580 2024-03-21T11:29:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.013+0000 7f20df400640 1 -- 172.21.15.47:0/1899654713 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20ec02cb20 con 0x55bd58b12580 2024-03-21T11:29:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.013+0000 7f20df400640 1 -- 172.21.15.47:0/1899654713 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f20f0097f80 con 0x55bd58b12580 2024-03-21T11:29:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.013+0000 7f2102fa0740 10 client.27538 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.013+0000 7f2102fa0740 10 client.27538 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:29:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.013+0000 7f2102fa0740 20 client.27538 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.013+0000 7f2102fa0740 1 --2- 172.21.15.47:0/1899654713 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55bd58b266a0 0x55bd58b28b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.013+0000 7f2102fa0740 4 client.27538 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:29:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.013+0000 7f2102fa0740 1 -- 172.21.15.47:0/1899654713 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd58a82800 con 0x55bd58b266a0 2024-03-21T11:29:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.013+0000 7f20fd800640 1 --2- 172.21.15.47:0/1899654713 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55bd58b266a0 0x55bd58b28b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.014+0000 7f20fd800640 1 --2- 172.21.15.47:0/1899654713 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55bd58b266a0 0x55bd58b28b10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de03609a676c7061 server_cookie=953e0681e144cc35 in_seq=0 out_seq=0 2024-03-21T11:29:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.014+0000 7f20df400640 10 client.27538 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:29:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.015+0000 7f20df400640 1 -- 172.21.15.47:0/1899654713 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bd58a82800 con 0x55bd58b266a0 2024-03-21T11:29:30.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.015+0000 7f20df400640 10 client.27538 handle_command_reply: tid=0 2024-03-21T11:29:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.015+0000 7f2102fa0740 1 client.27538 shutdown 2024-03-21T11:29:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.015+0000 7f2102fa0740 1 -- 172.21.15.47:0/1899654713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 msgr2=0x55bd58b22150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.015+0000 7f2102fa0740 1 --2- 172.21.15.47:0/1899654713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b22150 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f20f001b000 tx=0x7f20f009c260 comp rx=0 tx=0).stop 2024-03-21T11:29:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.015+0000 7f2102fa0740 1 -- 172.21.15.47:0/1899654713 shutdown_connections 2024-03-21T11:29:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.015+0000 7f2102fa0740 1 --2- 172.21.15.47:0/1899654713 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55bd58b266a0 0x55bd58b28b10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.015+0000 7f2102fa0740 1 --2- 172.21.15.47:0/1899654713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b22150 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.015+0000 7f2102fa0740 1 --2- 172.21.15.47:0/1899654713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b1b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.015+0000 7f2102fa0740 1 --2- 172.21.15.47:0/1899654713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 0x55bd58b1abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.015+0000 7f20df400640 0 client.27538 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:29:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.015+0000 7f2102fa0740 1 -- 172.21.15.47:0/1899654713 >> 172.21.15.47:0/1899654713 conn(0x55bd58b0bde0 msgr2=0x55bd58b0b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.016+0000 7f2102fa0740 1 -- 172.21.15.47:0/1899654713 shutdown_connections 2024-03-21T11:29:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.016+0000 7f2102fa0740 1 -- 172.21.15.47:0/1899654713 wait complete. 2024-03-21T11:29:30.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.016+0000 7f2102fa0740 20 client.27538 trim_cache size 0 max 16384 2024-03-21T11:29:30.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.032+0000 7f2102fa0740 1 Processor -- start 2024-03-21T11:29:30.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.032+0000 7f2102fa0740 1 -- start start 2024-03-21T11:29:30.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.032+0000 7f2102fa0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 0x55bd58b0f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.032+0000 7f2102fa0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b22110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.032+0000 7f2102fa0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b2d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.032+0000 7f2102fa0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f81b0490 con 0x55bd58b109f0 2024-03-21T11:29:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.032+0000 7f2102fa0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f81b1e10 con 0x55bd58b11bc0 2024-03-21T11:29:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.032+0000 7f2102fa0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f81b0af0 con 0x55bd58b12580 2024-03-21T11:29:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.032+0000 7f20fe200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b2d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.032+0000 7f20fce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b22110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.032+0000 7f20fd800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 0x55bd58b0f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.032+0000 7f20fce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b22110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43336/0 (socket says 172.21.15.47:43336) 2024-03-21T11:29:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.033+0000 7f20fd800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 0x55bd58b0f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34378/0 (socket says 172.21.15.47:34378) 2024-03-21T11:29:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.032+0000 7f20fe200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b2d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45410/0 (socket says 172.21.15.47:45410) 2024-03-21T11:29:30.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.033+0000 7f20fce00640 1 -- 172.21.15.47:0/4074214285 learned_addr learned my addr 172.21.15.47:0/4074214285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.033+0000 7f20fe200640 1 -- 172.21.15.47:0/4074214285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 msgr2=0x55bd58b22110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.033+0000 7f20fe200640 1 --2- 172.21.15.47:0/4074214285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b22110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.033+0000 7f20fe200640 1 -- 172.21.15.47:0/4074214285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 msgr2=0x55bd58b0f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.033+0000 7f20fe200640 1 --2- 172.21.15.47:0/4074214285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 0x55bd58b0f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.033+0000 7f20fe200640 1 -- 172.21.15.47:0/4074214285 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd58b11640 con 0x55bd58b12580 2024-03-21T11:29:30.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.033+0000 7f20fe200640 1 --2- 172.21.15.47:0/4074214285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b2d3d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f20f0098600 tx=0x7f20f00a4d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a5028f7e02715f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.033+0000 7f20ea000640 1 -- 172.21.15.47:0/4074214285 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f00980f0 con 0x55bd58b12580 2024-03-21T11:29:30.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.033+0000 7f20ea000640 1 -- 172.21.15.47:0/4074214285 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20f00a4720 con 0x55bd58b12580 2024-03-21T11:29:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.034+0000 7f20ea000640 1 -- 172.21.15.47:0/4074214285 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f00bc0c0 con 0x55bd58b12580 2024-03-21T11:29:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.034+0000 7f2102fa0740 1 -- 172.21.15.47:0/4074214285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 msgr2=0x55bd58b2d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.034+0000 7f2102fa0740 1 --2- 172.21.15.47:0/4074214285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b2d3d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f20f0098600 tx=0x7f20f00a4d30 comp rx=0 tx=0).stop 2024-03-21T11:29:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.034+0000 7f2102fa0740 1 -- 172.21.15.47:0/4074214285 shutdown_connections 2024-03-21T11:29:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.034+0000 7f2102fa0740 1 --2- 172.21.15.47:0/4074214285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b12580 0x55bd58b2d3d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.034+0000 7f2102fa0740 1 --2- 172.21.15.47:0/4074214285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b22110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.034+0000 7f2102fa0740 1 --2- 172.21.15.47:0/4074214285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b109f0 0x55bd58b0f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.035+0000 7f2102fa0740 1 -- 172.21.15.47:0/4074214285 >> 172.21.15.47:0/4074214285 conn(0x55bd58b0bde0 msgr2=0x55bd58b0f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.035+0000 7f2102fa0740 1 -- 172.21.15.47:0/4074214285 shutdown_connections 2024-03-21T11:29:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.035+0000 7f2102fa0740 1 -- 172.21.15.47:0/4074214285 wait complete. 2024-03-21T11:29:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.035+0000 7f2102fa0740 1 -- start start 2024-03-21T11:29:30.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.035+0000 7f2102fa0740 1 Processor -- start 2024-03-21T11:29:30.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.036+0000 7f2102fa0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b109f0 0x55bd58b21c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.036+0000 7f2102fa0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b1e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.036+0000 7f2102fa0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b12580 0x55bd58b1ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.036+0000 7f2102fa0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f81b17b0 con 0x55bd58b12580 2024-03-21T11:29:30.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.036+0000 7f2102fa0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f81b1150 con 0x55bd58b11bc0 2024-03-21T11:29:30.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.036+0000 7f2102fa0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f81ae650 con 0x55bd58b109f0 2024-03-21T11:29:30.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.036+0000 7f2102fa0740 10 client.0 _ll_register_callbacks cb 0x7f20f0005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:30.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.036+0000 7f20fce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b1e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.036+0000 7f20fd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b109f0 0x55bd58b21c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.036+0000 7f20fd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b109f0 0x55bd58b21c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45420/0 (socket says 172.21.15.47:45420) 2024-03-21T11:29:30.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.036+0000 7f20fd800640 1 -- 172.21.15.47:0/2571339781 learned_addr learned my addr 172.21.15.47:0/2571339781 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.036+0000 7f20fe200640 1 --2- 172.21.15.47:0/2571339781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b12580 0x55bd58b1ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.037+0000 7f20fd800640 1 -- 172.21.15.47:0/2571339781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 msgr2=0x55bd58b1e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.037+0000 7f20fd800640 1 --2- 172.21.15.47:0/2571339781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b1e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.037+0000 7f20fd800640 1 -- 172.21.15.47:0/2571339781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b12580 msgr2=0x55bd58b1ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.037+0000 7f20fd800640 1 --2- 172.21.15.47:0/2571339781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b12580 0x55bd58b1ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.037+0000 7f20fd800640 1 -- 172.21.15.47:0/2571339781 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd58b222c0 con 0x55bd58b109f0 2024-03-21T11:29:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.037+0000 7f20fce00640 1 --2- 172.21.15.47:0/2571339781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b1e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.037+0000 7f20fe200640 1 --2- 172.21.15.47:0/2571339781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b12580 0x55bd58b1ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:29:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.037+0000 7f20fd800640 1 --2- 172.21.15.47:0/2571339781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b109f0 0x55bd58b21c90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009900 tx=0x7f20ec005990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb98eea4bc925ae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.037+0000 7f20df400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:29:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.037+0000 7f20df400640 1 -- 172.21.15.47:0/2571339781 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec055060 con 0x55bd58b109f0 2024-03-21T11:29:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.037+0000 7f20df400640 1 -- 172.21.15.47:0/2571339781 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f20ec005040 con 0x55bd58b109f0 2024-03-21T11:29:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.037+0000 7f2102fa0740 1 -- 172.21.15.47:0/2571339781 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20f00a4720 con 0x55bd58b109f0 2024-03-21T11:29:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.038+0000 7f20df400640 1 -- 172.21.15.47:0/2571339781 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20ec05c050 con 0x55bd58b109f0 2024-03-21T11:29:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.038+0000 7f2102fa0740 10 client.27544 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.038+0000 7f2102fa0740 1 -- 172.21.15.47:0/2571339781 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bd58b82f20 con 0x55bd58b109f0 2024-03-21T11:29:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.038+0000 7f20df400640 1 -- 172.21.15.47:0/2571339781 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20f00a4720 con 0x55bd58b109f0 2024-03-21T11:29:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.038+0000 7f20df400640 1 -- 172.21.15.47:0/2571339781 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f20ec03f440 con 0x55bd58b109f0 2024-03-21T11:29:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.038+0000 7f2102fa0740 10 client.27544 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.038+0000 7f2102fa0740 10 client.27544 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:29:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.038+0000 7f2102fa0740 20 client.27544 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.038+0000 7f2102fa0740 1 --2- 172.21.15.47:0/2571339781 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55bd58b83530 0x55bd58b85920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.038+0000 7f2102fa0740 4 client.27544 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:29:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.038+0000 7f2102fa0740 1 -- 172.21.15.47:0/2571339781 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bd58a7cb50 con 0x55bd58b83530 2024-03-21T11:29:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.039+0000 7f20fce00640 1 --2- 172.21.15.47:0/2571339781 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55bd58b83530 0x55bd58b85920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.039+0000 7f20fce00640 1 --2- 172.21.15.47:0/2571339781 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55bd58b83530 0x55bd58b85920 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dd349f1adea44232 server_cookie=d828663d2802747e in_seq=0 out_seq=0 2024-03-21T11:29:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.039+0000 7f20df400640 10 client.27544 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:29:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.039+0000 7f20df400640 1 -- 172.21.15.47:0/2571339781 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bd58a7cb50 con 0x55bd58b83530 2024-03-21T11:29:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.039+0000 7f20df400640 10 client.27544 handle_command_reply: tid=0 2024-03-21T11:29:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.039+0000 7f2102fa0740 1 client.27544 shutdown 2024-03-21T11:29:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.040+0000 7f2102fa0740 1 -- 172.21.15.47:0/2571339781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b109f0 msgr2=0x55bd58b21c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.040+0000 7f2102fa0740 1 --2- 172.21.15.47:0/2571339781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b109f0 0x55bd58b21c90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009900 tx=0x7f20ec005990 comp rx=0 tx=0).stop 2024-03-21T11:29:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.040+0000 7f2102fa0740 1 -- 172.21.15.47:0/2571339781 shutdown_connections 2024-03-21T11:29:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.040+0000 7f2102fa0740 1 --2- 172.21.15.47:0/2571339781 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55bd58b83530 0x55bd58b85920 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.040+0000 7f2102fa0740 1 --2- 172.21.15.47:0/2571339781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bd58b12580 0x55bd58b1ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.040+0000 7f2102fa0740 1 --2- 172.21.15.47:0/2571339781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd58b11bc0 0x55bd58b1e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.040+0000 7f2102fa0740 1 --2- 172.21.15.47:0/2571339781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd58b109f0 0x55bd58b21c90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.040+0000 7f20df400640 0 client.27544 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:29:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.040+0000 7f2102fa0740 1 -- 172.21.15.47:0/2571339781 >> 172.21.15.47:0/2571339781 conn(0x55bd58b0bde0 msgr2=0x55bd58b0f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.040+0000 7f2102fa0740 1 -- 172.21.15.47:0/2571339781 shutdown_connections 2024-03-21T11:29:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.040+0000 7f2102fa0740 1 -- 172.21.15.47:0/2571339781 wait complete. 2024-03-21T11:29:30.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.040+0000 7f2102fa0740 20 client.27544 trim_cache size 0 max 16384 2024-03-21T11:29:30.044 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:29:30.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.041+0000 7f20fec00640 1 -- 172.21.15.47:0/3037483520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f20e00a2fa0 msgr2=0x7f20e00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.041+0000 7f20fec00640 1 --2- 172.21.15.47:0/3037483520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f20e00a2fa0 0x7f20e00a5460 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55bd58b11ac0 tx=0x7f20e4009530 comp rx=0 tx=0).stop 2024-03-21T11:29:30.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.041+0000 7f20fec00640 1 -- 172.21.15.47:0/3037483520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20f8103150 msgr2=0x7f20f8073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.041+0000 7f20fec00640 1 --2- 172.21.15.47:0/3037483520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20f8103150 0x7f20f8073fd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009fd0 tx=0x7f20f000ef10 comp rx=0 tx=0).stop 2024-03-21T11:29:30.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.041+0000 7f20fec00640 1 -- 172.21.15.47:0/3037483520 shutdown_connections 2024-03-21T11:29:30.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.041+0000 7f20fec00640 1 --2- 172.21.15.47:0/3037483520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f20e00a2fa0 0x7f20e00a5460 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.041+0000 7f20fec00640 1 --2- 172.21.15.47:0/3037483520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20f8103150 0x7f20f8073fd0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.041+0000 7f20fec00640 1 --2- 172.21.15.47:0/3037483520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20f8102790 0x7f20f8073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.041+0000 7f20fec00640 1 --2- 172.21.15.47:0/3037483520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20f81015a0 0x7f20f8075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.041+0000 7f20fec00640 1 -- 172.21.15.47:0/3037483520 >> 172.21.15.47:0/3037483520 conn(0x7f20f80fcb10 msgr2=0x7f20f8105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.041+0000 7f20fec00640 1 -- 172.21.15.47:0/3037483520 shutdown_connections 2024-03-21T11:29:30.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.041+0000 7f20fec00640 1 -- 172.21.15.47:0/3037483520 wait complete. 2024-03-21T11:29:30.059 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:29:30.059 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:29:30.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.281+0000 7fde6b200640 1 -- 172.21.15.47:0/4022208102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde6c1015a0 msgr2=0x7fde6c1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.281+0000 7fde6b200640 1 --2- 172.21.15.47:0/4022208102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde6c1015a0 0x7fde6c1019c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009900 tx=0x7fde5c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:29:30.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.281+0000 7fde6b200640 1 -- 172.21.15.47:0/4022208102 shutdown_connections 2024-03-21T11:29:30.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.281+0000 7fde6b200640 1 --2- 172.21.15.47:0/4022208102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde6c103150 0x7fde6c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.281+0000 7fde6b200640 1 --2- 172.21.15.47:0/4022208102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c102790 0x7fde6c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.281+0000 7fde6b200640 1 --2- 172.21.15.47:0/4022208102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde6c1015a0 0x7fde6c1019c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.281+0000 7fde6b200640 1 -- 172.21.15.47:0/4022208102 >> 172.21.15.47:0/4022208102 conn(0x7fde6c0fcb10 msgr2=0x7fde6c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.281+0000 7fde6b200640 1 -- 172.21.15.47:0/4022208102 shutdown_connections 2024-03-21T11:29:30.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.281+0000 7fde6b200640 1 -- 172.21.15.47:0/4022208102 wait complete. 2024-03-21T11:29:30.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.282+0000 7fde6b200640 1 Processor -- start 2024-03-21T11:29:30.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.283+0000 7fde6b200640 1 -- start start 2024-03-21T11:29:30.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.283+0000 7fde6b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde6c1015a0 0x7fde6c199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.283+0000 7fde6b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c102790 0x7fde6c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.283+0000 7fde6b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde6c103150 0x7fde6c1a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.283+0000 7fde6b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c10d240 con 0x7fde6c1015a0 2024-03-21T11:29:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.283+0000 7fde6b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c10d0c0 con 0x7fde6c103150 2024-03-21T11:29:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.283+0000 7fde6b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c10d3c0 con 0x7fde6c102790 2024-03-21T11:29:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.283+0000 7fde69400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c102790 0x7fde6c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.283+0000 7fde69e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde6c1015a0 0x7fde6c199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.283+0000 7fde6a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde6c103150 0x7fde6c1a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.283+0000 7fde69400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c102790 0x7fde6c19a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45432/0 (socket says 172.21.15.47:45432) 2024-03-21T11:29:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.283+0000 7fde69400640 1 -- 172.21.15.47:0/1246328868 learned_addr learned my addr 172.21.15.47:0/1246328868 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.283+0000 7fde69e00640 1 --2- 172.21.15.47:0/1246328868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde6c1015a0 0x7fde6c199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34404/0 (socket says 172.21.15.47:34404) 2024-03-21T11:29:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.284+0000 7fde69400640 1 -- 172.21.15.47:0/1246328868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde6c103150 msgr2=0x7fde6c1a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.284+0000 7fde69400640 1 --2- 172.21.15.47:0/1246328868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde6c103150 0x7fde6c1a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.284+0000 7fde69400640 1 -- 172.21.15.47:0/1246328868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde6c1015a0 msgr2=0x7fde6c199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.284+0000 7fde69400640 1 --2- 172.21.15.47:0/1246328868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde6c1015a0 0x7fde6c199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.284+0000 7fde69400640 1 -- 172.21.15.47:0/1246328868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c1a1740 con 0x7fde6c102790 2024-03-21T11:29:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.284+0000 7fde69400640 1 --2- 172.21.15.47:0/1246328868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c102790 0x7fde6c19a060 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fde5800ea60 tx=0x7fde5800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.284+0000 7fde62a00640 1 -- 172.21.15.47:0/1246328868 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5800cde0 con 0x7fde6c102790 2024-03-21T11:29:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.284+0000 7fde6b200640 1 -- 172.21.15.47:0/1246328868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c1a1a30 con 0x7fde6c102790 2024-03-21T11:29:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.284+0000 7fde62a00640 1 -- 172.21.15.47:0/1246328868 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde580035f0 con 0x7fde6c102790 2024-03-21T11:29:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.284+0000 7fde62a00640 1 -- 172.21.15.47:0/1246328868 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58010670 con 0x7fde6c102790 2024-03-21T11:29:30.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.284+0000 7fde6b200640 1 -- 172.21.15.47:0/1246328868 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c1a1fd0 con 0x7fde6c102790 2024-03-21T11:29:30.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.288+0000 7fde62a00640 1 -- 172.21.15.47:0/1246328868 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fde580037a0 con 0x7fde6c102790 2024-03-21T11:29:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.288+0000 7fde73999740 1 Processor -- start 2024-03-21T11:29:30.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.288+0000 7fde73999740 1 -- start start 2024-03-21T11:29:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.288+0000 7fde73999740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd69f0 0x560df6fd6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.288+0000 7fde73999740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd7bc0 0x560df6fd8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.288+0000 7fde73999740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd8580 0x560df6fdee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.289+0000 7fde73999740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df6f3b2c0 con 0x560df6fd7bc0 2024-03-21T11:29:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.289+0000 7fde73999740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df6ee3c90 con 0x560df6fd69f0 2024-03-21T11:29:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.289+0000 7fde73999740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df6f37360 con 0x560df6fd8580 2024-03-21T11:29:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.289+0000 7fde6a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd69f0 0x560df6fd6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.289+0000 7fde6a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd69f0 0x560df6fd6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43366/0 (socket says 172.21.15.47:43366) 2024-03-21T11:29:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.289+0000 7fde6a800640 1 -- 172.21.15.47:0/1561383172 learned_addr learned my addr 172.21.15.47:0/1561383172 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.289+0000 7fde69400640 1 --2- 172.21.15.47:0/1561383172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd8580 0x560df6fdee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.289+0000 7fde69e00640 1 --2- 172.21.15.47:0/1561383172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd7bc0 0x560df6fd8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.289+0000 7fde6a800640 1 -- 172.21.15.47:0/1561383172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd8580 msgr2=0x560df6fdee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.289+0000 7fde6a800640 1 --2- 172.21.15.47:0/1561383172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd8580 0x560df6fdee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.289+0000 7fde6a800640 1 -- 172.21.15.47:0/1561383172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd7bc0 msgr2=0x560df6fd8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.289+0000 7fde6a800640 1 --2- 172.21.15.47:0/1561383172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd7bc0 0x560df6fd8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.289+0000 7fde6a800640 1 -- 172.21.15.47:0/1561383172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560df6f08080 con 0x560df6fd69f0 2024-03-21T11:29:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.289+0000 7fde62a00640 1 --2- 172.21.15.47:0/1246328868 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fde500a2f20 0x7fde500a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.290+0000 7fde6a800640 1 --2- 172.21.15.47:0/1561383172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd69f0 0x560df6fd6e10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fde64009fd0 tx=0x7fde640118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ae418545e8b58e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.290+0000 7fde62a00640 1 -- 172.21.15.47:0/1246328868 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fde580cfac0 con 0x7fde6c102790 2024-03-21T11:29:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.290+0000 7fde69e00640 1 --2- 172.21.15.47:0/1246328868 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fde500a2f20 0x7fde500a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.290+0000 7fde57000640 1 -- 172.21.15.47:0/1561383172 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64004d20 con 0x560df6fd69f0 2024-03-21T11:29:30.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.290+0000 7fde57000640 1 -- 172.21.15.47:0/1561383172 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde64004050 con 0x560df6fd69f0 2024-03-21T11:29:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.290+0000 7fde57000640 1 -- 172.21.15.47:0/1561383172 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64005260 con 0x560df6fd69f0 2024-03-21T11:29:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.291+0000 7fde69e00640 1 --2- 172.21.15.47:0/1246328868 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fde500a2f20 0x7fde500a53e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fde5c02c170 tx=0x7fde5c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.291+0000 7fde73999740 1 -- 172.21.15.47:0/1561383172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd69f0 msgr2=0x560df6fd6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.291+0000 7fde73999740 1 --2- 172.21.15.47:0/1561383172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd69f0 0x560df6fd6e10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fde64009fd0 tx=0x7fde640118d0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.291+0000 7fde73999740 1 -- 172.21.15.47:0/1561383172 shutdown_connections 2024-03-21T11:29:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.291+0000 7fde73999740 1 --2- 172.21.15.47:0/1561383172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd8580 0x560df6fdee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.291+0000 7fde73999740 1 --2- 172.21.15.47:0/1561383172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd7bc0 0x560df6fd8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.291+0000 7fde73999740 1 --2- 172.21.15.47:0/1561383172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd69f0 0x560df6fd6e10 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.291+0000 7fde73999740 1 -- 172.21.15.47:0/1561383172 >> 172.21.15.47:0/1561383172 conn(0x560df6fd1de0 msgr2=0x560df6fd21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.291+0000 7fde73999740 1 -- 172.21.15.47:0/1561383172 shutdown_connections 2024-03-21T11:29:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.291+0000 7fde73999740 1 -- 172.21.15.47:0/1561383172 wait complete. 2024-03-21T11:29:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.291+0000 7fde73999740 1 -- start start 2024-03-21T11:29:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.292+0000 7fde73999740 1 Processor -- start 2024-03-21T11:29:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.292+0000 7fde73999740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd69f0 0x560df6fe0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.292+0000 7fde73999740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd7bc0 0x560df6fe11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.292+0000 7fde73999740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd8580 0x560df6fe8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.292+0000 7fde73999740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df6fdf590 con 0x560df6fd69f0 2024-03-21T11:29:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.292+0000 7fde73999740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df6ef70d0 con 0x560df6fd8580 2024-03-21T11:29:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.292+0000 7fde73999740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df6fdf710 con 0x560df6fd7bc0 2024-03-21T11:29:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.292+0000 7fde73999740 10 client.0 _ll_register_callbacks cb 0x560df6efbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.292+0000 7fde69400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd7bc0 0x560df6fe11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.292+0000 7fde69e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd69f0 0x560df6fe0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.292+0000 7fde69400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd7bc0 0x560df6fe11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45458/0 (socket says 172.21.15.47:45458) 2024-03-21T11:29:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.292+0000 7fde69400640 1 -- 172.21.15.47:0/2561754210 learned_addr learned my addr 172.21.15.47:0/2561754210 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.293+0000 7fde6a800640 1 --2- 172.21.15.47:0/2561754210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd8580 0x560df6fe8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.293+0000 7fde69400640 1 -- 172.21.15.47:0/2561754210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd8580 msgr2=0x560df6fe8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.293+0000 7fde69400640 1 --2- 172.21.15.47:0/2561754210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd8580 0x560df6fe8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.293+0000 7fde69400640 1 -- 172.21.15.47:0/2561754210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd69f0 msgr2=0x560df6fe0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.293+0000 7fde69400640 1 --2- 172.21.15.47:0/2561754210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd69f0 0x560df6fe0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.293+0000 7fde69400640 1 -- 172.21.15.47:0/2561754210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560df6c4f3f0 con 0x560df6fd7bc0 2024-03-21T11:29:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.293+0000 7fde69400640 1 --2- 172.21.15.47:0/2561754210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd7bc0 0x560df6fe11e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fde5809bd90 tx=0x7fde58009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1945285231a03aec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.293+0000 7fde4b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:29:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.293+0000 7fde4b400640 1 -- 172.21.15.47:0/2561754210 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5809b940 con 0x560df6fd7bc0 2024-03-21T11:29:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.293+0000 7fde4b400640 1 -- 172.21.15.47:0/2561754210 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fde58003b30 con 0x560df6fd7bc0 2024-03-21T11:29:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.293+0000 7fde73999740 1 -- 172.21.15.47:0/2561754210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde64004050 con 0x560df6fd7bc0 2024-03-21T11:29:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.294+0000 7fde4b400640 1 -- 172.21.15.47:0/2561754210 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde5809b760 con 0x560df6fd7bc0 2024-03-21T11:29:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.294+0000 7fde73999740 10 client.27556 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.294+0000 7fde73999740 1 -- 172.21.15.47:0/2561754210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560df6d197d0 con 0x560df6fd7bc0 2024-03-21T11:29:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.294+0000 7fde4b400640 1 -- 172.21.15.47:0/2561754210 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde64004050 con 0x560df6fd7bc0 2024-03-21T11:29:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.294+0000 7fde4b400640 1 -- 172.21.15.47:0/2561754210 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fde58098470 con 0x560df6fd7bc0 2024-03-21T11:29:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.294+0000 7fde73999740 10 client.27556 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.294+0000 7fde73999740 10 client.27556 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:29:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.294+0000 7fde73999740 20 client.27556 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.294+0000 7fde73999740 1 --2- 172.21.15.47:0/2561754210 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560df6fec650 0x560df6feeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.294+0000 7fde73999740 4 client.27556 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:29:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.294+0000 7fde73999740 1 -- 172.21.15.47:0/2561754210 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560df6f48800 con 0x560df6fec650 2024-03-21T11:29:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.294+0000 7fde69e00640 1 --2- 172.21.15.47:0/2561754210 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560df6fec650 0x560df6feeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.295+0000 7fde69e00640 1 --2- 172.21.15.47:0/2561754210 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560df6fec650 0x560df6feeab0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd399163a80e08ca server_cookie=8809da7fddaabe70 in_seq=0 out_seq=0 2024-03-21T11:29:30.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.295+0000 7fde4b400640 10 client.27556 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:29:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.296+0000 7fde4b400640 1 -- 172.21.15.47:0/2561754210 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560df6f48800 con 0x560df6fec650 2024-03-21T11:29:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.296+0000 7fde4b400640 10 client.27556 handle_command_reply: tid=0 2024-03-21T11:29:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.296+0000 7fde73999740 1 client.27556 shutdown 2024-03-21T11:29:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.296+0000 7fde73999740 1 -- 172.21.15.47:0/2561754210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd7bc0 msgr2=0x560df6fe11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.296+0000 7fde73999740 1 --2- 172.21.15.47:0/2561754210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd7bc0 0x560df6fe11e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fde5809bd90 tx=0x7fde58009550 comp rx=0 tx=0).stop 2024-03-21T11:29:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.296+0000 7fde73999740 1 -- 172.21.15.47:0/2561754210 shutdown_connections 2024-03-21T11:29:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.296+0000 7fde73999740 1 --2- 172.21.15.47:0/2561754210 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560df6fec650 0x560df6feeab0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.296+0000 7fde73999740 1 --2- 172.21.15.47:0/2561754210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd8580 0x560df6fe8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.296+0000 7fde73999740 1 --2- 172.21.15.47:0/2561754210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd7bc0 0x560df6fe11e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.296+0000 7fde4b400640 0 client.27556 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:29:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.296+0000 7fde73999740 1 --2- 172.21.15.47:0/2561754210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd69f0 0x560df6fe0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.296+0000 7fde73999740 1 -- 172.21.15.47:0/2561754210 >> 172.21.15.47:0/2561754210 conn(0x560df6fd1de0 msgr2=0x560df6fd1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.296+0000 7fde73999740 1 -- 172.21.15.47:0/2561754210 shutdown_connections 2024-03-21T11:29:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.297+0000 7fde73999740 1 -- 172.21.15.47:0/2561754210 wait complete. 2024-03-21T11:29:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.297+0000 7fde73999740 20 client.27556 trim_cache size 0 max 16384 2024-03-21T11:29:30.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.310+0000 7fde73999740 1 Processor -- start 2024-03-21T11:29:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.310+0000 7fde73999740 1 -- start start 2024-03-21T11:29:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde73999740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd69f0 0x560df6fd5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde73999740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd7bc0 0x560df6fe81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde73999740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd8580 0x560df6ff3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde73999740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1ac190 con 0x560df6fd8580 2024-03-21T11:29:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde73999740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1adb10 con 0x560df6fd7bc0 2024-03-21T11:29:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde73999740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1ac7f0 con 0x560df6fd69f0 2024-03-21T11:29:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde69e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd69f0 0x560df6fd5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde69400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd7bc0 0x560df6fe81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde69e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd69f0 0x560df6fd5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45468/0 (socket says 172.21.15.47:45468) 2024-03-21T11:29:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde69e00640 1 -- 172.21.15.47:0/3646214381 learned_addr learned my addr 172.21.15.47:0/3646214381 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde6a800640 1 --2- 172.21.15.47:0/3646214381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd8580 0x560df6ff3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde69e00640 1 -- 172.21.15.47:0/3646214381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd7bc0 msgr2=0x560df6fe81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde69e00640 1 --2- 172.21.15.47:0/3646214381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd7bc0 0x560df6fe81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde69e00640 1 -- 172.21.15.47:0/3646214381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd8580 msgr2=0x560df6ff3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde69e00640 1 --2- 172.21.15.47:0/3646214381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd8580 0x560df6ff3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde69e00640 1 -- 172.21.15.47:0/3646214381 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560df6fd65f0 con 0x560df6fd69f0 2024-03-21T11:29:30.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde6a800640 1 --2- 172.21.15.47:0/3646214381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd8580 0x560df6ff3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:29:30.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.311+0000 7fde69400640 1 --2- 172.21.15.47:0/3646214381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd7bc0 0x560df6fe81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:30.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.312+0000 7fde69e00640 1 --2- 172.21.15.47:0/3646214381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd69f0 0x560df6fd5750 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fde5c005ec0 tx=0x7fde5c042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d3781d599a9ac7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.312+0000 7fde56600640 1 -- 172.21.15.47:0/3646214381 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c046070 con 0x560df6fd69f0 2024-03-21T11:29:30.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.312+0000 7fde56600640 1 -- 172.21.15.47:0/3646214381 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde5c05b070 con 0x560df6fd69f0 2024-03-21T11:29:30.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.312+0000 7fde56600640 1 -- 172.21.15.47:0/3646214381 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c004ab0 con 0x560df6fd69f0 2024-03-21T11:29:30.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.313+0000 7fde73999740 1 -- 172.21.15.47:0/3646214381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd69f0 msgr2=0x560df6fd5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.313+0000 7fde73999740 1 --2- 172.21.15.47:0/3646214381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd69f0 0x560df6fd5750 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fde5c005ec0 tx=0x7fde5c042000 comp rx=0 tx=0).stop 2024-03-21T11:29:30.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.313+0000 7fde73999740 1 -- 172.21.15.47:0/3646214381 shutdown_connections 2024-03-21T11:29:30.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.313+0000 7fde73999740 1 --2- 172.21.15.47:0/3646214381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd8580 0x560df6ff3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.313+0000 7fde73999740 1 --2- 172.21.15.47:0/3646214381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd7bc0 0x560df6fe81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.313+0000 7fde73999740 1 --2- 172.21.15.47:0/3646214381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd69f0 0x560df6fd5750 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.313+0000 7fde73999740 1 -- 172.21.15.47:0/3646214381 >> 172.21.15.47:0/3646214381 conn(0x560df6fd1de0 msgr2=0x560df6fd5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.313+0000 7fde73999740 1 -- 172.21.15.47:0/3646214381 shutdown_connections 2024-03-21T11:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.313+0000 7fde73999740 1 -- 172.21.15.47:0/3646214381 wait complete. 2024-03-21T11:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.313+0000 7fde73999740 1 -- start start 2024-03-21T11:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.313+0000 7fde73999740 1 Processor -- start 2024-03-21T11:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.314+0000 7fde73999740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd69f0 0x560df6fe7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.314+0000 7fde73999740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd7bc0 0x560df6fe47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.314+0000 7fde73999740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd8580 0x560df6fe4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.314+0000 7fde73999740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1ad4b0 con 0x560df6fd69f0 2024-03-21T11:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.314+0000 7fde73999740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1ace50 con 0x560df6fd8580 2024-03-21T11:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.314+0000 7fde73999740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1aa350 con 0x560df6fd7bc0 2024-03-21T11:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.314+0000 7fde73999740 10 client.0 _ll_register_callbacks cb 0x7fde58003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.314+0000 7fde69e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd69f0 0x560df6fe7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.314+0000 7fde6a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd8580 0x560df6fe4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.314+0000 7fde6a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd8580 0x560df6fe4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43386/0 (socket says 172.21.15.47:43386) 2024-03-21T11:29:30.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.314+0000 7fde6a800640 1 -- 172.21.15.47:0/2973590101 learned_addr learned my addr 172.21.15.47:0/2973590101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.314+0000 7fde69400640 1 --2- 172.21.15.47:0/2973590101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd7bc0 0x560df6fe47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.315+0000 7fde6a800640 1 -- 172.21.15.47:0/2973590101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd7bc0 msgr2=0x560df6fe47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.315+0000 7fde6a800640 1 --2- 172.21.15.47:0/2973590101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd7bc0 0x560df6fe47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.315+0000 7fde6a800640 1 -- 172.21.15.47:0/2973590101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd69f0 msgr2=0x560df6fe7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.315+0000 7fde6a800640 1 --2- 172.21.15.47:0/2973590101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd69f0 0x560df6fe7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.315+0000 7fde6a800640 1 -- 172.21.15.47:0/2973590101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560df6fe8370 con 0x560df6fd8580 2024-03-21T11:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.315+0000 7fde69400640 1 --2- 172.21.15.47:0/2973590101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd7bc0 0x560df6fe47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.315+0000 7fde6a800640 1 --2- 172.21.15.47:0/2973590101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd8580 0x560df6fe4d20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fde64006010 tx=0x7fde64009d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=184e08b3f502fa9d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.315+0000 7fde4b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.315+0000 7fde4b400640 1 -- 172.21.15.47:0/2973590101 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64004aa0 con 0x560df6fd8580 2024-03-21T11:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.315+0000 7fde4b400640 1 -- 172.21.15.47:0/2973590101 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fde64004c40 con 0x560df6fd8580 2024-03-21T11:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.315+0000 7fde73999740 1 -- 172.21.15.47:0/2973590101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde5c05b070 con 0x560df6fd8580 2024-03-21T11:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.316+0000 7fde4b400640 1 -- 172.21.15.47:0/2973590101 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde64004e40 con 0x560df6fd8580 2024-03-21T11:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.316+0000 7fde73999740 10 client.27542 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.316+0000 7fde73999740 1 -- 172.21.15.47:0/2973590101 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560df7048e40 con 0x560df6fd8580 2024-03-21T11:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.316+0000 7fde4b400640 1 -- 172.21.15.47:0/2973590101 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde5c05b070 con 0x560df6fd8580 2024-03-21T11:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.316+0000 7fde4b400640 1 -- 172.21.15.47:0/2973590101 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fde640054b0 con 0x560df6fd8580 2024-03-21T11:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.316+0000 7fde73999740 10 client.27542 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.316+0000 7fde73999740 10 client.27542 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:29:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.316+0000 7fde73999740 20 client.27542 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.316+0000 7fde73999740 1 --2- 172.21.15.47:0/2973590101 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560df7049660 0x560df704ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.316+0000 7fde73999740 4 client.27542 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:29:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.316+0000 7fde73999740 1 -- 172.21.15.47:0/2973590101 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560df6f42b50 con 0x560df7049660 2024-03-21T11:29:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde69e00640 1 --2- 172.21.15.47:0/2973590101 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560df7049660 0x560df704ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde69e00640 1 --2- 172.21.15.47:0/2973590101 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560df7049660 0x560df704ba80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=148b7b8fd08cbe5a server_cookie=c7c1ae63da3cb0a6 in_seq=0 out_seq=0 2024-03-21T11:29:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde4b400640 10 client.27542 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:29:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde4b400640 1 -- 172.21.15.47:0/2973590101 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560df6f42b50 con 0x560df7049660 2024-03-21T11:29:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde4b400640 10 client.27542 handle_command_reply: tid=0 2024-03-21T11:29:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde73999740 1 client.27542 shutdown 2024-03-21T11:29:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde73999740 1 -- 172.21.15.47:0/2973590101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd8580 msgr2=0x560df6fe4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde73999740 1 --2- 172.21.15.47:0/2973590101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd8580 0x560df6fe4d20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fde64006010 tx=0x7fde64009d60 comp rx=0 tx=0).stop 2024-03-21T11:29:30.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde73999740 1 -- 172.21.15.47:0/2973590101 shutdown_connections 2024-03-21T11:29:30.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde73999740 1 --2- 172.21.15.47:0/2973590101 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560df7049660 0x560df704ba80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde73999740 1 --2- 172.21.15.47:0/2973590101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560df6fd8580 0x560df6fe4d20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde73999740 1 --2- 172.21.15.47:0/2973590101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560df6fd7bc0 0x560df6fe47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde73999740 1 --2- 172.21.15.47:0/2973590101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560df6fd69f0 0x560df6fe7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde4b400640 0 client.27542 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:29:30.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.317+0000 7fde73999740 1 -- 172.21.15.47:0/2973590101 >> 172.21.15.47:0/2973590101 conn(0x560df6fd1de0 msgr2=0x560df6fd58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.318+0000 7fde73999740 1 -- 172.21.15.47:0/2973590101 shutdown_connections 2024-03-21T11:29:30.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.318+0000 7fde73999740 1 -- 172.21.15.47:0/2973590101 wait complete. 2024-03-21T11:29:30.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.318+0000 7fde73999740 20 client.27542 trim_cache size 0 max 16384 2024-03-21T11:29:30.323 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:29:30.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.318+0000 7fde6b200640 1 -- 172.21.15.47:0/1246328868 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fde500a2f20 msgr2=0x7fde500a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.318+0000 7fde6b200640 1 --2- 172.21.15.47:0/1246328868 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fde500a2f20 0x7fde500a53e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fde5c02c170 tx=0x7fde5c02df90 comp rx=0 tx=0).stop 2024-03-21T11:29:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.318+0000 7fde6b200640 1 -- 172.21.15.47:0/1246328868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c102790 msgr2=0x7fde6c19a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.318+0000 7fde6b200640 1 --2- 172.21.15.47:0/1246328868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c102790 0x7fde6c19a060 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fde5800ea60 tx=0x7fde5800ef20 comp rx=0 tx=0).stop 2024-03-21T11:29:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.319+0000 7fde6b200640 1 -- 172.21.15.47:0/1246328868 shutdown_connections 2024-03-21T11:29:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.319+0000 7fde6b200640 1 --2- 172.21.15.47:0/1246328868 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fde500a2f20 0x7fde500a53e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.319+0000 7fde6b200640 1 --2- 172.21.15.47:0/1246328868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde6c103150 0x7fde6c1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.319+0000 7fde6b200640 1 --2- 172.21.15.47:0/1246328868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c102790 0x7fde6c19a060 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.319+0000 7fde6b200640 1 --2- 172.21.15.47:0/1246328868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde6c1015a0 0x7fde6c199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.319+0000 7fde6b200640 1 -- 172.21.15.47:0/1246328868 >> 172.21.15.47:0/1246328868 conn(0x7fde6c0fcb10 msgr2=0x7fde6c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.319+0000 7fde6b200640 1 -- 172.21.15.47:0/1246328868 shutdown_connections 2024-03-21T11:29:30.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.319+0000 7fde6b200640 1 -- 172.21.15.47:0/1246328868 wait complete. 2024-03-21T11:29:30.336 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:29:30.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:29:30.520 INFO:tasks.workunit.client.0.smithi047.stdout: 1 401447 8.29 MB/sec execute 134 sec latency 18.894 ms 2024-03-21T11:29:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.563+0000 7f31db800640 1 -- 172.21.15.47:0/2322530613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d410b490 msgr2=0x7f31d4114330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.563+0000 7f31db800640 1 --2- 172.21.15.47:0/2322530613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d410b490 0x7f31d4114330 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009ca0 tx=0x7f31cc02c0f0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.564+0000 7f31db800640 1 -- 172.21.15.47:0/2322530613 shutdown_connections 2024-03-21T11:29:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.564+0000 7f31db800640 1 --2- 172.21.15.47:0/2322530613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d410b490 0x7f31d4114330 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.564+0000 7f31db800640 1 --2- 172.21.15.47:0/2322530613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d410b010 0x7f31d4111de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.564+0000 7f31db800640 1 --2- 172.21.15.47:0/2322530613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d41086a0 0x7f31d410aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.564+0000 7f31db800640 1 -- 172.21.15.47:0/2322530613 >> 172.21.15.47:0/2322530613 conn(0x7f31d406e8a0 msgr2=0x7f31d4070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.564+0000 7f31db800640 1 -- 172.21.15.47:0/2322530613 shutdown_connections 2024-03-21T11:29:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.564+0000 7f31db800640 1 -- 172.21.15.47:0/2322530613 wait complete. 2024-03-21T11:29:30.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.565+0000 7f31db800640 1 Processor -- start 2024-03-21T11:29:30.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.565+0000 7f31db800640 1 -- start start 2024-03-21T11:29:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.565+0000 7f31db800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d41086a0 0x7f31d41a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.565+0000 7f31db800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d410b010 0x7f31d41a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.565+0000 7f31db800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d410b490 0x7f31d41a9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.565+0000 7f31db800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d4117d40 con 0x7f31d41086a0 2024-03-21T11:29:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.565+0000 7f31db800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d4117bc0 con 0x7f31d410b010 2024-03-21T11:29:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.565+0000 7f31db800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d4117ec0 con 0x7f31d410b490 2024-03-21T11:29:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.565+0000 7f31d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d410b010 0x7f31d41a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.565+0000 7f31d8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d41086a0 0x7f31d41a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.565+0000 7f31d9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d410b490 0x7f31d41a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.565+0000 7f31d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d410b010 0x7f31d41a28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43400/0 (socket says 172.21.15.47:43400) 2024-03-21T11:29:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.565+0000 7f31d3e00640 1 -- 172.21.15.47:0/3886114753 learned_addr learned my addr 172.21.15.47:0/3886114753 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.566+0000 7f31d9800640 1 -- 172.21.15.47:0/3886114753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d410b010 msgr2=0x7f31d41a28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.566+0000 7f31d9800640 1 --2- 172.21.15.47:0/3886114753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d410b010 0x7f31d41a28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.566+0000 7f31d9800640 1 -- 172.21.15.47:0/3886114753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d41086a0 msgr2=0x7f31d41a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.566+0000 7f31d9800640 1 --2- 172.21.15.47:0/3886114753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d41086a0 0x7f31d41a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.566+0000 7f31d9800640 1 -- 172.21.15.47:0/3886114753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d41a9fa0 con 0x7f31d410b490 2024-03-21T11:29:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.566+0000 7f31d3e00640 1 --2- 172.21.15.47:0/3886114753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d410b010 0x7f31d41a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.566+0000 7f31d9800640 1 --2- 172.21.15.47:0/3886114753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d410b490 0x7f31d41a9930 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f31cc037040 tx=0x7f31cc032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.567+0000 7f31d1600640 1 -- 172.21.15.47:0/3886114753 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc041020 con 0x7f31d410b490 2024-03-21T11:29:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.567+0000 7f31d1600640 1 -- 172.21.15.47:0/3886114753 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31cc034cd0 con 0x7f31d410b490 2024-03-21T11:29:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.567+0000 7f31db800640 1 -- 172.21.15.47:0/3886114753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d41aa230 con 0x7f31d410b490 2024-03-21T11:29:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.567+0000 7f31d1600640 1 -- 172.21.15.47:0/3886114753 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc039040 con 0x7f31d410b490 2024-03-21T11:29:30.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.567+0000 7f31db800640 1 -- 172.21.15.47:0/3886114753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d41aa740 con 0x7f31d410b490 2024-03-21T11:29:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.570+0000 7f31de5ac740 1 Processor -- start 2024-03-21T11:29:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.570+0000 7f31d1600640 1 -- 172.21.15.47:0/3886114753 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f31cc044020 con 0x7f31d410b490 2024-03-21T11:29:30.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.570+0000 7f31de5ac740 1 -- start start 2024-03-21T11:29:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.571+0000 7f31de5ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35d810 0x5607ae35dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.571+0000 7f31de5ac740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35e9e0 0x5607ae35ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.571+0000 7f31de5ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 0x5607ae365c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.571+0000 7f31de5ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ae2c22c0 con 0x5607ae35f3a0 2024-03-21T11:29:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.571+0000 7f31de5ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ae26ac90 con 0x5607ae35d810 2024-03-21T11:29:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.571+0000 7f31de5ac740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ae2be360 con 0x5607ae35e9e0 2024-03-21T11:29:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.571+0000 7f31d8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35e9e0 0x5607ae35ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.571+0000 7f31d8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35e9e0 0x5607ae35ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45506/0 (socket says 172.21.15.47:45506) 2024-03-21T11:29:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.571+0000 7f31d8e00640 1 -- 172.21.15.47:0/1265681736 learned_addr learned my addr 172.21.15.47:0/1265681736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.571+0000 7f31d3e00640 1 --2- 172.21.15.47:0/1265681736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35d810 0x5607ae35dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.571+0000 7f31d3e00640 1 --2- 172.21.15.47:0/1265681736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 0x5607ae365c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.571+0000 7f31d8e00640 1 -- 172.21.15.47:0/1265681736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35d810 msgr2=0x5607ae35dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.571+0000 7f31d8e00640 1 --2- 172.21.15.47:0/1265681736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35d810 0x5607ae35dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.572+0000 7f31d8e00640 1 -- 172.21.15.47:0/1265681736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 msgr2=0x5607ae365c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.572+0000 7f31d8e00640 1 --2- 172.21.15.47:0/1265681736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 0x5607ae365c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.572+0000 7f31d8e00640 1 -- 172.21.15.47:0/1265681736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607ae28f080 con 0x5607ae35e9e0 2024-03-21T11:29:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.572+0000 7f31d8e00640 1 --2- 172.21.15.47:0/1265681736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35e9e0 0x5607ae35ee60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009cf0 tx=0x7f31c8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9df39c3bd1604ae8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.572+0000 7f31d1600640 1 --2- 172.21.15.47:0/3886114753 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31bc0a2ff0 0x7f31bc0a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.572+0000 7f31c6000640 1 -- 172.21.15.47:0/1265681736 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8004230 con 0x5607ae35e9e0 2024-03-21T11:29:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.572+0000 7f31c6000640 1 -- 172.21.15.47:0/1265681736 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31c80043d0 con 0x5607ae35e9e0 2024-03-21T11:29:30.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.572+0000 7f31c6000640 1 -- 172.21.15.47:0/1265681736 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c80198c0 con 0x5607ae35e9e0 2024-03-21T11:29:30.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.572+0000 7f31d3e00640 1 --2- 172.21.15.47:0/3886114753 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31bc0a2ff0 0x7f31bc0a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.573+0000 7f31d1600640 1 -- 172.21.15.47:0/3886114753 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f31cc0f13d0 con 0x7f31d410b490 2024-03-21T11:29:30.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.573+0000 7f31de5ac740 1 -- 172.21.15.47:0/1265681736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35e9e0 msgr2=0x5607ae35ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.573+0000 7f31de5ac740 1 --2- 172.21.15.47:0/1265681736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35e9e0 0x5607ae35ee60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009cf0 tx=0x7f31c8011810 comp rx=0 tx=0).stop 2024-03-21T11:29:30.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.573+0000 7f31de5ac740 1 -- 172.21.15.47:0/1265681736 shutdown_connections 2024-03-21T11:29:30.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.573+0000 7f31de5ac740 1 --2- 172.21.15.47:0/1265681736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 0x5607ae365c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.573+0000 7f31de5ac740 1 --2- 172.21.15.47:0/1265681736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35e9e0 0x5607ae35ee60 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.573+0000 7f31de5ac740 1 --2- 172.21.15.47:0/1265681736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35d810 0x5607ae35dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.573+0000 7f31de5ac740 1 -- 172.21.15.47:0/1265681736 >> 172.21.15.47:0/1265681736 conn(0x5607ae358c00 msgr2=0x5607ae359000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.573+0000 7f31d3e00640 1 --2- 172.21.15.47:0/3886114753 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31bc0a2ff0 0x7f31bc0a54b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x5607ae35e8e0 tx=0x7f31c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.573+0000 7f31de5ac740 1 -- 172.21.15.47:0/1265681736 shutdown_connections 2024-03-21T11:29:30.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.573+0000 7f31de5ac740 1 -- 172.21.15.47:0/1265681736 wait complete. 2024-03-21T11:29:30.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.574+0000 7f31de5ac740 1 -- start start 2024-03-21T11:29:30.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.574+0000 7f31de5ac740 1 Processor -- start 2024-03-21T11:29:30.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.574+0000 7f31de5ac740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35d810 0x5607ae367a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.574+0000 7f31de5ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35e9e0 0x5607ae367f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.574+0000 7f31de5ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 0x5607ae36efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.574+0000 7f31de5ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ae3663b0 con 0x5607ae35f3a0 2024-03-21T11:29:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.575+0000 7f31de5ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ae27e0d0 con 0x5607ae35e9e0 2024-03-21T11:29:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.575+0000 7f31de5ac740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ae366530 con 0x5607ae35d810 2024-03-21T11:29:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.575+0000 7f31de5ac740 10 client.0 _ll_register_callbacks cb 0x5607ae282e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.575+0000 7f31d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35e9e0 0x5607ae367f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.575+0000 7f31d8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35d810 0x5607ae367a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.575+0000 7f31d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35e9e0 0x5607ae367f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43406/0 (socket says 172.21.15.47:43406) 2024-03-21T11:29:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.575+0000 7f31d3e00640 1 -- 172.21.15.47:0/3516314925 learned_addr learned my addr 172.21.15.47:0/3516314925 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.575+0000 7f31d9800640 1 --2- 172.21.15.47:0/3516314925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 0x5607ae36efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.575+0000 7f31d3e00640 1 -- 172.21.15.47:0/3516314925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35d810 msgr2=0x5607ae367a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.575+0000 7f31d3e00640 1 --2- 172.21.15.47:0/3516314925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35d810 0x5607ae367a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.575+0000 7f31d3e00640 1 -- 172.21.15.47:0/3516314925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 msgr2=0x5607ae36efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.575+0000 7f31d3e00640 1 --2- 172.21.15.47:0/3516314925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 0x5607ae36efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.575+0000 7f31d3e00640 1 -- 172.21.15.47:0/3516314925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607adfd63f0 con 0x5607ae35e9e0 2024-03-21T11:29:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.575+0000 7f31d8e00640 1 --2- 172.21.15.47:0/3516314925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35d810 0x5607ae367a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.576+0000 7f31d3e00640 1 --2- 172.21.15.47:0/3516314925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35e9e0 0x5607ae367f90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f31c0021700 tx=0x7f31c0009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2218fd2824c54bd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.576+0000 7f31baa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:29:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.576+0000 7f31baa00640 1 -- 172.21.15.47:0/3516314925 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0004020 con 0x5607ae35e9e0 2024-03-21T11:29:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.576+0000 7f31baa00640 1 -- 172.21.15.47:0/3516314925 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f31c00041c0 con 0x5607ae35e9e0 2024-03-21T11:29:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.576+0000 7f31de5ac740 1 -- 172.21.15.47:0/3516314925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31c80043d0 con 0x5607ae35e9e0 2024-03-21T11:29:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.576+0000 7f31baa00640 1 -- 172.21.15.47:0/3516314925 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31c0034050 con 0x5607ae35e9e0 2024-03-21T11:29:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.576+0000 7f31de5ac740 10 client.27554 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.576+0000 7f31de5ac740 1 -- 172.21.15.47:0/3516314925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5607ae0a07d0 con 0x5607ae35e9e0 2024-03-21T11:29:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.577+0000 7f31baa00640 1 -- 172.21.15.47:0/3516314925 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31c80043d0 con 0x5607ae35e9e0 2024-03-21T11:29:30.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.577+0000 7f31baa00640 1 -- 172.21.15.47:0/3516314925 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f31c0003820 con 0x5607ae35e9e0 2024-03-21T11:29:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.577+0000 7f31de5ac740 10 client.27554 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.577+0000 7f31de5ac740 10 client.27554 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:29:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.577+0000 7f31de5ac740 20 client.27554 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.577+0000 7f31de5ac740 1 --2- 172.21.15.47:0/3516314925 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5607ae373510 0x5607ae375980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.577+0000 7f31de5ac740 4 client.27554 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:29:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.577+0000 7f31de5ac740 1 -- 172.21.15.47:0/3516314925 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607ae2cf800 con 0x5607ae373510 2024-03-21T11:29:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.577+0000 7f31d8e00640 1 --2- 172.21.15.47:0/3516314925 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5607ae373510 0x5607ae375980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.578+0000 7f31d8e00640 1 --2- 172.21.15.47:0/3516314925 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5607ae373510 0x5607ae375980 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8187ddc877c5c438 server_cookie=48012ab969349694 in_seq=0 out_seq=0 2024-03-21T11:29:30.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.578+0000 7f31baa00640 10 client.27554 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:29:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.579+0000 7f31baa00640 1 -- 172.21.15.47:0/3516314925 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5607ae2cf800 con 0x5607ae373510 2024-03-21T11:29:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.579+0000 7f31baa00640 10 client.27554 handle_command_reply: tid=0 2024-03-21T11:29:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.579+0000 7f31de5ac740 1 client.27554 shutdown 2024-03-21T11:29:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.579+0000 7f31de5ac740 1 -- 172.21.15.47:0/3516314925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35e9e0 msgr2=0x5607ae367f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.579+0000 7f31de5ac740 1 --2- 172.21.15.47:0/3516314925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35e9e0 0x5607ae367f90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f31c0021700 tx=0x7f31c0009850 comp rx=0 tx=0).stop 2024-03-21T11:29:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.579+0000 7f31de5ac740 1 -- 172.21.15.47:0/3516314925 shutdown_connections 2024-03-21T11:29:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.579+0000 7f31de5ac740 1 --2- 172.21.15.47:0/3516314925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 0x5607ae36efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.579+0000 7f31de5ac740 1 --2- 172.21.15.47:0/3516314925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35e9e0 0x5607ae367f90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.579+0000 7f31de5ac740 1 --2- 172.21.15.47:0/3516314925 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5607ae373510 0x5607ae375980 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.579+0000 7f31de5ac740 1 --2- 172.21.15.47:0/3516314925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35d810 0x5607ae367a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.579+0000 7f31de5ac740 1 -- 172.21.15.47:0/3516314925 >> 172.21.15.47:0/3516314925 conn(0x5607ae358c00 msgr2=0x5607ae357eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.579+0000 7f31baa00640 0 client.27554 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:29:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.579+0000 7f31de5ac740 1 -- 172.21.15.47:0/3516314925 shutdown_connections 2024-03-21T11:29:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.579+0000 7f31de5ac740 1 -- 172.21.15.47:0/3516314925 wait complete. 2024-03-21T11:29:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.580+0000 7f31de5ac740 20 client.27554 trim_cache size 0 max 16384 2024-03-21T11:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.594+0000 7f31de5ac740 1 Processor -- start 2024-03-21T11:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.594+0000 7f31de5ac740 1 -- start start 2024-03-21T11:29:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.595+0000 7f31de5ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35d810 0x5607ae357620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.595+0000 7f31de5ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35e9e0 0x5607ae36ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.595+0000 7f31de5ac740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35f3a0 0x5607ae357b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.595+0000 7f31de5ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d41b4b10 con 0x5607ae35e9e0 2024-03-21T11:29:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.595+0000 7f31de5ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d41b6490 con 0x5607ae35d810 2024-03-21T11:29:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.595+0000 7f31de5ac740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d41b5170 con 0x5607ae35f3a0 2024-03-21T11:29:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.595+0000 7f31d8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35d810 0x5607ae357620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.595+0000 7f31d3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35e9e0 0x5607ae36ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.595+0000 7f31d8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35d810 0x5607ae357620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43408/0 (socket says 172.21.15.47:43408) 2024-03-21T11:29:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.595+0000 7f31d8e00640 1 -- 172.21.15.47:0/1142827563 learned_addr learned my addr 172.21.15.47:0/1142827563 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.595+0000 7f31d9800640 1 --2- 172.21.15.47:0/1142827563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35f3a0 0x5607ae357b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.596+0000 7f31d8e00640 1 -- 172.21.15.47:0/1142827563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35f3a0 msgr2=0x5607ae357b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.596+0000 7f31d8e00640 1 --2- 172.21.15.47:0/1142827563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35f3a0 0x5607ae357b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.596+0000 7f31d8e00640 1 -- 172.21.15.47:0/1142827563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35e9e0 msgr2=0x5607ae36ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.596+0000 7f31d8e00640 1 --2- 172.21.15.47:0/1142827563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35e9e0 0x5607ae36ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.596+0000 7f31d8e00640 1 -- 172.21.15.47:0/1142827563 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607ae35e460 con 0x5607ae35d810 2024-03-21T11:29:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.596+0000 7f31d8e00640 1 --2- 172.21.15.47:0/1142827563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35d810 0x5607ae357620 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009250 tx=0x7f31c8009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6263a3d36de1ce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.596+0000 7f31c5600640 1 -- 172.21.15.47:0/1142827563 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8031070 con 0x5607ae35d810 2024-03-21T11:29:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.596+0000 7f31c5600640 1 -- 172.21.15.47:0/1142827563 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31c8003ad0 con 0x5607ae35d810 2024-03-21T11:29:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.596+0000 7f31c5600640 1 -- 172.21.15.47:0/1142827563 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c802ba40 con 0x5607ae35d810 2024-03-21T11:29:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.597+0000 7f31de5ac740 1 -- 172.21.15.47:0/1142827563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35d810 msgr2=0x5607ae357620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.597+0000 7f31de5ac740 1 --2- 172.21.15.47:0/1142827563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35d810 0x5607ae357620 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009250 tx=0x7f31c8009640 comp rx=0 tx=0).stop 2024-03-21T11:29:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.597+0000 7f31de5ac740 1 -- 172.21.15.47:0/1142827563 shutdown_connections 2024-03-21T11:29:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.597+0000 7f31de5ac740 1 --2- 172.21.15.47:0/1142827563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35f3a0 0x5607ae357b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.597+0000 7f31de5ac740 1 --2- 172.21.15.47:0/1142827563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35e9e0 0x5607ae36ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.597+0000 7f31de5ac740 1 --2- 172.21.15.47:0/1142827563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35d810 0x5607ae357620 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.597+0000 7f31de5ac740 1 -- 172.21.15.47:0/1142827563 >> 172.21.15.47:0/1142827563 conn(0x5607ae358c00 msgr2=0x5607ae367a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.597+0000 7f31de5ac740 1 -- 172.21.15.47:0/1142827563 shutdown_connections 2024-03-21T11:29:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.597+0000 7f31de5ac740 1 -- 172.21.15.47:0/1142827563 wait complete. 2024-03-21T11:29:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.598+0000 7f31de5ac740 1 -- start start 2024-03-21T11:29:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.598+0000 7f31de5ac740 1 Processor -- start 2024-03-21T11:29:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.598+0000 7f31de5ac740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35d810 0x5607ae36a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.598+0000 7f31de5ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35e9e0 0x5607ae36aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.598+0000 7f31de5ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 0x5607ae36da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.598+0000 7f31de5ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d41b5e30 con 0x5607ae35f3a0 2024-03-21T11:29:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.598+0000 7f31de5ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d41b57d0 con 0x5607ae35e9e0 2024-03-21T11:29:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.598+0000 7f31de5ac740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d41b2cd0 con 0x5607ae35d810 2024-03-21T11:29:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.599+0000 7f31de5ac740 10 client.0 _ll_register_callbacks cb 0x7f31c000fda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:30.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.599+0000 7f31d8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35d810 0x5607ae36a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.599+0000 7f31d8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35d810 0x5607ae36a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45532/0 (socket says 172.21.15.47:45532) 2024-03-21T11:29:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.599+0000 7f31d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35e9e0 0x5607ae36aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.599+0000 7f31d8e00640 1 -- 172.21.15.47:0/3098322822 learned_addr learned my addr 172.21.15.47:0/3098322822 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.599+0000 7f31d9800640 1 --2- 172.21.15.47:0/3098322822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 0x5607ae36da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.599+0000 7f31d3e00640 1 -- 172.21.15.47:0/3098322822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35d810 msgr2=0x5607ae36a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.599+0000 7f31d3e00640 1 --2- 172.21.15.47:0/3098322822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35d810 0x5607ae36a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.599+0000 7f31d3e00640 1 -- 172.21.15.47:0/3098322822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 msgr2=0x5607ae36da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.599+0000 7f31d3e00640 1 --2- 172.21.15.47:0/3098322822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 0x5607ae36da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.599+0000 7f31d3e00640 1 -- 172.21.15.47:0/3098322822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607ae36dff0 con 0x5607ae35e9e0 2024-03-21T11:29:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.599+0000 7f31d8e00640 1 --2- 172.21.15.47:0/3098322822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35d810 0x5607ae36a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.599+0000 7f31d3e00640 1 --2- 172.21.15.47:0/3098322822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35e9e0 0x5607ae36aa50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f31c00216d0 tx=0x7f31c001ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc03e31ef665211 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.599+0000 7f31baa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:29:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.600+0000 7f31baa00640 1 -- 172.21.15.47:0/3098322822 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c00035f0 con 0x5607ae35e9e0 2024-03-21T11:29:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.600+0000 7f31baa00640 1 -- 172.21.15.47:0/3098322822 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f31c0003790 con 0x5607ae35e9e0 2024-03-21T11:29:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.600+0000 7f31de5ac740 1 -- 172.21.15.47:0/3098322822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31c8003ad0 con 0x5607ae35e9e0 2024-03-21T11:29:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.600+0000 7f31baa00640 1 -- 172.21.15.47:0/3098322822 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31c0003990 con 0x5607ae35e9e0 2024-03-21T11:29:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.600+0000 7f31de5ac740 10 client.27560 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.600+0000 7f31de5ac740 1 -- 172.21.15.47:0/3098322822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5607ae3d7620 con 0x5607ae35e9e0 2024-03-21T11:29:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.600+0000 7f31baa00640 1 -- 172.21.15.47:0/3098322822 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31c8003ad0 con 0x5607ae35e9e0 2024-03-21T11:29:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.600+0000 7f31baa00640 1 -- 172.21.15.47:0/3098322822 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f31c00041a0 con 0x5607ae35e9e0 2024-03-21T11:29:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.600+0000 7f31de5ac740 10 client.27560 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:30.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.600+0000 7f31de5ac740 10 client.27560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:29:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.601+0000 7f31de5ac740 20 client.27560 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.601+0000 7f31de5ac740 1 --2- 172.21.15.47:0/3098322822 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5607ae3d7e10 0x5607ae3da230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.601+0000 7f31de5ac740 4 client.27560 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:29:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.601+0000 7f31de5ac740 1 -- 172.21.15.47:0/3098322822 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5607ae2c9b50 con 0x5607ae3d7e10 2024-03-21T11:29:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.601+0000 7f31d8e00640 1 --2- 172.21.15.47:0/3098322822 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5607ae3d7e10 0x5607ae3da230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31baa00640 10 client.27560 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:29:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31d8e00640 1 --2- 172.21.15.47:0/3098322822 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5607ae3d7e10 0x5607ae3da230 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f21d9b3f23247a server_cookie=2423e5e25103c1b2 in_seq=0 out_seq=0 2024-03-21T11:29:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31baa00640 1 -- 172.21.15.47:0/3098322822 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5607ae2c9b50 con 0x5607ae3d7e10 2024-03-21T11:29:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31baa00640 10 client.27560 handle_command_reply: tid=0 2024-03-21T11:29:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31de5ac740 1 client.27560 shutdown 2024-03-21T11:29:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31de5ac740 1 -- 172.21.15.47:0/3098322822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35e9e0 msgr2=0x5607ae36aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31de5ac740 1 --2- 172.21.15.47:0/3098322822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35e9e0 0x5607ae36aa50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f31c00216d0 tx=0x7f31c001ee40 comp rx=0 tx=0).stop 2024-03-21T11:29:30.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31de5ac740 1 -- 172.21.15.47:0/3098322822 shutdown_connections 2024-03-21T11:29:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31de5ac740 1 --2- 172.21.15.47:0/3098322822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607ae35f3a0 0x5607ae36da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31de5ac740 1 --2- 172.21.15.47:0/3098322822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607ae35e9e0 0x5607ae36aa50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31de5ac740 1 --2- 172.21.15.47:0/3098322822 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5607ae3d7e10 0x5607ae3da230 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31de5ac740 1 --2- 172.21.15.47:0/3098322822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607ae35d810 0x5607ae36a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31de5ac740 1 -- 172.21.15.47:0/3098322822 >> 172.21.15.47:0/3098322822 conn(0x5607ae358c00 msgr2=0x5607ae35be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31baa00640 0 client.27560 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:29:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31de5ac740 1 -- 172.21.15.47:0/3098322822 shutdown_connections 2024-03-21T11:29:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31de5ac740 1 -- 172.21.15.47:0/3098322822 wait complete. 2024-03-21T11:29:30.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.602+0000 7f31de5ac740 20 client.27560 trim_cache size 0 max 16384 2024-03-21T11:29:30.606 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:29:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.603+0000 7f31db800640 1 -- 172.21.15.47:0/3886114753 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31bc0a2ff0 msgr2=0x7f31bc0a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.603+0000 7f31db800640 1 --2- 172.21.15.47:0/3886114753 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31bc0a2ff0 0x7f31bc0a54b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x5607ae35e8e0 tx=0x7f31c0005f90 comp rx=0 tx=0).stop 2024-03-21T11:29:30.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.603+0000 7f31db800640 1 -- 172.21.15.47:0/3886114753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d410b490 msgr2=0x7f31d41a9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.603+0000 7f31db800640 1 --2- 172.21.15.47:0/3886114753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d410b490 0x7f31d41a9930 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f31cc037040 tx=0x7f31cc032ea0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.603+0000 7f31db800640 1 -- 172.21.15.47:0/3886114753 shutdown_connections 2024-03-21T11:29:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.603+0000 7f31db800640 1 --2- 172.21.15.47:0/3886114753 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31bc0a2ff0 0x7f31bc0a54b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.603+0000 7f31db800640 1 --2- 172.21.15.47:0/3886114753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d410b490 0x7f31d41a9930 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.603+0000 7f31db800640 1 --2- 172.21.15.47:0/3886114753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31d410b010 0x7f31d41a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.603+0000 7f31db800640 1 --2- 172.21.15.47:0/3886114753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31d41086a0 0x7f31d41a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.603+0000 7f31db800640 1 -- 172.21.15.47:0/3886114753 >> 172.21.15.47:0/3886114753 conn(0x7f31d406e8a0 msgr2=0x7f31d406f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.604+0000 7f31db800640 1 -- 172.21.15.47:0/3886114753 shutdown_connections 2024-03-21T11:29:30.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.604+0000 7f31db800640 1 -- 172.21.15.47:0/3886114753 wait complete. 2024-03-21T11:29:30.622 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:29:30.622 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:29:30.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.850+0000 7f081fc00640 1 -- 172.21.15.47:0/1246874475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0818102140 msgr2=0x7f0818104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.850+0000 7f081fc00640 1 --2- 172.21.15.47:0/1246874475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0818102140 0x7f0818104530 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f080c009900 tx=0x7f080c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:29:30.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.850+0000 7f081fc00640 1 -- 172.21.15.47:0/1246874475 shutdown_connections 2024-03-21T11:29:30.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.850+0000 7f081fc00640 1 --2- 172.21.15.47:0/1246874475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08181073a0 0x7f08181097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.850+0000 7f081fc00640 1 --2- 172.21.15.47:0/1246874475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0818104a70 0x7f0818106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.850+0000 7f081fc00640 1 --2- 172.21.15.47:0/1246874475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0818102140 0x7f0818104530 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.850+0000 7f081fc00640 1 -- 172.21.15.47:0/1246874475 >> 172.21.15.47:0/1246874475 conn(0x7f08180fbb30 msgr2=0x7f08180fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.850+0000 7f081fc00640 1 -- 172.21.15.47:0/1246874475 shutdown_connections 2024-03-21T11:29:30.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.850+0000 7f081fc00640 1 -- 172.21.15.47:0/1246874475 wait complete. 2024-03-21T11:29:30.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.851+0000 7f081fc00640 1 Processor -- start 2024-03-21T11:29:30.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.851+0000 7f081fc00640 1 -- start start 2024-03-21T11:29:30.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.852+0000 7f081fc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0818102140 0x7f0818199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.852+0000 7f081fc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0818104a70 0x7f081819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.852+0000 7f081fc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08181073a0 0x7f08181a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.852+0000 7f081fc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081810d3c0 con 0x7f0818102140 2024-03-21T11:29:30.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.852+0000 7f081fc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081810d240 con 0x7f0818104a70 2024-03-21T11:29:30.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.852+0000 7f081fc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081810d540 con 0x7f08181073a0 2024-03-21T11:29:30.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.852+0000 7f0817e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0818104a70 0x7f081819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.852+0000 7f0817e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0818104a70 0x7f081819a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43432/0 (socket says 172.21.15.47:43432) 2024-03-21T11:29:30.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.852+0000 7f0817e00640 1 -- 172.21.15.47:0/1819342018 learned_addr learned my addr 172.21.15.47:0/1819342018 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.852+0000 7f081d200640 1 --2- 172.21.15.47:0/1819342018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0818102140 0x7f0818199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.852+0000 7f081dc00640 1 --2- 172.21.15.47:0/1819342018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08181073a0 0x7f08181a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.853+0000 7f0817e00640 1 -- 172.21.15.47:0/1819342018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08181073a0 msgr2=0x7f08181a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.853+0000 7f0817e00640 1 --2- 172.21.15.47:0/1819342018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08181073a0 0x7f08181a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.853+0000 7f0817e00640 1 -- 172.21.15.47:0/1819342018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0818102140 msgr2=0x7f0818199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.853+0000 7f0817e00640 1 --2- 172.21.15.47:0/1819342018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0818102140 0x7f0818199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.853+0000 7f0817e00640 1 -- 172.21.15.47:0/1819342018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08181a18d0 con 0x7f0818104a70 2024-03-21T11:29:30.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.853+0000 7f0817e00640 1 --2- 172.21.15.47:0/1819342018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0818104a70 0x7f081819a1d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f080400e9f0 tx=0x7f080400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.853+0000 7f0815600640 1 -- 172.21.15.47:0/1819342018 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080400ccc0 con 0x7f0818104a70 2024-03-21T11:29:30.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.854+0000 7f0815600640 1 -- 172.21.15.47:0/1819342018 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0804005500 con 0x7f0818104a70 2024-03-21T11:29:30.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.854+0000 7f081fc00640 1 -- 172.21.15.47:0/1819342018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08181a1bc0 con 0x7f0818104a70 2024-03-21T11:29:30.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.854+0000 7f0815600640 1 -- 172.21.15.47:0/1819342018 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804010690 con 0x7f0818104a70 2024-03-21T11:29:30.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.854+0000 7f081fc00640 1 -- 172.21.15.47:0/1819342018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08181a20e0 con 0x7f0818104a70 2024-03-21T11:29:30.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.857+0000 7f0815600640 1 -- 172.21.15.47:0/1819342018 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f08040046c0 con 0x7f0818104a70 2024-03-21T11:29:30.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.858+0000 7f0822972740 1 Processor -- start 2024-03-21T11:29:30.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.858+0000 7f0822972740 1 -- start start 2024-03-21T11:29:30.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.858+0000 7f0822972740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.858+0000 7f0822972740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441cf9e0 0x556d441cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.858+0000 7f0822972740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441d03a0 0x556d441d6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.858+0000 7f0822972740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d441332c0 con 0x556d441d03a0 2024-03-21T11:29:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.858+0000 7f0822972740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d440dbc90 con 0x556d441cf9e0 2024-03-21T11:29:30.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.858+0000 7f0822972740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d4412f360 con 0x556d441ce810 2024-03-21T11:29:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.859+0000 7f081dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.859+0000 7f081dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45554/0 (socket says 172.21.15.47:45554) 2024-03-21T11:29:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.859+0000 7f081dc00640 1 -- 172.21.15.47:0/2821391385 learned_addr learned my addr 172.21.15.47:0/2821391385 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.859+0000 7f081d200640 1 --2- 172.21.15.47:0/2821391385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441cf9e0 0x556d441cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.859+0000 7f081dc00640 1 -- 172.21.15.47:0/2821391385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441cf9e0 msgr2=0x556d441cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.859+0000 7f081dc00640 1 --2- 172.21.15.47:0/2821391385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441cf9e0 0x556d441cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.859+0000 7f081dc00640 1 -- 172.21.15.47:0/2821391385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441d03a0 msgr2=0x556d441d6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.859+0000 7f081dc00640 1 --2- 172.21.15.47:0/2821391385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441d03a0 0x556d441d6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.859+0000 7f081dc00640 1 -- 172.21.15.47:0/2821391385 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d44100080 con 0x556d441ce810 2024-03-21T11:29:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.860+0000 7f081dc00640 1 --2- 172.21.15.47:0/2821391385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441cec30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0810010370 tx=0x7f0810010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d790b4dc5c648ad8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.860+0000 7f0815600640 1 --2- 172.21.15.47:0/1819342018 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f08000a2fa0 0x7f08000a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.860+0000 7f080a000640 1 -- 172.21.15.47:0/2821391385 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810016070 con 0x556d441ce810 2024-03-21T11:29:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.860+0000 7f080a000640 1 -- 172.21.15.47:0/2821391385 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0810004d20 con 0x556d441ce810 2024-03-21T11:29:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.860+0000 7f080a000640 1 -- 172.21.15.47:0/2821391385 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810005480 con 0x556d441ce810 2024-03-21T11:29:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.860+0000 7f0815600640 1 -- 172.21.15.47:0/1819342018 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f08040cfa10 con 0x7f0818104a70 2024-03-21T11:29:30.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.860+0000 7f081d200640 1 --2- 172.21.15.47:0/1819342018 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f08000a2fa0 0x7f08000a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.861+0000 7f0822972740 1 -- 172.21.15.47:0/2821391385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 msgr2=0x556d441cec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.861+0000 7f0822972740 1 --2- 172.21.15.47:0/2821391385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441cec30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0810010370 tx=0x7f0810010830 comp rx=0 tx=0).stop 2024-03-21T11:29:30.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.861+0000 7f0822972740 1 -- 172.21.15.47:0/2821391385 shutdown_connections 2024-03-21T11:29:30.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.861+0000 7f0822972740 1 --2- 172.21.15.47:0/2821391385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441d03a0 0x556d441d6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.861+0000 7f0822972740 1 --2- 172.21.15.47:0/2821391385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441cf9e0 0x556d441cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.861+0000 7f0822972740 1 --2- 172.21.15.47:0/2821391385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441cec30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.861+0000 7f0822972740 1 -- 172.21.15.47:0/2821391385 >> 172.21.15.47:0/2821391385 conn(0x556d441c9c00 msgr2=0x556d441ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.861+0000 7f081d200640 1 --2- 172.21.15.47:0/1819342018 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f08000a2fa0 0x7f08000a5460 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f080c000c00 tx=0x7f080c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.861+0000 7f0822972740 1 -- 172.21.15.47:0/2821391385 shutdown_connections 2024-03-21T11:29:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.861+0000 7f0822972740 1 -- 172.21.15.47:0/2821391385 wait complete. 2024-03-21T11:29:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.862+0000 7f0822972740 1 -- start start 2024-03-21T11:29:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.862+0000 7f0822972740 1 Processor -- start 2024-03-21T11:29:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0822972740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441ce810 0x556d441d8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0822972740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441cf9e0 0x556d441d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0822972740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441d03a0 0x556d441e00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0822972740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d441d73b0 con 0x556d441ce810 2024-03-21T11:29:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0822972740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d440ef0d0 con 0x556d441d03a0 2024-03-21T11:29:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0822972740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d441d7530 con 0x556d441cf9e0 2024-03-21T11:29:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0822972740 10 client.0 _ll_register_callbacks cb 0x556d440f3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0817e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441cf9e0 0x556d441d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f081dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441d03a0 0x556d441e00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0817e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441cf9e0 0x556d441d9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45556/0 (socket says 172.21.15.47:45556) 2024-03-21T11:29:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0817e00640 1 -- 172.21.15.47:0/2077871051 learned_addr learned my addr 172.21.15.47:0/2077871051 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f081d200640 1 --2- 172.21.15.47:0/2077871051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441ce810 0x556d441d8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0817e00640 1 -- 172.21.15.47:0/2077871051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441d03a0 msgr2=0x556d441e00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0817e00640 1 --2- 172.21.15.47:0/2077871051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441d03a0 0x556d441e00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0817e00640 1 -- 172.21.15.47:0/2077871051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441ce810 msgr2=0x556d441d8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0817e00640 1 --2- 172.21.15.47:0/2077871051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441ce810 0x556d441d8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f0817e00640 1 -- 172.21.15.47:0/2077871051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d43e473f0 con 0x556d441cf9e0 2024-03-21T11:29:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.863+0000 7f081dc00640 1 --2- 172.21.15.47:0/2077871051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441d03a0 0x556d441e00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.864+0000 7f0817e00640 1 --2- 172.21.15.47:0/2077871051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441cf9e0 0x556d441d9090 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f080409bab0 tx=0x7f080409b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2217dbce106099d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.864+0000 7f07fea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:29:30.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.864+0000 7f07fea00640 1 -- 172.21.15.47:0/2077871051 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08040a4530 con 0x556d441cf9e0 2024-03-21T11:29:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.864+0000 7f07fea00640 1 -- 172.21.15.47:0/2077871051 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f080409b200 con 0x556d441cf9e0 2024-03-21T11:29:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.864+0000 7f0822972740 1 -- 172.21.15.47:0/2077871051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0810004d20 con 0x556d441cf9e0 2024-03-21T11:29:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.864+0000 7f07fea00640 1 -- 172.21.15.47:0/2077871051 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f080409bdb0 con 0x556d441cf9e0 2024-03-21T11:29:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.864+0000 7f0822972740 10 client.27592 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.864+0000 7f0822972740 1 -- 172.21.15.47:0/2077871051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556d43f117d0 con 0x556d441cf9e0 2024-03-21T11:29:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.865+0000 7f07fea00640 1 -- 172.21.15.47:0/2077871051 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0810004d20 con 0x556d441cf9e0 2024-03-21T11:29:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.865+0000 7f07fea00640 1 -- 172.21.15.47:0/2077871051 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0804097b00 con 0x556d441cf9e0 2024-03-21T11:29:30.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.865+0000 7f0822972740 10 client.27592 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.865+0000 7f0822972740 10 client.27592 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:29:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.865+0000 7f0822972740 20 client.27592 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.865+0000 7f0822972740 1 --2- 172.21.15.47:0/2077871051 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556d441e4650 0x556d441e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.865+0000 7f0822972740 4 client.27592 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:29:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.865+0000 7f0822972740 1 -- 172.21.15.47:0/2077871051 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d44140800 con 0x556d441e4650 2024-03-21T11:29:30.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.865+0000 7f081d200640 1 --2- 172.21.15.47:0/2077871051 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556d441e4650 0x556d441e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.866+0000 7f081d200640 1 --2- 172.21.15.47:0/2077871051 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556d441e4650 0x556d441e6a40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbb2bef239641752 server_cookie=379ff15cecbf183d in_seq=0 out_seq=0 2024-03-21T11:29:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.866+0000 7f07fea00640 10 client.27592 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:29:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.867+0000 7f07fea00640 1 -- 172.21.15.47:0/2077871051 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556d44140800 con 0x556d441e4650 2024-03-21T11:29:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.867+0000 7f07fea00640 10 client.27592 handle_command_reply: tid=0 2024-03-21T11:29:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.867+0000 7f0822972740 1 client.27592 shutdown 2024-03-21T11:29:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.867+0000 7f0822972740 1 -- 172.21.15.47:0/2077871051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441cf9e0 msgr2=0x556d441d9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.867+0000 7f0822972740 1 --2- 172.21.15.47:0/2077871051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441cf9e0 0x556d441d9090 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f080409bab0 tx=0x7f080409b710 comp rx=0 tx=0).stop 2024-03-21T11:29:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.867+0000 7f0822972740 1 -- 172.21.15.47:0/2077871051 shutdown_connections 2024-03-21T11:29:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.867+0000 7f0822972740 1 --2- 172.21.15.47:0/2077871051 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556d441e4650 0x556d441e6a40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.867+0000 7f0822972740 1 --2- 172.21.15.47:0/2077871051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441d03a0 0x556d441e00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.867+0000 7f0822972740 1 --2- 172.21.15.47:0/2077871051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441cf9e0 0x556d441d9090 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.867+0000 7f0822972740 1 --2- 172.21.15.47:0/2077871051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441ce810 0x556d441d8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.867+0000 7f07fea00640 0 client.27592 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:29:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.867+0000 7f0822972740 1 -- 172.21.15.47:0/2077871051 >> 172.21.15.47:0/2077871051 conn(0x556d441c9c00 msgr2=0x556d441c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.868+0000 7f0822972740 1 -- 172.21.15.47:0/2077871051 shutdown_connections 2024-03-21T11:29:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.868+0000 7f0822972740 1 -- 172.21.15.47:0/2077871051 wait complete. 2024-03-21T11:29:30.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.868+0000 7f0822972740 20 client.27592 trim_cache size 0 max 16384 2024-03-21T11:29:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.882+0000 7f0822972740 1 Processor -- start 2024-03-21T11:29:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.883+0000 7f0822972740 1 -- start start 2024-03-21T11:29:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.883+0000 7f0822972740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441eb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.883+0000 7f0822972740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441cf9e0 0x556d441e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.883+0000 7f0822972740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441d03a0 0x556d441eb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.883+0000 7f0822972740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08181ac2e0 con 0x556d441cf9e0 2024-03-21T11:29:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.883+0000 7f0822972740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08181adc60 con 0x556d441d03a0 2024-03-21T11:29:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.883+0000 7f0822972740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08181ac940 con 0x556d441ce810 2024-03-21T11:29:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.883+0000 7f0817e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441cf9e0 0x556d441e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.883+0000 7f081dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441d03a0 0x556d441eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.883+0000 7f0817e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441cf9e0 0x556d441e0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34572/0 (socket says 172.21.15.47:34572) 2024-03-21T11:29:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.883+0000 7f0817e00640 1 -- 172.21.15.47:0/1415038779 learned_addr learned my addr 172.21.15.47:0/1415038779 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.883+0000 7f081d200640 1 --2- 172.21.15.47:0/1415038779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441eb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.883+0000 7f0817e00640 1 -- 172.21.15.47:0/1415038779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 msgr2=0x556d441eb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.884+0000 7f0817e00640 1 --2- 172.21.15.47:0/1415038779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441eb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.884+0000 7f0817e00640 1 -- 172.21.15.47:0/1415038779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441d03a0 msgr2=0x556d441eb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.884+0000 7f0817e00640 1 --2- 172.21.15.47:0/1415038779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441d03a0 0x556d441eb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.884+0000 7f0817e00640 1 -- 172.21.15.47:0/1415038779 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d441c5510 con 0x556d441cf9e0 2024-03-21T11:29:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.884+0000 7f081dc00640 1 --2- 172.21.15.47:0/1415038779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441d03a0 0x556d441eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.884+0000 7f0817e00640 1 --2- 172.21.15.47:0/1415038779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441cf9e0 0x556d441e0090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0804098210 tx=0x7f080409a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb6676279a50de4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.884+0000 7f0809600640 1 -- 172.21.15.47:0/1415038779 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080407f030 con 0x556d441cf9e0 2024-03-21T11:29:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.884+0000 7f0809600640 1 -- 172.21.15.47:0/1415038779 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f080409f030 con 0x556d441cf9e0 2024-03-21T11:29:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.884+0000 7f0809600640 1 -- 172.21.15.47:0/1415038779 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08040a4330 con 0x556d441cf9e0 2024-03-21T11:29:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.885+0000 7f0822972740 1 -- 172.21.15.47:0/1415038779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441cf9e0 msgr2=0x556d441e0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.885+0000 7f0822972740 1 --2- 172.21.15.47:0/1415038779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441cf9e0 0x556d441e0090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0804098210 tx=0x7f080409a2c0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.885+0000 7f0822972740 1 -- 172.21.15.47:0/1415038779 shutdown_connections 2024-03-21T11:29:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.885+0000 7f0822972740 1 --2- 172.21.15.47:0/1415038779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441d03a0 0x556d441eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.885+0000 7f0822972740 1 --2- 172.21.15.47:0/1415038779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441cf9e0 0x556d441e0090 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.885+0000 7f0822972740 1 --2- 172.21.15.47:0/1415038779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441eb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.885+0000 7f0822972740 1 -- 172.21.15.47:0/1415038779 >> 172.21.15.47:0/1415038779 conn(0x556d441c9c00 msgr2=0x556d441cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.885+0000 7f0822972740 1 -- 172.21.15.47:0/1415038779 shutdown_connections 2024-03-21T11:29:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.885+0000 7f0822972740 1 -- 172.21.15.47:0/1415038779 wait complete. 2024-03-21T11:29:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.886+0000 7f0822972740 1 -- start start 2024-03-21T11:29:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.886+0000 7f0822972740 1 Processor -- start 2024-03-21T11:29:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.886+0000 7f0822972740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441db9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.886+0000 7f0822972740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441cf9e0 0x556d441dbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.886+0000 7f0822972740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441d03a0 0x556d441def50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.886+0000 7f0822972740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08181ad600 con 0x556d441d03a0 2024-03-21T11:29:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.886+0000 7f0822972740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08181acfa0 con 0x556d441cf9e0 2024-03-21T11:29:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.886+0000 7f0822972740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08181aa4a0 con 0x556d441ce810 2024-03-21T11:29:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.886+0000 7f0822972740 10 client.0 _ll_register_callbacks cb 0x7f080409bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.887+0000 7f081d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441db9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.887+0000 7f0817e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441cf9e0 0x556d441dbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.887+0000 7f081d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441db9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45562/0 (socket says 172.21.15.47:45562) 2024-03-21T11:29:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.887+0000 7f081d200640 1 -- 172.21.15.47:0/2605118586 learned_addr learned my addr 172.21.15.47:0/2605118586 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.887+0000 7f081dc00640 1 --2- 172.21.15.47:0/2605118586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441d03a0 0x556d441def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.887+0000 7f081d200640 1 -- 172.21.15.47:0/2605118586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441cf9e0 msgr2=0x556d441dbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.887+0000 7f081d200640 1 --2- 172.21.15.47:0/2605118586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441cf9e0 0x556d441dbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.887+0000 7f081d200640 1 -- 172.21.15.47:0/2605118586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441d03a0 msgr2=0x556d441def50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.887+0000 7f081d200640 1 --2- 172.21.15.47:0/2605118586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441d03a0 0x556d441def50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.887+0000 7f081d200640 1 -- 172.21.15.47:0/2605118586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d441df540 con 0x556d441ce810 2024-03-21T11:29:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.887+0000 7f081d200640 1 --2- 172.21.15.47:0/2605118586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441db9f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f080c005c00 tx=0x7f080c042070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83536ef227e63a34 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.887+0000 7f07fea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:29:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.888+0000 7f07fea00640 1 -- 172.21.15.47:0/2605118586 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c046070 con 0x556d441ce810 2024-03-21T11:29:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.888+0000 7f0822972740 1 -- 172.21.15.47:0/2605118586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f080409f030 con 0x556d441ce810 2024-03-21T11:29:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.888+0000 7f07fea00640 1 -- 172.21.15.47:0/2605118586 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f080c0046c0 con 0x556d441ce810 2024-03-21T11:29:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.888+0000 7f07fea00640 1 -- 172.21.15.47:0/2605118586 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f080c0048c0 con 0x556d441ce810 2024-03-21T11:29:30.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.888+0000 7f0822972740 10 client.27598 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.888+0000 7f0822972740 1 -- 172.21.15.47:0/2605118586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556d44248980 con 0x556d441ce810 2024-03-21T11:29:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.888+0000 7f07fea00640 1 -- 172.21.15.47:0/2605118586 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f080409f030 con 0x556d441ce810 2024-03-21T11:29:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.888+0000 7f07fea00640 1 -- 172.21.15.47:0/2605118586 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f080c0586f0 con 0x556d441ce810 2024-03-21T11:29:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.888+0000 7f0822972740 10 client.27598 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.888+0000 7f0822972740 10 client.27598 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:29:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.888+0000 7f0822972740 20 client.27598 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.888+0000 7f0822972740 1 --2- 172.21.15.47:0/2605118586 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556d442492d0 0x556d4424b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.888+0000 7f0822972740 4 client.27598 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:29:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.888+0000 7f0822972740 1 -- 172.21.15.47:0/2605118586 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556d4413ab50 con 0x556d442492d0 2024-03-21T11:29:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.889+0000 7f0817e00640 1 --2- 172.21.15.47:0/2605118586 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556d442492d0 0x556d4424b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.889+0000 7f07fea00640 10 client.27598 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:29:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.889+0000 7f0817e00640 1 --2- 172.21.15.47:0/2605118586 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556d442492d0 0x556d4424b6f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58a99563e878bc73 server_cookie=10f8877c952c328c in_seq=0 out_seq=0 2024-03-21T11:29:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.889+0000 7f07fea00640 1 -- 172.21.15.47:0/2605118586 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556d4413ab50 con 0x556d442492d0 2024-03-21T11:29:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.889+0000 7f07fea00640 10 client.27598 handle_command_reply: tid=0 2024-03-21T11:29:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.889+0000 7f0822972740 1 client.27598 shutdown 2024-03-21T11:29:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.890+0000 7f0822972740 1 -- 172.21.15.47:0/2605118586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 msgr2=0x556d441db9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.890+0000 7f0822972740 1 --2- 172.21.15.47:0/2605118586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441db9f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f080c005c00 tx=0x7f080c042070 comp rx=0 tx=0).stop 2024-03-21T11:29:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.890+0000 7f0822972740 1 -- 172.21.15.47:0/2605118586 shutdown_connections 2024-03-21T11:29:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.890+0000 7f0822972740 1 --2- 172.21.15.47:0/2605118586 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556d442492d0 0x556d4424b6f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.890+0000 7f0822972740 1 --2- 172.21.15.47:0/2605118586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d441d03a0 0x556d441def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.890+0000 7f0822972740 1 --2- 172.21.15.47:0/2605118586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d441cf9e0 0x556d441dbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.890+0000 7f0822972740 1 --2- 172.21.15.47:0/2605118586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d441ce810 0x556d441db9f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.890+0000 7f07fea00640 0 client.27598 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:29:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.890+0000 7f0822972740 1 -- 172.21.15.47:0/2605118586 >> 172.21.15.47:0/2605118586 conn(0x556d441c9c00 msgr2=0x556d441d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.890+0000 7f0822972740 1 -- 172.21.15.47:0/2605118586 shutdown_connections 2024-03-21T11:29:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.890+0000 7f0822972740 1 -- 172.21.15.47:0/2605118586 wait complete. 2024-03-21T11:29:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.890+0000 7f0822972740 20 client.27598 trim_cache size 0 max 16384 2024-03-21T11:29:30.906 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:29:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.891+0000 7f081fc00640 1 -- 172.21.15.47:0/1819342018 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f08000a2fa0 msgr2=0x7f08000a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.891+0000 7f081fc00640 1 --2- 172.21.15.47:0/1819342018 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f08000a2fa0 0x7f08000a5460 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f080c000c00 tx=0x7f080c02df90 comp rx=0 tx=0).stop 2024-03-21T11:29:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.891+0000 7f081fc00640 1 -- 172.21.15.47:0/1819342018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0818104a70 msgr2=0x7f081819a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.891+0000 7f081fc00640 1 --2- 172.21.15.47:0/1819342018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0818104a70 0x7f081819a1d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f080400e9f0 tx=0x7f080400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.891+0000 7f081fc00640 1 -- 172.21.15.47:0/1819342018 shutdown_connections 2024-03-21T11:29:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.891+0000 7f081fc00640 1 --2- 172.21.15.47:0/1819342018 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f08000a2fa0 0x7f08000a5460 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.891+0000 7f081fc00640 1 --2- 172.21.15.47:0/1819342018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08181073a0 0x7f08181a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.891+0000 7f081fc00640 1 --2- 172.21.15.47:0/1819342018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0818104a70 0x7f081819a1d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.891+0000 7f081fc00640 1 --2- 172.21.15.47:0/1819342018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0818102140 0x7f0818199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.891+0000 7f081fc00640 1 -- 172.21.15.47:0/1819342018 >> 172.21.15.47:0/1819342018 conn(0x7f08180fbb30 msgr2=0x7f0818101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.891+0000 7f081fc00640 1 -- 172.21.15.47:0/1819342018 shutdown_connections 2024-03-21T11:29:30.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:30.891+0000 7f081fc00640 1 -- 172.21.15.47:0/1819342018 wait complete. 2024-03-21T11:29:30.909 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:30 smithi047 ceph-mon[31351]: pgmap v585: 129 pgs: 129 active+clean; 35 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 289 op/s 2024-03-21T11:29:30.909 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:30 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3350974012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:29:30.911 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:29:30.912 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:29:30.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:30 smithi102 ceph-mon[36748]: pgmap v585: 129 pgs: 129 active+clean; 35 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 289 op/s 2024-03-21T11:29:30.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:30 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3350974012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:29:31.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.130+0000 7f4882c00640 1 -- 172.21.15.47:0/3747184097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c104a50 msgr2=0x7f487c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.130+0000 7f4882c00640 1 --2- 172.21.15.47:0/3747184097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c104a50 0x7f487c106e40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4868009900 tx=0x7f486802bc50 comp rx=0 tx=0).stop 2024-03-21T11:29:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.130+0000 7f4882c00640 1 -- 172.21.15.47:0/3747184097 shutdown_connections 2024-03-21T11:29:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.130+0000 7f4882c00640 1 --2- 172.21.15.47:0/3747184097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c107380 0x7f487c1097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.130+0000 7f4882c00640 1 --2- 172.21.15.47:0/3747184097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c104a50 0x7f487c106e40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.130+0000 7f4882c00640 1 --2- 172.21.15.47:0/3747184097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487c102120 0x7f487c104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.130+0000 7f4882c00640 1 -- 172.21.15.47:0/3747184097 >> 172.21.15.47:0/3747184097 conn(0x7f487c0fbb30 msgr2=0x7f487c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.130+0000 7f4882c00640 1 -- 172.21.15.47:0/3747184097 shutdown_connections 2024-03-21T11:29:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.130+0000 7f4882c00640 1 -- 172.21.15.47:0/3747184097 wait complete. 2024-03-21T11:29:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.132+0000 7f4882c00640 1 Processor -- start 2024-03-21T11:29:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.132+0000 7f4882c00640 1 -- start start 2024-03-21T11:29:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f4882c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c102120 0x7f487c197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f4882c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c104a50 0x7f487c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f4882c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487c107380 0x7f487c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f4882c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c10d360 con 0x7f487c107380 2024-03-21T11:29:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f4882c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c10d1e0 con 0x7f487c104a50 2024-03-21T11:29:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f4882c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c10d4e0 con 0x7f487c102120 2024-03-21T11:29:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f487be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c102120 0x7f487c197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f487b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c104a50 0x7f487c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f487be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c102120 0x7f487c197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45582/0 (socket says 172.21.15.47:45582) 2024-03-21T11:29:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f487be00640 1 -- 172.21.15.47:0/3378129922 learned_addr learned my addr 172.21.15.47:0/3378129922 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f4880c00640 1 --2- 172.21.15.47:0/3378129922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487c107380 0x7f487c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f487be00640 1 -- 172.21.15.47:0/3378129922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c104a50 msgr2=0x7f487c197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f487be00640 1 --2- 172.21.15.47:0/3378129922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c104a50 0x7f487c197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f487be00640 1 -- 172.21.15.47:0/3378129922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487c107380 msgr2=0x7f487c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f487be00640 1 --2- 172.21.15.47:0/3378129922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487c107380 0x7f487c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.133+0000 7f487be00640 1 -- 172.21.15.47:0/3378129922 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c19f630 con 0x7f487c102120 2024-03-21T11:29:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.134+0000 7f487be00640 1 --2- 172.21.15.47:0/3378129922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c102120 0x7f487c197a10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f486c00d950 tx=0x7f486c00dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.134+0000 7f4878c00640 1 -- 172.21.15.47:0/3378129922 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c017070 con 0x7f487c102120 2024-03-21T11:29:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.134+0000 7f4882c00640 1 -- 172.21.15.47:0/3378129922 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487c19f920 con 0x7f487c102120 2024-03-21T11:29:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.134+0000 7f4878c00640 1 -- 172.21.15.47:0/3378129922 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f486c00de80 con 0x7f487c102120 2024-03-21T11:29:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.134+0000 7f4878c00640 1 -- 172.21.15.47:0/3378129922 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c00f620 con 0x7f487c102120 2024-03-21T11:29:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.134+0000 7f4882c00640 1 -- 172.21.15.47:0/3378129922 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487c19fe40 con 0x7f487c102120 2024-03-21T11:29:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.139+0000 7f4885965740 1 Processor -- start 2024-03-21T11:29:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.139+0000 7f4885965740 1 -- start start 2024-03-21T11:29:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.139+0000 7f4885965740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f767ec20 0x55d7f767f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.139+0000 7f4885965740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f7680270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.139+0000 7f4885965740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f76807b0 0x55d7f7687040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.139+0000 7f4885965740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f75df2c0 con 0x55d7f767ec20 2024-03-21T11:29:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.139+0000 7f4885965740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f7587c90 con 0x55d7f76807b0 2024-03-21T11:29:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.139+0000 7f4878c00640 1 -- 172.21.15.47:0/3378129922 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f486c009c30 con 0x7f487c102120 2024-03-21T11:29:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.139+0000 7f4885965740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f75db360 con 0x55d7f767fdf0 2024-03-21T11:29:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.140+0000 7f4880c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f767ec20 0x55d7f767f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.140+0000 7f487b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f76807b0 0x55d7f7687040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.140+0000 7f487b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f76807b0 0x55d7f7687040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43488/0 (socket says 172.21.15.47:43488) 2024-03-21T11:29:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.140+0000 7f487be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f7680270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.140+0000 7f487b400640 1 -- 172.21.15.47:0/206085564 learned_addr learned my addr 172.21.15.47:0/206085564 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.140+0000 7f487b400640 1 -- 172.21.15.47:0/206085564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 msgr2=0x55d7f7680270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.140+0000 7f487b400640 1 --2- 172.21.15.47:0/206085564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f7680270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.140+0000 7f487b400640 1 -- 172.21.15.47:0/206085564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f767ec20 msgr2=0x55d7f767f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.140+0000 7f487b400640 1 --2- 172.21.15.47:0/206085564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f767ec20 0x55d7f767f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.140+0000 7f487b400640 1 -- 172.21.15.47:0/206085564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7f75ac080 con 0x55d7f76807b0 2024-03-21T11:29:31.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.141+0000 7f487b400640 1 --2- 172.21.15.47:0/206085564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f76807b0 0x55d7f7687040 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f486802c260 tx=0x7f48680095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c735b55211533fe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.141+0000 7f4871200640 1 -- 172.21.15.47:0/206085564 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868042070 con 0x55d7f76807b0 2024-03-21T11:29:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.141+0000 7f4871200640 1 -- 172.21.15.47:0/206085564 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f486802cba0 con 0x55d7f76807b0 2024-03-21T11:29:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.141+0000 7f4871200640 1 -- 172.21.15.47:0/206085564 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48680050f0 con 0x55d7f76807b0 2024-03-21T11:29:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.141+0000 7f4878c00640 1 --2- 172.21.15.47:0/3378129922 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48600a2f20 0x7f48600a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.141+0000 7f4878c00640 1 -- 172.21.15.47:0/3378129922 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f486c0cf3c0 con 0x7f487c102120 2024-03-21T11:29:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.142+0000 7f4885965740 1 -- 172.21.15.47:0/206085564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f76807b0 msgr2=0x55d7f7687040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:31.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.142+0000 7f4885965740 1 --2- 172.21.15.47:0/206085564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f76807b0 0x55d7f7687040 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f486802c260 tx=0x7f48680095e0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.142+0000 7f4880c00640 1 --2- 172.21.15.47:0/3378129922 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48600a2f20 0x7f48600a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.142+0000 7f4885965740 1 -- 172.21.15.47:0/206085564 shutdown_connections 2024-03-21T11:29:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.142+0000 7f4885965740 1 --2- 172.21.15.47:0/206085564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f76807b0 0x55d7f7687040 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.142+0000 7f4885965740 1 --2- 172.21.15.47:0/206085564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f7680270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.142+0000 7f4885965740 1 --2- 172.21.15.47:0/206085564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f767ec20 0x55d7f767f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.142+0000 7f4885965740 1 -- 172.21.15.47:0/206085564 >> 172.21.15.47:0/206085564 conn(0x55d7f7675c00 msgr2=0x55d7f7676000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.142+0000 7f4885965740 1 -- 172.21.15.47:0/206085564 shutdown_connections 2024-03-21T11:29:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.142+0000 7f4885965740 1 -- 172.21.15.47:0/206085564 wait complete. 2024-03-21T11:29:31.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.143+0000 7f4880c00640 1 --2- 172.21.15.47:0/3378129922 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48600a2f20 0x7f48600a53e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55d7f767fcf0 tx=0x7f4874009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:31.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.143+0000 7f4885965740 1 -- start start 2024-03-21T11:29:31.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:30 smithi100 ceph-mon[36245]: pgmap v585: 129 pgs: 129 active+clean; 35 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 289 op/s 2024-03-21T11:29:31.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:30 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3350974012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:29:31.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.143+0000 7f4885965740 1 Processor -- start 2024-03-21T11:29:31.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.143+0000 7f4885965740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f7688de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.143+0000 7f4885965740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f767fdf0 0x55d7f7689320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.143+0000 7f4885965740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f76807b0 0x55d7f7690360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.143+0000 7f4885965740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f7687760 con 0x55d7f767fdf0 2024-03-21T11:29:31.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.143+0000 7f4885965740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f759b0d0 con 0x55d7f767ec20 2024-03-21T11:29:31.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.143+0000 7f4885965740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f76878e0 con 0x55d7f76807b0 2024-03-21T11:29:31.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.143+0000 7f4885965740 10 client.0 _ll_register_callbacks cb 0x55d7f759fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:31.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.144+0000 7f487be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f7688de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.144+0000 7f4880c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f76807b0 0x55d7f7690360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.144+0000 7f487be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f7688de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43502/0 (socket says 172.21.15.47:43502) 2024-03-21T11:29:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.144+0000 7f487b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f767fdf0 0x55d7f7689320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.144+0000 7f487be00640 1 -- 172.21.15.47:0/2738334614 learned_addr learned my addr 172.21.15.47:0/2738334614 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.144+0000 7f487be00640 1 -- 172.21.15.47:0/2738334614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f76807b0 msgr2=0x55d7f7690360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.144+0000 7f487be00640 1 --2- 172.21.15.47:0/2738334614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f76807b0 0x55d7f7690360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.144+0000 7f487be00640 1 -- 172.21.15.47:0/2738334614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f767fdf0 msgr2=0x55d7f7689320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.144+0000 7f487be00640 1 --2- 172.21.15.47:0/2738334614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f767fdf0 0x55d7f7689320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.144+0000 7f487be00640 1 -- 172.21.15.47:0/2738334614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7f72f33f0 con 0x55d7f767ec20 2024-03-21T11:29:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.144+0000 7f4880c00640 1 --2- 172.21.15.47:0/2738334614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f76807b0 0x55d7f7690360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.144+0000 7f487be00640 1 --2- 172.21.15.47:0/2738334614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f7688de0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f486c09b690 tx=0x7f486c0d2930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bc4435a753532fb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.144+0000 7f4865600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:29:31.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.145+0000 7f4865600640 1 -- 172.21.15.47:0/2738334614 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c020030 con 0x55d7f767ec20 2024-03-21T11:29:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.145+0000 7f4865600640 1 -- 172.21.15.47:0/2738334614 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f486c0ccbe0 con 0x55d7f767ec20 2024-03-21T11:29:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.145+0000 7f4885965740 1 -- 172.21.15.47:0/2738334614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f486802cba0 con 0x55d7f767ec20 2024-03-21T11:29:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.145+0000 7f4865600640 1 -- 172.21.15.47:0/2738334614 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f486c09b060 con 0x55d7f767ec20 2024-03-21T11:29:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.145+0000 7f4885965740 10 client.27590 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.145+0000 7f4885965740 1 -- 172.21.15.47:0/2738334614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d7f73bd7d0 con 0x55d7f767ec20 2024-03-21T11:29:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.145+0000 7f4865600640 1 -- 172.21.15.47:0/2738334614 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f486802cba0 con 0x55d7f767ec20 2024-03-21T11:29:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.145+0000 7f4865600640 1 -- 172.21.15.47:0/2738334614 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f486c097d70 con 0x55d7f767ec20 2024-03-21T11:29:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.145+0000 7f4885965740 10 client.27590 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.145+0000 7f4885965740 10 client.27590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.145+0000 7f4885965740 20 client.27590 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.146+0000 7f4885965740 1 --2- 172.21.15.47:0/2738334614 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7f76948a0 0x55d7f7696cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.146+0000 7f4885965740 4 client.27590 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:29:31.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.146+0000 7f4885965740 1 -- 172.21.15.47:0/2738334614 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7f75ec800 con 0x55d7f76948a0 2024-03-21T11:29:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.146+0000 7f487b400640 1 --2- 172.21.15.47:0/2738334614 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7f76948a0 0x55d7f7696cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.146+0000 7f487b400640 1 --2- 172.21.15.47:0/2738334614 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7f76948a0 0x55d7f7696cf0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36a1107535fb9dcd server_cookie=54228e0e80afd897 in_seq=0 out_seq=0 2024-03-21T11:29:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.146+0000 7f4865600640 10 client.27590 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.147+0000 7f4865600640 1 -- 172.21.15.47:0/2738334614 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d7f75ec800 con 0x55d7f76948a0 2024-03-21T11:29:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.147+0000 7f4865600640 10 client.27590 handle_command_reply: tid=0 2024-03-21T11:29:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.147+0000 7f4885965740 1 client.27590 shutdown 2024-03-21T11:29:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.148+0000 7f4885965740 1 -- 172.21.15.47:0/2738334614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 msgr2=0x55d7f7688de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.148+0000 7f4885965740 1 --2- 172.21.15.47:0/2738334614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f7688de0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f486c09b690 tx=0x7f486c0d2930 comp rx=0 tx=0).stop 2024-03-21T11:29:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.148+0000 7f4885965740 1 -- 172.21.15.47:0/2738334614 shutdown_connections 2024-03-21T11:29:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.148+0000 7f4885965740 1 --2- 172.21.15.47:0/2738334614 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7f76948a0 0x55d7f7696cf0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.148+0000 7f4885965740 1 --2- 172.21.15.47:0/2738334614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f76807b0 0x55d7f7690360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.148+0000 7f4885965740 1 --2- 172.21.15.47:0/2738334614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f767fdf0 0x55d7f7689320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.148+0000 7f4865600640 0 client.27590 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.148+0000 7f4885965740 1 --2- 172.21.15.47:0/2738334614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f7688de0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.148+0000 7f4885965740 1 -- 172.21.15.47:0/2738334614 >> 172.21.15.47:0/2738334614 conn(0x55d7f7675c00 msgr2=0x55d7f7674eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.148+0000 7f4885965740 1 -- 172.21.15.47:0/2738334614 shutdown_connections 2024-03-21T11:29:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.148+0000 7f4885965740 1 -- 172.21.15.47:0/2738334614 wait complete. 2024-03-21T11:29:31.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.149+0000 7f4885965740 20 client.27590 trim_cache size 0 max 16384 2024-03-21T11:29:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.163+0000 7f4885965740 1 Processor -- start 2024-03-21T11:29:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.163+0000 7f4885965740 1 -- start start 2024-03-21T11:29:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.163+0000 7f4885965740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f7683020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.163+0000 7f4885965740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f7690320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.163+0000 7f4885965740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f76807b0 0x55d7f7683560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.163+0000 7f4885965740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1a7d70 con 0x55d7f76807b0 2024-03-21T11:29:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.163+0000 7f4885965740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1a96f0 con 0x55d7f767ec20 2024-03-21T11:29:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.163+0000 7f4885965740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1a83d0 con 0x55d7f767fdf0 2024-03-21T11:29:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.163+0000 7f487b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f7690320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.163+0000 7f487b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f7690320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45610/0 (socket says 172.21.15.47:45610) 2024-03-21T11:29:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.163+0000 7f487b400640 1 -- 172.21.15.47:0/573078901 learned_addr learned my addr 172.21.15.47:0/573078901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.163+0000 7f4880c00640 1 --2- 172.21.15.47:0/573078901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f76807b0 0x55d7f7683560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.163+0000 7f487be00640 1 --2- 172.21.15.47:0/573078901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f7683020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.164+0000 7f487b400640 1 -- 172.21.15.47:0/573078901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 msgr2=0x55d7f7683020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.164+0000 7f487b400640 1 --2- 172.21.15.47:0/573078901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f7683020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.164+0000 7f487b400640 1 -- 172.21.15.47:0/573078901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f76807b0 msgr2=0x55d7f7683560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.164+0000 7f487b400640 1 --2- 172.21.15.47:0/573078901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f76807b0 0x55d7f7683560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.164+0000 7f487b400640 1 -- 172.21.15.47:0/573078901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7f767f870 con 0x55d7f767fdf0 2024-03-21T11:29:31.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.164+0000 7f487be00640 1 --2- 172.21.15.47:0/573078901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f7683020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:29:31.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.164+0000 7f487b400640 1 --2- 172.21.15.47:0/573078901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f7690320 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f486802c730 tx=0x7f486802e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba96ee14ab785da8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:31.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.164+0000 7f4867e00640 1 -- 172.21.15.47:0/573078901 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868056070 con 0x55d7f767fdf0 2024-03-21T11:29:31.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.165+0000 7f4867e00640 1 -- 172.21.15.47:0/573078901 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f486802cd20 con 0x55d7f767fdf0 2024-03-21T11:29:31.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.165+0000 7f4867e00640 1 -- 172.21.15.47:0/573078901 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868042040 con 0x55d7f767fdf0 2024-03-21T11:29:31.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.165+0000 7f4885965740 1 -- 172.21.15.47:0/573078901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 msgr2=0x55d7f7690320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:31.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.165+0000 7f4885965740 1 --2- 172.21.15.47:0/573078901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f7690320 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f486802c730 tx=0x7f486802e040 comp rx=0 tx=0).stop 2024-03-21T11:29:31.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.165+0000 7f4885965740 1 -- 172.21.15.47:0/573078901 shutdown_connections 2024-03-21T11:29:31.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.165+0000 7f4885965740 1 --2- 172.21.15.47:0/573078901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f76807b0 0x55d7f7683560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.165+0000 7f4885965740 1 --2- 172.21.15.47:0/573078901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f7690320 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.165+0000 7f4885965740 1 --2- 172.21.15.47:0/573078901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f7683020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.165+0000 7f4885965740 1 -- 172.21.15.47:0/573078901 >> 172.21.15.47:0/573078901 conn(0x55d7f7675c00 msgr2=0x55d7f76796f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:31.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.166+0000 7f4885965740 1 -- 172.21.15.47:0/573078901 shutdown_connections 2024-03-21T11:29:31.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.166+0000 7f4885965740 1 -- 172.21.15.47:0/573078901 wait complete. 2024-03-21T11:29:31.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.166+0000 7f4885965740 1 -- start start 2024-03-21T11:29:31.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.166+0000 7f4885965740 1 Processor -- start 2024-03-21T11:29:31.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f4885965740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f76900a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f4885965740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f768e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f4885965740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f76807b0 0x55d7f768ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f4885965740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1a9090 con 0x55d7f76807b0 2024-03-21T11:29:31.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f4885965740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1a8a30 con 0x55d7f767ec20 2024-03-21T11:29:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f4885965740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1a5f30 con 0x55d7f767fdf0 2024-03-21T11:29:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f4885965740 10 client.0 _ll_register_callbacks cb 0x7f486c0d2af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f487be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f76900a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f487b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f768e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f487be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f76900a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43532/0 (socket says 172.21.15.47:43532) 2024-03-21T11:29:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f487b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f768e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45622/0 (socket says 172.21.15.47:45622) 2024-03-21T11:29:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f487be00640 1 -- 172.21.15.47:0/1521209988 learned_addr learned my addr 172.21.15.47:0/1521209988 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f4880c00640 1 --2- 172.21.15.47:0/1521209988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f76807b0 0x55d7f768ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f487b400640 1 -- 172.21.15.47:0/1521209988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 msgr2=0x55d7f76900a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f487b400640 1 --2- 172.21.15.47:0/1521209988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f76900a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f487b400640 1 -- 172.21.15.47:0/1521209988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f76807b0 msgr2=0x55d7f768ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f487b400640 1 --2- 172.21.15.47:0/1521209988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f76807b0 0x55d7f768ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.167+0000 7f487b400640 1 -- 172.21.15.47:0/1521209988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7f768f1b0 con 0x55d7f767fdf0 2024-03-21T11:29:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.168+0000 7f487b400640 1 --2- 172.21.15.47:0/1521209988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f768e6b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55d7f759ff00 tx=0x7f48680058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d12766581ec60477 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.168+0000 7f4865600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:29:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.168+0000 7f4865600640 1 -- 172.21.15.47:0/1521209988 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868056070 con 0x55d7f767fdf0 2024-03-21T11:29:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.168+0000 7f4885965740 1 -- 172.21.15.47:0/1521209988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f486802cd20 con 0x55d7f767fdf0 2024-03-21T11:29:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.168+0000 7f4865600640 1 -- 172.21.15.47:0/1521209988 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4868004ae0 con 0x55d7f767fdf0 2024-03-21T11:29:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.168+0000 7f4865600640 1 -- 172.21.15.47:0/1521209988 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f486805f050 con 0x55d7f767fdf0 2024-03-21T11:29:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.168+0000 7f4885965740 10 client.27616 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.168+0000 7f4885965740 1 -- 172.21.15.47:0/1521209988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d7f76f0f40 con 0x55d7f767fdf0 2024-03-21T11:29:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.168+0000 7f4865600640 1 -- 172.21.15.47:0/1521209988 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f486802cd20 con 0x55d7f767fdf0 2024-03-21T11:29:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.169+0000 7f4865600640 1 -- 172.21.15.47:0/1521209988 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4868059020 con 0x55d7f767fdf0 2024-03-21T11:29:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.169+0000 7f4885965740 10 client.27616 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.169+0000 7f4885965740 10 client.27616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.169+0000 7f4885965740 20 client.27616 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.169+0000 7f4885965740 1 --2- 172.21.15.47:0/1521209988 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7f76f1390 0x55d7f76f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.169+0000 7f4885965740 4 client.27616 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:29:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.169+0000 7f4885965740 1 -- 172.21.15.47:0/1521209988 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d7f75e6b50 con 0x55d7f76f1390 2024-03-21T11:29:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.169+0000 7f487be00640 1 --2- 172.21.15.47:0/1521209988 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7f76f1390 0x55d7f76f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.169+0000 7f487be00640 1 --2- 172.21.15.47:0/1521209988 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7f76f1390 0x55d7f76f37b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5101f39aea3a157f server_cookie=861ba788eabbddb5 in_seq=0 out_seq=0 2024-03-21T11:29:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.169+0000 7f4865600640 10 client.27616 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4865600640 1 -- 172.21.15.47:0/1521209988 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d7f75e6b50 con 0x55d7f76f1390 2024-03-21T11:29:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4865600640 10 client.27616 handle_command_reply: tid=0 2024-03-21T11:29:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4885965740 1 client.27616 shutdown 2024-03-21T11:29:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4885965740 1 -- 172.21.15.47:0/1521209988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 msgr2=0x55d7f768e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4885965740 1 --2- 172.21.15.47:0/1521209988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f768e6b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55d7f759ff00 tx=0x7f48680058e0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4885965740 1 -- 172.21.15.47:0/1521209988 shutdown_connections 2024-03-21T11:29:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4885965740 1 --2- 172.21.15.47:0/1521209988 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7f76f1390 0x55d7f76f37b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4885965740 1 --2- 172.21.15.47:0/1521209988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7f76807b0 0x55d7f768ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4885965740 1 --2- 172.21.15.47:0/1521209988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7f767fdf0 0x55d7f768e6b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4885965740 1 --2- 172.21.15.47:0/1521209988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7f767ec20 0x55d7f76900a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4885965740 1 -- 172.21.15.47:0/1521209988 >> 172.21.15.47:0/1521209988 conn(0x55d7f7675c00 msgr2=0x55d7f7674620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:31.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4865600640 0 client.27616 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:31.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4885965740 1 -- 172.21.15.47:0/1521209988 shutdown_connections 2024-03-21T11:29:31.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4885965740 1 -- 172.21.15.47:0/1521209988 wait complete. 2024-03-21T11:29:31.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.170+0000 7f4885965740 20 client.27616 trim_cache size 0 max 16384 2024-03-21T11:29:31.178 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:29:31.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.171+0000 7f4882c00640 1 -- 172.21.15.47:0/3378129922 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48600a2f20 msgr2=0x7f48600a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:31.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.171+0000 7f4882c00640 1 --2- 172.21.15.47:0/3378129922 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48600a2f20 0x7f48600a53e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55d7f767fcf0 tx=0x7f4874009500 comp rx=0 tx=0).stop 2024-03-21T11:29:31.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.171+0000 7f4882c00640 1 -- 172.21.15.47:0/3378129922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c102120 msgr2=0x7f487c197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:31.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.171+0000 7f4882c00640 1 --2- 172.21.15.47:0/3378129922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c102120 0x7f487c197a10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f486c00d950 tx=0x7f486c00dd50 comp rx=0 tx=0).stop 2024-03-21T11:29:31.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.171+0000 7f4882c00640 1 -- 172.21.15.47:0/3378129922 shutdown_connections 2024-03-21T11:29:31.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.171+0000 7f4882c00640 1 --2- 172.21.15.47:0/3378129922 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48600a2f20 0x7f48600a53e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.171+0000 7f4882c00640 1 --2- 172.21.15.47:0/3378129922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487c107380 0x7f487c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.171+0000 7f4882c00640 1 --2- 172.21.15.47:0/3378129922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c104a50 0x7f487c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.171+0000 7f4882c00640 1 --2- 172.21.15.47:0/3378129922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c102120 0x7f487c197a10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:31.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.171+0000 7f4882c00640 1 -- 172.21.15.47:0/3378129922 >> 172.21.15.47:0/3378129922 conn(0x7f487c0fbb30 msgr2=0x7f487c102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:31.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.171+0000 7f4882c00640 1 -- 172.21.15.47:0/3378129922 shutdown_connections 2024-03-21T11:29:31.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:31.171+0000 7f4882c00640 1 -- 172.21.15.47:0/3378129922 wait complete. 2024-03-21T11:29:31.520 INFO:tasks.workunit.client.0.smithi047.stdout: 1 403356 8.28 MB/sec execute 135 sec latency 11.432 ms 2024-03-21T11:29:32.191 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:29:32.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:29:32.191 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:29:32.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.454+0000 7f1ddc200640 1 -- 172.21.15.47:0/3716765853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dd4108eb0 msgr2=0x7f1dd4109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.454+0000 7f1ddc200640 1 --2- 172.21.15.47:0/3716765853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dd4108eb0 0x7f1dd4109330 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009900 tx=0x7f1dc802bc30 comp rx=0 tx=0).stop 2024-03-21T11:29:32.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.455+0000 7f1ddc200640 1 -- 172.21.15.47:0/3716765853 shutdown_connections 2024-03-21T11:29:32.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.455+0000 7f1ddc200640 1 --2- 172.21.15.47:0/3716765853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1dd4109870 0x7f1dd4110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.455+0000 7f1ddc200640 1 --2- 172.21.15.47:0/3716765853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dd4108eb0 0x7f1dd4109330 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.455+0000 7f1ddc200640 1 --2- 172.21.15.47:0/3716765853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dd4107cc0 0x7f1dd41080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.455+0000 7f1ddc200640 1 -- 172.21.15.47:0/3716765853 >> 172.21.15.47:0/3716765853 conn(0x7f1dd4076c60 msgr2=0x7f1dd40790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:32.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.456+0000 7f1ddc200640 1 -- 172.21.15.47:0/3716765853 shutdown_connections 2024-03-21T11:29:32.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.456+0000 7f1ddc200640 1 -- 172.21.15.47:0/3716765853 wait complete. 2024-03-21T11:29:32.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.458+0000 7f1ddc200640 1 Processor -- start 2024-03-21T11:29:32.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.458+0000 7f1ddc200640 1 -- start start 2024-03-21T11:29:32.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.459+0000 7f1ddc200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dd4107cc0 0x7f1dd419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.459+0000 7f1ddc200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dd4108eb0 0x7f1dd419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.459+0000 7f1ddc200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1dd4109870 0x7f1dd41a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.459+0000 7f1ddc200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd4113970 con 0x7f1dd4109870 2024-03-21T11:29:32.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.459+0000 7f1ddc200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd41137f0 con 0x7f1dd4107cc0 2024-03-21T11:29:32.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.459+0000 7f1ddc200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd4113af0 con 0x7f1dd4108eb0 2024-03-21T11:29:32.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.459+0000 7f1dd9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dd4107cc0 0x7f1dd419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.459+0000 7f1dd9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dd4107cc0 0x7f1dd419dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43546/0 (socket says 172.21.15.47:43546) 2024-03-21T11:29:32.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.459+0000 7f1dd9800640 1 -- 172.21.15.47:0/1267713835 learned_addr learned my addr 172.21.15.47:0/1267713835 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:32.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.459+0000 7f1dd8e00640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dd4108eb0 0x7f1dd419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.459+0000 7f1dda200640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1dd4109870 0x7f1dd41a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.460+0000 7f1dd8e00640 1 -- 172.21.15.47:0/1267713835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dd4107cc0 msgr2=0x7f1dd419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.460+0000 7f1dd8e00640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dd4107cc0 0x7f1dd419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.460+0000 7f1dd8e00640 1 -- 172.21.15.47:0/1267713835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1dd4109870 msgr2=0x7f1dd41a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.460+0000 7f1dd8e00640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1dd4109870 0x7f1dd41a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.460+0000 7f1dd8e00640 1 -- 172.21.15.47:0/1267713835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd41a5be0 con 0x7f1dd4108eb0 2024-03-21T11:29:32.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.460+0000 7f1dd9800640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dd4107cc0 0x7f1dd419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:32.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.460+0000 7f1dd8e00640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dd4108eb0 0x7f1dd419e500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8036040 tx=0x7f1dc8004ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:32.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.461+0000 7f1dd2000640 1 -- 172.21.15.47:0/1267713835 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8043020 con 0x7f1dd4108eb0 2024-03-21T11:29:32.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.461+0000 7f1ddc200640 1 -- 172.21.15.47:0/1267713835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd41a5e70 con 0x7f1dd4108eb0 2024-03-21T11:29:32.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.461+0000 7f1dd2000640 1 -- 172.21.15.47:0/1267713835 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1dc80054e0 con 0x7f1dd4108eb0 2024-03-21T11:29:32.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.461+0000 7f1dd2000640 1 -- 172.21.15.47:0/1267713835 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8003c60 con 0x7f1dd4108eb0 2024-03-21T11:29:32.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.461+0000 7f1ddc200640 1 -- 172.21.15.47:0/1267713835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd41a6380 con 0x7f1dd4108eb0 2024-03-21T11:29:32.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.464+0000 7f1ddee56740 1 Processor -- start 2024-03-21T11:29:32.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.464+0000 7f1ddee56740 1 -- start start 2024-03-21T11:29:32.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.464+0000 7f1dd2000640 1 -- 172.21.15.47:0/1267713835 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1dc8046020 con 0x7f1dd4108eb0 2024-03-21T11:29:32.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1ddee56740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1ddee56740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1ddee56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1ddee56740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2012beb0 con 0x556d201c60a0 2024-03-21T11:29:32.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1ddee56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d200d4a60 con 0x556d201cbc00 2024-03-21T11:29:32.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1ddee56740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d20127f50 con 0x556d201c6a90 2024-03-21T11:29:32.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1dd8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1dd9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1dd8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43560/0 (socket says 172.21.15.47:43560) 2024-03-21T11:29:32.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1dd8e00640 1 -- 172.21.15.47:0/1084912841 learned_addr learned my addr 172.21.15.47:0/1084912841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:32.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1dda200640 1 --2- 172.21.15.47:0/1084912841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1dd9800640 1 -- 172.21.15.47:0/1084912841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 msgr2=0x556d201ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1dd9800640 1 --2- 172.21.15.47:0/1084912841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1dd9800640 1 -- 172.21.15.47:0/1084912841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 msgr2=0x556d201c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.465+0000 7f1dd9800640 1 --2- 172.21.15.47:0/1084912841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.466+0000 7f1dd9800640 1 -- 172.21.15.47:0/1084912841 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d1ff087e0 con 0x556d201c6a90 2024-03-21T11:29:32.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.466+0000 7f1dda200640 1 --2- 172.21.15.47:0/1084912841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:29:32.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.466+0000 7f1dd9800640 1 --2- 172.21.15.47:0/1084912841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201cb590 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00e7c0 tx=0x7f1dcc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f3b908ac22fad57 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:32.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.466+0000 7f1dc2a00640 1 -- 172.21.15.47:0/1084912841 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc004d20 con 0x556d201c6a90 2024-03-21T11:29:32.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.466+0000 7f1dc2a00640 1 -- 172.21.15.47:0/1084912841 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1dcc010e30 con 0x556d201c6a90 2024-03-21T11:29:32.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.466+0000 7f1dc2a00640 1 -- 172.21.15.47:0/1084912841 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc005980 con 0x556d201c6a90 2024-03-21T11:29:32.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.467+0000 7f1dd2000640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1dbc0a3070 0x7f1dbc0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.467+0000 7f1dd2000640 1 -- 172.21.15.47:0/1267713835 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1dc80f18a0 con 0x7f1dd4108eb0 2024-03-21T11:29:32.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.467+0000 7f1dda200640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1dbc0a3070 0x7f1dbc0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.467+0000 7f1ddee56740 1 -- 172.21.15.47:0/1084912841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 msgr2=0x556d201cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.467+0000 7f1ddee56740 1 --2- 172.21.15.47:0/1084912841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201cb590 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00e7c0 tx=0x7f1dcc00ec80 comp rx=0 tx=0).stop 2024-03-21T11:29:32.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.468+0000 7f1ddee56740 1 -- 172.21.15.47:0/1084912841 shutdown_connections 2024-03-21T11:29:32.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.468+0000 7f1ddee56740 1 --2- 172.21.15.47:0/1084912841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.468+0000 7f1ddee56740 1 --2- 172.21.15.47:0/1084912841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201cb590 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.468+0000 7f1ddee56740 1 --2- 172.21.15.47:0/1084912841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.468+0000 7f1ddee56740 1 -- 172.21.15.47:0/1084912841 >> 172.21.15.47:0/1084912841 conn(0x556d201c1610 msgr2=0x556d201c1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:32.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.468+0000 7f1ddee56740 1 -- 172.21.15.47:0/1084912841 shutdown_connections 2024-03-21T11:29:32.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.468+0000 7f1ddee56740 1 -- 172.21.15.47:0/1084912841 wait complete. 2024-03-21T11:29:32.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.468+0000 7f1dda200640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1dbc0a3070 0x7f1dbc0a5530 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x556d201c72f0 tx=0x7f1dc4005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:32.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.468+0000 7f1ddee56740 1 -- start start 2024-03-21T11:29:32.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.469+0000 7f1ddee56740 1 Processor -- start 2024-03-21T11:29:32.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.469+0000 7f1ddee56740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.469+0000 7f1ddee56740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.469+0000 7f1ddee56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.469+0000 7f1ddee56740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d201cebb0 con 0x556d201c60a0 2024-03-21T11:29:32.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.469+0000 7f1ddee56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d200e7eb0 con 0x556d201cbc00 2024-03-21T11:29:32.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.469+0000 7f1ddee56740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d201ced30 con 0x556d201c6a90 2024-03-21T11:29:32.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.469+0000 7f1ddee56740 10 client.0 _ll_register_callbacks cb 0x556d200dc150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.469+0000 7f1dd9800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.470+0000 7f1dda200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.470+0000 7f1dd9800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201d02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34684/0 (socket says 172.21.15.47:34684) 2024-03-21T11:29:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.470+0000 7f1dd9800640 1 -- 172.21.15.47:0/1096328830 learned_addr learned my addr 172.21.15.47:0/1096328830 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.470+0000 7f1dd8e00640 1 --2- 172.21.15.47:0/1096328830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.470+0000 7f1dda200640 1 -- 172.21.15.47:0/1096328830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 msgr2=0x556d201d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.470+0000 7f1dda200640 1 --2- 172.21.15.47:0/1096328830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.470+0000 7f1dda200640 1 -- 172.21.15.47:0/1096328830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 msgr2=0x556d201d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.470+0000 7f1dda200640 1 --2- 172.21.15.47:0/1096328830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.470+0000 7f1dda200640 1 -- 172.21.15.47:0/1096328830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d1ff68f70 con 0x556d201cbc00 2024-03-21T11:29:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.470+0000 7f1dda200640 1 --2- 172.21.15.47:0/1096328830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201d7860 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4012980 tx=0x7f1dc40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd44f90c8165390a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.470+0000 7f1db7400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:29:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.471+0000 7f1db7400640 1 -- 172.21.15.47:0/1096328830 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400fce0 con 0x556d201cbc00 2024-03-21T11:29:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.471+0000 7f1ddee56740 1 -- 172.21.15.47:0/1096328830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1dcc010e30 con 0x556d201cbc00 2024-03-21T11:29:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.471+0000 7f1db7400640 1 -- 172.21.15.47:0/1096328830 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1dc401f420 con 0x556d201cbc00 2024-03-21T11:29:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.471+0000 7f1db7400640 1 -- 172.21.15.47:0/1096328830 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dc4030050 con 0x556d201cbc00 2024-03-21T11:29:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.471+0000 7f1ddee56740 10 client.27608 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.471+0000 7f1ddee56740 1 -- 172.21.15.47:0/1096328830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556d1fe00170 con 0x556d201cbc00 2024-03-21T11:29:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.471+0000 7f1db7400640 1 -- 172.21.15.47:0/1096328830 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1dcc010e30 con 0x556d201cbc00 2024-03-21T11:29:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.472+0000 7f1db7400640 1 -- 172.21.15.47:0/1096328830 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1dc401d4d0 con 0x556d201cbc00 2024-03-21T11:29:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.472+0000 7f1ddee56740 10 client.27608 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.472+0000 7f1ddee56740 10 client.27608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:32.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.472+0000 7f1ddee56740 20 client.27608 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:32.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.472+0000 7f1ddee56740 1 --2- 172.21.15.47:0/1096328830 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d201dc150 0x556d201de5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.472+0000 7f1ddee56740 4 client.27608 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:29:32.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.472+0000 7f1ddee56740 1 -- 172.21.15.47:0/1096328830 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d201393f0 con 0x556d201dc150 2024-03-21T11:29:32.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.472+0000 7f1dd9800640 1 --2- 172.21.15.47:0/1096328830 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d201dc150 0x556d201de5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.473+0000 7f1dd9800640 1 --2- 172.21.15.47:0/1096328830 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d201dc150 0x556d201de5f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c08fd68148102e server_cookie=dcc81a5db7592c21 in_seq=0 out_seq=0 2024-03-21T11:29:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.473+0000 7f1db7400640 10 client.27608 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1db7400640 1 -- 172.21.15.47:0/1096328830 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556d201393f0 con 0x556d201dc150 2024-03-21T11:29:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1db7400640 10 client.27608 handle_command_reply: tid=0 2024-03-21T11:29:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1ddee56740 1 client.27608 shutdown 2024-03-21T11:29:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1ddee56740 1 -- 172.21.15.47:0/1096328830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 msgr2=0x556d201d7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1ddee56740 1 --2- 172.21.15.47:0/1096328830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201d7860 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4012980 tx=0x7f1dc40046c0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1ddee56740 1 -- 172.21.15.47:0/1096328830 shutdown_connections 2024-03-21T11:29:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1ddee56740 1 --2- 172.21.15.47:0/1096328830 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d201dc150 0x556d201de5f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1ddee56740 1 --2- 172.21.15.47:0/1096328830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201d7860 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1ddee56740 1 --2- 172.21.15.47:0/1096328830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1ddee56740 1 --2- 172.21.15.47:0/1096328830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1ddee56740 1 -- 172.21.15.47:0/1096328830 >> 172.21.15.47:0/1096328830 conn(0x556d201c1610 msgr2=0x556d201c0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1db7400640 0 client.27608 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1ddee56740 1 -- 172.21.15.47:0/1096328830 shutdown_connections 2024-03-21T11:29:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1ddee56740 1 -- 172.21.15.47:0/1096328830 wait complete. 2024-03-21T11:29:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.474+0000 7f1ddee56740 20 client.27608 trim_cache size 0 max 16384 2024-03-21T11:29:32.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.492+0000 7f1ddee56740 1 Processor -- start 2024-03-21T11:29:32.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.492+0000 7f1ddee56740 1 -- start start 2024-03-21T11:29:32.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1ddee56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201c60a0 0x556d201c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1ddee56740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201d7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1ddee56740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201cbc00 0x556d201c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1ddee56740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd41b0600 con 0x556d201cbc00 2024-03-21T11:29:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1ddee56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd41b1f80 con 0x556d201c60a0 2024-03-21T11:29:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1ddee56740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd41b0c60 con 0x556d201c6a90 2024-03-21T11:29:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1dd9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201c60a0 0x556d201c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1dda200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201cbc00 0x556d201c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1dd9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201c60a0 0x556d201c7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43580/0 (socket says 172.21.15.47:43580) 2024-03-21T11:29:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1dd9800640 1 -- 172.21.15.47:0/3426684664 learned_addr learned my addr 172.21.15.47:0/3426684664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1dd8e00640 1 --2- 172.21.15.47:0/3426684664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201d7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1dd9800640 1 -- 172.21.15.47:0/3426684664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 msgr2=0x556d201d7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1dd9800640 1 --2- 172.21.15.47:0/3426684664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201d7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1dd9800640 1 -- 172.21.15.47:0/3426684664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201cbc00 msgr2=0x556d201c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1dd9800640 1 --2- 172.21.15.47:0/3426684664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201cbc00 0x556d201c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.493+0000 7f1dd9800640 1 -- 172.21.15.47:0/3426684664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d201dbef0 con 0x556d201c60a0 2024-03-21T11:29:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.494+0000 7f1dd9800640 1 --2- 172.21.15.47:0/3426684664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201c60a0 0x556d201c7500 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009590 tx=0x7f1dcc0097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cf4c2b21e63050c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.494+0000 7f1dc2000640 1 -- 172.21.15.47:0/3426684664 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc003970 con 0x556d201c60a0 2024-03-21T11:29:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.494+0000 7f1dc2000640 1 -- 172.21.15.47:0/3426684664 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1dcc010870 con 0x556d201c60a0 2024-03-21T11:29:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.494+0000 7f1dc2000640 1 -- 172.21.15.47:0/3426684664 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc02a960 con 0x556d201c60a0 2024-03-21T11:29:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.495+0000 7f1ddee56740 1 -- 172.21.15.47:0/3426684664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201c60a0 msgr2=0x556d201c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.495+0000 7f1ddee56740 1 --2- 172.21.15.47:0/3426684664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201c60a0 0x556d201c7500 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009590 tx=0x7f1dcc0097a0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.495+0000 7f1ddee56740 1 -- 172.21.15.47:0/3426684664 shutdown_connections 2024-03-21T11:29:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.495+0000 7f1ddee56740 1 --2- 172.21.15.47:0/3426684664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201cbc00 0x556d201c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.495+0000 7f1ddee56740 1 --2- 172.21.15.47:0/3426684664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201d7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.495+0000 7f1ddee56740 1 --2- 172.21.15.47:0/3426684664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201c60a0 0x556d201c7500 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.495+0000 7f1ddee56740 1 -- 172.21.15.47:0/3426684664 >> 172.21.15.47:0/3426684664 conn(0x556d201c1610 msgr2=0x556d201c5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.495+0000 7f1ddee56740 1 -- 172.21.15.47:0/3426684664 shutdown_connections 2024-03-21T11:29:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.495+0000 7f1ddee56740 1 -- 172.21.15.47:0/3426684664 wait complete. 2024-03-21T11:29:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.495+0000 7f1ddee56740 1 -- start start 2024-03-21T11:29:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.495+0000 7f1ddee56740 1 Processor -- start 2024-03-21T11:29:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1ddee56740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201d70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1ddee56740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201d7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1ddee56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201d56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1ddee56740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd41b1920 con 0x556d201c60a0 2024-03-21T11:29:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1ddee56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd41b12c0 con 0x556d201cbc00 2024-03-21T11:29:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1ddee56740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd41ae7c0 con 0x556d201c6a90 2024-03-21T11:29:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1ddee56740 10 client.0 _ll_register_callbacks cb 0x7f1dc400a770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1dda200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201d56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1dd8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201d7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1dda200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201d56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43586/0 (socket says 172.21.15.47:43586) 2024-03-21T11:29:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1dda200640 1 -- 172.21.15.47:0/576426153 learned_addr learned my addr 172.21.15.47:0/576426153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1dd9800640 1 --2- 172.21.15.47:0/576426153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201d70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1dda200640 1 -- 172.21.15.47:0/576426153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 msgr2=0x556d201d7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1dda200640 1 --2- 172.21.15.47:0/576426153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201d7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1dda200640 1 -- 172.21.15.47:0/576426153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 msgr2=0x556d201d70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1dda200640 1 --2- 172.21.15.47:0/576426153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201d70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1dda200640 1 -- 172.21.15.47:0/576426153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d201d5d50 con 0x556d201cbc00 2024-03-21T11:29:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.496+0000 7f1dd9800640 1 --2- 172.21.15.47:0/576426153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201d70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:29:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.497+0000 7f1dda200640 1 --2- 172.21.15.47:0/576426153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201d56e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1dc401d840 tx=0x7f1dc401d640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f83afce5f09dbf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.497+0000 7f1db7400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:29:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.497+0000 7f1db7400640 1 -- 172.21.15.47:0/576426153 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4011e90 con 0x556d201cbc00 2024-03-21T11:29:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.497+0000 7f1db7400640 1 -- 172.21.15.47:0/576426153 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1dc400fcf0 con 0x556d201cbc00 2024-03-21T11:29:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.497+0000 7f1ddee56740 1 -- 172.21.15.47:0/576426153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1dcc010870 con 0x556d201cbc00 2024-03-21T11:29:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.497+0000 7f1db7400640 1 -- 172.21.15.47:0/576426153 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dc4045050 con 0x556d201cbc00 2024-03-21T11:29:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.497+0000 7f1ddee56740 10 client.27614 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.497+0000 7f1ddee56740 1 -- 172.21.15.47:0/576426153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556d202384b0 con 0x556d201cbc00 2024-03-21T11:29:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.498+0000 7f1db7400640 1 -- 172.21.15.47:0/576426153 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1dcc010870 con 0x556d201cbc00 2024-03-21T11:29:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.498+0000 7f1db7400640 1 -- 172.21.15.47:0/576426153 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1dc401fb30 con 0x556d201cbc00 2024-03-21T11:29:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.498+0000 7f1ddee56740 10 client.27614 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.498+0000 7f1ddee56740 10 client.27614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.498+0000 7f1ddee56740 20 client.27614 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.498+0000 7f1ddee56740 1 --2- 172.21.15.47:0/576426153 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d20238de0 0x556d2023b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.498+0000 7f1ddee56740 4 client.27614 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:29:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.498+0000 7f1ddee56740 1 -- 172.21.15.47:0/576426153 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556d20133740 con 0x556d20238de0 2024-03-21T11:29:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.498+0000 7f1dd9800640 1 --2- 172.21.15.47:0/576426153 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d20238de0 0x556d2023b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.499+0000 7f1dd9800640 1 --2- 172.21.15.47:0/576426153 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d20238de0 0x556d2023b200 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=638ec67a56d77ba1 server_cookie=3af0c5edbee2c08e in_seq=0 out_seq=0 2024-03-21T11:29:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.499+0000 7f1db7400640 10 client.27614 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.500+0000 7f1db7400640 1 -- 172.21.15.47:0/576426153 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556d20133740 con 0x556d20238de0 2024-03-21T11:29:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.500+0000 7f1db7400640 10 client.27614 handle_command_reply: tid=0 2024-03-21T11:29:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.500+0000 7f1ddee56740 1 client.27614 shutdown 2024-03-21T11:29:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.500+0000 7f1ddee56740 1 -- 172.21.15.47:0/576426153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 msgr2=0x556d201d56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.500+0000 7f1ddee56740 1 --2- 172.21.15.47:0/576426153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201d56e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1dc401d840 tx=0x7f1dc401d640 comp rx=0 tx=0).stop 2024-03-21T11:29:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.500+0000 7f1ddee56740 1 -- 172.21.15.47:0/576426153 shutdown_connections 2024-03-21T11:29:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.500+0000 7f1ddee56740 1 --2- 172.21.15.47:0/576426153 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d20238de0 0x556d2023b200 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.500+0000 7f1ddee56740 1 --2- 172.21.15.47:0/576426153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d201cbc00 0x556d201d56e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.500+0000 7f1ddee56740 1 --2- 172.21.15.47:0/576426153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d201c6a90 0x556d201d7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.500+0000 7f1ddee56740 1 --2- 172.21.15.47:0/576426153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d201c60a0 0x556d201d70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.500+0000 7f1db7400640 0 client.27614 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.500+0000 7f1ddee56740 1 -- 172.21.15.47:0/576426153 >> 172.21.15.47:0/576426153 conn(0x556d201c1610 msgr2=0x556d201bffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.501+0000 7f1ddee56740 1 -- 172.21.15.47:0/576426153 shutdown_connections 2024-03-21T11:29:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.501+0000 7f1ddee56740 1 -- 172.21.15.47:0/576426153 wait complete. 2024-03-21T11:29:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.501+0000 7f1ddee56740 20 client.27614 trim_cache size 0 max 16384 2024-03-21T11:29:32.505 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:29:32.505 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:29:32.505 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "741ab37d-da7d-487c-9754-b31a08e77045", 2024-03-21T11:29:32.506 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:29:32.506 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:29:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.501+0000 7f1ddc200640 1 -- 172.21.15.47:0/1267713835 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1dbc0a3070 msgr2=0x7f1dbc0a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.501+0000 7f1ddc200640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1dbc0a3070 0x7f1dbc0a5530 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x556d201c72f0 tx=0x7f1dc4005f60 comp rx=0 tx=0).stop 2024-03-21T11:29:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.501+0000 7f1ddc200640 1 -- 172.21.15.47:0/1267713835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dd4108eb0 msgr2=0x7f1dd419e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.501+0000 7f1ddc200640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dd4108eb0 0x7f1dd419e500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8036040 tx=0x7f1dc8004ec0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.502+0000 7f1ddc200640 1 -- 172.21.15.47:0/1267713835 shutdown_connections 2024-03-21T11:29:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.502+0000 7f1ddc200640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1dbc0a3070 0x7f1dbc0a5530 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.502+0000 7f1ddc200640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1dd4109870 0x7f1dd41a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.502+0000 7f1ddc200640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1dd4108eb0 0x7f1dd419e500 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.502+0000 7f1ddc200640 1 --2- 172.21.15.47:0/1267713835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dd4107cc0 0x7f1dd419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.502+0000 7f1ddc200640 1 -- 172.21.15.47:0/1267713835 >> 172.21.15.47:0/1267713835 conn(0x7f1dd4076c60 msgr2=0x7f1dd410c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.502+0000 7f1ddc200640 1 -- 172.21.15.47:0/1267713835 shutdown_connections 2024-03-21T11:29:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.502+0000 7f1ddc200640 1 -- 172.21.15.47:0/1267713835 wait complete. 2024-03-21T11:29:32.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:29:32.520 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:29:32.521 INFO:tasks.workunit.client.0.smithi047.stdout: 1 404799 8.31 MB/sec execute 136 sec latency 19.042 ms 2024-03-21T11:29:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.723+0000 7fc339800640 1 -- 172.21.15.47:0/4089065998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc334107cc0 msgr2=0x7fc3341080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.723+0000 7fc339800640 1 --2- 172.21.15.47:0/4089065998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc334107cc0 0x7fc3341080e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc324009900 tx=0x7fc32402bc30 comp rx=0 tx=0).stop 2024-03-21T11:29:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.723+0000 7fc339800640 1 -- 172.21.15.47:0/4089065998 shutdown_connections 2024-03-21T11:29:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.723+0000 7fc339800640 1 --2- 172.21.15.47:0/4089065998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334109870 0x7fc334110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.723+0000 7fc339800640 1 --2- 172.21.15.47:0/4089065998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc334108eb0 0x7fc334109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.723+0000 7fc339800640 1 --2- 172.21.15.47:0/4089065998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc334107cc0 0x7fc3341080e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.723+0000 7fc339800640 1 -- 172.21.15.47:0/4089065998 >> 172.21.15.47:0/4089065998 conn(0x7fc334076c60 msgr2=0x7fc3340790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.723+0000 7fc339800640 1 -- 172.21.15.47:0/4089065998 shutdown_connections 2024-03-21T11:29:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.723+0000 7fc339800640 1 -- 172.21.15.47:0/4089065998 wait complete. 2024-03-21T11:29:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.724+0000 7fc339800640 1 Processor -- start 2024-03-21T11:29:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.724+0000 7fc339800640 1 -- start start 2024-03-21T11:29:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.724+0000 7fc339800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc334107cc0 0x7fc33419e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.724+0000 7fc339800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc334108eb0 0x7fc33419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.724+0000 7fc339800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334109870 0x7fc3341a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.724+0000 7fc339800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc334113960 con 0x7fc334107cc0 2024-03-21T11:29:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.724+0000 7fc339800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3341137e0 con 0x7fc334108eb0 2024-03-21T11:29:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.724+0000 7fc339800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc334113ae0 con 0x7fc334109870 2024-03-21T11:29:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.725+0000 7fc333400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334109870 0x7fc3341a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.725+0000 7fc32a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc334108eb0 0x7fc33419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.725+0000 7fc332a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc334107cc0 0x7fc33419e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.725+0000 7fc333400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334109870 0x7fc3341a55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45712/0 (socket says 172.21.15.47:45712) 2024-03-21T11:29:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.725+0000 7fc32a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc334108eb0 0x7fc33419e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43590/0 (socket says 172.21.15.47:43590) 2024-03-21T11:29:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.725+0000 7fc333400640 1 -- 172.21.15.47:0/729053253 learned_addr learned my addr 172.21.15.47:0/729053253 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.725+0000 7fc333400640 1 -- 172.21.15.47:0/729053253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc334108eb0 msgr2=0x7fc33419e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.725+0000 7fc333400640 1 --2- 172.21.15.47:0/729053253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc334108eb0 0x7fc33419e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.725+0000 7fc333400640 1 -- 172.21.15.47:0/729053253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc334107cc0 msgr2=0x7fc33419e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.725+0000 7fc333400640 1 --2- 172.21.15.47:0/729053253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc334107cc0 0x7fc33419e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.725+0000 7fc333400640 1 -- 172.21.15.47:0/729053253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3341a5c30 con 0x7fc334109870 2024-03-21T11:29:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.725+0000 7fc333400640 1 --2- 172.21.15.47:0/729053253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334109870 0x7fc3341a55c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00efd0 tx=0x7fc32c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.726+0000 7fc32be00640 1 -- 172.21.15.47:0/729053253 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c003b70 con 0x7fc334109870 2024-03-21T11:29:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.726+0000 7fc32be00640 1 -- 172.21.15.47:0/729053253 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc32c003d10 con 0x7fc334109870 2024-03-21T11:29:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.726+0000 7fc32be00640 1 -- 172.21.15.47:0/729053253 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0109f0 con 0x7fc334109870 2024-03-21T11:29:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.726+0000 7fc339800640 1 -- 172.21.15.47:0/729053253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3341a5f20 con 0x7fc334109870 2024-03-21T11:29:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.726+0000 7fc339800640 1 -- 172.21.15.47:0/729053253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3341a6390 con 0x7fc334109870 2024-03-21T11:29:32.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.729+0000 7fc33c5ed740 1 Processor -- start 2024-03-21T11:29:32.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.729+0000 7fc32be00640 1 -- 172.21.15.47:0/729053253 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc32c004020 con 0x7fc334109870 2024-03-21T11:29:32.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.729+0000 7fc33c5ed740 1 -- start start 2024-03-21T11:29:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.730+0000 7fc33c5ed740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c13e80 0x5579d1c142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.730+0000 7fc33c5ed740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1c14870 0x5579d1c19370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.730+0000 7fc33c5ed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c199e0 0x5579d1c1be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.730+0000 7fc33c5ed740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d1b79250 con 0x5579d1c14870 2024-03-21T11:29:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.730+0000 7fc33c5ed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d1b21a30 con 0x5579d1c199e0 2024-03-21T11:29:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.730+0000 7fc33c5ed740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d1b752f0 con 0x5579d1c13e80 2024-03-21T11:29:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.730+0000 7fc32a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c13e80 0x5579d1c142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.730+0000 7fc32a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c13e80 0x5579d1c142a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45716/0 (socket says 172.21.15.47:45716) 2024-03-21T11:29:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.730+0000 7fc32a000640 1 -- 172.21.15.47:0/2933423918 learned_addr learned my addr 172.21.15.47:0/2933423918 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.730+0000 7fc332a00640 1 --2- 172.21.15.47:0/2933423918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1c14870 0x5579d1c19370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.730+0000 7fc32a000640 1 --2- 172.21.15.47:0/2933423918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c199e0 0x5579d1c1be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.731+0000 7fc32a000640 1 -- 172.21.15.47:0/2933423918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c199e0 msgr2=0x5579d1c1be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.731+0000 7fc32a000640 1 --2- 172.21.15.47:0/2933423918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c199e0 0x5579d1c1be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.731+0000 7fc32a000640 1 -- 172.21.15.47:0/2933423918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1c14870 msgr2=0x5579d1c19370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.731+0000 7fc32a000640 1 --2- 172.21.15.47:0/2933423918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1c14870 0x5579d1c19370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.731+0000 7fc32a000640 1 -- 172.21.15.47:0/2933423918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579d1b0c690 con 0x5579d1c13e80 2024-03-21T11:29:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.731+0000 7fc32a000640 1 --2- 172.21.15.47:0/2933423918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c13e80 0x5579d1c142a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0058c0 tx=0x7fc31c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=499d5ab44ce7a623 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.731+0000 7fc32be00640 1 --2- 172.21.15.47:0/729053253 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3180a2fa0 0x7fc3180a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.731+0000 7fc323e00640 1 -- 172.21.15.47:0/2933423918 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c004030 con 0x5579d1c13e80 2024-03-21T11:29:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.731+0000 7fc323e00640 1 -- 172.21.15.47:0/2933423918 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc31c005500 con 0x5579d1c13e80 2024-03-21T11:29:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.731+0000 7fc323e00640 1 -- 172.21.15.47:0/2933423918 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c004c10 con 0x5579d1c13e80 2024-03-21T11:29:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.731+0000 7fc32be00640 1 -- 172.21.15.47:0/729053253 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc32c0cfb10 con 0x7fc334109870 2024-03-21T11:29:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.732+0000 7fc332a00640 1 --2- 172.21.15.47:0/729053253 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3180a2fa0 0x7fc3180a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.732+0000 7fc33c5ed740 1 -- 172.21.15.47:0/2933423918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c13e80 msgr2=0x5579d1c142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.732+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/2933423918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c13e80 0x5579d1c142a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0058c0 tx=0x7fc31c005850 comp rx=0 tx=0).stop 2024-03-21T11:29:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.732+0000 7fc33c5ed740 1 -- 172.21.15.47:0/2933423918 shutdown_connections 2024-03-21T11:29:32.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.732+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/2933423918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c199e0 0x5579d1c1be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.732+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/2933423918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1c14870 0x5579d1c19370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.732+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/2933423918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c13e80 0x5579d1c142a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.732+0000 7fc33c5ed740 1 -- 172.21.15.47:0/2933423918 >> 172.21.15.47:0/2933423918 conn(0x5579d1bdc200 msgr2=0x5579d1c0f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.733+0000 7fc332a00640 1 --2- 172.21.15.47:0/729053253 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3180a2fa0 0x7fc3180a5460 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc32402c170 tx=0x7fc32402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.733+0000 7fc33c5ed740 1 -- 172.21.15.47:0/2933423918 shutdown_connections 2024-03-21T11:29:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.733+0000 7fc33c5ed740 1 -- 172.21.15.47:0/2933423918 wait complete. 2024-03-21T11:29:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.733+0000 7fc33c5ed740 1 -- start start 2024-03-21T11:29:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.733+0000 7fc33c5ed740 1 Processor -- start 2024-03-21T11:29:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc33c5ed740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c13e80 0x5579d1c1de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc33c5ed740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1c14870 0x5579d1c1e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc33c5ed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c199e0 0x5579d1c253a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc33c5ed740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d1c1c990 con 0x5579d1c14870 2024-03-21T11:29:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc33c5ed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d1b34e80 con 0x5579d1c199e0 2024-03-21T11:29:32.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc33c5ed740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d1c1cb10 con 0x5579d1c13e80 2024-03-21T11:29:32.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc33c5ed740 10 client.0 _ll_register_callbacks cb 0x5579d1b47130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:32.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc332a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c13e80 0x5579d1c1de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc333400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c199e0 0x5579d1c253a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc332a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c13e80 0x5579d1c1de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45728/0 (socket says 172.21.15.47:45728) 2024-03-21T11:29:32.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc332a00640 1 -- 172.21.15.47:0/2542182740 learned_addr learned my addr 172.21.15.47:0/2542182740 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:32.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc32a000640 1 --2- 172.21.15.47:0/2542182740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1c14870 0x5579d1c1e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc333400640 1 -- 172.21.15.47:0/2542182740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c13e80 msgr2=0x5579d1c1de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc333400640 1 --2- 172.21.15.47:0/2542182740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c13e80 0x5579d1c1de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc333400640 1 -- 172.21.15.47:0/2542182740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1c14870 msgr2=0x5579d1c1e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc333400640 1 --2- 172.21.15.47:0/2542182740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1c14870 0x5579d1c1e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc333400640 1 -- 172.21.15.47:0/2542182740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579d19577d0 con 0x5579d1c199e0 2024-03-21T11:29:32.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.734+0000 7fc32a000640 1 --2- 172.21.15.47:0/2542182740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1c14870 0x5579d1c1e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:29:32.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.735+0000 7fc333400640 1 --2- 172.21.15.47:0/2542182740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c199e0 0x5579d1c253a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0cd280 tx=0x7fc32c09bae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93eae79b766de8b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:32.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.735+0000 7fc320c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:29:32.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.735+0000 7fc320c00640 1 -- 172.21.15.47:0/2542182740 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0b4020 con 0x5579d1c199e0 2024-03-21T11:29:32.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.735+0000 7fc320c00640 1 -- 172.21.15.47:0/2542182740 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc32c09a1b0 con 0x5579d1c199e0 2024-03-21T11:29:32.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.735+0000 7fc33c5ed740 1 -- 172.21.15.47:0/2542182740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc31c005500 con 0x5579d1c199e0 2024-03-21T11:29:32.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.735+0000 7fc320c00640 1 -- 172.21.15.47:0/2542182740 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc32c09beb0 con 0x5579d1c199e0 2024-03-21T11:29:32.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.735+0000 7fc33c5ed740 10 client.27626 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:32.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.735+0000 7fc33c5ed740 1 -- 172.21.15.47:0/2542182740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5579d188d3f0 con 0x5579d1c199e0 2024-03-21T11:29:32.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.735+0000 7fc320c00640 1 -- 172.21.15.47:0/2542182740 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc31c005500 con 0x5579d1c199e0 2024-03-21T11:29:32.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.736+0000 7fc320c00640 1 -- 172.21.15.47:0/2542182740 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc32c01f020 con 0x5579d1c199e0 2024-03-21T11:29:32.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.736+0000 7fc33c5ed740 10 client.27626 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:32.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.736+0000 7fc33c5ed740 10 client.27626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:32.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.736+0000 7fc33c5ed740 20 client.27626 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:32.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.736+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/2542182740 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579d1c29be0 0x5579d1c2bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.736+0000 7fc33c5ed740 4 client.27626 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:29:32.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.736+0000 7fc33c5ed740 1 -- 172.21.15.47:0/2542182740 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579d1b86790 con 0x5579d1c29be0 2024-03-21T11:29:32.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.736+0000 7fc332a00640 1 --2- 172.21.15.47:0/2542182740 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579d1c29be0 0x5579d1c2bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.737+0000 7fc332a00640 1 --2- 172.21.15.47:0/2542182740 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579d1c29be0 0x5579d1c2bfd0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3946d984cfcaad7f server_cookie=c9e82a2aaab34f61 in_seq=0 out_seq=0 2024-03-21T11:29:32.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.737+0000 7fc320c00640 10 client.27626 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.741+0000 7fc320c00640 1 -- 172.21.15.47:0/2542182740 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5579d1b86790 con 0x5579d1c29be0 2024-03-21T11:29:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.741+0000 7fc320c00640 10 client.27626 handle_command_reply: tid=0 2024-03-21T11:29:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.742+0000 7fc33c5ed740 1 client.27626 shutdown 2024-03-21T11:29:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.742+0000 7fc33c5ed740 1 -- 172.21.15.47:0/2542182740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c199e0 msgr2=0x5579d1c253a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.742+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/2542182740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c199e0 0x5579d1c253a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0cd280 tx=0x7fc32c09bae0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.742+0000 7fc33c5ed740 1 -- 172.21.15.47:0/2542182740 shutdown_connections 2024-03-21T11:29:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.742+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/2542182740 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579d1c29be0 0x5579d1c2bfd0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.742+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/2542182740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c199e0 0x5579d1c253a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.742+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/2542182740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1c14870 0x5579d1c1e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.742+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/2542182740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c13e80 0x5579d1c1de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.742+0000 7fc33c5ed740 1 -- 172.21.15.47:0/2542182740 >> 172.21.15.47:0/2542182740 conn(0x5579d1bdc200 msgr2=0x5579d1c0ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:32.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.742+0000 7fc320c00640 0 client.27626 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:32.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.742+0000 7fc33c5ed740 1 -- 172.21.15.47:0/2542182740 shutdown_connections 2024-03-21T11:29:32.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.742+0000 7fc33c5ed740 1 -- 172.21.15.47:0/2542182740 wait complete. 2024-03-21T11:29:32.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.742+0000 7fc33c5ed740 20 client.27626 trim_cache size 0 max 16384 2024-03-21T11:29:32.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.760+0000 7fc33c5ed740 1 Processor -- start 2024-03-21T11:29:32.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.760+0000 7fc33c5ed740 1 -- start start 2024-03-21T11:29:32.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc33c5ed740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1bdc200 0x5579d1c1d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc33c5ed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c13e80 0x5579d1c25360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc33c5ed740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c14870 0x5579d1c159f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc33c5ed740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3341b0680 con 0x5579d1bdc200 2024-03-21T11:29:32.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc33c5ed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3341b2000 con 0x5579d1c13e80 2024-03-21T11:29:32.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc33c5ed740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3341b0ce0 con 0x5579d1c14870 2024-03-21T11:29:32.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc333400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c14870 0x5579d1c159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc32a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c13e80 0x5579d1c25360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc333400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c14870 0x5579d1c159f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45740/0 (socket says 172.21.15.47:45740) 2024-03-21T11:29:32.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc333400640 1 -- 172.21.15.47:0/1723829043 learned_addr learned my addr 172.21.15.47:0/1723829043 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:32.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc32a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c13e80 0x5579d1c25360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43628/0 (socket says 172.21.15.47:43628) 2024-03-21T11:29:32.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc332a00640 1 --2- 172.21.15.47:0/1723829043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1bdc200 0x5579d1c1d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc333400640 1 -- 172.21.15.47:0/1723829043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c13e80 msgr2=0x5579d1c25360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc333400640 1 --2- 172.21.15.47:0/1723829043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c13e80 0x5579d1c25360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc333400640 1 -- 172.21.15.47:0/1723829043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1bdc200 msgr2=0x5579d1c1d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc333400640 1 --2- 172.21.15.47:0/1723829043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1bdc200 0x5579d1c1d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.761+0000 7fc333400640 1 -- 172.21.15.47:0/1723829043 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579d184d0f0 con 0x5579d1c14870 2024-03-21T11:29:32.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.762+0000 7fc333400640 1 --2- 172.21.15.47:0/1723829043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c14870 0x5579d1c159f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc32c09bbb0 tx=0x7fc32c098190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cadfa8b297fd7baf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:32.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.762+0000 7fc323400640 1 -- 172.21.15.47:0/1723829043 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0b4030 con 0x5579d1c14870 2024-03-21T11:29:32.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.762+0000 7fc323400640 1 -- 172.21.15.47:0/1723829043 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc32c009820 con 0x5579d1c14870 2024-03-21T11:29:32.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.762+0000 7fc323400640 1 -- 172.21.15.47:0/1723829043 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c09b520 con 0x5579d1c14870 2024-03-21T11:29:32.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.763+0000 7fc33c5ed740 1 -- 172.21.15.47:0/1723829043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c14870 msgr2=0x5579d1c159f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.763+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/1723829043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c14870 0x5579d1c159f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc32c09bbb0 tx=0x7fc32c098190 comp rx=0 tx=0).stop 2024-03-21T11:29:32.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.763+0000 7fc33c5ed740 1 -- 172.21.15.47:0/1723829043 shutdown_connections 2024-03-21T11:29:32.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.763+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/1723829043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c14870 0x5579d1c159f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.763+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/1723829043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c13e80 0x5579d1c25360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.763+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/1723829043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1bdc200 0x5579d1c1d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.763+0000 7fc33c5ed740 1 -- 172.21.15.47:0/1723829043 >> 172.21.15.47:0/1723829043 conn(0x5579d1c19520 msgr2=0x5579d1c152e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.763+0000 7fc33c5ed740 1 -- 172.21.15.47:0/1723829043 shutdown_connections 2024-03-21T11:29:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.763+0000 7fc33c5ed740 1 -- 172.21.15.47:0/1723829043 wait complete. 2024-03-21T11:29:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.763+0000 7fc33c5ed740 1 -- start start 2024-03-21T11:29:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.764+0000 7fc33c5ed740 1 Processor -- start 2024-03-21T11:29:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.764+0000 7fc33c5ed740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1bdc200 0x5579d1c22a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.764+0000 7fc33c5ed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c13e80 0x5579d1c22fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.764+0000 7fc33c5ed740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c14870 0x5579d1c23fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.764+0000 7fc33c5ed740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3341b19a0 con 0x5579d1bdc200 2024-03-21T11:29:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.764+0000 7fc33c5ed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3341b1340 con 0x5579d1c13e80 2024-03-21T11:29:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.764+0000 7fc33c5ed740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3341ae840 con 0x5579d1c14870 2024-03-21T11:29:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.764+0000 7fc33c5ed740 10 client.0 _ll_register_callbacks cb 0x7fc32c009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.764+0000 7fc32a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c13e80 0x5579d1c22fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.764+0000 7fc332a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1bdc200 0x5579d1c22a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.764+0000 7fc32a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c13e80 0x5579d1c22fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43634/0 (socket says 172.21.15.47:43634) 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.764+0000 7fc32a000640 1 -- 172.21.15.47:0/4153147789 learned_addr learned my addr 172.21.15.47:0/4153147789 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.764+0000 7fc333400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c14870 0x5579d1c23fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.765+0000 7fc333400640 1 -- 172.21.15.47:0/4153147789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c13e80 msgr2=0x5579d1c22fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.765+0000 7fc333400640 1 --2- 172.21.15.47:0/4153147789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c13e80 0x5579d1c22fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.765+0000 7fc333400640 1 -- 172.21.15.47:0/4153147789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1bdc200 msgr2=0x5579d1c22a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.765+0000 7fc333400640 1 --2- 172.21.15.47:0/4153147789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1bdc200 0x5579d1c22a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.765+0000 7fc333400640 1 -- 172.21.15.47:0/4153147789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579d1c245c0 con 0x5579d1c14870 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.765+0000 7fc32a000640 1 --2- 172.21.15.47:0/4153147789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c13e80 0x5579d1c22fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.765+0000 7fc333400640 1 --2- 172.21.15.47:0/4153147789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c14870 0x5579d1c23fd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00cba0 tx=0x7fc32c097ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a720dc7d20566bc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.765+0000 7fc320c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.765+0000 7fc320c00640 1 -- 172.21.15.47:0/4153147789 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0b4070 con 0x5579d1c14870 2024-03-21T11:29:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.765+0000 7fc320c00640 1 -- 172.21.15.47:0/4153147789 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc32c09b300 con 0x5579d1c14870 2024-03-21T11:29:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.765+0000 7fc33c5ed740 1 -- 172.21.15.47:0/4153147789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc32c009820 con 0x5579d1c14870 2024-03-21T11:29:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.766+0000 7fc320c00640 1 -- 172.21.15.47:0/4153147789 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc32c0a48c0 con 0x5579d1c14870 2024-03-21T11:29:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.766+0000 7fc33c5ed740 10 client.27652 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.766+0000 7fc33c5ed740 1 -- 172.21.15.47:0/4153147789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5579d1c85460 con 0x5579d1c14870 2024-03-21T11:29:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.766+0000 7fc320c00640 1 -- 172.21.15.47:0/4153147789 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc32c009820 con 0x5579d1c14870 2024-03-21T11:29:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.766+0000 7fc320c00640 1 -- 172.21.15.47:0/4153147789 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc32c01f070 con 0x5579d1c14870 2024-03-21T11:29:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.766+0000 7fc33c5ed740 10 client.27652 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.766+0000 7fc33c5ed740 10 client.27652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.766+0000 7fc33c5ed740 20 client.27652 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.766+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/4153147789 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579d1c199e0 0x5579d1c87aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.766+0000 7fc33c5ed740 4 client.27652 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:29:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.766+0000 7fc33c5ed740 1 -- 172.21.15.47:0/4153147789 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5579d1b80ae0 con 0x5579d1c199e0 2024-03-21T11:29:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.766+0000 7fc332a00640 1 --2- 172.21.15.47:0/4153147789 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579d1c199e0 0x5579d1c87aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.767+0000 7fc332a00640 1 --2- 172.21.15.47:0/4153147789 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579d1c199e0 0x5579d1c87aa0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af16c836c50cc5b0 server_cookie=32a806799fe2c2e1 in_seq=0 out_seq=0 2024-03-21T11:29:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.767+0000 7fc320c00640 10 client.27652 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc320c00640 1 -- 172.21.15.47:0/4153147789 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5579d1b80ae0 con 0x5579d1c199e0 2024-03-21T11:29:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc320c00640 10 client.27652 handle_command_reply: tid=0 2024-03-21T11:29:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc33c5ed740 1 client.27652 shutdown 2024-03-21T11:29:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc33c5ed740 1 -- 172.21.15.47:0/4153147789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c14870 msgr2=0x5579d1c23fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/4153147789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c14870 0x5579d1c23fd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00cba0 tx=0x7fc32c097ad0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc33c5ed740 1 -- 172.21.15.47:0/4153147789 shutdown_connections 2024-03-21T11:29:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/4153147789 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5579d1c199e0 0x5579d1c87aa0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/4153147789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5579d1c14870 0x5579d1c23fd0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/4153147789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5579d1c13e80 0x5579d1c22fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc33c5ed740 1 --2- 172.21.15.47:0/4153147789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5579d1bdc200 0x5579d1c22a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc33c5ed740 1 -- 172.21.15.47:0/4153147789 >> 172.21.15.47:0/4153147789 conn(0x5579d1c19520 msgr2=0x5579d1c0ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:32.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc320c00640 0 client.27652 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:32.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc33c5ed740 1 -- 172.21.15.47:0/4153147789 shutdown_connections 2024-03-21T11:29:32.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc33c5ed740 1 -- 172.21.15.47:0/4153147789 wait complete. 2024-03-21T11:29:32.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.770+0000 7fc33c5ed740 20 client.27652 trim_cache size 0 max 16384 2024-03-21T11:29:32.772 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:29:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:29:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:29:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "741ab37d-da7d-487c-9754-b31a08e77045": { 2024-03-21T11:29:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:29:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "741ab37d-da7d-487c-9754-b31a08e77045", 2024-03-21T11:29:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:29:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:29:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:29:32.773 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:29:32.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.771+0000 7fc339800640 1 -- 172.21.15.47:0/729053253 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3180a2fa0 msgr2=0x7fc3180a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.771+0000 7fc339800640 1 --2- 172.21.15.47:0/729053253 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3180a2fa0 0x7fc3180a5460 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc32402c170 tx=0x7fc32402df90 comp rx=0 tx=0).stop 2024-03-21T11:29:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.771+0000 7fc339800640 1 -- 172.21.15.47:0/729053253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334109870 msgr2=0x7fc3341a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.771+0000 7fc339800640 1 --2- 172.21.15.47:0/729053253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334109870 0x7fc3341a55c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00efd0 tx=0x7fc32c00c6a0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.771+0000 7fc339800640 1 -- 172.21.15.47:0/729053253 shutdown_connections 2024-03-21T11:29:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.771+0000 7fc339800640 1 --2- 172.21.15.47:0/729053253 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3180a2fa0 0x7fc3180a5460 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.771+0000 7fc339800640 1 --2- 172.21.15.47:0/729053253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc334109870 0x7fc3341a55c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.771+0000 7fc339800640 1 --2- 172.21.15.47:0/729053253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc334108eb0 0x7fc33419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.771+0000 7fc339800640 1 --2- 172.21.15.47:0/729053253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc334107cc0 0x7fc33419e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.771+0000 7fc339800640 1 -- 172.21.15.47:0/729053253 >> 172.21.15.47:0/729053253 conn(0x7fc334076c60 msgr2=0x7fc33410c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.771+0000 7fc339800640 1 -- 172.21.15.47:0/729053253 shutdown_connections 2024-03-21T11:29:32.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:29:32.771+0000 7fc339800640 1 -- 172.21.15.47:0/729053253 wait complete. 2024-03-21T11:29:32.794 INFO:tasks.cephfs.filesystem:scrub status for tag:741ab37d-da7d-487c-9754-b31a08e77045 - {'path': '/', 'tag': '741ab37d-da7d-487c-9754-b31a08e77045', 'options': 'recursive,force'} 2024-03-21T11:29:32.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:32 smithi102 ceph-mon[36748]: pgmap v586: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 284 op/s 2024-03-21T11:29:33.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:32 smithi047 ceph-mon[31351]: pgmap v586: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 284 op/s 2024-03-21T11:29:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:32 smithi100 ceph-mon[36245]: pgmap v586: 129 pgs: 129 active+clean; 36 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 284 op/s 2024-03-21T11:29:33.520 INFO:tasks.workunit.client.0.smithi047.stdout: 1 405906 8.28 MB/sec execute 137 sec latency 41.137 ms 2024-03-21T11:29:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:33 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:29:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:33 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:29:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:33 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:29:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:33 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:29:34.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:33 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:29:34.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:33 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:29:34.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:33 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:29:34.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:33 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:29:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:33 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:29:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:33 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:29:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:33 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:29:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:33 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:29:34.521 INFO:tasks.workunit.client.0.smithi047.stdout: 1 407943 8.31 MB/sec execute 138 sec latency 17.805 ms 2024-03-21T11:29:34.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:34 smithi102 ceph-mon[36748]: pgmap v587: 129 pgs: 129 active+clean; 38 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 308 op/s 2024-03-21T11:29:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:34 smithi047 ceph-mon[31351]: pgmap v587: 129 pgs: 129 active+clean; 38 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 308 op/s 2024-03-21T11:29:35.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:34 smithi100 ceph-mon[36245]: pgmap v587: 129 pgs: 129 active+clean; 38 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 308 op/s 2024-03-21T11:29:35.521 INFO:tasks.workunit.client.0.smithi047.stdout: 1 408768 8.27 MB/sec execute 139 sec latency 28.375 ms 2024-03-21T11:29:36.521 INFO:tasks.workunit.client.0.smithi047.stdout: 1 410715 8.28 MB/sec execute 140 sec latency 14.833 ms 2024-03-21T11:29:36.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:36 smithi102 ceph-mon[36748]: pgmap v588: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2024-03-21T11:29:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.033+0000 7f7d1d000640 1 -- 172.21.15.47:0/1630386905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d18101f70 msgr2=0x7f7d18104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.033+0000 7f7d1d000640 1 --2- 172.21.15.47:0/1630386905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d18101f70 0x7f7d18104360 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009900 tx=0x7f7d0802bc30 comp rx=0 tx=0).stop 2024-03-21T11:29:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.033+0000 7f7d1d000640 1 -- 172.21.15.47:0/1630386905 shutdown_connections 2024-03-21T11:29:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.033+0000 7f7d1d000640 1 --2- 172.21.15.47:0/1630386905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d181071d0 0x7f7d18109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.033+0000 7f7d1d000640 1 --2- 172.21.15.47:0/1630386905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d181048a0 0x7f7d18106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.033+0000 7f7d1d000640 1 --2- 172.21.15.47:0/1630386905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d18101f70 0x7f7d18104360 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.033+0000 7f7d1d000640 1 -- 172.21.15.47:0/1630386905 >> 172.21.15.47:0/1630386905 conn(0x7f7d180fb980 msgr2=0x7f7d180fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.033+0000 7f7d1d000640 1 -- 172.21.15.47:0/1630386905 shutdown_connections 2024-03-21T11:29:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.033+0000 7f7d1d000640 1 -- 172.21.15.47:0/1630386905 wait complete. 2024-03-21T11:29:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.034+0000 7f7d1d000640 1 Processor -- start 2024-03-21T11:29:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.034+0000 7f7d1d000640 1 -- start start 2024-03-21T11:29:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.035+0000 7f7d1d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d18101f70 0x7f7d18199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.035+0000 7f7d1d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d181048a0 0x7f7d18199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.035+0000 7f7d1d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d181071d0 0x7f7d181a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.035+0000 7f7d1d000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1810d180 con 0x7f7d18101f70 2024-03-21T11:29:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.035+0000 7f7d1d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1810d000 con 0x7f7d181071d0 2024-03-21T11:29:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.035+0000 7f7d1d000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1810d300 con 0x7f7d181048a0 2024-03-21T11:29:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.035+0000 7f7d0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d181048a0 0x7f7d18199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.035+0000 7f7d17e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d181071d0 0x7f7d181a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.035+0000 7f7d0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d181048a0 0x7f7d18199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45780/0 (socket says 172.21.15.47:45780) 2024-03-21T11:29:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.035+0000 7f7d0fe00640 1 -- 172.21.15.47:0/2360159288 learned_addr learned my addr 172.21.15.47:0/2360159288 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.035+0000 7f7d17400640 1 --2- 172.21.15.47:0/2360159288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d18101f70 0x7f7d18199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.035+0000 7f7d0fe00640 1 -- 172.21.15.47:0/2360159288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d181071d0 msgr2=0x7f7d181a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.035+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/2360159288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d181071d0 0x7f7d181a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.036+0000 7f7d0fe00640 1 -- 172.21.15.47:0/2360159288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d18101f70 msgr2=0x7f7d18199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.036+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/2360159288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d18101f70 0x7f7d18199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.036+0000 7f7d0fe00640 1 -- 172.21.15.47:0/2360159288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d181a1670 con 0x7f7d181048a0 2024-03-21T11:29:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.036+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/2360159288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d181048a0 0x7f7d18199f90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000d770 tx=0x7f7d0000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.036+0000 7f7d14c00640 1 -- 172.21.15.47:0/2360159288 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00017070 con 0x7f7d181048a0 2024-03-21T11:29:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.036+0000 7f7d14c00640 1 -- 172.21.15.47:0/2360159288 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d00009c30 con 0x7f7d181048a0 2024-03-21T11:29:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.036+0000 7f7d14c00640 1 -- 172.21.15.47:0/2360159288 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000f620 con 0x7f7d181048a0 2024-03-21T11:29:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.036+0000 7f7d1d000640 1 -- 172.21.15.47:0/2360159288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d181a1960 con 0x7f7d181048a0 2024-03-21T11:29:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.036+0000 7f7d1d000640 1 -- 172.21.15.47:0/2360159288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d181a1e60 con 0x7f7d181048a0 2024-03-21T11:29:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.040+0000 7f7d14c00640 1 -- 172.21.15.47:0/2360159288 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7d0001a070 con 0x7f7d181048a0 2024-03-21T11:29:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.040+0000 7f7d2139f740 1 Processor -- start 2024-03-21T11:29:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.040+0000 7f7d2139f740 1 -- start start 2024-03-21T11:29:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.040+0000 7f7d2139f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71b3f0 0x558f1a71b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.040+0000 7f7d2139f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71bde0 0x558f1a7208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.040+0000 7f7d2139f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a720f50 0x558f1a7233e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.040+0000 7f7d2139f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1a6de3b0 con 0x558f1a71b3f0 2024-03-21T11:29:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.040+0000 7f7d2139f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1a680830 con 0x558f1a71bde0 2024-03-21T11:29:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.040+0000 7f7d2139f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1a6295d0 con 0x558f1a720f50 2024-03-21T11:29:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.041+0000 7f7d17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71bde0 0x558f1a7208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.041+0000 7f7d17e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71b3f0 0x558f1a71b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.041+0000 7f7d0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a720f50 0x558f1a7233e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.041+0000 7f7d17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71bde0 0x558f1a7208e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43660/0 (socket says 172.21.15.47:43660) 2024-03-21T11:29:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.041+0000 7f7d17e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71b3f0 0x558f1a71b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34802/0 (socket says 172.21.15.47:34802) 2024-03-21T11:29:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.041+0000 7f7d17400640 1 -- 172.21.15.47:0/2427150317 learned_addr learned my addr 172.21.15.47:0/2427150317 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.041+0000 7f7d0fe00640 1 -- 172.21.15.47:0/2427150317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71bde0 msgr2=0x558f1a7208e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.041+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/2427150317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71bde0 0x558f1a7208e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.041+0000 7f7d0fe00640 1 -- 172.21.15.47:0/2427150317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71b3f0 msgr2=0x558f1a71b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.041+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/2427150317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71b3f0 0x558f1a71b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.041+0000 7f7d0fe00640 1 -- 172.21.15.47:0/2427150317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f1a6412c0 con 0x558f1a720f50 2024-03-21T11:29:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.041+0000 7f7d17400640 1 --2- 172.21.15.47:0/2427150317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71bde0 0x558f1a7208e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.042+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/2427150317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a720f50 0x558f1a7233e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7d000cc900 tx=0x7f7d000d7910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c3fb65a64bcbcf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.042+0000 7f7d14c00640 1 --2- 172.21.15.47:0/2360159288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7cfc0a2f10 0x7f7cfc0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.042+0000 7f7d0cc00640 1 -- 172.21.15.47:0/2427150317 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000e2070 con 0x558f1a720f50 2024-03-21T11:29:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.042+0000 7f7d17e00640 1 --2- 172.21.15.47:0/2360159288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7cfc0a2f10 0x7f7cfc0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.042+0000 7f7d14c00640 1 -- 172.21.15.47:0/2360159288 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7d000cf1c0 con 0x7f7d181048a0 2024-03-21T11:29:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.042+0000 7f7d0cc00640 1 -- 172.21.15.47:0/2427150317 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d00009de0 con 0x558f1a720f50 2024-03-21T11:29:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.042+0000 7f7d0cc00640 1 -- 172.21.15.47:0/2427150317 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0009af70 con 0x558f1a720f50 2024-03-21T11:29:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.043+0000 7f7d2139f740 1 -- 172.21.15.47:0/2427150317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a720f50 msgr2=0x558f1a7233e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.043+0000 7f7d17e00640 1 --2- 172.21.15.47:0/2360159288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7cfc0a2f10 0x7f7cfc0a53d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x558f1a71c640 tx=0x7f7d10009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.043+0000 7f7d2139f740 1 --2- 172.21.15.47:0/2427150317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a720f50 0x558f1a7233e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7d000cc900 tx=0x7f7d000d7910 comp rx=0 tx=0).stop 2024-03-21T11:29:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.043+0000 7f7d2139f740 1 -- 172.21.15.47:0/2427150317 shutdown_connections 2024-03-21T11:29:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.043+0000 7f7d2139f740 1 --2- 172.21.15.47:0/2427150317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a720f50 0x558f1a7233e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.043+0000 7f7d2139f740 1 --2- 172.21.15.47:0/2427150317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71bde0 0x558f1a7208e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.043+0000 7f7d2139f740 1 --2- 172.21.15.47:0/2427150317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71b3f0 0x558f1a71b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.043+0000 7f7d2139f740 1 -- 172.21.15.47:0/2427150317 >> 172.21.15.47:0/2427150317 conn(0x558f1a716960 msgr2=0x558f1a716d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.043+0000 7f7d2139f740 1 -- 172.21.15.47:0/2427150317 shutdown_connections 2024-03-21T11:29:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.043+0000 7f7d2139f740 1 -- 172.21.15.47:0/2427150317 wait complete. 2024-03-21T11:29:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.043+0000 7f7d2139f740 1 -- start start 2024-03-21T11:29:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.044+0000 7f7d2139f740 1 Processor -- start 2024-03-21T11:29:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.044+0000 7f7d2139f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71b3f0 0x558f1a7255a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.044+0000 7f7d2139f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71bde0 0x558f1a725ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.044+0000 7f7d2139f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a720f50 0x558f1a72cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.044+0000 7f7d2139f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1a63ca20 con 0x558f1a71bde0 2024-03-21T11:29:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.044+0000 7f7d2139f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1a67c8d0 con 0x558f1a71b3f0 2024-03-21T11:29:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.044+0000 7f7d2139f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f1a723f00 con 0x558f1a720f50 2024-03-21T11:29:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.044+0000 7f7d2139f740 10 client.0 _ll_register_callbacks cb 0x558f1a6791c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.044+0000 7f7d17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71b3f0 0x558f1a7255a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.045+0000 7f7d17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71b3f0 0x558f1a7255a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43674/0 (socket says 172.21.15.47:43674) 2024-03-21T11:29:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.045+0000 7f7d17400640 1 -- 172.21.15.47:0/1677939344 learned_addr learned my addr 172.21.15.47:0/1677939344 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.045+0000 7f7d17e00640 1 --2- 172.21.15.47:0/1677939344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a720f50 0x558f1a72cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.045+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/1677939344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71bde0 0x558f1a725ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.045+0000 7f7d17400640 1 -- 172.21.15.47:0/1677939344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a720f50 msgr2=0x558f1a72cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.045+0000 7f7d17400640 1 --2- 172.21.15.47:0/1677939344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a720f50 0x558f1a72cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.045+0000 7f7d17400640 1 -- 172.21.15.47:0/1677939344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71bde0 msgr2=0x558f1a725ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.045+0000 7f7d17400640 1 --2- 172.21.15.47:0/1677939344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71bde0 0x558f1a725ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.045+0000 7f7d17400640 1 -- 172.21.15.47:0/1677939344 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f1a395210 con 0x558f1a71b3f0 2024-03-21T11:29:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.045+0000 7f7d17e00640 1 --2- 172.21.15.47:0/1677939344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a720f50 0x558f1a72cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:29:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.045+0000 7f7d17400640 1 --2- 172.21.15.47:0/1677939344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71b3f0 0x558f1a7255a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009230 tx=0x7f7d08004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace5bb348936b937 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.045+0000 7f7d05600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:29:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.046+0000 7f7d05600640 1 -- 172.21.15.47:0/1677939344 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08005230 con 0x558f1a71b3f0 2024-03-21T11:29:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.046+0000 7f7d05600640 1 -- 172.21.15.47:0/1677939344 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7d080053d0 con 0x558f1a71b3f0 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.046+0000 7f7d2139f740 1 -- 172.21.15.47:0/1677939344 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d00009de0 con 0x558f1a71b3f0 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.046+0000 7f7d05600640 1 -- 172.21.15.47:0/1677939344 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d0804e050 con 0x558f1a71b3f0 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.046+0000 7f7d2139f740 10 client.27644 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.046+0000 7f7d2139f740 1 -- 172.21.15.47:0/1677939344 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558f1a461620 con 0x558f1a71b3f0 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.046+0000 7f7d05600640 1 -- 172.21.15.47:0/1677939344 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d00009de0 con 0x558f1a71b3f0 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.046+0000 7f7d05600640 1 -- 172.21.15.47:0/1677939344 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7d080037b0 con 0x558f1a71b3f0 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.046+0000 7f7d2139f740 10 client.27644 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.046+0000 7f7d2139f740 10 client.27644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.046+0000 7f7d2139f740 20 client.27644 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.046+0000 7f7d2139f740 1 --2- 172.21.15.47:0/1677939344 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558f1a7313a0 0x558f1a733790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.046+0000 7f7d2139f740 4 client.27644 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.046+0000 7f7d2139f740 1 -- 172.21.15.47:0/1677939344 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f1a68dd70 con 0x558f1a7313a0 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.047+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/1677939344 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558f1a7313a0 0x558f1a733790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.047+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/1677939344 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558f1a7313a0 0x558f1a733790 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf3668bebd6ae69 server_cookie=8ca0014f06ee29a0 in_seq=0 out_seq=0 2024-03-21T11:29:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.047+0000 7f7d05600640 10 client.27644 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.048+0000 7f7d05600640 1 -- 172.21.15.47:0/1677939344 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558f1a68dd70 con 0x558f1a7313a0 2024-03-21T11:29:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.048+0000 7f7d05600640 10 client.27644 handle_command_reply: tid=0 2024-03-21T11:29:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.048+0000 7f7d2139f740 1 client.27644 shutdown 2024-03-21T11:29:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.048+0000 7f7d2139f740 1 -- 172.21.15.47:0/1677939344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71b3f0 msgr2=0x558f1a7255a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.048+0000 7f7d2139f740 1 --2- 172.21.15.47:0/1677939344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71b3f0 0x558f1a7255a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009230 tx=0x7f7d08004270 comp rx=0 tx=0).stop 2024-03-21T11:29:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.048+0000 7f7d2139f740 1 -- 172.21.15.47:0/1677939344 shutdown_connections 2024-03-21T11:29:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.048+0000 7f7d2139f740 1 --2- 172.21.15.47:0/1677939344 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558f1a7313a0 0x558f1a733790 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.048+0000 7f7d2139f740 1 --2- 172.21.15.47:0/1677939344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a720f50 0x558f1a72cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.048+0000 7f7d2139f740 1 --2- 172.21.15.47:0/1677939344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71bde0 0x558f1a725ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.048+0000 7f7d2139f740 1 --2- 172.21.15.47:0/1677939344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71b3f0 0x558f1a7255a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.048+0000 7f7d05600640 0 client.27644 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.048+0000 7f7d2139f740 1 -- 172.21.15.47:0/1677939344 >> 172.21.15.47:0/1677939344 conn(0x558f1a716960 msgr2=0x558f1a715b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.048+0000 7f7d2139f740 1 -- 172.21.15.47:0/1677939344 shutdown_connections 2024-03-21T11:29:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.048+0000 7f7d2139f740 1 -- 172.21.15.47:0/1677939344 wait complete. 2024-03-21T11:29:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.049+0000 7f7d2139f740 20 client.27644 trim_cache size 0 max 16384 2024-03-21T11:29:37.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:36 smithi047 ceph-mon[31351]: pgmap v588: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2024-03-21T11:29:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.066+0000 7f7d2139f740 1 Processor -- start 2024-03-21T11:29:37.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.066+0000 7f7d2139f740 1 -- start start 2024-03-21T11:29:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d2139f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a716960 0x558f1a719bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d2139f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a71b3f0 0x558f1a71a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d2139f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71bde0 0x558f1a738070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d2139f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d181ac240 con 0x558f1a71bde0 2024-03-21T11:29:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d2139f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d181adbc0 con 0x558f1a716960 2024-03-21T11:29:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d2139f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d181ac8a0 con 0x558f1a71b3f0 2024-03-21T11:29:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a71b3f0 0x558f1a71a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a716960 0x558f1a719bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a71b3f0 0x558f1a71a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45798/0 (socket says 172.21.15.47:45798) 2024-03-21T11:29:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a716960 0x558f1a719bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43680/0 (socket says 172.21.15.47:43680) 2024-03-21T11:29:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d0fe00640 1 -- 172.21.15.47:0/2294303203 learned_addr learned my addr 172.21.15.47:0/2294303203 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d0fe00640 1 -- 172.21.15.47:0/2294303203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a716960 msgr2=0x558f1a719bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/2294303203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a716960 0x558f1a719bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d17e00640 1 --2- 172.21.15.47:0/2294303203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71bde0 0x558f1a738070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.067+0000 7f7d0fe00640 1 -- 172.21.15.47:0/2294303203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71bde0 msgr2=0x558f1a738070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.068+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/2294303203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71bde0 0x558f1a738070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.068+0000 7f7d0fe00640 1 -- 172.21.15.47:0/2294303203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f1a7176d0 con 0x558f1a71b3f0 2024-03-21T11:29:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.068+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/2294303203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a71b3f0 0x558f1a71a100 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7d000ccca0 tx=0x7f7d00010cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=383a9a813e01aae9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.068+0000 7f7d07e00640 1 -- 172.21.15.47:0/2294303203 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00021040 con 0x558f1a71b3f0 2024-03-21T11:29:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.068+0000 7f7d07e00640 1 -- 172.21.15.47:0/2294303203 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d00097040 con 0x558f1a71b3f0 2024-03-21T11:29:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.068+0000 7f7d07e00640 1 -- 172.21.15.47:0/2294303203 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0005ef20 con 0x558f1a71b3f0 2024-03-21T11:29:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.069+0000 7f7d2139f740 1 -- 172.21.15.47:0/2294303203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a71b3f0 msgr2=0x558f1a71a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.069+0000 7f7d2139f740 1 --2- 172.21.15.47:0/2294303203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a71b3f0 0x558f1a71a100 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7d000ccca0 tx=0x7f7d00010cd0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.069+0000 7f7d2139f740 1 -- 172.21.15.47:0/2294303203 shutdown_connections 2024-03-21T11:29:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.069+0000 7f7d2139f740 1 --2- 172.21.15.47:0/2294303203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71bde0 0x558f1a738070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.069+0000 7f7d2139f740 1 --2- 172.21.15.47:0/2294303203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a71b3f0 0x558f1a71a100 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.069+0000 7f7d2139f740 1 --2- 172.21.15.47:0/2294303203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a716960 0x558f1a719bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.069+0000 7f7d2139f740 1 -- 172.21.15.47:0/2294303203 >> 172.21.15.47:0/2294303203 conn(0x558f1a720a90 msgr2=0x558f1a716d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.069+0000 7f7d2139f740 1 -- 172.21.15.47:0/2294303203 shutdown_connections 2024-03-21T11:29:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.069+0000 7f7d2139f740 1 -- 172.21.15.47:0/2294303203 wait complete. 2024-03-21T11:29:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.069+0000 7f7d2139f740 1 -- start start 2024-03-21T11:29:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.070+0000 7f7d2139f740 1 Processor -- start 2024-03-21T11:29:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.070+0000 7f7d2139f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a716960 0x558f1a72a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.070+0000 7f7d2139f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71b3f0 0x558f1a72a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.070+0000 7f7d2139f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71bde0 0x558f1a72b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.070+0000 7f7d2139f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d181ad560 con 0x558f1a71b3f0 2024-03-21T11:29:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.070+0000 7f7d2139f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d181acf00 con 0x558f1a71bde0 2024-03-21T11:29:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.070+0000 7f7d2139f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d181aa400 con 0x558f1a716960 2024-03-21T11:29:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.070+0000 7f7d2139f740 10 client.0 _ll_register_callbacks cb 0x7f7d0802c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.070+0000 7f7d17400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a716960 0x558f1a72a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.070+0000 7f7d17400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a716960 0x558f1a72a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45804/0 (socket says 172.21.15.47:45804) 2024-03-21T11:29:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.070+0000 7f7d17400640 1 -- 172.21.15.47:0/869088964 learned_addr learned my addr 172.21.15.47:0/869088964 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.070+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/869088964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71b3f0 0x558f1a72a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.071+0000 7f7d17e00640 1 --2- 172.21.15.47:0/869088964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71bde0 0x558f1a72b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.071+0000 7f7d17400640 1 -- 172.21.15.47:0/869088964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71bde0 msgr2=0x558f1a72b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.071+0000 7f7d17400640 1 --2- 172.21.15.47:0/869088964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71bde0 0x558f1a72b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.071+0000 7f7d17400640 1 -- 172.21.15.47:0/869088964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71b3f0 msgr2=0x558f1a72a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.071+0000 7f7d17400640 1 --2- 172.21.15.47:0/869088964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71b3f0 0x558f1a72a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.071+0000 7f7d17400640 1 -- 172.21.15.47:0/869088964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f1a72bc60 con 0x558f1a716960 2024-03-21T11:29:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.071+0000 7f7d17e00640 1 --2- 172.21.15.47:0/869088964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71bde0 0x558f1a72b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:29:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.071+0000 7f7d17400640 1 --2- 172.21.15.47:0/869088964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a716960 0x558f1a72a190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7d0802c7b0 tx=0x7f7d08003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f93a2ccacfc0a576 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.071+0000 7f7d05600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:29:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.071+0000 7f7d05600640 1 -- 172.21.15.47:0/869088964 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08057070 con 0x558f1a716960 2024-03-21T11:29:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.072+0000 7f7d05600640 1 -- 172.21.15.47:0/869088964 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7d08003800 con 0x558f1a716960 2024-03-21T11:29:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.072+0000 7f7d2139f740 1 -- 172.21.15.47:0/869088964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d00097040 con 0x558f1a716960 2024-03-21T11:29:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.072+0000 7f7d05600640 1 -- 172.21.15.47:0/869088964 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d08063050 con 0x558f1a716960 2024-03-21T11:29:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.072+0000 7f7d2139f740 10 client.27670 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.072+0000 7f7d2139f740 1 -- 172.21.15.47:0/869088964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558f1a78d3e0 con 0x558f1a716960 2024-03-21T11:29:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.072+0000 7f7d05600640 1 -- 172.21.15.47:0/869088964 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d00097040 con 0x558f1a716960 2024-03-21T11:29:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.072+0000 7f7d05600640 1 -- 172.21.15.47:0/869088964 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7d08004750 con 0x558f1a716960 2024-03-21T11:29:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.072+0000 7f7d2139f740 10 client.27670 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.072+0000 7f7d2139f740 10 client.27670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.072+0000 7f7d2139f740 20 client.27670 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.072+0000 7f7d2139f740 1 --2- 172.21.15.47:0/869088964 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558f1a720f50 0x558f1a78fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.072+0000 7f7d2139f740 4 client.27670 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:29:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.072+0000 7f7d2139f740 1 -- 172.21.15.47:0/869088964 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558f1a6880c0 con 0x558f1a720f50 2024-03-21T11:29:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.073+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/869088964 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558f1a720f50 0x558f1a78fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.073+0000 7f7d0fe00640 1 --2- 172.21.15.47:0/869088964 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558f1a720f50 0x558f1a78fd10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46baab98f7c63460 server_cookie=ffe8c4ed96128f6f in_seq=0 out_seq=0 2024-03-21T11:29:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.073+0000 7f7d05600640 10 client.27670 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:37.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:36 smithi100 ceph-mon[36245]: pgmap v588: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2024-03-21T11:29:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.276+0000 7f7d05600640 1 -- 172.21.15.47:0/869088964 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558f1a6880c0 con 0x558f1a720f50 2024-03-21T11:29:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.276+0000 7f7d05600640 10 client.27670 handle_command_reply: tid=0 2024-03-21T11:29:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.276+0000 7f7d2139f740 1 client.27670 shutdown 2024-03-21T11:29:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.276+0000 7f7d2139f740 1 -- 172.21.15.47:0/869088964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a716960 msgr2=0x558f1a72a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.276+0000 7f7d2139f740 1 --2- 172.21.15.47:0/869088964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a716960 0x558f1a72a190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7d0802c7b0 tx=0x7f7d08003f50 comp rx=0 tx=0).stop 2024-03-21T11:29:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.276+0000 7f7d2139f740 1 -- 172.21.15.47:0/869088964 shutdown_connections 2024-03-21T11:29:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.276+0000 7f7d2139f740 1 --2- 172.21.15.47:0/869088964 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558f1a720f50 0x558f1a78fd10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.276+0000 7f7d2139f740 1 --2- 172.21.15.47:0/869088964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f1a71bde0 0x558f1a72b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.276+0000 7f7d2139f740 1 --2- 172.21.15.47:0/869088964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f1a71b3f0 0x558f1a72a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.276+0000 7f7d2139f740 1 --2- 172.21.15.47:0/869088964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558f1a716960 0x558f1a72a190 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.276+0000 7f7d05600640 0 client.27670 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.276+0000 7f7d2139f740 1 -- 172.21.15.47:0/869088964 >> 172.21.15.47:0/869088964 conn(0x558f1a720a90 msgr2=0x558f1a719bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.276+0000 7f7d2139f740 1 -- 172.21.15.47:0/869088964 shutdown_connections 2024-03-21T11:29:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.276+0000 7f7d2139f740 1 -- 172.21.15.47:0/869088964 wait complete. 2024-03-21T11:29:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.277+0000 7f7d2139f740 20 client.27670 trim_cache size 0 max 16384 2024-03-21T11:29:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:29:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:29:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:29:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:29:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.277+0000 7f7d1d000640 1 -- 172.21.15.47:0/2360159288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7cfc0a2f10 msgr2=0x7f7cfc0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.277+0000 7f7d1d000640 1 --2- 172.21.15.47:0/2360159288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7cfc0a2f10 0x7f7cfc0a53d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x558f1a71c640 tx=0x7f7d10009500 comp rx=0 tx=0).stop 2024-03-21T11:29:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.277+0000 7f7d1d000640 1 -- 172.21.15.47:0/2360159288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d181048a0 msgr2=0x7f7d18199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.277+0000 7f7d1d000640 1 --2- 172.21.15.47:0/2360159288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d181048a0 0x7f7d18199f90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000d770 tx=0x7f7d0000dc30 comp rx=0 tx=0).stop 2024-03-21T11:29:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.277+0000 7f7d1d000640 1 -- 172.21.15.47:0/2360159288 shutdown_connections 2024-03-21T11:29:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.277+0000 7f7d1d000640 1 --2- 172.21.15.47:0/2360159288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7cfc0a2f10 0x7f7cfc0a53d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.277+0000 7f7d1d000640 1 --2- 172.21.15.47:0/2360159288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d181071d0 0x7f7d181a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.277+0000 7f7d1d000640 1 --2- 172.21.15.47:0/2360159288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d181048a0 0x7f7d18199f90 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.277+0000 7f7d1d000640 1 --2- 172.21.15.47:0/2360159288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d18101f70 0x7f7d18199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.277+0000 7f7d1d000640 1 -- 172.21.15.47:0/2360159288 >> 172.21.15.47:0/2360159288 conn(0x7f7d180fb980 msgr2=0x7f7d18101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.278+0000 7f7d1d000640 1 -- 172.21.15.47:0/2360159288 shutdown_connections 2024-03-21T11:29:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:37.278+0000 7f7d1d000640 1 -- 172.21.15.47:0/2360159288 wait complete. 2024-03-21T11:29:37.521 INFO:tasks.workunit.client.0.smithi047.stdout: 1 411925 8.28 MB/sec execute 141 sec latency 141.126 ms 2024-03-21T11:29:37.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:37 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:29:38.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:37 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:29:38.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:37 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:29:38.521 INFO:tasks.workunit.client.0.smithi047.stdout: 1 412997 8.26 MB/sec execute 142 sec latency 79.182 ms 2024-03-21T11:29:38.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:38 smithi102 ceph-mon[36748]: pgmap v589: 129 pgs: 129 active+clean; 40 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 308 op/s 2024-03-21T11:29:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:38 smithi047 ceph-mon[31351]: pgmap v589: 129 pgs: 129 active+clean; 40 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 308 op/s 2024-03-21T11:29:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:38 smithi100 ceph-mon[36245]: pgmap v589: 129 pgs: 129 active+clean; 40 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 308 op/s 2024-03-21T11:29:39.521 INFO:tasks.workunit.client.0.smithi047.stdout: 1 415038 8.29 MB/sec execute 143 sec latency 24.451 ms 2024-03-21T11:29:40.521 INFO:tasks.workunit.client.0.smithi047.stdout: 1 415953 8.25 MB/sec execute 144 sec latency 17.454 ms 2024-03-21T11:29:40.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:40 smithi102 ceph-mon[36748]: pgmap v590: 129 pgs: 129 active+clean; 37 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 290 op/s 2024-03-21T11:29:41.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:40 smithi047 ceph-mon[31351]: pgmap v590: 129 pgs: 129 active+clean; 37 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 290 op/s 2024-03-21T11:29:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:40 smithi100 ceph-mon[36245]: pgmap v590: 129 pgs: 129 active+clean; 37 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 290 op/s 2024-03-21T11:29:41.521 INFO:tasks.workunit.client.0.smithi047.stdout: 1 417972 8.27 MB/sec execute 145 sec latency 12.875 ms 2024-03-21T11:29:42.521 INFO:tasks.workunit.client.0.smithi047.stdout: 1 419179 8.26 MB/sec execute 146 sec latency 13.470 ms 2024-03-21T11:29:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:42 smithi102 ceph-mon[36748]: pgmap v591: 129 pgs: 129 active+clean; 34 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 255 op/s 2024-03-21T11:29:43.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:42 smithi047 ceph-mon[31351]: pgmap v591: 129 pgs: 129 active+clean; 34 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 255 op/s 2024-03-21T11:29:43.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:42 smithi100 ceph-mon[36245]: pgmap v591: 129 pgs: 129 active+clean; 34 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 255 op/s 2024-03-21T11:29:43.522 INFO:tasks.workunit.client.0.smithi047.stdout: 1 420929 8.25 MB/sec execute 147 sec latency 47.421 ms 2024-03-21T11:29:44.522 INFO:tasks.workunit.client.0.smithi047.stdout: 1 422406 8.28 MB/sec execute 148 sec latency 58.260 ms 2024-03-21T11:29:44.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:44 smithi102 ceph-mon[36748]: pgmap v592: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 334 op/s 2024-03-21T11:29:45.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:44 smithi047 ceph-mon[31351]: pgmap v592: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 334 op/s 2024-03-21T11:29:45.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:44 smithi100 ceph-mon[36245]: pgmap v592: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 334 op/s 2024-03-21T11:29:45.522 INFO:tasks.workunit.client.0.smithi047.stdout: 1 423383 8.24 MB/sec execute 149 sec latency 12.149 ms 2024-03-21T11:29:46.523 INFO:tasks.workunit.client.0.smithi047.stdout: 1 425471 8.27 MB/sec execute 150 sec latency 12.970 ms 2024-03-21T11:29:47.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:46 smithi047 ceph-mon[31351]: pgmap v593: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 271 op/s 2024-03-21T11:29:47.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:46 smithi100 ceph-mon[36245]: pgmap v593: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 271 op/s 2024-03-21T11:29:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:46 smithi102 ceph-mon[36748]: pgmap v593: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 271 op/s 2024-03-21T11:29:47.522 INFO:tasks.workunit.client.0.smithi047.stdout: 1 426350 8.24 MB/sec execute 151 sec latency 151.705 ms 2024-03-21T11:29:48.522 INFO:tasks.workunit.client.0.smithi047.stdout: 1 428104 8.23 MB/sec execute 152 sec latency 19.552 ms 2024-03-21T11:29:49.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:48 smithi047 ceph-mon[31351]: pgmap v594: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 287 op/s 2024-03-21T11:29:49.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:48 smithi100 ceph-mon[36245]: pgmap v594: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 287 op/s 2024-03-21T11:29:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:48 smithi102 ceph-mon[36748]: pgmap v594: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 287 op/s 2024-03-21T11:29:49.522 INFO:tasks.workunit.client.0.smithi047.stdout: 1 429482 8.26 MB/sec execute 153 sec latency 40.341 ms 2024-03-21T11:29:50.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:29:50.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:49 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:50.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:29:50.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:49 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:50.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:29:50.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:49 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.435+0000 7f1ca0e00640 1 -- 172.21.15.47:0/422742260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c9c104910 msgr2=0x7f1c9c104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.435+0000 7f1ca0e00640 1 --2- 172.21.15.47:0/422742260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c9c104910 0x7f1c9c104d90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009900 tx=0x7f1c8802bc30 comp rx=0 tx=0).stop 2024-03-21T11:29:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.435+0000 7f1ca0e00640 1 -- 172.21.15.47:0/422742260 shutdown_connections 2024-03-21T11:29:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.435+0000 7f1ca0e00640 1 --2- 172.21.15.47:0/422742260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c9c1052d0 0x7f1c9c10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.435+0000 7f1ca0e00640 1 --2- 172.21.15.47:0/422742260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c9c104910 0x7f1c9c104d90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.435+0000 7f1ca0e00640 1 --2- 172.21.15.47:0/422742260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c9c103720 0x7f1c9c103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.435+0000 7f1ca0e00640 1 -- 172.21.15.47:0/422742260 >> 172.21.15.47:0/422742260 conn(0x7f1c9c0fecb0 msgr2=0x7f1c9c1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.435+0000 7f1ca0e00640 1 -- 172.21.15.47:0/422742260 shutdown_connections 2024-03-21T11:29:50.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.435+0000 7f1ca0e00640 1 -- 172.21.15.47:0/422742260 wait complete. 2024-03-21T11:29:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.436+0000 7f1ca0e00640 1 Processor -- start 2024-03-21T11:29:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.437+0000 7f1ca0e00640 1 -- start start 2024-03-21T11:29:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.437+0000 7f1ca0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c9c103720 0x7f1c9c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.437+0000 7f1ca0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c9c104910 0x7f1c9c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.437+0000 7f1ca0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c9c1052d0 0x7f1c9c1a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.437+0000 7f1ca0e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c10f380 con 0x7f1c9c1052d0 2024-03-21T11:29:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.437+0000 7f1ca0e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c10f200 con 0x7f1c9c103720 2024-03-21T11:29:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.437+0000 7f1ca0e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c10f500 con 0x7f1c9c104910 2024-03-21T11:29:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.437+0000 7f1c9b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c9c103720 0x7f1c9c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.437+0000 7f1c9b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c9c103720 0x7f1c9c199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53538/0 (socket says 172.21.15.47:53538) 2024-03-21T11:29:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.437+0000 7f1c9b400640 1 -- 172.21.15.47:0/2135204330 learned_addr learned my addr 172.21.15.47:0/2135204330 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.437+0000 7f1c9aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c9c104910 0x7f1c9c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.437+0000 7f1c9be00640 1 --2- 172.21.15.47:0/2135204330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c9c1052d0 0x7f1c9c1a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.438+0000 7f1c9b400640 1 -- 172.21.15.47:0/2135204330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c9c104910 msgr2=0x7f1c9c199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.438+0000 7f1c9b400640 1 --2- 172.21.15.47:0/2135204330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c9c104910 0x7f1c9c199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.438+0000 7f1c9b400640 1 -- 172.21.15.47:0/2135204330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c9c1052d0 msgr2=0x7f1c9c1a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.438+0000 7f1c9b400640 1 --2- 172.21.15.47:0/2135204330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c9c1052d0 0x7f1c9c1a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.438+0000 7f1c9b400640 1 -- 172.21.15.47:0/2135204330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c1a1630 con 0x7f1c9c103720 2024-03-21T11:29:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.438+0000 7f1c9aa00640 1 --2- 172.21.15.47:0/2135204330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c9c104910 0x7f1c9c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:29:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.438+0000 7f1c9b400640 1 --2- 172.21.15.47:0/2135204330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c9c103720 0x7f1c9c199a10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00e9c0 tx=0x7f1c8c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.438+0000 7f1c93e00640 1 -- 172.21.15.47:0/2135204330 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c00cd80 con 0x7f1c9c103720 2024-03-21T11:29:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.438+0000 7f1c93e00640 1 -- 172.21.15.47:0/2135204330 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c8c0035f0 con 0x7f1c9c103720 2024-03-21T11:29:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.438+0000 7f1c93e00640 1 -- 172.21.15.47:0/2135204330 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c010690 con 0x7f1c9c103720 2024-03-21T11:29:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.438+0000 7f1ca0e00640 1 -- 172.21.15.47:0/2135204330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9c1a1920 con 0x7f1c9c103720 2024-03-21T11:29:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.438+0000 7f1ca0e00640 1 -- 172.21.15.47:0/2135204330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9c1a1df0 con 0x7f1c9c103720 2024-03-21T11:29:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.442+0000 7f1ca500b740 1 Processor -- start 2024-03-21T11:29:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.442+0000 7f1ca500b740 1 -- start start 2024-03-21T11:29:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.442+0000 7f1c93e00640 1 -- 172.21.15.47:0/2135204330 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1c8c0037a0 con 0x7f1c9c103720 2024-03-21T11:29:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.442+0000 7f1ca500b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b54810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1ca500b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b54de0 0x5601f9b598e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1ca500b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b59f50 0x5601f9b5c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1ca500b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f9b173b0 con 0x5601f9b54de0 2024-03-21T11:29:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1ca500b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f9ab9830 con 0x5601f9b543f0 2024-03-21T11:29:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1ca500b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f9a625d0 con 0x5601f9b59f50 2024-03-21T11:29:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1c9aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b59f50 0x5601f9b5c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1c9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b54810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1c9b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b54de0 0x5601f9b598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1c9aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b59f50 0x5601f9b5c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37174/0 (socket says 172.21.15.47:37174) 2024-03-21T11:29:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1c9aa00640 1 -- 172.21.15.47:0/2108479931 learned_addr learned my addr 172.21.15.47:0/2108479931 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1c9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b54810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53552/0 (socket says 172.21.15.47:53552) 2024-03-21T11:29:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1c9be00640 1 -- 172.21.15.47:0/2108479931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b59f50 msgr2=0x5601f9b5c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1c9be00640 1 --2- 172.21.15.47:0/2108479931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b59f50 0x5601f9b5c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1c9be00640 1 -- 172.21.15.47:0/2108479931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b54de0 msgr2=0x5601f9b598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1c9be00640 1 --2- 172.21.15.47:0/2108479931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b54de0 0x5601f9b598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.443+0000 7f1c9be00640 1 -- 172.21.15.47:0/2108479931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601f9a7a2c0 con 0x5601f9b543f0 2024-03-21T11:29:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.444+0000 7f1c9be00640 1 --2- 172.21.15.47:0/2108479931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b54810 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009e20 tx=0x7f1c94010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41cd3b350aa0866a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.444+0000 7f1c90c00640 1 -- 172.21.15.47:0/2108479931 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94016070 con 0x5601f9b543f0 2024-03-21T11:29:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.444+0000 7f1c90c00640 1 -- 172.21.15.47:0/2108479931 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c94004d20 con 0x5601f9b543f0 2024-03-21T11:29:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.444+0000 7f1c90c00640 1 -- 172.21.15.47:0/2108479931 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c940128b0 con 0x5601f9b543f0 2024-03-21T11:29:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.444+0000 7f1c93e00640 1 --2- 172.21.15.47:0/2135204330 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1c800a2f90 0x7f1c800a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.445+0000 7f1c93e00640 1 -- 172.21.15.47:0/2135204330 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1c8c0cfcb0 con 0x7f1c9c103720 2024-03-21T11:29:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.445+0000 7f1c9b400640 1 --2- 172.21.15.47:0/2135204330 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1c800a2f90 0x7f1c800a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.445+0000 7f1ca500b740 1 -- 172.21.15.47:0/2108479931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 msgr2=0x5601f9b54810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.445+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2108479931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b54810 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009e20 tx=0x7f1c94010740 comp rx=0 tx=0).stop 2024-03-21T11:29:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.445+0000 7f1ca500b740 1 -- 172.21.15.47:0/2108479931 shutdown_connections 2024-03-21T11:29:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.445+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2108479931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b59f50 0x5601f9b5c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.445+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2108479931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b54de0 0x5601f9b598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.445+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2108479931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b54810 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.445+0000 7f1ca500b740 1 -- 172.21.15.47:0/2108479931 >> 172.21.15.47:0/2108479931 conn(0x5601f9b4f960 msgr2=0x5601f9b4fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.445+0000 7f1ca500b740 1 -- 172.21.15.47:0/2108479931 shutdown_connections 2024-03-21T11:29:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.445+0000 7f1ca500b740 1 -- 172.21.15.47:0/2108479931 wait complete. 2024-03-21T11:29:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.445+0000 7f1c9b400640 1 --2- 172.21.15.47:0/2135204330 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1c800a2f90 0x7f1c800a5450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0989c0 tx=0x7f1c8c0d36f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.446+0000 7f1ca500b740 1 -- start start 2024-03-21T11:29:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.446+0000 7f1ca500b740 1 Processor -- start 2024-03-21T11:29:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1ca500b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b5e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1ca500b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b54de0 0x5601f9b5eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1ca500b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b59f50 0x5601f9b65b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1ca500b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f9a75a20 con 0x5601f9b59f50 2024-03-21T11:29:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1ca500b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f9ab58d0 con 0x5601f9b543f0 2024-03-21T11:29:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1ca500b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f9b5cf00 con 0x5601f9b54de0 2024-03-21T11:29:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1ca500b740 10 client.0 _ll_register_callbacks cb 0x5601f9ab21c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1c9aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b54de0 0x5601f9b5eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1c9aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b54de0 0x5601f9b5eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37176/0 (socket says 172.21.15.47:37176) 2024-03-21T11:29:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1c9aa00640 1 -- 172.21.15.47:0/2348930039 learned_addr learned my addr 172.21.15.47:0/2348930039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1c9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b5e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1c9be00640 1 --2- 172.21.15.47:0/2348930039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b59f50 0x5601f9b65b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1c9aa00640 1 -- 172.21.15.47:0/2348930039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 msgr2=0x5601f9b5e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1c9aa00640 1 --2- 172.21.15.47:0/2348930039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b5e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1c9aa00640 1 -- 172.21.15.47:0/2348930039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b59f50 msgr2=0x5601f9b65b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1c9aa00640 1 --2- 172.21.15.47:0/2348930039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b59f50 0x5601f9b65b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1c9aa00640 1 -- 172.21.15.47:0/2348930039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601f97ce210 con 0x5601f9b54de0 2024-03-21T11:29:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.447+0000 7f1c9be00640 1 --2- 172.21.15.47:0/2348930039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b59f50 0x5601f9b65b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:29:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.448+0000 7f1c9aa00640 1 --2- 172.21.15.47:0/2348930039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b54de0 0x5601f9b5eaf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009230 tx=0x7f1c88004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16ad4ef73d81d2f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.448+0000 7f1c85600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:29:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.448+0000 7f1c85600640 1 -- 172.21.15.47:0/2348930039 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88005230 con 0x5601f9b54de0 2024-03-21T11:29:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.448+0000 7f1ca500b740 1 -- 172.21.15.47:0/2348930039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c94004d20 con 0x5601f9b54de0 2024-03-21T11:29:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.448+0000 7f1c85600640 1 -- 172.21.15.47:0/2348930039 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1c880053d0 con 0x5601f9b54de0 2024-03-21T11:29:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.448+0000 7f1c85600640 1 -- 172.21.15.47:0/2348930039 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c880055d0 con 0x5601f9b54de0 2024-03-21T11:29:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.448+0000 7f1ca500b740 10 client.27682 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.449+0000 7f1ca500b740 1 -- 172.21.15.47:0/2348930039 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5601f989a620 con 0x5601f9b54de0 2024-03-21T11:29:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.449+0000 7f1c85600640 1 -- 172.21.15.47:0/2348930039 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c94004d20 con 0x5601f9b54de0 2024-03-21T11:29:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.449+0000 7f1c85600640 1 -- 172.21.15.47:0/2348930039 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1c88003bc0 con 0x5601f9b54de0 2024-03-21T11:29:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.449+0000 7f1ca500b740 10 client.27682 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.449+0000 7f1ca500b740 10 client.27682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.449+0000 7f1ca500b740 20 client.27682 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.449+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2348930039 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601f9b6a4d0 0x5601f9b6c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.449+0000 7f1ca500b740 4 client.27682 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:29:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.449+0000 7f1ca500b740 1 -- 172.21.15.47:0/2348930039 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601f9ac6d70 con 0x5601f9b6a4d0 2024-03-21T11:29:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.449+0000 7f1c9be00640 1 --2- 172.21.15.47:0/2348930039 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601f9b6a4d0 0x5601f9b6c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.450+0000 7f1c9be00640 1 --2- 172.21.15.47:0/2348930039 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601f9b6a4d0 0x5601f9b6c8c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f9451a706a9b98f server_cookie=ac10d8765ba5f371 in_seq=0 out_seq=0 2024-03-21T11:29:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.450+0000 7f1c85600640 10 client.27682 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.451+0000 7f1c85600640 1 -- 172.21.15.47:0/2348930039 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5601f9ac6d70 con 0x5601f9b6a4d0 2024-03-21T11:29:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.451+0000 7f1c85600640 10 client.27682 handle_command_reply: tid=0 2024-03-21T11:29:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.451+0000 7f1ca500b740 1 client.27682 shutdown 2024-03-21T11:29:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.451+0000 7f1ca500b740 1 -- 172.21.15.47:0/2348930039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b54de0 msgr2=0x5601f9b5eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.451+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2348930039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b54de0 0x5601f9b5eaf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009230 tx=0x7f1c88004270 comp rx=0 tx=0).stop 2024-03-21T11:29:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.451+0000 7f1ca500b740 1 -- 172.21.15.47:0/2348930039 shutdown_connections 2024-03-21T11:29:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.451+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2348930039 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601f9b6a4d0 0x5601f9b6c8c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.451+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2348930039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b59f50 0x5601f9b65b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.451+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2348930039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b54de0 0x5601f9b5eaf0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.451+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2348930039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b5e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.451+0000 7f1c85600640 0 client.27682 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.451+0000 7f1ca500b740 1 -- 172.21.15.47:0/2348930039 >> 172.21.15.47:0/2348930039 conn(0x5601f9b4f960 msgr2=0x5601f9b4eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.452+0000 7f1ca500b740 1 -- 172.21.15.47:0/2348930039 shutdown_connections 2024-03-21T11:29:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.452+0000 7f1ca500b740 1 -- 172.21.15.47:0/2348930039 wait complete. 2024-03-21T11:29:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.452+0000 7f1ca500b740 20 client.27682 trim_cache size 0 max 16384 2024-03-21T11:29:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.471+0000 7f1ca500b740 1 Processor -- start 2024-03-21T11:29:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.471+0000 7f1ca500b740 1 -- start start 2024-03-21T11:29:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.471+0000 7f1ca500b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b56100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.471+0000 7f1ca500b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b54de0 0x5601f9b65af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.471+0000 7f1ca500b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b59f50 0x5601f9b56640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.471+0000 7f1ca500b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c1ac130 con 0x5601f9b54de0 2024-03-21T11:29:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.471+0000 7f1ca500b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c1adab0 con 0x5601f9b543f0 2024-03-21T11:29:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.471+0000 7f1ca500b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c1ac790 con 0x5601f9b59f50 2024-03-21T11:29:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.471+0000 7f1c9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b56100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.472+0000 7f1c9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b56100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53560/0 (socket says 172.21.15.47:53560) 2024-03-21T11:29:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.472+0000 7f1c9be00640 1 -- 172.21.15.47:0/1380723996 learned_addr learned my addr 172.21.15.47:0/1380723996 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.472+0000 7f1c9aa00640 1 --2- 172.21.15.47:0/1380723996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b54de0 0x5601f9b65af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.472+0000 7f1c9be00640 1 --2- 172.21.15.47:0/1380723996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b59f50 0x5601f9b56640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.472+0000 7f1c9be00640 1 -- 172.21.15.47:0/1380723996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b59f50 msgr2=0x5601f9b56640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.472+0000 7f1c9be00640 1 --2- 172.21.15.47:0/1380723996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b59f50 0x5601f9b56640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.472+0000 7f1c9be00640 1 -- 172.21.15.47:0/1380723996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b54de0 msgr2=0x5601f9b65af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.472+0000 7f1c9be00640 1 --2- 172.21.15.47:0/1380723996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b54de0 0x5601f9b65af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.472+0000 7f1c9be00640 1 -- 172.21.15.47:0/1380723996 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601f9b55bf0 con 0x5601f9b543f0 2024-03-21T11:29:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.472+0000 7f1c9be00640 1 --2- 172.21.15.47:0/1380723996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b56100 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009660 tx=0x7f1c9401c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b45c755b72d0ee6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.472+0000 7f1c87e00640 1 -- 172.21.15.47:0/1380723996 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94016070 con 0x5601f9b543f0 2024-03-21T11:29:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.473+0000 7f1c87e00640 1 -- 172.21.15.47:0/1380723996 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c94005200 con 0x5601f9b543f0 2024-03-21T11:29:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.473+0000 7f1c87e00640 1 -- 172.21.15.47:0/1380723996 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94012980 con 0x5601f9b543f0 2024-03-21T11:29:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.473+0000 7f1ca500b740 1 -- 172.21.15.47:0/1380723996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 msgr2=0x5601f9b56100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.473+0000 7f1ca500b740 1 --2- 172.21.15.47:0/1380723996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b56100 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009660 tx=0x7f1c9401c040 comp rx=0 tx=0).stop 2024-03-21T11:29:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.474+0000 7f1ca500b740 1 -- 172.21.15.47:0/1380723996 shutdown_connections 2024-03-21T11:29:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.474+0000 7f1ca500b740 1 --2- 172.21.15.47:0/1380723996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b59f50 0x5601f9b56640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.474+0000 7f1ca500b740 1 --2- 172.21.15.47:0/1380723996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b54de0 0x5601f9b65af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.474+0000 7f1ca500b740 1 --2- 172.21.15.47:0/1380723996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b543f0 0x5601f9b56100 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.474+0000 7f1ca500b740 1 -- 172.21.15.47:0/1380723996 >> 172.21.15.47:0/1380723996 conn(0x5601f9b4f960 msgr2=0x5601f9b4fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.474+0000 7f1ca500b740 1 -- 172.21.15.47:0/1380723996 shutdown_connections 2024-03-21T11:29:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.474+0000 7f1ca500b740 1 -- 172.21.15.47:0/1380723996 wait complete. 2024-03-21T11:29:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.475+0000 7f1ca500b740 1 -- start start 2024-03-21T11:29:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.475+0000 7f1ca500b740 1 Processor -- start 2024-03-21T11:29:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.475+0000 7f1ca500b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b543f0 0x5601f9b657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.475+0000 7f1ca500b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b54de0 0x5601f9b63ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.475+0000 7f1ca500b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b59f50 0x5601f9b643e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.476+0000 7f1ca500b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c1ad450 con 0x5601f9b59f50 2024-03-21T11:29:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.476+0000 7f1ca500b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c1acdf0 con 0x5601f9b54de0 2024-03-21T11:29:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.476+0000 7f1ca500b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c1aa2f0 con 0x5601f9b543f0 2024-03-21T11:29:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.476+0000 7f1ca500b740 10 client.0 _ll_register_callbacks cb 0x7f1c88009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:29:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.476+0000 7f1c9aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b54de0 0x5601f9b63ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.476+0000 7f1c9aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b54de0 0x5601f9b63ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53566/0 (socket says 172.21.15.47:53566) 2024-03-21T11:29:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.476+0000 7f1c9aa00640 1 -- 172.21.15.47:0/2399107259 learned_addr learned my addr 172.21.15.47:0/2399107259 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:29:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.476+0000 7f1c9be00640 1 --2- 172.21.15.47:0/2399107259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b543f0 0x5601f9b657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.476+0000 7f1c9be00640 1 --2- 172.21.15.47:0/2399107259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b59f50 0x5601f9b643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.477+0000 7f1c9aa00640 1 -- 172.21.15.47:0/2399107259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b543f0 msgr2=0x5601f9b657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.477+0000 7f1c9aa00640 1 --2- 172.21.15.47:0/2399107259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b543f0 0x5601f9b657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.477+0000 7f1c9aa00640 1 -- 172.21.15.47:0/2399107259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b59f50 msgr2=0x5601f9b643e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:29:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.477+0000 7f1c9aa00640 1 --2- 172.21.15.47:0/2399107259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b59f50 0x5601f9b643e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.477+0000 7f1c9aa00640 1 -- 172.21.15.47:0/2399107259 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601f9b6cc30 con 0x5601f9b54de0 2024-03-21T11:29:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.477+0000 7f1c9be00640 1 --2- 172.21.15.47:0/2399107259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b59f50 0x5601f9b643e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:29:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.477+0000 7f1c9aa00640 1 --2- 172.21.15.47:0/2399107259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b54de0 0x5601f9b63ea0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802f020 tx=0x7f1c8802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95aab955a032b847 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:29:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.477+0000 7f1c85600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:29:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.478+0000 7f1c85600640 1 -- 172.21.15.47:0/2399107259 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88058060 con 0x5601f9b54de0 2024-03-21T11:29:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.478+0000 7f1ca500b740 1 -- 172.21.15.47:0/2399107259 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c94005200 con 0x5601f9b54de0 2024-03-21T11:29:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.478+0000 7f1c85600640 1 -- 172.21.15.47:0/2399107259 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1c88005370 con 0x5601f9b54de0 2024-03-21T11:29:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.478+0000 7f1c85600640 1 -- 172.21.15.47:0/2399107259 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c8805f050 con 0x5601f9b54de0 2024-03-21T11:29:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.478+0000 7f1ca500b740 10 client.27668 fetch_fsmap learned FSMap version 28 2024-03-21T11:29:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.478+0000 7f1ca500b740 1 -- 172.21.15.47:0/2399107259 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5601f9bcf0f0 con 0x5601f9b54de0 2024-03-21T11:29:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.479+0000 7f1c85600640 1 -- 172.21.15.47:0/2399107259 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c94005200 con 0x5601f9b54de0 2024-03-21T11:29:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.480+0000 7f1c85600640 1 -- 172.21.15.47:0/2399107259 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1c880035f0 con 0x5601f9b54de0 2024-03-21T11:29:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.480+0000 7f1ca500b740 10 client.27668 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:29:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.480+0000 7f1ca500b740 10 client.27668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:29:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.480+0000 7f1ca500b740 20 client.27668 populate_metadata read hostname 'smithi047' 2024-03-21T11:29:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.480+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2399107259 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601f9bcfa20 0x5601f9bd1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:29:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.480+0000 7f1ca500b740 4 client.27668 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:29:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.480+0000 7f1ca500b740 1 -- 172.21.15.47:0/2399107259 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5601f9ac10c0 con 0x5601f9bcfa20 2024-03-21T11:29:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.481+0000 7f1c9be00640 1 --2- 172.21.15.47:0/2399107259 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601f9bcfa20 0x5601f9bd1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:29:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.482+0000 7f1c9be00640 1 --2- 172.21.15.47:0/2399107259 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601f9bcfa20 0x5601f9bd1e40 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f9c0400f9281a8d server_cookie=1d4ed8a251a62630 in_seq=0 out_seq=0 2024-03-21T11:29:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.482+0000 7f1c85600640 10 client.27668 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:50.522 INFO:tasks.workunit.client.0.smithi047.stdout: 1 430648 8.24 MB/sec execute 154 sec latency 24.636 ms 2024-03-21T11:29:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.692+0000 7f1c85600640 1 -- 172.21.15.47:0/2399107259 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5601f9ac10c0 con 0x5601f9bcfa20 2024-03-21T11:29:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.693+0000 7f1c85600640 10 client.27668 handle_command_reply: tid=0 2024-03-21T11:29:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.693+0000 7f1ca500b740 1 client.27668 shutdown 2024-03-21T11:29:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.693+0000 7f1ca500b740 1 -- 172.21.15.47:0/2399107259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b54de0 msgr2=0x5601f9b63ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.693+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2399107259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b54de0 0x5601f9b63ea0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802f020 tx=0x7f1c8802bc30 comp rx=0 tx=0).stop 2024-03-21T11:29:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.693+0000 7f1ca500b740 1 -- 172.21.15.47:0/2399107259 shutdown_connections 2024-03-21T11:29:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.693+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2399107259 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5601f9bcfa20 0x5601f9bd1e40 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.693+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2399107259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601f9b59f50 0x5601f9b643e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.693+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2399107259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601f9b54de0 0x5601f9b63ea0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.693+0000 7f1c85600640 0 client.27668 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:29:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.693+0000 7f1ca500b740 1 --2- 172.21.15.47:0/2399107259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601f9b543f0 0x5601f9b657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.693+0000 7f1ca500b740 1 -- 172.21.15.47:0/2399107259 >> 172.21.15.47:0/2399107259 conn(0x5601f9b4f960 msgr2=0x5601f9b4fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.693+0000 7f1ca500b740 1 -- 172.21.15.47:0/2399107259 shutdown_connections 2024-03-21T11:29:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.693+0000 7f1ca500b740 1 -- 172.21.15.47:0/2399107259 wait complete. 2024-03-21T11:29:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.693+0000 7f1ca500b740 20 client.27668 trim_cache size 0 max 16384 2024-03-21T11:29:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:29:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:29:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:29:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:29:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.694+0000 7f1ca0e00640 1 -- 172.21.15.47:0/2135204330 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1c800a2f90 msgr2=0x7f1c800a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.694+0000 7f1ca0e00640 1 --2- 172.21.15.47:0/2135204330 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1c800a2f90 0x7f1c800a5450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0989c0 tx=0x7f1c8c0d36f0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.694+0000 7f1ca0e00640 1 -- 172.21.15.47:0/2135204330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c9c103720 msgr2=0x7f1c9c199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:29:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.694+0000 7f1ca0e00640 1 --2- 172.21.15.47:0/2135204330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c9c103720 0x7f1c9c199a10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00e9c0 tx=0x7f1c8c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:29:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.694+0000 7f1ca0e00640 1 -- 172.21.15.47:0/2135204330 shutdown_connections 2024-03-21T11:29:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.694+0000 7f1ca0e00640 1 --2- 172.21.15.47:0/2135204330 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1c800a2f90 0x7f1c800a5450 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.694+0000 7f1ca0e00640 1 --2- 172.21.15.47:0/2135204330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c9c1052d0 0x7f1c9c1a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.694+0000 7f1ca0e00640 1 --2- 172.21.15.47:0/2135204330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c9c104910 0x7f1c9c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.694+0000 7f1ca0e00640 1 --2- 172.21.15.47:0/2135204330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c9c103720 0x7f1c9c199a10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:29:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.694+0000 7f1ca0e00640 1 -- 172.21.15.47:0/2135204330 >> 172.21.15.47:0/2135204330 conn(0x7f1c9c0fecb0 msgr2=0x7f1c9c107ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:29:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.695+0000 7f1ca0e00640 1 -- 172.21.15.47:0/2135204330 shutdown_connections 2024-03-21T11:29:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:29:50.695+0000 7f1ca0e00640 1 -- 172.21.15.47:0/2135204330 wait complete. 2024-03-21T11:29:51.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:50 smithi047 ceph-mon[31351]: pgmap v595: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 290 op/s 2024-03-21T11:29:51.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:50 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi100 to 174.1M 2024-03-21T11:29:51.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:50 smithi047 ceph-mon[31351]: Unable to set osd_memory_target on smithi100 to 182575411: error parsing value: Value '182575411' is below minimum 939524096 2024-03-21T11:29:51.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:29:51.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:29:51.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:29:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:50 smithi100 ceph-mon[36245]: pgmap v595: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 290 op/s 2024-03-21T11:29:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:50 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi100 to 174.1M 2024-03-21T11:29:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:50 smithi100 ceph-mon[36245]: Unable to set osd_memory_target on smithi100 to 182575411: error parsing value: Value '182575411' is below minimum 939524096 2024-03-21T11:29:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:29:51.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:29:51.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:29:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:50 smithi102 ceph-mon[36748]: pgmap v595: 129 pgs: 129 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 290 op/s 2024-03-21T11:29:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:50 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi100 to 174.1M 2024-03-21T11:29:51.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:50 smithi102 ceph-mon[36748]: Unable to set osd_memory_target on smithi100 to 182575411: error parsing value: Value '182575411' is below minimum 939524096 2024-03-21T11:29:51.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:29:51.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:29:51.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:29:51.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:29:51.522 INFO:tasks.workunit.client.0.smithi047.stdout: 1 432443 8.24 MB/sec execute 155 sec latency 111.204 ms 2024-03-21T11:29:52.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:51 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi047 to 177.8M 2024-03-21T11:29:52.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:51 smithi047 ceph-mon[31351]: Unable to set osd_memory_target on smithi047 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T11:29:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:51 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi047 to 177.8M 2024-03-21T11:29:52.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:51 smithi100 ceph-mon[36245]: Unable to set osd_memory_target on smithi047 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T11:29:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:51 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi047 to 177.8M 2024-03-21T11:29:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:51 smithi102 ceph-mon[36748]: Unable to set osd_memory_target on smithi047 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T11:29:52.523 INFO:tasks.workunit.client.0.smithi047.stdout: 1 433482 8.22 MB/sec execute 156 sec latency 12.493 ms 2024-03-21T11:29:53.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:52 smithi047 ceph-mon[31351]: pgmap v596: 129 pgs: 129 active+clean; 38 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-03-21T11:29:53.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:52 smithi100 ceph-mon[36245]: pgmap v596: 129 pgs: 129 active+clean; 38 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-03-21T11:29:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:52 smithi102 ceph-mon[36748]: pgmap v596: 129 pgs: 129 active+clean; 38 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-03-21T11:29:53.523 INFO:tasks.workunit.client.0.smithi047.stdout: 1 435207 8.22 MB/sec execute 157 sec latency 22.793 ms 2024-03-21T11:29:54.523 INFO:tasks.workunit.client.0.smithi047.stdout: 1 436589 8.24 MB/sec execute 158 sec latency 47.201 ms 2024-03-21T11:29:55.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:54 smithi047 ceph-mon[31351]: pgmap v597: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 304 op/s 2024-03-21T11:29:55.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:54 smithi100 ceph-mon[36245]: pgmap v597: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 304 op/s 2024-03-21T11:29:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:54 smithi102 ceph-mon[36748]: pgmap v597: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 304 op/s 2024-03-21T11:29:55.523 INFO:tasks.workunit.client.0.smithi047.stdout: 1 437421 8.20 MB/sec execute 159 sec latency 80.577 ms 2024-03-21T11:29:56.523 INFO:tasks.workunit.client.0.smithi047.stdout: 1 439293 8.21 MB/sec execute 160 sec latency 14.272 ms 2024-03-21T11:29:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:56 smithi047 ceph-mon[31351]: pgmap v598: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1.8 MiB/s wr, 234 op/s 2024-03-21T11:29:57.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:56 smithi100 ceph-mon[36245]: pgmap v598: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1.8 MiB/s wr, 234 op/s 2024-03-21T11:29:57.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:56 smithi102 ceph-mon[36748]: pgmap v598: 129 pgs: 129 active+clean; 36 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1.8 MiB/s wr, 234 op/s 2024-03-21T11:29:57.524 INFO:tasks.workunit.client.0.smithi047.stdout: 1 440409 8.20 MB/sec execute 161 sec latency 20.633 ms 2024-03-21T11:29:58.524 INFO:tasks.workunit.client.0.smithi047.stdout: 1 441725 8.19 MB/sec execute 162 sec latency 20.671 ms 2024-03-21T11:29:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:29:58 smithi047 ceph-mon[31351]: pgmap v599: 129 pgs: 129 active+clean; 38 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2024-03-21T11:29:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:29:58 smithi100 ceph-mon[36245]: pgmap v599: 129 pgs: 129 active+clean; 38 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2024-03-21T11:29:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:29:58 smithi102 ceph-mon[36748]: pgmap v599: 129 pgs: 129 active+clean; 38 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2024-03-21T11:29:59.524 INFO:tasks.workunit.client.0.smithi047.stdout: 1 443479 8.21 MB/sec execute 163 sec latency 40.975 ms 2024-03-21T11:30:00.524 INFO:tasks.workunit.client.0.smithi047.stdout: 1 444354 8.18 MB/sec execute 164 sec latency 14.679 ms 2024-03-21T11:30:01.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:00 smithi047 ceph-mon[31351]: pgmap v600: 129 pgs: 129 active+clean; 37 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 282 op/s 2024-03-21T11:30:01.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:00 smithi047 ceph-mon[31351]: overall HEALTH_OK 2024-03-21T11:30:01.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:00 smithi100 ceph-mon[36245]: pgmap v600: 129 pgs: 129 active+clean; 37 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 282 op/s 2024-03-21T11:30:01.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:00 smithi100 ceph-mon[36245]: overall HEALTH_OK 2024-03-21T11:30:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:00 smithi102 ceph-mon[36748]: pgmap v600: 129 pgs: 129 active+clean; 37 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 282 op/s 2024-03-21T11:30:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:00 smithi102 ceph-mon[36748]: overall HEALTH_OK 2024-03-21T11:30:01.524 INFO:tasks.workunit.client.0.smithi047.stdout: 1 446425 8.20 MB/sec execute 165 sec latency 10.577 ms 2024-03-21T11:30:02.525 INFO:tasks.workunit.client.0.smithi047.stdout: 1 447623 8.19 MB/sec execute 166 sec latency 25.186 ms 2024-03-21T11:30:02.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:30:02.796 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:30:03.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.029+0000 7fc144600640 1 -- 172.21.15.47:0/3569395062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc13c076060 msgr2=0x7fc13c076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.029+0000 7fc144600640 1 --2- 172.21.15.47:0/3569395062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc13c076060 0x7fc13c076480 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc12c009ca0 tx=0x7fc12c02c0f0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.029+0000 7fc144600640 1 -- 172.21.15.47:0/3569395062 shutdown_connections 2024-03-21T11:30:03.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.029+0000 7fc144600640 1 --2- 172.21.15.47:0/3569395062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc13c077c40 0x7fc13c0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.029+0000 7fc144600640 1 --2- 172.21.15.47:0/3569395062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c0772a0 0x7fc13c077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.029+0000 7fc144600640 1 --2- 172.21.15.47:0/3569395062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc13c076060 0x7fc13c076480 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.029+0000 7fc144600640 1 -- 172.21.15.47:0/3569395062 >> 172.21.15.47:0/3569395062 conn(0x7fc13c0fee50 msgr2=0x7fc13c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.029+0000 7fc144600640 1 -- 172.21.15.47:0/3569395062 shutdown_connections 2024-03-21T11:30:03.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.029+0000 7fc144600640 1 -- 172.21.15.47:0/3569395062 wait complete. 2024-03-21T11:30:03.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.030+0000 7fc144600640 1 Processor -- start 2024-03-21T11:30:03.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.030+0000 7fc144600640 1 -- start start 2024-03-21T11:30:03.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.030+0000 7fc144600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc13c076060 0x7fc13c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.030+0000 7fc144600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c0772a0 0x7fc13c19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.030+0000 7fc144600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc13c077c40 0x7fc13c1a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.030+0000 7fc144600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c113970 con 0x7fc13c077c40 2024-03-21T11:30:03.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.030+0000 7fc144600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1137f0 con 0x7fc13c0772a0 2024-03-21T11:30:03.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.030+0000 7fc144600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c113af0 con 0x7fc13c076060 2024-03-21T11:30:03.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.031+0000 7fc141200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c0772a0 0x7fc13c19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.031+0000 7fc141c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc13c076060 0x7fc13c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.031+0000 7fc141200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c0772a0 0x7fc13c19e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51910/0 (socket says 172.21.15.47:51910) 2024-03-21T11:30:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.031+0000 7fc141200640 1 -- 172.21.15.47:0/3381499368 learned_addr learned my addr 172.21.15.47:0/3381499368 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.031+0000 7fc141c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc13c076060 0x7fc13c19dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59652/0 (socket says 172.21.15.47:59652) 2024-03-21T11:30:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.031+0000 7fc142600640 1 --2- 172.21.15.47:0/3381499368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc13c077c40 0x7fc13c1a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.031+0000 7fc141c00640 1 -- 172.21.15.47:0/3381499368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c0772a0 msgr2=0x7fc13c19e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.031+0000 7fc141c00640 1 --2- 172.21.15.47:0/3381499368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c0772a0 0x7fc13c19e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.031+0000 7fc141c00640 1 -- 172.21.15.47:0/3381499368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc13c077c40 msgr2=0x7fc13c1a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.031+0000 7fc141c00640 1 --2- 172.21.15.47:0/3381499368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc13c077c40 0x7fc13c1a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.031+0000 7fc141c00640 1 -- 172.21.15.47:0/3381499368 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c1a5bf0 con 0x7fc13c076060 2024-03-21T11:30:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.031+0000 7fc141200640 1 --2- 172.21.15.47:0/3381499368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c0772a0 0x7fc13c19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.031+0000 7fc141c00640 1 --2- 172.21.15.47:0/3381499368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc13c076060 0x7fc13c19dfd0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc12c02c630 tx=0x7fc12c032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.032+0000 7fc13a000640 1 -- 172.21.15.47:0/3381499368 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c041020 con 0x7fc13c076060 2024-03-21T11:30:03.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.032+0000 7fc13a000640 1 -- 172.21.15.47:0/3381499368 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc12c034cd0 con 0x7fc13c076060 2024-03-21T11:30:03.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.032+0000 7fc144600640 1 -- 172.21.15.47:0/3381499368 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13c1a5e80 con 0x7fc13c076060 2024-03-21T11:30:03.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.032+0000 7fc13a000640 1 -- 172.21.15.47:0/3381499368 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c039040 con 0x7fc13c076060 2024-03-21T11:30:03.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.032+0000 7fc144600640 1 -- 172.21.15.47:0/3381499368 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13c1a62e0 con 0x7fc13c076060 2024-03-21T11:30:03.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.036+0000 7fc147378740 1 Processor -- start 2024-03-21T11:30:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.036+0000 7fc147378740 1 -- start start 2024-03-21T11:30:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.037+0000 7fc147378740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66ae80 0x55c58d66b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.037+0000 7fc147378740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d66b870 0x55c58d670370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.037+0000 7fc147378740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d6709e0 0x55c58d672e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.037+0000 7fc147378740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58d5d0250 con 0x55c58d6709e0 2024-03-21T11:30:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.037+0000 7fc147378740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58d578a30 con 0x55c58d66b870 2024-03-21T11:30:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.037+0000 7fc147378740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58d5cc2f0 con 0x55c58d66ae80 2024-03-21T11:30:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.037+0000 7fc142600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66ae80 0x55c58d66b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.037+0000 7fc13a000640 1 -- 172.21.15.47:0/3381499368 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc12c044020 con 0x7fc13c076060 2024-03-21T11:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.037+0000 7fc142600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66ae80 0x55c58d66b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59658/0 (socket says 172.21.15.47:59658) 2024-03-21T11:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.037+0000 7fc142600640 1 -- 172.21.15.47:0/2195143179 learned_addr learned my addr 172.21.15.47:0/2195143179 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.037+0000 7fc141200640 1 --2- 172.21.15.47:0/2195143179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d6709e0 0x55c58d672e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.038+0000 7fc142600640 1 -- 172.21.15.47:0/2195143179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d66b870 msgr2=0x55c58d670370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.038+0000 7fc141c00640 1 --2- 172.21.15.47:0/2195143179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d66b870 0x55c58d670370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.038+0000 7fc142600640 1 --2- 172.21.15.47:0/2195143179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d66b870 0x55c58d670370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.038+0000 7fc142600640 1 -- 172.21.15.47:0/2195143179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d6709e0 msgr2=0x55c58d672e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.038+0000 7fc142600640 1 --2- 172.21.15.47:0/2195143179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d6709e0 0x55c58d672e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.038+0000 7fc142600640 1 -- 172.21.15.47:0/2195143179 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c58d563690 con 0x55c58d66ae80 2024-03-21T11:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.038+0000 7fc141c00640 1 --2- 172.21.15.47:0/2195143179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d66b870 0x55c58d670370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.038+0000 7fc141200640 1 --2- 172.21.15.47:0/2195143179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d6709e0 0x55c58d672e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.038+0000 7fc142600640 1 --2- 172.21.15.47:0/2195143179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66ae80 0x55c58d66b2a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc134009ca0 tx=0x7fc134011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecb002b0fc451f15 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.038+0000 7fc132a00640 1 -- 172.21.15.47:0/2195143179 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13401e070 con 0x55c58d66ae80 2024-03-21T11:30:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.038+0000 7fc132a00640 1 -- 172.21.15.47:0/2195143179 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc134009e50 con 0x55c58d66ae80 2024-03-21T11:30:03.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.038+0000 7fc132a00640 1 -- 172.21.15.47:0/2195143179 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13400eac0 con 0x55c58d66ae80 2024-03-21T11:30:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.039+0000 7fc13a000640 1 --2- 172.21.15.47:0/3381499368 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc1200a2fa0 0x7fc1200a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.039+0000 7fc147378740 1 -- 172.21.15.47:0/2195143179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66ae80 msgr2=0x55c58d66b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.039+0000 7fc147378740 1 --2- 172.21.15.47:0/2195143179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66ae80 0x55c58d66b2a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc134009ca0 tx=0x7fc134011810 comp rx=0 tx=0).stop 2024-03-21T11:30:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.039+0000 7fc13a000640 1 -- 172.21.15.47:0/3381499368 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc12c0f14a0 con 0x7fc13c076060 2024-03-21T11:30:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.039+0000 7fc147378740 1 -- 172.21.15.47:0/2195143179 shutdown_connections 2024-03-21T11:30:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.039+0000 7fc147378740 1 --2- 172.21.15.47:0/2195143179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d6709e0 0x55c58d672e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.039+0000 7fc147378740 1 --2- 172.21.15.47:0/2195143179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d66b870 0x55c58d670370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.039+0000 7fc147378740 1 --2- 172.21.15.47:0/2195143179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66ae80 0x55c58d66b2a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.039+0000 7fc147378740 1 -- 172.21.15.47:0/2195143179 >> 172.21.15.47:0/2195143179 conn(0x55c58d633200 msgr2=0x55c58d6667f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.039+0000 7fc141c00640 1 --2- 172.21.15.47:0/3381499368 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc1200a2fa0 0x7fc1200a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.039+0000 7fc147378740 1 -- 172.21.15.47:0/2195143179 shutdown_connections 2024-03-21T11:30:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.040+0000 7fc147378740 1 -- 172.21.15.47:0/2195143179 wait complete. 2024-03-21T11:30:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.040+0000 7fc147378740 1 -- start start 2024-03-21T11:30:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.040+0000 7fc141c00640 1 --2- 172.21.15.47:0/3381499368 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc1200a2fa0 0x7fc1200a5460 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc12c006010 tx=0x7fc12c02c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.040+0000 7fc147378740 1 Processor -- start 2024-03-21T11:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.041+0000 7fc147378740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d66ae80 0x55c58d674f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.041+0000 7fc147378740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66b870 0x55c58d675480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.041+0000 7fc147378740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d6709e0 0x55c58d67c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.041+0000 7fc147378740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58d673990 con 0x55c58d66ae80 2024-03-21T11:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.041+0000 7fc147378740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58d58be80 con 0x55c58d6709e0 2024-03-21T11:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.041+0000 7fc147378740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58d673b10 con 0x55c58d66b870 2024-03-21T11:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.041+0000 7fc147378740 10 client.0 _ll_register_callbacks cb 0x55c58d59e130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.041+0000 7fc141200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66b870 0x55c58d675480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.041+0000 7fc142600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d66ae80 0x55c58d674f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.041+0000 7fc141200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66b870 0x55c58d675480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59664/0 (socket says 172.21.15.47:59664) 2024-03-21T11:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.041+0000 7fc141200640 1 -- 172.21.15.47:0/1010662109 learned_addr learned my addr 172.21.15.47:0/1010662109 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.041+0000 7fc142600640 1 --2- 172.21.15.47:0/1010662109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d6709e0 0x55c58d67c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.042+0000 7fc141200640 1 -- 172.21.15.47:0/1010662109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d6709e0 msgr2=0x55c58d67c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.042+0000 7fc141200640 1 --2- 172.21.15.47:0/1010662109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d6709e0 0x55c58d67c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.042+0000 7fc141200640 1 -- 172.21.15.47:0/1010662109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d66ae80 msgr2=0x55c58d674f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.042+0000 7fc141200640 1 --2- 172.21.15.47:0/1010662109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d66ae80 0x55c58d674f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.042+0000 7fc141200640 1 -- 172.21.15.47:0/1010662109 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c58d3ae7d0 con 0x55c58d66b870 2024-03-21T11:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.042+0000 7fc141200640 1 --2- 172.21.15.47:0/1010662109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66b870 0x55c58d675480 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc128012d20 tx=0x7fc12800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a07e498a8425d35a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.042+0000 7fc127400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.042+0000 7fc127400640 1 -- 172.21.15.47:0/1010662109 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1280120d0 con 0x55c58d66b870 2024-03-21T11:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.042+0000 7fc127400640 1 -- 172.21.15.47:0/1010662109 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc128010430 con 0x55c58d66b870 2024-03-21T11:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.043+0000 7fc147378740 1 -- 172.21.15.47:0/1010662109 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc134009e50 con 0x55c58d66b870 2024-03-21T11:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.043+0000 7fc127400640 1 -- 172.21.15.47:0/1010662109 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc128012270 con 0x55c58d66b870 2024-03-21T11:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.043+0000 7fc147378740 10 client.27700 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.043+0000 7fc147378740 1 -- 172.21.15.47:0/1010662109 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c58d2e43f0 con 0x55c58d66b870 2024-03-21T11:30:03.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.043+0000 7fc127400640 1 -- 172.21.15.47:0/1010662109 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc134009e50 con 0x55c58d66b870 2024-03-21T11:30:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.043+0000 7fc127400640 1 -- 172.21.15.47:0/1010662109 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc12801c780 con 0x55c58d66b870 2024-03-21T11:30:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.043+0000 7fc147378740 10 client.27700 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.044+0000 7fc147378740 10 client.27700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.044+0000 7fc147378740 20 client.27700 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.044+0000 7fc147378740 1 --2- 172.21.15.47:0/1010662109 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c58d680ce0 0x55c58d683140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.044+0000 7fc147378740 4 client.27700 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.044+0000 7fc147378740 1 -- 172.21.15.47:0/1010662109 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c58d5dd790 con 0x55c58d680ce0 2024-03-21T11:30:03.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.044+0000 7fc142600640 1 --2- 172.21.15.47:0/1010662109 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c58d680ce0 0x55c58d683140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.044+0000 7fc142600640 1 --2- 172.21.15.47:0/1010662109 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c58d680ce0 0x55c58d683140 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20a096e21ba1e818 server_cookie=240d56f1bf595e96 in_seq=0 out_seq=0 2024-03-21T11:30:03.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.044+0000 7fc127400640 10 client.27700 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:03.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.046+0000 7fc127400640 1 -- 172.21.15.47:0/1010662109 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c58d5dd790 con 0x55c58d680ce0 2024-03-21T11:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.046+0000 7fc127400640 10 client.27700 handle_command_reply: tid=0 2024-03-21T11:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.046+0000 7fc147378740 1 client.27700 shutdown 2024-03-21T11:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.046+0000 7fc147378740 1 -- 172.21.15.47:0/1010662109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66b870 msgr2=0x55c58d675480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.046+0000 7fc147378740 1 --2- 172.21.15.47:0/1010662109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66b870 0x55c58d675480 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc128012d20 tx=0x7fc12800f790 comp rx=0 tx=0).stop 2024-03-21T11:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.046+0000 7fc147378740 1 -- 172.21.15.47:0/1010662109 shutdown_connections 2024-03-21T11:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.046+0000 7fc147378740 1 --2- 172.21.15.47:0/1010662109 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c58d680ce0 0x55c58d683140 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.046+0000 7fc127400640 0 client.27700 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.046+0000 7fc147378740 1 --2- 172.21.15.47:0/1010662109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d6709e0 0x55c58d67c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.046+0000 7fc147378740 1 --2- 172.21.15.47:0/1010662109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66b870 0x55c58d675480 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.046+0000 7fc147378740 1 --2- 172.21.15.47:0/1010662109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d66ae80 0x55c58d674f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.046+0000 7fc147378740 1 -- 172.21.15.47:0/1010662109 >> 172.21.15.47:0/1010662109 conn(0x55c58d633200 msgr2=0x55c58d665a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.047+0000 7fc147378740 1 -- 172.21.15.47:0/1010662109 shutdown_connections 2024-03-21T11:30:03.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.047+0000 7fc147378740 1 -- 172.21.15.47:0/1010662109 wait complete. 2024-03-21T11:30:03.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.047+0000 7fc147378740 20 client.27700 trim_cache size 0 max 16384 2024-03-21T11:30:03.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:02 smithi047 ceph-mon[31351]: pgmap v601: 129 pgs: 129 active+clean; 38 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.7 MiB/s wr, 240 op/s 2024-03-21T11:30:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.065+0000 7fc147378740 1 Processor -- start 2024-03-21T11:30:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.065+0000 7fc147378740 1 -- start start 2024-03-21T11:30:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.065+0000 7fc147378740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d66ae80 0x55c58d6745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.065+0000 7fc147378740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66b870 0x55c58d67c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.065+0000 7fc147378740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d6709e0 0x55c58d66c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.065+0000 7fc147378740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1b06e0 con 0x55c58d66ae80 2024-03-21T11:30:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.065+0000 7fc147378740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1b2060 con 0x55c58d6709e0 2024-03-21T11:30:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.065+0000 7fc147378740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1b0d40 con 0x55c58d66b870 2024-03-21T11:30:03.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.066+0000 7fc142600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d6709e0 0x55c58d66c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.066+0000 7fc142600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d6709e0 0x55c58d66c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51932/0 (socket says 172.21.15.47:51932) 2024-03-21T11:30:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.066+0000 7fc142600640 1 -- 172.21.15.47:0/2029809222 learned_addr learned my addr 172.21.15.47:0/2029809222 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.066+0000 7fc141200640 1 --2- 172.21.15.47:0/2029809222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66b870 0x55c58d67c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.066+0000 7fc142600640 1 --2- 172.21.15.47:0/2029809222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d66ae80 0x55c58d6745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.066+0000 7fc141200640 1 -- 172.21.15.47:0/2029809222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d6709e0 msgr2=0x55c58d66c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.066+0000 7fc141200640 1 --2- 172.21.15.47:0/2029809222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d6709e0 0x55c58d66c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.066+0000 7fc141200640 1 -- 172.21.15.47:0/2029809222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d66ae80 msgr2=0x55c58d6745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.066+0000 7fc141200640 1 --2- 172.21.15.47:0/2029809222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d66ae80 0x55c58d6745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.066+0000 7fc141200640 1 -- 172.21.15.47:0/2029809222 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c58d2a40f0 con 0x55c58d66b870 2024-03-21T11:30:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.066+0000 7fc141200640 1 --2- 172.21.15.47:0/2029809222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66b870 0x55c58d67c410 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc12801cc40 tx=0x7fc128012990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc808a78c40dad1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.067+0000 7fc132000640 1 -- 172.21.15.47:0/2029809222 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12801c420 con 0x55c58d66b870 2024-03-21T11:30:03.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.067+0000 7fc132000640 1 -- 172.21.15.47:0/2029809222 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc128005030 con 0x55c58d66b870 2024-03-21T11:30:03.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.067+0000 7fc132000640 1 -- 172.21.15.47:0/2029809222 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128003850 con 0x55c58d66b870 2024-03-21T11:30:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.067+0000 7fc147378740 1 -- 172.21.15.47:0/2029809222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66b870 msgr2=0x55c58d67c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.067+0000 7fc147378740 1 --2- 172.21.15.47:0/2029809222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66b870 0x55c58d67c410 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc12801cc40 tx=0x7fc128012990 comp rx=0 tx=0).stop 2024-03-21T11:30:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.068+0000 7fc147378740 1 -- 172.21.15.47:0/2029809222 shutdown_connections 2024-03-21T11:30:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.068+0000 7fc147378740 1 --2- 172.21.15.47:0/2029809222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d6709e0 0x55c58d66c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.068+0000 7fc147378740 1 --2- 172.21.15.47:0/2029809222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66b870 0x55c58d67c410 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.068+0000 7fc147378740 1 --2- 172.21.15.47:0/2029809222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d66ae80 0x55c58d6745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.068+0000 7fc147378740 1 -- 172.21.15.47:0/2029809222 >> 172.21.15.47:0/2029809222 conn(0x55c58d633200 msgr2=0x55c58d673ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.068+0000 7fc147378740 1 -- 172.21.15.47:0/2029809222 shutdown_connections 2024-03-21T11:30:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.068+0000 7fc147378740 1 -- 172.21.15.47:0/2029809222 wait complete. 2024-03-21T11:30:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.068+0000 7fc147378740 1 -- start start 2024-03-21T11:30:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.068+0000 7fc147378740 1 Processor -- start 2024-03-21T11:30:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.069+0000 7fc147378740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66ae80 0x55c58d679840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.069+0000 7fc147378740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d66b870 0x55c58d679d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.069+0000 7fc147378740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d6709e0 0x55c58d67ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.069+0000 7fc147378740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1b1a00 con 0x55c58d6709e0 2024-03-21T11:30:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.069+0000 7fc147378740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1b13a0 con 0x55c58d66b870 2024-03-21T11:30:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.069+0000 7fc147378740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1ae8a0 con 0x55c58d66ae80 2024-03-21T11:30:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.069+0000 7fc147378740 10 client.0 _ll_register_callbacks cb 0x7fc128009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.069+0000 7fc141200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d66b870 0x55c58d679d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.069+0000 7fc141200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d66b870 0x55c58d679d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51942/0 (socket says 172.21.15.47:51942) 2024-03-21T11:30:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.069+0000 7fc141200640 1 -- 172.21.15.47:0/3972157333 learned_addr learned my addr 172.21.15.47:0/3972157333 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.069+0000 7fc142600640 1 --2- 172.21.15.47:0/3972157333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66ae80 0x55c58d679840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.069+0000 7fc142600640 1 --2- 172.21.15.47:0/3972157333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d6709e0 0x55c58d67ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.070+0000 7fc141200640 1 -- 172.21.15.47:0/3972157333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66ae80 msgr2=0x55c58d679840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.070+0000 7fc141200640 1 --2- 172.21.15.47:0/3972157333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66ae80 0x55c58d679840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.070+0000 7fc141200640 1 -- 172.21.15.47:0/3972157333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d6709e0 msgr2=0x55c58d67ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.070+0000 7fc141200640 1 --2- 172.21.15.47:0/3972157333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d6709e0 0x55c58d67ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.070+0000 7fc141200640 1 -- 172.21.15.47:0/3972157333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c58d67b330 con 0x55c58d66b870 2024-03-21T11:30:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.070+0000 7fc141200640 1 --2- 172.21.15.47:0/3972157333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d66b870 0x55c58d679d80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc12801cbc0 tx=0x7fc128005630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=948eb636f88134d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.070+0000 7fc127400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:03.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.070+0000 7fc127400640 1 -- 172.21.15.47:0/3972157333 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12801c960 con 0x55c58d66b870 2024-03-21T11:30:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.070+0000 7fc147378740 1 -- 172.21.15.47:0/3972157333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc128005030 con 0x55c58d66b870 2024-03-21T11:30:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.070+0000 7fc127400640 1 -- 172.21.15.47:0/3972157333 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc128018040 con 0x55c58d66b870 2024-03-21T11:30:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.071+0000 7fc127400640 1 -- 172.21.15.47:0/3972157333 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1280035f0 con 0x55c58d66b870 2024-03-21T11:30:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.071+0000 7fc147378740 10 client.27683 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.071+0000 7fc147378740 1 -- 172.21.15.47:0/3972157333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c58d67c640 con 0x55c58d66b870 2024-03-21T11:30:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.071+0000 7fc127400640 1 -- 172.21.15.47:0/3972157333 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc128005030 con 0x55c58d66b870 2024-03-21T11:30:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.071+0000 7fc127400640 1 -- 172.21.15.47:0/3972157333 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc12801b860 con 0x55c58d66b870 2024-03-21T11:30:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.071+0000 7fc147378740 10 client.27683 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.071+0000 7fc147378740 10 client.27683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.071+0000 7fc147378740 20 client.27683 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.071+0000 7fc147378740 1 --2- 172.21.15.47:0/3972157333 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c58d680ce0 0x55c58d6dea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.071+0000 7fc147378740 4 client.27683 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:30:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.071+0000 7fc147378740 1 -- 172.21.15.47:0/3972157333 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c58d5d7ae0 con 0x55c58d680ce0 2024-03-21T11:30:03.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.071+0000 7fc142600640 1 --2- 172.21.15.47:0/3972157333 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c58d680ce0 0x55c58d6dea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.072+0000 7fc127400640 10 client.27683 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.072+0000 7fc142600640 1 --2- 172.21.15.47:0/3972157333 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c58d680ce0 0x55c58d6dea80 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=575e0cb161afe006 server_cookie=7850de82293edce2 in_seq=0 out_seq=0 2024-03-21T11:30:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc127400640 1 -- 172.21.15.47:0/3972157333 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c58d5d7ae0 con 0x55c58d680ce0 2024-03-21T11:30:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc127400640 10 client.27683 handle_command_reply: tid=0 2024-03-21T11:30:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc147378740 1 client.27683 shutdown 2024-03-21T11:30:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc147378740 1 -- 172.21.15.47:0/3972157333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d66b870 msgr2=0x55c58d679d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc147378740 1 --2- 172.21.15.47:0/3972157333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d66b870 0x55c58d679d80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc12801cbc0 tx=0x7fc128005630 comp rx=0 tx=0).stop 2024-03-21T11:30:03.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc147378740 1 -- 172.21.15.47:0/3972157333 shutdown_connections 2024-03-21T11:30:03.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc147378740 1 --2- 172.21.15.47:0/3972157333 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c58d680ce0 0x55c58d6dea80 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc147378740 1 --2- 172.21.15.47:0/3972157333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c58d6709e0 0x55c58d67ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc147378740 1 --2- 172.21.15.47:0/3972157333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c58d66b870 0x55c58d679d80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc147378740 1 --2- 172.21.15.47:0/3972157333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c58d66ae80 0x55c58d679840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc147378740 1 -- 172.21.15.47:0/3972157333 >> 172.21.15.47:0/3972157333 conn(0x55c58d633200 msgr2=0x55c58d66c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc127400640 0 client.27683 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:03.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc147378740 1 -- 172.21.15.47:0/3972157333 shutdown_connections 2024-03-21T11:30:03.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc147378740 1 -- 172.21.15.47:0/3972157333 wait complete. 2024-03-21T11:30:03.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.074+0000 7fc147378740 20 client.27683 trim_cache size 0 max 16384 2024-03-21T11:30:03.079 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:30:03.079 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:30:03.079 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:30:03.079 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:30:03.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.075+0000 7fc144600640 1 -- 172.21.15.47:0/3381499368 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc1200a2fa0 msgr2=0x7fc1200a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.075+0000 7fc144600640 1 --2- 172.21.15.47:0/3381499368 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc1200a2fa0 0x7fc1200a5460 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc12c006010 tx=0x7fc12c02c0f0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.075+0000 7fc144600640 1 -- 172.21.15.47:0/3381499368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc13c076060 msgr2=0x7fc13c19dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.075+0000 7fc144600640 1 --2- 172.21.15.47:0/3381499368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc13c076060 0x7fc13c19dfd0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc12c02c630 tx=0x7fc12c032f20 comp rx=0 tx=0).stop 2024-03-21T11:30:03.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.075+0000 7fc144600640 1 -- 172.21.15.47:0/3381499368 shutdown_connections 2024-03-21T11:30:03.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.075+0000 7fc144600640 1 --2- 172.21.15.47:0/3381499368 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc1200a2fa0 0x7fc1200a5460 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.075+0000 7fc144600640 1 --2- 172.21.15.47:0/3381499368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc13c077c40 0x7fc13c1a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.075+0000 7fc144600640 1 --2- 172.21.15.47:0/3381499368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc13c0772a0 0x7fc13c19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.075+0000 7fc144600640 1 --2- 172.21.15.47:0/3381499368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc13c076060 0x7fc13c19dfd0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.075+0000 7fc144600640 1 -- 172.21.15.47:0/3381499368 >> 172.21.15.47:0/3381499368 conn(0x7fc13c0fee50 msgr2=0x7fc13c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.075+0000 7fc144600640 1 -- 172.21.15.47:0/3381499368 shutdown_connections 2024-03-21T11:30:03.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.075+0000 7fc144600640 1 -- 172.21.15.47:0/3381499368 wait complete. 2024-03-21T11:30:03.095 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:30:03.095 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:30:03.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:02 smithi100 ceph-mon[36245]: pgmap v601: 129 pgs: 129 active+clean; 38 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.7 MiB/s wr, 240 op/s 2024-03-21T11:30:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:02 smithi102 ceph-mon[36748]: pgmap v601: 129 pgs: 129 active+clean; 38 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.7 MiB/s wr, 240 op/s 2024-03-21T11:30:03.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.332+0000 7f472ac00640 1 -- 172.21.15.47:0/2128820966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47241096b0 msgr2=0x7f472410ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.332+0000 7f472ac00640 1 --2- 172.21.15.47:0/2128820966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47241096b0 0x7f472410ff40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f471c009ca0 tx=0x7f471c02c0f0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.332+0000 7f472ac00640 1 -- 172.21.15.47:0/2128820966 shutdown_connections 2024-03-21T11:30:03.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.332+0000 7f472ac00640 1 --2- 172.21.15.47:0/2128820966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47241096b0 0x7f472410ff40 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.332+0000 7f472ac00640 1 --2- 172.21.15.47:0/2128820966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4724108cf0 0x7f4724109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.332+0000 7f472ac00640 1 --2- 172.21.15.47:0/2128820966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4724107b00 0x7f4724107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.332+0000 7f472ac00640 1 -- 172.21.15.47:0/2128820966 >> 172.21.15.47:0/2128820966 conn(0x7f4724076ab0 msgr2=0x7f4724078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.333+0000 7f472ac00640 1 -- 172.21.15.47:0/2128820966 shutdown_connections 2024-03-21T11:30:03.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.333+0000 7f472ac00640 1 -- 172.21.15.47:0/2128820966 wait complete. 2024-03-21T11:30:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.334+0000 7f472ac00640 1 Processor -- start 2024-03-21T11:30:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.334+0000 7f472ac00640 1 -- start start 2024-03-21T11:30:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.334+0000 7f472ac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4724107b00 0x7f472419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.334+0000 7f472ac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4724108cf0 0x7f472419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.334+0000 7f472ac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47241096b0 0x7f47241a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.334+0000 7f472ac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4724113760 con 0x7f4724108cf0 2024-03-21T11:30:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.334+0000 7f472ac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47241135e0 con 0x7f47241096b0 2024-03-21T11:30:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.334+0000 7f472ac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47241138e0 con 0x7f4724107b00 2024-03-21T11:30:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.334+0000 7f4729800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4724107b00 0x7f472419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.334+0000 7f4729800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4724107b00 0x7f472419dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59690/0 (socket says 172.21.15.47:59690) 2024-03-21T11:30:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.334+0000 7f4729800640 1 -- 172.21.15.47:0/1727268342 learned_addr learned my addr 172.21.15.47:0/1727268342 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.334+0000 7f472a200640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47241096b0 0x7f47241a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.335+0000 7f4728e00640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4724108cf0 0x7f472419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.335+0000 7f4729800640 1 -- 172.21.15.47:0/1727268342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47241096b0 msgr2=0x7f47241a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.335+0000 7f4729800640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47241096b0 0x7f47241a5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.335+0000 7f4729800640 1 -- 172.21.15.47:0/1727268342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4724108cf0 msgr2=0x7f472419e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.335+0000 7f4729800640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4724108cf0 0x7f472419e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.335+0000 7f4729800640 1 -- 172.21.15.47:0/1727268342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47241a5990 con 0x7f4724107b00 2024-03-21T11:30:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.335+0000 7f472a200640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47241096b0 0x7f47241a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.335+0000 7f4729800640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4724107b00 0x7f472419dd70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f471400d950 tx=0x7f471400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.335+0000 7f4722000640 1 -- 172.21.15.47:0/1727268342 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714017020 con 0x7f4724107b00 2024-03-21T11:30:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.336+0000 7f4722000640 1 -- 172.21.15.47:0/1727268342 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4714009c30 con 0x7f4724107b00 2024-03-21T11:30:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.336+0000 7f472ac00640 1 -- 172.21.15.47:0/1727268342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47241a5c80 con 0x7f4724107b00 2024-03-21T11:30:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.336+0000 7f4722000640 1 -- 172.21.15.47:0/1727268342 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471400f720 con 0x7f4724107b00 2024-03-21T11:30:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.336+0000 7f472ac00640 1 -- 172.21.15.47:0/1727268342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47241a61d0 con 0x7f4724107b00 2024-03-21T11:30:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.337+0000 7f472ac00640 1 -- 172.21.15.47:0/1727268342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472410ee20 con 0x7f4724107b00 2024-03-21T11:30:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.339+0000 7f4722000640 1 -- 172.21.15.47:0/1727268342 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f471401a020 con 0x7f4724107b00 2024-03-21T11:30:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.340+0000 7f4722000640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f470c0a2fa0 0x7f470c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.340+0000 7f4722000640 1 -- 172.21.15.47:0/1727268342 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f47140cf5e0 con 0x7f4724107b00 2024-03-21T11:30:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.341+0000 7f4728e00640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f470c0a2fa0 0x7f470c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.341+0000 7f4728e00640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f470c0a2fa0 0x7f470c0a5460 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f472419f280 tx=0x7f4718005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.346+0000 7f4722000640 1 -- 172.21.15.47:0/1727268342 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4714021020 con 0x7f4724107b00 2024-03-21T11:30:03.525 INFO:tasks.workunit.client.0.smithi047.stdout: 1 449713 8.20 MB/sec execute 167 sec latency 27.451 ms 2024-03-21T11:30:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.610+0000 7f472ac00640 1 -- 172.21.15.47:0/1727268342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f472402d070 con 0x7f4724107b00 2024-03-21T11:30:03.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.612+0000 7f4722000640 1 -- 172.21.15.47:0/1727268342 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f4714009de0 con 0x7f4724107b00 2024-03-21T11:30:03.613 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:30:03.613 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:30:03.613 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:30:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.616+0000 7f472ac00640 1 -- 172.21.15.47:0/1727268342 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f470c0a2fa0 msgr2=0x7f470c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.616+0000 7f472ac00640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f470c0a2fa0 0x7f470c0a5460 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f472419f280 tx=0x7f4718005f90 comp rx=0 tx=0).stop 2024-03-21T11:30:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.616+0000 7f472ac00640 1 -- 172.21.15.47:0/1727268342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4724107b00 msgr2=0x7f472419dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.616+0000 7f472ac00640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4724107b00 0x7f472419dd70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f471400d950 tx=0x7f471400de10 comp rx=0 tx=0).stop 2024-03-21T11:30:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.616+0000 7f472ac00640 1 -- 172.21.15.47:0/1727268342 shutdown_connections 2024-03-21T11:30:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.616+0000 7f472ac00640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f470c0a2fa0 0x7f470c0a5460 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.616+0000 7f472ac00640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47241096b0 0x7f47241a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.616+0000 7f472ac00640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4724108cf0 0x7f472419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.616+0000 7f472ac00640 1 --2- 172.21.15.47:0/1727268342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4724107b00 0x7f472419dd70 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.616+0000 7f472ac00640 1 -- 172.21.15.47:0/1727268342 >> 172.21.15.47:0/1727268342 conn(0x7f4724076ab0 msgr2=0x7f472410bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.616+0000 7f472ac00640 1 -- 172.21.15.47:0/1727268342 shutdown_connections 2024-03-21T11:30:03.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.616+0000 7f472ac00640 1 -- 172.21.15.47:0/1727268342 wait complete. 2024-03-21T11:30:03.640 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:30:03.640 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:30:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.841+0000 7f3a87000640 1 -- 172.21.15.47:0/4150407203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a80074080 msgr2=0x7f3a800744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.841+0000 7f3a87000640 1 --2- 172.21.15.47:0/4150407203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a80074080 0x7f3a800744e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009900 tx=0x7f3a6802bcd0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.841+0000 7f3a87000640 1 -- 172.21.15.47:0/4150407203 shutdown_connections 2024-03-21T11:30:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.841+0000 7f3a87000640 1 --2- 172.21.15.47:0/4150407203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a80109580 0x7f3a8010ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.841+0000 7f3a87000640 1 --2- 172.21.15.47:0/4150407203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a80074080 0x7f3a800744e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.841+0000 7f3a87000640 1 --2- 172.21.15.47:0/4150407203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a80075660 0x7f3a80073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.841+0000 7f3a87000640 1 -- 172.21.15.47:0/4150407203 >> 172.21.15.47:0/4150407203 conn(0x7f3a800fcc60 msgr2=0x7f3a800ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.841+0000 7f3a87000640 1 -- 172.21.15.47:0/4150407203 shutdown_connections 2024-03-21T11:30:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.841+0000 7f3a87000640 1 -- 172.21.15.47:0/4150407203 wait complete. 2024-03-21T11:30:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.842+0000 7f3a87000640 1 Processor -- start 2024-03-21T11:30:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.842+0000 7f3a87000640 1 -- start start 2024-03-21T11:30:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.842+0000 7f3a87000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a80074080 0x7f3a80199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.842+0000 7f3a87000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a80075660 0x7f3a8019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.842+0000 7f3a87000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a80109580 0x7f3a801a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.842+0000 7f3a87000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8010f5b0 con 0x7f3a80074080 2024-03-21T11:30:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.842+0000 7f3a87000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8010f430 con 0x7f3a80075660 2024-03-21T11:30:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.842+0000 7f3a87000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8010f730 con 0x7f3a80109580 2024-03-21T11:30:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.843+0000 7f3a7f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a80075660 0x7f3a8019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.843+0000 7f3a85000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a80109580 0x7f3a801a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.843+0000 7f3a7f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a80075660 0x7f3a8019a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51992/0 (socket says 172.21.15.47:51992) 2024-03-21T11:30:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.843+0000 7f3a7f400640 1 -- 172.21.15.47:0/1709692943 learned_addr learned my addr 172.21.15.47:0/1709692943 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.843+0000 7f3a7fe00640 1 --2- 172.21.15.47:0/1709692943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a80074080 0x7f3a80199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.843+0000 7f3a85000640 1 -- 172.21.15.47:0/1709692943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a80075660 msgr2=0x7f3a8019a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.843+0000 7f3a85000640 1 --2- 172.21.15.47:0/1709692943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a80075660 0x7f3a8019a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.843+0000 7f3a85000640 1 -- 172.21.15.47:0/1709692943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a80074080 msgr2=0x7f3a80199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.843+0000 7f3a85000640 1 --2- 172.21.15.47:0/1709692943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a80074080 0x7f3a80199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.843+0000 7f3a85000640 1 -- 172.21.15.47:0/1709692943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a801a1720 con 0x7f3a80109580 2024-03-21T11:30:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.843+0000 7f3a7f400640 1 --2- 172.21.15.47:0/1709692943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a80075660 0x7f3a8019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.843+0000 7f3a85000640 1 --2- 172.21.15.47:0/1709692943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a80109580 0x7f3a801a10b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009fd0 tx=0x7f3a7800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.844+0000 7f3a7cc00640 1 -- 172.21.15.47:0/1709692943 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800ce20 con 0x7f3a80109580 2024-03-21T11:30:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.844+0000 7f3a7cc00640 1 -- 172.21.15.47:0/1709692943 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a78005020 con 0x7f3a80109580 2024-03-21T11:30:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.844+0000 7f3a7cc00640 1 -- 172.21.15.47:0/1709692943 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a78010720 con 0x7f3a80109580 2024-03-21T11:30:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.844+0000 7f3a87000640 1 -- 172.21.15.47:0/1709692943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a801a1a10 con 0x7f3a80109580 2024-03-21T11:30:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.844+0000 7f3a87000640 1 -- 172.21.15.47:0/1709692943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a801a1f10 con 0x7f3a80109580 2024-03-21T11:30:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.847+0000 7f3a7cc00640 1 -- 172.21.15.47:0/1709692943 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3a78005540 con 0x7f3a80109580 2024-03-21T11:30:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.847+0000 7f3a89dd8740 1 Processor -- start 2024-03-21T11:30:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.848+0000 7f3a89dd8740 1 -- start start 2024-03-21T11:30:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.848+0000 7f3a89dd8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 0x5636b6efde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.848+0000 7f3a89dd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f02ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.848+0000 7f3a89dd8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6f03550 0x5636b6f059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.848+0000 7f3a89dd8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b6eb83b0 con 0x5636b6f03550 2024-03-21T11:30:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.848+0000 7f3a89dd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b6e5a830 con 0x5636b6efe3e0 2024-03-21T11:30:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.848+0000 7f3a89dd8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b6e035d0 con 0x5636b6efd9f0 2024-03-21T11:30:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.848+0000 7f3a7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f02ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.848+0000 7f3a7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f02ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52008/0 (socket says 172.21.15.47:52008) 2024-03-21T11:30:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.848+0000 7f3a7f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 0x5636b6efde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.848+0000 7f3a7fe00640 1 -- 172.21.15.47:0/3979021229 learned_addr learned my addr 172.21.15.47:0/3979021229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.848+0000 7f3a7cc00640 1 --2- 172.21.15.47:0/1709692943 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a640a3040 0x7f3a640a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.849+0000 7f3a7f400640 1 --2- 172.21.15.47:0/3979021229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6f03550 0x5636b6f059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.849+0000 7f3a7cc00640 1 -- 172.21.15.47:0/1709692943 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3a780cfd50 con 0x7f3a80109580 2024-03-21T11:30:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.849+0000 7f3a7fe00640 1 -- 172.21.15.47:0/3979021229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 msgr2=0x5636b6efde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.849+0000 7f3a7fe00640 1 --2- 172.21.15.47:0/3979021229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 0x5636b6efde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.849+0000 7f3a7fe00640 1 -- 172.21.15.47:0/3979021229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6f03550 msgr2=0x5636b6f059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.849+0000 7f3a85000640 1 --2- 172.21.15.47:0/1709692943 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a640a3040 0x7f3a640a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.849+0000 7f3a7fe00640 1 --2- 172.21.15.47:0/3979021229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6f03550 0x5636b6f059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.849+0000 7f3a7fe00640 1 -- 172.21.15.47:0/3979021229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636b6e1b2c0 con 0x5636b6efe3e0 2024-03-21T11:30:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.849+0000 7f3a7f400640 1 --2- 172.21.15.47:0/3979021229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6f03550 0x5636b6f059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:30:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.849+0000 7f3a7fe00640 1 --2- 172.21.15.47:0/3979021229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f02ee0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009cf0 tx=0x7f3a70011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9deb24447989f6d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.850+0000 7f3a85000640 1 --2- 172.21.15.47:0/1709692943 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a640a3040 0x7f3a640a5500 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3a780058f0 tx=0x7f3a780054b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.850+0000 7f3a75600640 1 -- 172.21.15.47:0/3979021229 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70004230 con 0x5636b6efe3e0 2024-03-21T11:30:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.850+0000 7f3a75600640 1 -- 172.21.15.47:0/3979021229 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a700043d0 con 0x5636b6efe3e0 2024-03-21T11:30:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.850+0000 7f3a75600640 1 -- 172.21.15.47:0/3979021229 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a700198c0 con 0x5636b6efe3e0 2024-03-21T11:30:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.851+0000 7f3a89dd8740 1 -- 172.21.15.47:0/3979021229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 msgr2=0x5636b6f02ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.851+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/3979021229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f02ee0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009cf0 tx=0x7f3a70011810 comp rx=0 tx=0).stop 2024-03-21T11:30:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.851+0000 7f3a89dd8740 1 -- 172.21.15.47:0/3979021229 shutdown_connections 2024-03-21T11:30:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.851+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/3979021229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6f03550 0x5636b6f059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.851+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/3979021229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f02ee0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.851+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/3979021229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 0x5636b6efde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.851+0000 7f3a89dd8740 1 -- 172.21.15.47:0/3979021229 >> 172.21.15.47:0/3979021229 conn(0x5636b6ef0750 msgr2=0x5636b6ef0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.851+0000 7f3a89dd8740 1 -- 172.21.15.47:0/3979021229 shutdown_connections 2024-03-21T11:30:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.851+0000 7f3a89dd8740 1 -- 172.21.15.47:0/3979021229 wait complete. 2024-03-21T11:30:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.851+0000 7f3a89dd8740 1 -- start start 2024-03-21T11:30:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.852+0000 7f3a89dd8740 1 Processor -- start 2024-03-21T11:30:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.852+0000 7f3a89dd8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6efd9f0 0x5636b6f07b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.852+0000 7f3a89dd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f08090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.852+0000 7f3a89dd8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6f03550 0x5636b6f0f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.852+0000 7f3a89dd8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b6e16a20 con 0x5636b6efd9f0 2024-03-21T11:30:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.852+0000 7f3a89dd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b6e568d0 con 0x5636b6efe3e0 2024-03-21T11:30:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.852+0000 7f3a89dd8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b6f06500 con 0x5636b6f03550 2024-03-21T11:30:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.852+0000 7f3a89dd8740 10 client.0 _ll_register_callbacks cb 0x5636b6e531c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.853+0000 7f3a7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f08090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.853+0000 7f3a7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f08090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52018/0 (socket says 172.21.15.47:52018) 2024-03-21T11:30:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.853+0000 7f3a7fe00640 1 -- 172.21.15.47:0/546381163 learned_addr learned my addr 172.21.15.47:0/546381163 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.853+0000 7f3a7f400640 1 --2- 172.21.15.47:0/546381163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6f03550 0x5636b6f0f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.853+0000 7f3a7f400640 1 --2- 172.21.15.47:0/546381163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6efd9f0 0x5636b6f07b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.853+0000 7f3a7fe00640 1 -- 172.21.15.47:0/546381163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6f03550 msgr2=0x5636b6f0f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.853+0000 7f3a7fe00640 1 --2- 172.21.15.47:0/546381163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6f03550 0x5636b6f0f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.853+0000 7f3a7fe00640 1 -- 172.21.15.47:0/546381163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6efd9f0 msgr2=0x5636b6f07b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.853+0000 7f3a7fe00640 1 --2- 172.21.15.47:0/546381163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6efd9f0 0x5636b6f07b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.853+0000 7f3a7fe00640 1 -- 172.21.15.47:0/546381163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636b6b6f210 con 0x5636b6efe3e0 2024-03-21T11:30:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.854+0000 7f3a7fe00640 1 --2- 172.21.15.47:0/546381163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f08090 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009f50 tx=0x7f3a70003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8dd8d194f42a748 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.854+0000 7f3a6e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.854+0000 7f3a6e000640 1 -- 172.21.15.47:0/546381163 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70027070 con 0x5636b6efe3e0 2024-03-21T11:30:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.854+0000 7f3a89dd8740 1 -- 172.21.15.47:0/546381163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a700043d0 con 0x5636b6efe3e0 2024-03-21T11:30:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.854+0000 7f3a6e000640 1 -- 172.21.15.47:0/546381163 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3a70005240 con 0x5636b6efe3e0 2024-03-21T11:30:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.854+0000 7f3a6e000640 1 -- 172.21.15.47:0/546381163 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a70033050 con 0x5636b6efe3e0 2024-03-21T11:30:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.854+0000 7f3a89dd8740 10 client.27701 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.854+0000 7f3a89dd8740 1 -- 172.21.15.47:0/546381163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5636b6c3b620 con 0x5636b6efe3e0 2024-03-21T11:30:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.855+0000 7f3a6e000640 1 -- 172.21.15.47:0/546381163 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a700043d0 con 0x5636b6efe3e0 2024-03-21T11:30:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.855+0000 7f3a6e000640 1 -- 172.21.15.47:0/546381163 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3a70004580 con 0x5636b6efe3e0 2024-03-21T11:30:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.855+0000 7f3a89dd8740 10 client.27701 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.855+0000 7f3a89dd8740 10 client.27701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.855+0000 7f3a89dd8740 20 client.27701 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.855+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/546381163 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5636b6f136e0 0x5636b6f15af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.855+0000 7f3a89dd8740 4 client.27701 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.855+0000 7f3a89dd8740 1 -- 172.21.15.47:0/546381163 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636b6e67d70 con 0x5636b6f136e0 2024-03-21T11:30:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.856+0000 7f3a7f400640 1 --2- 172.21.15.47:0/546381163 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5636b6f136e0 0x5636b6f15af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.856+0000 7f3a7f400640 1 --2- 172.21.15.47:0/546381163 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5636b6f136e0 0x5636b6f15af0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b9e065b391904e server_cookie=dfe934350f26c05 in_seq=0 out_seq=0 2024-03-21T11:30:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.856+0000 7f3a6e000640 10 client.27701 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.857+0000 7f1e61c00640 1 -- 172.21.15.47:0/882552543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e5c0772a0 msgr2=0x7f1e5c077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.857+0000 7f1e61c00640 1 --2- 172.21.15.47:0/882552543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e5c0772a0 0x7f1e5c077700 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009900 tx=0x7f1e4402bc30 comp rx=0 tx=0).stop 2024-03-21T11:30:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.857+0000 7f1e61c00640 1 -- 172.21.15.47:0/882552543 shutdown_connections 2024-03-21T11:30:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.857+0000 7f1e61c00640 1 --2- 172.21.15.47:0/882552543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e5c077c40 0x7f1e5c0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.857+0000 7f1e61c00640 1 --2- 172.21.15.47:0/882552543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e5c0772a0 0x7f1e5c077700 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.857+0000 7f1e61c00640 1 --2- 172.21.15.47:0/882552543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e5c076060 0x7f1e5c076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.857+0000 7f1e61c00640 1 -- 172.21.15.47:0/882552543 >> 172.21.15.47:0/882552543 conn(0x7f1e5c0fee50 msgr2=0x7f1e5c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.857+0000 7f1e61c00640 1 -- 172.21.15.47:0/882552543 shutdown_connections 2024-03-21T11:30:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.857+0000 7f3a6e000640 1 -- 172.21.15.47:0/546381163 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5636b6e67d70 con 0x5636b6f136e0 2024-03-21T11:30:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.857+0000 7f3a6e000640 10 client.27701 handle_command_reply: tid=0 2024-03-21T11:30:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.857+0000 7f1e61c00640 1 -- 172.21.15.47:0/882552543 wait complete. 2024-03-21T11:30:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.857+0000 7f3a89dd8740 1 client.27701 shutdown 2024-03-21T11:30:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.858+0000 7f3a89dd8740 1 -- 172.21.15.47:0/546381163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 msgr2=0x5636b6f08090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.858+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/546381163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f08090 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009f50 tx=0x7f3a70003820 comp rx=0 tx=0).stop 2024-03-21T11:30:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.858+0000 7f3a89dd8740 1 -- 172.21.15.47:0/546381163 shutdown_connections 2024-03-21T11:30:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.858+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/546381163 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5636b6f136e0 0x5636b6f15af0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.858+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/546381163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6f03550 0x5636b6f0f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.858+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/546381163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f08090 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.858+0000 7f3a6e000640 0 client.27701 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.858+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/546381163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6efd9f0 0x5636b6f07b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.858+0000 7f3a89dd8740 1 -- 172.21.15.47:0/546381163 >> 172.21.15.47:0/546381163 conn(0x5636b6ef0750 msgr2=0x5636b6eef950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.858+0000 7f3a89dd8740 1 -- 172.21.15.47:0/546381163 shutdown_connections 2024-03-21T11:30:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.858+0000 7f3a89dd8740 1 -- 172.21.15.47:0/546381163 wait complete. 2024-03-21T11:30:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.858+0000 7f3a89dd8740 20 client.27701 trim_cache size 0 max 16384 2024-03-21T11:30:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.858+0000 7f1e61c00640 1 Processor -- start 2024-03-21T11:30:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.859+0000 7f1e61c00640 1 -- start start 2024-03-21T11:30:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.859+0000 7f1e61c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e5c076060 0x7f1e5c19e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.859+0000 7f1e61c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e5c0772a0 0x7f1e5c19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.859+0000 7f1e61c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e5c077c40 0x7f1e5c1a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.859+0000 7f1e61c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c1139b0 con 0x7f1e5c0772a0 2024-03-21T11:30:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.859+0000 7f1e61c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c113830 con 0x7f1e5c076060 2024-03-21T11:30:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.859+0000 7f1e61c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c113b30 con 0x7f1e5c077c40 2024-03-21T11:30:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.859+0000 7f1e5aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e5c076060 0x7f1e5c19e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.859+0000 7f1e5aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e5c076060 0x7f1e5c19e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52036/0 (socket says 172.21.15.47:52036) 2024-03-21T11:30:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.859+0000 7f1e5aa00640 1 -- 172.21.15.47:0/206910640 learned_addr learned my addr 172.21.15.47:0/206910640 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.859+0000 7f1e5b400640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e5c077c40 0x7f1e5c1a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.859+0000 7f1e52000640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e5c0772a0 0x7f1e5c19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.860+0000 7f1e5aa00640 1 -- 172.21.15.47:0/206910640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e5c077c40 msgr2=0x7f1e5c1a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.860+0000 7f1e5aa00640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e5c077c40 0x7f1e5c1a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.860+0000 7f1e5aa00640 1 -- 172.21.15.47:0/206910640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e5c0772a0 msgr2=0x7f1e5c19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.860+0000 7f1e5aa00640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e5c0772a0 0x7f1e5c19e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.860+0000 7f1e5aa00640 1 -- 172.21.15.47:0/206910640 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c1a5c30 con 0x7f1e5c076060 2024-03-21T11:30:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.860+0000 7f1e5b400640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e5c077c40 0x7f1e5c1a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.860+0000 7f1e5aa00640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e5c076060 0x7f1e5c19e010 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00e9f0 tx=0x7f1e4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.860+0000 7f1e53e00640 1 -- 172.21.15.47:0/206910640 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c00cd80 con 0x7f1e5c076060 2024-03-21T11:30:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.860+0000 7f1e53e00640 1 -- 172.21.15.47:0/206910640 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e4c010910 con 0x7f1e5c076060 2024-03-21T11:30:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.860+0000 7f1e53e00640 1 -- 172.21.15.47:0/206910640 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c018990 con 0x7f1e5c076060 2024-03-21T11:30:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.860+0000 7f1e61c00640 1 -- 172.21.15.47:0/206910640 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c1a5f20 con 0x7f1e5c076060 2024-03-21T11:30:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.861+0000 7f1e61c00640 1 -- 172.21.15.47:0/206910640 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c1a6330 con 0x7f1e5c076060 2024-03-21T11:30:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.864+0000 7f1e53e00640 1 -- 172.21.15.47:0/206910640 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1e4c010ac0 con 0x7f1e5c076060 2024-03-21T11:30:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.866+0000 7f1e649e9740 1 Processor -- start 2024-03-21T11:30:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.866+0000 7f1e649e9740 1 -- start start 2024-03-21T11:30:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.866+0000 7f1e53e00640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1e400a2fa0 0x7f1e400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.866+0000 7f1e53e00640 1 -- 172.21.15.47:0/206910640 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1e4c0cfa40 con 0x7f1e5c076060 2024-03-21T11:30:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.866+0000 7f1e649e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb4810 0x55dba7fb4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.866+0000 7f1e649e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb59e0 0x55dba7fb5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.866+0000 7f1e649e9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb63a0 0x55dba7fbcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.866+0000 7f1e649e9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba7f192c0 con 0x55dba7fb63a0 2024-03-21T11:30:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.866+0000 7f1e649e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba7ec1c90 con 0x55dba7fb4810 2024-03-21T11:30:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.866+0000 7f1e649e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba7f15360 con 0x55dba7fb59e0 2024-03-21T11:30:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.867+0000 7f1e5b400640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1e400a2fa0 0x7f1e400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.867+0000 7f1e5aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb4810 0x55dba7fb4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.867+0000 7f1e52000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb59e0 0x55dba7fb5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.867+0000 7f1e5aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb4810 0x55dba7fb4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52052/0 (socket says 172.21.15.47:52052) 2024-03-21T11:30:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.867+0000 7f1e5aa00640 1 -- 172.21.15.47:0/2999014392 learned_addr learned my addr 172.21.15.47:0/2999014392 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.867+0000 7f1e5aa00640 1 -- 172.21.15.47:0/2999014392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb59e0 msgr2=0x55dba7fb5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.867+0000 7f1e5aa00640 1 --2- 172.21.15.47:0/2999014392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb59e0 0x55dba7fb5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.867+0000 7f1e5aa00640 1 -- 172.21.15.47:0/2999014392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb63a0 msgr2=0x55dba7fbcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.867+0000 7f1e5aa00640 1 --2- 172.21.15.47:0/2999014392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb63a0 0x55dba7fbcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.867+0000 7f1e5aa00640 1 -- 172.21.15.47:0/2999014392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dba7ee6080 con 0x55dba7fb4810 2024-03-21T11:30:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.867+0000 7f1e5aa00640 1 --2- 172.21.15.47:0/2999014392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb4810 0x55dba7fb4c30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0a4260 tx=0x7f1e4c09bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da32e992a0412dcd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.868+0000 7f1e4be00640 1 -- 172.21.15.47:0/2999014392 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c09a130 con 0x55dba7fb4810 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.868+0000 7f1e4be00640 1 -- 172.21.15.47:0/2999014392 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e4c0984a0 con 0x55dba7fb4810 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.868+0000 7f1e4be00640 1 -- 172.21.15.47:0/2999014392 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c09b4e0 con 0x55dba7fb4810 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.869+0000 7f1e649e9740 1 -- 172.21.15.47:0/2999014392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb4810 msgr2=0x55dba7fb4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.869+0000 7f1e649e9740 1 --2- 172.21.15.47:0/2999014392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb4810 0x55dba7fb4c30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0a4260 tx=0x7f1e4c09bef0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.869+0000 7f1e5b400640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1e400a2fa0 0x7f1e400a5460 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1e54005fd0 tx=0x7f1e54009e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.869+0000 7f1e649e9740 1 -- 172.21.15.47:0/2999014392 shutdown_connections 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.869+0000 7f1e649e9740 1 --2- 172.21.15.47:0/2999014392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb63a0 0x55dba7fbcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.869+0000 7f1e649e9740 1 --2- 172.21.15.47:0/2999014392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb59e0 0x55dba7fb5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.869+0000 7f1e649e9740 1 --2- 172.21.15.47:0/2999014392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb4810 0x55dba7fb4c30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.869+0000 7f1e649e9740 1 -- 172.21.15.47:0/2999014392 >> 172.21.15.47:0/2999014392 conn(0x55dba7fafc00 msgr2=0x55dba7fb0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.869+0000 7f1e649e9740 1 -- 172.21.15.47:0/2999014392 shutdown_connections 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.869+0000 7f1e649e9740 1 -- 172.21.15.47:0/2999014392 wait complete. 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.869+0000 7f1e649e9740 1 -- start start 2024-03-21T11:30:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.870+0000 7f1e649e9740 1 Processor -- start 2024-03-21T11:30:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.870+0000 7f1e649e9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 0x55dba7fbea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.870+0000 7f1e649e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fbef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.870+0000 7f1e649e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fc5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.870+0000 7f1e649e9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba7fbd350 con 0x55dba7fb4810 2024-03-21T11:30:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.870+0000 7f1e649e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba7ed50d0 con 0x55dba7fb59e0 2024-03-21T11:30:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.870+0000 7f1e649e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba7fbd4d0 con 0x55dba7fb63a0 2024-03-21T11:30:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.870+0000 7f1e649e9740 10 client.0 _ll_register_callbacks cb 0x55dba7ed9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.870+0000 7f1e5aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 0x55dba7fbea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.870+0000 7f1e5b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fc5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.870+0000 7f1e5b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fc5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59740/0 (socket says 172.21.15.47:59740) 2024-03-21T11:30:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.870+0000 7f1e5b400640 1 -- 172.21.15.47:0/1274891832 learned_addr learned my addr 172.21.15.47:0/1274891832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.871+0000 7f1e52000640 1 --2- 172.21.15.47:0/1274891832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fbef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.871+0000 7f1e5b400640 1 -- 172.21.15.47:0/1274891832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 msgr2=0x55dba7fbef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.871+0000 7f1e5b400640 1 --2- 172.21.15.47:0/1274891832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fbef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.871+0000 7f1e5b400640 1 -- 172.21.15.47:0/1274891832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 msgr2=0x55dba7fbea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.871+0000 7f1e5b400640 1 --2- 172.21.15.47:0/1274891832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 0x55dba7fbea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.871+0000 7f1e5b400640 1 -- 172.21.15.47:0/1274891832 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dba7c2d3f0 con 0x55dba7fb63a0 2024-03-21T11:30:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.871+0000 7f1e5aa00640 1 --2- 172.21.15.47:0/1274891832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 0x55dba7fbea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.871+0000 7f1e52000640 1 --2- 172.21.15.47:0/1274891832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fbef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.871+0000 7f1e5b400640 1 --2- 172.21.15.47:0/1274891832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fc5fb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1e54005f40 tx=0x7f1e54004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d948c3cc8b3dadfb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.871+0000 7f1e48c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.872+0000 7f1e48c00640 1 -- 172.21.15.47:0/1274891832 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54023030 con 0x55dba7fb63a0 2024-03-21T11:30:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.872+0000 7f1e48c00640 1 -- 172.21.15.47:0/1274891832 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1e54004540 con 0x55dba7fb63a0 2024-03-21T11:30:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.872+0000 7f1e649e9740 1 -- 172.21.15.47:0/1274891832 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e4c0984a0 con 0x55dba7fb63a0 2024-03-21T11:30:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.872+0000 7f1e48c00640 1 -- 172.21.15.47:0/1274891832 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e54030050 con 0x55dba7fb63a0 2024-03-21T11:30:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.872+0000 7f1e649e9740 10 client.27733 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.872+0000 7f1e649e9740 1 -- 172.21.15.47:0/1274891832 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dba7cf77d0 con 0x55dba7fb63a0 2024-03-21T11:30:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.873+0000 7f1e48c00640 1 -- 172.21.15.47:0/1274891832 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e4c0984a0 con 0x55dba7fb63a0 2024-03-21T11:30:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.873+0000 7f1e48c00640 1 -- 172.21.15.47:0/1274891832 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1e5401c050 con 0x55dba7fb63a0 2024-03-21T11:30:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.873+0000 7f1e649e9740 10 client.27733 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.873+0000 7f1e649e9740 10 client.27733 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:30:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.873+0000 7f1e649e9740 20 client.27733 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.873+0000 7f1e649e9740 1 --2- 172.21.15.47:0/1274891832 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dba7fca570 0x55dba7fcc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.873+0000 7f1e649e9740 4 client.27733 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.873+0000 7f1e649e9740 1 -- 172.21.15.47:0/1274891832 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dba7f26800 con 0x55dba7fca570 2024-03-21T11:30:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.873+0000 7f1e5aa00640 1 --2- 172.21.15.47:0/1274891832 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dba7fca570 0x55dba7fcc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.874+0000 7f1e5aa00640 1 --2- 172.21.15.47:0/1274891832 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dba7fca570 0x55dba7fcc9b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=21d2c1e0a0229263 server_cookie=8bd156b409e386fe in_seq=0 out_seq=0 2024-03-21T11:30:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.874+0000 7f1e48c00640 10 client.27733 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:30:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.874+0000 7f1e48c00640 1 -- 172.21.15.47:0/1274891832 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dba7f26800 con 0x55dba7fca570 2024-03-21T11:30:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.874+0000 7f1e48c00640 10 client.27733 handle_command_reply: tid=0 2024-03-21T11:30:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.875+0000 7f1e649e9740 1 client.27733 shutdown 2024-03-21T11:30:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.875+0000 7f1e649e9740 1 -- 172.21.15.47:0/1274891832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 msgr2=0x55dba7fc5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.875+0000 7f1e649e9740 1 --2- 172.21.15.47:0/1274891832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fc5fb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1e54005f40 tx=0x7f1e54004ab0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.875+0000 7f1e649e9740 1 -- 172.21.15.47:0/1274891832 shutdown_connections 2024-03-21T11:30:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.875+0000 7f1e649e9740 1 --2- 172.21.15.47:0/1274891832 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dba7fca570 0x55dba7fcc9b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.875+0000 7f1e649e9740 1 --2- 172.21.15.47:0/1274891832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fc5fb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.875+0000 7f1e48c00640 0 client.27733 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:30:03.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.875+0000 7f1e649e9740 1 --2- 172.21.15.47:0/1274891832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fbef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.875+0000 7f1e649e9740 1 --2- 172.21.15.47:0/1274891832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 0x55dba7fbea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.875+0000 7f1e649e9740 1 -- 172.21.15.47:0/1274891832 >> 172.21.15.47:0/1274891832 conn(0x55dba7fafc00 msgr2=0x55dba7faeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.875+0000 7f1e649e9740 1 -- 172.21.15.47:0/1274891832 shutdown_connections 2024-03-21T11:30:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.875+0000 7f1e649e9740 1 -- 172.21.15.47:0/1274891832 wait complete. 2024-03-21T11:30:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.875+0000 7f1e649e9740 20 client.27733 trim_cache size 0 max 16384 2024-03-21T11:30:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.884+0000 7f3a89dd8740 1 Processor -- start 2024-03-21T11:30:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.885+0000 7f3a89dd8740 1 -- start start 2024-03-21T11:30:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.885+0000 7f3a89dd8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 0x5636b6ef40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.885+0000 7f3a89dd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f0f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.885+0000 7f3a89dd8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6f03550 0x5636b6f1a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.885+0000 7f3a89dd8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a801a7e80 con 0x5636b6f03550 2024-03-21T11:30:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.885+0000 7f3a89dd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a801a9800 con 0x5636b6efe3e0 2024-03-21T11:30:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.885+0000 7f3a89dd8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a801a84e0 con 0x5636b6efd9f0 2024-03-21T11:30:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.885+0000 7f3a7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f0f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.885+0000 7f3a7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f0f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52076/0 (socket says 172.21.15.47:52076) 2024-03-21T11:30:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.885+0000 7f3a7fe00640 1 -- 172.21.15.47:0/378558501 learned_addr learned my addr 172.21.15.47:0/378558501 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.885+0000 7f3a7f400640 1 --2- 172.21.15.47:0/378558501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 0x5636b6ef40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.885+0000 7f3a7f400640 1 --2- 172.21.15.47:0/378558501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6f03550 0x5636b6f1a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.886+0000 7f3a7fe00640 1 -- 172.21.15.47:0/378558501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 msgr2=0x5636b6ef40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.886+0000 7f3a7fe00640 1 --2- 172.21.15.47:0/378558501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 0x5636b6ef40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.886+0000 7f3a7fe00640 1 -- 172.21.15.47:0/378558501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6f03550 msgr2=0x5636b6f1a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.886+0000 7f3a7fe00640 1 --2- 172.21.15.47:0/378558501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6f03550 0x5636b6f1a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.886+0000 7f3a7fe00640 1 -- 172.21.15.47:0/378558501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636b6f03270 con 0x5636b6efe3e0 2024-03-21T11:30:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.886+0000 7f3a7fe00640 1 --2- 172.21.15.47:0/378558501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f0f090 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3a700049a0 tx=0x7f3a70004d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a70d7cb2326ef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.886+0000 7f3a74c00640 1 -- 172.21.15.47:0/378558501 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70030070 con 0x5636b6efe3e0 2024-03-21T11:30:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.886+0000 7f3a74c00640 1 -- 172.21.15.47:0/378558501 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a70004fc0 con 0x5636b6efe3e0 2024-03-21T11:30:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.886+0000 7f3a74c00640 1 -- 172.21.15.47:0/378558501 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70027070 con 0x5636b6efe3e0 2024-03-21T11:30:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.887+0000 7f3a89dd8740 1 -- 172.21.15.47:0/378558501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 msgr2=0x5636b6f0f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.887+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/378558501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f0f090 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3a700049a0 tx=0x7f3a70004d20 comp rx=0 tx=0).stop 2024-03-21T11:30:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.887+0000 7f3a89dd8740 1 -- 172.21.15.47:0/378558501 shutdown_connections 2024-03-21T11:30:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.887+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/378558501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6f03550 0x5636b6f1a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.887+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/378558501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6efe3e0 0x5636b6f0f090 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.887+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/378558501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 0x5636b6ef40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.887+0000 7f3a89dd8740 1 -- 172.21.15.47:0/378558501 >> 172.21.15.47:0/378558501 conn(0x5636b6ef0750 msgr2=0x5636b6ef39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.888+0000 7f3a89dd8740 1 -- 172.21.15.47:0/378558501 shutdown_connections 2024-03-21T11:30:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.888+0000 7f3a89dd8740 1 -- 172.21.15.47:0/378558501 wait complete. 2024-03-21T11:30:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.888+0000 7f3a89dd8740 1 -- start start 2024-03-21T11:30:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.888+0000 7f3a89dd8740 1 Processor -- start 2024-03-21T11:30:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a89dd8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 0x5636b6f0ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a89dd8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6efe3e0 0x5636b6f0cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a89dd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6f03550 0x5636b6f0df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a89dd8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a801a91a0 con 0x5636b6efe3e0 2024-03-21T11:30:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a89dd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a801a8b40 con 0x5636b6f03550 2024-03-21T11:30:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a89dd8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a801a6040 con 0x5636b6efd9f0 2024-03-21T11:30:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a89dd8740 10 client.0 _ll_register_callbacks cb 0x7f3a70009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a7f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 0x5636b6f0ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a7fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6efe3e0 0x5636b6f0cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a7f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 0x5636b6f0ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59764/0 (socket says 172.21.15.47:59764) 2024-03-21T11:30:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a7f400640 1 -- 172.21.15.47:0/3353769436 learned_addr learned my addr 172.21.15.47:0/3353769436 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a7f400640 1 --2- 172.21.15.47:0/3353769436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6f03550 0x5636b6f0df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a7f400640 1 -- 172.21.15.47:0/3353769436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6f03550 msgr2=0x5636b6f0df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a7f400640 1 --2- 172.21.15.47:0/3353769436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6f03550 0x5636b6f0df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a7f400640 1 -- 172.21.15.47:0/3353769436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6efe3e0 msgr2=0x5636b6f0cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a7f400640 1 --2- 172.21.15.47:0/3353769436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6efe3e0 0x5636b6f0cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.889+0000 7f3a7f400640 1 -- 172.21.15.47:0/3353769436 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636b6f0e580 con 0x5636b6efd9f0 2024-03-21T11:30:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.890+0000 7f3a7f400640 1 --2- 172.21.15.47:0/3353769436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 0x5636b6f0ca90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3a680059d0 tx=0x7f3a68045040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9788f84464a53848 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.890+0000 7f3a6e000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.890+0000 7f3a6e000640 1 -- 172.21.15.47:0/3353769436 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6802cdf0 con 0x5636b6efd9f0 2024-03-21T11:30:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.890+0000 7f3a89dd8740 1 -- 172.21.15.47:0/3353769436 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a70004fc0 con 0x5636b6efd9f0 2024-03-21T11:30:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.890+0000 7f3a6e000640 1 -- 172.21.15.47:0/3353769436 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3a68004050 con 0x5636b6efd9f0 2024-03-21T11:30:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.890+0000 7f3a6e000640 1 -- 172.21.15.47:0/3353769436 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a6805c050 con 0x5636b6efd9f0 2024-03-21T11:30:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.890+0000 7f3a89dd8740 10 client.27739 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.890+0000 7f3a89dd8740 1 -- 172.21.15.47:0/3353769436 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5636b6f6f340 con 0x5636b6efd9f0 2024-03-21T11:30:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.891+0000 7f3a6e000640 1 -- 172.21.15.47:0/3353769436 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a70004fc0 con 0x5636b6efd9f0 2024-03-21T11:30:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.891+0000 7f3a6e000640 1 -- 172.21.15.47:0/3353769436 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3a680035f0 con 0x5636b6efd9f0 2024-03-21T11:30:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.891+0000 7f3a89dd8740 10 client.27739 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.891+0000 7f3a89dd8740 10 client.27739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.891+0000 7f3a89dd8740 20 client.27739 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.891+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/3353769436 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5636b6f6fc40 0x5636b6f72060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.891+0000 7f3a89dd8740 4 client.27739 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:30:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.891+0000 7f3a89dd8740 1 -- 172.21.15.47:0/3353769436 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5636b6e620c0 con 0x5636b6f6fc40 2024-03-21T11:30:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.891+0000 7f3a7fe00640 1 --2- 172.21.15.47:0/3353769436 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5636b6f6fc40 0x5636b6f72060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.892+0000 7f3a7fe00640 1 --2- 172.21.15.47:0/3353769436 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5636b6f6fc40 0x5636b6f72060 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48fead35ec420515 server_cookie=1e051247c5d28142 in_seq=0 out_seq=0 2024-03-21T11:30:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:03.892+0000 7f3a6e000640 10 client.27739 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:03.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.896+0000 7f1e649e9740 1 Processor -- start 2024-03-21T11:30:03.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.896+0000 7f1e649e9740 1 -- start start 2024-03-21T11:30:03.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e649e9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 0x55dba7faed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e649e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fc5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e649e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fb8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e649e9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c1b0780 con 0x55dba7fb4810 2024-03-21T11:30:03.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e649e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c1b2100 con 0x55dba7fb59e0 2024-03-21T11:30:03.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e649e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c1b0de0 con 0x55dba7fb63a0 2024-03-21T11:30:03.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e52000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fc5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e5aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 0x55dba7faed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e52000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fc5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52092/0 (socket says 172.21.15.47:52092) 2024-03-21T11:30:03.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e52000640 1 -- 172.21.15.47:0/1988833893 learned_addr learned my addr 172.21.15.47:0/1988833893 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e5b400640 1 --2- 172.21.15.47:0/1988833893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fb8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e52000640 1 -- 172.21.15.47:0/1988833893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 msgr2=0x55dba7fb8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e52000640 1 --2- 172.21.15.47:0/1988833893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fb8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e52000640 1 -- 172.21.15.47:0/1988833893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 msgr2=0x55dba7faed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e52000640 1 --2- 172.21.15.47:0/1988833893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 0x55dba7faed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e52000640 1 -- 172.21.15.47:0/1988833893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dba7fbd600 con 0x55dba7fb59e0 2024-03-21T11:30:03.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e5aa00640 1 --2- 172.21.15.47:0/1988833893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 0x55dba7faed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:03.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.897+0000 7f1e5b400640 1 --2- 172.21.15.47:0/1988833893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fb8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:03.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.898+0000 7f1e52000640 1 --2- 172.21.15.47:0/1988833893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fc5f70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1e4402c7b0 tx=0x7f1e440092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5af352c8cf42af server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.898+0000 7f1e4b400640 1 -- 172.21.15.47:0/1988833893 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4402ca40 con 0x55dba7fb59e0 2024-03-21T11:30:03.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.898+0000 7f1e4b400640 1 -- 172.21.15.47:0/1988833893 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e4402cbe0 con 0x55dba7fb59e0 2024-03-21T11:30:03.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.898+0000 7f1e4b400640 1 -- 172.21.15.47:0/1988833893 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44004eb0 con 0x55dba7fb59e0 2024-03-21T11:30:03.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.899+0000 7f1e649e9740 1 -- 172.21.15.47:0/1988833893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 msgr2=0x55dba7fc5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.899+0000 7f1e649e9740 1 --2- 172.21.15.47:0/1988833893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fc5f70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1e4402c7b0 tx=0x7f1e440092d0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.899+0000 7f1e649e9740 1 -- 172.21.15.47:0/1988833893 shutdown_connections 2024-03-21T11:30:03.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.899+0000 7f1e649e9740 1 --2- 172.21.15.47:0/1988833893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fb8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.899+0000 7f1e649e9740 1 --2- 172.21.15.47:0/1988833893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fc5f70 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.899+0000 7f1e649e9740 1 --2- 172.21.15.47:0/1988833893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 0x55dba7faed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.899+0000 7f1e649e9740 1 -- 172.21.15.47:0/1988833893 >> 172.21.15.47:0/1988833893 conn(0x55dba7fafc00 msgr2=0x55dba7fae620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.899+0000 7f1e649e9740 1 -- 172.21.15.47:0/1988833893 shutdown_connections 2024-03-21T11:30:03.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.899+0000 7f1e649e9740 1 -- 172.21.15.47:0/1988833893 wait complete. 2024-03-21T11:30:03.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.899+0000 7f1e649e9740 1 -- start start 2024-03-21T11:30:03.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.900+0000 7f1e649e9740 1 Processor -- start 2024-03-21T11:30:03.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.900+0000 7f1e649e9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 0x55dba7fc1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.900+0000 7f1e649e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fc1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.900+0000 7f1e649e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fc4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.900+0000 7f1e649e9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c1b1aa0 con 0x55dba7fb4810 2024-03-21T11:30:03.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.900+0000 7f1e649e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c1b1440 con 0x55dba7fb59e0 2024-03-21T11:30:03.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.900+0000 7f1e649e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c1ae940 con 0x55dba7fb63a0 2024-03-21T11:30:03.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.900+0000 7f1e649e9740 10 client.0 _ll_register_callbacks cb 0x7f1e54014d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:03.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.900+0000 7f1e52000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fc1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.900+0000 7f1e52000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fc1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52094/0 (socket says 172.21.15.47:52094) 2024-03-21T11:30:03.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.900+0000 7f1e5aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 0x55dba7fc1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.900+0000 7f1e52000640 1 -- 172.21.15.47:0/2150611661 learned_addr learned my addr 172.21.15.47:0/2150611661 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:03.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.900+0000 7f1e5b400640 1 --2- 172.21.15.47:0/2150611661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fc4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.901+0000 7f1e52000640 1 -- 172.21.15.47:0/2150611661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 msgr2=0x55dba7fc4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.901+0000 7f1e52000640 1 --2- 172.21.15.47:0/2150611661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fc4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.901+0000 7f1e52000640 1 -- 172.21.15.47:0/2150611661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 msgr2=0x55dba7fc1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:03.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.901+0000 7f1e52000640 1 --2- 172.21.15.47:0/2150611661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 0x55dba7fc1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.901+0000 7f1e52000640 1 -- 172.21.15.47:0/2150611661 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dba7fc53b0 con 0x55dba7fb59e0 2024-03-21T11:30:03.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.901+0000 7f1e52000640 1 --2- 172.21.15.47:0/2150611661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fc1da0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1e4402c240 tx=0x7f1e440095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a18c13aa60906009 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:03.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.901+0000 7f1e48c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.901+0000 7f1e48c00640 1 -- 172.21.15.47:0/2150611661 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4402ca40 con 0x55dba7fb59e0 2024-03-21T11:30:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.901+0000 7f1e649e9740 1 -- 172.21.15.47:0/2150611661 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e4402cbe0 con 0x55dba7fb59e0 2024-03-21T11:30:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.901+0000 7f1e48c00640 1 -- 172.21.15.47:0/2150611661 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1e440050b0 con 0x55dba7fb59e0 2024-03-21T11:30:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.901+0000 7f1e48c00640 1 -- 172.21.15.47:0/2150611661 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e44004ca0 con 0x55dba7fb59e0 2024-03-21T11:30:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.901+0000 7f1e649e9740 10 client.27725 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.901+0000 7f1e649e9740 1 -- 172.21.15.47:0/2150611661 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dba8026080 con 0x55dba7fb59e0 2024-03-21T11:30:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.902+0000 7f1e48c00640 1 -- 172.21.15.47:0/2150611661 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e4402cbe0 con 0x55dba7fb59e0 2024-03-21T11:30:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.902+0000 7f1e48c00640 1 -- 172.21.15.47:0/2150611661 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1e44047070 con 0x55dba7fb59e0 2024-03-21T11:30:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.902+0000 7f1e649e9740 10 client.27725 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.902+0000 7f1e649e9740 10 client.27725 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:30:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.902+0000 7f1e649e9740 20 client.27725 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.902+0000 7f1e649e9740 1 --2- 172.21.15.47:0/2150611661 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dba7fca570 0x55dba80289f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.902+0000 7f1e649e9740 4 client.27725 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:30:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.902+0000 7f1e649e9740 1 -- 172.21.15.47:0/2150611661 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dba7f20b50 con 0x55dba7fca570 2024-03-21T11:30:03.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.902+0000 7f1e5aa00640 1 --2- 172.21.15.47:0/2150611661 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dba7fca570 0x55dba80289f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:03.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e5aa00640 1 --2- 172.21.15.47:0/2150611661 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dba7fca570 0x55dba80289f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=981aa8a39c02d153 server_cookie=6187eeab406e2ab7 in_seq=0 out_seq=0 2024-03-21T11:30:03.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e48c00640 10 client.27725 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:30:03.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e48c00640 1 -- 172.21.15.47:0/2150611661 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dba7f20b50 con 0x55dba7fca570 2024-03-21T11:30:03.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e48c00640 10 client.27725 handle_command_reply: tid=0 2024-03-21T11:30:03.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e649e9740 1 client.27725 shutdown 2024-03-21T11:30:03.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e649e9740 1 -- 172.21.15.47:0/2150611661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 msgr2=0x55dba7fc1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e649e9740 1 --2- 172.21.15.47:0/2150611661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fc1da0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1e4402c240 tx=0x7f1e440095d0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e649e9740 1 -- 172.21.15.47:0/2150611661 shutdown_connections 2024-03-21T11:30:03.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e649e9740 1 --2- 172.21.15.47:0/2150611661 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55dba7fca570 0x55dba80289f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e649e9740 1 --2- 172.21.15.47:0/2150611661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dba7fb63a0 0x55dba7fc4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e649e9740 1 --2- 172.21.15.47:0/2150611661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dba7fb59e0 0x55dba7fc1da0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e48c00640 0 client.27725 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:30:03.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e649e9740 1 --2- 172.21.15.47:0/2150611661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dba7fb4810 0x55dba7fc1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e649e9740 1 -- 172.21.15.47:0/2150611661 >> 172.21.15.47:0/2150611661 conn(0x55dba7fafc00 msgr2=0x55dba7faeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e649e9740 1 -- 172.21.15.47:0/2150611661 shutdown_connections 2024-03-21T11:30:03.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e649e9740 1 -- 172.21.15.47:0/2150611661 wait complete. 2024-03-21T11:30:03.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.903+0000 7f1e649e9740 20 client.27725 trim_cache size 0 max 16384 2024-03-21T11:30:03.909 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:30:03.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.904+0000 7f1e61c00640 1 -- 172.21.15.47:0/206910640 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1e400a2fa0 msgr2=0x7f1e400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.904+0000 7f1e61c00640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1e400a2fa0 0x7f1e400a5460 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1e54005fd0 tx=0x7f1e54009e70 comp rx=0 tx=0).stop 2024-03-21T11:30:03.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.904+0000 7f1e61c00640 1 -- 172.21.15.47:0/206910640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e5c076060 msgr2=0x7f1e5c19e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:03.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.904+0000 7f1e61c00640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e5c076060 0x7f1e5c19e010 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00e9f0 tx=0x7f1e4c00eeb0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.904+0000 7f1e61c00640 1 -- 172.21.15.47:0/206910640 shutdown_connections 2024-03-21T11:30:03.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.904+0000 7f1e61c00640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1e400a2fa0 0x7f1e400a5460 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.904+0000 7f1e61c00640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e5c077c40 0x7f1e5c1a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.904+0000 7f1e61c00640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e5c0772a0 0x7f1e5c19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.904+0000 7f1e61c00640 1 --2- 172.21.15.47:0/206910640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e5c076060 0x7f1e5c19e010 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:03.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.904+0000 7f1e61c00640 1 -- 172.21.15.47:0/206910640 >> 172.21.15.47:0/206910640 conn(0x7f1e5c0fee50 msgr2=0x7f1e5c100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:03.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.904+0000 7f1e61c00640 1 -- 172.21.15.47:0/206910640 shutdown_connections 2024-03-21T11:30:03.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:03.904+0000 7f1e61c00640 1 -- 172.21.15.47:0/206910640 wait complete. 2024-03-21T11:30:03.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:03 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1727268342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:30:03.922 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:30:03.922 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:30:04.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:03 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1727268342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:30:04.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:03 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1727268342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:30:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.245+0000 7f3a6e000640 1 -- 172.21.15.47:0/3353769436 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5636b6e620c0 con 0x5636b6f6fc40 2024-03-21T11:30:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.245+0000 7f3a6e000640 10 client.27739 handle_command_reply: tid=0 2024-03-21T11:30:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.245+0000 7f3a89dd8740 1 client.27739 shutdown 2024-03-21T11:30:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.246+0000 7f3a89dd8740 1 -- 172.21.15.47:0/3353769436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 msgr2=0x5636b6f0ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.246+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/3353769436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 0x5636b6f0ca90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3a680059d0 tx=0x7f3a68045040 comp rx=0 tx=0).stop 2024-03-21T11:30:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.246+0000 7f3a89dd8740 1 -- 172.21.15.47:0/3353769436 shutdown_connections 2024-03-21T11:30:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.246+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/3353769436 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5636b6f6fc40 0x5636b6f72060 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.246+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/3353769436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636b6f03550 0x5636b6f0df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.246+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/3353769436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5636b6efe3e0 0x5636b6f0cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.246+0000 7f3a89dd8740 1 --2- 172.21.15.47:0/3353769436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636b6efd9f0 0x5636b6f0ca90 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.246+0000 7f3a89dd8740 1 -- 172.21.15.47:0/3353769436 >> 172.21.15.47:0/3353769436 conn(0x5636b6ef0750 msgr2=0x5636b6ef4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.247+0000 7f3a6e000640 0 client.27739 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.247+0000 7f3a89dd8740 1 -- 172.21.15.47:0/3353769436 shutdown_connections 2024-03-21T11:30:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.247+0000 7f3a89dd8740 1 -- 172.21.15.47:0/3353769436 wait complete. 2024-03-21T11:30:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.248+0000 7f3a89dd8740 20 client.27739 trim_cache size 0 max 16384 2024-03-21T11:30:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:30:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:30:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:30:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:30:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.250+0000 7f3a87000640 1 -- 172.21.15.47:0/1709692943 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a640a3040 msgr2=0x7f3a640a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.250+0000 7f3a87000640 1 --2- 172.21.15.47:0/1709692943 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a640a3040 0x7f3a640a5500 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3a780058f0 tx=0x7f3a780054b0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.250+0000 7f3a87000640 1 -- 172.21.15.47:0/1709692943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a80109580 msgr2=0x7f3a801a10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.251+0000 7f3a87000640 1 --2- 172.21.15.47:0/1709692943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a80109580 0x7f3a801a10b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009fd0 tx=0x7f3a7800ef10 comp rx=0 tx=0).stop 2024-03-21T11:30:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.251+0000 7f3a87000640 1 -- 172.21.15.47:0/1709692943 shutdown_connections 2024-03-21T11:30:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.251+0000 7f3a87000640 1 --2- 172.21.15.47:0/1709692943 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a640a3040 0x7f3a640a5500 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.251+0000 7f3a87000640 1 --2- 172.21.15.47:0/1709692943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a80109580 0x7f3a801a10b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.251+0000 7f3a87000640 1 --2- 172.21.15.47:0/1709692943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a80075660 0x7f3a8019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.251+0000 7f3a87000640 1 --2- 172.21.15.47:0/1709692943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a80074080 0x7f3a80199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.251+0000 7f3a87000640 1 -- 172.21.15.47:0/1709692943 >> 172.21.15.47:0/1709692943 conn(0x7f3a800fcc60 msgr2=0x7f3a80105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.252+0000 7f3a87000640 1 -- 172.21.15.47:0/1709692943 shutdown_connections 2024-03-21T11:30:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:04.252+0000 7f3a87000640 1 -- 172.21.15.47:0/1709692943 wait complete. 2024-03-21T11:30:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.259+0000 7f1983400640 1 -- 172.21.15.47:0/952236452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f197c104ad0 msgr2=0x7f197c104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.259+0000 7f1983400640 1 --2- 172.21.15.47:0/952236452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f197c104ad0 0x7f197c104f50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1968009900 tx=0x7f196802bc40 comp rx=0 tx=0).stop 2024-03-21T11:30:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.259+0000 7f1983400640 1 -- 172.21.15.47:0/952236452 shutdown_connections 2024-03-21T11:30:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.259+0000 7f1983400640 1 --2- 172.21.15.47:0/952236452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f197c105490 0x7f197c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.259+0000 7f1983400640 1 --2- 172.21.15.47:0/952236452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f197c104ad0 0x7f197c104f50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.259+0000 7f1983400640 1 --2- 172.21.15.47:0/952236452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f197c1038e0 0x7f197c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.259+0000 7f1983400640 1 -- 172.21.15.47:0/952236452 >> 172.21.15.47:0/952236452 conn(0x7f197c0fee50 msgr2=0x7f197c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.259+0000 7f1983400640 1 -- 172.21.15.47:0/952236452 shutdown_connections 2024-03-21T11:30:04.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.259+0000 7f1983400640 1 -- 172.21.15.47:0/952236452 wait complete. 2024-03-21T11:30:04.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.261+0000 7f1983400640 1 Processor -- start 2024-03-21T11:30:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.261+0000 7f1983400640 1 -- start start 2024-03-21T11:30:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.261+0000 7f1983400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f197c1038e0 0x7f197c199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.261+0000 7f1983400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f197c104ad0 0x7f197c19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.261+0000 7f1983400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f197c105490 0x7f197c1a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.261+0000 7f1983400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c10f5b0 con 0x7f197c104ad0 2024-03-21T11:30:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.261+0000 7f1983400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c10f430 con 0x7f197c105490 2024-03-21T11:30:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.261+0000 7f1983400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c10f730 con 0x7f197c1038e0 2024-03-21T11:30:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.262+0000 7f1981400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f197c105490 0x7f197c1a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.262+0000 7f1980a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f197c1038e0 0x7f197c199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.262+0000 7f1981400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f197c105490 0x7f197c1a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52122/0 (socket says 172.21.15.47:52122) 2024-03-21T11:30:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.262+0000 7f1981400640 1 -- 172.21.15.47:0/3405229098 learned_addr learned my addr 172.21.15.47:0/3405229098 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.262+0000 7f197be00640 1 --2- 172.21.15.47:0/3405229098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f197c104ad0 0x7f197c19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.262+0000 7f1980a00640 1 -- 172.21.15.47:0/3405229098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f197c105490 msgr2=0x7f197c1a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.262+0000 7f1980a00640 1 --2- 172.21.15.47:0/3405229098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f197c105490 0x7f197c1a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.262+0000 7f1980a00640 1 -- 172.21.15.47:0/3405229098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f197c104ad0 msgr2=0x7f197c19a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.262+0000 7f1980a00640 1 --2- 172.21.15.47:0/3405229098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f197c104ad0 0x7f197c19a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.262+0000 7f1980a00640 1 -- 172.21.15.47:0/3405229098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c1a17f0 con 0x7f197c1038e0 2024-03-21T11:30:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.262+0000 7f1980a00640 1 --2- 172.21.15.47:0/3405229098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f197c1038e0 0x7f197c199bd0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f197000e9c0 tx=0x7f197000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.263+0000 7f1979600640 1 -- 172.21.15.47:0/3405229098 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197000ccc0 con 0x7f197c1038e0 2024-03-21T11:30:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.263+0000 7f1979600640 1 -- 172.21.15.47:0/3405229098 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19700035f0 con 0x7f197c1038e0 2024-03-21T11:30:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.263+0000 7f1979600640 1 -- 172.21.15.47:0/3405229098 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197000ce60 con 0x7f197c1038e0 2024-03-21T11:30:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.263+0000 7f1983400640 1 -- 172.21.15.47:0/3405229098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c1a1ae0 con 0x7f197c1038e0 2024-03-21T11:30:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.263+0000 7f1983400640 1 -- 172.21.15.47:0/3405229098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c1a1fb0 con 0x7f197c1038e0 2024-03-21T11:30:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.266+0000 7f1979600640 1 -- 172.21.15.47:0/3405229098 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f19700037a0 con 0x7f197c1038e0 2024-03-21T11:30:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.266+0000 7f1986030740 1 Processor -- start 2024-03-21T11:30:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.267+0000 7f1986030740 1 -- start start 2024-03-21T11:30:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.267+0000 7f1986030740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 0x55926c07fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.267+0000 7f1986030740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0809e0 0x55926c080e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.267+0000 7f1986030740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c087c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.267+0000 7f1986030740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55926bfe42c0 con 0x55926c07f810 2024-03-21T11:30:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.267+0000 7f1986030740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55926bf8cc90 con 0x55926c0809e0 2024-03-21T11:30:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.267+0000 7f1986030740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55926bfe0360 con 0x55926c0813a0 2024-03-21T11:30:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.267+0000 7f1980a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0809e0 0x55926c080e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.268+0000 7f197be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c087c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.268+0000 7f1981400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 0x55926c07fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.268+0000 7f1980a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0809e0 0x55926c080e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52124/0 (socket says 172.21.15.47:52124) 2024-03-21T11:30:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.268+0000 7f1980a00640 1 -- 172.21.15.47:0/1403296250 learned_addr learned my addr 172.21.15.47:0/1403296250 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.268+0000 7f1980a00640 1 -- 172.21.15.47:0/1403296250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 msgr2=0x55926c087c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.268+0000 7f1980a00640 1 --2- 172.21.15.47:0/1403296250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c087c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.268+0000 7f1980a00640 1 -- 172.21.15.47:0/1403296250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 msgr2=0x55926c07fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.268+0000 7f1980a00640 1 --2- 172.21.15.47:0/1403296250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 0x55926c07fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.268+0000 7f1980a00640 1 -- 172.21.15.47:0/1403296250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55926bfb1080 con 0x55926c0809e0 2024-03-21T11:30:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.268+0000 7f197be00640 1 --2- 172.21.15.47:0/1403296250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c087c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.268+0000 7f1980a00640 1 --2- 172.21.15.47:0/1403296250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0809e0 0x55926c080e60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f19700d3a40 tx=0x7f19700d3a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d90a844d1edbbfd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.269+0000 7f196dc00640 1 -- 172.21.15.47:0/1403296250 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19700e3070 con 0x55926c0809e0 2024-03-21T11:30:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.269+0000 7f196dc00640 1 -- 172.21.15.47:0/1403296250 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19700d8cb0 con 0x55926c0809e0 2024-03-21T11:30:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.269+0000 7f196dc00640 1 -- 172.21.15.47:0/1403296250 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19700dee40 con 0x55926c0809e0 2024-03-21T11:30:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.269+0000 7f1979600640 1 --2- 172.21.15.47:0/3405229098 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f19640a2fa0 0x7f19640a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.269+0000 7f1979600640 1 -- 172.21.15.47:0/3405229098 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f19700cfaa0 con 0x7f197c1038e0 2024-03-21T11:30:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.269+0000 7f1981400640 1 --2- 172.21.15.47:0/3405229098 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f19640a2fa0 0x7f19640a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.270+0000 7f1986030740 1 -- 172.21.15.47:0/1403296250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0809e0 msgr2=0x55926c080e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.270+0000 7f1986030740 1 --2- 172.21.15.47:0/1403296250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0809e0 0x55926c080e60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f19700d3a40 tx=0x7f19700d3a70 comp rx=0 tx=0).stop 2024-03-21T11:30:04.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.270+0000 7f1986030740 1 -- 172.21.15.47:0/1403296250 shutdown_connections 2024-03-21T11:30:04.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.270+0000 7f1986030740 1 --2- 172.21.15.47:0/1403296250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c087c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.270+0000 7f1986030740 1 --2- 172.21.15.47:0/1403296250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0809e0 0x55926c080e60 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.270+0000 7f1986030740 1 --2- 172.21.15.47:0/1403296250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 0x55926c07fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.270+0000 7f1986030740 1 -- 172.21.15.47:0/1403296250 >> 172.21.15.47:0/1403296250 conn(0x55926c07ac00 msgr2=0x55926c07b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.270+0000 7f1986030740 1 -- 172.21.15.47:0/1403296250 shutdown_connections 2024-03-21T11:30:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.270+0000 7f1981400640 1 --2- 172.21.15.47:0/3405229098 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f19640a2fa0 0x7f19640a5460 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55926c0808e0 tx=0x7f1974009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.270+0000 7f1986030740 1 -- 172.21.15.47:0/1403296250 wait complete. 2024-03-21T11:30:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.271+0000 7f1986030740 1 -- start start 2024-03-21T11:30:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.271+0000 7f1986030740 1 Processor -- start 2024-03-21T11:30:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.271+0000 7f1986030740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 0x55926c089a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.271+0000 7f1986030740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0809e0 0x55926c089fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.271+0000 7f1986030740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c090fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.271+0000 7f1986030740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55926c0883b0 con 0x55926c07f810 2024-03-21T11:30:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f1986030740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55926bfa00d0 con 0x55926c0809e0 2024-03-21T11:30:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f1986030740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55926c088530 con 0x55926c0813a0 2024-03-21T11:30:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f1986030740 10 client.0 _ll_register_callbacks cb 0x55926bfa4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f197be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0809e0 0x55926c089fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f1981400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c090fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f1981400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c090fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59812/0 (socket says 172.21.15.47:59812) 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f1981400640 1 -- 172.21.15.47:0/1122956060 learned_addr learned my addr 172.21.15.47:0/1122956060 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f197be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0809e0 0x55926c089fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52138/0 (socket says 172.21.15.47:52138) 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f1980a00640 1 --2- 172.21.15.47:0/1122956060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 0x55926c089a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f1981400640 1 -- 172.21.15.47:0/1122956060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0809e0 msgr2=0x55926c089fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f1981400640 1 --2- 172.21.15.47:0/1122956060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0809e0 0x55926c089fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f1981400640 1 -- 172.21.15.47:0/1122956060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 msgr2=0x55926c089a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f1981400640 1 --2- 172.21.15.47:0/1122956060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 0x55926c089a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f1981400640 1 -- 172.21.15.47:0/1122956060 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55926bcf83f0 con 0x55926c0813a0 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f1980a00640 1 --2- 172.21.15.47:0/1122956060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 0x55926c089a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.272+0000 7f197be00640 1 --2- 172.21.15.47:0/1122956060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0809e0 0x55926c089fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.273+0000 7f1981400640 1 --2- 172.21.15.47:0/1122956060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c090fe0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f197400fb50 tx=0x7f1974004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1130b65ae5659a97 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.273+0000 7f1962000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.273+0000 7f1962000640 1 -- 172.21.15.47:0/1122956060 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197402e060 con 0x55926c0813a0 2024-03-21T11:30:04.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.273+0000 7f1986030740 1 -- 172.21.15.47:0/1122956060 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19700d8cb0 con 0x55926c0813a0 2024-03-21T11:30:04.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.273+0000 7f1962000640 1 -- 172.21.15.47:0/1122956060 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1974018030 con 0x55926c0813a0 2024-03-21T11:30:04.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.273+0000 7f1962000640 1 -- 172.21.15.47:0/1122956060 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f197400fdc0 con 0x55926c0813a0 2024-03-21T11:30:04.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.273+0000 7f1986030740 10 client.27757 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:04.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.273+0000 7f1986030740 1 -- 172.21.15.47:0/1122956060 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55926bdc27d0 con 0x55926c0813a0 2024-03-21T11:30:04.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.274+0000 7f1962000640 1 -- 172.21.15.47:0/1122956060 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19700d8cb0 con 0x55926c0813a0 2024-03-21T11:30:04.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.274+0000 7f1962000640 1 -- 172.21.15.47:0/1122956060 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1974005220 con 0x55926c0813a0 2024-03-21T11:30:04.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.274+0000 7f1986030740 10 client.27757 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.274+0000 7f1986030740 10 client.27757 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:30:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.274+0000 7f1986030740 20 client.27757 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.274+0000 7f1986030740 1 --2- 172.21.15.47:0/1122956060 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55926c095530 0x55926c097980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.274+0000 7f1986030740 4 client.27757 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.274+0000 7f1986030740 1 -- 172.21.15.47:0/1122956060 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55926bff1800 con 0x55926c095530 2024-03-21T11:30:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.275+0000 7f1980a00640 1 --2- 172.21.15.47:0/1122956060 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55926c095530 0x55926c097980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.275+0000 7f1980a00640 1 --2- 172.21.15.47:0/1122956060 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55926c095530 0x55926c097980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=daabdb2aa6d7cc39 server_cookie=a7fe740544eed1f4 in_seq=0 out_seq=0 2024-03-21T11:30:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.275+0000 7f1962000640 10 client.27757 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:30:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.276+0000 7f1962000640 1 -- 172.21.15.47:0/1122956060 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55926bff1800 con 0x55926c095530 2024-03-21T11:30:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.276+0000 7f1962000640 10 client.27757 handle_command_reply: tid=0 2024-03-21T11:30:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.276+0000 7f1986030740 1 client.27757 shutdown 2024-03-21T11:30:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.276+0000 7f1986030740 1 -- 172.21.15.47:0/1122956060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 msgr2=0x55926c090fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.276+0000 7f1986030740 1 --2- 172.21.15.47:0/1122956060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c090fe0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f197400fb50 tx=0x7f1974004ab0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.276+0000 7f1986030740 1 -- 172.21.15.47:0/1122956060 shutdown_connections 2024-03-21T11:30:04.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.276+0000 7f1986030740 1 --2- 172.21.15.47:0/1122956060 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55926c095530 0x55926c097980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.276+0000 7f1986030740 1 --2- 172.21.15.47:0/1122956060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c090fe0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.276+0000 7f1986030740 1 --2- 172.21.15.47:0/1122956060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0809e0 0x55926c089fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.276+0000 7f1986030740 1 --2- 172.21.15.47:0/1122956060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 0x55926c089a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.276+0000 7f1986030740 1 -- 172.21.15.47:0/1122956060 >> 172.21.15.47:0/1122956060 conn(0x55926c07ac00 msgr2=0x55926c079eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.276+0000 7f1962000640 0 client.27757 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:30:04.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.277+0000 7f1986030740 1 -- 172.21.15.47:0/1122956060 shutdown_connections 2024-03-21T11:30:04.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.277+0000 7f1986030740 1 -- 172.21.15.47:0/1122956060 wait complete. 2024-03-21T11:30:04.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.277+0000 7f1986030740 20 client.27757 trim_cache size 0 max 16384 2024-03-21T11:30:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.292+0000 7f1986030740 1 Processor -- start 2024-03-21T11:30:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.292+0000 7f1986030740 1 -- start start 2024-03-21T11:30:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.292+0000 7f1986030740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 0x55926c087900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.292+0000 7f1986030740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0809e0 0x55926c090f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.292+0000 7f1986030740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0813a0 0x55926c083a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.292+0000 7f1986030740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c1ac2f0 con 0x55926c07f810 2024-03-21T11:30:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.292+0000 7f1986030740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c1adc70 con 0x55926c0813a0 2024-03-21T11:30:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.292+0000 7f1986030740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c1ac950 con 0x55926c0809e0 2024-03-21T11:30:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.292+0000 7f197be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0809e0 0x55926c090f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.292+0000 7f1980a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 0x55926c087900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.292+0000 7f197be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0809e0 0x55926c090f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59826/0 (socket says 172.21.15.47:59826) 2024-03-21T11:30:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.292+0000 7f197be00640 1 -- 172.21.15.47:0/3207871934 learned_addr learned my addr 172.21.15.47:0/3207871934 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.292+0000 7f1981400640 1 --2- 172.21.15.47:0/3207871934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0813a0 0x55926c083a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.293+0000 7f197be00640 1 -- 172.21.15.47:0/3207871934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0813a0 msgr2=0x55926c083a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.293+0000 7f197be00640 1 --2- 172.21.15.47:0/3207871934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0813a0 0x55926c083a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.293+0000 7f197be00640 1 -- 172.21.15.47:0/3207871934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 msgr2=0x55926c087900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.293+0000 7f197be00640 1 --2- 172.21.15.47:0/3207871934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 0x55926c087900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.293+0000 7f197be00640 1 -- 172.21.15.47:0/3207871934 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55926c08a310 con 0x55926c0809e0 2024-03-21T11:30:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.293+0000 7f197be00640 1 --2- 172.21.15.47:0/3207871934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0809e0 0x55926c090f00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1968009900 tx=0x7f196802cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a86314316008758 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.293+0000 7f196d200640 1 -- 172.21.15.47:0/3207871934 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196804d070 con 0x55926c0809e0 2024-03-21T11:30:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.293+0000 7f196d200640 1 -- 172.21.15.47:0/3207871934 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19680046c0 con 0x55926c0809e0 2024-03-21T11:30:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.293+0000 7f196d200640 1 -- 172.21.15.47:0/3207871934 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968005360 con 0x55926c0809e0 2024-03-21T11:30:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.294+0000 7f1986030740 1 -- 172.21.15.47:0/3207871934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0809e0 msgr2=0x55926c090f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.294+0000 7f1986030740 1 --2- 172.21.15.47:0/3207871934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0809e0 0x55926c090f00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1968009900 tx=0x7f196802cf90 comp rx=0 tx=0).stop 2024-03-21T11:30:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.294+0000 7f1986030740 1 -- 172.21.15.47:0/3207871934 shutdown_connections 2024-03-21T11:30:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.294+0000 7f1986030740 1 --2- 172.21.15.47:0/3207871934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c0813a0 0x55926c083a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.294+0000 7f1986030740 1 --2- 172.21.15.47:0/3207871934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0809e0 0x55926c090f00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.294+0000 7f1986030740 1 --2- 172.21.15.47:0/3207871934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c07f810 0x55926c087900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.294+0000 7f1986030740 1 -- 172.21.15.47:0/3207871934 >> 172.21.15.47:0/3207871934 conn(0x55926c07ac00 msgr2=0x55926c07e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.294+0000 7f1986030740 1 -- 172.21.15.47:0/3207871934 shutdown_connections 2024-03-21T11:30:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.294+0000 7f1986030740 1 -- 172.21.15.47:0/3207871934 wait complete. 2024-03-21T11:30:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.295+0000 7f1986030740 1 -- start start 2024-03-21T11:30:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.295+0000 7f1986030740 1 Processor -- start 2024-03-21T11:30:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.295+0000 7f1986030740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c07f810 0x55926c08cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.295+0000 7f1986030740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c0809e0 0x55926c08d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.295+0000 7f1986030740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c0902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.295+0000 7f1986030740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c1ad610 con 0x55926c0809e0 2024-03-21T11:30:04.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.295+0000 7f1986030740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c1acfb0 con 0x55926c07f810 2024-03-21T11:30:04.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.295+0000 7f1986030740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c1aa4b0 con 0x55926c0813a0 2024-03-21T11:30:04.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.295+0000 7f1986030740 10 client.0 _ll_register_callbacks cb 0x7f1974009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:04.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.296+0000 7f1980a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c07f810 0x55926c08cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.296+0000 7f1981400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c0902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.296+0000 7f1980a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c07f810 0x55926c08cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52146/0 (socket says 172.21.15.47:52146) 2024-03-21T11:30:04.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.296+0000 7f1980a00640 1 -- 172.21.15.47:0/4163575274 learned_addr learned my addr 172.21.15.47:0/4163575274 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.296+0000 7f197be00640 1 --2- 172.21.15.47:0/4163575274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c0809e0 0x55926c08d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.296+0000 7f1980a00640 1 -- 172.21.15.47:0/4163575274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 msgr2=0x55926c0902a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.296+0000 7f1980a00640 1 --2- 172.21.15.47:0/4163575274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c0902a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.296+0000 7f1980a00640 1 -- 172.21.15.47:0/4163575274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c0809e0 msgr2=0x55926c08d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.296+0000 7f1980a00640 1 --2- 172.21.15.47:0/4163575274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c0809e0 0x55926c08d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.296+0000 7f1980a00640 1 -- 172.21.15.47:0/4163575274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55926c090890 con 0x55926c07f810 2024-03-21T11:30:04.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.296+0000 7f197be00640 1 --2- 172.21.15.47:0/4163575274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c0809e0 0x55926c08d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:04.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.296+0000 7f1980a00640 1 --2- 172.21.15.47:0/4163575274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c07f810 0x55926c08cd40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f19700987b0 tx=0x7f19700de6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5317d4cb36ee31 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.296+0000 7f1962000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:04.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.296+0000 7f1962000640 1 -- 172.21.15.47:0/4163575274 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970098080 con 0x55926c07f810 2024-03-21T11:30:04.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.297+0000 7f1986030740 1 -- 172.21.15.47:0/4163575274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19680046c0 con 0x55926c07f810 2024-03-21T11:30:04.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.297+0000 7f1962000640 1 -- 172.21.15.47:0/4163575274 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f197009a0e0 con 0x55926c07f810 2024-03-21T11:30:04.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.297+0000 7f1962000640 1 -- 172.21.15.47:0/4163575274 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f197009b740 con 0x55926c07f810 2024-03-21T11:30:04.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.297+0000 7f1986030740 10 client.27743 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:04.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.297+0000 7f1986030740 1 -- 172.21.15.47:0/4163575274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55926c0f14f0 con 0x55926c07f810 2024-03-21T11:30:04.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.297+0000 7f1962000640 1 -- 172.21.15.47:0/4163575274 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19680046c0 con 0x55926c07f810 2024-03-21T11:30:04.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.297+0000 7f1962000640 1 -- 172.21.15.47:0/4163575274 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1970098450 con 0x55926c07f810 2024-03-21T11:30:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.297+0000 7f1986030740 10 client.27743 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.297+0000 7f1986030740 10 client.27743 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:30:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.297+0000 7f1986030740 20 client.27743 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.297+0000 7f1986030740 1 --2- 172.21.15.47:0/4163575274 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55926c0f1a50 0x55926c0f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.297+0000 7f1986030740 4 client.27743 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:30:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.297+0000 7f1986030740 1 -- 172.21.15.47:0/4163575274 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55926bfebb50 con 0x55926c0f1a50 2024-03-21T11:30:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.298+0000 7f197be00640 1 --2- 172.21.15.47:0/4163575274 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55926c0f1a50 0x55926c0f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.298+0000 7f197be00640 1 --2- 172.21.15.47:0/4163575274 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55926c0f1a50 0x55926c0f3e40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64a781b8cfce9852 server_cookie=ee054f46051d8285 in_seq=0 out_seq=0 2024-03-21T11:30:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.298+0000 7f1962000640 10 client.27743 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:30:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.298+0000 7f1962000640 1 -- 172.21.15.47:0/4163575274 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55926bfebb50 con 0x55926c0f1a50 2024-03-21T11:30:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.298+0000 7f1962000640 10 client.27743 handle_command_reply: tid=0 2024-03-21T11:30:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.298+0000 7f1986030740 1 client.27743 shutdown 2024-03-21T11:30:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.298+0000 7f1986030740 1 -- 172.21.15.47:0/4163575274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c07f810 msgr2=0x55926c08cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.298+0000 7f1986030740 1 --2- 172.21.15.47:0/4163575274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c07f810 0x55926c08cd40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f19700987b0 tx=0x7f19700de6e0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.299+0000 7f1986030740 1 -- 172.21.15.47:0/4163575274 shutdown_connections 2024-03-21T11:30:04.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.299+0000 7f1986030740 1 --2- 172.21.15.47:0/4163575274 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55926c0f1a50 0x55926c0f3e40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.299+0000 7f1986030740 1 --2- 172.21.15.47:0/4163575274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55926c0813a0 0x55926c0902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.299+0000 7f1962000640 0 client.27743 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:30:04.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.299+0000 7f1986030740 1 --2- 172.21.15.47:0/4163575274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55926c0809e0 0x55926c08d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.299+0000 7f1986030740 1 --2- 172.21.15.47:0/4163575274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55926c07f810 0x55926c08cd40 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.299+0000 7f1986030740 1 -- 172.21.15.47:0/4163575274 >> 172.21.15.47:0/4163575274 conn(0x55926c07ac00 msgr2=0x55926c07e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.299+0000 7f1986030740 1 -- 172.21.15.47:0/4163575274 shutdown_connections 2024-03-21T11:30:04.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.299+0000 7f1986030740 1 -- 172.21.15.47:0/4163575274 wait complete. 2024-03-21T11:30:04.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.299+0000 7f1986030740 20 client.27743 trim_cache size 0 max 16384 2024-03-21T11:30:04.305 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:30:04.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.300+0000 7f1983400640 1 -- 172.21.15.47:0/3405229098 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f19640a2fa0 msgr2=0x7f19640a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.300+0000 7f1983400640 1 --2- 172.21.15.47:0/3405229098 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f19640a2fa0 0x7f19640a5460 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55926c0808e0 tx=0x7f1974009500 comp rx=0 tx=0).stop 2024-03-21T11:30:04.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.300+0000 7f1983400640 1 -- 172.21.15.47:0/3405229098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f197c1038e0 msgr2=0x7f197c199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.300+0000 7f1983400640 1 --2- 172.21.15.47:0/3405229098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f197c1038e0 0x7f197c199bd0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f197000e9c0 tx=0x7f197000ee80 comp rx=0 tx=0).stop 2024-03-21T11:30:04.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.300+0000 7f1983400640 1 -- 172.21.15.47:0/3405229098 shutdown_connections 2024-03-21T11:30:04.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.300+0000 7f1983400640 1 --2- 172.21.15.47:0/3405229098 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f19640a2fa0 0x7f19640a5460 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.300+0000 7f1983400640 1 --2- 172.21.15.47:0/3405229098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f197c105490 0x7f197c1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.300+0000 7f1983400640 1 --2- 172.21.15.47:0/3405229098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f197c104ad0 0x7f197c19a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.300+0000 7f1983400640 1 --2- 172.21.15.47:0/3405229098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f197c1038e0 0x7f197c199bd0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.300+0000 7f1983400640 1 -- 172.21.15.47:0/3405229098 >> 172.21.15.47:0/3405229098 conn(0x7f197c0fee50 msgr2=0x7f197c107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.300+0000 7f1983400640 1 -- 172.21.15.47:0/3405229098 shutdown_connections 2024-03-21T11:30:04.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.300+0000 7f1983400640 1 -- 172.21.15.47:0/3405229098 wait complete. 2024-03-21T11:30:04.318 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:30:04.319 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:30:04.525 INFO:tasks.workunit.client.0.smithi047.stdout: 1 450813 8.20 MB/sec execute 168 sec latency 149.938 ms 2024-03-21T11:30:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.527+0000 7f22b5a00640 1 -- 172.21.15.47:0/3008894937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22b01028a0 msgr2=0x7f22b0102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.527+0000 7f22b5a00640 1 --2- 172.21.15.47:0/3008894937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22b01028a0 0x7f22b0102d20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f229c009860 tx=0x7f229c02bb70 comp rx=0 tx=0).stop 2024-03-21T11:30:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.527+0000 7f22b5a00640 1 -- 172.21.15.47:0/3008894937 shutdown_connections 2024-03-21T11:30:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.527+0000 7f22b5a00640 1 --2- 172.21.15.47:0/3008894937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b0103260 0x7f22b0109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.527+0000 7f22b5a00640 1 --2- 172.21.15.47:0/3008894937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22b01028a0 0x7f22b0102d20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.527+0000 7f22b5a00640 1 --2- 172.21.15.47:0/3008894937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b01016b0 0x7f22b0101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.527+0000 7f22b5a00640 1 -- 172.21.15.47:0/3008894937 >> 172.21.15.47:0/3008894937 conn(0x7f22b00fcc40 msgr2=0x7f22b00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.528+0000 7f22b5a00640 1 -- 172.21.15.47:0/3008894937 shutdown_connections 2024-03-21T11:30:04.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.528+0000 7f22b5a00640 1 -- 172.21.15.47:0/3008894937 wait complete. 2024-03-21T11:30:04.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.529+0000 7f22b5a00640 1 Processor -- start 2024-03-21T11:30:04.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.530+0000 7f22b5a00640 1 -- start start 2024-03-21T11:30:04.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.530+0000 7f22b5a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b01016b0 0x7f22b0110570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.530+0000 7f22b5a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b01028a0 0x7f22b0110ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.530+0000 7f22b5a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22b0103260 0x7f22b010d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.531+0000 7f22b5a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b0074030 con 0x7f22b01028a0 2024-03-21T11:30:04.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.531+0000 7f22b5a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b0073eb0 con 0x7f22b0103260 2024-03-21T11:30:04.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.531+0000 7f22b5a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00741b0 con 0x7f22b01016b0 2024-03-21T11:30:04.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.531+0000 7f22aea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b01016b0 0x7f22b0110570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.531+0000 7f22af400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22b0103260 0x7f22b010d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.531+0000 7f22aea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b01016b0 0x7f22b0110570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59856/0 (socket says 172.21.15.47:59856) 2024-03-21T11:30:04.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.531+0000 7f22aea00640 1 -- 172.21.15.47:0/4270935608 learned_addr learned my addr 172.21.15.47:0/4270935608 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.532+0000 7f22ae000640 1 --2- 172.21.15.47:0/4270935608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b01028a0 0x7f22b0110ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.532+0000 7f22aea00640 1 -- 172.21.15.47:0/4270935608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22b0103260 msgr2=0x7f22b010d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.532+0000 7f22aea00640 1 --2- 172.21.15.47:0/4270935608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22b0103260 0x7f22b010d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.532+0000 7f22aea00640 1 -- 172.21.15.47:0/4270935608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b01028a0 msgr2=0x7f22b0110ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.532+0000 7f22aea00640 1 --2- 172.21.15.47:0/4270935608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b01028a0 0x7f22b0110ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.532+0000 7f22aea00640 1 -- 172.21.15.47:0/4270935608 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b010d9a0 con 0x7f22b01016b0 2024-03-21T11:30:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.534+0000 7f22aea00640 1 --2- 172.21.15.47:0/4270935608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b01016b0 0x7f22b0110570 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f22a000e9c0 tx=0x7f22a000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.535+0000 7f22a7400640 1 -- 172.21.15.47:0/4270935608 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a000cd80 con 0x7f22b01016b0 2024-03-21T11:30:04.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.535+0000 7f22a7400640 1 -- 172.21.15.47:0/4270935608 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22a00035f0 con 0x7f22b01016b0 2024-03-21T11:30:04.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.535+0000 7f22a7400640 1 -- 172.21.15.47:0/4270935608 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0010690 con 0x7f22b01016b0 2024-03-21T11:30:04.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.535+0000 7f22b5a00640 1 -- 172.21.15.47:0/4270935608 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b01a6050 con 0x7f22b01016b0 2024-03-21T11:30:04.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.535+0000 7f22b5a00640 1 -- 172.21.15.47:0/4270935608 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b01a6460 con 0x7f22b01016b0 2024-03-21T11:30:04.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.542+0000 7f22a7400640 1 -- 172.21.15.47:0/4270935608 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f22a0004020 con 0x7f22b01016b0 2024-03-21T11:30:04.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.546+0000 7f22b8680740 1 Processor -- start 2024-03-21T11:30:04.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.546+0000 7f22b8680740 1 -- start start 2024-03-21T11:30:04.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.546+0000 7f22b8680740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b8810 0x55e7393b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.546+0000 7f22b8680740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393b99e0 0x55e7393b9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.546+0000 7f22b8680740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393ba3a0 0x55e7393c0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.546+0000 7f22b8680740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73931d2c0 con 0x55e7393b8810 2024-03-21T11:30:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.546+0000 7f22b8680740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7392c5c90 con 0x55e7393ba3a0 2024-03-21T11:30:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.546+0000 7f22b8680740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e739319360 con 0x55e7393b99e0 2024-03-21T11:30:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.547+0000 7f22a7400640 1 --2- 172.21.15.47:0/4270935608 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22940a3070 0x7f22940a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.547+0000 7f22af400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b8810 0x55e7393b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.547+0000 7f22af400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b8810 0x55e7393b8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48696/0 (socket says 172.21.15.47:48696) 2024-03-21T11:30:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.547+0000 7f22af400640 1 -- 172.21.15.47:0/150590196 learned_addr learned my addr 172.21.15.47:0/150590196 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.547+0000 7f22a7400640 1 -- 172.21.15.47:0/4270935608 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f22a00cfc80 con 0x7f22b01016b0 2024-03-21T11:30:04.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.547+0000 7f22ae000640 1 --2- 172.21.15.47:0/150590196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393ba3a0 0x55e7393c0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.548+0000 7f22aea00640 1 --2- 172.21.15.47:0/150590196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393b99e0 0x55e7393b9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.548+0000 7f22af400640 1 -- 172.21.15.47:0/150590196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393b99e0 msgr2=0x55e7393b9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.549+0000 7f22af400640 1 --2- 172.21.15.47:0/150590196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393b99e0 0x55e7393b9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.549+0000 7f22af400640 1 -- 172.21.15.47:0/150590196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393ba3a0 msgr2=0x55e7393c0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.549+0000 7f22af400640 1 --2- 172.21.15.47:0/150590196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393ba3a0 0x55e7393c0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.549+0000 7f22af400640 1 -- 172.21.15.47:0/150590196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7392ea080 con 0x55e7393b8810 2024-03-21T11:30:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.549+0000 7f22af400640 1 --2- 172.21.15.47:0/4270935608 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22940a3070 0x7f22940a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.549+0000 7f22af400640 1 --2- 172.21.15.47:0/150590196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b8810 0x55e7393b8c30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f22a800f5c0 tx=0x7f22a800fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da06f2ea145365c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.550+0000 7f229be00640 1 -- 172.21.15.47:0/150590196 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a801f040 con 0x55e7393b8810 2024-03-21T11:30:04.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.551+0000 7f229be00640 1 -- 172.21.15.47:0/150590196 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22a8020020 con 0x55e7393b8810 2024-03-21T11:30:04.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.551+0000 7f22af400640 1 --2- 172.21.15.47:0/4270935608 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22940a3070 0x7f22940a5530 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f22a8021000 tx=0x7f22a8022000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.551+0000 7f229be00640 1 -- 172.21.15.47:0/150590196 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a800d3e0 con 0x55e7393b8810 2024-03-21T11:30:04.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.552+0000 7f22b8680740 1 -- 172.21.15.47:0/150590196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b8810 msgr2=0x55e7393b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.552+0000 7f22b8680740 1 --2- 172.21.15.47:0/150590196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b8810 0x55e7393b8c30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f22a800f5c0 tx=0x7f22a800fa80 comp rx=0 tx=0).stop 2024-03-21T11:30:04.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.552+0000 7f22b8680740 1 -- 172.21.15.47:0/150590196 shutdown_connections 2024-03-21T11:30:04.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.552+0000 7f22b8680740 1 --2- 172.21.15.47:0/150590196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393ba3a0 0x55e7393c0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.552+0000 7f22b8680740 1 --2- 172.21.15.47:0/150590196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393b99e0 0x55e7393b9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.552+0000 7f22b8680740 1 --2- 172.21.15.47:0/150590196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b8810 0x55e7393b8c30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.552+0000 7f22b8680740 1 -- 172.21.15.47:0/150590196 >> 172.21.15.47:0/150590196 conn(0x55e7393b3c00 msgr2=0x55e7393b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.553+0000 7f22b8680740 1 -- 172.21.15.47:0/150590196 shutdown_connections 2024-03-21T11:30:04.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.553+0000 7f22b8680740 1 -- 172.21.15.47:0/150590196 wait complete. 2024-03-21T11:30:04.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.553+0000 7f22b8680740 1 -- start start 2024-03-21T11:30:04.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.554+0000 7f22b8680740 1 Processor -- start 2024-03-21T11:30:04.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.555+0000 7f22b8680740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.555+0000 7f22b8680740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.555+0000 7f22b8680740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.555+0000 7f22b8680740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7393c13b0 con 0x55e7393b99e0 2024-03-21T11:30:04.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.555+0000 7f22b8680740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7392d90d0 con 0x55e7393b8810 2024-03-21T11:30:04.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.555+0000 7f22b8680740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7393c1530 con 0x55e7393ba3a0 2024-03-21T11:30:04.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.555+0000 7f22aea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.555+0000 7f22b8680740 10 client.0 _ll_register_callbacks cb 0x55e7392dde80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:04.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.555+0000 7f22aea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52190/0 (socket says 172.21.15.47:52190) 2024-03-21T11:30:04.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.555+0000 7f22aea00640 1 -- 172.21.15.47:0/2124975222 learned_addr learned my addr 172.21.15.47:0/2124975222 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.555+0000 7f22af400640 1 --2- 172.21.15.47:0/2124975222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.556+0000 7f22aea00640 1 -- 172.21.15.47:0/2124975222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 msgr2=0x55e7393c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.556+0000 7f22aea00640 1 --2- 172.21.15.47:0/2124975222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.556+0000 7f22aea00640 1 -- 172.21.15.47:0/2124975222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 msgr2=0x55e7393c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.556+0000 7f22aea00640 1 --2- 172.21.15.47:0/2124975222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.556+0000 7f22aea00640 1 -- 172.21.15.47:0/2124975222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7390313f0 con 0x55e7393b8810 2024-03-21T11:30:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.556+0000 7f22af400640 1 --2- 172.21.15.47:0/2124975222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:30:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.556+0000 7f22aea00640 1 --2- 172.21.15.47:0/2124975222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c29e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f22a009bcc0 tx=0x7f22a000ff60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0ff10ef0f67e46b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.557+0000 7f2298c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:04.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.557+0000 7f2298c00640 1 -- 172.21.15.47:0/2124975222 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a00a49d0 con 0x55e7393b8810 2024-03-21T11:30:04.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.557+0000 7f2298c00640 1 -- 172.21.15.47:0/2124975222 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f22a0003b10 con 0x55e7393b8810 2024-03-21T11:30:04.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.558+0000 7f22b8680740 1 -- 172.21.15.47:0/2124975222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22a8020020 con 0x55e7393b8810 2024-03-21T11:30:04.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.559+0000 7f2298c00640 1 -- 172.21.15.47:0/2124975222 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22a00bc0c0 con 0x55e7393b8810 2024-03-21T11:30:04.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.559+0000 7f22b8680740 10 client.27755 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:04.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.559+0000 7f22b8680740 1 -- 172.21.15.47:0/2124975222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e7390fb7d0 con 0x55e7393b8810 2024-03-21T11:30:04.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.560+0000 7f2298c00640 1 -- 172.21.15.47:0/2124975222 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22a8020020 con 0x55e7393b8810 2024-03-21T11:30:04.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.560+0000 7f2298c00640 1 -- 172.21.15.47:0/2124975222 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f22a00b6020 con 0x55e7393b8810 2024-03-21T11:30:04.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.560+0000 7f22b8680740 10 client.27755 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:04.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.560+0000 7f22b8680740 10 client.27755 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:30:04.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.560+0000 7f22b8680740 20 client.27755 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:04.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.560+0000 7f22b8680740 1 --2- 172.21.15.47:0/2124975222 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e7393ce460 0x55e7393d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.560+0000 7f22b8680740 4 client.27755 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:04.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.560+0000 7f22b8680740 1 -- 172.21.15.47:0/2124975222 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e73932a800 con 0x55e7393ce460 2024-03-21T11:30:04.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.561+0000 7f22ae000640 1 --2- 172.21.15.47:0/2124975222 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e7393ce460 0x55e7393d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.562+0000 7f2298c00640 10 client.27755 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:30:04.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.563+0000 7f22ae000640 1 --2- 172.21.15.47:0/2124975222 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e7393ce460 0x55e7393d08e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ed7fe6a91a1c43a server_cookie=117629f503a0d362 in_seq=0 out_seq=0 2024-03-21T11:30:04.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.566+0000 7f2298c00640 1 -- 172.21.15.47:0/2124975222 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e73932a800 con 0x55e7393ce460 2024-03-21T11:30:04.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.566+0000 7f2298c00640 10 client.27755 handle_command_reply: tid=0 2024-03-21T11:30:04.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.566+0000 7f22b8680740 1 client.27755 shutdown 2024-03-21T11:30:04.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.567+0000 7f22b8680740 1 -- 172.21.15.47:0/2124975222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 msgr2=0x55e7393c29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.567+0000 7f22b8680740 1 --2- 172.21.15.47:0/2124975222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c29e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f22a009bcc0 tx=0x7f22a000ff60 comp rx=0 tx=0).stop 2024-03-21T11:30:04.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.567+0000 7f22b8680740 1 -- 172.21.15.47:0/2124975222 shutdown_connections 2024-03-21T11:30:04.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.567+0000 7f22b8680740 1 --2- 172.21.15.47:0/2124975222 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e7393ce460 0x55e7393d08e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.567+0000 7f22b8680740 1 --2- 172.21.15.47:0/2124975222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.567+0000 7f22b8680740 1 --2- 172.21.15.47:0/2124975222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.567+0000 7f22b8680740 1 --2- 172.21.15.47:0/2124975222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c29e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.567+0000 7f22b8680740 1 -- 172.21.15.47:0/2124975222 >> 172.21.15.47:0/2124975222 conn(0x55e7393b3c00 msgr2=0x55e7393b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.567+0000 7f2298c00640 0 client.27755 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:30:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.567+0000 7f22b8680740 1 -- 172.21.15.47:0/2124975222 shutdown_connections 2024-03-21T11:30:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.568+0000 7f22b8680740 1 -- 172.21.15.47:0/2124975222 wait complete. 2024-03-21T11:30:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.568+0000 7f22b8680740 20 client.27755 trim_cache size 0 max 16384 2024-03-21T11:30:04.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.602+0000 7f22b8680740 1 Processor -- start 2024-03-21T11:30:04.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.602+0000 7f22b8680740 1 -- start start 2024-03-21T11:30:04.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.603+0000 7f22b8680740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.603+0000 7f22b8680740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.603+0000 7f22b8680740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.603+0000 7f22b8680740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01b0790 con 0x55e7393b99e0 2024-03-21T11:30:04.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.603+0000 7f22b8680740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01b2110 con 0x55e7393b8810 2024-03-21T11:30:04.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.603+0000 7f22aea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.604+0000 7f22b8680740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01b0df0 con 0x55e7393ba3a0 2024-03-21T11:30:04.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.603+0000 7f22ae000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.604+0000 7f22ae000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48720/0 (socket says 172.21.15.47:48720) 2024-03-21T11:30:04.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.604+0000 7f22ae000640 1 -- 172.21.15.47:0/3693068513 learned_addr learned my addr 172.21.15.47:0/3693068513 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.604+0000 7f22aea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52192/0 (socket says 172.21.15.47:52192) 2024-03-21T11:30:04.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.604+0000 7f22af400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.605+0000 7f22ae000640 1 -- 172.21.15.47:0/3693068513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 msgr2=0x55e7393b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.605+0000 7f22ae000640 1 --2- 172.21.15.47:0/3693068513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.605+0000 7f22ae000640 1 -- 172.21.15.47:0/3693068513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 msgr2=0x55e7393c2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.605+0000 7f22ae000640 1 --2- 172.21.15.47:0/3693068513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.605+0000 7f22ae000640 1 -- 172.21.15.47:0/3693068513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7393c0fa0 con 0x55e7393b99e0 2024-03-21T11:30:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.606+0000 7f22af400640 1 --2- 172.21.15.47:0/3693068513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.606+0000 7f22ae000640 1 --2- 172.21.15.47:0/3693068513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c9f20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f229c02c150 tx=0x7f229c02c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423ac662b8830a81 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.606+0000 7f229b400640 1 -- 172.21.15.47:0/3693068513 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c046020 con 0x55e7393b99e0 2024-03-21T11:30:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.607+0000 7f229b400640 1 -- 172.21.15.47:0/3693068513 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f229c034bb0 con 0x55e7393b99e0 2024-03-21T11:30:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.607+0000 7f229b400640 1 -- 172.21.15.47:0/3693068513 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c039040 con 0x55e7393b99e0 2024-03-21T11:30:04.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.608+0000 7f22b8680740 1 -- 172.21.15.47:0/3693068513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 msgr2=0x55e7393c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.608+0000 7f22b8680740 1 --2- 172.21.15.47:0/3693068513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c9f20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f229c02c150 tx=0x7f229c02c310 comp rx=0 tx=0).stop 2024-03-21T11:30:04.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.608+0000 7f22b8680740 1 -- 172.21.15.47:0/3693068513 shutdown_connections 2024-03-21T11:30:04.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.608+0000 7f22b8680740 1 --2- 172.21.15.47:0/3693068513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.608+0000 7f22b8680740 1 --2- 172.21.15.47:0/3693068513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c9f20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.608+0000 7f22b8680740 1 --2- 172.21.15.47:0/3693068513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.608+0000 7f22b8680740 1 -- 172.21.15.47:0/3693068513 >> 172.21.15.47:0/3693068513 conn(0x55e7393b3c00 msgr2=0x55e7393b76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.608+0000 7f22b8680740 1 -- 172.21.15.47:0/3693068513 shutdown_connections 2024-03-21T11:30:04.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.608+0000 7f22b8680740 1 -- 172.21.15.47:0/3693068513 wait complete. 2024-03-21T11:30:04.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.609+0000 7f22b8680740 1 -- start start 2024-03-21T11:30:04.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.609+0000 7f22b8680740 1 Processor -- start 2024-03-21T11:30:04.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.610+0000 7f22b8680740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.610+0000 7f22b8680740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.610+0000 7f22b8680740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393c9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.610+0000 7f22b8680740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01b1ab0 con 0x55e7393b99e0 2024-03-21T11:30:04.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.610+0000 7f22b8680740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01b1450 con 0x55e7393b8810 2024-03-21T11:30:04.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.610+0000 7f22b8680740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01ae950 con 0x55e7393ba3a0 2024-03-21T11:30:04.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.610+0000 7f22ae000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.610+0000 7f22ae000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48730/0 (socket says 172.21.15.47:48730) 2024-03-21T11:30:04.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.610+0000 7f22ae000640 1 -- 172.21.15.47:0/830821257 learned_addr learned my addr 172.21.15.47:0/830821257 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.610+0000 7f22b8680740 10 client.0 _ll_register_callbacks cb 0x7f22a00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:04.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.611+0000 7f22af400640 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.612+0000 7f22ae000640 1 -- 172.21.15.47:0/830821257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 msgr2=0x55e7393c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.612+0000 7f22aea00640 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.612+0000 7f22ae000640 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.612+0000 7f22ae000640 1 -- 172.21.15.47:0/830821257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 msgr2=0x55e7393c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.612+0000 7f22ae000640 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.612+0000 7f22ae000640 1 -- 172.21.15.47:0/830821257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7393c9920 con 0x55e7393b99e0 2024-03-21T11:30:04.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.612+0000 7f22aea00640 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:30:04.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.612+0000 7f22af400640 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.613+0000 7f22ae000640 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c8320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f229c037040 tx=0x7f229c005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a490c02f24325094 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.613+0000 7f2298c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:30:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.613+0000 7f2298c00640 1 -- 172.21.15.47:0/830821257 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c046070 con 0x55e7393b99e0 2024-03-21T11:30:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.613+0000 7f2298c00640 1 -- 172.21.15.47:0/830821257 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f229c034d60 con 0x55e7393b99e0 2024-03-21T11:30:04.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.613+0000 7f22b8680740 1 -- 172.21.15.47:0/830821257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f229c034bb0 con 0x55e7393b99e0 2024-03-21T11:30:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.614+0000 7f2298c00640 1 -- 172.21.15.47:0/830821257 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f229c039040 con 0x55e7393b99e0 2024-03-21T11:30:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.614+0000 7f22b8680740 10 client.17475 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.614+0000 7f22b8680740 1 -- 172.21.15.47:0/830821257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e73943b540 con 0x55e7393b99e0 2024-03-21T11:30:04.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.614+0000 7f2298c00640 1 -- 172.21.15.47:0/830821257 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f229c034bb0 con 0x55e7393b99e0 2024-03-21T11:30:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.615+0000 7f2298c00640 1 -- 172.21.15.47:0/830821257 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f229c052030 con 0x55e7393b99e0 2024-03-21T11:30:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.615+0000 7f22b8680740 10 client.17475 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.615+0000 7f22b8680740 10 client.17475 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:30:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.615+0000 7f22b8680740 20 client.17475 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:04.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.615+0000 7f22b8680740 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e73943be60 0x55e73943e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.616+0000 7f22b8680740 4 client.17475 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:30:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.616+0000 7f22b8680740 1 -- 172.21.15.47:0/830821257 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e739324b50 con 0x55e73943be60 2024-03-21T11:30:04.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.616+0000 7f22aea00640 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e73943be60 0x55e73943e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.618+0000 7f22aea00640 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e73943be60 0x55e73943e280 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef3f4c3a8c51c00 server_cookie=8d0ab111bdb57fc6 in_seq=0 out_seq=0 2024-03-21T11:30:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.618+0000 7f2298c00640 10 client.17475 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:30:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.618+0000 7f2298c00640 1 -- 172.21.15.47:0/830821257 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e739324b50 con 0x55e73943be60 2024-03-21T11:30:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.618+0000 7f2298c00640 10 client.17475 handle_command_reply: tid=0 2024-03-21T11:30:04.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.618+0000 7f22b8680740 1 client.17475 shutdown 2024-03-21T11:30:04.621 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:30:04.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.619+0000 7f22b8680740 1 -- 172.21.15.47:0/830821257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 msgr2=0x55e7393c8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.619+0000 7f22b8680740 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c8320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f229c037040 tx=0x7f229c005da0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.619+0000 7f22b8680740 1 -- 172.21.15.47:0/830821257 shutdown_connections 2024-03-21T11:30:04.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.619+0000 7f22b8680740 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e73943be60 0x55e73943e280 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.619+0000 7f22b8680740 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7393ba3a0 0x55e7393c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.619+0000 7f22b8680740 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7393b99e0 0x55e7393c8320 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.619+0000 7f22b8680740 1 --2- 172.21.15.47:0/830821257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7393b8810 0x55e7393c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.619+0000 7f22b8680740 1 -- 172.21.15.47:0/830821257 >> 172.21.15.47:0/830821257 conn(0x55e7393b3c00 msgr2=0x55e7393b76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.619+0000 7f2298c00640 0 client.17475 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:30:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.619+0000 7f22b8680740 1 -- 172.21.15.47:0/830821257 shutdown_connections 2024-03-21T11:30:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.619+0000 7f22b8680740 1 -- 172.21.15.47:0/830821257 wait complete. 2024-03-21T11:30:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.620+0000 7f22b8680740 20 client.17475 trim_cache size 0 max 16384 2024-03-21T11:30:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.621+0000 7f22b5a00640 1 -- 172.21.15.47:0/4270935608 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22940a3070 msgr2=0x7f22940a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.621+0000 7f22b5a00640 1 --2- 172.21.15.47:0/4270935608 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22940a3070 0x7f22940a5530 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f22a8021000 tx=0x7f22a8022000 comp rx=0 tx=0).stop 2024-03-21T11:30:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.621+0000 7f22b5a00640 1 -- 172.21.15.47:0/4270935608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b01016b0 msgr2=0x7f22b0110570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.621+0000 7f22b5a00640 1 --2- 172.21.15.47:0/4270935608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b01016b0 0x7f22b0110570 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f22a000e9c0 tx=0x7f22a000ee80 comp rx=0 tx=0).stop 2024-03-21T11:30:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.621+0000 7f22b5a00640 1 -- 172.21.15.47:0/4270935608 shutdown_connections 2024-03-21T11:30:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.621+0000 7f22b5a00640 1 --2- 172.21.15.47:0/4270935608 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22940a3070 0x7f22940a5530 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.621+0000 7f22b5a00640 1 --2- 172.21.15.47:0/4270935608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22b0103260 0x7f22b010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.621+0000 7f22b5a00640 1 --2- 172.21.15.47:0/4270935608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b01028a0 0x7f22b0110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.621+0000 7f22b5a00640 1 --2- 172.21.15.47:0/4270935608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22b01016b0 0x7f22b0110570 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.621+0000 7f22b5a00640 1 -- 172.21.15.47:0/4270935608 >> 172.21.15.47:0/4270935608 conn(0x7f22b00fcc40 msgr2=0x7f22b00fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.621+0000 7f22b5a00640 1 -- 172.21.15.47:0/4270935608 shutdown_connections 2024-03-21T11:30:04.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.622+0000 7f22b5a00640 1 -- 172.21.15.47:0/4270935608 wait complete. 2024-03-21T11:30:04.652 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:30:04.652 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:30:04.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.884+0000 7f4b67200640 1 -- 172.21.15.47:0/194291114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b60104ab0 msgr2=0x7f4b60104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.884+0000 7f4b67200640 1 --2- 172.21.15.47:0/194291114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b60104ab0 0x7f4b60104f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009900 tx=0x7f4b4802bc30 comp rx=0 tx=0).stop 2024-03-21T11:30:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.884+0000 7f4b67200640 1 -- 172.21.15.47:0/194291114 shutdown_connections 2024-03-21T11:30:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.884+0000 7f4b67200640 1 --2- 172.21.15.47:0/194291114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b60105470 0x7f4b6010bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.884+0000 7f4b67200640 1 --2- 172.21.15.47:0/194291114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b60104ab0 0x7f4b60104f30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.884+0000 7f4b67200640 1 --2- 172.21.15.47:0/194291114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b601038c0 0x7f4b60103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.884+0000 7f4b67200640 1 -- 172.21.15.47:0/194291114 >> 172.21.15.47:0/194291114 conn(0x7f4b600fee50 msgr2=0x7f4b60101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.884+0000 7f4b67200640 1 -- 172.21.15.47:0/194291114 shutdown_connections 2024-03-21T11:30:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.884+0000 7f4b67200640 1 -- 172.21.15.47:0/194291114 wait complete. 2024-03-21T11:30:04.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.885+0000 7f4b67200640 1 Processor -- start 2024-03-21T11:30:04.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b67200640 1 -- start start 2024-03-21T11:30:04.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b67200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b601038c0 0x7f4b60199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b67200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b60104ab0 0x7f4b6019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b67200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b60105470 0x7f4b601a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b67200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6010f570 con 0x7f4b601038c0 2024-03-21T11:30:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b67200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6010f3f0 con 0x7f4b60105470 2024-03-21T11:30:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b67200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6010f6f0 con 0x7f4b60104ab0 2024-03-21T11:30:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b57400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b60104ab0 0x7f4b6019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b65200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b60105470 0x7f4b601a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b5fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b601038c0 0x7f4b60199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b57400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b60104ab0 0x7f4b6019a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59918/0 (socket says 172.21.15.47:59918) 2024-03-21T11:30:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b57400640 1 -- 172.21.15.47:0/33447755 learned_addr learned my addr 172.21.15.47:0/33447755 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b65200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b60105470 0x7f4b601a1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52218/0 (socket says 172.21.15.47:52218) 2024-03-21T11:30:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b57400640 1 -- 172.21.15.47:0/33447755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b60105470 msgr2=0x7f4b601a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b57400640 1 --2- 172.21.15.47:0/33447755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b60105470 0x7f4b601a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b57400640 1 -- 172.21.15.47:0/33447755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b601038c0 msgr2=0x7f4b60199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b57400640 1 --2- 172.21.15.47:0/33447755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b601038c0 0x7f4b60199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.886+0000 7f4b57400640 1 -- 172.21.15.47:0/33447755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b601a17b0 con 0x7f4b60104ab0 2024-03-21T11:30:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.887+0000 7f4b57400640 1 --2- 172.21.15.47:0/33447755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b60104ab0 0x7f4b6019a0d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4b48000c00 tx=0x7f4b4802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.887+0000 7f4b5d600640 1 -- 172.21.15.47:0/33447755 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48043020 con 0x7f4b60104ab0 2024-03-21T11:30:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.887+0000 7f4b5d600640 1 -- 172.21.15.47:0/33447755 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b480054e0 con 0x7f4b60104ab0 2024-03-21T11:30:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.887+0000 7f4b67200640 1 -- 172.21.15.47:0/33447755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b601a1a40 con 0x7f4b60104ab0 2024-03-21T11:30:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.887+0000 7f4b5d600640 1 -- 172.21.15.47:0/33447755 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48003c40 con 0x7f4b60104ab0 2024-03-21T11:30:04.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.887+0000 7f4b67200640 1 -- 172.21.15.47:0/33447755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b601a1f90 con 0x7f4b60104ab0 2024-03-21T11:30:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.890+0000 7f4b69e5b740 1 Processor -- start 2024-03-21T11:30:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.891+0000 7f4b69e5b740 1 -- start start 2024-03-21T11:30:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.891+0000 7f4b5d600640 1 -- 172.21.15.47:0/33447755 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4b48046020 con 0x7f4b60104ab0 2024-03-21T11:30:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.891+0000 7f4b69e5b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0a810 0x556a38f0ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.891+0000 7f4b69e5b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0b9e0 0x556a38f0be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.891+0000 7f4b69e5b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0c3a0 0x556a38f12c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.891+0000 7f4b69e5b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a38e6f2c0 con 0x556a38f0a810 2024-03-21T11:30:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.891+0000 7f4b69e5b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a38e17c90 con 0x556a38f0c3a0 2024-03-21T11:30:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.891+0000 7f4b69e5b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a38e6b360 con 0x556a38f0b9e0 2024-03-21T11:30:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.891+0000 7f4b65200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0a810 0x556a38f0ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.891+0000 7f4b5fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0b9e0 0x556a38f0be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.891+0000 7f4b5fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0b9e0 0x556a38f0be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59928/0 (socket says 172.21.15.47:59928) 2024-03-21T11:30:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.891+0000 7f4b5fe00640 1 -- 172.21.15.47:0/1008570138 learned_addr learned my addr 172.21.15.47:0/1008570138 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.891+0000 7f4b57400640 1 --2- 172.21.15.47:0/1008570138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0c3a0 0x556a38f12c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.892+0000 7f4b5fe00640 1 -- 172.21.15.47:0/1008570138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0c3a0 msgr2=0x556a38f12c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.892+0000 7f4b5fe00640 1 --2- 172.21.15.47:0/1008570138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0c3a0 0x556a38f12c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.892+0000 7f4b5fe00640 1 -- 172.21.15.47:0/1008570138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0a810 msgr2=0x556a38f0ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.892+0000 7f4b5fe00640 1 --2- 172.21.15.47:0/1008570138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0a810 0x556a38f0ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.892+0000 7f4b5fe00640 1 -- 172.21.15.47:0/1008570138 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a38e3c080 con 0x556a38f0b9e0 2024-03-21T11:30:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.892+0000 7f4b65200640 1 --2- 172.21.15.47:0/1008570138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0a810 0x556a38f0ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.892+0000 7f4b5fe00640 1 --2- 172.21.15.47:0/1008570138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0b9e0 0x556a38f0be60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4b50011470 tx=0x7f4b50011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d6f97279cb1559 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.892+0000 7f4b55600640 1 -- 172.21.15.47:0/1008570138 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5001e070 con 0x556a38f0b9e0 2024-03-21T11:30:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.892+0000 7f4b55600640 1 -- 172.21.15.47:0/1008570138 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b50003c30 con 0x556a38f0b9e0 2024-03-21T11:30:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.892+0000 7f4b55600640 1 -- 172.21.15.47:0/1008570138 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5000fa30 con 0x556a38f0b9e0 2024-03-21T11:30:04.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.892+0000 7f4b5d600640 1 --2- 172.21.15.47:0/33447755 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b440a2f20 0x7f4b440a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.893+0000 7f4b5d600640 1 -- 172.21.15.47:0/33447755 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4b480f17a0 con 0x7f4b60104ab0 2024-03-21T11:30:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.893+0000 7f4b65200640 1 --2- 172.21.15.47:0/33447755 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b440a2f20 0x7f4b440a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.893+0000 7f4b69e5b740 1 -- 172.21.15.47:0/1008570138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0b9e0 msgr2=0x556a38f0be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.893+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/1008570138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0b9e0 0x556a38f0be60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4b50011470 tx=0x7f4b50011930 comp rx=0 tx=0).stop 2024-03-21T11:30:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.893+0000 7f4b69e5b740 1 -- 172.21.15.47:0/1008570138 shutdown_connections 2024-03-21T11:30:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.893+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/1008570138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0c3a0 0x556a38f12c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.893+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/1008570138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0b9e0 0x556a38f0be60 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.893+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/1008570138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0a810 0x556a38f0ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.893+0000 7f4b69e5b740 1 -- 172.21.15.47:0/1008570138 >> 172.21.15.47:0/1008570138 conn(0x556a38f05c00 msgr2=0x556a38f06000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.893+0000 7f4b65200640 1 --2- 172.21.15.47:0/33447755 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b440a2f20 0x7f4b440a53e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x556a38f0b8e0 tx=0x7f4b58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.893+0000 7f4b69e5b740 1 -- 172.21.15.47:0/1008570138 shutdown_connections 2024-03-21T11:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.893+0000 7f4b69e5b740 1 -- 172.21.15.47:0/1008570138 wait complete. 2024-03-21T11:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.894+0000 7f4b69e5b740 1 -- start start 2024-03-21T11:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.894+0000 7f4b69e5b740 1 Processor -- start 2024-03-21T11:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.894+0000 7f4b69e5b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0a810 0x556a38f14ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.894+0000 7f4b69e5b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0b9e0 0x556a38f15020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.894+0000 7f4b69e5b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0c3a0 0x556a38f1bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.894+0000 7f4b69e5b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a38f133b0 con 0x556a38f0a810 2024-03-21T11:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.894+0000 7f4b69e5b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a38e2b0d0 con 0x556a38f0b9e0 2024-03-21T11:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.894+0000 7f4b69e5b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a38f13530 con 0x556a38f0c3a0 2024-03-21T11:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.894+0000 7f4b69e5b740 10 client.0 _ll_register_callbacks cb 0x556a38e2fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.895+0000 7f4b65200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0c3a0 0x556a38f1bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.895+0000 7f4b57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0b9e0 0x556a38f15020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.895+0000 7f4b57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0b9e0 0x556a38f15020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52228/0 (socket says 172.21.15.47:52228) 2024-03-21T11:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.895+0000 7f4b57400640 1 -- 172.21.15.47:0/44145168 learned_addr learned my addr 172.21.15.47:0/44145168 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.895+0000 7f4b5fe00640 1 --2- 172.21.15.47:0/44145168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0a810 0x556a38f14ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.895+0000 7f4b57400640 1 -- 172.21.15.47:0/44145168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0c3a0 msgr2=0x556a38f1bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.895+0000 7f4b57400640 1 --2- 172.21.15.47:0/44145168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0c3a0 0x556a38f1bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.895+0000 7f4b57400640 1 -- 172.21.15.47:0/44145168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0a810 msgr2=0x556a38f14ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.895+0000 7f4b57400640 1 --2- 172.21.15.47:0/44145168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0a810 0x556a38f14ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.895+0000 7f4b57400640 1 -- 172.21.15.47:0/44145168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a38b833f0 con 0x556a38f0b9e0 2024-03-21T11:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.895+0000 7f4b57400640 1 --2- 172.21.15.47:0/44145168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0b9e0 0x556a38f15020 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4b480b9fa0 tx=0x7f4b480b9de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=635a380b62c38255 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.895+0000 7f4b4e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.895+0000 7f4b4e000640 1 -- 172.21.15.47:0/44145168 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b480bd4a0 con 0x556a38f0b9e0 2024-03-21T11:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.895+0000 7f4b4e000640 1 -- 172.21.15.47:0/44145168 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4b480bbe40 con 0x556a38f0b9e0 2024-03-21T11:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.896+0000 7f4b69e5b740 1 -- 172.21.15.47:0/44145168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b50003c30 con 0x556a38f0b9e0 2024-03-21T11:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.896+0000 7f4b4e000640 1 -- 172.21.15.47:0/44145168 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b48004250 con 0x556a38f0b9e0 2024-03-21T11:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.896+0000 7f4b69e5b740 10 client.27770 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.896+0000 7f4b69e5b740 1 -- 172.21.15.47:0/44145168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556a38c4d7d0 con 0x556a38f0b9e0 2024-03-21T11:30:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.896+0000 7f4b4e000640 1 -- 172.21.15.47:0/44145168 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b50003c30 con 0x556a38f0b9e0 2024-03-21T11:30:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.896+0000 7f4b4e000640 1 -- 172.21.15.47:0/44145168 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4b480c60c0 con 0x556a38f0b9e0 2024-03-21T11:30:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.896+0000 7f4b69e5b740 10 client.27770 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.896+0000 7f4b69e5b740 10 client.27770 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:30:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.896+0000 7f4b69e5b740 20 client.27770 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.896+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/44145168 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556a38f204e0 0x556a38f22920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.896+0000 7f4b69e5b740 4 client.27770 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.896+0000 7f4b69e5b740 1 -- 172.21.15.47:0/44145168 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a38e7c800 con 0x556a38f204e0 2024-03-21T11:30:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.897+0000 7f4b5fe00640 1 --2- 172.21.15.47:0/44145168 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556a38f204e0 0x556a38f22920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.897+0000 7f4b5fe00640 1 --2- 172.21.15.47:0/44145168 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556a38f204e0 0x556a38f22920 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eac6ee30a9676e7a server_cookie=a4e094ca151116b4 in_seq=0 out_seq=0 2024-03-21T11:30:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.897+0000 7f4b4e000640 10 client.27770 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:30:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b4e000640 1 -- 172.21.15.47:0/44145168 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556a38e7c800 con 0x556a38f204e0 2024-03-21T11:30:04.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b4e000640 10 client.27770 handle_command_reply: tid=0 2024-03-21T11:30:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b69e5b740 1 client.27770 shutdown 2024-03-21T11:30:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b69e5b740 1 -- 172.21.15.47:0/44145168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0b9e0 msgr2=0x556a38f15020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/44145168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0b9e0 0x556a38f15020 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4b480b9fa0 tx=0x7f4b480b9de0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b69e5b740 1 -- 172.21.15.47:0/44145168 shutdown_connections 2024-03-21T11:30:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/44145168 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556a38f204e0 0x556a38f22920 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/44145168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0c3a0 0x556a38f1bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/44145168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0b9e0 0x556a38f15020 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/44145168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0a810 0x556a38f14ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b4e000640 0 client.27770 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:30:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b69e5b740 1 -- 172.21.15.47:0/44145168 >> 172.21.15.47:0/44145168 conn(0x556a38f05c00 msgr2=0x556a38f04eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b69e5b740 1 -- 172.21.15.47:0/44145168 shutdown_connections 2024-03-21T11:30:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b69e5b740 1 -- 172.21.15.47:0/44145168 wait complete. 2024-03-21T11:30:04.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.898+0000 7f4b69e5b740 20 client.27770 trim_cache size 0 max 16384 2024-03-21T11:30:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.912+0000 7f4b69e5b740 1 Processor -- start 2024-03-21T11:30:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.912+0000 7f4b69e5b740 1 -- start start 2024-03-21T11:30:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.912+0000 7f4b69e5b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0a810 0x556a38f09570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.912+0000 7f4b69e5b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0b9e0 0x556a38f1bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.912+0000 7f4b69e5b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0c3a0 0x556a38f271e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.912+0000 7f4b69e5b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601ac210 con 0x556a38f0b9e0 2024-03-21T11:30:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.912+0000 7f4b69e5b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601adb90 con 0x556a38f0a810 2024-03-21T11:30:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.912+0000 7f4b69e5b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601ac870 con 0x556a38f0c3a0 2024-03-21T11:30:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.913+0000 7f4b57400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0b9e0 0x556a38f1bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.913+0000 7f4b5fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0a810 0x556a38f09570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.913+0000 7f4b57400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0b9e0 0x556a38f1bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48782/0 (socket says 172.21.15.47:48782) 2024-03-21T11:30:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.913+0000 7f4b57400640 1 -- 172.21.15.47:0/240447861 learned_addr learned my addr 172.21.15.47:0/240447861 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.913+0000 7f4b5fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0a810 0x556a38f09570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52234/0 (socket says 172.21.15.47:52234) 2024-03-21T11:30:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.913+0000 7f4b57400640 1 -- 172.21.15.47:0/240447861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0c3a0 msgr2=0x556a38f271e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.913+0000 7f4b57400640 1 --2- 172.21.15.47:0/240447861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0c3a0 0x556a38f271e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.913+0000 7f4b57400640 1 -- 172.21.15.47:0/240447861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0a810 msgr2=0x556a38f09570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.913+0000 7f4b57400640 1 --2- 172.21.15.47:0/240447861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0a810 0x556a38f09570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.913+0000 7f4b57400640 1 -- 172.21.15.47:0/240447861 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a38f0b460 con 0x556a38f0b9e0 2024-03-21T11:30:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.913+0000 7f4b5fe00640 1 --2- 172.21.15.47:0/240447861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0a810 0x556a38f09570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:04.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.914+0000 7f4b57400640 1 --2- 172.21.15.47:0/240447861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0b9e0 0x556a38f1bfb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b48006010 tx=0x7f4b4802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7001fbb4e4621c75 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.914+0000 7f4b54c00640 1 -- 172.21.15.47:0/240447861 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b480bd4a0 con 0x556a38f0b9e0 2024-03-21T11:30:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.914+0000 7f4b54c00640 1 -- 172.21.15.47:0/240447861 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b480be240 con 0x556a38f0b9e0 2024-03-21T11:30:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.914+0000 7f4b54c00640 1 -- 172.21.15.47:0/240447861 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b480c6550 con 0x556a38f0b9e0 2024-03-21T11:30:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.915+0000 7f4b69e5b740 1 -- 172.21.15.47:0/240447861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0b9e0 msgr2=0x556a38f1bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.915+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/240447861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0b9e0 0x556a38f1bfb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b48006010 tx=0x7f4b4802bc30 comp rx=0 tx=0).stop 2024-03-21T11:30:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.915+0000 7f4b69e5b740 1 -- 172.21.15.47:0/240447861 shutdown_connections 2024-03-21T11:30:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.915+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/240447861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0c3a0 0x556a38f271e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.915+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/240447861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0b9e0 0x556a38f1bfb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.915+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/240447861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0a810 0x556a38f09570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.915+0000 7f4b69e5b740 1 -- 172.21.15.47:0/240447861 >> 172.21.15.47:0/240447861 conn(0x556a38f05c00 msgr2=0x556a38f08e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.915+0000 7f4b69e5b740 1 -- 172.21.15.47:0/240447861 shutdown_connections 2024-03-21T11:30:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.915+0000 7f4b69e5b740 1 -- 172.21.15.47:0/240447861 wait complete. 2024-03-21T11:30:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.915+0000 7f4b69e5b740 1 -- start start 2024-03-21T11:30:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.915+0000 7f4b69e5b740 1 Processor -- start 2024-03-21T11:30:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b69e5b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0a810 0x556a38f1bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b69e5b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0b9e0 0x556a38f189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b69e5b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0c3a0 0x556a38f18f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b69e5b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601ad530 con 0x556a38f0b9e0 2024-03-21T11:30:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b69e5b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601aced0 con 0x556a38f0c3a0 2024-03-21T11:30:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b69e5b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601aa3d0 con 0x556a38f0a810 2024-03-21T11:30:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b69e5b740 10 client.0 _ll_register_callbacks cb 0x7f4b48005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b5fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0a810 0x556a38f1bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b65200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0c3a0 0x556a38f18f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b65200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0c3a0 0x556a38f18f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52248/0 (socket says 172.21.15.47:52248) 2024-03-21T11:30:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b65200640 1 -- 172.21.15.47:0/4232262751 learned_addr learned my addr 172.21.15.47:0/4232262751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b57400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0b9e0 0x556a38f189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b5fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0a810 0x556a38f1bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59942/0 (socket says 172.21.15.47:59942) 2024-03-21T11:30:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b65200640 1 -- 172.21.15.47:0/4232262751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0a810 msgr2=0x556a38f1bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b65200640 1 --2- 172.21.15.47:0/4232262751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0a810 0x556a38f1bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b65200640 1 -- 172.21.15.47:0/4232262751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0b9e0 msgr2=0x556a38f189e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b65200640 1 --2- 172.21.15.47:0/4232262751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0b9e0 0x556a38f189e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.916+0000 7f4b65200640 1 -- 172.21.15.47:0/4232262751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a38f7bdb0 con 0x556a38f0c3a0 2024-03-21T11:30:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.917+0000 7f4b65200640 1 --2- 172.21.15.47:0/4232262751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0c3a0 0x556a38f18f60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4b58004650 tx=0x7f4b580035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce2c54cf483a1674 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.917+0000 7f4b4e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.917+0000 7f4b4e000640 1 -- 172.21.15.47:0/4232262751 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58005020 con 0x556a38f0c3a0 2024-03-21T11:30:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.917+0000 7f4b4e000640 1 -- 172.21.15.47:0/4232262751 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4b58018030 con 0x556a38f0c3a0 2024-03-21T11:30:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.917+0000 7f4b69e5b740 1 -- 172.21.15.47:0/4232262751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a38e76b50 con 0x556a38f0c3a0 2024-03-21T11:30:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.917+0000 7f4b4e000640 1 -- 172.21.15.47:0/4232262751 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b58035050 con 0x556a38f0c3a0 2024-03-21T11:30:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.917+0000 7f4b69e5b740 10 client.27776 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.917+0000 7f4b69e5b740 1 -- 172.21.15.47:0/4232262751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556a38f7d180 con 0x556a38f0c3a0 2024-03-21T11:30:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.917+0000 7f4b4e000640 1 -- 172.21.15.47:0/4232262751 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556a38e76b50 con 0x556a38f0c3a0 2024-03-21T11:30:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.918+0000 7f4b4e000640 1 -- 172.21.15.47:0/4232262751 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4b5801d410 con 0x556a38f0c3a0 2024-03-21T11:30:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.918+0000 7f4b69e5b740 10 client.27776 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.918+0000 7f4b69e5b740 10 client.27776 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:30:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.918+0000 7f4b69e5b740 20 client.27776 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.918+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/4232262751 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556a38f7db90 0x556a38f7ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.918+0000 7f4b69e5b740 4 client.27776 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:30:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.918+0000 7f4b69e5b740 1 -- 172.21.15.47:0/4232262751 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556a38df4500 con 0x556a38f7db90 2024-03-21T11:30:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.918+0000 7f4b5fe00640 1 --2- 172.21.15.47:0/4232262751 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556a38f7db90 0x556a38f7ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:04.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.918+0000 7f4b5fe00640 1 --2- 172.21.15.47:0/4232262751 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556a38f7db90 0x556a38f7ffb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c3b1954070bfef3 server_cookie=c22f91ecf96fcd2f in_seq=0 out_seq=0 2024-03-21T11:30:04.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.918+0000 7f4b4e000640 10 client.27776 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:30:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b4e000640 1 -- 172.21.15.47:0/4232262751 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556a38df4500 con 0x556a38f7db90 2024-03-21T11:30:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b4e000640 10 client.27776 handle_command_reply: tid=0 2024-03-21T11:30:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b69e5b740 1 client.27776 shutdown 2024-03-21T11:30:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b69e5b740 1 -- 172.21.15.47:0/4232262751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0c3a0 msgr2=0x556a38f18f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/4232262751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0c3a0 0x556a38f18f60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4b58004650 tx=0x7f4b580035f0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b69e5b740 1 -- 172.21.15.47:0/4232262751 shutdown_connections 2024-03-21T11:30:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/4232262751 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x556a38f7db90 0x556a38f7ffb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/4232262751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a38f0c3a0 0x556a38f18f60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b4e000640 0 client.27776 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:30:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/4232262751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a38f0b9e0 0x556a38f189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b69e5b740 1 --2- 172.21.15.47:0/4232262751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a38f0a810 0x556a38f1bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b69e5b740 1 -- 172.21.15.47:0/4232262751 >> 172.21.15.47:0/4232262751 conn(0x556a38f05c00 msgr2=0x556a38f096f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b69e5b740 1 -- 172.21.15.47:0/4232262751 shutdown_connections 2024-03-21T11:30:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b69e5b740 1 -- 172.21.15.47:0/4232262751 wait complete. 2024-03-21T11:30:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.919+0000 7f4b69e5b740 20 client.27776 trim_cache size 0 max 16384 2024-03-21T11:30:04.925 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:30:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.920+0000 7f4b67200640 1 -- 172.21.15.47:0/33447755 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b440a2f20 msgr2=0x7f4b440a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.920+0000 7f4b67200640 1 --2- 172.21.15.47:0/33447755 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b440a2f20 0x7f4b440a53e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x556a38f0b8e0 tx=0x7f4b58005f90 comp rx=0 tx=0).stop 2024-03-21T11:30:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.920+0000 7f4b67200640 1 -- 172.21.15.47:0/33447755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b60104ab0 msgr2=0x7f4b6019a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.920+0000 7f4b67200640 1 --2- 172.21.15.47:0/33447755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b60104ab0 0x7f4b6019a0d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4b48000c00 tx=0x7f4b4802c650 comp rx=0 tx=0).stop 2024-03-21T11:30:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.920+0000 7f4b67200640 1 -- 172.21.15.47:0/33447755 shutdown_connections 2024-03-21T11:30:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.920+0000 7f4b67200640 1 --2- 172.21.15.47:0/33447755 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b440a2f20 0x7f4b440a53e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.920+0000 7f4b67200640 1 --2- 172.21.15.47:0/33447755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b60105470 0x7f4b601a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.920+0000 7f4b67200640 1 --2- 172.21.15.47:0/33447755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b60104ab0 0x7f4b6019a0d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.920+0000 7f4b67200640 1 --2- 172.21.15.47:0/33447755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b601038c0 0x7f4b60199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.920+0000 7f4b67200640 1 -- 172.21.15.47:0/33447755 >> 172.21.15.47:0/33447755 conn(0x7f4b600fee50 msgr2=0x7f4b60107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.920+0000 7f4b67200640 1 -- 172.21.15.47:0/33447755 shutdown_connections 2024-03-21T11:30:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:04.920+0000 7f4b67200640 1 -- 172.21.15.47:0/33447755 wait complete. 2024-03-21T11:30:04.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:04 smithi047 ceph-mon[31351]: pgmap v602: 129 pgs: 129 active+clean; 41 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 293 op/s 2024-03-21T11:30:04.936 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:30:04.936 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:30:05.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.139+0000 7f81c5400640 1 -- 172.21.15.47:0/1548670310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81c0104990 msgr2=0x7f81c0104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:05.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.139+0000 7f81c5400640 1 --2- 172.21.15.47:0/1548670310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81c0104990 0x7f81c0104e10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009900 tx=0x7f81a802bc30 comp rx=0 tx=0).stop 2024-03-21T11:30:05.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.140+0000 7f81c5400640 1 -- 172.21.15.47:0/1548670310 shutdown_connections 2024-03-21T11:30:05.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.140+0000 7f81c5400640 1 --2- 172.21.15.47:0/1548670310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c0105350 0x7f81c010bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.140+0000 7f81c5400640 1 --2- 172.21.15.47:0/1548670310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81c0104990 0x7f81c0104e10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.140+0000 7f81c5400640 1 --2- 172.21.15.47:0/1548670310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c01037a0 0x7f81c0103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.140+0000 7f81c5400640 1 -- 172.21.15.47:0/1548670310 >> 172.21.15.47:0/1548670310 conn(0x7f81c00fed10 msgr2=0x7f81c0101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:05.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.140+0000 7f81c5400640 1 -- 172.21.15.47:0/1548670310 shutdown_connections 2024-03-21T11:30:05.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.140+0000 7f81c5400640 1 -- 172.21.15.47:0/1548670310 wait complete. 2024-03-21T11:30:05.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.141+0000 7f81c5400640 1 Processor -- start 2024-03-21T11:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.141+0000 7f81c5400640 1 -- start start 2024-03-21T11:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.141+0000 7f81c5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81c01037a0 0x7f81c0199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.141+0000 7f81c5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c0104990 0x7f81c0199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.141+0000 7f81c5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c0105350 0x7f81c01a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.141+0000 7f81c5400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c010f450 con 0x7f81c0104990 2024-03-21T11:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.141+0000 7f81c5400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c010f2d0 con 0x7f81c0105350 2024-03-21T11:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.141+0000 7f81c5400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c010f5d0 con 0x7f81c01037a0 2024-03-21T11:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.141+0000 7f81bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81c01037a0 0x7f81c0199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.141+0000 7f81bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81c01037a0 0x7f81c0199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59962/0 (socket says 172.21.15.47:59962) 2024-03-21T11:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.141+0000 7f81bfe00640 1 -- 172.21.15.47:0/59700166 learned_addr learned my addr 172.21.15.47:0/59700166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.141+0000 7f81c4a00640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c0105350 0x7f81c01a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.141+0000 7f81bf400640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c0104990 0x7f81c0199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.142+0000 7f81bfe00640 1 -- 172.21.15.47:0/59700166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c0105350 msgr2=0x7f81c01a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.142+0000 7f81bfe00640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c0105350 0x7f81c01a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.142+0000 7f81bfe00640 1 -- 172.21.15.47:0/59700166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c0104990 msgr2=0x7f81c0199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.142+0000 7f81bfe00640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c0104990 0x7f81c0199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.142+0000 7f81bfe00640 1 -- 172.21.15.47:0/59700166 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c01a16b0 con 0x7f81c01037a0 2024-03-21T11:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.142+0000 7f81bf400640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c0104990 0x7f81c0199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.142+0000 7f81bfe00640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81c01037a0 0x7f81c0199a90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f81b000ebc0 tx=0x7f81b000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.142+0000 7f81bcc00640 1 -- 172.21.15.47:0/59700166 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b0003b70 con 0x7f81c01037a0 2024-03-21T11:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.142+0000 7f81bcc00640 1 -- 172.21.15.47:0/59700166 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81b0003d10 con 0x7f81c01037a0 2024-03-21T11:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.142+0000 7f81c5400640 1 -- 172.21.15.47:0/59700166 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c01a19a0 con 0x7f81c01037a0 2024-03-21T11:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.142+0000 7f81bcc00640 1 -- 172.21.15.47:0/59700166 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b0019980 con 0x7f81c01037a0 2024-03-21T11:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.142+0000 7f81c5400640 1 -- 172.21.15.47:0/59700166 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c01a1f10 con 0x7f81c01037a0 2024-03-21T11:30:05.146 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:04 smithi100 ceph-mon[36245]: pgmap v602: 129 pgs: 129 active+clean; 41 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 293 op/s 2024-03-21T11:30:05.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.146+0000 7f81c97a3740 1 Processor -- start 2024-03-21T11:30:05.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.146+0000 7f81c97a3740 1 -- start start 2024-03-21T11:30:05.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.146+0000 7f81bcc00640 1 -- 172.21.15.47:0/59700166 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f81b0019b20 con 0x7f81c01037a0 2024-03-21T11:30:05.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.146+0000 7f81c97a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2188e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.146+0000 7f81c97a3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c2189bc0 0x5612c218a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.146+0000 7f81c97a3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c218a580 0x5612c2190e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.146+0000 7f81c97a3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c20ed2c0 con 0x5612c218a580 2024-03-21T11:30:05.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.146+0000 7f81c97a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c2095c90 con 0x5612c21889f0 2024-03-21T11:30:05.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.146+0000 7f81c97a3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c20e9360 con 0x5612c2189bc0 2024-03-21T11:30:05.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.146+0000 7f81c4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2188e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.147+0000 7f81bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c2189bc0 0x5612c218a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.147+0000 7f81c4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2188e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52280/0 (socket says 172.21.15.47:52280) 2024-03-21T11:30:05.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.147+0000 7f81c4a00640 1 -- 172.21.15.47:0/3688536789 learned_addr learned my addr 172.21.15.47:0/3688536789 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:05.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.147+0000 7f81bf400640 1 --2- 172.21.15.47:0/3688536789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c218a580 0x5612c2190e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.147+0000 7f81c4a00640 1 -- 172.21.15.47:0/3688536789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c2189bc0 msgr2=0x5612c218a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:05.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.147+0000 7f81c4a00640 1 --2- 172.21.15.47:0/3688536789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c2189bc0 0x5612c218a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.147+0000 7f81c4a00640 1 -- 172.21.15.47:0/3688536789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c218a580 msgr2=0x5612c2190e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:05.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.147+0000 7f81c4a00640 1 --2- 172.21.15.47:0/3688536789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c218a580 0x5612c2190e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.147+0000 7f81c4a00640 1 -- 172.21.15.47:0/3688536789 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612c20ba080 con 0x5612c21889f0 2024-03-21T11:30:05.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.147+0000 7f81bf400640 1 --2- 172.21.15.47:0/3688536789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c218a580 0x5612c2190e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:05.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.147+0000 7f81c4a00640 1 --2- 172.21.15.47:0/3688536789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2188e10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009ca0 tx=0x7f81b8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=495de79242a5f80e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:05.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.148+0000 7f81b5200640 1 -- 172.21.15.47:0/3688536789 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b801e070 con 0x5612c21889f0 2024-03-21T11:30:05.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.148+0000 7f81b5200640 1 -- 172.21.15.47:0/3688536789 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81b8009e50 con 0x5612c21889f0 2024-03-21T11:30:05.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.148+0000 7f81b5200640 1 -- 172.21.15.47:0/3688536789 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b800eac0 con 0x5612c21889f0 2024-03-21T11:30:05.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.148+0000 7f81bcc00640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81a00a2fa0 0x7f81a00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.148+0000 7f81bcc00640 1 -- 172.21.15.47:0/59700166 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f81b00cffb0 con 0x7f81c01037a0 2024-03-21T11:30:05.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.148+0000 7f81bfe00640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81a00a2fa0 0x7f81a00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.148+0000 7f81c97a3740 1 -- 172.21.15.47:0/3688536789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 msgr2=0x5612c2188e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:05.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.148+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3688536789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2188e10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009ca0 tx=0x7f81b8011810 comp rx=0 tx=0).stop 2024-03-21T11:30:05.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.148+0000 7f81c97a3740 1 -- 172.21.15.47:0/3688536789 shutdown_connections 2024-03-21T11:30:05.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.148+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3688536789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c218a580 0x5612c2190e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.148+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3688536789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c2189bc0 0x5612c218a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.149+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3688536789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2188e10 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.149+0000 7f81c97a3740 1 -- 172.21.15.47:0/3688536789 >> 172.21.15.47:0/3688536789 conn(0x5612c2183de0 msgr2=0x5612c21841e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:05.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.149+0000 7f81c97a3740 1 -- 172.21.15.47:0/3688536789 shutdown_connections 2024-03-21T11:30:05.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.149+0000 7f81c97a3740 1 -- 172.21.15.47:0/3688536789 wait complete. 2024-03-21T11:30:05.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.149+0000 7f81bfe00640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81a00a2fa0 0x7f81a00a5460 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f81b00cd720 tx=0x7f81b00d3a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:05.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.149+0000 7f81c97a3740 1 -- start start 2024-03-21T11:30:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.149+0000 7f81c97a3740 1 Processor -- start 2024-03-21T11:30:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.150+0000 7f81c97a3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c21889f0 0x5612c2192c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.150+0000 7f81c97a3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 0x5612c21931a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.150+0000 7f81c97a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c218a580 0x5612c219a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.150+0000 7f81c97a3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c2191590 con 0x5612c2189bc0 2024-03-21T11:30:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.150+0000 7f81c97a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c20a90d0 con 0x5612c218a580 2024-03-21T11:30:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.150+0000 7f81c97a3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c2191710 con 0x5612c21889f0 2024-03-21T11:30:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.150+0000 7f81c97a3740 10 client.0 _ll_register_callbacks cb 0x5612c20ade80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.150+0000 7f81c4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c21889f0 0x5612c2192c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.150+0000 7f81c4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c21889f0 0x5612c2192c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59982/0 (socket says 172.21.15.47:59982) 2024-03-21T11:30:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.150+0000 7f81c4a00640 1 -- 172.21.15.47:0/3465528211 learned_addr learned my addr 172.21.15.47:0/3465528211 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.150+0000 7f81bf400640 1 --2- 172.21.15.47:0/3465528211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 0x5612c21931a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.150+0000 7f81c4a00640 1 --2- 172.21.15.47:0/3465528211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c218a580 0x5612c219a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.151+0000 7f81bf400640 1 -- 172.21.15.47:0/3465528211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c21889f0 msgr2=0x5612c2192c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:05.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.151+0000 7f81bf400640 1 --2- 172.21.15.47:0/3465528211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c21889f0 0x5612c2192c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.151+0000 7f81bf400640 1 -- 172.21.15.47:0/3465528211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c218a580 msgr2=0x5612c219a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:05.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.151+0000 7f81bf400640 1 --2- 172.21.15.47:0/3465528211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c218a580 0x5612c219a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.151+0000 7f81bf400640 1 -- 172.21.15.47:0/3465528211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612c1e013f0 con 0x5612c2189bc0 2024-03-21T11:30:05.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.151+0000 7f81bf400640 1 --2- 172.21.15.47:0/3465528211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 0x5612c21931a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f81a8036040 tx=0x7f81a80042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddc97f463896dd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:05.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.151+0000 7f81ad600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:30:05.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.151+0000 7f81ad600640 1 -- 172.21.15.47:0/3465528211 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8005d50 con 0x5612c2189bc0 2024-03-21T11:30:05.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.151+0000 7f81ad600640 1 -- 172.21.15.47:0/3465528211 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f81a8005450 con 0x5612c2189bc0 2024-03-21T11:30:05.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.151+0000 7f81c97a3740 1 -- 172.21.15.47:0/3465528211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81b8009e50 con 0x5612c2189bc0 2024-03-21T11:30:05.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.152+0000 7f81ad600640 1 -- 172.21.15.47:0/3465528211 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81a8051050 con 0x5612c2189bc0 2024-03-21T11:30:05.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.152+0000 7f81c97a3740 10 client.17505 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:05.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.152+0000 7f81c97a3740 1 -- 172.21.15.47:0/3465528211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5612c1ecb7d0 con 0x5612c2189bc0 2024-03-21T11:30:05.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.152+0000 7f81ad600640 1 -- 172.21.15.47:0/3465528211 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81b8009e50 con 0x5612c2189bc0 2024-03-21T11:30:05.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.152+0000 7f81ad600640 1 -- 172.21.15.47:0/3465528211 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f81a80035f0 con 0x5612c2189bc0 2024-03-21T11:30:05.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.152+0000 7f81c97a3740 10 client.17505 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:05.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.152+0000 7f81c97a3740 10 client.17505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:05.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.152+0000 7f81c97a3740 20 client.17505 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:05.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.152+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3465528211 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5612c219e760 0x5612c21a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.152+0000 7f81c97a3740 4 client.17505 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:05.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.152+0000 7f81c97a3740 1 -- 172.21.15.47:0/3465528211 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612c20fa800 con 0x5612c219e760 2024-03-21T11:30:05.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.153+0000 7f81c4a00640 1 --2- 172.21.15.47:0/3465528211 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5612c219e760 0x5612c21a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.153+0000 7f81c4a00640 1 --2- 172.21.15.47:0/3465528211 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5612c219e760 0x5612c21a0b80 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f5e298fc07e1c0c server_cookie=7083f6ff0d8b9e5e in_seq=0 out_seq=0 2024-03-21T11:30:05.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.153+0000 7f81ad600640 10 client.17505 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:05.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81ad600640 1 -- 172.21.15.47:0/3465528211 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5612c20fa800 con 0x5612c219e760 2024-03-21T11:30:05.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81ad600640 10 client.17505 handle_command_reply: tid=0 2024-03-21T11:30:05.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81c97a3740 1 client.17505 shutdown 2024-03-21T11:30:05.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81c97a3740 1 -- 172.21.15.47:0/3465528211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 msgr2=0x5612c21931a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:05.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3465528211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 0x5612c21931a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f81a8036040 tx=0x7f81a80042d0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81c97a3740 1 -- 172.21.15.47:0/3465528211 shutdown_connections 2024-03-21T11:30:05.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3465528211 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5612c219e760 0x5612c21a0b80 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3465528211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c218a580 0x5612c219a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3465528211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 0x5612c21931a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81ad600640 0 client.17505 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:05.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3465528211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c21889f0 0x5612c2192c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81c97a3740 1 -- 172.21.15.47:0/3465528211 >> 172.21.15.47:0/3465528211 conn(0x5612c2183de0 msgr2=0x5612c2183160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:05.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81c97a3740 1 -- 172.21.15.47:0/3465528211 shutdown_connections 2024-03-21T11:30:05.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81c97a3740 1 -- 172.21.15.47:0/3465528211 wait complete. 2024-03-21T11:30:05.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.154+0000 7f81c97a3740 20 client.17505 trim_cache size 0 max 16384 2024-03-21T11:30:05.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.168+0000 7f81c97a3740 1 Processor -- start 2024-03-21T11:30:05.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.168+0000 7f81c97a3740 1 -- start start 2024-03-21T11:30:05.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c97a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2192280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c97a3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 0x5612c219a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c97a3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c218a580 0x5612c218cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c97a3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c01ac0c0 con 0x5612c2189bc0 2024-03-21T11:30:05.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c97a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c01ada40 con 0x5612c21889f0 2024-03-21T11:30:05.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c97a3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c01ac720 con 0x5612c218a580 2024-03-21T11:30:05.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2192280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2192280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52296/0 (socket says 172.21.15.47:52296) 2024-03-21T11:30:05.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c4a00640 1 -- 172.21.15.47:0/3014615044 learned_addr learned my addr 172.21.15.47:0/3014615044 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:05.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c4a00640 1 --2- 172.21.15.47:0/3014615044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c218a580 0x5612c218cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c4a00640 1 -- 172.21.15.47:0/3014615044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c218a580 msgr2=0x5612c218cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:05.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c4a00640 1 --2- 172.21.15.47:0/3014615044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c218a580 0x5612c218cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c4a00640 1 -- 172.21.15.47:0/3014615044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 msgr2=0x5612c219a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:05.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c4a00640 1 --2- 172.21.15.47:0/3014615044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 0x5612c219a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.169+0000 7f81c4a00640 1 -- 172.21.15.47:0/3014615044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612c217f8b0 con 0x5612c21889f0 2024-03-21T11:30:05.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.170+0000 7f81c4a00640 1 --2- 172.21.15.47:0/3014615044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2192280 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009ca0 tx=0x7f81b8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a523177ab4826a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:05.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.170+0000 7f81afe00640 1 -- 172.21.15.47:0/3014615044 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b803e070 con 0x5612c21889f0 2024-03-21T11:30:05.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.170+0000 7f81afe00640 1 -- 172.21.15.47:0/3014615044 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81b800e880 con 0x5612c21889f0 2024-03-21T11:30:05.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.170+0000 7f81afe00640 1 -- 172.21.15.47:0/3014615044 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b801e040 con 0x5612c21889f0 2024-03-21T11:30:05.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.171+0000 7f81c97a3740 1 -- 172.21.15.47:0/3014615044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 msgr2=0x5612c2192280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:05.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.171+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3014615044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2192280 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009ca0 tx=0x7f81b8000bc0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.171+0000 7f81c97a3740 1 -- 172.21.15.47:0/3014615044 shutdown_connections 2024-03-21T11:30:05.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.171+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3014615044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c218a580 0x5612c218cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.171+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3014615044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 0x5612c219a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.171+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3014615044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2192280 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.171+0000 7f81c97a3740 1 -- 172.21.15.47:0/3014615044 >> 172.21.15.47:0/3014615044 conn(0x5612c2183de0 msgr2=0x5612c21878d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:05.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.171+0000 7f81c97a3740 1 -- 172.21.15.47:0/3014615044 shutdown_connections 2024-03-21T11:30:05.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.172+0000 7f81c97a3740 1 -- 172.21.15.47:0/3014615044 wait complete. 2024-03-21T11:30:05.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.172+0000 7f81c97a3740 1 -- start start 2024-03-21T11:30:05.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.172+0000 7f81c97a3740 1 Processor -- start 2024-03-21T11:30:05.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.173+0000 7f81c97a3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2199da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.173+0000 7f81c97a3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 0x5612c21983b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.173+0000 7f81c97a3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c218a580 0x5612c21988f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.173+0000 7f81c97a3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c01ad3e0 con 0x5612c2189bc0 2024-03-21T11:30:05.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.173+0000 7f81c97a3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c01acd80 con 0x5612c21889f0 2024-03-21T11:30:05.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.173+0000 7f81c97a3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c01aa280 con 0x5612c218a580 2024-03-21T11:30:05.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.173+0000 7f81c97a3740 10 client.0 _ll_register_callbacks cb 0x7f81b8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:05.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.173+0000 7f81c4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.173+0000 7f81c4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52310/0 (socket says 172.21.15.47:52310) 2024-03-21T11:30:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.173+0000 7f81c4a00640 1 -- 172.21.15.47:0/3276907825 learned_addr learned my addr 172.21.15.47:0/3276907825 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.173+0000 7f81c4a00640 1 --2- 172.21.15.47:0/3276907825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c218a580 0x5612c21988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.173+0000 7f81bf400640 1 --2- 172.21.15.47:0/3276907825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 0x5612c21983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.174+0000 7f81c4a00640 1 -- 172.21.15.47:0/3276907825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c218a580 msgr2=0x5612c21988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.174+0000 7f81c4a00640 1 --2- 172.21.15.47:0/3276907825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c218a580 0x5612c21988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.174+0000 7f81c4a00640 1 -- 172.21.15.47:0/3276907825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 msgr2=0x5612c21983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.174+0000 7f81c4a00640 1 --2- 172.21.15.47:0/3276907825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 0x5612c21983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.174+0000 7f81c4a00640 1 -- 172.21.15.47:0/3276907825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612c2198e30 con 0x5612c21889f0 2024-03-21T11:30:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.174+0000 7f81c4a00640 1 --2- 172.21.15.47:0/3276907825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2199da0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009ab0 tx=0x7f81b8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea42c686e2e83a74 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.174+0000 7f81ad600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.174+0000 7f81ad600640 1 -- 172.21.15.47:0/3276907825 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b803e020 con 0x5612c21889f0 2024-03-21T11:30:05.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.174+0000 7f81ad600640 1 -- 172.21.15.47:0/3276907825 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f81b80054d0 con 0x5612c21889f0 2024-03-21T11:30:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.174+0000 7f81c97a3740 1 -- 172.21.15.47:0/3276907825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81b800e880 con 0x5612c21889f0 2024-03-21T11:30:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.174+0000 7f81ad600640 1 -- 172.21.15.47:0/3276907825 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81b801e070 con 0x5612c21889f0 2024-03-21T11:30:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.174+0000 7f81c97a3740 10 client.27794 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.174+0000 7f81c97a3740 1 -- 172.21.15.47:0/3276907825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5612c21fad40 con 0x5612c21889f0 2024-03-21T11:30:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.175+0000 7f81ad600640 1 -- 172.21.15.47:0/3276907825 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81b800e880 con 0x5612c21889f0 2024-03-21T11:30:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.175+0000 7f81ad600640 1 -- 172.21.15.47:0/3276907825 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f81b8024050 con 0x5612c21889f0 2024-03-21T11:30:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.175+0000 7f81c97a3740 10 client.27794 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.175+0000 7f81c97a3740 10 client.27794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.175+0000 7f81c97a3740 20 client.27794 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.175+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3276907825 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5612c21fb360 0x5612c21fd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.175+0000 7f81c97a3740 4 client.27794 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:30:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.175+0000 7f81c97a3740 1 -- 172.21.15.47:0/3276907825 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5612c2192a50 con 0x5612c21fb360 2024-03-21T11:30:05.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.175+0000 7f81bf400640 1 --2- 172.21.15.47:0/3276907825 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5612c21fb360 0x5612c21fd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81bf400640 1 --2- 172.21.15.47:0/3276907825 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5612c21fb360 0x5612c21fd750 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ffcdd858474f37 server_cookie=61d0d04ca0f50831 in_seq=0 out_seq=0 2024-03-21T11:30:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81ad600640 10 client.27794 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81ad600640 1 -- 172.21.15.47:0/3276907825 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5612c2192a50 con 0x5612c21fb360 2024-03-21T11:30:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81ad600640 10 client.27794 handle_command_reply: tid=0 2024-03-21T11:30:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81c97a3740 1 client.27794 shutdown 2024-03-21T11:30:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81c97a3740 1 -- 172.21.15.47:0/3276907825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 msgr2=0x5612c2199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3276907825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2199da0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009ab0 tx=0x7f81b8009310 comp rx=0 tx=0).stop 2024-03-21T11:30:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81c97a3740 1 -- 172.21.15.47:0/3276907825 shutdown_connections 2024-03-21T11:30:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3276907825 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5612c21fb360 0x5612c21fd750 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3276907825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612c218a580 0x5612c21988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3276907825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612c2189bc0 0x5612c21983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81ad600640 0 client.27794 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:05.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81c97a3740 1 --2- 172.21.15.47:0/3276907825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612c21889f0 0x5612c2199da0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81c97a3740 1 -- 172.21.15.47:0/3276907825 >> 172.21.15.47:0/3276907825 conn(0x5612c2183de0 msgr2=0x5612c21878d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:05.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.176+0000 7f81c97a3740 1 -- 172.21.15.47:0/3276907825 shutdown_connections 2024-03-21T11:30:05.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.177+0000 7f81c97a3740 1 -- 172.21.15.47:0/3276907825 wait complete. 2024-03-21T11:30:05.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.177+0000 7f81c97a3740 20 client.27794 trim_cache size 0 max 16384 2024-03-21T11:30:05.182 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:30:05.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.178+0000 7f81c5400640 1 -- 172.21.15.47:0/59700166 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81a00a2fa0 msgr2=0x7f81a00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:05.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.178+0000 7f81c5400640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81a00a2fa0 0x7f81a00a5460 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f81b00cd720 tx=0x7f81b00d3a90 comp rx=0 tx=0).stop 2024-03-21T11:30:05.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.178+0000 7f81c5400640 1 -- 172.21.15.47:0/59700166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81c01037a0 msgr2=0x7f81c0199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:05.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.178+0000 7f81c5400640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81c01037a0 0x7f81c0199a90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f81b000ebc0 tx=0x7f81b000c6a0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.178+0000 7f81c5400640 1 -- 172.21.15.47:0/59700166 shutdown_connections 2024-03-21T11:30:05.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.178+0000 7f81c5400640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81a00a2fa0 0x7f81a00a5460 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.178+0000 7f81c5400640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c0105350 0x7f81c01a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.178+0000 7f81c5400640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c0104990 0x7f81c0199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.178+0000 7f81c5400640 1 --2- 172.21.15.47:0/59700166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81c01037a0 0x7f81c0199a90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:05.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.178+0000 7f81c5400640 1 -- 172.21.15.47:0/59700166 >> 172.21.15.47:0/59700166 conn(0x7f81c00fed10 msgr2=0x7f81c0107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:05.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.178+0000 7f81c5400640 1 -- 172.21.15.47:0/59700166 shutdown_connections 2024-03-21T11:30:05.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:05.179+0000 7f81c5400640 1 -- 172.21.15.47:0/59700166 wait complete. 2024-03-21T11:30:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:04 smithi102 ceph-mon[36748]: pgmap v602: 129 pgs: 129 active+clean; 41 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 293 op/s 2024-03-21T11:30:05.525 INFO:tasks.workunit.client.0.smithi047.stdout: 1 452104 8.19 MB/sec execute 169 sec latency 18.536 ms 2024-03-21T11:30:06.199 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:30:06.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:30:06.200 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:30:06.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.436+0000 7f81b7e00640 1 -- 172.21.15.47:0/1005394947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b01005a0 msgr2=0x7f81b01009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.436+0000 7f81b7e00640 1 --2- 172.21.15.47:0/1005394947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b01005a0 0x7f81b01009c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009ca0 tx=0x7f81a402c0f0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.436+0000 7f81b7e00640 1 -- 172.21.15.47:0/1005394947 shutdown_connections 2024-03-21T11:30:06.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.436+0000 7f81b7e00640 1 --2- 172.21.15.47:0/1005394947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b0102150 0x7f81b01089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.436+0000 7f81b7e00640 1 --2- 172.21.15.47:0/1005394947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81b0101790 0x7f81b0101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.436+0000 7f81b7e00640 1 --2- 172.21.15.47:0/1005394947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b01005a0 0x7f81b01009c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.436+0000 7f81b7e00640 1 -- 172.21.15.47:0/1005394947 >> 172.21.15.47:0/1005394947 conn(0x7f81b00fbb10 msgr2=0x7f81b00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:06.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.436+0000 7f81b7e00640 1 -- 172.21.15.47:0/1005394947 shutdown_connections 2024-03-21T11:30:06.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.436+0000 7f81b7e00640 1 -- 172.21.15.47:0/1005394947 wait complete. 2024-03-21T11:30:06.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.437+0000 7f81b7e00640 1 Processor -- start 2024-03-21T11:30:06.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.438+0000 7f81b7e00640 1 -- start start 2024-03-21T11:30:06.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.438+0000 7f81b7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b01005a0 0x7f81b0199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.438+0000 7f81b7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b0101790 0x7f81b019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.438+0000 7f81b7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81b0102150 0x7f81b01a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.438+0000 7f81b7e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b010c230 con 0x7f81b01005a0 2024-03-21T11:30:06.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.438+0000 7f81b7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b010c0b0 con 0x7f81b0101790 2024-03-21T11:30:06.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.438+0000 7f81b7e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b010c3b0 con 0x7f81b0102150 2024-03-21T11:30:06.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.438+0000 7f81b4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b0101790 0x7f81b019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.438+0000 7f81b4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b0101790 0x7f81b019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52324/0 (socket says 172.21.15.47:52324) 2024-03-21T11:30:06.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.438+0000 7f81b4a00640 1 -- 172.21.15.47:0/703432807 learned_addr learned my addr 172.21.15.47:0/703432807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.438+0000 7f81b5e00640 1 --2- 172.21.15.47:0/703432807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81b0102150 0x7f81b01a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.438+0000 7f81b5400640 1 --2- 172.21.15.47:0/703432807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b01005a0 0x7f81b0199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.439+0000 7f81b4a00640 1 -- 172.21.15.47:0/703432807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81b0102150 msgr2=0x7f81b01a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.439+0000 7f81b4a00640 1 --2- 172.21.15.47:0/703432807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81b0102150 0x7f81b01a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.439+0000 7f81b4a00640 1 -- 172.21.15.47:0/703432807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b01005a0 msgr2=0x7f81b0199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.439+0000 7f81b4a00640 1 --2- 172.21.15.47:0/703432807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b01005a0 0x7f81b0199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.439+0000 7f81b4a00640 1 -- 172.21.15.47:0/703432807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b01a1850 con 0x7f81b0101790 2024-03-21T11:30:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.439+0000 7f81b4a00640 1 --2- 172.21.15.47:0/703432807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b0101790 0x7f81b019a180 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f819c00ea60 tx=0x7f819c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:06.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.439+0000 7f81ae000640 1 -- 172.21.15.47:0/703432807 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c00cde0 con 0x7f81b0101790 2024-03-21T11:30:06.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.440+0000 7f81ae000640 1 -- 172.21.15.47:0/703432807 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f819c0035f0 con 0x7f81b0101790 2024-03-21T11:30:06.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.440+0000 7f81ae000640 1 -- 172.21.15.47:0/703432807 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c010670 con 0x7f81b0101790 2024-03-21T11:30:06.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.440+0000 7f81b7e00640 1 -- 172.21.15.47:0/703432807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b01a1b40 con 0x7f81b0101790 2024-03-21T11:30:06.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.440+0000 7f81b7e00640 1 -- 172.21.15.47:0/703432807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b01a2010 con 0x7f81b0101790 2024-03-21T11:30:06.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.444+0000 7f81bab78740 1 Processor -- start 2024-03-21T11:30:06.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.445+0000 7f81bab78740 1 -- start start 2024-03-21T11:30:06.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.445+0000 7f81ae000640 1 -- 172.21.15.47:0/703432807 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f819c0037a0 con 0x7f81b0101790 2024-03-21T11:30:06.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.445+0000 7f81bab78740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9b0a0 0x55fe36c9b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.445+0000 7f81bab78740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9ba90 0x55fe36ca0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.445+0000 7f81bab78740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36ca0c00 0x55fe36ca3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.445+0000 7f81bab78740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe36c00eb0 con 0x55fe36c9b0a0 2024-03-21T11:30:06.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.445+0000 7f81bab78740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe36ba9a60 con 0x55fe36ca0c00 2024-03-21T11:30:06.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.445+0000 7f81bab78740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe36bfcf50 con 0x55fe36c9ba90 2024-03-21T11:30:06.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.445+0000 7f81b5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9ba90 0x55fe36ca0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.445+0000 7f81b5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9ba90 0x55fe36ca0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60036/0 (socket says 172.21.15.47:60036) 2024-03-21T11:30:06.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.445+0000 7f81b5400640 1 -- 172.21.15.47:0/50104295 learned_addr learned my addr 172.21.15.47:0/50104295 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:06.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.445+0000 7f81b4a00640 1 --2- 172.21.15.47:0/50104295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36ca0c00 0x55fe36ca3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.445+0000 7f81b5e00640 1 --2- 172.21.15.47:0/50104295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9b0a0 0x55fe36c9b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.446+0000 7f81b5400640 1 -- 172.21.15.47:0/50104295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36ca0c00 msgr2=0x55fe36ca3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.446+0000 7f81b5400640 1 --2- 172.21.15.47:0/50104295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36ca0c00 0x55fe36ca3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.446+0000 7f81b5400640 1 -- 172.21.15.47:0/50104295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9b0a0 msgr2=0x55fe36c9b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.446+0000 7f81b5400640 1 --2- 172.21.15.47:0/50104295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9b0a0 0x55fe36c9b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.446+0000 7f81b5400640 1 -- 172.21.15.47:0/50104295 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe369dd7e0 con 0x55fe36c9ba90 2024-03-21T11:30:06.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.446+0000 7f81b5400640 1 --2- 172.21.15.47:0/50104295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9ba90 0x55fe36ca0590 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f81a402c700 tx=0x7f81a4005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61f347f84e286267 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:06.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.446+0000 7f81a2600640 1 -- 172.21.15.47:0/50104295 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4043070 con 0x55fe36c9ba90 2024-03-21T11:30:06.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.447+0000 7f81a2600640 1 -- 172.21.15.47:0/50104295 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81a4032c90 con 0x55fe36c9ba90 2024-03-21T11:30:06.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.447+0000 7f81a2600640 1 -- 172.21.15.47:0/50104295 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4039040 con 0x55fe36c9ba90 2024-03-21T11:30:06.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.447+0000 7f81ae000640 1 --2- 172.21.15.47:0/703432807 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81980a2fa0 0x7f81980a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.447+0000 7f81ae000640 1 -- 172.21.15.47:0/703432807 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f819c0cfae0 con 0x7f81b0101790 2024-03-21T11:30:06.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.447+0000 7f81b5e00640 1 --2- 172.21.15.47:0/703432807 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81980a2fa0 0x7f81980a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.447+0000 7f81bab78740 1 -- 172.21.15.47:0/50104295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9ba90 msgr2=0x55fe36ca0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.447+0000 7f81bab78740 1 --2- 172.21.15.47:0/50104295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9ba90 0x55fe36ca0590 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f81a402c700 tx=0x7f81a4005ef0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.448+0000 7f81bab78740 1 -- 172.21.15.47:0/50104295 shutdown_connections 2024-03-21T11:30:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.448+0000 7f81bab78740 1 --2- 172.21.15.47:0/50104295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36ca0c00 0x55fe36ca3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.448+0000 7f81bab78740 1 --2- 172.21.15.47:0/50104295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9ba90 0x55fe36ca0590 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.448+0000 7f81bab78740 1 --2- 172.21.15.47:0/50104295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9b0a0 0x55fe36c9b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.448+0000 7f81bab78740 1 -- 172.21.15.47:0/50104295 >> 172.21.15.47:0/50104295 conn(0x55fe36c96610 msgr2=0x55fe36c96a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.448+0000 7f81bab78740 1 -- 172.21.15.47:0/50104295 shutdown_connections 2024-03-21T11:30:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.448+0000 7f81bab78740 1 -- 172.21.15.47:0/50104295 wait complete. 2024-03-21T11:30:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.448+0000 7f81b5e00640 1 --2- 172.21.15.47:0/703432807 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81980a2fa0 0x7f81980a5460 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x55fe36c9c2f0 tx=0x7f81a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.448+0000 7f81bab78740 1 -- start start 2024-03-21T11:30:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.448+0000 7f81bab78740 1 Processor -- start 2024-03-21T11:30:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.449+0000 7f81bab78740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9b0a0 0x55fe36ca5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.449+0000 7f81bab78740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9ba90 0x55fe36ca57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.449+0000 7f81bab78740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36ca0c00 0x55fe36cac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.449+0000 7f81bab78740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe36ca3b50 con 0x55fe36c9b0a0 2024-03-21T11:30:06.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.449+0000 7f81bab78740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe36bbceb0 con 0x55fe36ca0c00 2024-03-21T11:30:06.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.449+0000 7f81bab78740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe36ca3cd0 con 0x55fe36c9ba90 2024-03-21T11:30:06.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.449+0000 7f81bab78740 10 client.0 _ll_register_callbacks cb 0x55fe36bb1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:06.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.449+0000 7f81b5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36ca0c00 0x55fe36cac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.449+0000 7f81b5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9b0a0 0x55fe36ca5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.449+0000 7f81b5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9b0a0 0x55fe36ca5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48870/0 (socket says 172.21.15.47:48870) 2024-03-21T11:30:06.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.449+0000 7f81b5400640 1 -- 172.21.15.47:0/3125918757 learned_addr learned my addr 172.21.15.47:0/3125918757 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:06.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.449+0000 7f81b5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36ca0c00 0x55fe36cac770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52336/0 (socket says 172.21.15.47:52336) 2024-03-21T11:30:06.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.449+0000 7f81b4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9ba90 0x55fe36ca57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.450+0000 7f81b4a00640 1 -- 172.21.15.47:0/3125918757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36ca0c00 msgr2=0x55fe36cac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.450+0000 7f81b4a00640 1 --2- 172.21.15.47:0/3125918757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36ca0c00 0x55fe36cac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.450+0000 7f81b4a00640 1 -- 172.21.15.47:0/3125918757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9b0a0 msgr2=0x55fe36ca5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.450+0000 7f81b4a00640 1 --2- 172.21.15.47:0/3125918757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9b0a0 0x55fe36ca5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.450+0000 7f81b4a00640 1 -- 172.21.15.47:0/3125918757 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe36a3df70 con 0x55fe36c9ba90 2024-03-21T11:30:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.450+0000 7f81b4a00640 1 --2- 172.21.15.47:0/3125918757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9ba90 0x55fe36ca57a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f819c01f000 tx=0x7f819c098030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e044e7a09675ce13 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.450+0000 7f8196a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.450+0000 7f8196a00640 1 -- 172.21.15.47:0/3125918757 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c09b960 con 0x55fe36c9ba90 2024-03-21T11:30:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.450+0000 7f81bab78740 1 -- 172.21.15.47:0/3125918757 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81a4032c90 con 0x55fe36c9ba90 2024-03-21T11:30:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.450+0000 7f8196a00640 1 -- 172.21.15.47:0/3125918757 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f819c0bab90 con 0x55fe36c9ba90 2024-03-21T11:30:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.451+0000 7f8196a00640 1 -- 172.21.15.47:0/3125918757 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f819c0ba050 con 0x55fe36c9ba90 2024-03-21T11:30:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.451+0000 7f81bab78740 10 client.27823 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.451+0000 7f81bab78740 1 -- 172.21.15.47:0/3125918757 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55fe368d5170 con 0x55fe36c9ba90 2024-03-21T11:30:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.451+0000 7f8196a00640 1 -- 172.21.15.47:0/3125918757 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81a4032c90 con 0x55fe36c9ba90 2024-03-21T11:30:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.451+0000 7f8196a00640 1 -- 172.21.15.47:0/3125918757 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f819c09b270 con 0x55fe36c9ba90 2024-03-21T11:30:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.451+0000 7f81bab78740 10 client.27823 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.451+0000 7f81bab78740 10 client.27823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.451+0000 7f81bab78740 20 client.27823 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.451+0000 7f81bab78740 1 --2- 172.21.15.47:0/3125918757 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe36cb1030 0x55fe36cb34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.451+0000 7f81bab78740 4 client.27823 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.451+0000 7f81bab78740 1 -- 172.21.15.47:0/3125918757 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe36c0e3f0 con 0x55fe36cb1030 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.452+0000 7f81b5400640 1 --2- 172.21.15.47:0/3125918757 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe36cb1030 0x55fe36cb34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.452+0000 7f81b5400640 1 --2- 172.21.15.47:0/3125918757 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe36cb1030 0x55fe36cb34b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=897c72476c75f7bd server_cookie=8579fd2c01d348e in_seq=0 out_seq=0 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.452+0000 7f8196a00640 10 client.27823 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.453+0000 7f8196a00640 1 -- 172.21.15.47:0/3125918757 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fe36c0e3f0 con 0x55fe36cb1030 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.453+0000 7f8196a00640 10 client.27823 handle_command_reply: tid=0 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.453+0000 7f81bab78740 1 client.27823 shutdown 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.453+0000 7f81bab78740 1 -- 172.21.15.47:0/3125918757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9ba90 msgr2=0x55fe36ca57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.453+0000 7f81bab78740 1 --2- 172.21.15.47:0/3125918757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9ba90 0x55fe36ca57a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f819c01f000 tx=0x7f819c098030 comp rx=0 tx=0).stop 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.453+0000 7f81bab78740 1 -- 172.21.15.47:0/3125918757 shutdown_connections 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.453+0000 7f81bab78740 1 --2- 172.21.15.47:0/3125918757 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe36cb1030 0x55fe36cb34b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.453+0000 7f81bab78740 1 --2- 172.21.15.47:0/3125918757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36ca0c00 0x55fe36cac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.453+0000 7f81bab78740 1 --2- 172.21.15.47:0/3125918757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9ba90 0x55fe36ca57a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.453+0000 7f81bab78740 1 --2- 172.21.15.47:0/3125918757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9b0a0 0x55fe36ca5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.453+0000 7f81bab78740 1 -- 172.21.15.47:0/3125918757 >> 172.21.15.47:0/3125918757 conn(0x55fe36c96610 msgr2=0x55fe36c95830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:06.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.453+0000 7f8196a00640 0 client.27823 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:06.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.453+0000 7f81bab78740 1 -- 172.21.15.47:0/3125918757 shutdown_connections 2024-03-21T11:30:06.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.454+0000 7f81bab78740 1 -- 172.21.15.47:0/3125918757 wait complete. 2024-03-21T11:30:06.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.454+0000 7f81bab78740 20 client.27823 trim_cache size 0 max 16384 2024-03-21T11:30:06.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.473+0000 7f81bab78740 1 Processor -- start 2024-03-21T11:30:06.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.473+0000 7f81bab78740 1 -- start start 2024-03-21T11:30:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.474+0000 7f81bab78740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36c96610 0x55fe36cb7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.474+0000 7f81bab78740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9b0a0 0x55fe36cac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.474+0000 7f81bab78740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9ba90 0x55fe36cb82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.474+0000 7f81bab78740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b01ac320 con 0x55fe36c9ba90 2024-03-21T11:30:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.474+0000 7f81bab78740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b01adca0 con 0x55fe36c96610 2024-03-21T11:30:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.474+0000 7f81bab78740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b01ac980 con 0x55fe36c9b0a0 2024-03-21T11:30:06.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.474+0000 7f81b5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36c96610 0x55fe36cb7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.474+0000 7f81b5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36c96610 0x55fe36cb7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52348/0 (socket says 172.21.15.47:52348) 2024-03-21T11:30:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.474+0000 7f81b5400640 1 -- 172.21.15.47:0/487101337 learned_addr learned my addr 172.21.15.47:0/487101337 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.474+0000 7f81b4a00640 1 --2- 172.21.15.47:0/487101337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9b0a0 0x55fe36cac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.474+0000 7f81b5e00640 1 --2- 172.21.15.47:0/487101337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9ba90 0x55fe36cb82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.475+0000 7f81b5400640 1 -- 172.21.15.47:0/487101337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9b0a0 msgr2=0x55fe36cac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.475+0000 7f81b5400640 1 --2- 172.21.15.47:0/487101337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9b0a0 0x55fe36cac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.475+0000 7f81b5400640 1 -- 172.21.15.47:0/487101337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9ba90 msgr2=0x55fe36cb82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.475+0000 7f81b5400640 1 --2- 172.21.15.47:0/487101337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9ba90 0x55fe36cb82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.475+0000 7f81b5400640 1 -- 172.21.15.47:0/487101337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe36c91c20 con 0x55fe36c96610 2024-03-21T11:30:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.475+0000 7f81b5e00640 1 --2- 172.21.15.47:0/487101337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9ba90 0x55fe36cb82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:30:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.475+0000 7f81b5400640 1 --2- 172.21.15.47:0/487101337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36c96610 0x55fe36cb7d70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f81a402c700 tx=0x7f81a4009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a269be678f02b434 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.475+0000 7f81a1c00640 1 -- 172.21.15.47:0/487101337 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4056070 con 0x55fe36c96610 2024-03-21T11:30:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.475+0000 7f81a1c00640 1 -- 172.21.15.47:0/487101337 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81a402cca0 con 0x55fe36c96610 2024-03-21T11:30:06.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.475+0000 7f81a1c00640 1 -- 172.21.15.47:0/487101337 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4039040 con 0x55fe36c96610 2024-03-21T11:30:06.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.476+0000 7f81bab78740 1 -- 172.21.15.47:0/487101337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36c96610 msgr2=0x55fe36cb7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.476+0000 7f81bab78740 1 --2- 172.21.15.47:0/487101337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36c96610 0x55fe36cb7d70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f81a402c700 tx=0x7f81a4009360 comp rx=0 tx=0).stop 2024-03-21T11:30:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.476+0000 7f81bab78740 1 -- 172.21.15.47:0/487101337 shutdown_connections 2024-03-21T11:30:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.476+0000 7f81bab78740 1 --2- 172.21.15.47:0/487101337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c9ba90 0x55fe36cb82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.476+0000 7f81bab78740 1 --2- 172.21.15.47:0/487101337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9b0a0 0x55fe36cac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.476+0000 7f81bab78740 1 --2- 172.21.15.47:0/487101337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36c96610 0x55fe36cb7d70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.476+0000 7f81bab78740 1 -- 172.21.15.47:0/487101337 >> 172.21.15.47:0/487101337 conn(0x55fe36c9a4a0 msgr2=0x55fe36c95830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.476+0000 7f81bab78740 1 -- 172.21.15.47:0/487101337 shutdown_connections 2024-03-21T11:30:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.476+0000 7f81bab78740 1 -- 172.21.15.47:0/487101337 wait complete. 2024-03-21T11:30:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.477+0000 7f81bab78740 1 -- start start 2024-03-21T11:30:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.477+0000 7f81bab78740 1 Processor -- start 2024-03-21T11:30:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.477+0000 7f81bab78740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c96610 0x55fe36cab950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.477+0000 7f81bab78740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9b0a0 0x55fe36cabe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.477+0000 7f81bab78740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36c9ba90 0x55fe36d152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.477+0000 7f81bab78740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b01ad640 con 0x55fe36c96610 2024-03-21T11:30:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.477+0000 7f81bab78740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b01acfe0 con 0x55fe36c9ba90 2024-03-21T11:30:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.477+0000 7f81bab78740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b01aa4e0 con 0x55fe36c9b0a0 2024-03-21T11:30:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.477+0000 7f81bab78740 10 client.0 _ll_register_callbacks cb 0x7f819c00e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.478+0000 7f81b5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c96610 0x55fe36cab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.478+0000 7f81b5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36c9ba90 0x55fe36d152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.478+0000 7f81b5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c96610 0x55fe36cab950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48898/0 (socket says 172.21.15.47:48898) 2024-03-21T11:30:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.478+0000 7f81b5400640 1 -- 172.21.15.47:0/2473653195 learned_addr learned my addr 172.21.15.47:0/2473653195 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.478+0000 7f81b4a00640 1 --2- 172.21.15.47:0/2473653195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9b0a0 0x55fe36cabe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.478+0000 7f81b5400640 1 -- 172.21.15.47:0/2473653195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9b0a0 msgr2=0x55fe36cabe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.478+0000 7f81b5400640 1 --2- 172.21.15.47:0/2473653195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9b0a0 0x55fe36cabe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.478+0000 7f81b5400640 1 -- 172.21.15.47:0/2473653195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36c9ba90 msgr2=0x55fe36d152f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.478+0000 7f81b5400640 1 --2- 172.21.15.47:0/2473653195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36c9ba90 0x55fe36d152f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.478+0000 7f81b5400640 1 -- 172.21.15.47:0/2473653195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe36d15900 con 0x55fe36c96610 2024-03-21T11:30:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.478+0000 7f81b5400640 1 --2- 172.21.15.47:0/2473653195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c96610 0x55fe36cab950 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f81a402c630 tx=0x7f81a4005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad9232c3fc70f47 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.478+0000 7f8196a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:30:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.479+0000 7f8196a00640 1 -- 172.21.15.47:0/2473653195 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4056020 con 0x55fe36c96610 2024-03-21T11:30:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.479+0000 7f81bab78740 1 -- 172.21.15.47:0/2473653195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81a402cca0 con 0x55fe36c96610 2024-03-21T11:30:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.479+0000 7f8196a00640 1 -- 172.21.15.47:0/2473653195 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f81a4032690 con 0x55fe36c96610 2024-03-21T11:30:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.479+0000 7f8196a00640 1 -- 172.21.15.47:0/2473653195 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81a4039040 con 0x55fe36c96610 2024-03-21T11:30:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.479+0000 7f81bab78740 10 client.17523 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.479+0000 7f81bab78740 1 -- 172.21.15.47:0/2473653195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55fe36d16bb0 con 0x55fe36c96610 2024-03-21T11:30:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.479+0000 7f8196a00640 1 -- 172.21.15.47:0/2473653195 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81a402cca0 con 0x55fe36c96610 2024-03-21T11:30:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.480+0000 7f8196a00640 1 -- 172.21.15.47:0/2473653195 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f81a4048730 con 0x55fe36c96610 2024-03-21T11:30:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.480+0000 7f81bab78740 10 client.17523 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.480+0000 7f81bab78740 10 client.17523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.480+0000 7f81bab78740 20 client.17523 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.480+0000 7f81bab78740 1 --2- 172.21.15.47:0/2473653195 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe36ca0c00 0x55fe36d19180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.480+0000 7f81bab78740 4 client.17523 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:30:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.480+0000 7f81bab78740 1 -- 172.21.15.47:0/2473653195 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fe36c08740 con 0x55fe36ca0c00 2024-03-21T11:30:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.480+0000 7f81b4a00640 1 --2- 172.21.15.47:0/2473653195 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe36ca0c00 0x55fe36d19180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.480+0000 7f81b4a00640 1 --2- 172.21.15.47:0/2473653195 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe36ca0c00 0x55fe36d19180 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=177a7bb476b57969 server_cookie=17caee42b895826d in_seq=0 out_seq=0 2024-03-21T11:30:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.480+0000 7f8196a00640 10 client.17523 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.483+0000 7f8196a00640 1 -- 172.21.15.47:0/2473653195 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fe36c08740 con 0x55fe36ca0c00 2024-03-21T11:30:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.483+0000 7f8196a00640 10 client.17523 handle_command_reply: tid=0 2024-03-21T11:30:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.483+0000 7f81bab78740 1 client.17523 shutdown 2024-03-21T11:30:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.483+0000 7f81bab78740 1 -- 172.21.15.47:0/2473653195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c96610 msgr2=0x55fe36cab950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.483+0000 7f81bab78740 1 --2- 172.21.15.47:0/2473653195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c96610 0x55fe36cab950 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f81a402c630 tx=0x7f81a4005f70 comp rx=0 tx=0).stop 2024-03-21T11:30:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.483+0000 7f81bab78740 1 -- 172.21.15.47:0/2473653195 shutdown_connections 2024-03-21T11:30:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.483+0000 7f81bab78740 1 --2- 172.21.15.47:0/2473653195 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe36ca0c00 0x55fe36d19180 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.483+0000 7f81bab78740 1 --2- 172.21.15.47:0/2473653195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe36c9ba90 0x55fe36d152f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.483+0000 7f81bab78740 1 --2- 172.21.15.47:0/2473653195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe36c9b0a0 0x55fe36cabe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.483+0000 7f81bab78740 1 --2- 172.21.15.47:0/2473653195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe36c96610 0x55fe36cab950 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.483+0000 7f8196a00640 0 client.17523 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:06.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.483+0000 7f81bab78740 1 -- 172.21.15.47:0/2473653195 >> 172.21.15.47:0/2473653195 conn(0x55fe36c9a4a0 msgr2=0x55fe36c9c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:06.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.484+0000 7f81bab78740 1 -- 172.21.15.47:0/2473653195 shutdown_connections 2024-03-21T11:30:06.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.484+0000 7f81bab78740 1 -- 172.21.15.47:0/2473653195 wait complete. 2024-03-21T11:30:06.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.484+0000 7f81bab78740 20 client.17523 trim_cache size 0 max 16384 2024-03-21T11:30:06.486 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:30:06.487 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:30:06.487 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "4ad9ce15-8e03-4595-88c5-760c5e5b0a15", 2024-03-21T11:30:06.487 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:30:06.487 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:30:06.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.484+0000 7f81b7e00640 1 -- 172.21.15.47:0/703432807 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81980a2fa0 msgr2=0x7f81980a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.484+0000 7f81b7e00640 1 --2- 172.21.15.47:0/703432807 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81980a2fa0 0x7f81980a5460 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x55fe36c9c2f0 tx=0x7f81a8005f90 comp rx=0 tx=0).stop 2024-03-21T11:30:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.484+0000 7f81b7e00640 1 -- 172.21.15.47:0/703432807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b0101790 msgr2=0x7f81b019a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.484+0000 7f81b7e00640 1 --2- 172.21.15.47:0/703432807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b0101790 0x7f81b019a180 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f819c00ea60 tx=0x7f819c00ef20 comp rx=0 tx=0).stop 2024-03-21T11:30:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.484+0000 7f81b7e00640 1 -- 172.21.15.47:0/703432807 shutdown_connections 2024-03-21T11:30:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.484+0000 7f81b7e00640 1 --2- 172.21.15.47:0/703432807 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81980a2fa0 0x7f81980a5460 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.484+0000 7f81b7e00640 1 --2- 172.21.15.47:0/703432807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81b0102150 0x7f81b01a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.484+0000 7f81b7e00640 1 --2- 172.21.15.47:0/703432807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b0101790 0x7f81b019a180 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.485+0000 7f81b7e00640 1 --2- 172.21.15.47:0/703432807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b01005a0 0x7f81b0199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.485+0000 7f81b7e00640 1 -- 172.21.15.47:0/703432807 >> 172.21.15.47:0/703432807 conn(0x7f81b00fbb10 msgr2=0x7f81b0069cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.485+0000 7f81b7e00640 1 -- 172.21.15.47:0/703432807 shutdown_connections 2024-03-21T11:30:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.485+0000 7f81b7e00640 1 -- 172.21.15.47:0/703432807 wait complete. 2024-03-21T11:30:06.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:30:06.504 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:30:06.525 INFO:tasks.workunit.client.0.smithi047.stdout: 1 454121 8.21 MB/sec execute 170 sec latency 13.295 ms 2024-03-21T11:30:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.738+0000 7fd2f0e00640 1 -- 172.21.15.47:0/3903457377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2ec10ea10 msgr2=0x7fd2ec110ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.738+0000 7fd2f0e00640 1 --2- 172.21.15.47:0/3903457377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2ec10ea10 0x7fd2ec110ed0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009900 tx=0x7fd2e402bc40 comp rx=0 tx=0).stop 2024-03-21T11:30:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.738+0000 7fd2f0e00640 1 -- 172.21.15.47:0/3903457377 shutdown_connections 2024-03-21T11:30:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.738+0000 7fd2f0e00640 1 --2- 172.21.15.47:0/3903457377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2ec10ea10 0x7fd2ec110ed0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.738+0000 7fd2f0e00640 1 --2- 172.21.15.47:0/3903457377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec10c0e0 0x7fd2ec10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.738+0000 7fd2f0e00640 1 --2- 172.21.15.47:0/3903457377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec1097b0 0x7fd2ec10bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.738+0000 7fd2f0e00640 1 -- 172.21.15.47:0/3903457377 >> 172.21.15.47:0/3903457377 conn(0x7fd2ec06e8a0 msgr2=0x7fd2ec070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.738+0000 7fd2f0e00640 1 -- 172.21.15.47:0/3903457377 shutdown_connections 2024-03-21T11:30:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.738+0000 7fd2f0e00640 1 -- 172.21.15.47:0/3903457377 wait complete. 2024-03-21T11:30:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.739+0000 7fd2f0e00640 1 Processor -- start 2024-03-21T11:30:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.739+0000 7fd2f0e00640 1 -- start start 2024-03-21T11:30:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2f0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2ec1097b0 0x7fd2ec10b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2f0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec10c0e0 0x7fd2ec109d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2f0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec10ea10 0x7fd2ec10a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2f0e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec114a70 con 0x7fd2ec10c0e0 2024-03-21T11:30:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2f0e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec1148f0 con 0x7fd2ec10ea10 2024-03-21T11:30:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2f0e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec114bf0 con 0x7fd2ec1097b0 2024-03-21T11:30:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2e9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2ec1097b0 0x7fd2ec10b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2e9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2ec1097b0 0x7fd2ec10b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60080/0 (socket says 172.21.15.47:60080) 2024-03-21T11:30:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2e9e00640 1 -- 172.21.15.47:0/3197672638 learned_addr learned my addr 172.21.15.47:0/3197672638 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2ea800640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec10ea10 0x7fd2ec10a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2e9400640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec10c0e0 0x7fd2ec109d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2e9e00640 1 -- 172.21.15.47:0/3197672638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec10ea10 msgr2=0x7fd2ec10a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2e9e00640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec10ea10 0x7fd2ec10a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2e9e00640 1 -- 172.21.15.47:0/3197672638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec10c0e0 msgr2=0x7fd2ec109d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2e9e00640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec10c0e0 0x7fd2ec109d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2e9e00640 1 -- 172.21.15.47:0/3197672638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec10ab20 con 0x7fd2ec1097b0 2024-03-21T11:30:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.740+0000 7fd2e9400640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec10c0e0 0x7fd2ec109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.741+0000 7fd2e9e00640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2ec1097b0 0x7fd2ec10b710 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009d50 tx=0x7fd2dc00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.741+0000 7fd2e2a00640 1 -- 172.21.15.47:0/3197672638 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc003b70 con 0x7fd2ec1097b0 2024-03-21T11:30:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.741+0000 7fd2e2a00640 1 -- 172.21.15.47:0/3197672638 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2dc003d10 con 0x7fd2ec1097b0 2024-03-21T11:30:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.741+0000 7fd2e2a00640 1 -- 172.21.15.47:0/3197672638 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc010a20 con 0x7fd2ec1097b0 2024-03-21T11:30:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.741+0000 7fd2f0e00640 1 -- 172.21.15.47:0/3197672638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ec1aa290 con 0x7fd2ec1097b0 2024-03-21T11:30:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.741+0000 7fd2f0e00640 1 -- 172.21.15.47:0/3197672638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec1aa790 con 0x7fd2ec1097b0 2024-03-21T11:30:06.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.746+0000 7fd2f3b81740 1 Processor -- start 2024-03-21T11:30:06.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.746+0000 7fd2f3b81740 1 -- start start 2024-03-21T11:30:06.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.746+0000 7fd2e2a00640 1 -- 172.21.15.47:0/3197672638 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd2dc003ec0 con 0x7fd2ec1097b0 2024-03-21T11:30:06.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.746+0000 7fd2f3b81740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.746+0000 7fd2f3b81740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2f0870 0x55c8eb2f5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.746+0000 7fd2f3b81740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb2f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.746+0000 7fd2f3b81740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8eb255250 con 0x55c8eb2efe80 2024-03-21T11:30:06.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.746+0000 7fd2f3b81740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8eb1fda30 con 0x55c8eb2f59e0 2024-03-21T11:30:06.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.746+0000 7fd2f3b81740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8eb2512f0 con 0x55c8eb2f0870 2024-03-21T11:30:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.747+0000 7fd2e9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2f0870 0x55c8eb2f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.747+0000 7fd2ea800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.747+0000 7fd2e9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb2f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.747+0000 7fd2ea800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48926/0 (socket says 172.21.15.47:48926) 2024-03-21T11:30:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.747+0000 7fd2ea800640 1 -- 172.21.15.47:0/173918619 learned_addr learned my addr 172.21.15.47:0/173918619 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.747+0000 7fd2e9400640 1 -- 172.21.15.47:0/173918619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2f0870 msgr2=0x55c8eb2f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.747+0000 7fd2e9400640 1 --2- 172.21.15.47:0/173918619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2f0870 0x55c8eb2f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.747+0000 7fd2e9400640 1 -- 172.21.15.47:0/173918619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2efe80 msgr2=0x55c8eb2f02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.747+0000 7fd2e9400640 1 --2- 172.21.15.47:0/173918619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.747+0000 7fd2e9400640 1 -- 172.21.15.47:0/173918619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8eb1e8690 con 0x55c8eb2f59e0 2024-03-21T11:30:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.748+0000 7fd2e9400640 1 --2- 172.21.15.47:0/173918619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb2f7e70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec073390 tx=0x7fd2d80117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64d9514bdb6cd203 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:06.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.748+0000 7fd2d7400640 1 -- 172.21.15.47:0/173918619 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8012450 con 0x55c8eb2f59e0 2024-03-21T11:30:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.748+0000 7fd2d7400640 1 -- 172.21.15.47:0/173918619 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2d8012ad0 con 0x55c8eb2f59e0 2024-03-21T11:30:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.748+0000 7fd2d7400640 1 -- 172.21.15.47:0/173918619 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d800fb60 con 0x55c8eb2f59e0 2024-03-21T11:30:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.748+0000 7fd2e2a00640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2d00a2fa0 0x7fd2d00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.748+0000 7fd2e2a00640 1 -- 172.21.15.47:0/3197672638 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd2dc0cfee0 con 0x7fd2ec1097b0 2024-03-21T11:30:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.748+0000 7fd2ea800640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2d00a2fa0 0x7fd2d00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.749+0000 7fd2f3b81740 1 -- 172.21.15.47:0/173918619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 msgr2=0x55c8eb2f7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.749+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/173918619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb2f7e70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec073390 tx=0x7fd2d80117b0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.749+0000 7fd2f3b81740 1 -- 172.21.15.47:0/173918619 shutdown_connections 2024-03-21T11:30:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.749+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/173918619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb2f7e70 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.749+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/173918619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2f0870 0x55c8eb2f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.749+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/173918619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.749+0000 7fd2f3b81740 1 -- 172.21.15.47:0/173918619 >> 172.21.15.47:0/173918619 conn(0x55c8eb2b8200 msgr2=0x55c8eb2eb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.749+0000 7fd2f3b81740 1 -- 172.21.15.47:0/173918619 shutdown_connections 2024-03-21T11:30:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.749+0000 7fd2ea800640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2d00a2fa0 0x7fd2d00a5460 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55c8eb2f10d0 tx=0x7fd2e402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.749+0000 7fd2f3b81740 1 -- 172.21.15.47:0/173918619 wait complete. 2024-03-21T11:30:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.750+0000 7fd2f3b81740 1 -- start start 2024-03-21T11:30:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.750+0000 7fd2f3b81740 1 Processor -- start 2024-03-21T11:30:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.750+0000 7fd2f3b81740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.750+0000 7fd2f3b81740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f0870 0x55c8eb2fa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.750+0000 7fd2f3b81740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2f59e0 0x55c8eb301380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.750+0000 7fd2f3b81740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8eb2f8990 con 0x55c8eb2f59e0 2024-03-21T11:30:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.750+0000 7fd2f3b81740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8eb210e80 con 0x55c8eb2f0870 2024-03-21T11:30:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.750+0000 7fd2f3b81740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8eb2f8b10 con 0x55c8eb2efe80 2024-03-21T11:30:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.751+0000 7fd2e9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.751+0000 7fd2f3b81740 10 client.0 _ll_register_callbacks cb 0x55c8eb223130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.751+0000 7fd2e9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f0870 0x55c8eb2fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.751+0000 7fd2e9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60112/0 (socket says 172.21.15.47:60112) 2024-03-21T11:30:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.751+0000 7fd2e9e00640 1 -- 172.21.15.47:0/788520815 learned_addr learned my addr 172.21.15.47:0/788520815 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.751+0000 7fd2ea800640 1 --2- 172.21.15.47:0/788520815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2f59e0 0x55c8eb301380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.751+0000 7fd2e9e00640 1 -- 172.21.15.47:0/788520815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f0870 msgr2=0x55c8eb2fa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.751+0000 7fd2e9e00640 1 --2- 172.21.15.47:0/788520815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f0870 0x55c8eb2fa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.751+0000 7fd2e9e00640 1 -- 172.21.15.47:0/788520815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2f59e0 msgr2=0x55c8eb301380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.751+0000 7fd2e9e00640 1 --2- 172.21.15.47:0/788520815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2f59e0 0x55c8eb301380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.751+0000 7fd2e9e00640 1 -- 172.21.15.47:0/788520815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8eb0337d0 con 0x55c8eb2efe80 2024-03-21T11:30:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.751+0000 7fd2e9400640 1 --2- 172.21.15.47:0/788520815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f0870 0x55c8eb2fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.751+0000 7fd2e9e00640 1 --2- 172.21.15.47:0/788520815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f9e00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc01f000 tx=0x7fd2dc005f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76a1a80a03b972d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.751+0000 7fd2cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.752+0000 7fd2cbe00640 1 -- 172.21.15.47:0/788520815 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc09bdc0 con 0x55c8eb2efe80 2024-03-21T11:30:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.752+0000 7fd2cbe00640 1 -- 172.21.15.47:0/788520815 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd2dc0988f0 con 0x55c8eb2efe80 2024-03-21T11:30:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.752+0000 7fd2f3b81740 1 -- 172.21.15.47:0/788520815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2d8012ad0 con 0x55c8eb2efe80 2024-03-21T11:30:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.752+0000 7fd2cbe00640 1 -- 172.21.15.47:0/788520815 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2dc09bbe0 con 0x55c8eb2efe80 2024-03-21T11:30:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.752+0000 7fd2f3b81740 10 client.27841 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.752+0000 7fd2f3b81740 1 -- 172.21.15.47:0/788520815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c8eaf693f0 con 0x55c8eb2efe80 2024-03-21T11:30:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.752+0000 7fd2cbe00640 1 -- 172.21.15.47:0/788520815 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2d8012ad0 con 0x55c8eb2efe80 2024-03-21T11:30:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.752+0000 7fd2cbe00640 1 -- 172.21.15.47:0/788520815 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd2dc010bc0 con 0x55c8eb2efe80 2024-03-21T11:30:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.753+0000 7fd2f3b81740 10 client.27841 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.753+0000 7fd2f3b81740 10 client.27841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.753+0000 7fd2f3b81740 20 client.27841 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.753+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/788520815 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8eb305ce0 0x55c8eb308150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.753+0000 7fd2f3b81740 4 client.27841 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.753+0000 7fd2f3b81740 1 -- 172.21.15.47:0/788520815 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8eb262790 con 0x55c8eb305ce0 2024-03-21T11:30:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.753+0000 7fd2e9400640 1 --2- 172.21.15.47:0/788520815 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8eb305ce0 0x55c8eb308150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.754+0000 7fd2e9400640 1 --2- 172.21.15.47:0/788520815 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8eb305ce0 0x55c8eb308150 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eece145a7c15834 server_cookie=3754282a8ee217d6 in_seq=0 out_seq=0 2024-03-21T11:30:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.754+0000 7fd2cbe00640 10 client.27841 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.756+0000 7fd2cbe00640 1 -- 172.21.15.47:0/788520815 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c8eb262790 con 0x55c8eb305ce0 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.756+0000 7fd2cbe00640 10 client.27841 handle_command_reply: tid=0 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.756+0000 7fd2f3b81740 1 client.27841 shutdown 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.756+0000 7fd2f3b81740 1 -- 172.21.15.47:0/788520815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2efe80 msgr2=0x55c8eb2f9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.756+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/788520815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f9e00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc01f000 tx=0x7fd2dc005f30 comp rx=0 tx=0).stop 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.756+0000 7fd2f3b81740 1 -- 172.21.15.47:0/788520815 shutdown_connections 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.756+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/788520815 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8eb305ce0 0x55c8eb308150 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.756+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/788520815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2f59e0 0x55c8eb301380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.756+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/788520815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f0870 0x55c8eb2fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.756+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/788520815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f9e00 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.756+0000 7fd2f3b81740 1 -- 172.21.15.47:0/788520815 >> 172.21.15.47:0/788520815 conn(0x55c8eb2b8200 msgr2=0x55c8eb2eaa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.756+0000 7fd2cbe00640 0 client.27841 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.756+0000 7fd2f3b81740 1 -- 172.21.15.47:0/788520815 shutdown_connections 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.757+0000 7fd2f3b81740 1 -- 172.21.15.47:0/788520815 wait complete. 2024-03-21T11:30:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.757+0000 7fd2f3b81740 20 client.27841 trim_cache size 0 max 16384 2024-03-21T11:30:06.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.775+0000 7fd2f3b81740 1 Processor -- start 2024-03-21T11:30:06.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.775+0000 7fd2f3b81740 1 -- start start 2024-03-21T11:30:06.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.775+0000 7fd2f3b81740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.775+0000 7fd2f3b81740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2f0870 0x55c8eb301340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.775+0000 7fd2f3b81740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb2ef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.775+0000 7fd2f3b81740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec1b4ac0 con 0x55c8eb2efe80 2024-03-21T11:30:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.775+0000 7fd2f3b81740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec1b6440 con 0x55c8eb2f59e0 2024-03-21T11:30:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.775+0000 7fd2f3b81740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec1b5120 con 0x55c8eb2f0870 2024-03-21T11:30:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.776+0000 7fd2e9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2f0870 0x55c8eb301340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.776+0000 7fd2e9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2f0870 0x55c8eb301340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60122/0 (socket says 172.21.15.47:60122) 2024-03-21T11:30:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.776+0000 7fd2e9400640 1 -- 172.21.15.47:0/2595769195 learned_addr learned my addr 172.21.15.47:0/2595769195 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.776+0000 7fd2e9e00640 1 --2- 172.21.15.47:0/2595769195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.776+0000 7fd2ea800640 1 --2- 172.21.15.47:0/2595769195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb2ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.776+0000 7fd2e9400640 1 -- 172.21.15.47:0/2595769195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 msgr2=0x55c8eb2ef140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.776+0000 7fd2e9400640 1 --2- 172.21.15.47:0/2595769195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb2ef140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.776+0000 7fd2e9400640 1 -- 172.21.15.47:0/2595769195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2efe80 msgr2=0x55c8eb2f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.776+0000 7fd2e9400640 1 --2- 172.21.15.47:0/2595769195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.776+0000 7fd2e9400640 1 -- 172.21.15.47:0/2595769195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8eaf290f0 con 0x55c8eb2f0870 2024-03-21T11:30:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.776+0000 7fd2ea800640 1 --2- 172.21.15.47:0/2595769195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb2ef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.776+0000 7fd2e9e00640 1 --2- 172.21.15.47:0/2595769195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.776+0000 7fd2e9400640 1 --2- 172.21.15.47:0/2595769195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2f0870 0x55c8eb301340 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8005860 tx=0x7fd2d8014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78cc0ffd572d1bee server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.777+0000 7fd2d6a00640 1 -- 172.21.15.47:0/2595769195 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d80095a0 con 0x55c8eb2f0870 2024-03-21T11:30:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.777+0000 7fd2d6a00640 1 -- 172.21.15.47:0/2595769195 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2d802ad60 con 0x55c8eb2f0870 2024-03-21T11:30:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.777+0000 7fd2d6a00640 1 -- 172.21.15.47:0/2595769195 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d80053c0 con 0x55c8eb2f0870 2024-03-21T11:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.777+0000 7fd2f3b81740 1 -- 172.21.15.47:0/2595769195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2f0870 msgr2=0x55c8eb301340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.777+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/2595769195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2f0870 0x55c8eb301340 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8005860 tx=0x7fd2d8014040 comp rx=0 tx=0).stop 2024-03-21T11:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.777+0000 7fd2f3b81740 1 -- 172.21.15.47:0/2595769195 shutdown_connections 2024-03-21T11:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.777+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/2595769195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb2ef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.777+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/2595769195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2f0870 0x55c8eb301340 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.777+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/2595769195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2efe80 0x55c8eb2f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.777+0000 7fd2f3b81740 1 -- 172.21.15.47:0/2595769195 >> 172.21.15.47:0/2595769195 conn(0x55c8eb2b8200 msgr2=0x55c8eb2eea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.778+0000 7fd2f3b81740 1 -- 172.21.15.47:0/2595769195 shutdown_connections 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.778+0000 7fd2f3b81740 1 -- 172.21.15.47:0/2595769195 wait complete. 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.778+0000 7fd2f3b81740 1 -- start start 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.778+0000 7fd2f3b81740 1 Processor -- start 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2f3b81740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2efe80 0x55c8eb2fe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2f3b81740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2f0870 0x55c8eb2fe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2f3b81740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb35f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2f3b81740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec1b5de0 con 0x55c8eb2f0870 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2f3b81740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec1b5780 con 0x55c8eb2f59e0 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2f3b81740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec1b2c80 con 0x55c8eb2efe80 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2f3b81740 10 client.0 _ll_register_callbacks cb 0x7fd2dc0048f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2ea800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb35f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2ea800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb35f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52420/0 (socket says 172.21.15.47:52420) 2024-03-21T11:30:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2ea800640 1 -- 172.21.15.47:0/3829300407 learned_addr learned my addr 172.21.15.47:0/3829300407 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2e9e00640 1 --2- 172.21.15.47:0/3829300407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2efe80 0x55c8eb2fe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2e9400640 1 --2- 172.21.15.47:0/3829300407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2f0870 0x55c8eb2fe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2ea800640 1 -- 172.21.15.47:0/3829300407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2efe80 msgr2=0x55c8eb2fe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2ea800640 1 --2- 172.21.15.47:0/3829300407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2efe80 0x55c8eb2fe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2ea800640 1 -- 172.21.15.47:0/3829300407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2f0870 msgr2=0x55c8eb2fe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2ea800640 1 --2- 172.21.15.47:0/3829300407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2f0870 0x55c8eb2fe660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.779+0000 7fd2ea800640 1 -- 172.21.15.47:0/3829300407 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8eb35fae0 con 0x55c8eb2f59e0 2024-03-21T11:30:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.780+0000 7fd2e9e00640 1 --2- 172.21.15.47:0/3829300407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2efe80 0x55c8eb2fe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.780+0000 7fd2ea800640 1 --2- 172.21.15.47:0/3829300407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb35f5a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd2e402c180 tx=0x7fd2e4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcb5c1770e8ce1ce server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.780+0000 7fd2cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.780+0000 7fd2cbe00640 1 -- 172.21.15.47:0/3829300407 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4005650 con 0x55c8eb2f59e0 2024-03-21T11:30:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.780+0000 7fd2f3b81740 1 -- 172.21.15.47:0/3829300407 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2d802ad60 con 0x55c8eb2f59e0 2024-03-21T11:30:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.780+0000 7fd2cbe00640 1 -- 172.21.15.47:0/3829300407 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd2e4034440 con 0x55c8eb2f59e0 2024-03-21T11:30:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.780+0000 7fd2cbe00640 1 -- 172.21.15.47:0/3829300407 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2e4034640 con 0x55c8eb2f59e0 2024-03-21T11:30:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.780+0000 7fd2f3b81740 10 client.27830 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.780+0000 7fd2f3b81740 1 -- 172.21.15.47:0/3829300407 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c8eb360d00 con 0x55c8eb2f59e0 2024-03-21T11:30:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.781+0000 7fd2cbe00640 1 -- 172.21.15.47:0/3829300407 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2d802ad60 con 0x55c8eb2f59e0 2024-03-21T11:30:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.781+0000 7fd2cbe00640 1 -- 172.21.15.47:0/3829300407 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd2e4046040 con 0x55c8eb2f59e0 2024-03-21T11:30:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.781+0000 7fd2f3b81740 10 client.27830 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.781+0000 7fd2f3b81740 10 client.27830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.781+0000 7fd2f3b81740 20 client.27830 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.781+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/3829300407 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8eb361640 0x55c8eb363a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.781+0000 7fd2f3b81740 4 client.27830 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:30:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.781+0000 7fd2f3b81740 1 -- 172.21.15.47:0/3829300407 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c8eb25cae0 con 0x55c8eb361640 2024-03-21T11:30:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.781+0000 7fd2e9e00640 1 --2- 172.21.15.47:0/3829300407 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8eb361640 0x55c8eb363a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:06.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.782+0000 7fd2e9e00640 1 --2- 172.21.15.47:0/3829300407 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8eb361640 0x55c8eb363a60 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3faaa5ddba3550c7 server_cookie=9d4131735128d2bc in_seq=0 out_seq=0 2024-03-21T11:30:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.782+0000 7fd2cbe00640 10 client.27830 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.782+0000 7fd2cbe00640 1 -- 172.21.15.47:0/3829300407 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c8eb25cae0 con 0x55c8eb361640 2024-03-21T11:30:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.782+0000 7fd2cbe00640 10 client.27830 handle_command_reply: tid=0 2024-03-21T11:30:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.782+0000 7fd2f3b81740 1 client.27830 shutdown 2024-03-21T11:30:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.783+0000 7fd2f3b81740 1 -- 172.21.15.47:0/3829300407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 msgr2=0x55c8eb35f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.783+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/3829300407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb35f5a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd2e402c180 tx=0x7fd2e4009260 comp rx=0 tx=0).stop 2024-03-21T11:30:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.783+0000 7fd2f3b81740 1 -- 172.21.15.47:0/3829300407 shutdown_connections 2024-03-21T11:30:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.783+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/3829300407 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8eb361640 0x55c8eb363a60 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.783+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/3829300407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8eb2f59e0 0x55c8eb35f5a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.783+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/3829300407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8eb2f0870 0x55c8eb2fe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.783+0000 7fd2f3b81740 1 --2- 172.21.15.47:0/3829300407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8eb2efe80 0x55c8eb2fe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.783+0000 7fd2cbe00640 0 client.27830 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.783+0000 7fd2f3b81740 1 -- 172.21.15.47:0/3829300407 >> 172.21.15.47:0/3829300407 conn(0x55c8eb2b8200 msgr2=0x55c8eb2f9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.783+0000 7fd2f3b81740 1 -- 172.21.15.47:0/3829300407 shutdown_connections 2024-03-21T11:30:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.783+0000 7fd2f3b81740 1 -- 172.21.15.47:0/3829300407 wait complete. 2024-03-21T11:30:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.783+0000 7fd2f3b81740 20 client.27830 trim_cache size 0 max 16384 2024-03-21T11:30:06.788 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:30:06.788 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:30:06.788 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:30:06.788 INFO:teuthology.orchestra.run.smithi047.stdout: "4ad9ce15-8e03-4595-88c5-760c5e5b0a15": { 2024-03-21T11:30:06.788 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:30:06.788 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "4ad9ce15-8e03-4595-88c5-760c5e5b0a15", 2024-03-21T11:30:06.788 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:30:06.789 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:30:06.789 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:30:06.789 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:30:06.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.784+0000 7fd2f0e00640 1 -- 172.21.15.47:0/3197672638 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2d00a2fa0 msgr2=0x7fd2d00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.784+0000 7fd2f0e00640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2d00a2fa0 0x7fd2d00a5460 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55c8eb2f10d0 tx=0x7fd2e402cf90 comp rx=0 tx=0).stop 2024-03-21T11:30:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.784+0000 7fd2f0e00640 1 -- 172.21.15.47:0/3197672638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2ec1097b0 msgr2=0x7fd2ec10b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.784+0000 7fd2f0e00640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2ec1097b0 0x7fd2ec10b710 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009d50 tx=0x7fd2dc00c6a0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.784+0000 7fd2f0e00640 1 -- 172.21.15.47:0/3197672638 shutdown_connections 2024-03-21T11:30:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.784+0000 7fd2f0e00640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd2d00a2fa0 0x7fd2d00a5460 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.784+0000 7fd2f0e00640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec10ea10 0x7fd2ec10a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.784+0000 7fd2f0e00640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec10c0e0 0x7fd2ec109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.784+0000 7fd2f0e00640 1 --2- 172.21.15.47:0/3197672638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2ec1097b0 0x7fd2ec10b710 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.784+0000 7fd2f0e00640 1 -- 172.21.15.47:0/3197672638 >> 172.21.15.47:0/3197672638 conn(0x7fd2ec06e8a0 msgr2=0x7fd2ec108bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.784+0000 7fd2f0e00640 1 -- 172.21.15.47:0/3197672638 shutdown_connections 2024-03-21T11:30:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:06.784+0000 7fd2f0e00640 1 -- 172.21.15.47:0/3197672638 wait complete. 2024-03-21T11:30:06.803 INFO:tasks.cephfs.filesystem:scrub status for tag:4ad9ce15-8e03-4595-88c5-760c5e5b0a15 - {'path': '/', 'tag': '4ad9ce15-8e03-4595-88c5-760c5e5b0a15', 'options': 'recursive,force'} 2024-03-21T11:30:07.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:06 smithi047 ceph-mon[31351]: pgmap v603: 129 pgs: 129 active+clean; 40 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-03-21T11:30:07.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:06 smithi100 ceph-mon[36245]: pgmap v603: 129 pgs: 129 active+clean; 40 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-03-21T11:30:07.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:06 smithi102 ceph-mon[36748]: pgmap v603: 129 pgs: 129 active+clean; 40 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-03-21T11:30:07.525 INFO:tasks.workunit.client.0.smithi047.stdout: 1 455016 8.18 MB/sec execute 171 sec latency 15.655 ms 2024-03-21T11:30:08.141 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:07 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:30:08.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:07 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:30:08.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:07 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:30:08.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:07 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:30:08.142 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:07 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:30:08.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:07 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:30:08.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:07 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:30:08.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:07 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:30:08.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:07 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:30:08.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:07 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:30:08.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:07 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:30:08.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:07 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:30:08.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:07 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:30:08.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:07 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:30:08.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:07 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:30:08.525 INFO:tasks.workunit.client.0.smithi047.stdout: 1 457211 8.20 MB/sec execute 172 sec latency 12.092 ms 2024-03-21T11:30:09.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:08 smithi100 ceph-mon[36245]: pgmap v604: 129 pgs: 129 active+clean; 36 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 252 op/s 2024-03-21T11:30:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:08 smithi102 ceph-mon[36748]: pgmap v604: 129 pgs: 129 active+clean; 36 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 252 op/s 2024-03-21T11:30:09.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:08 smithi047 ceph-mon[31351]: pgmap v604: 129 pgs: 129 active+clean; 36 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 252 op/s 2024-03-21T11:30:09.525 INFO:tasks.workunit.client.0.smithi047.stdout: 1 458187 8.19 MB/sec execute 173 sec latency 45.643 ms 2024-03-21T11:30:10.526 INFO:tasks.workunit.client.0.smithi047.stdout: 1 459142 8.27 MB/sec execute 174 sec latency 55.790 ms 2024-03-21T11:30:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:10 smithi100 ceph-mon[36245]: pgmap v605: 129 pgs: 129 active+clean; 34 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 307 op/s 2024-03-21T11:30:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:10 smithi102 ceph-mon[36748]: pgmap v605: 129 pgs: 129 active+clean; 34 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 307 op/s 2024-03-21T11:30:11.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:10 smithi047 ceph-mon[31351]: pgmap v605: 129 pgs: 129 active+clean; 34 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 307 op/s 2024-03-21T11:30:11.526 INFO:tasks.workunit.client.0.smithi047.stdout: 1 461328 8.29 MB/sec execute 175 sec latency 9.844 ms 2024-03-21T11:30:12.526 INFO:tasks.workunit.client.0.smithi047.stdout: 1 462655 8.28 MB/sec execute 176 sec latency 10.054 ms 2024-03-21T11:30:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:12 smithi100 ceph-mon[36245]: pgmap v606: 129 pgs: 129 active+clean; 34 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 301 op/s 2024-03-21T11:30:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:12 smithi102 ceph-mon[36748]: pgmap v606: 129 pgs: 129 active+clean; 34 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 301 op/s 2024-03-21T11:30:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:12 smithi047 ceph-mon[31351]: pgmap v606: 129 pgs: 129 active+clean; 34 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 301 op/s 2024-03-21T11:30:13.526 INFO:tasks.workunit.client.0.smithi047.stdout: 1 464800 8.30 MB/sec execute 177 sec latency 12.322 ms 2024-03-21T11:30:14.526 INFO:tasks.workunit.client.0.smithi047.stdout: 1 466000 8.29 MB/sec execute 178 sec latency 64.939 ms 2024-03-21T11:30:15.119 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:14 smithi100 ceph-mon[36245]: pgmap v607: 129 pgs: 129 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 352 op/s 2024-03-21T11:30:15.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:14 smithi102 ceph-mon[36748]: pgmap v607: 129 pgs: 129 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 352 op/s 2024-03-21T11:30:15.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:14 smithi047 ceph-mon[31351]: pgmap v607: 129 pgs: 129 active+clean; 37 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 352 op/s 2024-03-21T11:30:15.526 INFO:tasks.workunit.client.0.smithi047.stdout: 1 467888 8.29 MB/sec execute 179 sec latency 24.851 ms 2024-03-21T11:30:16.526 INFO:tasks.workunit.client.0.smithi047.stdout: 1 469400 8.31 MB/sec execute 180 sec latency 11.102 ms 2024-03-21T11:30:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:16 smithi100 ceph-mon[36245]: pgmap v608: 129 pgs: 129 active+clean; 35 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 291 op/s 2024-03-21T11:30:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:16 smithi102 ceph-mon[36748]: pgmap v608: 129 pgs: 129 active+clean; 35 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 291 op/s 2024-03-21T11:30:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:16 smithi047 ceph-mon[31351]: pgmap v608: 129 pgs: 129 active+clean; 35 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 291 op/s 2024-03-21T11:30:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.415+0000 7fecd3200640 1 -- 172.21.15.47:0/801022496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc104af0 msgr2=0x7feccc104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.415+0000 7fecd3200640 1 --2- 172.21.15.47:0/801022496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc104af0 0x7feccc104f70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fecb4009900 tx=0x7fecb402bc40 comp rx=0 tx=0).stop 2024-03-21T11:30:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.415+0000 7fecd3200640 1 -- 172.21.15.47:0/801022496 shutdown_connections 2024-03-21T11:30:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.415+0000 7fecd3200640 1 --2- 172.21.15.47:0/801022496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feccc1054b0 0x7feccc10bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.415+0000 7fecd3200640 1 --2- 172.21.15.47:0/801022496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc104af0 0x7feccc104f70 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.415+0000 7fecd3200640 1 --2- 172.21.15.47:0/801022496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feccc103900 0x7feccc103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.415+0000 7fecd3200640 1 -- 172.21.15.47:0/801022496 >> 172.21.15.47:0/801022496 conn(0x7feccc0fee90 msgr2=0x7feccc1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.415+0000 7fecd3200640 1 -- 172.21.15.47:0/801022496 shutdown_connections 2024-03-21T11:30:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.415+0000 7fecd3200640 1 -- 172.21.15.47:0/801022496 wait complete. 2024-03-21T11:30:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.416+0000 7fecd3200640 1 Processor -- start 2024-03-21T11:30:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.417+0000 7fecd3200640 1 -- start start 2024-03-21T11:30:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.417+0000 7fecd3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc103900 0x7feccc199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.417+0000 7fecd3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feccc104af0 0x7feccc19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.417+0000 7fecd3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feccc1054b0 0x7feccc1a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.417+0000 7fecd3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc10f580 con 0x7feccc1054b0 2024-03-21T11:30:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.417+0000 7fecd3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc10f400 con 0x7feccc104af0 2024-03-21T11:30:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.417+0000 7fecd3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc10f700 con 0x7feccc103900 2024-03-21T11:30:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.417+0000 7feccbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc103900 0x7feccc199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.417+0000 7feccbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc103900 0x7feccc199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54516/0 (socket says 172.21.15.47:54516) 2024-03-21T11:30:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.417+0000 7feccbe00640 1 -- 172.21.15.47:0/757375677 learned_addr learned my addr 172.21.15.47:0/757375677 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.417+0000 7feccb400640 1 --2- 172.21.15.47:0/757375677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feccc104af0 0x7feccc19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.417+0000 7fecd1200640 1 --2- 172.21.15.47:0/757375677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feccc1054b0 0x7feccc1a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.418+0000 7feccbe00640 1 -- 172.21.15.47:0/757375677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feccc104af0 msgr2=0x7feccc19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.418+0000 7feccbe00640 1 --2- 172.21.15.47:0/757375677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feccc104af0 0x7feccc19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.418+0000 7feccbe00640 1 -- 172.21.15.47:0/757375677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feccc1054b0 msgr2=0x7feccc1a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.418+0000 7feccbe00640 1 --2- 172.21.15.47:0/757375677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feccc1054b0 0x7feccc1a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.418+0000 7feccbe00640 1 -- 172.21.15.47:0/757375677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc1a1760 con 0x7feccc103900 2024-03-21T11:30:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.418+0000 7feccbe00640 1 --2- 172.21.15.47:0/757375677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc103900 0x7feccc199b40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00e9c0 tx=0x7fecbc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.418+0000 7fecc8c00640 1 -- 172.21.15.47:0/757375677 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc00cd80 con 0x7feccc103900 2024-03-21T11:30:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.418+0000 7fecc8c00640 1 -- 172.21.15.47:0/757375677 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecbc0035f0 con 0x7feccc103900 2024-03-21T11:30:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.418+0000 7fecc8c00640 1 -- 172.21.15.47:0/757375677 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc010690 con 0x7feccc103900 2024-03-21T11:30:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.418+0000 7fecd3200640 1 -- 172.21.15.47:0/757375677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feccc1a1a50 con 0x7feccc103900 2024-03-21T11:30:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.419+0000 7fecd3200640 1 -- 172.21.15.47:0/757375677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feccc1a1f20 con 0x7feccc103900 2024-03-21T11:30:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.422+0000 7fecd5f77740 1 Processor -- start 2024-03-21T11:30:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.422+0000 7fecc8c00640 1 -- 172.21.15.47:0/757375677 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fecbc0037a0 con 0x7feccc103900 2024-03-21T11:30:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.422+0000 7fecd5f77740 1 -- start start 2024-03-21T11:30:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.422+0000 7fecd5f77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616546751e0 0x561654675600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.422+0000 7fecd5f77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 0x56165467a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.423+0000 7fecd5f77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56165467ad40 0x56165467d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.423+0000 7fecd5f77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616546383b0 con 0x561654675bd0 2024-03-21T11:30:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.423+0000 7fecd5f77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616545da830 con 0x56165467ad40 2024-03-21T11:30:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.423+0000 7fecd5f77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616545835d0 con 0x5616546751e0 2024-03-21T11:30:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.423+0000 7fecd1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616546751e0 0x561654675600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.423+0000 7feccbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 0x56165467a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.423+0000 7fecd1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616546751e0 0x561654675600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54518/0 (socket says 172.21.15.47:54518) 2024-03-21T11:30:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.423+0000 7fecd1200640 1 -- 172.21.15.47:0/1883434089 learned_addr learned my addr 172.21.15.47:0/1883434089 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.423+0000 7feccb400640 1 --2- 172.21.15.47:0/1883434089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56165467ad40 0x56165467d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.423+0000 7fecd1200640 1 -- 172.21.15.47:0/1883434089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56165467ad40 msgr2=0x56165467d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.423+0000 7fecd1200640 1 --2- 172.21.15.47:0/1883434089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56165467ad40 0x56165467d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.423+0000 7fecd1200640 1 -- 172.21.15.47:0/1883434089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 msgr2=0x56165467a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.423+0000 7fecd1200640 1 --2- 172.21.15.47:0/1883434089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 0x56165467a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.423+0000 7fecd1200640 1 -- 172.21.15.47:0/1883434089 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56165459b2c0 con 0x5616546751e0 2024-03-21T11:30:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.424+0000 7fecd1200640 1 --2- 172.21.15.47:0/1883434089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616546751e0 0x561654675600 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fecc4010370 tx=0x7fecc4010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e95297c6358df2b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.424+0000 7fecc8c00640 1 --2- 172.21.15.47:0/757375677 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fecac0a2fc0 0x7fecac0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.424+0000 7fecc1600640 1 -- 172.21.15.47:0/1883434089 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4016070 con 0x5616546751e0 2024-03-21T11:30:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.424+0000 7fecc1600640 1 -- 172.21.15.47:0/1883434089 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecc4004d20 con 0x5616546751e0 2024-03-21T11:30:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.424+0000 7fecc1600640 1 -- 172.21.15.47:0/1883434089 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4005480 con 0x5616546751e0 2024-03-21T11:30:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.424+0000 7fecc8c00640 1 -- 172.21.15.47:0/757375677 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fecbc0cfb40 con 0x7feccc103900 2024-03-21T11:30:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.424+0000 7feccbe00640 1 --2- 172.21.15.47:0/757375677 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fecac0a2fc0 0x7fecac0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.424+0000 7fecd5f77740 1 -- 172.21.15.47:0/1883434089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616546751e0 msgr2=0x561654675600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.424+0000 7fecd5f77740 1 --2- 172.21.15.47:0/1883434089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616546751e0 0x561654675600 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fecc4010370 tx=0x7fecc4010830 comp rx=0 tx=0).stop 2024-03-21T11:30:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.425+0000 7fecd5f77740 1 -- 172.21.15.47:0/1883434089 shutdown_connections 2024-03-21T11:30:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.425+0000 7fecd5f77740 1 --2- 172.21.15.47:0/1883434089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56165467ad40 0x56165467d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.425+0000 7fecd5f77740 1 --2- 172.21.15.47:0/1883434089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 0x56165467a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.425+0000 7fecd5f77740 1 --2- 172.21.15.47:0/1883434089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616546751e0 0x561654675600 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.425+0000 7fecd5f77740 1 -- 172.21.15.47:0/1883434089 >> 172.21.15.47:0/1883434089 conn(0x561654670750 msgr2=0x561654670b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.425+0000 7fecd5f77740 1 -- 172.21.15.47:0/1883434089 shutdown_connections 2024-03-21T11:30:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.425+0000 7fecd5f77740 1 -- 172.21.15.47:0/1883434089 wait complete. 2024-03-21T11:30:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.425+0000 7feccbe00640 1 --2- 172.21.15.47:0/757375677 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fecac0a2fc0 0x7fecac0a5480 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0a4360 tx=0x7fecbc0d3580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.425+0000 7fecd5f77740 1 -- start start 2024-03-21T11:30:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.425+0000 7fecd5f77740 1 Processor -- start 2024-03-21T11:30:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.426+0000 7fecd5f77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616546751e0 0x56165467f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.426+0000 7fecd5f77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561654675bd0 0x56165467f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.426+0000 7fecd5f77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56165467ad40 0x5616546868d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.426+0000 7fecd5f77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561654596a20 con 0x5616546751e0 2024-03-21T11:30:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.426+0000 7fecd5f77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616545d68d0 con 0x56165467ad40 2024-03-21T11:30:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.426+0000 7fecd5f77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56165467dcf0 con 0x561654675bd0 2024-03-21T11:30:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.426+0000 7fecd5f77740 10 client.0 _ll_register_callbacks cb 0x5616545d31c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.426+0000 7feccb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561654675bd0 0x56165467f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.426+0000 7fecd1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616546751e0 0x56165467f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.426+0000 7feccb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561654675bd0 0x56165467f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54522/0 (socket says 172.21.15.47:54522) 2024-03-21T11:30:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.426+0000 7fecd1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616546751e0 0x56165467f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42774/0 (socket says 172.21.15.47:42774) 2024-03-21T11:30:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.426+0000 7feccb400640 1 -- 172.21.15.47:0/2878262344 learned_addr learned my addr 172.21.15.47:0/2878262344 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.426+0000 7fecd1200640 1 --2- 172.21.15.47:0/2878262344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56165467ad40 0x5616546868d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.427+0000 7feccb400640 1 -- 172.21.15.47:0/2878262344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56165467ad40 msgr2=0x5616546868d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.427+0000 7feccb400640 1 --2- 172.21.15.47:0/2878262344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56165467ad40 0x5616546868d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.427+0000 7feccb400640 1 -- 172.21.15.47:0/2878262344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616546751e0 msgr2=0x56165467f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.427+0000 7feccb400640 1 --2- 172.21.15.47:0/2878262344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616546751e0 0x56165467f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.427+0000 7feccb400640 1 -- 172.21.15.47:0/2878262344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616542ef210 con 0x561654675bd0 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.427+0000 7feccb400640 1 --2- 172.21.15.47:0/2878262344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561654675bd0 0x56165467f900 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fecb4036040 tx=0x7fecb4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd233e8bf5352bca server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.427+0000 7fecba000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.427+0000 7fecba000640 1 -- 172.21.15.47:0/2878262344 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb4005d50 con 0x561654675bd0 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.427+0000 7fecba000640 1 -- 172.21.15.47:0/2878262344 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fecb4005230 con 0x561654675bd0 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.427+0000 7fecd5f77740 1 -- 172.21.15.47:0/2878262344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecc4004d20 con 0x561654675bd0 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.428+0000 7fecba000640 1 -- 172.21.15.47:0/2878262344 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecb4005430 con 0x561654675bd0 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.428+0000 7fecd5f77740 10 client.27859 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.428+0000 7fecd5f77740 1 -- 172.21.15.47:0/2878262344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5616543bb620 con 0x561654675bd0 2024-03-21T11:30:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.428+0000 7fecba000640 1 -- 172.21.15.47:0/2878262344 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecc4004d20 con 0x561654675bd0 2024-03-21T11:30:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.428+0000 7fecba000640 1 -- 172.21.15.47:0/2878262344 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fecb4003850 con 0x561654675bd0 2024-03-21T11:30:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.428+0000 7fecd5f77740 10 client.27859 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.428+0000 7fecd5f77740 10 client.27859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.428+0000 7fecd5f77740 20 client.27859 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.428+0000 7fecd5f77740 1 --2- 172.21.15.47:0/2878262344 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56165468b1b0 0x56165468d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.428+0000 7fecd5f77740 4 client.27859 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.428+0000 7fecd5f77740 1 -- 172.21.15.47:0/2878262344 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616545e7d70 con 0x56165468b1b0 2024-03-21T11:30:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.428+0000 7fecd1200640 1 --2- 172.21.15.47:0/2878262344 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56165468b1b0 0x56165468d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.429+0000 7fecd1200640 1 --2- 172.21.15.47:0/2878262344 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56165468b1b0 0x56165468d5f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc255daca1bfbb41 server_cookie=cc626e8173039812 in_seq=0 out_seq=0 2024-03-21T11:30:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.429+0000 7fecba000640 10 client.27859 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.430+0000 7fecba000640 1 -- 172.21.15.47:0/2878262344 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5616545e7d70 con 0x56165468b1b0 2024-03-21T11:30:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.430+0000 7fecba000640 10 client.27859 handle_command_reply: tid=0 2024-03-21T11:30:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.430+0000 7fecd5f77740 1 client.27859 shutdown 2024-03-21T11:30:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.430+0000 7fecd5f77740 1 -- 172.21.15.47:0/2878262344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561654675bd0 msgr2=0x56165467f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.430+0000 7fecd5f77740 1 --2- 172.21.15.47:0/2878262344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561654675bd0 0x56165467f900 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fecb4036040 tx=0x7fecb4004270 comp rx=0 tx=0).stop 2024-03-21T11:30:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.430+0000 7fecd5f77740 1 -- 172.21.15.47:0/2878262344 shutdown_connections 2024-03-21T11:30:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.430+0000 7fecd5f77740 1 --2- 172.21.15.47:0/2878262344 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56165468b1b0 0x56165468d5f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.430+0000 7fecd5f77740 1 --2- 172.21.15.47:0/2878262344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56165467ad40 0x5616546868d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.430+0000 7fecd5f77740 1 --2- 172.21.15.47:0/2878262344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561654675bd0 0x56165467f900 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.430+0000 7fecd5f77740 1 --2- 172.21.15.47:0/2878262344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616546751e0 0x56165467f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.430+0000 7fecd5f77740 1 -- 172.21.15.47:0/2878262344 >> 172.21.15.47:0/2878262344 conn(0x561654670750 msgr2=0x56165466f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.430+0000 7fecba000640 0 client.27859 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.431+0000 7fecd5f77740 1 -- 172.21.15.47:0/2878262344 shutdown_connections 2024-03-21T11:30:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.431+0000 7fecd5f77740 1 -- 172.21.15.47:0/2878262344 wait complete. 2024-03-21T11:30:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.431+0000 7fecd5f77740 20 client.27859 trim_cache size 0 max 16384 2024-03-21T11:30:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.447+0000 7fecd5f77740 1 Processor -- start 2024-03-21T11:30:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.447+0000 7fecd5f77740 1 -- start start 2024-03-21T11:30:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.447+0000 7fecd5f77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616546751e0 0x56165467f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.447+0000 7fecd5f77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 0x56165467f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.447+0000 7fecd5f77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56165467ad40 0x561654686890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.447+0000 7fecd5f77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc1ac200 con 0x561654675bd0 2024-03-21T11:30:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.447+0000 7fecd5f77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc1adb80 con 0x5616546751e0 2024-03-21T11:30:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.447+0000 7fecd5f77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc1ac860 con 0x56165467ad40 2024-03-21T11:30:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.448+0000 7fecd1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616546751e0 0x56165467f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.448+0000 7fecd1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616546751e0 0x56165467f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38808/0 (socket says 172.21.15.47:38808) 2024-03-21T11:30:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.448+0000 7fecd1200640 1 -- 172.21.15.47:0/475618347 learned_addr learned my addr 172.21.15.47:0/475618347 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.448+0000 7fecd1200640 1 --2- 172.21.15.47:0/475618347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56165467ad40 0x561654686890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.448+0000 7feccb400640 1 --2- 172.21.15.47:0/475618347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 0x56165467f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.448+0000 7fecd1200640 1 -- 172.21.15.47:0/475618347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56165467ad40 msgr2=0x561654686890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.448+0000 7fecd1200640 1 --2- 172.21.15.47:0/475618347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56165467ad40 0x561654686890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.448+0000 7fecd1200640 1 -- 172.21.15.47:0/475618347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 msgr2=0x56165467f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.448+0000 7fecd1200640 1 --2- 172.21.15.47:0/475618347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 0x56165467f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.449+0000 7fecd1200640 1 -- 172.21.15.47:0/475618347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616546769e0 con 0x5616546751e0 2024-03-21T11:30:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.449+0000 7fecd1200640 1 --2- 172.21.15.47:0/475618347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616546751e0 0x56165467f3c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fecc40098b0 tx=0x7fecc4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65249135f2e24b9a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.449+0000 7fecc0c00640 1 -- 172.21.15.47:0/475618347 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4016050 con 0x5616546751e0 2024-03-21T11:30:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.449+0000 7fecc0c00640 1 -- 172.21.15.47:0/475618347 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecc402ed20 con 0x5616546751e0 2024-03-21T11:30:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.449+0000 7fecc0c00640 1 -- 172.21.15.47:0/475618347 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc40129e0 con 0x5616546751e0 2024-03-21T11:30:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.450+0000 7fecd5f77740 1 -- 172.21.15.47:0/475618347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616546751e0 msgr2=0x56165467f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.450+0000 7fecd5f77740 1 --2- 172.21.15.47:0/475618347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616546751e0 0x56165467f3c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fecc40098b0 tx=0x7fecc4009840 comp rx=0 tx=0).stop 2024-03-21T11:30:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.450+0000 7fecd5f77740 1 -- 172.21.15.47:0/475618347 shutdown_connections 2024-03-21T11:30:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.450+0000 7fecd5f77740 1 --2- 172.21.15.47:0/475618347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56165467ad40 0x561654686890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.450+0000 7fecd5f77740 1 --2- 172.21.15.47:0/475618347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 0x56165467f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.450+0000 7fecd5f77740 1 --2- 172.21.15.47:0/475618347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616546751e0 0x56165467f3c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.450+0000 7fecd5f77740 1 -- 172.21.15.47:0/475618347 >> 172.21.15.47:0/475618347 conn(0x561654670750 msgr2=0x56165468d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.450+0000 7fecd5f77740 1 -- 172.21.15.47:0/475618347 shutdown_connections 2024-03-21T11:30:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.450+0000 7fecd5f77740 1 -- 172.21.15.47:0/475618347 wait complete. 2024-03-21T11:30:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.451+0000 7fecd5f77740 1 -- start start 2024-03-21T11:30:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.451+0000 7fecd5f77740 1 Processor -- start 2024-03-21T11:30:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.451+0000 7fecd5f77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616546751e0 0x561654683ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.451+0000 7fecd5f77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 0x561654684400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.451+0000 7fecd5f77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56165467ad40 0x5616546853c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.451+0000 7fecd5f77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc1ad520 con 0x561654675bd0 2024-03-21T11:30:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.451+0000 7fecd5f77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc1acec0 con 0x5616546751e0 2024-03-21T11:30:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.451+0000 7fecd5f77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc1aa3c0 con 0x56165467ad40 2024-03-21T11:30:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.452+0000 7fecd5f77740 10 client.0 _ll_register_callbacks cb 0x7fecb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.452+0000 7fecd1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616546751e0 0x561654683ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.452+0000 7fecd1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616546751e0 0x561654683ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38818/0 (socket says 172.21.15.47:38818) 2024-03-21T11:30:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.452+0000 7fecd1200640 1 -- 172.21.15.47:0/3898957303 learned_addr learned my addr 172.21.15.47:0/3898957303 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.452+0000 7feccb400640 1 --2- 172.21.15.47:0/3898957303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 0x561654684400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.452+0000 7fecd1200640 1 --2- 172.21.15.47:0/3898957303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56165467ad40 0x5616546853c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.452+0000 7fecd1200640 1 -- 172.21.15.47:0/3898957303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56165467ad40 msgr2=0x5616546853c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.452+0000 7fecd1200640 1 --2- 172.21.15.47:0/3898957303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56165467ad40 0x5616546853c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.452+0000 7fecd1200640 1 -- 172.21.15.47:0/3898957303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 msgr2=0x561654684400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.452+0000 7fecd1200640 1 --2- 172.21.15.47:0/3898957303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 0x561654684400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.452+0000 7fecd1200640 1 -- 172.21.15.47:0/3898957303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616546859b0 con 0x5616546751e0 2024-03-21T11:30:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.453+0000 7fecd1200640 1 --2- 172.21.15.47:0/3898957303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616546751e0 0x561654683ec0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fecc4006010 tx=0x7fecc4032040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cc7b76204928780 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.453+0000 7fecba000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.453+0000 7fecba000640 1 -- 172.21.15.47:0/3898957303 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4016040 con 0x5616546751e0 2024-03-21T11:30:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.453+0000 7fecd5f77740 1 -- 172.21.15.47:0/3898957303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecc402ed20 con 0x5616546751e0 2024-03-21T11:30:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.453+0000 7fecba000640 1 -- 172.21.15.47:0/3898957303 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fecc4012760 con 0x5616546751e0 2024-03-21T11:30:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.453+0000 7fecba000640 1 -- 172.21.15.47:0/3898957303 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecc4012960 con 0x5616546751e0 2024-03-21T11:30:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.453+0000 7fecd5f77740 10 client.27848 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.453+0000 7fecd5f77740 1 -- 172.21.15.47:0/3898957303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561654686cc0 con 0x5616546751e0 2024-03-21T11:30:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.453+0000 7fecba000640 1 -- 172.21.15.47:0/3898957303 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecc402ed20 con 0x5616546751e0 2024-03-21T11:30:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.454+0000 7fecba000640 1 -- 172.21.15.47:0/3898957303 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fecc4028040 con 0x5616546751e0 2024-03-21T11:30:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.454+0000 7fecd5f77740 10 client.27848 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.454+0000 7fecd5f77740 10 client.27848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.454+0000 7fecd5f77740 20 client.27848 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.454+0000 7fecd5f77740 1 --2- 172.21.15.47:0/3898957303 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5616546e6f10 0x5616546e9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.454+0000 7fecd5f77740 4 client.27848 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:30:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.454+0000 7fecd5f77740 1 -- 172.21.15.47:0/3898957303 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5616545e20c0 con 0x5616546e6f10 2024-03-21T11:30:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.454+0000 7feccb400640 1 --2- 172.21.15.47:0/3898957303 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5616546e6f10 0x5616546e9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.455+0000 7feccb400640 1 --2- 172.21.15.47:0/3898957303 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5616546e6f10 0x5616546e9300 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caa1e891ecd3508b server_cookie=ae46a99c093a9b0c in_seq=0 out_seq=0 2024-03-21T11:30:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.455+0000 7fecba000640 10 client.27848 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:17.527 INFO:tasks.workunit.client.0.smithi047.stdout: 1 470489 8.29 MB/sec execute 181 sec latency 72.682 ms 2024-03-21T11:30:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.722+0000 7fecba000640 1 -- 172.21.15.47:0/3898957303 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5616545e20c0 con 0x5616546e6f10 2024-03-21T11:30:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.722+0000 7fecba000640 10 client.27848 handle_command_reply: tid=0 2024-03-21T11:30:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.722+0000 7fecd5f77740 1 client.27848 shutdown 2024-03-21T11:30:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.723+0000 7fecd5f77740 1 -- 172.21.15.47:0/3898957303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616546751e0 msgr2=0x561654683ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.723+0000 7fecd5f77740 1 --2- 172.21.15.47:0/3898957303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616546751e0 0x561654683ec0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fecc4006010 tx=0x7fecc4032040 comp rx=0 tx=0).stop 2024-03-21T11:30:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.723+0000 7fecd5f77740 1 -- 172.21.15.47:0/3898957303 shutdown_connections 2024-03-21T11:30:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.723+0000 7fecd5f77740 1 --2- 172.21.15.47:0/3898957303 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5616546e6f10 0x5616546e9300 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.723+0000 7fecd5f77740 1 --2- 172.21.15.47:0/3898957303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56165467ad40 0x5616546853c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.723+0000 7fecd5f77740 1 --2- 172.21.15.47:0/3898957303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561654675bd0 0x561654684400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.723+0000 7fecd5f77740 1 --2- 172.21.15.47:0/3898957303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616546751e0 0x561654683ec0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.723+0000 7fecba000640 0 client.27848 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.723+0000 7fecd5f77740 1 -- 172.21.15.47:0/3898957303 >> 172.21.15.47:0/3898957303 conn(0x561654670750 msgr2=0x561654676ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.723+0000 7fecd5f77740 1 -- 172.21.15.47:0/3898957303 shutdown_connections 2024-03-21T11:30:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.723+0000 7fecd5f77740 1 -- 172.21.15.47:0/3898957303 wait complete. 2024-03-21T11:30:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.723+0000 7fecd5f77740 20 client.27848 trim_cache size 0 max 16384 2024-03-21T11:30:17.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:30:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:30:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:30:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:30:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.724+0000 7fecd3200640 1 -- 172.21.15.47:0/757375677 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fecac0a2fc0 msgr2=0x7fecac0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.724+0000 7fecd3200640 1 --2- 172.21.15.47:0/757375677 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fecac0a2fc0 0x7fecac0a5480 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0a4360 tx=0x7fecbc0d3580 comp rx=0 tx=0).stop 2024-03-21T11:30:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.724+0000 7fecd3200640 1 -- 172.21.15.47:0/757375677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc103900 msgr2=0x7feccc199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.724+0000 7fecd3200640 1 --2- 172.21.15.47:0/757375677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc103900 0x7feccc199b40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00e9c0 tx=0x7fecbc00ee80 comp rx=0 tx=0).stop 2024-03-21T11:30:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.724+0000 7fecd3200640 1 -- 172.21.15.47:0/757375677 shutdown_connections 2024-03-21T11:30:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.724+0000 7fecd3200640 1 --2- 172.21.15.47:0/757375677 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fecac0a2fc0 0x7fecac0a5480 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.724+0000 7fecd3200640 1 --2- 172.21.15.47:0/757375677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feccc1054b0 0x7feccc1a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.724+0000 7fecd3200640 1 --2- 172.21.15.47:0/757375677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feccc104af0 0x7feccc19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.724+0000 7fecd3200640 1 --2- 172.21.15.47:0/757375677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feccc103900 0x7feccc199b40 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.724+0000 7fecd3200640 1 -- 172.21.15.47:0/757375677 >> 172.21.15.47:0/757375677 conn(0x7feccc0fee90 msgr2=0x7feccc107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.725+0000 7fecd3200640 1 -- 172.21.15.47:0/757375677 shutdown_connections 2024-03-21T11:30:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:17.725+0000 7fecd3200640 1 -- 172.21.15.47:0/757375677 wait complete. 2024-03-21T11:30:18.529 INFO:tasks.workunit.client.0.smithi047.stdout: 1 472345 8.29 MB/sec execute 182 sec latency 187.286 ms 2024-03-21T11:30:19.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:18 smithi100 ceph-mon[36245]: pgmap v609: 129 pgs: 129 active+clean; 29 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 298 op/s 2024-03-21T11:30:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:18 smithi102 ceph-mon[36748]: pgmap v609: 129 pgs: 129 active+clean; 29 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 298 op/s 2024-03-21T11:30:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:18 smithi047 ceph-mon[31351]: pgmap v609: 129 pgs: 129 active+clean; 29 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 298 op/s 2024-03-21T11:30:19.527 INFO:tasks.workunit.client.0.smithi047.stdout: 1 473281 8.28 MB/sec execute 183 sec latency 51.998 ms 2024-03-21T11:30:20.527 INFO:tasks.workunit.client.0.smithi047.stdout: 1 475017 8.27 MB/sec execute 184 sec latency 24.197 ms 2024-03-21T11:30:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:20 smithi100 ceph-mon[36245]: pgmap v610: 129 pgs: 129 active+clean; 19 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 331 op/s 2024-03-21T11:30:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:20 smithi102 ceph-mon[36748]: pgmap v610: 129 pgs: 129 active+clean; 19 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 331 op/s 2024-03-21T11:30:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:20 smithi047 ceph-mon[31351]: pgmap v610: 129 pgs: 129 active+clean; 19 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 331 op/s 2024-03-21T11:30:21.527 INFO:tasks.workunit.client.0.smithi047.stdout: 1 476502 8.29 MB/sec execute 185 sec latency 13.913 ms 2024-03-21T11:30:22.528 INFO:tasks.workunit.client.0.smithi047.stdout: 1 477794 8.27 MB/sec execute 186 sec latency 35.792 ms 2024-03-21T11:30:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:22 smithi100 ceph-mon[36245]: pgmap v611: 129 pgs: 129 active+clean; 19 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 2.1 MiB/s wr, 266 op/s 2024-03-21T11:30:23.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:22 smithi102 ceph-mon[36748]: pgmap v611: 129 pgs: 129 active+clean; 19 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 2.1 MiB/s wr, 266 op/s 2024-03-21T11:30:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:22 smithi047 ceph-mon[31351]: pgmap v611: 129 pgs: 129 active+clean; 19 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 2.1 MiB/s wr, 266 op/s 2024-03-21T11:30:23.527 INFO:tasks.workunit.client.0.smithi047.stdout: 1 479872 8.30 MB/sec execute 187 sec latency 12.870 ms 2024-03-21T11:30:24.528 INFO:tasks.workunit.client.0.smithi047.stdout: 1 480727 8.27 MB/sec execute 188 sec latency 21.331 ms 2024-03-21T11:30:25.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:24 smithi047 ceph-mon[31351]: pgmap v612: 129 pgs: 129 active+clean; 17 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.7 MiB/s wr, 363 op/s 2024-03-21T11:30:25.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:24 smithi100 ceph-mon[36245]: pgmap v612: 129 pgs: 129 active+clean; 17 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.7 MiB/s wr, 363 op/s 2024-03-21T11:30:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:24 smithi102 ceph-mon[36748]: pgmap v612: 129 pgs: 129 active+clean; 17 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.7 MiB/s wr, 363 op/s 2024-03-21T11:30:25.528 INFO:tasks.workunit.client.0.smithi047.stdout: 1 482826 8.28 MB/sec execute 189 sec latency 71.635 ms 2024-03-21T11:30:26.528 INFO:tasks.workunit.client.0.smithi047.stdout: 1 483960 8.27 MB/sec execute 190 sec latency 22.975 ms 2024-03-21T11:30:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:26 smithi100 ceph-mon[36245]: pgmap v613: 129 pgs: 129 active+clean; 17 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 2.0 MiB/s wr, 312 op/s 2024-03-21T11:30:27.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:26 smithi102 ceph-mon[36748]: pgmap v613: 129 pgs: 129 active+clean; 17 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 2.0 MiB/s wr, 312 op/s 2024-03-21T11:30:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:26 smithi047 ceph-mon[31351]: pgmap v613: 129 pgs: 129 active+clean; 17 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 2.0 MiB/s wr, 312 op/s 2024-03-21T11:30:27.528 INFO:tasks.workunit.client.0.smithi047.stdout: 1 485910 8.28 MB/sec execute 191 sec latency 23.174 ms 2024-03-21T11:30:28.528 INFO:tasks.workunit.client.0.smithi047.stdout: 1 487328 8.29 MB/sec execute 192 sec latency 10.666 ms 2024-03-21T11:30:29.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:28 smithi100 ceph-mon[36245]: pgmap v614: 129 pgs: 129 active+clean; 17 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-03-21T11:30:29.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:28 smithi102 ceph-mon[36748]: pgmap v614: 129 pgs: 129 active+clean; 17 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-03-21T11:30:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:28 smithi047 ceph-mon[31351]: pgmap v614: 129 pgs: 129 active+clean; 17 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-03-21T11:30:29.528 INFO:tasks.workunit.client.0.smithi047.stdout: 1 489169 8.28 MB/sec execute 193 sec latency 13.116 ms 2024-03-21T11:30:30.528 INFO:tasks.workunit.client.0.smithi047.stdout: 1 490628 8.30 MB/sec execute 194 sec latency 90.594 ms 2024-03-21T11:30:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.901+0000 7f2883000640 1 -- 172.21.15.47:0/3844447629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f287c104ac0 msgr2=0x7f287c104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.901+0000 7f2883000640 1 --2- 172.21.15.47:0/3844447629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f287c104ac0 0x7f287c104f40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2868009900 tx=0x7f286802bc30 comp rx=0 tx=0).stop 2024-03-21T11:30:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.901+0000 7f2883000640 1 -- 172.21.15.47:0/3844447629 shutdown_connections 2024-03-21T11:30:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.901+0000 7f2883000640 1 --2- 172.21.15.47:0/3844447629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f287c105480 0x7f287c10bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.901+0000 7f2883000640 1 --2- 172.21.15.47:0/3844447629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f287c104ac0 0x7f287c104f40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.901+0000 7f2883000640 1 --2- 172.21.15.47:0/3844447629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f287c1038d0 0x7f287c103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.901+0000 7f2883000640 1 -- 172.21.15.47:0/3844447629 >> 172.21.15.47:0/3844447629 conn(0x7f287c0fee60 msgr2=0x7f287c101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.901+0000 7f2883000640 1 -- 172.21.15.47:0/3844447629 shutdown_connections 2024-03-21T11:30:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.901+0000 7f2883000640 1 -- 172.21.15.47:0/3844447629 wait complete. 2024-03-21T11:30:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.902+0000 7f2883000640 1 Processor -- start 2024-03-21T11:30:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.902+0000 7f2883000640 1 -- start start 2024-03-21T11:30:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f2883000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f287c1038d0 0x7f287c199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f2883000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f287c104ac0 0x7f287c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f2883000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f287c105480 0x7f287c1a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f2883000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c10f5a0 con 0x7f287c1038d0 2024-03-21T11:30:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f2883000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c10f420 con 0x7f287c105480 2024-03-21T11:30:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f2883000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c10f720 con 0x7f287c104ac0 2024-03-21T11:30:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f287b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f287c104ac0 0x7f287c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f287be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f287c1038d0 0x7f287c199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f2881000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f287c105480 0x7f287c1a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f287b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f287c104ac0 0x7f287c19a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34378/0 (socket says 172.21.15.47:34378) 2024-03-21T11:30:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f287b400640 1 -- 172.21.15.47:0/751737988 learned_addr learned my addr 172.21.15.47:0/751737988 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f287b400640 1 -- 172.21.15.47:0/751737988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f287c105480 msgr2=0x7f287c1a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f287b400640 1 --2- 172.21.15.47:0/751737988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f287c105480 0x7f287c1a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f287b400640 1 -- 172.21.15.47:0/751737988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f287c1038d0 msgr2=0x7f287c199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f287b400640 1 --2- 172.21.15.47:0/751737988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f287c1038d0 0x7f287c199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f287b400640 1 -- 172.21.15.47:0/751737988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c1a1740 con 0x7f287c104ac0 2024-03-21T11:30:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.903+0000 7f2881000640 1 --2- 172.21.15.47:0/751737988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f287c105480 0x7f287c1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.904+0000 7f287b400640 1 --2- 172.21.15.47:0/751737988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f287c104ac0 0x7f287c19a060 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2868005ab0 tx=0x7f286802c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.904+0000 7f2878c00640 1 -- 172.21.15.47:0/751737988 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868043020 con 0x7f287c104ac0 2024-03-21T11:30:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.904+0000 7f2878c00640 1 -- 172.21.15.47:0/751737988 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28680054e0 con 0x7f287c104ac0 2024-03-21T11:30:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.904+0000 7f2883000640 1 -- 172.21.15.47:0/751737988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c1a19d0 con 0x7f287c104ac0 2024-03-21T11:30:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.904+0000 7f2878c00640 1 -- 172.21.15.47:0/751737988 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868003ce0 con 0x7f287c104ac0 2024-03-21T11:30:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.904+0000 7f2883000640 1 -- 172.21.15.47:0/751737988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287c1a1f20 con 0x7f287c104ac0 2024-03-21T11:30:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.908+0000 7f2885d94740 1 Processor -- start 2024-03-21T11:30:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.908+0000 7f2885d94740 1 -- start start 2024-03-21T11:30:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.908+0000 7f2885d94740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf034600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.908+0000 7f2885d94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf034bd0 0x562daf0396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.908+0000 7f2878c00640 1 -- 172.21.15.47:0/751737988 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2868046020 con 0x7f287c104ac0 2024-03-21T11:30:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.908+0000 7f2885d94740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf039d40 0x562daf03c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.908+0000 7f2885d94740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562daeff73b0 con 0x562daf0341e0 2024-03-21T11:30:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.908+0000 7f2885d94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562daef99830 con 0x562daf034bd0 2024-03-21T11:30:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.908+0000 7f2885d94740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562daef425d0 con 0x562daf039d40 2024-03-21T11:30:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.908+0000 7f2881000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf034600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.908+0000 7f287b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf039d40 0x562daf03c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.908+0000 7f287be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf034bd0 0x562daf0396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.908+0000 7f2881000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf034600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50858/0 (socket says 172.21.15.47:50858) 2024-03-21T11:30:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.908+0000 7f2881000640 1 -- 172.21.15.47:0/4221497487 learned_addr learned my addr 172.21.15.47:0/4221497487 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.909+0000 7f287be00640 1 -- 172.21.15.47:0/4221497487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf039d40 msgr2=0x562daf03c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.909+0000 7f287be00640 1 --2- 172.21.15.47:0/4221497487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf039d40 0x562daf03c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.909+0000 7f287be00640 1 -- 172.21.15.47:0/4221497487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 msgr2=0x562daf034600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.909+0000 7f287be00640 1 --2- 172.21.15.47:0/4221497487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf034600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.909+0000 7f287be00640 1 -- 172.21.15.47:0/4221497487 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562daef5a2c0 con 0x562daf034bd0 2024-03-21T11:30:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.909+0000 7f287be00640 1 --2- 172.21.15.47:0/4221497487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf034bd0 0x562daf0396d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f287c104920 tx=0x7f286c00bbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e97813cf6edcaa19 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.909+0000 7f2871600640 1 -- 172.21.15.47:0/4221497487 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c003dd0 con 0x562daf034bd0 2024-03-21T11:30:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.910+0000 7f2871600640 1 -- 172.21.15.47:0/4221497487 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f286c015070 con 0x562daf034bd0 2024-03-21T11:30:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.910+0000 7f2871600640 1 -- 172.21.15.47:0/4221497487 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c011680 con 0x562daf034bd0 2024-03-21T11:30:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.910+0000 7f2878c00640 1 --2- 172.21.15.47:0/751737988 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28600a2fc0 0x7f28600a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.910+0000 7f2878c00640 1 -- 172.21.15.47:0/751737988 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f28680f18e0 con 0x7f287c104ac0 2024-03-21T11:30:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.910+0000 7f2881000640 1 --2- 172.21.15.47:0/751737988 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28600a2fc0 0x7f28600a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.910+0000 7f2885d94740 1 -- 172.21.15.47:0/4221497487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf034bd0 msgr2=0x562daf0396d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.910+0000 7f2885d94740 1 --2- 172.21.15.47:0/4221497487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf034bd0 0x562daf0396d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f287c104920 tx=0x7f286c00bbe0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.910+0000 7f2885d94740 1 -- 172.21.15.47:0/4221497487 shutdown_connections 2024-03-21T11:30:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.910+0000 7f2885d94740 1 --2- 172.21.15.47:0/4221497487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf039d40 0x562daf03c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.910+0000 7f2885d94740 1 --2- 172.21.15.47:0/4221497487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf034bd0 0x562daf0396d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.910+0000 7f2885d94740 1 --2- 172.21.15.47:0/4221497487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf034600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.910+0000 7f2885d94740 1 -- 172.21.15.47:0/4221497487 >> 172.21.15.47:0/4221497487 conn(0x562daf02f750 msgr2=0x562daf02fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.911+0000 7f2885d94740 1 -- 172.21.15.47:0/4221497487 shutdown_connections 2024-03-21T11:30:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.911+0000 7f2885d94740 1 -- 172.21.15.47:0/4221497487 wait complete. 2024-03-21T11:30:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.911+0000 7f2881000640 1 --2- 172.21.15.47:0/751737988 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28600a2fc0 0x7f28600a5480 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2874009fd0 tx=0x7f2874009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.911+0000 7f2885d94740 1 -- start start 2024-03-21T11:30:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.911+0000 7f2885d94740 1 Processor -- start 2024-03-21T11:30:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.912+0000 7f2885d94740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf03e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.912+0000 7f2885d94740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf034bd0 0x562daf03e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.912+0000 7f2885d94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf039d40 0x562daf045930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.912+0000 7f2885d94740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562daef55a20 con 0x562daf0341e0 2024-03-21T11:30:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.912+0000 7f2885d94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562daef958d0 con 0x562daf039d40 2024-03-21T11:30:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.912+0000 7f2885d94740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562daf03ccf0 con 0x562daf034bd0 2024-03-21T11:30:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.912+0000 7f2885d94740 10 client.0 _ll_register_callbacks cb 0x562daef921c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.912+0000 7f287be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf03e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.912+0000 7f287b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf034bd0 0x562daf03e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.912+0000 7f287be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf03e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50868/0 (socket says 172.21.15.47:50868) 2024-03-21T11:30:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.912+0000 7f287be00640 1 -- 172.21.15.47:0/1643502555 learned_addr learned my addr 172.21.15.47:0/1643502555 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.912+0000 7f2881000640 1 --2- 172.21.15.47:0/1643502555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf039d40 0x562daf045930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.913+0000 7f287be00640 1 -- 172.21.15.47:0/1643502555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf034bd0 msgr2=0x562daf03e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.913+0000 7f287be00640 1 --2- 172.21.15.47:0/1643502555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf034bd0 0x562daf03e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.913+0000 7f287be00640 1 -- 172.21.15.47:0/1643502555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf039d40 msgr2=0x562daf045930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.913+0000 7f287be00640 1 --2- 172.21.15.47:0/1643502555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf039d40 0x562daf045930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.913+0000 7f287be00640 1 -- 172.21.15.47:0/1643502555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562daecae210 con 0x562daf0341e0 2024-03-21T11:30:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.913+0000 7f287b400640 1 --2- 172.21.15.47:0/1643502555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf034bd0 0x562daf03e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.913+0000 7f2881000640 1 --2- 172.21.15.47:0/1643502555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf039d40 0x562daf045930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.913+0000 7f287be00640 1 --2- 172.21.15.47:0/1643502555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf03e420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f286c000c00 tx=0x7f286c003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccfa56c15acb612 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.913+0000 7f2866000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:30:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.913+0000 7f2866000640 1 -- 172.21.15.47:0/1643502555 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c01a880 con 0x562daf0341e0 2024-03-21T11:30:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.913+0000 7f2866000640 1 -- 172.21.15.47:0/1643502555 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f286c005040 con 0x562daf0341e0 2024-03-21T11:30:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.913+0000 7f2885d94740 1 -- 172.21.15.47:0/1643502555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f286c015070 con 0x562daf0341e0 2024-03-21T11:30:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.914+0000 7f2866000640 1 -- 172.21.15.47:0/1643502555 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f286c031050 con 0x562daf0341e0 2024-03-21T11:30:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.914+0000 7f2885d94740 10 client.17571 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.914+0000 7f2885d94740 1 -- 172.21.15.47:0/1643502555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562daed7a620 con 0x562daf0341e0 2024-03-21T11:30:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.915+0000 7f2866000640 1 -- 172.21.15.47:0/1643502555 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f286c015070 con 0x562daf0341e0 2024-03-21T11:30:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.915+0000 7f2866000640 1 -- 172.21.15.47:0/1643502555 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f286c004b00 con 0x562daf0341e0 2024-03-21T11:30:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.915+0000 7f2885d94740 10 client.17571 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.915+0000 7f2885d94740 10 client.17571 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.915+0000 7f2885d94740 20 client.17571 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.915+0000 7f2885d94740 1 --2- 172.21.15.47:0/1643502555 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562daf049f20 0x562daf04c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.915+0000 7f2885d94740 4 client.17571 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.915+0000 7f2885d94740 1 -- 172.21.15.47:0/1643502555 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562daefa6d70 con 0x562daf049f20 2024-03-21T11:30:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.915+0000 7f287b400640 1 --2- 172.21.15.47:0/1643502555 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562daf049f20 0x562daf04c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.916+0000 7f287b400640 1 --2- 172.21.15.47:0/1643502555 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562daf049f20 0x562daf04c310 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=171d1ec29528c515 server_cookie=979cbe4008ac258f in_seq=0 out_seq=0 2024-03-21T11:30:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.916+0000 7f2866000640 10 client.17571 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.918+0000 7f2866000640 1 -- 172.21.15.47:0/1643502555 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562daefa6d70 con 0x562daf049f20 2024-03-21T11:30:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.918+0000 7f2866000640 10 client.17571 handle_command_reply: tid=0 2024-03-21T11:30:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.919+0000 7f2885d94740 1 client.17571 shutdown 2024-03-21T11:30:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.919+0000 7f2885d94740 1 -- 172.21.15.47:0/1643502555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 msgr2=0x562daf03e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.919+0000 7f2885d94740 1 --2- 172.21.15.47:0/1643502555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf03e420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f286c000c00 tx=0x7f286c003e00 comp rx=0 tx=0).stop 2024-03-21T11:30:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.919+0000 7f2885d94740 1 -- 172.21.15.47:0/1643502555 shutdown_connections 2024-03-21T11:30:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.919+0000 7f2885d94740 1 --2- 172.21.15.47:0/1643502555 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562daf049f20 0x562daf04c310 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.919+0000 7f2885d94740 1 --2- 172.21.15.47:0/1643502555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf039d40 0x562daf045930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.919+0000 7f2885d94740 1 --2- 172.21.15.47:0/1643502555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf034bd0 0x562daf03e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.919+0000 7f2885d94740 1 --2- 172.21.15.47:0/1643502555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf03e420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.919+0000 7f2885d94740 1 -- 172.21.15.47:0/1643502555 >> 172.21.15.47:0/1643502555 conn(0x562daf02f750 msgr2=0x562daf02e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.919+0000 7f2866000640 0 client.17571 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.919+0000 7f2885d94740 1 -- 172.21.15.47:0/1643502555 shutdown_connections 2024-03-21T11:30:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.919+0000 7f2885d94740 1 -- 172.21.15.47:0/1643502555 wait complete. 2024-03-21T11:30:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.919+0000 7f2885d94740 20 client.17571 trim_cache size 0 max 16384 2024-03-21T11:30:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.935+0000 7f2885d94740 1 Processor -- start 2024-03-21T11:30:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.935+0000 7f2885d94740 1 -- start start 2024-03-21T11:30:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.935+0000 7f2885d94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf0341e0 0x562daf0330c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.935+0000 7f2885d94740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf034bd0 0x562daf050ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.935+0000 7f2885d94740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf039d40 0x562daf051020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.935+0000 7f2885d94740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1ac220 con 0x562daf039d40 2024-03-21T11:30:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.935+0000 7f2885d94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1adba0 con 0x562daf0341e0 2024-03-21T11:30:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.935+0000 7f2885d94740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1ac880 con 0x562daf034bd0 2024-03-21T11:30:30.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.935+0000 7f287be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf0341e0 0x562daf0330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.935+0000 7f287be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf0341e0 0x562daf0330c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58744/0 (socket says 172.21.15.47:58744) 2024-03-21T11:30:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.935+0000 7f287b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf034bd0 0x562daf050ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.935+0000 7f287be00640 1 -- 172.21.15.47:0/3954928783 learned_addr learned my addr 172.21.15.47:0/3954928783 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.935+0000 7f2881000640 1 --2- 172.21.15.47:0/3954928783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf039d40 0x562daf051020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.936+0000 7f287be00640 1 -- 172.21.15.47:0/3954928783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf034bd0 msgr2=0x562daf050ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.936+0000 7f287be00640 1 --2- 172.21.15.47:0/3954928783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf034bd0 0x562daf050ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.936+0000 7f287be00640 1 -- 172.21.15.47:0/3954928783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf039d40 msgr2=0x562daf051020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.936+0000 7f287be00640 1 --2- 172.21.15.47:0/3954928783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf039d40 0x562daf051020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.936+0000 7f287be00640 1 -- 172.21.15.47:0/3954928783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562daf0304c0 con 0x562daf0341e0 2024-03-21T11:30:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.936+0000 7f287b400640 1 --2- 172.21.15.47:0/3954928783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf034bd0 0x562daf050ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.936+0000 7f287be00640 1 --2- 172.21.15.47:0/3954928783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf0341e0 0x562daf0330c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f286c01ad10 tx=0x7f286c0044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee7080887e4fce5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.936+0000 7f2870c00640 1 -- 172.21.15.47:0/3954928783 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c003e70 con 0x562daf0341e0 2024-03-21T11:30:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.936+0000 7f2870c00640 1 -- 172.21.15.47:0/3954928783 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f286c015040 con 0x562daf0341e0 2024-03-21T11:30:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.936+0000 7f2870c00640 1 -- 172.21.15.47:0/3954928783 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c0048b0 con 0x562daf0341e0 2024-03-21T11:30:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.937+0000 7f2885d94740 1 -- 172.21.15.47:0/3954928783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf0341e0 msgr2=0x562daf0330c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.937+0000 7f2885d94740 1 --2- 172.21.15.47:0/3954928783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf0341e0 0x562daf0330c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f286c01ad10 tx=0x7f286c0044f0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.937+0000 7f2885d94740 1 -- 172.21.15.47:0/3954928783 shutdown_connections 2024-03-21T11:30:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.937+0000 7f2885d94740 1 --2- 172.21.15.47:0/3954928783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf039d40 0x562daf051020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.937+0000 7f2885d94740 1 --2- 172.21.15.47:0/3954928783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf034bd0 0x562daf050ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.937+0000 7f2885d94740 1 --2- 172.21.15.47:0/3954928783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf0341e0 0x562daf0330c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.937+0000 7f2885d94740 1 -- 172.21.15.47:0/3954928783 >> 172.21.15.47:0/3954928783 conn(0x562daf02f750 msgr2=0x562daf0329b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.937+0000 7f2885d94740 1 -- 172.21.15.47:0/3954928783 shutdown_connections 2024-03-21T11:30:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.937+0000 7f2885d94740 1 -- 172.21.15.47:0/3954928783 wait complete. 2024-03-21T11:30:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.938+0000 7f2885d94740 1 -- start start 2024-03-21T11:30:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.938+0000 7f2885d94740 1 Processor -- start 2024-03-21T11:30:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.938+0000 7f2885d94740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf042280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.938+0000 7f2885d94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf034bd0 0x562daf0427c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.938+0000 7f2885d94740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf039d40 0x562daf043760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.938+0000 7f2885d94740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1ad540 con 0x562daf0341e0 2024-03-21T11:30:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.938+0000 7f2885d94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1acee0 con 0x562daf034bd0 2024-03-21T11:30:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.938+0000 7f2885d94740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1aa3e0 con 0x562daf039d40 2024-03-21T11:30:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.938+0000 7f2885d94740 10 client.0 _ll_register_callbacks cb 0x7f2874009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.939+0000 7f2881000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf039d40 0x562daf043760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.939+0000 7f287b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf034bd0 0x562daf0427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.939+0000 7f287be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf042280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.939+0000 7f2881000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf039d40 0x562daf043760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34414/0 (socket says 172.21.15.47:34414) 2024-03-21T11:30:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.939+0000 7f2881000640 1 -- 172.21.15.47:0/1609997572 learned_addr learned my addr 172.21.15.47:0/1609997572 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.939+0000 7f287b400640 1 -- 172.21.15.47:0/1609997572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf039d40 msgr2=0x562daf043760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.939+0000 7f287b400640 1 --2- 172.21.15.47:0/1609997572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf039d40 0x562daf043760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.939+0000 7f287b400640 1 -- 172.21.15.47:0/1609997572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 msgr2=0x562daf042280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.939+0000 7f287b400640 1 --2- 172.21.15.47:0/1609997572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf042280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.939+0000 7f287b400640 1 -- 172.21.15.47:0/1609997572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562daf043d50 con 0x562daf034bd0 2024-03-21T11:30:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.939+0000 7f2881000640 1 --2- 172.21.15.47:0/1609997572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf039d40 0x562daf043760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.939+0000 7f287b400640 1 --2- 172.21.15.47:0/1609997572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf034bd0 0x562daf0427c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f28680eeff0 tx=0x7f28680f7530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1154840742a89b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.939+0000 7f2866000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.939+0000 7f2866000640 1 -- 172.21.15.47:0/1609997572 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28680bd760 con 0x562daf034bd0 2024-03-21T11:30:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.940+0000 7f2866000640 1 -- 172.21.15.47:0/1609997572 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f28680bbf20 con 0x562daf034bd0 2024-03-21T11:30:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.940+0000 7f2885d94740 1 -- 172.21.15.47:0/1609997572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f286c015040 con 0x562daf034bd0 2024-03-21T11:30:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.940+0000 7f2866000640 1 -- 172.21.15.47:0/1609997572 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28680bd580 con 0x562daf034bd0 2024-03-21T11:30:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.940+0000 7f2885d94740 10 client.27866 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.940+0000 7f2885d94740 1 -- 172.21.15.47:0/1609997572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562daf0450d0 con 0x562daf034bd0 2024-03-21T11:30:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.940+0000 7f2866000640 1 -- 172.21.15.47:0/1609997572 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f286c015040 con 0x562daf034bd0 2024-03-21T11:30:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.940+0000 7f2866000640 1 -- 172.21.15.47:0/1609997572 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f28680c61a0 con 0x562daf034bd0 2024-03-21T11:30:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.940+0000 7f2885d94740 10 client.27866 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.940+0000 7f2885d94740 10 client.27866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.940+0000 7f2885d94740 20 client.27866 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.940+0000 7f2885d94740 1 --2- 172.21.15.47:0/1609997572 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562daf0a5ac0 0x562daf0a7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.940+0000 7f2885d94740 4 client.27866 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:30:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.940+0000 7f2885d94740 1 -- 172.21.15.47:0/1609997572 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562daefa10c0 con 0x562daf0a5ac0 2024-03-21T11:30:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.941+0000 7f287be00640 1 --2- 172.21.15.47:0/1609997572 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562daf0a5ac0 0x562daf0a7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.941+0000 7f287be00640 1 --2- 172.21.15.47:0/1609997572 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562daf0a5ac0 0x562daf0a7ee0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0cf47106d4b62f2 server_cookie=58823ebbbbc1b7bb in_seq=0 out_seq=0 2024-03-21T11:30:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:30.941+0000 7f2866000640 10 client.27866 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:31.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:30 smithi100 ceph-mon[36245]: pgmap v615: 129 pgs: 129 active+clean; 19 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.4 MiB/s wr, 357 op/s 2024-03-21T11:30:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.205+0000 7f2866000640 1 -- 172.21.15.47:0/1609997572 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562daefa10c0 con 0x562daf0a5ac0 2024-03-21T11:30:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.205+0000 7f2866000640 10 client.27866 handle_command_reply: tid=0 2024-03-21T11:30:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.205+0000 7f2885d94740 1 client.27866 shutdown 2024-03-21T11:30:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.206+0000 7f2885d94740 1 -- 172.21.15.47:0/1609997572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf034bd0 msgr2=0x562daf0427c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.206+0000 7f2885d94740 1 --2- 172.21.15.47:0/1609997572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf034bd0 0x562daf0427c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f28680eeff0 tx=0x7f28680f7530 comp rx=0 tx=0).stop 2024-03-21T11:30:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.206+0000 7f2885d94740 1 -- 172.21.15.47:0/1609997572 shutdown_connections 2024-03-21T11:30:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.206+0000 7f2885d94740 1 --2- 172.21.15.47:0/1609997572 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562daf0a5ac0 0x562daf0a7ee0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.206+0000 7f2885d94740 1 --2- 172.21.15.47:0/1609997572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562daf039d40 0x562daf043760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.206+0000 7f2885d94740 1 --2- 172.21.15.47:0/1609997572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562daf034bd0 0x562daf0427c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.206+0000 7f2885d94740 1 --2- 172.21.15.47:0/1609997572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562daf0341e0 0x562daf042280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.206+0000 7f2866000640 0 client.27866 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.206+0000 7f2885d94740 1 -- 172.21.15.47:0/1609997572 >> 172.21.15.47:0/1609997572 conn(0x562daf02f750 msgr2=0x562daf033240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.206+0000 7f2885d94740 1 -- 172.21.15.47:0/1609997572 shutdown_connections 2024-03-21T11:30:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.206+0000 7f2885d94740 1 -- 172.21.15.47:0/1609997572 wait complete. 2024-03-21T11:30:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.207+0000 7f2885d94740 20 client.27866 trim_cache size 0 max 16384 2024-03-21T11:30:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:30:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:30:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:30:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:30:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.208+0000 7f2883000640 1 -- 172.21.15.47:0/751737988 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28600a2fc0 msgr2=0x7f28600a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.208+0000 7f2883000640 1 --2- 172.21.15.47:0/751737988 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28600a2fc0 0x7f28600a5480 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2874009fd0 tx=0x7f2874009500 comp rx=0 tx=0).stop 2024-03-21T11:30:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.208+0000 7f2883000640 1 -- 172.21.15.47:0/751737988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f287c104ac0 msgr2=0x7f287c19a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.208+0000 7f2883000640 1 --2- 172.21.15.47:0/751737988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f287c104ac0 0x7f287c19a060 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2868005ab0 tx=0x7f286802c6c0 comp rx=0 tx=0).stop 2024-03-21T11:30:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.208+0000 7f2883000640 1 -- 172.21.15.47:0/751737988 shutdown_connections 2024-03-21T11:30:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.208+0000 7f2883000640 1 --2- 172.21.15.47:0/751737988 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28600a2fc0 0x7f28600a5480 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.208+0000 7f2883000640 1 --2- 172.21.15.47:0/751737988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f287c105480 0x7f287c1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.208+0000 7f2883000640 1 --2- 172.21.15.47:0/751737988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f287c104ac0 0x7f287c19a060 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.208+0000 7f2883000640 1 --2- 172.21.15.47:0/751737988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f287c1038d0 0x7f287c199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.208+0000 7f2883000640 1 -- 172.21.15.47:0/751737988 >> 172.21.15.47:0/751737988 conn(0x7f287c0fee60 msgr2=0x7f287c107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.208+0000 7f2883000640 1 -- 172.21.15.47:0/751737988 shutdown_connections 2024-03-21T11:30:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:31.208+0000 7f2883000640 1 -- 172.21.15.47:0/751737988 wait complete. 2024-03-21T11:30:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:30 smithi102 ceph-mon[36748]: pgmap v615: 129 pgs: 129 active+clean; 19 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.4 MiB/s wr, 357 op/s 2024-03-21T11:30:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:30 smithi047 ceph-mon[31351]: pgmap v615: 129 pgs: 129 active+clean; 19 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 2.4 MiB/s wr, 357 op/s 2024-03-21T11:30:31.529 INFO:tasks.workunit.client.0.smithi047.stdout: 1 491432 8.27 MB/sec execute 195 sec latency 137.617 ms 2024-03-21T11:30:32.529 INFO:tasks.workunit.client.0.smithi047.stdout: 1 493640 8.29 MB/sec execute 196 sec latency 16.651 ms 2024-03-21T11:30:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:32 smithi100 ceph-mon[36245]: pgmap v616: 129 pgs: 129 active+clean; 18 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.7 MiB/s wr, 251 op/s 2024-03-21T11:30:33.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:32 smithi102 ceph-mon[36748]: pgmap v616: 129 pgs: 129 active+clean; 18 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.7 MiB/s wr, 251 op/s 2024-03-21T11:30:33.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:32 smithi047 ceph-mon[31351]: pgmap v616: 129 pgs: 129 active+clean; 18 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.7 MiB/s wr, 251 op/s 2024-03-21T11:30:33.529 INFO:tasks.workunit.client.0.smithi047.stdout: 1 494642 8.27 MB/sec execute 197 sec latency 11.477 ms 2024-03-21T11:30:34.529 INFO:tasks.workunit.client.0.smithi047.stdout: 1 496422 8.27 MB/sec execute 198 sec latency 13.115 ms 2024-03-21T11:30:35.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:34 smithi100 ceph-mon[36245]: pgmap v617: 129 pgs: 129 active+clean; 15 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 2.3 MiB/s wr, 333 op/s 2024-03-21T11:30:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:34 smithi102 ceph-mon[36748]: pgmap v617: 129 pgs: 129 active+clean; 15 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 2.3 MiB/s wr, 333 op/s 2024-03-21T11:30:35.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:34 smithi047 ceph-mon[31351]: pgmap v617: 129 pgs: 129 active+clean; 15 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 2.3 MiB/s wr, 333 op/s 2024-03-21T11:30:35.529 INFO:tasks.workunit.client.0.smithi047.stdout: 1 497778 8.28 MB/sec execute 199 sec latency 63.719 ms 2024-03-21T11:30:36.529 INFO:tasks.workunit.client.0.smithi047.stdout: 1 498888 8.27 MB/sec execute 200 sec latency 34.259 ms 2024-03-21T11:30:36.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:30:36.804 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:30:37.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.045+0000 7f4b78c00640 1 -- 172.21.15.47:0/509533686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b741007d0 msgr2=0x7f4b74100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.045+0000 7f4b78c00640 1 --2- 172.21.15.47:0/509533686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b741007d0 0x7f4b74100c30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009900 tx=0x7f4b6002bcd0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.046+0000 7f4b78c00640 1 -- 172.21.15.47:0/509533686 shutdown_connections 2024-03-21T11:30:37.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.046+0000 7f4b78c00640 1 --2- 172.21.15.47:0/509533686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b74101170 0x7f4b741015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.046+0000 7f4b78c00640 1 --2- 172.21.15.47:0/509533686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b741007d0 0x7f4b74100c30 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.046+0000 7f4b78c00640 1 --2- 172.21.15.47:0/509533686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b740ffe70 0x7f4b74100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.046+0000 7f4b78c00640 1 -- 172.21.15.47:0/509533686 >> 172.21.15.47:0/509533686 conn(0x7f4b740fba10 msgr2=0x7f4b740fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:37.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.046+0000 7f4b78c00640 1 -- 172.21.15.47:0/509533686 shutdown_connections 2024-03-21T11:30:37.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.046+0000 7f4b78c00640 1 -- 172.21.15.47:0/509533686 wait complete. 2024-03-21T11:30:37.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.047+0000 7f4b78c00640 1 Processor -- start 2024-03-21T11:30:37.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.047+0000 7f4b78c00640 1 -- start start 2024-03-21T11:30:37.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.047+0000 7f4b78c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b740ffe70 0x7f4b74199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.047+0000 7f4b78c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b741007d0 0x7f4b74199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.047+0000 7f4b78c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b74101170 0x7f4b741a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.047+0000 7f4b78c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7410f450 con 0x7f4b74101170 2024-03-21T11:30:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.047+0000 7f4b78c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7410f2d0 con 0x7f4b740ffe70 2024-03-21T11:30:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.047+0000 7f4b78c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7410f5d0 con 0x7f4b741007d0 2024-03-21T11:30:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.048+0000 7f4b73400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b740ffe70 0x7f4b74199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.048+0000 7f4b73400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b740ffe70 0x7f4b74199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58760/0 (socket says 172.21.15.47:58760) 2024-03-21T11:30:37.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.048+0000 7f4b73400640 1 -- 172.21.15.47:0/432130148 learned_addr learned my addr 172.21.15.47:0/432130148 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.048+0000 7f4b72a00640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b741007d0 0x7f4b74199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.048+0000 7f4b73e00640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b74101170 0x7f4b741a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.048+0000 7f4b73400640 1 -- 172.21.15.47:0/432130148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b741007d0 msgr2=0x7f4b74199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.048+0000 7f4b73400640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b741007d0 0x7f4b74199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.048+0000 7f4b73400640 1 -- 172.21.15.47:0/432130148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b74101170 msgr2=0x7f4b741a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.048+0000 7f4b73400640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b74101170 0x7f4b741a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.048+0000 7f4b73400640 1 -- 172.21.15.47:0/432130148 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b741a1680 con 0x7f4b740ffe70 2024-03-21T11:30:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.048+0000 7f4b73e00640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b74101170 0x7f4b741a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.048+0000 7f4b73400640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b740ffe70 0x7f4b74199a60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400e9f0 tx=0x7f4b6400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.049+0000 7f4b6be00640 1 -- 172.21.15.47:0/432130148 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400cd80 con 0x7f4b740ffe70 2024-03-21T11:30:37.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.049+0000 7f4b6be00640 1 -- 172.21.15.47:0/432130148 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b64010910 con 0x7f4b740ffe70 2024-03-21T11:30:37.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.049+0000 7f4b78c00640 1 -- 172.21.15.47:0/432130148 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b741a1970 con 0x7f4b740ffe70 2024-03-21T11:30:37.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.049+0000 7f4b6be00640 1 -- 172.21.15.47:0/432130148 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64018990 con 0x7f4b740ffe70 2024-03-21T11:30:37.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.049+0000 7f4b78c00640 1 -- 172.21.15.47:0/432130148 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b741a1e90 con 0x7f4b740ffe70 2024-03-21T11:30:37.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.053+0000 7f4b7cdfd740 1 Processor -- start 2024-03-21T11:30:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.053+0000 7f4b7cdfd740 1 -- start start 2024-03-21T11:30:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.054+0000 7f4b6be00640 1 -- 172.21.15.47:0/432130148 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4b64010ac0 con 0x7f4b740ffe70 2024-03-21T11:30:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.054+0000 7f4b7cdfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.054+0000 7f4b7cdfd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d0a40 0x562be61d5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.054+0000 7f4b7cdfd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d5bb0 0x562be61d8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.054+0000 7f4b7cdfd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be6135250 con 0x562be61d0a40 2024-03-21T11:30:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.054+0000 7f4b7cdfd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be60dda30 con 0x562be61d5bb0 2024-03-21T11:30:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.054+0000 7f4b7cdfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be61312f0 con 0x562be61d0050 2024-03-21T11:30:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.054+0000 7f4b72a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d5bb0 0x562be61d8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.054+0000 7f4b73400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d0a40 0x562be61d5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.054+0000 7f4b73e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.054+0000 7f4b72a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d5bb0 0x562be61d8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58762/0 (socket says 172.21.15.47:58762) 2024-03-21T11:30:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.054+0000 7f4b72a00640 1 -- 172.21.15.47:0/3836203632 learned_addr learned my addr 172.21.15.47:0/3836203632 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:37.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.055+0000 7f4b72a00640 1 -- 172.21.15.47:0/3836203632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 msgr2=0x562be61d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.055+0000 7f4b72a00640 1 --2- 172.21.15.47:0/3836203632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.055+0000 7f4b72a00640 1 -- 172.21.15.47:0/3836203632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d0a40 msgr2=0x562be61d5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.055+0000 7f4b72a00640 1 --2- 172.21.15.47:0/3836203632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d0a40 0x562be61d5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.055+0000 7f4b72a00640 1 -- 172.21.15.47:0/3836203632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562be60c8690 con 0x562be61d5bb0 2024-03-21T11:30:37.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.055+0000 7f4b72a00640 1 --2- 172.21.15.47:0/3836203632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d5bb0 0x562be61d8040 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4b60000c00 tx=0x7f4b6002c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c227a68be9b2de77 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:37.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.055+0000 7f4b5fe00640 1 -- 172.21.15.47:0/3836203632 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60043070 con 0x562be61d5bb0 2024-03-21T11:30:37.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.056+0000 7f4b5fe00640 1 -- 172.21.15.47:0/3836203632 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b60004040 con 0x562be61d5bb0 2024-03-21T11:30:37.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.056+0000 7f4b5fe00640 1 -- 172.21.15.47:0/3836203632 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60004c60 con 0x562be61d5bb0 2024-03-21T11:30:37.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.056+0000 7f4b6be00640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b580ab730 0x7f4b580adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.056+0000 7f4b6be00640 1 -- 172.21.15.47:0/432130148 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4b640cfb20 con 0x7f4b740ffe70 2024-03-21T11:30:37.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.056+0000 7f4b73e00640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b580ab730 0x7f4b580adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.057+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/3836203632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d5bb0 msgr2=0x562be61d8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.057+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/3836203632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d5bb0 0x562be61d8040 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4b60000c00 tx=0x7f4b6002c700 comp rx=0 tx=0).stop 2024-03-21T11:30:37.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.057+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/3836203632 shutdown_connections 2024-03-21T11:30:37.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.057+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/3836203632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d5bb0 0x562be61d8040 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.057+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/3836203632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d0a40 0x562be61d5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.057+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/3836203632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.057+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/3836203632 >> 172.21.15.47:0/3836203632 conn(0x562be6198200 msgr2=0x562be61cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.057+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/3836203632 shutdown_connections 2024-03-21T11:30:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.057+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/3836203632 wait complete. 2024-03-21T11:30:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.057+0000 7f4b73e00640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b580ab730 0x7f4b580adbf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x562be61d12a0 tx=0x7f4b6c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.057+0000 7f4b7cdfd740 1 -- start start 2024-03-21T11:30:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.058+0000 7f4b7cdfd740 1 Processor -- start 2024-03-21T11:30:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.058+0000 7f4b7cdfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61d9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.058+0000 7f4b7cdfd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d0a40 0x562be61da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.058+0000 7f4b7cdfd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d5bb0 0x562be61e14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.058+0000 7f4b7cdfd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be61d8b00 con 0x562be61d0a40 2024-03-21T11:30:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.058+0000 7f4b7cdfd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be60f0e80 con 0x562be61d5bb0 2024-03-21T11:30:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.058+0000 7f4b7cdfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be61d8c80 con 0x562be61d0050 2024-03-21T11:30:37.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.058+0000 7f4b7cdfd740 10 client.0 _ll_register_callbacks cb 0x562be6103130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.058+0000 7f4b73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61d9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.058+0000 7f4b73e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d5bb0 0x562be61e14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.058+0000 7f4b73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61d9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34450/0 (socket says 172.21.15.47:34450) 2024-03-21T11:30:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.058+0000 7f4b73400640 1 -- 172.21.15.47:0/1389194401 learned_addr learned my addr 172.21.15.47:0/1389194401 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.059+0000 7f4b72a00640 1 --2- 172.21.15.47:0/1389194401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d0a40 0x562be61da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.059+0000 7f4b73400640 1 -- 172.21.15.47:0/1389194401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d5bb0 msgr2=0x562be61e14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.059+0000 7f4b73400640 1 --2- 172.21.15.47:0/1389194401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d5bb0 0x562be61e14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.059+0000 7f4b73400640 1 -- 172.21.15.47:0/1389194401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d0a40 msgr2=0x562be61da4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.059+0000 7f4b73400640 1 --2- 172.21.15.47:0/1389194401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d0a40 0x562be61da4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.059+0000 7f4b73400640 1 -- 172.21.15.47:0/1389194401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562be5f137d0 con 0x562be61d0050 2024-03-21T11:30:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.059+0000 7f4b73400640 1 --2- 172.21.15.47:0/1389194401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61d9f60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400e9c0 tx=0x7f4b640981e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e17527dc9f4714a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:37.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.059+0000 7f4b5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:37.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.059+0000 7f4b5cc00640 1 -- 172.21.15.47:0/1389194401 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b640a4640 con 0x562be61d0050 2024-03-21T11:30:37.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.059+0000 7f4b5cc00640 1 -- 172.21.15.47:0/1389194401 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4b64018b30 con 0x562be61d0050 2024-03-21T11:30:37.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.059+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/1389194401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b60004040 con 0x562be61d0050 2024-03-21T11:30:37.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.060+0000 7f4b5cc00640 1 -- 172.21.15.47:0/1389194401 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b6409c5c0 con 0x562be61d0050 2024-03-21T11:30:37.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.060+0000 7f4b7cdfd740 10 client.27895 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:37.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.060+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/1389194401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562be5e493f0 con 0x562be61d0050 2024-03-21T11:30:37.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.060+0000 7f4b5cc00640 1 -- 172.21.15.47:0/1389194401 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b640d5030 con 0x562be61d0050 2024-03-21T11:30:37.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.060+0000 7f4b5cc00640 1 -- 172.21.15.47:0/1389194401 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4b6409b310 con 0x562be61d0050 2024-03-21T11:30:37.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.060+0000 7f4b7cdfd740 10 client.27895 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:37.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.060+0000 7f4b7cdfd740 10 client.27895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:37.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.060+0000 7f4b7cdfd740 20 client.27895 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:37.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.060+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/1389194401 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562be61e5df0 0x562be61e82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.060+0000 7f4b7cdfd740 4 client.27895 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:37.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.060+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/1389194401 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562be6142790 con 0x562be61e5df0 2024-03-21T11:30:37.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.060+0000 7f4b72a00640 1 --2- 172.21.15.47:0/1389194401 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562be61e5df0 0x562be61e82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.061+0000 7f4b72a00640 1 --2- 172.21.15.47:0/1389194401 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562be61e5df0 0x562be61e82b0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b851e43f2f9e650 server_cookie=f6834089ebfb14e3 in_seq=0 out_seq=0 2024-03-21T11:30:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.061+0000 7f4b5cc00640 10 client.27895 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.062+0000 7f4b5cc00640 1 -- 172.21.15.47:0/1389194401 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562be6142790 con 0x562be61e5df0 2024-03-21T11:30:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.062+0000 7f4b5cc00640 10 client.27895 handle_command_reply: tid=0 2024-03-21T11:30:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.062+0000 7f4b7cdfd740 1 client.27895 shutdown 2024-03-21T11:30:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.062+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/1389194401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 msgr2=0x562be61d9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.062+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/1389194401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61d9f60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400e9c0 tx=0x7f4b640981e0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.062+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/1389194401 shutdown_connections 2024-03-21T11:30:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.062+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/1389194401 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562be61e5df0 0x562be61e82b0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.062+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/1389194401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d5bb0 0x562be61e14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.062+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/1389194401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d0a40 0x562be61da4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.062+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/1389194401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61d9f60 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.062+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/1389194401 >> 172.21.15.47:0/1389194401 conn(0x562be6198200 msgr2=0x562be61cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.062+0000 7f4b5cc00640 0 client.27895 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.062+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/1389194401 shutdown_connections 2024-03-21T11:30:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.062+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/1389194401 wait complete. 2024-03-21T11:30:37.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.063+0000 7f4b7cdfd740 20 client.27895 trim_cache size 0 max 16384 2024-03-21T11:30:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.082+0000 7f4b7cdfd740 1 Processor -- start 2024-03-21T11:30:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.082+0000 7f4b7cdfd740 1 -- start start 2024-03-21T11:30:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.082+0000 7f4b7cdfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.082+0000 7f4b7cdfd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d0a40 0x562be61e14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.082+0000 7f4b7cdfd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d5bb0 0x562be61d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.082+0000 7f4b7cdfd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741a7c30 con 0x562be61d5bb0 2024-03-21T11:30:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.082+0000 7f4b7cdfd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741a95b0 con 0x562be61d0a40 2024-03-21T11:30:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.082+0000 7f4b7cdfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741a8290 con 0x562be61d0050 2024-03-21T11:30:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.082+0000 7f4b73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.082+0000 7f4b73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34460/0 (socket says 172.21.15.47:34460) 2024-03-21T11:30:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.082+0000 7f4b73400640 1 -- 172.21.15.47:0/3489792695 learned_addr learned my addr 172.21.15.47:0/3489792695 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.082+0000 7f4b72a00640 1 --2- 172.21.15.47:0/3489792695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d0a40 0x562be61e14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.082+0000 7f4b73e00640 1 --2- 172.21.15.47:0/3489792695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d5bb0 0x562be61d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.083+0000 7f4b72a00640 1 -- 172.21.15.47:0/3489792695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 msgr2=0x562be61cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.083+0000 7f4b72a00640 1 --2- 172.21.15.47:0/3489792695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.083+0000 7f4b72a00640 1 -- 172.21.15.47:0/3489792695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d5bb0 msgr2=0x562be61d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.083+0000 7f4b72a00640 1 --2- 172.21.15.47:0/3489792695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d5bb0 0x562be61d19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.083+0000 7f4b72a00640 1 -- 172.21.15.47:0/3489792695 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562be5e090f0 con 0x562be61d0a40 2024-03-21T11:30:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.083+0000 7f4b73400640 1 --2- 172.21.15.47:0/3489792695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.083+0000 7f4b72a00640 1 --2- 172.21.15.47:0/3489792695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d0a40 0x562be61e14a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4b60005ab0 tx=0x7f4b6002bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4202222fac1d7a89 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.083+0000 7f4b5f400640 1 -- 172.21.15.47:0/3489792695 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60043040 con 0x562be61d0a40 2024-03-21T11:30:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.083+0000 7f4b5f400640 1 -- 172.21.15.47:0/3489792695 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b60004310 con 0x562be61d0a40 2024-03-21T11:30:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.083+0000 7f4b5f400640 1 -- 172.21.15.47:0/3489792695 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60004f10 con 0x562be61d0a40 2024-03-21T11:30:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.084+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/3489792695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d0a40 msgr2=0x562be61e14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.084+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/3489792695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d0a40 0x562be61e14a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4b60005ab0 tx=0x7f4b6002bcd0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.084+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/3489792695 shutdown_connections 2024-03-21T11:30:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.084+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/3489792695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d5bb0 0x562be61d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.084+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/3489792695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d0a40 0x562be61e14a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.084+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/3489792695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0050 0x562be61cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.084+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/3489792695 >> 172.21.15.47:0/3489792695 conn(0x562be6198200 msgr2=0x562be61d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.085+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/3489792695 shutdown_connections 2024-03-21T11:30:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.085+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/3489792695 wait complete. 2024-03-21T11:30:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.085+0000 7f4b7cdfd740 1 -- start start 2024-03-21T11:30:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.085+0000 7f4b7cdfd740 1 Processor -- start 2024-03-21T11:30:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.086+0000 7f4b7cdfd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d0050 0x562be61df280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.086+0000 7f4b7cdfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0a40 0x562be61df7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.086+0000 7f4b7cdfd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d5bb0 0x562be61e07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.086+0000 7f4b7cdfd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741a8f50 con 0x562be61d5bb0 2024-03-21T11:30:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.086+0000 7f4b7cdfd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741a88f0 con 0x562be61d0050 2024-03-21T11:30:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.086+0000 7f4b7cdfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b741a5df0 con 0x562be61d0a40 2024-03-21T11:30:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.086+0000 7f4b7cdfd740 10 client.0 _ll_register_callbacks cb 0x7f4b640d32f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.086+0000 7f4b72a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0a40 0x562be61df7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.086+0000 7f4b72a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0a40 0x562be61df7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34470/0 (socket says 172.21.15.47:34470) 2024-03-21T11:30:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.086+0000 7f4b72a00640 1 -- 172.21.15.47:0/880706822 learned_addr learned my addr 172.21.15.47:0/880706822 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.086+0000 7f4b73400640 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d0050 0x562be61df280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.086+0000 7f4b73e00640 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d5bb0 0x562be61e07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.087+0000 7f4b72a00640 1 -- 172.21.15.47:0/880706822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d0050 msgr2=0x562be61df280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.087+0000 7f4b72a00640 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d0050 0x562be61df280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.087+0000 7f4b72a00640 1 -- 172.21.15.47:0/880706822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d5bb0 msgr2=0x562be61e07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.087+0000 7f4b72a00640 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d5bb0 0x562be61e07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.087+0000 7f4b72a00640 1 -- 172.21.15.47:0/880706822 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562be61e0dc0 con 0x562be61d0a40 2024-03-21T11:30:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.087+0000 7f4b73400640 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d0050 0x562be61df280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.087+0000 7f4b73e00640 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d5bb0 0x562be61e07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.087+0000 7f4b72a00640 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0a40 0x562be61df7c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4b600096c0 tx=0x7f4b600058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a865de0e4e1613f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.087+0000 7f4b5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.088+0000 7f4b5cc00640 1 -- 172.21.15.47:0/880706822 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6002ca20 con 0x562be61d0a40 2024-03-21T11:30:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.088+0000 7f4b5cc00640 1 -- 172.21.15.47:0/880706822 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4b6002cbc0 con 0x562be61d0a40 2024-03-21T11:30:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.088+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/880706822 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562be613cae0 con 0x562be61d0a40 2024-03-21T11:30:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.088+0000 7f4b5cc00640 1 -- 172.21.15.47:0/880706822 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b6002cdc0 con 0x562be61d0a40 2024-03-21T11:30:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.088+0000 7f4b7cdfd740 10 client.27901 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.088+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/880706822 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562be6241d40 con 0x562be61d0a40 2024-03-21T11:30:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.088+0000 7f4b5cc00640 1 -- 172.21.15.47:0/880706822 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562be613cae0 con 0x562be61d0a40 2024-03-21T11:30:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.088+0000 7f4b5cc00640 1 -- 172.21.15.47:0/880706822 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4b60003860 con 0x562be61d0a40 2024-03-21T11:30:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.088+0000 7f4b7cdfd740 10 client.27901 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.088+0000 7f4b7cdfd740 10 client.27901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.088+0000 7f4b7cdfd740 20 client.27901 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.088+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562be62422f0 0x562be62446e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.088+0000 7f4b7cdfd740 4 client.27901 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:30:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.088+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/880706822 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562be60ba3a0 con 0x562be62422f0 2024-03-21T11:30:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.089+0000 7f4b73400640 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562be62422f0 0x562be62446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.089+0000 7f4b73400640 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562be62422f0 0x562be62446e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a8fc9a921cbd10 server_cookie=957ad0948e3ead4c in_seq=0 out_seq=0 2024-03-21T11:30:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.089+0000 7f4b5cc00640 10 client.27901 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.091+0000 7f4b5cc00640 1 -- 172.21.15.47:0/880706822 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562be60ba3a0 con 0x562be62422f0 2024-03-21T11:30:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.091+0000 7f4b5cc00640 10 client.27901 handle_command_reply: tid=0 2024-03-21T11:30:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.091+0000 7f4b7cdfd740 1 client.27901 shutdown 2024-03-21T11:30:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.091+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/880706822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0a40 msgr2=0x562be61df7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.091+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0a40 0x562be61df7c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4b600096c0 tx=0x7f4b600058e0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.091+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/880706822 shutdown_connections 2024-03-21T11:30:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.091+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562be62422f0 0x562be62446e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.091+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562be61d5bb0 0x562be61e07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.091+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562be61d0a40 0x562be61df7c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.091+0000 7f4b7cdfd740 1 --2- 172.21.15.47:0/880706822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562be61d0050 0x562be61df280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.091+0000 7f4b5cc00640 0 client.27901 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.091+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/880706822 >> 172.21.15.47:0/880706822 conn(0x562be6198200 msgr2=0x562be61caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.092+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/880706822 shutdown_connections 2024-03-21T11:30:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.092+0000 7f4b7cdfd740 1 -- 172.21.15.47:0/880706822 wait complete. 2024-03-21T11:30:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.092+0000 7f4b7cdfd740 20 client.27901 trim_cache size 0 max 16384 2024-03-21T11:30:37.096 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:30:37.096 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:30:37.096 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:30:37.096 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:30:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.093+0000 7f4b78c00640 1 -- 172.21.15.47:0/432130148 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b580ab730 msgr2=0x7f4b580adbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.093+0000 7f4b78c00640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b580ab730 0x7f4b580adbf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x562be61d12a0 tx=0x7f4b6c009500 comp rx=0 tx=0).stop 2024-03-21T11:30:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.093+0000 7f4b78c00640 1 -- 172.21.15.47:0/432130148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b740ffe70 msgr2=0x7f4b74199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.093+0000 7f4b78c00640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b740ffe70 0x7f4b74199a60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400e9f0 tx=0x7f4b6400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.093+0000 7f4b78c00640 1 -- 172.21.15.47:0/432130148 shutdown_connections 2024-03-21T11:30:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.093+0000 7f4b78c00640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b580ab730 0x7f4b580adbf0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.093+0000 7f4b78c00640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b74101170 0x7f4b741a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.093+0000 7f4b78c00640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b741007d0 0x7f4b74199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.093+0000 7f4b78c00640 1 --2- 172.21.15.47:0/432130148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b740ffe70 0x7f4b74199a60 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.093+0000 7f4b78c00640 1 -- 172.21.15.47:0/432130148 >> 172.21.15.47:0/432130148 conn(0x7f4b740fba10 msgr2=0x7f4b740fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.093+0000 7f4b78c00640 1 -- 172.21.15.47:0/432130148 shutdown_connections 2024-03-21T11:30:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.093+0000 7f4b78c00640 1 -- 172.21.15.47:0/432130148 wait complete. 2024-03-21T11:30:37.114 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:30:37.114 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:30:37.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:36 smithi100 ceph-mon[36245]: pgmap v618: 129 pgs: 129 active+clean; 15 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.7 MiB/s wr, 250 op/s 2024-03-21T11:30:37.193 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:36 smithi047 ceph-mon[31351]: pgmap v618: 129 pgs: 129 active+clean; 15 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.7 MiB/s wr, 250 op/s 2024-03-21T11:30:37.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:36 smithi102 ceph-mon[36748]: pgmap v618: 129 pgs: 129 active+clean; 15 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.7 MiB/s wr, 250 op/s 2024-03-21T11:30:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.342+0000 7f9d44400640 1 -- 172.21.15.47:0/2000024045 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d3c1028a0 msgr2=0x7f9d3c104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.342+0000 7f9d44400640 1 --2- 172.21.15.47:0/2000024045 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d3c1028a0 0x7f9d3c104d10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009900 tx=0x7f9d2802bc30 comp rx=0 tx=0).stop 2024-03-21T11:30:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.342+0000 7f9d44400640 1 -- 172.21.15.47:0/2000024045 shutdown_connections 2024-03-21T11:30:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.342+0000 7f9d44400640 1 --2- 172.21.15.47:0/2000024045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d3c105250 0x7f9d3c1056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.342+0000 7f9d44400640 1 --2- 172.21.15.47:0/2000024045 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d3c1028a0 0x7f9d3c104d10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.342+0000 7f9d44400640 1 --2- 172.21.15.47:0/2000024045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d3c0fff30 0x7f9d3c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.342+0000 7f9d44400640 1 -- 172.21.15.47:0/2000024045 >> 172.21.15.47:0/2000024045 conn(0x7f9d3c0fbb30 msgr2=0x7f9d3c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:37.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.342+0000 7f9d44400640 1 -- 172.21.15.47:0/2000024045 shutdown_connections 2024-03-21T11:30:37.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.342+0000 7f9d44400640 1 -- 172.21.15.47:0/2000024045 wait complete. 2024-03-21T11:30:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.343+0000 7f9d44400640 1 Processor -- start 2024-03-21T11:30:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.343+0000 7f9d44400640 1 -- start start 2024-03-21T11:30:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.344+0000 7f9d44400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d3c0fff30 0x7f9d3c19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.344+0000 7f9d44400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d3c1028a0 0x7f9d3c19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.344+0000 7f9d44400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d3c105250 0x7f9d3c1a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.344+0000 7f9d44400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c113980 con 0x7f9d3c0fff30 2024-03-21T11:30:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.344+0000 7f9d44400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c113800 con 0x7f9d3c1028a0 2024-03-21T11:30:37.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.344+0000 7f9d44400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c113b00 con 0x7f9d3c105250 2024-03-21T11:30:37.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.344+0000 7f9d41000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d3c1028a0 0x7f9d3c19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.344+0000 7f9d41a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d3c0fff30 0x7f9d3c19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.344+0000 7f9d42400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d3c105250 0x7f9d3c1a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.344+0000 7f9d41000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d3c1028a0 0x7f9d3c19e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58804/0 (socket says 172.21.15.47:58804) 2024-03-21T11:30:37.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.344+0000 7f9d41000640 1 -- 172.21.15.47:0/2929136374 learned_addr learned my addr 172.21.15.47:0/2929136374 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.345+0000 7f9d42400640 1 -- 172.21.15.47:0/2929136374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d3c1028a0 msgr2=0x7f9d3c19e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.345+0000 7f9d42400640 1 --2- 172.21.15.47:0/2929136374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d3c1028a0 0x7f9d3c19e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.345+0000 7f9d42400640 1 -- 172.21.15.47:0/2929136374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d3c0fff30 msgr2=0x7f9d3c19dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.345+0000 7f9d42400640 1 --2- 172.21.15.47:0/2929136374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d3c0fff30 0x7f9d3c19dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.345+0000 7f9d42400640 1 -- 172.21.15.47:0/2929136374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c1a5be0 con 0x7f9d3c105250 2024-03-21T11:30:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.345+0000 7f9d41a00640 1 --2- 172.21.15.47:0/2929136374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d3c0fff30 0x7f9d3c19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.345+0000 7f9d42400640 1 --2- 172.21.15.47:0/2929136374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d3c105250 0x7f9d3c1a5570 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009fd0 tx=0x7f9d3400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.346+0000 7f9d3a000640 1 -- 172.21.15.47:0/2929136374 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3400ce20 con 0x7f9d3c105250 2024-03-21T11:30:37.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.346+0000 7f9d3a000640 1 -- 172.21.15.47:0/2929136374 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d34005020 con 0x7f9d3c105250 2024-03-21T11:30:37.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.346+0000 7f9d3a000640 1 -- 172.21.15.47:0/2929136374 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d34010720 con 0x7f9d3c105250 2024-03-21T11:30:37.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.346+0000 7f9d44400640 1 -- 172.21.15.47:0/2929136374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3c1a5ed0 con 0x7f9d3c105250 2024-03-21T11:30:37.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.346+0000 7f9d44400640 1 -- 172.21.15.47:0/2929136374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3c1a6340 con 0x7f9d3c105250 2024-03-21T11:30:37.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.348+0000 7f9d44400640 1 -- 172.21.15.47:0/2929136374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3c0fd620 con 0x7f9d3c105250 2024-03-21T11:30:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.350+0000 7f9d3a000640 1 -- 172.21.15.47:0/2929136374 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9d34005540 con 0x7f9d3c105250 2024-03-21T11:30:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.352+0000 7f9d3a000640 1 --2- 172.21.15.47:0/2929136374 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9d240a2fa0 0x7f9d240a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.352+0000 7f9d3a000640 1 -- 172.21.15.47:0/2929136374 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9d340cfcf0 con 0x7f9d3c105250 2024-03-21T11:30:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.352+0000 7f9d41a00640 1 --2- 172.21.15.47:0/2929136374 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9d240a2fa0 0x7f9d240a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.353+0000 7f9d41a00640 1 --2- 172.21.15.47:0/2929136374 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9d240a2fa0 0x7f9d240a5460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009fc0 tx=0x7f9d30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.358+0000 7f9d3a000640 1 -- 172.21.15.47:0/2929136374 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9d340d4070 con 0x7f9d3c105250 2024-03-21T11:30:37.529 INFO:tasks.workunit.client.0.smithi047.stdout: 1 501061 8.29 MB/sec execute 201 sec latency 15.908 ms 2024-03-21T11:30:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.630+0000 7f9d44400640 1 -- 172.21.15.47:0/2929136374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d3c02d070 con 0x7f9d3c105250 2024-03-21T11:30:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.631+0000 7f9d3a000640 1 -- 172.21.15.47:0/2929136374 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f9d3409c790 con 0x7f9d3c105250 2024-03-21T11:30:37.632 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:30:37.632 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:30:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:30:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.634+0000 7f9d44400640 1 -- 172.21.15.47:0/2929136374 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9d240a2fa0 msgr2=0x7f9d240a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.634+0000 7f9d44400640 1 --2- 172.21.15.47:0/2929136374 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9d240a2fa0 0x7f9d240a5460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009fc0 tx=0x7f9d30005f90 comp rx=0 tx=0).stop 2024-03-21T11:30:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.634+0000 7f9d44400640 1 -- 172.21.15.47:0/2929136374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d3c105250 msgr2=0x7f9d3c1a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.634+0000 7f9d44400640 1 --2- 172.21.15.47:0/2929136374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d3c105250 0x7f9d3c1a5570 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009fd0 tx=0x7f9d3400ef10 comp rx=0 tx=0).stop 2024-03-21T11:30:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.634+0000 7f9d44400640 1 -- 172.21.15.47:0/2929136374 shutdown_connections 2024-03-21T11:30:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.634+0000 7f9d44400640 1 --2- 172.21.15.47:0/2929136374 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9d240a2fa0 0x7f9d240a5460 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.634+0000 7f9d44400640 1 --2- 172.21.15.47:0/2929136374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d3c105250 0x7f9d3c1a5570 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.634+0000 7f9d44400640 1 --2- 172.21.15.47:0/2929136374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d3c1028a0 0x7f9d3c19e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.634+0000 7f9d44400640 1 --2- 172.21.15.47:0/2929136374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d3c0fff30 0x7f9d3c19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.634+0000 7f9d44400640 1 -- 172.21.15.47:0/2929136374 >> 172.21.15.47:0/2929136374 conn(0x7f9d3c0fbb30 msgr2=0x7f9d3c1008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:37.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.634+0000 7f9d44400640 1 -- 172.21.15.47:0/2929136374 shutdown_connections 2024-03-21T11:30:37.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.635+0000 7f9d44400640 1 -- 172.21.15.47:0/2929136374 wait complete. 2024-03-21T11:30:37.655 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:30:37.655 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:30:37.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.894+0000 7f4d13400640 1 -- 172.21.15.47:0/3815729570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d0c1028a0 msgr2=0x7f4d0c102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.894+0000 7f4d13400640 1 --2- 172.21.15.47:0/3815729570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d0c1028a0 0x7f4d0c102d20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009900 tx=0x7f4cf802bc40 comp rx=0 tx=0).stop 2024-03-21T11:30:37.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.894+0000 7f4d13400640 1 -- 172.21.15.47:0/3815729570 shutdown_connections 2024-03-21T11:30:37.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.894+0000 7f4d13400640 1 --2- 172.21.15.47:0/3815729570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d0c103260 0x7f4d0c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.894+0000 7f4d13400640 1 --2- 172.21.15.47:0/3815729570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d0c1028a0 0x7f4d0c102d20 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.894+0000 7f4d13400640 1 --2- 172.21.15.47:0/3815729570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d0c1016b0 0x7f4d0c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.894+0000 7f4d13400640 1 -- 172.21.15.47:0/3815729570 >> 172.21.15.47:0/3815729570 conn(0x7f4d0c0fcc40 msgr2=0x7f4d0c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:37.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.895+0000 7f4d13400640 1 -- 172.21.15.47:0/3815729570 shutdown_connections 2024-03-21T11:30:37.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.895+0000 7f4d13400640 1 -- 172.21.15.47:0/3815729570 wait complete. 2024-03-21T11:30:37.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.896+0000 7f4d13400640 1 Processor -- start 2024-03-21T11:30:37.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.896+0000 7f4d13400640 1 -- start start 2024-03-21T11:30:37.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.896+0000 7f4d13400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d0c1016b0 0x7f4d0c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.896+0000 7f4d13400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d0c1028a0 0x7f4d0c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.897+0000 7f4d13400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d0c103260 0x7f4d0c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.897+0000 7f4d13400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c10d380 con 0x7f4d0c1028a0 2024-03-21T11:30:37.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.897+0000 7f4d13400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c10d200 con 0x7f4d0c1016b0 2024-03-21T11:30:37.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.897+0000 7f4d13400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c10d500 con 0x7f4d0c103260 2024-03-21T11:30:37.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.897+0000 7f4d10a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d0c1016b0 0x7f4d0c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.897+0000 7f4d11400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d0c103260 0x7f4d0c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.897+0000 7f4d11400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d0c103260 0x7f4d0c19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34510/0 (socket says 172.21.15.47:34510) 2024-03-21T11:30:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.897+0000 7f4d11400640 1 -- 172.21.15.47:0/3897437493 learned_addr learned my addr 172.21.15.47:0/3897437493 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.897+0000 7f4d10a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d0c1016b0 0x7f4d0c1979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58826/0 (socket says 172.21.15.47:58826) 2024-03-21T11:30:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.897+0000 7f4d0be00640 1 --2- 172.21.15.47:0/3897437493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d0c1028a0 0x7f4d0c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.897+0000 7f4d10a00640 1 -- 172.21.15.47:0/3897437493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d0c103260 msgr2=0x7f4d0c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.897+0000 7f4d10a00640 1 --2- 172.21.15.47:0/3897437493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d0c103260 0x7f4d0c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.898+0000 7f4d10a00640 1 -- 172.21.15.47:0/3897437493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d0c1028a0 msgr2=0x7f4d0c197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.898+0000 7f4d10a00640 1 --2- 172.21.15.47:0/3897437493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d0c1028a0 0x7f4d0c197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.898+0000 7f4d10a00640 1 -- 172.21.15.47:0/3897437493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c0738d0 con 0x7f4d0c1016b0 2024-03-21T11:30:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.898+0000 7f4d11400640 1 --2- 172.21.15.47:0/3897437493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d0c103260 0x7f4d0c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.898+0000 7f4d10a00640 1 --2- 172.21.15.47:0/3897437493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d0c1016b0 0x7f4d0c1979c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000e9c0 tx=0x7f4d0000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.901+0000 7f4d09600640 1 -- 172.21.15.47:0/3897437493 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000ccc0 con 0x7f4d0c1016b0 2024-03-21T11:30:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.901+0000 7f4d09600640 1 -- 172.21.15.47:0/3897437493 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d000035f0 con 0x7f4d0c1016b0 2024-03-21T11:30:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.901+0000 7f4d09600640 1 -- 172.21.15.47:0/3897437493 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000ce60 con 0x7f4d0c1016b0 2024-03-21T11:30:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.901+0000 7f4d13400640 1 -- 172.21.15.47:0/3897437493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0c073bc0 con 0x7f4d0c1016b0 2024-03-21T11:30:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.901+0000 7f4d13400640 1 -- 172.21.15.47:0/3897437493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0c074090 con 0x7f4d0c1016b0 2024-03-21T11:30:37.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.904+0000 7f4d09600640 1 -- 172.21.15.47:0/3897437493 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4d000037a0 con 0x7f4d0c1016b0 2024-03-21T11:30:37.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.904+0000 7f4d15ff5740 1 Processor -- start 2024-03-21T11:30:37.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.904+0000 7f4d15ff5740 1 -- start start 2024-03-21T11:30:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.905+0000 7f4d15ff5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 0x564c796dcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.905+0000 7f4d15ff5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796dde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.905+0000 7f4d15ff5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 0x564c796e4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.905+0000 7f4d15ff5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c796412c0 con 0x564c796dc810 2024-03-21T11:30:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.905+0000 7f4d15ff5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c795e9c90 con 0x564c796de3a0 2024-03-21T11:30:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.905+0000 7f4d15ff5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7963d360 con 0x564c796dd9e0 2024-03-21T11:30:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.905+0000 7f4d10a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796dde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.905+0000 7f4d11400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 0x564c796dcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.905+0000 7f4d10a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796dde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34512/0 (socket says 172.21.15.47:34512) 2024-03-21T11:30:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.905+0000 7f4d10a00640 1 -- 172.21.15.47:0/698822935 learned_addr learned my addr 172.21.15.47:0/698822935 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.905+0000 7f4d0be00640 1 --2- 172.21.15.47:0/698822935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 0x564c796e4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.906+0000 7f4d10a00640 1 -- 172.21.15.47:0/698822935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 msgr2=0x564c796e4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.906+0000 7f4d10a00640 1 --2- 172.21.15.47:0/698822935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 0x564c796e4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.906+0000 7f4d10a00640 1 -- 172.21.15.47:0/698822935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 msgr2=0x564c796dcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.906+0000 7f4d10a00640 1 --2- 172.21.15.47:0/698822935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 0x564c796dcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.906+0000 7f4d10a00640 1 -- 172.21.15.47:0/698822935 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c7960e080 con 0x564c796dd9e0 2024-03-21T11:30:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.906+0000 7f4d10a00640 1 --2- 172.21.15.47:0/698822935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796dde60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4d000d3a40 tx=0x7f4d000d3a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce027cf352ac1cb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.906+0000 7f4d09600640 1 --2- 172.21.15.47:0/3897437493 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4cf40a2fa0 0x7f4cf40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.906+0000 7f4cfdc00640 1 -- 172.21.15.47:0/698822935 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d000e3070 con 0x564c796dd9e0 2024-03-21T11:30:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.906+0000 7f4cfdc00640 1 -- 172.21.15.47:0/698822935 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d00085370 con 0x564c796dd9e0 2024-03-21T11:30:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.906+0000 7f4cfdc00640 1 -- 172.21.15.47:0/698822935 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d000893a0 con 0x564c796dd9e0 2024-03-21T11:30:37.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.906+0000 7f4d09600640 1 -- 172.21.15.47:0/3897437493 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4d000cfaa0 con 0x7f4d0c1016b0 2024-03-21T11:30:37.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.906+0000 7f4d11400640 1 --2- 172.21.15.47:0/3897437493 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4cf40a2fa0 0x7f4cf40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.907+0000 7f4d15ff5740 1 -- 172.21.15.47:0/698822935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 msgr2=0x564c796dde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.907+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/698822935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796dde60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4d000d3a40 tx=0x7f4d000d3a70 comp rx=0 tx=0).stop 2024-03-21T11:30:37.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.907+0000 7f4d15ff5740 1 -- 172.21.15.47:0/698822935 shutdown_connections 2024-03-21T11:30:37.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.907+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/698822935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 0x564c796e4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.907+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/698822935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796dde60 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.907+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/698822935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 0x564c796dcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.907+0000 7f4d11400640 1 --2- 172.21.15.47:0/3897437493 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4cf40a2fa0 0x7f4cf40a5460 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x564c796dd8e0 tx=0x7f4d04009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:37.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.907+0000 7f4d15ff5740 1 -- 172.21.15.47:0/698822935 >> 172.21.15.47:0/698822935 conn(0x564c796d7c00 msgr2=0x564c796d8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:37.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.908+0000 7f4d15ff5740 1 -- 172.21.15.47:0/698822935 shutdown_connections 2024-03-21T11:30:37.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.908+0000 7f4d15ff5740 1 -- 172.21.15.47:0/698822935 wait complete. 2024-03-21T11:30:37.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.908+0000 7f4d15ff5740 1 -- start start 2024-03-21T11:30:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.908+0000 7f4d15ff5740 1 Processor -- start 2024-03-21T11:30:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d15ff5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796dc810 0x564c796e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d15ff5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dd9e0 0x564c796e6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d15ff5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796de3a0 0x564c796edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d15ff5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c796e53b0 con 0x564c796dd9e0 2024-03-21T11:30:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d15ff5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c795fd0d0 con 0x564c796dc810 2024-03-21T11:30:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d15ff5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c796e5530 con 0x564c796de3a0 2024-03-21T11:30:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d15ff5740 10 client.0 _ll_register_callbacks cb 0x564c79601e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d10a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796dc810 0x564c796e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d10a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796dc810 0x564c796e6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58832/0 (socket says 172.21.15.47:58832) 2024-03-21T11:30:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d0be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dd9e0 0x564c796e6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d10a00640 1 -- 172.21.15.47:0/3986540783 learned_addr learned my addr 172.21.15.47:0/3986540783 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:37.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d11400640 1 --2- 172.21.15.47:0/3986540783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796de3a0 0x564c796edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d10a00640 1 -- 172.21.15.47:0/3986540783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796de3a0 msgr2=0x564c796edfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d10a00640 1 --2- 172.21.15.47:0/3986540783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796de3a0 0x564c796edfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d10a00640 1 -- 172.21.15.47:0/3986540783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dd9e0 msgr2=0x564c796e6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d10a00640 1 --2- 172.21.15.47:0/3986540783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dd9e0 0x564c796e6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.909+0000 7f4d10a00640 1 -- 172.21.15.47:0/3986540783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c793553f0 con 0x564c796dc810 2024-03-21T11:30:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.910+0000 7f4d10a00640 1 --2- 172.21.15.47:0/3986540783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796dc810 0x564c796e6a20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4d00009770 tx=0x7f4d00005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e01878ed4cfac5b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.910+0000 7f4cf2000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.910+0000 7f4cf2000640 1 -- 172.21.15.47:0/3986540783 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00085040 con 0x564c796dc810 2024-03-21T11:30:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.910+0000 7f4cf2000640 1 -- 172.21.15.47:0/3986540783 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4d000cd2c0 con 0x564c796dc810 2024-03-21T11:30:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.910+0000 7f4d15ff5740 1 -- 172.21.15.47:0/3986540783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d00085370 con 0x564c796dc810 2024-03-21T11:30:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.910+0000 7f4cf2000640 1 -- 172.21.15.47:0/3986540783 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d000891a0 con 0x564c796dc810 2024-03-21T11:30:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.910+0000 7f4d15ff5740 10 client.27902 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:37.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.910+0000 7f4d15ff5740 1 -- 172.21.15.47:0/3986540783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564c7941f7d0 con 0x564c796dc810 2024-03-21T11:30:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.911+0000 7f4cf2000640 1 -- 172.21.15.47:0/3986540783 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d00085370 con 0x564c796dc810 2024-03-21T11:30:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.911+0000 7f4cf2000640 1 -- 172.21.15.47:0/3986540783 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4d00098450 con 0x564c796dc810 2024-03-21T11:30:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.911+0000 7f4d15ff5740 10 client.27902 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.911+0000 7f4d15ff5740 10 client.27902 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:30:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.911+0000 7f4d15ff5740 20 client.27902 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.911+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/3986540783 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564c796f2100 0x564c796f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.911+0000 7f4d15ff5740 4 client.27902 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.911+0000 7f4d15ff5740 1 -- 172.21.15.47:0/3986540783 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c7964e800 con 0x564c796f2100 2024-03-21T11:30:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.911+0000 7f4d0be00640 1 --2- 172.21.15.47:0/3986540783 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564c796f2100 0x564c796f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.912+0000 7f4d0be00640 1 --2- 172.21.15.47:0/3986540783 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564c796f2100 0x564c796f44f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e8037ebb5ae0132d server_cookie=384e4a1dd68f83ba in_seq=0 out_seq=0 2024-03-21T11:30:37.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.912+0000 7f4cf2000640 10 client.27902 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:30:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.912+0000 7f4cf2000640 1 -- 172.21.15.47:0/3986540783 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564c7964e800 con 0x564c796f2100 2024-03-21T11:30:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.912+0000 7f4cf2000640 10 client.27902 handle_command_reply: tid=0 2024-03-21T11:30:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.912+0000 7f4d15ff5740 1 client.27902 shutdown 2024-03-21T11:30:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.913+0000 7f4d15ff5740 1 -- 172.21.15.47:0/3986540783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796dc810 msgr2=0x564c796e6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.913+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/3986540783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796dc810 0x564c796e6a20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4d00009770 tx=0x7f4d00005b20 comp rx=0 tx=0).stop 2024-03-21T11:30:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.913+0000 7f4d15ff5740 1 -- 172.21.15.47:0/3986540783 shutdown_connections 2024-03-21T11:30:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.913+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/3986540783 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564c796f2100 0x564c796f44f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.913+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/3986540783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796de3a0 0x564c796edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.913+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/3986540783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dd9e0 0x564c796e6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.913+0000 7f4cf2000640 0 client.27902 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:30:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.913+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/3986540783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796dc810 0x564c796e6a20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.913+0000 7f4d15ff5740 1 -- 172.21.15.47:0/3986540783 >> 172.21.15.47:0/3986540783 conn(0x564c796d7c00 msgr2=0x564c796d6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.913+0000 7f4d15ff5740 1 -- 172.21.15.47:0/3986540783 shutdown_connections 2024-03-21T11:30:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.913+0000 7f4d15ff5740 1 -- 172.21.15.47:0/3986540783 wait complete. 2024-03-21T11:30:37.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.913+0000 7f4d15ff5740 20 client.27902 trim_cache size 0 max 16384 2024-03-21T11:30:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.929+0000 7f4d15ff5740 1 Processor -- start 2024-03-21T11:30:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.929+0000 7f4d15ff5740 1 -- start start 2024-03-21T11:30:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.929+0000 7f4d15ff5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 0x564c796e6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.929+0000 7f4d15ff5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796edf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.929+0000 7f4d15ff5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 0x564c796db570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.929+0000 7f4d15ff5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c1ac350 con 0x564c796dc810 2024-03-21T11:30:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.929+0000 7f4d15ff5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c1adcd0 con 0x564c796de3a0 2024-03-21T11:30:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.929+0000 7f4d15ff5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c1ac9b0 con 0x564c796dd9e0 2024-03-21T11:30:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.929+0000 7f4d0be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.929+0000 7f4d11400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 0x564c796db570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.929+0000 7f4d10a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 0x564c796e6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.929+0000 7f4d0be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796edf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34524/0 (socket says 172.21.15.47:34524) 2024-03-21T11:30:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.929+0000 7f4d0be00640 1 -- 172.21.15.47:0/1107111735 learned_addr learned my addr 172.21.15.47:0/1107111735 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.930+0000 7f4d0be00640 1 -- 172.21.15.47:0/1107111735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 msgr2=0x564c796db570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.930+0000 7f4d0be00640 1 --2- 172.21.15.47:0/1107111735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 0x564c796db570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.930+0000 7f4d0be00640 1 -- 172.21.15.47:0/1107111735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 msgr2=0x564c796e6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.930+0000 7f4d0be00640 1 --2- 172.21.15.47:0/1107111735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 0x564c796e6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.930+0000 7f4d0be00640 1 -- 172.21.15.47:0/1107111735 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c796d2d40 con 0x564c796dd9e0 2024-03-21T11:30:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.930+0000 7f4d0be00640 1 --2- 172.21.15.47:0/1107111735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796edf60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4cf802c8f0 tx=0x7f4cf802cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5fb23d19df05d82 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.930+0000 7f4cfd200640 1 -- 172.21.15.47:0/1107111735 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8050070 con 0x564c796dd9e0 2024-03-21T11:30:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.930+0000 7f4cfd200640 1 -- 172.21.15.47:0/1107111735 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4cf804a110 con 0x564c796dd9e0 2024-03-21T11:30:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.930+0000 7f4cfd200640 1 -- 172.21.15.47:0/1107111735 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf804a6f0 con 0x564c796dd9e0 2024-03-21T11:30:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.931+0000 7f4d15ff5740 1 -- 172.21.15.47:0/1107111735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 msgr2=0x564c796edf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.931+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/1107111735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796edf60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4cf802c8f0 tx=0x7f4cf802cf90 comp rx=0 tx=0).stop 2024-03-21T11:30:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.931+0000 7f4d15ff5740 1 -- 172.21.15.47:0/1107111735 shutdown_connections 2024-03-21T11:30:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.931+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/1107111735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 0x564c796db570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.931+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/1107111735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796edf60 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.931+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/1107111735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 0x564c796e6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.931+0000 7f4d15ff5740 1 -- 172.21.15.47:0/1107111735 >> 172.21.15.47:0/1107111735 conn(0x564c796d7c00 msgr2=0x564c796dae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.931+0000 7f4d15ff5740 1 -- 172.21.15.47:0/1107111735 shutdown_connections 2024-03-21T11:30:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.931+0000 7f4d15ff5740 1 -- 172.21.15.47:0/1107111735 wait complete. 2024-03-21T11:30:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.932+0000 7f4d15ff5740 1 -- start start 2024-03-21T11:30:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.932+0000 7f4d15ff5740 1 Processor -- start 2024-03-21T11:30:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.932+0000 7f4d15ff5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 0x564c796e9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.932+0000 7f4d15ff5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796ea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.932+0000 7f4d15ff5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 0x564c796ed450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.932+0000 7f4d15ff5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c1ad670 con 0x564c796dc810 2024-03-21T11:30:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.932+0000 7f4d15ff5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c1ad010 con 0x564c796de3a0 2024-03-21T11:30:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.932+0000 7f4d15ff5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c1aa510 con 0x564c796dd9e0 2024-03-21T11:30:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.932+0000 7f4d15ff5740 10 client.0 _ll_register_callbacks cb 0x7f4d000e1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.933+0000 7f4d0be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796ea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.933+0000 7f4d0be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796ea430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34534/0 (socket says 172.21.15.47:34534) 2024-03-21T11:30:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.933+0000 7f4d0be00640 1 -- 172.21.15.47:0/971845094 learned_addr learned my addr 172.21.15.47:0/971845094 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.933+0000 7f4d11400640 1 --2- 172.21.15.47:0/971845094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 0x564c796ed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.933+0000 7f4d10a00640 1 --2- 172.21.15.47:0/971845094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 0x564c796e9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.933+0000 7f4d0be00640 1 -- 172.21.15.47:0/971845094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 msgr2=0x564c796ed450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.933+0000 7f4d0be00640 1 --2- 172.21.15.47:0/971845094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 0x564c796ed450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.933+0000 7f4d0be00640 1 -- 172.21.15.47:0/971845094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 msgr2=0x564c796e9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.933+0000 7f4d0be00640 1 --2- 172.21.15.47:0/971845094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 0x564c796e9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.933+0000 7f4d0be00640 1 -- 172.21.15.47:0/971845094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c796eda40 con 0x564c796dd9e0 2024-03-21T11:30:37.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.933+0000 7f4d0be00640 1 --2- 172.21.15.47:0/971845094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796ea430 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009230 tx=0x7f4cf804a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93c000e4f5f7d44b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.933+0000 7f4cf2000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.934+0000 7f4cf2000640 1 -- 172.21.15.47:0/971845094 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8050070 con 0x564c796dd9e0 2024-03-21T11:30:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.934+0000 7f4d15ff5740 1 -- 172.21.15.47:0/971845094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cf804a110 con 0x564c796dd9e0 2024-03-21T11:30:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.934+0000 7f4cf2000640 1 -- 172.21.15.47:0/971845094 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4cf804a6f0 con 0x564c796dd9e0 2024-03-21T11:30:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.934+0000 7f4cf2000640 1 -- 172.21.15.47:0/971845094 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cf805e050 con 0x564c796dd9e0 2024-03-21T11:30:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.934+0000 7f4d15ff5740 10 client.27925 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.934+0000 7f4d15ff5740 1 -- 172.21.15.47:0/971845094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564c7974df60 con 0x564c796dd9e0 2024-03-21T11:30:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.935+0000 7f4cf2000640 1 -- 172.21.15.47:0/971845094 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4cf804a110 con 0x564c796dd9e0 2024-03-21T11:30:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.935+0000 7f4cf2000640 1 -- 172.21.15.47:0/971845094 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4cf8047070 con 0x564c796dd9e0 2024-03-21T11:30:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.935+0000 7f4d15ff5740 10 client.27925 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.935+0000 7f4d15ff5740 10 client.27925 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:30:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.935+0000 7f4d15ff5740 20 client.27925 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.935+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/971845094 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564c7974e7b0 0x564c79750c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.935+0000 7f4d15ff5740 4 client.27925 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:30:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.935+0000 7f4d15ff5740 1 -- 172.21.15.47:0/971845094 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c79648b50 con 0x564c7974e7b0 2024-03-21T11:30:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.935+0000 7f4d10a00640 1 --2- 172.21.15.47:0/971845094 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564c7974e7b0 0x564c79750c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4d10a00640 1 --2- 172.21.15.47:0/971845094 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564c7974e7b0 0x564c79750c10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=211030c557a38d28 server_cookie=dade2bce74a6c62f in_seq=0 out_seq=0 2024-03-21T11:30:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4cf2000640 10 client.27925 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:30:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4cf2000640 1 -- 172.21.15.47:0/971845094 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c79648b50 con 0x564c7974e7b0 2024-03-21T11:30:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4cf2000640 10 client.27925 handle_command_reply: tid=0 2024-03-21T11:30:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4d15ff5740 1 client.27925 shutdown 2024-03-21T11:30:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4d15ff5740 1 -- 172.21.15.47:0/971845094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 msgr2=0x564c796ea430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/971845094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796ea430 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009230 tx=0x7f4cf804a2c0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4d15ff5740 1 -- 172.21.15.47:0/971845094 shutdown_connections 2024-03-21T11:30:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/971845094 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564c7974e7b0 0x564c79750c10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/971845094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c796de3a0 0x564c796ed450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/971845094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c796dd9e0 0x564c796ea430 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4cf2000640 0 client.27925 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:30:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4d15ff5740 1 --2- 172.21.15.47:0/971845094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c796dc810 0x564c796e9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4d15ff5740 1 -- 172.21.15.47:0/971845094 >> 172.21.15.47:0/971845094 conn(0x564c796d7c00 msgr2=0x564c796e6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4d15ff5740 1 -- 172.21.15.47:0/971845094 shutdown_connections 2024-03-21T11:30:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.936+0000 7f4d15ff5740 1 -- 172.21.15.47:0/971845094 wait complete. 2024-03-21T11:30:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.937+0000 7f4d15ff5740 20 client.27925 trim_cache size 0 max 16384 2024-03-21T11:30:37.941 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:30:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.937+0000 7f4d13400640 1 -- 172.21.15.47:0/3897437493 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4cf40a2fa0 msgr2=0x7f4cf40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.937+0000 7f4d13400640 1 --2- 172.21.15.47:0/3897437493 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4cf40a2fa0 0x7f4cf40a5460 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x564c796dd8e0 tx=0x7f4d04009500 comp rx=0 tx=0).stop 2024-03-21T11:30:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.937+0000 7f4d13400640 1 -- 172.21.15.47:0/3897437493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d0c1016b0 msgr2=0x7f4d0c1979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.937+0000 7f4d13400640 1 --2- 172.21.15.47:0/3897437493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d0c1016b0 0x7f4d0c1979c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000e9c0 tx=0x7f4d0000ee80 comp rx=0 tx=0).stop 2024-03-21T11:30:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.937+0000 7f4d13400640 1 -- 172.21.15.47:0/3897437493 shutdown_connections 2024-03-21T11:30:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.937+0000 7f4d13400640 1 --2- 172.21.15.47:0/3897437493 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4cf40a2fa0 0x7f4cf40a5460 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.937+0000 7f4d13400640 1 --2- 172.21.15.47:0/3897437493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d0c103260 0x7f4d0c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.937+0000 7f4d13400640 1 --2- 172.21.15.47:0/3897437493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d0c1028a0 0x7f4d0c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.937+0000 7f4d13400640 1 --2- 172.21.15.47:0/3897437493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d0c1016b0 0x7f4d0c1979c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.937+0000 7f4d13400640 1 -- 172.21.15.47:0/3897437493 >> 172.21.15.47:0/3897437493 conn(0x7f4d0c0fcc40 msgr2=0x7f4d0c105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.938+0000 7f4d13400640 1 -- 172.21.15.47:0/3897437493 shutdown_connections 2024-03-21T11:30:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:37.938+0000 7f4d13400640 1 -- 172.21.15.47:0/3897437493 wait complete. 2024-03-21T11:30:37.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:37 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2929136374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:30:37.956 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:30:37.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:30:38.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.170+0000 7f53b7000640 1 -- 172.21.15.47:0/2972698636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53b01037c0 msgr2=0x7f53b0103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.170+0000 7f53b7000640 1 --2- 172.21.15.47:0/2972698636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53b01037c0 0x7f53b0103be0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009860 tx=0x7f53a402bb90 comp rx=0 tx=0).stop 2024-03-21T11:30:38.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.170+0000 7f53b7000640 1 -- 172.21.15.47:0/2972698636 shutdown_connections 2024-03-21T11:30:38.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.170+0000 7f53b7000640 1 --2- 172.21.15.47:0/2972698636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53b0105370 0x7f53b010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.170+0000 7f53b7000640 1 --2- 172.21.15.47:0/2972698636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53b01049b0 0x7f53b0104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.170+0000 7f53b7000640 1 --2- 172.21.15.47:0/2972698636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53b01037c0 0x7f53b0103be0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.170+0000 7f53b7000640 1 -- 172.21.15.47:0/2972698636 >> 172.21.15.47:0/2972698636 conn(0x7f53b00fed50 msgr2=0x7f53b0101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.170+0000 7f53b7000640 1 -- 172.21.15.47:0/2972698636 shutdown_connections 2024-03-21T11:30:38.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.170+0000 7f53b7000640 1 -- 172.21.15.47:0/2972698636 wait complete. 2024-03-21T11:30:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.171+0000 7f53b7000640 1 Processor -- start 2024-03-21T11:30:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.171+0000 7f53b7000640 1 -- start start 2024-03-21T11:30:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.171+0000 7f53b7000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53b01037c0 0x7f53b0199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.171+0000 7f53b7000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53b01049b0 0x7f53b0199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.171+0000 7f53b7000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53b0105370 0x7f53b01a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.171+0000 7f53b7000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b010f410 con 0x7f53b01037c0 2024-03-21T11:30:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.171+0000 7f53b7000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b010f290 con 0x7f53b0105370 2024-03-21T11:30:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.171+0000 7f53b7000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b010f590 con 0x7f53b01049b0 2024-03-21T11:30:38.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.172+0000 7f53b5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53b01049b0 0x7f53b0199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.172+0000 7f53b5c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53b01037c0 0x7f53b0199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.172+0000 7f53b6600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53b0105370 0x7f53b01a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.172+0000 7f53b6600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53b0105370 0x7f53b01a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58870/0 (socket says 172.21.15.47:58870) 2024-03-21T11:30:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.172+0000 7f53b5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53b01049b0 0x7f53b0199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34558/0 (socket says 172.21.15.47:34558) 2024-03-21T11:30:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.172+0000 7f53b6600640 1 -- 172.21.15.47:0/4228685324 learned_addr learned my addr 172.21.15.47:0/4228685324 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.172+0000 7f53b6600640 1 -- 172.21.15.47:0/4228685324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53b01049b0 msgr2=0x7f53b0199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.172+0000 7f53b6600640 1 --2- 172.21.15.47:0/4228685324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53b01049b0 0x7f53b0199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.172+0000 7f53b6600640 1 -- 172.21.15.47:0/4228685324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53b01037c0 msgr2=0x7f53b0199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.172+0000 7f53b6600640 1 --2- 172.21.15.47:0/4228685324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53b01037c0 0x7f53b0199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.172+0000 7f53b6600640 1 -- 172.21.15.47:0/4228685324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b01a16b0 con 0x7f53b0105370 2024-03-21T11:30:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.172+0000 7f53b5200640 1 --2- 172.21.15.47:0/4228685324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53b01049b0 0x7f53b0199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.173+0000 7f53b6600640 1 --2- 172.21.15.47:0/4228685324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53b0105370 0x7f53b01a1040 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f53a800eb10 tx=0x7f53a800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.173+0000 7f53ae000640 1 -- 172.21.15.47:0/4228685324 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8004b20 con 0x7f53b0105370 2024-03-21T11:30:38.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.173+0000 7f53ae000640 1 -- 172.21.15.47:0/4228685324 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53a8004cc0 con 0x7f53b0105370 2024-03-21T11:30:38.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.173+0000 7f53ae000640 1 -- 172.21.15.47:0/4228685324 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8010760 con 0x7f53b0105370 2024-03-21T11:30:38.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.173+0000 7f53b7000640 1 -- 172.21.15.47:0/4228685324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b01a19a0 con 0x7f53b0105370 2024-03-21T11:30:38.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.173+0000 7f53b7000640 1 -- 172.21.15.47:0/4228685324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b01a1ea0 con 0x7f53b0105370 2024-03-21T11:30:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.178+0000 7f53ae000640 1 -- 172.21.15.47:0/4228685324 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f53a8005020 con 0x7f53b0105370 2024-03-21T11:30:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.178+0000 7f53bb3aa740 1 Processor -- start 2024-03-21T11:30:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.178+0000 7f53bb3aa740 1 -- start start 2024-03-21T11:30:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.178+0000 7f53bb3aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.178+0000 7f53bb3aa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f6bc0 0x561f1d2f7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.178+0000 7f53bb3aa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f7580 0x561f1d2fde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.178+0000 7f53bb3aa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1d25a2c0 con 0x561f1d2f6bc0 2024-03-21T11:30:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.178+0000 7f53bb3aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1d202c90 con 0x561f1d2f59f0 2024-03-21T11:30:38.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.178+0000 7f53bb3aa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1d256360 con 0x561f1d2f7580 2024-03-21T11:30:38.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.179+0000 7f53b5c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f6bc0 0x561f1d2f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.178+0000 7f53b5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.179+0000 7f53b5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58880/0 (socket says 172.21.15.47:58880) 2024-03-21T11:30:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.179+0000 7f53b5c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f6bc0 0x561f1d2f7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51064/0 (socket says 172.21.15.47:51064) 2024-03-21T11:30:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.179+0000 7f53b5200640 1 -- 172.21.15.47:0/2700989305 learned_addr learned my addr 172.21.15.47:0/2700989305 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.179+0000 7f53b5200640 1 --2- 172.21.15.47:0/2700989305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f7580 0x561f1d2fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.179+0000 7f53b5200640 1 -- 172.21.15.47:0/2700989305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f7580 msgr2=0x561f1d2fde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.179+0000 7f53b5200640 1 --2- 172.21.15.47:0/2700989305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f7580 0x561f1d2fde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.179+0000 7f53b5200640 1 -- 172.21.15.47:0/2700989305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f6bc0 msgr2=0x561f1d2f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.179+0000 7f53b5200640 1 --2- 172.21.15.47:0/2700989305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f6bc0 0x561f1d2f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.179+0000 7f53b5200640 1 -- 172.21.15.47:0/2700989305 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f1d227080 con 0x561f1d2f59f0 2024-03-21T11:30:38.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.179+0000 7f53b5c00640 1 --2- 172.21.15.47:0/2700989305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f6bc0 0x561f1d2f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:38.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.179+0000 7f53ae000640 1 --2- 172.21.15.47:0/4228685324 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53980a2fa0 0x7f53980a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.180+0000 7f53b5200640 1 --2- 172.21.15.47:0/2700989305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2f5e10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f539c011bc0 tx=0x7f539c0095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62ca281e00f8a040 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.180+0000 7f53ae000640 1 -- 172.21.15.47:0/4228685324 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f53a80cfc60 con 0x7f53b0105370 2024-03-21T11:30:38.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.180+0000 7f53b5c00640 1 --2- 172.21.15.47:0/4228685324 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53980a2fa0 0x7f53980a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.180+0000 7f53a2a00640 1 -- 172.21.15.47:0/2700989305 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c00fc50 con 0x561f1d2f59f0 2024-03-21T11:30:38.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.180+0000 7f53a2a00640 1 -- 172.21.15.47:0/2700989305 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f539c00fdf0 con 0x561f1d2f59f0 2024-03-21T11:30:38.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.180+0000 7f53a2a00640 1 -- 172.21.15.47:0/2700989305 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c004a00 con 0x561f1d2f59f0 2024-03-21T11:30:38.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.181+0000 7f53b5c00640 1 --2- 172.21.15.47:0/4228685324 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53980a2fa0 0x7f53980a5460 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f53a40349e0 tx=0x7f53a4034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.181+0000 7f53bb3aa740 1 -- 172.21.15.47:0/2700989305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 msgr2=0x561f1d2f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.181+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/2700989305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2f5e10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f539c011bc0 tx=0x7f539c0095f0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.181+0000 7f53bb3aa740 1 -- 172.21.15.47:0/2700989305 shutdown_connections 2024-03-21T11:30:38.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.181+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/2700989305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f7580 0x561f1d2fde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.181+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/2700989305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f6bc0 0x561f1d2f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.181+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/2700989305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2f5e10 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.181+0000 7f53bb3aa740 1 -- 172.21.15.47:0/2700989305 >> 172.21.15.47:0/2700989305 conn(0x561f1d2f0de0 msgr2=0x561f1d2f11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.181+0000 7f53bb3aa740 1 -- 172.21.15.47:0/2700989305 shutdown_connections 2024-03-21T11:30:38.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.181+0000 7f53bb3aa740 1 -- 172.21.15.47:0/2700989305 wait complete. 2024-03-21T11:30:38.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.182+0000 7f53bb3aa740 1 -- start start 2024-03-21T11:30:38.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.182+0000 7f53bb3aa740 1 Processor -- start 2024-03-21T11:30:38.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.183+0000 7f53bb3aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.183+0000 7f53bb3aa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f6bc0 0x561f1d300210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.183+0000 7f53bb3aa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f7580 0x561f1d307250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.183+0000 7f53bb3aa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1d2fe590 con 0x561f1d2f6bc0 2024-03-21T11:30:38.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.183+0000 7f53bb3aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1d2160d0 con 0x561f1d2f59f0 2024-03-21T11:30:38.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.183+0000 7f53bb3aa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1d2fe710 con 0x561f1d2f7580 2024-03-21T11:30:38.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.183+0000 7f53bb3aa740 10 client.0 _ll_register_callbacks cb 0x561f1d21ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:38.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.184+0000 7f53b6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f7580 0x561f1d307250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.184+0000 7f53b5200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f6bc0 0x561f1d300210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.184+0000 7f53b6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f7580 0x561f1d307250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34568/0 (socket says 172.21.15.47:34568) 2024-03-21T11:30:38.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.184+0000 7f53b6600640 1 -- 172.21.15.47:0/1516903490 learned_addr learned my addr 172.21.15.47:0/1516903490 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.184+0000 7f53b5c00640 1 --2- 172.21.15.47:0/1516903490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.184+0000 7f53b6600640 1 -- 172.21.15.47:0/1516903490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 msgr2=0x561f1d2ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.184+0000 7f53b6600640 1 --2- 172.21.15.47:0/1516903490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.185+0000 7f53b6600640 1 -- 172.21.15.47:0/1516903490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f6bc0 msgr2=0x561f1d300210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.185+0000 7f53b6600640 1 --2- 172.21.15.47:0/1516903490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f6bc0 0x561f1d300210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.185+0000 7f53b6600640 1 -- 172.21.15.47:0/1516903490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f1cf6e3f0 con 0x561f1d2f7580 2024-03-21T11:30:38.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.185+0000 7f53b5c00640 1 --2- 172.21.15.47:0/1516903490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:38.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.185+0000 7f53b5200640 1 --2- 172.21.15.47:0/1516903490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f6bc0 0x561f1d300210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:38.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.185+0000 7f53b6600640 1 --2- 172.21.15.47:0/1516903490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f7580 0x561f1d307250 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f53a8098460 tx=0x7f53a809ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd7d7f5dfe79ce9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.185+0000 7f5397400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:38.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.186+0000 7f5397400640 1 -- 172.21.15.47:0/1516903490 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a80a0150 con 0x561f1d2f7580 2024-03-21T11:30:38.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.186+0000 7f5397400640 1 -- 172.21.15.47:0/1516903490 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f53a80a4580 con 0x561f1d2f7580 2024-03-21T11:30:38.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.186+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1516903490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f539c00fdf0 con 0x561f1d2f7580 2024-03-21T11:30:38.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.186+0000 7f5397400640 1 -- 172.21.15.47:0/1516903490 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53a80a4a10 con 0x561f1d2f7580 2024-03-21T11:30:38.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.186+0000 7f53bb3aa740 10 client.27937 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:38.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.186+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1516903490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561f1d0387d0 con 0x561f1d2f7580 2024-03-21T11:30:38.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.186+0000 7f5397400640 1 -- 172.21.15.47:0/1516903490 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f539c00fdf0 con 0x561f1d2f7580 2024-03-21T11:30:38.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.187+0000 7f5397400640 1 -- 172.21.15.47:0/1516903490 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f53a8005310 con 0x561f1d2f7580 2024-03-21T11:30:38.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.187+0000 7f53bb3aa740 10 client.27937 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:38.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.187+0000 7f53bb3aa740 10 client.27937 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:30:38.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.187+0000 7f53bb3aa740 20 client.27937 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:38.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.187+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/1516903490 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561f1d30b7c0 0x561f1d30dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.187+0000 7f53bb3aa740 4 client.27937 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:38.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.187+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1516903490 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f1d267800 con 0x561f1d30b7c0 2024-03-21T11:30:38.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.187+0000 7f53b5c00640 1 --2- 172.21.15.47:0/1516903490 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561f1d30b7c0 0x561f1d30dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.188+0000 7f53b5c00640 1 --2- 172.21.15.47:0/1516903490 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561f1d30b7c0 0x561f1d30dbd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ef6126c7e8f8c55 server_cookie=c698f9752fe25705 in_seq=0 out_seq=0 2024-03-21T11:30:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.188+0000 7f5397400640 10 client.27937 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:30:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.189+0000 7f5397400640 1 -- 172.21.15.47:0/1516903490 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561f1d267800 con 0x561f1d30b7c0 2024-03-21T11:30:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.189+0000 7f5397400640 10 client.27937 handle_command_reply: tid=0 2024-03-21T11:30:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.189+0000 7f53bb3aa740 1 client.27937 shutdown 2024-03-21T11:30:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.189+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1516903490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f7580 msgr2=0x561f1d307250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.189+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/1516903490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f7580 0x561f1d307250 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f53a8098460 tx=0x7f53a809ba20 comp rx=0 tx=0).stop 2024-03-21T11:30:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.189+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1516903490 shutdown_connections 2024-03-21T11:30:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.189+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/1516903490 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561f1d30b7c0 0x561f1d30dbd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.189+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/1516903490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f7580 0x561f1d307250 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.189+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/1516903490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f6bc0 0x561f1d300210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.189+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/1516903490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.189+0000 7f5397400640 0 client.27937 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:30:38.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.189+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1516903490 >> 172.21.15.47:0/1516903490 conn(0x561f1d2f0de0 msgr2=0x561f1d2f0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.189+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1516903490 shutdown_connections 2024-03-21T11:30:38.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.190+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1516903490 wait complete. 2024-03-21T11:30:38.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.190+0000 7f53bb3aa740 20 client.27937 trim_cache size 0 max 16384 2024-03-21T11:30:38.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.206+0000 7f53bb3aa740 1 Processor -- start 2024-03-21T11:30:38.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.207+0000 7f53bb3aa740 1 -- start start 2024-03-21T11:30:38.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.207+0000 7f53bb3aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.207+0000 7f53bb3aa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f6bc0 0x561f1d307210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.207+0000 7f53bb3aa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f7580 0x561f1d2efe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.207+0000 7f53bb3aa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b01b0520 con 0x561f1d2f7580 2024-03-21T11:30:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.207+0000 7f53bb3aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b01b1ea0 con 0x561f1d2f59f0 2024-03-21T11:30:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.207+0000 7f53bb3aa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b01b0b80 con 0x561f1d2f6bc0 2024-03-21T11:30:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.207+0000 7f53b5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.207+0000 7f53b5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2ef8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58894/0 (socket says 172.21.15.47:58894) 2024-03-21T11:30:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.207+0000 7f53b5c00640 1 -- 172.21.15.47:0/3793719767 learned_addr learned my addr 172.21.15.47:0/3793719767 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.207+0000 7f53b6600640 1 --2- 172.21.15.47:0/3793719767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f7580 0x561f1d2efe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.207+0000 7f53b5200640 1 --2- 172.21.15.47:0/3793719767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f6bc0 0x561f1d307210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.208+0000 7f53b5c00640 1 -- 172.21.15.47:0/3793719767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f6bc0 msgr2=0x561f1d307210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.208+0000 7f53b5c00640 1 --2- 172.21.15.47:0/3793719767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f6bc0 0x561f1d307210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.208+0000 7f53b5c00640 1 -- 172.21.15.47:0/3793719767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f7580 msgr2=0x561f1d2efe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.208+0000 7f53b5c00640 1 --2- 172.21.15.47:0/3793719767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f7580 0x561f1d2efe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.208+0000 7f53b5c00640 1 -- 172.21.15.47:0/3793719767 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f1d2f1830 con 0x561f1d2f59f0 2024-03-21T11:30:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.208+0000 7f53b6600640 1 --2- 172.21.15.47:0/3793719767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f7580 0x561f1d2efe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.208+0000 7f53b5c00640 1 --2- 172.21.15.47:0/3793719767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2ef8d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009860 tx=0x7f53a403f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a0fe6b2ebbfa5e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.208+0000 7f53a2000640 1 -- 172.21.15.47:0/3793719767 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4057070 con 0x561f1d2f59f0 2024-03-21T11:30:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.208+0000 7f53a2000640 1 -- 172.21.15.47:0/3793719767 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53a4051e50 con 0x561f1d2f59f0 2024-03-21T11:30:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.208+0000 7f53a2000640 1 -- 172.21.15.47:0/3793719767 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4039040 con 0x561f1d2f59f0 2024-03-21T11:30:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.209+0000 7f53bb3aa740 1 -- 172.21.15.47:0/3793719767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 msgr2=0x561f1d2ef8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.209+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/3793719767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2ef8d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009860 tx=0x7f53a403f000 comp rx=0 tx=0).stop 2024-03-21T11:30:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.209+0000 7f53bb3aa740 1 -- 172.21.15.47:0/3793719767 shutdown_connections 2024-03-21T11:30:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.209+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/3793719767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f7580 0x561f1d2efe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.209+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/3793719767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f6bc0 0x561f1d307210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.209+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/3793719767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d2ef8d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.209+0000 7f53bb3aa740 1 -- 172.21.15.47:0/3793719767 >> 172.21.15.47:0/3793719767 conn(0x561f1d2f0de0 msgr2=0x561f1d2ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.209+0000 7f53bb3aa740 1 -- 172.21.15.47:0/3793719767 shutdown_connections 2024-03-21T11:30:38.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.209+0000 7f53bb3aa740 1 -- 172.21.15.47:0/3793719767 wait complete. 2024-03-21T11:30:38.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.210+0000 7f53bb3aa740 1 -- start start 2024-03-21T11:30:38.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.210+0000 7f53bb3aa740 1 Processor -- start 2024-03-21T11:30:38.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.210+0000 7f53bb3aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d302e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.210+0000 7f53bb3aa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f6bc0 0x561f1d303390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.210+0000 7f53bb3aa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f7580 0x561f1d3063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.210+0000 7f53bb3aa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b01b1840 con 0x561f1d2f7580 2024-03-21T11:30:38.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.210+0000 7f53bb3aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b01b11e0 con 0x561f1d2f59f0 2024-03-21T11:30:38.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.210+0000 7f53bb3aa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b01ae700 con 0x561f1d2f6bc0 2024-03-21T11:30:38.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f53bb3aa740 10 client.0 _ll_register_callbacks cb 0x7f53a80057d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:38.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f53b5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f6bc0 0x561f1d303390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f53b5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d302e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f53b5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d302e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58904/0 (socket says 172.21.15.47:58904) 2024-03-21T11:30:38.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f53b5c00640 1 -- 172.21.15.47:0/1277506709 learned_addr learned my addr 172.21.15.47:0/1277506709 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f53b5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f6bc0 0x561f1d303390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34584/0 (socket says 172.21.15.47:34584) 2024-03-21T11:30:38.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f53b6600640 1 --2- 172.21.15.47:0/1277506709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f7580 0x561f1d3063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f53b5c00640 1 -- 172.21.15.47:0/1277506709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f6bc0 msgr2=0x561f1d303390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f53b5c00640 1 --2- 172.21.15.47:0/1277506709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f6bc0 0x561f1d303390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f53b5c00640 1 -- 172.21.15.47:0/1277506709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f7580 msgr2=0x561f1d3063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f53b5c00640 1 --2- 172.21.15.47:0/1277506709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f7580 0x561f1d3063b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f53b5c00640 1 -- 172.21.15.47:0/1277506709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f1d3069a0 con 0x561f1d2f59f0 2024-03-21T11:30:38.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f53b6600640 1 --2- 172.21.15.47:0/1277506709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f7580 0x561f1d3063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:38.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f53b5c00640 1 --2- 172.21.15.47:0/1277506709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d302e50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f53a4037040 tx=0x7f53a4034750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d19202824f497b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.211+0000 7f5397400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:38.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.212+0000 7f5397400640 1 -- 172.21.15.47:0/1277506709 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4057030 con 0x561f1d2f59f0 2024-03-21T11:30:38.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.212+0000 7f5397400640 1 -- 172.21.15.47:0/1277506709 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f53a402cb80 con 0x561f1d2f59f0 2024-03-21T11:30:38.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.212+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1277506709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53a4051e50 con 0x561f1d2f59f0 2024-03-21T11:30:38.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.212+0000 7f5397400640 1 -- 172.21.15.47:0/1277506709 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53a4039040 con 0x561f1d2f59f0 2024-03-21T11:30:38.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.212+0000 7f53bb3aa740 10 client.27926 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:38.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.212+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1277506709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561f1d3675b0 con 0x561f1d2f59f0 2024-03-21T11:30:38.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.212+0000 7f5397400640 1 -- 172.21.15.47:0/1277506709 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53a4051e50 con 0x561f1d2f59f0 2024-03-21T11:30:38.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.212+0000 7f5397400640 1 -- 172.21.15.47:0/1277506709 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f53a404a040 con 0x561f1d2f59f0 2024-03-21T11:30:38.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.212+0000 7f53bb3aa740 10 client.27926 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:38.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.212+0000 7f53bb3aa740 10 client.27926 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:30:38.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.212+0000 7f53bb3aa740 20 client.27926 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:38.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.212+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/1277506709 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561f1d367ee0 0x561f1d36a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.212+0000 7f53bb3aa740 4 client.27926 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:30:38.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.212+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1277506709 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561f1d261b50 con 0x561f1d367ee0 2024-03-21T11:30:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.213+0000 7f53b5200640 1 --2- 172.21.15.47:0/1277506709 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561f1d367ee0 0x561f1d36a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.213+0000 7f53b5200640 1 --2- 172.21.15.47:0/1277506709 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561f1d367ee0 0x561f1d36a300 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32950fe16637b2de server_cookie=8b8f2665180922e4 in_seq=0 out_seq=0 2024-03-21T11:30:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.213+0000 7f5397400640 10 client.27926 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:30:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.213+0000 7f5397400640 1 -- 172.21.15.47:0/1277506709 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561f1d261b50 con 0x561f1d367ee0 2024-03-21T11:30:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.214+0000 7f5397400640 10 client.27926 handle_command_reply: tid=0 2024-03-21T11:30:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.214+0000 7f53bb3aa740 1 client.27926 shutdown 2024-03-21T11:30:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.214+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1277506709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 msgr2=0x561f1d302e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.214+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/1277506709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d302e50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f53a4037040 tx=0x7f53a4034750 comp rx=0 tx=0).stop 2024-03-21T11:30:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.214+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1277506709 shutdown_connections 2024-03-21T11:30:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.214+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/1277506709 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561f1d367ee0 0x561f1d36a300 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.214+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/1277506709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561f1d2f7580 0x561f1d3063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.214+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/1277506709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f1d2f6bc0 0x561f1d303390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.214+0000 7f53bb3aa740 1 --2- 172.21.15.47:0/1277506709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f1d2f59f0 0x561f1d302e50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.214+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1277506709 >> 172.21.15.47:0/1277506709 conn(0x561f1d2f0de0 msgr2=0x561f1d2f4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.214+0000 7f5397400640 0 client.27926 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:30:38.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.214+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1277506709 shutdown_connections 2024-03-21T11:30:38.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.214+0000 7f53bb3aa740 1 -- 172.21.15.47:0/1277506709 wait complete. 2024-03-21T11:30:38.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.214+0000 7f53bb3aa740 20 client.27926 trim_cache size 0 max 16384 2024-03-21T11:30:38.220 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:30:38.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.215+0000 7f53b7000640 1 -- 172.21.15.47:0/4228685324 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53980a2fa0 msgr2=0x7f53980a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.215+0000 7f53b7000640 1 --2- 172.21.15.47:0/4228685324 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53980a2fa0 0x7f53980a5460 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f53a40349e0 tx=0x7f53a4034ab0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.215+0000 7f53b7000640 1 -- 172.21.15.47:0/4228685324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53b0105370 msgr2=0x7f53b01a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.215+0000 7f53b7000640 1 --2- 172.21.15.47:0/4228685324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53b0105370 0x7f53b01a1040 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f53a800eb10 tx=0x7f53a800c2d0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.215+0000 7f53b7000640 1 -- 172.21.15.47:0/4228685324 shutdown_connections 2024-03-21T11:30:38.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.215+0000 7f53b7000640 1 --2- 172.21.15.47:0/4228685324 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53980a2fa0 0x7f53980a5460 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.215+0000 7f53b7000640 1 --2- 172.21.15.47:0/4228685324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53b0105370 0x7f53b01a1040 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.215+0000 7f53b7000640 1 --2- 172.21.15.47:0/4228685324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53b01049b0 0x7f53b0199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.215+0000 7f53b7000640 1 --2- 172.21.15.47:0/4228685324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53b01037c0 0x7f53b0199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.215+0000 7f53b7000640 1 -- 172.21.15.47:0/4228685324 >> 172.21.15.47:0/4228685324 conn(0x7f53b00fed50 msgr2=0x7f53b0107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.215+0000 7f53b7000640 1 -- 172.21.15.47:0/4228685324 shutdown_connections 2024-03-21T11:30:38.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.215+0000 7f53b7000640 1 -- 172.21.15.47:0/4228685324 wait complete. 2024-03-21T11:30:38.234 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:30:38.234 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:30:38.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:37 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2929136374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:30:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:37 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2929136374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:30:38.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.451+0000 7fc011e00640 1 -- 172.21.15.47:0/578668393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc00c0ff9a0 msgr2=0x7fc00c0ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.451+0000 7fc011e00640 1 --2- 172.21.15.47:0/578668393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc00c0ff9a0 0x7fc00c0ffdc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009860 tx=0x7fbffc02bb70 comp rx=0 tx=0).stop 2024-03-21T11:30:38.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.451+0000 7fc011e00640 1 -- 172.21.15.47:0/578668393 shutdown_connections 2024-03-21T11:30:38.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.451+0000 7fc011e00640 1 --2- 172.21.15.47:0/578668393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc00c1033d0 0x7fc00c109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.451+0000 7fc011e00640 1 --2- 172.21.15.47:0/578668393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc00c102a10 0x7fc00c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.451+0000 7fc011e00640 1 --2- 172.21.15.47:0/578668393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc00c0ff9a0 0x7fc00c0ffdc0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.451+0000 7fc011e00640 1 -- 172.21.15.47:0/578668393 >> 172.21.15.47:0/578668393 conn(0x7fc00c0fb290 msgr2=0x7fc00c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.451+0000 7fc011e00640 1 -- 172.21.15.47:0/578668393 shutdown_connections 2024-03-21T11:30:38.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.451+0000 7fc011e00640 1 -- 172.21.15.47:0/578668393 wait complete. 2024-03-21T11:30:38.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.452+0000 7fc011e00640 1 Processor -- start 2024-03-21T11:30:38.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.453+0000 7fc011e00640 1 -- start start 2024-03-21T11:30:38.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.453+0000 7fc011e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc00c0ff9a0 0x7fc00c102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.453+0000 7fc011e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc00c102a10 0x7fc00c1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.453+0000 7fc011e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc00c1033d0 0x7fc00c100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.453+0000 7fc011e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c10d330 con 0x7fc00c0ff9a0 2024-03-21T11:30:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.453+0000 7fc011e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c10d1b0 con 0x7fc00c102a10 2024-03-21T11:30:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.453+0000 7fc011e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c10d4b0 con 0x7fc00c1033d0 2024-03-21T11:30:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.453+0000 7fc00a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc00c102a10 0x7fc00c1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.453+0000 7fc00b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc00c1033d0 0x7fc00c100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.453+0000 7fc00a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc00c102a10 0x7fc00c1009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58916/0 (socket says 172.21.15.47:58916) 2024-03-21T11:30:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.453+0000 7fc00a000640 1 -- 172.21.15.47:0/109124323 learned_addr learned my addr 172.21.15.47:0/109124323 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.453+0000 7fc00aa00640 1 --2- 172.21.15.47:0/109124323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc00c0ff9a0 0x7fc00c102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.454+0000 7fc00a000640 1 -- 172.21.15.47:0/109124323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc00c1033d0 msgr2=0x7fc00c100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.454+0000 7fc00a000640 1 --2- 172.21.15.47:0/109124323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc00c1033d0 0x7fc00c100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.454+0000 7fc00a000640 1 -- 172.21.15.47:0/109124323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc00c0ff9a0 msgr2=0x7fc00c102380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.454+0000 7fc00a000640 1 --2- 172.21.15.47:0/109124323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc00c0ff9a0 0x7fc00c102380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.454+0000 7fc00a000640 1 -- 172.21.15.47:0/109124323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c101790 con 0x7fc00c102a10 2024-03-21T11:30:38.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.454+0000 7fc00aa00640 1 --2- 172.21.15.47:0/109124323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc00c0ff9a0 0x7fc00c102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:38.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.454+0000 7fc00a000640 1 --2- 172.21.15.47:0/109124323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc00c102a10 0x7fc00c1009c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbff800e9c0 tx=0x7fbff800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.454+0000 7fc010a00640 1 -- 172.21.15.47:0/109124323 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff800ccc0 con 0x7fc00c102a10 2024-03-21T11:30:38.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.454+0000 7fc010a00640 1 -- 172.21.15.47:0/109124323 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbff8005500 con 0x7fc00c102a10 2024-03-21T11:30:38.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.454+0000 7fc010a00640 1 -- 172.21.15.47:0/109124323 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8010430 con 0x7fc00c102a10 2024-03-21T11:30:38.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.454+0000 7fc011e00640 1 -- 172.21.15.47:0/109124323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00c1a1b50 con 0x7fc00c102a10 2024-03-21T11:30:38.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.454+0000 7fc011e00640 1 -- 172.21.15.47:0/109124323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00c1a2050 con 0x7fc00c102a10 2024-03-21T11:30:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.458+0000 7fc014b6c740 1 Processor -- start 2024-03-21T11:30:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.458+0000 7fc014b6c740 1 -- start start 2024-03-21T11:30:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.458+0000 7fc010a00640 1 -- 172.21.15.47:0/109124323 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbff80046c0 con 0x7fc00c102a10 2024-03-21T11:30:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc014b6c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x55747742cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc014b6c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x55747742de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc014b6c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 0x557477434c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc014b6c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574773912c0 con 0x55747742d9e0 2024-03-21T11:30:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc014b6c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557477339c90 con 0x55747742e3a0 2024-03-21T11:30:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc014b6c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747738d360 con 0x55747742c810 2024-03-21T11:30:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc00b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x55747742cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc00b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x55747742cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34598/0 (socket says 172.21.15.47:34598) 2024-03-21T11:30:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc00b400640 1 -- 172.21.15.47:0/2041827501 learned_addr learned my addr 172.21.15.47:0/2041827501 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc00a000640 1 --2- 172.21.15.47:0/2041827501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 0x557477434c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc00aa00640 1 --2- 172.21.15.47:0/2041827501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x55747742de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc00b400640 1 -- 172.21.15.47:0/2041827501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 msgr2=0x557477434c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc00b400640 1 --2- 172.21.15.47:0/2041827501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 0x557477434c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc00b400640 1 -- 172.21.15.47:0/2041827501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 msgr2=0x55747742de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc00b400640 1 --2- 172.21.15.47:0/2041827501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x55747742de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.459+0000 7fc00b400640 1 -- 172.21.15.47:0/2041827501 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55747735e080 con 0x55747742c810 2024-03-21T11:30:38.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.460+0000 7fc00b400640 1 --2- 172.21.15.47:0/2041827501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x55747742cc30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc004009fd0 tx=0x7fc0040118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72804feec20025c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.460+0000 7fbff7e00640 1 -- 172.21.15.47:0/2041827501 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004004d20 con 0x55747742c810 2024-03-21T11:30:38.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.460+0000 7fbff7e00640 1 -- 172.21.15.47:0/2041827501 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc004004050 con 0x55747742c810 2024-03-21T11:30:38.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.460+0000 7fbff7e00640 1 -- 172.21.15.47:0/2041827501 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004005240 con 0x55747742c810 2024-03-21T11:30:38.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.460+0000 7fc010a00640 1 --2- 172.21.15.47:0/109124323 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbff00a2fa0 0x7fbff00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.460+0000 7fc010a00640 1 -- 172.21.15.47:0/109124323 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbff80cfb20 con 0x7fc00c102a10 2024-03-21T11:30:38.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.461+0000 7fc00aa00640 1 --2- 172.21.15.47:0/109124323 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbff00a2fa0 0x7fbff00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.461+0000 7fc014b6c740 1 -- 172.21.15.47:0/2041827501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 msgr2=0x55747742cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.461+0000 7fc014b6c740 1 --2- 172.21.15.47:0/2041827501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x55747742cc30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc004009fd0 tx=0x7fc0040118d0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.461+0000 7fc014b6c740 1 -- 172.21.15.47:0/2041827501 shutdown_connections 2024-03-21T11:30:38.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.461+0000 7fc014b6c740 1 --2- 172.21.15.47:0/2041827501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 0x557477434c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.461+0000 7fc014b6c740 1 --2- 172.21.15.47:0/2041827501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x55747742de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.461+0000 7fc014b6c740 1 --2- 172.21.15.47:0/2041827501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x55747742cc30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.461+0000 7fc014b6c740 1 -- 172.21.15.47:0/2041827501 >> 172.21.15.47:0/2041827501 conn(0x557477427c00 msgr2=0x557477428000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.461+0000 7fc014b6c740 1 -- 172.21.15.47:0/2041827501 shutdown_connections 2024-03-21T11:30:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.461+0000 7fc014b6c740 1 -- 172.21.15.47:0/2041827501 wait complete. 2024-03-21T11:30:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.461+0000 7fc00aa00640 1 --2- 172.21.15.47:0/109124323 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbff00a2fa0 0x7fbff00a5460 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbffc034a10 tx=0x7fbffc034ae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.462+0000 7fc014b6c740 1 -- start start 2024-03-21T11:30:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.462+0000 7fc014b6c740 1 Processor -- start 2024-03-21T11:30:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc014b6c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x557477436a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc014b6c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x557477436fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc014b6c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 0x55747743dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc014b6c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574774353b0 con 0x55747742d9e0 2024-03-21T11:30:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc014b6c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747734d0d0 con 0x55747742e3a0 2024-03-21T11:30:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc014b6c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557477435530 con 0x55747742c810 2024-03-21T11:30:38.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc014b6c740 10 client.0 _ll_register_callbacks cb 0x557477351e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc00aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x557477436a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc00aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x557477436a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34604/0 (socket says 172.21.15.47:34604) 2024-03-21T11:30:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc00aa00640 1 -- 172.21.15.47:0/3157976237 learned_addr learned my addr 172.21.15.47:0/3157976237 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc00b400640 1 --2- 172.21.15.47:0/3157976237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 0x55747743dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc00a000640 1 --2- 172.21.15.47:0/3157976237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x557477436fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc00aa00640 1 -- 172.21.15.47:0/3157976237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 msgr2=0x55747743dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc00aa00640 1 --2- 172.21.15.47:0/3157976237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 0x55747743dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc00aa00640 1 -- 172.21.15.47:0/3157976237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 msgr2=0x557477436fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc00aa00640 1 --2- 172.21.15.47:0/3157976237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x557477436fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc00aa00640 1 -- 172.21.15.47:0/3157976237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574770a53f0 con 0x55747742c810 2024-03-21T11:30:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.463+0000 7fc00b400640 1 --2- 172.21.15.47:0/3157976237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 0x55747743dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.464+0000 7fc00aa00640 1 --2- 172.21.15.47:0/3157976237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x557477436a60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fbffc02c180 tx=0x7fbffc02bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24b85708f1d36c91 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.464+0000 7fbff4c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:38.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.464+0000 7fbff4c00640 1 -- 172.21.15.47:0/3157976237 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc04d030 con 0x55747742c810 2024-03-21T11:30:38.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.464+0000 7fbff4c00640 1 -- 172.21.15.47:0/3157976237 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbffc005020 con 0x55747742c810 2024-03-21T11:30:38.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.464+0000 7fc014b6c740 1 -- 172.21.15.47:0/3157976237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc004004050 con 0x55747742c810 2024-03-21T11:30:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.464+0000 7fbff4c00640 1 -- 172.21.15.47:0/3157976237 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbffc057050 con 0x55747742c810 2024-03-21T11:30:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.464+0000 7fc014b6c740 10 client.27955 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.464+0000 7fc014b6c740 1 -- 172.21.15.47:0/3157976237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55747716f7d0 con 0x55747742c810 2024-03-21T11:30:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.465+0000 7fbff4c00640 1 -- 172.21.15.47:0/3157976237 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc004004050 con 0x55747742c810 2024-03-21T11:30:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.465+0000 7fbff4c00640 1 -- 172.21.15.47:0/3157976237 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbffc004750 con 0x55747742c810 2024-03-21T11:30:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.465+0000 7fc014b6c740 10 client.27955 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.465+0000 7fc014b6c740 10 client.27955 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:30:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.465+0000 7fc014b6c740 20 client.27955 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.465+0000 7fc014b6c740 1 --2- 172.21.15.47:0/3157976237 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x557477442530 0x5574774449a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.465+0000 7fc014b6c740 4 client.27955 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:38.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.465+0000 7fc014b6c740 1 -- 172.21.15.47:0/3157976237 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55747739e800 con 0x557477442530 2024-03-21T11:30:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.466+0000 7fc00a000640 1 --2- 172.21.15.47:0/3157976237 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x557477442530 0x5574774449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.466+0000 7fc00a000640 1 --2- 172.21.15.47:0/3157976237 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x557477442530 0x5574774449a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc175e2673f24c77 server_cookie=9d9f9f573eb04305 in_seq=0 out_seq=0 2024-03-21T11:30:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.466+0000 7fbff4c00640 10 client.27955 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:30:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.467+0000 7fbff4c00640 1 -- 172.21.15.47:0/3157976237 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55747739e800 con 0x557477442530 2024-03-21T11:30:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.467+0000 7fbff4c00640 10 client.27955 handle_command_reply: tid=0 2024-03-21T11:30:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.467+0000 7fc014b6c740 1 client.27955 shutdown 2024-03-21T11:30:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.467+0000 7fc014b6c740 1 -- 172.21.15.47:0/3157976237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 msgr2=0x557477436a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.467+0000 7fc014b6c740 1 --2- 172.21.15.47:0/3157976237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x557477436a60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fbffc02c180 tx=0x7fbffc02bb70 comp rx=0 tx=0).stop 2024-03-21T11:30:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.467+0000 7fc014b6c740 1 -- 172.21.15.47:0/3157976237 shutdown_connections 2024-03-21T11:30:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.467+0000 7fc014b6c740 1 --2- 172.21.15.47:0/3157976237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 0x55747743dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.467+0000 7fc014b6c740 1 --2- 172.21.15.47:0/3157976237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x557477436fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.467+0000 7fc014b6c740 1 --2- 172.21.15.47:0/3157976237 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x557477442530 0x5574774449a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.467+0000 7fc014b6c740 1 --2- 172.21.15.47:0/3157976237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x557477436a60 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.467+0000 7fbff4c00640 0 client.27955 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:30:38.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.467+0000 7fc014b6c740 1 -- 172.21.15.47:0/3157976237 >> 172.21.15.47:0/3157976237 conn(0x557477427c00 msgr2=0x557477426eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.467+0000 7fc014b6c740 1 -- 172.21.15.47:0/3157976237 shutdown_connections 2024-03-21T11:30:38.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.468+0000 7fc014b6c740 1 -- 172.21.15.47:0/3157976237 wait complete. 2024-03-21T11:30:38.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.468+0000 7fc014b6c740 20 client.27955 trim_cache size 0 max 16384 2024-03-21T11:30:38.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.483+0000 7fc014b6c740 1 Processor -- start 2024-03-21T11:30:38.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.483+0000 7fc014b6c740 1 -- start start 2024-03-21T11:30:38.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.483+0000 7fc014b6c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742c810 0x557477430c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.483+0000 7fc014b6c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x55747743dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.483+0000 7fc014b6c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742e3a0 0x557477431150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.483+0000 7fc014b6c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c1ac2d0 con 0x55747742d9e0 2024-03-21T11:30:38.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.483+0000 7fc014b6c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c1adc50 con 0x55747742c810 2024-03-21T11:30:38.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.483+0000 7fc014b6c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c1ac930 con 0x55747742e3a0 2024-03-21T11:30:38.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.484+0000 7fc00aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742c810 0x557477430c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.484+0000 7fc00aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742c810 0x557477430c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58950/0 (socket says 172.21.15.47:58950) 2024-03-21T11:30:38.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.484+0000 7fc00aa00640 1 -- 172.21.15.47:0/1306226712 learned_addr learned my addr 172.21.15.47:0/1306226712 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.484+0000 7fc00b400640 1 --2- 172.21.15.47:0/1306226712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742e3a0 0x557477431150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.484+0000 7fc00a000640 1 --2- 172.21.15.47:0/1306226712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x55747743dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.484+0000 7fc00aa00640 1 -- 172.21.15.47:0/1306226712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742e3a0 msgr2=0x557477431150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.484+0000 7fc00aa00640 1 --2- 172.21.15.47:0/1306226712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742e3a0 0x557477431150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.484+0000 7fc00aa00640 1 -- 172.21.15.47:0/1306226712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 msgr2=0x55747743dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.484+0000 7fc00aa00640 1 --2- 172.21.15.47:0/1306226712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x55747743dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.484+0000 7fc00aa00640 1 -- 172.21.15.47:0/1306226712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557477435b30 con 0x55747742c810 2024-03-21T11:30:38.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.484+0000 7fc00aa00640 1 --2- 172.21.15.47:0/1306226712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742c810 0x557477430c10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbffc000c00 tx=0x7fbffc035bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad54508a3fd6308 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.485+0000 7fbff7400640 1 -- 172.21.15.47:0/1306226712 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc04d040 con 0x55747742c810 2024-03-21T11:30:38.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.485+0000 7fbff7400640 1 -- 172.21.15.47:0/1306226712 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbffc004490 con 0x55747742c810 2024-03-21T11:30:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.485+0000 7fbff7400640 1 -- 172.21.15.47:0/1306226712 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc039040 con 0x55747742c810 2024-03-21T11:30:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.486+0000 7fc014b6c740 1 -- 172.21.15.47:0/1306226712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742c810 msgr2=0x557477430c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.486+0000 7fc014b6c740 1 --2- 172.21.15.47:0/1306226712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742c810 0x557477430c10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbffc000c00 tx=0x7fbffc035bf0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.486+0000 7fc014b6c740 1 -- 172.21.15.47:0/1306226712 shutdown_connections 2024-03-21T11:30:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.486+0000 7fc014b6c740 1 --2- 172.21.15.47:0/1306226712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742e3a0 0x557477431150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.486+0000 7fc014b6c740 1 --2- 172.21.15.47:0/1306226712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x55747743dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.486+0000 7fc014b6c740 1 --2- 172.21.15.47:0/1306226712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742c810 0x557477430c10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.486+0000 7fc014b6c740 1 -- 172.21.15.47:0/1306226712 >> 172.21.15.47:0/1306226712 conn(0x557477427c00 msgr2=0x55747742b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.486+0000 7fc014b6c740 1 -- 172.21.15.47:0/1306226712 shutdown_connections 2024-03-21T11:30:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.486+0000 7fc014b6c740 1 -- 172.21.15.47:0/1306226712 wait complete. 2024-03-21T11:30:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.486+0000 7fc014b6c740 1 -- start start 2024-03-21T11:30:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.486+0000 7fc014b6c740 1 Processor -- start 2024-03-21T11:30:38.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc014b6c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x55747743d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc014b6c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x55747743df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc014b6c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 0x55747743c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc014b6c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c1ad5f0 con 0x55747742d9e0 2024-03-21T11:30:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc014b6c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c1acf90 con 0x55747742e3a0 2024-03-21T11:30:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc014b6c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c1aa490 con 0x55747742c810 2024-03-21T11:30:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc014b6c740 10 client.0 _ll_register_callbacks cb 0x7fbffc037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc00b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 0x55747743c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc00aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x55747743d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc00b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 0x55747743c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58958/0 (socket says 172.21.15.47:58958) 2024-03-21T11:30:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc00b400640 1 -- 172.21.15.47:0/4273662381 learned_addr learned my addr 172.21.15.47:0/4273662381 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc00aa00640 1 --2- 172.21.15.47:0/4273662381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x55747743d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34612/0 (socket says 172.21.15.47:34612) 2024-03-21T11:30:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc00a000640 1 --2- 172.21.15.47:0/4273662381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x55747743df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc00aa00640 1 -- 172.21.15.47:0/4273662381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 msgr2=0x55747743c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc00aa00640 1 --2- 172.21.15.47:0/4273662381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 0x55747743c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc00aa00640 1 -- 172.21.15.47:0/4273662381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 msgr2=0x55747743df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc00aa00640 1 --2- 172.21.15.47:0/4273662381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x55747743df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.487+0000 7fc00aa00640 1 -- 172.21.15.47:0/4273662381 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55747743c8c0 con 0x55747742c810 2024-03-21T11:30:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.488+0000 7fc00aa00640 1 --2- 172.21.15.47:0/4273662381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x55747743d9e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbffc004b90 tx=0x7fbffc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b74ec03cc92591da server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.488+0000 7fbff4c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.488+0000 7fbff4c00640 1 -- 172.21.15.47:0/4273662381 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc04d030 con 0x55747742c810 2024-03-21T11:30:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.488+0000 7fc014b6c740 1 -- 172.21.15.47:0/4273662381 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbffc004490 con 0x55747742c810 2024-03-21T11:30:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.488+0000 7fbff4c00640 1 -- 172.21.15.47:0/4273662381 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbffc032d10 con 0x55747742c810 2024-03-21T11:30:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.488+0000 7fbff4c00640 1 -- 172.21.15.47:0/4273662381 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbffc02c510 con 0x55747742c810 2024-03-21T11:30:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.488+0000 7fc014b6c740 10 client.27961 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.488+0000 7fc014b6c740 1 -- 172.21.15.47:0/4273662381 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55747749e8f0 con 0x55747742c810 2024-03-21T11:30:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.489+0000 7fbff4c00640 1 -- 172.21.15.47:0/4273662381 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbffc004490 con 0x55747742c810 2024-03-21T11:30:38.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.489+0000 7fbff4c00640 1 -- 172.21.15.47:0/4273662381 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbffc0052d0 con 0x55747742c810 2024-03-21T11:30:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.489+0000 7fc014b6c740 10 client.27961 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.489+0000 7fc014b6c740 10 client.27961 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:30:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.489+0000 7fc014b6c740 20 client.27961 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.489+0000 7fc014b6c740 1 --2- 172.21.15.47:0/4273662381 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55747749f180 0x5574774a1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.489+0000 7fc014b6c740 4 client.27961 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:30:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.489+0000 7fc014b6c740 1 -- 172.21.15.47:0/4273662381 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557477398b50 con 0x55747749f180 2024-03-21T11:30:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.489+0000 7fc00a000640 1 --2- 172.21.15.47:0/4273662381 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55747749f180 0x5574774a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fc00a000640 1 --2- 172.21.15.47:0/4273662381 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55747749f180 0x5574774a1570 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75f55ba926d5bb17 server_cookie=1c861eb6e07ce4cf in_seq=0 out_seq=0 2024-03-21T11:30:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fbff4c00640 10 client.27961 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:30:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fbff4c00640 1 -- 172.21.15.47:0/4273662381 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557477398b50 con 0x55747749f180 2024-03-21T11:30:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fbff4c00640 10 client.27961 handle_command_reply: tid=0 2024-03-21T11:30:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fc014b6c740 1 client.27961 shutdown 2024-03-21T11:30:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fc014b6c740 1 -- 172.21.15.47:0/4273662381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 msgr2=0x55747743d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fc014b6c740 1 --2- 172.21.15.47:0/4273662381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x55747743d9e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbffc004b90 tx=0x7fbffc004750 comp rx=0 tx=0).stop 2024-03-21T11:30:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fc014b6c740 1 -- 172.21.15.47:0/4273662381 shutdown_connections 2024-03-21T11:30:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fc014b6c740 1 --2- 172.21.15.47:0/4273662381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55747742e3a0 0x55747743c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fc014b6c740 1 --2- 172.21.15.47:0/4273662381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55747742d9e0 0x55747743df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fc014b6c740 1 --2- 172.21.15.47:0/4273662381 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55747749f180 0x5574774a1570 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fc014b6c740 1 --2- 172.21.15.47:0/4273662381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747742c810 0x55747743d9e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fbff4c00640 0 client.27961 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:30:38.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fc014b6c740 1 -- 172.21.15.47:0/4273662381 >> 172.21.15.47:0/4273662381 conn(0x557477427c00 msgr2=0x557477426620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fc014b6c740 1 -- 172.21.15.47:0/4273662381 shutdown_connections 2024-03-21T11:30:38.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.490+0000 7fc014b6c740 1 -- 172.21.15.47:0/4273662381 wait complete. 2024-03-21T11:30:38.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.491+0000 7fc014b6c740 20 client.27961 trim_cache size 0 max 16384 2024-03-21T11:30:38.496 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:30:38.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.491+0000 7fc011e00640 1 -- 172.21.15.47:0/109124323 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbff00a2fa0 msgr2=0x7fbff00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.491+0000 7fc011e00640 1 --2- 172.21.15.47:0/109124323 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbff00a2fa0 0x7fbff00a5460 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbffc034a10 tx=0x7fbffc034ae0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.491+0000 7fc011e00640 1 -- 172.21.15.47:0/109124323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc00c102a10 msgr2=0x7fc00c1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.491+0000 7fc011e00640 1 --2- 172.21.15.47:0/109124323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc00c102a10 0x7fc00c1009c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbff800e9c0 tx=0x7fbff800ee80 comp rx=0 tx=0).stop 2024-03-21T11:30:38.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.491+0000 7fc011e00640 1 -- 172.21.15.47:0/109124323 shutdown_connections 2024-03-21T11:30:38.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.491+0000 7fc011e00640 1 --2- 172.21.15.47:0/109124323 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbff00a2fa0 0x7fbff00a5460 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.491+0000 7fc011e00640 1 --2- 172.21.15.47:0/109124323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc00c1033d0 0x7fc00c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.491+0000 7fc011e00640 1 --2- 172.21.15.47:0/109124323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc00c102a10 0x7fc00c1009c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.491+0000 7fc011e00640 1 --2- 172.21.15.47:0/109124323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc00c0ff9a0 0x7fc00c102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.491+0000 7fc011e00640 1 -- 172.21.15.47:0/109124323 >> 172.21.15.47:0/109124323 conn(0x7fc00c0fb290 msgr2=0x7fc00c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.491+0000 7fc011e00640 1 -- 172.21.15.47:0/109124323 shutdown_connections 2024-03-21T11:30:38.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.491+0000 7fc011e00640 1 -- 172.21.15.47:0/109124323 wait complete. 2024-03-21T11:30:38.513 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:30:38.513 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:30:38.529 INFO:tasks.workunit.client.0.smithi047.stdout: 1 502222 8.26 MB/sec execute 202 sec latency 11.418 ms 2024-03-21T11:30:38.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.745+0000 7fde35400640 1 -- 172.21.15.47:0/223075071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde30102080 msgr2=0x7fde30104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.745+0000 7fde35400640 1 --2- 172.21.15.47:0/223075071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde30102080 0x7fde30104470 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fde20009900 tx=0x7fde2002bc30 comp rx=0 tx=0).stop 2024-03-21T11:30:38.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.745+0000 7fde35400640 1 -- 172.21.15.47:0/223075071 shutdown_connections 2024-03-21T11:30:38.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.745+0000 7fde35400640 1 --2- 172.21.15.47:0/223075071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde301072e0 0x7fde301096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.745+0000 7fde35400640 1 --2- 172.21.15.47:0/223075071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde301049b0 0x7fde30106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.745+0000 7fde35400640 1 --2- 172.21.15.47:0/223075071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde30102080 0x7fde30104470 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.745+0000 7fde35400640 1 -- 172.21.15.47:0/223075071 >> 172.21.15.47:0/223075071 conn(0x7fde300fba10 msgr2=0x7fde300fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.745+0000 7fde35400640 1 -- 172.21.15.47:0/223075071 shutdown_connections 2024-03-21T11:30:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.745+0000 7fde35400640 1 -- 172.21.15.47:0/223075071 wait complete. 2024-03-21T11:30:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.746+0000 7fde35400640 1 Processor -- start 2024-03-21T11:30:38.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.746+0000 7fde35400640 1 -- start start 2024-03-21T11:30:38.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.747+0000 7fde35400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde30102080 0x7fde301019f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.747+0000 7fde35400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde301049b0 0x7fde30100030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.747+0000 7fde35400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde301072e0 0x7fde30100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.747+0000 7fde35400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3010d260 con 0x7fde30102080 2024-03-21T11:30:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.747+0000 7fde35400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3010d0e0 con 0x7fde301049b0 2024-03-21T11:30:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.747+0000 7fde35400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3010d3e0 con 0x7fde301072e0 2024-03-21T11:30:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.747+0000 7fde2fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde30102080 0x7fde301019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.747+0000 7fde2f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde301049b0 0x7fde30100030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.747+0000 7fde2f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde301049b0 0x7fde30100030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58970/0 (socket says 172.21.15.47:58970) 2024-03-21T11:30:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.747+0000 7fde2f400640 1 -- 172.21.15.47:0/2450342394 learned_addr learned my addr 172.21.15.47:0/2450342394 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.747+0000 7fde34a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde301072e0 0x7fde30100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.748+0000 7fde2f400640 1 -- 172.21.15.47:0/2450342394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde301072e0 msgr2=0x7fde30100570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.748+0000 7fde2f400640 1 --2- 172.21.15.47:0/2450342394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde301072e0 0x7fde30100570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.748+0000 7fde2f400640 1 -- 172.21.15.47:0/2450342394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde30102080 msgr2=0x7fde301019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.748+0000 7fde2f400640 1 --2- 172.21.15.47:0/2450342394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde30102080 0x7fde301019f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.748+0000 7fde2f400640 1 -- 172.21.15.47:0/2450342394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde30100e00 con 0x7fde301049b0 2024-03-21T11:30:38.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.748+0000 7fde34a00640 1 --2- 172.21.15.47:0/2450342394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde301072e0 0x7fde30100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:38.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.748+0000 7fde2f400640 1 --2- 172.21.15.47:0/2450342394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde301049b0 0x7fde30100030 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fde1c005fd0 tx=0x7fde1c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.748+0000 7fde2cc00640 1 -- 172.21.15.47:0/2450342394 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c017070 con 0x7fde301049b0 2024-03-21T11:30:38.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.749+0000 7fde2cc00640 1 -- 172.21.15.47:0/2450342394 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde1c009c30 con 0x7fde301049b0 2024-03-21T11:30:38.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.749+0000 7fde2cc00640 1 -- 172.21.15.47:0/2450342394 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c010460 con 0x7fde301049b0 2024-03-21T11:30:38.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.749+0000 7fde35400640 1 -- 172.21.15.47:0/2450342394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde301a1a00 con 0x7fde301049b0 2024-03-21T11:30:38.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.749+0000 7fde35400640 1 -- 172.21.15.47:0/2450342394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde301a1f00 con 0x7fde301049b0 2024-03-21T11:30:38.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.753+0000 7fde39836740 1 Processor -- start 2024-03-21T11:30:38.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.753+0000 7fde39836740 1 -- start start 2024-03-21T11:30:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.753+0000 7fde39836740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2a9f0 0x5646e0e2ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.753+0000 7fde39836740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 0x5646e0e2c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.753+0000 7fde39836740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2c580 0x5646e0e32e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.753+0000 7fde39836740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e0d8f2c0 con 0x5646e0e2c580 2024-03-21T11:30:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.753+0000 7fde39836740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e0d37c90 con 0x5646e0e2bbc0 2024-03-21T11:30:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.753+0000 7fde39836740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e0d8b360 con 0x5646e0e2a9f0 2024-03-21T11:30:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.753+0000 7fde34a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2a9f0 0x5646e0e2ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.753+0000 7fde34a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2a9f0 0x5646e0e2ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34630/0 (socket says 172.21.15.47:34630) 2024-03-21T11:30:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.753+0000 7fde34a00640 1 -- 172.21.15.47:0/4108475119 learned_addr learned my addr 172.21.15.47:0/4108475119 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.753+0000 7fde2fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 0x5646e0e2c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.754+0000 7fde34a00640 1 -- 172.21.15.47:0/4108475119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 msgr2=0x5646e0e2c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.754+0000 7fde34a00640 1 --2- 172.21.15.47:0/4108475119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 0x5646e0e2c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.754+0000 7fde34a00640 1 -- 172.21.15.47:0/4108475119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2c580 msgr2=0x5646e0e32e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:30:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.754+0000 7fde34a00640 1 --2- 172.21.15.47:0/4108475119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2c580 0x5646e0e32e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.754+0000 7fde34a00640 1 -- 172.21.15.47:0/4108475119 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646e0d5c080 con 0x5646e0e2a9f0 2024-03-21T11:30:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.754+0000 7fde34a00640 1 --2- 172.21.15.47:0/4108475119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2a9f0 0x5646e0e2ae10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fde28011790 tx=0x7fde28011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e866bbda3fd69b89 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.754+0000 7fde2cc00640 1 -- 172.21.15.47:0/2450342394 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fde1c003e70 con 0x7fde301049b0 2024-03-21T11:30:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.754+0000 7fde25200640 1 -- 172.21.15.47:0/4108475119 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28003c30 con 0x5646e0e2a9f0 2024-03-21T11:30:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.754+0000 7fde25200640 1 -- 172.21.15.47:0/4108475119 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde2800fe80 con 0x5646e0e2a9f0 2024-03-21T11:30:38.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.754+0000 7fde25200640 1 -- 172.21.15.47:0/4108475119 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde280193e0 con 0x5646e0e2a9f0 2024-03-21T11:30:38.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.755+0000 7fde39836740 1 -- 172.21.15.47:0/4108475119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2a9f0 msgr2=0x5646e0e2ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.755+0000 7fde39836740 1 --2- 172.21.15.47:0/4108475119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2a9f0 0x5646e0e2ae10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fde28011790 tx=0x7fde28011c50 comp rx=0 tx=0).stop 2024-03-21T11:30:38.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.755+0000 7fde39836740 1 -- 172.21.15.47:0/4108475119 shutdown_connections 2024-03-21T11:30:38.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.755+0000 7fde39836740 1 --2- 172.21.15.47:0/4108475119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2c580 0x5646e0e32e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.755+0000 7fde39836740 1 --2- 172.21.15.47:0/4108475119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 0x5646e0e2c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.755+0000 7fde39836740 1 --2- 172.21.15.47:0/4108475119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2a9f0 0x5646e0e2ae10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.755+0000 7fde39836740 1 -- 172.21.15.47:0/4108475119 >> 172.21.15.47:0/4108475119 conn(0x5646e0e25de0 msgr2=0x5646e0e261e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.755+0000 7fde39836740 1 -- 172.21.15.47:0/4108475119 shutdown_connections 2024-03-21T11:30:38.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.756+0000 7fde39836740 1 -- 172.21.15.47:0/4108475119 wait complete. 2024-03-21T11:30:38.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.756+0000 7fde39836740 1 -- start start 2024-03-21T11:30:38.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.756+0000 7fde2cc00640 1 --2- 172.21.15.47:0/2450342394 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fde140a2eb0 0x7fde140a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.756+0000 7fde39836740 1 Processor -- start 2024-03-21T11:30:38.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.756+0000 7fde2cc00640 1 -- 172.21.15.47:0/2450342394 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fde1c0cf3d0 con 0x7fde301049b0 2024-03-21T11:30:38.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.756+0000 7fde34a00640 1 --2- 172.21.15.47:0/2450342394 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fde140a2eb0 0x7fde140a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.757+0000 7fde39836740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2a9f0 0x5646e0e34cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.757+0000 7fde39836740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2bbc0 0x5646e0e35210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.757+0000 7fde39836740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2c580 0x5646e0e3c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.757+0000 7fde39836740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e0e33590 con 0x5646e0e2c580 2024-03-21T11:30:38.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.757+0000 7fde39836740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e0d4b0d0 con 0x5646e0e2a9f0 2024-03-21T11:30:38.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.757+0000 7fde39836740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e0e33710 con 0x5646e0e2bbc0 2024-03-21T11:30:38.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.757+0000 7fde39836740 10 client.0 _ll_register_callbacks cb 0x5646e0d4fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:38.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.757+0000 7fde2fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2a9f0 0x5646e0e34cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.757+0000 7fde2fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2a9f0 0x5646e0e34cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58988/0 (socket says 172.21.15.47:58988) 2024-03-21T11:30:38.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.757+0000 7fde2fe00640 1 -- 172.21.15.47:0/3625366116 learned_addr learned my addr 172.21.15.47:0/3625366116 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.757+0000 7fde2f400640 1 --2- 172.21.15.47:0/3625366116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2bbc0 0x5646e0e35210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.758+0000 7fde2fe00640 1 -- 172.21.15.47:0/3625366116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2bbc0 msgr2=0x5646e0e35210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.758+0000 7fde2fe00640 1 --2- 172.21.15.47:0/3625366116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2bbc0 0x5646e0e35210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.758+0000 7fde2fe00640 1 -- 172.21.15.47:0/3625366116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2c580 msgr2=0x5646e0e3c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.758+0000 7fde2fe00640 1 --2- 172.21.15.47:0/3625366116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2c580 0x5646e0e3c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.758+0000 7fde2fe00640 1 -- 172.21.15.47:0/3625366116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646e0aa33f0 con 0x5646e0e2a9f0 2024-03-21T11:30:38.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.758+0000 7fde2f400640 1 --2- 172.21.15.47:0/3625366116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2bbc0 0x5646e0e35210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:38.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.758+0000 7fde34a00640 1 --2- 172.21.15.47:0/2450342394 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fde140a2eb0 0x7fde140a5370 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fde28011760 tx=0x7fde28005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.758+0000 7fde2fe00640 1 --2- 172.21.15.47:0/3625366116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2a9f0 0x5646e0e34cd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fde20005ab0 tx=0x7fde20009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c81dfd6b316180b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.758+0000 7fde19600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:38.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.759+0000 7fde19600640 1 -- 172.21.15.47:0/3625366116 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde200046c0 con 0x5646e0e2a9f0 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.759+0000 7fde19600640 1 -- 172.21.15.47:0/3625366116 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fde20004860 con 0x5646e0e2a9f0 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.759+0000 7fde39836740 1 -- 172.21.15.47:0/3625366116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646e0d9c800 con 0x5646e0e2a9f0 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.759+0000 7fde19600640 1 -- 172.21.15.47:0/3625366116 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde20004a60 con 0x5646e0e2a9f0 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.759+0000 7fde39836740 10 client.27956 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.759+0000 7fde39836740 1 -- 172.21.15.47:0/3625366116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5646e0b6d7d0 con 0x5646e0e2a9f0 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.759+0000 7fde19600640 1 -- 172.21.15.47:0/3625366116 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5646e0d9c800 con 0x5646e0e2a9f0 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.760+0000 7fde19600640 1 -- 172.21.15.47:0/3625366116 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fde20003850 con 0x5646e0e2a9f0 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.760+0000 7fde39836740 10 client.27956 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.760+0000 7fde39836740 10 client.27956 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.760+0000 7fde39836740 20 client.27956 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.760+0000 7fde39836740 1 --2- 172.21.15.47:0/3625366116 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5646e0e40670 0x5646e0e42a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.760+0000 7fde39836740 4 client.27956 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.760+0000 7fde39836740 1 -- 172.21.15.47:0/3625366116 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646e0d96b50 con 0x5646e0e40670 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.760+0000 7fde2f400640 1 --2- 172.21.15.47:0/3625366116 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5646e0e40670 0x5646e0e42a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.761+0000 7fde19600640 10 client.27956 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:30:38.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.761+0000 7fde2f400640 1 --2- 172.21.15.47:0/3625366116 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5646e0e40670 0x5646e0e42a80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5114489cd1386e0 server_cookie=7719a1d10b387bd3 in_seq=0 out_seq=0 2024-03-21T11:30:38.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.762+0000 7fde19600640 1 -- 172.21.15.47:0/3625366116 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5646e0d96b50 con 0x5646e0e40670 2024-03-21T11:30:38.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.762+0000 7fde19600640 10 client.27956 handle_command_reply: tid=0 2024-03-21T11:30:38.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.762+0000 7fde39836740 1 client.27956 shutdown 2024-03-21T11:30:38.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.762+0000 7fde39836740 1 -- 172.21.15.47:0/3625366116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2a9f0 msgr2=0x5646e0e34cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.762+0000 7fde39836740 1 --2- 172.21.15.47:0/3625366116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2a9f0 0x5646e0e34cd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fde20005ab0 tx=0x7fde20009680 comp rx=0 tx=0).stop 2024-03-21T11:30:38.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.762+0000 7fde39836740 1 -- 172.21.15.47:0/3625366116 shutdown_connections 2024-03-21T11:30:38.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.762+0000 7fde39836740 1 --2- 172.21.15.47:0/3625366116 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5646e0e40670 0x5646e0e42a80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.762+0000 7fde39836740 1 --2- 172.21.15.47:0/3625366116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2c580 0x5646e0e3c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.763+0000 7fde19600640 0 client.27956 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:30:38.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.762+0000 7fde39836740 1 --2- 172.21.15.47:0/3625366116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2bbc0 0x5646e0e35210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.763+0000 7fde39836740 1 --2- 172.21.15.47:0/3625366116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2a9f0 0x5646e0e34cd0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.763+0000 7fde39836740 1 -- 172.21.15.47:0/3625366116 >> 172.21.15.47:0/3625366116 conn(0x5646e0e25de0 msgr2=0x5646e0e25160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.763+0000 7fde39836740 1 -- 172.21.15.47:0/3625366116 shutdown_connections 2024-03-21T11:30:38.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.763+0000 7fde39836740 1 -- 172.21.15.47:0/3625366116 wait complete. 2024-03-21T11:30:38.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.763+0000 7fde39836740 20 client.27956 trim_cache size 0 max 16384 2024-03-21T11:30:38.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.778+0000 7fde39836740 1 Processor -- start 2024-03-21T11:30:38.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.778+0000 7fde39836740 1 -- start start 2024-03-21T11:30:38.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.778+0000 7fde39836740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2a9f0 0x5646e0e341f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.778+0000 7fde39836740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 0x5646e0e3c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.778+0000 7fde39836740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2c580 0x5646e0e248d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.778+0000 7fde39836740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde301ac240 con 0x5646e0e2a9f0 2024-03-21T11:30:38.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.778+0000 7fde39836740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde301adbc0 con 0x5646e0e2bbc0 2024-03-21T11:30:38.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.778+0000 7fde39836740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde301ac8a0 con 0x5646e0e2c580 2024-03-21T11:30:38.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.779+0000 7fde2fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2a9f0 0x5646e0e341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.779+0000 7fde34a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2c580 0x5646e0e248d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.779+0000 7fde2f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 0x5646e0e3c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.779+0000 7fde34a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2c580 0x5646e0e248d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34648/0 (socket says 172.21.15.47:34648) 2024-03-21T11:30:38.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.779+0000 7fde2fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2a9f0 0x5646e0e341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51184/0 (socket says 172.21.15.47:51184) 2024-03-21T11:30:38.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.779+0000 7fde34a00640 1 -- 172.21.15.47:0/1763205143 learned_addr learned my addr 172.21.15.47:0/1763205143 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.779+0000 7fde2f400640 1 -- 172.21.15.47:0/1763205143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2c580 msgr2=0x5646e0e248d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.779+0000 7fde2f400640 1 --2- 172.21.15.47:0/1763205143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2c580 0x5646e0e248d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.779+0000 7fde2f400640 1 -- 172.21.15.47:0/1763205143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2a9f0 msgr2=0x5646e0e341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.779+0000 7fde2f400640 1 --2- 172.21.15.47:0/1763205143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2a9f0 0x5646e0e341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.779+0000 7fde2f400640 1 -- 172.21.15.47:0/1763205143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646e0e33470 con 0x5646e0e2bbc0 2024-03-21T11:30:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.779+0000 7fde34a00640 1 --2- 172.21.15.47:0/1763205143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2c580 0x5646e0e248d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.780+0000 7fde2f400640 1 --2- 172.21.15.47:0/1763205143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 0x5646e0e3c1c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0ccbd0 tx=0x7fde1c0ce040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5acbc238034a2feb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.780+0000 7fde1be00640 1 -- 172.21.15.47:0/1763205143 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c0035f0 con 0x5646e0e2bbc0 2024-03-21T11:30:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.780+0000 7fde1be00640 1 -- 172.21.15.47:0/1763205143 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde1c0ccc90 con 0x5646e0e2bbc0 2024-03-21T11:30:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.780+0000 7fde1be00640 1 -- 172.21.15.47:0/1763205143 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c09ac00 con 0x5646e0e2bbc0 2024-03-21T11:30:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.781+0000 7fde39836740 1 -- 172.21.15.47:0/1763205143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 msgr2=0x5646e0e3c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.781+0000 7fde39836740 1 --2- 172.21.15.47:0/1763205143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 0x5646e0e3c1c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0ccbd0 tx=0x7fde1c0ce040 comp rx=0 tx=0).stop 2024-03-21T11:30:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.781+0000 7fde39836740 1 -- 172.21.15.47:0/1763205143 shutdown_connections 2024-03-21T11:30:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.781+0000 7fde39836740 1 --2- 172.21.15.47:0/1763205143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2c580 0x5646e0e248d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.781+0000 7fde39836740 1 --2- 172.21.15.47:0/1763205143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 0x5646e0e3c1c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.781+0000 7fde39836740 1 --2- 172.21.15.47:0/1763205143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2a9f0 0x5646e0e341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.781+0000 7fde39836740 1 -- 172.21.15.47:0/1763205143 >> 172.21.15.47:0/1763205143 conn(0x5646e0e25de0 msgr2=0x5646e0e33ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.781+0000 7fde39836740 1 -- 172.21.15.47:0/1763205143 shutdown_connections 2024-03-21T11:30:38.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.781+0000 7fde39836740 1 -- 172.21.15.47:0/1763205143 wait complete. 2024-03-21T11:30:38.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.782+0000 7fde39836740 1 -- start start 2024-03-21T11:30:38.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.782+0000 7fde39836740 1 Processor -- start 2024-03-21T11:30:38.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.782+0000 7fde39836740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2a9f0 0x5646e0e375e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.782+0000 7fde39836740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 0x5646e0e37b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.782+0000 7fde39836740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2c580 0x5646e0e3aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.782+0000 7fde39836740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde301ad560 con 0x5646e0e2c580 2024-03-21T11:30:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.782+0000 7fde39836740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde301acf00 con 0x5646e0e2bbc0 2024-03-21T11:30:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.782+0000 7fde39836740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde301aa400 con 0x5646e0e2a9f0 2024-03-21T11:30:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.782+0000 7fde39836740 10 client.0 _ll_register_callbacks cb 0x7fde20009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.782+0000 7fde2fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2a9f0 0x5646e0e375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.783+0000 7fde2fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2a9f0 0x5646e0e375e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34652/0 (socket says 172.21.15.47:34652) 2024-03-21T11:30:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.783+0000 7fde2fe00640 1 -- 172.21.15.47:0/3471195617 learned_addr learned my addr 172.21.15.47:0/3471195617 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.783+0000 7fde2f400640 1 --2- 172.21.15.47:0/3471195617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 0x5646e0e37b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.783+0000 7fde34a00640 1 --2- 172.21.15.47:0/3471195617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2c580 0x5646e0e3aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.783+0000 7fde2fe00640 1 -- 172.21.15.47:0/3471195617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 msgr2=0x5646e0e37b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.783+0000 7fde2fe00640 1 --2- 172.21.15.47:0/3471195617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 0x5646e0e37b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.783+0000 7fde2fe00640 1 -- 172.21.15.47:0/3471195617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2c580 msgr2=0x5646e0e3aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.783+0000 7fde2fe00640 1 --2- 172.21.15.47:0/3471195617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2c580 0x5646e0e3aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.783+0000 7fde2fe00640 1 -- 172.21.15.47:0/3471195617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646e0e3b0c0 con 0x5646e0e2a9f0 2024-03-21T11:30:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.783+0000 7fde34a00640 1 --2- 172.21.15.47:0/3471195617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2c580 0x5646e0e3aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.783+0000 7fde2fe00640 1 --2- 172.21.15.47:0/3471195617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2a9f0 0x5646e0e375e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fde2002c240 tx=0x7fde20005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10f89bf29faa7ea8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.783+0000 7fde19600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.783+0000 7fde19600640 1 -- 172.21.15.47:0/3471195617 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2005d060 con 0x5646e0e2a9f0 2024-03-21T11:30:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.784+0000 7fde19600640 1 -- 172.21.15.47:0/3471195617 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fde200037f0 con 0x5646e0e2a9f0 2024-03-21T11:30:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.784+0000 7fde39836740 1 -- 172.21.15.47:0/3471195617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde1c0ccc90 con 0x5646e0e2a9f0 2024-03-21T11:30:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.784+0000 7fde19600640 1 -- 172.21.15.47:0/3471195617 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde200039f0 con 0x5646e0e2a9f0 2024-03-21T11:30:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.784+0000 7fde39836740 10 client.27979 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.784+0000 7fde39836740 1 -- 172.21.15.47:0/3471195617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5646e0e3c3c0 con 0x5646e0e2a9f0 2024-03-21T11:30:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.784+0000 7fde19600640 1 -- 172.21.15.47:0/3471195617 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde1c0ccc90 con 0x5646e0e2a9f0 2024-03-21T11:30:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.784+0000 7fde19600640 1 -- 172.21.15.47:0/3471195617 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fde2004b6f0 con 0x5646e0e2a9f0 2024-03-21T11:30:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.784+0000 7fde39836740 10 client.27979 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.784+0000 7fde39836740 10 client.27979 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:30:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.784+0000 7fde39836740 20 client.27979 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.784+0000 7fde39836740 1 --2- 172.21.15.47:0/3471195617 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5646e0e9c570 0x5646e0e9e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.784+0000 7fde39836740 4 client.27979 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:30:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.784+0000 7fde39836740 1 -- 172.21.15.47:0/3471195617 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5646e0d14500 con 0x5646e0e9c570 2024-03-21T11:30:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.785+0000 7fde2f400640 1 --2- 172.21.15.47:0/3471195617 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5646e0e9c570 0x5646e0e9e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.785+0000 7fde19600640 10 client.27979 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:30:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.785+0000 7fde2f400640 1 --2- 172.21.15.47:0/3471195617 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5646e0e9c570 0x5646e0e9e990 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bd290896ad4d47f server_cookie=34a1be6dfb340f15 in_seq=0 out_seq=0 2024-03-21T11:30:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.785+0000 7fde19600640 1 -- 172.21.15.47:0/3471195617 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5646e0d14500 con 0x5646e0e9c570 2024-03-21T11:30:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.785+0000 7fde19600640 10 client.27979 handle_command_reply: tid=0 2024-03-21T11:30:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.785+0000 7fde39836740 1 client.27979 shutdown 2024-03-21T11:30:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.785+0000 7fde39836740 1 -- 172.21.15.47:0/3471195617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2a9f0 msgr2=0x5646e0e375e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.785+0000 7fde39836740 1 --2- 172.21.15.47:0/3471195617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2a9f0 0x5646e0e375e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fde2002c240 tx=0x7fde20005fd0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.786+0000 7fde39836740 1 -- 172.21.15.47:0/3471195617 shutdown_connections 2024-03-21T11:30:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.786+0000 7fde39836740 1 --2- 172.21.15.47:0/3471195617 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5646e0e9c570 0x5646e0e9e990 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.786+0000 7fde39836740 1 --2- 172.21.15.47:0/3471195617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646e0e2c580 0x5646e0e3aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.786+0000 7fde39836740 1 --2- 172.21.15.47:0/3471195617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646e0e2bbc0 0x5646e0e37b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.786+0000 7fde39836740 1 --2- 172.21.15.47:0/3471195617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5646e0e2a9f0 0x5646e0e375e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.786+0000 7fde39836740 1 -- 172.21.15.47:0/3471195617 >> 172.21.15.47:0/3471195617 conn(0x5646e0e25de0 msgr2=0x5646e0e29040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.786+0000 7fde19600640 0 client.27979 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:30:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.786+0000 7fde39836740 1 -- 172.21.15.47:0/3471195617 shutdown_connections 2024-03-21T11:30:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.786+0000 7fde39836740 1 -- 172.21.15.47:0/3471195617 wait complete. 2024-03-21T11:30:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.786+0000 7fde39836740 20 client.27979 trim_cache size 0 max 16384 2024-03-21T11:30:38.792 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:30:38.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.787+0000 7fde35400640 1 -- 172.21.15.47:0/2450342394 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fde140a2eb0 msgr2=0x7fde140a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.787+0000 7fde35400640 1 --2- 172.21.15.47:0/2450342394 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fde140a2eb0 0x7fde140a5370 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fde28011760 tx=0x7fde28005fd0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.787+0000 7fde35400640 1 -- 172.21.15.47:0/2450342394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde301049b0 msgr2=0x7fde30100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.787+0000 7fde35400640 1 --2- 172.21.15.47:0/2450342394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde301049b0 0x7fde30100030 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fde1c005fd0 tx=0x7fde1c00dde0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.787+0000 7fde35400640 1 -- 172.21.15.47:0/2450342394 shutdown_connections 2024-03-21T11:30:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.787+0000 7fde35400640 1 --2- 172.21.15.47:0/2450342394 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fde140a2eb0 0x7fde140a5370 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.787+0000 7fde35400640 1 --2- 172.21.15.47:0/2450342394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde301072e0 0x7fde30100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.787+0000 7fde35400640 1 --2- 172.21.15.47:0/2450342394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde301049b0 0x7fde30100030 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.787+0000 7fde35400640 1 --2- 172.21.15.47:0/2450342394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde30102080 0x7fde301019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.787+0000 7fde35400640 1 -- 172.21.15.47:0/2450342394 >> 172.21.15.47:0/2450342394 conn(0x7fde300fba10 msgr2=0x7fde301029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.787+0000 7fde35400640 1 -- 172.21.15.47:0/2450342394 shutdown_connections 2024-03-21T11:30:38.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:38.787+0000 7fde35400640 1 -- 172.21.15.47:0/2450342394 wait complete. 2024-03-21T11:30:38.805 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:30:38.806 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:30:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:38 smithi047 ceph-mon[31351]: pgmap v619: 129 pgs: 129 active+clean; 16 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 2.0 MiB/s wr, 279 op/s 2024-03-21T11:30:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.069+0000 7f3b3fe00640 1 -- 172.21.15.47:0/900872329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b40101580 msgr2=0x7f3b401019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.069+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/900872329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b40101580 0x7f3b401019a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009900 tx=0x7f3b3002bc30 comp rx=0 tx=0).stop 2024-03-21T11:30:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.069+0000 7f3b3fe00640 1 -- 172.21.15.47:0/900872329 shutdown_connections 2024-03-21T11:30:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.069+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/900872329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b40103130 0x7f3b401099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.069+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/900872329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b40102770 0x7f3b40102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.069+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/900872329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b40101580 0x7f3b401019a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.069+0000 7f3b3fe00640 1 -- 172.21.15.47:0/900872329 >> 172.21.15.47:0/900872329 conn(0x7f3b400fcb10 msgr2=0x7f3b400fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.069+0000 7f3b3fe00640 1 -- 172.21.15.47:0/900872329 shutdown_connections 2024-03-21T11:30:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.069+0000 7f3b3fe00640 1 -- 172.21.15.47:0/900872329 wait complete. 2024-03-21T11:30:39.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.071+0000 7f3b3fe00640 1 Processor -- start 2024-03-21T11:30:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.071+0000 7f3b3fe00640 1 -- start start 2024-03-21T11:30:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.071+0000 7f3b3fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b40101580 0x7f3b40197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.071+0000 7f3b3fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b40102770 0x7f3b40197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.071+0000 7f3b3fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b40103130 0x7f3b4019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.071+0000 7f3b3fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4010d220 con 0x7f3b40102770 2024-03-21T11:30:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.071+0000 7f3b3fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4010d0a0 con 0x7f3b40103130 2024-03-21T11:30:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.071+0000 7f3b3fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4010d3a0 con 0x7f3b40101580 2024-03-21T11:30:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.072+0000 7f3b3ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b40101580 0x7f3b40197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.072+0000 7f3b3ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b40101580 0x7f3b40197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51808/0 (socket says 172.21.15.47:51808) 2024-03-21T11:30:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.072+0000 7f3b3ea00640 1 -- 172.21.15.47:0/2360033917 learned_addr learned my addr 172.21.15.47:0/2360033917 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.072+0000 7f3b36000640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b40102770 0x7f3b40197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.072+0000 7f3b3f400640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b40103130 0x7f3b4019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.072+0000 7f3b3ea00640 1 -- 172.21.15.47:0/2360033917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b40103130 msgr2=0x7f3b4019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.072+0000 7f3b3ea00640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b40103130 0x7f3b4019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.072+0000 7f3b3ea00640 1 -- 172.21.15.47:0/2360033917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b40102770 msgr2=0x7f3b40197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.072+0000 7f3b3ea00640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b40102770 0x7f3b40197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.072+0000 7f3b3ea00640 1 -- 172.21.15.47:0/2360033917 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4019f490 con 0x7f3b40101580 2024-03-21T11:30:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.072+0000 7f3b36000640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b40102770 0x7f3b40197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.072+0000 7f3b3ea00640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b40101580 0x7f3b40197870 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3b30000c00 tx=0x7f3b3002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.073+0000 7f3b37e00640 1 -- 172.21.15.47:0/2360033917 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b30043020 con 0x7f3b40101580 2024-03-21T11:30:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.073+0000 7f3b3fe00640 1 -- 172.21.15.47:0/2360033917 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4019f720 con 0x7f3b40101580 2024-03-21T11:30:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.073+0000 7f3b37e00640 1 -- 172.21.15.47:0/2360033917 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b300054e0 con 0x7f3b40101580 2024-03-21T11:30:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.073+0000 7f3b37e00640 1 -- 172.21.15.47:0/2360033917 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b30003c40 con 0x7f3b40101580 2024-03-21T11:30:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.073+0000 7f3b3fe00640 1 -- 172.21.15.47:0/2360033917 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4019fc70 con 0x7f3b40101580 2024-03-21T11:30:39.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.076+0000 7f3b48416740 1 Processor -- start 2024-03-21T11:30:39.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.077+0000 7f3b48416740 1 -- start start 2024-03-21T11:30:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.077+0000 7f3b37e00640 1 -- 172.21.15.47:0/2360033917 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3b30046020 con 0x7f3b40101580 2024-03-21T11:30:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.077+0000 7f3b48416740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.077+0000 7f3b48416740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a7bc0 0x5615de6a8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.077+0000 7f3b48416740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a8580 0x5615de6aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.077+0000 7f3b48416740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615de60b2c0 con 0x5615de6a7bc0 2024-03-21T11:30:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.077+0000 7f3b48416740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615de5b3c90 con 0x5615de6a69f0 2024-03-21T11:30:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.077+0000 7f3b48416740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615de607360 con 0x5615de6a8580 2024-03-21T11:30:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.077+0000 7f3b3f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.077+0000 7f3b3f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41104/0 (socket says 172.21.15.47:41104) 2024-03-21T11:30:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.077+0000 7f3b3f400640 1 -- 172.21.15.47:0/219530270 learned_addr learned my addr 172.21.15.47:0/219530270 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.077+0000 7f3b36000640 1 --2- 172.21.15.47:0/219530270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a8580 0x5615de6aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.077+0000 7f3b3ea00640 1 --2- 172.21.15.47:0/219530270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a7bc0 0x5615de6a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.078+0000 7f3b3f400640 1 -- 172.21.15.47:0/219530270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a8580 msgr2=0x5615de6aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:39.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.078+0000 7f3b3f400640 1 --2- 172.21.15.47:0/219530270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a8580 0x5615de6aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.078+0000 7f3b3f400640 1 -- 172.21.15.47:0/219530270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a7bc0 msgr2=0x5615de6a8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:39.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.078+0000 7f3b3f400640 1 --2- 172.21.15.47:0/219530270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a7bc0 0x5615de6a8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.078+0000 7f3b3f400640 1 -- 172.21.15.47:0/219530270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615de5d8080 con 0x5615de6a69f0 2024-03-21T11:30:39.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.078+0000 7f3b36000640 1 --2- 172.21.15.47:0/219530270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a8580 0x5615de6aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:39.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.078+0000 7f3b3f400640 1 --2- 172.21.15.47:0/219530270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6a6e10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009fd0 tx=0x7f3b380118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0c3345a5c3ae8b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:39.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.078+0000 7f3b2f400640 1 -- 172.21.15.47:0/219530270 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b38004d20 con 0x5615de6a69f0 2024-03-21T11:30:39.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.078+0000 7f3b2f400640 1 -- 172.21.15.47:0/219530270 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b38004050 con 0x5615de6a69f0 2024-03-21T11:30:39.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.078+0000 7f3b2f400640 1 -- 172.21.15.47:0/219530270 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b38005240 con 0x5615de6a69f0 2024-03-21T11:30:39.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.079+0000 7f3b37e00640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b240a2f20 0x7f3b240a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.079+0000 7f3b37e00640 1 -- 172.21.15.47:0/2360033917 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3b300f1870 con 0x7f3b40101580 2024-03-21T11:30:39.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.079+0000 7f3b3ea00640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b240a2f20 0x7f3b240a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.079+0000 7f3b48416740 1 -- 172.21.15.47:0/219530270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 msgr2=0x5615de6a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:39.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.079+0000 7f3b48416740 1 --2- 172.21.15.47:0/219530270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6a6e10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009fd0 tx=0x7f3b380118d0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.079+0000 7f3b48416740 1 -- 172.21.15.47:0/219530270 shutdown_connections 2024-03-21T11:30:39.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.079+0000 7f3b48416740 1 --2- 172.21.15.47:0/219530270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a8580 0x5615de6aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.079+0000 7f3b48416740 1 --2- 172.21.15.47:0/219530270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a7bc0 0x5615de6a8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.079+0000 7f3b48416740 1 --2- 172.21.15.47:0/219530270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6a6e10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.079+0000 7f3b48416740 1 -- 172.21.15.47:0/219530270 >> 172.21.15.47:0/219530270 conn(0x5615de6a1de0 msgr2=0x5615de6a21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.079+0000 7f3b48416740 1 -- 172.21.15.47:0/219530270 shutdown_connections 2024-03-21T11:30:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.079+0000 7f3b48416740 1 -- 172.21.15.47:0/219530270 wait complete. 2024-03-21T11:30:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.080+0000 7f3b3ea00640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b240a2f20 0x7f3b240a53e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3b300ba010 tx=0x7f3b30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.080+0000 7f3b48416740 1 -- start start 2024-03-21T11:30:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.080+0000 7f3b48416740 1 Processor -- start 2024-03-21T11:30:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.080+0000 7f3b48416740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b48416740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a7bc0 0x5615de6b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b48416740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a8580 0x5615de6bc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b48416740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615de6af590 con 0x5615de6a7bc0 2024-03-21T11:30:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b48416740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615de5c70d0 con 0x5615de6a69f0 2024-03-21T11:30:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b48416740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615de6af710 con 0x5615de6a8580 2024-03-21T11:30:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b48416740 10 client.0 _ll_register_callbacks cb 0x5615de5cbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b3f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b3f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41116/0 (socket says 172.21.15.47:41116) 2024-03-21T11:30:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b3f400640 1 -- 172.21.15.47:0/3997069880 learned_addr learned my addr 172.21.15.47:0/3997069880 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b3f400640 1 --2- 172.21.15.47:0/3997069880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a8580 0x5615de6bc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b36000640 1 --2- 172.21.15.47:0/3997069880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a7bc0 0x5615de6b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b3f400640 1 -- 172.21.15.47:0/3997069880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a8580 msgr2=0x5615de6bc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b3f400640 1 --2- 172.21.15.47:0/3997069880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a8580 0x5615de6bc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b3f400640 1 -- 172.21.15.47:0/3997069880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a7bc0 msgr2=0x5615de6b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b3f400640 1 --2- 172.21.15.47:0/3997069880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a7bc0 0x5615de6b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.081+0000 7f3b3f400640 1 -- 172.21.15.47:0/3997069880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615de31f3f0 con 0x5615de6a69f0 2024-03-21T11:30:39.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.082+0000 7f3b3f400640 1 --2- 172.21.15.47:0/3997069880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6b5050 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3b38003e70 tx=0x7f3b38000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e08ea3e1e73cd28 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.082+0000 7f3b23e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.082+0000 7f3b23e00640 1 -- 172.21.15.47:0/3997069880 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b380045c0 con 0x5615de6a69f0 2024-03-21T11:30:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.082+0000 7f3b23e00640 1 -- 172.21.15.47:0/3997069880 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3b38004c50 con 0x5615de6a69f0 2024-03-21T11:30:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.082+0000 7f3b48416740 1 -- 172.21.15.47:0/3997069880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b38004050 con 0x5615de6a69f0 2024-03-21T11:30:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.082+0000 7f3b23e00640 1 -- 172.21.15.47:0/3997069880 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b38036050 con 0x5615de6a69f0 2024-03-21T11:30:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.082+0000 7f3b48416740 10 client.27974 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.082+0000 7f3b48416740 1 -- 172.21.15.47:0/3997069880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5615de3e97d0 con 0x5615de6a69f0 2024-03-21T11:30:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.083+0000 7f3b23e00640 1 -- 172.21.15.47:0/3997069880 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b38004050 con 0x5615de6a69f0 2024-03-21T11:30:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.083+0000 7f3b23e00640 1 -- 172.21.15.47:0/3997069880 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3b38028570 con 0x5615de6a69f0 2024-03-21T11:30:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.083+0000 7f3b48416740 10 client.27974 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.083+0000 7f3b48416740 10 client.27974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.083+0000 7f3b48416740 20 client.27974 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.083+0000 7f3b48416740 1 --2- 172.21.15.47:0/3997069880 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615de6c07e0 0x5615de6c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.083+0000 7f3b48416740 4 client.27974 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.083+0000 7f3b48416740 1 -- 172.21.15.47:0/3997069880 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615de618800 con 0x5615de6c07e0 2024-03-21T11:30:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.083+0000 7f3b36000640 1 --2- 172.21.15.47:0/3997069880 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615de6c07e0 0x5615de6c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.084+0000 7f3b36000640 1 --2- 172.21.15.47:0/3997069880 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615de6c07e0 0x5615de6c2bd0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd24ef81b33fe2d9 server_cookie=29548fa2b7f25a2a in_seq=0 out_seq=0 2024-03-21T11:30:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.084+0000 7f3b23e00640 10 client.27974 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b23e00640 1 -- 172.21.15.47:0/3997069880 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5615de618800 con 0x5615de6c07e0 2024-03-21T11:30:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b23e00640 10 client.27974 handle_command_reply: tid=0 2024-03-21T11:30:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b48416740 1 client.27974 shutdown 2024-03-21T11:30:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b48416740 1 -- 172.21.15.47:0/3997069880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 msgr2=0x5615de6b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:39.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b48416740 1 --2- 172.21.15.47:0/3997069880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6b5050 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3b38003e70 tx=0x7f3b38000f40 comp rx=0 tx=0).stop 2024-03-21T11:30:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b48416740 1 -- 172.21.15.47:0/3997069880 shutdown_connections 2024-03-21T11:30:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b48416740 1 --2- 172.21.15.47:0/3997069880 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615de6c07e0 0x5615de6c2bd0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b48416740 1 --2- 172.21.15.47:0/3997069880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a8580 0x5615de6bc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b48416740 1 --2- 172.21.15.47:0/3997069880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a7bc0 0x5615de6b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b48416740 1 --2- 172.21.15.47:0/3997069880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6b5050 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b23e00640 0 client.27974 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b48416740 1 -- 172.21.15.47:0/3997069880 >> 172.21.15.47:0/3997069880 conn(0x5615de6a1de0 msgr2=0x5615de6a1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b48416740 1 -- 172.21.15.47:0/3997069880 shutdown_connections 2024-03-21T11:30:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b48416740 1 -- 172.21.15.47:0/3997069880 wait complete. 2024-03-21T11:30:39.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.085+0000 7f3b48416740 20 client.27974 trim_cache size 0 max 16384 2024-03-21T11:30:39.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.100+0000 7f3b48416740 1 Processor -- start 2024-03-21T11:30:39.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b48416740 1 -- start start 2024-03-21T11:30:39.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b48416740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b48416740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a7bc0 0x5615de6a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b48416740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a8580 0x5615de6a1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b48416740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b401a7de0 con 0x5615de6a8580 2024-03-21T11:30:39.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b48416740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b401a9760 con 0x5615de6a69f0 2024-03-21T11:30:39.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b48416740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b401a8440 con 0x5615de6a7bc0 2024-03-21T11:30:39.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b36000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a7bc0 0x5615de6a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b36000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a7bc0 0x5615de6a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51832/0 (socket says 172.21.15.47:51832) 2024-03-21T11:30:39.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b36000640 1 -- 172.21.15.47:0/4251654417 learned_addr learned my addr 172.21.15.47:0/4251654417 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:39.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b3f400640 1 --2- 172.21.15.47:0/4251654417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b36000640 1 -- 172.21.15.47:0/4251654417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 msgr2=0x5615de6a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:39.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b36000640 1 --2- 172.21.15.47:0/4251654417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b36000640 1 -- 172.21.15.47:0/4251654417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a8580 msgr2=0x5615de6a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:39.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b36000640 1 --2- 172.21.15.47:0/4251654417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a8580 0x5615de6a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b36000640 1 -- 172.21.15.47:0/4251654417 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615de6b4c60 con 0x5615de6a7bc0 2024-03-21T11:30:39.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.101+0000 7f3b3f400640 1 --2- 172.21.15.47:0/4251654417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:39.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.102+0000 7f3b36000640 1 --2- 172.21.15.47:0/4251654417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a7bc0 0x5615de6a0e10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x5615de6c3440 tx=0x7f3b28005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4a5c6f3ccbdc925 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:39.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.102+0000 7f3b2ea00640 1 -- 172.21.15.47:0/4251654417 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b280200f0 con 0x5615de6a7bc0 2024-03-21T11:30:39.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.102+0000 7f3b2ea00640 1 -- 172.21.15.47:0/4251654417 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b28024020 con 0x5615de6a7bc0 2024-03-21T11:30:39.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.102+0000 7f3b2ea00640 1 -- 172.21.15.47:0/4251654417 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28011b50 con 0x5615de6a7bc0 2024-03-21T11:30:39.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.103+0000 7f3b48416740 1 -- 172.21.15.47:0/4251654417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a7bc0 msgr2=0x5615de6a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:39.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.103+0000 7f3b48416740 1 --2- 172.21.15.47:0/4251654417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a7bc0 0x5615de6a0e10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x5615de6c3440 tx=0x7f3b28005f90 comp rx=0 tx=0).stop 2024-03-21T11:30:39.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.103+0000 7f3b48416740 1 -- 172.21.15.47:0/4251654417 shutdown_connections 2024-03-21T11:30:39.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.103+0000 7f3b48416740 1 --2- 172.21.15.47:0/4251654417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a8580 0x5615de6a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.103+0000 7f3b48416740 1 --2- 172.21.15.47:0/4251654417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a7bc0 0x5615de6a0e10 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.103+0000 7f3b48416740 1 --2- 172.21.15.47:0/4251654417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.103+0000 7f3b48416740 1 -- 172.21.15.47:0/4251654417 >> 172.21.15.47:0/4251654417 conn(0x5615de6a1de0 msgr2=0x5615de6a58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:39.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.103+0000 7f3b48416740 1 -- 172.21.15.47:0/4251654417 shutdown_connections 2024-03-21T11:30:39.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.103+0000 7f3b48416740 1 -- 172.21.15.47:0/4251654417 wait complete. 2024-03-21T11:30:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.104+0000 7f3b48416740 1 -- start start 2024-03-21T11:30:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.104+0000 7f3b48416740 1 Processor -- start 2024-03-21T11:30:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.104+0000 7f3b48416740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.104+0000 7f3b48416740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a7bc0 0x5615de6b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.104+0000 7f3b48416740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a8580 0x5615de6bb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.104+0000 7f3b48416740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b401a9100 con 0x5615de6a8580 2024-03-21T11:30:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.104+0000 7f3b48416740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b401a8aa0 con 0x5615de6a69f0 2024-03-21T11:30:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.104+0000 7f3b48416740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b401a5fa0 con 0x5615de6a7bc0 2024-03-21T11:30:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.104+0000 7f3b48416740 10 client.0 _ll_register_callbacks cb 0x7f3b280095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.105+0000 7f3b36000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a7bc0 0x5615de6b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.105+0000 7f3b36000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a7bc0 0x5615de6b6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51842/0 (socket says 172.21.15.47:51842) 2024-03-21T11:30:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.105+0000 7f3b36000640 1 -- 172.21.15.47:0/2807969337 learned_addr learned my addr 172.21.15.47:0/2807969337 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.105+0000 7f3b3f400640 1 --2- 172.21.15.47:0/2807969337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.105+0000 7f3b3f400640 1 --2- 172.21.15.47:0/2807969337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a8580 0x5615de6bb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.105+0000 7f3b36000640 1 -- 172.21.15.47:0/2807969337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 msgr2=0x5615de6b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:39.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.105+0000 7f3b36000640 1 --2- 172.21.15.47:0/2807969337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.105+0000 7f3b36000640 1 -- 172.21.15.47:0/2807969337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a8580 msgr2=0x5615de6bb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:39.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.105+0000 7f3b36000640 1 --2- 172.21.15.47:0/2807969337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a8580 0x5615de6bb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.105+0000 7f3b36000640 1 -- 172.21.15.47:0/2807969337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615de6bbc10 con 0x5615de6a7bc0 2024-03-21T11:30:39.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.105+0000 7f3b3f400640 1 --2- 172.21.15.47:0/2807969337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a8580 0x5615de6bb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:30:39.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.105+0000 7f3b36000640 1 --2- 172.21.15.47:0/2807969337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a7bc0 0x5615de6b6cc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3b280095a0 tx=0x7f3b280115a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83930ee9c9261e80 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:39.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.105+0000 7f3b23e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:39.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.105+0000 7f3b23e00640 1 -- 172.21.15.47:0/2807969337 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28011ad0 con 0x5615de6a7bc0 2024-03-21T11:30:39.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.106+0000 7f3b48416740 1 -- 172.21.15.47:0/2807969337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b28024020 con 0x5615de6a7bc0 2024-03-21T11:30:39.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.106+0000 7f3b23e00640 1 -- 172.21.15.47:0/2807969337 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3b28011c70 con 0x5615de6a7bc0 2024-03-21T11:30:39.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.106+0000 7f3b23e00640 1 -- 172.21.15.47:0/2807969337 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b28039050 con 0x5615de6a7bc0 2024-03-21T11:30:39.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.106+0000 7f3b48416740 10 client.27997 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:39.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.106+0000 7f3b48416740 1 -- 172.21.15.47:0/2807969337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5615de720b50 con 0x5615de6a7bc0 2024-03-21T11:30:39.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.106+0000 7f3b23e00640 1 -- 172.21.15.47:0/2807969337 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b28024020 con 0x5615de6a7bc0 2024-03-21T11:30:39.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.106+0000 7f3b23e00640 1 -- 172.21.15.47:0/2807969337 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3b28004730 con 0x5615de6a7bc0 2024-03-21T11:30:39.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.106+0000 7f3b48416740 10 client.27997 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:39.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.106+0000 7f3b48416740 10 client.27997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:39.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.106+0000 7f3b48416740 20 client.27997 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:39.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.106+0000 7f3b48416740 1 --2- 172.21.15.47:0/2807969337 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615de7214d0 0x5615de7238f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:39.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.106+0000 7f3b48416740 4 client.27997 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:30:39.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.106+0000 7f3b48416740 1 -- 172.21.15.47:0/2807969337 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5615de612b50 con 0x5615de7214d0 2024-03-21T11:30:39.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.107+0000 7f3b3f400640 1 --2- 172.21.15.47:0/2807969337 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615de7214d0 0x5615de7238f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:39.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.107+0000 7f3b3f400640 1 --2- 172.21.15.47:0/2807969337 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615de7214d0 0x5615de7238f0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8312f3a8965f43f server_cookie=b64ad601fd27aac1 in_seq=0 out_seq=0 2024-03-21T11:30:39.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.107+0000 7f3b23e00640 10 client.27997 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:39.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.108+0000 7f3b23e00640 1 -- 172.21.15.47:0/2807969337 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5615de612b50 con 0x5615de7214d0 2024-03-21T11:30:39.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.108+0000 7f3b23e00640 10 client.27997 handle_command_reply: tid=0 2024-03-21T11:30:39.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.108+0000 7f3b48416740 1 client.27997 shutdown 2024-03-21T11:30:39.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.109+0000 7f3b48416740 1 -- 172.21.15.47:0/2807969337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a7bc0 msgr2=0x5615de6b6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:39.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.109+0000 7f3b48416740 1 --2- 172.21.15.47:0/2807969337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a7bc0 0x5615de6b6cc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3b280095a0 tx=0x7f3b280115a0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.109+0000 7f3b48416740 1 -- 172.21.15.47:0/2807969337 shutdown_connections 2024-03-21T11:30:39.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.109+0000 7f3b48416740 1 --2- 172.21.15.47:0/2807969337 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615de7214d0 0x5615de7238f0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.109+0000 7f3b48416740 1 --2- 172.21.15.47:0/2807969337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615de6a8580 0x5615de6bb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.109+0000 7f3b48416740 1 --2- 172.21.15.47:0/2807969337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615de6a7bc0 0x5615de6b6cc0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.109+0000 7f3b48416740 1 --2- 172.21.15.47:0/2807969337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615de6a69f0 0x5615de6b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.109+0000 7f3b23e00640 0 client.27997 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:39.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.109+0000 7f3b48416740 1 -- 172.21.15.47:0/2807969337 >> 172.21.15.47:0/2807969337 conn(0x5615de6a1de0 msgr2=0x5615de6a08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:39.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.109+0000 7f3b48416740 1 -- 172.21.15.47:0/2807969337 shutdown_connections 2024-03-21T11:30:39.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.109+0000 7f3b48416740 1 -- 172.21.15.47:0/2807969337 wait complete. 2024-03-21T11:30:39.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.109+0000 7f3b48416740 20 client.27997 trim_cache size 0 max 16384 2024-03-21T11:30:39.114 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:30:39.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.110+0000 7f3b3fe00640 1 -- 172.21.15.47:0/2360033917 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b240a2f20 msgr2=0x7f3b240a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:39.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.110+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b240a2f20 0x7f3b240a53e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3b300ba010 tx=0x7f3b30009260 comp rx=0 tx=0).stop 2024-03-21T11:30:39.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.110+0000 7f3b3fe00640 1 -- 172.21.15.47:0/2360033917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b40101580 msgr2=0x7f3b40197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:39.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.110+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b40101580 0x7f3b40197870 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3b30000c00 tx=0x7f3b3002c650 comp rx=0 tx=0).stop 2024-03-21T11:30:39.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.110+0000 7f3b3fe00640 1 -- 172.21.15.47:0/2360033917 shutdown_connections 2024-03-21T11:30:39.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.110+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b240a2f20 0x7f3b240a53e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.110+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b40103130 0x7f3b4019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.110+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b40102770 0x7f3b40197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.110+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/2360033917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b40101580 0x7f3b40197870 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:39.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.110+0000 7f3b3fe00640 1 -- 172.21.15.47:0/2360033917 >> 172.21.15.47:0/2360033917 conn(0x7f3b400fcb10 msgr2=0x7f3b40105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:39.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.110+0000 7f3b3fe00640 1 -- 172.21.15.47:0/2360033917 shutdown_connections 2024-03-21T11:30:39.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:39.110+0000 7f3b3fe00640 1 -- 172.21.15.47:0/2360033917 wait complete. 2024-03-21T11:30:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:38 smithi102 ceph-mon[36748]: pgmap v619: 129 pgs: 129 active+clean; 16 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 2.0 MiB/s wr, 279 op/s 2024-03-21T11:30:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:38 smithi100 ceph-mon[36245]: pgmap v619: 129 pgs: 129 active+clean; 16 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 2.0 MiB/s wr, 279 op/s 2024-03-21T11:30:39.529 INFO:tasks.workunit.client.0.smithi047.stdout: 1 504496 8.30 MB/sec execute 203 sec latency 10.895 ms 2024-03-21T11:30:40.132 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:30:40.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:30:40.132 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:30:40.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.379+0000 7fa28bc00640 1 -- 172.21.15.47:0/314590912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa284101790 msgr2=0x7fa284101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.379+0000 7fa28bc00640 1 --2- 172.21.15.47:0/314590912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa284101790 0x7fa284101c10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa270009900 tx=0x7fa27002bc30 comp rx=0 tx=0).stop 2024-03-21T11:30:40.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.379+0000 7fa28bc00640 1 -- 172.21.15.47:0/314590912 shutdown_connections 2024-03-21T11:30:40.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.379+0000 7fa28bc00640 1 --2- 172.21.15.47:0/314590912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa284102150 0x7fa2841089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.379+0000 7fa28bc00640 1 --2- 172.21.15.47:0/314590912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa284101790 0x7fa284101c10 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.379+0000 7fa28bc00640 1 --2- 172.21.15.47:0/314590912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2841005a0 0x7fa2841009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.379+0000 7fa28bc00640 1 -- 172.21.15.47:0/314590912 >> 172.21.15.47:0/314590912 conn(0x7fa2840fbb30 msgr2=0x7fa2840fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:40.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.379+0000 7fa28bc00640 1 -- 172.21.15.47:0/314590912 shutdown_connections 2024-03-21T11:30:40.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.379+0000 7fa28bc00640 1 -- 172.21.15.47:0/314590912 wait complete. 2024-03-21T11:30:40.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.380+0000 7fa28bc00640 1 Processor -- start 2024-03-21T11:30:40.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.380+0000 7fa28bc00640 1 -- start start 2024-03-21T11:30:40.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.381+0000 7fa28bc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2841005a0 0x7fa284199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.381+0000 7fa28bc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa284101790 0x7fa28419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.381+0000 7fa28bc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa284102150 0x7fa2841a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.381+0000 7fa28bc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28410d350 con 0x7fa284102150 2024-03-21T11:30:40.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.381+0000 7fa28bc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28410d1d0 con 0x7fa284101790 2024-03-21T11:30:40.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.381+0000 7fa28bc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28410d4d0 con 0x7fa2841005a0 2024-03-21T11:30:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.381+0000 7fa283e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa284101790 0x7fa28419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.381+0000 7fa289200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2841005a0 0x7fa284199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.381+0000 7fa289200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2841005a0 0x7fa284199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51868/0 (socket says 172.21.15.47:51868) 2024-03-21T11:30:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.381+0000 7fa289200640 1 -- 172.21.15.47:0/4090121865 learned_addr learned my addr 172.21.15.47:0/4090121865 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.381+0000 7fa283e00640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa284101790 0x7fa28419a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41150/0 (socket says 172.21.15.47:41150) 2024-03-21T11:30:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.381+0000 7fa289c00640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa284102150 0x7fa2841a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.382+0000 7fa283e00640 1 -- 172.21.15.47:0/4090121865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2841005a0 msgr2=0x7fa284199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.382+0000 7fa283e00640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2841005a0 0x7fa284199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.382+0000 7fa283e00640 1 -- 172.21.15.47:0/4090121865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa284102150 msgr2=0x7fa2841a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.382+0000 7fa283e00640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa284102150 0x7fa2841a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.382+0000 7fa283e00640 1 -- 172.21.15.47:0/4090121865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2841a17c0 con 0x7fa284101790 2024-03-21T11:30:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.382+0000 7fa289c00640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa284102150 0x7fa2841a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.382+0000 7fa289200640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2841005a0 0x7fa284199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.382+0000 7fa283e00640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa284101790 0x7fa28419a0e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa270009230 tx=0x7fa27002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.383+0000 7fa281600640 1 -- 172.21.15.47:0/4090121865 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270041070 con 0x7fa284101790 2024-03-21T11:30:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.383+0000 7fa281600640 1 -- 172.21.15.47:0/4090121865 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa270004450 con 0x7fa284101790 2024-03-21T11:30:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.383+0000 7fa28bc00640 1 -- 172.21.15.47:0/4090121865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2841a1a50 con 0x7fa284101790 2024-03-21T11:30:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.383+0000 7fa281600640 1 -- 172.21.15.47:0/4090121865 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270003df0 con 0x7fa284101790 2024-03-21T11:30:40.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.383+0000 7fa28bc00640 1 -- 172.21.15.47:0/4090121865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2841a1f90 con 0x7fa284101790 2024-03-21T11:30:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.386+0000 7fa28e844740 1 Processor -- start 2024-03-21T11:30:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.386+0000 7fa28e844740 1 -- start start 2024-03-21T11:30:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.386+0000 7fa281600640 1 -- 172.21.15.47:0/4090121865 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa270044070 con 0x7fa284101790 2024-03-21T11:30:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa28e844740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e2500a0 0x55a69e2504c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa28e844740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e250a90 0x55a69e255590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa28e844740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e255c00 0x55a69e258090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa28e844740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69e1b5eb0 con 0x55a69e250a90 2024-03-21T11:30:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa28e844740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69e15ea60 con 0x55a69e2500a0 2024-03-21T11:30:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa28e844740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69e1b1f50 con 0x55a69e255c00 2024-03-21T11:30:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa289c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e2500a0 0x55a69e2504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa289c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e2500a0 0x55a69e2504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41152/0 (socket says 172.21.15.47:41152) 2024-03-21T11:30:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa289c00640 1 -- 172.21.15.47:0/4007174023 learned_addr learned my addr 172.21.15.47:0/4007174023 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa283e00640 1 --2- 172.21.15.47:0/4007174023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e255c00 0x55a69e258090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa289200640 1 --2- 172.21.15.47:0/4007174023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e250a90 0x55a69e255590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa289c00640 1 -- 172.21.15.47:0/4007174023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e255c00 msgr2=0x55a69e258090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa289c00640 1 --2- 172.21.15.47:0/4007174023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e255c00 0x55a69e258090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa289c00640 1 -- 172.21.15.47:0/4007174023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e250a90 msgr2=0x55a69e255590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa289c00640 1 --2- 172.21.15.47:0/4007174023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e250a90 0x55a69e255590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa289c00640 1 -- 172.21.15.47:0/4007174023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a69df927e0 con 0x55a69e2500a0 2024-03-21T11:30:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.387+0000 7fa289200640 1 --2- 172.21.15.47:0/4007174023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e250a90 0x55a69e255590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.388+0000 7fa289c00640 1 --2- 172.21.15.47:0/4007174023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e2500a0 0x55a69e2504c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009fd0 tx=0x7fa27c0108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44015f0a5c08e33e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.388+0000 7fa276000640 1 -- 172.21.15.47:0/4007174023 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c011760 con 0x55a69e2500a0 2024-03-21T11:30:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.388+0000 7fa276000640 1 -- 172.21.15.47:0/4007174023 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa27c011de0 con 0x55a69e2500a0 2024-03-21T11:30:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.388+0000 7fa276000640 1 -- 172.21.15.47:0/4007174023 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00e3c0 con 0x55a69e2500a0 2024-03-21T11:30:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.388+0000 7fa281600640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa26c0a2fa0 0x7fa26c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.389+0000 7fa281600640 1 -- 172.21.15.47:0/4090121865 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa2700f1280 con 0x7fa284101790 2024-03-21T11:30:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.389+0000 7fa28e844740 1 -- 172.21.15.47:0/4007174023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e2500a0 msgr2=0x55a69e2504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.389+0000 7fa28e844740 1 --2- 172.21.15.47:0/4007174023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e2500a0 0x55a69e2504c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009fd0 tx=0x7fa27c0108e0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.389+0000 7fa289200640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa26c0a2fa0 0x7fa26c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.389+0000 7fa28e844740 1 -- 172.21.15.47:0/4007174023 shutdown_connections 2024-03-21T11:30:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.389+0000 7fa28e844740 1 --2- 172.21.15.47:0/4007174023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e255c00 0x55a69e258090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.389+0000 7fa28e844740 1 --2- 172.21.15.47:0/4007174023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e250a90 0x55a69e255590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.389+0000 7fa28e844740 1 --2- 172.21.15.47:0/4007174023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e2500a0 0x55a69e2504c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.389+0000 7fa28e844740 1 -- 172.21.15.47:0/4007174023 >> 172.21.15.47:0/4007174023 conn(0x55a69e24b610 msgr2=0x55a69e24ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.389+0000 7fa28e844740 1 -- 172.21.15.47:0/4007174023 shutdown_connections 2024-03-21T11:30:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.389+0000 7fa28e844740 1 -- 172.21.15.47:0/4007174023 wait complete. 2024-03-21T11:30:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.390+0000 7fa28e844740 1 -- start start 2024-03-21T11:30:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.390+0000 7fa289200640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa26c0a2fa0 0x7fa26c0a5460 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55a69e24c580 tx=0x7fa278005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.390+0000 7fa28e844740 1 Processor -- start 2024-03-21T11:30:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.390+0000 7fa28e844740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e2500a0 0x55a69e25a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.390+0000 7fa28e844740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 0x55a69e25a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.390+0000 7fa28e844740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e255c00 0x55a69e261860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.390+0000 7fa28e844740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69e258bb0 con 0x55a69e255c00 2024-03-21T11:30:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.390+0000 7fa28e844740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69e171eb0 con 0x55a69e250a90 2024-03-21T11:30:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.390+0000 7fa28e844740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69e258d30 con 0x55a69e2500a0 2024-03-21T11:30:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.390+0000 7fa28e844740 10 client.0 _ll_register_callbacks cb 0x55a69e166150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.391+0000 7fa289200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e2500a0 0x55a69e25a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.391+0000 7fa289c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e255c00 0x55a69e261860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.391+0000 7fa289200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e2500a0 0x55a69e25a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51882/0 (socket says 172.21.15.47:51882) 2024-03-21T11:30:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.391+0000 7fa289200640 1 -- 172.21.15.47:0/3305497992 learned_addr learned my addr 172.21.15.47:0/3305497992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.391+0000 7fa283e00640 1 --2- 172.21.15.47:0/3305497992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 0x55a69e25a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.391+0000 7fa289200640 1 -- 172.21.15.47:0/3305497992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 msgr2=0x55a69e25a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.391+0000 7fa289200640 1 --2- 172.21.15.47:0/3305497992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 0x55a69e25a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.391+0000 7fa289200640 1 -- 172.21.15.47:0/3305497992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e255c00 msgr2=0x55a69e261860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.391+0000 7fa289200640 1 --2- 172.21.15.47:0/3305497992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e255c00 0x55a69e261860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.391+0000 7fa289200640 1 -- 172.21.15.47:0/3305497992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a69dff2f70 con 0x55a69e2500a0 2024-03-21T11:30:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.391+0000 7fa289200640 1 --2- 172.21.15.47:0/3305497992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e2500a0 0x55a69e25a2e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa27800fbd0 tx=0x7fa278004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb74334285d59c60 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.391+0000 7fa26aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.392+0000 7fa26aa00640 1 -- 172.21.15.47:0/3305497992 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278005020 con 0x55a69e2500a0 2024-03-21T11:30:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.392+0000 7fa28e844740 1 -- 172.21.15.47:0/3305497992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa27c011de0 con 0x55a69e2500a0 2024-03-21T11:30:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.392+0000 7fa26aa00640 1 -- 172.21.15.47:0/3305497992 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa278018030 con 0x55a69e2500a0 2024-03-21T11:30:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.392+0000 7fa26aa00640 1 -- 172.21.15.47:0/3305497992 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa278034050 con 0x55a69e2500a0 2024-03-21T11:30:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.392+0000 7fa28e844740 10 client.28009 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.392+0000 7fa28e844740 1 -- 172.21.15.47:0/3305497992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a69de8a170 con 0x55a69e2500a0 2024-03-21T11:30:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.392+0000 7fa26aa00640 1 -- 172.21.15.47:0/3305497992 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa27c011de0 con 0x55a69e2500a0 2024-03-21T11:30:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.392+0000 7fa26aa00640 1 -- 172.21.15.47:0/3305497992 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa27801e620 con 0x55a69e2500a0 2024-03-21T11:30:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.393+0000 7fa28e844740 10 client.28009 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:40.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.393+0000 7fa28e844740 10 client.28009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.393+0000 7fa28e844740 20 client.28009 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.393+0000 7fa28e844740 1 --2- 172.21.15.47:0/3305497992 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a69e266140 0x55a69e2685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.393+0000 7fa28e844740 4 client.28009 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.393+0000 7fa28e844740 1 -- 172.21.15.47:0/3305497992 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a69e1c33f0 con 0x55a69e266140 2024-03-21T11:30:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.393+0000 7fa283e00640 1 --2- 172.21.15.47:0/3305497992 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a69e266140 0x55a69e2685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.393+0000 7fa283e00640 1 --2- 172.21.15.47:0/3305497992 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a69e266140 0x55a69e2685e0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=803647c2566b5322 server_cookie=d54328bc8345a73b in_seq=0 out_seq=0 2024-03-21T11:30:40.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.393+0000 7fa26aa00640 10 client.28009 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:40.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.394+0000 7fa26aa00640 1 -- 172.21.15.47:0/3305497992 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a69e1c33f0 con 0x55a69e266140 2024-03-21T11:30:40.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.394+0000 7fa26aa00640 10 client.28009 handle_command_reply: tid=0 2024-03-21T11:30:40.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.394+0000 7fa28e844740 1 client.28009 shutdown 2024-03-21T11:30:40.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.395+0000 7fa28e844740 1 -- 172.21.15.47:0/3305497992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e2500a0 msgr2=0x55a69e25a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.395+0000 7fa28e844740 1 --2- 172.21.15.47:0/3305497992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e2500a0 0x55a69e25a2e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa27800fbd0 tx=0x7fa278004730 comp rx=0 tx=0).stop 2024-03-21T11:30:40.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.395+0000 7fa28e844740 1 -- 172.21.15.47:0/3305497992 shutdown_connections 2024-03-21T11:30:40.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.395+0000 7fa28e844740 1 --2- 172.21.15.47:0/3305497992 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a69e266140 0x55a69e2685e0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.395+0000 7fa28e844740 1 --2- 172.21.15.47:0/3305497992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e255c00 0x55a69e261860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.395+0000 7fa28e844740 1 --2- 172.21.15.47:0/3305497992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 0x55a69e25a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.395+0000 7fa28e844740 1 --2- 172.21.15.47:0/3305497992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e2500a0 0x55a69e25a2e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.395+0000 7fa28e844740 1 -- 172.21.15.47:0/3305497992 >> 172.21.15.47:0/3305497992 conn(0x55a69e24b610 msgr2=0x55a69e24a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:40.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.395+0000 7fa26aa00640 0 client.28009 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:40.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.395+0000 7fa28e844740 1 -- 172.21.15.47:0/3305497992 shutdown_connections 2024-03-21T11:30:40.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.395+0000 7fa28e844740 1 -- 172.21.15.47:0/3305497992 wait complete. 2024-03-21T11:30:40.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.395+0000 7fa28e844740 20 client.28009 trim_cache size 0 max 16384 2024-03-21T11:30:40.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.419+0000 7fa28e844740 1 Processor -- start 2024-03-21T11:30:40.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.419+0000 7fa28e844740 1 -- start start 2024-03-21T11:30:40.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.419+0000 7fa28e844740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e2500a0 0x55a69e25a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.419+0000 7fa28e844740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 0x55a69e24ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.419+0000 7fa28e844740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e255c00 0x55a69e251320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.419+0000 7fa28e844740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2841ac280 con 0x55a69e255c00 2024-03-21T11:30:40.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.419+0000 7fa28e844740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2841adc00 con 0x55a69e250a90 2024-03-21T11:30:40.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.419+0000 7fa28e844740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2841ac8e0 con 0x55a69e2500a0 2024-03-21T11:30:40.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.420+0000 7fa289200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e2500a0 0x55a69e25a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.420+0000 7fa289200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e2500a0 0x55a69e25a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51890/0 (socket says 172.21.15.47:51890) 2024-03-21T11:30:40.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.420+0000 7fa289200640 1 -- 172.21.15.47:0/1512133744 learned_addr learned my addr 172.21.15.47:0/1512133744 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:40.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.420+0000 7fa283e00640 1 --2- 172.21.15.47:0/1512133744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 0x55a69e24ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.420+0000 7fa289c00640 1 --2- 172.21.15.47:0/1512133744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e255c00 0x55a69e251320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.420+0000 7fa289200640 1 -- 172.21.15.47:0/1512133744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 msgr2=0x55a69e24ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.420+0000 7fa289200640 1 --2- 172.21.15.47:0/1512133744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 0x55a69e24ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.420+0000 7fa289200640 1 -- 172.21.15.47:0/1512133744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e255c00 msgr2=0x55a69e251320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.420+0000 7fa289200640 1 --2- 172.21.15.47:0/1512133744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e255c00 0x55a69e251320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.420+0000 7fa289200640 1 -- 172.21.15.47:0/1512133744 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a69e24c380 con 0x55a69e2500a0 2024-03-21T11:30:40.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.421+0000 7fa289200640 1 --2- 172.21.15.47:0/1512133744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e2500a0 0x55a69e25a780 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa278009c80 tx=0x7fa278009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a257cc25dadae70d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:40.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.421+0000 7fa275600640 1 -- 172.21.15.47:0/1512133744 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27803d070 con 0x55a69e2500a0 2024-03-21T11:30:40.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.421+0000 7fa275600640 1 -- 172.21.15.47:0/1512133744 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa27801e620 con 0x55a69e2500a0 2024-03-21T11:30:40.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.421+0000 7fa275600640 1 -- 172.21.15.47:0/1512133744 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278005430 con 0x55a69e2500a0 2024-03-21T11:30:40.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.422+0000 7fa28e844740 1 -- 172.21.15.47:0/1512133744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e2500a0 msgr2=0x55a69e25a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.422+0000 7fa28e844740 1 --2- 172.21.15.47:0/1512133744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e2500a0 0x55a69e25a780 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa278009c80 tx=0x7fa278009f30 comp rx=0 tx=0).stop 2024-03-21T11:30:40.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.422+0000 7fa28e844740 1 -- 172.21.15.47:0/1512133744 shutdown_connections 2024-03-21T11:30:40.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.423+0000 7fa28e844740 1 --2- 172.21.15.47:0/1512133744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e255c00 0x55a69e251320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.423+0000 7fa28e844740 1 --2- 172.21.15.47:0/1512133744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 0x55a69e24ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.423+0000 7fa28e844740 1 --2- 172.21.15.47:0/1512133744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e2500a0 0x55a69e25a780 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.423+0000 7fa28e844740 1 -- 172.21.15.47:0/1512133744 >> 172.21.15.47:0/1512133744 conn(0x55a69e24b610 msgr2=0x55a69e24e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.423+0000 7fa28e844740 1 -- 172.21.15.47:0/1512133744 shutdown_connections 2024-03-21T11:30:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.423+0000 7fa28e844740 1 -- 172.21.15.47:0/1512133744 wait complete. 2024-03-21T11:30:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.423+0000 7fa28e844740 1 -- start start 2024-03-21T11:30:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.423+0000 7fa28e844740 1 Processor -- start 2024-03-21T11:30:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.424+0000 7fa28e844740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e2500a0 0x55a69e2607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.424+0000 7fa28e844740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 0x55a69e260d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.424+0000 7fa28e844740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e255c00 0x55a69e25edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.424+0000 7fa28e844740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2841ad5a0 con 0x55a69e2500a0 2024-03-21T11:30:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.424+0000 7fa28e844740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2841acf40 con 0x55a69e250a90 2024-03-21T11:30:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.424+0000 7fa28e844740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2841aa440 con 0x55a69e255c00 2024-03-21T11:30:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.424+0000 7fa28e844740 10 client.0 _ll_register_callbacks cb 0x7fa27800f540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.424+0000 7fa283e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 0x55a69e260d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.425+0000 7fa283e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 0x55a69e260d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41196/0 (socket says 172.21.15.47:41196) 2024-03-21T11:30:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.425+0000 7fa283e00640 1 -- 172.21.15.47:0/3324014003 learned_addr learned my addr 172.21.15.47:0/3324014003 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.425+0000 7fa289200640 1 --2- 172.21.15.47:0/3324014003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e2500a0 0x55a69e2607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.425+0000 7fa289c00640 1 --2- 172.21.15.47:0/3324014003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e255c00 0x55a69e25edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.425+0000 7fa283e00640 1 -- 172.21.15.47:0/3324014003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e255c00 msgr2=0x55a69e25edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.425+0000 7fa283e00640 1 --2- 172.21.15.47:0/3324014003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e255c00 0x55a69e25edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.425+0000 7fa283e00640 1 -- 172.21.15.47:0/3324014003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e2500a0 msgr2=0x55a69e2607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.425+0000 7fa283e00640 1 --2- 172.21.15.47:0/3324014003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e2500a0 0x55a69e2607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.425+0000 7fa283e00640 1 -- 172.21.15.47:0/3324014003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a69e25f430 con 0x55a69e250a90 2024-03-21T11:30:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.425+0000 7fa289c00640 1 --2- 172.21.15.47:0/3324014003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e255c00 0x55a69e25edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.425+0000 7fa283e00640 1 --2- 172.21.15.47:0/3324014003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 0x55a69e260d20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa2700b9f90 tx=0x7fa2700a8040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15f299cc7cd472c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.425+0000 7fa26aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.426+0000 7fa26aa00640 1 -- 172.21.15.47:0/3324014003 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27004b040 con 0x55a69e250a90 2024-03-21T11:30:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.426+0000 7fa26aa00640 1 -- 172.21.15.47:0/3324014003 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa2700f4d30 con 0x55a69e250a90 2024-03-21T11:30:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.426+0000 7fa28e844740 1 -- 172.21.15.47:0/3324014003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa27801e620 con 0x55a69e250a90 2024-03-21T11:30:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.426+0000 7fa26aa00640 1 -- 172.21.15.47:0/3324014003 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2700bd390 con 0x55a69e250a90 2024-03-21T11:30:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.426+0000 7fa28e844740 10 client.27998 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.426+0000 7fa28e844740 1 -- 172.21.15.47:0/3324014003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a69e2cac90 con 0x55a69e250a90 2024-03-21T11:30:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.426+0000 7fa26aa00640 1 -- 172.21.15.47:0/3324014003 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa27801e620 con 0x55a69e250a90 2024-03-21T11:30:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.427+0000 7fa26aa00640 1 -- 172.21.15.47:0/3324014003 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa2700b9c30 con 0x55a69e250a90 2024-03-21T11:30:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.427+0000 7fa28e844740 10 client.27998 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.427+0000 7fa28e844740 10 client.27998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.427+0000 7fa28e844740 20 client.27998 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.427+0000 7fa28e844740 1 --2- 172.21.15.47:0/3324014003 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a69e2cb5a0 0x55a69e2cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.427+0000 7fa28e844740 4 client.27998 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:30:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.427+0000 7fa28e844740 1 -- 172.21.15.47:0/3324014003 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a69e1bd740 con 0x55a69e2cb5a0 2024-03-21T11:30:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.427+0000 7fa289200640 1 --2- 172.21.15.47:0/3324014003 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a69e2cb5a0 0x55a69e2cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.428+0000 7fa289200640 1 --2- 172.21.15.47:0/3324014003 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a69e2cb5a0 0x55a69e2cd9c0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ed3f09bdd7b5a1 server_cookie=38c672967600e08a in_seq=0 out_seq=0 2024-03-21T11:30:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.428+0000 7fa26aa00640 10 client.27998 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.429+0000 7fa26aa00640 1 -- 172.21.15.47:0/3324014003 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a69e1bd740 con 0x55a69e2cb5a0 2024-03-21T11:30:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.429+0000 7fa26aa00640 10 client.27998 handle_command_reply: tid=0 2024-03-21T11:30:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.429+0000 7fa28e844740 1 client.27998 shutdown 2024-03-21T11:30:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.430+0000 7fa28e844740 1 -- 172.21.15.47:0/3324014003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 msgr2=0x55a69e260d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.430+0000 7fa28e844740 1 --2- 172.21.15.47:0/3324014003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 0x55a69e260d20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa2700b9f90 tx=0x7fa2700a8040 comp rx=0 tx=0).stop 2024-03-21T11:30:40.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.430+0000 7fa28e844740 1 -- 172.21.15.47:0/3324014003 shutdown_connections 2024-03-21T11:30:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.430+0000 7fa28e844740 1 --2- 172.21.15.47:0/3324014003 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a69e2cb5a0 0x55a69e2cd9c0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.430+0000 7fa28e844740 1 --2- 172.21.15.47:0/3324014003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a69e255c00 0x55a69e25edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.430+0000 7fa28e844740 1 --2- 172.21.15.47:0/3324014003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a69e250a90 0x55a69e260d20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.430+0000 7fa28e844740 1 --2- 172.21.15.47:0/3324014003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a69e2500a0 0x55a69e2607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.430+0000 7fa28e844740 1 -- 172.21.15.47:0/3324014003 >> 172.21.15.47:0/3324014003 conn(0x55a69e24b610 msgr2=0x55a69e24f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.430+0000 7fa26aa00640 0 client.27998 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.430+0000 7fa28e844740 1 -- 172.21.15.47:0/3324014003 shutdown_connections 2024-03-21T11:30:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.430+0000 7fa28e844740 1 -- 172.21.15.47:0/3324014003 wait complete. 2024-03-21T11:30:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.430+0000 7fa28e844740 20 client.27998 trim_cache size 0 max 16384 2024-03-21T11:30:40.435 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:30:40.435 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:30:40.435 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "2b0acf29-9448-4b7f-aa12-36dc954884b5", 2024-03-21T11:30:40.435 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:30:40.435 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:30:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.431+0000 7fa28bc00640 1 -- 172.21.15.47:0/4090121865 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa26c0a2fa0 msgr2=0x7fa26c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.431+0000 7fa28bc00640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa26c0a2fa0 0x7fa26c0a5460 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55a69e24c580 tx=0x7fa278005f90 comp rx=0 tx=0).stop 2024-03-21T11:30:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.431+0000 7fa28bc00640 1 -- 172.21.15.47:0/4090121865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa284101790 msgr2=0x7fa28419a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.431+0000 7fa28bc00640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa284101790 0x7fa28419a0e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa270009230 tx=0x7fa27002c650 comp rx=0 tx=0).stop 2024-03-21T11:30:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.431+0000 7fa28bc00640 1 -- 172.21.15.47:0/4090121865 shutdown_connections 2024-03-21T11:30:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.432+0000 7fa28bc00640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa26c0a2fa0 0x7fa26c0a5460 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.432+0000 7fa28bc00640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa284102150 0x7fa2841a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.432+0000 7fa28bc00640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa284101790 0x7fa28419a0e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.432+0000 7fa28bc00640 1 --2- 172.21.15.47:0/4090121865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2841005a0 0x7fa284199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.432+0000 7fa28bc00640 1 -- 172.21.15.47:0/4090121865 >> 172.21.15.47:0/4090121865 conn(0x7fa2840fbb30 msgr2=0x7fa28410aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.432+0000 7fa28bc00640 1 -- 172.21.15.47:0/4090121865 shutdown_connections 2024-03-21T11:30:40.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.432+0000 7fa28bc00640 1 -- 172.21.15.47:0/4090121865 wait complete. 2024-03-21T11:30:40.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:30:40.455 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:30:40.529 INFO:tasks.workunit.client.0.smithi047.stdout: 1 505393 8.27 MB/sec execute 204 sec latency 43.340 ms 2024-03-21T11:30:40.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.708+0000 7f7a92c00640 1 -- 172.21.15.47:0/1896161996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a8c0756e0 msgr2=0x7f7a8c073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.708+0000 7f7a92c00640 1 --2- 172.21.15.47:0/1896161996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a8c0756e0 0x7f7a8c073b30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009860 tx=0x7f7a7c02bb70 comp rx=0 tx=0).stop 2024-03-21T11:30:40.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.709+0000 7f7a92c00640 1 -- 172.21.15.47:0/1896161996 shutdown_connections 2024-03-21T11:30:40.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.709+0000 7f7a92c00640 1 --2- 172.21.15.47:0/1896161996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a8c10dc80 0x7f7a8c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.709+0000 7f7a92c00640 1 --2- 172.21.15.47:0/1896161996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a8c074100 0x7f7a8c074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.709+0000 7f7a92c00640 1 --2- 172.21.15.47:0/1896161996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a8c0756e0 0x7f7a8c073b30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.709+0000 7f7a92c00640 1 -- 172.21.15.47:0/1896161996 >> 172.21.15.47:0/1896161996 conn(0x7f7a8c0fcc00 msgr2=0x7f7a8c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:40.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.710+0000 7f7a92c00640 1 -- 172.21.15.47:0/1896161996 shutdown_connections 2024-03-21T11:30:40.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.710+0000 7f7a92c00640 1 -- 172.21.15.47:0/1896161996 wait complete. 2024-03-21T11:30:40.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.711+0000 7f7a92c00640 1 Processor -- start 2024-03-21T11:30:40.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.711+0000 7f7a92c00640 1 -- start start 2024-03-21T11:30:40.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.712+0000 7f7a92c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a8c074100 0x7f7a8c19e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.712+0000 7f7a92c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a8c0756e0 0x7f7a8c19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.712+0000 7f7a92c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a8c10dc80 0x7f7a8c1a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.712+0000 7f7a92c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c113900 con 0x7f7a8c074100 2024-03-21T11:30:40.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.712+0000 7f7a92c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c113780 con 0x7f7a8c10dc80 2024-03-21T11:30:40.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.712+0000 7f7a92c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c113a80 con 0x7f7a8c0756e0 2024-03-21T11:30:40.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.712+0000 7f7a8b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a8c0756e0 0x7f7a8c19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.712+0000 7f7a90c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a8c10dc80 0x7f7a8c1a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.712+0000 7f7a8b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a8c0756e0 0x7f7a8c19e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51924/0 (socket says 172.21.15.47:51924) 2024-03-21T11:30:40.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.712+0000 7f7a8b400640 1 -- 172.21.15.47:0/2980547042 learned_addr learned my addr 172.21.15.47:0/2980547042 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:40.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.713+0000 7f7a8be00640 1 --2- 172.21.15.47:0/2980547042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a8c074100 0x7f7a8c19e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.713+0000 7f7a8b400640 1 -- 172.21.15.47:0/2980547042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a8c10dc80 msgr2=0x7f7a8c1a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.713+0000 7f7a8b400640 1 --2- 172.21.15.47:0/2980547042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a8c10dc80 0x7f7a8c1a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.713+0000 7f7a8b400640 1 -- 172.21.15.47:0/2980547042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a8c074100 msgr2=0x7f7a8c19e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.713+0000 7f7a8b400640 1 --2- 172.21.15.47:0/2980547042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a8c074100 0x7f7a8c19e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.713+0000 7f7a8b400640 1 -- 172.21.15.47:0/2980547042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c06b1f0 con 0x7f7a8c0756e0 2024-03-21T11:30:40.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.714+0000 7f7a8b400640 1 --2- 172.21.15.47:0/2980547042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a8c0756e0 0x7f7a8c19e590 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800e9f0 tx=0x7f7a7800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:40.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.714+0000 7f7a88c00640 1 -- 172.21.15.47:0/2980547042 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800cd80 con 0x7f7a8c0756e0 2024-03-21T11:30:40.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.714+0000 7f7a88c00640 1 -- 172.21.15.47:0/2980547042 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a78010910 con 0x7f7a8c0756e0 2024-03-21T11:30:40.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.714+0000 7f7a88c00640 1 -- 172.21.15.47:0/2980547042 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78018990 con 0x7f7a8c0756e0 2024-03-21T11:30:40.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.714+0000 7f7a92c00640 1 -- 172.21.15.47:0/2980547042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8c06b4e0 con 0x7f7a8c0756e0 2024-03-21T11:30:40.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.714+0000 7f7a92c00640 1 -- 172.21.15.47:0/2980547042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8c06b9e0 con 0x7f7a8c0756e0 2024-03-21T11:30:40.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.720+0000 7f7a88c00640 1 -- 172.21.15.47:0/2980547042 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7a78010ac0 con 0x7f7a8c0756e0 2024-03-21T11:30:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.721+0000 7f7a959d2740 1 Processor -- start 2024-03-21T11:30:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.721+0000 7f7a959d2740 1 -- start start 2024-03-21T11:30:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.722+0000 7f7a959d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdbf82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.722+0000 7f7a959d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbf8870 0x55d1bdbfd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.722+0000 7f7a959d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdbffe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.722+0000 7f7a959d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1bdb5d250 con 0x55d1bdbf8870 2024-03-21T11:30:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.723+0000 7f7a959d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1bdb05a30 con 0x55d1bdbfd9e0 2024-03-21T11:30:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.723+0000 7f7a959d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1bdb592f0 con 0x55d1bdbf7e80 2024-03-21T11:30:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.723+0000 7f7a90c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdbf82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.723+0000 7f7a90c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdbf82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51940/0 (socket says 172.21.15.47:51940) 2024-03-21T11:30:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.723+0000 7f7a90c00640 1 -- 172.21.15.47:0/2239748972 learned_addr learned my addr 172.21.15.47:0/2239748972 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.723+0000 7f7a8be00640 1 --2- 172.21.15.47:0/2239748972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbf8870 0x55d1bdbfd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.723+0000 7f7a8b400640 1 --2- 172.21.15.47:0/2239748972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdbffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.724+0000 7f7a90c00640 1 -- 172.21.15.47:0/2239748972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbfd9e0 msgr2=0x55d1bdbffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.724+0000 7f7a90c00640 1 --2- 172.21.15.47:0/2239748972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdbffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.724+0000 7f7a90c00640 1 -- 172.21.15.47:0/2239748972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbf8870 msgr2=0x55d1bdbfd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.724+0000 7f7a90c00640 1 --2- 172.21.15.47:0/2239748972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbf8870 0x55d1bdbfd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.724+0000 7f7a90c00640 1 -- 172.21.15.47:0/2239748972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1bdaf0690 con 0x55d1bdbf7e80 2024-03-21T11:30:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.724+0000 7f7a8be00640 1 --2- 172.21.15.47:0/2239748972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbf8870 0x55d1bdbfd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.724+0000 7f7a88c00640 1 --2- 172.21.15.47:0/2980547042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7a700a2f20 0x7f7a700a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.724+0000 7f7a8b400640 1 --2- 172.21.15.47:0/2239748972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdbffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.724+0000 7f7a88c00640 1 -- 172.21.15.47:0/2980547042 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7a780cfac0 con 0x7f7a8c0756e0 2024-03-21T11:30:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.725+0000 7f7a8be00640 1 --2- 172.21.15.47:0/2980547042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7a700a2f20 0x7f7a700a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.725+0000 7f7a90c00640 1 --2- 172.21.15.47:0/2239748972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdbf82a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009f00 tx=0x7f7a8400f7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=963dc1be8800e238 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.726+0000 7f7a81200640 1 -- 172.21.15.47:0/2239748972 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8400c620 con 0x55d1bdbf7e80 2024-03-21T11:30:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.726+0000 7f7a81200640 1 -- 172.21.15.47:0/2239748972 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a8400cca0 con 0x55d1bdbf7e80 2024-03-21T11:30:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.726+0000 7f7a81200640 1 -- 172.21.15.47:0/2239748972 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8400e3f0 con 0x55d1bdbf7e80 2024-03-21T11:30:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.727+0000 7f7a8be00640 1 --2- 172.21.15.47:0/2980547042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7a700a2f20 0x7f7a700a53e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009830 tx=0x7f7a7c034ae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.727+0000 7f7a959d2740 1 -- 172.21.15.47:0/2239748972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 msgr2=0x55d1bdbf82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.727+0000 7f7a959d2740 1 --2- 172.21.15.47:0/2239748972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdbf82a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009f00 tx=0x7f7a8400f7d0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.727+0000 7f7a959d2740 1 -- 172.21.15.47:0/2239748972 shutdown_connections 2024-03-21T11:30:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.727+0000 7f7a959d2740 1 --2- 172.21.15.47:0/2239748972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdbffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.727+0000 7f7a959d2740 1 --2- 172.21.15.47:0/2239748972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbf8870 0x55d1bdbfd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.727+0000 7f7a959d2740 1 --2- 172.21.15.47:0/2239748972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdbf82a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.727+0000 7f7a959d2740 1 -- 172.21.15.47:0/2239748972 >> 172.21.15.47:0/2239748972 conn(0x55d1bdbc0200 msgr2=0x55d1bdbf37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.727+0000 7f7a959d2740 1 -- 172.21.15.47:0/2239748972 shutdown_connections 2024-03-21T11:30:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.727+0000 7f7a959d2740 1 -- 172.21.15.47:0/2239748972 wait complete. 2024-03-21T11:30:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.728+0000 7f7a959d2740 1 -- start start 2024-03-21T11:30:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.729+0000 7f7a959d2740 1 Processor -- start 2024-03-21T11:30:40.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.731+0000 7f7a959d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdc01ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.731+0000 7f7a959d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbf8870 0x55d1bdc02420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.731+0000 7f7a959d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdc09460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.731+0000 7f7a959d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1bdc00990 con 0x55d1bdbfd9e0 2024-03-21T11:30:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.731+0000 7f7a959d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1bdb18e80 con 0x55d1bdbf8870 2024-03-21T11:30:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.731+0000 7f7a959d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1bdc00b10 con 0x55d1bdbf7e80 2024-03-21T11:30:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.731+0000 7f7a959d2740 10 client.0 _ll_register_callbacks cb 0x55d1bdb2b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.731+0000 7f7a8b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbf8870 0x55d1bdc02420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.731+0000 7f7a8b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbf8870 0x55d1bdc02420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41226/0 (socket says 172.21.15.47:41226) 2024-03-21T11:30:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.731+0000 7f7a8b400640 1 -- 172.21.15.47:0/2818348758 learned_addr learned my addr 172.21.15.47:0/2818348758 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.731+0000 7f7a8be00640 1 --2- 172.21.15.47:0/2818348758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdc01ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.732+0000 7f7a90c00640 1 --2- 172.21.15.47:0/2818348758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdc09460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.732+0000 7f7a8b400640 1 -- 172.21.15.47:0/2818348758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 msgr2=0x55d1bdc01ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.732+0000 7f7a8b400640 1 --2- 172.21.15.47:0/2818348758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdc01ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.732+0000 7f7a8b400640 1 -- 172.21.15.47:0/2818348758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbfd9e0 msgr2=0x55d1bdc09460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.732+0000 7f7a8b400640 1 --2- 172.21.15.47:0/2818348758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdc09460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.732+0000 7f7a8b400640 1 -- 172.21.15.47:0/2818348758 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1bd93b7d0 con 0x55d1bdbf8870 2024-03-21T11:30:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.733+0000 7f7a8b400640 1 --2- 172.21.15.47:0/2818348758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbf8870 0x55d1bdc02420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7a780cd1d0 tx=0x7f7a7809bae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe02d5a4e0290762 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.733+0000 7f7a75600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.733+0000 7f7a75600640 1 -- 172.21.15.47:0/2818348758 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a780a4810 con 0x55d1bdbf8870 2024-03-21T11:30:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.733+0000 7f7a959d2740 1 -- 172.21.15.47:0/2818348758 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a8400cca0 con 0x55d1bdbf8870 2024-03-21T11:30:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.733+0000 7f7a75600640 1 -- 172.21.15.47:0/2818348758 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7a780cd2e0 con 0x55d1bdbf8870 2024-03-21T11:30:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.735+0000 7f7a75600640 1 -- 172.21.15.47:0/2818348758 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a780bab70 con 0x55d1bdbf8870 2024-03-21T11:30:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.735+0000 7f7a959d2740 10 client.28010 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.735+0000 7f7a959d2740 1 -- 172.21.15.47:0/2818348758 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d1bd8713f0 con 0x55d1bdbf8870 2024-03-21T11:30:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.736+0000 7f7a75600640 1 -- 172.21.15.47:0/2818348758 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a780b7080 con 0x55d1bdbf8870 2024-03-21T11:30:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.736+0000 7f7a75600640 1 -- 172.21.15.47:0/2818348758 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7a7809b300 con 0x55d1bdbf8870 2024-03-21T11:30:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.736+0000 7f7a959d2740 10 client.28010 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.736+0000 7f7a959d2740 10 client.28010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.736+0000 7f7a959d2740 20 client.28010 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.736+0000 7f7a959d2740 1 --2- 172.21.15.47:0/2818348758 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d1bdc0dc40 0x55d1bdc10060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.736+0000 7f7a959d2740 4 client.28010 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.736+0000 7f7a959d2740 1 -- 172.21.15.47:0/2818348758 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1bdb6a790 con 0x55d1bdc0dc40 2024-03-21T11:30:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.736+0000 7f7a8be00640 1 --2- 172.21.15.47:0/2818348758 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d1bdc0dc40 0x55d1bdc10060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.738+0000 7f7a8be00640 1 --2- 172.21.15.47:0/2818348758 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d1bdc0dc40 0x55d1bdc10060 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=440c73440a966a0e server_cookie=da67bab9368c56a4 in_seq=0 out_seq=0 2024-03-21T11:30:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.738+0000 7f7a75600640 10 client.28010 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:40.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.740+0000 7f7a75600640 1 -- 172.21.15.47:0/2818348758 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d1bdb6a790 con 0x55d1bdc0dc40 2024-03-21T11:30:40.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.740+0000 7f7a75600640 10 client.28010 handle_command_reply: tid=0 2024-03-21T11:30:40.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.740+0000 7f7a959d2740 1 client.28010 shutdown 2024-03-21T11:30:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.740+0000 7f7a959d2740 1 -- 172.21.15.47:0/2818348758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbf8870 msgr2=0x55d1bdc02420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.740+0000 7f7a959d2740 1 --2- 172.21.15.47:0/2818348758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbf8870 0x55d1bdc02420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7a780cd1d0 tx=0x7f7a7809bae0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.740+0000 7f7a959d2740 1 -- 172.21.15.47:0/2818348758 shutdown_connections 2024-03-21T11:30:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.740+0000 7f7a959d2740 1 --2- 172.21.15.47:0/2818348758 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d1bdc0dc40 0x55d1bdc10060 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.740+0000 7f7a959d2740 1 --2- 172.21.15.47:0/2818348758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdc09460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.740+0000 7f7a75600640 0 client.28010 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.741+0000 7f7a959d2740 1 --2- 172.21.15.47:0/2818348758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbf8870 0x55d1bdc02420 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.741+0000 7f7a959d2740 1 --2- 172.21.15.47:0/2818348758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdc01ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.741+0000 7f7a959d2740 1 -- 172.21.15.47:0/2818348758 >> 172.21.15.47:0/2818348758 conn(0x55d1bdbc0200 msgr2=0x55d1bdbf2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.741+0000 7f7a959d2740 1 -- 172.21.15.47:0/2818348758 shutdown_connections 2024-03-21T11:30:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.741+0000 7f7a959d2740 1 -- 172.21.15.47:0/2818348758 wait complete. 2024-03-21T11:30:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.741+0000 7f7a959d2740 20 client.28010 trim_cache size 0 max 16384 2024-03-21T11:30:40.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.782+0000 7f7a959d2740 1 Processor -- start 2024-03-21T11:30:40.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.782+0000 7f7a959d2740 1 -- start start 2024-03-21T11:30:40.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.783+0000 7f7a959d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbf7e80 0x55d1bdbf9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.783+0000 7f7a959d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdc09420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.783+0000 7f7a959d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdc0dc40 0x55d1bdbf9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.783+0000 7f7a959d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c1b9060 con 0x55d1bdbfd9e0 2024-03-21T11:30:40.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.783+0000 7f7a959d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c1ba9e0 con 0x55d1bdbf7e80 2024-03-21T11:30:40.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.783+0000 7f7a959d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c1b96c0 con 0x55d1bdc0dc40 2024-03-21T11:30:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.783+0000 7f7a90c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdc0dc40 0x55d1bdbf9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.783+0000 7f7a90c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdc0dc40 0x55d1bdbf9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51954/0 (socket says 172.21.15.47:51954) 2024-03-21T11:30:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.783+0000 7f7a90c00640 1 -- 172.21.15.47:0/2318736384 learned_addr learned my addr 172.21.15.47:0/2318736384 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.784+0000 7f7a8b400640 1 --2- 172.21.15.47:0/2318736384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdc09420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.784+0000 7f7a8be00640 1 --2- 172.21.15.47:0/2318736384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbf7e80 0x55d1bdbf9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.784+0000 7f7a90c00640 1 -- 172.21.15.47:0/2318736384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbf7e80 msgr2=0x55d1bdbf9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.785+0000 7f7a90c00640 1 --2- 172.21.15.47:0/2318736384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbf7e80 0x55d1bdbf9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.785+0000 7f7a90c00640 1 -- 172.21.15.47:0/2318736384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbfd9e0 msgr2=0x55d1bdc09420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.785+0000 7f7a90c00640 1 --2- 172.21.15.47:0/2318736384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdc09420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.785+0000 7f7a90c00640 1 -- 172.21.15.47:0/2318736384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1bd8310f0 con 0x55d1bdc0dc40 2024-03-21T11:30:40.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.785+0000 7f7a8b400640 1 --2- 172.21.15.47:0/2318736384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdc09420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:40.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.785+0000 7f7a8be00640 1 --2- 172.21.15.47:0/2318736384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbf7e80 0x55d1bdbf9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:30:40.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.787+0000 7f7a90c00640 1 --2- 172.21.15.47:0/2318736384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdc0dc40 0x55d1bdbf9640 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009ea0 tx=0x7f7a8400cf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ab0219ed8207f2d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:40.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.787+0000 7f7a77e00640 1 -- 172.21.15.47:0/2318736384 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a84009b20 con 0x55d1bdc0dc40 2024-03-21T11:30:40.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.787+0000 7f7a77e00640 1 -- 172.21.15.47:0/2318736384 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a84026070 con 0x55d1bdc0dc40 2024-03-21T11:30:40.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.787+0000 7f7a77e00640 1 -- 172.21.15.47:0/2318736384 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a84005210 con 0x55d1bdc0dc40 2024-03-21T11:30:40.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.789+0000 7f7a959d2740 1 -- 172.21.15.47:0/2318736384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdc0dc40 msgr2=0x55d1bdbf9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.789+0000 7f7a959d2740 1 --2- 172.21.15.47:0/2318736384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdc0dc40 0x55d1bdbf9640 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009ea0 tx=0x7f7a8400cf00 comp rx=0 tx=0).stop 2024-03-21T11:30:40.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.790+0000 7f7a959d2740 1 -- 172.21.15.47:0/2318736384 shutdown_connections 2024-03-21T11:30:40.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.790+0000 7f7a959d2740 1 --2- 172.21.15.47:0/2318736384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdc0dc40 0x55d1bdbf9640 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.790+0000 7f7a959d2740 1 --2- 172.21.15.47:0/2318736384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdc09420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.790+0000 7f7a959d2740 1 --2- 172.21.15.47:0/2318736384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbf7e80 0x55d1bdbf9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.790+0000 7f7a959d2740 1 -- 172.21.15.47:0/2318736384 >> 172.21.15.47:0/2318736384 conn(0x55d1bdbc0200 msgr2=0x55d1bdbf72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:40.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.790+0000 7f7a959d2740 1 -- 172.21.15.47:0/2318736384 shutdown_connections 2024-03-21T11:30:40.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.790+0000 7f7a959d2740 1 -- 172.21.15.47:0/2318736384 wait complete. 2024-03-21T11:30:40.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.791+0000 7f7a959d2740 1 -- start start 2024-03-21T11:30:40.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.792+0000 7f7a959d2740 1 Processor -- start 2024-03-21T11:30:40.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.793+0000 7f7a959d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdc08c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.793+0000 7f7a959d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdc091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.793+0000 7f7a959d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdc0dc40 0x55d1bdc07360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.793+0000 7f7a959d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c1ba380 con 0x55d1bdc0dc40 2024-03-21T11:30:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.793+0000 7f7a959d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c1b9d20 con 0x55d1bdbfd9e0 2024-03-21T11:30:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.793+0000 7f7a959d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0736e0 con 0x55d1bdbf7e80 2024-03-21T11:30:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.793+0000 7f7a8be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdc08c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.793+0000 7f7a959d2740 10 client.0 _ll_register_callbacks cb 0x7f7a7809bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.793+0000 7f7a8be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdc08c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51966/0 (socket says 172.21.15.47:51966) 2024-03-21T11:30:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.793+0000 7f7a8be00640 1 -- 172.21.15.47:0/394248919 learned_addr learned my addr 172.21.15.47:0/394248919 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.793+0000 7f7a8b400640 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdc091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.794+0000 7f7a8be00640 1 -- 172.21.15.47:0/394248919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbfd9e0 msgr2=0x55d1bdc091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.794+0000 7f7a8be00640 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdc091d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.794+0000 7f7a8be00640 1 -- 172.21.15.47:0/394248919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdc0dc40 msgr2=0x55d1bdc07360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.794+0000 7f7a90c00640 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdc0dc40 0x55d1bdc07360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.794+0000 7f7a8be00640 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdc0dc40 0x55d1bdc07360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.794+0000 7f7a8be00640 1 -- 172.21.15.47:0/394248919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1bdc07a50 con 0x55d1bdbf7e80 2024-03-21T11:30:40.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.794+0000 7f7a8be00640 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdc08c90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c02c180 tx=0x7f7a7c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e6e08cb3a4f1e9a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:40.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.794+0000 7f7a8b400640 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdc091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:40.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.794+0000 7f7a75600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:40.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.795+0000 7f7a75600640 1 -- 172.21.15.47:0/394248919 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c057060 con 0x55d1bdbf7e80 2024-03-21T11:30:40.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.795+0000 7f7a959d2740 1 -- 172.21.15.47:0/394248919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a84026070 con 0x55d1bdbf7e80 2024-03-21T11:30:40.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.794+0000 7f7a90c00640 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdc0dc40 0x55d1bdc07360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:30:40.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.795+0000 7f7a75600640 1 -- 172.21.15.47:0/394248919 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7a7c02c510 con 0x55d1bdbf7e80 2024-03-21T11:30:40.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.796+0000 7f7a75600640 1 -- 172.21.15.47:0/394248919 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a7c05e050 con 0x55d1bdbf7e80 2024-03-21T11:30:40.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.796+0000 7f7a959d2740 10 client.28033 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:40.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.796+0000 7f7a959d2740 1 -- 172.21.15.47:0/394248919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d1bdc69ea0 con 0x55d1bdbf7e80 2024-03-21T11:30:40.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.797+0000 7f7a75600640 1 -- 172.21.15.47:0/394248919 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a84026070 con 0x55d1bdbf7e80 2024-03-21T11:30:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.797+0000 7f7a75600640 1 -- 172.21.15.47:0/394248919 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7a7c049040 con 0x55d1bdbf7e80 2024-03-21T11:30:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.797+0000 7f7a959d2740 10 client.28033 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.797+0000 7f7a959d2740 10 client.28033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.797+0000 7f7a959d2740 20 client.28033 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.797+0000 7f7a959d2740 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d1bdc6a800 0x55d1bdc6cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.797+0000 7f7a959d2740 4 client.28033 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:30:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.797+0000 7f7a959d2740 1 -- 172.21.15.47:0/394248919 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d1bdb64ae0 con 0x55d1bdc6a800 2024-03-21T11:30:40.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.798+0000 7f7a8b400640 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d1bdc6a800 0x55d1bdc6cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:40.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.800+0000 7f7a8b400640 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d1bdc6a800 0x55d1bdc6cc20 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b547ea1bb398c73c server_cookie=1bfd7245c8bd3899 in_seq=0 out_seq=0 2024-03-21T11:30:40.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.800+0000 7f7a75600640 10 client.28033 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:40.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.803+0000 7f7a75600640 1 -- 172.21.15.47:0/394248919 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d1bdb64ae0 con 0x55d1bdc6a800 2024-03-21T11:30:40.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.803+0000 7f7a75600640 10 client.28033 handle_command_reply: tid=0 2024-03-21T11:30:40.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.803+0000 7f7a959d2740 1 client.28033 shutdown 2024-03-21T11:30:40.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.804+0000 7f7a959d2740 1 -- 172.21.15.47:0/394248919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 msgr2=0x55d1bdc08c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.804+0000 7f7a959d2740 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdc08c90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c02c180 tx=0x7f7a7c005f90 comp rx=0 tx=0).stop 2024-03-21T11:30:40.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.804+0000 7f7a959d2740 1 -- 172.21.15.47:0/394248919 shutdown_connections 2024-03-21T11:30:40.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.804+0000 7f7a959d2740 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d1bdc6a800 0x55d1bdc6cc20 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.804+0000 7f7a959d2740 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1bdc0dc40 0x55d1bdc07360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.804+0000 7f7a959d2740 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1bdbfd9e0 0x55d1bdc091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.805+0000 7f7a959d2740 1 --2- 172.21.15.47:0/394248919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1bdbf7e80 0x55d1bdc08c90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.805+0000 7f7a75600640 0 client.28033 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:40.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.805+0000 7f7a959d2740 1 -- 172.21.15.47:0/394248919 >> 172.21.15.47:0/394248919 conn(0x55d1bdbc0200 msgr2=0x55d1bdbf2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:40.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.806+0000 7f7a959d2740 1 -- 172.21.15.47:0/394248919 shutdown_connections 2024-03-21T11:30:40.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.806+0000 7f7a959d2740 1 -- 172.21.15.47:0/394248919 wait complete. 2024-03-21T11:30:40.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.806+0000 7f7a959d2740 20 client.28033 trim_cache size 0 max 16384 2024-03-21T11:30:40.807 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:30:40.807 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:30:40.807 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:30:40.808 INFO:teuthology.orchestra.run.smithi047.stdout: "2b0acf29-9448-4b7f-aa12-36dc954884b5": { 2024-03-21T11:30:40.808 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:30:40.808 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "2b0acf29-9448-4b7f-aa12-36dc954884b5", 2024-03-21T11:30:40.808 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:30:40.808 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:30:40.808 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:30:40.808 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:30:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.807+0000 7f7a92c00640 1 -- 172.21.15.47:0/2980547042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7a700a2f20 msgr2=0x7f7a700a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.807+0000 7f7a92c00640 1 --2- 172.21.15.47:0/2980547042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7a700a2f20 0x7f7a700a53e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009830 tx=0x7f7a7c034ae0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.807+0000 7f7a92c00640 1 -- 172.21.15.47:0/2980547042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a8c0756e0 msgr2=0x7f7a8c19e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.807+0000 7f7a92c00640 1 --2- 172.21.15.47:0/2980547042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a8c0756e0 0x7f7a8c19e590 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800e9f0 tx=0x7f7a7800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.808+0000 7f7a92c00640 1 -- 172.21.15.47:0/2980547042 shutdown_connections 2024-03-21T11:30:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.808+0000 7f7a92c00640 1 --2- 172.21.15.47:0/2980547042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7a700a2f20 0x7f7a700a53e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.808+0000 7f7a92c00640 1 --2- 172.21.15.47:0/2980547042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a8c10dc80 0x7f7a8c1a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.808+0000 7f7a92c00640 1 --2- 172.21.15.47:0/2980547042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a8c0756e0 0x7f7a8c19e590 secure :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800e9f0 tx=0x7f7a7800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.808+0000 7f7a92c00640 1 --2- 172.21.15.47:0/2980547042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a8c074100 0x7f7a8c19e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:40.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.808+0000 7f7a92c00640 1 -- 172.21.15.47:0/2980547042 >> 172.21.15.47:0/2980547042 conn(0x7f7a8c0fcc00 msgr2=0x7f7a8c101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:40.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.809+0000 7f7a92c00640 1 -- 172.21.15.47:0/2980547042 shutdown_connections 2024-03-21T11:30:40.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:30:40.809+0000 7f7a92c00640 1 -- 172.21.15.47:0/2980547042 wait complete. 2024-03-21T11:30:40.838 INFO:tasks.cephfs.filesystem:scrub status for tag:2b0acf29-9448-4b7f-aa12-36dc954884b5 - {'path': '/', 'tag': '2b0acf29-9448-4b7f-aa12-36dc954884b5', 'options': 'recursive,force'} 2024-03-21T11:30:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:40 smithi102 ceph-mon[36748]: pgmap v620: 129 pgs: 129 active+clean; 18 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 2.3 MiB/s wr, 315 op/s 2024-03-21T11:30:41.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:40 smithi047 ceph-mon[31351]: pgmap v620: 129 pgs: 129 active+clean; 18 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 2.3 MiB/s wr, 315 op/s 2024-03-21T11:30:41.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:40 smithi100 ceph-mon[36245]: pgmap v620: 129 pgs: 129 active+clean; 18 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 2.3 MiB/s wr, 315 op/s 2024-03-21T11:30:41.530 INFO:tasks.workunit.client.0.smithi047.stdout: 1 507367 8.28 MB/sec execute 205 sec latency 21.648 ms 2024-03-21T11:30:42.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:41 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:30:42.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:41 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:30:42.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:41 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:30:42.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:41 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:30:42.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:41 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:30:42.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:41 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:30:42.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:41 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:30:42.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:41 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:30:42.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:41 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:30:42.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:41 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:30:42.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:41 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:30:42.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:41 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:30:42.530 INFO:tasks.workunit.client.0.smithi047.stdout: 1 508590 8.28 MB/sec execute 206 sec latency 10.487 ms 2024-03-21T11:30:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:42 smithi102 ceph-mon[36748]: pgmap v621: 129 pgs: 129 active+clean; 18 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.8 MiB/s wr, 269 op/s 2024-03-21T11:30:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:42 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:30:43.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:42 smithi047 ceph-mon[31351]: pgmap v621: 129 pgs: 129 active+clean; 18 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.8 MiB/s wr, 269 op/s 2024-03-21T11:30:43.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:42 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:30:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:42 smithi100 ceph-mon[36245]: pgmap v621: 129 pgs: 129 active+clean; 18 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.8 MiB/s wr, 269 op/s 2024-03-21T11:30:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:42 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:30:43.530 INFO:tasks.workunit.client.0.smithi047.stdout: 1 510412 8.27 MB/sec execute 207 sec latency 17.017 ms 2024-03-21T11:30:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.364+0000 7f63f0c00640 1 -- 172.21.15.47:0/1459135801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec104ae0 msgr2=0x7f63ec104f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.364+0000 7f63f0c00640 1 --2- 172.21.15.47:0/1459135801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec104ae0 0x7f63ec104f60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009900 tx=0x7f63d402bc40 comp rx=0 tx=0).stop 2024-03-21T11:30:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.364+0000 7f63f0c00640 1 -- 172.21.15.47:0/1459135801 shutdown_connections 2024-03-21T11:30:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.364+0000 7f63f0c00640 1 --2- 172.21.15.47:0/1459135801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec1054a0 0x7f63ec10bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.364+0000 7f63f0c00640 1 --2- 172.21.15.47:0/1459135801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec104ae0 0x7f63ec104f60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.364+0000 7f63f0c00640 1 --2- 172.21.15.47:0/1459135801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ec1038f0 0x7f63ec103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.364+0000 7f63f0c00640 1 -- 172.21.15.47:0/1459135801 >> 172.21.15.47:0/1459135801 conn(0x7f63ec0fee60 msgr2=0x7f63ec1012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.365+0000 7f63f0c00640 1 -- 172.21.15.47:0/1459135801 shutdown_connections 2024-03-21T11:30:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.365+0000 7f63f0c00640 1 -- 172.21.15.47:0/1459135801 wait complete. 2024-03-21T11:30:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.366+0000 7f63f0c00640 1 Processor -- start 2024-03-21T11:30:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.366+0000 7f63f0c00640 1 -- start start 2024-03-21T11:30:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.366+0000 7f63f0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec1038f0 0x7f63ec199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.366+0000 7f63f0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ec104ae0 0x7f63ec19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.366+0000 7f63f0c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec1054a0 0x7f63ec1a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.366+0000 7f63f0c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec10f570 con 0x7f63ec1038f0 2024-03-21T11:30:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.366+0000 7f63f0c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec10f3f0 con 0x7f63ec104ae0 2024-03-21T11:30:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.366+0000 7f63f0c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec10f6f0 con 0x7f63ec1054a0 2024-03-21T11:30:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.366+0000 7f63ea800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec1054a0 0x7f63ec1a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.366+0000 7f63e3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ec104ae0 0x7f63ec19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.366+0000 7f63ea800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec1054a0 0x7f63ec1a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51984/0 (socket says 172.21.15.47:51984) 2024-03-21T11:30:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.367+0000 7f63ea800640 1 -- 172.21.15.47:0/3761694779 learned_addr learned my addr 172.21.15.47:0/3761694779 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.367+0000 7f63e9e00640 1 --2- 172.21.15.47:0/3761694779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec1038f0 0x7f63ec199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.367+0000 7f63ea800640 1 -- 172.21.15.47:0/3761694779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ec104ae0 msgr2=0x7f63ec19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.367+0000 7f63ea800640 1 --2- 172.21.15.47:0/3761694779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ec104ae0 0x7f63ec19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.367+0000 7f63ea800640 1 -- 172.21.15.47:0/3761694779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec1038f0 msgr2=0x7f63ec199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.367+0000 7f63ea800640 1 --2- 172.21.15.47:0/3761694779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec1038f0 0x7f63ec199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.367+0000 7f63ea800640 1 -- 172.21.15.47:0/3761694779 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec1a17d0 con 0x7f63ec1054a0 2024-03-21T11:30:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.367+0000 7f63e3e00640 1 --2- 172.21.15.47:0/3761694779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ec104ae0 0x7f63ec19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.367+0000 7f63ea800640 1 --2- 172.21.15.47:0/3761694779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec1054a0 0x7f63ec1a1160 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009fd0 tx=0x7f63e400b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.368+0000 7f63e2a00640 1 -- 172.21.15.47:0/3761694779 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4017070 con 0x7f63ec1054a0 2024-03-21T11:30:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.368+0000 7f63e2a00640 1 -- 172.21.15.47:0/3761694779 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63e4003e50 con 0x7f63ec1054a0 2024-03-21T11:30:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.368+0000 7f63f0c00640 1 -- 172.21.15.47:0/3761694779 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ec1a1ac0 con 0x7f63ec1054a0 2024-03-21T11:30:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.368+0000 7f63e2a00640 1 -- 172.21.15.47:0/3761694779 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e40106c0 con 0x7f63ec1054a0 2024-03-21T11:30:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.368+0000 7f63f0c00640 1 -- 172.21.15.47:0/3761694779 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ec1a1fe0 con 0x7f63ec1054a0 2024-03-21T11:30:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.372+0000 7f63f398e740 1 Processor -- start 2024-03-21T11:30:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.372+0000 7f63e2a00640 1 -- 172.21.15.47:0/3761694779 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f63e4003970 con 0x7f63ec1054a0 2024-03-21T11:30:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.373+0000 7f63f398e740 1 -- start start 2024-03-21T11:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.373+0000 7f63f398e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e3911e0 0x559e9e391600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.373+0000 7f63f398e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 0x559e9e3966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.373+0000 7f63f398e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e396d40 0x559e9e3991d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.373+0000 7f63f398e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9e3543b0 con 0x559e9e3911e0 2024-03-21T11:30:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.373+0000 7f63f398e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9e2f6830 con 0x559e9e391bd0 2024-03-21T11:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.373+0000 7f63f398e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9e29f5d0 con 0x559e9e396d40 2024-03-21T11:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.373+0000 7f63e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 0x559e9e3966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.373+0000 7f63e3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e3911e0 0x559e9e391600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.373+0000 7f63e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 0x559e9e3966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41278/0 (socket says 172.21.15.47:41278) 2024-03-21T11:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.373+0000 7f63e9e00640 1 -- 172.21.15.47:0/3033384073 learned_addr learned my addr 172.21.15.47:0/3033384073 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.374+0000 7f63e3e00640 1 --2- 172.21.15.47:0/3033384073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e396d40 0x559e9e3991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.374+0000 7f63e9e00640 1 -- 172.21.15.47:0/3033384073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e396d40 msgr2=0x559e9e3991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.374+0000 7f63e9e00640 1 --2- 172.21.15.47:0/3033384073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e396d40 0x559e9e3991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.374+0000 7f63e9e00640 1 -- 172.21.15.47:0/3033384073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e3911e0 msgr2=0x559e9e391600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.374+0000 7f63e9e00640 1 --2- 172.21.15.47:0/3033384073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e3911e0 0x559e9e391600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.374+0000 7f63e9e00640 1 -- 172.21.15.47:0/3033384073 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e9e2b72c0 con 0x559e9e391bd0 2024-03-21T11:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.374+0000 7f63e3e00640 1 --2- 172.21.15.47:0/3033384073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e396d40 0x559e9e3991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:30:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.374+0000 7f63e2a00640 1 --2- 172.21.15.47:0/3761694779 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f63d00a3040 0x7f63d00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.374+0000 7f63e9e00640 1 --2- 172.21.15.47:0/3033384073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 0x559e9e3966d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009ca0 tx=0x7f63dc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aebbdf9525baa07e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.374+0000 7f63e2a00640 1 -- 172.21.15.47:0/3761694779 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f63e40cf370 con 0x7f63ec1054a0 2024-03-21T11:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.374+0000 7f63db000640 1 -- 172.21.15.47:0/3033384073 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc004220 con 0x559e9e391bd0 2024-03-21T11:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.374+0000 7f63e3e00640 1 --2- 172.21.15.47:0/3761694779 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f63d00a3040 0x7f63d00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.374+0000 7f63db000640 1 -- 172.21.15.47:0/3033384073 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63dc0043c0 con 0x559e9e391bd0 2024-03-21T11:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.374+0000 7f63db000640 1 -- 172.21.15.47:0/3033384073 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc019900 con 0x559e9e391bd0 2024-03-21T11:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.375+0000 7f63e3e00640 1 --2- 172.21.15.47:0/3761694779 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f63d00a3040 0x7f63d00a5500 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f63d402c7c0 tx=0x7f63d4005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.375+0000 7f63f398e740 1 -- 172.21.15.47:0/3033384073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 msgr2=0x559e9e3966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.375+0000 7f63f398e740 1 --2- 172.21.15.47:0/3033384073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 0x559e9e3966d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009ca0 tx=0x7f63dc011810 comp rx=0 tx=0).stop 2024-03-21T11:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.375+0000 7f63f398e740 1 -- 172.21.15.47:0/3033384073 shutdown_connections 2024-03-21T11:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.375+0000 7f63f398e740 1 --2- 172.21.15.47:0/3033384073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e396d40 0x559e9e3991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.375+0000 7f63f398e740 1 --2- 172.21.15.47:0/3033384073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 0x559e9e3966d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.375+0000 7f63f398e740 1 --2- 172.21.15.47:0/3033384073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e3911e0 0x559e9e391600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.375+0000 7f63f398e740 1 -- 172.21.15.47:0/3033384073 >> 172.21.15.47:0/3033384073 conn(0x559e9e38c750 msgr2=0x559e9e38cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.376+0000 7f63f398e740 1 -- 172.21.15.47:0/3033384073 shutdown_connections 2024-03-21T11:30:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.376+0000 7f63f398e740 1 -- 172.21.15.47:0/3033384073 wait complete. 2024-03-21T11:30:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.376+0000 7f63f398e740 1 -- start start 2024-03-21T11:30:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.376+0000 7f63f398e740 1 Processor -- start 2024-03-21T11:30:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63f398e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e3911e0 0x559e9e39b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63f398e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e391bd0 0x559e9e39b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63f398e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 0x559e9e3a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63f398e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9e2b2a20 con 0x559e9e396d40 2024-03-21T11:30:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63f398e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9e2f28d0 con 0x559e9e3911e0 2024-03-21T11:30:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63f398e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9e399cf0 con 0x559e9e391bd0 2024-03-21T11:30:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63f398e740 10 client.0 _ll_register_callbacks cb 0x559e9e2ef1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e3911e0 0x559e9e39b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63e3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e391bd0 0x559e9e39b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e3911e0 0x559e9e39b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41294/0 (socket says 172.21.15.47:41294) 2024-03-21T11:30:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63e9e00640 1 -- 172.21.15.47:0/3240098957 learned_addr learned my addr 172.21.15.47:0/3240098957 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63ea800640 1 --2- 172.21.15.47:0/3240098957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 0x559e9e3a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63e9e00640 1 -- 172.21.15.47:0/3240098957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e391bd0 msgr2=0x559e9e39b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63e9e00640 1 --2- 172.21.15.47:0/3240098957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e391bd0 0x559e9e39b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63e9e00640 1 -- 172.21.15.47:0/3240098957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 msgr2=0x559e9e3a28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.377+0000 7f63e9e00640 1 --2- 172.21.15.47:0/3240098957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 0x559e9e3a28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.378+0000 7f63e9e00640 1 -- 172.21.15.47:0/3240098957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e9e00b210 con 0x559e9e3911e0 2024-03-21T11:30:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.378+0000 7f63e9e00640 1 --2- 172.21.15.47:0/3240098957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e3911e0 0x559e9e39b360 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009c70 tx=0x7f63dc003ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f076de645642034 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.378+0000 7f63cf400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.378+0000 7f63cf400640 1 -- 172.21.15.47:0/3240098957 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc028070 con 0x559e9e3911e0 2024-03-21T11:30:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.378+0000 7f63cf400640 1 -- 172.21.15.47:0/3240098957 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f63dc005410 con 0x559e9e3911e0 2024-03-21T11:30:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.378+0000 7f63f398e740 1 -- 172.21.15.47:0/3240098957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63dc0043c0 con 0x559e9e3911e0 2024-03-21T11:30:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.378+0000 7f63cf400640 1 -- 172.21.15.47:0/3240098957 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63dc034050 con 0x559e9e3911e0 2024-03-21T11:30:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.379+0000 7f63f398e740 10 client.28025 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.379+0000 7f63f398e740 1 -- 172.21.15.47:0/3240098957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559e9e0d7620 con 0x559e9e3911e0 2024-03-21T11:30:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.379+0000 7f63cf400640 1 -- 172.21.15.47:0/3240098957 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63dc0043c0 con 0x559e9e3911e0 2024-03-21T11:30:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.379+0000 7f63cf400640 1 -- 172.21.15.47:0/3240098957 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f63dc004ad0 con 0x559e9e3911e0 2024-03-21T11:30:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.379+0000 7f63f398e740 10 client.28025 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.379+0000 7f63f398e740 10 client.28025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.379+0000 7f63f398e740 20 client.28025 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.379+0000 7f63f398e740 1 --2- 172.21.15.47:0/3240098957 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e9e3a6ea0 0x559e9e3a9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.379+0000 7f63f398e740 4 client.28025 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.379+0000 7f63f398e740 1 -- 172.21.15.47:0/3240098957 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e9e303d70 con 0x559e9e3a6ea0 2024-03-21T11:30:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.379+0000 7f63e3e00640 1 --2- 172.21.15.47:0/3240098957 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e9e3a6ea0 0x559e9e3a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.380+0000 7f63e3e00640 1 --2- 172.21.15.47:0/3240098957 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e9e3a6ea0 0x559e9e3a9290 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f65c835c8e03587 server_cookie=bcc71288700bcf6 in_seq=0 out_seq=0 2024-03-21T11:30:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.380+0000 7f63cf400640 10 client.28025 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.381+0000 7f63cf400640 1 -- 172.21.15.47:0/3240098957 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559e9e303d70 con 0x559e9e3a6ea0 2024-03-21T11:30:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.381+0000 7f63cf400640 10 client.28025 handle_command_reply: tid=0 2024-03-21T11:30:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.381+0000 7f63f398e740 1 client.28025 shutdown 2024-03-21T11:30:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.381+0000 7f63f398e740 1 -- 172.21.15.47:0/3240098957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e3911e0 msgr2=0x559e9e39b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.381+0000 7f63f398e740 1 --2- 172.21.15.47:0/3240098957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e3911e0 0x559e9e39b360 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009c70 tx=0x7f63dc003ea0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.381+0000 7f63f398e740 1 -- 172.21.15.47:0/3240098957 shutdown_connections 2024-03-21T11:30:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.381+0000 7f63f398e740 1 --2- 172.21.15.47:0/3240098957 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e9e3a6ea0 0x559e9e3a9290 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.381+0000 7f63f398e740 1 --2- 172.21.15.47:0/3240098957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 0x559e9e3a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.381+0000 7f63f398e740 1 --2- 172.21.15.47:0/3240098957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e391bd0 0x559e9e39b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.381+0000 7f63f398e740 1 --2- 172.21.15.47:0/3240098957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e3911e0 0x559e9e39b360 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.381+0000 7f63f398e740 1 -- 172.21.15.47:0/3240098957 >> 172.21.15.47:0/3240098957 conn(0x559e9e38c750 msgr2=0x559e9e38b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.382+0000 7f63cf400640 0 client.28025 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.382+0000 7f63f398e740 1 -- 172.21.15.47:0/3240098957 shutdown_connections 2024-03-21T11:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.382+0000 7f63f398e740 1 -- 172.21.15.47:0/3240098957 wait complete. 2024-03-21T11:30:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.382+0000 7f63f398e740 20 client.28025 trim_cache size 0 max 16384 2024-03-21T11:30:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.400+0000 7f63f398e740 1 Processor -- start 2024-03-21T11:30:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.400+0000 7f63f398e740 1 -- start start 2024-03-21T11:30:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.400+0000 7f63f398e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e3911e0 0x559e9e3900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.400+0000 7f63f398e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 0x559e9e3adb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.400+0000 7f63f398e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 0x559e9e3ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.400+0000 7f63f398e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec1ac2a0 con 0x559e9e396d40 2024-03-21T11:30:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.400+0000 7f63f398e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec1adc20 con 0x559e9e391bd0 2024-03-21T11:30:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.400+0000 7f63f398e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec1ac900 con 0x559e9e3911e0 2024-03-21T11:30:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.400+0000 7f63e9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e3911e0 0x559e9e3900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.400+0000 7f63e3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 0x559e9e3adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.400+0000 7f63e9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e3911e0 0x559e9e3900c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52010/0 (socket says 172.21.15.47:52010) 2024-03-21T11:30:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.400+0000 7f63e9e00640 1 -- 172.21.15.47:0/2225045536 learned_addr learned my addr 172.21.15.47:0/2225045536 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.400+0000 7f63ea800640 1 --2- 172.21.15.47:0/2225045536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 0x559e9e3ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.401+0000 7f63e9e00640 1 -- 172.21.15.47:0/2225045536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 msgr2=0x559e9e3adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.401+0000 7f63e9e00640 1 --2- 172.21.15.47:0/2225045536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 0x559e9e3adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.401+0000 7f63e9e00640 1 -- 172.21.15.47:0/2225045536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 msgr2=0x559e9e3ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.401+0000 7f63e9e00640 1 --2- 172.21.15.47:0/2225045536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 0x559e9e3ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.401+0000 7f63e9e00640 1 -- 172.21.15.47:0/2225045536 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e9e38d4c0 con 0x559e9e3911e0 2024-03-21T11:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.401+0000 7f63e9e00640 1 --2- 172.21.15.47:0/2225045536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e3911e0 0x559e9e3900c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f63dc003e00 tx=0x7f63dc0097c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67a7917235df1385 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.401+0000 7f63da600640 1 -- 172.21.15.47:0/2225045536 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc03d070 con 0x559e9e3911e0 2024-03-21T11:30:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.401+0000 7f63da600640 1 -- 172.21.15.47:0/2225045536 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63dc004310 con 0x559e9e3911e0 2024-03-21T11:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.401+0000 7f63da600640 1 -- 172.21.15.47:0/2225045536 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc028040 con 0x559e9e3911e0 2024-03-21T11:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.402+0000 7f63f398e740 1 -- 172.21.15.47:0/2225045536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e3911e0 msgr2=0x559e9e3900c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.402+0000 7f63f398e740 1 --2- 172.21.15.47:0/2225045536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e3911e0 0x559e9e3900c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f63dc003e00 tx=0x7f63dc0097c0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.402+0000 7f63f398e740 1 -- 172.21.15.47:0/2225045536 shutdown_connections 2024-03-21T11:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.402+0000 7f63f398e740 1 --2- 172.21.15.47:0/2225045536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 0x559e9e3ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.402+0000 7f63f398e740 1 --2- 172.21.15.47:0/2225045536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 0x559e9e3adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.402+0000 7f63f398e740 1 --2- 172.21.15.47:0/2225045536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e3911e0 0x559e9e3900c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.402+0000 7f63f398e740 1 -- 172.21.15.47:0/2225045536 >> 172.21.15.47:0/2225045536 conn(0x559e9e38c750 msgr2=0x559e9e38f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.403+0000 7f63f398e740 1 -- 172.21.15.47:0/2225045536 shutdown_connections 2024-03-21T11:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.403+0000 7f63f398e740 1 -- 172.21.15.47:0/2225045536 wait complete. 2024-03-21T11:30:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.403+0000 7f63f398e740 1 -- start start 2024-03-21T11:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.403+0000 7f63f398e740 1 Processor -- start 2024-03-21T11:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63f398e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e3911e0 0x559e9e39f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63f398e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 0x559e9e39fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63f398e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 0x559e9e3a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63f398e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec1ad5c0 con 0x559e9e396d40 2024-03-21T11:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63f398e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec1acf60 con 0x559e9e391bd0 2024-03-21T11:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63f398e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec1aa460 con 0x559e9e3911e0 2024-03-21T11:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63f398e740 10 client.0 _ll_register_callbacks cb 0x7f63dc009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63e9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e3911e0 0x559e9e39f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63e9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e3911e0 0x559e9e39f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52018/0 (socket says 172.21.15.47:52018) 2024-03-21T11:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63e9e00640 1 -- 172.21.15.47:0/1801015398 learned_addr learned my addr 172.21.15.47:0/1801015398 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63e3e00640 1 --2- 172.21.15.47:0/1801015398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 0x559e9e39fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63ea800640 1 --2- 172.21.15.47:0/1801015398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 0x559e9e3a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63e9e00640 1 -- 172.21.15.47:0/1801015398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 msgr2=0x559e9e39fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63e9e00640 1 --2- 172.21.15.47:0/1801015398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 0x559e9e39fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63e9e00640 1 -- 172.21.15.47:0/1801015398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 msgr2=0x559e9e3a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63e9e00640 1 --2- 172.21.15.47:0/1801015398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 0x559e9e3a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.404+0000 7f63e9e00640 1 -- 172.21.15.47:0/1801015398 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e9e3a1360 con 0x559e9e3911e0 2024-03-21T11:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.405+0000 7f63e9e00640 1 --2- 172.21.15.47:0/1801015398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e3911e0 0x559e9e39f820 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009bc0 tx=0x7f63dc003ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82b12d6f717940af server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.405+0000 7f63cf400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.405+0000 7f63cf400640 1 -- 172.21.15.47:0/1801015398 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc03d070 con 0x559e9e3911e0 2024-03-21T11:30:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.405+0000 7f63f398e740 1 -- 172.21.15.47:0/1801015398 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63dc004310 con 0x559e9e3911e0 2024-03-21T11:30:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.405+0000 7f63cf400640 1 -- 172.21.15.47:0/1801015398 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f63dc0035f0 con 0x559e9e3911e0 2024-03-21T11:30:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.405+0000 7f63cf400640 1 -- 172.21.15.47:0/1801015398 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63dc049050 con 0x559e9e3911e0 2024-03-21T11:30:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.405+0000 7f63f398e740 10 client.28048 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.405+0000 7f63f398e740 1 -- 172.21.15.47:0/1801015398 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559e9e4027e0 con 0x559e9e3911e0 2024-03-21T11:30:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.406+0000 7f63cf400640 1 -- 172.21.15.47:0/1801015398 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63dc004310 con 0x559e9e3911e0 2024-03-21T11:30:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.406+0000 7f63cf400640 1 -- 172.21.15.47:0/1801015398 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f63dc004680 con 0x559e9e3911e0 2024-03-21T11:30:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.406+0000 7f63f398e740 10 client.28048 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.406+0000 7f63f398e740 10 client.28048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.406+0000 7f63f398e740 20 client.28048 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.406+0000 7f63f398e740 1 --2- 172.21.15.47:0/1801015398 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e9e402fe0 0x559e9e405400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.406+0000 7f63f398e740 4 client.28048 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:30:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.406+0000 7f63f398e740 1 -- 172.21.15.47:0/1801015398 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559e9e2fe0c0 con 0x559e9e402fe0 2024-03-21T11:30:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.406+0000 7f63e3e00640 1 --2- 172.21.15.47:0/1801015398 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e9e402fe0 0x559e9e405400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.407+0000 7f63e3e00640 1 --2- 172.21.15.47:0/1801015398 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e9e402fe0 0x559e9e405400 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dcceea0bb5b26d3 server_cookie=698cbad5064ab515 in_seq=0 out_seq=0 2024-03-21T11:30:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.407+0000 7f63cf400640 10 client.28048 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:44.530 INFO:tasks.workunit.client.0.smithi047.stdout: 1 511850 8.29 MB/sec execute 208 sec latency 83.216 ms 2024-03-21T11:30:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.649+0000 7f63cf400640 1 -- 172.21.15.47:0/1801015398 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559e9e2fe0c0 con 0x559e9e402fe0 2024-03-21T11:30:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.649+0000 7f63cf400640 10 client.28048 handle_command_reply: tid=0 2024-03-21T11:30:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.649+0000 7f63f398e740 1 client.28048 shutdown 2024-03-21T11:30:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.649+0000 7f63f398e740 1 -- 172.21.15.47:0/1801015398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e3911e0 msgr2=0x559e9e39f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.649+0000 7f63f398e740 1 --2- 172.21.15.47:0/1801015398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e3911e0 0x559e9e39f820 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009bc0 tx=0x7f63dc003ea0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.650+0000 7f63f398e740 1 -- 172.21.15.47:0/1801015398 shutdown_connections 2024-03-21T11:30:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.650+0000 7f63f398e740 1 --2- 172.21.15.47:0/1801015398 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e9e402fe0 0x559e9e405400 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.650+0000 7f63f398e740 1 --2- 172.21.15.47:0/1801015398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e9e396d40 0x559e9e3a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.650+0000 7f63f398e740 1 --2- 172.21.15.47:0/1801015398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9e391bd0 0x559e9e39fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.650+0000 7f63f398e740 1 --2- 172.21.15.47:0/1801015398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e9e3911e0 0x559e9e39f820 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.650+0000 7f63f398e740 1 -- 172.21.15.47:0/1801015398 >> 172.21.15.47:0/1801015398 conn(0x559e9e38c750 msgr2=0x559e9e38b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.650+0000 7f63cf400640 0 client.28048 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.650+0000 7f63f398e740 1 -- 172.21.15.47:0/1801015398 shutdown_connections 2024-03-21T11:30:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.650+0000 7f63f398e740 1 -- 172.21.15.47:0/1801015398 wait complete. 2024-03-21T11:30:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.650+0000 7f63f398e740 20 client.28048 trim_cache size 0 max 16384 2024-03-21T11:30:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:30:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:30:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:30:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:30:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.651+0000 7f63f0c00640 1 -- 172.21.15.47:0/3761694779 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f63d00a3040 msgr2=0x7f63d00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.651+0000 7f63f0c00640 1 --2- 172.21.15.47:0/3761694779 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f63d00a3040 0x7f63d00a5500 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f63d402c7c0 tx=0x7f63d4005b50 comp rx=0 tx=0).stop 2024-03-21T11:30:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.651+0000 7f63f0c00640 1 -- 172.21.15.47:0/3761694779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec1054a0 msgr2=0x7f63ec1a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.651+0000 7f63f0c00640 1 --2- 172.21.15.47:0/3761694779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec1054a0 0x7f63ec1a1160 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009fd0 tx=0x7f63e400b260 comp rx=0 tx=0).stop 2024-03-21T11:30:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.651+0000 7f63f0c00640 1 -- 172.21.15.47:0/3761694779 shutdown_connections 2024-03-21T11:30:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.651+0000 7f63f0c00640 1 --2- 172.21.15.47:0/3761694779 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f63d00a3040 0x7f63d00a5500 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.651+0000 7f63f0c00640 1 --2- 172.21.15.47:0/3761694779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec1054a0 0x7f63ec1a1160 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.651+0000 7f63f0c00640 1 --2- 172.21.15.47:0/3761694779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ec104ae0 0x7f63ec19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.652+0000 7f63f0c00640 1 --2- 172.21.15.47:0/3761694779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec1038f0 0x7f63ec199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.652+0000 7f63f0c00640 1 -- 172.21.15.47:0/3761694779 >> 172.21.15.47:0/3761694779 conn(0x7f63ec0fee60 msgr2=0x7f63ec107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.652+0000 7f63f0c00640 1 -- 172.21.15.47:0/3761694779 shutdown_connections 2024-03-21T11:30:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:44.652+0000 7f63f0c00640 1 -- 172.21.15.47:0/3761694779 wait complete. 2024-03-21T11:30:45.195 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:44 smithi100 ceph-mon[36245]: pgmap v622: 129 pgs: 129 active+clean; 20 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.3 MiB/s wr, 326 op/s 2024-03-21T11:30:45.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:44 smithi102 ceph-mon[36748]: pgmap v622: 129 pgs: 129 active+clean; 20 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.3 MiB/s wr, 326 op/s 2024-03-21T11:30:45.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:44 smithi047 ceph-mon[31351]: pgmap v622: 129 pgs: 129 active+clean; 20 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.3 MiB/s wr, 326 op/s 2024-03-21T11:30:45.530 INFO:tasks.workunit.client.0.smithi047.stdout: 1 512507 8.26 MB/sec execute 209 sec latency 118.603 ms 2024-03-21T11:30:46.530 INFO:tasks.workunit.client.0.smithi047.stdout: 1 514360 8.26 MB/sec execute 210 sec latency 27.116 ms 2024-03-21T11:30:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:46 smithi102 ceph-mon[36748]: pgmap v623: 129 pgs: 129 active+clean; 20 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-03-21T11:30:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:46 smithi047 ceph-mon[31351]: pgmap v623: 129 pgs: 129 active+clean; 20 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-03-21T11:30:47.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:46 smithi100 ceph-mon[36245]: pgmap v623: 129 pgs: 129 active+clean; 20 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-03-21T11:30:47.530 INFO:tasks.workunit.client.0.smithi047.stdout: 1 515658 8.26 MB/sec execute 211 sec latency 11.322 ms 2024-03-21T11:30:48.531 INFO:tasks.workunit.client.0.smithi047.stdout: 1 517080 8.25 MB/sec execute 212 sec latency 15.615 ms 2024-03-21T11:30:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:48 smithi102 ceph-mon[36748]: pgmap v624: 129 pgs: 129 active+clean; 29 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 2.6 MiB/s wr, 254 op/s 2024-03-21T11:30:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:48 smithi047 ceph-mon[31351]: pgmap v624: 129 pgs: 129 active+clean; 29 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 2.6 MiB/s wr, 254 op/s 2024-03-21T11:30:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:48 smithi100 ceph-mon[36245]: pgmap v624: 129 pgs: 129 active+clean; 29 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 2.6 MiB/s wr, 254 op/s 2024-03-21T11:30:49.530 INFO:tasks.workunit.client.0.smithi047.stdout: 1 518879 8.27 MB/sec execute 213 sec latency 18.154 ms 2024-03-21T11:30:50.531 INFO:tasks.workunit.client.0.smithi047.stdout: 1 519732 8.25 MB/sec execute 214 sec latency 11.045 ms 2024-03-21T11:30:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:50 smithi102 ceph-mon[36748]: pgmap v625: 129 pgs: 129 active+clean; 35 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-03-21T11:30:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:50 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:30:51.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:50 smithi047 ceph-mon[31351]: pgmap v625: 129 pgs: 129 active+clean; 35 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-03-21T11:30:51.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:50 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:30:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:50 smithi100 ceph-mon[36245]: pgmap v625: 129 pgs: 129 active+clean; 35 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-03-21T11:30:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:50 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:30:51.531 INFO:tasks.workunit.client.0.smithi047.stdout: 1 521601 8.26 MB/sec execute 215 sec latency 64.806 ms 2024-03-21T11:30:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:30:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:30:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:30:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:30:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:30:52.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:30:52.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:51 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:30:52.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:30:52.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:30:52.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:30:52.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:30:52.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:30:52.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:30:52.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:51 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:30:52.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:30:52.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:30:52.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:30:52.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:30:52.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:30:52.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:30:52.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:51 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:30:52.531 INFO:tasks.workunit.client.0.smithi047.stdout: 1 522819 8.25 MB/sec execute 216 sec latency 19.570 ms 2024-03-21T11:30:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:52 smithi102 ceph-mon[36748]: pgmap v626: 129 pgs: 129 active+clean; 34 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 3.4 MiB/s wr, 283 op/s 2024-03-21T11:30:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:52 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi102 to 174.1M 2024-03-21T11:30:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:52 smithi102 ceph-mon[36748]: Unable to set osd_memory_target on smithi102 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-03-21T11:30:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:52 smithi047 ceph-mon[31351]: pgmap v626: 129 pgs: 129 active+clean; 34 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 3.4 MiB/s wr, 283 op/s 2024-03-21T11:30:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:52 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi102 to 174.1M 2024-03-21T11:30:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:52 smithi047 ceph-mon[31351]: Unable to set osd_memory_target on smithi102 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-03-21T11:30:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:52 smithi100 ceph-mon[36245]: pgmap v626: 129 pgs: 129 active+clean; 34 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 3.4 MiB/s wr, 283 op/s 2024-03-21T11:30:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:52 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi102 to 174.1M 2024-03-21T11:30:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:52 smithi100 ceph-mon[36245]: Unable to set osd_memory_target on smithi102 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-03-21T11:30:53.531 INFO:tasks.workunit.client.0.smithi047.stdout: 1 524682 8.25 MB/sec execute 217 sec latency 12.046 ms 2024-03-21T11:30:54.531 INFO:tasks.workunit.client.0.smithi047.stdout: 1 526268 8.26 MB/sec execute 218 sec latency 11.364 ms 2024-03-21T11:30:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:54 smithi102 ceph-mon[36748]: pgmap v627: 129 pgs: 129 active+clean; 36 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.8 MiB/s wr, 331 op/s 2024-03-21T11:30:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:54 smithi047 ceph-mon[31351]: pgmap v627: 129 pgs: 129 active+clean; 36 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.8 MiB/s wr, 331 op/s 2024-03-21T11:30:55.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:54 smithi100 ceph-mon[36245]: pgmap v627: 129 pgs: 129 active+clean; 36 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.8 MiB/s wr, 331 op/s 2024-03-21T11:30:55.531 INFO:tasks.workunit.client.0.smithi047.stdout: 1 528062 8.25 MB/sec execute 219 sec latency 12.949 ms 2024-03-21T11:30:56.531 INFO:tasks.workunit.client.0.smithi047.stdout: 1 529517 8.27 MB/sec execute 220 sec latency 75.782 ms 2024-03-21T11:30:57.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:56 smithi102 ceph-mon[36748]: pgmap v628: 129 pgs: 129 active+clean; 36 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-03-21T11:30:57.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:56 smithi047 ceph-mon[31351]: pgmap v628: 129 pgs: 129 active+clean; 36 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-03-21T11:30:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:56 smithi100 ceph-mon[36245]: pgmap v628: 129 pgs: 129 active+clean; 36 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-03-21T11:30:57.532 INFO:tasks.workunit.client.0.smithi047.stdout: 1 530365 8.24 MB/sec execute 221 sec latency 20.572 ms 2024-03-21T11:30:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.810+0000 7f842ea00640 1 -- 172.21.15.47:0/3595891037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84281048a0 msgr2=0x7f8428106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.810+0000 7f842ea00640 1 --2- 172.21.15.47:0/3595891037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84281048a0 0x7f8428106c90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8414009900 tx=0x7f841402bc40 comp rx=0 tx=0).stop 2024-03-21T11:30:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.811+0000 7f842ea00640 1 -- 172.21.15.47:0/3595891037 shutdown_connections 2024-03-21T11:30:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.811+0000 7f842ea00640 1 --2- 172.21.15.47:0/3595891037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84281071d0 0x7f8428109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.811+0000 7f842ea00640 1 --2- 172.21.15.47:0/3595891037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84281048a0 0x7f8428106c90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.811+0000 7f842ea00640 1 --2- 172.21.15.47:0/3595891037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8428101f70 0x7f8428104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.811+0000 7f842ea00640 1 -- 172.21.15.47:0/3595891037 >> 172.21.15.47:0/3595891037 conn(0x7f84280fb980 msgr2=0x7f84280fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.811+0000 7f842ea00640 1 -- 172.21.15.47:0/3595891037 shutdown_connections 2024-03-21T11:30:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.811+0000 7f842ea00640 1 -- 172.21.15.47:0/3595891037 wait complete. 2024-03-21T11:30:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.812+0000 7f842ea00640 1 Processor -- start 2024-03-21T11:30:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.812+0000 7f842ea00640 1 -- start start 2024-03-21T11:30:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.812+0000 7f842ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8428101f70 0x7f8428199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.812+0000 7f842ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84281048a0 0x7f8428199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.812+0000 7f842ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84281071d0 0x7f84281a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.812+0000 7f842ea00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842810d140 con 0x7f84281048a0 2024-03-21T11:30:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.812+0000 7f842ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842810cfc0 con 0x7f84281071d0 2024-03-21T11:30:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.812+0000 7f842ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842810d2c0 con 0x7f8428101f70 2024-03-21T11:30:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.813+0000 7f842d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8428101f70 0x7f8428199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.813+0000 7f842e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84281071d0 0x7f84281a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.813+0000 7f842d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8428101f70 0x7f8428199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45932/0 (socket says 172.21.15.47:45932) 2024-03-21T11:30:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.813+0000 7f842d600640 1 -- 172.21.15.47:0/2269858469 learned_addr learned my addr 172.21.15.47:0/2269858469 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.813+0000 7f842cc00640 1 --2- 172.21.15.47:0/2269858469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84281048a0 0x7f8428199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.813+0000 7f842d600640 1 -- 172.21.15.47:0/2269858469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84281071d0 msgr2=0x7f84281a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.813+0000 7f842d600640 1 --2- 172.21.15.47:0/2269858469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84281071d0 0x7f84281a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.813+0000 7f842d600640 1 -- 172.21.15.47:0/2269858469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84281048a0 msgr2=0x7f8428199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.813+0000 7f842d600640 1 --2- 172.21.15.47:0/2269858469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84281048a0 0x7f8428199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.813+0000 7f842d600640 1 -- 172.21.15.47:0/2269858469 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84281a1680 con 0x7f8428101f70 2024-03-21T11:30:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.813+0000 7f842e000640 1 --2- 172.21.15.47:0/2269858469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84281071d0 0x7f84281a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:30:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.813+0000 7f842d600640 1 --2- 172.21.15.47:0/2269858469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8428101f70 0x7f8428199a60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f841c00e9f0 tx=0x7f841c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.814+0000 7f8426000640 1 -- 172.21.15.47:0/2269858469 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c00cd80 con 0x7f8428101f70 2024-03-21T11:30:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.814+0000 7f8426000640 1 -- 172.21.15.47:0/2269858469 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f841c010910 con 0x7f8428101f70 2024-03-21T11:30:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.814+0000 7f8426000640 1 -- 172.21.15.47:0/2269858469 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c018990 con 0x7f8428101f70 2024-03-21T11:30:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.814+0000 7f842ea00640 1 -- 172.21.15.47:0/2269858469 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84281a1970 con 0x7f8428101f70 2024-03-21T11:30:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.814+0000 7f842ea00640 1 -- 172.21.15.47:0/2269858469 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84281a1e40 con 0x7f8428101f70 2024-03-21T11:30:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.817+0000 7f8432dd1740 1 Processor -- start 2024-03-21T11:30:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.817+0000 7f8426000640 1 -- 172.21.15.47:0/2269858469 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f841c010ac0 con 0x7f8428101f70 2024-03-21T11:30:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.817+0000 7f8432dd1740 1 -- start start 2024-03-21T11:30:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.817+0000 7f8432dd1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21b3f0 0x55802a21b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.817+0000 7f8432dd1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21bde0 0x55802a2208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.818+0000 7f8432dd1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a220f50 0x55802a2233e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.818+0000 7f8432dd1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55802a1de3b0 con 0x55802a220f50 2024-03-21T11:30:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.818+0000 7f8432dd1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55802a180830 con 0x55802a21bde0 2024-03-21T11:30:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.818+0000 7f8432dd1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55802a1295d0 con 0x55802a21b3f0 2024-03-21T11:30:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.818+0000 7f842e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21b3f0 0x55802a21b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.818+0000 7f842d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21bde0 0x55802a2208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.818+0000 7f842e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21b3f0 0x55802a21b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45944/0 (socket says 172.21.15.47:45944) 2024-03-21T11:30:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.818+0000 7f842e000640 1 -- 172.21.15.47:0/993217514 learned_addr learned my addr 172.21.15.47:0/993217514 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.818+0000 7f842cc00640 1 --2- 172.21.15.47:0/993217514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a220f50 0x55802a2233e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.818+0000 7f842e000640 1 -- 172.21.15.47:0/993217514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21bde0 msgr2=0x55802a2208e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.818+0000 7f842e000640 1 --2- 172.21.15.47:0/993217514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21bde0 0x55802a2208e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.818+0000 7f842e000640 1 -- 172.21.15.47:0/993217514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a220f50 msgr2=0x55802a2233e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.818+0000 7f842e000640 1 --2- 172.21.15.47:0/993217514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a220f50 0x55802a2233e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.818+0000 7f842e000640 1 -- 172.21.15.47:0/993217514 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55802a1412c0 con 0x55802a21b3f0 2024-03-21T11:30:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.819+0000 7f842e000640 1 --2- 172.21.15.47:0/993217514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21b3f0 0x55802a21b810 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f84200117e0 tx=0x7f8420011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d95f8276b56a3fa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.819+0000 7f841a600640 1 -- 172.21.15.47:0/993217514 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420011310 con 0x55802a21b3f0 2024-03-21T11:30:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.819+0000 7f841a600640 1 -- 172.21.15.47:0/993217514 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8420003e50 con 0x55802a21b3f0 2024-03-21T11:30:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.819+0000 7f841a600640 1 -- 172.21.15.47:0/993217514 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420004620 con 0x55802a21b3f0 2024-03-21T11:30:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.819+0000 7f8426000640 1 --2- 172.21.15.47:0/2269858469 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84100a2f90 0x7f84100a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.819+0000 7f8426000640 1 -- 172.21.15.47:0/2269858469 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f841c0cf9d0 con 0x7f8428101f70 2024-03-21T11:30:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.820+0000 7f8432dd1740 1 -- 172.21.15.47:0/993217514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21b3f0 msgr2=0x55802a21b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.820+0000 7f8432dd1740 1 --2- 172.21.15.47:0/993217514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21b3f0 0x55802a21b810 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f84200117e0 tx=0x7f8420011ca0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.820+0000 7f842d600640 1 --2- 172.21.15.47:0/2269858469 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84100a2f90 0x7f84100a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.820+0000 7f8432dd1740 1 -- 172.21.15.47:0/993217514 shutdown_connections 2024-03-21T11:30:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.820+0000 7f8432dd1740 1 --2- 172.21.15.47:0/993217514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a220f50 0x55802a2233e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.820+0000 7f8432dd1740 1 --2- 172.21.15.47:0/993217514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21bde0 0x55802a2208e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.820+0000 7f8432dd1740 1 --2- 172.21.15.47:0/993217514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21b3f0 0x55802a21b810 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.820+0000 7f8432dd1740 1 -- 172.21.15.47:0/993217514 >> 172.21.15.47:0/993217514 conn(0x55802a216960 msgr2=0x55802a216d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.820+0000 7f8432dd1740 1 -- 172.21.15.47:0/993217514 shutdown_connections 2024-03-21T11:30:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.820+0000 7f8432dd1740 1 -- 172.21.15.47:0/993217514 wait complete. 2024-03-21T11:30:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.820+0000 7f842d600640 1 --2- 172.21.15.47:0/2269858469 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84100a2f90 0x7f84100a5450 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55802a21c1e0 tx=0x7f841c0d5550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.820+0000 7f8432dd1740 1 -- start start 2024-03-21T11:30:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.821+0000 7f8432dd1740 1 Processor -- start 2024-03-21T11:30:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.821+0000 7f8432dd1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a21b3f0 0x55802a225560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.821+0000 7f8432dd1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21bde0 0x55802a225aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.821+0000 7f8432dd1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a220f50 0x55802a22cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.821+0000 7f8432dd1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55802a13ca20 con 0x55802a21b3f0 2024-03-21T11:30:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.821+0000 7f8432dd1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55802a17c8d0 con 0x55802a21bde0 2024-03-21T11:30:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.821+0000 7f8432dd1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55802a223f00 con 0x55802a220f50 2024-03-21T11:30:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.822+0000 7f842cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21bde0 0x55802a225aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.822+0000 7f842cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21bde0 0x55802a225aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52230/0 (socket says 172.21.15.47:52230) 2024-03-21T11:30:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.822+0000 7f842cc00640 1 -- 172.21.15.47:0/1831483019 learned_addr learned my addr 172.21.15.47:0/1831483019 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.822+0000 7f8432dd1740 10 client.0 _ll_register_callbacks cb 0x55802a1791c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.822+0000 7f842e000640 1 --2- 172.21.15.47:0/1831483019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a21b3f0 0x55802a225560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.822+0000 7f842cc00640 1 -- 172.21.15.47:0/1831483019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a220f50 msgr2=0x55802a22cae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:30:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.822+0000 7f842cc00640 1 --2- 172.21.15.47:0/1831483019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a220f50 0x55802a22cae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.822+0000 7f842cc00640 1 -- 172.21.15.47:0/1831483019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a21b3f0 msgr2=0x55802a225560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.822+0000 7f842cc00640 1 --2- 172.21.15.47:0/1831483019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a21b3f0 0x55802a225560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.822+0000 7f842cc00640 1 -- 172.21.15.47:0/1831483019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558029e95210 con 0x55802a21bde0 2024-03-21T11:30:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.822+0000 7f842e000640 1 --2- 172.21.15.47:0/1831483019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a21b3f0 0x55802a225560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.823+0000 7f842cc00640 1 --2- 172.21.15.47:0/1831483019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21bde0 0x55802a225aa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f841402c7c0 tx=0x7f8414004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ee9461c6e4f8d28 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.823+0000 7f840ea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:30:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.823+0000 7f840ea00640 1 -- 172.21.15.47:0/1831483019 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414005d50 con 0x55802a21bde0 2024-03-21T11:30:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.823+0000 7f840ea00640 1 -- 172.21.15.47:0/1831483019 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8414005230 con 0x55802a21bde0 2024-03-21T11:30:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.823+0000 7f8432dd1740 1 -- 172.21.15.47:0/1831483019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8420003e50 con 0x55802a21bde0 2024-03-21T11:30:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.823+0000 7f840ea00640 1 -- 172.21.15.47:0/1831483019 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8414005430 con 0x55802a21bde0 2024-03-21T11:30:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.823+0000 7f8432dd1740 10 client.28043 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.823+0000 7f8432dd1740 1 -- 172.21.15.47:0/1831483019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558029f61620 con 0x55802a21bde0 2024-03-21T11:30:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.824+0000 7f840ea00640 1 -- 172.21.15.47:0/1831483019 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8420003e50 con 0x55802a21bde0 2024-03-21T11:30:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.824+0000 7f840ea00640 1 -- 172.21.15.47:0/1831483019 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f84140037b0 con 0x55802a21bde0 2024-03-21T11:30:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.824+0000 7f8432dd1740 10 client.28043 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.824+0000 7f8432dd1740 10 client.28043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.824+0000 7f8432dd1740 20 client.28043 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.824+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1831483019 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55802a2313d0 0x55802a233820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.824+0000 7f8432dd1740 4 client.28043 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:30:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.824+0000 7f8432dd1740 1 -- 172.21.15.47:0/1831483019 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55802a18dd70 con 0x55802a2313d0 2024-03-21T11:30:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.824+0000 7f842e000640 1 --2- 172.21.15.47:0/1831483019 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55802a2313d0 0x55802a233820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.825+0000 7f842e000640 1 --2- 172.21.15.47:0/1831483019 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55802a2313d0 0x55802a233820 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e3ff465f88f455a server_cookie=2c8230627c6514ed in_seq=0 out_seq=0 2024-03-21T11:30:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.825+0000 7f840ea00640 10 client.28043 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f840ea00640 1 -- 172.21.15.47:0/1831483019 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55802a18dd70 con 0x55802a2313d0 2024-03-21T11:30:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f840ea00640 10 client.28043 handle_command_reply: tid=0 2024-03-21T11:30:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f8432dd1740 1 client.28043 shutdown 2024-03-21T11:30:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f8432dd1740 1 -- 172.21.15.47:0/1831483019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21bde0 msgr2=0x55802a225aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1831483019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21bde0 0x55802a225aa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f841402c7c0 tx=0x7f8414004270 comp rx=0 tx=0).stop 2024-03-21T11:30:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f8432dd1740 1 -- 172.21.15.47:0/1831483019 shutdown_connections 2024-03-21T11:30:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1831483019 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55802a2313d0 0x55802a233820 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1831483019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a220f50 0x55802a22cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1831483019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21bde0 0x55802a225aa0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1831483019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a21b3f0 0x55802a225560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f840ea00640 0 client.28043 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f8432dd1740 1 -- 172.21.15.47:0/1831483019 >> 172.21.15.47:0/1831483019 conn(0x55802a216960 msgr2=0x55802a215b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f8432dd1740 1 -- 172.21.15.47:0/1831483019 shutdown_connections 2024-03-21T11:30:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f8432dd1740 1 -- 172.21.15.47:0/1831483019 wait complete. 2024-03-21T11:30:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.827+0000 7f8432dd1740 20 client.28043 trim_cache size 0 max 16384 2024-03-21T11:30:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.843+0000 7f8432dd1740 1 Processor -- start 2024-03-21T11:30:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.844+0000 7f8432dd1740 1 -- start start 2024-03-21T11:30:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.844+0000 7f8432dd1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21b3f0 0x55802a216d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.844+0000 7f8432dd1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21bde0 0x55802a22caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.844+0000 7f8432dd1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a220f50 0x55802a219bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.844+0000 7f8432dd1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281ac270 con 0x55802a220f50 2024-03-21T11:30:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.844+0000 7f8432dd1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281adbf0 con 0x55802a21b3f0 2024-03-21T11:30:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.844+0000 7f8432dd1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281ac8d0 con 0x55802a21bde0 2024-03-21T11:30:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.844+0000 7f842cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21bde0 0x55802a22caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.844+0000 7f842cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21bde0 0x55802a22caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45952/0 (socket says 172.21.15.47:45952) 2024-03-21T11:30:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.844+0000 7f842cc00640 1 -- 172.21.15.47:0/1859246822 learned_addr learned my addr 172.21.15.47:0/1859246822 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.844+0000 7f842e000640 1 --2- 172.21.15.47:0/1859246822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21b3f0 0x55802a216d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.844+0000 7f842e000640 1 --2- 172.21.15.47:0/1859246822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a220f50 0x55802a219bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.844+0000 7f842cc00640 1 -- 172.21.15.47:0/1859246822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21b3f0 msgr2=0x55802a216d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.844+0000 7f842cc00640 1 --2- 172.21.15.47:0/1859246822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21b3f0 0x55802a216d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.845+0000 7f842cc00640 1 -- 172.21.15.47:0/1859246822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a220f50 msgr2=0x55802a219bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.845+0000 7f842cc00640 1 --2- 172.21.15.47:0/1859246822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a220f50 0x55802a219bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.845+0000 7f842cc00640 1 -- 172.21.15.47:0/1859246822 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55802a21c670 con 0x55802a21bde0 2024-03-21T11:30:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.845+0000 7f842cc00640 1 --2- 172.21.15.47:0/1859246822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21bde0 0x55802a22caa0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f841402c180 tx=0x7f841402bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a98b0206243a62d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.845+0000 7f8419c00640 1 -- 172.21.15.47:0/1859246822 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414058070 con 0x55802a21bde0 2024-03-21T11:30:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.845+0000 7f8419c00640 1 -- 172.21.15.47:0/1859246822 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8414005850 con 0x55802a21bde0 2024-03-21T11:30:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.845+0000 7f8419c00640 1 -- 172.21.15.47:0/1859246822 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84140046c0 con 0x55802a21bde0 2024-03-21T11:30:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.846+0000 7f8432dd1740 1 -- 172.21.15.47:0/1859246822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21bde0 msgr2=0x55802a22caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.846+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1859246822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21bde0 0x55802a22caa0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f841402c180 tx=0x7f841402bc40 comp rx=0 tx=0).stop 2024-03-21T11:30:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.846+0000 7f8432dd1740 1 -- 172.21.15.47:0/1859246822 shutdown_connections 2024-03-21T11:30:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.846+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1859246822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a220f50 0x55802a219bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.846+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1859246822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21bde0 0x55802a22caa0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.846+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1859246822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a21b3f0 0x55802a216d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.846+0000 7f8432dd1740 1 -- 172.21.15.47:0/1859246822 >> 172.21.15.47:0/1859246822 conn(0x55802a216960 msgr2=0x55802a2116a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.846+0000 7f8432dd1740 1 -- 172.21.15.47:0/1859246822 shutdown_connections 2024-03-21T11:30:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.846+0000 7f8432dd1740 1 -- 172.21.15.47:0/1859246822 wait complete. 2024-03-21T11:30:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.847+0000 7f8432dd1740 1 -- start start 2024-03-21T11:30:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.847+0000 7f8432dd1740 1 Processor -- start 2024-03-21T11:30:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f8432dd1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a21b3f0 0x55802a22a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f8432dd1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21bde0 0x55802a22adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f8432dd1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a220f50 0x55802a22bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f8432dd1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281ad590 con 0x55802a21b3f0 2024-03-21T11:30:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f8432dd1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281acf30 con 0x55802a220f50 2024-03-21T11:30:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f8432dd1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281aa430 con 0x55802a21bde0 2024-03-21T11:30:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f8432dd1740 10 client.0 _ll_register_callbacks cb 0x7f84200093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:30:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f842cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21bde0 0x55802a22adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f842cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21bde0 0x55802a22adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45956/0 (socket says 172.21.15.47:45956) 2024-03-21T11:30:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f842cc00640 1 -- 172.21.15.47:0/1381522967 learned_addr learned my addr 172.21.15.47:0/1381522967 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:30:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f842e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a21b3f0 0x55802a22a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f842e000640 1 --2- 172.21.15.47:0/1381522967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a220f50 0x55802a22bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f842cc00640 1 -- 172.21.15.47:0/1381522967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a220f50 msgr2=0x55802a22bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f842cc00640 1 --2- 172.21.15.47:0/1381522967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a220f50 0x55802a22bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f842cc00640 1 -- 172.21.15.47:0/1381522967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a21b3f0 msgr2=0x55802a22a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:30:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f842cc00640 1 --2- 172.21.15.47:0/1381522967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a21b3f0 0x55802a22a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f842cc00640 1 -- 172.21.15.47:0/1381522967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55802a22c3b0 con 0x55802a21bde0 2024-03-21T11:30:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.848+0000 7f842e000640 1 --2- 172.21.15.47:0/1381522967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a220f50 0x55802a22bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:30:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.849+0000 7f842cc00640 1 --2- 172.21.15.47:0/1381522967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21bde0 0x55802a22adb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8414003ca0 tx=0x7f8414003d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9db1988b7667e81f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:30:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.849+0000 7f840ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:30:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.849+0000 7f840ea00640 1 -- 172.21.15.47:0/1381522967 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414058040 con 0x55802a21bde0 2024-03-21T11:30:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.849+0000 7f840ea00640 1 -- 172.21.15.47:0/1381522967 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8414003e10 con 0x55802a21bde0 2024-03-21T11:30:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.849+0000 7f8432dd1740 1 -- 172.21.15.47:0/1381522967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8414005850 con 0x55802a21bde0 2024-03-21T11:30:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.849+0000 7f840ea00640 1 -- 172.21.15.47:0/1381522967 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8414065050 con 0x55802a21bde0 2024-03-21T11:30:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.849+0000 7f8432dd1740 10 client.28063 fetch_fsmap learned FSMap version 28 2024-03-21T11:30:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.849+0000 7f8432dd1740 1 -- 172.21.15.47:0/1381522967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55802a28d3d0 con 0x55802a21bde0 2024-03-21T11:30:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.850+0000 7f840ea00640 1 -- 172.21.15.47:0/1381522967 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8414005850 con 0x55802a21bde0 2024-03-21T11:30:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.850+0000 7f840ea00640 1 -- 172.21.15.47:0/1381522967 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8414034700 con 0x55802a21bde0 2024-03-21T11:30:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.850+0000 7f8432dd1740 10 client.28063 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:30:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.850+0000 7f8432dd1740 10 client.28063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:30:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.850+0000 7f8432dd1740 20 client.28063 populate_metadata read hostname 'smithi047' 2024-03-21T11:30:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.850+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1381522967 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55802a28dd50 0x55802a290170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:30:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.850+0000 7f8432dd1740 4 client.28063 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:30:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.850+0000 7f8432dd1740 1 -- 172.21.15.47:0/1381522967 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55802a1880c0 con 0x55802a28dd50 2024-03-21T11:30:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.850+0000 7f842e000640 1 --2- 172.21.15.47:0/1381522967 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55802a28dd50 0x55802a290170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:30:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.851+0000 7f842e000640 1 --2- 172.21.15.47:0/1381522967 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55802a28dd50 0x55802a290170 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e32f9996075de288 server_cookie=7b9fe0da8c4895b6 in_seq=0 out_seq=0 2024-03-21T11:30:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:57.851+0000 7f840ea00640 10 client.28063 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.085+0000 7f840ea00640 1 -- 172.21.15.47:0/1381522967 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55802a1880c0 con 0x55802a28dd50 2024-03-21T11:30:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.085+0000 7f840ea00640 10 client.28063 handle_command_reply: tid=0 2024-03-21T11:30:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.085+0000 7f8432dd1740 1 client.28063 shutdown 2024-03-21T11:30:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.085+0000 7f8432dd1740 1 -- 172.21.15.47:0/1381522967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21bde0 msgr2=0x55802a22adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.085+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1381522967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21bde0 0x55802a22adb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8414003ca0 tx=0x7f8414003d20 comp rx=0 tx=0).stop 2024-03-21T11:30:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.085+0000 7f8432dd1740 1 -- 172.21.15.47:0/1381522967 shutdown_connections 2024-03-21T11:30:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.085+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1381522967 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55802a28dd50 0x55802a290170 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.085+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1381522967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55802a220f50 0x55802a22bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.085+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1381522967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55802a21bde0 0x55802a22adb0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.085+0000 7f8432dd1740 1 --2- 172.21.15.47:0/1381522967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55802a21b3f0 0x55802a22a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.085+0000 7f840ea00640 0 client.28063 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:30:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.085+0000 7f8432dd1740 1 -- 172.21.15.47:0/1381522967 >> 172.21.15.47:0/1381522967 conn(0x55802a216960 msgr2=0x55802a216d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.086+0000 7f8432dd1740 1 -- 172.21.15.47:0/1381522967 shutdown_connections 2024-03-21T11:30:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.086+0000 7f8432dd1740 1 -- 172.21.15.47:0/1381522967 wait complete. 2024-03-21T11:30:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.086+0000 7f8432dd1740 20 client.28063 trim_cache size 0 max 16384 2024-03-21T11:30:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:30:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:30:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:30:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:30:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.087+0000 7f842ea00640 1 -- 172.21.15.47:0/2269858469 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84100a2f90 msgr2=0x7f84100a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.087+0000 7f842ea00640 1 --2- 172.21.15.47:0/2269858469 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84100a2f90 0x7f84100a5450 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55802a21c1e0 tx=0x7f841c0d5550 comp rx=0 tx=0).stop 2024-03-21T11:30:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.087+0000 7f842ea00640 1 -- 172.21.15.47:0/2269858469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8428101f70 msgr2=0x7f8428199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:30:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.087+0000 7f842ea00640 1 --2- 172.21.15.47:0/2269858469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8428101f70 0x7f8428199a60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f841c00e9f0 tx=0x7f841c00eeb0 comp rx=0 tx=0).stop 2024-03-21T11:30:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.087+0000 7f842ea00640 1 -- 172.21.15.47:0/2269858469 shutdown_connections 2024-03-21T11:30:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.087+0000 7f842ea00640 1 --2- 172.21.15.47:0/2269858469 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84100a2f90 0x7f84100a5450 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.087+0000 7f842ea00640 1 --2- 172.21.15.47:0/2269858469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84281071d0 0x7f84281a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.087+0000 7f842ea00640 1 --2- 172.21.15.47:0/2269858469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84281048a0 0x7f8428199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.087+0000 7f842ea00640 1 --2- 172.21.15.47:0/2269858469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8428101f70 0x7f8428199a60 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:30:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.087+0000 7f842ea00640 1 -- 172.21.15.47:0/2269858469 >> 172.21.15.47:0/2269858469 conn(0x7f84280fb980 msgr2=0x7f8428101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:30:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.087+0000 7f842ea00640 1 -- 172.21.15.47:0/2269858469 shutdown_connections 2024-03-21T11:30:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:30:58.087+0000 7f842ea00640 1 -- 172.21.15.47:0/2269858469 wait complete. 2024-03-21T11:30:58.532 INFO:tasks.workunit.client.0.smithi047.stdout: 1 532337 8.26 MB/sec execute 222 sec latency 129.255 ms 2024-03-21T11:30:59.246 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:30:58 smithi102 ceph-mon[36748]: pgmap v629: 129 pgs: 129 active+clean; 38 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 319 op/s 2024-03-21T11:30:59.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:30:58 smithi047 ceph-mon[31351]: pgmap v629: 129 pgs: 129 active+clean; 38 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 319 op/s 2024-03-21T11:30:59.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:30:58 smithi100 ceph-mon[36245]: pgmap v629: 129 pgs: 129 active+clean; 38 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 319 op/s 2024-03-21T11:30:59.532 INFO:tasks.workunit.client.0.smithi047.stdout: 1 533507 8.25 MB/sec execute 223 sec latency 20.211 ms 2024-03-21T11:31:00.532 INFO:tasks.workunit.client.0.smithi047.stdout: 1 535265 8.24 MB/sec execute 224 sec latency 13.029 ms 2024-03-21T11:31:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:00 smithi102 ceph-mon[36748]: pgmap v630: 129 pgs: 129 active+clean; 38 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-03-21T11:31:01.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:00 smithi047 ceph-mon[31351]: pgmap v630: 129 pgs: 129 active+clean; 38 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-03-21T11:31:01.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:00 smithi100 ceph-mon[36245]: pgmap v630: 129 pgs: 129 active+clean; 38 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-03-21T11:31:01.532 INFO:tasks.workunit.client.0.smithi047.stdout: 1 536738 8.26 MB/sec execute 225 sec latency 81.124 ms 2024-03-21T11:31:02.532 INFO:tasks.workunit.client.0.smithi047.stdout: 1 537687 8.23 MB/sec execute 226 sec latency 17.447 ms 2024-03-21T11:31:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:02 smithi102 ceph-mon[36748]: pgmap v631: 129 pgs: 129 active+clean; 38 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 217 op/s 2024-03-21T11:31:03.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:02 smithi047 ceph-mon[31351]: pgmap v631: 129 pgs: 129 active+clean; 38 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 217 op/s 2024-03-21T11:31:03.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:02 smithi100 ceph-mon[36245]: pgmap v631: 129 pgs: 129 active+clean; 38 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 217 op/s 2024-03-21T11:31:03.532 INFO:tasks.workunit.client.0.smithi047.stdout: 1 539779 8.25 MB/sec execute 227 sec latency 14.332 ms 2024-03-21T11:31:04.533 INFO:tasks.workunit.client.0.smithi047.stdout: 1 540782 8.24 MB/sec execute 228 sec latency 15.549 ms 2024-03-21T11:31:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:04 smithi102 ceph-mon[36748]: pgmap v632: 129 pgs: 129 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-03-21T11:31:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:04 smithi047 ceph-mon[31351]: pgmap v632: 129 pgs: 129 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-03-21T11:31:05.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:04 smithi100 ceph-mon[36245]: pgmap v632: 129 pgs: 129 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-03-21T11:31:05.533 INFO:tasks.workunit.client.0.smithi047.stdout: 1 542653 8.24 MB/sec execute 229 sec latency 18.004 ms 2024-03-21T11:31:06.533 INFO:tasks.workunit.client.0.smithi047.stdout: 1 543904 8.25 MB/sec execute 230 sec latency 43.400 ms 2024-03-21T11:31:07.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:06 smithi102 ceph-mon[36748]: pgmap v633: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 298 op/s 2024-03-21T11:31:07.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:06 smithi047 ceph-mon[31351]: pgmap v633: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 298 op/s 2024-03-21T11:31:07.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:06 smithi100 ceph-mon[36245]: pgmap v633: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 298 op/s 2024-03-21T11:31:07.533 INFO:tasks.workunit.client.0.smithi047.stdout: 1 545188 8.23 MB/sec execute 231 sec latency 14.816 ms 2024-03-21T11:31:08.533 INFO:tasks.workunit.client.0.smithi047.stdout: 1 547305 8.26 MB/sec execute 232 sec latency 15.321 ms 2024-03-21T11:31:09.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:08 smithi047 ceph-mon[31351]: pgmap v634: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2024-03-21T11:31:09.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:08 smithi100 ceph-mon[36245]: pgmap v634: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2024-03-21T11:31:09.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:08 smithi102 ceph-mon[36748]: pgmap v634: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2024-03-21T11:31:09.533 INFO:tasks.workunit.client.0.smithi047.stdout: 1 548351 8.23 MB/sec execute 233 sec latency 14.674 ms 2024-03-21T11:31:10.533 INFO:tasks.workunit.client.0.smithi047.stdout: 1 550685 8.26 MB/sec execute 234 sec latency 20.556 ms 2024-03-21T11:31:10.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:31:10.839 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:31:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.093+0000 7f36e5400640 1 -- 172.21.15.47:0/273756093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36e0105470 msgr2=0x7f36e010bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.093+0000 7f36e5400640 1 --2- 172.21.15.47:0/273756093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36e0105470 0x7f36e010bd00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009900 tx=0x7f36d802bc40 comp rx=0 tx=0).stop 2024-03-21T11:31:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.094+0000 7f36e5400640 1 -- 172.21.15.47:0/273756093 shutdown_connections 2024-03-21T11:31:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.094+0000 7f36e5400640 1 --2- 172.21.15.47:0/273756093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36e0105470 0x7f36e010bd00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.094+0000 7f36e5400640 1 --2- 172.21.15.47:0/273756093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36e0104ab0 0x7f36e0104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.094+0000 7f36e5400640 1 --2- 172.21.15.47:0/273756093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36e01038c0 0x7f36e0103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.094+0000 7f36e5400640 1 -- 172.21.15.47:0/273756093 >> 172.21.15.47:0/273756093 conn(0x7f36e00fee50 msgr2=0x7f36e0101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.095+0000 7f36e5400640 1 -- 172.21.15.47:0/273756093 shutdown_connections 2024-03-21T11:31:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.095+0000 7f36e5400640 1 -- 172.21.15.47:0/273756093 wait complete. 2024-03-21T11:31:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.096+0000 7f36e5400640 1 Processor -- start 2024-03-21T11:31:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.098+0000 7f36e5400640 1 -- start start 2024-03-21T11:31:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.100+0000 7f36e5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36e01038c0 0x7f36e0110b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.100+0000 7f36e5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36e0104ab0 0x7f36e0111050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.100+0000 7f36e5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36e0105470 0x7f36e01adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.100+0000 7f36e5400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e00775e0 con 0x7f36e01038c0 2024-03-21T11:31:11.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.100+0000 7f36e5400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e0077460 con 0x7f36e0104ab0 2024-03-21T11:31:11.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.100+0000 7f36e5400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e0077760 con 0x7f36e0105470 2024-03-21T11:31:11.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.100+0000 7f36df200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36e0105470 0x7f36e01adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.101+0000 7f36df200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36e0105470 0x7f36e01adfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47994/0 (socket says 172.21.15.47:47994) 2024-03-21T11:31:11.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.101+0000 7f36df200640 1 -- 172.21.15.47:0/67107344 learned_addr learned my addr 172.21.15.47:0/67107344 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:11.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.101+0000 7f36dde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36e0104ab0 0x7f36e0111050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.101+0000 7f36de800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36e01038c0 0x7f36e0110b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.102+0000 7f36df200640 1 -- 172.21.15.47:0/67107344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36e0104ab0 msgr2=0x7f36e0111050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.102+0000 7f36df200640 1 --2- 172.21.15.47:0/67107344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36e0104ab0 0x7f36e0111050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.102+0000 7f36df200640 1 -- 172.21.15.47:0/67107344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36e01038c0 msgr2=0x7f36e0110b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.102+0000 7f36df200640 1 --2- 172.21.15.47:0/67107344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36e01038c0 0x7f36e0110b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.102+0000 7f36df200640 1 -- 172.21.15.47:0/67107344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e01ae520 con 0x7f36e0105470 2024-03-21T11:31:11.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.104+0000 7f36df200640 1 --2- 172.21.15.47:0/67107344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36e0105470 0x7f36e01adfe0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f36d802efd0 tx=0x7f36d8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:11.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.104+0000 7f36d7400640 1 -- 172.21.15.47:0/67107344 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8041070 con 0x7f36e0105470 2024-03-21T11:31:11.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.104+0000 7f36d7400640 1 -- 172.21.15.47:0/67107344 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36d80054e0 con 0x7f36e0105470 2024-03-21T11:31:11.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.104+0000 7f36e5400640 1 -- 172.21.15.47:0/67107344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e01ae6f0 con 0x7f36e0105470 2024-03-21T11:31:11.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.104+0000 7f36d7400640 1 -- 172.21.15.47:0/67107344 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8034440 con 0x7f36e0105470 2024-03-21T11:31:11.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.105+0000 7f36e5400640 1 -- 172.21.15.47:0/67107344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e01aebb0 con 0x7f36e0105470 2024-03-21T11:31:11.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.112+0000 7f36e8007740 1 Processor -- start 2024-03-21T11:31:11.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.113+0000 7f36e8007740 1 -- start start 2024-03-21T11:31:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.114+0000 7f36e8007740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259044e80 0x55d2590452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.114+0000 7f36e8007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d259045870 0x55d25904a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.114+0000 7f36e8007740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d25904a9e0 0x55d25904ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.114+0000 7f36e8007740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d258faa250 con 0x55d259044e80 2024-03-21T11:31:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.114+0000 7f36e8007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d258f52a30 con 0x55d259045870 2024-03-21T11:31:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.114+0000 7f36e8007740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d258fa62f0 con 0x55d25904a9e0 2024-03-21T11:31:11.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.114+0000 7f36d7400640 1 -- 172.21.15.47:0/67107344 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f36d8044070 con 0x7f36e0105470 2024-03-21T11:31:11.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.114+0000 7f36dde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259044e80 0x55d2590452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.115+0000 7f36de800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d259045870 0x55d25904a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.115+0000 7f36de800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d259045870 0x55d25904a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59776/0 (socket says 172.21.15.47:59776) 2024-03-21T11:31:11.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.115+0000 7f36de800640 1 -- 172.21.15.47:0/486765002 learned_addr learned my addr 172.21.15.47:0/486765002 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:11.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.115+0000 7f36dde00640 1 --2- 172.21.15.47:0/486765002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d25904a9e0 0x55d25904ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.116+0000 7f36de800640 1 -- 172.21.15.47:0/486765002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d25904a9e0 msgr2=0x55d25904ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.116+0000 7f36de800640 1 --2- 172.21.15.47:0/486765002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d25904a9e0 0x55d25904ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.116+0000 7f36de800640 1 -- 172.21.15.47:0/486765002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259044e80 msgr2=0x55d2590452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.116+0000 7f36de800640 1 --2- 172.21.15.47:0/486765002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259044e80 0x55d2590452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.116+0000 7f36de800640 1 -- 172.21.15.47:0/486765002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d258f3d690 con 0x55d259045870 2024-03-21T11:31:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.116+0000 7f36dde00640 1 --2- 172.21.15.47:0/486765002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259044e80 0x55d2590452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.118+0000 7f36de800640 1 --2- 172.21.15.47:0/486765002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d259045870 0x55d25904a370 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009f40 tx=0x7f36d0010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3666012a6c15d0e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.118+0000 7f36cf400640 1 -- 172.21.15.47:0/486765002 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0016070 con 0x55d259045870 2024-03-21T11:31:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.118+0000 7f36cf400640 1 -- 172.21.15.47:0/486765002 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36d0004d20 con 0x55d259045870 2024-03-21T11:31:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.119+0000 7f36cf400640 1 -- 172.21.15.47:0/486765002 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d00127e0 con 0x55d259045870 2024-03-21T11:31:11.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.119+0000 7f36d7400640 1 --2- 172.21.15.47:0/67107344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36c40a3120 0x7f36c40a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.119+0000 7f36e8007740 1 -- 172.21.15.47:0/486765002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d259045870 msgr2=0x55d25904a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.119+0000 7f36e8007740 1 --2- 172.21.15.47:0/486765002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d259045870 0x55d25904a370 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009f40 tx=0x7f36d0010800 comp rx=0 tx=0).stop 2024-03-21T11:31:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.120+0000 7f36d7400640 1 -- 172.21.15.47:0/67107344 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f36d80f1490 con 0x7f36e0105470 2024-03-21T11:31:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.120+0000 7f36e8007740 1 -- 172.21.15.47:0/486765002 shutdown_connections 2024-03-21T11:31:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.120+0000 7f36e8007740 1 --2- 172.21.15.47:0/486765002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d25904a9e0 0x55d25904ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.120+0000 7f36e8007740 1 --2- 172.21.15.47:0/486765002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d259045870 0x55d25904a370 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.120+0000 7f36e8007740 1 --2- 172.21.15.47:0/486765002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259044e80 0x55d2590452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.120+0000 7f36e8007740 1 -- 172.21.15.47:0/486765002 >> 172.21.15.47:0/486765002 conn(0x55d25900d200 msgr2=0x55d2590407f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.121+0000 7f36e8007740 1 -- 172.21.15.47:0/486765002 shutdown_connections 2024-03-21T11:31:11.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.121+0000 7f36e8007740 1 -- 172.21.15.47:0/486765002 wait complete. 2024-03-21T11:31:11.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.122+0000 7f36e8007740 1 -- start start 2024-03-21T11:31:11.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.122+0000 7f36e8007740 1 Processor -- start 2024-03-21T11:31:11.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.123+0000 7f36dde00640 1 --2- 172.21.15.47:0/67107344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36c40a3120 0x7f36c40a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.124+0000 7f36e8007740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259044e80 0x55d25904ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.124+0000 7f36e8007740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259045870 0x55d25904f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.124+0000 7f36e8007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d25904a9e0 0x55d259056240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.126+0000 7f36e8007740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25904d9e0 con 0x55d259045870 2024-03-21T11:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.126+0000 7f36e8007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d258f65e80 con 0x55d25904a9e0 2024-03-21T11:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.126+0000 7f36e8007740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25904db60 con 0x55d259044e80 2024-03-21T11:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.126+0000 7f36df200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d25904a9e0 0x55d259056240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.126+0000 7f36df200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d25904a9e0 0x55d259056240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59792/0 (socket says 172.21.15.47:59792) 2024-03-21T11:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.126+0000 7f36df200640 1 -- 172.21.15.47:0/1010413594 learned_addr learned my addr 172.21.15.47:0/1010413594 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.126+0000 7f36de800640 1 --2- 172.21.15.47:0/1010413594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259044e80 0x55d25904ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.126+0000 7f36e8007740 10 client.0 _ll_register_callbacks cb 0x55d258f78130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.126+0000 7f36dde00640 1 --2- 172.21.15.47:0/67107344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36c40a3120 0x7f36c40a55e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55d2590460d0 tx=0x7f36c8006a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.126+0000 7f36dde00640 1 --2- 172.21.15.47:0/1010413594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259045870 0x55d25904f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.127+0000 7f36df200640 1 -- 172.21.15.47:0/1010413594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259044e80 msgr2=0x55d25904ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.127+0000 7f36df200640 1 --2- 172.21.15.47:0/1010413594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259044e80 0x55d25904ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.127+0000 7f36df200640 1 -- 172.21.15.47:0/1010413594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259045870 msgr2=0x55d25904f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.127+0000 7f36df200640 1 --2- 172.21.15.47:0/1010413594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259045870 0x55d25904f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.127+0000 7f36df200640 1 -- 172.21.15.47:0/1010413594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d258d887d0 con 0x55d25904a9e0 2024-03-21T11:31:11.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.127+0000 7f36dde00640 1 --2- 172.21.15.47:0/1010413594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259045870 0x55d25904f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:31:11.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.127+0000 7f36df200640 1 --2- 172.21.15.47:0/1010413594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d25904a9e0 0x55d259056240 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f36d80c5cb0 tx=0x7f36d80bd660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830c53e630814473 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:11.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.127+0000 7f36c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:11.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.127+0000 7f36c3e00640 1 -- 172.21.15.47:0/1010413594 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d804b040 con 0x55d25904a9e0 2024-03-21T11:31:11.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.128+0000 7f36c3e00640 1 -- 172.21.15.47:0/1010413594 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f36d8003710 con 0x55d25904a9e0 2024-03-21T11:31:11.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.128+0000 7f36e8007740 1 -- 172.21.15.47:0/1010413594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36d0004d20 con 0x55d25904a9e0 2024-03-21T11:31:11.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.128+0000 7f36c3e00640 1 -- 172.21.15.47:0/1010413594 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36d8005690 con 0x55d25904a9e0 2024-03-21T11:31:11.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.128+0000 7f36e8007740 10 client.28061 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:11.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.128+0000 7f36e8007740 1 -- 172.21.15.47:0/1010413594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d258cbe3f0 con 0x55d25904a9e0 2024-03-21T11:31:11.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.129+0000 7f36c3e00640 1 -- 172.21.15.47:0/1010413594 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36d0004d20 con 0x55d25904a9e0 2024-03-21T11:31:11.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.129+0000 7f36c3e00640 1 -- 172.21.15.47:0/1010413594 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f36d80c5d50 con 0x55d25904a9e0 2024-03-21T11:31:11.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.129+0000 7f36e8007740 10 client.28061 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:11.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.129+0000 7f36e8007740 10 client.28061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:11.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.129+0000 7f36e8007740 20 client.28061 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:11.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.129+0000 7f36e8007740 1 --2- 172.21.15.47:0/1010413594 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d25905a990 0x55d25905cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.129+0000 7f36e8007740 4 client.28061 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:11.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.129+0000 7f36e8007740 1 -- 172.21.15.47:0/1010413594 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d258fb7790 con 0x55d25905a990 2024-03-21T11:31:11.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.130+0000 7f36de800640 1 --2- 172.21.15.47:0/1010413594 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d25905a990 0x55d25905cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.130+0000 7f36de800640 1 --2- 172.21.15.47:0/1010413594 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d25905a990 0x55d25905cdd0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3473480a2730a9fd server_cookie=51181f1d48e670cf in_seq=0 out_seq=0 2024-03-21T11:31:11.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.130+0000 7f36c3e00640 10 client.28061 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:11.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.131+0000 7f36c3e00640 1 -- 172.21.15.47:0/1010413594 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d258fb7790 con 0x55d25905a990 2024-03-21T11:31:11.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.131+0000 7f36c3e00640 10 client.28061 handle_command_reply: tid=0 2024-03-21T11:31:11.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.132+0000 7f36e8007740 1 client.28061 shutdown 2024-03-21T11:31:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.132+0000 7f36e8007740 1 -- 172.21.15.47:0/1010413594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d25904a9e0 msgr2=0x55d259056240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.132+0000 7f36e8007740 1 --2- 172.21.15.47:0/1010413594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d25904a9e0 0x55d259056240 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f36d80c5cb0 tx=0x7f36d80bd660 comp rx=0 tx=0).stop 2024-03-21T11:31:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.132+0000 7f36e8007740 1 -- 172.21.15.47:0/1010413594 shutdown_connections 2024-03-21T11:31:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.132+0000 7f36e8007740 1 --2- 172.21.15.47:0/1010413594 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d25905a990 0x55d25905cdd0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.132+0000 7f36e8007740 1 --2- 172.21.15.47:0/1010413594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d25904a9e0 0x55d259056240 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.132+0000 7f36e8007740 1 --2- 172.21.15.47:0/1010413594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259045870 0x55d25904f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.132+0000 7f36e8007740 1 --2- 172.21.15.47:0/1010413594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259044e80 0x55d25904ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.132+0000 7f36c3e00640 0 client.28061 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.132+0000 7f36e8007740 1 -- 172.21.15.47:0/1010413594 >> 172.21.15.47:0/1010413594 conn(0x55d25900d200 msgr2=0x55d25903fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.132+0000 7f36e8007740 1 -- 172.21.15.47:0/1010413594 shutdown_connections 2024-03-21T11:31:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.132+0000 7f36e8007740 1 -- 172.21.15.47:0/1010413594 wait complete. 2024-03-21T11:31:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.132+0000 7f36e8007740 20 client.28061 trim_cache size 0 max 16384 2024-03-21T11:31:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.151+0000 7f36e8007740 1 Processor -- start 2024-03-21T11:31:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.151+0000 7f36e8007740 1 -- start start 2024-03-21T11:31:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36e8007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d259044e80 0x55d25904a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36e8007740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259045870 0x55d259056200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36e8007740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d25904a9e0 0x55d259043a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36e8007740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e01b06f0 con 0x55d25904a9e0 2024-03-21T11:31:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36e8007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e01b2070 con 0x55d259044e80 2024-03-21T11:31:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36e8007740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e01b0d50 con 0x55d259045870 2024-03-21T11:31:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36de800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d259044e80 0x55d25904a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36dde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259045870 0x55d259056200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36de800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d259044e80 0x55d25904a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59808/0 (socket says 172.21.15.47:59808) 2024-03-21T11:31:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36de800640 1 -- 172.21.15.47:0/1699537435 learned_addr learned my addr 172.21.15.47:0/1699537435 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36df200640 1 --2- 172.21.15.47:0/1699537435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d25904a9e0 0x55d259043a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36de800640 1 -- 172.21.15.47:0/1699537435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259045870 msgr2=0x55d259056200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36de800640 1 --2- 172.21.15.47:0/1699537435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259045870 0x55d259056200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36de800640 1 -- 172.21.15.47:0/1699537435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d25904a9e0 msgr2=0x55d259043a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36de800640 1 --2- 172.21.15.47:0/1699537435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d25904a9e0 0x55d259043a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36de800640 1 -- 172.21.15.47:0/1699537435 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d258c7e0f0 con 0x55d259044e80 2024-03-21T11:31:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36df200640 1 --2- 172.21.15.47:0/1699537435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d25904a9e0 0x55d259043a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.152+0000 7f36dde00640 1 --2- 172.21.15.47:0/1699537435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259045870 0x55d259056200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.153+0000 7f36de800640 1 --2- 172.21.15.47:0/1699537435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d259044e80 0x55d25904a390 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009f40 tx=0x7f36d0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9527ad636b346dcd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.153+0000 7f36cea00640 1 -- 172.21.15.47:0/1699537435 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d002c070 con 0x55d259044e80 2024-03-21T11:31:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.153+0000 7f36cea00640 1 -- 172.21.15.47:0/1699537435 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36d00272d0 con 0x55d259044e80 2024-03-21T11:31:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.153+0000 7f36cea00640 1 -- 172.21.15.47:0/1699537435 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0016040 con 0x55d259044e80 2024-03-21T11:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.154+0000 7f36e8007740 1 -- 172.21.15.47:0/1699537435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d259044e80 msgr2=0x55d25904a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.154+0000 7f36e8007740 1 --2- 172.21.15.47:0/1699537435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d259044e80 0x55d25904a390 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009f40 tx=0x7f36d0005fd0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.154+0000 7f36e8007740 1 -- 172.21.15.47:0/1699537435 shutdown_connections 2024-03-21T11:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.154+0000 7f36e8007740 1 --2- 172.21.15.47:0/1699537435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d25904a9e0 0x55d259043a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.154+0000 7f36e8007740 1 --2- 172.21.15.47:0/1699537435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259045870 0x55d259056200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.154+0000 7f36e8007740 1 --2- 172.21.15.47:0/1699537435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d259044e80 0x55d25904a390 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.154+0000 7f36e8007740 1 -- 172.21.15.47:0/1699537435 >> 172.21.15.47:0/1699537435 conn(0x55d25900d200 msgr2=0x55d259049c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.154+0000 7f36e8007740 1 -- 172.21.15.47:0/1699537435 shutdown_connections 2024-03-21T11:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.154+0000 7f36e8007740 1 -- 172.21.15.47:0/1699537435 wait complete. 2024-03-21T11:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.155+0000 7f36e8007740 1 -- start start 2024-03-21T11:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.155+0000 7f36e8007740 1 Processor -- start 2024-03-21T11:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.155+0000 7f36e8007740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259044e80 0x55d259053dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.155+0000 7f36e8007740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259045870 0x55d259054310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.155+0000 7f36e8007740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d25904a9e0 0x55d2590552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.155+0000 7f36e8007740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e01b1a10 con 0x55d259045870 2024-03-21T11:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.155+0000 7f36e8007740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e01b13b0 con 0x55d25904a9e0 2024-03-21T11:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.155+0000 7f36e8007740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e0073370 con 0x55d259044e80 2024-03-21T11:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.155+0000 7f36e8007740 10 client.0 _ll_register_callbacks cb 0x7f36d8009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.156+0000 7f36de800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259044e80 0x55d259053dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.156+0000 7f36de800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259044e80 0x55d259053dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48038/0 (socket says 172.21.15.47:48038) 2024-03-21T11:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.156+0000 7f36de800640 1 -- 172.21.15.47:0/1990525552 learned_addr learned my addr 172.21.15.47:0/1990525552 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.156+0000 7f36dde00640 1 --2- 172.21.15.47:0/1990525552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259045870 0x55d259054310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.156+0000 7f36df200640 1 --2- 172.21.15.47:0/1990525552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d25904a9e0 0x55d2590552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.156+0000 7f36de800640 1 -- 172.21.15.47:0/1990525552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d25904a9e0 msgr2=0x55d2590552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.156+0000 7f36de800640 1 --2- 172.21.15.47:0/1990525552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d25904a9e0 0x55d2590552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.156+0000 7f36de800640 1 -- 172.21.15.47:0/1990525552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259045870 msgr2=0x55d259054310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.156+0000 7f36de800640 1 --2- 172.21.15.47:0/1990525552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259045870 0x55d259054310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.156+0000 7f36de800640 1 -- 172.21.15.47:0/1990525552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2590558c0 con 0x55d259044e80 2024-03-21T11:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.156+0000 7f36dde00640 1 --2- 172.21.15.47:0/1990525552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259045870 0x55d259054310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.156+0000 7f36de800640 1 --2- 172.21.15.47:0/1990525552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259044e80 0x55d259053dd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f36d0010d00 tx=0x7f36d00278c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fedb384d5931c675 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.156+0000 7f36c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.157+0000 7f36c3e00640 1 -- 172.21.15.47:0/1990525552 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0035060 con 0x55d259044e80 2024-03-21T11:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.157+0000 7f36e8007740 1 -- 172.21.15.47:0/1990525552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36d00272d0 con 0x55d259044e80 2024-03-21T11:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.157+0000 7f36c3e00640 1 -- 172.21.15.47:0/1990525552 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f36d0027ab0 con 0x55d259044e80 2024-03-21T11:31:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.157+0000 7f36c3e00640 1 -- 172.21.15.47:0/1990525552 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36d003c050 con 0x55d259044e80 2024-03-21T11:31:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.157+0000 7f36e8007740 10 client.28078 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.157+0000 7f36e8007740 1 -- 172.21.15.47:0/1990525552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d2590b6610 con 0x55d259044e80 2024-03-21T11:31:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.157+0000 7f36c3e00640 1 -- 172.21.15.47:0/1990525552 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36d00272d0 con 0x55d259044e80 2024-03-21T11:31:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.158+0000 7f36c3e00640 1 -- 172.21.15.47:0/1990525552 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f36d002f070 con 0x55d259044e80 2024-03-21T11:31:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.158+0000 7f36e8007740 10 client.28078 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.158+0000 7f36e8007740 10 client.28078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.158+0000 7f36e8007740 20 client.28078 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.158+0000 7f36e8007740 1 --2- 172.21.15.47:0/1990525552 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d2590b6d20 0x55d2590b9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.158+0000 7f36e8007740 4 client.28078 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:31:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.158+0000 7f36e8007740 1 -- 172.21.15.47:0/1990525552 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d258fb1ae0 con 0x55d2590b6d20 2024-03-21T11:31:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.158+0000 7f36dde00640 1 --2- 172.21.15.47:0/1990525552 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d2590b6d20 0x55d2590b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36dde00640 1 --2- 172.21.15.47:0/1990525552 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d2590b6d20 0x55d2590b9110 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=221484bcd9db7a3e server_cookie=7f26810675296a0c in_seq=0 out_seq=0 2024-03-21T11:31:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36c3e00640 10 client.28078 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36c3e00640 1 -- 172.21.15.47:0/1990525552 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d258fb1ae0 con 0x55d2590b6d20 2024-03-21T11:31:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36c3e00640 10 client.28078 handle_command_reply: tid=0 2024-03-21T11:31:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36e8007740 1 client.28078 shutdown 2024-03-21T11:31:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36e8007740 1 -- 172.21.15.47:0/1990525552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259044e80 msgr2=0x55d259053dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36e8007740 1 --2- 172.21.15.47:0/1990525552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259044e80 0x55d259053dd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f36d0010d00 tx=0x7f36d00278c0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36e8007740 1 -- 172.21.15.47:0/1990525552 shutdown_connections 2024-03-21T11:31:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36e8007740 1 --2- 172.21.15.47:0/1990525552 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d2590b6d20 0x55d2590b9110 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36e8007740 1 --2- 172.21.15.47:0/1990525552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d25904a9e0 0x55d2590552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36e8007740 1 --2- 172.21.15.47:0/1990525552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d259045870 0x55d259054310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36e8007740 1 --2- 172.21.15.47:0/1990525552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d259044e80 0x55d259053dd0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36e8007740 1 -- 172.21.15.47:0/1990525552 >> 172.21.15.47:0/1990525552 conn(0x55d25900d200 msgr2=0x55d25904a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36c3e00640 0 client.28078 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.159+0000 7f36e8007740 1 -- 172.21.15.47:0/1990525552 shutdown_connections 2024-03-21T11:31:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.160+0000 7f36e8007740 1 -- 172.21.15.47:0/1990525552 wait complete. 2024-03-21T11:31:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.160+0000 7f36e8007740 20 client.28078 trim_cache size 0 max 16384 2024-03-21T11:31:11.164 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:31:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:31:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:31:11.165 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:31:11.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.160+0000 7f36e5400640 1 -- 172.21.15.47:0/67107344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36c40a3120 msgr2=0x7f36c40a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.160+0000 7f36e5400640 1 --2- 172.21.15.47:0/67107344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36c40a3120 0x7f36c40a55e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55d2590460d0 tx=0x7f36c8006a30 comp rx=0 tx=0).stop 2024-03-21T11:31:11.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.160+0000 7f36e5400640 1 -- 172.21.15.47:0/67107344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36e0105470 msgr2=0x7f36e01adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.160+0000 7f36e5400640 1 --2- 172.21.15.47:0/67107344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36e0105470 0x7f36e01adfe0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f36d802efd0 tx=0x7f36d8004040 comp rx=0 tx=0).stop 2024-03-21T11:31:11.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.161+0000 7f36e5400640 1 -- 172.21.15.47:0/67107344 shutdown_connections 2024-03-21T11:31:11.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.161+0000 7f36e5400640 1 --2- 172.21.15.47:0/67107344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36c40a3120 0x7f36c40a55e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.161+0000 7f36e5400640 1 --2- 172.21.15.47:0/67107344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36e0105470 0x7f36e01adfe0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.161+0000 7f36e5400640 1 --2- 172.21.15.47:0/67107344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36e0104ab0 0x7f36e0111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.161+0000 7f36e5400640 1 --2- 172.21.15.47:0/67107344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36e01038c0 0x7f36e0110b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.161+0000 7f36e5400640 1 -- 172.21.15.47:0/67107344 >> 172.21.15.47:0/67107344 conn(0x7f36e00fee50 msgr2=0x7f36e0105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:11.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.161+0000 7f36e5400640 1 -- 172.21.15.47:0/67107344 shutdown_connections 2024-03-21T11:31:11.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.161+0000 7f36e5400640 1 -- 172.21.15.47:0/67107344 wait complete. 2024-03-21T11:31:11.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:11 smithi047 ceph-mon[31351]: pgmap v635: 129 pgs: 129 active+clean; 39 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 295 op/s 2024-03-21T11:31:11.179 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:31:11.179 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:31:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.235+0000 7f2cdfc00640 1 -- 172.21.15.47:0/3866959545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cd8104af0 msgr2=0x7f2cd8104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.235+0000 7f2cdfc00640 1 --2- 172.21.15.47:0/3866959545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cd8104af0 0x7f2cd8104f70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009900 tx=0x7f2cc402bc30 comp rx=0 tx=0).stop 2024-03-21T11:31:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.235+0000 7f2cdfc00640 1 -- 172.21.15.47:0/3866959545 shutdown_connections 2024-03-21T11:31:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.235+0000 7f2cdfc00640 1 --2- 172.21.15.47:0/3866959545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd81054b0 0x7f2cd810bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.235+0000 7f2cdfc00640 1 --2- 172.21.15.47:0/3866959545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cd8104af0 0x7f2cd8104f70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.235+0000 7f2cdfc00640 1 --2- 172.21.15.47:0/3866959545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cd8103900 0x7f2cd8103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.235+0000 7f2cdfc00640 1 -- 172.21.15.47:0/3866959545 >> 172.21.15.47:0/3866959545 conn(0x7f2cd80fee90 msgr2=0x7f2cd81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.235+0000 7f2cdfc00640 1 -- 172.21.15.47:0/3866959545 shutdown_connections 2024-03-21T11:31:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.235+0000 7f2cdfc00640 1 -- 172.21.15.47:0/3866959545 wait complete. 2024-03-21T11:31:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.236+0000 7f2cdfc00640 1 Processor -- start 2024-03-21T11:31:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.236+0000 7f2cdfc00640 1 -- start start 2024-03-21T11:31:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.237+0000 7f2cdfc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd8103900 0x7f2cd8199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.237+0000 7f2cdfc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cd8104af0 0x7f2cd819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.237+0000 7f2cdfc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cd81054b0 0x7f2cd81a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.237+0000 7f2cdfc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd810f560 con 0x7f2cd81054b0 2024-03-21T11:31:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.237+0000 7f2cdfc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd810f3e0 con 0x7f2cd8103900 2024-03-21T11:31:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.237+0000 7f2cdfc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd810f6e0 con 0x7f2cd8104af0 2024-03-21T11:31:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.237+0000 7f2cdd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd8103900 0x7f2cd8199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.237+0000 7f2cdd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd8103900 0x7f2cd8199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59820/0 (socket says 172.21.15.47:59820) 2024-03-21T11:31:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.237+0000 7f2cdd200640 1 -- 172.21.15.47:0/1247267653 learned_addr learned my addr 172.21.15.47:0/1247267653 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.237+0000 7f2cd7e00640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cd8104af0 0x7f2cd819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.237+0000 7f2cddc00640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cd81054b0 0x7f2cd81a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.237+0000 7f2cdd200640 1 -- 172.21.15.47:0/1247267653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cd8104af0 msgr2=0x7f2cd819a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.238+0000 7f2cdd200640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cd8104af0 0x7f2cd819a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.238+0000 7f2cdd200640 1 -- 172.21.15.47:0/1247267653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cd81054b0 msgr2=0x7f2cd81a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.238+0000 7f2cdd200640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cd81054b0 0x7f2cd81a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.238+0000 7f2cdd200640 1 -- 172.21.15.47:0/1247267653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd81a17b0 con 0x7f2cd8103900 2024-03-21T11:31:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.238+0000 7f2cddc00640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cd81054b0 0x7f2cd81a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.238+0000 7f2cdd200640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd8103900 0x7f2cd8199b90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00e9c0 tx=0x7f2ccc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.238+0000 7f2cd5600640 1 -- 172.21.15.47:0/1247267653 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc00cd80 con 0x7f2cd8103900 2024-03-21T11:31:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.238+0000 7f2cd5600640 1 -- 172.21.15.47:0/1247267653 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ccc0035f0 con 0x7f2cd8103900 2024-03-21T11:31:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.238+0000 7f2cdfc00640 1 -- 172.21.15.47:0/1247267653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd81a1aa0 con 0x7f2cd8103900 2024-03-21T11:31:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.238+0000 7f2cd5600640 1 -- 172.21.15.47:0/1247267653 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc010690 con 0x7f2cd8103900 2024-03-21T11:31:11.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.238+0000 7f2cdfc00640 1 -- 172.21.15.47:0/1247267653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd81a1fc0 con 0x7f2cd8103900 2024-03-21T11:31:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.242+0000 7f2cd5600640 1 -- 172.21.15.47:0/1247267653 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2ccc0037a0 con 0x7f2cd8103900 2024-03-21T11:31:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.242+0000 7f2ce27e0740 1 Processor -- start 2024-03-21T11:31:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.242+0000 7f2ce27e0740 1 -- start start 2024-03-21T11:31:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2ce27e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b91e0 0x5648254b9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2ce27e0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b9bd0 0x5648254be6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2ce27e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 0x5648254c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2ce27e0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482547c3b0 con 0x5648254b9bd0 2024-03-21T11:31:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2ce27e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482541e830 con 0x5648254bed40 2024-03-21T11:31:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2ce27e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648253c75d0 con 0x5648254b91e0 2024-03-21T11:31:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2cddc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b91e0 0x5648254b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2cddc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b91e0 0x5648254b9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48066/0 (socket says 172.21.15.47:48066) 2024-03-21T11:31:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2cddc00640 1 -- 172.21.15.47:0/3242423452 learned_addr learned my addr 172.21.15.47:0/3242423452 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2cd7e00640 1 --2- 172.21.15.47:0/3242423452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 0x5648254c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2cdd200640 1 --2- 172.21.15.47:0/3242423452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b9bd0 0x5648254be6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2cddc00640 1 -- 172.21.15.47:0/3242423452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 msgr2=0x5648254c11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2cddc00640 1 --2- 172.21.15.47:0/3242423452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 0x5648254c11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2cddc00640 1 -- 172.21.15.47:0/3242423452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b9bd0 msgr2=0x5648254be6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2cddc00640 1 --2- 172.21.15.47:0/3242423452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b9bd0 0x5648254be6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2cddc00640 1 -- 172.21.15.47:0/3242423452 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648253df2c0 con 0x5648254b91e0 2024-03-21T11:31:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.243+0000 7f2cdd200640 1 --2- 172.21.15.47:0/3242423452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b9bd0 0x5648254be6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:31:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.244+0000 7f2cd5600640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2cc00a2fc0 0x7f2cc00a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.244+0000 7f2cddc00640 1 --2- 172.21.15.47:0/3242423452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b91e0 0x5648254b9600 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009fd0 tx=0x7f2cd0010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3244ce6fbd8b4a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.244+0000 7f2cd5600640 1 -- 172.21.15.47:0/1247267653 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2ccc0cfc60 con 0x7f2cd8103900 2024-03-21T11:31:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.244+0000 7f2cdd200640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2cc00a2fc0 0x7f2cc00a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.244+0000 7f2cca000640 1 -- 172.21.15.47:0/3242423452 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0011700 con 0x5648254b91e0 2024-03-21T11:31:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.244+0000 7f2cca000640 1 -- 172.21.15.47:0/3242423452 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cd0011d80 con 0x5648254b91e0 2024-03-21T11:31:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.244+0000 7f2cca000640 1 -- 172.21.15.47:0/3242423452 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd000e3c0 con 0x5648254b91e0 2024-03-21T11:31:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.245+0000 7f2ce27e0740 1 -- 172.21.15.47:0/3242423452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b91e0 msgr2=0x5648254b9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.246+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/3242423452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b91e0 0x5648254b9600 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009fd0 tx=0x7f2cd0010860 comp rx=0 tx=0).stop 2024-03-21T11:31:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.245+0000 7f2cdd200640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2cc00a2fc0 0x7f2cc00a5480 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc098970 tx=0x7f2ccc0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.246+0000 7f2ce27e0740 1 -- 172.21.15.47:0/3242423452 shutdown_connections 2024-03-21T11:31:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.246+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/3242423452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 0x5648254c11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.246+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/3242423452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b9bd0 0x5648254be6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.246+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/3242423452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b91e0 0x5648254b9600 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.246+0000 7f2ce27e0740 1 -- 172.21.15.47:0/3242423452 >> 172.21.15.47:0/3242423452 conn(0x5648254b4750 msgr2=0x5648254b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.246+0000 7f2ce27e0740 1 -- 172.21.15.47:0/3242423452 shutdown_connections 2024-03-21T11:31:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.246+0000 7f2ce27e0740 1 -- 172.21.15.47:0/3242423452 wait complete. 2024-03-21T11:31:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.246+0000 7f2ce27e0740 1 -- start start 2024-03-21T11:31:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.247+0000 7f2ce27e0740 1 Processor -- start 2024-03-21T11:31:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.247+0000 7f2ce27e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b91e0 0x5648254c33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2ce27e0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b9bd0 0x5648254c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2ce27e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 0x5648254ca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2ce27e0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648253daa20 con 0x5648254b9bd0 2024-03-21T11:31:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2ce27e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482541a8d0 con 0x5648254bed40 2024-03-21T11:31:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2ce27e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648254c1cf0 con 0x5648254b91e0 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2ce27e0740 10 client.0 _ll_register_callbacks cb 0x5648254171c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2cddc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b91e0 0x5648254c33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2cddc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b91e0 0x5648254c33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48082/0 (socket says 172.21.15.47:48082) 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2cddc00640 1 -- 172.21.15.47:0/1816483083 learned_addr learned my addr 172.21.15.47:0/1816483083 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2cd7e00640 1 --2- 172.21.15.47:0/1816483083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b9bd0 0x5648254c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2cddc00640 1 --2- 172.21.15.47:0/1816483083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 0x5648254ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2cddc00640 1 -- 172.21.15.47:0/1816483083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 msgr2=0x5648254ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2cddc00640 1 --2- 172.21.15.47:0/1816483083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 0x5648254ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2cddc00640 1 -- 172.21.15.47:0/1816483083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b9bd0 msgr2=0x5648254c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2cddc00640 1 --2- 172.21.15.47:0/1816483083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b9bd0 0x5648254c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.248+0000 7f2cddc00640 1 -- 172.21.15.47:0/1816483083 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564825133210 con 0x5648254b91e0 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.249+0000 7f2cddc00640 1 --2- 172.21.15.47:0/1816483083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b91e0 0x5648254c33c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005cc0 tx=0x7f2cd0009280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d090da1c6cdae0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.249+0000 7f2cbea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.249+0000 7f2cbea00640 1 -- 172.21.15.47:0/1816483083 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd002e060 con 0x5648254b91e0 2024-03-21T11:31:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.249+0000 7f2cbea00640 1 -- 172.21.15.47:0/1816483083 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2cd0011700 con 0x5648254b91e0 2024-03-21T11:31:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.249+0000 7f2ce27e0740 1 -- 172.21.15.47:0/1816483083 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cd0011d80 con 0x5648254b91e0 2024-03-21T11:31:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.249+0000 7f2cbea00640 1 -- 172.21.15.47:0/1816483083 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cd0035050 con 0x5648254b91e0 2024-03-21T11:31:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.249+0000 7f2ce27e0740 10 client.28090 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.249+0000 7f2ce27e0740 1 -- 172.21.15.47:0/1816483083 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5648251ff620 con 0x5648254b91e0 2024-03-21T11:31:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.249+0000 7f2cbea00640 1 -- 172.21.15.47:0/1816483083 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cd0011d80 con 0x5648254b91e0 2024-03-21T11:31:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.250+0000 7f2cbea00640 1 -- 172.21.15.47:0/1816483083 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2cd000ed20 con 0x5648254b91e0 2024-03-21T11:31:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.250+0000 7f2ce27e0740 10 client.28090 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.250+0000 7f2ce27e0740 10 client.28090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.250+0000 7f2ce27e0740 20 client.28090 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.250+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/1816483083 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5648254cee60 0x5648254d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.250+0000 7f2ce27e0740 4 client.28090 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.250+0000 7f2ce27e0740 1 -- 172.21.15.47:0/1816483083 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56482542bd70 con 0x5648254cee60 2024-03-21T11:31:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.250+0000 7f2cd7e00640 1 --2- 172.21.15.47:0/1816483083 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5648254cee60 0x5648254d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.251+0000 7f2cd7e00640 1 --2- 172.21.15.47:0/1816483083 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5648254cee60 0x5648254d1250 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7021bf3cae9ef840 server_cookie=e3d7c22144b8bc62 in_seq=0 out_seq=0 2024-03-21T11:31:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.251+0000 7f2cbea00640 10 client.28090 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.252+0000 7f2cbea00640 1 -- 172.21.15.47:0/1816483083 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56482542bd70 con 0x5648254cee60 2024-03-21T11:31:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.252+0000 7f2cbea00640 10 client.28090 handle_command_reply: tid=0 2024-03-21T11:31:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.253+0000 7f2ce27e0740 1 client.28090 shutdown 2024-03-21T11:31:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.253+0000 7f2ce27e0740 1 -- 172.21.15.47:0/1816483083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b91e0 msgr2=0x5648254c33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.253+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/1816483083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b91e0 0x5648254c33c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005cc0 tx=0x7f2cd0009280 comp rx=0 tx=0).stop 2024-03-21T11:31:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.253+0000 7f2ce27e0740 1 -- 172.21.15.47:0/1816483083 shutdown_connections 2024-03-21T11:31:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.253+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/1816483083 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5648254cee60 0x5648254d1250 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.253+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/1816483083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 0x5648254ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.253+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/1816483083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b9bd0 0x5648254c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.253+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/1816483083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b91e0 0x5648254c33c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.253+0000 7f2cbea00640 0 client.28090 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.253+0000 7f2ce27e0740 1 -- 172.21.15.47:0/1816483083 >> 172.21.15.47:0/1816483083 conn(0x5648254b4750 msgr2=0x5648254b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.253+0000 7f2ce27e0740 1 -- 172.21.15.47:0/1816483083 shutdown_connections 2024-03-21T11:31:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.253+0000 7f2ce27e0740 1 -- 172.21.15.47:0/1816483083 wait complete. 2024-03-21T11:31:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.253+0000 7f2ce27e0740 20 client.28090 trim_cache size 0 max 16384 2024-03-21T11:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.269+0000 7f2ce27e0740 1 Processor -- start 2024-03-21T11:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.269+0000 7f2ce27e0740 1 -- start start 2024-03-21T11:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.269+0000 7f2ce27e0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b91e0 0x5648254c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.269+0000 7f2ce27e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254b9bd0 0x5648254ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.269+0000 7f2ce27e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254bed40 0x5648254d5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.269+0000 7f2ce27e0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd81ac320 con 0x5648254b91e0 2024-03-21T11:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.269+0000 7f2ce27e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd81adca0 con 0x5648254b9bd0 2024-03-21T11:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.269+0000 7f2ce27e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd81ac980 con 0x5648254bed40 2024-03-21T11:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.269+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254b9bd0 0x5648254ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.269+0000 7f2cddc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b91e0 0x5648254c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.269+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254b9bd0 0x5648254ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59846/0 (socket says 172.21.15.47:59846) 2024-03-21T11:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.269+0000 7f2cd7e00640 1 -- 172.21.15.47:0/3513259771 learned_addr learned my addr 172.21.15.47:0/3513259771 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.270+0000 7f2cddc00640 1 --2- 172.21.15.47:0/3513259771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254bed40 0x5648254d5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.270+0000 7f2cd7e00640 1 -- 172.21.15.47:0/3513259771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254bed40 msgr2=0x5648254d5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.270+0000 7f2cd7e00640 1 --2- 172.21.15.47:0/3513259771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254bed40 0x5648254d5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.270+0000 7f2cd7e00640 1 -- 172.21.15.47:0/3513259771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b91e0 msgr2=0x5648254c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.270+0000 7f2cd7e00640 1 --2- 172.21.15.47:0/3513259771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b91e0 0x5648254c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.270+0000 7f2cd7e00640 1 -- 172.21.15.47:0/3513259771 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648254d1830 con 0x5648254b9bd0 2024-03-21T11:31:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.270+0000 7f2cd7e00640 1 --2- 172.21.15.47:0/3513259771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254b9bd0 0x5648254ca900 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2cc402cfd0 tx=0x7f2cc402c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f407246e31dbc02 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.270+0000 7f2cc9600640 1 -- 172.21.15.47:0/3513259771 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4004040 con 0x5648254b9bd0 2024-03-21T11:31:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.271+0000 7f2cc9600640 1 -- 172.21.15.47:0/3513259771 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cc404b070 con 0x5648254b9bd0 2024-03-21T11:31:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.271+0000 7f2cc9600640 1 -- 172.21.15.47:0/3513259771 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4004920 con 0x5648254b9bd0 2024-03-21T11:31:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.272+0000 7f2ce27e0740 1 -- 172.21.15.47:0/3513259771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254b9bd0 msgr2=0x5648254ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.272+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/3513259771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254b9bd0 0x5648254ca900 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2cc402cfd0 tx=0x7f2cc402c680 comp rx=0 tx=0).stop 2024-03-21T11:31:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.272+0000 7f2ce27e0740 1 -- 172.21.15.47:0/3513259771 shutdown_connections 2024-03-21T11:31:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.272+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/3513259771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254bed40 0x5648254d5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.272+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/3513259771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254b9bd0 0x5648254ca900 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.272+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/3513259771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b91e0 0x5648254c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.272+0000 7f2ce27e0740 1 -- 172.21.15.47:0/3513259771 >> 172.21.15.47:0/3513259771 conn(0x5648254b4750 msgr2=0x5648254c2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.272+0000 7f2ce27e0740 1 -- 172.21.15.47:0/3513259771 shutdown_connections 2024-03-21T11:31:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.272+0000 7f2ce27e0740 1 -- 172.21.15.47:0/3513259771 wait complete. 2024-03-21T11:31:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.273+0000 7f2ce27e0740 1 -- start start 2024-03-21T11:31:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.273+0000 7f2ce27e0740 1 Processor -- start 2024-03-21T11:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2ce27e0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b91e0 0x5648254c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2ce27e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b9bd0 0x5648254c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2ce27e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 0x5648254c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2ce27e0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd81ad640 con 0x5648254b91e0 2024-03-21T11:31:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2ce27e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd81acfe0 con 0x5648254bed40 2024-03-21T11:31:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2ce27e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd81aa4e0 con 0x5648254b9bd0 2024-03-21T11:31:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2ce27e0740 10 client.0 _ll_register_callbacks cb 0x7f2cd00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b9bd0 0x5648254c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2cddc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b91e0 0x5648254c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b9bd0 0x5648254c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48106/0 (socket says 172.21.15.47:48106) 2024-03-21T11:31:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2cd7e00640 1 -- 172.21.15.47:0/2251076002 learned_addr learned my addr 172.21.15.47:0/2251076002 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2cddc00640 1 --2- 172.21.15.47:0/2251076002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 0x5648254c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2cd7e00640 1 -- 172.21.15.47:0/2251076002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 msgr2=0x5648254c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2cd7e00640 1 --2- 172.21.15.47:0/2251076002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 0x5648254c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2cd7e00640 1 -- 172.21.15.47:0/2251076002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b91e0 msgr2=0x5648254c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2cd7e00640 1 --2- 172.21.15.47:0/2251076002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b91e0 0x5648254c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2cd7e00640 1 -- 172.21.15.47:0/2251076002 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648254ca060 con 0x5648254b9bd0 2024-03-21T11:31:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.274+0000 7f2cddc00640 1 --2- 172.21.15.47:0/2251076002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 0x5648254c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.275+0000 7f2cd7e00640 1 --2- 172.21.15.47:0/2251076002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b9bd0 0x5648254c8a60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2cc402c170 tx=0x7f2cc4009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86adad9b4e0cd57e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.275+0000 7f2cbea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.275+0000 7f2cbea00640 1 -- 172.21.15.47:0/2251076002 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4053070 con 0x5648254b9bd0 2024-03-21T11:31:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.275+0000 7f2ce27e0740 1 -- 172.21.15.47:0/2251076002 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cc404b070 con 0x5648254b9bd0 2024-03-21T11:31:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.275+0000 7f2cbea00640 1 -- 172.21.15.47:0/2251076002 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2cc4046600 con 0x5648254b9bd0 2024-03-21T11:31:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.276+0000 7f2cbea00640 1 -- 172.21.15.47:0/2251076002 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cc405f050 con 0x5648254b9bd0 2024-03-21T11:31:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.276+0000 7f2ce27e0740 10 client.28096 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.276+0000 7f2ce27e0740 1 -- 172.21.15.47:0/2251076002 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56482552aba0 con 0x5648254b9bd0 2024-03-21T11:31:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.276+0000 7f2cbea00640 1 -- 172.21.15.47:0/2251076002 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2cc404b070 con 0x5648254b9bd0 2024-03-21T11:31:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.276+0000 7f2cbea00640 1 -- 172.21.15.47:0/2251076002 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2cc4003aa0 con 0x5648254b9bd0 2024-03-21T11:31:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.276+0000 7f2ce27e0740 10 client.28096 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.276+0000 7f2ce27e0740 10 client.28096 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.276+0000 7f2ce27e0740 20 client.28096 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.276+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/2251076002 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56482552b350 0x56482552d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.276+0000 7f2ce27e0740 4 client.28096 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:31:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.276+0000 7f2ce27e0740 1 -- 172.21.15.47:0/2251076002 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5648254260c0 con 0x56482552b350 2024-03-21T11:31:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.277+0000 7f2cddc00640 1 --2- 172.21.15.47:0/2251076002 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56482552b350 0x56482552d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.277+0000 7f2cddc00640 1 --2- 172.21.15.47:0/2251076002 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56482552b350 0x56482552d770 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e451607d23e99777 server_cookie=fe03b45cd2619093 in_seq=0 out_seq=0 2024-03-21T11:31:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.277+0000 7f2cbea00640 10 client.28096 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:11.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:11 smithi100 ceph-mon[36245]: pgmap v635: 129 pgs: 129 active+clean; 39 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 295 op/s 2024-03-21T11:31:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.406+0000 7f60f1a00640 1 -- 172.21.15.47:0/2063782407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec1086a0 msgr2=0x7f60ec10aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.406+0000 7f60f1a00640 1 --2- 172.21.15.47:0/2063782407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec1086a0 0x7f60ec10aad0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009900 tx=0x7f60d402bc30 comp rx=0 tx=0).stop 2024-03-21T11:31:11.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.410+0000 7f60f1a00640 1 -- 172.21.15.47:0/2063782407 shutdown_connections 2024-03-21T11:31:11.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.410+0000 7f60f1a00640 1 --2- 172.21.15.47:0/2063782407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec10b490 0x7f60ec114330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.410+0000 7f60f1a00640 1 --2- 172.21.15.47:0/2063782407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60ec10b010 0x7f60ec111de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.410+0000 7f60f1a00640 1 --2- 172.21.15.47:0/2063782407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec1086a0 0x7f60ec10aad0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.410+0000 7f60f1a00640 1 -- 172.21.15.47:0/2063782407 >> 172.21.15.47:0/2063782407 conn(0x7f60ec06e510 msgr2=0x7f60ec070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.410+0000 7f60f1a00640 1 -- 172.21.15.47:0/2063782407 shutdown_connections 2024-03-21T11:31:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.411+0000 7f60f1a00640 1 -- 172.21.15.47:0/2063782407 wait complete. 2024-03-21T11:31:11.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.412+0000 7f60f1a00640 1 Processor -- start 2024-03-21T11:31:11.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.413+0000 7f60f1a00640 1 -- start start 2024-03-21T11:31:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.414+0000 7f60f1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60ec1086a0 0x7f60ec10cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.414+0000 7f60f1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec10b010 0x7f60ec10d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.414+0000 7f60f1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec10b490 0x7f60ec1141e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.414+0000 7f60f1a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec117d90 con 0x7f60ec10b010 2024-03-21T11:31:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.414+0000 7f60f1a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec117c10 con 0x7f60ec10b490 2024-03-21T11:31:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.414+0000 7f60f1a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec117f10 con 0x7f60ec1086a0 2024-03-21T11:31:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.415+0000 7f60eb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec10b490 0x7f60ec1141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.415+0000 7f60eb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec10b490 0x7f60ec1141e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59876/0 (socket says 172.21.15.47:59876) 2024-03-21T11:31:11.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.415+0000 7f60eb400640 1 -- 172.21.15.47:0/4011191046 learned_addr learned my addr 172.21.15.47:0/4011191046 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:11.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.416+0000 7f60eaa00640 1 --2- 172.21.15.47:0/4011191046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60ec1086a0 0x7f60ec10cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.416+0000 7f60ea000640 1 --2- 172.21.15.47:0/4011191046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec10b010 0x7f60ec10d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.416+0000 7f60eaa00640 1 -- 172.21.15.47:0/4011191046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec10b490 msgr2=0x7f60ec1141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.416+0000 7f60eaa00640 1 --2- 172.21.15.47:0/4011191046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec10b490 0x7f60ec1141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.416+0000 7f60eaa00640 1 -- 172.21.15.47:0/4011191046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec10b010 msgr2=0x7f60ec10d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.416+0000 7f60eaa00640 1 --2- 172.21.15.47:0/4011191046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec10b010 0x7f60ec10d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.416+0000 7f60eaa00640 1 -- 172.21.15.47:0/4011191046 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec1b6c90 con 0x7f60ec1086a0 2024-03-21T11:31:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.417+0000 7f60eaa00640 1 --2- 172.21.15.47:0/4011191046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60ec1086a0 0x7f60ec10cc30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009230 tx=0x7f60d4005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.418+0000 7f60e3400640 1 -- 172.21.15.47:0/4011191046 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4043020 con 0x7f60ec1086a0 2024-03-21T11:31:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.418+0000 7f60e3400640 1 -- 172.21.15.47:0/4011191046 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60d40054e0 con 0x7f60ec1086a0 2024-03-21T11:31:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.418+0000 7f60e3400640 1 -- 172.21.15.47:0/4011191046 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4003ca0 con 0x7f60ec1086a0 2024-03-21T11:31:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.418+0000 7f60f1a00640 1 -- 172.21.15.47:0/4011191046 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ec1b6f20 con 0x7f60ec1086a0 2024-03-21T11:31:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.418+0000 7f60f1a00640 1 -- 172.21.15.47:0/4011191046 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ec1b7330 con 0x7f60ec1086a0 2024-03-21T11:31:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.421+0000 7f60f1a00640 1 -- 172.21.15.47:0/4011191046 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ec1a0b90 con 0x7f60ec1086a0 2024-03-21T11:31:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.426+0000 7f60e3400640 1 -- 172.21.15.47:0/4011191046 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f60d4046020 con 0x7f60ec1086a0 2024-03-21T11:31:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.432+0000 7f60e3400640 1 --2- 172.21.15.47:0/4011191046 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60d00a30a0 0x7f60d00a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.432+0000 7f60e3400640 1 -- 172.21.15.47:0/4011191046 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f60d40f1970 con 0x7f60ec1086a0 2024-03-21T11:31:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.433+0000 7f60ea000640 1 --2- 172.21.15.47:0/4011191046 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60d00a30a0 0x7f60d00a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.436+0000 7f60ea000640 1 --2- 172.21.15.47:0/4011191046 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60d00a30a0 0x7f60d00a5560 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f60dc005fd0 tx=0x7f60dc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.437+0000 7f60e3400640 1 -- 172.21.15.47:0/4011191046 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f60d404d020 con 0x7f60ec1086a0 2024-03-21T11:31:11.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:11 smithi102 ceph-mon[36748]: pgmap v635: 129 pgs: 129 active+clean; 39 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 295 op/s 2024-03-21T11:31:11.533 INFO:tasks.workunit.client.0.smithi047.stdout: 1 551290 8.23 MB/sec execute 235 sec latency 181.022 ms 2024-03-21T11:31:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.611+0000 7f2cbea00640 1 -- 172.21.15.47:0/2251076002 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5648254260c0 con 0x56482552b350 2024-03-21T11:31:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.611+0000 7f2cbea00640 10 client.28096 handle_command_reply: tid=0 2024-03-21T11:31:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.611+0000 7f2ce27e0740 1 client.28096 shutdown 2024-03-21T11:31:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.611+0000 7f2ce27e0740 1 -- 172.21.15.47:0/2251076002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b9bd0 msgr2=0x5648254c8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.611+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/2251076002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b9bd0 0x5648254c8a60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2cc402c170 tx=0x7f2cc4009680 comp rx=0 tx=0).stop 2024-03-21T11:31:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.611+0000 7f2ce27e0740 1 -- 172.21.15.47:0/2251076002 shutdown_connections 2024-03-21T11:31:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.611+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/2251076002 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56482552b350 0x56482552d770 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.611+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/2251076002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5648254bed40 0x5648254c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.611+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/2251076002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5648254b9bd0 0x5648254c8a60 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.611+0000 7f2ce27e0740 1 --2- 172.21.15.47:0/2251076002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5648254b91e0 0x5648254c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.611+0000 7f2cbea00640 0 client.28096 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.611+0000 7f2ce27e0740 1 -- 172.21.15.47:0/2251076002 >> 172.21.15.47:0/2251076002 conn(0x5648254b4750 msgr2=0x5648254b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.612+0000 7f2ce27e0740 1 -- 172.21.15.47:0/2251076002 shutdown_connections 2024-03-21T11:31:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.612+0000 7f2ce27e0740 1 -- 172.21.15.47:0/2251076002 wait complete. 2024-03-21T11:31:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.612+0000 7f2ce27e0740 20 client.28096 trim_cache size 0 max 16384 2024-03-21T11:31:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:31:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:31:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:31:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:31:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.613+0000 7f2cdfc00640 1 -- 172.21.15.47:0/1247267653 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2cc00a2fc0 msgr2=0x7f2cc00a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.613+0000 7f2cdfc00640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2cc00a2fc0 0x7f2cc00a5480 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc098970 tx=0x7f2ccc0046c0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.613+0000 7f2cdfc00640 1 -- 172.21.15.47:0/1247267653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd8103900 msgr2=0x7f2cd8199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.613+0000 7f2cdfc00640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd8103900 0x7f2cd8199b90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00e9c0 tx=0x7f2ccc00ee80 comp rx=0 tx=0).stop 2024-03-21T11:31:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.613+0000 7f2cdfc00640 1 -- 172.21.15.47:0/1247267653 shutdown_connections 2024-03-21T11:31:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.613+0000 7f2cdfc00640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2cc00a2fc0 0x7f2cc00a5480 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.613+0000 7f2cdfc00640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cd81054b0 0x7f2cd81a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.613+0000 7f2cdfc00640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cd8104af0 0x7f2cd819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.613+0000 7f2cdfc00640 1 --2- 172.21.15.47:0/1247267653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd8103900 0x7f2cd8199b90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.613+0000 7f2cdfc00640 1 -- 172.21.15.47:0/1247267653 >> 172.21.15.47:0/1247267653 conn(0x7f2cd80fee90 msgr2=0x7f2cd8107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.613+0000 7f2cdfc00640 1 -- 172.21.15.47:0/1247267653 shutdown_connections 2024-03-21T11:31:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:11.613+0000 7f2cdfc00640 1 -- 172.21.15.47:0/1247267653 wait complete. 2024-03-21T11:31:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.785+0000 7f60f1a00640 1 -- 172.21.15.47:0/4011191046 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60ec02d070 con 0x7f60ec1086a0 2024-03-21T11:31:11.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.786+0000 7f60e3400640 1 -- 172.21.15.47:0/4011191046 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f60d40be410 con 0x7f60ec1086a0 2024-03-21T11:31:11.788 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:31:11.788 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:31:11.788 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:31:11.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.790+0000 7f60f1a00640 1 -- 172.21.15.47:0/4011191046 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60d00a30a0 msgr2=0x7f60d00a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.790+0000 7f60f1a00640 1 --2- 172.21.15.47:0/4011191046 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60d00a30a0 0x7f60d00a5560 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f60dc005fd0 tx=0x7f60dc005850 comp rx=0 tx=0).stop 2024-03-21T11:31:11.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.790+0000 7f60f1a00640 1 -- 172.21.15.47:0/4011191046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60ec1086a0 msgr2=0x7f60ec10cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:11.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.790+0000 7f60f1a00640 1 --2- 172.21.15.47:0/4011191046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60ec1086a0 0x7f60ec10cc30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009230 tx=0x7f60d4005a40 comp rx=0 tx=0).stop 2024-03-21T11:31:11.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.791+0000 7f60f1a00640 1 -- 172.21.15.47:0/4011191046 shutdown_connections 2024-03-21T11:31:11.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.791+0000 7f60f1a00640 1 --2- 172.21.15.47:0/4011191046 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f60d00a30a0 0x7f60d00a5560 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.791+0000 7f60f1a00640 1 --2- 172.21.15.47:0/4011191046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec10b490 0x7f60ec1141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.791+0000 7f60f1a00640 1 --2- 172.21.15.47:0/4011191046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec10b010 0x7f60ec10d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.791+0000 7f60f1a00640 1 --2- 172.21.15.47:0/4011191046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60ec1086a0 0x7f60ec10cc30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.791+0000 7f60f1a00640 1 -- 172.21.15.47:0/4011191046 >> 172.21.15.47:0/4011191046 conn(0x7f60ec06e510 msgr2=0x7f60ec070730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.791+0000 7f60f1a00640 1 -- 172.21.15.47:0/4011191046 shutdown_connections 2024-03-21T11:31:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:11.792+0000 7f60f1a00640 1 -- 172.21.15.47:0/4011191046 wait complete. 2024-03-21T11:31:11.820 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:31:11.820 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:31:12.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.061+0000 7f7a61600640 1 -- 172.21.15.47:0/4089141272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5c1028a0 msgr2=0x7f7a5c102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.061+0000 7f7a61600640 1 --2- 172.21.15.47:0/4089141272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5c1028a0 0x7f7a5c102d20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009900 tx=0x7f7a4802bc30 comp rx=0 tx=0).stop 2024-03-21T11:31:12.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.061+0000 7f7a61600640 1 -- 172.21.15.47:0/4089141272 shutdown_connections 2024-03-21T11:31:12.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.061+0000 7f7a61600640 1 --2- 172.21.15.47:0/4089141272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a5c103260 0x7f7a5c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.061+0000 7f7a61600640 1 --2- 172.21.15.47:0/4089141272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5c1028a0 0x7f7a5c102d20 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.061+0000 7f7a61600640 1 --2- 172.21.15.47:0/4089141272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a5c1016b0 0x7f7a5c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.061+0000 7f7a61600640 1 -- 172.21.15.47:0/4089141272 >> 172.21.15.47:0/4089141272 conn(0x7f7a5c0fcc40 msgr2=0x7f7a5c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.061+0000 7f7a61600640 1 -- 172.21.15.47:0/4089141272 shutdown_connections 2024-03-21T11:31:12.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.061+0000 7f7a61600640 1 -- 172.21.15.47:0/4089141272 wait complete. 2024-03-21T11:31:12.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.062+0000 7f7a61600640 1 Processor -- start 2024-03-21T11:31:12.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.063+0000 7f7a61600640 1 -- start start 2024-03-21T11:31:12.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.063+0000 7f7a61600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5c1016b0 0x7f7a5c075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.063+0000 7f7a61600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a5c1028a0 0x7f7a5c073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.063+0000 7f7a61600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a5c103260 0x7f7a5c073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.063+0000 7f7a61600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c10d330 con 0x7f7a5c1028a0 2024-03-21T11:31:12.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.063+0000 7f7a61600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c10d1b0 con 0x7f7a5c103260 2024-03-21T11:31:12.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.063+0000 7f7a61600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c10d4b0 con 0x7f7a5c1016b0 2024-03-21T11:31:12.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.063+0000 7f7a5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5c1016b0 0x7f7a5c075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.063+0000 7f7a5a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a5c1028a0 0x7f7a5c073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.063+0000 7f7a5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5c1016b0 0x7f7a5c075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48130/0 (socket says 172.21.15.47:48130) 2024-03-21T11:31:12.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.063+0000 7f7a5aa00640 1 -- 172.21.15.47:0/3810741476 learned_addr learned my addr 172.21.15.47:0/3810741476 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.063+0000 7f7a5b400640 1 --2- 172.21.15.47:0/3810741476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a5c103260 0x7f7a5c073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.064+0000 7f7a5aa00640 1 -- 172.21.15.47:0/3810741476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a5c103260 msgr2=0x7f7a5c073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.064+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/3810741476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a5c103260 0x7f7a5c073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.064+0000 7f7a5aa00640 1 -- 172.21.15.47:0/3810741476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a5c1028a0 msgr2=0x7f7a5c073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.064+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/3810741476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a5c1028a0 0x7f7a5c073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.064+0000 7f7a5aa00640 1 -- 172.21.15.47:0/3810741476 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c074860 con 0x7f7a5c1016b0 2024-03-21T11:31:12.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.064+0000 7f7a5a000640 1 --2- 172.21.15.47:0/3810741476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a5c1028a0 0x7f7a5c073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:12.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.064+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/3810741476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5c1016b0 0x7f7a5c075450 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00e9c0 tx=0x7f7a4c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.064+0000 7f7a53400640 1 -- 172.21.15.47:0/3810741476 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00cd80 con 0x7f7a5c1016b0 2024-03-21T11:31:12.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.064+0000 7f7a53400640 1 -- 172.21.15.47:0/3810741476 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a4c0035f0 con 0x7f7a5c1016b0 2024-03-21T11:31:12.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.064+0000 7f7a53400640 1 -- 172.21.15.47:0/3810741476 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c010690 con 0x7f7a5c1016b0 2024-03-21T11:31:12.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.064+0000 7f7a61600640 1 -- 172.21.15.47:0/3810741476 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c1a1af0 con 0x7f7a5c1016b0 2024-03-21T11:31:12.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.064+0000 7f7a61600640 1 -- 172.21.15.47:0/3810741476 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c1a2010 con 0x7f7a5c1016b0 2024-03-21T11:31:12.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.068+0000 7f7a53400640 1 -- 172.21.15.47:0/3810741476 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7a4c0037a0 con 0x7f7a5c1016b0 2024-03-21T11:31:12.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.069+0000 7f7a643c7740 1 Processor -- start 2024-03-21T11:31:12.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.069+0000 7f7a643c7740 1 -- start start 2024-03-21T11:31:12.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.069+0000 7f7a53400640 1 --2- 172.21.15.47:0/3810741476 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7a400a2fa0 0x7f7a400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.069+0000 7f7a643c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb6244c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.069+0000 7f7a643c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62459e0 0x558bb6245e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.069+0000 7f7a53400640 1 -- 172.21.15.47:0/3810741476 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7a4c0cfae0 con 0x7f7a5c1016b0 2024-03-21T11:31:12.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.069+0000 7f7a643c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 0x558bb624cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a643c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb61a92c0 con 0x558bb62463a0 2024-03-21T11:31:12.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a643c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb6151c90 con 0x558bb6244810 2024-03-21T11:31:12.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a643c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb61a5360 con 0x558bb62459e0 2024-03-21T11:31:12.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a5b400640 1 --2- 172.21.15.47:0/3810741476 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7a400a2fa0 0x7f7a400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a5aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb6244c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a5aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb6244c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59898/0 (socket says 172.21.15.47:59898) 2024-03-21T11:31:12.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a5aa00640 1 -- 172.21.15.47:0/4233740839 learned_addr learned my addr 172.21.15.47:0/4233740839 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a5a000640 1 --2- 172.21.15.47:0/4233740839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62459e0 0x558bb6245e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a5aa00640 1 -- 172.21.15.47:0/4233740839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62459e0 msgr2=0x558bb6245e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a5b400640 1 --2- 172.21.15.47:0/4233740839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 0x558bb624cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/4233740839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62459e0 0x558bb6245e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a5aa00640 1 -- 172.21.15.47:0/4233740839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 msgr2=0x558bb624cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/4233740839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 0x558bb624cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a5aa00640 1 -- 172.21.15.47:0/4233740839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bb6176080 con 0x558bb6244810 2024-03-21T11:31:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.070+0000 7f7a5b400640 1 --2- 172.21.15.47:0/4233740839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 0x558bb624cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:31:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.071+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/4233740839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb6244c30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009810 tx=0x7f7a4c09c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85d6ab2b09a6d042 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.071+0000 7f7a5b400640 1 --2- 172.21.15.47:0/3810741476 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7a400a2fa0 0x7f7a400a5460 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7a54005fd0 tx=0x7f7a54009bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.071+0000 7f7a47400640 1 -- 172.21.15.47:0/4233740839 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c09ff70 con 0x558bb6244810 2024-03-21T11:31:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.071+0000 7f7a47400640 1 -- 172.21.15.47:0/4233740839 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a4c0a4830 con 0x558bb6244810 2024-03-21T11:31:12.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.071+0000 7f7a47400640 1 -- 172.21.15.47:0/4233740839 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0bab90 con 0x558bb6244810 2024-03-21T11:31:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.072+0000 7f7a643c7740 1 -- 172.21.15.47:0/4233740839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 msgr2=0x558bb6244c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.072+0000 7f7a643c7740 1 --2- 172.21.15.47:0/4233740839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb6244c30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009810 tx=0x7f7a4c09c520 comp rx=0 tx=0).stop 2024-03-21T11:31:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.072+0000 7f7a643c7740 1 -- 172.21.15.47:0/4233740839 shutdown_connections 2024-03-21T11:31:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.072+0000 7f7a643c7740 1 --2- 172.21.15.47:0/4233740839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 0x558bb624cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.072+0000 7f7a643c7740 1 --2- 172.21.15.47:0/4233740839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62459e0 0x558bb6245e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.072+0000 7f7a643c7740 1 --2- 172.21.15.47:0/4233740839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb6244c30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.072+0000 7f7a643c7740 1 -- 172.21.15.47:0/4233740839 >> 172.21.15.47:0/4233740839 conn(0x558bb623fc00 msgr2=0x558bb6240000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.072+0000 7f7a643c7740 1 -- 172.21.15.47:0/4233740839 shutdown_connections 2024-03-21T11:31:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.072+0000 7f7a643c7740 1 -- 172.21.15.47:0/4233740839 wait complete. 2024-03-21T11:31:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.072+0000 7f7a643c7740 1 -- start start 2024-03-21T11:31:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.072+0000 7f7a643c7740 1 Processor -- start 2024-03-21T11:31:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.073+0000 7f7a643c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb624eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.073+0000 7f7a643c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62459e0 0x558bb624eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.073+0000 7f7a643c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62463a0 0x558bb6256030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.073+0000 7f7a643c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb624d3b0 con 0x558bb62459e0 2024-03-21T11:31:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.073+0000 7f7a643c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb61650d0 con 0x558bb6244810 2024-03-21T11:31:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.073+0000 7f7a643c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb624d530 con 0x558bb62463a0 2024-03-21T11:31:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.073+0000 7f7a643c7740 10 client.0 _ll_register_callbacks cb 0x558bb6169e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.073+0000 7f7a5aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb624eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.073+0000 7f7a5aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb624eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59904/0 (socket says 172.21.15.47:59904) 2024-03-21T11:31:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.073+0000 7f7a5aa00640 1 -- 172.21.15.47:0/2394031511 learned_addr learned my addr 172.21.15.47:0/2394031511 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.073+0000 7f7a5a000640 1 --2- 172.21.15.47:0/2394031511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62459e0 0x558bb624eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.073+0000 7f7a5b400640 1 --2- 172.21.15.47:0/2394031511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62463a0 0x558bb6256030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.074+0000 7f7a5aa00640 1 -- 172.21.15.47:0/2394031511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62463a0 msgr2=0x558bb6256030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.074+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/2394031511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62463a0 0x558bb6256030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.074+0000 7f7a5aa00640 1 -- 172.21.15.47:0/2394031511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62459e0 msgr2=0x558bb624eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.074+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/2394031511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62459e0 0x558bb624eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.074+0000 7f7a5aa00640 1 -- 172.21.15.47:0/2394031511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bb5ebd3f0 con 0x558bb6244810 2024-03-21T11:31:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.074+0000 7f7a5a000640 1 --2- 172.21.15.47:0/2394031511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62459e0 0x558bb624eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.074+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/2394031511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb624eab0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0987f0 tx=0x7f7a4c098500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a149a67752f488 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.074+0000 7f7a3be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.074+0000 7f7a3be00640 1 -- 172.21.15.47:0/2394031511 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0b9070 con 0x558bb6244810 2024-03-21T11:31:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.074+0000 7f7a3be00640 1 -- 172.21.15.47:0/2394031511 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7a4c0bad90 con 0x558bb6244810 2024-03-21T11:31:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.074+0000 7f7a643c7740 1 -- 172.21.15.47:0/2394031511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a4c0a4830 con 0x558bb6244810 2024-03-21T11:31:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.074+0000 7f7a3be00640 1 -- 172.21.15.47:0/2394031511 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a4c09b270 con 0x558bb6244810 2024-03-21T11:31:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.074+0000 7f7a643c7740 10 client.28103 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.074+0000 7f7a643c7740 1 -- 172.21.15.47:0/2394031511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558bb5f877d0 con 0x558bb6244810 2024-03-21T11:31:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.075+0000 7f7a3be00640 1 -- 172.21.15.47:0/2394031511 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a4c0a4830 con 0x558bb6244810 2024-03-21T11:31:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.075+0000 7f7a3be00640 1 -- 172.21.15.47:0/2394031511 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7a4c097b70 con 0x558bb6244810 2024-03-21T11:31:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.075+0000 7f7a643c7740 10 client.28103 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.075+0000 7f7a643c7740 10 client.28103 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:31:12.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.075+0000 7f7a643c7740 20 client.28103 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:12.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.075+0000 7f7a643c7740 1 --2- 172.21.15.47:0/2394031511 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558bb625a100 0x558bb625c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.075+0000 7f7a643c7740 4 client.28103 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:12.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.075+0000 7f7a643c7740 1 -- 172.21.15.47:0/2394031511 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bb61b6800 con 0x558bb625a100 2024-03-21T11:31:12.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.075+0000 7f7a5a000640 1 --2- 172.21.15.47:0/2394031511 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558bb625a100 0x558bb625c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.076+0000 7f7a5a000640 1 --2- 172.21.15.47:0/2394031511 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558bb625a100 0x558bb625c4f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=86e756e7605c7be server_cookie=55567254e77fe9b4 in_seq=0 out_seq=0 2024-03-21T11:31:12.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.076+0000 7f7a3be00640 10 client.28103 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:31:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a3be00640 1 -- 172.21.15.47:0/2394031511 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558bb61b6800 con 0x558bb625a100 2024-03-21T11:31:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a3be00640 10 client.28103 handle_command_reply: tid=0 2024-03-21T11:31:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a643c7740 1 client.28103 shutdown 2024-03-21T11:31:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a643c7740 1 -- 172.21.15.47:0/2394031511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 msgr2=0x558bb624eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a643c7740 1 --2- 172.21.15.47:0/2394031511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb624eab0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0987f0 tx=0x7f7a4c098500 comp rx=0 tx=0).stop 2024-03-21T11:31:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a643c7740 1 -- 172.21.15.47:0/2394031511 shutdown_connections 2024-03-21T11:31:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a643c7740 1 --2- 172.21.15.47:0/2394031511 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558bb625a100 0x558bb625c4f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a643c7740 1 --2- 172.21.15.47:0/2394031511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62463a0 0x558bb6256030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a3be00640 0 client.28103 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:31:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a643c7740 1 --2- 172.21.15.47:0/2394031511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62459e0 0x558bb624eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a643c7740 1 --2- 172.21.15.47:0/2394031511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb624eab0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a643c7740 1 -- 172.21.15.47:0/2394031511 >> 172.21.15.47:0/2394031511 conn(0x558bb623fc00 msgr2=0x558bb623eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a643c7740 1 -- 172.21.15.47:0/2394031511 shutdown_connections 2024-03-21T11:31:12.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a643c7740 1 -- 172.21.15.47:0/2394031511 wait complete. 2024-03-21T11:31:12.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.077+0000 7f7a643c7740 20 client.28103 trim_cache size 0 max 16384 2024-03-21T11:31:12.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.092+0000 7f7a643c7740 1 Processor -- start 2024-03-21T11:31:12.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.092+0000 7f7a643c7740 1 -- start start 2024-03-21T11:31:12.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a643c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb623b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a643c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62459e0 0x558bb6255ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a643c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 0x558bb623ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a643c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1ac2d0 con 0x558bb62463a0 2024-03-21T11:31:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a643c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1adc50 con 0x558bb6244810 2024-03-21T11:31:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a643c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1ac930 con 0x558bb62459e0 2024-03-21T11:31:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a5aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb623b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a5a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62459e0 0x558bb6255ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a5aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb623b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59908/0 (socket says 172.21.15.47:59908) 2024-03-21T11:31:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a5aa00640 1 -- 172.21.15.47:0/668015614 learned_addr learned my addr 172.21.15.47:0/668015614 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a5b400640 1 --2- 172.21.15.47:0/668015614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 0x558bb623ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a5aa00640 1 -- 172.21.15.47:0/668015614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62459e0 msgr2=0x558bb6255ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/668015614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62459e0 0x558bb6255ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a5aa00640 1 -- 172.21.15.47:0/668015614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 msgr2=0x558bb623ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/668015614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 0x558bb623ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.093+0000 7f7a5aa00640 1 -- 172.21.15.47:0/668015614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bb6240520 con 0x558bb6244810 2024-03-21T11:31:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.094+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/668015614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb623b000 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c097f40 tx=0x7f7a4c09c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11fd9f467bd171fc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.094+0000 7f7a46a00640 1 -- 172.21.15.47:0/668015614 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0b9040 con 0x558bb6244810 2024-03-21T11:31:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.094+0000 7f7a46a00640 1 -- 172.21.15.47:0/668015614 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a4c098570 con 0x558bb6244810 2024-03-21T11:31:12.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.094+0000 7f7a46a00640 1 -- 172.21.15.47:0/668015614 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c09ad80 con 0x558bb6244810 2024-03-21T11:31:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.094+0000 7f7a643c7740 1 -- 172.21.15.47:0/668015614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 msgr2=0x558bb623b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.094+0000 7f7a643c7740 1 --2- 172.21.15.47:0/668015614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb623b000 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c097f40 tx=0x7f7a4c09c520 comp rx=0 tx=0).stop 2024-03-21T11:31:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.094+0000 7f7a643c7740 1 -- 172.21.15.47:0/668015614 shutdown_connections 2024-03-21T11:31:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.094+0000 7f7a643c7740 1 --2- 172.21.15.47:0/668015614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 0x558bb623ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.094+0000 7f7a643c7740 1 --2- 172.21.15.47:0/668015614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb62459e0 0x558bb6255ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.094+0000 7f7a643c7740 1 --2- 172.21.15.47:0/668015614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb6244810 0x558bb623b000 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.094+0000 7f7a643c7740 1 -- 172.21.15.47:0/668015614 >> 172.21.15.47:0/668015614 conn(0x558bb623fc00 msgr2=0x558bb62436f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.095+0000 7f7a643c7740 1 -- 172.21.15.47:0/668015614 shutdown_connections 2024-03-21T11:31:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.095+0000 7f7a643c7740 1 -- 172.21.15.47:0/668015614 wait complete. 2024-03-21T11:31:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.095+0000 7f7a643c7740 1 -- start start 2024-03-21T11:31:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.095+0000 7f7a643c7740 1 Processor -- start 2024-03-21T11:31:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.095+0000 7f7a643c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb6244810 0x558bb6255840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.095+0000 7f7a643c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb62459e0 0x558bb6255d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.095+0000 7f7a643c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 0x558bb62522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.095+0000 7f7a643c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1ad5f0 con 0x558bb62463a0 2024-03-21T11:31:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a643c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1acf90 con 0x558bb62459e0 2024-03-21T11:31:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a643c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1aa470 con 0x558bb6244810 2024-03-21T11:31:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a643c7740 10 client.0 _ll_register_callbacks cb 0x7f7a4c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:12.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb6244810 0x558bb6255840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a5a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb62459e0 0x558bb6255d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb6244810 0x558bb6255840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48166/0 (socket says 172.21.15.47:48166) 2024-03-21T11:31:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a5aa00640 1 -- 172.21.15.47:0/910680944 learned_addr learned my addr 172.21.15.47:0/910680944 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a5b400640 1 --2- 172.21.15.47:0/910680944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 0x558bb62522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a5aa00640 1 -- 172.21.15.47:0/910680944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb62459e0 msgr2=0x558bb6255d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/910680944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb62459e0 0x558bb6255d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a5aa00640 1 -- 172.21.15.47:0/910680944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 msgr2=0x558bb62522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/910680944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 0x558bb62522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a5aa00640 1 -- 172.21.15.47:0/910680944 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bb62529d0 con 0x558bb6244810 2024-03-21T11:31:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a5a000640 1 --2- 172.21.15.47:0/910680944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb62459e0 0x558bb6255d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a5aa00640 1 --2- 172.21.15.47:0/910680944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb6244810 0x558bb6255840 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c09a0d0 tx=0x7f7a4c0a43a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4f71f97249c1d0d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.096+0000 7f7a3be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:12.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.097+0000 7f7a3be00640 1 -- 172.21.15.47:0/910680944 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0b9070 con 0x558bb6244810 2024-03-21T11:31:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.097+0000 7f7a3be00640 1 -- 172.21.15.47:0/910680944 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7a4c0bab90 con 0x558bb6244810 2024-03-21T11:31:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.097+0000 7f7a643c7740 1 -- 172.21.15.47:0/910680944 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a4c098570 con 0x558bb6244810 2024-03-21T11:31:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.097+0000 7f7a3be00640 1 -- 172.21.15.47:0/910680944 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a4c0980c0 con 0x558bb6244810 2024-03-21T11:31:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.097+0000 7f7a643c7740 10 client.28120 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.097+0000 7f7a643c7740 1 -- 172.21.15.47:0/910680944 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558bb62529d0 con 0x558bb6244810 2024-03-21T11:31:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.097+0000 7f7a3be00640 1 -- 172.21.15.47:0/910680944 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a4c098570 con 0x558bb6244810 2024-03-21T11:31:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.097+0000 7f7a3be00640 1 -- 172.21.15.47:0/910680944 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7a4c086ca0 con 0x558bb6244810 2024-03-21T11:31:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.097+0000 7f7a643c7740 10 client.28120 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.097+0000 7f7a643c7740 10 client.28120 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:31:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.097+0000 7f7a643c7740 20 client.28120 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.097+0000 7f7a643c7740 1 --2- 172.21.15.47:0/910680944 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558bb62b6820 0x558bb62b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.097+0000 7f7a643c7740 4 client.28120 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:31:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.097+0000 7f7a643c7740 1 -- 172.21.15.47:0/910680944 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558bb61b0b50 con 0x558bb62b6820 2024-03-21T11:31:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.098+0000 7f7a5a000640 1 --2- 172.21.15.47:0/910680944 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558bb62b6820 0x558bb62b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.098+0000 7f7a5a000640 1 --2- 172.21.15.47:0/910680944 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558bb62b6820 0x558bb62b8c10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b11567e0a45aab3a server_cookie=dc0535be4407c5a in_seq=0 out_seq=0 2024-03-21T11:31:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.098+0000 7f7a3be00640 10 client.28120 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:31:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.098+0000 7f7a3be00640 1 -- 172.21.15.47:0/910680944 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558bb61b0b50 con 0x558bb62b6820 2024-03-21T11:31:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.098+0000 7f7a3be00640 10 client.28120 handle_command_reply: tid=0 2024-03-21T11:31:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.098+0000 7f7a643c7740 1 client.28120 shutdown 2024-03-21T11:31:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.098+0000 7f7a643c7740 1 -- 172.21.15.47:0/910680944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb6244810 msgr2=0x558bb6255840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.098+0000 7f7a643c7740 1 --2- 172.21.15.47:0/910680944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb6244810 0x558bb6255840 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c09a0d0 tx=0x7f7a4c0a43a0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a643c7740 1 -- 172.21.15.47:0/910680944 shutdown_connections 2024-03-21T11:31:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a643c7740 1 --2- 172.21.15.47:0/910680944 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558bb62b6820 0x558bb62b8c10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a643c7740 1 --2- 172.21.15.47:0/910680944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bb62463a0 0x558bb62522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a643c7740 1 --2- 172.21.15.47:0/910680944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bb62459e0 0x558bb6255d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a643c7740 1 --2- 172.21.15.47:0/910680944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bb6244810 0x558bb6255840 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a643c7740 1 -- 172.21.15.47:0/910680944 >> 172.21.15.47:0/910680944 conn(0x558bb623fc00 msgr2=0x558bb623e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a3be00640 0 client.28120 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:31:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a643c7740 1 -- 172.21.15.47:0/910680944 shutdown_connections 2024-03-21T11:31:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a643c7740 1 -- 172.21.15.47:0/910680944 wait complete. 2024-03-21T11:31:12.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a643c7740 20 client.28120 trim_cache size 0 max 16384 2024-03-21T11:31:12.105 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:31:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a61600640 1 -- 172.21.15.47:0/3810741476 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7a400a2fa0 msgr2=0x7f7a400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a61600640 1 --2- 172.21.15.47:0/3810741476 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7a400a2fa0 0x7f7a400a5460 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7a54005fd0 tx=0x7f7a54009bf0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a61600640 1 -- 172.21.15.47:0/3810741476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5c1016b0 msgr2=0x7f7a5c075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a61600640 1 --2- 172.21.15.47:0/3810741476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5c1016b0 0x7f7a5c075450 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00e9c0 tx=0x7f7a4c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:31:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a61600640 1 -- 172.21.15.47:0/3810741476 shutdown_connections 2024-03-21T11:31:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.099+0000 7f7a61600640 1 --2- 172.21.15.47:0/3810741476 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7a400a2fa0 0x7f7a400a5460 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.100+0000 7f7a61600640 1 --2- 172.21.15.47:0/3810741476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a5c103260 0x7f7a5c073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.100+0000 7f7a61600640 1 --2- 172.21.15.47:0/3810741476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a5c1028a0 0x7f7a5c073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.100+0000 7f7a61600640 1 --2- 172.21.15.47:0/3810741476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a5c1016b0 0x7f7a5c075450 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.100+0000 7f7a61600640 1 -- 172.21.15.47:0/3810741476 >> 172.21.15.47:0/3810741476 conn(0x7f7a5c0fcc40 msgr2=0x7f7a5c105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.100+0000 7f7a61600640 1 -- 172.21.15.47:0/3810741476 shutdown_connections 2024-03-21T11:31:12.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.100+0000 7f7a61600640 1 -- 172.21.15.47:0/3810741476 wait complete. 2024-03-21T11:31:12.115 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:12 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/4011191046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:31:12.117 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:31:12.117 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:31:12.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.317+0000 7fd16f200640 1 -- 172.21.15.47:0/601940290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1701015a0 msgr2=0x7fd1701019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.317+0000 7fd16f200640 1 --2- 172.21.15.47:0/601940290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1701015a0 0x7fd1701019c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd160009900 tx=0x7fd16002bc30 comp rx=0 tx=0).stop 2024-03-21T11:31:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.317+0000 7fd16f200640 1 -- 172.21.15.47:0/601940290 shutdown_connections 2024-03-21T11:31:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.317+0000 7fd16f200640 1 --2- 172.21.15.47:0/601940290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd170103150 0x7fd1701099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.317+0000 7fd16f200640 1 --2- 172.21.15.47:0/601940290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd170102790 0x7fd170102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.317+0000 7fd16f200640 1 --2- 172.21.15.47:0/601940290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1701015a0 0x7fd1701019c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.317+0000 7fd16f200640 1 -- 172.21.15.47:0/601940290 >> 172.21.15.47:0/601940290 conn(0x7fd1700fcb10 msgr2=0x7fd1700fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.317+0000 7fd16f200640 1 -- 172.21.15.47:0/601940290 shutdown_connections 2024-03-21T11:31:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.317+0000 7fd16f200640 1 -- 172.21.15.47:0/601940290 wait complete. 2024-03-21T11:31:12.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.319+0000 7fd16f200640 1 Processor -- start 2024-03-21T11:31:12.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.319+0000 7fd16f200640 1 -- start start 2024-03-21T11:31:12.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.319+0000 7fd16f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1701015a0 0x7fd170199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.319+0000 7fd16f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd170102790 0x7fd170199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.319+0000 7fd16f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd170103150 0x7fd1701a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.319+0000 7fd16f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17010d240 con 0x7fd1701015a0 2024-03-21T11:31:12.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.319+0000 7fd16f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17010d0c0 con 0x7fd170103150 2024-03-21T11:31:12.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.319+0000 7fd16f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17010d3c0 con 0x7fd170102790 2024-03-21T11:31:12.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.320+0000 7fd16de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1701015a0 0x7fd170199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.320+0000 7fd16e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd170103150 0x7fd1701a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.320+0000 7fd16de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1701015a0 0x7fd170199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53126/0 (socket says 172.21.15.47:53126) 2024-03-21T11:31:12.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.320+0000 7fd16de00640 1 -- 172.21.15.47:0/1560240950 learned_addr learned my addr 172.21.15.47:0/1560240950 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.320+0000 7fd16e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd170103150 0x7fd1701a1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59924/0 (socket says 172.21.15.47:59924) 2024-03-21T11:31:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.320+0000 7fd16de00640 1 -- 172.21.15.47:0/1560240950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd170102790 msgr2=0x7fd170199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.320+0000 7fd16d400640 1 --2- 172.21.15.47:0/1560240950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd170102790 0x7fd170199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.320+0000 7fd16de00640 1 --2- 172.21.15.47:0/1560240950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd170102790 0x7fd170199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.320+0000 7fd16de00640 1 -- 172.21.15.47:0/1560240950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd170103150 msgr2=0x7fd1701a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.320+0000 7fd16de00640 1 --2- 172.21.15.47:0/1560240950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd170103150 0x7fd1701a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.320+0000 7fd16de00640 1 -- 172.21.15.47:0/1560240950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1701a1690 con 0x7fd1701015a0 2024-03-21T11:31:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.320+0000 7fd16d400640 1 --2- 172.21.15.47:0/1560240950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd170102790 0x7fd170199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:31:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.321+0000 7fd16de00640 1 --2- 172.21.15.47:0/1560240950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1701015a0 0x7fd170199a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd160000c00 tx=0x7fd16002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.321+0000 7fd166a00640 1 -- 172.21.15.47:0/1560240950 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160043020 con 0x7fd1701015a0 2024-03-21T11:31:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.321+0000 7fd166a00640 1 -- 172.21.15.47:0/1560240950 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1600054e0 con 0x7fd1701015a0 2024-03-21T11:31:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.321+0000 7fd166a00640 1 -- 172.21.15.47:0/1560240950 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160003ce0 con 0x7fd1701015a0 2024-03-21T11:31:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.321+0000 7fd16f200640 1 -- 172.21.15.47:0/1560240950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1701a1920 con 0x7fd1701015a0 2024-03-21T11:31:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.322+0000 7fd16f200640 1 -- 172.21.15.47:0/1560240950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1701a1e40 con 0x7fd1701015a0 2024-03-21T11:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.325+0000 7fd166a00640 1 -- 172.21.15.47:0/1560240950 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd160046020 con 0x7fd1701015a0 2024-03-21T11:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.325+0000 7fd177dfe740 1 Processor -- start 2024-03-21T11:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.326+0000 7fd177dfe740 1 -- start start 2024-03-21T11:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.326+0000 7fd177dfe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c548e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.326+0000 7fd177dfe740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c54a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.326+0000 7fd177dfe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 0x55824c550e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.326+0000 7fd177dfe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824c4ad2c0 con 0x55824c54a580 2024-03-21T11:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.326+0000 7fd177dfe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824c455c90 con 0x55824c5489f0 2024-03-21T11:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.326+0000 7fd177dfe740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824c4a9360 con 0x55824c549bc0 2024-03-21T11:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.326+0000 7fd16e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c548e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.326+0000 7fd16de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c54a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.326+0000 7fd16e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c548e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59928/0 (socket says 172.21.15.47:59928) 2024-03-21T11:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.326+0000 7fd16e800640 1 -- 172.21.15.47:0/2629586481 learned_addr learned my addr 172.21.15.47:0/2629586481 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.327+0000 7fd16d400640 1 --2- 172.21.15.47:0/2629586481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 0x55824c550e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.327+0000 7fd16e800640 1 -- 172.21.15.47:0/2629586481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 msgr2=0x55824c54a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.327+0000 7fd16e800640 1 --2- 172.21.15.47:0/2629586481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c54a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.327+0000 7fd166a00640 1 --2- 172.21.15.47:0/1560240950 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1500a2f20 0x7fd1500a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.327+0000 7fd16e800640 1 -- 172.21.15.47:0/2629586481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 msgr2=0x55824c550e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.327+0000 7fd16e800640 1 --2- 172.21.15.47:0/2629586481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 0x55824c550e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.327+0000 7fd16e800640 1 -- 172.21.15.47:0/2629586481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55824c47a080 con 0x55824c5489f0 2024-03-21T11:31:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.327+0000 7fd166a00640 1 -- 172.21.15.47:0/1560240950 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd1600f1990 con 0x7fd1701015a0 2024-03-21T11:31:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.327+0000 7fd16e800640 1 --2- 172.21.15.47:0/1560240950 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1500a2f20 0x7fd1500a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.328+0000 7fd16e800640 1 --2- 172.21.15.47:0/2629586481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c548e10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd16800b700 tx=0x7fd16800bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=956480c21cf2e2dc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.328+0000 7fd15f400640 1 -- 172.21.15.47:0/2629586481 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168022070 con 0x55824c5489f0 2024-03-21T11:31:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.328+0000 7fd15f400640 1 -- 172.21.15.47:0/2629586481 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd168015070 con 0x55824c5489f0 2024-03-21T11:31:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.328+0000 7fd15f400640 1 -- 172.21.15.47:0/2629586481 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168010710 con 0x55824c5489f0 2024-03-21T11:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.329+0000 7fd177dfe740 1 -- 172.21.15.47:0/2629586481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 msgr2=0x55824c548e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.329+0000 7fd16e800640 1 --2- 172.21.15.47:0/1560240950 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1500a2f20 0x7fd1500a53e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd16801f000 tx=0x7fd16800bf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.329+0000 7fd177dfe740 1 --2- 172.21.15.47:0/2629586481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c548e10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd16800b700 tx=0x7fd16800bbc0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.329+0000 7fd177dfe740 1 -- 172.21.15.47:0/2629586481 shutdown_connections 2024-03-21T11:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.329+0000 7fd177dfe740 1 --2- 172.21.15.47:0/2629586481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 0x55824c550e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.329+0000 7fd177dfe740 1 --2- 172.21.15.47:0/2629586481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c54a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.329+0000 7fd177dfe740 1 --2- 172.21.15.47:0/2629586481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c548e10 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.329+0000 7fd177dfe740 1 -- 172.21.15.47:0/2629586481 >> 172.21.15.47:0/2629586481 conn(0x55824c543de0 msgr2=0x55824c5441e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.329+0000 7fd177dfe740 1 -- 172.21.15.47:0/2629586481 shutdown_connections 2024-03-21T11:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.329+0000 7fd177dfe740 1 -- 172.21.15.47:0/2629586481 wait complete. 2024-03-21T11:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.329+0000 7fd177dfe740 1 -- start start 2024-03-21T11:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.330+0000 7fd177dfe740 1 Processor -- start 2024-03-21T11:31:12.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.330+0000 7fd177dfe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c552c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.330+0000 7fd177dfe740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c5531a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.330+0000 7fd177dfe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 0x55824c55a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.330+0000 7fd177dfe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824c551590 con 0x55824c54a580 2024-03-21T11:31:12.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.330+0000 7fd177dfe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824c4690d0 con 0x55824c5489f0 2024-03-21T11:31:12.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.330+0000 7fd177dfe740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824c551710 con 0x55824c549bc0 2024-03-21T11:31:12.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.330+0000 7fd177dfe740 10 client.0 _ll_register_callbacks cb 0x55824c46de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:12.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.330+0000 7fd16de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c552c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.330+0000 7fd16de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c552c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59944/0 (socket says 172.21.15.47:59944) 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.330+0000 7fd16d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c5531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.330+0000 7fd16de00640 1 -- 172.21.15.47:0/1464748540 learned_addr learned my addr 172.21.15.47:0/1464748540 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.330+0000 7fd16e800640 1 --2- 172.21.15.47:0/1464748540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 0x55824c55a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.331+0000 7fd16de00640 1 -- 172.21.15.47:0/1464748540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 msgr2=0x55824c5531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.331+0000 7fd16de00640 1 --2- 172.21.15.47:0/1464748540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c5531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.331+0000 7fd16de00640 1 -- 172.21.15.47:0/1464748540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 msgr2=0x55824c55a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.331+0000 7fd16de00640 1 --2- 172.21.15.47:0/1464748540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 0x55824c55a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.331+0000 7fd16de00640 1 -- 172.21.15.47:0/1464748540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55824c1c13f0 con 0x55824c5489f0 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.331+0000 7fd16d400640 1 --2- 172.21.15.47:0/1464748540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c5531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.331+0000 7fd16de00640 1 --2- 172.21.15.47:0/1464748540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c552c60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd1600c61b0 tx=0x7fd160009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e9bb055c43ec4f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.331+0000 7fd157e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.331+0000 7fd157e00640 1 -- 172.21.15.47:0/1464748540 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1600bd810 con 0x55824c5489f0 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.331+0000 7fd157e00640 1 -- 172.21.15.47:0/1464748540 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd1600ef1b0 con 0x55824c5489f0 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.331+0000 7fd177dfe740 1 -- 172.21.15.47:0/1464748540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd168015070 con 0x55824c5489f0 2024-03-21T11:31:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.331+0000 7fd157e00640 1 -- 172.21.15.47:0/1464748540 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1600bd630 con 0x55824c5489f0 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.331+0000 7fd177dfe740 10 client.28121 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.331+0000 7fd177dfe740 1 -- 172.21.15.47:0/1464748540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55824c28b7d0 con 0x55824c5489f0 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.332+0000 7fd157e00640 1 -- 172.21.15.47:0/1464748540 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd168015070 con 0x55824c5489f0 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.332+0000 7fd157e00640 1 -- 172.21.15.47:0/1464748540 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd1600c6250 con 0x55824c5489f0 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.332+0000 7fd177dfe740 10 client.28121 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.332+0000 7fd177dfe740 10 client.28121 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.332+0000 7fd177dfe740 20 client.28121 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.332+0000 7fd177dfe740 1 --2- 172.21.15.47:0/1464748540 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55824c55e610 0x55824c560a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.332+0000 7fd177dfe740 4 client.28121 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.332+0000 7fd177dfe740 1 -- 172.21.15.47:0/1464748540 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55824c4ba800 con 0x55824c55e610 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.332+0000 7fd16d400640 1 --2- 172.21.15.47:0/1464748540 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55824c55e610 0x55824c560a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.333+0000 7fd16d400640 1 --2- 172.21.15.47:0/1464748540 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55824c55e610 0x55824c560a90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d5f5f95ab7b71144 server_cookie=84ce4813f285a74 in_seq=0 out_seq=0 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.333+0000 7fd157e00640 10 client.28121 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd157e00640 1 -- 172.21.15.47:0/1464748540 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55824c4ba800 con 0x55824c55e610 2024-03-21T11:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd157e00640 10 client.28121 handle_command_reply: tid=0 2024-03-21T11:31:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd177dfe740 1 client.28121 shutdown 2024-03-21T11:31:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd177dfe740 1 -- 172.21.15.47:0/1464748540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 msgr2=0x55824c552c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd177dfe740 1 --2- 172.21.15.47:0/1464748540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c552c60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd1600c61b0 tx=0x7fd160009560 comp rx=0 tx=0).stop 2024-03-21T11:31:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd177dfe740 1 -- 172.21.15.47:0/1464748540 shutdown_connections 2024-03-21T11:31:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd177dfe740 1 --2- 172.21.15.47:0/1464748540 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55824c55e610 0x55824c560a90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd177dfe740 1 --2- 172.21.15.47:0/1464748540 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 0x55824c55a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd177dfe740 1 --2- 172.21.15.47:0/1464748540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c5531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd177dfe740 1 --2- 172.21.15.47:0/1464748540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c552c60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd157e00640 0 client.28121 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:31:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd177dfe740 1 -- 172.21.15.47:0/1464748540 >> 172.21.15.47:0/1464748540 conn(0x55824c543de0 msgr2=0x55824c543160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd177dfe740 1 -- 172.21.15.47:0/1464748540 shutdown_connections 2024-03-21T11:31:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd177dfe740 1 -- 172.21.15.47:0/1464748540 wait complete. 2024-03-21T11:31:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.334+0000 7fd177dfe740 20 client.28121 trim_cache size 0 max 16384 2024-03-21T11:31:12.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.352+0000 7fd177dfe740 1 Processor -- start 2024-03-21T11:31:12.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.352+0000 7fd177dfe740 1 -- start start 2024-03-21T11:31:12.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.352+0000 7fd177dfe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c5489f0 0x55824c547750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.352+0000 7fd177dfe740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c55a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.352+0000 7fd177dfe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c54a580 0x55824c54cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.352+0000 7fd177dfe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1701ac050 con 0x55824c5489f0 2024-03-21T11:31:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.352+0000 7fd177dfe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1701ad9d0 con 0x55824c54a580 2024-03-21T11:31:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.352+0000 7fd177dfe740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1701ac6b0 con 0x55824c549bc0 2024-03-21T11:31:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.353+0000 7fd16d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c55a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.353+0000 7fd16e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c54a580 0x55824c54cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.353+0000 7fd16de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c5489f0 0x55824c547750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.353+0000 7fd16d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c55a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48222/0 (socket says 172.21.15.47:48222) 2024-03-21T11:31:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.353+0000 7fd16d400640 1 -- 172.21.15.47:0/4177610101 learned_addr learned my addr 172.21.15.47:0/4177610101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.353+0000 7fd16e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c54a580 0x55824c54cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59952/0 (socket says 172.21.15.47:59952) 2024-03-21T11:31:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.353+0000 7fd16d400640 1 -- 172.21.15.47:0/4177610101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c54a580 msgr2=0x55824c54cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.353+0000 7fd16d400640 1 --2- 172.21.15.47:0/4177610101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c54a580 0x55824c54cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.353+0000 7fd16d400640 1 -- 172.21.15.47:0/4177610101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c5489f0 msgr2=0x55824c547750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.353+0000 7fd16d400640 1 --2- 172.21.15.47:0/4177610101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c5489f0 0x55824c547750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.353+0000 7fd16d400640 1 -- 172.21.15.47:0/4177610101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55824c549640 con 0x55824c549bc0 2024-03-21T11:31:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.353+0000 7fd16d400640 1 --2- 172.21.15.47:0/4177610101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c55a1a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd158012600 tx=0x7fd158014b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93f1eb8d0f79a10e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.353+0000 7fd15ea00640 1 -- 172.21.15.47:0/4177610101 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158014390 con 0x55824c549bc0 2024-03-21T11:31:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.353+0000 7fd15ea00640 1 -- 172.21.15.47:0/4177610101 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd158003e50 con 0x55824c549bc0 2024-03-21T11:31:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.354+0000 7fd15ea00640 1 -- 172.21.15.47:0/4177610101 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15801d680 con 0x55824c549bc0 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.354+0000 7fd177dfe740 1 -- 172.21.15.47:0/4177610101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 msgr2=0x55824c55a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.354+0000 7fd177dfe740 1 --2- 172.21.15.47:0/4177610101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c55a1a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd158012600 tx=0x7fd158014b00 comp rx=0 tx=0).stop 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.354+0000 7fd177dfe740 1 -- 172.21.15.47:0/4177610101 shutdown_connections 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.354+0000 7fd177dfe740 1 --2- 172.21.15.47:0/4177610101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c54a580 0x55824c54cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.354+0000 7fd177dfe740 1 --2- 172.21.15.47:0/4177610101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c55a1a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.354+0000 7fd177dfe740 1 --2- 172.21.15.47:0/4177610101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c5489f0 0x55824c547750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.354+0000 7fd177dfe740 1 -- 172.21.15.47:0/4177610101 >> 172.21.15.47:0/4177610101 conn(0x55824c543de0 msgr2=0x55824c547040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.354+0000 7fd177dfe740 1 -- 172.21.15.47:0/4177610101 shutdown_connections 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.354+0000 7fd177dfe740 1 -- 172.21.15.47:0/4177610101 wait complete. 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.355+0000 7fd177dfe740 1 -- start start 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.355+0000 7fd177dfe740 1 Processor -- start 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.355+0000 7fd177dfe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c558100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.355+0000 7fd177dfe740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c558640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.355+0000 7fd177dfe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 0x55824c559590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.355+0000 7fd177dfe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1701ad370 con 0x55824c54a580 2024-03-21T11:31:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.355+0000 7fd177dfe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1701acd10 con 0x55824c5489f0 2024-03-21T11:31:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.355+0000 7fd177dfe740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1701aa210 con 0x55824c549bc0 2024-03-21T11:31:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.355+0000 7fd177dfe740 10 client.0 _ll_register_callbacks cb 0x7fd160005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd16d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c558640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd16de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c558100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd16de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c558100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59966/0 (socket says 172.21.15.47:59966) 2024-03-21T11:31:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd16d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c558640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48224/0 (socket says 172.21.15.47:48224) 2024-03-21T11:31:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd16de00640 1 -- 172.21.15.47:0/652977704 learned_addr learned my addr 172.21.15.47:0/652977704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd16e800640 1 --2- 172.21.15.47:0/652977704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 0x55824c559590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd16de00640 1 -- 172.21.15.47:0/652977704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 msgr2=0x55824c558640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd16de00640 1 --2- 172.21.15.47:0/652977704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c558640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd16de00640 1 -- 172.21.15.47:0/652977704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 msgr2=0x55824c559590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd16de00640 1 --2- 172.21.15.47:0/652977704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 0x55824c559590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd16de00640 1 -- 172.21.15.47:0/652977704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55824c559b80 con 0x55824c5489f0 2024-03-21T11:31:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd16d400640 1 --2- 172.21.15.47:0/652977704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c558640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd16e800640 1 --2- 172.21.15.47:0/652977704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 0x55824c559590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd16de00640 1 --2- 172.21.15.47:0/652977704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c558100 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd160006010 tx=0x7fd1600bd1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de626e5ca6f4065 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd157e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.356+0000 7fd157e00640 1 -- 172.21.15.47:0/652977704 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1600b9f70 con 0x55824c5489f0 2024-03-21T11:31:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.357+0000 7fd157e00640 1 -- 172.21.15.47:0/652977704 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd1600c6250 con 0x55824c5489f0 2024-03-21T11:31:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.357+0000 7fd177dfe740 1 -- 172.21.15.47:0/652977704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd158003e50 con 0x55824c5489f0 2024-03-21T11:31:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.357+0000 7fd157e00640 1 -- 172.21.15.47:0/652977704 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1600044a0 con 0x55824c5489f0 2024-03-21T11:31:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.357+0000 7fd177dfe740 10 client.28127 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.357+0000 7fd177dfe740 1 -- 172.21.15.47:0/652977704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55824c5ba6f0 con 0x55824c5489f0 2024-03-21T11:31:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.357+0000 7fd157e00640 1 -- 172.21.15.47:0/652977704 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd158003e50 con 0x55824c5489f0 2024-03-21T11:31:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.357+0000 7fd157e00640 1 -- 172.21.15.47:0/652977704 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd1600bcb90 con 0x55824c5489f0 2024-03-21T11:31:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.357+0000 7fd177dfe740 10 client.28127 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.357+0000 7fd177dfe740 10 client.28127 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:31:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.357+0000 7fd177dfe740 20 client.28127 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.357+0000 7fd177dfe740 1 --2- 172.21.15.47:0/652977704 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55824c5bae50 0x55824c5bd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.357+0000 7fd177dfe740 4 client.28127 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:31:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.357+0000 7fd177dfe740 1 -- 172.21.15.47:0/652977704 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55824c4b4b50 con 0x55824c5bae50 2024-03-21T11:31:12.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.358+0000 7fd16d400640 1 --2- 172.21.15.47:0/652977704 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55824c5bae50 0x55824c5bd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.358+0000 7fd16d400640 1 --2- 172.21.15.47:0/652977704 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55824c5bae50 0x55824c5bd270 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4b60ed9f639ea286 server_cookie=618cc847bd302779 in_seq=0 out_seq=0 2024-03-21T11:31:12.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.358+0000 7fd157e00640 10 client.28127 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:31:12.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.358+0000 7fd157e00640 1 -- 172.21.15.47:0/652977704 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55824c4b4b50 con 0x55824c5bae50 2024-03-21T11:31:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.358+0000 7fd157e00640 10 client.28127 handle_command_reply: tid=0 2024-03-21T11:31:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.358+0000 7fd177dfe740 1 client.28127 shutdown 2024-03-21T11:31:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.358+0000 7fd177dfe740 1 -- 172.21.15.47:0/652977704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 msgr2=0x55824c558100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.358+0000 7fd177dfe740 1 --2- 172.21.15.47:0/652977704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c558100 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd160006010 tx=0x7fd1600bd1a0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd177dfe740 1 -- 172.21.15.47:0/652977704 shutdown_connections 2024-03-21T11:31:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd177dfe740 1 --2- 172.21.15.47:0/652977704 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55824c5bae50 0x55824c5bd270 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd177dfe740 1 --2- 172.21.15.47:0/652977704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55824c54a580 0x55824c559590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd177dfe740 1 --2- 172.21.15.47:0/652977704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55824c549bc0 0x55824c558640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd157e00640 0 client.28127 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:31:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd177dfe740 1 --2- 172.21.15.47:0/652977704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55824c5489f0 0x55824c558100 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd177dfe740 1 -- 172.21.15.47:0/652977704 >> 172.21.15.47:0/652977704 conn(0x55824c543de0 msgr2=0x55824c5478d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd177dfe740 1 -- 172.21.15.47:0/652977704 shutdown_connections 2024-03-21T11:31:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd177dfe740 1 -- 172.21.15.47:0/652977704 wait complete. 2024-03-21T11:31:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd177dfe740 20 client.28127 trim_cache size 0 max 16384 2024-03-21T11:31:12.365 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:31:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd16f200640 1 -- 172.21.15.47:0/1560240950 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1500a2f20 msgr2=0x7fd1500a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd16f200640 1 --2- 172.21.15.47:0/1560240950 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1500a2f20 0x7fd1500a53e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd16801f000 tx=0x7fd16800bf90 comp rx=0 tx=0).stop 2024-03-21T11:31:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd16f200640 1 -- 172.21.15.47:0/1560240950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1701015a0 msgr2=0x7fd170199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd16f200640 1 --2- 172.21.15.47:0/1560240950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1701015a0 0x7fd170199a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd160000c00 tx=0x7fd16002c6c0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd16f200640 1 -- 172.21.15.47:0/1560240950 shutdown_connections 2024-03-21T11:31:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd16f200640 1 --2- 172.21.15.47:0/1560240950 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1500a2f20 0x7fd1500a53e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd16f200640 1 --2- 172.21.15.47:0/1560240950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd170103150 0x7fd1701a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd16f200640 1 --2- 172.21.15.47:0/1560240950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd170102790 0x7fd170199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.359+0000 7fd16f200640 1 --2- 172.21.15.47:0/1560240950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1701015a0 0x7fd170199a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.360+0000 7fd16f200640 1 -- 172.21.15.47:0/1560240950 >> 172.21.15.47:0/1560240950 conn(0x7fd1700fcb10 msgr2=0x7fd1700738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.360+0000 7fd16f200640 1 -- 172.21.15.47:0/1560240950 shutdown_connections 2024-03-21T11:31:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.360+0000 7fd16f200640 1 -- 172.21.15.47:0/1560240950 wait complete. 2024-03-21T11:31:12.376 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:31:12.376 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:31:12.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:12 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/4011191046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:31:12.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:12 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/4011191046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:31:12.534 INFO:tasks.workunit.client.0.smithi047.stdout: 1 553100 8.23 MB/sec execute 236 sec latency 216.505 ms 2024-03-21T11:31:12.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.582+0000 7f2f77000640 1 -- 172.21.15.47:0/339108082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f701086a0 msgr2=0x7f2f70108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.582+0000 7f2f77000640 1 --2- 172.21.15.47:0/339108082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f701086a0 0x7f2f70108ac0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009900 tx=0x7f2f6002bc40 comp rx=0 tx=0).stop 2024-03-21T11:31:12.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.582+0000 7f2f77000640 1 -- 172.21.15.47:0/339108082 shutdown_connections 2024-03-21T11:31:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.582+0000 7f2f77000640 1 --2- 172.21.15.47:0/339108082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f7010ff20 0x7f2f70112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.582+0000 7f2f77000640 1 --2- 172.21.15.47:0/339108082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f70109000 0x7f2f7010f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.582+0000 7f2f77000640 1 --2- 172.21.15.47:0/339108082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f701086a0 0x7f2f70108ac0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.582+0000 7f2f77000640 1 -- 172.21.15.47:0/339108082 >> 172.21.15.47:0/339108082 conn(0x7f2f7006e8a0 msgr2=0x7f2f70070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.582+0000 7f2f77000640 1 -- 172.21.15.47:0/339108082 shutdown_connections 2024-03-21T11:31:12.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.582+0000 7f2f77000640 1 -- 172.21.15.47:0/339108082 wait complete. 2024-03-21T11:31:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.583+0000 7f2f77000640 1 Processor -- start 2024-03-21T11:31:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.583+0000 7f2f77000640 1 -- start start 2024-03-21T11:31:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.583+0000 7f2f77000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f701086a0 0x7f2f701a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.583+0000 7f2f77000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f70109000 0x7f2f701a2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.583+0000 7f2f77000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f7010ff20 0x7f2f701a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.584+0000 7f2f77000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f70115ae0 con 0x7f2f70109000 2024-03-21T11:31:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.584+0000 7f2f77000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f70115960 con 0x7f2f701086a0 2024-03-21T11:31:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.584+0000 7f2f77000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f70115c60 con 0x7f2f7010ff20 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.584+0000 7f2f6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f701086a0 0x7f2f701a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.584+0000 7f2f75000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f7010ff20 0x7f2f701a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.584+0000 7f2f6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f701086a0 0x7f2f701a2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59978/0 (socket says 172.21.15.47:59978) 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.584+0000 7f2f6fe00640 1 -- 172.21.15.47:0/1070546607 learned_addr learned my addr 172.21.15.47:0/1070546607 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.584+0000 7f2f67e00640 1 --2- 172.21.15.47:0/1070546607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f70109000 0x7f2f701a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.584+0000 7f2f6fe00640 1 -- 172.21.15.47:0/1070546607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f7010ff20 msgr2=0x7f2f701a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.584+0000 7f2f6fe00640 1 --2- 172.21.15.47:0/1070546607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f7010ff20 0x7f2f701a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.584+0000 7f2f6fe00640 1 -- 172.21.15.47:0/1070546607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f70109000 msgr2=0x7f2f701a2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.584+0000 7f2f6fe00640 1 --2- 172.21.15.47:0/1070546607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f70109000 0x7f2f701a2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.584+0000 7f2f6fe00640 1 -- 172.21.15.47:0/1070546607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f701aa040 con 0x7f2f701086a0 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.584+0000 7f2f6fe00640 1 --2- 172.21.15.47:0/1070546607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f701086a0 0x7f2f701a2420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009230 tx=0x7f2f60005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.585+0000 7f2f6d600640 1 -- 172.21.15.47:0/1070546607 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f60041070 con 0x7f2f701086a0 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.585+0000 7f2f77000640 1 -- 172.21.15.47:0/1070546607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f701aa2d0 con 0x7f2f701086a0 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.585+0000 7f2f6d600640 1 -- 172.21.15.47:0/1070546607 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f600044d0 con 0x7f2f701086a0 2024-03-21T11:31:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.585+0000 7f2f6d600640 1 -- 172.21.15.47:0/1070546607 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f60034440 con 0x7f2f701086a0 2024-03-21T11:31:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.585+0000 7f2f77000640 1 -- 172.21.15.47:0/1070546607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f701aa810 con 0x7f2f701086a0 2024-03-21T11:31:12.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.588+0000 7f2f6d600640 1 -- 172.21.15.47:0/1070546607 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2f60044070 con 0x7f2f701086a0 2024-03-21T11:31:12.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.588+0000 7f2f79dc6740 1 Processor -- start 2024-03-21T11:31:12.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.589+0000 7f2f79dc6740 1 -- start start 2024-03-21T11:31:12.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.589+0000 7f2f79dc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cd810 0x5605194cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.589+0000 7f2f79dc6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194ce9e0 0x5605194cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.589+0000 7f2f79dc6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cf3a0 0x5605194d5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.589+0000 7f2f79dc6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605194322c0 con 0x5605194ce9e0 2024-03-21T11:31:12.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.589+0000 7f2f79dc6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605193dac90 con 0x5605194cf3a0 2024-03-21T11:31:12.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.589+0000 7f2f79dc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56051942e360 con 0x5605194cd810 2024-03-21T11:31:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.589+0000 7f2f75000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cd810 0x5605194cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.589+0000 7f2f75000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cd810 0x5605194cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48248/0 (socket says 172.21.15.47:48248) 2024-03-21T11:31:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.589+0000 7f2f75000640 1 -- 172.21.15.47:0/2494373787 learned_addr learned my addr 172.21.15.47:0/2494373787 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.589+0000 7f2f67e00640 1 --2- 172.21.15.47:0/2494373787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cf3a0 0x5605194d5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.589+0000 7f2f6fe00640 1 --2- 172.21.15.47:0/2494373787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194ce9e0 0x5605194cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.590+0000 7f2f75000640 1 -- 172.21.15.47:0/2494373787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cf3a0 msgr2=0x5605194d5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.590+0000 7f2f75000640 1 --2- 172.21.15.47:0/2494373787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cf3a0 0x5605194d5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.590+0000 7f2f75000640 1 -- 172.21.15.47:0/2494373787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194ce9e0 msgr2=0x5605194cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.590+0000 7f2f75000640 1 --2- 172.21.15.47:0/2494373787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194ce9e0 0x5605194cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.590+0000 7f2f75000640 1 -- 172.21.15.47:0/2494373787 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605193ff080 con 0x5605194cd810 2024-03-21T11:31:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.590+0000 7f2f6d600640 1 --2- 172.21.15.47:0/1070546607 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2f540a2fa0 0x7f2f540a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.590+0000 7f2f75000640 1 --2- 172.21.15.47:0/2494373787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cd810 0x5605194cdc30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2f68010370 tx=0x7f2f68010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab2fc6aaee7bab37 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.590+0000 7f2f6d600640 1 -- 172.21.15.47:0/1070546607 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2f600f1340 con 0x7f2f701086a0 2024-03-21T11:31:12.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.590+0000 7f2f65600640 1 -- 172.21.15.47:0/2494373787 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68016070 con 0x5605194cd810 2024-03-21T11:31:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.590+0000 7f2f65600640 1 -- 172.21.15.47:0/2494373787 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f68004d20 con 0x5605194cd810 2024-03-21T11:31:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.590+0000 7f2f6fe00640 1 --2- 172.21.15.47:0/1070546607 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2f540a2fa0 0x7f2f540a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.590+0000 7f2f65600640 1 -- 172.21.15.47:0/2494373787 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68005480 con 0x5605194cd810 2024-03-21T11:31:12.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.591+0000 7f2f79dc6740 1 -- 172.21.15.47:0/2494373787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cd810 msgr2=0x5605194cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.591+0000 7f2f6fe00640 1 --2- 172.21.15.47:0/1070546607 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2f540a2fa0 0x7f2f540a5460 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2f6002c7c0 tx=0x7f2f60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.591+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/2494373787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cd810 0x5605194cdc30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2f68010370 tx=0x7f2f68010830 comp rx=0 tx=0).stop 2024-03-21T11:31:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.591+0000 7f2f79dc6740 1 -- 172.21.15.47:0/2494373787 shutdown_connections 2024-03-21T11:31:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.591+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/2494373787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cf3a0 0x5605194d5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.591+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/2494373787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194ce9e0 0x5605194cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.591+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/2494373787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cd810 0x5605194cdc30 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.591+0000 7f2f79dc6740 1 -- 172.21.15.47:0/2494373787 >> 172.21.15.47:0/2494373787 conn(0x5605194c8c00 msgr2=0x5605194c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.591+0000 7f2f79dc6740 1 -- 172.21.15.47:0/2494373787 shutdown_connections 2024-03-21T11:31:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.591+0000 7f2f79dc6740 1 -- 172.21.15.47:0/2494373787 wait complete. 2024-03-21T11:31:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.592+0000 7f2f79dc6740 1 -- start start 2024-03-21T11:31:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.592+0000 7f2f79dc6740 1 Processor -- start 2024-03-21T11:31:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.592+0000 7f2f79dc6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cd810 0x5605194d7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.592+0000 7f2f79dc6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194ce9e0 0x5605194d7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.592+0000 7f2f79dc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cf3a0 0x5605194df010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.592+0000 7f2f79dc6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605194d63b0 con 0x5605194ce9e0 2024-03-21T11:31:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.592+0000 7f2f79dc6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605193ee0d0 con 0x5605194cd810 2024-03-21T11:31:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.592+0000 7f2f79dc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605194d6530 con 0x5605194cf3a0 2024-03-21T11:31:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.592+0000 7f2f79dc6740 10 client.0 _ll_register_callbacks cb 0x5605193f2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.593+0000 7f2f67e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194ce9e0 0x5605194d7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.593+0000 7f2f75000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cd810 0x5605194d7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.593+0000 7f2f67e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194ce9e0 0x5605194d7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53200/0 (socket says 172.21.15.47:53200) 2024-03-21T11:31:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.593+0000 7f2f67e00640 1 -- 172.21.15.47:0/1249431892 learned_addr learned my addr 172.21.15.47:0/1249431892 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.593+0000 7f2f75000640 1 --2- 172.21.15.47:0/1249431892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cf3a0 0x5605194df010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.593+0000 7f2f67e00640 1 -- 172.21.15.47:0/1249431892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cf3a0 msgr2=0x5605194df010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.593+0000 7f2f67e00640 1 --2- 172.21.15.47:0/1249431892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cf3a0 0x5605194df010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.593+0000 7f2f67e00640 1 -- 172.21.15.47:0/1249431892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cd810 msgr2=0x5605194d7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.593+0000 7f2f67e00640 1 --2- 172.21.15.47:0/1249431892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cd810 0x5605194d7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.593+0000 7f2f67e00640 1 -- 172.21.15.47:0/1249431892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605191463f0 con 0x5605194ce9e0 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.593+0000 7f2f75000640 1 --2- 172.21.15.47:0/1249431892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cd810 0x5605194d7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.593+0000 7f2f67e00640 1 --2- 172.21.15.47:0/1249431892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194ce9e0 0x5605194d7fd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f58012700 tx=0x7f2f58012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7569d9031aa7048c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.593+0000 7f2f5e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.594+0000 7f2f5e000640 1 -- 172.21.15.47:0/1249431892 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5800f8c0 con 0x5605194ce9e0 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.594+0000 7f2f5e000640 1 -- 172.21.15.47:0/1249431892 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2f5800fa60 con 0x5605194ce9e0 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.594+0000 7f2f79dc6740 1 -- 172.21.15.47:0/1249431892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f68004d20 con 0x5605194ce9e0 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.594+0000 7f2f5e000640 1 -- 172.21.15.47:0/1249431892 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f58025050 con 0x5605194ce9e0 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.594+0000 7f2f79dc6740 10 client.17826 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.594+0000 7f2f79dc6740 1 -- 172.21.15.47:0/1249431892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5605192107d0 con 0x5605194ce9e0 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.594+0000 7f2f5e000640 1 -- 172.21.15.47:0/1249431892 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f68004d20 con 0x5605194ce9e0 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.594+0000 7f2f5e000640 1 -- 172.21.15.47:0/1249431892 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2f58010430 con 0x5605194ce9e0 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.595+0000 7f2f79dc6740 10 client.17826 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.595+0000 7f2f79dc6740 10 client.17826 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:31:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.595+0000 7f2f79dc6740 20 client.17826 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.595+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/1249431892 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5605194e3550 0x5605194e59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.595+0000 7f2f79dc6740 4 client.17826 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.595+0000 7f2f79dc6740 1 -- 172.21.15.47:0/1249431892 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56051943f800 con 0x5605194e3550 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.595+0000 7f2f75000640 1 --2- 172.21.15.47:0/1249431892 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5605194e3550 0x5605194e59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.595+0000 7f2f75000640 1 --2- 172.21.15.47:0/1249431892 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5605194e3550 0x5605194e59b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4310bc61733d626d server_cookie=a50d58d48a53216b in_seq=0 out_seq=0 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.595+0000 7f2f5e000640 10 client.17826 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.596+0000 7f2f5e000640 1 -- 172.21.15.47:0/1249431892 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56051943f800 con 0x5605194e3550 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.596+0000 7f2f5e000640 10 client.17826 handle_command_reply: tid=0 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.596+0000 7f2f79dc6740 1 client.17826 shutdown 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.596+0000 7f2f79dc6740 1 -- 172.21.15.47:0/1249431892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194ce9e0 msgr2=0x5605194d7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.596+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/1249431892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194ce9e0 0x5605194d7fd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f58012700 tx=0x7f2f58012bc0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.596+0000 7f2f79dc6740 1 -- 172.21.15.47:0/1249431892 shutdown_connections 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.596+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/1249431892 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5605194e3550 0x5605194e59b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.596+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/1249431892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cf3a0 0x5605194df010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.596+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/1249431892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194ce9e0 0x5605194d7fd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.596+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/1249431892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cd810 0x5605194d7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.596+0000 7f2f5e000640 0 client.17826 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:31:12.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.596+0000 7f2f79dc6740 1 -- 172.21.15.47:0/1249431892 >> 172.21.15.47:0/1249431892 conn(0x5605194c8c00 msgr2=0x5605194c7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.597+0000 7f2f79dc6740 1 -- 172.21.15.47:0/1249431892 shutdown_connections 2024-03-21T11:31:12.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.597+0000 7f2f79dc6740 1 -- 172.21.15.47:0/1249431892 wait complete. 2024-03-21T11:31:12.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.597+0000 7f2f79dc6740 20 client.17826 trim_cache size 0 max 16384 2024-03-21T11:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.610+0000 7f2f79dc6740 1 Processor -- start 2024-03-21T11:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.610+0000 7f2f79dc6740 1 -- start start 2024-03-21T11:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f79dc6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cd810 0x5605194d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f79dc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194ce9e0 0x5605194defd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f79dc6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194cf3a0 0x5605194d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f79dc6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f701b4b60 con 0x5605194cf3a0 2024-03-21T11:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f79dc6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f701b64e0 con 0x5605194cd810 2024-03-21T11:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f79dc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f701b51c0 con 0x5605194ce9e0 2024-03-21T11:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194ce9e0 0x5605194defd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194ce9e0 0x5605194defd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48268/0 (socket says 172.21.15.47:48268) 2024-03-21T11:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f67e00640 1 -- 172.21.15.47:0/3205206899 learned_addr learned my addr 172.21.15.47:0/3205206899 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f75000640 1 --2- 172.21.15.47:0/3205206899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cd810 0x5605194d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f75000640 1 --2- 172.21.15.47:0/3205206899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194cf3a0 0x5605194d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f67e00640 1 -- 172.21.15.47:0/3205206899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cd810 msgr2=0x5605194d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f67e00640 1 --2- 172.21.15.47:0/3205206899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cd810 0x5605194d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f67e00640 1 -- 172.21.15.47:0/3205206899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194cf3a0 msgr2=0x5605194d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f67e00640 1 --2- 172.21.15.47:0/3205206899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194cf3a0 0x5605194d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f67e00640 1 -- 172.21.15.47:0/3205206899 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605194c32c0 con 0x5605194ce9e0 2024-03-21T11:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.611+0000 7f2f75000640 1 --2- 172.21.15.47:0/3205206899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194cf3a0 0x5605194d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.612+0000 7f2f67e00640 1 --2- 172.21.15.47:0/3205206899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194ce9e0 0x5605194defd0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2f58009e00 tx=0x7f2f58011db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2756ac6a3948c04a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.612+0000 7f2f64c00640 1 -- 172.21.15.47:0/3205206899 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58010a40 con 0x5605194ce9e0 2024-03-21T11:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.612+0000 7f2f64c00640 1 -- 172.21.15.47:0/3205206899 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f580055b0 con 0x5605194ce9e0 2024-03-21T11:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.612+0000 7f2f64c00640 1 -- 172.21.15.47:0/3205206899 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5800fb10 con 0x5605194ce9e0 2024-03-21T11:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.613+0000 7f2f79dc6740 1 -- 172.21.15.47:0/3205206899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194ce9e0 msgr2=0x5605194defd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.613+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/3205206899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194ce9e0 0x5605194defd0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2f58009e00 tx=0x7f2f58011db0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.613+0000 7f2f79dc6740 1 -- 172.21.15.47:0/3205206899 shutdown_connections 2024-03-21T11:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.613+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/3205206899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194cf3a0 0x5605194d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.613+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/3205206899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194ce9e0 0x5605194defd0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.613+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/3205206899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194cd810 0x5605194d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.613+0000 7f2f79dc6740 1 -- 172.21.15.47:0/3205206899 >> 172.21.15.47:0/3205206899 conn(0x5605194c8c00 msgr2=0x5605194cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.613+0000 7f2f79dc6740 1 -- 172.21.15.47:0/3205206899 shutdown_connections 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.613+0000 7f2f79dc6740 1 -- 172.21.15.47:0/3205206899 wait complete. 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.613+0000 7f2f79dc6740 1 -- start start 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.613+0000 7f2f79dc6740 1 Processor -- start 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f79dc6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194cd810 0x5605194daca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f79dc6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194ce9e0 0x5605194db1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f79dc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cf3a0 0x5605194de200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f79dc6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f701b5e80 con 0x5605194cd810 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f79dc6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f701b5820 con 0x5605194ce9e0 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f79dc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f701b2d20 con 0x5605194cf3a0 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f79dc6740 10 client.0 _ll_register_callbacks cb 0x7f2f58009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f67e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194ce9e0 0x5605194db1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f75000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194cd810 0x5605194daca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f67e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194ce9e0 0x5605194db1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60022/0 (socket says 172.21.15.47:60022) 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f67e00640 1 -- 172.21.15.47:0/3432144957 learned_addr learned my addr 172.21.15.47:0/3432144957 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f75000640 1 --2- 172.21.15.47:0/3432144957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cf3a0 0x5605194de200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f67e00640 1 -- 172.21.15.47:0/3432144957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cf3a0 msgr2=0x5605194de200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f67e00640 1 --2- 172.21.15.47:0/3432144957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cf3a0 0x5605194de200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f67e00640 1 -- 172.21.15.47:0/3432144957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194cd810 msgr2=0x5605194daca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f67e00640 1 --2- 172.21.15.47:0/3432144957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194cd810 0x5605194daca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.614+0000 7f2f67e00640 1 -- 172.21.15.47:0/3432144957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605194de7f0 con 0x5605194ce9e0 2024-03-21T11:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.615+0000 7f2f67e00640 1 --2- 172.21.15.47:0/3432144957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194ce9e0 0x5605194db1e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2f58010c20 tx=0x7f2f5800f830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1a8e29606c7a8e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.615+0000 7f2f5e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.615+0000 7f2f5e000640 1 -- 172.21.15.47:0/3432144957 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f580050d0 con 0x5605194ce9e0 2024-03-21T11:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.615+0000 7f2f79dc6740 1 -- 172.21.15.47:0/3432144957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f580055b0 con 0x5605194ce9e0 2024-03-21T11:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.615+0000 7f2f5e000640 1 -- 172.21.15.47:0/3432144957 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2f58018050 con 0x5605194ce9e0 2024-03-21T11:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.615+0000 7f2f5e000640 1 -- 172.21.15.47:0/3432144957 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f58005270 con 0x5605194ce9e0 2024-03-21T11:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.615+0000 7f2f79dc6740 10 client.28145 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.615+0000 7f2f79dc6740 1 -- 172.21.15.47:0/3432144957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560519547b70 con 0x5605194ce9e0 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.615+0000 7f2f5e000640 1 -- 172.21.15.47:0/3432144957 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f580055b0 con 0x5605194ce9e0 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.616+0000 7f2f5e000640 1 -- 172.21.15.47:0/3432144957 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2f58030070 con 0x5605194ce9e0 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.616+0000 7f2f79dc6740 10 client.28145 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.616+0000 7f2f79dc6740 10 client.28145 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.616+0000 7f2f79dc6740 20 client.28145 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.616+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/3432144957 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5605195483e0 0x56051954a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.616+0000 7f2f79dc6740 4 client.28145 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.616+0000 7f2f79dc6740 1 -- 172.21.15.47:0/3432144957 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560519439b50 con 0x5605195483e0 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.616+0000 7f2f75000640 1 --2- 172.21.15.47:0/3432144957 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5605195483e0 0x56051954a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.616+0000 7f2f75000640 1 --2- 172.21.15.47:0/3432144957 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5605195483e0 0x56051954a800 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3280bbf679662fdb server_cookie=f7c201dfe50f435 in_seq=0 out_seq=0 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.616+0000 7f2f5e000640 10 client.28145 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f5e000640 1 -- 172.21.15.47:0/3432144957 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560519439b50 con 0x5605195483e0 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f5e000640 10 client.28145 handle_command_reply: tid=0 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f79dc6740 1 client.28145 shutdown 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f79dc6740 1 -- 172.21.15.47:0/3432144957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194ce9e0 msgr2=0x5605194db1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/3432144957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194ce9e0 0x5605194db1e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2f58010c20 tx=0x7f2f5800f830 comp rx=0 tx=0).stop 2024-03-21T11:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f79dc6740 1 -- 172.21.15.47:0/3432144957 shutdown_connections 2024-03-21T11:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/3432144957 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5605195483e0 0x56051954a800 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/3432144957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605194cf3a0 0x5605194de200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/3432144957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605194ce9e0 0x5605194db1e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f5e000640 0 client.28145 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f79dc6740 1 --2- 172.21.15.47:0/3432144957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605194cd810 0x5605194daca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f79dc6740 1 -- 172.21.15.47:0/3432144957 >> 172.21.15.47:0/3432144957 conn(0x5605194c8c00 msgr2=0x5605194c7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f79dc6740 1 -- 172.21.15.47:0/3432144957 shutdown_connections 2024-03-21T11:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f79dc6740 1 -- 172.21.15.47:0/3432144957 wait complete. 2024-03-21T11:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.617+0000 7f2f79dc6740 20 client.28145 trim_cache size 0 max 16384 2024-03-21T11:31:12.623 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.618+0000 7f2f77000640 1 -- 172.21.15.47:0/1070546607 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2f540a2fa0 msgr2=0x7f2f540a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.618+0000 7f2f77000640 1 --2- 172.21.15.47:0/1070546607 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2f540a2fa0 0x7f2f540a5460 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2f6002c7c0 tx=0x7f2f60009260 comp rx=0 tx=0).stop 2024-03-21T11:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.618+0000 7f2f77000640 1 -- 172.21.15.47:0/1070546607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f701086a0 msgr2=0x7f2f701a2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.618+0000 7f2f77000640 1 --2- 172.21.15.47:0/1070546607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f701086a0 0x7f2f701a2420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009230 tx=0x7f2f60005d50 comp rx=0 tx=0).stop 2024-03-21T11:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.618+0000 7f2f77000640 1 -- 172.21.15.47:0/1070546607 shutdown_connections 2024-03-21T11:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.618+0000 7f2f77000640 1 --2- 172.21.15.47:0/1070546607 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2f540a2fa0 0x7f2f540a5460 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.618+0000 7f2f77000640 1 --2- 172.21.15.47:0/1070546607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f7010ff20 0x7f2f701a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.618+0000 7f2f77000640 1 --2- 172.21.15.47:0/1070546607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f70109000 0x7f2f701a2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.618+0000 7f2f77000640 1 --2- 172.21.15.47:0/1070546607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f701086a0 0x7f2f701a2420 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.618+0000 7f2f77000640 1 -- 172.21.15.47:0/1070546607 >> 172.21.15.47:0/1070546607 conn(0x7f2f7006e8a0 msgr2=0x7f2f70070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.618+0000 7f2f77000640 1 -- 172.21.15.47:0/1070546607 shutdown_connections 2024-03-21T11:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.618+0000 7f2f77000640 1 -- 172.21.15.47:0/1070546607 wait complete. 2024-03-21T11:31:12.637 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:31:12.637 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:31:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.854+0000 7f4c28c00640 1 -- 172.21.15.47:0/2286000604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c241038c0 msgr2=0x7f4c24103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.854+0000 7f4c28c00640 1 --2- 172.21.15.47:0/2286000604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c241038c0 0x7f4c24103ce0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009900 tx=0x7f4c1402bc30 comp rx=0 tx=0).stop 2024-03-21T11:31:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.854+0000 7f4c28c00640 1 -- 172.21.15.47:0/2286000604 shutdown_connections 2024-03-21T11:31:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.854+0000 7f4c28c00640 1 --2- 172.21.15.47:0/2286000604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c24105470 0x7f4c2410bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.854+0000 7f4c28c00640 1 --2- 172.21.15.47:0/2286000604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c24104ab0 0x7f4c24104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.854+0000 7f4c28c00640 1 --2- 172.21.15.47:0/2286000604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c241038c0 0x7f4c24103ce0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.854+0000 7f4c28c00640 1 -- 172.21.15.47:0/2286000604 >> 172.21.15.47:0/2286000604 conn(0x7f4c240fee50 msgr2=0x7f4c24101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.854+0000 7f4c28c00640 1 -- 172.21.15.47:0/2286000604 shutdown_connections 2024-03-21T11:31:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.855+0000 7f4c28c00640 1 -- 172.21.15.47:0/2286000604 wait complete. 2024-03-21T11:31:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.856+0000 7f4c28c00640 1 Processor -- start 2024-03-21T11:31:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.856+0000 7f4c28c00640 1 -- start start 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.856+0000 7f4c28c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c241038c0 0x7f4c24199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.856+0000 7f4c28c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c24104ab0 0x7f4c2419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.856+0000 7f4c28c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c24105470 0x7f4c241a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.856+0000 7f4c28c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2410f560 con 0x7f4c24105470 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.856+0000 7f4c28c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2410f3e0 con 0x7f4c241038c0 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.856+0000 7f4c28c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2410f6e0 con 0x7f4c24104ab0 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.856+0000 7f4c21e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c241038c0 0x7f4c24199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.857+0000 7f4c21400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c24104ab0 0x7f4c2419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.857+0000 7f4c21e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c241038c0 0x7f4c24199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60052/0 (socket says 172.21.15.47:60052) 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.857+0000 7f4c21e00640 1 -- 172.21.15.47:0/1407011163 learned_addr learned my addr 172.21.15.47:0/1407011163 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.857+0000 7f4c22800640 1 --2- 172.21.15.47:0/1407011163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c24105470 0x7f4c241a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.857+0000 7f4c21e00640 1 -- 172.21.15.47:0/1407011163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c24104ab0 msgr2=0x7f4c2419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.857+0000 7f4c21e00640 1 --2- 172.21.15.47:0/1407011163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c24104ab0 0x7f4c2419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.857+0000 7f4c21e00640 1 -- 172.21.15.47:0/1407011163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c24105470 msgr2=0x7f4c241a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.857+0000 7f4c21e00640 1 --2- 172.21.15.47:0/1407011163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c24105470 0x7f4c241a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.857+0000 7f4c21e00640 1 -- 172.21.15.47:0/1407011163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c241a17f0 con 0x7f4c241038c0 2024-03-21T11:31:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.857+0000 7f4c21400640 1 --2- 172.21.15.47:0/1407011163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c24104ab0 0x7f4c2419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.857+0000 7f4c21e00640 1 --2- 172.21.15.47:0/1407011163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c241038c0 0x7f4c24199bd0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4c14000c00 tx=0x7f4c1402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.858+0000 7f4c1aa00640 1 -- 172.21.15.47:0/1407011163 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14043020 con 0x7f4c241038c0 2024-03-21T11:31:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.858+0000 7f4c1aa00640 1 -- 172.21.15.47:0/1407011163 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c140054e0 con 0x7f4c241038c0 2024-03-21T11:31:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.858+0000 7f4c28c00640 1 -- 172.21.15.47:0/1407011163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c241a1a80 con 0x7f4c241038c0 2024-03-21T11:31:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.858+0000 7f4c1aa00640 1 -- 172.21.15.47:0/1407011163 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14003c40 con 0x7f4c241038c0 2024-03-21T11:31:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.858+0000 7f4c28c00640 1 -- 172.21.15.47:0/1407011163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c241a2020 con 0x7f4c241038c0 2024-03-21T11:31:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.862+0000 7f4c1aa00640 1 -- 172.21.15.47:0/1407011163 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4c14046020 con 0x7f4c241038c0 2024-03-21T11:31:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.862+0000 7f4c2b98a740 1 Processor -- start 2024-03-21T11:31:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.863+0000 7f4c2b98a740 1 -- start start 2024-03-21T11:31:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.863+0000 7f4c2b98a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf5ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.863+0000 7f4c2b98a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf5fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.863+0000 7f4c2b98a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 0x56296bf66c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.863+0000 7f4c2b98a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bec32c0 con 0x56296bf5f9e0 2024-03-21T11:31:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.863+0000 7f4c2b98a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296be6bc90 con 0x56296bf603a0 2024-03-21T11:31:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.863+0000 7f4c2b98a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bebf360 con 0x56296bf5e810 2024-03-21T11:31:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.863+0000 7f4c22800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf5ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.863+0000 7f4c22800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf5ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48314/0 (socket says 172.21.15.47:48314) 2024-03-21T11:31:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.863+0000 7f4c22800640 1 -- 172.21.15.47:0/2739035548 learned_addr learned my addr 172.21.15.47:0/2739035548 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.863+0000 7f4c21e00640 1 --2- 172.21.15.47:0/2739035548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf5fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.864+0000 7f4c21400640 1 --2- 172.21.15.47:0/2739035548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 0x56296bf66c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.864+0000 7f4c22800640 1 -- 172.21.15.47:0/2739035548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 msgr2=0x56296bf66c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.864+0000 7f4c22800640 1 --2- 172.21.15.47:0/2739035548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 0x56296bf66c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.864+0000 7f4c22800640 1 -- 172.21.15.47:0/2739035548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 msgr2=0x56296bf5fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.864+0000 7f4c22800640 1 --2- 172.21.15.47:0/2739035548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf5fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.864+0000 7f4c22800640 1 -- 172.21.15.47:0/2739035548 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56296be90080 con 0x56296bf5e810 2024-03-21T11:31:12.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.864+0000 7f4c21400640 1 --2- 172.21.15.47:0/2739035548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 0x56296bf66c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:12.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.864+0000 7f4c21e00640 1 --2- 172.21.15.47:0/2739035548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf5fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:12.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.864+0000 7f4c1aa00640 1 --2- 172.21.15.47:0/1407011163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c080a2fa0 0x7f4c080a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.864+0000 7f4c22800640 1 --2- 172.21.15.47:0/2739035548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf5ec30 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009fd0 tx=0x7f4c1c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef05a70dfc223b5c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.864+0000 7f4c1aa00640 1 -- 172.21.15.47:0/1407011163 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4c140f16a0 con 0x7f4c241038c0 2024-03-21T11:31:12.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.864+0000 7f4c21e00640 1 --2- 172.21.15.47:0/1407011163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c080a2fa0 0x7f4c080a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.864+0000 7f4c0f000640 1 -- 172.21.15.47:0/2739035548 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c004d20 con 0x56296bf5e810 2024-03-21T11:31:12.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.865+0000 7f4c0f000640 1 -- 172.21.15.47:0/2739035548 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c1c004050 con 0x56296bf5e810 2024-03-21T11:31:12.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.865+0000 7f4c0f000640 1 -- 172.21.15.47:0/2739035548 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c005220 con 0x56296bf5e810 2024-03-21T11:31:12.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.865+0000 7f4c2b98a740 1 -- 172.21.15.47:0/2739035548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 msgr2=0x56296bf5ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.865+0000 7f4c21e00640 1 --2- 172.21.15.47:0/1407011163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c080a2fa0 0x7f4c080a5460 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4c1402c170 tx=0x7f4c14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.865+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/2739035548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf5ec30 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009fd0 tx=0x7f4c1c0118d0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.865+0000 7f4c2b98a740 1 -- 172.21.15.47:0/2739035548 shutdown_connections 2024-03-21T11:31:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.865+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/2739035548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 0x56296bf66c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.865+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/2739035548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf5fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.865+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/2739035548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf5ec30 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.865+0000 7f4c2b98a740 1 -- 172.21.15.47:0/2739035548 >> 172.21.15.47:0/2739035548 conn(0x56296bf59c00 msgr2=0x56296bf5a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.866+0000 7f4c2b98a740 1 -- 172.21.15.47:0/2739035548 shutdown_connections 2024-03-21T11:31:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.866+0000 7f4c2b98a740 1 -- 172.21.15.47:0/2739035548 wait complete. 2024-03-21T11:31:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.866+0000 7f4c2b98a740 1 -- start start 2024-03-21T11:31:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.866+0000 7f4c2b98a740 1 Processor -- start 2024-03-21T11:31:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c2b98a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf68a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c2b98a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf68fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c2b98a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 0x56296bf70000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c2b98a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf673b0 con 0x56296bf5f9e0 2024-03-21T11:31:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c2b98a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296be7f0d0 con 0x56296bf603a0 2024-03-21T11:31:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c2b98a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf67530 con 0x56296bf5e810 2024-03-21T11:31:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c2b98a740 10 client.0 _ll_register_callbacks cb 0x56296be83e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c22800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf68a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c22800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf68a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48328/0 (socket says 172.21.15.47:48328) 2024-03-21T11:31:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c22800640 1 -- 172.21.15.47:0/3089221488 learned_addr learned my addr 172.21.15.47:0/3089221488 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c21400640 1 --2- 172.21.15.47:0/3089221488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf68fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c22800640 1 --2- 172.21.15.47:0/3089221488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 0x56296bf70000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c22800640 1 -- 172.21.15.47:0/3089221488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 msgr2=0x56296bf70000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c22800640 1 --2- 172.21.15.47:0/3089221488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 0x56296bf70000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c22800640 1 -- 172.21.15.47:0/3089221488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 msgr2=0x56296bf68fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c22800640 1 --2- 172.21.15.47:0/3089221488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf68fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c22800640 1 -- 172.21.15.47:0/3089221488 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56296bbd73f0 con 0x56296bf5e810 2024-03-21T11:31:12.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.867+0000 7f4c21400640 1 --2- 172.21.15.47:0/3089221488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf68fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.868+0000 7f4c22800640 1 --2- 172.21.15.47:0/3089221488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf68a80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c003eb0 tx=0x7f4c1c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a88ab8c034a66192 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.868+0000 7f4c03400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.868+0000 7f4c03400640 1 -- 172.21.15.47:0/3089221488 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c00f6e0 con 0x56296bf5e810 2024-03-21T11:31:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.868+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3089221488 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c1c004050 con 0x56296bf5e810 2024-03-21T11:31:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.868+0000 7f4c03400640 1 -- 172.21.15.47:0/3089221488 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4c1c00f880 con 0x56296bf5e810 2024-03-21T11:31:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.868+0000 7f4c03400640 1 -- 172.21.15.47:0/3089221488 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c1c036050 con 0x56296bf5e810 2024-03-21T11:31:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.868+0000 7f4c2b98a740 10 client.28165 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.868+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3089221488 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56296bca17d0 con 0x56296bf5e810 2024-03-21T11:31:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.868+0000 7f4c03400640 1 -- 172.21.15.47:0/3089221488 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c1c004050 con 0x56296bf5e810 2024-03-21T11:31:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.869+0000 7f4c03400640 1 -- 172.21.15.47:0/3089221488 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4c1c025040 con 0x56296bf5e810 2024-03-21T11:31:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.869+0000 7f4c2b98a740 10 client.28165 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.869+0000 7f4c2b98a740 10 client.28165 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:31:12.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.869+0000 7f4c2b98a740 20 client.28165 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.869+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3089221488 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56296bf74310 0x56296bf76700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.869+0000 7f4c2b98a740 4 client.28165 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.869+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3089221488 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56296bed0800 con 0x56296bf74310 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.869+0000 7f4c21400640 1 --2- 172.21.15.47:0/3089221488 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56296bf74310 0x56296bf76700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.870+0000 7f4c21400640 1 --2- 172.21.15.47:0/3089221488 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56296bf74310 0x56296bf76700 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75b21708a745c2ed server_cookie=722ad537eb7f078a in_seq=0 out_seq=0 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.870+0000 7f4c03400640 10 client.28165 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.870+0000 7f4c03400640 1 -- 172.21.15.47:0/3089221488 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56296bed0800 con 0x56296bf74310 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.870+0000 7f4c03400640 10 client.28165 handle_command_reply: tid=0 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.870+0000 7f4c2b98a740 1 client.28165 shutdown 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.870+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3089221488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 msgr2=0x56296bf68a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.870+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3089221488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf68a80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c003eb0 tx=0x7f4c1c000f40 comp rx=0 tx=0).stop 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.871+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3089221488 shutdown_connections 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.871+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3089221488 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56296bf74310 0x56296bf76700 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.871+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3089221488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 0x56296bf70000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.871+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3089221488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf68fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.871+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3089221488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf68a80 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.871+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3089221488 >> 172.21.15.47:0/3089221488 conn(0x56296bf59c00 msgr2=0x56296bf58eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.871+0000 7f4c03400640 0 client.28165 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:31:12.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.871+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3089221488 shutdown_connections 2024-03-21T11:31:12.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.871+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3089221488 wait complete. 2024-03-21T11:31:12.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.871+0000 7f4c2b98a740 20 client.28165 trim_cache size 0 max 16384 2024-03-21T11:31:12.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.886+0000 7f4c2b98a740 1 Processor -- start 2024-03-21T11:31:12.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.886+0000 7f4c2b98a740 1 -- start start 2024-03-21T11:31:12.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.886+0000 7f4c2b98a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf7afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.886+0000 7f4c2b98a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf7b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.886+0000 7f4c2b98a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 0x56296bf7ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.886+0000 7f4c2b98a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241ac380 con 0x56296bf5f9e0 2024-03-21T11:31:12.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.886+0000 7f4c2b98a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241add00 con 0x56296bf603a0 2024-03-21T11:31:12.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.886+0000 7f4c2b98a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241ac9e0 con 0x56296bf5e810 2024-03-21T11:31:12.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.887+0000 7f4c21400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf7b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.887+0000 7f4c22800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf7afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.887+0000 7f4c21400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf7b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53274/0 (socket says 172.21.15.47:53274) 2024-03-21T11:31:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.887+0000 7f4c21400640 1 -- 172.21.15.47:0/3232486889 learned_addr learned my addr 172.21.15.47:0/3232486889 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.887+0000 7f4c22800640 1 --2- 172.21.15.47:0/3232486889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 0x56296bf7ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.887+0000 7f4c21400640 1 -- 172.21.15.47:0/3232486889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 msgr2=0x56296bf7afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.887+0000 7f4c21400640 1 --2- 172.21.15.47:0/3232486889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf7afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.887+0000 7f4c21400640 1 -- 172.21.15.47:0/3232486889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 msgr2=0x56296bf7ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.887+0000 7f4c21400640 1 --2- 172.21.15.47:0/3232486889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 0x56296bf7ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.887+0000 7f4c21400640 1 -- 172.21.15.47:0/3232486889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56296bf76a70 con 0x56296bf5f9e0 2024-03-21T11:31:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.887+0000 7f4c21400640 1 --2- 172.21.15.47:0/3232486889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf7b4e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4c10012780 tx=0x7f4c10005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c8a11f58757cc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.888+0000 7f4c0e600640 1 -- 172.21.15.47:0/3232486889 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c100201d0 con 0x56296bf5f9e0 2024-03-21T11:31:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.888+0000 7f4c0e600640 1 -- 172.21.15.47:0/3232486889 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c10024020 con 0x56296bf5f9e0 2024-03-21T11:31:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.888+0000 7f4c0e600640 1 -- 172.21.15.47:0/3232486889 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10020a90 con 0x56296bf5f9e0 2024-03-21T11:31:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.888+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3232486889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 msgr2=0x56296bf7b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.888+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3232486889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf7b4e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4c10012780 tx=0x7f4c10005f90 comp rx=0 tx=0).stop 2024-03-21T11:31:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.889+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3232486889 shutdown_connections 2024-03-21T11:31:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.889+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3232486889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf603a0 0x56296bf7ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.889+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3232486889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5f9e0 0x56296bf7b4e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.889+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3232486889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf5e810 0x56296bf7afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.889+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3232486889 >> 172.21.15.47:0/3232486889 conn(0x56296bf59c00 msgr2=0x56296bf5d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.889+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3232486889 shutdown_connections 2024-03-21T11:31:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.889+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3232486889 wait complete. 2024-03-21T11:31:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.889+0000 7f4c2b98a740 1 -- start start 2024-03-21T11:31:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.889+0000 7f4c2b98a740 1 Processor -- start 2024-03-21T11:31:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.890+0000 7f4c2b98a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5e810 0x56296bf6d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.890+0000 7f4c2b98a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf5f9e0 0x56296bf6d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.890+0000 7f4c2b98a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf603a0 0x56296bf6e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.890+0000 7f4c2b98a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241ad6a0 con 0x56296bf5e810 2024-03-21T11:31:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.890+0000 7f4c2b98a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241ad040 con 0x56296bf5f9e0 2024-03-21T11:31:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.890+0000 7f4c2b98a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241aa540 con 0x56296bf603a0 2024-03-21T11:31:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.890+0000 7f4c2b98a740 10 client.0 _ll_register_callbacks cb 0x7f4c1c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.890+0000 7f4c22800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5e810 0x56296bf6d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.890+0000 7f4c21400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf5f9e0 0x56296bf6d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.890+0000 7f4c22800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5e810 0x56296bf6d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53278/0 (socket says 172.21.15.47:53278) 2024-03-21T11:31:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.890+0000 7f4c22800640 1 -- 172.21.15.47:0/3742752544 learned_addr learned my addr 172.21.15.47:0/3742752544 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.890+0000 7f4c22800640 1 --2- 172.21.15.47:0/3742752544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf603a0 0x56296bf6e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.891+0000 7f4c21400640 1 -- 172.21.15.47:0/3742752544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf603a0 msgr2=0x56296bf6e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.891+0000 7f4c21400640 1 --2- 172.21.15.47:0/3742752544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf603a0 0x56296bf6e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.891+0000 7f4c21400640 1 -- 172.21.15.47:0/3742752544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5e810 msgr2=0x56296bf6d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.891+0000 7f4c21400640 1 --2- 172.21.15.47:0/3742752544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5e810 0x56296bf6d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.891+0000 7f4c21400640 1 -- 172.21.15.47:0/3742752544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56296bf6ecd0 con 0x56296bf5f9e0 2024-03-21T11:31:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.891+0000 7f4c22800640 1 --2- 172.21.15.47:0/3742752544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5e810 0x56296bf6d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.891+0000 7f4c21400640 1 --2- 172.21.15.47:0/3742752544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf5f9e0 0x56296bf6d6d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x56296bf69f10 tx=0x7f4c10005710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dfe4a84aba5b4fe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.891+0000 7f4c03400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.891+0000 7f4c03400640 1 -- 172.21.15.47:0/3742752544 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1002f060 con 0x56296bf5f9e0 2024-03-21T11:31:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.891+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3742752544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c10024020 con 0x56296bf5f9e0 2024-03-21T11:31:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.891+0000 7f4c03400640 1 -- 172.21.15.47:0/3742752544 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4c10004030 con 0x56296bf5f9e0 2024-03-21T11:31:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.891+0000 7f4c03400640 1 -- 172.21.15.47:0/3742752544 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c10004230 con 0x56296bf5f9e0 2024-03-21T11:31:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.892+0000 7f4c2b98a740 10 client.28163 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.892+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3742752544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56296bfcfeb0 con 0x56296bf5f9e0 2024-03-21T11:31:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.892+0000 7f4c03400640 1 -- 172.21.15.47:0/3742752544 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c10024020 con 0x56296bf5f9e0 2024-03-21T11:31:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.892+0000 7f4c03400640 1 -- 172.21.15.47:0/3742752544 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4c100038c0 con 0x56296bf5f9e0 2024-03-21T11:31:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.892+0000 7f4c2b98a740 10 client.28163 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.892+0000 7f4c2b98a740 10 client.28163 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:31:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.892+0000 7f4c2b98a740 20 client.28163 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.892+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3742752544 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56296bfd06d0 0x56296bfd2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.892+0000 7f4c2b98a740 4 client.28163 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:31:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.892+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3742752544 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56296becab50 con 0x56296bfd06d0 2024-03-21T11:31:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.892+0000 7f4c22800640 1 --2- 172.21.15.47:0/3742752544 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56296bfd06d0 0x56296bfd2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:12.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c22800640 1 --2- 172.21.15.47:0/3742752544 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56296bfd06d0 0x56296bfd2b10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be5822e41f57d3fd server_cookie=80a77fe6e6e03ba3 in_seq=0 out_seq=0 2024-03-21T11:31:12.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c03400640 10 client.28163 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:31:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c03400640 1 -- 172.21.15.47:0/3742752544 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56296becab50 con 0x56296bfd06d0 2024-03-21T11:31:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c03400640 10 client.28163 handle_command_reply: tid=0 2024-03-21T11:31:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c2b98a740 1 client.28163 shutdown 2024-03-21T11:31:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3742752544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf5f9e0 msgr2=0x56296bf6d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3742752544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf5f9e0 0x56296bf6d6d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x56296bf69f10 tx=0x7f4c10005710 comp rx=0 tx=0).stop 2024-03-21T11:31:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3742752544 shutdown_connections 2024-03-21T11:31:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3742752544 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56296bfd06d0 0x56296bfd2b10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3742752544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56296bf603a0 0x56296bf6e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3742752544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56296bf5f9e0 0x56296bf6d6d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c03400640 0 client.28163 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:31:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c2b98a740 1 --2- 172.21.15.47:0/3742752544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56296bf5e810 0x56296bf6d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3742752544 >> 172.21.15.47:0/3742752544 conn(0x56296bf59c00 msgr2=0x56296bf62c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.893+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3742752544 shutdown_connections 2024-03-21T11:31:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.894+0000 7f4c2b98a740 1 -- 172.21.15.47:0/3742752544 wait complete. 2024-03-21T11:31:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.894+0000 7f4c2b98a740 20 client.28163 trim_cache size 0 max 16384 2024-03-21T11:31:12.899 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:31:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.894+0000 7f4c28c00640 1 -- 172.21.15.47:0/1407011163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c080a2fa0 msgr2=0x7f4c080a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.894+0000 7f4c28c00640 1 --2- 172.21.15.47:0/1407011163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c080a2fa0 0x7f4c080a5460 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4c1402c170 tx=0x7f4c14009260 comp rx=0 tx=0).stop 2024-03-21T11:31:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.894+0000 7f4c28c00640 1 -- 172.21.15.47:0/1407011163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c241038c0 msgr2=0x7f4c24199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.894+0000 7f4c28c00640 1 --2- 172.21.15.47:0/1407011163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c241038c0 0x7f4c24199bd0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4c14000c00 tx=0x7f4c1402c650 comp rx=0 tx=0).stop 2024-03-21T11:31:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.894+0000 7f4c28c00640 1 -- 172.21.15.47:0/1407011163 shutdown_connections 2024-03-21T11:31:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.894+0000 7f4c28c00640 1 --2- 172.21.15.47:0/1407011163 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c080a2fa0 0x7f4c080a5460 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.894+0000 7f4c28c00640 1 --2- 172.21.15.47:0/1407011163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c24105470 0x7f4c241a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.894+0000 7f4c28c00640 1 --2- 172.21.15.47:0/1407011163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c24104ab0 0x7f4c2419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.894+0000 7f4c28c00640 1 --2- 172.21.15.47:0/1407011163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c241038c0 0x7f4c24199bd0 secure :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4c14000c00 tx=0x7f4c1402c650 comp rx=0 tx=0).stop 2024-03-21T11:31:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.894+0000 7f4c28c00640 1 -- 172.21.15.47:0/1407011163 >> 172.21.15.47:0/1407011163 conn(0x7f4c240fee50 msgr2=0x7f4c24107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.895+0000 7f4c28c00640 1 -- 172.21.15.47:0/1407011163 shutdown_connections 2024-03-21T11:31:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:12.895+0000 7f4c28c00640 1 -- 172.21.15.47:0/1407011163 wait complete. 2024-03-21T11:31:12.914 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:31:12.914 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:31:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.133+0000 7fdec8a00640 1 -- 172.21.15.47:0/4164957216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec4077c40 msgr2=0x7fdec40784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.133+0000 7fdec8a00640 1 --2- 172.21.15.47:0/4164957216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec4077c40 0x7fdec40784e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009900 tx=0x7fdebc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:31:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.133+0000 7fdec8a00640 1 -- 172.21.15.47:0/4164957216 shutdown_connections 2024-03-21T11:31:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.133+0000 7fdec8a00640 1 --2- 172.21.15.47:0/4164957216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec4077c40 0x7fdec40784e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.133+0000 7fdec8a00640 1 --2- 172.21.15.47:0/4164957216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdec40772a0 0x7fdec4077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.133+0000 7fdec8a00640 1 --2- 172.21.15.47:0/4164957216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdec4076060 0x7fdec4076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.133+0000 7fdec8a00640 1 -- 172.21.15.47:0/4164957216 >> 172.21.15.47:0/4164957216 conn(0x7fdec40fed50 msgr2=0x7fdec4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.133+0000 7fdec8a00640 1 -- 172.21.15.47:0/4164957216 shutdown_connections 2024-03-21T11:31:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.133+0000 7fdec8a00640 1 -- 172.21.15.47:0/4164957216 wait complete. 2024-03-21T11:31:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.134+0000 7fdec8a00640 1 Processor -- start 2024-03-21T11:31:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.134+0000 7fdec8a00640 1 -- start start 2024-03-21T11:31:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec4076060 0x7fdec419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec8a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdec40772a0 0x7fdec419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdec4077c40 0x7fdec41a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec8a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec4113890 con 0x7fdec40772a0 2024-03-21T11:31:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec8a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec4113710 con 0x7fdec4077c40 2024-03-21T11:31:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec8a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec4113a10 con 0x7fdec4076060 2024-03-21T11:31:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec4076060 0x7fdec419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec4076060 0x7fdec419de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48368/0 (socket says 172.21.15.47:48368) 2024-03-21T11:31:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdec4077c40 0x7fdec41a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec3400640 1 -- 172.21.15.47:0/1841236672 learned_addr learned my addr 172.21.15.47:0/1841236672 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdebaa00640 1 --2- 172.21.15.47:0/1841236672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdec40772a0 0x7fdec419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec3400640 1 -- 172.21.15.47:0/1841236672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdec4077c40 msgr2=0x7fdec41a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec3400640 1 --2- 172.21.15.47:0/1841236672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdec4077c40 0x7fdec41a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec3400640 1 -- 172.21.15.47:0/1841236672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdec40772a0 msgr2=0x7fdec419e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec3400640 1 --2- 172.21.15.47:0/1841236672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdec40772a0 0x7fdec419e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.135+0000 7fdec3400640 1 -- 172.21.15.47:0/1841236672 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec41a5a30 con 0x7fdec4076060 2024-03-21T11:31:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.136+0000 7fdec3400640 1 --2- 172.21.15.47:0/1841236672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec4076060 0x7fdec419de10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400eb90 tx=0x7fdeb400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:13.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.136+0000 7fdec0c00640 1 -- 172.21.15.47:0/1841236672 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4004d20 con 0x7fdec4076060 2024-03-21T11:31:13.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.136+0000 7fdec0c00640 1 -- 172.21.15.47:0/1841236672 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdeb400ee90 con 0x7fdec4076060 2024-03-21T11:31:13.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.136+0000 7fdec8a00640 1 -- 172.21.15.47:0/1841236672 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec41a5d20 con 0x7fdec4076060 2024-03-21T11:31:13.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.136+0000 7fdec0c00640 1 -- 172.21.15.47:0/1841236672 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4010850 con 0x7fdec4076060 2024-03-21T11:31:13.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.136+0000 7fdec8a00640 1 -- 172.21.15.47:0/1841236672 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec41a61c0 con 0x7fdec4076060 2024-03-21T11:31:13.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.139+0000 7fdeccc18740 1 Processor -- start 2024-03-21T11:31:13.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.139+0000 7fdeccc18740 1 -- start start 2024-03-21T11:31:13.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.140+0000 7fdeccc18740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bea9f0 0x5574d6beae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.140+0000 7fdeccc18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bebbc0 0x5574d6bec040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.140+0000 7fdeccc18740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bec580 0x5574d6bf2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.140+0000 7fdeccc18740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d6b4f2c0 con 0x5574d6bea9f0 2024-03-21T11:31:13.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.140+0000 7fdeccc18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d6af7c90 con 0x5574d6bebbc0 2024-03-21T11:31:13.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.140+0000 7fdeccc18740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d6b4b360 con 0x5574d6bec580 2024-03-21T11:31:13.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.140+0000 7fdec0c00640 1 -- 172.21.15.47:0/1841236672 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdeb4005020 con 0x7fdec4076060 2024-03-21T11:31:13.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.140+0000 7fdec3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bea9f0 0x5574d6beae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.140+0000 7fdebaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bec580 0x5574d6bf2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.140+0000 7fdebaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bec580 0x5574d6bf2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48384/0 (socket says 172.21.15.47:48384) 2024-03-21T11:31:13.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.140+0000 7fdebaa00640 1 -- 172.21.15.47:0/1947909484 learned_addr learned my addr 172.21.15.47:0/1947909484 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:13.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.140+0000 7fdec3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bea9f0 0x5574d6beae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53308/0 (socket says 172.21.15.47:53308) 2024-03-21T11:31:13.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.140+0000 7fdec3400640 1 --2- 172.21.15.47:0/1947909484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bebbc0 0x5574d6bec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.141+0000 7fdebaa00640 1 -- 172.21.15.47:0/1947909484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bebbc0 msgr2=0x5574d6bec040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:13.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.141+0000 7fdebaa00640 1 --2- 172.21.15.47:0/1947909484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bebbc0 0x5574d6bec040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.141+0000 7fdebaa00640 1 -- 172.21.15.47:0/1947909484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bea9f0 msgr2=0x5574d6beae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:13.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.141+0000 7fdebaa00640 1 --2- 172.21.15.47:0/1947909484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bea9f0 0x5574d6beae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.141+0000 7fdebaa00640 1 -- 172.21.15.47:0/1947909484 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574d6b1c080 con 0x5574d6bec580 2024-03-21T11:31:13.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.141+0000 7fdebaa00640 1 --2- 172.21.15.47:0/1947909484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bec580 0x5574d6bf2e10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009c10 tx=0x7fdeac0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3de633deb116de41 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.141+0000 7fdeb3e00640 1 -- 172.21.15.47:0/1947909484 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac012450 con 0x5574d6bec580 2024-03-21T11:31:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.141+0000 7fdeb3e00640 1 -- 172.21.15.47:0/1947909484 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdeac012ad0 con 0x5574d6bec580 2024-03-21T11:31:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.141+0000 7fdeb3e00640 1 -- 172.21.15.47:0/1947909484 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00fbf0 con 0x5574d6bec580 2024-03-21T11:31:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.142+0000 7fdec0c00640 1 --2- 172.21.15.47:0/1841236672 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdea80a2f20 0x7fdea80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.142+0000 7fdeccc18740 1 -- 172.21.15.47:0/1947909484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bec580 msgr2=0x5574d6bf2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.142+0000 7fdeccc18740 1 --2- 172.21.15.47:0/1947909484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bec580 0x5574d6bf2e10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009c10 tx=0x7fdeac0117b0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.142+0000 7fdec3e00640 1 --2- 172.21.15.47:0/1841236672 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdea80a2f20 0x7fdea80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.142+0000 7fdec0c00640 1 -- 172.21.15.47:0/1841236672 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdeb40d0070 con 0x7fdec4076060 2024-03-21T11:31:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.142+0000 7fdeccc18740 1 -- 172.21.15.47:0/1947909484 shutdown_connections 2024-03-21T11:31:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.142+0000 7fdeccc18740 1 --2- 172.21.15.47:0/1947909484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bec580 0x5574d6bf2e10 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.142+0000 7fdeccc18740 1 --2- 172.21.15.47:0/1947909484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bebbc0 0x5574d6bec040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.142+0000 7fdeccc18740 1 --2- 172.21.15.47:0/1947909484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bea9f0 0x5574d6beae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.142+0000 7fdeccc18740 1 -- 172.21.15.47:0/1947909484 >> 172.21.15.47:0/1947909484 conn(0x5574d6be5de0 msgr2=0x5574d6be61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.142+0000 7fdeccc18740 1 -- 172.21.15.47:0/1947909484 shutdown_connections 2024-03-21T11:31:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.143+0000 7fdeccc18740 1 -- 172.21.15.47:0/1947909484 wait complete. 2024-03-21T11:31:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.143+0000 7fdeccc18740 1 -- start start 2024-03-21T11:31:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.143+0000 7fdec3e00640 1 --2- 172.21.15.47:0/1841236672 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdea80a2f20 0x7fdea80a53e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdebc036040 tx=0x7fdebc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.143+0000 7fdeccc18740 1 Processor -- start 2024-03-21T11:31:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.144+0000 7fdeccc18740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bea9f0 0x5574d6bf4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.144+0000 7fdeccc18740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bebbc0 0x5574d6bf5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.144+0000 7fdeccc18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bec580 0x5574d6bfc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.144+0000 7fdeccc18740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d6bf3590 con 0x5574d6bea9f0 2024-03-21T11:31:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.144+0000 7fdeccc18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d6b0b0d0 con 0x5574d6bec580 2024-03-21T11:31:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.144+0000 7fdeccc18740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d6bf3710 con 0x5574d6bebbc0 2024-03-21T11:31:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.144+0000 7fdeccc18740 10 client.0 _ll_register_callbacks cb 0x5574d6b0fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.144+0000 7fdec3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bea9f0 0x5574d6bf4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.144+0000 7fdec3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bea9f0 0x5574d6bf4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53312/0 (socket says 172.21.15.47:53312) 2024-03-21T11:31:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.144+0000 7fdebaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bebbc0 0x5574d6bf5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.144+0000 7fdec3400640 1 -- 172.21.15.47:0/4126051241 learned_addr learned my addr 172.21.15.47:0/4126051241 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.144+0000 7fdec3e00640 1 --2- 172.21.15.47:0/4126051241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bec580 0x5574d6bfc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.144+0000 7fdec3400640 1 -- 172.21.15.47:0/4126051241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bebbc0 msgr2=0x5574d6bf5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:13.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.145+0000 7fdec3400640 1 --2- 172.21.15.47:0/4126051241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bebbc0 0x5574d6bf5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.145+0000 7fdec3400640 1 -- 172.21.15.47:0/4126051241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bec580 msgr2=0x5574d6bfc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:13.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.145+0000 7fdec3400640 1 --2- 172.21.15.47:0/4126051241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bec580 0x5574d6bfc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.145+0000 7fdec3400640 1 -- 172.21.15.47:0/4126051241 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574d68633f0 con 0x5574d6bea9f0 2024-03-21T11:31:13.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.145+0000 7fdebaa00640 1 --2- 172.21.15.47:0/4126051241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bebbc0 0x5574d6bf5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.145+0000 7fdec3400640 1 --2- 172.21.15.47:0/4126051241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bea9f0 0x5574d6bf4c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40049d0 tx=0x7fdeb40d3740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20cbc151bf0749b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.145+0000 7fdeb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:31:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.145+0000 7fdeb0c00640 1 -- 172.21.15.47:0/4126051241 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb409c1e0 con 0x5574d6bea9f0 2024-03-21T11:31:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.145+0000 7fdeb0c00640 1 -- 172.21.15.47:0/4126051241 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdeb4005310 con 0x5574d6bea9f0 2024-03-21T11:31:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.145+0000 7fdeccc18740 1 -- 172.21.15.47:0/4126051241 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdeac012ad0 con 0x5574d6bea9f0 2024-03-21T11:31:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.145+0000 7fdeb0c00640 1 -- 172.21.15.47:0/4126051241 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeb409bd70 con 0x5574d6bea9f0 2024-03-21T11:31:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.145+0000 7fdeccc18740 10 client.17859 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.145+0000 7fdeccc18740 1 -- 172.21.15.47:0/4126051241 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5574d692d7d0 con 0x5574d6bea9f0 2024-03-21T11:31:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.146+0000 7fdeb0c00640 1 -- 172.21.15.47:0/4126051241 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdeac012ad0 con 0x5574d6bea9f0 2024-03-21T11:31:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.146+0000 7fdeb0c00640 1 -- 172.21.15.47:0/4126051241 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdeb40a0120 con 0x5574d6bea9f0 2024-03-21T11:31:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.146+0000 7fdeccc18740 10 client.17859 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.146+0000 7fdeccc18740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.146+0000 7fdeccc18740 20 client.17859 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.146+0000 7fdeccc18740 1 --2- 172.21.15.47:0/4126051241 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5574d6c00780 0x5574d6c02bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.146+0000 7fdeccc18740 4 client.17859 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.146+0000 7fdeccc18740 1 -- 172.21.15.47:0/4126051241 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574d6b5c800 con 0x5574d6c00780 2024-03-21T11:31:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.146+0000 7fdebaa00640 1 --2- 172.21.15.47:0/4126051241 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5574d6c00780 0x5574d6c02bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.147+0000 7fdebaa00640 1 --2- 172.21.15.47:0/4126051241 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5574d6c00780 0x5574d6c02bd0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68d2f2ed3f4b3346 server_cookie=30aedd180dafe62c in_seq=0 out_seq=0 2024-03-21T11:31:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.147+0000 7fdeb0c00640 10 client.17859 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.148+0000 7fdeb0c00640 1 -- 172.21.15.47:0/4126051241 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5574d6b5c800 con 0x5574d6c00780 2024-03-21T11:31:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.148+0000 7fdeb0c00640 10 client.17859 handle_command_reply: tid=0 2024-03-21T11:31:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.148+0000 7fdeccc18740 1 client.17859 shutdown 2024-03-21T11:31:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.148+0000 7fdeccc18740 1 -- 172.21.15.47:0/4126051241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bea9f0 msgr2=0x5574d6bf4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.148+0000 7fdeccc18740 1 --2- 172.21.15.47:0/4126051241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bea9f0 0x5574d6bf4c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40049d0 tx=0x7fdeb40d3740 comp rx=0 tx=0).stop 2024-03-21T11:31:13.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.148+0000 7fdeccc18740 1 -- 172.21.15.47:0/4126051241 shutdown_connections 2024-03-21T11:31:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.148+0000 7fdeccc18740 1 --2- 172.21.15.47:0/4126051241 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5574d6c00780 0x5574d6c02bd0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.148+0000 7fdeccc18740 1 --2- 172.21.15.47:0/4126051241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bec580 0x5574d6bfc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.148+0000 7fdeccc18740 1 --2- 172.21.15.47:0/4126051241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bebbc0 0x5574d6bf5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.148+0000 7fdeccc18740 1 --2- 172.21.15.47:0/4126051241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bea9f0 0x5574d6bf4c10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.148+0000 7fdeccc18740 1 -- 172.21.15.47:0/4126051241 >> 172.21.15.47:0/4126051241 conn(0x5574d6be5de0 msgr2=0x5574d6be5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.148+0000 7fdeb0c00640 0 client.17859 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.148+0000 7fdeccc18740 1 -- 172.21.15.47:0/4126051241 shutdown_connections 2024-03-21T11:31:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.148+0000 7fdeccc18740 1 -- 172.21.15.47:0/4126051241 wait complete. 2024-03-21T11:31:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.149+0000 7fdeccc18740 20 client.17859 trim_cache size 0 max 16384 2024-03-21T11:31:13.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.163+0000 7fdeccc18740 1 Processor -- start 2024-03-21T11:31:13.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.163+0000 7fdeccc18740 1 -- start start 2024-03-21T11:31:13.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdeccc18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bea9f0 0x5574d6beedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdeccc18740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bebbc0 0x5574d6bfc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdeccc18740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bec580 0x5574d6bef330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdeccc18740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec41b03e0 con 0x5574d6bec580 2024-03-21T11:31:13.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdeccc18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec41b1d60 con 0x5574d6bea9f0 2024-03-21T11:31:13.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdeccc18740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec41b0a40 con 0x5574d6bebbc0 2024-03-21T11:31:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdec3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bec580 0x5574d6bef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdebaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bebbc0 0x5574d6bfc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdec3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bec580 0x5574d6bef330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53324/0 (socket says 172.21.15.47:53324) 2024-03-21T11:31:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdec3e00640 1 -- 172.21.15.47:0/653219685 learned_addr learned my addr 172.21.15.47:0/653219685 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdec3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bea9f0 0x5574d6beedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdec3e00640 1 -- 172.21.15.47:0/653219685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bebbc0 msgr2=0x5574d6bfc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdec3e00640 1 --2- 172.21.15.47:0/653219685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bebbc0 0x5574d6bfc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdec3e00640 1 -- 172.21.15.47:0/653219685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bea9f0 msgr2=0x5574d6beedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:13.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.164+0000 7fdec3e00640 1 --2- 172.21.15.47:0/653219685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bea9f0 0x5574d6beedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.165+0000 7fdec3e00640 1 -- 172.21.15.47:0/653219685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574d6beb640 con 0x5574d6bec580 2024-03-21T11:31:13.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.165+0000 7fdec3e00640 1 --2- 172.21.15.47:0/653219685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bec580 0x5574d6bef330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005ab0 tx=0x7fdebc041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8180c22fd298c09c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:13.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.165+0000 7fdeb3400640 1 -- 172.21.15.47:0/653219685 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc045070 con 0x5574d6bec580 2024-03-21T11:31:13.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.165+0000 7fdeb3400640 1 -- 172.21.15.47:0/653219685 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdebc0041c0 con 0x5574d6bec580 2024-03-21T11:31:13.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.165+0000 7fdeb3400640 1 -- 172.21.15.47:0/653219685 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc034440 con 0x5574d6bec580 2024-03-21T11:31:13.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.166+0000 7fdeccc18740 1 -- 172.21.15.47:0/653219685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bec580 msgr2=0x5574d6bef330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:13.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.166+0000 7fdeccc18740 1 --2- 172.21.15.47:0/653219685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bec580 0x5574d6bef330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005ab0 tx=0x7fdebc041000 comp rx=0 tx=0).stop 2024-03-21T11:31:13.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.166+0000 7fdeccc18740 1 -- 172.21.15.47:0/653219685 shutdown_connections 2024-03-21T11:31:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.166+0000 7fdeccc18740 1 --2- 172.21.15.47:0/653219685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bec580 0x5574d6bef330 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.166+0000 7fdeccc18740 1 --2- 172.21.15.47:0/653219685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bebbc0 0x5574d6bfc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.166+0000 7fdeccc18740 1 --2- 172.21.15.47:0/653219685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bea9f0 0x5574d6beedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.166+0000 7fdeccc18740 1 -- 172.21.15.47:0/653219685 >> 172.21.15.47:0/653219685 conn(0x5574d6be5de0 msgr2=0x5574d6be98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.166+0000 7fdeccc18740 1 -- 172.21.15.47:0/653219685 shutdown_connections 2024-03-21T11:31:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.166+0000 7fdeccc18740 1 -- 172.21.15.47:0/653219685 wait complete. 2024-03-21T11:31:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.167+0000 7fdeccc18740 1 -- start start 2024-03-21T11:31:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.167+0000 7fdeccc18740 1 Processor -- start 2024-03-21T11:31:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.167+0000 7fdeccc18740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bea9f0 0x5574d6bfa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.167+0000 7fdeccc18740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bebbc0 0x5574d6bfa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.167+0000 7fdeccc18740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bec580 0x5574d6bfb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.167+0000 7fdeccc18740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec41b1700 con 0x5574d6bec580 2024-03-21T11:31:13.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.167+0000 7fdeccc18740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec41b10a0 con 0x5574d6bebbc0 2024-03-21T11:31:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.167+0000 7fdeccc18740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec41ae5a0 con 0x5574d6bea9f0 2024-03-21T11:31:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.167+0000 7fdeccc18740 10 client.0 _ll_register_callbacks cb 0x7fdeb4004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.167+0000 7fdec3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bea9f0 0x5574d6bfa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.167+0000 7fdec3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bea9f0 0x5574d6bfa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48414/0 (socket says 172.21.15.47:48414) 2024-03-21T11:31:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.167+0000 7fdebaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bebbc0 0x5574d6bfa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.167+0000 7fdec3400640 1 -- 172.21.15.47:0/3705899486 learned_addr learned my addr 172.21.15.47:0/3705899486 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.167+0000 7fdec3e00640 1 --2- 172.21.15.47:0/3705899486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bec580 0x5574d6bfb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.168+0000 7fdec3400640 1 -- 172.21.15.47:0/3705899486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bebbc0 msgr2=0x5574d6bfa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.168+0000 7fdec3400640 1 --2- 172.21.15.47:0/3705899486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bebbc0 0x5574d6bfa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.168+0000 7fdec3400640 1 -- 172.21.15.47:0/3705899486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bec580 msgr2=0x5574d6bfb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.168+0000 7fdec3400640 1 --2- 172.21.15.47:0/3705899486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bec580 0x5574d6bfb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.168+0000 7fdec3400640 1 -- 172.21.15.47:0/3705899486 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574d6bfbb80 con 0x5574d6bea9f0 2024-03-21T11:31:13.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.168+0000 7fdec3400640 1 --2- 172.21.15.47:0/3705899486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bea9f0 0x5574d6bfa090 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40980f0 tx=0x7fdeb4009a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5e640f2e7cde704 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.168+0000 7fdeb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.168+0000 7fdeb0c00640 1 -- 172.21.15.47:0/3705899486 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb409bf50 con 0x5574d6bea9f0 2024-03-21T11:31:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.168+0000 7fdeb0c00640 1 -- 172.21.15.47:0/3705899486 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdeb409b860 con 0x5574d6bea9f0 2024-03-21T11:31:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.168+0000 7fdeccc18740 1 -- 172.21.15.47:0/3705899486 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdebc0041c0 con 0x5574d6bea9f0 2024-03-21T11:31:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.168+0000 7fdeb0c00640 1 -- 172.21.15.47:0/3705899486 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeb409cb10 con 0x5574d6bea9f0 2024-03-21T11:31:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.168+0000 7fdeccc18740 10 client.28189 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.168+0000 7fdeccc18740 1 -- 172.21.15.47:0/3705899486 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5574d6c5c7d0 con 0x5574d6bea9f0 2024-03-21T11:31:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.169+0000 7fdeb0c00640 1 -- 172.21.15.47:0/3705899486 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdebc0041c0 con 0x5574d6bea9f0 2024-03-21T11:31:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.169+0000 7fdeb0c00640 1 -- 172.21.15.47:0/3705899486 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdeb4067110 con 0x5574d6bea9f0 2024-03-21T11:31:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.169+0000 7fdeccc18740 10 client.28189 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.169+0000 7fdeccc18740 10 client.28189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.169+0000 7fdeccc18740 20 client.28189 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:13.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.169+0000 7fdeccc18740 1 --2- 172.21.15.47:0/3705899486 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5574d6c5cf00 0x5574d6c5f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.169+0000 7fdeccc18740 4 client.28189 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:31:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.169+0000 7fdeccc18740 1 -- 172.21.15.47:0/3705899486 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5574d6b56b50 con 0x5574d6c5cf00 2024-03-21T11:31:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.169+0000 7fdebaa00640 1 --2- 172.21.15.47:0/3705899486 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5574d6c5cf00 0x5574d6c5f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.170+0000 7fdebaa00640 1 --2- 172.21.15.47:0/3705899486 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5574d6c5cf00 0x5574d6c5f320 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddc6b465c9c3cb76 server_cookie=926e37bcba643561 in_seq=0 out_seq=0 2024-03-21T11:31:13.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.170+0000 7fdeb0c00640 10 client.28189 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeb0c00640 1 -- 172.21.15.47:0/3705899486 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5574d6b56b50 con 0x5574d6c5cf00 2024-03-21T11:31:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeb0c00640 10 client.28189 handle_command_reply: tid=0 2024-03-21T11:31:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeccc18740 1 client.28189 shutdown 2024-03-21T11:31:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeccc18740 1 -- 172.21.15.47:0/3705899486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bea9f0 msgr2=0x5574d6bfa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeccc18740 1 --2- 172.21.15.47:0/3705899486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bea9f0 0x5574d6bfa090 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40980f0 tx=0x7fdeb4009a70 comp rx=0 tx=0).stop 2024-03-21T11:31:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeccc18740 1 -- 172.21.15.47:0/3705899486 shutdown_connections 2024-03-21T11:31:13.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeccc18740 1 --2- 172.21.15.47:0/3705899486 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5574d6c5cf00 0x5574d6c5f320 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeccc18740 1 --2- 172.21.15.47:0/3705899486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5574d6bec580 0x5574d6bfb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeccc18740 1 --2- 172.21.15.47:0/3705899486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574d6bebbc0 0x5574d6bfa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeccc18740 1 --2- 172.21.15.47:0/3705899486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5574d6bea9f0 0x5574d6bfa090 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeccc18740 1 -- 172.21.15.47:0/3705899486 >> 172.21.15.47:0/3705899486 conn(0x5574d6be5de0 msgr2=0x5574d6be48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeb0c00640 0 client.28189 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeccc18740 1 -- 172.21.15.47:0/3705899486 shutdown_connections 2024-03-21T11:31:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeccc18740 1 -- 172.21.15.47:0/3705899486 wait complete. 2024-03-21T11:31:13.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.171+0000 7fdeccc18740 20 client.28189 trim_cache size 0 max 16384 2024-03-21T11:31:13.176 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:31:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.172+0000 7fdec8a00640 1 -- 172.21.15.47:0/1841236672 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdea80a2f20 msgr2=0x7fdea80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.172+0000 7fdec8a00640 1 --2- 172.21.15.47:0/1841236672 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdea80a2f20 0x7fdea80a53e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdebc036040 tx=0x7fdebc02df90 comp rx=0 tx=0).stop 2024-03-21T11:31:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.172+0000 7fdec8a00640 1 -- 172.21.15.47:0/1841236672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec4076060 msgr2=0x7fdec419de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.172+0000 7fdec8a00640 1 --2- 172.21.15.47:0/1841236672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec4076060 0x7fdec419de10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400eb90 tx=0x7fdeb400c6a0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.172+0000 7fdec8a00640 1 -- 172.21.15.47:0/1841236672 shutdown_connections 2024-03-21T11:31:13.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.172+0000 7fdec8a00640 1 --2- 172.21.15.47:0/1841236672 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdea80a2f20 0x7fdea80a53e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.172+0000 7fdec8a00640 1 --2- 172.21.15.47:0/1841236672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdec4077c40 0x7fdec41a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.172+0000 7fdec8a00640 1 --2- 172.21.15.47:0/1841236672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdec40772a0 0x7fdec419e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.172+0000 7fdec8a00640 1 --2- 172.21.15.47:0/1841236672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec4076060 0x7fdec419de10 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:13.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.172+0000 7fdec8a00640 1 -- 172.21.15.47:0/1841236672 >> 172.21.15.47:0/1841236672 conn(0x7fdec40fed50 msgr2=0x7fdec4100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:13.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.172+0000 7fdec8a00640 1 -- 172.21.15.47:0/1841236672 shutdown_connections 2024-03-21T11:31:13.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:13.172+0000 7fdec8a00640 1 -- 172.21.15.47:0/1841236672 wait complete. 2024-03-21T11:31:13.188 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:13 smithi047 ceph-mon[31351]: pgmap v636: 129 pgs: 129 active+clean; 39 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-03-21T11:31:13.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:13 smithi100 ceph-mon[36245]: pgmap v636: 129 pgs: 129 active+clean; 39 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-03-21T11:31:13.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:13 smithi102 ceph-mon[36748]: pgmap v636: 129 pgs: 129 active+clean; 39 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2024-03-21T11:31:13.533 INFO:tasks.workunit.client.0.smithi047.stdout: 1 554712 8.24 MB/sec execute 237 sec latency 9.005 ms 2024-03-21T11:31:14.191 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:31:14.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:31:14.192 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:31:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.450+0000 7f7914e00640 1 -- 172.21.15.47:0/73755283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79101038e0 msgr2=0x7f7910103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.450+0000 7f7914e00640 1 --2- 172.21.15.47:0/73755283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79101038e0 0x7f7910103d00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7900009860 tx=0x7f790002bb70 comp rx=0 tx=0).stop 2024-03-21T11:31:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.450+0000 7f7914e00640 1 -- 172.21.15.47:0/73755283 shutdown_connections 2024-03-21T11:31:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.450+0000 7f7914e00640 1 --2- 172.21.15.47:0/73755283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7910105490 0x7f791010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.450+0000 7f7914e00640 1 --2- 172.21.15.47:0/73755283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7910104ad0 0x7f7910104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.451+0000 7f7914e00640 1 --2- 172.21.15.47:0/73755283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79101038e0 0x7f7910103d00 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.451+0000 7f7914e00640 1 -- 172.21.15.47:0/73755283 >> 172.21.15.47:0/73755283 conn(0x7f79100fee50 msgr2=0x7f7910101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.451+0000 7f7914e00640 1 -- 172.21.15.47:0/73755283 shutdown_connections 2024-03-21T11:31:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.451+0000 7f7914e00640 1 -- 172.21.15.47:0/73755283 wait complete. 2024-03-21T11:31:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.452+0000 7f7914e00640 1 Processor -- start 2024-03-21T11:31:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.452+0000 7f7914e00640 1 -- start start 2024-03-21T11:31:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.452+0000 7f7914e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79101038e0 0x7f7910199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.452+0000 7f7914e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7910104ad0 0x7f791019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.452+0000 7f7914e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7910105490 0x7f79101a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.452+0000 7f7914e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791010f530 con 0x7f7910105490 2024-03-21T11:31:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.452+0000 7f7914e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791010f3b0 con 0x7f79101038e0 2024-03-21T11:31:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.452+0000 7f7914e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791010f6b0 con 0x7f7910104ad0 2024-03-21T11:31:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.452+0000 7f790de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79101038e0 0x7f7910199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.452+0000 7f790d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7910104ad0 0x7f791019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.452+0000 7f790de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79101038e0 0x7f7910199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60172/0 (socket says 172.21.15.47:60172) 2024-03-21T11:31:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.452+0000 7f790e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7910105490 0x7f79101a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.452+0000 7f790de00640 1 -- 172.21.15.47:0/3332747882 learned_addr learned my addr 172.21.15.47:0/3332747882 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.452+0000 7f790d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7910104ad0 0x7f791019a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48446/0 (socket says 172.21.15.47:48446) 2024-03-21T11:31:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.453+0000 7f790de00640 1 -- 172.21.15.47:0/3332747882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7910104ad0 msgr2=0x7f791019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.453+0000 7f790de00640 1 --2- 172.21.15.47:0/3332747882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7910104ad0 0x7f791019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.453+0000 7f790de00640 1 -- 172.21.15.47:0/3332747882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7910105490 msgr2=0x7f79101a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.453+0000 7f790de00640 1 --2- 172.21.15.47:0/3332747882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7910105490 0x7f79101a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.453+0000 7f790de00640 1 -- 172.21.15.47:0/3332747882 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79101a1880 con 0x7f79101038e0 2024-03-21T11:31:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.453+0000 7f790de00640 1 --2- 172.21.15.47:0/3332747882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79101038e0 0x7f7910199c70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f79000349e0 tx=0x7f7900032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.453+0000 7f7906a00640 1 -- 172.21.15.47:0/3332747882 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900041020 con 0x7f79101038e0 2024-03-21T11:31:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.453+0000 7f7914e00640 1 -- 172.21.15.47:0/3332747882 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79101a1b10 con 0x7f79101038e0 2024-03-21T11:31:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.453+0000 7f7906a00640 1 -- 172.21.15.47:0/3332747882 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f790002c9f0 con 0x7f79101038e0 2024-03-21T11:31:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.454+0000 7f7906a00640 1 -- 172.21.15.47:0/3332747882 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900039040 con 0x7f79101038e0 2024-03-21T11:31:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.454+0000 7f7914e00640 1 -- 172.21.15.47:0/3332747882 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79101a2050 con 0x7f79101038e0 2024-03-21T11:31:14.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.457+0000 7f7906a00640 1 -- 172.21.15.47:0/3332747882 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7900044020 con 0x7f79101038e0 2024-03-21T11:31:14.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.457+0000 7f7917a22740 1 Processor -- start 2024-03-21T11:31:14.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.458+0000 7f7917a22740 1 -- start start 2024-03-21T11:31:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.458+0000 7f7917a22740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d1260a0 0x56516d1264c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.458+0000 7f7917a22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 0x56516d12b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.458+0000 7f7917a22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d12e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.458+0000 7f7917a22740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516d08beb0 con 0x56516d1260a0 2024-03-21T11:31:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.458+0000 7f7917a22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516d034a60 con 0x56516d126a90 2024-03-21T11:31:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.458+0000 7f7917a22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516d087f50 con 0x56516d12bc00 2024-03-21T11:31:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.458+0000 7f790d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d12e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.458+0000 7f790d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d12e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48454/0 (socket says 172.21.15.47:48454) 2024-03-21T11:31:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.458+0000 7f790de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 0x56516d12b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.458+0000 7f790d400640 1 -- 172.21.15.47:0/4138176672 learned_addr learned my addr 172.21.15.47:0/4138176672 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.458+0000 7f790e800640 1 --2- 172.21.15.47:0/4138176672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d1260a0 0x56516d1264c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.459+0000 7f790d400640 1 -- 172.21.15.47:0/4138176672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 msgr2=0x56516d12b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.459+0000 7f790d400640 1 --2- 172.21.15.47:0/4138176672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 0x56516d12b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.459+0000 7f790d400640 1 -- 172.21.15.47:0/4138176672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d1260a0 msgr2=0x56516d1264c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.459+0000 7f790d400640 1 --2- 172.21.15.47:0/4138176672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d1260a0 0x56516d1264c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.459+0000 7f790d400640 1 -- 172.21.15.47:0/4138176672 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56516ce687e0 con 0x56516d12bc00 2024-03-21T11:31:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.459+0000 7f790d400640 1 --2- 172.21.15.47:0/4138176672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d12e090 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009cd0 tx=0x7f78fc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d956ced44115fa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.459+0000 7f7906a00640 1 --2- 172.21.15.47:0/3332747882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f78f40a2fa0 0x7f78f40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.459+0000 7f7906a00640 1 -- 172.21.15.47:0/3332747882 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f79000f0fd0 con 0x7f79101038e0 2024-03-21T11:31:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.459+0000 7f78fb400640 1 -- 172.21.15.47:0/4138176672 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc01e070 con 0x56516d12bc00 2024-03-21T11:31:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.459+0000 7f78fb400640 1 -- 172.21.15.47:0/4138176672 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78fc019b10 con 0x56516d12bc00 2024-03-21T11:31:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.459+0000 7f78fb400640 1 -- 172.21.15.47:0/4138176672 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00eb50 con 0x56516d12bc00 2024-03-21T11:31:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.459+0000 7f790e800640 1 --2- 172.21.15.47:0/3332747882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f78f40a2fa0 0x7f78f40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.460+0000 7f7917a22740 1 -- 172.21.15.47:0/4138176672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 msgr2=0x56516d12e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.460+0000 7f7917a22740 1 --2- 172.21.15.47:0/4138176672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d12e090 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009cd0 tx=0x7f78fc011810 comp rx=0 tx=0).stop 2024-03-21T11:31:14.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.460+0000 7f790e800640 1 --2- 172.21.15.47:0/3332747882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f78f40a2fa0 0x7f78f40a5460 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x56516d1272f0 tx=0x7f7908009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.460+0000 7f7917a22740 1 -- 172.21.15.47:0/4138176672 shutdown_connections 2024-03-21T11:31:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.460+0000 7f7917a22740 1 --2- 172.21.15.47:0/4138176672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d12e090 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.460+0000 7f7917a22740 1 --2- 172.21.15.47:0/4138176672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 0x56516d12b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.460+0000 7f7917a22740 1 --2- 172.21.15.47:0/4138176672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d1260a0 0x56516d1264c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.460+0000 7f7917a22740 1 -- 172.21.15.47:0/4138176672 >> 172.21.15.47:0/4138176672 conn(0x56516d121610 msgr2=0x56516d121a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.460+0000 7f7917a22740 1 -- 172.21.15.47:0/4138176672 shutdown_connections 2024-03-21T11:31:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.460+0000 7f7917a22740 1 -- 172.21.15.47:0/4138176672 wait complete. 2024-03-21T11:31:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.461+0000 7f7917a22740 1 -- start start 2024-03-21T11:31:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.461+0000 7f7917a22740 1 Processor -- start 2024-03-21T11:31:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.461+0000 7f7917a22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d1260a0 0x56516d1302e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.461+0000 7f7917a22740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d126a90 0x56516d130820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.461+0000 7f7917a22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d137860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f7917a22740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516d12ebb0 con 0x56516d126a90 2024-03-21T11:31:14.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f7917a22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516d047eb0 con 0x56516d1260a0 2024-03-21T11:31:14.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f7917a22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516d12ed30 con 0x56516d12bc00 2024-03-21T11:31:14.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f7917a22740 10 client.0 _ll_register_callbacks cb 0x56516d03c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:14.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f790de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d1260a0 0x56516d1302e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f790e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d137860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f790de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d1260a0 0x56516d1302e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60200/0 (socket says 172.21.15.47:60200) 2024-03-21T11:31:14.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f790e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d137860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48464/0 (socket says 172.21.15.47:48464) 2024-03-21T11:31:14.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f790de00640 1 -- 172.21.15.47:0/3176700565 learned_addr learned my addr 172.21.15.47:0/3176700565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f790d400640 1 --2- 172.21.15.47:0/3176700565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d126a90 0x56516d130820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f790e800640 1 -- 172.21.15.47:0/3176700565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d1260a0 msgr2=0x56516d1302e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f790e800640 1 --2- 172.21.15.47:0/3176700565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d1260a0 0x56516d1302e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f790e800640 1 -- 172.21.15.47:0/3176700565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d126a90 msgr2=0x56516d130820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f790e800640 1 --2- 172.21.15.47:0/3176700565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d126a90 0x56516d130820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f790e800640 1 -- 172.21.15.47:0/3176700565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56516cec8f70 con 0x56516d12bc00 2024-03-21T11:31:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f790de00640 1 --2- 172.21.15.47:0/3176700565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d1260a0 0x56516d1302e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f790d400640 1 --2- 172.21.15.47:0/3176700565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d126a90 0x56516d130820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f790e800640 1 --2- 172.21.15.47:0/3176700565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d137860 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7908005fd0 tx=0x7f7908004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2ba86491cc74d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.462+0000 7f78efe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.463+0000 7f78efe00640 1 -- 172.21.15.47:0/3176700565 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790800f870 con 0x56516d12bc00 2024-03-21T11:31:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.463+0000 7f78efe00640 1 -- 172.21.15.47:0/3176700565 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f790800fa10 con 0x56516d12bc00 2024-03-21T11:31:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.463+0000 7f7917a22740 1 -- 172.21.15.47:0/3176700565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78fc019b10 con 0x56516d12bc00 2024-03-21T11:31:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.463+0000 7f78efe00640 1 -- 172.21.15.47:0/3176700565 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f790800fc10 con 0x56516d12bc00 2024-03-21T11:31:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.463+0000 7f7917a22740 10 client.28201 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.463+0000 7f7917a22740 1 -- 172.21.15.47:0/3176700565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56516cd60170 con 0x56516d12bc00 2024-03-21T11:31:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.463+0000 7f78efe00640 1 -- 172.21.15.47:0/3176700565 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78fc019b10 con 0x56516d12bc00 2024-03-21T11:31:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.464+0000 7f78efe00640 1 -- 172.21.15.47:0/3176700565 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7908026070 con 0x56516d12bc00 2024-03-21T11:31:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.464+0000 7f7917a22740 10 client.28201 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.464+0000 7f7917a22740 10 client.28201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.464+0000 7f7917a22740 20 client.28201 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.464+0000 7f7917a22740 1 --2- 172.21.15.47:0/3176700565 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56516d13c220 0x56516d13e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.464+0000 7f7917a22740 4 client.28201 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.464+0000 7f7917a22740 1 -- 172.21.15.47:0/3176700565 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56516d0993f0 con 0x56516d13c220 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.464+0000 7f790de00640 1 --2- 172.21.15.47:0/3176700565 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56516d13c220 0x56516d13e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.465+0000 7f790de00640 1 --2- 172.21.15.47:0/3176700565 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56516d13c220 0x56516d13e660 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79155102689f76ab server_cookie=2622e14cdf5ff007 in_seq=0 out_seq=0 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.465+0000 7f78efe00640 10 client.28201 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f78efe00640 1 -- 172.21.15.47:0/3176700565 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56516d0993f0 con 0x56516d13c220 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f78efe00640 10 client.28201 handle_command_reply: tid=0 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f7917a22740 1 client.28201 shutdown 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f7917a22740 1 -- 172.21.15.47:0/3176700565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 msgr2=0x56516d137860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f7917a22740 1 --2- 172.21.15.47:0/3176700565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d137860 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7908005fd0 tx=0x7f7908004ab0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f7917a22740 1 -- 172.21.15.47:0/3176700565 shutdown_connections 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f7917a22740 1 --2- 172.21.15.47:0/3176700565 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56516d13c220 0x56516d13e660 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f7917a22740 1 --2- 172.21.15.47:0/3176700565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d137860 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f7917a22740 1 --2- 172.21.15.47:0/3176700565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d126a90 0x56516d130820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f7917a22740 1 --2- 172.21.15.47:0/3176700565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d1260a0 0x56516d1302e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f78efe00640 0 client.28201 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:14.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f7917a22740 1 -- 172.21.15.47:0/3176700565 >> 172.21.15.47:0/3176700565 conn(0x56516d121610 msgr2=0x56516d120830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:14.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f7917a22740 1 -- 172.21.15.47:0/3176700565 shutdown_connections 2024-03-21T11:31:14.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f7917a22740 1 -- 172.21.15.47:0/3176700565 wait complete. 2024-03-21T11:31:14.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.466+0000 7f7917a22740 20 client.28201 trim_cache size 0 max 16384 2024-03-21T11:31:14.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.485+0000 7f7917a22740 1 Processor -- start 2024-03-21T11:31:14.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.485+0000 7f7917a22740 1 -- start start 2024-03-21T11:31:14.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.485+0000 7f7917a22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d1260a0 0x56516d127490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.485+0000 7f7917a22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 0x56516d137820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.485+0000 7f7917a22740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d12bc00 0x56516d1279d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.485+0000 7f7917a22740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101ac340 con 0x56516d12bc00 2024-03-21T11:31:14.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.485+0000 7f7917a22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101adcc0 con 0x56516d126a90 2024-03-21T11:31:14.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.485+0000 7f7917a22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101ac9a0 con 0x56516d1260a0 2024-03-21T11:31:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.486+0000 7f790d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 0x56516d137820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.486+0000 7f790de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d1260a0 0x56516d127490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.486+0000 7f790d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 0x56516d137820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60204/0 (socket says 172.21.15.47:60204) 2024-03-21T11:31:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.486+0000 7f790de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d1260a0 0x56516d127490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48472/0 (socket says 172.21.15.47:48472) 2024-03-21T11:31:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.486+0000 7f790d400640 1 -- 172.21.15.47:0/763418813 learned_addr learned my addr 172.21.15.47:0/763418813 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.486+0000 7f790e800640 1 --2- 172.21.15.47:0/763418813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d12bc00 0x56516d1279d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.486+0000 7f790de00640 1 -- 172.21.15.47:0/763418813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 msgr2=0x56516d137820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.486+0000 7f790de00640 1 --2- 172.21.15.47:0/763418813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 0x56516d137820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.486+0000 7f790de00640 1 -- 172.21.15.47:0/763418813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d12bc00 msgr2=0x56516d1279d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.486+0000 7f790de00640 1 --2- 172.21.15.47:0/763418813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d12bc00 0x56516d1279d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.486+0000 7f790de00640 1 -- 172.21.15.47:0/763418813 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56516d11cc20 con 0x56516d1260a0 2024-03-21T11:31:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.486+0000 7f790de00640 1 --2- 172.21.15.47:0/763418813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d1260a0 0x56516d127490 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f790000b230 tx=0x7f79000f0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c46b3943a55f99e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:14.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.487+0000 7f78faa00640 1 -- 172.21.15.47:0/763418813 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79000bd0e0 con 0x56516d1260a0 2024-03-21T11:31:14.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.487+0000 7f78faa00640 1 -- 172.21.15.47:0/763418813 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79000bda10 con 0x56516d1260a0 2024-03-21T11:31:14.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.487+0000 7f78faa00640 1 -- 172.21.15.47:0/763418813 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79000bb450 con 0x56516d1260a0 2024-03-21T11:31:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.487+0000 7f7917a22740 1 -- 172.21.15.47:0/763418813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d1260a0 msgr2=0x56516d127490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.487+0000 7f7917a22740 1 --2- 172.21.15.47:0/763418813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d1260a0 0x56516d127490 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f790000b230 tx=0x7f79000f0040 comp rx=0 tx=0).stop 2024-03-21T11:31:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.487+0000 7f7917a22740 1 -- 172.21.15.47:0/763418813 shutdown_connections 2024-03-21T11:31:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.487+0000 7f7917a22740 1 --2- 172.21.15.47:0/763418813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d12bc00 0x56516d1279d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.487+0000 7f7917a22740 1 --2- 172.21.15.47:0/763418813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 0x56516d137820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.487+0000 7f7917a22740 1 --2- 172.21.15.47:0/763418813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d1260a0 0x56516d127490 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.487+0000 7f7917a22740 1 -- 172.21.15.47:0/763418813 >> 172.21.15.47:0/763418813 conn(0x56516d121610 msgr2=0x56516d125100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.488+0000 7f7917a22740 1 -- 172.21.15.47:0/763418813 shutdown_connections 2024-03-21T11:31:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.488+0000 7f7917a22740 1 -- 172.21.15.47:0/763418813 wait complete. 2024-03-21T11:31:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.488+0000 7f7917a22740 1 -- start start 2024-03-21T11:31:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.488+0000 7f7917a22740 1 Processor -- start 2024-03-21T11:31:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.489+0000 7f7917a22740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d1260a0 0x56516d1372d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.489+0000 7f7917a22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 0x56516d137810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.489+0000 7f7917a22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d1359d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.489+0000 7f7917a22740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101ad660 con 0x56516d1260a0 2024-03-21T11:31:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.489+0000 7f7917a22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101ad000 con 0x56516d126a90 2024-03-21T11:31:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.489+0000 7f7917a22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79101aa500 con 0x56516d12bc00 2024-03-21T11:31:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.489+0000 7f7917a22740 10 client.0 _ll_register_callbacks cb 0x7f7908009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.489+0000 7f790e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d1359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.489+0000 7f790d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 0x56516d137810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.489+0000 7f790e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d1359d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48484/0 (socket says 172.21.15.47:48484) 2024-03-21T11:31:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.489+0000 7f790de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d1260a0 0x56516d1372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.489+0000 7f790e800640 1 -- 172.21.15.47:0/3983488277 learned_addr learned my addr 172.21.15.47:0/3983488277 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.490+0000 7f790e800640 1 -- 172.21.15.47:0/3983488277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 msgr2=0x56516d137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.490+0000 7f790e800640 1 --2- 172.21.15.47:0/3983488277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 0x56516d137810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.490+0000 7f790e800640 1 -- 172.21.15.47:0/3983488277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d1260a0 msgr2=0x56516d1372d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.490+0000 7f790e800640 1 --2- 172.21.15.47:0/3983488277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d1260a0 0x56516d1372d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.490+0000 7f790e800640 1 -- 172.21.15.47:0/3983488277 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56516d136200 con 0x56516d12bc00 2024-03-21T11:31:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.490+0000 7f790e800640 1 --2- 172.21.15.47:0/3983488277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d1359d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f790801e600 tx=0x7f790800fdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=186d2201e7f33549 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.490+0000 7f78efe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.490+0000 7f78efe00640 1 -- 172.21.15.47:0/3983488277 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908004d20 con 0x56516d12bc00 2024-03-21T11:31:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.490+0000 7f7917a22740 1 -- 172.21.15.47:0/3983488277 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79000bda10 con 0x56516d12bc00 2024-03-21T11:31:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.490+0000 7f78efe00640 1 -- 172.21.15.47:0/3983488277 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f790801e770 con 0x56516d12bc00 2024-03-21T11:31:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.490+0000 7f78efe00640 1 -- 172.21.15.47:0/3983488277 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f790801e970 con 0x56516d12bc00 2024-03-21T11:31:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.490+0000 7f7917a22740 10 client.28207 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.490+0000 7f7917a22740 1 -- 172.21.15.47:0/3983488277 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56516d198750 con 0x56516d12bc00 2024-03-21T11:31:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.491+0000 7f78efe00640 1 -- 172.21.15.47:0/3983488277 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79000bda10 con 0x56516d12bc00 2024-03-21T11:31:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.491+0000 7f78efe00640 1 -- 172.21.15.47:0/3983488277 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7908026040 con 0x56516d12bc00 2024-03-21T11:31:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.491+0000 7f7917a22740 10 client.28207 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:14.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.491+0000 7f7917a22740 10 client.28207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:14.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.491+0000 7f7917a22740 20 client.28207 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:14.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.491+0000 7f7917a22740 1 --2- 172.21.15.47:0/3983488277 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56516d198fa0 0x56516d19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.491+0000 7f7917a22740 4 client.28207 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:31:14.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.491+0000 7f7917a22740 1 -- 172.21.15.47:0/3983488277 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56516d093740 con 0x56516d198fa0 2024-03-21T11:31:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.491+0000 7f790de00640 1 --2- 172.21.15.47:0/3983488277 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56516d198fa0 0x56516d19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.492+0000 7f790de00640 1 --2- 172.21.15.47:0/3983488277 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56516d198fa0 0x56516d19b3c0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa619621500c5d26 server_cookie=3e286f07a881f35f in_seq=0 out_seq=0 2024-03-21T11:31:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.492+0000 7f78efe00640 10 client.28207 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.493+0000 7f78efe00640 1 -- 172.21.15.47:0/3983488277 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56516d093740 con 0x56516d198fa0 2024-03-21T11:31:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.493+0000 7f78efe00640 10 client.28207 handle_command_reply: tid=0 2024-03-21T11:31:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.493+0000 7f7917a22740 1 client.28207 shutdown 2024-03-21T11:31:14.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.493+0000 7f7917a22740 1 -- 172.21.15.47:0/3983488277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 msgr2=0x56516d1359d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.493+0000 7f7917a22740 1 --2- 172.21.15.47:0/3983488277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d1359d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f790801e600 tx=0x7f790800fdb0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.493+0000 7f7917a22740 1 -- 172.21.15.47:0/3983488277 shutdown_connections 2024-03-21T11:31:14.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.493+0000 7f7917a22740 1 --2- 172.21.15.47:0/3983488277 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56516d198fa0 0x56516d19b3c0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.493+0000 7f7917a22740 1 --2- 172.21.15.47:0/3983488277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56516d12bc00 0x56516d1359d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.493+0000 7f7917a22740 1 --2- 172.21.15.47:0/3983488277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56516d126a90 0x56516d137810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.493+0000 7f7917a22740 1 --2- 172.21.15.47:0/3983488277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56516d1260a0 0x56516d1372d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.493+0000 7f78efe00640 0 client.28207 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:14.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.493+0000 7f7917a22740 1 -- 172.21.15.47:0/3983488277 >> 172.21.15.47:0/3983488277 conn(0x56516d121610 msgr2=0x56516d11ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:14.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.493+0000 7f7917a22740 1 -- 172.21.15.47:0/3983488277 shutdown_connections 2024-03-21T11:31:14.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.493+0000 7f7917a22740 1 -- 172.21.15.47:0/3983488277 wait complete. 2024-03-21T11:31:14.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.494+0000 7f7917a22740 20 client.28207 trim_cache size 0 max 16384 2024-03-21T11:31:14.498 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:31:14.498 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:31:14.498 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "8beb702c-c6b2-4c20-921d-ab63af5eee5b", 2024-03-21T11:31:14.498 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:31:14.498 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:31:14.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.494+0000 7f7914e00640 1 -- 172.21.15.47:0/3332747882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f78f40a2fa0 msgr2=0x7f78f40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.494+0000 7f7914e00640 1 --2- 172.21.15.47:0/3332747882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f78f40a2fa0 0x7f78f40a5460 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x56516d1272f0 tx=0x7f7908009500 comp rx=0 tx=0).stop 2024-03-21T11:31:14.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.494+0000 7f7914e00640 1 -- 172.21.15.47:0/3332747882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79101038e0 msgr2=0x7f7910199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.494+0000 7f7914e00640 1 --2- 172.21.15.47:0/3332747882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79101038e0 0x7f7910199c70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f79000349e0 tx=0x7f7900032430 comp rx=0 tx=0).stop 2024-03-21T11:31:14.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.494+0000 7f7914e00640 1 -- 172.21.15.47:0/3332747882 shutdown_connections 2024-03-21T11:31:14.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.494+0000 7f7914e00640 1 --2- 172.21.15.47:0/3332747882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f78f40a2fa0 0x7f78f40a5460 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.494+0000 7f7914e00640 1 --2- 172.21.15.47:0/3332747882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7910105490 0x7f79101a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.494+0000 7f7914e00640 1 --2- 172.21.15.47:0/3332747882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7910104ad0 0x7f791019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.494+0000 7f7914e00640 1 --2- 172.21.15.47:0/3332747882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79101038e0 0x7f7910199c70 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.494+0000 7f7914e00640 1 -- 172.21.15.47:0/3332747882 >> 172.21.15.47:0/3332747882 conn(0x7f79100fee50 msgr2=0x7f7910107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:14.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.494+0000 7f7914e00640 1 -- 172.21.15.47:0/3332747882 shutdown_connections 2024-03-21T11:31:14.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.494+0000 7f7914e00640 1 -- 172.21.15.47:0/3332747882 wait complete. 2024-03-21T11:31:14.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:31:14.513 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:31:14.533 INFO:tasks.workunit.client.0.smithi047.stdout: 1 556530 8.24 MB/sec execute 238 sec latency 15.464 ms 2024-03-21T11:31:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.742+0000 7ff8d7200640 1 -- 172.21.15.47:0/2309080448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8d8102810 msgr2=0x7ff8d8102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.742+0000 7ff8d7200640 1 --2- 172.21.15.47:0/2309080448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8d8102810 0x7ff8d8102c90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009900 tx=0x7ff8c402bc30 comp rx=0 tx=0).stop 2024-03-21T11:31:14.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.742+0000 7ff8d7200640 1 -- 172.21.15.47:0/2309080448 shutdown_connections 2024-03-21T11:31:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.742+0000 7ff8d7200640 1 --2- 172.21.15.47:0/2309080448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d81031d0 0x7ff8d8109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.742+0000 7ff8d7200640 1 --2- 172.21.15.47:0/2309080448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8d8102810 0x7ff8d8102c90 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.742+0000 7ff8d7200640 1 --2- 172.21.15.47:0/2309080448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d80ff840 0x7ff8d80ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.742+0000 7ff8d7200640 1 -- 172.21.15.47:0/2309080448 >> 172.21.15.47:0/2309080448 conn(0x7ff8d80fb190 msgr2=0x7ff8d80fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.742+0000 7ff8d7200640 1 -- 172.21.15.47:0/2309080448 shutdown_connections 2024-03-21T11:31:14.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.742+0000 7ff8d7200640 1 -- 172.21.15.47:0/2309080448 wait complete. 2024-03-21T11:31:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.743+0000 7ff8d7200640 1 Processor -- start 2024-03-21T11:31:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.743+0000 7ff8d7200640 1 -- start start 2024-03-21T11:31:14.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.743+0000 7ff8d7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d80ff840 0x7ff8d8197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.743+0000 7ff8d7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8d8102810 0x7ff8d8197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.743+0000 7ff8d7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d81031d0 0x7ff8d819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d810d230 con 0x7ff8d81031d0 2024-03-21T11:31:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d810d0b0 con 0x7ff8d8102810 2024-03-21T11:31:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d810d3b0 con 0x7ff8d80ff840 2024-03-21T11:31:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d80ff840 0x7ff8d8197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d80ff840 0x7ff8d8197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48506/0 (socket says 172.21.15.47:48506) 2024-03-21T11:31:14.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d5e00640 1 -- 172.21.15.47:0/2854517828 learned_addr learned my addr 172.21.15.47:0/2854517828 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d5400640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8d8102810 0x7ff8d8197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d6800640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d81031d0 0x7ff8d819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d5e00640 1 -- 172.21.15.47:0/2854517828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8d8102810 msgr2=0x7ff8d8197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d5e00640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8d8102810 0x7ff8d8197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d5e00640 1 -- 172.21.15.47:0/2854517828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d81031d0 msgr2=0x7ff8d819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d5e00640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d81031d0 0x7ff8d819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d5e00640 1 -- 172.21.15.47:0/2854517828 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d819f4b0 con 0x7ff8d80ff840 2024-03-21T11:31:14.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d6800640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d81031d0 0x7ff8d819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:31:14.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.744+0000 7ff8d5e00640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d80ff840 0x7ff8d8197890 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800d950 tx=0x7ff8c800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.745+0000 7ff8cea00640 1 -- 172.21.15.47:0/2854517828 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8017020 con 0x7ff8d80ff840 2024-03-21T11:31:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.745+0000 7ff8d7200640 1 -- 172.21.15.47:0/2854517828 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d819f7a0 con 0x7ff8d80ff840 2024-03-21T11:31:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.745+0000 7ff8cea00640 1 -- 172.21.15.47:0/2854517828 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8c8009c30 con 0x7ff8d80ff840 2024-03-21T11:31:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.745+0000 7ff8cea00640 1 -- 172.21.15.47:0/2854517828 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c800f6b0 con 0x7ff8d80ff840 2024-03-21T11:31:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.745+0000 7ff8d7200640 1 -- 172.21.15.47:0/2854517828 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d819fd10 con 0x7ff8d80ff840 2024-03-21T11:31:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.748+0000 7ff8dffa6740 1 Processor -- start 2024-03-21T11:31:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.748+0000 7ff8dffa6740 1 -- start start 2024-03-21T11:31:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.748+0000 7ff8cea00640 1 -- 172.21.15.47:0/2854517828 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff8c801a020 con 0x7ff8d80ff840 2024-03-21T11:31:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8dffa6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895460 0x55ff25895880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8dffa6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895e50 0x55ff2589a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8dffa6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff2589afc0 0x55ff2589d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8dffa6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff257f6250 con 0x55ff2589afc0 2024-03-21T11:31:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8dffa6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2579ea30 con 0x55ff25895460 2024-03-21T11:31:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8dffa6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff257f22f0 con 0x55ff25895e50 2024-03-21T11:31:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8d6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895460 0x55ff25895880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8d6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895460 0x55ff25895880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60256/0 (socket says 172.21.15.47:60256) 2024-03-21T11:31:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8d5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895e50 0x55ff2589a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8d6800640 1 -- 172.21.15.47:0/3291304075 learned_addr learned my addr 172.21.15.47:0/3291304075 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8d5400640 1 --2- 172.21.15.47:0/3291304075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff2589afc0 0x55ff2589d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8d6800640 1 -- 172.21.15.47:0/3291304075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895e50 msgr2=0x55ff2589a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8d6800640 1 --2- 172.21.15.47:0/3291304075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895e50 0x55ff2589a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8d6800640 1 -- 172.21.15.47:0/3291304075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff2589afc0 msgr2=0x55ff2589d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8d6800640 1 --2- 172.21.15.47:0/3291304075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff2589afc0 0x55ff2589d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8d6800640 1 -- 172.21.15.47:0/3291304075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff25789690 con 0x55ff25895460 2024-03-21T11:31:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.749+0000 7ff8d5400640 1 --2- 172.21.15.47:0/3291304075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff2589afc0 0x55ff2589d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.750+0000 7ff8d6800640 1 --2- 172.21.15.47:0/3291304075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895460 0x55ff25895880 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009f10 tx=0x7ff8d000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51934e30f5990770 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.750+0000 7ff8c3400640 1 -- 172.21.15.47:0/3291304075 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0003b70 con 0x55ff25895460 2024-03-21T11:31:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.750+0000 7ff8c3400640 1 -- 172.21.15.47:0/3291304075 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8d0003d10 con 0x55ff25895460 2024-03-21T11:31:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.750+0000 7ff8c3400640 1 -- 172.21.15.47:0/3291304075 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0011a70 con 0x55ff25895460 2024-03-21T11:31:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.750+0000 7ff8cea00640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8bc0a2f20 0x7ff8bc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.751+0000 7ff8cea00640 1 -- 172.21.15.47:0/2854517828 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff8c80cf7d0 con 0x7ff8d80ff840 2024-03-21T11:31:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.751+0000 7ff8d5e00640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8bc0a2f20 0x7ff8bc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.751+0000 7ff8dffa6740 1 -- 172.21.15.47:0/3291304075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895460 msgr2=0x55ff25895880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.751+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/3291304075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895460 0x55ff25895880 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009f10 tx=0x7ff8d000f730 comp rx=0 tx=0).stop 2024-03-21T11:31:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.751+0000 7ff8dffa6740 1 -- 172.21.15.47:0/3291304075 shutdown_connections 2024-03-21T11:31:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.751+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/3291304075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff2589afc0 0x55ff2589d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.751+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/3291304075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895e50 0x55ff2589a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.751+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/3291304075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895460 0x55ff25895880 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.751+0000 7ff8dffa6740 1 -- 172.21.15.47:0/3291304075 >> 172.21.15.47:0/3291304075 conn(0x55ff25859200 msgr2=0x55ff2588c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.751+0000 7ff8dffa6740 1 -- 172.21.15.47:0/3291304075 shutdown_connections 2024-03-21T11:31:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.751+0000 7ff8dffa6740 1 -- 172.21.15.47:0/3291304075 wait complete. 2024-03-21T11:31:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.752+0000 7ff8d5e00640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8bc0a2f20 0x7ff8bc0a53e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80984e0 tx=0x7ff8c80d5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.752+0000 7ff8dffa6740 1 -- start start 2024-03-21T11:31:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.752+0000 7ff8dffa6740 1 Processor -- start 2024-03-21T11:31:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.752+0000 7ff8dffa6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff25895460 0x55ff2589f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.752+0000 7ff8dffa6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895e50 0x55ff2589fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.752+0000 7ff8dffa6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff2589afc0 0x55ff258a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.752+0000 7ff8dffa6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2589df70 con 0x55ff25895460 2024-03-21T11:31:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.752+0000 7ff8dffa6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff257b1e80 con 0x55ff25895e50 2024-03-21T11:31:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.752+0000 7ff8dffa6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2589e0f0 con 0x55ff2589afc0 2024-03-21T11:31:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.753+0000 7ff8dffa6740 10 client.0 _ll_register_callbacks cb 0x55ff257c4130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.753+0000 7ff8d5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895e50 0x55ff2589fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.753+0000 7ff8d5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895e50 0x55ff2589fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60272/0 (socket says 172.21.15.47:60272) 2024-03-21T11:31:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.753+0000 7ff8d5400640 1 -- 172.21.15.47:0/4005853285 learned_addr learned my addr 172.21.15.47:0/4005853285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.753+0000 7ff8d6800640 1 --2- 172.21.15.47:0/4005853285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff25895460 0x55ff2589f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.753+0000 7ff8d6800640 1 --2- 172.21.15.47:0/4005853285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff2589afc0 0x55ff258a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.753+0000 7ff8d5400640 1 -- 172.21.15.47:0/4005853285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff2589afc0 msgr2=0x55ff258a6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.753+0000 7ff8d5400640 1 --2- 172.21.15.47:0/4005853285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff2589afc0 0x55ff258a6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.753+0000 7ff8d5400640 1 -- 172.21.15.47:0/4005853285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff25895460 msgr2=0x55ff2589f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.753+0000 7ff8d5400640 1 --2- 172.21.15.47:0/4005853285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff25895460 0x55ff2589f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.753+0000 7ff8d5400640 1 -- 172.21.15.47:0/4005853285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff255d47d0 con 0x55ff25895e50 2024-03-21T11:31:14.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.753+0000 7ff8d6800640 1 --2- 172.21.15.47:0/4005853285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff2589afc0 0x55ff258a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:14.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.754+0000 7ff8d5400640 1 --2- 172.21.15.47:0/4005853285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895e50 0x55ff2589fa00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff8c402c7e0 tx=0x7ff8c4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd7faff71df9a28 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:14.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.754+0000 7ff8b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:14.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.754+0000 7ff8b7e00640 1 -- 172.21.15.47:0/4005853285 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4005d50 con 0x55ff25895e50 2024-03-21T11:31:14.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.754+0000 7ff8dffa6740 1 -- 172.21.15.47:0/4005853285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8d0003d10 con 0x55ff25895e50 2024-03-21T11:31:14.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.754+0000 7ff8b7e00640 1 -- 172.21.15.47:0/4005853285 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff8c40046c0 con 0x55ff25895e50 2024-03-21T11:31:14.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.754+0000 7ff8b7e00640 1 -- 172.21.15.47:0/4005853285 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8c4053050 con 0x55ff25895e50 2024-03-21T11:31:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.754+0000 7ff8dffa6740 10 client.28211 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.754+0000 7ff8dffa6740 1 -- 172.21.15.47:0/4005853285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ff2550a3f0 con 0x55ff25895e50 2024-03-21T11:31:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.754+0000 7ff8b7e00640 1 -- 172.21.15.47:0/4005853285 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8d0003d10 con 0x55ff25895e50 2024-03-21T11:31:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.755+0000 7ff8b7e00640 1 -- 172.21.15.47:0/4005853285 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff8c40037b0 con 0x55ff25895e50 2024-03-21T11:31:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.755+0000 7ff8dffa6740 10 client.28211 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.755+0000 7ff8dffa6740 10 client.28211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.755+0000 7ff8dffa6740 20 client.28211 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.755+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/4005853285 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff258ab1f0 0x55ff258ad630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.755+0000 7ff8dffa6740 4 client.28211 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.755+0000 7ff8dffa6740 1 -- 172.21.15.47:0/4005853285 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff25803790 con 0x55ff258ab1f0 2024-03-21T11:31:14.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.755+0000 7ff8d6800640 1 --2- 172.21.15.47:0/4005853285 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff258ab1f0 0x55ff258ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.756+0000 7ff8d6800640 1 --2- 172.21.15.47:0/4005853285 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff258ab1f0 0x55ff258ad630 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6bb1d41f212f9e3 server_cookie=4946afa45a51032a in_seq=0 out_seq=0 2024-03-21T11:31:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.756+0000 7ff8b7e00640 10 client.28211 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8b7e00640 1 -- 172.21.15.47:0/4005853285 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ff25803790 con 0x55ff258ab1f0 2024-03-21T11:31:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8b7e00640 10 client.28211 handle_command_reply: tid=0 2024-03-21T11:31:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8dffa6740 1 client.28211 shutdown 2024-03-21T11:31:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8dffa6740 1 -- 172.21.15.47:0/4005853285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895e50 msgr2=0x55ff2589fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/4005853285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895e50 0x55ff2589fa00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff8c402c7e0 tx=0x7ff8c4004270 comp rx=0 tx=0).stop 2024-03-21T11:31:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8dffa6740 1 -- 172.21.15.47:0/4005853285 shutdown_connections 2024-03-21T11:31:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/4005853285 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff258ab1f0 0x55ff258ad630 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/4005853285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff2589afc0 0x55ff258a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/4005853285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895e50 0x55ff2589fa00 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/4005853285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff25895460 0x55ff2589f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8dffa6740 1 -- 172.21.15.47:0/4005853285 >> 172.21.15.47:0/4005853285 conn(0x55ff25859200 msgr2=0x55ff2588bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8b7e00640 0 client.28211 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8dffa6740 1 -- 172.21.15.47:0/4005853285 shutdown_connections 2024-03-21T11:31:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8dffa6740 1 -- 172.21.15.47:0/4005853285 wait complete. 2024-03-21T11:31:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.757+0000 7ff8dffa6740 20 client.28211 trim_cache size 0 max 16384 2024-03-21T11:31:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.776+0000 7ff8dffa6740 1 Processor -- start 2024-03-21T11:31:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.776+0000 7ff8dffa6740 1 -- start start 2024-03-21T11:31:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.776+0000 7ff8dffa6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895460 0x55ff2588b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.776+0000 7ff8dffa6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895e50 0x55ff2588b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.776+0000 7ff8dffa6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff2589afc0 0x55ff2588bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.776+0000 7ff8dffa6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d81a7ce0 con 0x55ff2589afc0 2024-03-21T11:31:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.776+0000 7ff8dffa6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d81a9660 con 0x55ff25895e50 2024-03-21T11:31:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.776+0000 7ff8dffa6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d81a8340 con 0x55ff25895460 2024-03-21T11:31:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.777+0000 7ff8d6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895460 0x55ff2588b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.777+0000 7ff8d6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895460 0x55ff2588b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48536/0 (socket says 172.21.15.47:48536) 2024-03-21T11:31:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.777+0000 7ff8d6800640 1 -- 172.21.15.47:0/3317487138 learned_addr learned my addr 172.21.15.47:0/3317487138 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.777+0000 7ff8d5400640 1 --2- 172.21.15.47:0/3317487138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895e50 0x55ff2588b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.777+0000 7ff8d6800640 1 --2- 172.21.15.47:0/3317487138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff2589afc0 0x55ff2588bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.777+0000 7ff8d6800640 1 -- 172.21.15.47:0/3317487138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895e50 msgr2=0x55ff2588b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.777+0000 7ff8d6800640 1 --2- 172.21.15.47:0/3317487138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895e50 0x55ff2588b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.777+0000 7ff8d6800640 1 -- 172.21.15.47:0/3317487138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff2589afc0 msgr2=0x55ff2588bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.777+0000 7ff8d6800640 1 --2- 172.21.15.47:0/3317487138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff2589afc0 0x55ff2588bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.777+0000 7ff8d6800640 1 -- 172.21.15.47:0/3317487138 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff254ca0f0 con 0x55ff25895460 2024-03-21T11:31:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.777+0000 7ff8d5400640 1 --2- 172.21.15.47:0/3317487138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895e50 0x55ff2588b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.778+0000 7ff8d6800640 1 --2- 172.21.15.47:0/3317487138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895460 0x55ff2588b330 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009d20 tx=0x7ff8d0000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d500c67cf7d1d5f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.778+0000 7ff8c2a00640 1 -- 172.21.15.47:0/3317487138 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0031070 con 0x55ff25895460 2024-03-21T11:31:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.778+0000 7ff8c2a00640 1 -- 172.21.15.47:0/3317487138 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8d0026040 con 0x55ff25895460 2024-03-21T11:31:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.778+0000 7ff8c2a00640 1 -- 172.21.15.47:0/3317487138 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0025070 con 0x55ff25895460 2024-03-21T11:31:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.779+0000 7ff8dffa6740 1 -- 172.21.15.47:0/3317487138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895460 msgr2=0x55ff2588b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.779+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/3317487138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895460 0x55ff2588b330 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009d20 tx=0x7ff8d0000f40 comp rx=0 tx=0).stop 2024-03-21T11:31:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.779+0000 7ff8dffa6740 1 -- 172.21.15.47:0/3317487138 shutdown_connections 2024-03-21T11:31:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.779+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/3317487138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff2589afc0 0x55ff2588bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.779+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/3317487138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff25895e50 0x55ff2588b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.779+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/3317487138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895460 0x55ff2588b330 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.779+0000 7ff8dffa6740 1 -- 172.21.15.47:0/3317487138 >> 172.21.15.47:0/3317487138 conn(0x55ff25859200 msgr2=0x55ff25897180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.780+0000 7ff8dffa6740 1 -- 172.21.15.47:0/3317487138 shutdown_connections 2024-03-21T11:31:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.780+0000 7ff8dffa6740 1 -- 172.21.15.47:0/3317487138 wait complete. 2024-03-21T11:31:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.780+0000 7ff8dffa6740 1 -- start start 2024-03-21T11:31:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.780+0000 7ff8dffa6740 1 Processor -- start 2024-03-21T11:31:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8dffa6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895460 0x55ff258a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8dffa6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff25895e50 0x55ff258a5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8dffa6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff2589afc0 0x55ff258a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8dffa6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d81a9000 con 0x55ff25895e50 2024-03-21T11:31:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8dffa6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d81a89a0 con 0x55ff2589afc0 2024-03-21T11:31:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8dffa6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d81a5ea0 con 0x55ff25895460 2024-03-21T11:31:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8dffa6740 10 client.0 _ll_register_callbacks cb 0x7ff8c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8d6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895460 0x55ff258a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8d6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895460 0x55ff258a2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48552/0 (socket says 172.21.15.47:48552) 2024-03-21T11:31:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8d6800640 1 -- 172.21.15.47:0/138216524 learned_addr learned my addr 172.21.15.47:0/138216524 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8d6800640 1 --2- 172.21.15.47:0/138216524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff2589afc0 0x55ff258a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8d5400640 1 --2- 172.21.15.47:0/138216524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff25895e50 0x55ff258a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8d6800640 1 -- 172.21.15.47:0/138216524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff2589afc0 msgr2=0x55ff258a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8d6800640 1 --2- 172.21.15.47:0/138216524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff2589afc0 0x55ff258a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8d6800640 1 -- 172.21.15.47:0/138216524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff25895e50 msgr2=0x55ff258a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8d6800640 1 --2- 172.21.15.47:0/138216524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff25895e50 0x55ff258a5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.781+0000 7ff8d6800640 1 -- 172.21.15.47:0/138216524 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff258a5ef0 con 0x55ff25895460 2024-03-21T11:31:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.782+0000 7ff8d6800640 1 --2- 172.21.15.47:0/138216524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895460 0x55ff258a2c30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00116b0 tx=0x7ff8d0009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19770210c275a7b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.782+0000 7ff8b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.782+0000 7ff8b7e00640 1 -- 172.21.15.47:0/138216524 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0031040 con 0x55ff25895460 2024-03-21T11:31:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.782+0000 7ff8b7e00640 1 -- 172.21.15.47:0/138216524 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff8d0003970 con 0x55ff25895460 2024-03-21T11:31:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.782+0000 7ff8dffa6740 1 -- 172.21.15.47:0/138216524 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8d0026040 con 0x55ff25895460 2024-03-21T11:31:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.782+0000 7ff8b7e00640 1 -- 172.21.15.47:0/138216524 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8d0043050 con 0x55ff25895460 2024-03-21T11:31:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.782+0000 7ff8dffa6740 10 client.28225 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.782+0000 7ff8dffa6740 1 -- 172.21.15.47:0/138216524 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ff2590b480 con 0x55ff25895460 2024-03-21T11:31:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.782+0000 7ff8b7e00640 1 -- 172.21.15.47:0/138216524 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8d0026040 con 0x55ff25895460 2024-03-21T11:31:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.783+0000 7ff8b7e00640 1 -- 172.21.15.47:0/138216524 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff8d0034030 con 0x55ff25895460 2024-03-21T11:31:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.783+0000 7ff8dffa6740 10 client.28225 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.783+0000 7ff8dffa6740 10 client.28225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.783+0000 7ff8dffa6740 20 client.28225 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.783+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/138216524 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff2590baf0 0x55ff2590df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.783+0000 7ff8dffa6740 4 client.28225 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:31:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.783+0000 7ff8dffa6740 1 -- 172.21.15.47:0/138216524 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ff257fdae0 con 0x55ff2590baf0 2024-03-21T11:31:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.783+0000 7ff8d5400640 1 --2- 172.21.15.47:0/138216524 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff2590baf0 0x55ff2590df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.784+0000 7ff8d5400640 1 --2- 172.21.15.47:0/138216524 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff2590baf0 0x55ff2590df10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcaabe002898a07c server_cookie=1b44dfb820647275 in_seq=0 out_seq=0 2024-03-21T11:31:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.784+0000 7ff8b7e00640 10 client.28225 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8b7e00640 1 -- 172.21.15.47:0/138216524 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ff257fdae0 con 0x55ff2590baf0 2024-03-21T11:31:14.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8b7e00640 10 client.28225 handle_command_reply: tid=0 2024-03-21T11:31:14.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8dffa6740 1 client.28225 shutdown 2024-03-21T11:31:14.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8dffa6740 1 -- 172.21.15.47:0/138216524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895460 msgr2=0x55ff258a2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/138216524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895460 0x55ff258a2c30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00116b0 tx=0x7ff8d0009500 comp rx=0 tx=0).stop 2024-03-21T11:31:14.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8dffa6740 1 -- 172.21.15.47:0/138216524 shutdown_connections 2024-03-21T11:31:14.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/138216524 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff2590baf0 0x55ff2590df10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/138216524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff2589afc0 0x55ff258a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/138216524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff25895e50 0x55ff258a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8dffa6740 1 --2- 172.21.15.47:0/138216524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff25895460 0x55ff258a2c30 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8dffa6740 1 -- 172.21.15.47:0/138216524 >> 172.21.15.47:0/138216524 conn(0x55ff25859200 msgr2=0x55ff2588fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:14.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8b7e00640 0 client.28225 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:14.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8dffa6740 1 -- 172.21.15.47:0/138216524 shutdown_connections 2024-03-21T11:31:14.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8dffa6740 1 -- 172.21.15.47:0/138216524 wait complete. 2024-03-21T11:31:14.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.785+0000 7ff8dffa6740 20 client.28225 trim_cache size 0 max 16384 2024-03-21T11:31:14.811 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:31:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:31:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:31:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "8beb702c-c6b2-4c20-921d-ab63af5eee5b": { 2024-03-21T11:31:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:31:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "8beb702c-c6b2-4c20-921d-ab63af5eee5b", 2024-03-21T11:31:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:31:14.812 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:31:14.812 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:31:14.812 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:31:14.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.786+0000 7ff8d7200640 1 -- 172.21.15.47:0/2854517828 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8bc0a2f20 msgr2=0x7ff8bc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.786+0000 7ff8d7200640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8bc0a2f20 0x7ff8bc0a53e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80984e0 tx=0x7ff8c80d5530 comp rx=0 tx=0).stop 2024-03-21T11:31:14.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.786+0000 7ff8d7200640 1 -- 172.21.15.47:0/2854517828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d80ff840 msgr2=0x7ff8d8197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:14.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.786+0000 7ff8d7200640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d80ff840 0x7ff8d8197890 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800d950 tx=0x7ff8c800de10 comp rx=0 tx=0).stop 2024-03-21T11:31:14.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.786+0000 7ff8d7200640 1 -- 172.21.15.47:0/2854517828 shutdown_connections 2024-03-21T11:31:14.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.786+0000 7ff8d7200640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8bc0a2f20 0x7ff8bc0a53e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.786+0000 7ff8d7200640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d81031d0 0x7ff8d819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.786+0000 7ff8d7200640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8d8102810 0x7ff8d8197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.786+0000 7ff8d7200640 1 --2- 172.21.15.47:0/2854517828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d80ff840 0x7ff8d8197890 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:14.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.786+0000 7ff8d7200640 1 -- 172.21.15.47:0/2854517828 >> 172.21.15.47:0/2854517828 conn(0x7ff8d80fb190 msgr2=0x7ff8d8105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:14.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.786+0000 7ff8d7200640 1 -- 172.21.15.47:0/2854517828 shutdown_connections 2024-03-21T11:31:14.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:14.786+0000 7ff8d7200640 1 -- 172.21.15.47:0/2854517828 wait complete. 2024-03-21T11:31:14.816 INFO:tasks.cephfs.filesystem:scrub status for tag:8beb702c-c6b2-4c20-921d-ab63af5eee5b - {'path': '/', 'tag': '8beb702c-c6b2-4c20-921d-ab63af5eee5b', 'options': 'recursive,force'} 2024-03-21T11:31:15.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:15 smithi047 ceph-mon[31351]: pgmap v637: 129 pgs: 129 active+clean; 38 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 359 op/s 2024-03-21T11:31:15.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:15 smithi100 ceph-mon[36245]: pgmap v637: 129 pgs: 129 active+clean; 38 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 359 op/s 2024-03-21T11:31:15.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:15 smithi102 ceph-mon[36748]: pgmap v637: 129 pgs: 129 active+clean; 38 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 359 op/s 2024-03-21T11:31:15.534 INFO:tasks.workunit.client.0.smithi047.stdout: 1 558093 8.26 MB/sec execute 239 sec latency 12.781 ms 2024-03-21T11:31:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:16 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:31:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:16 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:31:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:16 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:31:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:16 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:31:16.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:16 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:31:16.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:16 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:31:16.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:16 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:31:16.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:16 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:31:16.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:16 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:31:16.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:16 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:31:16.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:16 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:31:16.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:16 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:31:16.534 INFO:tasks.workunit.client.0.smithi047.stdout: 1 559105 8.24 MB/sec execute 240 sec latency 68.947 ms 2024-03-21T11:31:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:17 smithi047 ceph-mon[31351]: pgmap v638: 129 pgs: 129 active+clean; 34 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 281 op/s 2024-03-21T11:31:17.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:17 smithi100 ceph-mon[36245]: pgmap v638: 129 pgs: 129 active+clean; 34 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 281 op/s 2024-03-21T11:31:17.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:17 smithi102 ceph-mon[36748]: pgmap v638: 129 pgs: 129 active+clean; 34 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 281 op/s 2024-03-21T11:31:17.534 INFO:tasks.workunit.client.0.smithi047.stdout: 1 561072 8.25 MB/sec execute 241 sec latency 16.198 ms 2024-03-21T11:31:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:18 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:31:18.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:18 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:31:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:18 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:31:18.534 INFO:tasks.workunit.client.0.smithi047.stdout: 1 562054 8.23 MB/sec execute 242 sec latency 14.354 ms 2024-03-21T11:31:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:19 smithi047 ceph-mon[31351]: pgmap v639: 129 pgs: 129 active+clean; 34 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2024-03-21T11:31:19.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:19 smithi100 ceph-mon[36245]: pgmap v639: 129 pgs: 129 active+clean; 34 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2024-03-21T11:31:19.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:19 smithi102 ceph-mon[36748]: pgmap v639: 129 pgs: 129 active+clean; 34 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2024-03-21T11:31:19.534 INFO:tasks.workunit.client.0.smithi047.stdout: 1 563972 8.24 MB/sec execute 243 sec latency 12.101 ms 2024-03-21T11:31:20.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:20 smithi100 ceph-mon[36245]: pgmap v640: 129 pgs: 129 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 310 op/s 2024-03-21T11:31:20.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:20 smithi102 ceph-mon[36748]: pgmap v640: 129 pgs: 129 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 310 op/s 2024-03-21T11:31:20.534 INFO:tasks.workunit.client.0.smithi047.stdout: 1 565267 8.24 MB/sec execute 244 sec latency 21.376 ms 2024-03-21T11:31:20.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:20 smithi047 ceph-mon[31351]: pgmap v640: 129 pgs: 129 active+clean; 35 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 310 op/s 2024-03-21T11:31:21.534 INFO:tasks.workunit.client.0.smithi047.stdout: 1 566498 8.23 MB/sec execute 245 sec latency 79.425 ms 2024-03-21T11:31:22.534 INFO:tasks.workunit.client.0.smithi047.stdout: 1 568460 8.25 MB/sec execute 246 sec latency 28.316 ms 2024-03-21T11:31:22.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:22 smithi100 ceph-mon[36245]: pgmap v641: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 306 op/s 2024-03-21T11:31:22.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:22 smithi102 ceph-mon[36748]: pgmap v641: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 306 op/s 2024-03-21T11:31:22.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:22 smithi047 ceph-mon[31351]: pgmap v641: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 306 op/s 2024-03-21T11:31:23.535 INFO:tasks.workunit.client.0.smithi047.stdout: 1 569323 8.22 MB/sec execute 247 sec latency 14.549 ms 2024-03-21T11:31:24.535 INFO:tasks.workunit.client.0.smithi047.stdout: 1 571250 8.24 MB/sec execute 248 sec latency 24.651 ms 2024-03-21T11:31:24.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:24 smithi100 ceph-mon[36245]: pgmap v642: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 325 op/s 2024-03-21T11:31:24.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:24 smithi102 ceph-mon[36748]: pgmap v642: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 325 op/s 2024-03-21T11:31:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.767+0000 7fef9e600640 1 -- 172.21.15.47:0/1264908278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef98102170 msgr2=0x7fef98108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.767+0000 7fef9e600640 1 --2- 172.21.15.47:0/1264908278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef98102170 0x7fef98108a00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fef90009be0 tx=0x7fef9002bef0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.767+0000 7fef9e600640 1 -- 172.21.15.47:0/1264908278 shutdown_connections 2024-03-21T11:31:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.767+0000 7fef9e600640 1 --2- 172.21.15.47:0/1264908278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef98102170 0x7fef98108a00 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.767+0000 7fef9e600640 1 --2- 172.21.15.47:0/1264908278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef981017b0 0x7fef98101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.767+0000 7fef9e600640 1 --2- 172.21.15.47:0/1264908278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef981005c0 0x7fef981009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.767+0000 7fef9e600640 1 -- 172.21.15.47:0/1264908278 >> 172.21.15.47:0/1264908278 conn(0x7fef980fbb30 msgr2=0x7fef980fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.768+0000 7fef9e600640 1 -- 172.21.15.47:0/1264908278 shutdown_connections 2024-03-21T11:31:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.768+0000 7fef9e600640 1 -- 172.21.15.47:0/1264908278 wait complete. 2024-03-21T11:31:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.769+0000 7fef9e600640 1 Processor -- start 2024-03-21T11:31:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.769+0000 7fef9e600640 1 -- start start 2024-03-21T11:31:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.769+0000 7fef9e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef981005c0 0x7fef9810ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.769+0000 7fef9e600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef981017b0 0x7fef98109280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.769+0000 7fef9e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef98102170 0x7fef981097c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.769+0000 7fef9e600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9810d360 con 0x7fef981017b0 2024-03-21T11:31:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.769+0000 7fef9e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9810d1e0 con 0x7fef981005c0 2024-03-21T11:31:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.769+0000 7fef9e600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9810d4e0 con 0x7fef98102170 2024-03-21T11:31:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.769+0000 7fef97400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef981005c0 0x7fef9810ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.769+0000 7fef97400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef981005c0 0x7fef9810ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40204/0 (socket says 172.21.15.47:40204) 2024-03-21T11:31:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.769+0000 7fef97400640 1 -- 172.21.15.47:0/884167827 learned_addr learned my addr 172.21.15.47:0/884167827 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.769+0000 7fef97e00640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef98102170 0x7fef981097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.769+0000 7fef96a00640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef981017b0 0x7fef98109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.770+0000 7fef97400640 1 -- 172.21.15.47:0/884167827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef98102170 msgr2=0x7fef981097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.770+0000 7fef97400640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef98102170 0x7fef981097c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.770+0000 7fef97400640 1 -- 172.21.15.47:0/884167827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef981017b0 msgr2=0x7fef98109280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.770+0000 7fef97400640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef981017b0 0x7fef98109280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.770+0000 7fef97400640 1 -- 172.21.15.47:0/884167827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9810a050 con 0x7fef981005c0 2024-03-21T11:31:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.770+0000 7fef96a00640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef981017b0 0x7fef98109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.770+0000 7fef97400640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef981005c0 0x7fef9810ac40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fef8800e9c0 tx=0x7fef8800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.770+0000 7fef8fe00640 1 -- 172.21.15.47:0/884167827 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800cd80 con 0x7fef981005c0 2024-03-21T11:31:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.770+0000 7fef8fe00640 1 -- 172.21.15.47:0/884167827 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef880035f0 con 0x7fef981005c0 2024-03-21T11:31:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.770+0000 7fef8fe00640 1 -- 172.21.15.47:0/884167827 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88010690 con 0x7fef981005c0 2024-03-21T11:31:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.770+0000 7fef9e600640 1 -- 172.21.15.47:0/884167827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef981a1a70 con 0x7fef981005c0 2024-03-21T11:31:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.771+0000 7fef9e600640 1 -- 172.21.15.47:0/884167827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef981a1f70 con 0x7fef981005c0 2024-03-21T11:31:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.774+0000 7fef8fe00640 1 -- 172.21.15.47:0/884167827 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fef880037a0 con 0x7fef981005c0 2024-03-21T11:31:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.774+0000 7fefa1221740 1 Processor -- start 2024-03-21T11:31:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.775+0000 7fefa1221740 1 -- start start 2024-03-21T11:31:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.775+0000 7fefa1221740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e31e0 0x55c5429e3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.775+0000 7fefa1221740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e3bd0 0x55c5429e86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.775+0000 7fefa1221740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e8d40 0x55c5429eb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.775+0000 7fefa1221740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5429a63b0 con 0x55c5429e31e0 2024-03-21T11:31:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.775+0000 7fefa1221740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c542948830 con 0x55c5429e8d40 2024-03-21T11:31:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.775+0000 7fefa1221740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5428f15d0 con 0x55c5429e3bd0 2024-03-21T11:31:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.775+0000 7fef97400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e3bd0 0x55c5429e86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.775+0000 7fef96a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e8d40 0x55c5429eb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.775+0000 7fef97e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e31e0 0x55c5429e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.775+0000 7fef96a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e8d40 0x55c5429eb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40206/0 (socket says 172.21.15.47:40206) 2024-03-21T11:31:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.775+0000 7fef97e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e31e0 0x55c5429e3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44184/0 (socket says 172.21.15.47:44184) 2024-03-21T11:31:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.775+0000 7fef96a00640 1 -- 172.21.15.47:0/3432168399 learned_addr learned my addr 172.21.15.47:0/3432168399 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:24.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.776+0000 7fef96a00640 1 -- 172.21.15.47:0/3432168399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e3bd0 msgr2=0x55c5429e86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.776+0000 7fef96a00640 1 --2- 172.21.15.47:0/3432168399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e3bd0 0x55c5429e86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.776+0000 7fef96a00640 1 -- 172.21.15.47:0/3432168399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e31e0 msgr2=0x55c5429e3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.776+0000 7fef96a00640 1 --2- 172.21.15.47:0/3432168399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e31e0 0x55c5429e3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.776+0000 7fef96a00640 1 -- 172.21.15.47:0/3432168399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5429092c0 con 0x55c5429e8d40 2024-03-21T11:31:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.776+0000 7fef96a00640 1 --2- 172.21.15.47:0/3432168399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e8d40 0x55c5429eb1d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fef84009cc0 tx=0x7fef840107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=933ce05cb66ad475 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:24.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.776+0000 7fef8fe00640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef7c0a3040 0x7fef7c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.776+0000 7fef8fe00640 1 -- 172.21.15.47:0/884167827 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fef880cfc40 con 0x7fef981005c0 2024-03-21T11:31:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.776+0000 7fef8cc00640 1 -- 172.21.15.47:0/3432168399 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8401c480 con 0x55c5429e8d40 2024-03-21T11:31:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.776+0000 7fef97e00640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef7c0a3040 0x7fef7c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.777+0000 7fef8cc00640 1 -- 172.21.15.47:0/3432168399 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef8401cb00 con 0x55c5429e8d40 2024-03-21T11:31:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.777+0000 7fef8cc00640 1 -- 172.21.15.47:0/3432168399 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84011d70 con 0x55c5429e8d40 2024-03-21T11:31:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.777+0000 7fefa1221740 1 -- 172.21.15.47:0/3432168399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e8d40 msgr2=0x55c5429eb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.777+0000 7fefa1221740 1 --2- 172.21.15.47:0/3432168399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e8d40 0x55c5429eb1d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fef84009cc0 tx=0x7fef840107a0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.777+0000 7fef97e00640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef7c0a3040 0x7fef7c0a5500 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fef90006010 tx=0x7fef90004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.778+0000 7fefa1221740 1 -- 172.21.15.47:0/3432168399 shutdown_connections 2024-03-21T11:31:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.778+0000 7fefa1221740 1 --2- 172.21.15.47:0/3432168399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e8d40 0x55c5429eb1d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.778+0000 7fefa1221740 1 --2- 172.21.15.47:0/3432168399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e3bd0 0x55c5429e86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.778+0000 7fefa1221740 1 --2- 172.21.15.47:0/3432168399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e31e0 0x55c5429e3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.778+0000 7fefa1221740 1 -- 172.21.15.47:0/3432168399 >> 172.21.15.47:0/3432168399 conn(0x55c5429de750 msgr2=0x55c5429deb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.778+0000 7fefa1221740 1 -- 172.21.15.47:0/3432168399 shutdown_connections 2024-03-21T11:31:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.778+0000 7fefa1221740 1 -- 172.21.15.47:0/3432168399 wait complete. 2024-03-21T11:31:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.778+0000 7fefa1221740 1 -- start start 2024-03-21T11:31:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.778+0000 7fefa1221740 1 Processor -- start 2024-03-21T11:31:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.779+0000 7fefa1221740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e31e0 0x55c5429ed360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.779+0000 7fefa1221740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e3bd0 0x55c5429ed8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.779+0000 7fefa1221740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e8d40 0x55c5429f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.779+0000 7fefa1221740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c542904a20 con 0x55c5429e8d40 2024-03-21T11:31:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.779+0000 7fefa1221740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5429448d0 con 0x55c5429e3bd0 2024-03-21T11:31:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.779+0000 7fefa1221740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5429ebcf0 con 0x55c5429e31e0 2024-03-21T11:31:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.779+0000 7fefa1221740 10 client.0 _ll_register_callbacks cb 0x55c5429411c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.779+0000 7fef97400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e31e0 0x55c5429ed360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.779+0000 7fef97400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e31e0 0x55c5429ed360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55602/0 (socket says 172.21.15.47:55602) 2024-03-21T11:31:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.779+0000 7fef97400640 1 -- 172.21.15.47:0/1856115658 learned_addr learned my addr 172.21.15.47:0/1856115658 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.779+0000 7fef96a00640 1 --2- 172.21.15.47:0/1856115658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e3bd0 0x55c5429ed8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.779+0000 7fef97e00640 1 --2- 172.21.15.47:0/1856115658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e8d40 0x55c5429f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.780+0000 7fef97400640 1 -- 172.21.15.47:0/1856115658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e3bd0 msgr2=0x55c5429ed8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.780+0000 7fef97400640 1 --2- 172.21.15.47:0/1856115658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e3bd0 0x55c5429ed8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.780+0000 7fef97400640 1 -- 172.21.15.47:0/1856115658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e8d40 msgr2=0x55c5429f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.780+0000 7fef97400640 1 --2- 172.21.15.47:0/1856115658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e8d40 0x55c5429f48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.780+0000 7fef97400640 1 -- 172.21.15.47:0/1856115658 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c54265d210 con 0x55c5429e31e0 2024-03-21T11:31:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.780+0000 7fef97e00640 1 --2- 172.21.15.47:0/1856115658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e8d40 0x55c5429f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.780+0000 7fef97400640 1 --2- 172.21.15.47:0/1856115658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e31e0 0x55c5429ed360 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fef880a4460 tx=0x7fef88098220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41031ed163ad24b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.780+0000 7fef81600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.780+0000 7fef81600640 1 -- 172.21.15.47:0/1856115658 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8809b8e0 con 0x55c5429e31e0 2024-03-21T11:31:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.780+0000 7fef81600640 1 -- 172.21.15.47:0/1856115658 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fef88003b30 con 0x55c5429e31e0 2024-03-21T11:31:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.780+0000 7fefa1221740 1 -- 172.21.15.47:0/1856115658 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef8401cb00 con 0x55c5429e31e0 2024-03-21T11:31:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.781+0000 7fef81600640 1 -- 172.21.15.47:0/1856115658 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef8809c680 con 0x55c5429e31e0 2024-03-21T11:31:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.781+0000 7fefa1221740 10 client.28237 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.781+0000 7fefa1221740 1 -- 172.21.15.47:0/1856115658 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c542729620 con 0x55c5429e31e0 2024-03-21T11:31:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.781+0000 7fef81600640 1 -- 172.21.15.47:0/1856115658 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef8401cb00 con 0x55c5429e31e0 2024-03-21T11:31:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.781+0000 7fef81600640 1 -- 172.21.15.47:0/1856115658 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef8809b560 con 0x55c5429e31e0 2024-03-21T11:31:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.781+0000 7fefa1221740 10 client.28237 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.781+0000 7fefa1221740 10 client.28237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.781+0000 7fefa1221740 20 client.28237 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.781+0000 7fefa1221740 1 --2- 172.21.15.47:0/1856115658 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5429f9210 0x55c5429fb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.781+0000 7fefa1221740 4 client.28237 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.781+0000 7fefa1221740 1 -- 172.21.15.47:0/1856115658 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c542955d70 con 0x55c5429f9210 2024-03-21T11:31:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.782+0000 7fef96a00640 1 --2- 172.21.15.47:0/1856115658 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5429f9210 0x55c5429fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.782+0000 7fef96a00640 1 --2- 172.21.15.47:0/1856115658 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5429f9210 0x55c5429fb600 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf21abf71abe7f5a server_cookie=858bb8e21a016bdb in_seq=0 out_seq=0 2024-03-21T11:31:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.782+0000 7fef81600640 10 client.28237 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.784+0000 7fef81600640 1 -- 172.21.15.47:0/1856115658 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c542955d70 con 0x55c5429f9210 2024-03-21T11:31:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.784+0000 7fef81600640 10 client.28237 handle_command_reply: tid=0 2024-03-21T11:31:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.784+0000 7fefa1221740 1 client.28237 shutdown 2024-03-21T11:31:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.785+0000 7fefa1221740 1 -- 172.21.15.47:0/1856115658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e31e0 msgr2=0x55c5429ed360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.785+0000 7fefa1221740 1 --2- 172.21.15.47:0/1856115658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e31e0 0x55c5429ed360 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fef880a4460 tx=0x7fef88098220 comp rx=0 tx=0).stop 2024-03-21T11:31:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.785+0000 7fefa1221740 1 -- 172.21.15.47:0/1856115658 shutdown_connections 2024-03-21T11:31:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.785+0000 7fefa1221740 1 --2- 172.21.15.47:0/1856115658 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5429f9210 0x55c5429fb600 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.785+0000 7fefa1221740 1 --2- 172.21.15.47:0/1856115658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e8d40 0x55c5429f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.785+0000 7fefa1221740 1 --2- 172.21.15.47:0/1856115658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e3bd0 0x55c5429ed8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.785+0000 7fefa1221740 1 --2- 172.21.15.47:0/1856115658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e31e0 0x55c5429ed360 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.785+0000 7fefa1221740 1 -- 172.21.15.47:0/1856115658 >> 172.21.15.47:0/1856115658 conn(0x55c5429de750 msgr2=0x55c5429dd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.785+0000 7fef81600640 0 client.28237 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.785+0000 7fefa1221740 1 -- 172.21.15.47:0/1856115658 shutdown_connections 2024-03-21T11:31:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.785+0000 7fefa1221740 1 -- 172.21.15.47:0/1856115658 wait complete. 2024-03-21T11:31:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.785+0000 7fefa1221740 20 client.28237 trim_cache size 0 max 16384 2024-03-21T11:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.802+0000 7fefa1221740 1 Processor -- start 2024-03-21T11:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.803+0000 7fefa1221740 1 -- start start 2024-03-21T11:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.803+0000 7fefa1221740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429de750 0x55c5429ed320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.803+0000 7fefa1221740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e31e0 0x55c5429ed860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.803+0000 7fefa1221740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e3bd0 0x55c5429f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.803+0000 7fefa1221740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981ac330 con 0x55c5429e3bd0 2024-03-21T11:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.803+0000 7fefa1221740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981adcb0 con 0x55c5429de750 2024-03-21T11:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.803+0000 7fefa1221740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981ac990 con 0x55c5429e31e0 2024-03-21T11:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.803+0000 7fef97400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429de750 0x55c5429ed320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.803+0000 7fef97400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429de750 0x55c5429ed320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40226/0 (socket says 172.21.15.47:40226) 2024-03-21T11:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.803+0000 7fef97400640 1 -- 172.21.15.47:0/1036542177 learned_addr learned my addr 172.21.15.47:0/1036542177 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.803+0000 7fef96a00640 1 --2- 172.21.15.47:0/1036542177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e31e0 0x55c5429ed860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.803+0000 7fef97e00640 1 --2- 172.21.15.47:0/1036542177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e3bd0 0x55c5429f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.803+0000 7fef96a00640 1 -- 172.21.15.47:0/1036542177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429de750 msgr2=0x55c5429ed320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.803+0000 7fef96a00640 1 --2- 172.21.15.47:0/1036542177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429de750 0x55c5429ed320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.804+0000 7fef96a00640 1 -- 172.21.15.47:0/1036542177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e3bd0 msgr2=0x55c5429f48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.804+0000 7fef96a00640 1 --2- 172.21.15.47:0/1036542177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e3bd0 0x55c5429f48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.804+0000 7fef96a00640 1 -- 172.21.15.47:0/1036542177 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5429ebbd0 con 0x55c5429e31e0 2024-03-21T11:31:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.804+0000 7fef96a00640 1 --2- 172.21.15.47:0/1036542177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e31e0 0x55c5429ed860 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fef84000c00 tx=0x7fef84013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66db62c87ef8d935 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.804+0000 7fef83e00640 1 -- 172.21.15.47:0/1036542177 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8402a110 con 0x55c5429e31e0 2024-03-21T11:31:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.804+0000 7fef83e00640 1 -- 172.21.15.47:0/1036542177 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef8402a2b0 con 0x55c5429e31e0 2024-03-21T11:31:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.804+0000 7fef83e00640 1 -- 172.21.15.47:0/1036542177 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84011cb0 con 0x55c5429e31e0 2024-03-21T11:31:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.805+0000 7fefa1221740 1 -- 172.21.15.47:0/1036542177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e31e0 msgr2=0x55c5429ed860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.805+0000 7fefa1221740 1 --2- 172.21.15.47:0/1036542177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e31e0 0x55c5429ed860 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fef84000c00 tx=0x7fef84013040 comp rx=0 tx=0).stop 2024-03-21T11:31:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.805+0000 7fefa1221740 1 -- 172.21.15.47:0/1036542177 shutdown_connections 2024-03-21T11:31:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.805+0000 7fefa1221740 1 --2- 172.21.15.47:0/1036542177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e3bd0 0x55c5429f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.805+0000 7fefa1221740 1 --2- 172.21.15.47:0/1036542177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429e31e0 0x55c5429ed860 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.805+0000 7fefa1221740 1 --2- 172.21.15.47:0/1036542177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429de750 0x55c5429ed320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.805+0000 7fefa1221740 1 -- 172.21.15.47:0/1036542177 >> 172.21.15.47:0/1036542177 conn(0x55c5429e8880 msgr2=0x55c5429e4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.805+0000 7fefa1221740 1 -- 172.21.15.47:0/1036542177 shutdown_connections 2024-03-21T11:31:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.805+0000 7fefa1221740 1 -- 172.21.15.47:0/1036542177 wait complete. 2024-03-21T11:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.806+0000 7fefa1221740 1 -- start start 2024-03-21T11:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.806+0000 7fefa1221740 1 Processor -- start 2024-03-21T11:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.806+0000 7fefa1221740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429de750 0x55c5429f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fefa1221740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e31e0 0x55c5429f2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fefa1221740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e3bd0 0x55c5429f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fefa1221740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981ad650 con 0x55c5429e31e0 2024-03-21T11:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fefa1221740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981acff0 con 0x55c5429e3bd0 2024-03-21T11:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fefa1221740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981aa4f0 con 0x55c5429de750 2024-03-21T11:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fefa1221740 10 client.0 _ll_register_callbacks cb 0x7fef880d3370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fef97400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429de750 0x55c5429f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fef97e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e3bd0 0x55c5429f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fef97400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429de750 0x55c5429f1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55610/0 (socket says 172.21.15.47:55610) 2024-03-21T11:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fef97400640 1 -- 172.21.15.47:0/1713599553 learned_addr learned my addr 172.21.15.47:0/1713599553 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fef96a00640 1 --2- 172.21.15.47:0/1713599553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e31e0 0x55c5429f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fef97400640 1 -- 172.21.15.47:0/1713599553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e3bd0 msgr2=0x55c5429f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fef97400640 1 --2- 172.21.15.47:0/1713599553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e3bd0 0x55c5429f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fef97400640 1 -- 172.21.15.47:0/1713599553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e31e0 msgr2=0x55c5429f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fef97400640 1 --2- 172.21.15.47:0/1713599553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e31e0 0x55c5429f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fef97400640 1 -- 172.21.15.47:0/1713599553 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5429f3990 con 0x55c5429de750 2024-03-21T11:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fef96a00640 1 --2- 172.21.15.47:0/1713599553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e31e0 0x55c5429f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:31:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fef97400640 1 --2- 172.21.15.47:0/1713599553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429de750 0x55c5429f1ec0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fef88005b60 tx=0x7fef8800c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d63841183595c6cf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.807+0000 7fef81600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.808+0000 7fef81600640 1 -- 172.21.15.47:0/1713599553 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8807e060 con 0x55c5429de750 2024-03-21T11:31:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.808+0000 7fef81600640 1 -- 172.21.15.47:0/1713599553 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fef8809b3d0 con 0x55c5429de750 2024-03-21T11:31:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.808+0000 7fefa1221740 1 -- 172.21.15.47:0/1713599553 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef8402a2b0 con 0x55c5429de750 2024-03-21T11:31:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.808+0000 7fef81600640 1 -- 172.21.15.47:0/1713599553 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef8809b8e0 con 0x55c5429de750 2024-03-21T11:31:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.808+0000 7fefa1221740 10 client.28243 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.808+0000 7fefa1221740 1 -- 172.21.15.47:0/1713599553 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c542a54ab0 con 0x55c5429de750 2024-03-21T11:31:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.808+0000 7fef81600640 1 -- 172.21.15.47:0/1713599553 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef8402a2b0 con 0x55c5429de750 2024-03-21T11:31:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.808+0000 7fef81600640 1 -- 172.21.15.47:0/1713599553 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef880a4be0 con 0x55c5429de750 2024-03-21T11:31:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.808+0000 7fefa1221740 10 client.28243 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.808+0000 7fefa1221740 10 client.28243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.808+0000 7fefa1221740 20 client.28243 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.808+0000 7fefa1221740 1 --2- 172.21.15.47:0/1713599553 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5429e8d40 0x55c542a570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.808+0000 7fefa1221740 4 client.28243 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:31:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.808+0000 7fefa1221740 1 -- 172.21.15.47:0/1713599553 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c5429500c0 con 0x55c5429e8d40 2024-03-21T11:31:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.809+0000 7fef96a00640 1 --2- 172.21.15.47:0/1713599553 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5429e8d40 0x55c542a570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.809+0000 7fef96a00640 1 --2- 172.21.15.47:0/1713599553 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5429e8d40 0x55c542a570f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a32c44b351f6294e server_cookie=b4a0d94985b23237 in_seq=0 out_seq=0 2024-03-21T11:31:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:24.809+0000 7fef81600640 10 client.28243 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:24.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:24 smithi047 ceph-mon[31351]: pgmap v642: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 325 op/s 2024-03-21T11:31:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.031+0000 7fef81600640 1 -- 172.21.15.47:0/1713599553 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c5429500c0 con 0x55c5429e8d40 2024-03-21T11:31:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.031+0000 7fef81600640 10 client.28243 handle_command_reply: tid=0 2024-03-21T11:31:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.031+0000 7fefa1221740 1 client.28243 shutdown 2024-03-21T11:31:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.031+0000 7fefa1221740 1 -- 172.21.15.47:0/1713599553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429de750 msgr2=0x55c5429f1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.031+0000 7fefa1221740 1 --2- 172.21.15.47:0/1713599553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429de750 0x55c5429f1ec0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fef88005b60 tx=0x7fef8800c7d0 comp rx=0 tx=0).stop 2024-03-21T11:31:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.031+0000 7fefa1221740 1 -- 172.21.15.47:0/1713599553 shutdown_connections 2024-03-21T11:31:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.031+0000 7fefa1221740 1 --2- 172.21.15.47:0/1713599553 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5429e8d40 0x55c542a570f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.031+0000 7fefa1221740 1 --2- 172.21.15.47:0/1713599553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5429e3bd0 0x55c5429f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.031+0000 7fefa1221740 1 --2- 172.21.15.47:0/1713599553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5429e31e0 0x55c5429f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.031+0000 7fefa1221740 1 --2- 172.21.15.47:0/1713599553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5429de750 0x55c5429f1ec0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.031+0000 7fefa1221740 1 -- 172.21.15.47:0/1713599553 >> 172.21.15.47:0/1713599553 conn(0x55c5429e8880 msgr2=0x55c5429ed2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.032+0000 7fef81600640 0 client.28243 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.032+0000 7fefa1221740 1 -- 172.21.15.47:0/1713599553 shutdown_connections 2024-03-21T11:31:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.032+0000 7fefa1221740 1 -- 172.21.15.47:0/1713599553 wait complete. 2024-03-21T11:31:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.032+0000 7fefa1221740 20 client.28243 trim_cache size 0 max 16384 2024-03-21T11:31:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:31:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:31:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:31:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:31:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.033+0000 7fef9e600640 1 -- 172.21.15.47:0/884167827 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef7c0a3040 msgr2=0x7fef7c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.033+0000 7fef9e600640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef7c0a3040 0x7fef7c0a5500 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fef90006010 tx=0x7fef90004a40 comp rx=0 tx=0).stop 2024-03-21T11:31:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.033+0000 7fef9e600640 1 -- 172.21.15.47:0/884167827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef981005c0 msgr2=0x7fef9810ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.033+0000 7fef9e600640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef981005c0 0x7fef9810ac40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fef8800e9c0 tx=0x7fef8800ee80 comp rx=0 tx=0).stop 2024-03-21T11:31:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.033+0000 7fef9e600640 1 -- 172.21.15.47:0/884167827 shutdown_connections 2024-03-21T11:31:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.033+0000 7fef9e600640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef7c0a3040 0x7fef7c0a5500 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.033+0000 7fef9e600640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef98102170 0x7fef981097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.033+0000 7fef9e600640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef981017b0 0x7fef98109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.033+0000 7fef9e600640 1 --2- 172.21.15.47:0/884167827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef981005c0 0x7fef9810ac40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.033+0000 7fef9e600640 1 -- 172.21.15.47:0/884167827 >> 172.21.15.47:0/884167827 conn(0x7fef980fbb30 msgr2=0x7fef981049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.033+0000 7fef9e600640 1 -- 172.21.15.47:0/884167827 shutdown_connections 2024-03-21T11:31:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:25.033+0000 7fef9e600640 1 -- 172.21.15.47:0/884167827 wait complete. 2024-03-21T11:31:25.535 INFO:tasks.workunit.client.0.smithi047.stdout: 1 572287 8.23 MB/sec execute 249 sec latency 115.413 ms 2024-03-21T11:31:26.535 INFO:tasks.workunit.client.0.smithi047.stdout: 1 573066 8.21 MB/sec execute 250 sec latency 64.198 ms 2024-03-21T11:31:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:26 smithi100 ceph-mon[36245]: pgmap v643: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 2.0 MiB/s wr, 197 op/s 2024-03-21T11:31:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:26 smithi102 ceph-mon[36748]: pgmap v643: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 2.0 MiB/s wr, 197 op/s 2024-03-21T11:31:26.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:26 smithi047 ceph-mon[31351]: pgmap v643: 129 pgs: 129 active+clean; 37 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 2.0 MiB/s wr, 197 op/s 2024-03-21T11:31:27.535 INFO:tasks.workunit.client.0.smithi047.stdout: 1 575043 8.22 MB/sec execute 251 sec latency 50.094 ms 2024-03-21T11:31:28.536 INFO:tasks.workunit.client.0.smithi047.stdout: 1 576036 8.21 MB/sec execute 252 sec latency 11.722 ms 2024-03-21T11:31:28.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:28 smithi100 ceph-mon[36245]: pgmap v644: 129 pgs: 129 active+clean; 36 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-03-21T11:31:28.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:28 smithi102 ceph-mon[36748]: pgmap v644: 129 pgs: 129 active+clean; 36 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-03-21T11:31:28.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:28 smithi047 ceph-mon[31351]: pgmap v644: 129 pgs: 129 active+clean; 36 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-03-21T11:31:29.536 INFO:tasks.workunit.client.0.smithi047.stdout: 1 577762 8.20 MB/sec execute 253 sec latency 14.525 ms 2024-03-21T11:31:30.536 INFO:tasks.workunit.client.0.smithi047.stdout: 1 579118 8.21 MB/sec execute 254 sec latency 15.626 ms 2024-03-21T11:31:30.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:30 smithi100 ceph-mon[36245]: pgmap v645: 129 pgs: 129 active+clean; 32 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 292 op/s 2024-03-21T11:31:30.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:30 smithi102 ceph-mon[36748]: pgmap v645: 129 pgs: 129 active+clean; 32 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 292 op/s 2024-03-21T11:31:30.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:30 smithi047 ceph-mon[31351]: pgmap v645: 129 pgs: 129 active+clean; 32 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 292 op/s 2024-03-21T11:31:31.537 INFO:tasks.workunit.client.0.smithi047.stdout: 1 579892 8.19 MB/sec execute 255 sec latency 24.287 ms 2024-03-21T11:31:32.537 INFO:tasks.workunit.client.0.smithi047.stdout: 1 581692 8.20 MB/sec execute 256 sec latency 50.130 ms 2024-03-21T11:31:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:32 smithi100 ceph-mon[36245]: pgmap v646: 129 pgs: 129 active+clean; 32 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 267 op/s 2024-03-21T11:31:32.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:32 smithi102 ceph-mon[36748]: pgmap v646: 129 pgs: 129 active+clean; 32 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 267 op/s 2024-03-21T11:31:32.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:32 smithi047 ceph-mon[31351]: pgmap v646: 129 pgs: 129 active+clean; 32 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 267 op/s 2024-03-21T11:31:33.537 INFO:tasks.workunit.client.0.smithi047.stdout: 1 582971 8.20 MB/sec execute 257 sec latency 15.889 ms 2024-03-21T11:31:34.538 INFO:tasks.workunit.client.0.smithi047.stdout: 1 584305 8.19 MB/sec execute 258 sec latency 14.275 ms 2024-03-21T11:31:34.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:34 smithi102 ceph-mon[36748]: pgmap v647: 129 pgs: 129 active+clean; 36 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 281 op/s 2024-03-21T11:31:34.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:34 smithi047 ceph-mon[31351]: pgmap v647: 129 pgs: 129 active+clean; 36 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 281 op/s 2024-03-21T11:31:34.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:34 smithi100 ceph-mon[36245]: pgmap v647: 129 pgs: 129 active+clean; 36 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 281 op/s 2024-03-21T11:31:35.538 INFO:tasks.workunit.client.0.smithi047.stdout: 1 586180 8.20 MB/sec execute 259 sec latency 13.642 ms 2024-03-21T11:31:36.538 INFO:tasks.workunit.client.0.smithi047.stdout: 1 586984 8.18 MB/sec execute 260 sec latency 21.343 ms 2024-03-21T11:31:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:36 smithi102 ceph-mon[36748]: pgmap v648: 129 pgs: 129 active+clean; 35 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.9 MiB/s wr, 261 op/s 2024-03-21T11:31:36.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:36 smithi047 ceph-mon[31351]: pgmap v648: 129 pgs: 129 active+clean; 35 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.9 MiB/s wr, 261 op/s 2024-03-21T11:31:36.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:36 smithi100 ceph-mon[36245]: pgmap v648: 129 pgs: 129 active+clean; 35 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.9 MiB/s wr, 261 op/s 2024-03-21T11:31:37.538 INFO:tasks.workunit.client.0.smithi047.stdout: 1 588642 8.18 MB/sec execute 261 sec latency 89.854 ms 2024-03-21T11:31:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.185+0000 7fb277200640 1 -- 172.21.15.47:0/2936762429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb278102700 msgr2=0x7fb278102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.185+0000 7fb277200640 1 --2- 172.21.15.47:0/2936762429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb278102700 0x7fb278102b80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb260009900 tx=0x7fb26002bc40 comp rx=0 tx=0).stop 2024-03-21T11:31:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.185+0000 7fb277200640 1 -- 172.21.15.47:0/2936762429 shutdown_connections 2024-03-21T11:31:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.185+0000 7fb277200640 1 --2- 172.21.15.47:0/2936762429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2781030c0 0x7fb278109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.185+0000 7fb277200640 1 --2- 172.21.15.47:0/2936762429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb278102700 0x7fb278102b80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.185+0000 7fb277200640 1 --2- 172.21.15.47:0/2936762429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb278101510 0x7fb278101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.185+0000 7fb277200640 1 -- 172.21.15.47:0/2936762429 >> 172.21.15.47:0/2936762429 conn(0x7fb2780fca80 msgr2=0x7fb2780feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.185+0000 7fb277200640 1 -- 172.21.15.47:0/2936762429 shutdown_connections 2024-03-21T11:31:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.185+0000 7fb277200640 1 -- 172.21.15.47:0/2936762429 wait complete. 2024-03-21T11:31:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.186+0000 7fb277200640 1 Processor -- start 2024-03-21T11:31:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.187+0000 7fb277200640 1 -- start start 2024-03-21T11:31:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.187+0000 7fb277200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb278101510 0x7fb2781977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.187+0000 7fb277200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb278102700 0x7fb278197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.187+0000 7fb277200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2781030c0 0x7fb27819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.187+0000 7fb277200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27810d1f0 con 0x7fb278102700 2024-03-21T11:31:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.187+0000 7fb277200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27810d070 con 0x7fb278101510 2024-03-21T11:31:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.187+0000 7fb277200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27810d370 con 0x7fb2781030c0 2024-03-21T11:31:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.187+0000 7fb275e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb278101510 0x7fb2781977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.187+0000 7fb276800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2781030c0 0x7fb27819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.187+0000 7fb275e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb278101510 0x7fb2781977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51860/0 (socket says 172.21.15.47:51860) 2024-03-21T11:31:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.187+0000 7fb275e00640 1 -- 172.21.15.47:0/3905757731 learned_addr learned my addr 172.21.15.47:0/3905757731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.187+0000 7fb26d400640 1 --2- 172.21.15.47:0/3905757731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb278102700 0x7fb278197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.188+0000 7fb276800640 1 -- 172.21.15.47:0/3905757731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb278101510 msgr2=0x7fb2781977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.188+0000 7fb276800640 1 --2- 172.21.15.47:0/3905757731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb278101510 0x7fb2781977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.188+0000 7fb276800640 1 -- 172.21.15.47:0/3905757731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb278102700 msgr2=0x7fb278197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.188+0000 7fb276800640 1 --2- 172.21.15.47:0/3905757731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb278102700 0x7fb278197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.188+0000 7fb276800640 1 -- 172.21.15.47:0/3905757731 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27819f3e0 con 0x7fb2781030c0 2024-03-21T11:31:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.188+0000 7fb275e00640 1 --2- 172.21.15.47:0/3905757731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb278101510 0x7fb2781977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.188+0000 7fb276800640 1 --2- 172.21.15.47:0/3905757731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2781030c0 0x7fb27819ed70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb27000e940 tx=0x7fb27000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.188+0000 7fb26f400640 1 -- 172.21.15.47:0/3905757731 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb270004d20 con 0x7fb2781030c0 2024-03-21T11:31:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.188+0000 7fb26f400640 1 -- 172.21.15.47:0/3905757731 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb270005020 con 0x7fb2781030c0 2024-03-21T11:31:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.189+0000 7fb26f400640 1 -- 172.21.15.47:0/3905757731 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2700116b0 con 0x7fb2781030c0 2024-03-21T11:31:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.189+0000 7fb277200640 1 -- 172.21.15.47:0/3905757731 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27819f6d0 con 0x7fb2781030c0 2024-03-21T11:31:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.189+0000 7fb277200640 1 -- 172.21.15.47:0/3905757731 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27819fba0 con 0x7fb2781030c0 2024-03-21T11:31:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.192+0000 7fb27f78f740 1 Processor -- start 2024-03-21T11:31:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.192+0000 7fb26f400640 1 -- 172.21.15.47:0/3905757731 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb2700054d0 con 0x7fb2781030c0 2024-03-21T11:31:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.192+0000 7fb27f78f740 1 -- start start 2024-03-21T11:31:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.193+0000 7fb27f78f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a3800 0x5577ac5a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.193+0000 7fb27f78f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a41f0 0x5577ac5a8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.193+0000 7fb27f78f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a9360 0x5577ac5ab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.193+0000 7fb27f78f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ac5623b0 con 0x5577ac5a3800 2024-03-21T11:31:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.193+0000 7fb27f78f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ac504830 con 0x5577ac5a41f0 2024-03-21T11:31:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.193+0000 7fb27f78f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ac4ad5d0 con 0x5577ac5a9360 2024-03-21T11:31:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.193+0000 7fb275e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a41f0 0x5577ac5a8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.193+0000 7fb275e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a41f0 0x5577ac5a8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51868/0 (socket says 172.21.15.47:51868) 2024-03-21T11:31:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.193+0000 7fb275e00640 1 -- 172.21.15.47:0/3535593327 learned_addr learned my addr 172.21.15.47:0/3535593327 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.193+0000 7fb26d400640 1 --2- 172.21.15.47:0/3535593327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a3800 0x5577ac5a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.193+0000 7fb275e00640 1 -- 172.21.15.47:0/3535593327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a9360 msgr2=0x5577ac5ab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.193+0000 7fb26d400640 1 --2- 172.21.15.47:0/3535593327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a9360 0x5577ac5ab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.194+0000 7fb275e00640 1 --2- 172.21.15.47:0/3535593327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a9360 0x5577ac5ab7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.194+0000 7fb275e00640 1 -- 172.21.15.47:0/3535593327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a3800 msgr2=0x5577ac5a3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.194+0000 7fb275e00640 1 --2- 172.21.15.47:0/3535593327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a3800 0x5577ac5a3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.194+0000 7fb275e00640 1 -- 172.21.15.47:0/3535593327 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577ac4c52c0 con 0x5577ac5a41f0 2024-03-21T11:31:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.194+0000 7fb26d400640 1 --2- 172.21.15.47:0/3535593327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a9360 0x5577ac5ab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:31:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.194+0000 7fb275e00640 1 --2- 172.21.15.47:0/3535593327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a41f0 0x5577ac5a8cf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb2680115c0 tx=0x7fb268011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=954be9c3809301b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.194+0000 7fb26f400640 1 --2- 172.21.15.47:0/3905757731 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2580a2f10 0x7fb2580a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.194+0000 7fb267000640 1 -- 172.21.15.47:0/3535593327 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268005020 con 0x5577ac5a41f0 2024-03-21T11:31:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.194+0000 7fb267000640 1 -- 172.21.15.47:0/3535593327 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2680047a0 con 0x5577ac5a41f0 2024-03-21T11:31:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.194+0000 7fb267000640 1 -- 172.21.15.47:0/3535593327 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26800f9b0 con 0x5577ac5a41f0 2024-03-21T11:31:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.194+0000 7fb26f400640 1 -- 172.21.15.47:0/3905757731 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb2700cf790 con 0x7fb2781030c0 2024-03-21T11:31:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.194+0000 7fb26d400640 1 --2- 172.21.15.47:0/3905757731 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2580a2f10 0x7fb2580a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.195+0000 7fb27f78f740 1 -- 172.21.15.47:0/3535593327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a41f0 msgr2=0x5577ac5a8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.195+0000 7fb27f78f740 1 --2- 172.21.15.47:0/3535593327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a41f0 0x5577ac5a8cf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb2680115c0 tx=0x7fb268011a80 comp rx=0 tx=0).stop 2024-03-21T11:31:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.195+0000 7fb27f78f740 1 -- 172.21.15.47:0/3535593327 shutdown_connections 2024-03-21T11:31:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.195+0000 7fb27f78f740 1 --2- 172.21.15.47:0/3535593327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a9360 0x5577ac5ab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.195+0000 7fb27f78f740 1 --2- 172.21.15.47:0/3535593327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a41f0 0x5577ac5a8cf0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.195+0000 7fb27f78f740 1 --2- 172.21.15.47:0/3535593327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a3800 0x5577ac5a3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.195+0000 7fb27f78f740 1 -- 172.21.15.47:0/3535593327 >> 172.21.15.47:0/3535593327 conn(0x5577ac59ed70 msgr2=0x5577ac59f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.195+0000 7fb26d400640 1 --2- 172.21.15.47:0/3905757731 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2580a2f10 0x7fb2580a53d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb26002c250 tx=0x7fb26002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.195+0000 7fb27f78f740 1 -- 172.21.15.47:0/3535593327 shutdown_connections 2024-03-21T11:31:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.195+0000 7fb27f78f740 1 -- 172.21.15.47:0/3535593327 wait complete. 2024-03-21T11:31:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.196+0000 7fb27f78f740 1 -- start start 2024-03-21T11:31:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.196+0000 7fb27f78f740 1 Processor -- start 2024-03-21T11:31:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.196+0000 7fb27f78f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a3800 0x5577ac5ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb27f78f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a41f0 0x5577ac5adf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb27f78f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a9360 0x5577ac5b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb27f78f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ac4c0a20 con 0x5577ac5a3800 2024-03-21T11:31:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb27f78f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ac5008d0 con 0x5577ac5a9360 2024-03-21T11:31:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb27f78f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ac5ac310 con 0x5577ac5a41f0 2024-03-21T11:31:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb27f78f740 10 client.0 _ll_register_callbacks cb 0x5577ac4fd1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb26d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a41f0 0x5577ac5adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb276800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a9360 0x5577ac5b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb275e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a3800 0x5577ac5ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb276800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a9360 0x5577ac5b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51882/0 (socket says 172.21.15.47:51882) 2024-03-21T11:31:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb276800640 1 -- 172.21.15.47:0/1804720646 learned_addr learned my addr 172.21.15.47:0/1804720646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb26d400640 1 -- 172.21.15.47:0/1804720646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a9360 msgr2=0x5577ac5b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb26d400640 1 --2- 172.21.15.47:0/1804720646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a9360 0x5577ac5b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb26d400640 1 -- 172.21.15.47:0/1804720646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a3800 msgr2=0x5577ac5ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb26d400640 1 --2- 172.21.15.47:0/1804720646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a3800 0x5577ac5ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb26d400640 1 -- 172.21.15.47:0/1804720646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577ac219210 con 0x5577ac5a41f0 2024-03-21T11:31:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.197+0000 7fb276800640 1 --2- 172.21.15.47:0/1804720646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a9360 0x5577ac5b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.198+0000 7fb26d400640 1 --2- 172.21.15.47:0/1804720646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a41f0 0x5577ac5adf00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb260036040 tx=0x7fb260005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d789e0bd5bdf5cdb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.198+0000 7fb25f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.198+0000 7fb25f400640 1 -- 172.21.15.47:0/1804720646 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb260004c50 con 0x5577ac5a41f0 2024-03-21T11:31:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.198+0000 7fb25f400640 1 -- 172.21.15.47:0/1804720646 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb260004ed0 con 0x5577ac5a41f0 2024-03-21T11:31:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.198+0000 7fb27f78f740 1 -- 172.21.15.47:0/1804720646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2680047a0 con 0x5577ac5a41f0 2024-03-21T11:31:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.198+0000 7fb25f400640 1 -- 172.21.15.47:0/1804720646 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2600050d0 con 0x5577ac5a41f0 2024-03-21T11:31:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.198+0000 7fb27f78f740 10 client.28252 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.198+0000 7fb27f78f740 1 -- 172.21.15.47:0/1804720646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5577ac2e5620 con 0x5577ac5a41f0 2024-03-21T11:31:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.198+0000 7fb25f400640 1 -- 172.21.15.47:0/1804720646 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2680047a0 con 0x5577ac5a41f0 2024-03-21T11:31:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.199+0000 7fb25f400640 1 -- 172.21.15.47:0/1804720646 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb260051070 con 0x5577ac5a41f0 2024-03-21T11:31:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.199+0000 7fb27f78f740 10 client.28252 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.199+0000 7fb27f78f740 10 client.28252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.199+0000 7fb27f78f740 20 client.28252 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.199+0000 7fb27f78f740 1 --2- 172.21.15.47:0/1804720646 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5577ac5b9790 0x5577ac5bbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.199+0000 7fb27f78f740 4 client.28252 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.199+0000 7fb27f78f740 1 -- 172.21.15.47:0/1804720646 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577ac511d70 con 0x5577ac5b9790 2024-03-21T11:31:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.199+0000 7fb275e00640 1 --2- 172.21.15.47:0/1804720646 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5577ac5b9790 0x5577ac5bbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.200+0000 7fb275e00640 1 --2- 172.21.15.47:0/1804720646 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5577ac5b9790 0x5577ac5bbbc0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ae718f9289658e0 server_cookie=5d3a998aef83e968 in_seq=0 out_seq=0 2024-03-21T11:31:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.200+0000 7fb25f400640 10 client.28252 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.201+0000 7fb25f400640 1 -- 172.21.15.47:0/1804720646 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5577ac511d70 con 0x5577ac5b9790 2024-03-21T11:31:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.201+0000 7fb25f400640 10 client.28252 handle_command_reply: tid=0 2024-03-21T11:31:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.201+0000 7fb27f78f740 1 client.28252 shutdown 2024-03-21T11:31:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.201+0000 7fb27f78f740 1 -- 172.21.15.47:0/1804720646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a41f0 msgr2=0x5577ac5adf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.201+0000 7fb27f78f740 1 --2- 172.21.15.47:0/1804720646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a41f0 0x5577ac5adf00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb260036040 tx=0x7fb260005fd0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.201+0000 7fb27f78f740 1 -- 172.21.15.47:0/1804720646 shutdown_connections 2024-03-21T11:31:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.201+0000 7fb27f78f740 1 --2- 172.21.15.47:0/1804720646 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5577ac5b9790 0x5577ac5bbbc0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.201+0000 7fb27f78f740 1 --2- 172.21.15.47:0/1804720646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a9360 0x5577ac5b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.201+0000 7fb27f78f740 1 --2- 172.21.15.47:0/1804720646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a41f0 0x5577ac5adf00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.201+0000 7fb27f78f740 1 --2- 172.21.15.47:0/1804720646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a3800 0x5577ac5ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.201+0000 7fb25f400640 0 client.28252 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.201+0000 7fb27f78f740 1 -- 172.21.15.47:0/1804720646 >> 172.21.15.47:0/1804720646 conn(0x5577ac59ed70 msgr2=0x5577ac599b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.201+0000 7fb27f78f740 1 -- 172.21.15.47:0/1804720646 shutdown_connections 2024-03-21T11:31:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.201+0000 7fb27f78f740 1 -- 172.21.15.47:0/1804720646 wait complete. 2024-03-21T11:31:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.202+0000 7fb27f78f740 20 client.28252 trim_cache size 0 max 16384 2024-03-21T11:31:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.218+0000 7fb27f78f740 1 Processor -- start 2024-03-21T11:31:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.218+0000 7fb27f78f740 1 -- start start 2024-03-21T11:31:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.218+0000 7fb27f78f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a3800 0x5577ac5a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.218+0000 7fb27f78f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a41f0 0x5577ac5b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.218+0000 7fb27f78f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a9360 0x5577ac5a5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.218+0000 7fb27f78f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2781a7d00 con 0x5577ac5a41f0 2024-03-21T11:31:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.218+0000 7fb27f78f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2781a9680 con 0x5577ac5a9360 2024-03-21T11:31:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.218+0000 7fb27f78f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2781a8360 con 0x5577ac5a3800 2024-03-21T11:31:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.218+0000 7fb275e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a3800 0x5577ac5a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.218+0000 7fb275e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a3800 0x5577ac5a3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43460/0 (socket says 172.21.15.47:43460) 2024-03-21T11:31:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.218+0000 7fb26d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a41f0 0x5577ac5b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.218+0000 7fb275e00640 1 -- 172.21.15.47:0/1844213166 learned_addr learned my addr 172.21.15.47:0/1844213166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.219+0000 7fb276800640 1 --2- 172.21.15.47:0/1844213166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a9360 0x5577ac5a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.219+0000 7fb275e00640 1 -- 172.21.15.47:0/1844213166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a9360 msgr2=0x5577ac5a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.219+0000 7fb275e00640 1 --2- 172.21.15.47:0/1844213166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a9360 0x5577ac5a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.219+0000 7fb275e00640 1 -- 172.21.15.47:0/1844213166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a41f0 msgr2=0x5577ac5b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.219+0000 7fb275e00640 1 --2- 172.21.15.47:0/1844213166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a41f0 0x5577ac5b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.219+0000 7fb275e00640 1 -- 172.21.15.47:0/1844213166 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577ac598f80 con 0x5577ac5a3800 2024-03-21T11:31:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.219+0000 7fb275e00640 1 --2- 172.21.15.47:0/1844213166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a3800 0x5577ac5a3be0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb268009250 tx=0x7fb268004f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f1ef67f17174367 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.219+0000 7fb266600640 1 -- 172.21.15.47:0/1844213166 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26801d040 con 0x5577ac5a3800 2024-03-21T11:31:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.219+0000 7fb266600640 1 -- 172.21.15.47:0/1844213166 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb26802e030 con 0x5577ac5a3800 2024-03-21T11:31:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.219+0000 7fb266600640 1 -- 172.21.15.47:0/1844213166 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268005000 con 0x5577ac5a3800 2024-03-21T11:31:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.220+0000 7fb27f78f740 1 -- 172.21.15.47:0/1844213166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a3800 msgr2=0x5577ac5a3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.220+0000 7fb27f78f740 1 --2- 172.21.15.47:0/1844213166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a3800 0x5577ac5a3be0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb268009250 tx=0x7fb268004f10 comp rx=0 tx=0).stop 2024-03-21T11:31:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.220+0000 7fb27f78f740 1 -- 172.21.15.47:0/1844213166 shutdown_connections 2024-03-21T11:31:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.220+0000 7fb27f78f740 1 --2- 172.21.15.47:0/1844213166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a9360 0x5577ac5a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.220+0000 7fb27f78f740 1 --2- 172.21.15.47:0/1844213166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a41f0 0x5577ac5b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.220+0000 7fb27f78f740 1 --2- 172.21.15.47:0/1844213166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a3800 0x5577ac5a3be0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.220+0000 7fb27f78f740 1 -- 172.21.15.47:0/1844213166 >> 172.21.15.47:0/1844213166 conn(0x5577ac59ed70 msgr2=0x5577ac59f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.220+0000 7fb27f78f740 1 -- 172.21.15.47:0/1844213166 shutdown_connections 2024-03-21T11:31:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.221+0000 7fb27f78f740 1 -- 172.21.15.47:0/1844213166 wait complete. 2024-03-21T11:31:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.221+0000 7fb27f78f740 1 -- start start 2024-03-21T11:31:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.221+0000 7fb27f78f740 1 Processor -- start 2024-03-21T11:31:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb27f78f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a3800 0x5577ac5b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb27f78f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a41f0 0x5577ac5b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb27f78f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a9360 0x5577ac5b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb27f78f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2781a9020 con 0x5577ac5a9360 2024-03-21T11:31:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb27f78f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2781a89c0 con 0x5577ac5a3800 2024-03-21T11:31:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb27f78f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2781a5ec0 con 0x5577ac5a41f0 2024-03-21T11:31:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb27f78f740 10 client.0 _ll_register_callbacks cb 0x7fb260009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb26d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a41f0 0x5577ac5b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb26d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a41f0 0x5577ac5b2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43474/0 (socket says 172.21.15.47:43474) 2024-03-21T11:31:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb26d400640 1 -- 172.21.15.47:0/4195211885 learned_addr learned my addr 172.21.15.47:0/4195211885 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb275e00640 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a3800 0x5577ac5b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb276800640 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a9360 0x5577ac5b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb26d400640 1 -- 172.21.15.47:0/4195211885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a3800 msgr2=0x5577ac5b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb26d400640 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a3800 0x5577ac5b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb26d400640 1 -- 172.21.15.47:0/4195211885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a9360 msgr2=0x5577ac5b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb26d400640 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a9360 0x5577ac5b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb26d400640 1 -- 172.21.15.47:0/4195211885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577ac5b44f0 con 0x5577ac5a41f0 2024-03-21T11:31:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb276800640 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a9360 0x5577ac5b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.222+0000 7fb275e00640 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a3800 0x5577ac5b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.223+0000 7fb26d400640 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a41f0 0x5577ac5b2f90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb260009900 tx=0x7fb260005480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb632ab9e98b5bc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.223+0000 7fb25f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.223+0000 7fb25f400640 1 -- 172.21.15.47:0/4195211885 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2600043e0 con 0x5577ac5a41f0 2024-03-21T11:31:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.223+0000 7fb25f400640 1 -- 172.21.15.47:0/4195211885 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb260003800 con 0x5577ac5a41f0 2024-03-21T11:31:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.223+0000 7fb27f78f740 1 -- 172.21.15.47:0/4195211885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb26802e030 con 0x5577ac5a41f0 2024-03-21T11:31:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.223+0000 7fb25f400640 1 -- 172.21.15.47:0/4195211885 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb26006a050 con 0x5577ac5a41f0 2024-03-21T11:31:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.223+0000 7fb27f78f740 10 client.28258 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.223+0000 7fb27f78f740 1 -- 172.21.15.47:0/4195211885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5577ac6154f0 con 0x5577ac5a41f0 2024-03-21T11:31:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.224+0000 7fb25f400640 1 -- 172.21.15.47:0/4195211885 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb26802e030 con 0x5577ac5a41f0 2024-03-21T11:31:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.224+0000 7fb25f400640 1 -- 172.21.15.47:0/4195211885 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb260051070 con 0x5577ac5a41f0 2024-03-21T11:31:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.224+0000 7fb27f78f740 10 client.28258 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.224+0000 7fb27f78f740 10 client.28258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.224+0000 7fb27f78f740 20 client.28258 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.224+0000 7fb27f78f740 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5577ac615e40 0x5577ac618260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.224+0000 7fb27f78f740 4 client.28258 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:31:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.224+0000 7fb27f78f740 1 -- 172.21.15.47:0/4195211885 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5577ac50c0c0 con 0x5577ac615e40 2024-03-21T11:31:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.224+0000 7fb275e00640 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5577ac615e40 0x5577ac618260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.225+0000 7fb275e00640 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5577ac615e40 0x5577ac618260 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ef28a246ef46af server_cookie=10139f5b3037f372 in_seq=0 out_seq=0 2024-03-21T11:31:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.225+0000 7fb25f400640 10 client.28258 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.436+0000 7fb25f400640 1 -- 172.21.15.47:0/4195211885 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5577ac50c0c0 con 0x5577ac615e40 2024-03-21T11:31:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.436+0000 7fb25f400640 10 client.28258 handle_command_reply: tid=0 2024-03-21T11:31:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.436+0000 7fb27f78f740 1 client.28258 shutdown 2024-03-21T11:31:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.436+0000 7fb27f78f740 1 -- 172.21.15.47:0/4195211885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a41f0 msgr2=0x5577ac5b2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.436+0000 7fb27f78f740 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a41f0 0x5577ac5b2f90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb260009900 tx=0x7fb260005480 comp rx=0 tx=0).stop 2024-03-21T11:31:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.436+0000 7fb27f78f740 1 -- 172.21.15.47:0/4195211885 shutdown_connections 2024-03-21T11:31:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.436+0000 7fb27f78f740 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5577ac615e40 0x5577ac618260 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.436+0000 7fb27f78f740 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577ac5a9360 0x5577ac5b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.436+0000 7fb27f78f740 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577ac5a41f0 0x5577ac5b2f90 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.436+0000 7fb27f78f740 1 --2- 172.21.15.47:0/4195211885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577ac5a3800 0x5577ac5b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.436+0000 7fb27f78f740 1 -- 172.21.15.47:0/4195211885 >> 172.21.15.47:0/4195211885 conn(0x5577ac59ed70 msgr2=0x5577ac5a8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.436+0000 7fb25f400640 0 client.28258 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.436+0000 7fb27f78f740 1 -- 172.21.15.47:0/4195211885 shutdown_connections 2024-03-21T11:31:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.436+0000 7fb27f78f740 1 -- 172.21.15.47:0/4195211885 wait complete. 2024-03-21T11:31:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.437+0000 7fb27f78f740 20 client.28258 trim_cache size 0 max 16384 2024-03-21T11:31:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:31:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:31:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:31:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:31:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.437+0000 7fb277200640 1 -- 172.21.15.47:0/3905757731 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2580a2f10 msgr2=0x7fb2580a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.437+0000 7fb277200640 1 --2- 172.21.15.47:0/3905757731 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2580a2f10 0x7fb2580a53d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb26002c250 tx=0x7fb26002cf90 comp rx=0 tx=0).stop 2024-03-21T11:31:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.438+0000 7fb277200640 1 -- 172.21.15.47:0/3905757731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2781030c0 msgr2=0x7fb27819ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.438+0000 7fb277200640 1 --2- 172.21.15.47:0/3905757731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2781030c0 0x7fb27819ed70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb27000e940 tx=0x7fb27000ee00 comp rx=0 tx=0).stop 2024-03-21T11:31:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.438+0000 7fb277200640 1 -- 172.21.15.47:0/3905757731 shutdown_connections 2024-03-21T11:31:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.438+0000 7fb277200640 1 --2- 172.21.15.47:0/3905757731 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2580a2f10 0x7fb2580a53d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.438+0000 7fb277200640 1 --2- 172.21.15.47:0/3905757731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2781030c0 0x7fb27819ed70 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.438+0000 7fb277200640 1 --2- 172.21.15.47:0/3905757731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb278102700 0x7fb278197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.438+0000 7fb277200640 1 --2- 172.21.15.47:0/3905757731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb278101510 0x7fb2781977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.438+0000 7fb277200640 1 -- 172.21.15.47:0/3905757731 >> 172.21.15.47:0/3905757731 conn(0x7fb2780fca80 msgr2=0x7fb278105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.438+0000 7fb277200640 1 -- 172.21.15.47:0/3905757731 shutdown_connections 2024-03-21T11:31:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:38.438+0000 7fb277200640 1 -- 172.21.15.47:0/3905757731 wait complete. 2024-03-21T11:31:38.538 INFO:tasks.workunit.client.0.smithi047.stdout: 1 589976 8.19 MB/sec execute 262 sec latency 139.464 ms 2024-03-21T11:31:38.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:38 smithi102 ceph-mon[36748]: pgmap v649: 129 pgs: 129 active+clean; 36 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 302 op/s 2024-03-21T11:31:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:38 smithi047 ceph-mon[31351]: pgmap v649: 129 pgs: 129 active+clean; 36 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 302 op/s 2024-03-21T11:31:38.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:38 smithi100 ceph-mon[36245]: pgmap v649: 129 pgs: 129 active+clean; 36 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 302 op/s 2024-03-21T11:31:39.538 INFO:tasks.workunit.client.0.smithi047.stdout: 1 591045 8.18 MB/sec execute 263 sec latency 13.736 ms 2024-03-21T11:31:40.538 INFO:tasks.workunit.client.0.smithi047.stdout: 1 592922 8.18 MB/sec execute 264 sec latency 13.162 ms 2024-03-21T11:31:40.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:40 smithi102 ceph-mon[36748]: pgmap v650: 129 pgs: 129 active+clean; 36 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-03-21T11:31:40.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:40 smithi047 ceph-mon[31351]: pgmap v650: 129 pgs: 129 active+clean; 36 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-03-21T11:31:40.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:40 smithi100 ceph-mon[36245]: pgmap v650: 129 pgs: 129 active+clean; 36 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-03-21T11:31:41.538 INFO:tasks.workunit.client.0.smithi047.stdout: 1 593882 8.17 MB/sec execute 265 sec latency 13.879 ms 2024-03-21T11:31:42.538 INFO:tasks.workunit.client.0.smithi047.stdout: 1 595589 8.16 MB/sec execute 266 sec latency 80.309 ms 2024-03-21T11:31:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:42 smithi102 ceph-mon[36748]: pgmap v651: 129 pgs: 129 active+clean; 38 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2024-03-21T11:31:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:42 smithi047 ceph-mon[31351]: pgmap v651: 129 pgs: 129 active+clean; 38 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2024-03-21T11:31:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:42 smithi100 ceph-mon[36245]: pgmap v651: 129 pgs: 129 active+clean; 38 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2024-03-21T11:31:43.542 INFO:tasks.workunit.client.0.smithi047.stdout: 1 597136 8.18 MB/sec execute 267 sec latency 12.949 ms 2024-03-21T11:31:44.539 INFO:tasks.workunit.client.0.smithi047.stdout: 1 598690 8.17 MB/sec execute 268 sec latency 21.609 ms 2024-03-21T11:31:44.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:31:44.817 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:31:45.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.061+0000 7f8e87a00640 1 -- 172.21.15.47:0/1151778185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e801038c0 msgr2=0x7f8e80103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.061+0000 7f8e87a00640 1 --2- 172.21.15.47:0/1151778185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e801038c0 0x7f8e80103ce0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009900 tx=0x7f8e6402bc50 comp rx=0 tx=0).stop 2024-03-21T11:31:45.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.061+0000 7f8e87a00640 1 -- 172.21.15.47:0/1151778185 shutdown_connections 2024-03-21T11:31:45.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.061+0000 7f8e87a00640 1 --2- 172.21.15.47:0/1151778185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e80105470 0x7f8e8010bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.061+0000 7f8e87a00640 1 --2- 172.21.15.47:0/1151778185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e80104ab0 0x7f8e80104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.061+0000 7f8e87a00640 1 --2- 172.21.15.47:0/1151778185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e801038c0 0x7f8e80103ce0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.061+0000 7f8e87a00640 1 -- 172.21.15.47:0/1151778185 >> 172.21.15.47:0/1151778185 conn(0x7f8e800fee50 msgr2=0x7f8e80101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:45.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.061+0000 7f8e87a00640 1 -- 172.21.15.47:0/1151778185 shutdown_connections 2024-03-21T11:31:45.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.061+0000 7f8e87a00640 1 -- 172.21.15.47:0/1151778185 wait complete. 2024-03-21T11:31:45.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.062+0000 7f8e87a00640 1 Processor -- start 2024-03-21T11:31:45.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.062+0000 7f8e87a00640 1 -- start start 2024-03-21T11:31:45.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e87a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e801038c0 0x7f8e80199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e87a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e80104ab0 0x7f8e8019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e87a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e80105470 0x7f8e801a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e87a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8010f510 con 0x7f8e80104ab0 2024-03-21T11:31:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e87a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8010f390 con 0x7f8e801038c0 2024-03-21T11:31:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e87a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8010f690 con 0x7f8e80105470 2024-03-21T11:31:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e85000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e801038c0 0x7f8e80199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e85000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e801038c0 0x7f8e80199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58326/0 (socket says 172.21.15.47:58326) 2024-03-21T11:31:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e85000640 1 -- 172.21.15.47:0/4285649536 learned_addr learned my addr 172.21.15.47:0/4285649536 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e85a00640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e80105470 0x7f8e801a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e77e00640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e80104ab0 0x7f8e8019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e85000640 1 -- 172.21.15.47:0/4285649536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e80105470 msgr2=0x7f8e801a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e85000640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e80105470 0x7f8e801a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e85000640 1 -- 172.21.15.47:0/4285649536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e80104ab0 msgr2=0x7f8e8019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.063+0000 7f8e85000640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e80104ab0 0x7f8e8019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.064+0000 7f8e85000640 1 -- 172.21.15.47:0/4285649536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e801a1880 con 0x7f8e801038c0 2024-03-21T11:31:45.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.064+0000 7f8e85a00640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e80105470 0x7f8e801a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:45.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.064+0000 7f8e85000640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e801038c0 0x7f8e80199c70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8e64000c00 tx=0x7f8e6402c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:45.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.064+0000 7f8e7e000640 1 -- 172.21.15.47:0/4285649536 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64042020 con 0x7f8e801038c0 2024-03-21T11:31:45.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.064+0000 7f8e7e000640 1 -- 172.21.15.47:0/4285649536 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e64004450 con 0x7f8e801038c0 2024-03-21T11:31:45.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.064+0000 7f8e87a00640 1 -- 172.21.15.47:0/4285649536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e801a1b10 con 0x7f8e801038c0 2024-03-21T11:31:45.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.064+0000 7f8e7e000640 1 -- 172.21.15.47:0/4285649536 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6403e460 con 0x7f8e801038c0 2024-03-21T11:31:45.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.064+0000 7f8e87a00640 1 -- 172.21.15.47:0/4285649536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e801a2060 con 0x7f8e801038c0 2024-03-21T11:31:45.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.067+0000 7f8e8a7e5740 1 Processor -- start 2024-03-21T11:31:45.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.067+0000 7f8e7e000640 1 -- 172.21.15.47:0/4285649536 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8e64045020 con 0x7f8e801038c0 2024-03-21T11:31:45.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.068+0000 7f8e8a7e5740 1 -- start start 2024-03-21T11:31:45.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.068+0000 7f8e8a7e5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 0x55ce8f2f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.068+0000 7f8e8a7e5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f2fc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.068+0000 7f8e8a7e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f2fee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.068+0000 7f8e8a7e5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8f25c250 con 0x55ce8f2f7870 2024-03-21T11:31:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.068+0000 7f8e8a7e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8f204a30 con 0x55ce8f2fc9e0 2024-03-21T11:31:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.068+0000 7f8e8a7e5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8f2582f0 con 0x55ce8f2f6e80 2024-03-21T11:31:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.068+0000 7f8e85a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 0x55ce8f2f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.068+0000 7f8e85a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 0x55ce8f2f72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44522/0 (socket says 172.21.15.47:44522) 2024-03-21T11:31:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.068+0000 7f8e85a00640 1 -- 172.21.15.47:0/646641753 learned_addr learned my addr 172.21.15.47:0/646641753 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.068+0000 7f8e85000640 1 --2- 172.21.15.47:0/646641753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f2fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.068+0000 7f8e77e00640 1 --2- 172.21.15.47:0/646641753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f2fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.069+0000 7f8e85a00640 1 -- 172.21.15.47:0/646641753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 msgr2=0x55ce8f2fee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.069+0000 7f8e85a00640 1 --2- 172.21.15.47:0/646641753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f2fee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.069+0000 7f8e85a00640 1 -- 172.21.15.47:0/646641753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 msgr2=0x55ce8f2fc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.069+0000 7f8e85a00640 1 --2- 172.21.15.47:0/646641753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f2fc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.069+0000 7f8e85a00640 1 -- 172.21.15.47:0/646641753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce8f1ef690 con 0x55ce8f2f6e80 2024-03-21T11:31:45.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.069+0000 7f8e85000640 1 --2- 172.21.15.47:0/646641753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f2fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:45.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.069+0000 7f8e85a00640 1 --2- 172.21.15.47:0/646641753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 0x55ce8f2f72a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8e780117e0 tx=0x7f8e78011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83774f200dfc91a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:45.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.069+0000 7f8e76000640 1 -- 172.21.15.47:0/646641753 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78011310 con 0x55ce8f2f6e80 2024-03-21T11:31:45.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.069+0000 7f8e76000640 1 -- 172.21.15.47:0/646641753 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e78003e50 con 0x55ce8f2f6e80 2024-03-21T11:31:45.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.069+0000 7f8e76000640 1 -- 172.21.15.47:0/646641753 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e780045c0 con 0x55ce8f2f6e80 2024-03-21T11:31:45.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.069+0000 7f8e7e000640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.070+0000 7f8e7e000640 1 -- 172.21.15.47:0/4285649536 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8e640f1780 con 0x7f8e801038c0 2024-03-21T11:31:45.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.070+0000 7f8e85000640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.070+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/646641753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 msgr2=0x55ce8f2f72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.070+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/646641753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 0x55ce8f2f72a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8e780117e0 tx=0x7f8e78011ca0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.070+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/646641753 shutdown_connections 2024-03-21T11:31:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.070+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/646641753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f2fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.070+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/646641753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f2fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.070+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/646641753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 0x55ce8f2f72a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.070+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/646641753 >> 172.21.15.47:0/646641753 conn(0x55ce8f2bf200 msgr2=0x55ce8f2f27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.070+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/646641753 shutdown_connections 2024-03-21T11:31:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.070+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/646641753 wait complete. 2024-03-21T11:31:45.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.070+0000 7f8e85000640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005ab0 tx=0x7f8e6402bc50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.071+0000 7f8e8a7e5740 1 -- start start 2024-03-21T11:31:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.071+0000 7f8e8a7e5740 1 Processor -- start 2024-03-21T11:31:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.071+0000 7f8e8a7e5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 0x55ce8f300e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.071+0000 7f8e8a7e5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f301390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.071+0000 7f8e8a7e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f3083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e8a7e5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8f2ff990 con 0x55ce8f2f7870 2024-03-21T11:31:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e8a7e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8f217e80 con 0x55ce8f2fc9e0 2024-03-21T11:31:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e8a7e5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8f2ffb10 con 0x55ce8f2f6e80 2024-03-21T11:31:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e8a7e5740 10 client.0 _ll_register_callbacks cb 0x55ce8f22a130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e85a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 0x55ce8f300e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e85a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f3083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e77e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f301390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e77e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f301390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45270/0 (socket says 172.21.15.47:45270) 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e77e00640 1 -- 172.21.15.47:0/1308065472 learned_addr learned my addr 172.21.15.47:0/1308065472 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e85a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f3083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58344/0 (socket says 172.21.15.47:58344) 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e77e00640 1 -- 172.21.15.47:0/1308065472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 msgr2=0x55ce8f300e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e77e00640 1 --2- 172.21.15.47:0/1308065472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 0x55ce8f300e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e77e00640 1 -- 172.21.15.47:0/1308065472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 msgr2=0x55ce8f3083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e77e00640 1 --2- 172.21.15.47:0/1308065472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f3083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e77e00640 1 -- 172.21.15.47:0/1308065472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce8f03a7d0 con 0x55ce8f2f7870 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.072+0000 7f8e85a00640 1 --2- 172.21.15.47:0/1308065472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f3083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.073+0000 7f8e77e00640 1 --2- 172.21.15.47:0/1308065472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f301390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0126e0 tx=0x7f8e6c012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da875c9b6f87c818 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.073+0000 7f8e72e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.073+0000 7f8e72e00640 1 -- 172.21.15.47:0/1308065472 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c012190 con 0x55ce8f2f7870 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.073+0000 7f8e72e00640 1 -- 172.21.15.47:0/1308065472 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8e6c00f8c0 con 0x55ce8f2f7870 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.073+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/1308065472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e78003e50 con 0x55ce8f2f7870 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.073+0000 7f8e72e00640 1 -- 172.21.15.47:0/1308065472 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e6c00fac0 con 0x55ce8f2f7870 2024-03-21T11:31:45.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.073+0000 7f8e8a7e5740 10 client.17937 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.073+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/1308065472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ce8ef703f0 con 0x55ce8f2f7870 2024-03-21T11:31:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.074+0000 7f8e72e00640 1 -- 172.21.15.47:0/1308065472 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e78003e50 con 0x55ce8f2f7870 2024-03-21T11:31:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.074+0000 7f8e72e00640 1 -- 172.21.15.47:0/1308065472 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8e6c01c420 con 0x55ce8f2f7870 2024-03-21T11:31:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.074+0000 7f8e8a7e5740 10 client.17937 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.074+0000 7f8e8a7e5740 10 client.17937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.074+0000 7f8e8a7e5740 20 client.17937 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.074+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/1308065472 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ce8f30cdb0 0x55ce8f30f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.074+0000 7f8e8a7e5740 4 client.17937 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.074+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/1308065472 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce8f269790 con 0x55ce8f30cdb0 2024-03-21T11:31:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.074+0000 7f8e85a00640 1 --2- 172.21.15.47:0/1308065472 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ce8f30cdb0 0x55ce8f30f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.075+0000 7f8e85a00640 1 --2- 172.21.15.47:0/1308065472 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ce8f30cdb0 0x55ce8f30f1f0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70cbf297f819a3e2 server_cookie=1641d7d7207bc1a1 in_seq=0 out_seq=0 2024-03-21T11:31:45.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.075+0000 7f8e72e00640 10 client.17937 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:45.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.076+0000 7f8e72e00640 1 -- 172.21.15.47:0/1308065472 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ce8f269790 con 0x55ce8f30cdb0 2024-03-21T11:31:45.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.076+0000 7f8e72e00640 10 client.17937 handle_command_reply: tid=0 2024-03-21T11:31:45.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.076+0000 7f8e8a7e5740 1 client.17937 shutdown 2024-03-21T11:31:45.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.076+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/1308065472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 msgr2=0x55ce8f301390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.076+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/1308065472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f301390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0126e0 tx=0x7f8e6c012ba0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.076+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/1308065472 shutdown_connections 2024-03-21T11:31:45.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.076+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/1308065472 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ce8f30cdb0 0x55ce8f30f1f0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.076+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/1308065472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f3083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.076+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/1308065472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f301390 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.076+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/1308065472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 0x55ce8f300e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.076+0000 7f8e72e00640 0 client.17937 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:45.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.076+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/1308065472 >> 172.21.15.47:0/1308065472 conn(0x55ce8f2bf200 msgr2=0x55ce8f2f1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:45.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.077+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/1308065472 shutdown_connections 2024-03-21T11:31:45.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.077+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/1308065472 wait complete. 2024-03-21T11:31:45.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.077+0000 7f8e8a7e5740 20 client.17937 trim_cache size 0 max 16384 2024-03-21T11:31:45.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.097+0000 7f8e8a7e5740 1 Processor -- start 2024-03-21T11:31:45.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.097+0000 7f8e8a7e5740 1 -- start start 2024-03-21T11:31:45.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.098+0000 7f8e8a7e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2f6e80 0x55ce8f2f1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.098+0000 7f8e8a7e5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f7870 0x55ce8f308390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.098+0000 7f8e8a7e5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f2f1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.098+0000 7f8e8a7e5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801ac330 con 0x55ce8f2fc9e0 2024-03-21T11:31:45.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.098+0000 7f8e8a7e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801adcb0 con 0x55ce8f2f6e80 2024-03-21T11:31:45.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.098+0000 7f8e8a7e5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801ac990 con 0x55ce8f2f7870 2024-03-21T11:31:45.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.098+0000 7f8e85a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2f6e80 0x55ce8f2f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.098+0000 7f8e85a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2f6e80 0x55ce8f2f1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58346/0 (socket says 172.21.15.47:58346) 2024-03-21T11:31:45.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.098+0000 7f8e85a00640 1 -- 172.21.15.47:0/1886430324 learned_addr learned my addr 172.21.15.47:0/1886430324 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:45.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.098+0000 7f8e77e00640 1 --2- 172.21.15.47:0/1886430324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f7870 0x55ce8f308390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.098+0000 7f8e85a00640 1 --2- 172.21.15.47:0/1886430324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f2f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.099+0000 7f8e85a00640 1 -- 172.21.15.47:0/1886430324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f7870 msgr2=0x55ce8f308390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.099+0000 7f8e85a00640 1 --2- 172.21.15.47:0/1886430324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f7870 0x55ce8f308390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.099+0000 7f8e85a00640 1 -- 172.21.15.47:0/1886430324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2fc9e0 msgr2=0x55ce8f2f1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.099+0000 7f8e85a00640 1 --2- 172.21.15.47:0/1886430324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f2f1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.099+0000 7f8e85a00640 1 -- 172.21.15.47:0/1886430324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce8ef300f0 con 0x55ce8f2f6e80 2024-03-21T11:31:45.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.099+0000 7f8e77e00640 1 --2- 172.21.15.47:0/1886430324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f7870 0x55ce8f308390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:45.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.099+0000 7f8e85a00640 1 --2- 172.21.15.47:0/1886430324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2f6e80 0x55ce8f2f1200 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8e780117b0 tx=0x7f8e78009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7331afc0b94e4f32 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:45.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.099+0000 7f8e75600640 1 -- 172.21.15.47:0/1886430324 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78023040 con 0x55ce8f2f6e80 2024-03-21T11:31:45.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.100+0000 7f8e75600640 1 -- 172.21.15.47:0/1886430324 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e7802f2b0 con 0x55ce8f2f6e80 2024-03-21T11:31:45.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.100+0000 7f8e75600640 1 -- 172.21.15.47:0/1886430324 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7802fda0 con 0x55ce8f2f6e80 2024-03-21T11:31:45.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.100+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/1886430324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2f6e80 msgr2=0x55ce8f2f1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.100+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/1886430324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2f6e80 0x55ce8f2f1200 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8e780117b0 tx=0x7f8e78009500 comp rx=0 tx=0).stop 2024-03-21T11:31:45.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.100+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/1886430324 shutdown_connections 2024-03-21T11:31:45.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.100+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/1886430324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f2f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.100+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/1886430324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f7870 0x55ce8f308390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.100+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/1886430324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2f6e80 0x55ce8f2f1200 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.100+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/1886430324 >> 172.21.15.47:0/1886430324 conn(0x55ce8f2bf200 msgr2=0x55ce8f2f8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:45.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.101+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/1886430324 shutdown_connections 2024-03-21T11:31:45.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.101+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/1886430324 wait complete. 2024-03-21T11:31:45.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.101+0000 7f8e8a7e5740 1 -- start start 2024-03-21T11:31:45.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.101+0000 7f8e8a7e5740 1 Processor -- start 2024-03-21T11:31:45.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.102+0000 7f8e8a7e5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 0x55ce8f3060d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.102+0000 7f8e8a7e5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f306610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.102+0000 7f8e8a7e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f3075b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.102+0000 7f8e8a7e5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801ad650 con 0x55ce8f2f7870 2024-03-21T11:31:45.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.102+0000 7f8e8a7e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801acff0 con 0x55ce8f2fc9e0 2024-03-21T11:31:45.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.102+0000 7f8e8a7e5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801aa4f0 con 0x55ce8f2f6e80 2024-03-21T11:31:45.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.102+0000 7f8e8a7e5740 10 client.0 _ll_register_callbacks cb 0x7f8e6c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:45.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.102+0000 7f8e85a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f3075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.102+0000 7f8e77e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f306610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.102+0000 7f8e85a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f3075b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58350/0 (socket says 172.21.15.47:58350) 2024-03-21T11:31:45.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.102+0000 7f8e85a00640 1 -- 172.21.15.47:0/3071805900 learned_addr learned my addr 172.21.15.47:0/3071805900 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:45.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.102+0000 7f8e85a00640 1 --2- 172.21.15.47:0/3071805900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 0x55ce8f3060d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.103+0000 7f8e85a00640 1 -- 172.21.15.47:0/3071805900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 msgr2=0x55ce8f3060d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.103+0000 7f8e85a00640 1 --2- 172.21.15.47:0/3071805900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 0x55ce8f3060d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.103+0000 7f8e85a00640 1 -- 172.21.15.47:0/3071805900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 msgr2=0x55ce8f306610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.103+0000 7f8e85a00640 1 --2- 172.21.15.47:0/3071805900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f306610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.103+0000 7f8e85a00640 1 -- 172.21.15.47:0/3071805900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce8f307ba0 con 0x55ce8f2fc9e0 2024-03-21T11:31:45.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.103+0000 7f8e85a00640 1 --2- 172.21.15.47:0/3071805900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f3075b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8e78014040 tx=0x7f8e78032040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5856cb387316090b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:45.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.103+0000 7f8e72e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:45.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.104+0000 7f8e72e00640 1 -- 172.21.15.47:0/3071805900 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78023040 con 0x55ce8f2fc9e0 2024-03-21T11:31:45.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.104+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/3071805900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e7802f2b0 con 0x55ce8f2fc9e0 2024-03-21T11:31:45.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.104+0000 7f8e72e00640 1 -- 172.21.15.47:0/3071805900 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8e7802fb20 con 0x55ce8f2fc9e0 2024-03-21T11:31:45.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.104+0000 7f8e72e00640 1 -- 172.21.15.47:0/3071805900 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e78047050 con 0x55ce8f2fc9e0 2024-03-21T11:31:45.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.104+0000 7f8e8a7e5740 10 client.28271 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:45.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.104+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/3071805900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ce8f368c60 con 0x55ce8f2fc9e0 2024-03-21T11:31:45.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.104+0000 7f8e72e00640 1 -- 172.21.15.47:0/3071805900 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e7802f2b0 con 0x55ce8f2fc9e0 2024-03-21T11:31:45.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.105+0000 7f8e72e00640 1 -- 172.21.15.47:0/3071805900 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8e7803c040 con 0x55ce8f2fc9e0 2024-03-21T11:31:45.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.105+0000 7f8e8a7e5740 10 client.28271 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:45.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.105+0000 7f8e8a7e5740 10 client.28271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:45.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.105+0000 7f8e8a7e5740 20 client.28271 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.105+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/3071805900 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ce8f3692c0 0x55ce8f36b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.105+0000 7f8e8a7e5740 4 client.28271 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.105+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/3071805900 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ce8f300c40 con 0x55ce8f3692c0 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.105+0000 7f8e77e00640 1 --2- 172.21.15.47:0/3071805900 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ce8f3692c0 0x55ce8f36b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.105+0000 7f8e77e00640 1 --2- 172.21.15.47:0/3071805900 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ce8f3692c0 0x55ce8f36b6e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6177b0876befc83e server_cookie=6b291bdd03261cc in_seq=0 out_seq=0 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.105+0000 7f8e72e00640 10 client.28271 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.106+0000 7f8e72e00640 1 -- 172.21.15.47:0/3071805900 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ce8f300c40 con 0x55ce8f3692c0 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.106+0000 7f8e72e00640 10 client.28271 handle_command_reply: tid=0 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.107+0000 7f8e8a7e5740 1 client.28271 shutdown 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.107+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/3071805900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 msgr2=0x55ce8f3075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.107+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/3071805900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f3075b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8e78014040 tx=0x7f8e78032040 comp rx=0 tx=0).stop 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.107+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/3071805900 shutdown_connections 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.107+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/3071805900 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ce8f3692c0 0x55ce8f36b6e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.107+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/3071805900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ce8f2fc9e0 0x55ce8f3075b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.107+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/3071805900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ce8f2f7870 0x55ce8f306610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.107+0000 7f8e8a7e5740 1 --2- 172.21.15.47:0/3071805900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ce8f2f6e80 0x55ce8f3060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.107+0000 7f8e72e00640 0 client.28271 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:45.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.107+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/3071805900 >> 172.21.15.47:0/3071805900 conn(0x55ce8f2bf200 msgr2=0x55ce8f2f8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:45.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.107+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/3071805900 shutdown_connections 2024-03-21T11:31:45.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.107+0000 7f8e8a7e5740 1 -- 172.21.15.47:0/3071805900 wait complete. 2024-03-21T11:31:45.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.107+0000 7f8e8a7e5740 20 client.28271 trim_cache size 0 max 16384 2024-03-21T11:31:45.109 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:31:45.109 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:31:45.109 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:31:45.110 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:31:45.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.108+0000 7f8e87a00640 1 -- 172.21.15.47:0/4285649536 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8e5c0a2f20 msgr2=0x7f8e5c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.108+0000 7f8e87a00640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005ab0 tx=0x7f8e6402bc50 comp rx=0 tx=0).stop 2024-03-21T11:31:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.108+0000 7f8e87a00640 1 -- 172.21.15.47:0/4285649536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e801038c0 msgr2=0x7f8e80199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.108+0000 7f8e87a00640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e801038c0 0x7f8e80199c70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8e64000c00 tx=0x7f8e6402c660 comp rx=0 tx=0).stop 2024-03-21T11:31:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.108+0000 7f8e87a00640 1 -- 172.21.15.47:0/4285649536 shutdown_connections 2024-03-21T11:31:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.108+0000 7f8e87a00640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.108+0000 7f8e87a00640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e80105470 0x7f8e801a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.108+0000 7f8e87a00640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e80104ab0 0x7f8e8019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.108+0000 7f8e87a00640 1 --2- 172.21.15.47:0/4285649536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e801038c0 0x7f8e80199c70 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.108+0000 7f8e87a00640 1 -- 172.21.15.47:0/4285649536 >> 172.21.15.47:0/4285649536 conn(0x7f8e800fee50 msgr2=0x7f8e80107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.108+0000 7f8e87a00640 1 -- 172.21.15.47:0/4285649536 shutdown_connections 2024-03-21T11:31:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.108+0000 7f8e87a00640 1 -- 172.21.15.47:0/4285649536 wait complete. 2024-03-21T11:31:45.122 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:44 smithi047 ceph-mon[31351]: pgmap v652: 129 pgs: 129 active+clean; 40 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-03-21T11:31:45.125 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:31:45.126 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:31:45.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:44 smithi100 ceph-mon[36245]: pgmap v652: 129 pgs: 129 active+clean; 40 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-03-21T11:31:45.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:44 smithi102 ceph-mon[36748]: pgmap v652: 129 pgs: 129 active+clean; 40 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-03-21T11:31:45.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.344+0000 7f819b200640 1 -- 172.21.15.47:0/4097036641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c105070 msgr2=0x7f819c1054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.344+0000 7f819b200640 1 --2- 172.21.15.47:0/4097036641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c105070 0x7f819c1054f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8194009900 tx=0x7f819402bc40 comp rx=0 tx=0).stop 2024-03-21T11:31:45.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.344+0000 7f819b200640 1 -- 172.21.15.47:0/4097036641 shutdown_connections 2024-03-21T11:31:45.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.344+0000 7f819b200640 1 --2- 172.21.15.47:0/4097036641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c105070 0x7f819c1054f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.344+0000 7f819b200640 1 --2- 172.21.15.47:0/4097036641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f819c1026c0 0x7f819c104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.344+0000 7f819b200640 1 --2- 172.21.15.47:0/4097036641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f819c0ffd50 0x7f819c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.344+0000 7f819b200640 1 -- 172.21.15.47:0/4097036641 >> 172.21.15.47:0/4097036641 conn(0x7f819c0fb950 msgr2=0x7f819c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:45.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.344+0000 7f819b200640 1 -- 172.21.15.47:0/4097036641 shutdown_connections 2024-03-21T11:31:45.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.344+0000 7f819b200640 1 -- 172.21.15.47:0/4097036641 wait complete. 2024-03-21T11:31:45.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.345+0000 7f819b200640 1 Processor -- start 2024-03-21T11:31:45.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.346+0000 7f819b200640 1 -- start start 2024-03-21T11:31:45.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.346+0000 7f819b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f819c0ffd50 0x7f819c19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.346+0000 7f819b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c1026c0 0x7f819c19e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.346+0000 7f819b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f819c105070 0x7f819c1a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.346+0000 7f819b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c113730 con 0x7f819c1026c0 2024-03-21T11:31:45.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.346+0000 7f819b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c1135b0 con 0x7f819c0ffd50 2024-03-21T11:31:45.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.346+0000 7f819b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c1138b0 con 0x7f819c105070 2024-03-21T11:31:45.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.346+0000 7f8199e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f819c0ffd50 0x7f819c19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.346+0000 7f819a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f819c105070 0x7f819c1a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.346+0000 7f8199400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c1026c0 0x7f819c19e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.346+0000 7f819a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f819c105070 0x7f819c1a52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44576/0 (socket says 172.21.15.47:44576) 2024-03-21T11:31:45.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.346+0000 7f819a800640 1 -- 172.21.15.47:0/1886402101 learned_addr learned my addr 172.21.15.47:0/1886402101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:45.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.346+0000 7f8199e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f819c0ffd50 0x7f819c19dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58370/0 (socket says 172.21.15.47:58370) 2024-03-21T11:31:45.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.347+0000 7f819a800640 1 -- 172.21.15.47:0/1886402101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f819c0ffd50 msgr2=0x7f819c19dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.347+0000 7f819a800640 1 --2- 172.21.15.47:0/1886402101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f819c0ffd50 0x7f819c19dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.347+0000 7f819a800640 1 -- 172.21.15.47:0/1886402101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c1026c0 msgr2=0x7f819c19e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.347+0000 7f819a800640 1 --2- 172.21.15.47:0/1886402101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c1026c0 0x7f819c19e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.347+0000 7f819a800640 1 -- 172.21.15.47:0/1886402101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c1a5960 con 0x7f819c105070 2024-03-21T11:31:45.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.347+0000 7f8199e00640 1 --2- 172.21.15.47:0/1886402101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f819c0ffd50 0x7f819c19dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:45.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.347+0000 7f8199400640 1 --2- 172.21.15.47:0/1886402101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c1026c0 0x7f819c19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:45.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.347+0000 7f819a800640 1 --2- 172.21.15.47:0/1886402101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f819c105070 0x7f819c1a52c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8194005ab0 tx=0x7f819402c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:45.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.348+0000 7f8192a00640 1 -- 172.21.15.47:0/1886402101 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194041070 con 0x7f819c105070 2024-03-21T11:31:45.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.348+0000 7f8192a00640 1 -- 172.21.15.47:0/1886402101 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8194004450 con 0x7f819c105070 2024-03-21T11:31:45.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.348+0000 7f8192a00640 1 -- 172.21.15.47:0/1886402101 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194003e40 con 0x7f819c105070 2024-03-21T11:31:45.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.348+0000 7f819b200640 1 -- 172.21.15.47:0/1886402101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819c1a5bf0 con 0x7f819c105070 2024-03-21T11:31:45.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.348+0000 7f819b200640 1 -- 172.21.15.47:0/1886402101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819c1a60e0 con 0x7f819c105070 2024-03-21T11:31:45.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.349+0000 7f819b200640 1 -- 172.21.15.47:0/1886402101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819c0fd440 con 0x7f819c105070 2024-03-21T11:31:45.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.351+0000 7f8192a00640 1 -- 172.21.15.47:0/1886402101 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8194044070 con 0x7f819c105070 2024-03-21T11:31:45.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.352+0000 7f8192a00640 1 --2- 172.21.15.47:0/1886402101 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81800a2f20 0x7f81800a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.353+0000 7f8192a00640 1 -- 172.21.15.47:0/1886402101 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f81940f1810 con 0x7f819c105070 2024-03-21T11:31:45.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.353+0000 7f8199e00640 1 --2- 172.21.15.47:0/1886402101 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81800a2f20 0x7f81800a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.353+0000 7f8199e00640 1 --2- 172.21.15.47:0/1886402101 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81800a2f20 0x7f81800a53e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f819c19f220 tx=0x7f818c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:45.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.359+0000 7f8192a00640 1 -- 172.21.15.47:0/1886402101 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f81940bdc00 con 0x7f819c105070 2024-03-21T11:31:45.539 INFO:tasks.workunit.client.0.smithi047.stdout: 1 600583 8.19 MB/sec execute 269 sec latency 13.805 ms 2024-03-21T11:31:45.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.613+0000 7f819b200640 1 -- 172.21.15.47:0/1886402101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f819c02d190 con 0x7f819c105070 2024-03-21T11:31:45.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.614+0000 7f8192a00640 1 -- 172.21.15.47:0/1886402101 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f81940c1ca0 con 0x7f819c105070 2024-03-21T11:31:45.614 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:31:45.615 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:31:45.615 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:31:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.617+0000 7f819b200640 1 -- 172.21.15.47:0/1886402101 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81800a2f20 msgr2=0x7f81800a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.617+0000 7f819b200640 1 --2- 172.21.15.47:0/1886402101 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81800a2f20 0x7f81800a53e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f819c19f220 tx=0x7f818c009500 comp rx=0 tx=0).stop 2024-03-21T11:31:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.617+0000 7f819b200640 1 -- 172.21.15.47:0/1886402101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f819c105070 msgr2=0x7f819c1a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.617+0000 7f819b200640 1 --2- 172.21.15.47:0/1886402101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f819c105070 0x7f819c1a52c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8194005ab0 tx=0x7f819402c6d0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.618+0000 7f819b200640 1 -- 172.21.15.47:0/1886402101 shutdown_connections 2024-03-21T11:31:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.618+0000 7f819b200640 1 --2- 172.21.15.47:0/1886402101 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f81800a2f20 0x7f81800a53e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.618+0000 7f819b200640 1 --2- 172.21.15.47:0/1886402101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f819c105070 0x7f819c1a52c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.618+0000 7f819b200640 1 --2- 172.21.15.47:0/1886402101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f819c1026c0 0x7f819c19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.618+0000 7f819b200640 1 --2- 172.21.15.47:0/1886402101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f819c0ffd50 0x7f819c19dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.618+0000 7f819b200640 1 -- 172.21.15.47:0/1886402101 >> 172.21.15.47:0/1886402101 conn(0x7f819c0fb950 msgr2=0x7f819c1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.618+0000 7f819b200640 1 -- 172.21.15.47:0/1886402101 shutdown_connections 2024-03-21T11:31:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.618+0000 7f819b200640 1 -- 172.21.15.47:0/1886402101 wait complete. 2024-03-21T11:31:45.638 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:31:45.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:31:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.883+0000 7fa64d200640 1 -- 172.21.15.47:0/3127742638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa648109000 msgr2=0x7fa64810f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.883+0000 7fa64d200640 1 --2- 172.21.15.47:0/3127742638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa648109000 0x7fa64810f9e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa630009900 tx=0x7fa63002bc40 comp rx=0 tx=0).stop 2024-03-21T11:31:45.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.884+0000 7fa64d200640 1 -- 172.21.15.47:0/3127742638 shutdown_connections 2024-03-21T11:31:45.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.884+0000 7fa64d200640 1 --2- 172.21.15.47:0/3127742638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64810ff20 0x7fa648112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.884+0000 7fa64d200640 1 --2- 172.21.15.47:0/3127742638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa648109000 0x7fa64810f9e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.884+0000 7fa64d200640 1 --2- 172.21.15.47:0/3127742638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6481086a0 0x7fa648108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.884+0000 7fa64d200640 1 -- 172.21.15.47:0/3127742638 >> 172.21.15.47:0/3127742638 conn(0x7fa64806e500 msgr2=0x7fa648070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:45.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.884+0000 7fa64d200640 1 -- 172.21.15.47:0/3127742638 shutdown_connections 2024-03-21T11:31:45.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.885+0000 7fa64d200640 1 -- 172.21.15.47:0/3127742638 wait complete. 2024-03-21T11:31:45.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.886+0000 7fa64d200640 1 Processor -- start 2024-03-21T11:31:45.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.887+0000 7fa64d200640 1 -- start start 2024-03-21T11:31:45.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.887+0000 7fa64d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6481086a0 0x7fa6481a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.887+0000 7fa64d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa648109000 0x7fa6481a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.887+0000 7fa64d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64810ff20 0x7fa6481ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.888+0000 7fa64d200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa648117d90 con 0x7fa648109000 2024-03-21T11:31:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.888+0000 7fa64d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa648117c10 con 0x7fa6481086a0 2024-03-21T11:31:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.888+0000 7fa64d200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa648117f10 con 0x7fa64810ff20 2024-03-21T11:31:45.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.888+0000 7fa646800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64810ff20 0x7fa6481ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.888+0000 7fa645e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6481086a0 0x7fa6481a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.888+0000 7fa645e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6481086a0 0x7fa6481a6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58386/0 (socket says 172.21.15.47:58386) 2024-03-21T11:31:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.888+0000 7fa646800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64810ff20 0x7fa6481ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44582/0 (socket says 172.21.15.47:44582) 2024-03-21T11:31:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.888+0000 7fa645e00640 1 -- 172.21.15.47:0/1552782809 learned_addr learned my addr 172.21.15.47:0/1552782809 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.888+0000 7fa63fe00640 1 --2- 172.21.15.47:0/1552782809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa648109000 0x7fa6481a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.889+0000 7fa645e00640 1 -- 172.21.15.47:0/1552782809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64810ff20 msgr2=0x7fa6481ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.889+0000 7fa645e00640 1 --2- 172.21.15.47:0/1552782809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64810ff20 0x7fa6481ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.889+0000 7fa645e00640 1 -- 172.21.15.47:0/1552782809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa648109000 msgr2=0x7fa6481a6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.889+0000 7fa645e00640 1 --2- 172.21.15.47:0/1552782809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa648109000 0x7fa6481a6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.889+0000 7fa645e00640 1 -- 172.21.15.47:0/1552782809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6481ae4a0 con 0x7fa6481086a0 2024-03-21T11:31:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.889+0000 7fa646800640 1 --2- 172.21.15.47:0/1552782809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64810ff20 0x7fa6481ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.889+0000 7fa645e00640 1 --2- 172.21.15.47:0/1552782809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6481086a0 0x7fa6481a6880 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa63800e9f0 tx=0x7fa63800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.890+0000 7fa63ea00640 1 -- 172.21.15.47:0/1552782809 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63800cd80 con 0x7fa6481086a0 2024-03-21T11:31:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.890+0000 7fa63ea00640 1 -- 172.21.15.47:0/1552782809 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa638010910 con 0x7fa6481086a0 2024-03-21T11:31:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.890+0000 7fa64d200640 1 -- 172.21.15.47:0/1552782809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6481ae790 con 0x7fa6481086a0 2024-03-21T11:31:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.890+0000 7fa63ea00640 1 -- 172.21.15.47:0/1552782809 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638018990 con 0x7fa6481086a0 2024-03-21T11:31:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.891+0000 7fa64d200640 1 -- 172.21.15.47:0/1552782809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6481aed00 con 0x7fa6481086a0 2024-03-21T11:31:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.896+0000 7fa65000d740 1 Processor -- start 2024-03-21T11:31:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.896+0000 7fa63ea00640 1 -- 172.21.15.47:0/1552782809 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa638018b30 con 0x7fa6481086a0 2024-03-21T11:31:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.896+0000 7fa65000d740 1 -- start start 2024-03-21T11:31:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.897+0000 7fa65000d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914c810 0x5605c914cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.897+0000 7fa65000d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914d9e0 0x5605c914de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.897+0000 7fa65000d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c9154c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.897+0000 7fa65000d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605c90b12c0 con 0x5605c914c810 2024-03-21T11:31:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.897+0000 7fa65000d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605c9059c90 con 0x5605c914d9e0 2024-03-21T11:31:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.897+0000 7fa65000d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605c90ad360 con 0x5605c914e3a0 2024-03-21T11:31:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.897+0000 7fa645e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914d9e0 0x5605c914de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.897+0000 7fa645e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914d9e0 0x5605c914de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58398/0 (socket says 172.21.15.47:58398) 2024-03-21T11:31:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.897+0000 7fa645e00640 1 -- 172.21.15.47:0/1120970461 learned_addr learned my addr 172.21.15.47:0/1120970461 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.898+0000 7fa646800640 1 --2- 172.21.15.47:0/1120970461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914c810 0x5605c914cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.898+0000 7fa63fe00640 1 --2- 172.21.15.47:0/1120970461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c9154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.899+0000 7fa645e00640 1 -- 172.21.15.47:0/1120970461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 msgr2=0x5605c9154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.900+0000 7fa63ea00640 1 --2- 172.21.15.47:0/1552782809 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa62c0a3020 0x7fa62c0a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.900+0000 7fa63ea00640 1 -- 172.21.15.47:0/1552782809 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa6380cfb80 con 0x7fa6481086a0 2024-03-21T11:31:45.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.901+0000 7fa645e00640 1 --2- 172.21.15.47:0/1120970461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c9154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.901+0000 7fa645e00640 1 -- 172.21.15.47:0/1120970461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914c810 msgr2=0x5605c914cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.901+0000 7fa645e00640 1 --2- 172.21.15.47:0/1120970461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914c810 0x5605c914cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.901+0000 7fa645e00640 1 -- 172.21.15.47:0/1120970461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605c907e080 con 0x5605c914d9e0 2024-03-21T11:31:45.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.901+0000 7fa646800640 1 --2- 172.21.15.47:0/1120970461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914c810 0x5605c914cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:45.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.902+0000 7fa645e00640 1 --2- 172.21.15.47:0/1120970461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914d9e0 0x5605c914de60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa638098890 tx=0x7fa638018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6ba87dae573b2a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:45.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.902+0000 7fa646800640 1 --2- 172.21.15.47:0/1552782809 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa62c0a3020 0x7fa62c0a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.904+0000 7fa637400640 1 -- 172.21.15.47:0/1120970461 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6380d8070 con 0x5605c914d9e0 2024-03-21T11:31:45.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.904+0000 7fa637400640 1 -- 172.21.15.47:0/1120970461 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6380d9040 con 0x5605c914d9e0 2024-03-21T11:31:45.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.904+0000 7fa637400640 1 -- 172.21.15.47:0/1120970461 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63809bec0 con 0x5605c914d9e0 2024-03-21T11:31:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.905+0000 7fa65000d740 1 -- 172.21.15.47:0/1120970461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914d9e0 msgr2=0x5605c914de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.905+0000 7fa65000d740 1 --2- 172.21.15.47:0/1120970461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914d9e0 0x5605c914de60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa638098890 tx=0x7fa638018420 comp rx=0 tx=0).stop 2024-03-21T11:31:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.906+0000 7fa646800640 1 --2- 172.21.15.47:0/1552782809 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa62c0a3020 0x7fa62c0a54e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x5605c914d8e0 tx=0x7fa640004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.906+0000 7fa65000d740 1 -- 172.21.15.47:0/1120970461 shutdown_connections 2024-03-21T11:31:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.906+0000 7fa65000d740 1 --2- 172.21.15.47:0/1120970461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c9154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.907+0000 7fa65000d740 1 --2- 172.21.15.47:0/1120970461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914d9e0 0x5605c914de60 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.907+0000 7fa65000d740 1 --2- 172.21.15.47:0/1120970461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914c810 0x5605c914cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.907+0000 7fa65000d740 1 -- 172.21.15.47:0/1120970461 >> 172.21.15.47:0/1120970461 conn(0x5605c9147c00 msgr2=0x5605c9148000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.907+0000 7fa65000d740 1 -- 172.21.15.47:0/1120970461 shutdown_connections 2024-03-21T11:31:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.908+0000 7fa65000d740 1 -- 172.21.15.47:0/1120970461 wait complete. 2024-03-21T11:31:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.908+0000 7fa65000d740 1 -- start start 2024-03-21T11:31:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.908+0000 7fa65000d740 1 Processor -- start 2024-03-21T11:31:45.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.910+0000 7fa65000d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914c810 0x5605c9156a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.910+0000 7fa65000d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914d9e0 0x5605c9156f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.911+0000 7fa65000d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c915dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.911+0000 7fa65000d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605c9155350 con 0x5605c914d9e0 2024-03-21T11:31:45.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.911+0000 7fa65000d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605c906d0d0 con 0x5605c914c810 2024-03-21T11:31:45.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.911+0000 7fa65000d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605c91554d0 con 0x5605c914e3a0 2024-03-21T11:31:45.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.911+0000 7fa65000d740 10 client.0 _ll_register_callbacks cb 0x5605c9071e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:45.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.911+0000 7fa645e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914c810 0x5605c9156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.911+0000 7fa645e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914c810 0x5605c9156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58402/0 (socket says 172.21.15.47:58402) 2024-03-21T11:31:45.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.911+0000 7fa645e00640 1 -- 172.21.15.47:0/1667618588 learned_addr learned my addr 172.21.15.47:0/1667618588 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:45.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.911+0000 7fa646800640 1 --2- 172.21.15.47:0/1667618588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c915dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.912+0000 7fa63fe00640 1 --2- 172.21.15.47:0/1667618588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914d9e0 0x5605c9156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.912+0000 7fa646800640 1 -- 172.21.15.47:0/1667618588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914c810 msgr2=0x5605c9156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.912+0000 7fa646800640 1 --2- 172.21.15.47:0/1667618588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914c810 0x5605c9156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.912+0000 7fa646800640 1 -- 172.21.15.47:0/1667618588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914d9e0 msgr2=0x5605c9156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.912+0000 7fa646800640 1 --2- 172.21.15.47:0/1667618588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914d9e0 0x5605c9156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.912+0000 7fa646800640 1 -- 172.21.15.47:0/1667618588 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605c8dc53f0 con 0x5605c914e3a0 2024-03-21T11:31:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.912+0000 7fa645e00640 1 --2- 172.21.15.47:0/1667618588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914c810 0x5605c9156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.912+0000 7fa646800640 1 --2- 172.21.15.47:0/1667618588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c915dfa0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa64806b390 tx=0x7fa640004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f437f38b39e6a34 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.912+0000 7fa62be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.913+0000 7fa62be00640 1 -- 172.21.15.47:0/1667618588 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64001b8e0 con 0x5605c914e3a0 2024-03-21T11:31:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.913+0000 7fa62be00640 1 -- 172.21.15.47:0/1667618588 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa64001ba80 con 0x5605c914e3a0 2024-03-21T11:31:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.913+0000 7fa65000d740 1 -- 172.21.15.47:0/1667618588 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6380d9040 con 0x5605c914e3a0 2024-03-21T11:31:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.913+0000 7fa62be00640 1 -- 172.21.15.47:0/1667618588 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa64001bc80 con 0x5605c914e3a0 2024-03-21T11:31:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.913+0000 7fa65000d740 10 client.28291 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.913+0000 7fa65000d740 1 -- 172.21.15.47:0/1667618588 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5605c8e8f7d0 con 0x5605c914e3a0 2024-03-21T11:31:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.914+0000 7fa62be00640 1 -- 172.21.15.47:0/1667618588 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6380d9040 con 0x5605c914e3a0 2024-03-21T11:31:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.914+0000 7fa62be00640 1 -- 172.21.15.47:0/1667618588 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa640025070 con 0x5605c914e3a0 2024-03-21T11:31:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.914+0000 7fa65000d740 10 client.28291 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.914+0000 7fa65000d740 10 client.28291 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:31:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.914+0000 7fa65000d740 20 client.28291 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.914+0000 7fa65000d740 1 --2- 172.21.15.47:0/1667618588 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5605c91625d0 0x5605c9164a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.914+0000 7fa65000d740 4 client.28291 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.914+0000 7fa65000d740 1 -- 172.21.15.47:0/1667618588 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605c90be800 con 0x5605c91625d0 2024-03-21T11:31:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.915+0000 7fa645e00640 1 --2- 172.21.15.47:0/1667618588 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5605c91625d0 0x5605c9164a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.916+0000 7fa645e00640 1 --2- 172.21.15.47:0/1667618588 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5605c91625d0 0x5605c9164a30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f432591c5a2b245c server_cookie=c54978129fb187b4 in_seq=0 out_seq=0 2024-03-21T11:31:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.916+0000 7fa62be00640 10 client.28291 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:31:45.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.918+0000 7fa62be00640 1 -- 172.21.15.47:0/1667618588 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5605c90be800 con 0x5605c91625d0 2024-03-21T11:31:45.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.918+0000 7fa62be00640 10 client.28291 handle_command_reply: tid=0 2024-03-21T11:31:45.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.918+0000 7fa65000d740 1 client.28291 shutdown 2024-03-21T11:31:45.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.919+0000 7fa65000d740 1 -- 172.21.15.47:0/1667618588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 msgr2=0x5605c915dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.919+0000 7fa65000d740 1 --2- 172.21.15.47:0/1667618588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c915dfa0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa64806b390 tx=0x7fa640004ab0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.920+0000 7fa65000d740 1 -- 172.21.15.47:0/1667618588 shutdown_connections 2024-03-21T11:31:45.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.920+0000 7fa65000d740 1 --2- 172.21.15.47:0/1667618588 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5605c91625d0 0x5605c9164a30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.920+0000 7fa65000d740 1 --2- 172.21.15.47:0/1667618588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c915dfa0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.920+0000 7fa65000d740 1 --2- 172.21.15.47:0/1667618588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914d9e0 0x5605c9156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.920+0000 7fa65000d740 1 --2- 172.21.15.47:0/1667618588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914c810 0x5605c9156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.920+0000 7fa65000d740 1 -- 172.21.15.47:0/1667618588 >> 172.21.15.47:0/1667618588 conn(0x5605c9147c00 msgr2=0x5605c9146eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:45.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.920+0000 7fa62be00640 0 client.28291 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:31:45.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.920+0000 7fa65000d740 1 -- 172.21.15.47:0/1667618588 shutdown_connections 2024-03-21T11:31:45.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.920+0000 7fa65000d740 1 -- 172.21.15.47:0/1667618588 wait complete. 2024-03-21T11:31:45.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.921+0000 7fa65000d740 20 client.28291 trim_cache size 0 max 16384 2024-03-21T11:31:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.943+0000 7fa65000d740 1 Processor -- start 2024-03-21T11:31:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.943+0000 7fa65000d740 1 -- start start 2024-03-21T11:31:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa65000d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914c810 0x5605c9150a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa65000d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914d9e0 0x5605c915df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa65000d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c9150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa65000d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6481b8fc0 con 0x5605c914c810 2024-03-21T11:31:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa65000d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6481ba940 con 0x5605c914d9e0 2024-03-21T11:31:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa65000d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6481b9620 con 0x5605c914e3a0 2024-03-21T11:31:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa646800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c9150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa63fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914d9e0 0x5605c915df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa646800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c9150fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44610/0 (socket says 172.21.15.47:44610) 2024-03-21T11:31:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa646800640 1 -- 172.21.15.47:0/258574073 learned_addr learned my addr 172.21.15.47:0/258574073 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa645e00640 1 --2- 172.21.15.47:0/258574073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914c810 0x5605c9150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa646800640 1 -- 172.21.15.47:0/258574073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914d9e0 msgr2=0x5605c915df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa646800640 1 --2- 172.21.15.47:0/258574073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914d9e0 0x5605c915df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa646800640 1 -- 172.21.15.47:0/258574073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914c810 msgr2=0x5605c9150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa646800640 1 --2- 172.21.15.47:0/258574073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914c810 0x5605c9150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa646800640 1 -- 172.21.15.47:0/258574073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605c9154580 con 0x5605c914e3a0 2024-03-21T11:31:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.944+0000 7fa645e00640 1 --2- 172.21.15.47:0/258574073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914c810 0x5605c9150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.945+0000 7fa646800640 1 --2- 172.21.15.47:0/258574073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c9150fd0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa640010680 tx=0x7fa64000aca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ed25d42ae88b00c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:45.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.945+0000 7fa636a00640 1 -- 172.21.15.47:0/258574073 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64001b9a0 con 0x5605c914e3a0 2024-03-21T11:31:45.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.945+0000 7fa636a00640 1 -- 172.21.15.47:0/258574073 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa640026d50 con 0x5605c914e3a0 2024-03-21T11:31:45.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.945+0000 7fa636a00640 1 -- 172.21.15.47:0/258574073 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640035e60 con 0x5605c914e3a0 2024-03-21T11:31:45.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.946+0000 7fa65000d740 1 -- 172.21.15.47:0/258574073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 msgr2=0x5605c9150fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.946+0000 7fa65000d740 1 --2- 172.21.15.47:0/258574073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c9150fd0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa640010680 tx=0x7fa64000aca0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.946+0000 7fa65000d740 1 -- 172.21.15.47:0/258574073 shutdown_connections 2024-03-21T11:31:45.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.946+0000 7fa65000d740 1 --2- 172.21.15.47:0/258574073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c9150fd0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.946+0000 7fa65000d740 1 --2- 172.21.15.47:0/258574073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914d9e0 0x5605c915df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.946+0000 7fa65000d740 1 --2- 172.21.15.47:0/258574073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914c810 0x5605c9150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.946+0000 7fa65000d740 1 -- 172.21.15.47:0/258574073 >> 172.21.15.47:0/258574073 conn(0x5605c9147c00 msgr2=0x5605c914b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:45.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.946+0000 7fa65000d740 1 -- 172.21.15.47:0/258574073 shutdown_connections 2024-03-21T11:31:45.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.946+0000 7fa65000d740 1 -- 172.21.15.47:0/258574073 wait complete. 2024-03-21T11:31:45.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.947+0000 7fa65000d740 1 -- start start 2024-03-21T11:31:45.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.947+0000 7fa65000d740 1 Processor -- start 2024-03-21T11:31:45.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.947+0000 7fa65000d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914c810 0x5605c9159c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.947+0000 7fa65000d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914d9e0 0x5605c915a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.947+0000 7fa65000d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c915d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.947+0000 7fa65000d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6481ba2e0 con 0x5605c914d9e0 2024-03-21T11:31:45.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.947+0000 7fa65000d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6481b9c80 con 0x5605c914c810 2024-03-21T11:31:45.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.947+0000 7fa65000d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6481b7180 con 0x5605c914e3a0 2024-03-21T11:31:45.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.947+0000 7fa65000d740 10 client.0 _ll_register_callbacks cb 0x7fa640010f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:45.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.947+0000 7fa645e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914c810 0x5605c9159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.947+0000 7fa646800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c915d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.948+0000 7fa63fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914d9e0 0x5605c915a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.948+0000 7fa645e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914c810 0x5605c9159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58416/0 (socket says 172.21.15.47:58416) 2024-03-21T11:31:45.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.948+0000 7fa645e00640 1 -- 172.21.15.47:0/492281420 learned_addr learned my addr 172.21.15.47:0/492281420 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:45.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.948+0000 7fa646800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c915d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44620/0 (socket says 172.21.15.47:44620) 2024-03-21T11:31:45.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.948+0000 7fa646800640 1 -- 172.21.15.47:0/492281420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914c810 msgr2=0x5605c9159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.948+0000 7fa646800640 1 --2- 172.21.15.47:0/492281420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914c810 0x5605c9159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.948+0000 7fa646800640 1 -- 172.21.15.47:0/492281420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914d9e0 msgr2=0x5605c915a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:45.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.948+0000 7fa646800640 1 --2- 172.21.15.47:0/492281420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914d9e0 0x5605c915a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.948+0000 7fa646800640 1 -- 172.21.15.47:0/492281420 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605c915d730 con 0x5605c914e3a0 2024-03-21T11:31:45.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.948+0000 7fa645e00640 1 --2- 172.21.15.47:0/492281420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914c810 0x5605c9159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:45.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.948+0000 7fa646800640 1 --2- 172.21.15.47:0/492281420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c915d140 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa640010650 tx=0x7fa64001e660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1399de05e717957e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:45.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.948+0000 7fa62be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:45.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.948+0000 7fa62be00640 1 -- 172.21.15.47:0/492281420 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640026870 con 0x5605c914e3a0 2024-03-21T11:31:45.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.949+0000 7fa65000d740 1 -- 172.21.15.47:0/492281420 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa640026d50 con 0x5605c914e3a0 2024-03-21T11:31:45.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.949+0000 7fa62be00640 1 -- 172.21.15.47:0/492281420 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa640026a10 con 0x5605c914e3a0 2024-03-21T11:31:45.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.949+0000 7fa62be00640 1 -- 172.21.15.47:0/492281420 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa64004b050 con 0x5605c914e3a0 2024-03-21T11:31:45.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.949+0000 7fa65000d740 10 client.28297 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:45.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.949+0000 7fa65000d740 1 -- 172.21.15.47:0/492281420 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5605c91be380 con 0x5605c914e3a0 2024-03-21T11:31:45.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.949+0000 7fa62be00640 1 -- 172.21.15.47:0/492281420 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa640026d50 con 0x5605c914e3a0 2024-03-21T11:31:45.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.949+0000 7fa62be00640 1 -- 172.21.15.47:0/492281420 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa640025040 con 0x5605c914e3a0 2024-03-21T11:31:45.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.949+0000 7fa65000d740 10 client.28297 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:45.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.949+0000 7fa65000d740 10 client.28297 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:31:45.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.949+0000 7fa65000d740 20 client.28297 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:45.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.949+0000 7fa65000d740 1 --2- 172.21.15.47:0/492281420 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5605c91bec70 0x5605c91c1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:45.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.949+0000 7fa65000d740 4 client.28297 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:31:45.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.949+0000 7fa65000d740 1 -- 172.21.15.47:0/492281420 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5605c90b8b50 con 0x5605c91bec70 2024-03-21T11:31:45.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.950+0000 7fa645e00640 1 --2- 172.21.15.47:0/492281420 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5605c91bec70 0x5605c91c1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:45.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.950+0000 7fa645e00640 1 --2- 172.21.15.47:0/492281420 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5605c91bec70 0x5605c91c1090 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6b80adcbec23577c server_cookie=55d5496e743c2415 in_seq=0 out_seq=0 2024-03-21T11:31:45.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.950+0000 7fa62be00640 10 client.28297 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:31:45.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.950+0000 7fa62be00640 1 -- 172.21.15.47:0/492281420 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5605c90b8b50 con 0x5605c91bec70 2024-03-21T11:31:45.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.950+0000 7fa62be00640 10 client.28297 handle_command_reply: tid=0 2024-03-21T11:31:45.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.950+0000 7fa65000d740 1 client.28297 shutdown 2024-03-21T11:31:45.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.951+0000 7fa65000d740 1 -- 172.21.15.47:0/492281420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 msgr2=0x5605c915d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.951+0000 7fa65000d740 1 --2- 172.21.15.47:0/492281420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c915d140 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa640010650 tx=0x7fa64001e660 comp rx=0 tx=0).stop 2024-03-21T11:31:45.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.951+0000 7fa65000d740 1 -- 172.21.15.47:0/492281420 shutdown_connections 2024-03-21T11:31:45.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.951+0000 7fa65000d740 1 --2- 172.21.15.47:0/492281420 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5605c91bec70 0x5605c91c1090 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.951+0000 7fa65000d740 1 --2- 172.21.15.47:0/492281420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5605c914e3a0 0x5605c915d140 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.951+0000 7fa65000d740 1 --2- 172.21.15.47:0/492281420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5605c914d9e0 0x5605c915a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.951+0000 7fa65000d740 1 --2- 172.21.15.47:0/492281420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605c914c810 0x5605c9159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.951+0000 7fa65000d740 1 -- 172.21.15.47:0/492281420 >> 172.21.15.47:0/492281420 conn(0x5605c9147c00 msgr2=0x5605c9146620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:45.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.951+0000 7fa62be00640 0 client.28297 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:31:45.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.951+0000 7fa65000d740 1 -- 172.21.15.47:0/492281420 shutdown_connections 2024-03-21T11:31:45.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.951+0000 7fa65000d740 1 -- 172.21.15.47:0/492281420 wait complete. 2024-03-21T11:31:45.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.951+0000 7fa65000d740 20 client.28297 trim_cache size 0 max 16384 2024-03-21T11:31:45.958 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:31:45.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.952+0000 7fa64d200640 1 -- 172.21.15.47:0/1552782809 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa62c0a3020 msgr2=0x7fa62c0a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.952+0000 7fa64d200640 1 --2- 172.21.15.47:0/1552782809 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa62c0a3020 0x7fa62c0a54e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x5605c914d8e0 tx=0x7fa640004a40 comp rx=0 tx=0).stop 2024-03-21T11:31:45.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.952+0000 7fa64d200640 1 -- 172.21.15.47:0/1552782809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6481086a0 msgr2=0x7fa6481a6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:45.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.952+0000 7fa64d200640 1 --2- 172.21.15.47:0/1552782809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6481086a0 0x7fa6481a6880 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa63800e9f0 tx=0x7fa63800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.952+0000 7fa64d200640 1 -- 172.21.15.47:0/1552782809 shutdown_connections 2024-03-21T11:31:45.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.952+0000 7fa64d200640 1 --2- 172.21.15.47:0/1552782809 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa62c0a3020 0x7fa62c0a54e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.952+0000 7fa64d200640 1 --2- 172.21.15.47:0/1552782809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa64810ff20 0x7fa6481ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.952+0000 7fa64d200640 1 --2- 172.21.15.47:0/1552782809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa648109000 0x7fa6481a6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.952+0000 7fa64d200640 1 --2- 172.21.15.47:0/1552782809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6481086a0 0x7fa6481a6880 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:45.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.952+0000 7fa64d200640 1 -- 172.21.15.47:0/1552782809 >> 172.21.15.47:0/1552782809 conn(0x7fa64806e500 msgr2=0x7fa648071760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:45.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.952+0000 7fa64d200640 1 -- 172.21.15.47:0/1552782809 shutdown_connections 2024-03-21T11:31:45.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:45.952+0000 7fa64d200640 1 -- 172.21.15.47:0/1552782809 wait complete. 2024-03-21T11:31:45.967 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:45 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1886402101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:31:45.970 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:31:45.970 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:31:46.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:45 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1886402101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:31:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.178+0000 7f9528200640 1 -- 172.21.15.47:0/2957552284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9520104ad0 msgr2=0x7f9520104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.178+0000 7f9528200640 1 --2- 172.21.15.47:0/2957552284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9520104ad0 0x7f9520104f50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f950c009900 tx=0x7f950c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:31:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.179+0000 7f9528200640 1 -- 172.21.15.47:0/2957552284 shutdown_connections 2024-03-21T11:31:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.179+0000 7f9528200640 1 --2- 172.21.15.47:0/2957552284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9520105490 0x7f952010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.179+0000 7f9528200640 1 --2- 172.21.15.47:0/2957552284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9520104ad0 0x7f9520104f50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.179+0000 7f9528200640 1 --2- 172.21.15.47:0/2957552284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95201038e0 0x7f9520103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.179+0000 7f9528200640 1 -- 172.21.15.47:0/2957552284 >> 172.21.15.47:0/2957552284 conn(0x7f95200fee50 msgr2=0x7f9520101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.179+0000 7f9528200640 1 -- 172.21.15.47:0/2957552284 shutdown_connections 2024-03-21T11:31:46.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.179+0000 7f9528200640 1 -- 172.21.15.47:0/2957552284 wait complete. 2024-03-21T11:31:46.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.180+0000 7f9528200640 1 Processor -- start 2024-03-21T11:31:46.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.180+0000 7f9528200640 1 -- start start 2024-03-21T11:31:46.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.181+0000 7f9528200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95201038e0 0x7f9520199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.181+0000 7f9528200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9520104ad0 0x7f952019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.181+0000 7f9528200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9520105490 0x7f95201a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.181+0000 7f9528200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952010f590 con 0x7f95201038e0 2024-03-21T11:31:46.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.181+0000 7f9528200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952010f410 con 0x7f9520104ad0 2024-03-21T11:31:46.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.181+0000 7f9528200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952010f710 con 0x7f9520105490 2024-03-21T11:31:46.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.181+0000 7f9525800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95201038e0 0x7f9520199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.181+0000 7f9524e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9520104ad0 0x7f952019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.181+0000 7f9524e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9520104ad0 0x7f952019a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58434/0 (socket says 172.21.15.47:58434) 2024-03-21T11:31:46.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.181+0000 7f9524e00640 1 -- 172.21.15.47:0/319583957 learned_addr learned my addr 172.21.15.47:0/319583957 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.181+0000 7f9526200640 1 --2- 172.21.15.47:0/319583957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9520105490 0x7f95201a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.181+0000 7f9524e00640 1 -- 172.21.15.47:0/319583957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9520105490 msgr2=0x7f95201a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.181+0000 7f9524e00640 1 --2- 172.21.15.47:0/319583957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9520105490 0x7f95201a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.182+0000 7f9524e00640 1 -- 172.21.15.47:0/319583957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95201038e0 msgr2=0x7f9520199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.182+0000 7f9524e00640 1 --2- 172.21.15.47:0/319583957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95201038e0 0x7f9520199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.182+0000 7f9524e00640 1 -- 172.21.15.47:0/319583957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95201a1830 con 0x7f9520104ad0 2024-03-21T11:31:46.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.182+0000 7f9525800640 1 --2- 172.21.15.47:0/319583957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95201038e0 0x7f9520199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:46.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.182+0000 7f9524e00640 1 --2- 172.21.15.47:0/319583957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9520104ad0 0x7f952019a160 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f950c02c8f0 tx=0x7f950c02c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.182+0000 7f951e000640 1 -- 172.21.15.47:0/319583957 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c041070 con 0x7f9520104ad0 2024-03-21T11:31:46.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.182+0000 7f951e000640 1 -- 172.21.15.47:0/319583957 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f950c004450 con 0x7f9520104ad0 2024-03-21T11:31:46.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.182+0000 7f9528200640 1 -- 172.21.15.47:0/319583957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95201a1ac0 con 0x7f9520104ad0 2024-03-21T11:31:46.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.182+0000 7f951e000640 1 -- 172.21.15.47:0/319583957 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c003df0 con 0x7f9520104ad0 2024-03-21T11:31:46.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.183+0000 7f9528200640 1 -- 172.21.15.47:0/319583957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95201a1fb0 con 0x7f9520104ad0 2024-03-21T11:31:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.186+0000 7f951e000640 1 -- 172.21.15.47:0/319583957 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f950c044070 con 0x7f9520104ad0 2024-03-21T11:31:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.186+0000 7f952afa7740 1 Processor -- start 2024-03-21T11:31:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.186+0000 7f952afa7740 1 -- start start 2024-03-21T11:31:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.187+0000 7f952afa7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b754c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.187+0000 7f952afa7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b755e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.187+0000 7f952afa7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 0x556a3b75cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.187+0000 7f952afa7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3b6b92c0 con 0x556a3b7563a0 2024-03-21T11:31:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.187+0000 7f952afa7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3b661c90 con 0x556a3b7559e0 2024-03-21T11:31:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.187+0000 7f952afa7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3b6b5360 con 0x556a3b754810 2024-03-21T11:31:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.187+0000 7f9526200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b754c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.187+0000 7f9525800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b755e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.187+0000 7f9526200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b754c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44640/0 (socket says 172.21.15.47:44640) 2024-03-21T11:31:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.187+0000 7f9526200640 1 -- 172.21.15.47:0/535552857 learned_addr learned my addr 172.21.15.47:0/535552857 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.187+0000 7f9525800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b755e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58436/0 (socket says 172.21.15.47:58436) 2024-03-21T11:31:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.187+0000 7f9524e00640 1 --2- 172.21.15.47:0/535552857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 0x556a3b75cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.188+0000 7f9526200640 1 -- 172.21.15.47:0/535552857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 msgr2=0x556a3b755e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.188+0000 7f9526200640 1 --2- 172.21.15.47:0/535552857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b755e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.188+0000 7f9526200640 1 -- 172.21.15.47:0/535552857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 msgr2=0x556a3b75cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.188+0000 7f9526200640 1 --2- 172.21.15.47:0/535552857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 0x556a3b75cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.188+0000 7f9526200640 1 -- 172.21.15.47:0/535552857 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a3b686080 con 0x556a3b754810 2024-03-21T11:31:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.188+0000 7f951e000640 1 --2- 172.21.15.47:0/319583957 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95080a2f20 0x7f95080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.188+0000 7f9526200640 1 --2- 172.21.15.47:0/535552857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b754c30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9518009c90 tx=0x7f9518011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c427e3111cbdb64 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.188+0000 7f951e000640 1 -- 172.21.15.47:0/319583957 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f950c0f13e0 con 0x7f9520104ad0 2024-03-21T11:31:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.188+0000 7f9512a00640 1 -- 172.21.15.47:0/535552857 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9518004230 con 0x556a3b754810 2024-03-21T11:31:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.188+0000 7f9512a00640 1 -- 172.21.15.47:0/535552857 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9518009e40 con 0x556a3b754810 2024-03-21T11:31:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.189+0000 7f9512a00640 1 -- 172.21.15.47:0/535552857 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95180197e0 con 0x556a3b754810 2024-03-21T11:31:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.189+0000 7f9525800640 1 --2- 172.21.15.47:0/319583957 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95080a2f20 0x7f95080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.189+0000 7f952afa7740 1 -- 172.21.15.47:0/535552857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 msgr2=0x556a3b754c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.189+0000 7f952afa7740 1 --2- 172.21.15.47:0/535552857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b754c30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9518009c90 tx=0x7f9518011810 comp rx=0 tx=0).stop 2024-03-21T11:31:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.190+0000 7f9525800640 1 --2- 172.21.15.47:0/319583957 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95080a2f20 0x7f95080a53e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f952019b0f0 tx=0x7f9514009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.190+0000 7f952afa7740 1 -- 172.21.15.47:0/535552857 shutdown_connections 2024-03-21T11:31:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.190+0000 7f952afa7740 1 --2- 172.21.15.47:0/535552857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 0x556a3b75cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.190+0000 7f952afa7740 1 --2- 172.21.15.47:0/535552857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b755e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.190+0000 7f952afa7740 1 --2- 172.21.15.47:0/535552857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b754c30 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.190+0000 7f952afa7740 1 -- 172.21.15.47:0/535552857 >> 172.21.15.47:0/535552857 conn(0x556a3b74fc00 msgr2=0x556a3b750000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.190+0000 7f952afa7740 1 -- 172.21.15.47:0/535552857 shutdown_connections 2024-03-21T11:31:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.190+0000 7f952afa7740 1 -- 172.21.15.47:0/535552857 wait complete. 2024-03-21T11:31:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.190+0000 7f952afa7740 1 -- start start 2024-03-21T11:31:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.191+0000 7f952afa7740 1 Processor -- start 2024-03-21T11:31:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.191+0000 7f952afa7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b75eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.191+0000 7f952afa7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b75eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.191+0000 7f952afa7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 0x556a3b766030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.191+0000 7f952afa7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3b75d3b0 con 0x556a3b7563a0 2024-03-21T11:31:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.191+0000 7f952afa7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3b6750d0 con 0x556a3b7559e0 2024-03-21T11:31:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.191+0000 7f952afa7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3b75d530 con 0x556a3b754810 2024-03-21T11:31:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.191+0000 7f952afa7740 10 client.0 _ll_register_callbacks cb 0x556a3b679e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.192+0000 7f9524e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b75eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.192+0000 7f9524e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b75eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58450/0 (socket says 172.21.15.47:58450) 2024-03-21T11:31:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.192+0000 7f9524e00640 1 -- 172.21.15.47:0/3242490583 learned_addr learned my addr 172.21.15.47:0/3242490583 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.192+0000 7f9526200640 1 --2- 172.21.15.47:0/3242490583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 0x556a3b766030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.192+0000 7f9525800640 1 --2- 172.21.15.47:0/3242490583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b75eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.192+0000 7f9524e00640 1 -- 172.21.15.47:0/3242490583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 msgr2=0x556a3b75eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.192+0000 7f9524e00640 1 --2- 172.21.15.47:0/3242490583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b75eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.192+0000 7f9524e00640 1 -- 172.21.15.47:0/3242490583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 msgr2=0x556a3b766030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.192+0000 7f9524e00640 1 --2- 172.21.15.47:0/3242490583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 0x556a3b766030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.192+0000 7f9524e00640 1 -- 172.21.15.47:0/3242490583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a3b3cd3f0 con 0x556a3b7559e0 2024-03-21T11:31:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.192+0000 7f9526200640 1 --2- 172.21.15.47:0/3242490583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 0x556a3b766030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.192+0000 7f9524e00640 1 --2- 172.21.15.47:0/3242490583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b75eff0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f950c02c180 tx=0x7f950c03c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40b98081a262b2ec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.192+0000 7f9507400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.193+0000 7f9507400640 1 -- 172.21.15.47:0/3242490583 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c04b050 con 0x556a3b7559e0 2024-03-21T11:31:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.193+0000 7f9507400640 1 -- 172.21.15.47:0/3242490583 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f950c004c60 con 0x556a3b7559e0 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.193+0000 7f952afa7740 1 -- 172.21.15.47:0/3242490583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a3b6c6800 con 0x556a3b7559e0 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.193+0000 7f9507400640 1 -- 172.21.15.47:0/3242490583 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f950c0bd2c0 con 0x556a3b7559e0 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.193+0000 7f952afa7740 10 client.28307 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.193+0000 7f952afa7740 1 -- 172.21.15.47:0/3242490583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556a3b4977d0 con 0x556a3b7559e0 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.193+0000 7f9507400640 1 -- 172.21.15.47:0/3242490583 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556a3b6c6800 con 0x556a3b7559e0 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.193+0000 7f9507400640 1 -- 172.21.15.47:0/3242490583 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f950c0c5d00 con 0x556a3b7559e0 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.193+0000 7f952afa7740 10 client.28307 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.193+0000 7f952afa7740 10 client.28307 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.193+0000 7f952afa7740 20 client.28307 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.194+0000 7f952afa7740 1 --2- 172.21.15.47:0/3242490583 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x556a3b772d10 0x556a3b775130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.194+0000 7f952afa7740 4 client.28307 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.194+0000 7f952afa7740 1 -- 172.21.15.47:0/3242490583 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a3b6c0b50 con 0x556a3b772d10 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.194+0000 7f9525800640 1 --2- 172.21.15.47:0/3242490583 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x556a3b772d10 0x556a3b775130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.194+0000 7f9525800640 1 --2- 172.21.15.47:0/3242490583 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x556a3b772d10 0x556a3b775130 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bebf8149a2870e3e server_cookie=7d830fef590bf2b8 in_seq=0 out_seq=0 2024-03-21T11:31:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.194+0000 7f9507400640 10 client.28307 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:31:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.195+0000 7f9507400640 1 -- 172.21.15.47:0/3242490583 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556a3b6c0b50 con 0x556a3b772d10 2024-03-21T11:31:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.195+0000 7f9507400640 10 client.28307 handle_command_reply: tid=0 2024-03-21T11:31:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.195+0000 7f952afa7740 1 client.28307 shutdown 2024-03-21T11:31:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.195+0000 7f952afa7740 1 -- 172.21.15.47:0/3242490583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 msgr2=0x556a3b75eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.195+0000 7f952afa7740 1 --2- 172.21.15.47:0/3242490583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b75eff0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f950c02c180 tx=0x7f950c03c000 comp rx=0 tx=0).stop 2024-03-21T11:31:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.196+0000 7f952afa7740 1 -- 172.21.15.47:0/3242490583 shutdown_connections 2024-03-21T11:31:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.196+0000 7f952afa7740 1 --2- 172.21.15.47:0/3242490583 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x556a3b772d10 0x556a3b775130 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.196+0000 7f952afa7740 1 --2- 172.21.15.47:0/3242490583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 0x556a3b766030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.196+0000 7f952afa7740 1 --2- 172.21.15.47:0/3242490583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b75eff0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.196+0000 7f952afa7740 1 --2- 172.21.15.47:0/3242490583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b75eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.196+0000 7f952afa7740 1 -- 172.21.15.47:0/3242490583 >> 172.21.15.47:0/3242490583 conn(0x556a3b74fc00 msgr2=0x556a3b74eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.196+0000 7f9507400640 0 client.28307 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:31:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.196+0000 7f952afa7740 1 -- 172.21.15.47:0/3242490583 shutdown_connections 2024-03-21T11:31:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.196+0000 7f952afa7740 1 -- 172.21.15.47:0/3242490583 wait complete. 2024-03-21T11:31:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.196+0000 7f952afa7740 20 client.28307 trim_cache size 0 max 16384 2024-03-21T11:31:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.211+0000 7f952afa7740 1 Processor -- start 2024-03-21T11:31:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.211+0000 7f952afa7740 1 -- start start 2024-03-21T11:31:46.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.211+0000 7f952afa7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b7589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.211+0000 7f952afa7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b765ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.211+0000 7f952afa7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 0x556a3b758f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.211+0000 7f952afa7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95201a7f00 con 0x556a3b7563a0 2024-03-21T11:31:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.211+0000 7f952afa7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95201a9880 con 0x556a3b7559e0 2024-03-21T11:31:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.211+0000 7f952afa7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95201a8560 con 0x556a3b754810 2024-03-21T11:31:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.211+0000 7f9525800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b7589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.211+0000 7f9525800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b7589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44660/0 (socket says 172.21.15.47:44660) 2024-03-21T11:31:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.211+0000 7f9525800640 1 -- 172.21.15.47:0/1320299280 learned_addr learned my addr 172.21.15.47:0/1320299280 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.211+0000 7f9526200640 1 --2- 172.21.15.47:0/1320299280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 0x556a3b758f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.211+0000 7f9524e00640 1 --2- 172.21.15.47:0/1320299280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b765ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.212+0000 7f9525800640 1 -- 172.21.15.47:0/1320299280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 msgr2=0x556a3b765ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.212+0000 7f9525800640 1 --2- 172.21.15.47:0/1320299280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b765ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.212+0000 7f9525800640 1 -- 172.21.15.47:0/1320299280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 msgr2=0x556a3b758f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.212+0000 7f9525800640 1 --2- 172.21.15.47:0/1320299280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 0x556a3b758f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.212+0000 7f9525800640 1 -- 172.21.15.47:0/1320299280 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a3b750970 con 0x556a3b754810 2024-03-21T11:31:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.212+0000 7f9525800640 1 --2- 172.21.15.47:0/1320299280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b7589e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9520104930 tx=0x7f9514019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93518c36d0550f8e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.212+0000 7f9512000640 1 -- 172.21.15.47:0/1320299280 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514004b20 con 0x556a3b754810 2024-03-21T11:31:46.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.212+0000 7f9512000640 1 -- 172.21.15.47:0/1320299280 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9514004cc0 con 0x556a3b754810 2024-03-21T11:31:46.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.212+0000 7f9512000640 1 -- 172.21.15.47:0/1320299280 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95140051a0 con 0x556a3b754810 2024-03-21T11:31:46.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.213+0000 7f952afa7740 1 -- 172.21.15.47:0/1320299280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 msgr2=0x556a3b7589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.213+0000 7f952afa7740 1 --2- 172.21.15.47:0/1320299280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b7589e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9520104930 tx=0x7f9514019000 comp rx=0 tx=0).stop 2024-03-21T11:31:46.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.213+0000 7f952afa7740 1 -- 172.21.15.47:0/1320299280 shutdown_connections 2024-03-21T11:31:46.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.213+0000 7f952afa7740 1 --2- 172.21.15.47:0/1320299280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b7563a0 0x556a3b758f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.213+0000 7f952afa7740 1 --2- 172.21.15.47:0/1320299280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b765ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.213+0000 7f952afa7740 1 --2- 172.21.15.47:0/1320299280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b754810 0x556a3b7589e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.213+0000 7f952afa7740 1 -- 172.21.15.47:0/1320299280 >> 172.21.15.47:0/1320299280 conn(0x556a3b74fc00 msgr2=0x556a3b7536f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.213+0000 7f952afa7740 1 -- 172.21.15.47:0/1320299280 shutdown_connections 2024-03-21T11:31:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.213+0000 7f952afa7740 1 -- 172.21.15.47:0/1320299280 wait complete. 2024-03-21T11:31:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.214+0000 7f952afa7740 1 -- start start 2024-03-21T11:31:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.214+0000 7f952afa7740 1 Processor -- start 2024-03-21T11:31:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.214+0000 7f952afa7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b754810 0x556a3b761c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.214+0000 7f952afa7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b762150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.214+0000 7f952afa7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b7563a0 0x556a3b7650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.214+0000 7f952afa7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95201a9220 con 0x556a3b754810 2024-03-21T11:31:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.214+0000 7f952afa7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95201a8bc0 con 0x556a3b7559e0 2024-03-21T11:31:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.214+0000 7f952afa7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95201a60c0 con 0x556a3b7563a0 2024-03-21T11:31:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.214+0000 7f952afa7740 10 client.0 _ll_register_callbacks cb 0x7f950c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.215+0000 7f9524e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b762150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.215+0000 7f9524e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b762150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58456/0 (socket says 172.21.15.47:58456) 2024-03-21T11:31:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.215+0000 7f9526200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b7563a0 0x556a3b7650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.215+0000 7f9524e00640 1 -- 172.21.15.47:0/220471556 learned_addr learned my addr 172.21.15.47:0/220471556 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.215+0000 7f9525800640 1 --2- 172.21.15.47:0/220471556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b754810 0x556a3b761c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.215+0000 7f9524e00640 1 -- 172.21.15.47:0/220471556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b7563a0 msgr2=0x556a3b7650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.215+0000 7f9524e00640 1 --2- 172.21.15.47:0/220471556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b7563a0 0x556a3b7650d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.215+0000 7f9524e00640 1 -- 172.21.15.47:0/220471556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b754810 msgr2=0x556a3b761c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.215+0000 7f9524e00640 1 --2- 172.21.15.47:0/220471556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b754810 0x556a3b761c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.215+0000 7f9524e00640 1 -- 172.21.15.47:0/220471556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a3b7656c0 con 0x556a3b7559e0 2024-03-21T11:31:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.215+0000 7f9526200640 1 --2- 172.21.15.47:0/220471556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b7563a0 0x556a3b7650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.215+0000 7f9524e00640 1 --2- 172.21.15.47:0/220471556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b762150 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f950c0c5c60 tx=0x7f950c0038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14eee1524a5ab079 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.215+0000 7f9507400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.216+0000 7f9507400640 1 -- 172.21.15.47:0/220471556 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c04b040 con 0x556a3b7559e0 2024-03-21T11:31:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.216+0000 7f9507400640 1 -- 172.21.15.47:0/220471556 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f950c0bcdd0 con 0x556a3b7559e0 2024-03-21T11:31:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.216+0000 7f952afa7740 1 -- 172.21.15.47:0/220471556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9514004cc0 con 0x556a3b7559e0 2024-03-21T11:31:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.216+0000 7f9507400640 1 -- 172.21.15.47:0/220471556 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f950c0bb8c0 con 0x556a3b7559e0 2024-03-21T11:31:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.216+0000 7f952afa7740 10 client.28313 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.216+0000 7f952afa7740 1 -- 172.21.15.47:0/220471556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556a3b7cea60 con 0x556a3b7559e0 2024-03-21T11:31:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.216+0000 7f9507400640 1 -- 172.21.15.47:0/220471556 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9514004cc0 con 0x556a3b7559e0 2024-03-21T11:31:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.216+0000 7f9507400640 1 -- 172.21.15.47:0/220471556 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f950c0dc0f0 con 0x556a3b7559e0 2024-03-21T11:31:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.217+0000 7f952afa7740 10 client.28313 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.217+0000 7f952afa7740 10 client.28313 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:31:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.217+0000 7f952afa7740 20 client.28313 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.217+0000 7f952afa7740 1 --2- 172.21.15.47:0/220471556 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x556a3b7ceeb0 0x556a3b7d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.217+0000 7f952afa7740 4 client.28313 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:31:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.217+0000 7f952afa7740 1 -- 172.21.15.47:0/220471556 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556a3b63e500 con 0x556a3b7ceeb0 2024-03-21T11:31:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.217+0000 7f9525800640 1 --2- 172.21.15.47:0/220471556 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x556a3b7ceeb0 0x556a3b7d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.217+0000 7f9525800640 1 --2- 172.21.15.47:0/220471556 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x556a3b7ceeb0 0x556a3b7d12d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=18c018fff452950a server_cookie=b7a382f04147b494 in_seq=0 out_seq=0 2024-03-21T11:31:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.217+0000 7f9507400640 10 client.28313 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:31:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f9507400640 1 -- 172.21.15.47:0/220471556 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556a3b63e500 con 0x556a3b7ceeb0 2024-03-21T11:31:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f9507400640 10 client.28313 handle_command_reply: tid=0 2024-03-21T11:31:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f952afa7740 1 client.28313 shutdown 2024-03-21T11:31:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f952afa7740 1 -- 172.21.15.47:0/220471556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 msgr2=0x556a3b762150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f952afa7740 1 --2- 172.21.15.47:0/220471556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b762150 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f950c0c5c60 tx=0x7f950c0038b0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f952afa7740 1 -- 172.21.15.47:0/220471556 shutdown_connections 2024-03-21T11:31:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f952afa7740 1 --2- 172.21.15.47:0/220471556 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x556a3b7ceeb0 0x556a3b7d12d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f952afa7740 1 --2- 172.21.15.47:0/220471556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a3b7563a0 0x556a3b7650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f952afa7740 1 --2- 172.21.15.47:0/220471556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a3b7559e0 0x556a3b762150 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f952afa7740 1 --2- 172.21.15.47:0/220471556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a3b754810 0x556a3b761c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f952afa7740 1 -- 172.21.15.47:0/220471556 >> 172.21.15.47:0/220471556 conn(0x556a3b74fc00 msgr2=0x556a3b74e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f9507400640 0 client.28313 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:31:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f952afa7740 1 -- 172.21.15.47:0/220471556 shutdown_connections 2024-03-21T11:31:46.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f952afa7740 1 -- 172.21.15.47:0/220471556 wait complete. 2024-03-21T11:31:46.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.218+0000 7f952afa7740 20 client.28313 trim_cache size 0 max 16384 2024-03-21T11:31:46.225 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:31:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.219+0000 7f9528200640 1 -- 172.21.15.47:0/319583957 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95080a2f20 msgr2=0x7f95080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.219+0000 7f9528200640 1 --2- 172.21.15.47:0/319583957 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95080a2f20 0x7f95080a53e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f952019b0f0 tx=0x7f9514009500 comp rx=0 tx=0).stop 2024-03-21T11:31:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.219+0000 7f9528200640 1 -- 172.21.15.47:0/319583957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9520104ad0 msgr2=0x7f952019a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.219+0000 7f9528200640 1 --2- 172.21.15.47:0/319583957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9520104ad0 0x7f952019a160 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f950c02c8f0 tx=0x7f950c02c660 comp rx=0 tx=0).stop 2024-03-21T11:31:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.219+0000 7f9528200640 1 -- 172.21.15.47:0/319583957 shutdown_connections 2024-03-21T11:31:46.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.219+0000 7f9528200640 1 --2- 172.21.15.47:0/319583957 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95080a2f20 0x7f95080a53e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.219+0000 7f9528200640 1 --2- 172.21.15.47:0/319583957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9520105490 0x7f95201a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.219+0000 7f9528200640 1 --2- 172.21.15.47:0/319583957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9520104ad0 0x7f952019a160 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.219+0000 7f9528200640 1 --2- 172.21.15.47:0/319583957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95201038e0 0x7f9520199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.219+0000 7f9528200640 1 -- 172.21.15.47:0/319583957 >> 172.21.15.47:0/319583957 conn(0x7f95200fee50 msgr2=0x7f9520107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.219+0000 7f9528200640 1 -- 172.21.15.47:0/319583957 shutdown_connections 2024-03-21T11:31:46.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.219+0000 7f9528200640 1 -- 172.21.15.47:0/319583957 wait complete. 2024-03-21T11:31:46.237 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:31:46.237 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:31:46.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:45 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1886402101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:31:46.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.456+0000 7fbc97800640 1 -- 172.21.15.47:0/178676964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc90105490 msgr2=0x7fbc9010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.456+0000 7fbc97800640 1 --2- 172.21.15.47:0/178676964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc90105490 0x7fbc9010bd20 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009ca0 tx=0x7fbc8802c100 comp rx=0 tx=0).stop 2024-03-21T11:31:46.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.456+0000 7fbc97800640 1 -- 172.21.15.47:0/178676964 shutdown_connections 2024-03-21T11:31:46.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.456+0000 7fbc97800640 1 --2- 172.21.15.47:0/178676964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc90105490 0x7fbc9010bd20 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.456+0000 7fbc97800640 1 --2- 172.21.15.47:0/178676964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc90104ad0 0x7fbc90104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.456+0000 7fbc97800640 1 --2- 172.21.15.47:0/178676964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc901038e0 0x7fbc90103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.456+0000 7fbc97800640 1 -- 172.21.15.47:0/178676964 >> 172.21.15.47:0/178676964 conn(0x7fbc900fee50 msgr2=0x7fbc90101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.456+0000 7fbc97800640 1 -- 172.21.15.47:0/178676964 shutdown_connections 2024-03-21T11:31:46.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.456+0000 7fbc97800640 1 -- 172.21.15.47:0/178676964 wait complete. 2024-03-21T11:31:46.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.457+0000 7fbc97800640 1 Processor -- start 2024-03-21T11:31:46.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.457+0000 7fbc97800640 1 -- start start 2024-03-21T11:31:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc97800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc901038e0 0x7fbc90199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc97800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc90104ad0 0x7fbc9019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc97800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90105490 0x7fbc901a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc97800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9010f540 con 0x7fbc90105490 2024-03-21T11:31:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc97800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9010f3c0 con 0x7fbc901038e0 2024-03-21T11:31:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc97800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9010f6c0 con 0x7fbc90104ad0 2024-03-21T11:31:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc94e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc901038e0 0x7fbc90199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc94e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc901038e0 0x7fbc90199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58470/0 (socket says 172.21.15.47:58470) 2024-03-21T11:31:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc94e00640 1 -- 172.21.15.47:0/650408779 learned_addr learned my addr 172.21.15.47:0/650408779 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc8fe00640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc90104ad0 0x7fbc9019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc95800640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90105490 0x7fbc901a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc94e00640 1 -- 172.21.15.47:0/650408779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc90104ad0 msgr2=0x7fbc9019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc94e00640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc90104ad0 0x7fbc9019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc94e00640 1 -- 172.21.15.47:0/650408779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90105490 msgr2=0x7fbc901a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc94e00640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90105490 0x7fbc901a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc94e00640 1 -- 172.21.15.47:0/650408779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc901a1780 con 0x7fbc901038e0 2024-03-21T11:31:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.458+0000 7fbc95800640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90105490 0x7fbc901a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.459+0000 7fbc94e00640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc901038e0 0x7fbc90199b60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400e9c0 tx=0x7fbc8400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.459+0000 7fbc8d600640 1 -- 172.21.15.47:0/650408779 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8400cde0 con 0x7fbc901038e0 2024-03-21T11:31:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.459+0000 7fbc8d600640 1 -- 172.21.15.47:0/650408779 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc84010910 con 0x7fbc901038e0 2024-03-21T11:31:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.459+0000 7fbc8d600640 1 -- 172.21.15.47:0/650408779 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84018a00 con 0x7fbc901038e0 2024-03-21T11:31:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.459+0000 7fbc97800640 1 -- 172.21.15.47:0/650408779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc901a1a70 con 0x7fbc901038e0 2024-03-21T11:31:46.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.459+0000 7fbc97800640 1 -- 172.21.15.47:0/650408779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc901a2010 con 0x7fbc901038e0 2024-03-21T11:31:46.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.462+0000 7fbc9a402740 1 Processor -- start 2024-03-21T11:31:46.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.462+0000 7fbc9a402740 1 -- start start 2024-03-21T11:31:46.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc8d600640 1 -- 172.21.15.47:0/650408779 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbc8401f020 con 0x7fbc901038e0 2024-03-21T11:31:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc9a402740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561760170810 0x561760170c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc9a402740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617601719e0 0x561760171e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc9a402740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601723a0 0x561760178c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc9a402740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617600d52c0 con 0x5617601723a0 2024-03-21T11:31:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc9a402740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176007dc90 con 0x5617601719e0 2024-03-21T11:31:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc9a402740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617600d1360 con 0x561760170810 2024-03-21T11:31:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc94e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617601719e0 0x561760171e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc95800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561760170810 0x561760170c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc95800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561760170810 0x561760170c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44712/0 (socket says 172.21.15.47:44712) 2024-03-21T11:31:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc95800640 1 -- 172.21.15.47:0/3712386685 learned_addr learned my addr 172.21.15.47:0/3712386685 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc94e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617601719e0 0x561760171e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58480/0 (socket says 172.21.15.47:58480) 2024-03-21T11:31:46.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc8fe00640 1 --2- 172.21.15.47:0/3712386685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601723a0 0x561760178c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc95800640 1 -- 172.21.15.47:0/3712386685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617601719e0 msgr2=0x561760171e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc95800640 1 --2- 172.21.15.47:0/3712386685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617601719e0 0x561760171e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc95800640 1 -- 172.21.15.47:0/3712386685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601723a0 msgr2=0x561760178c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc95800640 1 --2- 172.21.15.47:0/3712386685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601723a0 0x561760178c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.463+0000 7fbc95800640 1 -- 172.21.15.47:0/3712386685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617600a2080 con 0x561760170810 2024-03-21T11:31:46.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.464+0000 7fbc95800640 1 --2- 172.21.15.47:0/3712386685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561760170810 0x561760170c30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbc880324c0 tx=0x7fbc88005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e171679e340791 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.464+0000 7fbc82000640 1 -- 172.21.15.47:0/3712386685 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88043020 con 0x561760170810 2024-03-21T11:31:46.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.464+0000 7fbc82000640 1 -- 172.21.15.47:0/3712386685 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc88003970 con 0x561760170810 2024-03-21T11:31:46.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.464+0000 7fbc82000640 1 -- 172.21.15.47:0/3712386685 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88039040 con 0x561760170810 2024-03-21T11:31:46.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.465+0000 7fbc8d600640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbc780a2fa0 0x7fbc780a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.465+0000 7fbc8d600640 1 -- 172.21.15.47:0/650408779 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbc840cf980 con 0x7fbc901038e0 2024-03-21T11:31:46.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.465+0000 7fbc94e00640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbc780a2fa0 0x7fbc780a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.465+0000 7fbc9a402740 1 -- 172.21.15.47:0/3712386685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561760170810 msgr2=0x561760170c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.465+0000 7fbc9a402740 1 --2- 172.21.15.47:0/3712386685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561760170810 0x561760170c30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbc880324c0 tx=0x7fbc88005f70 comp rx=0 tx=0).stop 2024-03-21T11:31:46.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.465+0000 7fbc9a402740 1 -- 172.21.15.47:0/3712386685 shutdown_connections 2024-03-21T11:31:46.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.465+0000 7fbc9a402740 1 --2- 172.21.15.47:0/3712386685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601723a0 0x561760178c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.465+0000 7fbc9a402740 1 --2- 172.21.15.47:0/3712386685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617601719e0 0x561760171e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.465+0000 7fbc9a402740 1 --2- 172.21.15.47:0/3712386685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561760170810 0x561760170c30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.465+0000 7fbc9a402740 1 -- 172.21.15.47:0/3712386685 >> 172.21.15.47:0/3712386685 conn(0x56176016bc00 msgr2=0x56176016c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.465+0000 7fbc9a402740 1 -- 172.21.15.47:0/3712386685 shutdown_connections 2024-03-21T11:31:46.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.465+0000 7fbc9a402740 1 -- 172.21.15.47:0/3712386685 wait complete. 2024-03-21T11:31:46.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.466+0000 7fbc94e00640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbc780a2fa0 0x7fbc780a5460 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbc840986f0 tx=0x7fbc840d5550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.466+0000 7fbc9a402740 1 -- start start 2024-03-21T11:31:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.466+0000 7fbc9a402740 1 Processor -- start 2024-03-21T11:31:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.466+0000 7fbc9a402740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561760170810 0x56176017a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.466+0000 7fbc9a402740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5617601719e0 0x56176017af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.466+0000 7fbc9a402740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601723a0 0x561760181f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.466+0000 7fbc9a402740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561760179350 con 0x5617601723a0 2024-03-21T11:31:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc9a402740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617600910d0 con 0x561760170810 2024-03-21T11:31:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc9a402740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617601794d0 con 0x5617601719e0 2024-03-21T11:31:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc9a402740 10 client.0 _ll_register_callbacks cb 0x561760095e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc8fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5617601719e0 0x56176017af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc8fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5617601719e0 0x56176017af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44728/0 (socket says 172.21.15.47:44728) 2024-03-21T11:31:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc8fe00640 1 -- 172.21.15.47:0/177982459 learned_addr learned my addr 172.21.15.47:0/177982459 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc95800640 1 --2- 172.21.15.47:0/177982459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561760170810 0x56176017a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc95800640 1 --2- 172.21.15.47:0/177982459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601723a0 0x561760181f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc8fe00640 1 -- 172.21.15.47:0/177982459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561760170810 msgr2=0x56176017a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc8fe00640 1 --2- 172.21.15.47:0/177982459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561760170810 0x56176017a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc8fe00640 1 -- 172.21.15.47:0/177982459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601723a0 msgr2=0x561760181f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc8fe00640 1 --2- 172.21.15.47:0/177982459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601723a0 0x561760181f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc8fe00640 1 -- 172.21.15.47:0/177982459 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56175fde93f0 con 0x5617601719e0 2024-03-21T11:31:46.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc95800640 1 --2- 172.21.15.47:0/177982459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601723a0 0x561760181f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:31:46.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc8fe00640 1 --2- 172.21.15.47:0/177982459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5617601719e0 0x56176017af00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c012700 tx=0x7fbc7c012b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b1f8b871b85198e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.467+0000 7fbc76a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:46.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.468+0000 7fbc76a00640 1 -- 172.21.15.47:0/177982459 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c00f610 con 0x5617601719e0 2024-03-21T11:31:46.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.468+0000 7fbc9a402740 1 -- 172.21.15.47:0/177982459 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc88003970 con 0x5617601719e0 2024-03-21T11:31:46.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.468+0000 7fbc76a00640 1 -- 172.21.15.47:0/177982459 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbc7c00f7b0 con 0x5617601719e0 2024-03-21T11:31:46.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.468+0000 7fbc76a00640 1 -- 172.21.15.47:0/177982459 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc7c00f9b0 con 0x5617601719e0 2024-03-21T11:31:46.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.468+0000 7fbc9a402740 10 client.28327 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:46.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.468+0000 7fbc9a402740 1 -- 172.21.15.47:0/177982459 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56175feb37d0 con 0x5617601719e0 2024-03-21T11:31:46.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.468+0000 7fbc76a00640 1 -- 172.21.15.47:0/177982459 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc88003970 con 0x5617601719e0 2024-03-21T11:31:46.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.469+0000 7fbc76a00640 1 -- 172.21.15.47:0/177982459 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbc7c010ce0 con 0x5617601719e0 2024-03-21T11:31:46.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.469+0000 7fbc9a402740 10 client.28327 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:46.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.469+0000 7fbc9a402740 10 client.28327 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:31:46.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.469+0000 7fbc9a402740 20 client.28327 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:46.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.469+0000 7fbc9a402740 1 --2- 172.21.15.47:0/177982459 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561760186590 0x5617601889d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.469+0000 7fbc9a402740 4 client.28327 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:46.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.469+0000 7fbc9a402740 1 -- 172.21.15.47:0/177982459 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617600e2800 con 0x561760186590 2024-03-21T11:31:46.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.469+0000 7fbc95800640 1 --2- 172.21.15.47:0/177982459 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561760186590 0x5617601889d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.469+0000 7fbc95800640 1 --2- 172.21.15.47:0/177982459 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561760186590 0x5617601889d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4abcb367aa5b25ad server_cookie=201491c3cc89d137 in_seq=0 out_seq=0 2024-03-21T11:31:46.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.469+0000 7fbc76a00640 10 client.28327 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:31:46.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.470+0000 7fbc76a00640 1 -- 172.21.15.47:0/177982459 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5617600e2800 con 0x561760186590 2024-03-21T11:31:46.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.470+0000 7fbc76a00640 10 client.28327 handle_command_reply: tid=0 2024-03-21T11:31:46.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.470+0000 7fbc9a402740 1 client.28327 shutdown 2024-03-21T11:31:46.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.470+0000 7fbc9a402740 1 -- 172.21.15.47:0/177982459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5617601719e0 msgr2=0x56176017af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.470+0000 7fbc9a402740 1 --2- 172.21.15.47:0/177982459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5617601719e0 0x56176017af00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c012700 tx=0x7fbc7c012b00 comp rx=0 tx=0).stop 2024-03-21T11:31:46.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.471+0000 7fbc9a402740 1 -- 172.21.15.47:0/177982459 shutdown_connections 2024-03-21T11:31:46.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.471+0000 7fbc9a402740 1 --2- 172.21.15.47:0/177982459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601723a0 0x561760181f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.471+0000 7fbc9a402740 1 --2- 172.21.15.47:0/177982459 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561760186590 0x5617601889d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.471+0000 7fbc9a402740 1 --2- 172.21.15.47:0/177982459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5617601719e0 0x56176017af00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.471+0000 7fbc9a402740 1 --2- 172.21.15.47:0/177982459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561760170810 0x56176017a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.471+0000 7fbc76a00640 0 client.28327 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:31:46.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.471+0000 7fbc9a402740 1 -- 172.21.15.47:0/177982459 >> 172.21.15.47:0/177982459 conn(0x56176016bc00 msgr2=0x56176016aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.471+0000 7fbc9a402740 1 -- 172.21.15.47:0/177982459 shutdown_connections 2024-03-21T11:31:46.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.471+0000 7fbc9a402740 1 -- 172.21.15.47:0/177982459 wait complete. 2024-03-21T11:31:46.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.471+0000 7fbc9a402740 20 client.28327 trim_cache size 0 max 16384 2024-03-21T11:31:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.485+0000 7fbc9a402740 1 Processor -- start 2024-03-21T11:31:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.485+0000 7fbc9a402740 1 -- start start 2024-03-21T11:31:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.485+0000 7fbc9a402740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56176016bc00 0x56176016ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.485+0000 7fbc9a402740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561760170810 0x561760181f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.485+0000 7fbc9a402740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5617601719e0 0x56176018d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.485+0000 7fbc9a402740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901ac300 con 0x56176016bc00 2024-03-21T11:31:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.485+0000 7fbc9a402740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901adc80 con 0x561760170810 2024-03-21T11:31:46.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.485+0000 7fbc9a402740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901ac960 con 0x5617601719e0 2024-03-21T11:31:46.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.485+0000 7fbc95800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56176016bc00 0x56176016ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.485+0000 7fbc8fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561760170810 0x561760181f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.485+0000 7fbc95800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56176016bc00 0x56176016ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45458/0 (socket says 172.21.15.47:45458) 2024-03-21T11:31:46.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.485+0000 7fbc95800640 1 -- 172.21.15.47:0/3220651417 learned_addr learned my addr 172.21.15.47:0/3220651417 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.486+0000 7fbc95800640 1 --2- 172.21.15.47:0/3220651417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5617601719e0 0x56176018d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.486+0000 7fbc8fe00640 1 -- 172.21.15.47:0/3220651417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5617601719e0 msgr2=0x56176018d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.486+0000 7fbc8fe00640 1 --2- 172.21.15.47:0/3220651417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5617601719e0 0x56176018d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.486+0000 7fbc8fe00640 1 -- 172.21.15.47:0/3220651417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56176016bc00 msgr2=0x56176016ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.486+0000 7fbc8fe00640 1 --2- 172.21.15.47:0/3220651417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56176016bc00 0x56176016ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.486+0000 7fbc8fe00640 1 -- 172.21.15.47:0/3220651417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56176016cb70 con 0x561760170810 2024-03-21T11:31:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.486+0000 7fbc95800640 1 --2- 172.21.15.47:0/3220651417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56176016bc00 0x56176016ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.486+0000 7fbc8fe00640 1 --2- 172.21.15.47:0/3220651417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561760170810 0x561760181f00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c01c570 tx=0x7fbc7c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc1807efab04e981 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.486+0000 7fbc81600640 1 -- 172.21.15.47:0/3220651417 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c00f610 con 0x561760170810 2024-03-21T11:31:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.486+0000 7fbc81600640 1 -- 172.21.15.47:0/3220651417 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc7c00f7b0 con 0x561760170810 2024-03-21T11:31:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.486+0000 7fbc81600640 1 -- 172.21.15.47:0/3220651417 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0054a0 con 0x561760170810 2024-03-21T11:31:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.487+0000 7fbc9a402740 1 -- 172.21.15.47:0/3220651417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561760170810 msgr2=0x561760181f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.487+0000 7fbc9a402740 1 --2- 172.21.15.47:0/3220651417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561760170810 0x561760181f00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c01c570 tx=0x7fbc7c005d30 comp rx=0 tx=0).stop 2024-03-21T11:31:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.487+0000 7fbc9a402740 1 -- 172.21.15.47:0/3220651417 shutdown_connections 2024-03-21T11:31:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.487+0000 7fbc9a402740 1 --2- 172.21.15.47:0/3220651417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5617601719e0 0x56176018d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.487+0000 7fbc9a402740 1 --2- 172.21.15.47:0/3220651417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561760170810 0x561760181f00 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.487+0000 7fbc9a402740 1 --2- 172.21.15.47:0/3220651417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56176016bc00 0x56176016ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.487+0000 7fbc9a402740 1 -- 172.21.15.47:0/3220651417 >> 172.21.15.47:0/3220651417 conn(0x56176017a580 msgr2=0x56176016a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.487+0000 7fbc9a402740 1 -- 172.21.15.47:0/3220651417 shutdown_connections 2024-03-21T11:31:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.487+0000 7fbc9a402740 1 -- 172.21.15.47:0/3220651417 wait complete. 2024-03-21T11:31:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.488+0000 7fbc9a402740 1 -- start start 2024-03-21T11:31:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.488+0000 7fbc9a402740 1 Processor -- start 2024-03-21T11:31:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.488+0000 7fbc9a402740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56176016bc00 0x561760180650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.488+0000 7fbc9a402740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561760170810 0x561760180b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.488+0000 7fbc9a402740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601719e0 0x5617601810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.488+0000 7fbc9a402740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901ad620 con 0x5617601719e0 2024-03-21T11:31:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.488+0000 7fbc9a402740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901acfc0 con 0x56176016bc00 2024-03-21T11:31:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.488+0000 7fbc9a402740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901aa4c0 con 0x561760170810 2024-03-21T11:31:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.488+0000 7fbc9a402740 10 client.0 _ll_register_callbacks cb 0x7fbc7c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc95800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56176016bc00 0x561760180650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc8fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561760170810 0x561760180b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc95800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56176016bc00 0x561760180650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58496/0 (socket says 172.21.15.47:58496) 2024-03-21T11:31:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc95800640 1 -- 172.21.15.47:0/2667687098 learned_addr learned my addr 172.21.15.47:0/2667687098 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc95800640 1 --2- 172.21.15.47:0/2667687098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601719e0 0x5617601810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc95800640 1 -- 172.21.15.47:0/2667687098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561760170810 msgr2=0x561760180b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc95800640 1 --2- 172.21.15.47:0/2667687098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561760170810 0x561760180b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc95800640 1 -- 172.21.15.47:0/2667687098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601719e0 msgr2=0x5617601810d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc95800640 1 --2- 172.21.15.47:0/2667687098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601719e0 0x5617601810d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc95800640 1 -- 172.21.15.47:0/2667687098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617601e1e60 con 0x56176016bc00 2024-03-21T11:31:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc95800640 1 --2- 172.21.15.47:0/2667687098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56176016bc00 0x561760180650 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbc880324c0 tx=0x7fbc88005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e3da532895b9ce3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc76a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc76a00640 1 -- 172.21.15.47:0/2667687098 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88043070 con 0x56176016bc00 2024-03-21T11:31:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc9a402740 1 -- 172.21.15.47:0/2667687098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc7c00f7b0 con 0x56176016bc00 2024-03-21T11:31:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.489+0000 7fbc76a00640 1 -- 172.21.15.47:0/2667687098 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbc88032890 con 0x56176016bc00 2024-03-21T11:31:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.490+0000 7fbc76a00640 1 -- 172.21.15.47:0/2667687098 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc88039040 con 0x56176016bc00 2024-03-21T11:31:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.490+0000 7fbc9a402740 10 client.28331 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.490+0000 7fbc9a402740 1 -- 172.21.15.47:0/2667687098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5617601e3080 con 0x56176016bc00 2024-03-21T11:31:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.490+0000 7fbc76a00640 1 -- 172.21.15.47:0/2667687098 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc7c00f7b0 con 0x56176016bc00 2024-03-21T11:31:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.490+0000 7fbc76a00640 1 -- 172.21.15.47:0/2667687098 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbc8803f950 con 0x56176016bc00 2024-03-21T11:31:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.490+0000 7fbc9a402740 10 client.28331 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.490+0000 7fbc9a402740 10 client.28331 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:31:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.490+0000 7fbc9a402740 20 client.28331 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.490+0000 7fbc9a402740 1 --2- 172.21.15.47:0/2667687098 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5617601723a0 0x5617601e5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.490+0000 7fbc9a402740 4 client.28331 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:31:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.490+0000 7fbc9a402740 1 -- 172.21.15.47:0/2667687098 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5617600dcb50 con 0x5617601723a0 2024-03-21T11:31:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.490+0000 7fbc8fe00640 1 --2- 172.21.15.47:0/2667687098 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5617601723a0 0x5617601e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.491+0000 7fbc8fe00640 1 --2- 172.21.15.47:0/2667687098 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5617601723a0 0x5617601e5590 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f27352f598cd664b server_cookie=3f148b9cbf627663 in_seq=0 out_seq=0 2024-03-21T11:31:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.491+0000 7fbc76a00640 10 client.28331 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:31:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.491+0000 7fbc76a00640 1 -- 172.21.15.47:0/2667687098 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5617600dcb50 con 0x5617601723a0 2024-03-21T11:31:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.491+0000 7fbc76a00640 10 client.28331 handle_command_reply: tid=0 2024-03-21T11:31:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.491+0000 7fbc9a402740 1 client.28331 shutdown 2024-03-21T11:31:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.491+0000 7fbc9a402740 1 -- 172.21.15.47:0/2667687098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56176016bc00 msgr2=0x561760180650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.491+0000 7fbc9a402740 1 --2- 172.21.15.47:0/2667687098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56176016bc00 0x561760180650 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbc880324c0 tx=0x7fbc88005df0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.491+0000 7fbc9a402740 1 -- 172.21.15.47:0/2667687098 shutdown_connections 2024-03-21T11:31:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.491+0000 7fbc9a402740 1 --2- 172.21.15.47:0/2667687098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617601719e0 0x5617601810d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.491+0000 7fbc9a402740 1 --2- 172.21.15.47:0/2667687098 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5617601723a0 0x5617601e5590 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.491+0000 7fbc9a402740 1 --2- 172.21.15.47:0/2667687098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561760170810 0x561760180b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.491+0000 7fbc9a402740 1 --2- 172.21.15.47:0/2667687098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56176016bc00 0x561760180650 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.491+0000 7fbc76a00640 0 client.28331 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:31:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.491+0000 7fbc9a402740 1 -- 172.21.15.47:0/2667687098 >> 172.21.15.47:0/2667687098 conn(0x56176017a580 msgr2=0x5617601754a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.492+0000 7fbc9a402740 1 -- 172.21.15.47:0/2667687098 shutdown_connections 2024-03-21T11:31:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.492+0000 7fbc9a402740 1 -- 172.21.15.47:0/2667687098 wait complete. 2024-03-21T11:31:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.492+0000 7fbc9a402740 20 client.28331 trim_cache size 0 max 16384 2024-03-21T11:31:46.496 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:31:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.492+0000 7fbc97800640 1 -- 172.21.15.47:0/650408779 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbc780a2fa0 msgr2=0x7fbc780a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.492+0000 7fbc97800640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbc780a2fa0 0x7fbc780a5460 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbc840986f0 tx=0x7fbc840d5550 comp rx=0 tx=0).stop 2024-03-21T11:31:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.492+0000 7fbc97800640 1 -- 172.21.15.47:0/650408779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc901038e0 msgr2=0x7fbc90199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.492+0000 7fbc97800640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc901038e0 0x7fbc90199b60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400e9c0 tx=0x7fbc8400ee80 comp rx=0 tx=0).stop 2024-03-21T11:31:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.492+0000 7fbc97800640 1 -- 172.21.15.47:0/650408779 shutdown_connections 2024-03-21T11:31:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.492+0000 7fbc97800640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbc780a2fa0 0x7fbc780a5460 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.492+0000 7fbc97800640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbc90105490 0x7fbc901a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.492+0000 7fbc97800640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc90104ad0 0x7fbc9019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.492+0000 7fbc97800640 1 --2- 172.21.15.47:0/650408779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc901038e0 0x7fbc90199b60 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.492+0000 7fbc97800640 1 -- 172.21.15.47:0/650408779 >> 172.21.15.47:0/650408779 conn(0x7fbc900fee50 msgr2=0x7fbc90107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.493+0000 7fbc97800640 1 -- 172.21.15.47:0/650408779 shutdown_connections 2024-03-21T11:31:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.493+0000 7fbc97800640 1 -- 172.21.15.47:0/650408779 wait complete. 2024-03-21T11:31:46.509 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:31:46.510 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:31:46.538 INFO:tasks.workunit.client.0.smithi047.stdout: 1 602172 8.18 MB/sec execute 270 sec latency 16.084 ms 2024-03-21T11:31:46.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.729+0000 7f12cd000640 1 -- 172.21.15.47:0/1936051587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c8077c40 msgr2=0x7f12c80784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.730+0000 7f12cd000640 1 --2- 172.21.15.47:0/1936051587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c8077c40 0x7f12c80784e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009ca0 tx=0x7f12c002c0f0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.730+0000 7f12cd000640 1 -- 172.21.15.47:0/1936051587 shutdown_connections 2024-03-21T11:31:46.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.730+0000 7f12cd000640 1 --2- 172.21.15.47:0/1936051587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c8077c40 0x7f12c80784e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.730+0000 7f12cd000640 1 --2- 172.21.15.47:0/1936051587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12c80772a0 0x7f12c8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.730+0000 7f12cd000640 1 --2- 172.21.15.47:0/1936051587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c8076060 0x7f12c8076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.730+0000 7f12cd000640 1 -- 172.21.15.47:0/1936051587 >> 172.21.15.47:0/1936051587 conn(0x7f12c80fee50 msgr2=0x7f12c8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.730+0000 7f12cd000640 1 -- 172.21.15.47:0/1936051587 shutdown_connections 2024-03-21T11:31:46.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.730+0000 7f12cd000640 1 -- 172.21.15.47:0/1936051587 wait complete. 2024-03-21T11:31:46.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.731+0000 7f12cd000640 1 Processor -- start 2024-03-21T11:31:46.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.731+0000 7f12cd000640 1 -- start start 2024-03-21T11:31:46.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.731+0000 7f12cd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c8076060 0x7f12c819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.731+0000 7f12cd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c80772a0 0x7f12c819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.731+0000 7f12cd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12c8077c40 0x7f12c81a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.731+0000 7f12cd000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c8113970 con 0x7f12c80772a0 2024-03-21T11:31:46.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.731+0000 7f12cd000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c81137f0 con 0x7f12c8076060 2024-03-21T11:31:46.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.731+0000 7f12cd000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c8113af0 con 0x7f12c8077c40 2024-03-21T11:31:46.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.731+0000 7f12c5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c8076060 0x7f12c819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.732+0000 7f12c5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c8076060 0x7f12c819df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58504/0 (socket says 172.21.15.47:58504) 2024-03-21T11:31:46.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.732+0000 7f12c5e00640 1 -- 172.21.15.47:0/1359349391 learned_addr learned my addr 172.21.15.47:0/1359349391 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.732+0000 7f12c6800640 1 --2- 172.21.15.47:0/1359349391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12c8077c40 0x7f12c81a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.732+0000 7f12bfe00640 1 --2- 172.21.15.47:0/1359349391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c80772a0 0x7f12c819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.732+0000 7f12c5e00640 1 -- 172.21.15.47:0/1359349391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12c8077c40 msgr2=0x7f12c81a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.732+0000 7f12c5e00640 1 --2- 172.21.15.47:0/1359349391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12c8077c40 0x7f12c81a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.732+0000 7f12c5e00640 1 -- 172.21.15.47:0/1359349391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c80772a0 msgr2=0x7f12c819e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.732+0000 7f12c5e00640 1 --2- 172.21.15.47:0/1359349391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c80772a0 0x7f12c819e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.732+0000 7f12c5e00640 1 -- 172.21.15.47:0/1359349391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c81a5bb0 con 0x7f12c8076060 2024-03-21T11:31:46.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.732+0000 7f12c5e00640 1 --2- 172.21.15.47:0/1359349391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c8076060 0x7f12c819df90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f12b800d950 tx=0x7f12b800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.733+0000 7f12bea00640 1 -- 172.21.15.47:0/1359349391 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b8017070 con 0x7f12c8076060 2024-03-21T11:31:46.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.733+0000 7f12bea00640 1 -- 172.21.15.47:0/1359349391 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12b8009c30 con 0x7f12c8076060 2024-03-21T11:31:46.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.733+0000 7f12bea00640 1 -- 172.21.15.47:0/1359349391 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b800f6b0 con 0x7f12c8076060 2024-03-21T11:31:46.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.733+0000 7f12cd000640 1 -- 172.21.15.47:0/1359349391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c81a5ea0 con 0x7f12c8076060 2024-03-21T11:31:46.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.733+0000 7f12cd000640 1 -- 172.21.15.47:0/1359349391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c81a62b0 con 0x7f12c8076060 2024-03-21T11:31:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.736+0000 7f12cfc31740 1 Processor -- start 2024-03-21T11:31:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.736+0000 7f12cfc31740 1 -- start start 2024-03-21T11:31:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.737+0000 7f12bea00640 1 -- 172.21.15.47:0/1359349391 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f12b80109d0 con 0x7f12c8076060 2024-03-21T11:31:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.737+0000 7f12cfc31740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b869810 0x55856b869c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.737+0000 7f12cfc31740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 0x55856b86ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.737+0000 7f12cfc31740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b86b3a0 0x55856b871c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.737+0000 7f12cfc31740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856b7ce2c0 con 0x55856b86a9e0 2024-03-21T11:31:46.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.737+0000 7f12cfc31740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856b776c90 con 0x55856b86b3a0 2024-03-21T11:31:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.737+0000 7f12cfc31740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856b7ca360 con 0x55856b869810 2024-03-21T11:31:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.737+0000 7f12bfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b86b3a0 0x55856b871c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.737+0000 7f12c6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b869810 0x55856b869c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.737+0000 7f12bfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b86b3a0 0x55856b871c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58506/0 (socket says 172.21.15.47:58506) 2024-03-21T11:31:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.737+0000 7f12bfe00640 1 -- 172.21.15.47:0/1230965604 learned_addr learned my addr 172.21.15.47:0/1230965604 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.737+0000 7f12c5e00640 1 --2- 172.21.15.47:0/1230965604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 0x55856b86ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.737+0000 7f12bfe00640 1 -- 172.21.15.47:0/1230965604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b869810 msgr2=0x55856b869c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.738+0000 7f12bfe00640 1 --2- 172.21.15.47:0/1230965604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b869810 0x55856b869c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.738+0000 7f12bfe00640 1 -- 172.21.15.47:0/1230965604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 msgr2=0x55856b86ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.738+0000 7f12bfe00640 1 --2- 172.21.15.47:0/1230965604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 0x55856b86ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.738+0000 7f12bfe00640 1 -- 172.21.15.47:0/1230965604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55856b79b080 con 0x55856b86b3a0 2024-03-21T11:31:46.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.738+0000 7f12bfe00640 1 --2- 172.21.15.47:0/1230965604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b86b3a0 0x55856b871c30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009ca0 tx=0x7f12b0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb250cf90e5bf821 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.738+0000 7f12b7400640 1 -- 172.21.15.47:0/1230965604 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b001e070 con 0x55856b86b3a0 2024-03-21T11:31:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.738+0000 7f12b7400640 1 -- 172.21.15.47:0/1230965604 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12b0019b10 con 0x55856b86b3a0 2024-03-21T11:31:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.738+0000 7f12bea00640 1 --2- 172.21.15.47:0/1359349391 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12ac0a2fa0 0x7f12ac0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.738+0000 7f12b7400640 1 -- 172.21.15.47:0/1230965604 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b000eae0 con 0x55856b86b3a0 2024-03-21T11:31:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.738+0000 7f12bea00640 1 -- 172.21.15.47:0/1359349391 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f12b80cf280 con 0x7f12c8076060 2024-03-21T11:31:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.739+0000 7f12c6800640 1 --2- 172.21.15.47:0/1359349391 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12ac0a2fa0 0x7f12ac0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.739+0000 7f12cfc31740 1 -- 172.21.15.47:0/1230965604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b86b3a0 msgr2=0x55856b871c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.739+0000 7f12cfc31740 1 --2- 172.21.15.47:0/1230965604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b86b3a0 0x55856b871c30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009ca0 tx=0x7f12b0011810 comp rx=0 tx=0).stop 2024-03-21T11:31:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.739+0000 7f12cfc31740 1 -- 172.21.15.47:0/1230965604 shutdown_connections 2024-03-21T11:31:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.739+0000 7f12cfc31740 1 --2- 172.21.15.47:0/1230965604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b86b3a0 0x55856b871c30 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.739+0000 7f12cfc31740 1 --2- 172.21.15.47:0/1230965604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 0x55856b86ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.739+0000 7f12cfc31740 1 --2- 172.21.15.47:0/1230965604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b869810 0x55856b869c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.739+0000 7f12cfc31740 1 -- 172.21.15.47:0/1230965604 >> 172.21.15.47:0/1230965604 conn(0x55856b864c00 msgr2=0x55856b865000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.739+0000 7f12cfc31740 1 -- 172.21.15.47:0/1230965604 shutdown_connections 2024-03-21T11:31:46.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.739+0000 7f12c6800640 1 --2- 172.21.15.47:0/1359349391 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12ac0a2fa0 0x7f12ac0a5460 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f12c002c630 tx=0x7f12c0032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.739+0000 7f12cfc31740 1 -- 172.21.15.47:0/1230965604 wait complete. 2024-03-21T11:31:46.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.740+0000 7f12cfc31740 1 -- start start 2024-03-21T11:31:46.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.740+0000 7f12cfc31740 1 Processor -- start 2024-03-21T11:31:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12cfc31740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b869810 0x55856b873a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12cfc31740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86a9e0 0x55856b873fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12cfc31740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b86b3a0 0x55856b87b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12cfc31740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856b8723b0 con 0x55856b869810 2024-03-21T11:31:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12cfc31740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856b78a0d0 con 0x55856b86b3a0 2024-03-21T11:31:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12cfc31740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856b872530 con 0x55856b86a9e0 2024-03-21T11:31:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12cfc31740 10 client.0 _ll_register_callbacks cb 0x55856b78ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86a9e0 0x55856b873fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12c5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b869810 0x55856b873a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86a9e0 0x55856b873fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44774/0 (socket says 172.21.15.47:44774) 2024-03-21T11:31:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12bfe00640 1 -- 172.21.15.47:0/3152087709 learned_addr learned my addr 172.21.15.47:0/3152087709 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12c6800640 1 --2- 172.21.15.47:0/3152087709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b86b3a0 0x55856b87b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12bfe00640 1 -- 172.21.15.47:0/3152087709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b86b3a0 msgr2=0x55856b87b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12bfe00640 1 --2- 172.21.15.47:0/3152087709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b86b3a0 0x55856b87b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12bfe00640 1 -- 172.21.15.47:0/3152087709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b869810 msgr2=0x55856b873a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12bfe00640 1 --2- 172.21.15.47:0/3152087709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b869810 0x55856b873a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12bfe00640 1 -- 172.21.15.47:0/3152087709 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55856b4e23f0 con 0x55856b86a9e0 2024-03-21T11:31:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.741+0000 7f12c5e00640 1 --2- 172.21.15.47:0/3152087709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b869810 0x55856b873a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.742+0000 7f12bfe00640 1 --2- 172.21.15.47:0/3152087709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86a9e0 0x55856b873fc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009f50 tx=0x7f12b0019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64e9b818c4c57056 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.742+0000 7f12abe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.742+0000 7f12abe00640 1 -- 172.21.15.47:0/3152087709 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0028020 con 0x55856b86a9e0 2024-03-21T11:31:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.742+0000 7f12abe00640 1 -- 172.21.15.47:0/3152087709 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f12b0005050 con 0x55856b86a9e0 2024-03-21T11:31:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.742+0000 7f12cfc31740 1 -- 172.21.15.47:0/3152087709 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12b0019b10 con 0x55856b86a9e0 2024-03-21T11:31:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.742+0000 7f12abe00640 1 -- 172.21.15.47:0/3152087709 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12b0033050 con 0x55856b86a9e0 2024-03-21T11:31:46.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.742+0000 7f12cfc31740 10 client.28345 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.742+0000 7f12cfc31740 1 -- 172.21.15.47:0/3152087709 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55856b5ac7d0 con 0x55856b86a9e0 2024-03-21T11:31:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.743+0000 7f12abe00640 1 -- 172.21.15.47:0/3152087709 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12b0019b10 con 0x55856b86a9e0 2024-03-21T11:31:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.743+0000 7f12abe00640 1 -- 172.21.15.47:0/3152087709 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f12b00046c0 con 0x55856b86a9e0 2024-03-21T11:31:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.743+0000 7f12cfc31740 10 client.28345 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.743+0000 7f12cfc31740 10 client.28345 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:31:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.743+0000 7f12cfc31740 20 client.28345 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.743+0000 7f12cfc31740 1 --2- 172.21.15.47:0/3152087709 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55856b87f210 0x55856b881640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.743+0000 7f12cfc31740 4 client.28345 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.743+0000 7f12cfc31740 1 -- 172.21.15.47:0/3152087709 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55856b7db800 con 0x55856b87f210 2024-03-21T11:31:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.743+0000 7f12c5e00640 1 --2- 172.21.15.47:0/3152087709 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55856b87f210 0x55856b881640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.744+0000 7f12c5e00640 1 --2- 172.21.15.47:0/3152087709 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55856b87f210 0x55856b881640 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=553fa70736e07cb6 server_cookie=e968e27724381cf4 in_seq=0 out_seq=0 2024-03-21T11:31:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.744+0000 7f12abe00640 10 client.28345 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:31:46.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.745+0000 7f12abe00640 1 -- 172.21.15.47:0/3152087709 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55856b7db800 con 0x55856b87f210 2024-03-21T11:31:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.745+0000 7f12abe00640 10 client.28345 handle_command_reply: tid=0 2024-03-21T11:31:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.745+0000 7f12cfc31740 1 client.28345 shutdown 2024-03-21T11:31:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.745+0000 7f12cfc31740 1 -- 172.21.15.47:0/3152087709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86a9e0 msgr2=0x55856b873fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.745+0000 7f12cfc31740 1 --2- 172.21.15.47:0/3152087709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86a9e0 0x55856b873fc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009f50 tx=0x7f12b0019660 comp rx=0 tx=0).stop 2024-03-21T11:31:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.745+0000 7f12cfc31740 1 -- 172.21.15.47:0/3152087709 shutdown_connections 2024-03-21T11:31:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.745+0000 7f12cfc31740 1 --2- 172.21.15.47:0/3152087709 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55856b87f210 0x55856b881640 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.745+0000 7f12cfc31740 1 --2- 172.21.15.47:0/3152087709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b86b3a0 0x55856b87b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.745+0000 7f12cfc31740 1 --2- 172.21.15.47:0/3152087709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86a9e0 0x55856b873fc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.745+0000 7f12cfc31740 1 --2- 172.21.15.47:0/3152087709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b869810 0x55856b873a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.745+0000 7f12abe00640 0 client.28345 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:31:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.745+0000 7f12cfc31740 1 -- 172.21.15.47:0/3152087709 >> 172.21.15.47:0/3152087709 conn(0x55856b864c00 msgr2=0x55856b863eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.746+0000 7f12cfc31740 1 -- 172.21.15.47:0/3152087709 shutdown_connections 2024-03-21T11:31:46.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.746+0000 7f12cfc31740 1 -- 172.21.15.47:0/3152087709 wait complete. 2024-03-21T11:31:46.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.746+0000 7f12cfc31740 20 client.28345 trim_cache size 0 max 16384 2024-03-21T11:31:46.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.761+0000 7f12cfc31740 1 Processor -- start 2024-03-21T11:31:46.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.762+0000 7f12cfc31740 1 -- start start 2024-03-21T11:31:46.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.762+0000 7f12cfc31740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b869810 0x55856b8718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.762+0000 7f12cfc31740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 0x55856b87afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.762+0000 7f12cfc31740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86b3a0 0x55856b86dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.762+0000 7f12cfc31740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c81b0720 con 0x55856b86a9e0 2024-03-21T11:31:46.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.762+0000 7f12cfc31740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c81b20a0 con 0x55856b869810 2024-03-21T11:31:46.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.762+0000 7f12cfc31740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c81b0d80 con 0x55856b86b3a0 2024-03-21T11:31:46.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.762+0000 7f12c6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86b3a0 0x55856b86dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.762+0000 7f12bfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 0x55856b87afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.762+0000 7f12c5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b869810 0x55856b8718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.762+0000 7f12c5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b869810 0x55856b8718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58526/0 (socket says 172.21.15.47:58526) 2024-03-21T11:31:46.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.762+0000 7f12bfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 0x55856b87afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45516/0 (socket says 172.21.15.47:45516) 2024-03-21T11:31:46.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.762+0000 7f12c5e00640 1 -- 172.21.15.47:0/3461462973 learned_addr learned my addr 172.21.15.47:0/3461462973 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.762+0000 7f12c6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86b3a0 0x55856b86dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44778/0 (socket says 172.21.15.47:44778) 2024-03-21T11:31:46.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.763+0000 7f12c6800640 1 -- 172.21.15.47:0/3461462973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b869810 msgr2=0x55856b8718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.763+0000 7f12c6800640 1 --2- 172.21.15.47:0/3461462973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b869810 0x55856b8718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.763+0000 7f12c6800640 1 -- 172.21.15.47:0/3461462973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 msgr2=0x55856b87afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.763+0000 7f12c6800640 1 --2- 172.21.15.47:0/3461462973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 0x55856b87afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.763+0000 7f12c6800640 1 -- 172.21.15.47:0/3461462973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55856b872ea0 con 0x55856b86b3a0 2024-03-21T11:31:46.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.763+0000 7f12c6800640 1 --2- 172.21.15.47:0/3461462973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86b3a0 0x55856b86dc10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f12c0037040 tx=0x7f12c0041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19d23a898ab9a462 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.763+0000 7f12b6a00640 1 -- 172.21.15.47:0/3461462973 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0039040 con 0x55856b86b3a0 2024-03-21T11:31:46.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.763+0000 7f12b6a00640 1 -- 172.21.15.47:0/3461462973 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12c0003e50 con 0x55856b86b3a0 2024-03-21T11:31:46.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.763+0000 7f12b6a00640 1 -- 172.21.15.47:0/3461462973 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c004e460 con 0x55856b86b3a0 2024-03-21T11:31:46.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.764+0000 7f12cfc31740 1 -- 172.21.15.47:0/3461462973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86b3a0 msgr2=0x55856b86dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.764+0000 7f12cfc31740 1 --2- 172.21.15.47:0/3461462973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86b3a0 0x55856b86dc10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f12c0037040 tx=0x7f12c0041000 comp rx=0 tx=0).stop 2024-03-21T11:31:46.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.764+0000 7f12cfc31740 1 -- 172.21.15.47:0/3461462973 shutdown_connections 2024-03-21T11:31:46.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.764+0000 7f12cfc31740 1 --2- 172.21.15.47:0/3461462973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86b3a0 0x55856b86dc10 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.764+0000 7f12cfc31740 1 --2- 172.21.15.47:0/3461462973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 0x55856b87afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.764+0000 7f12cfc31740 1 --2- 172.21.15.47:0/3461462973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b869810 0x55856b8718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.764+0000 7f12cfc31740 1 -- 172.21.15.47:0/3461462973 >> 172.21.15.47:0/3461462973 conn(0x55856b864c00 msgr2=0x55856b8686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.764+0000 7f12cfc31740 1 -- 172.21.15.47:0/3461462973 shutdown_connections 2024-03-21T11:31:46.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.764+0000 7f12cfc31740 1 -- 172.21.15.47:0/3461462973 wait complete. 2024-03-21T11:31:46.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.765+0000 7f12cfc31740 1 -- start start 2024-03-21T11:31:46.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.765+0000 7f12cfc31740 1 Processor -- start 2024-03-21T11:31:46.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.765+0000 7f12cfc31740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b869810 0x55856b878810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.765+0000 7f12cfc31740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 0x55856b878d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.765+0000 7f12cfc31740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86b3a0 0x55856b879d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.765+0000 7f12cfc31740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c81b1a40 con 0x55856b86a9e0 2024-03-21T11:31:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.765+0000 7f12cfc31740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c81b13e0 con 0x55856b869810 2024-03-21T11:31:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.765+0000 7f12cfc31740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c81ae8e0 con 0x55856b86b3a0 2024-03-21T11:31:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.765+0000 7f12cfc31740 10 client.0 _ll_register_callbacks cb 0x7f12b001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.766+0000 7f12c5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b869810 0x55856b878810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.766+0000 7f12c5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b869810 0x55856b878810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58542/0 (socket says 172.21.15.47:58542) 2024-03-21T11:31:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.766+0000 7f12c5e00640 1 -- 172.21.15.47:0/1531213047 learned_addr learned my addr 172.21.15.47:0/1531213047 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.766+0000 7f12bfe00640 1 --2- 172.21.15.47:0/1531213047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 0x55856b878d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.766+0000 7f12c6800640 1 --2- 172.21.15.47:0/1531213047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86b3a0 0x55856b879d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.766+0000 7f12c5e00640 1 -- 172.21.15.47:0/1531213047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86b3a0 msgr2=0x55856b879d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.766+0000 7f12c5e00640 1 --2- 172.21.15.47:0/1531213047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86b3a0 0x55856b879d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.766+0000 7f12c5e00640 1 -- 172.21.15.47:0/1531213047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 msgr2=0x55856b878d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.766+0000 7f12c5e00640 1 --2- 172.21.15.47:0/1531213047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 0x55856b878d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.766+0000 7f12c5e00640 1 -- 172.21.15.47:0/1531213047 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55856b87a350 con 0x55856b869810 2024-03-21T11:31:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.766+0000 7f12c6800640 1 --2- 172.21.15.47:0/1531213047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86b3a0 0x55856b879d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.766+0000 7f12c5e00640 1 --2- 172.21.15.47:0/1531213047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b869810 0x55856b878810 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f12b80ccd60 tx=0x7f12b8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31eb71cb694abdd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.766+0000 7f12abe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.766+0000 7f12abe00640 1 -- 172.21.15.47:0/1531213047 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b8020040 con 0x55856b869810 2024-03-21T11:31:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.767+0000 7f12abe00640 1 -- 172.21.15.47:0/1531213047 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f12b80999d0 con 0x55856b869810 2024-03-21T11:31:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.767+0000 7f12cfc31740 1 -- 172.21.15.47:0/1531213047 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12c0003e50 con 0x55856b869810 2024-03-21T11:31:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.767+0000 7f12abe00640 1 -- 172.21.15.47:0/1531213047 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12b809f770 con 0x55856b869810 2024-03-21T11:31:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.767+0000 7f12cfc31740 10 client.28349 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.767+0000 7f12cfc31740 1 -- 172.21.15.47:0/1531213047 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55856b8dac50 con 0x55856b869810 2024-03-21T11:31:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.767+0000 7f12abe00640 1 -- 172.21.15.47:0/1531213047 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12c0003e50 con 0x55856b869810 2024-03-21T11:31:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.767+0000 7f12abe00640 1 -- 172.21.15.47:0/1531213047 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f12b809ac40 con 0x55856b869810 2024-03-21T11:31:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.767+0000 7f12cfc31740 10 client.28349 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.767+0000 7f12cfc31740 10 client.28349 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:31:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.767+0000 7f12cfc31740 20 client.28349 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.767+0000 7f12cfc31740 1 --2- 172.21.15.47:0/1531213047 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55856b8db4a0 0x55856b8dd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.767+0000 7f12cfc31740 4 client.28349 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:31:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.767+0000 7f12cfc31740 1 -- 172.21.15.47:0/1531213047 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55856b7d5b50 con 0x55856b8db4a0 2024-03-21T11:31:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12bfe00640 1 --2- 172.21.15.47:0/1531213047 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55856b8db4a0 0x55856b8dd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12bfe00640 1 --2- 172.21.15.47:0/1531213047 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55856b8db4a0 0x55856b8dd8c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa25e4b492ff3565 server_cookie=89b53c9f8c9b31 in_seq=0 out_seq=0 2024-03-21T11:31:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12abe00640 10 client.28349 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:31:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12abe00640 1 -- 172.21.15.47:0/1531213047 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55856b7d5b50 con 0x55856b8db4a0 2024-03-21T11:31:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12abe00640 10 client.28349 handle_command_reply: tid=0 2024-03-21T11:31:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12cfc31740 1 client.28349 shutdown 2024-03-21T11:31:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12cfc31740 1 -- 172.21.15.47:0/1531213047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b869810 msgr2=0x55856b878810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12cfc31740 1 --2- 172.21.15.47:0/1531213047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b869810 0x55856b878810 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f12b80ccd60 tx=0x7f12b8000bc0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12cfc31740 1 -- 172.21.15.47:0/1531213047 shutdown_connections 2024-03-21T11:31:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12cfc31740 1 --2- 172.21.15.47:0/1531213047 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55856b8db4a0 0x55856b8dd8c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12cfc31740 1 --2- 172.21.15.47:0/1531213047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55856b86b3a0 0x55856b879d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12cfc31740 1 --2- 172.21.15.47:0/1531213047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55856b86a9e0 0x55856b878d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12cfc31740 1 --2- 172.21.15.47:0/1531213047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55856b869810 0x55856b878810 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12abe00640 0 client.28349 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:31:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.768+0000 7f12cfc31740 1 -- 172.21.15.47:0/1531213047 >> 172.21.15.47:0/1531213047 conn(0x55856b864c00 msgr2=0x55856b8686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.769+0000 7f12cfc31740 1 -- 172.21.15.47:0/1531213047 shutdown_connections 2024-03-21T11:31:46.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.769+0000 7f12cfc31740 1 -- 172.21.15.47:0/1531213047 wait complete. 2024-03-21T11:31:46.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.769+0000 7f12cfc31740 20 client.28349 trim_cache size 0 max 16384 2024-03-21T11:31:46.776 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:31:46.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.769+0000 7f12cd000640 1 -- 172.21.15.47:0/1359349391 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12ac0a2fa0 msgr2=0x7f12ac0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.769+0000 7f12cd000640 1 --2- 172.21.15.47:0/1359349391 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12ac0a2fa0 0x7f12ac0a5460 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f12c002c630 tx=0x7f12c0032590 comp rx=0 tx=0).stop 2024-03-21T11:31:46.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.769+0000 7f12cd000640 1 -- 172.21.15.47:0/1359349391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c8076060 msgr2=0x7f12c819df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:46.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.769+0000 7f12cd000640 1 --2- 172.21.15.47:0/1359349391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c8076060 0x7f12c819df90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f12b800d950 tx=0x7f12b800de10 comp rx=0 tx=0).stop 2024-03-21T11:31:46.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.769+0000 7f12cd000640 1 -- 172.21.15.47:0/1359349391 shutdown_connections 2024-03-21T11:31:46.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.769+0000 7f12cd000640 1 --2- 172.21.15.47:0/1359349391 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f12ac0a2fa0 0x7f12ac0a5460 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.769+0000 7f12cd000640 1 --2- 172.21.15.47:0/1359349391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12c8077c40 0x7f12c81a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.770+0000 7f12cd000640 1 --2- 172.21.15.47:0/1359349391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c80772a0 0x7f12c819e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.770+0000 7f12cd000640 1 --2- 172.21.15.47:0/1359349391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c8076060 0x7f12c819df90 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:46.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.770+0000 7f12cd000640 1 -- 172.21.15.47:0/1359349391 >> 172.21.15.47:0/1359349391 conn(0x7f12c80fee50 msgr2=0x7f12c8100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:46.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.770+0000 7f12cd000640 1 -- 172.21.15.47:0/1359349391 shutdown_connections 2024-03-21T11:31:46.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:46.770+0000 7f12cd000640 1 -- 172.21.15.47:0/1359349391 wait complete. 2024-03-21T11:31:46.787 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:31:46.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:31:47.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:46 smithi047 ceph-mon[31351]: pgmap v653: 129 pgs: 129 active+clean; 40 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-03-21T11:31:47.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.082+0000 7f7d8de00640 1 -- 172.21.15.47:0/2278394646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d88104ad0 msgr2=0x7f7d88104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.082+0000 7f7d8de00640 1 --2- 172.21.15.47:0/2278394646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d88104ad0 0x7f7d88104f50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009860 tx=0x7f7d7402bb70 comp rx=0 tx=0).stop 2024-03-21T11:31:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.082+0000 7f7d8de00640 1 -- 172.21.15.47:0/2278394646 shutdown_connections 2024-03-21T11:31:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.082+0000 7f7d8de00640 1 --2- 172.21.15.47:0/2278394646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d88105490 0x7f7d8810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.082+0000 7f7d8de00640 1 --2- 172.21.15.47:0/2278394646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d88104ad0 0x7f7d88104f50 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.082+0000 7f7d8de00640 1 --2- 172.21.15.47:0/2278394646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d881038e0 0x7f7d88103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.082+0000 7f7d8de00640 1 -- 172.21.15.47:0/2278394646 >> 172.21.15.47:0/2278394646 conn(0x7f7d880fee50 msgr2=0x7f7d88101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.083+0000 7f7d8de00640 1 -- 172.21.15.47:0/2278394646 shutdown_connections 2024-03-21T11:31:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.083+0000 7f7d8de00640 1 -- 172.21.15.47:0/2278394646 wait complete. 2024-03-21T11:31:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.084+0000 7f7d8de00640 1 Processor -- start 2024-03-21T11:31:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.084+0000 7f7d8de00640 1 -- start start 2024-03-21T11:31:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.084+0000 7f7d8de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d881038e0 0x7f7d88199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.084+0000 7f7d8de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d88104ad0 0x7f7d8819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.084+0000 7f7d8de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d88105490 0x7f7d881a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.084+0000 7f7d8de00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8810f540 con 0x7f7d88104ad0 2024-03-21T11:31:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.084+0000 7f7d8de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8810f3c0 con 0x7f7d881038e0 2024-03-21T11:31:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.084+0000 7f7d8de00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8810f6c0 con 0x7f7d88105490 2024-03-21T11:31:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.084+0000 7f7d86a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d881038e0 0x7f7d88199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.084+0000 7f7d86a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d881038e0 0x7f7d88199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58556/0 (socket says 172.21.15.47:58556) 2024-03-21T11:31:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.084+0000 7f7d86a00640 1 -- 172.21.15.47:0/3620218360 learned_addr learned my addr 172.21.15.47:0/3620218360 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.084+0000 7f7d86000640 1 --2- 172.21.15.47:0/3620218360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d88104ad0 0x7f7d8819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.084+0000 7f7d87400640 1 --2- 172.21.15.47:0/3620218360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d88105490 0x7f7d881a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.085+0000 7f7d86a00640 1 -- 172.21.15.47:0/3620218360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d88105490 msgr2=0x7f7d881a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.085+0000 7f7d86a00640 1 --2- 172.21.15.47:0/3620218360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d88105490 0x7f7d881a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.085+0000 7f7d86a00640 1 -- 172.21.15.47:0/3620218360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d88104ad0 msgr2=0x7f7d8819a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.085+0000 7f7d86a00640 1 --2- 172.21.15.47:0/3620218360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d88104ad0 0x7f7d8819a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.085+0000 7f7d86a00640 1 -- 172.21.15.47:0/3620218360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d881a17b0 con 0x7f7d881038e0 2024-03-21T11:31:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.085+0000 7f7d86a00640 1 --2- 172.21.15.47:0/3620218360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d881038e0 0x7f7d88199b90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800e9f0 tx=0x7f7d7800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.085+0000 7f7d8ca00640 1 -- 172.21.15.47:0/3620218360 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800ccc0 con 0x7f7d881038e0 2024-03-21T11:31:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.086+0000 7f7d8de00640 1 -- 172.21.15.47:0/3620218360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d881a1aa0 con 0x7f7d881038e0 2024-03-21T11:31:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.086+0000 7f7d8ca00640 1 -- 172.21.15.47:0/3620218360 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d780035f0 con 0x7f7d881038e0 2024-03-21T11:31:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.086+0000 7f7d8ca00640 1 -- 172.21.15.47:0/3620218360 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800ce60 con 0x7f7d881038e0 2024-03-21T11:31:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.086+0000 7f7d8de00640 1 -- 172.21.15.47:0/3620218360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d881a1fc0 con 0x7f7d881038e0 2024-03-21T11:31:47.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.089+0000 7f7d90a31740 1 Processor -- start 2024-03-21T11:31:47.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.089+0000 7f7d8ca00640 1 -- 172.21.15.47:0/3620218360 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7d780037a0 con 0x7f7d881038e0 2024-03-21T11:31:47.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.089+0000 7f7d90a31740 1 -- start start 2024-03-21T11:31:47.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.089+0000 7f7d90a31740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6a810 0x55c044d6ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.089+0000 7f7d90a31740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6b9e0 0x55c044d6be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d90a31740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6c3a0 0x55c044d72c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d90a31740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c044ccf2c0 con 0x55c044d6c3a0 2024-03-21T11:31:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d90a31740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c044c77c90 con 0x55c044d6b9e0 2024-03-21T11:31:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d90a31740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c044ccb360 con 0x55c044d6a810 2024-03-21T11:31:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d87400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6a810 0x55c044d6ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d87400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6a810 0x55c044d6ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44816/0 (socket says 172.21.15.47:44816) 2024-03-21T11:31:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d87400640 1 -- 172.21.15.47:0/1376896823 learned_addr learned my addr 172.21.15.47:0/1376896823 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d86a00640 1 --2- 172.21.15.47:0/1376896823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6b9e0 0x55c044d6be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d86000640 1 --2- 172.21.15.47:0/1376896823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6c3a0 0x55c044d72c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d87400640 1 -- 172.21.15.47:0/1376896823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6b9e0 msgr2=0x55c044d6be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d87400640 1 --2- 172.21.15.47:0/1376896823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6b9e0 0x55c044d6be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d87400640 1 -- 172.21.15.47:0/1376896823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6c3a0 msgr2=0x55c044d72c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d87400640 1 --2- 172.21.15.47:0/1376896823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6c3a0 0x55c044d72c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d87400640 1 -- 172.21.15.47:0/1376896823 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c044c9c080 con 0x55c044d6a810 2024-03-21T11:31:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.090+0000 7f7d87400640 1 --2- 172.21.15.47:0/1376896823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6a810 0x55c044d6ac30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009fd0 tx=0x7f7d800118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afbe30649fc02c02 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.091+0000 7f7d73e00640 1 -- 172.21.15.47:0/1376896823 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80004d20 con 0x55c044d6a810 2024-03-21T11:31:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.091+0000 7f7d73e00640 1 -- 172.21.15.47:0/1376896823 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d80004050 con 0x55c044d6a810 2024-03-21T11:31:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.091+0000 7f7d73e00640 1 -- 172.21.15.47:0/1376896823 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80005260 con 0x55c044d6a810 2024-03-21T11:31:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.091+0000 7f7d8ca00640 1 --2- 172.21.15.47:0/3620218360 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d6c0a2f20 0x7f7d6c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.091+0000 7f7d8ca00640 1 -- 172.21.15.47:0/3620218360 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7d780cf9a0 con 0x7f7d881038e0 2024-03-21T11:31:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.091+0000 7f7d86a00640 1 --2- 172.21.15.47:0/3620218360 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d6c0a2f20 0x7f7d6c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.092+0000 7f7d90a31740 1 -- 172.21.15.47:0/1376896823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6a810 msgr2=0x55c044d6ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:47.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.092+0000 7f7d90a31740 1 --2- 172.21.15.47:0/1376896823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6a810 0x55c044d6ac30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009fd0 tx=0x7f7d800118d0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.092+0000 7f7d90a31740 1 -- 172.21.15.47:0/1376896823 shutdown_connections 2024-03-21T11:31:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.092+0000 7f7d90a31740 1 --2- 172.21.15.47:0/1376896823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6c3a0 0x55c044d72c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.092+0000 7f7d90a31740 1 --2- 172.21.15.47:0/1376896823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6b9e0 0x55c044d6be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.092+0000 7f7d90a31740 1 --2- 172.21.15.47:0/1376896823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6a810 0x55c044d6ac30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.092+0000 7f7d90a31740 1 -- 172.21.15.47:0/1376896823 >> 172.21.15.47:0/1376896823 conn(0x55c044d65c00 msgr2=0x55c044d66000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.092+0000 7f7d90a31740 1 -- 172.21.15.47:0/1376896823 shutdown_connections 2024-03-21T11:31:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.092+0000 7f7d90a31740 1 -- 172.21.15.47:0/1376896823 wait complete. 2024-03-21T11:31:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.092+0000 7f7d86a00640 1 --2- 172.21.15.47:0/3620218360 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d6c0a2f20 0x7f7d6c0a53e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7d780a4220 tx=0x7f7d780d2f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.092+0000 7f7d90a31740 1 -- start start 2024-03-21T11:31:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.093+0000 7f7d90a31740 1 Processor -- start 2024-03-21T11:31:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.093+0000 7f7d90a31740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d74a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.093+0000 7f7d90a31740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d74f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.093+0000 7f7d90a31740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 0x55c044d7bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.093+0000 7f7d90a31740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c044d733b0 con 0x55c044d6b9e0 2024-03-21T11:31:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.093+0000 7f7d90a31740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c044c8b0d0 con 0x55c044d6a810 2024-03-21T11:31:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.093+0000 7f7d90a31740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c044d73530 con 0x55c044d6c3a0 2024-03-21T11:31:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.093+0000 7f7d90a31740 10 client.0 _ll_register_callbacks cb 0x55c044c8fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.093+0000 7f7d86000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d74f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.093+0000 7f7d87400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d74a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.094+0000 7f7d86000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d74f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45572/0 (socket says 172.21.15.47:45572) 2024-03-21T11:31:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.094+0000 7f7d86000640 1 -- 172.21.15.47:0/131043732 learned_addr learned my addr 172.21.15.47:0/131043732 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.094+0000 7f7d87400640 1 --2- 172.21.15.47:0/131043732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 0x55c044d7bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.094+0000 7f7d86000640 1 -- 172.21.15.47:0/131043732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 msgr2=0x55c044d7bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.094+0000 7f7d86000640 1 --2- 172.21.15.47:0/131043732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 0x55c044d7bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.094+0000 7f7d86000640 1 -- 172.21.15.47:0/131043732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 msgr2=0x55c044d74a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.094+0000 7f7d86000640 1 --2- 172.21.15.47:0/131043732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d74a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.094+0000 7f7d86000640 1 -- 172.21.15.47:0/131043732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0449e33f0 con 0x55c044d6b9e0 2024-03-21T11:31:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.094+0000 7f7d87400640 1 --2- 172.21.15.47:0/131043732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 0x55c044d7bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.094+0000 7f7d86000640 1 --2- 172.21.15.47:0/131043732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d74f90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7d74037040 tx=0x7f7d74005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4402fe1e30e81b08 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.094+0000 7f7d70c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:31:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.095+0000 7f7d70c00640 1 -- 172.21.15.47:0/131043732 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74043070 con 0x55c044d6b9e0 2024-03-21T11:31:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.095+0000 7f7d70c00640 1 -- 172.21.15.47:0/131043732 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7d74004040 con 0x55c044d6b9e0 2024-03-21T11:31:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.095+0000 7f7d90a31740 1 -- 172.21.15.47:0/131043732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d80004050 con 0x55c044d6b9e0 2024-03-21T11:31:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.095+0000 7f7d70c00640 1 -- 172.21.15.47:0/131043732 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d7404f050 con 0x55c044d6b9e0 2024-03-21T11:31:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.095+0000 7f7d90a31740 10 client.18027 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:47.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.095+0000 7f7d90a31740 1 -- 172.21.15.47:0/131043732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c044aad7d0 con 0x55c044d6b9e0 2024-03-21T11:31:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.095+0000 7f7d70c00640 1 -- 172.21.15.47:0/131043732 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d80004050 con 0x55c044d6b9e0 2024-03-21T11:31:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.096+0000 7f7d70c00640 1 -- 172.21.15.47:0/131043732 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7d74032640 con 0x55c044d6b9e0 2024-03-21T11:31:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.096+0000 7f7d90a31740 10 client.18027 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.096+0000 7f7d90a31740 10 client.18027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.096+0000 7f7d90a31740 20 client.18027 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.096+0000 7f7d90a31740 1 --2- 172.21.15.47:0/131043732 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c044d80470 0x55c044d828a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.096+0000 7f7d90a31740 4 client.18027 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.096+0000 7f7d90a31740 1 -- 172.21.15.47:0/131043732 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c044cdc800 con 0x55c044d80470 2024-03-21T11:31:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.096+0000 7f7d87400640 1 --2- 172.21.15.47:0/131043732 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c044d80470 0x55c044d828a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.097+0000 7f7d87400640 1 --2- 172.21.15.47:0/131043732 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c044d80470 0x55c044d828a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78bcd28486506be4 server_cookie=6dcc4bdbc7e0f8d1 in_seq=0 out_seq=0 2024-03-21T11:31:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.097+0000 7f7d70c00640 10 client.18027 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.097+0000 7f7d70c00640 1 -- 172.21.15.47:0/131043732 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c044cdc800 con 0x55c044d80470 2024-03-21T11:31:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.097+0000 7f7d70c00640 10 client.18027 handle_command_reply: tid=0 2024-03-21T11:31:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.097+0000 7f7d90a31740 1 client.18027 shutdown 2024-03-21T11:31:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.098+0000 7f7d90a31740 1 -- 172.21.15.47:0/131043732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 msgr2=0x55c044d74f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.098+0000 7f7d90a31740 1 --2- 172.21.15.47:0/131043732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d74f90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7d74037040 tx=0x7f7d74005b00 comp rx=0 tx=0).stop 2024-03-21T11:31:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.098+0000 7f7d90a31740 1 -- 172.21.15.47:0/131043732 shutdown_connections 2024-03-21T11:31:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.098+0000 7f7d90a31740 1 --2- 172.21.15.47:0/131043732 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c044d80470 0x55c044d828a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.098+0000 7f7d90a31740 1 --2- 172.21.15.47:0/131043732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 0x55c044d7bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.098+0000 7f7d90a31740 1 --2- 172.21.15.47:0/131043732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d74f90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.098+0000 7f7d90a31740 1 --2- 172.21.15.47:0/131043732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d74a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.098+0000 7f7d70c00640 0 client.18027 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.098+0000 7f7d90a31740 1 -- 172.21.15.47:0/131043732 >> 172.21.15.47:0/131043732 conn(0x55c044d65c00 msgr2=0x55c044d64eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:47.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.098+0000 7f7d90a31740 1 -- 172.21.15.47:0/131043732 shutdown_connections 2024-03-21T11:31:47.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.098+0000 7f7d90a31740 1 -- 172.21.15.47:0/131043732 wait complete. 2024-03-21T11:31:47.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.098+0000 7f7d90a31740 20 client.18027 trim_cache size 0 max 16384 2024-03-21T11:31:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.112+0000 7f7d90a31740 1 Processor -- start 2024-03-21T11:31:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.112+0000 7f7d90a31740 1 -- start start 2024-03-21T11:31:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.112+0000 7f7d90a31740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d60d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.112+0000 7f7d90a31740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d7bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.112+0000 7f7d90a31740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 0x55c044d6ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.112+0000 7f7d90a31740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d881ac320 con 0x55c044d6b9e0 2024-03-21T11:31:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.112+0000 7f7d90a31740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d881adca0 con 0x55c044d6a810 2024-03-21T11:31:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.112+0000 7f7d90a31740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d881ac980 con 0x55c044d6c3a0 2024-03-21T11:31:47.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.113+0000 7f7d86000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d7bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.113+0000 7f7d87400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d60d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.113+0000 7f7d87400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d60d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58582/0 (socket says 172.21.15.47:58582) 2024-03-21T11:31:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.113+0000 7f7d86000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d7bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45574/0 (socket says 172.21.15.47:45574) 2024-03-21T11:31:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.113+0000 7f7d87400640 1 -- 172.21.15.47:0/1253903386 learned_addr learned my addr 172.21.15.47:0/1253903386 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.113+0000 7f7d87400640 1 --2- 172.21.15.47:0/1253903386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 0x55c044d6ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.113+0000 7f7d87400640 1 -- 172.21.15.47:0/1253903386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 msgr2=0x55c044d6ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.113+0000 7f7d87400640 1 --2- 172.21.15.47:0/1253903386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 0x55c044d6ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.113+0000 7f7d87400640 1 -- 172.21.15.47:0/1253903386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 msgr2=0x55c044d7bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.113+0000 7f7d87400640 1 --2- 172.21.15.47:0/1253903386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d7bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.113+0000 7f7d87400640 1 -- 172.21.15.47:0/1253903386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c044d61510 con 0x55c044d6a810 2024-03-21T11:31:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.113+0000 7f7d86000640 1 --2- 172.21.15.47:0/1253903386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d7bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.113+0000 7f7d87400640 1 --2- 172.21.15.47:0/1253903386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d60d40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7d80006010 tx=0x7f7d80009ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c63b22ce0fead55f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.114+0000 7f7d73400640 1 -- 172.21.15.47:0/1253903386 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80004050 con 0x55c044d6a810 2024-03-21T11:31:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.114+0000 7f7d73400640 1 -- 172.21.15.47:0/1253903386 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d800046d0 con 0x55c044d6a810 2024-03-21T11:31:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.114+0000 7f7d73400640 1 -- 172.21.15.47:0/1253903386 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8002fcd0 con 0x55c044d6a810 2024-03-21T11:31:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.114+0000 7f7d90a31740 1 -- 172.21.15.47:0/1253903386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 msgr2=0x55c044d60d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.114+0000 7f7d90a31740 1 --2- 172.21.15.47:0/1253903386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d60d40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7d80006010 tx=0x7f7d80009ac0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.114+0000 7f7d90a31740 1 -- 172.21.15.47:0/1253903386 shutdown_connections 2024-03-21T11:31:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.114+0000 7f7d90a31740 1 --2- 172.21.15.47:0/1253903386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 0x55c044d6ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.114+0000 7f7d90a31740 1 --2- 172.21.15.47:0/1253903386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d7bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.114+0000 7f7d90a31740 1 --2- 172.21.15.47:0/1253903386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d60d40 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.114+0000 7f7d90a31740 1 -- 172.21.15.47:0/1253903386 >> 172.21.15.47:0/1253903386 conn(0x55c044d65c00 msgr2=0x55c044d696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.115+0000 7f7d90a31740 1 -- 172.21.15.47:0/1253903386 shutdown_connections 2024-03-21T11:31:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.115+0000 7f7d90a31740 1 -- 172.21.15.47:0/1253903386 wait complete. 2024-03-21T11:31:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.115+0000 7f7d90a31740 1 -- start start 2024-03-21T11:31:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.115+0000 7f7d90a31740 1 Processor -- start 2024-03-21T11:31:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d90a31740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d79f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d90a31740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d7a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d90a31740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 0x55c044d7b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d90a31740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d881ad640 con 0x55c044d6b9e0 2024-03-21T11:31:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d90a31740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d881acfe0 con 0x55c044d6a810 2024-03-21T11:31:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d90a31740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d881aa4e0 con 0x55c044d6c3a0 2024-03-21T11:31:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d90a31740 10 client.0 _ll_register_callbacks cb 0x7f7d800093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d87400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d79f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d87400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d79f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58596/0 (socket says 172.21.15.47:58596) 2024-03-21T11:31:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d86000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d7a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d87400640 1 -- 172.21.15.47:0/3678976967 learned_addr learned my addr 172.21.15.47:0/3678976967 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d87400640 1 --2- 172.21.15.47:0/3678976967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 0x55c044d7b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d86000640 1 -- 172.21.15.47:0/3678976967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 msgr2=0x55c044d7b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d86000640 1 --2- 172.21.15.47:0/3678976967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 0x55c044d7b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d86000640 1 -- 172.21.15.47:0/3678976967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 msgr2=0x55c044d79f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d86000640 1 --2- 172.21.15.47:0/3678976967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d79f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d86000640 1 -- 172.21.15.47:0/3678976967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c044d7baa0 con 0x55c044d6b9e0 2024-03-21T11:31:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.116+0000 7f7d87400640 1 --2- 172.21.15.47:0/3678976967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d79f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.117+0000 7f7d86000640 1 --2- 172.21.15.47:0/3678976967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d7a4a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7d74032a10 tx=0x7f7d74032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97f09d8a0c6b5c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.117+0000 7f7d70c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:31:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.117+0000 7f7d70c00640 1 -- 172.21.15.47:0/3678976967 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74059070 con 0x55c044d6b9e0 2024-03-21T11:31:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.117+0000 7f7d70c00640 1 -- 172.21.15.47:0/3678976967 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7d74004040 con 0x55c044d6b9e0 2024-03-21T11:31:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.117+0000 7f7d90a31740 1 -- 172.21.15.47:0/3678976967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d800046d0 con 0x55c044d6b9e0 2024-03-21T11:31:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.117+0000 7f7d70c00640 1 -- 172.21.15.47:0/3678976967 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d74062050 con 0x55c044d6b9e0 2024-03-21T11:31:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.117+0000 7f7d90a31740 10 client.18033 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.117+0000 7f7d90a31740 1 -- 172.21.15.47:0/3678976967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c044ddc680 con 0x55c044d6b9e0 2024-03-21T11:31:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.118+0000 7f7d70c00640 1 -- 172.21.15.47:0/3678976967 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d800046d0 con 0x55c044d6b9e0 2024-03-21T11:31:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.118+0000 7f7d70c00640 1 -- 172.21.15.47:0/3678976967 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7d7405c020 con 0x55c044d6b9e0 2024-03-21T11:31:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.118+0000 7f7d90a31740 10 client.18033 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.118+0000 7f7d90a31740 10 client.18033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.118+0000 7f7d90a31740 20 client.18033 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.118+0000 7f7d90a31740 1 --2- 172.21.15.47:0/3678976967 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c044ddcfc0 0x55c044ddf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.118+0000 7f7d90a31740 4 client.18033 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:31:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.118+0000 7f7d90a31740 1 -- 172.21.15.47:0/3678976967 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c044cd6b50 con 0x55c044ddcfc0 2024-03-21T11:31:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.118+0000 7f7d87400640 1 --2- 172.21.15.47:0/3678976967 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c044ddcfc0 0x55c044ddf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.119+0000 7f7d87400640 1 --2- 172.21.15.47:0/3678976967 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c044ddcfc0 0x55c044ddf3e0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73d033120452b83e server_cookie=23ce3e6d63907a88 in_seq=0 out_seq=0 2024-03-21T11:31:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.119+0000 7f7d70c00640 10 client.18033 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.119+0000 7f7d70c00640 1 -- 172.21.15.47:0/3678976967 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c044cd6b50 con 0x55c044ddcfc0 2024-03-21T11:31:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.119+0000 7f7d70c00640 10 client.18033 handle_command_reply: tid=0 2024-03-21T11:31:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.119+0000 7f7d90a31740 1 client.18033 shutdown 2024-03-21T11:31:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.120+0000 7f7d90a31740 1 -- 172.21.15.47:0/3678976967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 msgr2=0x55c044d7a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.120+0000 7f7d90a31740 1 --2- 172.21.15.47:0/3678976967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d7a4a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7d74032a10 tx=0x7f7d74032e10 comp rx=0 tx=0).stop 2024-03-21T11:31:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.120+0000 7f7d90a31740 1 -- 172.21.15.47:0/3678976967 shutdown_connections 2024-03-21T11:31:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.120+0000 7f7d90a31740 1 --2- 172.21.15.47:0/3678976967 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c044ddcfc0 0x55c044ddf3e0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.120+0000 7f7d90a31740 1 --2- 172.21.15.47:0/3678976967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c044d6c3a0 0x55c044d7b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.120+0000 7f7d90a31740 1 --2- 172.21.15.47:0/3678976967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c044d6b9e0 0x55c044d7a4a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.120+0000 7f7d90a31740 1 --2- 172.21.15.47:0/3678976967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c044d6a810 0x55c044d79f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.120+0000 7f7d70c00640 0 client.18033 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.120+0000 7f7d90a31740 1 -- 172.21.15.47:0/3678976967 >> 172.21.15.47:0/3678976967 conn(0x55c044d65c00 msgr2=0x55c044d64620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.120+0000 7f7d90a31740 1 -- 172.21.15.47:0/3678976967 shutdown_connections 2024-03-21T11:31:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.120+0000 7f7d90a31740 1 -- 172.21.15.47:0/3678976967 wait complete. 2024-03-21T11:31:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.120+0000 7f7d90a31740 20 client.18033 trim_cache size 0 max 16384 2024-03-21T11:31:47.125 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:31:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.121+0000 7f7d8de00640 1 -- 172.21.15.47:0/3620218360 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d6c0a2f20 msgr2=0x7f7d6c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.121+0000 7f7d8de00640 1 --2- 172.21.15.47:0/3620218360 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d6c0a2f20 0x7f7d6c0a53e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7d780a4220 tx=0x7f7d780d2f90 comp rx=0 tx=0).stop 2024-03-21T11:31:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.121+0000 7f7d8de00640 1 -- 172.21.15.47:0/3620218360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d881038e0 msgr2=0x7f7d88199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.121+0000 7f7d8de00640 1 --2- 172.21.15.47:0/3620218360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d881038e0 0x7f7d88199b90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800e9f0 tx=0x7f7d7800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.121+0000 7f7d8de00640 1 -- 172.21.15.47:0/3620218360 shutdown_connections 2024-03-21T11:31:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.121+0000 7f7d8de00640 1 --2- 172.21.15.47:0/3620218360 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d6c0a2f20 0x7f7d6c0a53e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.121+0000 7f7d8de00640 1 --2- 172.21.15.47:0/3620218360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d88105490 0x7f7d881a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.121+0000 7f7d8de00640 1 --2- 172.21.15.47:0/3620218360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d88104ad0 0x7f7d8819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.121+0000 7f7d8de00640 1 --2- 172.21.15.47:0/3620218360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d881038e0 0x7f7d88199b90 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.121+0000 7f7d8de00640 1 -- 172.21.15.47:0/3620218360 >> 172.21.15.47:0/3620218360 conn(0x7f7d880fee50 msgr2=0x7f7d88107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.121+0000 7f7d8de00640 1 -- 172.21.15.47:0/3620218360 shutdown_connections 2024-03-21T11:31:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:47.121+0000 7f7d8de00640 1 -- 172.21.15.47:0/3620218360 wait complete. 2024-03-21T11:31:47.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:46 smithi100 ceph-mon[36245]: pgmap v653: 129 pgs: 129 active+clean; 40 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-03-21T11:31:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:46 smithi102 ceph-mon[36748]: pgmap v653: 129 pgs: 129 active+clean; 40 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-03-21T11:31:47.539 INFO:tasks.workunit.client.0.smithi047.stdout: 1 604062 8.20 MB/sec execute 271 sec latency 50.972 ms 2024-03-21T11:31:48.140 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:31:48.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:31:48.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:31:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.346+0000 7fc590400640 1 -- 172.21.15.47:0/700522551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc588074100 msgr2=0x7fc588074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.346+0000 7fc590400640 1 --2- 172.21.15.47:0/700522551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc588074100 0x7fc588074580 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc574009900 tx=0x7fc57402bc50 comp rx=0 tx=0).stop 2024-03-21T11:31:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.347+0000 7fc590400640 1 -- 172.21.15.47:0/700522551 shutdown_connections 2024-03-21T11:31:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.347+0000 7fc590400640 1 --2- 172.21.15.47:0/700522551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc58810dc90 0x7fc588110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.347+0000 7fc590400640 1 --2- 172.21.15.47:0/700522551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc588074100 0x7fc588074580 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.347+0000 7fc590400640 1 --2- 172.21.15.47:0/700522551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5880756e0 0x7fc588073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.347+0000 7fc590400640 1 -- 172.21.15.47:0/700522551 >> 172.21.15.47:0/700522551 conn(0x7fc5880fcc30 msgr2=0x7fc5880ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.347+0000 7fc590400640 1 -- 172.21.15.47:0/700522551 shutdown_connections 2024-03-21T11:31:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.347+0000 7fc590400640 1 -- 172.21.15.47:0/700522551 wait complete. 2024-03-21T11:31:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.348+0000 7fc590400640 1 Processor -- start 2024-03-21T11:31:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.349+0000 7fc590400640 1 -- start start 2024-03-21T11:31:48.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.349+0000 7fc590400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc588074100 0x7fc58819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.349+0000 7fc590400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5880756e0 0x7fc58819e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.350+0000 7fc590400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc58810dc90 0x7fc5881a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.350+0000 7fc590400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc588113960 con 0x7fc58810dc90 2024-03-21T11:31:48.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.350+0000 7fc590400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5881137e0 con 0x7fc5880756e0 2024-03-21T11:31:48.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.350+0000 7fc590400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc588113ae0 con 0x7fc588074100 2024-03-21T11:31:48.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.350+0000 7fc58da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc588074100 0x7fc58819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.350+0000 7fc58da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc588074100 0x7fc58819dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44860/0 (socket says 172.21.15.47:44860) 2024-03-21T11:31:48.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.350+0000 7fc58da00640 1 -- 172.21.15.47:0/734505885 learned_addr learned my addr 172.21.15.47:0/734505885 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:48.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.350+0000 7fc58d000640 1 --2- 172.21.15.47:0/734505885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5880756e0 0x7fc58819e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.350+0000 7fc58e400640 1 --2- 172.21.15.47:0/734505885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc58810dc90 0x7fc5881a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.350+0000 7fc58da00640 1 -- 172.21.15.47:0/734505885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5880756e0 msgr2=0x7fc58819e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.350+0000 7fc58da00640 1 --2- 172.21.15.47:0/734505885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5880756e0 0x7fc58819e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.350+0000 7fc58da00640 1 -- 172.21.15.47:0/734505885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc58810dc90 msgr2=0x7fc5881a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.350+0000 7fc58da00640 1 --2- 172.21.15.47:0/734505885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc58810dc90 0x7fc5881a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.350+0000 7fc58da00640 1 -- 172.21.15.47:0/734505885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5881a5bd0 con 0x7fc588074100 2024-03-21T11:31:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.351+0000 7fc58da00640 1 --2- 172.21.15.47:0/734505885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc588074100 0x7fc58819dfb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00db80 tx=0x7fc57c00b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.351+0000 7fc586000640 1 -- 172.21.15.47:0/734505885 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c017070 con 0x7fc588074100 2024-03-21T11:31:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.351+0000 7fc586000640 1 -- 172.21.15.47:0/734505885 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc57c003b70 con 0x7fc588074100 2024-03-21T11:31:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.351+0000 7fc590400640 1 -- 172.21.15.47:0/734505885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5881a5ec0 con 0x7fc588074100 2024-03-21T11:31:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.351+0000 7fc586000640 1 -- 172.21.15.47:0/734505885 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c0106c0 con 0x7fc588074100 2024-03-21T11:31:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.351+0000 7fc590400640 1 -- 172.21.15.47:0/734505885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5881a6330 con 0x7fc588074100 2024-03-21T11:31:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.355+0000 7fc586000640 1 -- 172.21.15.47:0/734505885 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc57c004020 con 0x7fc588074100 2024-03-21T11:31:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.355+0000 7fc5931c1740 1 Processor -- start 2024-03-21T11:31:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.356+0000 7fc5931c1740 1 -- start start 2024-03-21T11:31:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.356+0000 7fc5931c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd5d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.356+0000 7fc5931c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd5da90 0x56114bd62590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.356+0000 7fc5931c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd62c00 0x56114bd65090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.356+0000 7fc5931c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bcc2eb0 con 0x56114bd62c00 2024-03-21T11:31:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.356+0000 7fc5931c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bc6ba60 con 0x56114bd5d0a0 2024-03-21T11:31:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.356+0000 7fc5931c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bcbef50 con 0x56114bd5da90 2024-03-21T11:31:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.356+0000 7fc58e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd5d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.356+0000 7fc58e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd5d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58634/0 (socket says 172.21.15.47:58634) 2024-03-21T11:31:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.356+0000 7fc58e400640 1 -- 172.21.15.47:0/3231265475 learned_addr learned my addr 172.21.15.47:0/3231265475 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.356+0000 7fc58da00640 1 --2- 172.21.15.47:0/3231265475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd5da90 0x56114bd62590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.357+0000 7fc58d000640 1 --2- 172.21.15.47:0/3231265475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd62c00 0x56114bd65090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.357+0000 7fc58da00640 1 -- 172.21.15.47:0/3231265475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 msgr2=0x56114bd5d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.357+0000 7fc58da00640 1 --2- 172.21.15.47:0/3231265475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd5d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.357+0000 7fc58da00640 1 -- 172.21.15.47:0/3231265475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd62c00 msgr2=0x56114bd65090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.357+0000 7fc58da00640 1 --2- 172.21.15.47:0/3231265475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd62c00 0x56114bd65090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.357+0000 7fc58da00640 1 -- 172.21.15.47:0/3231265475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56114ba9f7e0 con 0x56114bd5da90 2024-03-21T11:31:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.357+0000 7fc58da00640 1 --2- 172.21.15.47:0/3231265475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd5da90 0x56114bd62590 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0ccc60 tx=0x7fc57c0d5b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69796d051fe4c6e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.358+0000 7fc586000640 1 --2- 172.21.15.47:0/734505885 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5700a2fa0 0x7fc5700a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.358+0000 7fc57aa00640 1 -- 172.21.15.47:0/3231265475 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c0e2070 con 0x56114bd5da90 2024-03-21T11:31:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.358+0000 7fc57aa00640 1 -- 172.21.15.47:0/3231265475 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc57c0cfba0 con 0x56114bd5da90 2024-03-21T11:31:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.358+0000 7fc57aa00640 1 -- 172.21.15.47:0/3231265475 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c09b250 con 0x56114bd5da90 2024-03-21T11:31:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.358+0000 7fc586000640 1 -- 172.21.15.47:0/734505885 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc57c0cf5b0 con 0x7fc588074100 2024-03-21T11:31:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.358+0000 7fc58e400640 1 --2- 172.21.15.47:0/734505885 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5700a2fa0 0x7fc5700a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.359+0000 7fc5931c1740 1 -- 172.21.15.47:0/3231265475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd5da90 msgr2=0x56114bd62590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.359+0000 7fc5931c1740 1 --2- 172.21.15.47:0/3231265475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd5da90 0x56114bd62590 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0ccc60 tx=0x7fc57c0d5b10 comp rx=0 tx=0).stop 2024-03-21T11:31:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.359+0000 7fc58e400640 1 --2- 172.21.15.47:0/734505885 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5700a2fa0 0x7fc5700a5460 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x56114bd5e2f0 tx=0x7fc580005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.359+0000 7fc5931c1740 1 -- 172.21.15.47:0/3231265475 shutdown_connections 2024-03-21T11:31:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.359+0000 7fc5931c1740 1 --2- 172.21.15.47:0/3231265475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd62c00 0x56114bd65090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.359+0000 7fc5931c1740 1 --2- 172.21.15.47:0/3231265475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd5da90 0x56114bd62590 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.359+0000 7fc5931c1740 1 --2- 172.21.15.47:0/3231265475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd5d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.359+0000 7fc5931c1740 1 -- 172.21.15.47:0/3231265475 >> 172.21.15.47:0/3231265475 conn(0x56114bd58610 msgr2=0x56114bd58a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.359+0000 7fc5931c1740 1 -- 172.21.15.47:0/3231265475 shutdown_connections 2024-03-21T11:31:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.359+0000 7fc5931c1740 1 -- 172.21.15.47:0/3231265475 wait complete. 2024-03-21T11:31:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.360+0000 7fc5931c1740 1 -- start start 2024-03-21T11:31:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.360+0000 7fc5931c1740 1 Processor -- start 2024-03-21T11:31:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.361+0000 7fc5931c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd67100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.361+0000 7fc5931c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 0x56114bd67640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.361+0000 7fc5931c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd62c00 0x56114bd6e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.361+0000 7fc5931c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bd65a30 con 0x56114bd5da90 2024-03-21T11:31:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.361+0000 7fc5931c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bc7eeb0 con 0x56114bd5d0a0 2024-03-21T11:31:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.361+0000 7fc5931c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114bd65bb0 con 0x56114bd62c00 2024-03-21T11:31:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.361+0000 7fc5931c1740 10 client.0 _ll_register_callbacks cb 0x56114bc73150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.361+0000 7fc58e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd62c00 0x56114bd6e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.361+0000 7fc58da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd67100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.361+0000 7fc58e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd62c00 0x56114bd6e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44874/0 (socket says 172.21.15.47:44874) 2024-03-21T11:31:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.361+0000 7fc58e400640 1 -- 172.21.15.47:0/1073461772 learned_addr learned my addr 172.21.15.47:0/1073461772 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.361+0000 7fc58d000640 1 --2- 172.21.15.47:0/1073461772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 0x56114bd67640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.361+0000 7fc58da00640 1 -- 172.21.15.47:0/1073461772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd62c00 msgr2=0x56114bd6e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.362+0000 7fc58da00640 1 --2- 172.21.15.47:0/1073461772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd62c00 0x56114bd6e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.362+0000 7fc58da00640 1 -- 172.21.15.47:0/1073461772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 msgr2=0x56114bd67640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.362+0000 7fc58da00640 1 --2- 172.21.15.47:0/1073461772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 0x56114bd67640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.362+0000 7fc58da00640 1 -- 172.21.15.47:0/1073461772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56114bafff70 con 0x56114bd5d0a0 2024-03-21T11:31:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.362+0000 7fc58e400640 1 --2- 172.21.15.47:0/1073461772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd62c00 0x56114bd6e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.362+0000 7fc58da00640 1 --2- 172.21.15.47:0/1073461772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd67100 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x56114bd5de90 tx=0x7fc57c00be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d71df3b78262c6b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.362+0000 7fc56f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.362+0000 7fc56f400640 1 -- 172.21.15.47:0/1073461772 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c099c90 con 0x56114bd5d0a0 2024-03-21T11:31:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.362+0000 7fc56f400640 1 -- 172.21.15.47:0/1073461772 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc57c0cf9a0 con 0x56114bd5d0a0 2024-03-21T11:31:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.362+0000 7fc5931c1740 1 -- 172.21.15.47:0/1073461772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc57c0cfba0 con 0x56114bd5d0a0 2024-03-21T11:31:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.363+0000 7fc56f400640 1 -- 172.21.15.47:0/1073461772 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc57c0b4050 con 0x56114bd5d0a0 2024-03-21T11:31:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.363+0000 7fc5931c1740 10 client.28379 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.363+0000 7fc5931c1740 1 -- 172.21.15.47:0/1073461772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56114b997170 con 0x56114bd5d0a0 2024-03-21T11:31:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.363+0000 7fc56f400640 1 -- 172.21.15.47:0/1073461772 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc57c0cfba0 con 0x56114bd5d0a0 2024-03-21T11:31:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.363+0000 7fc56f400640 1 -- 172.21.15.47:0/1073461772 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc57c0cf230 con 0x56114bd5d0a0 2024-03-21T11:31:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.363+0000 7fc5931c1740 10 client.28379 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.363+0000 7fc5931c1740 10 client.28379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.363+0000 7fc5931c1740 20 client.28379 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.363+0000 7fc5931c1740 1 --2- 172.21.15.47:0/1073461772 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56114bd72b70 0x56114bd74f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.363+0000 7fc5931c1740 4 client.28379 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.363+0000 7fc5931c1740 1 -- 172.21.15.47:0/1073461772 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56114bcd03f0 con 0x56114bd72b70 2024-03-21T11:31:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.364+0000 7fc58d000640 1 --2- 172.21.15.47:0/1073461772 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56114bd72b70 0x56114bd74f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.364+0000 7fc58d000640 1 --2- 172.21.15.47:0/1073461772 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56114bd72b70 0x56114bd74f60 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7254506031acec9 server_cookie=2774e346b81771a6 in_seq=0 out_seq=0 2024-03-21T11:31:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.364+0000 7fc56f400640 10 client.28379 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.365+0000 7fc56f400640 1 -- 172.21.15.47:0/1073461772 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56114bcd03f0 con 0x56114bd72b70 2024-03-21T11:31:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.365+0000 7fc56f400640 10 client.28379 handle_command_reply: tid=0 2024-03-21T11:31:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.365+0000 7fc5931c1740 1 client.28379 shutdown 2024-03-21T11:31:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.365+0000 7fc5931c1740 1 -- 172.21.15.47:0/1073461772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 msgr2=0x56114bd67100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.365+0000 7fc5931c1740 1 --2- 172.21.15.47:0/1073461772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd67100 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x56114bd5de90 tx=0x7fc57c00be60 comp rx=0 tx=0).stop 2024-03-21T11:31:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.365+0000 7fc5931c1740 1 -- 172.21.15.47:0/1073461772 shutdown_connections 2024-03-21T11:31:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.365+0000 7fc5931c1740 1 --2- 172.21.15.47:0/1073461772 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56114bd72b70 0x56114bd74f60 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.365+0000 7fc5931c1740 1 --2- 172.21.15.47:0/1073461772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd62c00 0x56114bd6e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.365+0000 7fc5931c1740 1 --2- 172.21.15.47:0/1073461772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 0x56114bd67640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.365+0000 7fc56f400640 0 client.28379 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.365+0000 7fc5931c1740 1 --2- 172.21.15.47:0/1073461772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd67100 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.365+0000 7fc5931c1740 1 -- 172.21.15.47:0/1073461772 >> 172.21.15.47:0/1073461772 conn(0x56114bd58610 msgr2=0x56114bd57830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.365+0000 7fc5931c1740 1 -- 172.21.15.47:0/1073461772 shutdown_connections 2024-03-21T11:31:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.365+0000 7fc5931c1740 1 -- 172.21.15.47:0/1073461772 wait complete. 2024-03-21T11:31:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.366+0000 7fc5931c1740 20 client.28379 trim_cache size 0 max 16384 2024-03-21T11:31:48.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.384+0000 7fc5931c1740 1 Processor -- start 2024-03-21T11:31:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.384+0000 7fc5931c1740 1 -- start start 2024-03-21T11:31:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc5931c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd5e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc5931c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 0x56114bd6e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc5931c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd62c00 0x56114bd5ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc5931c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5881b0690 con 0x56114bd5da90 2024-03-21T11:31:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc5931c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5881b2010 con 0x56114bd5d0a0 2024-03-21T11:31:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc5931c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5881b0cf0 con 0x56114bd62c00 2024-03-21T11:31:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc58da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd5e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc58e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd62c00 0x56114bd5ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc58da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd5e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58650/0 (socket says 172.21.15.47:58650) 2024-03-21T11:31:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc58da00640 1 -- 172.21.15.47:0/4168494974 learned_addr learned my addr 172.21.15.47:0/4168494974 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc58d000640 1 --2- 172.21.15.47:0/4168494974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 0x56114bd6e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc58da00640 1 -- 172.21.15.47:0/4168494974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd62c00 msgr2=0x56114bd5ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc58da00640 1 --2- 172.21.15.47:0/4168494974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd62c00 0x56114bd5ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc58da00640 1 -- 172.21.15.47:0/4168494974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 msgr2=0x56114bd6e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc58da00640 1 --2- 172.21.15.47:0/4168494974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 0x56114bd6e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc58da00640 1 -- 172.21.15.47:0/4168494974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56114bd52ae0 con 0x56114bd5d0a0 2024-03-21T11:31:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc58e400640 1 --2- 172.21.15.47:0/4168494974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd62c00 0x56114bd5ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.385+0000 7fc58d000640 1 --2- 172.21.15.47:0/4168494974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 0x56114bd6e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:48.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.386+0000 7fc58da00640 1 --2- 172.21.15.47:0/4168494974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd5e500 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00db50 tx=0x7fc57c0048a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=547a02a95aa2debf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.386+0000 7fc57a000640 1 -- 172.21.15.47:0/4168494974 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c099eb0 con 0x56114bd5d0a0 2024-03-21T11:31:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.386+0000 7fc57a000640 1 -- 172.21.15.47:0/4168494974 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc57c0ce020 con 0x56114bd5d0a0 2024-03-21T11:31:48.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.386+0000 7fc57a000640 1 -- 172.21.15.47:0/4168494974 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c09a0a0 con 0x56114bd5d0a0 2024-03-21T11:31:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.387+0000 7fc5931c1740 1 -- 172.21.15.47:0/4168494974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 msgr2=0x56114bd5e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.387+0000 7fc5931c1740 1 --2- 172.21.15.47:0/4168494974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd5e500 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00db50 tx=0x7fc57c0048a0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.387+0000 7fc5931c1740 1 -- 172.21.15.47:0/4168494974 shutdown_connections 2024-03-21T11:31:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.387+0000 7fc5931c1740 1 --2- 172.21.15.47:0/4168494974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd62c00 0x56114bd5ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.387+0000 7fc5931c1740 1 --2- 172.21.15.47:0/4168494974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 0x56114bd6e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.387+0000 7fc5931c1740 1 --2- 172.21.15.47:0/4168494974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd5d0a0 0x56114bd5e500 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.387+0000 7fc5931c1740 1 -- 172.21.15.47:0/4168494974 >> 172.21.15.47:0/4168494974 conn(0x56114bd58610 msgr2=0x56114bd5c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:48.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.387+0000 7fc5931c1740 1 -- 172.21.15.47:0/4168494974 shutdown_connections 2024-03-21T11:31:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.387+0000 7fc5931c1740 1 -- 172.21.15.47:0/4168494974 wait complete. 2024-03-21T11:31:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.387+0000 7fc5931c1740 1 -- start start 2024-03-21T11:31:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.388+0000 7fc5931c1740 1 Processor -- start 2024-03-21T11:31:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.388+0000 7fc5931c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd5d0a0 0x56114bd6e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.388+0000 7fc5931c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 0x56114bd6c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.388+0000 7fc5931c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd62c00 0x56114bd6cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.388+0000 7fc5931c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5881b19b0 con 0x56114bd5da90 2024-03-21T11:31:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.388+0000 7fc5931c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5881b1350 con 0x56114bd62c00 2024-03-21T11:31:48.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.388+0000 7fc5931c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5881ae850 con 0x56114bd5d0a0 2024-03-21T11:31:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.388+0000 7fc5931c1740 10 client.0 _ll_register_callbacks cb 0x7fc57c0e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.388+0000 7fc58da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd5d0a0 0x56114bd6e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.388+0000 7fc58d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 0x56114bd6c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.388+0000 7fc58e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd62c00 0x56114bd6cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.388+0000 7fc58da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd5d0a0 0x56114bd6e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44890/0 (socket says 172.21.15.47:44890) 2024-03-21T11:31:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.388+0000 7fc58da00640 1 -- 172.21.15.47:0/3882567770 learned_addr learned my addr 172.21.15.47:0/3882567770 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.389+0000 7fc58da00640 1 -- 172.21.15.47:0/3882567770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd62c00 msgr2=0x56114bd6cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.389+0000 7fc58da00640 1 --2- 172.21.15.47:0/3882567770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd62c00 0x56114bd6cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.389+0000 7fc58da00640 1 -- 172.21.15.47:0/3882567770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 msgr2=0x56114bd6c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.389+0000 7fc58da00640 1 --2- 172.21.15.47:0/3882567770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 0x56114bd6c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.389+0000 7fc58da00640 1 -- 172.21.15.47:0/3882567770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56114bd6d2d0 con 0x56114bd5d0a0 2024-03-21T11:31:48.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.389+0000 7fc58da00640 1 --2- 172.21.15.47:0/3882567770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd5d0a0 0x56114bd6e180 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00bb30 tx=0x7fc57c0d2b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a1c57a8ed0f2b10 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:48.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.389+0000 7fc56f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:48.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.389+0000 7fc56f400640 1 -- 172.21.15.47:0/3882567770 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c09a420 con 0x56114bd5d0a0 2024-03-21T11:31:48.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.389+0000 7fc56f400640 1 -- 172.21.15.47:0/3882567770 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc57c0cf270 con 0x56114bd5d0a0 2024-03-21T11:31:48.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.389+0000 7fc5931c1740 1 -- 172.21.15.47:0/3882567770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc57c0ce020 con 0x56114bd5d0a0 2024-03-21T11:31:48.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.389+0000 7fc56f400640 1 -- 172.21.15.47:0/3882567770 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc57c0b8030 con 0x56114bd5d0a0 2024-03-21T11:31:48.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.389+0000 7fc5931c1740 10 client.28387 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:48.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.389+0000 7fc5931c1740 1 -- 172.21.15.47:0/3882567770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56114bd6d2d0 con 0x56114bd5d0a0 2024-03-21T11:31:48.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.390+0000 7fc56f400640 1 -- 172.21.15.47:0/3882567770 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc57c0ce020 con 0x56114bd5d0a0 2024-03-21T11:31:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.390+0000 7fc56f400640 1 -- 172.21.15.47:0/3882567770 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc57c0ba2c0 con 0x56114bd5d0a0 2024-03-21T11:31:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.390+0000 7fc5931c1740 10 client.28387 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.390+0000 7fc5931c1740 10 client.28387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.390+0000 7fc5931c1740 20 client.28387 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.390+0000 7fc5931c1740 1 --2- 172.21.15.47:0/3882567770 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56114bdcf4b0 0x56114bdd18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.390+0000 7fc5931c1740 4 client.28387 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:31:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.390+0000 7fc5931c1740 1 -- 172.21.15.47:0/3882567770 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56114bcca740 con 0x56114bdcf4b0 2024-03-21T11:31:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.390+0000 7fc58d000640 1 --2- 172.21.15.47:0/3882567770 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56114bdcf4b0 0x56114bdd18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.391+0000 7fc58d000640 1 --2- 172.21.15.47:0/3882567770 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56114bdcf4b0 0x56114bdd18a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b03e69ade2215845 server_cookie=a0b57a71521fe9b3 in_seq=0 out_seq=0 2024-03-21T11:31:48.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.391+0000 7fc56f400640 10 client.28387 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:48.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc56f400640 1 -- 172.21.15.47:0/3882567770 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56114bcca740 con 0x56114bdcf4b0 2024-03-21T11:31:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc56f400640 10 client.28387 handle_command_reply: tid=0 2024-03-21T11:31:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc5931c1740 1 client.28387 shutdown 2024-03-21T11:31:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc5931c1740 1 -- 172.21.15.47:0/3882567770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd5d0a0 msgr2=0x56114bd6e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc5931c1740 1 --2- 172.21.15.47:0/3882567770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd5d0a0 0x56114bd6e180 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00bb30 tx=0x7fc57c0d2b60 comp rx=0 tx=0).stop 2024-03-21T11:31:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc5931c1740 1 -- 172.21.15.47:0/3882567770 shutdown_connections 2024-03-21T11:31:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc5931c1740 1 --2- 172.21.15.47:0/3882567770 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56114bdcf4b0 0x56114bdd18a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc5931c1740 1 --2- 172.21.15.47:0/3882567770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56114bd62c00 0x56114bd6cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc5931c1740 1 --2- 172.21.15.47:0/3882567770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56114bd5da90 0x56114bd6c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc5931c1740 1 --2- 172.21.15.47:0/3882567770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56114bd5d0a0 0x56114bd6e180 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc5931c1740 1 -- 172.21.15.47:0/3882567770 >> 172.21.15.47:0/3882567770 conn(0x56114bd58610 msgr2=0x56114bd56fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc56f400640 0 client.28387 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:48.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc5931c1740 1 -- 172.21.15.47:0/3882567770 shutdown_connections 2024-03-21T11:31:48.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc5931c1740 1 -- 172.21.15.47:0/3882567770 wait complete. 2024-03-21T11:31:48.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.393+0000 7fc5931c1740 20 client.28387 trim_cache size 0 max 16384 2024-03-21T11:31:48.399 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:31:48.399 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:31:48.399 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "1674dfd2-71aa-4617-a27c-42876be4eaa3", 2024-03-21T11:31:48.399 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:31:48.399 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:31:48.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.394+0000 7fc590400640 1 -- 172.21.15.47:0/734505885 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5700a2fa0 msgr2=0x7fc5700a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.394+0000 7fc590400640 1 --2- 172.21.15.47:0/734505885 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5700a2fa0 0x7fc5700a5460 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x56114bd5e2f0 tx=0x7fc580005f90 comp rx=0 tx=0).stop 2024-03-21T11:31:48.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.394+0000 7fc590400640 1 -- 172.21.15.47:0/734505885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc588074100 msgr2=0x7fc58819dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.394+0000 7fc590400640 1 --2- 172.21.15.47:0/734505885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc588074100 0x7fc58819dfb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00db80 tx=0x7fc57c00b630 comp rx=0 tx=0).stop 2024-03-21T11:31:48.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.394+0000 7fc590400640 1 -- 172.21.15.47:0/734505885 shutdown_connections 2024-03-21T11:31:48.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.394+0000 7fc590400640 1 --2- 172.21.15.47:0/734505885 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5700a2fa0 0x7fc5700a5460 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.394+0000 7fc590400640 1 --2- 172.21.15.47:0/734505885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc58810dc90 0x7fc5881a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.394+0000 7fc590400640 1 --2- 172.21.15.47:0/734505885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5880756e0 0x7fc58819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.394+0000 7fc590400640 1 --2- 172.21.15.47:0/734505885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc588074100 0x7fc58819dfb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.394+0000 7fc590400640 1 -- 172.21.15.47:0/734505885 >> 172.21.15.47:0/734505885 conn(0x7fc5880fcc30 msgr2=0x7fc588101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:48.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.394+0000 7fc590400640 1 -- 172.21.15.47:0/734505885 shutdown_connections 2024-03-21T11:31:48.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.394+0000 7fc590400640 1 -- 172.21.15.47:0/734505885 wait complete. 2024-03-21T11:31:48.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:31:48.414 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:31:48.539 INFO:tasks.workunit.client.0.smithi047.stdout: 1 605256 8.19 MB/sec execute 272 sec latency 15.453 ms 2024-03-21T11:31:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.645+0000 7f0ca7000640 1 -- 172.21.15.47:0/2662755384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ca01015a0 msgr2=0x7f0ca01019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.645+0000 7f0ca7000640 1 --2- 172.21.15.47:0/2662755384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ca01015a0 0x7f0ca01019c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009900 tx=0x7f0c9402bc30 comp rx=0 tx=0).stop 2024-03-21T11:31:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.646+0000 7f0ca7000640 1 -- 172.21.15.47:0/2662755384 shutdown_connections 2024-03-21T11:31:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.646+0000 7f0ca7000640 1 --2- 172.21.15.47:0/2662755384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ca0103150 0x7f0ca01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.646+0000 7f0ca7000640 1 --2- 172.21.15.47:0/2662755384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca0102790 0x7f0ca0102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.646+0000 7f0ca7000640 1 --2- 172.21.15.47:0/2662755384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ca01015a0 0x7f0ca01019c0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.646+0000 7f0ca7000640 1 -- 172.21.15.47:0/2662755384 >> 172.21.15.47:0/2662755384 conn(0x7f0ca00fcb10 msgr2=0x7f0ca00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.646+0000 7f0ca7000640 1 -- 172.21.15.47:0/2662755384 shutdown_connections 2024-03-21T11:31:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.646+0000 7f0ca7000640 1 -- 172.21.15.47:0/2662755384 wait complete. 2024-03-21T11:31:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.647+0000 7f0ca7000640 1 Processor -- start 2024-03-21T11:31:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.647+0000 7f0ca7000640 1 -- start start 2024-03-21T11:31:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.647+0000 7f0ca7000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ca01015a0 0x7f0ca0199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.647+0000 7f0ca7000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ca0102790 0x7f0ca019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.647+0000 7f0ca7000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca0103150 0x7f0ca01a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.647+0000 7f0ca7000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca010d240 con 0x7f0ca01015a0 2024-03-21T11:31:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.647+0000 7f0ca7000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca010d0c0 con 0x7f0ca0103150 2024-03-21T11:31:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.647+0000 7f0ca7000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca010d3c0 con 0x7f0ca0102790 2024-03-21T11:31:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.647+0000 7f0ca5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ca0102790 0x7f0ca019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.648+0000 7f0ca5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ca0102790 0x7f0ca019a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44910/0 (socket says 172.21.15.47:44910) 2024-03-21T11:31:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.648+0000 7f0ca5c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ca01015a0 0x7f0ca0199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.648+0000 7f0ca6600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca0103150 0x7f0ca01a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.648+0000 7f0ca5200640 1 -- 172.21.15.47:0/2738911197 learned_addr learned my addr 172.21.15.47:0/2738911197 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.648+0000 7f0ca6600640 1 -- 172.21.15.47:0/2738911197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ca0102790 msgr2=0x7f0ca019a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.648+0000 7f0ca6600640 1 --2- 172.21.15.47:0/2738911197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ca0102790 0x7f0ca019a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.648+0000 7f0ca6600640 1 -- 172.21.15.47:0/2738911197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ca01015a0 msgr2=0x7f0ca0199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.648+0000 7f0ca6600640 1 --2- 172.21.15.47:0/2738911197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ca01015a0 0x7f0ca0199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.648+0000 7f0ca6600640 1 -- 172.21.15.47:0/2738911197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca01a1720 con 0x7f0ca0103150 2024-03-21T11:31:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.648+0000 7f0ca5c00640 1 --2- 172.21.15.47:0/2738911197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ca01015a0 0x7f0ca0199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.648+0000 7f0ca6600640 1 --2- 172.21.15.47:0/2738911197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca0103150 0x7f0ca01a10b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009fd0 tx=0x7f0c9800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.649+0000 7f0c9e000640 1 -- 172.21.15.47:0/2738911197 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800ce20 con 0x7f0ca0103150 2024-03-21T11:31:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.649+0000 7f0ca7000640 1 -- 172.21.15.47:0/2738911197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca01a1a10 con 0x7f0ca0103150 2024-03-21T11:31:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.649+0000 7f0c9e000640 1 -- 172.21.15.47:0/2738911197 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c98005020 con 0x7f0ca0103150 2024-03-21T11:31:48.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.649+0000 7f0c9e000640 1 -- 172.21.15.47:0/2738911197 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98010720 con 0x7f0ca0103150 2024-03-21T11:31:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.649+0000 7f0ca7000640 1 -- 172.21.15.47:0/2738911197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca01a1f60 con 0x7f0ca0103150 2024-03-21T11:31:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.653+0000 7f0cab391740 1 Processor -- start 2024-03-21T11:31:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.653+0000 7f0c9e000640 1 -- 172.21.15.47:0/2738911197 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0c98005540 con 0x7f0ca0103150 2024-03-21T11:31:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.654+0000 7f0cab391740 1 -- start start 2024-03-21T11:31:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.654+0000 7f0cab391740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393bab5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.654+0000 7f0cab391740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bab5a40 0x56393baba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.654+0000 7f0cab391740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bababb0 0x56393babd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.654+0000 7f0cab391740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393ba1a250 con 0x56393bab5050 2024-03-21T11:31:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.654+0000 7f0cab391740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393b9c2a30 con 0x56393bab5a40 2024-03-21T11:31:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.654+0000 7f0cab391740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393ba162f0 con 0x56393bababb0 2024-03-21T11:31:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.654+0000 7f0ca5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bab5a40 0x56393baba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.654+0000 7f0ca5200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393bab5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.654+0000 7f0ca5200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393bab5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45662/0 (socket says 172.21.15.47:45662) 2024-03-21T11:31:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.654+0000 7f0ca5200640 1 -- 172.21.15.47:0/785172462 learned_addr learned my addr 172.21.15.47:0/785172462 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.654+0000 7f0ca5200640 1 --2- 172.21.15.47:0/785172462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bababb0 0x56393babd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.655+0000 7f0ca5c00640 1 -- 172.21.15.47:0/785172462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bababb0 msgr2=0x56393babd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.655+0000 7f0ca5c00640 1 --2- 172.21.15.47:0/785172462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bababb0 0x56393babd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.655+0000 7f0ca5c00640 1 -- 172.21.15.47:0/785172462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 msgr2=0x56393bab5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.655+0000 7f0ca5c00640 1 --2- 172.21.15.47:0/785172462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393bab5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.655+0000 7f0ca5c00640 1 -- 172.21.15.47:0/785172462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56393b9ad690 con 0x56393bab5a40 2024-03-21T11:31:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.655+0000 7f0ca5c00640 1 --2- 172.21.15.47:0/785172462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bab5a40 0x56393baba540 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009230 tx=0x7f0c94009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1b9db475af16a9a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.655+0000 7f0c9e000640 1 --2- 172.21.15.47:0/2738911197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c880a2fa0 0x7f0c880a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.655+0000 7f0c92a00640 1 -- 172.21.15.47:0/785172462 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94045070 con 0x56393bab5a40 2024-03-21T11:31:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.656+0000 7f0c92a00640 1 -- 172.21.15.47:0/785172462 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c9402cd30 con 0x56393bab5a40 2024-03-21T11:31:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.656+0000 7f0c92a00640 1 -- 172.21.15.47:0/785172462 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c940050d0 con 0x56393bab5a40 2024-03-21T11:31:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.656+0000 7f0c9e000640 1 -- 172.21.15.47:0/2738911197 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0c980cfdc0 con 0x7f0ca0103150 2024-03-21T11:31:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.656+0000 7f0ca5200640 1 --2- 172.21.15.47:0/2738911197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c880a2fa0 0x7f0c880a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.656+0000 7f0cab391740 1 -- 172.21.15.47:0/785172462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bab5a40 msgr2=0x56393baba540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.656+0000 7f0cab391740 1 --2- 172.21.15.47:0/785172462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bab5a40 0x56393baba540 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009230 tx=0x7f0c94009630 comp rx=0 tx=0).stop 2024-03-21T11:31:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.656+0000 7f0cab391740 1 -- 172.21.15.47:0/785172462 shutdown_connections 2024-03-21T11:31:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.656+0000 7f0cab391740 1 --2- 172.21.15.47:0/785172462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bababb0 0x56393babd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.656+0000 7f0cab391740 1 --2- 172.21.15.47:0/785172462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bab5a40 0x56393baba540 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.656+0000 7f0cab391740 1 --2- 172.21.15.47:0/785172462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393bab5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.656+0000 7f0cab391740 1 -- 172.21.15.47:0/785172462 >> 172.21.15.47:0/785172462 conn(0x56393ba7d200 msgr2=0x56393bab09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.657+0000 7f0cab391740 1 -- 172.21.15.47:0/785172462 shutdown_connections 2024-03-21T11:31:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.657+0000 7f0ca5200640 1 --2- 172.21.15.47:0/2738911197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c880a2fa0 0x7f0c880a5460 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x56393bab62a0 tx=0x7f0c8c004120 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.657+0000 7f0cab391740 1 -- 172.21.15.47:0/785172462 wait complete. 2024-03-21T11:31:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.657+0000 7f0cab391740 1 -- start start 2024-03-21T11:31:48.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.657+0000 7f0cab391740 1 Processor -- start 2024-03-21T11:31:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.658+0000 7f0cab391740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393babf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.658+0000 7f0cab391740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393babf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.658+0000 7f0cab391740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bababb0 0x56393bac6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.658+0000 7f0cab391740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393babdb00 con 0x56393bab5050 2024-03-21T11:31:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.658+0000 7f0cab391740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393b9d5e80 con 0x56393bababb0 2024-03-21T11:31:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.658+0000 7f0cab391740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393babdc80 con 0x56393bab5a40 2024-03-21T11:31:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.658+0000 7f0cab391740 10 client.0 _ll_register_callbacks cb 0x56393b9e8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.658+0000 7f0ca5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393babf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.658+0000 7f0ca5c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393babf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.658+0000 7f0ca6600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bababb0 0x56393bac6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.658+0000 7f0ca5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393babf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44932/0 (socket says 172.21.15.47:44932) 2024-03-21T11:31:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.658+0000 7f0ca5200640 1 -- 172.21.15.47:0/1228627505 learned_addr learned my addr 172.21.15.47:0/1228627505 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.659+0000 7f0ca5200640 1 -- 172.21.15.47:0/1228627505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bababb0 msgr2=0x56393bac6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.659+0000 7f0ca5200640 1 --2- 172.21.15.47:0/1228627505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bababb0 0x56393bac6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.659+0000 7f0ca5200640 1 -- 172.21.15.47:0/1228627505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 msgr2=0x56393babf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.659+0000 7f0ca5200640 1 --2- 172.21.15.47:0/1228627505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393babf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.659+0000 7f0ca5200640 1 -- 172.21.15.47:0/1228627505 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56393b7f87d0 con 0x56393bab5a40 2024-03-21T11:31:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.659+0000 7f0ca6600640 1 --2- 172.21.15.47:0/1228627505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bababb0 0x56393bac6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.659+0000 7f0ca5200640 1 --2- 172.21.15.47:0/1228627505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393babf5c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c010880 tx=0x7f0c8c00b4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2edb44b5bfb850d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.659+0000 7f0c87400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.659+0000 7f0c87400640 1 -- 172.21.15.47:0/1228627505 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c010e00 con 0x56393bab5a40 2024-03-21T11:31:48.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.659+0000 7f0cab391740 1 -- 172.21.15.47:0/1228627505 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c9402cd30 con 0x56393bab5a40 2024-03-21T11:31:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.659+0000 7f0c87400640 1 -- 172.21.15.47:0/1228627505 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0c8c01b030 con 0x56393bab5a40 2024-03-21T11:31:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.660+0000 7f0c87400640 1 -- 172.21.15.47:0/1228627505 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c8c034050 con 0x56393bab5a40 2024-03-21T11:31:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.660+0000 7f0cab391740 10 client.28399 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.660+0000 7f0cab391740 1 -- 172.21.15.47:0/1228627505 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56393b72e3f0 con 0x56393bab5a40 2024-03-21T11:31:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.660+0000 7f0c87400640 1 -- 172.21.15.47:0/1228627505 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c9402cd30 con 0x56393bab5a40 2024-03-21T11:31:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.660+0000 7f0c87400640 1 -- 172.21.15.47:0/1228627505 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0c8c01e3e0 con 0x56393bab5a40 2024-03-21T11:31:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.660+0000 7f0cab391740 10 client.28399 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.660+0000 7f0cab391740 10 client.28399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.660+0000 7f0cab391740 20 client.28399 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.660+0000 7f0cab391740 1 --2- 172.21.15.47:0/1228627505 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56393bacaea0 0x56393bacd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.660+0000 7f0cab391740 4 client.28399 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.660+0000 7f0cab391740 1 -- 172.21.15.47:0/1228627505 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56393ba27790 con 0x56393bacaea0 2024-03-21T11:31:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.661+0000 7f0ca5c00640 1 --2- 172.21.15.47:0/1228627505 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56393bacaea0 0x56393bacd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.661+0000 7f0ca5c00640 1 --2- 172.21.15.47:0/1228627505 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56393bacaea0 0x56393bacd310 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88fe79df51eeaf86 server_cookie=cb2d2178d32fb73a in_seq=0 out_seq=0 2024-03-21T11:31:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.661+0000 7f0c87400640 10 client.28399 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.663+0000 7f0c87400640 1 -- 172.21.15.47:0/1228627505 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56393ba27790 con 0x56393bacaea0 2024-03-21T11:31:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.663+0000 7f0c87400640 10 client.28399 handle_command_reply: tid=0 2024-03-21T11:31:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.663+0000 7f0cab391740 1 client.28399 shutdown 2024-03-21T11:31:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.664+0000 7f0cab391740 1 -- 172.21.15.47:0/1228627505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 msgr2=0x56393babf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.664+0000 7f0cab391740 1 --2- 172.21.15.47:0/1228627505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393babf5c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c010880 tx=0x7f0c8c00b4a0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.664+0000 7f0cab391740 1 -- 172.21.15.47:0/1228627505 shutdown_connections 2024-03-21T11:31:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.664+0000 7f0cab391740 1 --2- 172.21.15.47:0/1228627505 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56393bacaea0 0x56393bacd310 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.664+0000 7f0cab391740 1 --2- 172.21.15.47:0/1228627505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bababb0 0x56393bac6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.664+0000 7f0cab391740 1 --2- 172.21.15.47:0/1228627505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393babf5c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.664+0000 7f0cab391740 1 --2- 172.21.15.47:0/1228627505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393babf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.664+0000 7f0c87400640 0 client.28399 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.664+0000 7f0cab391740 1 -- 172.21.15.47:0/1228627505 >> 172.21.15.47:0/1228627505 conn(0x56393ba7d200 msgr2=0x56393baafbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:48.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.664+0000 7f0cab391740 1 -- 172.21.15.47:0/1228627505 shutdown_connections 2024-03-21T11:31:48.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.664+0000 7f0cab391740 1 -- 172.21.15.47:0/1228627505 wait complete. 2024-03-21T11:31:48.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.664+0000 7f0cab391740 20 client.28399 trim_cache size 0 max 16384 2024-03-21T11:31:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.682+0000 7f0cab391740 1 Processor -- start 2024-03-21T11:31:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.682+0000 7f0cab391740 1 -- start start 2024-03-21T11:31:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.682+0000 7f0cab391740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393bab4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.682+0000 7f0cab391740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393bac65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.682+0000 7f0cab391740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bababb0 0x56393baaf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.682+0000 7f0cab391740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01ac090 con 0x56393bab5050 2024-03-21T11:31:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.682+0000 7f0cab391740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01ada10 con 0x56393bababb0 2024-03-21T11:31:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.682+0000 7f0cab391740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01ac6f0 con 0x56393bab5a40 2024-03-21T11:31:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.682+0000 7f0ca5c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393bab4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.682+0000 7f0ca5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393bac65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.682+0000 7f0ca5c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393bab4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45682/0 (socket says 172.21.15.47:45682) 2024-03-21T11:31:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.682+0000 7f0ca5c00640 1 -- 172.21.15.47:0/2613813116 learned_addr learned my addr 172.21.15.47:0/2613813116 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.683+0000 7f0ca6600640 1 --2- 172.21.15.47:0/2613813116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bababb0 0x56393baaf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.683+0000 7f0ca5c00640 1 -- 172.21.15.47:0/2613813116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 msgr2=0x56393bac65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.683+0000 7f0ca5c00640 1 --2- 172.21.15.47:0/2613813116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393bac65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.683+0000 7f0ca5c00640 1 -- 172.21.15.47:0/2613813116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bababb0 msgr2=0x56393baaf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.683+0000 7f0ca5c00640 1 --2- 172.21.15.47:0/2613813116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bababb0 0x56393baaf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.683+0000 7f0ca5c00640 1 -- 172.21.15.47:0/2613813116 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56393b6ee0f0 con 0x56393bab5050 2024-03-21T11:31:48.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.683+0000 7f0ca5c00640 1 --2- 172.21.15.47:0/2613813116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393bab4310 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x56393bab64b0 tx=0x7f0c9402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d36d34d7ffb231d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:48.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.683+0000 7f0c92000640 1 -- 172.21.15.47:0/2613813116 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94045040 con 0x56393bab5050 2024-03-21T11:31:48.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.683+0000 7f0c92000640 1 -- 172.21.15.47:0/2613813116 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c9402cd30 con 0x56393bab5050 2024-03-21T11:31:48.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.683+0000 7f0c92000640 1 -- 172.21.15.47:0/2613813116 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c940051d0 con 0x56393bab5050 2024-03-21T11:31:48.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.684+0000 7f0cab391740 1 -- 172.21.15.47:0/2613813116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 msgr2=0x56393bab4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.684+0000 7f0cab391740 1 --2- 172.21.15.47:0/2613813116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393bab4310 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x56393bab64b0 tx=0x7f0c9402bc30 comp rx=0 tx=0).stop 2024-03-21T11:31:48.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.684+0000 7f0cab391740 1 -- 172.21.15.47:0/2613813116 shutdown_connections 2024-03-21T11:31:48.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.684+0000 7f0cab391740 1 --2- 172.21.15.47:0/2613813116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bababb0 0x56393baaf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.684+0000 7f0cab391740 1 --2- 172.21.15.47:0/2613813116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393bac65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.684+0000 7f0cab391740 1 --2- 172.21.15.47:0/2613813116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bab5050 0x56393bab4310 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.684+0000 7f0cab391740 1 -- 172.21.15.47:0/2613813116 >> 172.21.15.47:0/2613813116 conn(0x56393ba7d200 msgr2=0x56393bab3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.684+0000 7f0cab391740 1 -- 172.21.15.47:0/2613813116 shutdown_connections 2024-03-21T11:31:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.684+0000 7f0cab391740 1 -- 172.21.15.47:0/2613813116 wait complete. 2024-03-21T11:31:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.685+0000 7f0cab391740 1 -- start start 2024-03-21T11:31:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.685+0000 7f0cab391740 1 Processor -- start 2024-03-21T11:31:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.685+0000 7f0cab391740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bab5050 0x56393bac3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.685+0000 7f0cab391740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393bac4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.685+0000 7f0cab391740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bababb0 0x56393bac5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.685+0000 7f0cab391740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01ad3b0 con 0x56393bababb0 2024-03-21T11:31:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.685+0000 7f0cab391740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01acd50 con 0x56393bab5050 2024-03-21T11:31:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.685+0000 7f0cab391740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01aa250 con 0x56393bab5a40 2024-03-21T11:31:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.685+0000 7f0cab391740 10 client.0 _ll_register_callbacks cb 0x7f0c8c010500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.686+0000 7f0ca5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bab5050 0x56393bac3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.686+0000 7f0ca5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393bac4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.686+0000 7f0ca5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bab5050 0x56393bac3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58702/0 (socket says 172.21.15.47:58702) 2024-03-21T11:31:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.686+0000 7f0ca5c00640 1 -- 172.21.15.47:0/1137751779 learned_addr learned my addr 172.21.15.47:0/1137751779 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.686+0000 7f0ca6600640 1 --2- 172.21.15.47:0/1137751779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bababb0 0x56393bac5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.686+0000 7f0ca5c00640 1 -- 172.21.15.47:0/1137751779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 msgr2=0x56393bac4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.686+0000 7f0ca5c00640 1 --2- 172.21.15.47:0/1137751779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393bac4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.686+0000 7f0ca5c00640 1 -- 172.21.15.47:0/1137751779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bababb0 msgr2=0x56393bac5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:48.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.686+0000 7f0ca5c00640 1 --2- 172.21.15.47:0/1137751779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bababb0 0x56393bac5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.686+0000 7f0ca5c00640 1 -- 172.21.15.47:0/1137751779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56393bac5660 con 0x56393bab5050 2024-03-21T11:31:48.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.686+0000 7f0ca6600640 1 --2- 172.21.15.47:0/1137751779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bababb0 0x56393bac5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:48.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.686+0000 7f0ca5200640 1 --2- 172.21.15.47:0/1137751779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393bac4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:31:48.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.686+0000 7f0ca5c00640 1 --2- 172.21.15.47:0/1137751779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bab5050 0x56393bac3b20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009900 tx=0x7f0c94005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d0cc538ae9def66 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:48.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.686+0000 7f0c87400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:48.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.687+0000 7f0c87400640 1 -- 172.21.15.47:0/1137751779 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94045040 con 0x56393bab5050 2024-03-21T11:31:48.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.687+0000 7f0c87400640 1 -- 172.21.15.47:0/1137751779 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0c94005150 con 0x56393bab5050 2024-03-21T11:31:48.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.687+0000 7f0cab391740 1 -- 172.21.15.47:0/1137751779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c9402cd30 con 0x56393bab5050 2024-03-21T11:31:48.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.687+0000 7f0c87400640 1 -- 172.21.15.47:0/1137751779 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c9405f050 con 0x56393bab5050 2024-03-21T11:31:48.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.687+0000 7f0cab391740 10 client.28403 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:48.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.687+0000 7f0cab391740 1 -- 172.21.15.47:0/1137751779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56393bac6970 con 0x56393bab5050 2024-03-21T11:31:48.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.687+0000 7f0c87400640 1 -- 172.21.15.47:0/1137751779 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c9402cd30 con 0x56393bab5050 2024-03-21T11:31:48.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.687+0000 7f0c87400640 1 -- 172.21.15.47:0/1137751779 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0c94053040 con 0x56393bab5050 2024-03-21T11:31:48.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.687+0000 7f0cab391740 10 client.28403 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:48.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.687+0000 7f0cab391740 10 client.28403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:48.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.687+0000 7f0cab391740 20 client.28403 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:48.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.687+0000 7f0cab391740 1 --2- 172.21.15.47:0/1137751779 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56393bb26a20 0x56393bb28e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:48.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.687+0000 7f0cab391740 4 client.28403 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:31:48.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.687+0000 7f0cab391740 1 -- 172.21.15.47:0/1137751779 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56393ba21ae0 con 0x56393bb26a20 2024-03-21T11:31:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.688+0000 7f0ca5200640 1 --2- 172.21.15.47:0/1137751779 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56393bb26a20 0x56393bb28e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.688+0000 7f0ca5200640 1 --2- 172.21.15.47:0/1137751779 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56393bb26a20 0x56393bb28e10 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e52efc594adbd1f server_cookie=e5fd9b5cc1d2fc49 in_seq=0 out_seq=0 2024-03-21T11:31:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.688+0000 7f0c87400640 10 client.28403 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.688+0000 7f0c87400640 1 -- 172.21.15.47:0/1137751779 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56393ba21ae0 con 0x56393bb26a20 2024-03-21T11:31:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.688+0000 7f0c87400640 10 client.28403 handle_command_reply: tid=0 2024-03-21T11:31:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.688+0000 7f0cab391740 1 client.28403 shutdown 2024-03-21T11:31:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.689+0000 7f0cab391740 1 -- 172.21.15.47:0/1137751779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bab5050 msgr2=0x56393bac3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.689+0000 7f0cab391740 1 --2- 172.21.15.47:0/1137751779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bab5050 0x56393bac3b20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009900 tx=0x7f0c94005d50 comp rx=0 tx=0).stop 2024-03-21T11:31:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.689+0000 7f0cab391740 1 -- 172.21.15.47:0/1137751779 shutdown_connections 2024-03-21T11:31:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.689+0000 7f0cab391740 1 --2- 172.21.15.47:0/1137751779 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56393bb26a20 0x56393bb28e10 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.689+0000 7f0cab391740 1 --2- 172.21.15.47:0/1137751779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56393bababb0 0x56393bac5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.689+0000 7f0cab391740 1 --2- 172.21.15.47:0/1137751779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56393bab5a40 0x56393bac4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.689+0000 7f0cab391740 1 --2- 172.21.15.47:0/1137751779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56393bab5050 0x56393bac3b20 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.689+0000 7f0cab391740 1 -- 172.21.15.47:0/1137751779 >> 172.21.15.47:0/1137751779 conn(0x56393ba7d200 msgr2=0x56393bab6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.689+0000 7f0c87400640 0 client.28403 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.689+0000 7f0cab391740 1 -- 172.21.15.47:0/1137751779 shutdown_connections 2024-03-21T11:31:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.689+0000 7f0cab391740 1 -- 172.21.15.47:0/1137751779 wait complete. 2024-03-21T11:31:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.689+0000 7f0cab391740 20 client.28403 trim_cache size 0 max 16384 2024-03-21T11:31:48.696 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:31:48.697 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:31:48.697 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:31:48.697 INFO:teuthology.orchestra.run.smithi047.stdout: "1674dfd2-71aa-4617-a27c-42876be4eaa3": { 2024-03-21T11:31:48.697 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:31:48.697 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "1674dfd2-71aa-4617-a27c-42876be4eaa3", 2024-03-21T11:31:48.697 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:31:48.697 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:31:48.697 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:31:48.697 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:31:48.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.690+0000 7f0ca7000640 1 -- 172.21.15.47:0/2738911197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c880a2fa0 msgr2=0x7f0c880a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.690+0000 7f0ca7000640 1 --2- 172.21.15.47:0/2738911197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c880a2fa0 0x7f0c880a5460 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x56393bab62a0 tx=0x7f0c8c004120 comp rx=0 tx=0).stop 2024-03-21T11:31:48.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.690+0000 7f0ca7000640 1 -- 172.21.15.47:0/2738911197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca0103150 msgr2=0x7f0ca01a10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:48.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.690+0000 7f0ca7000640 1 --2- 172.21.15.47:0/2738911197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca0103150 0x7f0ca01a10b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009fd0 tx=0x7f0c9800ef10 comp rx=0 tx=0).stop 2024-03-21T11:31:48.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.690+0000 7f0ca7000640 1 -- 172.21.15.47:0/2738911197 shutdown_connections 2024-03-21T11:31:48.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.690+0000 7f0ca7000640 1 --2- 172.21.15.47:0/2738911197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c880a2fa0 0x7f0c880a5460 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.690+0000 7f0ca7000640 1 --2- 172.21.15.47:0/2738911197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca0103150 0x7f0ca01a10b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.690+0000 7f0ca7000640 1 --2- 172.21.15.47:0/2738911197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ca0102790 0x7f0ca019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.690+0000 7f0ca7000640 1 --2- 172.21.15.47:0/2738911197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ca01015a0 0x7f0ca0199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.690+0000 7f0ca7000640 1 -- 172.21.15.47:0/2738911197 >> 172.21.15.47:0/2738911197 conn(0x7f0ca00fcb10 msgr2=0x7f0ca00738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.690+0000 7f0ca7000640 1 -- 172.21.15.47:0/2738911197 shutdown_connections 2024-03-21T11:31:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:31:48.690+0000 7f0ca7000640 1 -- 172.21.15.47:0/2738911197 wait complete. 2024-03-21T11:31:48.711 INFO:tasks.cephfs.filesystem:scrub status for tag:1674dfd2-71aa-4617-a27c-42876be4eaa3 - {'path': '/', 'tag': '1674dfd2-71aa-4617-a27c-42876be4eaa3', 'options': 'recursive,force'} 2024-03-21T11:31:49.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:48 smithi100 ceph-mon[36245]: pgmap v654: 129 pgs: 129 active+clean; 39 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 256 op/s 2024-03-21T11:31:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:48 smithi102 ceph-mon[36748]: pgmap v654: 129 pgs: 129 active+clean; 39 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 256 op/s 2024-03-21T11:31:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:48 smithi047 ceph-mon[31351]: pgmap v654: 129 pgs: 129 active+clean; 39 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 256 op/s 2024-03-21T11:31:49.539 INFO:tasks.workunit.client.0.smithi047.stdout: 1 607581 8.21 MB/sec execute 273 sec latency 10.856 ms 2024-03-21T11:31:50.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:49 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:31:50.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:49 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:31:50.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:49 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:31:50.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:49 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:31:50.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:49 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:31:50.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:49 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:31:50.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:49 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:31:50.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:49 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:31:50.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:49 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:31:50.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:49 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:31:50.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:49 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:31:50.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:49 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:31:50.539 INFO:tasks.workunit.client.0.smithi047.stdout: 1 608602 8.19 MB/sec execute 274 sec latency 11.797 ms 2024-03-21T11:31:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:50 smithi102 ceph-mon[36748]: pgmap v655: 129 pgs: 129 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 272 op/s 2024-03-21T11:31:51.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:50 smithi047 ceph-mon[31351]: pgmap v655: 129 pgs: 129 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 272 op/s 2024-03-21T11:31:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:50 smithi100 ceph-mon[36245]: pgmap v655: 129 pgs: 129 active+clean; 42 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 272 op/s 2024-03-21T11:31:51.539 INFO:tasks.workunit.client.0.smithi047.stdout: 1 610859 8.21 MB/sec execute 275 sec latency 38.914 ms 2024-03-21T11:31:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.589+0000 7f6c19400640 1 -- 172.21.15.47:0/1698553079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c14108ef0 msgr2=0x7f6c14109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.589+0000 7f6c19400640 1 --2- 172.21.15.47:0/1698553079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c14108ef0 0x7f6c14109370 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009900 tx=0x7f6c0002bc40 comp rx=0 tx=0).stop 2024-03-21T11:31:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.589+0000 7f6c19400640 1 -- 172.21.15.47:0/1698553079 shutdown_connections 2024-03-21T11:31:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.589+0000 7f6c19400640 1 --2- 172.21.15.47:0/1698553079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c141098b0 0x7f6c14110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.589+0000 7f6c19400640 1 --2- 172.21.15.47:0/1698553079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c14108ef0 0x7f6c14109370 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.589+0000 7f6c19400640 1 --2- 172.21.15.47:0/1698553079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c14107d00 0x7f6c14108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.589+0000 7f6c19400640 1 -- 172.21.15.47:0/1698553079 >> 172.21.15.47:0/1698553079 conn(0x7f6c14076cb0 msgr2=0x7f6c140790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.589+0000 7f6c19400640 1 -- 172.21.15.47:0/1698553079 shutdown_connections 2024-03-21T11:31:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.589+0000 7f6c19400640 1 -- 172.21.15.47:0/1698553079 wait complete. 2024-03-21T11:31:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.590+0000 7f6c19400640 1 Processor -- start 2024-03-21T11:31:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.591+0000 7f6c19400640 1 -- start start 2024-03-21T11:31:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.591+0000 7f6c19400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c14107d00 0x7f6c1419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.591+0000 7f6c19400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c14108ef0 0x7f6c1419e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.591+0000 7f6c19400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c141098b0 0x7f6c141a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.591+0000 7f6c19400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c141139b0 con 0x7f6c14108ef0 2024-03-21T11:31:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.591+0000 7f6c19400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c14113830 con 0x7f6c141098b0 2024-03-21T11:31:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.591+0000 7f6c19400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c14113b30 con 0x7f6c14107d00 2024-03-21T11:31:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.591+0000 7f6c12800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c14107d00 0x7f6c1419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.591+0000 7f6c12800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c14107d00 0x7f6c1419df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56286/0 (socket says 172.21.15.47:56286) 2024-03-21T11:31:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.591+0000 7f6c12800640 1 -- 172.21.15.47:0/4076916768 learned_addr learned my addr 172.21.15.47:0/4076916768 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.591+0000 7f6c11e00640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c14108ef0 0x7f6c1419e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.591+0000 7f6c13200640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c141098b0 0x7f6c141a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.592+0000 7f6c12800640 1 -- 172.21.15.47:0/4076916768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c141098b0 msgr2=0x7f6c141a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.592+0000 7f6c12800640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c141098b0 0x7f6c141a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.592+0000 7f6c12800640 1 -- 172.21.15.47:0/4076916768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c14108ef0 msgr2=0x7f6c1419e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.592+0000 7f6c12800640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c14108ef0 0x7f6c1419e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.592+0000 7f6c12800640 1 -- 172.21.15.47:0/4076916768 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c141a5b60 con 0x7f6c14107d00 2024-03-21T11:31:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.592+0000 7f6c13200640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c141098b0 0x7f6c141a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.592+0000 7f6c12800640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c14107d00 0x7f6c1419df40 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400ea80 tx=0x7f6c0400ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.592+0000 7f6c0b400640 1 -- 172.21.15.47:0/4076916768 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400ce40 con 0x7f6c14107d00 2024-03-21T11:31:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.592+0000 7f6c0b400640 1 -- 172.21.15.47:0/4076916768 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c04011950 con 0x7f6c14107d00 2024-03-21T11:31:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.593+0000 7f6c19400640 1 -- 172.21.15.47:0/4076916768 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c141a5e50 con 0x7f6c14107d00 2024-03-21T11:31:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.593+0000 7f6c0b400640 1 -- 172.21.15.47:0/4076916768 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0401ab80 con 0x7f6c14107d00 2024-03-21T11:31:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.593+0000 7f6c19400640 1 -- 172.21.15.47:0/4076916768 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c141a62c0 con 0x7f6c14107d00 2024-03-21T11:31:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.596+0000 7f6c1c078740 1 Processor -- start 2024-03-21T11:31:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.596+0000 7f6c1c078740 1 -- start start 2024-03-21T11:31:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.596+0000 7f6c0b400640 1 -- 172.21.15.47:0/4076916768 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6c04011b00 con 0x7f6c14107d00 2024-03-21T11:31:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.596+0000 7f6c1c078740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bc3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.596+0000 7f6c1c078740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc3bd0 0x562200bc86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.596+0000 7f6c1c078740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc8d40 0x562200bcb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.596+0000 7f6c1c078740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562200b863b0 con 0x562200bc8d40 2024-03-21T11:31:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.596+0000 7f6c1c078740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562200b28830 con 0x562200bc3bd0 2024-03-21T11:31:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.596+0000 7f6c1c078740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562200ad15d0 con 0x562200bc31e0 2024-03-21T11:31:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.597+0000 7f6c12800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc3bd0 0x562200bc86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.597+0000 7f6c13200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bc3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.597+0000 7f6c13200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bc3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56292/0 (socket says 172.21.15.47:56292) 2024-03-21T11:31:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.597+0000 7f6c13200640 1 -- 172.21.15.47:0/1710899163 learned_addr learned my addr 172.21.15.47:0/1710899163 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.597+0000 7f6c11e00640 1 --2- 172.21.15.47:0/1710899163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc8d40 0x562200bcb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.597+0000 7f6c13200640 1 -- 172.21.15.47:0/1710899163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc3bd0 msgr2=0x562200bc86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.597+0000 7f6c13200640 1 --2- 172.21.15.47:0/1710899163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc3bd0 0x562200bc86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.597+0000 7f6c13200640 1 -- 172.21.15.47:0/1710899163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc8d40 msgr2=0x562200bcb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.597+0000 7f6c13200640 1 --2- 172.21.15.47:0/1710899163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc8d40 0x562200bcb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.597+0000 7f6c13200640 1 -- 172.21.15.47:0/1710899163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562200ae92c0 con 0x562200bc31e0 2024-03-21T11:31:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.598+0000 7f6c13200640 1 --2- 172.21.15.47:0/1710899163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bc3600 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009cc0 tx=0x7f6c0c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e4ca234d929593c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.598+0000 7f6c0b400640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bf80a3040 0x7f6bf80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.598+0000 7f6bff400640 1 -- 172.21.15.47:0/1710899163 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c012450 con 0x562200bc31e0 2024-03-21T11:31:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.598+0000 7f6bff400640 1 -- 172.21.15.47:0/1710899163 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c012ad0 con 0x562200bc31e0 2024-03-21T11:31:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.598+0000 7f6bff400640 1 -- 172.21.15.47:0/1710899163 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c00fc50 con 0x562200bc31e0 2024-03-21T11:31:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.598+0000 7f6c0b400640 1 -- 172.21.15.47:0/4076916768 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6c040cfd80 con 0x7f6c14107d00 2024-03-21T11:31:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.598+0000 7f6c12800640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bf80a3040 0x7f6bf80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.599+0000 7f6c1c078740 1 -- 172.21.15.47:0/1710899163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 msgr2=0x562200bc3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.599+0000 7f6c1c078740 1 --2- 172.21.15.47:0/1710899163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bc3600 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009cc0 tx=0x7f6c0c0117b0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.599+0000 7f6c12800640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bf80a3040 0x7f6bf80a5500 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6c0409bdc0 tx=0x7f6c040d37c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.599+0000 7f6c1c078740 1 -- 172.21.15.47:0/1710899163 shutdown_connections 2024-03-21T11:31:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.599+0000 7f6c1c078740 1 --2- 172.21.15.47:0/1710899163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc8d40 0x562200bcb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.599+0000 7f6c1c078740 1 --2- 172.21.15.47:0/1710899163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc3bd0 0x562200bc86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.599+0000 7f6c1c078740 1 --2- 172.21.15.47:0/1710899163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bc3600 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.599+0000 7f6c1c078740 1 -- 172.21.15.47:0/1710899163 >> 172.21.15.47:0/1710899163 conn(0x562200bbe750 msgr2=0x562200bbeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.599+0000 7f6c1c078740 1 -- 172.21.15.47:0/1710899163 shutdown_connections 2024-03-21T11:31:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.599+0000 7f6c1c078740 1 -- 172.21.15.47:0/1710899163 wait complete. 2024-03-21T11:31:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.599+0000 7f6c1c078740 1 -- start start 2024-03-21T11:31:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.600+0000 7f6c1c078740 1 Processor -- start 2024-03-21T11:31:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.600+0000 7f6c1c078740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc31e0 0x562200bcd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.600+0000 7f6c1c078740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc3bd0 0x562200bcd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.600+0000 7f6c1c078740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc8d40 0x562200bd48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.600+0000 7f6c1c078740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562200ae4a20 con 0x562200bc31e0 2024-03-21T11:31:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.600+0000 7f6c1c078740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562200b248d0 con 0x562200bc8d40 2024-03-21T11:31:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.600+0000 7f6c1c078740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562200bcbcf0 con 0x562200bc3bd0 2024-03-21T11:31:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.600+0000 7f6c1c078740 10 client.0 _ll_register_callbacks cb 0x562200b211c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.600+0000 7f6c11e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc3bd0 0x562200bcd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.601+0000 7f6c11e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc3bd0 0x562200bcd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56294/0 (socket says 172.21.15.47:56294) 2024-03-21T11:31:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.601+0000 7f6c11e00640 1 -- 172.21.15.47:0/4273855659 learned_addr learned my addr 172.21.15.47:0/4273855659 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.601+0000 7f6c13200640 1 --2- 172.21.15.47:0/4273855659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc31e0 0x562200bcd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.601+0000 7f6c13200640 1 --2- 172.21.15.47:0/4273855659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc8d40 0x562200bd48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.601+0000 7f6c11e00640 1 -- 172.21.15.47:0/4273855659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc8d40 msgr2=0x562200bd48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.601+0000 7f6c11e00640 1 --2- 172.21.15.47:0/4273855659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc8d40 0x562200bd48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.601+0000 7f6c11e00640 1 -- 172.21.15.47:0/4273855659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc31e0 msgr2=0x562200bcd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.601+0000 7f6c11e00640 1 --2- 172.21.15.47:0/4273855659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc31e0 0x562200bcd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.601+0000 7f6c11e00640 1 -- 172.21.15.47:0/4273855659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56220083d210 con 0x562200bc3bd0 2024-03-21T11:31:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.601+0000 7f6c13200640 1 --2- 172.21.15.47:0/4273855659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc31e0 0x562200bcd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.601+0000 7f6c11e00640 1 --2- 172.21.15.47:0/4273855659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc3bd0 0x562200bcd910 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6c00005ab0 tx=0x7f6c00004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=813bdf4efe9649f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.601+0000 7f6bf3e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:31:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.602+0000 7f6bf3e00640 1 -- 172.21.15.47:0/4273855659 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00005d50 con 0x562200bc3bd0 2024-03-21T11:31:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.602+0000 7f6bf3e00640 1 -- 172.21.15.47:0/4273855659 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6c000048c0 con 0x562200bc3bd0 2024-03-21T11:31:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.602+0000 7f6c1c078740 1 -- 172.21.15.47:0/4273855659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c0c012ad0 con 0x562200bc3bd0 2024-03-21T11:31:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.602+0000 7f6bf3e00640 1 -- 172.21.15.47:0/4273855659 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c00004b00 con 0x562200bc3bd0 2024-03-21T11:31:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.602+0000 7f6c1c078740 10 client.28417 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.602+0000 7f6c1c078740 1 -- 172.21.15.47:0/4273855659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562200909620 con 0x562200bc3bd0 2024-03-21T11:31:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.602+0000 7f6bf3e00640 1 -- 172.21.15.47:0/4273855659 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c012ad0 con 0x562200bc3bd0 2024-03-21T11:31:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.602+0000 7f6bf3e00640 1 -- 172.21.15.47:0/4273855659 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6c00046070 con 0x562200bc3bd0 2024-03-21T11:31:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.603+0000 7f6c1c078740 10 client.28417 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.603+0000 7f6c1c078740 10 client.28417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.603+0000 7f6c1c078740 20 client.28417 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.603+0000 7f6c1c078740 1 --2- 172.21.15.47:0/4273855659 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562200bd9170 0x562200bdb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.603+0000 7f6c1c078740 4 client.28417 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:31:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.603+0000 7f6c1c078740 1 -- 172.21.15.47:0/4273855659 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562200b35d70 con 0x562200bd9170 2024-03-21T11:31:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.603+0000 7f6c13200640 1 --2- 172.21.15.47:0/4273855659 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562200bd9170 0x562200bdb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.603+0000 7f6c13200640 1 --2- 172.21.15.47:0/4273855659 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562200bd9170 0x562200bdb5b0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec6d498d443e2e68 server_cookie=9934c9becc8176e8 in_seq=0 out_seq=0 2024-03-21T11:31:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.603+0000 7f6bf3e00640 10 client.28417 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6bf3e00640 1 -- 172.21.15.47:0/4273855659 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562200b35d70 con 0x562200bd9170 2024-03-21T11:31:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6bf3e00640 10 client.28417 handle_command_reply: tid=0 2024-03-21T11:31:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6c1c078740 1 client.28417 shutdown 2024-03-21T11:31:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6c1c078740 1 -- 172.21.15.47:0/4273855659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc3bd0 msgr2=0x562200bcd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6c1c078740 1 --2- 172.21.15.47:0/4273855659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc3bd0 0x562200bcd910 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6c00005ab0 tx=0x7f6c00004270 comp rx=0 tx=0).stop 2024-03-21T11:31:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6c1c078740 1 -- 172.21.15.47:0/4273855659 shutdown_connections 2024-03-21T11:31:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6c1c078740 1 --2- 172.21.15.47:0/4273855659 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562200bd9170 0x562200bdb5b0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6c1c078740 1 --2- 172.21.15.47:0/4273855659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc8d40 0x562200bd48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6c1c078740 1 --2- 172.21.15.47:0/4273855659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc3bd0 0x562200bcd910 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6c1c078740 1 --2- 172.21.15.47:0/4273855659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc31e0 0x562200bcd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6bf3e00640 0 client.28417 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6c1c078740 1 -- 172.21.15.47:0/4273855659 >> 172.21.15.47:0/4273855659 conn(0x562200bbe750 msgr2=0x562200bbd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:51.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6c1c078740 1 -- 172.21.15.47:0/4273855659 shutdown_connections 2024-03-21T11:31:51.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6c1c078740 1 -- 172.21.15.47:0/4273855659 wait complete. 2024-03-21T11:31:51.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.606+0000 7f6c1c078740 20 client.28417 trim_cache size 0 max 16384 2024-03-21T11:31:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.622+0000 7f6c1c078740 1 Processor -- start 2024-03-21T11:31:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.622+0000 7f6c1c078740 1 -- start start 2024-03-21T11:31:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.622+0000 7f6c1c078740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bc19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.622+0000 7f6c1c078740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc3bd0 0x562200bd48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.622+0000 7f6c1c078740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc8d40 0x562200bc1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.622+0000 7f6c1c078740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c141b05e0 con 0x562200bc3bd0 2024-03-21T11:31:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.622+0000 7f6c1c078740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c141b1f60 con 0x562200bc8d40 2024-03-21T11:31:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.622+0000 7f6c1c078740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c141b0c40 con 0x562200bc31e0 2024-03-21T11:31:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.622+0000 7f6c13200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bc19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.623+0000 7f6c13200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bc19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56298/0 (socket says 172.21.15.47:56298) 2024-03-21T11:31:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.623+0000 7f6c13200640 1 -- 172.21.15.47:0/453848704 learned_addr learned my addr 172.21.15.47:0/453848704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.623+0000 7f6c13200640 1 --2- 172.21.15.47:0/453848704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc8d40 0x562200bc1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.623+0000 7f6c11e00640 1 --2- 172.21.15.47:0/453848704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc3bd0 0x562200bd48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.623+0000 7f6c13200640 1 -- 172.21.15.47:0/453848704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc8d40 msgr2=0x562200bc1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.623+0000 7f6c13200640 1 --2- 172.21.15.47:0/453848704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc8d40 0x562200bc1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.623+0000 7f6c13200640 1 -- 172.21.15.47:0/453848704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc3bd0 msgr2=0x562200bd48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.623+0000 7f6c13200640 1 --2- 172.21.15.47:0/453848704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc3bd0 0x562200bd48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.623+0000 7f6c13200640 1 -- 172.21.15.47:0/453848704 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562200bc4460 con 0x562200bc31e0 2024-03-21T11:31:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.623+0000 7f6c13200640 1 --2- 172.21.15.47:0/453848704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bc19b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c000c00 tx=0x7f6c0c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ec64309d99ee6b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.623+0000 7f6bfea00640 1 -- 172.21.15.47:0/453848704 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c0097a0 con 0x562200bc31e0 2024-03-21T11:31:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.623+0000 7f6bfea00640 1 -- 172.21.15.47:0/453848704 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c02e140 con 0x562200bc31e0 2024-03-21T11:31:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.623+0000 7f6bfea00640 1 -- 172.21.15.47:0/453848704 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c00f940 con 0x562200bc31e0 2024-03-21T11:31:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.624+0000 7f6c1c078740 1 -- 172.21.15.47:0/453848704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 msgr2=0x562200bc19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.624+0000 7f6c1c078740 1 --2- 172.21.15.47:0/453848704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bc19b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c000c00 tx=0x7f6c0c0117b0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.624+0000 7f6c1c078740 1 -- 172.21.15.47:0/453848704 shutdown_connections 2024-03-21T11:31:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.624+0000 7f6c1c078740 1 --2- 172.21.15.47:0/453848704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc8d40 0x562200bc1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.624+0000 7f6c1c078740 1 --2- 172.21.15.47:0/453848704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc3bd0 0x562200bd48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.624+0000 7f6c1c078740 1 --2- 172.21.15.47:0/453848704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bc19b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.624+0000 7f6c1c078740 1 -- 172.21.15.47:0/453848704 >> 172.21.15.47:0/453848704 conn(0x562200bbe750 msgr2=0x562200bc4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.624+0000 7f6c1c078740 1 -- 172.21.15.47:0/453848704 shutdown_connections 2024-03-21T11:31:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.624+0000 7f6c1c078740 1 -- 172.21.15.47:0/453848704 wait complete. 2024-03-21T11:31:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.624+0000 7f6c1c078740 1 -- start start 2024-03-21T11:31:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.625+0000 7f6c1c078740 1 Processor -- start 2024-03-21T11:31:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.625+0000 7f6c1c078740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bd2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.625+0000 7f6c1c078740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc3bd0 0x562200bd2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.625+0000 7f6c1c078740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc8d40 0x562200bd3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.625+0000 7f6c1c078740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c141b1900 con 0x562200bc8d40 2024-03-21T11:31:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.625+0000 7f6c1c078740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c141b12a0 con 0x562200bc3bd0 2024-03-21T11:31:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.625+0000 7f6c1c078740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c141ae7a0 con 0x562200bc31e0 2024-03-21T11:31:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.625+0000 7f6c1c078740 10 client.0 _ll_register_callbacks cb 0x7f6c00009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:31:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.625+0000 7f6c11e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc3bd0 0x562200bd2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.625+0000 7f6c11e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc3bd0 0x562200bd2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35976/0 (socket says 172.21.15.47:35976) 2024-03-21T11:31:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.625+0000 7f6c11e00640 1 -- 172.21.15.47:0/1508623633 learned_addr learned my addr 172.21.15.47:0/1508623633 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:31:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.625+0000 7f6c13200640 1 --2- 172.21.15.47:0/1508623633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bd2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.626+0000 7f6c13200640 1 --2- 172.21.15.47:0/1508623633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc8d40 0x562200bd3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.626+0000 7f6c11e00640 1 -- 172.21.15.47:0/1508623633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 msgr2=0x562200bd2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.626+0000 7f6c11e00640 1 --2- 172.21.15.47:0/1508623633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bd2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.626+0000 7f6c11e00640 1 -- 172.21.15.47:0/1508623633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc8d40 msgr2=0x562200bd3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:31:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.626+0000 7f6c11e00640 1 --2- 172.21.15.47:0/1508623633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc8d40 0x562200bd3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.626+0000 7f6c11e00640 1 -- 172.21.15.47:0/1508623633 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562200bd4230 con 0x562200bc3bd0 2024-03-21T11:31:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.626+0000 7f6c13200640 1 --2- 172.21.15.47:0/1508623633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc8d40 0x562200bd3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:31:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.626+0000 7f6c11e00640 1 --2- 172.21.15.47:0/1508623633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc3bd0 0x562200bd2c80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009850 tx=0x7f6c000484c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=912dbdfbdf733f4e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:31:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.626+0000 7f6bf3e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:31:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.626+0000 7f6bf3e00640 1 -- 172.21.15.47:0/1508623633 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00005d50 con 0x562200bc3bd0 2024-03-21T11:31:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.626+0000 7f6c1c078740 1 -- 172.21.15.47:0/1508623633 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c0c02e140 con 0x562200bc3bd0 2024-03-21T11:31:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.626+0000 7f6bf3e00640 1 -- 172.21.15.47:0/1508623633 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6c000050e0 con 0x562200bc3bd0 2024-03-21T11:31:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.627+0000 7f6bf3e00640 1 -- 172.21.15.47:0/1508623633 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c0002cd30 con 0x562200bc3bd0 2024-03-21T11:31:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.627+0000 7f6c1c078740 10 client.28421 fetch_fsmap learned FSMap version 28 2024-03-21T11:31:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.627+0000 7f6c1c078740 1 -- 172.21.15.47:0/1508623633 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562200c35220 con 0x562200bc3bd0 2024-03-21T11:31:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.627+0000 7f6bf3e00640 1 -- 172.21.15.47:0/1508623633 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c02e140 con 0x562200bc3bd0 2024-03-21T11:31:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.627+0000 7f6bf3e00640 1 -- 172.21.15.47:0/1508623633 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6c00055030 con 0x562200bc3bd0 2024-03-21T11:31:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.627+0000 7f6c1c078740 10 client.28421 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:31:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.627+0000 7f6c1c078740 10 client.28421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:31:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.627+0000 7f6c1c078740 20 client.28421 populate_metadata read hostname 'smithi047' 2024-03-21T11:31:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.627+0000 7f6c1c078740 1 --2- 172.21.15.47:0/1508623633 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562200c35ba0 0x562200c37fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:31:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.627+0000 7f6c1c078740 4 client.28421 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:31:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.627+0000 7f6c1c078740 1 -- 172.21.15.47:0/1508623633 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562200b300c0 con 0x562200c35ba0 2024-03-21T11:31:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.627+0000 7f6c13200640 1 --2- 172.21.15.47:0/1508623633 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562200c35ba0 0x562200c37fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:31:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.628+0000 7f6c13200640 1 --2- 172.21.15.47:0/1508623633 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562200c35ba0 0x562200c37fc0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9265c377647308c1 server_cookie=731aa388e7641eea in_seq=0 out_seq=0 2024-03-21T11:31:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.628+0000 7f6bf3e00640 10 client.28421 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.909+0000 7f6bf3e00640 1 -- 172.21.15.47:0/1508623633 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562200b300c0 con 0x562200c35ba0 2024-03-21T11:31:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.909+0000 7f6bf3e00640 10 client.28421 handle_command_reply: tid=0 2024-03-21T11:31:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.909+0000 7f6c1c078740 1 client.28421 shutdown 2024-03-21T11:31:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.909+0000 7f6c1c078740 1 -- 172.21.15.47:0/1508623633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc3bd0 msgr2=0x562200bd2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.909+0000 7f6c1c078740 1 --2- 172.21.15.47:0/1508623633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc3bd0 0x562200bd2c80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009850 tx=0x7f6c000484c0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.909+0000 7f6c1c078740 1 -- 172.21.15.47:0/1508623633 shutdown_connections 2024-03-21T11:31:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.909+0000 7f6c1c078740 1 --2- 172.21.15.47:0/1508623633 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562200c35ba0 0x562200c37fc0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.909+0000 7f6c1c078740 1 --2- 172.21.15.47:0/1508623633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562200bc8d40 0x562200bd3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.909+0000 7f6c1c078740 1 --2- 172.21.15.47:0/1508623633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562200bc3bd0 0x562200bd2c80 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.909+0000 7f6c1c078740 1 --2- 172.21.15.47:0/1508623633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562200bc31e0 0x562200bd2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.909+0000 7f6bf3e00640 0 client.28421 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:31:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.909+0000 7f6c1c078740 1 -- 172.21.15.47:0/1508623633 >> 172.21.15.47:0/1508623633 conn(0x562200bbe750 msgr2=0x562200bc19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.910+0000 7f6c1c078740 1 -- 172.21.15.47:0/1508623633 shutdown_connections 2024-03-21T11:31:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.910+0000 7f6c1c078740 1 -- 172.21.15.47:0/1508623633 wait complete. 2024-03-21T11:31:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.910+0000 7f6c1c078740 20 client.28421 trim_cache size 0 max 16384 2024-03-21T11:31:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:31:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:31:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:31:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:31:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.911+0000 7f6c19400640 1 -- 172.21.15.47:0/4076916768 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bf80a3040 msgr2=0x7f6bf80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.911+0000 7f6c19400640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bf80a3040 0x7f6bf80a5500 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6c0409bdc0 tx=0x7f6c040d37c0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.911+0000 7f6c19400640 1 -- 172.21.15.47:0/4076916768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c14107d00 msgr2=0x7f6c1419df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:31:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.911+0000 7f6c19400640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c14107d00 0x7f6c1419df40 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400ea80 tx=0x7f6c0400ef40 comp rx=0 tx=0).stop 2024-03-21T11:31:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.911+0000 7f6c19400640 1 -- 172.21.15.47:0/4076916768 shutdown_connections 2024-03-21T11:31:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.911+0000 7f6c19400640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6bf80a3040 0x7f6bf80a5500 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.911+0000 7f6c19400640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c141098b0 0x7f6c141a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.911+0000 7f6c19400640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c14108ef0 0x7f6c1419e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.911+0000 7f6c19400640 1 --2- 172.21.15.47:0/4076916768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c14107d00 0x7f6c1419df40 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:31:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.911+0000 7f6c19400640 1 -- 172.21.15.47:0/4076916768 >> 172.21.15.47:0/4076916768 conn(0x7f6c14076cb0 msgr2=0x7f6c1410c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:31:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.911+0000 7f6c19400640 1 -- 172.21.15.47:0/4076916768 shutdown_connections 2024-03-21T11:31:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:31:51.911+0000 7f6c19400640 1 -- 172.21.15.47:0/4076916768 wait complete. 2024-03-21T11:31:52.539 INFO:tasks.workunit.client.0.smithi047.stdout: 1 611474 8.19 MB/sec execute 276 sec latency 151.757 ms 2024-03-21T11:31:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:52 smithi102 ceph-mon[36748]: pgmap v656: 129 pgs: 129 active+clean; 43 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-03-21T11:31:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:52 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:31:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:52 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:31:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:52 smithi047 ceph-mon[31351]: pgmap v656: 129 pgs: 129 active+clean; 43 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-03-21T11:31:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:52 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:31:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:52 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:31:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:52 smithi100 ceph-mon[36245]: pgmap v656: 129 pgs: 129 active+clean; 43 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-03-21T11:31:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:52 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:31:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:52 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:31:53.539 INFO:tasks.workunit.client.0.smithi047.stdout: 1 613229 8.18 MB/sec execute 277 sec latency 27.047 ms 2024-03-21T11:31:54.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:53 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:31:54.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:53 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:31:54.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:53 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:31:54.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:53 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:31:54.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:53 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:31:54.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:53 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:31:54.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:53 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:31:54.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:53 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:31:54.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:53 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:31:54.540 INFO:tasks.workunit.client.0.smithi047.stdout: 1 614804 8.20 MB/sec execute 278 sec latency 12.123 ms 2024-03-21T11:31:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:54 smithi102 ceph-mon[36748]: pgmap v657: 129 pgs: 129 active+clean; 39 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 321 op/s 2024-03-21T11:31:55.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:54 smithi100 ceph-mon[36245]: pgmap v657: 129 pgs: 129 active+clean; 39 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 321 op/s 2024-03-21T11:31:55.410 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:54 smithi047 ceph-mon[31351]: pgmap v657: 129 pgs: 129 active+clean; 39 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 321 op/s 2024-03-21T11:31:55.540 INFO:tasks.workunit.client.0.smithi047.stdout: 1 616132 8.19 MB/sec execute 279 sec latency 12.920 ms 2024-03-21T11:31:56.540 INFO:tasks.workunit.client.0.smithi047.stdout: 1 618233 8.21 MB/sec execute 280 sec latency 12.791 ms 2024-03-21T11:31:57.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:56 smithi102 ceph-mon[36748]: pgmap v658: 129 pgs: 129 active+clean; 38 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 268 op/s 2024-03-21T11:31:57.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:56 smithi047 ceph-mon[31351]: pgmap v658: 129 pgs: 129 active+clean; 38 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 268 op/s 2024-03-21T11:31:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:56 smithi100 ceph-mon[36245]: pgmap v658: 129 pgs: 129 active+clean; 38 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 268 op/s 2024-03-21T11:31:57.540 INFO:tasks.workunit.client.0.smithi047.stdout: 1 619201 8.19 MB/sec execute 281 sec latency 71.666 ms 2024-03-21T11:31:58.540 INFO:tasks.workunit.client.0.smithi047.stdout: 1 621007 8.19 MB/sec execute 282 sec latency 73.495 ms 2024-03-21T11:31:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:31:58 smithi102 ceph-mon[36748]: pgmap v659: 129 pgs: 129 active+clean; 37 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-03-21T11:31:59.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:31:58 smithi047 ceph-mon[31351]: pgmap v659: 129 pgs: 129 active+clean; 37 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-03-21T11:31:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:31:58 smithi100 ceph-mon[36245]: pgmap v659: 129 pgs: 129 active+clean; 37 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-03-21T11:31:59.551 INFO:tasks.workunit.client.0.smithi047.stdout: 1 622250 8.19 MB/sec execute 283 sec latency 13.801 ms 2024-03-21T11:32:00.541 INFO:tasks.workunit.client.0.smithi047.stdout: 1 623968 8.18 MB/sec execute 284 sec latency 16.202 ms 2024-03-21T11:32:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:00 smithi102 ceph-mon[36748]: pgmap v660: 129 pgs: 129 active+clean; 36 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 344 op/s 2024-03-21T11:32:01.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:00 smithi047 ceph-mon[31351]: pgmap v660: 129 pgs: 129 active+clean; 36 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 344 op/s 2024-03-21T11:32:01.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:00 smithi100 ceph-mon[36245]: pgmap v660: 129 pgs: 129 active+clean; 36 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 344 op/s 2024-03-21T11:32:01.541 INFO:tasks.workunit.client.0.smithi047.stdout: 1 625459 8.20 MB/sec execute 285 sec latency 12.030 ms 2024-03-21T11:32:02.541 INFO:tasks.workunit.client.0.smithi047.stdout: 1 626362 8.18 MB/sec execute 286 sec latency 54.574 ms 2024-03-21T11:32:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:02 smithi102 ceph-mon[36748]: pgmap v661: 129 pgs: 129 active+clean; 37 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 304 op/s 2024-03-21T11:32:03.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:02 smithi047 ceph-mon[31351]: pgmap v661: 129 pgs: 129 active+clean; 37 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 304 op/s 2024-03-21T11:32:03.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:02 smithi100 ceph-mon[36245]: pgmap v661: 129 pgs: 129 active+clean; 37 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 304 op/s 2024-03-21T11:32:03.541 INFO:tasks.workunit.client.0.smithi047.stdout: 1 628347 8.19 MB/sec execute 287 sec latency 35.012 ms 2024-03-21T11:32:04.541 INFO:tasks.workunit.client.0.smithi047.stdout: 1 629358 8.18 MB/sec execute 288 sec latency 14.657 ms 2024-03-21T11:32:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.068+0000 7f1d8ce00640 1 -- 172.21.15.47:0/4078714545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d88104af0 msgr2=0x7f1d88104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.068+0000 7f1d8ce00640 1 --2- 172.21.15.47:0/4078714545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d88104af0 0x7f1d88104f70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009900 tx=0x7f1d7402bc50 comp rx=0 tx=0).stop 2024-03-21T11:32:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.068+0000 7f1d8ce00640 1 -- 172.21.15.47:0/4078714545 shutdown_connections 2024-03-21T11:32:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.068+0000 7f1d8ce00640 1 --2- 172.21.15.47:0/4078714545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d881054b0 0x7f1d8810bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.068+0000 7f1d8ce00640 1 --2- 172.21.15.47:0/4078714545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d88104af0 0x7f1d88104f70 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.068+0000 7f1d8ce00640 1 --2- 172.21.15.47:0/4078714545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d88103900 0x7f1d88103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.068+0000 7f1d8ce00640 1 -- 172.21.15.47:0/4078714545 >> 172.21.15.47:0/4078714545 conn(0x7f1d880fee90 msgr2=0x7f1d881012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.068+0000 7f1d8ce00640 1 -- 172.21.15.47:0/4078714545 shutdown_connections 2024-03-21T11:32:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.068+0000 7f1d8ce00640 1 -- 172.21.15.47:0/4078714545 wait complete. 2024-03-21T11:32:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.069+0000 7f1d8ce00640 1 Processor -- start 2024-03-21T11:32:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.069+0000 7f1d8ce00640 1 -- start start 2024-03-21T11:32:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d8ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d88103900 0x7f1d88199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d8ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d88104af0 0x7f1d8819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d8ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d881054b0 0x7f1d881a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d8ce00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8810f560 con 0x7f1d88104af0 2024-03-21T11:32:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d8ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8810f3e0 con 0x7f1d88103900 2024-03-21T11:32:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d8ce00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8810f6e0 con 0x7f1d881054b0 2024-03-21T11:32:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d85e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d88103900 0x7f1d88199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d85e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d88103900 0x7f1d88199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53052/0 (socket says 172.21.15.47:53052) 2024-03-21T11:32:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d85e00640 1 -- 172.21.15.47:0/3017392145 learned_addr learned my addr 172.21.15.47:0/3017392145 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d86800640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d881054b0 0x7f1d881a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d85400640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d88104af0 0x7f1d8819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d86800640 1 -- 172.21.15.47:0/3017392145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d88103900 msgr2=0x7f1d88199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d86800640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d88103900 0x7f1d88199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d86800640 1 -- 172.21.15.47:0/3017392145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d88104af0 msgr2=0x7f1d8819a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d86800640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d88104af0 0x7f1d8819a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d86800640 1 -- 172.21.15.47:0/3017392145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d881a1800 con 0x7f1d881054b0 2024-03-21T11:32:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.070+0000 7f1d85e00640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d88103900 0x7f1d88199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.071+0000 7f1d86800640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d881054b0 0x7f1d881a1190 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009fd0 tx=0x7f1d8000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.071+0000 7f1d7ea00640 1 -- 172.21.15.47:0/3017392145 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000ce20 con 0x7f1d881054b0 2024-03-21T11:32:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.071+0000 7f1d7ea00640 1 -- 172.21.15.47:0/3017392145 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d80005020 con 0x7f1d881054b0 2024-03-21T11:32:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.071+0000 7f1d8ce00640 1 -- 172.21.15.47:0/3017392145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d881a1af0 con 0x7f1d881054b0 2024-03-21T11:32:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.071+0000 7f1d7ea00640 1 -- 172.21.15.47:0/3017392145 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80010720 con 0x7f1d881054b0 2024-03-21T11:32:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.071+0000 7f1d8ce00640 1 -- 172.21.15.47:0/3017392145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d881a1fc0 con 0x7f1d881054b0 2024-03-21T11:32:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.075+0000 7f1d8fa07740 1 Processor -- start 2024-03-21T11:32:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.075+0000 7f1d7ea00640 1 -- 172.21.15.47:0/3017392145 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1d80005540 con 0x7f1d881054b0 2024-03-21T11:32:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.075+0000 7f1d8fa07740 1 -- start start 2024-03-21T11:32:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.075+0000 7f1d8fa07740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e101e0 0x55e624e10600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.075+0000 7f1d8fa07740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e10bd0 0x55e624e156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.075+0000 7f1d8fa07740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e15d40 0x55e624e181d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.075+0000 7f1d8fa07740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e624dd33b0 con 0x55e624e10bd0 2024-03-21T11:32:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.075+0000 7f1d8fa07740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e624d75830 con 0x55e624e15d40 2024-03-21T11:32:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.075+0000 7f1d8fa07740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e624d1e5d0 con 0x55e624e101e0 2024-03-21T11:32:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.075+0000 7f1d85400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e101e0 0x55e624e10600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.076+0000 7f1d85400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e101e0 0x55e624e10600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41984/0 (socket says 172.21.15.47:41984) 2024-03-21T11:32:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.076+0000 7f1d85e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e10bd0 0x55e624e156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.076+0000 7f1d85400640 1 -- 172.21.15.47:0/908561578 learned_addr learned my addr 172.21.15.47:0/908561578 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.076+0000 7f1d85400640 1 --2- 172.21.15.47:0/908561578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e15d40 0x55e624e181d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.076+0000 7f1d85e00640 1 -- 172.21.15.47:0/908561578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e101e0 msgr2=0x55e624e10600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.076+0000 7f1d85e00640 1 --2- 172.21.15.47:0/908561578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e101e0 0x55e624e10600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.076+0000 7f1d85e00640 1 -- 172.21.15.47:0/908561578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e15d40 msgr2=0x55e624e181d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.076+0000 7f1d85e00640 1 --2- 172.21.15.47:0/908561578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e15d40 0x55e624e181d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.076+0000 7f1d85e00640 1 -- 172.21.15.47:0/908561578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e624d362c0 con 0x55e624e10bd0 2024-03-21T11:32:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.076+0000 7f1d85400640 1 --2- 172.21.15.47:0/908561578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e101e0 0x55e624e10600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.076+0000 7f1d7ea00640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1d6c0a2fc0 0x7f1d6c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.076+0000 7f1d7ea00640 1 -- 172.21.15.47:0/3017392145 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1d800cfed0 con 0x7f1d881054b0 2024-03-21T11:32:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.076+0000 7f1d85e00640 1 --2- 172.21.15.47:0/908561578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e10bd0 0x55e624e156d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1d78010450 tx=0x7f1d78010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f388687332efdd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.076+0000 7f1d85400640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1d6c0a2fc0 0x7f1d6c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.077+0000 7f1d73400640 1 -- 172.21.15.47:0/908561578 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78011820 con 0x55e624e10bd0 2024-03-21T11:32:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.077+0000 7f1d73400640 1 -- 172.21.15.47:0/908561578 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d780193d0 con 0x55e624e10bd0 2024-03-21T11:32:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.077+0000 7f1d73400640 1 -- 172.21.15.47:0/908561578 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7800f620 con 0x55e624e10bd0 2024-03-21T11:32:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.078+0000 7f1d85400640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1d6c0a2fc0 0x7f1d6c0a5480 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1d7402c730 tx=0x7f1d74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.078+0000 7f1d8fa07740 1 -- 172.21.15.47:0/908561578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e10bd0 msgr2=0x55e624e156d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.078+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/908561578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e10bd0 0x55e624e156d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1d78010450 tx=0x7f1d78010910 comp rx=0 tx=0).stop 2024-03-21T11:32:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.078+0000 7f1d8fa07740 1 -- 172.21.15.47:0/908561578 shutdown_connections 2024-03-21T11:32:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.078+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/908561578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e15d40 0x55e624e181d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.078+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/908561578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e10bd0 0x55e624e156d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.078+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/908561578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e101e0 0x55e624e10600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.078+0000 7f1d8fa07740 1 -- 172.21.15.47:0/908561578 >> 172.21.15.47:0/908561578 conn(0x55e624e0b750 msgr2=0x55e624e0bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.078+0000 7f1d8fa07740 1 -- 172.21.15.47:0/908561578 shutdown_connections 2024-03-21T11:32:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.078+0000 7f1d8fa07740 1 -- 172.21.15.47:0/908561578 wait complete. 2024-03-21T11:32:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.078+0000 7f1d8fa07740 1 -- start start 2024-03-21T11:32:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.079+0000 7f1d8fa07740 1 Processor -- start 2024-03-21T11:32:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.079+0000 7f1d8fa07740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e101e0 0x55e624e1a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.079+0000 7f1d8fa07740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e10bd0 0x55e624e1a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.079+0000 7f1d8fa07740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e15d40 0x55e624e21930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.079+0000 7f1d8fa07740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e624d31a20 con 0x55e624e101e0 2024-03-21T11:32:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.079+0000 7f1d8fa07740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e624d718d0 con 0x55e624e10bd0 2024-03-21T11:32:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.079+0000 7f1d8fa07740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e624e18cf0 con 0x55e624e15d40 2024-03-21T11:32:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.079+0000 7f1d8fa07740 10 client.0 _ll_register_callbacks cb 0x55e624d6e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.079+0000 7f1d85400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e10bd0 0x55e624e1a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.079+0000 7f1d85e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e101e0 0x55e624e1a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d86800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e15d40 0x55e624e21930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d85400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e10bd0 0x55e624e1a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53064/0 (socket says 172.21.15.47:53064) 2024-03-21T11:32:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d85e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e101e0 0x55e624e1a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55866/0 (socket says 172.21.15.47:55866) 2024-03-21T11:32:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d85400640 1 -- 172.21.15.47:0/1293276355 learned_addr learned my addr 172.21.15.47:0/1293276355 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d86800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e15d40 0x55e624e21930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41998/0 (socket says 172.21.15.47:41998) 2024-03-21T11:32:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d86800640 1 -- 172.21.15.47:0/1293276355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e10bd0 msgr2=0x55e624e1a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d86800640 1 --2- 172.21.15.47:0/1293276355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e10bd0 0x55e624e1a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d86800640 1 -- 172.21.15.47:0/1293276355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e101e0 msgr2=0x55e624e1a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d86800640 1 --2- 172.21.15.47:0/1293276355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e101e0 0x55e624e1a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d86800640 1 -- 172.21.15.47:0/1293276355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e624a8a210 con 0x55e624e15d40 2024-03-21T11:32:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d86800640 1 --2- 172.21.15.47:0/1293276355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e15d40 0x55e624e21930 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1d8009bf10 tx=0x7f1d8009c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=661706e7cb322661 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d67e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d67e00640 1 -- 172.21.15.47:0/1293276355 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800a4790 con 0x55e624e15d40 2024-03-21T11:32:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d67e00640 1 -- 172.21.15.47:0/1293276355 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1d800a4930 con 0x55e624e15d40 2024-03-21T11:32:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.080+0000 7f1d8fa07740 1 -- 172.21.15.47:0/1293276355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d780193d0 con 0x55e624e15d40 2024-03-21T11:32:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.081+0000 7f1d67e00640 1 -- 172.21.15.47:0/1293276355 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d800b5200 con 0x55e624e15d40 2024-03-21T11:32:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.081+0000 7f1d8fa07740 10 client.28435 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.081+0000 7f1d8fa07740 1 -- 172.21.15.47:0/1293276355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e624b56620 con 0x55e624e15d40 2024-03-21T11:32:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.081+0000 7f1d67e00640 1 -- 172.21.15.47:0/1293276355 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d780193d0 con 0x55e624e15d40 2024-03-21T11:32:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.081+0000 7f1d67e00640 1 -- 172.21.15.47:0/1293276355 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1d8009b0d0 con 0x55e624e15d40 2024-03-21T11:32:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.081+0000 7f1d8fa07740 10 client.28435 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.081+0000 7f1d8fa07740 10 client.28435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.081+0000 7f1d8fa07740 20 client.28435 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.081+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/1293276355 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e624e26290 0x55e624e28680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.081+0000 7f1d8fa07740 4 client.28435 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.081+0000 7f1d8fa07740 1 -- 172.21.15.47:0/1293276355 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e624d82d70 con 0x55e624e26290 2024-03-21T11:32:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.082+0000 7f1d85e00640 1 --2- 172.21.15.47:0/1293276355 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e624e26290 0x55e624e28680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.082+0000 7f1d85e00640 1 --2- 172.21.15.47:0/1293276355 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e624e26290 0x55e624e28680 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c417bfead9df6c server_cookie=872801690a4b7988 in_seq=0 out_seq=0 2024-03-21T11:32:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.082+0000 7f1d67e00640 10 client.28435 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.083+0000 7f1d67e00640 1 -- 172.21.15.47:0/1293276355 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e624d82d70 con 0x55e624e26290 2024-03-21T11:32:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.083+0000 7f1d67e00640 10 client.28435 handle_command_reply: tid=0 2024-03-21T11:32:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.083+0000 7f1d8fa07740 1 client.28435 shutdown 2024-03-21T11:32:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.083+0000 7f1d8fa07740 1 -- 172.21.15.47:0/1293276355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e15d40 msgr2=0x55e624e21930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.083+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/1293276355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e15d40 0x55e624e21930 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1d8009bf10 tx=0x7f1d8009c2d0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.083+0000 7f1d8fa07740 1 -- 172.21.15.47:0/1293276355 shutdown_connections 2024-03-21T11:32:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.083+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/1293276355 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e624e26290 0x55e624e28680 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.083+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/1293276355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e15d40 0x55e624e21930 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.083+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/1293276355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e10bd0 0x55e624e1a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.083+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/1293276355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e101e0 0x55e624e1a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.083+0000 7f1d8fa07740 1 -- 172.21.15.47:0/1293276355 >> 172.21.15.47:0/1293276355 conn(0x55e624e0b750 msgr2=0x55e624e0a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.084+0000 7f1d67e00640 0 client.28435 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.084+0000 7f1d8fa07740 1 -- 172.21.15.47:0/1293276355 shutdown_connections 2024-03-21T11:32:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.084+0000 7f1d8fa07740 1 -- 172.21.15.47:0/1293276355 wait complete. 2024-03-21T11:32:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.084+0000 7f1d8fa07740 20 client.28435 trim_cache size 0 max 16384 2024-03-21T11:32:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.103+0000 7f1d8fa07740 1 Processor -- start 2024-03-21T11:32:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.103+0000 7f1d8fa07740 1 -- start start 2024-03-21T11:32:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d8fa07740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e101e0 0x55e624e1a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d8fa07740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e10bd0 0x55e624e1a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d8fa07740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e15d40 0x55e624e218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d8fa07740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881ac1d0 con 0x55e624e15d40 2024-03-21T11:32:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d8fa07740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881adb50 con 0x55e624e101e0 2024-03-21T11:32:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d8fa07740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881ac830 con 0x55e624e10bd0 2024-03-21T11:32:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d85e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e101e0 0x55e624e1a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d85400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e10bd0 0x55e624e1a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d85e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e101e0 0x55e624e1a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53072/0 (socket says 172.21.15.47:53072) 2024-03-21T11:32:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d86800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e15d40 0x55e624e218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d85e00640 1 -- 172.21.15.47:0/288748901 learned_addr learned my addr 172.21.15.47:0/288748901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d85400640 1 -- 172.21.15.47:0/288748901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e101e0 msgr2=0x55e624e1a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d85400640 1 --2- 172.21.15.47:0/288748901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e101e0 0x55e624e1a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d85400640 1 -- 172.21.15.47:0/288748901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e15d40 msgr2=0x55e624e218f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d85400640 1 --2- 172.21.15.47:0/288748901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e15d40 0x55e624e218f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d85400640 1 -- 172.21.15.47:0/288748901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e624e119e0 con 0x55e624e10bd0 2024-03-21T11:32:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d85e00640 1 --2- 172.21.15.47:0/288748901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e101e0 0x55e624e1a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.104+0000 7f1d86800640 1 --2- 172.21.15.47:0/288748901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e15d40 0x55e624e218f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.105+0000 7f1d85400640 1 --2- 172.21.15.47:0/288748901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e10bd0 0x55e624e1a960 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x55e624e1b890 tx=0x7f1d7403c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68ea043e13d2663 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.105+0000 7f1d72a00640 1 -- 172.21.15.47:0/288748901 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74047030 con 0x55e624e10bd0 2024-03-21T11:32:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.105+0000 7f1d72a00640 1 -- 172.21.15.47:0/288748901 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d74048040 con 0x55e624e10bd0 2024-03-21T11:32:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.105+0000 7f1d72a00640 1 -- 172.21.15.47:0/288748901 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74003b80 con 0x55e624e10bd0 2024-03-21T11:32:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.106+0000 7f1d8fa07740 1 -- 172.21.15.47:0/288748901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e10bd0 msgr2=0x55e624e1a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.106+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/288748901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e10bd0 0x55e624e1a960 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x55e624e1b890 tx=0x7f1d7403c000 comp rx=0 tx=0).stop 2024-03-21T11:32:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.106+0000 7f1d8fa07740 1 -- 172.21.15.47:0/288748901 shutdown_connections 2024-03-21T11:32:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.106+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/288748901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e15d40 0x55e624e218f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.106+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/288748901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e10bd0 0x55e624e1a960 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.106+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/288748901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e101e0 0x55e624e1a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.106+0000 7f1d8fa07740 1 -- 172.21.15.47:0/288748901 >> 172.21.15.47:0/288748901 conn(0x55e624e0b750 msgr2=0x55e624e190f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.106+0000 7f1d8fa07740 1 -- 172.21.15.47:0/288748901 shutdown_connections 2024-03-21T11:32:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.106+0000 7f1d8fa07740 1 -- 172.21.15.47:0/288748901 wait complete. 2024-03-21T11:32:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.106+0000 7f1d8fa07740 1 -- start start 2024-03-21T11:32:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.107+0000 7f1d8fa07740 1 Processor -- start 2024-03-21T11:32:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.107+0000 7f1d8fa07740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e101e0 0x55e624e1ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.107+0000 7f1d8fa07740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e10bd0 0x55e624e1f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.107+0000 7f1d8fa07740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e15d40 0x55e624e20240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.107+0000 7f1d8fa07740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881ad4f0 con 0x55e624e101e0 2024-03-21T11:32:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.107+0000 7f1d8fa07740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881ace90 con 0x55e624e10bd0 2024-03-21T11:32:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.107+0000 7f1d8fa07740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881aa390 con 0x55e624e15d40 2024-03-21T11:32:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.107+0000 7f1d8fa07740 10 client.0 _ll_register_callbacks cb 0x7f1d800984b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d85e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e101e0 0x55e624e1ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d85400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e10bd0 0x55e624e1f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d86800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e15d40 0x55e624e20240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d85400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e10bd0 0x55e624e1f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53084/0 (socket says 172.21.15.47:53084) 2024-03-21T11:32:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d85e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e101e0 0x55e624e1ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55870/0 (socket says 172.21.15.47:55870) 2024-03-21T11:32:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d85400640 1 -- 172.21.15.47:0/3903080590 learned_addr learned my addr 172.21.15.47:0/3903080590 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d86800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e15d40 0x55e624e20240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42018/0 (socket says 172.21.15.47:42018) 2024-03-21T11:32:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d86800640 1 -- 172.21.15.47:0/3903080590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e10bd0 msgr2=0x55e624e1f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d86800640 1 --2- 172.21.15.47:0/3903080590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e10bd0 0x55e624e1f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d86800640 1 -- 172.21.15.47:0/3903080590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e101e0 msgr2=0x55e624e1ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d86800640 1 --2- 172.21.15.47:0/3903080590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e101e0 0x55e624e1ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d86800640 1 -- 172.21.15.47:0/3903080590 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e624e20830 con 0x55e624e15d40 2024-03-21T11:32:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d85e00640 1 --2- 172.21.15.47:0/3903080590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e101e0 0x55e624e1ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d85400640 1 --2- 172.21.15.47:0/3903080590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e10bd0 0x55e624e1f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d86800640 1 --2- 172.21.15.47:0/3903080590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e15d40 0x55e624e20240 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1d800a46f0 tx=0x7f1d8009a720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9309e13b0f1eae6c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d67e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.108+0000 7f1d67e00640 1 -- 172.21.15.47:0/3903080590 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8007e030 con 0x55e624e15d40 2024-03-21T11:32:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.109+0000 7f1d8fa07740 1 -- 172.21.15.47:0/3903080590 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d74048040 con 0x55e624e15d40 2024-03-21T11:32:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.109+0000 7f1d67e00640 1 -- 172.21.15.47:0/3903080590 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1d8009b660 con 0x55e624e15d40 2024-03-21T11:32:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.109+0000 7f1d67e00640 1 -- 172.21.15.47:0/3903080590 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d8009bfe0 con 0x55e624e15d40 2024-03-21T11:32:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.109+0000 7f1d8fa07740 10 client.28441 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.109+0000 7f1d8fa07740 1 -- 172.21.15.47:0/3903080590 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e624e21bb0 con 0x55e624e15d40 2024-03-21T11:32:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.109+0000 7f1d67e00640 1 -- 172.21.15.47:0/3903080590 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d74048040 con 0x55e624e15d40 2024-03-21T11:32:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.109+0000 7f1d67e00640 1 -- 172.21.15.47:0/3903080590 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1d800a4800 con 0x55e624e15d40 2024-03-21T11:32:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.109+0000 7f1d8fa07740 10 client.28441 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.109+0000 7f1d8fa07740 10 client.28441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.109+0000 7f1d8fa07740 20 client.28441 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.109+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/3903080590 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e624e82180 0x55e624e84570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.109+0000 7f1d8fa07740 4 client.28441 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:32:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.109+0000 7f1d8fa07740 1 -- 172.21.15.47:0/3903080590 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e624d7d0c0 con 0x55e624e82180 2024-03-21T11:32:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.110+0000 7f1d85e00640 1 --2- 172.21.15.47:0/3903080590 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e624e82180 0x55e624e84570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.110+0000 7f1d85e00640 1 --2- 172.21.15.47:0/3903080590 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e624e82180 0x55e624e84570 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f3dbbc2bf0bc40 server_cookie=4c8d8dbd107404c1 in_seq=0 out_seq=0 2024-03-21T11:32:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.110+0000 7f1d67e00640 10 client.28441 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:04 smithi102 ceph-mon[36748]: pgmap v662: 129 pgs: 129 active+clean; 39 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-03-21T11:32:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:04 smithi047 ceph-mon[31351]: pgmap v662: 129 pgs: 129 active+clean; 39 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-03-21T11:32:05.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d67e00640 1 -- 172.21.15.47:0/3903080590 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e624d7d0c0 con 0x55e624e82180 2024-03-21T11:32:05.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d67e00640 10 client.28441 handle_command_reply: tid=0 2024-03-21T11:32:05.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d8fa07740 1 client.28441 shutdown 2024-03-21T11:32:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d8fa07740 1 -- 172.21.15.47:0/3903080590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e15d40 msgr2=0x55e624e20240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/3903080590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e15d40 0x55e624e20240 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1d800a46f0 tx=0x7f1d8009a720 comp rx=0 tx=0).stop 2024-03-21T11:32:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d8fa07740 1 -- 172.21.15.47:0/3903080590 shutdown_connections 2024-03-21T11:32:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/3903080590 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e624e82180 0x55e624e84570 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/3903080590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e624e15d40 0x55e624e20240 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/3903080590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e624e10bd0 0x55e624e1f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d8fa07740 1 --2- 172.21.15.47:0/3903080590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e624e101e0 0x55e624e1ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d67e00640 0 client.28441 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d8fa07740 1 -- 172.21.15.47:0/3903080590 >> 172.21.15.47:0/3903080590 conn(0x55e624e0b750 msgr2=0x55e624e1a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d8fa07740 1 -- 172.21.15.47:0/3903080590 shutdown_connections 2024-03-21T11:32:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d8fa07740 1 -- 172.21.15.47:0/3903080590 wait complete. 2024-03-21T11:32:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.361+0000 7f1d8fa07740 20 client.28441 trim_cache size 0 max 16384 2024-03-21T11:32:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:32:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:32:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:32:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:32:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.362+0000 7f1d8ce00640 1 -- 172.21.15.47:0/3017392145 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1d6c0a2fc0 msgr2=0x7f1d6c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.362+0000 7f1d8ce00640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1d6c0a2fc0 0x7f1d6c0a5480 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1d7402c730 tx=0x7f1d74009260 comp rx=0 tx=0).stop 2024-03-21T11:32:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.362+0000 7f1d8ce00640 1 -- 172.21.15.47:0/3017392145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d881054b0 msgr2=0x7f1d881a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.362+0000 7f1d8ce00640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d881054b0 0x7f1d881a1190 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009fd0 tx=0x7f1d8000ef10 comp rx=0 tx=0).stop 2024-03-21T11:32:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.362+0000 7f1d8ce00640 1 -- 172.21.15.47:0/3017392145 shutdown_connections 2024-03-21T11:32:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.362+0000 7f1d8ce00640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1d6c0a2fc0 0x7f1d6c0a5480 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.362+0000 7f1d8ce00640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d881054b0 0x7f1d881a1190 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.362+0000 7f1d8ce00640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d88104af0 0x7f1d8819a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.362+0000 7f1d8ce00640 1 --2- 172.21.15.47:0/3017392145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d88103900 0x7f1d88199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.363+0000 7f1d8ce00640 1 -- 172.21.15.47:0/3017392145 >> 172.21.15.47:0/3017392145 conn(0x7f1d880fee90 msgr2=0x7f1d88107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.363+0000 7f1d8ce00640 1 -- 172.21.15.47:0/3017392145 shutdown_connections 2024-03-21T11:32:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:05.363+0000 7f1d8ce00640 1 -- 172.21.15.47:0/3017392145 wait complete. 2024-03-21T11:32:05.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:04 smithi100 ceph-mon[36245]: pgmap v662: 129 pgs: 129 active+clean; 39 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-03-21T11:32:05.541 INFO:tasks.workunit.client.0.smithi047.stdout: 1 631013 8.17 MB/sec execute 289 sec latency 180.604 ms 2024-03-21T11:32:06.541 INFO:tasks.workunit.client.0.smithi047.stdout: 1 632539 8.19 MB/sec execute 290 sec latency 17.480 ms 2024-03-21T11:32:07.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:06 smithi102 ceph-mon[36748]: pgmap v663: 129 pgs: 129 active+clean; 39 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-03-21T11:32:07.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:06 smithi047 ceph-mon[31351]: pgmap v663: 129 pgs: 129 active+clean; 39 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-03-21T11:32:07.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:06 smithi100 ceph-mon[36245]: pgmap v663: 129 pgs: 129 active+clean; 39 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-03-21T11:32:07.541 INFO:tasks.workunit.client.0.smithi047.stdout: 1 633689 8.18 MB/sec execute 291 sec latency 31.709 ms 2024-03-21T11:32:08.541 INFO:tasks.workunit.client.0.smithi047.stdout: 1 635919 8.20 MB/sec execute 292 sec latency 16.197 ms 2024-03-21T11:32:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:08 smithi102 ceph-mon[36748]: pgmap v664: 129 pgs: 129 active+clean; 41 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 293 op/s 2024-03-21T11:32:09.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:08 smithi047 ceph-mon[31351]: pgmap v664: 129 pgs: 129 active+clean; 41 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 293 op/s 2024-03-21T11:32:09.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:08 smithi100 ceph-mon[36245]: pgmap v664: 129 pgs: 129 active+clean; 41 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 293 op/s 2024-03-21T11:32:09.541 INFO:tasks.workunit.client.0.smithi047.stdout: 1 636941 8.18 MB/sec execute 293 sec latency 10.607 ms 2024-03-21T11:32:10.541 INFO:tasks.workunit.client.0.smithi047.stdout: 1 639344 8.20 MB/sec execute 294 sec latency 10.131 ms 2024-03-21T11:32:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:10 smithi102 ceph-mon[36748]: pgmap v665: 129 pgs: 129 active+clean; 41 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 296 op/s 2024-03-21T11:32:11.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:10 smithi047 ceph-mon[31351]: pgmap v665: 129 pgs: 129 active+clean; 41 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 296 op/s 2024-03-21T11:32:11.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:10 smithi100 ceph-mon[36245]: pgmap v665: 129 pgs: 129 active+clean; 41 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 296 op/s 2024-03-21T11:32:11.542 INFO:tasks.workunit.client.0.smithi047.stdout: 1 640324 8.19 MB/sec execute 295 sec latency 15.225 ms 2024-03-21T11:32:12.542 INFO:tasks.workunit.client.0.smithi047.stdout: 1 642464 8.20 MB/sec execute 296 sec latency 28.206 ms 2024-03-21T11:32:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:12 smithi102 ceph-mon[36748]: pgmap v666: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.8 MiB/s wr, 287 op/s 2024-03-21T11:32:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:12 smithi047 ceph-mon[31351]: pgmap v666: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.8 MiB/s wr, 287 op/s 2024-03-21T11:32:13.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:12 smithi100 ceph-mon[36245]: pgmap v666: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.8 MiB/s wr, 287 op/s 2024-03-21T11:32:13.542 INFO:tasks.workunit.client.0.smithi047.stdout: 1 643522 8.19 MB/sec execute 297 sec latency 48.284 ms 2024-03-21T11:32:14.542 INFO:tasks.workunit.client.0.smithi047.stdout: 1 645390 8.19 MB/sec execute 298 sec latency 11.979 ms 2024-03-21T11:32:15.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:14 smithi102 ceph-mon[36748]: pgmap v667: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-03-21T11:32:15.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:14 smithi047 ceph-mon[31351]: pgmap v667: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-03-21T11:32:15.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:14 smithi100 ceph-mon[36245]: pgmap v667: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-03-21T11:32:15.542 INFO:tasks.workunit.client.0.smithi047.stdout: 1 646902 8.20 MB/sec execute 299 sec latency 10.980 ms 2024-03-21T11:32:16.542 INFO:tasks.workunit.client.0.smithi047.stdout: 1 648056 8.19 MB/sec execute 300 sec latency 59.916 ms 2024-03-21T11:32:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:16 smithi102 ceph-mon[36748]: pgmap v668: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 2.0 MiB/s wr, 281 op/s 2024-03-21T11:32:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:16 smithi047 ceph-mon[31351]: pgmap v668: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 2.0 MiB/s wr, 281 op/s 2024-03-21T11:32:17.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:16 smithi100 ceph-mon[36245]: pgmap v668: 129 pgs: 129 active+clean; 42 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 2.0 MiB/s wr, 281 op/s 2024-03-21T11:32:17.543 INFO:tasks.workunit.client.0.smithi047.stdout: 1 650115 8.21 MB/sec execute 301 sec latency 35.832 ms 2024-03-21T11:32:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.511+0000 7efc0fe00640 1 -- 172.21.15.47:0/540057392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc081073c0 msgr2=0x7efc08109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.511+0000 7efc0fe00640 1 --2- 172.21.15.47:0/540057392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc081073c0 0x7efc08109810 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009900 tx=0x7efbf802bc30 comp rx=0 tx=0).stop 2024-03-21T11:32:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.511+0000 7efc0fe00640 1 -- 172.21.15.47:0/540057392 shutdown_connections 2024-03-21T11:32:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.511+0000 7efc0fe00640 1 --2- 172.21.15.47:0/540057392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc081073c0 0x7efc08109810 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.511+0000 7efc0fe00640 1 --2- 172.21.15.47:0/540057392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc08104a90 0x7efc08106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.511+0000 7efc0fe00640 1 --2- 172.21.15.47:0/540057392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc08102160 0x7efc08104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.511+0000 7efc0fe00640 1 -- 172.21.15.47:0/540057392 >> 172.21.15.47:0/540057392 conn(0x7efc080fbb50 msgr2=0x7efc080fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.511+0000 7efc0fe00640 1 -- 172.21.15.47:0/540057392 shutdown_connections 2024-03-21T11:32:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.511+0000 7efc0fe00640 1 -- 172.21.15.47:0/540057392 wait complete. 2024-03-21T11:32:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.512+0000 7efc0fe00640 1 Processor -- start 2024-03-21T11:32:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.513+0000 7efc0fe00640 1 -- start start 2024-03-21T11:32:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.513+0000 7efc0fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc08102160 0x7efc08199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.513+0000 7efc0fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc08104a90 0x7efc0819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.513+0000 7efc0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc081073c0 0x7efc081a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.513+0000 7efc0fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0810d3d0 con 0x7efc08104a90 2024-03-21T11:32:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.513+0000 7efc0fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0810d250 con 0x7efc08102160 2024-03-21T11:32:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.513+0000 7efc0fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0810d550 con 0x7efc081073c0 2024-03-21T11:32:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.513+0000 7efc0d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc08102160 0x7efc08199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.513+0000 7efc0d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc08102160 0x7efc08199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59150/0 (socket says 172.21.15.47:59150) 2024-03-21T11:32:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.513+0000 7efc0d400640 1 -- 172.21.15.47:0/278992792 learned_addr learned my addr 172.21.15.47:0/278992792 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.513+0000 7efc0ca00640 1 --2- 172.21.15.47:0/278992792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc08104a90 0x7efc0819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.513+0000 7efc0de00640 1 --2- 172.21.15.47:0/278992792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc081073c0 0x7efc081a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.514+0000 7efc0d400640 1 -- 172.21.15.47:0/278992792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc081073c0 msgr2=0x7efc081a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.514+0000 7efc0d400640 1 --2- 172.21.15.47:0/278992792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc081073c0 0x7efc081a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.514+0000 7efc0d400640 1 -- 172.21.15.47:0/278992792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc08104a90 msgr2=0x7efc0819a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.514+0000 7efc0d400640 1 --2- 172.21.15.47:0/278992792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc08104a90 0x7efc0819a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.514+0000 7efc0d400640 1 -- 172.21.15.47:0/278992792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc081a1740 con 0x7efc08102160 2024-03-21T11:32:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.514+0000 7efc0d400640 1 --2- 172.21.15.47:0/278992792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc08102160 0x7efc08199b20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efc0000ebc0 tx=0x7efc0000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.514+0000 7efc06000640 1 -- 172.21.15.47:0/278992792 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc00010630 con 0x7efc08102160 2024-03-21T11:32:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.514+0000 7efc0fe00640 1 -- 172.21.15.47:0/278992792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc081a1a30 con 0x7efc08102160 2024-03-21T11:32:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.514+0000 7efc06000640 1 -- 172.21.15.47:0/278992792 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc00010cb0 con 0x7efc08102160 2024-03-21T11:32:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.514+0000 7efc06000640 1 -- 172.21.15.47:0/278992792 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc00018bd0 con 0x7efc08102160 2024-03-21T11:32:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.515+0000 7efc0fe00640 1 -- 172.21.15.47:0/278992792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc081a1f80 con 0x7efc08102160 2024-03-21T11:32:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.518+0000 7efc12bc5740 1 Processor -- start 2024-03-21T11:32:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.518+0000 7efc12bc5740 1 -- start start 2024-03-21T11:32:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.518+0000 7efc06000640 1 -- 172.21.15.47:0/278992792 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7efc000107d0 con 0x7efc08102160 2024-03-21T11:32:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.518+0000 7efc12bc5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad871e0 0x559f6ad87600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.518+0000 7efc12bc5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad8c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.518+0000 7efc12bc5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 0x559f6ad8f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.518+0000 7efc12bc5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6ad4a3b0 con 0x559f6ad8cd40 2024-03-21T11:32:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.518+0000 7efc12bc5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6acec830 con 0x559f6ad871e0 2024-03-21T11:32:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.518+0000 7efc12bc5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6ac955d0 con 0x559f6ad87bd0 2024-03-21T11:32:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.518+0000 7efc0de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad871e0 0x559f6ad87600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.518+0000 7efc0de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad871e0 0x559f6ad87600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59162/0 (socket says 172.21.15.47:59162) 2024-03-21T11:32:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.518+0000 7efc0de00640 1 -- 172.21.15.47:0/714863290 learned_addr learned my addr 172.21.15.47:0/714863290 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.518+0000 7efc0d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad8c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.519+0000 7efc0ca00640 1 --2- 172.21.15.47:0/714863290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 0x559f6ad8f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.519+0000 7efc0de00640 1 -- 172.21.15.47:0/714863290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 msgr2=0x559f6ad8c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.519+0000 7efc0de00640 1 --2- 172.21.15.47:0/714863290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad8c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.519+0000 7efc0de00640 1 -- 172.21.15.47:0/714863290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 msgr2=0x559f6ad8f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.519+0000 7efc0de00640 1 --2- 172.21.15.47:0/714863290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 0x559f6ad8f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.519+0000 7efc0de00640 1 -- 172.21.15.47:0/714863290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f6acad2c0 con 0x559f6ad871e0 2024-03-21T11:32:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.519+0000 7efc0de00640 1 --2- 172.21.15.47:0/714863290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad871e0 0x559f6ad87600 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efbf802c240 tx=0x7efbf802c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6ccc4ce7fe9477a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.520+0000 7efbfe600640 1 -- 172.21.15.47:0/714863290 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf8044020 con 0x559f6ad871e0 2024-03-21T11:32:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.520+0000 7efbfe600640 1 -- 172.21.15.47:0/714863290 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efbf802cb80 con 0x559f6ad871e0 2024-03-21T11:32:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.520+0000 7efbfe600640 1 -- 172.21.15.47:0/714863290 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf8004920 con 0x559f6ad871e0 2024-03-21T11:32:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.520+0000 7efc06000640 1 --2- 172.21.15.47:0/278992792 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efbec0a2fc0 0x7efbec0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.520+0000 7efc06000640 1 -- 172.21.15.47:0/278992792 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efc000cfec0 con 0x7efc08102160 2024-03-21T11:32:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.520+0000 7efc0d400640 1 --2- 172.21.15.47:0/278992792 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efbec0a2fc0 0x7efbec0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.520+0000 7efc12bc5740 1 -- 172.21.15.47:0/714863290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad871e0 msgr2=0x559f6ad87600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.520+0000 7efc12bc5740 1 --2- 172.21.15.47:0/714863290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad871e0 0x559f6ad87600 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efbf802c240 tx=0x7efbf802c680 comp rx=0 tx=0).stop 2024-03-21T11:32:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.521+0000 7efc12bc5740 1 -- 172.21.15.47:0/714863290 shutdown_connections 2024-03-21T11:32:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.521+0000 7efc12bc5740 1 --2- 172.21.15.47:0/714863290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 0x559f6ad8f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.521+0000 7efc12bc5740 1 --2- 172.21.15.47:0/714863290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad8c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.521+0000 7efc12bc5740 1 --2- 172.21.15.47:0/714863290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad871e0 0x559f6ad87600 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.521+0000 7efc12bc5740 1 -- 172.21.15.47:0/714863290 >> 172.21.15.47:0/714863290 conn(0x559f6ad82750 msgr2=0x559f6ad82b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.521+0000 7efc12bc5740 1 -- 172.21.15.47:0/714863290 shutdown_connections 2024-03-21T11:32:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.521+0000 7efc12bc5740 1 -- 172.21.15.47:0/714863290 wait complete. 2024-03-21T11:32:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.521+0000 7efc0d400640 1 --2- 172.21.15.47:0/278992792 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efbec0a2fc0 0x7efbec0a5480 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7efc000cd9a0 tx=0x7efc000d39a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.521+0000 7efc12bc5740 1 -- start start 2024-03-21T11:32:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.521+0000 7efc12bc5740 1 Processor -- start 2024-03-21T11:32:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.522+0000 7efc12bc5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad871e0 0x559f6ad91340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.522+0000 7efc12bc5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad91880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.522+0000 7efc12bc5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad8cd40 0x559f6ad988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.522+0000 7efc12bc5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6aca8a20 con 0x559f6ad871e0 2024-03-21T11:32:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.522+0000 7efc12bc5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6ace88d0 con 0x559f6ad8cd40 2024-03-21T11:32:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.522+0000 7efc12bc5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6ad8fc90 con 0x559f6ad87bd0 2024-03-21T11:32:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.522+0000 7efc12bc5740 10 client.0 _ll_register_callbacks cb 0x559f6ace51c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.522+0000 7efc0ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad91880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.522+0000 7efc0de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad871e0 0x559f6ad91340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.522+0000 7efc0ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad91880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59326/0 (socket says 172.21.15.47:59326) 2024-03-21T11:32:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.522+0000 7efc0ca00640 1 -- 172.21.15.47:0/2264823939 learned_addr learned my addr 172.21.15.47:0/2264823939 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.523+0000 7efc0de00640 1 --2- 172.21.15.47:0/2264823939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad8cd40 0x559f6ad988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.523+0000 7efc0ca00640 1 -- 172.21.15.47:0/2264823939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad8cd40 msgr2=0x559f6ad988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.523+0000 7efc0ca00640 1 --2- 172.21.15.47:0/2264823939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad8cd40 0x559f6ad988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.523+0000 7efc0ca00640 1 -- 172.21.15.47:0/2264823939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad871e0 msgr2=0x559f6ad91340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.523+0000 7efc0ca00640 1 --2- 172.21.15.47:0/2264823939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad871e0 0x559f6ad91340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.523+0000 7efc0ca00640 1 -- 172.21.15.47:0/2264823939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f6aa01210 con 0x559f6ad87bd0 2024-03-21T11:32:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.523+0000 7efc0de00640 1 --2- 172.21.15.47:0/2264823939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad8cd40 0x559f6ad988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:32:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.523+0000 7efc0ca00640 1 --2- 172.21.15.47:0/2264823939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad91880 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7efbf4013710 tx=0x7efbf4013bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cacfb85feb67cc46 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.523+0000 7efbf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.523+0000 7efbf2a00640 1 -- 172.21.15.47:0/2264823939 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4004440 con 0x559f6ad87bd0 2024-03-21T11:32:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.523+0000 7efc12bc5740 1 -- 172.21.15.47:0/2264823939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efbf802cb80 con 0x559f6ad87bd0 2024-03-21T11:32:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.523+0000 7efbf2a00640 1 -- 172.21.15.47:0/2264823939 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efbf4011960 con 0x559f6ad87bd0 2024-03-21T11:32:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.523+0000 7efbf2a00640 1 -- 172.21.15.47:0/2264823939 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbf4011b60 con 0x559f6ad87bd0 2024-03-21T11:32:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.523+0000 7efc12bc5740 10 client.28453 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.524+0000 7efc12bc5740 1 -- 172.21.15.47:0/2264823939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559f6aacd620 con 0x559f6ad87bd0 2024-03-21T11:32:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.524+0000 7efbf2a00640 1 -- 172.21.15.47:0/2264823939 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efbf802cb80 con 0x559f6ad87bd0 2024-03-21T11:32:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.524+0000 7efbf2a00640 1 -- 172.21.15.47:0/2264823939 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efbf401a070 con 0x559f6ad87bd0 2024-03-21T11:32:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.524+0000 7efc12bc5740 10 client.28453 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.524+0000 7efc12bc5740 10 client.28453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.524+0000 7efc12bc5740 20 client.28453 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.524+0000 7efc12bc5740 1 --2- 172.21.15.47:0/2264823939 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f6ad9d280 0x559f6ad9f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.524+0000 7efc12bc5740 4 client.28453 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.524+0000 7efc12bc5740 1 -- 172.21.15.47:0/2264823939 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f6acf9d70 con 0x559f6ad9d280 2024-03-21T11:32:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.524+0000 7efc0de00640 1 --2- 172.21.15.47:0/2264823939 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f6ad9d280 0x559f6ad9f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.525+0000 7efc0de00640 1 --2- 172.21.15.47:0/2264823939 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f6ad9d280 0x559f6ad9f6c0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3709bf3365a95b24 server_cookie=ac9a7ef1f3a14631 in_seq=0 out_seq=0 2024-03-21T11:32:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.525+0000 7efbf2a00640 10 client.28453 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.525+0000 7efbf2a00640 1 -- 172.21.15.47:0/2264823939 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559f6acf9d70 con 0x559f6ad9d280 2024-03-21T11:32:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.525+0000 7efbf2a00640 10 client.28453 handle_command_reply: tid=0 2024-03-21T11:32:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.526+0000 7efc12bc5740 1 client.28453 shutdown 2024-03-21T11:32:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.526+0000 7efc12bc5740 1 -- 172.21.15.47:0/2264823939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 msgr2=0x559f6ad91880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.526+0000 7efc12bc5740 1 --2- 172.21.15.47:0/2264823939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad91880 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7efbf4013710 tx=0x7efbf4013bd0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.526+0000 7efc12bc5740 1 -- 172.21.15.47:0/2264823939 shutdown_connections 2024-03-21T11:32:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.526+0000 7efc12bc5740 1 --2- 172.21.15.47:0/2264823939 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f6ad9d280 0x559f6ad9f6c0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.526+0000 7efc12bc5740 1 --2- 172.21.15.47:0/2264823939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad8cd40 0x559f6ad988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.526+0000 7efc12bc5740 1 --2- 172.21.15.47:0/2264823939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad91880 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.526+0000 7efbf2a00640 0 client.28453 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.526+0000 7efc12bc5740 1 --2- 172.21.15.47:0/2264823939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad871e0 0x559f6ad91340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.526+0000 7efc12bc5740 1 -- 172.21.15.47:0/2264823939 >> 172.21.15.47:0/2264823939 conn(0x559f6ad82750 msgr2=0x559f6ad81950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.526+0000 7efc12bc5740 1 -- 172.21.15.47:0/2264823939 shutdown_connections 2024-03-21T11:32:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.526+0000 7efc12bc5740 1 -- 172.21.15.47:0/2264823939 wait complete. 2024-03-21T11:32:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.526+0000 7efc12bc5740 20 client.28453 trim_cache size 0 max 16384 2024-03-21T11:32:18.542 INFO:tasks.workunit.client.0.smithi047.stdout: 1 650834 8.18 MB/sec execute 302 sec latency 41.339 ms 2024-03-21T11:32:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.545+0000 7efc12bc5740 1 Processor -- start 2024-03-21T11:32:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.545+0000 7efc12bc5740 1 -- start start 2024-03-21T11:32:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.545+0000 7efc12bc5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad871e0 0x559f6ad859b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.545+0000 7efc12bc5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad87bd0 0x559f6ad98880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.545+0000 7efc12bc5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 0x559f6ad85ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.545+0000 7efc12bc5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc081ac2e0 con 0x559f6ad8cd40 2024-03-21T11:32:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.545+0000 7efc12bc5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc081adc60 con 0x559f6ad87bd0 2024-03-21T11:32:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.545+0000 7efc12bc5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc081ac940 con 0x559f6ad871e0 2024-03-21T11:32:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.545+0000 7efc0ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad87bd0 0x559f6ad98880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.545+0000 7efc0ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad87bd0 0x559f6ad98880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59186/0 (socket says 172.21.15.47:59186) 2024-03-21T11:32:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.545+0000 7efc0ca00640 1 -- 172.21.15.47:0/1881847046 learned_addr learned my addr 172.21.15.47:0/1881847046 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.545+0000 7efc0de00640 1 --2- 172.21.15.47:0/1881847046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad871e0 0x559f6ad859b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.545+0000 7efc0de00640 1 --2- 172.21.15.47:0/1881847046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 0x559f6ad85ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.546+0000 7efc0ca00640 1 -- 172.21.15.47:0/1881847046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad871e0 msgr2=0x559f6ad859b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.546+0000 7efc0ca00640 1 --2- 172.21.15.47:0/1881847046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad871e0 0x559f6ad859b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.546+0000 7efc0ca00640 1 -- 172.21.15.47:0/1881847046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 msgr2=0x559f6ad85ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.546+0000 7efc0ca00640 1 --2- 172.21.15.47:0/1881847046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 0x559f6ad85ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.546+0000 7efc0ca00640 1 -- 172.21.15.47:0/1881847046 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f6ad88460 con 0x559f6ad87bd0 2024-03-21T11:32:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.546+0000 7efc0ca00640 1 --2- 172.21.15.47:0/1881847046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad87bd0 0x559f6ad98880 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7efbf40136e0 tx=0x7efbf400f9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20145da0b1ccd619 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.546+0000 7efbfdc00640 1 -- 172.21.15.47:0/1881847046 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf402f020 con 0x559f6ad87bd0 2024-03-21T11:32:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.547+0000 7efbfdc00640 1 -- 172.21.15.47:0/1881847046 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efbf40117b0 con 0x559f6ad87bd0 2024-03-21T11:32:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.547+0000 7efbfdc00640 1 -- 172.21.15.47:0/1881847046 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4003830 con 0x559f6ad87bd0 2024-03-21T11:32:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.547+0000 7efc12bc5740 1 -- 172.21.15.47:0/1881847046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad87bd0 msgr2=0x559f6ad98880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.547+0000 7efc12bc5740 1 --2- 172.21.15.47:0/1881847046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad87bd0 0x559f6ad98880 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7efbf40136e0 tx=0x7efbf400f9b0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.547+0000 7efc12bc5740 1 -- 172.21.15.47:0/1881847046 shutdown_connections 2024-03-21T11:32:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.547+0000 7efc12bc5740 1 --2- 172.21.15.47:0/1881847046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 0x559f6ad85ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.547+0000 7efc12bc5740 1 --2- 172.21.15.47:0/1881847046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad87bd0 0x559f6ad98880 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.547+0000 7efc12bc5740 1 --2- 172.21.15.47:0/1881847046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad871e0 0x559f6ad859b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.547+0000 7efc12bc5740 1 -- 172.21.15.47:0/1881847046 >> 172.21.15.47:0/1881847046 conn(0x559f6ad82750 msgr2=0x559f6ad91340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.548+0000 7efc12bc5740 1 -- 172.21.15.47:0/1881847046 shutdown_connections 2024-03-21T11:32:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.548+0000 7efc12bc5740 1 -- 172.21.15.47:0/1881847046 wait complete. 2024-03-21T11:32:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.548+0000 7efc12bc5740 1 -- start start 2024-03-21T11:32:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.548+0000 7efc12bc5740 1 Processor -- start 2024-03-21T11:32:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc12bc5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad871e0 0x559f6ad96510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc12bc5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad96a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc12bc5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 0x559f6ad97a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc12bc5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc081ad600 con 0x559f6ad8cd40 2024-03-21T11:32:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc12bc5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc081acfa0 con 0x559f6ad871e0 2024-03-21T11:32:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc12bc5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc081aa4a0 con 0x559f6ad87bd0 2024-03-21T11:32:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc12bc5740 10 client.0 _ll_register_callbacks cb 0x7efbf4009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc0ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad96a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc0de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad871e0 0x559f6ad96510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc0ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad96a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59344/0 (socket says 172.21.15.47:59344) 2024-03-21T11:32:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc0ca00640 1 -- 172.21.15.47:0/2583300363 learned_addr learned my addr 172.21.15.47:0/2583300363 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc0de00640 1 --2- 172.21.15.47:0/2583300363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 0x559f6ad97a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc0ca00640 1 -- 172.21.15.47:0/2583300363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad871e0 msgr2=0x559f6ad96510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc0ca00640 1 --2- 172.21.15.47:0/2583300363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad871e0 0x559f6ad96510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc0ca00640 1 -- 172.21.15.47:0/2583300363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 msgr2=0x559f6ad97a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc0ca00640 1 --2- 172.21.15.47:0/2583300363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 0x559f6ad97a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc0ca00640 1 -- 172.21.15.47:0/2583300363 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f6ad98050 con 0x559f6ad87bd0 2024-03-21T11:32:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.549+0000 7efc0de00640 1 --2- 172.21.15.47:0/2583300363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 0x559f6ad97a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.550+0000 7efc0ca00640 1 --2- 172.21.15.47:0/2583300363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad96a50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7efbf400f7e0 tx=0x7efbf4011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a388b9c6bc98fbd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.550+0000 7efbf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.550+0000 7efbf2a00640 1 -- 172.21.15.47:0/2583300363 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4000e60 con 0x559f6ad87bd0 2024-03-21T11:32:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.550+0000 7efc12bc5740 1 -- 172.21.15.47:0/2583300363 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efbf40117b0 con 0x559f6ad87bd0 2024-03-21T11:32:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.550+0000 7efbf2a00640 1 -- 172.21.15.47:0/2583300363 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efbf40035f0 con 0x559f6ad87bd0 2024-03-21T11:32:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.550+0000 7efbf2a00640 1 -- 172.21.15.47:0/2583300363 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbf4045050 con 0x559f6ad87bd0 2024-03-21T11:32:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.550+0000 7efc12bc5740 10 client.28459 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.550+0000 7efc12bc5740 1 -- 172.21.15.47:0/2583300363 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559f6ae01900 con 0x559f6ad87bd0 2024-03-21T11:32:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.551+0000 7efbf2a00640 1 -- 172.21.15.47:0/2583300363 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efbf40117b0 con 0x559f6ad87bd0 2024-03-21T11:32:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.551+0000 7efbf2a00640 1 -- 172.21.15.47:0/2583300363 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efbf401a050 con 0x559f6ad87bd0 2024-03-21T11:32:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.551+0000 7efc12bc5740 10 client.28459 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.551+0000 7efc12bc5740 10 client.28459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.551+0000 7efc12bc5740 20 client.28459 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.551+0000 7efc12bc5740 1 --2- 172.21.15.47:0/2583300363 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f6ae02200 0x559f6ae04620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.551+0000 7efc12bc5740 4 client.28459 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:32:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.551+0000 7efc12bc5740 1 -- 172.21.15.47:0/2583300363 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559f6acf40c0 con 0x559f6ae02200 2024-03-21T11:32:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.551+0000 7efc0de00640 1 --2- 172.21.15.47:0/2583300363 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f6ae02200 0x559f6ae04620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.552+0000 7efc0de00640 1 --2- 172.21.15.47:0/2583300363 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f6ae02200 0x559f6ae04620 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12113c678ae4c09a server_cookie=608873b23aa732b7 in_seq=0 out_seq=0 2024-03-21T11:32:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.552+0000 7efbf2a00640 10 client.28459 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:18.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:32:18.712 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:32:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.806+0000 7efbf2a00640 1 -- 172.21.15.47:0/2583300363 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559f6acf40c0 con 0x559f6ae02200 2024-03-21T11:32:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.806+0000 7efbf2a00640 10 client.28459 handle_command_reply: tid=0 2024-03-21T11:32:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.806+0000 7efc12bc5740 1 client.28459 shutdown 2024-03-21T11:32:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.807+0000 7efc12bc5740 1 -- 172.21.15.47:0/2583300363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 msgr2=0x559f6ad96a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.807+0000 7efc12bc5740 1 --2- 172.21.15.47:0/2583300363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad96a50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7efbf400f7e0 tx=0x7efbf4011a50 comp rx=0 tx=0).stop 2024-03-21T11:32:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.807+0000 7efc12bc5740 1 -- 172.21.15.47:0/2583300363 shutdown_connections 2024-03-21T11:32:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.807+0000 7efc12bc5740 1 --2- 172.21.15.47:0/2583300363 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f6ae02200 0x559f6ae04620 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.807+0000 7efc12bc5740 1 --2- 172.21.15.47:0/2583300363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f6ad8cd40 0x559f6ad97a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.807+0000 7efc12bc5740 1 --2- 172.21.15.47:0/2583300363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f6ad87bd0 0x559f6ad96a50 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.807+0000 7efc12bc5740 1 --2- 172.21.15.47:0/2583300363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ad871e0 0x559f6ad96510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.807+0000 7efc12bc5740 1 -- 172.21.15.47:0/2583300363 >> 172.21.15.47:0/2583300363 conn(0x559f6ad82750 msgr2=0x559f6ad859b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.807+0000 7efbf2a00640 0 client.28459 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.807+0000 7efc12bc5740 1 -- 172.21.15.47:0/2583300363 shutdown_connections 2024-03-21T11:32:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.807+0000 7efc12bc5740 1 -- 172.21.15.47:0/2583300363 wait complete. 2024-03-21T11:32:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.807+0000 7efc12bc5740 20 client.28459 trim_cache size 0 max 16384 2024-03-21T11:32:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:32:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:32:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:32:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:32:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.811+0000 7efc0fe00640 1 -- 172.21.15.47:0/278992792 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efbec0a2fc0 msgr2=0x7efbec0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.811+0000 7efc0fe00640 1 --2- 172.21.15.47:0/278992792 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efbec0a2fc0 0x7efbec0a5480 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7efc000cd9a0 tx=0x7efc000d39a0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.811+0000 7efc0fe00640 1 -- 172.21.15.47:0/278992792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc08102160 msgr2=0x7efc08199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.811+0000 7efc0fe00640 1 --2- 172.21.15.47:0/278992792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc08102160 0x7efc08199b20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efc0000ebc0 tx=0x7efc0000c6a0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.811+0000 7efc0fe00640 1 -- 172.21.15.47:0/278992792 shutdown_connections 2024-03-21T11:32:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.811+0000 7efc0fe00640 1 --2- 172.21.15.47:0/278992792 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efbec0a2fc0 0x7efbec0a5480 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.811+0000 7efc0fe00640 1 --2- 172.21.15.47:0/278992792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc081073c0 0x7efc081a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.811+0000 7efc0fe00640 1 --2- 172.21.15.47:0/278992792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc08104a90 0x7efc0819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.811+0000 7efc0fe00640 1 --2- 172.21.15.47:0/278992792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc08102160 0x7efc08199b20 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.811+0000 7efc0fe00640 1 -- 172.21.15.47:0/278992792 >> 172.21.15.47:0/278992792 conn(0x7efc080fbb50 msgr2=0x7efc08101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.811+0000 7efc0fe00640 1 -- 172.21.15.47:0/278992792 shutdown_connections 2024-03-21T11:32:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:18.812+0000 7efc0fe00640 1 -- 172.21.15.47:0/278992792 wait complete. 2024-03-21T11:32:18.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.947+0000 7f1876600640 1 -- 172.21.15.47:0/2565375513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18701073a0 msgr2=0x7f18701097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.947+0000 7f1876600640 1 --2- 172.21.15.47:0/2565375513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18701073a0 0x7f18701097f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1868009ca0 tx=0x7f186802c100 comp rx=0 tx=0).stop 2024-03-21T11:32:18.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.947+0000 7f1876600640 1 -- 172.21.15.47:0/2565375513 shutdown_connections 2024-03-21T11:32:18.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.947+0000 7f1876600640 1 --2- 172.21.15.47:0/2565375513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18701073a0 0x7f18701097f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.947+0000 7f1876600640 1 --2- 172.21.15.47:0/2565375513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1870104a70 0x7f1870106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.947+0000 7f1876600640 1 --2- 172.21.15.47:0/2565375513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1870102140 0x7f1870104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.947+0000 7f1876600640 1 -- 172.21.15.47:0/2565375513 >> 172.21.15.47:0/2565375513 conn(0x7f18700fbb30 msgr2=0x7f18700fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:18.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.947+0000 7f1876600640 1 -- 172.21.15.47:0/2565375513 shutdown_connections 2024-03-21T11:32:18.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.947+0000 7f1876600640 1 -- 172.21.15.47:0/2565375513 wait complete. 2024-03-21T11:32:18.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.948+0000 7f1876600640 1 Processor -- start 2024-03-21T11:32:18.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.948+0000 7f1876600640 1 -- start start 2024-03-21T11:32:18.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f1876600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1870102140 0x7f1870199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f1876600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1870104a70 0x7f187019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f1876600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18701073a0 0x7f18701a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f1876600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187010d340 con 0x7f1870102140 2024-03-21T11:32:18.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f1876600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187010d1c0 con 0x7f1870104a70 2024-03-21T11:32:18.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f1876600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187010d4c0 con 0x7f18701073a0 2024-03-21T11:32:18.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f186f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1870102140 0x7f1870199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f186fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18701073a0 0x7f18701a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f186fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18701073a0 0x7f18701a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52646/0 (socket says 172.21.15.47:52646) 2024-03-21T11:32:18.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f186fe00640 1 -- 172.21.15.47:0/3163542226 learned_addr learned my addr 172.21.15.47:0/3163542226 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:18.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f186ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1870104a70 0x7f187019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f186fe00640 1 -- 172.21.15.47:0/3163542226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1870104a70 msgr2=0x7f187019a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f186fe00640 1 --2- 172.21.15.47:0/3163542226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1870104a70 0x7f187019a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f186fe00640 1 -- 172.21.15.47:0/3163542226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1870102140 msgr2=0x7f1870199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f186fe00640 1 --2- 172.21.15.47:0/3163542226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1870102140 0x7f1870199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.949+0000 7f186fe00640 1 -- 172.21.15.47:0/3163542226 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18701a1810 con 0x7f18701073a0 2024-03-21T11:32:18.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.950+0000 7f186fe00640 1 --2- 172.21.15.47:0/3163542226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18701073a0 0x7f18701a11a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1868005cc0 tx=0x7f1868035d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:18.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.950+0000 7f1867e00640 1 -- 172.21.15.47:0/3163542226 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868039040 con 0x7f18701073a0 2024-03-21T11:32:18.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.950+0000 7f1867e00640 1 -- 172.21.15.47:0/3163542226 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1868044070 con 0x7f18701073a0 2024-03-21T11:32:18.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.950+0000 7f1867e00640 1 -- 172.21.15.47:0/3163542226 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868004400 con 0x7f18701073a0 2024-03-21T11:32:18.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.950+0000 7f1876600640 1 -- 172.21.15.47:0/3163542226 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18701a1aa0 con 0x7f18701073a0 2024-03-21T11:32:18.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.950+0000 7f1876600640 1 -- 172.21.15.47:0/3163542226 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18701a1f10 con 0x7f18701073a0 2024-03-21T11:32:18.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.954+0000 7f1879430740 1 Processor -- start 2024-03-21T11:32:18.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.954+0000 7f1867e00640 1 -- 172.21.15.47:0/3163542226 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f18680045a0 con 0x7f18701073a0 2024-03-21T11:32:18.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.954+0000 7f1879430740 1 -- start start 2024-03-21T11:32:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.954+0000 7f1879430740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563363022e80 0x5633630232a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.954+0000 7f1879430740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363023870 0x563363028370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.954+0000 7f1879430740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633630289e0 0x56336302ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.954+0000 7f1879430740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563362f88250 con 0x563363022e80 2024-03-21T11:32:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.954+0000 7f1879430740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563362f30a30 con 0x5633630289e0 2024-03-21T11:32:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.954+0000 7f1879430740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563362f842f0 con 0x563363023870 2024-03-21T11:32:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.955+0000 7f186ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563363022e80 0x5633630232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.955+0000 7f186f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363023870 0x563363028370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.955+0000 7f186f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363023870 0x563363028370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52660/0 (socket says 172.21.15.47:52660) 2024-03-21T11:32:18.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.955+0000 7f186f400640 1 -- 172.21.15.47:0/183820321 learned_addr learned my addr 172.21.15.47:0/183820321 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:18.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.955+0000 7f186ea00640 1 --2- 172.21.15.47:0/183820321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633630289e0 0x56336302ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.955+0000 7f186f400640 1 -- 172.21.15.47:0/183820321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633630289e0 msgr2=0x56336302ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.955+0000 7f186f400640 1 --2- 172.21.15.47:0/183820321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633630289e0 0x56336302ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.955+0000 7f186f400640 1 -- 172.21.15.47:0/183820321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563363022e80 msgr2=0x5633630232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.955+0000 7f186f400640 1 --2- 172.21.15.47:0/183820321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563363022e80 0x5633630232a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.955+0000 7f186f400640 1 -- 172.21.15.47:0/183820321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563362f1b690 con 0x563363023870 2024-03-21T11:32:18.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.956+0000 7f186f400640 1 --2- 172.21.15.47:0/183820321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363023870 0x563363028370 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1860009cf0 tx=0x7f1860011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e0a7aa28bcbcd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:18.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.956+0000 7f1867e00640 1 --2- 172.21.15.47:0/3163542226 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18540a2f20 0x7f18540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.956+0000 7f1864c00640 1 -- 172.21.15.47:0/183820321 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186001e070 con 0x563363023870 2024-03-21T11:32:18.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.956+0000 7f1864c00640 1 -- 172.21.15.47:0/183820321 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1860019b10 con 0x563363023870 2024-03-21T11:32:18.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.956+0000 7f1864c00640 1 -- 172.21.15.47:0/183820321 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186000e9f0 con 0x563363023870 2024-03-21T11:32:18.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.956+0000 7f1867e00640 1 -- 172.21.15.47:0/3163542226 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f18680f1da0 con 0x7f18701073a0 2024-03-21T11:32:18.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.956+0000 7f186ea00640 1 --2- 172.21.15.47:0/3163542226 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18540a2f20 0x7f18540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.957+0000 7f1879430740 1 -- 172.21.15.47:0/183820321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363023870 msgr2=0x563363028370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.957+0000 7f1879430740 1 --2- 172.21.15.47:0/183820321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363023870 0x563363028370 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1860009cf0 tx=0x7f1860011810 comp rx=0 tx=0).stop 2024-03-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.957+0000 7f1879430740 1 -- 172.21.15.47:0/183820321 shutdown_connections 2024-03-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.957+0000 7f1879430740 1 --2- 172.21.15.47:0/183820321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633630289e0 0x56336302ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.957+0000 7f1879430740 1 --2- 172.21.15.47:0/183820321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363023870 0x563363028370 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.957+0000 7f1879430740 1 --2- 172.21.15.47:0/183820321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563363022e80 0x5633630232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.957+0000 7f1879430740 1 -- 172.21.15.47:0/183820321 >> 172.21.15.47:0/183820321 conn(0x563362feb200 msgr2=0x56336301e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.957+0000 7f1879430740 1 -- 172.21.15.47:0/183820321 shutdown_connections 2024-03-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.957+0000 7f186ea00640 1 --2- 172.21.15.47:0/3163542226 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18540a2f20 0x7f18540a53e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x5633630240d0 tx=0x7f185c0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.957+0000 7f1879430740 1 -- 172.21.15.47:0/183820321 wait complete. 2024-03-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.957+0000 7f1879430740 1 -- start start 2024-03-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.958+0000 7f1879430740 1 Processor -- start 2024-03-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.958+0000 7f1879430740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363022e80 0x56336302ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.958+0000 7f1879430740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563363023870 0x56336302d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.958+0000 7f1879430740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633630289e0 0x5633630343d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.958+0000 7f1879430740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336302b990 con 0x563363023870 2024-03-21T11:32:18.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.958+0000 7f1879430740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563362f43e80 con 0x563363022e80 2024-03-21T11:32:18.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.958+0000 7f1879430740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336302bb10 con 0x5633630289e0 2024-03-21T11:32:18.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.958+0000 7f1879430740 10 client.0 _ll_register_callbacks cb 0x563362f56130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:18.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.958+0000 7f186f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363022e80 0x56336302ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.958+0000 7f186fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633630289e0 0x5633630343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.958+0000 7f186f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363022e80 0x56336302ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34450/0 (socket says 172.21.15.47:34450) 2024-03-21T11:32:18.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.958+0000 7f186f400640 1 -- 172.21.15.47:0/637211930 learned_addr learned my addr 172.21.15.47:0/637211930 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:18.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.958+0000 7f186ea00640 1 --2- 172.21.15.47:0/637211930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563363023870 0x56336302d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.959+0000 7f186f400640 1 -- 172.21.15.47:0/637211930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633630289e0 msgr2=0x5633630343d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.959+0000 7f186f400640 1 --2- 172.21.15.47:0/637211930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633630289e0 0x5633630343d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.959+0000 7f186f400640 1 -- 172.21.15.47:0/637211930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563363023870 msgr2=0x56336302d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.959+0000 7f186f400640 1 --2- 172.21.15.47:0/637211930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563363023870 0x56336302d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.959+0000 7f186f400640 1 -- 172.21.15.47:0/637211930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563362d667d0 con 0x563363022e80 2024-03-21T11:32:18.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.959+0000 7f186fe00640 1 --2- 172.21.15.47:0/637211930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633630289e0 0x5633630343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:18.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.959+0000 7f186f400640 1 --2- 172.21.15.47:0/637211930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363022e80 0x56336302ce50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f18600097b0 tx=0x7f186000e460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=414d4eb0ef9e5a19 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:18.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.959+0000 7f1859600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:32:18.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.959+0000 7f1859600640 1 -- 172.21.15.47:0/637211930 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1860027070 con 0x563363022e80 2024-03-21T11:32:18.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.959+0000 7f1859600640 1 -- 172.21.15.47:0/637211930 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1860019840 con 0x563363022e80 2024-03-21T11:32:18.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.959+0000 7f1879430740 1 -- 172.21.15.47:0/637211930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1860019b10 con 0x563363022e80 2024-03-21T11:32:18.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.960+0000 7f1859600640 1 -- 172.21.15.47:0/637211930 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1860033050 con 0x563363022e80 2024-03-21T11:32:18.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.960+0000 7f1879430740 10 client.28466 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:18.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.960+0000 7f1879430740 1 -- 172.21.15.47:0/637211930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563362c9c3f0 con 0x563363022e80 2024-03-21T11:32:18.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.960+0000 7f1859600640 1 -- 172.21.15.47:0/637211930 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1860019b10 con 0x563363022e80 2024-03-21T11:32:18.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.960+0000 7f1859600640 1 -- 172.21.15.47:0/637211930 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f18600046c0 con 0x563363022e80 2024-03-21T11:32:18.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.960+0000 7f1879430740 10 client.28466 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:18.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.960+0000 7f1879430740 10 client.28466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:18.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.960+0000 7f1879430740 20 client.28466 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:18.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.960+0000 7f1879430740 1 --2- 172.21.15.47:0/637211930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563363038930 0x56336303adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.961+0000 7f1879430740 4 client.28466 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:18.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.961+0000 7f1879430740 1 -- 172.21.15.47:0/637211930 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563362f95790 con 0x563363038930 2024-03-21T11:32:18.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.961+0000 7f186ea00640 1 --2- 172.21.15.47:0/637211930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563363038930 0x56336303adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.961+0000 7f186ea00640 1 --2- 172.21.15.47:0/637211930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563363038930 0x56336303adb0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac13a48a4ae2eb96 server_cookie=705fb63575e9c77a in_seq=0 out_seq=0 2024-03-21T11:32:18.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.961+0000 7f1859600640 10 client.28466 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:18.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1859600640 1 -- 172.21.15.47:0/637211930 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563362f95790 con 0x563363038930 2024-03-21T11:32:18.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1859600640 10 client.28466 handle_command_reply: tid=0 2024-03-21T11:32:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1879430740 1 client.28466 shutdown 2024-03-21T11:32:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1879430740 1 -- 172.21.15.47:0/637211930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363022e80 msgr2=0x56336302ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1879430740 1 --2- 172.21.15.47:0/637211930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363022e80 0x56336302ce50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f18600097b0 tx=0x7f186000e460 comp rx=0 tx=0).stop 2024-03-21T11:32:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1879430740 1 -- 172.21.15.47:0/637211930 shutdown_connections 2024-03-21T11:32:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1879430740 1 --2- 172.21.15.47:0/637211930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563363038930 0x56336303adb0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1879430740 1 --2- 172.21.15.47:0/637211930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633630289e0 0x5633630343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1879430740 1 --2- 172.21.15.47:0/637211930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563363023870 0x56336302d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1859600640 0 client.28466 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1879430740 1 --2- 172.21.15.47:0/637211930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363022e80 0x56336302ce50 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1879430740 1 -- 172.21.15.47:0/637211930 >> 172.21.15.47:0/637211930 conn(0x563362feb200 msgr2=0x56336301da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:18.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1879430740 1 -- 172.21.15.47:0/637211930 shutdown_connections 2024-03-21T11:32:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1879430740 1 -- 172.21.15.47:0/637211930 wait complete. 2024-03-21T11:32:18.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.963+0000 7f1879430740 20 client.28466 trim_cache size 0 max 16384 2024-03-21T11:32:18.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.983+0000 7f1879430740 1 Processor -- start 2024-03-21T11:32:18.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.983+0000 7f1879430740 1 -- start start 2024-03-21T11:32:18.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.983+0000 7f1879430740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363022e80 0x5633630242e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.983+0000 7f1879430740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363023870 0x5633630342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.983+0000 7f1879430740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633630289e0 0x56336303f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.983+0000 7f1879430740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18701ac340 con 0x5633630289e0 2024-03-21T11:32:18.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.983+0000 7f1879430740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18701adcc0 con 0x563363022e80 2024-03-21T11:32:18.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.983+0000 7f1879430740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18701ac9a0 con 0x563363023870 2024-03-21T11:32:18.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.984+0000 7f186f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363022e80 0x5633630242e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.984+0000 7f186fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633630289e0 0x56336303f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.984+0000 7f186ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363023870 0x5633630342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.984+0000 7f186f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363022e80 0x5633630242e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34452/0 (socket says 172.21.15.47:34452) 2024-03-21T11:32:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.984+0000 7f186f400640 1 -- 172.21.15.47:0/3578142565 learned_addr learned my addr 172.21.15.47:0/3578142565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.984+0000 7f186fe00640 1 -- 172.21.15.47:0/3578142565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363023870 msgr2=0x5633630342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.984+0000 7f186fe00640 1 --2- 172.21.15.47:0/3578142565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363023870 0x5633630342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.984+0000 7f186fe00640 1 -- 172.21.15.47:0/3578142565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363022e80 msgr2=0x5633630242e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.984+0000 7f186fe00640 1 --2- 172.21.15.47:0/3578142565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363022e80 0x5633630242e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.984+0000 7f186fe00640 1 -- 172.21.15.47:0/3578142565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563362c5c0f0 con 0x5633630289e0 2024-03-21T11:32:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.984+0000 7f186ea00640 1 --2- 172.21.15.47:0/3578142565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363023870 0x5633630342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.984+0000 7f186fe00640 1 --2- 172.21.15.47:0/3578142565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633630289e0 0x56336303f670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1868032bc0 tx=0x7f1868009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bde79fde6bea8d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.985+0000 7f185be00640 1 -- 172.21.15.47:0/3578142565 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868032890 con 0x5633630289e0 2024-03-21T11:32:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.985+0000 7f185be00640 1 -- 172.21.15.47:0/3578142565 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18680dce50 con 0x5633630289e0 2024-03-21T11:32:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.985+0000 7f185be00640 1 -- 172.21.15.47:0/3578142565 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18680de0e0 con 0x5633630289e0 2024-03-21T11:32:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.985+0000 7f1879430740 1 -- 172.21.15.47:0/3578142565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633630289e0 msgr2=0x56336303f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.985+0000 7f1879430740 1 --2- 172.21.15.47:0/3578142565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633630289e0 0x56336303f670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1868032bc0 tx=0x7f1868009a20 comp rx=0 tx=0).stop 2024-03-21T11:32:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.985+0000 7f1879430740 1 -- 172.21.15.47:0/3578142565 shutdown_connections 2024-03-21T11:32:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.985+0000 7f1879430740 1 --2- 172.21.15.47:0/3578142565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633630289e0 0x56336303f670 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.985+0000 7f1879430740 1 --2- 172.21.15.47:0/3578142565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363023870 0x5633630342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.985+0000 7f1879430740 1 --2- 172.21.15.47:0/3578142565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363022e80 0x5633630242e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.985+0000 7f1879430740 1 -- 172.21.15.47:0/3578142565 >> 172.21.15.47:0/3578142565 conn(0x563362feb200 msgr2=0x56336301da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.986+0000 7f1879430740 1 -- 172.21.15.47:0/3578142565 shutdown_connections 2024-03-21T11:32:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.986+0000 7f1879430740 1 -- 172.21.15.47:0/3578142565 wait complete. 2024-03-21T11:32:18.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.986+0000 7f1879430740 1 -- start start 2024-03-21T11:32:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.986+0000 7f1879430740 1 Processor -- start 2024-03-21T11:32:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.986+0000 7f1879430740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363022e80 0x563363031920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.986+0000 7f1879430740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363023870 0x563363031e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.986+0000 7f1879430740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633630289e0 0x563363032e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f1879430740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18701ad660 con 0x5633630289e0 2024-03-21T11:32:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f1879430740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18701ad000 con 0x563363023870 2024-03-21T11:32:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f1879430740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18701aa500 con 0x563363022e80 2024-03-21T11:32:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f1879430740 10 client.0 _ll_register_callbacks cb 0x7f186001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:18.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f186f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363022e80 0x563363031920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f186f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363022e80 0x563363031920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52678/0 (socket says 172.21.15.47:52678) 2024-03-21T11:32:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f186ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363023870 0x563363031e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f186f400640 1 -- 172.21.15.47:0/3477471295 learned_addr learned my addr 172.21.15.47:0/3477471295 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f186fe00640 1 --2- 172.21.15.47:0/3477471295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633630289e0 0x563363032e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f186f400640 1 -- 172.21.15.47:0/3477471295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363023870 msgr2=0x563363031e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f186f400640 1 --2- 172.21.15.47:0/3477471295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363023870 0x563363031e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f186f400640 1 -- 172.21.15.47:0/3477471295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633630289e0 msgr2=0x563363032e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f186f400640 1 --2- 172.21.15.47:0/3477471295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633630289e0 0x563363032e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f186f400640 1 -- 172.21.15.47:0/3477471295 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563363033460 con 0x563363022e80 2024-03-21T11:32:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f186ea00640 1 --2- 172.21.15.47:0/3477471295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363023870 0x563363031e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f186fe00640 1 --2- 172.21.15.47:0/3477471295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633630289e0 0x563363032e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f186f400640 1 --2- 172.21.15.47:0/3477471295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363022e80 0x563363031920 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f1860004ae0 tx=0x7f1860009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5beb9768a095e43d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.987+0000 7f1859600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:18.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.988+0000 7f1859600640 1 -- 172.21.15.47:0/3477471295 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1860027040 con 0x563363022e80 2024-03-21T11:32:18.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.988+0000 7f1879430740 1 -- 172.21.15.47:0/3477471295 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18680dce50 con 0x563363022e80 2024-03-21T11:32:18.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.988+0000 7f1859600640 1 -- 172.21.15.47:0/3477471295 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1860005410 con 0x563363022e80 2024-03-21T11:32:18.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.993+0000 7f1859600640 1 -- 172.21.15.47:0/3477471295 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1860040050 con 0x563363022e80 2024-03-21T11:32:18.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.993+0000 7f1879430740 10 client.28477 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:18.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.993+0000 7f1879430740 1 -- 172.21.15.47:0/3477471295 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563363034770 con 0x563363022e80 2024-03-21T11:32:18.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.993+0000 7f1859600640 1 -- 172.21.15.47:0/3477471295 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18680dce50 con 0x563363022e80 2024-03-21T11:32:18.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.993+0000 7f1859600640 1 -- 172.21.15.47:0/3477471295 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f18600035f0 con 0x563363022e80 2024-03-21T11:32:18.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.993+0000 7f1879430740 10 client.28477 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.993+0000 7f1879430740 10 client.28477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.993+0000 7f1879430740 20 client.28477 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.993+0000 7f1879430740 1 --2- 172.21.15.47:0/3477471295 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563363094930 0x563363096d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.993+0000 7f1879430740 4 client.28477 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:32:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.993+0000 7f1879430740 1 -- 172.21.15.47:0/3477471295 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563362f8fae0 con 0x563363094930 2024-03-21T11:32:18.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.994+0000 7f186ea00640 1 --2- 172.21.15.47:0/3477471295 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563363094930 0x563363096d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.994+0000 7f186ea00640 1 --2- 172.21.15.47:0/3477471295 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563363094930 0x563363096d50 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dee35d1c4661f4db server_cookie=a6801eb330c130b8 in_seq=0 out_seq=0 2024-03-21T11:32:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.994+0000 7f1859600640 10 client.28477 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.995+0000 7f1859600640 1 -- 172.21.15.47:0/3477471295 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563362f8fae0 con 0x563363094930 2024-03-21T11:32:18.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.995+0000 7f1859600640 10 client.28477 handle_command_reply: tid=0 2024-03-21T11:32:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.995+0000 7f1879430740 1 client.28477 shutdown 2024-03-21T11:32:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.995+0000 7f1879430740 1 -- 172.21.15.47:0/3477471295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363022e80 msgr2=0x563363031920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.995+0000 7f1879430740 1 --2- 172.21.15.47:0/3477471295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363022e80 0x563363031920 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f1860004ae0 tx=0x7f1860009c90 comp rx=0 tx=0).stop 2024-03-21T11:32:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.995+0000 7f1879430740 1 -- 172.21.15.47:0/3477471295 shutdown_connections 2024-03-21T11:32:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.995+0000 7f1879430740 1 --2- 172.21.15.47:0/3477471295 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563363094930 0x563363096d50 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.995+0000 7f1879430740 1 --2- 172.21.15.47:0/3477471295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633630289e0 0x563363032e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.995+0000 7f1879430740 1 --2- 172.21.15.47:0/3477471295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563363023870 0x563363031e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.995+0000 7f1879430740 1 --2- 172.21.15.47:0/3477471295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563363022e80 0x563363031920 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.995+0000 7f1859600640 0 client.28477 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.995+0000 7f1879430740 1 -- 172.21.15.47:0/3477471295 >> 172.21.15.47:0/3477471295 conn(0x563362feb200 msgr2=0x5633630222c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.995+0000 7f1879430740 1 -- 172.21.15.47:0/3477471295 shutdown_connections 2024-03-21T11:32:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.995+0000 7f1879430740 1 -- 172.21.15.47:0/3477471295 wait complete. 2024-03-21T11:32:18.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.996+0000 7f1879430740 20 client.28477 trim_cache size 0 max 16384 2024-03-21T11:32:18.998 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:32:18.998 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:32:18.998 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:32:18.998 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:32:18.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.996+0000 7f1876600640 1 -- 172.21.15.47:0/3163542226 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18540a2f20 msgr2=0x7f18540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.996+0000 7f1876600640 1 --2- 172.21.15.47:0/3163542226 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18540a2f20 0x7f18540a53e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x5633630240d0 tx=0x7f185c0046c0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.996+0000 7f1876600640 1 -- 172.21.15.47:0/3163542226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18701073a0 msgr2=0x7f18701a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:18.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.996+0000 7f1876600640 1 --2- 172.21.15.47:0/3163542226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18701073a0 0x7f18701a11a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1868005cc0 tx=0x7f1868035d90 comp rx=0 tx=0).stop 2024-03-21T11:32:18.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.996+0000 7f1876600640 1 -- 172.21.15.47:0/3163542226 shutdown_connections 2024-03-21T11:32:18.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.996+0000 7f1876600640 1 --2- 172.21.15.47:0/3163542226 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18540a2f20 0x7f18540a53e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.996+0000 7f1876600640 1 --2- 172.21.15.47:0/3163542226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18701073a0 0x7f18701a11a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.996+0000 7f1876600640 1 --2- 172.21.15.47:0/3163542226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1870104a70 0x7f187019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.996+0000 7f1876600640 1 --2- 172.21.15.47:0/3163542226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1870102140 0x7f1870199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:18.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.996+0000 7f1876600640 1 -- 172.21.15.47:0/3163542226 >> 172.21.15.47:0/3163542226 conn(0x7f18700fbb30 msgr2=0x7f18701017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:18.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.997+0000 7f1876600640 1 -- 172.21.15.47:0/3163542226 shutdown_connections 2024-03-21T11:32:19.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:18.997+0000 7f1876600640 1 -- 172.21.15.47:0/3163542226 wait complete. 2024-03-21T11:32:19.016 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:32:19.016 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:32:19.089 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:18 smithi047 ceph-mon[31351]: pgmap v669: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 2.0 MiB/s wr, 291 op/s 2024-03-21T11:32:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.235+0000 7fd810c00640 1 -- 172.21.15.47:0/1739157822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80c101510 msgr2=0x7fd80c101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.235+0000 7fd810c00640 1 --2- 172.21.15.47:0/1739157822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80c101510 0x7fd80c101930 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009900 tx=0x7fd7fc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:32:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.236+0000 7fd810c00640 1 -- 172.21.15.47:0/1739157822 shutdown_connections 2024-03-21T11:32:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.236+0000 7fd810c00640 1 --2- 172.21.15.47:0/1739157822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd80c1030c0 0x7fd80c109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.236+0000 7fd810c00640 1 --2- 172.21.15.47:0/1739157822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd80c102700 0x7fd80c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.236+0000 7fd810c00640 1 --2- 172.21.15.47:0/1739157822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80c101510 0x7fd80c101930 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.236+0000 7fd810c00640 1 -- 172.21.15.47:0/1739157822 >> 172.21.15.47:0/1739157822 conn(0x7fd80c0fca80 msgr2=0x7fd80c0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.236+0000 7fd810c00640 1 -- 172.21.15.47:0/1739157822 shutdown_connections 2024-03-21T11:32:19.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.236+0000 7fd810c00640 1 -- 172.21.15.47:0/1739157822 wait complete. 2024-03-21T11:32:19.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.237+0000 7fd810c00640 1 Processor -- start 2024-03-21T11:32:19.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.237+0000 7fd810c00640 1 -- start start 2024-03-21T11:32:19.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.237+0000 7fd810c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd80c101510 0x7fd80c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.237+0000 7fd810c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80c102700 0x7fd80c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.237+0000 7fd810c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd80c1030c0 0x7fd80c1a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.237+0000 7fd810c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c10d1f0 con 0x7fd80c101510 2024-03-21T11:32:19.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.237+0000 7fd810c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c10d070 con 0x7fd80c102700 2024-03-21T11:32:19.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.237+0000 7fd810c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c10d370 con 0x7fd80c1030c0 2024-03-21T11:32:19.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.238+0000 7fd802a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80c102700 0x7fd80c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.238+0000 7fd80be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd80c1030c0 0x7fd80c1a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.238+0000 7fd802a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80c102700 0x7fd80c199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34488/0 (socket says 172.21.15.47:34488) 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.238+0000 7fd80b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd80c101510 0x7fd80c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.238+0000 7fd802a00640 1 -- 172.21.15.47:0/556620423 learned_addr learned my addr 172.21.15.47:0/556620423 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.238+0000 7fd80be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd80c1030c0 0x7fd80c1a1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52688/0 (socket says 172.21.15.47:52688) 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.238+0000 7fd802a00640 1 -- 172.21.15.47:0/556620423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd80c1030c0 msgr2=0x7fd80c1a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.238+0000 7fd802a00640 1 --2- 172.21.15.47:0/556620423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd80c1030c0 0x7fd80c1a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.238+0000 7fd802a00640 1 -- 172.21.15.47:0/556620423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd80c101510 msgr2=0x7fd80c199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.238+0000 7fd802a00640 1 --2- 172.21.15.47:0/556620423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd80c101510 0x7fd80c199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.238+0000 7fd802a00640 1 -- 172.21.15.47:0/556620423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c1a16a0 con 0x7fd80c102700 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.238+0000 7fd802a00640 1 --2- 172.21.15.47:0/556620423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80c102700 0x7fd80c199fc0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400e9f0 tx=0x7fd7f400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.239+0000 7fd808c00640 1 -- 172.21.15.47:0/556620423 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f400cd80 con 0x7fd80c102700 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.239+0000 7fd808c00640 1 -- 172.21.15.47:0/556620423 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7f4005500 con 0x7fd80c102700 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.239+0000 7fd808c00640 1 -- 172.21.15.47:0/556620423 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4010690 con 0x7fd80c102700 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.239+0000 7fd810c00640 1 -- 172.21.15.47:0/556620423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80c1a1990 con 0x7fd80c102700 2024-03-21T11:32:19.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.239+0000 7fd810c00640 1 -- 172.21.15.47:0/556620423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80c1a1ee0 con 0x7fd80c102700 2024-03-21T11:32:19.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:18 smithi102 ceph-mon[36748]: pgmap v669: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 2.0 MiB/s wr, 291 op/s 2024-03-21T11:32:19.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.240+0000 7fd810c00640 1 -- 172.21.15.47:0/556620423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80c1a2170 con 0x7fd80c102700 2024-03-21T11:32:19.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.243+0000 7fd808c00640 1 -- 172.21.15.47:0/556620423 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd7f40046c0 con 0x7fd80c102700 2024-03-21T11:32:19.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.244+0000 7fd808c00640 1 --2- 172.21.15.47:0/556620423 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7f00a2fa0 0x7fd7f00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.244+0000 7fd808c00640 1 -- 172.21.15.47:0/556620423 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd7f40cfb40 con 0x7fd80c102700 2024-03-21T11:32:19.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.244+0000 7fd80b400640 1 --2- 172.21.15.47:0/556620423 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7f00a2fa0 0x7fd7f00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.245+0000 7fd80b400640 1 --2- 172.21.15.47:0/556620423 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7f00a2fa0 0x7fd7f00a5460 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc02c170 tx=0x7fd7fc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:19.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.250+0000 7fd808c00640 1 -- 172.21.15.47:0/556620423 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd7f409c580 con 0x7fd80c102700 2024-03-21T11:32:19.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:18 smithi100 ceph-mon[36245]: pgmap v669: 129 pgs: 129 active+clean; 42 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 2.0 MiB/s wr, 291 op/s 2024-03-21T11:32:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.509+0000 7fd810c00640 1 -- 172.21.15.47:0/556620423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd80c1a2340 con 0x7fd80c102700 2024-03-21T11:32:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.510+0000 7fd808c00640 1 -- 172.21.15.47:0/556620423 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fd7f409be80 con 0x7fd80c102700 2024-03-21T11:32:19.511 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:32:19.511 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:32:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:32:19.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.513+0000 7fd810c00640 1 -- 172.21.15.47:0/556620423 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7f00a2fa0 msgr2=0x7fd7f00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:19.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.513+0000 7fd810c00640 1 --2- 172.21.15.47:0/556620423 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7f00a2fa0 0x7fd7f00a5460 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc02c170 tx=0x7fd7fc02df90 comp rx=0 tx=0).stop 2024-03-21T11:32:19.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.513+0000 7fd810c00640 1 -- 172.21.15.47:0/556620423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80c102700 msgr2=0x7fd80c199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:19.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.513+0000 7fd810c00640 1 --2- 172.21.15.47:0/556620423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80c102700 0x7fd80c199fc0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400e9f0 tx=0x7fd7f400edf0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.513+0000 7fd810c00640 1 -- 172.21.15.47:0/556620423 shutdown_connections 2024-03-21T11:32:19.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.513+0000 7fd810c00640 1 --2- 172.21.15.47:0/556620423 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7f00a2fa0 0x7fd7f00a5460 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.513+0000 7fd810c00640 1 --2- 172.21.15.47:0/556620423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd80c1030c0 0x7fd80c1a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.513+0000 7fd810c00640 1 --2- 172.21.15.47:0/556620423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80c102700 0x7fd80c199fc0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.513+0000 7fd810c00640 1 --2- 172.21.15.47:0/556620423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd80c101510 0x7fd80c199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.513+0000 7fd810c00640 1 -- 172.21.15.47:0/556620423 >> 172.21.15.47:0/556620423 conn(0x7fd80c0fca80 msgr2=0x7fd80c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:19.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.513+0000 7fd810c00640 1 -- 172.21.15.47:0/556620423 shutdown_connections 2024-03-21T11:32:19.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.513+0000 7fd810c00640 1 -- 172.21.15.47:0/556620423 wait complete. 2024-03-21T11:32:19.533 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:32:19.533 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:32:19.542 INFO:tasks.workunit.client.0.smithi047.stdout: 1 652785 8.19 MB/sec execute 303 sec latency 103.233 ms 2024-03-21T11:32:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.765+0000 7f506ee00640 1 -- 172.21.15.47:0/814382646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5068105350 msgr2=0x7f506810bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.765+0000 7f506ee00640 1 --2- 172.21.15.47:0/814382646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5068105350 0x7f506810bbe0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5060009ca0 tx=0x7f506002c100 comp rx=0 tx=0).stop 2024-03-21T11:32:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.765+0000 7f506ee00640 1 -- 172.21.15.47:0/814382646 shutdown_connections 2024-03-21T11:32:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.765+0000 7f506ee00640 1 --2- 172.21.15.47:0/814382646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5068105350 0x7f506810bbe0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.766+0000 7f506ee00640 1 --2- 172.21.15.47:0/814382646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068104990 0x7f5068104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.766+0000 7f506ee00640 1 --2- 172.21.15.47:0/814382646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50681037a0 0x7f5068103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.766+0000 7f506ee00640 1 -- 172.21.15.47:0/814382646 >> 172.21.15.47:0/814382646 conn(0x7f50680fed10 msgr2=0x7f5068101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.766+0000 7f506ee00640 1 -- 172.21.15.47:0/814382646 shutdown_connections 2024-03-21T11:32:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.766+0000 7f506ee00640 1 -- 172.21.15.47:0/814382646 wait complete. 2024-03-21T11:32:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.767+0000 7f506ee00640 1 Processor -- start 2024-03-21T11:32:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.767+0000 7f506ee00640 1 -- start start 2024-03-21T11:32:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.767+0000 7f506ee00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50681037a0 0x7f5068199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.767+0000 7f506ee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068104990 0x7f5068199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.767+0000 7f506ee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5068105350 0x7f50681a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.767+0000 7f506ee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506810f420 con 0x7f5068105350 2024-03-21T11:32:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.767+0000 7f506ee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506810f2a0 con 0x7f5068104990 2024-03-21T11:32:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.767+0000 7f506ee00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506810f5a0 con 0x7f50681037a0 2024-03-21T11:32:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.767+0000 7f506da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50681037a0 0x7f5068199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.767+0000 7f506da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50681037a0 0x7f5068199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52708/0 (socket says 172.21.15.47:52708) 2024-03-21T11:32:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.767+0000 7f506da00640 1 -- 172.21.15.47:0/235499602 learned_addr learned my addr 172.21.15.47:0/235499602 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.767+0000 7f506d000640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068104990 0x7f5068199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.768+0000 7f506e400640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5068105350 0x7f50681a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.768+0000 7f506da00640 1 -- 172.21.15.47:0/235499602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068104990 msgr2=0x7f5068199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.768+0000 7f506da00640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068104990 0x7f5068199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.768+0000 7f506da00640 1 -- 172.21.15.47:0/235499602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5068105350 msgr2=0x7f50681a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.768+0000 7f506da00640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5068105350 0x7f50681a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.768+0000 7f506da00640 1 -- 172.21.15.47:0/235499602 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50681a1620 con 0x7f50681037a0 2024-03-21T11:32:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.768+0000 7f506e400640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5068105350 0x7f50681a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.768+0000 7f506da00640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50681037a0 0x7f5068199a00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f505c00e9c0 tx=0x7f505c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.768+0000 7f5066000640 1 -- 172.21.15.47:0/235499602 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c00cde0 con 0x7f50681037a0 2024-03-21T11:32:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.768+0000 7f5066000640 1 -- 172.21.15.47:0/235499602 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f505c010910 con 0x7f50681037a0 2024-03-21T11:32:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.768+0000 7f5066000640 1 -- 172.21.15.47:0/235499602 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c018a00 con 0x7f50681037a0 2024-03-21T11:32:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.768+0000 7f506ee00640 1 -- 172.21.15.47:0/235499602 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50681a1910 con 0x7f50681037a0 2024-03-21T11:32:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.769+0000 7f506ee00640 1 -- 172.21.15.47:0/235499602 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50681a1e10 con 0x7f50681037a0 2024-03-21T11:32:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.773+0000 7f507306c740 1 Processor -- start 2024-03-21T11:32:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.773+0000 7f507306c740 1 -- start start 2024-03-21T11:32:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.773+0000 7f507306c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 0x55c632817620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.773+0000 7f507306c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632818850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.773+0000 7f507306c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 0x55c63281f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.773+0000 7f507306c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6327732c0 con 0x55c632817200 2024-03-21T11:32:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.773+0000 7f507306c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63271bc90 con 0x55c632818d90 2024-03-21T11:32:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.773+0000 7f507306c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63276f360 con 0x55c6328183d0 2024-03-21T11:32:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.774+0000 7f5066000640 1 -- 172.21.15.47:0/235499602 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f505c01f020 con 0x7f50681037a0 2024-03-21T11:32:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.774+0000 7f506e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 0x55c632817620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.774+0000 7f506e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 0x55c632817620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37782/0 (socket says 172.21.15.47:37782) 2024-03-21T11:32:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.774+0000 7f506e400640 1 -- 172.21.15.47:0/497258156 learned_addr learned my addr 172.21.15.47:0/497258156 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.774+0000 7f506d000640 1 --2- 172.21.15.47:0/497258156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 0x55c63281f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.774+0000 7f506da00640 1 --2- 172.21.15.47:0/497258156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632818850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.774+0000 7f506e400640 1 -- 172.21.15.47:0/497258156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 msgr2=0x55c632818850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.774+0000 7f506e400640 1 --2- 172.21.15.47:0/497258156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632818850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.774+0000 7f506e400640 1 -- 172.21.15.47:0/497258156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 msgr2=0x55c63281f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.774+0000 7f506e400640 1 --2- 172.21.15.47:0/497258156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 0x55c63281f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.774+0000 7f506e400640 1 -- 172.21.15.47:0/497258156 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c632740080 con 0x55c632817200 2024-03-21T11:32:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.774+0000 7f506d000640 1 --2- 172.21.15.47:0/497258156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 0x55c63281f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.774+0000 7f506e400640 1 --2- 172.21.15.47:0/497258156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 0x55c632817620 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5060009ca0 tx=0x7f5060009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdffdb62d94dd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.775+0000 7f505aa00640 1 -- 172.21.15.47:0/497258156 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060043020 con 0x55c632817200 2024-03-21T11:32:19.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.775+0000 7f505aa00640 1 -- 172.21.15.47:0/497258156 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5060003970 con 0x55c632817200 2024-03-21T11:32:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.775+0000 7f505aa00640 1 -- 172.21.15.47:0/497258156 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060039040 con 0x55c632817200 2024-03-21T11:32:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.775+0000 7f5066000640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50500a2f20 0x7f50500a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.775+0000 7f5066000640 1 -- 172.21.15.47:0/235499602 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f505c0cfa60 con 0x7f50681037a0 2024-03-21T11:32:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.775+0000 7f507306c740 1 -- 172.21.15.47:0/497258156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 msgr2=0x55c632817620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.775+0000 7f507306c740 1 --2- 172.21.15.47:0/497258156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 0x55c632817620 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5060009ca0 tx=0x7f5060009800 comp rx=0 tx=0).stop 2024-03-21T11:32:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.775+0000 7f506da00640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50500a2f20 0x7f50500a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.776+0000 7f507306c740 1 -- 172.21.15.47:0/497258156 shutdown_connections 2024-03-21T11:32:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.776+0000 7f507306c740 1 --2- 172.21.15.47:0/497258156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 0x55c63281f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.776+0000 7f507306c740 1 --2- 172.21.15.47:0/497258156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632818850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.776+0000 7f507306c740 1 --2- 172.21.15.47:0/497258156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 0x55c632817620 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.776+0000 7f507306c740 1 -- 172.21.15.47:0/497258156 >> 172.21.15.47:0/497258156 conn(0x55c632809de0 msgr2=0x55c63280a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.776+0000 7f507306c740 1 -- 172.21.15.47:0/497258156 shutdown_connections 2024-03-21T11:32:19.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.776+0000 7f507306c740 1 -- 172.21.15.47:0/497258156 wait complete. 2024-03-21T11:32:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.776+0000 7f506da00640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50500a2f20 0x7f50500a53e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f505c0cd170 tx=0x7f505c00efb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.776+0000 7f507306c740 1 -- start start 2024-03-21T11:32:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.777+0000 7f507306c740 1 Processor -- start 2024-03-21T11:32:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.777+0000 7f507306c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 0x55c632821410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.777+0000 7f507306c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632821950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.777+0000 7f507306c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 0x55c632828990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.777+0000 7f507306c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63281fd40 con 0x55c632817200 2024-03-21T11:32:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.777+0000 7f507306c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63272f0d0 con 0x55c632818d90 2024-03-21T11:32:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.777+0000 7f507306c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63281fec0 con 0x55c6328183d0 2024-03-21T11:32:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.777+0000 7f507306c740 10 client.0 _ll_register_callbacks cb 0x55c632733e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.777+0000 7f506d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632821950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.777+0000 7f506d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632821950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52728/0 (socket says 172.21.15.47:52728) 2024-03-21T11:32:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.777+0000 7f506d000640 1 -- 172.21.15.47:0/3580451066 learned_addr learned my addr 172.21.15.47:0/3580451066 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.777+0000 7f506e400640 1 --2- 172.21.15.47:0/3580451066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 0x55c632821410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.777+0000 7f506e400640 1 --2- 172.21.15.47:0/3580451066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 0x55c632828990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.778+0000 7f506d000640 1 -- 172.21.15.47:0/3580451066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 msgr2=0x55c632828990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.778+0000 7f506d000640 1 --2- 172.21.15.47:0/3580451066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 0x55c632828990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.778+0000 7f506d000640 1 -- 172.21.15.47:0/3580451066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 msgr2=0x55c632821410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:19.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.778+0000 7f506d000640 1 --2- 172.21.15.47:0/3580451066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 0x55c632821410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.778+0000 7f506d000640 1 -- 172.21.15.47:0/3580451066 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6324873f0 con 0x55c6328183d0 2024-03-21T11:32:19.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.778+0000 7f506d000640 1 --2- 172.21.15.47:0/3580451066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632821950 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5054012700 tx=0x7f5054012b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5385008833b95d3a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:19.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.778+0000 7f504f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:19.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.778+0000 7f504f400640 1 -- 172.21.15.47:0/3580451066 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054004050 con 0x55c6328183d0 2024-03-21T11:32:19.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.778+0000 7f507306c740 1 -- 172.21.15.47:0/3580451066 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5060003970 con 0x55c6328183d0 2024-03-21T11:32:19.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.778+0000 7f504f400640 1 -- 172.21.15.47:0/3580451066 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f50540041f0 con 0x55c6328183d0 2024-03-21T11:32:19.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.778+0000 7f504f400640 1 -- 172.21.15.47:0/3580451066 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5054025050 con 0x55c6328183d0 2024-03-21T11:32:19.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.778+0000 7f507306c740 10 client.28495 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:19.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.778+0000 7f507306c740 1 -- 172.21.15.47:0/3580451066 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c6325517d0 con 0x55c6328183d0 2024-03-21T11:32:19.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.779+0000 7f504f400640 1 -- 172.21.15.47:0/3580451066 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5060003970 con 0x55c6328183d0 2024-03-21T11:32:19.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.779+0000 7f504f400640 1 -- 172.21.15.47:0/3580451066 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5054003800 con 0x55c6328183d0 2024-03-21T11:32:19.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.779+0000 7f507306c740 10 client.28495 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:19.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.779+0000 7f507306c740 10 client.28495 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:32:19.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.779+0000 7f507306c740 20 client.28495 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:19.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.779+0000 7f507306c740 1 --2- 172.21.15.47:0/3580451066 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c63282ce70 0x55c63282f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.779+0000 7f507306c740 4 client.28495 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:19.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.779+0000 7f507306c740 1 -- 172.21.15.47:0/3580451066 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c632780800 con 0x55c63282ce70 2024-03-21T11:32:19.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.779+0000 7f506e400640 1 --2- 172.21.15.47:0/3580451066 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c63282ce70 0x55c63282f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.780+0000 7f506e400640 1 --2- 172.21.15.47:0/3580451066 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c63282ce70 0x55c63282f2b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ab7ad2c16b5df5d server_cookie=7f697844cd09dde2 in_seq=0 out_seq=0 2024-03-21T11:32:19.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.780+0000 7f504f400640 10 client.28495 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:32:19.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f504f400640 1 -- 172.21.15.47:0/3580451066 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c632780800 con 0x55c63282ce70 2024-03-21T11:32:19.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f504f400640 10 client.28495 handle_command_reply: tid=0 2024-03-21T11:32:19.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f507306c740 1 client.28495 shutdown 2024-03-21T11:32:19.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f507306c740 1 -- 172.21.15.47:0/3580451066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 msgr2=0x55c632821950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:19.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f507306c740 1 --2- 172.21.15.47:0/3580451066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632821950 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5054012700 tx=0x7f5054012b00 comp rx=0 tx=0).stop 2024-03-21T11:32:19.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f507306c740 1 -- 172.21.15.47:0/3580451066 shutdown_connections 2024-03-21T11:32:19.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f507306c740 1 --2- 172.21.15.47:0/3580451066 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c63282ce70 0x55c63282f2b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f507306c740 1 --2- 172.21.15.47:0/3580451066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 0x55c632828990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f507306c740 1 --2- 172.21.15.47:0/3580451066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632821950 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f507306c740 1 --2- 172.21.15.47:0/3580451066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 0x55c632821410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f507306c740 1 -- 172.21.15.47:0/3580451066 >> 172.21.15.47:0/3580451066 conn(0x55c632809de0 msgr2=0x55c632809160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:19.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f504f400640 0 client.28495 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:32:19.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f507306c740 1 -- 172.21.15.47:0/3580451066 shutdown_connections 2024-03-21T11:32:19.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f507306c740 1 -- 172.21.15.47:0/3580451066 wait complete. 2024-03-21T11:32:19.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.781+0000 7f507306c740 20 client.28495 trim_cache size 0 max 16384 2024-03-21T11:32:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.795+0000 7f507306c740 1 Processor -- start 2024-03-21T11:32:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.795+0000 7f507306c740 1 -- start start 2024-03-21T11:32:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.795+0000 7f507306c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632817200 0x55c63281b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.795+0000 7f507306c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6328183d0 0x55c632828950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.795+0000 7f507306c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c632818d90 0x55c63281bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.795+0000 7f507306c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50681a7c90 con 0x55c6328183d0 2024-03-21T11:32:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.795+0000 7f507306c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50681a9610 con 0x55c632817200 2024-03-21T11:32:19.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.795+0000 7f507306c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50681a82f0 con 0x55c632818d90 2024-03-21T11:32:19.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.796+0000 7f506e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632817200 0x55c63281b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.796+0000 7f506e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632817200 0x55c63281b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34526/0 (socket says 172.21.15.47:34526) 2024-03-21T11:32:19.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.796+0000 7f506e400640 1 -- 172.21.15.47:0/1987867518 learned_addr learned my addr 172.21.15.47:0/1987867518 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:19.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.796+0000 7f506e400640 1 --2- 172.21.15.47:0/1987867518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c632818d90 0x55c63281bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.796+0000 7f506d000640 1 --2- 172.21.15.47:0/1987867518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6328183d0 0x55c632828950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.796+0000 7f506e400640 1 -- 172.21.15.47:0/1987867518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c632818d90 msgr2=0x55c63281bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.796+0000 7f506e400640 1 --2- 172.21.15.47:0/1987867518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c632818d90 0x55c63281bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.796+0000 7f506e400640 1 -- 172.21.15.47:0/1987867518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6328183d0 msgr2=0x55c632828950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.796+0000 7f506e400640 1 --2- 172.21.15.47:0/1987867518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6328183d0 0x55c632828950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.796+0000 7f506e400640 1 -- 172.21.15.47:0/1987867518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c63280ad50 con 0x55c632817200 2024-03-21T11:32:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.796+0000 7f506e400640 1 --2- 172.21.15.47:0/1987867518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632817200 0x55c63281b600 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5060033000 tx=0x7f50600099f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1650730f1e2247e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.797+0000 7f505a000640 1 -- 172.21.15.47:0/1987867518 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060039040 con 0x55c632817200 2024-03-21T11:32:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.797+0000 7f505a000640 1 -- 172.21.15.47:0/1987867518 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5060003e50 con 0x55c632817200 2024-03-21T11:32:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.797+0000 7f505a000640 1 -- 172.21.15.47:0/1987867518 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060043040 con 0x55c632817200 2024-03-21T11:32:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.797+0000 7f507306c740 1 -- 172.21.15.47:0/1987867518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632817200 msgr2=0x55c63281b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.797+0000 7f507306c740 1 --2- 172.21.15.47:0/1987867518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632817200 0x55c63281b600 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5060033000 tx=0x7f50600099f0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.797+0000 7f507306c740 1 -- 172.21.15.47:0/1987867518 shutdown_connections 2024-03-21T11:32:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.797+0000 7f507306c740 1 --2- 172.21.15.47:0/1987867518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c632818d90 0x55c63281bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.797+0000 7f507306c740 1 --2- 172.21.15.47:0/1987867518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6328183d0 0x55c632828950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.797+0000 7f507306c740 1 --2- 172.21.15.47:0/1987867518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632817200 0x55c63281b600 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.797+0000 7f507306c740 1 -- 172.21.15.47:0/1987867518 >> 172.21.15.47:0/1987867518 conn(0x55c632809de0 msgr2=0x55c63280d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:19.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.797+0000 7f507306c740 1 -- 172.21.15.47:0/1987867518 shutdown_connections 2024-03-21T11:32:19.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.797+0000 7f507306c740 1 -- 172.21.15.47:0/1987867518 wait complete. 2024-03-21T11:32:19.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.798+0000 7f507306c740 1 -- start start 2024-03-21T11:32:19.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.798+0000 7f507306c740 1 Processor -- start 2024-03-21T11:32:19.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.798+0000 7f507306c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 0x55c6328283c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f507306c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632828900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f507306c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 0x55c632826aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f507306c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50681a8fb0 con 0x55c632817200 2024-03-21T11:32:19.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f507306c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50681a8950 con 0x55c632818d90 2024-03-21T11:32:19.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f507306c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50681a5e70 con 0x55c6328183d0 2024-03-21T11:32:19.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f507306c740 10 client.0 _ll_register_callbacks cb 0x7f5054009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:19.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f506d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632828900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f506d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632828900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52742/0 (socket says 172.21.15.47:52742) 2024-03-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f506d000640 1 -- 172.21.15.47:0/4192021906 learned_addr learned my addr 172.21.15.47:0/4192021906 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f506e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 0x55c6328283c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f506e400640 1 --2- 172.21.15.47:0/4192021906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 0x55c632826aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f506d000640 1 -- 172.21.15.47:0/4192021906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 msgr2=0x55c632826aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f506d000640 1 --2- 172.21.15.47:0/4192021906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 0x55c632826aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f506d000640 1 -- 172.21.15.47:0/4192021906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 msgr2=0x55c6328283c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f506d000640 1 --2- 172.21.15.47:0/4192021906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 0x55c6328283c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f506d000640 1 -- 172.21.15.47:0/4192021906 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6328272d0 con 0x55c6328183d0 2024-03-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.799+0000 7f506e400640 1 --2- 172.21.15.47:0/4192021906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 0x55c632826aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.800+0000 7f506d000640 1 --2- 172.21.15.47:0/4192021906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632828900 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5054012420 tx=0x7f505400f8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1c1b724e2b90f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.800+0000 7f504f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.800+0000 7f504f400640 1 -- 172.21.15.47:0/4192021906 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505400fde0 con 0x55c6328183d0 2024-03-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.800+0000 7f507306c740 1 -- 172.21.15.47:0/4192021906 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5060003e50 con 0x55c6328183d0 2024-03-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.800+0000 7f504f400640 1 -- 172.21.15.47:0/4192021906 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5054018040 con 0x55c6328183d0 2024-03-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.800+0000 7f504f400640 1 -- 172.21.15.47:0/4192021906 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5054037050 con 0x55c6328183d0 2024-03-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.800+0000 7f507306c740 10 client.28501 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.800+0000 7f507306c740 1 -- 172.21.15.47:0/4192021906 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c6328893c0 con 0x55c6328183d0 2024-03-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.800+0000 7f504f400640 1 -- 172.21.15.47:0/4192021906 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5060003e50 con 0x55c6328183d0 2024-03-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.801+0000 7f504f400640 1 -- 172.21.15.47:0/4192021906 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5054029430 con 0x55c6328183d0 2024-03-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.801+0000 7f507306c740 10 client.28501 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.801+0000 7f507306c740 10 client.28501 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.801+0000 7f507306c740 20 client.28501 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.801+0000 7f507306c740 1 --2- 172.21.15.47:0/4192021906 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c632889d40 0x55c63288c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.801+0000 7f507306c740 4 client.28501 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.801+0000 7f507306c740 1 -- 172.21.15.47:0/4192021906 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c63277ab50 con 0x55c632889d40 2024-03-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.801+0000 7f506e400640 1 --2- 172.21.15.47:0/4192021906 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c632889d40 0x55c63288c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.801+0000 7f506e400640 1 --2- 172.21.15.47:0/4192021906 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c632889d40 0x55c63288c160 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a14e22fb9c84e48 server_cookie=c8996da2ff77785f in_seq=0 out_seq=0 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.801+0000 7f504f400640 10 client.28501 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f504f400640 1 -- 172.21.15.47:0/4192021906 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c63277ab50 con 0x55c632889d40 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f504f400640 10 client.28501 handle_command_reply: tid=0 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f507306c740 1 client.28501 shutdown 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f507306c740 1 -- 172.21.15.47:0/4192021906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 msgr2=0x55c632828900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f507306c740 1 --2- 172.21.15.47:0/4192021906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632828900 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5054012420 tx=0x7f505400f8c0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f507306c740 1 -- 172.21.15.47:0/4192021906 shutdown_connections 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f507306c740 1 --2- 172.21.15.47:0/4192021906 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c632889d40 0x55c63288c160 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f507306c740 1 --2- 172.21.15.47:0/4192021906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c632818d90 0x55c632826aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f507306c740 1 --2- 172.21.15.47:0/4192021906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6328183d0 0x55c632828900 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f507306c740 1 --2- 172.21.15.47:0/4192021906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c632817200 0x55c6328283c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f504f400640 0 client.28501 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f507306c740 1 -- 172.21.15.47:0/4192021906 >> 172.21.15.47:0/4192021906 conn(0x55c632809de0 msgr2=0x55c6328088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f507306c740 1 -- 172.21.15.47:0/4192021906 shutdown_connections 2024-03-21T11:32:19.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f507306c740 1 -- 172.21.15.47:0/4192021906 wait complete. 2024-03-21T11:32:19.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.802+0000 7f507306c740 20 client.28501 trim_cache size 0 max 16384 2024-03-21T11:32:19.812 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:32:19.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.803+0000 7f506ee00640 1 -- 172.21.15.47:0/235499602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50500a2f20 msgr2=0x7f50500a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:19.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.803+0000 7f506ee00640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50500a2f20 0x7f50500a53e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f505c0cd170 tx=0x7f505c00efb0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.803+0000 7f506ee00640 1 -- 172.21.15.47:0/235499602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50681037a0 msgr2=0x7f5068199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:19.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.803+0000 7f506ee00640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50681037a0 0x7f5068199a00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f505c00e9c0 tx=0x7f505c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:32:19.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.803+0000 7f506ee00640 1 -- 172.21.15.47:0/235499602 shutdown_connections 2024-03-21T11:32:19.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.803+0000 7f506ee00640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50500a2f20 0x7f50500a53e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.803+0000 7f506ee00640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5068105350 0x7f50681a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.803+0000 7f506ee00640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5068104990 0x7f5068199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.803+0000 7f506ee00640 1 --2- 172.21.15.47:0/235499602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50681037a0 0x7f5068199a00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:19.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.803+0000 7f506ee00640 1 -- 172.21.15.47:0/235499602 >> 172.21.15.47:0/235499602 conn(0x7f50680fed10 msgr2=0x7f5068107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:19.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.803+0000 7f506ee00640 1 -- 172.21.15.47:0/235499602 shutdown_connections 2024-03-21T11:32:19.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:19.803+0000 7f506ee00640 1 -- 172.21.15.47:0/235499602 wait complete. 2024-03-21T11:32:19.824 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:32:19.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:32:20.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.096+0000 7feda7200640 1 -- 172.21.15.47:0/1222591731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feda80772a0 msgr2=0x7feda8077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.096+0000 7feda7200640 1 --2- 172.21.15.47:0/1222591731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feda80772a0 0x7feda8077700 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fed94009900 tx=0x7fed9402bc50 comp rx=0 tx=0).stop 2024-03-21T11:32:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.096+0000 7feda7200640 1 -- 172.21.15.47:0/1222591731 shutdown_connections 2024-03-21T11:32:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.096+0000 7feda7200640 1 --2- 172.21.15.47:0/1222591731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda8077c40 0x7feda80784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.096+0000 7feda7200640 1 --2- 172.21.15.47:0/1222591731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feda80772a0 0x7feda8077700 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.096+0000 7feda7200640 1 --2- 172.21.15.47:0/1222591731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feda8076060 0x7feda8076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.096+0000 7feda7200640 1 -- 172.21.15.47:0/1222591731 >> 172.21.15.47:0/1222591731 conn(0x7feda80fed10 msgr2=0x7feda8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.097+0000 7feda7200640 1 -- 172.21.15.47:0/1222591731 shutdown_connections 2024-03-21T11:32:20.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.097+0000 7feda7200640 1 -- 172.21.15.47:0/1222591731 wait complete. 2024-03-21T11:32:20.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.097+0000 7feda7200640 1 Processor -- start 2024-03-21T11:32:20.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.098+0000 7feda7200640 1 -- start start 2024-03-21T11:32:20.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.098+0000 7feda7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feda8076060 0x7feda819df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.098+0000 7feda7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda80772a0 0x7feda819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.098+0000 7feda7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feda8077c40 0x7feda81a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.098+0000 7feda7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda8113830 con 0x7feda80772a0 2024-03-21T11:32:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.098+0000 7feda7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda81136b0 con 0x7feda8076060 2024-03-21T11:32:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.098+0000 7feda7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda81139b0 con 0x7feda8077c40 2024-03-21T11:32:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.098+0000 7feda5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feda8076060 0x7feda819df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.098+0000 7feda5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda80772a0 0x7feda819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.098+0000 7feda6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feda8077c40 0x7feda81a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.098+0000 7feda6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feda8077c40 0x7feda81a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52760/0 (socket says 172.21.15.47:52760) 2024-03-21T11:32:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.098+0000 7feda6800640 1 -- 172.21.15.47:0/2655930222 learned_addr learned my addr 172.21.15.47:0/2655930222 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.098+0000 7feda5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda80772a0 0x7feda819e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37824/0 (socket says 172.21.15.47:37824) 2024-03-21T11:32:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.098+0000 7feda5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feda8076060 0x7feda819df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34556/0 (socket says 172.21.15.47:34556) 2024-03-21T11:32:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.099+0000 7feda6800640 1 -- 172.21.15.47:0/2655930222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feda8076060 msgr2=0x7feda819df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.099+0000 7feda6800640 1 --2- 172.21.15.47:0/2655930222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feda8076060 0x7feda819df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.099+0000 7feda6800640 1 -- 172.21.15.47:0/2655930222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda80772a0 msgr2=0x7feda819e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.099+0000 7feda6800640 1 --2- 172.21.15.47:0/2655930222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda80772a0 0x7feda819e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.099+0000 7feda6800640 1 -- 172.21.15.47:0/2655930222 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda81a5b40 con 0x7feda8077c40 2024-03-21T11:32:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.099+0000 7feda5e00640 1 --2- 172.21.15.47:0/2655930222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feda8076060 0x7feda819df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.099+0000 7feda6800640 1 --2- 172.21.15.47:0/2655930222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feda8077c40 0x7feda81a54d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7feda000eaa0 tx=0x7feda000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.099+0000 7fed9ea00640 1 -- 172.21.15.47:0/2655930222 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda000cdb0 con 0x7feda8077c40 2024-03-21T11:32:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.099+0000 7fed9ea00640 1 -- 172.21.15.47:0/2655930222 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feda0005020 con 0x7feda8077c40 2024-03-21T11:32:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.099+0000 7feda7200640 1 -- 172.21.15.47:0/2655930222 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda81a5e30 con 0x7feda8077c40 2024-03-21T11:32:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.099+0000 7fed9ea00640 1 -- 172.21.15.47:0/2655930222 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda00116b0 con 0x7feda8077c40 2024-03-21T11:32:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.099+0000 7feda7200640 1 -- 172.21.15.47:0/2655930222 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda81a62f0 con 0x7feda8077c40 2024-03-21T11:32:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.102+0000 7fed9ea00640 1 -- 172.21.15.47:0/2655930222 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7feda0005560 con 0x7feda8077c40 2024-03-21T11:32:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.103+0000 7fedafde2740 1 Processor -- start 2024-03-21T11:32:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.103+0000 7fedafde2740 1 -- start start 2024-03-21T11:32:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.103+0000 7fedafde2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581198609f0 0x558119860e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.103+0000 7fedafde2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558119861bc0 0x558119862040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.103+0000 7fedafde2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119862580 0x558119868e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.103+0000 7fedafde2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581197c52c0 con 0x558119861bc0 2024-03-21T11:32:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.103+0000 7fedafde2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811976dc90 con 0x5581198609f0 2024-03-21T11:32:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.103+0000 7fedafde2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581197c1360 con 0x558119862580 2024-03-21T11:32:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.103+0000 7feda5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581198609f0 0x558119860e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.103+0000 7feda5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558119861bc0 0x558119862040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.104+0000 7feda5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581198609f0 0x558119860e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34570/0 (socket says 172.21.15.47:34570) 2024-03-21T11:32:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.104+0000 7feda5400640 1 -- 172.21.15.47:0/442053576 learned_addr learned my addr 172.21.15.47:0/442053576 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.104+0000 7feda5400640 1 --2- 172.21.15.47:0/442053576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119862580 0x558119868e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.104+0000 7feda5e00640 1 -- 172.21.15.47:0/442053576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119862580 msgr2=0x558119868e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.104+0000 7feda5e00640 1 --2- 172.21.15.47:0/442053576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119862580 0x558119868e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.104+0000 7feda5e00640 1 -- 172.21.15.47:0/442053576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581198609f0 msgr2=0x558119860e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.104+0000 7feda5e00640 1 --2- 172.21.15.47:0/442053576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581198609f0 0x558119860e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.104+0000 7feda5e00640 1 -- 172.21.15.47:0/442053576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558119792080 con 0x558119861bc0 2024-03-21T11:32:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.104+0000 7fed9ea00640 1 --2- 172.21.15.47:0/2655930222 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed8c0a2fa0 0x7fed8c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.104+0000 7fed9ea00640 1 -- 172.21.15.47:0/2655930222 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7feda00cf900 con 0x7feda8077c40 2024-03-21T11:32:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.104+0000 7feda5e00640 1 --2- 172.21.15.47:0/442053576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558119861bc0 0x558119862040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fed980103d0 tx=0x7fed98010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c54cb3994b6e0c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.105+0000 7feda5400640 1 --2- 172.21.15.47:0/2655930222 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed8c0a2fa0 0x7fed8c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.105+0000 7fed93400640 1 -- 172.21.15.47:0/442053576 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98011700 con 0x558119861bc0 2024-03-21T11:32:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.105+0000 7fed93400640 1 -- 172.21.15.47:0/442053576 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed98011d80 con 0x558119861bc0 2024-03-21T11:32:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.105+0000 7fed93400640 1 -- 172.21.15.47:0/442053576 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9801a380 con 0x558119861bc0 2024-03-21T11:32:20.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.105+0000 7fedafde2740 1 -- 172.21.15.47:0/442053576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558119861bc0 msgr2=0x558119862040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.105+0000 7fedafde2740 1 --2- 172.21.15.47:0/442053576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558119861bc0 0x558119862040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fed980103d0 tx=0x7fed98010890 comp rx=0 tx=0).stop 2024-03-21T11:32:20.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.105+0000 7feda5400640 1 --2- 172.21.15.47:0/2655930222 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed8c0a2fa0 0x7fed8c0a5460 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fed94005f70 tx=0x7fed94005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.106+0000 7fedafde2740 1 -- 172.21.15.47:0/442053576 shutdown_connections 2024-03-21T11:32:20.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.106+0000 7fedafde2740 1 --2- 172.21.15.47:0/442053576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119862580 0x558119868e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.106+0000 7fedafde2740 1 --2- 172.21.15.47:0/442053576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558119861bc0 0x558119862040 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.106+0000 7fedafde2740 1 --2- 172.21.15.47:0/442053576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581198609f0 0x558119860e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.106+0000 7fedafde2740 1 -- 172.21.15.47:0/442053576 >> 172.21.15.47:0/442053576 conn(0x55811985bde0 msgr2=0x55811985c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.106+0000 7fedafde2740 1 -- 172.21.15.47:0/442053576 shutdown_connections 2024-03-21T11:32:20.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.106+0000 7fedafde2740 1 -- 172.21.15.47:0/442053576 wait complete. 2024-03-21T11:32:20.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.106+0000 7fedafde2740 1 -- start start 2024-03-21T11:32:20.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.106+0000 7fedafde2740 1 Processor -- start 2024-03-21T11:32:20.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7fedafde2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581198609f0 0x55811986ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7fedafde2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x55811986b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7fedafde2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558119862580 0x5581198720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7fedafde2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558119869590 con 0x5581198609f0 2024-03-21T11:32:20.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7fedafde2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581197810d0 con 0x558119862580 2024-03-21T11:32:20.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7fedafde2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558119869710 con 0x558119861bc0 2024-03-21T11:32:20.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7fedafde2740 10 client.0 _ll_register_callbacks cb 0x558119785e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7feda5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x55811986b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7feda6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558119862580 0x5581198720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7feda5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581198609f0 0x55811986ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7feda5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x55811986b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52784/0 (socket says 172.21.15.47:52784) 2024-03-21T11:32:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7feda5400640 1 -- 172.21.15.47:0/2073054635 learned_addr learned my addr 172.21.15.47:0/2073054635 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7feda6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558119862580 0x5581198720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34576/0 (socket says 172.21.15.47:34576) 2024-03-21T11:32:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7feda5400640 1 -- 172.21.15.47:0/2073054635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558119862580 msgr2=0x5581198720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7feda5400640 1 --2- 172.21.15.47:0/2073054635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558119862580 0x5581198720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7feda5400640 1 -- 172.21.15.47:0/2073054635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581198609f0 msgr2=0x55811986ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.107+0000 7feda5400640 1 --2- 172.21.15.47:0/2073054635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581198609f0 0x55811986ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.108+0000 7feda5400640 1 -- 172.21.15.47:0/2073054635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581194d93f0 con 0x558119861bc0 2024-03-21T11:32:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.108+0000 7feda6800640 1 --2- 172.21.15.47:0/2073054635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558119862580 0x5581198720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.108+0000 7feda5400640 1 --2- 172.21.15.47:0/2073054635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x55811986b0b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fed9402ccf0 tx=0x7fed9402bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1baba116d30bb3af server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.108+0000 7fed87e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.108+0000 7fed87e00640 1 -- 172.21.15.47:0/2073054635 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94040890 con 0x558119861bc0 2024-03-21T11:32:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.108+0000 7fed87e00640 1 -- 172.21.15.47:0/2073054635 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fed94040a30 con 0x558119861bc0 2024-03-21T11:32:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.108+0000 7fedafde2740 1 -- 172.21.15.47:0/2073054635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed98011d80 con 0x558119861bc0 2024-03-21T11:32:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.108+0000 7fed87e00640 1 -- 172.21.15.47:0/2073054635 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed9405b050 con 0x558119861bc0 2024-03-21T11:32:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.108+0000 7fedafde2740 10 client.28513 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.108+0000 7fedafde2740 1 -- 172.21.15.47:0/2073054635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5581195a37d0 con 0x558119861bc0 2024-03-21T11:32:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.109+0000 7fed87e00640 1 -- 172.21.15.47:0/2073054635 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed98011d80 con 0x558119861bc0 2024-03-21T11:32:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.109+0000 7fed87e00640 1 -- 172.21.15.47:0/2073054635 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fed94043c00 con 0x558119861bc0 2024-03-21T11:32:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.109+0000 7fedafde2740 10 client.28513 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.109+0000 7fedafde2740 10 client.28513 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:32:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.109+0000 7fedafde2740 20 client.28513 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.109+0000 7fedafde2740 1 --2- 172.21.15.47:0/2073054635 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558119876640 0x558119878ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.109+0000 7fedafde2740 4 client.28513 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.109+0000 7fedafde2740 1 -- 172.21.15.47:0/2073054635 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581197d2800 con 0x558119876640 2024-03-21T11:32:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.109+0000 7feda5e00640 1 --2- 172.21.15.47:0/2073054635 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558119876640 0x558119878ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.110+0000 7feda5e00640 1 --2- 172.21.15.47:0/2073054635 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558119876640 0x558119878ae0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3be70676091700da server_cookie=1d1fed38419edb5c in_seq=0 out_seq=0 2024-03-21T11:32:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.110+0000 7fed87e00640 10 client.28513 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:32:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fed87e00640 1 -- 172.21.15.47:0/2073054635 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5581197d2800 con 0x558119876640 2024-03-21T11:32:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fed87e00640 10 client.28513 handle_command_reply: tid=0 2024-03-21T11:32:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fedafde2740 1 client.28513 shutdown 2024-03-21T11:32:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fedafde2740 1 -- 172.21.15.47:0/2073054635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 msgr2=0x55811986b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fedafde2740 1 --2- 172.21.15.47:0/2073054635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x55811986b0b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fed9402ccf0 tx=0x7fed9402bc50 comp rx=0 tx=0).stop 2024-03-21T11:32:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fedafde2740 1 -- 172.21.15.47:0/2073054635 shutdown_connections 2024-03-21T11:32:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fedafde2740 1 --2- 172.21.15.47:0/2073054635 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x558119876640 0x558119878ae0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fedafde2740 1 --2- 172.21.15.47:0/2073054635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558119862580 0x5581198720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fedafde2740 1 --2- 172.21.15.47:0/2073054635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x55811986b0b0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fedafde2740 1 --2- 172.21.15.47:0/2073054635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581198609f0 0x55811986ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fed87e00640 0 client.28513 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:32:20.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fedafde2740 1 -- 172.21.15.47:0/2073054635 >> 172.21.15.47:0/2073054635 conn(0x55811985bde0 msgr2=0x55811985b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fedafde2740 1 -- 172.21.15.47:0/2073054635 shutdown_connections 2024-03-21T11:32:20.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fedafde2740 1 -- 172.21.15.47:0/2073054635 wait complete. 2024-03-21T11:32:20.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.111+0000 7fedafde2740 20 client.28513 trim_cache size 0 max 16384 2024-03-21T11:32:20.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.125+0000 7fedafde2740 1 Processor -- start 2024-03-21T11:32:20.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.125+0000 7fedafde2740 1 -- start start 2024-03-21T11:32:20.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.125+0000 7fedafde2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581198609f0 0x55811985afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.125+0000 7fedafde2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x5581198720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.125+0000 7fedafde2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558119862580 0x55811987d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.125+0000 7fedafde2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda81b0550 con 0x558119862580 2024-03-21T11:32:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.125+0000 7fedafde2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda81b1ed0 con 0x5581198609f0 2024-03-21T11:32:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.125+0000 7fedafde2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda81b0bb0 con 0x558119861bc0 2024-03-21T11:32:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.125+0000 7feda5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581198609f0 0x55811985afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.125+0000 7feda5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x5581198720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.125+0000 7feda5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581198609f0 0x55811985afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34590/0 (socket says 172.21.15.47:34590) 2024-03-21T11:32:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.125+0000 7feda5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x5581198720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52800/0 (socket says 172.21.15.47:52800) 2024-03-21T11:32:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.125+0000 7feda5e00640 1 -- 172.21.15.47:0/2218357486 learned_addr learned my addr 172.21.15.47:0/2218357486 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.125+0000 7feda6800640 1 --2- 172.21.15.47:0/2218357486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558119862580 0x55811987d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.126+0000 7feda5e00640 1 -- 172.21.15.47:0/2218357486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 msgr2=0x5581198720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.126+0000 7feda5e00640 1 --2- 172.21.15.47:0/2218357486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x5581198720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.126+0000 7feda5e00640 1 -- 172.21.15.47:0/2218357486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558119862580 msgr2=0x55811987d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.126+0000 7feda5e00640 1 --2- 172.21.15.47:0/2218357486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558119862580 0x55811987d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.126+0000 7feda5e00640 1 -- 172.21.15.47:0/2218357486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55811985cb50 con 0x5581198609f0 2024-03-21T11:32:20.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.126+0000 7feda5400640 1 --2- 172.21.15.47:0/2218357486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x5581198720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:20.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.126+0000 7feda5e00640 1 --2- 172.21.15.47:0/2218357486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581198609f0 0x55811985afe0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x55811985cd50 tx=0x7fed9801c690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fe62197bbb742c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.126+0000 7fed92a00640 1 -- 172.21.15.47:0/2218357486 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9802a2d0 con 0x5581198609f0 2024-03-21T11:32:20.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.126+0000 7fed92a00640 1 -- 172.21.15.47:0/2218357486 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed98005520 con 0x5581198609f0 2024-03-21T11:32:20.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.126+0000 7fed92a00640 1 -- 172.21.15.47:0/2218357486 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9802a9e0 con 0x5581198609f0 2024-03-21T11:32:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.127+0000 7fedafde2740 1 -- 172.21.15.47:0/2218357486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581198609f0 msgr2=0x55811985afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.127+0000 7fedafde2740 1 --2- 172.21.15.47:0/2218357486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581198609f0 0x55811985afe0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x55811985cd50 tx=0x7fed9801c690 comp rx=0 tx=0).stop 2024-03-21T11:32:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.127+0000 7fedafde2740 1 -- 172.21.15.47:0/2218357486 shutdown_connections 2024-03-21T11:32:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.127+0000 7fedafde2740 1 --2- 172.21.15.47:0/2218357486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558119862580 0x55811987d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.127+0000 7fedafde2740 1 --2- 172.21.15.47:0/2218357486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x5581198720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.127+0000 7fedafde2740 1 --2- 172.21.15.47:0/2218357486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581198609f0 0x55811985afe0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.127+0000 7fedafde2740 1 -- 172.21.15.47:0/2218357486 >> 172.21.15.47:0/2218357486 conn(0x55811985bde0 msgr2=0x55811985a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.127+0000 7fedafde2740 1 -- 172.21.15.47:0/2218357486 shutdown_connections 2024-03-21T11:32:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.127+0000 7fedafde2740 1 -- 172.21.15.47:0/2218357486 wait complete. 2024-03-21T11:32:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.127+0000 7fedafde2740 1 -- start start 2024-03-21T11:32:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.128+0000 7fedafde2740 1 Processor -- start 2024-03-21T11:32:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.128+0000 7fedafde2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581198609f0 0x558119871ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.128+0000 7fedafde2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x558119870340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.128+0000 7fedafde2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558119862580 0x558119870880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.128+0000 7fedafde2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda81b1870 con 0x5581198609f0 2024-03-21T11:32:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.128+0000 7fedafde2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda81b1210 con 0x558119862580 2024-03-21T11:32:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.128+0000 7fedafde2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda81ae710 con 0x558119861bc0 2024-03-21T11:32:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.128+0000 7fedafde2740 10 client.0 _ll_register_callbacks cb 0x7fed94004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.128+0000 7feda6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558119862580 0x558119870880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.128+0000 7feda5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581198609f0 0x558119871ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.128+0000 7feda5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x558119870340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.128+0000 7feda6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558119862580 0x558119870880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34598/0 (socket says 172.21.15.47:34598) 2024-03-21T11:32:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.128+0000 7feda6800640 1 -- 172.21.15.47:0/2040288860 learned_addr learned my addr 172.21.15.47:0/2040288860 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.129+0000 7feda5400640 1 -- 172.21.15.47:0/2040288860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558119862580 msgr2=0x558119870880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.129+0000 7feda5400640 1 --2- 172.21.15.47:0/2040288860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558119862580 0x558119870880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.129+0000 7feda5400640 1 -- 172.21.15.47:0/2040288860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581198609f0 msgr2=0x558119871ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.129+0000 7feda5400640 1 --2- 172.21.15.47:0/2040288860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581198609f0 0x558119871ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.129+0000 7feda5400640 1 -- 172.21.15.47:0/2040288860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558119870df0 con 0x558119861bc0 2024-03-21T11:32:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.129+0000 7feda6800640 1 --2- 172.21.15.47:0/2040288860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558119862580 0x558119870880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.129+0000 7feda5e00640 1 --2- 172.21.15.47:0/2040288860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581198609f0 0x558119871ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.129+0000 7feda5400640 1 --2- 172.21.15.47:0/2040288860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x558119870340 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fed94009900 tx=0x7fed94034b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6198dbeaeeba3da3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.129+0000 7fed87e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.129+0000 7fed87e00640 1 -- 172.21.15.47:0/2040288860 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94066020 con 0x558119861bc0 2024-03-21T11:32:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.129+0000 7fedafde2740 1 -- 172.21.15.47:0/2040288860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed98005520 con 0x558119861bc0 2024-03-21T11:32:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.129+0000 7fed87e00640 1 -- 172.21.15.47:0/2040288860 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fed94040890 con 0x558119861bc0 2024-03-21T11:32:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.130+0000 7fed87e00640 1 -- 172.21.15.47:0/2040288860 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed94071050 con 0x558119861bc0 2024-03-21T11:32:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.130+0000 7fedafde2740 10 client.28519 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.130+0000 7fedafde2740 1 -- 172.21.15.47:0/2040288860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5581198d2cf0 con 0x558119861bc0 2024-03-21T11:32:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.130+0000 7fed87e00640 1 -- 172.21.15.47:0/2040288860 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed98005520 con 0x558119861bc0 2024-03-21T11:32:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.130+0000 7fed87e00640 1 -- 172.21.15.47:0/2040288860 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fed94043450 con 0x558119861bc0 2024-03-21T11:32:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.130+0000 7fedafde2740 10 client.28519 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.130+0000 7fedafde2740 10 client.28519 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:32:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.130+0000 7fedafde2740 20 client.28519 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.130+0000 7fedafde2740 1 --2- 172.21.15.47:0/2040288860 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5581198d3670 0x5581198d5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.130+0000 7fedafde2740 4 client.28519 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:32:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.130+0000 7fedafde2740 1 -- 172.21.15.47:0/2040288860 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5581197ccb50 con 0x5581198d3670 2024-03-21T11:32:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.130+0000 7feda5e00640 1 --2- 172.21.15.47:0/2040288860 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5581198d3670 0x5581198d5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.131+0000 7feda5e00640 1 --2- 172.21.15.47:0/2040288860 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5581198d3670 0x5581198d5a90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5cc1ac136b0b1de1 server_cookie=2d331e34393c08b1 in_seq=0 out_seq=0 2024-03-21T11:32:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.131+0000 7fed87e00640 10 client.28519 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:32:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.131+0000 7fed87e00640 1 -- 172.21.15.47:0/2040288860 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5581197ccb50 con 0x5581198d3670 2024-03-21T11:32:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.131+0000 7fed87e00640 10 client.28519 handle_command_reply: tid=0 2024-03-21T11:32:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.131+0000 7fedafde2740 1 client.28519 shutdown 2024-03-21T11:32:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.131+0000 7fedafde2740 1 -- 172.21.15.47:0/2040288860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 msgr2=0x558119870340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.131+0000 7fedafde2740 1 --2- 172.21.15.47:0/2040288860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x558119870340 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fed94009900 tx=0x7fed94034b40 comp rx=0 tx=0).stop 2024-03-21T11:32:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.131+0000 7fedafde2740 1 -- 172.21.15.47:0/2040288860 shutdown_connections 2024-03-21T11:32:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.131+0000 7fedafde2740 1 --2- 172.21.15.47:0/2040288860 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5581198d3670 0x5581198d5a90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.131+0000 7fedafde2740 1 --2- 172.21.15.47:0/2040288860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558119862580 0x558119870880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.131+0000 7fed87e00640 0 client.28519 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:32:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.131+0000 7fedafde2740 1 --2- 172.21.15.47:0/2040288860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119861bc0 0x558119870340 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.131+0000 7fedafde2740 1 --2- 172.21.15.47:0/2040288860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581198609f0 0x558119871ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.131+0000 7fedafde2740 1 -- 172.21.15.47:0/2040288860 >> 172.21.15.47:0/2040288860 conn(0x55811985bde0 msgr2=0x558119865680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.132+0000 7fedafde2740 1 -- 172.21.15.47:0/2040288860 shutdown_connections 2024-03-21T11:32:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.132+0000 7fedafde2740 1 -- 172.21.15.47:0/2040288860 wait complete. 2024-03-21T11:32:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.132+0000 7fedafde2740 20 client.28519 trim_cache size 0 max 16384 2024-03-21T11:32:20.187 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:32:20.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.132+0000 7feda7200640 1 -- 172.21.15.47:0/2655930222 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed8c0a2fa0 msgr2=0x7fed8c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.132+0000 7feda7200640 1 --2- 172.21.15.47:0/2655930222 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed8c0a2fa0 0x7fed8c0a5460 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fed94005f70 tx=0x7fed94005fd0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.132+0000 7feda7200640 1 -- 172.21.15.47:0/2655930222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feda8077c40 msgr2=0x7feda81a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.132+0000 7feda7200640 1 --2- 172.21.15.47:0/2655930222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feda8077c40 0x7feda81a54d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7feda000eaa0 tx=0x7feda000ef60 comp rx=0 tx=0).stop 2024-03-21T11:32:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.132+0000 7feda7200640 1 -- 172.21.15.47:0/2655930222 shutdown_connections 2024-03-21T11:32:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.132+0000 7feda7200640 1 --2- 172.21.15.47:0/2655930222 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fed8c0a2fa0 0x7fed8c0a5460 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.132+0000 7feda7200640 1 --2- 172.21.15.47:0/2655930222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feda8077c40 0x7feda81a54d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.132+0000 7feda7200640 1 --2- 172.21.15.47:0/2655930222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda80772a0 0x7feda819e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.132+0000 7feda7200640 1 --2- 172.21.15.47:0/2655930222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feda8076060 0x7feda819df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.132+0000 7feda7200640 1 -- 172.21.15.47:0/2655930222 >> 172.21.15.47:0/2655930222 conn(0x7feda80fed10 msgr2=0x7feda8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.133+0000 7feda7200640 1 -- 172.21.15.47:0/2655930222 shutdown_connections 2024-03-21T11:32:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.133+0000 7feda7200640 1 -- 172.21.15.47:0/2655930222 wait complete. 2024-03-21T11:32:20.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:19 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/556620423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:32:20.191 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:32:20.191 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:32:20.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:19 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/556620423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:32:20.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.423+0000 7f250f200640 1 -- 172.21.15.47:0/1217797573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25101037a0 msgr2=0x7f2510103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.423+0000 7f250f200640 1 --2- 172.21.15.47:0/1217797573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25101037a0 0x7f2510103bc0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2500009900 tx=0x7f250002bc30 comp rx=0 tx=0).stop 2024-03-21T11:32:20.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.423+0000 7f250f200640 1 -- 172.21.15.47:0/1217797573 shutdown_connections 2024-03-21T11:32:20.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.423+0000 7f250f200640 1 --2- 172.21.15.47:0/1217797573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2510105350 0x7f251010bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.423+0000 7f250f200640 1 --2- 172.21.15.47:0/1217797573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2510104990 0x7f2510104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.423+0000 7f250f200640 1 --2- 172.21.15.47:0/1217797573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25101037a0 0x7f2510103bc0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.423+0000 7f250f200640 1 -- 172.21.15.47:0/1217797573 >> 172.21.15.47:0/1217797573 conn(0x7f25100fed10 msgr2=0x7f2510101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.424+0000 7f250f200640 1 -- 172.21.15.47:0/1217797573 shutdown_connections 2024-03-21T11:32:20.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.424+0000 7f250f200640 1 -- 172.21.15.47:0/1217797573 wait complete. 2024-03-21T11:32:20.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.425+0000 7f250f200640 1 Processor -- start 2024-03-21T11:32:20.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.425+0000 7f250f200640 1 -- start start 2024-03-21T11:32:20.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.425+0000 7f250f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25101037a0 0x7f2510199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.425+0000 7f250f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2510104990 0x7f251019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.425+0000 7f250f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2510105350 0x7f25101a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.425+0000 7f250f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251010f460 con 0x7f25101037a0 2024-03-21T11:32:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.425+0000 7f250f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251010f2e0 con 0x7f2510105350 2024-03-21T11:32:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.425+0000 7f250f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251010f5e0 con 0x7f2510104990 2024-03-21T11:32:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.425+0000 7f250de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25101037a0 0x7f2510199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.425+0000 7f250d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2510104990 0x7f251019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.425+0000 7f250d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2510104990 0x7f251019a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52834/0 (socket says 172.21.15.47:52834) 2024-03-21T11:32:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.425+0000 7f250d400640 1 -- 172.21.15.47:0/1696711864 learned_addr learned my addr 172.21.15.47:0/1696711864 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.426+0000 7f250e800640 1 --2- 172.21.15.47:0/1696711864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2510105350 0x7f25101a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.426+0000 7f250d400640 1 -- 172.21.15.47:0/1696711864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2510105350 msgr2=0x7f25101a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.426+0000 7f250d400640 1 --2- 172.21.15.47:0/1696711864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2510105350 0x7f25101a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.426+0000 7f250d400640 1 -- 172.21.15.47:0/1696711864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25101037a0 msgr2=0x7f2510199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.426+0000 7f250d400640 1 --2- 172.21.15.47:0/1696711864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25101037a0 0x7f2510199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.426+0000 7f250d400640 1 -- 172.21.15.47:0/1696711864 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25101a1720 con 0x7f2510104990 2024-03-21T11:32:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.426+0000 7f250e800640 1 --2- 172.21.15.47:0/1696711864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2510105350 0x7f25101a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.426+0000 7f250d400640 1 --2- 172.21.15.47:0/1696711864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2510104990 0x7f251019a040 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f24f800e9f0 tx=0x7f24f800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.426+0000 7f2506a00640 1 -- 172.21.15.47:0/1696711864 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f800cd80 con 0x7f2510104990 2024-03-21T11:32:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.426+0000 7f2506a00640 1 -- 172.21.15.47:0/1696711864 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24f8005500 con 0x7f2510104990 2024-03-21T11:32:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.426+0000 7f2506a00640 1 -- 172.21.15.47:0/1696711864 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f8010690 con 0x7f2510104990 2024-03-21T11:32:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.426+0000 7f250f200640 1 -- 172.21.15.47:0/1696711864 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25101a1a10 con 0x7f2510104990 2024-03-21T11:32:20.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.427+0000 7f250f200640 1 -- 172.21.15.47:0/1696711864 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25101a1f10 con 0x7f2510104990 2024-03-21T11:32:20.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.430+0000 7f2517bd6740 1 Processor -- start 2024-03-21T11:32:20.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.430+0000 7f2517bd6740 1 -- start start 2024-03-21T11:32:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.430+0000 7f2517bd6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e179f0 0x561cb6e17e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.430+0000 7f2517bd6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e18bc0 0x561cb6e19040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.430+0000 7f2517bd6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e19580 0x561cb6e1fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.430+0000 7f2506a00640 1 -- 172.21.15.47:0/1696711864 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f24f80046c0 con 0x7f2510104990 2024-03-21T11:32:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.430+0000 7f2517bd6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb6d7c2c0 con 0x561cb6e179f0 2024-03-21T11:32:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.430+0000 7f2517bd6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb6d24c90 con 0x561cb6e18bc0 2024-03-21T11:32:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.430+0000 7f2517bd6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb6d78360 con 0x561cb6e19580 2024-03-21T11:32:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.431+0000 7f250e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e179f0 0x561cb6e17e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.431+0000 7f250e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e179f0 0x561cb6e17e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37878/0 (socket says 172.21.15.47:37878) 2024-03-21T11:32:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.431+0000 7f250e800640 1 -- 172.21.15.47:0/3895682721 learned_addr learned my addr 172.21.15.47:0/3895682721 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.431+0000 7f250de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e18bc0 0x561cb6e19040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.431+0000 7f250d400640 1 --2- 172.21.15.47:0/3895682721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e19580 0x561cb6e1fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.431+0000 7f250de00640 1 -- 172.21.15.47:0/3895682721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e19580 msgr2=0x561cb6e1fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.431+0000 7f250de00640 1 --2- 172.21.15.47:0/3895682721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e19580 0x561cb6e1fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.431+0000 7f250de00640 1 -- 172.21.15.47:0/3895682721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e179f0 msgr2=0x561cb6e17e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.431+0000 7f250de00640 1 --2- 172.21.15.47:0/3895682721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e179f0 0x561cb6e17e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.431+0000 7f250de00640 1 -- 172.21.15.47:0/3895682721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cb6d49080 con 0x561cb6e18bc0 2024-03-21T11:32:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.431+0000 7f250e800640 1 --2- 172.21.15.47:0/3895682721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e179f0 0x561cb6e17e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.432+0000 7f250de00640 1 --2- 172.21.15.47:0/3895682721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e18bc0 0x561cb6e19040 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2500009230 tx=0x7f2500009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc81f6dbfee7c4bd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.432+0000 7f24ff400640 1 -- 172.21.15.47:0/3895682721 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250002c980 con 0x561cb6e18bc0 2024-03-21T11:32:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.432+0000 7f24ff400640 1 -- 172.21.15.47:0/3895682721 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f250002cb20 con 0x561cb6e18bc0 2024-03-21T11:32:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.432+0000 7f24ff400640 1 -- 172.21.15.47:0/3895682721 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500004920 con 0x561cb6e18bc0 2024-03-21T11:32:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.432+0000 7f2506a00640 1 --2- 172.21.15.47:0/1696711864 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f24f00a2fa0 0x7f24f00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.433+0000 7f2506a00640 1 -- 172.21.15.47:0/1696711864 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f24f80cfb90 con 0x7f2510104990 2024-03-21T11:32:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.433+0000 7f2517bd6740 1 -- 172.21.15.47:0/3895682721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e18bc0 msgr2=0x561cb6e19040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.433+0000 7f2517bd6740 1 --2- 172.21.15.47:0/3895682721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e18bc0 0x561cb6e19040 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2500009230 tx=0x7f2500009560 comp rx=0 tx=0).stop 2024-03-21T11:32:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.433+0000 7f250e800640 1 --2- 172.21.15.47:0/1696711864 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f24f00a2fa0 0x7f24f00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.433+0000 7f2517bd6740 1 -- 172.21.15.47:0/3895682721 shutdown_connections 2024-03-21T11:32:20.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.433+0000 7f2517bd6740 1 --2- 172.21.15.47:0/3895682721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e19580 0x561cb6e1fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.433+0000 7f2517bd6740 1 --2- 172.21.15.47:0/3895682721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e18bc0 0x561cb6e19040 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.433+0000 7f2517bd6740 1 --2- 172.21.15.47:0/3895682721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e179f0 0x561cb6e17e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.433+0000 7f2517bd6740 1 -- 172.21.15.47:0/3895682721 >> 172.21.15.47:0/3895682721 conn(0x561cb6e12de0 msgr2=0x561cb6e131e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.433+0000 7f2517bd6740 1 -- 172.21.15.47:0/3895682721 shutdown_connections 2024-03-21T11:32:20.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.434+0000 7f2517bd6740 1 -- 172.21.15.47:0/3895682721 wait complete. 2024-03-21T11:32:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.434+0000 7f250e800640 1 --2- 172.21.15.47:0/1696711864 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f24f00a2fa0 0x7f24f00a5460 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2508009fd0 tx=0x7f2508009940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.434+0000 7f2517bd6740 1 -- start start 2024-03-21T11:32:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.434+0000 7f2517bd6740 1 Processor -- start 2024-03-21T11:32:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f2517bd6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e179f0 0x561cb6e21bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f2517bd6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e18bc0 0x561cb6e22110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f2517bd6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 0x561cb6e29150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f2517bd6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb6e20530 con 0x561cb6e19580 2024-03-21T11:32:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f2517bd6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb6d380d0 con 0x561cb6e179f0 2024-03-21T11:32:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f2517bd6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb6e206b0 con 0x561cb6e18bc0 2024-03-21T11:32:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f2517bd6740 10 client.0 _ll_register_callbacks cb 0x561cb6d3ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f250d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e18bc0 0x561cb6e22110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f250de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e179f0 0x561cb6e21bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f250e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 0x561cb6e29150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f250d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e18bc0 0x561cb6e22110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52862/0 (socket says 172.21.15.47:52862) 2024-03-21T11:32:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f250d400640 1 -- 172.21.15.47:0/556563748 learned_addr learned my addr 172.21.15.47:0/556563748 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f250d400640 1 -- 172.21.15.47:0/556563748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e179f0 msgr2=0x561cb6e21bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f250d400640 1 --2- 172.21.15.47:0/556563748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e179f0 0x561cb6e21bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f250d400640 1 -- 172.21.15.47:0/556563748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 msgr2=0x561cb6e29150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f250d400640 1 --2- 172.21.15.47:0/556563748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 0x561cb6e29150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.435+0000 7f250d400640 1 -- 172.21.15.47:0/556563748 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cb6a903f0 con 0x561cb6e18bc0 2024-03-21T11:32:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.436+0000 7f250d400640 1 --2- 172.21.15.47:0/556563748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e18bc0 0x561cb6e22110 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f24f80a4410 tx=0x7f24f80056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40710e157ae419c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.436+0000 7f24f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.436+0000 7f24f7e00640 1 -- 172.21.15.47:0/556563748 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f809bd00 con 0x561cb6e18bc0 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.436+0000 7f24f7e00640 1 -- 172.21.15.47:0/556563748 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f24f80a44b0 con 0x561cb6e18bc0 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.436+0000 7f2517bd6740 1 -- 172.21.15.47:0/556563748 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f250002cb20 con 0x561cb6e18bc0 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.436+0000 7f24f7e00640 1 -- 172.21.15.47:0/556563748 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24f809b890 con 0x561cb6e18bc0 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.436+0000 7f2517bd6740 10 client.28531 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.436+0000 7f2517bd6740 1 -- 172.21.15.47:0/556563748 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561cb6b5a7d0 con 0x561cb6e18bc0 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.436+0000 7f24f7e00640 1 -- 172.21.15.47:0/556563748 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f250002cb20 con 0x561cb6e18bc0 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.436+0000 7f24f7e00640 1 -- 172.21.15.47:0/556563748 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f24f809b380 con 0x561cb6e18bc0 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.437+0000 7f2517bd6740 10 client.28531 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.437+0000 7f2517bd6740 10 client.28531 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.437+0000 7f2517bd6740 20 client.28531 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.437+0000 7f2517bd6740 1 --2- 172.21.15.47:0/556563748 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561cb6e2d690 0x561cb6e2fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.437+0000 7f2517bd6740 4 client.28531 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.437+0000 7f2517bd6740 1 -- 172.21.15.47:0/556563748 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cb6d89800 con 0x561cb6e2d690 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.437+0000 7f250de00640 1 --2- 172.21.15.47:0/556563748 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561cb6e2d690 0x561cb6e2fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.438+0000 7f24f7e00640 10 client.28531 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:32:20.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.438+0000 7f250de00640 1 --2- 172.21.15.47:0/556563748 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561cb6e2d690 0x561cb6e2fad0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c57dada266477fb server_cookie=c3d46e2ccfcbf7b9 in_seq=0 out_seq=0 2024-03-21T11:32:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f24f7e00640 1 -- 172.21.15.47:0/556563748 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561cb6d89800 con 0x561cb6e2d690 2024-03-21T11:32:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f24f7e00640 10 client.28531 handle_command_reply: tid=0 2024-03-21T11:32:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f2517bd6740 1 client.28531 shutdown 2024-03-21T11:32:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f2517bd6740 1 -- 172.21.15.47:0/556563748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e18bc0 msgr2=0x561cb6e22110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f2517bd6740 1 --2- 172.21.15.47:0/556563748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e18bc0 0x561cb6e22110 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f24f80a4410 tx=0x7f24f80056b0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f2517bd6740 1 -- 172.21.15.47:0/556563748 shutdown_connections 2024-03-21T11:32:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f2517bd6740 1 --2- 172.21.15.47:0/556563748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 0x561cb6e29150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f2517bd6740 1 --2- 172.21.15.47:0/556563748 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561cb6e2d690 0x561cb6e2fad0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f2517bd6740 1 --2- 172.21.15.47:0/556563748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e18bc0 0x561cb6e22110 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f2517bd6740 1 --2- 172.21.15.47:0/556563748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e179f0 0x561cb6e21bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f2517bd6740 1 -- 172.21.15.47:0/556563748 >> 172.21.15.47:0/556563748 conn(0x561cb6e12de0 msgr2=0x561cb6e12160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f24f7e00640 0 client.28531 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:32:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f2517bd6740 1 -- 172.21.15.47:0/556563748 shutdown_connections 2024-03-21T11:32:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f2517bd6740 1 -- 172.21.15.47:0/556563748 wait complete. 2024-03-21T11:32:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.439+0000 7f2517bd6740 20 client.28531 trim_cache size 0 max 16384 2024-03-21T11:32:20.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.454+0000 7f2517bd6740 1 Processor -- start 2024-03-21T11:32:20.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.454+0000 7f2517bd6740 1 -- start start 2024-03-21T11:32:20.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.454+0000 7f2517bd6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e179f0 0x561cb6e21650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.454+0000 7f2517bd6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e18bc0 0x561cb6e29110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.454+0000 7f2517bd6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 0x561cb6e16750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.454+0000 7f2517bd6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25101abf80 con 0x561cb6e19580 2024-03-21T11:32:20.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.454+0000 7f2517bd6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25101ad900 con 0x561cb6e18bc0 2024-03-21T11:32:20.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.454+0000 7f2517bd6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25101ac5e0 con 0x561cb6e179f0 2024-03-21T11:32:20.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.454+0000 7f250de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e179f0 0x561cb6e21650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.454+0000 7f250d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e18bc0 0x561cb6e29110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.454+0000 7f250de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e179f0 0x561cb6e21650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52870/0 (socket says 172.21.15.47:52870) 2024-03-21T11:32:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.454+0000 7f250de00640 1 -- 172.21.15.47:0/2003506649 learned_addr learned my addr 172.21.15.47:0/2003506649 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.454+0000 7f250e800640 1 --2- 172.21.15.47:0/2003506649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 0x561cb6e16750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.455+0000 7f250de00640 1 -- 172.21.15.47:0/2003506649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e18bc0 msgr2=0x561cb6e29110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.455+0000 7f250de00640 1 --2- 172.21.15.47:0/2003506649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e18bc0 0x561cb6e29110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.455+0000 7f250de00640 1 -- 172.21.15.47:0/2003506649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 msgr2=0x561cb6e16750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.455+0000 7f250de00640 1 --2- 172.21.15.47:0/2003506649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 0x561cb6e16750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.455+0000 7f250de00640 1 -- 172.21.15.47:0/2003506649 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cb6e18640 con 0x561cb6e179f0 2024-03-21T11:32:20.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.455+0000 7f250d400640 1 --2- 172.21.15.47:0/2003506649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e18bc0 0x561cb6e29110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:20.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.455+0000 7f250de00640 1 --2- 172.21.15.47:0/2003506649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e179f0 0x561cb6e21650 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f250002c170 tx=0x7f2500005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4865eb1614c38738 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.455+0000 7f24fea00640 1 -- 172.21.15.47:0/2003506649 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500005d50 con 0x561cb6e179f0 2024-03-21T11:32:20.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.455+0000 7f24fea00640 1 -- 172.21.15.47:0/2003506649 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2500009520 con 0x561cb6e179f0 2024-03-21T11:32:20.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.455+0000 7f24fea00640 1 -- 172.21.15.47:0/2003506649 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500003ca0 con 0x561cb6e179f0 2024-03-21T11:32:20.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.456+0000 7f2517bd6740 1 -- 172.21.15.47:0/2003506649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e179f0 msgr2=0x561cb6e21650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.456+0000 7f2517bd6740 1 --2- 172.21.15.47:0/2003506649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e179f0 0x561cb6e21650 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f250002c170 tx=0x7f2500005fd0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.456+0000 7f2517bd6740 1 -- 172.21.15.47:0/2003506649 shutdown_connections 2024-03-21T11:32:20.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.456+0000 7f2517bd6740 1 --2- 172.21.15.47:0/2003506649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 0x561cb6e16750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.456+0000 7f2517bd6740 1 --2- 172.21.15.47:0/2003506649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e18bc0 0x561cb6e29110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.456+0000 7f2517bd6740 1 --2- 172.21.15.47:0/2003506649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e179f0 0x561cb6e21650 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.456+0000 7f2517bd6740 1 -- 172.21.15.47:0/2003506649 >> 172.21.15.47:0/2003506649 conn(0x561cb6e12de0 msgr2=0x561cb6e16040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.456+0000 7f2517bd6740 1 -- 172.21.15.47:0/2003506649 shutdown_connections 2024-03-21T11:32:20.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.456+0000 7f2517bd6740 1 -- 172.21.15.47:0/2003506649 wait complete. 2024-03-21T11:32:20.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.456+0000 7f2517bd6740 1 -- start start 2024-03-21T11:32:20.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.456+0000 7f2517bd6740 1 Processor -- start 2024-03-21T11:32:20.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.457+0000 7f2517bd6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e179f0 0x561cb6e28bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.457+0000 7f2517bd6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e18bc0 0x561cb6e29110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.457+0000 7f2517bd6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 0x561cb6e25760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.457+0000 7f2517bd6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25101ad2a0 con 0x561cb6e19580 2024-03-21T11:32:20.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.457+0000 7f2517bd6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25101acc40 con 0x561cb6e179f0 2024-03-21T11:32:20.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.457+0000 7f2517bd6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25101aa140 con 0x561cb6e18bc0 2024-03-21T11:32:20.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.457+0000 7f2517bd6740 10 client.0 _ll_register_callbacks cb 0x7f24f800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:20.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.457+0000 7f250de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e179f0 0x561cb6e28bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.457+0000 7f250d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e18bc0 0x561cb6e29110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.457+0000 7f250de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e179f0 0x561cb6e28bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34628/0 (socket says 172.21.15.47:34628) 2024-03-21T11:32:20.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.457+0000 7f250de00640 1 -- 172.21.15.47:0/1130275846 learned_addr learned my addr 172.21.15.47:0/1130275846 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.457+0000 7f250e800640 1 --2- 172.21.15.47:0/1130275846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 0x561cb6e25760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.457+0000 7f250d400640 1 -- 172.21.15.47:0/1130275846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e179f0 msgr2=0x561cb6e28bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.457+0000 7f250d400640 1 --2- 172.21.15.47:0/1130275846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e179f0 0x561cb6e28bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.458+0000 7f250d400640 1 -- 172.21.15.47:0/1130275846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 msgr2=0x561cb6e25760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.458+0000 7f250d400640 1 --2- 172.21.15.47:0/1130275846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 0x561cb6e25760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.458+0000 7f250d400640 1 -- 172.21.15.47:0/1130275846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cb6e25f90 con 0x561cb6e18bc0 2024-03-21T11:32:20.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.458+0000 7f250de00640 1 --2- 172.21.15.47:0/1130275846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e179f0 0x561cb6e28bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:20.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.458+0000 7f250d400640 1 --2- 172.21.15.47:0/1130275846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e18bc0 0x561cb6e29110 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f24f809a1e0 tx=0x7f24f80b50d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=992ead492a2d1c6b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.458+0000 7f24f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:20.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.458+0000 7f24f7e00640 1 -- 172.21.15.47:0/1130275846 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f809ba70 con 0x561cb6e18bc0 2024-03-21T11:32:20.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.458+0000 7f24f7e00640 1 -- 172.21.15.47:0/1130275846 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f24f8097c80 con 0x561cb6e18bc0 2024-03-21T11:32:20.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.458+0000 7f2517bd6740 1 -- 172.21.15.47:0/1130275846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2500009520 con 0x561cb6e18bc0 2024-03-21T11:32:20.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.458+0000 7f24f7e00640 1 -- 172.21.15.47:0/1130275846 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24f809bd00 con 0x561cb6e18bc0 2024-03-21T11:32:20.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.458+0000 7f2517bd6740 10 client.28537 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:20.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.458+0000 7f2517bd6740 1 -- 172.21.15.47:0/1130275846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561cb6e89e30 con 0x561cb6e18bc0 2024-03-21T11:32:20.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.458+0000 7f24f7e00640 1 -- 172.21.15.47:0/1130275846 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2500009520 con 0x561cb6e18bc0 2024-03-21T11:32:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.459+0000 7f24f7e00640 1 -- 172.21.15.47:0/1130275846 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f24f8097e80 con 0x561cb6e18bc0 2024-03-21T11:32:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.459+0000 7f2517bd6740 10 client.28537 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.459+0000 7f2517bd6740 10 client.28537 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:32:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.459+0000 7f2517bd6740 20 client.28537 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.459+0000 7f2517bd6740 1 --2- 172.21.15.47:0/1130275846 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561cb6e8a6e0 0x561cb6e8cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.459+0000 7f2517bd6740 4 client.28537 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:32:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.459+0000 7f2517bd6740 1 -- 172.21.15.47:0/1130275846 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561cb6d83b50 con 0x561cb6e8a6e0 2024-03-21T11:32:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.459+0000 7f250de00640 1 --2- 172.21.15.47:0/1130275846 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561cb6e8a6e0 0x561cb6e8cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.459+0000 7f250de00640 1 --2- 172.21.15.47:0/1130275846 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561cb6e8a6e0 0x561cb6e8cb00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c3c46ae619a1555 server_cookie=22757ec195d44f14 in_seq=0 out_seq=0 2024-03-21T11:32:20.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.459+0000 7f24f7e00640 10 client.28537 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:32:20.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f24f7e00640 1 -- 172.21.15.47:0/1130275846 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561cb6d83b50 con 0x561cb6e8a6e0 2024-03-21T11:32:20.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f24f7e00640 10 client.28537 handle_command_reply: tid=0 2024-03-21T11:32:20.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f2517bd6740 1 client.28537 shutdown 2024-03-21T11:32:20.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f2517bd6740 1 -- 172.21.15.47:0/1130275846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e18bc0 msgr2=0x561cb6e29110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f2517bd6740 1 --2- 172.21.15.47:0/1130275846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e18bc0 0x561cb6e29110 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f24f809a1e0 tx=0x7f24f80b50d0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f2517bd6740 1 -- 172.21.15.47:0/1130275846 shutdown_connections 2024-03-21T11:32:20.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f2517bd6740 1 --2- 172.21.15.47:0/1130275846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561cb6e19580 0x561cb6e25760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f2517bd6740 1 --2- 172.21.15.47:0/1130275846 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561cb6e8a6e0 0x561cb6e8cb00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f2517bd6740 1 --2- 172.21.15.47:0/1130275846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561cb6e18bc0 0x561cb6e29110 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f2517bd6740 1 --2- 172.21.15.47:0/1130275846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561cb6e179f0 0x561cb6e28bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f24f7e00640 0 client.28537 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:32:20.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f2517bd6740 1 -- 172.21.15.47:0/1130275846 >> 172.21.15.47:0/1130275846 conn(0x561cb6e12de0 msgr2=0x561cb6e168d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f2517bd6740 1 -- 172.21.15.47:0/1130275846 shutdown_connections 2024-03-21T11:32:20.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f2517bd6740 1 -- 172.21.15.47:0/1130275846 wait complete. 2024-03-21T11:32:20.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.460+0000 7f2517bd6740 20 client.28537 trim_cache size 0 max 16384 2024-03-21T11:32:20.467 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:32:20.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.461+0000 7f250f200640 1 -- 172.21.15.47:0/1696711864 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f24f00a2fa0 msgr2=0x7f24f00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.461+0000 7f250f200640 1 --2- 172.21.15.47:0/1696711864 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f24f00a2fa0 0x7f24f00a5460 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2508009fd0 tx=0x7f2508009940 comp rx=0 tx=0).stop 2024-03-21T11:32:20.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.461+0000 7f250f200640 1 -- 172.21.15.47:0/1696711864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2510104990 msgr2=0x7f251019a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.461+0000 7f250f200640 1 --2- 172.21.15.47:0/1696711864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2510104990 0x7f251019a040 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f24f800e9f0 tx=0x7f24f800edf0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.461+0000 7f250f200640 1 -- 172.21.15.47:0/1696711864 shutdown_connections 2024-03-21T11:32:20.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.461+0000 7f250f200640 1 --2- 172.21.15.47:0/1696711864 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f24f00a2fa0 0x7f24f00a5460 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.461+0000 7f250f200640 1 --2- 172.21.15.47:0/1696711864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2510105350 0x7f25101a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.461+0000 7f250f200640 1 --2- 172.21.15.47:0/1696711864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2510104990 0x7f251019a040 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.461+0000 7f250f200640 1 --2- 172.21.15.47:0/1696711864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25101037a0 0x7f2510199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.461+0000 7f250f200640 1 -- 172.21.15.47:0/1696711864 >> 172.21.15.47:0/1696711864 conn(0x7f25100fed10 msgr2=0x7f2510107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.461+0000 7f250f200640 1 -- 172.21.15.47:0/1696711864 shutdown_connections 2024-03-21T11:32:20.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.461+0000 7f250f200640 1 -- 172.21.15.47:0/1696711864 wait complete. 2024-03-21T11:32:20.478 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:32:20.478 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:32:20.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:19 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/556620423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:32:20.543 INFO:tasks.workunit.client.0.smithi047.stdout: 1 654102 8.19 MB/sec execute 304 sec latency 11.865 ms 2024-03-21T11:32:20.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.708+0000 7f1043600640 1 -- 172.21.15.47:0/3743449120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f103c109870 msgr2=0x7f103c110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.708+0000 7f1043600640 1 --2- 172.21.15.47:0/3743449120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f103c109870 0x7f103c110100 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1034009ca0 tx=0x7f103402c0f0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.708+0000 7f1043600640 1 -- 172.21.15.47:0/3743449120 shutdown_connections 2024-03-21T11:32:20.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.709+0000 7f1043600640 1 --2- 172.21.15.47:0/3743449120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f103c109870 0x7f103c110100 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.709+0000 7f1043600640 1 --2- 172.21.15.47:0/3743449120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f103c108eb0 0x7f103c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.709+0000 7f1043600640 1 --2- 172.21.15.47:0/3743449120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f103c107cc0 0x7f103c1080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.709+0000 7f1043600640 1 -- 172.21.15.47:0/3743449120 >> 172.21.15.47:0/3743449120 conn(0x7f103c076c60 msgr2=0x7f103c0790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.709+0000 7f1043600640 1 -- 172.21.15.47:0/3743449120 shutdown_connections 2024-03-21T11:32:20.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.709+0000 7f1043600640 1 -- 172.21.15.47:0/3743449120 wait complete. 2024-03-21T11:32:20.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.710+0000 7f1043600640 1 Processor -- start 2024-03-21T11:32:20.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.710+0000 7f1043600640 1 -- start start 2024-03-21T11:32:20.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.710+0000 7f1043600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f103c107cc0 0x7f103c19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.710+0000 7f1043600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f103c108eb0 0x7f103c19e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.710+0000 7f1043600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f103c109870 0x7f103c1a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.710+0000 7f1043600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c113920 con 0x7f103c108eb0 2024-03-21T11:32:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.710+0000 7f1043600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1137a0 con 0x7f103c107cc0 2024-03-21T11:32:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.710+0000 7f1043600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c113aa0 con 0x7f103c109870 2024-03-21T11:32:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.711+0000 7f1040c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f103c107cc0 0x7f103c19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.711+0000 7f1041600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f103c109870 0x7f103c1a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.711+0000 7f1040c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f103c107cc0 0x7f103c19df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34654/0 (socket says 172.21.15.47:34654) 2024-03-21T11:32:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.711+0000 7f1041600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f103c109870 0x7f103c1a5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52886/0 (socket says 172.21.15.47:52886) 2024-03-21T11:32:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.711+0000 7f1040c00640 1 -- 172.21.15.47:0/4042123399 learned_addr learned my addr 172.21.15.47:0/4042123399 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.711+0000 7f103be00640 1 --2- 172.21.15.47:0/4042123399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f103c108eb0 0x7f103c19e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.711+0000 7f1041600640 1 -- 172.21.15.47:0/4042123399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f103c107cc0 msgr2=0x7f103c19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.711+0000 7f1041600640 1 --2- 172.21.15.47:0/4042123399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f103c107cc0 0x7f103c19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.711+0000 7f1041600640 1 -- 172.21.15.47:0/4042123399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f103c108eb0 msgr2=0x7f103c19e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.711+0000 7f1041600640 1 --2- 172.21.15.47:0/4042123399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f103c108eb0 0x7f103c19e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.711+0000 7f1041600640 1 -- 172.21.15.47:0/4042123399 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c1a5b80 con 0x7f103c109870 2024-03-21T11:32:20.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.711+0000 7f103be00640 1 --2- 172.21.15.47:0/4042123399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f103c108eb0 0x7f103c19e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:20.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.711+0000 7f1041600640 1 --2- 172.21.15.47:0/4042123399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f103c109870 0x7f103c1a5510 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1034037040 tx=0x7f1034032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.712+0000 7f1039600640 1 -- 172.21.15.47:0/4042123399 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034041020 con 0x7f103c109870 2024-03-21T11:32:20.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.712+0000 7f1039600640 1 -- 172.21.15.47:0/4042123399 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1034034cd0 con 0x7f103c109870 2024-03-21T11:32:20.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.712+0000 7f1039600640 1 -- 172.21.15.47:0/4042123399 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034039040 con 0x7f103c109870 2024-03-21T11:32:20.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.712+0000 7f1043600640 1 -- 172.21.15.47:0/4042123399 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103c1a5e10 con 0x7f103c109870 2024-03-21T11:32:20.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.712+0000 7f1043600640 1 -- 172.21.15.47:0/4042123399 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103c1a6220 con 0x7f103c109870 2024-03-21T11:32:20.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.717+0000 7f1039600640 1 -- 172.21.15.47:0/4042123399 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1034044020 con 0x7f103c109870 2024-03-21T11:32:20.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.717+0000 7f10463ca740 1 Processor -- start 2024-03-21T11:32:20.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.717+0000 7f10463ca740 1 -- start start 2024-03-21T11:32:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.718+0000 7f10463ca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e5184810 0x5563e5184c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.718+0000 7f10463ca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5185e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.718+0000 7f10463ca740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e51863a0 0x5563e518cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.718+0000 7f10463ca740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e50e92c0 con 0x5563e51863a0 2024-03-21T11:32:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.718+0000 7f10463ca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e5091c90 con 0x5563e51859e0 2024-03-21T11:32:20.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.718+0000 7f10463ca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e50e5360 con 0x5563e5184810 2024-03-21T11:32:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.718+0000 7f103be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e5184810 0x5563e5184c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.718+0000 7f103be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e5184810 0x5563e5184c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52888/0 (socket says 172.21.15.47:52888) 2024-03-21T11:32:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.718+0000 7f103be00640 1 -- 172.21.15.47:0/2234117360 learned_addr learned my addr 172.21.15.47:0/2234117360 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.718+0000 7f103be00640 1 --2- 172.21.15.47:0/2234117360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e51863a0 0x5563e518cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.718+0000 7f1040c00640 1 --2- 172.21.15.47:0/2234117360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5185e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.719+0000 7f1040c00640 1 -- 172.21.15.47:0/2234117360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e5184810 msgr2=0x5563e5184c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.719+0000 7f1040c00640 1 --2- 172.21.15.47:0/2234117360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e5184810 0x5563e5184c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.719+0000 7f1040c00640 1 -- 172.21.15.47:0/2234117360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e51863a0 msgr2=0x5563e518cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.719+0000 7f1040c00640 1 --2- 172.21.15.47:0/2234117360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e51863a0 0x5563e518cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.719+0000 7f1040c00640 1 -- 172.21.15.47:0/2234117360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563e50b6080 con 0x5563e51859e0 2024-03-21T11:32:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.719+0000 7f103be00640 1 --2- 172.21.15.47:0/2234117360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e5184810 0x5563e5184c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.719+0000 7f1040c00640 1 --2- 172.21.15.47:0/2234117360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5185e60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f10300058c0 tx=0x7f1030011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f22b32a81b6c671a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.719+0000 7f1039600640 1 --2- 172.21.15.47:0/4042123399 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f10240a2f20 0x7f10240a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.720+0000 7f102dc00640 1 -- 172.21.15.47:0/2234117360 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030004220 con 0x5563e51859e0 2024-03-21T11:32:20.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.720+0000 7f102dc00640 1 -- 172.21.15.47:0/2234117360 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10300043c0 con 0x5563e51859e0 2024-03-21T11:32:20.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.720+0000 7f102dc00640 1 -- 172.21.15.47:0/2234117360 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030004db0 con 0x5563e51859e0 2024-03-21T11:32:20.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.720+0000 7f1039600640 1 -- 172.21.15.47:0/4042123399 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f10340f1610 con 0x7f103c109870 2024-03-21T11:32:20.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.721+0000 7f103be00640 1 --2- 172.21.15.47:0/4042123399 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f10240a2f20 0x7f10240a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.721+0000 7f10463ca740 1 -- 172.21.15.47:0/2234117360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 msgr2=0x5563e5185e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.721+0000 7f10463ca740 1 --2- 172.21.15.47:0/2234117360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5185e60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f10300058c0 tx=0x7f1030011810 comp rx=0 tx=0).stop 2024-03-21T11:32:20.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.721+0000 7f10463ca740 1 -- 172.21.15.47:0/2234117360 shutdown_connections 2024-03-21T11:32:20.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.721+0000 7f10463ca740 1 --2- 172.21.15.47:0/2234117360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e51863a0 0x5563e518cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.721+0000 7f10463ca740 1 --2- 172.21.15.47:0/2234117360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5185e60 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.721+0000 7f10463ca740 1 --2- 172.21.15.47:0/2234117360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e5184810 0x5563e5184c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.721+0000 7f10463ca740 1 -- 172.21.15.47:0/2234117360 >> 172.21.15.47:0/2234117360 conn(0x5563e517fc00 msgr2=0x5563e5180000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.721+0000 7f10463ca740 1 -- 172.21.15.47:0/2234117360 shutdown_connections 2024-03-21T11:32:20.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.722+0000 7f10463ca740 1 -- 172.21.15.47:0/2234117360 wait complete. 2024-03-21T11:32:20.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.722+0000 7f103be00640 1 --2- 172.21.15.47:0/4042123399 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f10240a2f20 0x7f10240a53e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1028009fd0 tx=0x7f1028005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.722+0000 7f10463ca740 1 -- start start 2024-03-21T11:32:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.723+0000 7f10463ca740 1 Processor -- start 2024-03-21T11:32:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.723+0000 7f10463ca740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e5184810 0x5563e518ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.723+0000 7f10463ca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e51859e0 0x5563e518efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.723+0000 7f10463ca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51863a0 0x5563e5196000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.723+0000 7f10463ca740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e518d3b0 con 0x5563e5184810 2024-03-21T11:32:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.723+0000 7f10463ca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e50a50d0 con 0x5563e51863a0 2024-03-21T11:32:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.723+0000 7f10463ca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e518d530 con 0x5563e51859e0 2024-03-21T11:32:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.723+0000 7f10463ca740 10 client.0 _ll_register_callbacks cb 0x5563e50a9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.723+0000 7f1041600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51863a0 0x5563e5196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.723+0000 7f1041600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51863a0 0x5563e5196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34672/0 (socket says 172.21.15.47:34672) 2024-03-21T11:32:20.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.723+0000 7f1041600640 1 -- 172.21.15.47:0/3184777111 learned_addr learned my addr 172.21.15.47:0/3184777111 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.724+0000 7f103be00640 1 --2- 172.21.15.47:0/3184777111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e51859e0 0x5563e518efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.724+0000 7f1040c00640 1 --2- 172.21.15.47:0/3184777111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e5184810 0x5563e518ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.724+0000 7f1041600640 1 -- 172.21.15.47:0/3184777111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e51859e0 msgr2=0x5563e518efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.724+0000 7f1041600640 1 --2- 172.21.15.47:0/3184777111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e51859e0 0x5563e518efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.724+0000 7f1041600640 1 -- 172.21.15.47:0/3184777111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e5184810 msgr2=0x5563e518ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.724+0000 7f1041600640 1 --2- 172.21.15.47:0/3184777111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e5184810 0x5563e518ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.724+0000 7f1041600640 1 -- 172.21.15.47:0/3184777111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563e4dfd3f0 con 0x5563e51863a0 2024-03-21T11:32:20.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.724+0000 7f1040c00640 1 --2- 172.21.15.47:0/3184777111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e5184810 0x5563e518ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:32:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.724+0000 7f1041600640 1 --2- 172.21.15.47:0/3184777111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51863a0 0x5563e5196000 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f10340ba3b0 tx=0x7f10340bd5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e474eba3de6921f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.724+0000 7f1022000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:32:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.725+0000 7f1022000640 1 -- 172.21.15.47:0/3184777111 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10340098b0 con 0x5563e51863a0 2024-03-21T11:32:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.725+0000 7f1022000640 1 -- 172.21.15.47:0/3184777111 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f10340f1320 con 0x5563e51863a0 2024-03-21T11:32:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.725+0000 7f10463ca740 1 -- 172.21.15.47:0/3184777111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10300043c0 con 0x5563e51863a0 2024-03-21T11:32:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.725+0000 7f1022000640 1 -- 172.21.15.47:0/3184777111 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10340bd9e0 con 0x5563e51863a0 2024-03-21T11:32:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.725+0000 7f10463ca740 10 client.28544 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.725+0000 7f10463ca740 1 -- 172.21.15.47:0/3184777111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5563e4ec77d0 con 0x5563e51863a0 2024-03-21T11:32:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.726+0000 7f1022000640 1 -- 172.21.15.47:0/3184777111 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10300043c0 con 0x5563e51863a0 2024-03-21T11:32:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.726+0000 7f1022000640 1 -- 172.21.15.47:0/3184777111 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f103403c030 con 0x5563e51863a0 2024-03-21T11:32:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.726+0000 7f10463ca740 10 client.28544 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.726+0000 7f10463ca740 10 client.28544 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:32:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.726+0000 7f10463ca740 20 client.28544 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.727+0000 7f10463ca740 1 --2- 172.21.15.47:0/3184777111 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5563e519a410 0x5563e519c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.727+0000 7f10463ca740 4 client.28544 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.727+0000 7f10463ca740 1 -- 172.21.15.47:0/3184777111 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563e50f6800 con 0x5563e519a410 2024-03-21T11:32:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.727+0000 7f1040c00640 1 --2- 172.21.15.47:0/3184777111 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5563e519a410 0x5563e519c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.728+0000 7f1040c00640 1 --2- 172.21.15.47:0/3184777111 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5563e519a410 0x5563e519c8a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4dacedd971b463a5 server_cookie=2982aa3c5ce2d8de in_seq=0 out_seq=0 2024-03-21T11:32:20.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.728+0000 7f1022000640 10 client.28544 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:32:20.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.728+0000 7f1022000640 1 -- 172.21.15.47:0/3184777111 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5563e50f6800 con 0x5563e519a410 2024-03-21T11:32:20.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.728+0000 7f1022000640 10 client.28544 handle_command_reply: tid=0 2024-03-21T11:32:20.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.729+0000 7f10463ca740 1 client.28544 shutdown 2024-03-21T11:32:20.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.729+0000 7f10463ca740 1 -- 172.21.15.47:0/3184777111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51863a0 msgr2=0x5563e5196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.729+0000 7f10463ca740 1 --2- 172.21.15.47:0/3184777111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51863a0 0x5563e5196000 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f10340ba3b0 tx=0x7f10340bd5a0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.729+0000 7f10463ca740 1 -- 172.21.15.47:0/3184777111 shutdown_connections 2024-03-21T11:32:20.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.729+0000 7f10463ca740 1 --2- 172.21.15.47:0/3184777111 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5563e519a410 0x5563e519c8a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.729+0000 7f10463ca740 1 --2- 172.21.15.47:0/3184777111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51863a0 0x5563e5196000 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.729+0000 7f10463ca740 1 --2- 172.21.15.47:0/3184777111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e51859e0 0x5563e518efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.729+0000 7f10463ca740 1 --2- 172.21.15.47:0/3184777111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e5184810 0x5563e518ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.729+0000 7f1022000640 0 client.28544 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.729+0000 7f10463ca740 1 -- 172.21.15.47:0/3184777111 >> 172.21.15.47:0/3184777111 conn(0x5563e517fc00 msgr2=0x5563e517eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.729+0000 7f10463ca740 1 -- 172.21.15.47:0/3184777111 shutdown_connections 2024-03-21T11:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.729+0000 7f10463ca740 1 -- 172.21.15.47:0/3184777111 wait complete. 2024-03-21T11:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.730+0000 7f10463ca740 20 client.28544 trim_cache size 0 max 16384 2024-03-21T11:32:20.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.743+0000 7f10463ca740 1 Processor -- start 2024-03-21T11:32:20.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.743+0000 7f10463ca740 1 -- start start 2024-03-21T11:32:20.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.744+0000 7f10463ca740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e5184810 0x5563e517ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.744+0000 7f10463ca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.744+0000 7f10463ca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e51863a0 0x5563e51a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.744+0000 7f10463ca740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1b06b0 con 0x5563e5184810 2024-03-21T11:32:20.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.744+0000 7f10463ca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1b2030 con 0x5563e51859e0 2024-03-21T11:32:20.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.744+0000 7f10463ca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1b0d10 con 0x5563e51863a0 2024-03-21T11:32:20.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.744+0000 7f103be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.744+0000 7f103be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5195fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34682/0 (socket says 172.21.15.47:34682) 2024-03-21T11:32:20.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.744+0000 7f103be00640 1 -- 172.21.15.47:0/1165847769 learned_addr learned my addr 172.21.15.47:0/1165847769 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.744+0000 7f1040c00640 1 --2- 172.21.15.47:0/1165847769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e5184810 0x5563e517ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.744+0000 7f1041600640 1 --2- 172.21.15.47:0/1165847769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e51863a0 0x5563e51a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.744+0000 7f103be00640 1 -- 172.21.15.47:0/1165847769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e51863a0 msgr2=0x5563e51a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.745+0000 7f103be00640 1 --2- 172.21.15.47:0/1165847769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e51863a0 0x5563e51a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.745+0000 7f103be00640 1 -- 172.21.15.47:0/1165847769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e5184810 msgr2=0x5563e517ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.745+0000 7f103be00640 1 --2- 172.21.15.47:0/1165847769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e5184810 0x5563e517ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.745+0000 7f103be00640 1 -- 172.21.15.47:0/1165847769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563e518dea0 con 0x5563e51859e0 2024-03-21T11:32:20.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.745+0000 7f1041600640 1 --2- 172.21.15.47:0/1165847769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e51863a0 0x5563e51a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:20.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.745+0000 7f103be00640 1 --2- 172.21.15.47:0/1165847769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5195fc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1028012540 tx=0x7f10280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51cc02610a31ad8a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.745+0000 7f102d200640 1 -- 172.21.15.47:0/1165847769 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028004020 con 0x5563e51859e0 2024-03-21T11:32:20.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.745+0000 7f102d200640 1 -- 172.21.15.47:0/1165847769 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10280041c0 con 0x5563e51859e0 2024-03-21T11:32:20.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.745+0000 7f102d200640 1 -- 172.21.15.47:0/1165847769 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102801f8f0 con 0x5563e51859e0 2024-03-21T11:32:20.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.746+0000 7f10463ca740 1 -- 172.21.15.47:0/1165847769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 msgr2=0x5563e5195fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.746+0000 7f10463ca740 1 --2- 172.21.15.47:0/1165847769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5195fc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1028012540 tx=0x7f10280046c0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.746+0000 7f10463ca740 1 -- 172.21.15.47:0/1165847769 shutdown_connections 2024-03-21T11:32:20.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.746+0000 7f10463ca740 1 --2- 172.21.15.47:0/1165847769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e51863a0 0x5563e51a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.746+0000 7f10463ca740 1 --2- 172.21.15.47:0/1165847769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5195fc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.746+0000 7f10463ca740 1 --2- 172.21.15.47:0/1165847769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e5184810 0x5563e517ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.746+0000 7f10463ca740 1 -- 172.21.15.47:0/1165847769 >> 172.21.15.47:0/1165847769 conn(0x5563e517fc00 msgr2=0x5563e517e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.746+0000 7f10463ca740 1 -- 172.21.15.47:0/1165847769 shutdown_connections 2024-03-21T11:32:20.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.746+0000 7f10463ca740 1 -- 172.21.15.47:0/1165847769 wait complete. 2024-03-21T11:32:20.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.747+0000 7f10463ca740 1 -- start start 2024-03-21T11:32:20.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.747+0000 7f10463ca740 1 Processor -- start 2024-03-21T11:32:20.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.747+0000 7f10463ca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e5184810 0x5563e5195920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.747+0000 7f10463ca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5195e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.747+0000 7f10463ca740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e51863a0 0x5563e51923c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.747+0000 7f10463ca740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1b19d0 con 0x5563e51863a0 2024-03-21T11:32:20.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.747+0000 7f10463ca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1b1370 con 0x5563e51859e0 2024-03-21T11:32:20.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.747+0000 7f10463ca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c1ae870 con 0x5563e5184810 2024-03-21T11:32:20.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.747+0000 7f10463ca740 10 client.0 _ll_register_callbacks cb 0x7f10340bd7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:20.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.747+0000 7f103be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.747+0000 7f1040c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e5184810 0x5563e5195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.747+0000 7f103be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34686/0 (socket says 172.21.15.47:34686) 2024-03-21T11:32:20.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.747+0000 7f103be00640 1 -- 172.21.15.47:0/3322438868 learned_addr learned my addr 172.21.15.47:0/3322438868 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:20.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.748+0000 7f1041600640 1 --2- 172.21.15.47:0/3322438868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e51863a0 0x5563e51923c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.748+0000 7f103be00640 1 -- 172.21.15.47:0/3322438868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e5184810 msgr2=0x5563e5195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.748+0000 7f103be00640 1 --2- 172.21.15.47:0/3322438868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e5184810 0x5563e5195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.748+0000 7f103be00640 1 -- 172.21.15.47:0/3322438868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e51863a0 msgr2=0x5563e51923c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:20.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.748+0000 7f103be00640 1 --2- 172.21.15.47:0/3322438868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e51863a0 0x5563e51923c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.748+0000 7f103be00640 1 -- 172.21.15.47:0/3322438868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563e5192bf0 con 0x5563e51859e0 2024-03-21T11:32:20.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.748+0000 7f103be00640 1 --2- 172.21.15.47:0/3322438868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5195e60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1028003f80 tx=0x7f1028003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15a52d3c47e560d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:20.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.748+0000 7f1022000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:32:20.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.748+0000 7f1022000640 1 -- 172.21.15.47:0/3322438868 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102801f830 con 0x5563e51859e0 2024-03-21T11:32:20.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.749+0000 7f1022000640 1 -- 172.21.15.47:0/3322438868 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f102801f9d0 con 0x5563e51859e0 2024-03-21T11:32:20.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.749+0000 7f10463ca740 1 -- 172.21.15.47:0/3322438868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10280041c0 con 0x5563e51859e0 2024-03-21T11:32:20.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.749+0000 7f1022000640 1 -- 172.21.15.47:0/3322438868 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1028003fe0 con 0x5563e51859e0 2024-03-21T11:32:20.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.749+0000 7f10463ca740 10 client.28550 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:20.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.749+0000 7f10463ca740 1 -- 172.21.15.47:0/3322438868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5563e51f6a70 con 0x5563e51859e0 2024-03-21T11:32:20.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.749+0000 7f1022000640 1 -- 172.21.15.47:0/3322438868 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10280041c0 con 0x5563e51859e0 2024-03-21T11:32:20.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.749+0000 7f1022000640 1 -- 172.21.15.47:0/3322438868 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1028027070 con 0x5563e51859e0 2024-03-21T11:32:20.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.749+0000 7f10463ca740 10 client.28550 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:20.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.749+0000 7f10463ca740 10 client.28550 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:32:20.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.749+0000 7f10463ca740 20 client.28550 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:20.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.749+0000 7f10463ca740 1 --2- 172.21.15.47:0/3322438868 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5563e51f73f0 0x5563e51f9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:20.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.749+0000 7f10463ca740 4 client.28550 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:32:20.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.749+0000 7f10463ca740 1 -- 172.21.15.47:0/3322438868 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5563e50f0b50 con 0x5563e51f73f0 2024-03-21T11:32:20.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.750+0000 7f1040c00640 1 --2- 172.21.15.47:0/3322438868 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5563e51f73f0 0x5563e51f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:20.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.750+0000 7f1040c00640 1 --2- 172.21.15.47:0/3322438868 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5563e51f73f0 0x5563e51f9810 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d0bcfb750b74dc4 server_cookie=9b77b0a83001b539 in_seq=0 out_seq=0 2024-03-21T11:32:20.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.750+0000 7f1022000640 10 client.28550 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:32:20.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.750+0000 7f1022000640 1 -- 172.21.15.47:0/3322438868 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5563e50f0b50 con 0x5563e51f73f0 2024-03-21T11:32:20.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.750+0000 7f1022000640 10 client.28550 handle_command_reply: tid=0 2024-03-21T11:32:20.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.750+0000 7f10463ca740 1 client.28550 shutdown 2024-03-21T11:32:20.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.751+0000 7f10463ca740 1 -- 172.21.15.47:0/3322438868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 msgr2=0x5563e5195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.751+0000 7f10463ca740 1 --2- 172.21.15.47:0/3322438868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5195e60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1028003f80 tx=0x7f1028003c80 comp rx=0 tx=0).stop 2024-03-21T11:32:20.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.751+0000 7f10463ca740 1 -- 172.21.15.47:0/3322438868 shutdown_connections 2024-03-21T11:32:20.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.751+0000 7f10463ca740 1 --2- 172.21.15.47:0/3322438868 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5563e51f73f0 0x5563e51f9810 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.751+0000 7f10463ca740 1 --2- 172.21.15.47:0/3322438868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563e51863a0 0x5563e51923c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.751+0000 7f10463ca740 1 --2- 172.21.15.47:0/3322438868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563e51859e0 0x5563e5195e60 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.751+0000 7f10463ca740 1 --2- 172.21.15.47:0/3322438868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563e5184810 0x5563e5195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.751+0000 7f10463ca740 1 -- 172.21.15.47:0/3322438868 >> 172.21.15.47:0/3322438868 conn(0x5563e517fc00 msgr2=0x5563e517e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.751+0000 7f1022000640 0 client.28550 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:32:20.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.751+0000 7f10463ca740 1 -- 172.21.15.47:0/3322438868 shutdown_connections 2024-03-21T11:32:20.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.751+0000 7f10463ca740 1 -- 172.21.15.47:0/3322438868 wait complete. 2024-03-21T11:32:20.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.751+0000 7f10463ca740 20 client.28550 trim_cache size 0 max 16384 2024-03-21T11:32:20.768 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:32:20.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.751+0000 7f1043600640 1 -- 172.21.15.47:0/4042123399 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f10240a2f20 msgr2=0x7f10240a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.751+0000 7f1043600640 1 --2- 172.21.15.47:0/4042123399 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f10240a2f20 0x7f10240a53e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1028009fd0 tx=0x7f1028005f90 comp rx=0 tx=0).stop 2024-03-21T11:32:20.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.752+0000 7f1043600640 1 -- 172.21.15.47:0/4042123399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f103c109870 msgr2=0x7f103c1a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:20.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.752+0000 7f1043600640 1 --2- 172.21.15.47:0/4042123399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f103c109870 0x7f103c1a5510 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1034037040 tx=0x7f1034032ea0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.752+0000 7f1043600640 1 -- 172.21.15.47:0/4042123399 shutdown_connections 2024-03-21T11:32:20.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.752+0000 7f1043600640 1 --2- 172.21.15.47:0/4042123399 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f10240a2f20 0x7f10240a53e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.752+0000 7f1043600640 1 --2- 172.21.15.47:0/4042123399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f103c109870 0x7f103c1a5510 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.752+0000 7f1043600640 1 --2- 172.21.15.47:0/4042123399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f103c108eb0 0x7f103c19e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.752+0000 7f1043600640 1 --2- 172.21.15.47:0/4042123399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f103c107cc0 0x7f103c19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:20.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.752+0000 7f1043600640 1 -- 172.21.15.47:0/4042123399 >> 172.21.15.47:0/4042123399 conn(0x7f103c076c60 msgr2=0x7f103c10c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:20.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.752+0000 7f1043600640 1 -- 172.21.15.47:0/4042123399 shutdown_connections 2024-03-21T11:32:20.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:20.752+0000 7f1043600640 1 -- 172.21.15.47:0/4042123399 wait complete. 2024-03-21T11:32:20.771 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:32:20.771 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:32:21.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.008+0000 7f3d00800640 1 -- 172.21.15.47:0/1787520657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf81005a0 msgr2=0x7f3cf81009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.008+0000 7f3d00800640 1 --2- 172.21.15.47:0/1787520657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf81005a0 0x7f3cf81009c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009860 tx=0x7f3cec02bb90 comp rx=0 tx=0).stop 2024-03-21T11:32:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.008+0000 7f3d00800640 1 -- 172.21.15.47:0/1787520657 shutdown_connections 2024-03-21T11:32:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.008+0000 7f3d00800640 1 --2- 172.21.15.47:0/1787520657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cf8102150 0x7f3cf81089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.008+0000 7f3d00800640 1 --2- 172.21.15.47:0/1787520657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf8101790 0x7f3cf8101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.008+0000 7f3d00800640 1 --2- 172.21.15.47:0/1787520657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf81005a0 0x7f3cf81009c0 secure :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009860 tx=0x7f3cec02bb90 comp rx=0 tx=0).stop 2024-03-21T11:32:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.008+0000 7f3d00800640 1 -- 172.21.15.47:0/1787520657 >> 172.21.15.47:0/1787520657 conn(0x7f3cf80fbb30 msgr2=0x7f3cf80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.009+0000 7f3d00800640 1 -- 172.21.15.47:0/1787520657 shutdown_connections 2024-03-21T11:32:21.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.009+0000 7f3d00800640 1 -- 172.21.15.47:0/1787520657 wait complete. 2024-03-21T11:32:21.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.010+0000 7f3d00800640 1 Processor -- start 2024-03-21T11:32:21.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.011+0000 7f3d00800640 1 -- start start 2024-03-21T11:32:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.011+0000 7f3d00800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cf8101790 0x7f3cf819e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.011+0000 7f3d00800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf8102150 0x7f3cf819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.011+0000 7f3d00800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf81a37e0 0x7f3cf81a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.011+0000 7f3d00800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf810b120 con 0x7f3cf8101790 2024-03-21T11:32:21.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.011+0000 7f3d00800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf810afa0 con 0x7f3cf8102150 2024-03-21T11:32:21.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.011+0000 7f3d00800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf810b2a0 con 0x7f3cf81a37e0 2024-03-21T11:32:21.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.012+0000 7f3cfe800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf81a37e0 0x7f3cf81a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.012+0000 7f3cfde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cf8101790 0x7f3cf819e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.012+0000 7f3cfe800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf81a37e0 0x7f3cf81a5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52934/0 (socket says 172.21.15.47:52934) 2024-03-21T11:32:21.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.012+0000 7f3cfe800640 1 -- 172.21.15.47:0/108388399 learned_addr learned my addr 172.21.15.47:0/108388399 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.012+0000 7f3cfd400640 1 --2- 172.21.15.47:0/108388399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf8102150 0x7f3cf819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.013+0000 7f3cfde00640 1 -- 172.21.15.47:0/108388399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf81a37e0 msgr2=0x7f3cf81a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:21.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.013+0000 7f3cfde00640 1 --2- 172.21.15.47:0/108388399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf81a37e0 0x7f3cf81a5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.013+0000 7f3cfde00640 1 -- 172.21.15.47:0/108388399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf8102150 msgr2=0x7f3cf819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.013+0000 7f3cfde00640 1 --2- 172.21.15.47:0/108388399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf8102150 0x7f3cf819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.013+0000 7f3cfde00640 1 -- 172.21.15.47:0/108388399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf81a6240 con 0x7f3cf8101790 2024-03-21T11:32:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.013+0000 7f3cfde00640 1 --2- 172.21.15.47:0/108388399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cf8101790 0x7f3cf819e240 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3cf81000e0 tx=0x7f3cec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.015+0000 7f3cf2a00640 1 -- 172.21.15.47:0/108388399 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec041020 con 0x7f3cf8101790 2024-03-21T11:32:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.015+0000 7f3cf2a00640 1 -- 172.21.15.47:0/108388399 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cec0035f0 con 0x7f3cf8101790 2024-03-21T11:32:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.015+0000 7f3cf2a00640 1 -- 172.21.15.47:0/108388399 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec039040 con 0x7f3cf8101790 2024-03-21T11:32:21.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.015+0000 7f3d00800640 1 -- 172.21.15.47:0/108388399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf81a64d0 con 0x7f3cf8101790 2024-03-21T11:32:21.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.015+0000 7f3d00800640 1 -- 172.21.15.47:0/108388399 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf81a6940 con 0x7f3cf8101790 2024-03-21T11:32:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.020+0000 7f3d03423740 1 Processor -- start 2024-03-21T11:32:21.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.020+0000 7f3d03423740 1 -- start start 2024-03-21T11:32:21.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.021+0000 7f3d03423740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 0x55e15e59dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.021+0000 7f3d03423740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59e9e0 0x55e15e59ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.021+0000 7f3d03423740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59f3a0 0x55e15e5a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.021+0000 7f3d03423740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e15e5022c0 con 0x55e15e59d810 2024-03-21T11:32:21.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.021+0000 7f3d03423740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e15e4aac90 con 0x55e15e59e9e0 2024-03-21T11:32:21.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.021+0000 7f3d03423740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e15e4fe360 con 0x55e15e59f3a0 2024-03-21T11:32:21.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.022+0000 7f3cfd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59f3a0 0x55e15e5a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.022+0000 7f3cfd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59f3a0 0x55e15e5a5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52942/0 (socket says 172.21.15.47:52942) 2024-03-21T11:32:21.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.022+0000 7f3cfd400640 1 -- 172.21.15.47:0/4206030159 learned_addr learned my addr 172.21.15.47:0/4206030159 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:21.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.022+0000 7f3cfe800640 1 --2- 172.21.15.47:0/4206030159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 0x55e15e59dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.022+0000 7f3cf2a00640 1 -- 172.21.15.47:0/108388399 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3cec044020 con 0x7f3cf8101790 2024-03-21T11:32:21.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.022+0000 7f3cfd400640 1 -- 172.21.15.47:0/4206030159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59e9e0 msgr2=0x55e15e59ee60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:32:21.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.023+0000 7f3cfd400640 1 --2- 172.21.15.47:0/4206030159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59e9e0 0x55e15e59ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.023+0000 7f3cfd400640 1 -- 172.21.15.47:0/4206030159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 msgr2=0x55e15e59dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:21.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.023+0000 7f3cfd400640 1 --2- 172.21.15.47:0/4206030159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 0x55e15e59dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.023+0000 7f3cfd400640 1 -- 172.21.15.47:0/4206030159 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e15e4cf080 con 0x55e15e59f3a0 2024-03-21T11:32:21.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.023+0000 7f3cfe800640 1 --2- 172.21.15.47:0/4206030159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 0x55e15e59dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:21.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.024+0000 7f3cfd400640 1 --2- 172.21.15.47:0/4206030159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59f3a0 0x55e15e5a5c30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009c90 tx=0x7f3ce8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e190035fe561222 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:21.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.025+0000 7f3ce7000640 1 -- 172.21.15.47:0/4206030159 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce801e070 con 0x55e15e59f3a0 2024-03-21T11:32:21.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.025+0000 7f3ce7000640 1 -- 172.21.15.47:0/4206030159 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ce8019b10 con 0x55e15e59f3a0 2024-03-21T11:32:21.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.025+0000 7f3ce7000640 1 -- 172.21.15.47:0/4206030159 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800eac0 con 0x55e15e59f3a0 2024-03-21T11:32:21.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.026+0000 7f3cf2a00640 1 --2- 172.21.15.47:0/108388399 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ce00a2fa0 0x7f3ce00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.026+0000 7f3cf2a00640 1 -- 172.21.15.47:0/108388399 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3cec04c060 con 0x7f3cf8101790 2024-03-21T11:32:21.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.026+0000 7f3d03423740 1 -- 172.21.15.47:0/4206030159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59f3a0 msgr2=0x55e15e5a5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:21.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.026+0000 7f3d03423740 1 --2- 172.21.15.47:0/4206030159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59f3a0 0x55e15e5a5c30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009c90 tx=0x7f3ce8011810 comp rx=0 tx=0).stop 2024-03-21T11:32:21.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.026+0000 7f3cfe800640 1 --2- 172.21.15.47:0/108388399 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ce00a2fa0 0x7f3ce00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.027+0000 7f3d03423740 1 -- 172.21.15.47:0/4206030159 shutdown_connections 2024-03-21T11:32:21.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.027+0000 7f3d03423740 1 --2- 172.21.15.47:0/4206030159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59f3a0 0x55e15e5a5c30 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.027+0000 7f3d03423740 1 --2- 172.21.15.47:0/4206030159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59e9e0 0x55e15e59ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.027+0000 7f3d03423740 1 --2- 172.21.15.47:0/4206030159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 0x55e15e59dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.027+0000 7f3d03423740 1 -- 172.21.15.47:0/4206030159 >> 172.21.15.47:0/4206030159 conn(0x55e15e598c00 msgr2=0x55e15e599000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:21.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.027+0000 7f3d03423740 1 -- 172.21.15.47:0/4206030159 shutdown_connections 2024-03-21T11:32:21.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.028+0000 7f3d03423740 1 -- 172.21.15.47:0/4206030159 wait complete. 2024-03-21T11:32:21.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.028+0000 7f3d03423740 1 -- start start 2024-03-21T11:32:21.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.028+0000 7f3cfe800640 1 --2- 172.21.15.47:0/108388399 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ce00a2fa0 0x7f3ce00a5460 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55e15e59e8e0 tx=0x7f3cf400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:21.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.028+0000 7f3d03423740 1 Processor -- start 2024-03-21T11:32:21.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.029+0000 7f3d03423740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 0x55e15e5a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.029+0000 7f3d03423740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59e9e0 0x55e15e5a8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.029+0000 7f3d03423740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59f3a0 0x55e15e5af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.029+0000 7f3d03423740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e15e5a63b0 con 0x55e15e59d810 2024-03-21T11:32:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.029+0000 7f3d03423740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e15e4be0d0 con 0x55e15e59f3a0 2024-03-21T11:32:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.029+0000 7f3d03423740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e15e5a6530 con 0x55e15e59e9e0 2024-03-21T11:32:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.029+0000 7f3d03423740 10 client.0 _ll_register_callbacks cb 0x55e15e4c2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.030+0000 7f3cfde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 0x55e15e5a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.030+0000 7f3cfd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59e9e0 0x55e15e5a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.030+0000 7f3cfd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59e9e0 0x55e15e5a8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52946/0 (socket says 172.21.15.47:52946) 2024-03-21T11:32:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.030+0000 7f3cfd400640 1 -- 172.21.15.47:0/925137130 learned_addr learned my addr 172.21.15.47:0/925137130 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.030+0000 7f3cfe800640 1 --2- 172.21.15.47:0/925137130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59f3a0 0x55e15e5af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.031+0000 7f3cfd400640 1 -- 172.21.15.47:0/925137130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59f3a0 msgr2=0x55e15e5af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.031+0000 7f3cfd400640 1 --2- 172.21.15.47:0/925137130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59f3a0 0x55e15e5af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.031+0000 7f3cfd400640 1 -- 172.21.15.47:0/925137130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 msgr2=0x55e15e5a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.031+0000 7f3cfd400640 1 --2- 172.21.15.47:0/925137130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 0x55e15e5a7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.031+0000 7f3cfd400640 1 -- 172.21.15.47:0/925137130 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e15e2163f0 con 0x55e15e59e9e0 2024-03-21T11:32:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.032+0000 7f3cfe800640 1 --2- 172.21.15.47:0/925137130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59f3a0 0x55e15e5af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.032+0000 7f3cfd400640 1 --2- 172.21.15.47:0/925137130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59e9e0 0x55e15e5a8080 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8019a20 tx=0x7f3ce8019a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11ff47cad587f5cb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.033+0000 7f3cdb400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.033+0000 7f3cdb400640 1 -- 172.21.15.47:0/925137130 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce801e070 con 0x55e15e59e9e0 2024-03-21T11:32:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.033+0000 7f3cdb400640 1 -- 172.21.15.47:0/925137130 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3ce8004230 con 0x55e15e59e9e0 2024-03-21T11:32:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.033+0000 7f3d03423740 1 -- 172.21.15.47:0/925137130 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ce8019b10 con 0x55e15e59e9e0 2024-03-21T11:32:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.033+0000 7f3cdb400640 1 -- 172.21.15.47:0/925137130 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ce8033050 con 0x55e15e59e9e0 2024-03-21T11:32:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.034+0000 7f3d03423740 10 client.28567 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:21.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.034+0000 7f3d03423740 1 -- 172.21.15.47:0/925137130 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e15e2e07d0 con 0x55e15e59e9e0 2024-03-21T11:32:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.035+0000 7f3cdb400640 1 -- 172.21.15.47:0/925137130 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ce8019b10 con 0x55e15e59e9e0 2024-03-21T11:32:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.037+0000 7f3cdb400640 1 -- 172.21.15.47:0/925137130 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3ce800ea00 con 0x55e15e59e9e0 2024-03-21T11:32:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.037+0000 7f3d03423740 10 client.28567 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.037+0000 7f3d03423740 10 client.28567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.037+0000 7f3d03423740 20 client.28567 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.037+0000 7f3d03423740 1 --2- 172.21.15.47:0/925137130 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e15e5b3920 0x55e15e5b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.037+0000 7f3d03423740 4 client.28567 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.037+0000 7f3d03423740 1 -- 172.21.15.47:0/925137130 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e15e50f800 con 0x55e15e5b3920 2024-03-21T11:32:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.037+0000 7f3cfde00640 1 --2- 172.21.15.47:0/925137130 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e15e5b3920 0x55e15e5b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.039+0000 7f3cfde00640 1 --2- 172.21.15.47:0/925137130 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e15e5b3920 0x55e15e5b5d40 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7278fcb100495bd4 server_cookie=38c025fe3afcf494 in_seq=0 out_seq=0 2024-03-21T11:32:21.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.039+0000 7f3cdb400640 10 client.28567 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:21.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.040+0000 7f3cdb400640 1 -- 172.21.15.47:0/925137130 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e15e50f800 con 0x55e15e5b3920 2024-03-21T11:32:21.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.041+0000 7f3cdb400640 10 client.28567 handle_command_reply: tid=0 2024-03-21T11:32:21.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.041+0000 7f3d03423740 1 client.28567 shutdown 2024-03-21T11:32:21.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.041+0000 7f3d03423740 1 -- 172.21.15.47:0/925137130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59e9e0 msgr2=0x55e15e5a8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:21.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.041+0000 7f3d03423740 1 --2- 172.21.15.47:0/925137130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59e9e0 0x55e15e5a8080 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8019a20 tx=0x7f3ce8019a50 comp rx=0 tx=0).stop 2024-03-21T11:32:21.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.042+0000 7f3d03423740 1 -- 172.21.15.47:0/925137130 shutdown_connections 2024-03-21T11:32:21.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.042+0000 7f3d03423740 1 --2- 172.21.15.47:0/925137130 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e15e5b3920 0x55e15e5b5d40 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.042+0000 7f3d03423740 1 --2- 172.21.15.47:0/925137130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59f3a0 0x55e15e5af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.042+0000 7f3d03423740 1 --2- 172.21.15.47:0/925137130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59e9e0 0x55e15e5a8080 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.042+0000 7f3d03423740 1 --2- 172.21.15.47:0/925137130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 0x55e15e5a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.042+0000 7f3cdb400640 0 client.28567 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.042+0000 7f3d03423740 1 -- 172.21.15.47:0/925137130 >> 172.21.15.47:0/925137130 conn(0x55e15e598c00 msgr2=0x55e15e597eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.042+0000 7f3d03423740 1 -- 172.21.15.47:0/925137130 shutdown_connections 2024-03-21T11:32:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.042+0000 7f3d03423740 1 -- 172.21.15.47:0/925137130 wait complete. 2024-03-21T11:32:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.042+0000 7f3d03423740 20 client.28567 trim_cache size 0 max 16384 2024-03-21T11:32:21.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.060+0000 7f3d03423740 1 Processor -- start 2024-03-21T11:32:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.060+0000 7f3d03423740 1 -- start start 2024-03-21T11:32:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.060+0000 7f3d03423740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59d810 0x55e15e597620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.060+0000 7f3d03423740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59e9e0 0x55e15e5af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.060+0000 7f3d03423740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59f3a0 0x55e15e597b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.060+0000 7f3d03423740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf81b0d80 con 0x55e15e59e9e0 2024-03-21T11:32:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.060+0000 7f3d03423740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf81b2700 con 0x55e15e59d810 2024-03-21T11:32:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.060+0000 7f3d03423740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf81b13e0 con 0x55e15e59f3a0 2024-03-21T11:32:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.061+0000 7f3cfd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59e9e0 0x55e15e5af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.061+0000 7f3cfde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59d810 0x55e15e597620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.061+0000 7f3cfe800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59f3a0 0x55e15e597b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.061+0000 7f3cfde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59d810 0x55e15e597620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34726/0 (socket says 172.21.15.47:34726) 2024-03-21T11:32:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.061+0000 7f3cfe800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59f3a0 0x55e15e597b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52950/0 (socket says 172.21.15.47:52950) 2024-03-21T11:32:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.061+0000 7f3cfd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59e9e0 0x55e15e5af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37986/0 (socket says 172.21.15.47:37986) 2024-03-21T11:32:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.061+0000 7f3cfde00640 1 -- 172.21.15.47:0/3028406642 learned_addr learned my addr 172.21.15.47:0/3028406642 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.061+0000 7f3cfd400640 1 -- 172.21.15.47:0/3028406642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59f3a0 msgr2=0x55e15e597b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.061+0000 7f3cfd400640 1 --2- 172.21.15.47:0/3028406642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59f3a0 0x55e15e597b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.061+0000 7f3cfd400640 1 -- 172.21.15.47:0/3028406642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59d810 msgr2=0x55e15e597620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.061+0000 7f3cfd400640 1 --2- 172.21.15.47:0/3028406642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59d810 0x55e15e597620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.061+0000 7f3cfd400640 1 -- 172.21.15.47:0/3028406642 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e15e599970 con 0x55e15e59e9e0 2024-03-21T11:32:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.061+0000 7f3cfde00640 1 --2- 172.21.15.47:0/3028406642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59d810 0x55e15e597620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.061+0000 7f3cfd400640 1 --2- 172.21.15.47:0/3028406642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59e9e0 0x55e15e5af080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55e15e599b40 tx=0x7f3ce8004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e982a9f15ce0cc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.062+0000 7f3ce6600640 1 -- 172.21.15.47:0/3028406642 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce801e070 con 0x55e15e59e9e0 2024-03-21T11:32:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.062+0000 7f3ce6600640 1 -- 172.21.15.47:0/3028406642 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ce801f040 con 0x55e15e59e9e0 2024-03-21T11:32:21.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.062+0000 7f3ce6600640 1 -- 172.21.15.47:0/3028406642 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8005280 con 0x55e15e59e9e0 2024-03-21T11:32:21.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.062+0000 7f3d03423740 1 -- 172.21.15.47:0/3028406642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59e9e0 msgr2=0x55e15e5af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:21.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.062+0000 7f3d03423740 1 --2- 172.21.15.47:0/3028406642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59e9e0 0x55e15e5af080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55e15e599b40 tx=0x7f3ce8004830 comp rx=0 tx=0).stop 2024-03-21T11:32:21.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.062+0000 7f3d03423740 1 -- 172.21.15.47:0/3028406642 shutdown_connections 2024-03-21T11:32:21.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.062+0000 7f3d03423740 1 --2- 172.21.15.47:0/3028406642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59f3a0 0x55e15e597b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.062+0000 7f3d03423740 1 --2- 172.21.15.47:0/3028406642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59e9e0 0x55e15e5af080 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.062+0000 7f3d03423740 1 --2- 172.21.15.47:0/3028406642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59d810 0x55e15e597620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.063+0000 7f3d03423740 1 -- 172.21.15.47:0/3028406642 >> 172.21.15.47:0/3028406642 conn(0x55e15e598c00 msgr2=0x55e15e598fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:21.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.063+0000 7f3d03423740 1 -- 172.21.15.47:0/3028406642 shutdown_connections 2024-03-21T11:32:21.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.063+0000 7f3d03423740 1 -- 172.21.15.47:0/3028406642 wait complete. 2024-03-21T11:32:21.068 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:21 smithi047 ceph-mon[31351]: pgmap v670: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-03-21T11:32:21.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.063+0000 7f3d03423740 1 -- start start 2024-03-21T11:32:21.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.063+0000 7f3d03423740 1 Processor -- start 2024-03-21T11:32:21.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.064+0000 7f3d03423740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 0x55e15e5aa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.064+0000 7f3d03423740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59e9e0 0x55e15e5aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.064+0000 7f3d03423740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59f3a0 0x55e15e5ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.064+0000 7f3d03423740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf81b20a0 con 0x55e15e59d810 2024-03-21T11:32:21.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.064+0000 7f3d03423740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf81b1a40 con 0x55e15e59f3a0 2024-03-21T11:32:21.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.064+0000 7f3d03423740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf81aef40 con 0x55e15e59e9e0 2024-03-21T11:32:21.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.064+0000 7f3d03423740 10 client.0 _ll_register_callbacks cb 0x7f3ce801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:21.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.064+0000 7f3cfd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59e9e0 0x55e15e5aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.064+0000 7f3cfde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 0x55e15e5aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.064+0000 7f3cfd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59e9e0 0x55e15e5aa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52960/0 (socket says 172.21.15.47:52960) 2024-03-21T11:32:21.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.064+0000 7f3cfd400640 1 -- 172.21.15.47:0/283272022 learned_addr learned my addr 172.21.15.47:0/283272022 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:21.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.064+0000 7f3cfe800640 1 --2- 172.21.15.47:0/283272022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59f3a0 0x55e15e5ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.064+0000 7f3cfd400640 1 -- 172.21.15.47:0/283272022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59f3a0 msgr2=0x55e15e5ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:21.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.064+0000 7f3cfd400640 1 --2- 172.21.15.47:0/283272022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59f3a0 0x55e15e5ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.065+0000 7f3cfd400640 1 -- 172.21.15.47:0/283272022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 msgr2=0x55e15e5aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:21.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.065+0000 7f3cfd400640 1 --2- 172.21.15.47:0/283272022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 0x55e15e5aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.065+0000 7f3cfd400640 1 -- 172.21.15.47:0/283272022 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e15e5a6c00 con 0x55e15e59e9e0 2024-03-21T11:32:21.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.065+0000 7f3cfd400640 1 --2- 172.21.15.47:0/283272022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59e9e0 0x55e15e5aa9d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8013040 tx=0x7f3ce8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60379c20c4f96dd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:21.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.065+0000 7f3cdb400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:21.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.065+0000 7f3cdb400640 1 -- 172.21.15.47:0/283272022 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce803b020 con 0x55e15e59e9e0 2024-03-21T11:32:21.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.065+0000 7f3d03423740 1 -- 172.21.15.47:0/283272022 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ce801f040 con 0x55e15e59e9e0 2024-03-21T11:32:21.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.065+0000 7f3cdb400640 1 -- 172.21.15.47:0/283272022 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3ce80053d0 con 0x55e15e59e9e0 2024-03-21T11:32:21.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.065+0000 7f3cdb400640 1 -- 172.21.15.47:0/283272022 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ce8046050 con 0x55e15e59e9e0 2024-03-21T11:32:21.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.065+0000 7f3d03423740 10 client.28573 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:21.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.065+0000 7f3d03423740 1 -- 172.21.15.47:0/283272022 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e15e5af110 con 0x55e15e59e9e0 2024-03-21T11:32:21.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.066+0000 7f3cdb400640 1 -- 172.21.15.47:0/283272022 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ce801f040 con 0x55e15e59e9e0 2024-03-21T11:32:21.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.066+0000 7f3cdb400640 1 -- 172.21.15.47:0/283272022 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3ce800e780 con 0x55e15e59e9e0 2024-03-21T11:32:21.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.066+0000 7f3d03423740 10 client.28573 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:21.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.066+0000 7f3d03423740 10 client.28573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:21.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.066+0000 7f3d03423740 20 client.28573 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:21.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.066+0000 7f3d03423740 1 --2- 172.21.15.47:0/283272022 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e15e60f620 0x55e15e611a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:21.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.066+0000 7f3d03423740 4 client.28573 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:32:21.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.066+0000 7f3d03423740 1 -- 172.21.15.47:0/283272022 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e15e509b50 con 0x55e15e60f620 2024-03-21T11:32:21.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.066+0000 7f3cfde00640 1 --2- 172.21.15.47:0/283272022 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e15e60f620 0x55e15e611a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:21.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.067+0000 7f3cfde00640 1 --2- 172.21.15.47:0/283272022 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e15e60f620 0x55e15e611a40 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cb424915aca5bd6 server_cookie=eee08f5276446c83 in_seq=0 out_seq=0 2024-03-21T11:32:21.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.067+0000 7f3cdb400640 10 client.28573 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:21.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.067+0000 7f3cdb400640 1 -- 172.21.15.47:0/283272022 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e15e509b50 con 0x55e15e60f620 2024-03-21T11:32:21.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.067+0000 7f3cdb400640 10 client.28573 handle_command_reply: tid=0 2024-03-21T11:32:21.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.067+0000 7f3d03423740 1 client.28573 shutdown 2024-03-21T11:32:21.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.067+0000 7f3d03423740 1 -- 172.21.15.47:0/283272022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59e9e0 msgr2=0x55e15e5aa9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:21.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.067+0000 7f3d03423740 1 --2- 172.21.15.47:0/283272022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59e9e0 0x55e15e5aa9d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8013040 tx=0x7f3ce8000bc0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.067+0000 7f3d03423740 1 -- 172.21.15.47:0/283272022 shutdown_connections 2024-03-21T11:32:21.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.067+0000 7f3d03423740 1 --2- 172.21.15.47:0/283272022 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e15e60f620 0x55e15e611a40 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.067+0000 7f3d03423740 1 --2- 172.21.15.47:0/283272022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e15e59f3a0 0x55e15e5ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.067+0000 7f3d03423740 1 --2- 172.21.15.47:0/283272022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e15e59e9e0 0x55e15e5aa9d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.067+0000 7f3d03423740 1 --2- 172.21.15.47:0/283272022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e15e59d810 0x55e15e5aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.067+0000 7f3cdb400640 0 client.28573 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:21.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.067+0000 7f3d03423740 1 -- 172.21.15.47:0/283272022 >> 172.21.15.47:0/283272022 conn(0x55e15e598c00 msgr2=0x55e15e59be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:21.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.068+0000 7f3d03423740 1 -- 172.21.15.47:0/283272022 shutdown_connections 2024-03-21T11:32:21.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.068+0000 7f3d03423740 1 -- 172.21.15.47:0/283272022 wait complete. 2024-03-21T11:32:21.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.068+0000 7f3d03423740 20 client.28573 trim_cache size 0 max 16384 2024-03-21T11:32:21.076 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:32:21.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.068+0000 7f3d00800640 1 -- 172.21.15.47:0/108388399 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ce00a2fa0 msgr2=0x7f3ce00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:21.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.068+0000 7f3d00800640 1 --2- 172.21.15.47:0/108388399 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ce00a2fa0 0x7f3ce00a5460 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55e15e59e8e0 tx=0x7f3cf400d040 comp rx=0 tx=0).stop 2024-03-21T11:32:21.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.068+0000 7f3d00800640 1 -- 172.21.15.47:0/108388399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cf8101790 msgr2=0x7f3cf819e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:21.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.068+0000 7f3d00800640 1 --2- 172.21.15.47:0/108388399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cf8101790 0x7f3cf819e240 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3cf81000e0 tx=0x7f3cec0042f0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.068+0000 7f3d00800640 1 -- 172.21.15.47:0/108388399 shutdown_connections 2024-03-21T11:32:21.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.068+0000 7f3d00800640 1 --2- 172.21.15.47:0/108388399 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3ce00a2fa0 0x7f3ce00a5460 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.068+0000 7f3d00800640 1 --2- 172.21.15.47:0/108388399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf81a37e0 0x7f3cf81a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.068+0000 7f3d00800640 1 --2- 172.21.15.47:0/108388399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf8102150 0x7f3cf819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.068+0000 7f3d00800640 1 --2- 172.21.15.47:0/108388399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cf8101790 0x7f3cf819e240 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:21.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.068+0000 7f3d00800640 1 -- 172.21.15.47:0/108388399 >> 172.21.15.47:0/108388399 conn(0x7f3cf80fbb30 msgr2=0x7f3cf8104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:21.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.069+0000 7f3d00800640 1 -- 172.21.15.47:0/108388399 shutdown_connections 2024-03-21T11:32:21.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:21.069+0000 7f3d00800640 1 -- 172.21.15.47:0/108388399 wait complete. 2024-03-21T11:32:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:21 smithi100 ceph-mon[36245]: pgmap v670: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-03-21T11:32:21.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:21 smithi102 ceph-mon[36748]: pgmap v670: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-03-21T11:32:21.543 INFO:tasks.workunit.client.0.smithi047.stdout: 1 655937 8.19 MB/sec execute 305 sec latency 15.165 ms 2024-03-21T11:32:22.088 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:32:22.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:32:22.088 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:32:22.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.317+0000 7fcbf6c00640 1 -- 172.21.15.47:0/2071038592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf01037a0 msgr2=0x7fcbf0103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.317+0000 7fcbf6c00640 1 --2- 172.21.15.47:0/2071038592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf01037a0 0x7fcbf0103bc0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009900 tx=0x7fcbe402bc30 comp rx=0 tx=0).stop 2024-03-21T11:32:22.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.317+0000 7fcbf6c00640 1 -- 172.21.15.47:0/2071038592 shutdown_connections 2024-03-21T11:32:22.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.317+0000 7fcbf6c00640 1 --2- 172.21.15.47:0/2071038592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0105350 0x7fcbf010bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.317+0000 7fcbf6c00640 1 --2- 172.21.15.47:0/2071038592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf0104990 0x7fcbf0104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.317+0000 7fcbf6c00640 1 --2- 172.21.15.47:0/2071038592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf01037a0 0x7fcbf0103bc0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.317+0000 7fcbf6c00640 1 -- 172.21.15.47:0/2071038592 >> 172.21.15.47:0/2071038592 conn(0x7fcbf00fed10 msgr2=0x7fcbf0101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:22.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.317+0000 7fcbf6c00640 1 -- 172.21.15.47:0/2071038592 shutdown_connections 2024-03-21T11:32:22.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.317+0000 7fcbf6c00640 1 -- 172.21.15.47:0/2071038592 wait complete. 2024-03-21T11:32:22.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.318+0000 7fcbf6c00640 1 Processor -- start 2024-03-21T11:32:22.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.319+0000 7fcbf6c00640 1 -- start start 2024-03-21T11:32:22.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.319+0000 7fcbf6c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf01037a0 0x7fcbf0199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.319+0000 7fcbf6c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0104990 0x7fcbf019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.319+0000 7fcbf6c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf0105350 0x7fcbf01a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.319+0000 7fcbf6c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf010f440 con 0x7fcbf01037a0 2024-03-21T11:32:22.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.319+0000 7fcbf6c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf010f2c0 con 0x7fcbf0104990 2024-03-21T11:32:22.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.319+0000 7fcbf6c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf010f5c0 con 0x7fcbf0105350 2024-03-21T11:32:22.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.319+0000 7fcbf4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0104990 0x7fcbf019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.319+0000 7fcbf4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0104990 0x7fcbf019a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34754/0 (socket says 172.21.15.47:34754) 2024-03-21T11:32:22.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.320+0000 7fcbf6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf0105350 0x7fcbf01a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.320+0000 7fcbf4e00640 1 -- 172.21.15.47:0/2164735363 learned_addr learned my addr 172.21.15.47:0/2164735363 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:22.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.320+0000 7fcbf5800640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf01037a0 0x7fcbf0199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.320+0000 7fcbf4e00640 1 -- 172.21.15.47:0/2164735363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf0105350 msgr2=0x7fcbf01a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.320+0000 7fcbf4e00640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf0105350 0x7fcbf01a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.320+0000 7fcbf4e00640 1 -- 172.21.15.47:0/2164735363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf01037a0 msgr2=0x7fcbf0199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.320+0000 7fcbf4e00640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf01037a0 0x7fcbf0199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.320+0000 7fcbf4e00640 1 -- 172.21.15.47:0/2164735363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf01a1700 con 0x7fcbf0104990 2024-03-21T11:32:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.320+0000 7fcbf5800640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf01037a0 0x7fcbf0199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.320+0000 7fcbf6200640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf0105350 0x7fcbf01a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.320+0000 7fcbf4e00640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0104990 0x7fcbf019a020 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00e9f0 tx=0x7fcbdc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.321+0000 7fcbee000640 1 -- 172.21.15.47:0/2164735363 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc00ccc0 con 0x7fcbf0104990 2024-03-21T11:32:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.321+0000 7fcbee000640 1 -- 172.21.15.47:0/2164735363 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbdc010910 con 0x7fcbf0104990 2024-03-21T11:32:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.321+0000 7fcbf6c00640 1 -- 172.21.15.47:0/2164735363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf01a19f0 con 0x7fcbf0104990 2024-03-21T11:32:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.321+0000 7fcbee000640 1 -- 172.21.15.47:0/2164735363 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0188a0 con 0x7fcbf0104990 2024-03-21T11:32:22.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.321+0000 7fcbf6c00640 1 -- 172.21.15.47:0/2164735363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf01a1f40 con 0x7fcbf0104990 2024-03-21T11:32:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.325+0000 7fcbfaf91740 1 Processor -- start 2024-03-21T11:32:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.325+0000 7fcbfaf91740 1 -- start start 2024-03-21T11:32:22.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.325+0000 7fcbee000640 1 -- 172.21.15.47:0/2164735363 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcbdc010ac0 con 0x7fcbf0104990 2024-03-21T11:32:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.325+0000 7fcbfaf91740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fceac0 0x557667fceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.325+0000 7fcbfaf91740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fcf4b0 0x557667fd3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.325+0000 7fcbfaf91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fd4620 0x557667fd6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.325+0000 7fcbfaf91740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557667f2beb0 con 0x557667fceac0 2024-03-21T11:32:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.325+0000 7fcbfaf91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557667ed4a60 con 0x557667fd4620 2024-03-21T11:32:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.325+0000 7fcbfaf91740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557667f27f50 con 0x557667fcf4b0 2024-03-21T11:32:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.326+0000 7fcbf5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fcf4b0 0x557667fd3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.326+0000 7fcbf4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fd4620 0x557667fd6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.326+0000 7fcbf6200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fceac0 0x557667fceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.326+0000 7fcbf5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fcf4b0 0x557667fd3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52992/0 (socket says 172.21.15.47:52992) 2024-03-21T11:32:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.326+0000 7fcbf5800640 1 -- 172.21.15.47:0/1320432287 learned_addr learned my addr 172.21.15.47:0/1320432287 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:22.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.326+0000 7fcbf5800640 1 -- 172.21.15.47:0/1320432287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fd4620 msgr2=0x557667fd6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.326+0000 7fcbf5800640 1 --2- 172.21.15.47:0/1320432287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fd4620 0x557667fd6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.326+0000 7fcbf5800640 1 -- 172.21.15.47:0/1320432287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fceac0 msgr2=0x557667fceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.326+0000 7fcbf5800640 1 --2- 172.21.15.47:0/1320432287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fceac0 0x557667fceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.326+0000 7fcbf5800640 1 -- 172.21.15.47:0/1320432287 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557667d087e0 con 0x557667fcf4b0 2024-03-21T11:32:22.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.326+0000 7fcbf4e00640 1 --2- 172.21.15.47:0/1320432287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fd4620 0x557667fd6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:22.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.327+0000 7fcbf5800640 1 --2- 172.21.15.47:0/1320432287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fcf4b0 0x557667fd3fb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4000c00 tx=0x7fcbe40095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=444a385eb84df376 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:22.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.327+0000 7fcbee000640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd80a2f20 0x7fcbd80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.327+0000 7fcbee000640 1 -- 172.21.15.47:0/2164735363 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcbdc0cfca0 con 0x7fcbf0104990 2024-03-21T11:32:22.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.327+0000 7fcbe2a00640 1 -- 172.21.15.47:0/1320432287 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4044020 con 0x557667fcf4b0 2024-03-21T11:32:22.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.327+0000 7fcbe2a00640 1 -- 172.21.15.47:0/1320432287 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbe402cc40 con 0x557667fcf4b0 2024-03-21T11:32:22.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.327+0000 7fcbe2a00640 1 -- 172.21.15.47:0/1320432287 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4004920 con 0x557667fcf4b0 2024-03-21T11:32:22.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.327+0000 7fcbf6200640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd80a2f20 0x7fcbd80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.328+0000 7fcbfaf91740 1 -- 172.21.15.47:0/1320432287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fcf4b0 msgr2=0x557667fd3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.328+0000 7fcbf6200640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd80a2f20 0x7fcbd80a53e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x557667fcfd10 tx=0x7fcbe8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:22.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.328+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/1320432287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fcf4b0 0x557667fd3fb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4000c00 tx=0x7fcbe40095e0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.328+0000 7fcbfaf91740 1 -- 172.21.15.47:0/1320432287 shutdown_connections 2024-03-21T11:32:22.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.328+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/1320432287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fd4620 0x557667fd6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.328+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/1320432287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fcf4b0 0x557667fd3fb0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.328+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/1320432287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fceac0 0x557667fceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.328+0000 7fcbfaf91740 1 -- 172.21.15.47:0/1320432287 >> 172.21.15.47:0/1320432287 conn(0x557667fca030 msgr2=0x557667fca430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:22.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.328+0000 7fcbfaf91740 1 -- 172.21.15.47:0/1320432287 shutdown_connections 2024-03-21T11:32:22.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.328+0000 7fcbfaf91740 1 -- 172.21.15.47:0/1320432287 wait complete. 2024-03-21T11:32:22.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.329+0000 7fcbfaf91740 1 -- start start 2024-03-21T11:32:22.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.329+0000 7fcbfaf91740 1 Processor -- start 2024-03-21T11:32:22.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.329+0000 7fcbfaf91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fceac0 0x557667fd8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.329+0000 7fcbfaf91740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fcf4b0 0x557667fd9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.329+0000 7fcbfaf91740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 0x557667fe0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.329+0000 7fcbfaf91740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557667fd7570 con 0x557667fcf4b0 2024-03-21T11:32:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.329+0000 7fcbfaf91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557667ee7eb0 con 0x557667fceac0 2024-03-21T11:32:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.329+0000 7fcbfaf91740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557667fd76f0 con 0x557667fd4620 2024-03-21T11:32:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.329+0000 7fcbfaf91740 10 client.0 _ll_register_callbacks cb 0x557667edc150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.330+0000 7fcbf5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fceac0 0x557667fd8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.330+0000 7fcbf6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 0x557667fe0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.330+0000 7fcbf5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fceac0 0x557667fd8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34768/0 (socket says 172.21.15.47:34768) 2024-03-21T11:32:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.330+0000 7fcbf5800640 1 -- 172.21.15.47:0/213258798 learned_addr learned my addr 172.21.15.47:0/213258798 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.330+0000 7fcbf4e00640 1 --2- 172.21.15.47:0/213258798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fcf4b0 0x557667fd9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.330+0000 7fcbf5800640 1 -- 172.21.15.47:0/213258798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 msgr2=0x557667fe0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.330+0000 7fcbf5800640 1 --2- 172.21.15.47:0/213258798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 0x557667fe0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.330+0000 7fcbf5800640 1 -- 172.21.15.47:0/213258798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fcf4b0 msgr2=0x557667fd9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.330+0000 7fcbf5800640 1 --2- 172.21.15.47:0/213258798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fcf4b0 0x557667fd9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.330+0000 7fcbf5800640 1 -- 172.21.15.47:0/213258798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557667d68f70 con 0x557667fceac0 2024-03-21T11:32:22.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.331+0000 7fcbf5800640 1 --2- 172.21.15.47:0/213258798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fceac0 0x557667fd8cd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009900 tx=0x7fcbe402cdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d351ce95f27a0ac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:22.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.331+0000 7fcbd7400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:32:22.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.331+0000 7fcbd7400640 1 -- 172.21.15.47:0/213258798 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe404d020 con 0x557667fceac0 2024-03-21T11:32:22.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.331+0000 7fcbd7400640 1 -- 172.21.15.47:0/213258798 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcbe4003800 con 0x557667fceac0 2024-03-21T11:32:22.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.331+0000 7fcbfaf91740 1 -- 172.21.15.47:0/213258798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbe402cc40 con 0x557667fceac0 2024-03-21T11:32:22.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.331+0000 7fcbd7400640 1 -- 172.21.15.47:0/213258798 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbe4058050 con 0x557667fceac0 2024-03-21T11:32:22.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.331+0000 7fcbfaf91740 10 client.28577 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:22.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.331+0000 7fcbfaf91740 1 -- 172.21.15.47:0/213258798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557667c00170 con 0x557667fceac0 2024-03-21T11:32:22.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.332+0000 7fcbd7400640 1 -- 172.21.15.47:0/213258798 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbe402cc40 con 0x557667fceac0 2024-03-21T11:32:22.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.332+0000 7fcbd7400640 1 -- 172.21.15.47:0/213258798 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcbe4049450 con 0x557667fceac0 2024-03-21T11:32:22.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.332+0000 7fcbfaf91740 10 client.28577 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:22.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.332+0000 7fcbfaf91740 10 client.28577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:22.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.332+0000 7fcbfaf91740 20 client.28577 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:22.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.332+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/213258798 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557667fe4740 0x557667fe6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.332+0000 7fcbfaf91740 4 client.28577 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:22.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.332+0000 7fcbfaf91740 1 -- 172.21.15.47:0/213258798 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557667f393f0 con 0x557667fe4740 2024-03-21T11:32:22.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.332+0000 7fcbf4e00640 1 --2- 172.21.15.47:0/213258798 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557667fe4740 0x557667fe6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.333+0000 7fcbf4e00640 1 --2- 172.21.15.47:0/213258798 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557667fe4740 0x557667fe6b30 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e70cf079e8cfb067 server_cookie=e39cc73f054a90d2 in_seq=0 out_seq=0 2024-03-21T11:32:22.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.333+0000 7fcbd7400640 10 client.28577 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:22.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.335+0000 7fcbd7400640 1 -- 172.21.15.47:0/213258798 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fcbdc07ae50 con 0x557667fe4740 2024-03-21T11:32:22.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.335+0000 7fcbd7400640 10 client.28577 handle_command_reply: tid=0 2024-03-21T11:32:22.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.335+0000 7fcbfaf91740 1 client.28577 shutdown 2024-03-21T11:32:22.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.336+0000 7fcbfaf91740 1 -- 172.21.15.47:0/213258798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fceac0 msgr2=0x557667fd8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.336+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/213258798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fceac0 0x557667fd8cd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009900 tx=0x7fcbe402cdf0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.336+0000 7fcbfaf91740 1 -- 172.21.15.47:0/213258798 shutdown_connections 2024-03-21T11:32:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.336+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/213258798 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557667fe4740 0x557667fe6b30 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.336+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/213258798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 0x557667fe0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.336+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/213258798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fcf4b0 0x557667fd9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.336+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/213258798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fceac0 0x557667fd8cd0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.336+0000 7fcbd7400640 0 client.28577 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.336+0000 7fcbfaf91740 1 -- 172.21.15.47:0/213258798 >> 172.21.15.47:0/213258798 conn(0x557667fca030 msgr2=0x557667fc0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.336+0000 7fcbfaf91740 1 -- 172.21.15.47:0/213258798 shutdown_connections 2024-03-21T11:32:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.336+0000 7fcbfaf91740 1 -- 172.21.15.47:0/213258798 wait complete. 2024-03-21T11:32:22.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.336+0000 7fcbfaf91740 20 client.28577 trim_cache size 0 max 16384 2024-03-21T11:32:22.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.356+0000 7fcbfaf91740 1 Processor -- start 2024-03-21T11:32:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.356+0000 7fcbfaf91740 1 -- start start 2024-03-21T11:32:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.356+0000 7fcbfaf91740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fceac0 0x557667feb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.356+0000 7fcbfaf91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fcf4b0 0x557667feb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.356+0000 7fcbfaf91740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 0x557667febd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbfaf91740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01a7cf0 con 0x557667fceac0 2024-03-21T11:32:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbfaf91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01a9670 con 0x557667fcf4b0 2024-03-21T11:32:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbfaf91740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01a8350 con 0x557667fd4620 2024-03-21T11:32:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbf4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fcf4b0 0x557667feb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbf4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fcf4b0 0x557667feb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34778/0 (socket says 172.21.15.47:34778) 2024-03-21T11:32:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbf4e00640 1 -- 172.21.15.47:0/4237559930 learned_addr learned my addr 172.21.15.47:0/4237559930 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbf5800640 1 --2- 172.21.15.47:0/4237559930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fceac0 0x557667feb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbf6200640 1 --2- 172.21.15.47:0/4237559930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 0x557667febd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbf4e00640 1 -- 172.21.15.47:0/4237559930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 msgr2=0x557667febd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbf4e00640 1 --2- 172.21.15.47:0/4237559930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 0x557667febd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbf4e00640 1 -- 172.21.15.47:0/4237559930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fceac0 msgr2=0x557667feb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbf4e00640 1 --2- 172.21.15.47:0/4237559930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fceac0 0x557667feb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbf4e00640 1 -- 172.21.15.47:0/4237559930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557667fe70c0 con 0x557667fcf4b0 2024-03-21T11:32:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbf6200640 1 --2- 172.21.15.47:0/4237559930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 0x557667febd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.357+0000 7fcbf4e00640 1 --2- 172.21.15.47:0/4237559930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fcf4b0 0x557667feb820 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc010f50 tx=0x7fcbdc0cd280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8732bac30a3e1b97 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.358+0000 7fcbe2000640 1 -- 172.21.15.47:0/4237559930 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc09b940 con 0x557667fcf4b0 2024-03-21T11:32:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.358+0000 7fcbe2000640 1 -- 172.21.15.47:0/4237559930 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557667f393f0 con 0x557667fcf4b0 2024-03-21T11:32:22.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.358+0000 7fcbe2000640 1 -- 172.21.15.47:0/4237559930 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0cd4c0 con 0x557667fcf4b0 2024-03-21T11:32:22.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.359+0000 7fcbfaf91740 1 -- 172.21.15.47:0/4237559930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fcf4b0 msgr2=0x557667feb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.359+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/4237559930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fcf4b0 0x557667feb820 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc010f50 tx=0x7fcbdc0cd280 comp rx=0 tx=0).stop 2024-03-21T11:32:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.359+0000 7fcbfaf91740 1 -- 172.21.15.47:0/4237559930 shutdown_connections 2024-03-21T11:32:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.359+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/4237559930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 0x557667febd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.359+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/4237559930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fcf4b0 0x557667feb820 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.359+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/4237559930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fceac0 0x557667feb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.359+0000 7fcbfaf91740 1 -- 172.21.15.47:0/4237559930 >> 172.21.15.47:0/4237559930 conn(0x557667fca030 msgr2=0x557667fc0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.359+0000 7fcbfaf91740 1 -- 172.21.15.47:0/4237559930 shutdown_connections 2024-03-21T11:32:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.359+0000 7fcbfaf91740 1 -- 172.21.15.47:0/4237559930 wait complete. 2024-03-21T11:32:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.359+0000 7fcbfaf91740 1 -- start start 2024-03-21T11:32:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.360+0000 7fcbfaf91740 1 Processor -- start 2024-03-21T11:32:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.360+0000 7fcbfaf91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fceac0 0x557667fdf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.360+0000 7fcbfaf91740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fcf4b0 0x557667fddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.360+0000 7fcbfaf91740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 0x557667fde2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.360+0000 7fcbfaf91740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01a9010 con 0x557667fcf4b0 2024-03-21T11:32:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.360+0000 7fcbfaf91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01a89b0 con 0x557667fceac0 2024-03-21T11:32:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.360+0000 7fcbfaf91740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01a5eb0 con 0x557667fd4620 2024-03-21T11:32:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.360+0000 7fcbfaf91740 10 client.0 _ll_register_callbacks cb 0x7fcbe8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.360+0000 7fcbf5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fceac0 0x557667fdf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.360+0000 7fcbf5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fceac0 0x557667fdf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34790/0 (socket says 172.21.15.47:34790) 2024-03-21T11:32:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.360+0000 7fcbf5800640 1 -- 172.21.15.47:0/1968714020 learned_addr learned my addr 172.21.15.47:0/1968714020 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.360+0000 7fcbf4e00640 1 --2- 172.21.15.47:0/1968714020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fcf4b0 0x557667fddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.360+0000 7fcbf6200640 1 --2- 172.21.15.47:0/1968714020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 0x557667fde2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.361+0000 7fcbf5800640 1 -- 172.21.15.47:0/1968714020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 msgr2=0x557667fde2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.361+0000 7fcbf5800640 1 --2- 172.21.15.47:0/1968714020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 0x557667fde2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.361+0000 7fcbf5800640 1 -- 172.21.15.47:0/1968714020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fcf4b0 msgr2=0x557667fddda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.361+0000 7fcbf5800640 1 --2- 172.21.15.47:0/1968714020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fcf4b0 0x557667fddda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.361+0000 7fcbf5800640 1 -- 172.21.15.47:0/1968714020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557667fde980 con 0x557667fceac0 2024-03-21T11:32:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.361+0000 7fcbf5800640 1 --2- 172.21.15.47:0/1968714020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fceac0 0x557667fdf790 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4049820 tx=0x7fcbe4049680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2511e3325e6eebc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.361+0000 7fcbd7400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:32:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.361+0000 7fcbd7400640 1 -- 172.21.15.47:0/1968714020 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe404d040 con 0x557667fceac0 2024-03-21T11:32:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.361+0000 7fcbfaf91740 1 -- 172.21.15.47:0/1968714020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557667f393f0 con 0x557667fceac0 2024-03-21T11:32:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.361+0000 7fcbd7400640 1 -- 172.21.15.47:0/1968714020 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcbe4049a10 con 0x557667fceac0 2024-03-21T11:32:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.362+0000 7fcbd7400640 1 -- 172.21.15.47:0/1968714020 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbe4044040 con 0x557667fceac0 2024-03-21T11:32:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.362+0000 7fcbfaf91740 10 client.28583 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.362+0000 7fcbfaf91740 1 -- 172.21.15.47:0/1968714020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557668040da0 con 0x557667fceac0 2024-03-21T11:32:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.362+0000 7fcbd7400640 1 -- 172.21.15.47:0/1968714020 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557667f393f0 con 0x557667fceac0 2024-03-21T11:32:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.362+0000 7fcbd7400640 1 -- 172.21.15.47:0/1968714020 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcbe4040470 con 0x557667fceac0 2024-03-21T11:32:22.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.362+0000 7fcbfaf91740 10 client.28583 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.362+0000 7fcbfaf91740 10 client.28583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.362+0000 7fcbfaf91740 20 client.28583 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.362+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/1968714020 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557668041610 0x557668043a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.362+0000 7fcbfaf91740 4 client.28583 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:32:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.362+0000 7fcbfaf91740 1 -- 172.21.15.47:0/1968714020 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557667f33740 con 0x557668041610 2024-03-21T11:32:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.362+0000 7fcbf4e00640 1 --2- 172.21.15.47:0/1968714020 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557668041610 0x557668043a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.363+0000 7fcbf4e00640 1 --2- 172.21.15.47:0/1968714020 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557668041610 0x557668043a30 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6f04a5f7787baa8 server_cookie=13baa205b4dc3da0 in_seq=0 out_seq=0 2024-03-21T11:32:22.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.363+0000 7fcbd7400640 10 client.28583 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:22.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.365+0000 7fcbd7400640 1 -- 172.21.15.47:0/1968714020 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557667f33740 con 0x557668041610 2024-03-21T11:32:22.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.365+0000 7fcbd7400640 10 client.28583 handle_command_reply: tid=0 2024-03-21T11:32:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.365+0000 7fcbfaf91740 1 client.28583 shutdown 2024-03-21T11:32:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.365+0000 7fcbfaf91740 1 -- 172.21.15.47:0/1968714020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fceac0 msgr2=0x557667fdf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.365+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/1968714020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fceac0 0x557667fdf790 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4049820 tx=0x7fcbe4049680 comp rx=0 tx=0).stop 2024-03-21T11:32:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.365+0000 7fcbfaf91740 1 -- 172.21.15.47:0/1968714020 shutdown_connections 2024-03-21T11:32:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.365+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/1968714020 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557668041610 0x557668043a30 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.365+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/1968714020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557667fd4620 0x557667fde2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.365+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/1968714020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557667fcf4b0 0x557667fddda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.365+0000 7fcbfaf91740 1 --2- 172.21.15.47:0/1968714020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557667fceac0 0x557667fdf790 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.365+0000 7fcbfaf91740 1 -- 172.21.15.47:0/1968714020 >> 172.21.15.47:0/1968714020 conn(0x557667fca030 msgr2=0x557667feb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.365+0000 7fcbd7400640 0 client.28583 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:22.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.365+0000 7fcbfaf91740 1 -- 172.21.15.47:0/1968714020 shutdown_connections 2024-03-21T11:32:22.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.366+0000 7fcbfaf91740 1 -- 172.21.15.47:0/1968714020 wait complete. 2024-03-21T11:32:22.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.366+0000 7fcbfaf91740 20 client.28583 trim_cache size 0 max 16384 2024-03-21T11:32:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:32:22.370 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:32:22.370 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "cd1556ad-9a35-4f6f-89c9-45bf3018ec8e", 2024-03-21T11:32:22.370 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:32:22.370 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:32:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.366+0000 7fcbf6c00640 1 -- 172.21.15.47:0/2164735363 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd80a2f20 msgr2=0x7fcbd80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.366+0000 7fcbf6c00640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd80a2f20 0x7fcbd80a53e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x557667fcfd10 tx=0x7fcbe8009500 comp rx=0 tx=0).stop 2024-03-21T11:32:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.366+0000 7fcbf6c00640 1 -- 172.21.15.47:0/2164735363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0104990 msgr2=0x7fcbf019a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.366+0000 7fcbf6c00640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0104990 0x7fcbf019a020 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00e9f0 tx=0x7fcbdc00edf0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.367+0000 7fcbf6c00640 1 -- 172.21.15.47:0/2164735363 shutdown_connections 2024-03-21T11:32:22.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.367+0000 7fcbf6c00640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd80a2f20 0x7fcbd80a53e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.367+0000 7fcbf6c00640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf0105350 0x7fcbf01a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.367+0000 7fcbf6c00640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0104990 0x7fcbf019a020 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.367+0000 7fcbf6c00640 1 --2- 172.21.15.47:0/2164735363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf01037a0 0x7fcbf0199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.367+0000 7fcbf6c00640 1 -- 172.21.15.47:0/2164735363 >> 172.21.15.47:0/2164735363 conn(0x7fcbf00fed10 msgr2=0x7fcbf0107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:22.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.367+0000 7fcbf6c00640 1 -- 172.21.15.47:0/2164735363 shutdown_connections 2024-03-21T11:32:22.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.367+0000 7fcbf6c00640 1 -- 172.21.15.47:0/2164735363 wait complete. 2024-03-21T11:32:22.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:32:22.390 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:32:22.543 INFO:tasks.workunit.client.0.smithi047.stdout: 1 657392 8.20 MB/sec execute 306 sec latency 26.781 ms 2024-03-21T11:32:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.608+0000 7ff148c00640 1 -- 172.21.15.47:0/1515748043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1440772a0 msgr2=0x7ff144077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.608+0000 7ff148c00640 1 --2- 172.21.15.47:0/1515748043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1440772a0 0x7ff144077700 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009900 tx=0x7ff12c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:32:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.608+0000 7ff148c00640 1 -- 172.21.15.47:0/1515748043 shutdown_connections 2024-03-21T11:32:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.608+0000 7ff148c00640 1 --2- 172.21.15.47:0/1515748043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff144077c40 0x7ff1440784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.608+0000 7ff148c00640 1 --2- 172.21.15.47:0/1515748043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1440772a0 0x7ff144077700 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.608+0000 7ff148c00640 1 --2- 172.21.15.47:0/1515748043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144076060 0x7ff144076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.608+0000 7ff148c00640 1 -- 172.21.15.47:0/1515748043 >> 172.21.15.47:0/1515748043 conn(0x7ff1440fee50 msgr2=0x7ff144101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.609+0000 7ff148c00640 1 -- 172.21.15.47:0/1515748043 shutdown_connections 2024-03-21T11:32:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.609+0000 7ff148c00640 1 -- 172.21.15.47:0/1515748043 wait complete. 2024-03-21T11:32:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.610+0000 7ff148c00640 1 Processor -- start 2024-03-21T11:32:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.610+0000 7ff148c00640 1 -- start start 2024-03-21T11:32:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.610+0000 7ff148c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144076060 0x7ff14419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.610+0000 7ff148c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff144077c40 0x7ff14419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.610+0000 7ff148c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1441a3780 0x7ff1441a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.610+0000 7ff148c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1441139b0 con 0x7ff144077c40 2024-03-21T11:32:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.610+0000 7ff148c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff144113830 con 0x7ff144076060 2024-03-21T11:32:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.610+0000 7ff148c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff144113b30 con 0x7ff1441a3780 2024-03-21T11:32:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.611+0000 7ff141e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144076060 0x7ff14419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.611+0000 7ff141e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144076060 0x7ff14419e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34816/0 (socket says 172.21.15.47:34816) 2024-03-21T11:32:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.611+0000 7ff141e00640 1 -- 172.21.15.47:0/1903304957 learned_addr learned my addr 172.21.15.47:0/1903304957 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.611+0000 7ff142800640 1 --2- 172.21.15.47:0/1903304957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1441a3780 0x7ff1441a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.611+0000 7ff141400640 1 --2- 172.21.15.47:0/1903304957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff144077c40 0x7ff14419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.611+0000 7ff141e00640 1 -- 172.21.15.47:0/1903304957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1441a3780 msgr2=0x7ff1441a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.611+0000 7ff141e00640 1 --2- 172.21.15.47:0/1903304957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1441a3780 0x7ff1441a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.611+0000 7ff141e00640 1 -- 172.21.15.47:0/1903304957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff144077c40 msgr2=0x7ff14419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.611+0000 7ff141e00640 1 --2- 172.21.15.47:0/1903304957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff144077c40 0x7ff14419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.611+0000 7ff141e00640 1 -- 172.21.15.47:0/1903304957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1441a61e0 con 0x7ff144076060 2024-03-21T11:32:22.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.611+0000 7ff141e00640 1 --2- 172.21.15.47:0/1903304957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144076060 0x7ff14419e1e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff13400e9f0 tx=0x7ff13400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:22.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.612+0000 7ff13aa00640 1 -- 172.21.15.47:0/1903304957 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13400ccc0 con 0x7ff144076060 2024-03-21T11:32:22.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.612+0000 7ff148c00640 1 -- 172.21.15.47:0/1903304957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1441a64d0 con 0x7ff144076060 2024-03-21T11:32:22.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.612+0000 7ff13aa00640 1 -- 172.21.15.47:0/1903304957 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff134010910 con 0x7ff144076060 2024-03-21T11:32:22.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.612+0000 7ff13aa00640 1 -- 172.21.15.47:0/1903304957 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1340188a0 con 0x7ff144076060 2024-03-21T11:32:22.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.612+0000 7ff148c00640 1 -- 172.21.15.47:0/1903304957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1441a69f0 con 0x7ff144076060 2024-03-21T11:32:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.615+0000 7ff14b96d740 1 Processor -- start 2024-03-21T11:32:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.615+0000 7ff13aa00640 1 -- 172.21.15.47:0/1903304957 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff134010ac0 con 0x7ff144076060 2024-03-21T11:32:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.616+0000 7ff14b96d740 1 -- start start 2024-03-21T11:32:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.616+0000 7ff14b96d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a925e80 0x564d2a9262a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.616+0000 7ff14b96d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a926870 0x564d2a92b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.616+0000 7ff14b96d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a92b9e0 0x564d2a92de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.616+0000 7ff14b96d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2a88b250 con 0x564d2a92b9e0 2024-03-21T11:32:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.616+0000 7ff14b96d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2a833a30 con 0x564d2a925e80 2024-03-21T11:32:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.616+0000 7ff14b96d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2a8872f0 con 0x564d2a926870 2024-03-21T11:32:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.616+0000 7ff142800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a925e80 0x564d2a9262a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.616+0000 7ff142800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a925e80 0x564d2a9262a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34820/0 (socket says 172.21.15.47:34820) 2024-03-21T11:32:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.616+0000 7ff141e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a926870 0x564d2a92b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.616+0000 7ff142800640 1 -- 172.21.15.47:0/1201347533 learned_addr learned my addr 172.21.15.47:0/1201347533 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.616+0000 7ff141400640 1 --2- 172.21.15.47:0/1201347533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a92b9e0 0x564d2a92de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.617+0000 7ff142800640 1 -- 172.21.15.47:0/1201347533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a926870 msgr2=0x564d2a92b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.617+0000 7ff142800640 1 --2- 172.21.15.47:0/1201347533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a926870 0x564d2a92b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.617+0000 7ff142800640 1 -- 172.21.15.47:0/1201347533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a92b9e0 msgr2=0x564d2a92de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.617+0000 7ff142800640 1 --2- 172.21.15.47:0/1201347533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a92b9e0 0x564d2a92de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.617+0000 7ff142800640 1 -- 172.21.15.47:0/1201347533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d2a81e690 con 0x564d2a925e80 2024-03-21T11:32:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.617+0000 7ff141400640 1 --2- 172.21.15.47:0/1201347533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a92b9e0 0x564d2a92de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.617+0000 7ff142800640 1 --2- 172.21.15.47:0/1201347533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a925e80 0x564d2a9262a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff13c011440 tx=0x7ff13c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c792ff54e929aac6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.617+0000 7ff13aa00640 1 --2- 172.21.15.47:0/1903304957 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff1280a2f20 0x7ff1280a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.617+0000 7ff133000640 1 -- 172.21.15.47:0/1201347533 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c003c30 con 0x564d2a925e80 2024-03-21T11:32:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.617+0000 7ff133000640 1 -- 172.21.15.47:0/1201347533 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff13c003dd0 con 0x564d2a925e80 2024-03-21T11:32:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.617+0000 7ff133000640 1 -- 172.21.15.47:0/1201347533 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c00fb50 con 0x564d2a925e80 2024-03-21T11:32:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.618+0000 7ff13aa00640 1 -- 172.21.15.47:0/1903304957 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff1340cfb90 con 0x7ff144076060 2024-03-21T11:32:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.618+0000 7ff141e00640 1 --2- 172.21.15.47:0/1903304957 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff1280a2f20 0x7ff1280a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.618+0000 7ff14b96d740 1 -- 172.21.15.47:0/1201347533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a925e80 msgr2=0x564d2a9262a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.618+0000 7ff141e00640 1 --2- 172.21.15.47:0/1903304957 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff1280a2f20 0x7ff1280a53e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x564d2a926c70 tx=0x7ff1340d3660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.618+0000 7ff14b96d740 1 --2- 172.21.15.47:0/1201347533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a925e80 0x564d2a9262a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff13c011440 tx=0x7ff13c011900 comp rx=0 tx=0).stop 2024-03-21T11:32:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.618+0000 7ff14b96d740 1 -- 172.21.15.47:0/1201347533 shutdown_connections 2024-03-21T11:32:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.618+0000 7ff14b96d740 1 --2- 172.21.15.47:0/1201347533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a92b9e0 0x564d2a92de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.619+0000 7ff14b96d740 1 --2- 172.21.15.47:0/1201347533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a926870 0x564d2a92b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.619+0000 7ff14b96d740 1 --2- 172.21.15.47:0/1201347533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a925e80 0x564d2a9262a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.619+0000 7ff14b96d740 1 -- 172.21.15.47:0/1201347533 >> 172.21.15.47:0/1201347533 conn(0x564d2a8ee200 msgr2=0x564d2a9217f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.619+0000 7ff14b96d740 1 -- 172.21.15.47:0/1201347533 shutdown_connections 2024-03-21T11:32:22.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.619+0000 7ff14b96d740 1 -- 172.21.15.47:0/1201347533 wait complete. 2024-03-21T11:32:22.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.619+0000 7ff14b96d740 1 -- start start 2024-03-21T11:32:22.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.619+0000 7ff14b96d740 1 Processor -- start 2024-03-21T11:32:22.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.620+0000 7ff14b96d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a925e80 0x564d2a92fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.620+0000 7ff14b96d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a926870 0x564d2a930390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.620+0000 7ff14b96d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a92b9e0 0x564d2a9373d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.620+0000 7ff14b96d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2a92e990 con 0x564d2a925e80 2024-03-21T11:32:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.620+0000 7ff14b96d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2a846e80 con 0x564d2a92b9e0 2024-03-21T11:32:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.620+0000 7ff14b96d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2a92eb10 con 0x564d2a926870 2024-03-21T11:32:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.620+0000 7ff14b96d740 10 client.0 _ll_register_callbacks cb 0x564d2a859130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.620+0000 7ff142800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a925e80 0x564d2a92fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.620+0000 7ff141400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a926870 0x564d2a930390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.620+0000 7ff141400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a926870 0x564d2a930390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53048/0 (socket says 172.21.15.47:53048) 2024-03-21T11:32:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.620+0000 7ff142800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a925e80 0x564d2a92fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38050/0 (socket says 172.21.15.47:38050) 2024-03-21T11:32:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.620+0000 7ff141400640 1 -- 172.21.15.47:0/396468626 learned_addr learned my addr 172.21.15.47:0/396468626 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.620+0000 7ff142800640 1 --2- 172.21.15.47:0/396468626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a92b9e0 0x564d2a9373d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.621+0000 7ff141400640 1 -- 172.21.15.47:0/396468626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a92b9e0 msgr2=0x564d2a9373d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.621+0000 7ff141400640 1 --2- 172.21.15.47:0/396468626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a92b9e0 0x564d2a9373d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.621+0000 7ff141400640 1 -- 172.21.15.47:0/396468626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a925e80 msgr2=0x564d2a92fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.621+0000 7ff141400640 1 --2- 172.21.15.47:0/396468626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a925e80 0x564d2a92fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.621+0000 7ff141400640 1 -- 172.21.15.47:0/396468626 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d2a6697d0 con 0x564d2a926870 2024-03-21T11:32:22.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.621+0000 7ff142800640 1 --2- 172.21.15.47:0/396468626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a925e80 0x564d2a92fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:22.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.621+0000 7ff141400640 1 --2- 172.21.15.47:0/396468626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a926870 0x564d2a930390 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff12c02c240 tx=0x7ff12c0042d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3da3b25e9888e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:22.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.621+0000 7ff127400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:22.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.621+0000 7ff127400640 1 -- 172.21.15.47:0/396468626 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c005d50 con 0x564d2a926870 2024-03-21T11:32:22.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.621+0000 7ff127400640 1 -- 172.21.15.47:0/396468626 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff12c005250 con 0x564d2a926870 2024-03-21T11:32:22.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.621+0000 7ff14b96d740 1 -- 172.21.15.47:0/396468626 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff13c003dd0 con 0x564d2a926870 2024-03-21T11:32:22.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.622+0000 7ff127400640 1 -- 172.21.15.47:0/396468626 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff12c0044c0 con 0x564d2a926870 2024-03-21T11:32:22.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.622+0000 7ff14b96d740 10 client.28603 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.622+0000 7ff14b96d740 1 -- 172.21.15.47:0/396468626 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564d2a59f3f0 con 0x564d2a926870 2024-03-21T11:32:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.622+0000 7ff127400640 1 -- 172.21.15.47:0/396468626 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff13c003dd0 con 0x564d2a926870 2024-03-21T11:32:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.622+0000 7ff127400640 1 -- 172.21.15.47:0/396468626 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff12c0035f0 con 0x564d2a926870 2024-03-21T11:32:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.623+0000 7ff14b96d740 10 client.28603 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.623+0000 7ff14b96d740 10 client.28603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.623+0000 7ff14b96d740 20 client.28603 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.623+0000 7ff14b96d740 1 --2- 172.21.15.47:0/396468626 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d2a93bc70 0x564d2a93e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.623+0000 7ff14b96d740 4 client.28603 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.623+0000 7ff14b96d740 1 -- 172.21.15.47:0/396468626 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d2a898790 con 0x564d2a93bc70 2024-03-21T11:32:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.623+0000 7ff142800640 1 --2- 172.21.15.47:0/396468626 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d2a93bc70 0x564d2a93e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.623+0000 7ff142800640 1 --2- 172.21.15.47:0/396468626 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d2a93bc70 0x564d2a93e090 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f4d8c731b4bd0c7 server_cookie=765a3c97e3190d95 in_seq=0 out_seq=0 2024-03-21T11:32:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.623+0000 7ff127400640 10 client.28603 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff127400640 1 -- 172.21.15.47:0/396468626 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564d2a898790 con 0x564d2a93bc70 2024-03-21T11:32:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff127400640 10 client.28603 handle_command_reply: tid=0 2024-03-21T11:32:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff14b96d740 1 client.28603 shutdown 2024-03-21T11:32:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff14b96d740 1 -- 172.21.15.47:0/396468626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a926870 msgr2=0x564d2a930390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff14b96d740 1 --2- 172.21.15.47:0/396468626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a926870 0x564d2a930390 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff12c02c240 tx=0x7ff12c0042d0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff14b96d740 1 -- 172.21.15.47:0/396468626 shutdown_connections 2024-03-21T11:32:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff14b96d740 1 --2- 172.21.15.47:0/396468626 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d2a93bc70 0x564d2a93e090 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff14b96d740 1 --2- 172.21.15.47:0/396468626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a92b9e0 0x564d2a9373d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff14b96d740 1 --2- 172.21.15.47:0/396468626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a926870 0x564d2a930390 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff14b96d740 1 --2- 172.21.15.47:0/396468626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a925e80 0x564d2a92fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff14b96d740 1 -- 172.21.15.47:0/396468626 >> 172.21.15.47:0/396468626 conn(0x564d2a8ee200 msgr2=0x564d2a920a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff127400640 0 client.28603 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff14b96d740 1 -- 172.21.15.47:0/396468626 shutdown_connections 2024-03-21T11:32:22.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff14b96d740 1 -- 172.21.15.47:0/396468626 wait complete. 2024-03-21T11:32:22.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.626+0000 7ff14b96d740 20 client.28603 trim_cache size 0 max 16384 2024-03-21T11:32:22.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.647+0000 7ff14b96d740 1 Processor -- start 2024-03-21T11:32:22.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.647+0000 7ff14b96d740 1 -- start start 2024-03-21T11:32:22.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff14b96d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a925e80 0x564d2a925140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff14b96d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a926870 0x564d2a937390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff14b96d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a92b9e0 0x564d2a942950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff14b96d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1441b0ce0 con 0x564d2a926870 2024-03-21T11:32:22.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff14b96d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1441b2660 con 0x564d2a92b9e0 2024-03-21T11:32:22.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff14b96d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1441b1340 con 0x564d2a925e80 2024-03-21T11:32:22.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff141400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a926870 0x564d2a937390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff142800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a925e80 0x564d2a925140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff142800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a925e80 0x564d2a925140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53064/0 (socket says 172.21.15.47:53064) 2024-03-21T11:32:22.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff141400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a926870 0x564d2a937390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38060/0 (socket says 172.21.15.47:38060) 2024-03-21T11:32:22.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff142800640 1 -- 172.21.15.47:0/2433843742 learned_addr learned my addr 172.21.15.47:0/2433843742 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:22.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff142800640 1 --2- 172.21.15.47:0/2433843742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a92b9e0 0x564d2a942950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff142800640 1 -- 172.21.15.47:0/2433843742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a92b9e0 msgr2=0x564d2a942950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff142800640 1 --2- 172.21.15.47:0/2433843742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a92b9e0 0x564d2a942950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff142800640 1 -- 172.21.15.47:0/2433843742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a926870 msgr2=0x564d2a937390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff142800640 1 --2- 172.21.15.47:0/2433843742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a926870 0x564d2a937390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff142800640 1 -- 172.21.15.47:0/2433843742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d2a55f0f0 con 0x564d2a925e80 2024-03-21T11:32:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.648+0000 7ff141400640 1 --2- 172.21.15.47:0/2433843742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a926870 0x564d2a937390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:22.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.649+0000 7ff142800640 1 --2- 172.21.15.47:0/2433843742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a925e80 0x564d2a925140 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009f80 tx=0x7ff13c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e5a48dbf9641191 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.649+0000 7ff132600640 1 -- 172.21.15.47:0/2433843742 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c033070 con 0x564d2a925e80 2024-03-21T11:32:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.649+0000 7ff132600640 1 -- 172.21.15.47:0/2433843742 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff13c004730 con 0x564d2a925e80 2024-03-21T11:32:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.649+0000 7ff132600640 1 -- 172.21.15.47:0/2433843742 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c00fd10 con 0x564d2a925e80 2024-03-21T11:32:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.650+0000 7ff14b96d740 1 -- 172.21.15.47:0/2433843742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a925e80 msgr2=0x564d2a925140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.650+0000 7ff14b96d740 1 --2- 172.21.15.47:0/2433843742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a925e80 0x564d2a925140 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009f80 tx=0x7ff13c000f40 comp rx=0 tx=0).stop 2024-03-21T11:32:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.650+0000 7ff14b96d740 1 -- 172.21.15.47:0/2433843742 shutdown_connections 2024-03-21T11:32:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.650+0000 7ff14b96d740 1 --2- 172.21.15.47:0/2433843742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a92b9e0 0x564d2a942950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.650+0000 7ff14b96d740 1 --2- 172.21.15.47:0/2433843742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a926870 0x564d2a937390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.650+0000 7ff14b96d740 1 --2- 172.21.15.47:0/2433843742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a925e80 0x564d2a925140 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.650+0000 7ff14b96d740 1 -- 172.21.15.47:0/2433843742 >> 172.21.15.47:0/2433843742 conn(0x564d2a8ee200 msgr2=0x564d2a924a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.650+0000 7ff14b96d740 1 -- 172.21.15.47:0/2433843742 shutdown_connections 2024-03-21T11:32:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.650+0000 7ff14b96d740 1 -- 172.21.15.47:0/2433843742 wait complete. 2024-03-21T11:32:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.651+0000 7ff14b96d740 1 -- start start 2024-03-21T11:32:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.651+0000 7ff14b96d740 1 Processor -- start 2024-03-21T11:32:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.651+0000 7ff14b96d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a925e80 0x564d2a937200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.651+0000 7ff14b96d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a926870 0x564d2a935880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff14b96d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a92b9e0 0x564d2a935dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff14b96d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1441b2000 con 0x564d2a926870 2024-03-21T11:32:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff14b96d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1441b19a0 con 0x564d2a925e80 2024-03-21T11:32:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff14b96d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1441aeea0 con 0x564d2a92b9e0 2024-03-21T11:32:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff142800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a925e80 0x564d2a937200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff141400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a926870 0x564d2a935880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff14b96d740 10 client.0 _ll_register_callbacks cb 0x7ff134005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff142800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a925e80 0x564d2a937200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34846/0 (socket says 172.21.15.47:34846) 2024-03-21T11:32:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff142800640 1 -- 172.21.15.47:0/3295564453 learned_addr learned my addr 172.21.15.47:0/3295564453 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff142800640 1 --2- 172.21.15.47:0/3295564453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a92b9e0 0x564d2a935dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff141400640 1 -- 172.21.15.47:0/3295564453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a92b9e0 msgr2=0x564d2a935dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff141400640 1 --2- 172.21.15.47:0/3295564453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a92b9e0 0x564d2a935dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff141400640 1 -- 172.21.15.47:0/3295564453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a925e80 msgr2=0x564d2a937200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff141400640 1 --2- 172.21.15.47:0/3295564453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a925e80 0x564d2a937200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff141400640 1 -- 172.21.15.47:0/3295564453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d2a936550 con 0x564d2a926870 2024-03-21T11:32:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.652+0000 7ff142800640 1 --2- 172.21.15.47:0/3295564453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a925e80 0x564d2a937200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.653+0000 7ff141400640 1 --2- 172.21.15.47:0/3295564453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a926870 0x564d2a935880 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff12c003a40 tx=0x7ff12c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4b2643c24e4abe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.653+0000 7ff127400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:32:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.653+0000 7ff127400640 1 -- 172.21.15.47:0/3295564453 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c005d50 con 0x564d2a926870 2024-03-21T11:32:22.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.653+0000 7ff14b96d740 1 -- 172.21.15.47:0/3295564453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff13c004730 con 0x564d2a926870 2024-03-21T11:32:22.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.653+0000 7ff127400640 1 -- 172.21.15.47:0/3295564453 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff12c005660 con 0x564d2a926870 2024-03-21T11:32:22.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.653+0000 7ff127400640 1 -- 172.21.15.47:0/3295564453 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff12c063050 con 0x564d2a926870 2024-03-21T11:32:22.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.653+0000 7ff14b96d740 10 client.18267 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:22.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.653+0000 7ff14b96d740 1 -- 172.21.15.47:0/3295564453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564d2a998570 con 0x564d2a926870 2024-03-21T11:32:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.654+0000 7ff127400640 1 -- 172.21.15.47:0/3295564453 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff13c004730 con 0x564d2a926870 2024-03-21T11:32:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.654+0000 7ff127400640 1 -- 172.21.15.47:0/3295564453 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff12c05e030 con 0x564d2a926870 2024-03-21T11:32:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.654+0000 7ff14b96d740 10 client.18267 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.654+0000 7ff14b96d740 10 client.18267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.654+0000 7ff14b96d740 20 client.18267 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.654+0000 7ff14b96d740 1 --2- 172.21.15.47:0/3295564453 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d2a998e60 0x564d2a99b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.654+0000 7ff14b96d740 4 client.18267 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:32:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.654+0000 7ff14b96d740 1 -- 172.21.15.47:0/3295564453 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564d2a892ae0 con 0x564d2a998e60 2024-03-21T11:32:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.654+0000 7ff142800640 1 --2- 172.21.15.47:0/3295564453 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d2a998e60 0x564d2a99b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.655+0000 7ff142800640 1 --2- 172.21.15.47:0/3295564453 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d2a998e60 0x564d2a99b250 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e84f7cded3e3b43e server_cookie=8cb35a987338a2b in_seq=0 out_seq=0 2024-03-21T11:32:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.655+0000 7ff127400640 10 client.18267 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.655+0000 7ff127400640 1 -- 172.21.15.47:0/3295564453 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564d2a892ae0 con 0x564d2a998e60 2024-03-21T11:32:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.655+0000 7ff127400640 10 client.18267 handle_command_reply: tid=0 2024-03-21T11:32:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.655+0000 7ff14b96d740 1 client.18267 shutdown 2024-03-21T11:32:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.655+0000 7ff14b96d740 1 -- 172.21.15.47:0/3295564453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a926870 msgr2=0x564d2a935880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.655+0000 7ff14b96d740 1 --2- 172.21.15.47:0/3295564453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a926870 0x564d2a935880 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff12c003a40 tx=0x7ff12c003820 comp rx=0 tx=0).stop 2024-03-21T11:32:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.655+0000 7ff14b96d740 1 -- 172.21.15.47:0/3295564453 shutdown_connections 2024-03-21T11:32:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.655+0000 7ff14b96d740 1 --2- 172.21.15.47:0/3295564453 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d2a998e60 0x564d2a99b250 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.655+0000 7ff14b96d740 1 --2- 172.21.15.47:0/3295564453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d2a92b9e0 0x564d2a935dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.655+0000 7ff14b96d740 1 --2- 172.21.15.47:0/3295564453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d2a926870 0x564d2a935880 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.655+0000 7ff14b96d740 1 --2- 172.21.15.47:0/3295564453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d2a925e80 0x564d2a937200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.655+0000 7ff127400640 0 client.18267 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.655+0000 7ff14b96d740 1 -- 172.21.15.47:0/3295564453 >> 172.21.15.47:0/3295564453 conn(0x564d2a8ee200 msgr2=0x564d2a9252c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.656+0000 7ff14b96d740 1 -- 172.21.15.47:0/3295564453 shutdown_connections 2024-03-21T11:32:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.656+0000 7ff14b96d740 1 -- 172.21.15.47:0/3295564453 wait complete. 2024-03-21T11:32:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.656+0000 7ff14b96d740 20 client.18267 trim_cache size 0 max 16384 2024-03-21T11:32:22.661 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:32:22.661 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:32:22.661 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:32:22.661 INFO:teuthology.orchestra.run.smithi047.stdout: "cd1556ad-9a35-4f6f-89c9-45bf3018ec8e": { 2024-03-21T11:32:22.661 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:32:22.661 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "cd1556ad-9a35-4f6f-89c9-45bf3018ec8e", 2024-03-21T11:32:22.661 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:32:22.661 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:32:22.661 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:32:22.661 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:32:22.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.656+0000 7ff148c00640 1 -- 172.21.15.47:0/1903304957 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff1280a2f20 msgr2=0x7ff1280a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.656+0000 7ff148c00640 1 --2- 172.21.15.47:0/1903304957 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff1280a2f20 0x7ff1280a53e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x564d2a926c70 tx=0x7ff1340d3660 comp rx=0 tx=0).stop 2024-03-21T11:32:22.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.656+0000 7ff148c00640 1 -- 172.21.15.47:0/1903304957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144076060 msgr2=0x7ff14419e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:22.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.656+0000 7ff148c00640 1 --2- 172.21.15.47:0/1903304957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144076060 0x7ff14419e1e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff13400e9f0 tx=0x7ff13400edf0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.657+0000 7ff148c00640 1 -- 172.21.15.47:0/1903304957 shutdown_connections 2024-03-21T11:32:22.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.657+0000 7ff148c00640 1 --2- 172.21.15.47:0/1903304957 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff1280a2f20 0x7ff1280a53e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.657+0000 7ff148c00640 1 --2- 172.21.15.47:0/1903304957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1441a3780 0x7ff1441a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.657+0000 7ff148c00640 1 --2- 172.21.15.47:0/1903304957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff144077c40 0x7ff14419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.657+0000 7ff148c00640 1 --2- 172.21.15.47:0/1903304957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff144076060 0x7ff14419e1e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:22.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.657+0000 7ff148c00640 1 -- 172.21.15.47:0/1903304957 >> 172.21.15.47:0/1903304957 conn(0x7ff1440fee50 msgr2=0x7ff1441005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:22.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.657+0000 7ff148c00640 1 -- 172.21.15.47:0/1903304957 shutdown_connections 2024-03-21T11:32:22.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:22.657+0000 7ff148c00640 1 -- 172.21.15.47:0/1903304957 wait complete. 2024-03-21T11:32:22.676 INFO:tasks.cephfs.filesystem:scrub status for tag:cd1556ad-9a35-4f6f-89c9-45bf3018ec8e - {'path': '/', 'tag': 'cd1556ad-9a35-4f6f-89c9-45bf3018ec8e', 'options': 'recursive,force'} 2024-03-21T11:32:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:23 smithi047 ceph-mon[31351]: pgmap v671: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-03-21T11:32:23.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:23 smithi100 ceph-mon[36245]: pgmap v671: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-03-21T11:32:23.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:23 smithi102 ceph-mon[36748]: pgmap v671: 129 pgs: 129 active+clean; 44 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-03-21T11:32:23.543 INFO:tasks.workunit.client.0.smithi047.stdout: 1 658452 8.19 MB/sec execute 307 sec latency 34.922 ms 2024-03-21T11:32:24.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:24 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:32:24.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:24 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:32:24.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:24 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:32:24.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:24 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:32:24.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:24 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:32:24.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:24 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:32:24.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:24 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:32:24.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:24 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:32:24.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:24 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:32:24.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:24 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:32:24.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:24 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:32:24.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:24 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:32:24.543 INFO:tasks.workunit.client.0.smithi047.stdout: 1 660282 8.19 MB/sec execute 308 sec latency 23.001 ms 2024-03-21T11:32:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:25 smithi047 ceph-mon[31351]: pgmap v672: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-03-21T11:32:25.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:25 smithi100 ceph-mon[36245]: pgmap v672: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-03-21T11:32:25.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:25 smithi102 ceph-mon[36748]: pgmap v672: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-03-21T11:32:25.544 INFO:tasks.workunit.client.0.smithi047.stdout: 1 661149 8.18 MB/sec execute 309 sec latency 86.248 ms 2024-03-21T11:32:26.544 INFO:tasks.workunit.client.0.smithi047.stdout: 1 662779 8.17 MB/sec execute 310 sec latency 13.801 ms 2024-03-21T11:32:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:27 smithi047 ceph-mon[31351]: pgmap v673: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-03-21T11:32:27.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:27 smithi100 ceph-mon[36245]: pgmap v673: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-03-21T11:32:27.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:27 smithi102 ceph-mon[36748]: pgmap v673: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-03-21T11:32:27.544 INFO:tasks.workunit.client.0.smithi047.stdout: 1 664285 8.19 MB/sec execute 311 sec latency 21.931 ms 2024-03-21T11:32:28.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:28 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:32:28.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:28 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:32:28.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:28 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:32:28.544 INFO:tasks.workunit.client.0.smithi047.stdout: 1 664863 8.16 MB/sec execute 312 sec latency 119.723 ms 2024-03-21T11:32:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:29 smithi047 ceph-mon[31351]: pgmap v674: 129 pgs: 129 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 315 op/s 2024-03-21T11:32:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:29 smithi100 ceph-mon[36245]: pgmap v674: 129 pgs: 129 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 315 op/s 2024-03-21T11:32:29.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:29 smithi102 ceph-mon[36748]: pgmap v674: 129 pgs: 129 active+clean; 40 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 315 op/s 2024-03-21T11:32:29.544 INFO:tasks.workunit.client.0.smithi047.stdout: 1 666474 8.16 MB/sec execute 313 sec latency 15.568 ms 2024-03-21T11:32:30.544 INFO:tasks.workunit.client.0.smithi047.stdout: 1 667882 8.17 MB/sec execute 314 sec latency 15.507 ms 2024-03-21T11:32:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:31 smithi047 ceph-mon[31351]: pgmap v675: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 315 op/s 2024-03-21T11:32:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:31 smithi100 ceph-mon[36245]: pgmap v675: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 315 op/s 2024-03-21T11:32:31.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:31 smithi102 ceph-mon[36748]: pgmap v675: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 315 op/s 2024-03-21T11:32:31.544 INFO:tasks.workunit.client.0.smithi047.stdout: 1 668758 8.15 MB/sec execute 315 sec latency 42.109 ms 2024-03-21T11:32:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.962+0000 7f6b07200640 1 -- 172.21.15.47:0/3199328648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b00104af0 msgr2=0x7f6b00104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.962+0000 7f6b07200640 1 --2- 172.21.15.47:0/3199328648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b00104af0 0x7f6b00104f70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009900 tx=0x7f6aec02bc30 comp rx=0 tx=0).stop 2024-03-21T11:32:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.962+0000 7f6b07200640 1 -- 172.21.15.47:0/3199328648 shutdown_connections 2024-03-21T11:32:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.962+0000 7f6b07200640 1 --2- 172.21.15.47:0/3199328648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b001054b0 0x7f6b0010bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.962+0000 7f6b07200640 1 --2- 172.21.15.47:0/3199328648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b00104af0 0x7f6b00104f70 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.962+0000 7f6b07200640 1 --2- 172.21.15.47:0/3199328648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b00103900 0x7f6b00103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.962+0000 7f6b07200640 1 -- 172.21.15.47:0/3199328648 >> 172.21.15.47:0/3199328648 conn(0x7f6b000fee90 msgr2=0x7f6b001012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.962+0000 7f6b07200640 1 -- 172.21.15.47:0/3199328648 shutdown_connections 2024-03-21T11:32:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.962+0000 7f6b07200640 1 -- 172.21.15.47:0/3199328648 wait complete. 2024-03-21T11:32:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.963+0000 7f6b07200640 1 Processor -- start 2024-03-21T11:32:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.963+0000 7f6b07200640 1 -- start start 2024-03-21T11:32:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6b07200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b00103900 0x7f6b00199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6b07200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b00104af0 0x7f6b0019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6b07200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b001054b0 0x7f6b001a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6b07200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0010f5d0 con 0x7f6b00104af0 2024-03-21T11:32:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6b07200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0010f450 con 0x7f6b00103900 2024-03-21T11:32:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6b07200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0010f750 con 0x7f6b001054b0 2024-03-21T11:32:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6affe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b00103900 0x7f6b00199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6affe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b00103900 0x7f6b00199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42826/0 (socket says 172.21.15.47:42826) 2024-03-21T11:32:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6affe00640 1 -- 172.21.15.47:0/3238686154 learned_addr learned my addr 172.21.15.47:0/3238686154 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6aff400640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b00104af0 0x7f6b0019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6b05200640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b001054b0 0x7f6b001a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6affe00640 1 -- 172.21.15.47:0/3238686154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b001054b0 msgr2=0x7f6b001a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6affe00640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b001054b0 0x7f6b001a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6affe00640 1 -- 172.21.15.47:0/3238686154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b00104af0 msgr2=0x7f6b0019a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6affe00640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b00104af0 0x7f6b0019a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6affe00640 1 -- 172.21.15.47:0/3238686154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b001a1740 con 0x7f6b00103900 2024-03-21T11:32:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.964+0000 7f6b05200640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b001054b0 0x7f6b001a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.965+0000 7f6affe00640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b00103900 0x7f6b00199b20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6af000e9c0 tx=0x7f6af000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.965+0000 7f6afcc00640 1 -- 172.21.15.47:0/3238686154 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af000cd80 con 0x7f6b00103900 2024-03-21T11:32:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.965+0000 7f6b07200640 1 -- 172.21.15.47:0/3238686154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b001a1a30 con 0x7f6b00103900 2024-03-21T11:32:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.965+0000 7f6afcc00640 1 -- 172.21.15.47:0/3238686154 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6af00035f0 con 0x7f6b00103900 2024-03-21T11:32:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.965+0000 7f6afcc00640 1 -- 172.21.15.47:0/3238686154 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0010690 con 0x7f6b00103900 2024-03-21T11:32:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.965+0000 7f6b07200640 1 -- 172.21.15.47:0/3238686154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b001a1f50 con 0x7f6b00103900 2024-03-21T11:32:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.968+0000 7f6b09f8b740 1 Processor -- start 2024-03-21T11:32:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.968+0000 7f6b09f8b740 1 -- start start 2024-03-21T11:32:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.968+0000 7f6afcc00640 1 -- 172.21.15.47:0/3238686154 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6af00037a0 con 0x7f6b00103900 2024-03-21T11:32:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.969+0000 7f6b09f8b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c061e0 0x55cc18c06600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.969+0000 7f6b09f8b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c0b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.969+0000 7f6b09f8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c0bd40 0x55cc18c0e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.969+0000 7f6b09f8b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc18bc93b0 con 0x55cc18c061e0 2024-03-21T11:32:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.969+0000 7f6b09f8b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc18b6b830 con 0x55cc18c06bd0 2024-03-21T11:32:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.969+0000 7f6b09f8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc18b145d0 con 0x55cc18c0bd40 2024-03-21T11:32:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.969+0000 7f6aff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c0bd40 0x55cc18c0e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.969+0000 7f6affe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c0b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.969+0000 7f6b05200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c061e0 0x55cc18c06600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.969+0000 7f6aff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c0bd40 0x55cc18c0e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33968/0 (socket says 172.21.15.47:33968) 2024-03-21T11:32:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.969+0000 7f6aff400640 1 -- 172.21.15.47:0/3906247043 learned_addr learned my addr 172.21.15.47:0/3906247043 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.970+0000 7f6aff400640 1 -- 172.21.15.47:0/3906247043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 msgr2=0x55cc18c0b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.970+0000 7f6aff400640 1 --2- 172.21.15.47:0/3906247043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c0b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.970+0000 7f6aff400640 1 -- 172.21.15.47:0/3906247043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c061e0 msgr2=0x55cc18c06600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.970+0000 7f6aff400640 1 --2- 172.21.15.47:0/3906247043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c061e0 0x55cc18c06600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.970+0000 7f6aff400640 1 -- 172.21.15.47:0/3906247043 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc18b2c2c0 con 0x55cc18c0bd40 2024-03-21T11:32:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.970+0000 7f6aff400640 1 --2- 172.21.15.47:0/3906247043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c0bd40 0x55cc18c0e1d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6aec000c00 tx=0x7f6aec009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2976d792ccaaf5ad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.970+0000 7f6af5600640 1 -- 172.21.15.47:0/3906247043 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec045070 con 0x55cc18c0bd40 2024-03-21T11:32:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.970+0000 7f6af5600640 1 -- 172.21.15.47:0/3906247043 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6aec004040 con 0x55cc18c0bd40 2024-03-21T11:32:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.970+0000 7f6af5600640 1 -- 172.21.15.47:0/3906247043 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec004eb0 con 0x55cc18c0bd40 2024-03-21T11:32:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.970+0000 7f6afcc00640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ae40a2fc0 0x7f6ae40a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.971+0000 7f6afcc00640 1 -- 172.21.15.47:0/3238686154 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6af00cfb70 con 0x7f6b00103900 2024-03-21T11:32:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.971+0000 7f6b05200640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ae40a2fc0 0x7f6ae40a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.971+0000 7f6b09f8b740 1 -- 172.21.15.47:0/3906247043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c0bd40 msgr2=0x55cc18c0e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.971+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/3906247043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c0bd40 0x55cc18c0e1d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6aec000c00 tx=0x7f6aec009560 comp rx=0 tx=0).stop 2024-03-21T11:32:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.971+0000 7f6b09f8b740 1 -- 172.21.15.47:0/3906247043 shutdown_connections 2024-03-21T11:32:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.971+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/3906247043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c0bd40 0x55cc18c0e1d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.971+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/3906247043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c0b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.971+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/3906247043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c061e0 0x55cc18c06600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.971+0000 7f6b09f8b740 1 -- 172.21.15.47:0/3906247043 >> 172.21.15.47:0/3906247043 conn(0x55cc18c01750 msgr2=0x55cc18c01b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.972+0000 7f6b09f8b740 1 -- 172.21.15.47:0/3906247043 shutdown_connections 2024-03-21T11:32:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.972+0000 7f6b09f8b740 1 -- 172.21.15.47:0/3906247043 wait complete. 2024-03-21T11:32:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.972+0000 7f6b05200640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ae40a2fc0 0x7f6ae40a5480 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55cc18c07430 tx=0x7f6af8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.972+0000 7f6b09f8b740 1 -- start start 2024-03-21T11:32:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.972+0000 7f6b09f8b740 1 Processor -- start 2024-03-21T11:32:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.973+0000 7f6b09f8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c10330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.973+0000 7f6b09f8b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c10870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.973+0000 7f6b09f8b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 0x55cc18c178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.973+0000 7f6b09f8b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc18b27a20 con 0x55cc18c0bd40 2024-03-21T11:32:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.973+0000 7f6b09f8b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc18b678d0 con 0x55cc18c06bd0 2024-03-21T11:32:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.973+0000 7f6b09f8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc18c0ec90 con 0x55cc18c061e0 2024-03-21T11:32:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.973+0000 7f6b09f8b740 10 client.0 _ll_register_callbacks cb 0x55cc18b641c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.973+0000 7f6affe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c10330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.973+0000 7f6affe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c10330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33980/0 (socket says 172.21.15.47:33980) 2024-03-21T11:32:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.973+0000 7f6affe00640 1 -- 172.21.15.47:0/2502529225 learned_addr learned my addr 172.21.15.47:0/2502529225 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.973+0000 7f6aff400640 1 --2- 172.21.15.47:0/2502529225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c10870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.973+0000 7f6b05200640 1 --2- 172.21.15.47:0/2502529225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 0x55cc18c178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.974+0000 7f6affe00640 1 -- 172.21.15.47:0/2502529225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 msgr2=0x55cc18c10870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.974+0000 7f6affe00640 1 --2- 172.21.15.47:0/2502529225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c10870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.974+0000 7f6affe00640 1 -- 172.21.15.47:0/2502529225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 msgr2=0x55cc18c178b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.974+0000 7f6affe00640 1 --2- 172.21.15.47:0/2502529225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 0x55cc18c178b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.974+0000 7f6affe00640 1 -- 172.21.15.47:0/2502529225 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc18880210 con 0x55cc18c061e0 2024-03-21T11:32:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.974+0000 7f6aff400640 1 --2- 172.21.15.47:0/2502529225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c10870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.974+0000 7f6affe00640 1 --2- 172.21.15.47:0/2502529225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c10330 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6af00a4390 tx=0x7f6af0098210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d7a55adff36245e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.974+0000 7f6aea000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.974+0000 7f6aea000640 1 -- 172.21.15.47:0/2502529225 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af00d7040 con 0x55cc18c061e0 2024-03-21T11:32:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.975+0000 7f6aea000640 1 -- 172.21.15.47:0/2502529225 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6af0003b30 con 0x55cc18c061e0 2024-03-21T11:32:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.975+0000 7f6b09f8b740 1 -- 172.21.15.47:0/2502529225 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6aec004040 con 0x55cc18c061e0 2024-03-21T11:32:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.975+0000 7f6aea000640 1 -- 172.21.15.47:0/2502529225 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6af009c5b0 con 0x55cc18c061e0 2024-03-21T11:32:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.975+0000 7f6b09f8b740 10 client.28621 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.975+0000 7f6b09f8b740 1 -- 172.21.15.47:0/2502529225 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cc1894c620 con 0x55cc18c061e0 2024-03-21T11:32:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.975+0000 7f6aea000640 1 -- 172.21.15.47:0/2502529225 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6aec004040 con 0x55cc18c061e0 2024-03-21T11:32:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.975+0000 7f6aea000640 1 -- 172.21.15.47:0/2502529225 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6af009b390 con 0x55cc18c061e0 2024-03-21T11:32:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.976+0000 7f6b09f8b740 10 client.28621 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.976+0000 7f6b09f8b740 10 client.28621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.976+0000 7f6b09f8b740 20 client.28621 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.976+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/2502529225 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cc18c1c200 0x55cc18c1e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.976+0000 7f6b09f8b740 4 client.28621 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.976+0000 7f6b09f8b740 1 -- 172.21.15.47:0/2502529225 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc18b78d70 con 0x55cc18c1c200 2024-03-21T11:32:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.976+0000 7f6aff400640 1 --2- 172.21.15.47:0/2502529225 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cc18c1c200 0x55cc18c1e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.977+0000 7f6aff400640 1 --2- 172.21.15.47:0/2502529225 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cc18c1c200 0x55cc18c1e5f0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f53dd4e588319e server_cookie=5f79bbbd6443a5cf in_seq=0 out_seq=0 2024-03-21T11:32:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.977+0000 7f6aea000640 10 client.28621 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6aea000640 1 -- 172.21.15.47:0/2502529225 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cc18b78d70 con 0x55cc18c1c200 2024-03-21T11:32:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6aea000640 10 client.28621 handle_command_reply: tid=0 2024-03-21T11:32:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6b09f8b740 1 client.28621 shutdown 2024-03-21T11:32:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6b09f8b740 1 -- 172.21.15.47:0/2502529225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 msgr2=0x55cc18c10330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/2502529225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c10330 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6af00a4390 tx=0x7f6af0098210 comp rx=0 tx=0).stop 2024-03-21T11:32:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6b09f8b740 1 -- 172.21.15.47:0/2502529225 shutdown_connections 2024-03-21T11:32:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/2502529225 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cc18c1c200 0x55cc18c1e5f0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/2502529225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 0x55cc18c178b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/2502529225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c10870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6aea000640 0 client.28621 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/2502529225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c10330 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6b09f8b740 1 -- 172.21.15.47:0/2502529225 >> 172.21.15.47:0/2502529225 conn(0x55cc18c01750 msgr2=0x55cc18c00950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6b09f8b740 1 -- 172.21.15.47:0/2502529225 shutdown_connections 2024-03-21T11:32:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6b09f8b740 1 -- 172.21.15.47:0/2502529225 wait complete. 2024-03-21T11:32:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.978+0000 7f6b09f8b740 20 client.28621 trim_cache size 0 max 16384 2024-03-21T11:32:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.994+0000 7f6b09f8b740 1 Processor -- start 2024-03-21T11:32:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.994+0000 7f6b09f8b740 1 -- start start 2024-03-21T11:32:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6b09f8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c0ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6b09f8b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c17870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6b09f8b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 0x55cc18c10480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6b09f8b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b001ac1e0 con 0x55cc18c0bd40 2024-03-21T11:32:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6b09f8b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b001adb60 con 0x55cc18c06bd0 2024-03-21T11:32:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6b09f8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b001ac840 con 0x55cc18c061e0 2024-03-21T11:32:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6affe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c0ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6affe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c0ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33990/0 (socket says 172.21.15.47:33990) 2024-03-21T11:32:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6affe00640 1 -- 172.21.15.47:0/1819780132 learned_addr learned my addr 172.21.15.47:0/1819780132 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6aff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c17870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6b05200640 1 --2- 172.21.15.47:0/1819780132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 0x55cc18c10480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6affe00640 1 -- 172.21.15.47:0/1819780132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 msgr2=0x55cc18c17870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6affe00640 1 --2- 172.21.15.47:0/1819780132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c17870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6affe00640 1 -- 172.21.15.47:0/1819780132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 msgr2=0x55cc18c10480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.995+0000 7f6affe00640 1 --2- 172.21.15.47:0/1819780132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 0x55cc18c10480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.996+0000 7f6affe00640 1 -- 172.21.15.47:0/1819780132 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc18c0ebd0 con 0x55cc18c061e0 2024-03-21T11:32:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.996+0000 7f6aff400640 1 --2- 172.21.15.47:0/1819780132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c17870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.996+0000 7f6affe00640 1 --2- 172.21.15.47:0/1819780132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c0ff40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6af0098880 tx=0x7f6af000c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87a1ba63772321ee server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.996+0000 7f6af4c00640 1 -- 172.21.15.47:0/1819780132 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af009b9f0 con 0x55cc18c061e0 2024-03-21T11:32:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.996+0000 7f6af4c00640 1 -- 172.21.15.47:0/1819780132 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6af009beb0 con 0x55cc18c061e0 2024-03-21T11:32:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.996+0000 7f6af4c00640 1 -- 172.21.15.47:0/1819780132 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0003c30 con 0x55cc18c061e0 2024-03-21T11:32:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.997+0000 7f6b09f8b740 1 -- 172.21.15.47:0/1819780132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 msgr2=0x55cc18c0ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.997+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/1819780132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c0ff40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6af0098880 tx=0x7f6af000c7d0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.997+0000 7f6b09f8b740 1 -- 172.21.15.47:0/1819780132 shutdown_connections 2024-03-21T11:32:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.997+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/1819780132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 0x55cc18c10480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.997+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/1819780132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c17870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.997+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/1819780132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c0ff40 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.997+0000 7f6b09f8b740 1 -- 172.21.15.47:0/1819780132 >> 172.21.15.47:0/1819780132 conn(0x55cc18c01750 msgr2=0x55cc18c07e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.997+0000 7f6b09f8b740 1 -- 172.21.15.47:0/1819780132 shutdown_connections 2024-03-21T11:32:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.997+0000 7f6b09f8b740 1 -- 172.21.15.47:0/1819780132 wait complete. 2024-03-21T11:32:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.998+0000 7f6b09f8b740 1 -- start start 2024-03-21T11:32:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.998+0000 7f6b09f8b740 1 Processor -- start 2024-03-21T11:32:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.998+0000 7f6b09f8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c17130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.998+0000 7f6b09f8b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c17670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.998+0000 7f6b09f8b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 0x55cc18c15740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.998+0000 7f6b09f8b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b001ad500 con 0x55cc18c0bd40 2024-03-21T11:32:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.998+0000 7f6b09f8b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b001acea0 con 0x55cc18c06bd0 2024-03-21T11:32:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.998+0000 7f6b09f8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b001aa3a0 con 0x55cc18c061e0 2024-03-21T11:32:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.998+0000 7f6b09f8b740 10 client.0 _ll_register_callbacks cb 0x7f6af00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.999+0000 7f6affe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c17130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.999+0000 7f6aff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c17670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.999+0000 7f6aff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c17670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42860/0 (socket says 172.21.15.47:42860) 2024-03-21T11:32:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.999+0000 7f6aff400640 1 -- 172.21.15.47:0/3071649303 learned_addr learned my addr 172.21.15.47:0/3071649303 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.999+0000 7f6affe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c17130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33998/0 (socket says 172.21.15.47:33998) 2024-03-21T11:32:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.999+0000 7f6b05200640 1 --2- 172.21.15.47:0/3071649303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 0x55cc18c15740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.999+0000 7f6aff400640 1 -- 172.21.15.47:0/3071649303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 msgr2=0x55cc18c17130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.999+0000 7f6aff400640 1 --2- 172.21.15.47:0/3071649303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c17130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.999+0000 7f6aff400640 1 -- 172.21.15.47:0/3071649303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 msgr2=0x55cc18c15740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.999+0000 7f6aff400640 1 --2- 172.21.15.47:0/3071649303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 0x55cc18c15740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.999+0000 7f6aff400640 1 -- 172.21.15.47:0/3071649303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc18c15e80 con 0x55cc18c06bd0 2024-03-21T11:32:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.999+0000 7f6affe00640 1 --2- 172.21.15.47:0/3071649303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c17130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.999+0000 7f6aff400640 1 --2- 172.21.15.47:0/3071649303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c17670 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6aec02c8e0 tx=0x7f6aec005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cb6dec100d9d7f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:31.999+0000 7f6aea000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:32:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.000+0000 7f6aea000640 1 -- 172.21.15.47:0/3071649303 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec045040 con 0x55cc18c06bd0 2024-03-21T11:32:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.000+0000 7f6aea000640 1 -- 172.21.15.47:0/3071649303 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6aec004470 con 0x55cc18c06bd0 2024-03-21T11:32:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.000+0000 7f6b09f8b740 1 -- 172.21.15.47:0/3071649303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6af009beb0 con 0x55cc18c06bd0 2024-03-21T11:32:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.000+0000 7f6aea000640 1 -- 172.21.15.47:0/3071649303 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6aec05f050 con 0x55cc18c06bd0 2024-03-21T11:32:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.000+0000 7f6b09f8b740 10 client.28619 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.000+0000 7f6b09f8b740 1 -- 172.21.15.47:0/3071649303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cc18c78510 con 0x55cc18c06bd0 2024-03-21T11:32:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.000+0000 7f6aea000640 1 -- 172.21.15.47:0/3071649303 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6af009beb0 con 0x55cc18c06bd0 2024-03-21T11:32:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.001+0000 7f6aea000640 1 -- 172.21.15.47:0/3071649303 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6aec052590 con 0x55cc18c06bd0 2024-03-21T11:32:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.001+0000 7f6b09f8b740 10 client.28619 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.001+0000 7f6b09f8b740 10 client.28619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.001+0000 7f6b09f8b740 20 client.28619 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.001+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/3071649303 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cc18c78b90 0x55cc18c7af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.001+0000 7f6b09f8b740 4 client.28619 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:32:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.001+0000 7f6b09f8b740 1 -- 172.21.15.47:0/3071649303 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cc18b730c0 con 0x55cc18c78b90 2024-03-21T11:32:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.001+0000 7f6affe00640 1 --2- 172.21.15.47:0/3071649303 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cc18c78b90 0x55cc18c7af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.001+0000 7f6affe00640 1 --2- 172.21.15.47:0/3071649303 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cc18c78b90 0x55cc18c7af80 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f31faa3b7b5d810 server_cookie=c65d9d153e748308 in_seq=0 out_seq=0 2024-03-21T11:32:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.001+0000 7f6aea000640 10 client.28619 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.242+0000 7f6aea000640 1 -- 172.21.15.47:0/3071649303 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cc18b730c0 con 0x55cc18c78b90 2024-03-21T11:32:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.242+0000 7f6aea000640 10 client.28619 handle_command_reply: tid=0 2024-03-21T11:32:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.243+0000 7f6b09f8b740 1 client.28619 shutdown 2024-03-21T11:32:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.243+0000 7f6b09f8b740 1 -- 172.21.15.47:0/3071649303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 msgr2=0x55cc18c17670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.243+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/3071649303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c17670 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6aec02c8e0 tx=0x7f6aec005fd0 comp rx=0 tx=0).stop 2024-03-21T11:32:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.243+0000 7f6b09f8b740 1 -- 172.21.15.47:0/3071649303 shutdown_connections 2024-03-21T11:32:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.243+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/3071649303 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cc18c78b90 0x55cc18c7af80 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.243+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/3071649303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc18c0bd40 0x55cc18c15740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.243+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/3071649303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc18c06bd0 0x55cc18c17670 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.243+0000 7f6b09f8b740 1 --2- 172.21.15.47:0/3071649303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cc18c061e0 0x55cc18c17130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.243+0000 7f6aea000640 0 client.28619 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.243+0000 7f6b09f8b740 1 -- 172.21.15.47:0/3071649303 >> 172.21.15.47:0/3071649303 conn(0x55cc18c01750 msgr2=0x55cc18c0ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.243+0000 7f6b09f8b740 1 -- 172.21.15.47:0/3071649303 shutdown_connections 2024-03-21T11:32:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.243+0000 7f6b09f8b740 1 -- 172.21.15.47:0/3071649303 wait complete. 2024-03-21T11:32:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.243+0000 7f6b09f8b740 20 client.28619 trim_cache size 0 max 16384 2024-03-21T11:32:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:32:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:32:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:32:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:32:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.244+0000 7f6b07200640 1 -- 172.21.15.47:0/3238686154 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ae40a2fc0 msgr2=0x7f6ae40a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.244+0000 7f6b07200640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ae40a2fc0 0x7f6ae40a5480 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55cc18c07430 tx=0x7f6af8009500 comp rx=0 tx=0).stop 2024-03-21T11:32:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.244+0000 7f6b07200640 1 -- 172.21.15.47:0/3238686154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b00103900 msgr2=0x7f6b00199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.244+0000 7f6b07200640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b00103900 0x7f6b00199b20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6af000e9c0 tx=0x7f6af000ee80 comp rx=0 tx=0).stop 2024-03-21T11:32:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.245+0000 7f6b07200640 1 -- 172.21.15.47:0/3238686154 shutdown_connections 2024-03-21T11:32:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.245+0000 7f6b07200640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ae40a2fc0 0x7f6ae40a5480 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.245+0000 7f6b07200640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b001054b0 0x7f6b001a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.245+0000 7f6b07200640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b00104af0 0x7f6b0019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.245+0000 7f6b07200640 1 --2- 172.21.15.47:0/3238686154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b00103900 0x7f6b00199b20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.245+0000 7f6b07200640 1 -- 172.21.15.47:0/3238686154 >> 172.21.15.47:0/3238686154 conn(0x7f6b000fee90 msgr2=0x7f6b00107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.245+0000 7f6b07200640 1 -- 172.21.15.47:0/3238686154 shutdown_connections 2024-03-21T11:32:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:32.245+0000 7f6b07200640 1 -- 172.21.15.47:0/3238686154 wait complete. 2024-03-21T11:32:32.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:32 smithi100 ceph-mon[36245]: pgmap v676: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 256 op/s 2024-03-21T11:32:32.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:32 smithi102 ceph-mon[36748]: pgmap v676: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 256 op/s 2024-03-21T11:32:32.544 INFO:tasks.workunit.client.0.smithi047.stdout: 1 670615 8.16 MB/sec execute 316 sec latency 194.641 ms 2024-03-21T11:32:32.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:32 smithi047 ceph-mon[31351]: pgmap v676: 129 pgs: 129 active+clean; 38 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 256 op/s 2024-03-21T11:32:33.544 INFO:tasks.workunit.client.0.smithi047.stdout: 1 671739 8.15 MB/sec execute 317 sec latency 48.231 ms 2024-03-21T11:32:34.545 INFO:tasks.workunit.client.0.smithi047.stdout: 1 673588 8.15 MB/sec execute 318 sec latency 22.684 ms 2024-03-21T11:32:34.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:34 smithi102 ceph-mon[36748]: pgmap v677: 129 pgs: 129 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 313 op/s 2024-03-21T11:32:34.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:34 smithi047 ceph-mon[31351]: pgmap v677: 129 pgs: 129 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 313 op/s 2024-03-21T11:32:34.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:34 smithi100 ceph-mon[36245]: pgmap v677: 129 pgs: 129 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 313 op/s 2024-03-21T11:32:35.545 INFO:tasks.workunit.client.0.smithi047.stdout: 1 675155 8.16 MB/sec execute 319 sec latency 10.677 ms 2024-03-21T11:32:36.545 INFO:tasks.workunit.client.0.smithi047.stdout: 1 676434 8.15 MB/sec execute 320 sec latency 20.429 ms 2024-03-21T11:32:36.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:36 smithi102 ceph-mon[36748]: pgmap v678: 129 pgs: 129 active+clean; 36 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-03-21T11:32:36.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:36 smithi047 ceph-mon[31351]: pgmap v678: 129 pgs: 129 active+clean; 36 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-03-21T11:32:36.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:36 smithi100 ceph-mon[36245]: pgmap v678: 129 pgs: 129 active+clean; 36 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-03-21T11:32:37.545 INFO:tasks.workunit.client.0.smithi047.stdout: 1 678547 8.17 MB/sec execute 321 sec latency 10.761 ms 2024-03-21T11:32:38.545 INFO:tasks.workunit.client.0.smithi047.stdout: 1 679347 8.15 MB/sec execute 322 sec latency 66.102 ms 2024-03-21T11:32:38.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:38 smithi102 ceph-mon[36748]: pgmap v679: 129 pgs: 129 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 287 op/s 2024-03-21T11:32:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:38 smithi047 ceph-mon[31351]: pgmap v679: 129 pgs: 129 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 287 op/s 2024-03-21T11:32:38.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:38 smithi100 ceph-mon[36245]: pgmap v679: 129 pgs: 129 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 287 op/s 2024-03-21T11:32:39.545 INFO:tasks.workunit.client.0.smithi047.stdout: 1 681318 8.16 MB/sec execute 323 sec latency 22.365 ms 2024-03-21T11:32:40.545 INFO:tasks.workunit.client.0.smithi047.stdout: 1 682543 8.16 MB/sec execute 324 sec latency 13.143 ms 2024-03-21T11:32:40.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:40 smithi102 ceph-mon[36748]: pgmap v680: 129 pgs: 129 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 256 op/s 2024-03-21T11:32:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:40 smithi047 ceph-mon[31351]: pgmap v680: 129 pgs: 129 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 256 op/s 2024-03-21T11:32:40.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:40 smithi100 ceph-mon[36245]: pgmap v680: 129 pgs: 129 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 256 op/s 2024-03-21T11:32:41.546 INFO:tasks.workunit.client.0.smithi047.stdout: 1 684299 8.15 MB/sec execute 325 sec latency 17.377 ms 2024-03-21T11:32:42.546 INFO:tasks.workunit.client.0.smithi047.stdout: 1 685782 8.16 MB/sec execute 326 sec latency 14.640 ms 2024-03-21T11:32:42.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:42 smithi102 ceph-mon[36748]: pgmap v681: 129 pgs: 129 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 253 op/s 2024-03-21T11:32:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:42 smithi047 ceph-mon[31351]: pgmap v681: 129 pgs: 129 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 253 op/s 2024-03-21T11:32:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:42 smithi100 ceph-mon[36245]: pgmap v681: 129 pgs: 129 active+clean; 35 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 253 op/s 2024-03-21T11:32:43.546 INFO:tasks.workunit.client.0.smithi047.stdout: 1 686662 8.15 MB/sec execute 327 sec latency 77.923 ms 2024-03-21T11:32:44.546 INFO:tasks.workunit.client.0.smithi047.stdout: 1 688694 8.16 MB/sec execute 328 sec latency 31.929 ms 2024-03-21T11:32:44.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:44 smithi102 ceph-mon[36748]: pgmap v682: 129 pgs: 129 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 290 op/s 2024-03-21T11:32:44.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:44 smithi047 ceph-mon[31351]: pgmap v682: 129 pgs: 129 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 290 op/s 2024-03-21T11:32:44.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:44 smithi100 ceph-mon[36245]: pgmap v682: 129 pgs: 129 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 290 op/s 2024-03-21T11:32:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.395+0000 7fd7cde00640 1 -- 172.21.15.47:0/723442265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c8075680 msgr2=0x7fd7c8073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.395+0000 7fd7cde00640 1 --2- 172.21.15.47:0/723442265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c8075680 0x7fd7c8073ad0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009ca0 tx=0x7fd7b802c190 comp rx=0 tx=0).stop 2024-03-21T11:32:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.395+0000 7fd7cde00640 1 -- 172.21.15.47:0/723442265 shutdown_connections 2024-03-21T11:32:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.395+0000 7fd7cde00640 1 --2- 172.21.15.47:0/723442265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7c81093b0 0x7fd7c810b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.395+0000 7fd7cde00640 1 --2- 172.21.15.47:0/723442265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7c80740a0 0x7fd7c8074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.395+0000 7fd7cde00640 1 --2- 172.21.15.47:0/723442265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c8075680 0x7fd7c8073ad0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.395+0000 7fd7cde00640 1 -- 172.21.15.47:0/723442265 >> 172.21.15.47:0/723442265 conn(0x7fd7c80fcab0 msgr2=0x7fd7c80feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.395+0000 7fd7cde00640 1 -- 172.21.15.47:0/723442265 shutdown_connections 2024-03-21T11:32:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.395+0000 7fd7cde00640 1 -- 172.21.15.47:0/723442265 wait complete. 2024-03-21T11:32:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.396+0000 7fd7cde00640 1 Processor -- start 2024-03-21T11:32:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.396+0000 7fd7cde00640 1 -- start start 2024-03-21T11:32:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.396+0000 7fd7cde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7c80740a0 0x7fd7c8199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.396+0000 7fd7cde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7c8075680 0x7fd7c8199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.397+0000 7fd7cde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c81093b0 0x7fd7c81a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.397+0000 7fd7cde00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c810f390 con 0x7fd7c8075680 2024-03-21T11:32:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.397+0000 7fd7cde00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c810f210 con 0x7fd7c80740a0 2024-03-21T11:32:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.397+0000 7fd7cde00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c810f510 con 0x7fd7c81093b0 2024-03-21T11:32:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.397+0000 7fd7cca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7c80740a0 0x7fd7c8199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.397+0000 7fd7cca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7c80740a0 0x7fd7c8199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58472/0 (socket says 172.21.15.47:58472) 2024-03-21T11:32:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.397+0000 7fd7bfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7c8075680 0x7fd7c8199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.397+0000 7fd7cca00640 1 -- 172.21.15.47:0/2764705351 learned_addr learned my addr 172.21.15.47:0/2764705351 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.397+0000 7fd7cd400640 1 --2- 172.21.15.47:0/2764705351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c81093b0 0x7fd7c81a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.397+0000 7fd7cca00640 1 -- 172.21.15.47:0/2764705351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c81093b0 msgr2=0x7fd7c81a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.397+0000 7fd7cca00640 1 --2- 172.21.15.47:0/2764705351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c81093b0 0x7fd7c81a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.397+0000 7fd7cca00640 1 -- 172.21.15.47:0/2764705351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7c8075680 msgr2=0x7fd7c8199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.397+0000 7fd7cca00640 1 --2- 172.21.15.47:0/2764705351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7c8075680 0x7fd7c8199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.397+0000 7fd7cca00640 1 -- 172.21.15.47:0/2764705351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c81a1640 con 0x7fd7c80740a0 2024-03-21T11:32:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.398+0000 7fd7cca00640 1 --2- 172.21.15.47:0/2764705351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7c80740a0 0x7fd7c8199a20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80326d0 tx=0x7fd7b8032360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.398+0000 7fd7c6000640 1 -- 172.21.15.47:0/2764705351 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8039040 con 0x7fd7c80740a0 2024-03-21T11:32:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.398+0000 7fd7c6000640 1 -- 172.21.15.47:0/2764705351 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7b8044070 con 0x7fd7c80740a0 2024-03-21T11:32:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.398+0000 7fd7cde00640 1 -- 172.21.15.47:0/2764705351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c81a18d0 con 0x7fd7c80740a0 2024-03-21T11:32:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.398+0000 7fd7c6000640 1 -- 172.21.15.47:0/2764705351 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8004500 con 0x7fd7c80740a0 2024-03-21T11:32:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.398+0000 7fd7cde00640 1 -- 172.21.15.47:0/2764705351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c81a1d90 con 0x7fd7c80740a0 2024-03-21T11:32:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.401+0000 7fd7d207b740 1 Processor -- start 2024-03-21T11:32:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.401+0000 7fd7d207b740 1 -- start start 2024-03-21T11:32:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7d207b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec41f3f0 0x55caec41f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7d207b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 0x55caec4248e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7d207b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec424f50 0x55caec4273e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7d207b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caec3e23b0 con 0x55caec41fde0 2024-03-21T11:32:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7d207b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caec384830 con 0x55caec41f3f0 2024-03-21T11:32:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7d207b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caec32d5d0 con 0x55caec424f50 2024-03-21T11:32:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7cd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec41f3f0 0x55caec41f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec424f50 0x55caec4273e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec424f50 0x55caec4273e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36092/0 (socket says 172.21.15.47:36092) 2024-03-21T11:32:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7bfe00640 1 -- 172.21.15.47:0/2793870535 learned_addr learned my addr 172.21.15.47:0/2793870535 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7cd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec41f3f0 0x55caec41f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58474/0 (socket says 172.21.15.47:58474) 2024-03-21T11:32:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7c6000640 1 -- 172.21.15.47:0/2764705351 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd7b8003bf0 con 0x7fd7c80740a0 2024-03-21T11:32:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7bfe00640 1 -- 172.21.15.47:0/2793870535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec41f3f0 msgr2=0x55caec41f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7bfe00640 1 --2- 172.21.15.47:0/2793870535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec41f3f0 0x55caec41f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.402+0000 7fd7bfe00640 1 -- 172.21.15.47:0/2793870535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 msgr2=0x55caec4248e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.403+0000 7fd7cca00640 1 --2- 172.21.15.47:0/2793870535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 0x55caec4248e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.403+0000 7fd7bfe00640 1 --2- 172.21.15.47:0/2793870535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 0x55caec4248e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.403+0000 7fd7bfe00640 1 -- 172.21.15.47:0/2793870535 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caec3452c0 con 0x55caec424f50 2024-03-21T11:32:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.403+0000 7fd7cca00640 1 --2- 172.21.15.47:0/2793870535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 0x55caec4248e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:32:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.403+0000 7fd7bfe00640 1 --2- 172.21.15.47:0/2793870535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec424f50 0x55caec4273e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009ca0 tx=0x7fd7b0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10ebd0ad8ff36da9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.403+0000 7fd7bdc00640 1 -- 172.21.15.47:0/2793870535 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0005220 con 0x55caec424f50 2024-03-21T11:32:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.403+0000 7fd7bdc00640 1 -- 172.21.15.47:0/2793870535 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7b0004c80 con 0x55caec424f50 2024-03-21T11:32:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.403+0000 7fd7bdc00640 1 -- 172.21.15.47:0/2793870535 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0019830 con 0x55caec424f50 2024-03-21T11:32:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.404+0000 7fd7c6000640 1 --2- 172.21.15.47:0/2764705351 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7ac0a2f10 0x7fd7ac0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.404+0000 7fd7d207b740 1 -- 172.21.15.47:0/2793870535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec424f50 msgr2=0x55caec4273e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.404+0000 7fd7d207b740 1 --2- 172.21.15.47:0/2793870535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec424f50 0x55caec4273e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009ca0 tx=0x7fd7b0011810 comp rx=0 tx=0).stop 2024-03-21T11:32:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.404+0000 7fd7c6000640 1 -- 172.21.15.47:0/2764705351 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd7b80f1cb0 con 0x7fd7c80740a0 2024-03-21T11:32:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.404+0000 7fd7cd400640 1 --2- 172.21.15.47:0/2764705351 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7ac0a2f10 0x7fd7ac0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.404+0000 7fd7d207b740 1 -- 172.21.15.47:0/2793870535 shutdown_connections 2024-03-21T11:32:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.404+0000 7fd7d207b740 1 --2- 172.21.15.47:0/2793870535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec424f50 0x55caec4273e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.404+0000 7fd7d207b740 1 --2- 172.21.15.47:0/2793870535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 0x55caec4248e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.404+0000 7fd7d207b740 1 --2- 172.21.15.47:0/2793870535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec41f3f0 0x55caec41f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.404+0000 7fd7d207b740 1 -- 172.21.15.47:0/2793870535 >> 172.21.15.47:0/2793870535 conn(0x55caec41a960 msgr2=0x55caec41ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.405+0000 7fd7d207b740 1 -- 172.21.15.47:0/2793870535 shutdown_connections 2024-03-21T11:32:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.405+0000 7fd7d207b740 1 -- 172.21.15.47:0/2793870535 wait complete. 2024-03-21T11:32:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.405+0000 7fd7d207b740 1 -- start start 2024-03-21T11:32:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.405+0000 7fd7cd400640 1 --2- 172.21.15.47:0/2764705351 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7ac0a2f10 0x7fd7ac0a53d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55caec420640 tx=0x7fd7c000b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.406+0000 7fd7d207b740 1 Processor -- start 2024-03-21T11:32:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.406+0000 7fd7d207b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41f3f0 0x55caec4295a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.406+0000 7fd7d207b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec41fde0 0x55caec429ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.406+0000 7fd7d207b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec424f50 0x55caec430b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.406+0000 7fd7d207b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caec340a20 con 0x55caec41f3f0 2024-03-21T11:32:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.406+0000 7fd7d207b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caec3808d0 con 0x55caec41fde0 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.406+0000 7fd7d207b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caec427f00 con 0x55caec424f50 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.406+0000 7fd7d207b740 10 client.0 _ll_register_callbacks cb 0x55caec37d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.406+0000 7fd7bfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec41fde0 0x55caec429ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.406+0000 7fd7cd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec424f50 0x55caec430b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.406+0000 7fd7bfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec41fde0 0x55caec429ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58476/0 (socket says 172.21.15.47:58476) 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.406+0000 7fd7bfe00640 1 -- 172.21.15.47:0/1189007500 learned_addr learned my addr 172.21.15.47:0/1189007500 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.406+0000 7fd7cca00640 1 --2- 172.21.15.47:0/1189007500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41f3f0 0x55caec4295a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.407+0000 7fd7cd400640 1 -- 172.21.15.47:0/1189007500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec41fde0 msgr2=0x55caec429ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.407+0000 7fd7cd400640 1 --2- 172.21.15.47:0/1189007500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec41fde0 0x55caec429ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.407+0000 7fd7cd400640 1 -- 172.21.15.47:0/1189007500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41f3f0 msgr2=0x55caec4295a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.407+0000 7fd7cd400640 1 --2- 172.21.15.47:0/1189007500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41f3f0 0x55caec4295a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.407+0000 7fd7cd400640 1 -- 172.21.15.47:0/1189007500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caec099210 con 0x55caec424f50 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.407+0000 7fd7cd400640 1 --2- 172.21.15.47:0/1189007500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec424f50 0x55caec430b20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0003c50 tx=0x7fd7c0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af71497baf5c3159 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.407+0000 7fd7b6000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.407+0000 7fd7b6000640 1 -- 172.21.15.47:0/1189007500 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c002c060 con 0x55caec424f50 2024-03-21T11:32:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.407+0000 7fd7b6000640 1 -- 172.21.15.47:0/1189007500 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd7c0017030 con 0x55caec424f50 2024-03-21T11:32:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.407+0000 7fd7d207b740 1 -- 172.21.15.47:0/1189007500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7b0004c80 con 0x55caec424f50 2024-03-21T11:32:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.408+0000 7fd7b6000640 1 -- 172.21.15.47:0/1189007500 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7c0033050 con 0x55caec424f50 2024-03-21T11:32:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.408+0000 7fd7d207b740 10 client.28639 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.408+0000 7fd7d207b740 1 -- 172.21.15.47:0/1189007500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55caec165620 con 0x55caec424f50 2024-03-21T11:32:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.408+0000 7fd7b6000640 1 -- 172.21.15.47:0/1189007500 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7b0004c80 con 0x55caec424f50 2024-03-21T11:32:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.408+0000 7fd7b6000640 1 -- 172.21.15.47:0/1189007500 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd7c0011440 con 0x55caec424f50 2024-03-21T11:32:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.408+0000 7fd7d207b740 10 client.28639 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.408+0000 7fd7d207b740 10 client.28639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.408+0000 7fd7d207b740 20 client.28639 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.408+0000 7fd7d207b740 1 --2- 172.21.15.47:0/1189007500 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55caec4354a0 0x55caec437920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.408+0000 7fd7d207b740 4 client.28639 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.408+0000 7fd7d207b740 1 -- 172.21.15.47:0/1189007500 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55caec391d70 con 0x55caec4354a0 2024-03-21T11:32:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.409+0000 7fd7cca00640 1 --2- 172.21.15.47:0/1189007500 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55caec4354a0 0x55caec437920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.409+0000 7fd7cca00640 1 --2- 172.21.15.47:0/1189007500 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55caec4354a0 0x55caec437920 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6430e8447cffe051 server_cookie=2e607e91c8a9ff13 in_seq=0 out_seq=0 2024-03-21T11:32:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.409+0000 7fd7b6000640 10 client.28639 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.412+0000 7fd7b6000640 1 -- 172.21.15.47:0/1189007500 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55caec391d70 con 0x55caec4354a0 2024-03-21T11:32:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.412+0000 7fd7b6000640 10 client.28639 handle_command_reply: tid=0 2024-03-21T11:32:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.413+0000 7fd7d207b740 1 client.28639 shutdown 2024-03-21T11:32:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.413+0000 7fd7d207b740 1 -- 172.21.15.47:0/1189007500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec424f50 msgr2=0x55caec430b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.413+0000 7fd7d207b740 1 --2- 172.21.15.47:0/1189007500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec424f50 0x55caec430b20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0003c50 tx=0x7fd7c0004730 comp rx=0 tx=0).stop 2024-03-21T11:32:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.413+0000 7fd7d207b740 1 -- 172.21.15.47:0/1189007500 shutdown_connections 2024-03-21T11:32:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.413+0000 7fd7d207b740 1 --2- 172.21.15.47:0/1189007500 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55caec4354a0 0x55caec437920 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.413+0000 7fd7d207b740 1 --2- 172.21.15.47:0/1189007500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec424f50 0x55caec430b20 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.413+0000 7fd7d207b740 1 --2- 172.21.15.47:0/1189007500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec41fde0 0x55caec429ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.413+0000 7fd7d207b740 1 --2- 172.21.15.47:0/1189007500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41f3f0 0x55caec4295a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.413+0000 7fd7d207b740 1 -- 172.21.15.47:0/1189007500 >> 172.21.15.47:0/1189007500 conn(0x55caec41a960 msgr2=0x55caec419b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.413+0000 7fd7b6000640 0 client.28639 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.413+0000 7fd7d207b740 1 -- 172.21.15.47:0/1189007500 shutdown_connections 2024-03-21T11:32:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.413+0000 7fd7d207b740 1 -- 172.21.15.47:0/1189007500 wait complete. 2024-03-21T11:32:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.413+0000 7fd7d207b740 20 client.28639 trim_cache size 0 max 16384 2024-03-21T11:32:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.431+0000 7fd7d207b740 1 Processor -- start 2024-03-21T11:32:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.431+0000 7fd7d207b740 1 -- start start 2024-03-21T11:32:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7d207b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec41f3f0 0x55caec421100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7d207b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 0x55caec421640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7d207b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec424f50 0x55caec430ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7d207b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81ac000 con 0x55caec41fde0 2024-03-21T11:32:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7d207b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81ad980 con 0x55caec424f50 2024-03-21T11:32:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7d207b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81ac660 con 0x55caec41f3f0 2024-03-21T11:32:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7cca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec41f3f0 0x55caec421100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7cca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec41f3f0 0x55caec421100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36118/0 (socket says 172.21.15.47:36118) 2024-03-21T11:32:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7cca00640 1 -- 172.21.15.47:0/755451172 learned_addr learned my addr 172.21.15.47:0/755451172 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7cd400640 1 --2- 172.21.15.47:0/755451172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec424f50 0x55caec430ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7bfe00640 1 --2- 172.21.15.47:0/755451172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 0x55caec421640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7cca00640 1 -- 172.21.15.47:0/755451172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec424f50 msgr2=0x55caec430ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7cca00640 1 --2- 172.21.15.47:0/755451172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec424f50 0x55caec430ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7cca00640 1 -- 172.21.15.47:0/755451172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 msgr2=0x55caec421640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7cca00640 1 --2- 172.21.15.47:0/755451172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 0x55caec421640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.432+0000 7fd7cca00640 1 -- 172.21.15.47:0/755451172 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caec420bf0 con 0x55caec41f3f0 2024-03-21T11:32:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.433+0000 7fd7cca00640 1 --2- 172.21.15.47:0/755451172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec41f3f0 0x55caec421100 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8005e30 tx=0x7fd7b802c190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b79bfdcce5fa0dc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.433+0000 7fd7bd200640 1 -- 172.21.15.47:0/755451172 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80be030 con 0x55caec41f3f0 2024-03-21T11:32:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.433+0000 7fd7bd200640 1 -- 172.21.15.47:0/755451172 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7b80c2180 con 0x55caec41f3f0 2024-03-21T11:32:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.433+0000 7fd7bd200640 1 -- 172.21.15.47:0/755451172 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80de0c0 con 0x55caec41f3f0 2024-03-21T11:32:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.434+0000 7fd7d207b740 1 -- 172.21.15.47:0/755451172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec41f3f0 msgr2=0x55caec421100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.434+0000 7fd7d207b740 1 --2- 172.21.15.47:0/755451172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec41f3f0 0x55caec421100 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8005e30 tx=0x7fd7b802c190 comp rx=0 tx=0).stop 2024-03-21T11:32:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.434+0000 7fd7d207b740 1 -- 172.21.15.47:0/755451172 shutdown_connections 2024-03-21T11:32:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.434+0000 7fd7d207b740 1 --2- 172.21.15.47:0/755451172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec424f50 0x55caec430ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.434+0000 7fd7d207b740 1 --2- 172.21.15.47:0/755451172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 0x55caec421640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.434+0000 7fd7d207b740 1 --2- 172.21.15.47:0/755451172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec41f3f0 0x55caec421100 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.434+0000 7fd7d207b740 1 -- 172.21.15.47:0/755451172 >> 172.21.15.47:0/755451172 conn(0x55caec41a960 msgr2=0x55caec41ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.434+0000 7fd7d207b740 1 -- 172.21.15.47:0/755451172 shutdown_connections 2024-03-21T11:32:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.434+0000 7fd7d207b740 1 -- 172.21.15.47:0/755451172 wait complete. 2024-03-21T11:32:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.434+0000 7fd7d207b740 1 -- start start 2024-03-21T11:32:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.435+0000 7fd7d207b740 1 Processor -- start 2024-03-21T11:32:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.435+0000 7fd7d207b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec41f3f0 0x55caec42e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.435+0000 7fd7d207b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 0x55caec42e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.435+0000 7fd7d207b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec424f50 0x55caec42f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.435+0000 7fd7d207b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81ad320 con 0x55caec41fde0 2024-03-21T11:32:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.435+0000 7fd7d207b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81accc0 con 0x55caec424f50 2024-03-21T11:32:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.435+0000 7fd7d207b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81aa1c0 con 0x55caec41f3f0 2024-03-21T11:32:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.435+0000 7fd7d207b740 10 client.0 _ll_register_callbacks cb 0x7fd7c0003b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.435+0000 7fd7cca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec41f3f0 0x55caec42e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.435+0000 7fd7cca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec41f3f0 0x55caec42e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36128/0 (socket says 172.21.15.47:36128) 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.435+0000 7fd7cca00640 1 -- 172.21.15.47:0/1381263531 learned_addr learned my addr 172.21.15.47:0/1381263531 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.435+0000 7fd7cd400640 1 --2- 172.21.15.47:0/1381263531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec424f50 0x55caec42f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.435+0000 7fd7bfe00640 1 --2- 172.21.15.47:0/1381263531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 0x55caec42e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.436+0000 7fd7cca00640 1 -- 172.21.15.47:0/1381263531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec424f50 msgr2=0x55caec42f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.436+0000 7fd7cca00640 1 --2- 172.21.15.47:0/1381263531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec424f50 0x55caec42f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.436+0000 7fd7cca00640 1 -- 172.21.15.47:0/1381263531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 msgr2=0x55caec42e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.436+0000 7fd7cca00640 1 --2- 172.21.15.47:0/1381263531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 0x55caec42e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.436+0000 7fd7cca00640 1 -- 172.21.15.47:0/1381263531 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caec42ff50 con 0x55caec41f3f0 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.436+0000 7fd7cd400640 1 --2- 172.21.15.47:0/1381263531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec424f50 0x55caec42f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.436+0000 7fd7cca00640 1 --2- 172.21.15.47:0/1381263531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec41f3f0 0x55caec42e410 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8006010 tx=0x7fd7b80be870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=674c884d85e0c11b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.436+0000 7fd7b6000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.436+0000 7fd7b6000640 1 -- 172.21.15.47:0/1381263531 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80046a0 con 0x55caec41f3f0 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.436+0000 7fd7d207b740 1 -- 172.21.15.47:0/1381263531 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7b80c2180 con 0x55caec41f3f0 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.436+0000 7fd7b6000640 1 -- 172.21.15.47:0/1381263531 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd7b80de050 con 0x55caec41f3f0 2024-03-21T11:32:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.436+0000 7fd7b6000640 1 -- 172.21.15.47:0/1381263531 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7b80de250 con 0x55caec41f3f0 2024-03-21T11:32:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.436+0000 7fd7d207b740 10 client.28645 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.436+0000 7fd7d207b740 1 -- 172.21.15.47:0/1381263531 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55caec490ff0 con 0x55caec41f3f0 2024-03-21T11:32:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.437+0000 7fd7b6000640 1 -- 172.21.15.47:0/1381263531 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7b80c2180 con 0x55caec41f3f0 2024-03-21T11:32:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.437+0000 7fd7b6000640 1 -- 172.21.15.47:0/1381263531 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd7b803c030 con 0x55caec41f3f0 2024-03-21T11:32:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.437+0000 7fd7d207b740 10 client.28645 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.437+0000 7fd7d207b740 10 client.28645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.437+0000 7fd7d207b740 20 client.28645 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.437+0000 7fd7d207b740 1 --2- 172.21.15.47:0/1381263531 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55caec491860 0x55caec493cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.437+0000 7fd7d207b740 4 client.28645 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:32:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.437+0000 7fd7d207b740 1 -- 172.21.15.47:0/1381263531 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55caec38c0c0 con 0x55caec491860 2024-03-21T11:32:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.437+0000 7fd7bfe00640 1 --2- 172.21.15.47:0/1381263531 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55caec491860 0x55caec493cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.438+0000 7fd7bfe00640 1 --2- 172.21.15.47:0/1381263531 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55caec491860 0x55caec493cc0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b7b18a9af5fb2c server_cookie=22f5fa1d7a180c30 in_seq=0 out_seq=0 2024-03-21T11:32:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.438+0000 7fd7b6000640 10 client.28645 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:45.546 INFO:tasks.workunit.client.0.smithi047.stdout: 1 689625 8.15 MB/sec execute 329 sec latency 65.252 ms 2024-03-21T11:32:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.686+0000 7fd7b6000640 1 -- 172.21.15.47:0/1381263531 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55caec38c0c0 con 0x55caec491860 2024-03-21T11:32:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.686+0000 7fd7b6000640 10 client.28645 handle_command_reply: tid=0 2024-03-21T11:32:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.686+0000 7fd7d207b740 1 client.28645 shutdown 2024-03-21T11:32:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.686+0000 7fd7d207b740 1 -- 172.21.15.47:0/1381263531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec41f3f0 msgr2=0x55caec42e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.686+0000 7fd7d207b740 1 --2- 172.21.15.47:0/1381263531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec41f3f0 0x55caec42e410 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8006010 tx=0x7fd7b80be870 comp rx=0 tx=0).stop 2024-03-21T11:32:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.686+0000 7fd7d207b740 1 -- 172.21.15.47:0/1381263531 shutdown_connections 2024-03-21T11:32:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.686+0000 7fd7d207b740 1 --2- 172.21.15.47:0/1381263531 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55caec491860 0x55caec493cc0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.686+0000 7fd7d207b740 1 --2- 172.21.15.47:0/1381263531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55caec424f50 0x55caec42f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.686+0000 7fd7d207b740 1 --2- 172.21.15.47:0/1381263531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55caec41fde0 0x55caec42e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.686+0000 7fd7d207b740 1 --2- 172.21.15.47:0/1381263531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caec41f3f0 0x55caec42e410 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.686+0000 7fd7b6000640 0 client.28645 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.686+0000 7fd7d207b740 1 -- 172.21.15.47:0/1381263531 >> 172.21.15.47:0/1381263531 conn(0x55caec41a960 msgr2=0x55caec424320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.686+0000 7fd7d207b740 1 -- 172.21.15.47:0/1381263531 shutdown_connections 2024-03-21T11:32:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.687+0000 7fd7d207b740 1 -- 172.21.15.47:0/1381263531 wait complete. 2024-03-21T11:32:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.687+0000 7fd7d207b740 20 client.28645 trim_cache size 0 max 16384 2024-03-21T11:32:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:32:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:32:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:32:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:32:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.688+0000 7fd7cde00640 1 -- 172.21.15.47:0/2764705351 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7ac0a2f10 msgr2=0x7fd7ac0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.688+0000 7fd7cde00640 1 --2- 172.21.15.47:0/2764705351 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7ac0a2f10 0x7fd7ac0a53d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55caec420640 tx=0x7fd7c000b2e0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.688+0000 7fd7cde00640 1 -- 172.21.15.47:0/2764705351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7c80740a0 msgr2=0x7fd7c8199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.688+0000 7fd7cde00640 1 --2- 172.21.15.47:0/2764705351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7c80740a0 0x7fd7c8199a20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80326d0 tx=0x7fd7b8032360 comp rx=0 tx=0).stop 2024-03-21T11:32:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.688+0000 7fd7cde00640 1 -- 172.21.15.47:0/2764705351 shutdown_connections 2024-03-21T11:32:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.688+0000 7fd7cde00640 1 --2- 172.21.15.47:0/2764705351 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7ac0a2f10 0x7fd7ac0a53d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.688+0000 7fd7cde00640 1 --2- 172.21.15.47:0/2764705351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7c81093b0 0x7fd7c81a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.688+0000 7fd7cde00640 1 --2- 172.21.15.47:0/2764705351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7c8075680 0x7fd7c8199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.688+0000 7fd7cde00640 1 --2- 172.21.15.47:0/2764705351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7c80740a0 0x7fd7c8199a20 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.688+0000 7fd7cde00640 1 -- 172.21.15.47:0/2764705351 >> 172.21.15.47:0/2764705351 conn(0x7fd7c80fcab0 msgr2=0x7fd7c8105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.688+0000 7fd7cde00640 1 -- 172.21.15.47:0/2764705351 shutdown_connections 2024-03-21T11:32:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:45.688+0000 7fd7cde00640 1 -- 172.21.15.47:0/2764705351 wait complete. 2024-03-21T11:32:46.546 INFO:tasks.workunit.client.0.smithi047.stdout: 1 691283 8.14 MB/sec execute 330 sec latency 162.260 ms 2024-03-21T11:32:46.694 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:46 smithi100 ceph-mon[36245]: pgmap v683: 129 pgs: 129 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-03-21T11:32:46.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:46 smithi102 ceph-mon[36748]: pgmap v683: 129 pgs: 129 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-03-21T11:32:46.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:46 smithi047 ceph-mon[31351]: pgmap v683: 129 pgs: 129 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-03-21T11:32:47.546 INFO:tasks.workunit.client.0.smithi047.stdout: 1 692787 8.16 MB/sec execute 331 sec latency 13.768 ms 2024-03-21T11:32:48.546 INFO:tasks.workunit.client.0.smithi047.stdout: 1 693655 8.14 MB/sec execute 332 sec latency 57.064 ms 2024-03-21T11:32:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:48 smithi102 ceph-mon[36748]: pgmap v684: 129 pgs: 129 active+clean; 35 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-03-21T11:32:48.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:48 smithi047 ceph-mon[31351]: pgmap v684: 129 pgs: 129 active+clean; 35 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-03-21T11:32:48.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:48 smithi100 ceph-mon[36245]: pgmap v684: 129 pgs: 129 active+clean; 35 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-03-21T11:32:49.547 INFO:tasks.workunit.client.0.smithi047.stdout: 1 695514 8.15 MB/sec execute 333 sec latency 21.850 ms 2024-03-21T11:32:50.547 INFO:tasks.workunit.client.0.smithi047.stdout: 1 696722 8.14 MB/sec execute 334 sec latency 15.569 ms 2024-03-21T11:32:50.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:50 smithi102 ceph-mon[36748]: pgmap v685: 129 pgs: 129 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 2.3 MiB/s wr, 247 op/s 2024-03-21T11:32:50.876 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:50 smithi047 ceph-mon[31351]: pgmap v685: 129 pgs: 129 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 2.3 MiB/s wr, 247 op/s 2024-03-21T11:32:50.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:50 smithi100 ceph-mon[36245]: pgmap v685: 129 pgs: 129 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 2.3 MiB/s wr, 247 op/s 2024-03-21T11:32:51.547 INFO:tasks.workunit.client.0.smithi047.stdout: 1 698463 8.14 MB/sec execute 335 sec latency 22.229 ms 2024-03-21T11:32:52.547 INFO:tasks.workunit.client.0.smithi047.stdout: 1 699909 8.15 MB/sec execute 336 sec latency 12.152 ms 2024-03-21T11:32:52.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:32:52.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:32:52.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:52 smithi102 ceph-mon[36748]: pgmap v686: 129 pgs: 129 active+clean; 38 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.2 MiB/s wr, 275 op/s 2024-03-21T11:32:52.771 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:52 smithi047 ceph-mon[31351]: pgmap v686: 129 pgs: 129 active+clean; 38 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.2 MiB/s wr, 275 op/s 2024-03-21T11:32:52.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:52 smithi100 ceph-mon[36245]: pgmap v686: 129 pgs: 129 active+clean; 38 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.2 MiB/s wr, 275 op/s 2024-03-21T11:32:52.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.919+0000 7fd163e00640 1 -- 172.21.15.47:0/4084889481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd15c102180 msgr2=0x7fd15c104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:52.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.919+0000 7fd163e00640 1 --2- 172.21.15.47:0/4084889481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd15c102180 0x7fd15c104570 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd150009900 tx=0x7fd15002bc30 comp rx=0 tx=0).stop 2024-03-21T11:32:52.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.919+0000 7fd163e00640 1 -- 172.21.15.47:0/4084889481 shutdown_connections 2024-03-21T11:32:52.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.919+0000 7fd163e00640 1 --2- 172.21.15.47:0/4084889481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd15c1073e0 0x7fd15c1097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.919+0000 7fd163e00640 1 --2- 172.21.15.47:0/4084889481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd15c104ab0 0x7fd15c106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.919+0000 7fd163e00640 1 --2- 172.21.15.47:0/4084889481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd15c102180 0x7fd15c104570 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.919+0000 7fd163e00640 1 -- 172.21.15.47:0/4084889481 >> 172.21.15.47:0/4084889481 conn(0x7fd15c0fbb30 msgr2=0x7fd15c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:52.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.920+0000 7fd163e00640 1 -- 172.21.15.47:0/4084889481 shutdown_connections 2024-03-21T11:32:52.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.920+0000 7fd163e00640 1 -- 172.21.15.47:0/4084889481 wait complete. 2024-03-21T11:32:52.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.921+0000 7fd163e00640 1 Processor -- start 2024-03-21T11:32:52.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.921+0000 7fd163e00640 1 -- start start 2024-03-21T11:32:52.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.921+0000 7fd163e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd15c102180 0x7fd15c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.921+0000 7fd163e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd15c104ab0 0x7fd15c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.921+0000 7fd163e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd15c1073e0 0x7fd15c1a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.921+0000 7fd163e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c10d370 con 0x7fd15c104ab0 2024-03-21T11:32:52.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.921+0000 7fd163e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c10d1f0 con 0x7fd15c1073e0 2024-03-21T11:32:52.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.921+0000 7fd163e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c10d4f0 con 0x7fd15c102180 2024-03-21T11:32:52.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.921+0000 7fd161400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd15c102180 0x7fd15c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.921+0000 7fd161400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd15c102180 0x7fd15c199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50452/0 (socket says 172.21.15.47:50452) 2024-03-21T11:32:52.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.922+0000 7fd161400640 1 -- 172.21.15.47:0/2007663470 learned_addr learned my addr 172.21.15.47:0/2007663470 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:52.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.922+0000 7fd161e00640 1 --2- 172.21.15.47:0/2007663470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd15c1073e0 0x7fd15c1a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.922+0000 7fd160a00640 1 --2- 172.21.15.47:0/2007663470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd15c104ab0 0x7fd15c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.922+0000 7fd161400640 1 -- 172.21.15.47:0/2007663470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd15c1073e0 msgr2=0x7fd15c1a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:52.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.922+0000 7fd161400640 1 --2- 172.21.15.47:0/2007663470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd15c1073e0 0x7fd15c1a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.922+0000 7fd161400640 1 -- 172.21.15.47:0/2007663470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd15c104ab0 msgr2=0x7fd15c19a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:52.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.922+0000 7fd161400640 1 --2- 172.21.15.47:0/2007663470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd15c104ab0 0x7fd15c19a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.922+0000 7fd161400640 1 -- 172.21.15.47:0/2007663470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c1a1850 con 0x7fd15c102180 2024-03-21T11:32:52.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.922+0000 7fd161400640 1 --2- 172.21.15.47:0/2007663470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd15c102180 0x7fd15c199c40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd150000c00 tx=0x7fd150005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:52.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.923+0000 7fd15a000640 1 -- 172.21.15.47:0/2007663470 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd150043020 con 0x7fd15c102180 2024-03-21T11:32:52.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.923+0000 7fd15a000640 1 -- 172.21.15.47:0/2007663470 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1500054e0 con 0x7fd15c102180 2024-03-21T11:32:52.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.923+0000 7fd15a000640 1 -- 172.21.15.47:0/2007663470 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd150003e60 con 0x7fd15c102180 2024-03-21T11:32:52.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.923+0000 7fd163e00640 1 -- 172.21.15.47:0/2007663470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15c1a1ae0 con 0x7fd15c102180 2024-03-21T11:32:52.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.923+0000 7fd163e00640 1 -- 172.21.15.47:0/2007663470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15c1a1fe0 con 0x7fd15c102180 2024-03-21T11:32:52.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.926+0000 7fd15a000640 1 -- 172.21.15.47:0/2007663470 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd150046020 con 0x7fd15c102180 2024-03-21T11:32:52.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.926+0000 7fd166a77740 1 Processor -- start 2024-03-21T11:32:52.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.927+0000 7fd166a77740 1 -- start start 2024-03-21T11:32:52.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.927+0000 7fd166a77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c04e80 0x55f041c052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.927+0000 7fd166a77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c05870 0x55f041c0a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.927+0000 7fd166a77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c0a9e0 0x55f041c0ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.927+0000 7fd166a77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f041b6a250 con 0x55f041c05870 2024-03-21T11:32:52.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.927+0000 7fd166a77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f041b12a30 con 0x55f041c04e80 2024-03-21T11:32:52.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.927+0000 7fd166a77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f041b662f0 con 0x55f041c0a9e0 2024-03-21T11:32:52.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.927+0000 7fd161e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c04e80 0x55f041c052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.927+0000 7fd161e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c04e80 0x55f041c052a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60728/0 (socket says 172.21.15.47:60728) 2024-03-21T11:32:52.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.927+0000 7fd161e00640 1 -- 172.21.15.47:0/2916634944 learned_addr learned my addr 172.21.15.47:0/2916634944 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:52.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.927+0000 7fd160a00640 1 --2- 172.21.15.47:0/2916634944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c0a9e0 0x55f041c0ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.927+0000 7fd161400640 1 --2- 172.21.15.47:0/2916634944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c05870 0x55f041c0a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.928+0000 7fd161e00640 1 -- 172.21.15.47:0/2916634944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c0a9e0 msgr2=0x55f041c0ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:52.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.928+0000 7fd161e00640 1 --2- 172.21.15.47:0/2916634944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c0a9e0 0x55f041c0ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.928+0000 7fd161e00640 1 -- 172.21.15.47:0/2916634944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c05870 msgr2=0x55f041c0a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:52.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.928+0000 7fd161e00640 1 --2- 172.21.15.47:0/2916634944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c05870 0x55f041c0a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.928+0000 7fd161e00640 1 -- 172.21.15.47:0/2916634944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f041afd690 con 0x55f041c04e80 2024-03-21T11:32:52.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.928+0000 7fd161e00640 1 --2- 172.21.15.47:0/2916634944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c04e80 0x55f041c052a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd154009f70 tx=0x7fd154010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4815262028a0b5c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:52.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.928+0000 7fd15a000640 1 --2- 172.21.15.47:0/2007663470 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1440a2fa0 0x7fd1440a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.928+0000 7fd14e600640 1 -- 172.21.15.47:0/2916634944 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154016070 con 0x55f041c04e80 2024-03-21T11:32:52.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.928+0000 7fd14e600640 1 -- 172.21.15.47:0/2916634944 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd154004d20 con 0x55f041c04e80 2024-03-21T11:32:52.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.928+0000 7fd15a000640 1 -- 172.21.15.47:0/2007663470 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd1500f19e0 con 0x7fd15c102180 2024-03-21T11:32:52.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.928+0000 7fd14e600640 1 -- 172.21.15.47:0/2916634944 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154012910 con 0x55f041c04e80 2024-03-21T11:32:52.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.928+0000 7fd161400640 1 --2- 172.21.15.47:0/2007663470 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1440a2fa0 0x7fd1440a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.929+0000 7fd166a77740 1 -- 172.21.15.47:0/2916634944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c04e80 msgr2=0x55f041c052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.929+0000 7fd166a77740 1 --2- 172.21.15.47:0/2916634944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c04e80 0x55f041c052a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd154009f70 tx=0x7fd154010800 comp rx=0 tx=0).stop 2024-03-21T11:32:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.929+0000 7fd161400640 1 --2- 172.21.15.47:0/2007663470 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1440a2fa0 0x7fd1440a5460 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd1500bda20 tx=0x7fd1500de040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.929+0000 7fd166a77740 1 -- 172.21.15.47:0/2916634944 shutdown_connections 2024-03-21T11:32:52.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.929+0000 7fd166a77740 1 --2- 172.21.15.47:0/2916634944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c0a9e0 0x55f041c0ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.929+0000 7fd166a77740 1 --2- 172.21.15.47:0/2916634944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c05870 0x55f041c0a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.929+0000 7fd166a77740 1 --2- 172.21.15.47:0/2916634944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c04e80 0x55f041c052a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.929+0000 7fd166a77740 1 -- 172.21.15.47:0/2916634944 >> 172.21.15.47:0/2916634944 conn(0x55f041bcd200 msgr2=0x55f041c007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:52.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.930+0000 7fd166a77740 1 -- 172.21.15.47:0/2916634944 shutdown_connections 2024-03-21T11:32:52.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.930+0000 7fd166a77740 1 -- 172.21.15.47:0/2916634944 wait complete. 2024-03-21T11:32:52.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.930+0000 7fd166a77740 1 -- start start 2024-03-21T11:32:52.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.930+0000 7fd166a77740 1 Processor -- start 2024-03-21T11:32:52.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.931+0000 7fd166a77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c04e80 0x55f041c0ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.931+0000 7fd166a77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c05870 0x55f041c0f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.931+0000 7fd166a77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c0a9e0 0x55f041c16400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.931+0000 7fd166a77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f041c0d990 con 0x55f041c0a9e0 2024-03-21T11:32:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.931+0000 7fd166a77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f041b25e80 con 0x55f041c04e80 2024-03-21T11:32:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.931+0000 7fd166a77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f041c0db10 con 0x55f041c05870 2024-03-21T11:32:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.931+0000 7fd166a77740 10 client.0 _ll_register_callbacks cb 0x55f041b38130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.931+0000 7fd160a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c05870 0x55f041c0f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.931+0000 7fd160a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c05870 0x55f041c0f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50476/0 (socket says 172.21.15.47:50476) 2024-03-21T11:32:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.931+0000 7fd160a00640 1 -- 172.21.15.47:0/638179408 learned_addr learned my addr 172.21.15.47:0/638179408 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.931+0000 7fd161e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c04e80 0x55f041c0ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.932+0000 7fd161e00640 1 --2- 172.21.15.47:0/638179408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c0a9e0 0x55f041c16400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.932+0000 7fd160a00640 1 -- 172.21.15.47:0/638179408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c04e80 msgr2=0x55f041c0ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.932+0000 7fd160a00640 1 --2- 172.21.15.47:0/638179408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c04e80 0x55f041c0ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.932+0000 7fd160a00640 1 -- 172.21.15.47:0/638179408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c0a9e0 msgr2=0x55f041c16400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.932+0000 7fd160a00640 1 --2- 172.21.15.47:0/638179408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c0a9e0 0x55f041c16400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.932+0000 7fd160a00640 1 -- 172.21.15.47:0/638179408 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0419487d0 con 0x55f041c05870 2024-03-21T11:32:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.932+0000 7fd160a00640 1 --2- 172.21.15.47:0/638179408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c05870 0x55f041c0f3c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd14800bf80 tx=0x7fd1480118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a09941f8d92a49e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.932+0000 7fd142a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.933+0000 7fd142a00640 1 -- 172.21.15.47:0/638179408 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148004020 con 0x55f041c05870 2024-03-21T11:32:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.933+0000 7fd166a77740 1 -- 172.21.15.47:0/638179408 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd154004d20 con 0x55f041c05870 2024-03-21T11:32:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.933+0000 7fd142a00640 1 -- 172.21.15.47:0/638179408 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd1480041c0 con 0x55f041c05870 2024-03-21T11:32:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.933+0000 7fd142a00640 1 -- 172.21.15.47:0/638179408 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd148024050 con 0x55f041c05870 2024-03-21T11:32:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.933+0000 7fd166a77740 10 client.28657 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:52.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.933+0000 7fd166a77740 1 -- 172.21.15.47:0/638179408 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f04187e3f0 con 0x55f041c05870 2024-03-21T11:32:52.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.933+0000 7fd142a00640 1 -- 172.21.15.47:0/638179408 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd154004d20 con 0x55f041c05870 2024-03-21T11:32:52.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.934+0000 7fd142a00640 1 -- 172.21.15.47:0/638179408 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd148004be0 con 0x55f041c05870 2024-03-21T11:32:52.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.934+0000 7fd166a77740 10 client.28657 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:52.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.934+0000 7fd166a77740 10 client.28657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:52.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.934+0000 7fd166a77740 20 client.28657 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:52.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.934+0000 7fd166a77740 1 --2- 172.21.15.47:0/638179408 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f041c1ad80 0x55f041c1d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.934+0000 7fd166a77740 4 client.28657 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:52.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.934+0000 7fd166a77740 1 -- 172.21.15.47:0/638179408 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f041b77790 con 0x55f041c1ad80 2024-03-21T11:32:52.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.934+0000 7fd161e00640 1 --2- 172.21.15.47:0/638179408 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f041c1ad80 0x55f041c1d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.935+0000 7fd161e00640 1 --2- 172.21.15.47:0/638179408 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f041c1ad80 0x55f041c1d1a0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc3a6588cae656f server_cookie=bcc5a84d36451dba in_seq=0 out_seq=0 2024-03-21T11:32:52.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.935+0000 7fd142a00640 10 client.28657 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:52.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.935+0000 7fd142a00640 1 -- 172.21.15.47:0/638179408 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f041b77790 con 0x55f041c1ad80 2024-03-21T11:32:52.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.935+0000 7fd142a00640 10 client.28657 handle_command_reply: tid=0 2024-03-21T11:32:52.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.936+0000 7fd166a77740 1 client.28657 shutdown 2024-03-21T11:32:52.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.936+0000 7fd166a77740 1 -- 172.21.15.47:0/638179408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c05870 msgr2=0x55f041c0f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:52.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.936+0000 7fd166a77740 1 --2- 172.21.15.47:0/638179408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c05870 0x55f041c0f3c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd14800bf80 tx=0x7fd1480118d0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.936+0000 7fd166a77740 1 -- 172.21.15.47:0/638179408 shutdown_connections 2024-03-21T11:32:52.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.936+0000 7fd166a77740 1 --2- 172.21.15.47:0/638179408 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f041c1ad80 0x55f041c1d1a0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.936+0000 7fd166a77740 1 --2- 172.21.15.47:0/638179408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c0a9e0 0x55f041c16400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.936+0000 7fd166a77740 1 --2- 172.21.15.47:0/638179408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c05870 0x55f041c0f3c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.936+0000 7fd166a77740 1 --2- 172.21.15.47:0/638179408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c04e80 0x55f041c0ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.936+0000 7fd166a77740 1 -- 172.21.15.47:0/638179408 >> 172.21.15.47:0/638179408 conn(0x55f041bcd200 msgr2=0x55f041bffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.936+0000 7fd142a00640 0 client.28657 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.936+0000 7fd166a77740 1 -- 172.21.15.47:0/638179408 shutdown_connections 2024-03-21T11:32:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.936+0000 7fd166a77740 1 -- 172.21.15.47:0/638179408 wait complete. 2024-03-21T11:32:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.936+0000 7fd166a77740 20 client.28657 trim_cache size 0 max 16384 2024-03-21T11:32:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.960+0000 7fd166a77740 1 Processor -- start 2024-03-21T11:32:52.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.960+0000 7fd166a77740 1 -- start start 2024-03-21T11:32:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd166a77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c04e80 0x55f041c09e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd166a77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c05870 0x55f041c163c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd166a77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c0a9e0 0x55f041c03a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd166a77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c1ac340 con 0x55f041c04e80 2024-03-21T11:32:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd166a77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c1adcc0 con 0x55f041c0a9e0 2024-03-21T11:32:52.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd166a77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c1ac9a0 con 0x55f041c05870 2024-03-21T11:32:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd160a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c05870 0x55f041c163c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd160a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c05870 0x55f041c163c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50488/0 (socket says 172.21.15.47:50488) 2024-03-21T11:32:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd160a00640 1 -- 172.21.15.47:0/3482361759 learned_addr learned my addr 172.21.15.47:0/3482361759 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd161e00640 1 --2- 172.21.15.47:0/3482361759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c04e80 0x55f041c09e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd161e00640 1 --2- 172.21.15.47:0/3482361759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c0a9e0 0x55f041c03a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd160a00640 1 -- 172.21.15.47:0/3482361759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c0a9e0 msgr2=0x55f041c03a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:52.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd160a00640 1 --2- 172.21.15.47:0/3482361759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c0a9e0 0x55f041c03a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd160a00640 1 -- 172.21.15.47:0/3482361759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c04e80 msgr2=0x55f041c09e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:52.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd160a00640 1 --2- 172.21.15.47:0/3482361759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c04e80 0x55f041c09e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.961+0000 7fd160a00640 1 -- 172.21.15.47:0/3482361759 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f04183e0f0 con 0x55f041c05870 2024-03-21T11:32:52.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.962+0000 7fd160a00640 1 --2- 172.21.15.47:0/3482361759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c05870 0x55f041c163c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd14800bf50 tx=0x7fd148011e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a474a85d3bfa0d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:52.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.962+0000 7fd14dc00640 1 -- 172.21.15.47:0/3482361759 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148005590 con 0x55f041c05870 2024-03-21T11:32:52.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.962+0000 7fd14dc00640 1 -- 172.21.15.47:0/3482361759 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd148010d60 con 0x55f041c05870 2024-03-21T11:32:52.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.962+0000 7fd14dc00640 1 -- 172.21.15.47:0/3482361759 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148004440 con 0x55f041c05870 2024-03-21T11:32:52.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.963+0000 7fd166a77740 1 -- 172.21.15.47:0/3482361759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c05870 msgr2=0x55f041c163c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:52.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.963+0000 7fd166a77740 1 --2- 172.21.15.47:0/3482361759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c05870 0x55f041c163c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd14800bf50 tx=0x7fd148011e50 comp rx=0 tx=0).stop 2024-03-21T11:32:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.963+0000 7fd166a77740 1 -- 172.21.15.47:0/3482361759 shutdown_connections 2024-03-21T11:32:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.963+0000 7fd166a77740 1 --2- 172.21.15.47:0/3482361759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c0a9e0 0x55f041c03a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.963+0000 7fd166a77740 1 --2- 172.21.15.47:0/3482361759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c05870 0x55f041c163c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.963+0000 7fd166a77740 1 --2- 172.21.15.47:0/3482361759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c04e80 0x55f041c09e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.963+0000 7fd166a77740 1 -- 172.21.15.47:0/3482361759 >> 172.21.15.47:0/3482361759 conn(0x55f041bcd200 msgr2=0x55f041c06ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.963+0000 7fd166a77740 1 -- 172.21.15.47:0/3482361759 shutdown_connections 2024-03-21T11:32:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.963+0000 7fd166a77740 1 -- 172.21.15.47:0/3482361759 wait complete. 2024-03-21T11:32:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.964+0000 7fd166a77740 1 -- start start 2024-03-21T11:32:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.964+0000 7fd166a77740 1 Processor -- start 2024-03-21T11:32:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.964+0000 7fd166a77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c04e80 0x55f041c15f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.964+0000 7fd166a77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c05870 0x55f041c14590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.964+0000 7fd166a77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c0a9e0 0x55f041c14ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.964+0000 7fd166a77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c1ad660 con 0x55f041c04e80 2024-03-21T11:32:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.964+0000 7fd166a77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c1ad000 con 0x55f041c05870 2024-03-21T11:32:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.964+0000 7fd166a77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c1aa500 con 0x55f041c0a9e0 2024-03-21T11:32:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.964+0000 7fd166a77740 10 client.0 _ll_register_callbacks cb 0x7fd14800bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.965+0000 7fd161e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c04e80 0x55f041c15f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.965+0000 7fd160a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c05870 0x55f041c14590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.965+0000 7fd160a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c05870 0x55f041c14590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60744/0 (socket says 172.21.15.47:60744) 2024-03-21T11:32:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.965+0000 7fd160a00640 1 -- 172.21.15.47:0/3954792539 learned_addr learned my addr 172.21.15.47:0/3954792539 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.965+0000 7fd161e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c04e80 0x55f041c15f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33280/0 (socket says 172.21.15.47:33280) 2024-03-21T11:32:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.965+0000 7fd161e00640 1 --2- 172.21.15.47:0/3954792539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c0a9e0 0x55f041c14ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.965+0000 7fd161e00640 1 -- 172.21.15.47:0/3954792539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c0a9e0 msgr2=0x55f041c14ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.965+0000 7fd161e00640 1 --2- 172.21.15.47:0/3954792539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c0a9e0 0x55f041c14ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.965+0000 7fd161e00640 1 -- 172.21.15.47:0/3954792539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c05870 msgr2=0x55f041c14590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.965+0000 7fd161e00640 1 --2- 172.21.15.47:0/3954792539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c05870 0x55f041c14590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.965+0000 7fd161e00640 1 -- 172.21.15.47:0/3954792539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f041c15010 con 0x55f041c04e80 2024-03-21T11:32:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.965+0000 7fd160a00640 1 --2- 172.21.15.47:0/3954792539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c05870 0x55f041c14590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.965+0000 7fd161e00640 1 --2- 172.21.15.47:0/3954792539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c04e80 0x55f041c15f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd154000f80 tx=0x7fd154010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afaa193a5216fbca server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.965+0000 7fd142a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:32:52.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.966+0000 7fd142a00640 1 -- 172.21.15.47:0/3954792539 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154016040 con 0x55f041c04e80 2024-03-21T11:32:52.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.966+0000 7fd142a00640 1 -- 172.21.15.47:0/3954792539 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd15402fb40 con 0x55f041c04e80 2024-03-21T11:32:52.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.966+0000 7fd166a77740 1 -- 172.21.15.47:0/3954792539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd148010d60 con 0x55f041c04e80 2024-03-21T11:32:52.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.966+0000 7fd142a00640 1 -- 172.21.15.47:0/3954792539 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd154023040 con 0x55f041c04e80 2024-03-21T11:32:52.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.966+0000 7fd166a77740 10 client.18318 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:52.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.966+0000 7fd166a77740 1 -- 172.21.15.47:0/3954792539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f041c77350 con 0x55f041c04e80 2024-03-21T11:32:52.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.966+0000 7fd142a00640 1 -- 172.21.15.47:0/3954792539 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd148010d60 con 0x55f041c04e80 2024-03-21T11:32:52.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.966+0000 7fd142a00640 1 -- 172.21.15.47:0/3954792539 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd154029040 con 0x55f041c04e80 2024-03-21T11:32:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.967+0000 7fd166a77740 10 client.18318 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.967+0000 7fd166a77740 10 client.18318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.967+0000 7fd166a77740 20 client.18318 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.967+0000 7fd166a77740 1 --2- 172.21.15.47:0/3954792539 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f041c77af0 0x55f041c79f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.967+0000 7fd166a77740 4 client.18318 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:32:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.967+0000 7fd166a77740 1 -- 172.21.15.47:0/3954792539 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f041b71ae0 con 0x55f041c77af0 2024-03-21T11:32:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.967+0000 7fd160a00640 1 --2- 172.21.15.47:0/3954792539 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f041c77af0 0x55f041c79f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.967+0000 7fd160a00640 1 --2- 172.21.15.47:0/3954792539 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f041c77af0 0x55f041c79f10 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f7202ed0e74d379 server_cookie=81880ecf90eac6e4 in_seq=0 out_seq=0 2024-03-21T11:32:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.967+0000 7fd142a00640 10 client.18318 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd142a00640 1 -- 172.21.15.47:0/3954792539 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f041b71ae0 con 0x55f041c77af0 2024-03-21T11:32:52.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd142a00640 10 client.18318 handle_command_reply: tid=0 2024-03-21T11:32:52.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd166a77740 1 client.18318 shutdown 2024-03-21T11:32:52.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd166a77740 1 -- 172.21.15.47:0/3954792539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c04e80 msgr2=0x55f041c15f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:52.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd166a77740 1 --2- 172.21.15.47:0/3954792539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c04e80 0x55f041c15f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd154000f80 tx=0x7fd154010800 comp rx=0 tx=0).stop 2024-03-21T11:32:52.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd166a77740 1 -- 172.21.15.47:0/3954792539 shutdown_connections 2024-03-21T11:32:52.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd166a77740 1 --2- 172.21.15.47:0/3954792539 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f041c77af0 0x55f041c79f10 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd166a77740 1 --2- 172.21.15.47:0/3954792539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f041c0a9e0 0x55f041c14ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd166a77740 1 --2- 172.21.15.47:0/3954792539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f041c05870 0x55f041c14590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd166a77740 1 --2- 172.21.15.47:0/3954792539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f041c04e80 0x55f041c15f80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd166a77740 1 -- 172.21.15.47:0/3954792539 >> 172.21.15.47:0/3954792539 conn(0x55f041bcd200 msgr2=0x55f041c06ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd142a00640 0 client.18318 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd166a77740 1 -- 172.21.15.47:0/3954792539 shutdown_connections 2024-03-21T11:32:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd166a77740 1 -- 172.21.15.47:0/3954792539 wait complete. 2024-03-21T11:32:52.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.968+0000 7fd166a77740 20 client.18318 trim_cache size 0 max 16384 2024-03-21T11:32:52.977 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:32:52.977 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:32:52.977 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:32:52.977 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:32:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.969+0000 7fd163e00640 1 -- 172.21.15.47:0/2007663470 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1440a2fa0 msgr2=0x7fd1440a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:52.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.969+0000 7fd163e00640 1 --2- 172.21.15.47:0/2007663470 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1440a2fa0 0x7fd1440a5460 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd1500bda20 tx=0x7fd1500de040 comp rx=0 tx=0).stop 2024-03-21T11:32:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.969+0000 7fd163e00640 1 -- 172.21.15.47:0/2007663470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd15c102180 msgr2=0x7fd15c199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.969+0000 7fd163e00640 1 --2- 172.21.15.47:0/2007663470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd15c102180 0x7fd15c199c40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd150000c00 tx=0x7fd150005dd0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.969+0000 7fd163e00640 1 -- 172.21.15.47:0/2007663470 shutdown_connections 2024-03-21T11:32:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.969+0000 7fd163e00640 1 --2- 172.21.15.47:0/2007663470 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1440a2fa0 0x7fd1440a5460 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.969+0000 7fd163e00640 1 --2- 172.21.15.47:0/2007663470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd15c1073e0 0x7fd15c1a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.969+0000 7fd163e00640 1 --2- 172.21.15.47:0/2007663470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd15c104ab0 0x7fd15c19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.969+0000 7fd163e00640 1 --2- 172.21.15.47:0/2007663470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd15c102180 0x7fd15c199c40 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.969+0000 7fd163e00640 1 -- 172.21.15.47:0/2007663470 >> 172.21.15.47:0/2007663470 conn(0x7fd15c0fbb30 msgr2=0x7fd15c101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.969+0000 7fd163e00640 1 -- 172.21.15.47:0/2007663470 shutdown_connections 2024-03-21T11:32:52.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:52.969+0000 7fd163e00640 1 -- 172.21.15.47:0/2007663470 wait complete. 2024-03-21T11:32:52.987 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:32:52.988 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:32:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.200+0000 7feecf200640 1 -- 172.21.15.47:0/708280851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feec8076060 msgr2=0x7feec8076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.200+0000 7feecf200640 1 --2- 172.21.15.47:0/708280851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feec8076060 0x7feec8076480 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009900 tx=0x7feeb802bc50 comp rx=0 tx=0).stop 2024-03-21T11:32:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.200+0000 7feecf200640 1 -- 172.21.15.47:0/708280851 shutdown_connections 2024-03-21T11:32:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.200+0000 7feecf200640 1 --2- 172.21.15.47:0/708280851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec8077c40 0x7feec80784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.200+0000 7feecf200640 1 --2- 172.21.15.47:0/708280851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec80772a0 0x7feec8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.200+0000 7feecf200640 1 --2- 172.21.15.47:0/708280851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feec8076060 0x7feec8076480 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.200+0000 7feecf200640 1 -- 172.21.15.47:0/708280851 >> 172.21.15.47:0/708280851 conn(0x7feec80fee50 msgr2=0x7feec8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.201+0000 7feecf200640 1 -- 172.21.15.47:0/708280851 shutdown_connections 2024-03-21T11:32:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.201+0000 7feecf200640 1 -- 172.21.15.47:0/708280851 wait complete. 2024-03-21T11:32:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.201+0000 7feecf200640 1 Processor -- start 2024-03-21T11:32:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.202+0000 7feecf200640 1 -- start start 2024-03-21T11:32:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.202+0000 7feecf200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec8076060 0x7feec819e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.202+0000 7feecf200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feec80772a0 0x7feec819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.202+0000 7feecf200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec8077c40 0x7feec81a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.202+0000 7feecf200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec8113970 con 0x7feec8077c40 2024-03-21T11:32:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.202+0000 7feecf200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec81137f0 con 0x7feec8076060 2024-03-21T11:32:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.202+0000 7feecf200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec8113af0 con 0x7feec80772a0 2024-03-21T11:32:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.202+0000 7feec7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec8076060 0x7feec819e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.202+0000 7feec7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec8076060 0x7feec819e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60768/0 (socket says 172.21.15.47:60768) 2024-03-21T11:32:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.202+0000 7feec7e00640 1 -- 172.21.15.47:0/2246166564 learned_addr learned my addr 172.21.15.47:0/2246166564 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.202+0000 7feec7400640 1 --2- 172.21.15.47:0/2246166564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feec80772a0 0x7feec819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.202+0000 7feecd200640 1 --2- 172.21.15.47:0/2246166564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec8077c40 0x7feec81a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.203+0000 7feec7e00640 1 -- 172.21.15.47:0/2246166564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feec80772a0 msgr2=0x7feec819e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.203+0000 7feec7e00640 1 --2- 172.21.15.47:0/2246166564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feec80772a0 0x7feec819e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.203+0000 7feec7e00640 1 -- 172.21.15.47:0/2246166564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec8077c40 msgr2=0x7feec81a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.203+0000 7feec7e00640 1 --2- 172.21.15.47:0/2246166564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec8077c40 0x7feec81a55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.203+0000 7feec7e00640 1 -- 172.21.15.47:0/2246166564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec81a5c60 con 0x7feec8076060 2024-03-21T11:32:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.203+0000 7feec7e00640 1 --2- 172.21.15.47:0/2246166564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec8076060 0x7feec819e040 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009230 tx=0x7feeb8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.203+0000 7feec4c00640 1 -- 172.21.15.47:0/2246166564 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8042020 con 0x7feec8076060 2024-03-21T11:32:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.203+0000 7feec4c00640 1 -- 172.21.15.47:0/2246166564 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feeb80044d0 con 0x7feec8076060 2024-03-21T11:32:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.203+0000 7feecf200640 1 -- 172.21.15.47:0/2246166564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec81a5ef0 con 0x7feec8076060 2024-03-21T11:32:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.203+0000 7feec4c00640 1 -- 172.21.15.47:0/2246166564 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb803e460 con 0x7feec8076060 2024-03-21T11:32:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.204+0000 7feecf200640 1 -- 172.21.15.47:0/2246166564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec81a6350 con 0x7feec8076060 2024-03-21T11:32:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.205+0000 7feecf200640 1 -- 172.21.15.47:0/2246166564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec81103f0 con 0x7feec8076060 2024-03-21T11:32:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.207+0000 7feec4c00640 1 -- 172.21.15.47:0/2246166564 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7feeb8045020 con 0x7feec8076060 2024-03-21T11:32:53.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.208+0000 7feec4c00640 1 --2- 172.21.15.47:0/2246166564 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feea80a2f20 0x7feea80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.208+0000 7feec4c00640 1 -- 172.21.15.47:0/2246166564 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7feeb80f1820 con 0x7feec8076060 2024-03-21T11:32:53.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.208+0000 7feec7400640 1 --2- 172.21.15.47:0/2246166564 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feea80a2f20 0x7feea80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.209+0000 7feec7400640 1 --2- 172.21.15.47:0/2246166564 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feea80a2f20 0x7feea80a53e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7feec819f550 tx=0x7feeb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:53.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.215+0000 7feec4c00640 1 -- 172.21.15.47:0/2246166564 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feeb804c020 con 0x7feec8076060 2024-03-21T11:32:53.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.543+0000 7feecf200640 1 -- 172.21.15.47:0/2246166564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feec802d070 con 0x7feec8076060 2024-03-21T11:32:53.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.544+0000 7feec4c00640 1 -- 172.21.15.47:0/2246166564 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7feeb80be260 con 0x7feec8076060 2024-03-21T11:32:53.546 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:32:53.546 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:32:53.546 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:32:53.547 INFO:tasks.workunit.client.0.smithi047.stdout: 1 701010 8.14 MB/sec execute 337 sec latency 80.904 ms 2024-03-21T11:32:53.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.547+0000 7feecf200640 1 -- 172.21.15.47:0/2246166564 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feea80a2f20 msgr2=0x7feea80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:53.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.547+0000 7feecf200640 1 --2- 172.21.15.47:0/2246166564 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feea80a2f20 0x7feea80a53e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7feec819f550 tx=0x7feeb0005f90 comp rx=0 tx=0).stop 2024-03-21T11:32:53.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.547+0000 7feecf200640 1 -- 172.21.15.47:0/2246166564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec8076060 msgr2=0x7feec819e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:53.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.547+0000 7feecf200640 1 --2- 172.21.15.47:0/2246166564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec8076060 0x7feec819e040 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009230 tx=0x7feeb8005d50 comp rx=0 tx=0).stop 2024-03-21T11:32:53.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.547+0000 7feecf200640 1 -- 172.21.15.47:0/2246166564 shutdown_connections 2024-03-21T11:32:53.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.547+0000 7feecf200640 1 --2- 172.21.15.47:0/2246166564 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feea80a2f20 0x7feea80a53e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.547+0000 7feecf200640 1 --2- 172.21.15.47:0/2246166564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feec8077c40 0x7feec81a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.547+0000 7feecf200640 1 --2- 172.21.15.47:0/2246166564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feec80772a0 0x7feec819e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.547+0000 7feecf200640 1 --2- 172.21.15.47:0/2246166564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec8076060 0x7feec819e040 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.547+0000 7feecf200640 1 -- 172.21.15.47:0/2246166564 >> 172.21.15.47:0/2246166564 conn(0x7feec80fee50 msgr2=0x7feec81005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:53.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.548+0000 7feecf200640 1 -- 172.21.15.47:0/2246166564 shutdown_connections 2024-03-21T11:32:53.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.548+0000 7feecf200640 1 -- 172.21.15.47:0/2246166564 wait complete. 2024-03-21T11:32:53.568 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:32:53.568 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:32:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.774+0000 7f0aa2400640 1 -- 172.21.15.47:0/2331576816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c1016b0 msgr2=0x7f0a9c101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.774+0000 7f0aa2400640 1 --2- 172.21.15.47:0/2331576816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c1016b0 0x7f0a9c101ad0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009860 tx=0x7f0a8c02bb70 comp rx=0 tx=0).stop 2024-03-21T11:32:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.775+0000 7f0aa2400640 1 -- 172.21.15.47:0/2331576816 shutdown_connections 2024-03-21T11:32:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.775+0000 7f0aa2400640 1 --2- 172.21.15.47:0/2331576816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a9c103260 0x7f0a9c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.775+0000 7f0aa2400640 1 --2- 172.21.15.47:0/2331576816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a9c1028a0 0x7f0a9c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.775+0000 7f0aa2400640 1 --2- 172.21.15.47:0/2331576816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c1016b0 0x7f0a9c101ad0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.775+0000 7f0aa2400640 1 -- 172.21.15.47:0/2331576816 >> 172.21.15.47:0/2331576816 conn(0x7f0a9c0fcc40 msgr2=0x7f0a9c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.775+0000 7f0aa2400640 1 -- 172.21.15.47:0/2331576816 shutdown_connections 2024-03-21T11:32:53.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.775+0000 7f0aa2400640 1 -- 172.21.15.47:0/2331576816 wait complete. 2024-03-21T11:32:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.776+0000 7f0aa2400640 1 Processor -- start 2024-03-21T11:32:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.776+0000 7f0aa2400640 1 -- start start 2024-03-21T11:32:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.776+0000 7f0aa2400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a9c1016b0 0x7f0a9c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.776+0000 7f0aa2400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c1028a0 0x7f0a9c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.776+0000 7f0aa2400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a9c103260 0x7f0a9c1a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.776+0000 7f0aa2400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c10d330 con 0x7f0a9c103260 2024-03-21T11:32:53.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.776+0000 7f0aa2400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c10d1b0 con 0x7f0a9c1028a0 2024-03-21T11:32:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.776+0000 7f0aa2400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c10d4b0 con 0x7f0a9c1016b0 2024-03-21T11:32:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.777+0000 7f0a9b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a9c1016b0 0x7f0a9c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.777+0000 7f0a9b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a9c1016b0 0x7f0a9c199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50528/0 (socket says 172.21.15.47:50528) 2024-03-21T11:32:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.777+0000 7f0a9aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c1028a0 0x7f0a9c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.777+0000 7f0a9b400640 1 -- 172.21.15.47:0/1498246076 learned_addr learned my addr 172.21.15.47:0/1498246076 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.777+0000 7f0a9be00640 1 --2- 172.21.15.47:0/1498246076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a9c103260 0x7f0a9c1a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.777+0000 7f0a9b400640 1 -- 172.21.15.47:0/1498246076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c1028a0 msgr2=0x7f0a9c199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.777+0000 7f0a9b400640 1 --2- 172.21.15.47:0/1498246076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c1028a0 0x7f0a9c199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.777+0000 7f0a9b400640 1 -- 172.21.15.47:0/1498246076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a9c103260 msgr2=0x7f0a9c1a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.777+0000 7f0a9b400640 1 --2- 172.21.15.47:0/1498246076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a9c103260 0x7f0a9c1a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.777+0000 7f0a9b400640 1 -- 172.21.15.47:0/1498246076 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c1a1630 con 0x7f0a9c1016b0 2024-03-21T11:32:53.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.777+0000 7f0a9b400640 1 --2- 172.21.15.47:0/1498246076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a9c1016b0 0x7f0a9c199a10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c02c180 tx=0x7f0a8c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.778+0000 7f0a93e00640 1 -- 172.21.15.47:0/1498246076 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c041020 con 0x7f0a9c1016b0 2024-03-21T11:32:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.778+0000 7f0a93e00640 1 -- 172.21.15.47:0/1498246076 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a8c0044b0 con 0x7f0a9c1016b0 2024-03-21T11:32:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.778+0000 7f0a93e00640 1 -- 172.21.15.47:0/1498246076 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c039040 con 0x7f0a9c1016b0 2024-03-21T11:32:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.778+0000 7f0aa2400640 1 -- 172.21.15.47:0/1498246076 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c1a18c0 con 0x7f0a9c1016b0 2024-03-21T11:32:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.778+0000 7f0aa2400640 1 -- 172.21.15.47:0/1498246076 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c1a1db0 con 0x7f0a9c1016b0 2024-03-21T11:32:53.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.781+0000 7f0a93e00640 1 -- 172.21.15.47:0/1498246076 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0a8c044020 con 0x7f0a9c1016b0 2024-03-21T11:32:53.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.781+0000 7f0aa5193740 1 Processor -- start 2024-03-21T11:32:53.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.781+0000 7f0aa5193740 1 -- start start 2024-03-21T11:32:53.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.782+0000 7f0aa5193740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83c810 0x56387a83cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.782+0000 7f0aa5193740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83d9e0 0x56387a83de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.782+0000 7f0aa5193740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83e3a0 0x56387a844c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.782+0000 7f0aa5193740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387a7a12c0 con 0x56387a83c810 2024-03-21T11:32:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.782+0000 7f0aa5193740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387a749c90 con 0x56387a83d9e0 2024-03-21T11:32:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.782+0000 7f0aa5193740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387a79d360 con 0x56387a83e3a0 2024-03-21T11:32:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.782+0000 7f0a9aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83e3a0 0x56387a844c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.782+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83c810 0x56387a83cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.782+0000 7f0a9aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83e3a0 0x56387a844c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50544/0 (socket says 172.21.15.47:50544) 2024-03-21T11:32:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.782+0000 7f0a9aa00640 1 -- 172.21.15.47:0/3933109212 learned_addr learned my addr 172.21.15.47:0/3933109212 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.782+0000 7f0a9b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83d9e0 0x56387a83de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.783+0000 7f0a9aa00640 1 -- 172.21.15.47:0/3933109212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83d9e0 msgr2=0x56387a83de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:53.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.783+0000 7f0a9aa00640 1 --2- 172.21.15.47:0/3933109212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83d9e0 0x56387a83de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.783+0000 7f0a9aa00640 1 -- 172.21.15.47:0/3933109212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83c810 msgr2=0x56387a83cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:53.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.783+0000 7f0a9aa00640 1 --2- 172.21.15.47:0/3933109212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83c810 0x56387a83cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.783+0000 7f0a9aa00640 1 -- 172.21.15.47:0/3933109212 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56387a76e080 con 0x56387a83e3a0 2024-03-21T11:32:53.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.783+0000 7f0a9aa00640 1 --2- 172.21.15.47:0/3933109212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83e3a0 0x56387a844c30 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009c90 tx=0x7f0a88011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad503e54a91a00e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:53.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.783+0000 7f0a93e00640 1 --2- 172.21.15.47:0/1498246076 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a800a2f20 0x7f0a800a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.783+0000 7f0a90c00640 1 -- 172.21.15.47:0/3933109212 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8801e070 con 0x56387a83e3a0 2024-03-21T11:32:53.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.784+0000 7f0a93e00640 1 -- 172.21.15.47:0/1498246076 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0a8c0f1320 con 0x7f0a9c1016b0 2024-03-21T11:32:53.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.784+0000 7f0a90c00640 1 -- 172.21.15.47:0/3933109212 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a88019b10 con 0x56387a83e3a0 2024-03-21T11:32:53.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.784+0000 7f0a90c00640 1 -- 172.21.15.47:0/3933109212 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8800ea70 con 0x56387a83e3a0 2024-03-21T11:32:53.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.784+0000 7f0a9be00640 1 --2- 172.21.15.47:0/1498246076 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a800a2f20 0x7f0a800a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.784+0000 7f0aa5193740 1 -- 172.21.15.47:0/3933109212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83e3a0 msgr2=0x56387a844c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:53.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.784+0000 7f0aa5193740 1 --2- 172.21.15.47:0/3933109212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83e3a0 0x56387a844c30 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009c90 tx=0x7f0a88011810 comp rx=0 tx=0).stop 2024-03-21T11:32:53.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.785+0000 7f0aa5193740 1 -- 172.21.15.47:0/3933109212 shutdown_connections 2024-03-21T11:32:53.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.785+0000 7f0aa5193740 1 --2- 172.21.15.47:0/3933109212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83e3a0 0x56387a844c30 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.785+0000 7f0aa5193740 1 --2- 172.21.15.47:0/3933109212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83d9e0 0x56387a83de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.785+0000 7f0aa5193740 1 --2- 172.21.15.47:0/3933109212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83c810 0x56387a83cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.785+0000 7f0a9be00640 1 --2- 172.21.15.47:0/1498246076 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a800a2f20 0x7f0a800a53e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x56387a83d8e0 tx=0x7f0a94010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:53.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.785+0000 7f0aa5193740 1 -- 172.21.15.47:0/3933109212 >> 172.21.15.47:0/3933109212 conn(0x56387a837c00 msgr2=0x56387a838000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:53.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.785+0000 7f0aa5193740 1 -- 172.21.15.47:0/3933109212 shutdown_connections 2024-03-21T11:32:53.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.785+0000 7f0aa5193740 1 -- 172.21.15.47:0/3933109212 wait complete. 2024-03-21T11:32:53.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.785+0000 7f0aa5193740 1 -- start start 2024-03-21T11:32:53.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.786+0000 7f0aa5193740 1 Processor -- start 2024-03-21T11:32:53.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.786+0000 7f0aa5193740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83c810 0x56387a846a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.786+0000 7f0aa5193740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 0x56387a846fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.786+0000 7f0aa5193740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83e3a0 0x56387a84dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.786+0000 7f0aa5193740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387a8453b0 con 0x56387a83d9e0 2024-03-21T11:32:53.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.786+0000 7f0aa5193740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387a75d0d0 con 0x56387a83e3a0 2024-03-21T11:32:53.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.786+0000 7f0aa5193740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387a845530 con 0x56387a83c810 2024-03-21T11:32:53.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.786+0000 7f0aa5193740 10 client.0 _ll_register_callbacks cb 0x56387a761e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:53.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.786+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83e3a0 0x56387a84dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.787+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83e3a0 0x56387a84dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60808/0 (socket says 172.21.15.47:60808) 2024-03-21T11:32:53.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.787+0000 7f0a9be00640 1 -- 172.21.15.47:0/2266574302 learned_addr learned my addr 172.21.15.47:0/2266574302 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:53.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.787+0000 7f0a9b400640 1 --2- 172.21.15.47:0/2266574302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83c810 0x56387a846a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.787+0000 7f0a9aa00640 1 --2- 172.21.15.47:0/2266574302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 0x56387a846fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.787+0000 7f0a9be00640 1 -- 172.21.15.47:0/2266574302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83c810 msgr2=0x56387a846a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:53.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.787+0000 7f0a9be00640 1 --2- 172.21.15.47:0/2266574302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83c810 0x56387a846a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.787+0000 7f0a9be00640 1 -- 172.21.15.47:0/2266574302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 msgr2=0x56387a846fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:53.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.787+0000 7f0a9be00640 1 --2- 172.21.15.47:0/2266574302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 0x56387a846fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.787+0000 7f0a9be00640 1 -- 172.21.15.47:0/2266574302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56387a4b53f0 con 0x56387a83e3a0 2024-03-21T11:32:53.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.787+0000 7f0a9aa00640 1 --2- 172.21.15.47:0/2266574302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 0x56387a846fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:53.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.788+0000 7f0a9be00640 1 --2- 172.21.15.47:0/2266574302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83e3a0 0x56387a84dfe0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0a940099d0 tx=0x7f0a94004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a790e54595527cc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:53.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.788+0000 7f0a85600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:32:53.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.788+0000 7f0a85600640 1 -- 172.21.15.47:0/2266574302 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9402a060 con 0x56387a83e3a0 2024-03-21T11:32:53.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.788+0000 7f0aa5193740 1 -- 172.21.15.47:0/2266574302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a88019b10 con 0x56387a83e3a0 2024-03-21T11:32:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.788+0000 7f0a85600640 1 -- 172.21.15.47:0/2266574302 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0a9401a450 con 0x56387a83e3a0 2024-03-21T11:32:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.788+0000 7f0a85600640 1 -- 172.21.15.47:0/2266574302 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a94031050 con 0x56387a83e3a0 2024-03-21T11:32:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.788+0000 7f0aa5193740 10 client.28673 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.788+0000 7f0aa5193740 1 -- 172.21.15.47:0/2266574302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56387a57f7d0 con 0x56387a83e3a0 2024-03-21T11:32:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.789+0000 7f0a85600640 1 -- 172.21.15.47:0/2266574302 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a88019b10 con 0x56387a83e3a0 2024-03-21T11:32:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.789+0000 7f0a85600640 1 -- 172.21.15.47:0/2266574302 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0a9400b040 con 0x56387a83e3a0 2024-03-21T11:32:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.789+0000 7f0aa5193740 10 client.28673 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.789+0000 7f0aa5193740 10 client.28673 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:32:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.789+0000 7f0aa5193740 20 client.28673 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.789+0000 7f0aa5193740 1 --2- 172.21.15.47:0/2266574302 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x56387a852520 0x56387a8549c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.789+0000 7f0aa5193740 4 client.28673 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.789+0000 7f0aa5193740 1 -- 172.21.15.47:0/2266574302 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56387a7ae800 con 0x56387a852520 2024-03-21T11:32:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.790+0000 7f0a9b400640 1 --2- 172.21.15.47:0/2266574302 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x56387a852520 0x56387a8549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.790+0000 7f0a9b400640 1 --2- 172.21.15.47:0/2266574302 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x56387a852520 0x56387a8549c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ff5de4dbdd4493ac server_cookie=9a65bfdf329b3f06 in_seq=0 out_seq=0 2024-03-21T11:32:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.790+0000 7f0a85600640 10 client.28673 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:32:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.791+0000 7f0a85600640 1 -- 172.21.15.47:0/2266574302 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56387a7ae800 con 0x56387a852520 2024-03-21T11:32:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.791+0000 7f0a85600640 10 client.28673 handle_command_reply: tid=0 2024-03-21T11:32:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.791+0000 7f0aa5193740 1 client.28673 shutdown 2024-03-21T11:32:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.791+0000 7f0aa5193740 1 -- 172.21.15.47:0/2266574302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83e3a0 msgr2=0x56387a84dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.791+0000 7f0aa5193740 1 --2- 172.21.15.47:0/2266574302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83e3a0 0x56387a84dfe0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0a940099d0 tx=0x7f0a94004ab0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.791+0000 7f0aa5193740 1 -- 172.21.15.47:0/2266574302 shutdown_connections 2024-03-21T11:32:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.791+0000 7f0aa5193740 1 --2- 172.21.15.47:0/2266574302 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x56387a852520 0x56387a8549c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.791+0000 7f0aa5193740 1 --2- 172.21.15.47:0/2266574302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83e3a0 0x56387a84dfe0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.791+0000 7f0aa5193740 1 --2- 172.21.15.47:0/2266574302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 0x56387a846fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.791+0000 7f0aa5193740 1 --2- 172.21.15.47:0/2266574302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83c810 0x56387a846a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.791+0000 7f0a85600640 0 client.28673 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:32:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.791+0000 7f0aa5193740 1 -- 172.21.15.47:0/2266574302 >> 172.21.15.47:0/2266574302 conn(0x56387a837c00 msgr2=0x56387a836eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:53.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.792+0000 7f0aa5193740 1 -- 172.21.15.47:0/2266574302 shutdown_connections 2024-03-21T11:32:53.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.792+0000 7f0aa5193740 1 -- 172.21.15.47:0/2266574302 wait complete. 2024-03-21T11:32:53.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.792+0000 7f0aa5193740 20 client.28673 trim_cache size 0 max 16384 2024-03-21T11:32:53.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.807+0000 7f0aa5193740 1 Processor -- start 2024-03-21T11:32:53.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.807+0000 7f0aa5193740 1 -- start start 2024-03-21T11:32:53.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.807+0000 7f0aa5193740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83c810 0x56387a8448f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0aa5193740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 0x56387a84dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0aa5193740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83e3a0 0x56387a840a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0aa5193740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1ac120 con 0x56387a83d9e0 2024-03-21T11:32:53.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0aa5193740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1adaa0 con 0x56387a83e3a0 2024-03-21T11:32:53.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0aa5193740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1ac780 con 0x56387a83c810 2024-03-21T11:32:53.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0a9b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83c810 0x56387a8448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83e3a0 0x56387a840a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0a9b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83c810 0x56387a8448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50558/0 (socket says 172.21.15.47:50558) 2024-03-21T11:32:53.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0a9b400640 1 -- 172.21.15.47:0/3626585810 learned_addr learned my addr 172.21.15.47:0/3626585810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:53.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83e3a0 0x56387a840a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60822/0 (socket says 172.21.15.47:60822) 2024-03-21T11:32:53.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0a9aa00640 1 --2- 172.21.15.47:0/3626585810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 0x56387a84dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0a9be00640 1 -- 172.21.15.47:0/3626585810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83c810 msgr2=0x56387a8448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:53.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0a9be00640 1 --2- 172.21.15.47:0/3626585810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83c810 0x56387a8448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0a9be00640 1 -- 172.21.15.47:0/3626585810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 msgr2=0x56387a84dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:53.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0a9be00640 1 --2- 172.21.15.47:0/3626585810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 0x56387a84dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.808+0000 7f0a9be00640 1 -- 172.21.15.47:0/3626585810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56387a838970 con 0x56387a83e3a0 2024-03-21T11:32:53.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.809+0000 7f0a9b400640 1 --2- 172.21.15.47:0/3626585810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83c810 0x56387a8448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:53.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.809+0000 7f0a9be00640 1 --2- 172.21.15.47:0/3626585810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83e3a0 0x56387a840a80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400a010 tx=0x7f0a94009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6694c51547dc6889 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:53.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.809+0000 7f0a87e00640 1 -- 172.21.15.47:0/3626585810 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94039070 con 0x56387a83e3a0 2024-03-21T11:32:53.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.809+0000 7f0a87e00640 1 -- 172.21.15.47:0/3626585810 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a94021040 con 0x56387a83e3a0 2024-03-21T11:32:53.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.809+0000 7f0a87e00640 1 -- 172.21.15.47:0/3626585810 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94005020 con 0x56387a83e3a0 2024-03-21T11:32:53.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.810+0000 7f0aa5193740 1 -- 172.21.15.47:0/3626585810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83e3a0 msgr2=0x56387a840a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:53.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.810+0000 7f0aa5193740 1 --2- 172.21.15.47:0/3626585810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83e3a0 0x56387a840a80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400a010 tx=0x7f0a94009280 comp rx=0 tx=0).stop 2024-03-21T11:32:53.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.810+0000 7f0aa5193740 1 -- 172.21.15.47:0/3626585810 shutdown_connections 2024-03-21T11:32:53.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.810+0000 7f0aa5193740 1 --2- 172.21.15.47:0/3626585810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83e3a0 0x56387a840a80 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.810+0000 7f0aa5193740 1 --2- 172.21.15.47:0/3626585810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 0x56387a84dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.810+0000 7f0aa5193740 1 --2- 172.21.15.47:0/3626585810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83c810 0x56387a8448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.810+0000 7f0aa5193740 1 -- 172.21.15.47:0/3626585810 >> 172.21.15.47:0/3626585810 conn(0x56387a837c00 msgr2=0x56387a83b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:53.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.810+0000 7f0aa5193740 1 -- 172.21.15.47:0/3626585810 shutdown_connections 2024-03-21T11:32:53.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.811+0000 7f0aa5193740 1 -- 172.21.15.47:0/3626585810 wait complete. 2024-03-21T11:32:53.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.811+0000 7f0aa5193740 1 -- start start 2024-03-21T11:32:53.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.811+0000 7f0aa5193740 1 Processor -- start 2024-03-21T11:32:53.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.812+0000 7f0aa5193740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83c810 0x56387a849b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.812+0000 7f0aa5193740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 0x56387a84a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.812+0000 7f0aa5193740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83e3a0 0x56387a84d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.812+0000 7f0aa5193740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1ad440 con 0x56387a83d9e0 2024-03-21T11:32:53.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.812+0000 7f0aa5193740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1acde0 con 0x56387a83c810 2024-03-21T11:32:53.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.812+0000 7f0aa5193740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1aa2e0 con 0x56387a83e3a0 2024-03-21T11:32:53.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.812+0000 7f0aa5193740 10 client.0 _ll_register_callbacks cb 0x7f0a94009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:53.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.812+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83e3a0 0x56387a84d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.812+0000 7f0a9be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83e3a0 0x56387a84d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50572/0 (socket says 172.21.15.47:50572) 2024-03-21T11:32:53.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.812+0000 7f0a9be00640 1 -- 172.21.15.47:0/1660289818 learned_addr learned my addr 172.21.15.47:0/1660289818 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:53.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.812+0000 7f0a9b400640 1 --2- 172.21.15.47:0/1660289818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83c810 0x56387a849b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.813+0000 7f0a9aa00640 1 --2- 172.21.15.47:0/1660289818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 0x56387a84a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.813+0000 7f0a9be00640 1 -- 172.21.15.47:0/1660289818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83c810 msgr2=0x56387a849b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:53.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.813+0000 7f0a9be00640 1 --2- 172.21.15.47:0/1660289818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83c810 0x56387a849b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.813+0000 7f0a9be00640 1 -- 172.21.15.47:0/1660289818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 msgr2=0x56387a84a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:53.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.813+0000 7f0a9be00640 1 --2- 172.21.15.47:0/1660289818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 0x56387a84a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.813+0000 7f0a9be00640 1 -- 172.21.15.47:0/1660289818 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56387a84d660 con 0x56387a83e3a0 2024-03-21T11:32:53.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.813+0000 7f0a9aa00640 1 --2- 172.21.15.47:0/1660289818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 0x56387a84a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:53.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.813+0000 7f0a9be00640 1 --2- 172.21.15.47:0/1660289818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83e3a0 0x56387a84d070 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0a94025c80 tx=0x7f0a94016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1306c38ff8699f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:53.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.813+0000 7f0a85600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:53.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.813+0000 7f0a85600640 1 -- 172.21.15.47:0/1660289818 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94041070 con 0x56387a83e3a0 2024-03-21T11:32:53.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.814+0000 7f0a85600640 1 -- 172.21.15.47:0/1660289818 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0a94004c10 con 0x56387a83e3a0 2024-03-21T11:32:53.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.814+0000 7f0aa5193740 1 -- 172.21.15.47:0/1660289818 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a94021040 con 0x56387a83e3a0 2024-03-21T11:32:53.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.814+0000 7f0a85600640 1 -- 172.21.15.47:0/1660289818 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a9404d050 con 0x56387a83e3a0 2024-03-21T11:32:53.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.814+0000 7f0aa5193740 10 client.28687 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:53.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.814+0000 7f0aa5193740 1 -- 172.21.15.47:0/1660289818 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56387a8ae280 con 0x56387a83e3a0 2024-03-21T11:32:53.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.814+0000 7f0a85600640 1 -- 172.21.15.47:0/1660289818 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a94021040 con 0x56387a83e3a0 2024-03-21T11:32:53.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.814+0000 7f0a85600640 1 -- 172.21.15.47:0/1660289818 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0a9400b030 con 0x56387a83e3a0 2024-03-21T11:32:53.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.814+0000 7f0aa5193740 10 client.28687 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:53.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.814+0000 7f0aa5193740 10 client.28687 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:32:53.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.814+0000 7f0aa5193740 20 client.28687 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:53.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.815+0000 7f0aa5193740 1 --2- 172.21.15.47:0/1660289818 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x56387a8aeac0 0x56387a8b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:53.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.815+0000 7f0aa5193740 4 client.28687 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:32:53.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.815+0000 7f0aa5193740 1 -- 172.21.15.47:0/1660289818 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56387a7a8b50 con 0x56387a8aeac0 2024-03-21T11:32:53.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.815+0000 7f0a9b400640 1 --2- 172.21.15.47:0/1660289818 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x56387a8aeac0 0x56387a8b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:53.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.815+0000 7f0a9b400640 1 --2- 172.21.15.47:0/1660289818 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x56387a8aeac0 0x56387a8b0ee0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d675d00e4788fc2a server_cookie=7d1b09497f931451 in_seq=0 out_seq=0 2024-03-21T11:32:53.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.815+0000 7f0a85600640 10 client.28687 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:32:53.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.815+0000 7f0a85600640 1 -- 172.21.15.47:0/1660289818 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56387a7a8b50 con 0x56387a8aeac0 2024-03-21T11:32:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.815+0000 7f0a85600640 10 client.28687 handle_command_reply: tid=0 2024-03-21T11:32:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.816+0000 7f0aa5193740 1 client.28687 shutdown 2024-03-21T11:32:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.816+0000 7f0aa5193740 1 -- 172.21.15.47:0/1660289818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83e3a0 msgr2=0x56387a84d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.816+0000 7f0aa5193740 1 --2- 172.21.15.47:0/1660289818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83e3a0 0x56387a84d070 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0a94025c80 tx=0x7f0a94016000 comp rx=0 tx=0).stop 2024-03-21T11:32:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.816+0000 7f0aa5193740 1 -- 172.21.15.47:0/1660289818 shutdown_connections 2024-03-21T11:32:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.816+0000 7f0aa5193740 1 --2- 172.21.15.47:0/1660289818 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x56387a8aeac0 0x56387a8b0ee0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.816+0000 7f0aa5193740 1 --2- 172.21.15.47:0/1660289818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56387a83e3a0 0x56387a84d070 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.816+0000 7f0aa5193740 1 --2- 172.21.15.47:0/1660289818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56387a83d9e0 0x56387a84a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.816+0000 7f0aa5193740 1 --2- 172.21.15.47:0/1660289818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56387a83c810 0x56387a849b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.816+0000 7f0aa5193740 1 -- 172.21.15.47:0/1660289818 >> 172.21.15.47:0/1660289818 conn(0x56387a837c00 msgr2=0x56387a83b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.816+0000 7f0a85600640 0 client.28687 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:32:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.816+0000 7f0aa5193740 1 -- 172.21.15.47:0/1660289818 shutdown_connections 2024-03-21T11:32:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.816+0000 7f0aa5193740 1 -- 172.21.15.47:0/1660289818 wait complete. 2024-03-21T11:32:53.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.816+0000 7f0aa5193740 20 client.28687 trim_cache size 0 max 16384 2024-03-21T11:32:53.822 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:32:53.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.817+0000 7f0aa2400640 1 -- 172.21.15.47:0/1498246076 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a800a2f20 msgr2=0x7f0a800a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:53.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.817+0000 7f0aa2400640 1 --2- 172.21.15.47:0/1498246076 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a800a2f20 0x7f0a800a53e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x56387a83d8e0 tx=0x7f0a94010040 comp rx=0 tx=0).stop 2024-03-21T11:32:53.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.817+0000 7f0aa2400640 1 -- 172.21.15.47:0/1498246076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a9c1016b0 msgr2=0x7f0a9c199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:53.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.817+0000 7f0aa2400640 1 --2- 172.21.15.47:0/1498246076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a9c1016b0 0x7f0a9c199a10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c02c180 tx=0x7f0a8c0040c0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.817+0000 7f0aa2400640 1 -- 172.21.15.47:0/1498246076 shutdown_connections 2024-03-21T11:32:53.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.817+0000 7f0aa2400640 1 --2- 172.21.15.47:0/1498246076 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0a800a2f20 0x7f0a800a53e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.817+0000 7f0aa2400640 1 --2- 172.21.15.47:0/1498246076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a9c103260 0x7f0a9c1a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.817+0000 7f0aa2400640 1 --2- 172.21.15.47:0/1498246076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a9c1028a0 0x7f0a9c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.817+0000 7f0aa2400640 1 --2- 172.21.15.47:0/1498246076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a9c1016b0 0x7f0a9c199a10 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:53.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.817+0000 7f0aa2400640 1 -- 172.21.15.47:0/1498246076 >> 172.21.15.47:0/1498246076 conn(0x7f0a9c0fcc40 msgr2=0x7f0a9c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:53.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.817+0000 7f0aa2400640 1 -- 172.21.15.47:0/1498246076 shutdown_connections 2024-03-21T11:32:53.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:53.817+0000 7f0aa2400640 1 -- 172.21.15.47:0/1498246076 wait complete. 2024-03-21T11:32:53.838 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:32:53.838 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:32:54.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.046+0000 7ffb15000640 1 -- 172.21.15.47:0/3591349566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb101015a0 msgr2=0x7ffb101019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.046+0000 7ffb15000640 1 --2- 172.21.15.47:0/3591349566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb101015a0 0x7ffb101019c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009900 tx=0x7ffb0002bc50 comp rx=0 tx=0).stop 2024-03-21T11:32:54.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.046+0000 7ffb15000640 1 -- 172.21.15.47:0/3591349566 shutdown_connections 2024-03-21T11:32:54.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.046+0000 7ffb15000640 1 --2- 172.21.15.47:0/3591349566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb10103150 0x7ffb101099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.046+0000 7ffb15000640 1 --2- 172.21.15.47:0/3591349566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb10102790 0x7ffb10102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.046+0000 7ffb15000640 1 --2- 172.21.15.47:0/3591349566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb101015a0 0x7ffb101019c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.046+0000 7ffb15000640 1 -- 172.21.15.47:0/3591349566 >> 172.21.15.47:0/3591349566 conn(0x7ffb100fcb10 msgr2=0x7ffb100fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.046+0000 7ffb15000640 1 -- 172.21.15.47:0/3591349566 shutdown_connections 2024-03-21T11:32:54.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.046+0000 7ffb15000640 1 -- 172.21.15.47:0/3591349566 wait complete. 2024-03-21T11:32:54.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.047+0000 7ffb15000640 1 Processor -- start 2024-03-21T11:32:54.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.047+0000 7ffb15000640 1 -- start start 2024-03-21T11:32:54.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.047+0000 7ffb15000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb101015a0 0x7ffb10075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.047+0000 7ffb15000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb10102790 0x7ffb10073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.047+0000 7ffb15000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb10103150 0x7ffb10073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.047+0000 7ffb15000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1010d240 con 0x7ffb10102790 2024-03-21T11:32:54.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.047+0000 7ffb15000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1010d0c0 con 0x7ffb10103150 2024-03-21T11:32:54.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.047+0000 7ffb15000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1010d3c0 con 0x7ffb101015a0 2024-03-21T11:32:54.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.048+0000 7ffb0f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb101015a0 0x7ffb10075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.048+0000 7ffb0fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb10103150 0x7ffb10073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.048+0000 7ffb0f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb101015a0 0x7ffb10075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50596/0 (socket says 172.21.15.47:50596) 2024-03-21T11:32:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.048+0000 7ffb0fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb10103150 0x7ffb10073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60852/0 (socket says 172.21.15.47:60852) 2024-03-21T11:32:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.048+0000 7ffb0f400640 1 -- 172.21.15.47:0/1284931635 learned_addr learned my addr 172.21.15.47:0/1284931635 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.048+0000 7ffb0ea00640 1 --2- 172.21.15.47:0/1284931635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb10102790 0x7ffb10073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.048+0000 7ffb0fe00640 1 -- 172.21.15.47:0/1284931635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb101015a0 msgr2=0x7ffb10075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.048+0000 7ffb0fe00640 1 --2- 172.21.15.47:0/1284931635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb101015a0 0x7ffb10075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.048+0000 7ffb0fe00640 1 -- 172.21.15.47:0/1284931635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb10102790 msgr2=0x7ffb10073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.048+0000 7ffb0fe00640 1 --2- 172.21.15.47:0/1284931635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb10102790 0x7ffb10073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.048+0000 7ffb0fe00640 1 -- 172.21.15.47:0/1284931635 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb10074860 con 0x7ffb10103150 2024-03-21T11:32:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.048+0000 7ffb0f400640 1 --2- 172.21.15.47:0/1284931635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb101015a0 0x7ffb10075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:54.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.048+0000 7ffb0fe00640 1 --2- 172.21.15.47:0/1284931635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb10103150 0x7ffb10073fd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009fd0 tx=0x7ffb0800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.049+0000 7ffb07e00640 1 -- 172.21.15.47:0/1284931635 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0800cd60 con 0x7ffb10103150 2024-03-21T11:32:54.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.049+0000 7ffb07e00640 1 -- 172.21.15.47:0/1284931635 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb08005020 con 0x7ffb10103150 2024-03-21T11:32:54.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.049+0000 7ffb15000640 1 -- 172.21.15.47:0/1284931635 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb101a1970 con 0x7ffb10103150 2024-03-21T11:32:54.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.049+0000 7ffb07e00640 1 -- 172.21.15.47:0/1284931635 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08010690 con 0x7ffb10103150 2024-03-21T11:32:54.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.049+0000 7ffb15000640 1 -- 172.21.15.47:0/1284931635 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb101a1e70 con 0x7ffb10103150 2024-03-21T11:32:54.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.052+0000 7ffb193ad740 1 Processor -- start 2024-03-21T11:32:54.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.052+0000 7ffb07e00640 1 -- 172.21.15.47:0/1284931635 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ffb08005540 con 0x7ffb10103150 2024-03-21T11:32:54.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.053+0000 7ffb193ad740 1 -- start start 2024-03-21T11:32:54.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.053+0000 7ffb193ad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad926e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.053+0000 7ffb193ad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad928040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.053+0000 7ffb193ad740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad928580 0x562fad92ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.053+0000 7ffb193ad740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fad88b2c0 con 0x562fad928580 2024-03-21T11:32:54.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.053+0000 7ffb193ad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fad833c90 con 0x562fad9269f0 2024-03-21T11:32:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.053+0000 7ffb193ad740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fad887360 con 0x562fad927bc0 2024-03-21T11:32:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.053+0000 7ffb0f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad928040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.053+0000 7ffb0f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad928040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50598/0 (socket says 172.21.15.47:50598) 2024-03-21T11:32:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.053+0000 7ffb0f400640 1 -- 172.21.15.47:0/3717399512 learned_addr learned my addr 172.21.15.47:0/3717399512 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.054+0000 7ffb0ea00640 1 --2- 172.21.15.47:0/3717399512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad926e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.054+0000 7ffb0ea00640 1 --2- 172.21.15.47:0/3717399512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad928580 0x562fad92ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.054+0000 7ffb0f400640 1 -- 172.21.15.47:0/3717399512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 msgr2=0x562fad926e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.054+0000 7ffb0f400640 1 --2- 172.21.15.47:0/3717399512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad926e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.054+0000 7ffb0f400640 1 -- 172.21.15.47:0/3717399512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad928580 msgr2=0x562fad92ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.054+0000 7ffb0f400640 1 --2- 172.21.15.47:0/3717399512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad928580 0x562fad92ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.054+0000 7ffb0f400640 1 -- 172.21.15.47:0/3717399512 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fad858080 con 0x562fad927bc0 2024-03-21T11:32:54.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.054+0000 7ffb0ea00640 1 --2- 172.21.15.47:0/3717399512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad928580 0x562fad92ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:54.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.054+0000 7ffb0f400640 1 --2- 172.21.15.47:0/3717399512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad928040 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ffb00036040 tx=0x7ffb00005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dfaaf015205b185 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.055+0000 7ffb07e00640 1 --2- 172.21.15.47:0/1284931635 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffaf40a2f20 0x7ffaf40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.055+0000 7ffb07e00640 1 -- 172.21.15.47:0/1284931635 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffb080cfe90 con 0x7ffb10103150 2024-03-21T11:32:54.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.055+0000 7ffb0ea00640 1 --2- 172.21.15.47:0/1284931635 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffaf40a2f20 0x7ffaf40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.055+0000 7ffb04c00640 1 -- 172.21.15.47:0/3717399512 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00042070 con 0x562fad927bc0 2024-03-21T11:32:54.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.055+0000 7ffb04c00640 1 -- 172.21.15.47:0/3717399512 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb00004ba0 con 0x562fad927bc0 2024-03-21T11:32:54.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.055+0000 7ffb04c00640 1 -- 172.21.15.47:0/3717399512 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb000054e0 con 0x562fad927bc0 2024-03-21T11:32:54.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.056+0000 7ffb193ad740 1 -- 172.21.15.47:0/3717399512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 msgr2=0x562fad928040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.056+0000 7ffb193ad740 1 --2- 172.21.15.47:0/3717399512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad928040 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ffb00036040 tx=0x7ffb00005dd0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.056+0000 7ffb0ea00640 1 --2- 172.21.15.47:0/1284931635 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffaf40a2f20 0x7ffaf40a53e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x562fad927ac0 tx=0x7ffafc00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.056+0000 7ffb193ad740 1 -- 172.21.15.47:0/3717399512 shutdown_connections 2024-03-21T11:32:54.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.056+0000 7ffb193ad740 1 --2- 172.21.15.47:0/3717399512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad928580 0x562fad92ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.056+0000 7ffb193ad740 1 --2- 172.21.15.47:0/3717399512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad928040 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.056+0000 7ffb193ad740 1 --2- 172.21.15.47:0/3717399512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad926e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.056+0000 7ffb193ad740 1 -- 172.21.15.47:0/3717399512 >> 172.21.15.47:0/3717399512 conn(0x562fad921de0 msgr2=0x562fad9221e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.056+0000 7ffb193ad740 1 -- 172.21.15.47:0/3717399512 shutdown_connections 2024-03-21T11:32:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.056+0000 7ffb193ad740 1 -- 172.21.15.47:0/3717399512 wait complete. 2024-03-21T11:32:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.057+0000 7ffb193ad740 1 -- start start 2024-03-21T11:32:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.057+0000 7ffb193ad740 1 Processor -- start 2024-03-21T11:32:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb193ad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad930be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb193ad740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad927bc0 0x562fad931120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb193ad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad928580 0x562fad938160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb193ad740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fad92f530 con 0x562fad927bc0 2024-03-21T11:32:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb193ad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fad8470d0 con 0x562fad9269f0 2024-03-21T11:32:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb193ad740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fad92f6b0 con 0x562fad928580 2024-03-21T11:32:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb193ad740 10 client.0 _ll_register_callbacks cb 0x562fad84be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad928580 0x562fad938160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb0f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad930be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb0ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad927bc0 0x562fad931120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb0f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad930be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60876/0 (socket says 172.21.15.47:60876) 2024-03-21T11:32:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad928580 0x562fad938160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50600/0 (socket says 172.21.15.47:50600) 2024-03-21T11:32:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb0f400640 1 -- 172.21.15.47:0/2369636769 learned_addr learned my addr 172.21.15.47:0/2369636769 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb0f400640 1 -- 172.21.15.47:0/2369636769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad928580 msgr2=0x562fad938160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb0f400640 1 --2- 172.21.15.47:0/2369636769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad928580 0x562fad938160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb0f400640 1 -- 172.21.15.47:0/2369636769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad927bc0 msgr2=0x562fad931120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb0f400640 1 --2- 172.21.15.47:0/2369636769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad927bc0 0x562fad931120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb0f400640 1 -- 172.21.15.47:0/2369636769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fad59f3f0 con 0x562fad9269f0 2024-03-21T11:32:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.058+0000 7ffb0fe00640 1 --2- 172.21.15.47:0/2369636769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad928580 0x562fad938160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.059+0000 7ffb0f400640 1 --2- 172.21.15.47:0/2369636769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad930be0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ffb0002c900 tx=0x7ffb00005490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b209b908708199c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.059+0000 7ffaf9600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:32:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.059+0000 7ffaf9600640 1 -- 172.21.15.47:0/2369636769 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00042030 con 0x562fad9269f0 2024-03-21T11:32:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.059+0000 7ffb193ad740 1 -- 172.21.15.47:0/2369636769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb00004ba0 con 0x562fad9269f0 2024-03-21T11:32:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.059+0000 7ffaf9600640 1 -- 172.21.15.47:0/2369636769 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffb00004450 con 0x562fad9269f0 2024-03-21T11:32:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.059+0000 7ffaf9600640 1 -- 172.21.15.47:0/2369636769 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb00059050 con 0x562fad9269f0 2024-03-21T11:32:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.059+0000 7ffb193ad740 10 client.28688 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.059+0000 7ffb193ad740 1 -- 172.21.15.47:0/2369636769 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562fad6697d0 con 0x562fad9269f0 2024-03-21T11:32:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.060+0000 7ffaf9600640 1 -- 172.21.15.47:0/2369636769 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb00004ba0 con 0x562fad9269f0 2024-03-21T11:32:54.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.060+0000 7ffaf9600640 1 -- 172.21.15.47:0/2369636769 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffb00047460 con 0x562fad9269f0 2024-03-21T11:32:54.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.060+0000 7ffb193ad740 10 client.28688 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:54.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.060+0000 7ffb193ad740 10 client.28688 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:32:54.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.060+0000 7ffb193ad740 20 client.28688 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:54.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.060+0000 7ffb193ad740 1 --2- 172.21.15.47:0/2369636769 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562fad93c2c0 0x562fad93e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.060+0000 7ffb193ad740 4 client.28688 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:54.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.060+0000 7ffb193ad740 1 -- 172.21.15.47:0/2369636769 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fad898800 con 0x562fad93c2c0 2024-03-21T11:32:54.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.060+0000 7ffb0ea00640 1 --2- 172.21.15.47:0/2369636769 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562fad93c2c0 0x562fad93e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.061+0000 7ffb0ea00640 1 --2- 172.21.15.47:0/2369636769 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562fad93c2c0 0x562fad93e6b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f993a3b831788ef9 server_cookie=32c6f0537a7cea05 in_seq=0 out_seq=0 2024-03-21T11:32:54.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.061+0000 7ffaf9600640 10 client.28688 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:32:54.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.062+0000 7ffaf9600640 1 -- 172.21.15.47:0/2369636769 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562fad898800 con 0x562fad93c2c0 2024-03-21T11:32:54.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.062+0000 7ffaf9600640 10 client.28688 handle_command_reply: tid=0 2024-03-21T11:32:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.062+0000 7ffb193ad740 1 client.28688 shutdown 2024-03-21T11:32:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.062+0000 7ffb193ad740 1 -- 172.21.15.47:0/2369636769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 msgr2=0x562fad930be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.062+0000 7ffb193ad740 1 --2- 172.21.15.47:0/2369636769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad930be0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ffb0002c900 tx=0x7ffb00005490 comp rx=0 tx=0).stop 2024-03-21T11:32:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.062+0000 7ffb193ad740 1 -- 172.21.15.47:0/2369636769 shutdown_connections 2024-03-21T11:32:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.062+0000 7ffb193ad740 1 --2- 172.21.15.47:0/2369636769 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562fad93c2c0 0x562fad93e6b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.062+0000 7ffb193ad740 1 --2- 172.21.15.47:0/2369636769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad928580 0x562fad938160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.062+0000 7ffb193ad740 1 --2- 172.21.15.47:0/2369636769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad927bc0 0x562fad931120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.062+0000 7ffaf9600640 0 client.28688 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:32:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.062+0000 7ffb193ad740 1 --2- 172.21.15.47:0/2369636769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad930be0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.062+0000 7ffb193ad740 1 -- 172.21.15.47:0/2369636769 >> 172.21.15.47:0/2369636769 conn(0x562fad921de0 msgr2=0x562fad921160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.062+0000 7ffb193ad740 1 -- 172.21.15.47:0/2369636769 shutdown_connections 2024-03-21T11:32:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.063+0000 7ffb193ad740 1 -- 172.21.15.47:0/2369636769 wait complete. 2024-03-21T11:32:54.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.063+0000 7ffb193ad740 20 client.28688 trim_cache size 0 max 16384 2024-03-21T11:32:54.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.079+0000 7ffb193ad740 1 Processor -- start 2024-03-21T11:32:54.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.079+0000 7ffb193ad740 1 -- start start 2024-03-21T11:32:54.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.079+0000 7ffb193ad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad921de0 0x562fad925040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.079+0000 7ffb193ad740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad9269f0 0x562fad938080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.079+0000 7ffb193ad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad942dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.079+0000 7ffb193ad740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb101ac000 con 0x562fad9269f0 2024-03-21T11:32:54.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.079+0000 7ffb193ad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb101ad980 con 0x562fad921de0 2024-03-21T11:32:54.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.079+0000 7ffb193ad740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb101ac660 con 0x562fad927bc0 2024-03-21T11:32:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.079+0000 7ffb0f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad921de0 0x562fad925040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.079+0000 7ffb0f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad921de0 0x562fad925040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60890/0 (socket says 172.21.15.47:60890) 2024-03-21T11:32:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.079+0000 7ffb0f400640 1 -- 172.21.15.47:0/3799734595 learned_addr learned my addr 172.21.15.47:0/3799734595 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.079+0000 7ffb0fe00640 1 --2- 172.21.15.47:0/3799734595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad942dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.079+0000 7ffb0ea00640 1 --2- 172.21.15.47:0/3799734595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad9269f0 0x562fad938080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.080+0000 7ffb0f400640 1 -- 172.21.15.47:0/3799734595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 msgr2=0x562fad942dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.080+0000 7ffb0f400640 1 --2- 172.21.15.47:0/3799734595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad942dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.080+0000 7ffb0f400640 1 -- 172.21.15.47:0/3799734595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad9269f0 msgr2=0x562fad938080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.080+0000 7ffb0f400640 1 --2- 172.21.15.47:0/3799734595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad9269f0 0x562fad938080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.080+0000 7ffb0f400640 1 -- 172.21.15.47:0/3799734595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fad922b50 con 0x562fad921de0 2024-03-21T11:32:54.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.080+0000 7ffb0f400640 1 --2- 172.21.15.47:0/3799734595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad921de0 0x562fad925040 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ffb0002f020 tx=0x7ffb00005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7132a68423db6a83 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.080+0000 7ffafbe00640 1 -- 172.21.15.47:0/3799734595 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00042040 con 0x562fad921de0 2024-03-21T11:32:54.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.080+0000 7ffafbe00640 1 -- 172.21.15.47:0/3799734595 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb00004ef0 con 0x562fad921de0 2024-03-21T11:32:54.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.080+0000 7ffafbe00640 1 -- 172.21.15.47:0/3799734595 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00004250 con 0x562fad921de0 2024-03-21T11:32:54.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.081+0000 7ffb193ad740 1 -- 172.21.15.47:0/3799734595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad921de0 msgr2=0x562fad925040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.081+0000 7ffb193ad740 1 --2- 172.21.15.47:0/3799734595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad921de0 0x562fad925040 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ffb0002f020 tx=0x7ffb00005a20 comp rx=0 tx=0).stop 2024-03-21T11:32:54.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.081+0000 7ffb193ad740 1 -- 172.21.15.47:0/3799734595 shutdown_connections 2024-03-21T11:32:54.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.081+0000 7ffb193ad740 1 --2- 172.21.15.47:0/3799734595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad942dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.081+0000 7ffb193ad740 1 --2- 172.21.15.47:0/3799734595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad9269f0 0x562fad938080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.081+0000 7ffb193ad740 1 --2- 172.21.15.47:0/3799734595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad921de0 0x562fad925040 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.081+0000 7ffb193ad740 1 -- 172.21.15.47:0/3799734595 >> 172.21.15.47:0/3799734595 conn(0x562fad92fdf0 msgr2=0x562fad921160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.081+0000 7ffb193ad740 1 -- 172.21.15.47:0/3799734595 shutdown_connections 2024-03-21T11:32:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.082+0000 7ffb193ad740 1 -- 172.21.15.47:0/3799734595 wait complete. 2024-03-21T11:32:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.082+0000 7ffb193ad740 1 -- start start 2024-03-21T11:32:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.082+0000 7ffb193ad740 1 Processor -- start 2024-03-21T11:32:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb193ad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad9341c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb193ad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad934700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb193ad740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad928580 0x562fad9376b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb193ad740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb101ad320 con 0x562fad928580 2024-03-21T11:32:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb193ad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb101accc0 con 0x562fad9269f0 2024-03-21T11:32:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb193ad740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb101aa1c0 con 0x562fad927bc0 2024-03-21T11:32:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb193ad740 10 client.0 _ll_register_callbacks cb 0x7ffafc00a9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb0f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad9341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb0ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad934700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb0f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad9341c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60898/0 (socket says 172.21.15.47:60898) 2024-03-21T11:32:54.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb0f400640 1 -- 172.21.15.47:0/1382492262 learned_addr learned my addr 172.21.15.47:0/1382492262 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb0ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad934700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50626/0 (socket says 172.21.15.47:50626) 2024-03-21T11:32:54.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb0fe00640 1 --2- 172.21.15.47:0/1382492262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad928580 0x562fad9376b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb0ea00640 1 -- 172.21.15.47:0/1382492262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 msgr2=0x562fad9341c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb0ea00640 1 --2- 172.21.15.47:0/1382492262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad9341c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb0ea00640 1 -- 172.21.15.47:0/1382492262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad928580 msgr2=0x562fad9376b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.083+0000 7ffb0ea00640 1 --2- 172.21.15.47:0/1382492262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad928580 0x562fad9376b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.084+0000 7ffb0ea00640 1 -- 172.21.15.47:0/1382492262 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fad937ca0 con 0x562fad927bc0 2024-03-21T11:32:54.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.084+0000 7ffb0f400640 1 --2- 172.21.15.47:0/1382492262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad9341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:54.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.084+0000 7ffb0ea00640 1 --2- 172.21.15.47:0/1382492262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad934700 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x562fad93ed10 tx=0x7ffafc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1797192da3902b26 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.084+0000 7ffaf9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:54.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.084+0000 7ffaf9600640 1 -- 172.21.15.47:0/1382492262 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc02cbc0 con 0x562fad927bc0 2024-03-21T11:32:54.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.084+0000 7ffb193ad740 1 -- 172.21.15.47:0/1382492262 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb00004ef0 con 0x562fad927bc0 2024-03-21T11:32:54.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.084+0000 7ffaf9600640 1 -- 172.21.15.47:0/1382492262 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffafc02cd60 con 0x562fad927bc0 2024-03-21T11:32:54.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.084+0000 7ffaf9600640 1 -- 172.21.15.47:0/1382492262 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffafc035440 con 0x562fad927bc0 2024-03-21T11:32:54.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.084+0000 7ffb193ad740 10 client.28705 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:54.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.084+0000 7ffb193ad740 1 -- 172.21.15.47:0/1382492262 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562fad9982c0 con 0x562fad927bc0 2024-03-21T11:32:54.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.085+0000 7ffaf9600640 1 -- 172.21.15.47:0/1382492262 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb00004ef0 con 0x562fad927bc0 2024-03-21T11:32:54.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.085+0000 7ffaf9600640 1 -- 172.21.15.47:0/1382492262 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffafc033070 con 0x562fad927bc0 2024-03-21T11:32:54.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.085+0000 7ffb193ad740 10 client.28705 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:54.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.085+0000 7ffb193ad740 10 client.28705 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:32:54.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.085+0000 7ffb193ad740 20 client.28705 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:54.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.085+0000 7ffb193ad740 1 --2- 172.21.15.47:0/1382492262 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562fad998c70 0x562fad99b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.085+0000 7ffb193ad740 4 client.28705 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:32:54.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.085+0000 7ffb193ad740 1 -- 172.21.15.47:0/1382492262 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562fad892b50 con 0x562fad998c70 2024-03-21T11:32:54.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.085+0000 7ffb0f400640 1 --2- 172.21.15.47:0/1382492262 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562fad998c70 0x562fad99b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.086+0000 7ffb0f400640 1 --2- 172.21.15.47:0/1382492262 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562fad998c70 0x562fad99b090 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a450949cbb597ee server_cookie=56e00eda445f7a5f in_seq=0 out_seq=0 2024-03-21T11:32:54.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.086+0000 7ffaf9600640 10 client.28705 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:32:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.086+0000 7ffaf9600640 1 -- 172.21.15.47:0/1382492262 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562fad892b50 con 0x562fad998c70 2024-03-21T11:32:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.086+0000 7ffaf9600640 10 client.28705 handle_command_reply: tid=0 2024-03-21T11:32:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.086+0000 7ffb193ad740 1 client.28705 shutdown 2024-03-21T11:32:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.086+0000 7ffb193ad740 1 -- 172.21.15.47:0/1382492262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 msgr2=0x562fad934700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.086+0000 7ffb193ad740 1 --2- 172.21.15.47:0/1382492262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad934700 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x562fad93ed10 tx=0x7ffafc004730 comp rx=0 tx=0).stop 2024-03-21T11:32:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.087+0000 7ffb193ad740 1 -- 172.21.15.47:0/1382492262 shutdown_connections 2024-03-21T11:32:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.087+0000 7ffb193ad740 1 --2- 172.21.15.47:0/1382492262 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562fad998c70 0x562fad99b090 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.087+0000 7ffb193ad740 1 --2- 172.21.15.47:0/1382492262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562fad928580 0x562fad9376b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.087+0000 7ffb193ad740 1 --2- 172.21.15.47:0/1382492262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fad927bc0 0x562fad934700 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.087+0000 7ffb193ad740 1 --2- 172.21.15.47:0/1382492262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562fad9269f0 0x562fad9341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.087+0000 7ffaf9600640 0 client.28705 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:32:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.087+0000 7ffb193ad740 1 -- 172.21.15.47:0/1382492262 >> 172.21.15.47:0/1382492262 conn(0x562fad92fdf0 msgr2=0x562fad92adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.087+0000 7ffb193ad740 1 -- 172.21.15.47:0/1382492262 shutdown_connections 2024-03-21T11:32:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.087+0000 7ffb193ad740 1 -- 172.21.15.47:0/1382492262 wait complete. 2024-03-21T11:32:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.087+0000 7ffb193ad740 20 client.28705 trim_cache size 0 max 16384 2024-03-21T11:32:54.096 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:32:54.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.087+0000 7ffb15000640 1 -- 172.21.15.47:0/1284931635 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffaf40a2f20 msgr2=0x7ffaf40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.087+0000 7ffb15000640 1 --2- 172.21.15.47:0/1284931635 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffaf40a2f20 0x7ffaf40a53e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x562fad927ac0 tx=0x7ffafc00a2d0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.088+0000 7ffb15000640 1 -- 172.21.15.47:0/1284931635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb10103150 msgr2=0x7ffb10073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.088+0000 7ffb15000640 1 --2- 172.21.15.47:0/1284931635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb10103150 0x7ffb10073fd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009fd0 tx=0x7ffb0800ee50 comp rx=0 tx=0).stop 2024-03-21T11:32:54.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.088+0000 7ffb15000640 1 -- 172.21.15.47:0/1284931635 shutdown_connections 2024-03-21T11:32:54.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.088+0000 7ffb15000640 1 --2- 172.21.15.47:0/1284931635 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffaf40a2f20 0x7ffaf40a53e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.088+0000 7ffb15000640 1 --2- 172.21.15.47:0/1284931635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb10103150 0x7ffb10073fd0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.088+0000 7ffb15000640 1 --2- 172.21.15.47:0/1284931635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb10102790 0x7ffb10073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.088+0000 7ffb15000640 1 --2- 172.21.15.47:0/1284931635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb101015a0 0x7ffb10075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.088+0000 7ffb15000640 1 -- 172.21.15.47:0/1284931635 >> 172.21.15.47:0/1284931635 conn(0x7ffb100fcb10 msgr2=0x7ffb10105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.088+0000 7ffb15000640 1 -- 172.21.15.47:0/1284931635 shutdown_connections 2024-03-21T11:32:54.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.088+0000 7ffb15000640 1 -- 172.21.15.47:0/1284931635 wait complete. 2024-03-21T11:32:54.109 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:32:54.109 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:32:54.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.403+0000 7f732f400640 1 -- 172.21.15.47:0/1930014757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7328109000 msgr2=0x7f732810f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.403+0000 7f732f400640 1 --2- 172.21.15.47:0/1930014757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7328109000 0x7f732810f9e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7314009900 tx=0x7f731402bc50 comp rx=0 tx=0).stop 2024-03-21T11:32:54.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.403+0000 7f732f400640 1 -- 172.21.15.47:0/1930014757 shutdown_connections 2024-03-21T11:32:54.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.403+0000 7f732f400640 1 --2- 172.21.15.47:0/1930014757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f732810ff20 0x7f7328112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.403+0000 7f732f400640 1 --2- 172.21.15.47:0/1930014757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7328109000 0x7f732810f9e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.403+0000 7f732f400640 1 --2- 172.21.15.47:0/1930014757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73281086a0 0x7f7328108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.403+0000 7f732f400640 1 -- 172.21.15.47:0/1930014757 >> 172.21.15.47:0/1930014757 conn(0x7f732806e500 msgr2=0x7f7328070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.404+0000 7f732f400640 1 -- 172.21.15.47:0/1930014757 shutdown_connections 2024-03-21T11:32:54.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.404+0000 7f732f400640 1 -- 172.21.15.47:0/1930014757 wait complete. 2024-03-21T11:32:54.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.406+0000 7f732f400640 1 Processor -- start 2024-03-21T11:32:54.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.406+0000 7f732f400640 1 -- start start 2024-03-21T11:32:54.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.406+0000 7f732f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281086a0 0x7f732810b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.406+0000 7f732f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7328109000 0x7f7328109970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.406+0000 7f732f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732810ff20 0x7f7328109eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.406+0000 7f732f400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7328115b50 con 0x7f732810ff20 2024-03-21T11:32:54.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.406+0000 7f732f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73281159d0 con 0x7f73281086a0 2024-03-21T11:32:54.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.406+0000 7f732f400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7328115cd0 con 0x7f7328109000 2024-03-21T11:32:54.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.407+0000 7f732ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281086a0 0x7f732810b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.407+0000 7f732ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281086a0 0x7f732810b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60908/0 (socket says 172.21.15.47:60908) 2024-03-21T11:32:54.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.407+0000 7f7327e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7328109000 0x7f7328109970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.407+0000 7f732ca00640 1 -- 172.21.15.47:0/3754017335 learned_addr learned my addr 172.21.15.47:0/3754017335 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.407+0000 7f732d400640 1 --2- 172.21.15.47:0/3754017335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732810ff20 0x7f7328109eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.407+0000 7f7327e00640 1 -- 172.21.15.47:0/3754017335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281086a0 msgr2=0x7f732810b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.407+0000 7f7327e00640 1 --2- 172.21.15.47:0/3754017335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281086a0 0x7f732810b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.407+0000 7f7327e00640 1 -- 172.21.15.47:0/3754017335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732810ff20 msgr2=0x7f7328109eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.407+0000 7f7327e00640 1 --2- 172.21.15.47:0/3754017335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732810ff20 0x7f7328109eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.407+0000 7f7327e00640 1 -- 172.21.15.47:0/3754017335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732810a3f0 con 0x7f7328109000 2024-03-21T11:32:54.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.407+0000 7f732d400640 1 --2- 172.21.15.47:0/3754017335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732810ff20 0x7f7328109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:32:54.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.408+0000 7f7327e00640 1 --2- 172.21.15.47:0/3754017335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7328109000 0x7f7328109970 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7314036040 tx=0x7f7314004eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.408+0000 7f7325600640 1 -- 172.21.15.47:0/3754017335 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314043020 con 0x7f7328109000 2024-03-21T11:32:54.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.408+0000 7f732f400640 1 -- 172.21.15.47:0/3754017335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732810a680 con 0x7f7328109000 2024-03-21T11:32:54.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.408+0000 7f732f400640 1 -- 172.21.15.47:0/3754017335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73281aec20 con 0x7f7328109000 2024-03-21T11:32:54.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.408+0000 7f7325600640 1 -- 172.21.15.47:0/3754017335 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f73140043d0 con 0x7f7328109000 2024-03-21T11:32:54.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.409+0000 7f7325600640 1 -- 172.21.15.47:0/3754017335 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314003de0 con 0x7f7328109000 2024-03-21T11:32:54.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.411+0000 7f7332016740 1 Processor -- start 2024-03-21T11:32:54.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.412+0000 7f7332016740 1 -- start start 2024-03-21T11:32:54.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.412+0000 7f7325600640 1 -- 172.21.15.47:0/3754017335 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7314046020 con 0x7f7328109000 2024-03-21T11:32:54.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.412+0000 7f7332016740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e67810 0x558a11e67c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.412+0000 7f7332016740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e689e0 0x558a11e68e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.412+0000 7f7332016740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e693a0 0x558a11e6fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.412+0000 7f7332016740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a11dcc2c0 con 0x558a11e693a0 2024-03-21T11:32:54.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.412+0000 7f7332016740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a11d74c90 con 0x558a11e689e0 2024-03-21T11:32:54.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.412+0000 7f7332016740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a11dc8360 con 0x558a11e67810 2024-03-21T11:32:54.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.412+0000 7f732d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e67810 0x558a11e67c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.412+0000 7f732d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e67810 0x558a11e67c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50648/0 (socket says 172.21.15.47:50648) 2024-03-21T11:32:54.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.412+0000 7f732d400640 1 -- 172.21.15.47:0/391509852 learned_addr learned my addr 172.21.15.47:0/391509852 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.412+0000 7f732ca00640 1 --2- 172.21.15.47:0/391509852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e689e0 0x558a11e68e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.412+0000 7f7327e00640 1 --2- 172.21.15.47:0/391509852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e693a0 0x558a11e6fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.413+0000 7f732d400640 1 -- 172.21.15.47:0/391509852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e689e0 msgr2=0x558a11e68e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.413+0000 7f732d400640 1 --2- 172.21.15.47:0/391509852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e689e0 0x558a11e68e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.413+0000 7f732d400640 1 -- 172.21.15.47:0/391509852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e693a0 msgr2=0x558a11e6fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.413+0000 7f732d400640 1 --2- 172.21.15.47:0/391509852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e693a0 0x558a11e6fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.413+0000 7f732d400640 1 -- 172.21.15.47:0/391509852 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a11d99080 con 0x558a11e67810 2024-03-21T11:32:54.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.413+0000 7f732ca00640 1 --2- 172.21.15.47:0/391509852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e689e0 0x558a11e68e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:54.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.413+0000 7f7327e00640 1 --2- 172.21.15.47:0/391509852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e693a0 0x558a11e6fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:54.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.413+0000 7f732d400640 1 --2- 172.21.15.47:0/391509852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e67810 0x558a11e67c30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7320009f30 tx=0x7f732000f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e570087ea6c7225 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.413+0000 7f7325600640 1 --2- 172.21.15.47:0/3754017335 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73100a2fa0 0x7f73100a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.413+0000 7f7319c00640 1 -- 172.21.15.47:0/391509852 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320003b70 con 0x558a11e67810 2024-03-21T11:32:54.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.413+0000 7f7325600640 1 -- 172.21.15.47:0/3754017335 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f73140f19a0 con 0x7f7328109000 2024-03-21T11:32:54.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.413+0000 7f7319c00640 1 -- 172.21.15.47:0/391509852 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7320003d10 con 0x558a11e67810 2024-03-21T11:32:54.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.413+0000 7f7319c00640 1 -- 172.21.15.47:0/391509852 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73200119a0 con 0x558a11e67810 2024-03-21T11:32:54.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.414+0000 7f732ca00640 1 --2- 172.21.15.47:0/3754017335 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73100a2fa0 0x7f73100a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.414+0000 7f7332016740 1 -- 172.21.15.47:0/391509852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e67810 msgr2=0x558a11e67c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.414+0000 7f7332016740 1 --2- 172.21.15.47:0/391509852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e67810 0x558a11e67c30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7320009f30 tx=0x7f732000f7f0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.414+0000 7f7332016740 1 -- 172.21.15.47:0/391509852 shutdown_connections 2024-03-21T11:32:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.414+0000 7f7332016740 1 --2- 172.21.15.47:0/391509852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e693a0 0x558a11e6fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.414+0000 7f7332016740 1 --2- 172.21.15.47:0/391509852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e689e0 0x558a11e68e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.414+0000 7f7332016740 1 --2- 172.21.15.47:0/391509852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e67810 0x558a11e67c30 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.414+0000 7f7332016740 1 -- 172.21.15.47:0/391509852 >> 172.21.15.47:0/391509852 conn(0x558a11e62c00 msgr2=0x558a11e63000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.414+0000 7f7332016740 1 -- 172.21.15.47:0/391509852 shutdown_connections 2024-03-21T11:32:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.414+0000 7f732ca00640 1 --2- 172.21.15.47:0/3754017335 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73100a2fa0 0x7f73100a5460 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f731c00afb0 tx=0x7f731c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.414+0000 7f7332016740 1 -- 172.21.15.47:0/391509852 wait complete. 2024-03-21T11:32:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.415+0000 7f7332016740 1 -- start start 2024-03-21T11:32:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.415+0000 7f7332016740 1 Processor -- start 2024-03-21T11:32:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.415+0000 7f7332016740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e67810 0x558a11e71ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7332016740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e689e0 0x558a11e72000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7332016740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e693a0 0x558a11e79040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7332016740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a11e703b0 con 0x558a11e693a0 2024-03-21T11:32:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7332016740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a11d880d0 con 0x558a11e67810 2024-03-21T11:32:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7332016740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a11e70530 con 0x558a11e689e0 2024-03-21T11:32:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7332016740 10 client.0 _ll_register_callbacks cb 0x558a11d8ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7327e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e689e0 0x558a11e72000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7327e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e689e0 0x558a11e72000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50658/0 (socket says 172.21.15.47:50658) 2024-03-21T11:32:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7327e00640 1 -- 172.21.15.47:0/3167070441 learned_addr learned my addr 172.21.15.47:0/3167070441 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f732ca00640 1 --2- 172.21.15.47:0/3167070441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e67810 0x558a11e71ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f732d400640 1 --2- 172.21.15.47:0/3167070441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e693a0 0x558a11e79040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7327e00640 1 -- 172.21.15.47:0/3167070441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e67810 msgr2=0x558a11e71ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7327e00640 1 --2- 172.21.15.47:0/3167070441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e67810 0x558a11e71ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7327e00640 1 -- 172.21.15.47:0/3167070441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e693a0 msgr2=0x558a11e79040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7327e00640 1 --2- 172.21.15.47:0/3167070441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e693a0 0x558a11e79040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7327e00640 1 -- 172.21.15.47:0/3167070441 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a11ae03f0 con 0x558a11e689e0 2024-03-21T11:32:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f732d400640 1 --2- 172.21.15.47:0/3167070441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e693a0 0x558a11e79040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f7327e00640 1 --2- 172.21.15.47:0/3167070441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e689e0 0x558a11e72000 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f73140ba710 tx=0x7f73140bdbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3366cab67572b447 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.416+0000 7f730e000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.417+0000 7f730e000640 1 -- 172.21.15.47:0/3167070441 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73140bdd40 con 0x558a11e689e0 2024-03-21T11:32:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.417+0000 7f730e000640 1 -- 172.21.15.47:0/3167070441 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f73140035f0 con 0x558a11e689e0 2024-03-21T11:32:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.417+0000 7f7332016740 1 -- 172.21.15.47:0/3167070441 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7320003d10 con 0x558a11e689e0 2024-03-21T11:32:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.417+0000 7f730e000640 1 -- 172.21.15.47:0/3167070441 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73140c1e90 con 0x558a11e689e0 2024-03-21T11:32:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.417+0000 7f7332016740 10 client.28717 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.417+0000 7f7332016740 1 -- 172.21.15.47:0/3167070441 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558a11baa7d0 con 0x558a11e689e0 2024-03-21T11:32:54.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.417+0000 7f730e000640 1 -- 172.21.15.47:0/3167070441 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7320003d10 con 0x558a11e689e0 2024-03-21T11:32:54.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.417+0000 7f730e000640 1 -- 172.21.15.47:0/3167070441 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f73140bd220 con 0x558a11e689e0 2024-03-21T11:32:54.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.417+0000 7f7332016740 10 client.28717 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:54.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.417+0000 7f7332016740 10 client.28717 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:32:54.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.417+0000 7f7332016740 20 client.28717 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:54.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.417+0000 7f7332016740 1 --2- 172.21.15.47:0/3167070441 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x558a11e7d580 0x558a11e7f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.417+0000 7f7332016740 4 client.28717 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:54.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.417+0000 7f7332016740 1 -- 172.21.15.47:0/3167070441 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a11dd9800 con 0x558a11e7d580 2024-03-21T11:32:54.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.418+0000 7f732ca00640 1 --2- 172.21.15.47:0/3167070441 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x558a11e7d580 0x558a11e7f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.418+0000 7f730e000640 10 client.28717 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:32:54.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.418+0000 7f732ca00640 1 --2- 172.21.15.47:0/3167070441 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x558a11e7d580 0x558a11e7f970 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e45ad39b7c8d79d2 server_cookie=cdd6326d9bbbffc7 in_seq=0 out_seq=0 2024-03-21T11:32:54.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f730e000640 1 -- 172.21.15.47:0/3167070441 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558a11dd9800 con 0x558a11e7d580 2024-03-21T11:32:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f730e000640 10 client.28717 handle_command_reply: tid=0 2024-03-21T11:32:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f7332016740 1 client.28717 shutdown 2024-03-21T11:32:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f7332016740 1 -- 172.21.15.47:0/3167070441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e689e0 msgr2=0x558a11e72000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f7332016740 1 --2- 172.21.15.47:0/3167070441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e689e0 0x558a11e72000 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f73140ba710 tx=0x7f73140bdbd0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f7332016740 1 -- 172.21.15.47:0/3167070441 shutdown_connections 2024-03-21T11:32:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f7332016740 1 --2- 172.21.15.47:0/3167070441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e693a0 0x558a11e79040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f7332016740 1 --2- 172.21.15.47:0/3167070441 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x558a11e7d580 0x558a11e7f970 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f7332016740 1 --2- 172.21.15.47:0/3167070441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e689e0 0x558a11e72000 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f7332016740 1 --2- 172.21.15.47:0/3167070441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e67810 0x558a11e71ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f730e000640 0 client.28717 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:32:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f7332016740 1 -- 172.21.15.47:0/3167070441 >> 172.21.15.47:0/3167070441 conn(0x558a11e62c00 msgr2=0x558a11e61eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f7332016740 1 -- 172.21.15.47:0/3167070441 shutdown_connections 2024-03-21T11:32:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f7332016740 1 -- 172.21.15.47:0/3167070441 wait complete. 2024-03-21T11:32:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.420+0000 7f7332016740 20 client.28717 trim_cache size 0 max 16384 2024-03-21T11:32:54.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.435+0000 7f7332016740 1 Processor -- start 2024-03-21T11:32:54.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.435+0000 7f7332016740 1 -- start start 2024-03-21T11:32:54.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.435+0000 7f7332016740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e67810 0x558a11e6c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.435+0000 7f7332016740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e689e0 0x558a11e79000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.435+0000 7f7332016740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e693a0 0x558a11e6c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.435+0000 7f7332016740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73281b8d50 con 0x558a11e67810 2024-03-21T11:32:54.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.435+0000 7f7332016740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73281ba6d0 con 0x558a11e693a0 2024-03-21T11:32:54.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.435+0000 7f7332016740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73281b93b0 con 0x558a11e689e0 2024-03-21T11:32:54.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.436+0000 7f7327e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e689e0 0x558a11e79000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.436+0000 7f732d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e693a0 0x558a11e6c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.436+0000 7f732d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e693a0 0x558a11e6c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60934/0 (socket says 172.21.15.47:60934) 2024-03-21T11:32:54.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.436+0000 7f732d400640 1 -- 172.21.15.47:0/3692286763 learned_addr learned my addr 172.21.15.47:0/3692286763 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.436+0000 7f732ca00640 1 --2- 172.21.15.47:0/3692286763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e67810 0x558a11e6c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.436+0000 7f732d400640 1 -- 172.21.15.47:0/3692286763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e689e0 msgr2=0x558a11e79000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.436+0000 7f732d400640 1 --2- 172.21.15.47:0/3692286763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e689e0 0x558a11e79000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.436+0000 7f732d400640 1 -- 172.21.15.47:0/3692286763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e67810 msgr2=0x558a11e6c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.436+0000 7f732d400640 1 --2- 172.21.15.47:0/3692286763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e67810 0x558a11e6c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.436+0000 7f732d400640 1 -- 172.21.15.47:0/3692286763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a11e7fff0 con 0x558a11e693a0 2024-03-21T11:32:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.436+0000 7f7327e00640 1 --2- 172.21.15.47:0/3692286763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e689e0 0x558a11e79000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.436+0000 7f732d400640 1 --2- 172.21.15.47:0/3692286763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e693a0 0x558a11e6c6d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7320009ed0 tx=0x7f7320009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53c8986bddfcafc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.437+0000 7f7319200640 1 -- 172.21.15.47:0/3692286763 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320003970 con 0x558a11e693a0 2024-03-21T11:32:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.437+0000 7f7319200640 1 -- 172.21.15.47:0/3692286763 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7320011690 con 0x558a11e693a0 2024-03-21T11:32:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.437+0000 7f7319200640 1 -- 172.21.15.47:0/3692286763 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73200052a0 con 0x558a11e693a0 2024-03-21T11:32:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.438+0000 7f7332016740 1 -- 172.21.15.47:0/3692286763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e693a0 msgr2=0x558a11e6c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.438+0000 7f7332016740 1 --2- 172.21.15.47:0/3692286763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e693a0 0x558a11e6c6d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7320009ed0 tx=0x7f7320009740 comp rx=0 tx=0).stop 2024-03-21T11:32:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.438+0000 7f7332016740 1 -- 172.21.15.47:0/3692286763 shutdown_connections 2024-03-21T11:32:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.438+0000 7f7332016740 1 --2- 172.21.15.47:0/3692286763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e693a0 0x558a11e6c6d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.438+0000 7f7332016740 1 --2- 172.21.15.47:0/3692286763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e689e0 0x558a11e79000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.438+0000 7f7332016740 1 --2- 172.21.15.47:0/3692286763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e67810 0x558a11e6c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.438+0000 7f7332016740 1 -- 172.21.15.47:0/3692286763 >> 172.21.15.47:0/3692286763 conn(0x558a11e62c00 msgr2=0x558a11e6ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.438+0000 7f7332016740 1 -- 172.21.15.47:0/3692286763 shutdown_connections 2024-03-21T11:32:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.438+0000 7f7332016740 1 -- 172.21.15.47:0/3692286763 wait complete. 2024-03-21T11:32:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.438+0000 7f7332016740 1 -- start start 2024-03-21T11:32:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.438+0000 7f7332016740 1 Processor -- start 2024-03-21T11:32:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f7332016740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e67810 0x558a11e75120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f7332016740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e689e0 0x558a11e75660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f7332016740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e693a0 0x558a11e78680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f7332016740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73281ba070 con 0x558a11e689e0 2024-03-21T11:32:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f7332016740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73281b9a10 con 0x558a11e693a0 2024-03-21T11:32:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f7332016740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73281b6f10 con 0x558a11e67810 2024-03-21T11:32:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f7332016740 10 client.0 _ll_register_callbacks cb 0x7f7314009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f732ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e67810 0x558a11e75120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f732ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e67810 0x558a11e75120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50676/0 (socket says 172.21.15.47:50676) 2024-03-21T11:32:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f732ca00640 1 -- 172.21.15.47:0/2883652572 learned_addr learned my addr 172.21.15.47:0/2883652572 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f732d400640 1 --2- 172.21.15.47:0/2883652572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e693a0 0x558a11e78680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f7327e00640 1 --2- 172.21.15.47:0/2883652572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e689e0 0x558a11e75660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f732ca00640 1 -- 172.21.15.47:0/2883652572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e693a0 msgr2=0x558a11e78680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f732ca00640 1 --2- 172.21.15.47:0/2883652572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e693a0 0x558a11e78680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f732ca00640 1 -- 172.21.15.47:0/2883652572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e689e0 msgr2=0x558a11e75660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f732ca00640 1 --2- 172.21.15.47:0/2883652572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e689e0 0x558a11e75660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.439+0000 7f732ca00640 1 -- 172.21.15.47:0/2883652572 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a11e78c70 con 0x558a11e67810 2024-03-21T11:32:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.440+0000 7f732ca00640 1 --2- 172.21.15.47:0/2883652572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e67810 0x558a11e75120 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f731c00ad90 tx=0x7f731c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391d112a6232c29 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.440+0000 7f730e000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.440+0000 7f730e000640 1 -- 172.21.15.47:0/2883652572 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c029260 con 0x558a11e67810 2024-03-21T11:32:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.440+0000 7f730e000640 1 -- 172.21.15.47:0/2883652572 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f731c01b670 con 0x558a11e67810 2024-03-21T11:32:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.440+0000 7f7332016740 1 -- 172.21.15.47:0/2883652572 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7320011690 con 0x558a11e67810 2024-03-21T11:32:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.440+0000 7f730e000640 1 -- 172.21.15.47:0/2883652572 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f731c03a050 con 0x558a11e67810 2024-03-21T11:32:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.440+0000 7f7332016740 10 client.28723 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.440+0000 7f7332016740 1 -- 172.21.15.47:0/2883652572 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558a11ed9950 con 0x558a11e67810 2024-03-21T11:32:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.441+0000 7f730e000640 1 -- 172.21.15.47:0/2883652572 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7320011690 con 0x558a11e67810 2024-03-21T11:32:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.441+0000 7f730e000640 1 -- 172.21.15.47:0/2883652572 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f731c032940 con 0x558a11e67810 2024-03-21T11:32:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.441+0000 7f7332016740 10 client.28723 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.441+0000 7f7332016740 10 client.28723 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:32:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.441+0000 7f7332016740 20 client.28723 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.441+0000 7f7332016740 1 --2- 172.21.15.47:0/2883652572 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x558a11eda330 0x558a11edc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.441+0000 7f7332016740 4 client.28723 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:32:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.441+0000 7f7332016740 1 -- 172.21.15.47:0/2883652572 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558a11dd3b50 con 0x558a11eda330 2024-03-21T11:32:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.441+0000 7f7327e00640 1 --2- 172.21.15.47:0/2883652572 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x558a11eda330 0x558a11edc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f7327e00640 1 --2- 172.21.15.47:0/2883652572 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x558a11eda330 0x558a11edc750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec60bd78d4ef7494 server_cookie=2469b58a1e74c08a in_seq=0 out_seq=0 2024-03-21T11:32:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f730e000640 10 client.28723 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:32:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f730e000640 1 -- 172.21.15.47:0/2883652572 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558a11dd3b50 con 0x558a11eda330 2024-03-21T11:32:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f730e000640 10 client.28723 handle_command_reply: tid=0 2024-03-21T11:32:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f7332016740 1 client.28723 shutdown 2024-03-21T11:32:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f7332016740 1 -- 172.21.15.47:0/2883652572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e67810 msgr2=0x558a11e75120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f7332016740 1 --2- 172.21.15.47:0/2883652572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e67810 0x558a11e75120 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f731c00ad90 tx=0x7f731c004730 comp rx=0 tx=0).stop 2024-03-21T11:32:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f7332016740 1 -- 172.21.15.47:0/2883652572 shutdown_connections 2024-03-21T11:32:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f7332016740 1 --2- 172.21.15.47:0/2883652572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a11e693a0 0x558a11e78680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f7332016740 1 --2- 172.21.15.47:0/2883652572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a11e689e0 0x558a11e75660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f7332016740 1 --2- 172.21.15.47:0/2883652572 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x558a11eda330 0x558a11edc750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f7332016740 1 --2- 172.21.15.47:0/2883652572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a11e67810 0x558a11e75120 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f7332016740 1 -- 172.21.15.47:0/2883652572 >> 172.21.15.47:0/2883652572 conn(0x558a11e62c00 msgr2=0x558a11e84b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f730e000640 0 client.28723 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:32:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f7332016740 1 -- 172.21.15.47:0/2883652572 shutdown_connections 2024-03-21T11:32:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.442+0000 7f7332016740 1 -- 172.21.15.47:0/2883652572 wait complete. 2024-03-21T11:32:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.443+0000 7f7332016740 20 client.28723 trim_cache size 0 max 16384 2024-03-21T11:32:54.447 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:32:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.443+0000 7f732f400640 1 -- 172.21.15.47:0/3754017335 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73100a2fa0 msgr2=0x7f73100a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.443+0000 7f732f400640 1 --2- 172.21.15.47:0/3754017335 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73100a2fa0 0x7f73100a5460 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f731c00afb0 tx=0x7f731c00a380 comp rx=0 tx=0).stop 2024-03-21T11:32:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.443+0000 7f732f400640 1 -- 172.21.15.47:0/3754017335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7328109000 msgr2=0x7f7328109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.443+0000 7f732f400640 1 --2- 172.21.15.47:0/3754017335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7328109000 0x7f7328109970 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7314036040 tx=0x7f7314004eb0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.443+0000 7f732f400640 1 -- 172.21.15.47:0/3754017335 shutdown_connections 2024-03-21T11:32:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.443+0000 7f732f400640 1 --2- 172.21.15.47:0/3754017335 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f73100a2fa0 0x7f73100a5460 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.443+0000 7f732f400640 1 --2- 172.21.15.47:0/3754017335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732810ff20 0x7f7328109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.443+0000 7f732f400640 1 --2- 172.21.15.47:0/3754017335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7328109000 0x7f7328109970 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.443+0000 7f732f400640 1 --2- 172.21.15.47:0/3754017335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281086a0 0x7f732810b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.443+0000 7f732f400640 1 -- 172.21.15.47:0/3754017335 >> 172.21.15.47:0/3754017335 conn(0x7f732806e500 msgr2=0x7f7328070160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.444+0000 7f732f400640 1 -- 172.21.15.47:0/3754017335 shutdown_connections 2024-03-21T11:32:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.444+0000 7f732f400640 1 -- 172.21.15.47:0/3754017335 wait complete. 2024-03-21T11:32:54.464 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:32:54.464 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:32:54.547 INFO:tasks.workunit.client.0.smithi047.stdout: 1 702952 8.14 MB/sec execute 338 sec latency 40.037 ms 2024-03-21T11:32:54.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.660+0000 7f8996c00640 1 -- 172.21.15.47:0/233136880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89901015a0 msgr2=0x7f89901019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.660+0000 7f8996c00640 1 --2- 172.21.15.47:0/233136880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89901015a0 0x7f89901019c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8984009900 tx=0x7f898402bc30 comp rx=0 tx=0).stop 2024-03-21T11:32:54.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.660+0000 7f8996c00640 1 -- 172.21.15.47:0/233136880 shutdown_connections 2024-03-21T11:32:54.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.660+0000 7f8996c00640 1 --2- 172.21.15.47:0/233136880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8990103150 0x7f89901099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.660+0000 7f8996c00640 1 --2- 172.21.15.47:0/233136880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8990102790 0x7f8990102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.660+0000 7f8996c00640 1 --2- 172.21.15.47:0/233136880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89901015a0 0x7f89901019c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.660+0000 7f8996c00640 1 -- 172.21.15.47:0/233136880 >> 172.21.15.47:0/233136880 conn(0x7f89900fcb10 msgr2=0x7f89900fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.660+0000 7f8996c00640 1 -- 172.21.15.47:0/233136880 shutdown_connections 2024-03-21T11:32:54.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.660+0000 7f8996c00640 1 -- 172.21.15.47:0/233136880 wait complete. 2024-03-21T11:32:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.661+0000 7f8996c00640 1 Processor -- start 2024-03-21T11:32:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.661+0000 7f8996c00640 1 -- start start 2024-03-21T11:32:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.661+0000 7f8996c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89901015a0 0x7f8990075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.661+0000 7f8996c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8990102790 0x7f8990073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.661+0000 7f8996c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8990103150 0x7f8990073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.661+0000 7f8996c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899010d260 con 0x7f89901015a0 2024-03-21T11:32:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.661+0000 7f8996c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899010d0e0 con 0x7f8990103150 2024-03-21T11:32:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.661+0000 7f8996c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899010d3e0 con 0x7f8990102790 2024-03-21T11:32:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.662+0000 7f8994e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8990102790 0x7f8990073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.662+0000 7f8996200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8990103150 0x7f8990073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.662+0000 7f8994e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8990102790 0x7f8990073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50704/0 (socket says 172.21.15.47:50704) 2024-03-21T11:32:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.662+0000 7f8994e00640 1 -- 172.21.15.47:0/1084641623 learned_addr learned my addr 172.21.15.47:0/1084641623 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.662+0000 7f8995800640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89901015a0 0x7f8990075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.662+0000 7f8994e00640 1 -- 172.21.15.47:0/1084641623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8990103150 msgr2=0x7f8990073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.662+0000 7f8994e00640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8990103150 0x7f8990073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.662+0000 7f8994e00640 1 -- 172.21.15.47:0/1084641623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89901015a0 msgr2=0x7f8990075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.662+0000 7f8994e00640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89901015a0 0x7f8990075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.662+0000 7f8994e00640 1 -- 172.21.15.47:0/1084641623 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8990074510 con 0x7f8990102790 2024-03-21T11:32:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.662+0000 7f8996200640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8990103150 0x7f8990073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.662+0000 7f8995800640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89901015a0 0x7f8990075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.662+0000 7f8994e00640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8990102790 0x7f8990073a90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f897c00e9f0 tx=0x7f897c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.663+0000 7f898e000640 1 -- 172.21.15.47:0/1084641623 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c00cd80 con 0x7f8990102790 2024-03-21T11:32:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.663+0000 7f898e000640 1 -- 172.21.15.47:0/1084641623 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f897c005500 con 0x7f8990102790 2024-03-21T11:32:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.663+0000 7f8996c00640 1 -- 172.21.15.47:0/1084641623 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8990074800 con 0x7f8990102790 2024-03-21T11:32:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.663+0000 7f898e000640 1 -- 172.21.15.47:0/1084641623 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c010690 con 0x7f8990102790 2024-03-21T11:32:54.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.663+0000 7f8996c00640 1 -- 172.21.15.47:0/1084641623 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89901a6340 con 0x7f8990102790 2024-03-21T11:32:54.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.667+0000 7f899ae51740 1 Processor -- start 2024-03-21T11:32:54.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.667+0000 7f899ae51740 1 -- start start 2024-03-21T11:32:54.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.667+0000 7f899ae51740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbda9f0 0x55720bbdae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.667+0000 7f899ae51740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbdbbc0 0x55720bbdc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.667+0000 7f899ae51740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdc580 0x55720bbe2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.667+0000 7f899ae51740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720bb3f2c0 con 0x55720bbda9f0 2024-03-21T11:32:54.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.668+0000 7f899ae51740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720bae7c90 con 0x55720bbdc580 2024-03-21T11:32:54.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.668+0000 7f899ae51740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720bb3b360 con 0x55720bbdbbc0 2024-03-21T11:32:54.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.668+0000 7f8996200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbda9f0 0x55720bbdae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.668+0000 7f8995800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbdbbc0 0x55720bbdc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.668+0000 7f8995800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbdbbc0 0x55720bbdc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50706/0 (socket says 172.21.15.47:50706) 2024-03-21T11:32:54.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.668+0000 7f8995800640 1 -- 172.21.15.47:0/2653817159 learned_addr learned my addr 172.21.15.47:0/2653817159 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.668+0000 7f898e000640 1 -- 172.21.15.47:0/1084641623 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f897c0046c0 con 0x7f8990102790 2024-03-21T11:32:54.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.668+0000 7f8995800640 1 -- 172.21.15.47:0/2653817159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdc580 msgr2=0x55720bbe2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.668+0000 7f8995800640 1 --2- 172.21.15.47:0/2653817159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdc580 0x55720bbe2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.668+0000 7f8995800640 1 -- 172.21.15.47:0/2653817159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbda9f0 msgr2=0x55720bbdae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.668+0000 7f8995800640 1 --2- 172.21.15.47:0/2653817159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbda9f0 0x55720bbdae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.668+0000 7f8995800640 1 -- 172.21.15.47:0/2653817159 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55720bb0c080 con 0x55720bbdbbc0 2024-03-21T11:32:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.669+0000 7f8995800640 1 --2- 172.21.15.47:0/2653817159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbdbbc0 0x55720bbdc040 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8984000c00 tx=0x7f89840095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3849aa7f778b4464 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.669+0000 7f8982a00640 1 -- 172.21.15.47:0/2653817159 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984044020 con 0x55720bbdbbc0 2024-03-21T11:32:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.669+0000 7f8982a00640 1 -- 172.21.15.47:0/2653817159 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f898402cc40 con 0x55720bbdbbc0 2024-03-21T11:32:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.669+0000 7f8982a00640 1 -- 172.21.15.47:0/2653817159 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984004920 con 0x55720bbdbbc0 2024-03-21T11:32:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.670+0000 7f898e000640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f89780a2f20 0x7f89780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.670+0000 7f898e000640 1 -- 172.21.15.47:0/1084641623 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f897c0cfbf0 con 0x7f8990102790 2024-03-21T11:32:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.670+0000 7f899ae51740 1 -- 172.21.15.47:0/2653817159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbdbbc0 msgr2=0x55720bbdc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.670+0000 7f899ae51740 1 --2- 172.21.15.47:0/2653817159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbdbbc0 0x55720bbdc040 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8984000c00 tx=0x7f89840095e0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.670+0000 7f8996200640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f89780a2f20 0x7f89780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.670+0000 7f899ae51740 1 -- 172.21.15.47:0/2653817159 shutdown_connections 2024-03-21T11:32:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.670+0000 7f899ae51740 1 --2- 172.21.15.47:0/2653817159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdc580 0x55720bbe2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.670+0000 7f899ae51740 1 --2- 172.21.15.47:0/2653817159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbdbbc0 0x55720bbdc040 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.670+0000 7f899ae51740 1 --2- 172.21.15.47:0/2653817159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbda9f0 0x55720bbdae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.670+0000 7f899ae51740 1 -- 172.21.15.47:0/2653817159 >> 172.21.15.47:0/2653817159 conn(0x55720bbd5de0 msgr2=0x55720bbd61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.670+0000 7f899ae51740 1 -- 172.21.15.47:0/2653817159 shutdown_connections 2024-03-21T11:32:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.670+0000 7f899ae51740 1 -- 172.21.15.47:0/2653817159 wait complete. 2024-03-21T11:32:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.671+0000 7f899ae51740 1 -- start start 2024-03-21T11:32:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.671+0000 7f8996200640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f89780a2f20 0x7f89780a53e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55720bbdbac0 tx=0x7f89880096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.671+0000 7f899ae51740 1 Processor -- start 2024-03-21T11:32:54.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f899ae51740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbda9f0 0x55720bbe4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f899ae51740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbdbbc0 0x55720bbe51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f899ae51740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdc580 0x55720bbec220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f899ae51740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720bbe3530 con 0x55720bbdbbc0 2024-03-21T11:32:54.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f899ae51740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720bafb0d0 con 0x55720bbdc580 2024-03-21T11:32:54.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f899ae51740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720bbe36b0 con 0x55720bbda9f0 2024-03-21T11:32:54.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f899ae51740 10 client.0 _ll_register_callbacks cb 0x55720baffe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:54.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f8995800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbda9f0 0x55720bbe4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f8995800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbda9f0 0x55720bbe4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50714/0 (socket says 172.21.15.47:50714) 2024-03-21T11:32:54.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f8995800640 1 -- 172.21.15.47:0/2017068668 learned_addr learned my addr 172.21.15.47:0/2017068668 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f8996200640 1 --2- 172.21.15.47:0/2017068668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdc580 0x55720bbec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f8994e00640 1 --2- 172.21.15.47:0/2017068668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbdbbc0 0x55720bbe51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f8995800640 1 -- 172.21.15.47:0/2017068668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdc580 msgr2=0x55720bbec220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f8995800640 1 --2- 172.21.15.47:0/2017068668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdc580 0x55720bbec220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f8995800640 1 -- 172.21.15.47:0/2017068668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbdbbc0 msgr2=0x55720bbe51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f8995800640 1 --2- 172.21.15.47:0/2017068668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbdbbc0 0x55720bbe51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.672+0000 7f8995800640 1 -- 172.21.15.47:0/2017068668 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55720b8533f0 con 0x55720bbda9f0 2024-03-21T11:32:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.673+0000 7f8995800640 1 --2- 172.21.15.47:0/2017068668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbda9f0 0x55720bbe4ca0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8984036040 tx=0x7f898402cdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e14c255317408da server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.673+0000 7f8977400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.673+0000 7f8977400640 1 -- 172.21.15.47:0/2017068668 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898404d070 con 0x55720bbda9f0 2024-03-21T11:32:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.673+0000 7f8977400640 1 -- 172.21.15.47:0/2017068668 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8984003800 con 0x55720bbda9f0 2024-03-21T11:32:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.673+0000 7f899ae51740 1 -- 172.21.15.47:0/2017068668 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f898402cc40 con 0x55720bbda9f0 2024-03-21T11:32:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.673+0000 7f8977400640 1 -- 172.21.15.47:0/2017068668 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8984059050 con 0x55720bbda9f0 2024-03-21T11:32:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.673+0000 7f899ae51740 10 client.28735 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.673+0000 7f899ae51740 1 -- 172.21.15.47:0/2017068668 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55720b91d7d0 con 0x55720bbda9f0 2024-03-21T11:32:54.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.674+0000 7f8977400640 1 -- 172.21.15.47:0/2017068668 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f898402cc40 con 0x55720bbda9f0 2024-03-21T11:32:54.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.674+0000 7f8977400640 1 -- 172.21.15.47:0/2017068668 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8984040bc0 con 0x55720bbda9f0 2024-03-21T11:32:54.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.674+0000 7f899ae51740 10 client.28735 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:54.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.674+0000 7f899ae51740 10 client.28735 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:32:54.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.674+0000 7f899ae51740 20 client.28735 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:54.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.674+0000 7f899ae51740 1 --2- 172.21.15.47:0/2017068668 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720bbf0380 0x55720bbf2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.674+0000 7f899ae51740 4 client.28735 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:54.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.674+0000 7f899ae51740 1 -- 172.21.15.47:0/2017068668 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55720bb4c800 con 0x55720bbf0380 2024-03-21T11:32:54.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.674+0000 7f8994e00640 1 --2- 172.21.15.47:0/2017068668 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720bbf0380 0x55720bbf2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.675+0000 7f8994e00640 1 --2- 172.21.15.47:0/2017068668 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720bbf0380 0x55720bbf2770 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=20c29e8a8f635870 server_cookie=15d2a3e818c37119 in_seq=0 out_seq=0 2024-03-21T11:32:54.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.675+0000 7f8977400640 10 client.28735 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:32:54.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f8977400640 1 -- 172.21.15.47:0/2017068668 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55720bb4c800 con 0x55720bbf0380 2024-03-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f8977400640 10 client.28735 handle_command_reply: tid=0 2024-03-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f899ae51740 1 client.28735 shutdown 2024-03-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f899ae51740 1 -- 172.21.15.47:0/2017068668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbda9f0 msgr2=0x55720bbe4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f899ae51740 1 --2- 172.21.15.47:0/2017068668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbda9f0 0x55720bbe4ca0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8984036040 tx=0x7f898402cdf0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f899ae51740 1 -- 172.21.15.47:0/2017068668 shutdown_connections 2024-03-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f899ae51740 1 --2- 172.21.15.47:0/2017068668 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720bbf0380 0x55720bbf2770 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f899ae51740 1 --2- 172.21.15.47:0/2017068668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdc580 0x55720bbec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f899ae51740 1 --2- 172.21.15.47:0/2017068668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbdbbc0 0x55720bbe51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f899ae51740 1 --2- 172.21.15.47:0/2017068668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbda9f0 0x55720bbe4ca0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f8977400640 0 client.28735 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f899ae51740 1 -- 172.21.15.47:0/2017068668 >> 172.21.15.47:0/2017068668 conn(0x55720bbd5de0 msgr2=0x55720bbd5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f899ae51740 1 -- 172.21.15.47:0/2017068668 shutdown_connections 2024-03-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f899ae51740 1 -- 172.21.15.47:0/2017068668 wait complete. 2024-03-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.676+0000 7f899ae51740 20 client.28735 trim_cache size 0 max 16384 2024-03-21T11:32:54.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.690+0000 7f899ae51740 1 Processor -- start 2024-03-21T11:32:54.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.690+0000 7f899ae51740 1 -- start start 2024-03-21T11:32:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f899ae51740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbda9f0 0x55720bbdec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f899ae51740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbdbbc0 0x55720bbdf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f899ae51740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdc580 0x55720bbdf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f899ae51740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89901b0400 con 0x55720bbda9f0 2024-03-21T11:32:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f899ae51740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89901b1d80 con 0x55720bbdc580 2024-03-21T11:32:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f899ae51740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89901b0a60 con 0x55720bbdbbc0 2024-03-21T11:32:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f8994e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbdbbc0 0x55720bbdf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f8995800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbda9f0 0x55720bbdec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f8996200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdc580 0x55720bbdf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f8994e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbdbbc0 0x55720bbdf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50724/0 (socket says 172.21.15.47:50724) 2024-03-21T11:32:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f8994e00640 1 -- 172.21.15.47:0/2222672258 learned_addr learned my addr 172.21.15.47:0/2222672258 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f8994e00640 1 -- 172.21.15.47:0/2222672258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdc580 msgr2=0x55720bbdf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f8994e00640 1 --2- 172.21.15.47:0/2222672258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdc580 0x55720bbdf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f8994e00640 1 -- 172.21.15.47:0/2222672258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbda9f0 msgr2=0x55720bbdec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f8994e00640 1 --2- 172.21.15.47:0/2222672258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbda9f0 0x55720bbdec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.691+0000 7f8994e00640 1 -- 172.21.15.47:0/2222672258 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55720bbdb640 con 0x55720bbdbbc0 2024-03-21T11:32:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.692+0000 7f8994e00640 1 --2- 172.21.15.47:0/2222672258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbdbbc0 0x55720bbdf1a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f897c098960 tx=0x7f897c009720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=915d7c075b801152 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.692+0000 7f8982000640 1 -- 172.21.15.47:0/2222672258 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c0b4070 con 0x55720bbdbbc0 2024-03-21T11:32:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.692+0000 7f8982000640 1 -- 172.21.15.47:0/2222672258 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f897c09bf90 con 0x55720bbdbbc0 2024-03-21T11:32:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.692+0000 7f8982000640 1 -- 172.21.15.47:0/2222672258 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c098600 con 0x55720bbdbbc0 2024-03-21T11:32:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.693+0000 7f899ae51740 1 -- 172.21.15.47:0/2222672258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbdbbc0 msgr2=0x55720bbdf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.693+0000 7f899ae51740 1 --2- 172.21.15.47:0/2222672258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbdbbc0 0x55720bbdf1a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f897c098960 tx=0x7f897c009720 comp rx=0 tx=0).stop 2024-03-21T11:32:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.693+0000 7f899ae51740 1 -- 172.21.15.47:0/2222672258 shutdown_connections 2024-03-21T11:32:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.693+0000 7f899ae51740 1 --2- 172.21.15.47:0/2222672258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdc580 0x55720bbdf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.693+0000 7f899ae51740 1 --2- 172.21.15.47:0/2222672258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbdbbc0 0x55720bbdf1a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.693+0000 7f899ae51740 1 --2- 172.21.15.47:0/2222672258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbda9f0 0x55720bbdec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.693+0000 7f899ae51740 1 -- 172.21.15.47:0/2222672258 >> 172.21.15.47:0/2222672258 conn(0x55720bbd5de0 msgr2=0x55720bbd98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.693+0000 7f899ae51740 1 -- 172.21.15.47:0/2222672258 shutdown_connections 2024-03-21T11:32:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.693+0000 7f899ae51740 1 -- 172.21.15.47:0/2222672258 wait complete. 2024-03-21T11:32:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.693+0000 7f899ae51740 1 -- start start 2024-03-21T11:32:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.693+0000 7f899ae51740 1 Processor -- start 2024-03-21T11:32:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f899ae51740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbda9f0 0x55720bbe7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f899ae51740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdbbc0 0x55720bbe7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f899ae51740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbdc580 0x55720bbeaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f899ae51740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89901b1720 con 0x55720bbdc580 2024-03-21T11:32:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f899ae51740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89901b10c0 con 0x55720bbdbbc0 2024-03-21T11:32:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f899ae51740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89901ae5c0 con 0x55720bbda9f0 2024-03-21T11:32:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f899ae51740 10 client.0 _ll_register_callbacks cb 0x7f8988009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f8994e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdbbc0 0x55720bbe7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f8995800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbda9f0 0x55720bbe7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f8994e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdbbc0 0x55720bbe7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60996/0 (socket says 172.21.15.47:60996) 2024-03-21T11:32:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f8994e00640 1 -- 172.21.15.47:0/2279720354 learned_addr learned my addr 172.21.15.47:0/2279720354 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f8996200640 1 --2- 172.21.15.47:0/2279720354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbdc580 0x55720bbeaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f8995800640 1 -- 172.21.15.47:0/2279720354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdbbc0 msgr2=0x55720bbe7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f8995800640 1 --2- 172.21.15.47:0/2279720354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdbbc0 0x55720bbe7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f8995800640 1 -- 172.21.15.47:0/2279720354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbdc580 msgr2=0x55720bbeaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f8995800640 1 --2- 172.21.15.47:0/2279720354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbdc580 0x55720bbeaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f8995800640 1 -- 172.21.15.47:0/2279720354 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55720bbeb070 con 0x55720bbda9f0 2024-03-21T11:32:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.694+0000 7f8996200640 1 --2- 172.21.15.47:0/2279720354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbdc580 0x55720bbeaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.695+0000 7f8995800640 1 --2- 172.21.15.47:0/2279720354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbda9f0 0x55720bbe7520 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f898402c8e0 tx=0x7f898402c320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e3b8202b0bb7e88 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.695+0000 7f8977400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.695+0000 7f8977400640 1 -- 172.21.15.47:0/2279720354 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984056020 con 0x55720bbda9f0 2024-03-21T11:32:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.695+0000 7f899ae51740 1 -- 172.21.15.47:0/2279720354 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f897c09bf90 con 0x55720bbda9f0 2024-03-21T11:32:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.695+0000 7f8977400640 1 -- 172.21.15.47:0/2279720354 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f89840349b0 con 0x55720bbda9f0 2024-03-21T11:32:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.695+0000 7f8977400640 1 -- 172.21.15.47:0/2279720354 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f898404d040 con 0x55720bbda9f0 2024-03-21T11:32:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.695+0000 7f899ae51740 10 client.28741 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.695+0000 7f899ae51740 1 -- 172.21.15.47:0/2279720354 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55720bc4bed0 con 0x55720bbda9f0 2024-03-21T11:32:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.695+0000 7f8977400640 1 -- 172.21.15.47:0/2279720354 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f897c09bf90 con 0x55720bbda9f0 2024-03-21T11:32:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.696+0000 7f8977400640 1 -- 172.21.15.47:0/2279720354 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8984040470 con 0x55720bbda9f0 2024-03-21T11:32:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.696+0000 7f899ae51740 10 client.28741 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.696+0000 7f899ae51740 10 client.28741 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:32:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.696+0000 7f899ae51740 20 client.28741 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:54.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.696+0000 7f899ae51740 1 --2- 172.21.15.47:0/2279720354 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720bc4c4c0 0x55720bc4e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.696+0000 7f899ae51740 4 client.28741 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:32:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.696+0000 7f899ae51740 1 -- 172.21.15.47:0/2279720354 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55720bb46b50 con 0x55720bc4c4c0 2024-03-21T11:32:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.696+0000 7f8994e00640 1 --2- 172.21.15.47:0/2279720354 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720bc4c4c0 0x55720bc4e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.696+0000 7f8994e00640 1 --2- 172.21.15.47:0/2279720354 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720bc4c4c0 0x55720bc4e960 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af9a94aedcc9d97 server_cookie=715df19dd7cd6e34 in_seq=0 out_seq=0 2024-03-21T11:32:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.696+0000 7f8977400640 10 client.28741 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:32:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f8977400640 1 -- 172.21.15.47:0/2279720354 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55720bb46b50 con 0x55720bc4c4c0 2024-03-21T11:32:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f8977400640 10 client.28741 handle_command_reply: tid=0 2024-03-21T11:32:54.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f899ae51740 1 client.28741 shutdown 2024-03-21T11:32:54.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f899ae51740 1 -- 172.21.15.47:0/2279720354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbda9f0 msgr2=0x55720bbe7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f899ae51740 1 --2- 172.21.15.47:0/2279720354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbda9f0 0x55720bbe7520 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f898402c8e0 tx=0x7f898402c320 comp rx=0 tx=0).stop 2024-03-21T11:32:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f899ae51740 1 -- 172.21.15.47:0/2279720354 shutdown_connections 2024-03-21T11:32:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f899ae51740 1 --2- 172.21.15.47:0/2279720354 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720bc4c4c0 0x55720bc4e960 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f899ae51740 1 --2- 172.21.15.47:0/2279720354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720bbdc580 0x55720bbeaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f899ae51740 1 --2- 172.21.15.47:0/2279720354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720bbdbbc0 0x55720bbe7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f8977400640 0 client.28741 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:32:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f899ae51740 1 --2- 172.21.15.47:0/2279720354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720bbda9f0 0x55720bbe7520 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f899ae51740 1 -- 172.21.15.47:0/2279720354 >> 172.21.15.47:0/2279720354 conn(0x55720bbd5de0 msgr2=0x55720bbd48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f899ae51740 1 -- 172.21.15.47:0/2279720354 shutdown_connections 2024-03-21T11:32:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f899ae51740 1 -- 172.21.15.47:0/2279720354 wait complete. 2024-03-21T11:32:54.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.697+0000 7f899ae51740 20 client.28741 trim_cache size 0 max 16384 2024-03-21T11:32:54.708 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:32:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.698+0000 7f8996c00640 1 -- 172.21.15.47:0/1084641623 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f89780a2f20 msgr2=0x7f89780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.698+0000 7f8996c00640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f89780a2f20 0x7f89780a53e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55720bbdbac0 tx=0x7f89880096a0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.698+0000 7f8996c00640 1 -- 172.21.15.47:0/1084641623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8990102790 msgr2=0x7f8990073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.698+0000 7f8996c00640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8990102790 0x7f8990073a90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f897c00e9f0 tx=0x7f897c00edf0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.698+0000 7f8996c00640 1 -- 172.21.15.47:0/1084641623 shutdown_connections 2024-03-21T11:32:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.698+0000 7f8996c00640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f89780a2f20 0x7f89780a53e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.698+0000 7f8996c00640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8990103150 0x7f8990073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.698+0000 7f8996c00640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8990102790 0x7f8990073a90 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.698+0000 7f8996c00640 1 --2- 172.21.15.47:0/1084641623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89901015a0 0x7f8990075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.698+0000 7f8996c00640 1 -- 172.21.15.47:0/1084641623 >> 172.21.15.47:0/1084641623 conn(0x7f89900fcb10 msgr2=0x7f8990105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.698+0000 7f8996c00640 1 -- 172.21.15.47:0/1084641623 shutdown_connections 2024-03-21T11:32:54.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.698+0000 7f8996c00640 1 -- 172.21.15.47:0/1084641623 wait complete. 2024-03-21T11:32:54.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:54 smithi047 ceph-mon[31351]: pgmap v687: 129 pgs: 129 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 317 op/s 2024-03-21T11:32:54.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:54 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2246166564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:32:54.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:54 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:32:54.716 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:32:54.716 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:32:54.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:54 smithi102 ceph-mon[36748]: pgmap v687: 129 pgs: 129 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 317 op/s 2024-03-21T11:32:54.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:54 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2246166564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:32:54.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:54 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:32:54.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:54 smithi100 ceph-mon[36245]: pgmap v687: 129 pgs: 129 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 317 op/s 2024-03-21T11:32:54.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:54 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2246166564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:32:54.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:54 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:32:54.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.923+0000 7effcb000640 1 -- 172.21.15.47:0/2961549695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effc4074100 msgr2=0x7effc4074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.924+0000 7effcb000640 1 --2- 172.21.15.47:0/2961549695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effc4074100 0x7effc4074580 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7effb0009900 tx=0x7effb002bc30 comp rx=0 tx=0).stop 2024-03-21T11:32:54.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.924+0000 7effcb000640 1 -- 172.21.15.47:0/2961549695 shutdown_connections 2024-03-21T11:32:54.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.924+0000 7effcb000640 1 --2- 172.21.15.47:0/2961549695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effc410db70 0x7effc410ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.924+0000 7effcb000640 1 --2- 172.21.15.47:0/2961549695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effc4074100 0x7effc4074580 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.924+0000 7effcb000640 1 --2- 172.21.15.47:0/2961549695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc40756e0 0x7effc4073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.924+0000 7effcb000640 1 -- 172.21.15.47:0/2961549695 >> 172.21.15.47:0/2961549695 conn(0x7effc40fcb10 msgr2=0x7effc40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.924+0000 7effcb000640 1 -- 172.21.15.47:0/2961549695 shutdown_connections 2024-03-21T11:32:54.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.924+0000 7effcb000640 1 -- 172.21.15.47:0/2961549695 wait complete. 2024-03-21T11:32:54.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.925+0000 7effcb000640 1 Processor -- start 2024-03-21T11:32:54.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.925+0000 7effcb000640 1 -- start start 2024-03-21T11:32:54.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.925+0000 7effcb000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effc4074100 0x7effc419def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.925+0000 7effcb000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effc40756e0 0x7effc419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.925+0000 7effcb000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc410db70 0x7effc41a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.925+0000 7effcb000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc4113840 con 0x7effc40756e0 2024-03-21T11:32:54.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.925+0000 7effcb000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc41136c0 con 0x7effc410db70 2024-03-21T11:32:54.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.925+0000 7effcb000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc41139c0 con 0x7effc4074100 2024-03-21T11:32:54.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.925+0000 7effc9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effc4074100 0x7effc419def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.925+0000 7effca600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc410db70 0x7effc41a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.925+0000 7effc9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effc4074100 0x7effc419def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50756/0 (socket says 172.21.15.47:50756) 2024-03-21T11:32:54.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.925+0000 7effca600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc410db70 0x7effc41a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32776/0 (socket says 172.21.15.47:32776) 2024-03-21T11:32:54.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.925+0000 7effc9c00640 1 -- 172.21.15.47:0/2721718461 learned_addr learned my addr 172.21.15.47:0/2721718461 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.925+0000 7effc9200640 1 --2- 172.21.15.47:0/2721718461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effc40756e0 0x7effc419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.926+0000 7effca600640 1 -- 172.21.15.47:0/2721718461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effc4074100 msgr2=0x7effc419def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.926+0000 7effca600640 1 --2- 172.21.15.47:0/2721718461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effc4074100 0x7effc419def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.926+0000 7effca600640 1 -- 172.21.15.47:0/2721718461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effc40756e0 msgr2=0x7effc419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.926+0000 7effca600640 1 --2- 172.21.15.47:0/2721718461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effc40756e0 0x7effc419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.926+0000 7effca600640 1 -- 172.21.15.47:0/2721718461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc41a5b10 con 0x7effc410db70 2024-03-21T11:32:54.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.926+0000 7effc9c00640 1 --2- 172.21.15.47:0/2721718461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effc4074100 0x7effc419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:54.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.926+0000 7effca600640 1 --2- 172.21.15.47:0/2721718461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc410db70 0x7effc41a54a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7effbc009fd0 tx=0x7effbc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.926+0000 7effc2000640 1 -- 172.21.15.47:0/2721718461 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc00ce20 con 0x7effc410db70 2024-03-21T11:32:54.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.926+0000 7effc2000640 1 -- 172.21.15.47:0/2721718461 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effbc005020 con 0x7effc410db70 2024-03-21T11:32:54.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.926+0000 7effc2000640 1 -- 172.21.15.47:0/2721718461 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc010720 con 0x7effc410db70 2024-03-21T11:32:54.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.926+0000 7effcb000640 1 -- 172.21.15.47:0/2721718461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc41a5e00 con 0x7effc410db70 2024-03-21T11:32:54.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.927+0000 7effcb000640 1 -- 172.21.15.47:0/2721718461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc41a6270 con 0x7effc410db70 2024-03-21T11:32:54.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.930+0000 7effcf26a740 1 Processor -- start 2024-03-21T11:32:54.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.930+0000 7effc2000640 1 -- 172.21.15.47:0/2721718461 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7effbc005540 con 0x7effc410db70 2024-03-21T11:32:54.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.930+0000 7effcf26a740 1 -- start start 2024-03-21T11:32:54.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effcf26a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 0x55f074f8de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effcf26a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8ebc0 0x55f074f8f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effcf26a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8f580 0x55f074f95e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effcf26a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074ef22c0 con 0x55f074f8d9f0 2024-03-21T11:32:54.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effcf26a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074e9ac90 con 0x55f074f8f580 2024-03-21T11:32:54.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effcf26a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074eee360 con 0x55f074f8ebc0 2024-03-21T11:32:54.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effc9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8ebc0 0x55f074f8f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effc9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8ebc0 0x55f074f8f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50760/0 (socket says 172.21.15.47:50760) 2024-03-21T11:32:54.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effc9c00640 1 -- 172.21.15.47:0/2035059056 learned_addr learned my addr 172.21.15.47:0/2035059056 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effc9200640 1 --2- 172.21.15.47:0/2035059056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 0x55f074f8de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effc9c00640 1 -- 172.21.15.47:0/2035059056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8f580 msgr2=0x55f074f95e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effc9200640 1 --2- 172.21.15.47:0/2035059056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8f580 0x55f074f95e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effc9c00640 1 --2- 172.21.15.47:0/2035059056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8f580 0x55f074f95e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effc9c00640 1 -- 172.21.15.47:0/2035059056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 msgr2=0x55f074f8de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effc9c00640 1 --2- 172.21.15.47:0/2035059056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 0x55f074f8de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effc9c00640 1 -- 172.21.15.47:0/2035059056 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f074ebf080 con 0x55f074f8ebc0 2024-03-21T11:32:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.931+0000 7effc9200640 1 --2- 172.21.15.47:0/2035059056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8f580 0x55f074f95e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:32:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.932+0000 7effc9c00640 1 --2- 172.21.15.47:0/2035059056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8ebc0 0x55f074f8f040 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7effb4009fd0 tx=0x7effb4011ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaaabb94335ab7a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.932+0000 7effc2000640 1 --2- 172.21.15.47:0/2721718461 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7effa80a2fa0 0x7effa80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.932+0000 7effbaa00640 1 -- 172.21.15.47:0/2035059056 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb401e070 con 0x55f074f8ebc0 2024-03-21T11:32:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.932+0000 7effbaa00640 1 -- 172.21.15.47:0/2035059056 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effb400fbd0 con 0x55f074f8ebc0 2024-03-21T11:32:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.932+0000 7effbaa00640 1 -- 172.21.15.47:0/2035059056 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4019d80 con 0x55f074f8ebc0 2024-03-21T11:32:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.932+0000 7effc2000640 1 -- 172.21.15.47:0/2721718461 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7effbc0cfe30 con 0x7effc410db70 2024-03-21T11:32:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.932+0000 7effc9200640 1 --2- 172.21.15.47:0/2721718461 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7effa80a2fa0 0x7effa80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.933+0000 7effcf26a740 1 -- 172.21.15.47:0/2035059056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8ebc0 msgr2=0x55f074f8f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.933+0000 7effcf26a740 1 --2- 172.21.15.47:0/2035059056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8ebc0 0x55f074f8f040 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7effb4009fd0 tx=0x7effb4011ae0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.933+0000 7effc9200640 1 --2- 172.21.15.47:0/2721718461 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7effa80a2fa0 0x7effa80a5460 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7effb002c240 tx=0x7effb002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.933+0000 7effcf26a740 1 -- 172.21.15.47:0/2035059056 shutdown_connections 2024-03-21T11:32:54.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.933+0000 7effcf26a740 1 --2- 172.21.15.47:0/2035059056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8f580 0x55f074f95e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.933+0000 7effcf26a740 1 --2- 172.21.15.47:0/2035059056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8ebc0 0x55f074f8f040 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.933+0000 7effcf26a740 1 --2- 172.21.15.47:0/2035059056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 0x55f074f8de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.933+0000 7effcf26a740 1 -- 172.21.15.47:0/2035059056 >> 172.21.15.47:0/2035059056 conn(0x55f074f88de0 msgr2=0x55f074f891e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.933+0000 7effcf26a740 1 -- 172.21.15.47:0/2035059056 shutdown_connections 2024-03-21T11:32:54.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.933+0000 7effcf26a740 1 -- 172.21.15.47:0/2035059056 wait complete. 2024-03-21T11:32:54.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.934+0000 7effcf26a740 1 -- start start 2024-03-21T11:32:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.934+0000 7effcf26a740 1 Processor -- start 2024-03-21T11:32:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.934+0000 7effcf26a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 0x55f074f97c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.934+0000 7effcf26a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 0x55f074f98190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.934+0000 7effcf26a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8f580 0x55f074f9f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.934+0000 7effcf26a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074f96590 con 0x55f074f8d9f0 2024-03-21T11:32:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.934+0000 7effcf26a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074eae0d0 con 0x55f074f8ebc0 2024-03-21T11:32:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.934+0000 7effcf26a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074f96710 con 0x55f074f8f580 2024-03-21T11:32:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.935+0000 7effcf26a740 10 client.0 _ll_register_callbacks cb 0x55f074eb2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.935+0000 7effc9c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 0x55f074f97c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.935+0000 7effca600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8f580 0x55f074f9f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.935+0000 7effca600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8f580 0x55f074f9f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50764/0 (socket says 172.21.15.47:50764) 2024-03-21T11:32:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.935+0000 7effc9c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 0x55f074f97c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33536/0 (socket says 172.21.15.47:33536) 2024-03-21T11:32:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.935+0000 7effca600640 1 -- 172.21.15.47:0/2445440446 learned_addr learned my addr 172.21.15.47:0/2445440446 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.935+0000 7effc9200640 1 --2- 172.21.15.47:0/2445440446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 0x55f074f98190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.935+0000 7effca600640 1 -- 172.21.15.47:0/2445440446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 msgr2=0x55f074f98190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.935+0000 7effca600640 1 --2- 172.21.15.47:0/2445440446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 0x55f074f98190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.935+0000 7effca600640 1 -- 172.21.15.47:0/2445440446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 msgr2=0x55f074f97c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.935+0000 7effca600640 1 --2- 172.21.15.47:0/2445440446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 0x55f074f97c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.935+0000 7effca600640 1 -- 172.21.15.47:0/2445440446 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f074c063f0 con 0x55f074f8f580 2024-03-21T11:32:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.935+0000 7effc9c00640 1 --2- 172.21.15.47:0/2445440446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 0x55f074f97c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.936+0000 7effca600640 1 --2- 172.21.15.47:0/2445440446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8f580 0x55f074f9f160 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7effbc0985d0 tx=0x7effbc09c2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f62e0583b623f2d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.936+0000 7effaf400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.936+0000 7effaf400640 1 -- 172.21.15.47:0/2445440446 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc0b5060 con 0x55f074f8f580 2024-03-21T11:32:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.936+0000 7effaf400640 1 -- 172.21.15.47:0/2445440446 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7effbc0051d0 con 0x55f074f8f580 2024-03-21T11:32:54.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.936+0000 7effcf26a740 1 -- 172.21.15.47:0/2445440446 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effb400fbd0 con 0x55f074f8f580 2024-03-21T11:32:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.936+0000 7effaf400640 1 -- 172.21.15.47:0/2445440446 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7effbc0a46f0 con 0x55f074f8f580 2024-03-21T11:32:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.936+0000 7effcf26a740 10 client.28753 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.936+0000 7effcf26a740 1 -- 172.21.15.47:0/2445440446 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f074cd07d0 con 0x55f074f8f580 2024-03-21T11:32:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.936+0000 7effaf400640 1 -- 172.21.15.47:0/2445440446 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effb400fbd0 con 0x55f074f8f580 2024-03-21T11:32:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.936+0000 7effaf400640 1 -- 172.21.15.47:0/2445440446 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7effbc096ce0 con 0x55f074f8f580 2024-03-21T11:32:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.937+0000 7effcf26a740 10 client.28753 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.937+0000 7effcf26a740 10 client.28753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.937+0000 7effcf26a740 20 client.28753 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.937+0000 7effcf26a740 1 --2- 172.21.15.47:0/2445440446 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f074fa35f0 0x55f074fa5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.937+0000 7effcf26a740 4 client.28753 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.937+0000 7effcf26a740 1 -- 172.21.15.47:0/2445440446 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f074eff800 con 0x55f074fa35f0 2024-03-21T11:32:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.937+0000 7effc9c00640 1 --2- 172.21.15.47:0/2445440446 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f074fa35f0 0x55f074fa5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.938+0000 7effc9c00640 1 --2- 172.21.15.47:0/2445440446 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f074fa35f0 0x55f074fa5a60 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c1e310b13d57b82 server_cookie=d0ff3f63073b627c in_seq=0 out_seq=0 2024-03-21T11:32:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.938+0000 7effaf400640 10 client.28753 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effaf400640 1 -- 172.21.15.47:0/2445440446 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f074eff800 con 0x55f074fa35f0 2024-03-21T11:32:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effaf400640 10 client.28753 handle_command_reply: tid=0 2024-03-21T11:32:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effcf26a740 1 client.28753 shutdown 2024-03-21T11:32:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effcf26a740 1 -- 172.21.15.47:0/2445440446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8f580 msgr2=0x55f074f9f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effcf26a740 1 --2- 172.21.15.47:0/2445440446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8f580 0x55f074f9f160 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7effbc0985d0 tx=0x7effbc09c2b0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effcf26a740 1 -- 172.21.15.47:0/2445440446 shutdown_connections 2024-03-21T11:32:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effcf26a740 1 --2- 172.21.15.47:0/2445440446 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f074fa35f0 0x55f074fa5a60 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effcf26a740 1 --2- 172.21.15.47:0/2445440446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8f580 0x55f074f9f160 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effcf26a740 1 --2- 172.21.15.47:0/2445440446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 0x55f074f98190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effcf26a740 1 --2- 172.21.15.47:0/2445440446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 0x55f074f97c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effaf400640 0 client.28753 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:54.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effcf26a740 1 -- 172.21.15.47:0/2445440446 >> 172.21.15.47:0/2445440446 conn(0x55f074f88de0 msgr2=0x55f074f88160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effcf26a740 1 -- 172.21.15.47:0/2445440446 shutdown_connections 2024-03-21T11:32:54.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effcf26a740 1 -- 172.21.15.47:0/2445440446 wait complete. 2024-03-21T11:32:54.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.940+0000 7effcf26a740 20 client.28753 trim_cache size 0 max 16384 2024-03-21T11:32:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.955+0000 7effcf26a740 1 Processor -- start 2024-03-21T11:32:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.955+0000 7effcf26a740 1 -- start start 2024-03-21T11:32:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.955+0000 7effcf26a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8d9f0 0x55f074f8c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.955+0000 7effcf26a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 0x55f074f9f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.955+0000 7effcf26a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8f580 0x55f074f91c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.955+0000 7effcf26a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc41b05b0 con 0x55f074f8f580 2024-03-21T11:32:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.955+0000 7effcf26a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc41b1f30 con 0x55f074f8ebc0 2024-03-21T11:32:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.955+0000 7effcf26a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc41b0c10 con 0x55f074f8d9f0 2024-03-21T11:32:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.955+0000 7effc9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8d9f0 0x55f074f8c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.955+0000 7effc9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8d9f0 0x55f074f8c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50772/0 (socket says 172.21.15.47:50772) 2024-03-21T11:32:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.955+0000 7effc9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 0x55f074f9f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.955+0000 7effc9c00640 1 -- 172.21.15.47:0/1807708334 learned_addr learned my addr 172.21.15.47:0/1807708334 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.956+0000 7effca600640 1 --2- 172.21.15.47:0/1807708334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8f580 0x55f074f91c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.956+0000 7effc9c00640 1 -- 172.21.15.47:0/1807708334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 msgr2=0x55f074f9f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.956+0000 7effc9c00640 1 --2- 172.21.15.47:0/1807708334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 0x55f074f9f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.956+0000 7effc9c00640 1 -- 172.21.15.47:0/1807708334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8f580 msgr2=0x55f074f91c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.956+0000 7effc9c00640 1 --2- 172.21.15.47:0/1807708334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8f580 0x55f074f91c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.956+0000 7effc9c00640 1 -- 172.21.15.47:0/1807708334 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f074f89830 con 0x55f074f8d9f0 2024-03-21T11:32:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.956+0000 7effca600640 1 --2- 172.21.15.47:0/1807708334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8f580 0x55f074f91c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.956+0000 7effc9c00640 1 --2- 172.21.15.47:0/1807708334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8d9f0 0x55f074f8c750 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x55f074f89d50 tx=0x7effb4009470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59e610b2370d8de9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.956+0000 7effba000640 1 -- 172.21.15.47:0/1807708334 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb402f070 con 0x55f074f8d9f0 2024-03-21T11:32:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.956+0000 7effba000640 1 -- 172.21.15.47:0/1807708334 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effb4005c50 con 0x55f074f8d9f0 2024-03-21T11:32:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.956+0000 7effba000640 1 -- 172.21.15.47:0/1807708334 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4029070 con 0x55f074f8d9f0 2024-03-21T11:32:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.957+0000 7effcf26a740 1 -- 172.21.15.47:0/1807708334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8d9f0 msgr2=0x55f074f8c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.957+0000 7effcf26a740 1 --2- 172.21.15.47:0/1807708334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8d9f0 0x55f074f8c750 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x55f074f89d50 tx=0x7effb4009470 comp rx=0 tx=0).stop 2024-03-21T11:32:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.957+0000 7effcf26a740 1 -- 172.21.15.47:0/1807708334 shutdown_connections 2024-03-21T11:32:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.957+0000 7effcf26a740 1 --2- 172.21.15.47:0/1807708334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8f580 0x55f074f91c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.957+0000 7effcf26a740 1 --2- 172.21.15.47:0/1807708334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 0x55f074f9f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.957+0000 7effcf26a740 1 --2- 172.21.15.47:0/1807708334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8d9f0 0x55f074f8c750 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.957+0000 7effcf26a740 1 -- 172.21.15.47:0/1807708334 >> 172.21.15.47:0/1807708334 conn(0x55f074f88de0 msgr2=0x55f074f8c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.957+0000 7effcf26a740 1 -- 172.21.15.47:0/1807708334 shutdown_connections 2024-03-21T11:32:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.957+0000 7effcf26a740 1 -- 172.21.15.47:0/1807708334 wait complete. 2024-03-21T11:32:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.958+0000 7effcf26a740 1 -- start start 2024-03-21T11:32:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.958+0000 7effcf26a740 1 Processor -- start 2024-03-21T11:32:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.958+0000 7effcf26a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 0x55f074f9afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.958+0000 7effcf26a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 0x55f074f9b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.958+0000 7effcf26a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8f580 0x55f074f9e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.958+0000 7effcf26a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc41b18d0 con 0x55f074f8d9f0 2024-03-21T11:32:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.958+0000 7effcf26a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc41b1270 con 0x55f074f8ebc0 2024-03-21T11:32:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.958+0000 7effcf26a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc41ae770 con 0x55f074f8f580 2024-03-21T11:32:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.958+0000 7effcf26a740 10 client.0 _ll_register_callbacks cb 0x7effbc098410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.959+0000 7effc9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 0x55f074f9b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.959+0000 7effc9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 0x55f074f9b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32802/0 (socket says 172.21.15.47:32802) 2024-03-21T11:32:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.959+0000 7effc9200640 1 -- 172.21.15.47:0/2051988210 learned_addr learned my addr 172.21.15.47:0/2051988210 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.959+0000 7effc9c00640 1 --2- 172.21.15.47:0/2051988210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 0x55f074f9afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.959+0000 7effca600640 1 --2- 172.21.15.47:0/2051988210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8f580 0x55f074f9e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.959+0000 7effc9200640 1 -- 172.21.15.47:0/2051988210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8f580 msgr2=0x55f074f9e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.959+0000 7effc9200640 1 --2- 172.21.15.47:0/2051988210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8f580 0x55f074f9e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.959+0000 7effc9200640 1 -- 172.21.15.47:0/2051988210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 msgr2=0x55f074f9afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.959+0000 7effc9200640 1 --2- 172.21.15.47:0/2051988210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 0x55f074f9afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.959+0000 7effc9200640 1 -- 172.21.15.47:0/2051988210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f074f9ea40 con 0x55f074f8ebc0 2024-03-21T11:32:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.959+0000 7effca600640 1 --2- 172.21.15.47:0/2051988210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8f580 0x55f074f9e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.960+0000 7effc9200640 1 --2- 172.21.15.47:0/2051988210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 0x55f074f9b4f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7effb002c7e0 tx=0x7effb0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66ec163f8fa29eac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.960+0000 7effaf400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:32:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.960+0000 7effaf400640 1 -- 172.21.15.47:0/2051988210 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0005650 con 0x55f074f8ebc0 2024-03-21T11:32:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.960+0000 7effaf400640 1 -- 172.21.15.47:0/2051988210 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7effb0004e10 con 0x55f074f8ebc0 2024-03-21T11:32:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.960+0000 7effcf26a740 1 -- 172.21.15.47:0/2051988210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effb4005c50 con 0x55f074f8ebc0 2024-03-21T11:32:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.960+0000 7effaf400640 1 -- 172.21.15.47:0/2051988210 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7effb002cd50 con 0x55f074f8ebc0 2024-03-21T11:32:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.960+0000 7effcf26a740 10 client.28742 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.960+0000 7effcf26a740 1 -- 172.21.15.47:0/2051988210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f074fff640 con 0x55f074f8ebc0 2024-03-21T11:32:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.961+0000 7effaf400640 1 -- 172.21.15.47:0/2051988210 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effb4005c50 con 0x55f074f8ebc0 2024-03-21T11:32:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.961+0000 7effaf400640 1 -- 172.21.15.47:0/2051988210 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7effb0056020 con 0x55f074f8ebc0 2024-03-21T11:32:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.961+0000 7effcf26a740 10 client.28742 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.961+0000 7effcf26a740 10 client.28742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.961+0000 7effcf26a740 20 client.28742 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.961+0000 7effcf26a740 1 --2- 172.21.15.47:0/2051988210 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f074ffff60 0x55f075002380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.961+0000 7effcf26a740 4 client.28742 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:32:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.961+0000 7effcf26a740 1 -- 172.21.15.47:0/2051988210 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f074ef9b50 con 0x55f074ffff60 2024-03-21T11:32:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.961+0000 7effc9c00640 1 --2- 172.21.15.47:0/2051988210 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f074ffff60 0x55f075002380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.962+0000 7effc9c00640 1 --2- 172.21.15.47:0/2051988210 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f074ffff60 0x55f075002380 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbe0e615a4c33664 server_cookie=d839f5d6299141ca in_seq=0 out_seq=0 2024-03-21T11:32:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.962+0000 7effaf400640 10 client.28742 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.962+0000 7effaf400640 1 -- 172.21.15.47:0/2051988210 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f074ef9b50 con 0x55f074ffff60 2024-03-21T11:32:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.962+0000 7effaf400640 10 client.28742 handle_command_reply: tid=0 2024-03-21T11:32:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.962+0000 7effcf26a740 1 client.28742 shutdown 2024-03-21T11:32:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.963+0000 7effcf26a740 1 -- 172.21.15.47:0/2051988210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 msgr2=0x55f074f9b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.963+0000 7effcf26a740 1 --2- 172.21.15.47:0/2051988210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 0x55f074f9b4f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7effb002c7e0 tx=0x7effb0005fd0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.963+0000 7effcf26a740 1 -- 172.21.15.47:0/2051988210 shutdown_connections 2024-03-21T11:32:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.963+0000 7effcf26a740 1 --2- 172.21.15.47:0/2051988210 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f074ffff60 0x55f075002380 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.963+0000 7effcf26a740 1 --2- 172.21.15.47:0/2051988210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f074f8f580 0x55f074f9e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.963+0000 7effcf26a740 1 --2- 172.21.15.47:0/2051988210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f074f8ebc0 0x55f074f9b4f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.963+0000 7effcf26a740 1 --2- 172.21.15.47:0/2051988210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f074f8d9f0 0x55f074f9afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.963+0000 7effaf400640 0 client.28742 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.963+0000 7effcf26a740 1 -- 172.21.15.47:0/2051988210 >> 172.21.15.47:0/2051988210 conn(0x55f074f88de0 msgr2=0x55f074f8c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.963+0000 7effcf26a740 1 -- 172.21.15.47:0/2051988210 shutdown_connections 2024-03-21T11:32:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.964+0000 7effcf26a740 1 -- 172.21.15.47:0/2051988210 wait complete. 2024-03-21T11:32:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.964+0000 7effcf26a740 20 client.28742 trim_cache size 0 max 16384 2024-03-21T11:32:54.968 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:32:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.965+0000 7effcb000640 1 -- 172.21.15.47:0/2721718461 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7effa80a2fa0 msgr2=0x7effa80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.965+0000 7effcb000640 1 --2- 172.21.15.47:0/2721718461 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7effa80a2fa0 0x7effa80a5460 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7effb002c240 tx=0x7effb002e040 comp rx=0 tx=0).stop 2024-03-21T11:32:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.965+0000 7effcb000640 1 -- 172.21.15.47:0/2721718461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc410db70 msgr2=0x7effc41a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.965+0000 7effcb000640 1 --2- 172.21.15.47:0/2721718461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc410db70 0x7effc41a54a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7effbc009fd0 tx=0x7effbc00ef10 comp rx=0 tx=0).stop 2024-03-21T11:32:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.965+0000 7effcb000640 1 -- 172.21.15.47:0/2721718461 shutdown_connections 2024-03-21T11:32:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.965+0000 7effcb000640 1 --2- 172.21.15.47:0/2721718461 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7effa80a2fa0 0x7effa80a5460 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.965+0000 7effcb000640 1 --2- 172.21.15.47:0/2721718461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc410db70 0x7effc41a54a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.965+0000 7effcb000640 1 --2- 172.21.15.47:0/2721718461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7effc40756e0 0x7effc419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.965+0000 7effcb000640 1 --2- 172.21.15.47:0/2721718461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effc4074100 0x7effc419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.965+0000 7effcb000640 1 -- 172.21.15.47:0/2721718461 >> 172.21.15.47:0/2721718461 conn(0x7effc40fcb10 msgr2=0x7effc4100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.965+0000 7effcb000640 1 -- 172.21.15.47:0/2721718461 shutdown_connections 2024-03-21T11:32:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:54.965+0000 7effcb000640 1 -- 172.21.15.47:0/2721718461 wait complete. 2024-03-21T11:32:55.547 INFO:tasks.workunit.client.0.smithi047.stdout: 1 703911 8.13 MB/sec execute 339 sec latency 13.003 ms 2024-03-21T11:32:55.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:55 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:32:55.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:55 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:32:55.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:55 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:32:55.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:55 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:32:55.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:55 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:32:55.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:55 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:32:55.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:55 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:32:55.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:55 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:32:55.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:55 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:32:55.996 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:32:55.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:32:55.997 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:32:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.255+0000 7f5b29c00640 1 -- 172.21.15.47:0/2739572230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b241005e0 msgr2=0x7f5b24100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.255+0000 7f5b29c00640 1 --2- 172.21.15.47:0/2739572230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b241005e0 0x7f5b24100a00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009900 tx=0x7f5b1402bc50 comp rx=0 tx=0).stop 2024-03-21T11:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.255+0000 7f5b29c00640 1 -- 172.21.15.47:0/2739572230 shutdown_connections 2024-03-21T11:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.255+0000 7f5b29c00640 1 --2- 172.21.15.47:0/2739572230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b24102190 0x7f5b24108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.255+0000 7f5b29c00640 1 --2- 172.21.15.47:0/2739572230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b241017d0 0x7f5b24101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.255+0000 7f5b29c00640 1 --2- 172.21.15.47:0/2739572230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b241005e0 0x7f5b24100a00 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.255+0000 7f5b29c00640 1 -- 172.21.15.47:0/2739572230 >> 172.21.15.47:0/2739572230 conn(0x7f5b240fbb50 msgr2=0x7f5b240fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.255+0000 7f5b29c00640 1 -- 172.21.15.47:0/2739572230 shutdown_connections 2024-03-21T11:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.255+0000 7f5b29c00640 1 -- 172.21.15.47:0/2739572230 wait complete. 2024-03-21T11:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.256+0000 7f5b29c00640 1 Processor -- start 2024-03-21T11:32:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.256+0000 7f5b29c00640 1 -- start start 2024-03-21T11:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.257+0000 7f5b29c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b241005e0 0x7f5b24104570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.257+0000 7f5b29c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b241017d0 0x7f5b24102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.257+0000 7f5b29c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b24102190 0x7f5b241030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.257+0000 7f5b29c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2410c230 con 0x7f5b24102190 2024-03-21T11:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.257+0000 7f5b29c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2410c0b0 con 0x7f5b241005e0 2024-03-21T11:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.257+0000 7f5b29c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2410c3b0 con 0x7f5b241017d0 2024-03-21T11:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.257+0000 7f5b1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b241017d0 0x7f5b24102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.257+0000 7f5b22a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b241005e0 0x7f5b24104570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.257+0000 7f5b1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b241017d0 0x7f5b24102bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50794/0 (socket says 172.21.15.47:50794) 2024-03-21T11:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.257+0000 7f5b1be00640 1 -- 172.21.15.47:0/31149326 learned_addr learned my addr 172.21.15.47:0/31149326 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.257+0000 7f5b23400640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b24102190 0x7f5b241030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.258+0000 7f5b1be00640 1 -- 172.21.15.47:0/31149326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b241005e0 msgr2=0x7f5b24104570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.258+0000 7f5b1be00640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b241005e0 0x7f5b24104570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.258+0000 7f5b1be00640 1 -- 172.21.15.47:0/31149326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b24102190 msgr2=0x7f5b241030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.258+0000 7f5b1be00640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b24102190 0x7f5b241030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.258+0000 7f5b1be00640 1 -- 172.21.15.47:0/31149326 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b24103980 con 0x7f5b241017d0 2024-03-21T11:32:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.258+0000 7f5b23400640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b24102190 0x7f5b241030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.258+0000 7f5b22a00640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b241005e0 0x7f5b24104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.258+0000 7f5b1be00640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b241017d0 0x7f5b24102bb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00ea60 tx=0x7f5b0c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.258+0000 7f5b1b400640 1 -- 172.21.15.47:0/31149326 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00cde0 con 0x7f5b241017d0 2024-03-21T11:32:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.258+0000 7f5b1b400640 1 -- 172.21.15.47:0/31149326 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b0c0035f0 con 0x7f5b241017d0 2024-03-21T11:32:56.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.258+0000 7f5b1b400640 1 -- 172.21.15.47:0/31149326 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c010670 con 0x7f5b241017d0 2024-03-21T11:32:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.258+0000 7f5b29c00640 1 -- 172.21.15.47:0/31149326 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b241a1c80 con 0x7f5b241017d0 2024-03-21T11:32:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.259+0000 7f5b29c00640 1 -- 172.21.15.47:0/31149326 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b241a2180 con 0x7f5b241017d0 2024-03-21T11:32:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.262+0000 7f5b1b400640 1 -- 172.21.15.47:0/31149326 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5b0c0037a0 con 0x7f5b241017d0 2024-03-21T11:32:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.262+0000 7f5b2c986740 1 Processor -- start 2024-03-21T11:32:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.262+0000 7f5b2c986740 1 -- start start 2024-03-21T11:32:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.263+0000 7f5b2c986740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa9880a0 0x5576fa9884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.263+0000 7f5b2c986740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa988a90 0x5576fa98d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.263+0000 7f5b2c986740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa98dc00 0x5576fa990090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.263+0000 7f5b2c986740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576fa8edeb0 con 0x5576fa9880a0 2024-03-21T11:32:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.263+0000 7f5b2c986740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576fa896a60 con 0x5576fa988a90 2024-03-21T11:32:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.263+0000 7f5b2c986740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576fa8e9f50 con 0x5576fa98dc00 2024-03-21T11:32:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.263+0000 7f5b22a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa988a90 0x5576fa98d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.263+0000 7f5b22a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa988a90 0x5576fa98d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32838/0 (socket says 172.21.15.47:32838) 2024-03-21T11:32:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.263+0000 7f5b23400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa9880a0 0x5576fa9884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.263+0000 7f5b22a00640 1 -- 172.21.15.47:0/1115264700 learned_addr learned my addr 172.21.15.47:0/1115264700 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.263+0000 7f5b1be00640 1 --2- 172.21.15.47:0/1115264700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa98dc00 0x5576fa990090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.263+0000 7f5b22a00640 1 -- 172.21.15.47:0/1115264700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa98dc00 msgr2=0x5576fa990090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.263+0000 7f5b22a00640 1 --2- 172.21.15.47:0/1115264700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa98dc00 0x5576fa990090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.264+0000 7f5b22a00640 1 -- 172.21.15.47:0/1115264700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa9880a0 msgr2=0x5576fa9884c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.264+0000 7f5b22a00640 1 --2- 172.21.15.47:0/1115264700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa9880a0 0x5576fa9884c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.264+0000 7f5b22a00640 1 -- 172.21.15.47:0/1115264700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576fa6ca7e0 con 0x5576fa988a90 2024-03-21T11:32:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.264+0000 7f5b1be00640 1 --2- 172.21.15.47:0/1115264700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa98dc00 0x5576fa990090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:32:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.264+0000 7f5b22a00640 1 --2- 172.21.15.47:0/1115264700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa988a90 0x5576fa98d590 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009230 tx=0x7f5b14009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58ac2ef03dcc0493 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.264+0000 7f5b1b400640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5b080a2f20 0x7f5b080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.264+0000 7f5b13e00640 1 -- 172.21.15.47:0/1115264700 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14004230 con 0x5576fa988a90 2024-03-21T11:32:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.264+0000 7f5b1b400640 1 -- 172.21.15.47:0/31149326 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5b0c0cfba0 con 0x7f5b241017d0 2024-03-21T11:32:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.264+0000 7f5b13e00640 1 -- 172.21.15.47:0/1115264700 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b140043d0 con 0x5576fa988a90 2024-03-21T11:32:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.264+0000 7f5b13e00640 1 -- 172.21.15.47:0/1115264700 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b140053f0 con 0x5576fa988a90 2024-03-21T11:32:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.264+0000 7f5b23400640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5b080a2f20 0x7f5b080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.265+0000 7f5b2c986740 1 -- 172.21.15.47:0/1115264700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa988a90 msgr2=0x5576fa98d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.265+0000 7f5b23400640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5b080a2f20 0x7f5b080a53e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x5576fa9892f0 tx=0x7f5b1c0098a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.265+0000 7f5b2c986740 1 --2- 172.21.15.47:0/1115264700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa988a90 0x5576fa98d590 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009230 tx=0x7f5b14009630 comp rx=0 tx=0).stop 2024-03-21T11:32:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.266+0000 7f5b2c986740 1 -- 172.21.15.47:0/1115264700 shutdown_connections 2024-03-21T11:32:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.266+0000 7f5b2c986740 1 --2- 172.21.15.47:0/1115264700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa98dc00 0x5576fa990090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.266+0000 7f5b2c986740 1 --2- 172.21.15.47:0/1115264700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa988a90 0x5576fa98d590 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.266+0000 7f5b2c986740 1 --2- 172.21.15.47:0/1115264700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa9880a0 0x5576fa9884c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.266+0000 7f5b2c986740 1 -- 172.21.15.47:0/1115264700 >> 172.21.15.47:0/1115264700 conn(0x5576fa983610 msgr2=0x5576fa983a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.266+0000 7f5b2c986740 1 -- 172.21.15.47:0/1115264700 shutdown_connections 2024-03-21T11:32:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.266+0000 7f5b2c986740 1 -- 172.21.15.47:0/1115264700 wait complete. 2024-03-21T11:32:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.266+0000 7f5b2c986740 1 -- start start 2024-03-21T11:32:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.266+0000 7f5b2c986740 1 Processor -- start 2024-03-21T11:32:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.267+0000 7f5b2c986740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa9880a0 0x5576fa9921f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.267+0000 7f5b2c986740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa988a90 0x5576fa992730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.267+0000 7f5b2c986740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa98dc00 0x5576fa999770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.267+0000 7f5b2c986740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576fa990b50 con 0x5576fa98dc00 2024-03-21T11:32:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.267+0000 7f5b2c986740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576fa8a9eb0 con 0x5576fa988a90 2024-03-21T11:32:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.267+0000 7f5b2c986740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576fa990cd0 con 0x5576fa9880a0 2024-03-21T11:32:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.267+0000 7f5b2c986740 10 client.0 _ll_register_callbacks cb 0x5576fa89e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.267+0000 7f5b1be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa988a90 0x5576fa992730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.267+0000 7f5b22a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa9880a0 0x5576fa9921f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.267+0000 7f5b1be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa988a90 0x5576fa992730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32840/0 (socket says 172.21.15.47:32840) 2024-03-21T11:32:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.267+0000 7f5b22a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa9880a0 0x5576fa9921f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50810/0 (socket says 172.21.15.47:50810) 2024-03-21T11:32:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.267+0000 7f5b1be00640 1 -- 172.21.15.47:0/950508787 learned_addr learned my addr 172.21.15.47:0/950508787 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.267+0000 7f5b23400640 1 --2- 172.21.15.47:0/950508787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa98dc00 0x5576fa999770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.268+0000 7f5b1be00640 1 -- 172.21.15.47:0/950508787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa9880a0 msgr2=0x5576fa9921f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.268+0000 7f5b1be00640 1 --2- 172.21.15.47:0/950508787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa9880a0 0x5576fa9921f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.268+0000 7f5b1be00640 1 -- 172.21.15.47:0/950508787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa98dc00 msgr2=0x5576fa999770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.268+0000 7f5b1be00640 1 --2- 172.21.15.47:0/950508787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa98dc00 0x5576fa999770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.268+0000 7f5b1be00640 1 -- 172.21.15.47:0/950508787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576fa72af70 con 0x5576fa988a90 2024-03-21T11:32:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.268+0000 7f5b22a00640 1 --2- 172.21.15.47:0/950508787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa9880a0 0x5576fa9921f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.268+0000 7f5b1be00640 1 --2- 172.21.15.47:0/950508787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa988a90 0x5576fa992730 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c09bbe0 tx=0x7f5b0c09bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4324fb3f5546bfd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.268+0000 7f5b10c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:32:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.268+0000 7f5b10c00640 1 -- 172.21.15.47:0/950508787 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c09bee0 con 0x5576fa988a90 2024-03-21T11:32:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.268+0000 7f5b2c986740 1 -- 172.21.15.47:0/950508787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b140043d0 con 0x5576fa988a90 2024-03-21T11:32:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.268+0000 7f5b10c00640 1 -- 172.21.15.47:0/950508787 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5b0c003b30 con 0x5576fa988a90 2024-03-21T11:32:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.268+0000 7f5b10c00640 1 -- 172.21.15.47:0/950508787 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b0c0a0030 con 0x5576fa988a90 2024-03-21T11:32:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.268+0000 7f5b2c986740 10 client.28754 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.268+0000 7f5b2c986740 1 -- 172.21.15.47:0/950508787 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5576fa5c2170 con 0x5576fa988a90 2024-03-21T11:32:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.269+0000 7f5b10c00640 1 -- 172.21.15.47:0/950508787 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b140043d0 con 0x5576fa988a90 2024-03-21T11:32:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.269+0000 7f5b10c00640 1 -- 172.21.15.47:0/950508787 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5b0c097c30 con 0x5576fa988a90 2024-03-21T11:32:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.269+0000 7f5b2c986740 10 client.28754 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.269+0000 7f5b2c986740 10 client.28754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.269+0000 7f5b2c986740 20 client.28754 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.269+0000 7f5b2c986740 1 --2- 172.21.15.47:0/950508787 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576fa99e040 0x5576fa9a0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.269+0000 7f5b2c986740 4 client.28754 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.269+0000 7f5b2c986740 1 -- 172.21.15.47:0/950508787 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576fa8fb3f0 con 0x5576fa99e040 2024-03-21T11:32:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.269+0000 7f5b22a00640 1 --2- 172.21.15.47:0/950508787 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576fa99e040 0x5576fa9a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.270+0000 7f5b22a00640 1 --2- 172.21.15.47:0/950508787 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576fa99e040 0x5576fa9a0470 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d99898547c12b59d server_cookie=2d9b204e105ff2a0 in_seq=0 out_seq=0 2024-03-21T11:32:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.270+0000 7f5b10c00640 10 client.28754 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.271+0000 7f5b10c00640 1 -- 172.21.15.47:0/950508787 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5576fa8fb3f0 con 0x5576fa99e040 2024-03-21T11:32:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.271+0000 7f5b10c00640 10 client.28754 handle_command_reply: tid=0 2024-03-21T11:32:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.271+0000 7f5b2c986740 1 client.28754 shutdown 2024-03-21T11:32:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.271+0000 7f5b2c986740 1 -- 172.21.15.47:0/950508787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa988a90 msgr2=0x5576fa992730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.271+0000 7f5b2c986740 1 --2- 172.21.15.47:0/950508787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa988a90 0x5576fa992730 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c09bbe0 tx=0x7f5b0c09bda0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.272+0000 7f5b2c986740 1 -- 172.21.15.47:0/950508787 shutdown_connections 2024-03-21T11:32:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.272+0000 7f5b2c986740 1 --2- 172.21.15.47:0/950508787 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576fa99e040 0x5576fa9a0470 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.272+0000 7f5b2c986740 1 --2- 172.21.15.47:0/950508787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa98dc00 0x5576fa999770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.272+0000 7f5b2c986740 1 --2- 172.21.15.47:0/950508787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa988a90 0x5576fa992730 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.272+0000 7f5b2c986740 1 --2- 172.21.15.47:0/950508787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa9880a0 0x5576fa9921f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.272+0000 7f5b10c00640 0 client.28754 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.272+0000 7f5b2c986740 1 -- 172.21.15.47:0/950508787 >> 172.21.15.47:0/950508787 conn(0x5576fa983610 msgr2=0x5576fa982830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.272+0000 7f5b2c986740 1 -- 172.21.15.47:0/950508787 shutdown_connections 2024-03-21T11:32:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.272+0000 7f5b2c986740 1 -- 172.21.15.47:0/950508787 wait complete. 2024-03-21T11:32:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.272+0000 7f5b2c986740 20 client.28754 trim_cache size 0 max 16384 2024-03-21T11:32:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.291+0000 7f5b2c986740 1 Processor -- start 2024-03-21T11:32:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.292+0000 7f5b2c986740 1 -- start start 2024-03-21T11:32:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.292+0000 7f5b2c986740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa9880a0 0x5576fa986ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.292+0000 7f5b2c986740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa988a90 0x5576fa999730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.292+0000 7f5b2c986740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa98dc00 0x5576fa9a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.292+0000 7f5b2c986740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241ac3e0 con 0x5576fa98dc00 2024-03-21T11:32:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.292+0000 7f5b2c986740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241add60 con 0x5576fa9880a0 2024-03-21T11:32:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.292+0000 7f5b2c986740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241aca40 con 0x5576fa988a90 2024-03-21T11:32:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.292+0000 7f5b22a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa9880a0 0x5576fa986ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.292+0000 7f5b1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa988a90 0x5576fa999730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.292+0000 7f5b22a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa9880a0 0x5576fa986ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32844/0 (socket says 172.21.15.47:32844) 2024-03-21T11:32:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.292+0000 7f5b22a00640 1 -- 172.21.15.47:0/4163297924 learned_addr learned my addr 172.21.15.47:0/4163297924 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.293+0000 7f5b23400640 1 --2- 172.21.15.47:0/4163297924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa98dc00 0x5576fa9a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.293+0000 7f5b22a00640 1 -- 172.21.15.47:0/4163297924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa988a90 msgr2=0x5576fa999730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.293+0000 7f5b22a00640 1 --2- 172.21.15.47:0/4163297924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa988a90 0x5576fa999730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.293+0000 7f5b22a00640 1 -- 172.21.15.47:0/4163297924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa98dc00 msgr2=0x5576fa9a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.293+0000 7f5b22a00640 1 --2- 172.21.15.47:0/4163297924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa98dc00 0x5576fa9a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.293+0000 7f5b22a00640 1 -- 172.21.15.47:0/4163297924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576fa97ec20 con 0x5576fa9880a0 2024-03-21T11:32:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.293+0000 7f5b22a00640 1 --2- 172.21.15.47:0/4163297924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa9880a0 0x5576fa986ee0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5b1402c900 tx=0x7f5b14005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92bf70a561fda269 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.293+0000 7f5b13400640 1 -- 172.21.15.47:0/4163297924 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14004f40 con 0x5576fa9880a0 2024-03-21T11:32:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.293+0000 7f5b13400640 1 -- 172.21.15.47:0/4163297924 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b14004340 con 0x5576fa9880a0 2024-03-21T11:32:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.293+0000 7f5b13400640 1 -- 172.21.15.47:0/4163297924 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14004c80 con 0x5576fa9880a0 2024-03-21T11:32:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.294+0000 7f5b2c986740 1 -- 172.21.15.47:0/4163297924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa9880a0 msgr2=0x5576fa986ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.294+0000 7f5b2c986740 1 --2- 172.21.15.47:0/4163297924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa9880a0 0x5576fa986ee0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5b1402c900 tx=0x7f5b14005fd0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.294+0000 7f5b2c986740 1 -- 172.21.15.47:0/4163297924 shutdown_connections 2024-03-21T11:32:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.294+0000 7f5b2c986740 1 --2- 172.21.15.47:0/4163297924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa98dc00 0x5576fa9a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.294+0000 7f5b2c986740 1 --2- 172.21.15.47:0/4163297924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa988a90 0x5576fa999730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.294+0000 7f5b2c986740 1 --2- 172.21.15.47:0/4163297924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa9880a0 0x5576fa986ee0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.294+0000 7f5b2c986740 1 -- 172.21.15.47:0/4163297924 >> 172.21.15.47:0/4163297924 conn(0x5576fa983610 msgr2=0x5576fa982830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.294+0000 7f5b2c986740 1 -- 172.21.15.47:0/4163297924 shutdown_connections 2024-03-21T11:32:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.294+0000 7f5b2c986740 1 -- 172.21.15.47:0/4163297924 wait complete. 2024-03-21T11:32:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.295+0000 7f5b2c986740 1 -- start start 2024-03-21T11:32:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.295+0000 7f5b2c986740 1 Processor -- start 2024-03-21T11:32:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.295+0000 7f5b2c986740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa9880a0 0x5576fa999460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.295+0000 7f5b2c986740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa988a90 0x5576fa997a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b2c986740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa98dc00 0x5576fa997fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b2c986740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241ad700 con 0x5576fa988a90 2024-03-21T11:32:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b2c986740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241ad0a0 con 0x5576fa98dc00 2024-03-21T11:32:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b2c986740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241aa5a0 con 0x5576fa9880a0 2024-03-21T11:32:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b2c986740 10 client.0 _ll_register_callbacks cb 0x7f5b0c003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b22a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa9880a0 0x5576fa999460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b22a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa9880a0 0x5576fa999460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50828/0 (socket says 172.21.15.47:50828) 2024-03-21T11:32:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b22a00640 1 -- 172.21.15.47:0/409458757 learned_addr learned my addr 172.21.15.47:0/409458757 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b1be00640 1 --2- 172.21.15.47:0/409458757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa988a90 0x5576fa997a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b23400640 1 --2- 172.21.15.47:0/409458757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa98dc00 0x5576fa997fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b22a00640 1 -- 172.21.15.47:0/409458757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa98dc00 msgr2=0x5576fa997fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b22a00640 1 --2- 172.21.15.47:0/409458757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa98dc00 0x5576fa997fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b22a00640 1 -- 172.21.15.47:0/409458757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa988a90 msgr2=0x5576fa997a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b22a00640 1 --2- 172.21.15.47:0/409458757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa988a90 0x5576fa997a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.296+0000 7f5b22a00640 1 -- 172.21.15.47:0/409458757 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576fa998620 con 0x5576fa9880a0 2024-03-21T11:32:56.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.297+0000 7f5b22a00640 1 --2- 172.21.15.47:0/409458757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa9880a0 0x5576fa999460 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5b1402c7d0 tx=0x7f5b14004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab97aa74b613a060 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.297+0000 7f5b10c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.297+0000 7f5b10c00640 1 -- 172.21.15.47:0/409458757 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b140550d0 con 0x5576fa9880a0 2024-03-21T11:32:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.297+0000 7f5b10c00640 1 -- 172.21.15.47:0/409458757 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5b14055270 con 0x5576fa9880a0 2024-03-21T11:32:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.297+0000 7f5b2c986740 1 -- 172.21.15.47:0/409458757 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b14004340 con 0x5576fa9880a0 2024-03-21T11:32:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.297+0000 7f5b10c00640 1 -- 172.21.15.47:0/409458757 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b1406b050 con 0x5576fa9880a0 2024-03-21T11:32:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.297+0000 7f5b2c986740 10 client.28777 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.297+0000 7f5b2c986740 1 -- 172.21.15.47:0/409458757 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5576fa9fa7d0 con 0x5576fa9880a0 2024-03-21T11:32:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.298+0000 7f5b10c00640 1 -- 172.21.15.47:0/409458757 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b14004340 con 0x5576fa9880a0 2024-03-21T11:32:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.298+0000 7f5b10c00640 1 -- 172.21.15.47:0/409458757 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5b14051040 con 0x5576fa9880a0 2024-03-21T11:32:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.298+0000 7f5b2c986740 10 client.28777 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.298+0000 7f5b2c986740 10 client.28777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.298+0000 7f5b2c986740 20 client.28777 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.298+0000 7f5b2c986740 1 --2- 172.21.15.47:0/409458757 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576fa9facc0 0x5576fa9fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.298+0000 7f5b2c986740 4 client.28777 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:32:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.298+0000 7f5b2c986740 1 -- 172.21.15.47:0/409458757 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5576fa8f5740 con 0x5576fa9facc0 2024-03-21T11:32:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.298+0000 7f5b1be00640 1 --2- 172.21.15.47:0/409458757 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576fa9facc0 0x5576fa9fd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.299+0000 7f5b1be00640 1 --2- 172.21.15.47:0/409458757 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576fa9facc0 0x5576fa9fd0b0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3fc46557eecca9d server_cookie=3d22408eb2f09858 in_seq=0 out_seq=0 2024-03-21T11:32:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.299+0000 7f5b10c00640 10 client.28777 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:56.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.300+0000 7f5b10c00640 1 -- 172.21.15.47:0/409458757 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5576fa8f5740 con 0x5576fa9facc0 2024-03-21T11:32:56.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.300+0000 7f5b10c00640 10 client.28777 handle_command_reply: tid=0 2024-03-21T11:32:56.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.300+0000 7f5b2c986740 1 client.28777 shutdown 2024-03-21T11:32:56.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.301+0000 7f5b2c986740 1 -- 172.21.15.47:0/409458757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa9880a0 msgr2=0x5576fa999460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.301+0000 7f5b2c986740 1 --2- 172.21.15.47:0/409458757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa9880a0 0x5576fa999460 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5b1402c7d0 tx=0x7f5b14004740 comp rx=0 tx=0).stop 2024-03-21T11:32:56.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.301+0000 7f5b2c986740 1 -- 172.21.15.47:0/409458757 shutdown_connections 2024-03-21T11:32:56.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.301+0000 7f5b2c986740 1 --2- 172.21.15.47:0/409458757 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5576fa9facc0 0x5576fa9fd0b0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.301+0000 7f5b2c986740 1 --2- 172.21.15.47:0/409458757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576fa98dc00 0x5576fa997fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.301+0000 7f5b2c986740 1 --2- 172.21.15.47:0/409458757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576fa988a90 0x5576fa997a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.301+0000 7f5b2c986740 1 --2- 172.21.15.47:0/409458757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576fa9880a0 0x5576fa999460 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.301+0000 7f5b10c00640 0 client.28777 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.301+0000 7f5b2c986740 1 -- 172.21.15.47:0/409458757 >> 172.21.15.47:0/409458757 conn(0x5576fa983610 msgr2=0x5576fa989320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.301+0000 7f5b2c986740 1 -- 172.21.15.47:0/409458757 shutdown_connections 2024-03-21T11:32:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.301+0000 7f5b2c986740 1 -- 172.21.15.47:0/409458757 wait complete. 2024-03-21T11:32:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.301+0000 7f5b2c986740 20 client.28777 trim_cache size 0 max 16384 2024-03-21T11:32:56.308 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:32:56.308 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:32:56.308 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "f1708e33-c09b-4a29-be8f-e098a641c48e", 2024-03-21T11:32:56.308 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:32:56.309 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:32:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.302+0000 7f5b29c00640 1 -- 172.21.15.47:0/31149326 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5b080a2f20 msgr2=0x7f5b080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.302+0000 7f5b29c00640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5b080a2f20 0x7f5b080a53e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x5576fa9892f0 tx=0x7f5b1c0098a0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.302+0000 7f5b29c00640 1 -- 172.21.15.47:0/31149326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b241017d0 msgr2=0x7f5b24102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.302+0000 7f5b29c00640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b241017d0 0x7f5b24102bb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00ea60 tx=0x7f5b0c00ef20 comp rx=0 tx=0).stop 2024-03-21T11:32:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.302+0000 7f5b29c00640 1 -- 172.21.15.47:0/31149326 shutdown_connections 2024-03-21T11:32:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.302+0000 7f5b29c00640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5b080a2f20 0x7f5b080a53e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.302+0000 7f5b29c00640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b24102190 0x7f5b241030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.302+0000 7f5b29c00640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b241017d0 0x7f5b24102bb0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.302+0000 7f5b29c00640 1 --2- 172.21.15.47:0/31149326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b241005e0 0x7f5b24104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.302+0000 7f5b29c00640 1 -- 172.21.15.47:0/31149326 >> 172.21.15.47:0/31149326 conn(0x7f5b240fbb50 msgr2=0x7f5b24069a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.302+0000 7f5b29c00640 1 -- 172.21.15.47:0/31149326 shutdown_connections 2024-03-21T11:32:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.302+0000 7f5b29c00640 1 -- 172.21.15.47:0/31149326 wait complete. 2024-03-21T11:32:56.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:32:56.326 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:32:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.545+0000 7f5432000640 1 -- 172.21.15.47:0/1951997925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542c0740a0 msgr2=0x7f542c074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.545+0000 7f5432000640 1 --2- 172.21.15.47:0/1951997925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542c0740a0 0x7f542c074500 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5418009900 tx=0x7f541802bcc0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.545+0000 7f5432000640 1 -- 172.21.15.47:0/1951997925 shutdown_connections 2024-03-21T11:32:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.545+0000 7f5432000640 1 --2- 172.21.15.47:0/1951997925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f542c109430 0x7f542c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.545+0000 7f5432000640 1 --2- 172.21.15.47:0/1951997925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542c0740a0 0x7f542c074500 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.545+0000 7f5432000640 1 --2- 172.21.15.47:0/1951997925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542c075680 0x7f542c073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.545+0000 7f5432000640 1 -- 172.21.15.47:0/1951997925 >> 172.21.15.47:0/1951997925 conn(0x7f542c0fcb10 msgr2=0x7f542c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:56.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.545+0000 7f5432000640 1 -- 172.21.15.47:0/1951997925 shutdown_connections 2024-03-21T11:32:56.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.545+0000 7f5432000640 1 -- 172.21.15.47:0/1951997925 wait complete. 2024-03-21T11:32:56.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.546+0000 7f5432000640 1 Processor -- start 2024-03-21T11:32:56.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.546+0000 7f5432000640 1 -- start start 2024-03-21T11:32:56.551 INFO:tasks.workunit.client.0.smithi047.stdout: 1 705713 8.13 MB/sec execute 340 sec latency 16.040 ms 2024-03-21T11:32:56.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.547+0000 7f5432000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542c0740a0 0x7f542c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.547+0000 7f5432000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f542c075680 0x7f542c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.547+0000 7f5432000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542c109430 0x7f542c1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.547+0000 7f5432000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c10f470 con 0x7f542c0740a0 2024-03-21T11:32:56.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.547+0000 7f5432000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c10f2f0 con 0x7f542c109430 2024-03-21T11:32:56.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.547+0000 7f5432000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c10f5f0 con 0x7f542c075680 2024-03-21T11:32:56.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.547+0000 7f5430c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542c0740a0 0x7f542c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.547+0000 7f542be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f542c075680 0x7f542c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.547+0000 7f5431600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542c109430 0x7f542c1a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.547+0000 7f5430c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542c0740a0 0x7f542c199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33606/0 (socket says 172.21.15.47:33606) 2024-03-21T11:32:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.547+0000 7f542be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f542c075680 0x7f542c199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50848/0 (socket says 172.21.15.47:50848) 2024-03-21T11:32:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.547+0000 7f5430c00640 1 -- 172.21.15.47:0/3194614034 learned_addr learned my addr 172.21.15.47:0/3194614034 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.548+0000 7f542be00640 1 -- 172.21.15.47:0/3194614034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542c109430 msgr2=0x7f542c1a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.548+0000 7f542be00640 1 --2- 172.21.15.47:0/3194614034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542c109430 0x7f542c1a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.548+0000 7f542be00640 1 -- 172.21.15.47:0/3194614034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542c0740a0 msgr2=0x7f542c199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.548+0000 7f542be00640 1 --2- 172.21.15.47:0/3194614034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542c0740a0 0x7f542c199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.548+0000 7f542be00640 1 -- 172.21.15.47:0/3194614034 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c1a1670 con 0x7f542c075680 2024-03-21T11:32:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.548+0000 7f5430c00640 1 --2- 172.21.15.47:0/3194614034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542c0740a0 0x7f542c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.548+0000 7f5431600640 1 --2- 172.21.15.47:0/3194614034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542c109430 0x7f542c1a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.548+0000 7f542be00640 1 --2- 172.21.15.47:0/3194614034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f542c075680 0x7f542c199f90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5418036040 tx=0x7f541802c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.548+0000 7f5429600640 1 -- 172.21.15.47:0/3194614034 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418043020 con 0x7f542c075680 2024-03-21T11:32:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.548+0000 7f5429600640 1 -- 172.21.15.47:0/3194614034 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54180054e0 con 0x7f542c075680 2024-03-21T11:32:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.548+0000 7f5432000640 1 -- 172.21.15.47:0/3194614034 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c1a1900 con 0x7f542c075680 2024-03-21T11:32:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.548+0000 7f5429600640 1 -- 172.21.15.47:0/3194614034 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418003b80 con 0x7f542c075680 2024-03-21T11:32:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.549+0000 7f5432000640 1 -- 172.21.15.47:0/3194614034 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c1a1e20 con 0x7f542c075680 2024-03-21T11:32:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.553+0000 7f54363ed740 1 Processor -- start 2024-03-21T11:32:56.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.553+0000 7f54363ed740 1 -- start start 2024-03-21T11:32:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.553+0000 7f5429600640 1 -- 172.21.15.47:0/3194614034 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5418046020 con 0x7f542c075680 2024-03-21T11:32:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.553+0000 7f54363ed740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85d050 0x5583bd85d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.553+0000 7f54363ed740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd85da40 0x5583bd862540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.553+0000 7f54363ed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd862bb0 0x5583bd865040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.553+0000 7f54363ed740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583bd7c2250 con 0x5583bd85d050 2024-03-21T11:32:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.553+0000 7f54363ed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583bd76aa30 con 0x5583bd862bb0 2024-03-21T11:32:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.553+0000 7f54363ed740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583bd7be2f0 con 0x5583bd85da40 2024-03-21T11:32:56.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.554+0000 7f5430c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd85da40 0x5583bd862540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.554+0000 7f542be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd862bb0 0x5583bd865040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.554+0000 7f5430c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd85da40 0x5583bd862540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50862/0 (socket says 172.21.15.47:50862) 2024-03-21T11:32:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.554+0000 7f5430c00640 1 -- 172.21.15.47:0/1156177301 learned_addr learned my addr 172.21.15.47:0/1156177301 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.554+0000 7f5431600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85d050 0x5583bd85d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.554+0000 7f5430c00640 1 -- 172.21.15.47:0/1156177301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd862bb0 msgr2=0x5583bd865040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.554+0000 7f5430c00640 1 --2- 172.21.15.47:0/1156177301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd862bb0 0x5583bd865040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.554+0000 7f5430c00640 1 -- 172.21.15.47:0/1156177301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85d050 msgr2=0x5583bd85d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.554+0000 7f5430c00640 1 --2- 172.21.15.47:0/1156177301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85d050 0x5583bd85d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.554+0000 7f5430c00640 1 -- 172.21.15.47:0/1156177301 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583bd755690 con 0x5583bd85da40 2024-03-21T11:32:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.554+0000 7f5430c00640 1 --2- 172.21.15.47:0/1156177301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd85da40 0x5583bd862540 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5420011650 tx=0x7f5420009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9f487a4421be385 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.555+0000 7f541dc00640 1 -- 172.21.15.47:0/1156177301 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542001e070 con 0x5583bd85da40 2024-03-21T11:32:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.555+0000 7f541dc00640 1 -- 172.21.15.47:0/1156177301 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f542000fba0 con 0x5583bd85da40 2024-03-21T11:32:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.555+0000 7f541dc00640 1 -- 172.21.15.47:0/1156177301 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420019cd0 con 0x5583bd85da40 2024-03-21T11:32:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.555+0000 7f54363ed740 1 -- 172.21.15.47:0/1156177301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd85da40 msgr2=0x5583bd862540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.555+0000 7f54363ed740 1 --2- 172.21.15.47:0/1156177301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd85da40 0x5583bd862540 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5420011650 tx=0x7f5420009f90 comp rx=0 tx=0).stop 2024-03-21T11:32:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.555+0000 7f5429600640 1 --2- 172.21.15.47:0/3194614034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54140a2f20 0x7f54140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.556+0000 7f5429600640 1 -- 172.21.15.47:0/3194614034 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f54180f1760 con 0x7f542c075680 2024-03-21T11:32:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.556+0000 7f54363ed740 1 -- 172.21.15.47:0/1156177301 shutdown_connections 2024-03-21T11:32:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.556+0000 7f54363ed740 1 --2- 172.21.15.47:0/1156177301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd862bb0 0x5583bd865040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.556+0000 7f54363ed740 1 --2- 172.21.15.47:0/1156177301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd85da40 0x5583bd862540 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.556+0000 7f54363ed740 1 --2- 172.21.15.47:0/1156177301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85d050 0x5583bd85d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.556+0000 7f54363ed740 1 -- 172.21.15.47:0/1156177301 >> 172.21.15.47:0/1156177301 conn(0x5583bd825200 msgr2=0x5583bd8589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.556+0000 7f5431600640 1 --2- 172.21.15.47:0/3194614034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54140a2f20 0x7f54140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.556+0000 7f54363ed740 1 -- 172.21.15.47:0/1156177301 shutdown_connections 2024-03-21T11:32:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.556+0000 7f54363ed740 1 -- 172.21.15.47:0/1156177301 wait complete. 2024-03-21T11:32:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.557+0000 7f54363ed740 1 -- start start 2024-03-21T11:32:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.557+0000 7f5431600640 1 --2- 172.21.15.47:0/3194614034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54140a2f20 0x7f54140a53e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x5583bd85e2a0 tx=0x7f5424009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.557+0000 7f54363ed740 1 Processor -- start 2024-03-21T11:32:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.557+0000 7f54363ed740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85d050 0x5583bd867050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.557+0000 7f54363ed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85da40 0x5583bd867590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f54363ed740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd862bb0 0x5583bd86e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f54363ed740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583bd865b60 con 0x5583bd85d050 2024-03-21T11:32:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f54363ed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583bd77de80 con 0x5583bd85da40 2024-03-21T11:32:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f54363ed740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583bd865ce0 con 0x5583bd862bb0 2024-03-21T11:32:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f54363ed740 10 client.0 _ll_register_callbacks cb 0x5583bd790130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f542be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85da40 0x5583bd867590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f5430c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85d050 0x5583bd867050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f5431600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd862bb0 0x5583bd86e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f542be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85da40 0x5583bd867590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32878/0 (socket says 172.21.15.47:32878) 2024-03-21T11:32:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f542be00640 1 -- 172.21.15.47:0/3131491596 learned_addr learned my addr 172.21.15.47:0/3131491596 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f542be00640 1 -- 172.21.15.47:0/3131491596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd862bb0 msgr2=0x5583bd86e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f542be00640 1 --2- 172.21.15.47:0/3131491596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd862bb0 0x5583bd86e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f542be00640 1 -- 172.21.15.47:0/3131491596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85d050 msgr2=0x5583bd867050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f542be00640 1 --2- 172.21.15.47:0/3131491596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85d050 0x5583bd867050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f542be00640 1 -- 172.21.15.47:0/3131491596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583bd5a07d0 con 0x5583bd85da40 2024-03-21T11:32:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f5431600640 1 --2- 172.21.15.47:0/3131491596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd862bb0 0x5583bd86e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.558+0000 7f5430c00640 1 --2- 172.21.15.47:0/3131491596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85d050 0x5583bd867050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.559+0000 7f542be00640 1 --2- 172.21.15.47:0/3131491596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85da40 0x5583bd867590 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f54180bda30 tx=0x7f54180b9dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1adbadd62d75e41 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.559+0000 7f5412000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:32:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.559+0000 7f5412000640 1 -- 172.21.15.47:0/3131491596 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418005690 con 0x5583bd85da40 2024-03-21T11:32:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.559+0000 7f5412000640 1 -- 172.21.15.47:0/3131491596 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5418003d20 con 0x5583bd85da40 2024-03-21T11:32:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.559+0000 7f54363ed740 1 -- 172.21.15.47:0/3131491596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f542000fba0 con 0x5583bd85da40 2024-03-21T11:32:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.559+0000 7f5412000640 1 -- 172.21.15.47:0/3131491596 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54180be1a0 con 0x5583bd85da40 2024-03-21T11:32:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.559+0000 7f54363ed740 10 client.28772 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.559+0000 7f54363ed740 1 -- 172.21.15.47:0/3131491596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5583bd4d63f0 con 0x5583bd85da40 2024-03-21T11:32:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.559+0000 7f5412000640 1 -- 172.21.15.47:0/3131491596 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f542000fba0 con 0x5583bd85da40 2024-03-21T11:32:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.560+0000 7f5412000640 1 -- 172.21.15.47:0/3131491596 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f54180bdd90 con 0x5583bd85da40 2024-03-21T11:32:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.560+0000 7f54363ed740 10 client.28772 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.560+0000 7f54363ed740 10 client.28772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.560+0000 7f54363ed740 20 client.28772 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.560+0000 7f54363ed740 1 --2- 172.21.15.47:0/3131491596 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583bd872e00 0x5583bd875250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.560+0000 7f54363ed740 4 client.28772 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.560+0000 7f54363ed740 1 -- 172.21.15.47:0/3131491596 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583bd7cf790 con 0x5583bd872e00 2024-03-21T11:32:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.560+0000 7f5430c00640 1 --2- 172.21.15.47:0/3131491596 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583bd872e00 0x5583bd875250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.561+0000 7f5430c00640 1 --2- 172.21.15.47:0/3131491596 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583bd872e00 0x5583bd875250 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dc3586eb0b9d446 server_cookie=6a8ae3feded31a3e in_seq=0 out_seq=0 2024-03-21T11:32:56.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.561+0000 7f5412000640 10 client.28772 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.562+0000 7f5412000640 1 -- 172.21.15.47:0/3131491596 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5583bd7cf790 con 0x5583bd872e00 2024-03-21T11:32:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.562+0000 7f5412000640 10 client.28772 handle_command_reply: tid=0 2024-03-21T11:32:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.562+0000 7f54363ed740 1 client.28772 shutdown 2024-03-21T11:32:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.562+0000 7f54363ed740 1 -- 172.21.15.47:0/3131491596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85da40 msgr2=0x5583bd867590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.562+0000 7f54363ed740 1 --2- 172.21.15.47:0/3131491596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85da40 0x5583bd867590 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f54180bda30 tx=0x7f54180b9dc0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.562+0000 7f54363ed740 1 -- 172.21.15.47:0/3131491596 shutdown_connections 2024-03-21T11:32:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.562+0000 7f54363ed740 1 --2- 172.21.15.47:0/3131491596 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583bd872e00 0x5583bd875250 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.562+0000 7f54363ed740 1 --2- 172.21.15.47:0/3131491596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd862bb0 0x5583bd86e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.562+0000 7f54363ed740 1 --2- 172.21.15.47:0/3131491596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85da40 0x5583bd867590 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.562+0000 7f5412000640 0 client.28772 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.562+0000 7f54363ed740 1 --2- 172.21.15.47:0/3131491596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85d050 0x5583bd867050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.562+0000 7f54363ed740 1 -- 172.21.15.47:0/3131491596 >> 172.21.15.47:0/3131491596 conn(0x5583bd825200 msgr2=0x5583bd857bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.563+0000 7f54363ed740 1 -- 172.21.15.47:0/3131491596 shutdown_connections 2024-03-21T11:32:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.563+0000 7f54363ed740 1 -- 172.21.15.47:0/3131491596 wait complete. 2024-03-21T11:32:56.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.563+0000 7f54363ed740 20 client.28772 trim_cache size 0 max 16384 2024-03-21T11:32:56.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.582+0000 7f54363ed740 1 Processor -- start 2024-03-21T11:32:56.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.582+0000 7f54363ed740 1 -- start start 2024-03-21T11:32:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.582+0000 7f54363ed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85d050 0x5583bd85ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.582+0000 7f54363ed740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd85da40 0x5583bd86e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.582+0000 7f54363ed740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd862bb0 0x5583bd85bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.582+0000 7f54363ed740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c1ac020 con 0x5583bd862bb0 2024-03-21T11:32:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.582+0000 7f54363ed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c1ad9a0 con 0x5583bd85d050 2024-03-21T11:32:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.582+0000 7f54363ed740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c1ac680 con 0x5583bd85da40 2024-03-21T11:32:56.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.583+0000 7f5430c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85d050 0x5583bd85ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.583+0000 7f542be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd85da40 0x5583bd86e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.583+0000 7f5431600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd862bb0 0x5583bd85bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.583+0000 7f5430c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85d050 0x5583bd85ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32888/0 (socket says 172.21.15.47:32888) 2024-03-21T11:32:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.583+0000 7f5430c00640 1 -- 172.21.15.47:0/2174410928 learned_addr learned my addr 172.21.15.47:0/2174410928 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.583+0000 7f5430c00640 1 -- 172.21.15.47:0/2174410928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd85da40 msgr2=0x5583bd86e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.583+0000 7f5430c00640 1 --2- 172.21.15.47:0/2174410928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd85da40 0x5583bd86e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.583+0000 7f5430c00640 1 -- 172.21.15.47:0/2174410928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd862bb0 msgr2=0x5583bd85bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.583+0000 7f5430c00640 1 --2- 172.21.15.47:0/2174410928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd862bb0 0x5583bd85bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.583+0000 7f5430c00640 1 -- 172.21.15.47:0/2174410928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583bd4960f0 con 0x5583bd85d050 2024-03-21T11:32:56.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.583+0000 7f542be00640 1 --2- 172.21.15.47:0/2174410928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd85da40 0x5583bd86e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:56.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.583+0000 7f5430c00640 1 --2- 172.21.15.47:0/2174410928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85d050 0x5583bd85ebf0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f542001c040 tx=0x7f5420009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=609bee623810b9ab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:56.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.584+0000 7f541d200640 1 -- 172.21.15.47:0/2174410928 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420029070 con 0x5583bd85d050 2024-03-21T11:32:56.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.584+0000 7f541d200640 1 -- 172.21.15.47:0/2174410928 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f542000fb30 con 0x5583bd85d050 2024-03-21T11:32:56.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.584+0000 7f541d200640 1 -- 172.21.15.47:0/2174410928 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542001e040 con 0x5583bd85d050 2024-03-21T11:32:56.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.584+0000 7f54363ed740 1 -- 172.21.15.47:0/2174410928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85d050 msgr2=0x5583bd85ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.584+0000 7f54363ed740 1 --2- 172.21.15.47:0/2174410928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85d050 0x5583bd85ebf0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f542001c040 tx=0x7f5420009530 comp rx=0 tx=0).stop 2024-03-21T11:32:56.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.585+0000 7f54363ed740 1 -- 172.21.15.47:0/2174410928 shutdown_connections 2024-03-21T11:32:56.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.585+0000 7f54363ed740 1 --2- 172.21.15.47:0/2174410928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd862bb0 0x5583bd85bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.585+0000 7f54363ed740 1 --2- 172.21.15.47:0/2174410928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd85da40 0x5583bd86e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.585+0000 7f54363ed740 1 --2- 172.21.15.47:0/2174410928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85d050 0x5583bd85ebf0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.585+0000 7f54363ed740 1 -- 172.21.15.47:0/2174410928 >> 172.21.15.47:0/2174410928 conn(0x5583bd825200 msgr2=0x5583bd85e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:56.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.585+0000 7f54363ed740 1 -- 172.21.15.47:0/2174410928 shutdown_connections 2024-03-21T11:32:56.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.585+0000 7f54363ed740 1 -- 172.21.15.47:0/2174410928 wait complete. 2024-03-21T11:32:56.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.585+0000 7f54363ed740 1 -- start start 2024-03-21T11:32:56.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.585+0000 7f54363ed740 1 Processor -- start 2024-03-21T11:32:56.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.586+0000 7f54363ed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85d050 0x5583bd86b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.586+0000 7f54363ed740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85da40 0x5583bd86bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.586+0000 7f54363ed740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd862bb0 0x5583bd86cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.586+0000 7f54363ed740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c1ad340 con 0x5583bd85da40 2024-03-21T11:32:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.586+0000 7f54363ed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c1acce0 con 0x5583bd85d050 2024-03-21T11:32:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.586+0000 7f54363ed740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c1aa1e0 con 0x5583bd862bb0 2024-03-21T11:32:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.586+0000 7f54363ed740 10 client.0 _ll_register_callbacks cb 0x7f5418009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.586+0000 7f5431600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd862bb0 0x5583bd86cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.586+0000 7f542be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85da40 0x5583bd86bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.586+0000 7f5430c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85d050 0x5583bd86b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.586+0000 7f542be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85da40 0x5583bd86bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33640/0 (socket says 172.21.15.47:33640) 2024-03-21T11:32:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.586+0000 7f542be00640 1 -- 172.21.15.47:0/1097556313 learned_addr learned my addr 172.21.15.47:0/1097556313 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.587+0000 7f5430c00640 1 -- 172.21.15.47:0/1097556313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd862bb0 msgr2=0x5583bd86cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.587+0000 7f5430c00640 1 --2- 172.21.15.47:0/1097556313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd862bb0 0x5583bd86cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.587+0000 7f5430c00640 1 -- 172.21.15.47:0/1097556313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85da40 msgr2=0x5583bd86bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.587+0000 7f5430c00640 1 --2- 172.21.15.47:0/1097556313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85da40 0x5583bd86bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.587+0000 7f5430c00640 1 -- 172.21.15.47:0/1097556313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583bd86d170 con 0x5583bd85d050 2024-03-21T11:32:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.587+0000 7f5431600640 1 --2- 172.21.15.47:0/1097556313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd862bb0 0x5583bd86cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.587+0000 7f542be00640 1 --2- 172.21.15.47:0/1097556313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85da40 0x5583bd86bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.587+0000 7f5430c00640 1 --2- 172.21.15.47:0/1097556313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85d050 0x5583bd86b630 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5420009250 tx=0x7f5420005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c561c1661455960 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.587+0000 7f5412000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:32:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.587+0000 7f5412000640 1 -- 172.21.15.47:0/1097556313 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542001e070 con 0x5583bd85d050 2024-03-21T11:32:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.587+0000 7f54363ed740 1 -- 172.21.15.47:0/1097556313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f542000fb30 con 0x5583bd85d050 2024-03-21T11:32:56.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.587+0000 7f5412000640 1 -- 172.21.15.47:0/1097556313 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f542002a260 con 0x5583bd85d050 2024-03-21T11:32:56.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.588+0000 7f5412000640 1 -- 172.21.15.47:0/1097556313 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5420039050 con 0x5583bd85d050 2024-03-21T11:32:56.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.588+0000 7f54363ed740 10 client.28778 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:56.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.588+0000 7f54363ed740 1 -- 172.21.15.47:0/1097556313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5583bd86e480 con 0x5583bd85d050 2024-03-21T11:32:56.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.588+0000 7f5412000640 1 -- 172.21.15.47:0/1097556313 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f542000fb30 con 0x5583bd85d050 2024-03-21T11:32:56.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.588+0000 7f5412000640 1 -- 172.21.15.47:0/1097556313 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5420019610 con 0x5583bd85d050 2024-03-21T11:32:56.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.588+0000 7f54363ed740 10 client.28778 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.588+0000 7f54363ed740 10 client.28778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.588+0000 7f54363ed740 20 client.28778 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.588+0000 7f54363ed740 1 --2- 172.21.15.47:0/1097556313 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583bd8ce7b0 0x5583bd8d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.588+0000 7f54363ed740 4 client.28778 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:32:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.588+0000 7f54363ed740 1 -- 172.21.15.47:0/1097556313 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5583bd7c9ae0 con 0x5583bd8ce7b0 2024-03-21T11:32:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.588+0000 7f542be00640 1 --2- 172.21.15.47:0/1097556313 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583bd8ce7b0 0x5583bd8d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:56.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.589+0000 7f542be00640 1 --2- 172.21.15.47:0/1097556313 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583bd8ce7b0 0x5583bd8d0ba0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd025b53d76d3159 server_cookie=3e7fd4b6facde54e in_seq=0 out_seq=0 2024-03-21T11:32:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.589+0000 7f5412000640 10 client.28778 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.589+0000 7f5412000640 1 -- 172.21.15.47:0/1097556313 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5583bd7c9ae0 con 0x5583bd8ce7b0 2024-03-21T11:32:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.589+0000 7f5412000640 10 client.28778 handle_command_reply: tid=0 2024-03-21T11:32:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.589+0000 7f54363ed740 1 client.28778 shutdown 2024-03-21T11:32:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.589+0000 7f54363ed740 1 -- 172.21.15.47:0/1097556313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85d050 msgr2=0x5583bd86b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.589+0000 7f54363ed740 1 --2- 172.21.15.47:0/1097556313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85d050 0x5583bd86b630 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5420009250 tx=0x7f5420005f10 comp rx=0 tx=0).stop 2024-03-21T11:32:56.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.590+0000 7f54363ed740 1 -- 172.21.15.47:0/1097556313 shutdown_connections 2024-03-21T11:32:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.590+0000 7f54363ed740 1 --2- 172.21.15.47:0/1097556313 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583bd8ce7b0 0x5583bd8d0ba0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.590+0000 7f54363ed740 1 --2- 172.21.15.47:0/1097556313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583bd862bb0 0x5583bd86cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.590+0000 7f54363ed740 1 --2- 172.21.15.47:0/1097556313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583bd85da40 0x5583bd86bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.590+0000 7f54363ed740 1 --2- 172.21.15.47:0/1097556313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583bd85d050 0x5583bd86b630 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.590+0000 7f54363ed740 1 -- 172.21.15.47:0/1097556313 >> 172.21.15.47:0/1097556313 conn(0x5583bd825200 msgr2=0x5583bd857330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.590+0000 7f5412000640 0 client.28778 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.590+0000 7f54363ed740 1 -- 172.21.15.47:0/1097556313 shutdown_connections 2024-03-21T11:32:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.590+0000 7f54363ed740 1 -- 172.21.15.47:0/1097556313 wait complete. 2024-03-21T11:32:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.590+0000 7f54363ed740 20 client.28778 trim_cache size 0 max 16384 2024-03-21T11:32:56.607 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:32:56.607 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:32:56.607 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:32:56.607 INFO:teuthology.orchestra.run.smithi047.stdout: "f1708e33-c09b-4a29-be8f-e098a641c48e": { 2024-03-21T11:32:56.608 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:32:56.608 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "f1708e33-c09b-4a29-be8f-e098a641c48e", 2024-03-21T11:32:56.608 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:32:56.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:32:56.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:32:56.608 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:32:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.590+0000 7f5432000640 1 -- 172.21.15.47:0/3194614034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54140a2f20 msgr2=0x7f54140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.590+0000 7f5432000640 1 --2- 172.21.15.47:0/3194614034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54140a2f20 0x7f54140a53e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x5583bd85e2a0 tx=0x7f5424009500 comp rx=0 tx=0).stop 2024-03-21T11:32:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.590+0000 7f5432000640 1 -- 172.21.15.47:0/3194614034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f542c075680 msgr2=0x7f542c199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.590+0000 7f5432000640 1 --2- 172.21.15.47:0/3194614034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f542c075680 0x7f542c199f90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5418036040 tx=0x7f541802c6e0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.591+0000 7f5432000640 1 -- 172.21.15.47:0/3194614034 shutdown_connections 2024-03-21T11:32:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.591+0000 7f5432000640 1 --2- 172.21.15.47:0/3194614034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54140a2f20 0x7f54140a53e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.591+0000 7f5432000640 1 --2- 172.21.15.47:0/3194614034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f542c109430 0x7f542c1a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.591+0000 7f5432000640 1 --2- 172.21.15.47:0/3194614034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f542c075680 0x7f542c199f90 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.591+0000 7f5432000640 1 --2- 172.21.15.47:0/3194614034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542c0740a0 0x7f542c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.591+0000 7f5432000640 1 -- 172.21.15.47:0/3194614034 >> 172.21.15.47:0/3194614034 conn(0x7f542c0fcb10 msgr2=0x7f542c1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.591+0000 7f5432000640 1 -- 172.21.15.47:0/3194614034 shutdown_connections 2024-03-21T11:32:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:32:56.591+0000 7f5432000640 1 -- 172.21.15.47:0/3194614034 wait complete. 2024-03-21T11:32:56.613 INFO:tasks.cephfs.filesystem:scrub status for tag:f1708e33-c09b-4a29-be8f-e098a641c48e - {'path': '/', 'tag': 'f1708e33-c09b-4a29-be8f-e098a641c48e', 'options': 'recursive,force'} 2024-03-21T11:32:57.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:57 smithi047 ceph-mon[31351]: pgmap v688: 129 pgs: 129 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 2.0 MiB/s wr, 276 op/s 2024-03-21T11:32:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:57 smithi100 ceph-mon[36245]: pgmap v688: 129 pgs: 129 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 2.0 MiB/s wr, 276 op/s 2024-03-21T11:32:57.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:57 smithi102 ceph-mon[36748]: pgmap v688: 129 pgs: 129 active+clean; 37 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 2.0 MiB/s wr, 276 op/s 2024-03-21T11:32:57.548 INFO:tasks.workunit.client.0.smithi047.stdout: 1 707096 8.14 MB/sec execute 341 sec latency 12.531 ms 2024-03-21T11:32:58.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:58 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:32:58.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:58 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:32:58.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:58 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:32:58.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:58 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:32:58.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:58 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:32:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:58 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:32:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:58 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:32:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:58 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:32:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:58 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:32:58.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:58 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:32:58.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:58 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:32:58.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:58 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:32:58.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:58 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:32:58.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:58 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:32:58.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:58 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:32:58.548 INFO:tasks.workunit.client.0.smithi047.stdout: 1 708196 8.13 MB/sec execute 342 sec latency 30.981 ms 2024-03-21T11:32:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.830+0000 7f10abe00640 1 -- 172.21.15.47:0/3786596285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a4103280 msgr2=0x7f10a4109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.830+0000 7f10abe00640 1 --2- 172.21.15.47:0/3786596285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a4103280 0x7f10a4109b10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f109c009ca0 tx=0x7f109c02c0f0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.830+0000 7f10abe00640 1 -- 172.21.15.47:0/3786596285 shutdown_connections 2024-03-21T11:32:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.830+0000 7f10abe00640 1 --2- 172.21.15.47:0/3786596285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a4103280 0x7f10a4109b10 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.830+0000 7f10abe00640 1 --2- 172.21.15.47:0/3786596285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a41028c0 0x7f10a4102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.830+0000 7f10abe00640 1 --2- 172.21.15.47:0/3786596285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a41016d0 0x7f10a4101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.830+0000 7f10abe00640 1 -- 172.21.15.47:0/3786596285 >> 172.21.15.47:0/3786596285 conn(0x7f10a40fcc60 msgr2=0x7f10a40ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.831+0000 7f10abe00640 1 -- 172.21.15.47:0/3786596285 shutdown_connections 2024-03-21T11:32:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.831+0000 7f10abe00640 1 -- 172.21.15.47:0/3786596285 wait complete. 2024-03-21T11:32:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.832+0000 7f10abe00640 1 Processor -- start 2024-03-21T11:32:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.832+0000 7f10abe00640 1 -- start start 2024-03-21T11:32:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.832+0000 7f10abe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a41016d0 0x7f10a4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.832+0000 7f10abe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a41028c0 0x7f10a4199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.832+0000 7f10abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a4103280 0x7f10a41a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.832+0000 7f10abe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a410d330 con 0x7f10a41016d0 2024-03-21T11:32:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.832+0000 7f10abe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a410d1b0 con 0x7f10a4103280 2024-03-21T11:32:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.832+0000 7f10abe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a410d4b0 con 0x7f10a41028c0 2024-03-21T11:32:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.832+0000 7f10a8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a41028c0 0x7f10a4199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.832+0000 7f10a9400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a41016d0 0x7f10a4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.832+0000 7f10a9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a4103280 0x7f10a41a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.832+0000 7f10a8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a41028c0 0x7f10a4199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50894/0 (socket says 172.21.15.47:50894) 2024-03-21T11:32:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.832+0000 7f10a8a00640 1 -- 172.21.15.47:0/2068293963 learned_addr learned my addr 172.21.15.47:0/2068293963 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.833+0000 7f10a9e00640 1 -- 172.21.15.47:0/2068293963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a41028c0 msgr2=0x7f10a4199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.833+0000 7f10a9e00640 1 --2- 172.21.15.47:0/2068293963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a41028c0 0x7f10a4199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.833+0000 7f10a9e00640 1 -- 172.21.15.47:0/2068293963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a41016d0 msgr2=0x7f10a4199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.833+0000 7f10a9e00640 1 --2- 172.21.15.47:0/2068293963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a41016d0 0x7f10a4199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.833+0000 7f10a9e00640 1 -- 172.21.15.47:0/2068293963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a41a16d0 con 0x7f10a4103280 2024-03-21T11:32:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.833+0000 7f10a8a00640 1 --2- 172.21.15.47:0/2068293963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a41028c0 0x7f10a4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.833+0000 7f10a9e00640 1 --2- 172.21.15.47:0/2068293963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a4103280 0x7f10a41a1060 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f109c037040 tx=0x7f109c032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:58.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.833+0000 7f10a2000640 1 -- 172.21.15.47:0/2068293963 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c041020 con 0x7f10a4103280 2024-03-21T11:32:58.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.834+0000 7f10a2000640 1 -- 172.21.15.47:0/2068293963 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f109c034cd0 con 0x7f10a4103280 2024-03-21T11:32:58.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.834+0000 7f10a2000640 1 -- 172.21.15.47:0/2068293963 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c039040 con 0x7f10a4103280 2024-03-21T11:32:58.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.834+0000 7f10abe00640 1 -- 172.21.15.47:0/2068293963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a41a1960 con 0x7f10a4103280 2024-03-21T11:32:58.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.834+0000 7f10abe00640 1 -- 172.21.15.47:0/2068293963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a41a1e70 con 0x7f10a4103280 2024-03-21T11:32:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.837+0000 7f10ae9f3740 1 Processor -- start 2024-03-21T11:32:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.837+0000 7f10a2000640 1 -- 172.21.15.47:0/2068293963 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f109c044020 con 0x7f10a4103280 2024-03-21T11:32:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.837+0000 7f10ae9f3740 1 -- start start 2024-03-21T11:32:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.837+0000 7f10ae9f3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c01e0 0x555f8a2c0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.837+0000 7f10ae9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c0bd0 0x555f8a2c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.837+0000 7f10ae9f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c5d40 0x555f8a2c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.837+0000 7f10ae9f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a2833b0 con 0x555f8a2c5d40 2024-03-21T11:32:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.837+0000 7f10ae9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a225830 con 0x555f8a2c0bd0 2024-03-21T11:32:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.837+0000 7f10ae9f3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a1ce5d0 con 0x555f8a2c01e0 2024-03-21T11:32:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.838+0000 7f10a8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c01e0 0x555f8a2c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.838+0000 7f10a9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c0bd0 0x555f8a2c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.838+0000 7f10a8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c01e0 0x555f8a2c0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50902/0 (socket says 172.21.15.47:50902) 2024-03-21T11:32:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.838+0000 7f10a8a00640 1 -- 172.21.15.47:0/580339281 learned_addr learned my addr 172.21.15.47:0/580339281 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.838+0000 7f10a8a00640 1 --2- 172.21.15.47:0/580339281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c5d40 0x555f8a2c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.838+0000 7f10a8a00640 1 -- 172.21.15.47:0/580339281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c0bd0 msgr2=0x555f8a2c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.838+0000 7f10a8a00640 1 --2- 172.21.15.47:0/580339281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c0bd0 0x555f8a2c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.838+0000 7f10a8a00640 1 -- 172.21.15.47:0/580339281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c5d40 msgr2=0x555f8a2c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.838+0000 7f10a8a00640 1 --2- 172.21.15.47:0/580339281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c5d40 0x555f8a2c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.838+0000 7f10a8a00640 1 -- 172.21.15.47:0/580339281 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f8a1e62c0 con 0x555f8a2c01e0 2024-03-21T11:32:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.838+0000 7f10a9400640 1 --2- 172.21.15.47:0/580339281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c0bd0 0x555f8a2c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.839+0000 7f10a8a00640 1 --2- 172.21.15.47:0/580339281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c01e0 0x555f8a2c0600 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1094011ca0 tx=0x7f1094009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5fa72f22339c765 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.839+0000 7f10a2000640 1 --2- 172.21.15.47:0/2068293963 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f108c0a3040 0x7f108c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.839+0000 7f10a2000640 1 -- 172.21.15.47:0/2068293963 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f109c0f1590 con 0x7f10a4103280 2024-03-21T11:32:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.839+0000 7f1092600640 1 -- 172.21.15.47:0/580339281 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109400fc50 con 0x555f8a2c01e0 2024-03-21T11:32:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.839+0000 7f10a9400640 1 --2- 172.21.15.47:0/2068293963 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f108c0a3040 0x7f108c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.839+0000 7f1092600640 1 -- 172.21.15.47:0/580339281 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f109400fdf0 con 0x555f8a2c01e0 2024-03-21T11:32:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.839+0000 7f1092600640 1 -- 172.21.15.47:0/580339281 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109400e6b0 con 0x555f8a2c01e0 2024-03-21T11:32:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.840+0000 7f10ae9f3740 1 -- 172.21.15.47:0/580339281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c01e0 msgr2=0x555f8a2c0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.840+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/580339281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c01e0 0x555f8a2c0600 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1094011ca0 tx=0x7f1094009c60 comp rx=0 tx=0).stop 2024-03-21T11:32:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.840+0000 7f10a9400640 1 --2- 172.21.15.47:0/2068293963 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f108c0a3040 0x7f108c0a5500 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x555f8a2c0fd0 tx=0x7f1098005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.840+0000 7f10ae9f3740 1 -- 172.21.15.47:0/580339281 shutdown_connections 2024-03-21T11:32:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.840+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/580339281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c5d40 0x555f8a2c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.840+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/580339281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c0bd0 0x555f8a2c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.840+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/580339281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c01e0 0x555f8a2c0600 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.840+0000 7f10ae9f3740 1 -- 172.21.15.47:0/580339281 >> 172.21.15.47:0/580339281 conn(0x555f8a2bb750 msgr2=0x555f8a2bbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.840+0000 7f10ae9f3740 1 -- 172.21.15.47:0/580339281 shutdown_connections 2024-03-21T11:32:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.840+0000 7f10ae9f3740 1 -- 172.21.15.47:0/580339281 wait complete. 2024-03-21T11:32:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.840+0000 7f10ae9f3740 1 -- start start 2024-03-21T11:32:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.841+0000 7f10ae9f3740 1 Processor -- start 2024-03-21T11:32:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.841+0000 7f10ae9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c01e0 0x555f8a2ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.841+0000 7f10ae9f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c0bd0 0x555f8a2ca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.841+0000 7f10ae9f3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c5d40 0x555f8a2d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.841+0000 7f10ae9f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a1e1a20 con 0x555f8a2c0bd0 2024-03-21T11:32:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.841+0000 7f10ae9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a2218d0 con 0x555f8a2c01e0 2024-03-21T11:32:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.841+0000 7f10ae9f3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8a2c8cf0 con 0x555f8a2c5d40 2024-03-21T11:32:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.841+0000 7f10ae9f3740 10 client.0 _ll_register_callbacks cb 0x555f8a21e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.841+0000 7f10a9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c01e0 0x555f8a2ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.841+0000 7f10a9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c5d40 0x555f8a2d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.841+0000 7f10a9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c01e0 0x555f8a2ca420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32924/0 (socket says 172.21.15.47:32924) 2024-03-21T11:32:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.841+0000 7f10a9400640 1 -- 172.21.15.47:0/838615777 learned_addr learned my addr 172.21.15.47:0/838615777 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.841+0000 7f10a8a00640 1 --2- 172.21.15.47:0/838615777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c0bd0 0x555f8a2ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.842+0000 7f10a9400640 1 -- 172.21.15.47:0/838615777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c5d40 msgr2=0x555f8a2d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.842+0000 7f10a9400640 1 --2- 172.21.15.47:0/838615777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c5d40 0x555f8a2d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.842+0000 7f10a9400640 1 -- 172.21.15.47:0/838615777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c0bd0 msgr2=0x555f8a2ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.842+0000 7f10a9400640 1 --2- 172.21.15.47:0/838615777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c0bd0 0x555f8a2ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.842+0000 7f10a9400640 1 -- 172.21.15.47:0/838615777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f89f3a210 con 0x555f8a2c01e0 2024-03-21T11:32:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.842+0000 7f10a9400640 1 --2- 172.21.15.47:0/838615777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c01e0 0x555f8a2ca420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f109801e9f0 tx=0x7f1098004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b3532531623b163 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.842+0000 7f1086a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:32:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.842+0000 7f1086a00640 1 -- 172.21.15.47:0/838615777 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109802e060 con 0x555f8a2c01e0 2024-03-21T11:32:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.843+0000 7f10ae9f3740 1 -- 172.21.15.47:0/838615777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f109400fdf0 con 0x555f8a2c01e0 2024-03-21T11:32:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.843+0000 7f1086a00640 1 -- 172.21.15.47:0/838615777 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1098018030 con 0x555f8a2c01e0 2024-03-21T11:32:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.843+0000 7f1086a00640 1 -- 172.21.15.47:0/838615777 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1098035050 con 0x555f8a2c01e0 2024-03-21T11:32:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.843+0000 7f10ae9f3740 10 client.28790 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.843+0000 7f10ae9f3740 1 -- 172.21.15.47:0/838615777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555f8a006620 con 0x555f8a2c01e0 2024-03-21T11:32:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.843+0000 7f1086a00640 1 -- 172.21.15.47:0/838615777 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f109400fdf0 con 0x555f8a2c01e0 2024-03-21T11:32:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.843+0000 7f1086a00640 1 -- 172.21.15.47:0/838615777 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f109801c560 con 0x555f8a2c01e0 2024-03-21T11:32:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.843+0000 7f10ae9f3740 10 client.28790 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.843+0000 7f10ae9f3740 10 client.28790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.843+0000 7f10ae9f3740 20 client.28790 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.844+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/838615777 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f8a2d6280 0x555f8a2d8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.844+0000 7f10ae9f3740 4 client.28790 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:32:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.844+0000 7f10ae9f3740 1 -- 172.21.15.47:0/838615777 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f8a232d70 con 0x555f8a2d6280 2024-03-21T11:32:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.844+0000 7f10a8a00640 1 --2- 172.21.15.47:0/838615777 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f8a2d6280 0x555f8a2d8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.844+0000 7f10a8a00640 1 --2- 172.21.15.47:0/838615777 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f8a2d6280 0x555f8a2d8670 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2faaf8c3c4b8e8da server_cookie=dc087ea37345b7f3 in_seq=0 out_seq=0 2024-03-21T11:32:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.844+0000 7f1086a00640 10 client.28790 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f1086a00640 1 -- 172.21.15.47:0/838615777 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555f8a232d70 con 0x555f8a2d6280 2024-03-21T11:32:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f1086a00640 10 client.28790 handle_command_reply: tid=0 2024-03-21T11:32:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f10ae9f3740 1 client.28790 shutdown 2024-03-21T11:32:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f10ae9f3740 1 -- 172.21.15.47:0/838615777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c01e0 msgr2=0x555f8a2ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/838615777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c01e0 0x555f8a2ca420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f109801e9f0 tx=0x7f1098004730 comp rx=0 tx=0).stop 2024-03-21T11:32:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f10ae9f3740 1 -- 172.21.15.47:0/838615777 shutdown_connections 2024-03-21T11:32:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/838615777 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f8a2d6280 0x555f8a2d8670 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/838615777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c5d40 0x555f8a2d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/838615777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c0bd0 0x555f8a2ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/838615777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c01e0 0x555f8a2ca420 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f1086a00640 0 client.28790 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f10ae9f3740 1 -- 172.21.15.47:0/838615777 >> 172.21.15.47:0/838615777 conn(0x555f8a2bb750 msgr2=0x555f8a2ba950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f10ae9f3740 1 -- 172.21.15.47:0/838615777 shutdown_connections 2024-03-21T11:32:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f10ae9f3740 1 -- 172.21.15.47:0/838615777 wait complete. 2024-03-21T11:32:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.846+0000 7f10ae9f3740 20 client.28790 trim_cache size 0 max 16384 2024-03-21T11:32:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.865+0000 7f10ae9f3740 1 Processor -- start 2024-03-21T11:32:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.865+0000 7f10ae9f3740 1 -- start start 2024-03-21T11:32:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.865+0000 7f10ae9f3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c01e0 0x555f8a2ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.865+0000 7f10ae9f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c0bd0 0x555f8a2d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.865+0000 7f10ae9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c5d40 0x555f8a2bf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.865+0000 7f10ae9f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a41ac210 con 0x555f8a2c0bd0 2024-03-21T11:32:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.865+0000 7f10ae9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a41adb90 con 0x555f8a2c5d40 2024-03-21T11:32:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.865+0000 7f10ae9f3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a41ac870 con 0x555f8a2c01e0 2024-03-21T11:32:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.866+0000 7f10a9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c01e0 0x555f8a2ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.866+0000 7f10a9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c5d40 0x555f8a2bf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.866+0000 7f10a9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c01e0 0x555f8a2ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50914/0 (socket says 172.21.15.47:50914) 2024-03-21T11:32:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.866+0000 7f10a9400640 1 -- 172.21.15.47:0/1861250911 learned_addr learned my addr 172.21.15.47:0/1861250911 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.866+0000 7f10a8a00640 1 --2- 172.21.15.47:0/1861250911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c0bd0 0x555f8a2d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.866+0000 7f10a9400640 1 -- 172.21.15.47:0/1861250911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c5d40 msgr2=0x555f8a2bf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.866+0000 7f10a9400640 1 --2- 172.21.15.47:0/1861250911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c5d40 0x555f8a2bf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.866+0000 7f10a9400640 1 -- 172.21.15.47:0/1861250911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c0bd0 msgr2=0x555f8a2d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.866+0000 7f10a9400640 1 --2- 172.21.15.47:0/1861250911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c0bd0 0x555f8a2d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.866+0000 7f10a9400640 1 -- 172.21.15.47:0/1861250911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f8a2c8e30 con 0x555f8a2c01e0 2024-03-21T11:32:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.866+0000 7f10a9400640 1 --2- 172.21.15.47:0/1861250911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c01e0 0x555f8a2ca8c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1098019040 tx=0x7f1098009d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2735cbb7dd07b1e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.866+0000 7f1091c00640 1 -- 172.21.15.47:0/1861250911 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109801cc30 con 0x555f8a2c01e0 2024-03-21T11:32:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.867+0000 7f1091c00640 1 -- 172.21.15.47:0/1861250911 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f109801c3a0 con 0x555f8a2c01e0 2024-03-21T11:32:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.867+0000 7f1091c00640 1 -- 172.21.15.47:0/1861250911 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098005020 con 0x555f8a2c01e0 2024-03-21T11:32:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.867+0000 7f10ae9f3740 1 -- 172.21.15.47:0/1861250911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c01e0 msgr2=0x555f8a2ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.867+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/1861250911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c01e0 0x555f8a2ca8c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1098019040 tx=0x7f1098009d30 comp rx=0 tx=0).stop 2024-03-21T11:32:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.867+0000 7f10ae9f3740 1 -- 172.21.15.47:0/1861250911 shutdown_connections 2024-03-21T11:32:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.867+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/1861250911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c5d40 0x555f8a2bf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.867+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/1861250911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c0bd0 0x555f8a2d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.867+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/1861250911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c01e0 0x555f8a2ca8c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.867+0000 7f10ae9f3740 1 -- 172.21.15.47:0/1861250911 >> 172.21.15.47:0/1861250911 conn(0x555f8a2bb750 msgr2=0x555f8a2be9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.868+0000 7f10ae9f3740 1 -- 172.21.15.47:0/1861250911 shutdown_connections 2024-03-21T11:32:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.868+0000 7f10ae9f3740 1 -- 172.21.15.47:0/1861250911 wait complete. 2024-03-21T11:32:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.868+0000 7f10ae9f3740 1 -- start start 2024-03-21T11:32:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.868+0000 7f10ae9f3740 1 Processor -- start 2024-03-21T11:32:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10ae9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c01e0 0x555f8a2cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10ae9f3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c0bd0 0x555f8a2cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10ae9f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c5d40 0x555f8a2d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10ae9f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a41ad530 con 0x555f8a2c5d40 2024-03-21T11:32:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10ae9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a41aced0 con 0x555f8a2c01e0 2024-03-21T11:32:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10ae9f3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a41aa3d0 con 0x555f8a2c0bd0 2024-03-21T11:32:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10ae9f3740 10 client.0 _ll_register_callbacks cb 0x7f1098009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:32:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10a8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c0bd0 0x555f8a2cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10a9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c01e0 0x555f8a2cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10a8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c0bd0 0x555f8a2cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54870/0 (socket says 172.21.15.47:54870) 2024-03-21T11:32:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10a9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c01e0 0x555f8a2cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46184/0 (socket says 172.21.15.47:46184) 2024-03-21T11:32:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10a8a00640 1 -- 172.21.15.47:0/1580335141 learned_addr learned my addr 172.21.15.47:0/1580335141 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:32:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10a9e00640 1 --2- 172.21.15.47:0/1580335141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c5d40 0x555f8a2d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10a8a00640 1 -- 172.21.15.47:0/1580335141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c01e0 msgr2=0x555f8a2cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10a8a00640 1 --2- 172.21.15.47:0/1580335141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c01e0 0x555f8a2cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10a8a00640 1 -- 172.21.15.47:0/1580335141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c5d40 msgr2=0x555f8a2d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:32:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10a8a00640 1 --2- 172.21.15.47:0/1580335141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c5d40 0x555f8a2d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10a8a00640 1 -- 172.21.15.47:0/1580335141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f8a2d1200 con 0x555f8a2c0bd0 2024-03-21T11:32:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.869+0000 7f10a9400640 1 --2- 172.21.15.47:0/1580335141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c01e0 0x555f8a2cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:32:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.870+0000 7f10a8a00640 1 --2- 172.21.15.47:0/1580335141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c0bd0 0x555f8a2cfc00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f10940097f0 tx=0x7f1094011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24f6eb3309bbbef5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:32:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.870+0000 7f1086a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:32:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.870+0000 7f1086a00640 1 -- 172.21.15.47:0/1580335141 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109400fe20 con 0x555f8a2c0bd0 2024-03-21T11:32:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.870+0000 7f10ae9f3740 1 -- 172.21.15.47:0/1580335141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f109801c3a0 con 0x555f8a2c0bd0 2024-03-21T11:32:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.870+0000 7f1086a00640 1 -- 172.21.15.47:0/1580335141 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f109402c030 con 0x555f8a2c0bd0 2024-03-21T11:32:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.870+0000 7f1086a00640 1 -- 172.21.15.47:0/1580335141 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10940115b0 con 0x555f8a2c0bd0 2024-03-21T11:32:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.870+0000 7f10ae9f3740 10 client.28813 fetch_fsmap learned FSMap version 28 2024-03-21T11:32:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.870+0000 7f10ae9f3740 1 -- 172.21.15.47:0/1580335141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555f8a332180 con 0x555f8a2c0bd0 2024-03-21T11:32:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.871+0000 7f1086a00640 1 -- 172.21.15.47:0/1580335141 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f109801c3a0 con 0x555f8a2c0bd0 2024-03-21T11:32:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.871+0000 7f1086a00640 1 -- 172.21.15.47:0/1580335141 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f109401d020 con 0x555f8a2c0bd0 2024-03-21T11:32:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.871+0000 7f10ae9f3740 10 client.28813 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:32:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.871+0000 7f10ae9f3740 10 client.28813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:32:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.871+0000 7f10ae9f3740 20 client.28813 populate_metadata read hostname 'smithi047' 2024-03-21T11:32:58.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.871+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/1580335141 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f8a332950 0x555f8a334d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:32:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.871+0000 7f10ae9f3740 4 client.28813 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:32:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.871+0000 7f10ae9f3740 1 -- 172.21.15.47:0/1580335141 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555f8a22d0c0 con 0x555f8a332950 2024-03-21T11:32:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.871+0000 7f10a9400640 1 --2- 172.21.15.47:0/1580335141 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f8a332950 0x555f8a334d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:32:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.872+0000 7f10a9400640 1 --2- 172.21.15.47:0/1580335141 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f8a332950 0x555f8a334d70 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e51886db461cc96f server_cookie=59022f16d8bbc38c in_seq=0 out_seq=0 2024-03-21T11:32:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:58.872+0000 7f1086a00640 10 client.28813 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.096+0000 7f1086a00640 1 -- 172.21.15.47:0/1580335141 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555f8a22d0c0 con 0x555f8a332950 2024-03-21T11:32:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.096+0000 7f1086a00640 10 client.28813 handle_command_reply: tid=0 2024-03-21T11:32:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.096+0000 7f10ae9f3740 1 client.28813 shutdown 2024-03-21T11:32:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.096+0000 7f10ae9f3740 1 -- 172.21.15.47:0/1580335141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c0bd0 msgr2=0x555f8a2cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.096+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/1580335141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c0bd0 0x555f8a2cfc00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f10940097f0 tx=0x7f1094011810 comp rx=0 tx=0).stop 2024-03-21T11:32:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.096+0000 7f10ae9f3740 1 -- 172.21.15.47:0/1580335141 shutdown_connections 2024-03-21T11:32:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.096+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/1580335141 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f8a332950 0x555f8a334d70 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.096+0000 7f1086a00640 0 client.28813 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:32:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.096+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/1580335141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f8a2c5d40 0x555f8a2d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.096+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/1580335141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f8a2c0bd0 0x555f8a2cfc00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.096+0000 7f10ae9f3740 1 --2- 172.21.15.47:0/1580335141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f8a2c01e0 0x555f8a2cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.096+0000 7f10ae9f3740 1 -- 172.21.15.47:0/1580335141 >> 172.21.15.47:0/1580335141 conn(0x555f8a2bb750 msgr2=0x555f8a2be9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.097+0000 7f10ae9f3740 1 -- 172.21.15.47:0/1580335141 shutdown_connections 2024-03-21T11:32:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.097+0000 7f10ae9f3740 1 -- 172.21.15.47:0/1580335141 wait complete. 2024-03-21T11:32:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.097+0000 7f10ae9f3740 20 client.28813 trim_cache size 0 max 16384 2024-03-21T11:32:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:32:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:32:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:32:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:32:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.098+0000 7f10abe00640 1 -- 172.21.15.47:0/2068293963 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f108c0a3040 msgr2=0x7f108c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.098+0000 7f10abe00640 1 --2- 172.21.15.47:0/2068293963 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f108c0a3040 0x7f108c0a5500 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x555f8a2c0fd0 tx=0x7f1098005f90 comp rx=0 tx=0).stop 2024-03-21T11:32:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.098+0000 7f10abe00640 1 -- 172.21.15.47:0/2068293963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a4103280 msgr2=0x7f10a41a1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:32:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.098+0000 7f10abe00640 1 --2- 172.21.15.47:0/2068293963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a4103280 0x7f10a41a1060 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f109c037040 tx=0x7f109c032ea0 comp rx=0 tx=0).stop 2024-03-21T11:32:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.098+0000 7f10abe00640 1 -- 172.21.15.47:0/2068293963 shutdown_connections 2024-03-21T11:32:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.098+0000 7f10abe00640 1 --2- 172.21.15.47:0/2068293963 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f108c0a3040 0x7f108c0a5500 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.098+0000 7f10abe00640 1 --2- 172.21.15.47:0/2068293963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a4103280 0x7f10a41a1060 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.098+0000 7f10abe00640 1 --2- 172.21.15.47:0/2068293963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a41028c0 0x7f10a4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.098+0000 7f10abe00640 1 --2- 172.21.15.47:0/2068293963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a41016d0 0x7f10a4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:32:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.098+0000 7f10abe00640 1 -- 172.21.15.47:0/2068293963 >> 172.21.15.47:0/2068293963 conn(0x7f10a40fcc60 msgr2=0x7f10a40738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:32:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.098+0000 7f10abe00640 1 -- 172.21.15.47:0/2068293963 shutdown_connections 2024-03-21T11:32:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:32:59.100+0000 7f10abe00640 1 -- 172.21.15.47:0/2068293963 wait complete. 2024-03-21T11:32:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:32:59 smithi047 ceph-mon[31351]: pgmap v689: 129 pgs: 129 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-03-21T11:32:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:32:59 smithi100 ceph-mon[36245]: pgmap v689: 129 pgs: 129 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-03-21T11:32:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:32:59 smithi102 ceph-mon[36748]: pgmap v689: 129 pgs: 129 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-03-21T11:32:59.548 INFO:tasks.workunit.client.0.smithi047.stdout: 1 709670 8.13 MB/sec execute 343 sec latency 98.575 ms 2024-03-21T11:33:00.548 INFO:tasks.workunit.client.0.smithi047.stdout: 1 710806 8.13 MB/sec execute 344 sec latency 18.546 ms 2024-03-21T11:33:01.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:01 smithi047 ceph-mon[31351]: pgmap v690: 129 pgs: 129 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 265 op/s 2024-03-21T11:33:01.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:01 smithi100 ceph-mon[36245]: pgmap v690: 129 pgs: 129 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 265 op/s 2024-03-21T11:33:01.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:01 smithi102 ceph-mon[36748]: pgmap v690: 129 pgs: 129 active+clean; 39 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 265 op/s 2024-03-21T11:33:01.548 INFO:tasks.workunit.client.0.smithi047.stdout: 1 712340 8.12 MB/sec execute 345 sec latency 17.064 ms 2024-03-21T11:33:02.548 INFO:tasks.workunit.client.0.smithi047.stdout: 1 713987 8.13 MB/sec execute 346 sec latency 13.095 ms 2024-03-21T11:33:03.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:03 smithi047 ceph-mon[31351]: pgmap v691: 129 pgs: 129 active+clean; 37 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-03-21T11:33:03.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:03 smithi100 ceph-mon[36245]: pgmap v691: 129 pgs: 129 active+clean; 37 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-03-21T11:33:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:03 smithi102 ceph-mon[36748]: pgmap v691: 129 pgs: 129 active+clean; 37 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-03-21T11:33:03.548 INFO:tasks.workunit.client.0.smithi047.stdout: 1 714709 8.11 MB/sec execute 347 sec latency 16.328 ms 2024-03-21T11:33:04.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:04 smithi100 ceph-mon[36245]: pgmap v692: 129 pgs: 129 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 282 op/s 2024-03-21T11:33:04.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:04 smithi102 ceph-mon[36748]: pgmap v692: 129 pgs: 129 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 282 op/s 2024-03-21T11:33:04.548 INFO:tasks.workunit.client.0.smithi047.stdout: 1 716318 8.11 MB/sec execute 348 sec latency 42.843 ms 2024-03-21T11:33:04.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:04 smithi047 ceph-mon[31351]: pgmap v692: 129 pgs: 129 active+clean; 33 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 282 op/s 2024-03-21T11:33:05.548 INFO:tasks.workunit.client.0.smithi047.stdout: 1 717674 8.12 MB/sec execute 349 sec latency 13.431 ms 2024-03-21T11:33:06.549 INFO:tasks.workunit.client.0.smithi047.stdout: 1 718806 8.11 MB/sec execute 350 sec latency 16.260 ms 2024-03-21T11:33:06.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:06 smithi102 ceph-mon[36748]: pgmap v693: 129 pgs: 129 active+clean; 34 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.8 MiB/s wr, 239 op/s 2024-03-21T11:33:06.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:06 smithi047 ceph-mon[31351]: pgmap v693: 129 pgs: 129 active+clean; 34 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.8 MiB/s wr, 239 op/s 2024-03-21T11:33:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:06 smithi100 ceph-mon[36245]: pgmap v693: 129 pgs: 129 active+clean; 34 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.8 MiB/s wr, 239 op/s 2024-03-21T11:33:07.549 INFO:tasks.workunit.client.0.smithi047.stdout: 1 720936 8.12 MB/sec execute 351 sec latency 10.014 ms 2024-03-21T11:33:08.549 INFO:tasks.workunit.client.0.smithi047.stdout: 1 721836 8.11 MB/sec execute 352 sec latency 19.100 ms 2024-03-21T11:33:08.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:08 smithi102 ceph-mon[36748]: pgmap v694: 129 pgs: 129 active+clean; 36 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 282 op/s 2024-03-21T11:33:08.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:08 smithi047 ceph-mon[31351]: pgmap v694: 129 pgs: 129 active+clean; 36 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 282 op/s 2024-03-21T11:33:08.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:08 smithi100 ceph-mon[36245]: pgmap v694: 129 pgs: 129 active+clean; 36 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 282 op/s 2024-03-21T11:33:09.549 INFO:tasks.workunit.client.0.smithi047.stdout: 1 723618 8.11 MB/sec execute 353 sec latency 46.589 ms 2024-03-21T11:33:10.549 INFO:tasks.workunit.client.0.smithi047.stdout: 1 724948 8.11 MB/sec execute 354 sec latency 10.767 ms 2024-03-21T11:33:10.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:10 smithi102 ceph-mon[36748]: pgmap v695: 129 pgs: 129 active+clean; 36 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-03-21T11:33:10.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:10 smithi047 ceph-mon[31351]: pgmap v695: 129 pgs: 129 active+clean; 36 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-03-21T11:33:10.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:10 smithi100 ceph-mon[36245]: pgmap v695: 129 pgs: 129 active+clean; 36 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-03-21T11:33:11.549 INFO:tasks.workunit.client.0.smithi047.stdout: 1 726294 8.10 MB/sec execute 355 sec latency 17.391 ms 2024-03-21T11:33:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.266+0000 7f42fd200640 1 -- 172.21.15.47:0/2994330854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f8104af0 msgr2=0x7f42f8104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.266+0000 7f42fd200640 1 --2- 172.21.15.47:0/2994330854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f8104af0 0x7f42f8104f70 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009900 tx=0x7f42e802bc50 comp rx=0 tx=0).stop 2024-03-21T11:33:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.266+0000 7f42fd200640 1 -- 172.21.15.47:0/2994330854 shutdown_connections 2024-03-21T11:33:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.266+0000 7f42fd200640 1 --2- 172.21.15.47:0/2994330854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42f81054b0 0x7f42f810bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.266+0000 7f42fd200640 1 --2- 172.21.15.47:0/2994330854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f8104af0 0x7f42f8104f70 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.266+0000 7f42fd200640 1 --2- 172.21.15.47:0/2994330854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42f8103900 0x7f42f8103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.266+0000 7f42fd200640 1 -- 172.21.15.47:0/2994330854 >> 172.21.15.47:0/2994330854 conn(0x7f42f80fee90 msgr2=0x7f42f81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.266+0000 7f42fd200640 1 -- 172.21.15.47:0/2994330854 shutdown_connections 2024-03-21T11:33:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.266+0000 7f42fd200640 1 -- 172.21.15.47:0/2994330854 wait complete. 2024-03-21T11:33:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.267+0000 7f42fd200640 1 Processor -- start 2024-03-21T11:33:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.267+0000 7f42fd200640 1 -- start start 2024-03-21T11:33:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42fd200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42f8103900 0x7f42f8199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42fd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f8104af0 0x7f42f819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42fd200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42f81054b0 0x7f42f81a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42fd200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f810f560 con 0x7f42f81054b0 2024-03-21T11:33:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42fd200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f810f3e0 con 0x7f42f8104af0 2024-03-21T11:33:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42fd200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f810f6e0 con 0x7f42f8103900 2024-03-21T11:33:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42f5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42f8103900 0x7f42f8199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42f5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42f8103900 0x7f42f8199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38690/0 (socket says 172.21.15.47:38690) 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42f5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f8104af0 0x7f42f819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42f5e00640 1 -- 172.21.15.47:0/298813287 learned_addr learned my addr 172.21.15.47:0/298813287 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42f6800640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42f81054b0 0x7f42f81a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42f5e00640 1 -- 172.21.15.47:0/298813287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f8104af0 msgr2=0x7f42f819a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42f5e00640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f8104af0 0x7f42f819a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42f5e00640 1 -- 172.21.15.47:0/298813287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42f81054b0 msgr2=0x7f42f81a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42f5e00640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42f81054b0 0x7f42f81a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42f5e00640 1 -- 172.21.15.47:0/298813287 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f81a1840 con 0x7f42f8103900 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42f5400640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f8104af0 0x7f42f819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.268+0000 7f42f6800640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42f81054b0 0x7f42f81a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.269+0000 7f42f5e00640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42f8103900 0x7f42f8199c30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f42e400e9c0 tx=0x7f42e400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.269+0000 7f42eea00640 1 -- 172.21.15.47:0/298813287 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e400cde0 con 0x7f42f8103900 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.269+0000 7f42eea00640 1 -- 172.21.15.47:0/298813287 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e4010910 con 0x7f42f8103900 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.269+0000 7f42fd200640 1 -- 172.21.15.47:0/298813287 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f81a1b30 con 0x7f42f8103900 2024-03-21T11:33:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.269+0000 7f42eea00640 1 -- 172.21.15.47:0/298813287 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4018a00 con 0x7f42f8103900 2024-03-21T11:33:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.269+0000 7f42fd200640 1 -- 172.21.15.47:0/298813287 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f81a2080 con 0x7f42f8103900 2024-03-21T11:33:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.272+0000 7f42ffe59740 1 Processor -- start 2024-03-21T11:33:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.273+0000 7f42ffe59740 1 -- start start 2024-03-21T11:33:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.273+0000 7f42ffe59740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d751e0 0x558b22d75600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.273+0000 7f42ffe59740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d7a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.273+0000 7f42eea00640 1 -- 172.21.15.47:0/298813287 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f42e401f020 con 0x7f42f8103900 2024-03-21T11:33:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.273+0000 7f42ffe59740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d7ad40 0x558b22d7d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.273+0000 7f42ffe59740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b22d383b0 con 0x558b22d75bd0 2024-03-21T11:33:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.273+0000 7f42ffe59740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b22cda830 con 0x558b22d751e0 2024-03-21T11:33:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.273+0000 7f42ffe59740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b22c835d0 con 0x558b22d7ad40 2024-03-21T11:33:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.273+0000 7f42f6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d751e0 0x558b22d75600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.273+0000 7f42f5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d7a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.273+0000 7f42f5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d7ad40 0x558b22d7d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.273+0000 7f42f6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d751e0 0x558b22d75600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34656/0 (socket says 172.21.15.47:34656) 2024-03-21T11:33:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.273+0000 7f42f6800640 1 -- 172.21.15.47:0/1708880139 learned_addr learned my addr 172.21.15.47:0/1708880139 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.274+0000 7f42f5400640 1 -- 172.21.15.47:0/1708880139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d751e0 msgr2=0x558b22d75600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.274+0000 7f42f5400640 1 --2- 172.21.15.47:0/1708880139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d751e0 0x558b22d75600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.274+0000 7f42f5400640 1 -- 172.21.15.47:0/1708880139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 msgr2=0x558b22d7a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.274+0000 7f42f5400640 1 --2- 172.21.15.47:0/1708880139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d7a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.274+0000 7f42f5400640 1 -- 172.21.15.47:0/1708880139 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b22c9b2c0 con 0x558b22d7ad40 2024-03-21T11:33:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.274+0000 7f42f5400640 1 --2- 172.21.15.47:0/1708880139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d7ad40 0x558b22d7d1d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009230 tx=0x7f42e802c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e9a836d49fcc6f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.274+0000 7f42e3400640 1 -- 172.21.15.47:0/1708880139 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8004030 con 0x558b22d7ad40 2024-03-21T11:33:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.274+0000 7f42e3400640 1 -- 172.21.15.47:0/1708880139 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e8005520 con 0x558b22d7ad40 2024-03-21T11:33:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.274+0000 7f42e3400640 1 -- 172.21.15.47:0/1708880139 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8004cb0 con 0x558b22d7ad40 2024-03-21T11:33:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.275+0000 7f42eea00640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f42dc0a2fc0 0x7f42dc0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.275+0000 7f42eea00640 1 -- 172.21.15.47:0/298813287 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f42e40cfaf0 con 0x7f42f8103900 2024-03-21T11:33:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.275+0000 7f42f6800640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f42dc0a2fc0 0x7f42dc0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.275+0000 7f42ffe59740 1 -- 172.21.15.47:0/1708880139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d7ad40 msgr2=0x558b22d7d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.275+0000 7f42ffe59740 1 --2- 172.21.15.47:0/1708880139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d7ad40 0x558b22d7d1d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009230 tx=0x7f42e802c6f0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.275+0000 7f42ffe59740 1 -- 172.21.15.47:0/1708880139 shutdown_connections 2024-03-21T11:33:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.275+0000 7f42ffe59740 1 --2- 172.21.15.47:0/1708880139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d7ad40 0x558b22d7d1d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.275+0000 7f42ffe59740 1 --2- 172.21.15.47:0/1708880139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d7a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.275+0000 7f42ffe59740 1 --2- 172.21.15.47:0/1708880139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d751e0 0x558b22d75600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.275+0000 7f42ffe59740 1 -- 172.21.15.47:0/1708880139 >> 172.21.15.47:0/1708880139 conn(0x558b22d70750 msgr2=0x558b22d70b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.276+0000 7f42ffe59740 1 -- 172.21.15.47:0/1708880139 shutdown_connections 2024-03-21T11:33:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.276+0000 7f42ffe59740 1 -- 172.21.15.47:0/1708880139 wait complete. 2024-03-21T11:33:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.276+0000 7f42f6800640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f42dc0a2fc0 0x7f42dc0a5480 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x558b22d76430 tx=0x7f42f0009940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.276+0000 7f42ffe59740 1 -- start start 2024-03-21T11:33:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.276+0000 7f42ffe59740 1 Processor -- start 2024-03-21T11:33:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42ffe59740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d751e0 0x558b22d7f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42ffe59740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d7f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42ffe59740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d7ad40 0x558b22d86820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42ffe59740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b22c96a20 con 0x558b22d75bd0 2024-03-21T11:33:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42ffe59740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b22cd68d0 con 0x558b22d7ad40 2024-03-21T11:33:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42ffe59740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b22d7dc90 con 0x558b22d751e0 2024-03-21T11:33:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42ffe59740 10 client.0 _ll_register_callbacks cb 0x558b22cd31c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42f5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d751e0 0x558b22d7f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42f5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d751e0 0x558b22d7f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38700/0 (socket says 172.21.15.47:38700) 2024-03-21T11:33:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42f5e00640 1 -- 172.21.15.47:0/1476842406 learned_addr learned my addr 172.21.15.47:0/1476842406 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42f6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d7ad40 0x558b22d86820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42f5400640 1 --2- 172.21.15.47:0/1476842406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d7f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42f5e00640 1 -- 172.21.15.47:0/1476842406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d7ad40 msgr2=0x558b22d86820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42f5e00640 1 --2- 172.21.15.47:0/1476842406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d7ad40 0x558b22d86820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42f5e00640 1 -- 172.21.15.47:0/1476842406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 msgr2=0x558b22d7f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42f5e00640 1 --2- 172.21.15.47:0/1476842406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d7f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.277+0000 7f42f5e00640 1 -- 172.21.15.47:0/1476842406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b229ef210 con 0x558b22d751e0 2024-03-21T11:33:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.278+0000 7f42f5400640 1 --2- 172.21.15.47:0/1476842406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d7f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:33:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.278+0000 7f42f5e00640 1 --2- 172.21.15.47:0/1476842406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d751e0 0x558b22d7f2a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f42e4098770 tx=0x7f42e4009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c18499d4cb2a790 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.278+0000 7f42d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:33:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.278+0000 7f42d7e00640 1 -- 172.21.15.47:0/1476842406 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e40d9040 con 0x558b22d751e0 2024-03-21T11:33:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.278+0000 7f42d7e00640 1 -- 172.21.15.47:0/1476842406 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f42e40a43b0 con 0x558b22d751e0 2024-03-21T11:33:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.278+0000 7f42ffe59740 1 -- 172.21.15.47:0/1476842406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42e8005520 con 0x558b22d751e0 2024-03-21T11:33:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.278+0000 7f42d7e00640 1 -- 172.21.15.47:0/1476842406 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42e409b790 con 0x558b22d751e0 2024-03-21T11:33:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.278+0000 7f42ffe59740 10 client.28825 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.278+0000 7f42ffe59740 1 -- 172.21.15.47:0/1476842406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558b22abb620 con 0x558b22d751e0 2024-03-21T11:33:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.279+0000 7f42d7e00640 1 -- 172.21.15.47:0/1476842406 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e8005520 con 0x558b22d751e0 2024-03-21T11:33:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.279+0000 7f42d7e00640 1 -- 172.21.15.47:0/1476842406 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f42e40baba0 con 0x558b22d751e0 2024-03-21T11:33:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.279+0000 7f42ffe59740 10 client.28825 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.279+0000 7f42ffe59740 10 client.28825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.279+0000 7f42ffe59740 20 client.28825 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.279+0000 7f42ffe59740 1 --2- 172.21.15.47:0/1476842406 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558b22d8b150 0x558b22d8d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.279+0000 7f42ffe59740 4 client.28825 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:33:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.279+0000 7f42ffe59740 1 -- 172.21.15.47:0/1476842406 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b22ce7d70 con 0x558b22d8b150 2024-03-21T11:33:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.279+0000 7f42f5400640 1 --2- 172.21.15.47:0/1476842406 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558b22d8b150 0x558b22d8d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.280+0000 7f42f5400640 1 --2- 172.21.15.47:0/1476842406 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558b22d8b150 0x558b22d8d580 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf9d8c1a023ee0e7 server_cookie=389f14d3c5a53ea9 in_seq=0 out_seq=0 2024-03-21T11:33:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.280+0000 7f42d7e00640 10 client.28825 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42d7e00640 1 -- 172.21.15.47:0/1476842406 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558b22ce7d70 con 0x558b22d8b150 2024-03-21T11:33:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42d7e00640 10 client.28825 handle_command_reply: tid=0 2024-03-21T11:33:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42ffe59740 1 client.28825 shutdown 2024-03-21T11:33:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42ffe59740 1 -- 172.21.15.47:0/1476842406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d751e0 msgr2=0x558b22d7f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42ffe59740 1 --2- 172.21.15.47:0/1476842406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d751e0 0x558b22d7f2a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f42e4098770 tx=0x7f42e4009550 comp rx=0 tx=0).stop 2024-03-21T11:33:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42ffe59740 1 -- 172.21.15.47:0/1476842406 shutdown_connections 2024-03-21T11:33:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42ffe59740 1 --2- 172.21.15.47:0/1476842406 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558b22d8b150 0x558b22d8d580 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42ffe59740 1 --2- 172.21.15.47:0/1476842406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d7ad40 0x558b22d86820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42ffe59740 1 --2- 172.21.15.47:0/1476842406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d7f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42ffe59740 1 --2- 172.21.15.47:0/1476842406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d751e0 0x558b22d7f2a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42d7e00640 0 client.28825 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42ffe59740 1 -- 172.21.15.47:0/1476842406 >> 172.21.15.47:0/1476842406 conn(0x558b22d70750 msgr2=0x558b22d6f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42ffe59740 1 -- 172.21.15.47:0/1476842406 shutdown_connections 2024-03-21T11:33:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42ffe59740 1 -- 172.21.15.47:0/1476842406 wait complete. 2024-03-21T11:33:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.281+0000 7f42ffe59740 20 client.28825 trim_cache size 0 max 16384 2024-03-21T11:33:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.298+0000 7f42ffe59740 1 Processor -- start 2024-03-21T11:33:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.298+0000 7f42ffe59740 1 -- start start 2024-03-21T11:33:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.298+0000 7f42ffe59740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d751e0 0x558b22d70b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.298+0000 7f42ffe59740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.298+0000 7f42ffe59740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d7ad40 0x558b22d739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.298+0000 7f42ffe59740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f81ac190 con 0x558b22d75bd0 2024-03-21T11:33:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.298+0000 7f42ffe59740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f81adb10 con 0x558b22d751e0 2024-03-21T11:33:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.298+0000 7f42ffe59740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f81ac7f0 con 0x558b22d7ad40 2024-03-21T11:33:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.299+0000 7f42f5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.299+0000 7f42f6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d7ad40 0x558b22d739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.299+0000 7f42f5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d867b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47050/0 (socket says 172.21.15.47:47050) 2024-03-21T11:33:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.299+0000 7f42f5400640 1 -- 172.21.15.47:0/1162127498 learned_addr learned my addr 172.21.15.47:0/1162127498 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.299+0000 7f42f5e00640 1 --2- 172.21.15.47:0/1162127498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d751e0 0x558b22d70b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.299+0000 7f42f6800640 1 -- 172.21.15.47:0/1162127498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d751e0 msgr2=0x558b22d70b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.299+0000 7f42f6800640 1 --2- 172.21.15.47:0/1162127498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d751e0 0x558b22d70b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.299+0000 7f42f6800640 1 -- 172.21.15.47:0/1162127498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 msgr2=0x558b22d867b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.299+0000 7f42f6800640 1 --2- 172.21.15.47:0/1162127498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d867b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.299+0000 7f42f6800640 1 -- 172.21.15.47:0/1162127498 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b22d75fb0 con 0x558b22d7ad40 2024-03-21T11:33:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.299+0000 7f42f5400640 1 --2- 172.21.15.47:0/1162127498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d867b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.299+0000 7f42f6800640 1 --2- 172.21.15.47:0/1162127498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d7ad40 0x558b22d739b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009f30 tx=0x7f42f0015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f0ddc45b1baf6bf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.300+0000 7f42e2a00640 1 -- 172.21.15.47:0/1162127498 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0004b20 con 0x558b22d7ad40 2024-03-21T11:33:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.300+0000 7f42e2a00640 1 -- 172.21.15.47:0/1162127498 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42f0004cc0 con 0x558b22d7ad40 2024-03-21T11:33:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.300+0000 7f42e2a00640 1 -- 172.21.15.47:0/1162127498 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f001e670 con 0x558b22d7ad40 2024-03-21T11:33:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.300+0000 7f42ffe59740 1 -- 172.21.15.47:0/1162127498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d7ad40 msgr2=0x558b22d739b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.300+0000 7f42ffe59740 1 --2- 172.21.15.47:0/1162127498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d7ad40 0x558b22d739b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009f30 tx=0x7f42f0015000 comp rx=0 tx=0).stop 2024-03-21T11:33:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.300+0000 7f42ffe59740 1 -- 172.21.15.47:0/1162127498 shutdown_connections 2024-03-21T11:33:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.300+0000 7f42ffe59740 1 --2- 172.21.15.47:0/1162127498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d7ad40 0x558b22d739b0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.300+0000 7f42ffe59740 1 --2- 172.21.15.47:0/1162127498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d867b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.300+0000 7f42ffe59740 1 --2- 172.21.15.47:0/1162127498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d751e0 0x558b22d70b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.300+0000 7f42ffe59740 1 -- 172.21.15.47:0/1162127498 >> 172.21.15.47:0/1162127498 conn(0x558b22d70750 msgr2=0x558b22d7e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.301+0000 7f42ffe59740 1 -- 172.21.15.47:0/1162127498 shutdown_connections 2024-03-21T11:33:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.301+0000 7f42ffe59740 1 -- 172.21.15.47:0/1162127498 wait complete. 2024-03-21T11:33:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.301+0000 7f42ffe59740 1 -- start start 2024-03-21T11:33:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.301+0000 7f42ffe59740 1 Processor -- start 2024-03-21T11:33:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42ffe59740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d751e0 0x558b22d84410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42ffe59740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d84950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42ffe59740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d7ad40 0x558b22d85910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42ffe59740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f81ad4b0 con 0x558b22d75bd0 2024-03-21T11:33:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42ffe59740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f81ace50 con 0x558b22d7ad40 2024-03-21T11:33:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42ffe59740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f81aa350 con 0x558b22d751e0 2024-03-21T11:33:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42ffe59740 10 client.0 _ll_register_callbacks cb 0x7f42e4010e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42f5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d751e0 0x558b22d84410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42f6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d7ad40 0x558b22d85910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42f5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d751e0 0x558b22d84410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38716/0 (socket says 172.21.15.47:38716) 2024-03-21T11:33:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42f5e00640 1 -- 172.21.15.47:0/3726160403 learned_addr learned my addr 172.21.15.47:0/3726160403 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42f5400640 1 --2- 172.21.15.47:0/3726160403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d84950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42f5e00640 1 -- 172.21.15.47:0/3726160403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d7ad40 msgr2=0x558b22d85910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42f5e00640 1 --2- 172.21.15.47:0/3726160403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d7ad40 0x558b22d85910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42f5e00640 1 -- 172.21.15.47:0/3726160403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 msgr2=0x558b22d84950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42f5e00640 1 --2- 172.21.15.47:0/3726160403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d84950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42f5e00640 1 -- 172.21.15.47:0/3726160403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b22d85f00 con 0x558b22d751e0 2024-03-21T11:33:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42f5e00640 1 --2- 172.21.15.47:0/3726160403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d751e0 0x558b22d84410 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f42e400c6a0 tx=0x7f42e409ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd79eb44a8a62716 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.302+0000 7f42d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:33:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.303+0000 7f42d7e00640 1 -- 172.21.15.47:0/3726160403 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4099f70 con 0x558b22d751e0 2024-03-21T11:33:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.303+0000 7f42d7e00640 1 -- 172.21.15.47:0/3726160403 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f42e4010430 con 0x558b22d751e0 2024-03-21T11:33:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.303+0000 7f42ffe59740 1 -- 172.21.15.47:0/3726160403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42f0004cc0 con 0x558b22d751e0 2024-03-21T11:33:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.303+0000 7f42d7e00640 1 -- 172.21.15.47:0/3726160403 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42e409bc00 con 0x558b22d751e0 2024-03-21T11:33:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.303+0000 7f42ffe59740 10 client.28831 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.303+0000 7f42ffe59740 1 -- 172.21.15.47:0/3726160403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558b22de6fe0 con 0x558b22d751e0 2024-03-21T11:33:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.303+0000 7f42d7e00640 1 -- 172.21.15.47:0/3726160403 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42f0004cc0 con 0x558b22d751e0 2024-03-21T11:33:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.304+0000 7f42d7e00640 1 -- 172.21.15.47:0/3726160403 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f42e409acf0 con 0x558b22d751e0 2024-03-21T11:33:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.304+0000 7f42ffe59740 10 client.28831 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.304+0000 7f42ffe59740 10 client.28831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.304+0000 7f42ffe59740 20 client.28831 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.304+0000 7f42ffe59740 1 --2- 172.21.15.47:0/3726160403 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558b22de7820 0x558b22de9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.304+0000 7f42ffe59740 4 client.28831 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:33:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.304+0000 7f42ffe59740 1 -- 172.21.15.47:0/3726160403 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558b22ce20c0 con 0x558b22de7820 2024-03-21T11:33:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.304+0000 7f42f5400640 1 --2- 172.21.15.47:0/3726160403 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558b22de7820 0x558b22de9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.305+0000 7f42f5400640 1 --2- 172.21.15.47:0/3726160403 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558b22de7820 0x558b22de9ce0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=409b51b827b8100e server_cookie=8fe3540debbbd52f in_seq=0 out_seq=0 2024-03-21T11:33:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.305+0000 7f42d7e00640 10 client.28831 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.537+0000 7f42d7e00640 1 -- 172.21.15.47:0/3726160403 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558b22ce20c0 con 0x558b22de7820 2024-03-21T11:33:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.537+0000 7f42d7e00640 10 client.28831 handle_command_reply: tid=0 2024-03-21T11:33:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.537+0000 7f42ffe59740 1 client.28831 shutdown 2024-03-21T11:33:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.537+0000 7f42ffe59740 1 -- 172.21.15.47:0/3726160403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d751e0 msgr2=0x558b22d84410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.537+0000 7f42ffe59740 1 --2- 172.21.15.47:0/3726160403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d751e0 0x558b22d84410 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f42e400c6a0 tx=0x7f42e409ba30 comp rx=0 tx=0).stop 2024-03-21T11:33:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.538+0000 7f42ffe59740 1 -- 172.21.15.47:0/3726160403 shutdown_connections 2024-03-21T11:33:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.538+0000 7f42ffe59740 1 --2- 172.21.15.47:0/3726160403 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558b22de7820 0x558b22de9ce0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.538+0000 7f42ffe59740 1 --2- 172.21.15.47:0/3726160403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558b22d7ad40 0x558b22d85910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.538+0000 7f42ffe59740 1 --2- 172.21.15.47:0/3726160403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b22d75bd0 0x558b22d84950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.538+0000 7f42d7e00640 0 client.28831 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.538+0000 7f42ffe59740 1 --2- 172.21.15.47:0/3726160403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558b22d751e0 0x558b22d84410 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.538+0000 7f42ffe59740 1 -- 172.21.15.47:0/3726160403 >> 172.21.15.47:0/3726160403 conn(0x558b22d70750 msgr2=0x558b22d70b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.538+0000 7f42ffe59740 1 -- 172.21.15.47:0/3726160403 shutdown_connections 2024-03-21T11:33:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.538+0000 7f42ffe59740 1 -- 172.21.15.47:0/3726160403 wait complete. 2024-03-21T11:33:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.538+0000 7f42ffe59740 20 client.28831 trim_cache size 0 max 16384 2024-03-21T11:33:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:33:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:33:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:33:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:33:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.539+0000 7f42fd200640 1 -- 172.21.15.47:0/298813287 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f42dc0a2fc0 msgr2=0x7f42dc0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.539+0000 7f42fd200640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f42dc0a2fc0 0x7f42dc0a5480 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x558b22d76430 tx=0x7f42f0009940 comp rx=0 tx=0).stop 2024-03-21T11:33:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.539+0000 7f42fd200640 1 -- 172.21.15.47:0/298813287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42f8103900 msgr2=0x7f42f8199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.539+0000 7f42fd200640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42f8103900 0x7f42f8199c30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f42e400e9c0 tx=0x7f42e400ee80 comp rx=0 tx=0).stop 2024-03-21T11:33:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.539+0000 7f42fd200640 1 -- 172.21.15.47:0/298813287 shutdown_connections 2024-03-21T11:33:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.539+0000 7f42fd200640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f42dc0a2fc0 0x7f42dc0a5480 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.539+0000 7f42fd200640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42f81054b0 0x7f42f81a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.539+0000 7f42fd200640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f8104af0 0x7f42f819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.539+0000 7f42fd200640 1 --2- 172.21.15.47:0/298813287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42f8103900 0x7f42f8199c30 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.539+0000 7f42fd200640 1 -- 172.21.15.47:0/298813287 >> 172.21.15.47:0/298813287 conn(0x7f42f80fee90 msgr2=0x7f42f8107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.539+0000 7f42fd200640 1 -- 172.21.15.47:0/298813287 shutdown_connections 2024-03-21T11:33:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:12.539+0000 7f42fd200640 1 -- 172.21.15.47:0/298813287 wait complete. 2024-03-21T11:33:12.549 INFO:tasks.workunit.client.0.smithi047.stdout: 1 728019 8.12 MB/sec execute 356 sec latency 125.583 ms 2024-03-21T11:33:12.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:12 smithi102 ceph-mon[36748]: pgmap v696: 129 pgs: 129 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-03-21T11:33:12.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:12 smithi047 ceph-mon[31351]: pgmap v696: 129 pgs: 129 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-03-21T11:33:12.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:12 smithi100 ceph-mon[36245]: pgmap v696: 129 pgs: 129 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-03-21T11:33:13.549 INFO:tasks.workunit.client.0.smithi047.stdout: 1 728925 8.10 MB/sec execute 357 sec latency 12.757 ms 2024-03-21T11:33:14.550 INFO:tasks.workunit.client.0.smithi047.stdout: 1 730824 8.11 MB/sec execute 358 sec latency 67.698 ms 2024-03-21T11:33:14.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:14 smithi102 ceph-mon[36748]: pgmap v697: 129 pgs: 129 active+clean; 38 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 275 op/s 2024-03-21T11:33:14.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:14 smithi047 ceph-mon[31351]: pgmap v697: 129 pgs: 129 active+clean; 38 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 275 op/s 2024-03-21T11:33:14.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:14 smithi100 ceph-mon[36245]: pgmap v697: 129 pgs: 129 active+clean; 38 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 275 op/s 2024-03-21T11:33:15.549 INFO:tasks.workunit.client.0.smithi047.stdout: 1 731956 8.11 MB/sec execute 359 sec latency 12.787 ms 2024-03-21T11:33:16.550 INFO:tasks.workunit.client.0.smithi047.stdout: 1 733201 8.10 MB/sec execute 360 sec latency 13.817 ms 2024-03-21T11:33:16.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:16 smithi102 ceph-mon[36748]: pgmap v698: 129 pgs: 129 active+clean; 37 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-03-21T11:33:16.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:16 smithi047 ceph-mon[31351]: pgmap v698: 129 pgs: 129 active+clean; 37 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-03-21T11:33:16.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:16 smithi100 ceph-mon[36245]: pgmap v698: 129 pgs: 129 active+clean; 37 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-03-21T11:33:17.550 INFO:tasks.workunit.client.0.smithi047.stdout: 1 735029 8.11 MB/sec execute 361 sec latency 23.847 ms 2024-03-21T11:33:18.550 INFO:tasks.workunit.client.0.smithi047.stdout: 1 735823 8.09 MB/sec execute 362 sec latency 16.141 ms 2024-03-21T11:33:18.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:18 smithi102 ceph-mon[36748]: pgmap v699: 129 pgs: 129 active+clean; 37 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-03-21T11:33:18.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:18 smithi047 ceph-mon[31351]: pgmap v699: 129 pgs: 129 active+clean; 37 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-03-21T11:33:18.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:18 smithi100 ceph-mon[36245]: pgmap v699: 129 pgs: 129 active+clean; 37 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-03-21T11:33:19.550 INFO:tasks.workunit.client.0.smithi047.stdout: 1 737406 8.09 MB/sec execute 363 sec latency 65.250 ms 2024-03-21T11:33:20.550 INFO:tasks.workunit.client.0.smithi047.stdout: 1 738806 8.10 MB/sec execute 364 sec latency 14.708 ms 2024-03-21T11:33:20.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:20 smithi102 ceph-mon[36748]: pgmap v700: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2024-03-21T11:33:20.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:20 smithi047 ceph-mon[31351]: pgmap v700: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2024-03-21T11:33:20.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:20 smithi100 ceph-mon[36245]: pgmap v700: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2024-03-21T11:33:21.550 INFO:tasks.workunit.client.0.smithi047.stdout: 1 739567 8.08 MB/sec execute 365 sec latency 20.728 ms 2024-03-21T11:33:22.550 INFO:tasks.workunit.client.0.smithi047.stdout: 1 741370 8.09 MB/sec execute 366 sec latency 15.936 ms 2024-03-21T11:33:22.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:22 smithi102 ceph-mon[36748]: pgmap v701: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-03-21T11:33:22.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:22 smithi047 ceph-mon[31351]: pgmap v701: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-03-21T11:33:22.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:22 smithi100 ceph-mon[36245]: pgmap v701: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-03-21T11:33:23.550 INFO:tasks.workunit.client.0.smithi047.stdout: 1 742566 8.09 MB/sec execute 367 sec latency 18.420 ms 2024-03-21T11:33:24.550 INFO:tasks.workunit.client.0.smithi047.stdout: 1 743563 8.08 MB/sec execute 368 sec latency 169.130 ms 2024-03-21T11:33:24.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:24 smithi102 ceph-mon[36748]: pgmap v702: 129 pgs: 129 active+clean; 38 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 277 op/s 2024-03-21T11:33:24.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:24 smithi047 ceph-mon[31351]: pgmap v702: 129 pgs: 129 active+clean; 38 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 277 op/s 2024-03-21T11:33:24.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:24 smithi100 ceph-mon[36245]: pgmap v702: 129 pgs: 129 active+clean; 38 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 277 op/s 2024-03-21T11:33:25.551 INFO:tasks.workunit.client.0.smithi047.stdout: 1 745505 8.08 MB/sec execute 369 sec latency 21.745 ms 2024-03-21T11:33:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.689+0000 7f33b3200640 1 -- 172.21.15.47:0/3697697106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33b4101fa0 msgr2=0x7f33b4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.689+0000 7f33b3200640 1 --2- 172.21.15.47:0/3697697106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33b4101fa0 0x7f33b4108830 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009ca0 tx=0x7f33ac02c100 comp rx=0 tx=0).stop 2024-03-21T11:33:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.690+0000 7f33b3200640 1 -- 172.21.15.47:0/3697697106 shutdown_connections 2024-03-21T11:33:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.690+0000 7f33b3200640 1 --2- 172.21.15.47:0/3697697106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33b4101fa0 0x7f33b4108830 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.690+0000 7f33b3200640 1 --2- 172.21.15.47:0/3697697106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b41015e0 0x7f33b4101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.690+0000 7f33b3200640 1 --2- 172.21.15.47:0/3697697106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33b41003f0 0x7f33b4100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.690+0000 7f33b3200640 1 -- 172.21.15.47:0/3697697106 >> 172.21.15.47:0/3697697106 conn(0x7f33b40fb980 msgr2=0x7f33b40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.690+0000 7f33b3200640 1 -- 172.21.15.47:0/3697697106 shutdown_connections 2024-03-21T11:33:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.690+0000 7f33b3200640 1 -- 172.21.15.47:0/3697697106 wait complete. 2024-03-21T11:33:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.691+0000 7f33b3200640 1 Processor -- start 2024-03-21T11:33:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.691+0000 7f33b3200640 1 -- start start 2024-03-21T11:33:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.691+0000 7f33b3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33b41003f0 0x7f33b41999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b41015e0 0x7f33b4199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33b4101fa0 0x7f33b41a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b410c050 con 0x7f33b41015e0 2024-03-21T11:33:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b410bed0 con 0x7f33b4101fa0 2024-03-21T11:33:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b410c1d0 con 0x7f33b41003f0 2024-03-21T11:33:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33b41003f0 0x7f33b41999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b41015e0 0x7f33b4199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33b41003f0 0x7f33b41999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41510/0 (socket says 172.21.15.47:41510) 2024-03-21T11:33:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b1e00640 1 -- 172.21.15.47:0/3907398279 learned_addr learned my addr 172.21.15.47:0/3907398279 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b2800640 1 --2- 172.21.15.47:0/3907398279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33b4101fa0 0x7f33b41a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b1e00640 1 -- 172.21.15.47:0/3907398279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33b4101fa0 msgr2=0x7f33b41a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b1e00640 1 --2- 172.21.15.47:0/3907398279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33b4101fa0 0x7f33b41a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b1e00640 1 -- 172.21.15.47:0/3907398279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b41015e0 msgr2=0x7f33b4199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b1e00640 1 --2- 172.21.15.47:0/3907398279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b41015e0 0x7f33b4199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b1e00640 1 -- 172.21.15.47:0/3907398279 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b41a15e0 con 0x7f33b41003f0 2024-03-21T11:33:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.692+0000 7f33b2800640 1 --2- 172.21.15.47:0/3907398279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33b4101fa0 0x7f33b41a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.693+0000 7f33b1e00640 1 --2- 172.21.15.47:0/3907398279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33b41003f0 0x7f33b41999c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f33a400e9c0 tx=0x7f33a400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.693+0000 7f33aaa00640 1 -- 172.21.15.47:0/3907398279 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a400cd80 con 0x7f33b41003f0 2024-03-21T11:33:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.693+0000 7f33b3200640 1 -- 172.21.15.47:0/3907398279 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b41a18d0 con 0x7f33b41003f0 2024-03-21T11:33:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.693+0000 7f33aaa00640 1 -- 172.21.15.47:0/3907398279 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33a40035f0 con 0x7f33b41003f0 2024-03-21T11:33:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.693+0000 7f33aaa00640 1 -- 172.21.15.47:0/3907398279 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4010690 con 0x7f33b41003f0 2024-03-21T11:33:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.693+0000 7f33b3200640 1 -- 172.21.15.47:0/3907398279 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b41a1e20 con 0x7f33b41003f0 2024-03-21T11:33:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.696+0000 7f33aaa00640 1 -- 172.21.15.47:0/3907398279 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f33a40037a0 con 0x7f33b41003f0 2024-03-21T11:33:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.697+0000 7f33bb864740 1 Processor -- start 2024-03-21T11:33:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.697+0000 7f33bb864740 1 -- start start 2024-03-21T11:33:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.697+0000 7f33bb864740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb688810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.697+0000 7f33bb864740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb688de0 0x55f4eb68d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.697+0000 7f33bb864740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb68df50 0x55f4eb6903e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.697+0000 7f33bb864740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4eb64b3b0 con 0x55f4eb688de0 2024-03-21T11:33:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.697+0000 7f33bb864740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4eb5ed830 con 0x55f4eb6883f0 2024-03-21T11:33:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.697+0000 7f33bb864740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4eb5965d0 con 0x55f4eb68df50 2024-03-21T11:33:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.697+0000 7f33b2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb688810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.697+0000 7f33b1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb68df50 0x55f4eb6903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.697+0000 7f33b2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb688810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52144/0 (socket says 172.21.15.47:52144) 2024-03-21T11:33:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.697+0000 7f33b1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb688de0 0x55f4eb68d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.697+0000 7f33b2800640 1 -- 172.21.15.47:0/2156141916 learned_addr learned my addr 172.21.15.47:0/2156141916 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.698+0000 7f33b1400640 1 -- 172.21.15.47:0/2156141916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 msgr2=0x55f4eb688810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.698+0000 7f33b1400640 1 --2- 172.21.15.47:0/2156141916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb688810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.698+0000 7f33b1400640 1 -- 172.21.15.47:0/2156141916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb688de0 msgr2=0x55f4eb68d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.698+0000 7f33b1400640 1 --2- 172.21.15.47:0/2156141916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb688de0 0x55f4eb68d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.698+0000 7f33b1400640 1 -- 172.21.15.47:0/2156141916 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4eb5ae2c0 con 0x55f4eb68df50 2024-03-21T11:33:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.698+0000 7f33b1400640 1 --2- 172.21.15.47:0/2156141916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb68df50 0x55f4eb6903e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009bf0 tx=0x7f33a0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c5800d739dee385 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.698+0000 7f339f000640 1 -- 172.21.15.47:0/2156141916 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0004050 con 0x55f4eb68df50 2024-03-21T11:33:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.698+0000 7f339f000640 1 -- 172.21.15.47:0/2156141916 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33a0005500 con 0x55f4eb68df50 2024-03-21T11:33:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.698+0000 7f339f000640 1 -- 172.21.15.47:0/2156141916 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0004e40 con 0x55f4eb68df50 2024-03-21T11:33:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.699+0000 7f33aaa00640 1 --2- 172.21.15.47:0/3907398279 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33980a2f10 0x7f33980a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.699+0000 7f33aaa00640 1 -- 172.21.15.47:0/3907398279 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f33a40cfb70 con 0x7f33b41003f0 2024-03-21T11:33:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.699+0000 7f33b2800640 1 --2- 172.21.15.47:0/3907398279 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33980a2f10 0x7f33980a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.699+0000 7f33bb864740 1 -- 172.21.15.47:0/2156141916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb68df50 msgr2=0x55f4eb6903e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.699+0000 7f33bb864740 1 --2- 172.21.15.47:0/2156141916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb68df50 0x55f4eb6903e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009bf0 tx=0x7f33a0011810 comp rx=0 tx=0).stop 2024-03-21T11:33:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.699+0000 7f33bb864740 1 -- 172.21.15.47:0/2156141916 shutdown_connections 2024-03-21T11:33:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.699+0000 7f33bb864740 1 --2- 172.21.15.47:0/2156141916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb68df50 0x55f4eb6903e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.699+0000 7f33bb864740 1 --2- 172.21.15.47:0/2156141916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb688de0 0x55f4eb68d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.699+0000 7f33bb864740 1 --2- 172.21.15.47:0/2156141916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb688810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.699+0000 7f33bb864740 1 -- 172.21.15.47:0/2156141916 >> 172.21.15.47:0/2156141916 conn(0x55f4eb683960 msgr2=0x55f4eb683d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.699+0000 7f33bb864740 1 -- 172.21.15.47:0/2156141916 shutdown_connections 2024-03-21T11:33:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.699+0000 7f33bb864740 1 -- 172.21.15.47:0/2156141916 wait complete. 2024-03-21T11:33:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.700+0000 7f33b2800640 1 --2- 172.21.15.47:0/3907398279 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33980a2f10 0x7f33980a53d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f33ac006010 tx=0x7f33ac032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.700+0000 7f33bb864740 1 -- start start 2024-03-21T11:33:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.700+0000 7f33bb864740 1 Processor -- start 2024-03-21T11:33:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.700+0000 7f33bb864740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb692540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33bb864740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb688de0 0x55f4eb692a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33bb864740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb68df50 0x55f4eb699ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33bb864740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4eb5a9a20 con 0x55f4eb688de0 2024-03-21T11:33:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33bb864740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4eb5e98d0 con 0x55f4eb6883f0 2024-03-21T11:33:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33bb864740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4eb690f00 con 0x55f4eb68df50 2024-03-21T11:33:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33bb864740 10 client.0 _ll_register_callbacks cb 0x55f4eb5e61c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33b1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb692540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33b2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb68df50 0x55f4eb699ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33b1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb692540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52152/0 (socket says 172.21.15.47:52152) 2024-03-21T11:33:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33b1e00640 1 -- 172.21.15.47:0/982687911 learned_addr learned my addr 172.21.15.47:0/982687911 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33b1400640 1 --2- 172.21.15.47:0/982687911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb688de0 0x55f4eb692a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33b2800640 1 -- 172.21.15.47:0/982687911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 msgr2=0x55f4eb692540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33b2800640 1 --2- 172.21.15.47:0/982687911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb692540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33b2800640 1 -- 172.21.15.47:0/982687911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb688de0 msgr2=0x55f4eb692a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33b2800640 1 --2- 172.21.15.47:0/982687911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb688de0 0x55f4eb692a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33b2800640 1 -- 172.21.15.47:0/982687911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4eb302210 con 0x55f4eb68df50 2024-03-21T11:33:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.701+0000 7f33b1e00640 1 --2- 172.21.15.47:0/982687911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb692540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.702+0000 7f33b2800640 1 --2- 172.21.15.47:0/982687911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb68df50 0x55f4eb699ac0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f33ac032420 tx=0x7f33ac0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a534b86abb9e84ef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.702+0000 7f3393400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:33:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.702+0000 7f3393400640 1 -- 172.21.15.47:0/982687911 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac051020 con 0x55f4eb68df50 2024-03-21T11:33:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.702+0000 7f33bb864740 1 -- 172.21.15.47:0/982687911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33a0005500 con 0x55f4eb68df50 2024-03-21T11:33:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.702+0000 7f3393400640 1 -- 172.21.15.47:0/982687911 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f33ac0349f0 con 0x55f4eb68df50 2024-03-21T11:33:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.702+0000 7f3393400640 1 -- 172.21.15.47:0/982687911 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33ac05c050 con 0x55f4eb68df50 2024-03-21T11:33:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.702+0000 7f33bb864740 10 client.28843 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.702+0000 7f33bb864740 1 -- 172.21.15.47:0/982687911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f4eb3ce620 con 0x55f4eb68df50 2024-03-21T11:33:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.702+0000 7f3393400640 1 -- 172.21.15.47:0/982687911 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33a0005500 con 0x55f4eb68df50 2024-03-21T11:33:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.703+0000 7f3393400640 1 -- 172.21.15.47:0/982687911 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f33ac040430 con 0x55f4eb68df50 2024-03-21T11:33:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.703+0000 7f33bb864740 10 client.28843 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.703+0000 7f33bb864740 10 client.28843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.703+0000 7f33bb864740 20 client.28843 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.703+0000 7f33bb864740 1 --2- 172.21.15.47:0/982687911 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f4eb69e410 0x55f4eb6a0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.703+0000 7f33bb864740 4 client.28843 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:33:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.703+0000 7f33bb864740 1 -- 172.21.15.47:0/982687911 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4eb5fad70 con 0x55f4eb69e410 2024-03-21T11:33:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.703+0000 7f33b1e00640 1 --2- 172.21.15.47:0/982687911 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f4eb69e410 0x55f4eb6a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.704+0000 7f33b1e00640 1 --2- 172.21.15.47:0/982687911 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f4eb69e410 0x55f4eb6a0800 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2d4bc068dbb443 server_cookie=c9697d2cb38f5373 in_seq=0 out_seq=0 2024-03-21T11:33:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.704+0000 7f3393400640 10 client.28843 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f3393400640 1 -- 172.21.15.47:0/982687911 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f4eb5fad70 con 0x55f4eb69e410 2024-03-21T11:33:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f3393400640 10 client.28843 handle_command_reply: tid=0 2024-03-21T11:33:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f33bb864740 1 client.28843 shutdown 2024-03-21T11:33:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f33bb864740 1 -- 172.21.15.47:0/982687911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb68df50 msgr2=0x55f4eb699ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f33bb864740 1 --2- 172.21.15.47:0/982687911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb68df50 0x55f4eb699ac0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f33ac032420 tx=0x7f33ac0095c0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f33bb864740 1 -- 172.21.15.47:0/982687911 shutdown_connections 2024-03-21T11:33:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f33bb864740 1 --2- 172.21.15.47:0/982687911 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f4eb69e410 0x55f4eb6a0800 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f33bb864740 1 --2- 172.21.15.47:0/982687911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb68df50 0x55f4eb699ac0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f33bb864740 1 --2- 172.21.15.47:0/982687911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb688de0 0x55f4eb692a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f33bb864740 1 --2- 172.21.15.47:0/982687911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb692540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f33bb864740 1 -- 172.21.15.47:0/982687911 >> 172.21.15.47:0/982687911 conn(0x55f4eb683960 msgr2=0x55f4eb682b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f3393400640 0 client.28843 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f33bb864740 1 -- 172.21.15.47:0/982687911 shutdown_connections 2024-03-21T11:33:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f33bb864740 1 -- 172.21.15.47:0/982687911 wait complete. 2024-03-21T11:33:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.705+0000 7f33bb864740 20 client.28843 trim_cache size 0 max 16384 2024-03-21T11:33:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.722+0000 7f33bb864740 1 Processor -- start 2024-03-21T11:33:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.723+0000 7f33bb864740 1 -- start start 2024-03-21T11:33:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.723+0000 7f33bb864740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb6883f0 0x55f4eb691300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.723+0000 7f33bb864740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb688de0 0x55f4eb6999e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.723+0000 7f33bb864740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb68df50 0x55f4eb67e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.723+0000 7f33bb864740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41ac0a0 con 0x55f4eb6883f0 2024-03-21T11:33:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.723+0000 7f33bb864740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41ada20 con 0x55f4eb68df50 2024-03-21T11:33:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.723+0000 7f33bb864740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41ac700 con 0x55f4eb688de0 2024-03-21T11:33:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.723+0000 7f33b1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb688de0 0x55f4eb6999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.723+0000 7f33b2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb68df50 0x55f4eb67e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.723+0000 7f33b1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb688de0 0x55f4eb6999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41530/0 (socket says 172.21.15.47:41530) 2024-03-21T11:33:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.723+0000 7f33b2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb68df50 0x55f4eb67e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52154/0 (socket says 172.21.15.47:52154) 2024-03-21T11:33:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.723+0000 7f33b1400640 1 -- 172.21.15.47:0/2784557432 learned_addr learned my addr 172.21.15.47:0/2784557432 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.723+0000 7f33b1e00640 1 --2- 172.21.15.47:0/2784557432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb6883f0 0x55f4eb691300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.724+0000 7f33b1400640 1 -- 172.21.15.47:0/2784557432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb68df50 msgr2=0x55f4eb67e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.724+0000 7f33b1400640 1 --2- 172.21.15.47:0/2784557432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb68df50 0x55f4eb67e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.724+0000 7f33b1400640 1 -- 172.21.15.47:0/2784557432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb6883f0 msgr2=0x55f4eb691300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.724+0000 7f33b1400640 1 --2- 172.21.15.47:0/2784557432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb6883f0 0x55f4eb691300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.724+0000 7f33b1400640 1 -- 172.21.15.47:0/2784557432 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4eb6a0e60 con 0x55f4eb688de0 2024-03-21T11:33:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.724+0000 7f33b2800640 1 --2- 172.21.15.47:0/2784557432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb68df50 0x55f4eb67e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.724+0000 7f33b1e00640 1 --2- 172.21.15.47:0/2784557432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb6883f0 0x55f4eb691300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:33:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.724+0000 7f33b1400640 1 --2- 172.21.15.47:0/2784557432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb688de0 0x55f4eb6999e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009250 tx=0x7f33a0009970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8378da9dd8d01c6d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.724+0000 7f339e600640 1 -- 172.21.15.47:0/2784557432 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0004250 con 0x55f4eb688de0 2024-03-21T11:33:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.724+0000 7f339e600640 1 -- 172.21.15.47:0/2784557432 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33a0009e60 con 0x55f4eb688de0 2024-03-21T11:33:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.724+0000 7f339e600640 1 -- 172.21.15.47:0/2784557432 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0004dd0 con 0x55f4eb688de0 2024-03-21T11:33:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.725+0000 7f33bb864740 1 -- 172.21.15.47:0/2784557432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb688de0 msgr2=0x55f4eb6999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.725+0000 7f33bb864740 1 --2- 172.21.15.47:0/2784557432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb688de0 0x55f4eb6999e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009250 tx=0x7f33a0009970 comp rx=0 tx=0).stop 2024-03-21T11:33:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.725+0000 7f33bb864740 1 -- 172.21.15.47:0/2784557432 shutdown_connections 2024-03-21T11:33:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.725+0000 7f33bb864740 1 --2- 172.21.15.47:0/2784557432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb68df50 0x55f4eb67e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.725+0000 7f33bb864740 1 --2- 172.21.15.47:0/2784557432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb688de0 0x55f4eb6999e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.725+0000 7f33bb864740 1 --2- 172.21.15.47:0/2784557432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb6883f0 0x55f4eb691300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.725+0000 7f33bb864740 1 -- 172.21.15.47:0/2784557432 >> 172.21.15.47:0/2784557432 conn(0x55f4eb683960 msgr2=0x55f4eb68d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.725+0000 7f33bb864740 1 -- 172.21.15.47:0/2784557432 shutdown_connections 2024-03-21T11:33:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.725+0000 7f33bb864740 1 -- 172.21.15.47:0/2784557432 wait complete. 2024-03-21T11:33:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.726+0000 7f33bb864740 1 -- start start 2024-03-21T11:33:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.726+0000 7f33bb864740 1 Processor -- start 2024-03-21T11:33:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.726+0000 7f33bb864740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb696880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.726+0000 7f33bb864740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb688de0 0x55f4eb696dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.726+0000 7f33bb864740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb68df50 0x55f4eb6f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.726+0000 7f33bb864740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41ad3c0 con 0x55f4eb68df50 2024-03-21T11:33:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.726+0000 7f33bb864740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41acd60 con 0x55f4eb6883f0 2024-03-21T11:33:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.726+0000 7f33bb864740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41aa260 con 0x55f4eb688de0 2024-03-21T11:33:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.726+0000 7f33bb864740 10 client.0 _ll_register_callbacks cb 0x7f33ac044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.726+0000 7f33b1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb696880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f33b1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb688de0 0x55f4eb696dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f33b1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb696880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52164/0 (socket says 172.21.15.47:52164) 2024-03-21T11:33:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f33b1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb688de0 0x55f4eb696dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41542/0 (socket says 172.21.15.47:41542) 2024-03-21T11:33:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f33b1e00640 1 -- 172.21.15.47:0/2144176129 learned_addr learned my addr 172.21.15.47:0/2144176129 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f33b2800640 1 --2- 172.21.15.47:0/2144176129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb68df50 0x55f4eb6f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f33b1e00640 1 -- 172.21.15.47:0/2144176129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb688de0 msgr2=0x55f4eb696dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f33b1e00640 1 --2- 172.21.15.47:0/2144176129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb688de0 0x55f4eb696dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f33b1e00640 1 -- 172.21.15.47:0/2144176129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb68df50 msgr2=0x55f4eb6f7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f33b1e00640 1 --2- 172.21.15.47:0/2144176129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb68df50 0x55f4eb6f7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f33b1e00640 1 -- 172.21.15.47:0/2144176129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4eb6f81c0 con 0x55f4eb6883f0 2024-03-21T11:33:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f33b1400640 1 --2- 172.21.15.47:0/2144176129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb688de0 0x55f4eb696dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f33b1e00640 1 --2- 172.21.15.47:0/2144176129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb696880 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f33a4098310 tx=0x7f33a4003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72de0a90d398fefc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f3393400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:33:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f3393400640 1 -- 172.21.15.47:0/2144176129 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a40d7040 con 0x55f4eb6883f0 2024-03-21T11:33:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f33bb864740 1 -- 172.21.15.47:0/2144176129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33a0009e60 con 0x55f4eb6883f0 2024-03-21T11:33:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.727+0000 7f3393400640 1 -- 172.21.15.47:0/2144176129 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f33a40cd390 con 0x55f4eb6883f0 2024-03-21T11:33:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.728+0000 7f3393400640 1 -- 172.21.15.47:0/2144176129 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33a409b810 con 0x55f4eb6883f0 2024-03-21T11:33:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.728+0000 7f33bb864740 10 client.28832 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.728+0000 7f33bb864740 1 -- 172.21.15.47:0/2144176129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f4eb6f94a0 con 0x55f4eb6883f0 2024-03-21T11:33:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.728+0000 7f3393400640 1 -- 172.21.15.47:0/2144176129 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33a0009e60 con 0x55f4eb6883f0 2024-03-21T11:33:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.728+0000 7f3393400640 1 -- 172.21.15.47:0/2144176129 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f33ac032d10 con 0x55f4eb6883f0 2024-03-21T11:33:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.728+0000 7f33bb864740 10 client.28832 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.728+0000 7f33bb864740 10 client.28832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.728+0000 7f33bb864740 20 client.28832 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.728+0000 7f33bb864740 1 --2- 172.21.15.47:0/2144176129 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f4eb6f9d60 0x55f4eb6fc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.728+0000 7f33bb864740 4 client.28832 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:33:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.728+0000 7f33bb864740 1 -- 172.21.15.47:0/2144176129 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f4eb5f50c0 con 0x55f4eb6f9d60 2024-03-21T11:33:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.729+0000 7f33b1400640 1 --2- 172.21.15.47:0/2144176129 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f4eb6f9d60 0x55f4eb6fc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.729+0000 7f33b1400640 1 --2- 172.21.15.47:0/2144176129 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f4eb6f9d60 0x55f4eb6fc150 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=392761fb2c86c320 server_cookie=811b03a3a4ae07bc in_seq=0 out_seq=0 2024-03-21T11:33:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.729+0000 7f3393400640 10 client.28832 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.975+0000 7f3393400640 1 -- 172.21.15.47:0/2144176129 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f33a0031040 con 0x55f4eb6f9d60 2024-03-21T11:33:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.975+0000 7f3393400640 10 client.28832 handle_command_reply: tid=0 2024-03-21T11:33:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.975+0000 7f33bb864740 1 client.28832 shutdown 2024-03-21T11:33:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.976+0000 7f33bb864740 1 -- 172.21.15.47:0/2144176129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 msgr2=0x55f4eb696880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.976+0000 7f33bb864740 1 --2- 172.21.15.47:0/2144176129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb696880 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f33a4098310 tx=0x7f33a4003a90 comp rx=0 tx=0).stop 2024-03-21T11:33:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.976+0000 7f33bb864740 1 -- 172.21.15.47:0/2144176129 shutdown_connections 2024-03-21T11:33:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.976+0000 7f33bb864740 1 --2- 172.21.15.47:0/2144176129 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f4eb6f9d60 0x55f4eb6fc150 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.976+0000 7f33bb864740 1 --2- 172.21.15.47:0/2144176129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f4eb68df50 0x55f4eb6f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.976+0000 7f33bb864740 1 --2- 172.21.15.47:0/2144176129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4eb688de0 0x55f4eb696dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.976+0000 7f3393400640 0 client.28832 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.976+0000 7f33bb864740 1 --2- 172.21.15.47:0/2144176129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f4eb6883f0 0x55f4eb696880 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.976+0000 7f33bb864740 1 -- 172.21.15.47:0/2144176129 >> 172.21.15.47:0/2144176129 conn(0x55f4eb683960 msgr2=0x55f4eb691300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.976+0000 7f33bb864740 1 -- 172.21.15.47:0/2144176129 shutdown_connections 2024-03-21T11:33:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.976+0000 7f33bb864740 1 -- 172.21.15.47:0/2144176129 wait complete. 2024-03-21T11:33:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.976+0000 7f33bb864740 20 client.28832 trim_cache size 0 max 16384 2024-03-21T11:33:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:33:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:33:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:33:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:33:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.977+0000 7f33b3200640 1 -- 172.21.15.47:0/3907398279 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33980a2f10 msgr2=0x7f33980a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.977+0000 7f33b3200640 1 --2- 172.21.15.47:0/3907398279 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33980a2f10 0x7f33980a53d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f33ac006010 tx=0x7f33ac032590 comp rx=0 tx=0).stop 2024-03-21T11:33:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.977+0000 7f33b3200640 1 -- 172.21.15.47:0/3907398279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33b41003f0 msgr2=0x7f33b41999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.977+0000 7f33b3200640 1 --2- 172.21.15.47:0/3907398279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33b41003f0 0x7f33b41999c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f33a400e9c0 tx=0x7f33a400ee80 comp rx=0 tx=0).stop 2024-03-21T11:33:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.977+0000 7f33b3200640 1 -- 172.21.15.47:0/3907398279 shutdown_connections 2024-03-21T11:33:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.977+0000 7f33b3200640 1 --2- 172.21.15.47:0/3907398279 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33980a2f10 0x7f33980a53d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.977+0000 7f33b3200640 1 --2- 172.21.15.47:0/3907398279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33b4101fa0 0x7f33b41a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.977+0000 7f33b3200640 1 --2- 172.21.15.47:0/3907398279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b41015e0 0x7f33b4199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.977+0000 7f33b3200640 1 --2- 172.21.15.47:0/3907398279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33b41003f0 0x7f33b41999c0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.977+0000 7f33b3200640 1 -- 172.21.15.47:0/3907398279 >> 172.21.15.47:0/3907398279 conn(0x7f33b40fb980 msgr2=0x7f33b40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.977+0000 7f33b3200640 1 -- 172.21.15.47:0/3907398279 shutdown_connections 2024-03-21T11:33:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:25.977+0000 7f33b3200640 1 -- 172.21.15.47:0/3907398279 wait complete. 2024-03-21T11:33:26.551 INFO:tasks.workunit.client.0.smithi047.stdout: 1 746374 8.07 MB/sec execute 370 sec latency 173.480 ms 2024-03-21T11:33:26.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:33:26.614 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:33:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:26 smithi102 ceph-mon[36748]: pgmap v703: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-03-21T11:33:26.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:26 smithi047 ceph-mon[31351]: pgmap v703: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-03-21T11:33:26.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.862+0000 7f454da00640 1 -- 172.21.15.47:0/1647530216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4548107cc0 msgr2=0x7f45481080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:26.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.862+0000 7f454da00640 1 --2- 172.21.15.47:0/1647530216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4548107cc0 0x7f45481080e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4538009900 tx=0x7f453802bc30 comp rx=0 tx=0).stop 2024-03-21T11:33:26.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.862+0000 7f454da00640 1 -- 172.21.15.47:0/1647530216 shutdown_connections 2024-03-21T11:33:26.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.862+0000 7f454da00640 1 --2- 172.21.15.47:0/1647530216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4548109870 0x7f4548110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.862+0000 7f454da00640 1 --2- 172.21.15.47:0/1647530216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4548108eb0 0x7f4548109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.862+0000 7f454da00640 1 --2- 172.21.15.47:0/1647530216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4548107cc0 0x7f45481080e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.862+0000 7f454da00640 1 -- 172.21.15.47:0/1647530216 >> 172.21.15.47:0/1647530216 conn(0x7f4548076c60 msgr2=0x7f45480790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:26.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.862+0000 7f454da00640 1 -- 172.21.15.47:0/1647530216 shutdown_connections 2024-03-21T11:33:26.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.862+0000 7f454da00640 1 -- 172.21.15.47:0/1647530216 wait complete. 2024-03-21T11:33:26.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.863+0000 7f454da00640 1 Processor -- start 2024-03-21T11:33:26.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.863+0000 7f454da00640 1 -- start start 2024-03-21T11:33:26.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.863+0000 7f454da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4548107cc0 0x7f454819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f454da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4548108eb0 0x7f454819e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f454da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4548109870 0x7f45481a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f454da00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4548113960 con 0x7f4548108eb0 2024-03-21T11:33:26.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f454da00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45481137e0 con 0x7f4548107cc0 2024-03-21T11:33:26.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f454da00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4548113ae0 con 0x7f4548109870 2024-03-21T11:33:26.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f4546a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4548107cc0 0x7f454819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f4546a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4548107cc0 0x7f454819dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52194/0 (socket says 172.21.15.47:52194) 2024-03-21T11:33:26.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f4546a00640 1 -- 172.21.15.47:0/1903282089 learned_addr learned my addr 172.21.15.47:0/1903282089 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:26.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f4547400640 1 --2- 172.21.15.47:0/1903282089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4548109870 0x7f45481a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f4546000640 1 --2- 172.21.15.47:0/1903282089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4548108eb0 0x7f454819e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f4546a00640 1 -- 172.21.15.47:0/1903282089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4548109870 msgr2=0x7f45481a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:26.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f4546a00640 1 --2- 172.21.15.47:0/1903282089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4548109870 0x7f45481a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f4546a00640 1 -- 172.21.15.47:0/1903282089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4548108eb0 msgr2=0x7f454819e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:26.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f4546a00640 1 --2- 172.21.15.47:0/1903282089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4548108eb0 0x7f454819e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.864+0000 7f4546a00640 1 -- 172.21.15.47:0/1903282089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45481a5bd0 con 0x7f4548107cc0 2024-03-21T11:33:26.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.865+0000 7f4546a00640 1 --2- 172.21.15.47:0/1903282089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4548107cc0 0x7f454819dfb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4538000c00 tx=0x7f453802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:26.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.865+0000 7f453f400640 1 -- 172.21.15.47:0/1903282089 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538043020 con 0x7f4548107cc0 2024-03-21T11:33:26.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.865+0000 7f453f400640 1 -- 172.21.15.47:0/1903282089 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45380054e0 con 0x7f4548107cc0 2024-03-21T11:33:26.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.865+0000 7f454da00640 1 -- 172.21.15.47:0/1903282089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45481a5e60 con 0x7f4548107cc0 2024-03-21T11:33:26.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.865+0000 7f453f400640 1 -- 172.21.15.47:0/1903282089 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538003c40 con 0x7f4548107cc0 2024-03-21T11:33:26.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.865+0000 7f454da00640 1 -- 172.21.15.47:0/1903282089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45481a62c0 con 0x7f4548107cc0 2024-03-21T11:33:26.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f455065b740 1 Processor -- start 2024-03-21T11:33:26.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f453f400640 1 -- 172.21.15.47:0/1903282089 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4538046020 con 0x7f4548107cc0 2024-03-21T11:33:26.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f455065b740 1 -- start start 2024-03-21T11:33:26.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f455065b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 0x562ef80aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f455065b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f455065b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80af9e0 0x562ef80b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f455065b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef800f250 con 0x562ef80a9e80 2024-03-21T11:33:26.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f455065b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef7fb7a30 con 0x562ef80aa870 2024-03-21T11:33:26.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f455065b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef800b2f0 con 0x562ef80af9e0 2024-03-21T11:33:26.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f4546a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f4546000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80af9e0 0x562ef80b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f4547400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 0x562ef80aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f4546000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80af9e0 0x562ef80b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41566/0 (socket says 172.21.15.47:41566) 2024-03-21T11:33:26.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f4546a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80af370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52208/0 (socket says 172.21.15.47:52208) 2024-03-21T11:33:26.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.869+0000 7f4546000640 1 -- 172.21.15.47:0/4055035589 learned_addr learned my addr 172.21.15.47:0/4055035589 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:26.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.870+0000 7f4546000640 1 -- 172.21.15.47:0/4055035589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 msgr2=0x562ef80af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:26.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.870+0000 7f4546000640 1 --2- 172.21.15.47:0/4055035589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.870+0000 7f4546000640 1 -- 172.21.15.47:0/4055035589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 msgr2=0x562ef80aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:26.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.870+0000 7f4546000640 1 --2- 172.21.15.47:0/4055035589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 0x562ef80aa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.870+0000 7f4546000640 1 -- 172.21.15.47:0/4055035589 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ef7fa2690 con 0x562ef80af9e0 2024-03-21T11:33:26.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.870+0000 7f453f400640 1 --2- 172.21.15.47:0/1903282089 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f452c0a2f20 0x7f452c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.870+0000 7f4546000640 1 --2- 172.21.15.47:0/4055035589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80af9e0 0x562ef80b1e70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4534009c90 tx=0x7f4534011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49822c6d5c5e3227 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:26.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.870+0000 7f453f400640 1 -- 172.21.15.47:0/1903282089 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f45380f17c0 con 0x7f4548107cc0 2024-03-21T11:33:26.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.871+0000 7f4533e00640 1 -- 172.21.15.47:0/4055035589 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453401e070 con 0x562ef80af9e0 2024-03-21T11:33:26.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.871+0000 7f4547400640 1 --2- 172.21.15.47:0/1903282089 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f452c0a2f20 0x7f452c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.871+0000 7f4533e00640 1 -- 172.21.15.47:0/4055035589 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4534019b10 con 0x562ef80af9e0 2024-03-21T11:33:26.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.871+0000 7f4533e00640 1 -- 172.21.15.47:0/4055035589 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453400eac0 con 0x562ef80af9e0 2024-03-21T11:33:26.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.871+0000 7f455065b740 1 -- 172.21.15.47:0/4055035589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80af9e0 msgr2=0x562ef80b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:26.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.871+0000 7f455065b740 1 --2- 172.21.15.47:0/4055035589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80af9e0 0x562ef80b1e70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4534009c90 tx=0x7f4534011810 comp rx=0 tx=0).stop 2024-03-21T11:33:26.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.871+0000 7f4547400640 1 --2- 172.21.15.47:0/1903282089 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f452c0a2f20 0x7f452c0a53e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x562ef80ab0d0 tx=0x7f4540009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:26.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.872+0000 7f455065b740 1 -- 172.21.15.47:0/4055035589 shutdown_connections 2024-03-21T11:33:26.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.872+0000 7f455065b740 1 --2- 172.21.15.47:0/4055035589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80af9e0 0x562ef80b1e70 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.872+0000 7f455065b740 1 --2- 172.21.15.47:0/4055035589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.872+0000 7f455065b740 1 --2- 172.21.15.47:0/4055035589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 0x562ef80aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.872+0000 7f455065b740 1 -- 172.21.15.47:0/4055035589 >> 172.21.15.47:0/4055035589 conn(0x562ef8072200 msgr2=0x562ef80a57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:26.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.872+0000 7f455065b740 1 -- 172.21.15.47:0/4055035589 shutdown_connections 2024-03-21T11:33:26.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.872+0000 7f455065b740 1 -- 172.21.15.47:0/4055035589 wait complete. 2024-03-21T11:33:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.872+0000 7f455065b740 1 -- start start 2024-03-21T11:33:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.873+0000 7f455065b740 1 Processor -- start 2024-03-21T11:33:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.873+0000 7f455065b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 0x562ef80b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.873+0000 7f455065b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80aa870 0x562ef80b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.873+0000 7f455065b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80af9e0 0x562ef80bb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.873+0000 7f455065b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef80b2990 con 0x562ef80a9e80 2024-03-21T11:33:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.873+0000 7f455065b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef7fcae80 con 0x562ef80af9e0 2024-03-21T11:33:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.873+0000 7f455065b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef80b2b10 con 0x562ef80aa870 2024-03-21T11:33:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.873+0000 7f455065b740 10 client.0 _ll_register_callbacks cb 0x562ef7fdd130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.873+0000 7f4546000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80aa870 0x562ef80b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.873+0000 7f4546000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80aa870 0x562ef80b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41576/0 (socket says 172.21.15.47:41576) 2024-03-21T11:33:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.873+0000 7f4547400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80af9e0 0x562ef80bb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.873+0000 7f4546000640 1 -- 172.21.15.47:0/965152347 learned_addr learned my addr 172.21.15.47:0/965152347 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.873+0000 7f4546a00640 1 --2- 172.21.15.47:0/965152347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 0x562ef80b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.874+0000 7f4546000640 1 -- 172.21.15.47:0/965152347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80af9e0 msgr2=0x562ef80bb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.874+0000 7f4546000640 1 --2- 172.21.15.47:0/965152347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80af9e0 0x562ef80bb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.874+0000 7f4546000640 1 -- 172.21.15.47:0/965152347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 msgr2=0x562ef80b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.874+0000 7f4546000640 1 --2- 172.21.15.47:0/965152347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 0x562ef80b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.874+0000 7f4546000640 1 -- 172.21.15.47:0/965152347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ef7ded7d0 con 0x562ef80aa870 2024-03-21T11:33:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.874+0000 7f4546000640 1 --2- 172.21.15.47:0/965152347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80aa870 0x562ef80b4360 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4534019a20 tx=0x7f4534019a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d43f138ca13ff650 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:26.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.874+0000 7f4530c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:33:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.874+0000 7f4530c00640 1 -- 172.21.15.47:0/965152347 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453401e070 con 0x562ef80aa870 2024-03-21T11:33:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.874+0000 7f4530c00640 1 -- 172.21.15.47:0/965152347 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4534004230 con 0x562ef80aa870 2024-03-21T11:33:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.875+0000 7f455065b740 1 -- 172.21.15.47:0/965152347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4534019b10 con 0x562ef80aa870 2024-03-21T11:33:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.875+0000 7f4530c00640 1 -- 172.21.15.47:0/965152347 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f453400e3b0 con 0x562ef80aa870 2024-03-21T11:33:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.875+0000 7f455065b740 10 client.28861 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.875+0000 7f455065b740 1 -- 172.21.15.47:0/965152347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562ef7d233f0 con 0x562ef80aa870 2024-03-21T11:33:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.875+0000 7f4530c00640 1 -- 172.21.15.47:0/965152347 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4534019b10 con 0x562ef80aa870 2024-03-21T11:33:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.875+0000 7f4530c00640 1 -- 172.21.15.47:0/965152347 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f453400eb20 con 0x562ef80aa870 2024-03-21T11:33:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.875+0000 7f455065b740 10 client.28861 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.875+0000 7f455065b740 10 client.28861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:26.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.875+0000 7f455065b740 20 client.28861 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.876+0000 7f455065b740 1 --2- 172.21.15.47:0/965152347 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ef80bf990 0x562ef80c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.876+0000 7f455065b740 4 client.28861 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:33:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.876+0000 7f455065b740 1 -- 172.21.15.47:0/965152347 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ef801c790 con 0x562ef80bf990 2024-03-21T11:33:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.876+0000 7f4546a00640 1 --2- 172.21.15.47:0/965152347 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ef80bf990 0x562ef80c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.876+0000 7f4546a00640 1 --2- 172.21.15.47:0/965152347 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ef80bf990 0x562ef80c1de0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66d0002628352eb0 server_cookie=7313bb2adc7cd9b7 in_seq=0 out_seq=0 2024-03-21T11:33:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.876+0000 7f4530c00640 10 client.28861 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.877+0000 7f4530c00640 1 -- 172.21.15.47:0/965152347 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562ef801c790 con 0x562ef80bf990 2024-03-21T11:33:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.877+0000 7f4530c00640 10 client.28861 handle_command_reply: tid=0 2024-03-21T11:33:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.877+0000 7f455065b740 1 client.28861 shutdown 2024-03-21T11:33:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.878+0000 7f455065b740 1 -- 172.21.15.47:0/965152347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80aa870 msgr2=0x562ef80b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.878+0000 7f455065b740 1 --2- 172.21.15.47:0/965152347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80aa870 0x562ef80b4360 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4534019a20 tx=0x7f4534019a50 comp rx=0 tx=0).stop 2024-03-21T11:33:26.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.878+0000 7f455065b740 1 -- 172.21.15.47:0/965152347 shutdown_connections 2024-03-21T11:33:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.878+0000 7f455065b740 1 --2- 172.21.15.47:0/965152347 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ef80bf990 0x562ef80c1de0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.878+0000 7f455065b740 1 --2- 172.21.15.47:0/965152347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80af9e0 0x562ef80bb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.878+0000 7f455065b740 1 --2- 172.21.15.47:0/965152347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80aa870 0x562ef80b4360 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.878+0000 7f4530c00640 0 client.28861 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.878+0000 7f455065b740 1 --2- 172.21.15.47:0/965152347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 0x562ef80b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.878+0000 7f455065b740 1 -- 172.21.15.47:0/965152347 >> 172.21.15.47:0/965152347 conn(0x562ef8072200 msgr2=0x562ef80a4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.878+0000 7f455065b740 1 -- 172.21.15.47:0/965152347 shutdown_connections 2024-03-21T11:33:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.878+0000 7f455065b740 1 -- 172.21.15.47:0/965152347 wait complete. 2024-03-21T11:33:26.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.878+0000 7f455065b740 20 client.28861 trim_cache size 0 max 16384 2024-03-21T11:33:26.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:26 smithi100 ceph-mon[36245]: pgmap v703: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-03-21T11:33:26.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.896+0000 7f455065b740 1 Processor -- start 2024-03-21T11:33:26.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f455065b740 1 -- start start 2024-03-21T11:33:26.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f455065b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80a9e80 0x562ef80b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f455065b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80bb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f455065b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80af9e0 0x562ef80a4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f455065b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45481b0640 con 0x562ef80af9e0 2024-03-21T11:33:26.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f455065b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45481b1fc0 con 0x562ef80aa870 2024-03-21T11:33:26.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f455065b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45481b0ca0 con 0x562ef80a9e80 2024-03-21T11:33:26.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f4546a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80a9e80 0x562ef80b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f4546a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80a9e80 0x562ef80b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41584/0 (socket says 172.21.15.47:41584) 2024-03-21T11:33:26.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f4546a00640 1 -- 172.21.15.47:0/1215850032 learned_addr learned my addr 172.21.15.47:0/1215850032 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:26.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f4546000640 1 --2- 172.21.15.47:0/1215850032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80bb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f4546a00640 1 -- 172.21.15.47:0/1215850032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 msgr2=0x562ef80bb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:26.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f4546a00640 1 --2- 172.21.15.47:0/1215850032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80bb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f4546a00640 1 -- 172.21.15.47:0/1215850032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80af9e0 msgr2=0x562ef80a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:26.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f4547400640 1 --2- 172.21.15.47:0/1215850032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80af9e0 0x562ef80a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f4546a00640 1 --2- 172.21.15.47:0/1215850032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80af9e0 0x562ef80a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.897+0000 7f4546a00640 1 -- 172.21.15.47:0/1215850032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ef7ce30f0 con 0x562ef80a9e80 2024-03-21T11:33:26.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.898+0000 7f4547400640 1 --2- 172.21.15.47:0/1215850032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80af9e0 0x562ef80a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:33:26.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.898+0000 7f4546a00640 1 --2- 172.21.15.47:0/1215850032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80a9e80 0x562ef80b3e20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4538006010 tx=0x7f45380f3040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8189e22500c2780 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:26.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.898+0000 7f4533400640 1 -- 172.21.15.47:0/1215850032 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45380f7020 con 0x562ef80a9e80 2024-03-21T11:33:26.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.898+0000 7f4533400640 1 -- 172.21.15.47:0/1215850032 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4538004250 con 0x562ef80a9e80 2024-03-21T11:33:26.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.898+0000 7f4533400640 1 -- 172.21.15.47:0/1215850032 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45380be200 con 0x562ef80a9e80 2024-03-21T11:33:26.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.899+0000 7f455065b740 1 -- 172.21.15.47:0/1215850032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80a9e80 msgr2=0x562ef80b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:26.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.899+0000 7f455065b740 1 --2- 172.21.15.47:0/1215850032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80a9e80 0x562ef80b3e20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4538006010 tx=0x7f45380f3040 comp rx=0 tx=0).stop 2024-03-21T11:33:26.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.899+0000 7f455065b740 1 -- 172.21.15.47:0/1215850032 shutdown_connections 2024-03-21T11:33:26.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.899+0000 7f455065b740 1 --2- 172.21.15.47:0/1215850032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80af9e0 0x562ef80a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.899+0000 7f455065b740 1 --2- 172.21.15.47:0/1215850032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80bb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.899+0000 7f455065b740 1 --2- 172.21.15.47:0/1215850032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80a9e80 0x562ef80b3e20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.899+0000 7f455065b740 1 -- 172.21.15.47:0/1215850032 >> 172.21.15.47:0/1215850032 conn(0x562ef8072200 msgr2=0x562ef80a4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:26.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.899+0000 7f455065b740 1 -- 172.21.15.47:0/1215850032 shutdown_connections 2024-03-21T11:33:26.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.899+0000 7f455065b740 1 -- 172.21.15.47:0/1215850032 wait complete. 2024-03-21T11:33:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.899+0000 7f455065b740 1 -- start start 2024-03-21T11:33:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.900+0000 7f455065b740 1 Processor -- start 2024-03-21T11:33:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.900+0000 7f455065b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 0x562ef80b91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.900+0000 7f455065b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.900+0000 7f455065b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80af9e0 0x562ef80ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.900+0000 7f455065b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45481b1960 con 0x562ef80a9e80 2024-03-21T11:33:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.900+0000 7f455065b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45481b1300 con 0x562ef80aa870 2024-03-21T11:33:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.900+0000 7f455065b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45481ae800 con 0x562ef80af9e0 2024-03-21T11:33:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.900+0000 7f455065b740 10 client.0 _ll_register_callbacks cb 0x7f453401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.900+0000 7f4546a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 0x562ef80b91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.900+0000 7f4546000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.900+0000 7f4547400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80af9e0 0x562ef80ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.901+0000 7f4546000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80b9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52230/0 (socket says 172.21.15.47:52230) 2024-03-21T11:33:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.901+0000 7f4547400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80af9e0 0x562ef80ba6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41592/0 (socket says 172.21.15.47:41592) 2024-03-21T11:33:26.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.901+0000 7f4546000640 1 -- 172.21.15.47:0/842189080 learned_addr learned my addr 172.21.15.47:0/842189080 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.901+0000 7f4546a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 0x562ef80b91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35530/0 (socket says 172.21.15.47:35530) 2024-03-21T11:33:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.901+0000 7f4546000640 1 -- 172.21.15.47:0/842189080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80af9e0 msgr2=0x562ef80ba6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.901+0000 7f4546000640 1 --2- 172.21.15.47:0/842189080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80af9e0 0x562ef80ba6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.901+0000 7f4546000640 1 -- 172.21.15.47:0/842189080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 msgr2=0x562ef80b91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.901+0000 7f4546000640 1 --2- 172.21.15.47:0/842189080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 0x562ef80b91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.901+0000 7f4546000640 1 -- 172.21.15.47:0/842189080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ef80bacc0 con 0x562ef80aa870 2024-03-21T11:33:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.901+0000 7f4546000640 1 --2- 172.21.15.47:0/842189080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80b9710 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f45340058c0 tx=0x7f4534011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12789fc0109dc219 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.901+0000 7f4530c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:33:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.902+0000 7f4530c00640 1 -- 172.21.15.47:0/842189080 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453403d060 con 0x562ef80aa870 2024-03-21T11:33:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.902+0000 7f4530c00640 1 -- 172.21.15.47:0/842189080 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4534009e40 con 0x562ef80aa870 2024-03-21T11:33:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.902+0000 7f455065b740 1 -- 172.21.15.47:0/842189080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ef8016ae0 con 0x562ef80aa870 2024-03-21T11:33:26.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.902+0000 7f4530c00640 1 -- 172.21.15.47:0/842189080 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4534044050 con 0x562ef80aa870 2024-03-21T11:33:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.902+0000 7f455065b740 10 client.28850 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.902+0000 7f455065b740 1 -- 172.21.15.47:0/842189080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562ef811b8f0 con 0x562ef80aa870 2024-03-21T11:33:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.902+0000 7f4530c00640 1 -- 172.21.15.47:0/842189080 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562ef8016ae0 con 0x562ef80aa870 2024-03-21T11:33:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.902+0000 7f4530c00640 1 -- 172.21.15.47:0/842189080 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f45340238f0 con 0x562ef80aa870 2024-03-21T11:33:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.902+0000 7f455065b740 10 client.28850 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.902+0000 7f455065b740 10 client.28850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.902+0000 7f455065b740 20 client.28850 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.903+0000 7f455065b740 1 --2- 172.21.15.47:0/842189080 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ef811c1e0 0x562ef811e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.903+0000 7f455065b740 4 client.28850 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:33:26.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.903+0000 7f455065b740 1 -- 172.21.15.47:0/842189080 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562ef7f943a0 con 0x562ef811c1e0 2024-03-21T11:33:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.903+0000 7f4546a00640 1 --2- 172.21.15.47:0/842189080 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ef811c1e0 0x562ef811e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.903+0000 7f4546a00640 1 --2- 172.21.15.47:0/842189080 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ef811c1e0 0x562ef811e600 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a66b95ed28d9f9b server_cookie=c1f6a9894a11740c in_seq=0 out_seq=0 2024-03-21T11:33:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.903+0000 7f4530c00640 10 client.28850 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.904+0000 7f4530c00640 1 -- 172.21.15.47:0/842189080 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562ef7f943a0 con 0x562ef811c1e0 2024-03-21T11:33:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.904+0000 7f4530c00640 10 client.28850 handle_command_reply: tid=0 2024-03-21T11:33:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.904+0000 7f455065b740 1 client.28850 shutdown 2024-03-21T11:33:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.905+0000 7f455065b740 1 -- 172.21.15.47:0/842189080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 msgr2=0x562ef80b9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.905+0000 7f455065b740 1 --2- 172.21.15.47:0/842189080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80b9710 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f45340058c0 tx=0x7f4534011d10 comp rx=0 tx=0).stop 2024-03-21T11:33:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.905+0000 7f455065b740 1 -- 172.21.15.47:0/842189080 shutdown_connections 2024-03-21T11:33:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.905+0000 7f455065b740 1 --2- 172.21.15.47:0/842189080 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562ef811c1e0 0x562ef811e600 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.905+0000 7f455065b740 1 --2- 172.21.15.47:0/842189080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562ef80af9e0 0x562ef80ba6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.905+0000 7f455065b740 1 --2- 172.21.15.47:0/842189080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ef80aa870 0x562ef80b9710 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.905+0000 7f4530c00640 0 client.28850 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.905+0000 7f455065b740 1 --2- 172.21.15.47:0/842189080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562ef80a9e80 0x562ef80b91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.905+0000 7f455065b740 1 -- 172.21.15.47:0/842189080 >> 172.21.15.47:0/842189080 conn(0x562ef8072200 msgr2=0x562ef80ab810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.905+0000 7f455065b740 1 -- 172.21.15.47:0/842189080 shutdown_connections 2024-03-21T11:33:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.905+0000 7f455065b740 1 -- 172.21.15.47:0/842189080 wait complete. 2024-03-21T11:33:26.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.905+0000 7f455065b740 20 client.28850 trim_cache size 0 max 16384 2024-03-21T11:33:26.912 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:33:26.913 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:33:26.913 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:33:26.913 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:33:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.906+0000 7f454da00640 1 -- 172.21.15.47:0/1903282089 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f452c0a2f20 msgr2=0x7f452c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.906+0000 7f454da00640 1 --2- 172.21.15.47:0/1903282089 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f452c0a2f20 0x7f452c0a53e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x562ef80ab0d0 tx=0x7f4540009500 comp rx=0 tx=0).stop 2024-03-21T11:33:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.906+0000 7f454da00640 1 -- 172.21.15.47:0/1903282089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4548107cc0 msgr2=0x7f454819dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.906+0000 7f454da00640 1 --2- 172.21.15.47:0/1903282089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4548107cc0 0x7f454819dfb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4538000c00 tx=0x7f453802c650 comp rx=0 tx=0).stop 2024-03-21T11:33:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.906+0000 7f454da00640 1 -- 172.21.15.47:0/1903282089 shutdown_connections 2024-03-21T11:33:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.906+0000 7f454da00640 1 --2- 172.21.15.47:0/1903282089 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f452c0a2f20 0x7f452c0a53e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.906+0000 7f454da00640 1 --2- 172.21.15.47:0/1903282089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4548109870 0x7f45481a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.906+0000 7f454da00640 1 --2- 172.21.15.47:0/1903282089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4548108eb0 0x7f454819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.906+0000 7f454da00640 1 --2- 172.21.15.47:0/1903282089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4548107cc0 0x7f454819dfb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:26.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.906+0000 7f454da00640 1 -- 172.21.15.47:0/1903282089 >> 172.21.15.47:0/1903282089 conn(0x7f4548076c60 msgr2=0x7f454810c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:26.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.906+0000 7f454da00640 1 -- 172.21.15.47:0/1903282089 shutdown_connections 2024-03-21T11:33:26.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:26.906+0000 7f454da00640 1 -- 172.21.15.47:0/1903282089 wait complete. 2024-03-21T11:33:26.924 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:33:26.925 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:33:27.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.148+0000 7f7fa1800640 1 -- 172.21.15.47:0/2622018349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f9c105490 msgr2=0x7f7f9c10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.148+0000 7f7fa1800640 1 --2- 172.21.15.47:0/2622018349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f9c105490 0x7f7f9c10bd20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f7f94009900 tx=0x7f7f9402bc40 comp rx=0 tx=0).stop 2024-03-21T11:33:27.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.148+0000 7f7fa1800640 1 -- 172.21.15.47:0/2622018349 shutdown_connections 2024-03-21T11:33:27.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.148+0000 7f7fa1800640 1 --2- 172.21.15.47:0/2622018349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f9c105490 0x7f7f9c10bd20 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.148+0000 7f7fa1800640 1 --2- 172.21.15.47:0/2622018349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f9c104ad0 0x7f7f9c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.148+0000 7f7fa1800640 1 --2- 172.21.15.47:0/2622018349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f9c1038e0 0x7f7f9c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.148+0000 7f7fa1800640 1 -- 172.21.15.47:0/2622018349 >> 172.21.15.47:0/2622018349 conn(0x7f7f9c0fee90 msgr2=0x7f7f9c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:27.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.148+0000 7f7fa1800640 1 -- 172.21.15.47:0/2622018349 shutdown_connections 2024-03-21T11:33:27.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.148+0000 7f7fa1800640 1 -- 172.21.15.47:0/2622018349 wait complete. 2024-03-21T11:33:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.149+0000 7f7fa1800640 1 Processor -- start 2024-03-21T11:33:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.149+0000 7f7fa1800640 1 -- start start 2024-03-21T11:33:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.150+0000 7f7fa1800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f9c1038e0 0x7f7f9c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.150+0000 7f7fa1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f9c104ad0 0x7f7f9c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.150+0000 7f7fa1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f9c105490 0x7f7f9c1a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.150+0000 7f7fa1800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c10f560 con 0x7f7f9c104ad0 2024-03-21T11:33:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.150+0000 7f7fa1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c10f3e0 con 0x7f7f9c105490 2024-03-21T11:33:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.150+0000 7f7fa1800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c10f6e0 con 0x7f7f9c1038e0 2024-03-21T11:33:27.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.150+0000 7f7f9b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f9c105490 0x7f7f9c1a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.150+0000 7f7f9aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f9c1038e0 0x7f7f9c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.150+0000 7f7f9b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f9c105490 0x7f7f9c1a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52258/0 (socket says 172.21.15.47:52258) 2024-03-21T11:33:27.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.150+0000 7f7f9aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f9c1038e0 0x7f7f9c199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41606/0 (socket says 172.21.15.47:41606) 2024-03-21T11:33:27.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.150+0000 7f7f9b400640 1 -- 172.21.15.47:0/2331685218 learned_addr learned my addr 172.21.15.47:0/2331685218 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:27.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.150+0000 7f7f92000640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f9c104ad0 0x7f7f9c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.151+0000 7f7f9aa00640 1 -- 172.21.15.47:0/2331685218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f9c105490 msgr2=0x7f7f9c1a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.151+0000 7f7f9aa00640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f9c105490 0x7f7f9c1a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.151+0000 7f7f9aa00640 1 -- 172.21.15.47:0/2331685218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f9c104ad0 msgr2=0x7f7f9c19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.151+0000 7f7f9aa00640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f9c104ad0 0x7f7f9c19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.151+0000 7f7f9aa00640 1 -- 172.21.15.47:0/2331685218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c1a1790 con 0x7f7f9c1038e0 2024-03-21T11:33:27.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.151+0000 7f7f9b400640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f9c105490 0x7f7f9c1a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:27.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.151+0000 7f7f92000640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f9c104ad0 0x7f7f9c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:27.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.151+0000 7f7f9aa00640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f9c1038e0 0x7f7f9c199b70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00e9c0 tx=0x7f7f8c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.151+0000 7f7f93e00640 1 -- 172.21.15.47:0/2331685218 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00ccc0 con 0x7f7f9c1038e0 2024-03-21T11:33:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.151+0000 7f7f93e00640 1 -- 172.21.15.47:0/2331685218 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f8c0035f0 con 0x7f7f9c1038e0 2024-03-21T11:33:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.151+0000 7f7fa1800640 1 -- 172.21.15.47:0/2331685218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9c1a1a80 con 0x7f7f9c1038e0 2024-03-21T11:33:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.151+0000 7f7f93e00640 1 -- 172.21.15.47:0/2331685218 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00ce60 con 0x7f7f9c1038e0 2024-03-21T11:33:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.152+0000 7f7fa1800640 1 -- 172.21.15.47:0/2331685218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9c1a1fd0 con 0x7f7f9c1038e0 2024-03-21T11:33:27.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.153+0000 7f7fa1800640 1 -- 172.21.15.47:0/2331685218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9c10ac00 con 0x7f7f9c1038e0 2024-03-21T11:33:27.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.155+0000 7f7f93e00640 1 -- 172.21.15.47:0/2331685218 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7f8c0037a0 con 0x7f7f9c1038e0 2024-03-21T11:33:27.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.156+0000 7f7f93e00640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f800a2fa0 0x7f7f800a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.156+0000 7f7f93e00640 1 -- 172.21.15.47:0/2331685218 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7f8c0cfc90 con 0x7f7f9c1038e0 2024-03-21T11:33:27.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.157+0000 7f7f92000640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f800a2fa0 0x7f7f800a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.158+0000 7f7f92000640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f800a2fa0 0x7f7f800a5460 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7f840099a0 tx=0x7f7f84009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:27.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.160+0000 7f7f93e00640 1 -- 172.21.15.47:0/2331685218 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f8c09c6d0 con 0x7f7f9c1038e0 2024-03-21T11:33:27.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.432+0000 7f7fa1800640 1 -- 172.21.15.47:0/2331685218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f9c02d070 con 0x7f7f9c1038e0 2024-03-21T11:33:27.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.434+0000 7f7f93e00640 1 -- 172.21.15.47:0/2331685218 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f7f8c09bfd0 con 0x7f7f9c1038e0 2024-03-21T11:33:27.435 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:33:27.435 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:33:27.435 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:33:27.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.436+0000 7f7fa1800640 1 -- 172.21.15.47:0/2331685218 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f800a2fa0 msgr2=0x7f7f800a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.436+0000 7f7fa1800640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f800a2fa0 0x7f7f800a5460 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7f840099a0 tx=0x7f7f84009500 comp rx=0 tx=0).stop 2024-03-21T11:33:27.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.436+0000 7f7fa1800640 1 -- 172.21.15.47:0/2331685218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f9c1038e0 msgr2=0x7f7f9c199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.436+0000 7f7fa1800640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f9c1038e0 0x7f7f9c199b70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00e9c0 tx=0x7f7f8c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:33:27.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.437+0000 7f7fa1800640 1 -- 172.21.15.47:0/2331685218 shutdown_connections 2024-03-21T11:33:27.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.437+0000 7f7fa1800640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f800a2fa0 0x7f7f800a5460 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.437+0000 7f7fa1800640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f9c105490 0x7f7f9c1a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.437+0000 7f7fa1800640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f9c104ad0 0x7f7f9c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.437+0000 7f7fa1800640 1 --2- 172.21.15.47:0/2331685218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f9c1038e0 0x7f7f9c199b70 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.437+0000 7f7fa1800640 1 -- 172.21.15.47:0/2331685218 >> 172.21.15.47:0/2331685218 conn(0x7f7f9c0fee90 msgr2=0x7f7f9c107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:27.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.437+0000 7f7fa1800640 1 -- 172.21.15.47:0/2331685218 shutdown_connections 2024-03-21T11:33:27.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.437+0000 7f7fa1800640 1 -- 172.21.15.47:0/2331685218 wait complete. 2024-03-21T11:33:27.457 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:33:27.457 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:33:27.551 INFO:tasks.workunit.client.0.smithi047.stdout: 1 748284 8.07 MB/sec execute 371 sec latency 15.460 ms 2024-03-21T11:33:27.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.690+0000 7ff7f7e00640 1 -- 172.21.15.47:0/132556436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7f8108db0 msgr2=0x7ff7f8109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.690+0000 7ff7f7e00640 1 --2- 172.21.15.47:0/132556436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7f8108db0 0x7ff7f8109230 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009900 tx=0x7ff7e402bc30 comp rx=0 tx=0).stop 2024-03-21T11:33:27.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.690+0000 7ff7f7e00640 1 -- 172.21.15.47:0/132556436 shutdown_connections 2024-03-21T11:33:27.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.690+0000 7ff7f7e00640 1 --2- 172.21.15.47:0/132556436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8109770 0x7ff7f8110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.690+0000 7ff7f7e00640 1 --2- 172.21.15.47:0/132556436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7f8108db0 0x7ff7f8109230 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.690+0000 7ff7f7e00640 1 --2- 172.21.15.47:0/132556436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7f8107bc0 0x7ff7f8107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.690+0000 7ff7f7e00640 1 -- 172.21.15.47:0/132556436 >> 172.21.15.47:0/132556436 conn(0x7ff7f8076b70 msgr2=0x7ff7f8078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:27.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.690+0000 7ff7f7e00640 1 -- 172.21.15.47:0/132556436 shutdown_connections 2024-03-21T11:33:27.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.690+0000 7ff7f7e00640 1 -- 172.21.15.47:0/132556436 wait complete. 2024-03-21T11:33:27.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.691+0000 7ff7f7e00640 1 Processor -- start 2024-03-21T11:33:27.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.691+0000 7ff7f7e00640 1 -- start start 2024-03-21T11:33:27.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7f8107bc0 0x7ff7f819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7f8108db0 0x7ff7f819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8109770 0x7ff7f81a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f7e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f8113870 con 0x7ff7f8108db0 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f81136f0 con 0x7ff7f8107bc0 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f7e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f81139f0 con 0x7ff7f8109770 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7f8107bc0 0x7ff7f819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7f8108db0 0x7ff7f819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8109770 0x7ff7f81a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7f8108db0 0x7ff7f819e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35574/0 (socket says 172.21.15.47:35574) 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7f8107bc0 0x7ff7f819de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52282/0 (socket says 172.21.15.47:52282) 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f6000640 1 -- 172.21.15.47:0/2049409781 learned_addr learned my addr 172.21.15.47:0/2049409781 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8109770 0x7ff7f81a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41616/0 (socket says 172.21.15.47:41616) 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f7400640 1 -- 172.21.15.47:0/2049409781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7f8107bc0 msgr2=0x7ff7f819de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f7400640 1 --2- 172.21.15.47:0/2049409781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7f8107bc0 0x7ff7f819de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f7400640 1 -- 172.21.15.47:0/2049409781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7f8108db0 msgr2=0x7ff7f819e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f7400640 1 --2- 172.21.15.47:0/2049409781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7f8108db0 0x7ff7f819e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.692+0000 7ff7f7400640 1 -- 172.21.15.47:0/2049409781 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f81a5ab0 con 0x7ff7f8109770 2024-03-21T11:33:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.693+0000 7ff7f7400640 1 --2- 172.21.15.47:0/2049409781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8109770 0x7ff7f81a5440 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009fd0 tx=0x7ff7f000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:27.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.693+0000 7ff7ef400640 1 -- 172.21.15.47:0/2049409781 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f000ce20 con 0x7ff7f8109770 2024-03-21T11:33:27.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.693+0000 7ff7ef400640 1 -- 172.21.15.47:0/2049409781 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7f0005020 con 0x7ff7f8109770 2024-03-21T11:33:27.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.693+0000 7ff7f7e00640 1 -- 172.21.15.47:0/2049409781 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f81a5da0 con 0x7ff7f8109770 2024-03-21T11:33:27.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.693+0000 7ff7ef400640 1 -- 172.21.15.47:0/2049409781 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0010720 con 0x7ff7f8109770 2024-03-21T11:33:27.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.693+0000 7ff7f7e00640 1 -- 172.21.15.47:0/2049409781 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f81a6260 con 0x7ff7f8109770 2024-03-21T11:33:27.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.697+0000 7ff8007f1740 1 Processor -- start 2024-03-21T11:33:27.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.698+0000 7ff7ef400640 1 -- 172.21.15.47:0/2049409781 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff7f0005540 con 0x7ff7f8109770 2024-03-21T11:33:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.698+0000 7ff8007f1740 1 -- start start 2024-03-21T11:33:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.699+0000 7ff8007f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf17449f0 0x557cf1744e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.699+0000 7ff8007f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf1745bc0 0x557cf1746040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.699+0000 7ff8007f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1746580 0x557cf174ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.699+0000 7ff8007f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf16a92c0 con 0x557cf17449f0 2024-03-21T11:33:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.699+0000 7ff8007f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf1651c90 con 0x557cf1746580 2024-03-21T11:33:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.699+0000 7ff8007f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf16a5360 con 0x557cf1745bc0 2024-03-21T11:33:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.699+0000 7ff7f6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf1745bc0 0x557cf1746040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.699+0000 7ff7f6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf1745bc0 0x557cf1746040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41626/0 (socket says 172.21.15.47:41626) 2024-03-21T11:33:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.699+0000 7ff7f6a00640 1 -- 172.21.15.47:0/2093142497 learned_addr learned my addr 172.21.15.47:0/2093142497 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.699+0000 7ff7f6000640 1 --2- 172.21.15.47:0/2093142497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf17449f0 0x557cf1744e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.699+0000 7ff7f6000640 1 --2- 172.21.15.47:0/2093142497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1746580 0x557cf174ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.699+0000 7ff7f6a00640 1 -- 172.21.15.47:0/2093142497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1746580 msgr2=0x557cf174ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.700+0000 7ff7f6a00640 1 --2- 172.21.15.47:0/2093142497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1746580 0x557cf174ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.700+0000 7ff7f6a00640 1 -- 172.21.15.47:0/2093142497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf17449f0 msgr2=0x557cf1744e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.700+0000 7ff7f6a00640 1 --2- 172.21.15.47:0/2093142497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf17449f0 0x557cf1744e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.700+0000 7ff7f6a00640 1 -- 172.21.15.47:0/2093142497 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cf1676080 con 0x557cf1745bc0 2024-03-21T11:33:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.700+0000 7ff7f6000640 1 --2- 172.21.15.47:0/2093142497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf17449f0 0x557cf1744e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:33:27.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.700+0000 7ff7f6a00640 1 --2- 172.21.15.47:0/2093142497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf1745bc0 0x557cf1746040 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009ca0 tx=0x7ff7e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=442191d9df57c1da server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:27.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.700+0000 7ff7ef400640 1 --2- 172.21.15.47:0/2049409781 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7dc0a2fa0 0x7ff7dc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.700+0000 7ff7ef400640 1 -- 172.21.15.47:0/2049409781 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff7f00cfe60 con 0x7ff7f8109770 2024-03-21T11:33:27.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.700+0000 7ff7e3e00640 1 -- 172.21.15.47:0/2093142497 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8004220 con 0x557cf1745bc0 2024-03-21T11:33:27.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.700+0000 7ff7f6000640 1 --2- 172.21.15.47:0/2049409781 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7dc0a2fa0 0x7ff7dc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.700+0000 7ff7e3e00640 1 -- 172.21.15.47:0/2093142497 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7e80043c0 con 0x557cf1745bc0 2024-03-21T11:33:27.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.700+0000 7ff7e3e00640 1 -- 172.21.15.47:0/2093142497 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8019930 con 0x557cf1745bc0 2024-03-21T11:33:27.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.701+0000 7ff8007f1740 1 -- 172.21.15.47:0/2093142497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf1745bc0 msgr2=0x557cf1746040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.701+0000 7ff8007f1740 1 --2- 172.21.15.47:0/2093142497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf1745bc0 0x557cf1746040 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009ca0 tx=0x7ff7e8011810 comp rx=0 tx=0).stop 2024-03-21T11:33:27.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.701+0000 7ff7f6000640 1 --2- 172.21.15.47:0/2049409781 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7dc0a2fa0 0x7ff7dc0a5460 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff7e402c8e0 tx=0x7ff7e4005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:27.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.702+0000 7ff8007f1740 1 -- 172.21.15.47:0/2093142497 shutdown_connections 2024-03-21T11:33:27.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.702+0000 7ff8007f1740 1 --2- 172.21.15.47:0/2093142497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1746580 0x557cf174ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.702+0000 7ff8007f1740 1 --2- 172.21.15.47:0/2093142497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf1745bc0 0x557cf1746040 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.702+0000 7ff8007f1740 1 --2- 172.21.15.47:0/2093142497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf17449f0 0x557cf1744e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.702+0000 7ff8007f1740 1 -- 172.21.15.47:0/2093142497 >> 172.21.15.47:0/2093142497 conn(0x557cf173fde0 msgr2=0x557cf17401e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:27.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.702+0000 7ff8007f1740 1 -- 172.21.15.47:0/2093142497 shutdown_connections 2024-03-21T11:33:27.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.702+0000 7ff8007f1740 1 -- 172.21.15.47:0/2093142497 wait complete. 2024-03-21T11:33:27.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.702+0000 7ff8007f1740 1 -- start start 2024-03-21T11:33:27.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.702+0000 7ff8007f1740 1 Processor -- start 2024-03-21T11:33:27.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.703+0000 7ff8007f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf17449f0 0x557cf174ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.703+0000 7ff8007f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1745bc0 0x557cf174f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.703+0000 7ff8007f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf1746580 0x557cf17561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.703+0000 7ff8007f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf174d590 con 0x557cf1746580 2024-03-21T11:33:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.703+0000 7ff8007f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf16650d0 con 0x557cf1745bc0 2024-03-21T11:33:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.703+0000 7ff8007f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf174d710 con 0x557cf17449f0 2024-03-21T11:33:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.703+0000 7ff8007f1740 10 client.0 _ll_register_callbacks cb 0x557cf1669e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.703+0000 7ff7f6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf17449f0 0x557cf174ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.703+0000 7ff7f6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1745bc0 0x557cf174f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.703+0000 7ff7f6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf17449f0 0x557cf174ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41640/0 (socket says 172.21.15.47:41640) 2024-03-21T11:33:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.703+0000 7ff7f6a00640 1 -- 172.21.15.47:0/1226567989 learned_addr learned my addr 172.21.15.47:0/1226567989 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.703+0000 7ff7f7400640 1 --2- 172.21.15.47:0/1226567989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf1746580 0x557cf17561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.704+0000 7ff7f6a00640 1 -- 172.21.15.47:0/1226567989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1745bc0 msgr2=0x557cf174f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.704+0000 7ff7f6a00640 1 --2- 172.21.15.47:0/1226567989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1745bc0 0x557cf174f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.704+0000 7ff7f6a00640 1 -- 172.21.15.47:0/1226567989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf1746580 msgr2=0x557cf17561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.704+0000 7ff7f6a00640 1 --2- 172.21.15.47:0/1226567989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf1746580 0x557cf17561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.704+0000 7ff7f6a00640 1 -- 172.21.15.47:0/1226567989 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cf13bd3f0 con 0x557cf17449f0 2024-03-21T11:33:27.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.704+0000 7ff7f6a00640 1 --2- 172.21.15.47:0/1226567989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf17449f0 0x557cf174ec60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8011d10 tx=0x7ff7e8003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac0966a40a405510 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:27.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.704+0000 7ff7e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:33:27.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.704+0000 7ff7e0c00640 1 -- 172.21.15.47:0/1226567989 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8028070 con 0x557cf17449f0 2024-03-21T11:33:27.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.704+0000 7ff7e0c00640 1 -- 172.21.15.47:0/1226567989 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff7e8005040 con 0x557cf17449f0 2024-03-21T11:33:27.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.704+0000 7ff8007f1740 1 -- 172.21.15.47:0/1226567989 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7e80043c0 con 0x557cf17449f0 2024-03-21T11:33:27.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.705+0000 7ff7e0c00640 1 -- 172.21.15.47:0/1226567989 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7e8034050 con 0x557cf17449f0 2024-03-21T11:33:27.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.705+0000 7ff8007f1740 10 client.28885 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:27.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.705+0000 7ff8007f1740 1 -- 172.21.15.47:0/1226567989 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557cf14877d0 con 0x557cf17449f0 2024-03-21T11:33:27.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.705+0000 7ff7e0c00640 1 -- 172.21.15.47:0/1226567989 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7e80043c0 con 0x557cf17449f0 2024-03-21T11:33:27.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.705+0000 7ff7e0c00640 1 -- 172.21.15.47:0/1226567989 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff7e8004aa0 con 0x557cf17449f0 2024-03-21T11:33:27.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.705+0000 7ff8007f1740 10 client.28885 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:27.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.705+0000 7ff8007f1740 10 client.28885 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:33:27.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.705+0000 7ff8007f1740 20 client.28885 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:27.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.705+0000 7ff8007f1740 1 --2- 172.21.15.47:0/1226567989 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x557cf175a3f0 0x557cf175c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.705+0000 7ff8007f1740 4 client.28885 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:33:27.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.705+0000 7ff8007f1740 1 -- 172.21.15.47:0/1226567989 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cf16b6800 con 0x557cf175a3f0 2024-03-21T11:33:27.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.705+0000 7ff7f6000640 1 --2- 172.21.15.47:0/1226567989 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x557cf175a3f0 0x557cf175c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.706+0000 7ff7f6000640 1 --2- 172.21.15.47:0/1226567989 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x557cf175a3f0 0x557cf175c820 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bef034ed9c87e7d8 server_cookie=5593013b76212c13 in_seq=0 out_seq=0 2024-03-21T11:33:27.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.706+0000 7ff7e0c00640 10 client.28885 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:33:27.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff7e0c00640 1 -- 172.21.15.47:0/1226567989 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557cf16b6800 con 0x557cf175a3f0 2024-03-21T11:33:27.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff7e0c00640 10 client.28885 handle_command_reply: tid=0 2024-03-21T11:33:27.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff8007f1740 1 client.28885 shutdown 2024-03-21T11:33:27.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff8007f1740 1 -- 172.21.15.47:0/1226567989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf17449f0 msgr2=0x557cf174ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff8007f1740 1 --2- 172.21.15.47:0/1226567989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf17449f0 0x557cf174ec60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8011d10 tx=0x7ff7e8003820 comp rx=0 tx=0).stop 2024-03-21T11:33:27.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff8007f1740 1 -- 172.21.15.47:0/1226567989 shutdown_connections 2024-03-21T11:33:27.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff8007f1740 1 --2- 172.21.15.47:0/1226567989 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x557cf175a3f0 0x557cf175c820 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff8007f1740 1 --2- 172.21.15.47:0/1226567989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf1746580 0x557cf17561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff8007f1740 1 --2- 172.21.15.47:0/1226567989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1745bc0 0x557cf174f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff8007f1740 1 --2- 172.21.15.47:0/1226567989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf17449f0 0x557cf174ec60 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff7e0c00640 0 client.28885 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:33:27.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff8007f1740 1 -- 172.21.15.47:0/1226567989 >> 172.21.15.47:0/1226567989 conn(0x557cf173fde0 msgr2=0x557cf173f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:27.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff8007f1740 1 -- 172.21.15.47:0/1226567989 shutdown_connections 2024-03-21T11:33:27.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff8007f1740 1 -- 172.21.15.47:0/1226567989 wait complete. 2024-03-21T11:33:27.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.707+0000 7ff8007f1740 20 client.28885 trim_cache size 0 max 16384 2024-03-21T11:33:27.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.723+0000 7ff8007f1740 1 Processor -- start 2024-03-21T11:33:27.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.723+0000 7ff8007f1740 1 -- start start 2024-03-21T11:33:27.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.723+0000 7ff8007f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf17449f0 0x557cf1743040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.723+0000 7ff8007f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf1745bc0 0x557cf17561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.723+0000 7ff8007f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1746580 0x557cf17610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.723+0000 7ff8007f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f81b04b0 con 0x557cf1745bc0 2024-03-21T11:33:27.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.723+0000 7ff8007f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f81b1e30 con 0x557cf1746580 2024-03-21T11:33:27.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.723+0000 7ff8007f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f81b0b10 con 0x557cf17449f0 2024-03-21T11:33:27.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.723+0000 7ff7f7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1746580 0x557cf17610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.724+0000 7ff7f6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf17449f0 0x557cf1743040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.724+0000 7ff7f6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf17449f0 0x557cf1743040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41646/0 (socket says 172.21.15.47:41646) 2024-03-21T11:33:27.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.724+0000 7ff7f7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1746580 0x557cf17610e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52306/0 (socket says 172.21.15.47:52306) 2024-03-21T11:33:27.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.724+0000 7ff7f6a00640 1 -- 172.21.15.47:0/500839835 learned_addr learned my addr 172.21.15.47:0/500839835 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:27.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.724+0000 7ff7f6000640 1 --2- 172.21.15.47:0/500839835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf1745bc0 0x557cf17561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.724+0000 7ff7f6a00640 1 -- 172.21.15.47:0/500839835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1746580 msgr2=0x557cf17610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:27.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.724+0000 7ff7f6a00640 1 --2- 172.21.15.47:0/500839835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1746580 0x557cf17610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.724+0000 7ff7f6a00640 1 -- 172.21.15.47:0/500839835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf1745bc0 msgr2=0x557cf17561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:27.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.724+0000 7ff7f6a00640 1 --2- 172.21.15.47:0/500839835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf1745bc0 0x557cf17561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.724+0000 7ff7f6a00640 1 -- 172.21.15.47:0/500839835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cf1745640 con 0x557cf17449f0 2024-03-21T11:33:27.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.724+0000 7ff7f6000640 1 --2- 172.21.15.47:0/500839835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf1745bc0 0x557cf17561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:33:27.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.724+0000 7ff7f6a00640 1 --2- 172.21.15.47:0/500839835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf17449f0 0x557cf1743040 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8000c00 tx=0x7ff7e8001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f01e9e5b1401902 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:27.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.725+0000 7ff7e3400640 1 -- 172.21.15.47:0/500839835 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8028030 con 0x557cf17449f0 2024-03-21T11:33:27.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.725+0000 7ff7e3400640 1 -- 172.21.15.47:0/500839835 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7e8009e50 con 0x557cf17449f0 2024-03-21T11:33:27.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.725+0000 7ff7e3400640 1 -- 172.21.15.47:0/500839835 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e80048b0 con 0x557cf17449f0 2024-03-21T11:33:27.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.725+0000 7ff8007f1740 1 -- 172.21.15.47:0/500839835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf17449f0 msgr2=0x557cf1743040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.725+0000 7ff8007f1740 1 --2- 172.21.15.47:0/500839835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf17449f0 0x557cf1743040 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8000c00 tx=0x7ff7e8001e50 comp rx=0 tx=0).stop 2024-03-21T11:33:27.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.725+0000 7ff8007f1740 1 -- 172.21.15.47:0/500839835 shutdown_connections 2024-03-21T11:33:27.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.725+0000 7ff8007f1740 1 --2- 172.21.15.47:0/500839835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1746580 0x557cf17610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.725+0000 7ff8007f1740 1 --2- 172.21.15.47:0/500839835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf1745bc0 0x557cf17561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.725+0000 7ff8007f1740 1 --2- 172.21.15.47:0/500839835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf17449f0 0x557cf1743040 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.725+0000 7ff8007f1740 1 -- 172.21.15.47:0/500839835 >> 172.21.15.47:0/500839835 conn(0x557cf173fde0 msgr2=0x557cf173f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:27.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.726+0000 7ff8007f1740 1 -- 172.21.15.47:0/500839835 shutdown_connections 2024-03-21T11:33:27.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.726+0000 7ff8007f1740 1 -- 172.21.15.47:0/500839835 wait complete. 2024-03-21T11:33:27.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.726+0000 7ff8007f1740 1 -- start start 2024-03-21T11:33:27.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.726+0000 7ff8007f1740 1 Processor -- start 2024-03-21T11:33:27.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff8007f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf17449f0 0x557cf1751f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff8007f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1745bc0 0x557cf1752440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff8007f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf1746580 0x557cf1755410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff8007f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f81b17d0 con 0x557cf17449f0 2024-03-21T11:33:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff8007f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f81b1170 con 0x557cf1745bc0 2024-03-21T11:33:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff8007f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f81ae670 con 0x557cf1746580 2024-03-21T11:33:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff8007f1740 10 client.0 _ll_register_callbacks cb 0x7ff7e8009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff7f6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1745bc0 0x557cf1752440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff7f7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf1746580 0x557cf1755410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff7f6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1745bc0 0x557cf1752440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52310/0 (socket says 172.21.15.47:52310) 2024-03-21T11:33:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff7f6000640 1 -- 172.21.15.47:0/2823528189 learned_addr learned my addr 172.21.15.47:0/2823528189 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff7f6a00640 1 --2- 172.21.15.47:0/2823528189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf17449f0 0x557cf1751f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff7f6000640 1 -- 172.21.15.47:0/2823528189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf1746580 msgr2=0x557cf1755410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff7f6000640 1 --2- 172.21.15.47:0/2823528189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf1746580 0x557cf1755410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff7f6000640 1 -- 172.21.15.47:0/2823528189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf17449f0 msgr2=0x557cf1751f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:27.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff7f6000640 1 --2- 172.21.15.47:0/2823528189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf17449f0 0x557cf1751f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff7f6000640 1 -- 172.21.15.47:0/2823528189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cf1755a00 con 0x557cf1745bc0 2024-03-21T11:33:27.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.727+0000 7ff7f7400640 1 --2- 172.21.15.47:0/2823528189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf1746580 0x557cf1755410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:27.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.728+0000 7ff7f6000640 1 --2- 172.21.15.47:0/2823528189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1745bc0 0x557cf1752440 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff7e402c240 tx=0x7ff7e402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e4a16538ad8858a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:27.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.728+0000 7ff7e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:33:27.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.728+0000 7ff7e0c00640 1 -- 172.21.15.47:0/2823528189 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4004a50 con 0x557cf1745bc0 2024-03-21T11:33:27.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.728+0000 7ff8007f1740 1 -- 172.21.15.47:0/2823528189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7e8009e50 con 0x557cf1745bc0 2024-03-21T11:33:27.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.728+0000 7ff7e0c00640 1 -- 172.21.15.47:0/2823528189 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff7e4004bf0 con 0x557cf1745bc0 2024-03-21T11:33:27.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.728+0000 7ff7e0c00640 1 -- 172.21.15.47:0/2823528189 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7e4069050 con 0x557cf1745bc0 2024-03-21T11:33:27.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.728+0000 7ff8007f1740 10 client.28874 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:27.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.728+0000 7ff8007f1740 1 -- 172.21.15.47:0/2823528189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557cf17b6310 con 0x557cf1745bc0 2024-03-21T11:33:27.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.729+0000 7ff7e0c00640 1 -- 172.21.15.47:0/2823528189 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7e8009e50 con 0x557cf1745bc0 2024-03-21T11:33:27.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.729+0000 7ff7e0c00640 1 -- 172.21.15.47:0/2823528189 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff7e4003ad0 con 0x557cf1745bc0 2024-03-21T11:33:27.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.729+0000 7ff8007f1740 10 client.28874 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.729+0000 7ff8007f1740 10 client.28874 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:33:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.729+0000 7ff8007f1740 20 client.28874 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.729+0000 7ff8007f1740 1 --2- 172.21.15.47:0/2823528189 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x557cf17b6c90 0x557cf17b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.729+0000 7ff8007f1740 4 client.28874 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:33:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.729+0000 7ff8007f1740 1 -- 172.21.15.47:0/2823528189 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557cf16b0b50 con 0x557cf17b6c90 2024-03-21T11:33:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.729+0000 7ff7f6a00640 1 --2- 172.21.15.47:0/2823528189 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x557cf17b6c90 0x557cf17b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff7f6a00640 1 --2- 172.21.15.47:0/2823528189 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x557cf17b6c90 0x557cf17b90b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dff0f2f487fd1005 server_cookie=210ebca418cca29 in_seq=0 out_seq=0 2024-03-21T11:33:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff7e0c00640 10 client.28874 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:33:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff7e0c00640 1 -- 172.21.15.47:0/2823528189 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557cf16b0b50 con 0x557cf17b6c90 2024-03-21T11:33:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff7e0c00640 10 client.28874 handle_command_reply: tid=0 2024-03-21T11:33:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff8007f1740 1 client.28874 shutdown 2024-03-21T11:33:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff8007f1740 1 -- 172.21.15.47:0/2823528189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1745bc0 msgr2=0x557cf1752440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff8007f1740 1 --2- 172.21.15.47:0/2823528189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1745bc0 0x557cf1752440 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff7e402c240 tx=0x7ff7e402bc30 comp rx=0 tx=0).stop 2024-03-21T11:33:27.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff8007f1740 1 -- 172.21.15.47:0/2823528189 shutdown_connections 2024-03-21T11:33:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff8007f1740 1 --2- 172.21.15.47:0/2823528189 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x557cf17b6c90 0x557cf17b90b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff8007f1740 1 --2- 172.21.15.47:0/2823528189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557cf1746580 0x557cf1755410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff8007f1740 1 --2- 172.21.15.47:0/2823528189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cf1745bc0 0x557cf1752440 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff8007f1740 1 --2- 172.21.15.47:0/2823528189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cf17449f0 0x557cf1751f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff7e0c00640 0 client.28874 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:33:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff8007f1740 1 -- 172.21.15.47:0/2823528189 >> 172.21.15.47:0/2823528189 conn(0x557cf173fde0 msgr2=0x557cf17610e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff8007f1740 1 -- 172.21.15.47:0/2823528189 shutdown_connections 2024-03-21T11:33:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.730+0000 7ff8007f1740 1 -- 172.21.15.47:0/2823528189 wait complete. 2024-03-21T11:33:27.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.731+0000 7ff8007f1740 20 client.28874 trim_cache size 0 max 16384 2024-03-21T11:33:27.736 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:33:27.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.731+0000 7ff7f7e00640 1 -- 172.21.15.47:0/2049409781 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7dc0a2fa0 msgr2=0x7ff7dc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.731+0000 7ff7f7e00640 1 --2- 172.21.15.47:0/2049409781 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7dc0a2fa0 0x7ff7dc0a5460 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff7e402c8e0 tx=0x7ff7e4005b50 comp rx=0 tx=0).stop 2024-03-21T11:33:27.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.731+0000 7ff7f7e00640 1 -- 172.21.15.47:0/2049409781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8109770 msgr2=0x7ff7f81a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.731+0000 7ff7f7e00640 1 --2- 172.21.15.47:0/2049409781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8109770 0x7ff7f81a5440 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009fd0 tx=0x7ff7f000ef10 comp rx=0 tx=0).stop 2024-03-21T11:33:27.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.731+0000 7ff7f7e00640 1 -- 172.21.15.47:0/2049409781 shutdown_connections 2024-03-21T11:33:27.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.731+0000 7ff7f7e00640 1 --2- 172.21.15.47:0/2049409781 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff7dc0a2fa0 0x7ff7dc0a5460 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.731+0000 7ff7f7e00640 1 --2- 172.21.15.47:0/2049409781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8109770 0x7ff7f81a5440 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.731+0000 7ff7f7e00640 1 --2- 172.21.15.47:0/2049409781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7f8108db0 0x7ff7f819e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.731+0000 7ff7f7e00640 1 --2- 172.21.15.47:0/2049409781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7f8107bc0 0x7ff7f819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.731+0000 7ff7f7e00640 1 -- 172.21.15.47:0/2049409781 >> 172.21.15.47:0/2049409781 conn(0x7ff7f8076b70 msgr2=0x7ff7f810c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:27.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.732+0000 7ff7f7e00640 1 -- 172.21.15.47:0/2049409781 shutdown_connections 2024-03-21T11:33:27.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.732+0000 7ff7f7e00640 1 -- 172.21.15.47:0/2049409781 wait complete. 2024-03-21T11:33:27.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:27 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2331685218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:33:27.749 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:27 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2331685218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:33:27.752 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:33:27.753 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:33:27.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:27 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2331685218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:33:27.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.985+0000 7fb2b6600640 1 -- 172.21.15.47:0/46807204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b01038c0 msgr2=0x7fb2b0103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.985+0000 7fb2b6600640 1 --2- 172.21.15.47:0/46807204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b01038c0 0x7fb2b0103ce0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009ca0 tx=0x7fb2a002c100 comp rx=0 tx=0).stop 2024-03-21T11:33:27.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.986+0000 7fb2b6600640 1 -- 172.21.15.47:0/46807204 shutdown_connections 2024-03-21T11:33:27.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.986+0000 7fb2b6600640 1 --2- 172.21.15.47:0/46807204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b0105470 0x7fb2b010bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.986+0000 7fb2b6600640 1 --2- 172.21.15.47:0/46807204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b0104ab0 0x7fb2b0104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.986+0000 7fb2b6600640 1 --2- 172.21.15.47:0/46807204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b01038c0 0x7fb2b0103ce0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.986+0000 7fb2b6600640 1 -- 172.21.15.47:0/46807204 >> 172.21.15.47:0/46807204 conn(0x7fb2b00fee50 msgr2=0x7fb2b0101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:27.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.986+0000 7fb2b6600640 1 -- 172.21.15.47:0/46807204 shutdown_connections 2024-03-21T11:33:27.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.986+0000 7fb2b6600640 1 -- 172.21.15.47:0/46807204 wait complete. 2024-03-21T11:33:27.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.987+0000 7fb2b6600640 1 Processor -- start 2024-03-21T11:33:27.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.987+0000 7fb2b6600640 1 -- start start 2024-03-21T11:33:27.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2b6600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b01038c0 0x7fb2b0199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2b6600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b0104ab0 0x7fb2b019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2b6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b0105470 0x7fb2b01a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2b6600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b010f510 con 0x7fb2b0104ab0 2024-03-21T11:33:27.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2b6600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b010f390 con 0x7fb2b01038c0 2024-03-21T11:33:27.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2b6600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b010f690 con 0x7fb2b0105470 2024-03-21T11:33:27.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2af400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b01038c0 0x7fb2b0199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2a6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b0104ab0 0x7fb2b019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b0105470 0x7fb2b01a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2af400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b01038c0 0x7fb2b0199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52336/0 (socket says 172.21.15.47:52336) 2024-03-21T11:33:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2af400640 1 -- 172.21.15.47:0/4087553508 learned_addr learned my addr 172.21.15.47:0/4087553508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2af400640 1 -- 172.21.15.47:0/4087553508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b0105470 msgr2=0x7fb2b01a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2af400640 1 --2- 172.21.15.47:0/4087553508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b0105470 0x7fb2b01a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2af400640 1 -- 172.21.15.47:0/4087553508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b0104ab0 msgr2=0x7fb2b019a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2af400640 1 --2- 172.21.15.47:0/4087553508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b0104ab0 0x7fb2b019a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.988+0000 7fb2af400640 1 -- 172.21.15.47:0/4087553508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b01a1810 con 0x7fb2b01038c0 2024-03-21T11:33:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.989+0000 7fb2afe00640 1 --2- 172.21.15.47:0/4087553508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b0105470 0x7fb2b01a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.989+0000 7fb2a6a00640 1 --2- 172.21.15.47:0/4087553508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b0104ab0 0x7fb2b019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.989+0000 7fb2af400640 1 --2- 172.21.15.47:0/4087553508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b01038c0 0x7fb2b0199bf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0000f80 tx=0x7fb2a0035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.989+0000 7fb2acc00640 1 -- 172.21.15.47:0/4087553508 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0039040 con 0x7fb2b01038c0 2024-03-21T11:33:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.989+0000 7fb2b6600640 1 -- 172.21.15.47:0/4087553508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b01a1aa0 con 0x7fb2b01038c0 2024-03-21T11:33:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.989+0000 7fb2acc00640 1 -- 172.21.15.47:0/4087553508 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2a0044070 con 0x7fb2b01038c0 2024-03-21T11:33:27.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.989+0000 7fb2acc00640 1 -- 172.21.15.47:0/4087553508 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a00045a0 con 0x7fb2b01038c0 2024-03-21T11:33:27.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.989+0000 7fb2b6600640 1 -- 172.21.15.47:0/4087553508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b01a1fb0 con 0x7fb2b01038c0 2024-03-21T11:33:27.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.993+0000 7fb2b93de740 1 Processor -- start 2024-03-21T11:33:27.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.994+0000 7fb2b93de740 1 -- start start 2024-03-21T11:33:27.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.994+0000 7fb2b93de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae5810 0x55aef8ae5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.994+0000 7fb2b93de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae69e0 0x55aef8ae6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.994+0000 7fb2b93de740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae73a0 0x55aef8aedc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:27.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.994+0000 7fb2b93de740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef8a4a2c0 con 0x55aef8ae73a0 2024-03-21T11:33:27.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.994+0000 7fb2b93de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef89f2c90 con 0x55aef8ae69e0 2024-03-21T11:33:27.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.994+0000 7fb2b93de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef8a46360 con 0x55aef8ae5810 2024-03-21T11:33:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.994+0000 7fb2acc00640 1 -- 172.21.15.47:0/4087553508 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb2a0032890 con 0x7fb2b01038c0 2024-03-21T11:33:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.994+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae5810 0x55aef8ae5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.994+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae5810 0x55aef8ae5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41686/0 (socket says 172.21.15.47:41686) 2024-03-21T11:33:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.994+0000 7fb2afe00640 1 -- 172.21.15.47:0/3599985874 learned_addr learned my addr 172.21.15.47:0/3599985874 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.994+0000 7fb2a6a00640 1 --2- 172.21.15.47:0/3599985874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae73a0 0x55aef8aedc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.995+0000 7fb2afe00640 1 -- 172.21.15.47:0/3599985874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae69e0 msgr2=0x55aef8ae6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.995+0000 7fb2afe00640 1 --2- 172.21.15.47:0/3599985874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae69e0 0x55aef8ae6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.995+0000 7fb2afe00640 1 -- 172.21.15.47:0/3599985874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae73a0 msgr2=0x55aef8aedc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.995+0000 7fb2afe00640 1 --2- 172.21.15.47:0/3599985874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae73a0 0x55aef8aedc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.995+0000 7fb2afe00640 1 -- 172.21.15.47:0/3599985874 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aef8a17080 con 0x55aef8ae5810 2024-03-21T11:33:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.995+0000 7fb2afe00640 1 --2- 172.21.15.47:0/3599985874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae5810 0x55aef8ae5c30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8010400 tx=0x7fb2a80108c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e2a1ce6e4f84905 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.995+0000 7fb2a4c00640 1 -- 172.21.15.47:0/3599985874 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8011760 con 0x55aef8ae5810 2024-03-21T11:33:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.995+0000 7fb2a4c00640 1 -- 172.21.15.47:0/3599985874 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2a8011de0 con 0x55aef8ae5810 2024-03-21T11:33:27.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.996+0000 7fb2a4c00640 1 -- 172.21.15.47:0/3599985874 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8018380 con 0x55aef8ae5810 2024-03-21T11:33:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.996+0000 7fb2acc00640 1 --2- 172.21.15.47:0/4087553508 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2940a2fa0 0x7fb2940a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.996+0000 7fb2acc00640 1 -- 172.21.15.47:0/4087553508 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb2a00f1d90 con 0x7fb2b01038c0 2024-03-21T11:33:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.996+0000 7fb2b93de740 1 -- 172.21.15.47:0/3599985874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae5810 msgr2=0x55aef8ae5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.996+0000 7fb2b93de740 1 --2- 172.21.15.47:0/3599985874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae5810 0x55aef8ae5c30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8010400 tx=0x7fb2a80108c0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.996+0000 7fb2af400640 1 --2- 172.21.15.47:0/4087553508 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2940a2fa0 0x7fb2940a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.997+0000 7fb2b93de740 1 -- 172.21.15.47:0/3599985874 shutdown_connections 2024-03-21T11:33:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.997+0000 7fb2b93de740 1 --2- 172.21.15.47:0/3599985874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae73a0 0x55aef8aedc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.997+0000 7fb2b93de740 1 --2- 172.21.15.47:0/3599985874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae69e0 0x55aef8ae6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.997+0000 7fb2b93de740 1 --2- 172.21.15.47:0/3599985874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae5810 0x55aef8ae5c30 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.997+0000 7fb2b93de740 1 -- 172.21.15.47:0/3599985874 >> 172.21.15.47:0/3599985874 conn(0x55aef8ae0c00 msgr2=0x55aef8ae1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.997+0000 7fb2b93de740 1 -- 172.21.15.47:0/3599985874 shutdown_connections 2024-03-21T11:33:28.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.997+0000 7fb2b93de740 1 -- 172.21.15.47:0/3599985874 wait complete. 2024-03-21T11:33:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.997+0000 7fb2b93de740 1 -- start start 2024-03-21T11:33:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.997+0000 7fb2af400640 1 --2- 172.21.15.47:0/4087553508 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2940a2fa0 0x7fb2940a5460 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00be060 tx=0x7fb2a00040d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.997+0000 7fb2b93de740 1 Processor -- start 2024-03-21T11:33:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.998+0000 7fb2b93de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae5810 0x55aef8aefa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.998+0000 7fb2b93de740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae69e0 0x55aef8aeffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.998+0000 7fb2b93de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae73a0 0x55aef8af6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.998+0000 7fb2b93de740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef8aee3b0 con 0x55aef8ae69e0 2024-03-21T11:33:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.998+0000 7fb2b93de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef8a060d0 con 0x55aef8ae5810 2024-03-21T11:33:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.998+0000 7fb2b93de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef8aee530 con 0x55aef8ae73a0 2024-03-21T11:33:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.998+0000 7fb2b93de740 10 client.0 _ll_register_callbacks cb 0x55aef8a0ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.998+0000 7fb2a6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae69e0 0x55aef8aeffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.998+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae5810 0x55aef8aefa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.998+0000 7fb2a6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae69e0 0x55aef8aeffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35636/0 (socket says 172.21.15.47:35636) 2024-03-21T11:33:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.998+0000 7fb2a6a00640 1 -- 172.21.15.47:0/2001671745 learned_addr learned my addr 172.21.15.47:0/2001671745 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.998+0000 7fb2afe00640 1 --2- 172.21.15.47:0/2001671745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae73a0 0x55aef8af6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.999+0000 7fb2a6a00640 1 -- 172.21.15.47:0/2001671745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae73a0 msgr2=0x55aef8af6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.999+0000 7fb2a6a00640 1 --2- 172.21.15.47:0/2001671745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae73a0 0x55aef8af6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.999+0000 7fb2a6a00640 1 -- 172.21.15.47:0/2001671745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae5810 msgr2=0x55aef8aefa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.999+0000 7fb2a6a00640 1 --2- 172.21.15.47:0/2001671745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae5810 0x55aef8aefa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.999+0000 7fb2a6a00640 1 -- 172.21.15.47:0/2001671745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aef875e3f0 con 0x55aef8ae69e0 2024-03-21T11:33:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.999+0000 7fb2afe00640 1 --2- 172.21.15.47:0/2001671745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae73a0 0x55aef8af6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:33:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.999+0000 7fb2a6a00640 1 --2- 172.21.15.47:0/2001671745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae69e0 0x55aef8aeffa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb298011830 tx=0x7fb298011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03ba75577e1e12d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.999+0000 7fb29d600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:33:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.999+0000 7fb29d600640 1 -- 172.21.15.47:0/2001671745 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298010c00 con 0x55aef8ae69e0 2024-03-21T11:33:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.999+0000 7fb29d600640 1 -- 172.21.15.47:0/2001671745 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb298010da0 con 0x55aef8ae69e0 2024-03-21T11:33:28.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:27.999+0000 7fb2b93de740 1 -- 172.21.15.47:0/2001671745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2a8011de0 con 0x55aef8ae69e0 2024-03-21T11:33:28.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.000+0000 7fb29d600640 1 -- 172.21.15.47:0/2001671745 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb298025050 con 0x55aef8ae69e0 2024-03-21T11:33:28.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.000+0000 7fb2b93de740 10 client.18549 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:28.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.000+0000 7fb2b93de740 1 -- 172.21.15.47:0/2001671745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55aef88287d0 con 0x55aef8ae69e0 2024-03-21T11:33:28.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.000+0000 7fb29d600640 1 -- 172.21.15.47:0/2001671745 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2a8011de0 con 0x55aef8ae69e0 2024-03-21T11:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.000+0000 7fb29d600640 1 -- 172.21.15.47:0/2001671745 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb29801a410 con 0x55aef8ae69e0 2024-03-21T11:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.000+0000 7fb2b93de740 10 client.18549 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.000+0000 7fb2b93de740 10 client.18549 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.001+0000 7fb2b93de740 20 client.18549 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.001+0000 7fb2b93de740 1 --2- 172.21.15.47:0/2001671745 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aef8afb530 0x55aef8afd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.001+0000 7fb2b93de740 4 client.18549 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.001+0000 7fb2b93de740 1 -- 172.21.15.47:0/2001671745 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aef8a57800 con 0x55aef8afb530 2024-03-21T11:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.001+0000 7fb2afe00640 1 --2- 172.21.15.47:0/2001671745 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aef8afb530 0x55aef8afd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.001+0000 7fb2afe00640 1 --2- 172.21.15.47:0/2001671745 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aef8afb530 0x55aef8afd990 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d1d8a00c82a2fdf server_cookie=3f2cf3b243b38d33 in_seq=0 out_seq=0 2024-03-21T11:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.001+0000 7fb29d600640 10 client.18549 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.002+0000 7fb29d600640 1 -- 172.21.15.47:0/2001671745 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55aef8a57800 con 0x55aef8afb530 2024-03-21T11:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.002+0000 7fb29d600640 10 client.18549 handle_command_reply: tid=0 2024-03-21T11:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.002+0000 7fb2b93de740 1 client.18549 shutdown 2024-03-21T11:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.002+0000 7fb2b93de740 1 -- 172.21.15.47:0/2001671745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae69e0 msgr2=0x55aef8aeffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.002+0000 7fb2b93de740 1 --2- 172.21.15.47:0/2001671745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae69e0 0x55aef8aeffa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb298011830 tx=0x7fb298011cf0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.002+0000 7fb2b93de740 1 -- 172.21.15.47:0/2001671745 shutdown_connections 2024-03-21T11:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.002+0000 7fb2b93de740 1 --2- 172.21.15.47:0/2001671745 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aef8afb530 0x55aef8afd990 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.002+0000 7fb2b93de740 1 --2- 172.21.15.47:0/2001671745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae73a0 0x55aef8af6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.002+0000 7fb29d600640 0 client.18549 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.002+0000 7fb2b93de740 1 --2- 172.21.15.47:0/2001671745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae69e0 0x55aef8aeffa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.002+0000 7fb2b93de740 1 --2- 172.21.15.47:0/2001671745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae5810 0x55aef8aefa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.002+0000 7fb2b93de740 1 -- 172.21.15.47:0/2001671745 >> 172.21.15.47:0/2001671745 conn(0x55aef8ae0c00 msgr2=0x55aef8adfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.002+0000 7fb2b93de740 1 -- 172.21.15.47:0/2001671745 shutdown_connections 2024-03-21T11:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.003+0000 7fb2b93de740 1 -- 172.21.15.47:0/2001671745 wait complete. 2024-03-21T11:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.003+0000 7fb2b93de740 20 client.18549 trim_cache size 0 max 16384 2024-03-21T11:33:28.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.017+0000 7fb2b93de740 1 Processor -- start 2024-03-21T11:33:28.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.017+0000 7fb2b93de740 1 -- start start 2024-03-21T11:33:28.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.017+0000 7fb2b93de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae5810 0x55aef8ae6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.017+0000 7fb2b93de740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae69e0 0x55aef8af6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.017+0000 7fb2b93de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae73a0 0x55aef8ae9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.017+0000 7fb2b93de740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b01ac2e0 con 0x55aef8ae69e0 2024-03-21T11:33:28.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.017+0000 7fb2b93de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b01adc60 con 0x55aef8ae5810 2024-03-21T11:33:28.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.017+0000 7fb2b93de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b01ac940 con 0x55aef8ae73a0 2024-03-21T11:33:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.017+0000 7fb2a6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae69e0 0x55aef8af6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.017+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae5810 0x55aef8ae6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.017+0000 7fb2a6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae69e0 0x55aef8af6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35638/0 (socket says 172.21.15.47:35638) 2024-03-21T11:33:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.017+0000 7fb2a6a00640 1 -- 172.21.15.47:0/495950366 learned_addr learned my addr 172.21.15.47:0/495950366 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.018+0000 7fb2afe00640 1 --2- 172.21.15.47:0/495950366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae73a0 0x55aef8ae9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.018+0000 7fb2a6a00640 1 -- 172.21.15.47:0/495950366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae73a0 msgr2=0x55aef8ae9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.018+0000 7fb2a6a00640 1 --2- 172.21.15.47:0/495950366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae73a0 0x55aef8ae9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.018+0000 7fb2a6a00640 1 -- 172.21.15.47:0/495950366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae5810 msgr2=0x55aef8ae6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.018+0000 7fb2a6a00640 1 --2- 172.21.15.47:0/495950366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae5810 0x55aef8ae6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.018+0000 7fb2a6a00640 1 -- 172.21.15.47:0/495950366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aef8ae5410 con 0x55aef8ae69e0 2024-03-21T11:33:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.018+0000 7fb2a6a00640 1 --2- 172.21.15.47:0/495950366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae69e0 0x55aef8af6fa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb298000c00 tx=0x7fb29800b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cc7189ff33d08c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.019+0000 7fb29fe00640 1 -- 172.21.15.47:0/495950366 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29802d030 con 0x55aef8ae69e0 2024-03-21T11:33:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.019+0000 7fb29fe00640 1 -- 172.21.15.47:0/495950366 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb29801a3e0 con 0x55aef8ae69e0 2024-03-21T11:33:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.019+0000 7fb29fe00640 1 -- 172.21.15.47:0/495950366 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298005030 con 0x55aef8ae69e0 2024-03-21T11:33:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.019+0000 7fb2b93de740 1 -- 172.21.15.47:0/495950366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae69e0 msgr2=0x55aef8af6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.019+0000 7fb2b93de740 1 --2- 172.21.15.47:0/495950366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae69e0 0x55aef8af6fa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb298000c00 tx=0x7fb29800b540 comp rx=0 tx=0).stop 2024-03-21T11:33:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.019+0000 7fb2b93de740 1 -- 172.21.15.47:0/495950366 shutdown_connections 2024-03-21T11:33:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.019+0000 7fb2b93de740 1 --2- 172.21.15.47:0/495950366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae73a0 0x55aef8ae9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.019+0000 7fb2b93de740 1 --2- 172.21.15.47:0/495950366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae69e0 0x55aef8af6fa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.019+0000 7fb2b93de740 1 --2- 172.21.15.47:0/495950366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae5810 0x55aef8ae6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.019+0000 7fb2b93de740 1 -- 172.21.15.47:0/495950366 >> 172.21.15.47:0/495950366 conn(0x55aef8ae0c00 msgr2=0x55aef8ae46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.020+0000 7fb2b93de740 1 -- 172.21.15.47:0/495950366 shutdown_connections 2024-03-21T11:33:28.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.020+0000 7fb2b93de740 1 -- 172.21.15.47:0/495950366 wait complete. 2024-03-21T11:33:28.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.020+0000 7fb2b93de740 1 -- start start 2024-03-21T11:33:28.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.020+0000 7fb2b93de740 1 Processor -- start 2024-03-21T11:33:28.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2b93de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae5810 0x55aef8af31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2b93de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae69e0 0x55aef8af36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2b93de740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae73a0 0x55aef8af6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2b93de740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b01ad600 con 0x55aef8ae73a0 2024-03-21T11:33:28.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2b93de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b01acfa0 con 0x55aef8ae69e0 2024-03-21T11:33:28.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2b93de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b01aa4a0 con 0x55aef8ae5810 2024-03-21T11:33:28.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2b93de740 10 client.0 _ll_register_callbacks cb 0x7fb29800bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:28.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae5810 0x55aef8af31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2a6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae69e0 0x55aef8af36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2afe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae5810 0x55aef8af31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41718/0 (socket says 172.21.15.47:41718) 2024-03-21T11:33:28.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2afe00640 1 -- 172.21.15.47:0/3254941283 learned_addr learned my addr 172.21.15.47:0/3254941283 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2afe00640 1 --2- 172.21.15.47:0/3254941283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae73a0 0x55aef8af6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2a6a00640 1 -- 172.21.15.47:0/3254941283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae5810 msgr2=0x55aef8af31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2a6a00640 1 --2- 172.21.15.47:0/3254941283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae5810 0x55aef8af31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2a6a00640 1 -- 172.21.15.47:0/3254941283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae73a0 msgr2=0x55aef8af6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2a6a00640 1 --2- 172.21.15.47:0/3254941283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae73a0 0x55aef8af6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2a6a00640 1 -- 172.21.15.47:0/3254941283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aef8af6c30 con 0x55aef8ae69e0 2024-03-21T11:33:28.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.021+0000 7fb2afe00640 1 --2- 172.21.15.47:0/3254941283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae5810 0x55aef8af31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:28.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.022+0000 7fb2a6a00640 1 --2- 172.21.15.47:0/3254941283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae69e0 0x55aef8af36e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x55aef8adc230 tx=0x7fb29800b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ceb59da0b044f65 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.022+0000 7fb29d600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:33:28.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.022+0000 7fb29d600640 1 -- 172.21.15.47:0/3254941283 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298036070 con 0x55aef8ae69e0 2024-03-21T11:33:28.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.022+0000 7fb2b93de740 1 -- 172.21.15.47:0/3254941283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb29801a3e0 con 0x55aef8ae69e0 2024-03-21T11:33:28.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.022+0000 7fb29d600640 1 -- 172.21.15.47:0/3254941283 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb29801ac80 con 0x55aef8ae69e0 2024-03-21T11:33:28.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.022+0000 7fb29d600640 1 -- 172.21.15.47:0/3254941283 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb298041050 con 0x55aef8ae69e0 2024-03-21T11:33:28.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.022+0000 7fb2b93de740 10 client.28889 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:28.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.022+0000 7fb2b93de740 1 -- 172.21.15.47:0/3254941283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55aef8b57820 con 0x55aef8ae69e0 2024-03-21T11:33:28.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.023+0000 7fb29d600640 1 -- 172.21.15.47:0/3254941283 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb29801a3e0 con 0x55aef8ae69e0 2024-03-21T11:33:28.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.023+0000 7fb29d600640 1 -- 172.21.15.47:0/3254941283 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb2980049b0 con 0x55aef8ae69e0 2024-03-21T11:33:28.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.023+0000 7fb2b93de740 10 client.28889 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:28.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.023+0000 7fb2b93de740 10 client.28889 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:33:28.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.023+0000 7fb2b93de740 20 client.28889 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:28.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.023+0000 7fb2b93de740 1 --2- 172.21.15.47:0/3254941283 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aef8b58030 0x55aef8b5a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.023+0000 7fb2b93de740 4 client.28889 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:33:28.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.023+0000 7fb2b93de740 1 -- 172.21.15.47:0/3254941283 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aef8a51b50 con 0x55aef8b58030 2024-03-21T11:33:28.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.023+0000 7fb2afe00640 1 --2- 172.21.15.47:0/3254941283 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aef8b58030 0x55aef8b5a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb2afe00640 1 --2- 172.21.15.47:0/3254941283 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aef8b58030 0x55aef8b5a450 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3693406f208a285d server_cookie=b5e40b7cc5cf10e6 in_seq=0 out_seq=0 2024-03-21T11:33:28.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb29d600640 10 client.28889 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:33:28.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb29d600640 1 -- 172.21.15.47:0/3254941283 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aef8a51b50 con 0x55aef8b58030 2024-03-21T11:33:28.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb29d600640 10 client.28889 handle_command_reply: tid=0 2024-03-21T11:33:28.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb2b93de740 1 client.28889 shutdown 2024-03-21T11:33:28.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb2b93de740 1 -- 172.21.15.47:0/3254941283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae69e0 msgr2=0x55aef8af36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb2b93de740 1 --2- 172.21.15.47:0/3254941283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae69e0 0x55aef8af36e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x55aef8adc230 tx=0x7fb29800b630 comp rx=0 tx=0).stop 2024-03-21T11:33:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb2b93de740 1 -- 172.21.15.47:0/3254941283 shutdown_connections 2024-03-21T11:33:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb2b93de740 1 --2- 172.21.15.47:0/3254941283 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55aef8b58030 0x55aef8b5a450 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb2b93de740 1 --2- 172.21.15.47:0/3254941283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aef8ae73a0 0x55aef8af6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb2b93de740 1 --2- 172.21.15.47:0/3254941283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aef8ae69e0 0x55aef8af36e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb2b93de740 1 --2- 172.21.15.47:0/3254941283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aef8ae5810 0x55aef8af31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb29d600640 0 client.28889 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:33:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb2b93de740 1 -- 172.21.15.47:0/3254941283 >> 172.21.15.47:0/3254941283 conn(0x55aef8ae0c00 msgr2=0x55aef8adf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.024+0000 7fb2b93de740 1 -- 172.21.15.47:0/3254941283 shutdown_connections 2024-03-21T11:33:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.025+0000 7fb2b93de740 1 -- 172.21.15.47:0/3254941283 wait complete. 2024-03-21T11:33:28.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.025+0000 7fb2b93de740 20 client.28889 trim_cache size 0 max 16384 2024-03-21T11:33:28.029 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:33:28.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.025+0000 7fb2b6600640 1 -- 172.21.15.47:0/4087553508 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2940a2fa0 msgr2=0x7fb2940a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.025+0000 7fb2b6600640 1 --2- 172.21.15.47:0/4087553508 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2940a2fa0 0x7fb2940a5460 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00be060 tx=0x7fb2a00040d0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.025+0000 7fb2b6600640 1 -- 172.21.15.47:0/4087553508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b01038c0 msgr2=0x7fb2b0199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.025+0000 7fb2b6600640 1 --2- 172.21.15.47:0/4087553508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b01038c0 0x7fb2b0199bf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0000f80 tx=0x7fb2a0035d10 comp rx=0 tx=0).stop 2024-03-21T11:33:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.025+0000 7fb2b6600640 1 -- 172.21.15.47:0/4087553508 shutdown_connections 2024-03-21T11:33:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.025+0000 7fb2b6600640 1 --2- 172.21.15.47:0/4087553508 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2940a2fa0 0x7fb2940a5460 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.025+0000 7fb2b6600640 1 --2- 172.21.15.47:0/4087553508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b0105470 0x7fb2b01a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.025+0000 7fb2b6600640 1 --2- 172.21.15.47:0/4087553508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b0104ab0 0x7fb2b019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.025+0000 7fb2b6600640 1 --2- 172.21.15.47:0/4087553508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b01038c0 0x7fb2b0199bf0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.025+0000 7fb2b6600640 1 -- 172.21.15.47:0/4087553508 >> 172.21.15.47:0/4087553508 conn(0x7fb2b00fee50 msgr2=0x7fb2b0107e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.026+0000 7fb2b6600640 1 -- 172.21.15.47:0/4087553508 shutdown_connections 2024-03-21T11:33:28.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.026+0000 7fb2b6600640 1 -- 172.21.15.47:0/4087553508 wait complete. 2024-03-21T11:33:28.046 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:33:28.046 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:33:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.304+0000 7fc27e800640 1 -- 172.21.15.47:0/1082586504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2781038e0 msgr2=0x7fc278103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.304+0000 7fc27e800640 1 --2- 172.21.15.47:0/1082586504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2781038e0 0x7fc278103d00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc268009900 tx=0x7fc26802bc30 comp rx=0 tx=0).stop 2024-03-21T11:33:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.305+0000 7fc27e800640 1 -- 172.21.15.47:0/1082586504 shutdown_connections 2024-03-21T11:33:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.305+0000 7fc27e800640 1 --2- 172.21.15.47:0/1082586504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc278105490 0x7fc27810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.305+0000 7fc27e800640 1 --2- 172.21.15.47:0/1082586504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc278104ad0 0x7fc278104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.305+0000 7fc27e800640 1 --2- 172.21.15.47:0/1082586504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2781038e0 0x7fc278103d00 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.305+0000 7fc27e800640 1 -- 172.21.15.47:0/1082586504 >> 172.21.15.47:0/1082586504 conn(0x7fc2780fee50 msgr2=0x7fc278101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.305+0000 7fc27e800640 1 -- 172.21.15.47:0/1082586504 shutdown_connections 2024-03-21T11:33:28.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.305+0000 7fc27e800640 1 -- 172.21.15.47:0/1082586504 wait complete. 2024-03-21T11:33:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.306+0000 7fc27e800640 1 Processor -- start 2024-03-21T11:33:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.306+0000 7fc27e800640 1 -- start start 2024-03-21T11:33:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.306+0000 7fc27e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2781038e0 0x7fc278199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.306+0000 7fc27e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc278104ad0 0x7fc27819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.306+0000 7fc27e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc278105490 0x7fc2781a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.306+0000 7fc27e800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27810f580 con 0x7fc278104ad0 2024-03-21T11:33:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.306+0000 7fc27e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27810f400 con 0x7fc2781038e0 2024-03-21T11:33:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.306+0000 7fc27e800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27810f700 con 0x7fc278105490 2024-03-21T11:33:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.306+0000 7fc277400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2781038e0 0x7fc278199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.306+0000 7fc277e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc278105490 0x7fc2781a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.306+0000 7fc277400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2781038e0 0x7fc278199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52390/0 (socket says 172.21.15.47:52390) 2024-03-21T11:33:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.306+0000 7fc276a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc278104ad0 0x7fc27819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.306+0000 7fc277e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc278105490 0x7fc2781a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41746/0 (socket says 172.21.15.47:41746) 2024-03-21T11:33:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.306+0000 7fc277400640 1 -- 172.21.15.47:0/2297387023 learned_addr learned my addr 172.21.15.47:0/2297387023 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.307+0000 7fc277400640 1 -- 172.21.15.47:0/2297387023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc278105490 msgr2=0x7fc2781a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.307+0000 7fc277400640 1 --2- 172.21.15.47:0/2297387023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc278105490 0x7fc2781a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.307+0000 7fc277400640 1 -- 172.21.15.47:0/2297387023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc278104ad0 msgr2=0x7fc27819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.307+0000 7fc277400640 1 --2- 172.21.15.47:0/2297387023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc278104ad0 0x7fc27819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.307+0000 7fc277400640 1 -- 172.21.15.47:0/2297387023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2781a1850 con 0x7fc2781038e0 2024-03-21T11:33:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.307+0000 7fc277e00640 1 --2- 172.21.15.47:0/2297387023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc278105490 0x7fc2781a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.307+0000 7fc277400640 1 --2- 172.21.15.47:0/2297387023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2781038e0 0x7fc278199c40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc268000c00 tx=0x7fc26802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.308+0000 7fc26fe00640 1 -- 172.21.15.47:0/2297387023 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268043020 con 0x7fc2781038e0 2024-03-21T11:33:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.308+0000 7fc26fe00640 1 -- 172.21.15.47:0/2297387023 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2680054e0 con 0x7fc2781038e0 2024-03-21T11:33:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.308+0000 7fc27e800640 1 -- 172.21.15.47:0/2297387023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2781a1ae0 con 0x7fc2781038e0 2024-03-21T11:33:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.308+0000 7fc26fe00640 1 -- 172.21.15.47:0/2297387023 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268003c40 con 0x7fc2781038e0 2024-03-21T11:33:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.308+0000 7fc27e800640 1 -- 172.21.15.47:0/2297387023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2781a2080 con 0x7fc2781038e0 2024-03-21T11:33:28.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.311+0000 7fc26fe00640 1 -- 172.21.15.47:0/2297387023 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc268046020 con 0x7fc2781038e0 2024-03-21T11:33:28.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.311+0000 7fc2815a4740 1 Processor -- start 2024-03-21T11:33:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.311+0000 7fc2815a4740 1 -- start start 2024-03-21T11:33:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.312+0000 7fc2815a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da3810 0x562a45da3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.312+0000 7fc2815a4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da49e0 0x562a45da4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.312+0000 7fc2815a4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da53a0 0x562a45dabc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.312+0000 7fc2815a4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a45d082c0 con 0x562a45da49e0 2024-03-21T11:33:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.312+0000 7fc2815a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a45cb0c90 con 0x562a45da3810 2024-03-21T11:33:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.312+0000 7fc2815a4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a45d04360 con 0x562a45da53a0 2024-03-21T11:33:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.312+0000 7fc277e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da3810 0x562a45da3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.312+0000 7fc277e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da3810 0x562a45da3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52392/0 (socket says 172.21.15.47:52392) 2024-03-21T11:33:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.312+0000 7fc277e00640 1 -- 172.21.15.47:0/3201528493 learned_addr learned my addr 172.21.15.47:0/3201528493 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.312+0000 7fc276a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da53a0 0x562a45dabc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.312+0000 7fc277400640 1 --2- 172.21.15.47:0/3201528493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da49e0 0x562a45da4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.313+0000 7fc276a00640 1 -- 172.21.15.47:0/3201528493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da3810 msgr2=0x562a45da3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.313+0000 7fc276a00640 1 --2- 172.21.15.47:0/3201528493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da3810 0x562a45da3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.313+0000 7fc276a00640 1 -- 172.21.15.47:0/3201528493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da49e0 msgr2=0x562a45da4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.313+0000 7fc276a00640 1 --2- 172.21.15.47:0/3201528493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da49e0 0x562a45da4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.313+0000 7fc276a00640 1 -- 172.21.15.47:0/3201528493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a45cd5080 con 0x562a45da53a0 2024-03-21T11:33:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.313+0000 7fc276a00640 1 --2- 172.21.15.47:0/3201528493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da53a0 0x562a45dabc30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc264009c20 tx=0x7fc264011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5be86d7281346b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.313+0000 7fc26fe00640 1 --2- 172.21.15.47:0/2297387023 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc25c0a2fa0 0x7fc25c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.313+0000 7fc26cc00640 1 -- 172.21.15.47:0/3201528493 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264009dd0 con 0x562a45da53a0 2024-03-21T11:33:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.313+0000 7fc26cc00640 1 -- 172.21.15.47:0/3201528493 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc264005020 con 0x562a45da53a0 2024-03-21T11:33:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.313+0000 7fc26cc00640 1 -- 172.21.15.47:0/3201528493 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264004ba0 con 0x562a45da53a0 2024-03-21T11:33:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.313+0000 7fc26fe00640 1 -- 172.21.15.47:0/2297387023 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc2680f1790 con 0x7fc2781038e0 2024-03-21T11:33:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.314+0000 7fc277e00640 1 --2- 172.21.15.47:0/2297387023 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc25c0a2fa0 0x7fc25c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.314+0000 7fc2815a4740 1 -- 172.21.15.47:0/3201528493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da53a0 msgr2=0x562a45dabc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.314+0000 7fc2815a4740 1 --2- 172.21.15.47:0/3201528493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da53a0 0x562a45dabc30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc264009c20 tx=0x7fc264011810 comp rx=0 tx=0).stop 2024-03-21T11:33:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.314+0000 7fc2815a4740 1 -- 172.21.15.47:0/3201528493 shutdown_connections 2024-03-21T11:33:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.314+0000 7fc2815a4740 1 --2- 172.21.15.47:0/3201528493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da53a0 0x562a45dabc30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.314+0000 7fc2815a4740 1 --2- 172.21.15.47:0/3201528493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da49e0 0x562a45da4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.314+0000 7fc2815a4740 1 --2- 172.21.15.47:0/3201528493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da3810 0x562a45da3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.314+0000 7fc2815a4740 1 -- 172.21.15.47:0/3201528493 >> 172.21.15.47:0/3201528493 conn(0x562a45d9ec00 msgr2=0x562a45d9f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.314+0000 7fc277e00640 1 --2- 172.21.15.47:0/2297387023 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc25c0a2fa0 0x7fc25c0a5460 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x562a45da48e0 tx=0x7fc270009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.314+0000 7fc2815a4740 1 -- 172.21.15.47:0/3201528493 shutdown_connections 2024-03-21T11:33:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.314+0000 7fc2815a4740 1 -- 172.21.15.47:0/3201528493 wait complete. 2024-03-21T11:33:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.315+0000 7fc2815a4740 1 -- start start 2024-03-21T11:33:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.315+0000 7fc2815a4740 1 Processor -- start 2024-03-21T11:33:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc2815a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da3810 0x562a45dada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc2815a4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da49e0 0x562a45dadf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc2815a4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da53a0 0x562a45db4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc2815a4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a45dac3b0 con 0x562a45da53a0 2024-03-21T11:33:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc2815a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a45cc40d0 con 0x562a45da3810 2024-03-21T11:33:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc2815a4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a45dac530 con 0x562a45da49e0 2024-03-21T11:33:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc2815a4740 10 client.0 _ll_register_callbacks cb 0x562a45cc8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc276a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da49e0 0x562a45dadf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc277e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da53a0 0x562a45db4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc276a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da49e0 0x562a45dadf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41764/0 (socket says 172.21.15.47:41764) 2024-03-21T11:33:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc276a00640 1 -- 172.21.15.47:0/3827848739 learned_addr learned my addr 172.21.15.47:0/3827848739 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc277400640 1 --2- 172.21.15.47:0/3827848739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da3810 0x562a45dada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc276a00640 1 -- 172.21.15.47:0/3827848739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da3810 msgr2=0x562a45dada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc276a00640 1 --2- 172.21.15.47:0/3827848739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da3810 0x562a45dada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.316+0000 7fc276a00640 1 -- 172.21.15.47:0/3827848739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da53a0 msgr2=0x562a45db4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.317+0000 7fc276a00640 1 --2- 172.21.15.47:0/3827848739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da53a0 0x562a45db4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.317+0000 7fc276a00640 1 -- 172.21.15.47:0/3827848739 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a45a1c3f0 con 0x562a45da49e0 2024-03-21T11:33:28.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.317+0000 7fc277e00640 1 --2- 172.21.15.47:0/3827848739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da53a0 0x562a45db4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:28.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.317+0000 7fc276a00640 1 --2- 172.21.15.47:0/3827848739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da49e0 0x562a45dadf40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc26401b040 tx=0x7fc2640098f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7fbd7938e209ba4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.317+0000 7fc261600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:33:28.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.317+0000 7fc261600640 1 -- 172.21.15.47:0/3827848739 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264029020 con 0x562a45da49e0 2024-03-21T11:33:28.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.317+0000 7fc261600640 1 -- 172.21.15.47:0/3827848739 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc264004b30 con 0x562a45da49e0 2024-03-21T11:33:28.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.317+0000 7fc2815a4740 1 -- 172.21.15.47:0/3827848739 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc264005020 con 0x562a45da49e0 2024-03-21T11:33:28.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.318+0000 7fc261600640 1 -- 172.21.15.47:0/3827848739 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc264034050 con 0x562a45da49e0 2024-03-21T11:33:28.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.318+0000 7fc2815a4740 10 client.28921 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:28.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.318+0000 7fc2815a4740 1 -- 172.21.15.47:0/3827848739 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562a45ae67d0 con 0x562a45da49e0 2024-03-21T11:33:28.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.318+0000 7fc261600640 1 -- 172.21.15.47:0/3827848739 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc264005020 con 0x562a45da49e0 2024-03-21T11:33:28.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.318+0000 7fc261600640 1 -- 172.21.15.47:0/3827848739 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc264023420 con 0x562a45da49e0 2024-03-21T11:33:28.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.318+0000 7fc2815a4740 10 client.28921 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:28.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.318+0000 7fc2815a4740 10 client.28921 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:33:28.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.318+0000 7fc2815a4740 20 client.28921 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:28.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.318+0000 7fc2815a4740 1 --2- 172.21.15.47:0/3827848739 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x562a45db9300 0x562a45dbb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.318+0000 7fc2815a4740 4 client.28921 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:33:28.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.318+0000 7fc2815a4740 1 -- 172.21.15.47:0/3827848739 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a45d15800 con 0x562a45db9300 2024-03-21T11:33:28.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.319+0000 7fc277400640 1 --2- 172.21.15.47:0/3827848739 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x562a45db9300 0x562a45dbb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.319+0000 7fc277400640 1 --2- 172.21.15.47:0/3827848739 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x562a45db9300 0x562a45dbb710 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb1a51c30d75387a server_cookie=e574cf4199d62aef in_seq=0 out_seq=0 2024-03-21T11:33:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.319+0000 7fc261600640 10 client.28921 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:33:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.320+0000 7fc261600640 1 -- 172.21.15.47:0/3827848739 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562a45d15800 con 0x562a45db9300 2024-03-21T11:33:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.320+0000 7fc261600640 10 client.28921 handle_command_reply: tid=0 2024-03-21T11:33:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.320+0000 7fc2815a4740 1 client.28921 shutdown 2024-03-21T11:33:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.320+0000 7fc2815a4740 1 -- 172.21.15.47:0/3827848739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da49e0 msgr2=0x562a45dadf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.320+0000 7fc2815a4740 1 --2- 172.21.15.47:0/3827848739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da49e0 0x562a45dadf40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc26401b040 tx=0x7fc2640098f0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.320+0000 7fc2815a4740 1 -- 172.21.15.47:0/3827848739 shutdown_connections 2024-03-21T11:33:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.320+0000 7fc2815a4740 1 --2- 172.21.15.47:0/3827848739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da53a0 0x562a45db4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.320+0000 7fc2815a4740 1 --2- 172.21.15.47:0/3827848739 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x562a45db9300 0x562a45dbb710 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.320+0000 7fc2815a4740 1 --2- 172.21.15.47:0/3827848739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da49e0 0x562a45dadf40 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.320+0000 7fc2815a4740 1 --2- 172.21.15.47:0/3827848739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da3810 0x562a45dada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.320+0000 7fc2815a4740 1 -- 172.21.15.47:0/3827848739 >> 172.21.15.47:0/3827848739 conn(0x562a45d9ec00 msgr2=0x562a45d9deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.320+0000 7fc261600640 0 client.28921 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:33:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.321+0000 7fc2815a4740 1 -- 172.21.15.47:0/3827848739 shutdown_connections 2024-03-21T11:33:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.321+0000 7fc2815a4740 1 -- 172.21.15.47:0/3827848739 wait complete. 2024-03-21T11:33:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.321+0000 7fc2815a4740 20 client.28921 trim_cache size 0 max 16384 2024-03-21T11:33:28.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.335+0000 7fc2815a4740 1 Processor -- start 2024-03-21T11:33:28.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.335+0000 7fc2815a4740 1 -- start start 2024-03-21T11:33:28.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.335+0000 7fc2815a4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da3810 0x562a45dad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.335+0000 7fc2815a4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da49e0 0x562a45db4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.335+0000 7fc2815a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da53a0 0x562a45da7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.335+0000 7fc2815a4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781ac3d0 con 0x562a45da3810 2024-03-21T11:33:28.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.335+0000 7fc2815a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781add50 con 0x562a45da53a0 2024-03-21T11:33:28.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.335+0000 7fc2815a4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781aca30 con 0x562a45da49e0 2024-03-21T11:33:28.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.335+0000 7fc277e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da53a0 0x562a45da7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.335+0000 7fc276a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da49e0 0x562a45db4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.335+0000 7fc277e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da53a0 0x562a45da7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52408/0 (socket says 172.21.15.47:52408) 2024-03-21T11:33:28.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.335+0000 7fc277e00640 1 -- 172.21.15.47:0/2603426035 learned_addr learned my addr 172.21.15.47:0/2603426035 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.335+0000 7fc277400640 1 --2- 172.21.15.47:0/2603426035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da3810 0x562a45dad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.336+0000 7fc277e00640 1 -- 172.21.15.47:0/2603426035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da49e0 msgr2=0x562a45db4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.336+0000 7fc277e00640 1 --2- 172.21.15.47:0/2603426035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da49e0 0x562a45db4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.336+0000 7fc277e00640 1 -- 172.21.15.47:0/2603426035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da3810 msgr2=0x562a45dad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.336+0000 7fc277e00640 1 --2- 172.21.15.47:0/2603426035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da3810 0x562a45dad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.336+0000 7fc277e00640 1 -- 172.21.15.47:0/2603426035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a45d9f970 con 0x562a45da53a0 2024-03-21T11:33:28.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.336+0000 7fc277e00640 1 --2- 172.21.15.47:0/2603426035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da53a0 0x562a45da7c10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc270011d90 tx=0x7fc270016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c9d9f99ae3db896 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.336+0000 7fc263e00640 1 -- 172.21.15.47:0/2603426035 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27001c910 con 0x562a45da53a0 2024-03-21T11:33:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.336+0000 7fc263e00640 1 -- 172.21.15.47:0/2603426035 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc27002d070 con 0x562a45da53a0 2024-03-21T11:33:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.336+0000 7fc263e00640 1 -- 172.21.15.47:0/2603426035 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270028e30 con 0x562a45da53a0 2024-03-21T11:33:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.337+0000 7fc2815a4740 1 -- 172.21.15.47:0/2603426035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da53a0 msgr2=0x562a45da7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.337+0000 7fc2815a4740 1 --2- 172.21.15.47:0/2603426035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da53a0 0x562a45da7c10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc270011d90 tx=0x7fc270016000 comp rx=0 tx=0).stop 2024-03-21T11:33:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.337+0000 7fc2815a4740 1 -- 172.21.15.47:0/2603426035 shutdown_connections 2024-03-21T11:33:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.337+0000 7fc2815a4740 1 --2- 172.21.15.47:0/2603426035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da53a0 0x562a45da7c10 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.337+0000 7fc2815a4740 1 --2- 172.21.15.47:0/2603426035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da49e0 0x562a45db4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.337+0000 7fc2815a4740 1 --2- 172.21.15.47:0/2603426035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da3810 0x562a45dad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.337+0000 7fc2815a4740 1 -- 172.21.15.47:0/2603426035 >> 172.21.15.47:0/2603426035 conn(0x562a45d9ec00 msgr2=0x562a45da26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.337+0000 7fc2815a4740 1 -- 172.21.15.47:0/2603426035 shutdown_connections 2024-03-21T11:33:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.337+0000 7fc2815a4740 1 -- 172.21.15.47:0/2603426035 wait complete. 2024-03-21T11:33:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.338+0000 7fc2815a4740 1 -- start start 2024-03-21T11:33:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.338+0000 7fc2815a4740 1 Processor -- start 2024-03-21T11:33:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.338+0000 7fc2815a4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da3810 0x562a45db2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.338+0000 7fc2815a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da49e0 0x562a45db3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.338+0000 7fc2815a4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da53a0 0x562a45db4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.338+0000 7fc2815a4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781ad6f0 con 0x562a45da53a0 2024-03-21T11:33:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.338+0000 7fc2815a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781ad090 con 0x562a45da49e0 2024-03-21T11:33:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.338+0000 7fc2815a4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781aa590 con 0x562a45da3810 2024-03-21T11:33:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.338+0000 7fc2815a4740 10 client.0 _ll_register_callbacks cb 0x7fc270009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.338+0000 7fc277400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da3810 0x562a45db2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.338+0000 7fc276a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da49e0 0x562a45db3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.338+0000 7fc277400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da3810 0x562a45db2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41782/0 (socket says 172.21.15.47:41782) 2024-03-21T11:33:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.338+0000 7fc277400640 1 -- 172.21.15.47:0/2071659370 learned_addr learned my addr 172.21.15.47:0/2071659370 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.338+0000 7fc277e00640 1 --2- 172.21.15.47:0/2071659370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da53a0 0x562a45db4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.339+0000 7fc276a00640 1 -- 172.21.15.47:0/2071659370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da3810 msgr2=0x562a45db2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.339+0000 7fc276a00640 1 --2- 172.21.15.47:0/2071659370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da3810 0x562a45db2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.339+0000 7fc276a00640 1 -- 172.21.15.47:0/2071659370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da53a0 msgr2=0x562a45db4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.339+0000 7fc276a00640 1 --2- 172.21.15.47:0/2071659370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da53a0 0x562a45db4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.339+0000 7fc276a00640 1 -- 172.21.15.47:0/2071659370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a45db4840 con 0x562a45da49e0 2024-03-21T11:33:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.339+0000 7fc277400640 1 --2- 172.21.15.47:0/2071659370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da3810 0x562a45db2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.339+0000 7fc277e00640 1 --2- 172.21.15.47:0/2071659370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da53a0 0x562a45db4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.339+0000 7fc276a00640 1 --2- 172.21.15.47:0/2071659370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da49e0 0x562a45db3290 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc264001eb0 tx=0x7fc264023c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=582ddc8ff520d2ac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.339+0000 7fc261600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:33:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.339+0000 7fc261600640 1 -- 172.21.15.47:0/2071659370 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264029040 con 0x562a45da49e0 2024-03-21T11:33:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.339+0000 7fc2815a4740 1 -- 172.21.15.47:0/2071659370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc27002d070 con 0x562a45da49e0 2024-03-21T11:33:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.339+0000 7fc261600640 1 -- 172.21.15.47:0/2071659370 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc264004040 con 0x562a45da49e0 2024-03-21T11:33:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.340+0000 7fc261600640 1 -- 172.21.15.47:0/2071659370 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc264005750 con 0x562a45da49e0 2024-03-21T11:33:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.340+0000 7fc2815a4740 10 client.28907 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.340+0000 7fc2815a4740 1 -- 172.21.15.47:0/2071659370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562a45e152a0 con 0x562a45da49e0 2024-03-21T11:33:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.340+0000 7fc261600640 1 -- 172.21.15.47:0/2071659370 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc27002d070 con 0x562a45da49e0 2024-03-21T11:33:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.340+0000 7fc261600640 1 -- 172.21.15.47:0/2071659370 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc264004360 con 0x562a45da49e0 2024-03-21T11:33:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.340+0000 7fc2815a4740 10 client.28907 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.340+0000 7fc2815a4740 10 client.28907 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:33:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.340+0000 7fc2815a4740 20 client.28907 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.340+0000 7fc2815a4740 1 --2- 172.21.15.47:0/2071659370 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x562a45e15b60 0x562a45e17f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.340+0000 7fc2815a4740 4 client.28907 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:33:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.340+0000 7fc2815a4740 1 -- 172.21.15.47:0/2071659370 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562a45dad7f0 con 0x562a45e15b60 2024-03-21T11:33:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.340+0000 7fc277400640 1 --2- 172.21.15.47:0/2071659370 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x562a45e15b60 0x562a45e17f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.341+0000 7fc277400640 1 --2- 172.21.15.47:0/2071659370 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x562a45e15b60 0x562a45e17f80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e09d10934cea9b6 server_cookie=7469457c84fe8b24 in_seq=0 out_seq=0 2024-03-21T11:33:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.341+0000 7fc261600640 10 client.28907 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:33:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.341+0000 7fc261600640 1 -- 172.21.15.47:0/2071659370 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562a45dad7f0 con 0x562a45e15b60 2024-03-21T11:33:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.341+0000 7fc261600640 10 client.28907 handle_command_reply: tid=0 2024-03-21T11:33:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.341+0000 7fc2815a4740 1 client.28907 shutdown 2024-03-21T11:33:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.341+0000 7fc2815a4740 1 -- 172.21.15.47:0/2071659370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da49e0 msgr2=0x562a45db3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.341+0000 7fc2815a4740 1 --2- 172.21.15.47:0/2071659370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da49e0 0x562a45db3290 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc264001eb0 tx=0x7fc264023c80 comp rx=0 tx=0).stop 2024-03-21T11:33:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.341+0000 7fc2815a4740 1 -- 172.21.15.47:0/2071659370 shutdown_connections 2024-03-21T11:33:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.341+0000 7fc2815a4740 1 --2- 172.21.15.47:0/2071659370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a45da53a0 0x562a45db4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.341+0000 7fc2815a4740 1 --2- 172.21.15.47:0/2071659370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a45da49e0 0x562a45db3290 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.341+0000 7fc2815a4740 1 --2- 172.21.15.47:0/2071659370 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x562a45e15b60 0x562a45e17f80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.341+0000 7fc2815a4740 1 --2- 172.21.15.47:0/2071659370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a45da3810 0x562a45db2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.341+0000 7fc2815a4740 1 -- 172.21.15.47:0/2071659370 >> 172.21.15.47:0/2071659370 conn(0x562a45d9ec00 msgr2=0x562a45da26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.341+0000 7fc261600640 0 client.28907 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:33:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.342+0000 7fc2815a4740 1 -- 172.21.15.47:0/2071659370 shutdown_connections 2024-03-21T11:33:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.342+0000 7fc2815a4740 1 -- 172.21.15.47:0/2071659370 wait complete. 2024-03-21T11:33:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.342+0000 7fc2815a4740 20 client.28907 trim_cache size 0 max 16384 2024-03-21T11:33:28.350 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:33:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.342+0000 7fc27e800640 1 -- 172.21.15.47:0/2297387023 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc25c0a2fa0 msgr2=0x7fc25c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.342+0000 7fc27e800640 1 --2- 172.21.15.47:0/2297387023 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc25c0a2fa0 0x7fc25c0a5460 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x562a45da48e0 tx=0x7fc270009500 comp rx=0 tx=0).stop 2024-03-21T11:33:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.342+0000 7fc27e800640 1 -- 172.21.15.47:0/2297387023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2781038e0 msgr2=0x7fc278199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.342+0000 7fc27e800640 1 --2- 172.21.15.47:0/2297387023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2781038e0 0x7fc278199c40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc268000c00 tx=0x7fc26802c650 comp rx=0 tx=0).stop 2024-03-21T11:33:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.343+0000 7fc27e800640 1 -- 172.21.15.47:0/2297387023 shutdown_connections 2024-03-21T11:33:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.343+0000 7fc27e800640 1 --2- 172.21.15.47:0/2297387023 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc25c0a2fa0 0x7fc25c0a5460 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.343+0000 7fc27e800640 1 --2- 172.21.15.47:0/2297387023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc278105490 0x7fc2781a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.343+0000 7fc27e800640 1 --2- 172.21.15.47:0/2297387023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc278104ad0 0x7fc27819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.343+0000 7fc27e800640 1 --2- 172.21.15.47:0/2297387023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2781038e0 0x7fc278199c40 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.343+0000 7fc27e800640 1 -- 172.21.15.47:0/2297387023 >> 172.21.15.47:0/2297387023 conn(0x7fc2780fee50 msgr2=0x7fc278107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.343+0000 7fc27e800640 1 -- 172.21.15.47:0/2297387023 shutdown_connections 2024-03-21T11:33:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.343+0000 7fc27e800640 1 -- 172.21.15.47:0/2297387023 wait complete. 2024-03-21T11:33:28.361 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:33:28.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:33:28.551 INFO:tasks.workunit.client.0.smithi047.stdout: 1 749790 8.08 MB/sec execute 372 sec latency 20.335 ms 2024-03-21T11:33:28.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.563+0000 7f1892a00640 1 -- 172.21.15.47:0/1490325764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f188c10db70 msgr2=0x7f188c10ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.563+0000 7f1892a00640 1 --2- 172.21.15.47:0/1490325764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f188c10db70 0x7f188c10ff60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1884009ca0 tx=0x7f188402c0f0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.563+0000 7f1892a00640 1 -- 172.21.15.47:0/1490325764 shutdown_connections 2024-03-21T11:33:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.563+0000 7f1892a00640 1 --2- 172.21.15.47:0/1490325764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f188c10db70 0x7f188c10ff60 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.563+0000 7f1892a00640 1 --2- 172.21.15.47:0/1490325764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f188c074100 0x7f188c074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.563+0000 7f1892a00640 1 --2- 172.21.15.47:0/1490325764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f188c0756e0 0x7f188c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.563+0000 7f1892a00640 1 -- 172.21.15.47:0/1490325764 >> 172.21.15.47:0/1490325764 conn(0x7f188c0fcb10 msgr2=0x7f188c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.563+0000 7f1892a00640 1 -- 172.21.15.47:0/1490325764 shutdown_connections 2024-03-21T11:33:28.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.563+0000 7f1892a00640 1 -- 172.21.15.47:0/1490325764 wait complete. 2024-03-21T11:33:28.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.564+0000 7f1892a00640 1 Processor -- start 2024-03-21T11:33:28.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.564+0000 7f1892a00640 1 -- start start 2024-03-21T11:33:28.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1892a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f188c074100 0x7f188c19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1892a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f188c0756e0 0x7f188c19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1892a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f188c10db70 0x7f188c1a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1892a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c113810 con 0x7f188c0756e0 2024-03-21T11:33:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1892a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c113690 con 0x7f188c074100 2024-03-21T11:33:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1892a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c113990 con 0x7f188c10db70 2024-03-21T11:33:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1892000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f188c10db70 0x7f188c1a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1891600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f188c074100 0x7f188c19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1892000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f188c10db70 0x7f188c1a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41790/0 (socket says 172.21.15.47:41790) 2024-03-21T11:33:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1892000640 1 -- 172.21.15.47:0/181355100 learned_addr learned my addr 172.21.15.47:0/181355100 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1890c00640 1 --2- 172.21.15.47:0/181355100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f188c0756e0 0x7f188c19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1892000640 1 -- 172.21.15.47:0/181355100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f188c074100 msgr2=0x7f188c19de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1892000640 1 --2- 172.21.15.47:0/181355100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f188c074100 0x7f188c19de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1892000640 1 -- 172.21.15.47:0/181355100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f188c0756e0 msgr2=0x7f188c19e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1892000640 1 --2- 172.21.15.47:0/181355100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f188c0756e0 0x7f188c19e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1892000640 1 -- 172.21.15.47:0/181355100 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c1a5a50 con 0x7f188c10db70 2024-03-21T11:33:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.565+0000 7f1891600640 1 --2- 172.21.15.47:0/181355100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f188c074100 0x7f188c19de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.566+0000 7f1892000640 1 --2- 172.21.15.47:0/181355100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f188c10db70 0x7f188c1a53e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1884037040 tx=0x7f1884032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.566+0000 7f188a000640 1 -- 172.21.15.47:0/181355100 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884042020 con 0x7f188c10db70 2024-03-21T11:33:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.566+0000 7f188a000640 1 -- 172.21.15.47:0/181355100 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1884034640 con 0x7f188c10db70 2024-03-21T11:33:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.566+0000 7f188a000640 1 -- 172.21.15.47:0/181355100 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884039040 con 0x7f188c10db70 2024-03-21T11:33:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.566+0000 7f1892a00640 1 -- 172.21.15.47:0/181355100 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188c1a5ce0 con 0x7f188c10db70 2024-03-21T11:33:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.566+0000 7f1892a00640 1 -- 172.21.15.47:0/181355100 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c1a61d0 con 0x7f188c10db70 2024-03-21T11:33:28.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.570+0000 7f188a000640 1 -- 172.21.15.47:0/181355100 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1884045020 con 0x7f188c10db70 2024-03-21T11:33:28.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.570+0000 7f1896dfd740 1 Processor -- start 2024-03-21T11:33:28.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.570+0000 7f1896dfd740 1 -- start start 2024-03-21T11:33:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.570+0000 7f1896dfd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b26e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.570+0000 7f1896dfd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b28040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.570+0000 7f1896dfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 0x5587f5b2ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.570+0000 7f1896dfd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f5a8b2c0 con 0x5587f5b27bc0 2024-03-21T11:33:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.570+0000 7f1896dfd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f5a33c90 con 0x5587f5b269f0 2024-03-21T11:33:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.570+0000 7f1896dfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f5a87360 con 0x5587f5b28580 2024-03-21T11:33:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.571+0000 7f1890c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b26e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.571+0000 7f1890c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b26e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52440/0 (socket says 172.21.15.47:52440) 2024-03-21T11:33:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.571+0000 7f1890c00640 1 -- 172.21.15.47:0/255425412 learned_addr learned my addr 172.21.15.47:0/255425412 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.571+0000 7f1891600640 1 --2- 172.21.15.47:0/255425412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b28040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.571+0000 7f1890c00640 1 --2- 172.21.15.47:0/255425412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 0x5587f5b2ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.571+0000 7f1890c00640 1 -- 172.21.15.47:0/255425412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 msgr2=0x5587f5b2ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.571+0000 7f1890c00640 1 --2- 172.21.15.47:0/255425412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 0x5587f5b2ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.571+0000 7f1890c00640 1 -- 172.21.15.47:0/255425412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 msgr2=0x5587f5b28040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.571+0000 7f1890c00640 1 --2- 172.21.15.47:0/255425412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b28040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.571+0000 7f1890c00640 1 -- 172.21.15.47:0/255425412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587f5a58080 con 0x5587f5b269f0 2024-03-21T11:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.571+0000 7f188a000640 1 --2- 172.21.15.47:0/181355100 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18740a2fa0 0x7f18740a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.571+0000 7f188a000640 1 -- 172.21.15.47:0/181355100 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f18840f1560 con 0x7f188c10db70 2024-03-21T11:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.571+0000 7f1890c00640 1 --2- 172.21.15.47:0/255425412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b26e10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1878013800 tx=0x7f1878005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=699f806726d1875c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.572+0000 7f1891600640 1 --2- 172.21.15.47:0/181355100 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18740a2fa0 0x7f18740a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.572+0000 7f187e600640 1 -- 172.21.15.47:0/255425412 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878013e40 con 0x5587f5b269f0 2024-03-21T11:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.572+0000 7f187e600640 1 -- 172.21.15.47:0/255425412 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1878013120 con 0x5587f5b269f0 2024-03-21T11:33:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.572+0000 7f187e600640 1 -- 172.21.15.47:0/255425412 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18780035f0 con 0x5587f5b269f0 2024-03-21T11:33:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.573+0000 7f1896dfd740 1 -- 172.21.15.47:0/255425412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 msgr2=0x5587f5b26e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.573+0000 7f1896dfd740 1 --2- 172.21.15.47:0/255425412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b26e10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1878013800 tx=0x7f1878005850 comp rx=0 tx=0).stop 2024-03-21T11:33:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.573+0000 7f1891600640 1 --2- 172.21.15.47:0/181355100 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18740a2fa0 0x7f18740a5460 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1880012640 tx=0x7f1880005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.573+0000 7f1896dfd740 1 -- 172.21.15.47:0/255425412 shutdown_connections 2024-03-21T11:33:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.573+0000 7f1896dfd740 1 --2- 172.21.15.47:0/255425412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 0x5587f5b2ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.573+0000 7f1896dfd740 1 --2- 172.21.15.47:0/255425412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b28040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.573+0000 7f1896dfd740 1 --2- 172.21.15.47:0/255425412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b26e10 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.573+0000 7f1896dfd740 1 -- 172.21.15.47:0/255425412 >> 172.21.15.47:0/255425412 conn(0x5587f5b21de0 msgr2=0x5587f5b221e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.573+0000 7f1896dfd740 1 -- 172.21.15.47:0/255425412 shutdown_connections 2024-03-21T11:33:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.573+0000 7f1896dfd740 1 -- 172.21.15.47:0/255425412 wait complete. 2024-03-21T11:33:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.573+0000 7f1896dfd740 1 -- start start 2024-03-21T11:33:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.574+0000 7f1896dfd740 1 Processor -- start 2024-03-21T11:33:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.574+0000 7f1896dfd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b30c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.574+0000 7f1896dfd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b311b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.574+0000 7f1896dfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 0x5587f5b381f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.574+0000 7f1896dfd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f5b2f530 con 0x5587f5b27bc0 2024-03-21T11:33:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.574+0000 7f1896dfd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f5a470d0 con 0x5587f5b269f0 2024-03-21T11:33:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.574+0000 7f1896dfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f5b2f6b0 con 0x5587f5b28580 2024-03-21T11:33:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.574+0000 7f1896dfd740 10 client.0 _ll_register_callbacks cb 0x5587f5a4be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.574+0000 7f1891600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b30c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.574+0000 7f1890c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b311b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.574+0000 7f1892000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 0x5587f5b381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.574+0000 7f1891600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b30c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52456/0 (socket says 172.21.15.47:52456) 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.575+0000 7f1891600640 1 -- 172.21.15.47:0/4097806647 learned_addr learned my addr 172.21.15.47:0/4097806647 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.575+0000 7f1891600640 1 -- 172.21.15.47:0/4097806647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 msgr2=0x5587f5b381f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.575+0000 7f1891600640 1 --2- 172.21.15.47:0/4097806647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 0x5587f5b381f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.575+0000 7f1891600640 1 -- 172.21.15.47:0/4097806647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 msgr2=0x5587f5b311b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.575+0000 7f1891600640 1 --2- 172.21.15.47:0/4097806647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b311b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.575+0000 7f1891600640 1 -- 172.21.15.47:0/4097806647 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587f579f3f0 con 0x5587f5b269f0 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.575+0000 7f1892000640 1 --2- 172.21.15.47:0/4097806647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 0x5587f5b381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.575+0000 7f1891600640 1 --2- 172.21.15.47:0/4097806647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b30c70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1880003730 tx=0x7f1880004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acea14ca44501ef2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.575+0000 7f1872a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.575+0000 7f1872a00640 1 -- 172.21.15.47:0/4097806647 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188000f8d0 con 0x5587f5b269f0 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.576+0000 7f1896dfd740 1 -- 172.21.15.47:0/4097806647 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1878013120 con 0x5587f5b269f0 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.576+0000 7f1872a00640 1 -- 172.21.15.47:0/4097806647 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f188000fa70 con 0x5587f5b269f0 2024-03-21T11:33:28.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.576+0000 7f1872a00640 1 -- 172.21.15.47:0/4097806647 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f188000fc70 con 0x5587f5b269f0 2024-03-21T11:33:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.576+0000 7f1896dfd740 10 client.28919 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.576+0000 7f1896dfd740 1 -- 172.21.15.47:0/4097806647 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5587f58697d0 con 0x5587f5b269f0 2024-03-21T11:33:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.576+0000 7f1872a00640 1 -- 172.21.15.47:0/4097806647 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1878013120 con 0x5587f5b269f0 2024-03-21T11:33:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.576+0000 7f1872a00640 1 -- 172.21.15.47:0/4097806647 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1880027070 con 0x5587f5b269f0 2024-03-21T11:33:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.576+0000 7f1896dfd740 10 client.28919 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.576+0000 7f1896dfd740 10 client.28919 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:33:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.576+0000 7f1896dfd740 20 client.28919 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.576+0000 7f1896dfd740 1 --2- 172.21.15.47:0/4097806647 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5587f5b3c730 0x5587f5b3eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.576+0000 7f1896dfd740 4 client.28919 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:33:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.577+0000 7f1896dfd740 1 -- 172.21.15.47:0/4097806647 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587f5a98800 con 0x5587f5b3c730 2024-03-21T11:33:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.577+0000 7f1890c00640 1 --2- 172.21.15.47:0/4097806647 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5587f5b3c730 0x5587f5b3eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.577+0000 7f1890c00640 1 --2- 172.21.15.47:0/4097806647 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5587f5b3c730 0x5587f5b3eb80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ed8c80b1909802f server_cookie=765e49a7f6f40439 in_seq=0 out_seq=0 2024-03-21T11:33:28.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.577+0000 7f1872a00640 10 client.28919 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:33:28.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.578+0000 7f1872a00640 1 -- 172.21.15.47:0/4097806647 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5587f5a98800 con 0x5587f5b3c730 2024-03-21T11:33:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.578+0000 7f1872a00640 10 client.28919 handle_command_reply: tid=0 2024-03-21T11:33:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.578+0000 7f1896dfd740 1 client.28919 shutdown 2024-03-21T11:33:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.578+0000 7f1896dfd740 1 -- 172.21.15.47:0/4097806647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 msgr2=0x5587f5b30c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.578+0000 7f1896dfd740 1 --2- 172.21.15.47:0/4097806647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b30c70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1880003730 tx=0x7f1880004730 comp rx=0 tx=0).stop 2024-03-21T11:33:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.578+0000 7f1896dfd740 1 -- 172.21.15.47:0/4097806647 shutdown_connections 2024-03-21T11:33:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.578+0000 7f1896dfd740 1 --2- 172.21.15.47:0/4097806647 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5587f5b3c730 0x5587f5b3eb80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.578+0000 7f1896dfd740 1 --2- 172.21.15.47:0/4097806647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 0x5587f5b381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.578+0000 7f1896dfd740 1 --2- 172.21.15.47:0/4097806647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b311b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.578+0000 7f1896dfd740 1 --2- 172.21.15.47:0/4097806647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b30c70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.578+0000 7f1872a00640 0 client.28919 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:33:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.578+0000 7f1896dfd740 1 -- 172.21.15.47:0/4097806647 >> 172.21.15.47:0/4097806647 conn(0x5587f5b21de0 msgr2=0x5587f5b21160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.579+0000 7f1896dfd740 1 -- 172.21.15.47:0/4097806647 shutdown_connections 2024-03-21T11:33:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.579+0000 7f1896dfd740 1 -- 172.21.15.47:0/4097806647 wait complete. 2024-03-21T11:33:28.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.579+0000 7f1896dfd740 20 client.28919 trim_cache size 0 max 16384 2024-03-21T11:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.597+0000 7f1896dfd740 1 Processor -- start 2024-03-21T11:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.597+0000 7f1896dfd740 1 -- start start 2024-03-21T11:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.597+0000 7f1896dfd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b25750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.597+0000 7f1896dfd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.597+0000 7f1896dfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 0x5587f5b2ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.597+0000 7f1896dfd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c1b04a0 con 0x5587f5b27bc0 2024-03-21T11:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.597+0000 7f1896dfd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c1b1e20 con 0x5587f5b269f0 2024-03-21T11:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.597+0000 7f1896dfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c1b0b00 con 0x5587f5b28580 2024-03-21T11:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.597+0000 7f1891600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b25750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.597+0000 7f1891600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b25750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52464/0 (socket says 172.21.15.47:52464) 2024-03-21T11:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.597+0000 7f1891600640 1 -- 172.21.15.47:0/1143680695 learned_addr learned my addr 172.21.15.47:0/1143680695 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.597+0000 7f1890c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.597+0000 7f1892000640 1 --2- 172.21.15.47:0/1143680695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 0x5587f5b2ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.598+0000 7f1891600640 1 -- 172.21.15.47:0/1143680695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 msgr2=0x5587f5b2ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.598+0000 7f1891600640 1 --2- 172.21.15.47:0/1143680695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 0x5587f5b2ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.598+0000 7f1891600640 1 -- 172.21.15.47:0/1143680695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 msgr2=0x5587f5b381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.598+0000 7f1891600640 1 --2- 172.21.15.47:0/1143680695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.598+0000 7f1891600640 1 -- 172.21.15.47:0/1143680695 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587f5b22d50 con 0x5587f5b269f0 2024-03-21T11:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.598+0000 7f1892000640 1 --2- 172.21.15.47:0/1143680695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 0x5587f5b2ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.598+0000 7f1890c00640 1 --2- 172.21.15.47:0/1143680695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.598+0000 7f1891600640 1 --2- 172.21.15.47:0/1143680695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b25750 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1880012f70 tx=0x7f1880009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10a490a78597d9a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.598+0000 7f187dc00640 1 -- 172.21.15.47:0/1143680695 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188000fe10 con 0x5587f5b269f0 2024-03-21T11:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.598+0000 7f187dc00640 1 -- 172.21.15.47:0/1143680695 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1880005500 con 0x5587f5b269f0 2024-03-21T11:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.599+0000 7f187dc00640 1 -- 172.21.15.47:0/1143680695 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18800047a0 con 0x5587f5b269f0 2024-03-21T11:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.599+0000 7f1896dfd740 1 -- 172.21.15.47:0/1143680695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 msgr2=0x5587f5b25750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.599+0000 7f1896dfd740 1 --2- 172.21.15.47:0/1143680695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b25750 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1880012f70 tx=0x7f1880009f50 comp rx=0 tx=0).stop 2024-03-21T11:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.599+0000 7f1896dfd740 1 -- 172.21.15.47:0/1143680695 shutdown_connections 2024-03-21T11:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.599+0000 7f1896dfd740 1 --2- 172.21.15.47:0/1143680695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b28580 0x5587f5b2ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.599+0000 7f1896dfd740 1 --2- 172.21.15.47:0/1143680695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.599+0000 7f1896dfd740 1 --2- 172.21.15.47:0/1143680695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b269f0 0x5587f5b25750 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.599+0000 7f1896dfd740 1 -- 172.21.15.47:0/1143680695 >> 172.21.15.47:0/1143680695 conn(0x5587f5b21de0 msgr2=0x5587f5b25040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.600+0000 7f1896dfd740 1 -- 172.21.15.47:0/1143680695 shutdown_connections 2024-03-21T11:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.600+0000 7f1896dfd740 1 -- 172.21.15.47:0/1143680695 wait complete. 2024-03-21T11:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.600+0000 7f1896dfd740 1 -- start start 2024-03-21T11:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.600+0000 7f1896dfd740 1 Processor -- start 2024-03-21T11:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1896dfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b269f0 0x5587f5b379f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1896dfd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b37f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1896dfd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b28580 0x5587f5b344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1896dfd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c1b17c0 con 0x5587f5b27bc0 2024-03-21T11:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1896dfd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c1b1160 con 0x5587f5b28580 2024-03-21T11:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1896dfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c1ae660 con 0x5587f5b269f0 2024-03-21T11:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1896dfd740 10 client.0 _ll_register_callbacks cb 0x7f1880009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1891600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b269f0 0x5587f5b379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1891600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b269f0 0x5587f5b379f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41828/0 (socket says 172.21.15.47:41828) 2024-03-21T11:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1891600640 1 -- 172.21.15.47:0/1073919202 learned_addr learned my addr 172.21.15.47:0/1073919202 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1892000640 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b28580 0x5587f5b344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1890c00640 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b37f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1891600640 1 -- 172.21.15.47:0/1073919202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b28580 msgr2=0x5587f5b344e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1891600640 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b28580 0x5587f5b344e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1891600640 1 -- 172.21.15.47:0/1073919202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 msgr2=0x5587f5b37f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1891600640 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b37f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1891600640 1 -- 172.21.15.47:0/1073919202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587f5b34b80 con 0x5587f5b269f0 2024-03-21T11:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1892000640 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b28580 0x5587f5b344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.601+0000 7f1890c00640 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b37f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.602+0000 7f1891600640 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b269f0 0x5587f5b379f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f188001f670 tx=0x7f18800059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb54f70680fb7ee8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.602+0000 7f1872a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.602+0000 7f1872a00640 1 -- 172.21.15.47:0/1073919202 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1880023070 con 0x5587f5b269f0 2024-03-21T11:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.602+0000 7f1872a00640 1 -- 172.21.15.47:0/1073919202 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1880005230 con 0x5587f5b269f0 2024-03-21T11:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.602+0000 7f1896dfd740 1 -- 172.21.15.47:0/1073919202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1880005500 con 0x5587f5b269f0 2024-03-21T11:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.602+0000 7f1872a00640 1 -- 172.21.15.47:0/1073919202 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f188000fe10 con 0x5587f5b269f0 2024-03-21T11:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.602+0000 7f1896dfd740 10 client.28945 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.602+0000 7f1896dfd740 1 -- 172.21.15.47:0/1073919202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5587f5b98bf0 con 0x5587f5b269f0 2024-03-21T11:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.602+0000 7f1872a00640 1 -- 172.21.15.47:0/1073919202 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1880005500 con 0x5587f5b269f0 2024-03-21T11:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.603+0000 7f1872a00640 1 -- 172.21.15.47:0/1073919202 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1880027070 con 0x5587f5b269f0 2024-03-21T11:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.603+0000 7f1896dfd740 10 client.28945 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.603+0000 7f1896dfd740 10 client.28945 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.603+0000 7f1896dfd740 20 client.28945 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.603+0000 7f1896dfd740 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5587f5b994f0 0x5587f5b9b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.603+0000 7f1896dfd740 4 client.28945 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.603+0000 7f1896dfd740 1 -- 172.21.15.47:0/1073919202 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5587f5a92b50 con 0x5587f5b994f0 2024-03-21T11:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.603+0000 7f1890c00640 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5587f5b994f0 0x5587f5b9b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.603+0000 7f1890c00640 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5587f5b994f0 0x5587f5b9b910 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38c19c097ad0b4c3 server_cookie=61bd116880912e35 in_seq=0 out_seq=0 2024-03-21T11:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.603+0000 7f1872a00640 10 client.28945 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1872a00640 1 -- 172.21.15.47:0/1073919202 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5587f5a92b50 con 0x5587f5b994f0 2024-03-21T11:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1872a00640 10 client.28945 handle_command_reply: tid=0 2024-03-21T11:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1896dfd740 1 client.28945 shutdown 2024-03-21T11:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1896dfd740 1 -- 172.21.15.47:0/1073919202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b269f0 msgr2=0x5587f5b379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1896dfd740 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b269f0 0x5587f5b379f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f188001f670 tx=0x7f18800059b0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1896dfd740 1 -- 172.21.15.47:0/1073919202 shutdown_connections 2024-03-21T11:33:28.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1896dfd740 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5587f5b994f0 0x5587f5b9b910 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1896dfd740 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5587f5b28580 0x5587f5b344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1896dfd740 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5587f5b27bc0 0x5587f5b37f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1896dfd740 1 --2- 172.21.15.47:0/1073919202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587f5b269f0 0x5587f5b379f0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1872a00640 0 client.28945 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:33:28.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1896dfd740 1 -- 172.21.15.47:0/1073919202 >> 172.21.15.47:0/1073919202 conn(0x5587f5b21de0 msgr2=0x5587f5b258d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1896dfd740 1 -- 172.21.15.47:0/1073919202 shutdown_connections 2024-03-21T11:33:28.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1896dfd740 1 -- 172.21.15.47:0/1073919202 wait complete. 2024-03-21T11:33:28.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.604+0000 7f1896dfd740 20 client.28945 trim_cache size 0 max 16384 2024-03-21T11:33:28.610 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:33:28.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.605+0000 7f1892a00640 1 -- 172.21.15.47:0/181355100 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18740a2fa0 msgr2=0x7f18740a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.605+0000 7f1892a00640 1 --2- 172.21.15.47:0/181355100 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18740a2fa0 0x7f18740a5460 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1880012640 tx=0x7f1880005f90 comp rx=0 tx=0).stop 2024-03-21T11:33:28.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.605+0000 7f1892a00640 1 -- 172.21.15.47:0/181355100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f188c10db70 msgr2=0x7f188c1a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.605+0000 7f1892a00640 1 --2- 172.21.15.47:0/181355100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f188c10db70 0x7f188c1a53e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1884037040 tx=0x7f1884032f20 comp rx=0 tx=0).stop 2024-03-21T11:33:28.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.605+0000 7f1892a00640 1 -- 172.21.15.47:0/181355100 shutdown_connections 2024-03-21T11:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.605+0000 7f1892a00640 1 --2- 172.21.15.47:0/181355100 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18740a2fa0 0x7f18740a5460 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.605+0000 7f1892a00640 1 --2- 172.21.15.47:0/181355100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f188c10db70 0x7f188c1a53e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.605+0000 7f1892a00640 1 --2- 172.21.15.47:0/181355100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f188c0756e0 0x7f188c19e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.605+0000 7f1892a00640 1 --2- 172.21.15.47:0/181355100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f188c074100 0x7f188c19de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.605+0000 7f1892a00640 1 -- 172.21.15.47:0/181355100 >> 172.21.15.47:0/181355100 conn(0x7f188c0fcb10 msgr2=0x7f188c100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.605+0000 7f1892a00640 1 -- 172.21.15.47:0/181355100 shutdown_connections 2024-03-21T11:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.605+0000 7f1892a00640 1 -- 172.21.15.47:0/181355100 wait complete. 2024-03-21T11:33:28.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:28 smithi047 ceph-mon[31351]: pgmap v704: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 283 op/s 2024-03-21T11:33:28.624 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:33:28.625 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:33:28.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:28 smithi102 ceph-mon[36748]: pgmap v704: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 283 op/s 2024-03-21T11:33:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.840+0000 7f0ed7200640 1 -- 172.21.15.47:0/2050988535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed8075660 msgr2=0x7f0ed8073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.840+0000 7f0ed7200640 1 --2- 172.21.15.47:0/2050988535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed8075660 0x7f0ed8073ab0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009900 tx=0x7f0ec802bcc0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.840+0000 7f0ed7200640 1 -- 172.21.15.47:0/2050988535 shutdown_connections 2024-03-21T11:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.840+0000 7f0ed7200640 1 --2- 172.21.15.47:0/2050988535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed8109450 0x7f0ed810b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.840+0000 7f0ed7200640 1 --2- 172.21.15.47:0/2050988535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ed8074080 0x7f0ed80744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.840+0000 7f0ed7200640 1 --2- 172.21.15.47:0/2050988535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed8075660 0x7f0ed8073ab0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.840+0000 7f0ed7200640 1 -- 172.21.15.47:0/2050988535 >> 172.21.15.47:0/2050988535 conn(0x7f0ed80fcb10 msgr2=0x7f0ed80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.840+0000 7f0ed7200640 1 -- 172.21.15.47:0/2050988535 shutdown_connections 2024-03-21T11:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.841+0000 7f0ed7200640 1 -- 172.21.15.47:0/2050988535 wait complete. 2024-03-21T11:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.842+0000 7f0ed7200640 1 Processor -- start 2024-03-21T11:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.842+0000 7f0ed7200640 1 -- start start 2024-03-21T11:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.842+0000 7f0ed7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ed8074080 0x7f0ed8104940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.842+0000 7f0ed7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed8075660 0x7f0ed8104e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.842+0000 7f0ed7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed8109450 0x7f0ed81014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.842+0000 7f0ed7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed810f420 con 0x7f0ed8074080 2024-03-21T11:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.842+0000 7f0ed7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed810f2a0 con 0x7f0ed8075660 2024-03-21T11:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.842+0000 7f0ed7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed810f5a0 con 0x7f0ed8109450 2024-03-21T11:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.843+0000 7f0ed5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed8075660 0x7f0ed8104e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.843+0000 7f0ed5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed8075660 0x7f0ed8104e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52476/0 (socket says 172.21.15.47:52476) 2024-03-21T11:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.843+0000 7f0ed5400640 1 -- 172.21.15.47:0/538891624 learned_addr learned my addr 172.21.15.47:0/538891624 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.843+0000 7f0ed5e00640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ed8074080 0x7f0ed8104940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.843+0000 7f0ed6800640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed8109450 0x7f0ed81014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.843+0000 7f0ed5400640 1 -- 172.21.15.47:0/538891624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed8109450 msgr2=0x7f0ed81014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.843+0000 7f0ed5400640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed8109450 0x7f0ed81014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.843+0000 7f0ed5400640 1 -- 172.21.15.47:0/538891624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ed8074080 msgr2=0x7f0ed8104940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.843+0000 7f0ed5400640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ed8074080 0x7f0ed8104940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.843+0000 7f0ed5400640 1 -- 172.21.15.47:0/538891624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed8101d70 con 0x7f0ed8075660 2024-03-21T11:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.843+0000 7f0ed5e00640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ed8074080 0x7f0ed8104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.844+0000 7f0ed5400640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed8075660 0x7f0ed8104e80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400d950 tx=0x7f0ec400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.844+0000 7f0ecea00640 1 -- 172.21.15.47:0/538891624 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4017070 con 0x7f0ed8075660 2024-03-21T11:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.844+0000 7f0ecea00640 1 -- 172.21.15.47:0/538891624 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ec4009c30 con 0x7f0ed8075660 2024-03-21T11:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.844+0000 7f0ed7200640 1 -- 172.21.15.47:0/538891624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed81a5ec0 con 0x7f0ed8075660 2024-03-21T11:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.844+0000 7f0ecea00640 1 -- 172.21.15.47:0/538891624 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400f6b0 con 0x7f0ed8075660 2024-03-21T11:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.844+0000 7f0ed7200640 1 -- 172.21.15.47:0/538891624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed81a62c0 con 0x7f0ed8075660 2024-03-21T11:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.849+0000 7f0ecea00640 1 -- 172.21.15.47:0/538891624 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0ec4010460 con 0x7f0ed8075660 2024-03-21T11:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.849+0000 7f0edf7b5740 1 Processor -- start 2024-03-21T11:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.849+0000 7f0edf7b5740 1 -- start start 2024-03-21T11:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.849+0000 7f0edf7b5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aee9f0 0x55d6a7aeee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.849+0000 7f0edf7b5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7aefbc0 0x55d6a7af0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.849+0000 7f0edf7b5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7af0580 0x55d6a7af6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.849+0000 7f0edf7b5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6a7a532c0 con 0x55d6a7aee9f0 2024-03-21T11:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.849+0000 7f0edf7b5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6a79fbc90 con 0x55d6a7aefbc0 2024-03-21T11:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.849+0000 7f0edf7b5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6a7a4f360 con 0x55d6a7af0580 2024-03-21T11:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.850+0000 7f0ed5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7aefbc0 0x55d6a7af0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.850+0000 7f0ed5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7af0580 0x55d6a7af6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.850+0000 7f0ed5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7aefbc0 0x55d6a7af0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52482/0 (socket says 172.21.15.47:52482) 2024-03-21T11:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.850+0000 7f0ed6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aee9f0 0x55d6a7aeee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.850+0000 7f0ed5e00640 1 -- 172.21.15.47:0/2736312899 learned_addr learned my addr 172.21.15.47:0/2736312899 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.850+0000 7f0ed5e00640 1 -- 172.21.15.47:0/2736312899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7af0580 msgr2=0x55d6a7af6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.850+0000 7f0ed5e00640 1 --2- 172.21.15.47:0/2736312899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7af0580 0x55d6a7af6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.850+0000 7f0ed5e00640 1 -- 172.21.15.47:0/2736312899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aee9f0 msgr2=0x55d6a7aeee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.850+0000 7f0ed5e00640 1 --2- 172.21.15.47:0/2736312899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aee9f0 0x55d6a7aeee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.850+0000 7f0ed5e00640 1 -- 172.21.15.47:0/2736312899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6a7a20080 con 0x55d6a7aefbc0 2024-03-21T11:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.851+0000 7f0ed5e00640 1 --2- 172.21.15.47:0/2736312899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7aefbc0 0x55d6a7af0040 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8000c00 tx=0x7f0ec8009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43040ea8f2adc18b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.851+0000 7f0ecea00640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0ebc0a2fa0 0x7f0ebc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.851+0000 7f0ecea00640 1 -- 172.21.15.47:0/538891624 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0ec40cf360 con 0x7f0ed8075660 2024-03-21T11:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.851+0000 7f0ec3000640 1 -- 172.21.15.47:0/2736312899 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8044020 con 0x55d6a7aefbc0 2024-03-21T11:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.851+0000 7f0ec3000640 1 -- 172.21.15.47:0/2736312899 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ec802cc10 con 0x55d6a7aefbc0 2024-03-21T11:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.851+0000 7f0ec3000640 1 -- 172.21.15.47:0/2736312899 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80048c0 con 0x55d6a7aefbc0 2024-03-21T11:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.851+0000 7f0ed6800640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0ebc0a2fa0 0x7f0ebc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.852+0000 7f0edf7b5740 1 -- 172.21.15.47:0/2736312899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7aefbc0 msgr2=0x55d6a7af0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.852+0000 7f0ed6800640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0ebc0a2fa0 0x7f0ebc0a5460 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55d6a7aefac0 tx=0x7f0ed0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.852+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/2736312899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7aefbc0 0x55d6a7af0040 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8000c00 tx=0x7f0ec8009560 comp rx=0 tx=0).stop 2024-03-21T11:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.852+0000 7f0edf7b5740 1 -- 172.21.15.47:0/2736312899 shutdown_connections 2024-03-21T11:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.852+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/2736312899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7af0580 0x55d6a7af6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.852+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/2736312899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7aefbc0 0x55d6a7af0040 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.852+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/2736312899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aee9f0 0x55d6a7aeee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.852+0000 7f0edf7b5740 1 -- 172.21.15.47:0/2736312899 >> 172.21.15.47:0/2736312899 conn(0x55d6a7ae9de0 msgr2=0x55d6a7aea1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.852+0000 7f0edf7b5740 1 -- 172.21.15.47:0/2736312899 shutdown_connections 2024-03-21T11:33:28.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.852+0000 7f0edf7b5740 1 -- 172.21.15.47:0/2736312899 wait complete. 2024-03-21T11:33:28.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.852+0000 7f0edf7b5740 1 -- start start 2024-03-21T11:33:28.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.853+0000 7f0edf7b5740 1 Processor -- start 2024-03-21T11:33:28.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.853+0000 7f0edf7b5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7aee9f0 0x55d6a7af8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.853+0000 7f0edf7b5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 0x55d6a7af90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.853+0000 7f0edf7b5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7af0580 0x55d6a7b00100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.853+0000 7f0edf7b5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6a7af7530 con 0x55d6a7aefbc0 2024-03-21T11:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.853+0000 7f0edf7b5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6a7a0f0d0 con 0x55d6a7af0580 2024-03-21T11:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.853+0000 7f0edf7b5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6a7af76b0 con 0x55d6a7aee9f0 2024-03-21T11:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.853+0000 7f0edf7b5740 10 client.0 _ll_register_callbacks cb 0x55d6a7a13e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.853+0000 7f0ed5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7aee9f0 0x55d6a7af8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.853+0000 7f0ed5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 0x55d6a7af90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.854+0000 7f0ed5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7aee9f0 0x55d6a7af8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41854/0 (socket says 172.21.15.47:41854) 2024-03-21T11:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.854+0000 7f0ed5e00640 1 -- 172.21.15.47:0/2403608005 learned_addr learned my addr 172.21.15.47:0/2403608005 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.854+0000 7f0ed6800640 1 --2- 172.21.15.47:0/2403608005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7af0580 0x55d6a7b00100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.854+0000 7f0ed5e00640 1 -- 172.21.15.47:0/2403608005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7af0580 msgr2=0x55d6a7b00100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.854+0000 7f0ed5e00640 1 --2- 172.21.15.47:0/2403608005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7af0580 0x55d6a7b00100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.854+0000 7f0ed5e00640 1 -- 172.21.15.47:0/2403608005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 msgr2=0x55d6a7af90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.854+0000 7f0ed5e00640 1 --2- 172.21.15.47:0/2403608005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 0x55d6a7af90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.854+0000 7f0ed5e00640 1 -- 172.21.15.47:0/2403608005 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6a77673f0 con 0x55d6a7aee9f0 2024-03-21T11:33:28.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.854+0000 7f0ed5400640 1 --2- 172.21.15.47:0/2403608005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 0x55d6a7af90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:28.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.854+0000 7f0ed5e00640 1 --2- 172.21.15.47:0/2403608005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7aee9f0 0x55d6a7af8b80 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8036040 tx=0x7f0ec8040c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=758daffffc06a0d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.854+0000 7f0eb7400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:33:28.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.855+0000 7f0eb7400640 1 -- 172.21.15.47:0/2403608005 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec804f070 con 0x55d6a7aee9f0 2024-03-21T11:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.855+0000 7f0eb7400640 1 -- 172.21.15.47:0/2403608005 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0ec804b440 con 0x55d6a7aee9f0 2024-03-21T11:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.855+0000 7f0edf7b5740 1 -- 172.21.15.47:0/2403608005 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ec802cc10 con 0x55d6a7aee9f0 2024-03-21T11:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.855+0000 7f0eb7400640 1 -- 172.21.15.47:0/2403608005 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ec805a050 con 0x55d6a7aee9f0 2024-03-21T11:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.855+0000 7f0edf7b5740 10 client.28957 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.855+0000 7f0edf7b5740 1 -- 172.21.15.47:0/2403608005 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d6a78317d0 con 0x55d6a7aee9f0 2024-03-21T11:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.855+0000 7f0eb7400640 1 -- 172.21.15.47:0/2403608005 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ec802cc10 con 0x55d6a7aee9f0 2024-03-21T11:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.855+0000 7f0eb7400640 1 -- 172.21.15.47:0/2403608005 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0ec80345e0 con 0x55d6a7aee9f0 2024-03-21T11:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.856+0000 7f0edf7b5740 10 client.28957 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.856+0000 7f0edf7b5740 10 client.28957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.856+0000 7f0edf7b5740 20 client.28957 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.856+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/2403608005 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d6a7b04260 0x55d6a7b06650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.856+0000 7f0edf7b5740 4 client.28957 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.856+0000 7f0edf7b5740 1 -- 172.21.15.47:0/2403608005 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6a7a60800 con 0x55d6a7b04260 2024-03-21T11:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.856+0000 7f0ed5400640 1 --2- 172.21.15.47:0/2403608005 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d6a7b04260 0x55d6a7b06650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.856+0000 7f0ed5400640 1 --2- 172.21.15.47:0/2403608005 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d6a7b04260 0x55d6a7b06650 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5feb24150da87d7 server_cookie=eb127514e5435a55 in_seq=0 out_seq=0 2024-03-21T11:33:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.856+0000 7f0eb7400640 10 client.28957 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.857+0000 7f0eb7400640 1 -- 172.21.15.47:0/2403608005 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d6a7a60800 con 0x55d6a7b04260 2024-03-21T11:33:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.857+0000 7f0eb7400640 10 client.28957 handle_command_reply: tid=0 2024-03-21T11:33:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.857+0000 7f0edf7b5740 1 client.28957 shutdown 2024-03-21T11:33:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.857+0000 7f0edf7b5740 1 -- 172.21.15.47:0/2403608005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7aee9f0 msgr2=0x55d6a7af8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.857+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/2403608005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7aee9f0 0x55d6a7af8b80 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8036040 tx=0x7f0ec8040c00 comp rx=0 tx=0).stop 2024-03-21T11:33:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.857+0000 7f0edf7b5740 1 -- 172.21.15.47:0/2403608005 shutdown_connections 2024-03-21T11:33:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.857+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/2403608005 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d6a7b04260 0x55d6a7b06650 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.857+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/2403608005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7af0580 0x55d6a7b00100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.857+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/2403608005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 0x55d6a7af90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.857+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/2403608005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7aee9f0 0x55d6a7af8b80 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.857+0000 7f0edf7b5740 1 -- 172.21.15.47:0/2403608005 >> 172.21.15.47:0/2403608005 conn(0x55d6a7ae9de0 msgr2=0x55d6a7ae9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.857+0000 7f0eb7400640 0 client.28957 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.858+0000 7f0edf7b5740 1 -- 172.21.15.47:0/2403608005 shutdown_connections 2024-03-21T11:33:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.858+0000 7f0edf7b5740 1 -- 172.21.15.47:0/2403608005 wait complete. 2024-03-21T11:33:28.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.858+0000 7f0edf7b5740 20 client.28957 trim_cache size 0 max 16384 2024-03-21T11:33:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.874+0000 7f0edf7b5740 1 Processor -- start 2024-03-21T11:33:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.874+0000 7f0edf7b5740 1 -- start start 2024-03-21T11:33:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.875+0000 7f0edf7b5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7aee9f0 0x55d6a7aed040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.875+0000 7f0edf7b5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 0x55d6a7b000c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.875+0000 7f0edf7b5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7af0580 0x55d6a7af81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.875+0000 7f0edf7b5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed81b0510 con 0x55d6a7aefbc0 2024-03-21T11:33:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.875+0000 7f0edf7b5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed81b1e90 con 0x55d6a7af0580 2024-03-21T11:33:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.875+0000 7f0edf7b5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed81b0b70 con 0x55d6a7aee9f0 2024-03-21T11:33:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.875+0000 7f0ed5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7aee9f0 0x55d6a7aed040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.875+0000 7f0ed5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7aee9f0 0x55d6a7aed040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52582/0 (socket says 172.21.15.47:52582) 2024-03-21T11:33:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.875+0000 7f0ed5e00640 1 -- 172.21.15.47:0/3681874041 learned_addr learned my addr 172.21.15.47:0/3681874041 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.875+0000 7f0ed6800640 1 --2- 172.21.15.47:0/3681874041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7af0580 0x55d6a7af81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.875+0000 7f0ed5400640 1 --2- 172.21.15.47:0/3681874041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 0x55d6a7b000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.876+0000 7f0ed6800640 1 -- 172.21.15.47:0/3681874041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7aee9f0 msgr2=0x55d6a7aed040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.876+0000 7f0ed6800640 1 --2- 172.21.15.47:0/3681874041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7aee9f0 0x55d6a7aed040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.876+0000 7f0ed6800640 1 -- 172.21.15.47:0/3681874041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 msgr2=0x55d6a7b000c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.876+0000 7f0ed6800640 1 --2- 172.21.15.47:0/3681874041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 0x55d6a7b000c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.876+0000 7f0ed6800640 1 -- 172.21.15.47:0/3681874041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6a7aea850 con 0x55d6a7af0580 2024-03-21T11:33:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.876+0000 7f0ed5400640 1 --2- 172.21.15.47:0/3681874041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 0x55d6a7b000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:33:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.876+0000 7f0ed5e00640 1 --2- 172.21.15.47:0/3681874041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7aee9f0 0x55d6a7aed040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.876+0000 7f0ed6800640 1 --2- 172.21.15.47:0/3681874041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7af0580 0x55d6a7af81a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0012010 tx=0x7f0ed001a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad1078b374897d19 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.876+0000 7f0ec2600640 1 -- 172.21.15.47:0/3681874041 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0004b20 con 0x55d6a7af0580 2024-03-21T11:33:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.877+0000 7f0ec2600640 1 -- 172.21.15.47:0/3681874041 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ed0004cc0 con 0x55d6a7af0580 2024-03-21T11:33:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.877+0000 7f0ec2600640 1 -- 172.21.15.47:0/3681874041 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0005280 con 0x55d6a7af0580 2024-03-21T11:33:28.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.877+0000 7f0edf7b5740 1 -- 172.21.15.47:0/3681874041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7af0580 msgr2=0x55d6a7af81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.877+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/3681874041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7af0580 0x55d6a7af81a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0012010 tx=0x7f0ed001a000 comp rx=0 tx=0).stop 2024-03-21T11:33:28.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.878+0000 7f0edf7b5740 1 -- 172.21.15.47:0/3681874041 shutdown_connections 2024-03-21T11:33:28.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.878+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/3681874041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7af0580 0x55d6a7af81a0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.878+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/3681874041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 0x55d6a7b000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.878+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/3681874041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7aee9f0 0x55d6a7aed040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.878+0000 7f0edf7b5740 1 -- 172.21.15.47:0/3681874041 >> 172.21.15.47:0/3681874041 conn(0x55d6a7ae9de0 msgr2=0x55d6a7ae9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.878+0000 7f0edf7b5740 1 -- 172.21.15.47:0/3681874041 shutdown_connections 2024-03-21T11:33:28.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.878+0000 7f0edf7b5740 1 -- 172.21.15.47:0/3681874041 wait complete. 2024-03-21T11:33:28.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.878+0000 7f0edf7b5740 1 -- start start 2024-03-21T11:33:28.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.879+0000 7f0edf7b5740 1 Processor -- start 2024-03-21T11:33:28.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.879+0000 7f0edf7b5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7aee9f0 0x55d6a7afbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.879+0000 7f0edf7b5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 0x55d6a7afc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.879+0000 7f0edf7b5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7af0580 0x55d6a7aff490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.879+0000 7f0edf7b5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed81b1830 con 0x55d6a7aefbc0 2024-03-21T11:33:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.879+0000 7f0edf7b5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed81b11d0 con 0x55d6a7aee9f0 2024-03-21T11:33:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.879+0000 7f0edf7b5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed81ae6d0 con 0x55d6a7af0580 2024-03-21T11:33:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.879+0000 7f0edf7b5740 10 client.0 _ll_register_callbacks cb 0x7f0ed0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.879+0000 7f0ed5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7aee9f0 0x55d6a7afbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.879+0000 7f0ed6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7af0580 0x55d6a7aff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.879+0000 7f0ed5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7aee9f0 0x55d6a7afbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33456/0 (socket says 172.21.15.47:33456) 2024-03-21T11:33:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.880+0000 7f0ed5e00640 1 -- 172.21.15.47:0/3741124228 learned_addr learned my addr 172.21.15.47:0/3741124228 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.880+0000 7f0ed5400640 1 --2- 172.21.15.47:0/3741124228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 0x55d6a7afc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.880+0000 7f0ed5e00640 1 -- 172.21.15.47:0/3741124228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7af0580 msgr2=0x55d6a7aff490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.880+0000 7f0ed5e00640 1 --2- 172.21.15.47:0/3741124228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7af0580 0x55d6a7aff490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.880+0000 7f0ed5e00640 1 -- 172.21.15.47:0/3741124228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 msgr2=0x55d6a7afc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.880+0000 7f0ed5e00640 1 --2- 172.21.15.47:0/3741124228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 0x55d6a7afc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.880+0000 7f0ed5e00640 1 -- 172.21.15.47:0/3741124228 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6a7affa80 con 0x55d6a7aee9f0 2024-03-21T11:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.880+0000 7f0ed6800640 1 --2- 172.21.15.47:0/3741124228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7af0580 0x55d6a7aff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.880+0000 7f0ed5e00640 1 --2- 172.21.15.47:0/3741124228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7aee9f0 0x55d6a7afbf30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0ec802c200 tx=0x7f0ec8040780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f46dfb53ef6a6f24 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.880+0000 7f0eb7400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.881+0000 7f0eb7400640 1 -- 172.21.15.47:0/3741124228 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec804f040 con 0x55d6a7aee9f0 2024-03-21T11:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.881+0000 7f0eb7400640 1 -- 172.21.15.47:0/3741124228 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0ec802ca10 con 0x55d6a7aee9f0 2024-03-21T11:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.881+0000 7f0edf7b5740 1 -- 172.21.15.47:0/3741124228 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ed0004cc0 con 0x55d6a7aee9f0 2024-03-21T11:33:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.881+0000 7f0eb7400640 1 -- 172.21.15.47:0/3741124228 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ec8066050 con 0x55d6a7aee9f0 2024-03-21T11:33:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.881+0000 7f0edf7b5740 10 client.28943 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.881+0000 7f0edf7b5740 1 -- 172.21.15.47:0/3741124228 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d6a7b60110 con 0x55d6a7aee9f0 2024-03-21T11:33:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.881+0000 7f0eb7400640 1 -- 172.21.15.47:0/3741124228 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ed0004cc0 con 0x55d6a7aee9f0 2024-03-21T11:33:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.882+0000 7f0eb7400640 1 -- 172.21.15.47:0/3741124228 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0ec80046c0 con 0x55d6a7aee9f0 2024-03-21T11:33:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.882+0000 7f0edf7b5740 10 client.28943 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.882+0000 7f0edf7b5740 10 client.28943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.882+0000 7f0edf7b5740 20 client.28943 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.882+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/3741124228 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d6a7b609e0 0x55d6a7b62e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.882+0000 7f0edf7b5740 4 client.28943 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.882+0000 7f0edf7b5740 1 -- 172.21.15.47:0/3741124228 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d6a7a5ab50 con 0x55d6a7b609e0 2024-03-21T11:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.882+0000 7f0ed5400640 1 --2- 172.21.15.47:0/3741124228 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d6a7b609e0 0x55d6a7b62e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.883+0000 7f0ed5400640 1 --2- 172.21.15.47:0/3741124228 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d6a7b609e0 0x55d6a7b62e00 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ce5819e2be8674 server_cookie=80e3bb1a4a94b344 in_seq=0 out_seq=0 2024-03-21T11:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.883+0000 7f0eb7400640 10 client.28943 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.883+0000 7f0eb7400640 1 -- 172.21.15.47:0/3741124228 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d6a7a5ab50 con 0x55d6a7b609e0 2024-03-21T11:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.883+0000 7f0eb7400640 10 client.28943 handle_command_reply: tid=0 2024-03-21T11:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.883+0000 7f0edf7b5740 1 client.28943 shutdown 2024-03-21T11:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.884+0000 7f0edf7b5740 1 -- 172.21.15.47:0/3741124228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7aee9f0 msgr2=0x55d6a7afbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.884+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/3741124228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7aee9f0 0x55d6a7afbf30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0ec802c200 tx=0x7f0ec8040780 comp rx=0 tx=0).stop 2024-03-21T11:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.884+0000 7f0edf7b5740 1 -- 172.21.15.47:0/3741124228 shutdown_connections 2024-03-21T11:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.884+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/3741124228 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d6a7b609e0 0x55d6a7b62e00 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.884+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/3741124228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6a7af0580 0x55d6a7aff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.884+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/3741124228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6a7aefbc0 0x55d6a7afc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.884+0000 7f0edf7b5740 1 --2- 172.21.15.47:0/3741124228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6a7aee9f0 0x55d6a7afbf30 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.884+0000 7f0eb7400640 0 client.28943 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:28.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.884+0000 7f0edf7b5740 1 -- 172.21.15.47:0/3741124228 >> 172.21.15.47:0/3741124228 conn(0x55d6a7ae9de0 msgr2=0x55d6a7af2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.884+0000 7f0edf7b5740 1 -- 172.21.15.47:0/3741124228 shutdown_connections 2024-03-21T11:33:28.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.884+0000 7f0edf7b5740 1 -- 172.21.15.47:0/3741124228 wait complete. 2024-03-21T11:33:28.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.884+0000 7f0edf7b5740 20 client.28943 trim_cache size 0 max 16384 2024-03-21T11:33:28.888 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:33:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.885+0000 7f0ed7200640 1 -- 172.21.15.47:0/538891624 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0ebc0a2fa0 msgr2=0x7f0ebc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.885+0000 7f0ed7200640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0ebc0a2fa0 0x7f0ebc0a5460 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55d6a7aefac0 tx=0x7f0ed0009500 comp rx=0 tx=0).stop 2024-03-21T11:33:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.885+0000 7f0ed7200640 1 -- 172.21.15.47:0/538891624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed8075660 msgr2=0x7f0ed8104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:28.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.885+0000 7f0ed7200640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed8075660 0x7f0ed8104e80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400d950 tx=0x7f0ec400de10 comp rx=0 tx=0).stop 2024-03-21T11:33:28.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.885+0000 7f0ed7200640 1 -- 172.21.15.47:0/538891624 shutdown_connections 2024-03-21T11:33:28.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.885+0000 7f0ed7200640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0ebc0a2fa0 0x7f0ebc0a5460 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.885+0000 7f0ed7200640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed8109450 0x7f0ed81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.885+0000 7f0ed7200640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ed8075660 0x7f0ed8104e80 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.885+0000 7f0ed7200640 1 --2- 172.21.15.47:0/538891624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ed8074080 0x7f0ed8104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:28.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.885+0000 7f0ed7200640 1 -- 172.21.15.47:0/538891624 >> 172.21.15.47:0/538891624 conn(0x7f0ed80fcb10 msgr2=0x7f0ed81053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:28.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.885+0000 7f0ed7200640 1 -- 172.21.15.47:0/538891624 shutdown_connections 2024-03-21T11:33:28.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:28.885+0000 7f0ed7200640 1 -- 172.21.15.47:0/538891624 wait complete. 2024-03-21T11:33:28.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:28 smithi100 ceph-mon[36245]: pgmap v704: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 283 op/s 2024-03-21T11:33:29.551 INFO:tasks.workunit.client.0.smithi047.stdout: 1 751554 8.08 MB/sec execute 373 sec latency 46.270 ms 2024-03-21T11:33:29.905 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:33:29.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:33:29.906 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:33:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.156+0000 7f6df6200640 1 -- 172.21.15.47:0/2898356535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df01037a0 msgr2=0x7f6df0103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.156+0000 7f6df6200640 1 --2- 172.21.15.47:0/2898356535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df01037a0 0x7f6df0103bc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009ca0 tx=0x7f6de002c0f0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.156+0000 7f6df6200640 1 -- 172.21.15.47:0/2898356535 shutdown_connections 2024-03-21T11:33:30.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.156+0000 7f6df6200640 1 --2- 172.21.15.47:0/2898356535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6df0105350 0x7f6df010bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.156+0000 7f6df6200640 1 --2- 172.21.15.47:0/2898356535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df0104990 0x7f6df0104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.156+0000 7f6df6200640 1 --2- 172.21.15.47:0/2898356535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df01037a0 0x7f6df0103bc0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.156+0000 7f6df6200640 1 -- 172.21.15.47:0/2898356535 >> 172.21.15.47:0/2898356535 conn(0x7f6df00fed10 msgr2=0x7f6df0101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:30.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.156+0000 7f6df6200640 1 -- 172.21.15.47:0/2898356535 shutdown_connections 2024-03-21T11:33:30.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.156+0000 7f6df6200640 1 -- 172.21.15.47:0/2898356535 wait complete. 2024-03-21T11:33:30.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.158+0000 7f6df6200640 1 Processor -- start 2024-03-21T11:33:30.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.158+0000 7f6df6200640 1 -- start start 2024-03-21T11:33:30.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.158+0000 7f6df6200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df01037a0 0x7f6df0199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.158+0000 7f6df6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df0104990 0x7f6df0199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.158+0000 7f6df6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6df0105350 0x7f6df01a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.158+0000 7f6df6200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df010f420 con 0x7f6df01037a0 2024-03-21T11:33:30.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.159+0000 7f6df6200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df010f2a0 con 0x7f6df0105350 2024-03-21T11:33:30.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.159+0000 7f6df4e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df01037a0 0x7f6df0199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.159+0000 7f6df6200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df010f5a0 con 0x7f6df0104990 2024-03-21T11:33:30.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.159+0000 7f6defe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df0104990 0x7f6df0199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.159+0000 7f6df4e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df01037a0 0x7f6df0199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58068/0 (socket says 172.21.15.47:58068) 2024-03-21T11:33:30.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.159+0000 7f6df4e00640 1 -- 172.21.15.47:0/1951422426 learned_addr learned my addr 172.21.15.47:0/1951422426 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:30.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.159+0000 7f6df5800640 1 --2- 172.21.15.47:0/1951422426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6df0105350 0x7f6df01a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.159+0000 7f6defe00640 1 -- 172.21.15.47:0/1951422426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6df0105350 msgr2=0x7f6df01a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.159+0000 7f6defe00640 1 --2- 172.21.15.47:0/1951422426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6df0105350 0x7f6df01a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.159+0000 7f6defe00640 1 -- 172.21.15.47:0/1951422426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df01037a0 msgr2=0x7f6df0199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.159+0000 7f6defe00640 1 --2- 172.21.15.47:0/1951422426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df01037a0 0x7f6df0199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.159+0000 7f6defe00640 1 -- 172.21.15.47:0/1951422426 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df01a16b0 con 0x7f6df0104990 2024-03-21T11:33:30.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.159+0000 7f6defe00640 1 --2- 172.21.15.47:0/1951422426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df0104990 0x7f6df0199fd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8005fd0 tx=0x7f6dd800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:30.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.160+0000 7f6ded600640 1 -- 172.21.15.47:0/1951422426 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8009c30 con 0x7f6df0104990 2024-03-21T11:33:30.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.160+0000 7f6ded600640 1 -- 172.21.15.47:0/1951422426 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6dd8009dd0 con 0x7f6df0104990 2024-03-21T11:33:30.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.160+0000 7f6df6200640 1 -- 172.21.15.47:0/1951422426 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df01a19a0 con 0x7f6df0104990 2024-03-21T11:33:30.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.160+0000 7f6ded600640 1 -- 172.21.15.47:0/1951422426 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8010460 con 0x7f6df0104990 2024-03-21T11:33:30.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.160+0000 7f6df6200640 1 -- 172.21.15.47:0/1951422426 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df01a1f40 con 0x7f6df0104990 2024-03-21T11:33:30.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.163+0000 7f6ded600640 1 -- 172.21.15.47:0/1951422426 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6dd8003ef0 con 0x7f6df0104990 2024-03-21T11:33:30.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.163+0000 7f6dfa446740 1 Processor -- start 2024-03-21T11:33:30.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.164+0000 7f6dfa446740 1 -- start start 2024-03-21T11:33:30.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.164+0000 7f6dfa446740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b42b0 0x55e6a35b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.164+0000 7f6dfa446740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b4ca0 0x55e6a35b97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.164+0000 7f6dfa446740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b9e10 0x55e6a35bc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.164+0000 7f6dfa446740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a3519eb0 con 0x55e6a35b42b0 2024-03-21T11:33:30.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.164+0000 7f6dfa446740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a34c2a60 con 0x55e6a35b4ca0 2024-03-21T11:33:30.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.164+0000 7f6dfa446740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a3515f50 con 0x55e6a35b9e10 2024-03-21T11:33:30.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.164+0000 7f6df4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b4ca0 0x55e6a35b97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.164+0000 7f6df5800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b42b0 0x55e6a35b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.164+0000 7f6df5800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b42b0 0x55e6a35b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58070/0 (socket says 172.21.15.47:58070) 2024-03-21T11:33:30.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.164+0000 7f6df5800640 1 -- 172.21.15.47:0/1756837880 learned_addr learned my addr 172.21.15.47:0/1756837880 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:30.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.164+0000 7f6defe00640 1 --2- 172.21.15.47:0/1756837880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b9e10 0x55e6a35bc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.164+0000 7f6df4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b4ca0 0x55e6a35b97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33482/0 (socket says 172.21.15.47:33482) 2024-03-21T11:33:30.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.165+0000 7f6ded600640 1 --2- 172.21.15.47:0/1951422426 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dd40a2fa0 0x7f6dd40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.165+0000 7f6df5800640 1 -- 172.21.15.47:0/1756837880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b9e10 msgr2=0x55e6a35bc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.165+0000 7f6df5800640 1 --2- 172.21.15.47:0/1756837880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b9e10 0x55e6a35bc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.165+0000 7f6df5800640 1 -- 172.21.15.47:0/1756837880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b4ca0 msgr2=0x55e6a35b97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.165+0000 7f6df5800640 1 --2- 172.21.15.47:0/1756837880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b4ca0 0x55e6a35b97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.165+0000 7f6df5800640 1 -- 172.21.15.47:0/1756837880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6a32f67e0 con 0x55e6a35b42b0 2024-03-21T11:33:30.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.165+0000 7f6defe00640 1 --2- 172.21.15.47:0/1756837880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b9e10 0x55e6a35bc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:30.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.165+0000 7f6ded600640 1 -- 172.21.15.47:0/1951422426 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6dd80cef00 con 0x7f6df0104990 2024-03-21T11:33:30.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.166+0000 7f6df5800640 1 --2- 172.21.15.47:0/1951422426 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dd40a2fa0 0x7f6dd40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.166+0000 7f6df5800640 1 --2- 172.21.15.47:0/1756837880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b42b0 0x55e6a35b46d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009c20 tx=0x7f6de8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345156e93efd54f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.166+0000 7f6de6000640 1 -- 172.21.15.47:0/1756837880 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de801b850 con 0x55e6a35b42b0 2024-03-21T11:33:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.166+0000 7f6de6000640 1 -- 172.21.15.47:0/1756837880 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6de8019db0 con 0x55e6a35b42b0 2024-03-21T11:33:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.166+0000 7f6de6000640 1 -- 172.21.15.47:0/1756837880 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de800eda0 con 0x55e6a35b42b0 2024-03-21T11:33:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.167+0000 7f6dfa446740 1 -- 172.21.15.47:0/1756837880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b42b0 msgr2=0x55e6a35b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.167+0000 7f6df5800640 1 --2- 172.21.15.47:0/1951422426 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dd40a2fa0 0x7f6dd40a5460 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009fd0 tx=0x7f6de8021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.167+0000 7f6dfa446740 1 --2- 172.21.15.47:0/1756837880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b42b0 0x55e6a35b46d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009c20 tx=0x7f6de8011810 comp rx=0 tx=0).stop 2024-03-21T11:33:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.167+0000 7f6dfa446740 1 -- 172.21.15.47:0/1756837880 shutdown_connections 2024-03-21T11:33:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.167+0000 7f6dfa446740 1 --2- 172.21.15.47:0/1756837880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b9e10 0x55e6a35bc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.167+0000 7f6dfa446740 1 --2- 172.21.15.47:0/1756837880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b4ca0 0x55e6a35b97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.167+0000 7f6dfa446740 1 --2- 172.21.15.47:0/1756837880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b42b0 0x55e6a35b46d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.167+0000 7f6dfa446740 1 -- 172.21.15.47:0/1756837880 >> 172.21.15.47:0/1756837880 conn(0x55e6a35af820 msgr2=0x55e6a35afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:30.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.167+0000 7f6dfa446740 1 -- 172.21.15.47:0/1756837880 shutdown_connections 2024-03-21T11:33:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.167+0000 7f6dfa446740 1 -- 172.21.15.47:0/1756837880 wait complete. 2024-03-21T11:33:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.168+0000 7f6dfa446740 1 -- start start 2024-03-21T11:33:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.168+0000 7f6dfa446740 1 Processor -- start 2024-03-21T11:33:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.168+0000 7f6dfa446740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b42b0 0x55e6a35be490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.168+0000 7f6dfa446740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b4ca0 0x55e6a35be9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.168+0000 7f6dfa446740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b9e10 0x55e6a35c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.168+0000 7f6dfa446740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a35bcdc0 con 0x55e6a35b9e10 2024-03-21T11:33:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.168+0000 7f6dfa446740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a34d5eb0 con 0x55e6a35b4ca0 2024-03-21T11:33:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.168+0000 7f6dfa446740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a35bcf40 con 0x55e6a35b42b0 2024-03-21T11:33:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.169+0000 7f6dfa446740 10 client.0 _ll_register_callbacks cb 0x55e6a34ca150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.169+0000 7f6df4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b42b0 0x55e6a35be490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.169+0000 7f6defe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b4ca0 0x55e6a35be9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.169+0000 7f6df4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b42b0 0x55e6a35be490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52624/0 (socket says 172.21.15.47:52624) 2024-03-21T11:33:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.169+0000 7f6df4e00640 1 -- 172.21.15.47:0/1682262701 learned_addr learned my addr 172.21.15.47:0/1682262701 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.169+0000 7f6df5800640 1 --2- 172.21.15.47:0/1682262701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b9e10 0x55e6a35c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.169+0000 7f6defe00640 1 -- 172.21.15.47:0/1682262701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b42b0 msgr2=0x55e6a35be490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.169+0000 7f6defe00640 1 --2- 172.21.15.47:0/1682262701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b42b0 0x55e6a35be490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.169+0000 7f6defe00640 1 -- 172.21.15.47:0/1682262701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b9e10 msgr2=0x55e6a35c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.169+0000 7f6defe00640 1 --2- 172.21.15.47:0/1682262701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b9e10 0x55e6a35c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.169+0000 7f6defe00640 1 -- 172.21.15.47:0/1682262701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6a3356f70 con 0x55e6a35b4ca0 2024-03-21T11:33:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.169+0000 7f6defe00640 1 --2- 172.21.15.47:0/1682262701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b4ca0 0x55e6a35be9d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6dd809b280 tx=0x7f6dd809b0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3afc705e3aa5f3d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.169+0000 7f6ddea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:33:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.170+0000 7f6ddea00640 1 -- 172.21.15.47:0/1682262701 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8020070 con 0x55e6a35b4ca0 2024-03-21T11:33:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.170+0000 7f6ddea00640 1 -- 172.21.15.47:0/1682262701 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6dd8097960 con 0x55e6a35b4ca0 2024-03-21T11:33:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.170+0000 7f6dfa446740 1 -- 172.21.15.47:0/1682262701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6de8019db0 con 0x55e6a35b4ca0 2024-03-21T11:33:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.170+0000 7f6ddea00640 1 -- 172.21.15.47:0/1682262701 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dd8095e60 con 0x55e6a35b4ca0 2024-03-21T11:33:30.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.170+0000 7f6dfa446740 10 client.28955 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.170+0000 7f6dfa446740 1 -- 172.21.15.47:0/1682262701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e6a31ee170 con 0x55e6a35b4ca0 2024-03-21T11:33:30.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.170+0000 7f6ddea00640 1 -- 172.21.15.47:0/1682262701 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6de8019db0 con 0x55e6a35b4ca0 2024-03-21T11:33:30.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.170+0000 7f6ddea00640 1 -- 172.21.15.47:0/1682262701 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6dd80d0030 con 0x55e6a35b4ca0 2024-03-21T11:33:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.170+0000 7f6dfa446740 10 client.28955 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.170+0000 7f6dfa446740 10 client.28955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.170+0000 7f6dfa446740 20 client.28955 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.170+0000 7f6dfa446740 1 --2- 172.21.15.47:0/1682262701 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e6a35ca2e0 0x55e6a35cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.171+0000 7f6dfa446740 4 client.28955 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:33:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.171+0000 7f6dfa446740 1 -- 172.21.15.47:0/1682262701 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6a35273f0 con 0x55e6a35ca2e0 2024-03-21T11:33:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.171+0000 7f6df4e00640 1 --2- 172.21.15.47:0/1682262701 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e6a35ca2e0 0x55e6a35cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.171+0000 7f6df4e00640 1 --2- 172.21.15.47:0/1682262701 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e6a35ca2e0 0x55e6a35cc6f0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f53bfb409504078 server_cookie=363badf8b7476fe8 in_seq=0 out_seq=0 2024-03-21T11:33:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.171+0000 7f6ddea00640 10 client.28955 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6ddea00640 1 -- 172.21.15.47:0/1682262701 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e6a35273f0 con 0x55e6a35ca2e0 2024-03-21T11:33:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6ddea00640 10 client.28955 handle_command_reply: tid=0 2024-03-21T11:33:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6dfa446740 1 client.28955 shutdown 2024-03-21T11:33:30.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6dfa446740 1 -- 172.21.15.47:0/1682262701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b4ca0 msgr2=0x55e6a35be9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6dfa446740 1 --2- 172.21.15.47:0/1682262701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b4ca0 0x55e6a35be9d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6dd809b280 tx=0x7f6dd809b0c0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6dfa446740 1 -- 172.21.15.47:0/1682262701 shutdown_connections 2024-03-21T11:33:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6dfa446740 1 --2- 172.21.15.47:0/1682262701 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e6a35ca2e0 0x55e6a35cc6f0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6dfa446740 1 --2- 172.21.15.47:0/1682262701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b9e10 0x55e6a35c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6ddea00640 0 client.28955 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6dfa446740 1 --2- 172.21.15.47:0/1682262701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b4ca0 0x55e6a35be9d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6dfa446740 1 --2- 172.21.15.47:0/1682262701 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b42b0 0x55e6a35be490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6dfa446740 1 -- 172.21.15.47:0/1682262701 >> 172.21.15.47:0/1682262701 conn(0x55e6a35af820 msgr2=0x55e6a35aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6dfa446740 1 -- 172.21.15.47:0/1682262701 shutdown_connections 2024-03-21T11:33:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6dfa446740 1 -- 172.21.15.47:0/1682262701 wait complete. 2024-03-21T11:33:30.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.174+0000 7f6dfa446740 20 client.28955 trim_cache size 0 max 16384 2024-03-21T11:33:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.195+0000 7f6dfa446740 1 Processor -- start 2024-03-21T11:33:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.196+0000 7f6dfa446740 1 -- start start 2024-03-21T11:33:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.196+0000 7f6dfa446740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b42b0 0x55e6a35b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.196+0000 7f6dfa446740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b4ca0 0x55e6a35c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.196+0000 7f6dfa446740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b9e10 0x55e6a35d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.196+0000 7f6dfa446740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df01ac160 con 0x55e6a35b4ca0 2024-03-21T11:33:30.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.196+0000 7f6dfa446740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df01adae0 con 0x55e6a35b9e10 2024-03-21T11:33:30.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.196+0000 7f6dfa446740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df01ac7c0 con 0x55e6a35b42b0 2024-03-21T11:33:30.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.196+0000 7f6df4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b42b0 0x55e6a35b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.196+0000 7f6df5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b9e10 0x55e6a35d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.196+0000 7f6df4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b42b0 0x55e6a35b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52628/0 (socket says 172.21.15.47:52628) 2024-03-21T11:33:30.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.196+0000 7f6df4e00640 1 -- 172.21.15.47:0/1395303074 learned_addr learned my addr 172.21.15.47:0/1395303074 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:30.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.196+0000 7f6defe00640 1 --2- 172.21.15.47:0/1395303074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b4ca0 0x55e6a35c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.197+0000 7f6df4e00640 1 -- 172.21.15.47:0/1395303074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b9e10 msgr2=0x55e6a35d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.197+0000 7f6df4e00640 1 --2- 172.21.15.47:0/1395303074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b9e10 0x55e6a35d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.197+0000 7f6df4e00640 1 -- 172.21.15.47:0/1395303074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b4ca0 msgr2=0x55e6a35c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.197+0000 7f6df4e00640 1 --2- 172.21.15.47:0/1395303074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b4ca0 0x55e6a35c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.197+0000 7f6df4e00640 1 -- 172.21.15.47:0/1395303074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6a35b5080 con 0x55e6a35b42b0 2024-03-21T11:33:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.197+0000 7f6df4e00640 1 --2- 172.21.15.47:0/1395303074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b42b0 0x55e6a35b30f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f6de0005e30 tx=0x7f6de002e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=575677b6fe42a015 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.197+0000 7f6de5600640 1 -- 172.21.15.47:0/1395303074 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de004e070 con 0x55e6a35b42b0 2024-03-21T11:33:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.198+0000 7f6de5600640 1 -- 172.21.15.47:0/1395303074 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6de0003970 con 0x55e6a35b42b0 2024-03-21T11:33:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.198+0000 7f6de5600640 1 -- 172.21.15.47:0/1395303074 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de0039040 con 0x55e6a35b42b0 2024-03-21T11:33:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.198+0000 7f6dfa446740 1 -- 172.21.15.47:0/1395303074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b42b0 msgr2=0x55e6a35b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.198+0000 7f6dfa446740 1 --2- 172.21.15.47:0/1395303074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b42b0 0x55e6a35b30f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f6de0005e30 tx=0x7f6de002e040 comp rx=0 tx=0).stop 2024-03-21T11:33:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.198+0000 7f6dfa446740 1 -- 172.21.15.47:0/1395303074 shutdown_connections 2024-03-21T11:33:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.198+0000 7f6dfa446740 1 --2- 172.21.15.47:0/1395303074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b9e10 0x55e6a35d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.198+0000 7f6dfa446740 1 --2- 172.21.15.47:0/1395303074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b4ca0 0x55e6a35c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.198+0000 7f6dfa446740 1 --2- 172.21.15.47:0/1395303074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b42b0 0x55e6a35b30f0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.198+0000 7f6dfa446740 1 -- 172.21.15.47:0/1395303074 >> 172.21.15.47:0/1395303074 conn(0x55e6a35af820 msgr2=0x55e6a35aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:30.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.199+0000 7f6dfa446740 1 -- 172.21.15.47:0/1395303074 shutdown_connections 2024-03-21T11:33:30.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.199+0000 7f6dfa446740 1 -- 172.21.15.47:0/1395303074 wait complete. 2024-03-21T11:33:30.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.199+0000 7f6dfa446740 1 -- start start 2024-03-21T11:33:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.199+0000 7f6dfa446740 1 Processor -- start 2024-03-21T11:33:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6dfa446740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b42b0 0x55e6a35c5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6dfa446740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b4ca0 0x55e6a35c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6dfa446740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b9e10 0x55e6a35c4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6dfa446740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df01ad480 con 0x55e6a35b4ca0 2024-03-21T11:33:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6dfa446740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df01ace20 con 0x55e6a35b42b0 2024-03-21T11:33:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6dfa446740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df01aa320 con 0x55e6a35b9e10 2024-03-21T11:33:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6dfa446740 10 client.0 _ll_register_callbacks cb 0x7f6dd8005d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6df4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b42b0 0x55e6a35c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6df4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b42b0 0x55e6a35c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33516/0 (socket says 172.21.15.47:33516) 2024-03-21T11:33:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6df4e00640 1 -- 172.21.15.47:0/24618064 learned_addr learned my addr 172.21.15.47:0/24618064 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6df5800640 1 --2- 172.21.15.47:0/24618064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b9e10 0x55e6a35c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6defe00640 1 --2- 172.21.15.47:0/24618064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b4ca0 0x55e6a35c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6df4e00640 1 -- 172.21.15.47:0/24618064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b9e10 msgr2=0x55e6a35c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6df4e00640 1 --2- 172.21.15.47:0/24618064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b9e10 0x55e6a35c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6df4e00640 1 -- 172.21.15.47:0/24618064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b4ca0 msgr2=0x55e6a35c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6df4e00640 1 --2- 172.21.15.47:0/24618064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b4ca0 0x55e6a35c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6df4e00640 1 -- 172.21.15.47:0/24618064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6a35c4d50 con 0x55e6a35b42b0 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.200+0000 7f6defe00640 1 --2- 172.21.15.47:0/24618064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b4ca0 0x55e6a35c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.201+0000 7f6df4e00640 1 --2- 172.21.15.47:0/24618064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b42b0 0x55e6a35c5950 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6de002c7e0 tx=0x7f6de0032d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bc7aa7ed3f7fc83 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.201+0000 7f6ddea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.201+0000 7f6ddea00640 1 -- 172.21.15.47:0/24618064 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de004e030 con 0x55e6a35b42b0 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.201+0000 7f6ddea00640 1 -- 172.21.15.47:0/24618064 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6de0003b20 con 0x55e6a35b42b0 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.201+0000 7f6dfa446740 1 -- 172.21.15.47:0/24618064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6de0003970 con 0x55e6a35b42b0 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.201+0000 7f6ddea00640 1 -- 172.21.15.47:0/24618064 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6de0039040 con 0x55e6a35b42b0 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.201+0000 7f6dfa446740 10 client.28961 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.201+0000 7f6dfa446740 1 -- 172.21.15.47:0/24618064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e6a3626ca0 con 0x55e6a35b42b0 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.201+0000 7f6ddea00640 1 -- 172.21.15.47:0/24618064 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6de0003970 con 0x55e6a35b42b0 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.202+0000 7f6ddea00640 1 -- 172.21.15.47:0/24618064 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6de0004660 con 0x55e6a35b42b0 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.202+0000 7f6dfa446740 10 client.28961 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.202+0000 7f6dfa446740 10 client.28961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.202+0000 7f6dfa446740 20 client.28961 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:30.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.202+0000 7f6dfa446740 1 --2- 172.21.15.47:0/24618064 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e6a3627600 0x55e6a3629a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.202+0000 7f6dfa446740 4 client.28961 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:33:30.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.202+0000 7f6dfa446740 1 -- 172.21.15.47:0/24618064 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e6a3521740 con 0x55e6a3627600 2024-03-21T11:33:30.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.202+0000 7f6defe00640 1 --2- 172.21.15.47:0/24618064 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e6a3627600 0x55e6a3629a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.203+0000 7f6defe00640 1 --2- 172.21.15.47:0/24618064 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e6a3627600 0x55e6a3629a20 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a6616b61f22b375 server_cookie=d82b05ee007bef05 in_seq=0 out_seq=0 2024-03-21T11:33:30.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.203+0000 7f6ddea00640 10 client.28961 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:30.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.219+0000 7f6ddea00640 1 -- 172.21.15.47:0/24618064 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e6a3521740 con 0x55e6a3627600 2024-03-21T11:33:30.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.219+0000 7f6ddea00640 10 client.28961 handle_command_reply: tid=0 2024-03-21T11:33:30.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.219+0000 7f6dfa446740 1 client.28961 shutdown 2024-03-21T11:33:30.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.220+0000 7f6dfa446740 1 -- 172.21.15.47:0/24618064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b42b0 msgr2=0x55e6a35c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.220+0000 7f6dfa446740 1 --2- 172.21.15.47:0/24618064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b42b0 0x55e6a35c5950 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6de002c7e0 tx=0x7f6de0032d10 comp rx=0 tx=0).stop 2024-03-21T11:33:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.220+0000 7f6dfa446740 1 -- 172.21.15.47:0/24618064 shutdown_connections 2024-03-21T11:33:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.220+0000 7f6dfa446740 1 --2- 172.21.15.47:0/24618064 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e6a3627600 0x55e6a3629a20 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.220+0000 7f6dfa446740 1 --2- 172.21.15.47:0/24618064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6a35b9e10 0x55e6a35c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.220+0000 7f6dfa446740 1 --2- 172.21.15.47:0/24618064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6a35b4ca0 0x55e6a35c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.220+0000 7f6dfa446740 1 --2- 172.21.15.47:0/24618064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e6a35b42b0 0x55e6a35c5950 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.220+0000 7f6dfa446740 1 -- 172.21.15.47:0/24618064 >> 172.21.15.47:0/24618064 conn(0x55e6a35af820 msgr2=0x55e6a35b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.220+0000 7f6ddea00640 0 client.28961 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.221+0000 7f6dfa446740 1 -- 172.21.15.47:0/24618064 shutdown_connections 2024-03-21T11:33:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.221+0000 7f6dfa446740 1 -- 172.21.15.47:0/24618064 wait complete. 2024-03-21T11:33:30.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.221+0000 7f6dfa446740 20 client.28961 trim_cache size 0 max 16384 2024-03-21T11:33:30.223 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:33:30.223 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:33:30.223 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "73463606-772b-4030-a544-e9d60dbc601a", 2024-03-21T11:33:30.223 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:33:30.223 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:33:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.222+0000 7f6df6200640 1 -- 172.21.15.47:0/1951422426 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dd40a2fa0 msgr2=0x7f6dd40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.222+0000 7f6df6200640 1 --2- 172.21.15.47:0/1951422426 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dd40a2fa0 0x7f6dd40a5460 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009fd0 tx=0x7f6de8021000 comp rx=0 tx=0).stop 2024-03-21T11:33:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.222+0000 7f6df6200640 1 -- 172.21.15.47:0/1951422426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df0104990 msgr2=0x7f6df0199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.222+0000 7f6df6200640 1 --2- 172.21.15.47:0/1951422426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df0104990 0x7f6df0199fd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8005fd0 tx=0x7f6dd800dde0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.222+0000 7f6df6200640 1 -- 172.21.15.47:0/1951422426 shutdown_connections 2024-03-21T11:33:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.222+0000 7f6df6200640 1 --2- 172.21.15.47:0/1951422426 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dd40a2fa0 0x7f6dd40a5460 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.222+0000 7f6df6200640 1 --2- 172.21.15.47:0/1951422426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6df0105350 0x7f6df01a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.222+0000 7f6df6200640 1 --2- 172.21.15.47:0/1951422426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df0104990 0x7f6df0199fd0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.222+0000 7f6df6200640 1 --2- 172.21.15.47:0/1951422426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df01037a0 0x7f6df0199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.222+0000 7f6df6200640 1 -- 172.21.15.47:0/1951422426 >> 172.21.15.47:0/1951422426 conn(0x7f6df00fed10 msgr2=0x7f6df0107ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:30.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.223+0000 7f6df6200640 1 -- 172.21.15.47:0/1951422426 shutdown_connections 2024-03-21T11:33:30.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.223+0000 7f6df6200640 1 -- 172.21.15.47:0/1951422426 wait complete. 2024-03-21T11:33:30.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:33:30.250 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:33:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.518+0000 7f507f000640 1 -- 172.21.15.47:0/1569997839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5078105110 msgr2=0x7f5078105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.518+0000 7f507f000640 1 --2- 172.21.15.47:0/1569997839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5078105110 0x7f5078105590 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5070009ca0 tx=0x7f507002c100 comp rx=0 tx=0).stop 2024-03-21T11:33:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.518+0000 7f507f000640 1 -- 172.21.15.47:0/1569997839 shutdown_connections 2024-03-21T11:33:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.518+0000 7f507f000640 1 --2- 172.21.15.47:0/1569997839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5078105110 0x7f5078105590 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.518+0000 7f507f000640 1 --2- 172.21.15.47:0/1569997839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5078102760 0x7f5078104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.518+0000 7f507f000640 1 --2- 172.21.15.47:0/1569997839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50780ffdf0 0x7f5078102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.518+0000 7f507f000640 1 -- 172.21.15.47:0/1569997839 >> 172.21.15.47:0/1569997839 conn(0x7f50780fba10 msgr2=0x7f50780fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.519+0000 7f507f000640 1 -- 172.21.15.47:0/1569997839 shutdown_connections 2024-03-21T11:33:30.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.519+0000 7f507f000640 1 -- 172.21.15.47:0/1569997839 wait complete. 2024-03-21T11:33:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.519+0000 7f507f000640 1 Processor -- start 2024-03-21T11:33:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.520+0000 7f507f000640 1 -- start start 2024-03-21T11:33:30.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.520+0000 7f507f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50780ffdf0 0x7f507819ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.520+0000 7f507f000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5078102760 0x7f507819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.520+0000 7f507f000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5078105110 0x7f50781a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.520+0000 7f507f000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50781137d0 con 0x7f5078105110 2024-03-21T11:33:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.520+0000 7f507f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5078113650 con 0x7f50780ffdf0 2024-03-21T11:33:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.520+0000 7f507f000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5078113950 con 0x7f5078102760 2024-03-21T11:33:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.520+0000 7f507dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50780ffdf0 0x7f507819ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.520+0000 7f507d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5078102760 0x7f507819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.520+0000 7f507e600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5078105110 0x7f50781a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.520+0000 7f507dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50780ffdf0 0x7f507819ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33526/0 (socket says 172.21.15.47:33526) 2024-03-21T11:33:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.520+0000 7f507dc00640 1 -- 172.21.15.47:0/440393691 learned_addr learned my addr 172.21.15.47:0/440393691 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.521+0000 7f507dc00640 1 -- 172.21.15.47:0/440393691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5078102760 msgr2=0x7f507819e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.521+0000 7f507dc00640 1 --2- 172.21.15.47:0/440393691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5078102760 0x7f507819e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.521+0000 7f507dc00640 1 -- 172.21.15.47:0/440393691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5078105110 msgr2=0x7f50781a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.521+0000 7f507dc00640 1 --2- 172.21.15.47:0/440393691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5078105110 0x7f50781a5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.521+0000 7f507dc00640 1 -- 172.21.15.47:0/440393691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50781a59e0 con 0x7f50780ffdf0 2024-03-21T11:33:30.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.521+0000 7f507dc00640 1 --2- 172.21.15.47:0/440393691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50780ffdf0 0x7f507819ddc0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f506c00e9f0 tx=0x7f506c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:30.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.521+0000 7f5076000640 1 -- 172.21.15.47:0/440393691 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c00cd80 con 0x7f50780ffdf0 2024-03-21T11:33:30.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.521+0000 7f5076000640 1 -- 172.21.15.47:0/440393691 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f506c010910 con 0x7f50780ffdf0 2024-03-21T11:33:30.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.521+0000 7f5076000640 1 -- 172.21.15.47:0/440393691 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c018990 con 0x7f50780ffdf0 2024-03-21T11:33:30.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.521+0000 7f507f000640 1 -- 172.21.15.47:0/440393691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50781a5cd0 con 0x7f50780ffdf0 2024-03-21T11:33:30.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.521+0000 7f507f000640 1 -- 172.21.15.47:0/440393691 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50781a61d0 con 0x7f50780ffdf0 2024-03-21T11:33:30.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.524+0000 7f5076000640 1 -- 172.21.15.47:0/440393691 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f506c010ac0 con 0x7f50780ffdf0 2024-03-21T11:33:30.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.525+0000 7f5083396740 1 Processor -- start 2024-03-21T11:33:30.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.525+0000 7f5083396740 1 -- start start 2024-03-21T11:33:30.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.525+0000 7f5083396740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301050 0x5631c8301470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.525+0000 7f5083396740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8301a40 0x5631c8306540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.525+0000 7f5083396740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8306bb0 0x5631c8309040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.525+0000 7f5083396740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c8266250 con 0x5631c8301050 2024-03-21T11:33:30.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.525+0000 7f5083396740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c820ea30 con 0x5631c8306bb0 2024-03-21T11:33:30.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.525+0000 7f5083396740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c82622f0 con 0x5631c8301a40 2024-03-21T11:33:30.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.525+0000 7f507e600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301050 0x5631c8301470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.525+0000 7f507e600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301050 0x5631c8301470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58106/0 (socket says 172.21.15.47:58106) 2024-03-21T11:33:30.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.525+0000 7f507e600640 1 -- 172.21.15.47:0/3703626696 learned_addr learned my addr 172.21.15.47:0/3703626696 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:30.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.525+0000 7f507d200640 1 --2- 172.21.15.47:0/3703626696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8306bb0 0x5631c8309040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.525+0000 7f507dc00640 1 --2- 172.21.15.47:0/3703626696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8301a40 0x5631c8306540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.526+0000 7f507d200640 1 -- 172.21.15.47:0/3703626696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8301a40 msgr2=0x5631c8306540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.526+0000 7f507d200640 1 --2- 172.21.15.47:0/3703626696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8301a40 0x5631c8306540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.526+0000 7f507d200640 1 -- 172.21.15.47:0/3703626696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301050 msgr2=0x5631c8301470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.526+0000 7f507d200640 1 --2- 172.21.15.47:0/3703626696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301050 0x5631c8301470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.526+0000 7f507d200640 1 -- 172.21.15.47:0/3703626696 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631c81f9690 con 0x5631c8306bb0 2024-03-21T11:33:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.526+0000 7f507e600640 1 --2- 172.21.15.47:0/3703626696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301050 0x5631c8301470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.526+0000 7f5076000640 1 --2- 172.21.15.47:0/440393691 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50600a2fa0 0x7f50600a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.526+0000 7f507d200640 1 --2- 172.21.15.47:0/3703626696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8306bb0 0x5631c8309040 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5064009bf0 tx=0x7f5064011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3724b6f7ba421eb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.526+0000 7f5076000640 1 -- 172.21.15.47:0/440393691 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f506c0cfa40 con 0x7f50780ffdf0 2024-03-21T11:33:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.526+0000 7f507e600640 1 --2- 172.21.15.47:0/440393691 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50600a2fa0 0x7f50600a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.527+0000 7f506aa00640 1 -- 172.21.15.47:0/3703626696 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064004050 con 0x5631c8306bb0 2024-03-21T11:33:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.527+0000 7f506aa00640 1 -- 172.21.15.47:0/3703626696 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5064005500 con 0x5631c8306bb0 2024-03-21T11:33:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.527+0000 7f506aa00640 1 -- 172.21.15.47:0/3703626696 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064004e40 con 0x5631c8306bb0 2024-03-21T11:33:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.528+0000 7f5083396740 1 -- 172.21.15.47:0/3703626696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8306bb0 msgr2=0x5631c8309040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.528+0000 7f507e600640 1 --2- 172.21.15.47:0/440393691 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50600a2fa0 0x7f50600a5460 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5070032420 tx=0x7f5070032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.528+0000 7f5083396740 1 --2- 172.21.15.47:0/3703626696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8306bb0 0x5631c8309040 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5064009bf0 tx=0x7f5064011810 comp rx=0 tx=0).stop 2024-03-21T11:33:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.528+0000 7f5083396740 1 -- 172.21.15.47:0/3703626696 shutdown_connections 2024-03-21T11:33:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.528+0000 7f5083396740 1 --2- 172.21.15.47:0/3703626696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8306bb0 0x5631c8309040 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.528+0000 7f5083396740 1 --2- 172.21.15.47:0/3703626696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8301a40 0x5631c8306540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.528+0000 7f5083396740 1 --2- 172.21.15.47:0/3703626696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301050 0x5631c8301470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.528+0000 7f5083396740 1 -- 172.21.15.47:0/3703626696 >> 172.21.15.47:0/3703626696 conn(0x5631c82c9200 msgr2=0x5631c82fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.528+0000 7f5083396740 1 -- 172.21.15.47:0/3703626696 shutdown_connections 2024-03-21T11:33:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.528+0000 7f5083396740 1 -- 172.21.15.47:0/3703626696 wait complete. 2024-03-21T11:33:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.528+0000 7f5083396740 1 -- start start 2024-03-21T11:33:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.529+0000 7f5083396740 1 Processor -- start 2024-03-21T11:33:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.529+0000 7f5083396740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8301050 0x5631c830b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.529+0000 7f5083396740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301a40 0x5631c830b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.529+0000 7f5083396740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8306bb0 0x5631c83125a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.529+0000 7f5083396740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c8309b60 con 0x5631c8301a40 2024-03-21T11:33:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.529+0000 7f5083396740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c8221e80 con 0x5631c8301050 2024-03-21T11:33:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.529+0000 7f5083396740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c8309ce0 con 0x5631c8306bb0 2024-03-21T11:33:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.529+0000 7f5083396740 10 client.0 _ll_register_callbacks cb 0x5631c8234130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.529+0000 7f507dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8301050 0x5631c830b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.529+0000 7f507e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8306bb0 0x5631c83125a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.529+0000 7f507dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8301050 0x5631c830b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33542/0 (socket says 172.21.15.47:33542) 2024-03-21T11:33:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.529+0000 7f507dc00640 1 -- 172.21.15.47:0/1308994021 learned_addr learned my addr 172.21.15.47:0/1308994021 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.529+0000 7f507d200640 1 --2- 172.21.15.47:0/1308994021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301a40 0x5631c830b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.530+0000 7f507e600640 1 -- 172.21.15.47:0/1308994021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8301050 msgr2=0x5631c830b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.530+0000 7f507e600640 1 --2- 172.21.15.47:0/1308994021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8301050 0x5631c830b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.530+0000 7f507e600640 1 -- 172.21.15.47:0/1308994021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301a40 msgr2=0x5631c830b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.530+0000 7f507e600640 1 --2- 172.21.15.47:0/1308994021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301a40 0x5631c830b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.530+0000 7f507e600640 1 -- 172.21.15.47:0/1308994021 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631c80447d0 con 0x5631c8306bb0 2024-03-21T11:33:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.530+0000 7f507dc00640 1 --2- 172.21.15.47:0/1308994021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8301050 0x5631c830b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.530+0000 7f507e600640 1 --2- 172.21.15.47:0/1308994021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8306bb0 0x5631c83125a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x5631c83022a0 tx=0x7f50700095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a73192d8b28f307 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.530+0000 7f505f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:33:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.530+0000 7f505f400640 1 -- 172.21.15.47:0/1308994021 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070050020 con 0x5631c8306bb0 2024-03-21T11:33:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.531+0000 7f5083396740 1 -- 172.21.15.47:0/1308994021 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5064005500 con 0x5631c8306bb0 2024-03-21T11:33:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.531+0000 7f505f400640 1 -- 172.21.15.47:0/1308994021 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5070003ec0 con 0x5631c8306bb0 2024-03-21T11:33:30.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.531+0000 7f505f400640 1 -- 172.21.15.47:0/1308994021 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f507005b050 con 0x5631c8306bb0 2024-03-21T11:33:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.531+0000 7f5083396740 10 client.28993 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.531+0000 7f5083396740 1 -- 172.21.15.47:0/1308994021 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5631c7f7a3f0 con 0x5631c8306bb0 2024-03-21T11:33:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.531+0000 7f505f400640 1 -- 172.21.15.47:0/1308994021 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5064005500 con 0x5631c8306bb0 2024-03-21T11:33:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.532+0000 7f505f400640 1 -- 172.21.15.47:0/1308994021 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f507004c8c0 con 0x5631c8306bb0 2024-03-21T11:33:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.532+0000 7f5083396740 10 client.28993 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.532+0000 7f5083396740 10 client.28993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.532+0000 7f5083396740 20 client.28993 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.532+0000 7f5083396740 1 --2- 172.21.15.47:0/1308994021 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5631c8316e90 0x5631c83192a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.532+0000 7f5083396740 4 client.28993 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:33:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.532+0000 7f5083396740 1 -- 172.21.15.47:0/1308994021 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631c8273790 con 0x5631c8316e90 2024-03-21T11:33:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.532+0000 7f507dc00640 1 --2- 172.21.15.47:0/1308994021 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5631c8316e90 0x5631c83192a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.533+0000 7f507dc00640 1 --2- 172.21.15.47:0/1308994021 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5631c8316e90 0x5631c83192a0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb4051a50a62dc8d server_cookie=2882c4dc208a2872 in_seq=0 out_seq=0 2024-03-21T11:33:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.533+0000 7f505f400640 10 client.28993 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.533+0000 7f505f400640 1 -- 172.21.15.47:0/1308994021 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5631c8273790 con 0x5631c8316e90 2024-03-21T11:33:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.533+0000 7f505f400640 10 client.28993 handle_command_reply: tid=0 2024-03-21T11:33:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.533+0000 7f5083396740 1 client.28993 shutdown 2024-03-21T11:33:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.533+0000 7f5083396740 1 -- 172.21.15.47:0/1308994021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8306bb0 msgr2=0x5631c83125a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.533+0000 7f5083396740 1 --2- 172.21.15.47:0/1308994021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8306bb0 0x5631c83125a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x5631c83022a0 tx=0x7f50700095c0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.534+0000 7f5083396740 1 -- 172.21.15.47:0/1308994021 shutdown_connections 2024-03-21T11:33:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.534+0000 7f5083396740 1 --2- 172.21.15.47:0/1308994021 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5631c8316e90 0x5631c83192a0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.534+0000 7f5083396740 1 --2- 172.21.15.47:0/1308994021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8306bb0 0x5631c83125a0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.534+0000 7f5083396740 1 --2- 172.21.15.47:0/1308994021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301a40 0x5631c830b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.534+0000 7f5083396740 1 --2- 172.21.15.47:0/1308994021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8301050 0x5631c830b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.534+0000 7f505f400640 0 client.28993 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.534+0000 7f5083396740 1 -- 172.21.15.47:0/1308994021 >> 172.21.15.47:0/1308994021 conn(0x5631c82c9200 msgr2=0x5631c82fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.534+0000 7f5083396740 1 -- 172.21.15.47:0/1308994021 shutdown_connections 2024-03-21T11:33:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.534+0000 7f5083396740 1 -- 172.21.15.47:0/1308994021 wait complete. 2024-03-21T11:33:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.534+0000 7f5083396740 20 client.28993 trim_cache size 0 max 16384 2024-03-21T11:33:30.551 INFO:tasks.workunit.client.0.smithi047.stdout: 1 752995 8.09 MB/sec execute 374 sec latency 25.473 ms 2024-03-21T11:33:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.553+0000 7f5083396740 1 Processor -- start 2024-03-21T11:33:30.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.553+0000 7f5083396740 1 -- start start 2024-03-21T11:33:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.553+0000 7f5083396740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8301050 0x5631c8302b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.553+0000 7f5083396740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301a40 0x5631c8312560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.553+0000 7f5083396740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8306bb0 0x5631c82ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.553+0000 7f5083396740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50781b03b0 con 0x5631c8301a40 2024-03-21T11:33:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.553+0000 7f5083396740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50781b1d30 con 0x5631c8306bb0 2024-03-21T11:33:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.553+0000 7f5083396740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50781b0a10 con 0x5631c8301050 2024-03-21T11:33:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.553+0000 7f507dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8301050 0x5631c8302b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.553+0000 7f507dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8301050 0x5631c8302b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52668/0 (socket says 172.21.15.47:52668) 2024-03-21T11:33:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.553+0000 7f507dc00640 1 -- 172.21.15.47:0/881762332 learned_addr learned my addr 172.21.15.47:0/881762332 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.553+0000 7f507e600640 1 --2- 172.21.15.47:0/881762332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8306bb0 0x5631c82ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.553+0000 7f507d200640 1 --2- 172.21.15.47:0/881762332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301a40 0x5631c8312560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.554+0000 7f507dc00640 1 -- 172.21.15.47:0/881762332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8306bb0 msgr2=0x5631c82ffc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.554+0000 7f507dc00640 1 --2- 172.21.15.47:0/881762332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8306bb0 0x5631c82ffc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.554+0000 7f507dc00640 1 -- 172.21.15.47:0/881762332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301a40 msgr2=0x5631c8312560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.554+0000 7f507dc00640 1 --2- 172.21.15.47:0/881762332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301a40 0x5631c8312560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.554+0000 7f507dc00640 1 -- 172.21.15.47:0/881762332 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631c7f3a0f0 con 0x5631c8301050 2024-03-21T11:33:30.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.554+0000 7f507dc00640 1 --2- 172.21.15.47:0/881762332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8301050 0x5631c8302b50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f506c00c6a0 tx=0x7f506c0d1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16cd8486c239bd11 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:30.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.554+0000 7f506a000640 1 -- 172.21.15.47:0/881762332 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c09be30 con 0x5631c8301050 2024-03-21T11:33:30.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.554+0000 7f506a000640 1 -- 172.21.15.47:0/881762332 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f506c09fed0 con 0x5631c8301050 2024-03-21T11:33:30.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.554+0000 7f506a000640 1 -- 172.21.15.47:0/881762332 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c09b280 con 0x5631c8301050 2024-03-21T11:33:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.555+0000 7f5083396740 1 -- 172.21.15.47:0/881762332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8301050 msgr2=0x5631c8302b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.555+0000 7f5083396740 1 --2- 172.21.15.47:0/881762332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8301050 0x5631c8302b50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f506c00c6a0 tx=0x7f506c0d1040 comp rx=0 tx=0).stop 2024-03-21T11:33:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.555+0000 7f5083396740 1 -- 172.21.15.47:0/881762332 shutdown_connections 2024-03-21T11:33:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.555+0000 7f5083396740 1 --2- 172.21.15.47:0/881762332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8306bb0 0x5631c82ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.555+0000 7f5083396740 1 --2- 172.21.15.47:0/881762332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301a40 0x5631c8312560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.555+0000 7f5083396740 1 --2- 172.21.15.47:0/881762332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8301050 0x5631c8302b50 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.555+0000 7f5083396740 1 -- 172.21.15.47:0/881762332 >> 172.21.15.47:0/881762332 conn(0x5631c82c9200 msgr2=0x5631c8302440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.555+0000 7f5083396740 1 -- 172.21.15.47:0/881762332 shutdown_connections 2024-03-21T11:33:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.555+0000 7f5083396740 1 -- 172.21.15.47:0/881762332 wait complete. 2024-03-21T11:33:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.556+0000 7f5083396740 1 -- start start 2024-03-21T11:33:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.556+0000 7f5083396740 1 Processor -- start 2024-03-21T11:33:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.556+0000 7f5083396740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301050 0x5631c830f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.556+0000 7f5083396740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8301a40 0x5631c830ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.556+0000 7f5083396740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8306bb0 0x5631c8310f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.556+0000 7f5083396740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50781b16d0 con 0x5631c8301050 2024-03-21T11:33:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.556+0000 7f5083396740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50781b1070 con 0x5631c8301a40 2024-03-21T11:33:30.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.556+0000 7f5083396740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50781ae570 con 0x5631c8306bb0 2024-03-21T11:33:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.556+0000 7f5083396740 10 client.0 _ll_register_callbacks cb 0x7f5070044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f507dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301050 0x5631c830f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f507d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8301a40 0x5631c830ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f507e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8306bb0 0x5631c8310f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f507d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8301a40 0x5631c830ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33566/0 (socket says 172.21.15.47:33566) 2024-03-21T11:33:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f507dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301050 0x5631c830f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58146/0 (socket says 172.21.15.47:58146) 2024-03-21T11:33:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f507d200640 1 -- 172.21.15.47:0/3610063040 learned_addr learned my addr 172.21.15.47:0/3610063040 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f507e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8306bb0 0x5631c8310f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52684/0 (socket says 172.21.15.47:52684) 2024-03-21T11:33:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f507e600640 1 -- 172.21.15.47:0/3610063040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8301a40 msgr2=0x5631c830ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f507e600640 1 --2- 172.21.15.47:0/3610063040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8301a40 0x5631c830ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f507e600640 1 -- 172.21.15.47:0/3610063040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301050 msgr2=0x5631c830f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f507e600640 1 --2- 172.21.15.47:0/3610063040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301050 0x5631c830f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f507e600640 1 -- 172.21.15.47:0/3610063040 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631c8311510 con 0x5631c8306bb0 2024-03-21T11:33:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f507dc00640 1 --2- 172.21.15.47:0/3610063040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301050 0x5631c830f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f507e600640 1 --2- 172.21.15.47:0/3610063040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8306bb0 0x5631c8310f20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f507004cd30 tx=0x7f507004c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2846fcc0f7dc7f67 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.557+0000 7f505f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:33:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.558+0000 7f505f400640 1 -- 172.21.15.47:0/3610063040 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070050070 con 0x5631c8306bb0 2024-03-21T11:33:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.558+0000 7f505f400640 1 -- 172.21.15.47:0/3610063040 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5070034640 con 0x5631c8306bb0 2024-03-21T11:33:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.558+0000 7f5083396740 1 -- 172.21.15.47:0/3610063040 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f506c09fed0 con 0x5631c8306bb0 2024-03-21T11:33:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.558+0000 7f505f400640 1 -- 172.21.15.47:0/3610063040 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5070039040 con 0x5631c8306bb0 2024-03-21T11:33:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.558+0000 7f5083396740 10 client.28999 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.558+0000 7f5083396740 1 -- 172.21.15.47:0/3610063040 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5631c8312890 con 0x5631c8306bb0 2024-03-21T11:33:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.558+0000 7f505f400640 1 -- 172.21.15.47:0/3610063040 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f506c09fed0 con 0x5631c8306bb0 2024-03-21T11:33:30.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.558+0000 7f505f400640 1 -- 172.21.15.47:0/3610063040 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5070040730 con 0x5631c8306bb0 2024-03-21T11:33:30.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.558+0000 7f5083396740 10 client.28999 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:30.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.558+0000 7f5083396740 10 client.28999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:30.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.558+0000 7f5083396740 20 client.28999 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:30.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.558+0000 7f5083396740 1 --2- 172.21.15.47:0/3610063040 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5631c8372df0 0x5631c8375210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:30.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.558+0000 7f5083396740 4 client.28999 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:33:30.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.558+0000 7f5083396740 1 -- 172.21.15.47:0/3610063040 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5631c826dae0 con 0x5631c8372df0 2024-03-21T11:33:30.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.559+0000 7f507dc00640 1 --2- 172.21.15.47:0/3610063040 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5631c8372df0 0x5631c8375210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.559+0000 7f507dc00640 1 --2- 172.21.15.47:0/3610063040 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5631c8372df0 0x5631c8375210 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85961a70ecedcc3c server_cookie=ebb4909d883629f0 in_seq=0 out_seq=0 2024-03-21T11:33:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.559+0000 7f505f400640 10 client.28999 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.560+0000 7f505f400640 1 -- 172.21.15.47:0/3610063040 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5631c826dae0 con 0x5631c8372df0 2024-03-21T11:33:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.560+0000 7f505f400640 10 client.28999 handle_command_reply: tid=0 2024-03-21T11:33:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.560+0000 7f5083396740 1 client.28999 shutdown 2024-03-21T11:33:30.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.561+0000 7f5083396740 1 -- 172.21.15.47:0/3610063040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8306bb0 msgr2=0x5631c8310f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.561+0000 7f5083396740 1 --2- 172.21.15.47:0/3610063040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8306bb0 0x5631c8310f20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f507004cd30 tx=0x7f507004c720 comp rx=0 tx=0).stop 2024-03-21T11:33:30.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.561+0000 7f5083396740 1 -- 172.21.15.47:0/3610063040 shutdown_connections 2024-03-21T11:33:30.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.561+0000 7f5083396740 1 --2- 172.21.15.47:0/3610063040 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5631c8372df0 0x5631c8375210 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.561+0000 7f5083396740 1 --2- 172.21.15.47:0/3610063040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631c8306bb0 0x5631c8310f20 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.561+0000 7f5083396740 1 --2- 172.21.15.47:0/3610063040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5631c8301a40 0x5631c830ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.561+0000 7f5083396740 1 --2- 172.21.15.47:0/3610063040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5631c8301050 0x5631c830f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.561+0000 7f505f400640 0 client.28999 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:30.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.561+0000 7f5083396740 1 -- 172.21.15.47:0/3610063040 >> 172.21.15.47:0/3610063040 conn(0x5631c82c9200 msgr2=0x5631c8302cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:30.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.561+0000 7f5083396740 1 -- 172.21.15.47:0/3610063040 shutdown_connections 2024-03-21T11:33:30.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.561+0000 7f5083396740 1 -- 172.21.15.47:0/3610063040 wait complete. 2024-03-21T11:33:30.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.561+0000 7f5083396740 20 client.28999 trim_cache size 0 max 16384 2024-03-21T11:33:30.566 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:33:30.566 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:33:30.566 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:33:30.566 INFO:teuthology.orchestra.run.smithi047.stdout: "73463606-772b-4030-a544-e9d60dbc601a": { 2024-03-21T11:33:30.566 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:33:30.566 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "73463606-772b-4030-a544-e9d60dbc601a", 2024-03-21T11:33:30.566 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:33:30.566 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:33:30.566 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:33:30.566 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:33:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.562+0000 7f507f000640 1 -- 172.21.15.47:0/440393691 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50600a2fa0 msgr2=0x7f50600a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.562+0000 7f507f000640 1 --2- 172.21.15.47:0/440393691 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50600a2fa0 0x7f50600a5460 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5070032420 tx=0x7f5070032590 comp rx=0 tx=0).stop 2024-03-21T11:33:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.562+0000 7f507f000640 1 -- 172.21.15.47:0/440393691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50780ffdf0 msgr2=0x7f507819ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.562+0000 7f507f000640 1 --2- 172.21.15.47:0/440393691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50780ffdf0 0x7f507819ddc0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f506c00e9f0 tx=0x7f506c00eeb0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.562+0000 7f507f000640 1 -- 172.21.15.47:0/440393691 shutdown_connections 2024-03-21T11:33:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.562+0000 7f507f000640 1 --2- 172.21.15.47:0/440393691 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50600a2fa0 0x7f50600a5460 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.562+0000 7f507f000640 1 --2- 172.21.15.47:0/440393691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5078105110 0x7f50781a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.562+0000 7f507f000640 1 --2- 172.21.15.47:0/440393691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5078102760 0x7f507819e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.562+0000 7f507f000640 1 --2- 172.21.15.47:0/440393691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50780ffdf0 0x7f507819ddc0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.562+0000 7f507f000640 1 -- 172.21.15.47:0/440393691 >> 172.21.15.47:0/440393691 conn(0x7f50780fba10 msgr2=0x7f5078100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.562+0000 7f507f000640 1 -- 172.21.15.47:0/440393691 shutdown_connections 2024-03-21T11:33:30.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:33:30.562+0000 7f507f000640 1 -- 172.21.15.47:0/440393691 wait complete. 2024-03-21T11:33:30.581 INFO:tasks.cephfs.filesystem:scrub status for tag:73463606-772b-4030-a544-e9d60dbc601a - {'path': '/', 'tag': '73463606-772b-4030-a544-e9d60dbc601a', 'options': 'recursive,force'} 2024-03-21T11:33:30.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:30 smithi102 ceph-mon[36748]: pgmap v705: 129 pgs: 129 active+clean; 37 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-03-21T11:33:30.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:30 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:33:30.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:30 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:33:30.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:30 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:33:30.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:30 smithi047 ceph-mon[31351]: pgmap v705: 129 pgs: 129 active+clean; 37 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-03-21T11:33:30.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:30 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:33:30.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:30 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:33:30.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:30 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:33:30.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:30 smithi100 ceph-mon[36245]: pgmap v705: 129 pgs: 129 active+clean; 37 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-03-21T11:33:30.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:30 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:33:30.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:30 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:33:30.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:30 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:33:31.551 INFO:tasks.workunit.client.0.smithi047.stdout: 1 753894 8.07 MB/sec execute 375 sec latency 16.706 ms 2024-03-21T11:33:31.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:31 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:33:31.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:31 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:33:31.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:31 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:33:32.554 INFO:tasks.workunit.client.0.smithi047.stdout: 1 755970 8.08 MB/sec execute 376 sec latency 15.152 ms 2024-03-21T11:33:32.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:32 smithi047 ceph-mon[31351]: pgmap v706: 129 pgs: 129 active+clean; 33 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 246 op/s 2024-03-21T11:33:32.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:32 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:33:32.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:32 smithi100 ceph-mon[36245]: pgmap v706: 129 pgs: 129 active+clean; 33 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 246 op/s 2024-03-21T11:33:32.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:32 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:33:32.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:32 smithi102 ceph-mon[36748]: pgmap v706: 129 pgs: 129 active+clean; 33 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 246 op/s 2024-03-21T11:33:32.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:32 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:33:33.551 INFO:tasks.workunit.client.0.smithi047.stdout: 1 757081 8.07 MB/sec execute 377 sec latency 13.669 ms 2024-03-21T11:33:34.552 INFO:tasks.workunit.client.0.smithi047.stdout: 1 758779 8.07 MB/sec execute 378 sec latency 70.283 ms 2024-03-21T11:33:34.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:34 smithi047 ceph-mon[31351]: pgmap v707: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 310 op/s 2024-03-21T11:33:34.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:34 smithi100 ceph-mon[36245]: pgmap v707: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 310 op/s 2024-03-21T11:33:34.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:34 smithi102 ceph-mon[36748]: pgmap v707: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 310 op/s 2024-03-21T11:33:35.552 INFO:tasks.workunit.client.0.smithi047.stdout: 1 760306 8.08 MB/sec execute 379 sec latency 16.632 ms 2024-03-21T11:33:36.552 INFO:tasks.workunit.client.0.smithi047.stdout: 1 761777 8.07 MB/sec execute 380 sec latency 12.055 ms 2024-03-21T11:33:36.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:36 smithi047 ceph-mon[31351]: pgmap v708: 129 pgs: 129 active+clean; 35 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-03-21T11:33:36.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:36 smithi100 ceph-mon[36245]: pgmap v708: 129 pgs: 129 active+clean; 35 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-03-21T11:33:36.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:36 smithi102 ceph-mon[36748]: pgmap v708: 129 pgs: 129 active+clean; 35 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-03-21T11:33:37.552 INFO:tasks.workunit.client.0.smithi047.stdout: 1 763720 8.09 MB/sec execute 381 sec latency 12.502 ms 2024-03-21T11:33:38.552 INFO:tasks.workunit.client.0.smithi047.stdout: 1 764671 8.07 MB/sec execute 382 sec latency 24.432 ms 2024-03-21T11:33:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:38 smithi047 ceph-mon[31351]: pgmap v709: 129 pgs: 129 active+clean; 35 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2024-03-21T11:33:38.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:38 smithi100 ceph-mon[36245]: pgmap v709: 129 pgs: 129 active+clean; 35 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2024-03-21T11:33:38.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:38 smithi102 ceph-mon[36748]: pgmap v709: 129 pgs: 129 active+clean; 35 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 273 op/s 2024-03-21T11:33:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.130+0000 7fda40000640 1 -- 172.21.15.47:0/4265680993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda381054b0 msgr2=0x7fda3810bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.130+0000 7fda40000640 1 --2- 172.21.15.47:0/4265680993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda381054b0 0x7fda3810bd40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fda30009ca0 tx=0x7fda3002c0f0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.130+0000 7fda40000640 1 -- 172.21.15.47:0/4265680993 shutdown_connections 2024-03-21T11:33:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.130+0000 7fda40000640 1 --2- 172.21.15.47:0/4265680993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda381054b0 0x7fda3810bd40 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.130+0000 7fda40000640 1 --2- 172.21.15.47:0/4265680993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda38104af0 0x7fda38104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.130+0000 7fda40000640 1 --2- 172.21.15.47:0/4265680993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda38103900 0x7fda38103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.130+0000 7fda40000640 1 -- 172.21.15.47:0/4265680993 >> 172.21.15.47:0/4265680993 conn(0x7fda380fee90 msgr2=0x7fda381012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.130+0000 7fda40000640 1 -- 172.21.15.47:0/4265680993 shutdown_connections 2024-03-21T11:33:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.130+0000 7fda40000640 1 -- 172.21.15.47:0/4265680993 wait complete. 2024-03-21T11:33:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.131+0000 7fda40000640 1 Processor -- start 2024-03-21T11:33:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.131+0000 7fda40000640 1 -- start start 2024-03-21T11:33:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.131+0000 7fda40000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda38103900 0x7fda38199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.131+0000 7fda40000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda38104af0 0x7fda3819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda40000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda381054b0 0x7fda381a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda40000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3810f560 con 0x7fda38103900 2024-03-21T11:33:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda40000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3810f3e0 con 0x7fda38104af0 2024-03-21T11:33:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda40000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3810f6e0 con 0x7fda381054b0 2024-03-21T11:33:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda3d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda38103900 0x7fda38199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda3d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda38103900 0x7fda38199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48488/0 (socket says 172.21.15.47:48488) 2024-03-21T11:33:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda3d600640 1 -- 172.21.15.47:0/3833979292 learned_addr learned my addr 172.21.15.47:0/3833979292 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda3e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda381054b0 0x7fda381a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda3cc00640 1 --2- 172.21.15.47:0/3833979292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda38104af0 0x7fda3819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda3e000640 1 -- 172.21.15.47:0/3833979292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda38104af0 msgr2=0x7fda3819a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda3e000640 1 --2- 172.21.15.47:0/3833979292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda38104af0 0x7fda3819a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda3e000640 1 -- 172.21.15.47:0/3833979292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda38103900 msgr2=0x7fda38199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda3e000640 1 --2- 172.21.15.47:0/3833979292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda38103900 0x7fda38199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda3e000640 1 -- 172.21.15.47:0/3833979292 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda381a1700 con 0x7fda381054b0 2024-03-21T11:33:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.132+0000 7fda3d600640 1 --2- 172.21.15.47:0/3833979292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda38103900 0x7fda38199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.133+0000 7fda3e000640 1 --2- 172.21.15.47:0/3833979292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda381054b0 0x7fda381a1090 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fda3002c630 tx=0x7fda30032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.133+0000 7fda36000640 1 -- 172.21.15.47:0/3833979292 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30041020 con 0x7fda381054b0 2024-03-21T11:33:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.133+0000 7fda36000640 1 -- 172.21.15.47:0/3833979292 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda30034cd0 con 0x7fda381054b0 2024-03-21T11:33:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.133+0000 7fda36000640 1 -- 172.21.15.47:0/3833979292 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30039040 con 0x7fda381054b0 2024-03-21T11:33:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.133+0000 7fda40000640 1 -- 172.21.15.47:0/3833979292 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda381a1990 con 0x7fda381054b0 2024-03-21T11:33:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.133+0000 7fda40000640 1 -- 172.21.15.47:0/3833979292 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda381a1e00 con 0x7fda381054b0 2024-03-21T11:33:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.136+0000 7fda36000640 1 -- 172.21.15.47:0/3833979292 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fda30044020 con 0x7fda381054b0 2024-03-21T11:33:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.136+0000 7fda42ddf740 1 Processor -- start 2024-03-21T11:33:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.137+0000 7fda42ddf740 1 -- start start 2024-03-21T11:33:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.137+0000 7fda42ddf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd41e0 0x5610c7cd4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.137+0000 7fda42ddf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd4bd0 0x5610c7cd96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.137+0000 7fda42ddf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd9d40 0x5610c7cdc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.137+0000 7fda42ddf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c7c973b0 con 0x5610c7cd4bd0 2024-03-21T11:33:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.137+0000 7fda42ddf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c7c39830 con 0x5610c7cd41e0 2024-03-21T11:33:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.137+0000 7fda42ddf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c7be25d0 con 0x5610c7cd9d40 2024-03-21T11:33:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.137+0000 7fda3cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd41e0 0x5610c7cd4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.137+0000 7fda3d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd4bd0 0x5610c7cd96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.137+0000 7fda3cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd41e0 0x5610c7cd4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40200/0 (socket says 172.21.15.47:40200) 2024-03-21T11:33:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.137+0000 7fda3cc00640 1 -- 172.21.15.47:0/3956951932 learned_addr learned my addr 172.21.15.47:0/3956951932 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.137+0000 7fda3cc00640 1 --2- 172.21.15.47:0/3956951932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd9d40 0x5610c7cdc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.138+0000 7fda3d600640 1 -- 172.21.15.47:0/3956951932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd9d40 msgr2=0x5610c7cdc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.138+0000 7fda3d600640 1 --2- 172.21.15.47:0/3956951932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd9d40 0x5610c7cdc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.138+0000 7fda3d600640 1 -- 172.21.15.47:0/3956951932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd41e0 msgr2=0x5610c7cd4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.138+0000 7fda3d600640 1 --2- 172.21.15.47:0/3956951932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd41e0 0x5610c7cd4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.138+0000 7fda3d600640 1 -- 172.21.15.47:0/3956951932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610c7bfa2c0 con 0x5610c7cd4bd0 2024-03-21T11:33:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.138+0000 7fda3cc00640 1 --2- 172.21.15.47:0/3956951932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd9d40 0x5610c7cdc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:33:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.138+0000 7fda36000640 1 --2- 172.21.15.47:0/3833979292 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda200a3040 0x7fda200a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.138+0000 7fda3d600640 1 --2- 172.21.15.47:0/3956951932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd4bd0 0x5610c7cd96d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0114c0 tx=0x7fda2c011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24feb6750937b0c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.138+0000 7fda36000640 1 -- 172.21.15.47:0/3833979292 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fda300f14d0 con 0x7fda381054b0 2024-03-21T11:33:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.138+0000 7fda3cc00640 1 --2- 172.21.15.47:0/3833979292 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda200a3040 0x7fda200a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.138+0000 7fda2a600640 1 -- 172.21.15.47:0/3956951932 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c005020 con 0x5610c7cd4bd0 2024-03-21T11:33:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.138+0000 7fda2a600640 1 -- 172.21.15.47:0/3956951932 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda2c0047a0 con 0x5610c7cd4bd0 2024-03-21T11:33:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.139+0000 7fda2a600640 1 -- 172.21.15.47:0/3956951932 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00f9d0 con 0x5610c7cd4bd0 2024-03-21T11:33:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.139+0000 7fda42ddf740 1 -- 172.21.15.47:0/3956951932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd4bd0 msgr2=0x5610c7cd96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.139+0000 7fda42ddf740 1 --2- 172.21.15.47:0/3956951932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd4bd0 0x5610c7cd96d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0114c0 tx=0x7fda2c011980 comp rx=0 tx=0).stop 2024-03-21T11:33:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.139+0000 7fda3cc00640 1 --2- 172.21.15.47:0/3833979292 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda200a3040 0x7fda200a5500 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x5610c7cd5430 tx=0x7fda24015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.139+0000 7fda42ddf740 1 -- 172.21.15.47:0/3956951932 shutdown_connections 2024-03-21T11:33:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.139+0000 7fda42ddf740 1 --2- 172.21.15.47:0/3956951932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd9d40 0x5610c7cdc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.139+0000 7fda42ddf740 1 --2- 172.21.15.47:0/3956951932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd4bd0 0x5610c7cd96d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.139+0000 7fda42ddf740 1 --2- 172.21.15.47:0/3956951932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd41e0 0x5610c7cd4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.139+0000 7fda42ddf740 1 -- 172.21.15.47:0/3956951932 >> 172.21.15.47:0/3956951932 conn(0x5610c7ccf750 msgr2=0x5610c7ccfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.139+0000 7fda42ddf740 1 -- 172.21.15.47:0/3956951932 shutdown_connections 2024-03-21T11:33:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.140+0000 7fda42ddf740 1 -- 172.21.15.47:0/3956951932 wait complete. 2024-03-21T11:33:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.140+0000 7fda42ddf740 1 -- start start 2024-03-21T11:33:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.140+0000 7fda42ddf740 1 Processor -- start 2024-03-21T11:33:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.140+0000 7fda42ddf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7cde330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.141+0000 7fda42ddf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd4bd0 0x5610c7cde870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.141+0000 7fda42ddf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd9d40 0x5610c7ce58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.141+0000 7fda42ddf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c7bf5a20 con 0x5610c7cd4bd0 2024-03-21T11:33:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.141+0000 7fda42ddf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c7c358d0 con 0x5610c7cd9d40 2024-03-21T11:33:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.141+0000 7fda42ddf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c7cdccf0 con 0x5610c7cd41e0 2024-03-21T11:33:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.141+0000 7fda42ddf740 10 client.0 _ll_register_callbacks cb 0x5610c7c321c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.141+0000 7fda3d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7cde330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.141+0000 7fda3d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7cde330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36802/0 (socket says 172.21.15.47:36802) 2024-03-21T11:33:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.141+0000 7fda3d600640 1 -- 172.21.15.47:0/2624876228 learned_addr learned my addr 172.21.15.47:0/2624876228 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.141+0000 7fda3e000640 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd9d40 0x5610c7ce58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.141+0000 7fda3cc00640 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd4bd0 0x5610c7cde870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.142+0000 7fda3d600640 1 -- 172.21.15.47:0/2624876228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd9d40 msgr2=0x5610c7ce58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.142+0000 7fda3d600640 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd9d40 0x5610c7ce58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.142+0000 7fda3d600640 1 -- 172.21.15.47:0/2624876228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd4bd0 msgr2=0x5610c7cde870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.142+0000 7fda3d600640 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd4bd0 0x5610c7cde870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.142+0000 7fda3d600640 1 -- 172.21.15.47:0/2624876228 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610c794e210 con 0x5610c7cd41e0 2024-03-21T11:33:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.142+0000 7fda3e000640 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd9d40 0x5610c7ce58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.142+0000 7fda3cc00640 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd4bd0 0x5610c7cde870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:33:39.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.142+0000 7fda3d600640 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7cde330 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fda2c004f20 tx=0x7fda2c005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f102ebcda9861b41 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:39.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.142+0000 7fda1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:33:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.142+0000 7fda1ea00640 1 -- 172.21.15.47:0/2624876228 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c028030 con 0x5610c7cd41e0 2024-03-21T11:33:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.143+0000 7fda1ea00640 1 -- 172.21.15.47:0/2624876228 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fda2c016040 con 0x5610c7cd41e0 2024-03-21T11:33:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.143+0000 7fda42ddf740 1 -- 172.21.15.47:0/2624876228 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda2c0047a0 con 0x5610c7cd41e0 2024-03-21T11:33:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.143+0000 7fda1ea00640 1 -- 172.21.15.47:0/2624876228 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda2c032050 con 0x5610c7cd41e0 2024-03-21T11:33:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.143+0000 7fda42ddf740 10 client.29011 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.143+0000 7fda42ddf740 1 -- 172.21.15.47:0/2624876228 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5610c7a1a620 con 0x5610c7cd41e0 2024-03-21T11:33:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.143+0000 7fda1ea00640 1 -- 172.21.15.47:0/2624876228 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda2c0047a0 con 0x5610c7cd41e0 2024-03-21T11:33:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.143+0000 7fda1ea00640 1 -- 172.21.15.47:0/2624876228 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fda2c023520 con 0x5610c7cd41e0 2024-03-21T11:33:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.143+0000 7fda42ddf740 10 client.29011 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.143+0000 7fda42ddf740 10 client.29011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.143+0000 7fda42ddf740 20 client.29011 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.143+0000 7fda42ddf740 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5610c7ce9f00 0x5610c7cec2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.143+0000 7fda42ddf740 4 client.29011 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:33:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.143+0000 7fda42ddf740 1 -- 172.21.15.47:0/2624876228 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610c7c46d70 con 0x5610c7ce9f00 2024-03-21T11:33:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.144+0000 7fda3cc00640 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5610c7ce9f00 0x5610c7cec2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.144+0000 7fda3cc00640 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5610c7ce9f00 0x5610c7cec2f0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=482254ceeb53c16d server_cookie=13dac517301813aa in_seq=0 out_seq=0 2024-03-21T11:33:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.144+0000 7fda1ea00640 10 client.29011 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.145+0000 7fda1ea00640 1 -- 172.21.15.47:0/2624876228 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5610c7c46d70 con 0x5610c7ce9f00 2024-03-21T11:33:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.145+0000 7fda1ea00640 10 client.29011 handle_command_reply: tid=0 2024-03-21T11:33:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.145+0000 7fda42ddf740 1 client.29011 shutdown 2024-03-21T11:33:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.145+0000 7fda42ddf740 1 -- 172.21.15.47:0/2624876228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 msgr2=0x5610c7cde330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.145+0000 7fda42ddf740 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7cde330 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fda2c004f20 tx=0x7fda2c005230 comp rx=0 tx=0).stop 2024-03-21T11:33:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.146+0000 7fda42ddf740 1 -- 172.21.15.47:0/2624876228 shutdown_connections 2024-03-21T11:33:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.146+0000 7fda42ddf740 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5610c7ce9f00 0x5610c7cec2f0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.146+0000 7fda42ddf740 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd9d40 0x5610c7ce58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.146+0000 7fda42ddf740 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd4bd0 0x5610c7cde870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.146+0000 7fda42ddf740 1 --2- 172.21.15.47:0/2624876228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7cde330 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.146+0000 7fda42ddf740 1 -- 172.21.15.47:0/2624876228 >> 172.21.15.47:0/2624876228 conn(0x5610c7ccf750 msgr2=0x5610c7cce950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.146+0000 7fda1ea00640 0 client.29011 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.146+0000 7fda42ddf740 1 -- 172.21.15.47:0/2624876228 shutdown_connections 2024-03-21T11:33:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.146+0000 7fda42ddf740 1 -- 172.21.15.47:0/2624876228 wait complete. 2024-03-21T11:33:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.146+0000 7fda42ddf740 20 client.29011 trim_cache size 0 max 16384 2024-03-21T11:33:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.162+0000 7fda42ddf740 1 Processor -- start 2024-03-21T11:33:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.163+0000 7fda42ddf740 1 -- start start 2024-03-21T11:33:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.163+0000 7fda42ddf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7cd29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.163+0000 7fda42ddf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd4bd0 0x5610c7ce57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.163+0000 7fda42ddf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd9d40 0x5610c7cd2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.163+0000 7fda42ddf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda381ac200 con 0x5610c7cd9d40 2024-03-21T11:33:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.163+0000 7fda42ddf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda381adb80 con 0x5610c7cd4bd0 2024-03-21T11:33:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.163+0000 7fda42ddf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda381ac860 con 0x5610c7cd41e0 2024-03-21T11:33:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.163+0000 7fda3cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd4bd0 0x5610c7ce57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.163+0000 7fda3d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7cd29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.163+0000 7fda3cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd4bd0 0x5610c7ce57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40218/0 (socket says 172.21.15.47:40218) 2024-03-21T11:33:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.163+0000 7fda3cc00640 1 -- 172.21.15.47:0/3469590844 learned_addr learned my addr 172.21.15.47:0/3469590844 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.163+0000 7fda3e000640 1 --2- 172.21.15.47:0/3469590844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd9d40 0x5610c7cd2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.164+0000 7fda3cc00640 1 -- 172.21.15.47:0/3469590844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 msgr2=0x5610c7cd29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.164+0000 7fda3cc00640 1 --2- 172.21.15.47:0/3469590844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7cd29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.164+0000 7fda3cc00640 1 -- 172.21.15.47:0/3469590844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd9d40 msgr2=0x5610c7cd2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.164+0000 7fda3cc00640 1 --2- 172.21.15.47:0/3469590844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd9d40 0x5610c7cd2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.164+0000 7fda3cc00640 1 -- 172.21.15.47:0/3469590844 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610c7cd04c0 con 0x5610c7cd4bd0 2024-03-21T11:33:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.164+0000 7fda3d600640 1 --2- 172.21.15.47:0/3469590844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7cd29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.164+0000 7fda3cc00640 1 --2- 172.21.15.47:0/3469590844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd4bd0 0x5610c7ce57d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fda240058c0 tx=0x7fda24004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fbd9cb974040565 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.164+0000 7fda29c00640 1 -- 172.21.15.47:0/3469590844 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2402c2d0 con 0x5610c7cd4bd0 2024-03-21T11:33:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.164+0000 7fda29c00640 1 -- 172.21.15.47:0/3469590844 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda24004160 con 0x5610c7cd4bd0 2024-03-21T11:33:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.164+0000 7fda29c00640 1 -- 172.21.15.47:0/3469590844 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2402c920 con 0x5610c7cd4bd0 2024-03-21T11:33:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.165+0000 7fda42ddf740 1 -- 172.21.15.47:0/3469590844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd4bd0 msgr2=0x5610c7ce57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.165+0000 7fda42ddf740 1 --2- 172.21.15.47:0/3469590844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd4bd0 0x5610c7ce57d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fda240058c0 tx=0x7fda24004730 comp rx=0 tx=0).stop 2024-03-21T11:33:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.165+0000 7fda42ddf740 1 -- 172.21.15.47:0/3469590844 shutdown_connections 2024-03-21T11:33:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.165+0000 7fda42ddf740 1 --2- 172.21.15.47:0/3469590844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd9d40 0x5610c7cd2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.165+0000 7fda42ddf740 1 --2- 172.21.15.47:0/3469590844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd4bd0 0x5610c7ce57d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.165+0000 7fda42ddf740 1 --2- 172.21.15.47:0/3469590844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7cd29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.165+0000 7fda42ddf740 1 -- 172.21.15.47:0/3469590844 >> 172.21.15.47:0/3469590844 conn(0x5610c7ccf750 msgr2=0x5610c7cd5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.165+0000 7fda42ddf740 1 -- 172.21.15.47:0/3469590844 shutdown_connections 2024-03-21T11:33:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.165+0000 7fda42ddf740 1 -- 172.21.15.47:0/3469590844 wait complete. 2024-03-21T11:33:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.166+0000 7fda42ddf740 1 -- start start 2024-03-21T11:33:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.166+0000 7fda42ddf740 1 Processor -- start 2024-03-21T11:33:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.166+0000 7fda42ddf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7ce32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda42ddf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd4bd0 0x5610c7ce37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda42ddf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd9d40 0x5610c7ce47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda42ddf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda381ad520 con 0x5610c7cd9d40 2024-03-21T11:33:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda42ddf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda381acec0 con 0x5610c7cd4bd0 2024-03-21T11:33:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda42ddf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda381aa3c0 con 0x5610c7cd41e0 2024-03-21T11:33:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda42ddf740 10 client.0 _ll_register_callbacks cb 0x7fda24009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda3cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd4bd0 0x5610c7ce37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda3d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7ce32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda3cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd4bd0 0x5610c7ce37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40220/0 (socket says 172.21.15.47:40220) 2024-03-21T11:33:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda3d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7ce32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36814/0 (socket says 172.21.15.47:36814) 2024-03-21T11:33:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda3cc00640 1 -- 172.21.15.47:0/497094011 learned_addr learned my addr 172.21.15.47:0/497094011 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda3e000640 1 --2- 172.21.15.47:0/497094011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd9d40 0x5610c7ce47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda3cc00640 1 -- 172.21.15.47:0/497094011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 msgr2=0x5610c7ce32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda3cc00640 1 --2- 172.21.15.47:0/497094011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7ce32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda3cc00640 1 -- 172.21.15.47:0/497094011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd9d40 msgr2=0x5610c7ce47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda3cc00640 1 --2- 172.21.15.47:0/497094011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd9d40 0x5610c7ce47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.167+0000 7fda3cc00640 1 -- 172.21.15.47:0/497094011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610c7ce4de0 con 0x5610c7cd4bd0 2024-03-21T11:33:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.168+0000 7fda3cc00640 1 --2- 172.21.15.47:0/497094011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd4bd0 0x5610c7ce37e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fda24006010 tx=0x7fda2402c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bfadc5e754a7a5b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.168+0000 7fda1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:33:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.168+0000 7fda1ea00640 1 -- 172.21.15.47:0/497094011 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2402c760 con 0x5610c7cd4bd0 2024-03-21T11:33:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.168+0000 7fda42ddf740 1 -- 172.21.15.47:0/497094011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda24004160 con 0x5610c7cd4bd0 2024-03-21T11:33:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.168+0000 7fda1ea00640 1 -- 172.21.15.47:0/497094011 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fda2402c900 con 0x5610c7cd4bd0 2024-03-21T11:33:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.169+0000 7fda1ea00640 1 -- 172.21.15.47:0/497094011 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda2402cb00 con 0x5610c7cd4bd0 2024-03-21T11:33:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.169+0000 7fda42ddf740 10 client.28997 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.169+0000 7fda42ddf740 1 -- 172.21.15.47:0/497094011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5610c7d45a60 con 0x5610c7cd4bd0 2024-03-21T11:33:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.169+0000 7fda1ea00640 1 -- 172.21.15.47:0/497094011 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda24004160 con 0x5610c7cd4bd0 2024-03-21T11:33:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.169+0000 7fda1ea00640 1 -- 172.21.15.47:0/497094011 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fda240234c0 con 0x5610c7cd4bd0 2024-03-21T11:33:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.169+0000 7fda42ddf740 10 client.28997 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.169+0000 7fda42ddf740 10 client.28997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.169+0000 7fda42ddf740 20 client.28997 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.169+0000 7fda42ddf740 1 --2- 172.21.15.47:0/497094011 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5610c7d46350 0x5610c7d48770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.169+0000 7fda42ddf740 4 client.28997 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:33:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.169+0000 7fda42ddf740 1 -- 172.21.15.47:0/497094011 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5610c7c410c0 con 0x5610c7d46350 2024-03-21T11:33:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.170+0000 7fda3d600640 1 --2- 172.21.15.47:0/497094011 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5610c7d46350 0x5610c7d48770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.170+0000 7fda3d600640 1 --2- 172.21.15.47:0/497094011 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5610c7d46350 0x5610c7d48770 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5950f75ebffe80ad server_cookie=14df2c4c3c2fd4a in_seq=0 out_seq=0 2024-03-21T11:33:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.170+0000 7fda1ea00640 10 client.28997 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.409+0000 7fda1ea00640 1 -- 172.21.15.47:0/497094011 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5610c7c410c0 con 0x5610c7d46350 2024-03-21T11:33:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.409+0000 7fda1ea00640 10 client.28997 handle_command_reply: tid=0 2024-03-21T11:33:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.409+0000 7fda42ddf740 1 client.28997 shutdown 2024-03-21T11:33:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.409+0000 7fda42ddf740 1 -- 172.21.15.47:0/497094011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd4bd0 msgr2=0x5610c7ce37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.409+0000 7fda42ddf740 1 --2- 172.21.15.47:0/497094011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd4bd0 0x5610c7ce37e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fda24006010 tx=0x7fda2402c2d0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.409+0000 7fda42ddf740 1 -- 172.21.15.47:0/497094011 shutdown_connections 2024-03-21T11:33:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.409+0000 7fda42ddf740 1 --2- 172.21.15.47:0/497094011 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5610c7d46350 0x5610c7d48770 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.409+0000 7fda42ddf740 1 --2- 172.21.15.47:0/497094011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610c7cd9d40 0x5610c7ce47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.409+0000 7fda42ddf740 1 --2- 172.21.15.47:0/497094011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610c7cd4bd0 0x5610c7ce37e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.409+0000 7fda42ddf740 1 --2- 172.21.15.47:0/497094011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610c7cd41e0 0x5610c7ce32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.409+0000 7fda42ddf740 1 -- 172.21.15.47:0/497094011 >> 172.21.15.47:0/497094011 conn(0x5610c7ccf750 msgr2=0x5610c7cd29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.409+0000 7fda1ea00640 0 client.28997 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.409+0000 7fda42ddf740 1 -- 172.21.15.47:0/497094011 shutdown_connections 2024-03-21T11:33:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.410+0000 7fda42ddf740 1 -- 172.21.15.47:0/497094011 wait complete. 2024-03-21T11:33:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.410+0000 7fda42ddf740 20 client.28997 trim_cache size 0 max 16384 2024-03-21T11:33:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:33:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:33:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:33:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:33:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.410+0000 7fda40000640 1 -- 172.21.15.47:0/3833979292 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda200a3040 msgr2=0x7fda200a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.410+0000 7fda40000640 1 --2- 172.21.15.47:0/3833979292 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda200a3040 0x7fda200a5500 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x5610c7cd5430 tx=0x7fda24015040 comp rx=0 tx=0).stop 2024-03-21T11:33:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.410+0000 7fda40000640 1 -- 172.21.15.47:0/3833979292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda381054b0 msgr2=0x7fda381a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.410+0000 7fda40000640 1 --2- 172.21.15.47:0/3833979292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda381054b0 0x7fda381a1090 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fda3002c630 tx=0x7fda30032f20 comp rx=0 tx=0).stop 2024-03-21T11:33:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.411+0000 7fda40000640 1 -- 172.21.15.47:0/3833979292 shutdown_connections 2024-03-21T11:33:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.411+0000 7fda40000640 1 --2- 172.21.15.47:0/3833979292 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda200a3040 0x7fda200a5500 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.411+0000 7fda40000640 1 --2- 172.21.15.47:0/3833979292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda381054b0 0x7fda381a1090 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.411+0000 7fda40000640 1 --2- 172.21.15.47:0/3833979292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda38104af0 0x7fda3819a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.411+0000 7fda40000640 1 --2- 172.21.15.47:0/3833979292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda38103900 0x7fda38199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.411+0000 7fda40000640 1 -- 172.21.15.47:0/3833979292 >> 172.21.15.47:0/3833979292 conn(0x7fda380fee90 msgr2=0x7fda38107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.411+0000 7fda40000640 1 -- 172.21.15.47:0/3833979292 shutdown_connections 2024-03-21T11:33:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:39.411+0000 7fda40000640 1 -- 172.21.15.47:0/3833979292 wait complete. 2024-03-21T11:33:39.552 INFO:tasks.workunit.client.0.smithi047.stdout: 1 766421 8.08 MB/sec execute 383 sec latency 118.193 ms 2024-03-21T11:33:40.552 INFO:tasks.workunit.client.0.smithi047.stdout: 1 767545 8.08 MB/sec execute 384 sec latency 84.565 ms 2024-03-21T11:33:40.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:40 smithi047 ceph-mon[31351]: pgmap v710: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 251 op/s 2024-03-21T11:33:40.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:40 smithi100 ceph-mon[36245]: pgmap v710: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 251 op/s 2024-03-21T11:33:40.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:40 smithi102 ceph-mon[36748]: pgmap v710: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 251 op/s 2024-03-21T11:33:41.552 INFO:tasks.workunit.client.0.smithi047.stdout: 1 769258 8.07 MB/sec execute 385 sec latency 13.472 ms 2024-03-21T11:33:42.552 INFO:tasks.workunit.client.0.smithi047.stdout: 1 770866 8.08 MB/sec execute 386 sec latency 12.827 ms 2024-03-21T11:33:42.792 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:42 smithi102 ceph-mon[36748]: pgmap v711: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 224 op/s 2024-03-21T11:33:42.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:42 smithi047 ceph-mon[31351]: pgmap v711: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 224 op/s 2024-03-21T11:33:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:42 smithi100 ceph-mon[36245]: pgmap v711: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 224 op/s 2024-03-21T11:33:43.553 INFO:tasks.workunit.client.0.smithi047.stdout: 1 772108 8.08 MB/sec execute 387 sec latency 16.024 ms 2024-03-21T11:33:44.553 INFO:tasks.workunit.client.0.smithi047.stdout: 1 774057 8.09 MB/sec execute 388 sec latency 74.472 ms 2024-03-21T11:33:45.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:44 smithi102 ceph-mon[36748]: pgmap v712: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 312 op/s 2024-03-21T11:33:45.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:44 smithi047 ceph-mon[31351]: pgmap v712: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 312 op/s 2024-03-21T11:33:45.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:44 smithi100 ceph-mon[36245]: pgmap v712: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 312 op/s 2024-03-21T11:33:45.553 INFO:tasks.workunit.client.0.smithi047.stdout: 1 774858 8.07 MB/sec execute 389 sec latency 59.428 ms 2024-03-21T11:33:46.553 INFO:tasks.workunit.client.0.smithi047.stdout: 1 776807 8.08 MB/sec execute 390 sec latency 11.775 ms 2024-03-21T11:33:47.195 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:46 smithi100 ceph-mon[36245]: pgmap v713: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 262 op/s 2024-03-21T11:33:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:46 smithi102 ceph-mon[36748]: pgmap v713: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 262 op/s 2024-03-21T11:33:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:46 smithi047 ceph-mon[31351]: pgmap v713: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 262 op/s 2024-03-21T11:33:47.553 INFO:tasks.workunit.client.0.smithi047.stdout: 1 778117 8.08 MB/sec execute 391 sec latency 58.563 ms 2024-03-21T11:33:48.553 INFO:tasks.workunit.client.0.smithi047.stdout: 1 779707 8.07 MB/sec execute 392 sec latency 20.958 ms 2024-03-21T11:33:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:48 smithi102 ceph-mon[36748]: pgmap v714: 129 pgs: 129 active+clean; 39 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 298 op/s 2024-03-21T11:33:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:48 smithi047 ceph-mon[31351]: pgmap v714: 129 pgs: 129 active+clean; 39 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 298 op/s 2024-03-21T11:33:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:48 smithi100 ceph-mon[36245]: pgmap v714: 129 pgs: 129 active+clean; 39 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 298 op/s 2024-03-21T11:33:49.553 INFO:tasks.workunit.client.0.smithi047.stdout: 1 781439 8.08 MB/sec execute 393 sec latency 12.375 ms 2024-03-21T11:33:50.553 INFO:tasks.workunit.client.0.smithi047.stdout: 1 782202 8.07 MB/sec execute 394 sec latency 68.321 ms 2024-03-21T11:33:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:50 smithi102 ceph-mon[36748]: pgmap v715: 129 pgs: 129 active+clean; 39 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 301 op/s 2024-03-21T11:33:51.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:50 smithi047 ceph-mon[31351]: pgmap v715: 129 pgs: 129 active+clean; 39 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 301 op/s 2024-03-21T11:33:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:50 smithi100 ceph-mon[36245]: pgmap v715: 129 pgs: 129 active+clean; 39 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 301 op/s 2024-03-21T11:33:51.554 INFO:tasks.workunit.client.0.smithi047.stdout: 1 784070 8.08 MB/sec execute 395 sec latency 15.645 ms 2024-03-21T11:33:52.557 INFO:tasks.workunit.client.0.smithi047.stdout: 1 785158 8.07 MB/sec execute 396 sec latency 13.061 ms 2024-03-21T11:33:52.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.569+0000 7f19c8400640 1 -- 172.21.15.47:0/1413169728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19c010c0e0 msgr2=0x7f19c010e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.569+0000 7f19c8400640 1 --2- 172.21.15.47:0/1413169728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19c010c0e0 0x7f19c010e4d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009900 tx=0x7f19ac02bc30 comp rx=0 tx=0).stop 2024-03-21T11:33:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.569+0000 7f19c8400640 1 -- 172.21.15.47:0/1413169728 shutdown_connections 2024-03-21T11:33:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.569+0000 7f19c8400640 1 --2- 172.21.15.47:0/1413169728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c010ea60 0x7f19c0110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.569+0000 7f19c8400640 1 --2- 172.21.15.47:0/1413169728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19c010c0e0 0x7f19c010e4d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.569+0000 7f19c8400640 1 --2- 172.21.15.47:0/1413169728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19c01097b0 0x7f19c010bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.569+0000 7f19c8400640 1 -- 172.21.15.47:0/1413169728 >> 172.21.15.47:0/1413169728 conn(0x7f19c006e8a0 msgr2=0x7f19c0070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.569+0000 7f19c8400640 1 -- 172.21.15.47:0/1413169728 shutdown_connections 2024-03-21T11:33:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.569+0000 7f19c8400640 1 -- 172.21.15.47:0/1413169728 wait complete. 2024-03-21T11:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.570+0000 7f19c8400640 1 Processor -- start 2024-03-21T11:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.570+0000 7f19c8400640 1 -- start start 2024-03-21T11:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c8400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19c01097b0 0x7f19c01a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c8400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19c010c0e0 0x7f19c01a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c8400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c010ea60 0x7f19c01a9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c8400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c0114a60 con 0x7f19c01097b0 2024-03-21T11:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c8400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c01148e0 con 0x7f19c010ea60 2024-03-21T11:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c8400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c0114be0 con 0x7f19c010c0e0 2024-03-21T11:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19c010c0e0 0x7f19c01a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c010ea60 0x7f19c01a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c5a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19c01097b0 0x7f19c01a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19c010c0e0 0x7f19c01a28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58544/0 (socket says 172.21.15.47:58544) 2024-03-21T11:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c5000640 1 -- 172.21.15.47:0/2952557021 learned_addr learned my addr 172.21.15.47:0/2952557021 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c6400640 1 -- 172.21.15.47:0/2952557021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19c010c0e0 msgr2=0x7f19c01a28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c6400640 1 --2- 172.21.15.47:0/2952557021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19c010c0e0 0x7f19c01a28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c6400640 1 -- 172.21.15.47:0/2952557021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19c01097b0 msgr2=0x7f19c01a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c6400640 1 --2- 172.21.15.47:0/2952557021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19c01097b0 0x7f19c01a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.571+0000 7f19c6400640 1 -- 172.21.15.47:0/2952557021 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c01a9f90 con 0x7f19c010ea60 2024-03-21T11:33:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.572+0000 7f19c6400640 1 --2- 172.21.15.47:0/2952557021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c010ea60 0x7f19c01a9920 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009fd0 tx=0x7f19b800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.572+0000 7f19be000640 1 -- 172.21.15.47:0/2952557021 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800ce20 con 0x7f19c010ea60 2024-03-21T11:33:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.572+0000 7f19c8400640 1 -- 172.21.15.47:0/2952557021 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c01aa280 con 0x7f19c010ea60 2024-03-21T11:33:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.572+0000 7f19be000640 1 -- 172.21.15.47:0/2952557021 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19b8005020 con 0x7f19c010ea60 2024-03-21T11:33:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.572+0000 7f19be000640 1 -- 172.21.15.47:0/2952557021 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b8010720 con 0x7f19c010ea60 2024-03-21T11:33:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.572+0000 7f19c8400640 1 -- 172.21.15.47:0/2952557021 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c01aa7f0 con 0x7f19c010ea60 2024-03-21T11:33:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.575+0000 7f19cb054740 1 Processor -- start 2024-03-21T11:33:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.576+0000 7f19be000640 1 -- 172.21.15.47:0/2952557021 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f19b8005540 con 0x7f19c010ea60 2024-03-21T11:33:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.576+0000 7f19cb054740 1 -- start start 2024-03-21T11:33:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.576+0000 7f19cb054740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1670941e0 0x55c167094600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.576+0000 7f19cb054740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167094bd0 0x55c1670996d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.576+0000 7f19cb054740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167099d40 0x55c16709c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.576+0000 7f19cb054740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1670573b0 con 0x55c167099d40 2024-03-21T11:33:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.576+0000 7f19cb054740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c166ff9830 con 0x55c1670941e0 2024-03-21T11:33:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.576+0000 7f19cb054740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c166fa25d0 con 0x55c167094bd0 2024-03-21T11:33:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.577+0000 7f19c5a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167094bd0 0x55c1670996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.577+0000 7f19c5a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167094bd0 0x55c1670996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58552/0 (socket says 172.21.15.47:58552) 2024-03-21T11:33:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.577+0000 7f19c5a00640 1 -- 172.21.15.47:0/2694616 learned_addr learned my addr 172.21.15.47:0/2694616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.577+0000 7f19c5000640 1 --2- 172.21.15.47:0/2694616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1670941e0 0x55c167094600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.577+0000 7f19c5000640 1 --2- 172.21.15.47:0/2694616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167099d40 0x55c16709c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.577+0000 7f19c5a00640 1 -- 172.21.15.47:0/2694616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1670941e0 msgr2=0x55c167094600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.577+0000 7f19c5a00640 1 --2- 172.21.15.47:0/2694616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1670941e0 0x55c167094600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.577+0000 7f19c5a00640 1 -- 172.21.15.47:0/2694616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167099d40 msgr2=0x55c16709c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.577+0000 7f19c5a00640 1 --2- 172.21.15.47:0/2694616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167099d40 0x55c16709c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.577+0000 7f19c5a00640 1 -- 172.21.15.47:0/2694616 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c166fba2c0 con 0x55c167094bd0 2024-03-21T11:33:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.577+0000 7f19c5a00640 1 --2- 172.21.15.47:0/2694616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167094bd0 0x55c1670996d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f19b400b750 tx=0x7f19b400bc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=418861e83b874ee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.578+0000 7f19b2a00640 1 -- 172.21.15.47:0/2694616 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b400be60 con 0x55c167094bd0 2024-03-21T11:33:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.578+0000 7f19b2a00640 1 -- 172.21.15.47:0/2694616 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19b4015070 con 0x55c167094bd0 2024-03-21T11:33:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.578+0000 7f19b2a00640 1 -- 172.21.15.47:0/2694616 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b4010770 con 0x55c167094bd0 2024-03-21T11:33:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.578+0000 7f19be000640 1 --2- 172.21.15.47:0/2952557021 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f19a80a2fc0 0x7f19a80a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.578+0000 7f19be000640 1 -- 172.21.15.47:0/2952557021 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f19b80cfd70 con 0x7f19c010ea60 2024-03-21T11:33:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.578+0000 7f19c5000640 1 --2- 172.21.15.47:0/2952557021 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f19a80a2fc0 0x7f19a80a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.578+0000 7f19cb054740 1 -- 172.21.15.47:0/2694616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167094bd0 msgr2=0x55c1670996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.578+0000 7f19cb054740 1 --2- 172.21.15.47:0/2694616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167094bd0 0x55c1670996d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f19b400b750 tx=0x7f19b400bc10 comp rx=0 tx=0).stop 2024-03-21T11:33:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.579+0000 7f19cb054740 1 -- 172.21.15.47:0/2694616 shutdown_connections 2024-03-21T11:33:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.579+0000 7f19cb054740 1 --2- 172.21.15.47:0/2694616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c167099d40 0x55c16709c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.579+0000 7f19cb054740 1 --2- 172.21.15.47:0/2694616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167094bd0 0x55c1670996d0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.579+0000 7f19cb054740 1 --2- 172.21.15.47:0/2694616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1670941e0 0x55c167094600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.579+0000 7f19cb054740 1 -- 172.21.15.47:0/2694616 >> 172.21.15.47:0/2694616 conn(0x55c16708f750 msgr2=0x55c16708fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.579+0000 7f19cb054740 1 -- 172.21.15.47:0/2694616 shutdown_connections 2024-03-21T11:33:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.579+0000 7f19cb054740 1 -- 172.21.15.47:0/2694616 wait complete. 2024-03-21T11:33:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.579+0000 7f19c5000640 1 --2- 172.21.15.47:0/2952557021 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f19a80a2fc0 0x7f19a80a5480 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f19ac000c00 tx=0x7f19ac02c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.580+0000 7f19cb054740 1 -- start start 2024-03-21T11:33:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.580+0000 7f19cb054740 1 Processor -- start 2024-03-21T11:33:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.580+0000 7f19cb054740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1670941e0 0x55c16709e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.580+0000 7f19cb054740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167094bd0 0x55c16709e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.580+0000 7f19cb054740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167099d40 0x55c1670a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.580+0000 7f19cb054740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c166fb5a20 con 0x55c1670941e0 2024-03-21T11:33:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.580+0000 7f19cb054740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c166ff58d0 con 0x55c167094bd0 2024-03-21T11:33:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.580+0000 7f19cb054740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16709ccf0 con 0x55c167099d40 2024-03-21T11:33:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.580+0000 7f19cb054740 10 client.0 _ll_register_callbacks cb 0x55c166ff21c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.581+0000 7f19c6400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167099d40 0x55c1670a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.581+0000 7f19c5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167094bd0 0x55c16709e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.581+0000 7f19c6400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167099d40 0x55c1670a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58560/0 (socket says 172.21.15.47:58560) 2024-03-21T11:33:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.581+0000 7f19c5a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1670941e0 0x55c16709e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.581+0000 7f19c6400640 1 -- 172.21.15.47:0/1424780901 learned_addr learned my addr 172.21.15.47:0/1424780901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.581+0000 7f19c6400640 1 -- 172.21.15.47:0/1424780901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167094bd0 msgr2=0x55c16709e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.581+0000 7f19c6400640 1 --2- 172.21.15.47:0/1424780901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167094bd0 0x55c16709e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.581+0000 7f19c6400640 1 -- 172.21.15.47:0/1424780901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1670941e0 msgr2=0x55c16709e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.581+0000 7f19c6400640 1 --2- 172.21.15.47:0/1424780901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1670941e0 0x55c16709e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.581+0000 7f19c6400640 1 -- 172.21.15.47:0/1424780901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c166d0e210 con 0x55c167099d40 2024-03-21T11:33:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.581+0000 7f19c6400640 1 --2- 172.21.15.47:0/1424780901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167099d40 0x55c1670a58e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f19b80a4590 tx=0x7f19b809c170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c915e36f53b8e9d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.581+0000 7f19a7400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:33:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.581+0000 7f19a7400640 1 -- 172.21.15.47:0/1424780901 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b80bc0f0 con 0x55c167099d40 2024-03-21T11:33:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.582+0000 7f19a7400640 1 -- 172.21.15.47:0/1424780901 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f19b80a4630 con 0x55c167099d40 2024-03-21T11:33:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.582+0000 7f19cb054740 1 -- 172.21.15.47:0/1424780901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19b4015070 con 0x55c167099d40 2024-03-21T11:33:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.582+0000 7f19a7400640 1 -- 172.21.15.47:0/1424780901 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19b80bc290 con 0x55c167099d40 2024-03-21T11:33:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.582+0000 7f19cb054740 10 client.29029 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.582+0000 7f19cb054740 1 -- 172.21.15.47:0/1424780901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c166dda620 con 0x55c167099d40 2024-03-21T11:33:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.582+0000 7f19a7400640 1 -- 172.21.15.47:0/1424780901 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19b4015070 con 0x55c167099d40 2024-03-21T11:33:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.582+0000 7f19a7400640 1 -- 172.21.15.47:0/1424780901 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f19b809af70 con 0x55c167099d40 2024-03-21T11:33:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.582+0000 7f19cb054740 10 client.29029 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.582+0000 7f19cb054740 10 client.29029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.582+0000 7f19cb054740 20 client.29029 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.582+0000 7f19cb054740 1 --2- 172.21.15.47:0/1424780901 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c1670aa250 0x55c1670ac680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.582+0000 7f19cb054740 4 client.29029 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:33:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.582+0000 7f19cb054740 1 -- 172.21.15.47:0/1424780901 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c167006d70 con 0x55c1670aa250 2024-03-21T11:33:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.583+0000 7f19c5a00640 1 --2- 172.21.15.47:0/1424780901 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c1670aa250 0x55c1670ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.583+0000 7f19c5a00640 1 --2- 172.21.15.47:0/1424780901 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c1670aa250 0x55c1670ac680 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47c31aa01506e275 server_cookie=a5fe7a3fb4208c50 in_seq=0 out_seq=0 2024-03-21T11:33:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.583+0000 7f19a7400640 10 client.29029 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.584+0000 7f19a7400640 1 -- 172.21.15.47:0/1424780901 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c167006d70 con 0x55c1670aa250 2024-03-21T11:33:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.584+0000 7f19a7400640 10 client.29029 handle_command_reply: tid=0 2024-03-21T11:33:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.584+0000 7f19cb054740 1 client.29029 shutdown 2024-03-21T11:33:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.584+0000 7f19cb054740 1 -- 172.21.15.47:0/1424780901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167099d40 msgr2=0x55c1670a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.584+0000 7f19cb054740 1 --2- 172.21.15.47:0/1424780901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167099d40 0x55c1670a58e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f19b80a4590 tx=0x7f19b809c170 comp rx=0 tx=0).stop 2024-03-21T11:33:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.584+0000 7f19cb054740 1 -- 172.21.15.47:0/1424780901 shutdown_connections 2024-03-21T11:33:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.584+0000 7f19cb054740 1 --2- 172.21.15.47:0/1424780901 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c1670aa250 0x55c1670ac680 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.584+0000 7f19cb054740 1 --2- 172.21.15.47:0/1424780901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167099d40 0x55c1670a58e0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.584+0000 7f19cb054740 1 --2- 172.21.15.47:0/1424780901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167094bd0 0x55c16709e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.584+0000 7f19cb054740 1 --2- 172.21.15.47:0/1424780901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1670941e0 0x55c16709e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.584+0000 7f19cb054740 1 -- 172.21.15.47:0/1424780901 >> 172.21.15.47:0/1424780901 conn(0x55c16708f750 msgr2=0x55c16708e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.584+0000 7f19a7400640 0 client.29029 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.585+0000 7f19cb054740 1 -- 172.21.15.47:0/1424780901 shutdown_connections 2024-03-21T11:33:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.585+0000 7f19cb054740 1 -- 172.21.15.47:0/1424780901 wait complete. 2024-03-21T11:33:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.585+0000 7f19cb054740 20 client.29029 trim_cache size 0 max 16384 2024-03-21T11:33:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.601+0000 7f19cb054740 1 Processor -- start 2024-03-21T11:33:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.602+0000 7f19cb054740 1 -- start start 2024-03-21T11:33:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.602+0000 7f19cb054740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c16708f750 0x55c1670929b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.602+0000 7f19cb054740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1670941e0 0x55c1670a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.602+0000 7f19cb054740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167094bd0 0x55c167092ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.602+0000 7f19cb054740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c01b49f0 con 0x55c16708f750 2024-03-21T11:33:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.602+0000 7f19cb054740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c01b6370 con 0x55c167094bd0 2024-03-21T11:33:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.602+0000 7f19cb054740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c01b5050 con 0x55c1670941e0 2024-03-21T11:33:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.602+0000 7f19c5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1670941e0 0x55c1670a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.602+0000 7f19c6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167094bd0 0x55c167092ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.602+0000 7f19c6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167094bd0 0x55c167092ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59470/0 (socket says 172.21.15.47:59470) 2024-03-21T11:33:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.602+0000 7f19c5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1670941e0 0x55c1670a58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58572/0 (socket says 172.21.15.47:58572) 2024-03-21T11:33:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.602+0000 7f19c6400640 1 -- 172.21.15.47:0/480612871 learned_addr learned my addr 172.21.15.47:0/480612871 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.602+0000 7f19c5a00640 1 --2- 172.21.15.47:0/480612871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c16708f750 0x55c1670929b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.603+0000 7f19c6400640 1 -- 172.21.15.47:0/480612871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1670941e0 msgr2=0x55c1670a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.603+0000 7f19c6400640 1 --2- 172.21.15.47:0/480612871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1670941e0 0x55c1670a58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.603+0000 7f19c6400640 1 -- 172.21.15.47:0/480612871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c16708f750 msgr2=0x55c1670929b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.603+0000 7f19c6400640 1 --2- 172.21.15.47:0/480612871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c16708f750 0x55c1670929b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.603+0000 7f19c6400640 1 -- 172.21.15.47:0/480612871 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c16709ec10 con 0x55c167094bd0 2024-03-21T11:33:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.603+0000 7f19c5a00640 1 --2- 172.21.15.47:0/480612871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c16708f750 0x55c1670929b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:33:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.603+0000 7f19c6400640 1 --2- 172.21.15.47:0/480612871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167094bd0 0x55c167092ef0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f19b8098a80 tx=0x7f19b809a5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d8a091a4c8fe32c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.603+0000 7f19b2000640 1 -- 172.21.15.47:0/480612871 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b80a14e0 con 0x55c167094bd0 2024-03-21T11:33:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.603+0000 7f19b2000640 1 -- 172.21.15.47:0/480612871 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19b80051d0 con 0x55c167094bd0 2024-03-21T11:33:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.604+0000 7f19b2000640 1 -- 172.21.15.47:0/480612871 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b80a4630 con 0x55c167094bd0 2024-03-21T11:33:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.605+0000 7f19cb054740 1 -- 172.21.15.47:0/480612871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167094bd0 msgr2=0x55c167092ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.605+0000 7f19cb054740 1 --2- 172.21.15.47:0/480612871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167094bd0 0x55c167092ef0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f19b8098a80 tx=0x7f19b809a5c0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.605+0000 7f19cb054740 1 -- 172.21.15.47:0/480612871 shutdown_connections 2024-03-21T11:33:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.605+0000 7f19cb054740 1 --2- 172.21.15.47:0/480612871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c167094bd0 0x55c167092ef0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.605+0000 7f19cb054740 1 --2- 172.21.15.47:0/480612871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1670941e0 0x55c1670a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.605+0000 7f19cb054740 1 --2- 172.21.15.47:0/480612871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c16708f750 0x55c1670929b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.605+0000 7f19cb054740 1 -- 172.21.15.47:0/480612871 >> 172.21.15.47:0/480612871 conn(0x55c167099840 msgr2=0x55c167095e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.605+0000 7f19cb054740 1 -- 172.21.15.47:0/480612871 shutdown_connections 2024-03-21T11:33:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.605+0000 7f19cb054740 1 -- 172.21.15.47:0/480612871 wait complete. 2024-03-21T11:33:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.606+0000 7f19cb054740 1 -- start start 2024-03-21T11:33:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.606+0000 7f19cb054740 1 Processor -- start 2024-03-21T11:33:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.606+0000 7f19cb054740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c16708f750 0x55c1670a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19cb054740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1670941e0 0x55c1670a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19cb054740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167094bd0 0x55c1670a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19cb054740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c01b5d10 con 0x55c16708f750 2024-03-21T11:33:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19cb054740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c01b56b0 con 0x55c1670941e0 2024-03-21T11:33:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19cb054740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c01b2bb0 con 0x55c167094bd0 2024-03-21T11:33:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19cb054740 10 client.0 _ll_register_callbacks cb 0x7f19b8005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:33:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19c6400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167094bd0 0x55c1670a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19c5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1670941e0 0x55c1670a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19c5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1670941e0 0x55c1670a3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59472/0 (socket says 172.21.15.47:59472) 2024-03-21T11:33:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19c6400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167094bd0 0x55c1670a4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58576/0 (socket says 172.21.15.47:58576) 2024-03-21T11:33:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19c5000640 1 -- 172.21.15.47:0/998597799 learned_addr learned my addr 172.21.15.47:0/998597799 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:33:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19c5a00640 1 --2- 172.21.15.47:0/998597799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c16708f750 0x55c1670a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19c6400640 1 -- 172.21.15.47:0/998597799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1670941e0 msgr2=0x55c1670a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19c6400640 1 --2- 172.21.15.47:0/998597799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1670941e0 0x55c1670a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19c6400640 1 -- 172.21.15.47:0/998597799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c16708f750 msgr2=0x55c1670a35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:33:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19c6400640 1 --2- 172.21.15.47:0/998597799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c16708f750 0x55c1670a35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19c6400640 1 -- 172.21.15.47:0/998597799 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1670a5110 con 0x55c167094bd0 2024-03-21T11:33:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.607+0000 7f19c5000640 1 --2- 172.21.15.47:0/998597799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1670941e0 0x55c1670a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:33:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.608+0000 7f19c6400640 1 --2- 172.21.15.47:0/998597799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167094bd0 0x55c1670a4b20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f19b809b330 tx=0x7f19b809b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4acabb675c8175bc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:33:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.608+0000 7f19a7400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:33:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.608+0000 7f19a7400640 1 -- 172.21.15.47:0/998597799 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b807e070 con 0x55c167094bd0 2024-03-21T11:33:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.608+0000 7f19a7400640 1 -- 172.21.15.47:0/998597799 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f19b809b700 con 0x55c167094bd0 2024-03-21T11:33:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.608+0000 7f19cb054740 1 -- 172.21.15.47:0/998597799 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19b80051d0 con 0x55c167094bd0 2024-03-21T11:33:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.608+0000 7f19a7400640 1 -- 172.21.15.47:0/998597799 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19b80bc0b0 con 0x55c167094bd0 2024-03-21T11:33:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.608+0000 7f19cb054740 10 client.29035 fetch_fsmap learned FSMap version 28 2024-03-21T11:33:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.608+0000 7f19cb054740 1 -- 172.21.15.47:0/998597799 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c1671061a0 con 0x55c167094bd0 2024-03-21T11:33:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.608+0000 7f19a7400640 1 -- 172.21.15.47:0/998597799 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19b80051d0 con 0x55c167094bd0 2024-03-21T11:33:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.608+0000 7f19a7400640 1 -- 172.21.15.47:0/998597799 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f19b8081020 con 0x55c167094bd0 2024-03-21T11:33:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.609+0000 7f19cb054740 10 client.29035 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:33:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.609+0000 7f19cb054740 10 client.29035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:33:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.609+0000 7f19cb054740 20 client.29035 populate_metadata read hostname 'smithi047' 2024-03-21T11:33:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.609+0000 7f19cb054740 1 --2- 172.21.15.47:0/998597799 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c167099d40 0x55c167108850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:33:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.609+0000 7f19cb054740 4 client.29035 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:33:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.609+0000 7f19cb054740 1 -- 172.21.15.47:0/998597799 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c1670010c0 con 0x55c167099d40 2024-03-21T11:33:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.609+0000 7f19c5a00640 1 --2- 172.21.15.47:0/998597799 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c167099d40 0x55c167108850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:33:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.609+0000 7f19c5a00640 1 --2- 172.21.15.47:0/998597799 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c167099d40 0x55c167108850 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9d482a41680e92 server_cookie=e2888ade5a8b78ac in_seq=0 out_seq=0 2024-03-21T11:33:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.609+0000 7f19a7400640 10 client.29035 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.893+0000 7f19a7400640 1 -- 172.21.15.47:0/998597799 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c1670010c0 con 0x55c167099d40 2024-03-21T11:33:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.893+0000 7f19a7400640 10 client.29035 handle_command_reply: tid=0 2024-03-21T11:33:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.893+0000 7f19cb054740 1 client.29035 shutdown 2024-03-21T11:33:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.893+0000 7f19cb054740 1 -- 172.21.15.47:0/998597799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167094bd0 msgr2=0x55c1670a4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.893+0000 7f19cb054740 1 --2- 172.21.15.47:0/998597799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167094bd0 0x55c1670a4b20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f19b809b330 tx=0x7f19b809b430 comp rx=0 tx=0).stop 2024-03-21T11:33:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.893+0000 7f19cb054740 1 -- 172.21.15.47:0/998597799 shutdown_connections 2024-03-21T11:33:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.893+0000 7f19cb054740 1 --2- 172.21.15.47:0/998597799 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c167099d40 0x55c167108850 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.893+0000 7f19cb054740 1 --2- 172.21.15.47:0/998597799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c167094bd0 0x55c1670a4b20 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.893+0000 7f19cb054740 1 --2- 172.21.15.47:0/998597799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1670941e0 0x55c1670a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.893+0000 7f19cb054740 1 --2- 172.21.15.47:0/998597799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c16708f750 0x55c1670a35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.893+0000 7f19a7400640 0 client.29035 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:33:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.893+0000 7f19cb054740 1 -- 172.21.15.47:0/998597799 >> 172.21.15.47:0/998597799 conn(0x55c167099840 msgr2=0x55c167095e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.893+0000 7f19cb054740 1 -- 172.21.15.47:0/998597799 shutdown_connections 2024-03-21T11:33:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.893+0000 7f19cb054740 1 -- 172.21.15.47:0/998597799 wait complete. 2024-03-21T11:33:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.894+0000 7f19cb054740 20 client.29035 trim_cache size 0 max 16384 2024-03-21T11:33:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:33:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:33:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:33:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:33:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.894+0000 7f19c8400640 1 -- 172.21.15.47:0/2952557021 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f19a80a2fc0 msgr2=0x7f19a80a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.894+0000 7f19c8400640 1 --2- 172.21.15.47:0/2952557021 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f19a80a2fc0 0x7f19a80a5480 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f19ac000c00 tx=0x7f19ac02c880 comp rx=0 tx=0).stop 2024-03-21T11:33:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.894+0000 7f19c8400640 1 -- 172.21.15.47:0/2952557021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c010ea60 msgr2=0x7f19c01a9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:33:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.894+0000 7f19c8400640 1 --2- 172.21.15.47:0/2952557021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c010ea60 0x7f19c01a9920 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009fd0 tx=0x7f19b800ef10 comp rx=0 tx=0).stop 2024-03-21T11:33:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.894+0000 7f19c8400640 1 -- 172.21.15.47:0/2952557021 shutdown_connections 2024-03-21T11:33:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.894+0000 7f19c8400640 1 --2- 172.21.15.47:0/2952557021 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f19a80a2fc0 0x7f19a80a5480 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.894+0000 7f19c8400640 1 --2- 172.21.15.47:0/2952557021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19c010ea60 0x7f19c01a9920 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.894+0000 7f19c8400640 1 --2- 172.21.15.47:0/2952557021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19c010c0e0 0x7f19c01a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.894+0000 7f19c8400640 1 --2- 172.21.15.47:0/2952557021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19c01097b0 0x7f19c01a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:33:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.894+0000 7f19c8400640 1 -- 172.21.15.47:0/2952557021 >> 172.21.15.47:0/2952557021 conn(0x7f19c006e8a0 msgr2=0x7f19c0108ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:33:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.895+0000 7f19c8400640 1 -- 172.21.15.47:0/2952557021 shutdown_connections 2024-03-21T11:33:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:33:52.895+0000 7f19c8400640 1 -- 172.21.15.47:0/2952557021 wait complete. 2024-03-21T11:33:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:52 smithi102 ceph-mon[36748]: pgmap v716: 129 pgs: 129 active+clean; 39 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-03-21T11:33:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:52 smithi047 ceph-mon[31351]: pgmap v716: 129 pgs: 129 active+clean; 39 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-03-21T11:33:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:52 smithi100 ceph-mon[36245]: pgmap v716: 129 pgs: 129 active+clean; 39 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-03-21T11:33:53.554 INFO:tasks.workunit.client.0.smithi047.stdout: 1 786134 8.07 MB/sec execute 397 sec latency 64.011 ms 2024-03-21T11:33:54.554 INFO:tasks.workunit.client.0.smithi047.stdout: 1 787883 8.07 MB/sec execute 398 sec latency 16.576 ms 2024-03-21T11:33:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:54 smithi102 ceph-mon[36748]: pgmap v717: 129 pgs: 129 active+clean; 40 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 307 op/s 2024-03-21T11:33:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:54 smithi047 ceph-mon[31351]: pgmap v717: 129 pgs: 129 active+clean; 40 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 307 op/s 2024-03-21T11:33:55.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:54 smithi100 ceph-mon[36245]: pgmap v717: 129 pgs: 129 active+clean; 40 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 307 op/s 2024-03-21T11:33:55.554 INFO:tasks.workunit.client.0.smithi047.stdout: 1 788669 8.06 MB/sec execute 399 sec latency 44.790 ms 2024-03-21T11:33:56.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:55 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:33:56.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:55 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:33:56.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:55 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:33:56.554 INFO:tasks.workunit.client.0.smithi047.stdout: 1 790055 8.05 MB/sec execute 400 sec latency 22.529 ms 2024-03-21T11:33:57.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:57 smithi047 ceph-mon[31351]: pgmap v718: 129 pgs: 129 active+clean; 41 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-03-21T11:33:57.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:57 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:57 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:57 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:57 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:57 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:57 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.369 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:57 smithi102 ceph-mon[36748]: pgmap v718: 129 pgs: 129 active+clean; 41 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-03-21T11:33:57.369 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:57 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.369 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:57 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.369 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:57 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.369 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:57 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.369 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:57 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.369 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:57 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:57 smithi100 ceph-mon[36245]: pgmap v718: 129 pgs: 129 active+clean; 41 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-03-21T11:33:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:57 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:57 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:57 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:57 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:57 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:57 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:57.554 INFO:tasks.workunit.client.0.smithi047.stdout: 1 791919 8.06 MB/sec execute 401 sec latency 13.957 ms 2024-03-21T11:33:58.554 INFO:tasks.workunit.client.0.smithi047.stdout: 1 792687 8.05 MB/sec execute 402 sec latency 18.354 ms 2024-03-21T11:33:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:59 smithi047 ceph-mon[31351]: pgmap v719: 129 pgs: 129 active+clean; 39 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 284 op/s 2024-03-21T11:33:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:33:59 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:59 smithi100 ceph-mon[36245]: pgmap v719: 129 pgs: 129 active+clean; 39 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 284 op/s 2024-03-21T11:33:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:33:59 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:59 smithi102 ceph-mon[36748]: pgmap v719: 129 pgs: 129 active+clean; 39 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 284 op/s 2024-03-21T11:33:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:33:59 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:33:59.554 INFO:tasks.workunit.client.0.smithi047.stdout: 1 794777 8.06 MB/sec execute 403 sec latency 56.228 ms 2024-03-21T11:34:00.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:00 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:34:00.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:00 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:34:00.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:00 smithi102 ceph-mon[36748]: pgmap v720: 129 pgs: 129 active+clean; 35 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 269 op/s 2024-03-21T11:34:00.554 INFO:tasks.workunit.client.0.smithi047.stdout: 1 795788 8.05 MB/sec execute 404 sec latency 54.524 ms 2024-03-21T11:34:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:00 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:34:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:00 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:34:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:00 smithi047 ceph-mon[31351]: pgmap v720: 129 pgs: 129 active+clean; 35 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 269 op/s 2024-03-21T11:34:00.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:34:00.583 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:34:00.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:00 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:34:00.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:00 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:34:00.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:00 smithi100 ceph-mon[36245]: pgmap v720: 129 pgs: 129 active+clean; 35 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 269 op/s 2024-03-21T11:34:00.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.803+0000 7fbf60400640 1 -- 172.21.15.47:0/1591835165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf580fffb0 msgr2=0x7fbf581003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:00.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.803+0000 7fbf60400640 1 --2- 172.21.15.47:0/1591835165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf580fffb0 0x7fbf581003d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009900 tx=0x7fbf4c02bcc0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.803+0000 7fbf60400640 1 -- 172.21.15.47:0/1591835165 shutdown_connections 2024-03-21T11:34:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.803+0000 7fbf60400640 1 --2- 172.21.15.47:0/1591835165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf581012b0 0x7fbf58101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.803+0000 7fbf60400640 1 --2- 172.21.15.47:0/1591835165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf58100910 0x7fbf58100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.803+0000 7fbf60400640 1 --2- 172.21.15.47:0/1591835165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf580fffb0 0x7fbf581003d0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.803+0000 7fbf60400640 1 -- 172.21.15.47:0/1591835165 >> 172.21.15.47:0/1591835165 conn(0x7fbf580fbb50 msgr2=0x7fbf580fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.804+0000 7fbf60400640 1 -- 172.21.15.47:0/1591835165 shutdown_connections 2024-03-21T11:34:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.804+0000 7fbf60400640 1 -- 172.21.15.47:0/1591835165 wait complete. 2024-03-21T11:34:00.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.805+0000 7fbf60400640 1 Processor -- start 2024-03-21T11:34:00.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.805+0000 7fbf60400640 1 -- start start 2024-03-21T11:34:00.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.805+0000 7fbf60400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf580fffb0 0x7fbf58199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.806+0000 7fbf60400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf58100910 0x7fbf5819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.806+0000 7fbf60400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf581012b0 0x7fbf581a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.806+0000 7fbf60400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5810f580 con 0x7fbf581012b0 2024-03-21T11:34:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.806+0000 7fbf60400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5810f400 con 0x7fbf58100910 2024-03-21T11:34:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.806+0000 7fbf60400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5810f700 con 0x7fbf580fffb0 2024-03-21T11:34:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.806+0000 7fbf5d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf58100910 0x7fbf5819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.806+0000 7fbf5d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf58100910 0x7fbf5819a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35860/0 (socket says 172.21.15.47:35860) 2024-03-21T11:34:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.806+0000 7fbf5d000640 1 -- 172.21.15.47:0/997276195 learned_addr learned my addr 172.21.15.47:0/997276195 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.806+0000 7fbf5da00640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf580fffb0 0x7fbf58199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.806+0000 7fbf5e400640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf581012b0 0x7fbf581a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.807+0000 7fbf5d000640 1 -- 172.21.15.47:0/997276195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf580fffb0 msgr2=0x7fbf58199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:00.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.807+0000 7fbf5d000640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf580fffb0 0x7fbf58199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.807+0000 7fbf5d000640 1 -- 172.21.15.47:0/997276195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf581012b0 msgr2=0x7fbf581a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:00.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.807+0000 7fbf5d000640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf581012b0 0x7fbf581a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.807+0000 7fbf5d000640 1 -- 172.21.15.47:0/997276195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf581a17b0 con 0x7fbf58100910 2024-03-21T11:34:00.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.807+0000 7fbf5da00640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf580fffb0 0x7fbf58199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:00.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.807+0000 7fbf5d000640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf58100910 0x7fbf5819a0d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400d950 tx=0x7fbf4400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:00.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.808+0000 7fbf56000640 1 -- 172.21.15.47:0/997276195 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44017070 con 0x7fbf58100910 2024-03-21T11:34:00.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.808+0000 7fbf56000640 1 -- 172.21.15.47:0/997276195 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf44009c30 con 0x7fbf58100910 2024-03-21T11:34:00.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.808+0000 7fbf56000640 1 -- 172.21.15.47:0/997276195 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4400f6b0 con 0x7fbf58100910 2024-03-21T11:34:00.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.808+0000 7fbf60400640 1 -- 172.21.15.47:0/997276195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf581a1aa0 con 0x7fbf58100910 2024-03-21T11:34:00.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.808+0000 7fbf60400640 1 -- 172.21.15.47:0/997276195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf581a1fa0 con 0x7fbf58100910 2024-03-21T11:34:00.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.812+0000 7fbf63052740 1 Processor -- start 2024-03-21T11:34:00.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.813+0000 7fbf56000640 1 -- 172.21.15.47:0/997276195 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbf440109d0 con 0x7fbf58100910 2024-03-21T11:34:00.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.813+0000 7fbf63052740 1 -- start start 2024-03-21T11:34:00.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.814+0000 7fbf63052740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbbf690 0x557edbbbfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.814+0000 7fbf63052740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbc0080 0x557edbbc4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.814+0000 7fbf63052740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbc51f0 0x557edbbc7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.814+0000 7fbf63052740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557edbb1c250 con 0x557edbbbf690 2024-03-21T11:34:00.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.814+0000 7fbf63052740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557edbac4a30 con 0x557edbbc0080 2024-03-21T11:34:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.814+0000 7fbf63052740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557edbb182f0 con 0x557edbbc51f0 2024-03-21T11:34:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.814+0000 7fbf5da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbc0080 0x557edbbc4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.814+0000 7fbf5d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbc51f0 0x557edbbc7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.814+0000 7fbf5da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbc0080 0x557edbbc4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35870/0 (socket says 172.21.15.47:35870) 2024-03-21T11:34:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.814+0000 7fbf5da00640 1 -- 172.21.15.47:0/2497238094 learned_addr learned my addr 172.21.15.47:0/2497238094 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.814+0000 7fbf5e400640 1 --2- 172.21.15.47:0/2497238094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbbf690 0x557edbbbfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.815+0000 7fbf5da00640 1 -- 172.21.15.47:0/2497238094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbc51f0 msgr2=0x557edbbc7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.815+0000 7fbf5da00640 1 --2- 172.21.15.47:0/2497238094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbc51f0 0x557edbbc7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.815+0000 7fbf5da00640 1 -- 172.21.15.47:0/2497238094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbbf690 msgr2=0x557edbbbfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.815+0000 7fbf5da00640 1 --2- 172.21.15.47:0/2497238094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbbf690 0x557edbbbfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.815+0000 7fbf5da00640 1 -- 172.21.15.47:0/2497238094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557edbaaf690 con 0x557edbbc0080 2024-03-21T11:34:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.815+0000 7fbf5d000640 1 --2- 172.21.15.47:0/2497238094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbc51f0 0x557edbbc7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.816+0000 7fbf56000640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbf400a2fa0 0x7fbf400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.816+0000 7fbf5da00640 1 --2- 172.21.15.47:0/2497238094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbc0080 0x557edbbc4b80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c02c840 tx=0x7fbf4c009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=397d8f575ea5f410 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.816+0000 7fbf56000640 1 -- 172.21.15.47:0/997276195 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbf440cf380 con 0x7fbf58100910 2024-03-21T11:34:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.816+0000 7fbf4aa00640 1 -- 172.21.15.47:0/2497238094 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c054070 con 0x557edbbc0080 2024-03-21T11:34:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.816+0000 7fbf4aa00640 1 -- 172.21.15.47:0/2497238094 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf4c004a70 con 0x557edbbc0080 2024-03-21T11:34:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.816+0000 7fbf4aa00640 1 -- 172.21.15.47:0/2497238094 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c005190 con 0x557edbbc0080 2024-03-21T11:34:00.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.817+0000 7fbf5e400640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbf400a2fa0 0x7fbf400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.817+0000 7fbf63052740 1 -- 172.21.15.47:0/2497238094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbc0080 msgr2=0x557edbbc4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:00.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.817+0000 7fbf63052740 1 --2- 172.21.15.47:0/2497238094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbc0080 0x557edbbc4b80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c02c840 tx=0x7fbf4c009610 comp rx=0 tx=0).stop 2024-03-21T11:34:00.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.817+0000 7fbf63052740 1 -- 172.21.15.47:0/2497238094 shutdown_connections 2024-03-21T11:34:00.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.817+0000 7fbf63052740 1 --2- 172.21.15.47:0/2497238094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbc51f0 0x557edbbc7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.817+0000 7fbf5e400640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbf400a2fa0 0x7fbf400a5460 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x557edbbc08e0 tx=0x7fbf50009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.817+0000 7fbf63052740 1 --2- 172.21.15.47:0/2497238094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbc0080 0x557edbbc4b80 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.817+0000 7fbf63052740 1 --2- 172.21.15.47:0/2497238094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbbf690 0x557edbbbfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.819+0000 7fbf63052740 1 -- 172.21.15.47:0/2497238094 >> 172.21.15.47:0/2497238094 conn(0x557edbb7f200 msgr2=0x557edbbbb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.819+0000 7fbf63052740 1 -- 172.21.15.47:0/2497238094 shutdown_connections 2024-03-21T11:34:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.819+0000 7fbf63052740 1 -- 172.21.15.47:0/2497238094 wait complete. 2024-03-21T11:34:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.819+0000 7fbf63052740 1 -- start start 2024-03-21T11:34:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf63052740 1 Processor -- start 2024-03-21T11:34:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf63052740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbbf690 0x557edbbb3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf63052740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbc0080 0x557edbbb3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf63052740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbc51f0 0x557edbbb8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf63052740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557edbbb2420 con 0x557edbbbf690 2024-03-21T11:34:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf63052740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557edbad7e80 con 0x557edbbc0080 2024-03-21T11:34:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf63052740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557edbbb25a0 con 0x557edbbc51f0 2024-03-21T11:34:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf63052740 10 client.0 _ll_register_callbacks cb 0x557edbaea130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf5d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbc0080 0x557edbbb3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf5d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbc0080 0x557edbbb3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35884/0 (socket says 172.21.15.47:35884) 2024-03-21T11:34:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf5d000640 1 -- 172.21.15.47:0/1362225333 learned_addr learned my addr 172.21.15.47:0/1362225333 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf5da00640 1 --2- 172.21.15.47:0/1362225333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbbf690 0x557edbbb3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf5e400640 1 --2- 172.21.15.47:0/1362225333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbc51f0 0x557edbbb8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf5d000640 1 -- 172.21.15.47:0/1362225333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbc51f0 msgr2=0x557edbbb8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf5d000640 1 --2- 172.21.15.47:0/1362225333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbc51f0 0x557edbbb8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf5d000640 1 -- 172.21.15.47:0/1362225333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbbf690 msgr2=0x557edbbb3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.820+0000 7fbf5d000640 1 --2- 172.21.15.47:0/1362225333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbbf690 0x557edbbb3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.821+0000 7fbf5d000640 1 -- 172.21.15.47:0/1362225333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557edb8fa7d0 con 0x557edbbc0080 2024-03-21T11:34:00.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.821+0000 7fbf5d000640 1 --2- 172.21.15.47:0/1362225333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbc0080 0x557edbbb3ec0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbf440a3c00 tx=0x7fbf44097a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa1f8092cbe8dd8a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.821+0000 7fbf3f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.821+0000 7fbf3f400640 1 -- 172.21.15.47:0/1362225333 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44020070 con 0x557edbbc0080 2024-03-21T11:34:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.821+0000 7fbf3f400640 1 -- 172.21.15.47:0/1362225333 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbf44099a20 con 0x557edbbc0080 2024-03-21T11:34:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.821+0000 7fbf63052740 1 -- 172.21.15.47:0/1362225333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf4c004a70 con 0x557edbbc0080 2024-03-21T11:34:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.821+0000 7fbf3f400640 1 -- 172.21.15.47:0/1362225333 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf44009de0 con 0x557edbbc0080 2024-03-21T11:34:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.821+0000 7fbf63052740 10 client.29027 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.821+0000 7fbf63052740 1 -- 172.21.15.47:0/1362225333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557edb8303f0 con 0x557edbbc0080 2024-03-21T11:34:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.822+0000 7fbf3f400640 1 -- 172.21.15.47:0/1362225333 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf4c004a70 con 0x557edbbc0080 2024-03-21T11:34:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.822+0000 7fbf3f400640 1 -- 172.21.15.47:0/1362225333 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbf44097d90 con 0x557edbbc0080 2024-03-21T11:34:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.822+0000 7fbf63052740 10 client.29027 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.822+0000 7fbf63052740 10 client.29027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.822+0000 7fbf63052740 20 client.29027 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.822+0000 7fbf63052740 1 --2- 172.21.15.47:0/1362225333 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557edbbccb20 0x557edbbcef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.822+0000 7fbf63052740 4 client.29027 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.822+0000 7fbf63052740 1 -- 172.21.15.47:0/1362225333 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557edbb29790 con 0x557edbbccb20 2024-03-21T11:34:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.823+0000 7fbf5da00640 1 --2- 172.21.15.47:0/1362225333 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557edbbccb20 0x557edbbcef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.823+0000 7fbf5da00640 1 --2- 172.21.15.47:0/1362225333 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557edbbccb20 0x557edbbcef40 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10d011bd55396c2d server_cookie=86da8e1d51e84140 in_seq=0 out_seq=0 2024-03-21T11:34:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.823+0000 7fbf3f400640 10 client.29027 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.824+0000 7fbf3f400640 1 -- 172.21.15.47:0/1362225333 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557edbb29790 con 0x557edbbccb20 2024-03-21T11:34:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.824+0000 7fbf3f400640 10 client.29027 handle_command_reply: tid=0 2024-03-21T11:34:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.824+0000 7fbf63052740 1 client.29027 shutdown 2024-03-21T11:34:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.825+0000 7fbf63052740 1 -- 172.21.15.47:0/1362225333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbc0080 msgr2=0x557edbbb3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.825+0000 7fbf63052740 1 --2- 172.21.15.47:0/1362225333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbc0080 0x557edbbb3ec0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbf440a3c00 tx=0x7fbf44097a80 comp rx=0 tx=0).stop 2024-03-21T11:34:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.825+0000 7fbf63052740 1 -- 172.21.15.47:0/1362225333 shutdown_connections 2024-03-21T11:34:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.825+0000 7fbf63052740 1 --2- 172.21.15.47:0/1362225333 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557edbbccb20 0x557edbbcef40 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.825+0000 7fbf3f400640 0 client.29027 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.825+0000 7fbf63052740 1 --2- 172.21.15.47:0/1362225333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbc51f0 0x557edbbb8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.825+0000 7fbf63052740 1 --2- 172.21.15.47:0/1362225333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbc0080 0x557edbbb3ec0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.825+0000 7fbf63052740 1 --2- 172.21.15.47:0/1362225333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbbf690 0x557edbbb3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.825+0000 7fbf63052740 1 -- 172.21.15.47:0/1362225333 >> 172.21.15.47:0/1362225333 conn(0x557edbb7f200 msgr2=0x557edbbb1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:00.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.825+0000 7fbf63052740 1 -- 172.21.15.47:0/1362225333 shutdown_connections 2024-03-21T11:34:00.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.825+0000 7fbf63052740 1 -- 172.21.15.47:0/1362225333 wait complete. 2024-03-21T11:34:00.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.825+0000 7fbf63052740 20 client.29027 trim_cache size 0 max 16384 2024-03-21T11:34:00.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.845+0000 7fbf63052740 1 Processor -- start 2024-03-21T11:34:00.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.845+0000 7fbf63052740 1 -- start start 2024-03-21T11:34:00.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.845+0000 7fbf63052740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbbf690 0x557edbbbe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.845+0000 7fbf63052740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbc51f0 0x557edbbbe780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.845+0000 7fbf63052740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbb3ff0 0x557edbbbecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.845+0000 7fbf63052740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf581a7d90 con 0x557edbbc51f0 2024-03-21T11:34:00.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.845+0000 7fbf63052740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf581a9710 con 0x557edbbbf690 2024-03-21T11:34:00.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.845+0000 7fbf63052740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf581a83f0 con 0x557edbbb3ff0 2024-03-21T11:34:00.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.846+0000 7fbf5da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbbf690 0x557edbbbe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.846+0000 7fbf5e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbb3ff0 0x557edbbbecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.846+0000 7fbf5d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbc51f0 0x557edbbbe780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.846+0000 7fbf5e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbb3ff0 0x557edbbbecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54624/0 (socket says 172.21.15.47:54624) 2024-03-21T11:34:00.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.846+0000 7fbf5e400640 1 -- 172.21.15.47:0/140073749 learned_addr learned my addr 172.21.15.47:0/140073749 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:00.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.846+0000 7fbf5da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbbf690 0x557edbbbe240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35886/0 (socket says 172.21.15.47:35886) 2024-03-21T11:34:00.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.846+0000 7fbf5d000640 1 -- 172.21.15.47:0/140073749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbb3ff0 msgr2=0x557edbbbecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:00.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.846+0000 7fbf5d000640 1 --2- 172.21.15.47:0/140073749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbb3ff0 0x557edbbbecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.846+0000 7fbf5d000640 1 -- 172.21.15.47:0/140073749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbbf690 msgr2=0x557edbbbe240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:00.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.846+0000 7fbf5d000640 1 --2- 172.21.15.47:0/140073749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbbf690 0x557edbbbe240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.846+0000 7fbf5d000640 1 -- 172.21.15.47:0/140073749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557edbbb3770 con 0x557edbbc51f0 2024-03-21T11:34:00.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.846+0000 7fbf5d000640 1 --2- 172.21.15.47:0/140073749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbc51f0 0x557edbbbe780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x557edbbb47a0 tx=0x7fbf44005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae39ef1c0243260 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:00.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.847+0000 7fbf4a000640 1 -- 172.21.15.47:0/140073749 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44020070 con 0x557edbbc51f0 2024-03-21T11:34:00.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.847+0000 7fbf4a000640 1 -- 172.21.15.47:0/140073749 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf4409be20 con 0x557edbbc51f0 2024-03-21T11:34:00.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.847+0000 7fbf4a000640 1 -- 172.21.15.47:0/140073749 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4409f870 con 0x557edbbc51f0 2024-03-21T11:34:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.847+0000 7fbf63052740 1 -- 172.21.15.47:0/140073749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbc51f0 msgr2=0x557edbbbe780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.847+0000 7fbf63052740 1 --2- 172.21.15.47:0/140073749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbc51f0 0x557edbbbe780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x557edbbb47a0 tx=0x7fbf44005f70 comp rx=0 tx=0).stop 2024-03-21T11:34:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.848+0000 7fbf63052740 1 -- 172.21.15.47:0/140073749 shutdown_connections 2024-03-21T11:34:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.848+0000 7fbf63052740 1 --2- 172.21.15.47:0/140073749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbb3ff0 0x557edbbbecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.848+0000 7fbf63052740 1 --2- 172.21.15.47:0/140073749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbc51f0 0x557edbbbe780 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.848+0000 7fbf63052740 1 --2- 172.21.15.47:0/140073749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbbbf690 0x557edbbbe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.848+0000 7fbf63052740 1 -- 172.21.15.47:0/140073749 >> 172.21.15.47:0/140073749 conn(0x557edbb7f200 msgr2=0x557edbbb1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.848+0000 7fbf63052740 1 -- 172.21.15.47:0/140073749 shutdown_connections 2024-03-21T11:34:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.848+0000 7fbf63052740 1 -- 172.21.15.47:0/140073749 wait complete. 2024-03-21T11:34:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.848+0000 7fbf63052740 1 -- start start 2024-03-21T11:34:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.848+0000 7fbf63052740 1 Processor -- start 2024-03-21T11:34:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.849+0000 7fbf63052740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbb3ff0 0x557edbbde5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.849+0000 7fbf63052740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbbf690 0x557edbbdeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.849+0000 7fbf63052740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbc25d50 0x557edbc28140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.849+0000 7fbf63052740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf581a90b0 con 0x557edbbb3ff0 2024-03-21T11:34:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.849+0000 7fbf63052740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf581a8a50 con 0x557edbc25d50 2024-03-21T11:34:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.849+0000 7fbf63052740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf581a5f50 con 0x557edbbbf690 2024-03-21T11:34:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.849+0000 7fbf63052740 10 client.0 _ll_register_callbacks cb 0x7fbf44097300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.849+0000 7fbf5d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbbf690 0x557edbbdeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.849+0000 7fbf5e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbc25d50 0x557edbc28140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.849+0000 7fbf5da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbb3ff0 0x557edbbde5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.849+0000 7fbf5d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbbf690 0x557edbbdeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54638/0 (socket says 172.21.15.47:54638) 2024-03-21T11:34:00.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.849+0000 7fbf5d000640 1 -- 172.21.15.47:0/3100458121 learned_addr learned my addr 172.21.15.47:0/3100458121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:00.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.850+0000 7fbf5e400640 1 -- 172.21.15.47:0/3100458121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbbf690 msgr2=0x557edbbdeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:00.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.850+0000 7fbf5e400640 1 --2- 172.21.15.47:0/3100458121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbbf690 0x557edbbdeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.850+0000 7fbf5e400640 1 -- 172.21.15.47:0/3100458121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbb3ff0 msgr2=0x557edbbde5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:00.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.850+0000 7fbf5e400640 1 --2- 172.21.15.47:0/3100458121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbb3ff0 0x557edbbde5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.850+0000 7fbf5e400640 1 -- 172.21.15.47:0/3100458121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557edb7f00f0 con 0x557edbc25d50 2024-03-21T11:34:00.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.850+0000 7fbf5d000640 1 --2- 172.21.15.47:0/3100458121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbbf690 0x557edbbdeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:00.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.850+0000 7fbf5e400640 1 --2- 172.21.15.47:0/3100458121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbc25d50 0x557edbc28140 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf50012c10 tx=0x7fbf50004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27258948861a73e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:00.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.850+0000 7fbf3f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:00.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.850+0000 7fbf3f400640 1 -- 172.21.15.47:0/3100458121 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50012e40 con 0x557edbc25d50 2024-03-21T11:34:00.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.850+0000 7fbf63052740 1 -- 172.21.15.47:0/3100458121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf4409be20 con 0x557edbc25d50 2024-03-21T11:34:00.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.850+0000 7fbf3f400640 1 -- 172.21.15.47:0/3100458121 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbf50004b20 con 0x557edbc25d50 2024-03-21T11:34:00.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.850+0000 7fbf3f400640 1 -- 172.21.15.47:0/3100458121 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf50032050 con 0x557edbc25d50 2024-03-21T11:34:00.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.851+0000 7fbf63052740 10 client.29033 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:00.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.851+0000 7fbf63052740 1 -- 172.21.15.47:0/3100458121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557edbc29940 con 0x557edbc25d50 2024-03-21T11:34:00.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.851+0000 7fbf3f400640 1 -- 172.21.15.47:0/3100458121 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf4409be20 con 0x557edbc25d50 2024-03-21T11:34:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.851+0000 7fbf3f400640 1 -- 172.21.15.47:0/3100458121 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbf50019040 con 0x557edbc25d50 2024-03-21T11:34:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.851+0000 7fbf63052740 10 client.29033 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.851+0000 7fbf63052740 10 client.29033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.851+0000 7fbf63052740 20 client.29033 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.851+0000 7fbf63052740 1 --2- 172.21.15.47:0/3100458121 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557edbc2a2a0 0x557edbc2c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.851+0000 7fbf63052740 4 client.29033 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:34:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.851+0000 7fbf63052740 1 -- 172.21.15.47:0/3100458121 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557edbb23ae0 con 0x557edbc2a2a0 2024-03-21T11:34:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.852+0000 7fbf5da00640 1 --2- 172.21.15.47:0/3100458121 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557edbc2a2a0 0x557edbc2c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.852+0000 7fbf5da00640 1 --2- 172.21.15.47:0/3100458121 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557edbc2a2a0 0x557edbc2c6c0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70d9f765a89c2fd7 server_cookie=95e13cb2952d4861 in_seq=0 out_seq=0 2024-03-21T11:34:00.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.852+0000 7fbf3f400640 10 client.29033 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:00.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.853+0000 7fbf3f400640 1 -- 172.21.15.47:0/3100458121 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557edbb23ae0 con 0x557edbc2a2a0 2024-03-21T11:34:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.853+0000 7fbf3f400640 10 client.29033 handle_command_reply: tid=0 2024-03-21T11:34:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.853+0000 7fbf63052740 1 client.29033 shutdown 2024-03-21T11:34:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.854+0000 7fbf63052740 1 -- 172.21.15.47:0/3100458121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbc25d50 msgr2=0x557edbc28140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.854+0000 7fbf63052740 1 --2- 172.21.15.47:0/3100458121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbc25d50 0x557edbc28140 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf50012c10 tx=0x7fbf50004a40 comp rx=0 tx=0).stop 2024-03-21T11:34:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.854+0000 7fbf63052740 1 -- 172.21.15.47:0/3100458121 shutdown_connections 2024-03-21T11:34:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.854+0000 7fbf63052740 1 --2- 172.21.15.47:0/3100458121 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557edbc2a2a0 0x557edbc2c6c0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.854+0000 7fbf63052740 1 --2- 172.21.15.47:0/3100458121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557edbc25d50 0x557edbc28140 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.854+0000 7fbf63052740 1 --2- 172.21.15.47:0/3100458121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557edbbbf690 0x557edbbdeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.854+0000 7fbf3f400640 0 client.29033 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.854+0000 7fbf63052740 1 --2- 172.21.15.47:0/3100458121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557edbbb3ff0 0x557edbbde5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.854+0000 7fbf63052740 1 -- 172.21.15.47:0/3100458121 >> 172.21.15.47:0/3100458121 conn(0x557edbb7f200 msgr2=0x557edbbc0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.854+0000 7fbf63052740 1 -- 172.21.15.47:0/3100458121 shutdown_connections 2024-03-21T11:34:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.854+0000 7fbf63052740 1 -- 172.21.15.47:0/3100458121 wait complete. 2024-03-21T11:34:00.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.854+0000 7fbf63052740 20 client.29033 trim_cache size 0 max 16384 2024-03-21T11:34:00.858 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:34:00.858 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:34:00.858 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:34:00.858 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:34:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.855+0000 7fbf60400640 1 -- 172.21.15.47:0/997276195 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbf400a2fa0 msgr2=0x7fbf400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.855+0000 7fbf60400640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbf400a2fa0 0x7fbf400a5460 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x557edbbc08e0 tx=0x7fbf50009500 comp rx=0 tx=0).stop 2024-03-21T11:34:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.855+0000 7fbf60400640 1 -- 172.21.15.47:0/997276195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf58100910 msgr2=0x7fbf5819a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.855+0000 7fbf60400640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf58100910 0x7fbf5819a0d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400d950 tx=0x7fbf4400de10 comp rx=0 tx=0).stop 2024-03-21T11:34:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.855+0000 7fbf60400640 1 -- 172.21.15.47:0/997276195 shutdown_connections 2024-03-21T11:34:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.855+0000 7fbf60400640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbf400a2fa0 0x7fbf400a5460 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.855+0000 7fbf60400640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf581012b0 0x7fbf581a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.855+0000 7fbf60400640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf58100910 0x7fbf5819a0d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.855+0000 7fbf60400640 1 --2- 172.21.15.47:0/997276195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf580fffb0 0x7fbf58199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:00.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.855+0000 7fbf60400640 1 -- 172.21.15.47:0/997276195 >> 172.21.15.47:0/997276195 conn(0x7fbf580fbb50 msgr2=0x7fbf580fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:00.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.855+0000 7fbf60400640 1 -- 172.21.15.47:0/997276195 shutdown_connections 2024-03-21T11:34:00.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:00.855+0000 7fbf60400640 1 -- 172.21.15.47:0/997276195 wait complete. 2024-03-21T11:34:00.878 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:34:00.878 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:34:01.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.094+0000 7fd8aaa00640 1 -- 172.21.15.47:0/2933035327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8a4103700 msgr2=0x7fd8a4103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.094+0000 7fd8aaa00640 1 --2- 172.21.15.47:0/2933035327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8a4103700 0x7fd8a4103b20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd898009900 tx=0x7fd89802bc30 comp rx=0 tx=0).stop 2024-03-21T11:34:01.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.094+0000 7fd8aaa00640 1 -- 172.21.15.47:0/2933035327 shutdown_connections 2024-03-21T11:34:01.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.094+0000 7fd8aaa00640 1 --2- 172.21.15.47:0/2933035327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8a41052b0 0x7fd8a410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.094+0000 7fd8aaa00640 1 --2- 172.21.15.47:0/2933035327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8a41048f0 0x7fd8a4104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.094+0000 7fd8aaa00640 1 --2- 172.21.15.47:0/2933035327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8a4103700 0x7fd8a4103b20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.094+0000 7fd8aaa00640 1 -- 172.21.15.47:0/2933035327 >> 172.21.15.47:0/2933035327 conn(0x7fd8a40fec70 msgr2=0x7fd8a41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:01.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.094+0000 7fd8aaa00640 1 -- 172.21.15.47:0/2933035327 shutdown_connections 2024-03-21T11:34:01.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.094+0000 7fd8aaa00640 1 -- 172.21.15.47:0/2933035327 wait complete. 2024-03-21T11:34:01.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.095+0000 7fd8aaa00640 1 Processor -- start 2024-03-21T11:34:01.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.096+0000 7fd8aaa00640 1 -- start start 2024-03-21T11:34:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.096+0000 7fd8aaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8a4103700 0x7fd8a41999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.096+0000 7fd8aaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8a41048f0 0x7fd8a4199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.096+0000 7fd8aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8a41052b0 0x7fd8a41a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.096+0000 7fd8aaa00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a410f3d0 con 0x7fd8a4103700 2024-03-21T11:34:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.096+0000 7fd8aaa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a410f250 con 0x7fd8a41048f0 2024-03-21T11:34:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.096+0000 7fd8aaa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a410f550 con 0x7fd8a41052b0 2024-03-21T11:34:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.096+0000 7fd8a8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8a41048f0 0x7fd8a4199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.096+0000 7fd8a9600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8a4103700 0x7fd8a41999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.096+0000 7fd8a8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8a41048f0 0x7fd8a4199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35926/0 (socket says 172.21.15.47:35926) 2024-03-21T11:34:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.096+0000 7fd8a8c00640 1 -- 172.21.15.47:0/3674766326 learned_addr learned my addr 172.21.15.47:0/3674766326 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.096+0000 7fd8aa000640 1 --2- 172.21.15.47:0/3674766326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8a41052b0 0x7fd8a41a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.097+0000 7fd8a8c00640 1 -- 172.21.15.47:0/3674766326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8a41052b0 msgr2=0x7fd8a41a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.097+0000 7fd8a8c00640 1 --2- 172.21.15.47:0/3674766326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8a41052b0 0x7fd8a41a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.097+0000 7fd8a8c00640 1 -- 172.21.15.47:0/3674766326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8a4103700 msgr2=0x7fd8a41999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.097+0000 7fd8a8c00640 1 --2- 172.21.15.47:0/3674766326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8a4103700 0x7fd8a41999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.097+0000 7fd8a8c00640 1 -- 172.21.15.47:0/3674766326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a41a1600 con 0x7fd8a41048f0 2024-03-21T11:34:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.097+0000 7fd8a8c00640 1 --2- 172.21.15.47:0/3674766326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8a41048f0 0x7fd8a4199f20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd89000d950 tx=0x7fd89000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.097+0000 7fd8a2000640 1 -- 172.21.15.47:0/3674766326 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890017030 con 0x7fd8a41048f0 2024-03-21T11:34:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.097+0000 7fd8a2000640 1 -- 172.21.15.47:0/3674766326 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd890004020 con 0x7fd8a41048f0 2024-03-21T11:34:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.097+0000 7fd8a2000640 1 -- 172.21.15.47:0/3674766326 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8900047a0 con 0x7fd8a41048f0 2024-03-21T11:34:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.097+0000 7fd8aaa00640 1 -- 172.21.15.47:0/3674766326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a41a18f0 con 0x7fd8a41048f0 2024-03-21T11:34:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.098+0000 7fd8aaa00640 1 -- 172.21.15.47:0/3674766326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a41a1e90 con 0x7fd8a41048f0 2024-03-21T11:34:01.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.099+0000 7fd8aaa00640 1 -- 172.21.15.47:0/3674766326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a410aa20 con 0x7fd8a41048f0 2024-03-21T11:34:01.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.101+0000 7fd8a2000640 1 -- 172.21.15.47:0/3674766326 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd8900041d0 con 0x7fd8a41048f0 2024-03-21T11:34:01.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.102+0000 7fd8a2000640 1 --2- 172.21.15.47:0/3674766326 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd88c0a2fa0 0x7fd88c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.102+0000 7fd8a2000640 1 -- 172.21.15.47:0/3674766326 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd8900cf380 con 0x7fd8a41048f0 2024-03-21T11:34:01.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.102+0000 7fd8a9600640 1 --2- 172.21.15.47:0/3674766326 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd88c0a2fa0 0x7fd88c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.103+0000 7fd8a9600640 1 --2- 172.21.15.47:0/3674766326 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd88c0a2fa0 0x7fd88c0a5460 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd89802c170 tx=0x7fd89802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:01.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.107+0000 7fd8a2000640 1 -- 172.21.15.47:0/3674766326 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd89009be40 con 0x7fd8a41048f0 2024-03-21T11:34:01.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.403+0000 7fd8aaa00640 1 -- 172.21.15.47:0/3674766326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8a402d2c0 con 0x7fd8a41048f0 2024-03-21T11:34:01.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.404+0000 7fd8a2000640 1 -- 172.21.15.47:0/3674766326 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fd89009b740 con 0x7fd8a41048f0 2024-03-21T11:34:01.404 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:34:01.404 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:34:01.404 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:34:01.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.406+0000 7fd8aaa00640 1 -- 172.21.15.47:0/3674766326 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd88c0a2fa0 msgr2=0x7fd88c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.406+0000 7fd8aaa00640 1 --2- 172.21.15.47:0/3674766326 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd88c0a2fa0 0x7fd88c0a5460 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd89802c170 tx=0x7fd89802df90 comp rx=0 tx=0).stop 2024-03-21T11:34:01.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.406+0000 7fd8aaa00640 1 -- 172.21.15.47:0/3674766326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8a41048f0 msgr2=0x7fd8a4199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.406+0000 7fd8aaa00640 1 --2- 172.21.15.47:0/3674766326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8a41048f0 0x7fd8a4199f20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd89000d950 tx=0x7fd89000de10 comp rx=0 tx=0).stop 2024-03-21T11:34:01.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.406+0000 7fd8aaa00640 1 -- 172.21.15.47:0/3674766326 shutdown_connections 2024-03-21T11:34:01.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.406+0000 7fd8aaa00640 1 --2- 172.21.15.47:0/3674766326 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd88c0a2fa0 0x7fd88c0a5460 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.406+0000 7fd8aaa00640 1 --2- 172.21.15.47:0/3674766326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8a41052b0 0x7fd8a41a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.406+0000 7fd8aaa00640 1 --2- 172.21.15.47:0/3674766326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8a41048f0 0x7fd8a4199f20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.406+0000 7fd8aaa00640 1 --2- 172.21.15.47:0/3674766326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8a4103700 0x7fd8a41999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.406+0000 7fd8aaa00640 1 -- 172.21.15.47:0/3674766326 >> 172.21.15.47:0/3674766326 conn(0x7fd8a40fec70 msgr2=0x7fd8a4107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:01.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.407+0000 7fd8aaa00640 1 -- 172.21.15.47:0/3674766326 shutdown_connections 2024-03-21T11:34:01.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.407+0000 7fd8aaa00640 1 -- 172.21.15.47:0/3674766326 wait complete. 2024-03-21T11:34:01.427 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:34:01.427 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:34:01.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:01 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:34:01.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:01 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:34:01.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:01 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:34:01.494 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:01 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:34:01.494 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:01 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:34:01.494 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:01 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:34:01.554 INFO:tasks.workunit.client.0.smithi047.stdout: 1 797462 8.05 MB/sec execute 405 sec latency 17.412 ms 2024-03-21T11:34:01.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.628+0000 7fe95c600640 1 -- 172.21.15.47:0/372011752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9541073e0 msgr2=0x7fe9541097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.628+0000 7fe95c600640 1 --2- 172.21.15.47:0/372011752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9541073e0 0x7fe9541097d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009be0 tx=0x7fe94c02bef0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.629+0000 7fe95c600640 1 -- 172.21.15.47:0/372011752 shutdown_connections 2024-03-21T11:34:01.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.629+0000 7fe95c600640 1 --2- 172.21.15.47:0/372011752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9541073e0 0x7fe9541097d0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.629+0000 7fe95c600640 1 --2- 172.21.15.47:0/372011752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe954104ab0 0x7fe954106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.629+0000 7fe95c600640 1 --2- 172.21.15.47:0/372011752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe954102180 0x7fe954104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.629+0000 7fe95c600640 1 -- 172.21.15.47:0/372011752 >> 172.21.15.47:0/372011752 conn(0x7fe9540fbb10 msgr2=0x7fe9540fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:01.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.629+0000 7fe95c600640 1 -- 172.21.15.47:0/372011752 shutdown_connections 2024-03-21T11:34:01.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.629+0000 7fe95c600640 1 -- 172.21.15.47:0/372011752 wait complete. 2024-03-21T11:34:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.630+0000 7fe95c600640 1 Processor -- start 2024-03-21T11:34:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.630+0000 7fe95c600640 1 -- start start 2024-03-21T11:34:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.630+0000 7fe95c600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe954102180 0x7fe9541979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.630+0000 7fe95c600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe954104ab0 0x7fe954197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.630+0000 7fe95c600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9541073e0 0x7fe95419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.630+0000 7fe95c600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95410d370 con 0x7fe954104ab0 2024-03-21T11:34:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.630+0000 7fe95c600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95410d1f0 con 0x7fe9541073e0 2024-03-21T11:34:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.630+0000 7fe95c600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95410d4f0 con 0x7fe954102180 2024-03-21T11:34:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.630+0000 7fe959c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe954102180 0x7fe9541979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.630+0000 7fe959c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe954102180 0x7fe9541979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54664/0 (socket says 172.21.15.47:54664) 2024-03-21T11:34:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.630+0000 7fe959c00640 1 -- 172.21.15.47:0/422338882 learned_addr learned my addr 172.21.15.47:0/422338882 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.630+0000 7fe95a600640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9541073e0 0x7fe95419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.631+0000 7fe959200640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe954104ab0 0x7fe954197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.631+0000 7fe959c00640 1 -- 172.21.15.47:0/422338882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9541073e0 msgr2=0x7fe95419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.631+0000 7fe959c00640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9541073e0 0x7fe95419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.631+0000 7fe959c00640 1 -- 172.21.15.47:0/422338882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe954104ab0 msgr2=0x7fe954197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.631+0000 7fe959c00640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe954104ab0 0x7fe954197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.631+0000 7fe959c00640 1 -- 172.21.15.47:0/422338882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95419f5e0 con 0x7fe954102180 2024-03-21T11:34:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.631+0000 7fe959200640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe954104ab0 0x7fe954197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.631+0000 7fe959c00640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe954102180 0x7fe9541979c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe94800d950 tx=0x7fe94800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.631+0000 7fe952000640 1 -- 172.21.15.47:0/422338882 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948017070 con 0x7fe954102180 2024-03-21T11:34:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.631+0000 7fe952000640 1 -- 172.21.15.47:0/422338882 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe948009c30 con 0x7fe954102180 2024-03-21T11:34:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.631+0000 7fe952000640 1 -- 172.21.15.47:0/422338882 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94800f6b0 con 0x7fe954102180 2024-03-21T11:34:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.631+0000 7fe95c600640 1 -- 172.21.15.47:0/422338882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95419f8d0 con 0x7fe954102180 2024-03-21T11:34:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.632+0000 7fe95c600640 1 -- 172.21.15.47:0/422338882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95419fdd0 con 0x7fe954102180 2024-03-21T11:34:01.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.635+0000 7fe95f24c740 1 Processor -- start 2024-03-21T11:34:01.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.635+0000 7fe95f24c740 1 -- start start 2024-03-21T11:34:01.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.635+0000 7fe952000640 1 -- 172.21.15.47:0/422338882 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe9480109d0 con 0x7fe954102180 2024-03-21T11:34:01.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.635+0000 7fe95f24c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c5718c20 0x5621c5719040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.635+0000 7fe95f24c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5719df0 0x5621c571a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.635+0000 7fe95f24c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c571a7b0 0x5621c5721040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.635+0000 7fe95f24c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c56792c0 con 0x5621c571a7b0 2024-03-21T11:34:01.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.635+0000 7fe95f24c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c5621c90 con 0x5621c5719df0 2024-03-21T11:34:01.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.635+0000 7fe95f24c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c5675360 con 0x5621c5718c20 2024-03-21T11:34:01.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.635+0000 7fe95a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c5718c20 0x5621c5719040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.635+0000 7fe95a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c5718c20 0x5621c5719040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54680/0 (socket says 172.21.15.47:54680) 2024-03-21T11:34:01.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.635+0000 7fe95a600640 1 -- 172.21.15.47:0/1932204089 learned_addr learned my addr 172.21.15.47:0/1932204089 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:01.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.636+0000 7fe959c00640 1 --2- 172.21.15.47:0/1932204089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5719df0 0x5621c571a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.636+0000 7fe959200640 1 --2- 172.21.15.47:0/1932204089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c571a7b0 0x5621c5721040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.636+0000 7fe95a600640 1 -- 172.21.15.47:0/1932204089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5719df0 msgr2=0x5621c571a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.636+0000 7fe95a600640 1 --2- 172.21.15.47:0/1932204089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5719df0 0x5621c571a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.636+0000 7fe95a600640 1 -- 172.21.15.47:0/1932204089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c571a7b0 msgr2=0x5621c5721040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.636+0000 7fe95a600640 1 --2- 172.21.15.47:0/1932204089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c571a7b0 0x5621c5721040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.636+0000 7fe95a600640 1 -- 172.21.15.47:0/1932204089 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621c5646080 con 0x5621c5718c20 2024-03-21T11:34:01.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.636+0000 7fe959c00640 1 --2- 172.21.15.47:0/1932204089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5719df0 0x5621c571a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:01.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.636+0000 7fe95a600640 1 --2- 172.21.15.47:0/1932204089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c5718c20 0x5621c5719040 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe94c02c500 tx=0x7fe94c009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e47c9e91d12e0710 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:01.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.637+0000 7fe946a00640 1 -- 172.21.15.47:0/1932204089 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c043030 con 0x5621c5718c20 2024-03-21T11:34:01.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.637+0000 7fe946a00640 1 -- 172.21.15.47:0/1932204089 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe94c033bc0 con 0x5621c5718c20 2024-03-21T11:34:01.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.637+0000 7fe946a00640 1 -- 172.21.15.47:0/1932204089 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c02c960 con 0x5621c5718c20 2024-03-21T11:34:01.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.637+0000 7fe952000640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe93c0a2fa0 0x7fe93c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.637+0000 7fe952000640 1 -- 172.21.15.47:0/422338882 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe9480cf330 con 0x7fe954102180 2024-03-21T11:34:01.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.637+0000 7fe959c00640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe93c0a2fa0 0x7fe93c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.637+0000 7fe95f24c740 1 -- 172.21.15.47:0/1932204089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c5718c20 msgr2=0x5621c5719040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.637+0000 7fe95f24c740 1 --2- 172.21.15.47:0/1932204089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c5718c20 0x5621c5719040 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe94c02c500 tx=0x7fe94c009330 comp rx=0 tx=0).stop 2024-03-21T11:34:01.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.638+0000 7fe95f24c740 1 -- 172.21.15.47:0/1932204089 shutdown_connections 2024-03-21T11:34:01.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.638+0000 7fe95f24c740 1 --2- 172.21.15.47:0/1932204089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c571a7b0 0x5621c5721040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.638+0000 7fe95f24c740 1 --2- 172.21.15.47:0/1932204089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5719df0 0x5621c571a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.638+0000 7fe95f24c740 1 --2- 172.21.15.47:0/1932204089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c5718c20 0x5621c5719040 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.638+0000 7fe95f24c740 1 -- 172.21.15.47:0/1932204089 >> 172.21.15.47:0/1932204089 conn(0x5621c570fc00 msgr2=0x5621c5710000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:01.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.638+0000 7fe95f24c740 1 -- 172.21.15.47:0/1932204089 shutdown_connections 2024-03-21T11:34:01.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.638+0000 7fe95f24c740 1 -- 172.21.15.47:0/1932204089 wait complete. 2024-03-21T11:34:01.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.638+0000 7fe959c00640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe93c0a2fa0 0x7fe93c0a5460 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe94809b660 tx=0x7fe948000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:01.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.638+0000 7fe95f24c740 1 -- start start 2024-03-21T11:34:01.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.638+0000 7fe95f24c740 1 Processor -- start 2024-03-21T11:34:01.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95f24c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c5722da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95f24c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 0x5621c57232e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95f24c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 0x5621c572a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95f24c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c5721760 con 0x5621c5719df0 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95f24c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c56350d0 con 0x5621c5718c20 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95f24c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c57218e0 con 0x5621c571a7b0 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95f24c740 10 client.0 _ll_register_callbacks cb 0x5621c5639e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c5722da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c5722da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35952/0 (socket says 172.21.15.47:35952) 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95a600640 1 -- 172.21.15.47:0/4040116454 learned_addr learned my addr 172.21.15.47:0/4040116454 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe959200640 1 --2- 172.21.15.47:0/4040116454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 0x5621c57232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95a600640 1 --2- 172.21.15.47:0/4040116454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 0x5621c572a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95a600640 1 -- 172.21.15.47:0/4040116454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 msgr2=0x5621c572a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95a600640 1 --2- 172.21.15.47:0/4040116454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 0x5621c572a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95a600640 1 -- 172.21.15.47:0/4040116454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 msgr2=0x5621c57232e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95a600640 1 --2- 172.21.15.47:0/4040116454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 0x5621c57232e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.639+0000 7fe95a600640 1 -- 172.21.15.47:0/4040116454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621c538d3f0 con 0x5621c5718c20 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.640+0000 7fe95a600640 1 --2- 172.21.15.47:0/4040116454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c5722da0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009be0 tx=0x7fe94c02bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47683c27faeb2b68 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.640+0000 7fe93b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.640+0000 7fe93b400640 1 -- 172.21.15.47:0/4040116454 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c04c070 con 0x5621c5718c20 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.640+0000 7fe93b400640 1 -- 172.21.15.47:0/4040116454 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe94c02cd10 con 0x5621c5718c20 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.640+0000 7fe95f24c740 1 -- 172.21.15.47:0/4040116454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe94c033bc0 con 0x5621c5718c20 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.640+0000 7fe93b400640 1 -- 172.21.15.47:0/4040116454 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe94c043040 con 0x5621c5718c20 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.640+0000 7fe95f24c740 10 client.29051 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.640+0000 7fe95f24c740 1 -- 172.21.15.47:0/4040116454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5621c54577d0 con 0x5621c5718c20 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.640+0000 7fe93b400640 1 -- 172.21.15.47:0/4040116454 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe94c033bc0 con 0x5621c5718c20 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.641+0000 7fe93b400640 1 -- 172.21.15.47:0/4040116454 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe94c034690 con 0x5621c5718c20 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.641+0000 7fe95f24c740 10 client.29051 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.641+0000 7fe95f24c740 10 client.29051 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.641+0000 7fe95f24c740 20 client.29051 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:01.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.641+0000 7fe95f24c740 1 --2- 172.21.15.47:0/4040116454 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5621c572e480 0x5621c5730870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.641+0000 7fe95f24c740 4 client.29051 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:01.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.641+0000 7fe95f24c740 1 -- 172.21.15.47:0/4040116454 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621c5686800 con 0x5621c572e480 2024-03-21T11:34:01.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.641+0000 7fe959200640 1 --2- 172.21.15.47:0/4040116454 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5621c572e480 0x5621c5730870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.645 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:01 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:34:01.645 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:01 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:34:01.645 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:01 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:34:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.642+0000 7fe959200640 1 --2- 172.21.15.47:0/4040116454 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5621c572e480 0x5621c5730870 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=71d20028dccac3d9 server_cookie=1e151478fb92367c in_seq=0 out_seq=0 2024-03-21T11:34:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.642+0000 7fe93b400640 10 client.29051 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:34:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.643+0000 7fe93b400640 1 -- 172.21.15.47:0/4040116454 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5621c5686800 con 0x5621c572e480 2024-03-21T11:34:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.643+0000 7fe93b400640 10 client.29051 handle_command_reply: tid=0 2024-03-21T11:34:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.643+0000 7fe95f24c740 1 client.29051 shutdown 2024-03-21T11:34:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.643+0000 7fe95f24c740 1 -- 172.21.15.47:0/4040116454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 msgr2=0x5621c5722da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.643+0000 7fe95f24c740 1 --2- 172.21.15.47:0/4040116454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c5722da0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009be0 tx=0x7fe94c02bef0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.643+0000 7fe95f24c740 1 -- 172.21.15.47:0/4040116454 shutdown_connections 2024-03-21T11:34:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.643+0000 7fe95f24c740 1 --2- 172.21.15.47:0/4040116454 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5621c572e480 0x5621c5730870 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.643+0000 7fe95f24c740 1 --2- 172.21.15.47:0/4040116454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 0x5621c572a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.643+0000 7fe95f24c740 1 --2- 172.21.15.47:0/4040116454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 0x5621c57232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.643+0000 7fe95f24c740 1 --2- 172.21.15.47:0/4040116454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c5722da0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.643+0000 7fe93b400640 0 client.29051 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:34:01.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.643+0000 7fe95f24c740 1 -- 172.21.15.47:0/4040116454 >> 172.21.15.47:0/4040116454 conn(0x5621c570fc00 msgr2=0x5621c570eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:01.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.643+0000 7fe95f24c740 1 -- 172.21.15.47:0/4040116454 shutdown_connections 2024-03-21T11:34:01.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.643+0000 7fe95f24c740 1 -- 172.21.15.47:0/4040116454 wait complete. 2024-03-21T11:34:01.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.644+0000 7fe95f24c740 20 client.29051 trim_cache size 0 max 16384 2024-03-21T11:34:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.660+0000 7fe95f24c740 1 Processor -- start 2024-03-21T11:34:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.660+0000 7fe95f24c740 1 -- start start 2024-03-21T11:34:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.660+0000 7fe95f24c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c5734fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.660+0000 7fe95f24c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 0x5621c572a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.661+0000 7fe95f24c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 0x5621c5735510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.661+0000 7fe95f24c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9541a7eb0 con 0x5621c5719df0 2024-03-21T11:34:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.661+0000 7fe95f24c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9541a9830 con 0x5621c5718c20 2024-03-21T11:34:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.661+0000 7fe95f24c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9541a8510 con 0x5621c571a7b0 2024-03-21T11:34:01.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.661+0000 7fe95a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c5734fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.661+0000 7fe95a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c5734fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35960/0 (socket says 172.21.15.47:35960) 2024-03-21T11:34:01.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.661+0000 7fe95a600640 1 -- 172.21.15.47:0/4000464122 learned_addr learned my addr 172.21.15.47:0/4000464122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:01.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.661+0000 7fe95a600640 1 --2- 172.21.15.47:0/4000464122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 0x5621c5735510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.661+0000 7fe959200640 1 --2- 172.21.15.47:0/4000464122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 0x5621c572a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.661+0000 7fe95a600640 1 -- 172.21.15.47:0/4000464122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 msgr2=0x5621c5735510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.661+0000 7fe95a600640 1 --2- 172.21.15.47:0/4000464122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 0x5621c5735510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.661+0000 7fe95a600640 1 -- 172.21.15.47:0/4000464122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 msgr2=0x5621c572a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.661+0000 7fe95a600640 1 --2- 172.21.15.47:0/4000464122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 0x5621c572a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.661+0000 7fe95a600640 1 -- 172.21.15.47:0/4000464122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621c5710670 con 0x5621c5718c20 2024-03-21T11:34:01.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.662+0000 7fe95a600640 1 --2- 172.21.15.47:0/4000464122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c5734fd0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0099a0 tx=0x7fe94c009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbb327a4ef15b26b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:01.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.662+0000 7fe946000640 1 -- 172.21.15.47:0/4000464122 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c04c070 con 0x5621c5718c20 2024-03-21T11:34:01.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.662+0000 7fe946000640 1 -- 172.21.15.47:0/4000464122 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe94c050070 con 0x5621c5718c20 2024-03-21T11:34:01.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.662+0000 7fe946000640 1 -- 172.21.15.47:0/4000464122 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c034830 con 0x5621c5718c20 2024-03-21T11:34:01.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.663+0000 7fe95f24c740 1 -- 172.21.15.47:0/4000464122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 msgr2=0x5621c5734fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.663+0000 7fe95f24c740 1 --2- 172.21.15.47:0/4000464122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c5734fd0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0099a0 tx=0x7fe94c009330 comp rx=0 tx=0).stop 2024-03-21T11:34:01.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.663+0000 7fe95f24c740 1 -- 172.21.15.47:0/4000464122 shutdown_connections 2024-03-21T11:34:01.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.663+0000 7fe95f24c740 1 --2- 172.21.15.47:0/4000464122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 0x5621c5735510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.663+0000 7fe95f24c740 1 --2- 172.21.15.47:0/4000464122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 0x5621c572a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.663+0000 7fe95f24c740 1 --2- 172.21.15.47:0/4000464122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c5734fd0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.663+0000 7fe95f24c740 1 -- 172.21.15.47:0/4000464122 >> 172.21.15.47:0/4000464122 conn(0x5621c570fc00 msgr2=0x5621c57136f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:01.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.663+0000 7fe95f24c740 1 -- 172.21.15.47:0/4000464122 shutdown_connections 2024-03-21T11:34:01.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.663+0000 7fe95f24c740 1 -- 172.21.15.47:0/4000464122 wait complete. 2024-03-21T11:34:01.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.663+0000 7fe95f24c740 1 -- start start 2024-03-21T11:34:01.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.663+0000 7fe95f24c740 1 Processor -- start 2024-03-21T11:34:01.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.664+0000 7fe95f24c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c57282c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.664+0000 7fe95f24c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 0x5621c5728800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.664+0000 7fe95f24c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 0x5621c5729810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.664+0000 7fe95f24c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9541a91d0 con 0x5621c5719df0 2024-03-21T11:34:01.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.664+0000 7fe95f24c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9541a8b70 con 0x5621c5718c20 2024-03-21T11:34:01.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.664+0000 7fe95f24c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9541a6070 con 0x5621c571a7b0 2024-03-21T11:34:01.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.664+0000 7fe95f24c740 10 client.0 _ll_register_callbacks cb 0x7fe94c009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:01.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.664+0000 7fe95a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c57282c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.664+0000 7fe95a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c57282c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35970/0 (socket says 172.21.15.47:35970) 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.664+0000 7fe95a600640 1 -- 172.21.15.47:0/3889220510 learned_addr learned my addr 172.21.15.47:0/3889220510 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.664+0000 7fe95a600640 1 --2- 172.21.15.47:0/3889220510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 0x5621c5729810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.664+0000 7fe959200640 1 --2- 172.21.15.47:0/3889220510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 0x5621c5728800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.665+0000 7fe95a600640 1 -- 172.21.15.47:0/3889220510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 msgr2=0x5621c5729810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.665+0000 7fe95a600640 1 --2- 172.21.15.47:0/3889220510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 0x5621c5729810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.665+0000 7fe95a600640 1 -- 172.21.15.47:0/3889220510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 msgr2=0x5621c5728800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.665+0000 7fe95a600640 1 --2- 172.21.15.47:0/3889220510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 0x5621c5728800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.665+0000 7fe95a600640 1 -- 172.21.15.47:0/3889220510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621c5729e00 con 0x5621c5718c20 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.665+0000 7fe959200640 1 --2- 172.21.15.47:0/3889220510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 0x5621c5728800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.665+0000 7fe95a600640 1 --2- 172.21.15.47:0/3889220510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c57282c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe94c006010 tx=0x7fe94c005cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f17624b940fbfed server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.665+0000 7fe93b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.665+0000 7fe93b400640 1 -- 172.21.15.47:0/3889220510 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c04c070 con 0x5621c5718c20 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.665+0000 7fe95f24c740 1 -- 172.21.15.47:0/3889220510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe94c050070 con 0x5621c5718c20 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.665+0000 7fe93b400640 1 -- 172.21.15.47:0/3889220510 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe94c033b40 con 0x5621c5718c20 2024-03-21T11:34:01.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.665+0000 7fe93b400640 1 -- 172.21.15.47:0/3889220510 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe94c034cf0 con 0x5621c5718c20 2024-03-21T11:34:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.665+0000 7fe95f24c740 10 client.29057 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.665+0000 7fe95f24c740 1 -- 172.21.15.47:0/3889220510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5621c578a440 con 0x5621c5718c20 2024-03-21T11:34:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.666+0000 7fe93b400640 1 -- 172.21.15.47:0/3889220510 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe94c050070 con 0x5621c5718c20 2024-03-21T11:34:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.666+0000 7fe93b400640 1 -- 172.21.15.47:0/3889220510 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe94c03f5d0 con 0x5621c5718c20 2024-03-21T11:34:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.666+0000 7fe95f24c740 10 client.29057 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.666+0000 7fe95f24c740 10 client.29057 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:34:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.666+0000 7fe95f24c740 20 client.29057 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.666+0000 7fe95f24c740 1 --2- 172.21.15.47:0/3889220510 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5621c578ab00 0x5621c578cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.666+0000 7fe95f24c740 4 client.29057 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:34:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.666+0000 7fe95f24c740 1 -- 172.21.15.47:0/3889220510 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5621c5680b50 con 0x5621c578ab00 2024-03-21T11:34:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.666+0000 7fe959200640 1 --2- 172.21.15.47:0/3889220510 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5621c578ab00 0x5621c578cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe959200640 1 --2- 172.21.15.47:0/3889220510 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5621c578ab00 0x5621c578cf20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=618bf55ad103e7f6 server_cookie=733577182967735b in_seq=0 out_seq=0 2024-03-21T11:34:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe93b400640 10 client.29057 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:34:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe93b400640 1 -- 172.21.15.47:0/3889220510 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5621c5680b50 con 0x5621c578ab00 2024-03-21T11:34:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe93b400640 10 client.29057 handle_command_reply: tid=0 2024-03-21T11:34:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe95f24c740 1 client.29057 shutdown 2024-03-21T11:34:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe95f24c740 1 -- 172.21.15.47:0/3889220510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 msgr2=0x5621c57282c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe95f24c740 1 --2- 172.21.15.47:0/3889220510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c57282c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe94c006010 tx=0x7fe94c005cf0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe95f24c740 1 -- 172.21.15.47:0/3889220510 shutdown_connections 2024-03-21T11:34:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe95f24c740 1 --2- 172.21.15.47:0/3889220510 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5621c578ab00 0x5621c578cf20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe95f24c740 1 --2- 172.21.15.47:0/3889220510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621c571a7b0 0x5621c5729810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe95f24c740 1 --2- 172.21.15.47:0/3889220510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621c5719df0 0x5621c5728800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe95f24c740 1 --2- 172.21.15.47:0/3889220510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621c5718c20 0x5621c57282c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe93b400640 0 client.29057 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:34:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe95f24c740 1 -- 172.21.15.47:0/3889220510 >> 172.21.15.47:0/3889220510 conn(0x5621c570fc00 msgr2=0x5621c571d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe95f24c740 1 -- 172.21.15.47:0/3889220510 shutdown_connections 2024-03-21T11:34:01.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe95f24c740 1 -- 172.21.15.47:0/3889220510 wait complete. 2024-03-21T11:34:01.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.667+0000 7fe95f24c740 20 client.29057 trim_cache size 0 max 16384 2024-03-21T11:34:01.672 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:34:01.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.668+0000 7fe95c600640 1 -- 172.21.15.47:0/422338882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe93c0a2fa0 msgr2=0x7fe93c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.668+0000 7fe95c600640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe93c0a2fa0 0x7fe93c0a5460 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe94809b660 tx=0x7fe948000bc0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.668+0000 7fe95c600640 1 -- 172.21.15.47:0/422338882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe954102180 msgr2=0x7fe9541979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.668+0000 7fe95c600640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe954102180 0x7fe9541979c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe94800d950 tx=0x7fe94800de10 comp rx=0 tx=0).stop 2024-03-21T11:34:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.668+0000 7fe95c600640 1 -- 172.21.15.47:0/422338882 shutdown_connections 2024-03-21T11:34:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.668+0000 7fe95c600640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe93c0a2fa0 0x7fe93c0a5460 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.668+0000 7fe95c600640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9541073e0 0x7fe95419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.668+0000 7fe95c600640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe954104ab0 0x7fe954197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.668+0000 7fe95c600640 1 --2- 172.21.15.47:0/422338882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe954102180 0x7fe9541979c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.668+0000 7fe95c600640 1 -- 172.21.15.47:0/422338882 >> 172.21.15.47:0/422338882 conn(0x7fe9540fbb10 msgr2=0x7fe954102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.668+0000 7fe95c600640 1 -- 172.21.15.47:0/422338882 shutdown_connections 2024-03-21T11:34:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.668+0000 7fe95c600640 1 -- 172.21.15.47:0/422338882 wait complete. 2024-03-21T11:34:01.685 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:34:01.685 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:34:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.897+0000 7f6dcf000640 1 -- 172.21.15.47:0/2787309148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81016b0 msgr2=0x7f6dc8101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.897+0000 7f6dcf000640 1 --2- 172.21.15.47:0/2787309148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81016b0 0x7f6dc8101ad0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009900 tx=0x7f6db802bc30 comp rx=0 tx=0).stop 2024-03-21T11:34:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.898+0000 7f6dcf000640 1 -- 172.21.15.47:0/2787309148 shutdown_connections 2024-03-21T11:34:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.898+0000 7f6dcf000640 1 --2- 172.21.15.47:0/2787309148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dc8103260 0x7f6dc8109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.898+0000 7f6dcf000640 1 --2- 172.21.15.47:0/2787309148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6dc81028a0 0x7f6dc8102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.898+0000 7f6dcf000640 1 --2- 172.21.15.47:0/2787309148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81016b0 0x7f6dc8101ad0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.898+0000 7f6dcf000640 1 -- 172.21.15.47:0/2787309148 >> 172.21.15.47:0/2787309148 conn(0x7f6dc80fcc40 msgr2=0x7f6dc80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.898+0000 7f6dcf000640 1 -- 172.21.15.47:0/2787309148 shutdown_connections 2024-03-21T11:34:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.898+0000 7f6dcf000640 1 -- 172.21.15.47:0/2787309148 wait complete. 2024-03-21T11:34:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.899+0000 7f6dcf000640 1 Processor -- start 2024-03-21T11:34:01.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.899+0000 7f6dcf000640 1 -- start start 2024-03-21T11:34:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.899+0000 7f6dcf000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6dc81016b0 0x7f6dc8199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.899+0000 7f6dcf000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81028a0 0x7f6dc819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.899+0000 7f6dcf000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dc8103260 0x7f6dc81a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.899+0000 7f6dcf000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc810d350 con 0x7f6dc81016b0 2024-03-21T11:34:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.899+0000 7f6dcf000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc810d1d0 con 0x7f6dc8103260 2024-03-21T11:34:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.899+0000 7f6dcf000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc810d4d0 con 0x7f6dc81028a0 2024-03-21T11:34:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.900+0000 7f6dc7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81028a0 0x7f6dc819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.900+0000 7f6dc7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6dc81016b0 0x7f6dc8199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.900+0000 7f6dc7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81028a0 0x7f6dc819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54724/0 (socket says 172.21.15.47:54724) 2024-03-21T11:34:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.900+0000 7f6dc7400640 1 -- 172.21.15.47:0/1164937521 learned_addr learned my addr 172.21.15.47:0/1164937521 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.900+0000 7f6dcd000640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dc8103260 0x7f6dc81a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.900+0000 7f6dc7400640 1 -- 172.21.15.47:0/1164937521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dc8103260 msgr2=0x7f6dc81a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.900+0000 7f6dc7400640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dc8103260 0x7f6dc81a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.900+0000 7f6dc7400640 1 -- 172.21.15.47:0/1164937521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6dc81016b0 msgr2=0x7f6dc8199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.900+0000 7f6dc7400640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6dc81016b0 0x7f6dc8199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.900+0000 7f6dc7400640 1 -- 172.21.15.47:0/1164937521 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc81a1870 con 0x7f6dc81028a0 2024-03-21T11:34:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.900+0000 7f6dc7e00640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6dc81016b0 0x7f6dc8199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.900+0000 7f6dcd000640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dc8103260 0x7f6dc81a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.900+0000 7f6dc7400640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81028a0 0x7f6dc819a1a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6db400e9f0 tx=0x7f6db400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.901+0000 7f6dc4c00640 1 -- 172.21.15.47:0/1164937521 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db400cd80 con 0x7f6dc81028a0 2024-03-21T11:34:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.901+0000 7f6dc4c00640 1 -- 172.21.15.47:0/1164937521 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6db4010910 con 0x7f6dc81028a0 2024-03-21T11:34:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.901+0000 7f6dc4c00640 1 -- 172.21.15.47:0/1164937521 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4018990 con 0x7f6dc81028a0 2024-03-21T11:34:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.901+0000 7f6dcf000640 1 -- 172.21.15.47:0/1164937521 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc81a1b60 con 0x7f6dc81028a0 2024-03-21T11:34:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.901+0000 7f6dcf000640 1 -- 172.21.15.47:0/1164937521 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc81a2030 con 0x7f6dc81028a0 2024-03-21T11:34:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.904+0000 7f6dd1c1e740 1 Processor -- start 2024-03-21T11:34:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.904+0000 7f6dd1c1e740 1 -- start start 2024-03-21T11:34:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.904+0000 7f6dd1c1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21dd810 0x55d5d21ddc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.904+0000 7f6dd1c1e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21dee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.904+0000 7f6dd1c1e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21df3a0 0x55d5d21e5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.904+0000 7f6dd1c1e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d21422c0 con 0x55d5d21df3a0 2024-03-21T11:34:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.904+0000 7f6dd1c1e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d20eac90 con 0x55d5d21de9e0 2024-03-21T11:34:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.904+0000 7f6dd1c1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d213e360 con 0x55d5d21dd810 2024-03-21T11:34:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.905+0000 7f6dcd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21dd810 0x55d5d21ddc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.905+0000 7f6dc7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.905+0000 7f6dcd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21dd810 0x55d5d21ddc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54740/0 (socket says 172.21.15.47:54740) 2024-03-21T11:34:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.905+0000 7f6dcd000640 1 -- 172.21.15.47:0/2942433096 learned_addr learned my addr 172.21.15.47:0/2942433096 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.905+0000 7f6dc4c00640 1 -- 172.21.15.47:0/1164937521 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6db4010ac0 con 0x7f6dc81028a0 2024-03-21T11:34:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.905+0000 7f6dc7e00640 1 -- 172.21.15.47:0/2942433096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21dd810 msgr2=0x55d5d21ddc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.905+0000 7f6dc7e00640 1 --2- 172.21.15.47:0/2942433096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21dd810 0x55d5d21ddc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.905+0000 7f6dc7e00640 1 -- 172.21.15.47:0/2942433096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21df3a0 msgr2=0x55d5d21e5c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:34:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.905+0000 7f6dc7e00640 1 --2- 172.21.15.47:0/2942433096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21df3a0 0x55d5d21e5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.905+0000 7f6dc7e00640 1 -- 172.21.15.47:0/2942433096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5d210f080 con 0x55d5d21de9e0 2024-03-21T11:34:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.905+0000 7f6dc7e00640 1 --2- 172.21.15.47:0/2942433096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21dee60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009230 tx=0x7f6db8009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbac2e69db897cee server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.906+0000 7f6dbd600640 1 -- 172.21.15.47:0/2942433096 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8045070 con 0x55d5d21de9e0 2024-03-21T11:34:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.906+0000 7f6dbd600640 1 -- 172.21.15.47:0/2942433096 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6db802cd30 con 0x55d5d21de9e0 2024-03-21T11:34:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.906+0000 7f6dbd600640 1 -- 172.21.15.47:0/2942433096 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db80050d0 con 0x55d5d21de9e0 2024-03-21T11:34:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.906+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/2942433096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 msgr2=0x55d5d21dee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.906+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/2942433096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21dee60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009230 tx=0x7f6db8009560 comp rx=0 tx=0).stop 2024-03-21T11:34:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.906+0000 7f6dc4c00640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dac0a2fa0 0x7f6dac0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.907+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/2942433096 shutdown_connections 2024-03-21T11:34:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.907+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/2942433096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21df3a0 0x55d5d21e5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.907+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/2942433096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21dee60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.907+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/2942433096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21dd810 0x55d5d21ddc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.907+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/2942433096 >> 172.21.15.47:0/2942433096 conn(0x55d5d21d8c00 msgr2=0x55d5d21d9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.907+0000 7f6dcd000640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dac0a2fa0 0x7f6dac0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.907+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/2942433096 shutdown_connections 2024-03-21T11:34:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.907+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/2942433096 wait complete. 2024-03-21T11:34:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.907+0000 7f6dc4c00640 1 -- 172.21.15.47:0/1164937521 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6db40cfa70 con 0x7f6dc81028a0 2024-03-21T11:34:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.907+0000 7f6dd1c1e740 1 -- start start 2024-03-21T11:34:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.907+0000 7f6dcd000640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dac0a2fa0 0x7f6dac0a5460 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005fd0 tx=0x7f6dc0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.908+0000 7f6dd1c1e740 1 Processor -- start 2024-03-21T11:34:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.908+0000 7f6dd1c1e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21dd810 0x55d5d21e7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.908+0000 7f6dd1c1e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21e7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.908+0000 7f6dd1c1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21df3a0 0x55d5d21ef010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.908+0000 7f6dd1c1e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d21e6350 con 0x55d5d21dd810 2024-03-21T11:34:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.908+0000 7f6dd1c1e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d20fe0d0 con 0x55d5d21de9e0 2024-03-21T11:34:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.908+0000 7f6dd1c1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d21e64d0 con 0x55d5d21df3a0 2024-03-21T11:34:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.908+0000 7f6dd1c1e740 10 client.0 _ll_register_callbacks cb 0x55d5d2102e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.908+0000 7f6dcd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21df3a0 0x55d5d21ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.908+0000 7f6dc7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21dd810 0x55d5d21e7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.908+0000 7f6dc7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21e7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.908+0000 7f6dcd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21df3a0 0x55d5d21ef010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54754/0 (socket says 172.21.15.47:54754) 2024-03-21T11:34:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.908+0000 7f6dcd000640 1 -- 172.21.15.47:0/1070270929 learned_addr learned my addr 172.21.15.47:0/1070270929 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.909+0000 7f6dcd000640 1 -- 172.21.15.47:0/1070270929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 msgr2=0x55d5d21e7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.909+0000 7f6dcd000640 1 --2- 172.21.15.47:0/1070270929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21e7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.909+0000 7f6dcd000640 1 -- 172.21.15.47:0/1070270929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21dd810 msgr2=0x55d5d21e7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.909+0000 7f6dcd000640 1 --2- 172.21.15.47:0/1070270929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21dd810 0x55d5d21e7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.909+0000 7f6dcd000640 1 -- 172.21.15.47:0/1070270929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5d1e563f0 con 0x55d5d21df3a0 2024-03-21T11:34:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.909+0000 7f6dcd000640 1 --2- 172.21.15.47:0/1070270929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21df3a0 0x55d5d21ef010 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6dc001d9b0 tx=0x7f6dc0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb301bb1da17785d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.909+0000 7f6db2000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.909+0000 7f6db2000640 1 -- 172.21.15.47:0/1070270929 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc002d060 con 0x55d5d21df3a0 2024-03-21T11:34:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.909+0000 7f6db2000640 1 -- 172.21.15.47:0/1070270929 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6dc0017030 con 0x55d5d21df3a0 2024-03-21T11:34:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.909+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/1070270929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6db802cd30 con 0x55d5d21df3a0 2024-03-21T11:34:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.910+0000 7f6db2000640 1 -- 172.21.15.47:0/1070270929 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dc0034050 con 0x55d5d21df3a0 2024-03-21T11:34:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.910+0000 7f6dd1c1e740 10 client.29089 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.910+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/1070270929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d5d1f207d0 con 0x55d5d21df3a0 2024-03-21T11:34:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.910+0000 7f6db2000640 1 -- 172.21.15.47:0/1070270929 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6db802cd30 con 0x55d5d21df3a0 2024-03-21T11:34:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.910+0000 7f6db2000640 1 -- 172.21.15.47:0/1070270929 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6dc0028780 con 0x55d5d21df3a0 2024-03-21T11:34:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.910+0000 7f6dd1c1e740 10 client.29089 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.910+0000 7f6dd1c1e740 10 client.29089 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:34:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.910+0000 7f6dd1c1e740 20 client.29089 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.910+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/1070270929 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55d5d21f3580 0x55d5d21f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.910+0000 7f6dd1c1e740 4 client.29089 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.910+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/1070270929 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5d214f800 con 0x55d5d21f3580 2024-03-21T11:34:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.911+0000 7f6dc7e00640 1 --2- 172.21.15.47:0/1070270929 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55d5d21f3580 0x55d5d21f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.911+0000 7f6dc7e00640 1 --2- 172.21.15.47:0/1070270929 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55d5d21f3580 0x55d5d21f59c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fdc75e79e0fd93b7 server_cookie=c8af769bcc6d39ca in_seq=0 out_seq=0 2024-03-21T11:34:01.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.911+0000 7f6db2000640 10 client.29089 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:34:01.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.912+0000 7f6db2000640 1 -- 172.21.15.47:0/1070270929 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d5d214f800 con 0x55d5d21f3580 2024-03-21T11:34:01.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.912+0000 7f6db2000640 10 client.29089 handle_command_reply: tid=0 2024-03-21T11:34:01.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.912+0000 7f6dd1c1e740 1 client.29089 shutdown 2024-03-21T11:34:01.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.912+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/1070270929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21df3a0 msgr2=0x55d5d21ef010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.912+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/1070270929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21df3a0 0x55d5d21ef010 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6dc001d9b0 tx=0x7f6dc0004ab0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.912+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/1070270929 shutdown_connections 2024-03-21T11:34:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.912+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/1070270929 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55d5d21f3580 0x55d5d21f59c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.912+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/1070270929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21df3a0 0x55d5d21ef010 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.912+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/1070270929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21e7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.912+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/1070270929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21dd810 0x55d5d21e7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.912+0000 7f6db2000640 0 client.29089 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:34:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.912+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/1070270929 >> 172.21.15.47:0/1070270929 conn(0x55d5d21d8c00 msgr2=0x55d5d21d7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.913+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/1070270929 shutdown_connections 2024-03-21T11:34:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.913+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/1070270929 wait complete. 2024-03-21T11:34:01.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.913+0000 7f6dd1c1e740 20 client.29089 trim_cache size 0 max 16384 2024-03-21T11:34:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.928+0000 7f6dd1c1e740 1 Processor -- start 2024-03-21T11:34:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.928+0000 7f6dd1c1e740 1 -- start start 2024-03-21T11:34:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dd1c1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21dd810 0x55d5d21dc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dd1c1e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21de9e0 0x55d5d21eefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dd1c1e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21df3a0 0x55d5d21e1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dd1c1e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc81ac290 con 0x55d5d21de9e0 2024-03-21T11:34:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dd1c1e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc81adc10 con 0x55d5d21df3a0 2024-03-21T11:34:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dd1c1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc81ac8f0 con 0x55d5d21dd810 2024-03-21T11:34:01.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dc7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21dd810 0x55d5d21dc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dc7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21de9e0 0x55d5d21eefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dcd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21df3a0 0x55d5d21e1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dc7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21dd810 0x55d5d21dc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54758/0 (socket says 172.21.15.47:54758) 2024-03-21T11:34:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dc7e00640 1 -- 172.21.15.47:0/1785340574 learned_addr learned my addr 172.21.15.47:0/1785340574 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dc7e00640 1 -- 172.21.15.47:0/1785340574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21df3a0 msgr2=0x55d5d21e1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dc7e00640 1 --2- 172.21.15.47:0/1785340574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21df3a0 0x55d5d21e1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dc7e00640 1 -- 172.21.15.47:0/1785340574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21de9e0 msgr2=0x55d5d21eefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dc7e00640 1 --2- 172.21.15.47:0/1785340574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21de9e0 0x55d5d21eefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dc7e00640 1 -- 172.21.15.47:0/1785340574 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5d21de460 con 0x55d5d21dd810 2024-03-21T11:34:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dc7400640 1 --2- 172.21.15.47:0/1785340574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21de9e0 0x55d5d21eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:01.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.929+0000 7f6dc7e00640 1 --2- 172.21.15.47:0/1785340574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21dd810 0x55d5d21dc570 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x55d5d21d9b70 tx=0x7f6db8041040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aac89a5f84fe3a7a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:01.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.930+0000 7f6dbcc00640 1 -- 172.21.15.47:0/1785340574 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8052070 con 0x55d5d21dd810 2024-03-21T11:34:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.930+0000 7f6dbcc00640 1 -- 172.21.15.47:0/1785340574 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6db802cd30 con 0x55d5d21dd810 2024-03-21T11:34:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.930+0000 7f6dbcc00640 1 -- 172.21.15.47:0/1785340574 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8045040 con 0x55d5d21dd810 2024-03-21T11:34:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.931+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/1785340574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21dd810 msgr2=0x55d5d21dc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.931+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/1785340574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21dd810 0x55d5d21dc570 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x55d5d21d9b70 tx=0x7f6db8041040 comp rx=0 tx=0).stop 2024-03-21T11:34:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.931+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/1785340574 shutdown_connections 2024-03-21T11:34:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.931+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/1785340574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21df3a0 0x55d5d21e1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.931+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/1785340574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21de9e0 0x55d5d21eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.931+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/1785340574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21dd810 0x55d5d21dc570 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.931+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/1785340574 >> 172.21.15.47:0/1785340574 conn(0x55d5d21d8c00 msgr2=0x55d5d21dbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.931+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/1785340574 shutdown_connections 2024-03-21T11:34:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.931+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/1785340574 wait complete. 2024-03-21T11:34:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.931+0000 7f6dd1c1e740 1 -- start start 2024-03-21T11:34:01.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.931+0000 7f6dd1c1e740 1 Processor -- start 2024-03-21T11:34:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dd1c1e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21eafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dd1c1e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21df3a0 0x55d5d21eb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dd1c1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21ee540 0x55d5d21ee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dd1c1e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc81ad5b0 con 0x55d5d21df3a0 2024-03-21T11:34:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dd1c1e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc81acf50 con 0x55d5d21de9e0 2024-03-21T11:34:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dd1c1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc81aa450 con 0x55d5d21ee540 2024-03-21T11:34:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dd1c1e740 10 client.0 _ll_register_callbacks cb 0x7f6dc0003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dc7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21eafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dc7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21eafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36034/0 (socket says 172.21.15.47:36034) 2024-03-21T11:34:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dc7e00640 1 -- 172.21.15.47:0/764285799 learned_addr learned my addr 172.21.15.47:0/764285799 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dc7400640 1 --2- 172.21.15.47:0/764285799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21df3a0 0x55d5d21eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dcd000640 1 --2- 172.21.15.47:0/764285799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21ee540 0x55d5d21ee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dc7e00640 1 -- 172.21.15.47:0/764285799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21ee540 msgr2=0x55d5d21ee9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dc7e00640 1 --2- 172.21.15.47:0/764285799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21ee540 0x55d5d21ee9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dc7e00640 1 -- 172.21.15.47:0/764285799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21df3a0 msgr2=0x55d5d21eb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dc7e00640 1 --2- 172.21.15.47:0/764285799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21df3a0 0x55d5d21eb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.932+0000 7f6dc7e00640 1 -- 172.21.15.47:0/764285799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5d21eef00 con 0x55d5d21de9e0 2024-03-21T11:34:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.933+0000 7f6dc7e00640 1 --2- 172.21.15.47:0/764285799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21eafe0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6db802c7b0 tx=0x7f6db80056e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40b1c7782c7759a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.933+0000 7f6db2000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.933+0000 7f6db2000640 1 -- 172.21.15.47:0/764285799 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8045030 con 0x55d5d21de9e0 2024-03-21T11:34:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.933+0000 7f6db2000640 1 -- 172.21.15.47:0/764285799 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6db8004040 con 0x55d5d21de9e0 2024-03-21T11:34:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.933+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/764285799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6db802cd30 con 0x55d5d21de9e0 2024-03-21T11:34:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.933+0000 7f6db2000640 1 -- 172.21.15.47:0/764285799 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6db8004240 con 0x55d5d21de9e0 2024-03-21T11:34:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.933+0000 7f6dd1c1e740 10 client.29075 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.933+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/764285799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d5d224fa50 con 0x55d5d21de9e0 2024-03-21T11:34:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.933+0000 7f6db2000640 1 -- 172.21.15.47:0/764285799 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6db802cd30 con 0x55d5d21de9e0 2024-03-21T11:34:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.934+0000 7f6db2000640 1 -- 172.21.15.47:0/764285799 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6db8003860 con 0x55d5d21de9e0 2024-03-21T11:34:01.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.934+0000 7f6dd1c1e740 10 client.29075 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.934+0000 7f6dd1c1e740 10 client.29075 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:34:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.934+0000 7f6dd1c1e740 20 client.29075 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.934+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/764285799 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55d5d224fef0 0x55d5d22522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.934+0000 7f6dd1c1e740 4 client.29075 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:34:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.934+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/764285799 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d5d2149b50 con 0x55d5d224fef0 2024-03-21T11:34:01.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.934+0000 7f6dc7400640 1 --2- 172.21.15.47:0/764285799 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55d5d224fef0 0x55d5d22522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6dc7400640 1 --2- 172.21.15.47:0/764285799 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55d5d224fef0 0x55d5d22522e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=48cc7c7e2ed1c7b1 server_cookie=8a602e66b78e369c in_seq=0 out_seq=0 2024-03-21T11:34:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6db2000640 10 client.29075 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:34:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6db2000640 1 -- 172.21.15.47:0/764285799 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d5d2149b50 con 0x55d5d224fef0 2024-03-21T11:34:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6db2000640 10 client.29075 handle_command_reply: tid=0 2024-03-21T11:34:01.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6dd1c1e740 1 client.29075 shutdown 2024-03-21T11:34:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/764285799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 msgr2=0x55d5d21eafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/764285799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21eafe0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6db802c7b0 tx=0x7f6db80056e0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/764285799 shutdown_connections 2024-03-21T11:34:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/764285799 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55d5d224fef0 0x55d5d22522e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/764285799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d5d21ee540 0x55d5d21ee9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/764285799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d5d21df3a0 0x55d5d21eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6db2000640 0 client.29075 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:34:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6dd1c1e740 1 --2- 172.21.15.47:0/764285799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5d21de9e0 0x55d5d21eafe0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/764285799 >> 172.21.15.47:0/764285799 conn(0x55d5d21d8c00 msgr2=0x55d5d21dc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/764285799 shutdown_connections 2024-03-21T11:34:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6dd1c1e740 1 -- 172.21.15.47:0/764285799 wait complete. 2024-03-21T11:34:01.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.935+0000 7f6dd1c1e740 20 client.29075 trim_cache size 0 max 16384 2024-03-21T11:34:01.940 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:34:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.936+0000 7f6dcf000640 1 -- 172.21.15.47:0/1164937521 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dac0a2fa0 msgr2=0x7f6dac0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.936+0000 7f6dcf000640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dac0a2fa0 0x7f6dac0a5460 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005fd0 tx=0x7f6dc0009500 comp rx=0 tx=0).stop 2024-03-21T11:34:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.936+0000 7f6dcf000640 1 -- 172.21.15.47:0/1164937521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81028a0 msgr2=0x7f6dc819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:01.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.936+0000 7f6dcf000640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81028a0 0x7f6dc819a1a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6db400e9f0 tx=0x7f6db400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.936+0000 7f6dcf000640 1 -- 172.21.15.47:0/1164937521 shutdown_connections 2024-03-21T11:34:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.936+0000 7f6dcf000640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dac0a2fa0 0x7f6dac0a5460 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.936+0000 7f6dcf000640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6dc8103260 0x7f6dc81a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.936+0000 7f6dcf000640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dc81028a0 0x7f6dc819a1a0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.936+0000 7f6dcf000640 1 --2- 172.21.15.47:0/1164937521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6dc81016b0 0x7f6dc8199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.936+0000 7f6dcf000640 1 -- 172.21.15.47:0/1164937521 >> 172.21.15.47:0/1164937521 conn(0x7f6dc80fcc40 msgr2=0x7f6dc80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.936+0000 7f6dcf000640 1 -- 172.21.15.47:0/1164937521 shutdown_connections 2024-03-21T11:34:01.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:01.936+0000 7f6dcf000640 1 -- 172.21.15.47:0/1164937521 wait complete. 2024-03-21T11:34:01.954 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:34:01.954 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:34:02.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.177+0000 7f18f2c00640 1 -- 172.21.15.47:0/3225289426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec1028a0 msgr2=0x7f18ec102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.177+0000 7f18f2c00640 1 --2- 172.21.15.47:0/3225289426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec1028a0 0x7f18ec102d20 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009900 tx=0x7f18d402bc40 comp rx=0 tx=0).stop 2024-03-21T11:34:02.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.177+0000 7f18f2c00640 1 -- 172.21.15.47:0/3225289426 shutdown_connections 2024-03-21T11:34:02.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.177+0000 7f18f2c00640 1 --2- 172.21.15.47:0/3225289426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ec103260 0x7f18ec109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.177+0000 7f18f2c00640 1 --2- 172.21.15.47:0/3225289426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec1028a0 0x7f18ec102d20 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.177+0000 7f18f2c00640 1 --2- 172.21.15.47:0/3225289426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18ec1016b0 0x7f18ec101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.177+0000 7f18f2c00640 1 -- 172.21.15.47:0/3225289426 >> 172.21.15.47:0/3225289426 conn(0x7f18ec0fcc40 msgr2=0x7f18ec0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.177+0000 7f18f2c00640 1 -- 172.21.15.47:0/3225289426 shutdown_connections 2024-03-21T11:34:02.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.178+0000 7f18f2c00640 1 -- 172.21.15.47:0/3225289426 wait complete. 2024-03-21T11:34:02.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.178+0000 7f18f2c00640 1 Processor -- start 2024-03-21T11:34:02.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.179+0000 7f18f2c00640 1 -- start start 2024-03-21T11:34:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.179+0000 7f18f2c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec1016b0 0x7f18ec199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.179+0000 7f18f2c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ec1028a0 0x7f18ec19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.179+0000 7f18f2c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18ec103260 0x7f18ec1a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.179+0000 7f18f2c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec10d380 con 0x7f18ec103260 2024-03-21T11:34:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.179+0000 7f18f2c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec10d200 con 0x7f18ec1028a0 2024-03-21T11:34:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.179+0000 7f18f2c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec10d500 con 0x7f18ec1016b0 2024-03-21T11:34:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.179+0000 7f18ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec1016b0 0x7f18ec199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.179+0000 7f18ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec1016b0 0x7f18ec199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54780/0 (socket says 172.21.15.47:54780) 2024-03-21T11:34:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.179+0000 7f18ebe00640 1 -- 172.21.15.47:0/3373447593 learned_addr learned my addr 172.21.15.47:0/3373447593 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.179+0000 7f18eb400640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ec1028a0 0x7f18ec19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.179+0000 7f18f0c00640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18ec103260 0x7f18ec1a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.180+0000 7f18eb400640 1 -- 172.21.15.47:0/3373447593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec1016b0 msgr2=0x7f18ec199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.180+0000 7f18eb400640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec1016b0 0x7f18ec199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.180+0000 7f18eb400640 1 -- 172.21.15.47:0/3373447593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18ec103260 msgr2=0x7f18ec1a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.180+0000 7f18eb400640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18ec103260 0x7f18ec1a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.180+0000 7f18eb400640 1 -- 172.21.15.47:0/3373447593 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec1a17f0 con 0x7f18ec1028a0 2024-03-21T11:34:02.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.180+0000 7f18ebe00640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec1016b0 0x7f18ec199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:02.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.180+0000 7f18eb400640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ec1028a0 0x7f18ec19a110 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f18d4036040 tx=0x7f18d402c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.180+0000 7f18e8c00640 1 -- 172.21.15.47:0/3373447593 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4041070 con 0x7f18ec1028a0 2024-03-21T11:34:02.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.180+0000 7f18e8c00640 1 -- 172.21.15.47:0/3373447593 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18d4004450 con 0x7f18ec1028a0 2024-03-21T11:34:02.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.180+0000 7f18f2c00640 1 -- 172.21.15.47:0/3373447593 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ec1a1a80 con 0x7f18ec1028a0 2024-03-21T11:34:02.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.180+0000 7f18e8c00640 1 -- 172.21.15.47:0/3373447593 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4003e60 con 0x7f18ec1028a0 2024-03-21T11:34:02.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.181+0000 7f18f2c00640 1 -- 172.21.15.47:0/3373447593 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec1a1f40 con 0x7f18ec1028a0 2024-03-21T11:34:02.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.184+0000 7f18f59a2740 1 Processor -- start 2024-03-21T11:34:02.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.184+0000 7f18f59a2740 1 -- start start 2024-03-21T11:34:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.184+0000 7f18f59a2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf331c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.184+0000 7f18f59a2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf332e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.184+0000 7f18f59a2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3333a0 0x5573cf339c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.184+0000 7f18f59a2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cf2962c0 con 0x5573cf3329e0 2024-03-21T11:34:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.184+0000 7f18f59a2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cf23ec90 con 0x5573cf331810 2024-03-21T11:34:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.184+0000 7f18f59a2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cf292360 con 0x5573cf3333a0 2024-03-21T11:34:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.184+0000 7f18f0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf331c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.184+0000 7f18f0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf331c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36074/0 (socket says 172.21.15.47:36074) 2024-03-21T11:34:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.184+0000 7f18f0c00640 1 -- 172.21.15.47:0/2984075343 learned_addr learned my addr 172.21.15.47:0/2984075343 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.184+0000 7f18ebe00640 1 --2- 172.21.15.47:0/2984075343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf332e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.184+0000 7f18e8c00640 1 -- 172.21.15.47:0/3373447593 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f18d4044070 con 0x7f18ec1028a0 2024-03-21T11:34:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.185+0000 7f18f0c00640 1 -- 172.21.15.47:0/2984075343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3333a0 msgr2=0x5573cf339c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:34:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.185+0000 7f18f0c00640 1 --2- 172.21.15.47:0/2984075343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3333a0 0x5573cf339c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.185+0000 7f18f0c00640 1 -- 172.21.15.47:0/2984075343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 msgr2=0x5573cf332e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.185+0000 7f18f0c00640 1 --2- 172.21.15.47:0/2984075343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf332e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.185+0000 7f18f0c00640 1 -- 172.21.15.47:0/2984075343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573cf263080 con 0x5573cf331810 2024-03-21T11:34:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.185+0000 7f18f0c00640 1 --2- 172.21.15.47:0/2984075343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf331c30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009ca0 tx=0x7f18dc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d166bffe1bf5e0b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.185+0000 7f18e1200640 1 -- 172.21.15.47:0/2984075343 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc01e070 con 0x5573cf331810 2024-03-21T11:34:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.185+0000 7f18e1200640 1 -- 172.21.15.47:0/2984075343 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18dc019910 con 0x5573cf331810 2024-03-21T11:34:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.185+0000 7f18e1200640 1 -- 172.21.15.47:0/2984075343 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00ea50 con 0x5573cf331810 2024-03-21T11:34:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.186+0000 7f18f59a2740 1 -- 172.21.15.47:0/2984075343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 msgr2=0x5573cf331c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.186+0000 7f18f59a2740 1 --2- 172.21.15.47:0/2984075343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf331c30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009ca0 tx=0x7f18dc011810 comp rx=0 tx=0).stop 2024-03-21T11:34:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.186+0000 7f18f59a2740 1 -- 172.21.15.47:0/2984075343 shutdown_connections 2024-03-21T11:34:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.186+0000 7f18f59a2740 1 --2- 172.21.15.47:0/2984075343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3333a0 0x5573cf339c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.186+0000 7f18f59a2740 1 --2- 172.21.15.47:0/2984075343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf332e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.186+0000 7f18f59a2740 1 --2- 172.21.15.47:0/2984075343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf331c30 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.186+0000 7f18e8c00640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18d00a2f20 0x7f18d00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.186+0000 7f18f59a2740 1 -- 172.21.15.47:0/2984075343 >> 172.21.15.47:0/2984075343 conn(0x5573cf32cc00 msgr2=0x5573cf32d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.186+0000 7f18e8c00640 1 -- 172.21.15.47:0/3373447593 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f18d40f1280 con 0x7f18ec1028a0 2024-03-21T11:34:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.186+0000 7f18f59a2740 1 -- 172.21.15.47:0/2984075343 shutdown_connections 2024-03-21T11:34:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.186+0000 7f18f59a2740 1 -- 172.21.15.47:0/2984075343 wait complete. 2024-03-21T11:34:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.186+0000 7f18f0c00640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18d00a2f20 0x7f18d00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.187+0000 7f18f59a2740 1 -- start start 2024-03-21T11:34:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.187+0000 7f18f0c00640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18d00a2f20 0x7f18d00a53e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00e4f0 tx=0x7f18dc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.187+0000 7f18f59a2740 1 Processor -- start 2024-03-21T11:34:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18f59a2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf33bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18f59a2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf33c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18f59a2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3333a0 0x5573cf3430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18f59a2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cf33a3b0 con 0x5573cf3329e0 2024-03-21T11:34:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18f59a2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cf2520d0 con 0x5573cf331810 2024-03-21T11:34:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18f59a2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573cf33a530 con 0x5573cf3333a0 2024-03-21T11:34:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18f59a2740 10 client.0 _ll_register_callbacks cb 0x5573cf256e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18eb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf33c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18f0c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3333a0 0x5573cf3430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18ebe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf33bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18f0c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3333a0 0x5573cf3430e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54800/0 (socket says 172.21.15.47:54800) 2024-03-21T11:34:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18ebe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf33bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36076/0 (socket says 172.21.15.47:36076) 2024-03-21T11:34:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18f0c00640 1 -- 172.21.15.47:0/1053466681 learned_addr learned my addr 172.21.15.47:0/1053466681 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18eb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf33c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55322/0 (socket says 172.21.15.47:55322) 2024-03-21T11:34:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18eb400640 1 -- 172.21.15.47:0/1053466681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3333a0 msgr2=0x5573cf3430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18eb400640 1 --2- 172.21.15.47:0/1053466681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3333a0 0x5573cf3430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18eb400640 1 -- 172.21.15.47:0/1053466681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 msgr2=0x5573cf33bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18eb400640 1 --2- 172.21.15.47:0/1053466681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf33bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18eb400640 1 -- 172.21.15.47:0/1053466681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573cefaa3f0 con 0x5573cf3329e0 2024-03-21T11:34:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.188+0000 7f18f0c00640 1 --2- 172.21.15.47:0/1053466681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3333a0 0x5573cf3430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.189+0000 7f18eb400640 1 --2- 172.21.15.47:0/1053466681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf33c0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f18d40ee990 tx=0x7f18d40bd500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418044ed925abef0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.189+0000 7f18d9600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:34:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.189+0000 7f18d9600640 1 -- 172.21.15.47:0/1053466681 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d404b040 con 0x5573cf3329e0 2024-03-21T11:34:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.189+0000 7f18d9600640 1 -- 172.21.15.47:0/1053466681 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f18d40c5b40 con 0x5573cf3329e0 2024-03-21T11:34:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.189+0000 7f18f59a2740 1 -- 172.21.15.47:0/1053466681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18dc019910 con 0x5573cf3329e0 2024-03-21T11:34:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.189+0000 7f18d9600640 1 -- 172.21.15.47:0/1053466681 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18d40c1710 con 0x5573cf3329e0 2024-03-21T11:34:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.189+0000 7f18f59a2740 10 client.18747 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.189+0000 7f18f59a2740 1 -- 172.21.15.47:0/1053466681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5573cf0747d0 con 0x5573cf3329e0 2024-03-21T11:34:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.189+0000 7f18d9600640 1 -- 172.21.15.47:0/1053466681 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18dc019910 con 0x5573cf3329e0 2024-03-21T11:34:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.189+0000 7f18d9600640 1 -- 172.21.15.47:0/1053466681 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f18d40bcac0 con 0x5573cf3329e0 2024-03-21T11:34:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.190+0000 7f18f59a2740 10 client.18747 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.190+0000 7f18f59a2740 10 client.18747 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:34:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.190+0000 7f18f59a2740 20 client.18747 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.190+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1053466681 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5573cf347510 0x5573cf349900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.190+0000 7f18f59a2740 4 client.18747 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.190+0000 7f18f59a2740 1 -- 172.21.15.47:0/1053466681 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573cf2a3800 con 0x5573cf347510 2024-03-21T11:34:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.190+0000 7f18ebe00640 1 --2- 172.21.15.47:0/1053466681 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5573cf347510 0x5573cf349900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.190+0000 7f18ebe00640 1 --2- 172.21.15.47:0/1053466681 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5573cf347510 0x5573cf349900 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa0a249e57ae2b47 server_cookie=dac3c7a96bc7a96 in_seq=0 out_seq=0 2024-03-21T11:34:02.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.190+0000 7f18d9600640 10 client.18747 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:34:02.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.191+0000 7f18d9600640 1 -- 172.21.15.47:0/1053466681 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5573cf2a3800 con 0x5573cf347510 2024-03-21T11:34:02.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.191+0000 7f18d9600640 10 client.18747 handle_command_reply: tid=0 2024-03-21T11:34:02.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.191+0000 7f18f59a2740 1 client.18747 shutdown 2024-03-21T11:34:02.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.191+0000 7f18f59a2740 1 -- 172.21.15.47:0/1053466681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 msgr2=0x5573cf33c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.191+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1053466681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf33c0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f18d40ee990 tx=0x7f18d40bd500 comp rx=0 tx=0).stop 2024-03-21T11:34:02.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.192+0000 7f18f59a2740 1 -- 172.21.15.47:0/1053466681 shutdown_connections 2024-03-21T11:34:02.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.192+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1053466681 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5573cf347510 0x5573cf349900 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.192+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1053466681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3333a0 0x5573cf3430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.192+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1053466681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf33c0a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.192+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1053466681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf33bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.192+0000 7f18f59a2740 1 -- 172.21.15.47:0/1053466681 >> 172.21.15.47:0/1053466681 conn(0x5573cf32cc00 msgr2=0x5573cf32beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.192+0000 7f18d9600640 0 client.18747 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:34:02.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.192+0000 7f18f59a2740 1 -- 172.21.15.47:0/1053466681 shutdown_connections 2024-03-21T11:34:02.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.192+0000 7f18f59a2740 1 -- 172.21.15.47:0/1053466681 wait complete. 2024-03-21T11:34:02.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.192+0000 7f18f59a2740 20 client.18747 trim_cache size 0 max 16384 2024-03-21T11:34:02.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.207+0000 7f18f59a2740 1 Processor -- start 2024-03-21T11:34:02.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.207+0000 7f18f59a2740 1 -- start start 2024-03-21T11:34:02.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.207+0000 7f18f59a2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf330570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.207+0000 7f18f59a2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3329e0 0x5573cf3430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.207+0000 7f18f59a2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3333a0 0x5573cf34e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.207+0000 7f18f59a2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec1ac210 con 0x5573cf3333a0 2024-03-21T11:34:02.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.207+0000 7f18f59a2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec1adb90 con 0x5573cf331810 2024-03-21T11:34:02.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.207+0000 7f18f59a2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec1ac870 con 0x5573cf3329e0 2024-03-21T11:34:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.207+0000 7f18eb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3329e0 0x5573cf3430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.207+0000 7f18ebe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf330570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.208+0000 7f18eb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3329e0 0x5573cf3430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54802/0 (socket says 172.21.15.47:54802) 2024-03-21T11:34:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.208+0000 7f18f0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3333a0 0x5573cf34e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.208+0000 7f18eb400640 1 -- 172.21.15.47:0/1303691679 learned_addr learned my addr 172.21.15.47:0/1303691679 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.208+0000 7f18eb400640 1 -- 172.21.15.47:0/1303691679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 msgr2=0x5573cf330570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.208+0000 7f18eb400640 1 --2- 172.21.15.47:0/1303691679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf330570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.208+0000 7f18eb400640 1 -- 172.21.15.47:0/1303691679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3333a0 msgr2=0x5573cf34e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.208+0000 7f18eb400640 1 --2- 172.21.15.47:0/1303691679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3333a0 0x5573cf34e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.208+0000 7f18eb400640 1 -- 172.21.15.47:0/1303691679 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573cf32d670 con 0x5573cf3329e0 2024-03-21T11:34:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.208+0000 7f18eb400640 1 --2- 172.21.15.47:0/1303691679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3329e0 0x5573cf3430a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f18d40b9ad0 tx=0x7f18d40bd570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4de5f2232f94041 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.208+0000 7f18dbe00640 1 -- 172.21.15.47:0/1303691679 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d40da070 con 0x5573cf3329e0 2024-03-21T11:34:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.208+0000 7f18dbe00640 1 -- 172.21.15.47:0/1303691679 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18d40eee50 con 0x5573cf3329e0 2024-03-21T11:34:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.209+0000 7f18dbe00640 1 -- 172.21.15.47:0/1303691679 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d404b070 con 0x5573cf3329e0 2024-03-21T11:34:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.209+0000 7f18f59a2740 1 -- 172.21.15.47:0/1303691679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3329e0 msgr2=0x5573cf3430a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.209+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1303691679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3329e0 0x5573cf3430a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f18d40b9ad0 tx=0x7f18d40bd570 comp rx=0 tx=0).stop 2024-03-21T11:34:02.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.209+0000 7f18f59a2740 1 -- 172.21.15.47:0/1303691679 shutdown_connections 2024-03-21T11:34:02.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.209+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1303691679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3333a0 0x5573cf34e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.209+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1303691679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf3329e0 0x5573cf3430a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.209+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1303691679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf331810 0x5573cf330570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.209+0000 7f18f59a2740 1 -- 172.21.15.47:0/1303691679 >> 172.21.15.47:0/1303691679 conn(0x5573cf32cc00 msgr2=0x5573cf32fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.209+0000 7f18f59a2740 1 -- 172.21.15.47:0/1303691679 shutdown_connections 2024-03-21T11:34:02.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.209+0000 7f18f59a2740 1 -- 172.21.15.47:0/1303691679 wait complete. 2024-03-21T11:34:02.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.210+0000 7f18f59a2740 1 -- start start 2024-03-21T11:34:02.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.210+0000 7f18f59a2740 1 Processor -- start 2024-03-21T11:34:02.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.210+0000 7f18f59a2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf331810 0x5573cf33e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.210+0000 7f18f59a2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf33eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.210+0000 7f18f59a2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf3333a0 0x5573cf341ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.210+0000 7f18f59a2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec1ad530 con 0x5573cf3329e0 2024-03-21T11:34:02.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.210+0000 7f18f59a2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec1aced0 con 0x5573cf3333a0 2024-03-21T11:34:02.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.210+0000 7f18f59a2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec1aa3d0 con 0x5573cf331810 2024-03-21T11:34:02.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.211+0000 7f18f59a2740 10 client.0 _ll_register_callbacks cb 0x7f18d40b91f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:02.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.211+0000 7f18ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf331810 0x5573cf33e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.211+0000 7f18ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf331810 0x5573cf33e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54804/0 (socket says 172.21.15.47:54804) 2024-03-21T11:34:02.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.211+0000 7f18ebe00640 1 -- 172.21.15.47:0/1904738262 learned_addr learned my addr 172.21.15.47:0/1904738262 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.211+0000 7f18f0c00640 1 --2- 172.21.15.47:0/1904738262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf3333a0 0x5573cf341ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.211+0000 7f18eb400640 1 --2- 172.21.15.47:0/1904738262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf33eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.211+0000 7f18ebe00640 1 -- 172.21.15.47:0/1904738262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf3333a0 msgr2=0x5573cf341ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.211+0000 7f18ebe00640 1 --2- 172.21.15.47:0/1904738262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf3333a0 0x5573cf341ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.211+0000 7f18ebe00640 1 -- 172.21.15.47:0/1904738262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 msgr2=0x5573cf33eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.211+0000 7f18ebe00640 1 --2- 172.21.15.47:0/1904738262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf33eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.211+0000 7f18ebe00640 1 -- 172.21.15.47:0/1904738262 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573cf3424c0 con 0x5573cf331810 2024-03-21T11:34:02.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.211+0000 7f18eb400640 1 --2- 172.21.15.47:0/1904738262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf33eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:02.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.211+0000 7f18ebe00640 1 --2- 172.21.15.47:0/1904738262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf331810 0x5573cf33e970 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x5573cf34a120 tx=0x7f18e400fd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4e7d6a3cddd72b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.211+0000 7f18d9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:02.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.212+0000 7f18d9600640 1 -- 172.21.15.47:0/1904738262 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4020970 con 0x5573cf331810 2024-03-21T11:34:02.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.212+0000 7f18f59a2740 1 -- 172.21.15.47:0/1904738262 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18d40eee50 con 0x5573cf331810 2024-03-21T11:34:02.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.212+0000 7f18d9600640 1 -- 172.21.15.47:0/1904738262 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f18e4020b10 con 0x5573cf331810 2024-03-21T11:34:02.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.212+0000 7f18d9600640 1 -- 172.21.15.47:0/1904738262 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18e4031050 con 0x5573cf331810 2024-03-21T11:34:02.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.212+0000 7f18f59a2740 10 client.29110 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:02.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.212+0000 7f18f59a2740 1 -- 172.21.15.47:0/1904738262 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5573cf3a2fe0 con 0x5573cf331810 2024-03-21T11:34:02.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.212+0000 7f18d9600640 1 -- 172.21.15.47:0/1904738262 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18d40eee50 con 0x5573cf331810 2024-03-21T11:34:02.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.213+0000 7f18d9600640 1 -- 172.21.15.47:0/1904738262 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f18e40104c0 con 0x5573cf331810 2024-03-21T11:34:02.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.213+0000 7f18f59a2740 10 client.29110 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:02.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.213+0000 7f18f59a2740 10 client.29110 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:34:02.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.213+0000 7f18f59a2740 20 client.29110 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:02.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.213+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1904738262 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5573cf3a38f0 0x5573cf3a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.213+0000 7f18f59a2740 4 client.29110 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:34:02.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.213+0000 7f18f59a2740 1 -- 172.21.15.47:0/1904738262 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5573cf33b950 con 0x5573cf3a38f0 2024-03-21T11:34:02.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.213+0000 7f18eb400640 1 --2- 172.21.15.47:0/1904738262 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5573cf3a38f0 0x5573cf3a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.213+0000 7f18eb400640 1 --2- 172.21.15.47:0/1904738262 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5573cf3a38f0 0x5573cf3a5d90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20f2331d762c492 server_cookie=b5e3b44f19195399 in_seq=0 out_seq=0 2024-03-21T11:34:02.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.213+0000 7f18d9600640 10 client.29110 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:34:02.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18d9600640 1 -- 172.21.15.47:0/1904738262 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5573cf33b950 con 0x5573cf3a38f0 2024-03-21T11:34:02.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18d9600640 10 client.29110 handle_command_reply: tid=0 2024-03-21T11:34:02.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18f59a2740 1 client.29110 shutdown 2024-03-21T11:34:02.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18f59a2740 1 -- 172.21.15.47:0/1904738262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf331810 msgr2=0x5573cf33e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1904738262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf331810 0x5573cf33e970 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x5573cf34a120 tx=0x7f18e400fd20 comp rx=0 tx=0).stop 2024-03-21T11:34:02.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18f59a2740 1 -- 172.21.15.47:0/1904738262 shutdown_connections 2024-03-21T11:34:02.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1904738262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573cf3333a0 0x5573cf341ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1904738262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5573cf3329e0 0x5573cf33eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1904738262 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5573cf3a38f0 0x5573cf3a5d90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18f59a2740 1 --2- 172.21.15.47:0/1904738262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5573cf331810 0x5573cf33e970 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18f59a2740 1 -- 172.21.15.47:0/1904738262 >> 172.21.15.47:0/1904738262 conn(0x5573cf32cc00 msgr2=0x5573cf3306f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18d9600640 0 client.29110 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:34:02.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18f59a2740 1 -- 172.21.15.47:0/1904738262 shutdown_connections 2024-03-21T11:34:02.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18f59a2740 1 -- 172.21.15.47:0/1904738262 wait complete. 2024-03-21T11:34:02.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.214+0000 7f18f59a2740 20 client.29110 trim_cache size 0 max 16384 2024-03-21T11:34:02.223 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:34:02.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.215+0000 7f18f2c00640 1 -- 172.21.15.47:0/3373447593 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18d00a2f20 msgr2=0x7f18d00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.215+0000 7f18f2c00640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18d00a2f20 0x7f18d00a53e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00e4f0 tx=0x7f18dc005850 comp rx=0 tx=0).stop 2024-03-21T11:34:02.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.215+0000 7f18f2c00640 1 -- 172.21.15.47:0/3373447593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ec1028a0 msgr2=0x7f18ec19a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.215+0000 7f18f2c00640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ec1028a0 0x7f18ec19a110 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f18d4036040 tx=0x7f18d402c660 comp rx=0 tx=0).stop 2024-03-21T11:34:02.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.215+0000 7f18f2c00640 1 -- 172.21.15.47:0/3373447593 shutdown_connections 2024-03-21T11:34:02.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.215+0000 7f18f2c00640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18d00a2f20 0x7f18d00a53e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.215+0000 7f18f2c00640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18ec103260 0x7f18ec1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.215+0000 7f18f2c00640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ec1028a0 0x7f18ec19a110 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.215+0000 7f18f2c00640 1 --2- 172.21.15.47:0/3373447593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ec1016b0 0x7f18ec199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.215+0000 7f18f2c00640 1 -- 172.21.15.47:0/3373447593 >> 172.21.15.47:0/3373447593 conn(0x7f18ec0fcc40 msgr2=0x7f18ec0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.215+0000 7f18f2c00640 1 -- 172.21.15.47:0/3373447593 shutdown_connections 2024-03-21T11:34:02.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.215+0000 7f18f2c00640 1 -- 172.21.15.47:0/3373447593 wait complete. 2024-03-21T11:34:02.231 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:34:02.232 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:34:02.299 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:02 smithi047 ceph-mon[31351]: pgmap v721: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-03-21T11:34:02.300 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:02 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3674766326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:34:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.439+0000 7f325ea00640 1 -- 172.21.15.47:0/4069835854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32581038e0 msgr2=0x7f3258103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.439+0000 7f325ea00640 1 --2- 172.21.15.47:0/4069835854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32581038e0 0x7f3258103d00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3248009900 tx=0x7f324802bc30 comp rx=0 tx=0).stop 2024-03-21T11:34:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.439+0000 7f325ea00640 1 -- 172.21.15.47:0/4069835854 shutdown_connections 2024-03-21T11:34:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.439+0000 7f325ea00640 1 --2- 172.21.15.47:0/4069835854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3258105490 0x7f325810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.439+0000 7f325ea00640 1 --2- 172.21.15.47:0/4069835854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3258104ad0 0x7f3258104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.439+0000 7f325ea00640 1 --2- 172.21.15.47:0/4069835854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32581038e0 0x7f3258103d00 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.439+0000 7f325ea00640 1 -- 172.21.15.47:0/4069835854 >> 172.21.15.47:0/4069835854 conn(0x7f32580fee50 msgr2=0x7f3258101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.439+0000 7f325ea00640 1 -- 172.21.15.47:0/4069835854 shutdown_connections 2024-03-21T11:34:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.439+0000 7f325ea00640 1 -- 172.21.15.47:0/4069835854 wait complete. 2024-03-21T11:34:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.440+0000 7f325ea00640 1 Processor -- start 2024-03-21T11:34:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.441+0000 7f325ea00640 1 -- start start 2024-03-21T11:34:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.441+0000 7f325ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32581038e0 0x7f3258199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.441+0000 7f325ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3258104ad0 0x7f325819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.441+0000 7f325ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3258105490 0x7f32581a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.441+0000 7f325ea00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325810f5b0 con 0x7f32581038e0 2024-03-21T11:34:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.441+0000 7f325ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325810f430 con 0x7f3258104ad0 2024-03-21T11:34:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.441+0000 7f325ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325810f730 con 0x7f3258105490 2024-03-21T11:34:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.441+0000 7f3257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3258104ad0 0x7f325819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.441+0000 7f3257e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32581038e0 0x7f3258199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.441+0000 7f325ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3258105490 0x7f32581a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.441+0000 7f325ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3258105490 0x7f32581a11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54816/0 (socket says 172.21.15.47:54816) 2024-03-21T11:34:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.441+0000 7f3257e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32581038e0 0x7f3258199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55368/0 (socket says 172.21.15.47:55368) 2024-03-21T11:34:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.441+0000 7f325ca00640 1 -- 172.21.15.47:0/973520824 learned_addr learned my addr 172.21.15.47:0/973520824 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.442+0000 7f325ca00640 1 -- 172.21.15.47:0/973520824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3258104ad0 msgr2=0x7f325819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.442+0000 7f325ca00640 1 --2- 172.21.15.47:0/973520824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3258104ad0 0x7f325819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.442+0000 7f325ca00640 1 -- 172.21.15.47:0/973520824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32581038e0 msgr2=0x7f3258199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.442+0000 7f325ca00640 1 --2- 172.21.15.47:0/973520824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32581038e0 0x7f3258199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.442+0000 7f325ca00640 1 -- 172.21.15.47:0/973520824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32581a1880 con 0x7f3258105490 2024-03-21T11:34:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.442+0000 7f3257400640 1 --2- 172.21.15.47:0/973520824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3258104ad0 0x7f325819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.442+0000 7f325ca00640 1 --2- 172.21.15.47:0/973520824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3258105490 0x7f32581a11e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f325000e970 tx=0x7f325000ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.442+0000 7f3254c00640 1 -- 172.21.15.47:0/973520824 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325000ccc0 con 0x7f3258105490 2024-03-21T11:34:02.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.442+0000 7f3254c00640 1 -- 172.21.15.47:0/973520824 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f325000ce60 con 0x7f3258105490 2024-03-21T11:34:02.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.442+0000 7f3254c00640 1 -- 172.21.15.47:0/973520824 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3250010490 con 0x7f3258105490 2024-03-21T11:34:02.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.442+0000 7f325ea00640 1 -- 172.21.15.47:0/973520824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32581a1b70 con 0x7f3258105490 2024-03-21T11:34:02.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.442+0000 7f325ea00640 1 -- 172.21.15.47:0/973520824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32581a2040 con 0x7f3258105490 2024-03-21T11:34:02.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.446+0000 7f32617cc740 1 Processor -- start 2024-03-21T11:34:02.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.446+0000 7f3254c00640 1 -- 172.21.15.47:0/973520824 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3250005020 con 0x7f3258105490 2024-03-21T11:34:02.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.446+0000 7f32617cc740 1 -- start start 2024-03-21T11:34:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.446+0000 7f32617cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770adf810 0x55b770adfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.446+0000 7f32617cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770ae09e0 0x55b770ae0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.446+0000 7f32617cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae13a0 0x55b770ae7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.446+0000 7f32617cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b770a442c0 con 0x55b770ae09e0 2024-03-21T11:34:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.446+0000 7f32617cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7709ecc90 con 0x55b770ae13a0 2024-03-21T11:34:02.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.446+0000 7f32617cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b770a40360 con 0x55b770adf810 2024-03-21T11:34:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.446+0000 7f3257400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770adf810 0x55b770adfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.446+0000 7f3257400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770adf810 0x55b770adfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54824/0 (socket says 172.21.15.47:54824) 2024-03-21T11:34:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.446+0000 7f3257e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770ae09e0 0x55b770ae0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.447+0000 7f3257400640 1 -- 172.21.15.47:0/4084948467 learned_addr learned my addr 172.21.15.47:0/4084948467 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.447+0000 7f3257400640 1 --2- 172.21.15.47:0/4084948467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae13a0 0x55b770ae7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.447+0000 7f3257400640 1 -- 172.21.15.47:0/4084948467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae13a0 msgr2=0x55b770ae7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.447+0000 7f3257400640 1 --2- 172.21.15.47:0/4084948467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae13a0 0x55b770ae7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.447+0000 7f3257400640 1 -- 172.21.15.47:0/4084948467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770ae09e0 msgr2=0x55b770ae0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.447+0000 7f3257400640 1 --2- 172.21.15.47:0/4084948467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770ae09e0 0x55b770ae0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.447+0000 7f3257400640 1 -- 172.21.15.47:0/4084948467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b770a11080 con 0x55b770adf810 2024-03-21T11:34:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.447+0000 7f3257400640 1 --2- 172.21.15.47:0/4084948467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770adf810 0x55b770adfc30 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3244012fc0 tx=0x7f3244009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c27bda31987eab5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.447+0000 7f324d200640 1 -- 172.21.15.47:0/4084948467 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32440150f0 con 0x55b770adf810 2024-03-21T11:34:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.448+0000 7f324d200640 1 -- 172.21.15.47:0/4084948467 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32440128b0 con 0x55b770adf810 2024-03-21T11:34:02.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.448+0000 7f324d200640 1 -- 172.21.15.47:0/4084948467 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244005850 con 0x55b770adf810 2024-03-21T11:34:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.448+0000 7f3254c00640 1 --2- 172.21.15.47:0/973520824 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f323c0a2f20 0x7f323c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.448+0000 7f3254c00640 1 -- 172.21.15.47:0/973520824 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f32500cfc10 con 0x7f3258105490 2024-03-21T11:34:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.448+0000 7f3257e00640 1 --2- 172.21.15.47:0/973520824 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f323c0a2f20 0x7f323c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.448+0000 7f32617cc740 1 -- 172.21.15.47:0/4084948467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770adf810 msgr2=0x55b770adfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.448+0000 7f32617cc740 1 --2- 172.21.15.47:0/4084948467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770adf810 0x55b770adfc30 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3244012fc0 tx=0x7f3244009ee0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.448+0000 7f32617cc740 1 -- 172.21.15.47:0/4084948467 shutdown_connections 2024-03-21T11:34:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.448+0000 7f32617cc740 1 --2- 172.21.15.47:0/4084948467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae13a0 0x55b770ae7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.448+0000 7f32617cc740 1 --2- 172.21.15.47:0/4084948467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770ae09e0 0x55b770ae0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.448+0000 7f32617cc740 1 --2- 172.21.15.47:0/4084948467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770adf810 0x55b770adfc30 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.448+0000 7f32617cc740 1 -- 172.21.15.47:0/4084948467 >> 172.21.15.47:0/4084948467 conn(0x55b770adac00 msgr2=0x55b770adb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.448+0000 7f32617cc740 1 -- 172.21.15.47:0/4084948467 shutdown_connections 2024-03-21T11:34:02.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.449+0000 7f32617cc740 1 -- 172.21.15.47:0/4084948467 wait complete. 2024-03-21T11:34:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.449+0000 7f32617cc740 1 -- start start 2024-03-21T11:34:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.449+0000 7f3257e00640 1 --2- 172.21.15.47:0/973520824 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f323c0a2f20 0x7f323c0a53e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f324802c170 tx=0x7f324802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.449+0000 7f32617cc740 1 Processor -- start 2024-03-21T11:34:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f32617cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770adf810 0x55b770ae9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f32617cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae09e0 0x55b770ae9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f32617cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770ae13a0 0x55b770af0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f32617cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b770ae8350 con 0x55b770ae13a0 2024-03-21T11:34:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f32617cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b770a000d0 con 0x55b770adf810 2024-03-21T11:34:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f32617cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b770ae84d0 con 0x55b770ae09e0 2024-03-21T11:34:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f32617cc740 10 client.0 _ll_register_callbacks cb 0x55b770a04e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f3257400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae09e0 0x55b770ae9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f325ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770ae13a0 0x55b770af0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f3257400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae09e0 0x55b770ae9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54838/0 (socket says 172.21.15.47:54838) 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f3257400640 1 -- 172.21.15.47:0/2772027925 learned_addr learned my addr 172.21.15.47:0/2772027925 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f3257e00640 1 --2- 172.21.15.47:0/2772027925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770adf810 0x55b770ae9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f3257400640 1 -- 172.21.15.47:0/2772027925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770adf810 msgr2=0x55b770ae9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f3257400640 1 --2- 172.21.15.47:0/2772027925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770adf810 0x55b770ae9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f3257400640 1 -- 172.21.15.47:0/2772027925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770ae13a0 msgr2=0x55b770af0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.450+0000 7f3257400640 1 --2- 172.21.15.47:0/2772027925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770ae13a0 0x55b770af0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.451+0000 7f3257400640 1 -- 172.21.15.47:0/2772027925 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7707583f0 con 0x55b770ae09e0 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.451+0000 7f325ca00640 1 --2- 172.21.15.47:0/2772027925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770ae13a0 0x55b770af0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.451+0000 7f3257400640 1 --2- 172.21.15.47:0/2772027925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae09e0 0x55b770ae9f70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3244009da0 tx=0x7f3244005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20cdef12dce086b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.451+0000 7f3241600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.451+0000 7f3241600640 1 -- 172.21.15.47:0/2772027925 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244024070 con 0x55b770ae09e0 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.451+0000 7f3241600640 1 -- 172.21.15.47:0/2772027925 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f32440055d0 con 0x55b770ae09e0 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.451+0000 7f32617cc740 1 -- 172.21.15.47:0/2772027925 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32440128b0 con 0x55b770ae09e0 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.452+0000 7f3241600640 1 -- 172.21.15.47:0/2772027925 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f324402f050 con 0x55b770ae09e0 2024-03-21T11:34:02.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.452+0000 7f32617cc740 10 client.29122 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:02.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.452+0000 7f32617cc740 1 -- 172.21.15.47:0/2772027925 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b7708227d0 con 0x55b770ae09e0 2024-03-21T11:34:02.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.452+0000 7f3241600640 1 -- 172.21.15.47:0/2772027925 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32440128b0 con 0x55b770ae09e0 2024-03-21T11:34:02.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.452+0000 7f3241600640 1 -- 172.21.15.47:0/2772027925 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f324400f040 con 0x55b770ae09e0 2024-03-21T11:34:02.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.452+0000 7f32617cc740 10 client.29122 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:02.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.452+0000 7f32617cc740 10 client.29122 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:34:02.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.452+0000 7f32617cc740 20 client.29122 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:02.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.452+0000 7f32617cc740 1 --2- 172.21.15.47:0/2772027925 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b770af5170 0x55b770af7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.452+0000 7f32617cc740 4 client.29122 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:02.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.452+0000 7f32617cc740 1 -- 172.21.15.47:0/2772027925 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b770a51800 con 0x55b770af5170 2024-03-21T11:34:02.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.453+0000 7f3257e00640 1 --2- 172.21.15.47:0/2772027925 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b770af5170 0x55b770af7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.453+0000 7f3257e00640 1 --2- 172.21.15.47:0/2772027925 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b770af5170 0x55b770af7560 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=206da20ef99ee840 server_cookie=be4eb508c966c000 in_seq=0 out_seq=0 2024-03-21T11:34:02.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.453+0000 7f3241600640 10 client.29122 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:34:02.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.454+0000 7f3241600640 1 -- 172.21.15.47:0/2772027925 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b770a51800 con 0x55b770af5170 2024-03-21T11:34:02.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.454+0000 7f3241600640 10 client.29122 handle_command_reply: tid=0 2024-03-21T11:34:02.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.454+0000 7f32617cc740 1 client.29122 shutdown 2024-03-21T11:34:02.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.454+0000 7f32617cc740 1 -- 172.21.15.47:0/2772027925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae09e0 msgr2=0x55b770ae9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.454+0000 7f32617cc740 1 --2- 172.21.15.47:0/2772027925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae09e0 0x55b770ae9f70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3244009da0 tx=0x7f3244005fd0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.454+0000 7f32617cc740 1 -- 172.21.15.47:0/2772027925 shutdown_connections 2024-03-21T11:34:02.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.454+0000 7f32617cc740 1 --2- 172.21.15.47:0/2772027925 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b770af5170 0x55b770af7560 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.454+0000 7f32617cc740 1 --2- 172.21.15.47:0/2772027925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770ae13a0 0x55b770af0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.454+0000 7f32617cc740 1 --2- 172.21.15.47:0/2772027925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae09e0 0x55b770ae9f70 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.454+0000 7f32617cc740 1 --2- 172.21.15.47:0/2772027925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770adf810 0x55b770ae9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.454+0000 7f3241600640 0 client.29122 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:34:02.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.454+0000 7f32617cc740 1 -- 172.21.15.47:0/2772027925 >> 172.21.15.47:0/2772027925 conn(0x55b770adac00 msgr2=0x55b770ad9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.454+0000 7f32617cc740 1 -- 172.21.15.47:0/2772027925 shutdown_connections 2024-03-21T11:34:02.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.454+0000 7f32617cc740 1 -- 172.21.15.47:0/2772027925 wait complete. 2024-03-21T11:34:02.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.455+0000 7f32617cc740 20 client.29122 trim_cache size 0 max 16384 2024-03-21T11:34:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.469+0000 7f32617cc740 1 Processor -- start 2024-03-21T11:34:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.469+0000 7f32617cc740 1 -- start start 2024-03-21T11:34:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.469+0000 7f32617cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770adf810 0x55b770ae3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.469+0000 7f32617cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae09e0 0x55b770af0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.469+0000 7f32617cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae13a0 0x55b770ae3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.469+0000 7f32617cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32581ac400 con 0x55b770adf810 2024-03-21T11:34:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.469+0000 7f32617cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32581add80 con 0x55b770ae13a0 2024-03-21T11:34:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.469+0000 7f32617cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32581aca60 con 0x55b770ae09e0 2024-03-21T11:34:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.469+0000 7f3257400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae09e0 0x55b770af0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.469+0000 7f3257400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae09e0 0x55b770af0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54850/0 (socket says 172.21.15.47:54850) 2024-03-21T11:34:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.469+0000 7f3257400640 1 -- 172.21.15.47:0/4014983689 learned_addr learned my addr 172.21.15.47:0/4014983689 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.469+0000 7f325ca00640 1 --2- 172.21.15.47:0/4014983689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae13a0 0x55b770ae3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.469+0000 7f3257e00640 1 --2- 172.21.15.47:0/4014983689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770adf810 0x55b770ae3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.470+0000 7f3257400640 1 -- 172.21.15.47:0/4014983689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae13a0 msgr2=0x55b770ae3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.470+0000 7f3257400640 1 --2- 172.21.15.47:0/4014983689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae13a0 0x55b770ae3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.470+0000 7f3257400640 1 -- 172.21.15.47:0/4014983689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770adf810 msgr2=0x55b770ae3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.470+0000 7f3257400640 1 --2- 172.21.15.47:0/4014983689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770adf810 0x55b770ae3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.470+0000 7f3257400640 1 -- 172.21.15.47:0/4014983689 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b770af7b90 con 0x55b770ae09e0 2024-03-21T11:34:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.470+0000 7f325ca00640 1 --2- 172.21.15.47:0/4014983689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae13a0 0x55b770ae3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.470+0000 7f3257e00640 1 --2- 172.21.15.47:0/4014983689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770adf810 0x55b770ae3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.470+0000 7f3257400640 1 --2- 172.21.15.47:0/4014983689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae09e0 0x55b770af0f70 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3244009d70 tx=0x7f3244004da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7403c088f49d81d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.470+0000 7f3243e00640 1 -- 172.21.15.47:0/4014983689 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244024040 con 0x55b770ae09e0 2024-03-21T11:34:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.470+0000 7f3243e00640 1 -- 172.21.15.47:0/4014983689 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3244003e50 con 0x55b770ae09e0 2024-03-21T11:34:02.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.470+0000 7f3243e00640 1 -- 172.21.15.47:0/4014983689 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32440044f0 con 0x55b770ae09e0 2024-03-21T11:34:02.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.471+0000 7f32617cc740 1 -- 172.21.15.47:0/4014983689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae09e0 msgr2=0x55b770af0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.471+0000 7f32617cc740 1 --2- 172.21.15.47:0/4014983689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae09e0 0x55b770af0f70 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3244009d70 tx=0x7f3244004da0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.471+0000 7f32617cc740 1 -- 172.21.15.47:0/4014983689 shutdown_connections 2024-03-21T11:34:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.471+0000 7f32617cc740 1 --2- 172.21.15.47:0/4014983689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae13a0 0x55b770ae3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.471+0000 7f32617cc740 1 --2- 172.21.15.47:0/4014983689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae09e0 0x55b770af0f70 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.471+0000 7f32617cc740 1 --2- 172.21.15.47:0/4014983689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770adf810 0x55b770ae3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.471+0000 7f32617cc740 1 -- 172.21.15.47:0/4014983689 >> 172.21.15.47:0/4014983689 conn(0x55b770adac00 msgr2=0x55b770ae9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.471+0000 7f32617cc740 1 -- 172.21.15.47:0/4014983689 shutdown_connections 2024-03-21T11:34:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.471+0000 7f32617cc740 1 -- 172.21.15.47:0/4014983689 wait complete. 2024-03-21T11:34:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.472+0000 7f32617cc740 1 -- start start 2024-03-21T11:34:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.472+0000 7f32617cc740 1 Processor -- start 2024-03-21T11:34:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.472+0000 7f32617cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770adf810 0x55b770aec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.472+0000 7f32617cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae09e0 0x55b770aecab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.472+0000 7f32617cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae13a0 0x55b770aefad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.472+0000 7f32617cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32581ad720 con 0x55b770adf810 2024-03-21T11:34:02.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.472+0000 7f32617cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32581ad0c0 con 0x55b770ae09e0 2024-03-21T11:34:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.472+0000 7f32617cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32581aa5c0 con 0x55b770ae13a0 2024-03-21T11:34:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.473+0000 7f32617cc740 10 client.0 _ll_register_callbacks cb 0x7f3248009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.473+0000 7f3257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae09e0 0x55b770aecab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.473+0000 7f3257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae09e0 0x55b770aecab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36146/0 (socket says 172.21.15.47:36146) 2024-03-21T11:34:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.473+0000 7f3257400640 1 -- 172.21.15.47:0/3485343500 learned_addr learned my addr 172.21.15.47:0/3485343500 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.473+0000 7f3257e00640 1 --2- 172.21.15.47:0/3485343500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770adf810 0x55b770aec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.473+0000 7f325ca00640 1 --2- 172.21.15.47:0/3485343500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae13a0 0x55b770aefad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.473+0000 7f3257400640 1 -- 172.21.15.47:0/3485343500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae13a0 msgr2=0x55b770aefad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.473+0000 7f3257400640 1 --2- 172.21.15.47:0/3485343500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae13a0 0x55b770aefad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.473+0000 7f3257400640 1 -- 172.21.15.47:0/3485343500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770adf810 msgr2=0x55b770aec570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.473+0000 7f3257400640 1 --2- 172.21.15.47:0/3485343500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770adf810 0x55b770aec570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.473+0000 7f3257400640 1 -- 172.21.15.47:0/3485343500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b770ae0200 con 0x55b770ae09e0 2024-03-21T11:34:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.473+0000 7f3257400640 1 --2- 172.21.15.47:0/3485343500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae09e0 0x55b770aecab0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3244002230 tx=0x7f3244005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26098f0b62f6a47e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.473+0000 7f3241600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.474+0000 7f3241600640 1 -- 172.21.15.47:0/3485343500 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244024040 con 0x55b770ae09e0 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.474+0000 7f3241600640 1 -- 172.21.15.47:0/3485343500 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3244003970 con 0x55b770ae09e0 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.474+0000 7f32617cc740 1 -- 172.21.15.47:0/3485343500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3244003e50 con 0x55b770ae09e0 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.474+0000 7f3241600640 1 -- 172.21.15.47:0/3485343500 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3244043050 con 0x55b770ae09e0 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.474+0000 7f32617cc740 10 client.29111 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.474+0000 7f32617cc740 1 -- 172.21.15.47:0/3485343500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b770af11f0 con 0x55b770ae09e0 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.474+0000 7f3241600640 1 -- 172.21.15.47:0/3485343500 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3244003e50 con 0x55b770ae09e0 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.475+0000 7f3241600640 1 -- 172.21.15.47:0/3485343500 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f324403b020 con 0x55b770ae09e0 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.475+0000 7f32617cc740 10 client.29111 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.475+0000 7f32617cc740 10 client.29111 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.475+0000 7f32617cc740 20 client.29111 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.475+0000 7f32617cc740 1 --2- 172.21.15.47:0/3485343500 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b770b50d10 0x55b770b53100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.475+0000 7f32617cc740 4 client.29111 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.475+0000 7f32617cc740 1 -- 172.21.15.47:0/3485343500 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b770a4bb50 con 0x55b770b50d10 2024-03-21T11:34:02.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.475+0000 7f3257e00640 1 --2- 172.21.15.47:0/3485343500 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b770b50d10 0x55b770b53100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.475+0000 7f3257e00640 1 --2- 172.21.15.47:0/3485343500 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b770b50d10 0x55b770b53100 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67660b341b3c8b56 server_cookie=1b0aece768634ecf in_seq=0 out_seq=0 2024-03-21T11:34:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.475+0000 7f3241600640 10 client.29111 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:34:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f3241600640 1 -- 172.21.15.47:0/3485343500 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b770a4bb50 con 0x55b770b50d10 2024-03-21T11:34:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f3241600640 10 client.29111 handle_command_reply: tid=0 2024-03-21T11:34:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f32617cc740 1 client.29111 shutdown 2024-03-21T11:34:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f32617cc740 1 -- 172.21.15.47:0/3485343500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae09e0 msgr2=0x55b770aecab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f32617cc740 1 --2- 172.21.15.47:0/3485343500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae09e0 0x55b770aecab0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3244002230 tx=0x7f3244005e70 comp rx=0 tx=0).stop 2024-03-21T11:34:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f32617cc740 1 -- 172.21.15.47:0/3485343500 shutdown_connections 2024-03-21T11:34:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f32617cc740 1 --2- 172.21.15.47:0/3485343500 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b770b50d10 0x55b770b53100 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f32617cc740 1 --2- 172.21.15.47:0/3485343500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b770ae13a0 0x55b770aefad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f32617cc740 1 --2- 172.21.15.47:0/3485343500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b770ae09e0 0x55b770aecab0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f32617cc740 1 --2- 172.21.15.47:0/3485343500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b770adf810 0x55b770aec570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f32617cc740 1 -- 172.21.15.47:0/3485343500 >> 172.21.15.47:0/3485343500 conn(0x55b770adac00 msgr2=0x55b770afbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f3241600640 0 client.29111 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:34:02.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f32617cc740 1 -- 172.21.15.47:0/3485343500 shutdown_connections 2024-03-21T11:34:02.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f32617cc740 1 -- 172.21.15.47:0/3485343500 wait complete. 2024-03-21T11:34:02.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.476+0000 7f32617cc740 20 client.29111 trim_cache size 0 max 16384 2024-03-21T11:34:02.480 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:34:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.477+0000 7f325ea00640 1 -- 172.21.15.47:0/973520824 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f323c0a2f20 msgr2=0x7f323c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.477+0000 7f325ea00640 1 --2- 172.21.15.47:0/973520824 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f323c0a2f20 0x7f323c0a53e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f324802c170 tx=0x7f324802df90 comp rx=0 tx=0).stop 2024-03-21T11:34:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.477+0000 7f325ea00640 1 -- 172.21.15.47:0/973520824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3258105490 msgr2=0x7f32581a11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.477+0000 7f325ea00640 1 --2- 172.21.15.47:0/973520824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3258105490 0x7f32581a11e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f325000e970 tx=0x7f325000ed70 comp rx=0 tx=0).stop 2024-03-21T11:34:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.477+0000 7f325ea00640 1 -- 172.21.15.47:0/973520824 shutdown_connections 2024-03-21T11:34:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.477+0000 7f325ea00640 1 --2- 172.21.15.47:0/973520824 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f323c0a2f20 0x7f323c0a53e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.477+0000 7f325ea00640 1 --2- 172.21.15.47:0/973520824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3258105490 0x7f32581a11e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.477+0000 7f325ea00640 1 --2- 172.21.15.47:0/973520824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3258104ad0 0x7f325819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.477+0000 7f325ea00640 1 --2- 172.21.15.47:0/973520824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32581038e0 0x7f3258199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.477+0000 7f325ea00640 1 -- 172.21.15.47:0/973520824 >> 172.21.15.47:0/973520824 conn(0x7f32580fee50 msgr2=0x7f3258107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.477+0000 7f325ea00640 1 -- 172.21.15.47:0/973520824 shutdown_connections 2024-03-21T11:34:02.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.477+0000 7f325ea00640 1 -- 172.21.15.47:0/973520824 wait complete. 2024-03-21T11:34:02.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:02 smithi102 ceph-mon[36748]: pgmap v721: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-03-21T11:34:02.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:02 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3674766326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:34:02.495 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:34:02.495 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:34:02.555 INFO:tasks.workunit.client.0.smithi047.stdout: 1 799174 8.06 MB/sec execute 406 sec latency 12.027 ms 2024-03-21T11:34:02.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:02 smithi100 ceph-mon[36245]: pgmap v721: 129 pgs: 129 active+clean; 36 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-03-21T11:34:02.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:02 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3674766326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:34:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.718+0000 7f8c04400640 1 -- 172.21.15.47:0/2579342494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bfc1005a0 msgr2=0x7f8bfc1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.718+0000 7f8c04400640 1 --2- 172.21.15.47:0/2579342494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bfc1005a0 0x7f8bfc1009c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009900 tx=0x7f8bec02bc30 comp rx=0 tx=0).stop 2024-03-21T11:34:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.718+0000 7f8c04400640 1 -- 172.21.15.47:0/2579342494 shutdown_connections 2024-03-21T11:34:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.718+0000 7f8c04400640 1 --2- 172.21.15.47:0/2579342494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bfc102150 0x7f8bfc1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.718+0000 7f8c04400640 1 --2- 172.21.15.47:0/2579342494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8bfc101790 0x7f8bfc101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.718+0000 7f8c04400640 1 --2- 172.21.15.47:0/2579342494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bfc1005a0 0x7f8bfc1009c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.718+0000 7f8c04400640 1 -- 172.21.15.47:0/2579342494 >> 172.21.15.47:0/2579342494 conn(0x7f8bfc0fbb30 msgr2=0x7f8bfc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.718+0000 7f8c04400640 1 -- 172.21.15.47:0/2579342494 shutdown_connections 2024-03-21T11:34:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.718+0000 7f8c04400640 1 -- 172.21.15.47:0/2579342494 wait complete. 2024-03-21T11:34:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.719+0000 7f8c04400640 1 Processor -- start 2024-03-21T11:34:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.719+0000 7f8c04400640 1 -- start start 2024-03-21T11:34:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.720+0000 7f8c04400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8bfc1005a0 0x7f8bfc197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.720+0000 7f8c04400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bfc101790 0x7f8bfc197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.720+0000 7f8c04400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bfc102150 0x7f8bfc19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.720+0000 7f8c04400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc10c260 con 0x7f8bfc1005a0 2024-03-21T11:34:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.720+0000 7f8c04400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc10c0e0 con 0x7f8bfc102150 2024-03-21T11:34:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.720+0000 7f8c04400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc10c3e0 con 0x7f8bfc101790 2024-03-21T11:34:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.720+0000 7f8c01000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bfc101790 0x7f8bfc197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.720+0000 7f8c01a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8bfc1005a0 0x7f8bfc197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.720+0000 7f8c01000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bfc101790 0x7f8bfc197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54884/0 (socket says 172.21.15.47:54884) 2024-03-21T11:34:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.720+0000 7f8c01000640 1 -- 172.21.15.47:0/3595225519 learned_addr learned my addr 172.21.15.47:0/3595225519 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.720+0000 7f8c02400640 1 --2- 172.21.15.47:0/3595225519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bfc102150 0x7f8bfc19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.720+0000 7f8c01000640 1 -- 172.21.15.47:0/3595225519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bfc102150 msgr2=0x7f8bfc19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.720+0000 7f8c01000640 1 --2- 172.21.15.47:0/3595225519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bfc102150 0x7f8bfc19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.720+0000 7f8c01000640 1 -- 172.21.15.47:0/3595225519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8bfc1005a0 msgr2=0x7f8bfc197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.721+0000 7f8c01000640 1 --2- 172.21.15.47:0/3595225519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8bfc1005a0 0x7f8bfc197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.721+0000 7f8c01000640 1 -- 172.21.15.47:0/3595225519 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc19f650 con 0x7f8bfc101790 2024-03-21T11:34:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.721+0000 7f8c01000640 1 --2- 172.21.15.47:0/3595225519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bfc101790 0x7f8bfc197f70 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8be800e9f0 tx=0x7f8be800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.721+0000 7f8bfa000640 1 -- 172.21.15.47:0/3595225519 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be800cd80 con 0x7f8bfc101790 2024-03-21T11:34:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.721+0000 7f8bfa000640 1 -- 172.21.15.47:0/3595225519 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8be8010910 con 0x7f8bfc101790 2024-03-21T11:34:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.721+0000 7f8bfa000640 1 -- 172.21.15.47:0/3595225519 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be8018990 con 0x7f8bfc101790 2024-03-21T11:34:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.721+0000 7f8c04400640 1 -- 172.21.15.47:0/3595225519 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bfc19f940 con 0x7f8bfc101790 2024-03-21T11:34:02.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.721+0000 7f8c04400640 1 -- 172.21.15.47:0/3595225519 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc19fee0 con 0x7f8bfc101790 2024-03-21T11:34:02.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.724+0000 7f8c071c4740 1 Processor -- start 2024-03-21T11:34:02.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.725+0000 7f8c071c4740 1 -- start start 2024-03-21T11:34:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.725+0000 7f8c071c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848ec20 0x56204848f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.725+0000 7f8c071c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848fdf0 0x562048490270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.725+0000 7f8c071c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620484907b0 0x562048497040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.725+0000 7f8c071c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620483ef2c0 con 0x56204848fdf0 2024-03-21T11:34:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.725+0000 7f8c071c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562048397c90 con 0x5620484907b0 2024-03-21T11:34:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.725+0000 7f8c071c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620483eb360 con 0x56204848ec20 2024-03-21T11:34:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.725+0000 7f8c01a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848fdf0 0x562048490270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.725+0000 7f8c02400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848ec20 0x56204848f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.725+0000 7f8c02400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848ec20 0x56204848f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54896/0 (socket says 172.21.15.47:54896) 2024-03-21T11:34:02.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.725+0000 7f8c01a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848fdf0 0x562048490270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55444/0 (socket says 172.21.15.47:55444) 2024-03-21T11:34:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.725+0000 7f8c02400640 1 -- 172.21.15.47:0/324908830 learned_addr learned my addr 172.21.15.47:0/324908830 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.726+0000 7f8bfa000640 1 -- 172.21.15.47:0/3595225519 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8be8010ac0 con 0x7f8bfc101790 2024-03-21T11:34:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.726+0000 7f8c01000640 1 --2- 172.21.15.47:0/324908830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620484907b0 0x562048497040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.726+0000 7f8c02400640 1 -- 172.21.15.47:0/324908830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620484907b0 msgr2=0x562048497040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.726+0000 7f8c02400640 1 --2- 172.21.15.47:0/324908830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620484907b0 0x562048497040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.726+0000 7f8c02400640 1 -- 172.21.15.47:0/324908830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848fdf0 msgr2=0x562048490270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.726+0000 7f8c02400640 1 --2- 172.21.15.47:0/324908830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848fdf0 0x562048490270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.726+0000 7f8c02400640 1 -- 172.21.15.47:0/324908830 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620483bc080 con 0x56204848ec20 2024-03-21T11:34:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.726+0000 7f8c01000640 1 --2- 172.21.15.47:0/324908830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620484907b0 0x562048497040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:34:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.726+0000 7f8c01a00640 1 --2- 172.21.15.47:0/324908830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848fdf0 0x562048490270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.726+0000 7f8c02400640 1 --2- 172.21.15.47:0/324908830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848ec20 0x56204848f040 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009fd0 tx=0x7f8bf40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f57c0cfd7087d84 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.727+0000 7f8bf2a00640 1 -- 172.21.15.47:0/324908830 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4004d20 con 0x56204848ec20 2024-03-21T11:34:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.727+0000 7f8bf2a00640 1 -- 172.21.15.47:0/324908830 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bf4004050 con 0x56204848ec20 2024-03-21T11:34:02.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.727+0000 7f8bf2a00640 1 -- 172.21.15.47:0/324908830 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4005260 con 0x56204848ec20 2024-03-21T11:34:02.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.727+0000 7f8bfa000640 1 --2- 172.21.15.47:0/3595225519 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8be00a3020 0x7f8be00a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.727+0000 7f8bfa000640 1 -- 172.21.15.47:0/3595225519 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8be80cfa40 con 0x7f8bfc101790 2024-03-21T11:34:02.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.728+0000 7f8c071c4740 1 -- 172.21.15.47:0/324908830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848ec20 msgr2=0x56204848f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.728+0000 7f8c071c4740 1 --2- 172.21.15.47:0/324908830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848ec20 0x56204848f040 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009fd0 tx=0x7f8bf40118d0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.728+0000 7f8c01a00640 1 --2- 172.21.15.47:0/3595225519 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8be00a3020 0x7f8be00a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.728+0000 7f8c071c4740 1 -- 172.21.15.47:0/324908830 shutdown_connections 2024-03-21T11:34:02.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.728+0000 7f8c071c4740 1 --2- 172.21.15.47:0/324908830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620484907b0 0x562048497040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.728+0000 7f8c071c4740 1 --2- 172.21.15.47:0/324908830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848fdf0 0x562048490270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.728+0000 7f8c071c4740 1 --2- 172.21.15.47:0/324908830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848ec20 0x56204848f040 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.728+0000 7f8c071c4740 1 -- 172.21.15.47:0/324908830 >> 172.21.15.47:0/324908830 conn(0x562048485c00 msgr2=0x562048486000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.728+0000 7f8c071c4740 1 -- 172.21.15.47:0/324908830 shutdown_connections 2024-03-21T11:34:02.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.728+0000 7f8c071c4740 1 -- 172.21.15.47:0/324908830 wait complete. 2024-03-21T11:34:02.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.728+0000 7f8c01a00640 1 --2- 172.21.15.47:0/3595225519 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8be00a3020 0x7f8be00a54e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8bec02c170 tx=0x7f8bec02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.729+0000 7f8c071c4740 1 -- start start 2024-03-21T11:34:02.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.729+0000 7f8c071c4740 1 Processor -- start 2024-03-21T11:34:02.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.729+0000 7f8c071c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56204848ec20 0x562048498ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.729+0000 7f8c071c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848fdf0 0x562048499400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.729+0000 7f8c071c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620484907b0 0x5620484a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.729+0000 7f8c071c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620484977c0 con 0x56204848fdf0 2024-03-21T11:34:02.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.729+0000 7f8c071c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620483ab0d0 con 0x56204848ec20 2024-03-21T11:34:02.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.729+0000 7f8c071c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562048497940 con 0x5620484907b0 2024-03-21T11:34:02.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.729+0000 7f8c071c4740 10 client.0 _ll_register_callbacks cb 0x5620483afe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:02.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.729+0000 7f8c02400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620484907b0 0x5620484a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.729+0000 7f8c01000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848fdf0 0x562048499400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.729+0000 7f8c02400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620484907b0 0x5620484a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54900/0 (socket says 172.21.15.47:54900) 2024-03-21T11:34:02.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.730+0000 7f8c02400640 1 -- 172.21.15.47:0/2906172633 learned_addr learned my addr 172.21.15.47:0/2906172633 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.730+0000 7f8c01a00640 1 --2- 172.21.15.47:0/2906172633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56204848ec20 0x562048498ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.730+0000 7f8c02400640 1 -- 172.21.15.47:0/2906172633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56204848ec20 msgr2=0x562048498ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.730+0000 7f8c02400640 1 --2- 172.21.15.47:0/2906172633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56204848ec20 0x562048498ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.730+0000 7f8c02400640 1 -- 172.21.15.47:0/2906172633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848fdf0 msgr2=0x562048499400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.730+0000 7f8c02400640 1 --2- 172.21.15.47:0/2906172633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848fdf0 0x562048499400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.730+0000 7f8c02400640 1 -- 172.21.15.47:0/2906172633 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620481033f0 con 0x5620484907b0 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.730+0000 7f8c02400640 1 --2- 172.21.15.47:0/2906172633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620484907b0 0x5620484a03d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009fa0 tx=0x7f8bf4004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cff346559966b6b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.730+0000 7f8be7400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.730+0000 7f8be7400640 1 -- 172.21.15.47:0/2906172633 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4028020 con 0x5620484907b0 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.731+0000 7f8c071c4740 1 -- 172.21.15.47:0/2906172633 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bf4004050 con 0x5620484907b0 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.731+0000 7f8be7400640 1 -- 172.21.15.47:0/2906172633 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8bf4004410 con 0x5620484907b0 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.731+0000 7f8be7400640 1 -- 172.21.15.47:0/2906172633 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf4033050 con 0x5620484907b0 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.731+0000 7f8c071c4740 10 client.29140 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.731+0000 7f8c071c4740 1 -- 172.21.15.47:0/2906172633 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5620481cd7d0 con 0x5620484907b0 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.731+0000 7f8be7400640 1 -- 172.21.15.47:0/2906172633 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bf4004050 con 0x5620484907b0 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.731+0000 7f8be7400640 1 -- 172.21.15.47:0/2906172633 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8bf400f4e0 con 0x5620484907b0 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.731+0000 7f8c071c4740 10 client.29140 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.731+0000 7f8c071c4740 10 client.29140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.731+0000 7f8c071c4740 20 client.29140 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:02.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.731+0000 7f8c071c4740 1 --2- 172.21.15.47:0/2906172633 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620484a4610 0x5620484a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.731+0000 7f8c071c4740 4 client.29140 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:02.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.731+0000 7f8c071c4740 1 -- 172.21.15.47:0/2906172633 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620483fc800 con 0x5620484a4610 2024-03-21T11:34:02.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.732+0000 7f8c01a00640 1 --2- 172.21.15.47:0/2906172633 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620484a4610 0x5620484a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.732+0000 7f8c01a00640 1 --2- 172.21.15.47:0/2906172633 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620484a4610 0x5620484a6a30 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aec7d30ec6df916 server_cookie=7810be35462e1f5e in_seq=0 out_seq=0 2024-03-21T11:34:02.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.732+0000 7f8be7400640 10 client.29140 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:02.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.733+0000 7f8be7400640 1 -- 172.21.15.47:0/2906172633 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5620483fc800 con 0x5620484a4610 2024-03-21T11:34:02.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.734+0000 7f8be7400640 10 client.29140 handle_command_reply: tid=0 2024-03-21T11:34:02.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.734+0000 7f8c071c4740 1 client.29140 shutdown 2024-03-21T11:34:02.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.734+0000 7f8c071c4740 1 -- 172.21.15.47:0/2906172633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620484907b0 msgr2=0x5620484a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.734+0000 7f8c071c4740 1 --2- 172.21.15.47:0/2906172633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620484907b0 0x5620484a03d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009fa0 tx=0x7f8bf4004cf0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.734+0000 7f8c071c4740 1 -- 172.21.15.47:0/2906172633 shutdown_connections 2024-03-21T11:34:02.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.734+0000 7f8c071c4740 1 --2- 172.21.15.47:0/2906172633 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620484a4610 0x5620484a6a30 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.734+0000 7f8c071c4740 1 --2- 172.21.15.47:0/2906172633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620484907b0 0x5620484a03d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.734+0000 7f8c071c4740 1 --2- 172.21.15.47:0/2906172633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848fdf0 0x562048499400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.734+0000 7f8c071c4740 1 --2- 172.21.15.47:0/2906172633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56204848ec20 0x562048498ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.734+0000 7f8be7400640 0 client.29140 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:02.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.734+0000 7f8c071c4740 1 -- 172.21.15.47:0/2906172633 >> 172.21.15.47:0/2906172633 conn(0x562048485c00 msgr2=0x562048484eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.734+0000 7f8c071c4740 1 -- 172.21.15.47:0/2906172633 shutdown_connections 2024-03-21T11:34:02.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.734+0000 7f8c071c4740 1 -- 172.21.15.47:0/2906172633 wait complete. 2024-03-21T11:34:02.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.734+0000 7f8c071c4740 20 client.29140 trim_cache size 0 max 16384 2024-03-21T11:34:02.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.749+0000 7f8c071c4740 1 Processor -- start 2024-03-21T11:34:02.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.749+0000 7f8c071c4740 1 -- start start 2024-03-21T11:34:02.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.749+0000 7f8c071c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848ec20 0x562048484d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.749+0000 7f8c071c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56204848fdf0 0x5620484a0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.749+0000 7f8c071c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620484907b0 0x5620484ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.749+0000 7f8c071c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1a7fb0 con 0x5620484907b0 2024-03-21T11:34:02.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.749+0000 7f8c071c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1a9930 con 0x56204848fdf0 2024-03-21T11:34:02.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.749+0000 7f8c071c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1a8610 con 0x56204848ec20 2024-03-21T11:34:02.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.749+0000 7f8c01a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848ec20 0x562048484d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.750+0000 7f8c01000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56204848fdf0 0x5620484a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.750+0000 7f8c02400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620484907b0 0x5620484ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.750+0000 7f8c01000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56204848fdf0 0x5620484a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36194/0 (socket says 172.21.15.47:36194) 2024-03-21T11:34:02.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.750+0000 7f8c01a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848ec20 0x562048484d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54902/0 (socket says 172.21.15.47:54902) 2024-03-21T11:34:02.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.750+0000 7f8c01000640 1 -- 172.21.15.47:0/2579433695 learned_addr learned my addr 172.21.15.47:0/2579433695 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.750+0000 7f8c01000640 1 -- 172.21.15.47:0/2579433695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848ec20 msgr2=0x562048484d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.750+0000 7f8c01000640 1 --2- 172.21.15.47:0/2579433695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848ec20 0x562048484d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.750+0000 7f8c01000640 1 -- 172.21.15.47:0/2579433695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620484907b0 msgr2=0x5620484ab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.750+0000 7f8c01000640 1 --2- 172.21.15.47:0/2579433695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620484907b0 0x5620484ab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.750+0000 7f8c01000640 1 -- 172.21.15.47:0/2579433695 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620484976a0 con 0x56204848fdf0 2024-03-21T11:34:02.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.750+0000 7f8c01000640 1 --2- 172.21.15.47:0/2579433695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56204848fdf0 0x5620484a0390 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8be8098290 tx=0x7f8be8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a749de692aa93a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.750+0000 7f8bf2000640 1 -- 172.21.15.47:0/2579433695 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be809a130 con 0x56204848fdf0 2024-03-21T11:34:02.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.751+0000 7f8bf2000640 1 -- 172.21.15.47:0/2579433695 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8be80984a0 con 0x56204848fdf0 2024-03-21T11:34:02.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.751+0000 7f8bf2000640 1 -- 172.21.15.47:0/2579433695 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be80baaf0 con 0x56204848fdf0 2024-03-21T11:34:02.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.751+0000 7f8c071c4740 1 -- 172.21.15.47:0/2579433695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56204848fdf0 msgr2=0x5620484a0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.751+0000 7f8c071c4740 1 --2- 172.21.15.47:0/2579433695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56204848fdf0 0x5620484a0390 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8be8098290 tx=0x7f8be8000bc0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.751+0000 7f8c071c4740 1 -- 172.21.15.47:0/2579433695 shutdown_connections 2024-03-21T11:34:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.751+0000 7f8c071c4740 1 --2- 172.21.15.47:0/2579433695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620484907b0 0x5620484ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.751+0000 7f8c071c4740 1 --2- 172.21.15.47:0/2579433695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56204848fdf0 0x5620484a0390 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.751+0000 7f8c071c4740 1 --2- 172.21.15.47:0/2579433695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848ec20 0x562048484d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.751+0000 7f8c071c4740 1 -- 172.21.15.47:0/2579433695 >> 172.21.15.47:0/2579433695 conn(0x562048485c00 msgr2=0x562048484620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.751+0000 7f8c071c4740 1 -- 172.21.15.47:0/2579433695 shutdown_connections 2024-03-21T11:34:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.752+0000 7f8c071c4740 1 -- 172.21.15.47:0/2579433695 wait complete. 2024-03-21T11:34:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.752+0000 7f8c071c4740 1 -- start start 2024-03-21T11:34:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.752+0000 7f8c071c4740 1 Processor -- start 2024-03-21T11:34:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.752+0000 7f8c071c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848ec20 0x56204849fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.752+0000 7f8c071c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848fdf0 0x5620484a0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.752+0000 7f8c071c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620484907b0 0x56204849c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.752+0000 7f8c071c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1a92d0 con 0x56204848ec20 2024-03-21T11:34:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.752+0000 7f8c071c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1a8c70 con 0x5620484907b0 2024-03-21T11:34:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8c071c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1a6170 con 0x56204848fdf0 2024-03-21T11:34:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8c071c4740 10 client.0 _ll_register_callbacks cb 0x7f8bf40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8c02400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620484907b0 0x56204849c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8c01a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848ec20 0x56204849fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8c02400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620484907b0 0x56204849c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36210/0 (socket says 172.21.15.47:36210) 2024-03-21T11:34:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8c02400640 1 -- 172.21.15.47:0/3463301773 learned_addr learned my addr 172.21.15.47:0/3463301773 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8c01000640 1 --2- 172.21.15.47:0/3463301773 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848fdf0 0x5620484a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8c02400640 1 -- 172.21.15.47:0/3463301773 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848fdf0 msgr2=0x5620484a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8c02400640 1 --2- 172.21.15.47:0/3463301773 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848fdf0 0x5620484a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8c02400640 1 -- 172.21.15.47:0/3463301773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848ec20 msgr2=0x56204849fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8c02400640 1 --2- 172.21.15.47:0/3463301773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848ec20 0x56204849fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8c02400640 1 -- 172.21.15.47:0/3463301773 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56204849cce0 con 0x5620484907b0 2024-03-21T11:34:02.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8c01a00640 1 --2- 172.21.15.47:0/3463301773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848ec20 0x56204849fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8c02400640 1 --2- 172.21.15.47:0/3463301773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620484907b0 0x56204849c6f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400f980 tx=0x7f8bf4011f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21e1a56ec2e2037e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.753+0000 7f8be7400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.754+0000 7f8be7400640 1 -- 172.21.15.47:0/3463301773 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4028040 con 0x5620484907b0 2024-03-21T11:34:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.754+0000 7f8be7400640 1 -- 172.21.15.47:0/3463301773 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8bf400f4e0 con 0x5620484907b0 2024-03-21T11:34:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.754+0000 7f8c071c4740 1 -- 172.21.15.47:0/3463301773 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8be80984a0 con 0x5620484907b0 2024-03-21T11:34:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.754+0000 7f8be7400640 1 -- 172.21.15.47:0/3463301773 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf4044050 con 0x5620484907b0 2024-03-21T11:34:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.754+0000 7f8c071c4740 10 client.29126 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.754+0000 7f8c071c4740 1 -- 172.21.15.47:0/3463301773 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562048500a70 con 0x5620484907b0 2024-03-21T11:34:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.754+0000 7f8be7400640 1 -- 172.21.15.47:0/3463301773 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8be80984a0 con 0x5620484907b0 2024-03-21T11:34:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.754+0000 7f8be7400640 1 -- 172.21.15.47:0/3463301773 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8bf403f030 con 0x5620484907b0 2024-03-21T11:34:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.755+0000 7f8c071c4740 10 client.29126 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.755+0000 7f8c071c4740 10 client.29126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:02.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.755+0000 7f8c071c4740 20 client.29126 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.755+0000 7f8c071c4740 1 --2- 172.21.15.47:0/3463301773 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562048501390 0x5620485037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.755+0000 7f8c071c4740 4 client.29126 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:34:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.755+0000 7f8c071c4740 1 -- 172.21.15.47:0/3463301773 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5620483f6b50 con 0x562048501390 2024-03-21T11:34:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.755+0000 7f8c01a00640 1 --2- 172.21.15.47:0/3463301773 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562048501390 0x5620485037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.755+0000 7f8c01a00640 1 --2- 172.21.15.47:0/3463301773 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562048501390 0x5620485037b0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adaf5299ebf56ddd server_cookie=9abfb62fd5a7a427 in_seq=0 out_seq=0 2024-03-21T11:34:02.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.755+0000 7f8be7400640 10 client.29126 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:02.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.757+0000 7f8be7400640 1 -- 172.21.15.47:0/3463301773 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5620483f6b50 con 0x562048501390 2024-03-21T11:34:02.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.757+0000 7f8be7400640 10 client.29126 handle_command_reply: tid=0 2024-03-21T11:34:02.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.757+0000 7f8c071c4740 1 client.29126 shutdown 2024-03-21T11:34:02.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.757+0000 7f8c071c4740 1 -- 172.21.15.47:0/3463301773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620484907b0 msgr2=0x56204849c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.757+0000 7f8c071c4740 1 --2- 172.21.15.47:0/3463301773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620484907b0 0x56204849c6f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400f980 tx=0x7f8bf4011f10 comp rx=0 tx=0).stop 2024-03-21T11:34:02.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.758+0000 7f8c071c4740 1 -- 172.21.15.47:0/3463301773 shutdown_connections 2024-03-21T11:34:02.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.758+0000 7f8c071c4740 1 --2- 172.21.15.47:0/3463301773 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562048501390 0x5620485037b0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.758+0000 7f8c071c4740 1 --2- 172.21.15.47:0/3463301773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620484907b0 0x56204849c6f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.758+0000 7f8c071c4740 1 --2- 172.21.15.47:0/3463301773 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56204848fdf0 0x5620484a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.758+0000 7f8c071c4740 1 --2- 172.21.15.47:0/3463301773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56204848ec20 0x56204849fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.758+0000 7f8be7400640 0 client.29126 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.758+0000 7f8c071c4740 1 -- 172.21.15.47:0/3463301773 >> 172.21.15.47:0/3463301773 conn(0x562048485c00 msgr2=0x562048484eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.758+0000 7f8c071c4740 1 -- 172.21.15.47:0/3463301773 shutdown_connections 2024-03-21T11:34:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.758+0000 7f8c071c4740 1 -- 172.21.15.47:0/3463301773 wait complete. 2024-03-21T11:34:02.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.758+0000 7f8c071c4740 20 client.29126 trim_cache size 0 max 16384 2024-03-21T11:34:02.761 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:34:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.759+0000 7f8c04400640 1 -- 172.21.15.47:0/3595225519 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8be00a3020 msgr2=0x7f8be00a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.759+0000 7f8c04400640 1 --2- 172.21.15.47:0/3595225519 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8be00a3020 0x7f8be00a54e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8bec02c170 tx=0x7f8bec02df90 comp rx=0 tx=0).stop 2024-03-21T11:34:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.759+0000 7f8c04400640 1 -- 172.21.15.47:0/3595225519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bfc101790 msgr2=0x7f8bfc197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.759+0000 7f8c04400640 1 --2- 172.21.15.47:0/3595225519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bfc101790 0x7f8bfc197f70 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8be800e9f0 tx=0x7f8be800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.759+0000 7f8c04400640 1 -- 172.21.15.47:0/3595225519 shutdown_connections 2024-03-21T11:34:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.759+0000 7f8c04400640 1 --2- 172.21.15.47:0/3595225519 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8be00a3020 0x7f8be00a54e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.759+0000 7f8c04400640 1 --2- 172.21.15.47:0/3595225519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bfc102150 0x7f8bfc19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.759+0000 7f8c04400640 1 --2- 172.21.15.47:0/3595225519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bfc101790 0x7f8bfc197f70 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.759+0000 7f8c04400640 1 --2- 172.21.15.47:0/3595225519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8bfc1005a0 0x7f8bfc197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:02.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.759+0000 7f8c04400640 1 -- 172.21.15.47:0/3595225519 >> 172.21.15.47:0/3595225519 conn(0x7f8bfc0fbb30 msgr2=0x7f8bfc0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:02.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.759+0000 7f8c04400640 1 -- 172.21.15.47:0/3595225519 shutdown_connections 2024-03-21T11:34:02.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:02.759+0000 7f8c04400640 1 -- 172.21.15.47:0/3595225519 wait complete. 2024-03-21T11:34:03.555 INFO:tasks.workunit.client.0.smithi047.stdout: 1 800264 8.05 MB/sec execute 407 sec latency 20.617 ms 2024-03-21T11:34:03.779 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:34:03.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:34:03.779 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:34:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.018+0000 7f8d53e00640 1 -- 172.21.15.47:0/1099883874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d541015a0 msgr2=0x7f8d541019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.018+0000 7f8d53e00640 1 --2- 172.21.15.47:0/1099883874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d541015a0 0x7f8d541019c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009900 tx=0x7f8d4402bc50 comp rx=0 tx=0).stop 2024-03-21T11:34:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.018+0000 7f8d53e00640 1 -- 172.21.15.47:0/1099883874 shutdown_connections 2024-03-21T11:34:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.018+0000 7f8d53e00640 1 --2- 172.21.15.47:0/1099883874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d54103150 0x7f8d541099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.018+0000 7f8d53e00640 1 --2- 172.21.15.47:0/1099883874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d54102790 0x7f8d54102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.018+0000 7f8d53e00640 1 --2- 172.21.15.47:0/1099883874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d541015a0 0x7f8d541019c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.018+0000 7f8d53e00640 1 -- 172.21.15.47:0/1099883874 >> 172.21.15.47:0/1099883874 conn(0x7f8d540fcb10 msgr2=0x7f8d540fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.018+0000 7f8d53e00640 1 -- 172.21.15.47:0/1099883874 shutdown_connections 2024-03-21T11:34:04.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.018+0000 7f8d53e00640 1 -- 172.21.15.47:0/1099883874 wait complete. 2024-03-21T11:34:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.019+0000 7f8d53e00640 1 Processor -- start 2024-03-21T11:34:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.019+0000 7f8d53e00640 1 -- start start 2024-03-21T11:34:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.019+0000 7f8d53e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d541015a0 0x7f8d54075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.019+0000 7f8d53e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d54102790 0x7f8d54073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.019+0000 7f8d53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d54103150 0x7f8d54073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.019+0000 7f8d53e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5410d230 con 0x7f8d54102790 2024-03-21T11:34:04.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.019+0000 7f8d53e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5410d0b0 con 0x7f8d54103150 2024-03-21T11:34:04.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.019+0000 7f8d53e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5410d3b0 con 0x7f8d541015a0 2024-03-21T11:34:04.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.020+0000 7f8d52a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d541015a0 0x7f8d54075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.020+0000 7f8d52a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d541015a0 0x7f8d54075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54922/0 (socket says 172.21.15.47:54922) 2024-03-21T11:34:04.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.020+0000 7f8d52a00640 1 -- 172.21.15.47:0/408974164 learned_addr learned my addr 172.21.15.47:0/408974164 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:04.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.020+0000 7f8d53400640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d54103150 0x7f8d54073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.020+0000 7f8d4a000640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d54102790 0x7f8d54073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.020+0000 7f8d53400640 1 -- 172.21.15.47:0/408974164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d541015a0 msgr2=0x7f8d54075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.020+0000 7f8d53400640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d541015a0 0x7f8d54075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.020+0000 7f8d53400640 1 -- 172.21.15.47:0/408974164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d54102790 msgr2=0x7f8d54073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.020+0000 7f8d53400640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d54102790 0x7f8d54073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.020+0000 7f8d53400640 1 -- 172.21.15.47:0/408974164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d54074510 con 0x7f8d54103150 2024-03-21T11:34:04.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.020+0000 7f8d52a00640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d541015a0 0x7f8d54075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:04.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.021+0000 7f8d53400640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d54103150 0x7f8d54073fd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00da10 tx=0x7f8d4c00ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:04.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.021+0000 7f8d58a00640 1 -- 172.21.15.47:0/408974164 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c00bd70 con 0x7f8d54103150 2024-03-21T11:34:04.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.021+0000 7f8d53e00640 1 -- 172.21.15.47:0/408974164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d54074800 con 0x7f8d54103150 2024-03-21T11:34:04.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.021+0000 7f8d58a00640 1 -- 172.21.15.47:0/408974164 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d4c003e50 con 0x7f8d54103150 2024-03-21T11:34:04.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.021+0000 7f8d58a00640 1 -- 172.21.15.47:0/408974164 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c0106a0 con 0x7f8d54103150 2024-03-21T11:34:04.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.021+0000 7f8d53e00640 1 -- 172.21.15.47:0/408974164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d541a6440 con 0x7f8d54103150 2024-03-21T11:34:04.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.024+0000 7f8d58a00640 1 -- 172.21.15.47:0/408974164 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8d4c003970 con 0x7f8d54103150 2024-03-21T11:34:04.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.025+0000 7f8d5ca4f740 1 Processor -- start 2024-03-21T11:34:04.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.025+0000 7f8d5ca4f740 1 -- start start 2024-03-21T11:34:04.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.025+0000 7f8d5ca4f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.025+0000 7f8d5ca4f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae67ca0 0x563c0ae6c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.025+0000 7f8d5ca4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae6ce10 0x563c0ae6f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.025+0000 7f8d5ca4f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c0adcceb0 con 0x563c0ae67ca0 2024-03-21T11:34:04.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.025+0000 7f8d5ca4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c0ad75a60 con 0x563c0ae6ce10 2024-03-21T11:34:04.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.025+0000 7f8d5ca4f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c0adc8f50 con 0x563c0ae672b0 2024-03-21T11:34:04.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.025+0000 7f8d4a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.025+0000 7f8d4a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae676d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54932/0 (socket says 172.21.15.47:54932) 2024-03-21T11:34:04.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.025+0000 7f8d4a000640 1 -- 172.21.15.47:0/1593121434 learned_addr learned my addr 172.21.15.47:0/1593121434 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:04.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.025+0000 7f8d52a00640 1 --2- 172.21.15.47:0/1593121434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae67ca0 0x563c0ae6c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.025+0000 7f8d4a000640 1 --2- 172.21.15.47:0/1593121434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae6ce10 0x563c0ae6f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.026+0000 7f8d4a000640 1 -- 172.21.15.47:0/1593121434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae6ce10 msgr2=0x563c0ae6f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.026+0000 7f8d4a000640 1 --2- 172.21.15.47:0/1593121434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae6ce10 0x563c0ae6f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.026+0000 7f8d4a000640 1 -- 172.21.15.47:0/1593121434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae67ca0 msgr2=0x563c0ae6c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.026+0000 7f8d4a000640 1 --2- 172.21.15.47:0/1593121434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae67ca0 0x563c0ae6c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.026+0000 7f8d4a000640 1 -- 172.21.15.47:0/1593121434 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c0aba97e0 con 0x563c0ae672b0 2024-03-21T11:34:04.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.026+0000 7f8d58a00640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d340a2f20 0x7f8d340a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.026+0000 7f8d4a000640 1 --2- 172.21.15.47:0/1593121434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae676d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c011810 tx=0x7f8d3c0097d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46e8ee7fb1617eb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:04.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.026+0000 7f8d58a00640 1 -- 172.21.15.47:0/408974164 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8d4c0cfb80 con 0x7f8d54103150 2024-03-21T11:34:04.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.026+0000 7f8d52a00640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d340a2f20 0x7f8d340a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.026+0000 7f8d43e00640 1 -- 172.21.15.47:0/1593121434 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c004030 con 0x563c0ae672b0 2024-03-21T11:34:04.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.027+0000 7f8d43e00640 1 -- 172.21.15.47:0/1593121434 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d3c011e50 con 0x563c0ae672b0 2024-03-21T11:34:04.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.027+0000 7f8d43e00640 1 -- 172.21.15.47:0/1593121434 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c003850 con 0x563c0ae672b0 2024-03-21T11:34:04.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.027+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/1593121434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 msgr2=0x563c0ae676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.027+0000 7f8d52a00640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d340a2f20 0x7f8d340a53e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8d4402c900 tx=0x7f8d44005b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:04.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.027+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/1593121434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae676d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c011810 tx=0x7f8d3c0097d0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.028+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/1593121434 shutdown_connections 2024-03-21T11:34:04.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.028+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/1593121434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae6ce10 0x563c0ae6f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.028+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/1593121434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae67ca0 0x563c0ae6c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.028+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/1593121434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae676d0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.028+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/1593121434 >> 172.21.15.47:0/1593121434 conn(0x563c0ae62820 msgr2=0x563c0ae62c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:04.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.028+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/1593121434 shutdown_connections 2024-03-21T11:34:04.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.028+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/1593121434 wait complete. 2024-03-21T11:34:04.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.028+0000 7f8d5ca4f740 1 -- start start 2024-03-21T11:34:04.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.028+0000 7f8d5ca4f740 1 Processor -- start 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.029+0000 7f8d5ca4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae672b0 0x563c0ae715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.029+0000 7f8d5ca4f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae67ca0 0x563c0ae71af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.029+0000 7f8d5ca4f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae6ce10 0x563c0ae78b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.029+0000 7f8d5ca4f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c0ae6fdc0 con 0x563c0ae67ca0 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.029+0000 7f8d5ca4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c0ad88eb0 con 0x563c0ae672b0 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.029+0000 7f8d5ca4f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c0ae6ff40 con 0x563c0ae6ce10 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.029+0000 7f8d5ca4f740 10 client.0 _ll_register_callbacks cb 0x563c0ad7d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.029+0000 7f8d52a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae672b0 0x563c0ae715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.029+0000 7f8d53400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae6ce10 0x563c0ae78b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.029+0000 7f8d52a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae672b0 0x563c0ae715b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36246/0 (socket says 172.21.15.47:36246) 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.029+0000 7f8d52a00640 1 -- 172.21.15.47:0/4031535785 learned_addr learned my addr 172.21.15.47:0/4031535785 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.029+0000 7f8d4a000640 1 --2- 172.21.15.47:0/4031535785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae67ca0 0x563c0ae71af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.030+0000 7f8d53400640 1 -- 172.21.15.47:0/4031535785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae672b0 msgr2=0x563c0ae715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.030+0000 7f8d53400640 1 --2- 172.21.15.47:0/4031535785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae672b0 0x563c0ae715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.030+0000 7f8d53400640 1 -- 172.21.15.47:0/4031535785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae67ca0 msgr2=0x563c0ae71af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.030+0000 7f8d53400640 1 --2- 172.21.15.47:0/4031535785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae67ca0 0x563c0ae71af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.030+0000 7f8d53400640 1 -- 172.21.15.47:0/4031535785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c0ac09f70 con 0x563c0ae6ce10 2024-03-21T11:34:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.030+0000 7f8d4a000640 1 --2- 172.21.15.47:0/4031535785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae67ca0 0x563c0ae71af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.030+0000 7f8d53400640 1 --2- 172.21.15.47:0/4031535785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae6ce10 0x563c0ae78b30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00b630 tx=0x7f8d4c09c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c29929d6a0b5891 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.030+0000 7f8d40c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.030+0000 7f8d40c00640 1 -- 172.21.15.47:0/4031535785 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c0bc0c0 con 0x563c0ae6ce10 2024-03-21T11:34:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.030+0000 7f8d40c00640 1 -- 172.21.15.47:0/4031535785 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8d4c0042e0 con 0x563c0ae6ce10 2024-03-21T11:34:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.030+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/4031535785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d3c011e50 con 0x563c0ae6ce10 2024-03-21T11:34:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.030+0000 7f8d40c00640 1 -- 172.21.15.47:0/4031535785 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d4c0a48d0 con 0x563c0ae6ce10 2024-03-21T11:34:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.030+0000 7f8d5ca4f740 10 client.29158 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.030+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/4031535785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563c0aaa1170 con 0x563c0ae6ce10 2024-03-21T11:34:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.031+0000 7f8d40c00640 1 -- 172.21.15.47:0/4031535785 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d3c011e50 con 0x563c0ae6ce10 2024-03-21T11:34:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.031+0000 7f8d40c00640 1 -- 172.21.15.47:0/4031535785 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8d4c024060 con 0x563c0ae6ce10 2024-03-21T11:34:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.031+0000 7f8d5ca4f740 10 client.29158 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.031+0000 7f8d5ca4f740 10 client.29158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.031+0000 7f8d5ca4f740 20 client.29158 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.031+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/4031535785 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563c0ae7d4a0 0x563c0ae7f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.031+0000 7f8d5ca4f740 4 client.29158 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.031+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/4031535785 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c0adda3f0 con 0x563c0ae7d4a0 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.031+0000 7f8d52a00640 1 --2- 172.21.15.47:0/4031535785 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563c0ae7d4a0 0x563c0ae7f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.032+0000 7f8d52a00640 1 --2- 172.21.15.47:0/4031535785 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563c0ae7d4a0 0x563c0ae7f8e0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27d9fbc390d87d30 server_cookie=80e6caaddc6fae6a in_seq=0 out_seq=0 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.032+0000 7f8d40c00640 10 client.29158 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.033+0000 7f8d40c00640 1 -- 172.21.15.47:0/4031535785 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563c0adda3f0 con 0x563c0ae7d4a0 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.033+0000 7f8d40c00640 10 client.29158 handle_command_reply: tid=0 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.033+0000 7f8d5ca4f740 1 client.29158 shutdown 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.033+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/4031535785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae6ce10 msgr2=0x563c0ae78b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.033+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/4031535785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae6ce10 0x563c0ae78b30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00b630 tx=0x7f8d4c09c680 comp rx=0 tx=0).stop 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.033+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/4031535785 shutdown_connections 2024-03-21T11:34:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.033+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/4031535785 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563c0ae7d4a0 0x563c0ae7f8e0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.033+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/4031535785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae6ce10 0x563c0ae78b30 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.033+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/4031535785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae67ca0 0x563c0ae71af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.033+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/4031535785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae672b0 0x563c0ae715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.033+0000 7f8d40c00640 0 client.29158 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.033+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/4031535785 >> 172.21.15.47:0/4031535785 conn(0x563c0ae62820 msgr2=0x563c0ae61a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.033+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/4031535785 shutdown_connections 2024-03-21T11:34:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.033+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/4031535785 wait complete. 2024-03-21T11:34:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.034+0000 7f8d5ca4f740 20 client.29158 trim_cache size 0 max 16384 2024-03-21T11:34:04.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.056+0000 7f8d5ca4f740 1 Processor -- start 2024-03-21T11:34:04.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.056+0000 7f8d5ca4f740 1 -- start start 2024-03-21T11:34:04.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.056+0000 7f8d5ca4f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae71030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.056+0000 7f8d5ca4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae67ca0 0x563c0ae78af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.056+0000 7f8d5ca4f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae6ce10 0x563c0ae841a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.056+0000 7f8d5ca4f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d541b0550 con 0x563c0ae6ce10 2024-03-21T11:34:04.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.056+0000 7f8d5ca4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d541b1ed0 con 0x563c0ae67ca0 2024-03-21T11:34:04.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.056+0000 7f8d5ca4f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d541b0bb0 con 0x563c0ae672b0 2024-03-21T11:34:04.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.057+0000 7f8d4a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae67ca0 0x563c0ae78af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.057+0000 7f8d52a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae71030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.057+0000 7f8d4a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae67ca0 0x563c0ae78af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36254/0 (socket says 172.21.15.47:36254) 2024-03-21T11:34:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.057+0000 7f8d4a000640 1 -- 172.21.15.47:0/1548096544 learned_addr learned my addr 172.21.15.47:0/1548096544 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.057+0000 7f8d53400640 1 --2- 172.21.15.47:0/1548096544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae6ce10 0x563c0ae841a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.057+0000 7f8d52a00640 1 -- 172.21.15.47:0/1548096544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae67ca0 msgr2=0x563c0ae78af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.057+0000 7f8d52a00640 1 --2- 172.21.15.47:0/1548096544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae67ca0 0x563c0ae78af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.057+0000 7f8d52a00640 1 -- 172.21.15.47:0/1548096544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae6ce10 msgr2=0x563c0ae841a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.057+0000 7f8d52a00640 1 --2- 172.21.15.47:0/1548096544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae6ce10 0x563c0ae841a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.057+0000 7f8d52a00640 1 -- 172.21.15.47:0/1548096544 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c0ae63290 con 0x563c0ae672b0 2024-03-21T11:34:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.057+0000 7f8d4a000640 1 --2- 172.21.15.47:0/1548096544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae67ca0 0x563c0ae78af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.057+0000 7f8d52a00640 1 --2- 172.21.15.47:0/1548096544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae71030 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x563c0ae80150 tx=0x7f8d44041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3006d7eae2785f7e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.058+0000 7f8d43400640 1 -- 172.21.15.47:0/1548096544 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d440046c0 con 0x563c0ae672b0 2024-03-21T11:34:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.058+0000 7f8d43400640 1 -- 172.21.15.47:0/1548096544 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d44004ed0 con 0x563c0ae672b0 2024-03-21T11:34:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.058+0000 7f8d43400640 1 -- 172.21.15.47:0/1548096544 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44003830 con 0x563c0ae672b0 2024-03-21T11:34:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.059+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/1548096544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 msgr2=0x563c0ae71030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.059+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/1548096544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae71030 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x563c0ae80150 tx=0x7f8d44041000 comp rx=0 tx=0).stop 2024-03-21T11:34:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.059+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/1548096544 shutdown_connections 2024-03-21T11:34:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.059+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/1548096544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae6ce10 0x563c0ae841a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.059+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/1548096544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae67ca0 0x563c0ae78af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.059+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/1548096544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae71030 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.059+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/1548096544 >> 172.21.15.47:0/1548096544 conn(0x563c0ae62820 msgr2=0x563c0ae66310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.059+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/1548096544 shutdown_connections 2024-03-21T11:34:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.059+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/1548096544 wait complete. 2024-03-21T11:34:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.059+0000 7f8d5ca4f740 1 -- start start 2024-03-21T11:34:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.060+0000 7f8d5ca4f740 1 Processor -- start 2024-03-21T11:34:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.060+0000 7f8d5ca4f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae78870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.060+0000 7f8d5ca4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae67ca0 0x563c0ae76ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.060+0000 7f8d5ca4f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae6ce10 0x563c0ae77430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.060+0000 7f8d5ca4f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d541b1870 con 0x563c0ae6ce10 2024-03-21T11:34:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.060+0000 7f8d5ca4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d541b1210 con 0x563c0ae67ca0 2024-03-21T11:34:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.060+0000 7f8d5ca4f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d541ae710 con 0x563c0ae672b0 2024-03-21T11:34:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.060+0000 7f8d5ca4f740 10 client.0 _ll_register_callbacks cb 0x7f8d4c097a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.060+0000 7f8d52a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae78870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.060+0000 7f8d4a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae67ca0 0x563c0ae76ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.060+0000 7f8d52a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae78870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54974/0 (socket says 172.21.15.47:54974) 2024-03-21T11:34:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.060+0000 7f8d4a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae67ca0 0x563c0ae76ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36270/0 (socket says 172.21.15.47:36270) 2024-03-21T11:34:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.060+0000 7f8d52a00640 1 -- 172.21.15.47:0/825719282 learned_addr learned my addr 172.21.15.47:0/825719282 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.060+0000 7f8d53400640 1 --2- 172.21.15.47:0/825719282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae6ce10 0x563c0ae77430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.061+0000 7f8d52a00640 1 -- 172.21.15.47:0/825719282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae67ca0 msgr2=0x563c0ae76ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.061+0000 7f8d52a00640 1 --2- 172.21.15.47:0/825719282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae67ca0 0x563c0ae76ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.061+0000 7f8d52a00640 1 -- 172.21.15.47:0/825719282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae6ce10 msgr2=0x563c0ae77430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.061+0000 7f8d52a00640 1 --2- 172.21.15.47:0/825719282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae6ce10 0x563c0ae77430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.061+0000 7f8d52a00640 1 -- 172.21.15.47:0/825719282 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c0ae77ad0 con 0x563c0ae672b0 2024-03-21T11:34:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.061+0000 7f8d4a000640 1 --2- 172.21.15.47:0/825719282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae67ca0 0x563c0ae76ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.061+0000 7f8d52a00640 1 --2- 172.21.15.47:0/825719282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae78870 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8d44005ab0 tx=0x7f8d4402bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8343ba0019c41630 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.061+0000 7f8d40c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.061+0000 7f8d40c00640 1 -- 172.21.15.47:0/825719282 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44063060 con 0x563c0ae672b0 2024-03-21T11:34:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.061+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/825719282 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d44004ed0 con 0x563c0ae672b0 2024-03-21T11:34:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.061+0000 7f8d40c00640 1 -- 172.21.15.47:0/825719282 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8d44054050 con 0x563c0ae672b0 2024-03-21T11:34:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.062+0000 7f8d40c00640 1 -- 172.21.15.47:0/825719282 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d44050020 con 0x563c0ae672b0 2024-03-21T11:34:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.062+0000 7f8d5ca4f740 10 client.29164 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.062+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/825719282 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563c0aed9cf0 con 0x563c0ae672b0 2024-03-21T11:34:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.062+0000 7f8d40c00640 1 -- 172.21.15.47:0/825719282 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d44004ed0 con 0x563c0ae672b0 2024-03-21T11:34:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.062+0000 7f8d40c00640 1 -- 172.21.15.47:0/825719282 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8d44003d00 con 0x563c0ae672b0 2024-03-21T11:34:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.062+0000 7f8d5ca4f740 10 client.29164 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.062+0000 7f8d5ca4f740 10 client.29164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.062+0000 7f8d5ca4f740 20 client.29164 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.062+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/825719282 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563c0aeda680 0x563c0aedcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.062+0000 7f8d5ca4f740 4 client.29164 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:34:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.062+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/825719282 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563c0add4740 con 0x563c0aeda680 2024-03-21T11:34:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.063+0000 7f8d4a000640 1 --2- 172.21.15.47:0/825719282 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563c0aeda680 0x563c0aedcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.063+0000 7f8d4a000640 1 --2- 172.21.15.47:0/825719282 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563c0aeda680 0x563c0aedcaa0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e71f7ed1ed03ddea server_cookie=84a521b2d7fc68b3 in_seq=0 out_seq=0 2024-03-21T11:34:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.063+0000 7f8d40c00640 10 client.29164 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.064+0000 7f8d40c00640 1 -- 172.21.15.47:0/825719282 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563c0add4740 con 0x563c0aeda680 2024-03-21T11:34:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.064+0000 7f8d40c00640 10 client.29164 handle_command_reply: tid=0 2024-03-21T11:34:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.064+0000 7f8d5ca4f740 1 client.29164 shutdown 2024-03-21T11:34:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.065+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/825719282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 msgr2=0x563c0ae78870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.065+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/825719282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae78870 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8d44005ab0 tx=0x7f8d4402bc50 comp rx=0 tx=0).stop 2024-03-21T11:34:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.065+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/825719282 shutdown_connections 2024-03-21T11:34:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.065+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/825719282 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563c0aeda680 0x563c0aedcaa0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.065+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/825719282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c0ae6ce10 0x563c0ae77430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.065+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/825719282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c0ae67ca0 0x563c0ae76ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.065+0000 7f8d5ca4f740 1 --2- 172.21.15.47:0/825719282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c0ae672b0 0x563c0ae78870 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.065+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/825719282 >> 172.21.15.47:0/825719282 conn(0x563c0ae62820 msgr2=0x563c0ae611f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.065+0000 7f8d40c00640 0 client.29164 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.065+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/825719282 shutdown_connections 2024-03-21T11:34:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.065+0000 7f8d5ca4f740 1 -- 172.21.15.47:0/825719282 wait complete. 2024-03-21T11:34:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.065+0000 7f8d5ca4f740 20 client.29164 trim_cache size 0 max 16384 2024-03-21T11:34:04.068 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:34:04.068 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:34:04.068 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "125057cb-93f4-4771-895a-7be62ef4b900", 2024-03-21T11:34:04.068 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:34:04.068 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:34:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.066+0000 7f8d53e00640 1 -- 172.21.15.47:0/408974164 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d340a2f20 msgr2=0x7f8d340a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.066+0000 7f8d53e00640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d340a2f20 0x7f8d340a53e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8d4402c900 tx=0x7f8d44005b70 comp rx=0 tx=0).stop 2024-03-21T11:34:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.066+0000 7f8d53e00640 1 -- 172.21.15.47:0/408974164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d54103150 msgr2=0x7f8d54073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.066+0000 7f8d53e00640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d54103150 0x7f8d54073fd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00da10 tx=0x7f8d4c00ded0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.066+0000 7f8d53e00640 1 -- 172.21.15.47:0/408974164 shutdown_connections 2024-03-21T11:34:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.066+0000 7f8d53e00640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d340a2f20 0x7f8d340a53e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.066+0000 7f8d53e00640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d54103150 0x7f8d54073fd0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.066+0000 7f8d53e00640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d54102790 0x7f8d54073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.066+0000 7f8d53e00640 1 --2- 172.21.15.47:0/408974164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d541015a0 0x7f8d54075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.066+0000 7f8d53e00640 1 -- 172.21.15.47:0/408974164 >> 172.21.15.47:0/408974164 conn(0x7f8d540fcb10 msgr2=0x7f8d54105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.066+0000 7f8d53e00640 1 -- 172.21.15.47:0/408974164 shutdown_connections 2024-03-21T11:34:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.066+0000 7f8d53e00640 1 -- 172.21.15.47:0/408974164 wait complete. 2024-03-21T11:34:04.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:34:04.088 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:34:04.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.313+0000 7f57d5600640 1 -- 172.21.15.47:0/1537194829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d01016d0 msgr2=0x7f57d0101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.313+0000 7f57d5600640 1 --2- 172.21.15.47:0/1537194829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d01016d0 0x7f57d0101af0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009ca0 tx=0x7f57c002c0f0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.313+0000 7f57d5600640 1 -- 172.21.15.47:0/1537194829 shutdown_connections 2024-03-21T11:34:04.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.313+0000 7f57d5600640 1 --2- 172.21.15.47:0/1537194829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57d0103280 0x7f57d0109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.313+0000 7f57d5600640 1 --2- 172.21.15.47:0/1537194829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d01028c0 0x7f57d0102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.313+0000 7f57d5600640 1 --2- 172.21.15.47:0/1537194829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d01016d0 0x7f57d0101af0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.313+0000 7f57d5600640 1 -- 172.21.15.47:0/1537194829 >> 172.21.15.47:0/1537194829 conn(0x7f57d00fcc40 msgr2=0x7f57d00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:04.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.313+0000 7f57d5600640 1 -- 172.21.15.47:0/1537194829 shutdown_connections 2024-03-21T11:34:04.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.313+0000 7f57d5600640 1 -- 172.21.15.47:0/1537194829 wait complete. 2024-03-21T11:34:04.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.314+0000 7f57d5600640 1 Processor -- start 2024-03-21T11:34:04.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.314+0000 7f57d5600640 1 -- start start 2024-03-21T11:34:04.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57d5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d01016d0 0x7f57d0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57d5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d01028c0 0x7f57d0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57d5600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57d0103280 0x7f57d019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57d5600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d010d340 con 0x7f57d0103280 2024-03-21T11:34:04.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57d5600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d010d1c0 con 0x7f57d01028c0 2024-03-21T11:34:04.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57d5600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d010d4c0 con 0x7f57d01016d0 2024-03-21T11:34:04.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57cf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57d0103280 0x7f57d019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57cea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d01016d0 0x7f57d0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57cea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d01016d0 0x7f57d0197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54982/0 (socket says 172.21.15.47:54982) 2024-03-21T11:34:04.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57cea00640 1 -- 172.21.15.47:0/1045890655 learned_addr learned my addr 172.21.15.47:0/1045890655 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:04.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57ce000640 1 --2- 172.21.15.47:0/1045890655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d01028c0 0x7f57d0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57cea00640 1 -- 172.21.15.47:0/1045890655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d01028c0 msgr2=0x7f57d0197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57cea00640 1 --2- 172.21.15.47:0/1045890655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d01028c0 0x7f57d0197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57cea00640 1 -- 172.21.15.47:0/1045890655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57d0103280 msgr2=0x7f57d019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57cea00640 1 --2- 172.21.15.47:0/1045890655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57d0103280 0x7f57d019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57cea00640 1 -- 172.21.15.47:0/1045890655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d019f640 con 0x7f57d01016d0 2024-03-21T11:34:04.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.315+0000 7f57cf400640 1 --2- 172.21.15.47:0/1045890655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57d0103280 0x7f57d019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:04.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.316+0000 7f57cea00640 1 --2- 172.21.15.47:0/1045890655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d01016d0 0x7f57d0197a20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f57c0000f80 tx=0x7f57c0032ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:04.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.316+0000 7f57c7400640 1 -- 172.21.15.47:0/1045890655 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0041020 con 0x7f57d01016d0 2024-03-21T11:34:04.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.316+0000 7f57c7400640 1 -- 172.21.15.47:0/1045890655 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57c0034cd0 con 0x7f57d01016d0 2024-03-21T11:34:04.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.316+0000 7f57d5600640 1 -- 172.21.15.47:0/1045890655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d019f8d0 con 0x7f57d01016d0 2024-03-21T11:34:04.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.316+0000 7f57c7400640 1 -- 172.21.15.47:0/1045890655 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0039040 con 0x7f57d01016d0 2024-03-21T11:34:04.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.316+0000 7f57d5600640 1 -- 172.21.15.47:0/1045890655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d019fd90 con 0x7f57d01016d0 2024-03-21T11:34:04.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.319+0000 7f57d8238740 1 Processor -- start 2024-03-21T11:34:04.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.320+0000 7f57d8238740 1 -- start start 2024-03-21T11:34:04.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.320+0000 7f57c7400640 1 -- 172.21.15.47:0/1045890655 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f57c0044020 con 0x7f57d01016d0 2024-03-21T11:34:04.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.320+0000 7f57d8238740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72d5290 0x55c2f72d56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.320+0000 7f57d8238740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72da780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.320+0000 7f57d8238740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72dadf0 0x55c2f72dd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.320+0000 7f57d8238740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f7236250 con 0x55c2f72dadf0 2024-03-21T11:34:04.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.320+0000 7f57d8238740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f71dea30 con 0x55c2f72d5c80 2024-03-21T11:34:04.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.320+0000 7f57d8238740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f72322f0 con 0x55c2f72d5290 2024-03-21T11:34:04.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.320+0000 7f57cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72da780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.320+0000 7f57cf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72d5290 0x55c2f72d56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.320+0000 7f57ce000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72dadf0 0x55c2f72dd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.320+0000 7f57cf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72d5290 0x55c2f72d56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54986/0 (socket says 172.21.15.47:54986) 2024-03-21T11:34:04.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.320+0000 7f57cf400640 1 -- 172.21.15.47:0/1855735972 learned_addr learned my addr 172.21.15.47:0/1855735972 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:04.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.320+0000 7f57cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72da780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36292/0 (socket says 172.21.15.47:36292) 2024-03-21T11:34:04.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.321+0000 7f57cf400640 1 -- 172.21.15.47:0/1855735972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 msgr2=0x55c2f72da780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.321+0000 7f57cf400640 1 --2- 172.21.15.47:0/1855735972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72da780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.321+0000 7f57cf400640 1 -- 172.21.15.47:0/1855735972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72dadf0 msgr2=0x55c2f72dd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.321+0000 7f57cf400640 1 --2- 172.21.15.47:0/1855735972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72dadf0 0x55c2f72dd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.321+0000 7f57cf400640 1 -- 172.21.15.47:0/1855735972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2f71c9690 con 0x55c2f72d5290 2024-03-21T11:34:04.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.321+0000 7f57cf400640 1 --2- 172.21.15.47:0/1855735972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72d5290 0x55c2f72d56b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f57c8010400 tx=0x7f57c80108c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46fa7ff333e3a4a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:04.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.321+0000 7f57bf400640 1 -- 172.21.15.47:0/1855735972 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8004230 con 0x55c2f72d5290 2024-03-21T11:34:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.321+0000 7f57bf400640 1 -- 172.21.15.47:0/1855735972 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57c80043d0 con 0x55c2f72d5290 2024-03-21T11:34:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.321+0000 7f57bf400640 1 -- 172.21.15.47:0/1855735972 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8003b00 con 0x55c2f72d5290 2024-03-21T11:34:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.322+0000 7f57c7400640 1 --2- 172.21.15.47:0/1045890655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f57b40a2f20 0x7f57b40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.322+0000 7f57c7400640 1 -- 172.21.15.47:0/1045890655 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f57c00f15f0 con 0x7f57d01016d0 2024-03-21T11:34:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.322+0000 7f57cea00640 1 --2- 172.21.15.47:0/1045890655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f57b40a2f20 0x7f57b40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.322+0000 7f57d8238740 1 -- 172.21.15.47:0/1855735972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72d5290 msgr2=0x55c2f72d56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.322+0000 7f57d8238740 1 --2- 172.21.15.47:0/1855735972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72d5290 0x55c2f72d56b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f57c8010400 tx=0x7f57c80108c0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.322+0000 7f57d8238740 1 -- 172.21.15.47:0/1855735972 shutdown_connections 2024-03-21T11:34:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.322+0000 7f57d8238740 1 --2- 172.21.15.47:0/1855735972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72dadf0 0x55c2f72dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.322+0000 7f57d8238740 1 --2- 172.21.15.47:0/1855735972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72da780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.322+0000 7f57d8238740 1 --2- 172.21.15.47:0/1855735972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72d5290 0x55c2f72d56b0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.322+0000 7f57d8238740 1 -- 172.21.15.47:0/1855735972 >> 172.21.15.47:0/1855735972 conn(0x55c2f7299200 msgr2=0x55c2f72cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.323+0000 7f57d8238740 1 -- 172.21.15.47:0/1855735972 shutdown_connections 2024-03-21T11:34:04.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.323+0000 7f57d8238740 1 -- 172.21.15.47:0/1855735972 wait complete. 2024-03-21T11:34:04.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.323+0000 7f57cea00640 1 --2- 172.21.15.47:0/1045890655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f57b40a2f20 0x7f57b40a53e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x55c2f72d6080 tx=0x7f57c00f70e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:04.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.323+0000 7f57d8238740 1 -- start start 2024-03-21T11:34:04.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.323+0000 7f57d8238740 1 Processor -- start 2024-03-21T11:34:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.324+0000 7f57d8238740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 0x55c2f72df260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.324+0000 7f57d8238740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72df7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.324+0000 7f57d8238740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72dadf0 0x55c2f72e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.324+0000 7f57d8238740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f72ddda0 con 0x55c2f72d5290 2024-03-21T11:34:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.324+0000 7f57d8238740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f71f1e80 con 0x55c2f72d5c80 2024-03-21T11:34:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.324+0000 7f57d8238740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f72ddf20 con 0x55c2f72dadf0 2024-03-21T11:34:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.324+0000 7f57d8238740 10 client.0 _ll_register_callbacks cb 0x55c2f7204130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.324+0000 7f57ce000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72df7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.324+0000 7f57ce000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72df7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36306/0 (socket says 172.21.15.47:36306) 2024-03-21T11:34:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.324+0000 7f57ce000640 1 -- 172.21.15.47:0/593495420 learned_addr learned my addr 172.21.15.47:0/593495420 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:04.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.324+0000 7f57cf400640 1 --2- 172.21.15.47:0/593495420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 0x55c2f72df260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.325+0000 7f57cf400640 1 --2- 172.21.15.47:0/593495420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72dadf0 0x55c2f72e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.325+0000 7f57ce000640 1 -- 172.21.15.47:0/593495420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72dadf0 msgr2=0x55c2f72e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.325+0000 7f57ce000640 1 --2- 172.21.15.47:0/593495420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72dadf0 0x55c2f72e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.325+0000 7f57ce000640 1 -- 172.21.15.47:0/593495420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 msgr2=0x55c2f72df260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.325+0000 7f57ce000640 1 --2- 172.21.15.47:0/593495420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 0x55c2f72df260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.325+0000 7f57ce000640 1 -- 172.21.15.47:0/593495420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2f70147d0 con 0x55c2f72d5c80 2024-03-21T11:34:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.325+0000 7f57cf400640 1 --2- 172.21.15.47:0/593495420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72dadf0 0x55c2f72e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:34:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.325+0000 7f57ce000640 1 --2- 172.21.15.47:0/593495420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72df7a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f57b80126e0 tx=0x7f57b8012ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa4fdac6380ca3c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.325+0000 7f57b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.325+0000 7f57b3e00640 1 -- 172.21.15.47:0/593495420 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b800f920 con 0x55c2f72d5c80 2024-03-21T11:34:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.325+0000 7f57d8238740 1 -- 172.21.15.47:0/593495420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57c80043d0 con 0x55c2f72d5c80 2024-03-21T11:34:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.325+0000 7f57b3e00640 1 -- 172.21.15.47:0/593495420 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f57b800fac0 con 0x55c2f72d5c80 2024-03-21T11:34:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.326+0000 7f57b3e00640 1 -- 172.21.15.47:0/593495420 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57b800fcc0 con 0x55c2f72d5c80 2024-03-21T11:34:04.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.326+0000 7f57d8238740 10 client.29156 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.326+0000 7f57d8238740 1 -- 172.21.15.47:0/593495420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c2f6f4a3f0 con 0x55c2f72d5c80 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.326+0000 7f57b3e00640 1 -- 172.21.15.47:0/593495420 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57c80043d0 con 0x55c2f72d5c80 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.326+0000 7f57b3e00640 1 -- 172.21.15.47:0/593495420 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f57b801c4b0 con 0x55c2f72d5c80 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.326+0000 7f57d8238740 10 client.29156 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.326+0000 7f57d8238740 10 client.29156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.326+0000 7f57d8238740 20 client.29156 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.326+0000 7f57d8238740 1 --2- 172.21.15.47:0/593495420 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c2f72eb000 0x55c2f72ed460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.326+0000 7f57d8238740 4 client.29156 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.326+0000 7f57d8238740 1 -- 172.21.15.47:0/593495420 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2f7243790 con 0x55c2f72eb000 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.327+0000 7f57cf400640 1 --2- 172.21.15.47:0/593495420 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c2f72eb000 0x55c2f72ed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.327+0000 7f57cf400640 1 --2- 172.21.15.47:0/593495420 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c2f72eb000 0x55c2f72ed460 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8613cc9ce36f8fcf server_cookie=dff40bbd1e90d37d in_seq=0 out_seq=0 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.327+0000 7f57b3e00640 10 client.29156 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.328+0000 7f57b3e00640 1 -- 172.21.15.47:0/593495420 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c2f7243790 con 0x55c2f72eb000 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.328+0000 7f57b3e00640 10 client.29156 handle_command_reply: tid=0 2024-03-21T11:34:04.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.328+0000 7f57d8238740 1 client.29156 shutdown 2024-03-21T11:34:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.328+0000 7f57d8238740 1 -- 172.21.15.47:0/593495420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 msgr2=0x55c2f72df7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.328+0000 7f57d8238740 1 --2- 172.21.15.47:0/593495420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72df7a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f57b80126e0 tx=0x7f57b8012ba0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.328+0000 7f57d8238740 1 -- 172.21.15.47:0/593495420 shutdown_connections 2024-03-21T11:34:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.328+0000 7f57d8238740 1 --2- 172.21.15.47:0/593495420 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c2f72eb000 0x55c2f72ed460 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.329+0000 7f57d8238740 1 --2- 172.21.15.47:0/593495420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72dadf0 0x55c2f72e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.329+0000 7f57d8238740 1 --2- 172.21.15.47:0/593495420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72df7a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.329+0000 7f57d8238740 1 --2- 172.21.15.47:0/593495420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 0x55c2f72df260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.329+0000 7f57b3e00640 0 client.29156 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.329+0000 7f57d8238740 1 -- 172.21.15.47:0/593495420 >> 172.21.15.47:0/593495420 conn(0x55c2f7299200 msgr2=0x55c2f72cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.329+0000 7f57d8238740 1 -- 172.21.15.47:0/593495420 shutdown_connections 2024-03-21T11:34:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.329+0000 7f57d8238740 1 -- 172.21.15.47:0/593495420 wait complete. 2024-03-21T11:34:04.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.329+0000 7f57d8238740 20 client.29156 trim_cache size 0 max 16384 2024-03-21T11:34:04.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.348+0000 7f57d8238740 1 Processor -- start 2024-03-21T11:34:04.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.348+0000 7f57d8238740 1 -- start start 2024-03-21T11:34:04.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.348+0000 7f57d8238740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 0x55c2f72cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.348+0000 7f57d8238740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72e67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.348+0000 7f57d8238740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72dadf0 0x55c2f72d6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.348+0000 7f57d8238740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d01a7e80 con 0x55c2f72d5290 2024-03-21T11:34:04.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.348+0000 7f57d8238740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d01a9800 con 0x55c2f72d5c80 2024-03-21T11:34:04.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.348+0000 7f57d8238740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d01a84e0 con 0x55c2f72dadf0 2024-03-21T11:34:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.348+0000 7f57cf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 0x55c2f72cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.348+0000 7f57ce000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72e67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.348+0000 7f57ce000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72e67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36318/0 (socket says 172.21.15.47:36318) 2024-03-21T11:34:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.348+0000 7f57ce000640 1 -- 172.21.15.47:0/2684201384 learned_addr learned my addr 172.21.15.47:0/2684201384 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.349+0000 7f57cf400640 1 --2- 172.21.15.47:0/2684201384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72dadf0 0x55c2f72d6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.349+0000 7f57ce000640 1 -- 172.21.15.47:0/2684201384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72dadf0 msgr2=0x55c2f72d6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.349+0000 7f57ce000640 1 --2- 172.21.15.47:0/2684201384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72dadf0 0x55c2f72d6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.349+0000 7f57ce000640 1 -- 172.21.15.47:0/2684201384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 msgr2=0x55c2f72cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.349+0000 7f57ce000640 1 --2- 172.21.15.47:0/2684201384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 0x55c2f72cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.349+0000 7f57ce000640 1 -- 172.21.15.47:0/2684201384 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2f6f0a0f0 con 0x55c2f72d5c80 2024-03-21T11:34:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.349+0000 7f57ce000640 1 --2- 172.21.15.47:0/2684201384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72e67a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f57b8000c00 tx=0x7f57b800f810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=364b86d0d7d2a668 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:04.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.350+0000 7f57bea00640 1 -- 172.21.15.47:0/2684201384 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b802f070 con 0x55c2f72d5c80 2024-03-21T11:34:04.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.350+0000 7f57bea00640 1 -- 172.21.15.47:0/2684201384 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57b801cbf0 con 0x55c2f72d5c80 2024-03-21T11:34:04.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.350+0000 7f57bea00640 1 -- 172.21.15.47:0/2684201384 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8010c20 con 0x55c2f72d5c80 2024-03-21T11:34:04.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.350+0000 7f57d8238740 1 -- 172.21.15.47:0/2684201384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 msgr2=0x55c2f72e67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.350+0000 7f57d8238740 1 --2- 172.21.15.47:0/2684201384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72e67a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f57b8000c00 tx=0x7f57b800f810 comp rx=0 tx=0).stop 2024-03-21T11:34:04.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.350+0000 7f57d8238740 1 -- 172.21.15.47:0/2684201384 shutdown_connections 2024-03-21T11:34:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.350+0000 7f57d8238740 1 --2- 172.21.15.47:0/2684201384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72dadf0 0x55c2f72d6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.350+0000 7f57d8238740 1 --2- 172.21.15.47:0/2684201384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72d5c80 0x55c2f72e67a0 secure :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f57b8000c00 tx=0x7f57b800f810 comp rx=0 tx=0).stop 2024-03-21T11:34:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.350+0000 7f57d8238740 1 --2- 172.21.15.47:0/2684201384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 0x55c2f72cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.350+0000 7f57d8238740 1 -- 172.21.15.47:0/2684201384 >> 172.21.15.47:0/2684201384 conn(0x55c2f7299200 msgr2=0x55c2f72d6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.351+0000 7f57d8238740 1 -- 172.21.15.47:0/2684201384 shutdown_connections 2024-03-21T11:34:04.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.351+0000 7f57d8238740 1 -- 172.21.15.47:0/2684201384 wait complete. 2024-03-21T11:34:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.351+0000 7f57d8238740 1 -- start start 2024-03-21T11:34:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.351+0000 7f57d8238740 1 Processor -- start 2024-03-21T11:34:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.352+0000 7f57d8238740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 0x55c2f72e4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.352+0000 7f57d8238740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72dadf0 0x55c2f72e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.352+0000 7f57d8238740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72e59b0 0x55c2f72e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.352+0000 7f57d8238740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d01a91a0 con 0x55c2f72d5290 2024-03-21T11:34:04.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.352+0000 7f57d8238740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d01a8b40 con 0x55c2f72dadf0 2024-03-21T11:34:04.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.352+0000 7f57d8238740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d01a6040 con 0x55c2f72e59b0 2024-03-21T11:34:04.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.352+0000 7f57d8238740 10 client.0 _ll_register_callbacks cb 0x7f57b8009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:04.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.352+0000 7f57ce000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72dadf0 0x55c2f72e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.352+0000 7f57cf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 0x55c2f72e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.352+0000 7f57ce000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72dadf0 0x55c2f72e49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36334/0 (socket says 172.21.15.47:36334) 2024-03-21T11:34:04.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.352+0000 7f57ce000640 1 -- 172.21.15.47:0/605623813 learned_addr learned my addr 172.21.15.47:0/605623813 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.352+0000 7f57cf400640 1 --2- 172.21.15.47:0/605623813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72e59b0 0x55c2f72e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.353+0000 7f57ce000640 1 -- 172.21.15.47:0/605623813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72e59b0 msgr2=0x55c2f72e5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.353+0000 7f57ce000640 1 --2- 172.21.15.47:0/605623813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72e59b0 0x55c2f72e5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.353+0000 7f57ce000640 1 -- 172.21.15.47:0/605623813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 msgr2=0x55c2f72e4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.353+0000 7f57ce000640 1 --2- 172.21.15.47:0/605623813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 0x55c2f72e4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.353+0000 7f57ce000640 1 -- 172.21.15.47:0/605623813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2f72e6370 con 0x55c2f72dadf0 2024-03-21T11:34:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.353+0000 7f57ce000640 1 --2- 172.21.15.47:0/605623813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72dadf0 0x55c2f72e49a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f57b801c8c0 tx=0x7f57b8011e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=289f95e467d5ee85 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.353+0000 7f57b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.353+0000 7f57b3e00640 1 -- 172.21.15.47:0/605623813 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b802f070 con 0x55c2f72dadf0 2024-03-21T11:34:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.353+0000 7f57b3e00640 1 -- 172.21.15.47:0/605623813 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f57b80109a0 con 0x55c2f72dadf0 2024-03-21T11:34:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.353+0000 7f57d8238740 1 -- 172.21.15.47:0/605623813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57b801cbf0 con 0x55c2f72dadf0 2024-03-21T11:34:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.354+0000 7f57b3e00640 1 -- 172.21.15.47:0/605623813 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57b8005320 con 0x55c2f72dadf0 2024-03-21T11:34:04.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.354+0000 7f57d8238740 10 client.29162 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.354+0000 7f57d8238740 1 -- 172.21.15.47:0/605623813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c2f73472a0 con 0x55c2f72dadf0 2024-03-21T11:34:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.354+0000 7f57b3e00640 1 -- 172.21.15.47:0/605623813 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57b801cbf0 con 0x55c2f72dadf0 2024-03-21T11:34:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.354+0000 7f57b3e00640 1 -- 172.21.15.47:0/605623813 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f57b80038c0 con 0x55c2f72dadf0 2024-03-21T11:34:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.354+0000 7f57d8238740 10 client.29162 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.354+0000 7f57d8238740 10 client.29162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:04.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.354+0000 7f57d8238740 20 client.29162 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:04.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.354+0000 7f57d8238740 1 --2- 172.21.15.47:0/605623813 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c2f7347b50 0x55c2f7349f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:04.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.354+0000 7f57d8238740 4 client.29162 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:34:04.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.354+0000 7f57d8238740 1 -- 172.21.15.47:0/605623813 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c2f723dae0 con 0x55c2f7347b50 2024-03-21T11:34:04.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.354+0000 7f57cf400640 1 --2- 172.21.15.47:0/605623813 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c2f7347b50 0x55c2f7349f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:04.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.355+0000 7f57cf400640 1 --2- 172.21.15.47:0/605623813 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c2f7347b50 0x55c2f7349f70 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bacc9034acef17e server_cookie=28c5791d560f8680 in_seq=0 out_seq=0 2024-03-21T11:34:04.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.355+0000 7f57b3e00640 10 client.29162 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:04.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.357+0000 7f57b3e00640 1 -- 172.21.15.47:0/605623813 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c2f723dae0 con 0x55c2f7347b50 2024-03-21T11:34:04.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.357+0000 7f57b3e00640 10 client.29162 handle_command_reply: tid=0 2024-03-21T11:34:04.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.357+0000 7f57d8238740 1 client.29162 shutdown 2024-03-21T11:34:04.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.357+0000 7f57d8238740 1 -- 172.21.15.47:0/605623813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72dadf0 msgr2=0x55c2f72e49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.357+0000 7f57d8238740 1 --2- 172.21.15.47:0/605623813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72dadf0 0x55c2f72e49a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f57b801c8c0 tx=0x7f57b8011e80 comp rx=0 tx=0).stop 2024-03-21T11:34:04.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.357+0000 7f57d8238740 1 -- 172.21.15.47:0/605623813 shutdown_connections 2024-03-21T11:34:04.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.357+0000 7f57d8238740 1 --2- 172.21.15.47:0/605623813 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c2f7347b50 0x55c2f7349f70 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.357+0000 7f57d8238740 1 --2- 172.21.15.47:0/605623813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c2f72e59b0 0x55c2f72e5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.357+0000 7f57d8238740 1 --2- 172.21.15.47:0/605623813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2f72dadf0 0x55c2f72e49a0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.357+0000 7f57d8238740 1 --2- 172.21.15.47:0/605623813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c2f72d5290 0x55c2f72e4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.357+0000 7f57d8238740 1 -- 172.21.15.47:0/605623813 >> 172.21.15.47:0/605623813 conn(0x55c2f7299200 msgr2=0x55c2f72d6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:04.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.357+0000 7f57b3e00640 0 client.29162 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:04.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.357+0000 7f57d8238740 1 -- 172.21.15.47:0/605623813 shutdown_connections 2024-03-21T11:34:04.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.357+0000 7f57d8238740 1 -- 172.21.15.47:0/605623813 wait complete. 2024-03-21T11:34:04.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.358+0000 7f57d8238740 20 client.29162 trim_cache size 0 max 16384 2024-03-21T11:34:04.367 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:34:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:34:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:34:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "125057cb-93f4-4771-895a-7be62ef4b900": { 2024-03-21T11:34:04.367 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:34:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "125057cb-93f4-4771-895a-7be62ef4b900", 2024-03-21T11:34:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:34:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:34:04.368 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:34:04.368 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:34:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.358+0000 7f57d5600640 1 -- 172.21.15.47:0/1045890655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f57b40a2f20 msgr2=0x7f57b40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.358+0000 7f57d5600640 1 --2- 172.21.15.47:0/1045890655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f57b40a2f20 0x7f57b40a53e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x55c2f72d6080 tx=0x7f57c00f70e0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.358+0000 7f57d5600640 1 -- 172.21.15.47:0/1045890655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d01016d0 msgr2=0x7f57d0197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.358+0000 7f57d5600640 1 --2- 172.21.15.47:0/1045890655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d01016d0 0x7f57d0197a20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f57c0000f80 tx=0x7f57c0032ec0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.358+0000 7f57d5600640 1 -- 172.21.15.47:0/1045890655 shutdown_connections 2024-03-21T11:34:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.358+0000 7f57d5600640 1 --2- 172.21.15.47:0/1045890655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f57b40a2f20 0x7f57b40a53e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.358+0000 7f57d5600640 1 --2- 172.21.15.47:0/1045890655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57d0103280 0x7f57d019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.358+0000 7f57d5600640 1 --2- 172.21.15.47:0/1045890655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d01028c0 0x7f57d0197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.358+0000 7f57d5600640 1 --2- 172.21.15.47:0/1045890655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57d01016d0 0x7f57d0197a20 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.358+0000 7f57d5600640 1 -- 172.21.15.47:0/1045890655 >> 172.21.15.47:0/1045890655 conn(0x7f57d00fcc40 msgr2=0x7f57d0105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.359+0000 7f57d5600640 1 -- 172.21.15.47:0/1045890655 shutdown_connections 2024-03-21T11:34:04.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:04.359+0000 7f57d5600640 1 -- 172.21.15.47:0/1045890655 wait complete. 2024-03-21T11:34:04.376 INFO:tasks.cephfs.filesystem:scrub status for tag:125057cb-93f4-4771-895a-7be62ef4b900 - {'path': '/', 'tag': '125057cb-93f4-4771-895a-7be62ef4b900', 'options': 'recursive,force'} 2024-03-21T11:34:04.555 INFO:tasks.workunit.client.0.smithi047.stdout: 1 802369 8.07 MB/sec execute 408 sec latency 13.088 ms 2024-03-21T11:34:04.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:04 smithi102 ceph-mon[36748]: pgmap v722: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 318 op/s 2024-03-21T11:34:04.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:04 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:34:04.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:04 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:34:04.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:04 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:34:04.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:04 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:34:04.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:04 smithi047 ceph-mon[31351]: pgmap v722: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 318 op/s 2024-03-21T11:34:04.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:04 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:34:04.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:04 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:34:04.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:04 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:34:04.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:04 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:34:04.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:04 smithi100 ceph-mon[36245]: pgmap v722: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 318 op/s 2024-03-21T11:34:04.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:04 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:34:04.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:04 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:34:04.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:04 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:34:04.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:04 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:34:05.555 INFO:tasks.workunit.client.0.smithi047.stdout: 1 803090 8.05 MB/sec execute 409 sec latency 44.642 ms 2024-03-21T11:34:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.048+0000 7fc957600640 1 -- 172.21.15.47:0/4073313597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc950107d00 msgr2=0x7fc950108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.048+0000 7fc957600640 1 --2- 172.21.15.47:0/4073313597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc950107d00 0x7fc950108120 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc940009900 tx=0x7fc94002bc40 comp rx=0 tx=0).stop 2024-03-21T11:34:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.048+0000 7fc957600640 1 -- 172.21.15.47:0/4073313597 shutdown_connections 2024-03-21T11:34:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.048+0000 7fc957600640 1 --2- 172.21.15.47:0/4073313597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9501098b0 0x7fc950110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.048+0000 7fc957600640 1 --2- 172.21.15.47:0/4073313597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc950108ef0 0x7fc950109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.048+0000 7fc957600640 1 --2- 172.21.15.47:0/4073313597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc950107d00 0x7fc950108120 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.048+0000 7fc957600640 1 -- 172.21.15.47:0/4073313597 >> 172.21.15.47:0/4073313597 conn(0x7fc950076c90 msgr2=0x7fc9500790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.049+0000 7fc957600640 1 -- 172.21.15.47:0/4073313597 shutdown_connections 2024-03-21T11:34:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.049+0000 7fc957600640 1 -- 172.21.15.47:0/4073313597 wait complete. 2024-03-21T11:34:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.050+0000 7fc957600640 1 Processor -- start 2024-03-21T11:34:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.050+0000 7fc957600640 1 -- start start 2024-03-21T11:34:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.050+0000 7fc957600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc950107d00 0x7fc95019df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.050+0000 7fc957600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc950108ef0 0x7fc95019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.050+0000 7fc957600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9501098b0 0x7fc9501a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.050+0000 7fc957600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9501139a0 con 0x7fc950108ef0 2024-03-21T11:34:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.050+0000 7fc957600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc950113820 con 0x7fc950107d00 2024-03-21T11:34:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.050+0000 7fc957600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc950113b20 con 0x7fc9501098b0 2024-03-21T11:34:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.050+0000 7fc954c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc950107d00 0x7fc95019df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.050+0000 7fc954c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc950107d00 0x7fc95019df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36354/0 (socket says 172.21.15.47:36354) 2024-03-21T11:34:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.050+0000 7fc955600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9501098b0 0x7fc9501a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.050+0000 7fc954c00640 1 -- 172.21.15.47:0/3282460756 learned_addr learned my addr 172.21.15.47:0/3282460756 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.050+0000 7fc947e00640 1 --2- 172.21.15.47:0/3282460756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc950108ef0 0x7fc95019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.051+0000 7fc955600640 1 -- 172.21.15.47:0/3282460756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc950107d00 msgr2=0x7fc95019df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.051+0000 7fc955600640 1 --2- 172.21.15.47:0/3282460756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc950107d00 0x7fc95019df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.051+0000 7fc955600640 1 -- 172.21.15.47:0/3282460756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc950108ef0 msgr2=0x7fc95019e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.051+0000 7fc955600640 1 --2- 172.21.15.47:0/3282460756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc950108ef0 0x7fc95019e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.051+0000 7fc955600640 1 -- 172.21.15.47:0/3282460756 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9501a5b50 con 0x7fc9501098b0 2024-03-21T11:34:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.051+0000 7fc955600640 1 --2- 172.21.15.47:0/3282460756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9501098b0 0x7fc9501a54e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc948009fd0 tx=0x7fc94800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.052+0000 7fc94e000640 1 -- 172.21.15.47:0/3282460756 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800ce20 con 0x7fc9501098b0 2024-03-21T11:34:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.052+0000 7fc94e000640 1 -- 172.21.15.47:0/3282460756 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc948005020 con 0x7fc9501098b0 2024-03-21T11:34:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.052+0000 7fc94e000640 1 -- 172.21.15.47:0/3282460756 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948010720 con 0x7fc9501098b0 2024-03-21T11:34:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.052+0000 7fc957600640 1 -- 172.21.15.47:0/3282460756 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9501a5e40 con 0x7fc9501098b0 2024-03-21T11:34:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.052+0000 7fc957600640 1 -- 172.21.15.47:0/3282460756 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9501a62b0 con 0x7fc9501098b0 2024-03-21T11:34:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.055+0000 7fc94e000640 1 -- 172.21.15.47:0/3282460756 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc948005540 con 0x7fc9501098b0 2024-03-21T11:34:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.055+0000 7fc95a22b740 1 Processor -- start 2024-03-21T11:34:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.056+0000 7fc95a22b740 1 -- start start 2024-03-21T11:34:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.056+0000 7fc95a22b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 0x55d70f13d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.056+0000 7fc95a22b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f13dbd0 0x55d70f1426d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.056+0000 7fc95a22b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f142d40 0x55d70f1451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.056+0000 7fc95a22b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70f1003b0 con 0x55d70f13dbd0 2024-03-21T11:34:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.056+0000 7fc95a22b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70f0a2830 con 0x55d70f142d40 2024-03-21T11:34:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.056+0000 7fc95a22b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70f04b5d0 con 0x55d70f13d1e0 2024-03-21T11:34:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.056+0000 7fc954c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f13dbd0 0x55d70f1426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.056+0000 7fc954c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f13dbd0 0x55d70f1426d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55588/0 (socket says 172.21.15.47:55588) 2024-03-21T11:34:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.056+0000 7fc954c00640 1 -- 172.21.15.47:0/4008874167 learned_addr learned my addr 172.21.15.47:0/4008874167 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.056+0000 7fc947e00640 1 --2- 172.21.15.47:0/4008874167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 0x55d70f13d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.057+0000 7fc947e00640 1 --2- 172.21.15.47:0/4008874167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f142d40 0x55d70f1451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.057+0000 7fc94e000640 1 --2- 172.21.15.47:0/3282460756 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9340a2fc0 0x7fc9340a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.057+0000 7fc954c00640 1 -- 172.21.15.47:0/4008874167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 msgr2=0x55d70f13d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.057+0000 7fc954c00640 1 --2- 172.21.15.47:0/4008874167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 0x55d70f13d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.057+0000 7fc954c00640 1 -- 172.21.15.47:0/4008874167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f142d40 msgr2=0x55d70f1451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.057+0000 7fc954c00640 1 --2- 172.21.15.47:0/4008874167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f142d40 0x55d70f1451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.057+0000 7fc954c00640 1 -- 172.21.15.47:0/4008874167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d70f0632c0 con 0x55d70f13dbd0 2024-03-21T11:34:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.057+0000 7fc947e00640 1 --2- 172.21.15.47:0/4008874167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f142d40 0x55d70f1451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.057+0000 7fc94e000640 1 -- 172.21.15.47:0/3282460756 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc9480cfea0 con 0x7fc9501098b0 2024-03-21T11:34:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.057+0000 7fc955600640 1 --2- 172.21.15.47:0/3282460756 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9340a2fc0 0x7fc9340a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.058+0000 7fc954c00640 1 --2- 172.21.15.47:0/4008874167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f13dbd0 0x55d70f1426d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc94002c180 tx=0x7fc940000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce5401e6a17f9dc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.058+0000 7fc945c00640 1 -- 172.21.15.47:0/4008874167 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940042070 con 0x55d70f13dbd0 2024-03-21T11:34:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.058+0000 7fc955600640 1 --2- 172.21.15.47:0/3282460756 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9340a2fc0 0x7fc9340a5480 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc94801a040 tx=0x7fc9480054b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.058+0000 7fc945c00640 1 -- 172.21.15.47:0/4008874167 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc94002cb90 con 0x55d70f13dbd0 2024-03-21T11:34:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.058+0000 7fc945c00640 1 -- 172.21.15.47:0/4008874167 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940005150 con 0x55d70f13dbd0 2024-03-21T11:34:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.059+0000 7fc95a22b740 1 -- 172.21.15.47:0/4008874167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f13dbd0 msgr2=0x55d70f1426d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.059+0000 7fc95a22b740 1 --2- 172.21.15.47:0/4008874167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f13dbd0 0x55d70f1426d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc94002c180 tx=0x7fc940000bc0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.059+0000 7fc95a22b740 1 -- 172.21.15.47:0/4008874167 shutdown_connections 2024-03-21T11:34:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.059+0000 7fc95a22b740 1 --2- 172.21.15.47:0/4008874167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f142d40 0x55d70f1451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.059+0000 7fc95a22b740 1 --2- 172.21.15.47:0/4008874167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f13dbd0 0x55d70f1426d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.059+0000 7fc95a22b740 1 --2- 172.21.15.47:0/4008874167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 0x55d70f13d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.059+0000 7fc95a22b740 1 -- 172.21.15.47:0/4008874167 >> 172.21.15.47:0/4008874167 conn(0x55d70f138750 msgr2=0x55d70f138b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.059+0000 7fc95a22b740 1 -- 172.21.15.47:0/4008874167 shutdown_connections 2024-03-21T11:34:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.059+0000 7fc95a22b740 1 -- 172.21.15.47:0/4008874167 wait complete. 2024-03-21T11:34:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.059+0000 7fc95a22b740 1 -- start start 2024-03-21T11:34:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.060+0000 7fc95a22b740 1 Processor -- start 2024-03-21T11:34:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.060+0000 7fc95a22b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 0x55d70f1473c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.060+0000 7fc95a22b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f13dbd0 0x55d70f147900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.060+0000 7fc95a22b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f142d40 0x55d70f14e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.060+0000 7fc95a22b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70f05ea20 con 0x55d70f142d40 2024-03-21T11:34:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.060+0000 7fc95a22b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70f09e8d0 con 0x55d70f13dbd0 2024-03-21T11:34:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.060+0000 7fc95a22b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70f145c90 con 0x55d70f13d1e0 2024-03-21T11:34:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.060+0000 7fc95a22b740 10 client.0 _ll_register_callbacks cb 0x55d70f09b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.060+0000 7fc954c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f13dbd0 0x55d70f147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.060+0000 7fc954c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f13dbd0 0x55d70f147900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36372/0 (socket says 172.21.15.47:36372) 2024-03-21T11:34:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.060+0000 7fc954c00640 1 -- 172.21.15.47:0/1182271068 learned_addr learned my addr 172.21.15.47:0/1182271068 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.060+0000 7fc947e00640 1 --2- 172.21.15.47:0/1182271068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 0x55d70f1473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.060+0000 7fc947e00640 1 --2- 172.21.15.47:0/1182271068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f142d40 0x55d70f14e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.061+0000 7fc954c00640 1 -- 172.21.15.47:0/1182271068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 msgr2=0x55d70f1473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.061+0000 7fc954c00640 1 --2- 172.21.15.47:0/1182271068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 0x55d70f1473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.061+0000 7fc954c00640 1 -- 172.21.15.47:0/1182271068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f142d40 msgr2=0x55d70f14e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.061+0000 7fc954c00640 1 --2- 172.21.15.47:0/1182271068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f142d40 0x55d70f14e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.061+0000 7fc954c00640 1 -- 172.21.15.47:0/1182271068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d70edb7210 con 0x55d70f13dbd0 2024-03-21T11:34:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.061+0000 7fc947e00640 1 --2- 172.21.15.47:0/1182271068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f142d40 0x55d70f14e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.061+0000 7fc954c00640 1 --2- 172.21.15.47:0/1182271068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f13dbd0 0x55d70f147900 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc940036040 tx=0x7fc940005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f075218f75cc8a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.061+0000 7fc93e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.061+0000 7fc93e000640 1 -- 172.21.15.47:0/1182271068 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940042040 con 0x55d70f13dbd0 2024-03-21T11:34:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.061+0000 7fc95a22b740 1 -- 172.21.15.47:0/1182271068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc94002cb90 con 0x55d70f13dbd0 2024-03-21T11:34:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.061+0000 7fc93e000640 1 -- 172.21.15.47:0/1182271068 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc940004ef0 con 0x55d70f13dbd0 2024-03-21T11:34:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.062+0000 7fc93e000640 1 -- 172.21.15.47:0/1182271068 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc94002cd40 con 0x55d70f13dbd0 2024-03-21T11:34:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.062+0000 7fc95a22b740 10 client.29174 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.062+0000 7fc95a22b740 1 -- 172.21.15.47:0/1182271068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d70ee83620 con 0x55d70f13dbd0 2024-03-21T11:34:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.062+0000 7fc93e000640 1 -- 172.21.15.47:0/1182271068 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc94002cb90 con 0x55d70f13dbd0 2024-03-21T11:34:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.062+0000 7fc93e000640 1 -- 172.21.15.47:0/1182271068 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc940003bc0 con 0x55d70f13dbd0 2024-03-21T11:34:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.062+0000 7fc95a22b740 10 client.29174 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.062+0000 7fc95a22b740 10 client.29174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.062+0000 7fc95a22b740 20 client.29174 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.062+0000 7fc95a22b740 1 --2- 172.21.15.47:0/1182271068 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d70f152dc0 0x55d70f1551b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.062+0000 7fc95a22b740 4 client.29174 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.062+0000 7fc95a22b740 1 -- 172.21.15.47:0/1182271068 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d70f0afd70 con 0x55d70f152dc0 2024-03-21T11:34:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.062+0000 7fc947e00640 1 --2- 172.21.15.47:0/1182271068 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d70f152dc0 0x55d70f1551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.063+0000 7fc947e00640 1 --2- 172.21.15.47:0/1182271068 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d70f152dc0 0x55d70f1551b0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db1dbda38ea6ed02 server_cookie=30046ea525e71f3f in_seq=0 out_seq=0 2024-03-21T11:34:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.063+0000 7fc93e000640 10 client.29174 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.064+0000 7fc93e000640 1 -- 172.21.15.47:0/1182271068 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d70f0afd70 con 0x55d70f152dc0 2024-03-21T11:34:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.064+0000 7fc93e000640 10 client.29174 handle_command_reply: tid=0 2024-03-21T11:34:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.064+0000 7fc95a22b740 1 client.29174 shutdown 2024-03-21T11:34:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.064+0000 7fc95a22b740 1 -- 172.21.15.47:0/1182271068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f13dbd0 msgr2=0x55d70f147900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.064+0000 7fc95a22b740 1 --2- 172.21.15.47:0/1182271068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f13dbd0 0x55d70f147900 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc940036040 tx=0x7fc940005850 comp rx=0 tx=0).stop 2024-03-21T11:34:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.064+0000 7fc95a22b740 1 -- 172.21.15.47:0/1182271068 shutdown_connections 2024-03-21T11:34:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.064+0000 7fc95a22b740 1 --2- 172.21.15.47:0/1182271068 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d70f152dc0 0x55d70f1551b0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.064+0000 7fc95a22b740 1 --2- 172.21.15.47:0/1182271068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f142d40 0x55d70f14e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.064+0000 7fc95a22b740 1 --2- 172.21.15.47:0/1182271068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f13dbd0 0x55d70f147900 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.064+0000 7fc95a22b740 1 --2- 172.21.15.47:0/1182271068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 0x55d70f1473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.064+0000 7fc95a22b740 1 -- 172.21.15.47:0/1182271068 >> 172.21.15.47:0/1182271068 conn(0x55d70f138750 msgr2=0x55d70f137950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.064+0000 7fc93e000640 0 client.29174 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.064+0000 7fc95a22b740 1 -- 172.21.15.47:0/1182271068 shutdown_connections 2024-03-21T11:34:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.065+0000 7fc95a22b740 1 -- 172.21.15.47:0/1182271068 wait complete. 2024-03-21T11:34:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.065+0000 7fc95a22b740 20 client.29174 trim_cache size 0 max 16384 2024-03-21T11:34:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.081+0000 7fc95a22b740 1 Processor -- start 2024-03-21T11:34:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.082+0000 7fc95a22b740 1 -- start start 2024-03-21T11:34:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.082+0000 7fc95a22b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f13d1e0 0x55d70f13ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.082+0000 7fc95a22b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13dbd0 0x55d70f13f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.082+0000 7fc95a22b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f142d40 0x55d70f14e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.082+0000 7fc95a22b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9501b0510 con 0x55d70f142d40 2024-03-21T11:34:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.082+0000 7fc95a22b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9501b1e90 con 0x55d70f13d1e0 2024-03-21T11:34:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.082+0000 7fc95a22b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9501b0b70 con 0x55d70f13dbd0 2024-03-21T11:34:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.082+0000 7fc954c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13dbd0 0x55d70f13f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.082+0000 7fc954c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13dbd0 0x55d70f13f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55054/0 (socket says 172.21.15.47:55054) 2024-03-21T11:34:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.082+0000 7fc954c00640 1 -- 172.21.15.47:0/4179749421 learned_addr learned my addr 172.21.15.47:0/4179749421 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.083+0000 7fc947e00640 1 --2- 172.21.15.47:0/4179749421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f142d40 0x55d70f14e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.083+0000 7fc954c00640 1 -- 172.21.15.47:0/4179749421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f13d1e0 msgr2=0x55d70f13ef50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:34:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.083+0000 7fc954c00640 1 --2- 172.21.15.47:0/4179749421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f13d1e0 0x55d70f13ef50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.083+0000 7fc954c00640 1 -- 172.21.15.47:0/4179749421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f142d40 msgr2=0x55d70f14e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.083+0000 7fc954c00640 1 --2- 172.21.15.47:0/4179749421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f142d40 0x55d70f14e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.083+0000 7fc954c00640 1 -- 172.21.15.47:0/4179749421 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d70f1329c0 con 0x55d70f13dbd0 2024-03-21T11:34:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.083+0000 7fc947e00640 1 --2- 172.21.15.47:0/4179749421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f142d40 0x55d70f14e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:34:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.083+0000 7fc954c00640 1 --2- 172.21.15.47:0/4179749421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13dbd0 0x55d70f13f490 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc940000c00 tx=0x7fc9400043d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ff2664ea66d16cb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.083+0000 7fc945200640 1 -- 172.21.15.47:0/4179749421 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940005d50 con 0x55d70f13dbd0 2024-03-21T11:34:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.083+0000 7fc945200640 1 -- 172.21.15.47:0/4179749421 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc940004a50 con 0x55d70f13dbd0 2024-03-21T11:34:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.083+0000 7fc945200640 1 -- 172.21.15.47:0/4179749421 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940047dc0 con 0x55d70f13dbd0 2024-03-21T11:34:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.084+0000 7fc95a22b740 1 -- 172.21.15.47:0/4179749421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13dbd0 msgr2=0x55d70f13f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.084+0000 7fc95a22b740 1 --2- 172.21.15.47:0/4179749421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13dbd0 0x55d70f13f490 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc940000c00 tx=0x7fc9400043d0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.084+0000 7fc95a22b740 1 -- 172.21.15.47:0/4179749421 shutdown_connections 2024-03-21T11:34:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.084+0000 7fc95a22b740 1 --2- 172.21.15.47:0/4179749421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f142d40 0x55d70f14e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.084+0000 7fc95a22b740 1 --2- 172.21.15.47:0/4179749421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13dbd0 0x55d70f13f490 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.084+0000 7fc95a22b740 1 --2- 172.21.15.47:0/4179749421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f13d1e0 0x55d70f13ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.084+0000 7fc95a22b740 1 -- 172.21.15.47:0/4179749421 >> 172.21.15.47:0/4179749421 conn(0x55d70f138750 msgr2=0x55d70f1473c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.084+0000 7fc95a22b740 1 -- 172.21.15.47:0/4179749421 shutdown_connections 2024-03-21T11:34:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.084+0000 7fc95a22b740 1 -- 172.21.15.47:0/4179749421 wait complete. 2024-03-21T11:34:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.085+0000 7fc95a22b740 1 -- start start 2024-03-21T11:34:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.085+0000 7fc95a22b740 1 Processor -- start 2024-03-21T11:34:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.085+0000 7fc95a22b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 0x55d70f14c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.085+0000 7fc95a22b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f13dbd0 0x55d70f14c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.085+0000 7fc95a22b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f142d40 0x55d70f14d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.085+0000 7fc95a22b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9501b1830 con 0x55d70f13dbd0 2024-03-21T11:34:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.085+0000 7fc95a22b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9501b11d0 con 0x55d70f142d40 2024-03-21T11:34:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.085+0000 7fc95a22b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9501ae6d0 con 0x55d70f13d1e0 2024-03-21T11:34:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.086+0000 7fc95a22b740 10 client.0 _ll_register_callbacks cb 0x7fc938009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.086+0000 7fc954c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f13dbd0 0x55d70f14c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.086+0000 7fc954c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f13dbd0 0x55d70f14c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55612/0 (socket says 172.21.15.47:55612) 2024-03-21T11:34:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.086+0000 7fc954c00640 1 -- 172.21.15.47:0/3564761282 learned_addr learned my addr 172.21.15.47:0/3564761282 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.086+0000 7fc947e00640 1 --2- 172.21.15.47:0/3564761282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 0x55d70f14c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.086+0000 7fc947e00640 1 --2- 172.21.15.47:0/3564761282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f142d40 0x55d70f14d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.086+0000 7fc954c00640 1 -- 172.21.15.47:0/3564761282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 msgr2=0x55d70f14c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.086+0000 7fc954c00640 1 --2- 172.21.15.47:0/3564761282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 0x55d70f14c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.086+0000 7fc954c00640 1 -- 172.21.15.47:0/3564761282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f142d40 msgr2=0x55d70f14d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.086+0000 7fc954c00640 1 --2- 172.21.15.47:0/3564761282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f142d40 0x55d70f14d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.086+0000 7fc954c00640 1 -- 172.21.15.47:0/3564761282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d70f14dce0 con 0x55d70f13dbd0 2024-03-21T11:34:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.086+0000 7fc947e00640 1 --2- 172.21.15.47:0/3564761282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f142d40 0x55d70f14d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.087+0000 7fc954c00640 1 --2- 172.21.15.47:0/3564761282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f13dbd0 0x55d70f14c6e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc94002e040 tx=0x7fc940004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e4ca4707a577b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.087+0000 7fc93e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:34:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.087+0000 7fc93e000640 1 -- 172.21.15.47:0/3564761282 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940060070 con 0x55d70f13dbd0 2024-03-21T11:34:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.087+0000 7fc93e000640 1 -- 172.21.15.47:0/3564761282 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc94002cb90 con 0x55d70f13dbd0 2024-03-21T11:34:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.087+0000 7fc95a22b740 1 -- 172.21.15.47:0/3564761282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc940004a50 con 0x55d70f13dbd0 2024-03-21T11:34:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.087+0000 7fc93e000640 1 -- 172.21.15.47:0/3564761282 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc94006b050 con 0x55d70f13dbd0 2024-03-21T11:34:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.087+0000 7fc95a22b740 10 client.18840 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.087+0000 7fc95a22b740 1 -- 172.21.15.47:0/3564761282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d70f1ae780 con 0x55d70f13dbd0 2024-03-21T11:34:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.087+0000 7fc93e000640 1 -- 172.21.15.47:0/3564761282 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc940004a50 con 0x55d70f13dbd0 2024-03-21T11:34:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.088+0000 7fc93e000640 1 -- 172.21.15.47:0/3564761282 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc9400051f0 con 0x55d70f13dbd0 2024-03-21T11:34:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.088+0000 7fc95a22b740 10 client.18840 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.088+0000 7fc95a22b740 10 client.18840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.088+0000 7fc95a22b740 20 client.18840 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.088+0000 7fc95a22b740 1 --2- 172.21.15.47:0/3564761282 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d70f1aede0 0x55d70f1b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.088+0000 7fc95a22b740 4 client.18840 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:34:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.088+0000 7fc95a22b740 1 -- 172.21.15.47:0/3564761282 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d70f0aa0c0 con 0x55d70f1aede0 2024-03-21T11:34:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.088+0000 7fc947e00640 1 --2- 172.21.15.47:0/3564761282 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d70f1aede0 0x55d70f1b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.089+0000 7fc93e000640 10 client.18840 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.089+0000 7fc947e00640 1 --2- 172.21.15.47:0/3564761282 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d70f1aede0 0x55d70f1b11d0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aa7a96ae02a0d1b server_cookie=ab0b8cdada6d84f2 in_seq=0 out_seq=0 2024-03-21T11:34:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.337+0000 7fc93e000640 1 -- 172.21.15.47:0/3564761282 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d70f0aa0c0 con 0x55d70f1aede0 2024-03-21T11:34:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.337+0000 7fc93e000640 10 client.18840 handle_command_reply: tid=0 2024-03-21T11:34:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.337+0000 7fc95a22b740 1 client.18840 shutdown 2024-03-21T11:34:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.338+0000 7fc95a22b740 1 -- 172.21.15.47:0/3564761282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f13dbd0 msgr2=0x55d70f14c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.338+0000 7fc95a22b740 1 --2- 172.21.15.47:0/3564761282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f13dbd0 0x55d70f14c6e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc94002e040 tx=0x7fc940004e30 comp rx=0 tx=0).stop 2024-03-21T11:34:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.338+0000 7fc95a22b740 1 -- 172.21.15.47:0/3564761282 shutdown_connections 2024-03-21T11:34:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.338+0000 7fc95a22b740 1 --2- 172.21.15.47:0/3564761282 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d70f1aede0 0x55d70f1b11d0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.338+0000 7fc95a22b740 1 --2- 172.21.15.47:0/3564761282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d70f142d40 0x55d70f14d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.338+0000 7fc95a22b740 1 --2- 172.21.15.47:0/3564761282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d70f13dbd0 0x55d70f14c6e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.338+0000 7fc95a22b740 1 --2- 172.21.15.47:0/3564761282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d70f13d1e0 0x55d70f14c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.338+0000 7fc95a22b740 1 -- 172.21.15.47:0/3564761282 >> 172.21.15.47:0/3564761282 conn(0x55d70f138750 msgr2=0x55d70f147290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.338+0000 7fc93e000640 0 client.18840 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.339+0000 7fc95a22b740 1 -- 172.21.15.47:0/3564761282 shutdown_connections 2024-03-21T11:34:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.339+0000 7fc95a22b740 1 -- 172.21.15.47:0/3564761282 wait complete. 2024-03-21T11:34:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.339+0000 7fc95a22b740 20 client.18840 trim_cache size 0 max 16384 2024-03-21T11:34:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:34:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:34:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:34:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:34:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.341+0000 7fc957600640 1 -- 172.21.15.47:0/3282460756 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9340a2fc0 msgr2=0x7fc9340a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.341+0000 7fc957600640 1 --2- 172.21.15.47:0/3282460756 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9340a2fc0 0x7fc9340a5480 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc94801a040 tx=0x7fc9480054b0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.341+0000 7fc957600640 1 -- 172.21.15.47:0/3282460756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9501098b0 msgr2=0x7fc9501a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.341+0000 7fc957600640 1 --2- 172.21.15.47:0/3282460756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9501098b0 0x7fc9501a54e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc948009fd0 tx=0x7fc94800ef10 comp rx=0 tx=0).stop 2024-03-21T11:34:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.341+0000 7fc957600640 1 -- 172.21.15.47:0/3282460756 shutdown_connections 2024-03-21T11:34:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.341+0000 7fc957600640 1 --2- 172.21.15.47:0/3282460756 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9340a2fc0 0x7fc9340a5480 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.341+0000 7fc957600640 1 --2- 172.21.15.47:0/3282460756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9501098b0 0x7fc9501a54e0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.341+0000 7fc957600640 1 --2- 172.21.15.47:0/3282460756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc950108ef0 0x7fc95019e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.341+0000 7fc957600640 1 --2- 172.21.15.47:0/3282460756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc950107d00 0x7fc95019df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.341+0000 7fc957600640 1 -- 172.21.15.47:0/3282460756 >> 172.21.15.47:0/3282460756 conn(0x7fc950076c90 msgr2=0x7fc95010c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.341+0000 7fc957600640 1 -- 172.21.15.47:0/3282460756 shutdown_connections 2024-03-21T11:34:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:06.342+0000 7fc957600640 1 -- 172.21.15.47:0/3282460756 wait complete. 2024-03-21T11:34:06.555 INFO:tasks.workunit.client.0.smithi047.stdout: 1 804347 8.04 MB/sec execute 410 sec latency 185.667 ms 2024-03-21T11:34:06.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:06 smithi102 ceph-mon[36748]: pgmap v723: 129 pgs: 129 active+clean; 37 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-03-21T11:34:06.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:06 smithi047 ceph-mon[31351]: pgmap v723: 129 pgs: 129 active+clean; 37 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-03-21T11:34:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:06 smithi100 ceph-mon[36245]: pgmap v723: 129 pgs: 129 active+clean; 37 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-03-21T11:34:07.555 INFO:tasks.workunit.client.0.smithi047.stdout: 1 806094 8.05 MB/sec execute 411 sec latency 15.127 ms 2024-03-21T11:34:07.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:07 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:34:07.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:07 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:34:07.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:07 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:34:08.556 INFO:tasks.workunit.client.0.smithi047.stdout: 1 806888 8.04 MB/sec execute 412 sec latency 14.996 ms 2024-03-21T11:34:08.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:08 smithi102 ceph-mon[36748]: pgmap v724: 129 pgs: 129 active+clean; 35 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 279 op/s 2024-03-21T11:34:08.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:08 smithi047 ceph-mon[31351]: pgmap v724: 129 pgs: 129 active+clean; 35 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 279 op/s 2024-03-21T11:34:08.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:08 smithi100 ceph-mon[36245]: pgmap v724: 129 pgs: 129 active+clean; 35 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 279 op/s 2024-03-21T11:34:09.556 INFO:tasks.workunit.client.0.smithi047.stdout: 1 808748 8.04 MB/sec execute 413 sec latency 28.929 ms 2024-03-21T11:34:10.556 INFO:tasks.workunit.client.0.smithi047.stdout: 1 809902 8.05 MB/sec execute 414 sec latency 44.457 ms 2024-03-21T11:34:10.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:10 smithi102 ceph-mon[36748]: pgmap v725: 129 pgs: 129 active+clean; 35 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 267 op/s 2024-03-21T11:34:10.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:10 smithi047 ceph-mon[31351]: pgmap v725: 129 pgs: 129 active+clean; 35 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 267 op/s 2024-03-21T11:34:10.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:10 smithi100 ceph-mon[36245]: pgmap v725: 129 pgs: 129 active+clean; 35 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 267 op/s 2024-03-21T11:34:11.556 INFO:tasks.workunit.client.0.smithi047.stdout: 1 811061 8.04 MB/sec execute 415 sec latency 24.593 ms 2024-03-21T11:34:12.556 INFO:tasks.workunit.client.0.smithi047.stdout: 1 813043 8.05 MB/sec execute 416 sec latency 20.442 ms 2024-03-21T11:34:12.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:12 smithi102 ceph-mon[36748]: pgmap v726: 129 pgs: 129 active+clean; 35 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-03-21T11:34:12.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:12 smithi047 ceph-mon[31351]: pgmap v726: 129 pgs: 129 active+clean; 35 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-03-21T11:34:12.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:12 smithi100 ceph-mon[36245]: pgmap v726: 129 pgs: 129 active+clean; 35 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 262 op/s 2024-03-21T11:34:13.556 INFO:tasks.workunit.client.0.smithi047.stdout: 1 813918 8.03 MB/sec execute 417 sec latency 14.051 ms 2024-03-21T11:34:14.556 INFO:tasks.workunit.client.0.smithi047.stdout: 1 815606 8.03 MB/sec execute 418 sec latency 16.171 ms 2024-03-21T11:34:14.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:14 smithi102 ceph-mon[36748]: pgmap v727: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-03-21T11:34:14.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:14 smithi047 ceph-mon[31351]: pgmap v727: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-03-21T11:34:14.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:14 smithi100 ceph-mon[36245]: pgmap v727: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 316 op/s 2024-03-21T11:34:15.557 INFO:tasks.workunit.client.0.smithi047.stdout: 1 816904 8.04 MB/sec execute 419 sec latency 49.153 ms 2024-03-21T11:34:16.557 INFO:tasks.workunit.client.0.smithi047.stdout: 1 817772 8.03 MB/sec execute 420 sec latency 28.465 ms 2024-03-21T11:34:16.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:16 smithi102 ceph-mon[36748]: pgmap v728: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 258 op/s 2024-03-21T11:34:16.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:16 smithi047 ceph-mon[31351]: pgmap v728: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 258 op/s 2024-03-21T11:34:16.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:16 smithi100 ceph-mon[36245]: pgmap v728: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 258 op/s 2024-03-21T11:34:17.557 INFO:tasks.workunit.client.0.smithi047.stdout: 1 820148 8.04 MB/sec execute 421 sec latency 9.583 ms 2024-03-21T11:34:18.557 INFO:tasks.workunit.client.0.smithi047.stdout: 1 821114 8.03 MB/sec execute 422 sec latency 12.739 ms 2024-03-21T11:34:18.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:18 smithi102 ceph-mon[36748]: pgmap v729: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.9 MiB/s wr, 258 op/s 2024-03-21T11:34:18.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:18 smithi047 ceph-mon[31351]: pgmap v729: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.9 MiB/s wr, 258 op/s 2024-03-21T11:34:18.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:18 smithi100 ceph-mon[36245]: pgmap v729: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.9 MiB/s wr, 258 op/s 2024-03-21T11:34:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.494+0000 7fc19c600640 1 -- 172.21.15.47:0/4150254858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194103900 msgr2=0x7fc194103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.494+0000 7fc19c600640 1 --2- 172.21.15.47:0/4150254858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194103900 0x7fc194103d20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc188009900 tx=0x7fc18802bc50 comp rx=0 tx=0).stop 2024-03-21T11:34:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.494+0000 7fc19c600640 1 -- 172.21.15.47:0/4150254858 shutdown_connections 2024-03-21T11:34:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.494+0000 7fc19c600640 1 --2- 172.21.15.47:0/4150254858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1941054b0 0x7fc19410bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.494+0000 7fc19c600640 1 --2- 172.21.15.47:0/4150254858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc194104af0 0x7fc194104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.494+0000 7fc19c600640 1 --2- 172.21.15.47:0/4150254858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc194103900 0x7fc194103d20 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.494+0000 7fc19c600640 1 -- 172.21.15.47:0/4150254858 >> 172.21.15.47:0/4150254858 conn(0x7fc1940fee90 msgr2=0x7fc1941012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.494+0000 7fc19c600640 1 -- 172.21.15.47:0/4150254858 shutdown_connections 2024-03-21T11:34:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.494+0000 7fc19c600640 1 -- 172.21.15.47:0/4150254858 wait complete. 2024-03-21T11:34:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.495+0000 7fc19c600640 1 Processor -- start 2024-03-21T11:34:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.495+0000 7fc19c600640 1 -- start start 2024-03-21T11:34:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc19c600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc194103900 0x7fc194199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc19c600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc194104af0 0x7fc19419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc19c600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1941054b0 0x7fc1941a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc19c600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19410f550 con 0x7fc194104af0 2024-03-21T11:34:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc19c600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19410f3d0 con 0x7fc1941054b0 2024-03-21T11:34:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc19c600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19410f6d0 con 0x7fc194103900 2024-03-21T11:34:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc199c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc194103900 0x7fc194199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc199c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc194103900 0x7fc194199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60296/0 (socket says 172.21.15.47:60296) 2024-03-21T11:34:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc199c00640 1 -- 172.21.15.47:0/3992028940 learned_addr learned my addr 172.21.15.47:0/3992028940 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc199200640 1 --2- 172.21.15.47:0/3992028940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc194104af0 0x7fc19419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc19a600640 1 --2- 172.21.15.47:0/3992028940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1941054b0 0x7fc1941a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc199c00640 1 -- 172.21.15.47:0/3992028940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1941054b0 msgr2=0x7fc1941a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc199c00640 1 --2- 172.21.15.47:0/3992028940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1941054b0 0x7fc1941a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc199c00640 1 -- 172.21.15.47:0/3992028940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc194104af0 msgr2=0x7fc19419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc199c00640 1 --2- 172.21.15.47:0/3992028940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc194104af0 0x7fc19419a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.496+0000 7fc199c00640 1 -- 172.21.15.47:0/3992028940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1941a1840 con 0x7fc194103900 2024-03-21T11:34:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.497+0000 7fc199c00640 1 --2- 172.21.15.47:0/3992028940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc194103900 0x7fc194199c30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc188000c00 tx=0x7fc18802c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.497+0000 7fc192000640 1 -- 172.21.15.47:0/3992028940 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188042020 con 0x7fc194103900 2024-03-21T11:34:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.497+0000 7fc19c600640 1 -- 172.21.15.47:0/3992028940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1941a1ad0 con 0x7fc194103900 2024-03-21T11:34:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.497+0000 7fc192000640 1 -- 172.21.15.47:0/3992028940 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc188004450 con 0x7fc194103900 2024-03-21T11:34:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.497+0000 7fc192000640 1 -- 172.21.15.47:0/3992028940 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18803e460 con 0x7fc194103900 2024-03-21T11:34:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.497+0000 7fc19c600640 1 -- 172.21.15.47:0/3992028940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1941a2020 con 0x7fc194103900 2024-03-21T11:34:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.500+0000 7fc19f23a740 1 Processor -- start 2024-03-21T11:34:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.500+0000 7fc19f23a740 1 -- start start 2024-03-21T11:34:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.501+0000 7fc192000640 1 -- 172.21.15.47:0/3992028940 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc188045020 con 0x7fc194103900 2024-03-21T11:34:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.501+0000 7fc19f23a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834b1e0 0x55e86834b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.501+0000 7fc19f23a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834bbd0 0x55e8683506d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.501+0000 7fc19f23a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e868350d40 0x55e8683531d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.501+0000 7fc19f23a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e86830e3b0 con 0x55e868350d40 2024-03-21T11:34:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.501+0000 7fc19f23a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8682b0830 con 0x55e86834bbd0 2024-03-21T11:34:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.501+0000 7fc19f23a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8682595d0 con 0x55e86834b1e0 2024-03-21T11:34:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.501+0000 7fc19a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834b1e0 0x55e86834b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.501+0000 7fc19a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834b1e0 0x55e86834b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60302/0 (socket says 172.21.15.47:60302) 2024-03-21T11:34:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.501+0000 7fc19a600640 1 -- 172.21.15.47:0/2977292916 learned_addr learned my addr 172.21.15.47:0/2977292916 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.501+0000 7fc199c00640 1 --2- 172.21.15.47:0/2977292916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834bbd0 0x55e8683506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.501+0000 7fc199200640 1 --2- 172.21.15.47:0/2977292916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e868350d40 0x55e8683531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.501+0000 7fc19a600640 1 -- 172.21.15.47:0/2977292916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834bbd0 msgr2=0x55e8683506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.502+0000 7fc19a600640 1 --2- 172.21.15.47:0/2977292916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834bbd0 0x55e8683506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.502+0000 7fc19a600640 1 -- 172.21.15.47:0/2977292916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e868350d40 msgr2=0x55e8683531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.502+0000 7fc19a600640 1 --2- 172.21.15.47:0/2977292916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e868350d40 0x55e8683531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.502+0000 7fc19a600640 1 -- 172.21.15.47:0/2977292916 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8682712c0 con 0x55e86834b1e0 2024-03-21T11:34:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.502+0000 7fc199c00640 1 --2- 172.21.15.47:0/2977292916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834bbd0 0x55e8683506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.502+0000 7fc19a600640 1 --2- 172.21.15.47:0/2977292916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834b1e0 0x55e86834b600 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009fd0 tx=0x7fc18c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=514ed75e5900f6d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.502+0000 7fc186a00640 1 -- 172.21.15.47:0/2977292916 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c004d20 con 0x55e86834b1e0 2024-03-21T11:34:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.502+0000 7fc186a00640 1 -- 172.21.15.47:0/2977292916 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc18c004050 con 0x55e86834b1e0 2024-03-21T11:34:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.502+0000 7fc186a00640 1 -- 172.21.15.47:0/2977292916 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c005220 con 0x55e86834b1e0 2024-03-21T11:34:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.502+0000 7fc192000640 1 --2- 172.21.15.47:0/3992028940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc17c0a2fc0 0x7fc17c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.503+0000 7fc199c00640 1 --2- 172.21.15.47:0/3992028940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc17c0a2fc0 0x7fc17c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.503+0000 7fc192000640 1 -- 172.21.15.47:0/3992028940 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc1880f1780 con 0x7fc194103900 2024-03-21T11:34:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.503+0000 7fc19f23a740 1 -- 172.21.15.47:0/2977292916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834b1e0 msgr2=0x55e86834b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:19.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.503+0000 7fc19f23a740 1 --2- 172.21.15.47:0/2977292916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834b1e0 0x55e86834b600 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009fd0 tx=0x7fc18c0118d0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.503+0000 7fc19f23a740 1 -- 172.21.15.47:0/2977292916 shutdown_connections 2024-03-21T11:34:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.503+0000 7fc19f23a740 1 --2- 172.21.15.47:0/2977292916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e868350d40 0x55e8683531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.503+0000 7fc19f23a740 1 --2- 172.21.15.47:0/2977292916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834bbd0 0x55e8683506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.503+0000 7fc19f23a740 1 --2- 172.21.15.47:0/2977292916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834b1e0 0x55e86834b600 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.503+0000 7fc19f23a740 1 -- 172.21.15.47:0/2977292916 >> 172.21.15.47:0/2977292916 conn(0x55e868346750 msgr2=0x55e868346b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.503+0000 7fc19f23a740 1 -- 172.21.15.47:0/2977292916 shutdown_connections 2024-03-21T11:34:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.503+0000 7fc19f23a740 1 -- 172.21.15.47:0/2977292916 wait complete. 2024-03-21T11:34:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.503+0000 7fc199c00640 1 --2- 172.21.15.47:0/3992028940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc17c0a2fc0 0x7fc17c0a5480 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc1880bd7c0 tx=0x7fc1880f7530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.504+0000 7fc19f23a740 1 -- start start 2024-03-21T11:34:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.504+0000 7fc19f23a740 1 Processor -- start 2024-03-21T11:34:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.504+0000 7fc19f23a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e86834b1e0 0x55e868355360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.504+0000 7fc19f23a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834bbd0 0x55e8683558a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.504+0000 7fc19f23a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e868350d40 0x55e86835c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.504+0000 7fc19f23a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e86826ca20 con 0x55e86834b1e0 2024-03-21T11:34:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.504+0000 7fc19f23a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8682ac8d0 con 0x55e868350d40 2024-03-21T11:34:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.504+0000 7fc19f23a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e868353cf0 con 0x55e86834bbd0 2024-03-21T11:34:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.505+0000 7fc19f23a740 10 client.0 _ll_register_callbacks cb 0x55e8682a91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.505+0000 7fc199200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834bbd0 0x55e8683558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.505+0000 7fc19a600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e86834b1e0 0x55e868355360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.505+0000 7fc199200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834bbd0 0x55e8683558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60306/0 (socket says 172.21.15.47:60306) 2024-03-21T11:34:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.505+0000 7fc199200640 1 -- 172.21.15.47:0/3213643177 learned_addr learned my addr 172.21.15.47:0/3213643177 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.505+0000 7fc19a600640 1 --2- 172.21.15.47:0/3213643177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e868350d40 0x55e86835c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.505+0000 7fc199200640 1 -- 172.21.15.47:0/3213643177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e868350d40 msgr2=0x55e86835c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.505+0000 7fc199200640 1 --2- 172.21.15.47:0/3213643177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e868350d40 0x55e86835c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.505+0000 7fc199200640 1 -- 172.21.15.47:0/3213643177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e86834b1e0 msgr2=0x55e868355360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.505+0000 7fc199200640 1 --2- 172.21.15.47:0/3213643177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e86834b1e0 0x55e868355360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.505+0000 7fc199200640 1 -- 172.21.15.47:0/3213643177 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e867fc5210 con 0x55e86834bbd0 2024-03-21T11:34:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.505+0000 7fc199200640 1 --2- 172.21.15.47:0/3213643177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834bbd0 0x55e8683558a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc1800126e0 tx=0x7fc180012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e20d03e000e59aa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.506+0000 7fc17b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.506+0000 7fc17b400640 1 -- 172.21.15.47:0/3213643177 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800120d0 con 0x55e86834bbd0 2024-03-21T11:34:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.506+0000 7fc17b400640 1 -- 172.21.15.47:0/3213643177 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc18000f8c0 con 0x55e86834bbd0 2024-03-21T11:34:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.506+0000 7fc19f23a740 1 -- 172.21.15.47:0/3213643177 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc18c004050 con 0x55e86834bbd0 2024-03-21T11:34:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.506+0000 7fc17b400640 1 -- 172.21.15.47:0/3213643177 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc180012270 con 0x55e86834bbd0 2024-03-21T11:34:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.506+0000 7fc19f23a740 10 client.29209 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.506+0000 7fc19f23a740 1 -- 172.21.15.47:0/3213643177 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e868091620 con 0x55e86834bbd0 2024-03-21T11:34:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.506+0000 7fc17b400640 1 -- 172.21.15.47:0/3213643177 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc18c004050 con 0x55e86834bbd0 2024-03-21T11:34:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.506+0000 7fc17b400640 1 -- 172.21.15.47:0/3213643177 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc18001c420 con 0x55e86834bbd0 2024-03-21T11:34:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.506+0000 7fc19f23a740 10 client.29209 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.506+0000 7fc19f23a740 10 client.29209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.507+0000 7fc19f23a740 20 client.29209 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.507+0000 7fc19f23a740 1 --2- 172.21.15.47:0/3213643177 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e8683612a0 0x55e8683636e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.507+0000 7fc19f23a740 4 client.29209 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.507+0000 7fc19f23a740 1 -- 172.21.15.47:0/3213643177 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8682bdd70 con 0x55e8683612a0 2024-03-21T11:34:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.507+0000 7fc19a600640 1 --2- 172.21.15.47:0/3213643177 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e8683612a0 0x55e8683636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.507+0000 7fc19a600640 1 --2- 172.21.15.47:0/3213643177 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e8683612a0 0x55e8683636e0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5110a25cf21cc07 server_cookie=d417582540de1562 in_seq=0 out_seq=0 2024-03-21T11:34:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.507+0000 7fc17b400640 10 client.29209 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.508+0000 7fc17b400640 1 -- 172.21.15.47:0/3213643177 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e8682bdd70 con 0x55e8683612a0 2024-03-21T11:34:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.508+0000 7fc17b400640 10 client.29209 handle_command_reply: tid=0 2024-03-21T11:34:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.508+0000 7fc19f23a740 1 client.29209 shutdown 2024-03-21T11:34:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.509+0000 7fc19f23a740 1 -- 172.21.15.47:0/3213643177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834bbd0 msgr2=0x55e8683558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.509+0000 7fc19f23a740 1 --2- 172.21.15.47:0/3213643177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834bbd0 0x55e8683558a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc1800126e0 tx=0x7fc180012ba0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.509+0000 7fc19f23a740 1 -- 172.21.15.47:0/3213643177 shutdown_connections 2024-03-21T11:34:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.509+0000 7fc19f23a740 1 --2- 172.21.15.47:0/3213643177 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e8683612a0 0x55e8683636e0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.509+0000 7fc19f23a740 1 --2- 172.21.15.47:0/3213643177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e868350d40 0x55e86835c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.509+0000 7fc19f23a740 1 --2- 172.21.15.47:0/3213643177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834bbd0 0x55e8683558a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.509+0000 7fc19f23a740 1 --2- 172.21.15.47:0/3213643177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e86834b1e0 0x55e868355360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.509+0000 7fc19f23a740 1 -- 172.21.15.47:0/3213643177 >> 172.21.15.47:0/3213643177 conn(0x55e868346750 msgr2=0x55e868345950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.509+0000 7fc17b400640 0 client.29209 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.509+0000 7fc19f23a740 1 -- 172.21.15.47:0/3213643177 shutdown_connections 2024-03-21T11:34:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.509+0000 7fc19f23a740 1 -- 172.21.15.47:0/3213643177 wait complete. 2024-03-21T11:34:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.509+0000 7fc19f23a740 20 client.29209 trim_cache size 0 max 16384 2024-03-21T11:34:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.526+0000 7fc19f23a740 1 Processor -- start 2024-03-21T11:34:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.526+0000 7fc19f23a740 1 -- start start 2024-03-21T11:34:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.527+0000 7fc19f23a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e868346750 0x55e86834cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.527+0000 7fc19f23a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834b1e0 0x55e86834d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.527+0000 7fc19f23a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834bbd0 0x55e86835c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.527+0000 7fc19f23a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1941ac310 con 0x55e868346750 2024-03-21T11:34:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.527+0000 7fc19f23a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1941adc90 con 0x55e86834b1e0 2024-03-21T11:34:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.527+0000 7fc19f23a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1941ac970 con 0x55e86834bbd0 2024-03-21T11:34:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.527+0000 7fc19a600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e868346750 0x55e86834cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.527+0000 7fc199200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834b1e0 0x55e86834d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.527+0000 7fc19a600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e868346750 0x55e86834cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39384/0 (socket says 172.21.15.47:39384) 2024-03-21T11:34:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.527+0000 7fc19a600640 1 -- 172.21.15.47:0/2948972400 learned_addr learned my addr 172.21.15.47:0/2948972400 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.527+0000 7fc19a600640 1 --2- 172.21.15.47:0/2948972400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834bbd0 0x55e86835c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.528+0000 7fc199200640 1 -- 172.21.15.47:0/2948972400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834bbd0 msgr2=0x55e86835c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.528+0000 7fc199200640 1 --2- 172.21.15.47:0/2948972400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834bbd0 0x55e86835c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.528+0000 7fc199200640 1 -- 172.21.15.47:0/2948972400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e868346750 msgr2=0x55e86834cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.528+0000 7fc199200640 1 --2- 172.21.15.47:0/2948972400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e868346750 0x55e86834cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.528+0000 7fc199200640 1 -- 172.21.15.47:0/2948972400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e86834c460 con 0x55e86834b1e0 2024-03-21T11:34:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.528+0000 7fc19a600640 1 --2- 172.21.15.47:0/2948972400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e868346750 0x55e86834cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.528+0000 7fc199200640 1 --2- 172.21.15.47:0/2948972400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834b1e0 0x55e86834d430 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc18001c960 tx=0x7fc180009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69231f87dd4b6967 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.528+0000 7fc186000640 1 -- 172.21.15.47:0/2948972400 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18001c420 con 0x55e86834b1e0 2024-03-21T11:34:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.528+0000 7fc186000640 1 -- 172.21.15.47:0/2948972400 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc18001c5c0 con 0x55e86834b1e0 2024-03-21T11:34:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.528+0000 7fc186000640 1 -- 172.21.15.47:0/2948972400 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800035f0 con 0x55e86834b1e0 2024-03-21T11:34:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.529+0000 7fc19f23a740 1 -- 172.21.15.47:0/2948972400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834b1e0 msgr2=0x55e86834d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.529+0000 7fc19f23a740 1 --2- 172.21.15.47:0/2948972400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834b1e0 0x55e86834d430 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc18001c960 tx=0x7fc180009f00 comp rx=0 tx=0).stop 2024-03-21T11:34:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.529+0000 7fc19f23a740 1 -- 172.21.15.47:0/2948972400 shutdown_connections 2024-03-21T11:34:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.529+0000 7fc19f23a740 1 --2- 172.21.15.47:0/2948972400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e86834bbd0 0x55e86835c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.529+0000 7fc19f23a740 1 --2- 172.21.15.47:0/2948972400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834b1e0 0x55e86834d430 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.529+0000 7fc19f23a740 1 --2- 172.21.15.47:0/2948972400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e868346750 0x55e86834cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.529+0000 7fc19f23a740 1 -- 172.21.15.47:0/2948972400 >> 172.21.15.47:0/2948972400 conn(0x55e868350880 msgr2=0x55e868355360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.529+0000 7fc19f23a740 1 -- 172.21.15.47:0/2948972400 shutdown_connections 2024-03-21T11:34:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.529+0000 7fc19f23a740 1 -- 172.21.15.47:0/2948972400 wait complete. 2024-03-21T11:34:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.529+0000 7fc19f23a740 1 -- start start 2024-03-21T11:34:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.530+0000 7fc19f23a740 1 Processor -- start 2024-03-21T11:34:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.530+0000 7fc19f23a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e868346750 0x55e86835a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.530+0000 7fc19f23a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834b1e0 0x55e86835ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.530+0000 7fc19f23a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e86834bbd0 0x55e86835bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.530+0000 7fc19f23a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1941ad630 con 0x55e86834bbd0 2024-03-21T11:34:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.530+0000 7fc19f23a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1941acfd0 con 0x55e86834b1e0 2024-03-21T11:34:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.530+0000 7fc19f23a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1941aa4d0 con 0x55e868346750 2024-03-21T11:34:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.530+0000 7fc19f23a740 10 client.0 _ll_register_callbacks cb 0x7fc180009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.530+0000 7fc199200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834b1e0 0x55e86835ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.530+0000 7fc199200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834b1e0 0x55e86835ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38806/0 (socket says 172.21.15.47:38806) 2024-03-21T11:34:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.530+0000 7fc199200640 1 -- 172.21.15.47:0/240905948 learned_addr learned my addr 172.21.15.47:0/240905948 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.531+0000 7fc19a600640 1 --2- 172.21.15.47:0/240905948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e868346750 0x55e86835a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.531+0000 7fc199200640 1 -- 172.21.15.47:0/240905948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e868346750 msgr2=0x55e86835a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.531+0000 7fc199200640 1 --2- 172.21.15.47:0/240905948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e868346750 0x55e86835a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.531+0000 7fc199200640 1 -- 172.21.15.47:0/240905948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e86834bbd0 msgr2=0x55e86835bd50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:34:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.531+0000 7fc199200640 1 --2- 172.21.15.47:0/240905948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e86834bbd0 0x55e86835bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.531+0000 7fc199200640 1 -- 172.21.15.47:0/240905948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e86835c340 con 0x55e86834b1e0 2024-03-21T11:34:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.531+0000 7fc199200640 1 --2- 172.21.15.47:0/240905948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834b1e0 0x55e86835ad90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc1800058c0 tx=0x7fc180005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=690e5738b4cb646 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.531+0000 7fc17b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.531+0000 7fc17b400640 1 -- 172.21.15.47:0/240905948 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180037060 con 0x55e86834b1e0 2024-03-21T11:34:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.532+0000 7fc19f23a740 1 -- 172.21.15.47:0/240905948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc18001c5c0 con 0x55e86834b1e0 2024-03-21T11:34:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.532+0000 7fc17b400640 1 -- 172.21.15.47:0/240905948 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc180005600 con 0x55e86834b1e0 2024-03-21T11:34:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.532+0000 7fc17b400640 1 -- 172.21.15.47:0/240905948 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc18003e050 con 0x55e86834b1e0 2024-03-21T11:34:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.532+0000 7fc19f23a740 10 client.29195 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.532+0000 7fc19f23a740 1 -- 172.21.15.47:0/240905948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e8683bd3e0 con 0x55e86834b1e0 2024-03-21T11:34:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.532+0000 7fc17b400640 1 -- 172.21.15.47:0/240905948 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc18001c5c0 con 0x55e86834b1e0 2024-03-21T11:34:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.532+0000 7fc17b400640 1 -- 172.21.15.47:0/240905948 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc18002a430 con 0x55e86834b1e0 2024-03-21T11:34:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.532+0000 7fc19f23a740 10 client.29195 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.532+0000 7fc19f23a740 10 client.29195 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.532+0000 7fc19f23a740 20 client.29195 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.532+0000 7fc19f23a740 1 --2- 172.21.15.47:0/240905948 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e868350d40 0x55e8683bfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.532+0000 7fc19f23a740 4 client.29195 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:34:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.532+0000 7fc19f23a740 1 -- 172.21.15.47:0/240905948 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e8682b80c0 con 0x55e868350d40 2024-03-21T11:34:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.533+0000 7fc19a600640 1 --2- 172.21.15.47:0/240905948 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e868350d40 0x55e8683bfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.533+0000 7fc19a600640 1 --2- 172.21.15.47:0/240905948 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e868350d40 0x55e8683bfcf0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70aab636ec988cb6 server_cookie=cda9a3e841ec7c82 in_seq=0 out_seq=0 2024-03-21T11:34:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.533+0000 7fc17b400640 10 client.29195 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:19.557 INFO:tasks.workunit.client.0.smithi047.stdout: 1 823474 8.04 MB/sec execute 423 sec latency 11.309 ms 2024-03-21T11:34:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.810+0000 7fc17b400640 1 -- 172.21.15.47:0/240905948 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e8682b80c0 con 0x55e868350d40 2024-03-21T11:34:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.810+0000 7fc17b400640 10 client.29195 handle_command_reply: tid=0 2024-03-21T11:34:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.810+0000 7fc19f23a740 1 client.29195 shutdown 2024-03-21T11:34:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.810+0000 7fc19f23a740 1 -- 172.21.15.47:0/240905948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834b1e0 msgr2=0x55e86835ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.810+0000 7fc19f23a740 1 --2- 172.21.15.47:0/240905948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834b1e0 0x55e86835ad90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc1800058c0 tx=0x7fc180005280 comp rx=0 tx=0).stop 2024-03-21T11:34:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.810+0000 7fc19f23a740 1 -- 172.21.15.47:0/240905948 shutdown_connections 2024-03-21T11:34:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.810+0000 7fc19f23a740 1 --2- 172.21.15.47:0/240905948 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e868350d40 0x55e8683bfcf0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.810+0000 7fc19f23a740 1 --2- 172.21.15.47:0/240905948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e86834bbd0 0x55e86835bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.810+0000 7fc19f23a740 1 --2- 172.21.15.47:0/240905948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e86834b1e0 0x55e86835ad90 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.810+0000 7fc19f23a740 1 --2- 172.21.15.47:0/240905948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e868346750 0x55e86835a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.810+0000 7fc17b400640 0 client.29195 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.810+0000 7fc19f23a740 1 -- 172.21.15.47:0/240905948 >> 172.21.15.47:0/240905948 conn(0x55e868350880 msgr2=0x55e868355320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.810+0000 7fc19f23a740 1 -- 172.21.15.47:0/240905948 shutdown_connections 2024-03-21T11:34:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.810+0000 7fc19f23a740 1 -- 172.21.15.47:0/240905948 wait complete. 2024-03-21T11:34:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.811+0000 7fc19f23a740 20 client.29195 trim_cache size 0 max 16384 2024-03-21T11:34:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:34:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:34:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:34:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:34:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.811+0000 7fc19c600640 1 -- 172.21.15.47:0/3992028940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc17c0a2fc0 msgr2=0x7fc17c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.811+0000 7fc19c600640 1 --2- 172.21.15.47:0/3992028940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc17c0a2fc0 0x7fc17c0a5480 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc1880bd7c0 tx=0x7fc1880f7530 comp rx=0 tx=0).stop 2024-03-21T11:34:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.811+0000 7fc19c600640 1 -- 172.21.15.47:0/3992028940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc194103900 msgr2=0x7fc194199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.811+0000 7fc19c600640 1 --2- 172.21.15.47:0/3992028940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc194103900 0x7fc194199c30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc188000c00 tx=0x7fc18802c660 comp rx=0 tx=0).stop 2024-03-21T11:34:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.811+0000 7fc19c600640 1 -- 172.21.15.47:0/3992028940 shutdown_connections 2024-03-21T11:34:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.811+0000 7fc19c600640 1 --2- 172.21.15.47:0/3992028940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc17c0a2fc0 0x7fc17c0a5480 secure :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc1880bd7c0 tx=0x7fc1880f7530 comp rx=0 tx=0).stop 2024-03-21T11:34:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.811+0000 7fc19c600640 1 --2- 172.21.15.47:0/3992028940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1941054b0 0x7fc1941a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.811+0000 7fc19c600640 1 --2- 172.21.15.47:0/3992028940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc194104af0 0x7fc19419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.811+0000 7fc19c600640 1 --2- 172.21.15.47:0/3992028940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc194103900 0x7fc194199c30 secure :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc188000c00 tx=0x7fc18802c660 comp rx=0 tx=0).stop 2024-03-21T11:34:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.811+0000 7fc19c600640 1 -- 172.21.15.47:0/3992028940 >> 172.21.15.47:0/3992028940 conn(0x7fc1940fee90 msgr2=0x7fc194107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.812+0000 7fc19c600640 1 -- 172.21.15.47:0/3992028940 shutdown_connections 2024-03-21T11:34:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:19.812+0000 7fc19c600640 1 -- 172.21.15.47:0/3992028940 wait complete. 2024-03-21T11:34:20.557 INFO:tasks.workunit.client.0.smithi047.stdout: 1 824221 8.04 MB/sec execute 424 sec latency 135.883 ms 2024-03-21T11:34:20.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:20 smithi047 ceph-mon[31351]: pgmap v730: 129 pgs: 129 active+clean; 40 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 242 op/s 2024-03-21T11:34:20.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:20 smithi100 ceph-mon[36245]: pgmap v730: 129 pgs: 129 active+clean; 40 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 242 op/s 2024-03-21T11:34:20.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:20 smithi102 ceph-mon[36748]: pgmap v730: 129 pgs: 129 active+clean; 40 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 242 op/s 2024-03-21T11:34:21.557 INFO:tasks.workunit.client.0.smithi047.stdout: 1 825724 8.03 MB/sec execute 425 sec latency 24.633 ms 2024-03-21T11:34:22.559 INFO:tasks.workunit.client.0.smithi047.stdout: 1 827604 8.04 MB/sec execute 426 sec latency 12.462 ms 2024-03-21T11:34:22.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:22 smithi047 ceph-mon[31351]: pgmap v731: 129 pgs: 129 active+clean; 40 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-03-21T11:34:22.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:22 smithi100 ceph-mon[36245]: pgmap v731: 129 pgs: 129 active+clean; 40 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-03-21T11:34:22.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:22 smithi102 ceph-mon[36748]: pgmap v731: 129 pgs: 129 active+clean; 40 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-03-21T11:34:23.557 INFO:tasks.workunit.client.0.smithi047.stdout: 1 828758 8.04 MB/sec execute 427 sec latency 61.192 ms 2024-03-21T11:34:24.558 INFO:tasks.workunit.client.0.smithi047.stdout: 1 831016 8.05 MB/sec execute 428 sec latency 11.705 ms 2024-03-21T11:34:24.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:24 smithi047 ceph-mon[31351]: pgmap v732: 129 pgs: 129 active+clean; 34 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 306 op/s 2024-03-21T11:34:24.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:24 smithi100 ceph-mon[36245]: pgmap v732: 129 pgs: 129 active+clean; 34 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 306 op/s 2024-03-21T11:34:24.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:24 smithi102 ceph-mon[36748]: pgmap v732: 129 pgs: 129 active+clean; 34 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 306 op/s 2024-03-21T11:34:25.558 INFO:tasks.workunit.client.0.smithi047.stdout: 1 831868 8.04 MB/sec execute 429 sec latency 14.049 ms 2024-03-21T11:34:26.558 INFO:tasks.workunit.client.0.smithi047.stdout: 1 833649 8.04 MB/sec execute 430 sec latency 49.447 ms 2024-03-21T11:34:26.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:26 smithi047 ceph-mon[31351]: pgmap v733: 129 pgs: 129 active+clean; 34 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-03-21T11:34:26.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:26 smithi100 ceph-mon[36245]: pgmap v733: 129 pgs: 129 active+clean; 34 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-03-21T11:34:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:26 smithi102 ceph-mon[36748]: pgmap v733: 129 pgs: 129 active+clean; 34 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-03-21T11:34:27.558 INFO:tasks.workunit.client.0.smithi047.stdout: 1 834873 8.04 MB/sec execute 431 sec latency 16.207 ms 2024-03-21T11:34:28.558 INFO:tasks.workunit.client.0.smithi047.stdout: 1 836337 8.03 MB/sec execute 432 sec latency 19.806 ms 2024-03-21T11:34:28.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:28 smithi047 ceph-mon[31351]: pgmap v734: 129 pgs: 129 active+clean; 36 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 286 op/s 2024-03-21T11:34:28.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:28 smithi100 ceph-mon[36245]: pgmap v734: 129 pgs: 129 active+clean; 36 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 286 op/s 2024-03-21T11:34:28.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:28 smithi102 ceph-mon[36748]: pgmap v734: 129 pgs: 129 active+clean; 36 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 286 op/s 2024-03-21T11:34:29.558 INFO:tasks.workunit.client.0.smithi047.stdout: 1 838117 8.04 MB/sec execute 433 sec latency 13.762 ms 2024-03-21T11:34:30.559 INFO:tasks.workunit.client.0.smithi047.stdout: 1 838881 8.03 MB/sec execute 434 sec latency 23.252 ms 2024-03-21T11:34:30.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:30 smithi047 ceph-mon[31351]: pgmap v735: 129 pgs: 129 active+clean; 37 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 308 op/s 2024-03-21T11:34:30.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:30 smithi100 ceph-mon[36245]: pgmap v735: 129 pgs: 129 active+clean; 37 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 308 op/s 2024-03-21T11:34:30.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:30 smithi102 ceph-mon[36748]: pgmap v735: 129 pgs: 129 active+clean; 37 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 308 op/s 2024-03-21T11:34:31.559 INFO:tasks.workunit.client.0.smithi047.stdout: 1 840515 8.03 MB/sec execute 435 sec latency 56.027 ms 2024-03-21T11:34:32.559 INFO:tasks.workunit.client.0.smithi047.stdout: 1 841923 8.03 MB/sec execute 436 sec latency 11.836 ms 2024-03-21T11:34:32.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:32 smithi047 ceph-mon[31351]: pgmap v736: 129 pgs: 129 active+clean; 37 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 260 op/s 2024-03-21T11:34:32.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:32 smithi100 ceph-mon[36245]: pgmap v736: 129 pgs: 129 active+clean; 37 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 260 op/s 2024-03-21T11:34:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.980+0000 7f7282000640 1 -- 172.21.15.47:0/1520622870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f727c0fff90 msgr2=0x7f727c1003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.980+0000 7f7282000640 1 --2- 172.21.15.47:0/1520622870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f727c0fff90 0x7f727c1003b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f726c009900 tx=0x7f726c02bc50 comp rx=0 tx=0).stop 2024-03-21T11:34:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.980+0000 7f7282000640 1 -- 172.21.15.47:0/1520622870 shutdown_connections 2024-03-21T11:34:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.980+0000 7f7282000640 1 --2- 172.21.15.47:0/1520622870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f727c101290 0x7f727c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.980+0000 7f7282000640 1 --2- 172.21.15.47:0/1520622870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f727c1008f0 0x7f727c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.980+0000 7f7282000640 1 --2- 172.21.15.47:0/1520622870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f727c0fff90 0x7f727c1003b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.980+0000 7f7282000640 1 -- 172.21.15.47:0/1520622870 >> 172.21.15.47:0/1520622870 conn(0x7f727c0fbb30 msgr2=0x7f727c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.980+0000 7f7282000640 1 -- 172.21.15.47:0/1520622870 shutdown_connections 2024-03-21T11:34:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.980+0000 7f7282000640 1 -- 172.21.15.47:0/1520622870 wait complete. 2024-03-21T11:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.981+0000 7f7282000640 1 Processor -- start 2024-03-21T11:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.981+0000 7f7282000640 1 -- start start 2024-03-21T11:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.981+0000 7f7282000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f727c0fff90 0x7f727c199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.981+0000 7f7282000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f727c1008f0 0x7f727c19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.981+0000 7f7282000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f727c101290 0x7f727c1a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.982+0000 7f7282000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c10f550 con 0x7f727c101290 2024-03-21T11:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.982+0000 7f7282000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c10f3d0 con 0x7f727c0fff90 2024-03-21T11:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.982+0000 7f7282000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c10f6d0 con 0x7f727c1008f0 2024-03-21T11:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.982+0000 7f727b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f727c0fff90 0x7f727c199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.982+0000 7f727aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f727c1008f0 0x7f727c19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.982+0000 7f727b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f727c0fff90 0x7f727c199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58636/0 (socket says 172.21.15.47:58636) 2024-03-21T11:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.982+0000 7f727b400640 1 -- 172.21.15.47:0/669644634 learned_addr learned my addr 172.21.15.47:0/669644634 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.982+0000 7f727be00640 1 --2- 172.21.15.47:0/669644634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f727c101290 0x7f727c1a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.982+0000 7f727b400640 1 -- 172.21.15.47:0/669644634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f727c1008f0 msgr2=0x7f727c19a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.982+0000 7f727b400640 1 --2- 172.21.15.47:0/669644634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f727c1008f0 0x7f727c19a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.982+0000 7f727b400640 1 -- 172.21.15.47:0/669644634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f727c101290 msgr2=0x7f727c1a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.982+0000 7f727b400640 1 --2- 172.21.15.47:0/669644634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f727c101290 0x7f727c1a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.982+0000 7f727b400640 1 -- 172.21.15.47:0/669644634 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c1a1730 con 0x7f727c0fff90 2024-03-21T11:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.983+0000 7f727b400640 1 --2- 172.21.15.47:0/669644634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f727c0fff90 0x7f727c199b10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f726c009850 tx=0x7f726c004e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.983+0000 7f7273e00640 1 -- 172.21.15.47:0/669644634 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c041070 con 0x7f727c0fff90 2024-03-21T11:34:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.983+0000 7f7273e00640 1 -- 172.21.15.47:0/669644634 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f726c0054e0 con 0x7f727c0fff90 2024-03-21T11:34:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.983+0000 7f7282000640 1 -- 172.21.15.47:0/669644634 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727c1a19c0 con 0x7f727c0fff90 2024-03-21T11:34:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.983+0000 7f7273e00640 1 -- 172.21.15.47:0/669644634 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c003e20 con 0x7f727c0fff90 2024-03-21T11:34:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.983+0000 7f7282000640 1 -- 172.21.15.47:0/669644634 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727c1a1f10 con 0x7f727c0fff90 2024-03-21T11:34:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.987+0000 7f7273e00640 1 -- 172.21.15.47:0/669644634 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f726c044070 con 0x7f727c0fff90 2024-03-21T11:34:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.988+0000 7f7284c68740 1 Processor -- start 2024-03-21T11:34:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.988+0000 7f7284c68740 1 -- start start 2024-03-21T11:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f7284c68740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d381e0 0x565487d38600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f7284c68740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d38bd0 0x565487d3d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f7284c68740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d3dd40 0x565487d401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f7284c68740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x565487cfb3b0 con 0x565487d381e0 2024-03-21T11:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f7284c68740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x565487c9d830 con 0x565487d38bd0 2024-03-21T11:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f7284c68740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x565487c465d0 con 0x565487d3dd40 2024-03-21T11:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f727b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d38bd0 0x565487d3d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f727aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d3dd40 0x565487d401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f727aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d3dd40 0x565487d401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58762/0 (socket says 172.21.15.47:58762) 2024-03-21T11:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f727aa00640 1 -- 172.21.15.47:0/1010249823 learned_addr learned my addr 172.21.15.47:0/1010249823 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f727be00640 1 --2- 172.21.15.47:0/1010249823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d381e0 0x565487d38600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f7273e00640 1 --2- 172.21.15.47:0/669644634 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72600a3090 0x7f72600a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f727aa00640 1 -- 172.21.15.47:0/1010249823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d38bd0 msgr2=0x565487d3d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f727aa00640 1 --2- 172.21.15.47:0/1010249823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d38bd0 0x565487d3d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f727aa00640 1 -- 172.21.15.47:0/1010249823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d381e0 msgr2=0x565487d38600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f727aa00640 1 --2- 172.21.15.47:0/1010249823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d381e0 0x565487d38600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.989+0000 7f727aa00640 1 -- 172.21.15.47:0/1010249823 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565487c5e2c0 con 0x565487d3dd40 2024-03-21T11:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.990+0000 7f7273e00640 1 -- 172.21.15.47:0/669644634 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f726c0f1300 con 0x7f727c0fff90 2024-03-21T11:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.990+0000 7f727be00640 1 --2- 172.21.15.47:0/669644634 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72600a3090 0x7f72600a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.990+0000 7f727aa00640 1 --2- 172.21.15.47:0/1010249823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d3dd40 0x565487d401d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7268009c90 tx=0x7f7268011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6d1c0510e0dc2b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.990+0000 7f7267e00640 1 -- 172.21.15.47:0/1010249823 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726801e070 con 0x565487d3dd40 2024-03-21T11:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.990+0000 7f7267e00640 1 -- 172.21.15.47:0/1010249823 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7268019b10 con 0x565487d3dd40 2024-03-21T11:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.990+0000 7f7267e00640 1 -- 172.21.15.47:0/1010249823 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726800eac0 con 0x565487d3dd40 2024-03-21T11:34:32.993 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:32 smithi102 ceph-mon[36748]: pgmap v736: 129 pgs: 129 active+clean; 37 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 260 op/s 2024-03-21T11:34:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.991+0000 7f7284c68740 1 -- 172.21.15.47:0/1010249823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d3dd40 msgr2=0x565487d401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.991+0000 7f7284c68740 1 --2- 172.21.15.47:0/1010249823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d3dd40 0x565487d401d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7268009c90 tx=0x7f7268011810 comp rx=0 tx=0).stop 2024-03-21T11:34:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.991+0000 7f727be00640 1 --2- 172.21.15.47:0/669644634 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72600a3090 0x7f72600a5550 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x565487d39430 tx=0x7f7274004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.991+0000 7f7284c68740 1 -- 172.21.15.47:0/1010249823 shutdown_connections 2024-03-21T11:34:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.991+0000 7f7284c68740 1 --2- 172.21.15.47:0/1010249823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d3dd40 0x565487d401d0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.991+0000 7f7284c68740 1 --2- 172.21.15.47:0/1010249823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d38bd0 0x565487d3d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.991+0000 7f7284c68740 1 --2- 172.21.15.47:0/1010249823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d381e0 0x565487d38600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.991+0000 7f7284c68740 1 -- 172.21.15.47:0/1010249823 >> 172.21.15.47:0/1010249823 conn(0x565487d33750 msgr2=0x565487d33b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.991+0000 7f7284c68740 1 -- 172.21.15.47:0/1010249823 shutdown_connections 2024-03-21T11:34:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.991+0000 7f7284c68740 1 -- 172.21.15.47:0/1010249823 wait complete. 2024-03-21T11:34:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.992+0000 7f7284c68740 1 -- start start 2024-03-21T11:34:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.992+0000 7f7284c68740 1 Processor -- start 2024-03-21T11:34:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.992+0000 7f7284c68740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d42390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.992+0000 7f7284c68740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d38bd0 0x565487d428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.992+0000 7f7284c68740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d3dd40 0x565487d49910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.992+0000 7f7284c68740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x565487c59a20 con 0x565487d38bd0 2024-03-21T11:34:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.992+0000 7f7284c68740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x565487c998d0 con 0x565487d3dd40 2024-03-21T11:34:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.992+0000 7f7284c68740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x565487d40cf0 con 0x565487d381e0 2024-03-21T11:34:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.992+0000 7f7284c68740 10 client.0 _ll_register_callbacks cb 0x565487c961c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.992+0000 7f727b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d42390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.992+0000 7f727b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d42390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58768/0 (socket says 172.21.15.47:58768) 2024-03-21T11:34:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.992+0000 7f727b400640 1 -- 172.21.15.47:0/2539444732 learned_addr learned my addr 172.21.15.47:0/2539444732 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.992+0000 7f727be00640 1 --2- 172.21.15.47:0/2539444732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d3dd40 0x565487d49910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.993+0000 7f727aa00640 1 --2- 172.21.15.47:0/2539444732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d38bd0 0x565487d428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.993+0000 7f727b400640 1 -- 172.21.15.47:0/2539444732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d3dd40 msgr2=0x565487d49910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.993+0000 7f727b400640 1 --2- 172.21.15.47:0/2539444732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d3dd40 0x565487d49910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.993+0000 7f727b400640 1 -- 172.21.15.47:0/2539444732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d38bd0 msgr2=0x565487d428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.993+0000 7f727b400640 1 --2- 172.21.15.47:0/2539444732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d38bd0 0x565487d428d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.993+0000 7f727b400640 1 -- 172.21.15.47:0/2539444732 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654879b2210 con 0x565487d381e0 2024-03-21T11:34:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.993+0000 7f727aa00640 1 --2- 172.21.15.47:0/2539444732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d38bd0 0x565487d428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.993+0000 7f727b400640 1 --2- 172.21.15.47:0/2539444732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d42390 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f726c006010 tx=0x7f726c02bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75a0b0c0bae79bb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.993+0000 7f7264c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.993+0000 7f7264c00640 1 -- 172.21.15.47:0/2539444732 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c04b040 con 0x565487d381e0 2024-03-21T11:34:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.993+0000 7f7264c00640 1 -- 172.21.15.47:0/2539444732 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f726c0bd8a0 con 0x565487d381e0 2024-03-21T11:34:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.993+0000 7f7284c68740 1 -- 172.21.15.47:0/2539444732 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7268019b10 con 0x565487d381e0 2024-03-21T11:34:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.994+0000 7f7264c00640 1 -- 172.21.15.47:0/2539444732 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f726c0bd410 con 0x565487d381e0 2024-03-21T11:34:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.994+0000 7f7284c68740 10 client.29227 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.994+0000 7f7284c68740 1 -- 172.21.15.47:0/2539444732 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x565487a7e620 con 0x565487d381e0 2024-03-21T11:34:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.994+0000 7f7264c00640 1 -- 172.21.15.47:0/2539444732 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7268019b10 con 0x565487d381e0 2024-03-21T11:34:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.994+0000 7f7264c00640 1 -- 172.21.15.47:0/2539444732 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f726c0a6bd0 con 0x565487d381e0 2024-03-21T11:34:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.994+0000 7f7284c68740 10 client.29227 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.994+0000 7f7284c68740 10 client.29227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.994+0000 7f7284c68740 20 client.29227 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.994+0000 7f7284c68740 1 --2- 172.21.15.47:0/2539444732 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x565487d4e190 0x565487d50630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.994+0000 7f7284c68740 4 client.29227 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.994+0000 7f7284c68740 1 -- 172.21.15.47:0/2539444732 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565487caad70 con 0x565487d4e190 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.995+0000 7f727aa00640 1 --2- 172.21.15.47:0/2539444732 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x565487d4e190 0x565487d50630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.995+0000 7f727aa00640 1 --2- 172.21.15.47:0/2539444732 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x565487d4e190 0x565487d50630 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a12b8112d9a2795b server_cookie=f642aa334c615c20 in_seq=0 out_seq=0 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.995+0000 7f7264c00640 10 client.29227 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.996+0000 7f7264c00640 1 -- 172.21.15.47:0/2539444732 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x565487caad70 con 0x565487d4e190 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.996+0000 7f7264c00640 10 client.29227 handle_command_reply: tid=0 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.996+0000 7f7284c68740 1 client.29227 shutdown 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.996+0000 7f7284c68740 1 -- 172.21.15.47:0/2539444732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 msgr2=0x565487d42390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.996+0000 7f7284c68740 1 --2- 172.21.15.47:0/2539444732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d42390 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f726c006010 tx=0x7f726c02bc50 comp rx=0 tx=0).stop 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.996+0000 7f7284c68740 1 -- 172.21.15.47:0/2539444732 shutdown_connections 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.996+0000 7f7284c68740 1 --2- 172.21.15.47:0/2539444732 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x565487d4e190 0x565487d50630 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.996+0000 7f7284c68740 1 --2- 172.21.15.47:0/2539444732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d3dd40 0x565487d49910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.996+0000 7f7284c68740 1 --2- 172.21.15.47:0/2539444732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d38bd0 0x565487d428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.996+0000 7f7264c00640 0 client.29227 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.996+0000 7f7284c68740 1 --2- 172.21.15.47:0/2539444732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d42390 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.996+0000 7f7284c68740 1 -- 172.21.15.47:0/2539444732 >> 172.21.15.47:0/2539444732 conn(0x565487d33750 msgr2=0x565487d32950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.997+0000 7f7284c68740 1 -- 172.21.15.47:0/2539444732 shutdown_connections 2024-03-21T11:34:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.997+0000 7f7284c68740 1 -- 172.21.15.47:0/2539444732 wait complete. 2024-03-21T11:34:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:32.997+0000 7f7284c68740 20 client.29227 trim_cache size 0 max 16384 2024-03-21T11:34:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.014+0000 7f7284c68740 1 Processor -- start 2024-03-21T11:34:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.014+0000 7f7284c68740 1 -- start start 2024-03-21T11:34:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.014+0000 7f7284c68740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d369b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.014+0000 7f7284c68740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d38bd0 0x565487d498d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.014+0000 7f7284c68740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d3dd40 0x565487d36ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.014+0000 7f7284c68740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c1ac250 con 0x565487d38bd0 2024-03-21T11:34:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.014+0000 7f7284c68740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c1adbd0 con 0x565487d3dd40 2024-03-21T11:34:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.014+0000 7f7284c68740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c1ac8b0 con 0x565487d381e0 2024-03-21T11:34:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.014+0000 7f727b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d369b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.015+0000 7f727b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d369b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58772/0 (socket says 172.21.15.47:58772) 2024-03-21T11:34:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.015+0000 7f727be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d3dd40 0x565487d36ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.015+0000 7f727b400640 1 -- 172.21.15.47:0/828301350 learned_addr learned my addr 172.21.15.47:0/828301350 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.015+0000 7f727aa00640 1 --2- 172.21.15.47:0/828301350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d38bd0 0x565487d498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.015+0000 7f727b400640 1 -- 172.21.15.47:0/828301350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d3dd40 msgr2=0x565487d36ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.015+0000 7f727b400640 1 --2- 172.21.15.47:0/828301350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d3dd40 0x565487d36ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.015+0000 7f727b400640 1 -- 172.21.15.47:0/828301350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d38bd0 msgr2=0x565487d498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.015+0000 7f727b400640 1 --2- 172.21.15.47:0/828301350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d38bd0 0x565487d498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.015+0000 7f727b400640 1 -- 172.21.15.47:0/828301350 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565487d3d190 con 0x565487d381e0 2024-03-21T11:34:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.015+0000 7f727aa00640 1 --2- 172.21.15.47:0/828301350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d38bd0 0x565487d498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.015+0000 7f727be00640 1 --2- 172.21.15.47:0/828301350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d3dd40 0x565487d36ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.015+0000 7f727b400640 1 --2- 172.21.15.47:0/828301350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d369b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f726c02f020 tx=0x7f726c0bbb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c6706f7ee5b57a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.015+0000 7f7267400640 1 -- 172.21.15.47:0/828301350 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c04b040 con 0x565487d381e0 2024-03-21T11:34:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.016+0000 7f7267400640 1 -- 172.21.15.47:0/828301350 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f726c0bdd40 con 0x565487d381e0 2024-03-21T11:34:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.016+0000 7f7267400640 1 -- 172.21.15.47:0/828301350 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c0bcfa0 con 0x565487d381e0 2024-03-21T11:34:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.016+0000 7f7284c68740 1 -- 172.21.15.47:0/828301350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 msgr2=0x565487d369b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.016+0000 7f7284c68740 1 --2- 172.21.15.47:0/828301350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d369b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f726c02f020 tx=0x7f726c0bbb50 comp rx=0 tx=0).stop 2024-03-21T11:34:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.017+0000 7f7284c68740 1 -- 172.21.15.47:0/828301350 shutdown_connections 2024-03-21T11:34:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.017+0000 7f7284c68740 1 --2- 172.21.15.47:0/828301350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d3dd40 0x565487d36ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.017+0000 7f7284c68740 1 --2- 172.21.15.47:0/828301350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d38bd0 0x565487d498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.017+0000 7f7284c68740 1 --2- 172.21.15.47:0/828301350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d369b0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.017+0000 7f7284c68740 1 -- 172.21.15.47:0/828301350 >> 172.21.15.47:0/828301350 conn(0x565487d33750 msgr2=0x565487d33b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.017+0000 7f7284c68740 1 -- 172.21.15.47:0/828301350 shutdown_connections 2024-03-21T11:34:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.017+0000 7f7284c68740 1 -- 172.21.15.47:0/828301350 wait complete. 2024-03-21T11:34:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.017+0000 7f7284c68740 1 -- start start 2024-03-21T11:34:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.017+0000 7f7284c68740 1 Processor -- start 2024-03-21T11:34:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.018+0000 7f7284c68740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.018+0000 7f7284c68740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d38bd0 0x565487d47c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.018+0000 7f7284c68740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d3dd40 0x565487d48c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.018+0000 7f7284c68740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c1ad570 con 0x565487d3dd40 2024-03-21T11:34:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.018+0000 7f7284c68740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c1acf10 con 0x565487d38bd0 2024-03-21T11:34:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.018+0000 7f7284c68740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c1aa410 con 0x565487d381e0 2024-03-21T11:34:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.018+0000 7f7284c68740 10 client.0 _ll_register_callbacks cb 0x7f726c0045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.018+0000 7f727b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.018+0000 7f727b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d476f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58786/0 (socket says 172.21.15.47:58786) 2024-03-21T11:34:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.018+0000 7f727b400640 1 -- 172.21.15.47:0/4120453751 learned_addr learned my addr 172.21.15.47:0/4120453751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.018+0000 7f727aa00640 1 --2- 172.21.15.47:0/4120453751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d38bd0 0x565487d47c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.018+0000 7f727be00640 1 --2- 172.21.15.47:0/4120453751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d3dd40 0x565487d48c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.019+0000 7f727be00640 1 -- 172.21.15.47:0/4120453751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 msgr2=0x565487d476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.020+0000 7f727be00640 1 --2- 172.21.15.47:0/4120453751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.020+0000 7f727be00640 1 -- 172.21.15.47:0/4120453751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d38bd0 msgr2=0x565487d47c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.020+0000 7f727be00640 1 --2- 172.21.15.47:0/4120453751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d38bd0 0x565487d47c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.020+0000 7f727be00640 1 -- 172.21.15.47:0/4120453751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565487d49230 con 0x565487d3dd40 2024-03-21T11:34:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.020+0000 7f727aa00640 1 --2- 172.21.15.47:0/4120453751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d38bd0 0x565487d47c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.020+0000 7f727be00640 1 --2- 172.21.15.47:0/4120453751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d3dd40 0x565487d48c40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7274012fd0 tx=0x7f7274004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac99670a64564c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.020+0000 7f7264c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:34:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.020+0000 7f7264c00640 1 -- 172.21.15.47:0/4120453751 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727402c060 con 0x565487d3dd40 2024-03-21T11:34:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.021+0000 7f7264c00640 1 -- 172.21.15.47:0/4120453751 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7274003b80 con 0x565487d3dd40 2024-03-21T11:34:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.021+0000 7f7284c68740 1 -- 172.21.15.47:0/4120453751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f726c0bdd40 con 0x565487d3dd40 2024-03-21T11:34:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.021+0000 7f7264c00640 1 -- 172.21.15.47:0/4120453751 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7274003d80 con 0x565487d3dd40 2024-03-21T11:34:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.021+0000 7f7284c68740 10 client.18873 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.021+0000 7f7284c68740 1 -- 172.21.15.47:0/4120453751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x565487db2b10 con 0x565487d3dd40 2024-03-21T11:34:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.022+0000 7f7264c00640 1 -- 172.21.15.47:0/4120453751 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f726c0bdd40 con 0x565487d3dd40 2024-03-21T11:34:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.022+0000 7f7264c00640 1 -- 172.21.15.47:0/4120453751 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7274027400 con 0x565487d3dd40 2024-03-21T11:34:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.022+0000 7f7284c68740 10 client.18873 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.022+0000 7f7284c68740 10 client.18873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.022+0000 7f7284c68740 20 client.18873 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.022+0000 7f7284c68740 1 --2- 172.21.15.47:0/4120453751 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x565487db34d0 0x565487db58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.022+0000 7f7284c68740 4 client.18873 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:34:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.022+0000 7f7284c68740 1 -- 172.21.15.47:0/4120453751 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565487ca50c0 con 0x565487db34d0 2024-03-21T11:34:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.022+0000 7f727b400640 1 --2- 172.21.15.47:0/4120453751 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x565487db34d0 0x565487db58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.023+0000 7f727b400640 1 --2- 172.21.15.47:0/4120453751 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x565487db34d0 0x565487db58f0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f39e5a3b73f653ba server_cookie=d1b2f5d131886900 in_seq=0 out_seq=0 2024-03-21T11:34:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.023+0000 7f7264c00640 10 client.18873 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:33.311 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.310+0000 7f7264c00640 1 -- 172.21.15.47:0/4120453751 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565487ca50c0 con 0x565487db34d0 2024-03-21T11:34:33.311 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.310+0000 7f7264c00640 10 client.18873 handle_command_reply: tid=0 2024-03-21T11:34:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.310+0000 7f7284c68740 1 client.18873 shutdown 2024-03-21T11:34:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.310+0000 7f7284c68740 1 -- 172.21.15.47:0/4120453751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d3dd40 msgr2=0x565487d48c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.310+0000 7f7284c68740 1 --2- 172.21.15.47:0/4120453751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d3dd40 0x565487d48c40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7274012fd0 tx=0x7f7274004ab0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.311+0000 7f7284c68740 1 -- 172.21.15.47:0/4120453751 shutdown_connections 2024-03-21T11:34:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.311+0000 7f7284c68740 1 --2- 172.21.15.47:0/4120453751 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x565487db34d0 0x565487db58f0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.311+0000 7f7284c68740 1 --2- 172.21.15.47:0/4120453751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565487d3dd40 0x565487d48c40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.311+0000 7f7284c68740 1 --2- 172.21.15.47:0/4120453751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565487d38bd0 0x565487d47c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.311+0000 7f7284c68740 1 --2- 172.21.15.47:0/4120453751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565487d381e0 0x565487d476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.311+0000 7f7284c68740 1 -- 172.21.15.47:0/4120453751 >> 172.21.15.47:0/4120453751 conn(0x565487d33750 msgr2=0x565487d369b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.311+0000 7f7264c00640 0 client.18873 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.311+0000 7f7284c68740 1 -- 172.21.15.47:0/4120453751 shutdown_connections 2024-03-21T11:34:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.311+0000 7f7284c68740 1 -- 172.21.15.47:0/4120453751 wait complete. 2024-03-21T11:34:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.311+0000 7f7284c68740 20 client.18873 trim_cache size 0 max 16384 2024-03-21T11:34:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:34:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:34:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:34:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:34:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.312+0000 7f7282000640 1 -- 172.21.15.47:0/669644634 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72600a3090 msgr2=0x7f72600a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.312+0000 7f7282000640 1 --2- 172.21.15.47:0/669644634 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72600a3090 0x7f72600a5550 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x565487d39430 tx=0x7f7274004a40 comp rx=0 tx=0).stop 2024-03-21T11:34:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.312+0000 7f7282000640 1 -- 172.21.15.47:0/669644634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f727c0fff90 msgr2=0x7f727c199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.312+0000 7f7282000640 1 --2- 172.21.15.47:0/669644634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f727c0fff90 0x7f727c199b10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f726c009850 tx=0x7f726c004e30 comp rx=0 tx=0).stop 2024-03-21T11:34:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.312+0000 7f7282000640 1 -- 172.21.15.47:0/669644634 shutdown_connections 2024-03-21T11:34:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.312+0000 7f7282000640 1 --2- 172.21.15.47:0/669644634 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72600a3090 0x7f72600a5550 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.312+0000 7f7282000640 1 --2- 172.21.15.47:0/669644634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f727c101290 0x7f727c1a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.312+0000 7f7282000640 1 --2- 172.21.15.47:0/669644634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f727c1008f0 0x7f727c19a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.312+0000 7f7282000640 1 --2- 172.21.15.47:0/669644634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f727c0fff90 0x7f727c199b10 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.312+0000 7f7282000640 1 -- 172.21.15.47:0/669644634 >> 172.21.15.47:0/669644634 conn(0x7f727c0fbb30 msgr2=0x7f727c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.312+0000 7f7282000640 1 -- 172.21.15.47:0/669644634 shutdown_connections 2024-03-21T11:34:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:33.312+0000 7f7282000640 1 -- 172.21.15.47:0/669644634 wait complete. 2024-03-21T11:34:33.559 INFO:tasks.workunit.client.0.smithi047.stdout: 1 842900 8.03 MB/sec execute 437 sec latency 126.300 ms 2024-03-21T11:34:34.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:34:34.377 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:34:34.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 845080 8.04 MB/sec execute 438 sec latency 12.046 ms 2024-03-21T11:34:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.600+0000 7f04f6a00640 1 -- 172.21.15.47:0/3495584436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f0108db0 msgr2=0x7f04f0109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.600+0000 7f04f6a00640 1 --2- 172.21.15.47:0/3495584436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f0108db0 0x7f04f0109230 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009900 tx=0x7f04dc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:34:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.600+0000 7f04f6a00640 1 -- 172.21.15.47:0/3495584436 shutdown_connections 2024-03-21T11:34:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.600+0000 7f04f6a00640 1 --2- 172.21.15.47:0/3495584436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04f0109770 0x7f04f0110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.601+0000 7f04f6a00640 1 --2- 172.21.15.47:0/3495584436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f0108db0 0x7f04f0109230 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.601+0000 7f04f6a00640 1 --2- 172.21.15.47:0/3495584436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04f0107bc0 0x7f04f0107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.601+0000 7f04f6a00640 1 -- 172.21.15.47:0/3495584436 >> 172.21.15.47:0/3495584436 conn(0x7f04f0076b70 msgr2=0x7f04f0078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.601+0000 7f04f6a00640 1 -- 172.21.15.47:0/3495584436 shutdown_connections 2024-03-21T11:34:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.601+0000 7f04f6a00640 1 -- 172.21.15.47:0/3495584436 wait complete. 2024-03-21T11:34:34.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.602+0000 7f04f6a00640 1 Processor -- start 2024-03-21T11:34:34.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.602+0000 7f04f6a00640 1 -- start start 2024-03-21T11:34:34.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.602+0000 7f04f6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f0107bc0 0x7f04f019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.602+0000 7f04f6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04f0108db0 0x7f04f019e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.602+0000 7f04f6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04f0109770 0x7f04f01a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.602+0000 7f04f6a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f0113870 con 0x7f04f0108db0 2024-03-21T11:34:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.602+0000 7f04f6a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f01136f0 con 0x7f04f0107bc0 2024-03-21T11:34:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.602+0000 7f04f6a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f01139f0 con 0x7f04f0109770 2024-03-21T11:34:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.602+0000 7f04f5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f0107bc0 0x7f04f019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.602+0000 7f04f5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f0107bc0 0x7f04f019de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58688/0 (socket says 172.21.15.47:58688) 2024-03-21T11:34:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.602+0000 7f04f4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04f0108db0 0x7f04f019e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.602+0000 7f04f5600640 1 -- 172.21.15.47:0/4249614447 learned_addr learned my addr 172.21.15.47:0/4249614447 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.602+0000 7f04f6000640 1 --2- 172.21.15.47:0/4249614447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04f0109770 0x7f04f01a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.603+0000 7f04f5600640 1 -- 172.21.15.47:0/4249614447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04f0109770 msgr2=0x7f04f01a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.603+0000 7f04f5600640 1 --2- 172.21.15.47:0/4249614447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04f0109770 0x7f04f01a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.603+0000 7f04f5600640 1 -- 172.21.15.47:0/4249614447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04f0108db0 msgr2=0x7f04f019e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.603+0000 7f04f5600640 1 --2- 172.21.15.47:0/4249614447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04f0108db0 0x7f04f019e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.603+0000 7f04f5600640 1 -- 172.21.15.47:0/4249614447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f01a5a70 con 0x7f04f0107bc0 2024-03-21T11:34:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.603+0000 7f04f5600640 1 --2- 172.21.15.47:0/4249614447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f0107bc0 0x7f04f019de50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f04e400e9c0 tx=0x7f04e400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:34.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.603+0000 7f04ee000640 1 -- 172.21.15.47:0/4249614447 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e400cd80 con 0x7f04f0107bc0 2024-03-21T11:34:34.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.603+0000 7f04ee000640 1 -- 172.21.15.47:0/4249614447 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04e40035f0 con 0x7f04f0107bc0 2024-03-21T11:34:34.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.603+0000 7f04ee000640 1 -- 172.21.15.47:0/4249614447 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4010690 con 0x7f04f0107bc0 2024-03-21T11:34:34.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.603+0000 7f04f6a00640 1 -- 172.21.15.47:0/4249614447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f01a5d60 con 0x7f04f0107bc0 2024-03-21T11:34:34.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.604+0000 7f04f6a00640 1 -- 172.21.15.47:0/4249614447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f01a6210 con 0x7f04f0107bc0 2024-03-21T11:34:34.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.607+0000 7f04fad96740 1 Processor -- start 2024-03-21T11:34:34.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.607+0000 7f04fad96740 1 -- start start 2024-03-21T11:34:34.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.607+0000 7f04ee000640 1 -- 172.21.15.47:0/4249614447 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f04e40037a0 con 0x7f04f0107bc0 2024-03-21T11:34:34.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.607+0000 7f04fad96740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e850470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.607+0000 7f04fad96740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e850a40 0x55ca2e855540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.607+0000 7f04fad96740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e855bb0 0x55ca2e858040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.607+0000 7f04fad96740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2e7b5250 con 0x55ca2e855bb0 2024-03-21T11:34:34.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.607+0000 7f04fad96740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2e75da30 con 0x55ca2e850a40 2024-03-21T11:34:34.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.607+0000 7f04fad96740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2e7b12f0 con 0x55ca2e850050 2024-03-21T11:34:34.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.607+0000 7f04f6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e850470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.608+0000 7f04f6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e850470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58820/0 (socket says 172.21.15.47:58820) 2024-03-21T11:34:34.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.608+0000 7f04f6000640 1 -- 172.21.15.47:0/1095569487 learned_addr learned my addr 172.21.15.47:0/1095569487 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:34.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.608+0000 7f04f5600640 1 --2- 172.21.15.47:0/1095569487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e850a40 0x55ca2e855540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.608+0000 7f04f4c00640 1 --2- 172.21.15.47:0/1095569487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e855bb0 0x55ca2e858040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.608+0000 7f04f6000640 1 -- 172.21.15.47:0/1095569487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e850a40 msgr2=0x55ca2e855540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:34.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.608+0000 7f04f6000640 1 --2- 172.21.15.47:0/1095569487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e850a40 0x55ca2e855540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.608+0000 7f04f6000640 1 -- 172.21.15.47:0/1095569487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e855bb0 msgr2=0x55ca2e858040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:34.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.608+0000 7f04f6000640 1 --2- 172.21.15.47:0/1095569487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e855bb0 0x55ca2e858040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.608+0000 7f04f6000640 1 -- 172.21.15.47:0/1095569487 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca2e748690 con 0x55ca2e850050 2024-03-21T11:34:34.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.608+0000 7f04f4c00640 1 --2- 172.21.15.47:0/1095569487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e855bb0 0x55ca2e858040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:34:34.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.608+0000 7f04f6000640 1 --2- 172.21.15.47:0/1095569487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e850470 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009fd0 tx=0x7f04e80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88a5ebdde7ab1fe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.608+0000 7f04e2600640 1 -- 172.21.15.47:0/1095569487 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e8004d20 con 0x55ca2e850050 2024-03-21T11:34:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.608+0000 7f04e2600640 1 -- 172.21.15.47:0/1095569487 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04e8004050 con 0x55ca2e850050 2024-03-21T11:34:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.608+0000 7f04e2600640 1 -- 172.21.15.47:0/1095569487 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e8005260 con 0x55ca2e850050 2024-03-21T11:34:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.609+0000 7f04ee000640 1 --2- 172.21.15.47:0/4249614447 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f04d80a2fa0 0x7f04d80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.609+0000 7f04ee000640 1 -- 172.21.15.47:0/4249614447 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f04e40cfb40 con 0x7f04f0107bc0 2024-03-21T11:34:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.609+0000 7f04f5600640 1 --2- 172.21.15.47:0/4249614447 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f04d80a2fa0 0x7f04d80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.609+0000 7f04fad96740 1 -- 172.21.15.47:0/1095569487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 msgr2=0x55ca2e850470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.609+0000 7f04fad96740 1 --2- 172.21.15.47:0/1095569487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e850470 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009fd0 tx=0x7f04e80118d0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.609+0000 7f04fad96740 1 -- 172.21.15.47:0/1095569487 shutdown_connections 2024-03-21T11:34:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.609+0000 7f04fad96740 1 --2- 172.21.15.47:0/1095569487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e855bb0 0x55ca2e858040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.609+0000 7f04fad96740 1 --2- 172.21.15.47:0/1095569487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e850a40 0x55ca2e855540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.609+0000 7f04fad96740 1 --2- 172.21.15.47:0/1095569487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e850470 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.609+0000 7f04fad96740 1 -- 172.21.15.47:0/1095569487 >> 172.21.15.47:0/1095569487 conn(0x55ca2e818200 msgr2=0x55ca2e84b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:34.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.609+0000 7f04fad96740 1 -- 172.21.15.47:0/1095569487 shutdown_connections 2024-03-21T11:34:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.610+0000 7f04fad96740 1 -- 172.21.15.47:0/1095569487 wait complete. 2024-03-21T11:34:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.610+0000 7f04f5600640 1 --2- 172.21.15.47:0/4249614447 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f04d80a2fa0 0x7f04d80a5460 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f04e4098850 tx=0x7f04e40d3580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.610+0000 7f04fad96740 1 -- start start 2024-03-21T11:34:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.610+0000 7f04fad96740 1 Processor -- start 2024-03-21T11:34:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.611+0000 7f04fad96740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e85a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.611+0000 7f04fad96740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850a40 0x55ca2e85a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.611+0000 7f04fad96740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 0x55ca2e861600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.611+0000 7f04fad96740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2e858b60 con 0x55ca2e850a40 2024-03-21T11:34:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.611+0000 7f04fad96740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2e770e80 con 0x55ca2e855bb0 2024-03-21T11:34:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.611+0000 7f04fad96740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2e858ce0 con 0x55ca2e850050 2024-03-21T11:34:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.611+0000 7f04fad96740 10 client.0 _ll_register_callbacks cb 0x55ca2e783130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.611+0000 7f04f6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e85a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.611+0000 7f04f6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e85a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58836/0 (socket says 172.21.15.47:58836) 2024-03-21T11:34:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.611+0000 7f04f6000640 1 -- 172.21.15.47:0/1884636882 learned_addr learned my addr 172.21.15.47:0/1884636882 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.611+0000 7f04f4c00640 1 --2- 172.21.15.47:0/1884636882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850a40 0x55ca2e85a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.611+0000 7f04f6000640 1 --2- 172.21.15.47:0/1884636882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 0x55ca2e861600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.611+0000 7f04f4c00640 1 -- 172.21.15.47:0/1884636882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 msgr2=0x55ca2e85a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.612+0000 7f04f4c00640 1 --2- 172.21.15.47:0/1884636882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e85a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.612+0000 7f04f4c00640 1 -- 172.21.15.47:0/1884636882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 msgr2=0x55ca2e861600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.612+0000 7f04f4c00640 1 --2- 172.21.15.47:0/1884636882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 0x55ca2e861600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.612+0000 7f04f4c00640 1 -- 172.21.15.47:0/1884636882 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca2e5937d0 con 0x55ca2e850a40 2024-03-21T11:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.612+0000 7f04f6000640 1 --2- 172.21.15.47:0/1884636882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e85a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.612+0000 7f04f4c00640 1 --2- 172.21.15.47:0/1884636882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850a40 0x55ca2e85a5c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f04dc036040 tx=0x7f04dc004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83f93c58b0235e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.612+0000 7f04d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.612+0000 7f04d6a00640 1 -- 172.21.15.47:0/1884636882 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc005d50 con 0x55ca2e850a40 2024-03-21T11:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.612+0000 7f04fad96740 1 -- 172.21.15.47:0/1884636882 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04e8004050 con 0x55ca2e850a40 2024-03-21T11:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.612+0000 7f04d6a00640 1 -- 172.21.15.47:0/1884636882 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f04dc0046c0 con 0x55ca2e850a40 2024-03-21T11:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.613+0000 7f04d6a00640 1 -- 172.21.15.47:0/1884636882 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04dc052050 con 0x55ca2e850a40 2024-03-21T11:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.613+0000 7f04fad96740 10 client.18882 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.613+0000 7f04fad96740 1 -- 172.21.15.47:0/1884636882 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ca2e4c93f0 con 0x55ca2e850a40 2024-03-21T11:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.613+0000 7f04d6a00640 1 -- 172.21.15.47:0/1884636882 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04e8004050 con 0x55ca2e850a40 2024-03-21T11:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.613+0000 7f04d6a00640 1 -- 172.21.15.47:0/1884636882 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f04dc0053f0 con 0x55ca2e850a40 2024-03-21T11:34:34.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.613+0000 7f04fad96740 10 client.18882 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:34.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.613+0000 7f04fad96740 10 client.18882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:34.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.613+0000 7f04fad96740 20 client.18882 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:34.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.613+0000 7f04fad96740 1 --2- 172.21.15.47:0/1884636882 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca2e865e80 0x55ca2e8682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.613+0000 7f04fad96740 4 client.18882 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:34.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.613+0000 7f04fad96740 1 -- 172.21.15.47:0/1884636882 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca2e7c2790 con 0x55ca2e865e80 2024-03-21T11:34:34.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.613+0000 7f04f6000640 1 --2- 172.21.15.47:0/1884636882 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca2e865e80 0x55ca2e8682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.614+0000 7f04f6000640 1 --2- 172.21.15.47:0/1884636882 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca2e865e80 0x55ca2e8682c0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e9aba714622bcd3 server_cookie=d84b4bc88d8eefcf in_seq=0 out_seq=0 2024-03-21T11:34:34.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.614+0000 7f04d6a00640 10 client.18882 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:34.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.616+0000 7f04d6a00640 1 -- 172.21.15.47:0/1884636882 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ca2e7c2790 con 0x55ca2e865e80 2024-03-21T11:34:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.616+0000 7f04d6a00640 10 client.18882 handle_command_reply: tid=0 2024-03-21T11:34:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.616+0000 7f04fad96740 1 client.18882 shutdown 2024-03-21T11:34:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.616+0000 7f04fad96740 1 -- 172.21.15.47:0/1884636882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850a40 msgr2=0x55ca2e85a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.616+0000 7f04fad96740 1 --2- 172.21.15.47:0/1884636882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850a40 0x55ca2e85a5c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f04dc036040 tx=0x7f04dc004250 comp rx=0 tx=0).stop 2024-03-21T11:34:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.616+0000 7f04fad96740 1 -- 172.21.15.47:0/1884636882 shutdown_connections 2024-03-21T11:34:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.616+0000 7f04fad96740 1 --2- 172.21.15.47:0/1884636882 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca2e865e80 0x55ca2e8682c0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.616+0000 7f04fad96740 1 --2- 172.21.15.47:0/1884636882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 0x55ca2e861600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.616+0000 7f04fad96740 1 --2- 172.21.15.47:0/1884636882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850a40 0x55ca2e85a5c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.616+0000 7f04fad96740 1 --2- 172.21.15.47:0/1884636882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e85a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.616+0000 7f04d6a00640 0 client.18882 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.616+0000 7f04fad96740 1 -- 172.21.15.47:0/1884636882 >> 172.21.15.47:0/1884636882 conn(0x55ca2e818200 msgr2=0x55ca2e84abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.616+0000 7f04fad96740 1 -- 172.21.15.47:0/1884636882 shutdown_connections 2024-03-21T11:34:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.617+0000 7f04fad96740 1 -- 172.21.15.47:0/1884636882 wait complete. 2024-03-21T11:34:34.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.617+0000 7f04fad96740 20 client.18882 trim_cache size 0 max 16384 2024-03-21T11:34:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.635+0000 7f04fad96740 1 Processor -- start 2024-03-21T11:34:34.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.635+0000 7f04fad96740 1 -- start start 2024-03-21T11:34:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.635+0000 7f04fad96740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850050 0x55ca2e84ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.635+0000 7f04fad96740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850a40 0x55ca2e8615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.635+0000 7f04fad96740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 0x55ca2e84f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.635+0000 7f04fad96740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f01b04a0 con 0x55ca2e850050 2024-03-21T11:34:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.635+0000 7f04fad96740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f01b1e20 con 0x55ca2e855bb0 2024-03-21T11:34:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.635+0000 7f04fad96740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f01b0b00 con 0x55ca2e850a40 2024-03-21T11:34:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.636+0000 7f04f6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850050 0x55ca2e84ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.636+0000 7f04f6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850050 0x55ca2e84ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42304/0 (socket says 172.21.15.47:42304) 2024-03-21T11:34:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.636+0000 7f04f6000640 1 -- 172.21.15.47:0/2480377042 learned_addr learned my addr 172.21.15.47:0/2480377042 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.636+0000 7f04f4c00640 1 --2- 172.21.15.47:0/2480377042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850a40 0x55ca2e8615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.636+0000 7f04f6000640 1 --2- 172.21.15.47:0/2480377042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 0x55ca2e84f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.636+0000 7f04f4c00640 1 -- 172.21.15.47:0/2480377042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 msgr2=0x55ca2e84f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.636+0000 7f04f4c00640 1 --2- 172.21.15.47:0/2480377042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 0x55ca2e84f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.636+0000 7f04f4c00640 1 -- 172.21.15.47:0/2480377042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850050 msgr2=0x55ca2e84ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.636+0000 7f04f4c00640 1 --2- 172.21.15.47:0/2480377042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850050 0x55ca2e84ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.636+0000 7f04f4c00640 1 -- 172.21.15.47:0/2480377042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca2e4890f0 con 0x55ca2e850a40 2024-03-21T11:34:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.636+0000 7f04f6000640 1 --2- 172.21.15.47:0/2480377042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850050 0x55ca2e84ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.637+0000 7f04f4c00640 1 --2- 172.21.15.47:0/2480377042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850a40 0x55ca2e8615c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f04dc02c240 tx=0x7f04dc02c690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=219e6420c50638ad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.637+0000 7f04e1c00640 1 -- 172.21.15.47:0/2480377042 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc05b070 con 0x55ca2e850a40 2024-03-21T11:34:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.637+0000 7f04e1c00640 1 -- 172.21.15.47:0/2480377042 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04dc009780 con 0x55ca2e850a40 2024-03-21T11:34:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.637+0000 7f04e1c00640 1 -- 172.21.15.47:0/2480377042 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc04f040 con 0x55ca2e850a40 2024-03-21T11:34:34.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.638+0000 7f04fad96740 1 -- 172.21.15.47:0/2480377042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850a40 msgr2=0x55ca2e8615c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.638+0000 7f04fad96740 1 --2- 172.21.15.47:0/2480377042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850a40 0x55ca2e8615c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f04dc02c240 tx=0x7f04dc02c690 comp rx=0 tx=0).stop 2024-03-21T11:34:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.638+0000 7f04fad96740 1 -- 172.21.15.47:0/2480377042 shutdown_connections 2024-03-21T11:34:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.638+0000 7f04fad96740 1 --2- 172.21.15.47:0/2480377042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 0x55ca2e84f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.638+0000 7f04fad96740 1 --2- 172.21.15.47:0/2480377042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850a40 0x55ca2e8615c0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.638+0000 7f04fad96740 1 --2- 172.21.15.47:0/2480377042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850050 0x55ca2e84ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.638+0000 7f04fad96740 1 -- 172.21.15.47:0/2480377042 >> 172.21.15.47:0/2480377042 conn(0x55ca2e818200 msgr2=0x55ca2e851d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.638+0000 7f04fad96740 1 -- 172.21.15.47:0/2480377042 shutdown_connections 2024-03-21T11:34:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.638+0000 7f04fad96740 1 -- 172.21.15.47:0/2480377042 wait complete. 2024-03-21T11:34:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.638+0000 7f04fad96740 1 -- start start 2024-03-21T11:34:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.638+0000 7f04fad96740 1 Processor -- start 2024-03-21T11:34:34.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04fad96740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e85f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04fad96740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850a40 0x55ca2e85f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04fad96740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 0x55ca2e860750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04fad96740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f01b17c0 con 0x55ca2e850a40 2024-03-21T11:34:34.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04fad96740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f01b1160 con 0x55ca2e855bb0 2024-03-21T11:34:34.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04fad96740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f01ae660 con 0x55ca2e850050 2024-03-21T11:34:34.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04fad96740 10 client.0 _ll_register_callbacks cb 0x7f04e4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:34.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04f6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e85f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04f6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e85f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58852/0 (socket says 172.21.15.47:58852) 2024-03-21T11:34:34.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04f6000640 1 -- 172.21.15.47:0/3594704918 learned_addr learned my addr 172.21.15.47:0/3594704918 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:34.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04f6000640 1 --2- 172.21.15.47:0/3594704918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 0x55ca2e860750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04f4c00640 1 --2- 172.21.15.47:0/3594704918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850a40 0x55ca2e85f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04f6000640 1 -- 172.21.15.47:0/3594704918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 msgr2=0x55ca2e860750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:34.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04f6000640 1 --2- 172.21.15.47:0/3594704918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 0x55ca2e860750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04f6000640 1 -- 172.21.15.47:0/3594704918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850a40 msgr2=0x55ca2e85f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04f6000640 1 --2- 172.21.15.47:0/3594704918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850a40 0x55ca2e85f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04f6000640 1 -- 172.21.15.47:0/3594704918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca2e860d40 con 0x55ca2e850050 2024-03-21T11:34:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.639+0000 7f04f4c00640 1 --2- 172.21.15.47:0/3594704918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850a40 0x55ca2e85f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.640+0000 7f04f6000640 1 --2- 172.21.15.47:0/3594704918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e85f200 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009690 tx=0x7f04e8004870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eca3bd2bb125b5f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.640+0000 7f04d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.640+0000 7f04d6a00640 1 -- 172.21.15.47:0/3594704918 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e802f050 con 0x55ca2e850050 2024-03-21T11:34:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.640+0000 7f04d6a00640 1 -- 172.21.15.47:0/3594704918 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f04e802f1f0 con 0x55ca2e850050 2024-03-21T11:34:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.640+0000 7f04fad96740 1 -- 172.21.15.47:0/3594704918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04dc009780 con 0x55ca2e850050 2024-03-21T11:34:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.640+0000 7f04d6a00640 1 -- 172.21.15.47:0/3594704918 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04e802f520 con 0x55ca2e850050 2024-03-21T11:34:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.640+0000 7f04fad96740 10 client.29251 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.640+0000 7f04fad96740 1 -- 172.21.15.47:0/3594704918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ca2e8c1c40 con 0x55ca2e850050 2024-03-21T11:34:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.640+0000 7f04d6a00640 1 -- 172.21.15.47:0/3594704918 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04dc009780 con 0x55ca2e850050 2024-03-21T11:34:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.641+0000 7f04d6a00640 1 -- 172.21.15.47:0/3594704918 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f04e8025040 con 0x55ca2e850050 2024-03-21T11:34:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.641+0000 7f04fad96740 10 client.29251 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.641+0000 7f04fad96740 10 client.29251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.641+0000 7f04fad96740 20 client.29251 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.641+0000 7f04fad96740 1 --2- 172.21.15.47:0/3594704918 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca2e8c2320 0x55ca2e8c4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.641+0000 7f04fad96740 4 client.29251 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:34:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.641+0000 7f04fad96740 1 -- 172.21.15.47:0/3594704918 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ca2e7bcae0 con 0x55ca2e8c2320 2024-03-21T11:34:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.641+0000 7f04f4c00640 1 --2- 172.21.15.47:0/3594704918 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca2e8c2320 0x55ca2e8c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.642+0000 7f04f4c00640 1 --2- 172.21.15.47:0/3594704918 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca2e8c2320 0x55ca2e8c4740 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67e2eb54a605f6bb server_cookie=1216abc654748f16 in_seq=0 out_seq=0 2024-03-21T11:34:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.642+0000 7f04d6a00640 10 client.29251 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.642+0000 7f04d6a00640 1 -- 172.21.15.47:0/3594704918 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ca2e7bcae0 con 0x55ca2e8c2320 2024-03-21T11:34:34.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.642+0000 7f04d6a00640 10 client.29251 handle_command_reply: tid=0 2024-03-21T11:34:34.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.642+0000 7f04fad96740 1 client.29251 shutdown 2024-03-21T11:34:34.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.642+0000 7f04fad96740 1 -- 172.21.15.47:0/3594704918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 msgr2=0x55ca2e85f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:34.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.642+0000 7f04fad96740 1 --2- 172.21.15.47:0/3594704918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e85f200 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009690 tx=0x7f04e8004870 comp rx=0 tx=0).stop 2024-03-21T11:34:34.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.643+0000 7f04fad96740 1 -- 172.21.15.47:0/3594704918 shutdown_connections 2024-03-21T11:34:34.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.643+0000 7f04fad96740 1 --2- 172.21.15.47:0/3594704918 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca2e8c2320 0x55ca2e8c4740 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.643+0000 7f04fad96740 1 --2- 172.21.15.47:0/3594704918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca2e855bb0 0x55ca2e860750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.643+0000 7f04fad96740 1 --2- 172.21.15.47:0/3594704918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca2e850a40 0x55ca2e85f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.643+0000 7f04d6a00640 0 client.29251 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:34.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.643+0000 7f04fad96740 1 --2- 172.21.15.47:0/3594704918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca2e850050 0x55ca2e85f200 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.643+0000 7f04fad96740 1 -- 172.21.15.47:0/3594704918 >> 172.21.15.47:0/3594704918 conn(0x55ca2e818200 msgr2=0x55ca2e84a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:34.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.643+0000 7f04fad96740 1 -- 172.21.15.47:0/3594704918 shutdown_connections 2024-03-21T11:34:34.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.643+0000 7f04fad96740 1 -- 172.21.15.47:0/3594704918 wait complete. 2024-03-21T11:34:34.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.643+0000 7f04fad96740 20 client.29251 trim_cache size 0 max 16384 2024-03-21T11:34:34.649 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:34:34.649 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:34:34.649 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:34:34.649 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:34:34.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.644+0000 7f04f6a00640 1 -- 172.21.15.47:0/4249614447 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f04d80a2fa0 msgr2=0x7f04d80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:34.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.644+0000 7f04f6a00640 1 --2- 172.21.15.47:0/4249614447 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f04d80a2fa0 0x7f04d80a5460 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f04e4098850 tx=0x7f04e40d3580 comp rx=0 tx=0).stop 2024-03-21T11:34:34.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.644+0000 7f04f6a00640 1 -- 172.21.15.47:0/4249614447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f0107bc0 msgr2=0x7f04f019de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:34.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.644+0000 7f04f6a00640 1 --2- 172.21.15.47:0/4249614447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f0107bc0 0x7f04f019de50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f04e400e9c0 tx=0x7f04e400ee80 comp rx=0 tx=0).stop 2024-03-21T11:34:34.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.644+0000 7f04f6a00640 1 -- 172.21.15.47:0/4249614447 shutdown_connections 2024-03-21T11:34:34.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.644+0000 7f04f6a00640 1 --2- 172.21.15.47:0/4249614447 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f04d80a2fa0 0x7f04d80a5460 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.644+0000 7f04f6a00640 1 --2- 172.21.15.47:0/4249614447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04f0109770 0x7f04f01a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.644+0000 7f04f6a00640 1 --2- 172.21.15.47:0/4249614447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04f0108db0 0x7f04f019e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.644+0000 7f04f6a00640 1 --2- 172.21.15.47:0/4249614447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04f0107bc0 0x7f04f019de50 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.644+0000 7f04f6a00640 1 -- 172.21.15.47:0/4249614447 >> 172.21.15.47:0/4249614447 conn(0x7f04f0076b70 msgr2=0x7f04f010c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:34.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.644+0000 7f04f6a00640 1 -- 172.21.15.47:0/4249614447 shutdown_connections 2024-03-21T11:34:34.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.644+0000 7f04f6a00640 1 -- 172.21.15.47:0/4249614447 wait complete. 2024-03-21T11:34:34.663 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:34:34.663 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:34:34.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:34 smithi047 ceph-mon[31351]: pgmap v737: 129 pgs: 129 active+clean; 39 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-03-21T11:34:34.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.879+0000 7f7148200640 1 -- 172.21.15.47:0/805756385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7140102200 msgr2=0x7f71401045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:34.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.879+0000 7f7148200640 1 --2- 172.21.15.47:0/805756385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7140102200 0x7f71401045f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7134009900 tx=0x7f713402bc40 comp rx=0 tx=0).stop 2024-03-21T11:34:34.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.879+0000 7f7148200640 1 -- 172.21.15.47:0/805756385 shutdown_connections 2024-03-21T11:34:34.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.879+0000 7f7148200640 1 --2- 172.21.15.47:0/805756385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7140107460 0x7f7140109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.879+0000 7f7148200640 1 --2- 172.21.15.47:0/805756385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7140104b30 0x7f7140106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.879+0000 7f7148200640 1 --2- 172.21.15.47:0/805756385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7140102200 0x7f71401045f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.879+0000 7f7148200640 1 -- 172.21.15.47:0/805756385 >> 172.21.15.47:0/805756385 conn(0x7f71400fbb30 msgr2=0x7f71400fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.879+0000 7f7148200640 1 -- 172.21.15.47:0/805756385 shutdown_connections 2024-03-21T11:34:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.879+0000 7f7148200640 1 -- 172.21.15.47:0/805756385 wait complete. 2024-03-21T11:34:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.880+0000 7f7148200640 1 Processor -- start 2024-03-21T11:34:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.880+0000 7f7148200640 1 -- start start 2024-03-21T11:34:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.880+0000 7f7148200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7140102200 0x7f7140199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.880+0000 7f7148200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7140104b30 0x7f714019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.880+0000 7f7148200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7140107460 0x7f71401a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.880+0000 7f7148200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714010d350 con 0x7f7140107460 2024-03-21T11:34:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.881+0000 7f7148200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714010d1d0 con 0x7f7140104b30 2024-03-21T11:34:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.881+0000 7f7148200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714010d4d0 con 0x7f7140102200 2024-03-21T11:34:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.881+0000 7f7145800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7140102200 0x7f7140199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.881+0000 7f7145800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7140102200 0x7f7140199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58862/0 (socket says 172.21.15.47:58862) 2024-03-21T11:34:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.881+0000 7f7145800640 1 -- 172.21.15.47:0/1308661471 learned_addr learned my addr 172.21.15.47:0/1308661471 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.881+0000 7f7146200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7140107460 0x7f71401a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.881+0000 7f7144e00640 1 --2- 172.21.15.47:0/1308661471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7140104b30 0x7f714019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.881+0000 7f7145800640 1 -- 172.21.15.47:0/1308661471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7140104b30 msgr2=0x7f714019a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.881+0000 7f7145800640 1 --2- 172.21.15.47:0/1308661471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7140104b30 0x7f714019a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.881+0000 7f7145800640 1 -- 172.21.15.47:0/1308661471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7140107460 msgr2=0x7f71401a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.881+0000 7f7145800640 1 --2- 172.21.15.47:0/1308661471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7140107460 0x7f71401a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.881+0000 7f7145800640 1 -- 172.21.15.47:0/1308661471 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71401a1870 con 0x7f7140102200 2024-03-21T11:34:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.882+0000 7f7145800640 1 --2- 172.21.15.47:0/1308661471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7140102200 0x7f7140199c60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7134009230 tx=0x7f7134005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.882+0000 7f713e000640 1 -- 172.21.15.47:0/1308661471 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134041070 con 0x7f7140102200 2024-03-21T11:34:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.882+0000 7f713e000640 1 -- 172.21.15.47:0/1308661471 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71340044d0 con 0x7f7140102200 2024-03-21T11:34:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.882+0000 7f7148200640 1 -- 172.21.15.47:0/1308661471 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71401a1b00 con 0x7f7140102200 2024-03-21T11:34:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.882+0000 7f713e000640 1 -- 172.21.15.47:0/1308661471 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134034440 con 0x7f7140102200 2024-03-21T11:34:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.882+0000 7f7148200640 1 -- 172.21.15.47:0/1308661471 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71401a1ff0 con 0x7f7140102200 2024-03-21T11:34:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.883+0000 7f7148200640 1 -- 172.21.15.47:0/1308661471 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7140108f50 con 0x7f7140102200 2024-03-21T11:34:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.885+0000 7f713e000640 1 -- 172.21.15.47:0/1308661471 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7134044070 con 0x7f7140102200 2024-03-21T11:34:34.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.887+0000 7f713e000640 1 --2- 172.21.15.47:0/1308661471 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f71280a2f20 0x7f71280a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:34.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.887+0000 7f713e000640 1 -- 172.21.15.47:0/1308661471 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f71340f17c0 con 0x7f7140102200 2024-03-21T11:34:34.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.887+0000 7f7144e00640 1 --2- 172.21.15.47:0/1308661471 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f71280a2f20 0x7f71280a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:34.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.888+0000 7f7144e00640 1 --2- 172.21.15.47:0/1308661471 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f71280a2f20 0x7f71280a53e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f71400692a0 tx=0x7f712c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:34.892 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:34 smithi100 ceph-mon[36245]: pgmap v737: 129 pgs: 129 active+clean; 39 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-03-21T11:34:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:34.893+0000 7f713e000640 1 -- 172.21.15.47:0/1308661471 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f71340bdbb0 con 0x7f7140102200 2024-03-21T11:34:34.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:34 smithi102 ceph-mon[36748]: pgmap v737: 129 pgs: 129 active+clean; 39 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-03-21T11:34:35.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.141+0000 7f7148200640 1 -- 172.21.15.47:0/1308661471 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f714002d070 con 0x7f7140102200 2024-03-21T11:34:35.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.143+0000 7f713e000640 1 -- 172.21.15.47:0/1308661471 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f71340c1c50 con 0x7f7140102200 2024-03-21T11:34:35.144 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:34:35.144 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:34:35.144 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:34:35.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.145+0000 7f7148200640 1 -- 172.21.15.47:0/1308661471 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f71280a2f20 msgr2=0x7f71280a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.145+0000 7f7148200640 1 --2- 172.21.15.47:0/1308661471 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f71280a2f20 0x7f71280a53e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f71400692a0 tx=0x7f712c005f90 comp rx=0 tx=0).stop 2024-03-21T11:34:35.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.145+0000 7f7148200640 1 -- 172.21.15.47:0/1308661471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7140102200 msgr2=0x7f7140199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.145+0000 7f7148200640 1 --2- 172.21.15.47:0/1308661471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7140102200 0x7f7140199c60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7134009230 tx=0x7f7134005d50 comp rx=0 tx=0).stop 2024-03-21T11:34:35.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.145+0000 7f7148200640 1 -- 172.21.15.47:0/1308661471 shutdown_connections 2024-03-21T11:34:35.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.145+0000 7f7148200640 1 --2- 172.21.15.47:0/1308661471 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f71280a2f20 0x7f71280a53e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.145+0000 7f7148200640 1 --2- 172.21.15.47:0/1308661471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7140107460 0x7f71401a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.145+0000 7f7148200640 1 --2- 172.21.15.47:0/1308661471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7140104b30 0x7f714019a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.146+0000 7f7148200640 1 --2- 172.21.15.47:0/1308661471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7140102200 0x7f7140199c60 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.146+0000 7f7148200640 1 -- 172.21.15.47:0/1308661471 >> 172.21.15.47:0/1308661471 conn(0x7f71400fbb30 msgr2=0x7f71401019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:35.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.146+0000 7f7148200640 1 -- 172.21.15.47:0/1308661471 shutdown_connections 2024-03-21T11:34:35.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.146+0000 7f7148200640 1 -- 172.21.15.47:0/1308661471 wait complete. 2024-03-21T11:34:35.166 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:34:35.166 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:34:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.410+0000 7feb0f800640 1 -- 172.21.15.47:0/1871067270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb08105470 msgr2=0x7feb0810bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.410+0000 7feb0f800640 1 --2- 172.21.15.47:0/1871067270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb08105470 0x7feb0810bd00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7feb00009ca0 tx=0x7feb0002c100 comp rx=0 tx=0).stop 2024-03-21T11:34:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.410+0000 7feb0f800640 1 -- 172.21.15.47:0/1871067270 shutdown_connections 2024-03-21T11:34:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.410+0000 7feb0f800640 1 --2- 172.21.15.47:0/1871067270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb08105470 0x7feb0810bd00 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.410+0000 7feb0f800640 1 --2- 172.21.15.47:0/1871067270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb08104ab0 0x7feb08104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.410+0000 7feb0f800640 1 --2- 172.21.15.47:0/1871067270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb081038c0 0x7feb08103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.410+0000 7feb0f800640 1 -- 172.21.15.47:0/1871067270 >> 172.21.15.47:0/1871067270 conn(0x7feb080fee50 msgr2=0x7feb08101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.410+0000 7feb0f800640 1 -- 172.21.15.47:0/1871067270 shutdown_connections 2024-03-21T11:34:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.410+0000 7feb0f800640 1 -- 172.21.15.47:0/1871067270 wait complete. 2024-03-21T11:34:35.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.411+0000 7feb0f800640 1 Processor -- start 2024-03-21T11:34:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.411+0000 7feb0f800640 1 -- start start 2024-03-21T11:34:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0f800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb081038c0 0x7feb08199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0f800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb08104ab0 0x7feb0819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0f800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb08105470 0x7feb081a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0f800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0810f540 con 0x7feb08104ab0 2024-03-21T11:34:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0f800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0810f3c0 con 0x7feb081038c0 2024-03-21T11:34:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0f800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0810f6c0 con 0x7feb08105470 2024-03-21T11:34:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb081038c0 0x7feb08199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb08105470 0x7feb081a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb081038c0 0x7feb08199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58744/0 (socket says 172.21.15.47:58744) 2024-03-21T11:34:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0ce00640 1 -- 172.21.15.47:0/2330605344 learned_addr learned my addr 172.21.15.47:0/2330605344 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb07e00640 1 --2- 172.21.15.47:0/2330605344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb08104ab0 0x7feb0819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0d800640 1 -- 172.21.15.47:0/2330605344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb081038c0 msgr2=0x7feb08199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0d800640 1 --2- 172.21.15.47:0/2330605344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb081038c0 0x7feb08199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0d800640 1 -- 172.21.15.47:0/2330605344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb08104ab0 msgr2=0x7feb0819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0d800640 1 --2- 172.21.15.47:0/2330605344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb08104ab0 0x7feb0819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0d800640 1 -- 172.21.15.47:0/2330605344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb081a1770 con 0x7feb08105470 2024-03-21T11:34:35.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.412+0000 7feb0ce00640 1 --2- 172.21.15.47:0/2330605344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb081038c0 0x7feb08199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:35.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.413+0000 7feb0d800640 1 --2- 172.21.15.47:0/2330605344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb08105470 0x7feb081a1100 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7feb00037040 tx=0x7feb00035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:35.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.413+0000 7feb05600640 1 -- 172.21.15.47:0/2330605344 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00039040 con 0x7feb08105470 2024-03-21T11:34:35.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.413+0000 7feb05600640 1 -- 172.21.15.47:0/2330605344 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb00044070 con 0x7feb08105470 2024-03-21T11:34:35.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.413+0000 7feb05600640 1 -- 172.21.15.47:0/2330605344 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00004350 con 0x7feb08105470 2024-03-21T11:34:35.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.413+0000 7feb0f800640 1 -- 172.21.15.47:0/2330605344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb081a1a00 con 0x7feb08105470 2024-03-21T11:34:35.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.413+0000 7feb0f800640 1 -- 172.21.15.47:0/2330605344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb081a1e70 con 0x7feb08105470 2024-03-21T11:34:35.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.416+0000 7feb05600640 1 -- 172.21.15.47:0/2330605344 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7feb000044f0 con 0x7feb08105470 2024-03-21T11:34:35.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.417+0000 7feb1240e740 1 Processor -- start 2024-03-21T11:34:35.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.417+0000 7feb1240e740 1 -- start start 2024-03-21T11:34:35.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.417+0000 7feb1240e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c00810 0x55c406c00c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.417+0000 7feb1240e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c019e0 0x55c406c01e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.417+0000 7feb1240e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 0x55c406c08c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.417+0000 7feb1240e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c406b652c0 con 0x55c406c023a0 2024-03-21T11:34:35.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.417+0000 7feb1240e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c406b0dc90 con 0x55c406c00810 2024-03-21T11:34:35.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.417+0000 7feb1240e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c406b61360 con 0x55c406c019e0 2024-03-21T11:34:35.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.418+0000 7feb0ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c019e0 0x55c406c01e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.418+0000 7feb0ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c019e0 0x55c406c01e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58880/0 (socket says 172.21.15.47:58880) 2024-03-21T11:34:35.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.418+0000 7feb0ce00640 1 -- 172.21.15.47:0/2472365718 learned_addr learned my addr 172.21.15.47:0/2472365718 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.418+0000 7feb07e00640 1 --2- 172.21.15.47:0/2472365718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c00810 0x55c406c00c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.418+0000 7feb0ce00640 1 -- 172.21.15.47:0/2472365718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c00810 msgr2=0x55c406c00c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.418+0000 7feb0ce00640 1 --2- 172.21.15.47:0/2472365718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c00810 0x55c406c00c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.418+0000 7feb0ce00640 1 -- 172.21.15.47:0/2472365718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 msgr2=0x55c406c08c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.418+0000 7feb07e00640 1 --2- 172.21.15.47:0/2472365718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 0x55c406c08c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.418+0000 7feb0ce00640 1 --2- 172.21.15.47:0/2472365718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 0x55c406c08c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.418+0000 7feb0ce00640 1 -- 172.21.15.47:0/2472365718 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c406b32080 con 0x55c406c019e0 2024-03-21T11:34:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.418+0000 7feb07e00640 1 --2- 172.21.15.47:0/2472365718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 0x55c406c08c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:34:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.418+0000 7feb0ce00640 1 --2- 172.21.15.47:0/2472365718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c019e0 0x55c406c01e60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7feafc0115f0 tx=0x7feafc011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cc3eea8f3c4cd31 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.418+0000 7feb05600640 1 --2- 172.21.15.47:0/2330605344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feaf00a2fa0 0x7feaf00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.418+0000 7feafa000640 1 -- 172.21.15.47:0/2472365718 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc004020 con 0x55c406c019e0 2024-03-21T11:34:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.419+0000 7feafa000640 1 -- 172.21.15.47:0/2472365718 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feafc0041c0 con 0x55c406c019e0 2024-03-21T11:34:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.419+0000 7feafa000640 1 -- 172.21.15.47:0/2472365718 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc0039d0 con 0x55c406c019e0 2024-03-21T11:34:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.419+0000 7feb05600640 1 -- 172.21.15.47:0/2330605344 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7feb000f1c40 con 0x7feb08105470 2024-03-21T11:34:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.419+0000 7feb07e00640 1 --2- 172.21.15.47:0/2330605344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feaf00a2fa0 0x7feaf00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.419+0000 7feb1240e740 1 -- 172.21.15.47:0/2472365718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c019e0 msgr2=0x55c406c01e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.419+0000 7feb1240e740 1 --2- 172.21.15.47:0/2472365718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c019e0 0x55c406c01e60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7feafc0115f0 tx=0x7feafc011ab0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.420+0000 7feb1240e740 1 -- 172.21.15.47:0/2472365718 shutdown_connections 2024-03-21T11:34:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.420+0000 7feb1240e740 1 --2- 172.21.15.47:0/2472365718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 0x55c406c08c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.420+0000 7feb1240e740 1 --2- 172.21.15.47:0/2472365718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c019e0 0x55c406c01e60 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.420+0000 7feb1240e740 1 --2- 172.21.15.47:0/2472365718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c00810 0x55c406c00c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.420+0000 7feb1240e740 1 -- 172.21.15.47:0/2472365718 >> 172.21.15.47:0/2472365718 conn(0x55c406bfbc00 msgr2=0x55c406bfc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:35.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.420+0000 7feb07e00640 1 --2- 172.21.15.47:0/2330605344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feaf00a2fa0 0x7feaf00a5460 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55c406c018e0 tx=0x7feaf4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.420+0000 7feb1240e740 1 -- 172.21.15.47:0/2472365718 shutdown_connections 2024-03-21T11:34:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.420+0000 7feb1240e740 1 -- 172.21.15.47:0/2472365718 wait complete. 2024-03-21T11:34:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.420+0000 7feb1240e740 1 -- start start 2024-03-21T11:34:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.420+0000 7feb1240e740 1 Processor -- start 2024-03-21T11:34:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.421+0000 7feb1240e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c00810 0x55c406c0aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.421+0000 7feb1240e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c019e0 0x55c406c0afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.421+0000 7feb1240e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 0x55c406c12000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.421+0000 7feb1240e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c406c093b0 con 0x55c406c023a0 2024-03-21T11:34:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.421+0000 7feb1240e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c406b210d0 con 0x55c406c019e0 2024-03-21T11:34:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.421+0000 7feb1240e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c406c09530 con 0x55c406c00810 2024-03-21T11:34:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.421+0000 7feb1240e740 10 client.0 _ll_register_callbacks cb 0x55c406b25e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:35.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.421+0000 7feb0ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c00810 0x55c406c0aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.421+0000 7feb0ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c00810 0x55c406c0aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58884/0 (socket says 172.21.15.47:58884) 2024-03-21T11:34:35.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.421+0000 7feb0ce00640 1 -- 172.21.15.47:0/3005860200 learned_addr learned my addr 172.21.15.47:0/3005860200 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:35.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.421+0000 7feb07e00640 1 --2- 172.21.15.47:0/3005860200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c019e0 0x55c406c0afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.421+0000 7feb0d800640 1 --2- 172.21.15.47:0/3005860200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 0x55c406c12000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.422+0000 7feb0ce00640 1 -- 172.21.15.47:0/3005860200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c019e0 msgr2=0x55c406c0afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.422+0000 7feb0ce00640 1 --2- 172.21.15.47:0/3005860200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c019e0 0x55c406c0afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.422+0000 7feb0ce00640 1 -- 172.21.15.47:0/3005860200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 msgr2=0x55c406c12000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.422+0000 7feb0ce00640 1 --2- 172.21.15.47:0/3005860200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 0x55c406c12000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.422+0000 7feb0ce00640 1 -- 172.21.15.47:0/3005860200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4068793f0 con 0x55c406c00810 2024-03-21T11:34:35.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.422+0000 7feb07e00640 1 --2- 172.21.15.47:0/3005860200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c019e0 0x55c406c0afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:35.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.422+0000 7feb0ce00640 1 --2- 172.21.15.47:0/3005860200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c00810 0x55c406c0aa80 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7feafc01b040 tx=0x7feafc0045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f210c2f08cb4cd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:35.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.422+0000 7feaeea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:35.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.422+0000 7feaeea00640 1 -- 172.21.15.47:0/3005860200 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc027020 con 0x55c406c00810 2024-03-21T11:34:35.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.422+0000 7feaeea00640 1 -- 172.21.15.47:0/3005860200 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7feafc016040 con 0x55c406c00810 2024-03-21T11:34:35.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.422+0000 7feb1240e740 1 -- 172.21.15.47:0/3005860200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feafc0041c0 con 0x55c406c00810 2024-03-21T11:34:35.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.423+0000 7feaeea00640 1 -- 172.21.15.47:0/3005860200 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7feafc032050 con 0x55c406c00810 2024-03-21T11:34:35.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.423+0000 7feb1240e740 10 client.29269 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:35.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.423+0000 7feb1240e740 1 -- 172.21.15.47:0/3005860200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c4069437d0 con 0x55c406c00810 2024-03-21T11:34:35.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.423+0000 7feaeea00640 1 -- 172.21.15.47:0/3005860200 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feafc0041c0 con 0x55c406c00810 2024-03-21T11:34:35.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.423+0000 7feaeea00640 1 -- 172.21.15.47:0/3005860200 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7feafc0036d0 con 0x55c406c00810 2024-03-21T11:34:35.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.423+0000 7feb1240e740 10 client.29269 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:35.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.423+0000 7feb1240e740 10 client.29269 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:34:35.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.423+0000 7feb1240e740 20 client.29269 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:35.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.423+0000 7feb1240e740 1 --2- 172.21.15.47:0/3005860200 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c406c16230 0x55c406c18680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.423+0000 7feb1240e740 4 client.29269 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:35.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.423+0000 7feb1240e740 1 -- 172.21.15.47:0/3005860200 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c406b72800 con 0x55c406c16230 2024-03-21T11:34:35.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.424+0000 7feb07e00640 1 --2- 172.21.15.47:0/3005860200 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c406c16230 0x55c406c18680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.424+0000 7feb07e00640 1 --2- 172.21.15.47:0/3005860200 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c406c16230 0x55c406c18680 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d6dd77fef06f9999 server_cookie=bf56aec8aa82328f in_seq=0 out_seq=0 2024-03-21T11:34:35.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.424+0000 7feaeea00640 10 client.29269 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:34:35.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.425+0000 7feaeea00640 1 -- 172.21.15.47:0/3005860200 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c406b72800 con 0x55c406c16230 2024-03-21T11:34:35.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.425+0000 7feaeea00640 10 client.29269 handle_command_reply: tid=0 2024-03-21T11:34:35.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.426+0000 7feb1240e740 1 client.29269 shutdown 2024-03-21T11:34:35.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.426+0000 7feb1240e740 1 -- 172.21.15.47:0/3005860200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c00810 msgr2=0x55c406c0aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.426+0000 7feb1240e740 1 --2- 172.21.15.47:0/3005860200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c00810 0x55c406c0aa80 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7feafc01b040 tx=0x7feafc0045a0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.426+0000 7feb1240e740 1 -- 172.21.15.47:0/3005860200 shutdown_connections 2024-03-21T11:34:35.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.426+0000 7feb1240e740 1 --2- 172.21.15.47:0/3005860200 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c406c16230 0x55c406c18680 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.426+0000 7feb1240e740 1 --2- 172.21.15.47:0/3005860200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 0x55c406c12000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.426+0000 7feb1240e740 1 --2- 172.21.15.47:0/3005860200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c019e0 0x55c406c0afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.426+0000 7feb1240e740 1 --2- 172.21.15.47:0/3005860200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c00810 0x55c406c0aa80 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.426+0000 7feaeea00640 0 client.29269 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:34:35.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.426+0000 7feb1240e740 1 -- 172.21.15.47:0/3005860200 >> 172.21.15.47:0/3005860200 conn(0x55c406bfbc00 msgr2=0x55c406bfaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:35.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.426+0000 7feb1240e740 1 -- 172.21.15.47:0/3005860200 shutdown_connections 2024-03-21T11:34:35.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.426+0000 7feb1240e740 1 -- 172.21.15.47:0/3005860200 wait complete. 2024-03-21T11:34:35.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.426+0000 7feb1240e740 20 client.29269 trim_cache size 0 max 16384 2024-03-21T11:34:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.441+0000 7feb1240e740 1 Processor -- start 2024-03-21T11:34:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.441+0000 7feb1240e740 1 -- start start 2024-03-21T11:34:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.441+0000 7feb1240e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c00810 0x55c406c088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.441+0000 7feb1240e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c019e0 0x55c406c04a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.442+0000 7feb1240e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 0x55c406c04fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.442+0000 7feb1240e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081ac270 con 0x55c406c023a0 2024-03-21T11:34:35.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.442+0000 7feb1240e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081adbf0 con 0x55c406c019e0 2024-03-21T11:34:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.442+0000 7feb1240e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081ac8d0 con 0x55c406c00810 2024-03-21T11:34:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.442+0000 7feb0ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c00810 0x55c406c088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.442+0000 7feb0ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c00810 0x55c406c088c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58894/0 (socket says 172.21.15.47:58894) 2024-03-21T11:34:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.442+0000 7feb0ce00640 1 -- 172.21.15.47:0/3446689278 learned_addr learned my addr 172.21.15.47:0/3446689278 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.442+0000 7feb0d800640 1 --2- 172.21.15.47:0/3446689278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 0x55c406c04fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.442+0000 7feb07e00640 1 --2- 172.21.15.47:0/3446689278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c019e0 0x55c406c04a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.442+0000 7feb0ce00640 1 -- 172.21.15.47:0/3446689278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c019e0 msgr2=0x55c406c04a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.442+0000 7feb0ce00640 1 --2- 172.21.15.47:0/3446689278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c019e0 0x55c406c04a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.442+0000 7feb0ce00640 1 -- 172.21.15.47:0/3446689278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 msgr2=0x55c406c04fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.442+0000 7feb0ce00640 1 --2- 172.21.15.47:0/3446689278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 0x55c406c04fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.442+0000 7feb0ce00640 1 -- 172.21.15.47:0/3446689278 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c406c09ea0 con 0x55c406c00810 2024-03-21T11:34:35.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.443+0000 7feb0ce00640 1 --2- 172.21.15.47:0/3446689278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c00810 0x55c406c088c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7feafc011510 tx=0x7feafc005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b1dbd500e7cdf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.443+0000 7feaf9600640 1 -- 172.21.15.47:0/3446689278 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc027040 con 0x55c406c00810 2024-03-21T11:34:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.443+0000 7feaf9600640 1 -- 172.21.15.47:0/3446689278 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feafc016030 con 0x55c406c00810 2024-03-21T11:34:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.443+0000 7feaf9600640 1 -- 172.21.15.47:0/3446689278 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc003fd0 con 0x55c406c00810 2024-03-21T11:34:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.444+0000 7feb1240e740 1 -- 172.21.15.47:0/3446689278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c00810 msgr2=0x55c406c088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.444+0000 7feb1240e740 1 --2- 172.21.15.47:0/3446689278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c00810 0x55c406c088c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7feafc011510 tx=0x7feafc005970 comp rx=0 tx=0).stop 2024-03-21T11:34:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.444+0000 7feb1240e740 1 -- 172.21.15.47:0/3446689278 shutdown_connections 2024-03-21T11:34:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.444+0000 7feb1240e740 1 --2- 172.21.15.47:0/3446689278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c023a0 0x55c406c04fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.444+0000 7feb1240e740 1 --2- 172.21.15.47:0/3446689278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c019e0 0x55c406c04a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.444+0000 7feb1240e740 1 --2- 172.21.15.47:0/3446689278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c00810 0x55c406c088c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.444+0000 7feb1240e740 1 -- 172.21.15.47:0/3446689278 >> 172.21.15.47:0/3446689278 conn(0x55c406bfbc00 msgr2=0x55c406bff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:35.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.444+0000 7feb1240e740 1 -- 172.21.15.47:0/3446689278 shutdown_connections 2024-03-21T11:34:35.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.444+0000 7feb1240e740 1 -- 172.21.15.47:0/3446689278 wait complete. 2024-03-21T11:34:35.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.444+0000 7feb1240e740 1 -- start start 2024-03-21T11:34:35.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.445+0000 7feb1240e740 1 Processor -- start 2024-03-21T11:34:35.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.445+0000 7feb1240e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c00810 0x55c406c0c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.445+0000 7feb1240e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c019e0 0x55c406c0cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.445+0000 7feb1240e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c023a0 0x55c406c0fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.445+0000 7feb1240e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081ad590 con 0x55c406c019e0 2024-03-21T11:34:35.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.445+0000 7feb1240e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081acf30 con 0x55c406c00810 2024-03-21T11:34:35.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.445+0000 7feb1240e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081aa430 con 0x55c406c023a0 2024-03-21T11:34:35.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.445+0000 7feb1240e740 10 client.0 _ll_register_callbacks cb 0x7feafc011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:35.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.445+0000 7feb0ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c00810 0x55c406c0c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.445+0000 7feb07e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c019e0 0x55c406c0cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.445+0000 7feb0ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c00810 0x55c406c0c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58780/0 (socket says 172.21.15.47:58780) 2024-03-21T11:34:35.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.445+0000 7feb0d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c023a0 0x55c406c0fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.445+0000 7feb0ce00640 1 -- 172.21.15.47:0/161655316 learned_addr learned my addr 172.21.15.47:0/161655316 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:35.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.446+0000 7feb0ce00640 1 -- 172.21.15.47:0/161655316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c023a0 msgr2=0x55c406c0fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.446+0000 7feb0ce00640 1 --2- 172.21.15.47:0/161655316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c023a0 0x55c406c0fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.446+0000 7feb0ce00640 1 -- 172.21.15.47:0/161655316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c019e0 msgr2=0x55c406c0cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.446+0000 7feb0ce00640 1 --2- 172.21.15.47:0/161655316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c019e0 0x55c406c0cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.446+0000 7feb0ce00640 1 -- 172.21.15.47:0/161655316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c406c10360 con 0x55c406c00810 2024-03-21T11:34:35.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.446+0000 7feb0d800640 1 --2- 172.21.15.47:0/161655316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c023a0 0x55c406c0fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:35.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.446+0000 7feb0ce00640 1 --2- 172.21.15.47:0/161655316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c00810 0x55c406c0c860 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7feafc003a70 tx=0x7feafc0092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9878d1e1e657019 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:35.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.446+0000 7feaeea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:35.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.446+0000 7feaeea00640 1 -- 172.21.15.47:0/161655316 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc027070 con 0x55c406c00810 2024-03-21T11:34:35.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.446+0000 7feb1240e740 1 -- 172.21.15.47:0/161655316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feafc016030 con 0x55c406c00810 2024-03-21T11:34:35.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.446+0000 7feaeea00640 1 -- 172.21.15.47:0/161655316 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7feafc005460 con 0x55c406c00810 2024-03-21T11:34:35.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.447+0000 7feaeea00640 1 -- 172.21.15.47:0/161655316 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7feafc005660 con 0x55c406c00810 2024-03-21T11:34:35.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.447+0000 7feb1240e740 10 client.29255 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:35.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.447+0000 7feb1240e740 1 -- 172.21.15.47:0/161655316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c406c11660 con 0x55c406c00810 2024-03-21T11:34:35.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.447+0000 7feaeea00640 1 -- 172.21.15.47:0/161655316 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feafc016030 con 0x55c406c00810 2024-03-21T11:34:35.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.447+0000 7feaeea00640 1 -- 172.21.15.47:0/161655316 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7feafc004c20 con 0x55c406c00810 2024-03-21T11:34:35.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.447+0000 7feb1240e740 10 client.29255 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:35.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.447+0000 7feb1240e740 10 client.29255 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:34:35.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.447+0000 7feb1240e740 20 client.29255 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:35.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.447+0000 7feb1240e740 1 --2- 172.21.15.47:0/161655316 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c406c71cb0 0x55c406c740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.447+0000 7feb1240e740 4 client.29255 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:34:35.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.447+0000 7feb1240e740 1 -- 172.21.15.47:0/161655316 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c406b6cb50 con 0x55c406c71cb0 2024-03-21T11:34:35.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.447+0000 7feb07e00640 1 --2- 172.21.15.47:0/161655316 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c406c71cb0 0x55c406c740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.448+0000 7feb07e00640 1 --2- 172.21.15.47:0/161655316 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c406c71cb0 0x55c406c740a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b98058834c67abfb server_cookie=130a7b2dde107a7b in_seq=0 out_seq=0 2024-03-21T11:34:35.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.448+0000 7feaeea00640 10 client.29255 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:34:35.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.448+0000 7feaeea00640 1 -- 172.21.15.47:0/161655316 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c406b6cb50 con 0x55c406c71cb0 2024-03-21T11:34:35.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.448+0000 7feaeea00640 10 client.29255 handle_command_reply: tid=0 2024-03-21T11:34:35.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.448+0000 7feb1240e740 1 client.29255 shutdown 2024-03-21T11:34:35.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.448+0000 7feb1240e740 1 -- 172.21.15.47:0/161655316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c00810 msgr2=0x55c406c0c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.448+0000 7feb1240e740 1 --2- 172.21.15.47:0/161655316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c00810 0x55c406c0c860 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7feafc003a70 tx=0x7feafc0092f0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.448+0000 7feb1240e740 1 -- 172.21.15.47:0/161655316 shutdown_connections 2024-03-21T11:34:35.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.448+0000 7feb1240e740 1 --2- 172.21.15.47:0/161655316 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c406c71cb0 0x55c406c740a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.448+0000 7feb1240e740 1 --2- 172.21.15.47:0/161655316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c406c023a0 0x55c406c0fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.448+0000 7feaeea00640 0 client.29255 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:34:35.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.448+0000 7feb1240e740 1 --2- 172.21.15.47:0/161655316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c406c019e0 0x55c406c0cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.448+0000 7feb1240e740 1 --2- 172.21.15.47:0/161655316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c406c00810 0x55c406c0c860 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.448+0000 7feb1240e740 1 -- 172.21.15.47:0/161655316 >> 172.21.15.47:0/161655316 conn(0x55c406bfbc00 msgr2=0x55c406bff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:35.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.449+0000 7feb1240e740 1 -- 172.21.15.47:0/161655316 shutdown_connections 2024-03-21T11:34:35.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.449+0000 7feb1240e740 1 -- 172.21.15.47:0/161655316 wait complete. 2024-03-21T11:34:35.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.449+0000 7feb1240e740 20 client.29255 trim_cache size 0 max 16384 2024-03-21T11:34:35.454 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:34:35.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.449+0000 7feb0f800640 1 -- 172.21.15.47:0/2330605344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feaf00a2fa0 msgr2=0x7feaf00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.449+0000 7feb0f800640 1 --2- 172.21.15.47:0/2330605344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feaf00a2fa0 0x7feaf00a5460 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55c406c018e0 tx=0x7feaf4005f90 comp rx=0 tx=0).stop 2024-03-21T11:34:35.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.449+0000 7feb0f800640 1 -- 172.21.15.47:0/2330605344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb08105470 msgr2=0x7feb081a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.449+0000 7feb0f800640 1 --2- 172.21.15.47:0/2330605344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb08105470 0x7feb081a1100 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7feb00037040 tx=0x7feb00035d10 comp rx=0 tx=0).stop 2024-03-21T11:34:35.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.449+0000 7feb0f800640 1 -- 172.21.15.47:0/2330605344 shutdown_connections 2024-03-21T11:34:35.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.449+0000 7feb0f800640 1 --2- 172.21.15.47:0/2330605344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7feaf00a2fa0 0x7feaf00a5460 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.449+0000 7feb0f800640 1 --2- 172.21.15.47:0/2330605344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb08105470 0x7feb081a1100 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.449+0000 7feb0f800640 1 --2- 172.21.15.47:0/2330605344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb08104ab0 0x7feb0819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.449+0000 7feb0f800640 1 --2- 172.21.15.47:0/2330605344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb081038c0 0x7feb08199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.449+0000 7feb0f800640 1 -- 172.21.15.47:0/2330605344 >> 172.21.15.47:0/2330605344 conn(0x7feb080fee50 msgr2=0x7feb08107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:35.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.450+0000 7feb0f800640 1 -- 172.21.15.47:0/2330605344 shutdown_connections 2024-03-21T11:34:35.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.450+0000 7feb0f800640 1 -- 172.21.15.47:0/2330605344 wait complete. 2024-03-21T11:34:35.472 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:34:35.472 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:34:35.560 INFO:tasks.workunit.client.0.smithi047.stdout: 1 846093 8.03 MB/sec execute 439 sec latency 10.669 ms 2024-03-21T11:34:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.696+0000 7f0c5f200640 1 -- 172.21.15.47:0/68161879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c600ffdf0 msgr2=0x7f0c60100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.696+0000 7f0c5f200640 1 --2- 172.21.15.47:0/68161879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c600ffdf0 0x7f0c60100210 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009900 tx=0x7f0c5002bc30 comp rx=0 tx=0).stop 2024-03-21T11:34:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.696+0000 7f0c5f200640 1 -- 172.21.15.47:0/68161879 shutdown_connections 2024-03-21T11:34:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.696+0000 7f0c5f200640 1 --2- 172.21.15.47:0/68161879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c601010f0 0x7f0c60101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.696+0000 7f0c5f200640 1 --2- 172.21.15.47:0/68161879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c60100750 0x7f0c60100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.696+0000 7f0c5f200640 1 --2- 172.21.15.47:0/68161879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c600ffdf0 0x7f0c60100210 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.696+0000 7f0c5f200640 1 -- 172.21.15.47:0/68161879 >> 172.21.15.47:0/68161879 conn(0x7f0c600fba10 msgr2=0x7f0c600fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.696+0000 7f0c5f200640 1 -- 172.21.15.47:0/68161879 shutdown_connections 2024-03-21T11:34:35.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.697+0000 7f0c5f200640 1 -- 172.21.15.47:0/68161879 wait complete. 2024-03-21T11:34:35.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.698+0000 7f0c5f200640 1 Processor -- start 2024-03-21T11:34:35.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.698+0000 7f0c5f200640 1 -- start start 2024-03-21T11:34:35.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.698+0000 7f0c5f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c600ffdf0 0x7f0c60105970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c60100750 0x7f0c60105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c601010f0 0x7f0c601024e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6010f410 con 0x7f0c601010f0 2024-03-21T11:34:35.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6010f290 con 0x7f0c600ffdf0 2024-03-21T11:34:35.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6010f590 con 0x7f0c60100750 2024-03-21T11:34:35.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c60100750 0x7f0c60105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c60100750 0x7f0c60105eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58914/0 (socket says 172.21.15.47:58914) 2024-03-21T11:34:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5d400640 1 -- 172.21.15.47:0/3811888988 learned_addr learned my addr 172.21.15.47:0/3811888988 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5de00640 1 --2- 172.21.15.47:0/3811888988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c600ffdf0 0x7f0c60105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5e800640 1 --2- 172.21.15.47:0/3811888988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c601010f0 0x7f0c601024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5d400640 1 -- 172.21.15.47:0/3811888988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c600ffdf0 msgr2=0x7f0c60105970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5d400640 1 --2- 172.21.15.47:0/3811888988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c600ffdf0 0x7f0c60105970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5d400640 1 -- 172.21.15.47:0/3811888988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c601010f0 msgr2=0x7f0c601024e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5d400640 1 --2- 172.21.15.47:0/3811888988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c601010f0 0x7f0c601024e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.699+0000 7f0c5d400640 1 -- 172.21.15.47:0/3811888988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c60102da0 con 0x7f0c60100750 2024-03-21T11:34:35.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.700+0000 7f0c5d400640 1 --2- 172.21.15.47:0/3811888988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c60100750 0x7f0c60105eb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800ea60 tx=0x7f0c4800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:35.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.700+0000 7f0c56a00640 1 -- 172.21.15.47:0/3811888988 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4800cde0 con 0x7f0c60100750 2024-03-21T11:34:35.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.700+0000 7f0c56a00640 1 -- 172.21.15.47:0/3811888988 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c480035f0 con 0x7f0c60100750 2024-03-21T11:34:35.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.700+0000 7f0c56a00640 1 -- 172.21.15.47:0/3811888988 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48010670 con 0x7f0c60100750 2024-03-21T11:34:35.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.700+0000 7f0c5f200640 1 -- 172.21.15.47:0/3811888988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c601a5eb0 con 0x7f0c60100750 2024-03-21T11:34:35.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.700+0000 7f0c5f200640 1 -- 172.21.15.47:0/3811888988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c601a62b0 con 0x7f0c60100750 2024-03-21T11:34:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.703+0000 7f0c67a0e740 1 Processor -- start 2024-03-21T11:34:35.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.704+0000 7f0c67a0e740 1 -- start start 2024-03-21T11:34:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.704+0000 7f0c67a0e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c69f0 0x55df3b0c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.704+0000 7f0c67a0e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c7bc0 0x55df3b0c8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.704+0000 7f0c67a0e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c8580 0x55df3b0cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.704+0000 7f0c67a0e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3b02b2c0 con 0x55df3b0c69f0 2024-03-21T11:34:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.704+0000 7f0c67a0e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3afd3c90 con 0x55df3b0c8580 2024-03-21T11:34:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.704+0000 7f0c67a0e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3b027360 con 0x55df3b0c7bc0 2024-03-21T11:34:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.704+0000 7f0c5de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c7bc0 0x55df3b0c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.704+0000 7f0c5e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c69f0 0x55df3b0c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.704+0000 7f0c5de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c7bc0 0x55df3b0c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58916/0 (socket says 172.21.15.47:58916) 2024-03-21T11:34:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.704+0000 7f0c5de00640 1 -- 172.21.15.47:0/2694814026 learned_addr learned my addr 172.21.15.47:0/2694814026 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.705+0000 7f0c56a00640 1 -- 172.21.15.47:0/3811888988 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0c480037a0 con 0x7f0c60100750 2024-03-21T11:34:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.705+0000 7f0c5de00640 1 -- 172.21.15.47:0/2694814026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c8580 msgr2=0x55df3b0cee10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:34:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.705+0000 7f0c5de00640 1 --2- 172.21.15.47:0/2694814026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c8580 0x55df3b0cee10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.705+0000 7f0c5de00640 1 -- 172.21.15.47:0/2694814026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c69f0 msgr2=0x55df3b0c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.705+0000 7f0c5de00640 1 --2- 172.21.15.47:0/2694814026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c69f0 0x55df3b0c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.705+0000 7f0c5de00640 1 -- 172.21.15.47:0/2694814026 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df3aff8080 con 0x55df3b0c7bc0 2024-03-21T11:34:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.705+0000 7f0c5e800640 1 --2- 172.21.15.47:0/2694814026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c69f0 0x55df3b0c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.705+0000 7f0c5de00640 1 --2- 172.21.15.47:0/2694814026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c7bc0 0x55df3b0c8040 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0c50000c00 tx=0x7f0c50009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adeb157991f42135 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.706+0000 7f0c4f400640 1 -- 172.21.15.47:0/2694814026 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50045070 con 0x55df3b0c7bc0 2024-03-21T11:34:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.706+0000 7f0c4f400640 1 -- 172.21.15.47:0/2694814026 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c50004040 con 0x55df3b0c7bc0 2024-03-21T11:34:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.706+0000 7f0c4f400640 1 -- 172.21.15.47:0/2694814026 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50004eb0 con 0x55df3b0c7bc0 2024-03-21T11:34:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.706+0000 7f0c67a0e740 1 -- 172.21.15.47:0/2694814026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c7bc0 msgr2=0x55df3b0c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.706+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/2694814026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c7bc0 0x55df3b0c8040 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0c50000c00 tx=0x7f0c50009560 comp rx=0 tx=0).stop 2024-03-21T11:34:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.706+0000 7f0c67a0e740 1 -- 172.21.15.47:0/2694814026 shutdown_connections 2024-03-21T11:34:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.706+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/2694814026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c8580 0x55df3b0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.706+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/2694814026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c7bc0 0x55df3b0c8040 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.706+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/2694814026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c69f0 0x55df3b0c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.706+0000 7f0c67a0e740 1 -- 172.21.15.47:0/2694814026 >> 172.21.15.47:0/2694814026 conn(0x55df3b0c1de0 msgr2=0x55df3b0c21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.707+0000 7f0c67a0e740 1 -- 172.21.15.47:0/2694814026 shutdown_connections 2024-03-21T11:34:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.707+0000 7f0c67a0e740 1 -- 172.21.15.47:0/2694814026 wait complete. 2024-03-21T11:34:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.707+0000 7f0c56a00640 1 --2- 172.21.15.47:0/3811888988 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c440a2eb0 0x7f0c440a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.707+0000 7f0c56a00640 1 -- 172.21.15.47:0/3811888988 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0c480cfa20 con 0x7f0c60100750 2024-03-21T11:34:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.707+0000 7f0c5de00640 1 --2- 172.21.15.47:0/3811888988 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c440a2eb0 0x7f0c440a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.707+0000 7f0c67a0e740 1 -- start start 2024-03-21T11:34:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.707+0000 7f0c67a0e740 1 Processor -- start 2024-03-21T11:34:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.708+0000 7f0c5de00640 1 --2- 172.21.15.47:0/3811888988 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c440a2eb0 0x7f0c440a5370 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0c50005ab0 tx=0x7f0c5002bc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.708+0000 7f0c67a0e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c69f0 0x55df3b0d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.708+0000 7f0c67a0e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.708+0000 7f0c67a0e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c8580 0x55df3b0d81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.708+0000 7f0c67a0e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3b0cf530 con 0x55df3b0c7bc0 2024-03-21T11:34:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.708+0000 7f0c67a0e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3afe70d0 con 0x55df3b0c8580 2024-03-21T11:34:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.708+0000 7f0c67a0e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3b0cf6b0 con 0x55df3b0c69f0 2024-03-21T11:34:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.708+0000 7f0c67a0e740 10 client.0 _ll_register_callbacks cb 0x55df3afebe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.708+0000 7f0c5de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c69f0 0x55df3b0d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.708+0000 7f0c5de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c69f0 0x55df3b0d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58918/0 (socket says 172.21.15.47:58918) 2024-03-21T11:34:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.708+0000 7f0c5de00640 1 -- 172.21.15.47:0/80197394 learned_addr learned my addr 172.21.15.47:0/80197394 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.708+0000 7f0c5e800640 1 --2- 172.21.15.47:0/80197394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c8580 0x55df3b0d81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.708+0000 7f0c5d400640 1 --2- 172.21.15.47:0/80197394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.709+0000 7f0c5de00640 1 -- 172.21.15.47:0/80197394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c8580 msgr2=0x55df3b0d81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.709+0000 7f0c5de00640 1 --2- 172.21.15.47:0/80197394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c8580 0x55df3b0d81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.709+0000 7f0c5de00640 1 -- 172.21.15.47:0/80197394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c7bc0 msgr2=0x55df3b0d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.709+0000 7f0c5de00640 1 --2- 172.21.15.47:0/80197394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.709+0000 7f0c5de00640 1 -- 172.21.15.47:0/80197394 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df3ad3f3f0 con 0x55df3b0c69f0 2024-03-21T11:34:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.709+0000 7f0c5de00640 1 --2- 172.21.15.47:0/80197394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c69f0 0x55df3b0d0c40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0c50044000 tx=0x7f0c50009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0fb66014de20a42 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.709+0000 7f0c43e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.709+0000 7f0c43e00640 1 -- 172.21.15.47:0/80197394 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50059020 con 0x55df3b0c69f0 2024-03-21T11:34:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.709+0000 7f0c67a0e740 1 -- 172.21.15.47:0/80197394 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c50004040 con 0x55df3b0c69f0 2024-03-21T11:34:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.709+0000 7f0c43e00640 1 -- 172.21.15.47:0/80197394 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0c500052c0 con 0x55df3b0c69f0 2024-03-21T11:34:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.709+0000 7f0c43e00640 1 -- 172.21.15.47:0/80197394 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c50064050 con 0x55df3b0c69f0 2024-03-21T11:34:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.709+0000 7f0c67a0e740 10 client.29287 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.709+0000 7f0c67a0e740 1 -- 172.21.15.47:0/80197394 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55df3ae097d0 con 0x55df3b0c69f0 2024-03-21T11:34:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.710+0000 7f0c43e00640 1 -- 172.21.15.47:0/80197394 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c50004040 con 0x55df3b0c69f0 2024-03-21T11:34:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.710+0000 7f0c43e00640 1 -- 172.21.15.47:0/80197394 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0c5004a450 con 0x55df3b0c69f0 2024-03-21T11:34:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.710+0000 7f0c67a0e740 10 client.29287 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.710+0000 7f0c67a0e740 10 client.29287 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:34:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.710+0000 7f0c67a0e740 20 client.29287 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.710+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/80197394 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55df3b0dc320 0x55df3b0de710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.710+0000 7f0c67a0e740 4 client.29287 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.710+0000 7f0c67a0e740 1 -- 172.21.15.47:0/80197394 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df3b038800 con 0x55df3b0dc320 2024-03-21T11:34:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.710+0000 7f0c5d400640 1 --2- 172.21.15.47:0/80197394 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55df3b0dc320 0x55df3b0de710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.711+0000 7f0c5d400640 1 --2- 172.21.15.47:0/80197394 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55df3b0dc320 0x55df3b0de710 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=37f90bd71f0d2ddc server_cookie=410b4306108ebb0d in_seq=0 out_seq=0 2024-03-21T11:34:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.711+0000 7f0c43e00640 10 client.29287 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:34:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.712+0000 7f0c43e00640 1 -- 172.21.15.47:0/80197394 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55df3b038800 con 0x55df3b0dc320 2024-03-21T11:34:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.712+0000 7f0c43e00640 10 client.29287 handle_command_reply: tid=0 2024-03-21T11:34:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.712+0000 7f0c67a0e740 1 client.29287 shutdown 2024-03-21T11:34:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.712+0000 7f0c67a0e740 1 -- 172.21.15.47:0/80197394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c69f0 msgr2=0x55df3b0d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.712+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/80197394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c69f0 0x55df3b0d0c40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0c50044000 tx=0x7f0c50009560 comp rx=0 tx=0).stop 2024-03-21T11:34:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.712+0000 7f0c67a0e740 1 -- 172.21.15.47:0/80197394 shutdown_connections 2024-03-21T11:34:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.712+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/80197394 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55df3b0dc320 0x55df3b0de710 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.712+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/80197394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c8580 0x55df3b0d81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.712+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/80197394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.712+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/80197394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c69f0 0x55df3b0d0c40 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.712+0000 7f0c43e00640 0 client.29287 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:34:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.712+0000 7f0c67a0e740 1 -- 172.21.15.47:0/80197394 >> 172.21.15.47:0/80197394 conn(0x55df3b0c1de0 msgr2=0x55df3b0c1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.713+0000 7f0c67a0e740 1 -- 172.21.15.47:0/80197394 shutdown_connections 2024-03-21T11:34:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.713+0000 7f0c67a0e740 1 -- 172.21.15.47:0/80197394 wait complete. 2024-03-21T11:34:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.713+0000 7f0c67a0e740 20 client.29287 trim_cache size 0 max 16384 2024-03-21T11:34:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.728+0000 7f0c67a0e740 1 Processor -- start 2024-03-21T11:34:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.728+0000 7f0c67a0e740 1 -- start start 2024-03-21T11:34:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c67a0e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c69f0 0x55df3b0cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c67a0e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c67a0e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c8580 0x55df3b0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c67a0e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c601b0460 con 0x55df3b0c69f0 2024-03-21T11:34:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c67a0e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c601b1de0 con 0x55df3b0c7bc0 2024-03-21T11:34:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c67a0e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c601b0ac0 con 0x55df3b0c8580 2024-03-21T11:34:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c5e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c8580 0x55df3b0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c5e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c8580 0x55df3b0cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58934/0 (socket says 172.21.15.47:58934) 2024-03-21T11:34:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c5e800640 1 -- 172.21.15.47:0/330921904 learned_addr learned my addr 172.21.15.47:0/330921904 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c5de00640 1 --2- 172.21.15.47:0/330921904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c69f0 0x55df3b0cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c5d400640 1 --2- 172.21.15.47:0/330921904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c5e800640 1 -- 172.21.15.47:0/330921904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c7bc0 msgr2=0x55df3b0d8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c5e800640 1 --2- 172.21.15.47:0/330921904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c5e800640 1 -- 172.21.15.47:0/330921904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c69f0 msgr2=0x55df3b0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c5e800640 1 --2- 172.21.15.47:0/330921904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c69f0 0x55df3b0cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c5e800640 1 -- 172.21.15.47:0/330921904 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df3b0bd8b0 con 0x55df3b0c8580 2024-03-21T11:34:35.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c5d400640 1 --2- 172.21.15.47:0/330921904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:35.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.729+0000 7f0c5de00640 1 --2- 172.21.15.47:0/330921904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c69f0 0x55df3b0cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:35.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.730+0000 7f0c5e800640 1 --2- 172.21.15.47:0/330921904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c8580 0x55df3b0cb1a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0c58011790 tx=0x7f0c58011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=727c2e0623741d69 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:35.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.730+0000 7f0c4ea00640 1 -- 172.21.15.47:0/330921904 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58011e40 con 0x55df3b0c8580 2024-03-21T11:34:35.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.730+0000 7f0c4ea00640 1 -- 172.21.15.47:0/330921904 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c58003e50 con 0x55df3b0c8580 2024-03-21T11:34:35.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.730+0000 7f0c4ea00640 1 -- 172.21.15.47:0/330921904 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c580044a0 con 0x55df3b0c8580 2024-03-21T11:34:35.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.731+0000 7f0c67a0e740 1 -- 172.21.15.47:0/330921904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c8580 msgr2=0x55df3b0cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.731+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/330921904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c8580 0x55df3b0cb1a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0c58011790 tx=0x7f0c58011c50 comp rx=0 tx=0).stop 2024-03-21T11:34:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.731+0000 7f0c67a0e740 1 -- 172.21.15.47:0/330921904 shutdown_connections 2024-03-21T11:34:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.731+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/330921904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c8580 0x55df3b0cb1a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.731+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/330921904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.731+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/330921904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c69f0 0x55df3b0cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.731+0000 7f0c67a0e740 1 -- 172.21.15.47:0/330921904 >> 172.21.15.47:0/330921904 conn(0x55df3b0c1de0 msgr2=0x55df3b0c58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.731+0000 7f0c67a0e740 1 -- 172.21.15.47:0/330921904 shutdown_connections 2024-03-21T11:34:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.731+0000 7f0c67a0e740 1 -- 172.21.15.47:0/330921904 wait complete. 2024-03-21T11:34:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.731+0000 7f0c67a0e740 1 -- start start 2024-03-21T11:34:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.731+0000 7f0c67a0e740 1 Processor -- start 2024-03-21T11:34:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.732+0000 7f0c67a0e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c69f0 0x55df3b0d4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.732+0000 7f0c67a0e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.732+0000 7f0c67a0e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c8580 0x55df3b0d77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.732+0000 7f0c67a0e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c601b1780 con 0x55df3b0c8580 2024-03-21T11:34:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.732+0000 7f0c67a0e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c601b1120 con 0x55df3b0c69f0 2024-03-21T11:34:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.732+0000 7f0c67a0e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c601ae620 con 0x55df3b0c7bc0 2024-03-21T11:34:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.732+0000 7f0c67a0e740 10 client.0 _ll_register_callbacks cb 0x7f0c58009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.732+0000 7f0c5de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c69f0 0x55df3b0d4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.732+0000 7f0c5e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c8580 0x55df3b0d77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.732+0000 7f0c5d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.732+0000 7f0c5de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c69f0 0x55df3b0d4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58842/0 (socket says 172.21.15.47:58842) 2024-03-21T11:34:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.732+0000 7f0c5d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58938/0 (socket says 172.21.15.47:58938) 2024-03-21T11:34:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.732+0000 7f0c5de00640 1 -- 172.21.15.47:0/151278774 learned_addr learned my addr 172.21.15.47:0/151278774 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.733+0000 7f0c5d400640 1 -- 172.21.15.47:0/151278774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c69f0 msgr2=0x55df3b0d4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.733+0000 7f0c5d400640 1 --2- 172.21.15.47:0/151278774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c69f0 0x55df3b0d4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.733+0000 7f0c5d400640 1 -- 172.21.15.47:0/151278774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c8580 msgr2=0x55df3b0d77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.733+0000 7f0c5d400640 1 --2- 172.21.15.47:0/151278774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c8580 0x55df3b0d77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.733+0000 7f0c5d400640 1 -- 172.21.15.47:0/151278774 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df3b0d7dd0 con 0x55df3b0c7bc0 2024-03-21T11:34:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.733+0000 7f0c5e800640 1 --2- 172.21.15.47:0/151278774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c8580 0x55df3b0d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.733+0000 7f0c5d400640 1 --2- 172.21.15.47:0/151278774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d47c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800ea30 tx=0x7f0c4809b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8bbd42c43e1892b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.733+0000 7f0c43e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.733+0000 7f0c43e00640 1 -- 172.21.15.47:0/151278774 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4809feb0 con 0x55df3b0c7bc0 2024-03-21T11:34:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.733+0000 7f0c43e00640 1 -- 172.21.15.47:0/151278774 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0c480cd240 con 0x55df3b0c7bc0 2024-03-21T11:34:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.733+0000 7f0c67a0e740 1 -- 172.21.15.47:0/151278774 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c58003e50 con 0x55df3b0c7bc0 2024-03-21T11:34:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.733+0000 7f0c43e00640 1 -- 172.21.15.47:0/151278774 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c48083050 con 0x55df3b0c7bc0 2024-03-21T11:34:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.733+0000 7f0c67a0e740 10 client.29293 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.733+0000 7f0c67a0e740 1 -- 172.21.15.47:0/151278774 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55df3b1385b0 con 0x55df3b0c7bc0 2024-03-21T11:34:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.734+0000 7f0c43e00640 1 -- 172.21.15.47:0/151278774 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c58003e50 con 0x55df3b0c7bc0 2024-03-21T11:34:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.734+0000 7f0c43e00640 1 -- 172.21.15.47:0/151278774 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0c48086040 con 0x55df3b0c7bc0 2024-03-21T11:34:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.734+0000 7f0c67a0e740 10 client.29293 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.734+0000 7f0c67a0e740 10 client.29293 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:34:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.734+0000 7f0c67a0e740 20 client.29293 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.734+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/151278774 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55df3b138ec0 0x55df3b13b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.734+0000 7f0c67a0e740 4 client.29293 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:34:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.734+0000 7f0c67a0e740 1 -- 172.21.15.47:0/151278774 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55df3b032b50 con 0x55df3b138ec0 2024-03-21T11:34:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.734+0000 7f0c5de00640 1 --2- 172.21.15.47:0/151278774 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55df3b138ec0 0x55df3b13b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c5de00640 1 --2- 172.21.15.47:0/151278774 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55df3b138ec0 0x55df3b13b2e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=67f04f2783f9cac7 server_cookie=8f72f9f8cd290baa in_seq=0 out_seq=0 2024-03-21T11:34:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c43e00640 10 client.29293 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:34:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c43e00640 1 -- 172.21.15.47:0/151278774 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55df3b032b50 con 0x55df3b138ec0 2024-03-21T11:34:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c43e00640 10 client.29293 handle_command_reply: tid=0 2024-03-21T11:34:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c67a0e740 1 client.29293 shutdown 2024-03-21T11:34:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c67a0e740 1 -- 172.21.15.47:0/151278774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c7bc0 msgr2=0x55df3b0d47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/151278774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d47c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800ea30 tx=0x7f0c4809b8a0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c67a0e740 1 -- 172.21.15.47:0/151278774 shutdown_connections 2024-03-21T11:34:35.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/151278774 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55df3b138ec0 0x55df3b13b2e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/151278774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df3b0c8580 0x55df3b0d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/151278774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df3b0c7bc0 0x55df3b0d47c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c67a0e740 1 --2- 172.21.15.47:0/151278774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df3b0c69f0 0x55df3b0d4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c67a0e740 1 -- 172.21.15.47:0/151278774 >> 172.21.15.47:0/151278774 conn(0x55df3b0c1de0 msgr2=0x55df3b0c08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:35.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c43e00640 0 client.29293 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:34:35.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.735+0000 7f0c67a0e740 1 -- 172.21.15.47:0/151278774 shutdown_connections 2024-03-21T11:34:35.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.736+0000 7f0c67a0e740 1 -- 172.21.15.47:0/151278774 wait complete. 2024-03-21T11:34:35.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.736+0000 7f0c67a0e740 20 client.29293 trim_cache size 0 max 16384 2024-03-21T11:34:35.744 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:34:35.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.736+0000 7f0c5f200640 1 -- 172.21.15.47:0/3811888988 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c440a2eb0 msgr2=0x7f0c440a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.736+0000 7f0c5f200640 1 --2- 172.21.15.47:0/3811888988 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c440a2eb0 0x7f0c440a5370 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0c50005ab0 tx=0x7f0c5002bc30 comp rx=0 tx=0).stop 2024-03-21T11:34:35.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.736+0000 7f0c5f200640 1 -- 172.21.15.47:0/3811888988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c60100750 msgr2=0x7f0c60105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.736+0000 7f0c5f200640 1 --2- 172.21.15.47:0/3811888988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c60100750 0x7f0c60105eb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800ea60 tx=0x7f0c4800ef20 comp rx=0 tx=0).stop 2024-03-21T11:34:35.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.736+0000 7f0c5f200640 1 -- 172.21.15.47:0/3811888988 shutdown_connections 2024-03-21T11:34:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.736+0000 7f0c5f200640 1 --2- 172.21.15.47:0/3811888988 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c440a2eb0 0x7f0c440a5370 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.736+0000 7f0c5f200640 1 --2- 172.21.15.47:0/3811888988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c601010f0 0x7f0c601024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.736+0000 7f0c5f200640 1 --2- 172.21.15.47:0/3811888988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c60100750 0x7f0c60105eb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.736+0000 7f0c5f200640 1 --2- 172.21.15.47:0/3811888988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c600ffdf0 0x7f0c60105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.736+0000 7f0c5f200640 1 -- 172.21.15.47:0/3811888988 >> 172.21.15.47:0/3811888988 conn(0x7f0c600fba10 msgr2=0x7f0c600fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.737+0000 7f0c5f200640 1 -- 172.21.15.47:0/3811888988 shutdown_connections 2024-03-21T11:34:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.737+0000 7f0c5f200640 1 -- 172.21.15.47:0/3811888988 wait complete. 2024-03-21T11:34:35.753 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:35 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1308661471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:34:35.757 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:34:35.758 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:34:35.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:35 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1308661471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:34:35.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:35 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1308661471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:34:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.998+0000 7f5d8b200640 1 -- 172.21.15.47:0/2438654952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d8c104990 msgr2=0x7f5d8c104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.998+0000 7f5d8b200640 1 --2- 172.21.15.47:0/2438654952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d8c104990 0x7f5d8c104e10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009860 tx=0x7f5d7402bb70 comp rx=0 tx=0).stop 2024-03-21T11:34:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.998+0000 7f5d8b200640 1 -- 172.21.15.47:0/2438654952 shutdown_connections 2024-03-21T11:34:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.998+0000 7f5d8b200640 1 --2- 172.21.15.47:0/2438654952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d8c105350 0x7f5d8c10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.998+0000 7f5d8b200640 1 --2- 172.21.15.47:0/2438654952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d8c104990 0x7f5d8c104e10 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.998+0000 7f5d8b200640 1 --2- 172.21.15.47:0/2438654952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d8c1037a0 0x7f5d8c103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.998+0000 7f5d8b200640 1 -- 172.21.15.47:0/2438654952 >> 172.21.15.47:0/2438654952 conn(0x7f5d8c0fed10 msgr2=0x7f5d8c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.998+0000 7f5d8b200640 1 -- 172.21.15.47:0/2438654952 shutdown_connections 2024-03-21T11:34:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.998+0000 7f5d8b200640 1 -- 172.21.15.47:0/2438654952 wait complete. 2024-03-21T11:34:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.999+0000 7f5d8b200640 1 Processor -- start 2024-03-21T11:34:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:35.999+0000 7f5d8b200640 1 -- start start 2024-03-21T11:34:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d8b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d8c1037a0 0x7f5d8c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d8b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d8c104990 0x7f5d8c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d8b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d8c105350 0x7f5d8c1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d8b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c10f420 con 0x7f5d8c104990 2024-03-21T11:34:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d8b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c10f2a0 con 0x7f5d8c105350 2024-03-21T11:34:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d8b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c10f5a0 con 0x7f5d8c1037a0 2024-03-21T11:34:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d89e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d8c1037a0 0x7f5d8c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d8a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d8c105350 0x7f5d8c1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d89e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d8c1037a0 0x7f5d8c199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58942/0 (socket says 172.21.15.47:58942) 2024-03-21T11:34:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d89e00640 1 -- 172.21.15.47:0/3397196112 learned_addr learned my addr 172.21.15.47:0/3397196112 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d81400640 1 --2- 172.21.15.47:0/3397196112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d8c104990 0x7f5d8c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d89e00640 1 -- 172.21.15.47:0/3397196112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d8c105350 msgr2=0x7f5d8c1a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d89e00640 1 --2- 172.21.15.47:0/3397196112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d8c105350 0x7f5d8c1a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d89e00640 1 -- 172.21.15.47:0/3397196112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d8c104990 msgr2=0x7f5d8c199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d89e00640 1 --2- 172.21.15.47:0/3397196112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d8c104990 0x7f5d8c199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.000+0000 7f5d89e00640 1 -- 172.21.15.47:0/3397196112 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c1a16b0 con 0x7f5d8c1037a0 2024-03-21T11:34:36.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.001+0000 7f5d89e00640 1 --2- 172.21.15.47:0/3397196112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d8c1037a0 0x7f5d8c199a90 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00e9c0 tx=0x7f5d7c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.001+0000 7f5d83400640 1 -- 172.21.15.47:0/3397196112 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c00cd80 con 0x7f5d8c1037a0 2024-03-21T11:34:36.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.001+0000 7f5d83400640 1 -- 172.21.15.47:0/3397196112 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d7c0035f0 con 0x7f5d8c1037a0 2024-03-21T11:34:36.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.001+0000 7f5d83400640 1 -- 172.21.15.47:0/3397196112 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c010690 con 0x7f5d8c1037a0 2024-03-21T11:34:36.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.001+0000 7f5d8b200640 1 -- 172.21.15.47:0/3397196112 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c1a19a0 con 0x7f5d8c1037a0 2024-03-21T11:34:36.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.001+0000 7f5d8b200640 1 -- 172.21.15.47:0/3397196112 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c1a1e70 con 0x7f5d8c1037a0 2024-03-21T11:34:36.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.005+0000 7f5d9397d740 1 Processor -- start 2024-03-21T11:34:36.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.005+0000 7f5d9397d740 1 -- start start 2024-03-21T11:34:36.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.005+0000 7f5d83400640 1 -- 172.21.15.47:0/3397196112 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5d7c0037a0 con 0x7f5d8c1037a0 2024-03-21T11:34:36.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.005+0000 7f5d9397d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b69f0 0x559a076b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.006+0000 7f5d9397d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 0x559a076b8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.006+0000 7f5d9397d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b8580 0x559a076bee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.006+0000 7f5d9397d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a0761b2c0 con 0x559a076b69f0 2024-03-21T11:34:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.006+0000 7f5d9397d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a075c3c90 con 0x559a076b8580 2024-03-21T11:34:36.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.006+0000 7f5d9397d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a07617360 con 0x559a076b7bc0 2024-03-21T11:34:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.006+0000 7f5d81400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b8580 0x559a076bee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.006+0000 7f5d89e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 0x559a076b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.006+0000 7f5d81400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b8580 0x559a076bee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58866/0 (socket says 172.21.15.47:58866) 2024-03-21T11:34:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.006+0000 7f5d81400640 1 -- 172.21.15.47:0/4079690825 learned_addr learned my addr 172.21.15.47:0/4079690825 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.006+0000 7f5d8a800640 1 --2- 172.21.15.47:0/4079690825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b69f0 0x559a076b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.007+0000 7f5d81400640 1 -- 172.21.15.47:0/4079690825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 msgr2=0x559a076b8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.007+0000 7f5d81400640 1 --2- 172.21.15.47:0/4079690825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 0x559a076b8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.007+0000 7f5d81400640 1 -- 172.21.15.47:0/4079690825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b69f0 msgr2=0x559a076b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.007+0000 7f5d81400640 1 --2- 172.21.15.47:0/4079690825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b69f0 0x559a076b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.007+0000 7f5d81400640 1 -- 172.21.15.47:0/4079690825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a075e8080 con 0x559a076b8580 2024-03-21T11:34:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.007+0000 7f5d89e00640 1 --2- 172.21.15.47:0/4079690825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 0x559a076b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.007+0000 7f5d81400640 1 --2- 172.21.15.47:0/4079690825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b8580 0x559a076bee10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5d74037040 tx=0x7f5d74005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=851726d0d1903dd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.008+0000 7f5d7b000640 1 -- 172.21.15.47:0/4079690825 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74039040 con 0x559a076b8580 2024-03-21T11:34:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.008+0000 7f5d7b000640 1 -- 172.21.15.47:0/4079690825 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d74034db0 con 0x559a076b8580 2024-03-21T11:34:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.008+0000 7f5d7b000640 1 -- 172.21.15.47:0/4079690825 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7402c8d0 con 0x559a076b8580 2024-03-21T11:34:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.008+0000 7f5d83400640 1 --2- 172.21.15.47:0/3397196112 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5d700a2fa0 0x7f5d700a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.008+0000 7f5d83400640 1 -- 172.21.15.47:0/3397196112 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5d7c0cfb40 con 0x7f5d8c1037a0 2024-03-21T11:34:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.009+0000 7f5d8a800640 1 --2- 172.21.15.47:0/3397196112 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5d700a2fa0 0x7f5d700a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.009+0000 7f5d9397d740 1 -- 172.21.15.47:0/4079690825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b8580 msgr2=0x559a076bee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.009+0000 7f5d9397d740 1 --2- 172.21.15.47:0/4079690825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b8580 0x559a076bee10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5d74037040 tx=0x7f5d74005b20 comp rx=0 tx=0).stop 2024-03-21T11:34:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.009+0000 7f5d9397d740 1 -- 172.21.15.47:0/4079690825 shutdown_connections 2024-03-21T11:34:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.009+0000 7f5d9397d740 1 --2- 172.21.15.47:0/4079690825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b8580 0x559a076bee10 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.009+0000 7f5d9397d740 1 --2- 172.21.15.47:0/4079690825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 0x559a076b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.009+0000 7f5d9397d740 1 --2- 172.21.15.47:0/4079690825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b69f0 0x559a076b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.009+0000 7f5d9397d740 1 -- 172.21.15.47:0/4079690825 >> 172.21.15.47:0/4079690825 conn(0x559a076b1de0 msgr2=0x559a076b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.009+0000 7f5d9397d740 1 -- 172.21.15.47:0/4079690825 shutdown_connections 2024-03-21T11:34:36.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.009+0000 7f5d9397d740 1 -- 172.21.15.47:0/4079690825 wait complete. 2024-03-21T11:34:36.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.009+0000 7f5d9397d740 1 -- start start 2024-03-21T11:34:36.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.009+0000 7f5d8a800640 1 --2- 172.21.15.47:0/3397196112 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5d700a2fa0 0x7f5d700a5460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x559a076b7ac0 tx=0x7f5d84010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.010+0000 7f5d9397d740 1 Processor -- start 2024-03-21T11:34:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.010+0000 7f5d9397d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b69f0 0x559a076c0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.010+0000 7f5d9397d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b7bc0 0x559a076c1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.010+0000 7f5d9397d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b8580 0x559a076c8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.010+0000 7f5d9397d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a076bf530 con 0x559a076b69f0 2024-03-21T11:34:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.010+0000 7f5d9397d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a075d70d0 con 0x559a076b7bc0 2024-03-21T11:34:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.010+0000 7f5d9397d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a076bf6b0 con 0x559a076b8580 2024-03-21T11:34:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.011+0000 7f5d9397d740 10 client.0 _ll_register_callbacks cb 0x559a075dbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.011+0000 7f5d81400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b7bc0 0x559a076c1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.011+0000 7f5d81400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b7bc0 0x559a076c1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58868/0 (socket says 172.21.15.47:58868) 2024-03-21T11:34:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.011+0000 7f5d81400640 1 -- 172.21.15.47:0/553674180 learned_addr learned my addr 172.21.15.47:0/553674180 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.011+0000 7f5d8a800640 1 --2- 172.21.15.47:0/553674180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b8580 0x559a076c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.011+0000 7f5d89e00640 1 --2- 172.21.15.47:0/553674180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b69f0 0x559a076c0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.011+0000 7f5d81400640 1 -- 172.21.15.47:0/553674180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b8580 msgr2=0x559a076c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.011+0000 7f5d81400640 1 --2- 172.21.15.47:0/553674180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b8580 0x559a076c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.011+0000 7f5d81400640 1 -- 172.21.15.47:0/553674180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b69f0 msgr2=0x559a076c0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.011+0000 7f5d81400640 1 --2- 172.21.15.47:0/553674180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b69f0 0x559a076c0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.011+0000 7f5d81400640 1 -- 172.21.15.47:0/553674180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a0732f3f0 con 0x559a076b7bc0 2024-03-21T11:34:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.011+0000 7f5d8a800640 1 --2- 172.21.15.47:0/553674180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b8580 0x559a076c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.012+0000 7f5d81400640 1 --2- 172.21.15.47:0/553674180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b7bc0 0x559a076c1170 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5d7402c0b0 tx=0x7f5d7402bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7430ea8a9e6358f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.012+0000 7f5d6f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.012+0000 7f5d6f400640 1 -- 172.21.15.47:0/553674180 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74039040 con 0x559a076b7bc0 2024-03-21T11:34:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.012+0000 7f5d9397d740 1 -- 172.21.15.47:0/553674180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d74034db0 con 0x559a076b7bc0 2024-03-21T11:34:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.012+0000 7f5d6f400640 1 -- 172.21.15.47:0/553674180 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5d7402c820 con 0x559a076b7bc0 2024-03-21T11:34:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.012+0000 7f5d6f400640 1 -- 172.21.15.47:0/553674180 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d7405a050 con 0x559a076b7bc0 2024-03-21T11:34:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.012+0000 7f5d9397d740 10 client.29282 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.012+0000 7f5d9397d740 1 -- 172.21.15.47:0/553674180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559a073f97d0 con 0x559a076b7bc0 2024-03-21T11:34:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.013+0000 7f5d6f400640 1 -- 172.21.15.47:0/553674180 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d74034db0 con 0x559a076b7bc0 2024-03-21T11:34:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.013+0000 7f5d6f400640 1 -- 172.21.15.47:0/553674180 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5d74004900 con 0x559a076b7bc0 2024-03-21T11:34:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.013+0000 7f5d9397d740 10 client.29282 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.013+0000 7f5d9397d740 10 client.29282 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:34:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.013+0000 7f5d9397d740 20 client.29282 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.013+0000 7f5d9397d740 1 --2- 172.21.15.47:0/553674180 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x559a076cc2a0 0x559a076ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.013+0000 7f5d9397d740 4 client.29282 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.013+0000 7f5d9397d740 1 -- 172.21.15.47:0/553674180 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a07628800 con 0x559a076cc2a0 2024-03-21T11:34:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.013+0000 7f5d89e00640 1 --2- 172.21.15.47:0/553674180 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x559a076cc2a0 0x559a076ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.014+0000 7f5d89e00640 1 --2- 172.21.15.47:0/553674180 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x559a076cc2a0 0x559a076ce690 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3bc250f1b5c187a server_cookie=7bf066c0d8143e2a in_seq=0 out_seq=0 2024-03-21T11:34:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.014+0000 7f5d6f400640 10 client.29282 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:34:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d6f400640 1 -- 172.21.15.47:0/553674180 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559a07628800 con 0x559a076cc2a0 2024-03-21T11:34:36.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d6f400640 10 client.29282 handle_command_reply: tid=0 2024-03-21T11:34:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d9397d740 1 client.29282 shutdown 2024-03-21T11:34:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d9397d740 1 -- 172.21.15.47:0/553674180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b7bc0 msgr2=0x559a076c1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d9397d740 1 --2- 172.21.15.47:0/553674180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b7bc0 0x559a076c1170 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5d7402c0b0 tx=0x7f5d7402bb70 comp rx=0 tx=0).stop 2024-03-21T11:34:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d9397d740 1 -- 172.21.15.47:0/553674180 shutdown_connections 2024-03-21T11:34:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d9397d740 1 --2- 172.21.15.47:0/553674180 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x559a076cc2a0 0x559a076ce690 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d9397d740 1 --2- 172.21.15.47:0/553674180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b8580 0x559a076c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d9397d740 1 --2- 172.21.15.47:0/553674180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b7bc0 0x559a076c1170 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d9397d740 1 --2- 172.21.15.47:0/553674180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b69f0 0x559a076c0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d6f400640 0 client.29282 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:34:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d9397d740 1 -- 172.21.15.47:0/553674180 >> 172.21.15.47:0/553674180 conn(0x559a076b1de0 msgr2=0x559a076b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d9397d740 1 -- 172.21.15.47:0/553674180 shutdown_connections 2024-03-21T11:34:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d9397d740 1 -- 172.21.15.47:0/553674180 wait complete. 2024-03-21T11:34:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.015+0000 7f5d9397d740 20 client.29282 trim_cache size 0 max 16384 2024-03-21T11:34:36.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.030+0000 7f5d9397d740 1 Processor -- start 2024-03-21T11:34:36.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.030+0000 7f5d9397d740 1 -- start start 2024-03-21T11:34:36.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.030+0000 7f5d9397d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b69f0 0x559a076bac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.030+0000 7f5d9397d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 0x559a076c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.030+0000 7f5d9397d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b8580 0x559a076bb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.030+0000 7f5d9397d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c1ac010 con 0x559a076b8580 2024-03-21T11:34:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.030+0000 7f5d9397d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c1ad990 con 0x559a076b69f0 2024-03-21T11:34:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.030+0000 7f5d9397d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c1ac670 con 0x559a076b7bc0 2024-03-21T11:34:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.030+0000 7f5d89e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b69f0 0x559a076bac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.030+0000 7f5d81400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 0x559a076c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.030+0000 7f5d8a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b8580 0x559a076bb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.030+0000 7f5d89e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b69f0 0x559a076bac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58882/0 (socket says 172.21.15.47:58882) 2024-03-21T11:34:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.030+0000 7f5d89e00640 1 -- 172.21.15.47:0/3514011016 learned_addr learned my addr 172.21.15.47:0/3514011016 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.031+0000 7f5d81400640 1 -- 172.21.15.47:0/3514011016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b69f0 msgr2=0x559a076bac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.031+0000 7f5d81400640 1 --2- 172.21.15.47:0/3514011016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b69f0 0x559a076bac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.031+0000 7f5d81400640 1 -- 172.21.15.47:0/3514011016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b8580 msgr2=0x559a076bb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.031+0000 7f5d81400640 1 --2- 172.21.15.47:0/3514011016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b8580 0x559a076bb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.031+0000 7f5d81400640 1 -- 172.21.15.47:0/3514011016 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a076b7640 con 0x559a076b7bc0 2024-03-21T11:34:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.031+0000 7f5d89e00640 1 --2- 172.21.15.47:0/3514011016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b69f0 0x559a076bac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.031+0000 7f5d81400640 1 --2- 172.21.15.47:0/3514011016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 0x559a076c8100 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5d740349e0 tx=0x7f5d74005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=999ab18e0f956626 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.031+0000 7f5d7a600640 1 -- 172.21.15.47:0/3514011016 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74063070 con 0x559a076b7bc0 2024-03-21T11:34:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.031+0000 7f5d7a600640 1 -- 172.21.15.47:0/3514011016 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d74034bb0 con 0x559a076b7bc0 2024-03-21T11:34:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.031+0000 7f5d7a600640 1 -- 172.21.15.47:0/3514011016 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74005120 con 0x559a076b7bc0 2024-03-21T11:34:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.032+0000 7f5d9397d740 1 -- 172.21.15.47:0/3514011016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 msgr2=0x559a076c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.032+0000 7f5d9397d740 1 --2- 172.21.15.47:0/3514011016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 0x559a076c8100 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5d740349e0 tx=0x7f5d74005b20 comp rx=0 tx=0).stop 2024-03-21T11:34:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.032+0000 7f5d9397d740 1 -- 172.21.15.47:0/3514011016 shutdown_connections 2024-03-21T11:34:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.032+0000 7f5d9397d740 1 --2- 172.21.15.47:0/3514011016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b8580 0x559a076bb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.032+0000 7f5d9397d740 1 --2- 172.21.15.47:0/3514011016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 0x559a076c8100 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.032+0000 7f5d9397d740 1 --2- 172.21.15.47:0/3514011016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b69f0 0x559a076bac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.032+0000 7f5d9397d740 1 -- 172.21.15.47:0/3514011016 >> 172.21.15.47:0/3514011016 conn(0x559a076b1de0 msgr2=0x559a076b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.032+0000 7f5d9397d740 1 -- 172.21.15.47:0/3514011016 shutdown_connections 2024-03-21T11:34:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.032+0000 7f5d9397d740 1 -- 172.21.15.47:0/3514011016 wait complete. 2024-03-21T11:34:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.032+0000 7f5d9397d740 1 -- start start 2024-03-21T11:34:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.033+0000 7f5d9397d740 1 Processor -- start 2024-03-21T11:34:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.033+0000 7f5d9397d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b69f0 0x559a076c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.033+0000 7f5d9397d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 0x559a076c4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.033+0000 7f5d9397d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b8580 0x559a076c7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.033+0000 7f5d9397d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c1ad330 con 0x559a076b8580 2024-03-21T11:34:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.033+0000 7f5d9397d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c1accd0 con 0x559a076b69f0 2024-03-21T11:34:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.033+0000 7f5d9397d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c1aa1d0 con 0x559a076b7bc0 2024-03-21T11:34:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.033+0000 7f5d9397d740 10 client.0 _ll_register_callbacks cb 0x7f5d7c0d3450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.033+0000 7f5d89e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b69f0 0x559a076c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.033+0000 7f5d8a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b8580 0x559a076c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.033+0000 7f5d89e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b69f0 0x559a076c3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58890/0 (socket says 172.21.15.47:58890) 2024-03-21T11:34:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.033+0000 7f5d89e00640 1 -- 172.21.15.47:0/1352737568 learned_addr learned my addr 172.21.15.47:0/1352737568 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.033+0000 7f5d81400640 1 --2- 172.21.15.47:0/1352737568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 0x559a076c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.034+0000 7f5d89e00640 1 -- 172.21.15.47:0/1352737568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 msgr2=0x559a076c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.034+0000 7f5d89e00640 1 --2- 172.21.15.47:0/1352737568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 0x559a076c4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.034+0000 7f5d89e00640 1 -- 172.21.15.47:0/1352737568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b8580 msgr2=0x559a076c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.034+0000 7f5d89e00640 1 --2- 172.21.15.47:0/1352737568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b8580 0x559a076c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.034+0000 7f5d89e00640 1 -- 172.21.15.47:0/1352737568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a076c7630 con 0x559a076b69f0 2024-03-21T11:34:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.034+0000 7f5d89e00640 1 --2- 172.21.15.47:0/1352737568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b69f0 0x559a076c3ae0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0a4360 tx=0x7f5d7c098240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=674ead829f89a16f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.034+0000 7f5d6f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.034+0000 7f5d6f400640 1 -- 172.21.15.47:0/1352737568 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c09b7e0 con 0x559a076b69f0 2024-03-21T11:34:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.034+0000 7f5d6f400640 1 -- 172.21.15.47:0/1352737568 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5d7c003b30 con 0x559a076b69f0 2024-03-21T11:34:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.034+0000 7f5d9397d740 1 -- 172.21.15.47:0/1352737568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d74034bb0 con 0x559a076b69f0 2024-03-21T11:34:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.034+0000 7f5d6f400640 1 -- 172.21.15.47:0/1352737568 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d7c0bc0b0 con 0x559a076b69f0 2024-03-21T11:34:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.034+0000 7f5d9397d740 10 client.29288 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.035+0000 7f5d9397d740 1 -- 172.21.15.47:0/1352737568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559a07727d90 con 0x559a076b69f0 2024-03-21T11:34:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.035+0000 7f5d6f400640 1 -- 172.21.15.47:0/1352737568 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d74034bb0 con 0x559a076b69f0 2024-03-21T11:34:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.035+0000 7f5d6f400640 1 -- 172.21.15.47:0/1352737568 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5d7c09b2d0 con 0x559a076b69f0 2024-03-21T11:34:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.035+0000 7f5d9397d740 10 client.29288 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.035+0000 7f5d9397d740 10 client.29288 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:34:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.035+0000 7f5d9397d740 20 client.29288 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.035+0000 7f5d9397d740 1 --2- 172.21.15.47:0/1352737568 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x559a07728510 0x559a0772a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.035+0000 7f5d9397d740 4 client.29288 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:34:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.035+0000 7f5d9397d740 1 -- 172.21.15.47:0/1352737568 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559a07622b50 con 0x559a07728510 2024-03-21T11:34:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.035+0000 7f5d81400640 1 --2- 172.21.15.47:0/1352737568 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x559a07728510 0x559a0772a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d81400640 1 --2- 172.21.15.47:0/1352737568 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x559a07728510 0x559a0772a9d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=311bb14e5dbd21dd server_cookie=77391bd72fc31555 in_seq=0 out_seq=0 2024-03-21T11:34:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d6f400640 10 client.29288 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:34:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d6f400640 1 -- 172.21.15.47:0/1352737568 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559a07622b50 con 0x559a07728510 2024-03-21T11:34:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d6f400640 10 client.29288 handle_command_reply: tid=0 2024-03-21T11:34:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d9397d740 1 client.29288 shutdown 2024-03-21T11:34:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d9397d740 1 -- 172.21.15.47:0/1352737568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b69f0 msgr2=0x559a076c3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d9397d740 1 --2- 172.21.15.47:0/1352737568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b69f0 0x559a076c3ae0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0a4360 tx=0x7f5d7c098240 comp rx=0 tx=0).stop 2024-03-21T11:34:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d9397d740 1 -- 172.21.15.47:0/1352737568 shutdown_connections 2024-03-21T11:34:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d9397d740 1 --2- 172.21.15.47:0/1352737568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a076b8580 0x559a076c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d9397d740 1 --2- 172.21.15.47:0/1352737568 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x559a07728510 0x559a0772a9d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d9397d740 1 --2- 172.21.15.47:0/1352737568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a076b7bc0 0x559a076c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d9397d740 1 --2- 172.21.15.47:0/1352737568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a076b69f0 0x559a076c3ae0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d6f400640 0 client.29288 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:34:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d9397d740 1 -- 172.21.15.47:0/1352737568 >> 172.21.15.47:0/1352737568 conn(0x559a076b1de0 msgr2=0x559a076b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d9397d740 1 -- 172.21.15.47:0/1352737568 shutdown_connections 2024-03-21T11:34:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d9397d740 1 -- 172.21.15.47:0/1352737568 wait complete. 2024-03-21T11:34:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.036+0000 7f5d9397d740 20 client.29288 trim_cache size 0 max 16384 2024-03-21T11:34:36.049 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:34:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.037+0000 7f5d8b200640 1 -- 172.21.15.47:0/3397196112 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5d700a2fa0 msgr2=0x7f5d700a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.037+0000 7f5d8b200640 1 --2- 172.21.15.47:0/3397196112 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5d700a2fa0 0x7f5d700a5460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x559a076b7ac0 tx=0x7f5d84010040 comp rx=0 tx=0).stop 2024-03-21T11:34:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.037+0000 7f5d8b200640 1 -- 172.21.15.47:0/3397196112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d8c1037a0 msgr2=0x7f5d8c199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.037+0000 7f5d8b200640 1 --2- 172.21.15.47:0/3397196112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d8c1037a0 0x7f5d8c199a90 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00e9c0 tx=0x7f5d7c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:34:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.037+0000 7f5d8b200640 1 -- 172.21.15.47:0/3397196112 shutdown_connections 2024-03-21T11:34:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.037+0000 7f5d8b200640 1 --2- 172.21.15.47:0/3397196112 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5d700a2fa0 0x7f5d700a5460 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.037+0000 7f5d8b200640 1 --2- 172.21.15.47:0/3397196112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d8c105350 0x7f5d8c1a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.037+0000 7f5d8b200640 1 --2- 172.21.15.47:0/3397196112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d8c104990 0x7f5d8c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.037+0000 7f5d8b200640 1 --2- 172.21.15.47:0/3397196112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d8c1037a0 0x7f5d8c199a90 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.037+0000 7f5d8b200640 1 -- 172.21.15.47:0/3397196112 >> 172.21.15.47:0/3397196112 conn(0x7f5d8c0fed10 msgr2=0x7f5d8c107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.037+0000 7f5d8b200640 1 -- 172.21.15.47:0/3397196112 shutdown_connections 2024-03-21T11:34:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.037+0000 7f5d8b200640 1 -- 172.21.15.47:0/3397196112 wait complete. 2024-03-21T11:34:36.057 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:34:36.057 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:34:36.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.264+0000 7fd739800640 1 -- 172.21.15.47:0/71123013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7341017b0 msgr2=0x7fd734101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.264+0000 7fd739800640 1 --2- 172.21.15.47:0/71123013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7341017b0 0x7fd734101c30 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd720009900 tx=0x7fd72002bc30 comp rx=0 tx=0).stop 2024-03-21T11:34:36.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.265+0000 7fd739800640 1 -- 172.21.15.47:0/71123013 shutdown_connections 2024-03-21T11:34:36.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.265+0000 7fd739800640 1 --2- 172.21.15.47:0/71123013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd734102170 0x7fd734108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.265+0000 7fd739800640 1 --2- 172.21.15.47:0/71123013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7341017b0 0x7fd734101c30 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.265+0000 7fd739800640 1 --2- 172.21.15.47:0/71123013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7341005c0 0x7fd7341009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.265+0000 7fd739800640 1 -- 172.21.15.47:0/71123013 >> 172.21.15.47:0/71123013 conn(0x7fd7340fbb50 msgr2=0x7fd7340fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.265+0000 7fd739800640 1 -- 172.21.15.47:0/71123013 shutdown_connections 2024-03-21T11:34:36.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.265+0000 7fd739800640 1 -- 172.21.15.47:0/71123013 wait complete. 2024-03-21T11:34:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.266+0000 7fd739800640 1 Processor -- start 2024-03-21T11:34:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.266+0000 7fd739800640 1 -- start start 2024-03-21T11:34:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.266+0000 7fd739800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7341005c0 0x7fd73410af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.266+0000 7fd739800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd734102170 0x7fd734109580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.266+0000 7fd739800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd734109ac0 0x7fd734109f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.266+0000 7fd739800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73410d360 con 0x7fd7341005c0 2024-03-21T11:34:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.266+0000 7fd739800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73410d1e0 con 0x7fd734109ac0 2024-03-21T11:34:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.266+0000 7fd739800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73410d4e0 con 0x7fd734102170 2024-03-21T11:34:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.267+0000 7fd733400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd734109ac0 0x7fd734109f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.267+0000 7fd732000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd734102170 0x7fd734109580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.267+0000 7fd733400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd734109ac0 0x7fd734109f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58912/0 (socket says 172.21.15.47:58912) 2024-03-21T11:34:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.267+0000 7fd733400640 1 -- 172.21.15.47:0/3526459982 learned_addr learned my addr 172.21.15.47:0/3526459982 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.267+0000 7fd732a00640 1 --2- 172.21.15.47:0/3526459982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7341005c0 0x7fd73410af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.267+0000 7fd733400640 1 -- 172.21.15.47:0/3526459982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd734102170 msgr2=0x7fd734109580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.267+0000 7fd733400640 1 --2- 172.21.15.47:0/3526459982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd734102170 0x7fd734109580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.267+0000 7fd733400640 1 -- 172.21.15.47:0/3526459982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7341005c0 msgr2=0x7fd73410af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.267+0000 7fd733400640 1 --2- 172.21.15.47:0/3526459982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7341005c0 0x7fd73410af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.267+0000 7fd733400640 1 -- 172.21.15.47:0/3526459982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7341a1dd0 con 0x7fd734109ac0 2024-03-21T11:34:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.267+0000 7fd733400640 1 --2- 172.21.15.47:0/3526459982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd734109ac0 0x7fd734109f70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00eb10 tx=0x7fd72c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.268+0000 7fd72b400640 1 -- 172.21.15.47:0/3526459982 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c004b20 con 0x7fd734109ac0 2024-03-21T11:34:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.268+0000 7fd739800640 1 -- 172.21.15.47:0/3526459982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7341a20c0 con 0x7fd734109ac0 2024-03-21T11:34:36.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.268+0000 7fd72b400640 1 -- 172.21.15.47:0/3526459982 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd72c004cc0 con 0x7fd734109ac0 2024-03-21T11:34:36.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.268+0000 7fd72b400640 1 -- 172.21.15.47:0/3526459982 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c010760 con 0x7fd734109ac0 2024-03-21T11:34:36.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.268+0000 7fd739800640 1 -- 172.21.15.47:0/3526459982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7341a2590 con 0x7fd734109ac0 2024-03-21T11:34:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.271+0000 7fd72b400640 1 -- 172.21.15.47:0/3526459982 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd72c005020 con 0x7fd734109ac0 2024-03-21T11:34:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.271+0000 7fd73c5d0740 1 Processor -- start 2024-03-21T11:34:36.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.271+0000 7fd73c5d0740 1 -- start start 2024-03-21T11:34:36.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd73c5d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38dc810 0x564db38dcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd73c5d0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dd9e0 0x564db38dde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd73c5d0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38de3a0 0x564db38e4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd73c5d0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db38412c0 con 0x564db38de3a0 2024-03-21T11:34:36.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd73c5d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db37e9c90 con 0x564db38dc810 2024-03-21T11:34:36.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd73c5d0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db383d360 con 0x564db38dd9e0 2024-03-21T11:34:36.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd732a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dd9e0 0x564db38dde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd732a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dd9e0 0x564db38dde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59008/0 (socket says 172.21.15.47:59008) 2024-03-21T11:34:36.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd732a00640 1 -- 172.21.15.47:0/485557388 learned_addr learned my addr 172.21.15.47:0/485557388 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd732000640 1 --2- 172.21.15.47:0/485557388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38dc810 0x564db38dcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd732000640 1 --2- 172.21.15.47:0/485557388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38de3a0 0x564db38e4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd732a00640 1 -- 172.21.15.47:0/485557388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38dc810 msgr2=0x564db38dcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd732a00640 1 --2- 172.21.15.47:0/485557388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38dc810 0x564db38dcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd732a00640 1 -- 172.21.15.47:0/485557388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38de3a0 msgr2=0x564db38e4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.272+0000 7fd732a00640 1 --2- 172.21.15.47:0/485557388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38de3a0 0x564db38e4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.273+0000 7fd732a00640 1 -- 172.21.15.47:0/485557388 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db380e080 con 0x564db38dd9e0 2024-03-21T11:34:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.273+0000 7fd732000640 1 --2- 172.21.15.47:0/485557388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38de3a0 0x564db38e4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:34:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.273+0000 7fd72b400640 1 --2- 172.21.15.47:0/3526459982 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7180a2f20 0x7fd7180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.273+0000 7fd732a00640 1 --2- 172.21.15.47:0/485557388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dd9e0 0x564db38dde60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd724009c80 tx=0x7fd724011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95ee96c36e200459 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.273+0000 7fd72b400640 1 -- 172.21.15.47:0/3526459982 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd72c0cfd30 con 0x7fd734109ac0 2024-03-21T11:34:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.273+0000 7fd732000640 1 --2- 172.21.15.47:0/3526459982 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7180a2f20 0x7fd7180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.273+0000 7fd71fe00640 1 -- 172.21.15.47:0/485557388 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72401e070 con 0x564db38dd9e0 2024-03-21T11:34:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.273+0000 7fd71fe00640 1 -- 172.21.15.47:0/485557388 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd724009e30 con 0x564db38dd9e0 2024-03-21T11:34:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.273+0000 7fd71fe00640 1 -- 172.21.15.47:0/485557388 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72400ea60 con 0x564db38dd9e0 2024-03-21T11:34:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.274+0000 7fd73c5d0740 1 -- 172.21.15.47:0/485557388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dd9e0 msgr2=0x564db38dde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.274+0000 7fd732000640 1 --2- 172.21.15.47:0/3526459982 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7180a2f20 0x7fd7180a53e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd720009230 tx=0x7fd720005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.274+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/485557388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dd9e0 0x564db38dde60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd724009c80 tx=0x7fd724011810 comp rx=0 tx=0).stop 2024-03-21T11:34:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.274+0000 7fd73c5d0740 1 -- 172.21.15.47:0/485557388 shutdown_connections 2024-03-21T11:34:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.274+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/485557388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38de3a0 0x564db38e4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.274+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/485557388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dd9e0 0x564db38dde60 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.274+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/485557388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38dc810 0x564db38dcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.274+0000 7fd73c5d0740 1 -- 172.21.15.47:0/485557388 >> 172.21.15.47:0/485557388 conn(0x564db38d7c00 msgr2=0x564db38d8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.274+0000 7fd73c5d0740 1 -- 172.21.15.47:0/485557388 shutdown_connections 2024-03-21T11:34:36.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.274+0000 7fd73c5d0740 1 -- 172.21.15.47:0/485557388 wait complete. 2024-03-21T11:34:36.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.275+0000 7fd73c5d0740 1 -- start start 2024-03-21T11:34:36.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.275+0000 7fd73c5d0740 1 Processor -- start 2024-03-21T11:34:36.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.275+0000 7fd73c5d0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.275+0000 7fd73c5d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38dd9e0 0x564db38e6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.275+0000 7fd73c5d0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38de3a0 0x564db38edfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.275+0000 7fd73c5d0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db38e53b0 con 0x564db38de3a0 2024-03-21T11:34:36.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd73c5d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db37fd0d0 con 0x564db38dd9e0 2024-03-21T11:34:36.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd73c5d0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db38e5530 con 0x564db38dc810 2024-03-21T11:34:36.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd73c5d0740 10 client.0 _ll_register_callbacks cb 0x564db3801e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:36.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd732a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd732000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38dd9e0 0x564db38e6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd732a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38e6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59012/0 (socket says 172.21.15.47:59012) 2024-03-21T11:34:36.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd732a00640 1 -- 172.21.15.47:0/150939818 learned_addr learned my addr 172.21.15.47:0/150939818 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd733400640 1 --2- 172.21.15.47:0/150939818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38de3a0 0x564db38edfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd732a00640 1 -- 172.21.15.47:0/150939818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38dd9e0 msgr2=0x564db38e6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd732a00640 1 --2- 172.21.15.47:0/150939818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38dd9e0 0x564db38e6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd732a00640 1 -- 172.21.15.47:0/150939818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38de3a0 msgr2=0x564db38edfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd732a00640 1 --2- 172.21.15.47:0/150939818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38de3a0 0x564db38edfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd732a00640 1 -- 172.21.15.47:0/150939818 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564db35553f0 con 0x564db38dc810 2024-03-21T11:34:36.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd733400640 1 --2- 172.21.15.47:0/150939818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38de3a0 0x564db38edfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:36.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.276+0000 7fd732a00640 1 --2- 172.21.15.47:0/150939818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38e6a30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd7240058c0 tx=0x7fd724005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7be64bc8a99643f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.277+0000 7fd71cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:36.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.277+0000 7fd71cc00640 1 -- 172.21.15.47:0/150939818 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72401e040 con 0x564db38dc810 2024-03-21T11:34:36.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.277+0000 7fd71cc00640 1 -- 172.21.15.47:0/150939818 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd724004230 con 0x564db38dc810 2024-03-21T11:34:36.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.277+0000 7fd73c5d0740 1 -- 172.21.15.47:0/150939818 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd724009e30 con 0x564db38dc810 2024-03-21T11:34:36.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.277+0000 7fd71cc00640 1 -- 172.21.15.47:0/150939818 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd724033050 con 0x564db38dc810 2024-03-21T11:34:36.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.277+0000 7fd73c5d0740 10 client.29323 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:36.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.277+0000 7fd73c5d0740 1 -- 172.21.15.47:0/150939818 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564db361f7d0 con 0x564db38dc810 2024-03-21T11:34:36.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.277+0000 7fd71cc00640 1 -- 172.21.15.47:0/150939818 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd724009e30 con 0x564db38dc810 2024-03-21T11:34:36.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.278+0000 7fd71cc00640 1 -- 172.21.15.47:0/150939818 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd724004a20 con 0x564db38dc810 2024-03-21T11:34:36.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.278+0000 7fd73c5d0740 10 client.29323 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:36.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.278+0000 7fd73c5d0740 10 client.29323 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:34:36.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.278+0000 7fd73c5d0740 20 client.29323 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:36.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.278+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/150939818 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564db38f20f0 0x564db38f4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.278+0000 7fd73c5d0740 4 client.29323 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:36.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.278+0000 7fd73c5d0740 1 -- 172.21.15.47:0/150939818 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564db384e800 con 0x564db38f20f0 2024-03-21T11:34:36.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.278+0000 7fd732000640 1 --2- 172.21.15.47:0/150939818 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564db38f20f0 0x564db38f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.279+0000 7fd732000640 1 --2- 172.21.15.47:0/150939818 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564db38f20f0 0x564db38f4520 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16637b42e0d542fc server_cookie=51b6749c7c6b2527 in_seq=0 out_seq=0 2024-03-21T11:34:36.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.279+0000 7fd71cc00640 10 client.29323 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:34:36.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd71cc00640 1 -- 172.21.15.47:0/150939818 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564db384e800 con 0x564db38f20f0 2024-03-21T11:34:36.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd71cc00640 10 client.29323 handle_command_reply: tid=0 2024-03-21T11:34:36.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd73c5d0740 1 client.29323 shutdown 2024-03-21T11:34:36.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd73c5d0740 1 -- 172.21.15.47:0/150939818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 msgr2=0x564db38e6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/150939818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38e6a30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd7240058c0 tx=0x7fd724005750 comp rx=0 tx=0).stop 2024-03-21T11:34:36.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd73c5d0740 1 -- 172.21.15.47:0/150939818 shutdown_connections 2024-03-21T11:34:36.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/150939818 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564db38f20f0 0x564db38f4520 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/150939818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38de3a0 0x564db38edfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd71cc00640 0 client.29323 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:34:36.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/150939818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38dd9e0 0x564db38e6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/150939818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38e6a30 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd73c5d0740 1 -- 172.21.15.47:0/150939818 >> 172.21.15.47:0/150939818 conn(0x564db38d7c00 msgr2=0x564db38d6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd73c5d0740 1 -- 172.21.15.47:0/150939818 shutdown_connections 2024-03-21T11:34:36.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd73c5d0740 1 -- 172.21.15.47:0/150939818 wait complete. 2024-03-21T11:34:36.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.280+0000 7fd73c5d0740 20 client.29323 trim_cache size 0 max 16384 2024-03-21T11:34:36.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.297+0000 7fd73c5d0740 1 Processor -- start 2024-03-21T11:34:36.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.297+0000 7fd73c5d0740 1 -- start start 2024-03-21T11:34:36.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.297+0000 7fd73c5d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38d7c00 0x564db38f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.297+0000 7fd73c5d0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38edf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.297+0000 7fd73c5d0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38dd9e0 0x564db38f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.297+0000 7fd73c5d0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7341ac8e0 con 0x564db38dd9e0 2024-03-21T11:34:36.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.297+0000 7fd73c5d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7341ae260 con 0x564db38d7c00 2024-03-21T11:34:36.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.297+0000 7fd73c5d0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7341acf40 con 0x564db38dc810 2024-03-21T11:34:36.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.298+0000 7fd732000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.298+0000 7fd732a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38d7c00 0x564db38f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.298+0000 7fd732000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38edf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59014/0 (socket says 172.21.15.47:59014) 2024-03-21T11:34:36.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.298+0000 7fd732000640 1 -- 172.21.15.47:0/2745166458 learned_addr learned my addr 172.21.15.47:0/2745166458 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.298+0000 7fd733400640 1 --2- 172.21.15.47:0/2745166458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38dd9e0 0x564db38f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.298+0000 7fd732000640 1 -- 172.21.15.47:0/2745166458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38d7c00 msgr2=0x564db38f8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.298+0000 7fd732000640 1 --2- 172.21.15.47:0/2745166458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38d7c00 0x564db38f8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.298+0000 7fd732000640 1 -- 172.21.15.47:0/2745166458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38dd9e0 msgr2=0x564db38f9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.298+0000 7fd732000640 1 --2- 172.21.15.47:0/2745166458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38dd9e0 0x564db38f9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.298+0000 7fd732000640 1 -- 172.21.15.47:0/2745166458 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db38d8670 con 0x564db38dc810 2024-03-21T11:34:36.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.298+0000 7fd732000640 1 --2- 172.21.15.47:0/2745166458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38edf70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd720005ab0 tx=0x7fd720041040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f2bbd9d24ae32b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.299+0000 7fd71f400640 1 -- 172.21.15.47:0/2745166458 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72005a070 con 0x564db38dc810 2024-03-21T11:34:36.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.299+0000 7fd71f400640 1 -- 172.21.15.47:0/2745166458 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd720004ba0 con 0x564db38dc810 2024-03-21T11:34:36.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.299+0000 7fd71f400640 1 -- 172.21.15.47:0/2745166458 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720005430 con 0x564db38dc810 2024-03-21T11:34:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.299+0000 7fd73c5d0740 1 -- 172.21.15.47:0/2745166458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 msgr2=0x564db38edf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.299+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/2745166458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38edf70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd720005ab0 tx=0x7fd720041040 comp rx=0 tx=0).stop 2024-03-21T11:34:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.299+0000 7fd73c5d0740 1 -- 172.21.15.47:0/2745166458 shutdown_connections 2024-03-21T11:34:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.299+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/2745166458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38dd9e0 0x564db38f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.299+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/2745166458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38edf70 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.299+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/2745166458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38d7c00 0x564db38f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.299+0000 7fd73c5d0740 1 -- 172.21.15.47:0/2745166458 >> 172.21.15.47:0/2745166458 conn(0x564db38e5c40 msgr2=0x564db38d6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.299+0000 7fd73c5d0740 1 -- 172.21.15.47:0/2745166458 shutdown_connections 2024-03-21T11:34:36.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.300+0000 7fd73c5d0740 1 -- 172.21.15.47:0/2745166458 wait complete. 2024-03-21T11:34:36.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.300+0000 7fd73c5d0740 1 -- start start 2024-03-21T11:34:36.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.300+0000 7fd73c5d0740 1 Processor -- start 2024-03-21T11:34:36.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.300+0000 7fd73c5d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38d7c00 0x564db38e9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.300+0000 7fd73c5d0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38ea260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd73c5d0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38dd9e0 0x564db38ed280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd73c5d0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7341adc00 con 0x564db38dd9e0 2024-03-21T11:34:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd73c5d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7341ad5a0 con 0x564db38d7c00 2024-03-21T11:34:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd73c5d0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7341aaaa0 con 0x564db38dc810 2024-03-21T11:34:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd73c5d0740 10 client.0 _ll_register_callbacks cb 0x7fd72401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd732a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38d7c00 0x564db38e9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd732a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38d7c00 0x564db38e9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58958/0 (socket says 172.21.15.47:58958) 2024-03-21T11:34:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd732a00640 1 -- 172.21.15.47:0/3049501796 learned_addr learned my addr 172.21.15.47:0/3049501796 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd733400640 1 --2- 172.21.15.47:0/3049501796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38dd9e0 0x564db38ed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd732000640 1 --2- 172.21.15.47:0/3049501796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38ea260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd732a00640 1 -- 172.21.15.47:0/3049501796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 msgr2=0x564db38ea260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd732a00640 1 --2- 172.21.15.47:0/3049501796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38ea260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd732a00640 1 -- 172.21.15.47:0/3049501796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38dd9e0 msgr2=0x564db38ed280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd732a00640 1 --2- 172.21.15.47:0/3049501796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38dd9e0 0x564db38ed280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.301+0000 7fd732a00640 1 -- 172.21.15.47:0/3049501796 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564db38ed870 con 0x564db38d7c00 2024-03-21T11:34:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.302+0000 7fd732a00640 1 --2- 172.21.15.47:0/3049501796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38d7c00 0x564db38e9d20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd724000c00 tx=0x7fd724009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddbdc5d33519dd94 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.302+0000 7fd71cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.302+0000 7fd71cc00640 1 -- 172.21.15.47:0/3049501796 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72403c060 con 0x564db38d7c00 2024-03-21T11:34:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.302+0000 7fd73c5d0740 1 -- 172.21.15.47:0/3049501796 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd720004ba0 con 0x564db38d7c00 2024-03-21T11:34:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.302+0000 7fd71cc00640 1 -- 172.21.15.47:0/3049501796 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd724003de0 con 0x564db38d7c00 2024-03-21T11:34:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.302+0000 7fd71cc00640 1 -- 172.21.15.47:0/3049501796 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd724043050 con 0x564db38d7c00 2024-03-21T11:34:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.302+0000 7fd73c5d0740 10 client.29306 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.302+0000 7fd73c5d0740 1 -- 172.21.15.47:0/3049501796 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564db394dfd0 con 0x564db38d7c00 2024-03-21T11:34:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.302+0000 7fd71cc00640 1 -- 172.21.15.47:0/3049501796 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd720004ba0 con 0x564db38d7c00 2024-03-21T11:34:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.303+0000 7fd71cc00640 1 -- 172.21.15.47:0/3049501796 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd724037b80 con 0x564db38d7c00 2024-03-21T11:34:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.303+0000 7fd73c5d0740 10 client.29306 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.303+0000 7fd73c5d0740 10 client.29306 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:34:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.303+0000 7fd73c5d0740 20 client.29306 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.303+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/3049501796 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564db38de3a0 0x564db3950880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.303+0000 7fd73c5d0740 4 client.29306 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:34:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.303+0000 7fd73c5d0740 1 -- 172.21.15.47:0/3049501796 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564db3848b50 con 0x564db38de3a0 2024-03-21T11:34:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.303+0000 7fd732000640 1 --2- 172.21.15.47:0/3049501796 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564db38de3a0 0x564db3950880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.303+0000 7fd732000640 1 --2- 172.21.15.47:0/3049501796 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564db38de3a0 0x564db3950880 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=807c27b82c38f286 server_cookie=e50ef0fd5ba6dd39 in_seq=0 out_seq=0 2024-03-21T11:34:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.303+0000 7fd71cc00640 10 client.29306 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:34:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd71cc00640 1 -- 172.21.15.47:0/3049501796 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564db3848b50 con 0x564db38de3a0 2024-03-21T11:34:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd71cc00640 10 client.29306 handle_command_reply: tid=0 2024-03-21T11:34:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd73c5d0740 1 client.29306 shutdown 2024-03-21T11:34:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd73c5d0740 1 -- 172.21.15.47:0/3049501796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38d7c00 msgr2=0x564db38e9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/3049501796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38d7c00 0x564db38e9d20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd724000c00 tx=0x7fd724009b60 comp rx=0 tx=0).stop 2024-03-21T11:34:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd73c5d0740 1 -- 172.21.15.47:0/3049501796 shutdown_connections 2024-03-21T11:34:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/3049501796 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564db38de3a0 0x564db3950880 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/3049501796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db38dd9e0 0x564db38ed280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/3049501796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db38dc810 0x564db38ea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd71cc00640 0 client.29306 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:34:36.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd73c5d0740 1 --2- 172.21.15.47:0/3049501796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db38d7c00 0x564db38e9d20 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd73c5d0740 1 -- 172.21.15.47:0/3049501796 >> 172.21.15.47:0/3049501796 conn(0x564db38e5c40 msgr2=0x564db38e0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd73c5d0740 1 -- 172.21.15.47:0/3049501796 shutdown_connections 2024-03-21T11:34:36.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd73c5d0740 1 -- 172.21.15.47:0/3049501796 wait complete. 2024-03-21T11:34:36.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.304+0000 7fd73c5d0740 20 client.29306 trim_cache size 0 max 16384 2024-03-21T11:34:36.312 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:34:36.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.305+0000 7fd739800640 1 -- 172.21.15.47:0/3526459982 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7180a2f20 msgr2=0x7fd7180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.305+0000 7fd739800640 1 --2- 172.21.15.47:0/3526459982 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7180a2f20 0x7fd7180a53e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd720009230 tx=0x7fd720005fd0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.305+0000 7fd739800640 1 -- 172.21.15.47:0/3526459982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd734109ac0 msgr2=0x7fd734109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.305+0000 7fd739800640 1 --2- 172.21.15.47:0/3526459982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd734109ac0 0x7fd734109f70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00eb10 tx=0x7fd72c00c2d0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.305+0000 7fd739800640 1 -- 172.21.15.47:0/3526459982 shutdown_connections 2024-03-21T11:34:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.305+0000 7fd739800640 1 --2- 172.21.15.47:0/3526459982 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7180a2f20 0x7fd7180a53e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.305+0000 7fd739800640 1 --2- 172.21.15.47:0/3526459982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd734109ac0 0x7fd734109f70 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.305+0000 7fd739800640 1 --2- 172.21.15.47:0/3526459982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd734102170 0x7fd734109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.305+0000 7fd739800640 1 --2- 172.21.15.47:0/3526459982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7341005c0 0x7fd73410af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.305+0000 7fd739800640 1 -- 172.21.15.47:0/3526459982 >> 172.21.15.47:0/3526459982 conn(0x7fd7340fbb50 msgr2=0x7fd734104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.305+0000 7fd739800640 1 -- 172.21.15.47:0/3526459982 shutdown_connections 2024-03-21T11:34:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.305+0000 7fd739800640 1 -- 172.21.15.47:0/3526459982 wait complete. 2024-03-21T11:34:36.324 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:34:36.324 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:34:36.559 INFO:tasks.workunit.client.0.smithi047.stdout: 1 848017 8.03 MB/sec execute 440 sec latency 36.251 ms 2024-03-21T11:34:36.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.613+0000 7f019ea00640 1 -- 172.21.15.47:0/3327912107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01981086a0 msgr2=0x7f019810aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.613+0000 7f019ea00640 1 --2- 172.21.15.47:0/3327912107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01981086a0 0x7f019810aad0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0180009900 tx=0x7f018002bc30 comp rx=0 tx=0).stop 2024-03-21T11:34:36.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.614+0000 7f019ea00640 1 -- 172.21.15.47:0/3327912107 shutdown_connections 2024-03-21T11:34:36.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.614+0000 7f019ea00640 1 --2- 172.21.15.47:0/3327912107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f019810d9c0 0x7f019810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.614+0000 7f019ea00640 1 --2- 172.21.15.47:0/3327912107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f019810b010 0x7f019810d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.614+0000 7f019ea00640 1 --2- 172.21.15.47:0/3327912107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01981086a0 0x7f019810aad0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.614+0000 7f019ea00640 1 -- 172.21.15.47:0/3327912107 >> 172.21.15.47:0/3327912107 conn(0x7f019806e500 msgr2=0x7f0198070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.614+0000 7f019ea00640 1 -- 172.21.15.47:0/3327912107 shutdown_connections 2024-03-21T11:34:36.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.614+0000 7f019ea00640 1 -- 172.21.15.47:0/3327912107 wait complete. 2024-03-21T11:34:36.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.615+0000 7f019ea00640 1 Processor -- start 2024-03-21T11:34:36.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.615+0000 7f019ea00640 1 -- start start 2024-03-21T11:34:36.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.615+0000 7f019ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01981086a0 0x7f01981a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f019ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f019810b010 0x7f01981a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f019ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019810d9c0 0x7f01981addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f019ea00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019811c160 con 0x7f01981086a0 2024-03-21T11:34:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f019ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019811bfe0 con 0x7f019810b010 2024-03-21T11:34:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f019ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019811c2e0 con 0x7f019810d9c0 2024-03-21T11:34:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f0197400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f019810b010 0x7f01981a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f0197e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01981086a0 0x7f01981a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f0197400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f019810b010 0x7f01981a6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58978/0 (socket says 172.21.15.47:58978) 2024-03-21T11:34:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f0197400640 1 -- 172.21.15.47:0/3869001715 learned_addr learned my addr 172.21.15.47:0/3869001715 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f019ca00640 1 --2- 172.21.15.47:0/3869001715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019810d9c0 0x7f01981addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f0197400640 1 -- 172.21.15.47:0/3869001715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019810d9c0 msgr2=0x7f01981addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f0197400640 1 --2- 172.21.15.47:0/3869001715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019810d9c0 0x7f01981addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f0197400640 1 -- 172.21.15.47:0/3869001715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01981086a0 msgr2=0x7f01981a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f0197400640 1 --2- 172.21.15.47:0/3869001715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01981086a0 0x7f01981a6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f0197400640 1 -- 172.21.15.47:0/3869001715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01981ae420 con 0x7f019810b010 2024-03-21T11:34:36.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.616+0000 7f0197e00640 1 --2- 172.21.15.47:0/3869001715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01981086a0 0x7f01981a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:36.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.617+0000 7f0197400640 1 --2- 172.21.15.47:0/3869001715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f019810b010 0x7f01981a6d90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f018800d950 tx=0x7f018800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.617+0000 7f0194c00640 1 -- 172.21.15.47:0/3869001715 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188017070 con 0x7f019810b010 2024-03-21T11:34:36.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.617+0000 7f019ea00640 1 -- 172.21.15.47:0/3869001715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01981ae710 con 0x7f019810b010 2024-03-21T11:34:36.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.617+0000 7f019ea00640 1 -- 172.21.15.47:0/3869001715 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01981aecb0 con 0x7f019810b010 2024-03-21T11:34:36.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.617+0000 7f0194c00640 1 -- 172.21.15.47:0/3869001715 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0188009c30 con 0x7f019810b010 2024-03-21T11:34:36.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.617+0000 7f0194c00640 1 -- 172.21.15.47:0/3869001715 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018800f6b0 con 0x7f019810b010 2024-03-21T11:34:36.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.621+0000 7f0194c00640 1 -- 172.21.15.47:0/3869001715 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f018800f850 con 0x7f019810b010 2024-03-21T11:34:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.621+0000 7f01a1322740 1 Processor -- start 2024-03-21T11:34:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.621+0000 7f01a1322740 1 -- start start 2024-03-21T11:34:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f01a1322740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa99810 0x55882fa99c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f01a1322740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa9a9e0 0x55882fa9ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f01a1322740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9b3a0 0x55882faa1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f01a1322740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882f9fe2c0 con 0x55882fa9a9e0 2024-03-21T11:34:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f01a1322740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882f9a6c90 con 0x55882fa99810 2024-03-21T11:34:36.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f01a1322740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882f9fa360 con 0x55882fa9b3a0 2024-03-21T11:34:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f019ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa99810 0x55882fa99c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f019ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa99810 0x55882fa99c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58988/0 (socket says 172.21.15.47:58988) 2024-03-21T11:34:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f019ca00640 1 -- 172.21.15.47:0/827933006 learned_addr learned my addr 172.21.15.47:0/827933006 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f0197400640 1 --2- 172.21.15.47:0/827933006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9b3a0 0x55882faa1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f0197e00640 1 --2- 172.21.15.47:0/827933006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa9a9e0 0x55882fa9ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f0194c00640 1 --2- 172.21.15.47:0/3869001715 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f017c0a2fa0 0x7f017c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f019ca00640 1 -- 172.21.15.47:0/827933006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9b3a0 msgr2=0x55882faa1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f019ca00640 1 --2- 172.21.15.47:0/827933006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9b3a0 0x55882faa1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.622+0000 7f019ca00640 1 -- 172.21.15.47:0/827933006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa9a9e0 msgr2=0x55882fa9ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.623+0000 7f019ca00640 1 --2- 172.21.15.47:0/827933006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa9a9e0 0x55882fa9ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.623+0000 7f019ca00640 1 -- 172.21.15.47:0/827933006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55882f9cb080 con 0x55882fa99810 2024-03-21T11:34:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.623+0000 7f0194c00640 1 -- 172.21.15.47:0/3869001715 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f01880cf670 con 0x7f019810b010 2024-03-21T11:34:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.623+0000 7f0197400640 1 --2- 172.21.15.47:0/827933006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9b3a0 0x55882faa1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:36.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.623+0000 7f0197e00640 1 --2- 172.21.15.47:0/827933006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa9a9e0 0x55882fa9ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:36.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.623+0000 7f019ca00640 1 --2- 172.21.15.47:0/3869001715 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f017c0a2fa0 0x7f017c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.623+0000 7f019ca00640 1 --2- 172.21.15.47:0/827933006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa99810 0x55882fa99c30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0190009f90 tx=0x7f01900118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6cd8b3e85ae0444 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.624+0000 7f018d200640 1 -- 172.21.15.47:0/827933006 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190003cb0 con 0x55882fa99810 2024-03-21T11:34:36.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.624+0000 7f018d200640 1 -- 172.21.15.47:0/827933006 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f019000fbc0 con 0x55882fa99810 2024-03-21T11:34:36.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.624+0000 7f018d200640 1 -- 172.21.15.47:0/827933006 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019001bba0 con 0x55882fa99810 2024-03-21T11:34:36.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.624+0000 7f01a1322740 1 -- 172.21.15.47:0/827933006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa99810 msgr2=0x55882fa99c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.624+0000 7f01a1322740 1 --2- 172.21.15.47:0/827933006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa99810 0x55882fa99c30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0190009f90 tx=0x7f01900118d0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.624+0000 7f019ca00640 1 --2- 172.21.15.47:0/3869001715 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f017c0a2fa0 0x7f017c0a5460 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0190020000 tx=0x7f0190021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.625+0000 7f01a1322740 1 -- 172.21.15.47:0/827933006 shutdown_connections 2024-03-21T11:34:36.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.625+0000 7f01a1322740 1 --2- 172.21.15.47:0/827933006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9b3a0 0x55882faa1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.625+0000 7f01a1322740 1 --2- 172.21.15.47:0/827933006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa9a9e0 0x55882fa9ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.625+0000 7f01a1322740 1 --2- 172.21.15.47:0/827933006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa99810 0x55882fa99c30 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.625+0000 7f01a1322740 1 -- 172.21.15.47:0/827933006 >> 172.21.15.47:0/827933006 conn(0x55882fa94c00 msgr2=0x55882fa95000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.625+0000 7f01a1322740 1 -- 172.21.15.47:0/827933006 shutdown_connections 2024-03-21T11:34:36.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.625+0000 7f01a1322740 1 -- 172.21.15.47:0/827933006 wait complete. 2024-03-21T11:34:36.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.625+0000 7f01a1322740 1 -- start start 2024-03-21T11:34:36.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.625+0000 7f01a1322740 1 Processor -- start 2024-03-21T11:34:36.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f01a1322740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa99810 0x55882faa3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f01a1322740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9a9e0 0x55882faa3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f01a1322740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9b3a0 0x55882faab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f01a1322740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882faa23b0 con 0x55882fa99810 2024-03-21T11:34:36.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f01a1322740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882f9ba0d0 con 0x55882fa9a9e0 2024-03-21T11:34:36.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f01a1322740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882faa2530 con 0x55882fa9b3a0 2024-03-21T11:34:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f01a1322740 10 client.0 _ll_register_callbacks cb 0x55882f9bee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f0197400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9a9e0 0x55882faa3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f0197e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa99810 0x55882faa3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f0197400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9a9e0 0x55882faa3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58994/0 (socket says 172.21.15.47:58994) 2024-03-21T11:34:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f0197400640 1 -- 172.21.15.47:0/522529446 learned_addr learned my addr 172.21.15.47:0/522529446 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f019ca00640 1 --2- 172.21.15.47:0/522529446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9b3a0 0x55882faab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f0197400640 1 -- 172.21.15.47:0/522529446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9b3a0 msgr2=0x55882faab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f0197400640 1 --2- 172.21.15.47:0/522529446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9b3a0 0x55882faab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.626+0000 7f0197400640 1 -- 172.21.15.47:0/522529446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa99810 msgr2=0x55882faa3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.627+0000 7f0197400640 1 --2- 172.21.15.47:0/522529446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa99810 0x55882faa3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.627+0000 7f0197400640 1 -- 172.21.15.47:0/522529446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55882f7123f0 con 0x55882fa9a9e0 2024-03-21T11:34:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.627+0000 7f0197e00640 1 --2- 172.21.15.47:0/522529446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa99810 0x55882faa3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.627+0000 7f0197400640 1 --2- 172.21.15.47:0/522529446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9a9e0 0x55882faa3fc0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f018809b760 tx=0x7f018800dfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c819ae1afb8daa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.627+0000 7f0185600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.627+0000 7f0185600640 1 -- 172.21.15.47:0/522529446 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188020070 con 0x55882fa9a9e0 2024-03-21T11:34:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.627+0000 7f0185600640 1 -- 172.21.15.47:0/522529446 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f01880980d0 con 0x55882fa9a9e0 2024-03-21T11:34:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.627+0000 7f01a1322740 1 -- 172.21.15.47:0/522529446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f019000fbc0 con 0x55882fa9a9e0 2024-03-21T11:34:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.627+0000 7f0185600640 1 -- 172.21.15.47:0/522529446 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01880b4050 con 0x55882fa9a9e0 2024-03-21T11:34:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.627+0000 7f01a1322740 10 client.29318 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.627+0000 7f01a1322740 1 -- 172.21.15.47:0/522529446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55882f7dc7d0 con 0x55882fa9a9e0 2024-03-21T11:34:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.628+0000 7f0185600640 1 -- 172.21.15.47:0/522529446 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f019000fbc0 con 0x55882fa9a9e0 2024-03-21T11:34:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.628+0000 7f0185600640 1 -- 172.21.15.47:0/522529446 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f01880b4ce0 con 0x55882fa9a9e0 2024-03-21T11:34:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.628+0000 7f01a1322740 10 client.29318 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.628+0000 7f01a1322740 10 client.29318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.628+0000 7f01a1322740 20 client.29318 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.628+0000 7f01a1322740 1 --2- 172.21.15.47:0/522529446 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55882faaf3d0 0x55882fab1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.628+0000 7f01a1322740 4 client.29318 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:36.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.628+0000 7f01a1322740 1 -- 172.21.15.47:0/522529446 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55882fa0b800 con 0x55882faaf3d0 2024-03-21T11:34:36.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.628+0000 7f0197e00640 1 --2- 172.21.15.47:0/522529446 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55882faaf3d0 0x55882fab1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.629+0000 7f0197e00640 1 --2- 172.21.15.47:0/522529446 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55882faaf3d0 0x55882fab1830 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afd410ead7f7e67a server_cookie=ab088de75118f666 in_seq=0 out_seq=0 2024-03-21T11:34:36.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.629+0000 7f0185600640 10 client.29318 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:36.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f0185600640 1 -- 172.21.15.47:0/522529446 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55882fa0b800 con 0x55882faaf3d0 2024-03-21T11:34:36.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f0185600640 10 client.29318 handle_command_reply: tid=0 2024-03-21T11:34:36.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f01a1322740 1 client.29318 shutdown 2024-03-21T11:34:36.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f01a1322740 1 -- 172.21.15.47:0/522529446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9a9e0 msgr2=0x55882faa3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f01a1322740 1 --2- 172.21.15.47:0/522529446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9a9e0 0x55882faa3fc0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f018809b760 tx=0x7f018800dfa0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f01a1322740 1 -- 172.21.15.47:0/522529446 shutdown_connections 2024-03-21T11:34:36.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f01a1322740 1 --2- 172.21.15.47:0/522529446 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55882faaf3d0 0x55882fab1830 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f01a1322740 1 --2- 172.21.15.47:0/522529446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9b3a0 0x55882faab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f01a1322740 1 --2- 172.21.15.47:0/522529446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9a9e0 0x55882faa3fc0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f01a1322740 1 --2- 172.21.15.47:0/522529446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa99810 0x55882faa3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f0185600640 0 client.29318 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f01a1322740 1 -- 172.21.15.47:0/522529446 >> 172.21.15.47:0/522529446 conn(0x55882fa94c00 msgr2=0x55882fa93eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f01a1322740 1 -- 172.21.15.47:0/522529446 shutdown_connections 2024-03-21T11:34:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f01a1322740 1 -- 172.21.15.47:0/522529446 wait complete. 2024-03-21T11:34:36.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.630+0000 7f01a1322740 20 client.29318 trim_cache size 0 max 16384 2024-03-21T11:34:36.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.646+0000 7f01a1322740 1 Processor -- start 2024-03-21T11:34:36.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.646+0000 7f01a1322740 1 -- start start 2024-03-21T11:34:36.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.646+0000 7f01a1322740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa99810 0x55882fa93d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.646+0000 7f01a1322740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9a9e0 0x55882faaafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.646+0000 7f01a1322740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9b3a0 0x55882fab60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.646+0000 7f01a1322740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01981b8f00 con 0x55882fa99810 2024-03-21T11:34:36.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.646+0000 7f01a1322740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01981ba880 con 0x55882fa9b3a0 2024-03-21T11:34:36.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.646+0000 7f01a1322740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01981b9560 con 0x55882fa9a9e0 2024-03-21T11:34:36.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.646+0000 7f0197400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9a9e0 0x55882faaafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.646+0000 7f019ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9b3a0 0x55882fab60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.646+0000 7f0197400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9a9e0 0x55882faaafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59040/0 (socket says 172.21.15.47:59040) 2024-03-21T11:34:36.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.647+0000 7f0197400640 1 -- 172.21.15.47:0/1911285310 learned_addr learned my addr 172.21.15.47:0/1911285310 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.647+0000 7f0197e00640 1 --2- 172.21.15.47:0/1911285310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa99810 0x55882fa93d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.647+0000 7f0197400640 1 -- 172.21.15.47:0/1911285310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9b3a0 msgr2=0x55882fab60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.647+0000 7f0197400640 1 --2- 172.21.15.47:0/1911285310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9b3a0 0x55882fab60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.647+0000 7f0197400640 1 -- 172.21.15.47:0/1911285310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa99810 msgr2=0x55882fa93d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.647+0000 7f0197400640 1 --2- 172.21.15.47:0/1911285310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa99810 0x55882fa93d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.647+0000 7f0197400640 1 -- 172.21.15.47:0/1911285310 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55882fa9a460 con 0x55882fa9a9e0 2024-03-21T11:34:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.647+0000 7f0197400640 1 --2- 172.21.15.47:0/1911285310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9a9e0 0x55882faaafc0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0188010c80 tx=0x7f0188010b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4339df440c33055 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.647+0000 7f0187e00640 1 -- 172.21.15.47:0/1911285310 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880b8070 con 0x55882fa9a9e0 2024-03-21T11:34:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.647+0000 7f0187e00640 1 -- 172.21.15.47:0/1911285310 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f018809fb00 con 0x55882fa9a9e0 2024-03-21T11:34:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.648+0000 7f0187e00640 1 -- 172.21.15.47:0/1911285310 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188020070 con 0x55882fa9a9e0 2024-03-21T11:34:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.648+0000 7f01a1322740 1 -- 172.21.15.47:0/1911285310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9a9e0 msgr2=0x55882faaafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.648+0000 7f01a1322740 1 --2- 172.21.15.47:0/1911285310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9a9e0 0x55882faaafc0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0188010c80 tx=0x7f0188010b80 comp rx=0 tx=0).stop 2024-03-21T11:34:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.648+0000 7f01a1322740 1 -- 172.21.15.47:0/1911285310 shutdown_connections 2024-03-21T11:34:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.648+0000 7f01a1322740 1 --2- 172.21.15.47:0/1911285310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9b3a0 0x55882fab60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.648+0000 7f01a1322740 1 --2- 172.21.15.47:0/1911285310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa9a9e0 0x55882faaafc0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.648+0000 7f01a1322740 1 --2- 172.21.15.47:0/1911285310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa99810 0x55882fa93d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.648+0000 7f01a1322740 1 -- 172.21.15.47:0/1911285310 >> 172.21.15.47:0/1911285310 conn(0x55882fa94c00 msgr2=0x55882fa93620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.648+0000 7f01a1322740 1 -- 172.21.15.47:0/1911285310 shutdown_connections 2024-03-21T11:34:36.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.648+0000 7f01a1322740 1 -- 172.21.15.47:0/1911285310 wait complete. 2024-03-21T11:34:36.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.649+0000 7f01a1322740 1 -- start start 2024-03-21T11:34:36.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.649+0000 7f01a1322740 1 Processor -- start 2024-03-21T11:34:36.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.649+0000 7f01a1322740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa99810 0x55882faaacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.649+0000 7f01a1322740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa9a9e0 0x55882faa92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.649+0000 7f01a1322740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9b3a0 0x55882faa9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.649+0000 7f01a1322740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01981ba220 con 0x55882fa9a9e0 2024-03-21T11:34:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.649+0000 7f01a1322740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01981b9bc0 con 0x55882fa9b3a0 2024-03-21T11:34:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.649+0000 7f01a1322740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01981b70c0 con 0x55882fa99810 2024-03-21T11:34:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.649+0000 7f01a1322740 10 client.0 _ll_register_callbacks cb 0x7f0188097d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f0197e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa99810 0x55882faaacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f019ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9b3a0 0x55882faa9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f0197400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa9a9e0 0x55882faa92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f019ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9b3a0 0x55882faa9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59002/0 (socket says 172.21.15.47:59002) 2024-03-21T11:34:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f0197e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa99810 0x55882faaacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59056/0 (socket says 172.21.15.47:59056) 2024-03-21T11:34:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f019ca00640 1 -- 172.21.15.47:0/4060267636 learned_addr learned my addr 172.21.15.47:0/4060267636 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f019ca00640 1 -- 172.21.15.47:0/4060267636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa99810 msgr2=0x55882faaacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f019ca00640 1 --2- 172.21.15.47:0/4060267636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa99810 0x55882faaacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f019ca00640 1 -- 172.21.15.47:0/4060267636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa9a9e0 msgr2=0x55882faa92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f019ca00640 1 --2- 172.21.15.47:0/4060267636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa9a9e0 0x55882faa92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f019ca00640 1 -- 172.21.15.47:0/4060267636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55882faa9e10 con 0x55882fa9b3a0 2024-03-21T11:34:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f0197e00640 1 --2- 172.21.15.47:0/4060267636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa99810 0x55882faaacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f0197400640 1 --2- 172.21.15.47:0/4060267636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa9a9e0 0x55882faa92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f019ca00640 1 --2- 172.21.15.47:0/4060267636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9b3a0 0x55882faa9820 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0190026000 tx=0x7f0190009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=688213f6c5d29c32 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.650+0000 7f0185600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.651+0000 7f0185600640 1 -- 172.21.15.47:0/4060267636 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190029040 con 0x55882fa9b3a0 2024-03-21T11:34:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.651+0000 7f01a1322740 1 -- 172.21.15.47:0/4060267636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f018809fb00 con 0x55882fa9b3a0 2024-03-21T11:34:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.651+0000 7f0185600640 1 -- 172.21.15.47:0/4060267636 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f019001b630 con 0x55882fa9b3a0 2024-03-21T11:34:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.651+0000 7f0185600640 1 -- 172.21.15.47:0/4060267636 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0190044050 con 0x55882fa9b3a0 2024-03-21T11:34:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.651+0000 7f01a1322740 10 client.29324 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.651+0000 7f01a1322740 1 -- 172.21.15.47:0/4060267636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55882fb0bb30 con 0x55882fa9b3a0 2024-03-21T11:34:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.651+0000 7f0185600640 1 -- 172.21.15.47:0/4060267636 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f018809fb00 con 0x55882fa9b3a0 2024-03-21T11:34:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.651+0000 7f0185600640 1 -- 172.21.15.47:0/4060267636 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0190031440 con 0x55882fa9b3a0 2024-03-21T11:34:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.652+0000 7f01a1322740 10 client.29324 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.652+0000 7f01a1322740 10 client.29324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.652+0000 7f01a1322740 20 client.29324 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.652+0000 7f01a1322740 1 --2- 172.21.15.47:0/4060267636 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55882fb0c2e0 0x55882fb0e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.652+0000 7f01a1322740 4 client.29324 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:34:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.652+0000 7f01a1322740 1 -- 172.21.15.47:0/4060267636 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55882fa05b50 con 0x55882fb0c2e0 2024-03-21T11:34:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.652+0000 7f0197e00640 1 --2- 172.21.15.47:0/4060267636 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55882fb0c2e0 0x55882fb0e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.652+0000 7f0197e00640 1 --2- 172.21.15.47:0/4060267636 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55882fb0c2e0 0x55882fb0e700 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18884914d501da server_cookie=8dd307cffc020f25 in_seq=0 out_seq=0 2024-03-21T11:34:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.652+0000 7f0185600640 10 client.29324 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f0185600640 1 -- 172.21.15.47:0/4060267636 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55882fa05b50 con 0x55882fb0c2e0 2024-03-21T11:34:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f0185600640 10 client.29324 handle_command_reply: tid=0 2024-03-21T11:34:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f01a1322740 1 client.29324 shutdown 2024-03-21T11:34:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f01a1322740 1 -- 172.21.15.47:0/4060267636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9b3a0 msgr2=0x55882faa9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f01a1322740 1 --2- 172.21.15.47:0/4060267636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9b3a0 0x55882faa9820 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0190026000 tx=0x7f0190009ca0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f01a1322740 1 -- 172.21.15.47:0/4060267636 shutdown_connections 2024-03-21T11:34:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f01a1322740 1 --2- 172.21.15.47:0/4060267636 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55882fb0c2e0 0x55882fb0e700 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f01a1322740 1 --2- 172.21.15.47:0/4060267636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55882fa9b3a0 0x55882faa9820 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f01a1322740 1 --2- 172.21.15.47:0/4060267636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55882fa9a9e0 0x55882faa92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f01a1322740 1 --2- 172.21.15.47:0/4060267636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55882fa99810 0x55882faaacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f01a1322740 1 -- 172.21.15.47:0/4060267636 >> 172.21.15.47:0/4060267636 conn(0x55882fa94c00 msgr2=0x55882fa93eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f0185600640 0 client.29324 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f01a1322740 1 -- 172.21.15.47:0/4060267636 shutdown_connections 2024-03-21T11:34:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f01a1322740 1 -- 172.21.15.47:0/4060267636 wait complete. 2024-03-21T11:34:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.653+0000 7f01a1322740 20 client.29324 trim_cache size 0 max 16384 2024-03-21T11:34:36.659 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:34:36.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.654+0000 7f019ea00640 1 -- 172.21.15.47:0/3869001715 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f017c0a2fa0 msgr2=0x7f017c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.654+0000 7f019ea00640 1 --2- 172.21.15.47:0/3869001715 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f017c0a2fa0 0x7f017c0a5460 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0190020000 tx=0x7f0190021000 comp rx=0 tx=0).stop 2024-03-21T11:34:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.654+0000 7f019ea00640 1 -- 172.21.15.47:0/3869001715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f019810b010 msgr2=0x7f01981a6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.654+0000 7f019ea00640 1 --2- 172.21.15.47:0/3869001715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f019810b010 0x7f01981a6d90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f018800d950 tx=0x7f018800de10 comp rx=0 tx=0).stop 2024-03-21T11:34:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.654+0000 7f019ea00640 1 -- 172.21.15.47:0/3869001715 shutdown_connections 2024-03-21T11:34:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.654+0000 7f019ea00640 1 --2- 172.21.15.47:0/3869001715 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f017c0a2fa0 0x7f017c0a5460 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.654+0000 7f019ea00640 1 --2- 172.21.15.47:0/3869001715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019810d9c0 0x7f01981addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.654+0000 7f019ea00640 1 --2- 172.21.15.47:0/3869001715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f019810b010 0x7f01981a6d90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.654+0000 7f019ea00640 1 --2- 172.21.15.47:0/3869001715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01981086a0 0x7f01981a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.654+0000 7f019ea00640 1 -- 172.21.15.47:0/3869001715 >> 172.21.15.47:0/3869001715 conn(0x7f019806e500 msgr2=0x7f019810c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.654+0000 7f019ea00640 1 -- 172.21.15.47:0/3869001715 shutdown_connections 2024-03-21T11:34:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:36.654+0000 7f019ea00640 1 -- 172.21.15.47:0/3869001715 wait complete. 2024-03-21T11:34:36.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:36 smithi047 ceph-mon[31351]: pgmap v738: 129 pgs: 129 active+clean; 39 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-03-21T11:34:36.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:36 smithi100 ceph-mon[36245]: pgmap v738: 129 pgs: 129 active+clean; 39 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-03-21T11:34:36.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:36 smithi102 ceph-mon[36748]: pgmap v738: 129 pgs: 129 active+clean; 39 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 197 op/s 2024-03-21T11:34:37.560 INFO:tasks.workunit.client.0.smithi047.stdout: 1 849262 8.03 MB/sec execute 441 sec latency 66.327 ms 2024-03-21T11:34:37.678 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:34:37.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:34:37.679 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:34:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.916+0000 7ffad7200640 1 -- 172.21.15.47:0/817718262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffad8104930 msgr2=0x7ffad8106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.916+0000 7ffad7200640 1 --2- 172.21.15.47:0/817718262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffad8104930 0x7ffad8106d20 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009900 tx=0x7ffac002bc30 comp rx=0 tx=0).stop 2024-03-21T11:34:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.916+0000 7ffad7200640 1 -- 172.21.15.47:0/817718262 shutdown_connections 2024-03-21T11:34:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.916+0000 7ffad7200640 1 --2- 172.21.15.47:0/817718262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8107260 0x7ffad81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.916+0000 7ffad7200640 1 --2- 172.21.15.47:0/817718262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffad8104930 0x7ffad8106d20 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.916+0000 7ffad7200640 1 --2- 172.21.15.47:0/817718262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffad8102000 0x7ffad81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.916+0000 7ffad7200640 1 -- 172.21.15.47:0/817718262 >> 172.21.15.47:0/817718262 conn(0x7ffad80fba10 msgr2=0x7ffad80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:37.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.916+0000 7ffad7200640 1 -- 172.21.15.47:0/817718262 shutdown_connections 2024-03-21T11:34:37.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.916+0000 7ffad7200640 1 -- 172.21.15.47:0/817718262 wait complete. 2024-03-21T11:34:37.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.917+0000 7ffad7200640 1 Processor -- start 2024-03-21T11:34:37.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.917+0000 7ffad7200640 1 -- start start 2024-03-21T11:34:37.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.917+0000 7ffad7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8102000 0x7ffad8199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.917+0000 7ffad7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffad8104930 0x7ffad819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.917+0000 7ffad7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffad8107260 0x7ffad81a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.917+0000 7ffad7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad810d260 con 0x7ffad8104930 2024-03-21T11:34:37.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.917+0000 7ffad7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad810d0e0 con 0x7ffad8102000 2024-03-21T11:34:37.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.917+0000 7ffad7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad810d3e0 con 0x7ffad8107260 2024-03-21T11:34:37.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.918+0000 7ffad5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8102000 0x7ffad8199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.918+0000 7ffad5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8102000 0x7ffad8199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59018/0 (socket says 172.21.15.47:59018) 2024-03-21T11:34:37.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.918+0000 7ffad5e00640 1 -- 172.21.15.47:0/2769482247 learned_addr learned my addr 172.21.15.47:0/2769482247 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:37.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.918+0000 7ffacd400640 1 --2- 172.21.15.47:0/2769482247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffad8104930 0x7ffad819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.918+0000 7ffad6800640 1 --2- 172.21.15.47:0/2769482247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffad8107260 0x7ffad81a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.918+0000 7ffad5e00640 1 -- 172.21.15.47:0/2769482247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffad8107260 msgr2=0x7ffad81a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:37.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.918+0000 7ffad5e00640 1 --2- 172.21.15.47:0/2769482247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffad8107260 0x7ffad81a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.918+0000 7ffad5e00640 1 -- 172.21.15.47:0/2769482247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffad8104930 msgr2=0x7ffad819a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:37.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.918+0000 7ffad5e00640 1 --2- 172.21.15.47:0/2769482247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffad8104930 0x7ffad819a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.918+0000 7ffad5e00640 1 -- 172.21.15.47:0/2769482247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad81a16f0 con 0x7ffad8102000 2024-03-21T11:34:37.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.918+0000 7ffad5e00640 1 --2- 172.21.15.47:0/2769482247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8102000 0x7ffad8199ad0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ffac800e9f0 tx=0x7ffac800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:37.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.919+0000 7ffacf400640 1 -- 172.21.15.47:0/2769482247 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac800cd80 con 0x7ffad8102000 2024-03-21T11:34:37.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.919+0000 7ffacf400640 1 -- 172.21.15.47:0/2769482247 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffac8010910 con 0x7ffad8102000 2024-03-21T11:34:37.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.919+0000 7ffacf400640 1 -- 172.21.15.47:0/2769482247 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8018990 con 0x7ffad8102000 2024-03-21T11:34:37.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.919+0000 7ffad7200640 1 -- 172.21.15.47:0/2769482247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad81a19e0 con 0x7ffad8102000 2024-03-21T11:34:37.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.919+0000 7ffad7200640 1 -- 172.21.15.47:0/2769482247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad81a1f00 con 0x7ffad8102000 2024-03-21T11:34:37.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.922+0000 7ffadff87740 1 Processor -- start 2024-03-21T11:34:37.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.922+0000 7ffacf400640 1 -- 172.21.15.47:0/2769482247 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ffac8010ac0 con 0x7ffad8102000 2024-03-21T11:34:37.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.922+0000 7ffadff87740 1 -- start start 2024-03-21T11:34:37.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffadff87740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5624780782b0 0x5624780786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffadff87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562478078ca0 0x56247807d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffadff87740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56247807de10 0x5624780802a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffadff87740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562477fddeb0 con 0x5624780782b0 2024-03-21T11:34:37.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffadff87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562477f86a60 con 0x562478078ca0 2024-03-21T11:34:37.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffadff87740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562477fd9f50 con 0x56247807de10 2024-03-21T11:34:37.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffad5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562478078ca0 0x56247807d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffad6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5624780782b0 0x5624780786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffad5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562478078ca0 0x56247807d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59026/0 (socket says 172.21.15.47:59026) 2024-03-21T11:34:37.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffad5e00640 1 -- 172.21.15.47:0/2298337296 learned_addr learned my addr 172.21.15.47:0/2298337296 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:37.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffacd400640 1 --2- 172.21.15.47:0/2298337296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56247807de10 0x5624780802a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffad5e00640 1 -- 172.21.15.47:0/2298337296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56247807de10 msgr2=0x5624780802a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffad5e00640 1 --2- 172.21.15.47:0/2298337296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56247807de10 0x5624780802a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffad5e00640 1 -- 172.21.15.47:0/2298337296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5624780782b0 msgr2=0x5624780786d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.923+0000 7ffad5e00640 1 --2- 172.21.15.47:0/2298337296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5624780782b0 0x5624780786d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.924+0000 7ffad5e00640 1 -- 172.21.15.47:0/2298337296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562477dba7e0 con 0x562478078ca0 2024-03-21T11:34:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.924+0000 7ffad5e00640 1 --2- 172.21.15.47:0/2298337296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562478078ca0 0x56247807d7a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ffac80d3b90 tx=0x7ffac80d8520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a34d41ddbd169ad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.924+0000 7ffacf400640 1 --2- 172.21.15.47:0/2769482247 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffabc0a2f20 0x7ffabc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.924+0000 7ffac7400640 1 -- 172.21.15.47:0/2298337296 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80e3070 con 0x562478078ca0 2024-03-21T11:34:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.924+0000 7ffac7400640 1 -- 172.21.15.47:0/2298337296 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffac809bec0 con 0x562478078ca0 2024-03-21T11:34:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.924+0000 7ffac7400640 1 -- 172.21.15.47:0/2298337296 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80806f0 con 0x562478078ca0 2024-03-21T11:34:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.924+0000 7ffacf400640 1 -- 172.21.15.47:0/2769482247 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffac80cfad0 con 0x7ffad8102000 2024-03-21T11:34:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.924+0000 7ffad6800640 1 --2- 172.21.15.47:0/2769482247 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffabc0a2f20 0x7ffabc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.925+0000 7ffadff87740 1 -- 172.21.15.47:0/2298337296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562478078ca0 msgr2=0x56247807d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.925+0000 7ffadff87740 1 --2- 172.21.15.47:0/2298337296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562478078ca0 0x56247807d7a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ffac80d3b90 tx=0x7ffac80d8520 comp rx=0 tx=0).stop 2024-03-21T11:34:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.925+0000 7ffadff87740 1 -- 172.21.15.47:0/2298337296 shutdown_connections 2024-03-21T11:34:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.925+0000 7ffadff87740 1 --2- 172.21.15.47:0/2298337296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56247807de10 0x5624780802a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.925+0000 7ffadff87740 1 --2- 172.21.15.47:0/2298337296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562478078ca0 0x56247807d7a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.925+0000 7ffadff87740 1 --2- 172.21.15.47:0/2298337296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5624780782b0 0x5624780786d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.925+0000 7ffadff87740 1 -- 172.21.15.47:0/2298337296 >> 172.21.15.47:0/2298337296 conn(0x562478073820 msgr2=0x562478073c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.925+0000 7ffad6800640 1 --2- 172.21.15.47:0/2769482247 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffabc0a2f20 0x7ffabc0a53e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x562478079500 tx=0x7ffad0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.925+0000 7ffadff87740 1 -- 172.21.15.47:0/2298337296 shutdown_connections 2024-03-21T11:34:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.925+0000 7ffadff87740 1 -- 172.21.15.47:0/2298337296 wait complete. 2024-03-21T11:34:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.926+0000 7ffadff87740 1 -- start start 2024-03-21T11:34:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.926+0000 7ffadff87740 1 Processor -- start 2024-03-21T11:34:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.926+0000 7ffadff87740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624780782b0 0x562478082470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.926+0000 7ffadff87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562478078ca0 0x5624780829b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.926+0000 7ffadff87740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 0x5624780899f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.926+0000 7ffadff87740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562478080dc0 con 0x56247807de10 2024-03-21T11:34:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffadff87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562477f99eb0 con 0x562478078ca0 2024-03-21T11:34:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffadff87740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562478080f40 con 0x5624780782b0 2024-03-21T11:34:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffadff87740 10 client.0 _ll_register_callbacks cb 0x562477f8e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffad5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624780782b0 0x562478082470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffad5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624780782b0 0x562478082470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59110/0 (socket says 172.21.15.47:59110) 2024-03-21T11:34:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffad5e00640 1 -- 172.21.15.47:0/3950868452 learned_addr learned my addr 172.21.15.47:0/3950868452 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffacd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562478078ca0 0x5624780829b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffad6800640 1 --2- 172.21.15.47:0/3950868452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 0x5624780899f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffad5e00640 1 -- 172.21.15.47:0/3950868452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562478078ca0 msgr2=0x5624780829b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffad5e00640 1 --2- 172.21.15.47:0/3950868452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562478078ca0 0x5624780829b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffad5e00640 1 -- 172.21.15.47:0/3950868452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 msgr2=0x5624780899f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffad5e00640 1 --2- 172.21.15.47:0/3950868452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 0x5624780899f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffad5e00640 1 -- 172.21.15.47:0/3950868452 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562477e1af70 con 0x5624780782b0 2024-03-21T11:34:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffad5e00640 1 --2- 172.21.15.47:0/3950868452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624780782b0 0x562478082470 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ffac80835d0 tx=0x7ffac80cfd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f60ca9499a52fe9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.927+0000 7ffabbe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.928+0000 7ffabbe00640 1 -- 172.21.15.47:0/3950868452 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80830b0 con 0x5624780782b0 2024-03-21T11:34:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.928+0000 7ffabbe00640 1 -- 172.21.15.47:0/3950868452 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffac80cd2f0 con 0x5624780782b0 2024-03-21T11:34:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.928+0000 7ffadff87740 1 -- 172.21.15.47:0/3950868452 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffac809bec0 con 0x5624780782b0 2024-03-21T11:34:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.928+0000 7ffabbe00640 1 -- 172.21.15.47:0/3950868452 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffac8083430 con 0x5624780782b0 2024-03-21T11:34:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.928+0000 7ffadff87740 10 client.29359 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.928+0000 7ffadff87740 1 -- 172.21.15.47:0/3950868452 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562477cb2170 con 0x5624780782b0 2024-03-21T11:34:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.928+0000 7ffabbe00640 1 -- 172.21.15.47:0/3950868452 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffac809bec0 con 0x5624780782b0 2024-03-21T11:34:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.928+0000 7ffabbe00640 1 -- 172.21.15.47:0/3950868452 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffac8098530 con 0x5624780782b0 2024-03-21T11:34:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.928+0000 7ffadff87740 10 client.29359 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.928+0000 7ffadff87740 10 client.29359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.929+0000 7ffadff87740 20 client.29359 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.929+0000 7ffadff87740 1 --2- 172.21.15.47:0/3950868452 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56247808dee0 0x5624780902d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.929+0000 7ffadff87740 4 client.29359 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.929+0000 7ffadff87740 1 -- 172.21.15.47:0/3950868452 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562477feb3f0 con 0x56247808dee0 2024-03-21T11:34:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.929+0000 7ffacd400640 1 --2- 172.21.15.47:0/3950868452 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56247808dee0 0x5624780902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.929+0000 7ffacd400640 1 --2- 172.21.15.47:0/3950868452 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56247808dee0 0x5624780902d0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6412eab49bcac56 server_cookie=d47e0559692dc2d3 in_seq=0 out_seq=0 2024-03-21T11:34:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.929+0000 7ffabbe00640 10 client.29359 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.930+0000 7ffabbe00640 1 -- 172.21.15.47:0/3950868452 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562477feb3f0 con 0x56247808dee0 2024-03-21T11:34:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.930+0000 7ffabbe00640 10 client.29359 handle_command_reply: tid=0 2024-03-21T11:34:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.930+0000 7ffadff87740 1 client.29359 shutdown 2024-03-21T11:34:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.930+0000 7ffadff87740 1 -- 172.21.15.47:0/3950868452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624780782b0 msgr2=0x562478082470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.931+0000 7ffadff87740 1 --2- 172.21.15.47:0/3950868452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624780782b0 0x562478082470 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ffac80835d0 tx=0x7ffac80cfd50 comp rx=0 tx=0).stop 2024-03-21T11:34:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.931+0000 7ffadff87740 1 -- 172.21.15.47:0/3950868452 shutdown_connections 2024-03-21T11:34:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.931+0000 7ffadff87740 1 --2- 172.21.15.47:0/3950868452 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56247808dee0 0x5624780902d0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.931+0000 7ffadff87740 1 --2- 172.21.15.47:0/3950868452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 0x5624780899f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.931+0000 7ffadff87740 1 --2- 172.21.15.47:0/3950868452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562478078ca0 0x5624780829b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.931+0000 7ffabbe00640 0 client.29359 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.931+0000 7ffadff87740 1 --2- 172.21.15.47:0/3950868452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624780782b0 0x562478082470 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.931+0000 7ffadff87740 1 -- 172.21.15.47:0/3950868452 >> 172.21.15.47:0/3950868452 conn(0x562478073820 msgr2=0x562478072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.931+0000 7ffadff87740 1 -- 172.21.15.47:0/3950868452 shutdown_connections 2024-03-21T11:34:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.931+0000 7ffadff87740 1 -- 172.21.15.47:0/3950868452 wait complete. 2024-03-21T11:34:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.931+0000 7ffadff87740 20 client.29359 trim_cache size 0 max 16384 2024-03-21T11:34:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.949+0000 7ffadff87740 1 Processor -- start 2024-03-21T11:34:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.950+0000 7ffadff87740 1 -- start start 2024-03-21T11:34:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.950+0000 7ffadff87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624780782b0 0x5624780770f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.950+0000 7ffadff87740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562478078ca0 0x5624780899b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.950+0000 7ffadff87740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 0x562478094920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.950+0000 7ffadff87740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81ac170 con 0x56247807de10 2024-03-21T11:34:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.950+0000 7ffadff87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81adaf0 con 0x5624780782b0 2024-03-21T11:34:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.950+0000 7ffadff87740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81ac7d0 con 0x562478078ca0 2024-03-21T11:34:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.950+0000 7ffad5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624780782b0 0x5624780770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.950+0000 7ffad5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624780782b0 0x5624780770f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59034/0 (socket says 172.21.15.47:59034) 2024-03-21T11:34:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.950+0000 7ffad5e00640 1 -- 172.21.15.47:0/3294169079 learned_addr learned my addr 172.21.15.47:0/3294169079 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.950+0000 7ffacd400640 1 --2- 172.21.15.47:0/3294169079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562478078ca0 0x5624780899b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.950+0000 7ffad6800640 1 --2- 172.21.15.47:0/3294169079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 0x562478094920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.951+0000 7ffacd400640 1 -- 172.21.15.47:0/3294169079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624780782b0 msgr2=0x5624780770f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.951+0000 7ffacd400640 1 --2- 172.21.15.47:0/3294169079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624780782b0 0x5624780770f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.951+0000 7ffacd400640 1 -- 172.21.15.47:0/3294169079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 msgr2=0x562478094920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.951+0000 7ffacd400640 1 --2- 172.21.15.47:0/3294169079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 0x562478094920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.951+0000 7ffacd400640 1 -- 172.21.15.47:0/3294169079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56247807a720 con 0x562478078ca0 2024-03-21T11:34:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.951+0000 7ffad5e00640 1 --2- 172.21.15.47:0/3294169079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624780782b0 0x5624780770f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:34:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.951+0000 7ffacd400640 1 --2- 172.21.15.47:0/3294169079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562478078ca0 0x5624780899b0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009230 tx=0x7ffac002df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9bb825723ea4618 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.951+0000 7ffac6a00640 1 -- 172.21.15.47:0/3294169079 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac004e070 con 0x562478078ca0 2024-03-21T11:34:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.951+0000 7ffac6a00640 1 -- 172.21.15.47:0/3294169079 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffac002cb20 con 0x562478078ca0 2024-03-21T11:34:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.951+0000 7ffac6a00640 1 -- 172.21.15.47:0/3294169079 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0005570 con 0x562478078ca0 2024-03-21T11:34:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.952+0000 7ffadff87740 1 -- 172.21.15.47:0/3294169079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562478078ca0 msgr2=0x5624780899b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.952+0000 7ffadff87740 1 --2- 172.21.15.47:0/3294169079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562478078ca0 0x5624780899b0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009230 tx=0x7ffac002df90 comp rx=0 tx=0).stop 2024-03-21T11:34:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.952+0000 7ffadff87740 1 -- 172.21.15.47:0/3294169079 shutdown_connections 2024-03-21T11:34:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.952+0000 7ffadff87740 1 --2- 172.21.15.47:0/3294169079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 0x562478094920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.952+0000 7ffadff87740 1 --2- 172.21.15.47:0/3294169079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562478078ca0 0x5624780899b0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.952+0000 7ffadff87740 1 --2- 172.21.15.47:0/3294169079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624780782b0 0x5624780770f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.952+0000 7ffadff87740 1 -- 172.21.15.47:0/3294169079 >> 172.21.15.47:0/3294169079 conn(0x562478073820 msgr2=0x562478072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.952+0000 7ffadff87740 1 -- 172.21.15.47:0/3294169079 shutdown_connections 2024-03-21T11:34:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.952+0000 7ffadff87740 1 -- 172.21.15.47:0/3294169079 wait complete. 2024-03-21T11:34:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.953+0000 7ffadff87740 1 -- start start 2024-03-21T11:34:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.953+0000 7ffadff87740 1 Processor -- start 2024-03-21T11:34:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.953+0000 7ffadff87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624780782b0 0x562478088230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.953+0000 7ffadff87740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562478078ca0 0x562478088770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.953+0000 7ffadff87740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 0x562478088cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.953+0000 7ffadff87740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81ad490 con 0x56247807de10 2024-03-21T11:34:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.953+0000 7ffadff87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81ace30 con 0x5624780782b0 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.953+0000 7ffadff87740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81aa330 con 0x562478078ca0 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.953+0000 7ffadff87740 10 client.0 _ll_register_callbacks cb 0x7ffac80e1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.954+0000 7ffad5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624780782b0 0x562478088230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.954+0000 7ffacd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562478078ca0 0x562478088770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.954+0000 7ffad5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624780782b0 0x562478088230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59036/0 (socket says 172.21.15.47:59036) 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.954+0000 7ffad5e00640 1 -- 172.21.15.47:0/1988673762 learned_addr learned my addr 172.21.15.47:0/1988673762 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.954+0000 7ffad6800640 1 --2- 172.21.15.47:0/1988673762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 0x562478088cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.954+0000 7ffad5e00640 1 -- 172.21.15.47:0/1988673762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562478078ca0 msgr2=0x562478088770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.954+0000 7ffad5e00640 1 --2- 172.21.15.47:0/1988673762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562478078ca0 0x562478088770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.954+0000 7ffad5e00640 1 -- 172.21.15.47:0/1988673762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 msgr2=0x562478088cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.954+0000 7ffad5e00640 1 --2- 172.21.15.47:0/1988673762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 0x562478088cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.954+0000 7ffad5e00640 1 -- 172.21.15.47:0/1988673762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624780e95c0 con 0x5624780782b0 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.954+0000 7ffad6800640 1 --2- 172.21.15.47:0/1988673762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 0x562478088cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.954+0000 7ffad5e00640 1 --2- 172.21.15.47:0/1988673762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624780782b0 0x562478088230 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffac800c6a0 tx=0x7ffac80987e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55bf9d10d6c47878 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.954+0000 7ffabbe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.954+0000 7ffabbe00640 1 -- 172.21.15.47:0/1988673762 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8083600 con 0x5624780782b0 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.955+0000 7ffadff87740 1 -- 172.21.15.47:0/1988673762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffac002cb20 con 0x5624780782b0 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.955+0000 7ffabbe00640 1 -- 172.21.15.47:0/1988673762 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffac80a4590 con 0x5624780782b0 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.955+0000 7ffabbe00640 1 -- 172.21.15.47:0/1988673762 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffac8082360 con 0x5624780782b0 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.955+0000 7ffadff87740 10 client.29342 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.955+0000 7ffadff87740 1 -- 172.21.15.47:0/1988673762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5624780ea900 con 0x5624780782b0 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.955+0000 7ffabbe00640 1 -- 172.21.15.47:0/1988673762 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffac002cb20 con 0x5624780782b0 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.955+0000 7ffabbe00640 1 -- 172.21.15.47:0/1988673762 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffac809c120 con 0x5624780782b0 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.955+0000 7ffadff87740 10 client.29342 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.955+0000 7ffadff87740 10 client.29342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.955+0000 7ffadff87740 20 client.29342 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.955+0000 7ffadff87740 1 --2- 172.21.15.47:0/1988673762 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5624780eafe0 0x5624780ed400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.955+0000 7ffadff87740 4 client.29342 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.955+0000 7ffadff87740 1 -- 172.21.15.47:0/1988673762 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562477fe5740 con 0x5624780eafe0 2024-03-21T11:34:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.956+0000 7ffacd400640 1 --2- 172.21.15.47:0/1988673762 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5624780eafe0 0x5624780ed400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.956+0000 7ffacd400640 1 --2- 172.21.15.47:0/1988673762 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5624780eafe0 0x5624780ed400 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f36ffdc92d4102f1 server_cookie=718afa3c703b07b3 in_seq=0 out_seq=0 2024-03-21T11:34:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.956+0000 7ffabbe00640 10 client.29342 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.957+0000 7ffabbe00640 1 -- 172.21.15.47:0/1988673762 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562477fe5740 con 0x5624780eafe0 2024-03-21T11:34:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.957+0000 7ffabbe00640 10 client.29342 handle_command_reply: tid=0 2024-03-21T11:34:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.957+0000 7ffadff87740 1 client.29342 shutdown 2024-03-21T11:34:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.957+0000 7ffadff87740 1 -- 172.21.15.47:0/1988673762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624780782b0 msgr2=0x562478088230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.957+0000 7ffadff87740 1 --2- 172.21.15.47:0/1988673762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624780782b0 0x562478088230 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffac800c6a0 tx=0x7ffac80987e0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.957+0000 7ffadff87740 1 -- 172.21.15.47:0/1988673762 shutdown_connections 2024-03-21T11:34:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.957+0000 7ffadff87740 1 --2- 172.21.15.47:0/1988673762 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5624780eafe0 0x5624780ed400 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.957+0000 7ffadff87740 1 --2- 172.21.15.47:0/1988673762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56247807de10 0x562478088cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.957+0000 7ffadff87740 1 --2- 172.21.15.47:0/1988673762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562478078ca0 0x562478088770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.957+0000 7ffadff87740 1 --2- 172.21.15.47:0/1988673762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624780782b0 0x562478088230 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.957+0000 7ffadff87740 1 -- 172.21.15.47:0/1988673762 >> 172.21.15.47:0/1988673762 conn(0x562478073820 msgr2=0x562478079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.957+0000 7ffabbe00640 0 client.29342 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.958+0000 7ffadff87740 1 -- 172.21.15.47:0/1988673762 shutdown_connections 2024-03-21T11:34:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.958+0000 7ffadff87740 1 -- 172.21.15.47:0/1988673762 wait complete. 2024-03-21T11:34:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.958+0000 7ffadff87740 20 client.29342 trim_cache size 0 max 16384 2024-03-21T11:34:37.963 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:34:37.964 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:34:37.964 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "a59322b9-9d2e-46af-a3d6-2be3cea9300c", 2024-03-21T11:34:37.964 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:34:37.964 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:34:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.958+0000 7ffad7200640 1 -- 172.21.15.47:0/2769482247 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffabc0a2f20 msgr2=0x7ffabc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.958+0000 7ffad7200640 1 --2- 172.21.15.47:0/2769482247 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffabc0a2f20 0x7ffabc0a53e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x562478079500 tx=0x7ffad0009500 comp rx=0 tx=0).stop 2024-03-21T11:34:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.958+0000 7ffad7200640 1 -- 172.21.15.47:0/2769482247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8102000 msgr2=0x7ffad8199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.958+0000 7ffad7200640 1 --2- 172.21.15.47:0/2769482247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8102000 0x7ffad8199ad0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ffac800e9f0 tx=0x7ffac800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.959+0000 7ffad7200640 1 -- 172.21.15.47:0/2769482247 shutdown_connections 2024-03-21T11:34:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.959+0000 7ffad7200640 1 --2- 172.21.15.47:0/2769482247 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffabc0a2f20 0x7ffabc0a53e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.959+0000 7ffad7200640 1 --2- 172.21.15.47:0/2769482247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffad8107260 0x7ffad81a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.959+0000 7ffad7200640 1 --2- 172.21.15.47:0/2769482247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffad8104930 0x7ffad819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.959+0000 7ffad7200640 1 --2- 172.21.15.47:0/2769482247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8102000 0x7ffad8199ad0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.959+0000 7ffad7200640 1 -- 172.21.15.47:0/2769482247 >> 172.21.15.47:0/2769482247 conn(0x7ffad80fba10 msgr2=0x7ffad81017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:37.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.959+0000 7ffad7200640 1 -- 172.21.15.47:0/2769482247 shutdown_connections 2024-03-21T11:34:37.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:37.959+0000 7ffad7200640 1 -- 172.21.15.47:0/2769482247 wait complete. 2024-03-21T11:34:37.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:34:37.980 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:34:38.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.194+0000 7ffb56600640 1 -- 172.21.15.47:0/3198205384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb501038e0 msgr2=0x7ffb50103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:38.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.194+0000 7ffb56600640 1 --2- 172.21.15.47:0/3198205384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb501038e0 0x7ffb50103d00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009860 tx=0x7ffb4002bb70 comp rx=0 tx=0).stop 2024-03-21T11:34:38.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.194+0000 7ffb56600640 1 -- 172.21.15.47:0/3198205384 shutdown_connections 2024-03-21T11:34:38.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.194+0000 7ffb56600640 1 --2- 172.21.15.47:0/3198205384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb50105490 0x7ffb5010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.194+0000 7ffb56600640 1 --2- 172.21.15.47:0/3198205384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb50104ad0 0x7ffb50104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.194+0000 7ffb56600640 1 --2- 172.21.15.47:0/3198205384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb501038e0 0x7ffb50103d00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.194+0000 7ffb56600640 1 -- 172.21.15.47:0/3198205384 >> 172.21.15.47:0/3198205384 conn(0x7ffb500fee50 msgr2=0x7ffb50101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:38.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.194+0000 7ffb56600640 1 -- 172.21.15.47:0/3198205384 shutdown_connections 2024-03-21T11:34:38.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.194+0000 7ffb56600640 1 -- 172.21.15.47:0/3198205384 wait complete. 2024-03-21T11:34:38.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.195+0000 7ffb56600640 1 Processor -- start 2024-03-21T11:34:38.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.195+0000 7ffb56600640 1 -- start start 2024-03-21T11:34:38.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb56600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb501038e0 0x7ffb50199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb56600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb50104ad0 0x7ffb5019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb56600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb50105490 0x7ffb501a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb56600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5010f530 con 0x7ffb50104ad0 2024-03-21T11:34:38.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb56600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5010f3b0 con 0x7ffb50105490 2024-03-21T11:34:38.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb56600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5010f6b0 con 0x7ffb501038e0 2024-03-21T11:34:38.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb4f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb501038e0 0x7ffb50199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb4f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb501038e0 0x7ffb50199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59154/0 (socket says 172.21.15.47:59154) 2024-03-21T11:34:38.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb4f400640 1 -- 172.21.15.47:0/3437667511 learned_addr learned my addr 172.21.15.47:0/3437667511 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:38.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb4fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb50105490 0x7ffb501a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb46a00640 1 --2- 172.21.15.47:0/3437667511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb50104ad0 0x7ffb5019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb4f400640 1 -- 172.21.15.47:0/3437667511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb50105490 msgr2=0x7ffb501a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:38.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb4f400640 1 --2- 172.21.15.47:0/3437667511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb50105490 0x7ffb501a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb4f400640 1 -- 172.21.15.47:0/3437667511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb50104ad0 msgr2=0x7ffb5019a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:38.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb4f400640 1 --2- 172.21.15.47:0/3437667511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb50104ad0 0x7ffb5019a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.196+0000 7ffb4f400640 1 -- 172.21.15.47:0/3437667511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb501a1830 con 0x7ffb501038e0 2024-03-21T11:34:38.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.197+0000 7ffb4f400640 1 --2- 172.21.15.47:0/3437667511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb501038e0 0x7ffb50199c20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ffb400349e0 tx=0x7ffb40032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:38.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.197+0000 7ffb4cc00640 1 -- 172.21.15.47:0/3437667511 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb40041020 con 0x7ffb501038e0 2024-03-21T11:34:38.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.197+0000 7ffb4cc00640 1 -- 172.21.15.47:0/3437667511 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb4002c9f0 con 0x7ffb501038e0 2024-03-21T11:34:38.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.197+0000 7ffb56600640 1 -- 172.21.15.47:0/3437667511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb501a1ac0 con 0x7ffb501038e0 2024-03-21T11:34:38.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.197+0000 7ffb4cc00640 1 -- 172.21.15.47:0/3437667511 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb40039040 con 0x7ffb501038e0 2024-03-21T11:34:38.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.197+0000 7ffb56600640 1 -- 172.21.15.47:0/3437667511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb501a2000 con 0x7ffb501038e0 2024-03-21T11:34:38.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.201+0000 7ffb59239740 1 Processor -- start 2024-03-21T11:34:38.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.201+0000 7ffb59239740 1 -- start start 2024-03-21T11:34:38.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.202+0000 7ffb4cc00640 1 -- 172.21.15.47:0/3437667511 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ffb40044020 con 0x7ffb501038e0 2024-03-21T11:34:38.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.202+0000 7ffb59239740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a3e80 0x557f4f2a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.202+0000 7ffb59239740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a4870 0x557f4f2a9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.202+0000 7ffb59239740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a99e0 0x557f4f2abe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.202+0000 7ffb59239740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f209250 con 0x557f4f2a99e0 2024-03-21T11:34:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.202+0000 7ffb59239740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f1b1a30 con 0x557f4f2a3e80 2024-03-21T11:34:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.202+0000 7ffb59239740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f2052f0 con 0x557f4f2a4870 2024-03-21T11:34:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.202+0000 7ffb4fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a3e80 0x557f4f2a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.202+0000 7ffb4fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a3e80 0x557f4f2a42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59052/0 (socket says 172.21.15.47:59052) 2024-03-21T11:34:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.202+0000 7ffb4fe00640 1 -- 172.21.15.47:0/681920457 learned_addr learned my addr 172.21.15.47:0/681920457 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.202+0000 7ffb4f400640 1 --2- 172.21.15.47:0/681920457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a4870 0x557f4f2a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.202+0000 7ffb46a00640 1 --2- 172.21.15.47:0/681920457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a99e0 0x557f4f2abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.202+0000 7ffb4fe00640 1 -- 172.21.15.47:0/681920457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a4870 msgr2=0x557f4f2a9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:38.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.202+0000 7ffb4fe00640 1 --2- 172.21.15.47:0/681920457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a4870 0x557f4f2a9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.203+0000 7ffb4fe00640 1 -- 172.21.15.47:0/681920457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a99e0 msgr2=0x557f4f2abe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:38.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.203+0000 7ffb4fe00640 1 --2- 172.21.15.47:0/681920457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a99e0 0x557f4f2abe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.203+0000 7ffb4fe00640 1 -- 172.21.15.47:0/681920457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f4f19c690 con 0x557f4f2a3e80 2024-03-21T11:34:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.203+0000 7ffb4fe00640 1 --2- 172.21.15.47:0/681920457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a3e80 0x557f4f2a42a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009fd0 tx=0x7ffb480118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8724ee01f67a28bf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.203+0000 7ffb44c00640 1 -- 172.21.15.47:0/681920457 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48004d20 con 0x557f4f2a3e80 2024-03-21T11:34:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.203+0000 7ffb44c00640 1 -- 172.21.15.47:0/681920457 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb48004050 con 0x557f4f2a3e80 2024-03-21T11:34:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.203+0000 7ffb44c00640 1 -- 172.21.15.47:0/681920457 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48005220 con 0x557f4f2a3e80 2024-03-21T11:34:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.204+0000 7ffb4cc00640 1 --2- 172.21.15.47:0/3437667511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb340a2fa0 0x7ffb340a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.204+0000 7ffb59239740 1 -- 172.21.15.47:0/681920457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a3e80 msgr2=0x557f4f2a42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.204+0000 7ffb59239740 1 --2- 172.21.15.47:0/681920457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a3e80 0x557f4f2a42a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009fd0 tx=0x7ffb480118d0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.204+0000 7ffb59239740 1 -- 172.21.15.47:0/681920457 shutdown_connections 2024-03-21T11:34:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.204+0000 7ffb59239740 1 --2- 172.21.15.47:0/681920457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a99e0 0x557f4f2abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.204+0000 7ffb59239740 1 --2- 172.21.15.47:0/681920457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a4870 0x557f4f2a9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.204+0000 7ffb59239740 1 --2- 172.21.15.47:0/681920457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a3e80 0x557f4f2a42a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.204+0000 7ffb59239740 1 -- 172.21.15.47:0/681920457 >> 172.21.15.47:0/681920457 conn(0x557f4f26c200 msgr2=0x557f4f29f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.204+0000 7ffb4cc00640 1 -- 172.21.15.47:0/3437667511 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffb400f0fe0 con 0x7ffb501038e0 2024-03-21T11:34:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.204+0000 7ffb4f400640 1 --2- 172.21.15.47:0/3437667511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb340a2fa0 0x7ffb340a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.204+0000 7ffb59239740 1 -- 172.21.15.47:0/681920457 shutdown_connections 2024-03-21T11:34:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.204+0000 7ffb59239740 1 -- 172.21.15.47:0/681920457 wait complete. 2024-03-21T11:34:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.205+0000 7ffb59239740 1 -- start start 2024-03-21T11:34:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.205+0000 7ffb4f400640 1 --2- 172.21.15.47:0/3437667511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb340a2fa0 0x7ffb340a5460 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffb4002c180 tx=0x7ffb400f4a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.205+0000 7ffb59239740 1 Processor -- start 2024-03-21T11:34:38.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.205+0000 7ffb59239740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a3e80 0x557f4f2addf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.205+0000 7ffb59239740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a4870 0x557f4f2ae330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.205+0000 7ffb59239740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a99e0 0x557f4f2b5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.205+0000 7ffb59239740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f2ac990 con 0x557f4f2a99e0 2024-03-21T11:34:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.205+0000 7ffb59239740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f1c4e80 con 0x557f4f2a4870 2024-03-21T11:34:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.205+0000 7ffb59239740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4f2acb10 con 0x557f4f2a3e80 2024-03-21T11:34:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.205+0000 7ffb59239740 10 client.0 _ll_register_callbacks cb 0x557f4f1d7130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.206+0000 7ffb46a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a4870 0x557f4f2ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.206+0000 7ffb46a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a4870 0x557f4f2ae330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59054/0 (socket says 172.21.15.47:59054) 2024-03-21T11:34:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.206+0000 7ffb46a00640 1 -- 172.21.15.47:0/3865470810 learned_addr learned my addr 172.21.15.47:0/3865470810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.206+0000 7ffb4fe00640 1 --2- 172.21.15.47:0/3865470810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a3e80 0x557f4f2addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.206+0000 7ffb4fe00640 1 --2- 172.21.15.47:0/3865470810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a99e0 0x557f4f2b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.206+0000 7ffb46a00640 1 -- 172.21.15.47:0/3865470810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a3e80 msgr2=0x557f4f2addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.206+0000 7ffb46a00640 1 --2- 172.21.15.47:0/3865470810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a3e80 0x557f4f2addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.206+0000 7ffb46a00640 1 -- 172.21.15.47:0/3865470810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a99e0 msgr2=0x557f4f2b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:38.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.206+0000 7ffb46a00640 1 --2- 172.21.15.47:0/3865470810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a99e0 0x557f4f2b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.206+0000 7ffb46a00640 1 -- 172.21.15.47:0/3865470810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f4efe77d0 con 0x557f4f2a4870 2024-03-21T11:34:38.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.206+0000 7ffb4fe00640 1 --2- 172.21.15.47:0/3865470810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a99e0 0x557f4f2b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:38.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.206+0000 7ffb46a00640 1 --2- 172.21.15.47:0/3865470810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a4870 0x557f4f2ae330 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ffb38011710 tx=0x7ffb38011bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bde7eace925906f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:38.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.206+0000 7ffb3d600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:38.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.206+0000 7ffb3d600640 1 -- 172.21.15.47:0/3865470810 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3801e060 con 0x557f4f2a4870 2024-03-21T11:34:38.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.207+0000 7ffb3d600640 1 -- 172.21.15.47:0/3865470810 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffb3800fb30 con 0x557f4f2a4870 2024-03-21T11:34:38.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.207+0000 7ffb59239740 1 -- 172.21.15.47:0/3865470810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb48004050 con 0x557f4f2a4870 2024-03-21T11:34:38.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.207+0000 7ffb3d600640 1 -- 172.21.15.47:0/3865470810 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb38025050 con 0x557f4f2a4870 2024-03-21T11:34:38.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.207+0000 7ffb59239740 10 client.29354 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:38.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.207+0000 7ffb59239740 1 -- 172.21.15.47:0/3865470810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557f4ef1d3f0 con 0x557f4f2a4870 2024-03-21T11:34:38.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.207+0000 7ffb3d600640 1 -- 172.21.15.47:0/3865470810 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb48004050 con 0x557f4f2a4870 2024-03-21T11:34:38.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.207+0000 7ffb3d600640 1 -- 172.21.15.47:0/3865470810 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffb38019070 con 0x557f4f2a4870 2024-03-21T11:34:38.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.208+0000 7ffb59239740 10 client.29354 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:38.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.208+0000 7ffb59239740 10 client.29354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:38.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.208+0000 7ffb59239740 20 client.29354 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:38.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.208+0000 7ffb59239740 1 --2- 172.21.15.47:0/3865470810 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557f4f2b9d00 0x557f4f2bc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.208+0000 7ffb59239740 4 client.29354 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:38.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.208+0000 7ffb59239740 1 -- 172.21.15.47:0/3865470810 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f4f216790 con 0x557f4f2b9d00 2024-03-21T11:34:38.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.208+0000 7ffb4fe00640 1 --2- 172.21.15.47:0/3865470810 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557f4f2b9d00 0x557f4f2bc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.208+0000 7ffb4fe00640 1 --2- 172.21.15.47:0/3865470810 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557f4f2b9d00 0x557f4f2bc0f0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f4699ad6b441b5 server_cookie=77c49d183b3ab5ae in_seq=0 out_seq=0 2024-03-21T11:34:38.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.208+0000 7ffb3d600640 10 client.29354 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:38.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb3d600640 1 -- 172.21.15.47:0/3865470810 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557f4f216790 con 0x557f4f2b9d00 2024-03-21T11:34:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb3d600640 10 client.29354 handle_command_reply: tid=0 2024-03-21T11:34:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb59239740 1 client.29354 shutdown 2024-03-21T11:34:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb59239740 1 -- 172.21.15.47:0/3865470810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a4870 msgr2=0x557f4f2ae330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb59239740 1 --2- 172.21.15.47:0/3865470810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a4870 0x557f4f2ae330 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ffb38011710 tx=0x7ffb38011bd0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb59239740 1 -- 172.21.15.47:0/3865470810 shutdown_connections 2024-03-21T11:34:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb59239740 1 --2- 172.21.15.47:0/3865470810 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557f4f2b9d00 0x557f4f2bc0f0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb59239740 1 --2- 172.21.15.47:0/3865470810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a99e0 0x557f4f2b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb59239740 1 --2- 172.21.15.47:0/3865470810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a4870 0x557f4f2ae330 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb59239740 1 --2- 172.21.15.47:0/3865470810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a3e80 0x557f4f2addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb59239740 1 -- 172.21.15.47:0/3865470810 >> 172.21.15.47:0/3865470810 conn(0x557f4f26c200 msgr2=0x557f4f29ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb3d600640 0 client.29354 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb59239740 1 -- 172.21.15.47:0/3865470810 shutdown_connections 2024-03-21T11:34:38.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb59239740 1 -- 172.21.15.47:0/3865470810 wait complete. 2024-03-21T11:34:38.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.210+0000 7ffb59239740 20 client.29354 trim_cache size 0 max 16384 2024-03-21T11:34:38.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.228+0000 7ffb59239740 1 Processor -- start 2024-03-21T11:34:38.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.228+0000 7ffb59239740 1 -- start start 2024-03-21T11:34:38.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.228+0000 7ffb59239740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f26c200 0x557f4f2a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.228+0000 7ffb59239740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a3e80 0x557f4f2b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.228+0000 7ffb59239740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a4870 0x557f4f2a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.228+0000 7ffb59239740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501ac2b0 con 0x557f4f26c200 2024-03-21T11:34:38.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.228+0000 7ffb59239740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501adc30 con 0x557f4f2a3e80 2024-03-21T11:34:38.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.228+0000 7ffb59239740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501ac910 con 0x557f4f2a4870 2024-03-21T11:34:38.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.228+0000 7ffb4fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f26c200 0x557f4f2a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.229+0000 7ffb46a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a3e80 0x557f4f2b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.229+0000 7ffb46a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a3e80 0x557f4f2b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59064/0 (socket says 172.21.15.47:59064) 2024-03-21T11:34:38.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.229+0000 7ffb46a00640 1 -- 172.21.15.47:0/2454097822 learned_addr learned my addr 172.21.15.47:0/2454097822 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:38.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.229+0000 7ffb4fe00640 1 --2- 172.21.15.47:0/2454097822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a4870 0x557f4f2a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.229+0000 7ffb46a00640 1 -- 172.21.15.47:0/2454097822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a4870 msgr2=0x557f4f2a2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:38.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.229+0000 7ffb46a00640 1 --2- 172.21.15.47:0/2454097822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a4870 0x557f4f2a2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.229+0000 7ffb46a00640 1 -- 172.21.15.47:0/2454097822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f26c200 msgr2=0x557f4f2a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:38.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.229+0000 7ffb46a00640 1 --2- 172.21.15.47:0/2454097822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f26c200 0x557f4f2a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.229+0000 7ffb46a00640 1 -- 172.21.15.47:0/2454097822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f4eedd0f0 con 0x557f4f2a3e80 2024-03-21T11:34:38.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.229+0000 7ffb46a00640 1 --2- 172.21.15.47:0/2454097822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a3e80 0x557f4f2b5330 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009ec0 tx=0x7ffb38000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5f94b6c0babcf40 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:38.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.230+0000 7ffb3fe00640 1 -- 172.21.15.47:0/2454097822 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3801e040 con 0x557f4f2a3e80 2024-03-21T11:34:38.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.230+0000 7ffb3fe00640 1 -- 172.21.15.47:0/2454097822 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb38003fb0 con 0x557f4f2a3e80 2024-03-21T11:34:38.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.230+0000 7ffb3fe00640 1 -- 172.21.15.47:0/2454097822 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38000e60 con 0x557f4f2a3e80 2024-03-21T11:34:38.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.230+0000 7ffb59239740 1 -- 172.21.15.47:0/2454097822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a3e80 msgr2=0x557f4f2b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:38.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.230+0000 7ffb59239740 1 --2- 172.21.15.47:0/2454097822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a3e80 0x557f4f2b5330 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009ec0 tx=0x7ffb38000bc0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.230+0000 7ffb59239740 1 -- 172.21.15.47:0/2454097822 shutdown_connections 2024-03-21T11:34:38.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.230+0000 7ffb59239740 1 --2- 172.21.15.47:0/2454097822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a4870 0x557f4f2a2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.230+0000 7ffb59239740 1 --2- 172.21.15.47:0/2454097822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f2a3e80 0x557f4f2b5330 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.230+0000 7ffb59239740 1 --2- 172.21.15.47:0/2454097822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f26c200 0x557f4f2a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.230+0000 7ffb59239740 1 -- 172.21.15.47:0/2454097822 >> 172.21.15.47:0/2454097822 conn(0x557f4f2ad9b0 msgr2=0x557f4f2a5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:38.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.230+0000 7ffb59239740 1 -- 172.21.15.47:0/2454097822 shutdown_connections 2024-03-21T11:34:38.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.230+0000 7ffb59239740 1 -- 172.21.15.47:0/2454097822 wait complete. 2024-03-21T11:34:38.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.231+0000 7ffb59239740 1 -- start start 2024-03-21T11:34:38.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.231+0000 7ffb59239740 1 Processor -- start 2024-03-21T11:34:38.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.231+0000 7ffb59239740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f26c200 0x557f4f2b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.231+0000 7ffb59239740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a3e80 0x557f4f2b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.231+0000 7ffb59239740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a4870 0x557f4f2b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.231+0000 7ffb59239740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501ad5d0 con 0x557f4f2a4870 2024-03-21T11:34:38.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.231+0000 7ffb59239740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501acf70 con 0x557f4f26c200 2024-03-21T11:34:38.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.231+0000 7ffb59239740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501aa470 con 0x557f4f2a3e80 2024-03-21T11:34:38.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.232+0000 7ffb59239740 10 client.0 _ll_register_callbacks cb 0x7ffb38009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:38.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.232+0000 7ffb46a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a3e80 0x557f4f2b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.232+0000 7ffb46a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a3e80 0x557f4f2b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59200/0 (socket says 172.21.15.47:59200) 2024-03-21T11:34:38.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.232+0000 7ffb46a00640 1 -- 172.21.15.47:0/4202211832 learned_addr learned my addr 172.21.15.47:0/4202211832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:38.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.232+0000 7ffb4fe00640 1 --2- 172.21.15.47:0/4202211832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f26c200 0x557f4f2b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.232+0000 7ffb4fe00640 1 --2- 172.21.15.47:0/4202211832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a4870 0x557f4f2b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.232+0000 7ffb46a00640 1 -- 172.21.15.47:0/4202211832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f26c200 msgr2=0x557f4f2b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:38.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.232+0000 7ffb46a00640 1 --2- 172.21.15.47:0/4202211832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f26c200 0x557f4f2b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.232+0000 7ffb46a00640 1 -- 172.21.15.47:0/4202211832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a4870 msgr2=0x557f4f2b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:38.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.232+0000 7ffb46a00640 1 --2- 172.21.15.47:0/4202211832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a4870 0x557f4f2b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.232+0000 7ffb46a00640 1 -- 172.21.15.47:0/4202211832 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f4f2b4da0 con 0x557f4f2a3e80 2024-03-21T11:34:38.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.232+0000 7ffb4fe00640 1 --2- 172.21.15.47:0/4202211832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f26c200 0x557f4f2b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:38.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.232+0000 7ffb46a00640 1 --2- 172.21.15.47:0/4202211832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a3e80 0x557f4f2b37a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009530 tx=0x7ffb38003ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=560570ee265dcda1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:38.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.232+0000 7ffb3d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:38.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.233+0000 7ffb3d600640 1 -- 172.21.15.47:0/4202211832 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38036070 con 0x557f4f2a3e80 2024-03-21T11:34:38.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.233+0000 7ffb59239740 1 -- 172.21.15.47:0/4202211832 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb38003fb0 con 0x557f4f2a3e80 2024-03-21T11:34:38.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.233+0000 7ffb3d600640 1 -- 172.21.15.47:0/4202211832 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffb38004370 con 0x557f4f2a3e80 2024-03-21T11:34:38.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.233+0000 7ffb3d600640 1 -- 172.21.15.47:0/4202211832 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb3801e030 con 0x557f4f2a3e80 2024-03-21T11:34:38.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.233+0000 7ffb59239740 10 client.29383 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:38.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.233+0000 7ffb59239740 1 -- 172.21.15.47:0/4202211832 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557f4f315d20 con 0x557f4f2a3e80 2024-03-21T11:34:38.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.233+0000 7ffb3d600640 1 -- 172.21.15.47:0/4202211832 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb38003fb0 con 0x557f4f2a3e80 2024-03-21T11:34:38.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.234+0000 7ffb3d600640 1 -- 172.21.15.47:0/4202211832 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffb38019040 con 0x557f4f2a3e80 2024-03-21T11:34:38.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.234+0000 7ffb59239740 10 client.29383 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:38.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.234+0000 7ffb59239740 10 client.29383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:38.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.234+0000 7ffb59239740 20 client.29383 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.234+0000 7ffb59239740 1 --2- 172.21.15.47:0/4202211832 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557f4f2a99e0 0x557f4f318710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.234+0000 7ffb59239740 4 client.29383 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.234+0000 7ffb59239740 1 -- 172.21.15.47:0/4202211832 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557f4f210ae0 con 0x557f4f2a99e0 2024-03-21T11:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.234+0000 7ffb4fe00640 1 --2- 172.21.15.47:0/4202211832 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557f4f2a99e0 0x557f4f318710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.234+0000 7ffb4fe00640 1 --2- 172.21.15.47:0/4202211832 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557f4f2a99e0 0x557f4f318710 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb36ec1498b27ed5 server_cookie=a803a2866d3a87e9 in_seq=0 out_seq=0 2024-03-21T11:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.234+0000 7ffb3d600640 10 client.29383 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb3d600640 1 -- 172.21.15.47:0/4202211832 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557f4f210ae0 con 0x557f4f2a99e0 2024-03-21T11:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb3d600640 10 client.29383 handle_command_reply: tid=0 2024-03-21T11:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb59239740 1 client.29383 shutdown 2024-03-21T11:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb59239740 1 -- 172.21.15.47:0/4202211832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a3e80 msgr2=0x557f4f2b37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb59239740 1 --2- 172.21.15.47:0/4202211832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a3e80 0x557f4f2b37a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009530 tx=0x7ffb38003ad0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb59239740 1 -- 172.21.15.47:0/4202211832 shutdown_connections 2024-03-21T11:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb59239740 1 --2- 172.21.15.47:0/4202211832 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557f4f2a99e0 0x557f4f318710 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb59239740 1 --2- 172.21.15.47:0/4202211832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f4f2a4870 0x557f4f2b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb59239740 1 --2- 172.21.15.47:0/4202211832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f4f2a3e80 0x557f4f2b37a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb3d600640 0 client.29383 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:38.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb59239740 1 --2- 172.21.15.47:0/4202211832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f4f26c200 0x557f4f2b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb59239740 1 -- 172.21.15.47:0/4202211832 >> 172.21.15.47:0/4202211832 conn(0x557f4f2ad9b0 msgr2=0x557f4f29e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:38.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb59239740 1 -- 172.21.15.47:0/4202211832 shutdown_connections 2024-03-21T11:34:38.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb59239740 1 -- 172.21.15.47:0/4202211832 wait complete. 2024-03-21T11:34:38.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.235+0000 7ffb59239740 20 client.29383 trim_cache size 0 max 16384 2024-03-21T11:34:38.241 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:34:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:34:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:34:38.242 INFO:teuthology.orchestra.run.smithi047.stdout: "a59322b9-9d2e-46af-a3d6-2be3cea9300c": { 2024-03-21T11:34:38.242 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:34:38.242 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "a59322b9-9d2e-46af-a3d6-2be3cea9300c", 2024-03-21T11:34:38.242 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:34:38.242 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:34:38.242 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:34:38.242 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:34:38.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.236+0000 7ffb56600640 1 -- 172.21.15.47:0/3437667511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb340a2fa0 msgr2=0x7ffb340a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:38.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.236+0000 7ffb56600640 1 --2- 172.21.15.47:0/3437667511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb340a2fa0 0x7ffb340a5460 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffb4002c180 tx=0x7ffb400f4a20 comp rx=0 tx=0).stop 2024-03-21T11:34:38.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.236+0000 7ffb56600640 1 -- 172.21.15.47:0/3437667511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb501038e0 msgr2=0x7ffb50199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:38.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.236+0000 7ffb56600640 1 --2- 172.21.15.47:0/3437667511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb501038e0 0x7ffb50199c20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ffb400349e0 tx=0x7ffb40032430 comp rx=0 tx=0).stop 2024-03-21T11:34:38.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.236+0000 7ffb56600640 1 -- 172.21.15.47:0/3437667511 shutdown_connections 2024-03-21T11:34:38.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.236+0000 7ffb56600640 1 --2- 172.21.15.47:0/3437667511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb340a2fa0 0x7ffb340a5460 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.236+0000 7ffb56600640 1 --2- 172.21.15.47:0/3437667511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb50105490 0x7ffb501a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.236+0000 7ffb56600640 1 --2- 172.21.15.47:0/3437667511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb50104ad0 0x7ffb5019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.236+0000 7ffb56600640 1 --2- 172.21.15.47:0/3437667511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb501038e0 0x7ffb50199c20 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:38.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.236+0000 7ffb56600640 1 -- 172.21.15.47:0/3437667511 >> 172.21.15.47:0/3437667511 conn(0x7ffb500fee50 msgr2=0x7ffb50107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:38.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.237+0000 7ffb56600640 1 -- 172.21.15.47:0/3437667511 shutdown_connections 2024-03-21T11:34:38.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:34:38.237+0000 7ffb56600640 1 -- 172.21.15.47:0/3437667511 wait complete. 2024-03-21T11:34:38.257 INFO:tasks.cephfs.filesystem:scrub status for tag:a59322b9-9d2e-46af-a3d6-2be3cea9300c - {'path': '/', 'tag': 'a59322b9-9d2e-46af-a3d6-2be3cea9300c', 'options': 'recursive,force'} 2024-03-21T11:34:38.560 INFO:tasks.workunit.client.0.smithi047.stdout: 1 851146 8.03 MB/sec execute 442 sec latency 12.425 ms 2024-03-21T11:34:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:38 smithi047 ceph-mon[31351]: pgmap v739: 129 pgs: 129 active+clean; 39 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-03-21T11:34:38.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:38 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:34:38.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:38 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:34:38.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:38 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:34:38.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:38 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:34:38.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:38 smithi100 ceph-mon[36245]: pgmap v739: 129 pgs: 129 active+clean; 39 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-03-21T11:34:38.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:38 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:34:38.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:38 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:34:38.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:38 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:34:38.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:38 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:34:38.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:38 smithi102 ceph-mon[36748]: pgmap v739: 129 pgs: 129 active+clean; 39 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-03-21T11:34:38.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:38 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:34:38.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:38 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:34:38.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:38 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:34:38.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:38 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:34:39.560 INFO:tasks.workunit.client.0.smithi047.stdout: 1 852647 8.04 MB/sec execute 443 sec latency 14.771 ms 2024-03-21T11:34:40.560 INFO:tasks.workunit.client.0.smithi047.stdout: 1 854090 8.03 MB/sec execute 444 sec latency 21.629 ms 2024-03-21T11:34:40.833 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:40 smithi047 ceph-mon[31351]: pgmap v740: 129 pgs: 129 active+clean; 38 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-03-21T11:34:40.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:40 smithi100 ceph-mon[36245]: pgmap v740: 129 pgs: 129 active+clean; 38 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-03-21T11:34:40.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:40 smithi102 ceph-mon[36748]: pgmap v740: 129 pgs: 129 active+clean; 38 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-03-21T11:34:41.560 INFO:tasks.workunit.client.0.smithi047.stdout: 1 855776 8.04 MB/sec execute 445 sec latency 42.437 ms 2024-03-21T11:34:42.560 INFO:tasks.workunit.client.0.smithi047.stdout: 1 856597 8.03 MB/sec execute 446 sec latency 26.030 ms 2024-03-21T11:34:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:42 smithi100 ceph-mon[36245]: pgmap v741: 129 pgs: 129 active+clean; 38 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2024-03-21T11:34:42.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:42 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:34:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:42 smithi102 ceph-mon[36748]: pgmap v741: 129 pgs: 129 active+clean; 38 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2024-03-21T11:34:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:42 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:34:43.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:42 smithi047 ceph-mon[31351]: pgmap v741: 129 pgs: 129 active+clean; 38 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2024-03-21T11:34:43.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:42 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:34:43.560 INFO:tasks.workunit.client.0.smithi047.stdout: 1 858435 8.03 MB/sec execute 447 sec latency 15.922 ms 2024-03-21T11:34:44.561 INFO:tasks.workunit.client.0.smithi047.stdout: 1 859648 8.03 MB/sec execute 448 sec latency 11.666 ms 2024-03-21T11:34:44.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:44 smithi100 ceph-mon[36245]: pgmap v742: 129 pgs: 129 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 336 op/s 2024-03-21T11:34:44.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:44 smithi102 ceph-mon[36748]: pgmap v742: 129 pgs: 129 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 336 op/s 2024-03-21T11:34:45.067 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:44 smithi047 ceph-mon[31351]: pgmap v742: 129 pgs: 129 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 336 op/s 2024-03-21T11:34:45.561 INFO:tasks.workunit.client.0.smithi047.stdout: 1 860852 8.03 MB/sec execute 449 sec latency 16.884 ms 2024-03-21T11:34:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.481+0000 7fc783200640 1 -- 172.21.15.47:0/1178082278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7840ff2a0 msgr2=0x7fc7840ff6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.481+0000 7fc783200640 1 --2- 172.21.15.47:0/1178082278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7840ff2a0 0x7fc7840ff6c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc774009900 tx=0x7fc77402bc40 comp rx=0 tx=0).stop 2024-03-21T11:34:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.481+0000 7fc783200640 1 -- 172.21.15.47:0/1178082278 shutdown_connections 2024-03-21T11:34:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.481+0000 7fc783200640 1 --2- 172.21.15.47:0/1178082278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc784107190 0x7fc784109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.481+0000 7fc783200640 1 --2- 172.21.15.47:0/1178082278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7840ffc00 0x7fc784106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.481+0000 7fc783200640 1 --2- 172.21.15.47:0/1178082278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7840ff2a0 0x7fc7840ff6c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.481+0000 7fc783200640 1 -- 172.21.15.47:0/1178082278 >> 172.21.15.47:0/1178082278 conn(0x7fc7840facb0 msgr2=0x7fc7840fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.481+0000 7fc783200640 1 -- 172.21.15.47:0/1178082278 shutdown_connections 2024-03-21T11:34:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.481+0000 7fc783200640 1 -- 172.21.15.47:0/1178082278 wait complete. 2024-03-21T11:34:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.482+0000 7fc783200640 1 Processor -- start 2024-03-21T11:34:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.482+0000 7fc783200640 1 -- start start 2024-03-21T11:34:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.482+0000 7fc783200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7840ff2a0 0x7fc784199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.482+0000 7fc783200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7840ffc00 0x7fc784199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.482+0000 7fc783200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc784107190 0x7fc7841a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.482+0000 7fc783200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78410d1e0 con 0x7fc784107190 2024-03-21T11:34:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.482+0000 7fc783200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78410d060 con 0x7fc7840ffc00 2024-03-21T11:34:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.482+0000 7fc783200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78410d360 con 0x7fc7840ff2a0 2024-03-21T11:34:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.483+0000 7fc781e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7840ff2a0 0x7fc784199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.483+0000 7fc781e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7840ff2a0 0x7fc784199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43600/0 (socket says 172.21.15.47:43600) 2024-03-21T11:34:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.483+0000 7fc781e00640 1 -- 172.21.15.47:0/1041193241 learned_addr learned my addr 172.21.15.47:0/1041193241 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.483+0000 7fc782800640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc784107190 0x7fc7841a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.483+0000 7fc781400640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7840ffc00 0x7fc784199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.483+0000 7fc781e00640 1 -- 172.21.15.47:0/1041193241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7840ffc00 msgr2=0x7fc784199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.483+0000 7fc781e00640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7840ffc00 0x7fc784199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.483+0000 7fc781e00640 1 -- 172.21.15.47:0/1041193241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc784107190 msgr2=0x7fc7841a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.483+0000 7fc781e00640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc784107190 0x7fc7841a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.483+0000 7fc781e00640 1 -- 172.21.15.47:0/1041193241 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7841a16a0 con 0x7fc7840ff2a0 2024-03-21T11:34:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.483+0000 7fc781400640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7840ffc00 0x7fc784199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.483+0000 7fc782800640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc784107190 0x7fc7841a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:34:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.483+0000 7fc781e00640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7840ff2a0 0x7fc784199a80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc77402c8f0 tx=0x7fc774005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.484+0000 7fc77aa00640 1 -- 172.21.15.47:0/1041193241 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774041070 con 0x7fc7840ff2a0 2024-03-21T11:34:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.484+0000 7fc77aa00640 1 -- 172.21.15.47:0/1041193241 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7740044d0 con 0x7fc7840ff2a0 2024-03-21T11:34:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.484+0000 7fc77aa00640 1 -- 172.21.15.47:0/1041193241 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774034440 con 0x7fc7840ff2a0 2024-03-21T11:34:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.484+0000 7fc783200640 1 -- 172.21.15.47:0/1041193241 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7841a1930 con 0x7fc7840ff2a0 2024-03-21T11:34:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.484+0000 7fc783200640 1 -- 172.21.15.47:0/1041193241 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7841a1dd0 con 0x7fc7840ff2a0 2024-03-21T11:34:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.487+0000 7fc77aa00640 1 -- 172.21.15.47:0/1041193241 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc774044070 con 0x7fc7840ff2a0 2024-03-21T11:34:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.487+0000 7fc78bc81740 1 Processor -- start 2024-03-21T11:34:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.487+0000 7fc78bc81740 1 -- start start 2024-03-21T11:34:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.487+0000 7fc78bc81740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18a3f0 0x56228e18a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.487+0000 7fc78bc81740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e18f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.487+0000 7fc78bc81740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 0x56228e1923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.487+0000 7fc78bc81740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228e14d3b0 con 0x56228e18ff50 2024-03-21T11:34:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.487+0000 7fc78bc81740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228e0ef830 con 0x56228e18ade0 2024-03-21T11:34:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.487+0000 7fc78bc81740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228e0985d0 con 0x56228e18a3f0 2024-03-21T11:34:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.487+0000 7fc782800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18a3f0 0x56228e18a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.488+0000 7fc782800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18a3f0 0x56228e18a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43608/0 (socket says 172.21.15.47:43608) 2024-03-21T11:34:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.488+0000 7fc781e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e18f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.488+0000 7fc782800640 1 -- 172.21.15.47:0/2874893028 learned_addr learned my addr 172.21.15.47:0/2874893028 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.488+0000 7fc781400640 1 --2- 172.21.15.47:0/2874893028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 0x56228e1923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.488+0000 7fc782800640 1 -- 172.21.15.47:0/2874893028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 msgr2=0x56228e18f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.488+0000 7fc782800640 1 --2- 172.21.15.47:0/2874893028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e18f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.488+0000 7fc782800640 1 -- 172.21.15.47:0/2874893028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 msgr2=0x56228e1923e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.488+0000 7fc782800640 1 --2- 172.21.15.47:0/2874893028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 0x56228e1923e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.488+0000 7fc782800640 1 -- 172.21.15.47:0/2874893028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56228e0b02c0 con 0x56228e18a3f0 2024-03-21T11:34:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.488+0000 7fc782800640 1 --2- 172.21.15.47:0/2874893028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18a3f0 0x56228e18a810 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009fb0 tx=0x7fc77c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97f54e22f3e6e688 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.488+0000 7fc77aa00640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7680a2f90 0x7fc7680a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.488+0000 7fc76f400640 1 -- 172.21.15.47:0/2874893028 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c016070 con 0x56228e18a3f0 2024-03-21T11:34:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.489+0000 7fc76f400640 1 -- 172.21.15.47:0/2874893028 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc77c012bf0 con 0x56228e18a3f0 2024-03-21T11:34:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.489+0000 7fc76f400640 1 -- 172.21.15.47:0/2874893028 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c01bd20 con 0x56228e18a3f0 2024-03-21T11:34:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.489+0000 7fc77aa00640 1 -- 172.21.15.47:0/1041193241 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc7740f1410 con 0x7fc7840ff2a0 2024-03-21T11:34:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.489+0000 7fc781e00640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7680a2f90 0x7fc7680a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.489+0000 7fc78bc81740 1 -- 172.21.15.47:0/2874893028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18a3f0 msgr2=0x56228e18a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.489+0000 7fc78bc81740 1 --2- 172.21.15.47:0/2874893028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18a3f0 0x56228e18a810 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009fb0 tx=0x7fc77c010800 comp rx=0 tx=0).stop 2024-03-21T11:34:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.489+0000 7fc781e00640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7680a2f90 0x7fc7680a5450 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc7740eeef0 tx=0x7fc7740f4ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.489+0000 7fc78bc81740 1 -- 172.21.15.47:0/2874893028 shutdown_connections 2024-03-21T11:34:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.489+0000 7fc78bc81740 1 --2- 172.21.15.47:0/2874893028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 0x56228e1923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.489+0000 7fc78bc81740 1 --2- 172.21.15.47:0/2874893028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e18f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.489+0000 7fc78bc81740 1 --2- 172.21.15.47:0/2874893028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18a3f0 0x56228e18a810 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.489+0000 7fc78bc81740 1 -- 172.21.15.47:0/2874893028 >> 172.21.15.47:0/2874893028 conn(0x56228e185960 msgr2=0x56228e185d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.490+0000 7fc78bc81740 1 -- 172.21.15.47:0/2874893028 shutdown_connections 2024-03-21T11:34:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.490+0000 7fc78bc81740 1 -- 172.21.15.47:0/2874893028 wait complete. 2024-03-21T11:34:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.490+0000 7fc78bc81740 1 -- start start 2024-03-21T11:34:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.490+0000 7fc78bc81740 1 Processor -- start 2024-03-21T11:34:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc78bc81740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18a3f0 0x56228e194680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc78bc81740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18ade0 0x56228e194bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc78bc81740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 0x56228e19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc78bc81740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228e0aba20 con 0x56228e18ff50 2024-03-21T11:34:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc78bc81740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228e0eb8d0 con 0x56228e18a3f0 2024-03-21T11:34:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc78bc81740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228e192f00 con 0x56228e18ade0 2024-03-21T11:34:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc78bc81740 10 client.0 _ll_register_callbacks cb 0x56228e0e81c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc781400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18ade0 0x56228e194bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc781400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18ade0 0x56228e194bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43624/0 (socket says 172.21.15.47:43624) 2024-03-21T11:34:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc781400640 1 -- 172.21.15.47:0/401545174 learned_addr learned my addr 172.21.15.47:0/401545174 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc782800640 1 --2- 172.21.15.47:0/401545174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18a3f0 0x56228e194680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc782800640 1 --2- 172.21.15.47:0/401545174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 0x56228e19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc781400640 1 -- 172.21.15.47:0/401545174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18a3f0 msgr2=0x56228e194680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc781400640 1 --2- 172.21.15.47:0/401545174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18a3f0 0x56228e194680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc781400640 1 -- 172.21.15.47:0/401545174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 msgr2=0x56228e19bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc781400640 1 --2- 172.21.15.47:0/401545174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 0x56228e19bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.491+0000 7fc781400640 1 -- 172.21.15.47:0/401545174 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56228de04210 con 0x56228e18ade0 2024-03-21T11:34:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.492+0000 7fc781400640 1 --2- 172.21.15.47:0/401545174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18ade0 0x56228e194bc0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc770012700 tx=0x7fc770012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=381aeddb7f80d0cc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.492+0000 7fc763e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:34:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.492+0000 7fc763e00640 1 -- 172.21.15.47:0/401545174 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770011c10 con 0x56228e18ade0 2024-03-21T11:34:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.492+0000 7fc78bc81740 1 -- 172.21.15.47:0/401545174 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc77c012bf0 con 0x56228e18ade0 2024-03-21T11:34:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.492+0000 7fc763e00640 1 -- 172.21.15.47:0/401545174 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc770011db0 con 0x56228e18ade0 2024-03-21T11:34:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.492+0000 7fc763e00640 1 -- 172.21.15.47:0/401545174 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc770005020 con 0x56228e18ade0 2024-03-21T11:34:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.492+0000 7fc78bc81740 10 client.29395 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.492+0000 7fc78bc81740 1 -- 172.21.15.47:0/401545174 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56228ded0620 con 0x56228e18ade0 2024-03-21T11:34:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.493+0000 7fc763e00640 1 -- 172.21.15.47:0/401545174 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc77c012bf0 con 0x56228e18ade0 2024-03-21T11:34:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.493+0000 7fc763e00640 1 -- 172.21.15.47:0/401545174 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc77001c470 con 0x56228e18ade0 2024-03-21T11:34:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.493+0000 7fc78bc81740 10 client.29395 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.493+0000 7fc78bc81740 10 client.29395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.493+0000 7fc78bc81740 20 client.29395 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.493+0000 7fc78bc81740 1 --2- 172.21.15.47:0/401545174 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228e1a05a0 0x56228e1a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.493+0000 7fc78bc81740 4 client.29395 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.493+0000 7fc78bc81740 1 -- 172.21.15.47:0/401545174 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56228e0fcd70 con 0x56228e1a05a0 2024-03-21T11:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.493+0000 7fc782800640 1 --2- 172.21.15.47:0/401545174 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228e1a05a0 0x56228e1a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.494+0000 7fc782800640 1 --2- 172.21.15.47:0/401545174 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228e1a05a0 0x56228e1a29c0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfc8a5225c83d982 server_cookie=bc3f275dbb694975 in_seq=0 out_seq=0 2024-03-21T11:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.494+0000 7fc763e00640 10 client.29395 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.494+0000 7fc763e00640 1 -- 172.21.15.47:0/401545174 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56228e0fcd70 con 0x56228e1a05a0 2024-03-21T11:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.494+0000 7fc763e00640 10 client.29395 handle_command_reply: tid=0 2024-03-21T11:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.494+0000 7fc78bc81740 1 client.29395 shutdown 2024-03-21T11:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.495+0000 7fc78bc81740 1 -- 172.21.15.47:0/401545174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18ade0 msgr2=0x56228e194bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.495+0000 7fc78bc81740 1 --2- 172.21.15.47:0/401545174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18ade0 0x56228e194bc0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc770012700 tx=0x7fc770012bc0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.495+0000 7fc78bc81740 1 -- 172.21.15.47:0/401545174 shutdown_connections 2024-03-21T11:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.495+0000 7fc78bc81740 1 --2- 172.21.15.47:0/401545174 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228e1a05a0 0x56228e1a29c0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.495+0000 7fc78bc81740 1 --2- 172.21.15.47:0/401545174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 0x56228e19bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.495+0000 7fc78bc81740 1 --2- 172.21.15.47:0/401545174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18ade0 0x56228e194bc0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.495+0000 7fc78bc81740 1 --2- 172.21.15.47:0/401545174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18a3f0 0x56228e194680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.495+0000 7fc763e00640 0 client.29395 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.495+0000 7fc78bc81740 1 -- 172.21.15.47:0/401545174 >> 172.21.15.47:0/401545174 conn(0x56228e185960 msgr2=0x56228e184b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.495+0000 7fc78bc81740 1 -- 172.21.15.47:0/401545174 shutdown_connections 2024-03-21T11:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.495+0000 7fc78bc81740 1 -- 172.21.15.47:0/401545174 wait complete. 2024-03-21T11:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.495+0000 7fc78bc81740 20 client.29395 trim_cache size 0 max 16384 2024-03-21T11:34:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.520+0000 7fc78bc81740 1 Processor -- start 2024-03-21T11:34:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.520+0000 7fc78bc81740 1 -- start start 2024-03-21T11:34:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.521+0000 7fc78bc81740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18a3f0 0x56228e1892d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.521+0000 7fc78bc81740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.521+0000 7fc78bc81740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18ff50 0x56228e1a7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.521+0000 7fc78bc81740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7841ac170 con 0x56228e18a3f0 2024-03-21T11:34:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.521+0000 7fc78bc81740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7841adaf0 con 0x56228e18ade0 2024-03-21T11:34:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.521+0000 7fc78bc81740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7841ac7d0 con 0x56228e18ff50 2024-03-21T11:34:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.522+0000 7fc781400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.522+0000 7fc781400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e19bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59584/0 (socket says 172.21.15.47:59584) 2024-03-21T11:34:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.522+0000 7fc781400640 1 -- 172.21.15.47:0/1392393561 learned_addr learned my addr 172.21.15.47:0/1392393561 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.522+0000 7fc782800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18a3f0 0x56228e1892d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.522+0000 7fc782800640 1 --2- 172.21.15.47:0/1392393561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18ff50 0x56228e1a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.523+0000 7fc781400640 1 -- 172.21.15.47:0/1392393561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18ff50 msgr2=0x56228e1a7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.523+0000 7fc781400640 1 --2- 172.21.15.47:0/1392393561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18ff50 0x56228e1a7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.523+0000 7fc781400640 1 -- 172.21.15.47:0/1392393561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18a3f0 msgr2=0x56228e1892d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.523+0000 7fc781400640 1 --2- 172.21.15.47:0/1392393561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18a3f0 0x56228e1892d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.523+0000 7fc781400640 1 -- 172.21.15.47:0/1392393561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56228e1866d0 con 0x56228e18ade0 2024-03-21T11:34:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.523+0000 7fc782800640 1 --2- 172.21.15.47:0/1392393561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18ff50 0x56228e1a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:34:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.524+0000 7fc781400640 1 --2- 172.21.15.47:0/1392393561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e19bb50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc770009ed0 tx=0x7fc7700118b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c69bda1529ce12b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.525+0000 7fc76ea00640 1 -- 172.21.15.47:0/1392393561 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770009680 con 0x56228e18ade0 2024-03-21T11:34:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.525+0000 7fc76ea00640 1 -- 172.21.15.47:0/1392393561 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc77001cad0 con 0x56228e18ade0 2024-03-21T11:34:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.525+0000 7fc76ea00640 1 -- 172.21.15.47:0/1392393561 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770004a90 con 0x56228e18ade0 2024-03-21T11:34:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.526+0000 7fc78bc81740 1 -- 172.21.15.47:0/1392393561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 msgr2=0x56228e19bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.526+0000 7fc78bc81740 1 --2- 172.21.15.47:0/1392393561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e19bb50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc770009ed0 tx=0x7fc7700118b0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.526+0000 7fc78bc81740 1 -- 172.21.15.47:0/1392393561 shutdown_connections 2024-03-21T11:34:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.526+0000 7fc78bc81740 1 --2- 172.21.15.47:0/1392393561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18ff50 0x56228e1a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.526+0000 7fc78bc81740 1 --2- 172.21.15.47:0/1392393561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e19bb50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.526+0000 7fc78bc81740 1 --2- 172.21.15.47:0/1392393561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18a3f0 0x56228e1892d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.526+0000 7fc78bc81740 1 -- 172.21.15.47:0/1392393561 >> 172.21.15.47:0/1392393561 conn(0x56228e185960 msgr2=0x56228e188bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.527+0000 7fc78bc81740 1 -- 172.21.15.47:0/1392393561 shutdown_connections 2024-03-21T11:34:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.527+0000 7fc78bc81740 1 -- 172.21.15.47:0/1392393561 wait complete. 2024-03-21T11:34:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.528+0000 7fc78bc81740 1 -- start start 2024-03-21T11:34:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.529+0000 7fc78bc81740 1 Processor -- start 2024-03-21T11:34:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.530+0000 7fc78bc81740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18a3f0 0x56228e19b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.530+0000 7fc78bc81740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e19bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.531+0000 7fc78bc81740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 0x56228e199e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.531+0000 7fc78bc81740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7841ad490 con 0x56228e18ff50 2024-03-21T11:34:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.531+0000 7fc78bc81740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7841ace30 con 0x56228e18ade0 2024-03-21T11:34:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.531+0000 7fc78bc81740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7841aa330 con 0x56228e18a3f0 2024-03-21T11:34:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.531+0000 7fc78bc81740 10 client.0 _ll_register_callbacks cb 0x7fc770009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:34:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.531+0000 7fc782800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18a3f0 0x56228e19b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.531+0000 7fc782800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18a3f0 0x56228e19b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43632/0 (socket says 172.21.15.47:43632) 2024-03-21T11:34:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.531+0000 7fc782800640 1 -- 172.21.15.47:0/3037613439 learned_addr learned my addr 172.21.15.47:0/3037613439 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:34:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.531+0000 7fc781400640 1 --2- 172.21.15.47:0/3037613439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e19bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.531+0000 7fc782800640 1 --2- 172.21.15.47:0/3037613439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 0x56228e199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.532+0000 7fc781400640 1 -- 172.21.15.47:0/3037613439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18a3f0 msgr2=0x56228e19b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.532+0000 7fc781400640 1 --2- 172.21.15.47:0/3037613439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18a3f0 0x56228e19b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.532+0000 7fc781400640 1 -- 172.21.15.47:0/3037613439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 msgr2=0x56228e199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:34:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.532+0000 7fc781400640 1 --2- 172.21.15.47:0/3037613439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 0x56228e199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.532+0000 7fc781400640 1 -- 172.21.15.47:0/3037613439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56228e19a670 con 0x56228e18ade0 2024-03-21T11:34:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.533+0000 7fc781400640 1 --2- 172.21.15.47:0/3037613439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e19bc10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc7700047e0 tx=0x7fc7700053d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eb8f9baa94352a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:34:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.534+0000 7fc763e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:34:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.536+0000 7fc763e00640 1 -- 172.21.15.47:0/3037613439 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77003f060 con 0x56228e18ade0 2024-03-21T11:34:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.536+0000 7fc763e00640 1 -- 172.21.15.47:0/3037613439 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc770018050 con 0x56228e18ade0 2024-03-21T11:34:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.536+0000 7fc78bc81740 1 -- 172.21.15.47:0/3037613439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc77001cad0 con 0x56228e18ade0 2024-03-21T11:34:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.537+0000 7fc763e00640 1 -- 172.21.15.47:0/3037613439 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc77003f060 con 0x56228e18ade0 2024-03-21T11:34:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.537+0000 7fc78bc81740 10 client.29378 fetch_fsmap learned FSMap version 28 2024-03-21T11:34:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.537+0000 7fc78bc81740 1 -- 172.21.15.47:0/3037613439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56228e2051e0 con 0x56228e18ade0 2024-03-21T11:34:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.538+0000 7fc763e00640 1 -- 172.21.15.47:0/3037613439 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc77001cad0 con 0x56228e18ade0 2024-03-21T11:34:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.538+0000 7fc763e00640 1 -- 172.21.15.47:0/3037613439 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc770042050 con 0x56228e18ade0 2024-03-21T11:34:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.538+0000 7fc78bc81740 10 client.29378 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:34:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.538+0000 7fc78bc81740 10 client.29378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:34:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.538+0000 7fc78bc81740 20 client.29378 populate_metadata read hostname 'smithi047' 2024-03-21T11:34:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.538+0000 7fc78bc81740 1 --2- 172.21.15.47:0/3037613439 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228e205ae0 0x56228e207f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:34:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.538+0000 7fc78bc81740 4 client.29378 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:34:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.538+0000 7fc78bc81740 1 -- 172.21.15.47:0/3037613439 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56228e0f70c0 con 0x56228e205ae0 2024-03-21T11:34:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.539+0000 7fc782800640 1 --2- 172.21.15.47:0/3037613439 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228e205ae0 0x56228e207f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:34:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.541+0000 7fc782800640 1 --2- 172.21.15.47:0/3037613439 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228e205ae0 0x56228e207f00 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c957f252346f60b8 server_cookie=f549b813a084fde0 in_seq=0 out_seq=0 2024-03-21T11:34:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.541+0000 7fc763e00640 10 client.29378 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:46.561 INFO:tasks.workunit.client.0.smithi047.stdout: 1 862538 8.03 MB/sec execute 450 sec latency 64.018 ms 2024-03-21T11:34:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.849+0000 7fc763e00640 1 -- 172.21.15.47:0/3037613439 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56228e0f70c0 con 0x56228e205ae0 2024-03-21T11:34:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.849+0000 7fc763e00640 10 client.29378 handle_command_reply: tid=0 2024-03-21T11:34:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.849+0000 7fc78bc81740 1 client.29378 shutdown 2024-03-21T11:34:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.849+0000 7fc78bc81740 1 -- 172.21.15.47:0/3037613439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 msgr2=0x56228e19bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.849+0000 7fc78bc81740 1 --2- 172.21.15.47:0/3037613439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e19bc10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc7700047e0 tx=0x7fc7700053d0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.849+0000 7fc78bc81740 1 -- 172.21.15.47:0/3037613439 shutdown_connections 2024-03-21T11:34:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.849+0000 7fc78bc81740 1 --2- 172.21.15.47:0/3037613439 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228e205ae0 0x56228e207f00 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.849+0000 7fc78bc81740 1 --2- 172.21.15.47:0/3037613439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228e18ff50 0x56228e199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.849+0000 7fc763e00640 0 client.29378 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:34:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.849+0000 7fc78bc81740 1 --2- 172.21.15.47:0/3037613439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228e18ade0 0x56228e19bc10 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.849+0000 7fc78bc81740 1 --2- 172.21.15.47:0/3037613439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228e18a3f0 0x56228e19b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.849+0000 7fc78bc81740 1 -- 172.21.15.47:0/3037613439 >> 172.21.15.47:0/3037613439 conn(0x56228e185960 msgr2=0x56228e1842e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.849+0000 7fc78bc81740 1 -- 172.21.15.47:0/3037613439 shutdown_connections 2024-03-21T11:34:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.850+0000 7fc78bc81740 1 -- 172.21.15.47:0/3037613439 wait complete. 2024-03-21T11:34:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.850+0000 7fc78bc81740 20 client.29378 trim_cache size 0 max 16384 2024-03-21T11:34:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:34:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:34:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:34:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:34:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.851+0000 7fc783200640 1 -- 172.21.15.47:0/1041193241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7680a2f90 msgr2=0x7fc7680a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.851+0000 7fc783200640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7680a2f90 0x7fc7680a5450 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc7740eeef0 tx=0x7fc7740f4ef0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.851+0000 7fc783200640 1 -- 172.21.15.47:0/1041193241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7840ff2a0 msgr2=0x7fc784199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:34:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.851+0000 7fc783200640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7840ff2a0 0x7fc784199a80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc77402c8f0 tx=0x7fc774005d50 comp rx=0 tx=0).stop 2024-03-21T11:34:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.851+0000 7fc783200640 1 -- 172.21.15.47:0/1041193241 shutdown_connections 2024-03-21T11:34:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.851+0000 7fc783200640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7680a2f90 0x7fc7680a5450 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.851+0000 7fc783200640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc784107190 0x7fc7841a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.851+0000 7fc783200640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7840ffc00 0x7fc784199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.851+0000 7fc783200640 1 --2- 172.21.15.47:0/1041193241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7840ff2a0 0x7fc784199a80 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:34:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.851+0000 7fc783200640 1 -- 172.21.15.47:0/1041193241 >> 172.21.15.47:0/1041193241 conn(0x7fc7840facb0 msgr2=0x7fc7840fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:34:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.851+0000 7fc783200640 1 -- 172.21.15.47:0/1041193241 shutdown_connections 2024-03-21T11:34:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:34:46.851+0000 7fc783200640 1 -- 172.21.15.47:0/1041193241 wait complete. 2024-03-21T11:34:46.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:46 smithi100 ceph-mon[36245]: pgmap v743: 129 pgs: 129 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 285 op/s 2024-03-21T11:34:46.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:46 smithi102 ceph-mon[36748]: pgmap v743: 129 pgs: 129 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 285 op/s 2024-03-21T11:34:47.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:46 smithi047 ceph-mon[31351]: pgmap v743: 129 pgs: 129 active+clean; 40 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 285 op/s 2024-03-21T11:34:47.561 INFO:tasks.workunit.client.0.smithi047.stdout: 1 863222 8.02 MB/sec execute 451 sec latency 170.024 ms 2024-03-21T11:34:48.561 INFO:tasks.workunit.client.0.smithi047.stdout: 1 864362 8.01 MB/sec execute 452 sec latency 62.726 ms 2024-03-21T11:34:48.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:48 smithi100 ceph-mon[36245]: pgmap v744: 129 pgs: 129 active+clean; 36 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 283 op/s 2024-03-21T11:34:48.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:48 smithi102 ceph-mon[36748]: pgmap v744: 129 pgs: 129 active+clean; 36 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 283 op/s 2024-03-21T11:34:49.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:48 smithi047 ceph-mon[31351]: pgmap v744: 129 pgs: 129 active+clean; 36 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 283 op/s 2024-03-21T11:34:49.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 866119 8.02 MB/sec execute 453 sec latency 20.068 ms 2024-03-21T11:34:50.561 INFO:tasks.workunit.client.0.smithi047.stdout: 1 867044 8.01 MB/sec execute 454 sec latency 15.581 ms 2024-03-21T11:34:50.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:50 smithi100 ceph-mon[36245]: pgmap v745: 129 pgs: 129 active+clean; 34 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 316 op/s 2024-03-21T11:34:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:50 smithi102 ceph-mon[36748]: pgmap v745: 129 pgs: 129 active+clean; 34 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 316 op/s 2024-03-21T11:34:51.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:50 smithi047 ceph-mon[31351]: pgmap v745: 129 pgs: 129 active+clean; 34 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 316 op/s 2024-03-21T11:34:51.561 INFO:tasks.workunit.client.0.smithi047.stdout: 1 868639 8.01 MB/sec execute 455 sec latency 66.578 ms 2024-03-21T11:34:52.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 870073 8.02 MB/sec execute 456 sec latency 15.315 ms 2024-03-21T11:34:52.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:52 smithi100 ceph-mon[36245]: pgmap v746: 129 pgs: 129 active+clean; 34 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.8 MiB/s wr, 250 op/s 2024-03-21T11:34:52.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:52 smithi102 ceph-mon[36748]: pgmap v746: 129 pgs: 129 active+clean; 34 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.8 MiB/s wr, 250 op/s 2024-03-21T11:34:53.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:52 smithi047 ceph-mon[31351]: pgmap v746: 129 pgs: 129 active+clean; 34 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.8 MiB/s wr, 250 op/s 2024-03-21T11:34:53.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 870825 8.00 MB/sec execute 457 sec latency 14.995 ms 2024-03-21T11:34:54.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 872631 8.01 MB/sec execute 458 sec latency 13.985 ms 2024-03-21T11:34:54.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:54 smithi100 ceph-mon[36245]: pgmap v747: 129 pgs: 129 active+clean; 36 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 295 op/s 2024-03-21T11:34:54.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:54 smithi102 ceph-mon[36748]: pgmap v747: 129 pgs: 129 active+clean; 36 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 295 op/s 2024-03-21T11:34:55.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:54 smithi047 ceph-mon[31351]: pgmap v747: 129 pgs: 129 active+clean; 36 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 295 op/s 2024-03-21T11:34:55.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 873796 8.01 MB/sec execute 459 sec latency 14.736 ms 2024-03-21T11:34:56.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 874539 7.99 MB/sec execute 460 sec latency 77.824 ms 2024-03-21T11:34:56.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:56 smithi100 ceph-mon[36245]: pgmap v748: 129 pgs: 129 active+clean; 36 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-03-21T11:34:56.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:56 smithi102 ceph-mon[36748]: pgmap v748: 129 pgs: 129 active+clean; 36 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-03-21T11:34:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:56 smithi047 ceph-mon[31351]: pgmap v748: 129 pgs: 129 active+clean; 36 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-03-21T11:34:57.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 876300 8.00 MB/sec execute 461 sec latency 26.250 ms 2024-03-21T11:34:58.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 877413 8.00 MB/sec execute 462 sec latency 16.296 ms 2024-03-21T11:34:58.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:34:58 smithi047 ceph-mon[31351]: pgmap v749: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-03-21T11:34:58.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:34:58 smithi100 ceph-mon[36245]: pgmap v749: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-03-21T11:34:58.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:34:58 smithi102 ceph-mon[36748]: pgmap v749: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-03-21T11:34:59.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 878567 7.99 MB/sec execute 463 sec latency 31.835 ms 2024-03-21T11:35:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.010+0000 7f4294e00640 1 -- 172.21.15.47:0/79510550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42901017b0 msgr2=0x7f4290101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.010+0000 7f4294e00640 1 --2- 172.21.15.47:0/79510550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42901017b0 0x7f4290101c30 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f427c009900 tx=0x7f427c02bc50 comp rx=0 tx=0).stop 2024-03-21T11:35:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.011+0000 7f4294e00640 1 -- 172.21.15.47:0/79510550 shutdown_connections 2024-03-21T11:35:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.011+0000 7f4294e00640 1 --2- 172.21.15.47:0/79510550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4290102170 0x7f4290108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.011+0000 7f4294e00640 1 --2- 172.21.15.47:0/79510550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42901017b0 0x7f4290101c30 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.011+0000 7f4294e00640 1 --2- 172.21.15.47:0/79510550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42901005c0 0x7f42901009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.011+0000 7f4294e00640 1 -- 172.21.15.47:0/79510550 >> 172.21.15.47:0/79510550 conn(0x7f42900fbb30 msgr2=0x7f42900fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.011+0000 7f4294e00640 1 -- 172.21.15.47:0/79510550 shutdown_connections 2024-03-21T11:35:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.011+0000 7f4294e00640 1 -- 172.21.15.47:0/79510550 wait complete. 2024-03-21T11:35:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.012+0000 7f4294e00640 1 Processor -- start 2024-03-21T11:35:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.012+0000 7f4294e00640 1 -- start start 2024-03-21T11:35:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.012+0000 7f4294e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42901005c0 0x7f4290199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.012+0000 7f4294e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42901017b0 0x7f429019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.012+0000 7f4294e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4290102170 0x7f42901a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.012+0000 7f4294e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429010c280 con 0x7f42901017b0 2024-03-21T11:35:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.012+0000 7f4294e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429010c100 con 0x7f42901005c0 2024-03-21T11:35:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.012+0000 7f4294e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429010c400 con 0x7f4290102170 2024-03-21T11:35:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.012+0000 7f428d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42901017b0 0x7f429019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.012+0000 7f428e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4290102170 0x7f42901a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.012+0000 7f428de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42901005c0 0x7f4290199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.013+0000 7f428de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42901005c0 0x7f4290199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55814/0 (socket says 172.21.15.47:55814) 2024-03-21T11:35:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.013+0000 7f428e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4290102170 0x7f42901a1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45638/0 (socket says 172.21.15.47:45638) 2024-03-21T11:35:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.013+0000 7f428de00640 1 -- 172.21.15.47:0/2613535632 learned_addr learned my addr 172.21.15.47:0/2613535632 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.013+0000 7f428de00640 1 -- 172.21.15.47:0/2613535632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4290102170 msgr2=0x7f42901a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.013+0000 7f428de00640 1 --2- 172.21.15.47:0/2613535632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4290102170 0x7f42901a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.013+0000 7f428de00640 1 -- 172.21.15.47:0/2613535632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42901017b0 msgr2=0x7f429019a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.013+0000 7f428de00640 1 --2- 172.21.15.47:0/2613535632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42901017b0 0x7f429019a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.013+0000 7f428de00640 1 -- 172.21.15.47:0/2613535632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42901a17b0 con 0x7f42901005c0 2024-03-21T11:35:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.013+0000 7f428de00640 1 --2- 172.21.15.47:0/2613535632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42901005c0 0x7f4290199b90 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f428000e9c0 tx=0x7f428000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.013+0000 7f4286a00640 1 -- 172.21.15.47:0/2613535632 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428000ccc0 con 0x7f42901005c0 2024-03-21T11:35:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.013+0000 7f4286a00640 1 -- 172.21.15.47:0/2613535632 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42800035f0 con 0x7f42901005c0 2024-03-21T11:35:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.013+0000 7f4286a00640 1 -- 172.21.15.47:0/2613535632 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428000ce60 con 0x7f42901005c0 2024-03-21T11:35:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.014+0000 7f4294e00640 1 -- 172.21.15.47:0/2613535632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42901a1aa0 con 0x7f42901005c0 2024-03-21T11:35:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.014+0000 7f4294e00640 1 -- 172.21.15.47:0/2613535632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42901a1fc0 con 0x7f42901005c0 2024-03-21T11:35:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.017+0000 7f4297b8e740 1 Processor -- start 2024-03-21T11:35:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.017+0000 7f4297b8e740 1 -- start start 2024-03-21T11:35:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f4297b8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d5291e0 0x55dc1d529600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f4297b8e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d52e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f4297b8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d52ed40 0x55dc1d5311d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f4286a00640 1 -- 172.21.15.47:0/2613535632 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f42800037a0 con 0x7f42901005c0 2024-03-21T11:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f4297b8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1d4ec3b0 con 0x55dc1d52ed40 2024-03-21T11:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f4297b8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1d48e830 con 0x55dc1d5291e0 2024-03-21T11:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f4297b8e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1d4375d0 con 0x55dc1d529bd0 2024-03-21T11:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f428e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d5291e0 0x55dc1d529600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f428e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d5291e0 0x55dc1d529600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55826/0 (socket says 172.21.15.47:55826) 2024-03-21T11:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f428e800640 1 -- 172.21.15.47:0/2663153070 learned_addr learned my addr 172.21.15.47:0/2663153070 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f428de00640 1 --2- 172.21.15.47:0/2663153070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d52e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f428d400640 1 --2- 172.21.15.47:0/2663153070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d52ed40 0x55dc1d5311d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f428e800640 1 -- 172.21.15.47:0/2663153070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 msgr2=0x55dc1d52e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f428e800640 1 --2- 172.21.15.47:0/2663153070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d52e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f428e800640 1 -- 172.21.15.47:0/2663153070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d52ed40 msgr2=0x55dc1d5311d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f428e800640 1 --2- 172.21.15.47:0/2663153070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d52ed40 0x55dc1d5311d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.018+0000 7f428e800640 1 -- 172.21.15.47:0/2663153070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc1d44f2c0 con 0x55dc1d5291e0 2024-03-21T11:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.019+0000 7f428e800640 1 --2- 172.21.15.47:0/2663153070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d5291e0 0x55dc1d529600 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4288011440 tx=0x7f4288011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21c1e8f87c452742 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.019+0000 7f427b400640 1 -- 172.21.15.47:0/2663153070 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288003c30 con 0x55dc1d5291e0 2024-03-21T11:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.019+0000 7f427b400640 1 -- 172.21.15.47:0/2663153070 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4288003dd0 con 0x55dc1d5291e0 2024-03-21T11:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.019+0000 7f427b400640 1 -- 172.21.15.47:0/2663153070 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428800fb50 con 0x55dc1d5291e0 2024-03-21T11:35:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.019+0000 7f4286a00640 1 --2- 172.21.15.47:0/2613535632 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f42740a2fc0 0x7f42740a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.020+0000 7f4286a00640 1 -- 172.21.15.47:0/2613535632 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f42800cfaa0 con 0x7f42901005c0 2024-03-21T11:35:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.020+0000 7f428de00640 1 --2- 172.21.15.47:0/2613535632 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f42740a2fc0 0x7f42740a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.020+0000 7f4297b8e740 1 -- 172.21.15.47:0/2663153070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d5291e0 msgr2=0x55dc1d529600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.020+0000 7f4297b8e740 1 --2- 172.21.15.47:0/2663153070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d5291e0 0x55dc1d529600 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4288011440 tx=0x7f4288011900 comp rx=0 tx=0).stop 2024-03-21T11:35:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.020+0000 7f4297b8e740 1 -- 172.21.15.47:0/2663153070 shutdown_connections 2024-03-21T11:35:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.020+0000 7f4297b8e740 1 --2- 172.21.15.47:0/2663153070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d52ed40 0x55dc1d5311d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.020+0000 7f4297b8e740 1 --2- 172.21.15.47:0/2663153070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d52e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.020+0000 7f4297b8e740 1 --2- 172.21.15.47:0/2663153070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d5291e0 0x55dc1d529600 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.020+0000 7f4297b8e740 1 -- 172.21.15.47:0/2663153070 >> 172.21.15.47:0/2663153070 conn(0x55dc1d524750 msgr2=0x55dc1d524b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.020+0000 7f4297b8e740 1 -- 172.21.15.47:0/2663153070 shutdown_connections 2024-03-21T11:35:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.020+0000 7f4297b8e740 1 -- 172.21.15.47:0/2663153070 wait complete. 2024-03-21T11:35:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.020+0000 7f428de00640 1 --2- 172.21.15.47:0/2613535632 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f42740a2fc0 0x7f42740a5480 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f42800cd580 tx=0x7f42800d3580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.021+0000 7f4297b8e740 1 -- start start 2024-03-21T11:35:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.021+0000 7f4297b8e740 1 Processor -- start 2024-03-21T11:35:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.021+0000 7f4297b8e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d5291e0 0x55dc1d533310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.021+0000 7f4297b8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d529bd0 0x55dc1d533850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.021+0000 7f4297b8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d52ed40 0x55dc1d53a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f4297b8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1d44aa20 con 0x55dc1d52ed40 2024-03-21T11:35:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f4297b8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1d48a8d0 con 0x55dc1d529bd0 2024-03-21T11:35:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f4297b8e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1d531cf0 con 0x55dc1d5291e0 2024-03-21T11:35:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f4297b8e740 10 client.0 _ll_register_callbacks cb 0x55dc1d4871c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f428d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d529bd0 0x55dc1d533850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f428d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d529bd0 0x55dc1d533850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55840/0 (socket says 172.21.15.47:55840) 2024-03-21T11:35:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f428d400640 1 -- 172.21.15.47:0/4055876766 learned_addr learned my addr 172.21.15.47:0/4055876766 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f428e800640 1 --2- 172.21.15.47:0/4055876766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d5291e0 0x55dc1d533310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f428e800640 1 --2- 172.21.15.47:0/4055876766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d52ed40 0x55dc1d53a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f428d400640 1 -- 172.21.15.47:0/4055876766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d5291e0 msgr2=0x55dc1d533310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f428d400640 1 --2- 172.21.15.47:0/4055876766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d5291e0 0x55dc1d533310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f428d400640 1 -- 172.21.15.47:0/4055876766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d52ed40 msgr2=0x55dc1d53a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f428d400640 1 --2- 172.21.15.47:0/4055876766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d52ed40 0x55dc1d53a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f428d400640 1 -- 172.21.15.47:0/4055876766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc1d1a3210 con 0x55dc1d529bd0 2024-03-21T11:35:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.022+0000 7f428e800640 1 --2- 172.21.15.47:0/4055876766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d5291e0 0x55dc1d533310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.023+0000 7f428d400640 1 --2- 172.21.15.47:0/4055876766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d529bd0 0x55dc1d533850 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f427c005ab0 tx=0x7f427c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c808bcc5cb22ff43 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.023+0000 7f426fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.023+0000 7f426fe00640 1 -- 172.21.15.47:0/4055876766 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c005d50 con 0x55dc1d529bd0 2024-03-21T11:35:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.023+0000 7f4297b8e740 1 -- 172.21.15.47:0/4055876766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4288003dd0 con 0x55dc1d529bd0 2024-03-21T11:35:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.023+0000 7f426fe00640 1 -- 172.21.15.47:0/4055876766 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f427c0046c0 con 0x55dc1d529bd0 2024-03-21T11:35:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.023+0000 7f426fe00640 1 -- 172.21.15.47:0/4055876766 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f427c0048c0 con 0x55dc1d529bd0 2024-03-21T11:35:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.023+0000 7f4297b8e740 10 client.29390 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.023+0000 7f4297b8e740 1 -- 172.21.15.47:0/4055876766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dc1d26f620 con 0x55dc1d529bd0 2024-03-21T11:35:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.024+0000 7f426fe00640 1 -- 172.21.15.47:0/4055876766 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4288003dd0 con 0x55dc1d529bd0 2024-03-21T11:35:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.024+0000 7f426fe00640 1 -- 172.21.15.47:0/4055876766 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f427c0035f0 con 0x55dc1d529bd0 2024-03-21T11:35:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.024+0000 7f4297b8e740 10 client.29390 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.024+0000 7f4297b8e740 10 client.29390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.024+0000 7f4297b8e740 20 client.29390 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.024+0000 7f4297b8e740 1 --2- 172.21.15.47:0/4055876766 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc1d53f160 0x55dc1d541580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.024+0000 7f4297b8e740 4 client.29390 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.024+0000 7f4297b8e740 1 -- 172.21.15.47:0/4055876766 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc1d49bd70 con 0x55dc1d53f160 2024-03-21T11:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.024+0000 7f428e800640 1 --2- 172.21.15.47:0/4055876766 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc1d53f160 0x55dc1d541580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.025+0000 7f428e800640 1 --2- 172.21.15.47:0/4055876766 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc1d53f160 0x55dc1d541580 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c27d06f8341bb1 server_cookie=d4584e4f919cbbc8 in_seq=0 out_seq=0 2024-03-21T11:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.025+0000 7f426fe00640 10 client.29390 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f426fe00640 1 -- 172.21.15.47:0/4055876766 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dc1d49bd70 con 0x55dc1d53f160 2024-03-21T11:35:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f426fe00640 10 client.29390 handle_command_reply: tid=0 2024-03-21T11:35:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f4297b8e740 1 client.29390 shutdown 2024-03-21T11:35:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f4297b8e740 1 -- 172.21.15.47:0/4055876766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d529bd0 msgr2=0x55dc1d533850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f4297b8e740 1 --2- 172.21.15.47:0/4055876766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d529bd0 0x55dc1d533850 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f427c005ab0 tx=0x7f427c004270 comp rx=0 tx=0).stop 2024-03-21T11:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f4297b8e740 1 -- 172.21.15.47:0/4055876766 shutdown_connections 2024-03-21T11:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f4297b8e740 1 --2- 172.21.15.47:0/4055876766 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc1d53f160 0x55dc1d541580 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f4297b8e740 1 --2- 172.21.15.47:0/4055876766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d52ed40 0x55dc1d53a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f4297b8e740 1 --2- 172.21.15.47:0/4055876766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d529bd0 0x55dc1d533850 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f4297b8e740 1 --2- 172.21.15.47:0/4055876766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d5291e0 0x55dc1d533310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f4297b8e740 1 -- 172.21.15.47:0/4055876766 >> 172.21.15.47:0/4055876766 conn(0x55dc1d524750 msgr2=0x55dc1d523950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f426fe00640 0 client.29390 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f4297b8e740 1 -- 172.21.15.47:0/4055876766 shutdown_connections 2024-03-21T11:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f4297b8e740 1 -- 172.21.15.47:0/4055876766 wait complete. 2024-03-21T11:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.026+0000 7f4297b8e740 20 client.29390 trim_cache size 0 max 16384 2024-03-21T11:35:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.043+0000 7f4297b8e740 1 Processor -- start 2024-03-21T11:35:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.043+0000 7f4297b8e740 1 -- start start 2024-03-21T11:35:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f4297b8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d5291e0 0x55dc1d52ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f4297b8e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d52b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f4297b8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d52ed40 0x55dc1d53a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f4297b8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42901ac300 con 0x55dc1d5291e0 2024-03-21T11:35:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f4297b8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42901adc80 con 0x55dc1d52ed40 2024-03-21T11:35:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f4297b8e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42901ac960 con 0x55dc1d529bd0 2024-03-21T11:35:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f428e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d5291e0 0x55dc1d52ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f428d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d52b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f428d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d52b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45658/0 (socket says 172.21.15.47:45658) 2024-03-21T11:35:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f428d400640 1 -- 172.21.15.47:0/526393007 learned_addr learned my addr 172.21.15.47:0/526393007 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f428e800640 1 --2- 172.21.15.47:0/526393007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d52ed40 0x55dc1d53a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f428d400640 1 -- 172.21.15.47:0/526393007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d52ed40 msgr2=0x55dc1d53a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f428d400640 1 --2- 172.21.15.47:0/526393007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d52ed40 0x55dc1d53a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f428d400640 1 -- 172.21.15.47:0/526393007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d5291e0 msgr2=0x55dc1d52ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f428d400640 1 --2- 172.21.15.47:0/526393007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d5291e0 0x55dc1d52ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.044+0000 7f428d400640 1 -- 172.21.15.47:0/526393007 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc1d52a460 con 0x55dc1d529bd0 2024-03-21T11:35:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.045+0000 7f428d400640 1 --2- 172.21.15.47:0/526393007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d52b390 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f427c02c900 tx=0x7f427c005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6238ec9585c315c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.045+0000 7f427aa00640 1 -- 172.21.15.47:0/526393007 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c005d50 con 0x55dc1d529bd0 2024-03-21T11:35:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.045+0000 7f427aa00640 1 -- 172.21.15.47:0/526393007 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f427c009520 con 0x55dc1d529bd0 2024-03-21T11:35:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.045+0000 7f427aa00640 1 -- 172.21.15.47:0/526393007 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c0053b0 con 0x55dc1d529bd0 2024-03-21T11:35:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.046+0000 7f4297b8e740 1 -- 172.21.15.47:0/526393007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 msgr2=0x55dc1d52b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.046+0000 7f4297b8e740 1 --2- 172.21.15.47:0/526393007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d52b390 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f427c02c900 tx=0x7f427c005c90 comp rx=0 tx=0).stop 2024-03-21T11:35:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.046+0000 7f4297b8e740 1 -- 172.21.15.47:0/526393007 shutdown_connections 2024-03-21T11:35:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.046+0000 7f4297b8e740 1 --2- 172.21.15.47:0/526393007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d52ed40 0x55dc1d53a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.046+0000 7f4297b8e740 1 --2- 172.21.15.47:0/526393007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d52b390 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.046+0000 7f4297b8e740 1 --2- 172.21.15.47:0/526393007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d5291e0 0x55dc1d52ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.046+0000 7f4297b8e740 1 -- 172.21.15.47:0/526393007 >> 172.21.15.47:0/526393007 conn(0x55dc1d524750 msgr2=0x55dc1d524b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.046+0000 7f4297b8e740 1 -- 172.21.15.47:0/526393007 shutdown_connections 2024-03-21T11:35:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.046+0000 7f4297b8e740 1 -- 172.21.15.47:0/526393007 wait complete. 2024-03-21T11:35:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.047+0000 7f4297b8e740 1 -- start start 2024-03-21T11:35:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.047+0000 7f4297b8e740 1 Processor -- start 2024-03-21T11:35:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.047+0000 7f4297b8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d5291e0 0x55dc1d538510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.047+0000 7f4297b8e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d538a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.047+0000 7f4297b8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d52ed40 0x55dc1d5399c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.047+0000 7f4297b8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42901ad620 con 0x55dc1d5291e0 2024-03-21T11:35:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.047+0000 7f4297b8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42901acfc0 con 0x55dc1d52ed40 2024-03-21T11:35:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.047+0000 7f4297b8e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42901aa4c0 con 0x55dc1d529bd0 2024-03-21T11:35:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.048+0000 7f4297b8e740 10 client.0 _ll_register_callbacks cb 0x7f42880093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.048+0000 7f428d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d538a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.048+0000 7f428d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d538a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45664/0 (socket says 172.21.15.47:45664) 2024-03-21T11:35:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.048+0000 7f428d400640 1 -- 172.21.15.47:0/261596688 learned_addr learned my addr 172.21.15.47:0/261596688 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.048+0000 7f428e800640 1 --2- 172.21.15.47:0/261596688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d5291e0 0x55dc1d538510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.048+0000 7f428e800640 1 --2- 172.21.15.47:0/261596688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d52ed40 0x55dc1d5399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.048+0000 7f428d400640 1 -- 172.21.15.47:0/261596688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d52ed40 msgr2=0x55dc1d5399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.048+0000 7f428d400640 1 --2- 172.21.15.47:0/261596688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d52ed40 0x55dc1d5399c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.048+0000 7f428d400640 1 -- 172.21.15.47:0/261596688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d5291e0 msgr2=0x55dc1d538510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.048+0000 7f428d400640 1 --2- 172.21.15.47:0/261596688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d5291e0 0x55dc1d538510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.048+0000 7f428d400640 1 -- 172.21.15.47:0/261596688 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc1d539fb0 con 0x55dc1d529bd0 2024-03-21T11:35:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.049+0000 7f428d400640 1 --2- 172.21.15.47:0/261596688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d538a50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f427c003a90 tx=0x7f427c003e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67904b27c62ca264 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.049+0000 7f426fe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.049+0000 7f426fe00640 1 -- 172.21.15.47:0/261596688 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c005850 con 0x55dc1d529bd0 2024-03-21T11:35:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.049+0000 7f426fe00640 1 -- 172.21.15.47:0/261596688 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f427c0046c0 con 0x55dc1d529bd0 2024-03-21T11:35:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.049+0000 7f4297b8e740 1 -- 172.21.15.47:0/261596688 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f427c009520 con 0x55dc1d529bd0 2024-03-21T11:35:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.049+0000 7f426fe00640 1 -- 172.21.15.47:0/261596688 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f427c06c050 con 0x55dc1d529bd0 2024-03-21T11:35:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.049+0000 7f4297b8e740 10 client.29416 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.049+0000 7f4297b8e740 1 -- 172.21.15.47:0/261596688 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dc1d59ae30 con 0x55dc1d529bd0 2024-03-21T11:35:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.049+0000 7f426fe00640 1 -- 172.21.15.47:0/261596688 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f427c009520 con 0x55dc1d529bd0 2024-03-21T11:35:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.050+0000 7f426fe00640 1 -- 172.21.15.47:0/261596688 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f427c048470 con 0x55dc1d529bd0 2024-03-21T11:35:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.050+0000 7f4297b8e740 10 client.29416 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.050+0000 7f4297b8e740 10 client.29416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.050+0000 7f4297b8e740 20 client.29416 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.050+0000 7f4297b8e740 1 --2- 172.21.15.47:0/261596688 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc1d59b740 0x55dc1d59dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.050+0000 7f4297b8e740 4 client.29416 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:35:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.050+0000 7f4297b8e740 1 -- 172.21.15.47:0/261596688 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dc1d4960c0 con 0x55dc1d59b740 2024-03-21T11:35:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.050+0000 7f428e800640 1 --2- 172.21.15.47:0/261596688 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc1d59b740 0x55dc1d59dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.050+0000 7f428e800640 1 --2- 172.21.15.47:0/261596688 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc1d59b740 0x55dc1d59dbe0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe458fc718b4071 server_cookie=22559b5ebb40eb55 in_seq=0 out_seq=0 2024-03-21T11:35:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.050+0000 7f426fe00640 10 client.29416 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.283+0000 7f426fe00640 1 -- 172.21.15.47:0/261596688 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dc1d4960c0 con 0x55dc1d59b740 2024-03-21T11:35:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.283+0000 7f426fe00640 10 client.29416 handle_command_reply: tid=0 2024-03-21T11:35:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.283+0000 7f4297b8e740 1 client.29416 shutdown 2024-03-21T11:35:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.283+0000 7f4297b8e740 1 -- 172.21.15.47:0/261596688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 msgr2=0x55dc1d538a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.283+0000 7f4297b8e740 1 --2- 172.21.15.47:0/261596688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d538a50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f427c003a90 tx=0x7f427c003e30 comp rx=0 tx=0).stop 2024-03-21T11:35:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.283+0000 7f4297b8e740 1 -- 172.21.15.47:0/261596688 shutdown_connections 2024-03-21T11:35:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.283+0000 7f4297b8e740 1 --2- 172.21.15.47:0/261596688 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55dc1d59b740 0x55dc1d59dbe0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.283+0000 7f4297b8e740 1 --2- 172.21.15.47:0/261596688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc1d52ed40 0x55dc1d5399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.283+0000 7f4297b8e740 1 --2- 172.21.15.47:0/261596688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dc1d529bd0 0x55dc1d538a50 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.283+0000 7f426fe00640 0 client.29416 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.283+0000 7f4297b8e740 1 --2- 172.21.15.47:0/261596688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dc1d5291e0 0x55dc1d538510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.283+0000 7f4297b8e740 1 -- 172.21.15.47:0/261596688 >> 172.21.15.47:0/261596688 conn(0x55dc1d524750 msgr2=0x55dc1d5331e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.283+0000 7f4297b8e740 1 -- 172.21.15.47:0/261596688 shutdown_connections 2024-03-21T11:35:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.283+0000 7f4297b8e740 1 -- 172.21.15.47:0/261596688 wait complete. 2024-03-21T11:35:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.284+0000 7f4297b8e740 20 client.29416 trim_cache size 0 max 16384 2024-03-21T11:35:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:35:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:35:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:35:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:35:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.285+0000 7f4294e00640 1 -- 172.21.15.47:0/2613535632 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f42740a2fc0 msgr2=0x7f42740a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.285+0000 7f4294e00640 1 --2- 172.21.15.47:0/2613535632 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f42740a2fc0 0x7f42740a5480 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f42800cd580 tx=0x7f42800d3580 comp rx=0 tx=0).stop 2024-03-21T11:35:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.285+0000 7f4294e00640 1 -- 172.21.15.47:0/2613535632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42901005c0 msgr2=0x7f4290199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.285+0000 7f4294e00640 1 --2- 172.21.15.47:0/2613535632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42901005c0 0x7f4290199b90 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f428000e9c0 tx=0x7f428000edc0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.285+0000 7f4294e00640 1 -- 172.21.15.47:0/2613535632 shutdown_connections 2024-03-21T11:35:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.285+0000 7f4294e00640 1 --2- 172.21.15.47:0/2613535632 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f42740a2fc0 0x7f42740a5480 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.285+0000 7f4294e00640 1 --2- 172.21.15.47:0/2613535632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4290102170 0x7f42901a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.285+0000 7f4294e00640 1 --2- 172.21.15.47:0/2613535632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42901017b0 0x7f429019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.285+0000 7f4294e00640 1 --2- 172.21.15.47:0/2613535632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42901005c0 0x7f4290199b90 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.285+0000 7f4294e00640 1 -- 172.21.15.47:0/2613535632 >> 172.21.15.47:0/2613535632 conn(0x7f42900fbb30 msgr2=0x7f42900fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.285+0000 7f4294e00640 1 -- 172.21.15.47:0/2613535632 shutdown_connections 2024-03-21T11:35:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:00.285+0000 7f4294e00640 1 -- 172.21.15.47:0/2613535632 wait complete. 2024-03-21T11:35:00.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 880237 7.99 MB/sec execute 464 sec latency 134.765 ms 2024-03-21T11:35:00.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:00 smithi100 ceph-mon[36245]: pgmap v750: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.2 MiB/s wr, 255 op/s 2024-03-21T11:35:00.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:00 smithi102 ceph-mon[36748]: pgmap v750: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.2 MiB/s wr, 255 op/s 2024-03-21T11:35:01.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:00 smithi047 ceph-mon[31351]: pgmap v750: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.2 MiB/s wr, 255 op/s 2024-03-21T11:35:01.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 881050 7.99 MB/sec execute 465 sec latency 36.571 ms 2024-03-21T11:35:01.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:01 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:35:01.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:01 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:35:02.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:01 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:35:02.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 882773 7.98 MB/sec execute 466 sec latency 31.573 ms 2024-03-21T11:35:02.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:02 smithi100 ceph-mon[36245]: pgmap v751: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-03-21T11:35:02.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:35:02.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:35:02.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:35:02.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:02 smithi102 ceph-mon[36748]: pgmap v751: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-03-21T11:35:02.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:35:02.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:35:02.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:35:03.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:02 smithi047 ceph-mon[31351]: pgmap v751: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-03-21T11:35:03.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:35:03.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:35:03.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:35:03.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 884253 7.99 MB/sec execute 467 sec latency 16.172 ms 2024-03-21T11:35:04.562 INFO:tasks.workunit.client.0.smithi047.stdout: 1 885010 7.98 MB/sec execute 468 sec latency 24.526 ms 2024-03-21T11:35:04.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:04 smithi100 ceph-mon[36245]: pgmap v752: 129 pgs: 129 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 263 op/s 2024-03-21T11:35:04.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:04 smithi102 ceph-mon[36748]: pgmap v752: 129 pgs: 129 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 263 op/s 2024-03-21T11:35:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:04 smithi047 ceph-mon[31351]: pgmap v752: 129 pgs: 129 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 263 op/s 2024-03-21T11:35:05.563 INFO:tasks.workunit.client.0.smithi047.stdout: 1 886947 7.98 MB/sec execute 469 sec latency 13.512 ms 2024-03-21T11:35:06.565 INFO:tasks.workunit.client.0.smithi047.stdout: 1 888106 7.98 MB/sec execute 470 sec latency 11.862 ms 2024-03-21T11:35:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:06 smithi100 ceph-mon[36245]: pgmap v753: 129 pgs: 129 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-03-21T11:35:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:06 smithi102 ceph-mon[36748]: pgmap v753: 129 pgs: 129 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-03-21T11:35:07.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:06 smithi047 ceph-mon[31351]: pgmap v753: 129 pgs: 129 active+clean; 38 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-03-21T11:35:07.565 INFO:tasks.workunit.client.0.smithi047.stdout: 1 889204 7.98 MB/sec execute 471 sec latency 39.167 ms 2024-03-21T11:35:08.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:35:08.258 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:35:08.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.503+0000 7f5100e00640 1 -- 172.21.15.47:0/585843536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc107bc0 msgr2=0x7f50fc107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:08.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.503+0000 7f5100e00640 1 --2- 172.21.15.47:0/585843536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc107bc0 0x7f50fc107fe0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009900 tx=0x7f50ec02bc40 comp rx=0 tx=0).stop 2024-03-21T11:35:08.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.503+0000 7f5100e00640 1 -- 172.21.15.47:0/585843536 shutdown_connections 2024-03-21T11:35:08.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.503+0000 7f5100e00640 1 --2- 172.21.15.47:0/585843536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50fc109770 0x7f50fc110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.503+0000 7f5100e00640 1 --2- 172.21.15.47:0/585843536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50fc108db0 0x7f50fc109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.503+0000 7f5100e00640 1 --2- 172.21.15.47:0/585843536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc107bc0 0x7f50fc107fe0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.503+0000 7f5100e00640 1 -- 172.21.15.47:0/585843536 >> 172.21.15.47:0/585843536 conn(0x7f50fc076b70 msgr2=0x7f50fc078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:08.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.503+0000 7f5100e00640 1 -- 172.21.15.47:0/585843536 shutdown_connections 2024-03-21T11:35:08.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.503+0000 7f5100e00640 1 -- 172.21.15.47:0/585843536 wait complete. 2024-03-21T11:35:08.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.504+0000 7f5100e00640 1 Processor -- start 2024-03-21T11:35:08.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.504+0000 7f5100e00640 1 -- start start 2024-03-21T11:35:08.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.505+0000 7f5100e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50fc107bc0 0x7f50fc19ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.505+0000 7f5100e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50fc108db0 0x7f50fc19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.505+0000 7f5100e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc109770 0x7f50fc1a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.505+0000 7f5100e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc113860 con 0x7f50fc107bc0 2024-03-21T11:35:08.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.505+0000 7f5100e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1136e0 con 0x7f50fc108db0 2024-03-21T11:35:08.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.505+0000 7f5100e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1139e0 con 0x7f50fc109770 2024-03-21T11:35:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.505+0000 7f50fb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50fc107bc0 0x7f50fc19ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.505+0000 7f50fbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc109770 0x7f50fc1a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.505+0000 7f50f3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50fc108db0 0x7f50fc19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.505+0000 7f50fb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50fc107bc0 0x7f50fc19ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48210/0 (socket says 172.21.15.47:48210) 2024-03-21T11:35:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.505+0000 7f50fb400640 1 -- 172.21.15.47:0/422373619 learned_addr learned my addr 172.21.15.47:0/422373619 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.505+0000 7f50f3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50fc108db0 0x7f50fc19e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55886/0 (socket says 172.21.15.47:55886) 2024-03-21T11:35:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.505+0000 7f50fbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc109770 0x7f50fc1a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45686/0 (socket says 172.21.15.47:45686) 2024-03-21T11:35:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.506+0000 7f50fbe00640 1 -- 172.21.15.47:0/422373619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50fc108db0 msgr2=0x7f50fc19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.506+0000 7f50fbe00640 1 --2- 172.21.15.47:0/422373619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50fc108db0 0x7f50fc19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.506+0000 7f50fbe00640 1 -- 172.21.15.47:0/422373619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50fc107bc0 msgr2=0x7f50fc19ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.506+0000 7f50fbe00640 1 --2- 172.21.15.47:0/422373619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50fc107bc0 0x7f50fc19ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.506+0000 7f50fbe00640 1 -- 172.21.15.47:0/422373619 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc1a5af0 con 0x7f50fc109770 2024-03-21T11:35:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.506+0000 7f50f3e00640 1 --2- 172.21.15.47:0/422373619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50fc108db0 0x7f50fc19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:08.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.506+0000 7f50fbe00640 1 --2- 172.21.15.47:0/422373619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc109770 0x7f50fc1a5480 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f50f400ec50 tx=0x7f50f400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:08.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.506+0000 7f50f8c00640 1 -- 172.21.15.47:0/422373619 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f4003b70 con 0x7f50fc109770 2024-03-21T11:35:08.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.506+0000 7f50f8c00640 1 -- 172.21.15.47:0/422373619 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50f4003d10 con 0x7f50fc109770 2024-03-21T11:35:08.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.506+0000 7f5100e00640 1 -- 172.21.15.47:0/422373619 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50fc1a5de0 con 0x7f50fc109770 2024-03-21T11:35:08.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.506+0000 7f50f8c00640 1 -- 172.21.15.47:0/422373619 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f4010890 con 0x7f50fc109770 2024-03-21T11:35:08.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.506+0000 7f5100e00640 1 -- 172.21.15.47:0/422373619 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50fc1a62a0 con 0x7f50fc109770 2024-03-21T11:35:08.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.510+0000 7f50f8c00640 1 -- 172.21.15.47:0/422373619 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f50f4003fc0 con 0x7f50fc109770 2024-03-21T11:35:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.510+0000 7f510517e740 1 Processor -- start 2024-03-21T11:35:08.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.510+0000 7f510517e740 1 -- start start 2024-03-21T11:35:08.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.510+0000 7f510517e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26050 0x560c44a26470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.510+0000 7f510517e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26a40 0x560c44a2b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.510+0000 7f510517e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 0x560c44a2e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.510+0000 7f510517e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4498b250 con 0x560c44a2bbb0 2024-03-21T11:35:08.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.510+0000 7f510517e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c44933a30 con 0x560c44a26050 2024-03-21T11:35:08.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.510+0000 7f510517e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c449872f0 con 0x560c44a26a40 2024-03-21T11:35:08.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.511+0000 7f50f3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26050 0x560c44a26470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.511+0000 7f50fb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26a40 0x560c44a2b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.511+0000 7f50fb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26a40 0x560c44a2b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45692/0 (socket says 172.21.15.47:45692) 2024-03-21T11:35:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.511+0000 7f50fb400640 1 -- 172.21.15.47:0/2985405725 learned_addr learned my addr 172.21.15.47:0/2985405725 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.511+0000 7f50f3e00640 1 --2- 172.21.15.47:0/2985405725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 0x560c44a2e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.511+0000 7f50fb400640 1 -- 172.21.15.47:0/2985405725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26050 msgr2=0x560c44a26470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.511+0000 7f50fb400640 1 --2- 172.21.15.47:0/2985405725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26050 0x560c44a26470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.511+0000 7f50fb400640 1 -- 172.21.15.47:0/2985405725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 msgr2=0x560c44a2e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.511+0000 7f50fb400640 1 --2- 172.21.15.47:0/2985405725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 0x560c44a2e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.511+0000 7f50fb400640 1 -- 172.21.15.47:0/2985405725 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c4491e690 con 0x560c44a26a40 2024-03-21T11:35:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.511+0000 7f50f3e00640 1 --2- 172.21.15.47:0/2985405725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 0x560c44a2e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.512+0000 7f50fb400640 1 --2- 172.21.15.47:0/2985405725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26a40 0x560c44a2b540 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f50ec02c180 tx=0x7f50ec000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b3c94b07fb3e2b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.512+0000 7f50f0c00640 1 -- 172.21.15.47:0/2985405725 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec042070 con 0x560c44a26a40 2024-03-21T11:35:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.512+0000 7f50f8c00640 1 --2- 172.21.15.47:0/422373619 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50e00a2f20 0x7f50e00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.512+0000 7f50f0c00640 1 -- 172.21.15.47:0/2985405725 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50ec02cb90 con 0x560c44a26a40 2024-03-21T11:35:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.512+0000 7f50f0c00640 1 -- 172.21.15.47:0/2985405725 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec005150 con 0x560c44a26a40 2024-03-21T11:35:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.512+0000 7f50f8c00640 1 -- 172.21.15.47:0/422373619 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f50f40cfd80 con 0x7f50fc109770 2024-03-21T11:35:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.512+0000 7f50f3e00640 1 --2- 172.21.15.47:0/422373619 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50e00a2f20 0x7f50e00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.513+0000 7f510517e740 1 -- 172.21.15.47:0/2985405725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26a40 msgr2=0x560c44a2b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.513+0000 7f510517e740 1 --2- 172.21.15.47:0/2985405725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26a40 0x560c44a2b540 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f50ec02c180 tx=0x7f50ec000bc0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.513+0000 7f510517e740 1 -- 172.21.15.47:0/2985405725 shutdown_connections 2024-03-21T11:35:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.513+0000 7f510517e740 1 --2- 172.21.15.47:0/2985405725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 0x560c44a2e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.513+0000 7f510517e740 1 --2- 172.21.15.47:0/2985405725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26a40 0x560c44a2b540 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.513+0000 7f510517e740 1 --2- 172.21.15.47:0/2985405725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26050 0x560c44a26470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.513+0000 7f510517e740 1 -- 172.21.15.47:0/2985405725 >> 172.21.15.47:0/2985405725 conn(0x560c449ee200 msgr2=0x560c44a219c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.513+0000 7f510517e740 1 -- 172.21.15.47:0/2985405725 shutdown_connections 2024-03-21T11:35:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.513+0000 7f510517e740 1 -- 172.21.15.47:0/2985405725 wait complete. 2024-03-21T11:35:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.514+0000 7f50f3e00640 1 --2- 172.21.15.47:0/422373619 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50e00a2f20 0x7f50e00a53e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x560c44a272a0 tx=0x7f50e4013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.514+0000 7f510517e740 1 -- start start 2024-03-21T11:35:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.514+0000 7f510517e740 1 Processor -- start 2024-03-21T11:35:08.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f510517e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a30050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f510517e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a26a40 0x560c44a30590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f510517e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a2bbb0 0x560c44a37560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f510517e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c44a2eb00 con 0x560c44a26a40 2024-03-21T11:35:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f510517e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c44946e80 con 0x560c44a2bbb0 2024-03-21T11:35:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f510517e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c44a2ec80 con 0x560c44a26050 2024-03-21T11:35:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f510517e740 10 client.0 _ll_register_callbacks cb 0x560c44959130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f50fb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a30050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f50fb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a30050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45708/0 (socket says 172.21.15.47:45708) 2024-03-21T11:35:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f50fb400640 1 -- 172.21.15.47:0/2550973996 learned_addr learned my addr 172.21.15.47:0/2550973996 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f50fbe00640 1 --2- 172.21.15.47:0/2550973996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a2bbb0 0x560c44a37560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f50f3e00640 1 --2- 172.21.15.47:0/2550973996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a26a40 0x560c44a30590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f50fb400640 1 -- 172.21.15.47:0/2550973996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a2bbb0 msgr2=0x560c44a37560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f50fb400640 1 --2- 172.21.15.47:0/2550973996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a2bbb0 0x560c44a37560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.515+0000 7f50fb400640 1 -- 172.21.15.47:0/2550973996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a26a40 msgr2=0x560c44a30590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.516+0000 7f50fb400640 1 --2- 172.21.15.47:0/2550973996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a26a40 0x560c44a30590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.516+0000 7f50fb400640 1 -- 172.21.15.47:0/2550973996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c447697d0 con 0x560c44a26050 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.516+0000 7f50fb400640 1 --2- 172.21.15.47:0/2550973996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a30050 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f50ec005ab0 tx=0x7f50ec0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eca5cff74160197 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.516+0000 7f50e9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.516+0000 7f50e9600640 1 -- 172.21.15.47:0/2550973996 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec042070 con 0x560c44a26050 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.516+0000 7f50e9600640 1 -- 172.21.15.47:0/2550973996 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f50ec005100 con 0x560c44a26050 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.516+0000 7f510517e740 1 -- 172.21.15.47:0/2550973996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50ec02cb90 con 0x560c44a26050 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.516+0000 7f50e9600640 1 -- 172.21.15.47:0/2550973996 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50ec058050 con 0x560c44a26050 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.516+0000 7f510517e740 10 client.29428 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.516+0000 7f510517e740 1 -- 172.21.15.47:0/2550973996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560c4469f3f0 con 0x560c44a26050 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.517+0000 7f50e9600640 1 -- 172.21.15.47:0/2550973996 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50ec02cb90 con 0x560c44a26050 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.517+0000 7f50e9600640 1 -- 172.21.15.47:0/2550973996 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f50ec003a10 con 0x560c44a26050 2024-03-21T11:35:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.517+0000 7f510517e740 10 client.29428 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.517+0000 7f510517e740 10 client.29428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.517+0000 7f510517e740 20 client.29428 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.517+0000 7f510517e740 1 --2- 172.21.15.47:0/2550973996 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c44a3ba50 0x560c44a3de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.517+0000 7f510517e740 4 client.29428 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.517+0000 7f510517e740 1 -- 172.21.15.47:0/2550973996 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c44998790 con 0x560c44a3ba50 2024-03-21T11:35:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.517+0000 7f50f3e00640 1 --2- 172.21.15.47:0/2550973996 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c44a3ba50 0x560c44a3de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.518+0000 7f50f3e00640 1 --2- 172.21.15.47:0/2550973996 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c44a3ba50 0x560c44a3de40 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b93a5f686c1f20 server_cookie=5a2ad739fc8647da in_seq=0 out_seq=0 2024-03-21T11:35:08.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.518+0000 7f50e9600640 10 client.29428 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:08.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.519+0000 7f50e9600640 1 -- 172.21.15.47:0/2550973996 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560c44998790 con 0x560c44a3ba50 2024-03-21T11:35:08.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.519+0000 7f50e9600640 10 client.29428 handle_command_reply: tid=0 2024-03-21T11:35:08.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.519+0000 7f510517e740 1 client.29428 shutdown 2024-03-21T11:35:08.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.519+0000 7f510517e740 1 -- 172.21.15.47:0/2550973996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 msgr2=0x560c44a30050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:08.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.519+0000 7f510517e740 1 --2- 172.21.15.47:0/2550973996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a30050 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f50ec005ab0 tx=0x7f50ec0048f0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.520+0000 7f510517e740 1 -- 172.21.15.47:0/2550973996 shutdown_connections 2024-03-21T11:35:08.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.520+0000 7f510517e740 1 --2- 172.21.15.47:0/2550973996 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c44a3ba50 0x560c44a3de40 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.520+0000 7f510517e740 1 --2- 172.21.15.47:0/2550973996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a2bbb0 0x560c44a37560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.520+0000 7f510517e740 1 --2- 172.21.15.47:0/2550973996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a26a40 0x560c44a30590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.520+0000 7f510517e740 1 --2- 172.21.15.47:0/2550973996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a30050 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.520+0000 7f510517e740 1 -- 172.21.15.47:0/2550973996 >> 172.21.15.47:0/2550973996 conn(0x560c449ee200 msgr2=0x560c44a20bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.520+0000 7f50e9600640 0 client.29428 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.520+0000 7f510517e740 1 -- 172.21.15.47:0/2550973996 shutdown_connections 2024-03-21T11:35:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.520+0000 7f510517e740 1 -- 172.21.15.47:0/2550973996 wait complete. 2024-03-21T11:35:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.520+0000 7f510517e740 20 client.29428 trim_cache size 0 max 16384 2024-03-21T11:35:08.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.539+0000 7f510517e740 1 Processor -- start 2024-03-21T11:35:08.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.539+0000 7f510517e740 1 -- start start 2024-03-21T11:35:08.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f510517e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a20bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f510517e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26a40 0x560c44a374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f510517e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 0x560c44a25310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f510517e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1b0400 con 0x560c44a2bbb0 2024-03-21T11:35:08.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f510517e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1b1d80 con 0x560c44a26a40 2024-03-21T11:35:08.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f510517e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1b0a60 con 0x560c44a26050 2024-03-21T11:35:08.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f50fb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a20bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f50fb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a20bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45710/0 (socket says 172.21.15.47:45710) 2024-03-21T11:35:08.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f50fb400640 1 -- 172.21.15.47:0/2503955568 learned_addr learned my addr 172.21.15.47:0/2503955568 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:08.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f50f3e00640 1 --2- 172.21.15.47:0/2503955568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26a40 0x560c44a374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f50fbe00640 1 --2- 172.21.15.47:0/2503955568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 0x560c44a25310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f50fb400640 1 -- 172.21.15.47:0/2503955568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26a40 msgr2=0x560c44a374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:08.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f50fb400640 1 --2- 172.21.15.47:0/2503955568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26a40 0x560c44a374f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f50fb400640 1 -- 172.21.15.47:0/2503955568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 msgr2=0x560c44a25310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:08.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f50fb400640 1 --2- 172.21.15.47:0/2503955568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 0x560c44a25310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.540+0000 7f50fb400640 1 -- 172.21.15.47:0/2503955568 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c4465f0f0 con 0x560c44a26050 2024-03-21T11:35:08.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.541+0000 7f50fb400640 1 --2- 172.21.15.47:0/2503955568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a20bc0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f50ec02c180 tx=0x7f50ec009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d608926e416c4b7a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:08.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.541+0000 7f50ebe00640 1 -- 172.21.15.47:0/2503955568 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec055070 con 0x560c44a26050 2024-03-21T11:35:08.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.541+0000 7f50ebe00640 1 -- 172.21.15.47:0/2503955568 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50ec004960 con 0x560c44a26050 2024-03-21T11:35:08.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.541+0000 7f50ebe00640 1 -- 172.21.15.47:0/2503955568 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec042070 con 0x560c44a26050 2024-03-21T11:35:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.542+0000 7f510517e740 1 -- 172.21.15.47:0/2503955568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 msgr2=0x560c44a20bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.542+0000 7f510517e740 1 --2- 172.21.15.47:0/2503955568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a20bc0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f50ec02c180 tx=0x7f50ec009780 comp rx=0 tx=0).stop 2024-03-21T11:35:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.542+0000 7f510517e740 1 -- 172.21.15.47:0/2503955568 shutdown_connections 2024-03-21T11:35:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.542+0000 7f510517e740 1 --2- 172.21.15.47:0/2503955568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 0x560c44a25310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.542+0000 7f510517e740 1 --2- 172.21.15.47:0/2503955568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26a40 0x560c44a374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.542+0000 7f510517e740 1 --2- 172.21.15.47:0/2503955568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a20bc0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.542+0000 7f510517e740 1 -- 172.21.15.47:0/2503955568 >> 172.21.15.47:0/2503955568 conn(0x560c449ee200 msgr2=0x560c44a24c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.542+0000 7f510517e740 1 -- 172.21.15.47:0/2503955568 shutdown_connections 2024-03-21T11:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.542+0000 7f510517e740 1 -- 172.21.15.47:0/2503955568 wait complete. 2024-03-21T11:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.542+0000 7f510517e740 1 -- start start 2024-03-21T11:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.543+0000 7f510517e740 1 Processor -- start 2024-03-21T11:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.543+0000 7f510517e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a34e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.543+0000 7f510517e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26a40 0x560c44a35360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.543+0000 7f510517e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 0x560c44a36320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.543+0000 7f510517e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1b1720 con 0x560c44a2bbb0 2024-03-21T11:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.543+0000 7f510517e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1b10c0 con 0x560c44a26a40 2024-03-21T11:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.543+0000 7f510517e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1ae5c0 con 0x560c44a26050 2024-03-21T11:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.543+0000 7f510517e740 10 client.0 _ll_register_callbacks cb 0x7f50e4009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.543+0000 7f50fb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a34e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.543+0000 7f50f3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26a40 0x560c44a35360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.543+0000 7f50fb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a34e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45714/0 (socket says 172.21.15.47:45714) 2024-03-21T11:35:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.543+0000 7f50fb400640 1 -- 172.21.15.47:0/830233508 learned_addr learned my addr 172.21.15.47:0/830233508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.543+0000 7f50fbe00640 1 --2- 172.21.15.47:0/830233508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 0x560c44a36320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.544+0000 7f50fb400640 1 -- 172.21.15.47:0/830233508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26a40 msgr2=0x560c44a35360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.544+0000 7f50fb400640 1 --2- 172.21.15.47:0/830233508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26a40 0x560c44a35360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.544+0000 7f50fb400640 1 -- 172.21.15.47:0/830233508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 msgr2=0x560c44a36320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.544+0000 7f50fb400640 1 --2- 172.21.15.47:0/830233508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 0x560c44a36320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.544+0000 7f50fb400640 1 -- 172.21.15.47:0/830233508 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c44a36910 con 0x560c44a26050 2024-03-21T11:35:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.544+0000 7f50f3e00640 1 --2- 172.21.15.47:0/830233508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26a40 0x560c44a35360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.544+0000 7f50fb400640 1 --2- 172.21.15.47:0/830233508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a34e20 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f50ec004d40 tx=0x7f50ec0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f949db065a717539 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.544+0000 7f50e9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.544+0000 7f50e9600640 1 -- 172.21.15.47:0/830233508 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec042070 con 0x560c44a26050 2024-03-21T11:35:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.544+0000 7f50e9600640 1 -- 172.21.15.47:0/830233508 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f50ec003b30 con 0x560c44a26050 2024-03-21T11:35:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.544+0000 7f510517e740 1 -- 172.21.15.47:0/830233508 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50ec004960 con 0x560c44a26050 2024-03-21T11:35:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.545+0000 7f50e9600640 1 -- 172.21.15.47:0/830233508 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50ec055040 con 0x560c44a26050 2024-03-21T11:35:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.545+0000 7f510517e740 10 client.29434 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.545+0000 7f510517e740 1 -- 172.21.15.47:0/830233508 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560c44a97440 con 0x560c44a26050 2024-03-21T11:35:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.545+0000 7f50e9600640 1 -- 172.21.15.47:0/830233508 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50ec004960 con 0x560c44a26050 2024-03-21T11:35:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.545+0000 7f50e9600640 1 -- 172.21.15.47:0/830233508 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f50ec047690 con 0x560c44a26050 2024-03-21T11:35:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.545+0000 7f510517e740 10 client.29434 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.545+0000 7f510517e740 10 client.29434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.545+0000 7f510517e740 20 client.29434 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.545+0000 7f510517e740 1 --2- 172.21.15.47:0/830233508 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c44a97b00 0x560c44a99f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.545+0000 7f510517e740 4 client.29434 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:35:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.545+0000 7f510517e740 1 -- 172.21.15.47:0/830233508 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560c44992ae0 con 0x560c44a97b00 2024-03-21T11:35:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.545+0000 7f50f3e00640 1 --2- 172.21.15.47:0/830233508 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c44a97b00 0x560c44a99f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.546+0000 7f50f3e00640 1 --2- 172.21.15.47:0/830233508 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c44a97b00 0x560c44a99f20 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f00a14aa0b80bf7f server_cookie=56f246491c010be2 in_seq=0 out_seq=0 2024-03-21T11:35:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.546+0000 7f50e9600640 10 client.29434 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.546+0000 7f50e9600640 1 -- 172.21.15.47:0/830233508 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560c44992ae0 con 0x560c44a97b00 2024-03-21T11:35:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.546+0000 7f50e9600640 10 client.29434 handle_command_reply: tid=0 2024-03-21T11:35:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.546+0000 7f510517e740 1 client.29434 shutdown 2024-03-21T11:35:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.546+0000 7f510517e740 1 -- 172.21.15.47:0/830233508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 msgr2=0x560c44a34e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.546+0000 7f510517e740 1 --2- 172.21.15.47:0/830233508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a34e20 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f50ec004d40 tx=0x7f50ec0048f0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.547+0000 7f510517e740 1 -- 172.21.15.47:0/830233508 shutdown_connections 2024-03-21T11:35:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.547+0000 7f510517e740 1 --2- 172.21.15.47:0/830233508 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560c44a97b00 0x560c44a99f20 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.547+0000 7f510517e740 1 --2- 172.21.15.47:0/830233508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560c44a2bbb0 0x560c44a36320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.547+0000 7f510517e740 1 --2- 172.21.15.47:0/830233508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c44a26a40 0x560c44a35360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.547+0000 7f510517e740 1 --2- 172.21.15.47:0/830233508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560c44a26050 0x560c44a34e20 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.547+0000 7f510517e740 1 -- 172.21.15.47:0/830233508 >> 172.21.15.47:0/830233508 conn(0x560c449ee200 msgr2=0x560c44a20a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:08.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.547+0000 7f50e9600640 0 client.29434 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:08.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.547+0000 7f510517e740 1 -- 172.21.15.47:0/830233508 shutdown_connections 2024-03-21T11:35:08.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.547+0000 7f510517e740 1 -- 172.21.15.47:0/830233508 wait complete. 2024-03-21T11:35:08.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.547+0000 7f510517e740 20 client.29434 trim_cache size 0 max 16384 2024-03-21T11:35:08.552 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:35:08.552 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:35:08.552 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:35:08.552 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:35:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.547+0000 7f5100e00640 1 -- 172.21.15.47:0/422373619 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50e00a2f20 msgr2=0x7f50e00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.548+0000 7f5100e00640 1 --2- 172.21.15.47:0/422373619 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50e00a2f20 0x7f50e00a53e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x560c44a272a0 tx=0x7f50e4013040 comp rx=0 tx=0).stop 2024-03-21T11:35:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.548+0000 7f5100e00640 1 -- 172.21.15.47:0/422373619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc109770 msgr2=0x7f50fc1a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.548+0000 7f5100e00640 1 --2- 172.21.15.47:0/422373619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc109770 0x7f50fc1a5480 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f50f400ec50 tx=0x7f50f400c6a0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.548+0000 7f5100e00640 1 -- 172.21.15.47:0/422373619 shutdown_connections 2024-03-21T11:35:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.548+0000 7f5100e00640 1 --2- 172.21.15.47:0/422373619 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50e00a2f20 0x7f50e00a53e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.548+0000 7f5100e00640 1 --2- 172.21.15.47:0/422373619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50fc109770 0x7f50fc1a5480 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.548+0000 7f5100e00640 1 --2- 172.21.15.47:0/422373619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50fc108db0 0x7f50fc19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.548+0000 7f5100e00640 1 --2- 172.21.15.47:0/422373619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50fc107bc0 0x7f50fc19ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.548+0000 7f5100e00640 1 -- 172.21.15.47:0/422373619 >> 172.21.15.47:0/422373619 conn(0x7f50fc076b70 msgr2=0x7f50fc10c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.548+0000 7f5100e00640 1 -- 172.21.15.47:0/422373619 shutdown_connections 2024-03-21T11:35:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.548+0000 7f5100e00640 1 -- 172.21.15.47:0/422373619 wait complete. 2024-03-21T11:35:08.565 INFO:tasks.workunit.client.0.smithi047.stdout: 1 891182 7.99 MB/sec execute 472 sec latency 16.197 ms 2024-03-21T11:35:08.566 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:35:08.566 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:35:08.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.775+0000 7f1ab6600640 1 -- 172.21.15.47:0/3690768113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab01028b0 msgr2=0x7f1ab0102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.775+0000 7f1ab6600640 1 --2- 172.21.15.47:0/3690768113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab01028b0 0x7f1ab0102d30 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009900 tx=0x7f1a9c02bc50 comp rx=0 tx=0).stop 2024-03-21T11:35:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.776+0000 7f1ab6600640 1 -- 172.21.15.47:0/3690768113 shutdown_connections 2024-03-21T11:35:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.776+0000 7f1ab6600640 1 --2- 172.21.15.47:0/3690768113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ab0103270 0x7f1ab0109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.776+0000 7f1ab6600640 1 --2- 172.21.15.47:0/3690768113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab01028b0 0x7f1ab0102d30 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.776+0000 7f1ab6600640 1 --2- 172.21.15.47:0/3690768113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ab01016c0 0x7f1ab0101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.776+0000 7f1ab6600640 1 -- 172.21.15.47:0/3690768113 >> 172.21.15.47:0/3690768113 conn(0x7f1ab00fcc50 msgr2=0x7f1ab00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.776+0000 7f1ab6600640 1 -- 172.21.15.47:0/3690768113 shutdown_connections 2024-03-21T11:35:08.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.776+0000 7f1ab6600640 1 -- 172.21.15.47:0/3690768113 wait complete. 2024-03-21T11:35:08.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.777+0000 7f1ab6600640 1 Processor -- start 2024-03-21T11:35:08.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.777+0000 7f1ab6600640 1 -- start start 2024-03-21T11:35:08.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1ab6600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ab01016c0 0x7f1ab0075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1ab6600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ab01028b0 0x7f1ab0073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1ab6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab0103270 0x7f1ab0073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1ab6600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab010d390 con 0x7f1ab01028b0 2024-03-21T11:35:08.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1ab6600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab010d210 con 0x7f1ab01016c0 2024-03-21T11:35:08.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1ab6600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab010d510 con 0x7f1ab0103270 2024-03-21T11:35:08.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1aaf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ab01016c0 0x7f1ab0075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1aaf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ab01016c0 0x7f1ab0075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55928/0 (socket says 172.21.15.47:55928) 2024-03-21T11:35:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1aaf400640 1 -- 172.21.15.47:0/1892539867 learned_addr learned my addr 172.21.15.47:0/1892539867 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1aaea00640 1 --2- 172.21.15.47:0/1892539867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ab01028b0 0x7f1ab0073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1aafe00640 1 --2- 172.21.15.47:0/1892539867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab0103270 0x7f1ab0073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1aaf400640 1 -- 172.21.15.47:0/1892539867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab0103270 msgr2=0x7f1ab0073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1aaf400640 1 --2- 172.21.15.47:0/1892539867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab0103270 0x7f1ab0073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1aaf400640 1 -- 172.21.15.47:0/1892539867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ab01028b0 msgr2=0x7f1ab0073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1aaf400640 1 --2- 172.21.15.47:0/1892539867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ab01028b0 0x7f1ab0073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:08.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.778+0000 7f1aaf400640 1 -- 172.21.15.47:0/1892539867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab0074860 con 0x7f1ab01016c0 2024-03-21T11:35:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.779+0000 7f1aaf400640 1 --2- 172.21.15.47:0/1892539867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ab01016c0 0x7f1ab0075450 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000e9c0 tx=0x7f1aa000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.779+0000 7f1aa7e00640 1 -- 172.21.15.47:0/1892539867 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa000cd80 con 0x7f1ab01016c0 2024-03-21T11:35:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.779+0000 7f1aa7e00640 1 -- 172.21.15.47:0/1892539867 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1aa00035f0 con 0x7f1ab01016c0 2024-03-21T11:35:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.779+0000 7f1aa7e00640 1 -- 172.21.15.47:0/1892539867 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0010690 con 0x7f1ab01016c0 2024-03-21T11:35:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.779+0000 7f1ab6600640 1 -- 172.21.15.47:0/1892539867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab01a1b20 con 0x7f1ab01016c0 2024-03-21T11:35:08.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.779+0000 7f1ab6600640 1 -- 172.21.15.47:0/1892539867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab01a1ff0 con 0x7f1ab01016c0 2024-03-21T11:35:08.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.781+0000 7f1ab6600640 1 -- 172.21.15.47:0/1892539867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab00688e0 con 0x7f1ab01016c0 2024-03-21T11:35:08.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.783+0000 7f1aa7e00640 1 -- 172.21.15.47:0/1892539867 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1aa00037a0 con 0x7f1ab01016c0 2024-03-21T11:35:08.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.784+0000 7f1aa7e00640 1 --2- 172.21.15.47:0/1892539867 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a940a2fa0 0x7f1a940a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:08.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.784+0000 7f1aa7e00640 1 -- 172.21.15.47:0/1892539867 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1aa00cfb10 con 0x7f1ab01016c0 2024-03-21T11:35:08.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.784+0000 7f1aaea00640 1 --2- 172.21.15.47:0/1892539867 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a940a2fa0 0x7f1a940a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:08.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.785+0000 7f1aaea00640 1 --2- 172.21.15.47:0/1892539867 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a940a2fa0 0x7f1a940a5460 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c02c260 tx=0x7f1a9c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:08.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:08.789+0000 7f1aa7e00640 1 -- 172.21.15.47:0/1892539867 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1aa009c550 con 0x7f1ab01016c0 2024-03-21T11:35:08.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:08 smithi102 ceph-mon[36748]: pgmap v754: 129 pgs: 129 active+clean; 40 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-03-21T11:35:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:08 smithi047 ceph-mon[31351]: pgmap v754: 129 pgs: 129 active+clean; 40 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-03-21T11:35:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.077+0000 7f1ab6600640 1 -- 172.21.15.47:0/1892539867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ab00622a0 con 0x7f1ab01016c0 2024-03-21T11:35:09.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.079+0000 7f1aa7e00640 1 -- 172.21.15.47:0/1892539867 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f1aa009be50 con 0x7f1ab01016c0 2024-03-21T11:35:09.080 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:35:09.080 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:35:09.080 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:35:09.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.082+0000 7f1ab6600640 1 -- 172.21.15.47:0/1892539867 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a940a2fa0 msgr2=0x7f1a940a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.082+0000 7f1ab6600640 1 --2- 172.21.15.47:0/1892539867 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a940a2fa0 0x7f1a940a5460 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c02c260 tx=0x7f1a9c000bc0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.082+0000 7f1ab6600640 1 -- 172.21.15.47:0/1892539867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ab01016c0 msgr2=0x7f1ab0075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.082+0000 7f1ab6600640 1 --2- 172.21.15.47:0/1892539867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ab01016c0 0x7f1ab0075450 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000e9c0 tx=0x7f1aa000ee80 comp rx=0 tx=0).stop 2024-03-21T11:35:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.082+0000 7f1ab6600640 1 -- 172.21.15.47:0/1892539867 shutdown_connections 2024-03-21T11:35:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.082+0000 7f1ab6600640 1 --2- 172.21.15.47:0/1892539867 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a940a2fa0 0x7f1a940a5460 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.082+0000 7f1ab6600640 1 --2- 172.21.15.47:0/1892539867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ab0103270 0x7f1ab0073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.082+0000 7f1ab6600640 1 --2- 172.21.15.47:0/1892539867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ab01028b0 0x7f1ab0073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.082+0000 7f1ab6600640 1 --2- 172.21.15.47:0/1892539867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ab01016c0 0x7f1ab0075450 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.082+0000 7f1ab6600640 1 -- 172.21.15.47:0/1892539867 >> 172.21.15.47:0/1892539867 conn(0x7f1ab00fcc50 msgr2=0x7f1ab0105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.082+0000 7f1ab6600640 1 -- 172.21.15.47:0/1892539867 shutdown_connections 2024-03-21T11:35:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.083+0000 7f1ab6600640 1 -- 172.21.15.47:0/1892539867 wait complete. 2024-03-21T11:35:09.104 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:35:09.104 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:35:09.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:08 smithi100 ceph-mon[36245]: pgmap v754: 129 pgs: 129 active+clean; 40 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-03-21T11:35:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.339+0000 7f50dd000640 1 -- 172.21.15.47:0/686787521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50d81016c0 msgr2=0x7f50d8101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.339+0000 7f50dd000640 1 --2- 172.21.15.47:0/686787521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50d81016c0 0x7f50d8101ae0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009be0 tx=0x7f50c802bef0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.339+0000 7f50dd000640 1 -- 172.21.15.47:0/686787521 shutdown_connections 2024-03-21T11:35:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.339+0000 7f50dd000640 1 --2- 172.21.15.47:0/686787521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50d8103270 0x7f50d8109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.339+0000 7f50dd000640 1 --2- 172.21.15.47:0/686787521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50d81028b0 0x7f50d8102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.339+0000 7f50dd000640 1 --2- 172.21.15.47:0/686787521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50d81016c0 0x7f50d8101ae0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.339+0000 7f50dd000640 1 -- 172.21.15.47:0/686787521 >> 172.21.15.47:0/686787521 conn(0x7f50d80fcc50 msgr2=0x7f50d80ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.340+0000 7f50dd000640 1 -- 172.21.15.47:0/686787521 shutdown_connections 2024-03-21T11:35:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.340+0000 7f50dd000640 1 -- 172.21.15.47:0/686787521 wait complete. 2024-03-21T11:35:09.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.341+0000 7f50dd000640 1 Processor -- start 2024-03-21T11:35:09.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.341+0000 7f50dd000640 1 -- start start 2024-03-21T11:35:09.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.342+0000 7f50dd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50d81016c0 0x7f50d819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.342+0000 7f50dd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50d81028b0 0x7f50d819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.342+0000 7f50dd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50d8103270 0x7f50d81a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.342+0000 7f50dd000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d810d360 con 0x7f50d8103270 2024-03-21T11:35:09.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.342+0000 7f50dd000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d810d1e0 con 0x7f50d81016c0 2024-03-21T11:35:09.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.342+0000 7f50dd000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d810d4e0 con 0x7f50d81028b0 2024-03-21T11:35:09.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.342+0000 7f50d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50d81028b0 0x7f50d819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.342+0000 7f50d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50d81028b0 0x7f50d819e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49842/0 (socket says 172.21.15.47:49842) 2024-03-21T11:35:09.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.342+0000 7f50d5400640 1 -- 172.21.15.47:0/3129779222 learned_addr learned my addr 172.21.15.47:0/3129779222 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.342+0000 7f50d5e00640 1 --2- 172.21.15.47:0/3129779222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50d81016c0 0x7f50d819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.342+0000 7f50d6800640 1 --2- 172.21.15.47:0/3129779222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50d8103270 0x7f50d81a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.343+0000 7f50d5e00640 1 -- 172.21.15.47:0/3129779222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50d81028b0 msgr2=0x7f50d819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.343+0000 7f50d5e00640 1 --2- 172.21.15.47:0/3129779222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50d81028b0 0x7f50d819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.343+0000 7f50d5e00640 1 -- 172.21.15.47:0/3129779222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50d8103270 msgr2=0x7f50d81a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.343+0000 7f50d5e00640 1 --2- 172.21.15.47:0/3129779222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50d8103270 0x7f50d81a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.343+0000 7f50d5e00640 1 -- 172.21.15.47:0/3129779222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d81a5c70 con 0x7f50d81016c0 2024-03-21T11:35:09.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.343+0000 7f50d5e00640 1 --2- 172.21.15.47:0/3129779222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50d81016c0 0x7f50d819e050 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009b30 tx=0x7f50c8033580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.343+0000 7f50cea00640 1 -- 172.21.15.47:0/3129779222 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8043020 con 0x7f50d81016c0 2024-03-21T11:35:09.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.343+0000 7f50cea00640 1 -- 172.21.15.47:0/3129779222 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50c8003e50 con 0x7f50d81016c0 2024-03-21T11:35:09.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.344+0000 7f50dd000640 1 -- 172.21.15.47:0/3129779222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d81a5f00 con 0x7f50d81016c0 2024-03-21T11:35:09.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.344+0000 7f50cea00640 1 -- 172.21.15.47:0/3129779222 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8034690 con 0x7f50d81016c0 2024-03-21T11:35:09.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.344+0000 7f50dd000640 1 -- 172.21.15.47:0/3129779222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d81a6370 con 0x7f50d81016c0 2024-03-21T11:35:09.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.349+0000 7f50cea00640 1 -- 172.21.15.47:0/3129779222 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f50c8046020 con 0x7f50d81016c0 2024-03-21T11:35:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.349+0000 7f50dfc2b740 1 Processor -- start 2024-03-21T11:35:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.349+0000 7f50dfc2b740 1 -- start start 2024-03-21T11:35:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.349+0000 7f50dfc2b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 0x55f42c682c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.349+0000 7f50dfc2b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6839e0 0x55f42c683e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.349+0000 7f50dfc2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6843a0 0x55f42c68ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.349+0000 7f50dfc2b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f42c5e72c0 con 0x55f42c6839e0 2024-03-21T11:35:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.349+0000 7f50dfc2b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f42c58fc90 con 0x55f42c682810 2024-03-21T11:35:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.349+0000 7f50dfc2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f42c5e3360 con 0x55f42c6843a0 2024-03-21T11:35:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.350+0000 7f50d6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 0x55f42c682c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.350+0000 7f50d6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 0x55f42c682c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54570/0 (socket says 172.21.15.47:54570) 2024-03-21T11:35:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.350+0000 7f50d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6843a0 0x55f42c68ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.350+0000 7f50d6800640 1 -- 172.21.15.47:0/4014257304 learned_addr learned my addr 172.21.15.47:0/4014257304 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.350+0000 7f50d5e00640 1 --2- 172.21.15.47:0/4014257304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6839e0 0x55f42c683e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.350+0000 7f50d6800640 1 -- 172.21.15.47:0/4014257304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6843a0 msgr2=0x55f42c68ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.350+0000 7f50d6800640 1 --2- 172.21.15.47:0/4014257304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6843a0 0x55f42c68ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.350+0000 7f50d6800640 1 -- 172.21.15.47:0/4014257304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6839e0 msgr2=0x55f42c683e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.350+0000 7f50d6800640 1 --2- 172.21.15.47:0/4014257304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6839e0 0x55f42c683e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.350+0000 7f50d6800640 1 -- 172.21.15.47:0/4014257304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f42c5b4080 con 0x55f42c682810 2024-03-21T11:35:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.350+0000 7f50cea00640 1 --2- 172.21.15.47:0/3129779222 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50bc0a3070 0x7f50bc0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.351+0000 7f50d6800640 1 --2- 172.21.15.47:0/4014257304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 0x55f42c682c30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009ca0 tx=0x7f50d0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57a59cb1150f3eaa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.351+0000 7f50cea00640 1 -- 172.21.15.47:0/3129779222 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f50c80f1e80 con 0x7f50d81016c0 2024-03-21T11:35:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.351+0000 7f50c3400640 1 -- 172.21.15.47:0/4014257304 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d001e070 con 0x55f42c682810 2024-03-21T11:35:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.351+0000 7f50c3400640 1 -- 172.21.15.47:0/4014257304 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50d0019910 con 0x55f42c682810 2024-03-21T11:35:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.351+0000 7f50c3400640 1 -- 172.21.15.47:0/4014257304 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d000ea50 con 0x55f42c682810 2024-03-21T11:35:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.351+0000 7f50d5e00640 1 --2- 172.21.15.47:0/3129779222 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50bc0a3070 0x7f50bc0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.352+0000 7f50dfc2b740 1 -- 172.21.15.47:0/4014257304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 msgr2=0x55f42c682c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.352+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/4014257304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 0x55f42c682c30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009ca0 tx=0x7f50d0011810 comp rx=0 tx=0).stop 2024-03-21T11:35:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.352+0000 7f50d5e00640 1 --2- 172.21.15.47:0/3129779222 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50bc0a3070 0x7f50bc0a5530 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f50c80be150 tx=0x7f50c8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.352+0000 7f50dfc2b740 1 -- 172.21.15.47:0/4014257304 shutdown_connections 2024-03-21T11:35:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.352+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/4014257304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6843a0 0x55f42c68ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.352+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/4014257304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6839e0 0x55f42c683e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.352+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/4014257304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 0x55f42c682c30 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.352+0000 7f50dfc2b740 1 -- 172.21.15.47:0/4014257304 >> 172.21.15.47:0/4014257304 conn(0x55f42c67dc00 msgr2=0x55f42c67e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.352+0000 7f50dfc2b740 1 -- 172.21.15.47:0/4014257304 shutdown_connections 2024-03-21T11:35:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.352+0000 7f50dfc2b740 1 -- 172.21.15.47:0/4014257304 wait complete. 2024-03-21T11:35:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.352+0000 7f50dfc2b740 1 -- start start 2024-03-21T11:35:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.353+0000 7f50dfc2b740 1 Processor -- start 2024-03-21T11:35:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.353+0000 7f50dfc2b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 0x55f42c68cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.353+0000 7f50dfc2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c68cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.353+0000 7f50dfc2b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6843a0 0x55f42c694030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.353+0000 7f50dfc2b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f42c68b3b0 con 0x55f42c6843a0 2024-03-21T11:35:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.353+0000 7f50dfc2b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f42c5a30d0 con 0x55f42c682810 2024-03-21T11:35:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.353+0000 7f50dfc2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f42c68b530 con 0x55f42c6839e0 2024-03-21T11:35:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.353+0000 7f50dfc2b740 10 client.0 _ll_register_callbacks cb 0x55f42c5a7e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.353+0000 7f50d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c68cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.353+0000 7f50d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c68cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49850/0 (socket says 172.21.15.47:49850) 2024-03-21T11:35:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.354+0000 7f50d5400640 1 -- 172.21.15.47:0/539076166 learned_addr learned my addr 172.21.15.47:0/539076166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.354+0000 7f50d6800640 1 --2- 172.21.15.47:0/539076166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 0x55f42c68cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.354+0000 7f50d6800640 1 --2- 172.21.15.47:0/539076166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6843a0 0x55f42c694030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.354+0000 7f50d5400640 1 -- 172.21.15.47:0/539076166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 msgr2=0x55f42c68cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.354+0000 7f50d5400640 1 --2- 172.21.15.47:0/539076166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 0x55f42c68cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.354+0000 7f50d5400640 1 -- 172.21.15.47:0/539076166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6843a0 msgr2=0x55f42c694030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.354+0000 7f50d5400640 1 --2- 172.21.15.47:0/539076166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6843a0 0x55f42c694030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.354+0000 7f50d5400640 1 -- 172.21.15.47:0/539076166 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f42c2fb3f0 con 0x55f42c6839e0 2024-03-21T11:35:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.354+0000 7f50d6800640 1 --2- 172.21.15.47:0/539076166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6843a0 0x55f42c694030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:35:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.354+0000 7f50d5400640 1 --2- 172.21.15.47:0/539076166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c68cff0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f50c400ba90 tx=0x7f50c400bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60f02601e8df6c9c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.354+0000 7f50b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.355+0000 7f50b7e00640 1 -- 172.21.15.47:0/539076166 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c401b470 con 0x55f42c6839e0 2024-03-21T11:35:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.355+0000 7f50dfc2b740 1 -- 172.21.15.47:0/539076166 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50d0019910 con 0x55f42c6839e0 2024-03-21T11:35:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.355+0000 7f50b7e00640 1 -- 172.21.15.47:0/539076166 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f50c4011440 con 0x55f42c6839e0 2024-03-21T11:35:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.355+0000 7f50b7e00640 1 -- 172.21.15.47:0/539076166 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50c4011640 con 0x55f42c6839e0 2024-03-21T11:35:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.355+0000 7f50dfc2b740 10 client.29452 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.355+0000 7f50dfc2b740 1 -- 172.21.15.47:0/539076166 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f42c3c57d0 con 0x55f42c6839e0 2024-03-21T11:35:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.355+0000 7f50b7e00640 1 -- 172.21.15.47:0/539076166 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50d0019910 con 0x55f42c6839e0 2024-03-21T11:35:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.355+0000 7f50b7e00640 1 -- 172.21.15.47:0/539076166 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f50c401bb30 con 0x55f42c6839e0 2024-03-21T11:35:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.355+0000 7f50dfc2b740 10 client.29452 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.355+0000 7f50dfc2b740 10 client.29452 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:35:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.355+0000 7f50dfc2b740 20 client.29452 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.355+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/539076166 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f42c698680 0x55f42c69aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.356+0000 7f50dfc2b740 4 client.29452 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.356+0000 7f50dfc2b740 1 -- 172.21.15.47:0/539076166 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f42c5f4800 con 0x55f42c698680 2024-03-21T11:35:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.356+0000 7f50d6800640 1 --2- 172.21.15.47:0/539076166 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f42c698680 0x55f42c69aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.356+0000 7f50d6800640 1 --2- 172.21.15.47:0/539076166 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f42c698680 0x55f42c69aaa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=36d912616f27f03e server_cookie=fb465778211bc637 in_seq=0 out_seq=0 2024-03-21T11:35:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.356+0000 7f50b7e00640 10 client.29452 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:35:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.357+0000 7f50b7e00640 1 -- 172.21.15.47:0/539076166 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f42c5f4800 con 0x55f42c698680 2024-03-21T11:35:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.357+0000 7f50b7e00640 10 client.29452 handle_command_reply: tid=0 2024-03-21T11:35:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.357+0000 7f50dfc2b740 1 client.29452 shutdown 2024-03-21T11:35:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.357+0000 7f50dfc2b740 1 -- 172.21.15.47:0/539076166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 msgr2=0x55f42c68cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.357+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/539076166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c68cff0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f50c400ba90 tx=0x7f50c400bf50 comp rx=0 tx=0).stop 2024-03-21T11:35:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.357+0000 7f50dfc2b740 1 -- 172.21.15.47:0/539076166 shutdown_connections 2024-03-21T11:35:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.357+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/539076166 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f42c698680 0x55f42c69aaa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.357+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/539076166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6843a0 0x55f42c694030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.357+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/539076166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c68cff0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.357+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/539076166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 0x55f42c68cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.357+0000 7f50b7e00640 0 client.29452 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:35:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.357+0000 7f50dfc2b740 1 -- 172.21.15.47:0/539076166 >> 172.21.15.47:0/539076166 conn(0x55f42c67dc00 msgr2=0x55f42c67ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.357+0000 7f50dfc2b740 1 -- 172.21.15.47:0/539076166 shutdown_connections 2024-03-21T11:35:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.357+0000 7f50dfc2b740 1 -- 172.21.15.47:0/539076166 wait complete. 2024-03-21T11:35:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.358+0000 7f50dfc2b740 20 client.29452 trim_cache size 0 max 16384 2024-03-21T11:35:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.373+0000 7f50dfc2b740 1 Processor -- start 2024-03-21T11:35:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.373+0000 7f50dfc2b740 1 -- start start 2024-03-21T11:35:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.373+0000 7f50dfc2b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c682810 0x55f42c686c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.373+0000 7f50dfc2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c693ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.373+0000 7f50dfc2b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c6843a0 0x55f42c687150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.373+0000 7f50dfc2b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81b0680 con 0x55f42c682810 2024-03-21T11:35:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.373+0000 7f50dfc2b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81b2000 con 0x55f42c6843a0 2024-03-21T11:35:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.373+0000 7f50dfc2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81b0ce0 con 0x55f42c6839e0 2024-03-21T11:35:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.373+0000 7f50d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c693ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.373+0000 7f50d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c693ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49866/0 (socket says 172.21.15.47:49866) 2024-03-21T11:35:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.373+0000 7f50d5400640 1 -- 172.21.15.47:0/1393780177 learned_addr learned my addr 172.21.15.47:0/1393780177 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.374+0000 7f50d6800640 1 --2- 172.21.15.47:0/1393780177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c682810 0x55f42c686c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.374+0000 7f50d6800640 1 --2- 172.21.15.47:0/1393780177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c6843a0 0x55f42c687150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.374+0000 7f50d5400640 1 -- 172.21.15.47:0/1393780177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c6843a0 msgr2=0x55f42c687150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.374+0000 7f50d5400640 1 --2- 172.21.15.47:0/1393780177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c6843a0 0x55f42c687150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.374+0000 7f50d5400640 1 -- 172.21.15.47:0/1393780177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c682810 msgr2=0x55f42c686c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.374+0000 7f50d5400640 1 --2- 172.21.15.47:0/1393780177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c682810 0x55f42c686c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.374+0000 7f50d5400640 1 -- 172.21.15.47:0/1393780177 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f42c696f50 con 0x55f42c6839e0 2024-03-21T11:35:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.374+0000 7f50d6800640 1 --2- 172.21.15.47:0/1393780177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c6843a0 0x55f42c687150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:35:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.374+0000 7f50d5400640 1 --2- 172.21.15.47:0/1393780177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c693ff0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f50c401cfd0 tx=0x7f50c4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bafeaab2def9bdbe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.375+0000 7f50c2a00640 1 -- 172.21.15.47:0/1393780177 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c402c030 con 0x55f42c6839e0 2024-03-21T11:35:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.375+0000 7f50c2a00640 1 -- 172.21.15.47:0/1393780177 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50c4011440 con 0x55f42c6839e0 2024-03-21T11:35:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.375+0000 7f50c2a00640 1 -- 172.21.15.47:0/1393780177 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c401b7b0 con 0x55f42c6839e0 2024-03-21T11:35:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.376+0000 7f50dfc2b740 1 -- 172.21.15.47:0/1393780177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 msgr2=0x55f42c693ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.376+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/1393780177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c693ff0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f50c401cfd0 tx=0x7f50c4005880 comp rx=0 tx=0).stop 2024-03-21T11:35:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.376+0000 7f50dfc2b740 1 -- 172.21.15.47:0/1393780177 shutdown_connections 2024-03-21T11:35:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.376+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/1393780177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c6843a0 0x55f42c687150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.376+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/1393780177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c693ff0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.376+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/1393780177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c682810 0x55f42c686c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.376+0000 7f50dfc2b740 1 -- 172.21.15.47:0/1393780177 >> 172.21.15.47:0/1393780177 conn(0x55f42c67dc00 msgr2=0x55f42c6816f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.376+0000 7f50dfc2b740 1 -- 172.21.15.47:0/1393780177 shutdown_connections 2024-03-21T11:35:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.376+0000 7f50dfc2b740 1 -- 172.21.15.47:0/1393780177 wait complete. 2024-03-21T11:35:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.377+0000 7f50dfc2b740 1 -- start start 2024-03-21T11:35:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.377+0000 7f50dfc2b740 1 Processor -- start 2024-03-21T11:35:09.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50dfc2b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 0x55f42c691910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50dfc2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c691e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50dfc2b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6843a0 0x55f42c692df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50dfc2b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81b19a0 con 0x55f42c6843a0 2024-03-21T11:35:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50dfc2b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81b1340 con 0x55f42c682810 2024-03-21T11:35:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50dfc2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d81ae840 con 0x55f42c6839e0 2024-03-21T11:35:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50dfc2b740 10 client.0 _ll_register_callbacks cb 0x7f50c4004220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c691e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c691e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49882/0 (socket says 172.21.15.47:49882) 2024-03-21T11:35:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50d5400640 1 -- 172.21.15.47:0/954168804 learned_addr learned my addr 172.21.15.47:0/954168804 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50d6800640 1 --2- 172.21.15.47:0/954168804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 0x55f42c691910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50d6800640 1 --2- 172.21.15.47:0/954168804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6843a0 0x55f42c692df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50d5400640 1 -- 172.21.15.47:0/954168804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 msgr2=0x55f42c691910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50d5400640 1 --2- 172.21.15.47:0/954168804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 0x55f42c691910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50d5400640 1 -- 172.21.15.47:0/954168804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6843a0 msgr2=0x55f42c692df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50d5400640 1 --2- 172.21.15.47:0/954168804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6843a0 0x55f42c692df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50d5400640 1 -- 172.21.15.47:0/954168804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f42c6933e0 con 0x55f42c6839e0 2024-03-21T11:35:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.378+0000 7f50d6800640 1 --2- 172.21.15.47:0/954168804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6843a0 0x55f42c692df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:35:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.379+0000 7f50d5400640 1 --2- 172.21.15.47:0/954168804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c691e50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f50c4011610 tx=0x7f50c40053b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb52857540293656 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.379+0000 7f50b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.379+0000 7f50b7e00640 1 -- 172.21.15.47:0/954168804 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c4035070 con 0x55f42c6839e0 2024-03-21T11:35:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.379+0000 7f50b7e00640 1 -- 172.21.15.47:0/954168804 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f50c401b6f0 con 0x55f42c6839e0 2024-03-21T11:35:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.379+0000 7f50dfc2b740 1 -- 172.21.15.47:0/954168804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50c4011440 con 0x55f42c6839e0 2024-03-21T11:35:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.380+0000 7f50b7e00640 1 -- 172.21.15.47:0/954168804 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50c402c040 con 0x55f42c6839e0 2024-03-21T11:35:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.380+0000 7f50dfc2b740 10 client.29458 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.380+0000 7f50dfc2b740 1 -- 172.21.15.47:0/954168804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f42c6f3ff0 con 0x55f42c6839e0 2024-03-21T11:35:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.380+0000 7f50b7e00640 1 -- 172.21.15.47:0/954168804 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50c4011440 con 0x55f42c6839e0 2024-03-21T11:35:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.381+0000 7f50b7e00640 1 -- 172.21.15.47:0/954168804 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f50c4042020 con 0x55f42c6839e0 2024-03-21T11:35:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.381+0000 7f50dfc2b740 10 client.29458 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.381+0000 7f50dfc2b740 10 client.29458 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:35:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.381+0000 7f50dfc2b740 20 client.29458 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.381+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/954168804 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f42c6f4860 0x55f42c6f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.381+0000 7f50dfc2b740 4 client.29458 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:35:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.381+0000 7f50dfc2b740 1 -- 172.21.15.47:0/954168804 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f42c5eeb50 con 0x55f42c6f4860 2024-03-21T11:35:09.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.381+0000 7f50d6800640 1 --2- 172.21.15.47:0/954168804 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f42c6f4860 0x55f42c6f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.382+0000 7f50d6800640 1 --2- 172.21.15.47:0/954168804 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f42c6f4860 0x55f42c6f6c80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9435c5acdbb50e94 server_cookie=8a58254b18928221 in_seq=0 out_seq=0 2024-03-21T11:35:09.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.382+0000 7f50b7e00640 10 client.29458 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:35:09.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.382+0000 7f50b7e00640 1 -- 172.21.15.47:0/954168804 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f42c5eeb50 con 0x55f42c6f4860 2024-03-21T11:35:09.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.382+0000 7f50b7e00640 10 client.29458 handle_command_reply: tid=0 2024-03-21T11:35:09.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.382+0000 7f50dfc2b740 1 client.29458 shutdown 2024-03-21T11:35:09.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.382+0000 7f50dfc2b740 1 -- 172.21.15.47:0/954168804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 msgr2=0x55f42c691e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.382+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/954168804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c691e50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f50c4011610 tx=0x7f50c40053b0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.382+0000 7f50dfc2b740 1 -- 172.21.15.47:0/954168804 shutdown_connections 2024-03-21T11:35:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.382+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/954168804 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f42c6f4860 0x55f42c6f6c80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.382+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/954168804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f42c6843a0 0x55f42c692df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.382+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/954168804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f42c6839e0 0x55f42c691e50 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.382+0000 7f50dfc2b740 1 --2- 172.21.15.47:0/954168804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f42c682810 0x55f42c691910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.382+0000 7f50dfc2b740 1 -- 172.21.15.47:0/954168804 >> 172.21.15.47:0/954168804 conn(0x55f42c67dc00 msgr2=0x55f42c67c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.382+0000 7f50b7e00640 0 client.29458 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:35:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.383+0000 7f50dfc2b740 1 -- 172.21.15.47:0/954168804 shutdown_connections 2024-03-21T11:35:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.383+0000 7f50dfc2b740 1 -- 172.21.15.47:0/954168804 wait complete. 2024-03-21T11:35:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.383+0000 7f50dfc2b740 20 client.29458 trim_cache size 0 max 16384 2024-03-21T11:35:09.389 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:35:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.384+0000 7f50dd000640 1 -- 172.21.15.47:0/3129779222 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50bc0a3070 msgr2=0x7f50bc0a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.384+0000 7f50dd000640 1 --2- 172.21.15.47:0/3129779222 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50bc0a3070 0x7f50bc0a5530 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f50c80be150 tx=0x7f50c8009500 comp rx=0 tx=0).stop 2024-03-21T11:35:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.384+0000 7f50dd000640 1 -- 172.21.15.47:0/3129779222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50d81016c0 msgr2=0x7f50d819e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.384+0000 7f50dd000640 1 --2- 172.21.15.47:0/3129779222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50d81016c0 0x7f50d819e050 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009b30 tx=0x7f50c8033580 comp rx=0 tx=0).stop 2024-03-21T11:35:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.384+0000 7f50dd000640 1 -- 172.21.15.47:0/3129779222 shutdown_connections 2024-03-21T11:35:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.384+0000 7f50dd000640 1 --2- 172.21.15.47:0/3129779222 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50bc0a3070 0x7f50bc0a5530 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.384+0000 7f50dd000640 1 --2- 172.21.15.47:0/3129779222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50d8103270 0x7f50d81a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.384+0000 7f50dd000640 1 --2- 172.21.15.47:0/3129779222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50d81028b0 0x7f50d819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.384+0000 7f50dd000640 1 --2- 172.21.15.47:0/3129779222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50d81016c0 0x7f50d819e050 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.384+0000 7f50dd000640 1 -- 172.21.15.47:0/3129779222 >> 172.21.15.47:0/3129779222 conn(0x7f50d80fcc50 msgr2=0x7f50d80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.384+0000 7f50dd000640 1 -- 172.21.15.47:0/3129779222 shutdown_connections 2024-03-21T11:35:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.384+0000 7f50dd000640 1 -- 172.21.15.47:0/3129779222 wait complete. 2024-03-21T11:35:09.410 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:35:09.410 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:35:09.565 INFO:tasks.workunit.client.0.smithi047.stdout: 1 892022 7.97 MB/sec execute 473 sec latency 16.659 ms 2024-03-21T11:35:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.644+0000 7f0650000640 1 -- 172.21.15.47:0/977903122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06480772a0 msgr2=0x7f0648077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.644+0000 7f0650000640 1 --2- 172.21.15.47:0/977903122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06480772a0 0x7f0648077700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0634009860 tx=0x7f063402bb70 comp rx=0 tx=0).stop 2024-03-21T11:35:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.644+0000 7f0650000640 1 -- 172.21.15.47:0/977903122 shutdown_connections 2024-03-21T11:35:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.644+0000 7f0650000640 1 --2- 172.21.15.47:0/977903122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0648077c40 0x7f06480784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.644+0000 7f0650000640 1 --2- 172.21.15.47:0/977903122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06480772a0 0x7f0648077700 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.644+0000 7f0650000640 1 --2- 172.21.15.47:0/977903122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0648076060 0x7f0648076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.644+0000 7f0650000640 1 -- 172.21.15.47:0/977903122 >> 172.21.15.47:0/977903122 conn(0x7f06480fee50 msgr2=0x7f0648101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.644+0000 7f0650000640 1 -- 172.21.15.47:0/977903122 shutdown_connections 2024-03-21T11:35:09.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.644+0000 7f0650000640 1 -- 172.21.15.47:0/977903122 wait complete. 2024-03-21T11:35:09.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.645+0000 7f0650000640 1 Processor -- start 2024-03-21T11:35:09.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.645+0000 7f0650000640 1 -- start start 2024-03-21T11:35:09.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.646+0000 7f0650000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0648076060 0x7f064819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.646+0000 7f0650000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06480772a0 0x7f064819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.646+0000 7f0650000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0648077c40 0x7f06481a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.646+0000 7f0650000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0648113930 con 0x7f0648077c40 2024-03-21T11:35:09.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.646+0000 7f0650000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481137b0 con 0x7f06480772a0 2024-03-21T11:35:09.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.646+0000 7f0650000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0648113ab0 con 0x7f0648076060 2024-03-21T11:35:09.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.646+0000 7f064d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0648076060 0x7f064819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.646+0000 7f064d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0648076060 0x7f064819dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49902/0 (socket says 172.21.15.47:49902) 2024-03-21T11:35:09.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.646+0000 7f064cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06480772a0 0x7f064819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.646+0000 7f064d600640 1 -- 172.21.15.47:0/3905700590 learned_addr learned my addr 172.21.15.47:0/3905700590 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.646+0000 7f064e000640 1 --2- 172.21.15.47:0/3905700590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0648077c40 0x7f06481a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.647+0000 7f064cc00640 1 -- 172.21.15.47:0/3905700590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0648076060 msgr2=0x7f064819dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.647+0000 7f064cc00640 1 --2- 172.21.15.47:0/3905700590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0648076060 0x7f064819dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.647+0000 7f064cc00640 1 -- 172.21.15.47:0/3905700590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0648077c40 msgr2=0x7f06481a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.647+0000 7f064cc00640 1 --2- 172.21.15.47:0/3905700590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0648077c40 0x7f06481a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.647+0000 7f064cc00640 1 -- 172.21.15.47:0/3905700590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06481a5c00 con 0x7f06480772a0 2024-03-21T11:35:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.647+0000 7f064d600640 1 --2- 172.21.15.47:0/3905700590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0648076060 0x7f064819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:09.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.647+0000 7f064cc00640 1 --2- 172.21.15.47:0/3905700590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06480772a0 0x7f064819e520 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f063402c0b0 tx=0x7f0634032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.650+0000 7f0646000640 1 -- 172.21.15.47:0/3905700590 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634041020 con 0x7f06480772a0 2024-03-21T11:35:09.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.650+0000 7f0646000640 1 -- 172.21.15.47:0/3905700590 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f063402c9f0 con 0x7f06480772a0 2024-03-21T11:35:09.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.651+0000 7f0650000640 1 -- 172.21.15.47:0/3905700590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06481a5e90 con 0x7f06480772a0 2024-03-21T11:35:09.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.651+0000 7f0646000640 1 -- 172.21.15.47:0/3905700590 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634039040 con 0x7f06480772a0 2024-03-21T11:35:09.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.651+0000 7f0650000640 1 -- 172.21.15.47:0/3905700590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06481a6300 con 0x7f06480772a0 2024-03-21T11:35:09.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.654+0000 7f0646000640 1 -- 172.21.15.47:0/3905700590 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0634044020 con 0x7f06480772a0 2024-03-21T11:35:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.654+0000 7f0652c79740 1 Processor -- start 2024-03-21T11:35:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.655+0000 7f0652c79740 1 -- start start 2024-03-21T11:35:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.655+0000 7f0652c79740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ea810 0x55c5b64eac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.655+0000 7f0652c79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 0x55c5b64ebe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.655+0000 7f0652c79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 0x55c5b64f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.655+0000 7f0652c79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b644f2c0 con 0x55c5b64eb9e0 2024-03-21T11:35:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.655+0000 7f0652c79740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b63f7c90 con 0x55c5b64ec3a0 2024-03-21T11:35:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.655+0000 7f0652c79740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b644b360 con 0x55c5b64ea810 2024-03-21T11:35:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.655+0000 7f064e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ea810 0x55c5b64eac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.655+0000 7f064cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 0x55c5b64f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.656+0000 7f064e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ea810 0x55c5b64eac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49912/0 (socket says 172.21.15.47:49912) 2024-03-21T11:35:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.656+0000 7f064d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 0x55c5b64ebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.656+0000 7f064e000640 1 -- 172.21.15.47:0/3227863266 learned_addr learned my addr 172.21.15.47:0/3227863266 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.656+0000 7f064e000640 1 -- 172.21.15.47:0/3227863266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 msgr2=0x55c5b64f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.656+0000 7f064e000640 1 --2- 172.21.15.47:0/3227863266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 0x55c5b64f2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.656+0000 7f064e000640 1 -- 172.21.15.47:0/3227863266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 msgr2=0x55c5b64ebe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.656+0000 7f064e000640 1 --2- 172.21.15.47:0/3227863266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 0x55c5b64ebe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.656+0000 7f064e000640 1 -- 172.21.15.47:0/3227863266 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5b641c080 con 0x55c5b64ea810 2024-03-21T11:35:09.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.656+0000 7f0646000640 1 --2- 172.21.15.47:0/3905700590 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f06300a2fa0 0x7f06300a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.656+0000 7f0646000640 1 -- 172.21.15.47:0/3905700590 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f06340f0f30 con 0x7f06480772a0 2024-03-21T11:35:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.656+0000 7f064e000640 1 --2- 172.21.15.47:0/3227863266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ea810 0x55c5b64eac30 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0640009fd0 tx=0x7f06400118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb58b02b69ccf29e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.656+0000 7f064d600640 1 --2- 172.21.15.47:0/3905700590 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f06300a2fa0 0x7f06300a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.657+0000 7f063a600640 1 -- 172.21.15.47:0/3227863266 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640004d20 con 0x55c5b64ea810 2024-03-21T11:35:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.657+0000 7f063a600640 1 -- 172.21.15.47:0/3227863266 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0640004050 con 0x55c5b64ea810 2024-03-21T11:35:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.657+0000 7f063a600640 1 -- 172.21.15.47:0/3227863266 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640005260 con 0x55c5b64ea810 2024-03-21T11:35:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.657+0000 7f0652c79740 1 -- 172.21.15.47:0/3227863266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ea810 msgr2=0x55c5b64eac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.657+0000 7f064d600640 1 --2- 172.21.15.47:0/3905700590 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f06300a2fa0 0x7f06300a5460 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f063c005c80 tx=0x7f063c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.657+0000 7f0652c79740 1 --2- 172.21.15.47:0/3227863266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ea810 0x55c5b64eac30 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0640009fd0 tx=0x7f06400118d0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.658+0000 7f0652c79740 1 -- 172.21.15.47:0/3227863266 shutdown_connections 2024-03-21T11:35:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.658+0000 7f0652c79740 1 --2- 172.21.15.47:0/3227863266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 0x55c5b64f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.658+0000 7f0652c79740 1 --2- 172.21.15.47:0/3227863266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 0x55c5b64ebe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.658+0000 7f0652c79740 1 --2- 172.21.15.47:0/3227863266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ea810 0x55c5b64eac30 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.658+0000 7f0652c79740 1 -- 172.21.15.47:0/3227863266 >> 172.21.15.47:0/3227863266 conn(0x55c5b64e5c00 msgr2=0x55c5b64e6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.658+0000 7f0652c79740 1 -- 172.21.15.47:0/3227863266 shutdown_connections 2024-03-21T11:35:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.658+0000 7f0652c79740 1 -- 172.21.15.47:0/3227863266 wait complete. 2024-03-21T11:35:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.658+0000 7f0652c79740 1 -- start start 2024-03-21T11:35:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.658+0000 7f0652c79740 1 Processor -- start 2024-03-21T11:35:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.659+0000 7f0652c79740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ea810 0x55c5b64f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.659+0000 7f0652c79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 0x55c5b64f5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.659+0000 7f0652c79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 0x55c5b64fc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.659+0000 7f0652c79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b64f33b0 con 0x55c5b64eb9e0 2024-03-21T11:35:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.659+0000 7f0652c79740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b640b0d0 con 0x55c5b64ec3a0 2024-03-21T11:35:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.659+0000 7f0652c79740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b64f3530 con 0x55c5b64ea810 2024-03-21T11:35:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.659+0000 7f0652c79740 10 client.0 _ll_register_callbacks cb 0x55c5b640fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.659+0000 7f064d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ea810 0x55c5b64f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.659+0000 7f064d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ea810 0x55c5b64f4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49926/0 (socket says 172.21.15.47:49926) 2024-03-21T11:35:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.659+0000 7f064d600640 1 -- 172.21.15.47:0/1280397607 learned_addr learned my addr 172.21.15.47:0/1280397607 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.659+0000 7f064e000640 1 --2- 172.21.15.47:0/1280397607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 0x55c5b64fc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.659+0000 7f064cc00640 1 --2- 172.21.15.47:0/1280397607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 0x55c5b64f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.660+0000 7f064d600640 1 -- 172.21.15.47:0/1280397607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 msgr2=0x55c5b64fc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.660+0000 7f064d600640 1 --2- 172.21.15.47:0/1280397607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 0x55c5b64fc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.660+0000 7f064d600640 1 -- 172.21.15.47:0/1280397607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 msgr2=0x55c5b64f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.660+0000 7f064d600640 1 --2- 172.21.15.47:0/1280397607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 0x55c5b64f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.660+0000 7f064d600640 1 -- 172.21.15.47:0/1280397607 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5b61633f0 con 0x55c5b64ea810 2024-03-21T11:35:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.660+0000 7f064d600640 1 --2- 172.21.15.47:0/1280397607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ea810 0x55c5b64f4b10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f063c0124c0 tx=0x7f063c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc8207908777efd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.660+0000 7f062ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.660+0000 7f062ea00640 1 -- 172.21.15.47:0/1280397607 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c0035f0 con 0x55c5b64ea810 2024-03-21T11:35:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.660+0000 7f062ea00640 1 -- 172.21.15.47:0/1280397607 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f063c003790 con 0x55c5b64ea810 2024-03-21T11:35:09.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.660+0000 7f0652c79740 1 -- 172.21.15.47:0/1280397607 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0640004050 con 0x55c5b64ea810 2024-03-21T11:35:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.660+0000 7f062ea00640 1 -- 172.21.15.47:0/1280397607 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f063c032050 con 0x55c5b64ea810 2024-03-21T11:35:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.660+0000 7f0652c79740 10 client.29470 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.661+0000 7f0652c79740 1 -- 172.21.15.47:0/1280397607 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c5b622d7d0 con 0x55c5b64ea810 2024-03-21T11:35:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.661+0000 7f062ea00640 1 -- 172.21.15.47:0/1280397607 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0640004050 con 0x55c5b64ea810 2024-03-21T11:35:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.661+0000 7f062ea00640 1 -- 172.21.15.47:0/1280397607 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f063c011bd0 con 0x55c5b64ea810 2024-03-21T11:35:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.661+0000 7f0652c79740 10 client.29470 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.661+0000 7f0652c79740 10 client.29470 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:35:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.661+0000 7f0652c79740 20 client.29470 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.661+0000 7f0652c79740 1 --2- 172.21.15.47:0/1280397607 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c5b6500670 0x55c5b6502ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.661+0000 7f0652c79740 4 client.29470 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.661+0000 7f0652c79740 1 -- 172.21.15.47:0/1280397607 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5b645c800 con 0x55c5b6500670 2024-03-21T11:35:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.662+0000 7f064cc00640 1 --2- 172.21.15.47:0/1280397607 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c5b6500670 0x55c5b6502ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.662+0000 7f064cc00640 1 --2- 172.21.15.47:0/1280397607 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c5b6500670 0x55c5b6502ab0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6f619cbea56e850f server_cookie=e84f2096a4902f6e in_seq=0 out_seq=0 2024-03-21T11:35:09.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.663+0000 7f062ea00640 10 client.29470 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:35:09.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.663+0000 7f062ea00640 1 -- 172.21.15.47:0/1280397607 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c5b645c800 con 0x55c5b6500670 2024-03-21T11:35:09.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.663+0000 7f062ea00640 10 client.29470 handle_command_reply: tid=0 2024-03-21T11:35:09.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.663+0000 7f0652c79740 1 client.29470 shutdown 2024-03-21T11:35:09.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.663+0000 7f0652c79740 1 -- 172.21.15.47:0/1280397607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ea810 msgr2=0x55c5b64f4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.663+0000 7f0652c79740 1 --2- 172.21.15.47:0/1280397607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ea810 0x55c5b64f4b10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f063c0124c0 tx=0x7f063c004730 comp rx=0 tx=0).stop 2024-03-21T11:35:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.663+0000 7f0652c79740 1 -- 172.21.15.47:0/1280397607 shutdown_connections 2024-03-21T11:35:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.663+0000 7f0652c79740 1 --2- 172.21.15.47:0/1280397607 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c5b6500670 0x55c5b6502ab0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.663+0000 7f0652c79740 1 --2- 172.21.15.47:0/1280397607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 0x55c5b64fc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.663+0000 7f0652c79740 1 --2- 172.21.15.47:0/1280397607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 0x55c5b64f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.663+0000 7f0652c79740 1 --2- 172.21.15.47:0/1280397607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ea810 0x55c5b64f4b10 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.663+0000 7f0652c79740 1 -- 172.21.15.47:0/1280397607 >> 172.21.15.47:0/1280397607 conn(0x55c5b64e5c00 msgr2=0x55c5b64e4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.663+0000 7f062ea00640 0 client.29470 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:35:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.664+0000 7f0652c79740 1 -- 172.21.15.47:0/1280397607 shutdown_connections 2024-03-21T11:35:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.664+0000 7f0652c79740 1 -- 172.21.15.47:0/1280397607 wait complete. 2024-03-21T11:35:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.664+0000 7f0652c79740 20 client.29470 trim_cache size 0 max 16384 2024-03-21T11:35:09.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.680+0000 7f0652c79740 1 Processor -- start 2024-03-21T11:35:09.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.680+0000 7f0652c79740 1 -- start start 2024-03-21T11:35:09.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.680+0000 7f0652c79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ea810 0x55c5b64eea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f0652c79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 0x55c5b64fc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f0652c79740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ec3a0 0x55c5b64eefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f0652c79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481b0630 con 0x55c5b64eb9e0 2024-03-21T11:35:09.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f0652c79740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481b1fb0 con 0x55c5b64ea810 2024-03-21T11:35:09.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f0652c79740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481b0c90 con 0x55c5b64ec3a0 2024-03-21T11:35:09.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f064d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ea810 0x55c5b64eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f064e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ec3a0 0x55c5b64eefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f064d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ea810 0x55c5b64eea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54640/0 (socket says 172.21.15.47:54640) 2024-03-21T11:35:09.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f064d600640 1 -- 172.21.15.47:0/1854940303 learned_addr learned my addr 172.21.15.47:0/1854940303 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f064cc00640 1 --2- 172.21.15.47:0/1854940303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 0x55c5b64fc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f064d600640 1 -- 172.21.15.47:0/1854940303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ec3a0 msgr2=0x55c5b64eefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f064d600640 1 --2- 172.21.15.47:0/1854940303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ec3a0 0x55c5b64eefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f064d600640 1 -- 172.21.15.47:0/1854940303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 msgr2=0x55c5b64fc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f064d600640 1 --2- 172.21.15.47:0/1854940303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 0x55c5b64fc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.681+0000 7f064d600640 1 -- 172.21.15.47:0/1854940303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5b64f36b0 con 0x55c5b64ea810 2024-03-21T11:35:09.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.682+0000 7f064d600640 1 --2- 172.21.15.47:0/1854940303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ea810 0x55c5b64eea80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f063c019040 tx=0x7f063c01f570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4616ea45fc82d8ba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.682+0000 7f0639c00640 1 -- 172.21.15.47:0/1854940303 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c00fe30 con 0x55c5b64ea810 2024-03-21T11:35:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.682+0000 7f0639c00640 1 -- 172.21.15.47:0/1854940303 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f063c03b070 con 0x55c5b64ea810 2024-03-21T11:35:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.682+0000 7f0639c00640 1 -- 172.21.15.47:0/1854940303 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c011b60 con 0x55c5b64ea810 2024-03-21T11:35:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.683+0000 7f0652c79740 1 -- 172.21.15.47:0/1854940303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ea810 msgr2=0x55c5b64eea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.683+0000 7f0652c79740 1 --2- 172.21.15.47:0/1854940303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ea810 0x55c5b64eea80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f063c019040 tx=0x7f063c01f570 comp rx=0 tx=0).stop 2024-03-21T11:35:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.683+0000 7f0652c79740 1 -- 172.21.15.47:0/1854940303 shutdown_connections 2024-03-21T11:35:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.683+0000 7f0652c79740 1 --2- 172.21.15.47:0/1854940303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64ec3a0 0x55c5b64eefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.683+0000 7f0652c79740 1 --2- 172.21.15.47:0/1854940303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64eb9e0 0x55c5b64fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.683+0000 7f0652c79740 1 --2- 172.21.15.47:0/1854940303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ea810 0x55c5b64eea80 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.683+0000 7f0652c79740 1 -- 172.21.15.47:0/1854940303 >> 172.21.15.47:0/1854940303 conn(0x55c5b64e5c00 msgr2=0x55c5b64e96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.683+0000 7f0652c79740 1 -- 172.21.15.47:0/1854940303 shutdown_connections 2024-03-21T11:35:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.683+0000 7f0652c79740 1 -- 172.21.15.47:0/1854940303 wait complete. 2024-03-21T11:35:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.684+0000 7f0652c79740 1 -- start start 2024-03-21T11:35:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.684+0000 7f0652c79740 1 Processor -- start 2024-03-21T11:35:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.684+0000 7f0652c79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64ea810 0x55c5b64fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.684+0000 7f0652c79740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64eb9e0 0x55c5b64fbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.684+0000 7f0652c79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 0x55c5b64f8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.684+0000 7f0652c79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481b1950 con 0x55c5b64ea810 2024-03-21T11:35:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.684+0000 7f0652c79740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481b12f0 con 0x55c5b64ec3a0 2024-03-21T11:35:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.684+0000 7f0652c79740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481ae7f0 con 0x55c5b64eb9e0 2024-03-21T11:35:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.684+0000 7f0652c79740 10 client.0 _ll_register_callbacks cb 0x7f063c005a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f064cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64eb9e0 0x55c5b64fbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f064cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64eb9e0 0x55c5b64fbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49938/0 (socket says 172.21.15.47:49938) 2024-03-21T11:35:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f064cc00640 1 -- 172.21.15.47:0/1647625972 learned_addr learned my addr 172.21.15.47:0/1647625972 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f064e000640 1 --2- 172.21.15.47:0/1647625972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 0x55c5b64f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f064d600640 1 --2- 172.21.15.47:0/1647625972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64ea810 0x55c5b64fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f064cc00640 1 -- 172.21.15.47:0/1647625972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 msgr2=0x55c5b64f8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f064cc00640 1 --2- 172.21.15.47:0/1647625972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 0x55c5b64f8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f064cc00640 1 -- 172.21.15.47:0/1647625972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64ea810 msgr2=0x55c5b64fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f064cc00640 1 --2- 172.21.15.47:0/1647625972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64ea810 0x55c5b64fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f064cc00640 1 -- 172.21.15.47:0/1647625972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5b64f8960 con 0x55c5b64eb9e0 2024-03-21T11:35:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f064d600640 1 --2- 172.21.15.47:0/1647625972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64ea810 0x55c5b64fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f064cc00640 1 --2- 172.21.15.47:0/1647625972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64eb9e0 0x55c5b64fbe10 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0634004630 tx=0x7f06340f4970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f2fa743265ea820 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f062ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f062ea00640 1 -- 172.21.15.47:0/1647625972 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06340bcdb0 con 0x55c5b64eb9e0 2024-03-21T11:35:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.685+0000 7f062ea00640 1 -- 172.21.15.47:0/1647625972 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f06340042c0 con 0x55c5b64eb9e0 2024-03-21T11:35:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.686+0000 7f0652c79740 1 -- 172.21.15.47:0/1647625972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f063c03b070 con 0x55c5b64eb9e0 2024-03-21T11:35:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.686+0000 7f062ea00640 1 -- 172.21.15.47:0/1647625972 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06340bcbd0 con 0x55c5b64eb9e0 2024-03-21T11:35:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.686+0000 7f0652c79740 10 client.29476 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.686+0000 7f0652c79740 1 -- 172.21.15.47:0/1647625972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c5b655cb50 con 0x55c5b64eb9e0 2024-03-21T11:35:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.686+0000 7f062ea00640 1 -- 172.21.15.47:0/1647625972 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f063c03b070 con 0x55c5b64eb9e0 2024-03-21T11:35:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.686+0000 7f062ea00640 1 -- 172.21.15.47:0/1647625972 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f063403c030 con 0x55c5b64eb9e0 2024-03-21T11:35:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.686+0000 7f0652c79740 10 client.29476 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.686+0000 7f0652c79740 10 client.29476 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:35:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.686+0000 7f0652c79740 20 client.29476 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.686+0000 7f0652c79740 1 --2- 172.21.15.47:0/1647625972 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c5b655d4b0 0x55c5b655f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.686+0000 7f0652c79740 4 client.29476 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:35:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.686+0000 7f0652c79740 1 -- 172.21.15.47:0/1647625972 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c5b6456b50 con 0x55c5b655d4b0 2024-03-21T11:35:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.686+0000 7f064d600640 1 --2- 172.21.15.47:0/1647625972 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c5b655d4b0 0x55c5b655f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.687+0000 7f064d600640 1 --2- 172.21.15.47:0/1647625972 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c5b655d4b0 0x55c5b655f8d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d42ba235a8bc17e server_cookie=1408278676959694 in_seq=0 out_seq=0 2024-03-21T11:35:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.687+0000 7f062ea00640 10 client.29476 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:35:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.687+0000 7f062ea00640 1 -- 172.21.15.47:0/1647625972 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c5b6456b50 con 0x55c5b655d4b0 2024-03-21T11:35:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.687+0000 7f062ea00640 10 client.29476 handle_command_reply: tid=0 2024-03-21T11:35:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.687+0000 7f0652c79740 1 client.29476 shutdown 2024-03-21T11:35:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.687+0000 7f0652c79740 1 -- 172.21.15.47:0/1647625972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64eb9e0 msgr2=0x55c5b64fbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.687+0000 7f0652c79740 1 --2- 172.21.15.47:0/1647625972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64eb9e0 0x55c5b64fbe10 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0634004630 tx=0x7f06340f4970 comp rx=0 tx=0).stop 2024-03-21T11:35:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.687+0000 7f0652c79740 1 -- 172.21.15.47:0/1647625972 shutdown_connections 2024-03-21T11:35:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.687+0000 7f0652c79740 1 --2- 172.21.15.47:0/1647625972 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c5b655d4b0 0x55c5b655f8d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.687+0000 7f0652c79740 1 --2- 172.21.15.47:0/1647625972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5b64ec3a0 0x55c5b64f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.687+0000 7f0652c79740 1 --2- 172.21.15.47:0/1647625972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5b64eb9e0 0x55c5b64fbe10 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.687+0000 7f062ea00640 0 client.29476 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:35:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.687+0000 7f0652c79740 1 --2- 172.21.15.47:0/1647625972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5b64ea810 0x55c5b64fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.687+0000 7f0652c79740 1 -- 172.21.15.47:0/1647625972 >> 172.21.15.47:0/1647625972 conn(0x55c5b64e5c00 msgr2=0x55c5b64e4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.688+0000 7f0652c79740 1 -- 172.21.15.47:0/1647625972 shutdown_connections 2024-03-21T11:35:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.688+0000 7f0652c79740 1 -- 172.21.15.47:0/1647625972 wait complete. 2024-03-21T11:35:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.688+0000 7f0652c79740 20 client.29476 trim_cache size 0 max 16384 2024-03-21T11:35:09.696 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:35:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.689+0000 7f0650000640 1 -- 172.21.15.47:0/3905700590 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f06300a2fa0 msgr2=0x7f06300a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.689+0000 7f0650000640 1 --2- 172.21.15.47:0/3905700590 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f06300a2fa0 0x7f06300a5460 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f063c005c80 tx=0x7f063c009260 comp rx=0 tx=0).stop 2024-03-21T11:35:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.689+0000 7f0650000640 1 -- 172.21.15.47:0/3905700590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06480772a0 msgr2=0x7f064819e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.689+0000 7f0650000640 1 --2- 172.21.15.47:0/3905700590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06480772a0 0x7f064819e520 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f063402c0b0 tx=0x7f0634032430 comp rx=0 tx=0).stop 2024-03-21T11:35:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.689+0000 7f0650000640 1 -- 172.21.15.47:0/3905700590 shutdown_connections 2024-03-21T11:35:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.689+0000 7f0650000640 1 --2- 172.21.15.47:0/3905700590 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f06300a2fa0 0x7f06300a5460 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.689+0000 7f0650000640 1 --2- 172.21.15.47:0/3905700590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0648077c40 0x7f06481a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.689+0000 7f0650000640 1 --2- 172.21.15.47:0/3905700590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06480772a0 0x7f064819e520 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.689+0000 7f0650000640 1 --2- 172.21.15.47:0/3905700590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0648076060 0x7f064819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.689+0000 7f0650000640 1 -- 172.21.15.47:0/3905700590 >> 172.21.15.47:0/3905700590 conn(0x7f06480fee50 msgr2=0x7f0648100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.689+0000 7f0650000640 1 -- 172.21.15.47:0/3905700590 shutdown_connections 2024-03-21T11:35:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.689+0000 7f0650000640 1 -- 172.21.15.47:0/3905700590 wait complete. 2024-03-21T11:35:09.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:09 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1892539867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:35:09.710 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:35:09.710 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:35:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.931+0000 7fd723000640 1 -- 172.21.15.47:0/1969500171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd71c1007b0 msgr2=0x7fd71c100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.931+0000 7fd723000640 1 --2- 172.21.15.47:0/1969500171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd71c1007b0 0x7fd71c100c10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd708009900 tx=0x7fd70802bc30 comp rx=0 tx=0).stop 2024-03-21T11:35:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.932+0000 7fd723000640 1 -- 172.21.15.47:0/1969500171 shutdown_connections 2024-03-21T11:35:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.932+0000 7fd723000640 1 --2- 172.21.15.47:0/1969500171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd71c101150 0x7fd71c1015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.932+0000 7fd723000640 1 --2- 172.21.15.47:0/1969500171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd71c1007b0 0x7fd71c100c10 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.932+0000 7fd723000640 1 --2- 172.21.15.47:0/1969500171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd71c0ffe50 0x7fd71c100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.932+0000 7fd723000640 1 -- 172.21.15.47:0/1969500171 >> 172.21.15.47:0/1969500171 conn(0x7fd71c0fba10 msgr2=0x7fd71c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.932+0000 7fd723000640 1 -- 172.21.15.47:0/1969500171 shutdown_connections 2024-03-21T11:35:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.932+0000 7fd723000640 1 -- 172.21.15.47:0/1969500171 wait complete. 2024-03-21T11:35:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.933+0000 7fd723000640 1 Processor -- start 2024-03-21T11:35:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.933+0000 7fd723000640 1 -- start start 2024-03-21T11:35:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.933+0000 7fd723000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd71c0ffe50 0x7fd71c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.933+0000 7fd723000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd71c1007b0 0x7fd71c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.933+0000 7fd723000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd71c101150 0x7fd71c1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.933+0000 7fd723000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c10f410 con 0x7fd71c1007b0 2024-03-21T11:35:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.933+0000 7fd723000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c10f290 con 0x7fd71c101150 2024-03-21T11:35:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.933+0000 7fd723000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c10f590 con 0x7fd71c0ffe50 2024-03-21T11:35:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.934+0000 7fd721200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd71c1007b0 0x7fd71c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.934+0000 7fd721200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd71c1007b0 0x7fd71c199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55872/0 (socket says 172.21.15.47:55872) 2024-03-21T11:35:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.934+0000 7fd721200640 1 -- 172.21.15.47:0/1584834578 learned_addr learned my addr 172.21.15.47:0/1584834578 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.934+0000 7fd722600640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd71c101150 0x7fd71c1a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.934+0000 7fd721c00640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd71c0ffe50 0x7fd71c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.934+0000 7fd722600640 1 -- 172.21.15.47:0/1584834578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd71c0ffe50 msgr2=0x7fd71c199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.934+0000 7fd722600640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd71c0ffe50 0x7fd71c199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.934+0000 7fd722600640 1 -- 172.21.15.47:0/1584834578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd71c1007b0 msgr2=0x7fd71c199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.934+0000 7fd722600640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd71c1007b0 0x7fd71c199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.934+0000 7fd722600640 1 -- 172.21.15.47:0/1584834578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c1a1670 con 0x7fd71c101150 2024-03-21T11:35:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.934+0000 7fd721200640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd71c1007b0 0x7fd71c199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.935+0000 7fd722600640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd71c101150 0x7fd71c1a1000 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd71400e940 tx=0x7fd71400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.935+0000 7fd71a000640 1 -- 172.21.15.47:0/1584834578 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714004d20 con 0x7fd71c101150 2024-03-21T11:35:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.935+0000 7fd71a000640 1 -- 172.21.15.47:0/1584834578 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd714005020 con 0x7fd71c101150 2024-03-21T11:35:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.935+0000 7fd71a000640 1 -- 172.21.15.47:0/1584834578 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7140116b0 con 0x7fd71c101150 2024-03-21T11:35:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.935+0000 7fd723000640 1 -- 172.21.15.47:0/1584834578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71c1a1960 con 0x7fd71c101150 2024-03-21T11:35:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.935+0000 7fd723000640 1 -- 172.21.15.47:0/1584834578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c1a1e30 con 0x7fd71c101150 2024-03-21T11:35:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.940+0000 7fd71a000640 1 -- 172.21.15.47:0/1584834578 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd7140054d0 con 0x7fd71c101150 2024-03-21T11:35:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.941+0000 7fd7273a0740 1 Processor -- start 2024-03-21T11:35:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.942+0000 7fd7273a0740 1 -- start start 2024-03-21T11:35:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.942+0000 7fd7273a0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc0949f0 0x55cefc094e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.942+0000 7fd7273a0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc095bc0 0x55cefc096040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.942+0000 7fd7273a0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc096580 0x55cefc09ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.942+0000 7fd7273a0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cefbff92c0 con 0x55cefc0949f0 2024-03-21T11:35:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.942+0000 7fd7273a0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cefbfa1c90 con 0x55cefc095bc0 2024-03-21T11:35:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.942+0000 7fd7273a0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cefbff5360 con 0x55cefc096580 2024-03-21T11:35:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.942+0000 7fd721c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc095bc0 0x55cefc096040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.942+0000 7fd721c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc095bc0 0x55cefc096040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54668/0 (socket says 172.21.15.47:54668) 2024-03-21T11:35:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.942+0000 7fd721c00640 1 -- 172.21.15.47:0/4208461342 learned_addr learned my addr 172.21.15.47:0/4208461342 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.942+0000 7fd721200640 1 --2- 172.21.15.47:0/4208461342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc0949f0 0x55cefc094e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.943+0000 7fd721200640 1 --2- 172.21.15.47:0/4208461342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc096580 0x55cefc09ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.943+0000 7fd721c00640 1 -- 172.21.15.47:0/4208461342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc096580 msgr2=0x55cefc09ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.943+0000 7fd721c00640 1 --2- 172.21.15.47:0/4208461342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc096580 0x55cefc09ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.943+0000 7fd721c00640 1 -- 172.21.15.47:0/4208461342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc0949f0 msgr2=0x55cefc094e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.943+0000 7fd721c00640 1 --2- 172.21.15.47:0/4208461342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc0949f0 0x55cefc094e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.943+0000 7fd721c00640 1 -- 172.21.15.47:0/4208461342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cefbfc6080 con 0x55cefc095bc0 2024-03-21T11:35:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.943+0000 7fd721200640 1 --2- 172.21.15.47:0/4208461342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc0949f0 0x55cefc094e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.943+0000 7fd71a000640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7040a2f20 0x7fd7040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.944+0000 7fd721c00640 1 --2- 172.21.15.47:0/4208461342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc095bc0 0x55cefc096040 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd710009cf0 tx=0x7fd710011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dd18e7cda7d875f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.944+0000 7fd71a000640 1 -- 172.21.15.47:0/1584834578 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd7140cf870 con 0x7fd71c101150 2024-03-21T11:35:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.944+0000 7fd70ea00640 1 -- 172.21.15.47:0/4208461342 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710004230 con 0x55cefc095bc0 2024-03-21T11:35:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.944+0000 7fd70ea00640 1 -- 172.21.15.47:0/4208461342 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7100043d0 con 0x55cefc095bc0 2024-03-21T11:35:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.944+0000 7fd70ea00640 1 -- 172.21.15.47:0/4208461342 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710019920 con 0x55cefc095bc0 2024-03-21T11:35:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.944+0000 7fd721200640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7040a2f20 0x7fd7040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.945+0000 7fd7273a0740 1 -- 172.21.15.47:0/4208461342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc095bc0 msgr2=0x55cefc096040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.945+0000 7fd7273a0740 1 --2- 172.21.15.47:0/4208461342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc095bc0 0x55cefc096040 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd710009cf0 tx=0x7fd710011810 comp rx=0 tx=0).stop 2024-03-21T11:35:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.945+0000 7fd7273a0740 1 -- 172.21.15.47:0/4208461342 shutdown_connections 2024-03-21T11:35:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.945+0000 7fd7273a0740 1 --2- 172.21.15.47:0/4208461342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc096580 0x55cefc09ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.945+0000 7fd7273a0740 1 --2- 172.21.15.47:0/4208461342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc095bc0 0x55cefc096040 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.945+0000 7fd7273a0740 1 --2- 172.21.15.47:0/4208461342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc0949f0 0x55cefc094e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.945+0000 7fd7273a0740 1 -- 172.21.15.47:0/4208461342 >> 172.21.15.47:0/4208461342 conn(0x55cefc08fde0 msgr2=0x55cefc0901e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.945+0000 7fd721200640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7040a2f20 0x7fd7040a53e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd70802c8e0 tx=0x7fd708005b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.945+0000 7fd7273a0740 1 -- 172.21.15.47:0/4208461342 shutdown_connections 2024-03-21T11:35:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.945+0000 7fd7273a0740 1 -- 172.21.15.47:0/4208461342 wait complete. 2024-03-21T11:35:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.946+0000 7fd7273a0740 1 -- start start 2024-03-21T11:35:09.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.946+0000 7fd7273a0740 1 Processor -- start 2024-03-21T11:35:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.946+0000 7fd7273a0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc0949f0 0x55cefc0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.946+0000 7fd7273a0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc095bc0 0x55cefc0b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.946+0000 7fd7273a0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc096580 0x55cefc0b7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.946+0000 7fd7273a0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cefc0ae5a0 con 0x55cefc095bc0 2024-03-21T11:35:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.946+0000 7fd7273a0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cefbfb50d0 con 0x55cefc0949f0 2024-03-21T11:35:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.946+0000 7fd7273a0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cefc0ae720 con 0x55cefc096580 2024-03-21T11:35:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.947+0000 7fd7273a0740 10 client.0 _ll_register_callbacks cb 0x55cefbfb9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.947+0000 7fd721c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc0949f0 0x55cefc0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.947+0000 7fd721c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc0949f0 0x55cefc0afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54672/0 (socket says 172.21.15.47:54672) 2024-03-21T11:35:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.947+0000 7fd721c00640 1 -- 172.21.15.47:0/3588686536 learned_addr learned my addr 172.21.15.47:0/3588686536 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.947+0000 7fd721200640 1 --2- 172.21.15.47:0/3588686536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc095bc0 0x55cefc0b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.947+0000 7fd722600640 1 --2- 172.21.15.47:0/3588686536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc096580 0x55cefc0b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.947+0000 7fd721c00640 1 -- 172.21.15.47:0/3588686536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc096580 msgr2=0x55cefc0b7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.947+0000 7fd721c00640 1 --2- 172.21.15.47:0/3588686536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc096580 0x55cefc0b7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.947+0000 7fd721c00640 1 -- 172.21.15.47:0/3588686536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc095bc0 msgr2=0x55cefc0b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.947+0000 7fd721c00640 1 --2- 172.21.15.47:0/3588686536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc095bc0 0x55cefc0b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.947+0000 7fd721c00640 1 -- 172.21.15.47:0/3588686536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cefbd0d3f0 con 0x55cefc0949f0 2024-03-21T11:35:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.947+0000 7fd722600640 1 --2- 172.21.15.47:0/3588686536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc096580 0x55cefc0b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.948+0000 7fd721c00640 1 --2- 172.21.15.47:0/3588686536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc0949f0 0x55cefc0afbf0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd7100096e0 tx=0x7fd710003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9686a0fce74e6614 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.948+0000 7fd703400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.948+0000 7fd703400640 1 -- 172.21.15.47:0/3588686536 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710027070 con 0x55cefc0949f0 2024-03-21T11:35:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.948+0000 7fd703400640 1 -- 172.21.15.47:0/3588686536 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd710005240 con 0x55cefc0949f0 2024-03-21T11:35:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.948+0000 7fd7273a0740 1 -- 172.21.15.47:0/3588686536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7100043d0 con 0x55cefc0949f0 2024-03-21T11:35:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.948+0000 7fd703400640 1 -- 172.21.15.47:0/3588686536 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd710033050 con 0x55cefc0949f0 2024-03-21T11:35:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.948+0000 7fd7273a0740 10 client.29465 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.948+0000 7fd7273a0740 1 -- 172.21.15.47:0/3588686536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cefbdd77d0 con 0x55cefc0949f0 2024-03-21T11:35:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.949+0000 7fd703400640 1 -- 172.21.15.47:0/3588686536 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7100043d0 con 0x55cefc0949f0 2024-03-21T11:35:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.949+0000 7fd703400640 1 -- 172.21.15.47:0/3588686536 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd710004c30 con 0x55cefc0949f0 2024-03-21T11:35:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.949+0000 7fd7273a0740 10 client.29465 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.949+0000 7fd7273a0740 10 client.29465 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:35:09.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.949+0000 7fd7273a0740 20 client.29465 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.949+0000 7fd7273a0740 1 --2- 172.21.15.47:0/3588686536 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55cefc0bb3f0 0x55cefc0bd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.949+0000 7fd7273a0740 4 client.29465 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.949+0000 7fd7273a0740 1 -- 172.21.15.47:0/3588686536 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cefc006800 con 0x55cefc0bb3f0 2024-03-21T11:35:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.949+0000 7fd721200640 1 --2- 172.21.15.47:0/3588686536 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55cefc0bb3f0 0x55cefc0bd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.950+0000 7fd721200640 1 --2- 172.21.15.47:0/3588686536 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55cefc0bb3f0 0x55cefc0bd840 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e3571221459c94d server_cookie=b10afd4d4ccea3b2 in_seq=0 out_seq=0 2024-03-21T11:35:09.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.950+0000 7fd703400640 10 client.29465 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:35:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.951+0000 7fd703400640 1 -- 172.21.15.47:0/3588686536 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cefc006800 con 0x55cefc0bb3f0 2024-03-21T11:35:09.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.951+0000 7fd703400640 10 client.29465 handle_command_reply: tid=0 2024-03-21T11:35:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.951+0000 7fd7273a0740 1 client.29465 shutdown 2024-03-21T11:35:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.951+0000 7fd7273a0740 1 -- 172.21.15.47:0/3588686536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc0949f0 msgr2=0x55cefc0afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.951+0000 7fd7273a0740 1 --2- 172.21.15.47:0/3588686536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc0949f0 0x55cefc0afbf0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd7100096e0 tx=0x7fd710003820 comp rx=0 tx=0).stop 2024-03-21T11:35:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.952+0000 7fd7273a0740 1 -- 172.21.15.47:0/3588686536 shutdown_connections 2024-03-21T11:35:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.952+0000 7fd7273a0740 1 --2- 172.21.15.47:0/3588686536 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55cefc0bb3f0 0x55cefc0bd840 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.952+0000 7fd7273a0740 1 --2- 172.21.15.47:0/3588686536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc096580 0x55cefc0b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.952+0000 7fd7273a0740 1 --2- 172.21.15.47:0/3588686536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc095bc0 0x55cefc0b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.952+0000 7fd7273a0740 1 --2- 172.21.15.47:0/3588686536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc0949f0 0x55cefc0afbf0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.952+0000 7fd7273a0740 1 -- 172.21.15.47:0/3588686536 >> 172.21.15.47:0/3588686536 conn(0x55cefc08fde0 msgr2=0x55cefc08f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.952+0000 7fd703400640 0 client.29465 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:35:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.952+0000 7fd7273a0740 1 -- 172.21.15.47:0/3588686536 shutdown_connections 2024-03-21T11:35:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.952+0000 7fd7273a0740 1 -- 172.21.15.47:0/3588686536 wait complete. 2024-03-21T11:35:09.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.952+0000 7fd7273a0740 20 client.29465 trim_cache size 0 max 16384 2024-03-21T11:35:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.969+0000 7fd7273a0740 1 Processor -- start 2024-03-21T11:35:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.969+0000 7fd7273a0740 1 -- start start 2024-03-21T11:35:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.969+0000 7fd7273a0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc0949f0 0x55cefc0c2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.969+0000 7fd7273a0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc095bc0 0x55cefc0b7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.969+0000 7fd7273a0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc096580 0x55cefc0c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.969+0000 7fd7273a0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1a38e0 con 0x55cefc095bc0 2024-03-21T11:35:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.969+0000 7fd7273a0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1a5260 con 0x55cefc096580 2024-03-21T11:35:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.969+0000 7fd7273a0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1a3f40 con 0x55cefc0949f0 2024-03-21T11:35:09.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.969+0000 7fd721c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc0949f0 0x55cefc0c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.969+0000 7fd722600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc096580 0x55cefc0c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.969+0000 7fd721c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc0949f0 0x55cefc0c2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49976/0 (socket says 172.21.15.47:49976) 2024-03-21T11:35:09.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.969+0000 7fd721c00640 1 -- 172.21.15.47:0/3718279044 learned_addr learned my addr 172.21.15.47:0/3718279044 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.969+0000 7fd721200640 1 --2- 172.21.15.47:0/3718279044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc095bc0 0x55cefc0b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.970+0000 7fd721c00640 1 -- 172.21.15.47:0/3718279044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc096580 msgr2=0x55cefc0c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.970+0000 7fd721c00640 1 --2- 172.21.15.47:0/3718279044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc096580 0x55cefc0c2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.970+0000 7fd721c00640 1 -- 172.21.15.47:0/3718279044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc095bc0 msgr2=0x55cefc0b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.970+0000 7fd721c00640 1 --2- 172.21.15.47:0/3718279044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc095bc0 0x55cefc0b7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.970+0000 7fd721c00640 1 -- 172.21.15.47:0/3718279044 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cefc095640 con 0x55cefc0949f0 2024-03-21T11:35:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.970+0000 7fd722600640 1 --2- 172.21.15.47:0/3718279044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc096580 0x55cefc0c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.970+0000 7fd721200640 1 --2- 172.21.15.47:0/3718279044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc095bc0 0x55cefc0b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.970+0000 7fd721c00640 1 --2- 172.21.15.47:0/3718279044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc0949f0 0x55cefc0c2100 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd7100058c0 tx=0x7fd7100041b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f340b82ceef80fa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.970+0000 7fd70e000640 1 -- 172.21.15.47:0/3718279044 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710030020 con 0x55cefc0949f0 2024-03-21T11:35:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.970+0000 7fd70e000640 1 -- 172.21.15.47:0/3718279044 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd710004290 con 0x55cefc0949f0 2024-03-21T11:35:09.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.970+0000 7fd70e000640 1 -- 172.21.15.47:0/3718279044 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710027040 con 0x55cefc0949f0 2024-03-21T11:35:09.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.971+0000 7fd7273a0740 1 -- 172.21.15.47:0/3718279044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc0949f0 msgr2=0x55cefc0c2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.971+0000 7fd7273a0740 1 --2- 172.21.15.47:0/3718279044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc0949f0 0x55cefc0c2100 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd7100058c0 tx=0x7fd7100041b0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.971+0000 7fd7273a0740 1 -- 172.21.15.47:0/3718279044 shutdown_connections 2024-03-21T11:35:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.972+0000 7fd7273a0740 1 --2- 172.21.15.47:0/3718279044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc096580 0x55cefc0c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.972+0000 7fd7273a0740 1 --2- 172.21.15.47:0/3718279044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc095bc0 0x55cefc0b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.972+0000 7fd7273a0740 1 --2- 172.21.15.47:0/3718279044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc0949f0 0x55cefc0c2100 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.972+0000 7fd7273a0740 1 -- 172.21.15.47:0/3718279044 >> 172.21.15.47:0/3718279044 conn(0x55cefc08fde0 msgr2=0x55cefc08f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.972+0000 7fd7273a0740 1 -- 172.21.15.47:0/3718279044 shutdown_connections 2024-03-21T11:35:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.972+0000 7fd7273a0740 1 -- 172.21.15.47:0/3718279044 wait complete. 2024-03-21T11:35:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.972+0000 7fd7273a0740 1 -- start start 2024-03-21T11:35:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.972+0000 7fd7273a0740 1 Processor -- start 2024-03-21T11:35:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd7273a0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc0949f0 0x55cefc0b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd7273a0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc095bc0 0x55cefc0b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd7273a0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc096580 0x55cefc0b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd7273a0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1a4c00 con 0x55cefc096580 2024-03-21T11:35:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd7273a0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1a45a0 con 0x55cefc0949f0 2024-03-21T11:35:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd7273a0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c073240 con 0x55cefc095bc0 2024-03-21T11:35:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd7273a0740 10 client.0 _ll_register_callbacks cb 0x7fd710009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd721200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc095bc0 0x55cefc0b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd721200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc095bc0 0x55cefc0b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49986/0 (socket says 172.21.15.47:49986) 2024-03-21T11:35:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd721c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc0949f0 0x55cefc0b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd721200640 1 -- 172.21.15.47:0/4160328028 learned_addr learned my addr 172.21.15.47:0/4160328028 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd722600640 1 --2- 172.21.15.47:0/4160328028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc096580 0x55cefc0b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd721200640 1 -- 172.21.15.47:0/4160328028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc0949f0 msgr2=0x55cefc0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd721200640 1 --2- 172.21.15.47:0/4160328028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc0949f0 0x55cefc0b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd721200640 1 -- 172.21.15.47:0/4160328028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc096580 msgr2=0x55cefc0b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd721200640 1 --2- 172.21.15.47:0/4160328028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc096580 0x55cefc0b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd721200640 1 -- 172.21.15.47:0/4160328028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cefc0b6a60 con 0x55cefc095bc0 2024-03-21T11:35:09.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd722600640 1 --2- 172.21.15.47:0/4160328028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc096580 0x55cefc0b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:09.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.973+0000 7fd721c00640 1 --2- 172.21.15.47:0/4160328028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc0949f0 0x55cefc0b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:09.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.974+0000 7fd721200640 1 --2- 172.21.15.47:0/4160328028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc095bc0 0x55cefc0b3450 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd70802c710 tx=0x7fd70802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8da2eb574f882a9f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:09.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.974+0000 7fd703400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:09.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.974+0000 7fd703400640 1 -- 172.21.15.47:0/4160328028 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708005260 con 0x55cefc095bc0 2024-03-21T11:35:09.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.974+0000 7fd7273a0740 1 -- 172.21.15.47:0/4160328028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd710004290 con 0x55cefc095bc0 2024-03-21T11:35:09.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.974+0000 7fd703400640 1 -- 172.21.15.47:0/4160328028 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd70803a040 con 0x55cefc095bc0 2024-03-21T11:35:09.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.974+0000 7fd703400640 1 -- 172.21.15.47:0/4160328028 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd708066050 con 0x55cefc095bc0 2024-03-21T11:35:09.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.974+0000 7fd7273a0740 10 client.29494 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:09.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.974+0000 7fd7273a0740 1 -- 172.21.15.47:0/4160328028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cefc117470 con 0x55cefc095bc0 2024-03-21T11:35:09.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.975+0000 7fd703400640 1 -- 172.21.15.47:0/4160328028 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd710004290 con 0x55cefc095bc0 2024-03-21T11:35:09.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.975+0000 7fd703400640 1 -- 172.21.15.47:0/4160328028 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd7080035f0 con 0x55cefc095bc0 2024-03-21T11:35:09.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.975+0000 7fd7273a0740 10 client.29494 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:09.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.975+0000 7fd7273a0740 10 client.29494 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:35:09.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.975+0000 7fd7273a0740 20 client.29494 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:09.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.975+0000 7fd7273a0740 1 --2- 172.21.15.47:0/4160328028 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55cefc117e50 0x55cefc11a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:09.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.975+0000 7fd7273a0740 4 client.29494 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:35:09.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.975+0000 7fd7273a0740 1 -- 172.21.15.47:0/4160328028 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cefc000b50 con 0x55cefc117e50 2024-03-21T11:35:09.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.975+0000 7fd721c00640 1 --2- 172.21.15.47:0/4160328028 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55cefc117e50 0x55cefc11a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:09.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.976+0000 7fd721c00640 1 --2- 172.21.15.47:0/4160328028 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55cefc117e50 0x55cefc11a270 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e46cb64928b7c88b server_cookie=9f8c46d7e6cf71a3 in_seq=0 out_seq=0 2024-03-21T11:35:09.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.976+0000 7fd703400640 10 client.29494 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:35:09.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.976+0000 7fd703400640 1 -- 172.21.15.47:0/4160328028 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cefc000b50 con 0x55cefc117e50 2024-03-21T11:35:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.976+0000 7fd703400640 10 client.29494 handle_command_reply: tid=0 2024-03-21T11:35:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.976+0000 7fd7273a0740 1 client.29494 shutdown 2024-03-21T11:35:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.976+0000 7fd7273a0740 1 -- 172.21.15.47:0/4160328028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc095bc0 msgr2=0x55cefc0b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.976+0000 7fd7273a0740 1 --2- 172.21.15.47:0/4160328028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc095bc0 0x55cefc0b3450 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd70802c710 tx=0x7fd70802bc30 comp rx=0 tx=0).stop 2024-03-21T11:35:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.976+0000 7fd7273a0740 1 -- 172.21.15.47:0/4160328028 shutdown_connections 2024-03-21T11:35:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.976+0000 7fd7273a0740 1 --2- 172.21.15.47:0/4160328028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cefc096580 0x55cefc0b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.976+0000 7fd7273a0740 1 --2- 172.21.15.47:0/4160328028 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55cefc117e50 0x55cefc11a270 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.976+0000 7fd7273a0740 1 --2- 172.21.15.47:0/4160328028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cefc095bc0 0x55cefc0b3450 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.976+0000 7fd7273a0740 1 --2- 172.21.15.47:0/4160328028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cefc0949f0 0x55cefc0b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.976+0000 7fd7273a0740 1 -- 172.21.15.47:0/4160328028 >> 172.21.15.47:0/4160328028 conn(0x55cefc08fde0 msgr2=0x55cefc098df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.976+0000 7fd703400640 0 client.29494 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:35:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.977+0000 7fd7273a0740 1 -- 172.21.15.47:0/4160328028 shutdown_connections 2024-03-21T11:35:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.977+0000 7fd7273a0740 1 -- 172.21.15.47:0/4160328028 wait complete. 2024-03-21T11:35:09.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.977+0000 7fd7273a0740 20 client.29494 trim_cache size 0 max 16384 2024-03-21T11:35:09.982 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:35:09.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.977+0000 7fd723000640 1 -- 172.21.15.47:0/1584834578 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7040a2f20 msgr2=0x7fd7040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.977+0000 7fd723000640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7040a2f20 0x7fd7040a53e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd70802c8e0 tx=0x7fd708005b70 comp rx=0 tx=0).stop 2024-03-21T11:35:09.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.977+0000 7fd723000640 1 -- 172.21.15.47:0/1584834578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd71c101150 msgr2=0x7fd71c1a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:09.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.977+0000 7fd723000640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd71c101150 0x7fd71c1a1000 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd71400e940 tx=0x7fd71400ee00 comp rx=0 tx=0).stop 2024-03-21T11:35:09.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.978+0000 7fd723000640 1 -- 172.21.15.47:0/1584834578 shutdown_connections 2024-03-21T11:35:09.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.978+0000 7fd723000640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7040a2f20 0x7fd7040a53e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.978+0000 7fd723000640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd71c101150 0x7fd71c1a1000 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.978+0000 7fd723000640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd71c1007b0 0x7fd71c199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.978+0000 7fd723000640 1 --2- 172.21.15.47:0/1584834578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd71c0ffe50 0x7fd71c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:09.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.978+0000 7fd723000640 1 -- 172.21.15.47:0/1584834578 >> 172.21.15.47:0/1584834578 conn(0x7fd71c0fba10 msgr2=0x7fd71c0fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:09.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.978+0000 7fd723000640 1 -- 172.21.15.47:0/1584834578 shutdown_connections 2024-03-21T11:35:09.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:09.978+0000 7fd723000640 1 -- 172.21.15.47:0/1584834578 wait complete. 2024-03-21T11:35:09.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:09 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1892539867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:35:09.998 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:35:09.998 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:35:10.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:09 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1892539867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:35:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.215+0000 7f0789c00640 1 -- 172.21.15.47:0/2493977729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07840740a0 msgr2=0x7f0784074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.215+0000 7f0789c00640 1 --2- 172.21.15.47:0/2493977729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07840740a0 0x7f0784074500 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0770009900 tx=0x7f077002bcc0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.216+0000 7f0789c00640 1 -- 172.21.15.47:0/2493977729 shutdown_connections 2024-03-21T11:35:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.216+0000 7f0789c00640 1 --2- 172.21.15.47:0/2493977729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0784109410 0x7f078410b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.216+0000 7f0789c00640 1 --2- 172.21.15.47:0/2493977729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07840740a0 0x7f0784074500 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.216+0000 7f0789c00640 1 --2- 172.21.15.47:0/2493977729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0784075680 0x7f0784073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.216+0000 7f0789c00640 1 -- 172.21.15.47:0/2493977729 >> 172.21.15.47:0/2493977729 conn(0x7f07840fcb10 msgr2=0x7f07840fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.216+0000 7f0789c00640 1 -- 172.21.15.47:0/2493977729 shutdown_connections 2024-03-21T11:35:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.216+0000 7f0789c00640 1 -- 172.21.15.47:0/2493977729 wait complete. 2024-03-21T11:35:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.217+0000 7f0789c00640 1 Processor -- start 2024-03-21T11:35:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.217+0000 7f0789c00640 1 -- start start 2024-03-21T11:35:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.217+0000 7f0789c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07840740a0 0x7f0784199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.217+0000 7f0789c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0784075680 0x7f0784199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.217+0000 7f0789c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0784109410 0x7f07841a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.217+0000 7f0789c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078410f430 con 0x7f07840740a0 2024-03-21T11:35:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.217+0000 7f0789c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078410f2b0 con 0x7f0784109410 2024-03-21T11:35:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.217+0000 7f0789c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078410f5b0 con 0x7f0784075680 2024-03-21T11:35:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.218+0000 7f0783e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07840740a0 0x7f0784199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.218+0000 7f0783400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0784075680 0x7f0784199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.218+0000 7f0783400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0784075680 0x7f0784199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50008/0 (socket says 172.21.15.47:50008) 2024-03-21T11:35:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.218+0000 7f0783400640 1 -- 172.21.15.47:0/336346890 learned_addr learned my addr 172.21.15.47:0/336346890 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.218+0000 7f0783e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07840740a0 0x7f0784199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55926/0 (socket says 172.21.15.47:55926) 2024-03-21T11:35:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.218+0000 7f0789200640 1 --2- 172.21.15.47:0/336346890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0784109410 0x7f07841a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.218+0000 7f0783400640 1 -- 172.21.15.47:0/336346890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0784109410 msgr2=0x7f07841a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.218+0000 7f0783400640 1 --2- 172.21.15.47:0/336346890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0784109410 0x7f07841a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.218+0000 7f0783400640 1 -- 172.21.15.47:0/336346890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07840740a0 msgr2=0x7f0784199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.218+0000 7f0783400640 1 --2- 172.21.15.47:0/336346890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07840740a0 0x7f0784199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.218+0000 7f0783400640 1 -- 172.21.15.47:0/336346890 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07841a1680 con 0x7f0784075680 2024-03-21T11:35:10.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.219+0000 7f0783400640 1 --2- 172.21.15.47:0/336346890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0784075680 0x7f0784199fa0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0770005ab0 tx=0x7f077002c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:10.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.219+0000 7f0780c00640 1 -- 172.21.15.47:0/336346890 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770043020 con 0x7f0784075680 2024-03-21T11:35:10.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.219+0000 7f0780c00640 1 -- 172.21.15.47:0/336346890 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07700054e0 con 0x7f0784075680 2024-03-21T11:35:10.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.219+0000 7f0789c00640 1 -- 172.21.15.47:0/336346890 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07841a1910 con 0x7f0784075680 2024-03-21T11:35:10.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.219+0000 7f0780c00640 1 -- 172.21.15.47:0/336346890 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770003ca0 con 0x7f0784075680 2024-03-21T11:35:10.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.219+0000 7f0789c00640 1 -- 172.21.15.47:0/336346890 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07841a1e60 con 0x7f0784075680 2024-03-21T11:35:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.225+0000 7f078df98740 1 Processor -- start 2024-03-21T11:35:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.225+0000 7f078df98740 1 -- start start 2024-03-21T11:35:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.225+0000 7f0780c00640 1 -- 172.21.15.47:0/336346890 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0770046020 con 0x7f0784075680 2024-03-21T11:35:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.225+0000 7f078df98740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e69f0 0x564cf92e6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.225+0000 7f078df98740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e7bc0 0x564cf92e8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.225+0000 7f078df98740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e8580 0x564cf92eee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.225+0000 7f078df98740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf924b2c0 con 0x564cf92e7bc0 2024-03-21T11:35:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.225+0000 7f078df98740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf91f3c90 con 0x564cf92e8580 2024-03-21T11:35:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.225+0000 7f078df98740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf9247360 con 0x564cf92e69f0 2024-03-21T11:35:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.225+0000 7f0789200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e69f0 0x564cf92e6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.225+0000 7f0783e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e7bc0 0x564cf92e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.225+0000 7f0783400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e8580 0x564cf92eee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.225+0000 7f0789200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e69f0 0x564cf92e6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50018/0 (socket says 172.21.15.47:50018) 2024-03-21T11:35:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.225+0000 7f0789200640 1 -- 172.21.15.47:0/2306144616 learned_addr learned my addr 172.21.15.47:0/2306144616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.226+0000 7f0789200640 1 -- 172.21.15.47:0/2306144616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e8580 msgr2=0x564cf92eee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.226+0000 7f0789200640 1 --2- 172.21.15.47:0/2306144616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e8580 0x564cf92eee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.226+0000 7f0789200640 1 -- 172.21.15.47:0/2306144616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e7bc0 msgr2=0x564cf92e8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.226+0000 7f0789200640 1 --2- 172.21.15.47:0/2306144616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e7bc0 0x564cf92e8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.226+0000 7f0789200640 1 -- 172.21.15.47:0/2306144616 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cf9218080 con 0x564cf92e69f0 2024-03-21T11:35:10.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.226+0000 7f0789200640 1 --2- 172.21.15.47:0/2306144616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e69f0 0x564cf92e6e10 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f077c009f50 tx=0x7f077c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82a02149b3548d47 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:10.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.226+0000 7f0779600640 1 -- 172.21.15.47:0/2306144616 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c004d20 con 0x564cf92e69f0 2024-03-21T11:35:10.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.226+0000 7f0779600640 1 -- 172.21.15.47:0/2306144616 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f077c004050 con 0x564cf92e69f0 2024-03-21T11:35:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.227+0000 7f0779600640 1 -- 172.21.15.47:0/2306144616 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c005280 con 0x564cf92e69f0 2024-03-21T11:35:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.227+0000 7f0780c00640 1 --2- 172.21.15.47:0/336346890 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07680a2f20 0x7f07680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.227+0000 7f0780c00640 1 -- 172.21.15.47:0/336346890 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f07700f18a0 con 0x7f0784075680 2024-03-21T11:35:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.227+0000 7f0783e00640 1 --2- 172.21.15.47:0/336346890 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07680a2f20 0x7f07680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.227+0000 7f078df98740 1 -- 172.21.15.47:0/2306144616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e69f0 msgr2=0x564cf92e6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.227+0000 7f078df98740 1 --2- 172.21.15.47:0/2306144616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e69f0 0x564cf92e6e10 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f077c009f50 tx=0x7f077c0118d0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.228+0000 7f078df98740 1 -- 172.21.15.47:0/2306144616 shutdown_connections 2024-03-21T11:35:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.228+0000 7f078df98740 1 --2- 172.21.15.47:0/2306144616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e8580 0x564cf92eee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.228+0000 7f078df98740 1 --2- 172.21.15.47:0/2306144616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e7bc0 0x564cf92e8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.228+0000 7f078df98740 1 --2- 172.21.15.47:0/2306144616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e69f0 0x564cf92e6e10 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.228+0000 7f078df98740 1 -- 172.21.15.47:0/2306144616 >> 172.21.15.47:0/2306144616 conn(0x564cf92e1de0 msgr2=0x564cf92e21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.228+0000 7f078df98740 1 -- 172.21.15.47:0/2306144616 shutdown_connections 2024-03-21T11:35:10.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.228+0000 7f078df98740 1 -- 172.21.15.47:0/2306144616 wait complete. 2024-03-21T11:35:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.228+0000 7f0783e00640 1 --2- 172.21.15.47:0/336346890 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07680a2f20 0x7f07680a53e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0774005d00 tx=0x7f077400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.228+0000 7f078df98740 1 -- start start 2024-03-21T11:35:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.228+0000 7f078df98740 1 Processor -- start 2024-03-21T11:35:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.229+0000 7f078df98740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e69f0 0x564cf92f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.229+0000 7f078df98740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e7bc0 0x564cf92f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.229+0000 7f078df98740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e8580 0x564cf92f8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.229+0000 7f078df98740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf92ef590 con 0x564cf92e7bc0 2024-03-21T11:35:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.229+0000 7f078df98740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf92070d0 con 0x564cf92e69f0 2024-03-21T11:35:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.229+0000 7f078df98740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf92ef710 con 0x564cf92e8580 2024-03-21T11:35:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.229+0000 7f078df98740 10 client.0 _ll_register_callbacks cb 0x564cf920be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.229+0000 7f0783e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e69f0 0x564cf92f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.229+0000 7f0783e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e69f0 0x564cf92f0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54738/0 (socket says 172.21.15.47:54738) 2024-03-21T11:35:10.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.229+0000 7f0783e00640 1 -- 172.21.15.47:0/2817034539 learned_addr learned my addr 172.21.15.47:0/2817034539 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.229+0000 7f0783400640 1 --2- 172.21.15.47:0/2817034539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e7bc0 0x564cf92f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.229+0000 7f0789200640 1 --2- 172.21.15.47:0/2817034539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e8580 0x564cf92f8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.230+0000 7f0783e00640 1 -- 172.21.15.47:0/2817034539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e8580 msgr2=0x564cf92f8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.230+0000 7f0783e00640 1 --2- 172.21.15.47:0/2817034539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e8580 0x564cf92f8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.230+0000 7f0783e00640 1 -- 172.21.15.47:0/2817034539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e7bc0 msgr2=0x564cf92f11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.230+0000 7f0783e00640 1 --2- 172.21.15.47:0/2817034539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e7bc0 0x564cf92f11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.230+0000 7f0783e00640 1 -- 172.21.15.47:0/2817034539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cf8f5f3f0 con 0x564cf92e69f0 2024-03-21T11:35:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.230+0000 7f0783400640 1 --2- 172.21.15.47:0/2817034539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e7bc0 0x564cf92f11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.230+0000 7f0783e00640 1 --2- 172.21.15.47:0/2817034539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e69f0 0x564cf92f0c90 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f077401f5a0 tx=0x7f0774004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b039c1db7ee81de7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.230+0000 7f076e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.230+0000 7f076e000640 1 -- 172.21.15.47:0/2817034539 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774005020 con 0x564cf92e69f0 2024-03-21T11:35:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.230+0000 7f078df98740 1 -- 172.21.15.47:0/2817034539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f077c004050 con 0x564cf92e69f0 2024-03-21T11:35:10.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.230+0000 7f076e000640 1 -- 172.21.15.47:0/2817034539 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0774017030 con 0x564cf92e69f0 2024-03-21T11:35:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.231+0000 7f076e000640 1 -- 172.21.15.47:0/2817034539 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0774032050 con 0x564cf92e69f0 2024-03-21T11:35:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.231+0000 7f078df98740 10 client.29483 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.231+0000 7f078df98740 1 -- 172.21.15.47:0/2817034539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564cf90297d0 con 0x564cf92e69f0 2024-03-21T11:35:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.231+0000 7f076e000640 1 -- 172.21.15.47:0/2817034539 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f077c004050 con 0x564cf92e69f0 2024-03-21T11:35:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.231+0000 7f076e000640 1 -- 172.21.15.47:0/2817034539 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0774010700 con 0x564cf92e69f0 2024-03-21T11:35:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.231+0000 7f078df98740 10 client.29483 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.231+0000 7f078df98740 10 client.29483 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:35:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.231+0000 7f078df98740 20 client.29483 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.231+0000 7f078df98740 1 --2- 172.21.15.47:0/2817034539 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564cf92fc750 0x564cf92febf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.231+0000 7f078df98740 4 client.29483 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.231+0000 7f078df98740 1 -- 172.21.15.47:0/2817034539 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cf9258800 con 0x564cf92fc750 2024-03-21T11:35:10.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.232+0000 7f0783400640 1 --2- 172.21.15.47:0/2817034539 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564cf92fc750 0x564cf92febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.232+0000 7f0783400640 1 --2- 172.21.15.47:0/2817034539 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564cf92fc750 0x564cf92febf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=248af4342b21dbd0 server_cookie=19755e8deb505ad0 in_seq=0 out_seq=0 2024-03-21T11:35:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.232+0000 7f076e000640 10 client.29483 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:35:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.233+0000 7f076e000640 1 -- 172.21.15.47:0/2817034539 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564cf9258800 con 0x564cf92fc750 2024-03-21T11:35:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.233+0000 7f076e000640 10 client.29483 handle_command_reply: tid=0 2024-03-21T11:35:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.233+0000 7f078df98740 1 client.29483 shutdown 2024-03-21T11:35:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.233+0000 7f078df98740 1 -- 172.21.15.47:0/2817034539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e69f0 msgr2=0x564cf92f0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.233+0000 7f078df98740 1 --2- 172.21.15.47:0/2817034539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e69f0 0x564cf92f0c90 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f077401f5a0 tx=0x7f0774004730 comp rx=0 tx=0).stop 2024-03-21T11:35:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.233+0000 7f078df98740 1 -- 172.21.15.47:0/2817034539 shutdown_connections 2024-03-21T11:35:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.233+0000 7f078df98740 1 --2- 172.21.15.47:0/2817034539 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564cf92fc750 0x564cf92febf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.233+0000 7f078df98740 1 --2- 172.21.15.47:0/2817034539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e8580 0x564cf92f8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.233+0000 7f076e000640 0 client.29483 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:35:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.233+0000 7f078df98740 1 --2- 172.21.15.47:0/2817034539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e7bc0 0x564cf92f11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.233+0000 7f078df98740 1 --2- 172.21.15.47:0/2817034539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e69f0 0x564cf92f0c90 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.233+0000 7f078df98740 1 -- 172.21.15.47:0/2817034539 >> 172.21.15.47:0/2817034539 conn(0x564cf92e1de0 msgr2=0x564cf92e1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:10.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.234+0000 7f078df98740 1 -- 172.21.15.47:0/2817034539 shutdown_connections 2024-03-21T11:35:10.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.234+0000 7f078df98740 1 -- 172.21.15.47:0/2817034539 wait complete. 2024-03-21T11:35:10.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.234+0000 7f078df98740 20 client.29483 trim_cache size 0 max 16384 2024-03-21T11:35:10.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.249+0000 7f078df98740 1 Processor -- start 2024-03-21T11:35:10.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.250+0000 7f078df98740 1 -- start start 2024-03-21T11:35:10.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.250+0000 7f078df98740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e69f0 0x564cf92e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.250+0000 7f078df98740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e7bc0 0x564cf92f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.250+0000 7f078df98740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e8580 0x564cf93034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.250+0000 7f078df98740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841abff0 con 0x564cf92e69f0 2024-03-21T11:35:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.250+0000 7f078df98740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841ad970 con 0x564cf92e8580 2024-03-21T11:35:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.250+0000 7f078df98740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841ac650 con 0x564cf92e7bc0 2024-03-21T11:35:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.250+0000 7f0789200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e8580 0x564cf93034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.250+0000 7f0783400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e7bc0 0x564cf92f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.250+0000 7f0789200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e8580 0x564cf93034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54740/0 (socket says 172.21.15.47:54740) 2024-03-21T11:35:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.250+0000 7f0783400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e7bc0 0x564cf92f81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50026/0 (socket says 172.21.15.47:50026) 2024-03-21T11:35:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.250+0000 7f0789200640 1 -- 172.21.15.47:0/303980745 learned_addr learned my addr 172.21.15.47:0/303980745 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.250+0000 7f0783e00640 1 --2- 172.21.15.47:0/303980745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e69f0 0x564cf92e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.251+0000 7f0789200640 1 -- 172.21.15.47:0/303980745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e7bc0 msgr2=0x564cf92f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.251+0000 7f0789200640 1 --2- 172.21.15.47:0/303980745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e7bc0 0x564cf92f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.251+0000 7f0789200640 1 -- 172.21.15.47:0/303980745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e69f0 msgr2=0x564cf92e0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.251+0000 7f0789200640 1 --2- 172.21.15.47:0/303980745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e69f0 0x564cf92e0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.251+0000 7f0789200640 1 -- 172.21.15.47:0/303980745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cf92e65f0 con 0x564cf92e8580 2024-03-21T11:35:10.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.251+0000 7f0783e00640 1 --2- 172.21.15.47:0/303980745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e69f0 0x564cf92e0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:10.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.251+0000 7f0783400640 1 --2- 172.21.15.47:0/303980745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e7bc0 0x564cf92f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:10.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.251+0000 7f0789200640 1 --2- 172.21.15.47:0/303980745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e8580 0x564cf93034b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f077c009f50 tx=0x7f077c009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae8fd9b191fd820e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:10.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.251+0000 7f0778c00640 1 -- 172.21.15.47:0/303980745 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c003b70 con 0x564cf92e8580 2024-03-21T11:35:10.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.251+0000 7f0778c00640 1 -- 172.21.15.47:0/303980745 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f077c005bd0 con 0x564cf92e8580 2024-03-21T11:35:10.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.251+0000 7f0778c00640 1 -- 172.21.15.47:0/303980745 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c00f7b0 con 0x564cf92e8580 2024-03-21T11:35:10.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.252+0000 7f078df98740 1 -- 172.21.15.47:0/303980745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e8580 msgr2=0x564cf93034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.252+0000 7f078df98740 1 --2- 172.21.15.47:0/303980745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e8580 0x564cf93034b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f077c009f50 tx=0x7f077c009c50 comp rx=0 tx=0).stop 2024-03-21T11:35:10.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.252+0000 7f078df98740 1 -- 172.21.15.47:0/303980745 shutdown_connections 2024-03-21T11:35:10.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.252+0000 7f078df98740 1 --2- 172.21.15.47:0/303980745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e8580 0x564cf93034b0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.252+0000 7f078df98740 1 --2- 172.21.15.47:0/303980745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e7bc0 0x564cf92f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.252+0000 7f078df98740 1 --2- 172.21.15.47:0/303980745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e69f0 0x564cf92e0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.252+0000 7f078df98740 1 -- 172.21.15.47:0/303980745 >> 172.21.15.47:0/303980745 conn(0x564cf92e1de0 msgr2=0x564cf92e08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:10.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.252+0000 7f078df98740 1 -- 172.21.15.47:0/303980745 shutdown_connections 2024-03-21T11:35:10.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.252+0000 7f078df98740 1 -- 172.21.15.47:0/303980745 wait complete. 2024-03-21T11:35:10.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.253+0000 7f078df98740 1 -- start start 2024-03-21T11:35:10.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.253+0000 7f078df98740 1 Processor -- start 2024-03-21T11:35:10.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.253+0000 7f078df98740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e69f0 0x564cf92f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.253+0000 7f078df98740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e7bc0 0x564cf92f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.253+0000 7f078df98740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e8580 0x564cf92f4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.253+0000 7f078df98740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841ad310 con 0x564cf92e8580 2024-03-21T11:35:10.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.253+0000 7f078df98740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841accb0 con 0x564cf92e69f0 2024-03-21T11:35:10.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.253+0000 7f078df98740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07841aa1b0 con 0x564cf92e7bc0 2024-03-21T11:35:10.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.253+0000 7f078df98740 10 client.0 _ll_register_callbacks cb 0x7f077400be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:10.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.254+0000 7f0783e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e69f0 0x564cf92f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.254+0000 7f0783e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e69f0 0x564cf92f79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54746/0 (socket says 172.21.15.47:54746) 2024-03-21T11:35:10.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.254+0000 7f0783e00640 1 -- 172.21.15.47:0/1167242510 learned_addr learned my addr 172.21.15.47:0/1167242510 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:10.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.254+0000 7f0783400640 1 --2- 172.21.15.47:0/1167242510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e7bc0 0x564cf92f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.254+0000 7f0789200640 1 --2- 172.21.15.47:0/1167242510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e8580 0x564cf92f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.254+0000 7f0783e00640 1 -- 172.21.15.47:0/1167242510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e7bc0 msgr2=0x564cf92f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:10.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.254+0000 7f0783e00640 1 --2- 172.21.15.47:0/1167242510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e7bc0 0x564cf92f7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.254+0000 7f0783e00640 1 -- 172.21.15.47:0/1167242510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e8580 msgr2=0x564cf92f4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:10.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.254+0000 7f0783e00640 1 --2- 172.21.15.47:0/1167242510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e8580 0x564cf92f4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.254+0000 7f0783e00640 1 -- 172.21.15.47:0/1167242510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cf92f4ad0 con 0x564cf92e69f0 2024-03-21T11:35:10.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.254+0000 7f0783e00640 1 --2- 172.21.15.47:0/1167242510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e69f0 0x564cf92f79c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0774018040 tx=0x7f0774010d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=800adbeb91af6375 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:10.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.254+0000 7f076e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:10.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.254+0000 7f076e000640 1 -- 172.21.15.47:0/1167242510 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077403b060 con 0x564cf92e69f0 2024-03-21T11:35:10.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.255+0000 7f076e000640 1 -- 172.21.15.47:0/1167242510 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f077401f240 con 0x564cf92e69f0 2024-03-21T11:35:10.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.255+0000 7f078df98740 1 -- 172.21.15.47:0/1167242510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f077c005bd0 con 0x564cf92e69f0 2024-03-21T11:35:10.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.255+0000 7f076e000640 1 -- 172.21.15.47:0/1167242510 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0774028450 con 0x564cf92e69f0 2024-03-21T11:35:10.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.255+0000 7f078df98740 10 client.29489 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:10.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.255+0000 7f078df98740 1 -- 172.21.15.47:0/1167242510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564cf9358c20 con 0x564cf92e69f0 2024-03-21T11:35:10.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.255+0000 7f076e000640 1 -- 172.21.15.47:0/1167242510 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f077c005bd0 con 0x564cf92e69f0 2024-03-21T11:35:10.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.255+0000 7f076e000640 1 -- 172.21.15.47:0/1167242510 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f077401e540 con 0x564cf92e69f0 2024-03-21T11:35:10.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.255+0000 7f078df98740 10 client.29489 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:10.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.256+0000 7f078df98740 10 client.29489 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:35:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.256+0000 7f078df98740 20 client.29489 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.256+0000 7f078df98740 1 --2- 172.21.15.47:0/1167242510 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564cf9359580 0x564cf935b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.256+0000 7f078df98740 4 client.29489 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:35:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.256+0000 7f078df98740 1 -- 172.21.15.47:0/1167242510 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564cf9252b50 con 0x564cf9359580 2024-03-21T11:35:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.256+0000 7f0783400640 1 --2- 172.21.15.47:0/1167242510 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564cf9359580 0x564cf935b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.256+0000 7f0783400640 1 --2- 172.21.15.47:0/1167242510 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564cf9359580 0x564cf935b9a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3121c61e7c4cd24 server_cookie=b1ef3e30dbff5ad in_seq=0 out_seq=0 2024-03-21T11:35:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.256+0000 7f076e000640 10 client.29489 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:35:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.257+0000 7f076e000640 1 -- 172.21.15.47:0/1167242510 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564cf9252b50 con 0x564cf9359580 2024-03-21T11:35:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.257+0000 7f076e000640 10 client.29489 handle_command_reply: tid=0 2024-03-21T11:35:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.257+0000 7f078df98740 1 client.29489 shutdown 2024-03-21T11:35:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.257+0000 7f078df98740 1 -- 172.21.15.47:0/1167242510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e69f0 msgr2=0x564cf92f79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.257+0000 7f078df98740 1 --2- 172.21.15.47:0/1167242510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e69f0 0x564cf92f79c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0774018040 tx=0x7f0774010d60 comp rx=0 tx=0).stop 2024-03-21T11:35:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.257+0000 7f078df98740 1 -- 172.21.15.47:0/1167242510 shutdown_connections 2024-03-21T11:35:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.257+0000 7f078df98740 1 --2- 172.21.15.47:0/1167242510 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x564cf9359580 0x564cf935b9a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.257+0000 7f078df98740 1 --2- 172.21.15.47:0/1167242510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564cf92e8580 0x564cf92f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.257+0000 7f078df98740 1 --2- 172.21.15.47:0/1167242510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cf92e7bc0 0x564cf92f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.257+0000 7f078df98740 1 --2- 172.21.15.47:0/1167242510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cf92e69f0 0x564cf92f79c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.257+0000 7f076e000640 0 client.29489 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:35:10.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.257+0000 7f078df98740 1 -- 172.21.15.47:0/1167242510 >> 172.21.15.47:0/1167242510 conn(0x564cf92e1de0 msgr2=0x564cf92e1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:10.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.257+0000 7f078df98740 1 -- 172.21.15.47:0/1167242510 shutdown_connections 2024-03-21T11:35:10.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.257+0000 7f078df98740 1 -- 172.21.15.47:0/1167242510 wait complete. 2024-03-21T11:35:10.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.258+0000 7f078df98740 20 client.29489 trim_cache size 0 max 16384 2024-03-21T11:35:10.262 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:35:10.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.258+0000 7f0789c00640 1 -- 172.21.15.47:0/336346890 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07680a2f20 msgr2=0x7f07680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.258+0000 7f0789c00640 1 --2- 172.21.15.47:0/336346890 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07680a2f20 0x7f07680a53e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0774005d00 tx=0x7f077400b410 comp rx=0 tx=0).stop 2024-03-21T11:35:10.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.258+0000 7f0789c00640 1 -- 172.21.15.47:0/336346890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0784075680 msgr2=0x7f0784199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.258+0000 7f0789c00640 1 --2- 172.21.15.47:0/336346890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0784075680 0x7f0784199fa0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0770005ab0 tx=0x7f077002c750 comp rx=0 tx=0).stop 2024-03-21T11:35:10.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.259+0000 7f0789c00640 1 -- 172.21.15.47:0/336346890 shutdown_connections 2024-03-21T11:35:10.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.259+0000 7f0789c00640 1 --2- 172.21.15.47:0/336346890 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07680a2f20 0x7f07680a53e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.259+0000 7f0789c00640 1 --2- 172.21.15.47:0/336346890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0784109410 0x7f07841a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.259+0000 7f0789c00640 1 --2- 172.21.15.47:0/336346890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0784075680 0x7f0784199fa0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.259+0000 7f0789c00640 1 --2- 172.21.15.47:0/336346890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07840740a0 0x7f0784199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.259+0000 7f0789c00640 1 -- 172.21.15.47:0/336346890 >> 172.21.15.47:0/336346890 conn(0x7f07840fcb10 msgr2=0x7f07841053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:10.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.259+0000 7f0789c00640 1 -- 172.21.15.47:0/336346890 shutdown_connections 2024-03-21T11:35:10.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.259+0000 7f0789c00640 1 -- 172.21.15.47:0/336346890 wait complete. 2024-03-21T11:35:10.278 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:35:10.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:35:10.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.501+0000 7f4e1e600640 1 -- 172.21.15.47:0/2148237003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e18104ab0 msgr2=0x7f4e18104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.501+0000 7f4e1e600640 1 --2- 172.21.15.47:0/2148237003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e18104ab0 0x7f4e18104f30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009900 tx=0x7f4e0002bc40 comp rx=0 tx=0).stop 2024-03-21T11:35:10.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.501+0000 7f4e1e600640 1 -- 172.21.15.47:0/2148237003 shutdown_connections 2024-03-21T11:35:10.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.501+0000 7f4e1e600640 1 --2- 172.21.15.47:0/2148237003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e18105470 0x7f4e1810bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.501+0000 7f4e1e600640 1 --2- 172.21.15.47:0/2148237003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e18104ab0 0x7f4e18104f30 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.501+0000 7f4e1e600640 1 --2- 172.21.15.47:0/2148237003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e181038c0 0x7f4e18103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.501+0000 7f4e1e600640 1 -- 172.21.15.47:0/2148237003 >> 172.21.15.47:0/2148237003 conn(0x7f4e180fee50 msgr2=0x7f4e18101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:10.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.501+0000 7f4e1e600640 1 -- 172.21.15.47:0/2148237003 shutdown_connections 2024-03-21T11:35:10.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.501+0000 7f4e1e600640 1 -- 172.21.15.47:0/2148237003 wait complete. 2024-03-21T11:35:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.502+0000 7f4e1e600640 1 Processor -- start 2024-03-21T11:35:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.502+0000 7f4e1e600640 1 -- start start 2024-03-21T11:35:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.503+0000 7f4e1e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e181038c0 0x7f4e18079450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.503+0000 7f4e1e600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e18104ab0 0x7f4e18079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.503+0000 7f4e1e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e18105470 0x7f4e18075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.503+0000 7f4e1e600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1810f570 con 0x7f4e18104ab0 2024-03-21T11:35:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.503+0000 7f4e1e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1810f3f0 con 0x7f4e181038c0 2024-03-21T11:35:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.503+0000 7f4e1e600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1810f6f0 con 0x7f4e18105470 2024-03-21T11:35:10.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.503+0000 7f4e17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e181038c0 0x7f4e18079450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.503+0000 7f4e17400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e181038c0 0x7f4e18079450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54770/0 (socket says 172.21.15.47:54770) 2024-03-21T11:35:10.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.503+0000 7f4e17400640 1 -- 172.21.15.47:0/425855589 learned_addr learned my addr 172.21.15.47:0/425855589 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:10.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.503+0000 7f4e17e00640 1 --2- 172.21.15.47:0/425855589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e18105470 0x7f4e18075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.503+0000 7f4e0ea00640 1 --2- 172.21.15.47:0/425855589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e18104ab0 0x7f4e18079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.504+0000 7f4e17400640 1 -- 172.21.15.47:0/425855589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e18105470 msgr2=0x7f4e18075fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.504+0000 7f4e17400640 1 --2- 172.21.15.47:0/425855589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e18105470 0x7f4e18075fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.504+0000 7f4e17400640 1 -- 172.21.15.47:0/425855589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e18104ab0 msgr2=0x7f4e18079990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.504+0000 7f4e17400640 1 --2- 172.21.15.47:0/425855589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e18104ab0 0x7f4e18079990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.504+0000 7f4e17400640 1 -- 172.21.15.47:0/425855589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e18076880 con 0x7f4e181038c0 2024-03-21T11:35:10.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.504+0000 7f4e17400640 1 --2- 172.21.15.47:0/425855589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e181038c0 0x7f4e18079450 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800e9c0 tx=0x7f4e0800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:10.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.504+0000 7f4e14c00640 1 -- 172.21.15.47:0/425855589 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0800ccc0 con 0x7f4e181038c0 2024-03-21T11:35:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.505+0000 7f4e1e600640 1 -- 172.21.15.47:0/425855589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e181a6030 con 0x7f4e181038c0 2024-03-21T11:35:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.505+0000 7f4e14c00640 1 -- 172.21.15.47:0/425855589 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e080035f0 con 0x7f4e181038c0 2024-03-21T11:35:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.505+0000 7f4e14c00640 1 -- 172.21.15.47:0/425855589 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0800ce60 con 0x7f4e181038c0 2024-03-21T11:35:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.505+0000 7f4e1e600640 1 -- 172.21.15.47:0/425855589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e181a6490 con 0x7f4e181038c0 2024-03-21T11:35:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.508+0000 7f4e21216740 1 Processor -- start 2024-03-21T11:35:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.508+0000 7f4e14c00640 1 -- 172.21.15.47:0/425855589 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4e080037a0 con 0x7f4e181038c0 2024-03-21T11:35:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.508+0000 7f4e21216740 1 -- start start 2024-03-21T11:35:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.509+0000 7f4e21216740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea7023810 0x55cea7023c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.509+0000 7f4e21216740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70249e0 0x55cea7024e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.509+0000 7f4e21216740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea70253a0 0x55cea702bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.509+0000 7f4e21216740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea6f882c0 con 0x55cea70249e0 2024-03-21T11:35:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.509+0000 7f4e21216740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea6f30c90 con 0x55cea70253a0 2024-03-21T11:35:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.509+0000 7f4e21216740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea6f84360 con 0x55cea7023810 2024-03-21T11:35:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.509+0000 7f4e17e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea7023810 0x55cea7023c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.509+0000 7f4e17e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea7023810 0x55cea7023c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50074/0 (socket says 172.21.15.47:50074) 2024-03-21T11:35:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.509+0000 7f4e17e00640 1 -- 172.21.15.47:0/1171383592 learned_addr learned my addr 172.21.15.47:0/1171383592 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.509+0000 7f4e17400640 1 --2- 172.21.15.47:0/1171383592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70249e0 0x55cea7024e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.510+0000 7f4e17e00640 1 -- 172.21.15.47:0/1171383592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea70253a0 msgr2=0x55cea702bc30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:35:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.510+0000 7f4e17e00640 1 --2- 172.21.15.47:0/1171383592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea70253a0 0x55cea702bc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.510+0000 7f4e17e00640 1 -- 172.21.15.47:0/1171383592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70249e0 msgr2=0x55cea7024e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.510+0000 7f4e17e00640 1 --2- 172.21.15.47:0/1171383592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70249e0 0x55cea7024e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.510+0000 7f4e17e00640 1 -- 172.21.15.47:0/1171383592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cea6f55080 con 0x55cea7023810 2024-03-21T11:35:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.510+0000 7f4e17e00640 1 --2- 172.21.15.47:0/1171383592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea7023810 0x55cea7023c30 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4e10011440 tx=0x7f4e10011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae5dbfe0e04c5a16 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.510+0000 7f4e0cc00640 1 -- 172.21.15.47:0/1171383592 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e10003c30 con 0x55cea7023810 2024-03-21T11:35:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.510+0000 7f4e14c00640 1 --2- 172.21.15.47:0/425855589 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4dfc0a2fa0 0x7f4dfc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.510+0000 7f4e0cc00640 1 -- 172.21.15.47:0/1171383592 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e10003dd0 con 0x55cea7023810 2024-03-21T11:35:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.510+0000 7f4e0cc00640 1 -- 172.21.15.47:0/1171383592 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1000fb50 con 0x55cea7023810 2024-03-21T11:35:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.511+0000 7f4e14c00640 1 -- 172.21.15.47:0/425855589 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4e080cfb10 con 0x7f4e181038c0 2024-03-21T11:35:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.511+0000 7f4e17400640 1 --2- 172.21.15.47:0/425855589 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4dfc0a2fa0 0x7f4dfc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.511+0000 7f4e21216740 1 -- 172.21.15.47:0/1171383592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea7023810 msgr2=0x55cea7023c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.511+0000 7f4e21216740 1 --2- 172.21.15.47:0/1171383592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea7023810 0x55cea7023c30 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4e10011440 tx=0x7f4e10011900 comp rx=0 tx=0).stop 2024-03-21T11:35:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.511+0000 7f4e21216740 1 -- 172.21.15.47:0/1171383592 shutdown_connections 2024-03-21T11:35:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.511+0000 7f4e21216740 1 --2- 172.21.15.47:0/1171383592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea70253a0 0x55cea702bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.511+0000 7f4e21216740 1 --2- 172.21.15.47:0/1171383592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70249e0 0x55cea7024e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.511+0000 7f4e17400640 1 --2- 172.21.15.47:0/425855589 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4dfc0a2fa0 0x7f4dfc0a5460 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4e0809bde0 tx=0x7f4e08098210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:10.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.511+0000 7f4e21216740 1 --2- 172.21.15.47:0/1171383592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea7023810 0x55cea7023c30 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.511+0000 7f4e21216740 1 -- 172.21.15.47:0/1171383592 >> 172.21.15.47:0/1171383592 conn(0x55cea701ec00 msgr2=0x55cea701f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:10.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.511+0000 7f4e21216740 1 -- 172.21.15.47:0/1171383592 shutdown_connections 2024-03-21T11:35:10.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.512+0000 7f4e21216740 1 -- 172.21.15.47:0/1171383592 wait complete. 2024-03-21T11:35:10.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.512+0000 7f4e21216740 1 -- start start 2024-03-21T11:35:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.512+0000 7f4e21216740 1 Processor -- start 2024-03-21T11:35:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e21216740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea7023810 0x55cea702dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e21216740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea70249e0 0x55cea702e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e21216740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70253a0 0x55cea7034ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e21216740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea702c3b0 con 0x55cea70253a0 2024-03-21T11:35:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e21216740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea6f440d0 con 0x55cea7023810 2024-03-21T11:35:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e21216740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea702c530 con 0x55cea70249e0 2024-03-21T11:35:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e21216740 10 client.0 _ll_register_callbacks cb 0x55cea6f48e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e0ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea70249e0 0x55cea702e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e0ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea70249e0 0x55cea702e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50086/0 (socket says 172.21.15.47:50086) 2024-03-21T11:35:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e0ea00640 1 -- 172.21.15.47:0/2486567229 learned_addr learned my addr 172.21.15.47:0/2486567229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e17e00640 1 --2- 172.21.15.47:0/2486567229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea7023810 0x55cea702dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e17e00640 1 --2- 172.21.15.47:0/2486567229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70253a0 0x55cea7034ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e0ea00640 1 -- 172.21.15.47:0/2486567229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea7023810 msgr2=0x55cea702dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e0ea00640 1 --2- 172.21.15.47:0/2486567229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea7023810 0x55cea702dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.513+0000 7f4e0ea00640 1 -- 172.21.15.47:0/2486567229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70253a0 msgr2=0x55cea7034ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.514+0000 7f4e0ea00640 1 --2- 172.21.15.47:0/2486567229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70253a0 0x55cea7034ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.514+0000 7f4e0ea00640 1 -- 172.21.15.47:0/2486567229 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cea6c9c3f0 con 0x55cea70249e0 2024-03-21T11:35:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.514+0000 7f4e17e00640 1 --2- 172.21.15.47:0/2486567229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70253a0 0x55cea7034ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:35:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.514+0000 7f4e0ea00640 1 --2- 172.21.15.47:0/2486567229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea70249e0 0x55cea702e020 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4e0002efd0 tx=0x7f4e00004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66a23fd9b19472e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.514+0000 7f4e05600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.514+0000 7f4e05600640 1 -- 172.21.15.47:0/2486567229 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00042070 con 0x55cea70249e0 2024-03-21T11:35:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.514+0000 7f4e05600640 1 -- 172.21.15.47:0/2486567229 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4e000048c0 con 0x55cea70249e0 2024-03-21T11:35:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.514+0000 7f4e21216740 1 -- 172.21.15.47:0/2486567229 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e10003dd0 con 0x55cea70249e0 2024-03-21T11:35:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.514+0000 7f4e05600640 1 -- 172.21.15.47:0/2486567229 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e00050050 con 0x55cea70249e0 2024-03-21T11:35:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.514+0000 7f4e21216740 10 client.29524 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.514+0000 7f4e21216740 1 -- 172.21.15.47:0/2486567229 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cea6d667d0 con 0x55cea70249e0 2024-03-21T11:35:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.515+0000 7f4e05600640 1 -- 172.21.15.47:0/2486567229 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e10003dd0 con 0x55cea70249e0 2024-03-21T11:35:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.515+0000 7f4e05600640 1 -- 172.21.15.47:0/2486567229 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4e00005440 con 0x55cea70249e0 2024-03-21T11:35:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.515+0000 7f4e21216740 10 client.29524 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.515+0000 7f4e21216740 10 client.29524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.515+0000 7f4e21216740 20 client.29524 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.515+0000 7f4e21216740 1 --2- 172.21.15.47:0/2486567229 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cea7039570 0x55cea703b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.515+0000 7f4e21216740 4 client.29524 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.515+0000 7f4e21216740 1 -- 172.21.15.47:0/2486567229 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cea6f95800 con 0x55cea7039570 2024-03-21T11:35:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.515+0000 7f4e17e00640 1 --2- 172.21.15.47:0/2486567229 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cea7039570 0x55cea703b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.516+0000 7f4e17e00640 1 --2- 172.21.15.47:0/2486567229 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cea7039570 0x55cea703b9b0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb82ad15125ad63f server_cookie=e4f70b0a9a134d88 in_seq=0 out_seq=0 2024-03-21T11:35:10.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.516+0000 7f4e05600640 10 client.29524 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.517+0000 7f4e05600640 1 -- 172.21.15.47:0/2486567229 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cea6f95800 con 0x55cea7039570 2024-03-21T11:35:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.517+0000 7f4e05600640 10 client.29524 handle_command_reply: tid=0 2024-03-21T11:35:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.517+0000 7f4e21216740 1 client.29524 shutdown 2024-03-21T11:35:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.517+0000 7f4e21216740 1 -- 172.21.15.47:0/2486567229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea70249e0 msgr2=0x55cea702e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.517+0000 7f4e21216740 1 --2- 172.21.15.47:0/2486567229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea70249e0 0x55cea702e020 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4e0002efd0 tx=0x7f4e00004270 comp rx=0 tx=0).stop 2024-03-21T11:35:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.518+0000 7f4e21216740 1 -- 172.21.15.47:0/2486567229 shutdown_connections 2024-03-21T11:35:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.518+0000 7f4e21216740 1 --2- 172.21.15.47:0/2486567229 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cea7039570 0x55cea703b9b0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.518+0000 7f4e21216740 1 --2- 172.21.15.47:0/2486567229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70253a0 0x55cea7034ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.518+0000 7f4e21216740 1 --2- 172.21.15.47:0/2486567229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea70249e0 0x55cea702e020 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.518+0000 7f4e05600640 0 client.29524 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.518+0000 7f4e21216740 1 --2- 172.21.15.47:0/2486567229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea7023810 0x55cea702dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.518+0000 7f4e21216740 1 -- 172.21.15.47:0/2486567229 >> 172.21.15.47:0/2486567229 conn(0x55cea701ec00 msgr2=0x55cea701deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:10.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.518+0000 7f4e21216740 1 -- 172.21.15.47:0/2486567229 shutdown_connections 2024-03-21T11:35:10.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.518+0000 7f4e21216740 1 -- 172.21.15.47:0/2486567229 wait complete. 2024-03-21T11:35:10.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.518+0000 7f4e21216740 20 client.29524 trim_cache size 0 max 16384 2024-03-21T11:35:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.535+0000 7f4e21216740 1 Processor -- start 2024-03-21T11:35:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.535+0000 7f4e21216740 1 -- start start 2024-03-21T11:35:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.535+0000 7f4e21216740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea7023810 0x55cea7027c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.535+0000 7f4e21216740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea70249e0 0x55cea7034fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.535+0000 7f4e21216740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea70253a0 0x55cea7028150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.535+0000 7f4e21216740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e181b06a0 con 0x55cea7023810 2024-03-21T11:35:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.535+0000 7f4e21216740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e181b2020 con 0x55cea70253a0 2024-03-21T11:35:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.535+0000 7f4e21216740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e181b0d00 con 0x55cea70249e0 2024-03-21T11:35:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.536+0000 7f4e0ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea70249e0 0x55cea7034fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.536+0000 7f4e0ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea70249e0 0x55cea7034fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50102/0 (socket says 172.21.15.47:50102) 2024-03-21T11:35:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.536+0000 7f4e0ea00640 1 -- 172.21.15.47:0/1395180310 learned_addr learned my addr 172.21.15.47:0/1395180310 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.536+0000 7f4e17e00640 1 --2- 172.21.15.47:0/1395180310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea7023810 0x55cea7027c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.536+0000 7f4e0ea00640 1 -- 172.21.15.47:0/1395180310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea70253a0 msgr2=0x55cea7028150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:35:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.536+0000 7f4e0ea00640 1 --2- 172.21.15.47:0/1395180310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea70253a0 0x55cea7028150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.536+0000 7f4e0ea00640 1 -- 172.21.15.47:0/1395180310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea7023810 msgr2=0x55cea7027c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.536+0000 7f4e0ea00640 1 --2- 172.21.15.47:0/1395180310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea7023810 0x55cea7027c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.536+0000 7f4e0ea00640 1 -- 172.21.15.47:0/1395180310 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cea702ead0 con 0x55cea70249e0 2024-03-21T11:35:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.537+0000 7f4e0ea00640 1 --2- 172.21.15.47:0/1395180310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea70249e0 0x55cea7034fb0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f4e0002c180 tx=0x7f4e00009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16b2c17908ad30e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.537+0000 7f4e07e00640 1 -- 172.21.15.47:0/1395180310 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00059070 con 0x55cea70249e0 2024-03-21T11:35:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.537+0000 7f4e07e00640 1 -- 172.21.15.47:0/1395180310 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e00009520 con 0x55cea70249e0 2024-03-21T11:35:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.537+0000 7f4e07e00640 1 -- 172.21.15.47:0/1395180310 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00042070 con 0x55cea70249e0 2024-03-21T11:35:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.538+0000 7f4e21216740 1 -- 172.21.15.47:0/1395180310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea70249e0 msgr2=0x55cea7034fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.538+0000 7f4e21216740 1 --2- 172.21.15.47:0/1395180310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea70249e0 0x55cea7034fb0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f4e0002c180 tx=0x7f4e00009800 comp rx=0 tx=0).stop 2024-03-21T11:35:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.538+0000 7f4e21216740 1 -- 172.21.15.47:0/1395180310 shutdown_connections 2024-03-21T11:35:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.538+0000 7f4e21216740 1 --2- 172.21.15.47:0/1395180310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea70253a0 0x55cea7028150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.538+0000 7f4e21216740 1 --2- 172.21.15.47:0/1395180310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea70249e0 0x55cea7034fb0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.538+0000 7f4e21216740 1 --2- 172.21.15.47:0/1395180310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea7023810 0x55cea7027c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.538+0000 7f4e21216740 1 -- 172.21.15.47:0/1395180310 >> 172.21.15.47:0/1395180310 conn(0x55cea701ec00 msgr2=0x55cea70226f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.538+0000 7f4e21216740 1 -- 172.21.15.47:0/1395180310 shutdown_connections 2024-03-21T11:35:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.538+0000 7f4e21216740 1 -- 172.21.15.47:0/1395180310 wait complete. 2024-03-21T11:35:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.538+0000 7f4e21216740 1 -- start start 2024-03-21T11:35:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.539+0000 7f4e21216740 1 Processor -- start 2024-03-21T11:35:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.539+0000 7f4e21216740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea7023810 0x55cea7032be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.539+0000 7f4e21216740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70249e0 0x55cea7033120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.539+0000 7f4e21216740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea70253a0 0x55cea70340c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.539+0000 7f4e21216740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e181b19c0 con 0x55cea70249e0 2024-03-21T11:35:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.539+0000 7f4e21216740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e181b1360 con 0x55cea70253a0 2024-03-21T11:35:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.539+0000 7f4e21216740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e181ae860 con 0x55cea7023810 2024-03-21T11:35:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.539+0000 7f4e21216740 10 client.0 _ll_register_callbacks cb 0x7f4e100093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.539+0000 7f4e17e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea7023810 0x55cea7032be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.539+0000 7f4e17e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea7023810 0x55cea7032be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50108/0 (socket says 172.21.15.47:50108) 2024-03-21T11:35:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.539+0000 7f4e17e00640 1 -- 172.21.15.47:0/2462102232 learned_addr learned my addr 172.21.15.47:0/2462102232 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.539+0000 7f4e0ea00640 1 --2- 172.21.15.47:0/2462102232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70249e0 0x55cea7033120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.539+0000 7f4e17e00640 1 --2- 172.21.15.47:0/2462102232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea70253a0 0x55cea70340c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.540+0000 7f4e17e00640 1 -- 172.21.15.47:0/2462102232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea70253a0 msgr2=0x55cea70340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.540+0000 7f4e17e00640 1 --2- 172.21.15.47:0/2462102232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea70253a0 0x55cea70340c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.540+0000 7f4e17e00640 1 -- 172.21.15.47:0/2462102232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70249e0 msgr2=0x55cea7033120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.540+0000 7f4e17e00640 1 --2- 172.21.15.47:0/2462102232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70249e0 0x55cea7033120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.540+0000 7f4e17e00640 1 -- 172.21.15.47:0/2462102232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cea70346b0 con 0x55cea7023810 2024-03-21T11:35:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.540+0000 7f4e0ea00640 1 --2- 172.21.15.47:0/2462102232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70249e0 0x55cea7033120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.540+0000 7f4e17e00640 1 --2- 172.21.15.47:0/2462102232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea7023810 0x55cea7032be0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4e10011440 tx=0x7f4e10013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=308f87c9fbfa5380 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.540+0000 7f4e05600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.540+0000 7f4e05600640 1 -- 172.21.15.47:0/2462102232 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e10005070 con 0x55cea7023810 2024-03-21T11:35:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.541+0000 7f4e05600640 1 -- 172.21.15.47:0/2462102232 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4e10005210 con 0x55cea7023810 2024-03-21T11:35:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.541+0000 7f4e21216740 1 -- 172.21.15.47:0/2462102232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e00009520 con 0x55cea7023810 2024-03-21T11:35:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.541+0000 7f4e05600640 1 -- 172.21.15.47:0/2462102232 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e1003b050 con 0x55cea7023810 2024-03-21T11:35:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.541+0000 7f4e21216740 10 client.29530 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.541+0000 7f4e21216740 1 -- 172.21.15.47:0/2462102232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cea7095270 con 0x55cea7023810 2024-03-21T11:35:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.541+0000 7f4e05600640 1 -- 172.21.15.47:0/2462102232 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e00009520 con 0x55cea7023810 2024-03-21T11:35:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.541+0000 7f4e05600640 1 -- 172.21.15.47:0/2462102232 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4e1000fad0 con 0x55cea7023810 2024-03-21T11:35:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.541+0000 7f4e21216740 10 client.29530 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.541+0000 7f4e21216740 10 client.29530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.541+0000 7f4e21216740 20 client.29530 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.541+0000 7f4e21216740 1 --2- 172.21.15.47:0/2462102232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cea7095a80 0x55cea7097ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.541+0000 7f4e21216740 4 client.29530 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:35:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.541+0000 7f4e21216740 1 -- 172.21.15.47:0/2462102232 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cea6f8fb50 con 0x55cea7095a80 2024-03-21T11:35:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.542+0000 7f4e0ea00640 1 --2- 172.21.15.47:0/2462102232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cea7095a80 0x55cea7097ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.542+0000 7f4e0ea00640 1 --2- 172.21.15.47:0/2462102232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cea7095a80 0x55cea7097ea0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20575d348df37326 server_cookie=a648f739e79cdc86 in_seq=0 out_seq=0 2024-03-21T11:35:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.542+0000 7f4e05600640 10 client.29530 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.542+0000 7f4e05600640 1 -- 172.21.15.47:0/2462102232 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cea6f8fb50 con 0x55cea7095a80 2024-03-21T11:35:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.542+0000 7f4e05600640 10 client.29530 handle_command_reply: tid=0 2024-03-21T11:35:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.543+0000 7f4e21216740 1 client.29530 shutdown 2024-03-21T11:35:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.543+0000 7f4e21216740 1 -- 172.21.15.47:0/2462102232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea7023810 msgr2=0x55cea7032be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.543+0000 7f4e21216740 1 --2- 172.21.15.47:0/2462102232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea7023810 0x55cea7032be0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4e10011440 tx=0x7f4e10013040 comp rx=0 tx=0).stop 2024-03-21T11:35:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.543+0000 7f4e21216740 1 -- 172.21.15.47:0/2462102232 shutdown_connections 2024-03-21T11:35:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.543+0000 7f4e21216740 1 --2- 172.21.15.47:0/2462102232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cea7095a80 0x55cea7097ea0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.543+0000 7f4e21216740 1 --2- 172.21.15.47:0/2462102232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cea70253a0 0x55cea70340c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.543+0000 7f4e21216740 1 --2- 172.21.15.47:0/2462102232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cea70249e0 0x55cea7033120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.543+0000 7f4e21216740 1 --2- 172.21.15.47:0/2462102232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea7023810 0x55cea7032be0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.543+0000 7f4e05600640 0 client.29530 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.543+0000 7f4e21216740 1 -- 172.21.15.47:0/2462102232 >> 172.21.15.47:0/2462102232 conn(0x55cea701ec00 msgr2=0x55cea701d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.543+0000 7f4e21216740 1 -- 172.21.15.47:0/2462102232 shutdown_connections 2024-03-21T11:35:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.543+0000 7f4e21216740 1 -- 172.21.15.47:0/2462102232 wait complete. 2024-03-21T11:35:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.543+0000 7f4e21216740 20 client.29530 trim_cache size 0 max 16384 2024-03-21T11:35:10.548 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:35:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.544+0000 7f4e1e600640 1 -- 172.21.15.47:0/425855589 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4dfc0a2fa0 msgr2=0x7f4dfc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.544+0000 7f4e1e600640 1 --2- 172.21.15.47:0/425855589 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4dfc0a2fa0 0x7f4dfc0a5460 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4e0809bde0 tx=0x7f4e08098210 comp rx=0 tx=0).stop 2024-03-21T11:35:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.544+0000 7f4e1e600640 1 -- 172.21.15.47:0/425855589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e181038c0 msgr2=0x7f4e18079450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.544+0000 7f4e1e600640 1 --2- 172.21.15.47:0/425855589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e181038c0 0x7f4e18079450 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800e9c0 tx=0x7f4e0800edc0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.544+0000 7f4e1e600640 1 -- 172.21.15.47:0/425855589 shutdown_connections 2024-03-21T11:35:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.544+0000 7f4e1e600640 1 --2- 172.21.15.47:0/425855589 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4dfc0a2fa0 0x7f4dfc0a5460 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.544+0000 7f4e1e600640 1 --2- 172.21.15.47:0/425855589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e18105470 0x7f4e18075fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.544+0000 7f4e1e600640 1 --2- 172.21.15.47:0/425855589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e18104ab0 0x7f4e18079990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.544+0000 7f4e1e600640 1 --2- 172.21.15.47:0/425855589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e181038c0 0x7f4e18079450 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.544+0000 7f4e1e600640 1 -- 172.21.15.47:0/425855589 >> 172.21.15.47:0/425855589 conn(0x7f4e180fee50 msgr2=0x7f4e18107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.545+0000 7f4e1e600640 1 -- 172.21.15.47:0/425855589 shutdown_connections 2024-03-21T11:35:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:10.545+0000 7f4e1e600640 1 -- 172.21.15.47:0/425855589 wait complete. 2024-03-21T11:35:10.565 INFO:tasks.workunit.client.0.smithi047.stdout: 1 894118 7.98 MB/sec execute 474 sec latency 12.992 ms 2024-03-21T11:35:10.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:10 smithi047 ceph-mon[31351]: pgmap v755: 129 pgs: 129 active+clean; 41 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 249 op/s 2024-03-21T11:35:10.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:10 smithi102 ceph-mon[36748]: pgmap v755: 129 pgs: 129 active+clean; 41 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 249 op/s 2024-03-21T11:35:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:10 smithi100 ceph-mon[36245]: pgmap v755: 129 pgs: 129 active+clean; 41 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 249 op/s 2024-03-21T11:35:11.565 INFO:tasks.workunit.client.0.smithi047.stdout: 1 895209 7.98 MB/sec execute 475 sec latency 16.845 ms 2024-03-21T11:35:11.567 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:35:11.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:35:11.568 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:35:11.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.849+0000 7f2822200640 1 -- 172.21.15.47:0/3686649096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281c1038e0 msgr2=0x7f281c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:11.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.849+0000 7f2822200640 1 --2- 172.21.15.47:0/3686649096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281c1038e0 0x7f281c103d00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f280c009900 tx=0x7f280c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:35:11.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.850+0000 7f2822200640 1 -- 172.21.15.47:0/3686649096 shutdown_connections 2024-03-21T11:35:11.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.850+0000 7f2822200640 1 --2- 172.21.15.47:0/3686649096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f281c105490 0x7f281c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.850+0000 7f2822200640 1 --2- 172.21.15.47:0/3686649096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c104ad0 0x7f281c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.850+0000 7f2822200640 1 --2- 172.21.15.47:0/3686649096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281c1038e0 0x7f281c103d00 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.850+0000 7f2822200640 1 -- 172.21.15.47:0/3686649096 >> 172.21.15.47:0/3686649096 conn(0x7f281c0fee50 msgr2=0x7f281c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:11.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.850+0000 7f2822200640 1 -- 172.21.15.47:0/3686649096 shutdown_connections 2024-03-21T11:35:11.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.850+0000 7f2822200640 1 -- 172.21.15.47:0/3686649096 wait complete. 2024-03-21T11:35:11.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.851+0000 7f2822200640 1 Processor -- start 2024-03-21T11:35:11.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.851+0000 7f2822200640 1 -- start start 2024-03-21T11:35:11.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.851+0000 7f2822200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f281c1038e0 0x7f281c110ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.851+0000 7f2822200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281c104ad0 0x7f281c111020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.851+0000 7f2822200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c105490 0x7f281c116080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.851+0000 7f2822200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c076e40 con 0x7f281c105490 2024-03-21T11:35:11.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.851+0000 7f2822200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c076cc0 con 0x7f281c104ad0 2024-03-21T11:35:11.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.851+0000 7f2822200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c076fc0 con 0x7f281c1038e0 2024-03-21T11:35:11.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.851+0000 7f281b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f281c1038e0 0x7f281c110ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.851+0000 7f2813e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281c104ad0 0x7f281c111020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.851+0000 7f281b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f281c1038e0 0x7f281c110ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50114/0 (socket says 172.21.15.47:50114) 2024-03-21T11:35:11.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.851+0000 7f281b400640 1 -- 172.21.15.47:0/1533025719 learned_addr learned my addr 172.21.15.47:0/1533025719 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:11.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.852+0000 7f281be00640 1 --2- 172.21.15.47:0/1533025719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c105490 0x7f281c116080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.852+0000 7f281b400640 1 -- 172.21.15.47:0/1533025719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281c104ad0 msgr2=0x7f281c111020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:11.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.852+0000 7f281b400640 1 --2- 172.21.15.47:0/1533025719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281c104ad0 0x7f281c111020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.852+0000 7f281b400640 1 -- 172.21.15.47:0/1533025719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c105490 msgr2=0x7f281c116080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:11.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.852+0000 7f281b400640 1 --2- 172.21.15.47:0/1533025719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c105490 0x7f281c116080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.852+0000 7f281b400640 1 -- 172.21.15.47:0/1533025719 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c1166f0 con 0x7f281c1038e0 2024-03-21T11:35:11.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.852+0000 7f281b400640 1 --2- 172.21.15.47:0/1533025719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f281c1038e0 0x7f281c110ae0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f280c000c00 tx=0x7f280c02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:11.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.852+0000 7f2818c00640 1 -- 172.21.15.47:0/1533025719 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c043020 con 0x7f281c1038e0 2024-03-21T11:35:11.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.852+0000 7f2818c00640 1 -- 172.21.15.47:0/1533025719 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f280c0054e0 con 0x7f281c1038e0 2024-03-21T11:35:11.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.852+0000 7f2822200640 1 -- 172.21.15.47:0/1533025719 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281c1ae760 con 0x7f281c1038e0 2024-03-21T11:35:11.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.853+0000 7f2818c00640 1 -- 172.21.15.47:0/1533025719 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c003c40 con 0x7f281c1038e0 2024-03-21T11:35:11.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.853+0000 7f2822200640 1 -- 172.21.15.47:0/1533025719 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281c1aec50 con 0x7f281c1038e0 2024-03-21T11:35:11.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.856+0000 7f2824e56740 1 Processor -- start 2024-03-21T11:35:11.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.857+0000 7f2824e56740 1 -- start start 2024-03-21T11:35:11.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.857+0000 7f2824e56740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc10a0 0x555f30fc14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.857+0000 7f2824e56740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc1a90 0x555f30fc6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.857+0000 7f2824e56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc6c00 0x555f30fc9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.857+0000 7f2824e56740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f30f26eb0 con 0x555f30fc1a90 2024-03-21T11:35:11.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.857+0000 7f2824e56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f30ecfa60 con 0x555f30fc6c00 2024-03-21T11:35:11.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.857+0000 7f2824e56740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f30f22f50 con 0x555f30fc10a0 2024-03-21T11:35:11.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.857+0000 7f281be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc10a0 0x555f30fc14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.857+0000 7f281be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc10a0 0x555f30fc14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50126/0 (socket says 172.21.15.47:50126) 2024-03-21T11:35:11.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.857+0000 7f281be00640 1 -- 172.21.15.47:0/3769048013 learned_addr learned my addr 172.21.15.47:0/3769048013 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:11.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.857+0000 7f2813e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc6c00 0x555f30fc9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.858+0000 7f281be00640 1 -- 172.21.15.47:0/3769048013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc6c00 msgr2=0x555f30fc9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:11.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.858+0000 7f281be00640 1 --2- 172.21.15.47:0/3769048013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc6c00 0x555f30fc9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.858+0000 7f281be00640 1 -- 172.21.15.47:0/3769048013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc1a90 msgr2=0x555f30fc6590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:35:11.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.858+0000 7f281be00640 1 --2- 172.21.15.47:0/3769048013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc1a90 0x555f30fc6590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.858+0000 7f2818c00640 1 -- 172.21.15.47:0/1533025719 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f280c046020 con 0x7f281c1038e0 2024-03-21T11:35:11.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.858+0000 7f281be00640 1 -- 172.21.15.47:0/3769048013 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f30d037e0 con 0x555f30fc10a0 2024-03-21T11:35:11.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.858+0000 7f281be00640 1 --2- 172.21.15.47:0/3769048013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc10a0 0x555f30fc14c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2814009fc0 tx=0x7f2814010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e8045e254fb10b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:11.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.858+0000 7f280be00640 1 -- 172.21.15.47:0/3769048013 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814003b70 con 0x555f30fc10a0 2024-03-21T11:35:11.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.858+0000 7f280be00640 1 -- 172.21.15.47:0/3769048013 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2814003d10 con 0x555f30fc10a0 2024-03-21T11:35:11.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.858+0000 7f280be00640 1 -- 172.21.15.47:0/3769048013 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814011aa0 con 0x555f30fc10a0 2024-03-21T11:35:11.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.859+0000 7f2824e56740 1 -- 172.21.15.47:0/3769048013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc10a0 msgr2=0x555f30fc14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:11.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.859+0000 7f2824e56740 1 --2- 172.21.15.47:0/3769048013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc10a0 0x555f30fc14c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2814009fc0 tx=0x7f2814010860 comp rx=0 tx=0).stop 2024-03-21T11:35:11.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.859+0000 7f2824e56740 1 -- 172.21.15.47:0/3769048013 shutdown_connections 2024-03-21T11:35:11.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.859+0000 7f2824e56740 1 --2- 172.21.15.47:0/3769048013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc6c00 0x555f30fc9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.859+0000 7f2824e56740 1 --2- 172.21.15.47:0/3769048013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc1a90 0x555f30fc6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.859+0000 7f2824e56740 1 --2- 172.21.15.47:0/3769048013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc10a0 0x555f30fc14c0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.859+0000 7f2824e56740 1 -- 172.21.15.47:0/3769048013 >> 172.21.15.47:0/3769048013 conn(0x555f30fbc610 msgr2=0x555f30fbca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.859+0000 7f2818c00640 1 --2- 172.21.15.47:0/1533025719 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28000a2fa0 0x7f28000a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.859+0000 7f2824e56740 1 -- 172.21.15.47:0/3769048013 shutdown_connections 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.859+0000 7f2818c00640 1 -- 172.21.15.47:0/1533025719 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f280c0f1800 con 0x7f281c1038e0 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.860+0000 7f2824e56740 1 -- 172.21.15.47:0/3769048013 wait complete. 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.860+0000 7f2813e00640 1 --2- 172.21.15.47:0/1533025719 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28000a2fa0 0x7f28000a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.860+0000 7f2824e56740 1 -- start start 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.861+0000 7f2824e56740 1 Processor -- start 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.861+0000 7f2813e00640 1 --2- 172.21.15.47:0/1533025719 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28000a2fa0 0x7f28000a5460 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x555f30fbd580 tx=0x7f2804005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.861+0000 7f2824e56740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc10a0 0x555f30fcb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.861+0000 7f2824e56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc1a90 0x555f30fcb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.861+0000 7f2824e56740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc6c00 0x555f30fd2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.861+0000 7f2824e56740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f30fc9bb0 con 0x555f30fc10a0 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.861+0000 7f2824e56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f30ee2eb0 con 0x555f30fc1a90 2024-03-21T11:35:11.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.861+0000 7f2824e56740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f30fc9d30 con 0x555f30fc6c00 2024-03-21T11:35:11.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.861+0000 7f2824e56740 10 client.0 _ll_register_callbacks cb 0x555f30ed7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:11.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.861+0000 7f2813e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc1a90 0x555f30fcb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.861+0000 7f2813e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc1a90 0x555f30fcb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54822/0 (socket says 172.21.15.47:54822) 2024-03-21T11:35:11.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.861+0000 7f2813e00640 1 -- 172.21.15.47:0/3068253163 learned_addr learned my addr 172.21.15.47:0/3068253163 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:11.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.862+0000 7f281be00640 1 --2- 172.21.15.47:0/3068253163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc6c00 0x555f30fd2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.862+0000 7f281b400640 1 --2- 172.21.15.47:0/3068253163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc10a0 0x555f30fcb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.862+0000 7f281be00640 1 -- 172.21.15.47:0/3068253163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc1a90 msgr2=0x555f30fcb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:11.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.862+0000 7f281be00640 1 --2- 172.21.15.47:0/3068253163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc1a90 0x555f30fcb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.862+0000 7f281be00640 1 -- 172.21.15.47:0/3068253163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc10a0 msgr2=0x555f30fcb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:11.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.862+0000 7f281be00640 1 --2- 172.21.15.47:0/3068253163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc10a0 0x555f30fcb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.862+0000 7f281be00640 1 -- 172.21.15.47:0/3068253163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f30d63f70 con 0x555f30fc6c00 2024-03-21T11:35:11.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.862+0000 7f281be00640 1 --2- 172.21.15.47:0/3068253163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc6c00 0x555f30fd2840 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2814010d60 tx=0x7f2814010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a6152284d2e910c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:11.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.862+0000 7f2808c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:11.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.865+0000 7f2808c00640 1 -- 172.21.15.47:0/3068253163 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28140119f0 con 0x555f30fc6c00 2024-03-21T11:35:11.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.865+0000 7f2808c00640 1 -- 172.21.15.47:0/3068253163 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2814011b90 con 0x555f30fc6c00 2024-03-21T11:35:11.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.865+0000 7f2824e56740 1 -- 172.21.15.47:0/3068253163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2814003d10 con 0x555f30fc6c00 2024-03-21T11:35:11.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.865+0000 7f2808c00640 1 -- 172.21.15.47:0/3068253163 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2814031050 con 0x555f30fc6c00 2024-03-21T11:35:11.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.865+0000 7f2824e56740 10 client.29542 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:11.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.865+0000 7f2824e56740 1 -- 172.21.15.47:0/3068253163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555f30bfb170 con 0x555f30fc6c00 2024-03-21T11:35:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.866+0000 7f2808c00640 1 -- 172.21.15.47:0/3068253163 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2814003d10 con 0x555f30fc6c00 2024-03-21T11:35:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.866+0000 7f2808c00640 1 -- 172.21.15.47:0/3068253163 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2814005040 con 0x555f30fc6c00 2024-03-21T11:35:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.866+0000 7f2824e56740 10 client.29542 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.866+0000 7f2824e56740 10 client.29542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.866+0000 7f2824e56740 20 client.29542 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.866+0000 7f2824e56740 1 --2- 172.21.15.47:0/3068253163 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f30fd6e30 0x555f30fd9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.866+0000 7f2824e56740 4 client.29542 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.866+0000 7f2824e56740 1 -- 172.21.15.47:0/3068253163 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f30f343f0 con 0x555f30fd6e30 2024-03-21T11:35:11.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.867+0000 7f281b400640 1 --2- 172.21.15.47:0/3068253163 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f30fd6e30 0x555f30fd9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.867+0000 7f281b400640 1 --2- 172.21.15.47:0/3068253163 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f30fd6e30 0x555f30fd9220 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=668ff0974dba8218 server_cookie=bd40392391d0bfb9 in_seq=0 out_seq=0 2024-03-21T11:35:11.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.867+0000 7f2808c00640 10 client.29542 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:11.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.868+0000 7f2808c00640 1 -- 172.21.15.47:0/3068253163 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555f30f343f0 con 0x555f30fd6e30 2024-03-21T11:35:11.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.868+0000 7f2808c00640 10 client.29542 handle_command_reply: tid=0 2024-03-21T11:35:11.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.868+0000 7f2824e56740 1 client.29542 shutdown 2024-03-21T11:35:11.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.869+0000 7f2824e56740 1 -- 172.21.15.47:0/3068253163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc6c00 msgr2=0x555f30fd2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:11.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.869+0000 7f2824e56740 1 --2- 172.21.15.47:0/3068253163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc6c00 0x555f30fd2840 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2814010d60 tx=0x7f2814010fd0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.869+0000 7f2824e56740 1 -- 172.21.15.47:0/3068253163 shutdown_connections 2024-03-21T11:35:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.869+0000 7f2824e56740 1 --2- 172.21.15.47:0/3068253163 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f30fd6e30 0x555f30fd9220 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.869+0000 7f2824e56740 1 --2- 172.21.15.47:0/3068253163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc6c00 0x555f30fd2840 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.869+0000 7f2824e56740 1 --2- 172.21.15.47:0/3068253163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc1a90 0x555f30fcb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.869+0000 7f2824e56740 1 --2- 172.21.15.47:0/3068253163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc10a0 0x555f30fcb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.869+0000 7f2824e56740 1 -- 172.21.15.47:0/3068253163 >> 172.21.15.47:0/3068253163 conn(0x555f30fbc610 msgr2=0x555f30fbb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.869+0000 7f2808c00640 0 client.29542 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.869+0000 7f2824e56740 1 -- 172.21.15.47:0/3068253163 shutdown_connections 2024-03-21T11:35:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.869+0000 7f2824e56740 1 -- 172.21.15.47:0/3068253163 wait complete. 2024-03-21T11:35:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.870+0000 7f2824e56740 20 client.29542 trim_cache size 0 max 16384 2024-03-21T11:35:11.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.895+0000 7f2824e56740 1 Processor -- start 2024-03-21T11:35:11.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.895+0000 7f2824e56740 1 -- start start 2024-03-21T11:35:11.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.895+0000 7f2824e56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc10a0 0x555f30fc2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.895+0000 7f2824e56740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc1a90 0x555f30fc3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.895+0000 7f2824e56740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc6c00 0x555f30fd2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.895+0000 7f2824e56740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c1b9000 con 0x555f30fc6c00 2024-03-21T11:35:11.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.895+0000 7f2824e56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c1ba980 con 0x555f30fc10a0 2024-03-21T11:35:11.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.895+0000 7f2824e56740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c1b9660 con 0x555f30fc1a90 2024-03-21T11:35:11.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.895+0000 7f2813e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc1a90 0x555f30fc3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.896+0000 7f281b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc10a0 0x555f30fc2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.896+0000 7f2813e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc1a90 0x555f30fc3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50152/0 (socket says 172.21.15.47:50152) 2024-03-21T11:35:11.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.896+0000 7f281b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc10a0 0x555f30fc2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54824/0 (socket says 172.21.15.47:54824) 2024-03-21T11:35:11.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.896+0000 7f2813e00640 1 -- 172.21.15.47:0/3205948829 learned_addr learned my addr 172.21.15.47:0/3205948829 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:11.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.896+0000 7f281be00640 1 --2- 172.21.15.47:0/3205948829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc6c00 0x555f30fd2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.896+0000 7f2813e00640 1 -- 172.21.15.47:0/3205948829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc10a0 msgr2=0x555f30fc2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:11.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.896+0000 7f2813e00640 1 --2- 172.21.15.47:0/3205948829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc10a0 0x555f30fc2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.896+0000 7f2813e00640 1 -- 172.21.15.47:0/3205948829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc6c00 msgr2=0x555f30fd2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:11.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.896+0000 7f2813e00640 1 --2- 172.21.15.47:0/3205948829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc6c00 0x555f30fd2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.896+0000 7f2813e00640 1 -- 172.21.15.47:0/3205948829 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f30fb7c20 con 0x555f30fc1a90 2024-03-21T11:35:11.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.896+0000 7f281be00640 1 --2- 172.21.15.47:0/3205948829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc6c00 0x555f30fd2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:11.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.896+0000 7f2813e00640 1 --2- 172.21.15.47:0/3205948829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc1a90 0x555f30fc3180 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2804004390 tx=0x7f280401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1236856f2a3c31a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:11.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.897+0000 7f280b400640 1 -- 172.21.15.47:0/3205948829 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804009e60 con 0x555f30fc1a90 2024-03-21T11:35:11.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.897+0000 7f280b400640 1 -- 172.21.15.47:0/3205948829 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28040043c0 con 0x555f30fc1a90 2024-03-21T11:35:11.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.897+0000 7f280b400640 1 -- 172.21.15.47:0/3205948829 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280401f430 con 0x555f30fc1a90 2024-03-21T11:35:11.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.898+0000 7f2824e56740 1 -- 172.21.15.47:0/3205948829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc1a90 msgr2=0x555f30fc3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:11.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.898+0000 7f2824e56740 1 --2- 172.21.15.47:0/3205948829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc1a90 0x555f30fc3180 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2804004390 tx=0x7f280401b000 comp rx=0 tx=0).stop 2024-03-21T11:35:11.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.898+0000 7f2824e56740 1 -- 172.21.15.47:0/3205948829 shutdown_connections 2024-03-21T11:35:11.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.898+0000 7f2824e56740 1 --2- 172.21.15.47:0/3205948829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc6c00 0x555f30fd2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.898+0000 7f2824e56740 1 --2- 172.21.15.47:0/3205948829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc1a90 0x555f30fc3180 secure :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2804004390 tx=0x7f280401b000 comp rx=0 tx=0).stop 2024-03-21T11:35:11.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.898+0000 7f2824e56740 1 --2- 172.21.15.47:0/3205948829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fc10a0 0x555f30fc2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.898+0000 7f2824e56740 1 -- 172.21.15.47:0/3205948829 >> 172.21.15.47:0/3205948829 conn(0x555f30fbc610 msgr2=0x555f30fc2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:11.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.898+0000 7f2824e56740 1 -- 172.21.15.47:0/3205948829 shutdown_connections 2024-03-21T11:35:11.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.898+0000 7f2824e56740 1 -- 172.21.15.47:0/3205948829 wait complete. 2024-03-21T11:35:11.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.899+0000 7f2824e56740 1 -- start start 2024-03-21T11:35:11.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.899+0000 7f2824e56740 1 Processor -- start 2024-03-21T11:35:11.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.900+0000 7f2824e56740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc10a0 0x555f30fd2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.900+0000 7f2824e56740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc6c00 0x555f30fd0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.900+0000 7f2824e56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fd11a0 0x555f30fd1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.900+0000 7f2824e56740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c1ba320 con 0x555f30fc6c00 2024-03-21T11:35:11.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.900+0000 7f2824e56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c1b9cc0 con 0x555f30fd11a0 2024-03-21T11:35:11.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.900+0000 7f2824e56740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c1b71c0 con 0x555f30fc10a0 2024-03-21T11:35:11.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.900+0000 7f2824e56740 10 client.0 _ll_register_callbacks cb 0x7f28040097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:11.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.900+0000 7f281b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc10a0 0x555f30fd2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.900+0000 7f281b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc10a0 0x555f30fd2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50168/0 (socket says 172.21.15.47:50168) 2024-03-21T11:35:11.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.900+0000 7f281b400640 1 -- 172.21.15.47:0/1311078543 learned_addr learned my addr 172.21.15.47:0/1311078543 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:11.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.900+0000 7f2813e00640 1 --2- 172.21.15.47:0/1311078543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc6c00 0x555f30fd0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.900+0000 7f281be00640 1 --2- 172.21.15.47:0/1311078543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fd11a0 0x555f30fd1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.901+0000 7f281be00640 1 -- 172.21.15.47:0/1311078543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc10a0 msgr2=0x555f30fd2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:11.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.901+0000 7f281be00640 1 --2- 172.21.15.47:0/1311078543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc10a0 0x555f30fd2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.901+0000 7f281be00640 1 -- 172.21.15.47:0/1311078543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc6c00 msgr2=0x555f30fd0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.901+0000 7f281be00640 1 --2- 172.21.15.47:0/1311078543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc6c00 0x555f30fd0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.901+0000 7f281be00640 1 -- 172.21.15.47:0/1311078543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f31032600 con 0x555f30fd11a0 2024-03-21T11:35:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.901+0000 7f281be00640 1 --2- 172.21.15.47:0/1311078543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fd11a0 0x555f30fd1620 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2814005440 tx=0x7f2814009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c952f00522cf4f71 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.901+0000 7f2808c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.902+0000 7f2808c00640 1 -- 172.21.15.47:0/1311078543 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814009bd0 con 0x555f30fd11a0 2024-03-21T11:35:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.902+0000 7f2808c00640 1 -- 172.21.15.47:0/1311078543 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f28140119f0 con 0x555f30fd11a0 2024-03-21T11:35:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.902+0000 7f2824e56740 1 -- 172.21.15.47:0/1311078543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28040043c0 con 0x555f30fd11a0 2024-03-21T11:35:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.902+0000 7f2808c00640 1 -- 172.21.15.47:0/1311078543 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2814044050 con 0x555f30fd11a0 2024-03-21T11:35:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.903+0000 7f2824e56740 10 client.29519 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.903+0000 7f2824e56740 1 -- 172.21.15.47:0/1311078543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555f31033920 con 0x555f30fd11a0 2024-03-21T11:35:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.903+0000 7f2808c00640 1 -- 172.21.15.47:0/1311078543 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28040043c0 con 0x555f30fd11a0 2024-03-21T11:35:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.903+0000 7f2808c00640 1 -- 172.21.15.47:0/1311078543 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f281401b540 con 0x555f30fd11a0 2024-03-21T11:35:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.903+0000 7f2824e56740 10 client.29519 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:11.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.903+0000 7f2824e56740 10 client.29519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:11.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.903+0000 7f2824e56740 20 client.29519 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:11.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.903+0000 7f2824e56740 1 --2- 172.21.15.47:0/1311078543 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f31034160 0x555f31036580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:11.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.903+0000 7f2824e56740 4 client.29519 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:35:11.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.903+0000 7f2824e56740 1 -- 172.21.15.47:0/1311078543 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555f30f2e740 con 0x555f31034160 2024-03-21T11:35:11.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.903+0000 7f281b400640 1 --2- 172.21.15.47:0/1311078543 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f31034160 0x555f31036580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:11.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.904+0000 7f281b400640 1 --2- 172.21.15.47:0/1311078543 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f31034160 0x555f31036580 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd38376ff6a125a8 server_cookie=7c6f0f244cdbd5ef in_seq=0 out_seq=0 2024-03-21T11:35:11.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.904+0000 7f2808c00640 10 client.29519 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:11.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.905+0000 7f2808c00640 1 -- 172.21.15.47:0/1311078543 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555f30f2e740 con 0x555f31034160 2024-03-21T11:35:11.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.905+0000 7f2808c00640 10 client.29519 handle_command_reply: tid=0 2024-03-21T11:35:11.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.905+0000 7f2824e56740 1 client.29519 shutdown 2024-03-21T11:35:11.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.905+0000 7f2824e56740 1 -- 172.21.15.47:0/1311078543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fd11a0 msgr2=0x555f30fd1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:11.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.905+0000 7f2824e56740 1 --2- 172.21.15.47:0/1311078543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fd11a0 0x555f30fd1620 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2814005440 tx=0x7f2814009610 comp rx=0 tx=0).stop 2024-03-21T11:35:11.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.906+0000 7f2824e56740 1 -- 172.21.15.47:0/1311078543 shutdown_connections 2024-03-21T11:35:11.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.906+0000 7f2824e56740 1 --2- 172.21.15.47:0/1311078543 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f31034160 0x555f31036580 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.906+0000 7f2824e56740 1 --2- 172.21.15.47:0/1311078543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f30fd11a0 0x555f30fd1620 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.906+0000 7f2824e56740 1 --2- 172.21.15.47:0/1311078543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f30fc6c00 0x555f30fd0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.906+0000 7f2824e56740 1 --2- 172.21.15.47:0/1311078543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f30fc10a0 0x555f30fd2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.906+0000 7f2824e56740 1 -- 172.21.15.47:0/1311078543 >> 172.21.15.47:0/1311078543 conn(0x555f30fbc610 msgr2=0x555f30fc2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:11.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.906+0000 7f2808c00640 0 client.29519 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:11.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.906+0000 7f2824e56740 1 -- 172.21.15.47:0/1311078543 shutdown_connections 2024-03-21T11:35:11.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.906+0000 7f2824e56740 1 -- 172.21.15.47:0/1311078543 wait complete. 2024-03-21T11:35:11.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.906+0000 7f2824e56740 20 client.29519 trim_cache size 0 max 16384 2024-03-21T11:35:11.909 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:35:11.909 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:35:11.909 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "d117b9e0-699c-4805-9075-247a6eecb14e", 2024-03-21T11:35:11.909 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:35:11.909 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:35:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.907+0000 7f2822200640 1 -- 172.21.15.47:0/1533025719 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28000a2fa0 msgr2=0x7f28000a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.907+0000 7f2822200640 1 --2- 172.21.15.47:0/1533025719 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28000a2fa0 0x7f28000a5460 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x555f30fbd580 tx=0x7f2804005f90 comp rx=0 tx=0).stop 2024-03-21T11:35:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.907+0000 7f2822200640 1 -- 172.21.15.47:0/1533025719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f281c1038e0 msgr2=0x7f281c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.907+0000 7f2822200640 1 --2- 172.21.15.47:0/1533025719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f281c1038e0 0x7f281c110ae0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f280c000c00 tx=0x7f280c02c650 comp rx=0 tx=0).stop 2024-03-21T11:35:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.907+0000 7f2822200640 1 -- 172.21.15.47:0/1533025719 shutdown_connections 2024-03-21T11:35:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.907+0000 7f2822200640 1 --2- 172.21.15.47:0/1533025719 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28000a2fa0 0x7f28000a5460 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.907+0000 7f2822200640 1 --2- 172.21.15.47:0/1533025719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f281c105490 0x7f281c116080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.907+0000 7f2822200640 1 --2- 172.21.15.47:0/1533025719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281c104ad0 0x7f281c111020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.907+0000 7f2822200640 1 --2- 172.21.15.47:0/1533025719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f281c1038e0 0x7f281c110ae0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.907+0000 7f2822200640 1 -- 172.21.15.47:0/1533025719 >> 172.21.15.47:0/1533025719 conn(0x7f281c0fee50 msgr2=0x7f281c105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.908+0000 7f2822200640 1 -- 172.21.15.47:0/1533025719 shutdown_connections 2024-03-21T11:35:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:11.908+0000 7f2822200640 1 -- 172.21.15.47:0/1533025719 wait complete. 2024-03-21T11:35:11.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:35:11.930 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:35:12.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.144+0000 7f4881800640 1 -- 172.21.15.47:0/3646575787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c1037a0 msgr2=0x7f487c103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:12.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.144+0000 7f4881800640 1 --2- 172.21.15.47:0/3646575787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c1037a0 0x7f487c103bc0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4874009900 tx=0x7f487402bc40 comp rx=0 tx=0).stop 2024-03-21T11:35:12.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.144+0000 7f4881800640 1 -- 172.21.15.47:0/3646575787 shutdown_connections 2024-03-21T11:35:12.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.144+0000 7f4881800640 1 --2- 172.21.15.47:0/3646575787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c105350 0x7f487c10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.144+0000 7f4881800640 1 --2- 172.21.15.47:0/3646575787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487c104990 0x7f487c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.144+0000 7f4881800640 1 --2- 172.21.15.47:0/3646575787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c1037a0 0x7f487c103bc0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.144+0000 7f4881800640 1 -- 172.21.15.47:0/3646575787 >> 172.21.15.47:0/3646575787 conn(0x7f487c0fed10 msgr2=0x7f487c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.144+0000 7f4881800640 1 -- 172.21.15.47:0/3646575787 shutdown_connections 2024-03-21T11:35:12.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.144+0000 7f4881800640 1 -- 172.21.15.47:0/3646575787 wait complete. 2024-03-21T11:35:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.145+0000 7f4881800640 1 Processor -- start 2024-03-21T11:35:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.145+0000 7f4881800640 1 -- start start 2024-03-21T11:35:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.145+0000 7f4881800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c1037a0 0x7f487c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.145+0000 7f4881800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c104990 0x7f487c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.145+0000 7f4881800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487c105350 0x7f487c1a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.145+0000 7f4881800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c10f470 con 0x7f487c105350 2024-03-21T11:35:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.145+0000 7f4881800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c10f2f0 con 0x7f487c1037a0 2024-03-21T11:35:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.145+0000 7f4881800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c10f5f0 con 0x7f487c104990 2024-03-21T11:35:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.146+0000 7f487b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c104990 0x7f487c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.146+0000 7f487be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c1037a0 0x7f487c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.146+0000 7f487b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c104990 0x7f487c199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50174/0 (socket says 172.21.15.47:50174) 2024-03-21T11:35:12.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.146+0000 7f487b400640 1 -- 172.21.15.47:0/2563359344 learned_addr learned my addr 172.21.15.47:0/2563359344 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.146+0000 7f4880e00640 1 --2- 172.21.15.47:0/2563359344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487c105350 0x7f487c1a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.146+0000 7f487b400640 1 -- 172.21.15.47:0/2563359344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c1037a0 msgr2=0x7f487c199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.146+0000 7f487b400640 1 --2- 172.21.15.47:0/2563359344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c1037a0 0x7f487c199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.146+0000 7f487b400640 1 -- 172.21.15.47:0/2563359344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487c105350 msgr2=0x7f487c1a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.146+0000 7f487b400640 1 --2- 172.21.15.47:0/2563359344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487c105350 0x7f487c1a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.146+0000 7f487b400640 1 -- 172.21.15.47:0/2563359344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c1a16a0 con 0x7f487c104990 2024-03-21T11:35:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.146+0000 7f487be00640 1 --2- 172.21.15.47:0/2563359344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c1037a0 0x7f487c199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.146+0000 7f487b400640 1 --2- 172.21.15.47:0/2563359344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c104990 0x7f487c199fc0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f486400ebc0 tx=0x7f486400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.147+0000 7f4878c00640 1 -- 172.21.15.47:0/2563359344 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864010630 con 0x7f487c104990 2024-03-21T11:35:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.147+0000 7f4878c00640 1 -- 172.21.15.47:0/2563359344 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4864010cb0 con 0x7f487c104990 2024-03-21T11:35:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.147+0000 7f4881800640 1 -- 172.21.15.47:0/2563359344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487c1a1990 con 0x7f487c104990 2024-03-21T11:35:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.147+0000 7f4878c00640 1 -- 172.21.15.47:0/2563359344 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864018bd0 con 0x7f487c104990 2024-03-21T11:35:12.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.147+0000 7f4881800640 1 -- 172.21.15.47:0/2563359344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487c1a1ee0 con 0x7f487c104990 2024-03-21T11:35:12.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.150+0000 7f4878c00640 1 -- 172.21.15.47:0/2563359344 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f48640107d0 con 0x7f487c104990 2024-03-21T11:35:12.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.151+0000 7f4885b6b740 1 Processor -- start 2024-03-21T11:35:12.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.151+0000 7f4885b6b740 1 -- start start 2024-03-21T11:35:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.151+0000 7f4885b6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae7c050 0x55e51ae7c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.151+0000 7f4885b6b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7ca40 0x55e51ae81540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.151+0000 7f4885b6b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae81bb0 0x55e51ae84040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.151+0000 7f4885b6b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e51ade1250 con 0x55e51ae81bb0 2024-03-21T11:35:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.151+0000 7f4885b6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e51ad89a30 con 0x55e51ae7c050 2024-03-21T11:35:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.151+0000 7f4885b6b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e51addd2f0 con 0x55e51ae7ca40 2024-03-21T11:35:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.152+0000 7f4880e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae7c050 0x55e51ae7c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.152+0000 7f487be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7ca40 0x55e51ae81540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.152+0000 7f4880e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae7c050 0x55e51ae7c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54866/0 (socket says 172.21.15.47:54866) 2024-03-21T11:35:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.152+0000 7f4880e00640 1 -- 172.21.15.47:0/3235059009 learned_addr learned my addr 172.21.15.47:0/3235059009 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:12.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.152+0000 7f487b400640 1 --2- 172.21.15.47:0/3235059009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae81bb0 0x55e51ae84040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.152+0000 7f4880e00640 1 -- 172.21.15.47:0/3235059009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7ca40 msgr2=0x55e51ae81540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.152+0000 7f4880e00640 1 --2- 172.21.15.47:0/3235059009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7ca40 0x55e51ae81540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.152+0000 7f4880e00640 1 -- 172.21.15.47:0/3235059009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae81bb0 msgr2=0x55e51ae84040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.152+0000 7f4880e00640 1 --2- 172.21.15.47:0/3235059009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae81bb0 0x55e51ae84040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.152+0000 7f4880e00640 1 -- 172.21.15.47:0/3235059009 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e51ad74690 con 0x55e51ae7c050 2024-03-21T11:35:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.152+0000 7f4878c00640 1 --2- 172.21.15.47:0/2563359344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f485c0a2f20 0x7f485c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.152+0000 7f4880e00640 1 --2- 172.21.15.47:0/3235059009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae7c050 0x55e51ae7c470 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f486c009ca0 tx=0x7f486c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919b5c4e97273b17 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.152+0000 7f4878c00640 1 -- 172.21.15.47:0/2563359344 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f48640cfd60 con 0x7f487c104990 2024-03-21T11:35:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.153+0000 7f487be00640 1 --2- 172.21.15.47:0/2563359344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f485c0a2f20 0x7f485c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.153+0000 7f4871600640 1 -- 172.21.15.47:0/3235059009 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c01e070 con 0x55e51ae7c050 2024-03-21T11:35:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.153+0000 7f4871600640 1 -- 172.21.15.47:0/3235059009 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f486c009e50 con 0x55e51ae7c050 2024-03-21T11:35:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.153+0000 7f4871600640 1 -- 172.21.15.47:0/3235059009 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c00eac0 con 0x55e51ae7c050 2024-03-21T11:35:12.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.153+0000 7f487be00640 1 --2- 172.21.15.47:0/2563359344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f485c0a2f20 0x7f485c0a53e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f487402c180 tx=0x7f487402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.153+0000 7f4885b6b740 1 -- 172.21.15.47:0/3235059009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae7c050 msgr2=0x55e51ae7c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.154+0000 7f4885b6b740 1 --2- 172.21.15.47:0/3235059009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae7c050 0x55e51ae7c470 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f486c009ca0 tx=0x7f486c011810 comp rx=0 tx=0).stop 2024-03-21T11:35:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.154+0000 7f4885b6b740 1 -- 172.21.15.47:0/3235059009 shutdown_connections 2024-03-21T11:35:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.154+0000 7f4885b6b740 1 --2- 172.21.15.47:0/3235059009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae81bb0 0x55e51ae84040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.154+0000 7f4885b6b740 1 --2- 172.21.15.47:0/3235059009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7ca40 0x55e51ae81540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.154+0000 7f4885b6b740 1 --2- 172.21.15.47:0/3235059009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae7c050 0x55e51ae7c470 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.154+0000 7f4885b6b740 1 -- 172.21.15.47:0/3235059009 >> 172.21.15.47:0/3235059009 conn(0x55e51ae44200 msgr2=0x55e51ae779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.154+0000 7f4885b6b740 1 -- 172.21.15.47:0/3235059009 shutdown_connections 2024-03-21T11:35:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.154+0000 7f4885b6b740 1 -- 172.21.15.47:0/3235059009 wait complete. 2024-03-21T11:35:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.154+0000 7f4885b6b740 1 -- start start 2024-03-21T11:35:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.155+0000 7f4885b6b740 1 Processor -- start 2024-03-21T11:35:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.155+0000 7f4885b6b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7c050 0x55e51ae860b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.155+0000 7f4885b6b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae7ca40 0x55e51ae865f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.155+0000 7f4885b6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae81bb0 0x55e51ae8d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.155+0000 7f4885b6b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e51ae84b60 con 0x55e51ae7ca40 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.155+0000 7f4885b6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e51ad9ce80 con 0x55e51ae81bb0 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.155+0000 7f4885b6b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e51ae84ce0 con 0x55e51ae7c050 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.155+0000 7f4885b6b740 10 client.0 _ll_register_callbacks cb 0x55e51adaf130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.155+0000 7f487be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7c050 0x55e51ae860b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.155+0000 7f4880e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae81bb0 0x55e51ae8d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.155+0000 7f487be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7c050 0x55e51ae860b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50194/0 (socket says 172.21.15.47:50194) 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.155+0000 7f487be00640 1 -- 172.21.15.47:0/508771930 learned_addr learned my addr 172.21.15.47:0/508771930 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.155+0000 7f487b400640 1 --2- 172.21.15.47:0/508771930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae7ca40 0x55e51ae865f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.156+0000 7f4880e00640 1 -- 172.21.15.47:0/508771930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7c050 msgr2=0x55e51ae860b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.156+0000 7f4880e00640 1 --2- 172.21.15.47:0/508771930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7c050 0x55e51ae860b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.156+0000 7f4880e00640 1 -- 172.21.15.47:0/508771930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae7ca40 msgr2=0x55e51ae865f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.156+0000 7f4880e00640 1 --2- 172.21.15.47:0/508771930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae7ca40 0x55e51ae865f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.156+0000 7f4880e00640 1 -- 172.21.15.47:0/508771930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e51abbf7d0 con 0x55e51ae81bb0 2024-03-21T11:35:12.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.156+0000 7f487be00640 1 --2- 172.21.15.47:0/508771930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7c050 0x55e51ae860b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:12.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.156+0000 7f487b400640 1 --2- 172.21.15.47:0/508771930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae7ca40 0x55e51ae865f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:12.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.156+0000 7f4880e00640 1 --2- 172.21.15.47:0/508771930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae81bb0 0x55e51ae8d630 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f486c01c040 tx=0x7f486c00eab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcbbc69d24f067f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:12.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.156+0000 7f486a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.156+0000 7f486a000640 1 -- 172.21.15.47:0/508771930 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c027020 con 0x55e51ae81bb0 2024-03-21T11:35:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.157+0000 7f486a000640 1 -- 172.21.15.47:0/508771930 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f486c0037f0 con 0x55e51ae81bb0 2024-03-21T11:35:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.157+0000 7f4885b6b740 1 -- 172.21.15.47:0/508771930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f486c009e50 con 0x55e51ae81bb0 2024-03-21T11:35:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.157+0000 7f486a000640 1 -- 172.21.15.47:0/508771930 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f486c032050 con 0x55e51ae81bb0 2024-03-21T11:35:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.157+0000 7f4885b6b740 10 client.29531 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.157+0000 7f4885b6b740 1 -- 172.21.15.47:0/508771930 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e51aaf53f0 con 0x55e51ae81bb0 2024-03-21T11:35:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.157+0000 7f486a000640 1 -- 172.21.15.47:0/508771930 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f486c009e50 con 0x55e51ae81bb0 2024-03-21T11:35:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.157+0000 7f486a000640 1 -- 172.21.15.47:0/508771930 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f486c019730 con 0x55e51ae81bb0 2024-03-21T11:35:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.158+0000 7f4885b6b740 10 client.29531 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.158+0000 7f4885b6b740 10 client.29531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.158+0000 7f4885b6b740 20 client.29531 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:12.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.158+0000 7f4885b6b740 1 --2- 172.21.15.47:0/508771930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e51ae91d10 0x55e51ae94100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.158+0000 7f4885b6b740 4 client.29531 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.158+0000 7f4885b6b740 1 -- 172.21.15.47:0/508771930 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e51adee790 con 0x55e51ae91d10 2024-03-21T11:35:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.158+0000 7f487be00640 1 --2- 172.21.15.47:0/508771930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e51ae91d10 0x55e51ae94100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.159+0000 7f487be00640 1 --2- 172.21.15.47:0/508771930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e51ae91d10 0x55e51ae94100 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de4be49b182953 server_cookie=98fce7bf23b46454 in_seq=0 out_seq=0 2024-03-21T11:35:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.159+0000 7f486a000640 10 client.29531 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.159+0000 7f486a000640 1 -- 172.21.15.47:0/508771930 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e51adee790 con 0x55e51ae91d10 2024-03-21T11:35:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.159+0000 7f486a000640 10 client.29531 handle_command_reply: tid=0 2024-03-21T11:35:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.160+0000 7f4885b6b740 1 client.29531 shutdown 2024-03-21T11:35:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.160+0000 7f4885b6b740 1 -- 172.21.15.47:0/508771930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae81bb0 msgr2=0x55e51ae8d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.160+0000 7f4885b6b740 1 --2- 172.21.15.47:0/508771930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae81bb0 0x55e51ae8d630 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f486c01c040 tx=0x7f486c00eab0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.160+0000 7f4885b6b740 1 -- 172.21.15.47:0/508771930 shutdown_connections 2024-03-21T11:35:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.160+0000 7f4885b6b740 1 --2- 172.21.15.47:0/508771930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e51ae91d10 0x55e51ae94100 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.160+0000 7f4885b6b740 1 --2- 172.21.15.47:0/508771930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae81bb0 0x55e51ae8d630 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.160+0000 7f4885b6b740 1 --2- 172.21.15.47:0/508771930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae7ca40 0x55e51ae865f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.160+0000 7f4885b6b740 1 --2- 172.21.15.47:0/508771930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7c050 0x55e51ae860b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.160+0000 7f4885b6b740 1 -- 172.21.15.47:0/508771930 >> 172.21.15.47:0/508771930 conn(0x55e51ae44200 msgr2=0x55e51ae76bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:12.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.160+0000 7f486a000640 0 client.29531 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:12.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.160+0000 7f4885b6b740 1 -- 172.21.15.47:0/508771930 shutdown_connections 2024-03-21T11:35:12.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.160+0000 7f4885b6b740 1 -- 172.21.15.47:0/508771930 wait complete. 2024-03-21T11:35:12.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.160+0000 7f4885b6b740 20 client.29531 trim_cache size 0 max 16384 2024-03-21T11:35:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.181+0000 7f4885b6b740 1 Processor -- start 2024-03-21T11:35:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.181+0000 7f4885b6b740 1 -- start start 2024-03-21T11:35:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f4885b6b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae7c050 0x55e51ae810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f4885b6b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7ca40 0x55e51ae8d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f4885b6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae81bb0 0x55e51ae76a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f4885b6b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1ac100 con 0x55e51ae7c050 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f4885b6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1ada80 con 0x55e51ae81bb0 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f4885b6b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1ac760 con 0x55e51ae7ca40 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f4880e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae81bb0 0x55e51ae76a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f487b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7ca40 0x55e51ae8d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f487be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae7c050 0x55e51ae810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f487b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7ca40 0x55e51ae8d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50200/0 (socket says 172.21.15.47:50200) 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f4880e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae81bb0 0x55e51ae76a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54892/0 (socket says 172.21.15.47:54892) 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f487b400640 1 -- 172.21.15.47:0/576749055 learned_addr learned my addr 172.21.15.47:0/576749055 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f4880e00640 1 -- 172.21.15.47:0/576749055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7ca40 msgr2=0x55e51ae8d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f4880e00640 1 --2- 172.21.15.47:0/576749055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7ca40 0x55e51ae8d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f4880e00640 1 -- 172.21.15.47:0/576749055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae7c050 msgr2=0x55e51ae810d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.182+0000 7f4880e00640 1 --2- 172.21.15.47:0/576749055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae7c050 0x55e51ae810d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.183+0000 7f4880e00640 1 -- 172.21.15.47:0/576749055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e51aab50f0 con 0x55e51ae81bb0 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.183+0000 7f487b400640 1 --2- 172.21.15.47:0/576749055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7ca40 0x55e51ae8d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.183+0000 7f4880e00640 1 --2- 172.21.15.47:0/576749055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae81bb0 0x55e51ae76a40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f486c005fd0 tx=0x7f486c005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29924df662305ab2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.183+0000 7f4870c00640 1 -- 172.21.15.47:0/576749055 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c015070 con 0x55e51ae81bb0 2024-03-21T11:35:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.183+0000 7f4870c00640 1 -- 172.21.15.47:0/576749055 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f486c01f040 con 0x55e51ae81bb0 2024-03-21T11:35:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.183+0000 7f4870c00640 1 -- 172.21.15.47:0/576749055 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c027070 con 0x55e51ae81bb0 2024-03-21T11:35:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.184+0000 7f4885b6b740 1 -- 172.21.15.47:0/576749055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae81bb0 msgr2=0x55e51ae76a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.184+0000 7f4885b6b740 1 --2- 172.21.15.47:0/576749055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae81bb0 0x55e51ae76a40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f486c005fd0 tx=0x7f486c005e70 comp rx=0 tx=0).stop 2024-03-21T11:35:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.184+0000 7f4885b6b740 1 -- 172.21.15.47:0/576749055 shutdown_connections 2024-03-21T11:35:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.184+0000 7f4885b6b740 1 --2- 172.21.15.47:0/576749055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae81bb0 0x55e51ae76a40 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.184+0000 7f4885b6b740 1 --2- 172.21.15.47:0/576749055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7ca40 0x55e51ae8d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.184+0000 7f4885b6b740 1 --2- 172.21.15.47:0/576749055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae7c050 0x55e51ae810d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.184+0000 7f4885b6b740 1 -- 172.21.15.47:0/576749055 >> 172.21.15.47:0/576749055 conn(0x55e51ae44200 msgr2=0x55e51ae76330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.184+0000 7f4885b6b740 1 -- 172.21.15.47:0/576749055 shutdown_connections 2024-03-21T11:35:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.184+0000 7f4885b6b740 1 -- 172.21.15.47:0/576749055 wait complete. 2024-03-21T11:35:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.184+0000 7f4885b6b740 1 -- start start 2024-03-21T11:35:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.185+0000 7f4885b6b740 1 Processor -- start 2024-03-21T11:35:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.185+0000 7f4885b6b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7c050 0x55e51ae8aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.185+0000 7f4885b6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae7ca40 0x55e51ae8aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.185+0000 7f4885b6b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae81bb0 0x55e51ae8bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.185+0000 7f4885b6b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1ad420 con 0x55e51ae81bb0 2024-03-21T11:35:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.185+0000 7f4885b6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1acdc0 con 0x55e51ae7ca40 2024-03-21T11:35:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.185+0000 7f4885b6b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1aa2c0 con 0x55e51ae7c050 2024-03-21T11:35:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.185+0000 7f4885b6b740 10 client.0 _ll_register_callbacks cb 0x7f486c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.185+0000 7f487be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7c050 0x55e51ae8aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.186+0000 7f487b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae7ca40 0x55e51ae8aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.186+0000 7f487be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7c050 0x55e51ae8aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50212/0 (socket says 172.21.15.47:50212) 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.186+0000 7f487be00640 1 -- 172.21.15.47:0/2089292690 learned_addr learned my addr 172.21.15.47:0/2089292690 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.186+0000 7f4880e00640 1 --2- 172.21.15.47:0/2089292690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae81bb0 0x55e51ae8bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.186+0000 7f487be00640 1 -- 172.21.15.47:0/2089292690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae7ca40 msgr2=0x55e51ae8aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.186+0000 7f487be00640 1 --2- 172.21.15.47:0/2089292690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae7ca40 0x55e51ae8aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.186+0000 7f487be00640 1 -- 172.21.15.47:0/2089292690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae81bb0 msgr2=0x55e51ae8bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.186+0000 7f487be00640 1 --2- 172.21.15.47:0/2089292690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae81bb0 0x55e51ae8bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.186+0000 7f487be00640 1 -- 172.21.15.47:0/2089292690 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e51ae8c580 con 0x55e51ae7c050 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.186+0000 7f487b400640 1 --2- 172.21.15.47:0/2089292690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae7ca40 0x55e51ae8aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.186+0000 7f487be00640 1 --2- 172.21.15.47:0/2089292690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7c050 0x55e51ae8aab0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4874036040 tx=0x7f4874009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3783defd073aa860 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.186+0000 7f486a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.187+0000 7f486a000640 1 -- 172.21.15.47:0/2089292690 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874005380 con 0x55e51ae7c050 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.187+0000 7f4885b6b740 1 -- 172.21.15.47:0/2089292690 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f486c01f040 con 0x55e51ae7c050 2024-03-21T11:35:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.187+0000 7f486a000640 1 -- 172.21.15.47:0/2089292690 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4874005520 con 0x55e51ae7c050 2024-03-21T11:35:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.187+0000 7f486a000640 1 -- 172.21.15.47:0/2089292690 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4874054610 con 0x55e51ae7c050 2024-03-21T11:35:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.187+0000 7f4885b6b740 10 client.29566 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.187+0000 7f4885b6b740 1 -- 172.21.15.47:0/2089292690 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e51ae8d900 con 0x55e51ae7c050 2024-03-21T11:35:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.187+0000 7f486a000640 1 -- 172.21.15.47:0/2089292690 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f486c01f040 con 0x55e51ae7c050 2024-03-21T11:35:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.187+0000 7f486a000640 1 -- 172.21.15.47:0/2089292690 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4874046040 con 0x55e51ae7c050 2024-03-21T11:35:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.188+0000 7f4885b6b740 10 client.29566 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.188+0000 7f4885b6b740 10 client.29566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.188+0000 7f4885b6b740 20 client.29566 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.188+0000 7f4885b6b740 1 --2- 172.21.15.47:0/2089292690 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e51aeedc50 0x55e51aef0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.188+0000 7f4885b6b740 4 client.29566 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:35:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.188+0000 7f4885b6b740 1 -- 172.21.15.47:0/2089292690 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e51ade8ae0 con 0x55e51aeedc50 2024-03-21T11:35:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.188+0000 7f487b400640 1 --2- 172.21.15.47:0/2089292690 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e51aeedc50 0x55e51aef0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.188+0000 7f487b400640 1 --2- 172.21.15.47:0/2089292690 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e51aeedc50 0x55e51aef0070 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a0bc7beae9ee6fe server_cookie=5a15ae45a273953d in_seq=0 out_seq=0 2024-03-21T11:35:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.188+0000 7f486a000640 10 client.29566 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.189+0000 7f486a000640 1 -- 172.21.15.47:0/2089292690 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e51ade8ae0 con 0x55e51aeedc50 2024-03-21T11:35:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.189+0000 7f486a000640 10 client.29566 handle_command_reply: tid=0 2024-03-21T11:35:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.189+0000 7f4885b6b740 1 client.29566 shutdown 2024-03-21T11:35:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.189+0000 7f4885b6b740 1 -- 172.21.15.47:0/2089292690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7c050 msgr2=0x55e51ae8aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.189+0000 7f4885b6b740 1 --2- 172.21.15.47:0/2089292690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7c050 0x55e51ae8aab0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4874036040 tx=0x7f4874009260 comp rx=0 tx=0).stop 2024-03-21T11:35:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.189+0000 7f4885b6b740 1 -- 172.21.15.47:0/2089292690 shutdown_connections 2024-03-21T11:35:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.189+0000 7f4885b6b740 1 --2- 172.21.15.47:0/2089292690 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e51aeedc50 0x55e51aef0070 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.189+0000 7f4885b6b740 1 --2- 172.21.15.47:0/2089292690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e51ae81bb0 0x55e51ae8bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.189+0000 7f4885b6b740 1 --2- 172.21.15.47:0/2089292690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e51ae7ca40 0x55e51ae8aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.189+0000 7f4885b6b740 1 --2- 172.21.15.47:0/2089292690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e51ae7c050 0x55e51ae8aab0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.189+0000 7f4885b6b740 1 -- 172.21.15.47:0/2089292690 >> 172.21.15.47:0/2089292690 conn(0x55e51ae44200 msgr2=0x55e51ae76bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.189+0000 7f486a000640 0 client.29566 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.190+0000 7f4885b6b740 1 -- 172.21.15.47:0/2089292690 shutdown_connections 2024-03-21T11:35:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.190+0000 7f4885b6b740 1 -- 172.21.15.47:0/2089292690 wait complete. 2024-03-21T11:35:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.190+0000 7f4885b6b740 20 client.29566 trim_cache size 0 max 16384 2024-03-21T11:35:12.196 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:35:12.196 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:35:12.196 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:35:12.196 INFO:teuthology.orchestra.run.smithi047.stdout: "d117b9e0-699c-4805-9075-247a6eecb14e": { 2024-03-21T11:35:12.196 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:35:12.196 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "d117b9e0-699c-4805-9075-247a6eecb14e", 2024-03-21T11:35:12.196 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:35:12.196 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:35:12.197 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:35:12.197 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:35:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.191+0000 7f4881800640 1 -- 172.21.15.47:0/2563359344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f485c0a2f20 msgr2=0x7f485c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.191+0000 7f4881800640 1 --2- 172.21.15.47:0/2563359344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f485c0a2f20 0x7f485c0a53e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f487402c180 tx=0x7f487402cf90 comp rx=0 tx=0).stop 2024-03-21T11:35:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.191+0000 7f4881800640 1 -- 172.21.15.47:0/2563359344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c104990 msgr2=0x7f487c199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.191+0000 7f4881800640 1 --2- 172.21.15.47:0/2563359344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c104990 0x7f487c199fc0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f486400ebc0 tx=0x7f486400c6a0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.191+0000 7f4881800640 1 -- 172.21.15.47:0/2563359344 shutdown_connections 2024-03-21T11:35:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.191+0000 7f4881800640 1 --2- 172.21.15.47:0/2563359344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f485c0a2f20 0x7f485c0a53e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.191+0000 7f4881800640 1 --2- 172.21.15.47:0/2563359344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487c105350 0x7f487c1a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.191+0000 7f4881800640 1 --2- 172.21.15.47:0/2563359344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f487c104990 0x7f487c199fc0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.191+0000 7f4881800640 1 --2- 172.21.15.47:0/2563359344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c1037a0 0x7f487c199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.191+0000 7f4881800640 1 -- 172.21.15.47:0/2563359344 >> 172.21.15.47:0/2563359344 conn(0x7f487c0fed10 msgr2=0x7f487c107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.191+0000 7f4881800640 1 -- 172.21.15.47:0/2563359344 shutdown_connections 2024-03-21T11:35:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:12.191+0000 7f4881800640 1 -- 172.21.15.47:0/2563359344 wait complete. 2024-03-21T11:35:12.211 INFO:tasks.cephfs.filesystem:scrub status for tag:d117b9e0-699c-4805-9075-247a6eecb14e - {'path': '/', 'tag': 'd117b9e0-699c-4805-9075-247a6eecb14e', 'options': 'recursive,force'} 2024-03-21T11:35:12.565 INFO:tasks.workunit.client.0.smithi047.stdout: 1 896361 7.97 MB/sec execute 476 sec latency 118.601 ms 2024-03-21T11:35:12.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:12 smithi102 ceph-mon[36748]: pgmap v756: 129 pgs: 129 active+clean; 41 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-03-21T11:35:12.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:12 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:35:12.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:12 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:35:12.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:12 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:35:12.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:12 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:35:12.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:12 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:35:13.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:12 smithi047 ceph-mon[31351]: pgmap v756: 129 pgs: 129 active+clean; 41 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-03-21T11:35:13.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:12 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:35:13.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:12 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:35:13.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:12 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:35:13.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:12 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:35:13.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:12 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:35:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:12 smithi100 ceph-mon[36245]: pgmap v756: 129 pgs: 129 active+clean; 41 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-03-21T11:35:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:12 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:35:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:12 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:35:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:12 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:35:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:12 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:35:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:12 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:35:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.448+0000 7f1495400640 1 -- 172.21.15.47:0/3198772166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149010b780 msgr2=0x7f149010db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.448+0000 7f1495400640 1 --2- 172.21.15.47:0/3198772166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149010b780 0x7f149010db70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f147c009900 tx=0x7f147c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:35:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.448+0000 7f1495400640 1 -- 172.21.15.47:0/3198772166 shutdown_connections 2024-03-21T11:35:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.448+0000 7f1495400640 1 --2- 172.21.15.47:0/3198772166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f149010e1e0 0x7f1490110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.448+0000 7f1495400640 1 --2- 172.21.15.47:0/3198772166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149010b780 0x7f149010db70 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.448+0000 7f1495400640 1 --2- 172.21.15.47:0/3198772166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149006a4a0 0x7f149010b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.448+0000 7f1495400640 1 -- 172.21.15.47:0/3198772166 >> 172.21.15.47:0/3198772166 conn(0x7f149006e8a0 msgr2=0x7f1490070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.448+0000 7f1495400640 1 -- 172.21.15.47:0/3198772166 shutdown_connections 2024-03-21T11:35:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.448+0000 7f1495400640 1 -- 172.21.15.47:0/3198772166 wait complete. 2024-03-21T11:35:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.449+0000 7f1495400640 1 Processor -- start 2024-03-21T11:35:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.449+0000 7f1495400640 1 -- start start 2024-03-21T11:35:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.449+0000 7f1495400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149006a4a0 0x7f14901a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.449+0000 7f1495400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149010b780 0x7f14901a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.449+0000 7f1495400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f149010e1e0 0x7f14901a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.449+0000 7f1495400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1490115bb0 con 0x7f149006a4a0 2024-03-21T11:35:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.449+0000 7f1495400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1490115a30 con 0x7f149010b780 2024-03-21T11:35:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.449+0000 7f1495400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1490115d30 con 0x7f149010e1e0 2024-03-21T11:35:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.450+0000 7f148de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149010b780 0x7f14901a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.450+0000 7f148f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f149010e1e0 0x7f14901a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.450+0000 7f148de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149010b780 0x7f14901a28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54928/0 (socket says 172.21.15.47:54928) 2024-03-21T11:35:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.450+0000 7f148de00640 1 -- 172.21.15.47:0/3881095157 learned_addr learned my addr 172.21.15.47:0/3881095157 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.450+0000 7f148e800640 1 --2- 172.21.15.47:0/3881095157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149006a4a0 0x7f14901a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.450+0000 7f148de00640 1 -- 172.21.15.47:0/3881095157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f149010e1e0 msgr2=0x7f14901a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.450+0000 7f148de00640 1 --2- 172.21.15.47:0/3881095157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f149010e1e0 0x7f14901a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.450+0000 7f148de00640 1 -- 172.21.15.47:0/3881095157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149006a4a0 msgr2=0x7f14901a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.450+0000 7f148de00640 1 --2- 172.21.15.47:0/3881095157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149006a4a0 0x7f14901a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.450+0000 7f148de00640 1 -- 172.21.15.47:0/3881095157 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14901a9f80 con 0x7f149010b780 2024-03-21T11:35:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.451+0000 7f148de00640 1 --2- 172.21.15.47:0/3881095157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149010b780 0x7f14901a28a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f147c000c00 tx=0x7f147c02c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.451+0000 7f1487400640 1 -- 172.21.15.47:0/3881095157 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c043020 con 0x7f149010b780 2024-03-21T11:35:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.451+0000 7f1487400640 1 -- 172.21.15.47:0/3881095157 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f147c0054e0 con 0x7f149010b780 2024-03-21T11:35:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.451+0000 7f1487400640 1 -- 172.21.15.47:0/3881095157 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c003ce0 con 0x7f149010b780 2024-03-21T11:35:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.451+0000 7f1495400640 1 -- 172.21.15.47:0/3881095157 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14901aa210 con 0x7f149010b780 2024-03-21T11:35:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.451+0000 7f1495400640 1 -- 172.21.15.47:0/3881095157 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14901aa6e0 con 0x7f149010b780 2024-03-21T11:35:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.455+0000 7f14981d3740 1 Processor -- start 2024-03-21T11:35:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.456+0000 7f14981d3740 1 -- start start 2024-03-21T11:35:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.456+0000 7f1487400640 1 -- 172.21.15.47:0/3881095157 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f147c046020 con 0x7f149010b780 2024-03-21T11:35:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.456+0000 7f14981d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af61e0 0x561943af6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.456+0000 7f14981d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af6bd0 0x561943afb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.456+0000 7f14981d3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943afbd40 0x561943afe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.456+0000 7f14981d3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561943ab93b0 con 0x561943afbd40 2024-03-21T11:35:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.456+0000 7f14981d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561943a5b830 con 0x561943af61e0 2024-03-21T11:35:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.456+0000 7f14981d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561943a045d0 con 0x561943af6bd0 2024-03-21T11:35:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.456+0000 7f148f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af61e0 0x561943af6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.456+0000 7f148f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af61e0 0x561943af6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54938/0 (socket says 172.21.15.47:54938) 2024-03-21T11:35:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.456+0000 7f148e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af6bd0 0x561943afb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.456+0000 7f148f200640 1 -- 172.21.15.47:0/2171046521 learned_addr learned my addr 172.21.15.47:0/2171046521 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.456+0000 7f148de00640 1 --2- 172.21.15.47:0/2171046521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943afbd40 0x561943afe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.457+0000 7f148f200640 1 -- 172.21.15.47:0/2171046521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af6bd0 msgr2=0x561943afb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.457+0000 7f148f200640 1 --2- 172.21.15.47:0/2171046521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af6bd0 0x561943afb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.457+0000 7f148f200640 1 -- 172.21.15.47:0/2171046521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943afbd40 msgr2=0x561943afe1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.457+0000 7f148f200640 1 --2- 172.21.15.47:0/2171046521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943afbd40 0x561943afe1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.457+0000 7f148f200640 1 -- 172.21.15.47:0/2171046521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561943a1c2c0 con 0x561943af61e0 2024-03-21T11:35:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.457+0000 7f148e800640 1 --2- 172.21.15.47:0/2171046521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af6bd0 0x561943afb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.457+0000 7f148de00640 1 --2- 172.21.15.47:0/2171046521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943afbd40 0x561943afe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.457+0000 7f148f200640 1 --2- 172.21.15.47:0/2171046521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af61e0 0x561943af6600 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1488009fd0 tx=0x7f14880118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c63e956b54904dd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.457+0000 7f147b400640 1 -- 172.21.15.47:0/2171046521 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488004d20 con 0x561943af61e0 2024-03-21T11:35:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.457+0000 7f147b400640 1 -- 172.21.15.47:0/2171046521 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1488004050 con 0x561943af61e0 2024-03-21T11:35:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.457+0000 7f147b400640 1 -- 172.21.15.47:0/2171046521 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488005280 con 0x561943af61e0 2024-03-21T11:35:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.458+0000 7f1487400640 1 --2- 172.21.15.47:0/3881095157 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14740a3040 0x7f14740a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.458+0000 7f1487400640 1 -- 172.21.15.47:0/3881095157 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f147c0f17a0 con 0x7f149010b780 2024-03-21T11:35:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.458+0000 7f148e800640 1 --2- 172.21.15.47:0/3881095157 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14740a3040 0x7f14740a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.458+0000 7f14981d3740 1 -- 172.21.15.47:0/2171046521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af61e0 msgr2=0x561943af6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.458+0000 7f14981d3740 1 --2- 172.21.15.47:0/2171046521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af61e0 0x561943af6600 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1488009fd0 tx=0x7f14880118d0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.458+0000 7f14981d3740 1 -- 172.21.15.47:0/2171046521 shutdown_connections 2024-03-21T11:35:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.458+0000 7f14981d3740 1 --2- 172.21.15.47:0/2171046521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943afbd40 0x561943afe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.458+0000 7f14981d3740 1 --2- 172.21.15.47:0/2171046521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af6bd0 0x561943afb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.458+0000 7f14981d3740 1 --2- 172.21.15.47:0/2171046521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af61e0 0x561943af6600 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.458+0000 7f14981d3740 1 -- 172.21.15.47:0/2171046521 >> 172.21.15.47:0/2171046521 conn(0x561943af1750 msgr2=0x561943af1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.459+0000 7f14981d3740 1 -- 172.21.15.47:0/2171046521 shutdown_connections 2024-03-21T11:35:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.459+0000 7f14981d3740 1 -- 172.21.15.47:0/2171046521 wait complete. 2024-03-21T11:35:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.459+0000 7f148e800640 1 --2- 172.21.15.47:0/3881095157 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14740a3040 0x7f14740a5500 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x561943af26c0 tx=0x7f148000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.459+0000 7f14981d3740 1 -- start start 2024-03-21T11:35:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.459+0000 7f14981d3740 1 Processor -- start 2024-03-21T11:35:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f14981d3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943af61e0 0x561943b003c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f14981d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af6bd0 0x561943b00900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f14981d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943afbd40 0x561943b078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f14981d3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561943a17a20 con 0x561943af61e0 2024-03-21T11:35:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f14981d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561943a578d0 con 0x561943af6bd0 2024-03-21T11:35:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f14981d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561943afecf0 con 0x561943afbd40 2024-03-21T11:35:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f148de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af6bd0 0x561943b00900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f14981d3740 10 client.0 _ll_register_callbacks cb 0x561943a541c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f148de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af6bd0 0x561943b00900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54954/0 (socket says 172.21.15.47:54954) 2024-03-21T11:35:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f148de00640 1 -- 172.21.15.47:0/954680498 learned_addr learned my addr 172.21.15.47:0/954680498 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f148e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943af61e0 0x561943b003c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f148f200640 1 --2- 172.21.15.47:0/954680498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943afbd40 0x561943b078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f148de00640 1 -- 172.21.15.47:0/954680498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943afbd40 msgr2=0x561943b078d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f148de00640 1 --2- 172.21.15.47:0/954680498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943afbd40 0x561943b078d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f148de00640 1 -- 172.21.15.47:0/954680498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943af61e0 msgr2=0x561943b003c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f148de00640 1 --2- 172.21.15.47:0/954680498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943af61e0 0x561943b003c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.460+0000 7f148de00640 1 -- 172.21.15.47:0/954680498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561943770210 con 0x561943af6bd0 2024-03-21T11:35:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.461+0000 7f148de00640 1 --2- 172.21.15.47:0/954680498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af6bd0 0x561943b00900 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f147c0b9fa0 tx=0x7f147c0b9ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f347c4d10c79a318 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.461+0000 7f146fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.461+0000 7f146fe00640 1 -- 172.21.15.47:0/954680498 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c004c80 con 0x561943af6bd0 2024-03-21T11:35:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.461+0000 7f146fe00640 1 -- 172.21.15.47:0/954680498 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f147c0bbe40 con 0x561943af6bd0 2024-03-21T11:35:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.461+0000 7f14981d3740 1 -- 172.21.15.47:0/954680498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1488004050 con 0x561943af6bd0 2024-03-21T11:35:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.461+0000 7f146fe00640 1 -- 172.21.15.47:0/954680498 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f147c0044a0 con 0x561943af6bd0 2024-03-21T11:35:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.461+0000 7f14981d3740 10 client.29549 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.461+0000 7f14981d3740 1 -- 172.21.15.47:0/954680498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56194383c620 con 0x561943af6bd0 2024-03-21T11:35:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.462+0000 7f146fe00640 1 -- 172.21.15.47:0/954680498 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1488004050 con 0x561943af6bd0 2024-03-21T11:35:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.462+0000 7f146fe00640 1 -- 172.21.15.47:0/954680498 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f147c0c60c0 con 0x561943af6bd0 2024-03-21T11:35:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.462+0000 7f14981d3740 10 client.29549 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.462+0000 7f14981d3740 10 client.29549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.462+0000 7f14981d3740 20 client.29549 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.462+0000 7f14981d3740 1 --2- 172.21.15.47:0/954680498 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561943b0c1a0 0x561943b0e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.462+0000 7f14981d3740 4 client.29549 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.462+0000 7f14981d3740 1 -- 172.21.15.47:0/954680498 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561943a68d70 con 0x561943b0c1a0 2024-03-21T11:35:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.462+0000 7f148e800640 1 --2- 172.21.15.47:0/954680498 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561943b0c1a0 0x561943b0e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.463+0000 7f148e800640 1 --2- 172.21.15.47:0/954680498 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561943b0c1a0 0x561943b0e590 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=158c8e4527700f0 server_cookie=a49bea475df6c38f in_seq=0 out_seq=0 2024-03-21T11:35:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.463+0000 7f146fe00640 10 client.29549 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.464+0000 7f146fe00640 1 -- 172.21.15.47:0/954680498 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561943a68d70 con 0x561943b0c1a0 2024-03-21T11:35:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.464+0000 7f146fe00640 10 client.29549 handle_command_reply: tid=0 2024-03-21T11:35:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.464+0000 7f14981d3740 1 client.29549 shutdown 2024-03-21T11:35:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.465+0000 7f14981d3740 1 -- 172.21.15.47:0/954680498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af6bd0 msgr2=0x561943b00900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.465+0000 7f14981d3740 1 --2- 172.21.15.47:0/954680498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af6bd0 0x561943b00900 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f147c0b9fa0 tx=0x7f147c0b9ea0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.465+0000 7f14981d3740 1 -- 172.21.15.47:0/954680498 shutdown_connections 2024-03-21T11:35:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.465+0000 7f14981d3740 1 --2- 172.21.15.47:0/954680498 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561943b0c1a0 0x561943b0e590 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.465+0000 7f14981d3740 1 --2- 172.21.15.47:0/954680498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943afbd40 0x561943b078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.465+0000 7f14981d3740 1 --2- 172.21.15.47:0/954680498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af6bd0 0x561943b00900 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.465+0000 7f146fe00640 0 client.29549 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.465+0000 7f14981d3740 1 --2- 172.21.15.47:0/954680498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943af61e0 0x561943b003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.465+0000 7f14981d3740 1 -- 172.21.15.47:0/954680498 >> 172.21.15.47:0/954680498 conn(0x561943af1750 msgr2=0x561943af0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.465+0000 7f14981d3740 1 -- 172.21.15.47:0/954680498 shutdown_connections 2024-03-21T11:35:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.465+0000 7f14981d3740 1 -- 172.21.15.47:0/954680498 wait complete. 2024-03-21T11:35:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.465+0000 7f14981d3740 20 client.29549 trim_cache size 0 max 16384 2024-03-21T11:35:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.483+0000 7f14981d3740 1 Processor -- start 2024-03-21T11:35:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.484+0000 7f14981d3740 1 -- start start 2024-03-21T11:35:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.484+0000 7f14981d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af61e0 0x561943af00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.484+0000 7f14981d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af6bd0 0x561943b07890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.484+0000 7f14981d3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943afbd40 0x561943af0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.484+0000 7f14981d3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14901b4a90 con 0x561943afbd40 2024-03-21T11:35:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.484+0000 7f14981d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14901b6410 con 0x561943af6bd0 2024-03-21T11:35:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.484+0000 7f14981d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14901b50f0 con 0x561943af61e0 2024-03-21T11:35:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.484+0000 7f148de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af6bd0 0x561943b07890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.484+0000 7f148e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af61e0 0x561943af00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.484+0000 7f148de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af6bd0 0x561943b07890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54964/0 (socket says 172.21.15.47:54964) 2024-03-21T11:35:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.484+0000 7f148de00640 1 -- 172.21.15.47:0/1136017679 learned_addr learned my addr 172.21.15.47:0/1136017679 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.484+0000 7f148f200640 1 --2- 172.21.15.47:0/1136017679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943afbd40 0x561943af0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.485+0000 7f148de00640 1 -- 172.21.15.47:0/1136017679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af61e0 msgr2=0x561943af00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.485+0000 7f148de00640 1 --2- 172.21.15.47:0/1136017679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af61e0 0x561943af00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.485+0000 7f148de00640 1 -- 172.21.15.47:0/1136017679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943afbd40 msgr2=0x561943af0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.485+0000 7f148de00640 1 --2- 172.21.15.47:0/1136017679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943afbd40 0x561943af0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.485+0000 7f148de00640 1 -- 172.21.15.47:0/1136017679 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561943af65c0 con 0x561943af6bd0 2024-03-21T11:35:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.485+0000 7f148e800640 1 --2- 172.21.15.47:0/1136017679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af61e0 0x561943af00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.485+0000 7f148de00640 1 --2- 172.21.15.47:0/1136017679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af6bd0 0x561943b07890 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f147c0bd840 tx=0x7f147c0eef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35fc2ad1a4ce4fbb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.485+0000 7f147aa00640 1 -- 172.21.15.47:0/1136017679 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c0bd680 con 0x561943af6bd0 2024-03-21T11:35:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.485+0000 7f147aa00640 1 -- 172.21.15.47:0/1136017679 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f147c004c80 con 0x561943af6bd0 2024-03-21T11:35:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.485+0000 7f147aa00640 1 -- 172.21.15.47:0/1136017679 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c0bd1d0 con 0x561943af6bd0 2024-03-21T11:35:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.486+0000 7f14981d3740 1 -- 172.21.15.47:0/1136017679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af6bd0 msgr2=0x561943b07890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.486+0000 7f14981d3740 1 --2- 172.21.15.47:0/1136017679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af6bd0 0x561943b07890 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f147c0bd840 tx=0x7f147c0eef60 comp rx=0 tx=0).stop 2024-03-21T11:35:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.486+0000 7f14981d3740 1 -- 172.21.15.47:0/1136017679 shutdown_connections 2024-03-21T11:35:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.486+0000 7f14981d3740 1 --2- 172.21.15.47:0/1136017679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943afbd40 0x561943af0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.486+0000 7f14981d3740 1 --2- 172.21.15.47:0/1136017679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943af6bd0 0x561943b07890 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.486+0000 7f14981d3740 1 --2- 172.21.15.47:0/1136017679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af61e0 0x561943af00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.486+0000 7f14981d3740 1 -- 172.21.15.47:0/1136017679 >> 172.21.15.47:0/1136017679 conn(0x561943af1750 msgr2=0x561943af1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.486+0000 7f14981d3740 1 -- 172.21.15.47:0/1136017679 shutdown_connections 2024-03-21T11:35:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.486+0000 7f14981d3740 1 -- 172.21.15.47:0/1136017679 wait complete. 2024-03-21T11:35:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.487+0000 7f14981d3740 1 -- start start 2024-03-21T11:35:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.487+0000 7f14981d3740 1 Processor -- start 2024-03-21T11:35:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.487+0000 7f14981d3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943af61e0 0x561943b055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.487+0000 7f14981d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af6bd0 0x561943b05b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.487+0000 7f14981d3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943afbd40 0x561943b06b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.487+0000 7f14981d3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14901b5db0 con 0x561943af61e0 2024-03-21T11:35:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.487+0000 7f14981d3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14901b5750 con 0x561943afbd40 2024-03-21T11:35:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.487+0000 7f14981d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14901b2c50 con 0x561943af6bd0 2024-03-21T11:35:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.487+0000 7f14981d3740 10 client.0 _ll_register_callbacks cb 0x7f147c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.488+0000 7f148de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af6bd0 0x561943b05b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.488+0000 7f148de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af6bd0 0x561943b05b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50278/0 (socket says 172.21.15.47:50278) 2024-03-21T11:35:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.488+0000 7f148de00640 1 -- 172.21.15.47:0/3922200430 learned_addr learned my addr 172.21.15.47:0/3922200430 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.488+0000 7f148f200640 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943afbd40 0x561943b06b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.488+0000 7f148e800640 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943af61e0 0x561943b055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.488+0000 7f148f200640 1 -- 172.21.15.47:0/3922200430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af6bd0 msgr2=0x561943b05b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.488+0000 7f148f200640 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af6bd0 0x561943b05b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.488+0000 7f148f200640 1 -- 172.21.15.47:0/3922200430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943af61e0 msgr2=0x561943b055f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.488+0000 7f148f200640 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943af61e0 0x561943b055f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.488+0000 7f148f200640 1 -- 172.21.15.47:0/3922200430 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561943b07130 con 0x561943afbd40 2024-03-21T11:35:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.488+0000 7f148e800640 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943af61e0 0x561943b055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.488+0000 7f148de00640 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af6bd0 0x561943b05b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.489+0000 7f148f200640 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943afbd40 0x561943b06b40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f148801b040 tx=0x7f1488009d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15fcb5caac13c5e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.489+0000 7f146fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:13.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.489+0000 7f146fe00640 1 -- 172.21.15.47:0/3922200430 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488005590 con 0x561943afbd40 2024-03-21T11:35:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.489+0000 7f146fe00640 1 -- 172.21.15.47:0/3922200430 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1488005730 con 0x561943afbd40 2024-03-21T11:35:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.489+0000 7f14981d3740 1 -- 172.21.15.47:0/3922200430 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f147c004c80 con 0x561943afbd40 2024-03-21T11:35:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.490+0000 7f146fe00640 1 -- 172.21.15.47:0/3922200430 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1488005930 con 0x561943afbd40 2024-03-21T11:35:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.490+0000 7f14981d3740 10 client.29555 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.490+0000 7f14981d3740 1 -- 172.21.15.47:0/3922200430 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561943b68120 con 0x561943afbd40 2024-03-21T11:35:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.490+0000 7f146fe00640 1 -- 172.21.15.47:0/3922200430 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f147c004c80 con 0x561943afbd40 2024-03-21T11:35:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.490+0000 7f146fe00640 1 -- 172.21.15.47:0/3922200430 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f148802d630 con 0x561943afbd40 2024-03-21T11:35:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.490+0000 7f14981d3740 10 client.29555 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.490+0000 7f14981d3740 10 client.29555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.490+0000 7f14981d3740 20 client.29555 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.490+0000 7f14981d3740 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561943b0c1a0 0x561943b6a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.490+0000 7f14981d3740 4 client.29555 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:35:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.490+0000 7f14981d3740 1 -- 172.21.15.47:0/3922200430 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561943a630c0 con 0x561943b0c1a0 2024-03-21T11:35:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.491+0000 7f148e800640 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561943b0c1a0 0x561943b6a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.491+0000 7f148e800640 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561943b0c1a0 0x561943b6a910 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c1605e9e365d8b2 server_cookie=e19222767be45345 in_seq=0 out_seq=0 2024-03-21T11:35:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.491+0000 7f146fe00640 10 client.29555 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:13.566 INFO:tasks.workunit.client.0.smithi047.stdout: 1 898165 7.98 MB/sec execute 477 sec latency 32.195 ms 2024-03-21T11:35:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.758+0000 7f146fe00640 1 -- 172.21.15.47:0/3922200430 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561943a630c0 con 0x561943b0c1a0 2024-03-21T11:35:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.758+0000 7f146fe00640 10 client.29555 handle_command_reply: tid=0 2024-03-21T11:35:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.759+0000 7f14981d3740 1 client.29555 shutdown 2024-03-21T11:35:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.759+0000 7f14981d3740 1 -- 172.21.15.47:0/3922200430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943afbd40 msgr2=0x561943b06b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.759+0000 7f14981d3740 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943afbd40 0x561943b06b40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f148801b040 tx=0x7f1488009d60 comp rx=0 tx=0).stop 2024-03-21T11:35:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.759+0000 7f14981d3740 1 -- 172.21.15.47:0/3922200430 shutdown_connections 2024-03-21T11:35:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.759+0000 7f14981d3740 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561943b0c1a0 0x561943b6a910 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.759+0000 7f14981d3740 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561943afbd40 0x561943b06b40 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.759+0000 7f14981d3740 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561943af6bd0 0x561943b05b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.759+0000 7f14981d3740 1 --2- 172.21.15.47:0/3922200430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561943af61e0 0x561943b055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.759+0000 7f146fe00640 0 client.29555 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.759+0000 7f14981d3740 1 -- 172.21.15.47:0/3922200430 >> 172.21.15.47:0/3922200430 conn(0x561943af1750 msgr2=0x561943af49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.759+0000 7f14981d3740 1 -- 172.21.15.47:0/3922200430 shutdown_connections 2024-03-21T11:35:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.759+0000 7f14981d3740 1 -- 172.21.15.47:0/3922200430 wait complete. 2024-03-21T11:35:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.760+0000 7f14981d3740 20 client.29555 trim_cache size 0 max 16384 2024-03-21T11:35:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:35:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:35:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:35:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:35:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.761+0000 7f1495400640 1 -- 172.21.15.47:0/3881095157 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14740a3040 msgr2=0x7f14740a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.761+0000 7f1495400640 1 --2- 172.21.15.47:0/3881095157 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14740a3040 0x7f14740a5500 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x561943af26c0 tx=0x7f148000b390 comp rx=0 tx=0).stop 2024-03-21T11:35:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.761+0000 7f1495400640 1 -- 172.21.15.47:0/3881095157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149010b780 msgr2=0x7f14901a28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.761+0000 7f1495400640 1 --2- 172.21.15.47:0/3881095157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149010b780 0x7f14901a28a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f147c000c00 tx=0x7f147c02c6c0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.761+0000 7f1495400640 1 -- 172.21.15.47:0/3881095157 shutdown_connections 2024-03-21T11:35:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.761+0000 7f1495400640 1 --2- 172.21.15.47:0/3881095157 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14740a3040 0x7f14740a5500 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.761+0000 7f1495400640 1 --2- 172.21.15.47:0/3881095157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f149010e1e0 0x7f14901a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.761+0000 7f1495400640 1 --2- 172.21.15.47:0/3881095157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149010b780 0x7f14901a28a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.761+0000 7f1495400640 1 --2- 172.21.15.47:0/3881095157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149006a4a0 0x7f14901a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.761+0000 7f1495400640 1 -- 172.21.15.47:0/3881095157 >> 172.21.15.47:0/3881095157 conn(0x7f149006e8a0 msgr2=0x7f1490113950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.761+0000 7f1495400640 1 -- 172.21.15.47:0/3881095157 shutdown_connections 2024-03-21T11:35:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:13.762+0000 7f1495400640 1 -- 172.21.15.47:0/3881095157 wait complete. 2024-03-21T11:35:14.566 INFO:tasks.workunit.client.0.smithi047.stdout: 1 898793 7.97 MB/sec execute 478 sec latency 122.041 ms 2024-03-21T11:35:14.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:14 smithi102 ceph-mon[36748]: pgmap v757: 129 pgs: 129 active+clean; 42 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 280 op/s 2024-03-21T11:35:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:14 smithi047 ceph-mon[31351]: pgmap v757: 129 pgs: 129 active+clean; 42 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 280 op/s 2024-03-21T11:35:15.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:14 smithi100 ceph-mon[36245]: pgmap v757: 129 pgs: 129 active+clean; 42 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 280 op/s 2024-03-21T11:35:15.566 INFO:tasks.workunit.client.0.smithi047.stdout: 1 900155 7.96 MB/sec execute 479 sec latency 16.044 ms 2024-03-21T11:35:16.566 INFO:tasks.workunit.client.0.smithi047.stdout: 1 901828 7.97 MB/sec execute 480 sec latency 16.538 ms 2024-03-21T11:35:16.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:16 smithi102 ceph-mon[36748]: pgmap v758: 129 pgs: 129 active+clean; 43 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-03-21T11:35:17.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:16 smithi047 ceph-mon[31351]: pgmap v758: 129 pgs: 129 active+clean; 43 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-03-21T11:35:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:16 smithi100 ceph-mon[36245]: pgmap v758: 129 pgs: 129 active+clean; 43 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-03-21T11:35:17.567 INFO:tasks.workunit.client.0.smithi047.stdout: 1 902486 7.96 MB/sec execute 481 sec latency 52.149 ms 2024-03-21T11:35:18.567 INFO:tasks.workunit.client.0.smithi047.stdout: 1 904150 7.96 MB/sec execute 482 sec latency 20.108 ms 2024-03-21T11:35:18.944 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:18 smithi100 ceph-mon[36245]: pgmap v759: 129 pgs: 129 active+clean; 36 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-03-21T11:35:18.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:18 smithi102 ceph-mon[36748]: pgmap v759: 129 pgs: 129 active+clean; 36 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-03-21T11:35:19.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:18 smithi047 ceph-mon[31351]: pgmap v759: 129 pgs: 129 active+clean; 36 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-03-21T11:35:19.567 INFO:tasks.workunit.client.0.smithi047.stdout: 1 905673 7.96 MB/sec execute 483 sec latency 10.790 ms 2024-03-21T11:35:20.567 INFO:tasks.workunit.client.0.smithi047.stdout: 1 906743 7.96 MB/sec execute 484 sec latency 23.538 ms 2024-03-21T11:35:20.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:20 smithi102 ceph-mon[36748]: pgmap v760: 129 pgs: 129 active+clean; 37 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-03-21T11:35:21.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:20 smithi047 ceph-mon[31351]: pgmap v760: 129 pgs: 129 active+clean; 37 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-03-21T11:35:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:20 smithi100 ceph-mon[36245]: pgmap v760: 129 pgs: 129 active+clean; 37 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-03-21T11:35:21.567 INFO:tasks.workunit.client.0.smithi047.stdout: 1 908790 7.97 MB/sec execute 485 sec latency 13.984 ms 2024-03-21T11:35:22.567 INFO:tasks.workunit.client.0.smithi047.stdout: 1 909576 7.96 MB/sec execute 486 sec latency 59.709 ms 2024-03-21T11:35:22.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:22 smithi102 ceph-mon[36748]: pgmap v761: 129 pgs: 129 active+clean; 37 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 252 op/s 2024-03-21T11:35:23.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:22 smithi047 ceph-mon[31351]: pgmap v761: 129 pgs: 129 active+clean; 37 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 252 op/s 2024-03-21T11:35:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:22 smithi100 ceph-mon[36245]: pgmap v761: 129 pgs: 129 active+clean; 37 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 252 op/s 2024-03-21T11:35:23.567 INFO:tasks.workunit.client.0.smithi047.stdout: 1 911317 7.95 MB/sec execute 487 sec latency 18.862 ms 2024-03-21T11:35:24.570 INFO:tasks.workunit.client.0.smithi047.stdout: 1 912708 7.96 MB/sec execute 488 sec latency 14.924 ms 2024-03-21T11:35:24.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:24 smithi102 ceph-mon[36748]: pgmap v762: 129 pgs: 129 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 304 op/s 2024-03-21T11:35:25.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:24 smithi047 ceph-mon[31351]: pgmap v762: 129 pgs: 129 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 304 op/s 2024-03-21T11:35:25.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:24 smithi100 ceph-mon[36245]: pgmap v762: 129 pgs: 129 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 304 op/s 2024-03-21T11:35:25.568 INFO:tasks.workunit.client.0.smithi047.stdout: 1 913834 7.96 MB/sec execute 489 sec latency 14.361 ms 2024-03-21T11:35:26.568 INFO:tasks.workunit.client.0.smithi047.stdout: 1 915744 7.96 MB/sec execute 490 sec latency 14.023 ms 2024-03-21T11:35:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.910+0000 7fb9bb000640 1 -- 172.21.15.47:0/2648514245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4107b20 msgr2=0x7fb9b4107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.910+0000 7fb9bb000640 1 --2- 172.21.15.47:0/2648514245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4107b20 0x7fb9b4107f40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009860 tx=0x7fb9a402bb70 comp rx=0 tx=0).stop 2024-03-21T11:35:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.910+0000 7fb9bb000640 1 -- 172.21.15.47:0/2648514245 shutdown_connections 2024-03-21T11:35:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.910+0000 7fb9bb000640 1 --2- 172.21.15.47:0/2648514245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b41096d0 0x7fb9b410ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.910+0000 7fb9bb000640 1 --2- 172.21.15.47:0/2648514245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9b4108d10 0x7fb9b4109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.910+0000 7fb9bb000640 1 --2- 172.21.15.47:0/2648514245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4107b20 0x7fb9b4107f40 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.910+0000 7fb9bb000640 1 -- 172.21.15.47:0/2648514245 >> 172.21.15.47:0/2648514245 conn(0x7fb9b4076ae0 msgr2=0x7fb9b4078f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.910+0000 7fb9bb000640 1 -- 172.21.15.47:0/2648514245 shutdown_connections 2024-03-21T11:35:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.910+0000 7fb9bb000640 1 -- 172.21.15.47:0/2648514245 wait complete. 2024-03-21T11:35:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.911+0000 7fb9bb000640 1 Processor -- start 2024-03-21T11:35:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.911+0000 7fb9bb000640 1 -- start start 2024-03-21T11:35:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9bb000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4107b20 0x7fb9b419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9bb000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b4108d10 0x7fb9b419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9bb000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9b41096d0 0x7fb9b41a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9bb000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41137a0 con 0x7fb9b41096d0 2024-03-21T11:35:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9bb000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b4113620 con 0x7fb9b4108d10 2024-03-21T11:35:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9bb000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b4113920 con 0x7fb9b4107b20 2024-03-21T11:35:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9b9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4107b20 0x7fb9b419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9b9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4107b20 0x7fb9b419de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46242/0 (socket says 172.21.15.47:46242) 2024-03-21T11:35:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9b9c00640 1 -- 172.21.15.47:0/1879707364 learned_addr learned my addr 172.21.15.47:0/1879707364 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9b9200640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b4108d10 0x7fb9b419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9ba600640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9b41096d0 0x7fb9b41a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9b9c00640 1 -- 172.21.15.47:0/1879707364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b4108d10 msgr2=0x7fb9b419e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9b9c00640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b4108d10 0x7fb9b419e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9b9c00640 1 -- 172.21.15.47:0/1879707364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9b41096d0 msgr2=0x7fb9b41a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9b9c00640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9b41096d0 0x7fb9b41a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9b9c00640 1 -- 172.21.15.47:0/1879707364 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b41a5a30 con 0x7fb9b4107b20 2024-03-21T11:35:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.912+0000 7fb9ba600640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9b41096d0 0x7fb9b41a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.913+0000 7fb9b9c00640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4107b20 0x7fb9b419de10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb9a402c0b0 tx=0x7fb9a4032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.913+0000 7fb9b2000640 1 -- 172.21.15.47:0/1879707364 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4041020 con 0x7fb9b4107b20 2024-03-21T11:35:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.913+0000 7fb9b2000640 1 -- 172.21.15.47:0/1879707364 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9a402c9f0 con 0x7fb9b4107b20 2024-03-21T11:35:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.913+0000 7fb9bb000640 1 -- 172.21.15.47:0/1879707364 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b41a5cc0 con 0x7fb9b4107b20 2024-03-21T11:35:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.913+0000 7fb9b2000640 1 -- 172.21.15.47:0/1879707364 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4039040 con 0x7fb9b4107b20 2024-03-21T11:35:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.913+0000 7fb9bb000640 1 -- 172.21.15.47:0/1879707364 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b41a61b0 con 0x7fb9b4107b20 2024-03-21T11:35:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.916+0000 7fb9b2000640 1 -- 172.21.15.47:0/1879707364 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb9a4044020 con 0x7fb9b4107b20 2024-03-21T11:35:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.917+0000 7fb9bf374740 1 Processor -- start 2024-03-21T11:35:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.917+0000 7fb9bf374740 1 -- start start 2024-03-21T11:35:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.917+0000 7fb9bf374740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c953f0 0x563016c95810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.917+0000 7fb9bf374740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c95de0 0x563016c9a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.917+0000 7fb9bf374740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c9af50 0x563016c9d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.917+0000 7fb9bf374740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563016c583b0 con 0x563016c953f0 2024-03-21T11:35:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.917+0000 7fb9bf374740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563016bfa830 con 0x563016c95de0 2024-03-21T11:35:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.917+0000 7fb9bf374740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563016ba35d0 con 0x563016c9af50 2024-03-21T11:35:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.918+0000 7fb9ba600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c953f0 0x563016c95810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.918+0000 7fb9b9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c95de0 0x563016c9a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.918+0000 7fb9b9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c9af50 0x563016c9d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.918+0000 7fb9ba600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c953f0 0x563016c95810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44786/0 (socket says 172.21.15.47:44786) 2024-03-21T11:35:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.918+0000 7fb9ba600640 1 -- 172.21.15.47:0/863298175 learned_addr learned my addr 172.21.15.47:0/863298175 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.918+0000 7fb9b9200640 1 -- 172.21.15.47:0/863298175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c95de0 msgr2=0x563016c9a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.918+0000 7fb9b9200640 1 --2- 172.21.15.47:0/863298175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c95de0 0x563016c9a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.918+0000 7fb9b9200640 1 -- 172.21.15.47:0/863298175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c953f0 msgr2=0x563016c95810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.918+0000 7fb9b9200640 1 --2- 172.21.15.47:0/863298175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c953f0 0x563016c95810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.918+0000 7fb9b9200640 1 -- 172.21.15.47:0/863298175 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563016bbb2c0 con 0x563016c9af50 2024-03-21T11:35:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.918+0000 7fb9b2000640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb99c0a2f90 0x7fb99c0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.918+0000 7fb9b9200640 1 --2- 172.21.15.47:0/863298175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c9af50 0x563016c9d3e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009bf0 tx=0x7fb9a8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=297443e57a5abea9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.919+0000 7fb9b2000640 1 -- 172.21.15.47:0/1879707364 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb9a40f1120 con 0x7fb9b4107b20 2024-03-21T11:35:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.919+0000 7fb9ba600640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb99c0a2f90 0x7fb99c0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.919+0000 7fb9a2a00640 1 -- 172.21.15.47:0/863298175 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8004050 con 0x563016c9af50 2024-03-21T11:35:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.919+0000 7fb9a2a00640 1 -- 172.21.15.47:0/863298175 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9a8005500 con 0x563016c9af50 2024-03-21T11:35:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.919+0000 7fb9a2a00640 1 -- 172.21.15.47:0/863298175 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8004e40 con 0x563016c9af50 2024-03-21T11:35:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.920+0000 7fb9bf374740 1 -- 172.21.15.47:0/863298175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c9af50 msgr2=0x563016c9d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.920+0000 7fb9ba600640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb99c0a2f90 0x7fb99c0a5450 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x563016c96640 tx=0x7fb9ac011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.920+0000 7fb9bf374740 1 --2- 172.21.15.47:0/863298175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c9af50 0x563016c9d3e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009bf0 tx=0x7fb9a8011810 comp rx=0 tx=0).stop 2024-03-21T11:35:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.920+0000 7fb9bf374740 1 -- 172.21.15.47:0/863298175 shutdown_connections 2024-03-21T11:35:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.920+0000 7fb9bf374740 1 --2- 172.21.15.47:0/863298175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c9af50 0x563016c9d3e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.920+0000 7fb9bf374740 1 --2- 172.21.15.47:0/863298175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c95de0 0x563016c9a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.920+0000 7fb9bf374740 1 --2- 172.21.15.47:0/863298175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c953f0 0x563016c95810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.920+0000 7fb9bf374740 1 -- 172.21.15.47:0/863298175 >> 172.21.15.47:0/863298175 conn(0x563016c90960 msgr2=0x563016c90d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.920+0000 7fb9bf374740 1 -- 172.21.15.47:0/863298175 shutdown_connections 2024-03-21T11:35:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.920+0000 7fb9bf374740 1 -- 172.21.15.47:0/863298175 wait complete. 2024-03-21T11:35:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.921+0000 7fb9bf374740 1 -- start start 2024-03-21T11:35:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.921+0000 7fb9bf374740 1 Processor -- start 2024-03-21T11:35:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.921+0000 7fb9bf374740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c953f0 0x563016c9f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.921+0000 7fb9bf374740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c95de0 0x563016c9fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.921+0000 7fb9bf374740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c9af50 0x563016ca6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.921+0000 7fb9bf374740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563016bb6a20 con 0x563016c9af50 2024-03-21T11:35:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.921+0000 7fb9bf374740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563016bf68d0 con 0x563016c953f0 2024-03-21T11:35:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.921+0000 7fb9bf374740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563016c9df00 con 0x563016c95de0 2024-03-21T11:35:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.921+0000 7fb9bf374740 10 client.0 _ll_register_callbacks cb 0x563016bf31c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.922+0000 7fb9b9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c953f0 0x563016c9f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.922+0000 7fb9ba600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c9af50 0x563016ca6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.922+0000 7fb9b9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c95de0 0x563016c9fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.922+0000 7fb9b9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c953f0 0x563016c9f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33612/0 (socket says 172.21.15.47:33612) 2024-03-21T11:35:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.922+0000 7fb9b9c00640 1 -- 172.21.15.47:0/199253568 learned_addr learned my addr 172.21.15.47:0/199253568 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.922+0000 7fb9b9c00640 1 -- 172.21.15.47:0/199253568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c95de0 msgr2=0x563016c9fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.922+0000 7fb9b9c00640 1 --2- 172.21.15.47:0/199253568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c95de0 0x563016c9fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.922+0000 7fb9b9c00640 1 -- 172.21.15.47:0/199253568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c9af50 msgr2=0x563016ca6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.922+0000 7fb9b9c00640 1 --2- 172.21.15.47:0/199253568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c9af50 0x563016ca6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.922+0000 7fb9b9c00640 1 -- 172.21.15.47:0/199253568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56301690f210 con 0x563016c953f0 2024-03-21T11:35:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.922+0000 7fb9b9c00640 1 --2- 172.21.15.47:0/199253568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c953f0 0x563016c9f5d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4000c00 tx=0x7fb9a4005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d17e1a1795d0f8ab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.922+0000 7fb997400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.922+0000 7fb997400640 1 -- 172.21.15.47:0/199253568 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a40bd230 con 0x563016c953f0 2024-03-21T11:35:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.923+0000 7fb9bf374740 1 -- 172.21.15.47:0/199253568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9a8005500 con 0x563016c953f0 2024-03-21T11:35:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.923+0000 7fb997400640 1 -- 172.21.15.47:0/199253568 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb9a40042c0 con 0x563016c953f0 2024-03-21T11:35:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.923+0000 7fb997400640 1 -- 172.21.15.47:0/199253568 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9a40bcdc0 con 0x563016c953f0 2024-03-21T11:35:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.923+0000 7fb9bf374740 10 client.29567 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.923+0000 7fb9bf374740 1 -- 172.21.15.47:0/199253568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5630169db620 con 0x563016c953f0 2024-03-21T11:35:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.923+0000 7fb997400640 1 -- 172.21.15.47:0/199253568 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9a8005500 con 0x563016c953f0 2024-03-21T11:35:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.923+0000 7fb997400640 1 -- 172.21.15.47:0/199253568 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb9a403c030 con 0x563016c953f0 2024-03-21T11:35:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.923+0000 7fb9bf374740 10 client.29567 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.923+0000 7fb9bf374740 10 client.29567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.924+0000 7fb9bf374740 20 client.29567 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.924+0000 7fb9bf374740 1 --2- 172.21.15.47:0/199253568 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563016cab380 0x563016cad820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.924+0000 7fb9bf374740 4 client.29567 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.924+0000 7fb9bf374740 1 -- 172.21.15.47:0/199253568 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563016c07d70 con 0x563016cab380 2024-03-21T11:35:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.924+0000 7fb9b9200640 1 --2- 172.21.15.47:0/199253568 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563016cab380 0x563016cad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.924+0000 7fb9b9200640 1 --2- 172.21.15.47:0/199253568 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563016cab380 0x563016cad820 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f13bc9ff57114e server_cookie=f51fd47b24f22a20 in_seq=0 out_seq=0 2024-03-21T11:35:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.924+0000 7fb997400640 10 client.29567 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.925+0000 7fb997400640 1 -- 172.21.15.47:0/199253568 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563016c07d70 con 0x563016cab380 2024-03-21T11:35:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.925+0000 7fb997400640 10 client.29567 handle_command_reply: tid=0 2024-03-21T11:35:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.925+0000 7fb9bf374740 1 client.29567 shutdown 2024-03-21T11:35:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.925+0000 7fb9bf374740 1 -- 172.21.15.47:0/199253568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c953f0 msgr2=0x563016c9f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.925+0000 7fb9bf374740 1 --2- 172.21.15.47:0/199253568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c953f0 0x563016c9f5d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4000c00 tx=0x7fb9a4005ab0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.926+0000 7fb9bf374740 1 -- 172.21.15.47:0/199253568 shutdown_connections 2024-03-21T11:35:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.926+0000 7fb9bf374740 1 --2- 172.21.15.47:0/199253568 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563016cab380 0x563016cad820 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.926+0000 7fb997400640 0 client.29567 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.926+0000 7fb9bf374740 1 --2- 172.21.15.47:0/199253568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c9af50 0x563016ca6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.926+0000 7fb9bf374740 1 --2- 172.21.15.47:0/199253568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c95de0 0x563016c9fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.926+0000 7fb9bf374740 1 --2- 172.21.15.47:0/199253568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c953f0 0x563016c9f5d0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.926+0000 7fb9bf374740 1 -- 172.21.15.47:0/199253568 >> 172.21.15.47:0/199253568 conn(0x563016c90960 msgr2=0x563016c8fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.926+0000 7fb9bf374740 1 -- 172.21.15.47:0/199253568 shutdown_connections 2024-03-21T11:35:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.926+0000 7fb9bf374740 1 -- 172.21.15.47:0/199253568 wait complete. 2024-03-21T11:35:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.926+0000 7fb9bf374740 20 client.29567 trim_cache size 0 max 16384 2024-03-21T11:35:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.942+0000 7fb9bf374740 1 Processor -- start 2024-03-21T11:35:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.942+0000 7fb9bf374740 1 -- start start 2024-03-21T11:35:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9bf374740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c953f0 0x563016c968a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9bf374740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c95de0 0x563016ca6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9bf374740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c9af50 0x563016c93bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9bf374740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41b0580 con 0x563016c95de0 2024-03-21T11:35:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9bf374740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41b1f00 con 0x563016c9af50 2024-03-21T11:35:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9bf374740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41b0be0 con 0x563016c953f0 2024-03-21T11:35:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9b9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c953f0 0x563016c968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9b9c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c953f0 0x563016c968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46260/0 (socket says 172.21.15.47:46260) 2024-03-21T11:35:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9b9c00640 1 -- 172.21.15.47:0/4061099333 learned_addr learned my addr 172.21.15.47:0/4061099333 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9ba600640 1 --2- 172.21.15.47:0/4061099333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c9af50 0x563016c93bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9b9200640 1 --2- 172.21.15.47:0/4061099333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c95de0 0x563016ca6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9b9c00640 1 -- 172.21.15.47:0/4061099333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c9af50 msgr2=0x563016c93bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9b9c00640 1 --2- 172.21.15.47:0/4061099333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c9af50 0x563016c93bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9b9c00640 1 -- 172.21.15.47:0/4061099333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c95de0 msgr2=0x563016ca6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9b9c00640 1 --2- 172.21.15.47:0/4061099333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c95de0 0x563016ca6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9b9c00640 1 -- 172.21.15.47:0/4061099333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563016c9a400 con 0x563016c953f0 2024-03-21T11:35:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.943+0000 7fb9b9200640 1 --2- 172.21.15.47:0/4061099333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c95de0 0x563016ca6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:35:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.944+0000 7fb9b9c00640 1 --2- 172.21.15.47:0/4061099333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c953f0 0x563016c968a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40b9e30 tx=0x7fb9a40f7000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=354a78f4836cce23 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.944+0000 7fb9a2000640 1 -- 172.21.15.47:0/4061099333 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a40bcfa0 con 0x563016c953f0 2024-03-21T11:35:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.944+0000 7fb9a2000640 1 -- 172.21.15.47:0/4061099333 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9a40bdb60 con 0x563016c953f0 2024-03-21T11:35:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.944+0000 7fb9a2000640 1 -- 172.21.15.47:0/4061099333 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a40c1160 con 0x563016c953f0 2024-03-21T11:35:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.945+0000 7fb9bf374740 1 -- 172.21.15.47:0/4061099333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c953f0 msgr2=0x563016c968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.945+0000 7fb9bf374740 1 --2- 172.21.15.47:0/4061099333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c953f0 0x563016c968a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40b9e30 tx=0x7fb9a40f7000 comp rx=0 tx=0).stop 2024-03-21T11:35:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.945+0000 7fb9bf374740 1 -- 172.21.15.47:0/4061099333 shutdown_connections 2024-03-21T11:35:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.945+0000 7fb9bf374740 1 --2- 172.21.15.47:0/4061099333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c9af50 0x563016c93bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.945+0000 7fb9bf374740 1 --2- 172.21.15.47:0/4061099333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c95de0 0x563016ca6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.945+0000 7fb9bf374740 1 --2- 172.21.15.47:0/4061099333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c953f0 0x563016c968a0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.945+0000 7fb9bf374740 1 -- 172.21.15.47:0/4061099333 >> 172.21.15.47:0/4061099333 conn(0x563016c90960 msgr2=0x563016c90d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.945+0000 7fb9bf374740 1 -- 172.21.15.47:0/4061099333 shutdown_connections 2024-03-21T11:35:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.945+0000 7fb9bf374740 1 -- 172.21.15.47:0/4061099333 wait complete. 2024-03-21T11:35:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.945+0000 7fb9bf374740 1 -- start start 2024-03-21T11:35:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.945+0000 7fb9bf374740 1 Processor -- start 2024-03-21T11:35:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9bf374740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c953f0 0x563016ca4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9bf374740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c95de0 0x563016ca4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9bf374740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c9af50 0x563016ca5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9bf374740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41b18a0 con 0x563016c953f0 2024-03-21T11:35:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9bf374740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41b1240 con 0x563016c95de0 2024-03-21T11:35:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9bf374740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41ae740 con 0x563016c9af50 2024-03-21T11:35:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9bf374740 10 client.0 _ll_register_callbacks cb 0x7fb9a40057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9b9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c95de0 0x563016ca4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9ba600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c9af50 0x563016ca5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9b9c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c953f0 0x563016ca4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9b9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c95de0 0x563016ca4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33622/0 (socket says 172.21.15.47:33622) 2024-03-21T11:35:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9ba600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c9af50 0x563016ca5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46262/0 (socket says 172.21.15.47:46262) 2024-03-21T11:35:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9b9200640 1 -- 172.21.15.47:0/2002865567 learned_addr learned my addr 172.21.15.47:0/2002865567 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9b9200640 1 -- 172.21.15.47:0/2002865567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c9af50 msgr2=0x563016ca5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.946+0000 7fb9b9200640 1 --2- 172.21.15.47:0/2002865567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c9af50 0x563016ca5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.947+0000 7fb9b9200640 1 -- 172.21.15.47:0/2002865567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c953f0 msgr2=0x563016ca4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.947+0000 7fb9b9200640 1 --2- 172.21.15.47:0/2002865567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c953f0 0x563016ca4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.947+0000 7fb9b9200640 1 -- 172.21.15.47:0/2002865567 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563016ca6250 con 0x563016c95de0 2024-03-21T11:35:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.947+0000 7fb9b9c00640 1 --2- 172.21.15.47:0/2002865567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c953f0 0x563016ca4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.947+0000 7fb9b9200640 1 --2- 172.21.15.47:0/2002865567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c95de0 0x563016ca4c50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009b40 tx=0x7fb9a8009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d2dbac5d5cbaed1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.947+0000 7fb997400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.947+0000 7fb997400640 1 -- 172.21.15.47:0/2002865567 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8034060 con 0x563016c95de0 2024-03-21T11:35:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.947+0000 7fb9bf374740 1 -- 172.21.15.47:0/2002865567 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9a40bdb60 con 0x563016c95de0 2024-03-21T11:35:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.947+0000 7fb997400640 1 -- 172.21.15.47:0/2002865567 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb9a8004250 con 0x563016c95de0 2024-03-21T11:35:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.947+0000 7fb997400640 1 -- 172.21.15.47:0/2002865567 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9a803b050 con 0x563016c95de0 2024-03-21T11:35:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.947+0000 7fb9bf374740 10 client.29573 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.947+0000 7fb9bf374740 1 -- 172.21.15.47:0/2002865567 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563016d07170 con 0x563016c95de0 2024-03-21T11:35:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.948+0000 7fb997400640 1 -- 172.21.15.47:0/2002865567 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9a40bdb60 con 0x563016c95de0 2024-03-21T11:35:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.948+0000 7fb997400640 1 -- 172.21.15.47:0/2002865567 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb9a802e5a0 con 0x563016c95de0 2024-03-21T11:35:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.948+0000 7fb9bf374740 10 client.29573 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.948+0000 7fb9bf374740 10 client.29573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.948+0000 7fb9bf374740 20 client.29573 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.948+0000 7fb9bf374740 1 --2- 172.21.15.47:0/2002865567 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563016cab380 0x563016d099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.948+0000 7fb9bf374740 4 client.29573 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:35:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.948+0000 7fb9bf374740 1 -- 172.21.15.47:0/2002865567 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563016c020c0 con 0x563016cab380 2024-03-21T11:35:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.948+0000 7fb9b9c00640 1 --2- 172.21.15.47:0/2002865567 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563016cab380 0x563016d099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.949+0000 7fb9b9c00640 1 --2- 172.21.15.47:0/2002865567 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563016cab380 0x563016d099c0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89bb3d52a08450d server_cookie=3bd494d0939ab816 in_seq=0 out_seq=0 2024-03-21T11:35:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:26.949+0000 7fb997400640 10 client.29573 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:26 smithi102 ceph-mon[36748]: pgmap v763: 129 pgs: 129 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-03-21T11:35:27.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:26 smithi047 ceph-mon[31351]: pgmap v763: 129 pgs: 129 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-03-21T11:35:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:26 smithi100 ceph-mon[36245]: pgmap v763: 129 pgs: 129 active+clean; 39 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-03-21T11:35:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.298+0000 7fb997400640 1 -- 172.21.15.47:0/2002865567 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563016c020c0 con 0x563016cab380 2024-03-21T11:35:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.298+0000 7fb997400640 10 client.29573 handle_command_reply: tid=0 2024-03-21T11:35:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.298+0000 7fb9bf374740 1 client.29573 shutdown 2024-03-21T11:35:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.298+0000 7fb9bf374740 1 -- 172.21.15.47:0/2002865567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c95de0 msgr2=0x563016ca4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.298+0000 7fb9bf374740 1 --2- 172.21.15.47:0/2002865567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c95de0 0x563016ca4c50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009b40 tx=0x7fb9a8009640 comp rx=0 tx=0).stop 2024-03-21T11:35:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.299+0000 7fb9bf374740 1 -- 172.21.15.47:0/2002865567 shutdown_connections 2024-03-21T11:35:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.299+0000 7fb9bf374740 1 --2- 172.21.15.47:0/2002865567 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563016cab380 0x563016d099c0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.299+0000 7fb9bf374740 1 --2- 172.21.15.47:0/2002865567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563016c9af50 0x563016ca5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.299+0000 7fb9bf374740 1 --2- 172.21.15.47:0/2002865567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563016c95de0 0x563016ca4c50 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.299+0000 7fb9bf374740 1 --2- 172.21.15.47:0/2002865567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563016c953f0 0x563016ca4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.299+0000 7fb9bf374740 1 -- 172.21.15.47:0/2002865567 >> 172.21.15.47:0/2002865567 conn(0x563016c90960 msgr2=0x563016c93bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.299+0000 7fb997400640 0 client.29573 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.299+0000 7fb9bf374740 1 -- 172.21.15.47:0/2002865567 shutdown_connections 2024-03-21T11:35:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.299+0000 7fb9bf374740 1 -- 172.21.15.47:0/2002865567 wait complete. 2024-03-21T11:35:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.299+0000 7fb9bf374740 20 client.29573 trim_cache size 0 max 16384 2024-03-21T11:35:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:35:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:35:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:35:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:35:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.300+0000 7fb9bb000640 1 -- 172.21.15.47:0/1879707364 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb99c0a2f90 msgr2=0x7fb99c0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.300+0000 7fb9bb000640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb99c0a2f90 0x7fb99c0a5450 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x563016c96640 tx=0x7fb9ac011040 comp rx=0 tx=0).stop 2024-03-21T11:35:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.300+0000 7fb9bb000640 1 -- 172.21.15.47:0/1879707364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4107b20 msgr2=0x7fb9b419de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.301+0000 7fb9bb000640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4107b20 0x7fb9b419de10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb9a402c0b0 tx=0x7fb9a4032430 comp rx=0 tx=0).stop 2024-03-21T11:35:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.301+0000 7fb9bb000640 1 -- 172.21.15.47:0/1879707364 shutdown_connections 2024-03-21T11:35:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.301+0000 7fb9bb000640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb99c0a2f90 0x7fb99c0a5450 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.301+0000 7fb9bb000640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9b41096d0 0x7fb9b41a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.301+0000 7fb9bb000640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b4108d10 0x7fb9b419e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.301+0000 7fb9bb000640 1 --2- 172.21.15.47:0/1879707364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9b4107b20 0x7fb9b419de10 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.301+0000 7fb9bb000640 1 -- 172.21.15.47:0/1879707364 >> 172.21.15.47:0/1879707364 conn(0x7fb9b4076ae0 msgr2=0x7fb9b410c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.302+0000 7fb9bb000640 1 -- 172.21.15.47:0/1879707364 shutdown_connections 2024-03-21T11:35:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:27.302+0000 7fb9bb000640 1 -- 172.21.15.47:0/1879707364 wait complete. 2024-03-21T11:35:27.568 INFO:tasks.workunit.client.0.smithi047.stdout: 1 916434 7.95 MB/sec execute 491 sec latency 128.354 ms 2024-03-21T11:35:28.568 INFO:tasks.workunit.client.0.smithi047.stdout: 1 917418 7.98 MB/sec execute 492 sec latency 52.350 ms 2024-03-21T11:35:28.993 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:28 smithi102 ceph-mon[36748]: pgmap v764: 129 pgs: 129 active+clean; 38 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 1.7 MiB/s wr, 235 op/s 2024-03-21T11:35:29.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:28 smithi047 ceph-mon[31351]: pgmap v764: 129 pgs: 129 active+clean; 38 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 1.7 MiB/s wr, 235 op/s 2024-03-21T11:35:29.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:28 smithi100 ceph-mon[36245]: pgmap v764: 129 pgs: 129 active+clean; 38 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 1.7 MiB/s wr, 235 op/s 2024-03-21T11:35:29.568 INFO:tasks.workunit.client.0.smithi047.stdout: 1 919456 7.98 MB/sec execute 493 sec latency 13.271 ms 2024-03-21T11:35:30.568 INFO:tasks.workunit.client.0.smithi047.stdout: 1 920912 7.99 MB/sec execute 494 sec latency 16.345 ms 2024-03-21T11:35:30.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:30 smithi102 ceph-mon[36748]: pgmap v765: 129 pgs: 129 active+clean; 40 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 266 op/s 2024-03-21T11:35:31.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:30 smithi047 ceph-mon[31351]: pgmap v765: 129 pgs: 129 active+clean; 40 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 266 op/s 2024-03-21T11:35:31.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:30 smithi100 ceph-mon[36245]: pgmap v765: 129 pgs: 129 active+clean; 40 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 266 op/s 2024-03-21T11:35:31.568 INFO:tasks.workunit.client.0.smithi047.stdout: 1 922737 7.98 MB/sec execute 495 sec latency 12.504 ms 2024-03-21T11:35:32.569 INFO:tasks.workunit.client.0.smithi047.stdout: 1 924208 7.99 MB/sec execute 496 sec latency 47.143 ms 2024-03-21T11:35:32.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:32 smithi102 ceph-mon[36748]: pgmap v766: 129 pgs: 129 active+clean; 36 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.5 MiB/s wr, 253 op/s 2024-03-21T11:35:33.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:32 smithi047 ceph-mon[31351]: pgmap v766: 129 pgs: 129 active+clean; 36 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.5 MiB/s wr, 253 op/s 2024-03-21T11:35:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:32 smithi100 ceph-mon[36245]: pgmap v766: 129 pgs: 129 active+clean; 36 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.5 MiB/s wr, 253 op/s 2024-03-21T11:35:33.569 INFO:tasks.workunit.client.0.smithi047.stdout: 1 925637 7.99 MB/sec execute 497 sec latency 17.995 ms 2024-03-21T11:35:34.571 INFO:tasks.workunit.client.0.smithi047.stdout: 1 927646 8.00 MB/sec execute 498 sec latency 11.809 ms 2024-03-21T11:35:34.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:34 smithi102 ceph-mon[36748]: pgmap v767: 129 pgs: 129 active+clean; 26 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 326 op/s 2024-03-21T11:35:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:34 smithi047 ceph-mon[31351]: pgmap v767: 129 pgs: 129 active+clean; 26 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 326 op/s 2024-03-21T11:35:35.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:34 smithi100 ceph-mon[36245]: pgmap v767: 129 pgs: 129 active+clean; 26 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 326 op/s 2024-03-21T11:35:35.571 INFO:tasks.workunit.client.0.smithi047.stdout: 1 928975 7.99 MB/sec execute 499 sec latency 12.477 ms 2024-03-21T11:35:36.571 INFO:tasks.workunit.client.0.smithi047.stdout: 1 930882 8.00 MB/sec execute 500 sec latency 15.195 ms 2024-03-21T11:35:36.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:36 smithi102 ceph-mon[36748]: pgmap v768: 129 pgs: 129 active+clean; 21 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.8 MiB/s wr, 273 op/s 2024-03-21T11:35:37.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:36 smithi047 ceph-mon[31351]: pgmap v768: 129 pgs: 129 active+clean; 21 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.8 MiB/s wr, 273 op/s 2024-03-21T11:35:37.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:36 smithi100 ceph-mon[36245]: pgmap v768: 129 pgs: 129 active+clean; 21 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.8 MiB/s wr, 273 op/s 2024-03-21T11:35:37.571 INFO:tasks.workunit.client.0.smithi047.stdout: 1 931605 7.99 MB/sec execute 501 sec latency 30.733 ms 2024-03-21T11:35:38.571 INFO:tasks.workunit.client.0.smithi047.stdout: 1 933227 7.99 MB/sec execute 502 sec latency 20.958 ms 2024-03-21T11:35:38.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:38 smithi102 ceph-mon[36748]: pgmap v769: 129 pgs: 129 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 1.8 MiB/s wr, 314 op/s 2024-03-21T11:35:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:38 smithi047 ceph-mon[31351]: pgmap v769: 129 pgs: 129 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 1.8 MiB/s wr, 314 op/s 2024-03-21T11:35:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:38 smithi100 ceph-mon[36245]: pgmap v769: 129 pgs: 129 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 1.8 MiB/s wr, 314 op/s 2024-03-21T11:35:39.571 INFO:tasks.workunit.client.0.smithi047.stdout: 1 934639 7.99 MB/sec execute 503 sec latency 15.631 ms 2024-03-21T11:35:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.465+0000 7fe934e00640 1 -- 172.21.15.47:0/1677022240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe930101510 msgr2=0x7fe930101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.465+0000 7fe934e00640 1 --2- 172.21.15.47:0/1677022240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe930101510 0x7fe930101930 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe920009900 tx=0x7fe92002bc30 comp rx=0 tx=0).stop 2024-03-21T11:35:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.465+0000 7fe934e00640 1 -- 172.21.15.47:0/1677022240 shutdown_connections 2024-03-21T11:35:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.465+0000 7fe934e00640 1 --2- 172.21.15.47:0/1677022240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9301030c0 0x7fe930109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.465+0000 7fe934e00640 1 --2- 172.21.15.47:0/1677022240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe930102700 0x7fe930102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.465+0000 7fe934e00640 1 --2- 172.21.15.47:0/1677022240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe930101510 0x7fe930101930 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.465+0000 7fe934e00640 1 -- 172.21.15.47:0/1677022240 >> 172.21.15.47:0/1677022240 conn(0x7fe9300fca80 msgr2=0x7fe9300feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.465+0000 7fe934e00640 1 -- 172.21.15.47:0/1677022240 shutdown_connections 2024-03-21T11:35:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.465+0000 7fe934e00640 1 -- 172.21.15.47:0/1677022240 wait complete. 2024-03-21T11:35:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.466+0000 7fe934e00640 1 Processor -- start 2024-03-21T11:35:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.466+0000 7fe934e00640 1 -- start start 2024-03-21T11:35:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.466+0000 7fe934e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe930101510 0x7fe930199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.466+0000 7fe934e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe930102700 0x7fe93019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.467+0000 7fe934e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9301030c0 0x7fe9301a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.467+0000 7fe934e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93010d1c0 con 0x7fe930102700 2024-03-21T11:35:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.467+0000 7fe934e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93010d040 con 0x7fe9301030c0 2024-03-21T11:35:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.467+0000 7fe934e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93010d340 con 0x7fe930101510 2024-03-21T11:35:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.467+0000 7fe92f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe930101510 0x7fe930199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.467+0000 7fe92f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe930101510 0x7fe930199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41288/0 (socket says 172.21.15.47:41288) 2024-03-21T11:35:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.467+0000 7fe92f400640 1 -- 172.21.15.47:0/917798267 learned_addr learned my addr 172.21.15.47:0/917798267 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.467+0000 7fe92fe00640 1 --2- 172.21.15.47:0/917798267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9301030c0 0x7fe9301a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.467+0000 7fe92ea00640 1 --2- 172.21.15.47:0/917798267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe930102700 0x7fe93019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.467+0000 7fe92f400640 1 -- 172.21.15.47:0/917798267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9301030c0 msgr2=0x7fe9301a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.467+0000 7fe92f400640 1 --2- 172.21.15.47:0/917798267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9301030c0 0x7fe9301a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.467+0000 7fe92f400640 1 -- 172.21.15.47:0/917798267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe930102700 msgr2=0x7fe93019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.467+0000 7fe92f400640 1 --2- 172.21.15.47:0/917798267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe930102700 0x7fe93019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.467+0000 7fe92f400640 1 -- 172.21.15.47:0/917798267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9301a16f0 con 0x7fe930101510 2024-03-21T11:35:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.468+0000 7fe92f400640 1 --2- 172.21.15.47:0/917798267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe930101510 0x7fe930199ad0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe920000c00 tx=0x7fe92002c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.468+0000 7fe927e00640 1 -- 172.21.15.47:0/917798267 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920043020 con 0x7fe930101510 2024-03-21T11:35:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.468+0000 7fe927e00640 1 -- 172.21.15.47:0/917798267 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9200054e0 con 0x7fe930101510 2024-03-21T11:35:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.468+0000 7fe927e00640 1 -- 172.21.15.47:0/917798267 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920003d70 con 0x7fe930101510 2024-03-21T11:35:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.468+0000 7fe934e00640 1 -- 172.21.15.47:0/917798267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9301a1980 con 0x7fe930101510 2024-03-21T11:35:40.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.468+0000 7fe934e00640 1 -- 172.21.15.47:0/917798267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9301a1ef0 con 0x7fe930101510 2024-03-21T11:35:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.473+0000 7fe939163740 1 Processor -- start 2024-03-21T11:35:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.473+0000 7fe939163740 1 -- start start 2024-03-21T11:35:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.474+0000 7fe927e00640 1 -- 172.21.15.47:0/917798267 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe920046020 con 0x7fe930101510 2024-03-21T11:35:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.474+0000 7fe939163740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea9813f0 0x5615ea981810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.474+0000 7fe939163740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea981de0 0x5615ea9868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.474+0000 7fe939163740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea986f50 0x5615ea9893e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.474+0000 7fe939163740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea9443b0 con 0x5615ea986f50 2024-03-21T11:35:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.474+0000 7fe939163740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea8e6830 con 0x5615ea981de0 2024-03-21T11:35:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.474+0000 7fe939163740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea88f5d0 con 0x5615ea9813f0 2024-03-21T11:35:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.474+0000 7fe92f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea981de0 0x5615ea9868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.474+0000 7fe92fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea9813f0 0x5615ea981810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.474+0000 7fe92fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea9813f0 0x5615ea981810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41304/0 (socket says 172.21.15.47:41304) 2024-03-21T11:35:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.474+0000 7fe92f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea981de0 0x5615ea9868e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49466/0 (socket says 172.21.15.47:49466) 2024-03-21T11:35:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.474+0000 7fe92fe00640 1 -- 172.21.15.47:0/2313225000 learned_addr learned my addr 172.21.15.47:0/2313225000 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.475+0000 7fe92fe00640 1 -- 172.21.15.47:0/2313225000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea981de0 msgr2=0x5615ea9868e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.475+0000 7fe92fe00640 1 --2- 172.21.15.47:0/2313225000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea981de0 0x5615ea9868e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.475+0000 7fe92fe00640 1 -- 172.21.15.47:0/2313225000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea986f50 msgr2=0x5615ea9893e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.475+0000 7fe92fe00640 1 --2- 172.21.15.47:0/2313225000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea986f50 0x5615ea9893e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.475+0000 7fe92fe00640 1 -- 172.21.15.47:0/2313225000 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615ea8a72c0 con 0x5615ea9813f0 2024-03-21T11:35:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.475+0000 7fe92fe00640 1 --2- 172.21.15.47:0/2313225000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea9813f0 0x5615ea981810 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe9280039f0 tx=0x7fe928003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3267106acf3236ee server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.475+0000 7fe924c00640 1 -- 172.21.15.47:0/2313225000 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92801a470 con 0x5615ea9813f0 2024-03-21T11:35:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.476+0000 7fe927e00640 1 --2- 172.21.15.47:0/917798267 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9140a2f90 0x7fe9140a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.476+0000 7fe924c00640 1 -- 172.21.15.47:0/2313225000 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe928015070 con 0x5615ea9813f0 2024-03-21T11:35:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.476+0000 7fe924c00640 1 -- 172.21.15.47:0/2313225000 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928010900 con 0x5615ea9813f0 2024-03-21T11:35:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.476+0000 7fe927e00640 1 -- 172.21.15.47:0/917798267 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe9200f1880 con 0x7fe930101510 2024-03-21T11:35:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.476+0000 7fe92f400640 1 --2- 172.21.15.47:0/917798267 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9140a2f90 0x7fe9140a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.477+0000 7fe939163740 1 -- 172.21.15.47:0/2313225000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea9813f0 msgr2=0x5615ea981810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.477+0000 7fe939163740 1 --2- 172.21.15.47:0/2313225000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea9813f0 0x5615ea981810 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe9280039f0 tx=0x7fe928003eb0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.477+0000 7fe939163740 1 -- 172.21.15.47:0/2313225000 shutdown_connections 2024-03-21T11:35:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.477+0000 7fe939163740 1 --2- 172.21.15.47:0/2313225000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea986f50 0x5615ea9893e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.477+0000 7fe939163740 1 --2- 172.21.15.47:0/2313225000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea981de0 0x5615ea9868e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.477+0000 7fe939163740 1 --2- 172.21.15.47:0/2313225000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea9813f0 0x5615ea981810 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.477+0000 7fe939163740 1 -- 172.21.15.47:0/2313225000 >> 172.21.15.47:0/2313225000 conn(0x5615ea97c960 msgr2=0x5615ea97cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.477+0000 7fe939163740 1 -- 172.21.15.47:0/2313225000 shutdown_connections 2024-03-21T11:35:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.477+0000 7fe92f400640 1 --2- 172.21.15.47:0/917798267 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9140a2f90 0x7fe9140a5450 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe9200ef360 tx=0x7fe9200f7550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.477+0000 7fe939163740 1 -- 172.21.15.47:0/2313225000 wait complete. 2024-03-21T11:35:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.478+0000 7fe939163740 1 -- start start 2024-03-21T11:35:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.478+0000 7fe939163740 1 Processor -- start 2024-03-21T11:35:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe939163740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea9813f0 0x5615ea98b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe939163740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea981de0 0x5615ea98bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe939163740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea986f50 0x5615ea992bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe939163740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea8a2a20 con 0x5615ea981de0 2024-03-21T11:35:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe939163740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea8e28d0 con 0x5615ea9813f0 2024-03-21T11:35:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe939163740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ea989f00 con 0x5615ea986f50 2024-03-21T11:35:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe939163740 10 client.0 _ll_register_callbacks cb 0x5615ea8df1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe92fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea9813f0 0x5615ea98b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe92fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea9813f0 0x5615ea98b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49468/0 (socket says 172.21.15.47:49468) 2024-03-21T11:35:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe92fe00640 1 -- 172.21.15.47:0/2363361389 learned_addr learned my addr 172.21.15.47:0/2363361389 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe92fe00640 1 --2- 172.21.15.47:0/2363361389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea986f50 0x5615ea992bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe92fe00640 1 -- 172.21.15.47:0/2363361389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea986f50 msgr2=0x5615ea992bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe92fe00640 1 --2- 172.21.15.47:0/2363361389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea986f50 0x5615ea992bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe92fe00640 1 -- 172.21.15.47:0/2363361389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea981de0 msgr2=0x5615ea98bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.479+0000 7fe92fe00640 1 --2- 172.21.15.47:0/2363361389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea981de0 0x5615ea98bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.480+0000 7fe92fe00640 1 -- 172.21.15.47:0/2363361389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615ea5fb210 con 0x5615ea9813f0 2024-03-21T11:35:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.480+0000 7fe92fe00640 1 --2- 172.21.15.47:0/2363361389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea9813f0 0x5615ea98b6a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe928011f60 tx=0x7fe928011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c78e3a2ff6e6bf0f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.480+0000 7fe919600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.480+0000 7fe919600640 1 -- 172.21.15.47:0/2363361389 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928010680 con 0x5615ea9813f0 2024-03-21T11:35:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.480+0000 7fe919600640 1 -- 172.21.15.47:0/2363361389 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe928010820 con 0x5615ea9813f0 2024-03-21T11:35:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.480+0000 7fe939163740 1 -- 172.21.15.47:0/2363361389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe928015070 con 0x5615ea9813f0 2024-03-21T11:35:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.481+0000 7fe919600640 1 -- 172.21.15.47:0/2363361389 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe928010a20 con 0x5615ea9813f0 2024-03-21T11:35:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.481+0000 7fe939163740 10 client.29585 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.481+0000 7fe939163740 1 -- 172.21.15.47:0/2363361389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5615ea6c7620 con 0x5615ea9813f0 2024-03-21T11:35:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.481+0000 7fe919600640 1 -- 172.21.15.47:0/2363361389 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe928015070 con 0x5615ea9813f0 2024-03-21T11:35:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.481+0000 7fe919600640 1 -- 172.21.15.47:0/2363361389 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe92801abb0 con 0x5615ea9813f0 2024-03-21T11:35:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.481+0000 7fe939163740 10 client.29585 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.481+0000 7fe939163740 10 client.29585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.481+0000 7fe939163740 20 client.29585 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.481+0000 7fe939163740 1 --2- 172.21.15.47:0/2363361389 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615ea997220 0x5615ea999610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.481+0000 7fe939163740 4 client.29585 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.481+0000 7fe939163740 1 -- 172.21.15.47:0/2363361389 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615ea8f3d70 con 0x5615ea997220 2024-03-21T11:35:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.482+0000 7fe92ea00640 1 --2- 172.21.15.47:0/2363361389 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615ea997220 0x5615ea999610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.482+0000 7fe92ea00640 1 --2- 172.21.15.47:0/2363361389 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615ea997220 0x5615ea999610 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae4b457de6545065 server_cookie=1a828856f5804afe in_seq=0 out_seq=0 2024-03-21T11:35:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.482+0000 7fe919600640 10 client.29585 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.484+0000 7fe919600640 1 -- 172.21.15.47:0/2363361389 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5615ea8f3d70 con 0x5615ea997220 2024-03-21T11:35:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.484+0000 7fe919600640 10 client.29585 handle_command_reply: tid=0 2024-03-21T11:35:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.484+0000 7fe939163740 1 client.29585 shutdown 2024-03-21T11:35:40.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.484+0000 7fe939163740 1 -- 172.21.15.47:0/2363361389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea9813f0 msgr2=0x5615ea98b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.484+0000 7fe939163740 1 --2- 172.21.15.47:0/2363361389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea9813f0 0x5615ea98b6a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe928011f60 tx=0x7fe928011d10 comp rx=0 tx=0).stop 2024-03-21T11:35:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.484+0000 7fe939163740 1 -- 172.21.15.47:0/2363361389 shutdown_connections 2024-03-21T11:35:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.484+0000 7fe939163740 1 --2- 172.21.15.47:0/2363361389 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615ea997220 0x5615ea999610 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.484+0000 7fe939163740 1 --2- 172.21.15.47:0/2363361389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea986f50 0x5615ea992bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.484+0000 7fe939163740 1 --2- 172.21.15.47:0/2363361389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea981de0 0x5615ea98bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.484+0000 7fe939163740 1 --2- 172.21.15.47:0/2363361389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea9813f0 0x5615ea98b6a0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.484+0000 7fe939163740 1 -- 172.21.15.47:0/2363361389 >> 172.21.15.47:0/2363361389 conn(0x5615ea97c960 msgr2=0x5615ea97bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.484+0000 7fe919600640 0 client.29585 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.485+0000 7fe939163740 1 -- 172.21.15.47:0/2363361389 shutdown_connections 2024-03-21T11:35:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.485+0000 7fe939163740 1 -- 172.21.15.47:0/2363361389 wait complete. 2024-03-21T11:35:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.485+0000 7fe939163740 20 client.29585 trim_cache size 0 max 16384 2024-03-21T11:35:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.504+0000 7fe939163740 1 Processor -- start 2024-03-21T11:35:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.504+0000 7fe939163740 1 -- start start 2024-03-21T11:35:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.504+0000 7fe939163740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea9813f0 0x5615ea98b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.504+0000 7fe939163740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea981de0 0x5615ea992b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.504+0000 7fe939163740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea986f50 0x5615ea98b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.504+0000 7fe939163740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9301ac240 con 0x5615ea9813f0 2024-03-21T11:35:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.504+0000 7fe939163740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9301adbc0 con 0x5615ea986f50 2024-03-21T11:35:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.504+0000 7fe939163740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9301ac8a0 con 0x5615ea981de0 2024-03-21T11:35:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.505+0000 7fe92ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea981de0 0x5615ea992b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.505+0000 7fe92ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea981de0 0x5615ea992b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41322/0 (socket says 172.21.15.47:41322) 2024-03-21T11:35:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.505+0000 7fe92ea00640 1 -- 172.21.15.47:0/3241431161 learned_addr learned my addr 172.21.15.47:0/3241431161 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.505+0000 7fe92fe00640 1 --2- 172.21.15.47:0/3241431161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea9813f0 0x5615ea98b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.505+0000 7fe92fe00640 1 --2- 172.21.15.47:0/3241431161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea986f50 0x5615ea98b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.505+0000 7fe92ea00640 1 -- 172.21.15.47:0/3241431161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea986f50 msgr2=0x5615ea98b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.505+0000 7fe92ea00640 1 --2- 172.21.15.47:0/3241431161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea986f50 0x5615ea98b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.505+0000 7fe92ea00640 1 -- 172.21.15.47:0/3241431161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea9813f0 msgr2=0x5615ea98b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.505+0000 7fe92ea00640 1 --2- 172.21.15.47:0/3241431161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea9813f0 0x5615ea98b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.505+0000 7fe92ea00640 1 -- 172.21.15.47:0/3241431161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615ea97af80 con 0x5615ea981de0 2024-03-21T11:35:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.505+0000 7fe92fe00640 1 --2- 172.21.15.47:0/3241431161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea986f50 0x5615ea98b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:35:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.505+0000 7fe92ea00640 1 --2- 172.21.15.47:0/3241431161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea981de0 0x5615ea992b40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x5615ea97d8d0 tx=0x7fe91c009730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31eb4104d04710f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.506+0000 7fe91be00640 1 -- 172.21.15.47:0/3241431161 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c004030 con 0x5615ea981de0 2024-03-21T11:35:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.506+0000 7fe91be00640 1 -- 172.21.15.47:0/3241431161 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe91c0041d0 con 0x5615ea981de0 2024-03-21T11:35:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.506+0000 7fe91be00640 1 -- 172.21.15.47:0/3241431161 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c012790 con 0x5615ea981de0 2024-03-21T11:35:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.506+0000 7fe939163740 1 -- 172.21.15.47:0/3241431161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea981de0 msgr2=0x5615ea992b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.506+0000 7fe939163740 1 --2- 172.21.15.47:0/3241431161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea981de0 0x5615ea992b40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x5615ea97d8d0 tx=0x7fe91c009730 comp rx=0 tx=0).stop 2024-03-21T11:35:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.507+0000 7fe939163740 1 -- 172.21.15.47:0/3241431161 shutdown_connections 2024-03-21T11:35:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.507+0000 7fe939163740 1 --2- 172.21.15.47:0/3241431161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea986f50 0x5615ea98b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.507+0000 7fe939163740 1 --2- 172.21.15.47:0/3241431161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea981de0 0x5615ea992b40 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.507+0000 7fe939163740 1 --2- 172.21.15.47:0/3241431161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea9813f0 0x5615ea98b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.507+0000 7fe939163740 1 -- 172.21.15.47:0/3241431161 >> 172.21.15.47:0/3241431161 conn(0x5615ea97c960 msgr2=0x5615ea97cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.507+0000 7fe939163740 1 -- 172.21.15.47:0/3241431161 shutdown_connections 2024-03-21T11:35:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.507+0000 7fe939163740 1 -- 172.21.15.47:0/3241431161 wait complete. 2024-03-21T11:35:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.507+0000 7fe939163740 1 -- start start 2024-03-21T11:35:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.507+0000 7fe939163740 1 Processor -- start 2024-03-21T11:35:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.508+0000 7fe939163740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea9813f0 0x5615ea98f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.508+0000 7fe939163740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea981de0 0x5615ea98fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.508+0000 7fe939163740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea986f50 0x5615ea9f0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.508+0000 7fe939163740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9301ad560 con 0x5615ea9813f0 2024-03-21T11:35:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.508+0000 7fe939163740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9301acf00 con 0x5615ea981de0 2024-03-21T11:35:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.508+0000 7fe939163740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9301aa400 con 0x5615ea986f50 2024-03-21T11:35:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.508+0000 7fe939163740 10 client.0 _ll_register_callbacks cb 0x7fe9200f7420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.508+0000 7fe92ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea981de0 0x5615ea98fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.508+0000 7fe92fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea9813f0 0x5615ea98f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.508+0000 7fe92ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea981de0 0x5615ea98fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49476/0 (socket says 172.21.15.47:49476) 2024-03-21T11:35:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.508+0000 7fe92ea00640 1 -- 172.21.15.47:0/433686711 learned_addr learned my addr 172.21.15.47:0/433686711 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.508+0000 7fe92fe00640 1 --2- 172.21.15.47:0/433686711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea986f50 0x5615ea9f0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.508+0000 7fe92ea00640 1 -- 172.21.15.47:0/433686711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea986f50 msgr2=0x5615ea9f0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.508+0000 7fe92ea00640 1 --2- 172.21.15.47:0/433686711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea986f50 0x5615ea9f0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.509+0000 7fe92ea00640 1 -- 172.21.15.47:0/433686711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea9813f0 msgr2=0x5615ea98f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.509+0000 7fe92ea00640 1 --2- 172.21.15.47:0/433686711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea9813f0 0x5615ea98f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.509+0000 7fe92ea00640 1 -- 172.21.15.47:0/433686711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615ea990510 con 0x5615ea981de0 2024-03-21T11:35:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.509+0000 7fe92ea00640 1 --2- 172.21.15.47:0/433686711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea981de0 0x5615ea98fea0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0058c0 tx=0x7fe91c009950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce3c0b3c571e9906 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.509+0000 7fe919600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.509+0000 7fe919600640 1 -- 172.21.15.47:0/433686711 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c005020 con 0x5615ea981de0 2024-03-21T11:35:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.509+0000 7fe939163740 1 -- 172.21.15.47:0/433686711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe91c0041d0 con 0x5615ea981de0 2024-03-21T11:35:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.509+0000 7fe919600640 1 -- 172.21.15.47:0/433686711 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe91c0051c0 con 0x5615ea981de0 2024-03-21T11:35:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.510+0000 7fe919600640 1 -- 172.21.15.47:0/433686711 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe91c037050 con 0x5615ea981de0 2024-03-21T11:35:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.510+0000 7fe939163740 10 client.29588 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.510+0000 7fe939163740 1 -- 172.21.15.47:0/433686711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5615ea9f20a0 con 0x5615ea981de0 2024-03-21T11:35:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.510+0000 7fe919600640 1 -- 172.21.15.47:0/433686711 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe91c0041d0 con 0x5615ea981de0 2024-03-21T11:35:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.510+0000 7fe919600640 1 -- 172.21.15.47:0/433686711 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe91c004900 con 0x5615ea981de0 2024-03-21T11:35:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.510+0000 7fe939163740 10 client.29588 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.510+0000 7fe939163740 10 client.29588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.510+0000 7fe939163740 20 client.29588 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.510+0000 7fe939163740 1 --2- 172.21.15.47:0/433686711 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615ea9f2990 0x5615ea9f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.510+0000 7fe939163740 4 client.29588 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:35:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.510+0000 7fe939163740 1 -- 172.21.15.47:0/433686711 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5615ea8ee0c0 con 0x5615ea9f2990 2024-03-21T11:35:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.511+0000 7fe92fe00640 1 --2- 172.21.15.47:0/433686711 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615ea9f2990 0x5615ea9f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.511+0000 7fe92fe00640 1 --2- 172.21.15.47:0/433686711 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615ea9f2990 0x5615ea9f4db0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=999e67dff6f40fde server_cookie=fdb857df2c449b7f in_seq=0 out_seq=0 2024-03-21T11:35:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.511+0000 7fe919600640 10 client.29588 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:40.572 INFO:tasks.workunit.client.0.smithi047.stdout: 1 935334 7.98 MB/sec execute 504 sec latency 49.936 ms 2024-03-21T11:35:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe919600640 1 -- 172.21.15.47:0/433686711 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5615ea8ee0c0 con 0x5615ea9f2990 2024-03-21T11:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe919600640 10 client.29588 handle_command_reply: tid=0 2024-03-21T11:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe939163740 1 client.29588 shutdown 2024-03-21T11:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe939163740 1 -- 172.21.15.47:0/433686711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea981de0 msgr2=0x5615ea98fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe939163740 1 --2- 172.21.15.47:0/433686711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea981de0 0x5615ea98fea0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0058c0 tx=0x7fe91c009950 comp rx=0 tx=0).stop 2024-03-21T11:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe939163740 1 -- 172.21.15.47:0/433686711 shutdown_connections 2024-03-21T11:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe939163740 1 --2- 172.21.15.47:0/433686711 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5615ea9f2990 0x5615ea9f4db0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe939163740 1 --2- 172.21.15.47:0/433686711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615ea986f50 0x5615ea9f0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe939163740 1 --2- 172.21.15.47:0/433686711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5615ea981de0 0x5615ea98fea0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe939163740 1 --2- 172.21.15.47:0/433686711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5615ea9813f0 0x5615ea98f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe919600640 0 client.29588 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe939163740 1 -- 172.21.15.47:0/433686711 >> 172.21.15.47:0/433686711 conn(0x5615ea97c960 msgr2=0x5615ea986410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe939163740 1 -- 172.21.15.47:0/433686711 shutdown_connections 2024-03-21T11:35:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe939163740 1 -- 172.21.15.47:0/433686711 wait complete. 2024-03-21T11:35:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.850+0000 7fe939163740 20 client.29588 trim_cache size 0 max 16384 2024-03-21T11:35:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:35:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:35:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:35:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.851+0000 7fe934e00640 1 -- 172.21.15.47:0/917798267 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9140a2f90 msgr2=0x7fe9140a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.851+0000 7fe934e00640 1 --2- 172.21.15.47:0/917798267 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9140a2f90 0x7fe9140a5450 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe9200ef360 tx=0x7fe9200f7550 comp rx=0 tx=0).stop 2024-03-21T11:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.851+0000 7fe934e00640 1 -- 172.21.15.47:0/917798267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe930101510 msgr2=0x7fe930199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.851+0000 7fe934e00640 1 --2- 172.21.15.47:0/917798267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe930101510 0x7fe930199ad0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe920000c00 tx=0x7fe92002c6c0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.851+0000 7fe934e00640 1 -- 172.21.15.47:0/917798267 shutdown_connections 2024-03-21T11:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.851+0000 7fe934e00640 1 --2- 172.21.15.47:0/917798267 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe9140a2f90 0x7fe9140a5450 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.851+0000 7fe934e00640 1 --2- 172.21.15.47:0/917798267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9301030c0 0x7fe9301a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.851+0000 7fe934e00640 1 --2- 172.21.15.47:0/917798267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe930102700 0x7fe93019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.851+0000 7fe934e00640 1 --2- 172.21.15.47:0/917798267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe930101510 0x7fe930199ad0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.851+0000 7fe934e00640 1 -- 172.21.15.47:0/917798267 >> 172.21.15.47:0/917798267 conn(0x7fe9300fca80 msgr2=0x7fe9300738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.852+0000 7fe934e00640 1 -- 172.21.15.47:0/917798267 shutdown_connections 2024-03-21T11:35:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:40.852+0000 7fe934e00640 1 -- 172.21.15.47:0/917798267 wait complete. 2024-03-21T11:35:40.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:40 smithi102 ceph-mon[36748]: pgmap v770: 129 pgs: 129 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 2.0 MiB/s wr, 305 op/s 2024-03-21T11:35:41.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:40 smithi047 ceph-mon[31351]: pgmap v770: 129 pgs: 129 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 2.0 MiB/s wr, 305 op/s 2024-03-21T11:35:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:40 smithi100 ceph-mon[36245]: pgmap v770: 129 pgs: 129 active+clean; 22 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 2.0 MiB/s wr, 305 op/s 2024-03-21T11:35:41.571 INFO:tasks.workunit.client.0.smithi047.stdout: 1 936826 7.98 MB/sec execute 505 sec latency 259.780 ms 2024-03-21T11:35:42.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:35:42.213 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:35:42.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.480+0000 7f5cc0200640 1 -- 172.21.15.47:0/178385373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb81086a0 msgr2=0x7f5cb810aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:42.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.480+0000 7f5cc0200640 1 --2- 172.21.15.47:0/178385373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb81086a0 0x7f5cb810aad0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009860 tx=0x7f5cac02bb70 comp rx=0 tx=0).stop 2024-03-21T11:35:42.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.481+0000 7f5cc0200640 1 -- 172.21.15.47:0/178385373 shutdown_connections 2024-03-21T11:35:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.482+0000 7f5cc0200640 1 --2- 172.21.15.47:0/178385373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cb810d9c0 0x7f5cb810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.482+0000 7f5cc0200640 1 --2- 172.21.15.47:0/178385373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5cb810b010 0x7f5cb810d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.482+0000 7f5cc0200640 1 --2- 172.21.15.47:0/178385373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb81086a0 0x7f5cb810aad0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.482+0000 7f5cc0200640 1 -- 172.21.15.47:0/178385373 >> 172.21.15.47:0/178385373 conn(0x7f5cb806e500 msgr2=0x7f5cb8070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.482+0000 7f5cc0200640 1 -- 172.21.15.47:0/178385373 shutdown_connections 2024-03-21T11:35:42.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.482+0000 7f5cc0200640 1 -- 172.21.15.47:0/178385373 wait complete. 2024-03-21T11:35:42.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.486+0000 7f5cc0200640 1 Processor -- start 2024-03-21T11:35:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.487+0000 7f5cc0200640 1 -- start start 2024-03-21T11:35:42.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.487+0000 7f5cc0200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5cb81086a0 0x7f5cb81a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.487+0000 7f5cc0200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cb810b010 0x7f5cb81a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.487+0000 7f5cc0200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb810d9c0 0x7f5cb81adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.487+0000 7f5cc0200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb811c150 con 0x7f5cb810b010 2024-03-21T11:35:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.487+0000 7f5cc0200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb811bfd0 con 0x7f5cb810d9c0 2024-03-21T11:35:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.487+0000 7f5cc0200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb811c2d0 con 0x7f5cb81086a0 2024-03-21T11:35:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.488+0000 7f5cbd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5cb81086a0 0x7f5cb81a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.488+0000 7f5cbd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5cb81086a0 0x7f5cb81a6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41344/0 (socket says 172.21.15.47:41344) 2024-03-21T11:35:42.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.488+0000 7f5cbd800640 1 -- 172.21.15.47:0/3617421093 learned_addr learned my addr 172.21.15.47:0/3617421093 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.488+0000 7f5cbd800640 1 -- 172.21.15.47:0/3617421093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb810d9c0 msgr2=0x7f5cb81adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.488+0000 7f5cbd800640 1 --2- 172.21.15.47:0/3617421093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb810d9c0 0x7f5cb81adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.488+0000 7f5cbd800640 1 -- 172.21.15.47:0/3617421093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cb810b010 msgr2=0x7f5cb81a6cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:35:42.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.489+0000 7f5cbd800640 1 --2- 172.21.15.47:0/3617421093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cb810b010 0x7f5cb81a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.489+0000 7f5cbd800640 1 -- 172.21.15.47:0/3617421093 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb81ae340 con 0x7f5cb81086a0 2024-03-21T11:35:42.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.491+0000 7f5cbd800640 1 --2- 172.21.15.47:0/3617421093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5cb81086a0 0x7f5cb81a6770 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0349e0 tx=0x7f5cac0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:42.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.493+0000 7f5cb6000640 1 -- 172.21.15.47:0/3617421093 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac041020 con 0x7f5cb81086a0 2024-03-21T11:35:42.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.494+0000 7f5cb6000640 1 -- 172.21.15.47:0/3617421093 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cac0347b0 con 0x7f5cb81086a0 2024-03-21T11:35:42.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.494+0000 7f5cb6000640 1 -- 172.21.15.47:0/3617421093 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac039040 con 0x7f5cb81086a0 2024-03-21T11:35:42.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.494+0000 7f5cc0200640 1 -- 172.21.15.47:0/3617421093 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb81ae5d0 con 0x7f5cb81086a0 2024-03-21T11:35:42.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.494+0000 7f5cc0200640 1 -- 172.21.15.47:0/3617421093 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb81aeb10 con 0x7f5cb81086a0 2024-03-21T11:35:42.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.502+0000 7f5cc2fcf740 1 Processor -- start 2024-03-21T11:35:42.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.502+0000 7f5cc2fcf740 1 -- start start 2024-03-21T11:35:42.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.503+0000 7f5cb6000640 1 -- 172.21.15.47:0/3617421093 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5cac044020 con 0x7f5cb81086a0 2024-03-21T11:35:42.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.503+0000 7f5cc2fcf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141db32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.504+0000 7f5cbe200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141db32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.504+0000 7f5cbe200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141db32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41352/0 (socket says 172.21.15.47:41352) 2024-03-21T11:35:42.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.504+0000 7f5cc2fcf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db3870 0x55b141db8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.506+0000 7f5cc2fcf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db89e0 0x55b141dbae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.506+0000 7f5cc2fcf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b141d18250 con 0x55b141db3870 2024-03-21T11:35:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.506+0000 7f5cc2fcf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b141cc0a30 con 0x55b141db89e0 2024-03-21T11:35:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.506+0000 7f5cc2fcf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b141d142f0 con 0x55b141db2e80 2024-03-21T11:35:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.506+0000 7f5cbd800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db3870 0x55b141db8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.506+0000 7f5cbe200640 1 -- 172.21.15.47:0/328594403 learned_addr learned my addr 172.21.15.47:0/328594403 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.507+0000 7f5cb6000640 1 --2- 172.21.15.47:0/3617421093 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5ca00a2fa0 0x7f5ca00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.507+0000 7f5cbce00640 1 --2- 172.21.15.47:0/328594403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db89e0 0x55b141dbae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.507+0000 7f5cbe200640 1 -- 172.21.15.47:0/328594403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db89e0 msgr2=0x55b141dbae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.507+0000 7f5cbe200640 1 --2- 172.21.15.47:0/328594403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db89e0 0x55b141dbae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.507+0000 7f5cbe200640 1 -- 172.21.15.47:0/328594403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db3870 msgr2=0x55b141db8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.507+0000 7f5cbe200640 1 --2- 172.21.15.47:0/328594403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db3870 0x55b141db8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.507+0000 7f5cbe200640 1 -- 172.21.15.47:0/328594403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b141cab690 con 0x55b141db2e80 2024-03-21T11:35:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.507+0000 7f5cb6000640 1 -- 172.21.15.47:0/3617421093 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5cac0f13b0 con 0x7f5cb81086a0 2024-03-21T11:35:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.508+0000 7f5cbce00640 1 --2- 172.21.15.47:0/328594403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db89e0 0x55b141dbae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:35:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.508+0000 7f5cbe200640 1 --2- 172.21.15.47:0/3617421093 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5ca00a2fa0 0x7f5ca00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.509+0000 7f5cbe200640 1 --2- 172.21.15.47:0/328594403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141db32a0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0007cc0 tx=0x7f5cb000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7ddb6767a7921ba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.510+0000 7f5caaa00640 1 -- 172.21.15.47:0/328594403 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0022070 con 0x55b141db2e80 2024-03-21T11:35:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.511+0000 7f5caaa00640 1 -- 172.21.15.47:0/328594403 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cb0014050 con 0x55b141db2e80 2024-03-21T11:35:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.511+0000 7f5caaa00640 1 -- 172.21.15.47:0/328594403 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0022070 con 0x55b141db2e80 2024-03-21T11:35:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.511+0000 7f5cbe200640 1 --2- 172.21.15.47:0/3617421093 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5ca00a2fa0 0x7f5ca00a5460 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0007fd0 tx=0x7f5cb001e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.512+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/328594403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 msgr2=0x55b141db32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.512+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/328594403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141db32a0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0007cc0 tx=0x7f5cb000c6a0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.513+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/328594403 shutdown_connections 2024-03-21T11:35:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.513+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/328594403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db89e0 0x55b141dbae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.513+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/328594403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db3870 0x55b141db8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.513+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/328594403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141db32a0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.513+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/328594403 >> 172.21.15.47:0/328594403 conn(0x55b141d7b200 msgr2=0x55b141dae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.513+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/328594403 shutdown_connections 2024-03-21T11:35:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.514+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/328594403 wait complete. 2024-03-21T11:35:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.514+0000 7f5cc2fcf740 1 -- start start 2024-03-21T11:35:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.515+0000 7f5cc2fcf740 1 Processor -- start 2024-03-21T11:35:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.516+0000 7f5cc2fcf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141dbce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.516+0000 7f5cc2fcf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db3870 0x55b141dbd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.516+0000 7f5cc2fcf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db89e0 0x55b141dc43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.516+0000 7f5cc2fcf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b141dbb990 con 0x55b141db89e0 2024-03-21T11:35:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.516+0000 7f5cc2fcf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b141cd3e80 con 0x55b141db3870 2024-03-21T11:35:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.516+0000 7f5cc2fcf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b141dbbb10 con 0x55b141db2e80 2024-03-21T11:35:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.516+0000 7f5cc2fcf740 10 client.0 _ll_register_callbacks cb 0x55b141ce6130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.517+0000 7f5cbd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141dbce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.517+0000 7f5cbd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141dbce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41368/0 (socket says 172.21.15.47:41368) 2024-03-21T11:35:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.517+0000 7f5cbd800640 1 -- 172.21.15.47:0/1117777860 learned_addr learned my addr 172.21.15.47:0/1117777860 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.517+0000 7f5cbe200640 1 --2- 172.21.15.47:0/1117777860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db89e0 0x55b141dc43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.517+0000 7f5cbce00640 1 --2- 172.21.15.47:0/1117777860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db3870 0x55b141dbd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.518+0000 7f5cbd800640 1 -- 172.21.15.47:0/1117777860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db3870 msgr2=0x55b141dbd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.518+0000 7f5cbd800640 1 --2- 172.21.15.47:0/1117777860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db3870 0x55b141dbd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.518+0000 7f5cbd800640 1 -- 172.21.15.47:0/1117777860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db89e0 msgr2=0x55b141dc43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.518+0000 7f5cbd800640 1 --2- 172.21.15.47:0/1117777860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db89e0 0x55b141dc43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.518+0000 7f5cbd800640 1 -- 172.21.15.47:0/1117777860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b141af67d0 con 0x55b141db2e80 2024-03-21T11:35:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.518+0000 7f5cbe200640 1 --2- 172.21.15.47:0/1117777860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db89e0 0x55b141dc43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.519+0000 7f5cbd800640 1 --2- 172.21.15.47:0/1117777860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141dbce30 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009830 tx=0x7f5cac009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=349d3053252b2a42 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.519+0000 7f5c9f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.520+0000 7f5c9f400640 1 -- 172.21.15.47:0/1117777860 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0bd290 con 0x55b141db2e80 2024-03-21T11:35:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.520+0000 7f5c9f400640 1 -- 172.21.15.47:0/1117777860 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5cac0a80b0 con 0x55b141db2e80 2024-03-21T11:35:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.520+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/1117777860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5cb0014050 con 0x55b141db2e80 2024-03-21T11:35:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.521+0000 7f5c9f400640 1 -- 172.21.15.47:0/1117777860 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cac0044e0 con 0x55b141db2e80 2024-03-21T11:35:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.521+0000 7f5cc2fcf740 10 client.29629 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.521+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/1117777860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b141a2c3f0 con 0x55b141db2e80 2024-03-21T11:35:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.522+0000 7f5c9f400640 1 -- 172.21.15.47:0/1117777860 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cb0014050 con 0x55b141db2e80 2024-03-21T11:35:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.522+0000 7f5c9f400640 1 -- 172.21.15.47:0/1117777860 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5cac0a6ce0 con 0x55b141db2e80 2024-03-21T11:35:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.522+0000 7f5cc2fcf740 10 client.29629 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.522+0000 7f5cc2fcf740 10 client.29629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.522+0000 7f5cc2fcf740 20 client.29629 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.522+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/1117777860 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b141dc8ba0 0x55b141dcaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.522+0000 7f5cc2fcf740 4 client.29629 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.522+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/1117777860 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b141d25790 con 0x55b141dc8ba0 2024-03-21T11:35:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.522+0000 7f5cbce00640 1 --2- 172.21.15.47:0/1117777860 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b141dc8ba0 0x55b141dcaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.524+0000 7f5cbce00640 1 --2- 172.21.15.47:0/1117777860 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b141dc8ba0 0x55b141dcaff0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=673e54153d0cbf37 server_cookie=20b9b334fc624f8a in_seq=0 out_seq=0 2024-03-21T11:35:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.524+0000 7f5c9f400640 10 client.29629 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.525+0000 7f5c9f400640 1 -- 172.21.15.47:0/1117777860 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b141d25790 con 0x55b141dc8ba0 2024-03-21T11:35:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.525+0000 7f5c9f400640 10 client.29629 handle_command_reply: tid=0 2024-03-21T11:35:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.525+0000 7f5cc2fcf740 1 client.29629 shutdown 2024-03-21T11:35:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.525+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/1117777860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 msgr2=0x55b141dbce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.525+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/1117777860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141dbce30 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009830 tx=0x7f5cac009490 comp rx=0 tx=0).stop 2024-03-21T11:35:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.526+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/1117777860 shutdown_connections 2024-03-21T11:35:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.526+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/1117777860 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b141dc8ba0 0x55b141dcaff0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.526+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/1117777860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db89e0 0x55b141dc43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.526+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/1117777860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db3870 0x55b141dbd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.526+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/1117777860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141dbce30 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.526+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/1117777860 >> 172.21.15.47:0/1117777860 conn(0x55b141d7b200 msgr2=0x55b141dada90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.526+0000 7f5c9f400640 0 client.29629 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:42.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.526+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/1117777860 shutdown_connections 2024-03-21T11:35:42.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.526+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/1117777860 wait complete. 2024-03-21T11:35:42.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.526+0000 7f5cc2fcf740 20 client.29629 trim_cache size 0 max 16384 2024-03-21T11:35:42.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.547+0000 7f5cc2fcf740 1 Processor -- start 2024-03-21T11:35:42.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.548+0000 7f5cc2fcf740 1 -- start start 2024-03-21T11:35:42.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.548+0000 7f5cc2fcf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141db1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.548+0000 7f5cc2fcf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db3870 0x55b141dc4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.548+0000 7f5cc2fcf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db89e0 0x55b141db1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.548+0000 7f5cc2fcf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb81b8e90 con 0x55b141db89e0 2024-03-21T11:35:42.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.548+0000 7f5cc2fcf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb81ba810 con 0x55b141db3870 2024-03-21T11:35:42.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.548+0000 7f5cc2fcf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb81b94f0 con 0x55b141db2e80 2024-03-21T11:35:42.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.548+0000 7f5cbce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db3870 0x55b141dc4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.548+0000 7f5cbe200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db89e0 0x55b141db1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.548+0000 7f5cbce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db3870 0x55b141dc4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49528/0 (socket says 172.21.15.47:49528) 2024-03-21T11:35:42.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.548+0000 7f5cbce00640 1 -- 172.21.15.47:0/2678614009 learned_addr learned my addr 172.21.15.47:0/2678614009 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:42.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.548+0000 7f5cbd800640 1 --2- 172.21.15.47:0/2678614009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141db1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.549+0000 7f5cbce00640 1 -- 172.21.15.47:0/2678614009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 msgr2=0x55b141db1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:42.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.549+0000 7f5cbce00640 1 --2- 172.21.15.47:0/2678614009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141db1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.549+0000 7f5cbce00640 1 -- 172.21.15.47:0/2678614009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db89e0 msgr2=0x55b141db1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:42.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.549+0000 7f5cbce00640 1 --2- 172.21.15.47:0/2678614009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db89e0 0x55b141db1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.549+0000 7f5cbce00640 1 -- 172.21.15.47:0/2678614009 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1419ec0f0 con 0x55b141db3870 2024-03-21T11:35:42.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.549+0000 7f5cbe200640 1 --2- 172.21.15.47:0/2678614009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db89e0 0x55b141db1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:42.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.549+0000 7f5cbce00640 1 --2- 172.21.15.47:0/2678614009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db3870 0x55b141dc4370 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x55b141daf740 tx=0x7f5ca40199d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d75df23d5424c47 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:42.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.550+0000 7f5caa000640 1 -- 172.21.15.47:0/2678614009 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4005220 con 0x55b141db3870 2024-03-21T11:35:42.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.550+0000 7f5caa000640 1 -- 172.21.15.47:0/2678614009 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ca4004c80 con 0x55b141db3870 2024-03-21T11:35:42.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.550+0000 7f5caa000640 1 -- 172.21.15.47:0/2678614009 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4010960 con 0x55b141db3870 2024-03-21T11:35:42.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.550+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/2678614009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db3870 msgr2=0x55b141dc4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:42.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.550+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/2678614009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db3870 0x55b141dc4370 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x55b141daf740 tx=0x7f5ca40199d0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.551+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/2678614009 shutdown_connections 2024-03-21T11:35:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.551+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/2678614009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db89e0 0x55b141db1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.551+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/2678614009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db3870 0x55b141dc4370 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.551+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/2678614009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141db1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.551+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/2678614009 >> 172.21.15.47:0/2678614009 conn(0x55b141d7b200 msgr2=0x55b141db4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.551+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/2678614009 shutdown_connections 2024-03-21T11:35:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.551+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/2678614009 wait complete. 2024-03-21T11:35:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.551+0000 7f5cc2fcf740 1 -- start start 2024-03-21T11:35:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.551+0000 7f5cc2fcf740 1 Processor -- start 2024-03-21T11:35:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cc2fcf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141dc0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cc2fcf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db3870 0x55b141dc0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cc2fcf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db89e0 0x55b141dc3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cc2fcf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb81ba1b0 con 0x55b141db3870 2024-03-21T11:35:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cc2fcf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb81b9b50 con 0x55b141db89e0 2024-03-21T11:35:42.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cc2fcf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb81b7050 con 0x55b141db2e80 2024-03-21T11:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cc2fcf740 10 client.0 _ll_register_callbacks cb 0x7f5cac0b9330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cbd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141dc0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cbce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db3870 0x55b141dc0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cbd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141dc0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41382/0 (socket says 172.21.15.47:41382) 2024-03-21T11:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cbd800640 1 -- 172.21.15.47:0/3370915670 learned_addr learned my addr 172.21.15.47:0/3370915670 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cbe200640 1 --2- 172.21.15.47:0/3370915670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db89e0 0x55b141dc3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cbd800640 1 -- 172.21.15.47:0/3370915670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db89e0 msgr2=0x55b141dc3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cbd800640 1 --2- 172.21.15.47:0/3370915670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db89e0 0x55b141dc3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cbd800640 1 -- 172.21.15.47:0/3370915670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db3870 msgr2=0x55b141dc0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cbd800640 1 --2- 172.21.15.47:0/3370915670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db3870 0x55b141dc0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cbd800640 1 -- 172.21.15.47:0/3370915670 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b141dc3b00 con 0x55b141db2e80 2024-03-21T11:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.552+0000 7f5cbe200640 1 --2- 172.21.15.47:0/3370915670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db89e0 0x55b141dc3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.553+0000 7f5cbd800640 1 --2- 172.21.15.47:0/3370915670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141dc0000 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f5cac02c180 tx=0x7f5cac02bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c7ca449041a3d91 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.553+0000 7f5c9f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.553+0000 7f5c9f400640 1 -- 172.21.15.47:0/3370915670 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0bd0b0 con 0x55b141db2e80 2024-03-21T11:35:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.553+0000 7f5c9f400640 1 -- 172.21.15.47:0/3370915670 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5cac0bcba0 con 0x55b141db2e80 2024-03-21T11:35:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.553+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/3370915670 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ca4004c80 con 0x55b141db2e80 2024-03-21T11:35:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.554+0000 7f5c9f400640 1 -- 172.21.15.47:0/3370915670 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cac0d5030 con 0x55b141db2e80 2024-03-21T11:35:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.554+0000 7f5cc2fcf740 10 client.29635 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.554+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/3370915670 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b141e35aa0 con 0x55b141db2e80 2024-03-21T11:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.554+0000 7f5c9f400640 1 -- 172.21.15.47:0/3370915670 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ca4004c80 con 0x55b141db2e80 2024-03-21T11:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.554+0000 7f5c9f400640 1 -- 172.21.15.47:0/3370915670 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5cac0a6ce0 con 0x55b141db2e80 2024-03-21T11:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.554+0000 7f5cc2fcf740 10 client.29635 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.554+0000 7f5cc2fcf740 10 client.29635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.554+0000 7f5cc2fcf740 20 client.29635 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.554+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/3370915670 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b141e36160 0x55b141e38580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.554+0000 7f5cc2fcf740 4 client.29635 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.554+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/3370915670 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b141dcb380 con 0x55b141e36160 2024-03-21T11:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.555+0000 7f5cbce00640 1 --2- 172.21.15.47:0/3370915670 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b141e36160 0x55b141e38580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.555+0000 7f5cbce00640 1 --2- 172.21.15.47:0/3370915670 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b141e36160 0x55b141e38580 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e369c62e5d008eb3 server_cookie=5c99670d441559b in_seq=0 out_seq=0 2024-03-21T11:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.555+0000 7f5c9f400640 10 client.29635 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.556+0000 7f5c9f400640 1 -- 172.21.15.47:0/3370915670 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b141dcb380 con 0x55b141e36160 2024-03-21T11:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.556+0000 7f5c9f400640 10 client.29635 handle_command_reply: tid=0 2024-03-21T11:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.556+0000 7f5cc2fcf740 1 client.29635 shutdown 2024-03-21T11:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.556+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/3370915670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 msgr2=0x55b141dc0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.556+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/3370915670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141dc0000 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f5cac02c180 tx=0x7f5cac02bb70 comp rx=0 tx=0).stop 2024-03-21T11:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.556+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/3370915670 shutdown_connections 2024-03-21T11:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.556+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/3370915670 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b141e36160 0x55b141e38580 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.556+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/3370915670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b141db89e0 0x55b141dc3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.556+0000 7f5c9f400640 0 client.29635 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.556+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/3370915670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b141db3870 0x55b141dc0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.556+0000 7f5cc2fcf740 1 --2- 172.21.15.47:0/3370915670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b141db2e80 0x55b141dc0000 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.556+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/3370915670 >> 172.21.15.47:0/3370915670 conn(0x55b141d7b200 msgr2=0x55b141db4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.557+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/3370915670 shutdown_connections 2024-03-21T11:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.557+0000 7f5cc2fcf740 1 -- 172.21.15.47:0/3370915670 wait complete. 2024-03-21T11:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.557+0000 7f5cc2fcf740 20 client.29635 trim_cache size 0 max 16384 2024-03-21T11:35:42.561 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:35:42.561 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:35:42.562 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:35:42.562 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.558+0000 7f5cc0200640 1 -- 172.21.15.47:0/3617421093 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5ca00a2fa0 msgr2=0x7f5ca00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.558+0000 7f5cc0200640 1 --2- 172.21.15.47:0/3617421093 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5ca00a2fa0 0x7f5ca00a5460 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0007fd0 tx=0x7f5cb001e000 comp rx=0 tx=0).stop 2024-03-21T11:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.558+0000 7f5cc0200640 1 -- 172.21.15.47:0/3617421093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5cb81086a0 msgr2=0x7f5cb81a6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.558+0000 7f5cc0200640 1 --2- 172.21.15.47:0/3617421093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5cb81086a0 0x7f5cb81a6770 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0349e0 tx=0x7f5cac0042f0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.558+0000 7f5cc0200640 1 -- 172.21.15.47:0/3617421093 shutdown_connections 2024-03-21T11:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.558+0000 7f5cc0200640 1 --2- 172.21.15.47:0/3617421093 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5ca00a2fa0 0x7f5ca00a5460 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.558+0000 7f5cc0200640 1 --2- 172.21.15.47:0/3617421093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb810d9c0 0x7f5cb81adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.558+0000 7f5cc0200640 1 --2- 172.21.15.47:0/3617421093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5cb810b010 0x7f5cb81a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.558+0000 7f5cc0200640 1 --2- 172.21.15.47:0/3617421093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5cb81086a0 0x7f5cb81a6770 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.558+0000 7f5cc0200640 1 -- 172.21.15.47:0/3617421093 >> 172.21.15.47:0/3617421093 conn(0x7f5cb806e500 msgr2=0x7f5cb810c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.558+0000 7f5cc0200640 1 -- 172.21.15.47:0/3617421093 shutdown_connections 2024-03-21T11:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.559+0000 7f5cc0200640 1 -- 172.21.15.47:0/3617421093 wait complete. 2024-03-21T11:35:42.571 INFO:tasks.workunit.client.0.smithi047.stdout: 1 938122 7.98 MB/sec execute 506 sec latency 90.891 ms 2024-03-21T11:35:42.580 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:35:42.581 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:35:42.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.804+0000 7f26e1400640 1 -- 172.21.15.47:0/2903768993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26dc0fffb0 msgr2=0x7f26dc1003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.804+0000 7f26e1400640 1 --2- 172.21.15.47:0/2903768993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26dc0fffb0 0x7f26dc1003d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009860 tx=0x7f26cc02bc20 comp rx=0 tx=0).stop 2024-03-21T11:35:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.804+0000 7f26e1400640 1 -- 172.21.15.47:0/2903768993 shutdown_connections 2024-03-21T11:35:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.804+0000 7f26e1400640 1 --2- 172.21.15.47:0/2903768993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26dc1012b0 0x7f26dc101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.804+0000 7f26e1400640 1 --2- 172.21.15.47:0/2903768993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc100910 0x7f26dc100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.804+0000 7f26e1400640 1 --2- 172.21.15.47:0/2903768993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26dc0fffb0 0x7f26dc1003d0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.804+0000 7f26e1400640 1 -- 172.21.15.47:0/2903768993 >> 172.21.15.47:0/2903768993 conn(0x7f26dc0fbb30 msgr2=0x7f26dc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.805+0000 7f26e1400640 1 -- 172.21.15.47:0/2903768993 shutdown_connections 2024-03-21T11:35:42.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.805+0000 7f26e1400640 1 -- 172.21.15.47:0/2903768993 wait complete. 2024-03-21T11:35:42.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.806+0000 7f26e1400640 1 Processor -- start 2024-03-21T11:35:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.806+0000 7f26e1400640 1 -- start start 2024-03-21T11:35:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.806+0000 7f26e1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26dc0fffb0 0x7f26dc199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.806+0000 7f26e1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc100910 0x7f26dc19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.806+0000 7f26e1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26dc1012b0 0x7f26dc1a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.806+0000 7f26e1400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc10f510 con 0x7f26dc0fffb0 2024-03-21T11:35:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.806+0000 7f26e1400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc10f390 con 0x7f26dc100910 2024-03-21T11:35:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.806+0000 7f26e1400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc10f690 con 0x7f26dc1012b0 2024-03-21T11:35:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.806+0000 7f26db200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26dc1012b0 0x7f26dc1a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.806+0000 7f26d9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc100910 0x7f26dc19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.807+0000 7f26db200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26dc1012b0 0x7f26dc1a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41408/0 (socket says 172.21.15.47:41408) 2024-03-21T11:35:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.807+0000 7f26d9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc100910 0x7f26dc19a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49552/0 (socket says 172.21.15.47:49552) 2024-03-21T11:35:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.807+0000 7f26db200640 1 -- 172.21.15.47:0/2580485837 learned_addr learned my addr 172.21.15.47:0/2580485837 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:42.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.807+0000 7f26da800640 1 --2- 172.21.15.47:0/2580485837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26dc0fffb0 0x7f26dc199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.807+0000 7f26db200640 1 -- 172.21.15.47:0/2580485837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc100910 msgr2=0x7f26dc19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.807+0000 7f26db200640 1 --2- 172.21.15.47:0/2580485837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc100910 0x7f26dc19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.807+0000 7f26db200640 1 -- 172.21.15.47:0/2580485837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26dc0fffb0 msgr2=0x7f26dc199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.807+0000 7f26db200640 1 --2- 172.21.15.47:0/2580485837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26dc0fffb0 0x7f26dc199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.807+0000 7f26db200640 1 -- 172.21.15.47:0/2580485837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26dc1a17d0 con 0x7f26dc1012b0 2024-03-21T11:35:42.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.807+0000 7f26d9e00640 1 --2- 172.21.15.47:0/2580485837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc100910 0x7f26dc19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.807+0000 7f26db200640 1 --2- 172.21.15.47:0/2580485837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26dc1012b0 0x7f26dc1a1160 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f26d400eaa0 tx=0x7f26d400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.807+0000 7f26d3400640 1 -- 172.21.15.47:0/2580485837 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d400cdb0 con 0x7f26dc1012b0 2024-03-21T11:35:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.807+0000 7f26d3400640 1 -- 172.21.15.47:0/2580485837 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26d4005020 con 0x7f26dc1012b0 2024-03-21T11:35:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.807+0000 7f26d3400640 1 -- 172.21.15.47:0/2580485837 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d40116b0 con 0x7f26dc1012b0 2024-03-21T11:35:42.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.808+0000 7f26e1400640 1 -- 172.21.15.47:0/2580485837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26dc1a1ac0 con 0x7f26dc1012b0 2024-03-21T11:35:42.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.808+0000 7f26e1400640 1 -- 172.21.15.47:0/2580485837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc1a1fc0 con 0x7f26dc1012b0 2024-03-21T11:35:42.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.809+0000 7f26e1400640 1 -- 172.21.15.47:0/2580485837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26dc10aee0 con 0x7f26dc1012b0 2024-03-21T11:35:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.812+0000 7f26d3400640 1 -- 172.21.15.47:0/2580485837 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f26d4005560 con 0x7f26dc1012b0 2024-03-21T11:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.813+0000 7f26d3400640 1 --2- 172.21.15.47:0/2580485837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26c00a2fa0 0x7f26c00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.814+0000 7f26d3400640 1 -- 172.21.15.47:0/2580485837 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f26d40cf8d0 con 0x7f26dc1012b0 2024-03-21T11:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.814+0000 7f26da800640 1 --2- 172.21.15.47:0/2580485837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26c00a2fa0 0x7f26c00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.814+0000 7f26da800640 1 --2- 172.21.15.47:0/2580485837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26c00a2fa0 0x7f26c00a5460 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009830 tx=0x7f26cc034b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:42.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:42.820+0000 7f26d3400640 1 -- 172.21.15.47:0/2580485837 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f26d4016030 con 0x7f26dc1012b0 2024-03-21T11:35:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:42 smithi047 ceph-mon[31351]: pgmap v771: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-03-21T11:35:43.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.110+0000 7f26e1400640 1 -- 172.21.15.47:0/2580485837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26dc02d070 con 0x7f26dc1012b0 2024-03-21T11:35:43.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.111+0000 7f26d3400640 1 -- 172.21.15.47:0/2580485837 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f26d409bd00 con 0x7f26dc1012b0 2024-03-21T11:35:43.112 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:35:43.112 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:35:43.112 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:35:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.114+0000 7f26e1400640 1 -- 172.21.15.47:0/2580485837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26c00a2fa0 msgr2=0x7f26c00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.114+0000 7f26e1400640 1 --2- 172.21.15.47:0/2580485837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26c00a2fa0 0x7f26c00a5460 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009830 tx=0x7f26cc034b50 comp rx=0 tx=0).stop 2024-03-21T11:35:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.114+0000 7f26e1400640 1 -- 172.21.15.47:0/2580485837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26dc1012b0 msgr2=0x7f26dc1a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.114+0000 7f26e1400640 1 --2- 172.21.15.47:0/2580485837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26dc1012b0 0x7f26dc1a1160 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f26d400eaa0 tx=0x7f26d400ef60 comp rx=0 tx=0).stop 2024-03-21T11:35:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.114+0000 7f26e1400640 1 -- 172.21.15.47:0/2580485837 shutdown_connections 2024-03-21T11:35:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.114+0000 7f26e1400640 1 --2- 172.21.15.47:0/2580485837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26c00a2fa0 0x7f26c00a5460 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.114+0000 7f26e1400640 1 --2- 172.21.15.47:0/2580485837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26dc1012b0 0x7f26dc1a1160 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.114+0000 7f26e1400640 1 --2- 172.21.15.47:0/2580485837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26dc100910 0x7f26dc19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.114+0000 7f26e1400640 1 --2- 172.21.15.47:0/2580485837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26dc0fffb0 0x7f26dc199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.114+0000 7f26e1400640 1 -- 172.21.15.47:0/2580485837 >> 172.21.15.47:0/2580485837 conn(0x7f26dc0fbb30 msgr2=0x7f26dc0fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:43.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.114+0000 7f26e1400640 1 -- 172.21.15.47:0/2580485837 shutdown_connections 2024-03-21T11:35:43.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.114+0000 7f26e1400640 1 -- 172.21.15.47:0/2580485837 wait complete. 2024-03-21T11:35:43.136 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:35:43.136 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:35:43.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:42 smithi100 ceph-mon[36245]: pgmap v771: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-03-21T11:35:43.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:42 smithi102 ceph-mon[36748]: pgmap v771: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-03-21T11:35:43.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.365+0000 7f33bfe00640 1 -- 172.21.15.47:0/3458768060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0101680 msgr2=0x7f33c0101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.365+0000 7f33bfe00640 1 --2- 172.21.15.47:0/3458768060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0101680 0x7f33c0101b00 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009900 tx=0x7f33ac02bc30 comp rx=0 tx=0).stop 2024-03-21T11:35:43.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.365+0000 7f33bfe00640 1 -- 172.21.15.47:0/3458768060 shutdown_connections 2024-03-21T11:35:43.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.365+0000 7f33bfe00640 1 --2- 172.21.15.47:0/3458768060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33c0102040 0x7f33c01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.365+0000 7f33bfe00640 1 --2- 172.21.15.47:0/3458768060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0101680 0x7f33c0101b00 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.365+0000 7f33bfe00640 1 --2- 172.21.15.47:0/3458768060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33c0100490 0x7f33c01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.365+0000 7f33bfe00640 1 -- 172.21.15.47:0/3458768060 >> 172.21.15.47:0/3458768060 conn(0x7f33c00fba20 msgr2=0x7f33c00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:43.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.365+0000 7f33bfe00640 1 -- 172.21.15.47:0/3458768060 shutdown_connections 2024-03-21T11:35:43.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.365+0000 7f33bfe00640 1 -- 172.21.15.47:0/3458768060 wait complete. 2024-03-21T11:35:43.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.366+0000 7f33bfe00640 1 Processor -- start 2024-03-21T11:35:43.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.366+0000 7f33bfe00640 1 -- start start 2024-03-21T11:35:43.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.367+0000 7f33bfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33c0100490 0x7f33c0199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.367+0000 7f33bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0101680 0x7f33c0199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.367+0000 7f33bfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33c0102040 0x7f33c01a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.367+0000 7f33bfe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c010c170 con 0x7f33c0100490 2024-03-21T11:35:43.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.367+0000 7f33bfe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c010bff0 con 0x7f33c0102040 2024-03-21T11:35:43.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.367+0000 7f33bfe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c010c2f0 con 0x7f33c0101680 2024-03-21T11:35:43.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.367+0000 7f33be000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0101680 0x7f33c0199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.367+0000 7f33be000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0101680 0x7f33c0199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41432/0 (socket says 172.21.15.47:41432) 2024-03-21T11:35:43.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.367+0000 7f33bea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33c0100490 0x7f33c0199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.367+0000 7f33be000640 1 -- 172.21.15.47:0/1539545283 learned_addr learned my addr 172.21.15.47:0/1539545283 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:43.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.367+0000 7f33bf400640 1 --2- 172.21.15.47:0/1539545283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33c0102040 0x7f33c01a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.368+0000 7f33be000640 1 -- 172.21.15.47:0/1539545283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33c0102040 msgr2=0x7f33c01a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.368+0000 7f33be000640 1 --2- 172.21.15.47:0/1539545283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33c0102040 0x7f33c01a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.368+0000 7f33be000640 1 -- 172.21.15.47:0/1539545283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33c0100490 msgr2=0x7f33c0199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.368+0000 7f33be000640 1 --2- 172.21.15.47:0/1539545283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33c0100490 0x7f33c0199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.368+0000 7f33be000640 1 -- 172.21.15.47:0/1539545283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c01a1650 con 0x7f33c0101680 2024-03-21T11:35:43.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.368+0000 7f33bf400640 1 --2- 172.21.15.47:0/1539545283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33c0102040 0x7f33c01a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.368+0000 7f33be000640 1 --2- 172.21.15.47:0/1539545283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0101680 0x7f33c0199f70 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f33ac005ab0 tx=0x7f33ac02c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.369+0000 7f33b7400640 1 -- 172.21.15.47:0/1539545283 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac043020 con 0x7f33c0101680 2024-03-21T11:35:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.369+0000 7f33b7400640 1 -- 172.21.15.47:0/1539545283 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33ac0054e0 con 0x7f33c0101680 2024-03-21T11:35:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.369+0000 7f33bfe00640 1 -- 172.21.15.47:0/1539545283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c01a18e0 con 0x7f33c0101680 2024-03-21T11:35:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.369+0000 7f33b7400640 1 -- 172.21.15.47:0/1539545283 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac003ce0 con 0x7f33c0101680 2024-03-21T11:35:43.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.369+0000 7f33bfe00640 1 -- 172.21.15.47:0/1539545283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c01a1e80 con 0x7f33c0101680 2024-03-21T11:35:43.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.373+0000 7f33c887e740 1 Processor -- start 2024-03-21T11:35:43.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.374+0000 7f33b7400640 1 -- 172.21.15.47:0/1539545283 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f33ac046020 con 0x7f33c0101680 2024-03-21T11:35:43.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.374+0000 7f33c887e740 1 -- start start 2024-03-21T11:35:43.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.374+0000 7f33c887e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d44e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.374+0000 7f33c887e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d45bc0 0x559b16d46040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.374+0000 7f33c887e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d46580 0x559b16d4ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.374+0000 7f33c887e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b16ca92c0 con 0x559b16d46580 2024-03-21T11:35:43.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.374+0000 7f33c887e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b16c51c90 con 0x559b16d449f0 2024-03-21T11:35:43.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.374+0000 7f33c887e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b16ca5360 con 0x559b16d45bc0 2024-03-21T11:35:43.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.374+0000 7f33bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d44e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.374+0000 7f33bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d44e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49576/0 (socket says 172.21.15.47:49576) 2024-03-21T11:35:43.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.374+0000 7f33bf400640 1 -- 172.21.15.47:0/1472888448 learned_addr learned my addr 172.21.15.47:0/1472888448 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:43.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.374+0000 7f33bea00640 1 --2- 172.21.15.47:0/1472888448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d45bc0 0x559b16d46040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.375+0000 7f33bf400640 1 -- 172.21.15.47:0/1472888448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d45bc0 msgr2=0x559b16d46040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.375+0000 7f33be000640 1 --2- 172.21.15.47:0/1472888448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d46580 0x559b16d4ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.375+0000 7f33bf400640 1 --2- 172.21.15.47:0/1472888448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d45bc0 0x559b16d46040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.375+0000 7f33bf400640 1 -- 172.21.15.47:0/1472888448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d46580 msgr2=0x559b16d4ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.375+0000 7f33bf400640 1 --2- 172.21.15.47:0/1472888448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d46580 0x559b16d4ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.375+0000 7f33bf400640 1 -- 172.21.15.47:0/1472888448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b16c76080 con 0x559b16d449f0 2024-03-21T11:35:43.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.375+0000 7f33be000640 1 --2- 172.21.15.47:0/1472888448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d46580 0x559b16d4ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:35:43.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.375+0000 7f33bf400640 1 --2- 172.21.15.47:0/1472888448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d44e10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f33b80117e0 tx=0x7f33b8011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd16474baf0b119 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.376+0000 7f33abe00640 1 -- 172.21.15.47:0/1472888448 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b8011310 con 0x559b16d449f0 2024-03-21T11:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.376+0000 7f33abe00640 1 -- 172.21.15.47:0/1472888448 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33b8003e50 con 0x559b16d449f0 2024-03-21T11:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.376+0000 7f33abe00640 1 -- 172.21.15.47:0/1472888448 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b80045b0 con 0x559b16d449f0 2024-03-21T11:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.376+0000 7f33b7400640 1 --2- 172.21.15.47:0/1539545283 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33a40a2fa0 0x7f33a40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.377+0000 7f33b7400640 1 -- 172.21.15.47:0/1539545283 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f33ac0f1820 con 0x7f33c0101680 2024-03-21T11:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.377+0000 7f33c887e740 1 -- 172.21.15.47:0/1472888448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 msgr2=0x559b16d44e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.377+0000 7f33c887e740 1 --2- 172.21.15.47:0/1472888448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d44e10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f33b80117e0 tx=0x7f33b8011ca0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.377+0000 7f33c887e740 1 -- 172.21.15.47:0/1472888448 shutdown_connections 2024-03-21T11:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.377+0000 7f33c887e740 1 --2- 172.21.15.47:0/1472888448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d46580 0x559b16d4ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.377+0000 7f33c887e740 1 --2- 172.21.15.47:0/1472888448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d45bc0 0x559b16d46040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.377+0000 7f33c887e740 1 --2- 172.21.15.47:0/1472888448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d44e10 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.377+0000 7f33c887e740 1 -- 172.21.15.47:0/1472888448 >> 172.21.15.47:0/1472888448 conn(0x559b16d3fde0 msgr2=0x559b16d401e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.377+0000 7f33bea00640 1 --2- 172.21.15.47:0/1539545283 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33a40a2fa0 0x7f33a40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.377+0000 7f33c887e740 1 -- 172.21.15.47:0/1472888448 shutdown_connections 2024-03-21T11:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.377+0000 7f33c887e740 1 -- 172.21.15.47:0/1472888448 wait complete. 2024-03-21T11:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.378+0000 7f33c887e740 1 -- start start 2024-03-21T11:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.378+0000 7f33c887e740 1 Processor -- start 2024-03-21T11:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.378+0000 7f33bea00640 1 --2- 172.21.15.47:0/1539545283 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33a40a2fa0 0x7f33a40a5460 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x559b16d40d50 tx=0x7f33b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.379+0000 7f33c887e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d4ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.379+0000 7f33c887e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d45bc0 0x559b16d4f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.379+0000 7f33c887e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d46580 0x559b16d56200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.379+0000 7f33c887e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b16d4d590 con 0x559b16d46580 2024-03-21T11:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.379+0000 7f33c887e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b16c650d0 con 0x559b16d449f0 2024-03-21T11:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.379+0000 7f33c887e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b16d4d710 con 0x559b16d45bc0 2024-03-21T11:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.379+0000 7f33c887e740 10 client.0 _ll_register_callbacks cb 0x559b16c69e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.379+0000 7f33be000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d45bc0 0x559b16d4f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.379+0000 7f33be000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d45bc0 0x559b16d4f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41442/0 (socket says 172.21.15.47:41442) 2024-03-21T11:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.379+0000 7f33be000640 1 -- 172.21.15.47:0/3811247713 learned_addr learned my addr 172.21.15.47:0/3811247713 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.380+0000 7f33bf400640 1 --2- 172.21.15.47:0/3811247713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d46580 0x559b16d56200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.380+0000 7f33bea00640 1 --2- 172.21.15.47:0/3811247713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d4ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.380+0000 7f33be000640 1 -- 172.21.15.47:0/3811247713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 msgr2=0x559b16d4ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.380+0000 7f33be000640 1 --2- 172.21.15.47:0/3811247713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d4ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.380+0000 7f33be000640 1 -- 172.21.15.47:0/3811247713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d46580 msgr2=0x559b16d56200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.380+0000 7f33be000640 1 --2- 172.21.15.47:0/3811247713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d46580 0x559b16d56200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.380+0000 7f33be000640 1 -- 172.21.15.47:0/3811247713 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b169bd3f0 con 0x559b16d45bc0 2024-03-21T11:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.380+0000 7f33bea00640 1 --2- 172.21.15.47:0/3811247713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d4ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.380+0000 7f33be000640 1 --2- 172.21.15.47:0/3811247713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d45bc0 0x559b16d4f1c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0b9fc0 tx=0x7f33ac0bd4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55253543a7044df2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.380+0000 7f33a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.381+0000 7f33a8c00640 1 -- 172.21.15.47:0/3811247713 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac005690 con 0x559b16d45bc0 2024-03-21T11:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.381+0000 7f33a8c00640 1 -- 172.21.15.47:0/3811247713 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f33ac0ef040 con 0x559b16d45bc0 2024-03-21T11:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.381+0000 7f33c887e740 1 -- 172.21.15.47:0/3811247713 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33b8003e50 con 0x559b16d45bc0 2024-03-21T11:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.381+0000 7f33a8c00640 1 -- 172.21.15.47:0/3811247713 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33ac0be260 con 0x559b16d45bc0 2024-03-21T11:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.381+0000 7f33c887e740 10 client.29653 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.382+0000 7f33c887e740 1 -- 172.21.15.47:0/3811247713 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559b16a877d0 con 0x559b16d45bc0 2024-03-21T11:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.382+0000 7f33a8c00640 1 -- 172.21.15.47:0/3811247713 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33b8003e50 con 0x559b16d45bc0 2024-03-21T11:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.382+0000 7f33a8c00640 1 -- 172.21.15.47:0/3811247713 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f33ac0d6070 con 0x559b16d45bc0 2024-03-21T11:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.383+0000 7f33c887e740 10 client.29653 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.383+0000 7f33c887e740 10 client.29653 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.383+0000 7f33c887e740 20 client.29653 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.383+0000 7f33c887e740 1 --2- 172.21.15.47:0/3811247713 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559b16d5a660 0x559b16d5ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.383+0000 7f33c887e740 4 client.29653 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.383+0000 7f33c887e740 1 -- 172.21.15.47:0/3811247713 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b16cb6800 con 0x559b16d5a660 2024-03-21T11:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.383+0000 7f33bea00640 1 --2- 172.21.15.47:0/3811247713 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559b16d5a660 0x559b16d5ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.384+0000 7f33a8c00640 10 client.29653 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.384+0000 7f33bea00640 1 --2- 172.21.15.47:0/3811247713 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559b16d5a660 0x559b16d5ca90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b1a0cf28a2805e99 server_cookie=816e7e587499ddbe in_seq=0 out_seq=0 2024-03-21T11:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.385+0000 7f33a8c00640 1 -- 172.21.15.47:0/3811247713 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559b16cb6800 con 0x559b16d5a660 2024-03-21T11:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.385+0000 7f33a8c00640 10 client.29653 handle_command_reply: tid=0 2024-03-21T11:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.385+0000 7f33c887e740 1 client.29653 shutdown 2024-03-21T11:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.385+0000 7f33c887e740 1 -- 172.21.15.47:0/3811247713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d45bc0 msgr2=0x559b16d4f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.385+0000 7f33c887e740 1 --2- 172.21.15.47:0/3811247713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d45bc0 0x559b16d4f1c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0b9fc0 tx=0x7f33ac0bd4c0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.385+0000 7f33c887e740 1 -- 172.21.15.47:0/3811247713 shutdown_connections 2024-03-21T11:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.385+0000 7f33c887e740 1 --2- 172.21.15.47:0/3811247713 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559b16d5a660 0x559b16d5ca90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.385+0000 7f33c887e740 1 --2- 172.21.15.47:0/3811247713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d46580 0x559b16d56200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.385+0000 7f33c887e740 1 --2- 172.21.15.47:0/3811247713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d45bc0 0x559b16d4f1c0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.385+0000 7f33c887e740 1 --2- 172.21.15.47:0/3811247713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d4ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.386+0000 7f33c887e740 1 -- 172.21.15.47:0/3811247713 >> 172.21.15.47:0/3811247713 conn(0x559b16d3fde0 msgr2=0x559b16d3f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.386+0000 7f33a8c00640 0 client.29653 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.386+0000 7f33c887e740 1 -- 172.21.15.47:0/3811247713 shutdown_connections 2024-03-21T11:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.386+0000 7f33c887e740 1 -- 172.21.15.47:0/3811247713 wait complete. 2024-03-21T11:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.386+0000 7f33c887e740 20 client.29653 trim_cache size 0 max 16384 2024-03-21T11:35:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.408+0000 7f33c887e740 1 Processor -- start 2024-03-21T11:35:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.408+0000 7f33c887e740 1 -- start start 2024-03-21T11:35:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.409+0000 7f33c887e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d449f0 0x559b16d3aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.409+0000 7f33c887e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d46580 0x559b16d561c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.409+0000 7f33c887e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d43750 0x559b16d61350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.409+0000 7f33c887e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c01ac0e0 con 0x559b16d449f0 2024-03-21T11:35:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.409+0000 7f33c887e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c01ada60 con 0x559b16d43750 2024-03-21T11:35:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.409+0000 7f33c887e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c01ac740 con 0x559b16d46580 2024-03-21T11:35:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.409+0000 7f33be000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d46580 0x559b16d561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.409+0000 7f33bea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d449f0 0x559b16d3aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.409+0000 7f33be000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d46580 0x559b16d561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41458/0 (socket says 172.21.15.47:41458) 2024-03-21T11:35:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.409+0000 7f33be000640 1 -- 172.21.15.47:0/3892032141 learned_addr learned my addr 172.21.15.47:0/3892032141 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.409+0000 7f33bf400640 1 --2- 172.21.15.47:0/3892032141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d43750 0x559b16d61350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.410+0000 7f33be000640 1 -- 172.21.15.47:0/3892032141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d43750 msgr2=0x559b16d61350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.410+0000 7f33be000640 1 --2- 172.21.15.47:0/3892032141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d43750 0x559b16d61350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.410+0000 7f33be000640 1 -- 172.21.15.47:0/3892032141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d449f0 msgr2=0x559b16d3aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.410+0000 7f33be000640 1 --2- 172.21.15.47:0/3892032141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d449f0 0x559b16d3aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.410+0000 7f33be000640 1 -- 172.21.15.47:0/3892032141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b16d4f510 con 0x559b16d46580 2024-03-21T11:35:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.410+0000 7f33bea00640 1 --2- 172.21.15.47:0/3892032141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d449f0 0x559b16d3aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.410+0000 7f33be000640 1 --2- 172.21.15.47:0/3892032141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d46580 0x559b16d561c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0bbe10 tx=0x7f33ac02c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0b4803351cae05e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.410+0000 7f33ab400640 1 -- 172.21.15.47:0/3892032141 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac005690 con 0x559b16d46580 2024-03-21T11:35:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.411+0000 7f33ab400640 1 -- 172.21.15.47:0/3892032141 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33ac0bd2b0 con 0x559b16d46580 2024-03-21T11:35:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.411+0000 7f33ab400640 1 -- 172.21.15.47:0/3892032141 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0c6570 con 0x559b16d46580 2024-03-21T11:35:43.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.412+0000 7f33c887e740 1 -- 172.21.15.47:0/3892032141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d46580 msgr2=0x559b16d561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.412+0000 7f33c887e740 1 --2- 172.21.15.47:0/3892032141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d46580 0x559b16d561c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0bbe10 tx=0x7f33ac02c880 comp rx=0 tx=0).stop 2024-03-21T11:35:43.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.412+0000 7f33c887e740 1 -- 172.21.15.47:0/3892032141 shutdown_connections 2024-03-21T11:35:43.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.412+0000 7f33c887e740 1 --2- 172.21.15.47:0/3892032141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d43750 0x559b16d61350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.412+0000 7f33c887e740 1 --2- 172.21.15.47:0/3892032141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d46580 0x559b16d561c0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.412+0000 7f33c887e740 1 --2- 172.21.15.47:0/3892032141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d449f0 0x559b16d3aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.412+0000 7f33c887e740 1 -- 172.21.15.47:0/3892032141 >> 172.21.15.47:0/3892032141 conn(0x559b16d3fde0 msgr2=0x559b16d43040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:43.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.412+0000 7f33c887e740 1 -- 172.21.15.47:0/3892032141 shutdown_connections 2024-03-21T11:35:43.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.412+0000 7f33c887e740 1 -- 172.21.15.47:0/3892032141 wait complete. 2024-03-21T11:35:43.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.412+0000 7f33c887e740 1 -- start start 2024-03-21T11:35:43.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.413+0000 7f33c887e740 1 Processor -- start 2024-03-21T11:35:43.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.413+0000 7f33c887e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d43750 0x559b16d51fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.413+0000 7f33c887e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d52500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.413+0000 7f33c887e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d46580 0x559b16d55520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.413+0000 7f33c887e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c01ad400 con 0x559b16d43750 2024-03-21T11:35:43.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.413+0000 7f33c887e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c01acda0 con 0x559b16d449f0 2024-03-21T11:35:43.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.413+0000 7f33c887e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c01aa2a0 con 0x559b16d46580 2024-03-21T11:35:43.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.413+0000 7f33c887e740 10 client.0 _ll_register_callbacks cb 0x7f33ac005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:43.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.413+0000 7f33be000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d52500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.413+0000 7f33bf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d46580 0x559b16d55520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.413+0000 7f33be000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d52500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49594/0 (socket says 172.21.15.47:49594) 2024-03-21T11:35:43.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.413+0000 7f33be000640 1 -- 172.21.15.47:0/1822971849 learned_addr learned my addr 172.21.15.47:0/1822971849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:43.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.414+0000 7f33bea00640 1 --2- 172.21.15.47:0/1822971849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d43750 0x559b16d51fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.414+0000 7f33be000640 1 -- 172.21.15.47:0/1822971849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d46580 msgr2=0x559b16d55520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.414+0000 7f33be000640 1 --2- 172.21.15.47:0/1822971849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d46580 0x559b16d55520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.414+0000 7f33be000640 1 -- 172.21.15.47:0/1822971849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d43750 msgr2=0x559b16d51fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.414+0000 7f33be000640 1 --2- 172.21.15.47:0/1822971849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d43750 0x559b16d51fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.414+0000 7f33be000640 1 -- 172.21.15.47:0/1822971849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b16d55b10 con 0x559b16d449f0 2024-03-21T11:35:43.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.414+0000 7f33bea00640 1 --2- 172.21.15.47:0/1822971849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d43750 0x559b16d51fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:43.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.414+0000 7f33be000640 1 --2- 172.21.15.47:0/1822971849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d52500 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f33ac02c4e0 tx=0x7f33ac0bd4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2bacdb221454628 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:43.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.414+0000 7f33a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:43.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.414+0000 7f33a8c00640 1 -- 172.21.15.47:0/1822971849 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac004c80 con 0x559b16d449f0 2024-03-21T11:35:43.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.414+0000 7f33a8c00640 1 -- 172.21.15.47:0/1822971849 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f33ac0b98b0 con 0x559b16d449f0 2024-03-21T11:35:43.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.414+0000 7f33c887e740 1 -- 172.21.15.47:0/1822971849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33ac0bd2b0 con 0x559b16d449f0 2024-03-21T11:35:43.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.415+0000 7f33a8c00640 1 -- 172.21.15.47:0/1822971849 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33ac005690 con 0x559b16d449f0 2024-03-21T11:35:43.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.415+0000 7f33c887e740 10 client.29621 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:43.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.415+0000 7f33c887e740 1 -- 172.21.15.47:0/1822971849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559b16d55b10 con 0x559b16d449f0 2024-03-21T11:35:43.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.415+0000 7f33a8c00640 1 -- 172.21.15.47:0/1822971849 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33ac0bd2b0 con 0x559b16d449f0 2024-03-21T11:35:43.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.415+0000 7f33a8c00640 1 -- 172.21.15.47:0/1822971849 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f33ac0d6040 con 0x559b16d449f0 2024-03-21T11:35:43.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.415+0000 7f33c887e740 10 client.29621 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:43.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.415+0000 7f33c887e740 10 client.29621 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:35:43.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.415+0000 7f33c887e740 20 client.29621 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:43.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.415+0000 7f33c887e740 1 --2- 172.21.15.47:0/1822971849 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559b16dbf300 0x559b16dc1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.415+0000 7f33c887e740 4 client.29621 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:35:43.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.415+0000 7f33c887e740 1 -- 172.21.15.47:0/1822971849 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559b16cb0b50 con 0x559b16dbf300 2024-03-21T11:35:43.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.415+0000 7f33bea00640 1 --2- 172.21.15.47:0/1822971849 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559b16dbf300 0x559b16dc1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.416+0000 7f33bea00640 1 --2- 172.21.15.47:0/1822971849 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559b16dbf300 0x559b16dc1720 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e33a81631f22cad5 server_cookie=38c3af140b28954e in_seq=0 out_seq=0 2024-03-21T11:35:43.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.416+0000 7f33a8c00640 10 client.29621 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:35:43.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.416+0000 7f33a8c00640 1 -- 172.21.15.47:0/1822971849 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559b16cb0b50 con 0x559b16dbf300 2024-03-21T11:35:43.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.416+0000 7f33a8c00640 10 client.29621 handle_command_reply: tid=0 2024-03-21T11:35:43.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.416+0000 7f33c887e740 1 client.29621 shutdown 2024-03-21T11:35:43.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.416+0000 7f33c887e740 1 -- 172.21.15.47:0/1822971849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 msgr2=0x559b16d52500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.416+0000 7f33c887e740 1 --2- 172.21.15.47:0/1822971849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d52500 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f33ac02c4e0 tx=0x7f33ac0bd4c0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.416+0000 7f33c887e740 1 -- 172.21.15.47:0/1822971849 shutdown_connections 2024-03-21T11:35:43.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.416+0000 7f33c887e740 1 --2- 172.21.15.47:0/1822971849 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559b16dbf300 0x559b16dc1720 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.416+0000 7f33c887e740 1 --2- 172.21.15.47:0/1822971849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b16d46580 0x559b16d55520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.416+0000 7f33c887e740 1 --2- 172.21.15.47:0/1822971849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b16d449f0 0x559b16d52500 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.416+0000 7f33a8c00640 0 client.29621 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:35:43.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.416+0000 7f33c887e740 1 --2- 172.21.15.47:0/1822971849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b16d43750 0x559b16d51fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.417+0000 7f33c887e740 1 -- 172.21.15.47:0/1822971849 >> 172.21.15.47:0/1822971849 conn(0x559b16d3fde0 msgr2=0x559b16d48df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:43.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.417+0000 7f33c887e740 1 -- 172.21.15.47:0/1822971849 shutdown_connections 2024-03-21T11:35:43.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.417+0000 7f33c887e740 1 -- 172.21.15.47:0/1822971849 wait complete. 2024-03-21T11:35:43.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.417+0000 7f33c887e740 20 client.29621 trim_cache size 0 max 16384 2024-03-21T11:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:35:43.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.418+0000 7f33bfe00640 1 -- 172.21.15.47:0/1539545283 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33a40a2fa0 msgr2=0x7f33a40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.418+0000 7f33bfe00640 1 --2- 172.21.15.47:0/1539545283 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33a40a2fa0 0x7f33a40a5460 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x559b16d40d50 tx=0x7f33b0005f90 comp rx=0 tx=0).stop 2024-03-21T11:35:43.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.418+0000 7f33bfe00640 1 -- 172.21.15.47:0/1539545283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0101680 msgr2=0x7f33c0199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.418+0000 7f33bfe00640 1 --2- 172.21.15.47:0/1539545283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0101680 0x7f33c0199f70 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f33ac005ab0 tx=0x7f33ac02c6c0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.418+0000 7f33bfe00640 1 -- 172.21.15.47:0/1539545283 shutdown_connections 2024-03-21T11:35:43.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.418+0000 7f33bfe00640 1 --2- 172.21.15.47:0/1539545283 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33a40a2fa0 0x7f33a40a5460 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.418+0000 7f33bfe00640 1 --2- 172.21.15.47:0/1539545283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33c0102040 0x7f33c01a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.418+0000 7f33bfe00640 1 --2- 172.21.15.47:0/1539545283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0101680 0x7f33c0199f70 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.418+0000 7f33bfe00640 1 --2- 172.21.15.47:0/1539545283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33c0100490 0x7f33c0199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.418+0000 7f33bfe00640 1 -- 172.21.15.47:0/1539545283 >> 172.21.15.47:0/1539545283 conn(0x7f33c00fba20 msgr2=0x7f33c00fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:43.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.418+0000 7f33bfe00640 1 -- 172.21.15.47:0/1539545283 shutdown_connections 2024-03-21T11:35:43.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.418+0000 7f33bfe00640 1 -- 172.21.15.47:0/1539545283 wait complete. 2024-03-21T11:35:43.438 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:35:43.438 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:35:43.571 INFO:tasks.workunit.client.0.smithi047.stdout: 1 938981 7.97 MB/sec execute 507 sec latency 35.789 ms 2024-03-21T11:35:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.698+0000 7f1c9ca00640 1 -- 172.21.15.47:0/173159209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c98101670 msgr2=0x7f1c98101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.698+0000 7f1c9ca00640 1 --2- 172.21.15.47:0/173159209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c98101670 0x7f1c98101af0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009900 tx=0x7f1c8802bc60 comp rx=0 tx=0).stop 2024-03-21T11:35:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.699+0000 7f1c9ca00640 1 -- 172.21.15.47:0/173159209 shutdown_connections 2024-03-21T11:35:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.699+0000 7f1c9ca00640 1 --2- 172.21.15.47:0/173159209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c98102030 0x7f1c981088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.699+0000 7f1c9ca00640 1 --2- 172.21.15.47:0/173159209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c98101670 0x7f1c98101af0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.699+0000 7f1c9ca00640 1 --2- 172.21.15.47:0/173159209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c98100480 0x7f1c981008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.699+0000 7f1c9ca00640 1 -- 172.21.15.47:0/173159209 >> 172.21.15.47:0/173159209 conn(0x7f1c980fba10 msgr2=0x7f1c980fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.700+0000 7f1c9ca00640 1 -- 172.21.15.47:0/173159209 shutdown_connections 2024-03-21T11:35:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.700+0000 7f1c9ca00640 1 -- 172.21.15.47:0/173159209 wait complete. 2024-03-21T11:35:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.701+0000 7f1c9ca00640 1 Processor -- start 2024-03-21T11:35:43.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.702+0000 7f1c9ca00640 1 -- start start 2024-03-21T11:35:43.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.703+0000 7f1c9ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c98100480 0x7f1c98195210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.703+0000 7f1c9ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c98101670 0x7f1c98195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.703+0000 7f1c9ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c98102030 0x7f1c98191d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.703+0000 7f1c9ca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9810c1c0 con 0x7f1c98101670 2024-03-21T11:35:43.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.703+0000 7f1c9ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9810c040 con 0x7f1c98100480 2024-03-21T11:35:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.703+0000 7f1c9ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9810c340 con 0x7f1c98102030 2024-03-21T11:35:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.703+0000 7f1c97200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c98102030 0x7f1c98191d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.703+0000 7f1c96800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c98100480 0x7f1c98195210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.703+0000 7f1c97200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c98102030 0x7f1c98191d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41486/0 (socket says 172.21.15.47:41486) 2024-03-21T11:35:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.703+0000 7f1c97200640 1 -- 172.21.15.47:0/181704848 learned_addr learned my addr 172.21.15.47:0/181704848 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.703+0000 7f1c96800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c98100480 0x7f1c98195210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49612/0 (socket says 172.21.15.47:49612) 2024-03-21T11:35:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.704+0000 7f1c96800640 1 -- 172.21.15.47:0/181704848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c98102030 msgr2=0x7f1c98191d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.704+0000 7f1c96800640 1 --2- 172.21.15.47:0/181704848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c98102030 0x7f1c98191d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.704+0000 7f1c96800640 1 -- 172.21.15.47:0/181704848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c98101670 msgr2=0x7f1c98195750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.704+0000 7f1c96800640 1 --2- 172.21.15.47:0/181704848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c98101670 0x7f1c98195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.704+0000 7f1c96800640 1 -- 172.21.15.47:0/181704848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c98192640 con 0x7f1c98100480 2024-03-21T11:35:43.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.704+0000 7f1c96800640 1 --2- 172.21.15.47:0/181704848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c98100480 0x7f1c98195210 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000b3c0 tx=0x7f1c8000b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:43.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.705+0000 7f1c8f400640 1 -- 172.21.15.47:0/181704848 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c80011830 con 0x7f1c98100480 2024-03-21T11:35:43.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.705+0000 7f1c8f400640 1 -- 172.21.15.47:0/181704848 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c80016070 con 0x7f1c98100480 2024-03-21T11:35:43.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.705+0000 7f1c8f400640 1 -- 172.21.15.47:0/181704848 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8000f2e0 con 0x7f1c98100480 2024-03-21T11:35:43.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.705+0000 7f1c9ca00640 1 -- 172.21.15.47:0/181704848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c98199700 con 0x7f1c98100480 2024-03-21T11:35:43.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.706+0000 7f1c9ca00640 1 -- 172.21.15.47:0/181704848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c98199ba0 con 0x7f1c98100480 2024-03-21T11:35:43.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.711+0000 7f1ca021c740 1 Processor -- start 2024-03-21T11:35:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.711+0000 7f1c8f400640 1 -- 172.21.15.47:0/181704848 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1c800119d0 con 0x7f1c98100480 2024-03-21T11:35:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.712+0000 7f1ca021c740 1 -- start start 2024-03-21T11:35:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.712+0000 7f1ca021c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b0769f0 0x55ca7b076e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.712+0000 7f1ca021c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b077bc0 0x55ca7b078040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.712+0000 7f1ca021c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b078580 0x55ca7b07ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.712+0000 7f1ca021c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7afdb2c0 con 0x55ca7b077bc0 2024-03-21T11:35:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.712+0000 7f1ca021c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7af83c90 con 0x55ca7b0769f0 2024-03-21T11:35:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.713+0000 7f1ca021c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7afd7360 con 0x55ca7b078580 2024-03-21T11:35:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.713+0000 7f1c97200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b0769f0 0x55ca7b076e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.714+0000 7f1c97200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b0769f0 0x55ca7b076e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49620/0 (socket says 172.21.15.47:49620) 2024-03-21T11:35:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.714+0000 7f1c97200640 1 -- 172.21.15.47:0/725062337 learned_addr learned my addr 172.21.15.47:0/725062337 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.714+0000 7f1c95e00640 1 --2- 172.21.15.47:0/725062337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b078580 0x55ca7b07ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.714+0000 7f1c8f400640 1 --2- 172.21.15.47:0/181704848 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1c7c0a2fa0 0x7f1c7c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.714+0000 7f1c97200640 1 -- 172.21.15.47:0/725062337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b078580 msgr2=0x55ca7b07ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.714+0000 7f1c97200640 1 --2- 172.21.15.47:0/725062337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b078580 0x55ca7b07ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.714+0000 7f1c97200640 1 -- 172.21.15.47:0/725062337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b077bc0 msgr2=0x55ca7b078040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:35:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.714+0000 7f1c8f400640 1 -- 172.21.15.47:0/181704848 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1c800ced10 con 0x7f1c98100480 2024-03-21T11:35:43.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.715+0000 7f1c96800640 1 --2- 172.21.15.47:0/725062337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b077bc0 0x55ca7b078040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.716+0000 7f1c97200640 1 --2- 172.21.15.47:0/725062337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b077bc0 0x55ca7b078040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.716+0000 7f1c97200640 1 -- 172.21.15.47:0/725062337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca7afa8080 con 0x55ca7b0769f0 2024-03-21T11:35:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.716+0000 7f1c97200640 1 --2- 172.21.15.47:0/181704848 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1c7c0a2fa0 0x7f1c7c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.717+0000 7f1c97200640 1 --2- 172.21.15.47:0/725062337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b0769f0 0x55ca7b076e10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1c90011570 tx=0x7f1c90011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ea947fafcc6add8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.718+0000 7f1c96800640 1 --2- 172.21.15.47:0/725062337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b077bc0 0x55ca7b078040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:35:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.718+0000 7f1c87400640 1 -- 172.21.15.47:0/725062337 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000fb00 con 0x55ca7b0769f0 2024-03-21T11:35:43.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.718+0000 7f1c87400640 1 -- 172.21.15.47:0/725062337 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c9000fd20 con 0x55ca7b0769f0 2024-03-21T11:35:43.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.718+0000 7f1c87400640 1 -- 172.21.15.47:0/725062337 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c900188f0 con 0x55ca7b0769f0 2024-03-21T11:35:43.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.719+0000 7f1ca021c740 1 -- 172.21.15.47:0/725062337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b0769f0 msgr2=0x55ca7b076e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.719+0000 7f1ca021c740 1 --2- 172.21.15.47:0/725062337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b0769f0 0x55ca7b076e10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1c90011570 tx=0x7f1c90011a30 comp rx=0 tx=0).stop 2024-03-21T11:35:43.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.719+0000 7f1ca021c740 1 -- 172.21.15.47:0/725062337 shutdown_connections 2024-03-21T11:35:43.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.719+0000 7f1ca021c740 1 --2- 172.21.15.47:0/725062337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b078580 0x55ca7b07ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.719+0000 7f1ca021c740 1 --2- 172.21.15.47:0/725062337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b077bc0 0x55ca7b078040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.719+0000 7f1ca021c740 1 --2- 172.21.15.47:0/725062337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b0769f0 0x55ca7b076e10 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.719+0000 7f1c97200640 1 --2- 172.21.15.47:0/181704848 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1c7c0a2fa0 0x7f1c7c0a5460 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1c90020000 tx=0x7f1c90018ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:43.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.719+0000 7f1ca021c740 1 -- 172.21.15.47:0/725062337 >> 172.21.15.47:0/725062337 conn(0x55ca7b071de0 msgr2=0x55ca7b0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:43.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.719+0000 7f1ca021c740 1 -- 172.21.15.47:0/725062337 shutdown_connections 2024-03-21T11:35:43.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.720+0000 7f1ca021c740 1 -- 172.21.15.47:0/725062337 wait complete. 2024-03-21T11:35:43.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.720+0000 7f1ca021c740 1 -- start start 2024-03-21T11:35:43.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.721+0000 7f1ca021c740 1 Processor -- start 2024-03-21T11:35:43.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.722+0000 7f1ca021c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b0769f0 0x55ca7b091b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.722+0000 7f1ca021c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b077bc0 0x55ca7b0920c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.722+0000 7f1ca021c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b078580 0x55ca7b099100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.722+0000 7f1ca021c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7b0905a0 con 0x55ca7b0769f0 2024-03-21T11:35:43.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.722+0000 7f1ca021c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7af970d0 con 0x55ca7b077bc0 2024-03-21T11:35:43.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.722+0000 7f1ca021c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7b090720 con 0x55ca7b078580 2024-03-21T11:35:43.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.722+0000 7f1ca021c740 10 client.0 _ll_register_callbacks cb 0x55ca7af9be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:43.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.722+0000 7f1c95e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b077bc0 0x55ca7b0920c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.722+0000 7f1c96800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b0769f0 0x55ca7b091b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.722+0000 7f1c96800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b0769f0 0x55ca7b091b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49718/0 (socket says 172.21.15.47:49718) 2024-03-21T11:35:43.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.723+0000 7f1c96800640 1 -- 172.21.15.47:0/2634397293 learned_addr learned my addr 172.21.15.47:0/2634397293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:43.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.723+0000 7f1c97200640 1 --2- 172.21.15.47:0/2634397293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b078580 0x55ca7b099100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.722+0000 7f1c95e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b077bc0 0x55ca7b0920c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49624/0 (socket says 172.21.15.47:49624) 2024-03-21T11:35:43.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.723+0000 7f1c96800640 1 -- 172.21.15.47:0/2634397293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b078580 msgr2=0x55ca7b099100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.723+0000 7f1c96800640 1 --2- 172.21.15.47:0/2634397293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b078580 0x55ca7b099100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.723+0000 7f1c96800640 1 -- 172.21.15.47:0/2634397293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b077bc0 msgr2=0x55ca7b0920c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.723+0000 7f1c96800640 1 --2- 172.21.15.47:0/2634397293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b077bc0 0x55ca7b0920c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.723+0000 7f1c96800640 1 -- 172.21.15.47:0/2634397293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca7acef3f0 con 0x55ca7b0769f0 2024-03-21T11:35:43.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.724+0000 7f1c96800640 1 --2- 172.21.15.47:0/2634397293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b0769f0 0x55ca7b091b80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c800a3590 tx=0x7f1c8009b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a660509ce11a1a96 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:43.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.724+0000 7f1c7be00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:35:43.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.725+0000 7f1c7be00640 1 -- 172.21.15.47:0/2634397293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c800a3ac0 con 0x55ca7b0769f0 2024-03-21T11:35:43.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.726+0000 7f1c7be00640 1 -- 172.21.15.47:0/2634397293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1c800105a0 con 0x55ca7b0769f0 2024-03-21T11:35:43.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.726+0000 7f1ca021c740 1 -- 172.21.15.47:0/2634397293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c9000fd20 con 0x55ca7b0769f0 2024-03-21T11:35:43.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.727+0000 7f1c7be00640 1 -- 172.21.15.47:0/2634397293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c800b40a0 con 0x55ca7b0769f0 2024-03-21T11:35:43.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.727+0000 7f1ca021c740 10 client.19263 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:43.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.727+0000 7f1ca021c740 1 -- 172.21.15.47:0/2634397293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ca7adb97d0 con 0x55ca7b0769f0 2024-03-21T11:35:43.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.728+0000 7f1c7be00640 1 -- 172.21.15.47:0/2634397293 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c9000fd20 con 0x55ca7b0769f0 2024-03-21T11:35:43.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.728+0000 7f1c7be00640 1 -- 172.21.15.47:0/2634397293 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1c800a3630 con 0x55ca7b0769f0 2024-03-21T11:35:43.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.728+0000 7f1ca021c740 10 client.19263 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:43.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.728+0000 7f1ca021c740 10 client.19263 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:35:43.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.728+0000 7f1ca021c740 20 client.19263 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:43.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.728+0000 7f1ca021c740 1 --2- 172.21.15.47:0/2634397293 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ca7b09d4a0 0x55ca7b09f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.729+0000 7f1ca021c740 4 client.19263 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:43.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.729+0000 7f1ca021c740 1 -- 172.21.15.47:0/2634397293 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca7afe8800 con 0x55ca7b09d4a0 2024-03-21T11:35:43.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.729+0000 7f1c95e00640 1 --2- 172.21.15.47:0/2634397293 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ca7b09d4a0 0x55ca7b09f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.731+0000 7f1c95e00640 1 --2- 172.21.15.47:0/2634397293 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ca7b09d4a0 0x55ca7b09f8f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f21086be6de63b server_cookie=e61c11c2e13767e4 in_seq=0 out_seq=0 2024-03-21T11:35:43.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.731+0000 7f1c7be00640 10 client.19263 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:35:43.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.732+0000 7f1c7be00640 1 -- 172.21.15.47:0/2634397293 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ca7afe8800 con 0x55ca7b09d4a0 2024-03-21T11:35:43.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.732+0000 7f1c7be00640 10 client.19263 handle_command_reply: tid=0 2024-03-21T11:35:43.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.732+0000 7f1ca021c740 1 client.19263 shutdown 2024-03-21T11:35:43.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.733+0000 7f1ca021c740 1 -- 172.21.15.47:0/2634397293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b0769f0 msgr2=0x55ca7b091b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.733+0000 7f1ca021c740 1 --2- 172.21.15.47:0/2634397293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b0769f0 0x55ca7b091b80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c800a3590 tx=0x7f1c8009b8f0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.733+0000 7f1ca021c740 1 -- 172.21.15.47:0/2634397293 shutdown_connections 2024-03-21T11:35:43.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.733+0000 7f1ca021c740 1 --2- 172.21.15.47:0/2634397293 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ca7b09d4a0 0x55ca7b09f8f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.733+0000 7f1ca021c740 1 --2- 172.21.15.47:0/2634397293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b078580 0x55ca7b099100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.733+0000 7f1ca021c740 1 --2- 172.21.15.47:0/2634397293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b077bc0 0x55ca7b0920c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.733+0000 7f1ca021c740 1 --2- 172.21.15.47:0/2634397293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b0769f0 0x55ca7b091b80 secure :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c800a3590 tx=0x7f1c8009b8f0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.733+0000 7f1ca021c740 1 -- 172.21.15.47:0/2634397293 >> 172.21.15.47:0/2634397293 conn(0x55ca7b071de0 msgr2=0x55ca7b071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:43.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.733+0000 7f1c7be00640 0 client.19263 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:35:43.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.735+0000 7f1ca021c740 1 -- 172.21.15.47:0/2634397293 shutdown_connections 2024-03-21T11:35:43.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.735+0000 7f1ca021c740 1 -- 172.21.15.47:0/2634397293 wait complete. 2024-03-21T11:35:43.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.735+0000 7f1ca021c740 20 client.19263 trim_cache size 0 max 16384 2024-03-21T11:35:43.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.755+0000 7f1ca021c740 1 Processor -- start 2024-03-21T11:35:43.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.755+0000 7f1ca021c740 1 -- start start 2024-03-21T11:35:43.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1ca021c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b077bc0 0x55ca7b075750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1ca021c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b078580 0x55ca7b0990c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1ca021c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b072620 0x55ca7b07adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1ca021c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9806cf50 con 0x55ca7b077bc0 2024-03-21T11:35:43.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1ca021c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9806e8d0 con 0x55ca7b078580 2024-03-21T11:35:43.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1ca021c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9806d5b0 con 0x55ca7b072620 2024-03-21T11:35:43.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1c96800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b077bc0 0x55ca7b075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1c96800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b077bc0 0x55ca7b075750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49720/0 (socket says 172.21.15.47:49720) 2024-03-21T11:35:43.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1c96800640 1 -- 172.21.15.47:0/2218154160 learned_addr learned my addr 172.21.15.47:0/2218154160 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:43.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1c97200640 1 --2- 172.21.15.47:0/2218154160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b072620 0x55ca7b07adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1c95e00640 1 --2- 172.21.15.47:0/2218154160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b078580 0x55ca7b0990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1c96800640 1 -- 172.21.15.47:0/2218154160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b072620 msgr2=0x55ca7b07adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1c96800640 1 --2- 172.21.15.47:0/2218154160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b072620 0x55ca7b07adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1c96800640 1 -- 172.21.15.47:0/2218154160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b078580 msgr2=0x55ca7b0990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1c96800640 1 --2- 172.21.15.47:0/2218154160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b078580 0x55ca7b0990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.756+0000 7f1c96800640 1 -- 172.21.15.47:0/2218154160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca7b06d8b0 con 0x55ca7b077bc0 2024-03-21T11:35:43.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.757+0000 7f1c96800640 1 --2- 172.21.15.47:0/2218154160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b077bc0 0x55ca7b075750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000bd80 tx=0x7f1c800995c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0f7e361612d93d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:43.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.757+0000 7f1c86a00640 1 -- 172.21.15.47:0/2218154160 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c800a3ac0 con 0x55ca7b077bc0 2024-03-21T11:35:43.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.757+0000 7f1c86a00640 1 -- 172.21.15.47:0/2218154160 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c80011cc0 con 0x55ca7b077bc0 2024-03-21T11:35:43.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.757+0000 7f1c86a00640 1 -- 172.21.15.47:0/2218154160 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c800a3630 con 0x55ca7b077bc0 2024-03-21T11:35:43.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.758+0000 7f1ca021c740 1 -- 172.21.15.47:0/2218154160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b077bc0 msgr2=0x55ca7b075750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.758+0000 7f1ca021c740 1 --2- 172.21.15.47:0/2218154160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b077bc0 0x55ca7b075750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000bd80 tx=0x7f1c800995c0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.758+0000 7f1ca021c740 1 -- 172.21.15.47:0/2218154160 shutdown_connections 2024-03-21T11:35:43.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.758+0000 7f1ca021c740 1 --2- 172.21.15.47:0/2218154160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b072620 0x55ca7b07adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.758+0000 7f1ca021c740 1 --2- 172.21.15.47:0/2218154160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b078580 0x55ca7b0990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.758+0000 7f1ca021c740 1 --2- 172.21.15.47:0/2218154160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b077bc0 0x55ca7b075750 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.758+0000 7f1ca021c740 1 -- 172.21.15.47:0/2218154160 >> 172.21.15.47:0/2218154160 conn(0x55ca7b071de0 msgr2=0x55ca7b075040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:43.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.758+0000 7f1ca021c740 1 -- 172.21.15.47:0/2218154160 shutdown_connections 2024-03-21T11:35:43.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.758+0000 7f1ca021c740 1 -- 172.21.15.47:0/2218154160 wait complete. 2024-03-21T11:35:43.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.759+0000 7f1ca021c740 1 -- start start 2024-03-21T11:35:43.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.759+0000 7f1ca021c740 1 Processor -- start 2024-03-21T11:35:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.759+0000 7f1ca021c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b072620 0x55ca7b097010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.759+0000 7f1ca021c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b077bc0 0x55ca7b097550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.759+0000 7f1ca021c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b078580 0x55ca7b098560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.759+0000 7f1ca021c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9806e270 con 0x55ca7b072620 2024-03-21T11:35:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.759+0000 7f1ca021c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9806dc10 con 0x55ca7b078580 2024-03-21T11:35:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.759+0000 7f1ca021c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c981a1700 con 0x55ca7b077bc0 2024-03-21T11:35:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.759+0000 7f1ca021c740 10 client.0 _ll_register_callbacks cb 0x7f1c80010f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.760+0000 7f1c95e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b077bc0 0x55ca7b097550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.760+0000 7f1c96800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b072620 0x55ca7b097010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.760+0000 7f1c95e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b077bc0 0x55ca7b097550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41514/0 (socket says 172.21.15.47:41514) 2024-03-21T11:35:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.760+0000 7f1c95e00640 1 -- 172.21.15.47:0/3126341130 learned_addr learned my addr 172.21.15.47:0/3126341130 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.760+0000 7f1c97200640 1 --2- 172.21.15.47:0/3126341130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b078580 0x55ca7b098560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.760+0000 7f1c95e00640 1 -- 172.21.15.47:0/3126341130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b078580 msgr2=0x55ca7b098560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.760+0000 7f1c95e00640 1 --2- 172.21.15.47:0/3126341130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b078580 0x55ca7b098560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.760+0000 7f1c95e00640 1 -- 172.21.15.47:0/3126341130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b072620 msgr2=0x55ca7b097010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.760+0000 7f1c95e00640 1 --2- 172.21.15.47:0/3126341130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b072620 0x55ca7b097010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.760+0000 7f1c95e00640 1 -- 172.21.15.47:0/3126341130 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca7b098b50 con 0x55ca7b077bc0 2024-03-21T11:35:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.760+0000 7f1c97200640 1 --2- 172.21.15.47:0/3126341130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b078580 0x55ca7b098560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.760+0000 7f1c95e00640 1 --2- 172.21.15.47:0/3126341130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b077bc0 0x55ca7b097550 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009900 tx=0x7f1c88004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca4493de8f80f9c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.760+0000 7f1c7be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:43.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.761+0000 7f1c7be00640 1 -- 172.21.15.47:0/3126341130 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88049020 con 0x55ca7b077bc0 2024-03-21T11:35:43.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.761+0000 7f1c7be00640 1 -- 172.21.15.47:0/3126341130 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1c880041b0 con 0x55ca7b077bc0 2024-03-21T11:35:43.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.762+0000 7f1ca021c740 1 -- 172.21.15.47:0/3126341130 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c80011cc0 con 0x55ca7b077bc0 2024-03-21T11:35:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.762+0000 7f1c7be00640 1 -- 172.21.15.47:0/3126341130 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c88039040 con 0x55ca7b077bc0 2024-03-21T11:35:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.762+0000 7f1ca021c740 10 client.29677 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.762+0000 7f1ca021c740 1 -- 172.21.15.47:0/3126341130 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ca7b0f9570 con 0x55ca7b077bc0 2024-03-21T11:35:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.762+0000 7f1c7be00640 1 -- 172.21.15.47:0/3126341130 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c80011cc0 con 0x55ca7b077bc0 2024-03-21T11:35:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.762+0000 7f1c7be00640 1 -- 172.21.15.47:0/3126341130 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1c880051e0 con 0x55ca7b077bc0 2024-03-21T11:35:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.762+0000 7f1ca021c740 10 client.29677 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.762+0000 7f1ca021c740 10 client.29677 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:35:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.763+0000 7f1ca021c740 20 client.29677 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.763+0000 7f1ca021c740 1 --2- 172.21.15.47:0/3126341130 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ca7b0f9f00 0x55ca7b0fc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.763+0000 7f1ca021c740 4 client.29677 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:35:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.763+0000 7f1ca021c740 1 -- 172.21.15.47:0/3126341130 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ca7afe2b50 con 0x55ca7b0f9f00 2024-03-21T11:35:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.763+0000 7f1c96800640 1 --2- 172.21.15.47:0/3126341130 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ca7b0f9f00 0x55ca7b0fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.763+0000 7f1c96800640 1 --2- 172.21.15.47:0/3126341130 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ca7b0f9f00 0x55ca7b0fc320 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8030804edfaf9139 server_cookie=acd749a2a0ec2ef4 in_seq=0 out_seq=0 2024-03-21T11:35:43.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.763+0000 7f1c7be00640 10 client.29677 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:35:43.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1c7be00640 1 -- 172.21.15.47:0/3126341130 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ca7afe2b50 con 0x55ca7b0f9f00 2024-03-21T11:35:43.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1c7be00640 10 client.29677 handle_command_reply: tid=0 2024-03-21T11:35:43.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1ca021c740 1 client.29677 shutdown 2024-03-21T11:35:43.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1ca021c740 1 -- 172.21.15.47:0/3126341130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b077bc0 msgr2=0x55ca7b097550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1ca021c740 1 --2- 172.21.15.47:0/3126341130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b077bc0 0x55ca7b097550 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009900 tx=0x7f1c88004040 comp rx=0 tx=0).stop 2024-03-21T11:35:43.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1ca021c740 1 -- 172.21.15.47:0/3126341130 shutdown_connections 2024-03-21T11:35:43.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1ca021c740 1 --2- 172.21.15.47:0/3126341130 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55ca7b0f9f00 0x55ca7b0fc320 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1ca021c740 1 --2- 172.21.15.47:0/3126341130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7b078580 0x55ca7b098560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1ca021c740 1 --2- 172.21.15.47:0/3126341130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7b077bc0 0x55ca7b097550 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1ca021c740 1 --2- 172.21.15.47:0/3126341130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7b072620 0x55ca7b097010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1ca021c740 1 -- 172.21.15.47:0/3126341130 >> 172.21.15.47:0/3126341130 conn(0x55ca7b071de0 msgr2=0x55ca7b0758d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:43.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1c7be00640 0 client.29677 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:35:43.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1ca021c740 1 -- 172.21.15.47:0/3126341130 shutdown_connections 2024-03-21T11:35:43.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1ca021c740 1 -- 172.21.15.47:0/3126341130 wait complete. 2024-03-21T11:35:43.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.764+0000 7f1ca021c740 20 client.29677 trim_cache size 0 max 16384 2024-03-21T11:35:43.768 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:35:43.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.765+0000 7f1c9ca00640 1 -- 172.21.15.47:0/181704848 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1c7c0a2fa0 msgr2=0x7f1c7c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.765+0000 7f1c9ca00640 1 --2- 172.21.15.47:0/181704848 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1c7c0a2fa0 0x7f1c7c0a5460 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1c90020000 tx=0x7f1c90018ee0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.765+0000 7f1c9ca00640 1 -- 172.21.15.47:0/181704848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c98100480 msgr2=0x7f1c98195210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:43.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.765+0000 7f1c9ca00640 1 --2- 172.21.15.47:0/181704848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c98100480 0x7f1c98195210 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000b3c0 tx=0x7f1c8000b880 comp rx=0 tx=0).stop 2024-03-21T11:35:43.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.765+0000 7f1c9ca00640 1 -- 172.21.15.47:0/181704848 shutdown_connections 2024-03-21T11:35:43.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.765+0000 7f1c9ca00640 1 --2- 172.21.15.47:0/181704848 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1c7c0a2fa0 0x7f1c7c0a5460 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.765+0000 7f1c9ca00640 1 --2- 172.21.15.47:0/181704848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c98102030 0x7f1c98191d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.765+0000 7f1c9ca00640 1 --2- 172.21.15.47:0/181704848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c98101670 0x7f1c98195750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.765+0000 7f1c9ca00640 1 --2- 172.21.15.47:0/181704848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c98100480 0x7f1c98195210 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:43.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.765+0000 7f1c9ca00640 1 -- 172.21.15.47:0/181704848 >> 172.21.15.47:0/181704848 conn(0x7f1c980fba10 msgr2=0x7f1c980fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:43.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.765+0000 7f1c9ca00640 1 -- 172.21.15.47:0/181704848 shutdown_connections 2024-03-21T11:35:43.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.766+0000 7f1c9ca00640 1 -- 172.21.15.47:0/181704848 wait complete. 2024-03-21T11:35:43.786 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:35:43.786 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:35:43.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.998+0000 7fd899800640 1 -- 172.21.15.47:0/2222483822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8941038e0 msgr2=0x7fd894103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.998+0000 7fd899800640 1 --2- 172.21.15.47:0/2222483822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8941038e0 0x7fd894103d00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd884009900 tx=0x7fd88402bc30 comp rx=0 tx=0).stop 2024-03-21T11:35:44.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.998+0000 7fd899800640 1 -- 172.21.15.47:0/2222483822 shutdown_connections 2024-03-21T11:35:44.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.998+0000 7fd899800640 1 --2- 172.21.15.47:0/2222483822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894105490 0x7fd89410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.998+0000 7fd899800640 1 --2- 172.21.15.47:0/2222483822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd894104ad0 0x7fd894104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.998+0000 7fd899800640 1 --2- 172.21.15.47:0/2222483822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8941038e0 0x7fd894103d00 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.998+0000 7fd899800640 1 -- 172.21.15.47:0/2222483822 >> 172.21.15.47:0/2222483822 conn(0x7fd8940fee50 msgr2=0x7fd894101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.998+0000 7fd899800640 1 -- 172.21.15.47:0/2222483822 shutdown_connections 2024-03-21T11:35:44.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.998+0000 7fd899800640 1 -- 172.21.15.47:0/2222483822 wait complete. 2024-03-21T11:35:44.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:43.999+0000 7fd899800640 1 Processor -- start 2024-03-21T11:35:44.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.000+0000 7fd899800640 1 -- start start 2024-03-21T11:35:44.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.000+0000 7fd899800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8941038e0 0x7fd894199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.000+0000 7fd899800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd894104ad0 0x7fd89419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.000+0000 7fd899800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894105490 0x7fd8941a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.000+0000 7fd899800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89410f5a0 con 0x7fd894105490 2024-03-21T11:35:44.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.000+0000 7fd899800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89410f420 con 0x7fd8941038e0 2024-03-21T11:35:44.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.000+0000 7fd899800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89410f720 con 0x7fd894104ad0 2024-03-21T11:35:44.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.000+0000 7fd892a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8941038e0 0x7fd894199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.000+0000 7fd892a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8941038e0 0x7fd894199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49670/0 (socket says 172.21.15.47:49670) 2024-03-21T11:35:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.000+0000 7fd892a00640 1 -- 172.21.15.47:0/132025254 learned_addr learned my addr 172.21.15.47:0/132025254 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.000+0000 7fd892000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd894104ad0 0x7fd89419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.001+0000 7fd893400640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894105490 0x7fd8941a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.001+0000 7fd892a00640 1 -- 172.21.15.47:0/132025254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd894104ad0 msgr2=0x7fd89419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.001+0000 7fd892a00640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd894104ad0 0x7fd89419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.001+0000 7fd892a00640 1 -- 172.21.15.47:0/132025254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894105490 msgr2=0x7fd8941a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.001+0000 7fd892a00640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894105490 0x7fd8941a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.001+0000 7fd892a00640 1 -- 172.21.15.47:0/132025254 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8941a1870 con 0x7fd8941038e0 2024-03-21T11:35:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.001+0000 7fd892000640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd894104ad0 0x7fd89419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.001+0000 7fd893400640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894105490 0x7fd8941a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.001+0000 7fd892a00640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8941038e0 0x7fd894199c60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd884000c00 tx=0x7fd88402c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.001+0000 7fd88b400640 1 -- 172.21.15.47:0/132025254 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884043020 con 0x7fd8941038e0 2024-03-21T11:35:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.001+0000 7fd88b400640 1 -- 172.21.15.47:0/132025254 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8840054e0 con 0x7fd8941038e0 2024-03-21T11:35:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.001+0000 7fd899800640 1 -- 172.21.15.47:0/132025254 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8941a1b00 con 0x7fd8941038e0 2024-03-21T11:35:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.002+0000 7fd88b400640 1 -- 172.21.15.47:0/132025254 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884003ce0 con 0x7fd8941038e0 2024-03-21T11:35:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.002+0000 7fd899800640 1 -- 172.21.15.47:0/132025254 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8941a2070 con 0x7fd8941038e0 2024-03-21T11:35:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.005+0000 7fd89c5e8740 1 Processor -- start 2024-03-21T11:35:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.005+0000 7fd88b400640 1 -- 172.21.15.47:0/132025254 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd884046020 con 0x7fd8941038e0 2024-03-21T11:35:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.005+0000 7fd89c5e8740 1 -- start start 2024-03-21T11:35:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.006+0000 7fd89c5e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149526e810 0x56149526ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.006+0000 7fd89c5e8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149526fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.006+0000 7fd89c5e8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614952703a0 0x561495276c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.006+0000 7fd89c5e8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614951d32c0 con 0x5614952703a0 2024-03-21T11:35:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.006+0000 7fd89c5e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149517bc90 con 0x56149526e810 2024-03-21T11:35:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.006+0000 7fd89c5e8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614951cf360 con 0x56149526f9e0 2024-03-21T11:35:44.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.006+0000 7fd892a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149526fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.006+0000 7fd893400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149526e810 0x56149526ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.006+0000 7fd893400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149526e810 0x56149526ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49680/0 (socket says 172.21.15.47:49680) 2024-03-21T11:35:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.006+0000 7fd892a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149526fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41528/0 (socket says 172.21.15.47:41528) 2024-03-21T11:35:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.006+0000 7fd893400640 1 -- 172.21.15.47:0/3986567103 learned_addr learned my addr 172.21.15.47:0/3986567103 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.006+0000 7fd892000640 1 --2- 172.21.15.47:0/3986567103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614952703a0 0x561495276c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.007+0000 7fd892a00640 1 -- 172.21.15.47:0/3986567103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149526e810 msgr2=0x56149526ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.007+0000 7fd892a00640 1 --2- 172.21.15.47:0/3986567103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149526e810 0x56149526ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.007+0000 7fd892a00640 1 -- 172.21.15.47:0/3986567103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614952703a0 msgr2=0x561495276c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.007+0000 7fd892a00640 1 --2- 172.21.15.47:0/3986567103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614952703a0 0x561495276c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.007+0000 7fd892a00640 1 -- 172.21.15.47:0/3986567103 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614951a0080 con 0x56149526f9e0 2024-03-21T11:35:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.007+0000 7fd893400640 1 --2- 172.21.15.47:0/3986567103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149526e810 0x56149526ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.007+0000 7fd892000640 1 --2- 172.21.15.47:0/3986567103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614952703a0 0x561495276c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.007+0000 7fd892a00640 1 --2- 172.21.15.47:0/3986567103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149526fe60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd8840ef040 tx=0x7fd8840f7b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b13c9e4740871a29 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.007+0000 7fd88b400640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8780a2fa0 0x7fd8780a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.007+0000 7fd87fe00640 1 -- 172.21.15.47:0/3986567103 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884100070 con 0x56149526f9e0 2024-03-21T11:35:44.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.007+0000 7fd87fe00640 1 -- 172.21.15.47:0/3986567103 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8840be3a0 con 0x56149526f9e0 2024-03-21T11:35:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.007+0000 7fd87fe00640 1 -- 172.21.15.47:0/3986567103 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8840bbde0 con 0x56149526f9e0 2024-03-21T11:35:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.007+0000 7fd88b400640 1 -- 172.21.15.47:0/132025254 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd8840f1900 con 0x7fd8941038e0 2024-03-21T11:35:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.007+0000 7fd893400640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8780a2fa0 0x7fd8780a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.008+0000 7fd89c5e8740 1 -- 172.21.15.47:0/3986567103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 msgr2=0x56149526fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.008+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/3986567103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149526fe60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd8840ef040 tx=0x7fd8840f7b40 comp rx=0 tx=0).stop 2024-03-21T11:35:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.008+0000 7fd893400640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8780a2fa0 0x7fd8780a5460 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009fd0 tx=0x7fd88c009940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.008+0000 7fd89c5e8740 1 -- 172.21.15.47:0/3986567103 shutdown_connections 2024-03-21T11:35:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.008+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/3986567103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614952703a0 0x561495276c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.008+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/3986567103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149526fe60 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.008+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/3986567103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149526e810 0x56149526ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.008+0000 7fd89c5e8740 1 -- 172.21.15.47:0/3986567103 >> 172.21.15.47:0/3986567103 conn(0x561495269c00 msgr2=0x56149526a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.008+0000 7fd89c5e8740 1 -- 172.21.15.47:0/3986567103 shutdown_connections 2024-03-21T11:35:44.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.008+0000 7fd89c5e8740 1 -- 172.21.15.47:0/3986567103 wait complete. 2024-03-21T11:35:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.009+0000 7fd89c5e8740 1 -- start start 2024-03-21T11:35:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.009+0000 7fd89c5e8740 1 Processor -- start 2024-03-21T11:35:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.009+0000 7fd89c5e8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526e810 0x561495278700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd89c5e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149526f9e0 0x561495278c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd89c5e8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614952703a0 0x56149527fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd89c5e8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561495277170 con 0x5614952703a0 2024-03-21T11:35:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd89c5e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56149518f0d0 con 0x56149526f9e0 2024-03-21T11:35:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd89c5e8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614952772f0 con 0x56149526e810 2024-03-21T11:35:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd89c5e8740 10 client.0 _ll_register_callbacks cb 0x561495193e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd892a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526e810 0x561495278700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd892a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526e810 0x561495278700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41532/0 (socket says 172.21.15.47:41532) 2024-03-21T11:35:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd892a00640 1 -- 172.21.15.47:0/779762183 learned_addr learned my addr 172.21.15.47:0/779762183 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd893400640 1 --2- 172.21.15.47:0/779762183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614952703a0 0x56149527fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd892000640 1 --2- 172.21.15.47:0/779762183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149526f9e0 0x561495278c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd892a00640 1 -- 172.21.15.47:0/779762183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149526f9e0 msgr2=0x561495278c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd892a00640 1 --2- 172.21.15.47:0/779762183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149526f9e0 0x561495278c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd892a00640 1 -- 172.21.15.47:0/779762183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614952703a0 msgr2=0x56149527fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd892a00640 1 --2- 172.21.15.47:0/779762183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614952703a0 0x56149527fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd892a00640 1 -- 172.21.15.47:0/779762183 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561494ee73f0 con 0x56149526e810 2024-03-21T11:35:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.010+0000 7fd892a00640 1 --2- 172.21.15.47:0/779762183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526e810 0x561495278700 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd8840ba070 tx=0x7fd8840f1c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a91150a8bb62a17 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.011+0000 7fd87cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.011+0000 7fd87cc00640 1 -- 172.21.15.47:0/779762183 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8840b0120 con 0x56149526e810 2024-03-21T11:35:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.011+0000 7fd87cc00640 1 -- 172.21.15.47:0/779762183 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd8840bbfa0 con 0x56149526e810 2024-03-21T11:35:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.011+0000 7fd89c5e8740 1 -- 172.21.15.47:0/779762183 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8840be3a0 con 0x56149526e810 2024-03-21T11:35:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.011+0000 7fd87cc00640 1 -- 172.21.15.47:0/779762183 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd884100070 con 0x56149526e810 2024-03-21T11:35:44.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.011+0000 7fd89c5e8740 10 client.29689 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:44.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.011+0000 7fd89c5e8740 1 -- 172.21.15.47:0/779762183 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561494fb17d0 con 0x56149526e810 2024-03-21T11:35:44.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.011+0000 7fd87cc00640 1 -- 172.21.15.47:0/779762183 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8840be3a0 con 0x56149526e810 2024-03-21T11:35:44.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.012+0000 7fd87cc00640 1 -- 172.21.15.47:0/779762183 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd8840a8210 con 0x56149526e810 2024-03-21T11:35:44.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.012+0000 7fd89c5e8740 10 client.29689 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:44.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.012+0000 7fd89c5e8740 10 client.29689 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:35:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.012+0000 7fd89c5e8740 20 client.29689 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.012+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/779762183 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561495283de0 0x5614952861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.012+0000 7fd89c5e8740 4 client.29689 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.012+0000 7fd89c5e8740 1 -- 172.21.15.47:0/779762183 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614951e0800 con 0x561495283de0 2024-03-21T11:35:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.012+0000 7fd892000640 1 --2- 172.21.15.47:0/779762183 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561495283de0 0x5614952861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.013+0000 7fd892000640 1 --2- 172.21.15.47:0/779762183 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561495283de0 0x5614952861d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa4b832dddfa450 server_cookie=2cfa313407808677 in_seq=0 out_seq=0 2024-03-21T11:35:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.013+0000 7fd87cc00640 10 client.29689 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:35:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.014+0000 7fd87cc00640 1 -- 172.21.15.47:0/779762183 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5614951e0800 con 0x561495283de0 2024-03-21T11:35:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.014+0000 7fd87cc00640 10 client.29689 handle_command_reply: tid=0 2024-03-21T11:35:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.014+0000 7fd89c5e8740 1 client.29689 shutdown 2024-03-21T11:35:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.014+0000 7fd89c5e8740 1 -- 172.21.15.47:0/779762183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526e810 msgr2=0x561495278700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.014+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/779762183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526e810 0x561495278700 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd8840ba070 tx=0x7fd8840f1c00 comp rx=0 tx=0).stop 2024-03-21T11:35:44.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.014+0000 7fd89c5e8740 1 -- 172.21.15.47:0/779762183 shutdown_connections 2024-03-21T11:35:44.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.014+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/779762183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5614952703a0 0x56149527fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.014+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/779762183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56149526f9e0 0x561495278c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.014+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/779762183 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561495283de0 0x5614952861d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.014+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/779762183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526e810 0x561495278700 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.014+0000 7fd89c5e8740 1 -- 172.21.15.47:0/779762183 >> 172.21.15.47:0/779762183 conn(0x561495269c00 msgr2=0x561495268eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.014+0000 7fd87cc00640 0 client.29689 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:35:44.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.014+0000 7fd89c5e8740 1 -- 172.21.15.47:0/779762183 shutdown_connections 2024-03-21T11:35:44.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.014+0000 7fd89c5e8740 1 -- 172.21.15.47:0/779762183 wait complete. 2024-03-21T11:35:44.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.015+0000 7fd89c5e8740 20 client.29689 trim_cache size 0 max 16384 2024-03-21T11:35:44.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.030+0000 7fd89c5e8740 1 Processor -- start 2024-03-21T11:35:44.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.030+0000 7fd89c5e8740 1 -- start start 2024-03-21T11:35:44.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.030+0000 7fd89c5e8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56149526e810 0x5614952729e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.030+0000 7fd89c5e8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149527fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.030+0000 7fd89c5e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614952703a0 0x561495272f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.030+0000 7fd89c5e8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8941a7ed0 con 0x56149526e810 2024-03-21T11:35:44.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.030+0000 7fd89c5e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8941a9850 con 0x5614952703a0 2024-03-21T11:35:44.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.030+0000 7fd89c5e8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8941a8530 con 0x56149526f9e0 2024-03-21T11:35:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.031+0000 7fd892000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149527fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.031+0000 7fd893400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614952703a0 0x561495272f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.031+0000 7fd892000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149527fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41544/0 (socket says 172.21.15.47:41544) 2024-03-21T11:35:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.031+0000 7fd892000640 1 -- 172.21.15.47:0/180139062 learned_addr learned my addr 172.21.15.47:0/180139062 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.031+0000 7fd892a00640 1 --2- 172.21.15.47:0/180139062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56149526e810 0x5614952729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.031+0000 7fd892000640 1 -- 172.21.15.47:0/180139062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614952703a0 msgr2=0x561495272f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.031+0000 7fd892000640 1 --2- 172.21.15.47:0/180139062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614952703a0 0x561495272f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.031+0000 7fd892000640 1 -- 172.21.15.47:0/180139062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56149526e810 msgr2=0x5614952729e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.031+0000 7fd892000640 1 --2- 172.21.15.47:0/180139062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56149526e810 0x5614952729e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.031+0000 7fd892000640 1 -- 172.21.15.47:0/180139062 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561495286800 con 0x56149526f9e0 2024-03-21T11:35:44.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.031+0000 7fd893400640 1 --2- 172.21.15.47:0/180139062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614952703a0 0x561495272f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:44.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.031+0000 7fd892000640 1 --2- 172.21.15.47:0/180139062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149527fc10 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd880005d00 tx=0x7fd880005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b155d6ca2fde112 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.032+0000 7fd87f400640 1 -- 172.21.15.47:0/180139062 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8800202d0 con 0x56149526f9e0 2024-03-21T11:35:44.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.032+0000 7fd87f400640 1 -- 172.21.15.47:0/180139062 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd880024030 con 0x56149526f9e0 2024-03-21T11:35:44.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.032+0000 7fd87f400640 1 -- 172.21.15.47:0/180139062 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880004bc0 con 0x56149526f9e0 2024-03-21T11:35:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.033+0000 7fd89c5e8740 1 -- 172.21.15.47:0/180139062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 msgr2=0x56149527fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.033+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/180139062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149527fc10 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd880005d00 tx=0x7fd880005f90 comp rx=0 tx=0).stop 2024-03-21T11:35:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.033+0000 7fd89c5e8740 1 -- 172.21.15.47:0/180139062 shutdown_connections 2024-03-21T11:35:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.033+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/180139062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614952703a0 0x561495272f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.033+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/180139062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149527fc10 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.033+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/180139062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56149526e810 0x5614952729e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.033+0000 7fd89c5e8740 1 -- 172.21.15.47:0/180139062 >> 172.21.15.47:0/180139062 conn(0x561495269c00 msgr2=0x56149526d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.033+0000 7fd89c5e8740 1 -- 172.21.15.47:0/180139062 shutdown_connections 2024-03-21T11:35:44.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.033+0000 7fd89c5e8740 1 -- 172.21.15.47:0/180139062 wait complete. 2024-03-21T11:35:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.033+0000 7fd89c5e8740 1 -- start start 2024-03-21T11:35:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.033+0000 7fd89c5e8740 1 Processor -- start 2024-03-21T11:35:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.034+0000 7fd89c5e8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56149526e810 0x56149527be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.034+0000 7fd89c5e8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149527c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.034+0000 7fd89c5e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614952703a0 0x56149527f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.034+0000 7fd89c5e8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8941a91f0 con 0x56149526e810 2024-03-21T11:35:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.034+0000 7fd89c5e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8941a8b90 con 0x5614952703a0 2024-03-21T11:35:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.034+0000 7fd89c5e8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8941a6090 con 0x56149526f9e0 2024-03-21T11:35:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.034+0000 7fd89c5e8740 10 client.0 _ll_register_callbacks cb 0x7fd88c0097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.034+0000 7fd892000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149527c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.034+0000 7fd892000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149527c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41560/0 (socket says 172.21.15.47:41560) 2024-03-21T11:35:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.034+0000 7fd892000640 1 -- 172.21.15.47:0/4194074548 learned_addr learned my addr 172.21.15.47:0/4194074548 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.034+0000 7fd893400640 1 --2- 172.21.15.47:0/4194074548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614952703a0 0x56149527f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.034+0000 7fd892a00640 1 --2- 172.21.15.47:0/4194074548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56149526e810 0x56149527be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.035+0000 7fd892000640 1 -- 172.21.15.47:0/4194074548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614952703a0 msgr2=0x56149527f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.035+0000 7fd892000640 1 --2- 172.21.15.47:0/4194074548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614952703a0 0x56149527f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.035+0000 7fd892000640 1 -- 172.21.15.47:0/4194074548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56149526e810 msgr2=0x56149527be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.035+0000 7fd892000640 1 --2- 172.21.15.47:0/4194074548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56149526e810 0x56149527be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.035+0000 7fd892000640 1 -- 172.21.15.47:0/4194074548 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56149527f970 con 0x56149526f9e0 2024-03-21T11:35:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.035+0000 7fd892a00640 1 --2- 172.21.15.47:0/4194074548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56149526e810 0x56149527be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.035+0000 7fd892000640 1 --2- 172.21.15.47:0/4194074548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149527c360 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd880000c00 tx=0x7fd880009c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d316def5b8b7be20 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.035+0000 7fd87cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.035+0000 7fd87cc00640 1 -- 172.21.15.47:0/4194074548 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880020310 con 0x56149526f9e0 2024-03-21T11:35:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.035+0000 7fd87cc00640 1 -- 172.21.15.47:0/4194074548 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd880011bb0 con 0x56149526f9e0 2024-03-21T11:35:44.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.035+0000 7fd89c5e8740 1 -- 172.21.15.47:0/4194074548 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd880024030 con 0x56149526f9e0 2024-03-21T11:35:44.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.036+0000 7fd87cc00640 1 -- 172.21.15.47:0/4194074548 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd880011df0 con 0x56149526f9e0 2024-03-21T11:35:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.036+0000 7fd89c5e8740 10 client.29695 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.036+0000 7fd89c5e8740 1 -- 172.21.15.47:0/4194074548 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5614952e01a0 con 0x56149526f9e0 2024-03-21T11:35:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.036+0000 7fd87cc00640 1 -- 172.21.15.47:0/4194074548 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd880024030 con 0x56149526f9e0 2024-03-21T11:35:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.036+0000 7fd87cc00640 1 -- 172.21.15.47:0/4194074548 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd8800107e0 con 0x56149526f9e0 2024-03-21T11:35:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.036+0000 7fd89c5e8740 10 client.29695 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.036+0000 7fd89c5e8740 10 client.29695 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:35:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.036+0000 7fd89c5e8740 20 client.29695 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.036+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/4194074548 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5614952e0b50 0x5614952e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.036+0000 7fd89c5e8740 4 client.29695 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:35:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.036+0000 7fd89c5e8740 1 -- 172.21.15.47:0/4194074548 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5614951dab50 con 0x5614952e0b50 2024-03-21T11:35:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.037+0000 7fd892a00640 1 --2- 172.21.15.47:0/4194074548 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5614952e0b50 0x5614952e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.037+0000 7fd892a00640 1 --2- 172.21.15.47:0/4194074548 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5614952e0b50 0x5614952e2f70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee8f5b50512f644 server_cookie=f08b98f677a423a1 in_seq=0 out_seq=0 2024-03-21T11:35:44.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.037+0000 7fd87cc00640 10 client.29695 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:35:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.037+0000 7fd87cc00640 1 -- 172.21.15.47:0/4194074548 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5614951dab50 con 0x5614952e0b50 2024-03-21T11:35:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.037+0000 7fd87cc00640 10 client.29695 handle_command_reply: tid=0 2024-03-21T11:35:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.037+0000 7fd89c5e8740 1 client.29695 shutdown 2024-03-21T11:35:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.038+0000 7fd89c5e8740 1 -- 172.21.15.47:0/4194074548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 msgr2=0x56149527c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.038+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/4194074548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149527c360 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd880000c00 tx=0x7fd880009c50 comp rx=0 tx=0).stop 2024-03-21T11:35:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.038+0000 7fd89c5e8740 1 -- 172.21.15.47:0/4194074548 shutdown_connections 2024-03-21T11:35:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.038+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/4194074548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5614952703a0 0x56149527f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.038+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/4194074548 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5614952e0b50 0x5614952e2f70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.038+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/4194074548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56149526f9e0 0x56149527c360 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.038+0000 7fd89c5e8740 1 --2- 172.21.15.47:0/4194074548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56149526e810 0x56149527be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.038+0000 7fd87cc00640 0 client.29695 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:35:44.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.038+0000 7fd89c5e8740 1 -- 172.21.15.47:0/4194074548 >> 172.21.15.47:0/4194074548 conn(0x561495269c00 msgr2=0x561495268620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.038+0000 7fd89c5e8740 1 -- 172.21.15.47:0/4194074548 shutdown_connections 2024-03-21T11:35:44.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.038+0000 7fd89c5e8740 1 -- 172.21.15.47:0/4194074548 wait complete. 2024-03-21T11:35:44.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.038+0000 7fd89c5e8740 20 client.29695 trim_cache size 0 max 16384 2024-03-21T11:35:44.043 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:35:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.039+0000 7fd899800640 1 -- 172.21.15.47:0/132025254 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8780a2fa0 msgr2=0x7fd8780a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.039+0000 7fd899800640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8780a2fa0 0x7fd8780a5460 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009fd0 tx=0x7fd88c009940 comp rx=0 tx=0).stop 2024-03-21T11:35:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.039+0000 7fd899800640 1 -- 172.21.15.47:0/132025254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8941038e0 msgr2=0x7fd894199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.039+0000 7fd899800640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8941038e0 0x7fd894199c60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd884000c00 tx=0x7fd88402c6c0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.039+0000 7fd899800640 1 -- 172.21.15.47:0/132025254 shutdown_connections 2024-03-21T11:35:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.039+0000 7fd899800640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8780a2fa0 0x7fd8780a5460 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.039+0000 7fd899800640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd894105490 0x7fd8941a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.039+0000 7fd899800640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd894104ad0 0x7fd89419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.039+0000 7fd899800640 1 --2- 172.21.15.47:0/132025254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8941038e0 0x7fd894199c60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.039+0000 7fd899800640 1 -- 172.21.15.47:0/132025254 >> 172.21.15.47:0/132025254 conn(0x7fd8940fee50 msgr2=0x7fd894107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.039+0000 7fd899800640 1 -- 172.21.15.47:0/132025254 shutdown_connections 2024-03-21T11:35:44.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.039+0000 7fd899800640 1 -- 172.21.15.47:0/132025254 wait complete. 2024-03-21T11:35:44.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:43 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2580485837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:35:44.060 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:35:44.060 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:35:44.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:43 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2580485837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:35:44.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:43 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2580485837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:35:44.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.293+0000 7f3aa0a00640 1 -- 172.21.15.47:0/3292408386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9c104ad0 msgr2=0x7f3a9c104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.293+0000 7f3aa0a00640 1 --2- 172.21.15.47:0/3292408386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9c104ad0 0x7f3a9c104f50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009900 tx=0x7f3a8802bc30 comp rx=0 tx=0).stop 2024-03-21T11:35:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.293+0000 7f3aa0a00640 1 -- 172.21.15.47:0/3292408386 shutdown_connections 2024-03-21T11:35:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.293+0000 7f3aa0a00640 1 --2- 172.21.15.47:0/3292408386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a9c105490 0x7f3a9c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.293+0000 7f3aa0a00640 1 --2- 172.21.15.47:0/3292408386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9c104ad0 0x7f3a9c104f50 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.293+0000 7f3aa0a00640 1 --2- 172.21.15.47:0/3292408386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c1038e0 0x7f3a9c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.293+0000 7f3aa0a00640 1 -- 172.21.15.47:0/3292408386 >> 172.21.15.47:0/3292408386 conn(0x7f3a9c0fee50 msgr2=0x7f3a9c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.293+0000 7f3aa0a00640 1 -- 172.21.15.47:0/3292408386 shutdown_connections 2024-03-21T11:35:44.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.293+0000 7f3aa0a00640 1 -- 172.21.15.47:0/3292408386 wait complete. 2024-03-21T11:35:44.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.294+0000 7f3aa0a00640 1 Processor -- start 2024-03-21T11:35:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.295+0000 7f3aa0a00640 1 -- start start 2024-03-21T11:35:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.295+0000 7f3aa0a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9c1038e0 0x7f3a9c199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.295+0000 7f3aa0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a9c104ad0 0x7f3a9c19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.295+0000 7f3aa0a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c105490 0x7f3a9c1a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.295+0000 7f3aa0a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c10f560 con 0x7f3a9c105490 2024-03-21T11:35:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.295+0000 7f3aa0a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c10f3e0 con 0x7f3a9c1038e0 2024-03-21T11:35:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.295+0000 7f3aa0a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c10f6e0 con 0x7f3a9c104ad0 2024-03-21T11:35:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.295+0000 7f3a99e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9c1038e0 0x7f3a9c199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.295+0000 7f3a99400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a9c104ad0 0x7f3a9c19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.295+0000 7f3a99e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9c1038e0 0x7f3a9c199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49714/0 (socket says 172.21.15.47:49714) 2024-03-21T11:35:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.295+0000 7f3a99e00640 1 -- 172.21.15.47:0/3829258751 learned_addr learned my addr 172.21.15.47:0/3829258751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.295+0000 7f3a99400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a9c104ad0 0x7f3a9c19a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41574/0 (socket says 172.21.15.47:41574) 2024-03-21T11:35:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.295+0000 7f3a9a800640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c105490 0x7f3a9c1a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.296+0000 7f3a99e00640 1 -- 172.21.15.47:0/3829258751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a9c104ad0 msgr2=0x7f3a9c19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.296+0000 7f3a99e00640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a9c104ad0 0x7f3a9c19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.296+0000 7f3a99e00640 1 -- 172.21.15.47:0/3829258751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c105490 msgr2=0x7f3a9c1a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.296+0000 7f3a99e00640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c105490 0x7f3a9c1a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.296+0000 7f3a99e00640 1 -- 172.21.15.47:0/3829258751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c1a17d0 con 0x7f3a9c1038e0 2024-03-21T11:35:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.296+0000 7f3a9a800640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c105490 0x7f3a9c1a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.296+0000 7f3a99400640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a9c104ad0 0x7f3a9c19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.296+0000 7f3a99e00640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9c1038e0 0x7f3a9c199bb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00e9c0 tx=0x7f3a8c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.296+0000 7f3a92a00640 1 -- 172.21.15.47:0/3829258751 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c00cd80 con 0x7f3a9c1038e0 2024-03-21T11:35:44.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.296+0000 7f3a92a00640 1 -- 172.21.15.47:0/3829258751 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a8c0035f0 con 0x7f3a9c1038e0 2024-03-21T11:35:44.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.296+0000 7f3aa0a00640 1 -- 172.21.15.47:0/3829258751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c1a1ac0 con 0x7f3a9c1038e0 2024-03-21T11:35:44.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.296+0000 7f3a92a00640 1 -- 172.21.15.47:0/3829258751 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c010690 con 0x7f3a9c1038e0 2024-03-21T11:35:44.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.297+0000 7f3aa0a00640 1 -- 172.21.15.47:0/3829258751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c1a1fe0 con 0x7f3a9c1038e0 2024-03-21T11:35:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.300+0000 7f3a92a00640 1 -- 172.21.15.47:0/3829258751 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3a8c0037a0 con 0x7f3a9c1038e0 2024-03-21T11:35:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.300+0000 7f3aa3773740 1 Processor -- start 2024-03-21T11:35:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.301+0000 7f3aa3773740 1 -- start start 2024-03-21T11:35:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.301+0000 7f3aa3773740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad6f810 0x562c1ad6fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.301+0000 7f3aa3773740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad70e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.301+0000 7f3aa3773740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad713a0 0x562c1ad77c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.301+0000 7f3aa3773740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1acd42c0 con 0x562c1ad709e0 2024-03-21T11:35:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.301+0000 7f3aa3773740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1ac7cc90 con 0x562c1ad6f810 2024-03-21T11:35:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.301+0000 7f3aa3773740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1acd0360 con 0x562c1ad713a0 2024-03-21T11:35:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.301+0000 7f3a9a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad6f810 0x562c1ad6fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.301+0000 7f3a9a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad6f810 0x562c1ad6fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49726/0 (socket says 172.21.15.47:49726) 2024-03-21T11:35:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.301+0000 7f3a9a800640 1 -- 172.21.15.47:0/2157935398 learned_addr learned my addr 172.21.15.47:0/2157935398 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.301+0000 7f3a99400640 1 --2- 172.21.15.47:0/2157935398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad713a0 0x562c1ad77c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.301+0000 7f3a99e00640 1 --2- 172.21.15.47:0/2157935398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad70e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.302+0000 7f3a9a800640 1 -- 172.21.15.47:0/2157935398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad713a0 msgr2=0x562c1ad77c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.302+0000 7f3a9a800640 1 --2- 172.21.15.47:0/2157935398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad713a0 0x562c1ad77c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.302+0000 7f3a9a800640 1 -- 172.21.15.47:0/2157935398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 msgr2=0x562c1ad70e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.302+0000 7f3a9a800640 1 --2- 172.21.15.47:0/2157935398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad70e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.302+0000 7f3a9a800640 1 -- 172.21.15.47:0/2157935398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c1aca1080 con 0x562c1ad6f810 2024-03-21T11:35:44.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.302+0000 7f3a99e00640 1 --2- 172.21.15.47:0/2157935398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad70e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:35:44.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.302+0000 7f3a9a800640 1 --2- 172.21.15.47:0/2157935398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad6f810 0x562c1ad6fc30 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009fd0 tx=0x7f3a940108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b8a91f95ec13c86 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.302+0000 7f3a87000640 1 -- 172.21.15.47:0/2157935398 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a94011760 con 0x562c1ad6f810 2024-03-21T11:35:44.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.302+0000 7f3a92a00640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a800a2f20 0x7f3a800a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.302+0000 7f3a87000640 1 -- 172.21.15.47:0/2157935398 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a94011de0 con 0x562c1ad6f810 2024-03-21T11:35:44.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.302+0000 7f3a87000640 1 -- 172.21.15.47:0/2157935398 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9400e3c0 con 0x562c1ad6f810 2024-03-21T11:35:44.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.302+0000 7f3a92a00640 1 -- 172.21.15.47:0/3829258751 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3a8c0cfc60 con 0x7f3a9c1038e0 2024-03-21T11:35:44.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.303+0000 7f3a99e00640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a800a2f20 0x7f3a800a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.303+0000 7f3aa3773740 1 -- 172.21.15.47:0/2157935398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad6f810 msgr2=0x562c1ad6fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.303+0000 7f3aa3773740 1 --2- 172.21.15.47:0/2157935398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad6f810 0x562c1ad6fc30 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009fd0 tx=0x7f3a940108e0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.303+0000 7f3aa3773740 1 -- 172.21.15.47:0/2157935398 shutdown_connections 2024-03-21T11:35:44.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.303+0000 7f3aa3773740 1 --2- 172.21.15.47:0/2157935398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad713a0 0x562c1ad77c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.303+0000 7f3aa3773740 1 --2- 172.21.15.47:0/2157935398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad70e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.303+0000 7f3aa3773740 1 --2- 172.21.15.47:0/2157935398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad6f810 0x562c1ad6fc30 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.303+0000 7f3aa3773740 1 -- 172.21.15.47:0/2157935398 >> 172.21.15.47:0/2157935398 conn(0x562c1ad6ac00 msgr2=0x562c1ad6b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.303+0000 7f3aa3773740 1 -- 172.21.15.47:0/2157935398 shutdown_connections 2024-03-21T11:35:44.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.303+0000 7f3aa3773740 1 -- 172.21.15.47:0/2157935398 wait complete. 2024-03-21T11:35:44.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.303+0000 7f3a99e00640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a800a2f20 0x7f3a800a53e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c000c00 tx=0x7f3a8c0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.304+0000 7f3aa3773740 1 -- start start 2024-03-21T11:35:44.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.304+0000 7f3aa3773740 1 Processor -- start 2024-03-21T11:35:44.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.304+0000 7f3aa3773740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad6f810 0x562c1ad79ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.304+0000 7f3aa3773740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad7a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3aa3773740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad713a0 0x562c1ad80ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3aa3773740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1ad783b0 con 0x562c1ad709e0 2024-03-21T11:35:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3aa3773740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1ac900d0 con 0x562c1ad713a0 2024-03-21T11:35:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3aa3773740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1ad78530 con 0x562c1ad6f810 2024-03-21T11:35:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3aa3773740 10 client.0 _ll_register_callbacks cb 0x562c1ac94e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3a9a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad6f810 0x562c1ad79ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3a9a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad6f810 0x562c1ad79ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41594/0 (socket says 172.21.15.47:41594) 2024-03-21T11:35:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3a9a800640 1 -- 172.21.15.47:0/270162126 learned_addr learned my addr 172.21.15.47:0/270162126 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3a9a800640 1 --2- 172.21.15.47:0/270162126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad713a0 0x562c1ad80ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3a99400640 1 --2- 172.21.15.47:0/270162126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad7a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3a9a800640 1 -- 172.21.15.47:0/270162126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad713a0 msgr2=0x562c1ad80ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3a9a800640 1 --2- 172.21.15.47:0/270162126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad713a0 0x562c1ad80ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3a9a800640 1 -- 172.21.15.47:0/270162126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 msgr2=0x562c1ad7a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3a9a800640 1 --2- 172.21.15.47:0/270162126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad7a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.305+0000 7f3a9a800640 1 -- 172.21.15.47:0/270162126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c1a9e83f0 con 0x562c1ad6f810 2024-03-21T11:35:44.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.306+0000 7f3a9a800640 1 --2- 172.21.15.47:0/270162126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad6f810 0x562c1ad79ae0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009df0 tx=0x7f3a94013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10ff04f769426921 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.306+0000 7f3a7b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:44.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.306+0000 7f3a7b400640 1 -- 172.21.15.47:0/270162126 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9402f060 con 0x562c1ad6f810 2024-03-21T11:35:44.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.306+0000 7f3a7b400640 1 -- 172.21.15.47:0/270162126 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3a940119d0 con 0x562c1ad6f810 2024-03-21T11:35:44.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.306+0000 7f3aa3773740 1 -- 172.21.15.47:0/270162126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a94011de0 con 0x562c1ad6f810 2024-03-21T11:35:44.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.306+0000 7f3a7b400640 1 -- 172.21.15.47:0/270162126 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a94011bd0 con 0x562c1ad6f810 2024-03-21T11:35:44.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.306+0000 7f3aa3773740 10 client.29707 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:44.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.306+0000 7f3aa3773740 1 -- 172.21.15.47:0/270162126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562c1aab27d0 con 0x562c1ad6f810 2024-03-21T11:35:44.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.307+0000 7f3a7b400640 1 -- 172.21.15.47:0/270162126 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a94011de0 con 0x562c1ad6f810 2024-03-21T11:35:44.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.307+0000 7f3a7b400640 1 -- 172.21.15.47:0/270162126 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3a94029040 con 0x562c1ad6f810 2024-03-21T11:35:44.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.307+0000 7f3aa3773740 10 client.29707 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:44.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.307+0000 7f3aa3773740 10 client.29707 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:35:44.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.307+0000 7f3aa3773740 20 client.29707 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.307+0000 7f3aa3773740 1 --2- 172.21.15.47:0/270162126 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562c1ad85120 0x562c1ad87510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.307+0000 7f3aa3773740 4 client.29707 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.307+0000 7f3aa3773740 1 -- 172.21.15.47:0/270162126 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c1ace1800 con 0x562c1ad85120 2024-03-21T11:35:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.308+0000 7f3a99400640 1 --2- 172.21.15.47:0/270162126 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562c1ad85120 0x562c1ad87510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.309+0000 7f3a99400640 1 --2- 172.21.15.47:0/270162126 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562c1ad85120 0x562c1ad87510 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54d809ce13789d2c server_cookie=69214203e866d53d in_seq=0 out_seq=0 2024-03-21T11:35:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.309+0000 7f3a7b400640 10 client.29707 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:35:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.309+0000 7f3a7b400640 1 -- 172.21.15.47:0/270162126 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562c1ace1800 con 0x562c1ad85120 2024-03-21T11:35:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.309+0000 7f3a7b400640 10 client.29707 handle_command_reply: tid=0 2024-03-21T11:35:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.309+0000 7f3aa3773740 1 client.29707 shutdown 2024-03-21T11:35:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.310+0000 7f3aa3773740 1 -- 172.21.15.47:0/270162126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad6f810 msgr2=0x562c1ad79ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.310+0000 7f3aa3773740 1 --2- 172.21.15.47:0/270162126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad6f810 0x562c1ad79ae0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009df0 tx=0x7f3a94013040 comp rx=0 tx=0).stop 2024-03-21T11:35:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.310+0000 7f3aa3773740 1 -- 172.21.15.47:0/270162126 shutdown_connections 2024-03-21T11:35:44.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.310+0000 7f3aa3773740 1 --2- 172.21.15.47:0/270162126 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562c1ad85120 0x562c1ad87510 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.310+0000 7f3aa3773740 1 --2- 172.21.15.47:0/270162126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad713a0 0x562c1ad80ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.310+0000 7f3aa3773740 1 --2- 172.21.15.47:0/270162126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad7a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.310+0000 7f3aa3773740 1 --2- 172.21.15.47:0/270162126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad6f810 0x562c1ad79ae0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.310+0000 7f3aa3773740 1 -- 172.21.15.47:0/270162126 >> 172.21.15.47:0/270162126 conn(0x562c1ad6ac00 msgr2=0x562c1ad69eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.310+0000 7f3a7b400640 0 client.29707 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:35:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.310+0000 7f3aa3773740 1 -- 172.21.15.47:0/270162126 shutdown_connections 2024-03-21T11:35:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.310+0000 7f3aa3773740 1 -- 172.21.15.47:0/270162126 wait complete. 2024-03-21T11:35:44.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.310+0000 7f3aa3773740 20 client.29707 trim_cache size 0 max 16384 2024-03-21T11:35:44.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.327+0000 7f3aa3773740 1 Processor -- start 2024-03-21T11:35:44.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.327+0000 7f3aa3773740 1 -- start start 2024-03-21T11:35:44.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.327+0000 7f3aa3773740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad6f810 0x562c1ad79d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.327+0000 7f3aa3773740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad80fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.327+0000 7f3aa3773740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad713a0 0x562c1ad8bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.327+0000 7f3aa3773740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c1ac310 con 0x562c1ad709e0 2024-03-21T11:35:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.327+0000 7f3aa3773740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c1adc90 con 0x562c1ad713a0 2024-03-21T11:35:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.327+0000 7f3aa3773740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c1ac970 con 0x562c1ad6f810 2024-03-21T11:35:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.328+0000 7f3a9a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad6f810 0x562c1ad79d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.328+0000 7f3a9a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad6f810 0x562c1ad79d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41608/0 (socket says 172.21.15.47:41608) 2024-03-21T11:35:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.328+0000 7f3a9a800640 1 -- 172.21.15.47:0/1514130471 learned_addr learned my addr 172.21.15.47:0/1514130471 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.328+0000 7f3a9a800640 1 --2- 172.21.15.47:0/1514130471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad713a0 0x562c1ad8bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.328+0000 7f3a99400640 1 --2- 172.21.15.47:0/1514130471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad80fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.328+0000 7f3a9a800640 1 -- 172.21.15.47:0/1514130471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad713a0 msgr2=0x562c1ad8bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.328+0000 7f3a9a800640 1 --2- 172.21.15.47:0/1514130471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad713a0 0x562c1ad8bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.328+0000 7f3a9a800640 1 -- 172.21.15.47:0/1514130471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 msgr2=0x562c1ad80fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.328+0000 7f3a9a800640 1 --2- 172.21.15.47:0/1514130471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad80fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.328+0000 7f3a9a800640 1 -- 172.21.15.47:0/1514130471 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c1ad7a2a0 con 0x562c1ad6f810 2024-03-21T11:35:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.328+0000 7f3a9a800640 1 --2- 172.21.15.47:0/1514130471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad6f810 0x562c1ad79d60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3a9401b810 tx=0x7f3a940108e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=979b76a3e0a89e76 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.329+0000 7f3a86600640 1 -- 172.21.15.47:0/1514130471 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9402f070 con 0x562c1ad6f810 2024-03-21T11:35:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.329+0000 7f3a86600640 1 -- 172.21.15.47:0/1514130471 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a94010e90 con 0x562c1ad6f810 2024-03-21T11:35:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.329+0000 7f3a86600640 1 -- 172.21.15.47:0/1514130471 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a94039080 con 0x562c1ad6f810 2024-03-21T11:35:44.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.330+0000 7f3aa3773740 1 -- 172.21.15.47:0/1514130471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad6f810 msgr2=0x562c1ad79d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.330+0000 7f3aa3773740 1 --2- 172.21.15.47:0/1514130471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad6f810 0x562c1ad79d60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3a9401b810 tx=0x7f3a940108e0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.330+0000 7f3aa3773740 1 -- 172.21.15.47:0/1514130471 shutdown_connections 2024-03-21T11:35:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.330+0000 7f3aa3773740 1 --2- 172.21.15.47:0/1514130471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad713a0 0x562c1ad8bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.330+0000 7f3aa3773740 1 --2- 172.21.15.47:0/1514130471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad80fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.330+0000 7f3aa3773740 1 --2- 172.21.15.47:0/1514130471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad6f810 0x562c1ad79d60 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.330+0000 7f3aa3773740 1 -- 172.21.15.47:0/1514130471 >> 172.21.15.47:0/1514130471 conn(0x562c1ad6ac00 msgr2=0x562c1ad6e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.330+0000 7f3aa3773740 1 -- 172.21.15.47:0/1514130471 shutdown_connections 2024-03-21T11:35:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.330+0000 7f3aa3773740 1 -- 172.21.15.47:0/1514130471 wait complete. 2024-03-21T11:35:44.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.330+0000 7f3aa3773740 1 -- start start 2024-03-21T11:35:44.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.331+0000 7f3aa3773740 1 Processor -- start 2024-03-21T11:35:44.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.331+0000 7f3aa3773740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad6f810 0x562c1ad7c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.331+0000 7f3aa3773740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad7ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.331+0000 7f3aa3773740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad713a0 0x562c1ad7fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.331+0000 7f3aa3773740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c1ad630 con 0x562c1ad709e0 2024-03-21T11:35:44.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.331+0000 7f3aa3773740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c1acfd0 con 0x562c1ad6f810 2024-03-21T11:35:44.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.331+0000 7f3aa3773740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c1aa4d0 con 0x562c1ad713a0 2024-03-21T11:35:44.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.331+0000 7f3aa3773740 10 client.0 _ll_register_callbacks cb 0x7f3a940093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:44.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.331+0000 7f3a9a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad6f810 0x562c1ad7c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.331+0000 7f3a9a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad6f810 0x562c1ad7c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49754/0 (socket says 172.21.15.47:49754) 2024-03-21T11:35:44.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.331+0000 7f3a99400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad7ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.331+0000 7f3a9a800640 1 -- 172.21.15.47:0/2353717595 learned_addr learned my addr 172.21.15.47:0/2353717595 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.331+0000 7f3a9a800640 1 --2- 172.21.15.47:0/2353717595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad713a0 0x562c1ad7fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.332+0000 7f3a9a800640 1 -- 172.21.15.47:0/2353717595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad713a0 msgr2=0x562c1ad7fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.332+0000 7f3a9a800640 1 --2- 172.21.15.47:0/2353717595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad713a0 0x562c1ad7fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.332+0000 7f3a9a800640 1 -- 172.21.15.47:0/2353717595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 msgr2=0x562c1ad7ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.332+0000 7f3a9a800640 1 --2- 172.21.15.47:0/2353717595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad7ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.332+0000 7f3a9a800640 1 -- 172.21.15.47:0/2353717595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c1ad80420 con 0x562c1ad6f810 2024-03-21T11:35:44.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.332+0000 7f3a99400640 1 --2- 172.21.15.47:0/2353717595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad7ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:44.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.332+0000 7f3a9a800640 1 --2- 172.21.15.47:0/2353717595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad6f810 0x562c1ad7c8d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f3a94011790 tx=0x7f3a94036040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c573cb275471094c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.332+0000 7f3a7b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:44.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.332+0000 7f3a7b400640 1 -- 172.21.15.47:0/2353717595 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a94039020 con 0x562c1ad6f810 2024-03-21T11:35:44.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.332+0000 7f3aa3773740 1 -- 172.21.15.47:0/2353717595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a94010e90 con 0x562c1ad6f810 2024-03-21T11:35:44.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.332+0000 7f3a7b400640 1 -- 172.21.15.47:0/2353717595 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3a94004a00 con 0x562c1ad6f810 2024-03-21T11:35:44.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.333+0000 7f3a7b400640 1 -- 172.21.15.47:0/2353717595 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a9402f040 con 0x562c1ad6f810 2024-03-21T11:35:44.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.333+0000 7f3aa3773740 10 client.29675 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:44.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.333+0000 7f3aa3773740 1 -- 172.21.15.47:0/2353717595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562c1ade0b20 con 0x562c1ad6f810 2024-03-21T11:35:44.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.333+0000 7f3a7b400640 1 -- 172.21.15.47:0/2353717595 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a94010e90 con 0x562c1ad6f810 2024-03-21T11:35:44.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.333+0000 7f3a7b400640 1 -- 172.21.15.47:0/2353717595 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3a94005000 con 0x562c1ad6f810 2024-03-21T11:35:44.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.333+0000 7f3aa3773740 10 client.29675 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:44.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.333+0000 7f3aa3773740 10 client.29675 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:35:44.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.333+0000 7f3aa3773740 20 client.29675 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:44.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.333+0000 7f3aa3773740 1 --2- 172.21.15.47:0/2353717595 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562c1ade1350 0x562c1ade3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.333+0000 7f3aa3773740 4 client.29675 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:35:44.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.333+0000 7f3aa3773740 1 -- 172.21.15.47:0/2353717595 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562c1acdbb50 con 0x562c1ade1350 2024-03-21T11:35:44.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.333+0000 7f3a99400640 1 --2- 172.21.15.47:0/2353717595 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562c1ade1350 0x562c1ade3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.334+0000 7f3a99400640 1 --2- 172.21.15.47:0/2353717595 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562c1ade1350 0x562c1ade3770 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c86a197839b722c0 server_cookie=e7a2a2842e3045e4 in_seq=0 out_seq=0 2024-03-21T11:35:44.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.334+0000 7f3a7b400640 10 client.29675 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:35:44.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.334+0000 7f3a7b400640 1 -- 172.21.15.47:0/2353717595 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562c1acdbb50 con 0x562c1ade1350 2024-03-21T11:35:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.334+0000 7f3a7b400640 10 client.29675 handle_command_reply: tid=0 2024-03-21T11:35:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.334+0000 7f3aa3773740 1 client.29675 shutdown 2024-03-21T11:35:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.334+0000 7f3aa3773740 1 -- 172.21.15.47:0/2353717595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad6f810 msgr2=0x562c1ad7c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.334+0000 7f3aa3773740 1 --2- 172.21.15.47:0/2353717595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad6f810 0x562c1ad7c8d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f3a94011790 tx=0x7f3a94036040 comp rx=0 tx=0).stop 2024-03-21T11:35:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.334+0000 7f3aa3773740 1 -- 172.21.15.47:0/2353717595 shutdown_connections 2024-03-21T11:35:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.334+0000 7f3aa3773740 1 --2- 172.21.15.47:0/2353717595 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562c1ade1350 0x562c1ade3770 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.334+0000 7f3aa3773740 1 --2- 172.21.15.47:0/2353717595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c1ad713a0 0x562c1ad7fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.334+0000 7f3aa3773740 1 --2- 172.21.15.47:0/2353717595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562c1ad709e0 0x562c1ad7ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.334+0000 7f3a7b400640 0 client.29675 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:35:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.334+0000 7f3aa3773740 1 --2- 172.21.15.47:0/2353717595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562c1ad6f810 0x562c1ad7c8d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.334+0000 7f3aa3773740 1 -- 172.21.15.47:0/2353717595 >> 172.21.15.47:0/2353717595 conn(0x562c1ad6ac00 msgr2=0x562c1ad73c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.335+0000 7f3aa3773740 1 -- 172.21.15.47:0/2353717595 shutdown_connections 2024-03-21T11:35:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.335+0000 7f3aa3773740 1 -- 172.21.15.47:0/2353717595 wait complete. 2024-03-21T11:35:44.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.335+0000 7f3aa3773740 20 client.29675 trim_cache size 0 max 16384 2024-03-21T11:35:44.340 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:35:44.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.335+0000 7f3aa0a00640 1 -- 172.21.15.47:0/3829258751 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a800a2f20 msgr2=0x7f3a800a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.335+0000 7f3aa0a00640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a800a2f20 0x7f3a800a53e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c000c00 tx=0x7f3a8c0046c0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.335+0000 7f3aa0a00640 1 -- 172.21.15.47:0/3829258751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9c1038e0 msgr2=0x7f3a9c199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.335+0000 7f3aa0a00640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9c1038e0 0x7f3a9c199bb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00e9c0 tx=0x7f3a8c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:35:44.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.336+0000 7f3aa0a00640 1 -- 172.21.15.47:0/3829258751 shutdown_connections 2024-03-21T11:35:44.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.336+0000 7f3aa0a00640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a800a2f20 0x7f3a800a53e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.336+0000 7f3aa0a00640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c105490 0x7f3a9c1a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.336+0000 7f3aa0a00640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a9c104ad0 0x7f3a9c19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.336+0000 7f3aa0a00640 1 --2- 172.21.15.47:0/3829258751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9c1038e0 0x7f3a9c199bb0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.336+0000 7f3aa0a00640 1 -- 172.21.15.47:0/3829258751 >> 172.21.15.47:0/3829258751 conn(0x7f3a9c0fee50 msgr2=0x7f3a9c107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.336+0000 7f3aa0a00640 1 -- 172.21.15.47:0/3829258751 shutdown_connections 2024-03-21T11:35:44.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.336+0000 7f3aa0a00640 1 -- 172.21.15.47:0/3829258751 wait complete. 2024-03-21T11:35:44.355 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:35:44.355 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:35:44.571 INFO:tasks.workunit.client.0.smithi047.stdout: 1 941011 7.98 MB/sec execute 508 sec latency 27.605 ms 2024-03-21T11:35:44.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.579+0000 7f80d2200640 1 -- 172.21.15.47:0/1732404611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80cc1037a0 msgr2=0x7f80cc103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.579+0000 7f80d2200640 1 --2- 172.21.15.47:0/1732404611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80cc1037a0 0x7f80cc103bc0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009900 tx=0x7f80c002bc30 comp rx=0 tx=0).stop 2024-03-21T11:35:44.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.579+0000 7f80d2200640 1 -- 172.21.15.47:0/1732404611 shutdown_connections 2024-03-21T11:35:44.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.579+0000 7f80d2200640 1 --2- 172.21.15.47:0/1732404611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80cc105350 0x7f80cc10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.579+0000 7f80d2200640 1 --2- 172.21.15.47:0/1732404611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80cc104990 0x7f80cc104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.579+0000 7f80d2200640 1 --2- 172.21.15.47:0/1732404611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80cc1037a0 0x7f80cc103bc0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.579+0000 7f80d2200640 1 -- 172.21.15.47:0/1732404611 >> 172.21.15.47:0/1732404611 conn(0x7f80cc0fed10 msgr2=0x7f80cc101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.579+0000 7f80d2200640 1 -- 172.21.15.47:0/1732404611 shutdown_connections 2024-03-21T11:35:44.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.579+0000 7f80d2200640 1 -- 172.21.15.47:0/1732404611 wait complete. 2024-03-21T11:35:44.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.580+0000 7f80d2200640 1 Processor -- start 2024-03-21T11:35:44.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.580+0000 7f80d2200640 1 -- start start 2024-03-21T11:35:44.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.581+0000 7f80d2200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80cc1037a0 0x7f80cc199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.581+0000 7f80d2200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80cc104990 0x7f80cc199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.581+0000 7f80d2200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80cc105350 0x7f80cc1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.581+0000 7f80d2200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc10f470 con 0x7f80cc104990 2024-03-21T11:35:44.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.581+0000 7f80d2200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc10f2f0 con 0x7f80cc105350 2024-03-21T11:35:44.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.581+0000 7f80d2200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc10f5f0 con 0x7f80cc1037a0 2024-03-21T11:35:44.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.581+0000 7f80d0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80cc1037a0 0x7f80cc199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.581+0000 7f80d0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80cc1037a0 0x7f80cc199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41626/0 (socket says 172.21.15.47:41626) 2024-03-21T11:35:44.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.581+0000 7f80d0e00640 1 -- 172.21.15.47:0/3673835532 learned_addr learned my addr 172.21.15.47:0/3673835532 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.581+0000 7f80d1800640 1 --2- 172.21.15.47:0/3673835532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80cc105350 0x7f80cc1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.581+0000 7f80cbe00640 1 --2- 172.21.15.47:0/3673835532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80cc104990 0x7f80cc199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.582+0000 7f80d0e00640 1 -- 172.21.15.47:0/3673835532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80cc105350 msgr2=0x7f80cc1a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.582+0000 7f80d0e00640 1 --2- 172.21.15.47:0/3673835532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80cc105350 0x7f80cc1a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.582+0000 7f80d0e00640 1 -- 172.21.15.47:0/3673835532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80cc104990 msgr2=0x7f80cc199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.582+0000 7f80d0e00640 1 --2- 172.21.15.47:0/3673835532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80cc104990 0x7f80cc199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.582+0000 7f80d0e00640 1 -- 172.21.15.47:0/3673835532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc1a1640 con 0x7f80cc1037a0 2024-03-21T11:35:44.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.582+0000 7f80d0e00640 1 --2- 172.21.15.47:0/3673835532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80cc1037a0 0x7f80cc199a20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f80c0000c00 tx=0x7f80c002c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.582+0000 7f80c9600640 1 -- 172.21.15.47:0/3673835532 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c0043020 con 0x7f80cc1037a0 2024-03-21T11:35:44.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.582+0000 7f80c9600640 1 -- 172.21.15.47:0/3673835532 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80c00054e0 con 0x7f80cc1037a0 2024-03-21T11:35:44.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.583+0000 7f80c9600640 1 -- 172.21.15.47:0/3673835532 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c0003ce0 con 0x7f80cc1037a0 2024-03-21T11:35:44.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.583+0000 7f80d2200640 1 -- 172.21.15.47:0/3673835532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc1a18d0 con 0x7f80cc1037a0 2024-03-21T11:35:44.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.583+0000 7f80d2200640 1 -- 172.21.15.47:0/3673835532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc1a1e40 con 0x7f80cc1037a0 2024-03-21T11:35:44.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.586+0000 7f80c9600640 1 -- 172.21.15.47:0/3673835532 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f80c0046020 con 0x7f80cc1037a0 2024-03-21T11:35:44.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.587+0000 7f80d65c1740 1 Processor -- start 2024-03-21T11:35:44.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.587+0000 7f80d65c1740 1 -- start start 2024-03-21T11:35:44.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.587+0000 7f80d65c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf38669f0 0x555bf3866e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.587+0000 7f80d65c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 0x555bf3868040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.587+0000 7f80d65c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf3868580 0x555bf386ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.587+0000 7f80d65c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf37cb2c0 con 0x555bf3867bc0 2024-03-21T11:35:44.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.587+0000 7f80d65c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf3773c90 con 0x555bf3868580 2024-03-21T11:35:44.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.587+0000 7f80d65c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf37c7360 con 0x555bf38669f0 2024-03-21T11:35:44.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.587+0000 7f80d1800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf38669f0 0x555bf3866e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.587+0000 7f80d1800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf38669f0 0x555bf3866e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41636/0 (socket says 172.21.15.47:41636) 2024-03-21T11:35:44.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.587+0000 7f80d0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 0x555bf3868040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.587+0000 7f80d1800640 1 -- 172.21.15.47:0/1598031912 learned_addr learned my addr 172.21.15.47:0/1598031912 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.587+0000 7f80cbe00640 1 --2- 172.21.15.47:0/1598031912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf3868580 0x555bf386ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.588+0000 7f80d1800640 1 -- 172.21.15.47:0/1598031912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf3868580 msgr2=0x555bf386ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.588+0000 7f80d1800640 1 --2- 172.21.15.47:0/1598031912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf3868580 0x555bf386ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.588+0000 7f80d1800640 1 -- 172.21.15.47:0/1598031912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 msgr2=0x555bf3868040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.588+0000 7f80d1800640 1 --2- 172.21.15.47:0/1598031912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 0x555bf3868040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.588+0000 7f80d1800640 1 -- 172.21.15.47:0/1598031912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bf3798080 con 0x555bf38669f0 2024-03-21T11:35:44.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.588+0000 7f80c9600640 1 --2- 172.21.15.47:0/3673835532 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80b40a2fa0 0x7f80b40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.588+0000 7f80c9600640 1 -- 172.21.15.47:0/3673835532 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f80c00f18d0 con 0x7f80cc1037a0 2024-03-21T11:35:44.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.588+0000 7f80d1800640 1 --2- 172.21.15.47:0/1598031912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf38669f0 0x555bf3866e10 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009fd0 tx=0x7f80c40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a024fec139230bb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.589+0000 7f80be000640 1 -- 172.21.15.47:0/1598031912 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4004d20 con 0x555bf38669f0 2024-03-21T11:35:44.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.589+0000 7f80d0e00640 1 --2- 172.21.15.47:0/3673835532 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80b40a2fa0 0x7f80b40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.589+0000 7f80be000640 1 -- 172.21.15.47:0/1598031912 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80c4004050 con 0x555bf38669f0 2024-03-21T11:35:44.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.589+0000 7f80be000640 1 -- 172.21.15.47:0/1598031912 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4005260 con 0x555bf38669f0 2024-03-21T11:35:44.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.590+0000 7f80d65c1740 1 -- 172.21.15.47:0/1598031912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf38669f0 msgr2=0x555bf3866e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.590+0000 7f80d0e00640 1 --2- 172.21.15.47:0/3673835532 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80b40a2fa0 0x7f80b40a5460 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f80c00c6210 tx=0x7f80c00f7530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.590+0000 7f80d65c1740 1 --2- 172.21.15.47:0/1598031912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf38669f0 0x555bf3866e10 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009fd0 tx=0x7f80c40118d0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.590+0000 7f80d65c1740 1 -- 172.21.15.47:0/1598031912 shutdown_connections 2024-03-21T11:35:44.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.590+0000 7f80d65c1740 1 --2- 172.21.15.47:0/1598031912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf3868580 0x555bf386ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.590+0000 7f80d65c1740 1 --2- 172.21.15.47:0/1598031912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 0x555bf3868040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.590+0000 7f80d65c1740 1 --2- 172.21.15.47:0/1598031912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf38669f0 0x555bf3866e10 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.590+0000 7f80d65c1740 1 -- 172.21.15.47:0/1598031912 >> 172.21.15.47:0/1598031912 conn(0x555bf3861de0 msgr2=0x555bf38621e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.590+0000 7f80d65c1740 1 -- 172.21.15.47:0/1598031912 shutdown_connections 2024-03-21T11:35:44.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.590+0000 7f80d65c1740 1 -- 172.21.15.47:0/1598031912 wait complete. 2024-03-21T11:35:44.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.591+0000 7f80d65c1740 1 -- start start 2024-03-21T11:35:44.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.591+0000 7f80d65c1740 1 Processor -- start 2024-03-21T11:35:44.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80d65c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf38669f0 0x555bf3870c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80d65c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3867bc0 0x555bf38711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80d65c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf3868580 0x555bf38781e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80d65c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf386f590 con 0x555bf38669f0 2024-03-21T11:35:44.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80d65c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf37870d0 con 0x555bf3868580 2024-03-21T11:35:44.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80d65c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf386f710 con 0x555bf3867bc0 2024-03-21T11:35:44.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80d65c1740 10 client.0 _ll_register_callbacks cb 0x555bf378be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:44.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80cbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3867bc0 0x555bf38711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80cbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3867bc0 0x555bf38711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41638/0 (socket says 172.21.15.47:41638) 2024-03-21T11:35:44.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80d1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf38669f0 0x555bf3870c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80cbe00640 1 -- 172.21.15.47:0/694401835 learned_addr learned my addr 172.21.15.47:0/694401835 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80d1800640 1 --2- 172.21.15.47:0/694401835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf3868580 0x555bf38781e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80cbe00640 1 -- 172.21.15.47:0/694401835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf3868580 msgr2=0x555bf38781e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80cbe00640 1 --2- 172.21.15.47:0/694401835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf3868580 0x555bf38781e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80cbe00640 1 -- 172.21.15.47:0/694401835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf38669f0 msgr2=0x555bf3870c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80cbe00640 1 --2- 172.21.15.47:0/694401835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf38669f0 0x555bf3870c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80cbe00640 1 -- 172.21.15.47:0/694401835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bf34df3f0 con 0x555bf3867bc0 2024-03-21T11:35:44.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.592+0000 7f80d1800640 1 --2- 172.21.15.47:0/694401835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf3868580 0x555bf38781e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.593+0000 7f80cbe00640 1 --2- 172.21.15.47:0/694401835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3867bc0 0x555bf38711a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f80b8004650 tx=0x7f80b80039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33e139933f93a34 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.593+0000 7f80b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.593+0000 7f80b2a00640 1 -- 172.21.15.47:0/694401835 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8011910 con 0x555bf3867bc0 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.593+0000 7f80b2a00640 1 -- 172.21.15.47:0/694401835 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f80b8011ab0 con 0x555bf3867bc0 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.593+0000 7f80d65c1740 1 -- 172.21.15.47:0/694401835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80c4004050 con 0x555bf3867bc0 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.593+0000 7f80b2a00640 1 -- 172.21.15.47:0/694401835 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80b8011cb0 con 0x555bf3867bc0 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.593+0000 7f80d65c1740 10 client.29725 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.593+0000 7f80d65c1740 1 -- 172.21.15.47:0/694401835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555bf35a97d0 con 0x555bf3867bc0 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.593+0000 7f80b2a00640 1 -- 172.21.15.47:0/694401835 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80c4004050 con 0x555bf3867bc0 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.594+0000 7f80b2a00640 1 -- 172.21.15.47:0/694401835 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f80b8018070 con 0x555bf3867bc0 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.594+0000 7f80d65c1740 10 client.29725 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.594+0000 7f80d65c1740 10 client.29725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.594+0000 7f80d65c1740 20 client.29725 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.594+0000 7f80d65c1740 1 --2- 172.21.15.47:0/694401835 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555bf387c7f0 0x555bf387ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.594+0000 7f80d65c1740 4 client.29725 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:44.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.594+0000 7f80d65c1740 1 -- 172.21.15.47:0/694401835 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bf37d8800 con 0x555bf387c7f0 2024-03-21T11:35:44.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.594+0000 7f80d1800640 1 --2- 172.21.15.47:0/694401835 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555bf387c7f0 0x555bf387ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.595+0000 7f80d1800640 1 --2- 172.21.15.47:0/694401835 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555bf387c7f0 0x555bf387ec50 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1a796013c37366b server_cookie=c81af09d20ed4221 in_seq=0 out_seq=0 2024-03-21T11:35:44.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.595+0000 7f80b2a00640 10 client.29725 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:44.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.596+0000 7f80b2a00640 1 -- 172.21.15.47:0/694401835 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555bf37d8800 con 0x555bf387c7f0 2024-03-21T11:35:44.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.596+0000 7f80b2a00640 10 client.29725 handle_command_reply: tid=0 2024-03-21T11:35:44.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.596+0000 7f80d65c1740 1 client.29725 shutdown 2024-03-21T11:35:44.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.596+0000 7f80d65c1740 1 -- 172.21.15.47:0/694401835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3867bc0 msgr2=0x555bf38711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.596+0000 7f80d65c1740 1 --2- 172.21.15.47:0/694401835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3867bc0 0x555bf38711a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f80b8004650 tx=0x7f80b80039c0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.596+0000 7f80d65c1740 1 -- 172.21.15.47:0/694401835 shutdown_connections 2024-03-21T11:35:44.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.596+0000 7f80d65c1740 1 --2- 172.21.15.47:0/694401835 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555bf387c7f0 0x555bf387ec50 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.596+0000 7f80d65c1740 1 --2- 172.21.15.47:0/694401835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf3868580 0x555bf38781e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.597+0000 7f80d65c1740 1 --2- 172.21.15.47:0/694401835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3867bc0 0x555bf38711a0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.597+0000 7f80d65c1740 1 --2- 172.21.15.47:0/694401835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf38669f0 0x555bf3870c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.597+0000 7f80d65c1740 1 -- 172.21.15.47:0/694401835 >> 172.21.15.47:0/694401835 conn(0x555bf3861de0 msgr2=0x555bf3861160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.597+0000 7f80b2a00640 0 client.29725 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:44.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.597+0000 7f80d65c1740 1 -- 172.21.15.47:0/694401835 shutdown_connections 2024-03-21T11:35:44.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.597+0000 7f80d65c1740 1 -- 172.21.15.47:0/694401835 wait complete. 2024-03-21T11:35:44.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.597+0000 7f80d65c1740 20 client.29725 trim_cache size 0 max 16384 2024-03-21T11:35:44.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.613+0000 7f80d65c1740 1 Processor -- start 2024-03-21T11:35:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.613+0000 7f80d65c1740 1 -- start start 2024-03-21T11:35:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.613+0000 7f80d65c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf38669f0 0x555bf38658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.613+0000 7f80d65c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 0x555bf38781a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.613+0000 7f80d65c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3868580 0x555bf386ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.613+0000 7f80d65c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc1abff0 con 0x555bf3867bc0 2024-03-21T11:35:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.613+0000 7f80d65c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc1ad970 con 0x555bf38669f0 2024-03-21T11:35:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.613+0000 7f80d65c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc1ac650 con 0x555bf3868580 2024-03-21T11:35:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.613+0000 7f80d1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf38669f0 0x555bf38658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.613+0000 7f80d1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf38669f0 0x555bf38658d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49790/0 (socket says 172.21.15.47:49790) 2024-03-21T11:35:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.613+0000 7f80d1800640 1 -- 172.21.15.47:0/1577037624 learned_addr learned my addr 172.21.15.47:0/1577037624 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.613+0000 7f80d1800640 1 --2- 172.21.15.47:0/1577037624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3868580 0x555bf386ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.613+0000 7f80cbe00640 1 --2- 172.21.15.47:0/1577037624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 0x555bf38781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.614+0000 7f80d1800640 1 -- 172.21.15.47:0/1577037624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3868580 msgr2=0x555bf386ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.614+0000 7f80d1800640 1 --2- 172.21.15.47:0/1577037624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3868580 0x555bf386ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.614+0000 7f80d1800640 1 -- 172.21.15.47:0/1577037624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 msgr2=0x555bf38781a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.614+0000 7f80d1800640 1 --2- 172.21.15.47:0/1577037624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 0x555bf38781a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.614+0000 7f80d1800640 1 -- 172.21.15.47:0/1577037624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bf386e750 con 0x555bf38669f0 2024-03-21T11:35:44.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.614+0000 7f80d1800640 1 --2- 172.21.15.47:0/1577037624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf38669f0 0x555bf38658d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f80c4014040 tx=0x7f80c4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22a5e205e69ccced server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.614+0000 7f80bd600640 1 -- 172.21.15.47:0/1577037624 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4003a20 con 0x555bf38669f0 2024-03-21T11:35:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.614+0000 7f80bd600640 1 -- 172.21.15.47:0/1577037624 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80c402cd90 con 0x555bf38669f0 2024-03-21T11:35:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.614+0000 7f80bd600640 1 -- 172.21.15.47:0/1577037624 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4004580 con 0x555bf38669f0 2024-03-21T11:35:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.615+0000 7f80d65c1740 1 -- 172.21.15.47:0/1577037624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf38669f0 msgr2=0x555bf38658d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.615+0000 7f80d65c1740 1 --2- 172.21.15.47:0/1577037624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf38669f0 0x555bf38658d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f80c4014040 tx=0x7f80c4005fd0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.615+0000 7f80d65c1740 1 -- 172.21.15.47:0/1577037624 shutdown_connections 2024-03-21T11:35:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.615+0000 7f80d65c1740 1 --2- 172.21.15.47:0/1577037624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3868580 0x555bf386ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.615+0000 7f80d65c1740 1 --2- 172.21.15.47:0/1577037624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 0x555bf38781a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.615+0000 7f80d65c1740 1 --2- 172.21.15.47:0/1577037624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf38669f0 0x555bf38658d0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.615+0000 7f80d65c1740 1 -- 172.21.15.47:0/1577037624 >> 172.21.15.47:0/1577037624 conn(0x555bf3861de0 msgr2=0x555bf3870120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.615+0000 7f80d65c1740 1 -- 172.21.15.47:0/1577037624 shutdown_connections 2024-03-21T11:35:44.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.615+0000 7f80d65c1740 1 -- 172.21.15.47:0/1577037624 wait complete. 2024-03-21T11:35:44.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.616+0000 7f80d65c1740 1 -- start start 2024-03-21T11:35:44.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.616+0000 7f80d65c1740 1 Processor -- start 2024-03-21T11:35:44.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.616+0000 7f80d65c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf38669f0 0x555bf3874210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80d65c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 0x555bf3874750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80d65c1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3868580 0x555bf3877770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80d65c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc1ad310 con 0x555bf3867bc0 2024-03-21T11:35:44.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80d65c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc1accb0 con 0x555bf38669f0 2024-03-21T11:35:44.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80d65c1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc1aa1b0 con 0x555bf3868580 2024-03-21T11:35:44.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80d65c1740 10 client.0 _ll_register_callbacks cb 0x7f80b8004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:44.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80d1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf38669f0 0x555bf3874210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80d1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf38669f0 0x555bf3874210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49802/0 (socket says 172.21.15.47:49802) 2024-03-21T11:35:44.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80d1800640 1 -- 172.21.15.47:0/1062182742 learned_addr learned my addr 172.21.15.47:0/1062182742 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:44.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80d1800640 1 --2- 172.21.15.47:0/1062182742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3868580 0x555bf3877770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80d1800640 1 -- 172.21.15.47:0/1062182742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3868580 msgr2=0x555bf3877770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80d1800640 1 --2- 172.21.15.47:0/1062182742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3868580 0x555bf3877770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80cbe00640 1 --2- 172.21.15.47:0/1062182742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 0x555bf3874750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80d1800640 1 -- 172.21.15.47:0/1062182742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 msgr2=0x555bf3874750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:44.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.617+0000 7f80d1800640 1 --2- 172.21.15.47:0/1062182742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 0x555bf3874750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.618+0000 7f80d1800640 1 -- 172.21.15.47:0/1062182742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bf3877d60 con 0x555bf38669f0 2024-03-21T11:35:44.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.618+0000 7f80cbe00640 1 --2- 172.21.15.47:0/1062182742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 0x555bf3874750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:35:44.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.618+0000 7f80d1800640 1 --2- 172.21.15.47:0/1062182742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf38669f0 0x555bf3874210 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009560 tx=0x7f80c4004050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86c32142ce6efa7f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:44.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.618+0000 7f80b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:44.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.618+0000 7f80b2a00640 1 -- 172.21.15.47:0/1062182742 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c402c2d0 con 0x555bf38669f0 2024-03-21T11:35:44.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.618+0000 7f80b2a00640 1 -- 172.21.15.47:0/1062182742 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f80c402c8b0 con 0x555bf38669f0 2024-03-21T11:35:44.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.618+0000 7f80d65c1740 1 -- 172.21.15.47:0/1062182742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80c402cd90 con 0x555bf38669f0 2024-03-21T11:35:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.619+0000 7f80b2a00640 1 -- 172.21.15.47:0/1062182742 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80c402cab0 con 0x555bf38669f0 2024-03-21T11:35:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.619+0000 7f80d65c1740 10 client.29693 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:44.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.619+0000 7f80d65c1740 1 -- 172.21.15.47:0/1062182742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555bf38d8a60 con 0x555bf38669f0 2024-03-21T11:35:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.619+0000 7f80b2a00640 1 -- 172.21.15.47:0/1062182742 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80c402cd90 con 0x555bf38669f0 2024-03-21T11:35:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.619+0000 7f80b2a00640 1 -- 172.21.15.47:0/1062182742 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f80c400fce0 con 0x555bf38669f0 2024-03-21T11:35:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.619+0000 7f80d65c1740 10 client.29693 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.619+0000 7f80d65c1740 10 client.29693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.619+0000 7f80d65c1740 20 client.29693 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.619+0000 7f80d65c1740 1 --2- 172.21.15.47:0/1062182742 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555bf38d90c0 0x555bf38db4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.619+0000 7f80d65c1740 4 client.29693 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:35:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.619+0000 7f80d65c1740 1 -- 172.21.15.47:0/1062182742 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555bf37d2b50 con 0x555bf38d90c0 2024-03-21T11:35:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.620+0000 7f80cbe00640 1 --2- 172.21.15.47:0/1062182742 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555bf38d90c0 0x555bf38db4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.620+0000 7f80cbe00640 1 --2- 172.21.15.47:0/1062182742 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555bf38d90c0 0x555bf38db4e0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=657ec7a636d15856 server_cookie=93842a8345f7f75f in_seq=0 out_seq=0 2024-03-21T11:35:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.620+0000 7f80b2a00640 10 client.29693 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.621+0000 7f80b2a00640 1 -- 172.21.15.47:0/1062182742 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555bf37d2b50 con 0x555bf38d90c0 2024-03-21T11:35:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.621+0000 7f80b2a00640 10 client.29693 handle_command_reply: tid=0 2024-03-21T11:35:44.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.621+0000 7f80d65c1740 1 client.29693 shutdown 2024-03-21T11:35:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.621+0000 7f80d65c1740 1 -- 172.21.15.47:0/1062182742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf38669f0 msgr2=0x555bf3874210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.621+0000 7f80d65c1740 1 --2- 172.21.15.47:0/1062182742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf38669f0 0x555bf3874210 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009560 tx=0x7f80c4004050 comp rx=0 tx=0).stop 2024-03-21T11:35:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.621+0000 7f80d65c1740 1 -- 172.21.15.47:0/1062182742 shutdown_connections 2024-03-21T11:35:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.621+0000 7f80d65c1740 1 --2- 172.21.15.47:0/1062182742 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555bf38d90c0 0x555bf38db4e0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.621+0000 7f80d65c1740 1 --2- 172.21.15.47:0/1062182742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bf3868580 0x555bf3877770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.621+0000 7f80d65c1740 1 --2- 172.21.15.47:0/1062182742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf3867bc0 0x555bf3874750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.621+0000 7f80d65c1740 1 --2- 172.21.15.47:0/1062182742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555bf38669f0 0x555bf3874210 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.621+0000 7f80d65c1740 1 -- 172.21.15.47:0/1062182742 >> 172.21.15.47:0/1062182742 conn(0x555bf3861de0 msgr2=0x555bf38658d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.621+0000 7f80b2a00640 0 client.29693 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.621+0000 7f80d65c1740 1 -- 172.21.15.47:0/1062182742 shutdown_connections 2024-03-21T11:35:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.621+0000 7f80d65c1740 1 -- 172.21.15.47:0/1062182742 wait complete. 2024-03-21T11:35:44.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.622+0000 7f80d65c1740 20 client.29693 trim_cache size 0 max 16384 2024-03-21T11:35:44.626 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:35:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.622+0000 7f80d2200640 1 -- 172.21.15.47:0/3673835532 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80b40a2fa0 msgr2=0x7f80b40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.622+0000 7f80d2200640 1 --2- 172.21.15.47:0/3673835532 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80b40a2fa0 0x7f80b40a5460 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f80c00c6210 tx=0x7f80c00f7530 comp rx=0 tx=0).stop 2024-03-21T11:35:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.622+0000 7f80d2200640 1 -- 172.21.15.47:0/3673835532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80cc1037a0 msgr2=0x7f80cc199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.622+0000 7f80d2200640 1 --2- 172.21.15.47:0/3673835532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80cc1037a0 0x7f80cc199a20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f80c0000c00 tx=0x7f80c002c6c0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.622+0000 7f80d2200640 1 -- 172.21.15.47:0/3673835532 shutdown_connections 2024-03-21T11:35:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.622+0000 7f80d2200640 1 --2- 172.21.15.47:0/3673835532 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80b40a2fa0 0x7f80b40a5460 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.622+0000 7f80d2200640 1 --2- 172.21.15.47:0/3673835532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80cc105350 0x7f80cc1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.622+0000 7f80d2200640 1 --2- 172.21.15.47:0/3673835532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80cc104990 0x7f80cc199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.622+0000 7f80d2200640 1 --2- 172.21.15.47:0/3673835532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80cc1037a0 0x7f80cc199a20 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:44.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.622+0000 7f80d2200640 1 -- 172.21.15.47:0/3673835532 >> 172.21.15.47:0/3673835532 conn(0x7f80cc0fed10 msgr2=0x7f80cc107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:44.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.623+0000 7f80d2200640 1 -- 172.21.15.47:0/3673835532 shutdown_connections 2024-03-21T11:35:44.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:44.623+0000 7f80d2200640 1 -- 172.21.15.47:0/3673835532 wait complete. 2024-03-21T11:35:45.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:44 smithi102 ceph-mon[36748]: pgmap v772: 129 pgs: 129 active+clean; 20 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 2.4 MiB/s wr, 315 op/s 2024-03-21T11:35:45.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:44 smithi047 ceph-mon[31351]: pgmap v772: 129 pgs: 129 active+clean; 20 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 2.4 MiB/s wr, 315 op/s 2024-03-21T11:35:45.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:44 smithi100 ceph-mon[36245]: pgmap v772: 129 pgs: 129 active+clean; 20 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 2.4 MiB/s wr, 315 op/s 2024-03-21T11:35:45.572 INFO:tasks.workunit.client.0.smithi047.stdout: 1 942117 7.98 MB/sec execute 509 sec latency 13.592 ms 2024-03-21T11:35:45.645 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:35:45.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:35:45.645 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:35:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.894+0000 7f4b82400640 1 -- 172.21.15.47:0/2363889352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b7c105370 msgr2=0x7f4b7c10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.894+0000 7f4b82400640 1 --2- 172.21.15.47:0/2363889352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b7c105370 0x7f4b7c10bc00 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009900 tx=0x7f4b7402bc40 comp rx=0 tx=0).stop 2024-03-21T11:35:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.894+0000 7f4b82400640 1 -- 172.21.15.47:0/2363889352 shutdown_connections 2024-03-21T11:35:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.894+0000 7f4b82400640 1 --2- 172.21.15.47:0/2363889352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b7c105370 0x7f4b7c10bc00 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.894+0000 7f4b82400640 1 --2- 172.21.15.47:0/2363889352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b7c1049b0 0x7f4b7c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.894+0000 7f4b82400640 1 --2- 172.21.15.47:0/2363889352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b7c1037c0 0x7f4b7c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.894+0000 7f4b82400640 1 -- 172.21.15.47:0/2363889352 >> 172.21.15.47:0/2363889352 conn(0x7f4b7c0fed50 msgr2=0x7f4b7c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.895+0000 7f4b82400640 1 -- 172.21.15.47:0/2363889352 shutdown_connections 2024-03-21T11:35:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.895+0000 7f4b82400640 1 -- 172.21.15.47:0/2363889352 wait complete. 2024-03-21T11:35:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.896+0000 7f4b82400640 1 Processor -- start 2024-03-21T11:35:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.896+0000 7f4b82400640 1 -- start start 2024-03-21T11:35:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.896+0000 7f4b82400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b7c1037c0 0x7f4b7c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.896+0000 7f4b82400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b7c1049b0 0x7f4b7c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.896+0000 7f4b82400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b7c105370 0x7f4b7c1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.896+0000 7f4b82400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c10f490 con 0x7f4b7c1037c0 2024-03-21T11:35:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.896+0000 7f4b82400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c10f310 con 0x7f4b7c105370 2024-03-21T11:35:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.896+0000 7f4b82400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c10f610 con 0x7f4b7c1049b0 2024-03-21T11:35:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.896+0000 7f4b81a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b7c105370 0x7f4b7c1a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.896+0000 7f4b81000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b7c1037c0 0x7f4b7c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.896+0000 7f4b7be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b7c1049b0 0x7f4b7c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.896+0000 7f4b81a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b7c105370 0x7f4b7c1a1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49816/0 (socket says 172.21.15.47:49816) 2024-03-21T11:35:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.896+0000 7f4b7be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b7c1049b0 0x7f4b7c199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41682/0 (socket says 172.21.15.47:41682) 2024-03-21T11:35:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.896+0000 7f4b81a00640 1 -- 172.21.15.47:0/2447508106 learned_addr learned my addr 172.21.15.47:0/2447508106 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.897+0000 7f4b7be00640 1 -- 172.21.15.47:0/2447508106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b7c105370 msgr2=0x7f4b7c1a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.897+0000 7f4b7be00640 1 --2- 172.21.15.47:0/2447508106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b7c105370 0x7f4b7c1a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.897+0000 7f4b7be00640 1 -- 172.21.15.47:0/2447508106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b7c1037c0 msgr2=0x7f4b7c199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.897+0000 7f4b7be00640 1 --2- 172.21.15.47:0/2447508106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b7c1037c0 0x7f4b7c199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.897+0000 7f4b7be00640 1 -- 172.21.15.47:0/2447508106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c1a1670 con 0x7f4b7c1049b0 2024-03-21T11:35:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.897+0000 7f4b7be00640 1 --2- 172.21.15.47:0/2447508106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b7c1049b0 0x7f4b7c199f90 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400e9f0 tx=0x7f4b6400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.897+0000 7f4b79600640 1 -- 172.21.15.47:0/2447508106 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400ccc0 con 0x7f4b7c1049b0 2024-03-21T11:35:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.897+0000 7f4b79600640 1 -- 172.21.15.47:0/2447508106 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b64010910 con 0x7f4b7c1049b0 2024-03-21T11:35:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.897+0000 7f4b79600640 1 -- 172.21.15.47:0/2447508106 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64018960 con 0x7f4b7c1049b0 2024-03-21T11:35:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.898+0000 7f4b82400640 1 -- 172.21.15.47:0/2447508106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7c1a1960 con 0x7f4b7c1049b0 2024-03-21T11:35:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.898+0000 7f4b82400640 1 -- 172.21.15.47:0/2447508106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7c1a1e60 con 0x7f4b7c1049b0 2024-03-21T11:35:45.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.901+0000 7f4b79600640 1 -- 172.21.15.47:0/2447508106 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4b64010ac0 con 0x7f4b7c1049b0 2024-03-21T11:35:45.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.901+0000 7f4b8676e740 1 Processor -- start 2024-03-21T11:35:45.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.902+0000 7f4b8676e740 1 -- start start 2024-03-21T11:35:45.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.902+0000 7f4b8676e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f889122b0 0x559f889126d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.902+0000 7f4b8676e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88912ca0 0x559f889177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.902+0000 7f4b8676e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88917e10 0x559f8891a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.902+0000 7f4b8676e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f88877eb0 con 0x559f88917e10 2024-03-21T11:35:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.902+0000 7f4b8676e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f88820a60 con 0x559f889122b0 2024-03-21T11:35:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.902+0000 7f4b8676e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f88873f50 con 0x559f88912ca0 2024-03-21T11:35:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.902+0000 7f4b79600640 1 --2- 172.21.15.47:0/2447508106 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b600a2fa0 0x7f4b600a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.902+0000 7f4b81a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f889122b0 0x559f889126d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.902+0000 7f4b81000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88912ca0 0x559f889177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.902+0000 7f4b81a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f889122b0 0x559f889126d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49828/0 (socket says 172.21.15.47:49828) 2024-03-21T11:35:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.902+0000 7f4b81000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88912ca0 0x559f889177a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41690/0 (socket says 172.21.15.47:41690) 2024-03-21T11:35:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.902+0000 7f4b81a00640 1 -- 172.21.15.47:0/152888478 learned_addr learned my addr 172.21.15.47:0/152888478 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.902+0000 7f4b7be00640 1 --2- 172.21.15.47:0/152888478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88917e10 0x559f8891a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.903+0000 7f4b79600640 1 -- 172.21.15.47:0/2447508106 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4b640cfab0 con 0x7f4b7c1049b0 2024-03-21T11:35:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.903+0000 7f4b81000640 1 -- 172.21.15.47:0/152888478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f889122b0 msgr2=0x559f889126d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.903+0000 7f4b81000640 1 --2- 172.21.15.47:0/152888478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f889122b0 0x559f889126d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.903+0000 7f4b81000640 1 -- 172.21.15.47:0/152888478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88917e10 msgr2=0x559f8891a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.903+0000 7f4b81000640 1 --2- 172.21.15.47:0/152888478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88917e10 0x559f8891a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.903+0000 7f4b81000640 1 -- 172.21.15.47:0/152888478 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f886547e0 con 0x559f88912ca0 2024-03-21T11:35:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.903+0000 7f4b81a00640 1 --2- 172.21.15.47:0/2447508106 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b600a2fa0 0x7f4b600a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.903+0000 7f4b81000640 1 --2- 172.21.15.47:0/152888478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88912ca0 0x559f889177a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c005fd0 tx=0x7f4b6c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b7a2158f06099c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.903+0000 7f4b72000640 1 -- 172.21.15.47:0/152888478 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c004b20 con 0x559f88912ca0 2024-03-21T11:35:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.904+0000 7f4b72000640 1 -- 172.21.15.47:0/152888478 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b6c003e50 con 0x559f88912ca0 2024-03-21T11:35:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.904+0000 7f4b72000640 1 -- 172.21.15.47:0/152888478 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c00f540 con 0x559f88912ca0 2024-03-21T11:35:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.904+0000 7f4b8676e740 1 -- 172.21.15.47:0/152888478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88912ca0 msgr2=0x559f889177a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.904+0000 7f4b8676e740 1 --2- 172.21.15.47:0/152888478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88912ca0 0x559f889177a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c005fd0 tx=0x7f4b6c0118d0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.904+0000 7f4b81a00640 1 --2- 172.21.15.47:0/2447508106 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b600a2fa0 0x7f4b600a5460 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4b740096c0 tx=0x7f4b7402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.904+0000 7f4b8676e740 1 -- 172.21.15.47:0/152888478 shutdown_connections 2024-03-21T11:35:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.904+0000 7f4b8676e740 1 --2- 172.21.15.47:0/152888478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88917e10 0x559f8891a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.904+0000 7f4b8676e740 1 --2- 172.21.15.47:0/152888478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88912ca0 0x559f889177a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.904+0000 7f4b8676e740 1 --2- 172.21.15.47:0/152888478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f889122b0 0x559f889126d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.904+0000 7f4b8676e740 1 -- 172.21.15.47:0/152888478 >> 172.21.15.47:0/152888478 conn(0x559f8890d820 msgr2=0x559f8890dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.905+0000 7f4b8676e740 1 -- 172.21.15.47:0/152888478 shutdown_connections 2024-03-21T11:35:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.905+0000 7f4b8676e740 1 -- 172.21.15.47:0/152888478 wait complete. 2024-03-21T11:35:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.905+0000 7f4b8676e740 1 -- start start 2024-03-21T11:35:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.905+0000 7f4b8676e740 1 Processor -- start 2024-03-21T11:35:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b8676e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f889122b0 0x559f8891c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b8676e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f88912ca0 0x559f8891ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b8676e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88917e10 0x559f88923ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b8676e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8891adc0 con 0x559f88917e10 2024-03-21T11:35:45.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b8676e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f88833eb0 con 0x559f88912ca0 2024-03-21T11:35:45.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b8676e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8891af40 con 0x559f889122b0 2024-03-21T11:35:45.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b8676e740 10 client.0 _ll_register_callbacks cb 0x559f88828150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:45.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b81000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f889122b0 0x559f8891c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b81000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f889122b0 0x559f8891c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41698/0 (socket says 172.21.15.47:41698) 2024-03-21T11:35:45.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b81000640 1 -- 172.21.15.47:0/1983185970 learned_addr learned my addr 172.21.15.47:0/1983185970 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:45.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b7be00640 1 --2- 172.21.15.47:0/1983185970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f88912ca0 0x559f8891ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b81a00640 1 --2- 172.21.15.47:0/1983185970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88917e10 0x559f88923ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b81000640 1 -- 172.21.15.47:0/1983185970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f88912ca0 msgr2=0x559f8891ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:45.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b81000640 1 --2- 172.21.15.47:0/1983185970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f88912ca0 0x559f8891ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b81000640 1 -- 172.21.15.47:0/1983185970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88917e10 msgr2=0x559f88923ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:45.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b81000640 1 --2- 172.21.15.47:0/1983185970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88917e10 0x559f88923ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b81000640 1 -- 172.21.15.47:0/1983185970 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f886b4f70 con 0x559f889122b0 2024-03-21T11:35:45.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.906+0000 7f4b7be00640 1 --2- 172.21.15.47:0/1983185970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f88912ca0 0x559f8891ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:45.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.907+0000 7f4b81000640 1 --2- 172.21.15.47:0/1983185970 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f889122b0 0x559f8891c550 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009250 tx=0x7f4b6c003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e0088d355da5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:45.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.907+0000 7f4b6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:45.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.907+0000 7f4b6aa00640 1 -- 172.21.15.47:0/1983185970 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c028020 con 0x559f889122b0 2024-03-21T11:35:45.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.907+0000 7f4b6aa00640 1 -- 172.21.15.47:0/1983185970 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4b6c004c50 con 0x559f889122b0 2024-03-21T11:35:45.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.907+0000 7f4b8676e740 1 -- 172.21.15.47:0/1983185970 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b6c003e50 con 0x559f889122b0 2024-03-21T11:35:45.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.907+0000 7f4b6aa00640 1 -- 172.21.15.47:0/1983185970 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b6c033050 con 0x559f889122b0 2024-03-21T11:35:45.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.907+0000 7f4b8676e740 10 client.29743 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:45.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.908+0000 7f4b8676e740 1 -- 172.21.15.47:0/1983185970 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559f8854c170 con 0x559f889122b0 2024-03-21T11:35:45.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.908+0000 7f4b6aa00640 1 -- 172.21.15.47:0/1983185970 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b6c003e50 con 0x559f889122b0 2024-03-21T11:35:45.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.908+0000 7f4b6aa00640 1 -- 172.21.15.47:0/1983185970 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4b6c005000 con 0x559f889122b0 2024-03-21T11:35:45.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.908+0000 7f4b8676e740 10 client.29743 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:45.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.908+0000 7f4b8676e740 10 client.29743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.908+0000 7f4b8676e740 20 client.29743 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.908+0000 7f4b8676e740 1 --2- 172.21.15.47:0/1983185970 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f88928170 0x559f8892a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.908+0000 7f4b8676e740 4 client.29743 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.908+0000 7f4b8676e740 1 -- 172.21.15.47:0/1983185970 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f888853f0 con 0x559f88928170 2024-03-21T11:35:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.908+0000 7f4b7be00640 1 --2- 172.21.15.47:0/1983185970 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f88928170 0x559f8892a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.909+0000 7f4b7be00640 1 --2- 172.21.15.47:0/1983185970 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f88928170 0x559f8892a5a0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8909950c840cd339 server_cookie=8d928e78eb4776bd in_seq=0 out_seq=0 2024-03-21T11:35:45.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.909+0000 7f4b6aa00640 10 client.29743 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:45.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.910+0000 7f4b6aa00640 1 -- 172.21.15.47:0/1983185970 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559f888853f0 con 0x559f88928170 2024-03-21T11:35:45.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.910+0000 7f4b6aa00640 10 client.29743 handle_command_reply: tid=0 2024-03-21T11:35:45.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.910+0000 7f4b8676e740 1 client.29743 shutdown 2024-03-21T11:35:45.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.910+0000 7f4b8676e740 1 -- 172.21.15.47:0/1983185970 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f889122b0 msgr2=0x559f8891c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.910+0000 7f4b8676e740 1 --2- 172.21.15.47:0/1983185970 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f889122b0 0x559f8891c550 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009250 tx=0x7f4b6c003ba0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.911+0000 7f4b8676e740 1 -- 172.21.15.47:0/1983185970 shutdown_connections 2024-03-21T11:35:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.911+0000 7f4b8676e740 1 --2- 172.21.15.47:0/1983185970 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f88928170 0x559f8892a5a0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.911+0000 7f4b8676e740 1 --2- 172.21.15.47:0/1983185970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88917e10 0x559f88923ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.911+0000 7f4b8676e740 1 --2- 172.21.15.47:0/1983185970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f88912ca0 0x559f8891ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.911+0000 7f4b6aa00640 0 client.29743 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:45.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.911+0000 7f4b8676e740 1 --2- 172.21.15.47:0/1983185970 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f889122b0 0x559f8891c550 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.911+0000 7f4b8676e740 1 -- 172.21.15.47:0/1983185970 >> 172.21.15.47:0/1983185970 conn(0x559f8890d820 msgr2=0x559f8890ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:45.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.911+0000 7f4b8676e740 1 -- 172.21.15.47:0/1983185970 shutdown_connections 2024-03-21T11:35:45.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.911+0000 7f4b8676e740 1 -- 172.21.15.47:0/1983185970 wait complete. 2024-03-21T11:35:45.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.911+0000 7f4b8676e740 20 client.29743 trim_cache size 0 max 16384 2024-03-21T11:35:45.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.930+0000 7f4b8676e740 1 Processor -- start 2024-03-21T11:35:45.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.930+0000 7f4b8676e740 1 -- start start 2024-03-21T11:35:45.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.930+0000 7f4b8676e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f889122b0 0x559f8890c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.930+0000 7f4b8676e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88912ca0 0x559f8892ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.930+0000 7f4b8676e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88917e10 0x559f8892f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.930+0000 7f4b8676e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c1ac110 con 0x559f88912ca0 2024-03-21T11:35:45.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.930+0000 7f4b8676e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c1ada90 con 0x559f889122b0 2024-03-21T11:35:45.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.930+0000 7f4b8676e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c1ac770 con 0x559f88917e10 2024-03-21T11:35:45.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.931+0000 7f4b7be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88912ca0 0x559f8892ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.931+0000 7f4b81a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88917e10 0x559f8892f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.931+0000 7f4b81000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f889122b0 0x559f8890c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.931+0000 7f4b7be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88912ca0 0x559f8892ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49892/0 (socket says 172.21.15.47:49892) 2024-03-21T11:35:45.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.931+0000 7f4b81a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88917e10 0x559f8892f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41710/0 (socket says 172.21.15.47:41710) 2024-03-21T11:35:45.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.931+0000 7f4b7be00640 1 -- 172.21.15.47:0/1122835365 learned_addr learned my addr 172.21.15.47:0/1122835365 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:45.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.931+0000 7f4b7be00640 1 -- 172.21.15.47:0/1122835365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88917e10 msgr2=0x559f8892f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:45.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.931+0000 7f4b7be00640 1 --2- 172.21.15.47:0/1122835365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88917e10 0x559f8892f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.931+0000 7f4b7be00640 1 -- 172.21.15.47:0/1122835365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f889122b0 msgr2=0x559f8890c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.931+0000 7f4b7be00640 1 --2- 172.21.15.47:0/1122835365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f889122b0 0x559f8890c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.931+0000 7f4b7be00640 1 -- 172.21.15.47:0/1122835365 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f8890e590 con 0x559f88912ca0 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.931+0000 7f4b7be00640 1 --2- 172.21.15.47:0/1122835365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88912ca0 0x559f8892ee60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4b640cd1c0 tx=0x7f4b6406ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cd83c82dd80873 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.932+0000 7f4b71600640 1 -- 172.21.15.47:0/1122835365 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64026040 con 0x559f88912ca0 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.932+0000 7f4b71600640 1 -- 172.21.15.47:0/1122835365 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b640a4800 con 0x559f88912ca0 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.932+0000 7f4b71600640 1 -- 172.21.15.47:0/1122835365 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b640badc0 con 0x559f88912ca0 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.932+0000 7f4b8676e740 1 -- 172.21.15.47:0/1122835365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88912ca0 msgr2=0x559f8892ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.932+0000 7f4b8676e740 1 --2- 172.21.15.47:0/1122835365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88912ca0 0x559f8892ee60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4b640cd1c0 tx=0x7f4b6406ebd0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.933+0000 7f4b8676e740 1 -- 172.21.15.47:0/1122835365 shutdown_connections 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.933+0000 7f4b8676e740 1 --2- 172.21.15.47:0/1122835365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88917e10 0x559f8892f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.933+0000 7f4b8676e740 1 --2- 172.21.15.47:0/1122835365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f88912ca0 0x559f8892ee60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.933+0000 7f4b8676e740 1 --2- 172.21.15.47:0/1122835365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f889122b0 0x559f8890c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.933+0000 7f4b8676e740 1 -- 172.21.15.47:0/1122835365 >> 172.21.15.47:0/1122835365 conn(0x559f8890d820 msgr2=0x559f8890c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:45.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.933+0000 7f4b8676e740 1 -- 172.21.15.47:0/1122835365 shutdown_connections 2024-03-21T11:35:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.933+0000 7f4b8676e740 1 -- 172.21.15.47:0/1122835365 wait complete. 2024-03-21T11:35:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.933+0000 7f4b8676e740 1 -- start start 2024-03-21T11:35:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.933+0000 7f4b8676e740 1 Processor -- start 2024-03-21T11:35:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.934+0000 7f4b8676e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f889122b0 0x559f88920d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.934+0000 7f4b8676e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f88912ca0 0x559f889212b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.934+0000 7f4b8676e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88917e10 0x559f889222c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.934+0000 7f4b8676e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c1ad430 con 0x559f889122b0 2024-03-21T11:35:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.934+0000 7f4b8676e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c1acdd0 con 0x559f88912ca0 2024-03-21T11:35:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.934+0000 7f4b8676e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c1aa2d0 con 0x559f88917e10 2024-03-21T11:35:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.934+0000 7f4b8676e740 10 client.0 _ll_register_callbacks cb 0x7f4b6c009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.934+0000 7f4b7be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f88912ca0 0x559f889212b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.934+0000 7f4b7be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f88912ca0 0x559f889212b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49856/0 (socket says 172.21.15.47:49856) 2024-03-21T11:35:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.934+0000 7f4b7be00640 1 -- 172.21.15.47:0/4123025898 learned_addr learned my addr 172.21.15.47:0/4123025898 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.934+0000 7f4b81a00640 1 --2- 172.21.15.47:0/4123025898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88917e10 0x559f889222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.934+0000 7f4b81000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f889122b0 0x559f88920d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.935+0000 7f4b81a00640 1 -- 172.21.15.47:0/4123025898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f88912ca0 msgr2=0x559f889212b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.935+0000 7f4b81a00640 1 --2- 172.21.15.47:0/4123025898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f88912ca0 0x559f889212b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.935+0000 7f4b81a00640 1 -- 172.21.15.47:0/4123025898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f889122b0 msgr2=0x559f88920d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.935+0000 7f4b81a00640 1 --2- 172.21.15.47:0/4123025898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f889122b0 0x559f88920d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.935+0000 7f4b81a00640 1 -- 172.21.15.47:0/4123025898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f8891b080 con 0x559f88917e10 2024-03-21T11:35:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.935+0000 7f4b7be00640 1 --2- 172.21.15.47:0/4123025898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f88912ca0 0x559f889212b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.935+0000 7f4b81a00640 1 --2- 172.21.15.47:0/4123025898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88917e10 0x559f889222c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4b7402efd0 tx=0x7f4b74009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93d548025f827b68 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.935+0000 7f4b6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.935+0000 7f4b6aa00640 1 -- 172.21.15.47:0/4123025898 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74004ee0 con 0x559f88917e10 2024-03-21T11:35:45.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.935+0000 7f4b8676e740 1 -- 172.21.15.47:0/4123025898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b640a4800 con 0x559f88917e10 2024-03-21T11:35:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.935+0000 7f4b6aa00640 1 -- 172.21.15.47:0/4123025898 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4b74005080 con 0x559f88917e10 2024-03-21T11:35:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.935+0000 7f4b6aa00640 1 -- 172.21.15.47:0/4123025898 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b74005280 con 0x559f88917e10 2024-03-21T11:35:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.936+0000 7f4b8676e740 10 client.29749 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.936+0000 7f4b8676e740 1 -- 172.21.15.47:0/4123025898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559f88983d30 con 0x559f88917e10 2024-03-21T11:35:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.936+0000 7f4b6aa00640 1 -- 172.21.15.47:0/4123025898 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b640a4800 con 0x559f88917e10 2024-03-21T11:35:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.936+0000 7f4b6aa00640 1 -- 172.21.15.47:0/4123025898 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4b74054020 con 0x559f88917e10 2024-03-21T11:35:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.936+0000 7f4b8676e740 10 client.29749 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.936+0000 7f4b8676e740 10 client.29749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.936+0000 7f4b8676e740 20 client.29749 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.936+0000 7f4b8676e740 1 --2- 172.21.15.47:0/4123025898 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f889845b0 0x559f88986a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.936+0000 7f4b8676e740 4 client.29749 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:35:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.936+0000 7f4b8676e740 1 -- 172.21.15.47:0/4123025898 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559f8887f740 con 0x559f889845b0 2024-03-21T11:35:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.936+0000 7f4b81000640 1 --2- 172.21.15.47:0/4123025898 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f889845b0 0x559f88986a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:45.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.937+0000 7f4b81000640 1 --2- 172.21.15.47:0/4123025898 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f889845b0 0x559f88986a10 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=120550bcedda61c5 server_cookie=eeeeff13dda0bf01 in_seq=0 out_seq=0 2024-03-21T11:35:45.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.937+0000 7f4b6aa00640 10 client.29749 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.939+0000 7f4b6aa00640 1 -- 172.21.15.47:0/4123025898 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559f8887f740 con 0x559f889845b0 2024-03-21T11:35:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.939+0000 7f4b6aa00640 10 client.29749 handle_command_reply: tid=0 2024-03-21T11:35:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.939+0000 7f4b8676e740 1 client.29749 shutdown 2024-03-21T11:35:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.939+0000 7f4b8676e740 1 -- 172.21.15.47:0/4123025898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88917e10 msgr2=0x559f889222c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.939+0000 7f4b8676e740 1 --2- 172.21.15.47:0/4123025898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88917e10 0x559f889222c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4b7402efd0 tx=0x7f4b74009260 comp rx=0 tx=0).stop 2024-03-21T11:35:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.940+0000 7f4b8676e740 1 -- 172.21.15.47:0/4123025898 shutdown_connections 2024-03-21T11:35:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.940+0000 7f4b8676e740 1 --2- 172.21.15.47:0/4123025898 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f889845b0 0x559f88986a10 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.940+0000 7f4b8676e740 1 --2- 172.21.15.47:0/4123025898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f88917e10 0x559f889222c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.940+0000 7f4b8676e740 1 --2- 172.21.15.47:0/4123025898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f88912ca0 0x559f889212b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.940+0000 7f4b8676e740 1 --2- 172.21.15.47:0/4123025898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f889122b0 0x559f88920d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.940+0000 7f4b6aa00640 0 client.29749 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:45.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.940+0000 7f4b8676e740 1 -- 172.21.15.47:0/4123025898 >> 172.21.15.47:0/4123025898 conn(0x559f8890d820 msgr2=0x559f8890ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:45.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.940+0000 7f4b8676e740 1 -- 172.21.15.47:0/4123025898 shutdown_connections 2024-03-21T11:35:45.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.940+0000 7f4b8676e740 1 -- 172.21.15.47:0/4123025898 wait complete. 2024-03-21T11:35:45.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.940+0000 7f4b8676e740 20 client.29749 trim_cache size 0 max 16384 2024-03-21T11:35:45.947 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:35:45.948 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:35:45.948 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "5a639d18-a42d-44cd-a480-b12a74cfce2a", 2024-03-21T11:35:45.948 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:35:45.948 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:35:45.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.941+0000 7f4b82400640 1 -- 172.21.15.47:0/2447508106 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b600a2fa0 msgr2=0x7f4b600a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:45.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.941+0000 7f4b82400640 1 --2- 172.21.15.47:0/2447508106 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b600a2fa0 0x7f4b600a5460 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4b740096c0 tx=0x7f4b7402e040 comp rx=0 tx=0).stop 2024-03-21T11:35:45.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.941+0000 7f4b82400640 1 -- 172.21.15.47:0/2447508106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b7c1049b0 msgr2=0x7f4b7c199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:45.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.941+0000 7f4b82400640 1 --2- 172.21.15.47:0/2447508106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b7c1049b0 0x7f4b7c199f90 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400e9f0 tx=0x7f4b6400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.941+0000 7f4b82400640 1 -- 172.21.15.47:0/2447508106 shutdown_connections 2024-03-21T11:35:45.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.941+0000 7f4b82400640 1 --2- 172.21.15.47:0/2447508106 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4b600a2fa0 0x7f4b600a5460 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.941+0000 7f4b82400640 1 --2- 172.21.15.47:0/2447508106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b7c105370 0x7f4b7c1a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.941+0000 7f4b82400640 1 --2- 172.21.15.47:0/2447508106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b7c1049b0 0x7f4b7c199f90 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.941+0000 7f4b82400640 1 --2- 172.21.15.47:0/2447508106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b7c1037c0 0x7f4b7c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:45.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.941+0000 7f4b82400640 1 -- 172.21.15.47:0/2447508106 >> 172.21.15.47:0/2447508106 conn(0x7f4b7c0fed50 msgr2=0x7f4b7c107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:45.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.941+0000 7f4b82400640 1 -- 172.21.15.47:0/2447508106 shutdown_connections 2024-03-21T11:35:45.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:45.941+0000 7f4b82400640 1 -- 172.21.15.47:0/2447508106 wait complete. 2024-03-21T11:35:45.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:35:45.965 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.184+0000 7f1a79c00640 1 -- 172.21.15.47:0/2746942152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a741016b0 msgr2=0x7f1a74101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.184+0000 7f1a79c00640 1 --2- 172.21.15.47:0/2746942152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a741016b0 0x7f1a74101ad0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009900 tx=0x7f1a6402bc40 comp rx=0 tx=0).stop 2024-03-21T11:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.185+0000 7f1a79c00640 1 -- 172.21.15.47:0/2746942152 shutdown_connections 2024-03-21T11:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.185+0000 7f1a79c00640 1 --2- 172.21.15.47:0/2746942152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a74103260 0x7f1a74109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.185+0000 7f1a79c00640 1 --2- 172.21.15.47:0/2746942152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a741028a0 0x7f1a74102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.185+0000 7f1a79c00640 1 --2- 172.21.15.47:0/2746942152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a741016b0 0x7f1a74101ad0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.185+0000 7f1a79c00640 1 -- 172.21.15.47:0/2746942152 >> 172.21.15.47:0/2746942152 conn(0x7f1a740fcc40 msgr2=0x7f1a740ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.185+0000 7f1a79c00640 1 -- 172.21.15.47:0/2746942152 shutdown_connections 2024-03-21T11:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.185+0000 7f1a79c00640 1 -- 172.21.15.47:0/2746942152 wait complete. 2024-03-21T11:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.186+0000 7f1a79c00640 1 Processor -- start 2024-03-21T11:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.186+0000 7f1a79c00640 1 -- start start 2024-03-21T11:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.187+0000 7f1a79c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a741016b0 0x7f1a74199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.187+0000 7f1a79c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a741028a0 0x7f1a7419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.187+0000 7f1a79c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a74103260 0x7f1a741a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.187+0000 7f1a79c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7410d300 con 0x7f1a741016b0 2024-03-21T11:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.187+0000 7f1a79c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7410d180 con 0x7f1a741028a0 2024-03-21T11:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.187+0000 7f1a79c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7410d480 con 0x7f1a74103260 2024-03-21T11:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.187+0000 7f1a72a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a741016b0 0x7f1a74199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.187+0000 7f1a73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a74103260 0x7f1a741a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.187+0000 7f1a72a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a741016b0 0x7f1a74199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49904/0 (socket says 172.21.15.47:49904) 2024-03-21T11:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.187+0000 7f1a72a00640 1 -- 172.21.15.47:0/1418564289 learned_addr learned my addr 172.21.15.47:0/1418564289 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.187+0000 7f1a72000640 1 --2- 172.21.15.47:0/1418564289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a741028a0 0x7f1a7419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.188+0000 7f1a73400640 1 -- 172.21.15.47:0/1418564289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a741028a0 msgr2=0x7f1a7419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.188+0000 7f1a73400640 1 --2- 172.21.15.47:0/1418564289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a741028a0 0x7f1a7419a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.188+0000 7f1a73400640 1 -- 172.21.15.47:0/1418564289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a741016b0 msgr2=0x7f1a74199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.188+0000 7f1a73400640 1 --2- 172.21.15.47:0/1418564289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a741016b0 0x7f1a74199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.188+0000 7f1a73400640 1 -- 172.21.15.47:0/1418564289 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a741a1840 con 0x7f1a74103260 2024-03-21T11:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.188+0000 7f1a72000640 1 --2- 172.21.15.47:0/1418564289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a741028a0 0x7f1a7419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.188+0000 7f1a73400640 1 --2- 172.21.15.47:0/1418564289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a74103260 0x7f1a741a11a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00ec50 tx=0x7f1a6c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.188+0000 7f1a6b400640 1 -- 172.21.15.47:0/1418564289 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c003b70 con 0x7f1a74103260 2024-03-21T11:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.188+0000 7f1a6b400640 1 -- 172.21.15.47:0/1418564289 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a6c003d10 con 0x7f1a74103260 2024-03-21T11:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.188+0000 7f1a79c00640 1 -- 172.21.15.47:0/1418564289 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a741a1b30 con 0x7f1a74103260 2024-03-21T11:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.188+0000 7f1a6b400640 1 -- 172.21.15.47:0/1418564289 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c010890 con 0x7f1a74103260 2024-03-21T11:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.189+0000 7f1a79c00640 1 -- 172.21.15.47:0/1418564289 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a741a2030 con 0x7f1a74103260 2024-03-21T11:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.192+0000 7f1a7ca25740 1 Processor -- start 2024-03-21T11:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.192+0000 7f1a6b400640 1 -- 172.21.15.47:0/1418564289 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1a6c003fc0 con 0x7f1a74103260 2024-03-21T11:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.192+0000 7f1a7ca25740 1 -- start start 2024-03-21T11:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.192+0000 7f1a7ca25740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 0x5630ecf362a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.192+0000 7f1a7ca25740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf3b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.193+0000 7f1a7ca25740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf3de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.193+0000 7f1a7ca25740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ece9b250 con 0x5630ecf35e80 2024-03-21T11:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.193+0000 7f1a7ca25740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ece43a30 con 0x5630ecf36870 2024-03-21T11:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.193+0000 7f1a7ca25740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ece972f0 con 0x5630ecf3b9e0 2024-03-21T11:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.193+0000 7f1a72000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 0x5630ecf362a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.193+0000 7f1a72a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf3b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.193+0000 7f1a72a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf3b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49882/0 (socket says 172.21.15.47:49882) 2024-03-21T11:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.193+0000 7f1a72a00640 1 -- 172.21.15.47:0/3286382137 learned_addr learned my addr 172.21.15.47:0/3286382137 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.193+0000 7f1a72000640 1 --2- 172.21.15.47:0/3286382137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf3de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.193+0000 7f1a72a00640 1 -- 172.21.15.47:0/3286382137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 msgr2=0x5630ecf3de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.193+0000 7f1a72a00640 1 --2- 172.21.15.47:0/3286382137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf3de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.193+0000 7f1a72a00640 1 -- 172.21.15.47:0/3286382137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 msgr2=0x5630ecf362a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.193+0000 7f1a72a00640 1 --2- 172.21.15.47:0/3286382137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 0x5630ecf362a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.193+0000 7f1a72a00640 1 -- 172.21.15.47:0/3286382137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630ece2e690 con 0x5630ecf36870 2024-03-21T11:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.194+0000 7f1a6b400640 1 --2- 172.21.15.47:0/1418564289 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a580a2f20 0x7f1a580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.194+0000 7f1a72a00640 1 --2- 172.21.15.47:0/3286382137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf3b370 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1a64000c00 tx=0x7f1a64009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fe960c9c5ff136e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.194+0000 7f1a6b400640 1 -- 172.21.15.47:0/1418564289 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1a6c0cfdd0 con 0x7f1a74103260 2024-03-21T11:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.194+0000 7f1a72000640 1 --2- 172.21.15.47:0/1418564289 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a580a2f20 0x7f1a580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.194+0000 7f1a5fe00640 1 -- 172.21.15.47:0/3286382137 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64042070 con 0x5630ecf36870 2024-03-21T11:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.194+0000 7f1a5fe00640 1 -- 172.21.15.47:0/3286382137 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a6402cb90 con 0x5630ecf36870 2024-03-21T11:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.194+0000 7f1a5fe00640 1 -- 172.21.15.47:0/3286382137 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64004ef0 con 0x5630ecf36870 2024-03-21T11:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.195+0000 7f1a7ca25740 1 -- 172.21.15.47:0/3286382137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 msgr2=0x5630ecf3b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.195+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/3286382137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf3b370 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1a64000c00 tx=0x7f1a64009560 comp rx=0 tx=0).stop 2024-03-21T11:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.195+0000 7f1a72000640 1 --2- 172.21.15.47:0/1418564289 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a580a2f20 0x7f1a580a53e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x5630ecf370d0 tx=0x7f1a60005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.195+0000 7f1a7ca25740 1 -- 172.21.15.47:0/3286382137 shutdown_connections 2024-03-21T11:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.195+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/3286382137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf3de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.195+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/3286382137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf3b370 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.195+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/3286382137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 0x5630ecf362a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.195+0000 7f1a7ca25740 1 -- 172.21.15.47:0/3286382137 >> 172.21.15.47:0/3286382137 conn(0x5630ecefe200 msgr2=0x5630ecf317f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.195+0000 7f1a7ca25740 1 -- 172.21.15.47:0/3286382137 shutdown_connections 2024-03-21T11:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.195+0000 7f1a7ca25740 1 -- 172.21.15.47:0/3286382137 wait complete. 2024-03-21T11:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.196+0000 7f1a7ca25740 1 -- start start 2024-03-21T11:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.196+0000 7f1a7ca25740 1 Processor -- start 2024-03-21T11:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.196+0000 7f1a7ca25740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 0x5630ecf3fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.196+0000 7f1a7ca25740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf40330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.196+0000 7f1a7ca25740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf47370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.196+0000 7f1a7ca25740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ecf3e930 con 0x5630ecf35e80 2024-03-21T11:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.196+0000 7f1a7ca25740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ece56e80 con 0x5630ecf36870 2024-03-21T11:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.196+0000 7f1a7ca25740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ecf3eab0 con 0x5630ecf3b9e0 2024-03-21T11:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a7ca25740 10 client.0 _ll_register_callbacks cb 0x5630ece69130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a72000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf40330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a72000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf40330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49894/0 (socket says 172.21.15.47:49894) 2024-03-21T11:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a72000640 1 -- 172.21.15.47:0/990989705 learned_addr learned my addr 172.21.15.47:0/990989705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a73400640 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf47370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a72a00640 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 0x5630ecf3fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a73400640 1 -- 172.21.15.47:0/990989705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 msgr2=0x5630ecf40330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a73400640 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf40330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a73400640 1 -- 172.21.15.47:0/990989705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 msgr2=0x5630ecf3fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a73400640 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 0x5630ecf3fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a73400640 1 -- 172.21.15.47:0/990989705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630ecc797d0 con 0x5630ecf3b9e0 2024-03-21T11:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a72000640 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf40330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a72a00640 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 0x5630ecf3fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a73400640 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf47370 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00cba0 tx=0x7f1a6c09bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9964b590e00817cd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.197+0000 7f1a5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.198+0000 7f1a5cc00640 1 -- 172.21.15.47:0/990989705 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c009820 con 0x5630ecf3b9e0 2024-03-21T11:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.198+0000 7f1a5cc00640 1 -- 172.21.15.47:0/990989705 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1a6c0a46f0 con 0x5630ecf3b9e0 2024-03-21T11:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.198+0000 7f1a7ca25740 1 -- 172.21.15.47:0/990989705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a6402cb90 con 0x5630ecf3b9e0 2024-03-21T11:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.198+0000 7f1a5cc00640 1 -- 172.21.15.47:0/990989705 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a6c09c870 con 0x5630ecf3b9e0 2024-03-21T11:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.198+0000 7f1a7ca25740 10 client.29761 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.198+0000 7f1a7ca25740 1 -- 172.21.15.47:0/990989705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5630ecbaf3f0 con 0x5630ecf3b9e0 2024-03-21T11:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.198+0000 7f1a5cc00640 1 -- 172.21.15.47:0/990989705 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a6402cb90 con 0x5630ecf3b9e0 2024-03-21T11:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.199+0000 7f1a5cc00640 1 -- 172.21.15.47:0/990989705 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1a6c0bc0c0 con 0x5630ecf3b9e0 2024-03-21T11:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.199+0000 7f1a7ca25740 10 client.29761 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.199+0000 7f1a7ca25740 10 client.29761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.199+0000 7f1a7ca25740 20 client.29761 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.199+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5630ecf4bbf0 0x5630ecf4e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.199+0000 7f1a7ca25740 4 client.29761 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.199+0000 7f1a7ca25740 1 -- 172.21.15.47:0/990989705 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630ecea8790 con 0x5630ecf4bbf0 2024-03-21T11:35:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.199+0000 7f1a72a00640 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5630ecf4bbf0 0x5630ecf4e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.200+0000 7f1a72a00640 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5630ecf4bbf0 0x5630ecf4e040 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53dcc8b1b8867613 server_cookie=3bbffa478291afc1 in_seq=0 out_seq=0 2024-03-21T11:35:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.200+0000 7f1a5cc00640 10 client.29761 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.201+0000 7f1a5cc00640 1 -- 172.21.15.47:0/990989705 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5630ecea8790 con 0x5630ecf4bbf0 2024-03-21T11:35:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.201+0000 7f1a5cc00640 10 client.29761 handle_command_reply: tid=0 2024-03-21T11:35:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.201+0000 7f1a7ca25740 1 client.29761 shutdown 2024-03-21T11:35:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.201+0000 7f1a7ca25740 1 -- 172.21.15.47:0/990989705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 msgr2=0x5630ecf47370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.201+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf47370 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00cba0 tx=0x7f1a6c09bcb0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.202+0000 7f1a7ca25740 1 -- 172.21.15.47:0/990989705 shutdown_connections 2024-03-21T11:35:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.202+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5630ecf4bbf0 0x5630ecf4e040 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.202+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf47370 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.202+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf40330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.202+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/990989705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 0x5630ecf3fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.202+0000 7f1a5cc00640 0 client.29761 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.202+0000 7f1a7ca25740 1 -- 172.21.15.47:0/990989705 >> 172.21.15.47:0/990989705 conn(0x5630ecefe200 msgr2=0x5630ecf30a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.202+0000 7f1a7ca25740 1 -- 172.21.15.47:0/990989705 shutdown_connections 2024-03-21T11:35:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.202+0000 7f1a7ca25740 1 -- 172.21.15.47:0/990989705 wait complete. 2024-03-21T11:35:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.202+0000 7f1a7ca25740 20 client.29761 trim_cache size 0 max 16384 2024-03-21T11:35:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.220+0000 7f1a7ca25740 1 Processor -- start 2024-03-21T11:35:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.220+0000 7f1a7ca25740 1 -- start start 2024-03-21T11:35:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.220+0000 7f1a7ca25740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 0x5630ecf34a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.220+0000 7f1a7ca25740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf47330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.220+0000 7f1a7ca25740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf34f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.220+0000 7f1a7ca25740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a741ac1b0 con 0x5630ecf35e80 2024-03-21T11:35:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.220+0000 7f1a7ca25740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a741adb30 con 0x5630ecf36870 2024-03-21T11:35:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.220+0000 7f1a7ca25740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a741ac810 con 0x5630ecf3b9e0 2024-03-21T11:35:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.221+0000 7f1a73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf34f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.221+0000 7f1a72000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf47330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.221+0000 7f1a73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf34f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41738/0 (socket says 172.21.15.47:41738) 2024-03-21T11:35:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.221+0000 7f1a72a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 0x5630ecf34a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.221+0000 7f1a73400640 1 -- 172.21.15.47:0/1728410799 learned_addr learned my addr 172.21.15.47:0/1728410799 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.221+0000 7f1a73400640 1 -- 172.21.15.47:0/1728410799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 msgr2=0x5630ecf47330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.221+0000 7f1a73400640 1 --2- 172.21.15.47:0/1728410799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf47330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.221+0000 7f1a73400640 1 -- 172.21.15.47:0/1728410799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 msgr2=0x5630ecf34a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.221+0000 7f1a73400640 1 --2- 172.21.15.47:0/1728410799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 0x5630ecf34a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.221+0000 7f1a73400640 1 -- 172.21.15.47:0/1728410799 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630ecb6f0f0 con 0x5630ecf3b9e0 2024-03-21T11:35:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.221+0000 7f1a72000640 1 --2- 172.21.15.47:0/1728410799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf47330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.221+0000 7f1a73400640 1 --2- 172.21.15.47:0/1728410799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf34f70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0a4650 tx=0x7f1a6c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa5f8f9cc620ad57 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:46.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.221+0000 7f1a5f400640 1 -- 172.21.15.47:0/1728410799 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c07f070 con 0x5630ecf3b9e0 2024-03-21T11:35:46.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.222+0000 7f1a5f400640 1 -- 172.21.15.47:0/1728410799 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a6c009820 con 0x5630ecf3b9e0 2024-03-21T11:35:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.222+0000 7f1a5f400640 1 -- 172.21.15.47:0/1728410799 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c0a4b80 con 0x5630ecf3b9e0 2024-03-21T11:35:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.222+0000 7f1a7ca25740 1 -- 172.21.15.47:0/1728410799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 msgr2=0x5630ecf34f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.222+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/1728410799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf34f70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0a4650 tx=0x7f1a6c009b50 comp rx=0 tx=0).stop 2024-03-21T11:35:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.223+0000 7f1a7ca25740 1 -- 172.21.15.47:0/1728410799 shutdown_connections 2024-03-21T11:35:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.223+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/1728410799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf3b9e0 0x5630ecf34f70 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.223+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/1728410799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf36870 0x5630ecf47330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.223+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/1728410799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf35e80 0x5630ecf34a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.223+0000 7f1a7ca25740 1 -- 172.21.15.47:0/1728410799 >> 172.21.15.47:0/1728410799 conn(0x5630ecefe200 msgr2=0x5630ecf37ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.223+0000 7f1a7ca25740 1 -- 172.21.15.47:0/1728410799 shutdown_connections 2024-03-21T11:35:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.223+0000 7f1a7ca25740 1 -- 172.21.15.47:0/1728410799 wait complete. 2024-03-21T11:35:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.223+0000 7f1a7ca25740 1 -- start start 2024-03-21T11:35:46.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.223+0000 7f1a7ca25740 1 Processor -- start 2024-03-21T11:35:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a7ca25740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf35e80 0x5630ecf44a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a7ca25740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf36870 0x5630ecf44f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a7ca25740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf3b9e0 0x5630ecf45f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a7ca25740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a741ad4d0 con 0x5630ecf3b9e0 2024-03-21T11:35:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a7ca25740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a741ace70 con 0x5630ecf35e80 2024-03-21T11:35:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a7ca25740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a741aa370 con 0x5630ecf36870 2024-03-21T11:35:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a7ca25740 10 client.0 _ll_register_callbacks cb 0x7f1a6c003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a72a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf35e80 0x5630ecf44a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a72a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf35e80 0x5630ecf44a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49908/0 (socket says 172.21.15.47:49908) 2024-03-21T11:35:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a72000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf36870 0x5630ecf44f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a72a00640 1 -- 172.21.15.47:0/925684897 learned_addr learned my addr 172.21.15.47:0/925684897 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a73400640 1 --2- 172.21.15.47:0/925684897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf3b9e0 0x5630ecf45f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a72a00640 1 -- 172.21.15.47:0/925684897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf36870 msgr2=0x5630ecf44f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:46.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a72a00640 1 --2- 172.21.15.47:0/925684897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf36870 0x5630ecf44f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a72a00640 1 -- 172.21.15.47:0/925684897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf3b9e0 msgr2=0x5630ecf45f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:46.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.224+0000 7f1a72a00640 1 --2- 172.21.15.47:0/925684897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf3b9e0 0x5630ecf45f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.225+0000 7f1a72a00640 1 -- 172.21.15.47:0/925684897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630ecf46560 con 0x5630ecf35e80 2024-03-21T11:35:46.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.225+0000 7f1a72000640 1 --2- 172.21.15.47:0/925684897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf36870 0x5630ecf44f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:46.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.225+0000 7f1a72a00640 1 --2- 172.21.15.47:0/925684897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf35e80 0x5630ecf44a20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1a64005ab0 tx=0x7f1a6402ca80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef753e0a36a6d4a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:46.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.225+0000 7f1a5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:46.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.225+0000 7f1a5cc00640 1 -- 172.21.15.47:0/925684897 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64042040 con 0x5630ecf35e80 2024-03-21T11:35:46.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.225+0000 7f1a5cc00640 1 -- 172.21.15.47:0/925684897 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1a64004040 con 0x5630ecf35e80 2024-03-21T11:35:46.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.225+0000 7f1a7ca25740 1 -- 172.21.15.47:0/925684897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a6c009820 con 0x5630ecf35e80 2024-03-21T11:35:46.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.225+0000 7f1a5cc00640 1 -- 172.21.15.47:0/925684897 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a6402cb70 con 0x5630ecf35e80 2024-03-21T11:35:46.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.226+0000 7f1a7ca25740 10 client.29729 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:46.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.226+0000 7f1a7ca25740 1 -- 172.21.15.47:0/925684897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5630ecfa74a0 con 0x5630ecf35e80 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.226+0000 7f1a5cc00640 1 -- 172.21.15.47:0/925684897 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a6c009820 con 0x5630ecf35e80 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.226+0000 7f1a5cc00640 1 -- 172.21.15.47:0/925684897 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1a64004240 con 0x5630ecf35e80 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.226+0000 7f1a7ca25740 10 client.29729 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.226+0000 7f1a7ca25740 10 client.29729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.226+0000 7f1a7ca25740 20 client.29729 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.226+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/925684897 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5630ecfa79c0 0x5630ecfa9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.226+0000 7f1a7ca25740 4 client.29729 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.226+0000 7f1a7ca25740 1 -- 172.21.15.47:0/925684897 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5630ecea2ae0 con 0x5630ecfa79c0 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.226+0000 7f1a72000640 1 --2- 172.21.15.47:0/925684897 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5630ecfa79c0 0x5630ecfa9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.227+0000 7f1a72000640 1 --2- 172.21.15.47:0/925684897 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5630ecfa79c0 0x5630ecfa9de0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bda10c0839cad8c server_cookie=28dd1650f25493a1 in_seq=0 out_seq=0 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.227+0000 7f1a5cc00640 10 client.29729 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.227+0000 7f1a5cc00640 1 -- 172.21.15.47:0/925684897 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5630ecea2ae0 con 0x5630ecfa79c0 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.227+0000 7f1a5cc00640 10 client.29729 handle_command_reply: tid=0 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.227+0000 7f1a7ca25740 1 client.29729 shutdown 2024-03-21T11:35:46.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.227+0000 7f1a7ca25740 1 -- 172.21.15.47:0/925684897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf35e80 msgr2=0x5630ecf44a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.227+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/925684897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf35e80 0x5630ecf44a20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1a64005ab0 tx=0x7f1a6402ca80 comp rx=0 tx=0).stop 2024-03-21T11:35:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.227+0000 7f1a7ca25740 1 -- 172.21.15.47:0/925684897 shutdown_connections 2024-03-21T11:35:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.227+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/925684897 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5630ecfa79c0 0x5630ecfa9de0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.227+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/925684897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630ecf3b9e0 0x5630ecf45f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.228+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/925684897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630ecf36870 0x5630ecf44f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.228+0000 7f1a7ca25740 1 --2- 172.21.15.47:0/925684897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630ecf35e80 0x5630ecf44a20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.228+0000 7f1a7ca25740 1 -- 172.21.15.47:0/925684897 >> 172.21.15.47:0/925684897 conn(0x5630ecefe200 msgr2=0x5630ecf37ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.228+0000 7f1a5cc00640 0 client.29729 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.228+0000 7f1a7ca25740 1 -- 172.21.15.47:0/925684897 shutdown_connections 2024-03-21T11:35:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.228+0000 7f1a7ca25740 1 -- 172.21.15.47:0/925684897 wait complete. 2024-03-21T11:35:46.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.228+0000 7f1a7ca25740 20 client.29729 trim_cache size 0 max 16384 2024-03-21T11:35:46.234 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:35:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:35:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:35:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "5a639d18-a42d-44cd-a480-b12a74cfce2a": { 2024-03-21T11:35:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:35:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "5a639d18-a42d-44cd-a480-b12a74cfce2a", 2024-03-21T11:35:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:35:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:35:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:35:46.234 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:35:46.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.228+0000 7f1a79c00640 1 -- 172.21.15.47:0/1418564289 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a580a2f20 msgr2=0x7f1a580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:46.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.228+0000 7f1a79c00640 1 --2- 172.21.15.47:0/1418564289 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a580a2f20 0x7f1a580a53e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x5630ecf370d0 tx=0x7f1a60005850 comp rx=0 tx=0).stop 2024-03-21T11:35:46.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.228+0000 7f1a79c00640 1 -- 172.21.15.47:0/1418564289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a74103260 msgr2=0x7f1a741a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:46.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.228+0000 7f1a79c00640 1 --2- 172.21.15.47:0/1418564289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a74103260 0x7f1a741a11a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00ec50 tx=0x7f1a6c00c6a0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.229+0000 7f1a79c00640 1 -- 172.21.15.47:0/1418564289 shutdown_connections 2024-03-21T11:35:46.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.229+0000 7f1a79c00640 1 --2- 172.21.15.47:0/1418564289 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a580a2f20 0x7f1a580a53e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.229+0000 7f1a79c00640 1 --2- 172.21.15.47:0/1418564289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a74103260 0x7f1a741a11a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.229+0000 7f1a79c00640 1 --2- 172.21.15.47:0/1418564289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a741028a0 0x7f1a7419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.229+0000 7f1a79c00640 1 --2- 172.21.15.47:0/1418564289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a741016b0 0x7f1a74199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:46.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.229+0000 7f1a79c00640 1 -- 172.21.15.47:0/1418564289 >> 172.21.15.47:0/1418564289 conn(0x7f1a740fcc40 msgr2=0x7f1a741080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:46.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.229+0000 7f1a79c00640 1 -- 172.21.15.47:0/1418564289 shutdown_connections 2024-03-21T11:35:46.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:35:46.229+0000 7f1a79c00640 1 -- 172.21.15.47:0/1418564289 wait complete. 2024-03-21T11:35:46.250 INFO:tasks.cephfs.filesystem:scrub status for tag:5a639d18-a42d-44cd-a480-b12a74cfce2a - {'path': '/', 'tag': '5a639d18-a42d-44cd-a480-b12a74cfce2a', 'options': 'recursive,force'} 2024-03-21T11:35:46.572 INFO:tasks.workunit.client.0.smithi047.stdout: 1 943898 7.97 MB/sec execute 510 sec latency 13.919 ms 2024-03-21T11:35:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:46 smithi102 ceph-mon[36748]: pgmap v773: 129 pgs: 129 active+clean; 19 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.9 MiB/s wr, 277 op/s 2024-03-21T11:35:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:46 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:35:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:46 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:35:47.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:46 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:35:47.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:46 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:35:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:46 smithi047 ceph-mon[31351]: pgmap v773: 129 pgs: 129 active+clean; 19 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.9 MiB/s wr, 277 op/s 2024-03-21T11:35:47.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:46 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:35:47.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:46 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:35:47.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:46 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:35:47.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:46 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:35:47.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:46 smithi100 ceph-mon[36245]: pgmap v773: 129 pgs: 129 active+clean; 19 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.9 MiB/s wr, 277 op/s 2024-03-21T11:35:47.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:46 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:35:47.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:46 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:35:47.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:46 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:35:47.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:46 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:35:47.572 INFO:tasks.workunit.client.0.smithi047.stdout: 1 945315 7.98 MB/sec execute 511 sec latency 39.261 ms 2024-03-21T11:35:48.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:47 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:35:48.294 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:47 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:35:48.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:47 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:35:48.572 INFO:tasks.workunit.client.0.smithi047.stdout: 1 946017 7.97 MB/sec execute 512 sec latency 62.777 ms 2024-03-21T11:35:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:48 smithi102 ceph-mon[36748]: pgmap v774: 129 pgs: 129 active+clean; 20 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.9 MiB/s wr, 309 op/s 2024-03-21T11:35:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:48 smithi047 ceph-mon[31351]: pgmap v774: 129 pgs: 129 active+clean; 20 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.9 MiB/s wr, 309 op/s 2024-03-21T11:35:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:48 smithi100 ceph-mon[36245]: pgmap v774: 129 pgs: 129 active+clean; 20 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.9 MiB/s wr, 309 op/s 2024-03-21T11:35:49.573 INFO:tasks.workunit.client.0.smithi047.stdout: 1 947707 7.97 MB/sec execute 513 sec latency 24.806 ms 2024-03-21T11:35:50.573 INFO:tasks.workunit.client.0.smithi047.stdout: 1 949070 7.97 MB/sec execute 514 sec latency 13.636 ms 2024-03-21T11:35:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:50 smithi102 ceph-mon[36748]: pgmap v775: 129 pgs: 129 active+clean; 19 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.9 MiB/s wr, 281 op/s 2024-03-21T11:35:51.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:50 smithi047 ceph-mon[31351]: pgmap v775: 129 pgs: 129 active+clean; 19 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.9 MiB/s wr, 281 op/s 2024-03-21T11:35:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:50 smithi100 ceph-mon[36245]: pgmap v775: 129 pgs: 129 active+clean; 19 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.9 MiB/s wr, 281 op/s 2024-03-21T11:35:51.572 INFO:tasks.workunit.client.0.smithi047.stdout: 1 950290 7.97 MB/sec execute 515 sec latency 14.552 ms 2024-03-21T11:35:52.573 INFO:tasks.workunit.client.0.smithi047.stdout: 1 952157 7.97 MB/sec execute 516 sec latency 21.757 ms 2024-03-21T11:35:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:52 smithi102 ceph-mon[36748]: pgmap v776: 129 pgs: 129 active+clean; 22 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 2.0 MiB/s wr, 306 op/s 2024-03-21T11:35:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:52 smithi047 ceph-mon[31351]: pgmap v776: 129 pgs: 129 active+clean; 22 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 2.0 MiB/s wr, 306 op/s 2024-03-21T11:35:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:52 smithi100 ceph-mon[36245]: pgmap v776: 129 pgs: 129 active+clean; 22 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 2.0 MiB/s wr, 306 op/s 2024-03-21T11:35:53.573 INFO:tasks.workunit.client.0.smithi047.stdout: 1 952918 7.96 MB/sec execute 517 sec latency 49.659 ms 2024-03-21T11:35:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.997+0000 7f2dae400640 1 -- 172.21.15.47:0/2104147689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da81038f0 msgr2=0x7f2da8103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.997+0000 7f2dae400640 1 --2- 172.21.15.47:0/2104147689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da81038f0 0x7f2da8103d10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009860 tx=0x7f2d9802bb70 comp rx=0 tx=0).stop 2024-03-21T11:35:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.997+0000 7f2dae400640 1 -- 172.21.15.47:0/2104147689 shutdown_connections 2024-03-21T11:35:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.997+0000 7f2dae400640 1 --2- 172.21.15.47:0/2104147689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da81054a0 0x7f2da810bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.997+0000 7f2dae400640 1 --2- 172.21.15.47:0/2104147689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da8104ae0 0x7f2da8104f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.997+0000 7f2dae400640 1 --2- 172.21.15.47:0/2104147689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da81038f0 0x7f2da8103d10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.997+0000 7f2dae400640 1 -- 172.21.15.47:0/2104147689 >> 172.21.15.47:0/2104147689 conn(0x7f2da80fee60 msgr2=0x7f2da81012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.997+0000 7f2dae400640 1 -- 172.21.15.47:0/2104147689 shutdown_connections 2024-03-21T11:35:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.997+0000 7f2dae400640 1 -- 172.21.15.47:0/2104147689 wait complete. 2024-03-21T11:35:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.999+0000 7f2dae400640 1 Processor -- start 2024-03-21T11:35:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.999+0000 7f2dae400640 1 -- start start 2024-03-21T11:35:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.999+0000 7f2dae400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da81038f0 0x7f2da8199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.999+0000 7f2dae400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da8104ae0 0x7f2da819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.999+0000 7f2dae400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da81054a0 0x7f2da81a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.999+0000 7f2dae400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da810f560 con 0x7f2da8104ae0 2024-03-21T11:35:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.999+0000 7f2dae400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da810f3e0 con 0x7f2da81054a0 2024-03-21T11:35:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:53.999+0000 7f2dae400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da810f6e0 con 0x7f2da81038f0 2024-03-21T11:35:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.000+0000 7f2da7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da81038f0 0x7f2da8199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.000+0000 7f2da7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da81038f0 0x7f2da8199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54222/0 (socket says 172.21.15.47:54222) 2024-03-21T11:35:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.000+0000 7f2da7400640 1 -- 172.21.15.47:0/339926474 learned_addr learned my addr 172.21.15.47:0/339926474 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.000+0000 7f2da7e00640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da81054a0 0x7f2da81a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.000+0000 7f2da6a00640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da8104ae0 0x7f2da819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.000+0000 7f2da7400640 1 -- 172.21.15.47:0/339926474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da81054a0 msgr2=0x7f2da81a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.000+0000 7f2da7400640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da81054a0 0x7f2da81a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.000+0000 7f2da7400640 1 -- 172.21.15.47:0/339926474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da8104ae0 msgr2=0x7f2da819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.000+0000 7f2da7400640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da8104ae0 0x7f2da819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.000+0000 7f2da7400640 1 -- 172.21.15.47:0/339926474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da81a1790 con 0x7f2da81038f0 2024-03-21T11:35:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.000+0000 7f2da7e00640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da81054a0 0x7f2da81a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:35:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.000+0000 7f2da7400640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da81038f0 0x7f2da8199b70 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2d980349e0 tx=0x7f2d98032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.001+0000 7f2d9fe00640 1 -- 172.21.15.47:0/339926474 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98041020 con 0x7f2da81038f0 2024-03-21T11:35:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.001+0000 7f2d9fe00640 1 -- 172.21.15.47:0/339926474 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d9802c9f0 con 0x7f2da81038f0 2024-03-21T11:35:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.001+0000 7f2d9fe00640 1 -- 172.21.15.47:0/339926474 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98039040 con 0x7f2da81038f0 2024-03-21T11:35:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.001+0000 7f2dae400640 1 -- 172.21.15.47:0/339926474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da81a1a20 con 0x7f2da81038f0 2024-03-21T11:35:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.001+0000 7f2dae400640 1 -- 172.21.15.47:0/339926474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da81a1ec0 con 0x7f2da81038f0 2024-03-21T11:35:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.004+0000 7f2d9fe00640 1 -- 172.21.15.47:0/339926474 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2d98044020 con 0x7f2da81038f0 2024-03-21T11:35:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.004+0000 7f2db1064740 1 Processor -- start 2024-03-21T11:35:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.005+0000 7f2db1064740 1 -- start start 2024-03-21T11:35:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.005+0000 7f2db1064740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00e1e0 0x556e3d00e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.005+0000 7f2db1064740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00ebd0 0x556e3d0136d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.005+0000 7f2db1064740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d013d40 0x556e3d0161d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.005+0000 7f2db1064740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3cfd13b0 con 0x556e3d00ebd0 2024-03-21T11:35:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.005+0000 7f2db1064740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3cf73830 con 0x556e3d00e1e0 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.005+0000 7f2db1064740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3cf1c5d0 con 0x556e3d013d40 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.005+0000 7f2da7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00e1e0 0x556e3d00e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.005+0000 7f2da7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00e1e0 0x556e3d00e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34450/0 (socket says 172.21.15.47:34450) 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.005+0000 7f2da7e00640 1 -- 172.21.15.47:0/1701392274 learned_addr learned my addr 172.21.15.47:0/1701392274 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.005+0000 7f2da6a00640 1 --2- 172.21.15.47:0/1701392274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d013d40 0x556e3d0161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.006+0000 7f2da7400640 1 --2- 172.21.15.47:0/1701392274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00ebd0 0x556e3d0136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.006+0000 7f2da7e00640 1 -- 172.21.15.47:0/1701392274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d013d40 msgr2=0x556e3d0161d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.006+0000 7f2da7e00640 1 --2- 172.21.15.47:0/1701392274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d013d40 0x556e3d0161d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.006+0000 7f2da7e00640 1 -- 172.21.15.47:0/1701392274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00ebd0 msgr2=0x556e3d0136d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.006+0000 7f2da7e00640 1 --2- 172.21.15.47:0/1701392274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00ebd0 0x556e3d0136d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.006+0000 7f2da7e00640 1 -- 172.21.15.47:0/1701392274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e3cf342c0 con 0x556e3d00e1e0 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.006+0000 7f2da7400640 1 --2- 172.21.15.47:0/1701392274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00ebd0 0x556e3d0136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.006+0000 7f2da7e00640 1 --2- 172.21.15.47:0/1701392274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00e1e0 0x556e3d00e600 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2da00117e0 tx=0x7f2da0011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec85351e08b3796b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.006+0000 7f2d9fe00640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d8c0a3040 0x7f2d8c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.006+0000 7f2d9cc00640 1 -- 172.21.15.47:0/1701392274 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0011310 con 0x556e3d00e1e0 2024-03-21T11:35:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.006+0000 7f2d9cc00640 1 -- 172.21.15.47:0/1701392274 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2da0003e50 con 0x556e3d00e1e0 2024-03-21T11:35:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.006+0000 7f2d9cc00640 1 -- 172.21.15.47:0/1701392274 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0004620 con 0x556e3d00e1e0 2024-03-21T11:35:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.006+0000 7f2d9fe00640 1 -- 172.21.15.47:0/339926474 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2d980f0f40 con 0x7f2da81038f0 2024-03-21T11:35:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.007+0000 7f2da7400640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d8c0a3040 0x7f2d8c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.007+0000 7f2da7400640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d8c0a3040 0x7f2d8c0a5500 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2d9802c180 tx=0x7f2d98005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.007+0000 7f2db1064740 1 -- 172.21.15.47:0/1701392274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00e1e0 msgr2=0x556e3d00e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.007+0000 7f2db1064740 1 --2- 172.21.15.47:0/1701392274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00e1e0 0x556e3d00e600 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2da00117e0 tx=0x7f2da0011ca0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.008+0000 7f2db1064740 1 -- 172.21.15.47:0/1701392274 shutdown_connections 2024-03-21T11:35:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.008+0000 7f2db1064740 1 --2- 172.21.15.47:0/1701392274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d013d40 0x556e3d0161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.008+0000 7f2db1064740 1 --2- 172.21.15.47:0/1701392274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00ebd0 0x556e3d0136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.008+0000 7f2db1064740 1 --2- 172.21.15.47:0/1701392274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00e1e0 0x556e3d00e600 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.008+0000 7f2db1064740 1 -- 172.21.15.47:0/1701392274 >> 172.21.15.47:0/1701392274 conn(0x556e3d009750 msgr2=0x556e3d009b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.008+0000 7f2db1064740 1 -- 172.21.15.47:0/1701392274 shutdown_connections 2024-03-21T11:35:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.008+0000 7f2db1064740 1 -- 172.21.15.47:0/1701392274 wait complete. 2024-03-21T11:35:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.008+0000 7f2db1064740 1 -- start start 2024-03-21T11:35:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.009+0000 7f2db1064740 1 Processor -- start 2024-03-21T11:35:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.009+0000 7f2db1064740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d00e1e0 0x556e3d0183a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.009+0000 7f2db1064740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00ebd0 0x556e3d0188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.009+0000 7f2db1064740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d013d40 0x556e3d01f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.009+0000 7f2db1064740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3cf2fa20 con 0x556e3d013d40 2024-03-21T11:35:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.009+0000 7f2db1064740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3cf6f8d0 con 0x556e3d00ebd0 2024-03-21T11:35:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.009+0000 7f2db1064740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3d016cf0 con 0x556e3d00e1e0 2024-03-21T11:35:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.009+0000 7f2db1064740 10 client.0 _ll_register_callbacks cb 0x556e3cf6c1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.009+0000 7f2da6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00ebd0 0x556e3d0188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.009+0000 7f2da7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d00e1e0 0x556e3d0183a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.009+0000 7f2da6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00ebd0 0x556e3d0188e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34464/0 (socket says 172.21.15.47:34464) 2024-03-21T11:35:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.009+0000 7f2da6a00640 1 -- 172.21.15.47:0/3959998093 learned_addr learned my addr 172.21.15.47:0/3959998093 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.009+0000 7f2da7e00640 1 --2- 172.21.15.47:0/3959998093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d013d40 0x556e3d01f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.010+0000 7f2da6a00640 1 -- 172.21.15.47:0/3959998093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d00e1e0 msgr2=0x556e3d0183a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.010+0000 7f2da6a00640 1 --2- 172.21.15.47:0/3959998093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d00e1e0 0x556e3d0183a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.010+0000 7f2da6a00640 1 -- 172.21.15.47:0/3959998093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d013d40 msgr2=0x556e3d01f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.010+0000 7f2da6a00640 1 --2- 172.21.15.47:0/3959998093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d013d40 0x556e3d01f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.010+0000 7f2da6a00640 1 -- 172.21.15.47:0/3959998093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e3cc88210 con 0x556e3d00ebd0 2024-03-21T11:35:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.010+0000 7f2da7e00640 1 --2- 172.21.15.47:0/3959998093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d013d40 0x556e3d01f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:35:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.010+0000 7f2da6a00640 1 --2- 172.21.15.47:0/3959998093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00ebd0 0x556e3d0188e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2d940116f0 tx=0x7f2d94011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe6203a7b037dda2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.010+0000 7f2d91600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.010+0000 7f2d91600640 1 -- 172.21.15.47:0/3959998093 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9401e020 con 0x556e3d00ebd0 2024-03-21T11:35:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.010+0000 7f2d91600640 1 -- 172.21.15.47:0/3959998093 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2d94017050 con 0x556e3d00ebd0 2024-03-21T11:35:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.011+0000 7f2db1064740 1 -- 172.21.15.47:0/3959998093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2da0003e50 con 0x556e3d00ebd0 2024-03-21T11:35:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.011+0000 7f2d91600640 1 -- 172.21.15.47:0/3959998093 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d94026050 con 0x556e3d00ebd0 2024-03-21T11:35:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.011+0000 7f2db1064740 10 client.29741 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.011+0000 7f2db1064740 1 -- 172.21.15.47:0/3959998093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556e3cd54620 con 0x556e3d00ebd0 2024-03-21T11:35:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.011+0000 7f2d91600640 1 -- 172.21.15.47:0/3959998093 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2da0003e50 con 0x556e3d00ebd0 2024-03-21T11:35:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.011+0000 7f2d91600640 1 -- 172.21.15.47:0/3959998093 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2d94019070 con 0x556e3d00ebd0 2024-03-21T11:35:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.011+0000 7f2db1064740 10 client.29741 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.011+0000 7f2db1064740 10 client.29741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.012+0000 7f2db1064740 20 client.29741 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.012+0000 7f2db1064740 1 --2- 172.21.15.47:0/3959998093 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3d024220 0x556e3d026610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.012+0000 7f2db1064740 4 client.29741 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:35:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.012+0000 7f2db1064740 1 -- 172.21.15.47:0/3959998093 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e3cf80d70 con 0x556e3d024220 2024-03-21T11:35:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.012+0000 7f2da7e00640 1 --2- 172.21.15.47:0/3959998093 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3d024220 0x556e3d026610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.012+0000 7f2da7e00640 1 --2- 172.21.15.47:0/3959998093 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3d024220 0x556e3d026610 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf08cd88b9ee1c3 server_cookie=81e5777f787d1a24 in_seq=0 out_seq=0 2024-03-21T11:35:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.012+0000 7f2d91600640 10 client.29741 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.013+0000 7f2d91600640 1 -- 172.21.15.47:0/3959998093 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556e3cf80d70 con 0x556e3d024220 2024-03-21T11:35:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.013+0000 7f2d91600640 10 client.29741 handle_command_reply: tid=0 2024-03-21T11:35:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.014+0000 7f2db1064740 1 client.29741 shutdown 2024-03-21T11:35:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.014+0000 7f2db1064740 1 -- 172.21.15.47:0/3959998093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00ebd0 msgr2=0x556e3d0188e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.014+0000 7f2db1064740 1 --2- 172.21.15.47:0/3959998093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00ebd0 0x556e3d0188e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2d940116f0 tx=0x7f2d94011bb0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.014+0000 7f2db1064740 1 -- 172.21.15.47:0/3959998093 shutdown_connections 2024-03-21T11:35:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.014+0000 7f2db1064740 1 --2- 172.21.15.47:0/3959998093 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3d024220 0x556e3d026610 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.014+0000 7f2db1064740 1 --2- 172.21.15.47:0/3959998093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d013d40 0x556e3d01f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.014+0000 7f2db1064740 1 --2- 172.21.15.47:0/3959998093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00ebd0 0x556e3d0188e0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.014+0000 7f2db1064740 1 --2- 172.21.15.47:0/3959998093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d00e1e0 0x556e3d0183a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.014+0000 7f2d91600640 0 client.29741 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.014+0000 7f2db1064740 1 -- 172.21.15.47:0/3959998093 >> 172.21.15.47:0/3959998093 conn(0x556e3d009750 msgr2=0x556e3d008950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.014+0000 7f2db1064740 1 -- 172.21.15.47:0/3959998093 shutdown_connections 2024-03-21T11:35:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.014+0000 7f2db1064740 1 -- 172.21.15.47:0/3959998093 wait complete. 2024-03-21T11:35:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.014+0000 7f2db1064740 20 client.29741 trim_cache size 0 max 16384 2024-03-21T11:35:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.031+0000 7f2db1064740 1 Processor -- start 2024-03-21T11:35:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.031+0000 7f2db1064740 1 -- start start 2024-03-21T11:35:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.031+0000 7f2db1064740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d00e1e0 0x556e3d00ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.031+0000 7f2db1064740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00ebd0 0x556e3d010470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.031+0000 7f2db1064740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d013d40 0x556e3d01f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.031+0000 7f2db1064740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da81ac2c0 con 0x556e3d00ebd0 2024-03-21T11:35:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.031+0000 7f2db1064740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da81adc40 con 0x556e3d013d40 2024-03-21T11:35:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.031+0000 7f2db1064740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da81ac920 con 0x556e3d00e1e0 2024-03-21T11:35:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.031+0000 7f2da7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d00e1e0 0x556e3d00ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.031+0000 7f2da6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00ebd0 0x556e3d010470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.031+0000 7f2da7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d00e1e0 0x556e3d00ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54244/0 (socket says 172.21.15.47:54244) 2024-03-21T11:35:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.032+0000 7f2da7e00640 1 -- 172.21.15.47:0/3476566271 learned_addr learned my addr 172.21.15.47:0/3476566271 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.032+0000 7f2da7e00640 1 --2- 172.21.15.47:0/3476566271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d013d40 0x556e3d01f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.032+0000 7f2da7e00640 1 -- 172.21.15.47:0/3476566271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d013d40 msgr2=0x556e3d01f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.032+0000 7f2da7e00640 1 --2- 172.21.15.47:0/3476566271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d013d40 0x556e3d01f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.032+0000 7f2da7e00640 1 -- 172.21.15.47:0/3476566271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00ebd0 msgr2=0x556e3d010470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.032+0000 7f2da7e00640 1 --2- 172.21.15.47:0/3476566271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00ebd0 0x556e3d010470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.032+0000 7f2da7e00640 1 -- 172.21.15.47:0/3476566271 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e3d026980 con 0x556e3d00e1e0 2024-03-21T11:35:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.032+0000 7f2da7e00640 1 --2- 172.21.15.47:0/3476566271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d00e1e0 0x556e3d00ff30 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2da00117b0 tx=0x7f2da0009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a426f475626803a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.032+0000 7f2d93e00640 1 -- 172.21.15.47:0/3476566271 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da002e0c0 con 0x556e3d00e1e0 2024-03-21T11:35:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.032+0000 7f2d93e00640 1 -- 172.21.15.47:0/3476566271 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2da002e260 con 0x556e3d00e1e0 2024-03-21T11:35:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.032+0000 7f2d93e00640 1 -- 172.21.15.47:0/3476566271 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da002ed10 con 0x556e3d00e1e0 2024-03-21T11:35:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.033+0000 7f2db1064740 1 -- 172.21.15.47:0/3476566271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d00e1e0 msgr2=0x556e3d00ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.033+0000 7f2db1064740 1 --2- 172.21.15.47:0/3476566271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d00e1e0 0x556e3d00ff30 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2da00117b0 tx=0x7f2da0009500 comp rx=0 tx=0).stop 2024-03-21T11:35:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.033+0000 7f2db1064740 1 -- 172.21.15.47:0/3476566271 shutdown_connections 2024-03-21T11:35:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.033+0000 7f2db1064740 1 --2- 172.21.15.47:0/3476566271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d013d40 0x556e3d01f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.033+0000 7f2db1064740 1 --2- 172.21.15.47:0/3476566271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00ebd0 0x556e3d010470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.033+0000 7f2db1064740 1 --2- 172.21.15.47:0/3476566271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d00e1e0 0x556e3d00ff30 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.033+0000 7f2db1064740 1 -- 172.21.15.47:0/3476566271 >> 172.21.15.47:0/3476566271 conn(0x556e3d009750 msgr2=0x556e3d009b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.033+0000 7f2db1064740 1 -- 172.21.15.47:0/3476566271 shutdown_connections 2024-03-21T11:35:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.033+0000 7f2db1064740 1 -- 172.21.15.47:0/3476566271 wait complete. 2024-03-21T11:35:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.034+0000 7f2db1064740 1 -- start start 2024-03-21T11:35:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.034+0000 7f2db1064740 1 Processor -- start 2024-03-21T11:35:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.034+0000 7f2db1064740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00e1e0 0x556e3d01f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.034+0000 7f2db1064740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00ebd0 0x556e3d01f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.034+0000 7f2db1064740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d013d40 0x556e3d01d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.034+0000 7f2db1064740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da81ad5e0 con 0x556e3d00e1e0 2024-03-21T11:35:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.034+0000 7f2db1064740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da81acf80 con 0x556e3d00ebd0 2024-03-21T11:35:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2db1064740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da81aa480 con 0x556e3d013d40 2024-03-21T11:35:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2db1064740 10 client.0 _ll_register_callbacks cb 0x7f2d94009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:35:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2da6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00ebd0 0x556e3d01f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2da6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00ebd0 0x556e3d01f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34486/0 (socket says 172.21.15.47:34486) 2024-03-21T11:35:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2da6a00640 1 -- 172.21.15.47:0/42266046 learned_addr learned my addr 172.21.15.47:0/42266046 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:35:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2da7e00640 1 --2- 172.21.15.47:0/42266046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00e1e0 0x556e3d01f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2da7e00640 1 --2- 172.21.15.47:0/42266046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d013d40 0x556e3d01d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2da6a00640 1 -- 172.21.15.47:0/42266046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d013d40 msgr2=0x556e3d01d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2da6a00640 1 --2- 172.21.15.47:0/42266046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d013d40 0x556e3d01d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2da6a00640 1 -- 172.21.15.47:0/42266046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00e1e0 msgr2=0x556e3d01f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2da6a00640 1 --2- 172.21.15.47:0/42266046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00e1e0 0x556e3d01f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2da6a00640 1 -- 172.21.15.47:0/42266046 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e3d01ded0 con 0x556e3d00ebd0 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2da7e00640 1 --2- 172.21.15.47:0/42266046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d013d40 0x556e3d01d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2da6a00640 1 --2- 172.21.15.47:0/42266046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00ebd0 0x556e3d01f6c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x556e3d019880 tx=0x7f2d94009e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfe8a42aa5449989 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.035+0000 7f2d91600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.036+0000 7f2d91600640 1 -- 172.21.15.47:0/42266046 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9402e070 con 0x556e3d00ebd0 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.036+0000 7f2d91600640 1 -- 172.21.15.47:0/42266046 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2d94005070 con 0x556e3d00ebd0 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.036+0000 7f2db1064740 1 -- 172.21.15.47:0/42266046 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2da002e260 con 0x556e3d00ebd0 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.036+0000 7f2d91600640 1 -- 172.21.15.47:0/42266046 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d9403a050 con 0x556e3d00ebd0 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.036+0000 7f2db1064740 10 client.29747 fetch_fsmap learned FSMap version 28 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.036+0000 7f2db1064740 1 -- 172.21.15.47:0/42266046 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556e3d080520 con 0x556e3d00ebd0 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.037+0000 7f2d91600640 1 -- 172.21.15.47:0/42266046 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2da002e260 con 0x556e3d00ebd0 2024-03-21T11:35:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.037+0000 7f2d91600640 1 -- 172.21.15.47:0/42266046 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2d94019040 con 0x556e3d00ebd0 2024-03-21T11:35:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.037+0000 7f2db1064740 10 client.29747 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:35:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.037+0000 7f2db1064740 10 client.29747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:35:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.037+0000 7f2db1064740 20 client.29747 populate_metadata read hostname 'smithi047' 2024-03-21T11:35:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.037+0000 7f2db1064740 1 --2- 172.21.15.47:0/42266046 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3d080e50 0x556e3d083270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:35:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.037+0000 7f2db1064740 4 client.29747 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:35:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.037+0000 7f2db1064740 1 -- 172.21.15.47:0/42266046 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556e3d004a80 con 0x556e3d080e50 2024-03-21T11:35:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.037+0000 7f2da7e00640 1 --2- 172.21.15.47:0/42266046 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3d080e50 0x556e3d083270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:35:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.037+0000 7f2da7e00640 1 --2- 172.21.15.47:0/42266046 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3d080e50 0x556e3d083270 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8133c260556f9fd2 server_cookie=f899d120127da463 in_seq=0 out_seq=0 2024-03-21T11:35:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.037+0000 7f2d91600640 10 client.29747 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.313+0000 7f2d91600640 1 -- 172.21.15.47:0/42266046 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556e3d004a80 con 0x556e3d080e50 2024-03-21T11:35:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.313+0000 7f2d91600640 10 client.29747 handle_command_reply: tid=0 2024-03-21T11:35:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.313+0000 7f2db1064740 1 client.29747 shutdown 2024-03-21T11:35:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.313+0000 7f2db1064740 1 -- 172.21.15.47:0/42266046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00ebd0 msgr2=0x556e3d01f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.313+0000 7f2db1064740 1 --2- 172.21.15.47:0/42266046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00ebd0 0x556e3d01f6c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x556e3d019880 tx=0x7f2d94009e00 comp rx=0 tx=0).stop 2024-03-21T11:35:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.314+0000 7f2db1064740 1 -- 172.21.15.47:0/42266046 shutdown_connections 2024-03-21T11:35:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.314+0000 7f2db1064740 1 --2- 172.21.15.47:0/42266046 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3d080e50 0x556e3d083270 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.314+0000 7f2db1064740 1 --2- 172.21.15.47:0/42266046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3d013d40 0x556e3d01d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.314+0000 7f2db1064740 1 --2- 172.21.15.47:0/42266046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3d00ebd0 0x556e3d01f6c0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.314+0000 7f2db1064740 1 --2- 172.21.15.47:0/42266046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3d00e1e0 0x556e3d01f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.314+0000 7f2db1064740 1 -- 172.21.15.47:0/42266046 >> 172.21.15.47:0/42266046 conn(0x556e3d009750 msgr2=0x556e3d018340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.314+0000 7f2d91600640 0 client.29747 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:35:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.314+0000 7f2db1064740 1 -- 172.21.15.47:0/42266046 shutdown_connections 2024-03-21T11:35:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.314+0000 7f2db1064740 1 -- 172.21.15.47:0/42266046 wait complete. 2024-03-21T11:35:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.314+0000 7f2db1064740 20 client.29747 trim_cache size 0 max 16384 2024-03-21T11:35:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:35:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:35:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:35:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:35:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.315+0000 7f2dae400640 1 -- 172.21.15.47:0/339926474 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d8c0a3040 msgr2=0x7f2d8c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.315+0000 7f2dae400640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d8c0a3040 0x7f2d8c0a5500 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2d9802c180 tx=0x7f2d98005c90 comp rx=0 tx=0).stop 2024-03-21T11:35:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.315+0000 7f2dae400640 1 -- 172.21.15.47:0/339926474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da81038f0 msgr2=0x7f2da8199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:35:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.315+0000 7f2dae400640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da81038f0 0x7f2da8199b70 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2d980349e0 tx=0x7f2d98032430 comp rx=0 tx=0).stop 2024-03-21T11:35:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.316+0000 7f2dae400640 1 -- 172.21.15.47:0/339926474 shutdown_connections 2024-03-21T11:35:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.316+0000 7f2dae400640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d8c0a3040 0x7f2d8c0a5500 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.316+0000 7f2dae400640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da81054a0 0x7f2da81a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.316+0000 7f2dae400640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da8104ae0 0x7f2da819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.316+0000 7f2dae400640 1 --2- 172.21.15.47:0/339926474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da81038f0 0x7f2da8199b70 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:35:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.316+0000 7f2dae400640 1 -- 172.21.15.47:0/339926474 >> 172.21.15.47:0/339926474 conn(0x7f2da80fee60 msgr2=0x7f2da8107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:35:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.316+0000 7f2dae400640 1 -- 172.21.15.47:0/339926474 shutdown_connections 2024-03-21T11:35:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:35:54.316+0000 7f2dae400640 1 -- 172.21.15.47:0/339926474 wait complete. 2024-03-21T11:35:54.573 INFO:tasks.workunit.client.0.smithi047.stdout: 1 954152 7.96 MB/sec execute 518 sec latency 146.706 ms 2024-03-21T11:35:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:54 smithi102 ceph-mon[36748]: pgmap v777: 129 pgs: 129 active+clean; 21 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 2.1 MiB/s wr, 311 op/s 2024-03-21T11:35:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:54 smithi047 ceph-mon[31351]: pgmap v777: 129 pgs: 129 active+clean; 21 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 2.1 MiB/s wr, 311 op/s 2024-03-21T11:35:55.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:54 smithi100 ceph-mon[36245]: pgmap v777: 129 pgs: 129 active+clean; 21 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 2.1 MiB/s wr, 311 op/s 2024-03-21T11:35:55.573 INFO:tasks.workunit.client.0.smithi047.stdout: 1 955956 7.97 MB/sec execute 519 sec latency 14.999 ms 2024-03-21T11:35:56.573 INFO:tasks.workunit.client.0.smithi047.stdout: 1 956851 7.96 MB/sec execute 520 sec latency 13.566 ms 2024-03-21T11:35:57.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:56 smithi102 ceph-mon[36748]: pgmap v778: 129 pgs: 129 active+clean; 20 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.6 MiB/s wr, 200 op/s 2024-03-21T11:35:57.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:56 smithi047 ceph-mon[31351]: pgmap v778: 129 pgs: 129 active+clean; 20 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.6 MiB/s wr, 200 op/s 2024-03-21T11:35:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:56 smithi100 ceph-mon[36245]: pgmap v778: 129 pgs: 129 active+clean; 20 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.6 MiB/s wr, 200 op/s 2024-03-21T11:35:57.573 INFO:tasks.workunit.client.0.smithi047.stdout: 1 958765 7.96 MB/sec execute 521 sec latency 29.389 ms 2024-03-21T11:35:58.574 INFO:tasks.workunit.client.0.smithi047.stdout: 1 959886 7.96 MB/sec execute 522 sec latency 43.525 ms 2024-03-21T11:35:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:35:58 smithi102 ceph-mon[36748]: pgmap v779: 129 pgs: 129 active+clean; 18 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-03-21T11:35:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:35:58 smithi047 ceph-mon[31351]: pgmap v779: 129 pgs: 129 active+clean; 18 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-03-21T11:35:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:35:58 smithi100 ceph-mon[36245]: pgmap v779: 129 pgs: 129 active+clean; 18 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-03-21T11:35:59.574 INFO:tasks.workunit.client.0.smithi047.stdout: 1 961640 7.96 MB/sec execute 523 sec latency 30.264 ms 2024-03-21T11:36:00.574 INFO:tasks.workunit.client.0.smithi047.stdout: 1 963144 7.97 MB/sec execute 524 sec latency 14.019 ms 2024-03-21T11:36:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:00 smithi102 ceph-mon[36748]: pgmap v780: 129 pgs: 129 active+clean; 21 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-03-21T11:36:01.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:00 smithi047 ceph-mon[31351]: pgmap v780: 129 pgs: 129 active+clean; 21 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-03-21T11:36:01.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:00 smithi100 ceph-mon[36245]: pgmap v780: 129 pgs: 129 active+clean; 21 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-03-21T11:36:01.574 INFO:tasks.workunit.client.0.smithi047.stdout: 1 964086 7.95 MB/sec execute 525 sec latency 16.571 ms 2024-03-21T11:36:02.574 INFO:tasks.workunit.client.0.smithi047.stdout: 1 966173 7.96 MB/sec execute 526 sec latency 16.145 ms 2024-03-21T11:36:03.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:02 smithi047 ceph-mon[31351]: pgmap v781: 129 pgs: 129 active+clean; 21 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.9 MiB/s wr, 267 op/s 2024-03-21T11:36:03.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:02 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:36:03.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:02 smithi100 ceph-mon[36245]: pgmap v781: 129 pgs: 129 active+clean; 21 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.9 MiB/s wr, 267 op/s 2024-03-21T11:36:03.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:02 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:36:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:02 smithi102 ceph-mon[36748]: pgmap v781: 129 pgs: 129 active+clean; 21 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.9 MiB/s wr, 267 op/s 2024-03-21T11:36:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:02 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:36:03.574 INFO:tasks.workunit.client.0.smithi047.stdout: 1 966974 7.96 MB/sec execute 527 sec latency 94.680 ms 2024-03-21T11:36:04.574 INFO:tasks.workunit.client.0.smithi047.stdout: 1 968579 7.95 MB/sec execute 528 sec latency 19.811 ms 2024-03-21T11:36:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:04 smithi047 ceph-mon[31351]: pgmap v782: 129 pgs: 129 active+clean; 22 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-03-21T11:36:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:04 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:36:05.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:04 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:36:05.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:04 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:36:05.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:04 smithi100 ceph-mon[36245]: pgmap v782: 129 pgs: 129 active+clean; 22 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-03-21T11:36:05.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:04 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:36:05.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:04 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:36:05.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:04 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:36:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:04 smithi102 ceph-mon[36748]: pgmap v782: 129 pgs: 129 active+clean; 22 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-03-21T11:36:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:04 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:36:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:04 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:36:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:04 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:36:05.574 INFO:tasks.workunit.client.0.smithi047.stdout: 1 970137 7.96 MB/sec execute 529 sec latency 30.117 ms 2024-03-21T11:36:06.574 INFO:tasks.workunit.client.0.smithi047.stdout: 1 970848 7.95 MB/sec execute 530 sec latency 15.503 ms 2024-03-21T11:36:07.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:07 smithi047 ceph-mon[31351]: pgmap v783: 129 pgs: 129 active+clean; 30 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 2.6 MiB/s wr, 266 op/s 2024-03-21T11:36:07.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:07 smithi100 ceph-mon[36245]: pgmap v783: 129 pgs: 129 active+clean; 30 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 2.6 MiB/s wr, 266 op/s 2024-03-21T11:36:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.473+0000 7fc333a00640 1 -- 172.21.15.47:0/3316964991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc32c104af0 msgr2=0x7fc32c104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.473+0000 7fc333a00640 1 --2- 172.21.15.47:0/3316964991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc32c104af0 0x7fc32c104f70 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc314009900 tx=0x7fc31402bc40 comp rx=0 tx=0).stop 2024-03-21T11:36:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.473+0000 7fc333a00640 1 -- 172.21.15.47:0/3316964991 shutdown_connections 2024-03-21T11:36:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.473+0000 7fc333a00640 1 --2- 172.21.15.47:0/3316964991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc32c1054b0 0x7fc32c10bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.473+0000 7fc333a00640 1 --2- 172.21.15.47:0/3316964991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc32c104af0 0x7fc32c104f70 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.473+0000 7fc333a00640 1 --2- 172.21.15.47:0/3316964991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc32c103900 0x7fc32c103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.473+0000 7fc333a00640 1 -- 172.21.15.47:0/3316964991 >> 172.21.15.47:0/3316964991 conn(0x7fc32c0fee90 msgr2=0x7fc32c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.473+0000 7fc333a00640 1 -- 172.21.15.47:0/3316964991 shutdown_connections 2024-03-21T11:36:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.473+0000 7fc333a00640 1 -- 172.21.15.47:0/3316964991 wait complete. 2024-03-21T11:36:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.474+0000 7fc333a00640 1 Processor -- start 2024-03-21T11:36:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.474+0000 7fc333a00640 1 -- start start 2024-03-21T11:36:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.475+0000 7fc333a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc32c103900 0x7fc32c199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.475+0000 7fc333a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc32c104af0 0x7fc32c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.475+0000 7fc333a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc32c1054b0 0x7fc32c1a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.475+0000 7fc333a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c10f5d0 con 0x7fc32c1054b0 2024-03-21T11:36:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.475+0000 7fc333a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c10f450 con 0x7fc32c103900 2024-03-21T11:36:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.475+0000 7fc333a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c10f750 con 0x7fc32c104af0 2024-03-21T11:36:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.475+0000 7fc331000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc32c103900 0x7fc32c199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.475+0000 7fc32be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc32c104af0 0x7fc32c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.475+0000 7fc331000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc32c103900 0x7fc32c199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47484/0 (socket says 172.21.15.47:47484) 2024-03-21T11:36:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.475+0000 7fc331000640 1 -- 172.21.15.47:0/955551274 learned_addr learned my addr 172.21.15.47:0/955551274 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.475+0000 7fc331a00640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc32c1054b0 0x7fc32c1a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.476+0000 7fc331000640 1 -- 172.21.15.47:0/955551274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc32c104af0 msgr2=0x7fc32c19a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.476+0000 7fc331000640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc32c104af0 0x7fc32c19a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.476+0000 7fc331000640 1 -- 172.21.15.47:0/955551274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc32c1054b0 msgr2=0x7fc32c1a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.476+0000 7fc331000640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc32c1054b0 0x7fc32c1a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.476+0000 7fc331000640 1 -- 172.21.15.47:0/955551274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c1a1740 con 0x7fc32c103900 2024-03-21T11:36:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.476+0000 7fc331a00640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc32c1054b0 0x7fc32c1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.476+0000 7fc32be00640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc32c104af0 0x7fc32c19a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.476+0000 7fc331000640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc32c103900 0x7fc32c199b20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00ebc0 tx=0x7fc31c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.476+0000 7fc329600640 1 -- 172.21.15.47:0/955551274 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c003b70 con 0x7fc32c103900 2024-03-21T11:36:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.476+0000 7fc329600640 1 -- 172.21.15.47:0/955551274 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc31c003d10 con 0x7fc32c103900 2024-03-21T11:36:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.476+0000 7fc333a00640 1 -- 172.21.15.47:0/955551274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32c1a1a30 con 0x7fc32c103900 2024-03-21T11:36:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.476+0000 7fc329600640 1 -- 172.21.15.47:0/955551274 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c019980 con 0x7fc32c103900 2024-03-21T11:36:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.476+0000 7fc333a00640 1 -- 172.21.15.47:0/955551274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32c1a1fa0 con 0x7fc32c103900 2024-03-21T11:36:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.480+0000 7fc329600640 1 -- 172.21.15.47:0/955551274 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc31c019b20 con 0x7fc32c103900 2024-03-21T11:36:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.480+0000 7fc33680f740 1 Processor -- start 2024-03-21T11:36:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.480+0000 7fc33680f740 1 -- start start 2024-03-21T11:36:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.480+0000 7fc33680f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be8814600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.481+0000 7fc33680f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be88196d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.481+0000 7fc33680f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8819d40 0x560be881c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.481+0000 7fc33680f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be87d73b0 con 0x560be8819d40 2024-03-21T11:36:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.481+0000 7fc33680f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be8779830 con 0x560be88141e0 2024-03-21T11:36:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.481+0000 7fc33680f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be87225d0 con 0x560be8814bd0 2024-03-21T11:36:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.481+0000 7fc331a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be8814600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.481+0000 7fc331a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be8814600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47486/0 (socket says 172.21.15.47:47486) 2024-03-21T11:36:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.481+0000 7fc331a00640 1 -- 172.21.15.47:0/922186659 learned_addr learned my addr 172.21.15.47:0/922186659 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.481+0000 7fc331000640 1 --2- 172.21.15.47:0/922186659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be88196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.481+0000 7fc32be00640 1 --2- 172.21.15.47:0/922186659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8819d40 0x560be881c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.482+0000 7fc331a00640 1 -- 172.21.15.47:0/922186659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 msgr2=0x560be88196d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.482+0000 7fc331a00640 1 --2- 172.21.15.47:0/922186659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be88196d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.482+0000 7fc331a00640 1 -- 172.21.15.47:0/922186659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8819d40 msgr2=0x560be881c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.482+0000 7fc331a00640 1 --2- 172.21.15.47:0/922186659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8819d40 0x560be881c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.482+0000 7fc331a00640 1 -- 172.21.15.47:0/922186659 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560be873a2c0 con 0x560be88141e0 2024-03-21T11:36:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.482+0000 7fc329600640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3100a3040 0x7fc3100a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.482+0000 7fc331a00640 1 --2- 172.21.15.47:0/922186659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be8814600 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc324009cc0 tx=0x7fc3240117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8397b2f8aa7137db server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.482+0000 7fc331000640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3100a3040 0x7fc3100a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.482+0000 7fc322000640 1 -- 172.21.15.47:0/922186659 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc324012450 con 0x560be88141e0 2024-03-21T11:36:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.482+0000 7fc329600640 1 -- 172.21.15.47:0/955551274 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc31c0cfff0 con 0x7fc32c103900 2024-03-21T11:36:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.482+0000 7fc322000640 1 -- 172.21.15.47:0/922186659 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc324012ad0 con 0x560be88141e0 2024-03-21T11:36:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.482+0000 7fc322000640 1 -- 172.21.15.47:0/922186659 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32400fc90 con 0x560be88141e0 2024-03-21T11:36:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.483+0000 7fc33680f740 1 -- 172.21.15.47:0/922186659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 msgr2=0x560be8814600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.483+0000 7fc331000640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3100a3040 0x7fc3100a5500 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0a4830 tx=0x7fc31c0d3a50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.483+0000 7fc33680f740 1 --2- 172.21.15.47:0/922186659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be8814600 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc324009cc0 tx=0x7fc3240117b0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.483+0000 7fc33680f740 1 -- 172.21.15.47:0/922186659 shutdown_connections 2024-03-21T11:36:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.483+0000 7fc33680f740 1 --2- 172.21.15.47:0/922186659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8819d40 0x560be881c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.483+0000 7fc33680f740 1 --2- 172.21.15.47:0/922186659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be88196d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.483+0000 7fc33680f740 1 --2- 172.21.15.47:0/922186659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be8814600 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.483+0000 7fc33680f740 1 -- 172.21.15.47:0/922186659 >> 172.21.15.47:0/922186659 conn(0x560be880f750 msgr2=0x560be880fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.484+0000 7fc33680f740 1 -- 172.21.15.47:0/922186659 shutdown_connections 2024-03-21T11:36:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.484+0000 7fc33680f740 1 -- 172.21.15.47:0/922186659 wait complete. 2024-03-21T11:36:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.484+0000 7fc33680f740 1 -- start start 2024-03-21T11:36:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.484+0000 7fc33680f740 1 Processor -- start 2024-03-21T11:36:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.485+0000 7fc33680f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be881e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.485+0000 7fc33680f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be881e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.485+0000 7fc33680f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8819d40 0x560be8825910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.485+0000 7fc33680f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be8735a20 con 0x560be8819d40 2024-03-21T11:36:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.485+0000 7fc33680f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be87758d0 con 0x560be88141e0 2024-03-21T11:36:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.485+0000 7fc33680f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be881ccf0 con 0x560be8814bd0 2024-03-21T11:36:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.485+0000 7fc33680f740 10 client.0 _ll_register_callbacks cb 0x560be87721c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.485+0000 7fc32be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be881e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.485+0000 7fc331a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be881e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.485+0000 7fc32be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be881e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:39186/0 (socket says 172.21.15.47:39186) 2024-03-21T11:36:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.485+0000 7fc32be00640 1 -- 172.21.15.47:0/185640505 learned_addr learned my addr 172.21.15.47:0/185640505 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.485+0000 7fc331a00640 1 --2- 172.21.15.47:0/185640505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8819d40 0x560be8825910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.486+0000 7fc32be00640 1 -- 172.21.15.47:0/185640505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 msgr2=0x560be881e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.486+0000 7fc32be00640 1 --2- 172.21.15.47:0/185640505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be881e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.486+0000 7fc32be00640 1 -- 172.21.15.47:0/185640505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8819d40 msgr2=0x560be8825910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.486+0000 7fc32be00640 1 --2- 172.21.15.47:0/185640505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8819d40 0x560be8825910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.486+0000 7fc32be00640 1 -- 172.21.15.47:0/185640505 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560be848e210 con 0x560be8814bd0 2024-03-21T11:36:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.486+0000 7fc331a00640 1 --2- 172.21.15.47:0/185640505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8819d40 0x560be8825910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.486+0000 7fc32be00640 1 --2- 172.21.15.47:0/185640505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be881e8d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc31402efd0 tx=0x7fc3140048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdde47ca210136a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.486+0000 7fc31aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.486+0000 7fc31aa00640 1 -- 172.21.15.47:0/185640505 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314005d50 con 0x560be8814bd0 2024-03-21T11:36:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.486+0000 7fc31aa00640 1 -- 172.21.15.47:0/185640505 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc314004ef0 con 0x560be8814bd0 2024-03-21T11:36:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.486+0000 7fc33680f740 1 -- 172.21.15.47:0/185640505 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc324012ad0 con 0x560be8814bd0 2024-03-21T11:36:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.487+0000 7fc31aa00640 1 -- 172.21.15.47:0/185640505 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc31402cd40 con 0x560be8814bd0 2024-03-21T11:36:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.487+0000 7fc33680f740 10 client.29794 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.487+0000 7fc33680f740 1 -- 172.21.15.47:0/185640505 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560be855a620 con 0x560be8814bd0 2024-03-21T11:36:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.487+0000 7fc31aa00640 1 -- 172.21.15.47:0/185640505 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc324012ad0 con 0x560be8814bd0 2024-03-21T11:36:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.487+0000 7fc31aa00640 1 -- 172.21.15.47:0/185640505 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc3140038c0 con 0x560be8814bd0 2024-03-21T11:36:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.487+0000 7fc33680f740 10 client.29794 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.487+0000 7fc33680f740 10 client.29794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.487+0000 7fc33680f740 20 client.29794 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.487+0000 7fc33680f740 1 --2- 172.21.15.47:0/185640505 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560be882a2b0 0x560be882c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.487+0000 7fc33680f740 4 client.29794 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.487+0000 7fc33680f740 1 -- 172.21.15.47:0/185640505 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560be8786d70 con 0x560be882a2b0 2024-03-21T11:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.488+0000 7fc331a00640 1 --2- 172.21.15.47:0/185640505 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560be882a2b0 0x560be882c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.488+0000 7fc331a00640 1 --2- 172.21.15.47:0/185640505 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560be882a2b0 0x560be882c6a0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=828fd5f1d810bdc6 server_cookie=e08e133d1ced7f1b in_seq=0 out_seq=0 2024-03-21T11:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.488+0000 7fc31aa00640 10 client.29794 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.489+0000 7fc31aa00640 1 -- 172.21.15.47:0/185640505 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560be8786d70 con 0x560be882a2b0 2024-03-21T11:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.489+0000 7fc31aa00640 10 client.29794 handle_command_reply: tid=0 2024-03-21T11:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.489+0000 7fc33680f740 1 client.29794 shutdown 2024-03-21T11:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.490+0000 7fc33680f740 1 -- 172.21.15.47:0/185640505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 msgr2=0x560be881e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.490+0000 7fc33680f740 1 --2- 172.21.15.47:0/185640505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be881e8d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc31402efd0 tx=0x7fc3140048f0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.490+0000 7fc33680f740 1 -- 172.21.15.47:0/185640505 shutdown_connections 2024-03-21T11:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.490+0000 7fc33680f740 1 --2- 172.21.15.47:0/185640505 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560be882a2b0 0x560be882c6a0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.490+0000 7fc33680f740 1 --2- 172.21.15.47:0/185640505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8819d40 0x560be8825910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.490+0000 7fc33680f740 1 --2- 172.21.15.47:0/185640505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be881e8d0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.490+0000 7fc33680f740 1 --2- 172.21.15.47:0/185640505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be881e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.490+0000 7fc33680f740 1 -- 172.21.15.47:0/185640505 >> 172.21.15.47:0/185640505 conn(0x560be880f750 msgr2=0x560be880e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.490+0000 7fc31aa00640 0 client.29794 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.490+0000 7fc33680f740 1 -- 172.21.15.47:0/185640505 shutdown_connections 2024-03-21T11:36:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.490+0000 7fc33680f740 1 -- 172.21.15.47:0/185640505 wait complete. 2024-03-21T11:36:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.490+0000 7fc33680f740 20 client.29794 trim_cache size 0 max 16384 2024-03-21T11:36:07.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:07 smithi102 ceph-mon[36748]: pgmap v783: 129 pgs: 129 active+clean; 30 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 2.6 MiB/s wr, 266 op/s 2024-03-21T11:36:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.507+0000 7fc33680f740 1 Processor -- start 2024-03-21T11:36:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.507+0000 7fc33680f740 1 -- start start 2024-03-21T11:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.507+0000 7fc33680f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be881e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.507+0000 7fc33680f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8814bd0 0x560be88258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.507+0000 7fc33680f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8819d40 0x560be881e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.507+0000 7fc33680f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c1ac210 con 0x560be8814bd0 2024-03-21T11:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.507+0000 7fc33680f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c1adb90 con 0x560be88141e0 2024-03-21T11:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.507+0000 7fc33680f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c1ac870 con 0x560be8819d40 2024-03-21T11:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.507+0000 7fc331a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be881e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.507+0000 7fc32be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8814bd0 0x560be88258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.508+0000 7fc331a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be881e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47494/0 (socket says 172.21.15.47:47494) 2024-03-21T11:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.508+0000 7fc331a00640 1 -- 172.21.15.47:0/127435437 learned_addr learned my addr 172.21.15.47:0/127435437 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.508+0000 7fc331a00640 1 --2- 172.21.15.47:0/127435437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8819d40 0x560be881e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.508+0000 7fc32be00640 1 -- 172.21.15.47:0/127435437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8819d40 msgr2=0x560be881e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.508+0000 7fc32be00640 1 --2- 172.21.15.47:0/127435437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8819d40 0x560be881e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.508+0000 7fc32be00640 1 -- 172.21.15.47:0/127435437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 msgr2=0x560be881e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.508+0000 7fc32be00640 1 --2- 172.21.15.47:0/127435437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be881e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.508+0000 7fc32be00640 1 -- 172.21.15.47:0/127435437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560be8815460 con 0x560be8814bd0 2024-03-21T11:36:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.508+0000 7fc32be00640 1 --2- 172.21.15.47:0/127435437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8814bd0 0x560be88258d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc31402c7c0 tx=0x7fc31402bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4545723a8fb1083 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.509+0000 7fc321600640 1 -- 172.21.15.47:0/127435437 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31405a070 con 0x560be8814bd0 2024-03-21T11:36:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.509+0000 7fc321600640 1 -- 172.21.15.47:0/127435437 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc31402cd40 con 0x560be8814bd0 2024-03-21T11:36:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.509+0000 7fc321600640 1 -- 172.21.15.47:0/127435437 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314004ef0 con 0x560be8814bd0 2024-03-21T11:36:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.510+0000 7fc33680f740 1 -- 172.21.15.47:0/127435437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8814bd0 msgr2=0x560be88258d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.510+0000 7fc33680f740 1 --2- 172.21.15.47:0/127435437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8814bd0 0x560be88258d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc31402c7c0 tx=0x7fc31402bc40 comp rx=0 tx=0).stop 2024-03-21T11:36:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.510+0000 7fc33680f740 1 -- 172.21.15.47:0/127435437 shutdown_connections 2024-03-21T11:36:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.510+0000 7fc33680f740 1 --2- 172.21.15.47:0/127435437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8819d40 0x560be881e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.510+0000 7fc33680f740 1 --2- 172.21.15.47:0/127435437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be8814bd0 0x560be88258d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.510+0000 7fc33680f740 1 --2- 172.21.15.47:0/127435437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be88141e0 0x560be881e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.510+0000 7fc33680f740 1 -- 172.21.15.47:0/127435437 >> 172.21.15.47:0/127435437 conn(0x560be880f750 msgr2=0x560be881d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.510+0000 7fc33680f740 1 -- 172.21.15.47:0/127435437 shutdown_connections 2024-03-21T11:36:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.510+0000 7fc33680f740 1 -- 172.21.15.47:0/127435437 wait complete. 2024-03-21T11:36:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.510+0000 7fc33680f740 1 -- start start 2024-03-21T11:36:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.511+0000 7fc33680f740 1 Processor -- start 2024-03-21T11:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.511+0000 7fc33680f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be88141e0 0x560be88230d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.511+0000 7fc33680f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be8823610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.511+0000 7fc33680f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be8819d40 0x560be8824620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.511+0000 7fc33680f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c1ad530 con 0x560be88141e0 2024-03-21T11:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.511+0000 7fc33680f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c1aced0 con 0x560be8819d40 2024-03-21T11:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.511+0000 7fc33680f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c1aa3d0 con 0x560be8814bd0 2024-03-21T11:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.511+0000 7fc33680f740 10 client.0 _ll_register_callbacks cb 0x7fc324005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.512+0000 7fc331a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be88141e0 0x560be88230d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.512+0000 7fc32be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be8823610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.512+0000 7fc32be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be8823610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:39204/0 (socket says 172.21.15.47:39204) 2024-03-21T11:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.512+0000 7fc331a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be88141e0 0x560be88230d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42516/0 (socket says 172.21.15.47:42516) 2024-03-21T11:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.512+0000 7fc32be00640 1 -- 172.21.15.47:0/1311212017 learned_addr learned my addr 172.21.15.47:0/1311212017 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.512+0000 7fc331a00640 1 --2- 172.21.15.47:0/1311212017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be8819d40 0x560be8824620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.512+0000 7fc32be00640 1 -- 172.21.15.47:0/1311212017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be8819d40 msgr2=0x560be8824620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.512+0000 7fc32be00640 1 --2- 172.21.15.47:0/1311212017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be8819d40 0x560be8824620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.512+0000 7fc32be00640 1 -- 172.21.15.47:0/1311212017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be88141e0 msgr2=0x560be88230d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.512+0000 7fc32be00640 1 --2- 172.21.15.47:0/1311212017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be88141e0 0x560be88230d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.512+0000 7fc32be00640 1 -- 172.21.15.47:0/1311212017 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560be8824c10 con 0x560be8814bd0 2024-03-21T11:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.512+0000 7fc32be00640 1 --2- 172.21.15.47:0/1311212017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be8823610 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc31402c180 tx=0x7fc314004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6da312f523d89286 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.512+0000 7fc31aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.513+0000 7fc31aa00640 1 -- 172.21.15.47:0/1311212017 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31405a040 con 0x560be8814bd0 2024-03-21T11:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.513+0000 7fc31aa00640 1 -- 172.21.15.47:0/1311212017 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc3140048d0 con 0x560be8814bd0 2024-03-21T11:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.513+0000 7fc33680f740 1 -- 172.21.15.47:0/1311212017 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc31402cd40 con 0x560be8814bd0 2024-03-21T11:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.513+0000 7fc31aa00640 1 -- 172.21.15.47:0/1311212017 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc31406a050 con 0x560be8814bd0 2024-03-21T11:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.513+0000 7fc33680f740 10 client.29800 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.513+0000 7fc33680f740 1 -- 172.21.15.47:0/1311212017 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560be8885c20 con 0x560be8814bd0 2024-03-21T11:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.514+0000 7fc31aa00640 1 -- 172.21.15.47:0/1311212017 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc31402cd40 con 0x560be8814bd0 2024-03-21T11:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.514+0000 7fc31aa00640 1 -- 172.21.15.47:0/1311212017 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc314034440 con 0x560be8814bd0 2024-03-21T11:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.514+0000 7fc33680f740 10 client.29800 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.514+0000 7fc33680f740 10 client.29800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.514+0000 7fc33680f740 20 client.29800 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.514+0000 7fc33680f740 1 --2- 172.21.15.47:0/1311212017 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560be8886420 0x560be8888840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.514+0000 7fc33680f740 4 client.29800 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.514+0000 7fc33680f740 1 -- 172.21.15.47:0/1311212017 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560be87810c0 con 0x560be8886420 2024-03-21T11:36:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.514+0000 7fc331a00640 1 --2- 172.21.15.47:0/1311212017 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560be8886420 0x560be8888840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.515+0000 7fc331a00640 1 --2- 172.21.15.47:0/1311212017 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560be8886420 0x560be8888840 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58382904bd3acb89 server_cookie=694783d087218001 in_seq=0 out_seq=0 2024-03-21T11:36:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.515+0000 7fc31aa00640 10 client.29800 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:07.577 INFO:tasks.workunit.client.0.smithi047.stdout: 1 972478 7.95 MB/sec execute 531 sec latency 26.784 ms 2024-03-21T11:36:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.859+0000 7fc31aa00640 1 -- 172.21.15.47:0/1311212017 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560be87810c0 con 0x560be8886420 2024-03-21T11:36:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.859+0000 7fc31aa00640 10 client.29800 handle_command_reply: tid=0 2024-03-21T11:36:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.859+0000 7fc33680f740 1 client.29800 shutdown 2024-03-21T11:36:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.860+0000 7fc33680f740 1 -- 172.21.15.47:0/1311212017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 msgr2=0x560be8823610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.860+0000 7fc33680f740 1 --2- 172.21.15.47:0/1311212017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be8823610 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc31402c180 tx=0x7fc314004610 comp rx=0 tx=0).stop 2024-03-21T11:36:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.860+0000 7fc33680f740 1 -- 172.21.15.47:0/1311212017 shutdown_connections 2024-03-21T11:36:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.860+0000 7fc33680f740 1 --2- 172.21.15.47:0/1311212017 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560be8886420 0x560be8888840 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.860+0000 7fc33680f740 1 --2- 172.21.15.47:0/1311212017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560be8819d40 0x560be8824620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.860+0000 7fc33680f740 1 --2- 172.21.15.47:0/1311212017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560be8814bd0 0x560be8823610 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.860+0000 7fc33680f740 1 --2- 172.21.15.47:0/1311212017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560be88141e0 0x560be88230d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.860+0000 7fc33680f740 1 -- 172.21.15.47:0/1311212017 >> 172.21.15.47:0/1311212017 conn(0x560be880f750 msgr2=0x560be88129b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.860+0000 7fc31aa00640 0 client.29800 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.860+0000 7fc33680f740 1 -- 172.21.15.47:0/1311212017 shutdown_connections 2024-03-21T11:36:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.860+0000 7fc33680f740 1 -- 172.21.15.47:0/1311212017 wait complete. 2024-03-21T11:36:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.860+0000 7fc33680f740 20 client.29800 trim_cache size 0 max 16384 2024-03-21T11:36:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:36:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:36:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:36:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:36:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.861+0000 7fc333a00640 1 -- 172.21.15.47:0/955551274 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3100a3040 msgr2=0x7fc3100a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.861+0000 7fc333a00640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3100a3040 0x7fc3100a5500 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0a4830 tx=0x7fc31c0d3a50 comp rx=0 tx=0).stop 2024-03-21T11:36:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.861+0000 7fc333a00640 1 -- 172.21.15.47:0/955551274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc32c103900 msgr2=0x7fc32c199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.861+0000 7fc333a00640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc32c103900 0x7fc32c199b20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00ebc0 tx=0x7fc31c00c6a0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.862+0000 7fc333a00640 1 -- 172.21.15.47:0/955551274 shutdown_connections 2024-03-21T11:36:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.862+0000 7fc333a00640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc3100a3040 0x7fc3100a5500 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.862+0000 7fc333a00640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc32c1054b0 0x7fc32c1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.862+0000 7fc333a00640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc32c104af0 0x7fc32c19a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.862+0000 7fc333a00640 1 --2- 172.21.15.47:0/955551274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc32c103900 0x7fc32c199b20 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.862+0000 7fc333a00640 1 -- 172.21.15.47:0/955551274 >> 172.21.15.47:0/955551274 conn(0x7fc32c0fee90 msgr2=0x7fc32c107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.862+0000 7fc333a00640 1 -- 172.21.15.47:0/955551274 shutdown_connections 2024-03-21T11:36:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:07.862+0000 7fc333a00640 1 -- 172.21.15.47:0/955551274 wait complete. 2024-03-21T11:36:08.578 INFO:tasks.workunit.client.0.smithi047.stdout: 1 973713 7.95 MB/sec execute 532 sec latency 188.633 ms 2024-03-21T11:36:09.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:09 smithi047 ceph-mon[31351]: pgmap v784: 129 pgs: 129 active+clean; 39 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 3.2 MiB/s wr, 280 op/s 2024-03-21T11:36:09.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:09 smithi102 ceph-mon[36748]: pgmap v784: 129 pgs: 129 active+clean; 39 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 3.2 MiB/s wr, 280 op/s 2024-03-21T11:36:09.577 INFO:tasks.workunit.client.0.smithi047.stdout: 1 974395 7.94 MB/sec execute 533 sec latency 36.864 ms 2024-03-21T11:36:09.970 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:09 smithi100 ceph-mon[36245]: pgmap v784: 129 pgs: 129 active+clean; 39 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 3.2 MiB/s wr, 280 op/s 2024-03-21T11:36:10.578 INFO:tasks.workunit.client.0.smithi047.stdout: 1 976063 7.94 MB/sec execute 534 sec latency 18.129 ms 2024-03-21T11:36:11.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:11 smithi047 ceph-mon[31351]: pgmap v785: 129 pgs: 129 active+clean; 39 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 3.6 MiB/s wr, 269 op/s 2024-03-21T11:36:11.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:11 smithi100 ceph-mon[36245]: pgmap v785: 129 pgs: 129 active+clean; 39 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 3.6 MiB/s wr, 269 op/s 2024-03-21T11:36:11.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:11 smithi102 ceph-mon[36748]: pgmap v785: 129 pgs: 129 active+clean; 39 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 3.6 MiB/s wr, 269 op/s 2024-03-21T11:36:11.578 INFO:tasks.workunit.client.0.smithi047.stdout: 1 977387 7.94 MB/sec execute 535 sec latency 13.684 ms 2024-03-21T11:36:12.578 INFO:tasks.workunit.client.0.smithi047.stdout: 1 978204 7.93 MB/sec execute 536 sec latency 17.739 ms 2024-03-21T11:36:13.292 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:13 smithi102 ceph-mon[36748]: pgmap v786: 129 pgs: 129 active+clean; 34 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 3.1 MiB/s wr, 222 op/s 2024-03-21T11:36:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:13 smithi047 ceph-mon[31351]: pgmap v786: 129 pgs: 129 active+clean; 34 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 3.1 MiB/s wr, 222 op/s 2024-03-21T11:36:13.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:13 smithi100 ceph-mon[36245]: pgmap v786: 129 pgs: 129 active+clean; 34 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 3.1 MiB/s wr, 222 op/s 2024-03-21T11:36:13.578 INFO:tasks.workunit.client.0.smithi047.stdout: 1 980014 7.94 MB/sec execute 537 sec latency 43.224 ms 2024-03-21T11:36:14.578 INFO:tasks.workunit.client.0.smithi047.stdout: 1 981144 7.94 MB/sec execute 538 sec latency 23.876 ms 2024-03-21T11:36:15.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:15 smithi047 ceph-mon[31351]: pgmap v787: 129 pgs: 129 active+clean; 35 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 3.6 MiB/s wr, 323 op/s 2024-03-21T11:36:15.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:15 smithi100 ceph-mon[36245]: pgmap v787: 129 pgs: 129 active+clean; 35 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 3.6 MiB/s wr, 323 op/s 2024-03-21T11:36:15.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:15 smithi102 ceph-mon[36748]: pgmap v787: 129 pgs: 129 active+clean; 35 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 3.6 MiB/s wr, 323 op/s 2024-03-21T11:36:15.578 INFO:tasks.workunit.client.0.smithi047.stdout: 1 982894 7.93 MB/sec execute 539 sec latency 14.342 ms 2024-03-21T11:36:16.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:36:16.252 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:36:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.491+0000 7f12acc00640 1 -- 172.21.15.47:0/3950671091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a81037a0 msgr2=0x7f12a8103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.491+0000 7f12acc00640 1 --2- 172.21.15.47:0/3950671091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a81037a0 0x7f12a8103bc0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1298009900 tx=0x7f129802bc30 comp rx=0 tx=0).stop 2024-03-21T11:36:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.491+0000 7f12acc00640 1 -- 172.21.15.47:0/3950671091 shutdown_connections 2024-03-21T11:36:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.491+0000 7f12acc00640 1 --2- 172.21.15.47:0/3950671091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a8105350 0x7f12a810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.491+0000 7f12acc00640 1 --2- 172.21.15.47:0/3950671091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a8104990 0x7f12a8104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.491+0000 7f12acc00640 1 --2- 172.21.15.47:0/3950671091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a81037a0 0x7f12a8103bc0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.491+0000 7f12acc00640 1 -- 172.21.15.47:0/3950671091 >> 172.21.15.47:0/3950671091 conn(0x7f12a80fed10 msgr2=0x7f12a8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.491+0000 7f12acc00640 1 -- 172.21.15.47:0/3950671091 shutdown_connections 2024-03-21T11:36:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.491+0000 7f12acc00640 1 -- 172.21.15.47:0/3950671091 wait complete. 2024-03-21T11:36:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.492+0000 7f12acc00640 1 Processor -- start 2024-03-21T11:36:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.492+0000 7f12acc00640 1 -- start start 2024-03-21T11:36:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.493+0000 7f12acc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a81037a0 0x7f12a8199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.493+0000 7f12acc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a8104990 0x7f12a8199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.493+0000 7f12acc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a8105350 0x7f12a81a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.493+0000 7f12acc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a810f460 con 0x7f12a8105350 2024-03-21T11:36:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.493+0000 7f12acc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a810f2e0 con 0x7f12a81037a0 2024-03-21T11:36:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.493+0000 7f12acc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a810f5e0 con 0x7f12a8104990 2024-03-21T11:36:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.493+0000 7f12a7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a81037a0 0x7f12a8199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.493+0000 7f12a7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a81037a0 0x7f12a8199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58814/0 (socket says 172.21.15.47:58814) 2024-03-21T11:36:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.493+0000 7f12a6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a8104990 0x7f12a8199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.493+0000 7f12a7400640 1 -- 172.21.15.47:0/2866552375 learned_addr learned my addr 172.21.15.47:0/2866552375 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.493+0000 7f12a7e00640 1 --2- 172.21.15.47:0/2866552375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a8105350 0x7f12a81a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.493+0000 7f12a7400640 1 -- 172.21.15.47:0/2866552375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a8104990 msgr2=0x7f12a8199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.494+0000 7f12a7400640 1 --2- 172.21.15.47:0/2866552375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a8104990 0x7f12a8199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.494+0000 7f12a7400640 1 -- 172.21.15.47:0/2866552375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a8105350 msgr2=0x7f12a81a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.494+0000 7f12a7400640 1 --2- 172.21.15.47:0/2866552375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a8105350 0x7f12a81a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.494+0000 7f12a7400640 1 -- 172.21.15.47:0/2866552375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a81a16d0 con 0x7f12a81037a0 2024-03-21T11:36:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.494+0000 7f12a6a00640 1 --2- 172.21.15.47:0/2866552375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a8104990 0x7f12a8199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.494+0000 7f12a7400640 1 --2- 172.21.15.47:0/2866552375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a81037a0 0x7f12a8199ab0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1298000c00 tx=0x7f129802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.494+0000 7f129fe00640 1 -- 172.21.15.47:0/2866552375 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298043020 con 0x7f12a81037a0 2024-03-21T11:36:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.494+0000 7f12acc00640 1 -- 172.21.15.47:0/2866552375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a81a1960 con 0x7f12a81037a0 2024-03-21T11:36:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.494+0000 7f129fe00640 1 -- 172.21.15.47:0/2866552375 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12980054e0 con 0x7f12a81037a0 2024-03-21T11:36:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.494+0000 7f129fe00640 1 -- 172.21.15.47:0/2866552375 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298003c40 con 0x7f12a81037a0 2024-03-21T11:36:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.495+0000 7f12acc00640 1 -- 172.21.15.47:0/2866552375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a81a1eb0 con 0x7f12a81037a0 2024-03-21T11:36:16.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.498+0000 7f129fe00640 1 -- 172.21.15.47:0/2866552375 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1298046020 con 0x7f12a81037a0 2024-03-21T11:36:16.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.498+0000 7f12b0f88740 1 Processor -- start 2024-03-21T11:36:16.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.498+0000 7f12b0f88740 1 -- start start 2024-03-21T11:36:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.498+0000 7f12b0f88740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 0x5620dc289470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.498+0000 7f12b0f88740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 0x5620dc28e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.498+0000 7f12b0f88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc28ebb0 0x5620dc291040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.498+0000 7f12b0f88740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dc1ee250 con 0x5620dc289a40 2024-03-21T11:36:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.499+0000 7f12b0f88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dc196a30 con 0x5620dc28ebb0 2024-03-21T11:36:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.499+0000 7f12b0f88740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dc1ea2f0 con 0x5620dc289050 2024-03-21T11:36:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.499+0000 7f12a6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc28ebb0 0x5620dc291040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.499+0000 7f12a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 0x5620dc289470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.499+0000 7f12a6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc28ebb0 0x5620dc291040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58820/0 (socket says 172.21.15.47:58820) 2024-03-21T11:36:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.499+0000 7f12a6a00640 1 -- 172.21.15.47:0/2895857736 learned_addr learned my addr 172.21.15.47:0/2895857736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.499+0000 7f12a7400640 1 --2- 172.21.15.47:0/2895857736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 0x5620dc28e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.499+0000 7f12a7e00640 1 -- 172.21.15.47:0/2895857736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc28ebb0 msgr2=0x5620dc291040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.499+0000 7f12a7e00640 1 --2- 172.21.15.47:0/2895857736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc28ebb0 0x5620dc291040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.499+0000 7f12a7e00640 1 -- 172.21.15.47:0/2895857736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 msgr2=0x5620dc28e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.499+0000 7f12a7e00640 1 --2- 172.21.15.47:0/2895857736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 0x5620dc28e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.499+0000 7f12a7e00640 1 -- 172.21.15.47:0/2895857736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620dc181690 con 0x5620dc289050 2024-03-21T11:36:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.499+0000 7f12a6a00640 1 --2- 172.21.15.47:0/2895857736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc28ebb0 0x5620dc291040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:16.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.500+0000 7f12a7e00640 1 --2- 172.21.15.47:0/2895857736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 0x5620dc289470 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009f00 tx=0x7f12a0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7dddc944e04dc48 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:16.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.500+0000 7f129fe00640 1 --2- 172.21.15.47:0/2866552375 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f128c0a2f20 0x7f128c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.500+0000 7f1293e00640 1 -- 172.21.15.47:0/2895857736 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0004d20 con 0x5620dc289050 2024-03-21T11:36:16.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.500+0000 7f1293e00640 1 -- 172.21.15.47:0/2895857736 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12a0004030 con 0x5620dc289050 2024-03-21T11:36:16.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.500+0000 7f1293e00640 1 -- 172.21.15.47:0/2895857736 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0005220 con 0x5620dc289050 2024-03-21T11:36:16.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.500+0000 7f12a7400640 1 --2- 172.21.15.47:0/2866552375 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f128c0a2f20 0x7f128c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.500+0000 7f129fe00640 1 -- 172.21.15.47:0/2866552375 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f12980f1800 con 0x7f12a81037a0 2024-03-21T11:36:16.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.501+0000 7f12b0f88740 1 -- 172.21.15.47:0/2895857736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 msgr2=0x5620dc289470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:16.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.501+0000 7f12a7400640 1 --2- 172.21.15.47:0/2866552375 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f128c0a2f20 0x7f128c0a53e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f12980ba510 tx=0x7f12980f7530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:16.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.501+0000 7f12b0f88740 1 --2- 172.21.15.47:0/2895857736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 0x5620dc289470 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009f00 tx=0x7f12a0011810 comp rx=0 tx=0).stop 2024-03-21T11:36:16.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.501+0000 7f12b0f88740 1 -- 172.21.15.47:0/2895857736 shutdown_connections 2024-03-21T11:36:16.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.501+0000 7f12b0f88740 1 --2- 172.21.15.47:0/2895857736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc28ebb0 0x5620dc291040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.501+0000 7f12b0f88740 1 --2- 172.21.15.47:0/2895857736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 0x5620dc28e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.501+0000 7f12b0f88740 1 --2- 172.21.15.47:0/2895857736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 0x5620dc289470 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.501+0000 7f12b0f88740 1 -- 172.21.15.47:0/2895857736 >> 172.21.15.47:0/2895857736 conn(0x5620dc251200 msgr2=0x5620dc2849c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.501+0000 7f12b0f88740 1 -- 172.21.15.47:0/2895857736 shutdown_connections 2024-03-21T11:36:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.501+0000 7f12b0f88740 1 -- 172.21.15.47:0/2895857736 wait complete. 2024-03-21T11:36:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.502+0000 7f12b0f88740 1 -- start start 2024-03-21T11:36:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.502+0000 7f12b0f88740 1 Processor -- start 2024-03-21T11:36:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.502+0000 7f12b0f88740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 0x5620dc293020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.502+0000 7f12b0f88740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 0x5620dc293560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.502+0000 7f12b0f88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc28ebb0 0x5620dc29a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.502+0000 7f12b0f88740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dc291b60 con 0x5620dc289a40 2024-03-21T11:36:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.502+0000 7f12b0f88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dc1a9e80 con 0x5620dc28ebb0 2024-03-21T11:36:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.502+0000 7f12b0f88740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620dc291ce0 con 0x5620dc289050 2024-03-21T11:36:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.502+0000 7f12b0f88740 10 client.0 _ll_register_callbacks cb 0x5620dc1bc130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.503+0000 7f12a6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 0x5620dc293560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.503+0000 7f12a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 0x5620dc293020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.503+0000 7f12a6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 0x5620dc293560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34832/0 (socket says 172.21.15.47:34832) 2024-03-21T11:36:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.503+0000 7f12a6a00640 1 -- 172.21.15.47:0/2378612037 learned_addr learned my addr 172.21.15.47:0/2378612037 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.503+0000 7f12a7e00640 1 --2- 172.21.15.47:0/2378612037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc28ebb0 0x5620dc29a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.503+0000 7f12a6a00640 1 -- 172.21.15.47:0/2378612037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 msgr2=0x5620dc293020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.503+0000 7f12a6a00640 1 --2- 172.21.15.47:0/2378612037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 0x5620dc293020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.503+0000 7f12a6a00640 1 -- 172.21.15.47:0/2378612037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc28ebb0 msgr2=0x5620dc29a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.503+0000 7f12a6a00640 1 --2- 172.21.15.47:0/2378612037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc28ebb0 0x5620dc29a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.503+0000 7f12a6a00640 1 -- 172.21.15.47:0/2378612037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620dbfcc7d0 con 0x5620dc289a40 2024-03-21T11:36:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.503+0000 7f12a6a00640 1 --2- 172.21.15.47:0/2378612037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 0x5620dc293560 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1294012d20 tx=0x7f129400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76079231fe0199c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.503+0000 7f1290c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:36:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.504+0000 7f1290c00640 1 -- 172.21.15.47:0/2378612037 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294012110 con 0x5620dc289a40 2024-03-21T11:36:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.504+0000 7f1290c00640 1 -- 172.21.15.47:0/2378612037 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1294004030 con 0x5620dc289a40 2024-03-21T11:36:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.504+0000 7f12b0f88740 1 -- 172.21.15.47:0/2378612037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12a0004030 con 0x5620dc289a40 2024-03-21T11:36:16.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.504+0000 7f1290c00640 1 -- 172.21.15.47:0/2378612037 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1294025050 con 0x5620dc289a40 2024-03-21T11:36:16.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.504+0000 7f12b0f88740 10 client.19395 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:16.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.504+0000 7f12b0f88740 1 -- 172.21.15.47:0/2378612037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5620dbf023f0 con 0x5620dc289a40 2024-03-21T11:36:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.504+0000 7f1290c00640 1 -- 172.21.15.47:0/2378612037 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12a0004030 con 0x5620dc289a40 2024-03-21T11:36:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.505+0000 7f1290c00640 1 -- 172.21.15.47:0/2378612037 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1294010430 con 0x5620dc289a40 2024-03-21T11:36:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.505+0000 7f12b0f88740 10 client.19395 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.505+0000 7f12b0f88740 10 client.19395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.505+0000 7f12b0f88740 20 client.19395 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.505+0000 7f12b0f88740 1 --2- 172.21.15.47:0/2378612037 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620dc29ef10 0x5620dc2a1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.505+0000 7f12b0f88740 4 client.19395 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.505+0000 7f12b0f88740 1 -- 172.21.15.47:0/2378612037 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620dc1fb790 con 0x5620dc29ef10 2024-03-21T11:36:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.505+0000 7f12a7e00640 1 --2- 172.21.15.47:0/2378612037 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620dc29ef10 0x5620dc2a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.506+0000 7f1290c00640 10 client.19395 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.506+0000 7f12a7e00640 1 --2- 172.21.15.47:0/2378612037 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620dc29ef10 0x5620dc2a1330 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df1fe355d0902e38 server_cookie=359237214e7ace6f in_seq=0 out_seq=0 2024-03-21T11:36:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.506+0000 7f1290c00640 1 -- 172.21.15.47:0/2378612037 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5620dc1fb790 con 0x5620dc29ef10 2024-03-21T11:36:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.507+0000 7f1290c00640 10 client.19395 handle_command_reply: tid=0 2024-03-21T11:36:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.507+0000 7f12b0f88740 1 client.19395 shutdown 2024-03-21T11:36:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.507+0000 7f12b0f88740 1 -- 172.21.15.47:0/2378612037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 msgr2=0x5620dc293560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.507+0000 7f12b0f88740 1 --2- 172.21.15.47:0/2378612037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 0x5620dc293560 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1294012d20 tx=0x7f129400f790 comp rx=0 tx=0).stop 2024-03-21T11:36:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.507+0000 7f12b0f88740 1 -- 172.21.15.47:0/2378612037 shutdown_connections 2024-03-21T11:36:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.507+0000 7f12b0f88740 1 --2- 172.21.15.47:0/2378612037 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620dc29ef10 0x5620dc2a1330 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.507+0000 7f12b0f88740 1 --2- 172.21.15.47:0/2378612037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc28ebb0 0x5620dc29a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.507+0000 7f12b0f88740 1 --2- 172.21.15.47:0/2378612037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 0x5620dc293560 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.507+0000 7f12b0f88740 1 --2- 172.21.15.47:0/2378612037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 0x5620dc293020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.507+0000 7f1290c00640 0 client.19395 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.507+0000 7f12b0f88740 1 -- 172.21.15.47:0/2378612037 >> 172.21.15.47:0/2378612037 conn(0x5620dc251200 msgr2=0x5620dc283bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.507+0000 7f12b0f88740 1 -- 172.21.15.47:0/2378612037 shutdown_connections 2024-03-21T11:36:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.507+0000 7f12b0f88740 1 -- 172.21.15.47:0/2378612037 wait complete. 2024-03-21T11:36:16.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.507+0000 7f12b0f88740 20 client.19395 trim_cache size 0 max 16384 2024-03-21T11:36:16.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.526+0000 7f12b0f88740 1 Processor -- start 2024-03-21T11:36:16.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.526+0000 7f12b0f88740 1 -- start start 2024-03-21T11:36:16.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.527+0000 7f12b0f88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc289050 0x5620dc27f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.527+0000 7f12b0f88740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 0x5620dc29a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.527+0000 7f12b0f88740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc28ebb0 0x5620dc287c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.527+0000 7f12b0f88740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a81ac0c0 con 0x5620dc289a40 2024-03-21T11:36:16.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.527+0000 7f12b0f88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a81ada40 con 0x5620dc289050 2024-03-21T11:36:16.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.527+0000 7f12b0f88740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a81ac720 con 0x5620dc28ebb0 2024-03-21T11:36:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.527+0000 7f12a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc289050 0x5620dc27f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.527+0000 7f12a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc289050 0x5620dc27f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58842/0 (socket says 172.21.15.47:58842) 2024-03-21T11:36:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.527+0000 7f12a7e00640 1 -- 172.21.15.47:0/607116718 learned_addr learned my addr 172.21.15.47:0/607116718 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.527+0000 7f12a7e00640 1 --2- 172.21.15.47:0/607116718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc28ebb0 0x5620dc287c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.527+0000 7f12a6a00640 1 --2- 172.21.15.47:0/607116718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 0x5620dc29a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.528+0000 7f12a7e00640 1 -- 172.21.15.47:0/607116718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc28ebb0 msgr2=0x5620dc287c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.528+0000 7f12a7e00640 1 --2- 172.21.15.47:0/607116718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc28ebb0 0x5620dc287c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.528+0000 7f12a7e00640 1 -- 172.21.15.47:0/607116718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 msgr2=0x5620dc29a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.528+0000 7f12a7e00640 1 --2- 172.21.15.47:0/607116718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 0x5620dc29a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.528+0000 7f12a7e00640 1 -- 172.21.15.47:0/607116718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620dbec20f0 con 0x5620dc289050 2024-03-21T11:36:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.528+0000 7f12a7e00640 1 --2- 172.21.15.47:0/607116718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc289050 0x5620dc27f920 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f12a0004920 tx=0x7f12a0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39088b7ddbda8a39 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.528+0000 7f1293400640 1 -- 172.21.15.47:0/607116718 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a002c020 con 0x5620dc289050 2024-03-21T11:36:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.528+0000 7f1293400640 1 -- 172.21.15.47:0/607116718 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12a001f040 con 0x5620dc289050 2024-03-21T11:36:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.528+0000 7f1293400640 1 -- 172.21.15.47:0/607116718 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0003e00 con 0x5620dc289050 2024-03-21T11:36:16.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.529+0000 7f12b0f88740 1 -- 172.21.15.47:0/607116718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc289050 msgr2=0x5620dc27f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:16.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.529+0000 7f12b0f88740 1 --2- 172.21.15.47:0/607116718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc289050 0x5620dc27f920 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f12a0004920 tx=0x7f12a0011810 comp rx=0 tx=0).stop 2024-03-21T11:36:16.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.529+0000 7f12b0f88740 1 -- 172.21.15.47:0/607116718 shutdown_connections 2024-03-21T11:36:16.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.529+0000 7f12b0f88740 1 --2- 172.21.15.47:0/607116718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc28ebb0 0x5620dc287c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.529+0000 7f12b0f88740 1 --2- 172.21.15.47:0/607116718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc289a40 0x5620dc29a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.529+0000 7f12b0f88740 1 --2- 172.21.15.47:0/607116718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc289050 0x5620dc27f920 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.529+0000 7f12b0f88740 1 -- 172.21.15.47:0/607116718 >> 172.21.15.47:0/607116718 conn(0x5620dc251200 msgr2=0x5620dc28ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:16.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.529+0000 7f12b0f88740 1 -- 172.21.15.47:0/607116718 shutdown_connections 2024-03-21T11:36:16.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.529+0000 7f12b0f88740 1 -- 172.21.15.47:0/607116718 wait complete. 2024-03-21T11:36:16.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.530+0000 7f12b0f88740 1 -- start start 2024-03-21T11:36:16.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.530+0000 7f12b0f88740 1 Processor -- start 2024-03-21T11:36:16.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.530+0000 7f12b0f88740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 0x5620dc298410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.530+0000 7f12b0f88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc289a40 0x5620dc298950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.530+0000 7f12b0f88740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc28ebb0 0x5620dc299960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.530+0000 7f12b0f88740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a81ad3e0 con 0x5620dc28ebb0 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.530+0000 7f12b0f88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a81acd80 con 0x5620dc289a40 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.530+0000 7f12b0f88740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a81aa280 con 0x5620dc289050 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.530+0000 7f12b0f88740 10 client.0 _ll_register_callbacks cb 0x7f1294009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.531+0000 7f12a6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc289a40 0x5620dc298950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.531+0000 7f12a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 0x5620dc298410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.531+0000 7f12a6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc289a40 0x5620dc298950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58850/0 (socket says 172.21.15.47:58850) 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.531+0000 7f12a6a00640 1 -- 172.21.15.47:0/1724795483 learned_addr learned my addr 172.21.15.47:0/1724795483 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.531+0000 7f12a7e00640 1 --2- 172.21.15.47:0/1724795483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc28ebb0 0x5620dc299960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.531+0000 7f12a6a00640 1 -- 172.21.15.47:0/1724795483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 msgr2=0x5620dc298410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.531+0000 7f12a6a00640 1 --2- 172.21.15.47:0/1724795483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 0x5620dc298410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.531+0000 7f12a6a00640 1 -- 172.21.15.47:0/1724795483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc28ebb0 msgr2=0x5620dc299960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.531+0000 7f12a6a00640 1 --2- 172.21.15.47:0/1724795483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc28ebb0 0x5620dc299960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.531+0000 7f12a6a00640 1 -- 172.21.15.47:0/1724795483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620dc299f50 con 0x5620dc289a40 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.531+0000 7f12a7e00640 1 --2- 172.21.15.47:0/1724795483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 0x5620dc298410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:16.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.531+0000 7f12a6a00640 1 --2- 172.21.15.47:0/1724795483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc289a40 0x5620dc298950 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x5620dc2858e0 tx=0x7f1294012320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82d5bb6930b13919 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.531+0000 7f1290c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.532+0000 7f1290c00640 1 -- 172.21.15.47:0/1724795483 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129402e060 con 0x5620dc289a40 2024-03-21T11:36:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.532+0000 7f12b0f88740 1 -- 172.21.15.47:0/1724795483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12a001f040 con 0x5620dc289a40 2024-03-21T11:36:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.532+0000 7f1290c00640 1 -- 172.21.15.47:0/1724795483 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1294012860 con 0x5620dc289a40 2024-03-21T11:36:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.532+0000 7f1290c00640 1 -- 172.21.15.47:0/1724795483 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1294035050 con 0x5620dc289a40 2024-03-21T11:36:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.532+0000 7f12b0f88740 10 client.29783 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.532+0000 7f12b0f88740 1 -- 172.21.15.47:0/1724795483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5620dc2faf70 con 0x5620dc289a40 2024-03-21T11:36:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.532+0000 7f1290c00640 1 -- 172.21.15.47:0/1724795483 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12a001f040 con 0x5620dc289a40 2024-03-21T11:36:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.532+0000 7f1290c00640 1 -- 172.21.15.47:0/1724795483 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f12940035f0 con 0x5620dc289a40 2024-03-21T11:36:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.532+0000 7f12b0f88740 10 client.29783 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.532+0000 7f12b0f88740 10 client.29783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.532+0000 7f12b0f88740 20 client.29783 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.532+0000 7f12b0f88740 1 --2- 172.21.15.47:0/1724795483 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620dc2fb890 0x5620dc2fdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.533+0000 7f12b0f88740 4 client.29783 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:36:16.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.533+0000 7f12b0f88740 1 -- 172.21.15.47:0/1724795483 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5620dc1f5ae0 con 0x5620dc2fb890 2024-03-21T11:36:16.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.533+0000 7f12a7e00640 1 --2- 172.21.15.47:0/1724795483 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620dc2fb890 0x5620dc2fdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.533+0000 7f12a7e00640 1 --2- 172.21.15.47:0/1724795483 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620dc2fb890 0x5620dc2fdcb0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91af923ff499677b server_cookie=681ff2f772929518 in_seq=0 out_seq=0 2024-03-21T11:36:16.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.533+0000 7f1290c00640 10 client.29783 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:16.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f1290c00640 1 -- 172.21.15.47:0/1724795483 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5620dc1f5ae0 con 0x5620dc2fb890 2024-03-21T11:36:16.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f1290c00640 10 client.29783 handle_command_reply: tid=0 2024-03-21T11:36:16.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f12b0f88740 1 client.29783 shutdown 2024-03-21T11:36:16.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f12b0f88740 1 -- 172.21.15.47:0/1724795483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc289a40 msgr2=0x5620dc298950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:16.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f12b0f88740 1 --2- 172.21.15.47:0/1724795483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc289a40 0x5620dc298950 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x5620dc2858e0 tx=0x7f1294012320 comp rx=0 tx=0).stop 2024-03-21T11:36:16.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f12b0f88740 1 -- 172.21.15.47:0/1724795483 shutdown_connections 2024-03-21T11:36:16.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f12b0f88740 1 --2- 172.21.15.47:0/1724795483 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5620dc2fb890 0x5620dc2fdcb0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f12b0f88740 1 --2- 172.21.15.47:0/1724795483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620dc28ebb0 0x5620dc299960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f12b0f88740 1 --2- 172.21.15.47:0/1724795483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620dc289a40 0x5620dc298950 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f12b0f88740 1 --2- 172.21.15.47:0/1724795483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5620dc289050 0x5620dc298410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f1290c00640 0 client.29783 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:16.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f12b0f88740 1 -- 172.21.15.47:0/1724795483 >> 172.21.15.47:0/1724795483 conn(0x5620dc251200 msgr2=0x5620dc28a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:16.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f12b0f88740 1 -- 172.21.15.47:0/1724795483 shutdown_connections 2024-03-21T11:36:16.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f12b0f88740 1 -- 172.21.15.47:0/1724795483 wait complete. 2024-03-21T11:36:16.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.534+0000 7f12b0f88740 20 client.29783 trim_cache size 0 max 16384 2024-03-21T11:36:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:36:16.540 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:36:16.540 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:36:16.540 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:36:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.535+0000 7f12acc00640 1 -- 172.21.15.47:0/2866552375 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f128c0a2f20 msgr2=0x7f128c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:16.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.535+0000 7f12acc00640 1 --2- 172.21.15.47:0/2866552375 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f128c0a2f20 0x7f128c0a53e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f12980ba510 tx=0x7f12980f7530 comp rx=0 tx=0).stop 2024-03-21T11:36:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.535+0000 7f12acc00640 1 -- 172.21.15.47:0/2866552375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a81037a0 msgr2=0x7f12a8199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.535+0000 7f12acc00640 1 --2- 172.21.15.47:0/2866552375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a81037a0 0x7f12a8199ab0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1298000c00 tx=0x7f129802c650 comp rx=0 tx=0).stop 2024-03-21T11:36:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.535+0000 7f12acc00640 1 -- 172.21.15.47:0/2866552375 shutdown_connections 2024-03-21T11:36:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.535+0000 7f12acc00640 1 --2- 172.21.15.47:0/2866552375 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f128c0a2f20 0x7f128c0a53e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.535+0000 7f12acc00640 1 --2- 172.21.15.47:0/2866552375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a8105350 0x7f12a81a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.535+0000 7f12acc00640 1 --2- 172.21.15.47:0/2866552375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a8104990 0x7f12a8199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.535+0000 7f12acc00640 1 --2- 172.21.15.47:0/2866552375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12a81037a0 0x7f12a8199ab0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.535+0000 7f12acc00640 1 -- 172.21.15.47:0/2866552375 >> 172.21.15.47:0/2866552375 conn(0x7f12a80fed10 msgr2=0x7f12a8107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.535+0000 7f12acc00640 1 -- 172.21.15.47:0/2866552375 shutdown_connections 2024-03-21T11:36:16.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.535+0000 7f12acc00640 1 -- 172.21.15.47:0/2866552375 wait complete. 2024-03-21T11:36:16.553 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:36:16.553 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:36:16.578 INFO:tasks.workunit.client.0.smithi047.stdout: 1 984368 7.94 MB/sec execute 540 sec latency 29.468 ms 2024-03-21T11:36:16.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.781+0000 7efedd800640 1 -- 172.21.15.47:0/3967790583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed8103700 msgr2=0x7efed8103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:16.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.781+0000 7efedd800640 1 --2- 172.21.15.47:0/3967790583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed8103700 0x7efed8103b20 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7efec8009900 tx=0x7efec802bc30 comp rx=0 tx=0).stop 2024-03-21T11:36:16.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.782+0000 7efedd800640 1 -- 172.21.15.47:0/3967790583 shutdown_connections 2024-03-21T11:36:16.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.782+0000 7efedd800640 1 --2- 172.21.15.47:0/3967790583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efed81052b0 0x7efed810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.782+0000 7efedd800640 1 --2- 172.21.15.47:0/3967790583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed81048f0 0x7efed8104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.782+0000 7efedd800640 1 --2- 172.21.15.47:0/3967790583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed8103700 0x7efed8103b20 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.782+0000 7efedd800640 1 -- 172.21.15.47:0/3967790583 >> 172.21.15.47:0/3967790583 conn(0x7efed80fec70 msgr2=0x7efed81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:16.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.782+0000 7efedd800640 1 -- 172.21.15.47:0/3967790583 shutdown_connections 2024-03-21T11:36:16.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.782+0000 7efedd800640 1 -- 172.21.15.47:0/3967790583 wait complete. 2024-03-21T11:36:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.783+0000 7efedd800640 1 Processor -- start 2024-03-21T11:36:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.783+0000 7efedd800640 1 -- start start 2024-03-21T11:36:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.783+0000 7efedd800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed8103700 0x7efed8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.783+0000 7efedd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed81048f0 0x7efed8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.783+0000 7efedd800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efed81052b0 0x7efed81a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.783+0000 7efedd800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed810f3c0 con 0x7efed81052b0 2024-03-21T11:36:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.783+0000 7efedd800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed810f240 con 0x7efed8103700 2024-03-21T11:36:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.783+0000 7efedd800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed810f540 con 0x7efed81048f0 2024-03-21T11:36:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.783+0000 7efed7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed8103700 0x7efed8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.783+0000 7efecfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed81048f0 0x7efed8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.783+0000 7efed7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed8103700 0x7efed8199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58860/0 (socket says 172.21.15.47:58860) 2024-03-21T11:36:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.783+0000 7efed7e00640 1 -- 172.21.15.47:0/3823484210 learned_addr learned my addr 172.21.15.47:0/3823484210 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.784+0000 7efedce00640 1 --2- 172.21.15.47:0/3823484210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efed81052b0 0x7efed81a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.784+0000 7efecfe00640 1 -- 172.21.15.47:0/3823484210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed8103700 msgr2=0x7efed8199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.784+0000 7efecfe00640 1 --2- 172.21.15.47:0/3823484210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed8103700 0x7efed8199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.784+0000 7efecfe00640 1 -- 172.21.15.47:0/3823484210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efed81052b0 msgr2=0x7efed81a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.784+0000 7efecfe00640 1 --2- 172.21.15.47:0/3823484210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efed81052b0 0x7efed81a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.784+0000 7efecfe00640 1 -- 172.21.15.47:0/3823484210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed81a1650 con 0x7efed81048f0 2024-03-21T11:36:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.784+0000 7efecfe00640 1 --2- 172.21.15.47:0/3823484210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed81048f0 0x7efed8199f70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7efec000e9f0 tx=0x7efec000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.784+0000 7efed5600640 1 -- 172.21.15.47:0/3823484210 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000ccc0 con 0x7efed81048f0 2024-03-21T11:36:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.785+0000 7efed5600640 1 -- 172.21.15.47:0/3823484210 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efec0005020 con 0x7efed81048f0 2024-03-21T11:36:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.785+0000 7efedd800640 1 -- 172.21.15.47:0/3823484210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed81a1940 con 0x7efed81048f0 2024-03-21T11:36:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.785+0000 7efed5600640 1 -- 172.21.15.47:0/3823484210 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00049e0 con 0x7efed81048f0 2024-03-21T11:36:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.785+0000 7efedd800640 1 -- 172.21.15.47:0/3823484210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed81a1e90 con 0x7efed81048f0 2024-03-21T11:36:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.786+0000 7efedd800640 1 -- 172.21.15.47:0/3823484210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed810aa20 con 0x7efed81048f0 2024-03-21T11:36:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.788+0000 7efed5600640 1 -- 172.21.15.47:0/3823484210 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7efec0005560 con 0x7efed81048f0 2024-03-21T11:36:16.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.790+0000 7efed5600640 1 --2- 172.21.15.47:0/3823484210 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efebc0a2fa0 0x7efebc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.790+0000 7efed5600640 1 -- 172.21.15.47:0/3823484210 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efec00cf770 con 0x7efed81048f0 2024-03-21T11:36:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.790+0000 7efed7e00640 1 --2- 172.21.15.47:0/3823484210 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efebc0a2fa0 0x7efebc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.791+0000 7efed7e00640 1 --2- 172.21.15.47:0/3823484210 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efebc0a2fa0 0x7efebc0a5460 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efec8006010 tx=0x7efec802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:16.799+0000 7efed5600640 1 -- 172.21.15.47:0/3823484210 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efec00051d0 con 0x7efed81048f0 2024-03-21T11:36:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.067+0000 7efedd800640 1 -- 172.21.15.47:0/3823484210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efed802d070 con 0x7efed81048f0 2024-03-21T11:36:17.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.068+0000 7efed5600640 1 -- 172.21.15.47:0/3823484210 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7efec009c1b0 con 0x7efed81048f0 2024-03-21T11:36:17.069 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:36:17.069 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:36:17.069 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:36:17.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.071+0000 7efedd800640 1 -- 172.21.15.47:0/3823484210 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efebc0a2fa0 msgr2=0x7efebc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.071+0000 7efedd800640 1 --2- 172.21.15.47:0/3823484210 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efebc0a2fa0 0x7efebc0a5460 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efec8006010 tx=0x7efec802df90 comp rx=0 tx=0).stop 2024-03-21T11:36:17.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.071+0000 7efedd800640 1 -- 172.21.15.47:0/3823484210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed81048f0 msgr2=0x7efed8199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.071+0000 7efedd800640 1 --2- 172.21.15.47:0/3823484210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed81048f0 0x7efed8199f70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7efec000e9f0 tx=0x7efec000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.071+0000 7efedd800640 1 -- 172.21.15.47:0/3823484210 shutdown_connections 2024-03-21T11:36:17.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.071+0000 7efedd800640 1 --2- 172.21.15.47:0/3823484210 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efebc0a2fa0 0x7efebc0a5460 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.071+0000 7efedd800640 1 --2- 172.21.15.47:0/3823484210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efed81052b0 0x7efed81a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.071+0000 7efedd800640 1 --2- 172.21.15.47:0/3823484210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed81048f0 0x7efed8199f70 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.071+0000 7efedd800640 1 --2- 172.21.15.47:0/3823484210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efed8103700 0x7efed8199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.072+0000 7efedd800640 1 -- 172.21.15.47:0/3823484210 >> 172.21.15.47:0/3823484210 conn(0x7efed80fec70 msgr2=0x7efed8107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.072+0000 7efedd800640 1 -- 172.21.15.47:0/3823484210 shutdown_connections 2024-03-21T11:36:17.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.072+0000 7efedd800640 1 -- 172.21.15.47:0/3823484210 wait complete. 2024-03-21T11:36:17.094 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:36:17.094 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:36:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:17 smithi047 ceph-mon[31351]: pgmap v788: 129 pgs: 129 active+clean; 35 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-03-21T11:36:17.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.362+0000 7f9595e00640 1 -- 172.21.15.47:0/625675286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9590105350 msgr2=0x7f959010bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.362+0000 7f9595e00640 1 --2- 172.21.15.47:0/625675286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9590105350 0x7f959010bbe0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f9588009ca0 tx=0x7f958802c100 comp rx=0 tx=0).stop 2024-03-21T11:36:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.362+0000 7f9595e00640 1 -- 172.21.15.47:0/625675286 shutdown_connections 2024-03-21T11:36:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.362+0000 7f9595e00640 1 --2- 172.21.15.47:0/625675286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9590105350 0x7f959010bbe0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.362+0000 7f9595e00640 1 --2- 172.21.15.47:0/625675286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9590104990 0x7f9590104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.362+0000 7f9595e00640 1 --2- 172.21.15.47:0/625675286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95901037a0 0x7f9590103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.362+0000 7f9595e00640 1 -- 172.21.15.47:0/625675286 >> 172.21.15.47:0/625675286 conn(0x7f95900fed10 msgr2=0x7f9590101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.363+0000 7f9595e00640 1 -- 172.21.15.47:0/625675286 shutdown_connections 2024-03-21T11:36:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.363+0000 7f9595e00640 1 -- 172.21.15.47:0/625675286 wait complete. 2024-03-21T11:36:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.363+0000 7f9595e00640 1 Processor -- start 2024-03-21T11:36:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.364+0000 7f9595e00640 1 -- start start 2024-03-21T11:36:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.364+0000 7f9595e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95901037a0 0x7f9590199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.364+0000 7f9595e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9590104990 0x7f9590199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.364+0000 7f9595e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9590105350 0x7f95901a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.364+0000 7f9595e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959010f420 con 0x7f95901037a0 2024-03-21T11:36:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.364+0000 7f9595e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959010f2a0 con 0x7f9590105350 2024-03-21T11:36:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.364+0000 7f9595e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959010f5a0 con 0x7f9590104990 2024-03-21T11:36:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.364+0000 7f9587e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9590104990 0x7f9590199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.364+0000 7f9595400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9590105350 0x7f95901a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.364+0000 7f9587e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9590104990 0x7f9590199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44638/0 (socket says 172.21.15.47:44638) 2024-03-21T11:36:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.364+0000 7f9587e00640 1 -- 172.21.15.47:0/525339742 learned_addr learned my addr 172.21.15.47:0/525339742 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.364+0000 7f9594a00640 1 --2- 172.21.15.47:0/525339742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95901037a0 0x7f9590199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.365+0000 7f9587e00640 1 -- 172.21.15.47:0/525339742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9590105350 msgr2=0x7f95901a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.365+0000 7f9587e00640 1 --2- 172.21.15.47:0/525339742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9590105350 0x7f95901a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.365+0000 7f9587e00640 1 -- 172.21.15.47:0/525339742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95901037a0 msgr2=0x7f9590199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.365+0000 7f9587e00640 1 --2- 172.21.15.47:0/525339742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95901037a0 0x7f9590199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.365+0000 7f9587e00640 1 -- 172.21.15.47:0/525339742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95901a1660 con 0x7f9590104990 2024-03-21T11:36:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.365+0000 7f9595400640 1 --2- 172.21.15.47:0/525339742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9590105350 0x7f95901a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:17.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.365+0000 7f9587e00640 1 --2- 172.21.15.47:0/525339742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9590104990 0x7f9590199f80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f957800e9f0 tx=0x7f957800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.365+0000 7f958e000640 1 -- 172.21.15.47:0/525339742 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957800cd80 con 0x7f9590104990 2024-03-21T11:36:17.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.365+0000 7f958e000640 1 -- 172.21.15.47:0/525339742 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9578010910 con 0x7f9590104990 2024-03-21T11:36:17.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.365+0000 7f958e000640 1 -- 172.21.15.47:0/525339742 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578018990 con 0x7f9590104990 2024-03-21T11:36:17.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.365+0000 7f9595e00640 1 -- 172.21.15.47:0/525339742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95901a1950 con 0x7f9590104990 2024-03-21T11:36:17.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.366+0000 7f9595e00640 1 -- 172.21.15.47:0/525339742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95901a1ef0 con 0x7f9590104990 2024-03-21T11:36:17.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.369+0000 7f958e000640 1 -- 172.21.15.47:0/525339742 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9578010ac0 con 0x7f9590104990 2024-03-21T11:36:17.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.369+0000 7f959a1ec740 1 Processor -- start 2024-03-21T11:36:17.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.369+0000 7f959a1ec740 1 -- start start 2024-03-21T11:36:17.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.369+0000 7f959a1ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613ece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.369+0000 7f959a1ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613ee040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.369+0000 7f959a1ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ee580 0x5634613f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.369+0000 7f959a1ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634613512c0 con 0x5634613ee580 2024-03-21T11:36:17.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.369+0000 7f959a1ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634612f9c90 con 0x5634613ec9f0 2024-03-21T11:36:17.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.369+0000 7f959a1ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56346134d360 con 0x5634613edbc0 2024-03-21T11:36:17.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.370+0000 7f9595400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613ece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.370+0000 7f9595400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613ece10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58892/0 (socket says 172.21.15.47:58892) 2024-03-21T11:36:17.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.370+0000 7f9595400640 1 -- 172.21.15.47:0/245906886 learned_addr learned my addr 172.21.15.47:0/245906886 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.370+0000 7f9594a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613ee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.370+0000 7f9587e00640 1 --2- 172.21.15.47:0/245906886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ee580 0x5634613f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.370+0000 7f9595400640 1 -- 172.21.15.47:0/245906886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 msgr2=0x5634613ee040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.370+0000 7f9595400640 1 --2- 172.21.15.47:0/245906886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613ee040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.370+0000 7f9595400640 1 -- 172.21.15.47:0/245906886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ee580 msgr2=0x5634613f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.370+0000 7f9595400640 1 --2- 172.21.15.47:0/245906886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ee580 0x5634613f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.370+0000 7f9595400640 1 -- 172.21.15.47:0/245906886 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56346131e080 con 0x5634613ec9f0 2024-03-21T11:36:17.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.370+0000 7f9587e00640 1 --2- 172.21.15.47:0/245906886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ee580 0x5634613f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:36:17.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.371+0000 7f958e000640 1 --2- 172.21.15.47:0/525339742 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95740a2fa0 0x7f95740a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.371+0000 7f958e000640 1 -- 172.21.15.47:0/525339742 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f95780cfa40 con 0x7f9590104990 2024-03-21T11:36:17.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.371+0000 7f9595400640 1 --2- 172.21.15.47:0/245906886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613ece10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f958802c710 tx=0x7f95880323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbf6de61a92c42eb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.371+0000 7f9594a00640 1 --2- 172.21.15.47:0/525339742 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95740a2fa0 0x7f95740a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.371+0000 7f9585c00640 1 -- 172.21.15.47:0/245906886 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588043070 con 0x5634613ec9f0 2024-03-21T11:36:17.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.371+0000 7f9585c00640 1 -- 172.21.15.47:0/245906886 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9588032c90 con 0x5634613ec9f0 2024-03-21T11:36:17.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.371+0000 7f9585c00640 1 -- 172.21.15.47:0/245906886 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588039040 con 0x5634613ec9f0 2024-03-21T11:36:17.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.372+0000 7f959a1ec740 1 -- 172.21.15.47:0/245906886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 msgr2=0x5634613ece10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.372+0000 7f959a1ec740 1 --2- 172.21.15.47:0/245906886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613ece10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f958802c710 tx=0x7f95880323d0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.372+0000 7f9594a00640 1 --2- 172.21.15.47:0/525339742 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95740a2fa0 0x7f95740a5460 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f95901047f0 tx=0x7f9580005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.372+0000 7f959a1ec740 1 -- 172.21.15.47:0/245906886 shutdown_connections 2024-03-21T11:36:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.372+0000 7f959a1ec740 1 --2- 172.21.15.47:0/245906886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ee580 0x5634613f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.372+0000 7f959a1ec740 1 --2- 172.21.15.47:0/245906886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613ee040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.372+0000 7f959a1ec740 1 --2- 172.21.15.47:0/245906886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613ece10 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.372+0000 7f959a1ec740 1 -- 172.21.15.47:0/245906886 >> 172.21.15.47:0/245906886 conn(0x5634613e7de0 msgr2=0x5634613e81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.372+0000 7f959a1ec740 1 -- 172.21.15.47:0/245906886 shutdown_connections 2024-03-21T11:36:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.372+0000 7f959a1ec740 1 -- 172.21.15.47:0/245906886 wait complete. 2024-03-21T11:36:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.373+0000 7f959a1ec740 1 -- start start 2024-03-21T11:36:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.373+0000 7f959a1ec740 1 Processor -- start 2024-03-21T11:36:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.373+0000 7f959a1ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.373+0000 7f959a1ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613f7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.373+0000 7f959a1ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ee580 0x5634613fe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.373+0000 7f959a1ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634613f5530 con 0x5634613ee580 2024-03-21T11:36:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.373+0000 7f959a1ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56346130d0d0 con 0x5634613ec9f0 2024-03-21T11:36:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.373+0000 7f959a1ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634613f56b0 con 0x5634613edbc0 2024-03-21T11:36:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.374+0000 7f959a1ec740 10 client.0 _ll_register_callbacks cb 0x563461311e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.374+0000 7f9587e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.374+0000 7f9594a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.374+0000 7f9587e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613f7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44664/0 (socket says 172.21.15.47:44664) 2024-03-21T11:36:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.374+0000 7f9587e00640 1 -- 172.21.15.47:0/627871345 learned_addr learned my addr 172.21.15.47:0/627871345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.374+0000 7f9595400640 1 --2- 172.21.15.47:0/627871345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ee580 0x5634613fe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.374+0000 7f9587e00640 1 -- 172.21.15.47:0/627871345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 msgr2=0x5634613f6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.374+0000 7f9587e00640 1 --2- 172.21.15.47:0/627871345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613f6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.374+0000 7f9587e00640 1 -- 172.21.15.47:0/627871345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ee580 msgr2=0x5634613fe180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.374+0000 7f9587e00640 1 --2- 172.21.15.47:0/627871345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ee580 0x5634613fe180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.374+0000 7f9587e00640 1 -- 172.21.15.47:0/627871345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634610653f0 con 0x5634613edbc0 2024-03-21T11:36:17.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.374+0000 7f9587e00640 1 --2- 172.21.15.47:0/627871345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613f7140 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f95780cd150 tx=0x7f957809bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c4766e6d64dab16 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.374+0000 7f957e000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:17.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.375+0000 7f957e000640 1 -- 172.21.15.47:0/627871345 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578018d30 con 0x5634613edbc0 2024-03-21T11:36:17.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.375+0000 7f957e000640 1 -- 172.21.15.47:0/627871345 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f957809b280 con 0x5634613edbc0 2024-03-21T11:36:17.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.375+0000 7f959a1ec740 1 -- 172.21.15.47:0/627871345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9588032c90 con 0x5634613edbc0 2024-03-21T11:36:17.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.375+0000 7f957e000640 1 -- 172.21.15.47:0/627871345 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f957809b4a0 con 0x5634613edbc0 2024-03-21T11:36:17.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.375+0000 7f959a1ec740 10 client.29836 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:17.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.375+0000 7f959a1ec740 1 -- 172.21.15.47:0/627871345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56346112f7d0 con 0x5634613edbc0 2024-03-21T11:36:17.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.375+0000 7f957e000640 1 -- 172.21.15.47:0/627871345 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95780b8080 con 0x5634613edbc0 2024-03-21T11:36:17.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.376+0000 7f957e000640 1 -- 172.21.15.47:0/627871345 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9578097ca0 con 0x5634613edbc0 2024-03-21T11:36:17.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.376+0000 7f959a1ec740 10 client.29836 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:17.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.376+0000 7f959a1ec740 10 client.29836 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:36:17.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.376+0000 7f959a1ec740 20 client.29836 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:17.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.376+0000 7f959a1ec740 1 --2- 172.21.15.47:0/627871345 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5634614026e0 0x563461404b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.376+0000 7f959a1ec740 4 client.29836 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:17.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.376+0000 7f959a1ec740 1 -- 172.21.15.47:0/627871345 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56346135e800 con 0x5634614026e0 2024-03-21T11:36:17.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.376+0000 7f9594a00640 1 --2- 172.21.15.47:0/627871345 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5634614026e0 0x563461404b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.377+0000 7f9594a00640 1 --2- 172.21.15.47:0/627871345 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5634614026e0 0x563461404b00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de8404ff60bba517 server_cookie=aac9ecf2b2a151f9 in_seq=0 out_seq=0 2024-03-21T11:36:17.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.377+0000 7f957e000640 10 client.29836 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:36:17.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f957e000640 1 -- 172.21.15.47:0/627871345 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56346135e800 con 0x5634614026e0 2024-03-21T11:36:17.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f957e000640 10 client.29836 handle_command_reply: tid=0 2024-03-21T11:36:17.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f959a1ec740 1 client.29836 shutdown 2024-03-21T11:36:17.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f959a1ec740 1 -- 172.21.15.47:0/627871345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 msgr2=0x5634613f7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f959a1ec740 1 --2- 172.21.15.47:0/627871345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613f7140 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f95780cd150 tx=0x7f957809bef0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f959a1ec740 1 -- 172.21.15.47:0/627871345 shutdown_connections 2024-03-21T11:36:17.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f959a1ec740 1 --2- 172.21.15.47:0/627871345 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5634614026e0 0x563461404b00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f959a1ec740 1 --2- 172.21.15.47:0/627871345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ee580 0x5634613fe180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f959a1ec740 1 --2- 172.21.15.47:0/627871345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613f7140 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f959a1ec740 1 --2- 172.21.15.47:0/627871345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f959a1ec740 1 -- 172.21.15.47:0/627871345 >> 172.21.15.47:0/627871345 conn(0x5634613e7de0 msgr2=0x5634613e7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f957e000640 0 client.29836 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:36:17.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f959a1ec740 1 -- 172.21.15.47:0/627871345 shutdown_connections 2024-03-21T11:36:17.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f959a1ec740 1 -- 172.21.15.47:0/627871345 wait complete. 2024-03-21T11:36:17.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.378+0000 7f959a1ec740 20 client.29836 trim_cache size 0 max 16384 2024-03-21T11:36:17.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:17 smithi100 ceph-mon[36245]: pgmap v788: 129 pgs: 129 active+clean; 35 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-03-21T11:36:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.394+0000 7f959a1ec740 1 Processor -- start 2024-03-21T11:36:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.394+0000 7f959a1ec740 1 -- start start 2024-03-21T11:36:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.394+0000 7f959a1ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613f12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.394+0000 7f959a1ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613edbc0 0x5634613fe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.394+0000 7f959a1ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613ee580 0x5634613f1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.394+0000 7f959a1ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95901ac070 con 0x5634613edbc0 2024-03-21T11:36:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.394+0000 7f959a1ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95901ad9f0 con 0x5634613ec9f0 2024-03-21T11:36:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.394+0000 7f959a1ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95901ac6d0 con 0x5634613ee580 2024-03-21T11:36:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.395+0000 7f9595400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613ee580 0x5634613f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.395+0000 7f9594a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613f12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.395+0000 7f9587e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613edbc0 0x5634613fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.395+0000 7f9594a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613f12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58920/0 (socket says 172.21.15.47:58920) 2024-03-21T11:36:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.395+0000 7f9595400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613ee580 0x5634613f1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44668/0 (socket says 172.21.15.47:44668) 2024-03-21T11:36:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.395+0000 7f9594a00640 1 -- 172.21.15.47:0/3940252830 learned_addr learned my addr 172.21.15.47:0/3940252830 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.395+0000 7f9595400640 1 -- 172.21.15.47:0/3940252830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 msgr2=0x5634613f12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.395+0000 7f9595400640 1 --2- 172.21.15.47:0/3940252830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613f12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.395+0000 7f9595400640 1 -- 172.21.15.47:0/3940252830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613edbc0 msgr2=0x5634613fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.395+0000 7f9595400640 1 --2- 172.21.15.47:0/3940252830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613edbc0 0x5634613fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.395+0000 7f9595400640 1 -- 172.21.15.47:0/3940252830 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634613f5140 con 0x5634613ee580 2024-03-21T11:36:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.395+0000 7f9594a00640 1 --2- 172.21.15.47:0/3940252830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613f12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.395+0000 7f9587e00640 1 --2- 172.21.15.47:0/3940252830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613edbc0 0x5634613fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.395+0000 7f9595400640 1 --2- 172.21.15.47:0/3940252830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613ee580 0x5634613f1810 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9588009ca0 tx=0x7f9588000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50d846b58ab4aa7d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.396+0000 7f9585200640 1 -- 172.21.15.47:0/3940252830 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588039030 con 0x5634613ee580 2024-03-21T11:36:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.396+0000 7f9585200640 1 -- 172.21.15.47:0/3940252830 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9588009880 con 0x5634613ee580 2024-03-21T11:36:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.396+0000 7f9585200640 1 -- 172.21.15.47:0/3940252830 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588043040 con 0x5634613ee580 2024-03-21T11:36:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.396+0000 7f959a1ec740 1 -- 172.21.15.47:0/3940252830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613ee580 msgr2=0x5634613f1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.396+0000 7f959a1ec740 1 --2- 172.21.15.47:0/3940252830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613ee580 0x5634613f1810 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9588009ca0 tx=0x7f9588000f40 comp rx=0 tx=0).stop 2024-03-21T11:36:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.397+0000 7f959a1ec740 1 -- 172.21.15.47:0/3940252830 shutdown_connections 2024-03-21T11:36:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.397+0000 7f959a1ec740 1 --2- 172.21.15.47:0/3940252830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613ee580 0x5634613f1810 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.397+0000 7f959a1ec740 1 --2- 172.21.15.47:0/3940252830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613edbc0 0x5634613fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.397+0000 7f959a1ec740 1 --2- 172.21.15.47:0/3940252830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ec9f0 0x5634613f12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.397+0000 7f959a1ec740 1 -- 172.21.15.47:0/3940252830 >> 172.21.15.47:0/3940252830 conn(0x5634613e7de0 msgr2=0x5634613f0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.397+0000 7f959a1ec740 1 -- 172.21.15.47:0/3940252830 shutdown_connections 2024-03-21T11:36:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.397+0000 7f959a1ec740 1 -- 172.21.15.47:0/3940252830 wait complete. 2024-03-21T11:36:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.397+0000 7f959a1ec740 1 -- start start 2024-03-21T11:36:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.397+0000 7f959a1ec740 1 Processor -- start 2024-03-21T11:36:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f959a1ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ec9f0 0x5634613fde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f959a1ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613fa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f959a1ec740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ee580 0x5634613fae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f959a1ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95901ad390 con 0x5634613ec9f0 2024-03-21T11:36:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f959a1ec740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95901acd30 con 0x5634613ee580 2024-03-21T11:36:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f959a1ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95901aa230 con 0x5634613edbc0 2024-03-21T11:36:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f959a1ec740 10 client.0 _ll_register_callbacks cb 0x7f95780980d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f9587e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613fa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f9587e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613fa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44676/0 (socket says 172.21.15.47:44676) 2024-03-21T11:36:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f9587e00640 1 -- 172.21.15.47:0/902599952 learned_addr learned my addr 172.21.15.47:0/902599952 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f9594a00640 1 --2- 172.21.15.47:0/902599952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ec9f0 0x5634613fde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f9595400640 1 --2- 172.21.15.47:0/902599952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ee580 0x5634613fae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f9587e00640 1 -- 172.21.15.47:0/902599952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ee580 msgr2=0x5634613fae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f9587e00640 1 --2- 172.21.15.47:0/902599952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ee580 0x5634613fae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f9587e00640 1 -- 172.21.15.47:0/902599952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ec9f0 msgr2=0x5634613fde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f9587e00640 1 --2- 172.21.15.47:0/902599952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ec9f0 0x5634613fde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.398+0000 7f9587e00640 1 -- 172.21.15.47:0/902599952 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634613fb4c0 con 0x5634613edbc0 2024-03-21T11:36:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.399+0000 7f9587e00640 1 --2- 172.21.15.47:0/902599952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613fa910 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9578098800 tx=0x7f9578018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40abff3ab555caf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.399+0000 7f957e000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.399+0000 7f957e000640 1 -- 172.21.15.47:0/902599952 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780a9060 con 0x5634613edbc0 2024-03-21T11:36:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.399+0000 7f957e000640 1 -- 172.21.15.47:0/902599952 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f95780baaf0 con 0x5634613edbc0 2024-03-21T11:36:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.399+0000 7f959a1ec740 1 -- 172.21.15.47:0/902599952 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9588009880 con 0x5634613edbc0 2024-03-21T11:36:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.399+0000 7f957e000640 1 -- 172.21.15.47:0/902599952 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95780b0050 con 0x5634613edbc0 2024-03-21T11:36:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.399+0000 7f959a1ec740 10 client.29842 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.399+0000 7f959a1ec740 1 -- 172.21.15.47:0/902599952 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56346145f070 con 0x5634613edbc0 2024-03-21T11:36:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.400+0000 7f957e000640 1 -- 172.21.15.47:0/902599952 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9588009880 con 0x5634613edbc0 2024-03-21T11:36:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.400+0000 7f957e000640 1 -- 172.21.15.47:0/902599952 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f95780a1060 con 0x5634613edbc0 2024-03-21T11:36:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.400+0000 7f959a1ec740 10 client.29842 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.400+0000 7f959a1ec740 10 client.29842 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:36:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.400+0000 7f959a1ec740 20 client.29842 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.400+0000 7f959a1ec740 1 --2- 172.21.15.47:0/902599952 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x56346145f830 0x563461461c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.400+0000 7f959a1ec740 4 client.29842 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:36:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.400+0000 7f959a1ec740 1 -- 172.21.15.47:0/902599952 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5634613f69f0 con 0x56346145f830 2024-03-21T11:36:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.400+0000 7f9594a00640 1 --2- 172.21.15.47:0/902599952 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x56346145f830 0x563461461c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f9594a00640 1 --2- 172.21.15.47:0/902599952 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x56346145f830 0x563461461c50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a116afc3ac828b70 server_cookie=85a17ff9c52dff3c in_seq=0 out_seq=0 2024-03-21T11:36:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f957e000640 10 client.29842 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:36:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f957e000640 1 -- 172.21.15.47:0/902599952 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5634613f69f0 con 0x56346145f830 2024-03-21T11:36:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f957e000640 10 client.29842 handle_command_reply: tid=0 2024-03-21T11:36:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f959a1ec740 1 client.29842 shutdown 2024-03-21T11:36:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f959a1ec740 1 -- 172.21.15.47:0/902599952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 msgr2=0x5634613fa910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f959a1ec740 1 --2- 172.21.15.47:0/902599952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613fa910 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9578098800 tx=0x7f9578018420 comp rx=0 tx=0).stop 2024-03-21T11:36:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f959a1ec740 1 -- 172.21.15.47:0/902599952 shutdown_connections 2024-03-21T11:36:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f959a1ec740 1 --2- 172.21.15.47:0/902599952 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x56346145f830 0x563461461c50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f959a1ec740 1 --2- 172.21.15.47:0/902599952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5634613ee580 0x5634613fae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f959a1ec740 1 --2- 172.21.15.47:0/902599952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5634613edbc0 0x5634613fa910 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f957e000640 0 client.29842 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:36:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f959a1ec740 1 --2- 172.21.15.47:0/902599952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5634613ec9f0 0x5634613fde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f959a1ec740 1 -- 172.21.15.47:0/902599952 >> 172.21.15.47:0/902599952 conn(0x5634613e7de0 msgr2=0x563461409c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f959a1ec740 1 -- 172.21.15.47:0/902599952 shutdown_connections 2024-03-21T11:36:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f959a1ec740 1 -- 172.21.15.47:0/902599952 wait complete. 2024-03-21T11:36:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.401+0000 7f959a1ec740 20 client.29842 trim_cache size 0 max 16384 2024-03-21T11:36:17.406 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:36:17.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.402+0000 7f9595e00640 1 -- 172.21.15.47:0/525339742 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95740a2fa0 msgr2=0x7f95740a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.402+0000 7f9595e00640 1 --2- 172.21.15.47:0/525339742 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95740a2fa0 0x7f95740a5460 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f95901047f0 tx=0x7f9580005f90 comp rx=0 tx=0).stop 2024-03-21T11:36:17.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.402+0000 7f9595e00640 1 -- 172.21.15.47:0/525339742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9590104990 msgr2=0x7f9590199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.402+0000 7f9595e00640 1 --2- 172.21.15.47:0/525339742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9590104990 0x7f9590199f80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f957800e9f0 tx=0x7f957800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.402+0000 7f9595e00640 1 -- 172.21.15.47:0/525339742 shutdown_connections 2024-03-21T11:36:17.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.402+0000 7f9595e00640 1 --2- 172.21.15.47:0/525339742 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f95740a2fa0 0x7f95740a5460 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.402+0000 7f9595e00640 1 --2- 172.21.15.47:0/525339742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9590105350 0x7f95901a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.402+0000 7f9595e00640 1 --2- 172.21.15.47:0/525339742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9590104990 0x7f9590199f80 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.402+0000 7f9595e00640 1 --2- 172.21.15.47:0/525339742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95901037a0 0x7f9590199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.402+0000 7f9595e00640 1 -- 172.21.15.47:0/525339742 >> 172.21.15.47:0/525339742 conn(0x7f95900fed10 msgr2=0x7f9590107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.402+0000 7f9595e00640 1 -- 172.21.15.47:0/525339742 shutdown_connections 2024-03-21T11:36:17.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.402+0000 7f9595e00640 1 -- 172.21.15.47:0/525339742 wait complete. 2024-03-21T11:36:17.420 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:36:17.420 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:36:17.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:17 smithi102 ceph-mon[36748]: pgmap v788: 129 pgs: 129 active+clean; 35 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-03-21T11:36:17.578 INFO:tasks.workunit.client.0.smithi047.stdout: 1 985581 7.94 MB/sec execute 541 sec latency 18.143 ms 2024-03-21T11:36:17.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.640+0000 7f565e200640 1 -- 172.21.15.47:0/1424927534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56581016b0 msgr2=0x7f5658101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.640+0000 7f565e200640 1 --2- 172.21.15.47:0/1424927534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56581016b0 0x7f5658101ad0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5648009900 tx=0x7f564802bc40 comp rx=0 tx=0).stop 2024-03-21T11:36:17.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.640+0000 7f565e200640 1 -- 172.21.15.47:0/1424927534 shutdown_connections 2024-03-21T11:36:17.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.640+0000 7f565e200640 1 --2- 172.21.15.47:0/1424927534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5658103260 0x7f5658109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.640+0000 7f565e200640 1 --2- 172.21.15.47:0/1424927534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56581028a0 0x7f5658102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.640+0000 7f565e200640 1 --2- 172.21.15.47:0/1424927534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56581016b0 0x7f5658101ad0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.640+0000 7f565e200640 1 -- 172.21.15.47:0/1424927534 >> 172.21.15.47:0/1424927534 conn(0x7f56580fcc20 msgr2=0x7f56580ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.641+0000 7f565e200640 1 -- 172.21.15.47:0/1424927534 shutdown_connections 2024-03-21T11:36:17.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.641+0000 7f565e200640 1 -- 172.21.15.47:0/1424927534 wait complete. 2024-03-21T11:36:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.642+0000 7f565e200640 1 Processor -- start 2024-03-21T11:36:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.642+0000 7f565e200640 1 -- start start 2024-03-21T11:36:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.642+0000 7f565e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56581016b0 0x7f5658199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.642+0000 7f565e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56581028a0 0x7f565819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.642+0000 7f565e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5658103260 0x7f56581a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.642+0000 7f565e200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565810d300 con 0x7f56581028a0 2024-03-21T11:36:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.642+0000 7f565e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565810d180 con 0x7f5658103260 2024-03-21T11:36:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.642+0000 7f565e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565810d480 con 0x7f56581016b0 2024-03-21T11:36:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.642+0000 7f5657400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56581016b0 0x7f5658199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.642+0000 7f5657400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56581016b0 0x7f5658199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44686/0 (socket says 172.21.15.47:44686) 2024-03-21T11:36:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.642+0000 7f5657400640 1 -- 172.21.15.47:0/2503596999 learned_addr learned my addr 172.21.15.47:0/2503596999 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.642+0000 7f5657e00640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5658103260 0x7f56581a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.642+0000 7f564ea00640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56581028a0 0x7f565819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.643+0000 7f5657400640 1 -- 172.21.15.47:0/2503596999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5658103260 msgr2=0x7f56581a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.643+0000 7f5657400640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5658103260 0x7f56581a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.643+0000 7f5657400640 1 -- 172.21.15.47:0/2503596999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56581028a0 msgr2=0x7f565819a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.643+0000 7f5657400640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56581028a0 0x7f565819a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.643+0000 7f5657400640 1 -- 172.21.15.47:0/2503596999 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56581a1840 con 0x7f56581016b0 2024-03-21T11:36:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.643+0000 7f5657e00640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5658103260 0x7f56581a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.643+0000 7f5657400640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56581016b0 0x7f5658199c30 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5648000c00 tx=0x7f564802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.643+0000 7f5654c00640 1 -- 172.21.15.47:0/2503596999 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648041070 con 0x7f56581016b0 2024-03-21T11:36:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.644+0000 7f5654c00640 1 -- 172.21.15.47:0/2503596999 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5648004450 con 0x7f56581016b0 2024-03-21T11:36:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.644+0000 7f5654c00640 1 -- 172.21.15.47:0/2503596999 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648034440 con 0x7f56581016b0 2024-03-21T11:36:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.644+0000 7f565e200640 1 -- 172.21.15.47:0/2503596999 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56581a1ad0 con 0x7f56581016b0 2024-03-21T11:36:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.644+0000 7f565e200640 1 -- 172.21.15.47:0/2503596999 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56581a1f70 con 0x7f56581016b0 2024-03-21T11:36:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.647+0000 7f5660f82740 1 Processor -- start 2024-03-21T11:36:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.647+0000 7f5660f82740 1 -- start start 2024-03-21T11:36:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.647+0000 7f5654c00640 1 -- 172.21.15.47:0/2503596999 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5648044070 con 0x7f56581016b0 2024-03-21T11:36:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5660f82740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8faa020 0x55b5b8faa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5660f82740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fab670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5660f82740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8fb2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5660f82740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b8f062c0 con 0x55b5b8fab1f0 2024-03-21T11:36:17.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5660f82740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b8eaec90 con 0x55b5b8faa020 2024-03-21T11:36:17.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5660f82740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b8f02360 con 0x55b5b8fabbb0 2024-03-21T11:36:17.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5657e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8faa020 0x55b5b8faa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5657e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8faa020 0x55b5b8faa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58944/0 (socket says 172.21.15.47:58944) 2024-03-21T11:36:17.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5657e00640 1 -- 172.21.15.47:0/2827563613 learned_addr learned my addr 172.21.15.47:0/2827563613 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f564ea00640 1 --2- 172.21.15.47:0/2827563613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8fb2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5657400640 1 --2- 172.21.15.47:0/2827563613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5657e00640 1 -- 172.21.15.47:0/2827563613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 msgr2=0x55b5b8fb2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5657e00640 1 --2- 172.21.15.47:0/2827563613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8fb2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5657e00640 1 -- 172.21.15.47:0/2827563613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fab1f0 msgr2=0x55b5b8fab670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5657e00640 1 --2- 172.21.15.47:0/2827563613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fab670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.648+0000 7f5657e00640 1 -- 172.21.15.47:0/2827563613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5b8ed3080 con 0x55b5b8faa020 2024-03-21T11:36:17.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.649+0000 7f564ea00640 1 --2- 172.21.15.47:0/2827563613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8fb2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:17.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.649+0000 7f5657e00640 1 --2- 172.21.15.47:0/2827563613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8faa020 0x55b5b8faa440 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f56500117e0 tx=0x7f5650011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c8ea4581f56e528 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.649+0000 7f5647e00640 1 -- 172.21.15.47:0/2827563613 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5650011310 con 0x55b5b8faa020 2024-03-21T11:36:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.649+0000 7f5647e00640 1 -- 172.21.15.47:0/2827563613 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5650003e50 con 0x55b5b8faa020 2024-03-21T11:36:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.649+0000 7f5647e00640 1 -- 172.21.15.47:0/2827563613 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5650004560 con 0x55b5b8faa020 2024-03-21T11:36:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.649+0000 7f5654c00640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f563c0a2fa0 0x7f563c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.649+0000 7f5654c00640 1 -- 172.21.15.47:0/2503596999 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f56480f12c0 con 0x7f56581016b0 2024-03-21T11:36:17.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.649+0000 7f5657400640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f563c0a2fa0 0x7f563c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.650+0000 7f5660f82740 1 -- 172.21.15.47:0/2827563613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8faa020 msgr2=0x55b5b8faa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.650+0000 7f5660f82740 1 --2- 172.21.15.47:0/2827563613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8faa020 0x55b5b8faa440 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f56500117e0 tx=0x7f5650011ca0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.650+0000 7f5660f82740 1 -- 172.21.15.47:0/2827563613 shutdown_connections 2024-03-21T11:36:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.650+0000 7f5660f82740 1 --2- 172.21.15.47:0/2827563613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8fb2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.650+0000 7f5660f82740 1 --2- 172.21.15.47:0/2827563613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.650+0000 7f5660f82740 1 --2- 172.21.15.47:0/2827563613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8faa020 0x55b5b8faa440 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.650+0000 7f5660f82740 1 -- 172.21.15.47:0/2827563613 >> 172.21.15.47:0/2827563613 conn(0x55b5b8f9cc00 msgr2=0x55b5b8f9d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.650+0000 7f5657400640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f563c0a2fa0 0x7f563c0a5460 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f56480b9b10 tx=0x7f56480f4d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.650+0000 7f5660f82740 1 -- 172.21.15.47:0/2827563613 shutdown_connections 2024-03-21T11:36:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.650+0000 7f5660f82740 1 -- 172.21.15.47:0/2827563613 wait complete. 2024-03-21T11:36:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.651+0000 7f5660f82740 1 -- start start 2024-03-21T11:36:17.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.651+0000 7f5660f82740 1 Processor -- start 2024-03-21T11:36:17.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.652+0000 7f5660f82740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8faa020 0x55b5b8fb42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.652+0000 7f5660f82740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fb4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.652+0000 7f5660f82740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8fbb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.652+0000 7f5660f82740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b8fb2bc0 con 0x55b5b8fab1f0 2024-03-21T11:36:17.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.652+0000 7f5660f82740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b8ec20d0 con 0x55b5b8faa020 2024-03-21T11:36:17.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.652+0000 7f5660f82740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b8fb2d40 con 0x55b5b8fabbb0 2024-03-21T11:36:17.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.652+0000 7f5660f82740 10 client.0 _ll_register_callbacks cb 0x55b5b8ec6e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:17.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.652+0000 7f5657e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8faa020 0x55b5b8fb42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.652+0000 7f564ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fb4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.652+0000 7f5657e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8faa020 0x55b5b8fb42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58948/0 (socket says 172.21.15.47:58948) 2024-03-21T11:36:17.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.652+0000 7f5657e00640 1 -- 172.21.15.47:0/2827391154 learned_addr learned my addr 172.21.15.47:0/2827391154 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.653+0000 7f5657e00640 1 --2- 172.21.15.47:0/2827391154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8fbb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.653+0000 7f564ea00640 1 -- 172.21.15.47:0/2827391154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 msgr2=0x55b5b8fbb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.653+0000 7f564ea00640 1 --2- 172.21.15.47:0/2827391154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8fbb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.653+0000 7f564ea00640 1 -- 172.21.15.47:0/2827391154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8faa020 msgr2=0x55b5b8fb42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.653+0000 7f564ea00640 1 --2- 172.21.15.47:0/2827391154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8faa020 0x55b5b8fb42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.653+0000 7f564ea00640 1 -- 172.21.15.47:0/2827391154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5b8c1a3f0 con 0x55b5b8fab1f0 2024-03-21T11:36:17.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.653+0000 7f5657e00640 1 --2- 172.21.15.47:0/2827391154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8fbb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:17.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.654+0000 7f564ea00640 1 --2- 172.21.15.47:0/2827391154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fb4800 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f56400126d0 tx=0x7f5640012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23f750b3f401ffa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.654+0000 7f5644c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:36:17.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.654+0000 7f5644c00640 1 -- 172.21.15.47:0/2827391154 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640004030 con 0x55b5b8fab1f0 2024-03-21T11:36:17.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.654+0000 7f5644c00640 1 -- 172.21.15.47:0/2827391154 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f56400041d0 con 0x55b5b8fab1f0 2024-03-21T11:36:17.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.654+0000 7f5660f82740 1 -- 172.21.15.47:0/2827391154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5650003e50 con 0x55b5b8fab1f0 2024-03-21T11:36:17.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.655+0000 7f5644c00640 1 -- 172.21.15.47:0/2827391154 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5640025050 con 0x55b5b8fab1f0 2024-03-21T11:36:17.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.655+0000 7f5660f82740 10 client.19434 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:17.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.655+0000 7f5660f82740 1 -- 172.21.15.47:0/2827391154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b5b8ce47d0 con 0x55b5b8fab1f0 2024-03-21T11:36:17.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.656+0000 7f5644c00640 1 -- 172.21.15.47:0/2827391154 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5650003e50 con 0x55b5b8fab1f0 2024-03-21T11:36:17.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.657+0000 7f5644c00640 1 -- 172.21.15.47:0/2827391154 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5640003a40 con 0x55b5b8fab1f0 2024-03-21T11:36:17.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.657+0000 7f5660f82740 10 client.19434 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:17.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.657+0000 7f5660f82740 10 client.19434 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:36:17.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.657+0000 7f5660f82740 20 client.19434 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:17.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.657+0000 7f5660f82740 1 --2- 172.21.15.47:0/2827391154 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55b5b8fbfd70 0x55b5b8fc21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.657+0000 7f5660f82740 4 client.19434 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:17.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.657+0000 7f5660f82740 1 -- 172.21.15.47:0/2827391154 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5b8f13800 con 0x55b5b8fbfd70 2024-03-21T11:36:17.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.657+0000 7f5657e00640 1 --2- 172.21.15.47:0/2827391154 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55b5b8fbfd70 0x55b5b8fc21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.658+0000 7f5657e00640 1 --2- 172.21.15.47:0/2827391154 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55b5b8fbfd70 0x55b5b8fc21b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=82520d5a288f3b3c server_cookie=4098488f9927b0d6 in_seq=0 out_seq=0 2024-03-21T11:36:17.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.658+0000 7f5644c00640 10 client.19434 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:36:17.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.659+0000 7f5644c00640 1 -- 172.21.15.47:0/2827391154 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b5b8f13800 con 0x55b5b8fbfd70 2024-03-21T11:36:17.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.659+0000 7f5644c00640 10 client.19434 handle_command_reply: tid=0 2024-03-21T11:36:17.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.660+0000 7f5660f82740 1 client.19434 shutdown 2024-03-21T11:36:17.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.660+0000 7f5660f82740 1 -- 172.21.15.47:0/2827391154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fab1f0 msgr2=0x55b5b8fb4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.660+0000 7f5660f82740 1 --2- 172.21.15.47:0/2827391154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fb4800 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f56400126d0 tx=0x7f5640012b90 comp rx=0 tx=0).stop 2024-03-21T11:36:17.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.660+0000 7f5660f82740 1 -- 172.21.15.47:0/2827391154 shutdown_connections 2024-03-21T11:36:17.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.660+0000 7f5660f82740 1 --2- 172.21.15.47:0/2827391154 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55b5b8fbfd70 0x55b5b8fc21b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.660+0000 7f5660f82740 1 --2- 172.21.15.47:0/2827391154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8fbb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.660+0000 7f5660f82740 1 --2- 172.21.15.47:0/2827391154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fb4800 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.660+0000 7f5660f82740 1 --2- 172.21.15.47:0/2827391154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8faa020 0x55b5b8fb42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.660+0000 7f5660f82740 1 -- 172.21.15.47:0/2827391154 >> 172.21.15.47:0/2827391154 conn(0x55b5b8f9cc00 msgr2=0x55b5b8f9beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.660+0000 7f5644c00640 0 client.19434 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:36:17.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.660+0000 7f5660f82740 1 -- 172.21.15.47:0/2827391154 shutdown_connections 2024-03-21T11:36:17.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.660+0000 7f5660f82740 1 -- 172.21.15.47:0/2827391154 wait complete. 2024-03-21T11:36:17.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.661+0000 7f5660f82740 20 client.19434 trim_cache size 0 max 16384 2024-03-21T11:36:17.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.677+0000 7f5660f82740 1 Processor -- start 2024-03-21T11:36:17.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.677+0000 7f5660f82740 1 -- start start 2024-03-21T11:36:17.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.678+0000 7f5660f82740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8faa020 0x55b5b8f9b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.678+0000 7f5660f82740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fbb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.678+0000 7f5660f82740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8f9bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.678+0000 7f5660f82740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56581a7fb0 con 0x55b5b8faa020 2024-03-21T11:36:17.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.678+0000 7f5660f82740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56581a9930 con 0x55b5b8fab1f0 2024-03-21T11:36:17.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.678+0000 7f5660f82740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56581a8610 con 0x55b5b8fabbb0 2024-03-21T11:36:17.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.679+0000 7f5657e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8faa020 0x55b5b8f9b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.679+0000 7f564ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fbb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.679+0000 7f564ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fbb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58950/0 (socket says 172.21.15.47:58950) 2024-03-21T11:36:17.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.679+0000 7f564ea00640 1 -- 172.21.15.47:0/2595613129 learned_addr learned my addr 172.21.15.47:0/2595613129 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.679+0000 7f5657e00640 1 --2- 172.21.15.47:0/2595613129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8f9bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.680+0000 7f564ea00640 1 -- 172.21.15.47:0/2595613129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 msgr2=0x55b5b8f9bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.680+0000 7f564ea00640 1 --2- 172.21.15.47:0/2595613129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8f9bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.680+0000 7f564ea00640 1 -- 172.21.15.47:0/2595613129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8faa020 msgr2=0x55b5b8f9b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.680+0000 7f564ea00640 1 --2- 172.21.15.47:0/2595613129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8faa020 0x55b5b8f9b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.680+0000 7f564ea00640 1 -- 172.21.15.47:0/2595613129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5b8fb2aa0 con 0x55b5b8fab1f0 2024-03-21T11:36:17.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.680+0000 7f5657e00640 1 --2- 172.21.15.47:0/2595613129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8f9bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:36:17.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.680+0000 7f564ea00640 1 --2- 172.21.15.47:0/2595613129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fbb800 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5640012420 tx=0x7f5640010630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85afa5d186a277bb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.681+0000 7f5647400640 1 -- 172.21.15.47:0/2595613129 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640022040 con 0x55b5b8fab1f0 2024-03-21T11:36:17.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.681+0000 7f5647400640 1 -- 172.21.15.47:0/2595613129 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5640010d40 con 0x55b5b8fab1f0 2024-03-21T11:36:17.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.681+0000 7f5647400640 1 -- 172.21.15.47:0/2595613129 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564000f9f0 con 0x55b5b8fab1f0 2024-03-21T11:36:17.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.682+0000 7f5660f82740 1 -- 172.21.15.47:0/2595613129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8fab1f0 msgr2=0x55b5b8fbb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.682+0000 7f5660f82740 1 --2- 172.21.15.47:0/2595613129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fbb800 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5640012420 tx=0x7f5640010630 comp rx=0 tx=0).stop 2024-03-21T11:36:17.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.682+0000 7f5660f82740 1 -- 172.21.15.47:0/2595613129 shutdown_connections 2024-03-21T11:36:17.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.682+0000 7f5660f82740 1 --2- 172.21.15.47:0/2595613129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8fabbb0 0x55b5b8f9bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.682+0000 7f5660f82740 1 --2- 172.21.15.47:0/2595613129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fbb800 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.682+0000 7f5660f82740 1 --2- 172.21.15.47:0/2595613129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8faa020 0x55b5b8f9b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.682+0000 7f5660f82740 1 -- 172.21.15.47:0/2595613129 >> 172.21.15.47:0/2595613129 conn(0x55b5b8f9cc00 msgr2=0x55b5b8f9cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.682+0000 7f5660f82740 1 -- 172.21.15.47:0/2595613129 shutdown_connections 2024-03-21T11:36:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.682+0000 7f5660f82740 1 -- 172.21.15.47:0/2595613129 wait complete. 2024-03-21T11:36:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.683+0000 7f5660f82740 1 -- start start 2024-03-21T11:36:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.683+0000 7f5660f82740 1 Processor -- start 2024-03-21T11:36:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.684+0000 7f5660f82740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8faa020 0x55b5b8fb7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.684+0000 7f5660f82740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fb79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.684+0000 7f5660f82740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fabbb0 0x55b5b8fba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.684+0000 7f5660f82740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56581a92d0 con 0x55b5b8fabbb0 2024-03-21T11:36:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.684+0000 7f5660f82740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56581a8c70 con 0x55b5b8fab1f0 2024-03-21T11:36:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.684+0000 7f5660f82740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56581a6190 con 0x55b5b8faa020 2024-03-21T11:36:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.684+0000 7f5660f82740 10 client.0 _ll_register_callbacks cb 0x7f5640009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.684+0000 7f564ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fb79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.684+0000 7f564ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fb79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58952/0 (socket says 172.21.15.47:58952) 2024-03-21T11:36:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.684+0000 7f564ea00640 1 -- 172.21.15.47:0/114308048 learned_addr learned my addr 172.21.15.47:0/114308048 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.685+0000 7f5657e00640 1 --2- 172.21.15.47:0/114308048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8faa020 0x55b5b8fb7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.685+0000 7f5657e00640 1 --2- 172.21.15.47:0/114308048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fabbb0 0x55b5b8fba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.685+0000 7f564ea00640 1 -- 172.21.15.47:0/114308048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8faa020 msgr2=0x55b5b8fb7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.685+0000 7f564ea00640 1 --2- 172.21.15.47:0/114308048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8faa020 0x55b5b8fb7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.685+0000 7f564ea00640 1 -- 172.21.15.47:0/114308048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fabbb0 msgr2=0x55b5b8fba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.685+0000 7f564ea00640 1 --2- 172.21.15.47:0/114308048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fabbb0 0x55b5b8fba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.685+0000 7f564ea00640 1 -- 172.21.15.47:0/114308048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5b8fbafc0 con 0x55b5b8fab1f0 2024-03-21T11:36:17.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.686+0000 7f564ea00640 1 --2- 172.21.15.47:0/114308048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fb79b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5640003e10 tx=0x7f5640003800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10b4856a3474216d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.686+0000 7f5644c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:17.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.686+0000 7f5644c00640 1 -- 172.21.15.47:0/114308048 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640010930 con 0x55b5b8fab1f0 2024-03-21T11:36:17.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.687+0000 7f5644c00640 1 -- 172.21.15.47:0/114308048 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5640010ad0 con 0x55b5b8fab1f0 2024-03-21T11:36:17.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.687+0000 7f5660f82740 1 -- 172.21.15.47:0/114308048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5640010d40 con 0x55b5b8fab1f0 2024-03-21T11:36:17.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.687+0000 7f5644c00640 1 -- 172.21.15.47:0/114308048 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5640005110 con 0x55b5b8fab1f0 2024-03-21T11:36:17.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.687+0000 7f5660f82740 10 client.29825 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:17.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.688+0000 7f5660f82740 1 -- 172.21.15.47:0/114308048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b5b901ba90 con 0x55b5b8fab1f0 2024-03-21T11:36:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.688+0000 7f5644c00640 1 -- 172.21.15.47:0/114308048 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5640010d40 con 0x55b5b8fab1f0 2024-03-21T11:36:17.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.689+0000 7f5644c00640 1 -- 172.21.15.47:0/114308048 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f56400052b0 con 0x55b5b8fab1f0 2024-03-21T11:36:17.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.690+0000 7f5660f82740 10 client.29825 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:17.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.690+0000 7f5660f82740 10 client.29825 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:36:17.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.690+0000 7f5660f82740 20 client.29825 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:17.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.690+0000 7f5660f82740 1 --2- 172.21.15.47:0/114308048 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55b5b901c280 0x55b5b901e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.690+0000 7f5660f82740 4 client.29825 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:36:17.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.690+0000 7f5660f82740 1 -- 172.21.15.47:0/114308048 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b5b8f0db50 con 0x55b5b901c280 2024-03-21T11:36:17.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.690+0000 7f5657e00640 1 --2- 172.21.15.47:0/114308048 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55b5b901c280 0x55b5b901e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.691+0000 7f5657e00640 1 --2- 172.21.15.47:0/114308048 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55b5b901c280 0x55b5b901e6a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e5fd5e31a2744cdb server_cookie=b2a8acaf75227dae in_seq=0 out_seq=0 2024-03-21T11:36:17.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.692+0000 7f5644c00640 10 client.29825 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:36:17.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.693+0000 7f5644c00640 1 -- 172.21.15.47:0/114308048 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b5b8f0db50 con 0x55b5b901c280 2024-03-21T11:36:17.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.693+0000 7f5644c00640 10 client.29825 handle_command_reply: tid=0 2024-03-21T11:36:17.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.693+0000 7f5660f82740 1 client.29825 shutdown 2024-03-21T11:36:17.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.693+0000 7f5660f82740 1 -- 172.21.15.47:0/114308048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8fab1f0 msgr2=0x55b5b8fb79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.693+0000 7f5660f82740 1 --2- 172.21.15.47:0/114308048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fb79b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5640003e10 tx=0x7f5640003800 comp rx=0 tx=0).stop 2024-03-21T11:36:17.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.694+0000 7f5660f82740 1 -- 172.21.15.47:0/114308048 shutdown_connections 2024-03-21T11:36:17.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.694+0000 7f5660f82740 1 --2- 172.21.15.47:0/114308048 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55b5b901c280 0x55b5b901e6a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.694+0000 7f5660f82740 1 --2- 172.21.15.47:0/114308048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5b8fabbb0 0x55b5b8fba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.694+0000 7f5660f82740 1 --2- 172.21.15.47:0/114308048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5b8fab1f0 0x55b5b8fb79b0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.694+0000 7f5644c00640 0 client.29825 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:36:17.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.694+0000 7f5660f82740 1 --2- 172.21.15.47:0/114308048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b5b8faa020 0x55b5b8fb7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.694+0000 7f5660f82740 1 -- 172.21.15.47:0/114308048 >> 172.21.15.47:0/114308048 conn(0x55b5b8f9cc00 msgr2=0x55b5b8f9fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.695+0000 7f5660f82740 1 -- 172.21.15.47:0/114308048 shutdown_connections 2024-03-21T11:36:17.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.695+0000 7f5660f82740 1 -- 172.21.15.47:0/114308048 wait complete. 2024-03-21T11:36:17.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.695+0000 7f5660f82740 20 client.29825 trim_cache size 0 max 16384 2024-03-21T11:36:17.695 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:36:17.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.697+0000 7f565e200640 1 -- 172.21.15.47:0/2503596999 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f563c0a2fa0 msgr2=0x7f563c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.697+0000 7f565e200640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f563c0a2fa0 0x7f563c0a5460 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f56480b9b10 tx=0x7f56480f4d00 comp rx=0 tx=0).stop 2024-03-21T11:36:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.697+0000 7f565e200640 1 -- 172.21.15.47:0/2503596999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56581016b0 msgr2=0x7f5658199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.697+0000 7f565e200640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56581016b0 0x7f5658199c30 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5648000c00 tx=0x7f564802c650 comp rx=0 tx=0).stop 2024-03-21T11:36:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.697+0000 7f565e200640 1 -- 172.21.15.47:0/2503596999 shutdown_connections 2024-03-21T11:36:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.697+0000 7f565e200640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f563c0a2fa0 0x7f563c0a5460 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.697+0000 7f565e200640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5658103260 0x7f56581a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.697+0000 7f565e200640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56581028a0 0x7f565819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.697+0000 7f565e200640 1 --2- 172.21.15.47:0/2503596999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56581016b0 0x7f5658199c30 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.697+0000 7f565e200640 1 -- 172.21.15.47:0/2503596999 >> 172.21.15.47:0/2503596999 conn(0x7f56580fcc20 msgr2=0x7f56580738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.698+0000 7f565e200640 1 -- 172.21.15.47:0/2503596999 shutdown_connections 2024-03-21T11:36:17.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.698+0000 7f565e200640 1 -- 172.21.15.47:0/2503596999 wait complete. 2024-03-21T11:36:17.724 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:36:17.725 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:36:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.942+0000 7f3e61200640 1 -- 172.21.15.47:0/1327602013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e5c1028a0 msgr2=0x7f3e5c102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.942+0000 7f3e61200640 1 --2- 172.21.15.47:0/1327602013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e5c1028a0 0x7f3e5c102d20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009900 tx=0x7f3e4402bc40 comp rx=0 tx=0).stop 2024-03-21T11:36:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.942+0000 7f3e61200640 1 -- 172.21.15.47:0/1327602013 shutdown_connections 2024-03-21T11:36:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.942+0000 7f3e61200640 1 --2- 172.21.15.47:0/1327602013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e5c103260 0x7f3e5c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.942+0000 7f3e61200640 1 --2- 172.21.15.47:0/1327602013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e5c1028a0 0x7f3e5c102d20 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.942+0000 7f3e61200640 1 --2- 172.21.15.47:0/1327602013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c1016b0 0x7f3e5c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.942+0000 7f3e61200640 1 -- 172.21.15.47:0/1327602013 >> 172.21.15.47:0/1327602013 conn(0x7f3e5c0fcc40 msgr2=0x7f3e5c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.942+0000 7f3e61200640 1 -- 172.21.15.47:0/1327602013 shutdown_connections 2024-03-21T11:36:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.942+0000 7f3e61200640 1 -- 172.21.15.47:0/1327602013 wait complete. 2024-03-21T11:36:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.943+0000 7f3e61200640 1 Processor -- start 2024-03-21T11:36:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.943+0000 7f3e61200640 1 -- start start 2024-03-21T11:36:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.943+0000 7f3e61200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e5c1016b0 0x7f3e5c199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.943+0000 7f3e61200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e5c1028a0 0x7f3e5c19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e61200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c103260 0x7f3e5c1a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e61200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c10d380 con 0x7f3e5c103260 2024-03-21T11:36:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e61200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c10d200 con 0x7f3e5c1028a0 2024-03-21T11:36:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e61200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c10d500 con 0x7f3e5c1016b0 2024-03-21T11:36:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e5c1028a0 0x7f3e5c19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e59e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e5c1016b0 0x7f3e5c199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e5c1028a0 0x7f3e5c19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58968/0 (socket says 172.21.15.47:58968) 2024-03-21T11:36:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e53e00640 1 -- 172.21.15.47:0/3152465511 learned_addr learned my addr 172.21.15.47:0/3152465511 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e5a800640 1 --2- 172.21.15.47:0/3152465511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c103260 0x7f3e5c1a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e59e00640 1 -- 172.21.15.47:0/3152465511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e5c1028a0 msgr2=0x7f3e5c19a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e59e00640 1 --2- 172.21.15.47:0/3152465511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e5c1028a0 0x7f3e5c19a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e59e00640 1 -- 172.21.15.47:0/3152465511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c103260 msgr2=0x7f3e5c1a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e59e00640 1 --2- 172.21.15.47:0/3152465511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c103260 0x7f3e5c1a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e59e00640 1 -- 172.21.15.47:0/3152465511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c1a1870 con 0x7f3e5c1016b0 2024-03-21T11:36:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.944+0000 7f3e53e00640 1 --2- 172.21.15.47:0/3152465511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e5c1028a0 0x7f3e5c19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:17.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.945+0000 7f3e59e00640 1 --2- 172.21.15.47:0/3152465511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e5c1016b0 0x7f3e5c199c60 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00e9c0 tx=0x7f3e4c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.945+0000 7f3e52a00640 1 -- 172.21.15.47:0/3152465511 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00ccc0 con 0x7f3e5c1016b0 2024-03-21T11:36:17.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.945+0000 7f3e52a00640 1 -- 172.21.15.47:0/3152465511 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e4c0035f0 con 0x7f3e5c1016b0 2024-03-21T11:36:17.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.945+0000 7f3e52a00640 1 -- 172.21.15.47:0/3152465511 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00ce60 con 0x7f3e5c1016b0 2024-03-21T11:36:17.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.945+0000 7f3e61200640 1 -- 172.21.15.47:0/3152465511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5c1a1b60 con 0x7f3e5c1016b0 2024-03-21T11:36:17.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.945+0000 7f3e61200640 1 -- 172.21.15.47:0/3152465511 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5c1a2060 con 0x7f3e5c1016b0 2024-03-21T11:36:17.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.948+0000 7f3e63f78740 1 Processor -- start 2024-03-21T11:36:17.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.949+0000 7f3e52a00640 1 -- 172.21.15.47:0/3152465511 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3e4c0037a0 con 0x7f3e5c1016b0 2024-03-21T11:36:17.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.949+0000 7f3e63f78740 1 -- start start 2024-03-21T11:36:17.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.949+0000 7f3e63f78740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d7810 0x55c3b40d7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.949+0000 7f3e63f78740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40d8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.949+0000 7f3e63f78740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 0x55c3b40dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.949+0000 7f3e63f78740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b403c2c0 con 0x55c3b40d93a0 2024-03-21T11:36:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.949+0000 7f3e63f78740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b3fe4c90 con 0x55c3b40d7810 2024-03-21T11:36:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.949+0000 7f3e63f78740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b4038360 con 0x55c3b40d89e0 2024-03-21T11:36:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.949+0000 7f3e59e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40d8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.949+0000 7f3e5a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d7810 0x55c3b40d7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.949+0000 7f3e5a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d7810 0x55c3b40d7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58976/0 (socket says 172.21.15.47:58976) 2024-03-21T11:36:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.949+0000 7f3e5a800640 1 -- 172.21.15.47:0/3631425945 learned_addr learned my addr 172.21.15.47:0/3631425945 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.949+0000 7f3e53e00640 1 --2- 172.21.15.47:0/3631425945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 0x55c3b40dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.950+0000 7f3e59e00640 1 -- 172.21.15.47:0/3631425945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d7810 msgr2=0x55c3b40d7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.950+0000 7f3e59e00640 1 --2- 172.21.15.47:0/3631425945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d7810 0x55c3b40d7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.950+0000 7f3e59e00640 1 -- 172.21.15.47:0/3631425945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 msgr2=0x55c3b40dfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.950+0000 7f3e59e00640 1 --2- 172.21.15.47:0/3631425945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 0x55c3b40dfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.950+0000 7f3e59e00640 1 -- 172.21.15.47:0/3631425945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3b4009080 con 0x55c3b40d89e0 2024-03-21T11:36:17.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.950+0000 7f3e59e00640 1 --2- 172.21.15.47:0/3631425945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40d8e60 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c0cd210 tx=0x7f3e4c0d3ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=100f0584542c41e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.950+0000 7f3e52a00640 1 --2- 172.21.15.47:0/3152465511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3e400a2fa0 0x7f3e400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.950+0000 7f3e4b400640 1 -- 172.21.15.47:0/3631425945 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c09b2c0 con 0x55c3b40d89e0 2024-03-21T11:36:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.951+0000 7f3e4b400640 1 -- 172.21.15.47:0/3631425945 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e4c0e1070 con 0x55c3b40d89e0 2024-03-21T11:36:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.951+0000 7f3e4b400640 1 -- 172.21.15.47:0/3631425945 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c09b5f0 con 0x55c3b40d89e0 2024-03-21T11:36:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.951+0000 7f3e52a00640 1 -- 172.21.15.47:0/3152465511 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3e4c0cfad0 con 0x7f3e5c1016b0 2024-03-21T11:36:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.951+0000 7f3e5a800640 1 --2- 172.21.15.47:0/3152465511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3e400a2fa0 0x7f3e400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.951+0000 7f3e63f78740 1 -- 172.21.15.47:0/3631425945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 msgr2=0x55c3b40d8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.951+0000 7f3e63f78740 1 --2- 172.21.15.47:0/3631425945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40d8e60 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c0cd210 tx=0x7f3e4c0d3ba0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.951+0000 7f3e5a800640 1 --2- 172.21.15.47:0/3152465511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3e400a2fa0 0x7f3e400a5460 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55c3b40d88e0 tx=0x7f3e54009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.952+0000 7f3e63f78740 1 -- 172.21.15.47:0/3631425945 shutdown_connections 2024-03-21T11:36:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.952+0000 7f3e63f78740 1 --2- 172.21.15.47:0/3631425945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 0x55c3b40dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.952+0000 7f3e63f78740 1 --2- 172.21.15.47:0/3631425945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40d8e60 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.952+0000 7f3e63f78740 1 --2- 172.21.15.47:0/3631425945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d7810 0x55c3b40d7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.952+0000 7f3e63f78740 1 -- 172.21.15.47:0/3631425945 >> 172.21.15.47:0/3631425945 conn(0x55c3b40d2c00 msgr2=0x55c3b40d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.952+0000 7f3e63f78740 1 -- 172.21.15.47:0/3631425945 shutdown_connections 2024-03-21T11:36:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.952+0000 7f3e63f78740 1 -- 172.21.15.47:0/3631425945 wait complete. 2024-03-21T11:36:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.952+0000 7f3e63f78740 1 -- start start 2024-03-21T11:36:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.952+0000 7f3e63f78740 1 Processor -- start 2024-03-21T11:36:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e63f78740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d7810 0x55c3b40e19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e63f78740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40e1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e63f78740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d93a0 0x55c3b40e8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e63f78740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b40e0350 con 0x55c3b40d7810 2024-03-21T11:36:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e63f78740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b3ff80d0 con 0x55c3b40d93a0 2024-03-21T11:36:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e63f78740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b40e04d0 con 0x55c3b40d89e0 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e63f78740 10 client.0 _ll_register_callbacks cb 0x55c3b3ffce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40e1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40e1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44742/0 (socket says 172.21.15.47:44742) 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e53e00640 1 -- 172.21.15.47:0/283176101 learned_addr learned my addr 172.21.15.47:0/283176101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e59e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d7810 0x55c3b40e19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e5a800640 1 --2- 172.21.15.47:0/283176101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d93a0 0x55c3b40e8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e59e00640 1 -- 172.21.15.47:0/283176101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 msgr2=0x55c3b40e1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e59e00640 1 --2- 172.21.15.47:0/283176101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40e1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e59e00640 1 -- 172.21.15.47:0/283176101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d93a0 msgr2=0x55c3b40e8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e59e00640 1 --2- 172.21.15.47:0/283176101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d93a0 0x55c3b40e8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e59e00640 1 -- 172.21.15.47:0/283176101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3b3d503f0 con 0x55c3b40d7810 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.953+0000 7f3e53e00640 1 --2- 172.21.15.47:0/283176101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40e1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.954+0000 7f3e59e00640 1 --2- 172.21.15.47:0/283176101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d7810 0x55c3b40e19a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c000c00 tx=0x7f3e4c004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b26472aefb5e58 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.954+0000 7f3e3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:36:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.954+0000 7f3e3fe00640 1 -- 172.21.15.47:0/283176101 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c09b7d0 con 0x55c3b40d7810 2024-03-21T11:36:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.954+0000 7f3e3fe00640 1 -- 172.21.15.47:0/283176101 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3e4c003b10 con 0x55c3b40d7810 2024-03-21T11:36:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.954+0000 7f3e63f78740 1 -- 172.21.15.47:0/283176101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e4c0e1070 con 0x55c3b40d7810 2024-03-21T11:36:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.954+0000 7f3e3fe00640 1 -- 172.21.15.47:0/283176101 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e4c0d3020 con 0x55c3b40d7810 2024-03-21T11:36:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.954+0000 7f3e63f78740 10 client.19449 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.954+0000 7f3e63f78740 1 -- 172.21.15.47:0/283176101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c3b3e1a7d0 con 0x55c3b40d7810 2024-03-21T11:36:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.955+0000 7f3e3fe00640 1 -- 172.21.15.47:0/283176101 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e4c0e1070 con 0x55c3b40d7810 2024-03-21T11:36:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.955+0000 7f3e3fe00640 1 -- 172.21.15.47:0/283176101 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3e4c0babe0 con 0x55c3b40d7810 2024-03-21T11:36:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.955+0000 7f3e63f78740 10 client.19449 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.955+0000 7f3e63f78740 10 client.19449 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:36:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.955+0000 7f3e63f78740 20 client.19449 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.955+0000 7f3e63f78740 1 --2- 172.21.15.47:0/283176101 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c3b40ed080 0x55c3b40ef470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.955+0000 7f3e63f78740 4 client.19449 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.955+0000 7f3e63f78740 1 -- 172.21.15.47:0/283176101 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3b4049800 con 0x55c3b40ed080 2024-03-21T11:36:17.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.955+0000 7f3e53e00640 1 --2- 172.21.15.47:0/283176101 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c3b40ed080 0x55c3b40ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.956+0000 7f3e53e00640 1 --2- 172.21.15.47:0/283176101 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c3b40ed080 0x55c3b40ef470 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23919eba4c51b78c server_cookie=4303dcc059155ad6 in_seq=0 out_seq=0 2024-03-21T11:36:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.956+0000 7f3e3fe00640 10 client.19449 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:36:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e3fe00640 1 -- 172.21.15.47:0/283176101 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c3b4049800 con 0x55c3b40ed080 2024-03-21T11:36:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e3fe00640 10 client.19449 handle_command_reply: tid=0 2024-03-21T11:36:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e63f78740 1 client.19449 shutdown 2024-03-21T11:36:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e63f78740 1 -- 172.21.15.47:0/283176101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d7810 msgr2=0x55c3b40e19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e63f78740 1 --2- 172.21.15.47:0/283176101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d7810 0x55c3b40e19a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c000c00 tx=0x7f3e4c004720 comp rx=0 tx=0).stop 2024-03-21T11:36:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e63f78740 1 -- 172.21.15.47:0/283176101 shutdown_connections 2024-03-21T11:36:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e63f78740 1 --2- 172.21.15.47:0/283176101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d93a0 0x55c3b40e8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e63f78740 1 --2- 172.21.15.47:0/283176101 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c3b40ed080 0x55c3b40ef470 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e63f78740 1 --2- 172.21.15.47:0/283176101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40e1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e63f78740 1 --2- 172.21.15.47:0/283176101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d7810 0x55c3b40e19a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e3fe00640 0 client.19449 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:36:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e63f78740 1 -- 172.21.15.47:0/283176101 >> 172.21.15.47:0/283176101 conn(0x55c3b40d2c00 msgr2=0x55c3b40d1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e63f78740 1 -- 172.21.15.47:0/283176101 shutdown_connections 2024-03-21T11:36:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e63f78740 1 -- 172.21.15.47:0/283176101 wait complete. 2024-03-21T11:36:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.957+0000 7f3e63f78740 20 client.19449 trim_cache size 0 max 16384 2024-03-21T11:36:17.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.972+0000 7f3e63f78740 1 Processor -- start 2024-03-21T11:36:17.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.972+0000 7f3e63f78740 1 -- start start 2024-03-21T11:36:17.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e63f78740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d7810 0x55c3b40db9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e63f78740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d89e0 0x55c3b40e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e63f78740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 0x55c3b40dbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e63f78740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c1ac360 con 0x55c3b40d93a0 2024-03-21T11:36:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e63f78740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c1adce0 con 0x55c3b40d89e0 2024-03-21T11:36:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e63f78740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c1ac9c0 con 0x55c3b40d7810 2024-03-21T11:36:17.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e59e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d7810 0x55c3b40db9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e59e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d7810 0x55c3b40db9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44748/0 (socket says 172.21.15.47:44748) 2024-03-21T11:36:17.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e59e00640 1 -- 172.21.15.47:0/3939163598 learned_addr learned my addr 172.21.15.47:0/3939163598 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d89e0 0x55c3b40e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e5a800640 1 --2- 172.21.15.47:0/3939163598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 0x55c3b40dbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e59e00640 1 -- 172.21.15.47:0/3939163598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d89e0 msgr2=0x55c3b40e8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e59e00640 1 --2- 172.21.15.47:0/3939163598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d89e0 0x55c3b40e8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e59e00640 1 -- 172.21.15.47:0/3939163598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 msgr2=0x55c3b40dbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e59e00640 1 --2- 172.21.15.47:0/3939163598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 0x55c3b40dbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.973+0000 7f3e59e00640 1 -- 172.21.15.47:0/3939163598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3b40e0600 con 0x55c3b40d7810 2024-03-21T11:36:17.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.974+0000 7f3e59e00640 1 --2- 172.21.15.47:0/3939163598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d7810 0x55c3b40db9e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c0cd5b0 tx=0x7f3e4c0d35b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77cec5a470ea1230 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.974+0000 7f3e4aa00640 1 -- 172.21.15.47:0/3939163598 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c09b9b0 con 0x55c3b40d7810 2024-03-21T11:36:17.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.974+0000 7f3e4aa00640 1 -- 172.21.15.47:0/3939163598 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e4c0e1070 con 0x55c3b40d7810 2024-03-21T11:36:17.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.974+0000 7f3e4aa00640 1 -- 172.21.15.47:0/3939163598 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c09bc40 con 0x55c3b40d7810 2024-03-21T11:36:17.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.975+0000 7f3e63f78740 1 -- 172.21.15.47:0/3939163598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d7810 msgr2=0x55c3b40db9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.975+0000 7f3e63f78740 1 --2- 172.21.15.47:0/3939163598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d7810 0x55c3b40db9e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c0cd5b0 tx=0x7f3e4c0d35b0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.975+0000 7f3e63f78740 1 -- 172.21.15.47:0/3939163598 shutdown_connections 2024-03-21T11:36:17.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.975+0000 7f3e63f78740 1 --2- 172.21.15.47:0/3939163598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 0x55c3b40dbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.975+0000 7f3e63f78740 1 --2- 172.21.15.47:0/3939163598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d89e0 0x55c3b40e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.975+0000 7f3e63f78740 1 --2- 172.21.15.47:0/3939163598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d7810 0x55c3b40db9e0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.975+0000 7f3e63f78740 1 -- 172.21.15.47:0/3939163598 >> 172.21.15.47:0/3939163598 conn(0x55c3b40d2c00 msgr2=0x55c3b40d66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.975+0000 7f3e63f78740 1 -- 172.21.15.47:0/3939163598 shutdown_connections 2024-03-21T11:36:17.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.975+0000 7f3e63f78740 1 -- 172.21.15.47:0/3939163598 wait complete. 2024-03-21T11:36:17.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.975+0000 7f3e63f78740 1 -- start start 2024-03-21T11:36:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.976+0000 7f3e63f78740 1 Processor -- start 2024-03-21T11:36:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.976+0000 7f3e63f78740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d7810 0x55c3b40e50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.976+0000 7f3e63f78740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40e55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.976+0000 7f3e63f78740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 0x55c3b40e8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.976+0000 7f3e63f78740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c1ad680 con 0x55c3b40d93a0 2024-03-21T11:36:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.976+0000 7f3e63f78740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c1ad020 con 0x55c3b40d7810 2024-03-21T11:36:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.976+0000 7f3e63f78740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c1aa520 con 0x55c3b40d89e0 2024-03-21T11:36:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.976+0000 7f3e63f78740 10 client.0 _ll_register_callbacks cb 0x7f3e540093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.976+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40e55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.976+0000 7f3e59e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d7810 0x55c3b40e50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.976+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40e55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44758/0 (socket says 172.21.15.47:44758) 2024-03-21T11:36:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.976+0000 7f3e53e00640 1 -- 172.21.15.47:0/3341594498 learned_addr learned my addr 172.21.15.47:0/3341594498 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:17.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.976+0000 7f3e5a800640 1 --2- 172.21.15.47:0/3341594498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 0x55c3b40e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.977+0000 7f3e53e00640 1 -- 172.21.15.47:0/3341594498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d7810 msgr2=0x55c3b40e50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.977+0000 7f3e53e00640 1 --2- 172.21.15.47:0/3341594498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d7810 0x55c3b40e50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.977+0000 7f3e53e00640 1 -- 172.21.15.47:0/3341594498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 msgr2=0x55c3b40e8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.977+0000 7f3e53e00640 1 --2- 172.21.15.47:0/3341594498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 0x55c3b40e8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.977+0000 7f3e53e00640 1 -- 172.21.15.47:0/3341594498 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3b40e8c00 con 0x55c3b40d89e0 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.977+0000 7f3e5a800640 1 --2- 172.21.15.47:0/3341594498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 0x55c3b40e8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.977+0000 7f3e59e00640 1 --2- 172.21.15.47:0/3341594498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d7810 0x55c3b40e50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.977+0000 7f3e53e00640 1 --2- 172.21.15.47:0/3341594498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40e55f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3e44005c40 tx=0x7f3e4402c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60eb4a296af81682 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.977+0000 7f3e3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.977+0000 7f3e3fe00640 1 -- 172.21.15.47:0/3341594498 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e44004ad0 con 0x55c3b40d89e0 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.977+0000 7f3e63f78740 1 -- 172.21.15.47:0/3341594498 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e4c0e1070 con 0x55c3b40d89e0 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.977+0000 7f3e3fe00640 1 -- 172.21.15.47:0/3341594498 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3e44004c70 con 0x55c3b40d89e0 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.978+0000 7f3e3fe00640 1 -- 172.21.15.47:0/3341594498 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e4405a050 con 0x55c3b40d89e0 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.978+0000 7f3e63f78740 10 client.29875 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.978+0000 7f3e63f78740 1 -- 172.21.15.47:0/3341594498 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c3b4149280 con 0x55c3b40d89e0 2024-03-21T11:36:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.978+0000 7f3e3fe00640 1 -- 172.21.15.47:0/3341594498 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e4c0e1070 con 0x55c3b40d89e0 2024-03-21T11:36:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.978+0000 7f3e3fe00640 1 -- 172.21.15.47:0/3341594498 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3e44047050 con 0x55c3b40d89e0 2024-03-21T11:36:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.978+0000 7f3e63f78740 10 client.29875 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.978+0000 7f3e63f78740 10 client.29875 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:36:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.978+0000 7f3e63f78740 20 client.29875 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.978+0000 7f3e63f78740 1 --2- 172.21.15.47:0/3341594498 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c3b4149be0 0x55c3b414c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.978+0000 7f3e63f78740 4 client.29875 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:36:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.978+0000 7f3e63f78740 1 -- 172.21.15.47:0/3341594498 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c3b4043b50 con 0x55c3b4149be0 2024-03-21T11:36:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.978+0000 7f3e59e00640 1 --2- 172.21.15.47:0/3341594498 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c3b4149be0 0x55c3b414c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.979+0000 7f3e59e00640 1 --2- 172.21.15.47:0/3341594498 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c3b4149be0 0x55c3b414c000 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e004629effd7f38 server_cookie=e7e988010a1d934f in_seq=0 out_seq=0 2024-03-21T11:36:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.979+0000 7f3e3fe00640 10 client.29875 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:36:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.979+0000 7f3e3fe00640 1 -- 172.21.15.47:0/3341594498 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c3b4043b50 con 0x55c3b4149be0 2024-03-21T11:36:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.979+0000 7f3e3fe00640 10 client.29875 handle_command_reply: tid=0 2024-03-21T11:36:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.979+0000 7f3e63f78740 1 client.29875 shutdown 2024-03-21T11:36:17.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.979+0000 7f3e63f78740 1 -- 172.21.15.47:0/3341594498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 msgr2=0x55c3b40e55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.979+0000 7f3e63f78740 1 --2- 172.21.15.47:0/3341594498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40e55f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3e44005c40 tx=0x7f3e4402c890 comp rx=0 tx=0).stop 2024-03-21T11:36:17.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.979+0000 7f3e63f78740 1 -- 172.21.15.47:0/3341594498 shutdown_connections 2024-03-21T11:36:17.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.979+0000 7f3e63f78740 1 --2- 172.21.15.47:0/3341594498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3b40d93a0 0x55c3b40e8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.979+0000 7f3e63f78740 1 --2- 172.21.15.47:0/3341594498 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c3b4149be0 0x55c3b414c000 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.979+0000 7f3e63f78740 1 --2- 172.21.15.47:0/3341594498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3b40d89e0 0x55c3b40e55f0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.979+0000 7f3e63f78740 1 --2- 172.21.15.47:0/3341594498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3b40d7810 0x55c3b40e50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.979+0000 7f3e3fe00640 0 client.29875 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:36:17.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.979+0000 7f3e63f78740 1 -- 172.21.15.47:0/3341594498 >> 172.21.15.47:0/3341594498 conn(0x55c3b40d2c00 msgr2=0x55c3b40d1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.980+0000 7f3e63f78740 1 -- 172.21.15.47:0/3341594498 shutdown_connections 2024-03-21T11:36:17.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.980+0000 7f3e63f78740 1 -- 172.21.15.47:0/3341594498 wait complete. 2024-03-21T11:36:17.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.980+0000 7f3e63f78740 20 client.29875 trim_cache size 0 max 16384 2024-03-21T11:36:17.984 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:36:17.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.980+0000 7f3e61200640 1 -- 172.21.15.47:0/3152465511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3e400a2fa0 msgr2=0x7f3e400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.980+0000 7f3e61200640 1 --2- 172.21.15.47:0/3152465511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3e400a2fa0 0x7f3e400a5460 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55c3b40d88e0 tx=0x7f3e54009500 comp rx=0 tx=0).stop 2024-03-21T11:36:17.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.980+0000 7f3e61200640 1 -- 172.21.15.47:0/3152465511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e5c1016b0 msgr2=0x7f3e5c199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:17.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.980+0000 7f3e61200640 1 --2- 172.21.15.47:0/3152465511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e5c1016b0 0x7f3e5c199c60 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00e9c0 tx=0x7f3e4c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:36:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.981+0000 7f3e61200640 1 -- 172.21.15.47:0/3152465511 shutdown_connections 2024-03-21T11:36:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.981+0000 7f3e61200640 1 --2- 172.21.15.47:0/3152465511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3e400a2fa0 0x7f3e400a5460 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.981+0000 7f3e61200640 1 --2- 172.21.15.47:0/3152465511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e5c103260 0x7f3e5c1a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.981+0000 7f3e61200640 1 --2- 172.21.15.47:0/3152465511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e5c1028a0 0x7f3e5c19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.981+0000 7f3e61200640 1 --2- 172.21.15.47:0/3152465511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e5c1016b0 0x7f3e5c199c60 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.981+0000 7f3e61200640 1 -- 172.21.15.47:0/3152465511 >> 172.21.15.47:0/3152465511 conn(0x7f3e5c0fcc40 msgr2=0x7f3e5c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.981+0000 7f3e61200640 1 -- 172.21.15.47:0/3152465511 shutdown_connections 2024-03-21T11:36:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:17.981+0000 7f3e61200640 1 -- 172.21.15.47:0/3152465511 wait complete. 2024-03-21T11:36:17.999 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:36:17.999 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:36:18.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.221+0000 7fe62e600640 1 -- 172.21.15.47:0/2348348084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe628104ab0 msgr2=0x7fe628104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.221+0000 7fe62e600640 1 --2- 172.21.15.47:0/2348348084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe628104ab0 0x7fe628104f30 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe614009900 tx=0x7fe61402bc30 comp rx=0 tx=0).stop 2024-03-21T11:36:18.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.222+0000 7fe62e600640 1 -- 172.21.15.47:0/2348348084 shutdown_connections 2024-03-21T11:36:18.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.222+0000 7fe62e600640 1 --2- 172.21.15.47:0/2348348084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe628105470 0x7fe62810bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.222+0000 7fe62e600640 1 --2- 172.21.15.47:0/2348348084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe628104ab0 0x7fe628104f30 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.222+0000 7fe62e600640 1 --2- 172.21.15.47:0/2348348084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6281038c0 0x7fe628103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.222+0000 7fe62e600640 1 -- 172.21.15.47:0/2348348084 >> 172.21.15.47:0/2348348084 conn(0x7fe6280fee50 msgr2=0x7fe628101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:18.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.222+0000 7fe62e600640 1 -- 172.21.15.47:0/2348348084 shutdown_connections 2024-03-21T11:36:18.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.222+0000 7fe62e600640 1 -- 172.21.15.47:0/2348348084 wait complete. 2024-03-21T11:36:18.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.223+0000 7fe62e600640 1 Processor -- start 2024-03-21T11:36:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.223+0000 7fe62e600640 1 -- start start 2024-03-21T11:36:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.223+0000 7fe62e600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6281038c0 0x7fe628199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.223+0000 7fe62e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe628104ab0 0x7fe62819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.223+0000 7fe62e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe628105470 0x7fe6281a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.223+0000 7fe62e600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62810f590 con 0x7fe6281038c0 2024-03-21T11:36:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.223+0000 7fe62e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62810f410 con 0x7fe628104ab0 2024-03-21T11:36:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.223+0000 7fe62e600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62810f710 con 0x7fe628105470 2024-03-21T11:36:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.224+0000 7fe626a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe628104ab0 0x7fe62819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.224+0000 7fe627400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6281038c0 0x7fe628199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.224+0000 7fe626a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe628104ab0 0x7fe62819a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59026/0 (socket says 172.21.15.47:59026) 2024-03-21T11:36:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.224+0000 7fe626a00640 1 -- 172.21.15.47:0/2845953597 learned_addr learned my addr 172.21.15.47:0/2845953597 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:18.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.224+0000 7fe627e00640 1 --2- 172.21.15.47:0/2845953597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe628105470 0x7fe6281a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.224+0000 7fe627e00640 1 -- 172.21.15.47:0/2845953597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe628104ab0 msgr2=0x7fe62819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.224+0000 7fe627e00640 1 --2- 172.21.15.47:0/2845953597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe628104ab0 0x7fe62819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.224+0000 7fe627e00640 1 -- 172.21.15.47:0/2845953597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6281038c0 msgr2=0x7fe628199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.224+0000 7fe627e00640 1 --2- 172.21.15.47:0/2845953597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6281038c0 0x7fe628199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.224+0000 7fe627e00640 1 -- 172.21.15.47:0/2845953597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6281a1830 con 0x7fe628105470 2024-03-21T11:36:18.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.224+0000 7fe626a00640 1 --2- 172.21.15.47:0/2845953597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe628104ab0 0x7fe62819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:18.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.225+0000 7fe627e00640 1 --2- 172.21.15.47:0/2845953597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe628105470 0x7fe6281a1190 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe620009fd0 tx=0x7fe62000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:18.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.225+0000 7fe61fe00640 1 -- 172.21.15.47:0/2845953597 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62000ce20 con 0x7fe628105470 2024-03-21T11:36:18.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.225+0000 7fe61fe00640 1 -- 172.21.15.47:0/2845953597 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe620005020 con 0x7fe628105470 2024-03-21T11:36:18.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.225+0000 7fe61fe00640 1 -- 172.21.15.47:0/2845953597 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe620010720 con 0x7fe628105470 2024-03-21T11:36:18.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.225+0000 7fe62e600640 1 -- 172.21.15.47:0/2845953597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6281a1b20 con 0x7fe628105470 2024-03-21T11:36:18.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.225+0000 7fe62e600640 1 -- 172.21.15.47:0/2845953597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6281a2020 con 0x7fe628105470 2024-03-21T11:36:18.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.229+0000 7fe61fe00640 1 -- 172.21.15.47:0/2845953597 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe620005540 con 0x7fe628105470 2024-03-21T11:36:18.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.229+0000 7fe63137a740 1 Processor -- start 2024-03-21T11:36:18.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.229+0000 7fe63137a740 1 -- start start 2024-03-21T11:36:18.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.230+0000 7fe63137a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fa810 0x5603786fac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.230+0000 7fe63137a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fb9e0 0x5603786fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.230+0000 7fe63137a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fc3a0 0x560378702c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.230+0000 7fe63137a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037865f2c0 con 0x5603786fa810 2024-03-21T11:36:18.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.230+0000 7fe63137a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560378607c90 con 0x5603786fb9e0 2024-03-21T11:36:18.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.230+0000 7fe63137a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037865b360 con 0x5603786fc3a0 2024-03-21T11:36:18.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.230+0000 7fe627400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fb9e0 0x5603786fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.230+0000 7fe627400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fb9e0 0x5603786fbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59040/0 (socket says 172.21.15.47:59040) 2024-03-21T11:36:18.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.230+0000 7fe627400640 1 -- 172.21.15.47:0/3668660887 learned_addr learned my addr 172.21.15.47:0/3668660887 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:18.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.230+0000 7fe626a00640 1 --2- 172.21.15.47:0/3668660887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fa810 0x5603786fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.231+0000 7fe626a00640 1 --2- 172.21.15.47:0/3668660887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fc3a0 0x560378702c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.231+0000 7fe627400640 1 -- 172.21.15.47:0/3668660887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fc3a0 msgr2=0x560378702c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.231+0000 7fe627400640 1 --2- 172.21.15.47:0/3668660887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fc3a0 0x560378702c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.231+0000 7fe627400640 1 -- 172.21.15.47:0/3668660887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fa810 msgr2=0x5603786fac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.231+0000 7fe627400640 1 --2- 172.21.15.47:0/3668660887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fa810 0x5603786fac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.231+0000 7fe627400640 1 -- 172.21.15.47:0/3668660887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56037862c080 con 0x5603786fb9e0 2024-03-21T11:36:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.231+0000 7fe61fe00640 1 --2- 172.21.15.47:0/2845953597 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe60c0a3020 0x7fe60c0a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.231+0000 7fe627400640 1 --2- 172.21.15.47:0/3668660887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fb9e0 0x5603786fbe60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe618009c60 tx=0x7fe618011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9737e64350ff80ab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.232+0000 7fe61fe00640 1 -- 172.21.15.47:0/2845953597 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe6200cfec0 con 0x7fe628105470 2024-03-21T11:36:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.232+0000 7fe626a00640 1 --2- 172.21.15.47:0/2845953597 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe60c0a3020 0x7fe60c0a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.233+0000 7fe626a00640 1 --2- 172.21.15.47:0/2845953597 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe60c0a3020 0x7fe60c0a54e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe61402c8e0 tx=0x7fe614005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.233+0000 7fe61cc00640 1 -- 172.21.15.47:0/3668660887 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe618004220 con 0x5603786fb9e0 2024-03-21T11:36:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.233+0000 7fe61cc00640 1 -- 172.21.15.47:0/3668660887 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6180043c0 con 0x5603786fb9e0 2024-03-21T11:36:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.233+0000 7fe61cc00640 1 -- 172.21.15.47:0/3668660887 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6180198d0 con 0x5603786fb9e0 2024-03-21T11:36:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.234+0000 7fe63137a740 1 -- 172.21.15.47:0/3668660887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fb9e0 msgr2=0x5603786fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.234+0000 7fe63137a740 1 --2- 172.21.15.47:0/3668660887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fb9e0 0x5603786fbe60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe618009c60 tx=0x7fe618011810 comp rx=0 tx=0).stop 2024-03-21T11:36:18.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.237+0000 7fe63137a740 1 -- 172.21.15.47:0/3668660887 shutdown_connections 2024-03-21T11:36:18.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.237+0000 7fe63137a740 1 --2- 172.21.15.47:0/3668660887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fc3a0 0x560378702c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.237+0000 7fe63137a740 1 --2- 172.21.15.47:0/3668660887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fb9e0 0x5603786fbe60 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.237+0000 7fe63137a740 1 --2- 172.21.15.47:0/3668660887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fa810 0x5603786fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.237+0000 7fe63137a740 1 -- 172.21.15.47:0/3668660887 >> 172.21.15.47:0/3668660887 conn(0x5603786f5c00 msgr2=0x5603786f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:18.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.238+0000 7fe63137a740 1 -- 172.21.15.47:0/3668660887 shutdown_connections 2024-03-21T11:36:18.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.239+0000 7fe63137a740 1 -- 172.21.15.47:0/3668660887 wait complete. 2024-03-21T11:36:18.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.240+0000 7fe63137a740 1 -- start start 2024-03-21T11:36:18.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.242+0000 7fe63137a740 1 Processor -- start 2024-03-21T11:36:18.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.243+0000 7fe63137a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fa810 0x560378704a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.243+0000 7fe63137a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fb9e0 0x560378704f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.243+0000 7fe63137a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fc3a0 0x56037870bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.243+0000 7fe63137a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603787033b0 con 0x5603786fc3a0 2024-03-21T11:36:18.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.243+0000 7fe63137a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037861b0d0 con 0x5603786fa810 2024-03-21T11:36:18.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.243+0000 7fe63137a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560378703530 con 0x5603786fb9e0 2024-03-21T11:36:18.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.244+0000 7fe63137a740 10 client.0 _ll_register_callbacks cb 0x56037861fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:18.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.245+0000 7fe627400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fa810 0x560378704a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.246+0000 7fe627400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fa810 0x560378704a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59042/0 (socket says 172.21.15.47:59042) 2024-03-21T11:36:18.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.246+0000 7fe627400640 1 -- 172.21.15.47:0/1951265301 learned_addr learned my addr 172.21.15.47:0/1951265301 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.246+0000 7fe627400640 1 -- 172.21.15.47:0/1951265301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fb9e0 msgr2=0x560378704f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:36:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.246+0000 7fe627400640 1 --2- 172.21.15.47:0/1951265301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fb9e0 0x560378704f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.246+0000 7fe627400640 1 -- 172.21.15.47:0/1951265301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fc3a0 msgr2=0x56037870bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.247+0000 7fe627e00640 1 --2- 172.21.15.47:0/1951265301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fc3a0 0x56037870bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.248+0000 7fe627400640 1 --2- 172.21.15.47:0/1951265301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fc3a0 0x56037870bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.248+0000 7fe627400640 1 -- 172.21.15.47:0/1951265301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603783733f0 con 0x5603786fa810 2024-03-21T11:36:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.249+0000 7fe627400640 1 --2- 172.21.15.47:0/1951265301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fa810 0x560378704a00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe618009350 tx=0x7fe618003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88ab0ed70a0f791 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.250+0000 7fe611600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.250+0000 7fe627e00640 1 --2- 172.21.15.47:0/1951265301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fc3a0 0x56037870bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:36:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.253+0000 7fe611600640 1 -- 172.21.15.47:0/1951265301 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe618028070 con 0x5603786fa810 2024-03-21T11:36:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.253+0000 7fe611600640 1 -- 172.21.15.47:0/1951265301 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe618005040 con 0x5603786fa810 2024-03-21T11:36:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.253+0000 7fe63137a740 1 -- 172.21.15.47:0/1951265301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6180043c0 con 0x5603786fa810 2024-03-21T11:36:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.254+0000 7fe611600640 1 -- 172.21.15.47:0/1951265301 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe618034050 con 0x5603786fa810 2024-03-21T11:36:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.254+0000 7fe63137a740 10 client.29855 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.254+0000 7fe63137a740 1 -- 172.21.15.47:0/1951265301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56037843d7d0 con 0x5603786fa810 2024-03-21T11:36:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.256+0000 7fe611600640 1 -- 172.21.15.47:0/1951265301 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6180043c0 con 0x5603786fa810 2024-03-21T11:36:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.258+0000 7fe611600640 1 -- 172.21.15.47:0/1951265301 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe618019b50 con 0x5603786fa810 2024-03-21T11:36:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.258+0000 7fe63137a740 10 client.29855 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.258+0000 7fe63137a740 10 client.29855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:36:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.258+0000 7fe63137a740 20 client.29855 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.258+0000 7fe63137a740 1 --2- 172.21.15.47:0/1951265301 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x560378710230 0x560378712680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.258+0000 7fe63137a740 4 client.29855 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.258+0000 7fe63137a740 1 -- 172.21.15.47:0/1951265301 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56037866c800 con 0x560378710230 2024-03-21T11:36:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.258+0000 7fe626a00640 1 --2- 172.21.15.47:0/1951265301 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x560378710230 0x560378712680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.259+0000 7fe626a00640 1 --2- 172.21.15.47:0/1951265301 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x560378710230 0x560378712680 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=857d1cdd9bf204ff server_cookie=8080b30080cf4f00 in_seq=0 out_seq=0 2024-03-21T11:36:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.259+0000 7fe611600640 10 client.29855 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:36:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.261+0000 7fe611600640 1 -- 172.21.15.47:0/1951265301 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56037866c800 con 0x560378710230 2024-03-21T11:36:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.261+0000 7fe611600640 10 client.29855 handle_command_reply: tid=0 2024-03-21T11:36:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.261+0000 7fe63137a740 1 client.29855 shutdown 2024-03-21T11:36:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.262+0000 7fe63137a740 1 -- 172.21.15.47:0/1951265301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fa810 msgr2=0x560378704a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.262+0000 7fe63137a740 1 --2- 172.21.15.47:0/1951265301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fa810 0x560378704a00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe618009350 tx=0x7fe618003820 comp rx=0 tx=0).stop 2024-03-21T11:36:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.262+0000 7fe63137a740 1 -- 172.21.15.47:0/1951265301 shutdown_connections 2024-03-21T11:36:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.262+0000 7fe63137a740 1 --2- 172.21.15.47:0/1951265301 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x560378710230 0x560378712680 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.262+0000 7fe63137a740 1 --2- 172.21.15.47:0/1951265301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fc3a0 0x56037870bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.262+0000 7fe63137a740 1 --2- 172.21.15.47:0/1951265301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fb9e0 0x560378704f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.262+0000 7fe63137a740 1 --2- 172.21.15.47:0/1951265301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fa810 0x560378704a00 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.262+0000 7fe611600640 0 client.29855 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:36:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.262+0000 7fe63137a740 1 -- 172.21.15.47:0/1951265301 >> 172.21.15.47:0/1951265301 conn(0x5603786f5c00 msgr2=0x5603786f4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.262+0000 7fe63137a740 1 -- 172.21.15.47:0/1951265301 shutdown_connections 2024-03-21T11:36:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.262+0000 7fe63137a740 1 -- 172.21.15.47:0/1951265301 wait complete. 2024-03-21T11:36:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.263+0000 7fe63137a740 20 client.29855 trim_cache size 0 max 16384 2024-03-21T11:36:18.294 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:18 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3823484210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:36:18.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.297+0000 7fe63137a740 1 Processor -- start 2024-03-21T11:36:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.298+0000 7fe63137a740 1 -- start start 2024-03-21T11:36:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.299+0000 7fe63137a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fa810 0x5603786fe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.299+0000 7fe63137a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fb9e0 0x56037870bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.299+0000 7fe63137a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fc3a0 0x5603786fef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.299+0000 7fe63137a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281ac370 con 0x5603786fb9e0 2024-03-21T11:36:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.299+0000 7fe63137a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281adcf0 con 0x5603786fa810 2024-03-21T11:36:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.299+0000 7fe63137a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281ac9d0 con 0x5603786fc3a0 2024-03-21T11:36:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.299+0000 7fe627e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fc3a0 0x5603786fef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.299+0000 7fe627e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fc3a0 0x5603786fef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44806/0 (socket says 172.21.15.47:44806) 2024-03-21T11:36:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.299+0000 7fe627e00640 1 -- 172.21.15.47:0/1871787520 learned_addr learned my addr 172.21.15.47:0/1871787520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.299+0000 7fe626a00640 1 --2- 172.21.15.47:0/1871787520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fb9e0 0x56037870bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.300+0000 7fe627e00640 1 -- 172.21.15.47:0/1871787520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fa810 msgr2=0x5603786fe9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:36:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.300+0000 7fe627e00640 1 --2- 172.21.15.47:0/1871787520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fa810 0x5603786fe9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.300+0000 7fe627e00640 1 -- 172.21.15.47:0/1871787520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fb9e0 msgr2=0x56037870bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.300+0000 7fe627e00640 1 --2- 172.21.15.47:0/1871787520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fb9e0 0x56037870bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.300+0000 7fe627e00640 1 -- 172.21.15.47:0/1871787520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603787024d0 con 0x5603786fc3a0 2024-03-21T11:36:18.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.300+0000 7fe627e00640 1 --2- 172.21.15.47:0/1871787520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fc3a0 0x5603786fef20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe6200a46e0 tx=0x7fe62009a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91c4aa6d4611ad38 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:18.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.300+0000 7fe613e00640 1 -- 172.21.15.47:0/1871787520 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6200a0350 con 0x5603786fc3a0 2024-03-21T11:36:18.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.301+0000 7fe613e00640 1 -- 172.21.15.47:0/1871787520 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6200a4c10 con 0x5603786fc3a0 2024-03-21T11:36:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.301+0000 7fe613e00640 1 -- 172.21.15.47:0/1871787520 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62009b650 con 0x5603786fc3a0 2024-03-21T11:36:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.302+0000 7fe63137a740 1 -- 172.21.15.47:0/1871787520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fc3a0 msgr2=0x5603786fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.302+0000 7fe63137a740 1 --2- 172.21.15.47:0/1871787520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fc3a0 0x5603786fef20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe6200a46e0 tx=0x7fe62009a400 comp rx=0 tx=0).stop 2024-03-21T11:36:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.302+0000 7fe63137a740 1 -- 172.21.15.47:0/1871787520 shutdown_connections 2024-03-21T11:36:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.302+0000 7fe63137a740 1 --2- 172.21.15.47:0/1871787520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fc3a0 0x5603786fef20 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.302+0000 7fe63137a740 1 --2- 172.21.15.47:0/1871787520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fb9e0 0x56037870bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.302+0000 7fe63137a740 1 --2- 172.21.15.47:0/1871787520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fa810 0x5603786fe9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.302+0000 7fe63137a740 1 -- 172.21.15.47:0/1871787520 >> 172.21.15.47:0/1871787520 conn(0x5603786f5c00 msgr2=0x5603786f96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.303+0000 7fe63137a740 1 -- 172.21.15.47:0/1871787520 shutdown_connections 2024-03-21T11:36:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.303+0000 7fe63137a740 1 -- 172.21.15.47:0/1871787520 wait complete. 2024-03-21T11:36:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.303+0000 7fe63137a740 1 -- start start 2024-03-21T11:36:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.304+0000 7fe63137a740 1 Processor -- start 2024-03-21T11:36:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.306+0000 7fe63137a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fa810 0x560378707a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.306+0000 7fe63137a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fb9e0 0x560378707f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.306+0000 7fe63137a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fc3a0 0x56037870af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.306+0000 7fe63137a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281ad690 con 0x5603786fc3a0 2024-03-21T11:36:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.306+0000 7fe63137a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281ad030 con 0x5603786fb9e0 2024-03-21T11:36:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.306+0000 7fe63137a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281aa530 con 0x5603786fa810 2024-03-21T11:36:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.306+0000 7fe63137a740 10 client.0 _ll_register_callbacks cb 0x7fe618005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.307+0000 7fe627e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fc3a0 0x56037870af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.307+0000 7fe627400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fa810 0x560378707a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.307+0000 7fe627400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fa810 0x560378707a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44810/0 (socket says 172.21.15.47:44810) 2024-03-21T11:36:18.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.307+0000 7fe627400640 1 -- 172.21.15.47:0/3332395742 learned_addr learned my addr 172.21.15.47:0/3332395742 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.307+0000 7fe626a00640 1 --2- 172.21.15.47:0/3332395742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fb9e0 0x560378707f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.307+0000 7fe627e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fc3a0 0x56037870af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35066/0 (socket says 172.21.15.47:35066) 2024-03-21T11:36:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.307+0000 7fe627400640 1 -- 172.21.15.47:0/3332395742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fb9e0 msgr2=0x560378707f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.307+0000 7fe627400640 1 --2- 172.21.15.47:0/3332395742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fb9e0 0x560378707f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.307+0000 7fe627400640 1 -- 172.21.15.47:0/3332395742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fc3a0 msgr2=0x56037870af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.307+0000 7fe627400640 1 --2- 172.21.15.47:0/3332395742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fc3a0 0x56037870af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.307+0000 7fe627400640 1 -- 172.21.15.47:0/3332395742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56037870b530 con 0x5603786fa810 2024-03-21T11:36:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.309+0000 7fe627400640 1 --2- 172.21.15.47:0/3332395742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fa810 0x560378707a30 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe618011d10 tx=0x7fe618009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d88661c85ae54c0c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.309+0000 7fe611600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.310+0000 7fe611600640 1 -- 172.21.15.47:0/3332395742 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61803d060 con 0x5603786fa810 2024-03-21T11:36:18.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.310+0000 7fe611600640 1 -- 172.21.15.47:0/3332395742 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe618019430 con 0x5603786fa810 2024-03-21T11:36:18.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.310+0000 7fe63137a740 1 -- 172.21.15.47:0/3332395742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6200a4c10 con 0x5603786fa810 2024-03-21T11:36:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.312+0000 7fe611600640 1 -- 172.21.15.47:0/3332395742 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe618044050 con 0x5603786fa810 2024-03-21T11:36:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.312+0000 7fe63137a740 10 client.29890 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.312+0000 7fe63137a740 1 -- 172.21.15.47:0/3332395742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5603787746c0 con 0x5603786fa810 2024-03-21T11:36:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.313+0000 7fe611600640 1 -- 172.21.15.47:0/3332395742 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6200a4c10 con 0x5603786fa810 2024-03-21T11:36:18.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.313+0000 7fe611600640 1 -- 172.21.15.47:0/3332395742 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe618019ba0 con 0x5603786fa810 2024-03-21T11:36:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.313+0000 7fe63137a740 10 client.29890 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.313+0000 7fe63137a740 10 client.29890 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:36:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.313+0000 7fe63137a740 20 client.29890 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.313+0000 7fe63137a740 1 --2- 172.21.15.47:0/3332395742 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x560378774ff0 0x560378777410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.313+0000 7fe63137a740 4 client.29890 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:36:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.313+0000 7fe63137a740 1 -- 172.21.15.47:0/3332395742 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560378712a10 con 0x560378774ff0 2024-03-21T11:36:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:18 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3823484210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:36:18.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.314+0000 7fe626a00640 1 --2- 172.21.15.47:0/3332395742 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x560378774ff0 0x560378777410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.318+0000 7fe626a00640 1 --2- 172.21.15.47:0/3332395742 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x560378774ff0 0x560378777410 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ad6ae1edd28f19d server_cookie=7e535a2aab3ddb41 in_seq=0 out_seq=0 2024-03-21T11:36:18.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.318+0000 7fe611600640 10 client.29890 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:36:18.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.319+0000 7fe611600640 1 -- 172.21.15.47:0/3332395742 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560378712a10 con 0x560378774ff0 2024-03-21T11:36:18.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.319+0000 7fe611600640 10 client.29890 handle_command_reply: tid=0 2024-03-21T11:36:18.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.319+0000 7fe63137a740 1 client.29890 shutdown 2024-03-21T11:36:18.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.320+0000 7fe63137a740 1 -- 172.21.15.47:0/3332395742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fa810 msgr2=0x560378707a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.320+0000 7fe63137a740 1 --2- 172.21.15.47:0/3332395742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fa810 0x560378707a30 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe618011d10 tx=0x7fe618009e80 comp rx=0 tx=0).stop 2024-03-21T11:36:18.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.321+0000 7fe63137a740 1 -- 172.21.15.47:0/3332395742 shutdown_connections 2024-03-21T11:36:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.321+0000 7fe63137a740 1 --2- 172.21.15.47:0/3332395742 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x560378774ff0 0x560378777410 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.321+0000 7fe63137a740 1 --2- 172.21.15.47:0/3332395742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5603786fc3a0 0x56037870af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.321+0000 7fe63137a740 1 --2- 172.21.15.47:0/3332395742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5603786fb9e0 0x560378707f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.321+0000 7fe63137a740 1 --2- 172.21.15.47:0/3332395742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5603786fa810 0x560378707a30 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.321+0000 7fe611600640 0 client.29890 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:36:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.321+0000 7fe63137a740 1 -- 172.21.15.47:0/3332395742 >> 172.21.15.47:0/3332395742 conn(0x5603786f5c00 msgr2=0x5603786f4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.322+0000 7fe63137a740 1 -- 172.21.15.47:0/3332395742 shutdown_connections 2024-03-21T11:36:18.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.322+0000 7fe63137a740 1 -- 172.21.15.47:0/3332395742 wait complete. 2024-03-21T11:36:18.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.322+0000 7fe63137a740 20 client.29890 trim_cache size 0 max 16384 2024-03-21T11:36:18.322 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:36:18.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.323+0000 7fe62e600640 1 -- 172.21.15.47:0/2845953597 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe60c0a3020 msgr2=0x7fe60c0a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.323+0000 7fe62e600640 1 --2- 172.21.15.47:0/2845953597 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe60c0a3020 0x7fe60c0a54e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe61402c8e0 tx=0x7fe614005bd0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.323+0000 7fe62e600640 1 -- 172.21.15.47:0/2845953597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe628105470 msgr2=0x7fe6281a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.323+0000 7fe62e600640 1 --2- 172.21.15.47:0/2845953597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe628105470 0x7fe6281a1190 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe620009fd0 tx=0x7fe62000ef10 comp rx=0 tx=0).stop 2024-03-21T11:36:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.324+0000 7fe62e600640 1 -- 172.21.15.47:0/2845953597 shutdown_connections 2024-03-21T11:36:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.324+0000 7fe62e600640 1 --2- 172.21.15.47:0/2845953597 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe60c0a3020 0x7fe60c0a54e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.324+0000 7fe62e600640 1 --2- 172.21.15.47:0/2845953597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe628105470 0x7fe6281a1190 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.324+0000 7fe62e600640 1 --2- 172.21.15.47:0/2845953597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe628104ab0 0x7fe62819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.324+0000 7fe62e600640 1 --2- 172.21.15.47:0/2845953597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe6281038c0 0x7fe628199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.324+0000 7fe62e600640 1 -- 172.21.15.47:0/2845953597 >> 172.21.15.47:0/2845953597 conn(0x7fe6280fee50 msgr2=0x7fe628107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.324+0000 7fe62e600640 1 -- 172.21.15.47:0/2845953597 shutdown_connections 2024-03-21T11:36:18.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.324+0000 7fe62e600640 1 -- 172.21.15.47:0/2845953597 wait complete. 2024-03-21T11:36:18.344 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:36:18.344 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:36:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:18 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3823484210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:36:18.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.548+0000 7fbe40c00640 1 -- 172.21.15.47:0/861834149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe3c1020f0 msgr2=0x7fbe3c1044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.548+0000 7fbe40c00640 1 --2- 172.21.15.47:0/861834149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe3c1020f0 0x7fbe3c1044e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009900 tx=0x7fbe2c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:36:18.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.548+0000 7fbe40c00640 1 -- 172.21.15.47:0/861834149 shutdown_connections 2024-03-21T11:36:18.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.548+0000 7fbe40c00640 1 --2- 172.21.15.47:0/861834149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c107350 0x7fbe3c109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.548+0000 7fbe40c00640 1 --2- 172.21.15.47:0/861834149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe3c104a20 0x7fbe3c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.548+0000 7fbe40c00640 1 --2- 172.21.15.47:0/861834149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe3c1020f0 0x7fbe3c1044e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.548+0000 7fbe40c00640 1 -- 172.21.15.47:0/861834149 >> 172.21.15.47:0/861834149 conn(0x7fbe3c0fba20 msgr2=0x7fbe3c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:18.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.548+0000 7fbe40c00640 1 -- 172.21.15.47:0/861834149 shutdown_connections 2024-03-21T11:36:18.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.549+0000 7fbe40c00640 1 -- 172.21.15.47:0/861834149 wait complete. 2024-03-21T11:36:18.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.549+0000 7fbe40c00640 1 Processor -- start 2024-03-21T11:36:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.550+0000 7fbe40c00640 1 -- start start 2024-03-21T11:36:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.550+0000 7fbe40c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe3c1020f0 0x7fbe3c199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.550+0000 7fbe40c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c104a20 0x7fbe3c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.550+0000 7fbe40c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe3c107350 0x7fbe3c1a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.550+0000 7fbe40c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c10d250 con 0x7fbe3c107350 2024-03-21T11:36:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.550+0000 7fbe40c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c10d0d0 con 0x7fbe3c104a20 2024-03-21T11:36:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.550+0000 7fbe40c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c10d3d0 con 0x7fbe3c1020f0 2024-03-21T11:36:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.550+0000 7fbe3b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe3c1020f0 0x7fbe3c199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.550+0000 7fbe3b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe3c1020f0 0x7fbe3c199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44840/0 (socket says 172.21.15.47:44840) 2024-03-21T11:36:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.550+0000 7fbe3b400640 1 -- 172.21.15.47:0/1598208309 learned_addr learned my addr 172.21.15.47:0/1598208309 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.550+0000 7fbe32a00640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c104a20 0x7fbe3c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.551+0000 7fbe3be00640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe3c107350 0x7fbe3c1a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.551+0000 7fbe3b400640 1 -- 172.21.15.47:0/1598208309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c104a20 msgr2=0x7fbe3c19a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.551+0000 7fbe3b400640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c104a20 0x7fbe3c19a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.551+0000 7fbe3b400640 1 -- 172.21.15.47:0/1598208309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe3c107350 msgr2=0x7fbe3c1a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.551+0000 7fbe3b400640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe3c107350 0x7fbe3c1a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.551+0000 7fbe3b400640 1 -- 172.21.15.47:0/1598208309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c1a1740 con 0x7fbe3c1020f0 2024-03-21T11:36:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.551+0000 7fbe3be00640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe3c107350 0x7fbe3c1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.551+0000 7fbe3b400640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe3c1020f0 0x7fbe3c199b20 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009230 tx=0x7fbe2c02c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.551+0000 7fbe38c00640 1 -- 172.21.15.47:0/1598208309 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c041070 con 0x7fbe3c1020f0 2024-03-21T11:36:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.552+0000 7fbe38c00640 1 -- 172.21.15.47:0/1598208309 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe2c004450 con 0x7fbe3c1020f0 2024-03-21T11:36:18.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.552+0000 7fbe40c00640 1 -- 172.21.15.47:0/1598208309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c1a19d0 con 0x7fbe3c1020f0 2024-03-21T11:36:18.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.552+0000 7fbe38c00640 1 -- 172.21.15.47:0/1598208309 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c003e00 con 0x7fbe3c1020f0 2024-03-21T11:36:18.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.552+0000 7fbe40c00640 1 -- 172.21.15.47:0/1598208309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c1a1ec0 con 0x7fbe3c1020f0 2024-03-21T11:36:18.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.555+0000 7fbe44ff2740 1 Processor -- start 2024-03-21T11:36:18.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.555+0000 7fbe38c00640 1 -- 172.21.15.47:0/1598208309 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbe2c044070 con 0x7fbe3c1020f0 2024-03-21T11:36:18.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.555+0000 7fbe44ff2740 1 -- start start 2024-03-21T11:36:18.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.555+0000 7fbe44ff2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577182399f0 0x557718239e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.555+0000 7fbe44ff2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823abc0 0x55771823b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.555+0000 7fbe44ff2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823b580 0x557718241e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.555+0000 7fbe44ff2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771819e2c0 con 0x55771823b580 2024-03-21T11:36:18.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.555+0000 7fbe44ff2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557718146c90 con 0x5577182399f0 2024-03-21T11:36:18.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.555+0000 7fbe44ff2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771819a360 con 0x55771823abc0 2024-03-21T11:36:18.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.556+0000 7fbe3be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577182399f0 0x557718239e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.556+0000 7fbe3b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823abc0 0x55771823b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.556+0000 7fbe3be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577182399f0 0x557718239e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59092/0 (socket says 172.21.15.47:59092) 2024-03-21T11:36:18.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.556+0000 7fbe3be00640 1 -- 172.21.15.47:0/676070348 learned_addr learned my addr 172.21.15.47:0/676070348 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:18.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.556+0000 7fbe32a00640 1 --2- 172.21.15.47:0/676070348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823b580 0x557718241e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.556+0000 7fbe3be00640 1 -- 172.21.15.47:0/676070348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823abc0 msgr2=0x55771823b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:18.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.556+0000 7fbe3be00640 1 --2- 172.21.15.47:0/676070348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823abc0 0x55771823b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.556+0000 7fbe3be00640 1 -- 172.21.15.47:0/676070348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823b580 msgr2=0x557718241e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:18.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.556+0000 7fbe3be00640 1 --2- 172.21.15.47:0/676070348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823b580 0x557718241e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.556+0000 7fbe3be00640 1 -- 172.21.15.47:0/676070348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55771816b080 con 0x5577182399f0 2024-03-21T11:36:18.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.556+0000 7fbe32a00640 1 --2- 172.21.15.47:0/676070348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823b580 0x557718241e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:18.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.556+0000 7fbe3b400640 1 --2- 172.21.15.47:0/676070348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823abc0 0x55771823b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:18.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.556+0000 7fbe3be00640 1 --2- 172.21.15.47:0/676070348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577182399f0 0x557718239e10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbe34010450 tx=0x7fbe34010910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=209870ce6dcaaa47 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:18.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.557+0000 7fbe2be00640 1 -- 172.21.15.47:0/676070348 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34017070 con 0x5577182399f0 2024-03-21T11:36:18.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.557+0000 7fbe38c00640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe200a2f20 0x7fbe200a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.557+0000 7fbe2be00640 1 -- 172.21.15.47:0/676070348 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe34011c40 con 0x5577182399f0 2024-03-21T11:36:18.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.557+0000 7fbe2be00640 1 -- 172.21.15.47:0/676070348 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3401ad30 con 0x5577182399f0 2024-03-21T11:36:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.557+0000 7fbe38c00640 1 -- 172.21.15.47:0/1598208309 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbe2c0f1370 con 0x7fbe3c1020f0 2024-03-21T11:36:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.557+0000 7fbe3b400640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe200a2f20 0x7fbe200a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.558+0000 7fbe44ff2740 1 -- 172.21.15.47:0/676070348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577182399f0 msgr2=0x557718239e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.558+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/676070348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577182399f0 0x557718239e10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbe34010450 tx=0x7fbe34010910 comp rx=0 tx=0).stop 2024-03-21T11:36:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.558+0000 7fbe3b400640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe200a2f20 0x7fbe200a53e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0bd4b0 tx=0x7fbe2c0f4ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.558+0000 7fbe44ff2740 1 -- 172.21.15.47:0/676070348 shutdown_connections 2024-03-21T11:36:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.558+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/676070348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823b580 0x557718241e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.558+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/676070348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823abc0 0x55771823b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.558+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/676070348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577182399f0 0x557718239e10 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.558+0000 7fbe44ff2740 1 -- 172.21.15.47:0/676070348 >> 172.21.15.47:0/676070348 conn(0x557718234de0 msgr2=0x5577182351e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.558+0000 7fbe44ff2740 1 -- 172.21.15.47:0/676070348 shutdown_connections 2024-03-21T11:36:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.558+0000 7fbe44ff2740 1 -- 172.21.15.47:0/676070348 wait complete. 2024-03-21T11:36:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.558+0000 7fbe44ff2740 1 -- start start 2024-03-21T11:36:18.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.558+0000 7fbe44ff2740 1 Processor -- start 2024-03-21T11:36:18.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.559+0000 7fbe44ff2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577182399f0 0x557718243c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.559+0000 7fbe44ff2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823abc0 0x5577182441d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.559+0000 7fbe44ff2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823b580 0x55771824b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.559+0000 7fbe44ff2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557718242590 con 0x55771823abc0 2024-03-21T11:36:18.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.559+0000 7fbe44ff2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771815a0d0 con 0x5577182399f0 2024-03-21T11:36:18.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.559+0000 7fbe44ff2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557718242710 con 0x55771823b580 2024-03-21T11:36:18.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.559+0000 7fbe44ff2740 10 client.0 _ll_register_callbacks cb 0x55771815ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:18.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.559+0000 7fbe3be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577182399f0 0x557718243c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.559+0000 7fbe32a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823abc0 0x5577182441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.559+0000 7fbe3be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577182399f0 0x557718243c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59108/0 (socket says 172.21.15.47:59108) 2024-03-21T11:36:18.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.559+0000 7fbe3be00640 1 -- 172.21.15.47:0/3295763320 learned_addr learned my addr 172.21.15.47:0/3295763320 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:18.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.559+0000 7fbe3be00640 1 --2- 172.21.15.47:0/3295763320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823b580 0x55771824b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.560+0000 7fbe3be00640 1 -- 172.21.15.47:0/3295763320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823b580 msgr2=0x55771824b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:18.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.560+0000 7fbe3be00640 1 --2- 172.21.15.47:0/3295763320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823b580 0x55771824b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.560+0000 7fbe3be00640 1 -- 172.21.15.47:0/3295763320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823abc0 msgr2=0x5577182441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:18.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.560+0000 7fbe3be00640 1 --2- 172.21.15.47:0/3295763320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823abc0 0x5577182441d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.560+0000 7fbe3be00640 1 -- 172.21.15.47:0/3295763320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557717eb23f0 con 0x5577182399f0 2024-03-21T11:36:18.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.560+0000 7fbe3be00640 1 --2- 172.21.15.47:0/3295763320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577182399f0 0x557718243c90 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009e20 tx=0x7fbe34009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab489753a86e3585 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:18.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.560+0000 7fbe28c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:18.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.560+0000 7fbe28c00640 1 -- 172.21.15.47:0/3295763320 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34017030 con 0x5577182399f0 2024-03-21T11:36:18.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.560+0000 7fbe44ff2740 1 -- 172.21.15.47:0/3295763320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe34011c40 con 0x5577182399f0 2024-03-21T11:36:18.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.560+0000 7fbe28c00640 1 -- 172.21.15.47:0/3295763320 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbe34003970 con 0x5577182399f0 2024-03-21T11:36:18.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.560+0000 7fbe28c00640 1 -- 172.21.15.47:0/3295763320 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe34003b70 con 0x5577182399f0 2024-03-21T11:36:18.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.561+0000 7fbe44ff2740 10 client.29870 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:18.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.561+0000 7fbe44ff2740 1 -- 172.21.15.47:0/3295763320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557717f7c7d0 con 0x5577182399f0 2024-03-21T11:36:18.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.561+0000 7fbe28c00640 1 -- 172.21.15.47:0/3295763320 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe34011c40 con 0x5577182399f0 2024-03-21T11:36:18.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.561+0000 7fbe28c00640 1 -- 172.21.15.47:0/3295763320 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbe34023030 con 0x5577182399f0 2024-03-21T11:36:18.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.561+0000 7fbe44ff2740 10 client.29870 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:18.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.561+0000 7fbe44ff2740 10 client.29870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:18.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.561+0000 7fbe44ff2740 20 client.29870 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:18.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.561+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/3295763320 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55771824f420 0x557718251810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.561+0000 7fbe44ff2740 4 client.29870 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:18.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.561+0000 7fbe44ff2740 1 -- 172.21.15.47:0/3295763320 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577181ab800 con 0x55771824f420 2024-03-21T11:36:18.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.561+0000 7fbe32a00640 1 --2- 172.21.15.47:0/3295763320 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55771824f420 0x557718251810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.562+0000 7fbe32a00640 1 --2- 172.21.15.47:0/3295763320 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55771824f420 0x557718251810 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cfdcff5c2cb6bb0 server_cookie=79b84e6b7f6972ac in_seq=0 out_seq=0 2024-03-21T11:36:18.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.562+0000 7fbe28c00640 10 client.29870 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:18.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.563+0000 7fbe28c00640 1 -- 172.21.15.47:0/3295763320 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5577181ab800 con 0x55771824f420 2024-03-21T11:36:18.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.563+0000 7fbe28c00640 10 client.29870 handle_command_reply: tid=0 2024-03-21T11:36:18.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.563+0000 7fbe44ff2740 1 client.29870 shutdown 2024-03-21T11:36:18.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.564+0000 7fbe44ff2740 1 -- 172.21.15.47:0/3295763320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577182399f0 msgr2=0x557718243c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.564+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/3295763320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577182399f0 0x557718243c90 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009e20 tx=0x7fbe34009c20 comp rx=0 tx=0).stop 2024-03-21T11:36:18.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.564+0000 7fbe44ff2740 1 -- 172.21.15.47:0/3295763320 shutdown_connections 2024-03-21T11:36:18.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.564+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/3295763320 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55771824f420 0x557718251810 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.564+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/3295763320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823b580 0x55771824b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.564+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/3295763320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823abc0 0x5577182441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.564+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/3295763320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577182399f0 0x557718243c90 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.564+0000 7fbe28c00640 0 client.29870 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:18.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.564+0000 7fbe44ff2740 1 -- 172.21.15.47:0/3295763320 >> 172.21.15.47:0/3295763320 conn(0x557718234de0 msgr2=0x557718234160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:18.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.564+0000 7fbe44ff2740 1 -- 172.21.15.47:0/3295763320 shutdown_connections 2024-03-21T11:36:18.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.564+0000 7fbe44ff2740 1 -- 172.21.15.47:0/3295763320 wait complete. 2024-03-21T11:36:18.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.564+0000 7fbe44ff2740 20 client.29870 trim_cache size 0 max 16384 2024-03-21T11:36:18.578 INFO:tasks.workunit.client.0.smithi047.stdout: 1 987723 7.95 MB/sec execute 542 sec latency 50.735 ms 2024-03-21T11:36:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.579+0000 7fbe44ff2740 1 Processor -- start 2024-03-21T11:36:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.579+0000 7fbe44ff2740 1 -- start start 2024-03-21T11:36:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.579+0000 7fbe44ff2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577182399f0 0x55771823dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.579+0000 7fbe44ff2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823abc0 0x55771824b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe44ff2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55771823b580 0x55771823e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe44ff2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c1ac150 con 0x5577182399f0 2024-03-21T11:36:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe44ff2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c1adad0 con 0x55771823b580 2024-03-21T11:36:18.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe44ff2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c1ac7b0 con 0x55771823abc0 2024-03-21T11:36:18.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe32a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823abc0 0x55771824b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe32a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823abc0 0x55771824b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44858/0 (socket says 172.21.15.47:44858) 2024-03-21T11:36:18.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe32a00640 1 -- 172.21.15.47:0/43715113 learned_addr learned my addr 172.21.15.47:0/43715113 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:18.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe3be00640 1 --2- 172.21.15.47:0/43715113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577182399f0 0x55771823dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe3be00640 1 --2- 172.21.15.47:0/43715113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55771823b580 0x55771823e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe32a00640 1 -- 172.21.15.47:0/43715113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55771823b580 msgr2=0x55771823e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:18.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe32a00640 1 --2- 172.21.15.47:0/43715113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55771823b580 0x55771823e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe32a00640 1 -- 172.21.15.47:0/43715113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577182399f0 msgr2=0x55771823dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:18.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe32a00640 1 --2- 172.21.15.47:0/43715113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577182399f0 0x55771823dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe32a00640 1 -- 172.21.15.47:0/43715113 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557718243080 con 0x55771823abc0 2024-03-21T11:36:18.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.580+0000 7fbe3be00640 1 --2- 172.21.15.47:0/43715113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55771823b580 0x55771823e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:36:18.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.581+0000 7fbe32a00640 1 --2- 172.21.15.47:0/43715113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823abc0 0x55771824b1a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c19b030 tx=0x7fbe24005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=559ddfa38914ae2e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:18.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.581+0000 7fbe2b400640 1 -- 172.21.15.47:0/43715113 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24026070 con 0x55771823abc0 2024-03-21T11:36:18.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.581+0000 7fbe2b400640 1 -- 172.21.15.47:0/43715113 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe24020e40 con 0x55771823abc0 2024-03-21T11:36:18.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.581+0000 7fbe2b400640 1 -- 172.21.15.47:0/43715113 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24003830 con 0x55771823abc0 2024-03-21T11:36:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.582+0000 7fbe44ff2740 1 -- 172.21.15.47:0/43715113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823abc0 msgr2=0x55771824b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.582+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/43715113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823abc0 0x55771824b1a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c19b030 tx=0x7fbe24005f90 comp rx=0 tx=0).stop 2024-03-21T11:36:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.582+0000 7fbe44ff2740 1 -- 172.21.15.47:0/43715113 shutdown_connections 2024-03-21T11:36:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.582+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/43715113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55771823b580 0x55771823e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.582+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/43715113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55771823abc0 0x55771824b1a0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.582+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/43715113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577182399f0 0x55771823dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.582+0000 7fbe44ff2740 1 -- 172.21.15.47:0/43715113 >> 172.21.15.47:0/43715113 conn(0x557718234de0 msgr2=0x5577182388d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.582+0000 7fbe44ff2740 1 -- 172.21.15.47:0/43715113 shutdown_connections 2024-03-21T11:36:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.582+0000 7fbe44ff2740 1 -- 172.21.15.47:0/43715113 wait complete. 2024-03-21T11:36:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.582+0000 7fbe44ff2740 1 -- start start 2024-03-21T11:36:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.582+0000 7fbe44ff2740 1 Processor -- start 2024-03-21T11:36:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe44ff2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577182399f0 0x557718247270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe44ff2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55771823abc0 0x5577182477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe44ff2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823b580 0x55771824a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe44ff2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c1ad470 con 0x55771823b580 2024-03-21T11:36:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe44ff2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c1ace10 con 0x55771823abc0 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe44ff2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c1aa310 con 0x5577182399f0 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe44ff2740 10 client.0 _ll_register_callbacks cb 0x7fbe2c0f4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe32a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55771823abc0 0x5577182477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe32a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55771823abc0 0x5577182477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59116/0 (socket says 172.21.15.47:59116) 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe32a00640 1 -- 172.21.15.47:0/2620548739 learned_addr learned my addr 172.21.15.47:0/2620548739 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe3be00640 1 --2- 172.21.15.47:0/2620548739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577182399f0 0x557718247270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe3be00640 1 --2- 172.21.15.47:0/2620548739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823b580 0x55771824a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe32a00640 1 -- 172.21.15.47:0/2620548739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577182399f0 msgr2=0x557718247270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe32a00640 1 --2- 172.21.15.47:0/2620548739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577182399f0 0x557718247270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.583+0000 7fbe32a00640 1 -- 172.21.15.47:0/2620548739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823b580 msgr2=0x55771824a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.584+0000 7fbe32a00640 1 --2- 172.21.15.47:0/2620548739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823b580 0x55771824a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.584+0000 7fbe32a00640 1 -- 172.21.15.47:0/2620548739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55771824ad50 con 0x55771823abc0 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.584+0000 7fbe3be00640 1 --2- 172.21.15.47:0/2620548739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823b580 0x55771824a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.584+0000 7fbe32a00640 1 --2- 172.21.15.47:0/2620548739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55771823abc0 0x5577182477b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fbe24024040 tx=0x7fbe24005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59788c8e0cc6ad6b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.584+0000 7fbe28c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.584+0000 7fbe28c00640 1 -- 172.21.15.47:0/2620548739 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24030060 con 0x55771823abc0 2024-03-21T11:36:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.584+0000 7fbe44ff2740 1 -- 172.21.15.47:0/2620548739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577181a5b50 con 0x55771823abc0 2024-03-21T11:36:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.584+0000 7fbe28c00640 1 -- 172.21.15.47:0/2620548739 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbe240200d0 con 0x55771823abc0 2024-03-21T11:36:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.584+0000 7fbe28c00640 1 -- 172.21.15.47:0/2620548739 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe24026040 con 0x55771823abc0 2024-03-21T11:36:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.584+0000 7fbe44ff2740 10 client.29873 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.584+0000 7fbe44ff2740 1 -- 172.21.15.47:0/2620548739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5577182ab4f0 con 0x55771823abc0 2024-03-21T11:36:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.585+0000 7fbe28c00640 1 -- 172.21.15.47:0/2620548739 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5577181a5b50 con 0x55771823abc0 2024-03-21T11:36:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.585+0000 7fbe28c00640 1 -- 172.21.15.47:0/2620548739 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbe24004b10 con 0x55771823abc0 2024-03-21T11:36:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.585+0000 7fbe44ff2740 10 client.29873 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.585+0000 7fbe44ff2740 10 client.29873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.585+0000 7fbe44ff2740 20 client.29873 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.585+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/2620548739 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5577182abda0 0x5577182ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.585+0000 7fbe44ff2740 4 client.29873 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:36:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.585+0000 7fbe44ff2740 1 -- 172.21.15.47:0/2620548739 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557718123500 con 0x5577182abda0 2024-03-21T11:36:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.585+0000 7fbe3be00640 1 --2- 172.21.15.47:0/2620548739 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5577182abda0 0x5577182ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.586+0000 7fbe3be00640 1 --2- 172.21.15.47:0/2620548739 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5577182abda0 0x5577182ae1c0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=372fcdc0ea1bf8b2 server_cookie=6e8747ec2cea47d6 in_seq=0 out_seq=0 2024-03-21T11:36:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.586+0000 7fbe28c00640 10 client.29873 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.586+0000 7fbe28c00640 1 -- 172.21.15.47:0/2620548739 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557718123500 con 0x5577182abda0 2024-03-21T11:36:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.586+0000 7fbe28c00640 10 client.29873 handle_command_reply: tid=0 2024-03-21T11:36:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.586+0000 7fbe44ff2740 1 client.29873 shutdown 2024-03-21T11:36:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.586+0000 7fbe44ff2740 1 -- 172.21.15.47:0/2620548739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55771823abc0 msgr2=0x5577182477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.586+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/2620548739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55771823abc0 0x5577182477b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fbe24024040 tx=0x7fbe24005d90 comp rx=0 tx=0).stop 2024-03-21T11:36:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.586+0000 7fbe44ff2740 1 -- 172.21.15.47:0/2620548739 shutdown_connections 2024-03-21T11:36:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.586+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/2620548739 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5577182abda0 0x5577182ae1c0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.586+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/2620548739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55771823b580 0x55771824a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.586+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/2620548739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55771823abc0 0x5577182477b0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.586+0000 7fbe44ff2740 1 --2- 172.21.15.47:0/2620548739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577182399f0 0x557718247270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.586+0000 7fbe28c00640 0 client.29873 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:18.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.586+0000 7fbe44ff2740 1 -- 172.21.15.47:0/2620548739 >> 172.21.15.47:0/2620548739 conn(0x557718234de0 msgr2=0x5577182388d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:18.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.587+0000 7fbe44ff2740 1 -- 172.21.15.47:0/2620548739 shutdown_connections 2024-03-21T11:36:18.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.587+0000 7fbe44ff2740 1 -- 172.21.15.47:0/2620548739 wait complete. 2024-03-21T11:36:18.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.587+0000 7fbe44ff2740 20 client.29873 trim_cache size 0 max 16384 2024-03-21T11:36:18.591 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:36:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.588+0000 7fbe40c00640 1 -- 172.21.15.47:0/1598208309 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe200a2f20 msgr2=0x7fbe200a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.588+0000 7fbe40c00640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe200a2f20 0x7fbe200a53e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0bd4b0 tx=0x7fbe2c0f4ee0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.588+0000 7fbe40c00640 1 -- 172.21.15.47:0/1598208309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe3c1020f0 msgr2=0x7fbe3c199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.588+0000 7fbe40c00640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe3c1020f0 0x7fbe3c199b20 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009230 tx=0x7fbe2c02c660 comp rx=0 tx=0).stop 2024-03-21T11:36:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.588+0000 7fbe40c00640 1 -- 172.21.15.47:0/1598208309 shutdown_connections 2024-03-21T11:36:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.588+0000 7fbe40c00640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe200a2f20 0x7fbe200a53e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.588+0000 7fbe40c00640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe3c107350 0x7fbe3c1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.588+0000 7fbe40c00640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe3c104a20 0x7fbe3c19a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.588+0000 7fbe40c00640 1 --2- 172.21.15.47:0/1598208309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe3c1020f0 0x7fbe3c199b20 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:18.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.588+0000 7fbe40c00640 1 -- 172.21.15.47:0/1598208309 >> 172.21.15.47:0/1598208309 conn(0x7fbe3c0fba20 msgr2=0x7fbe3c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:18.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.588+0000 7fbe40c00640 1 -- 172.21.15.47:0/1598208309 shutdown_connections 2024-03-21T11:36:18.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:18.588+0000 7fbe40c00640 1 -- 172.21.15.47:0/1598208309 wait complete. 2024-03-21T11:36:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:19 smithi047 ceph-mon[31351]: pgmap v789: 129 pgs: 129 active+clean; 36 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 2.7 MiB/s wr, 287 op/s 2024-03-21T11:36:19.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:19 smithi100 ceph-mon[36245]: pgmap v789: 129 pgs: 129 active+clean; 36 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 2.7 MiB/s wr, 287 op/s 2024-03-21T11:36:19.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:19 smithi102 ceph-mon[36748]: pgmap v789: 129 pgs: 129 active+clean; 36 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 2.7 MiB/s wr, 287 op/s 2024-03-21T11:36:19.579 INFO:tasks.workunit.client.0.smithi047.stdout: 1 988458 7.93 MB/sec execute 543 sec latency 26.341 ms 2024-03-21T11:36:19.608 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:36:19.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:36:19.609 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:36:19.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.846+0000 7fdb92600640 1 -- 172.21.15.47:0/1334504670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb8c1015a0 msgr2=0x7fdb8c1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:19.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.846+0000 7fdb92600640 1 --2- 172.21.15.47:0/1334504670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb8c1015a0 0x7fdb8c1019c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009860 tx=0x7fdb8002bb70 comp rx=0 tx=0).stop 2024-03-21T11:36:19.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.846+0000 7fdb92600640 1 -- 172.21.15.47:0/1334504670 shutdown_connections 2024-03-21T11:36:19.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.846+0000 7fdb92600640 1 --2- 172.21.15.47:0/1334504670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb8c103150 0x7fdb8c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.846+0000 7fdb92600640 1 --2- 172.21.15.47:0/1334504670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c102790 0x7fdb8c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.846+0000 7fdb92600640 1 --2- 172.21.15.47:0/1334504670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb8c1015a0 0x7fdb8c1019c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.847+0000 7fdb92600640 1 -- 172.21.15.47:0/1334504670 >> 172.21.15.47:0/1334504670 conn(0x7fdb8c0fcb10 msgr2=0x7fdb8c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:19.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.847+0000 7fdb92600640 1 -- 172.21.15.47:0/1334504670 shutdown_connections 2024-03-21T11:36:19.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.847+0000 7fdb92600640 1 -- 172.21.15.47:0/1334504670 wait complete. 2024-03-21T11:36:19.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.848+0000 7fdb92600640 1 Processor -- start 2024-03-21T11:36:19.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.848+0000 7fdb92600640 1 -- start start 2024-03-21T11:36:19.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb92600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c1015a0 0x7fdb8c1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb92600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb8c102790 0x7fdb8c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb92600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb8c103150 0x7fdb8c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb92600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c10d200 con 0x7fdb8c103150 2024-03-21T11:36:19.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb92600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c10d080 con 0x7fdb8c1015a0 2024-03-21T11:36:19.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb92600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c10d380 con 0x7fdb8c102790 2024-03-21T11:36:19.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb91200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c1015a0 0x7fdb8c1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb91200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c1015a0 0x7fdb8c1978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33616/0 (socket says 172.21.15.47:33616) 2024-03-21T11:36:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb91200640 1 -- 172.21.15.47:0/3939489928 learned_addr learned my addr 172.21.15.47:0/3939489928 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb8be00640 1 --2- 172.21.15.47:0/3939489928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb8c102790 0x7fdb8c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb91c00640 1 --2- 172.21.15.47:0/3939489928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb8c103150 0x7fdb8c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb91200640 1 -- 172.21.15.47:0/3939489928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb8c102790 msgr2=0x7fdb8c197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb91200640 1 --2- 172.21.15.47:0/3939489928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb8c102790 0x7fdb8c197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb91200640 1 -- 172.21.15.47:0/3939489928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb8c103150 msgr2=0x7fdb8c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb91200640 1 --2- 172.21.15.47:0/3939489928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb8c103150 0x7fdb8c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.849+0000 7fdb91200640 1 -- 172.21.15.47:0/3939489928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c19f4f0 con 0x7fdb8c1015a0 2024-03-21T11:36:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.850+0000 7fdb91200640 1 --2- 172.21.15.47:0/3939489928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c1015a0 0x7fdb8c1978d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fdb800349e0 tx=0x7fdb80032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.850+0000 7fdb89600640 1 -- 172.21.15.47:0/3939489928 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb80041020 con 0x7fdb8c1015a0 2024-03-21T11:36:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.850+0000 7fdb92600640 1 -- 172.21.15.47:0/3939489928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8c19f780 con 0x7fdb8c1015a0 2024-03-21T11:36:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.850+0000 7fdb89600640 1 -- 172.21.15.47:0/3939489928 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb8002c9f0 con 0x7fdb8c1015a0 2024-03-21T11:36:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.850+0000 7fdb89600640 1 -- 172.21.15.47:0/3939489928 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb80039040 con 0x7fdb8c1015a0 2024-03-21T11:36:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.850+0000 7fdb92600640 1 -- 172.21.15.47:0/3939489928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c19fc70 con 0x7fdb8c1015a0 2024-03-21T11:36:19.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.853+0000 7fdb89600640 1 -- 172.21.15.47:0/3939489928 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdb80044020 con 0x7fdb8c1015a0 2024-03-21T11:36:19.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.854+0000 7fdb9680f740 1 Processor -- start 2024-03-21T11:36:19.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.854+0000 7fdb9680f740 1 -- start start 2024-03-21T11:36:19.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.854+0000 7fdb9680f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d937ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.854+0000 7fdb9680f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d9380b0 0x56445d93cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.854+0000 7fdb9680f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d93d220 0x56445d93f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.854+0000 7fdb9680f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445d898eb0 con 0x56445d9380b0 2024-03-21T11:36:19.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.854+0000 7fdb9680f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445d841a60 con 0x56445d9376c0 2024-03-21T11:36:19.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.854+0000 7fdb9680f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445d894f50 con 0x56445d93d220 2024-03-21T11:36:19.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.854+0000 7fdb91c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d937ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.855+0000 7fdb91c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d937ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33626/0 (socket says 172.21.15.47:33626) 2024-03-21T11:36:19.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.855+0000 7fdb91c00640 1 -- 172.21.15.47:0/531121963 learned_addr learned my addr 172.21.15.47:0/531121963 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:19.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.855+0000 7fdb91200640 1 --2- 172.21.15.47:0/531121963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d9380b0 0x56445d93cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.855+0000 7fdb8be00640 1 --2- 172.21.15.47:0/531121963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d93d220 0x56445d93f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.855+0000 7fdb91c00640 1 -- 172.21.15.47:0/531121963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d93d220 msgr2=0x56445d93f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:19.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.855+0000 7fdb91c00640 1 --2- 172.21.15.47:0/531121963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d93d220 0x56445d93f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.855+0000 7fdb91c00640 1 -- 172.21.15.47:0/531121963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d9380b0 msgr2=0x56445d93cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:19.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.855+0000 7fdb91c00640 1 --2- 172.21.15.47:0/531121963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d9380b0 0x56445d93cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.855+0000 7fdb91c00640 1 -- 172.21.15.47:0/531121963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56445d6757e0 con 0x56445d9376c0 2024-03-21T11:36:19.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.855+0000 7fdb91c00640 1 --2- 172.21.15.47:0/531121963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d937ae0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009fc0 tx=0x7fdb84010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5761c54aca39c8e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:19.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.855+0000 7fdb89600640 1 --2- 172.21.15.47:0/3939489928 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdb740a2f20 0x7fdb740a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.856+0000 7fdb7e000640 1 -- 172.21.15.47:0/531121963 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84003b70 con 0x56445d9376c0 2024-03-21T11:36:19.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.856+0000 7fdb89600640 1 -- 172.21.15.47:0/3939489928 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdb800f0ff0 con 0x7fdb8c1015a0 2024-03-21T11:36:19.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.856+0000 7fdb7e000640 1 -- 172.21.15.47:0/531121963 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb84003d10 con 0x56445d9376c0 2024-03-21T11:36:19.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.856+0000 7fdb7e000640 1 -- 172.21.15.47:0/531121963 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84011aa0 con 0x56445d9376c0 2024-03-21T11:36:19.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.856+0000 7fdb91200640 1 --2- 172.21.15.47:0/3939489928 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdb740a2f20 0x7fdb740a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.856+0000 7fdb9680f740 1 -- 172.21.15.47:0/531121963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 msgr2=0x56445d937ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:19.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.856+0000 7fdb9680f740 1 --2- 172.21.15.47:0/531121963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d937ae0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009fc0 tx=0x7fdb84010860 comp rx=0 tx=0).stop 2024-03-21T11:36:19.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.857+0000 7fdb9680f740 1 -- 172.21.15.47:0/531121963 shutdown_connections 2024-03-21T11:36:19.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.857+0000 7fdb9680f740 1 --2- 172.21.15.47:0/531121963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d93d220 0x56445d93f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.857+0000 7fdb9680f740 1 --2- 172.21.15.47:0/531121963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d9380b0 0x56445d93cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.857+0000 7fdb9680f740 1 --2- 172.21.15.47:0/531121963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d937ae0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.857+0000 7fdb9680f740 1 -- 172.21.15.47:0/531121963 >> 172.21.15.47:0/531121963 conn(0x56445d932c30 msgr2=0x56445d933030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:19.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.857+0000 7fdb91200640 1 --2- 172.21.15.47:0/3939489928 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdb740a2f20 0x7fdb740a53e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdb800eead0 tx=0x7fdb800095b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:19.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.857+0000 7fdb9680f740 1 -- 172.21.15.47:0/531121963 shutdown_connections 2024-03-21T11:36:19.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.857+0000 7fdb9680f740 1 -- 172.21.15.47:0/531121963 wait complete. 2024-03-21T11:36:19.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.857+0000 7fdb9680f740 1 -- start start 2024-03-21T11:36:19.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.857+0000 7fdb9680f740 1 Processor -- start 2024-03-21T11:36:19.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.858+0000 7fdb9680f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d941900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.858+0000 7fdb9680f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9380b0 0x56445d941e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.858+0000 7fdb9680f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 0x56445d948e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.858+0000 7fdb9680f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445d940140 con 0x56445d93d220 2024-03-21T11:36:19.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.858+0000 7fdb9680f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445d854eb0 con 0x56445d9376c0 2024-03-21T11:36:19.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.858+0000 7fdb9680f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445d9402c0 con 0x56445d9380b0 2024-03-21T11:36:19.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.858+0000 7fdb9680f740 10 client.0 _ll_register_callbacks cb 0x56445d849150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:19.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.858+0000 7fdb8be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9380b0 0x56445d941e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.858+0000 7fdb8be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9380b0 0x56445d941e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43666/0 (socket says 172.21.15.47:43666) 2024-03-21T11:36:19.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.858+0000 7fdb8be00640 1 -- 172.21.15.47:0/2494373597 learned_addr learned my addr 172.21.15.47:0/2494373597 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:19.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.858+0000 7fdb91c00640 1 --2- 172.21.15.47:0/2494373597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d941900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.858+0000 7fdb91c00640 1 --2- 172.21.15.47:0/2494373597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 0x56445d948e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.859+0000 7fdb8be00640 1 -- 172.21.15.47:0/2494373597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 msgr2=0x56445d941900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:19.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.859+0000 7fdb8be00640 1 --2- 172.21.15.47:0/2494373597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d941900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.859+0000 7fdb8be00640 1 -- 172.21.15.47:0/2494373597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 msgr2=0x56445d948e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:19.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.859+0000 7fdb8be00640 1 --2- 172.21.15.47:0/2494373597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 0x56445d948e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.859+0000 7fdb8be00640 1 -- 172.21.15.47:0/2494373597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56445d6d5f70 con 0x56445d9380b0 2024-03-21T11:36:19.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.859+0000 7fdb91c00640 1 --2- 172.21.15.47:0/2494373597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d941900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:19.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.859+0000 7fdb8be00640 1 --2- 172.21.15.47:0/2494373597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9380b0 0x56445d941e40 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fdb780116d0 tx=0x7fdb78011b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1254ee6ec2904f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:19.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.859+0000 7fdb72a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:19.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.859+0000 7fdb72a00640 1 -- 172.21.15.47:0/2494373597 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7801e060 con 0x56445d9380b0 2024-03-21T11:36:19.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.859+0000 7fdb9680f740 1 -- 172.21.15.47:0/2494373597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb84003d10 con 0x56445d9380b0 2024-03-21T11:36:19.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.859+0000 7fdb72a00640 1 -- 172.21.15.47:0/2494373597 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdb7800fb90 con 0x56445d9380b0 2024-03-21T11:36:19.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.860+0000 7fdb72a00640 1 -- 172.21.15.47:0/2494373597 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb78025050 con 0x56445d9380b0 2024-03-21T11:36:19.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.860+0000 7fdb9680f740 10 client.29920 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:19.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.860+0000 7fdb9680f740 1 -- 172.21.15.47:0/2494373597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56445d56d170 con 0x56445d9380b0 2024-03-21T11:36:19.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.860+0000 7fdb72a00640 1 -- 172.21.15.47:0/2494373597 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb84003d10 con 0x56445d9380b0 2024-03-21T11:36:19.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.860+0000 7fdb72a00640 1 -- 172.21.15.47:0/2494373597 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdb78019070 con 0x56445d9380b0 2024-03-21T11:36:19.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.860+0000 7fdb9680f740 10 client.29920 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:19.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.860+0000 7fdb9680f740 10 client.29920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:19.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.860+0000 7fdb9680f740 20 client.29920 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:19.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.860+0000 7fdb9680f740 1 --2- 172.21.15.47:0/2494373597 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56445d94d860 0x56445d94fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.860+0000 7fdb9680f740 4 client.29920 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:19.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.860+0000 7fdb9680f740 1 -- 172.21.15.47:0/2494373597 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56445d8a63f0 con 0x56445d94d860 2024-03-21T11:36:19.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.861+0000 7fdb91c00640 1 --2- 172.21.15.47:0/2494373597 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56445d94d860 0x56445d94fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.861+0000 7fdb91c00640 1 --2- 172.21.15.47:0/2494373597 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56445d94d860 0x56445d94fc80 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29fb1d9e22bf1808 server_cookie=401c7c791bf1bee in_seq=0 out_seq=0 2024-03-21T11:36:19.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.861+0000 7fdb72a00640 10 client.29920 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:19.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.862+0000 7fdb72a00640 1 -- 172.21.15.47:0/2494373597 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56445d8a63f0 con 0x56445d94d860 2024-03-21T11:36:19.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.862+0000 7fdb72a00640 10 client.29920 handle_command_reply: tid=0 2024-03-21T11:36:19.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.862+0000 7fdb9680f740 1 client.29920 shutdown 2024-03-21T11:36:19.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.862+0000 7fdb9680f740 1 -- 172.21.15.47:0/2494373597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9380b0 msgr2=0x56445d941e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:19.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.862+0000 7fdb9680f740 1 --2- 172.21.15.47:0/2494373597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9380b0 0x56445d941e40 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fdb780116d0 tx=0x7fdb78011b90 comp rx=0 tx=0).stop 2024-03-21T11:36:19.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.862+0000 7fdb9680f740 1 -- 172.21.15.47:0/2494373597 shutdown_connections 2024-03-21T11:36:19.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.862+0000 7fdb9680f740 1 --2- 172.21.15.47:0/2494373597 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56445d94d860 0x56445d94fc80 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.862+0000 7fdb9680f740 1 --2- 172.21.15.47:0/2494373597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 0x56445d948e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.862+0000 7fdb9680f740 1 --2- 172.21.15.47:0/2494373597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9380b0 0x56445d941e40 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.862+0000 7fdb9680f740 1 --2- 172.21.15.47:0/2494373597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d941900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.862+0000 7fdb72a00640 0 client.29920 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:19.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.862+0000 7fdb9680f740 1 -- 172.21.15.47:0/2494373597 >> 172.21.15.47:0/2494373597 conn(0x56445d932c30 msgr2=0x56445d92da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:19.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.862+0000 7fdb9680f740 1 -- 172.21.15.47:0/2494373597 shutdown_connections 2024-03-21T11:36:19.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.862+0000 7fdb9680f740 1 -- 172.21.15.47:0/2494373597 wait complete. 2024-03-21T11:36:19.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.863+0000 7fdb9680f740 20 client.29920 trim_cache size 0 max 16384 2024-03-21T11:36:19.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.881+0000 7fdb9680f740 1 Processor -- start 2024-03-21T11:36:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.881+0000 7fdb9680f740 1 -- start start 2024-03-21T11:36:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.881+0000 7fdb9680f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d938b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb9680f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9380b0 0x56445d948e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb9680f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 0x56445d939090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb9680f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c1a7d00 con 0x56445d93d220 2024-03-21T11:36:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb9680f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c1a9680 con 0x56445d9376c0 2024-03-21T11:36:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb9680f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c1a8360 con 0x56445d9380b0 2024-03-21T11:36:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb91c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d938b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb8be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9380b0 0x56445d948e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb8be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9380b0 0x56445d948e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43676/0 (socket says 172.21.15.47:43676) 2024-03-21T11:36:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb8be00640 1 -- 172.21.15.47:0/1985903389 learned_addr learned my addr 172.21.15.47:0/1985903389 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb91c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d938b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33642/0 (socket says 172.21.15.47:33642) 2024-03-21T11:36:19.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb91c00640 1 --2- 172.21.15.47:0/1985903389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 0x56445d939090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb8be00640 1 -- 172.21.15.47:0/1985903389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 msgr2=0x56445d938b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:19.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb8be00640 1 --2- 172.21.15.47:0/1985903389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d938b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb8be00640 1 -- 172.21.15.47:0/1985903389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 msgr2=0x56445d939090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:19.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb8be00640 1 --2- 172.21.15.47:0/1985903389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 0x56445d939090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.882+0000 7fdb8be00640 1 -- 172.21.15.47:0/1985903389 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56445d938940 con 0x56445d9380b0 2024-03-21T11:36:19.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.883+0000 7fdb8be00640 1 --2- 172.21.15.47:0/1985903389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9380b0 0x56445d948e40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fdb78003da0 tx=0x7fdb780035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a234860643c6834e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:19.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.883+0000 7fdb7d600640 1 -- 172.21.15.47:0/1985903389 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb780056f0 con 0x56445d9380b0 2024-03-21T11:36:19.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.883+0000 7fdb7d600640 1 -- 172.21.15.47:0/1985903389 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb78004470 con 0x56445d9380b0 2024-03-21T11:36:19.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.883+0000 7fdb7d600640 1 -- 172.21.15.47:0/1985903389 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78003920 con 0x56445d9380b0 2024-03-21T11:36:19.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.884+0000 7fdb9680f740 1 -- 172.21.15.47:0/1985903389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9380b0 msgr2=0x56445d948e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:19.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.884+0000 7fdb9680f740 1 --2- 172.21.15.47:0/1985903389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9380b0 0x56445d948e40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fdb78003da0 tx=0x7fdb780035f0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.884+0000 7fdb9680f740 1 -- 172.21.15.47:0/1985903389 shutdown_connections 2024-03-21T11:36:19.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.884+0000 7fdb9680f740 1 --2- 172.21.15.47:0/1985903389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 0x56445d939090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.884+0000 7fdb9680f740 1 --2- 172.21.15.47:0/1985903389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9380b0 0x56445d948e40 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.884+0000 7fdb9680f740 1 --2- 172.21.15.47:0/1985903389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9376c0 0x56445d938b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.884+0000 7fdb9680f740 1 -- 172.21.15.47:0/1985903389 >> 172.21.15.47:0/1985903389 conn(0x56445d932c30 msgr2=0x56445d936720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.884+0000 7fdb9680f740 1 -- 172.21.15.47:0/1985903389 shutdown_connections 2024-03-21T11:36:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.884+0000 7fdb9680f740 1 -- 172.21.15.47:0/1985903389 wait complete. 2024-03-21T11:36:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.884+0000 7fdb9680f740 1 -- start start 2024-03-21T11:36:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.884+0000 7fdb9680f740 1 Processor -- start 2024-03-21T11:36:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.885+0000 7fdb9680f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9376c0 0x56445d948870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.885+0000 7fdb9680f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9380b0 0x56445d948db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.885+0000 7fdb9680f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 0x56445d946ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.885+0000 7fdb9680f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c1a9020 con 0x56445d93d220 2024-03-21T11:36:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.885+0000 7fdb9680f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c1a89c0 con 0x56445d9380b0 2024-03-21T11:36:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.885+0000 7fdb9680f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c1a5ec0 con 0x56445d9376c0 2024-03-21T11:36:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.885+0000 7fdb9680f740 10 client.0 _ll_register_callbacks cb 0x7fdb78009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.885+0000 7fdb8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9380b0 0x56445d948db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.885+0000 7fdb91c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9376c0 0x56445d948870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.885+0000 7fdb91c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9376c0 0x56445d948870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43684/0 (socket says 172.21.15.47:43684) 2024-03-21T11:36:19.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.885+0000 7fdb91c00640 1 -- 172.21.15.47:0/2863639137 learned_addr learned my addr 172.21.15.47:0/2863639137 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:19.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.885+0000 7fdb91c00640 1 --2- 172.21.15.47:0/2863639137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 0x56445d946ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.886+0000 7fdb91c00640 1 -- 172.21.15.47:0/2863639137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9380b0 msgr2=0x56445d948db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:19.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.886+0000 7fdb91c00640 1 --2- 172.21.15.47:0/2863639137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9380b0 0x56445d948db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.886+0000 7fdb91c00640 1 -- 172.21.15.47:0/2863639137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 msgr2=0x56445d946ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:19.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.886+0000 7fdb91c00640 1 --2- 172.21.15.47:0/2863639137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 0x56445d946ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.886+0000 7fdb91c00640 1 -- 172.21.15.47:0/2863639137 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56445d9476d0 con 0x56445d9376c0 2024-03-21T11:36:19.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.886+0000 7fdb8be00640 1 --2- 172.21.15.47:0/2863639137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9380b0 0x56445d948db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:19.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.886+0000 7fdb91c00640 1 --2- 172.21.15.47:0/2863639137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9376c0 0x56445d948870 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fdb8402a480 tx=0x7fdb8401d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7824541bd18c6793 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.886+0000 7fdb72a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.886+0000 7fdb72a00640 1 -- 172.21.15.47:0/2863639137 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84024040 con 0x56445d9376c0 2024-03-21T11:36:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.886+0000 7fdb72a00640 1 -- 172.21.15.47:0/2863639137 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdb84011840 con 0x56445d9376c0 2024-03-21T11:36:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.886+0000 7fdb9680f740 1 -- 172.21.15.47:0/2863639137 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb78004470 con 0x56445d9376c0 2024-03-21T11:36:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.886+0000 7fdb72a00640 1 -- 172.21.15.47:0/2863639137 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb84011a40 con 0x56445d9376c0 2024-03-21T11:36:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.886+0000 7fdb9680f740 10 client.29926 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.886+0000 7fdb9680f740 1 -- 172.21.15.47:0/2863639137 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56445d9a9c90 con 0x56445d9376c0 2024-03-21T11:36:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.887+0000 7fdb72a00640 1 -- 172.21.15.47:0/2863639137 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb78004470 con 0x56445d9376c0 2024-03-21T11:36:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.887+0000 7fdb72a00640 1 -- 172.21.15.47:0/2863639137 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdb8402a040 con 0x56445d9376c0 2024-03-21T11:36:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.887+0000 7fdb9680f740 10 client.29926 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.887+0000 7fdb9680f740 10 client.29926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.887+0000 7fdb9680f740 20 client.29926 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.887+0000 7fdb9680f740 1 --2- 172.21.15.47:0/2863639137 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56445d9aa2f0 0x56445d9ac6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:19.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.887+0000 7fdb9680f740 4 client.29926 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:36:19.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.887+0000 7fdb9680f740 1 -- 172.21.15.47:0/2863639137 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56445d8a0740 con 0x56445d9aa2f0 2024-03-21T11:36:19.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.887+0000 7fdb8be00640 1 --2- 172.21.15.47:0/2863639137 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56445d9aa2f0 0x56445d9ac6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:19.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.888+0000 7fdb8be00640 1 --2- 172.21.15.47:0/2863639137 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56445d9aa2f0 0x56445d9ac6e0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52ed68d4f546a83b server_cookie=eabbce4b74de073d in_seq=0 out_seq=0 2024-03-21T11:36:19.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.888+0000 7fdb72a00640 10 client.29926 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:19.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.888+0000 7fdb72a00640 1 -- 172.21.15.47:0/2863639137 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56445d8a0740 con 0x56445d9aa2f0 2024-03-21T11:36:19.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.889+0000 7fdb72a00640 10 client.29926 handle_command_reply: tid=0 2024-03-21T11:36:19.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.889+0000 7fdb9680f740 1 client.29926 shutdown 2024-03-21T11:36:19.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.889+0000 7fdb9680f740 1 -- 172.21.15.47:0/2863639137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9376c0 msgr2=0x56445d948870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:19.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.889+0000 7fdb9680f740 1 --2- 172.21.15.47:0/2863639137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9376c0 0x56445d948870 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fdb8402a480 tx=0x7fdb8401d040 comp rx=0 tx=0).stop 2024-03-21T11:36:19.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.889+0000 7fdb9680f740 1 -- 172.21.15.47:0/2863639137 shutdown_connections 2024-03-21T11:36:19.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.889+0000 7fdb9680f740 1 --2- 172.21.15.47:0/2863639137 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56445d9aa2f0 0x56445d9ac6e0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.889+0000 7fdb9680f740 1 --2- 172.21.15.47:0/2863639137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56445d93d220 0x56445d946ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.889+0000 7fdb9680f740 1 --2- 172.21.15.47:0/2863639137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56445d9380b0 0x56445d948db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.889+0000 7fdb9680f740 1 --2- 172.21.15.47:0/2863639137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56445d9376c0 0x56445d948870 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.889+0000 7fdb9680f740 1 -- 172.21.15.47:0/2863639137 >> 172.21.15.47:0/2863639137 conn(0x56445d932c30 msgr2=0x56445d92d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:19.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.889+0000 7fdb72a00640 0 client.29926 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:19.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.889+0000 7fdb9680f740 1 -- 172.21.15.47:0/2863639137 shutdown_connections 2024-03-21T11:36:19.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.889+0000 7fdb9680f740 1 -- 172.21.15.47:0/2863639137 wait complete. 2024-03-21T11:36:19.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.889+0000 7fdb9680f740 20 client.29926 trim_cache size 0 max 16384 2024-03-21T11:36:19.894 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:36:19.894 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:36:19.895 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "1bb9c2fb-9bad-42a3-8780-030156e27250", 2024-03-21T11:36:19.895 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:36:19.895 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:36:19.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.890+0000 7fdb92600640 1 -- 172.21.15.47:0/3939489928 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdb740a2f20 msgr2=0x7fdb740a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:19.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.890+0000 7fdb92600640 1 --2- 172.21.15.47:0/3939489928 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdb740a2f20 0x7fdb740a53e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdb800eead0 tx=0x7fdb800095b0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.890+0000 7fdb92600640 1 -- 172.21.15.47:0/3939489928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c1015a0 msgr2=0x7fdb8c1978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:19.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.890+0000 7fdb92600640 1 --2- 172.21.15.47:0/3939489928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c1015a0 0x7fdb8c1978d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fdb800349e0 tx=0x7fdb80032430 comp rx=0 tx=0).stop 2024-03-21T11:36:19.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.890+0000 7fdb92600640 1 -- 172.21.15.47:0/3939489928 shutdown_connections 2024-03-21T11:36:19.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.890+0000 7fdb92600640 1 --2- 172.21.15.47:0/3939489928 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdb740a2f20 0x7fdb740a53e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.890+0000 7fdb92600640 1 --2- 172.21.15.47:0/3939489928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb8c103150 0x7fdb8c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.890+0000 7fdb92600640 1 --2- 172.21.15.47:0/3939489928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb8c102790 0x7fdb8c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.890+0000 7fdb92600640 1 --2- 172.21.15.47:0/3939489928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c1015a0 0x7fdb8c1978d0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:19.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.890+0000 7fdb92600640 1 -- 172.21.15.47:0/3939489928 >> 172.21.15.47:0/3939489928 conn(0x7fdb8c0fcb10 msgr2=0x7fdb8c105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:19.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.890+0000 7fdb92600640 1 -- 172.21.15.47:0/3939489928 shutdown_connections 2024-03-21T11:36:19.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:19.890+0000 7fdb92600640 1 -- 172.21.15.47:0/3939489928 wait complete. 2024-03-21T11:36:19.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:36:19.910 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:36:20.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.131+0000 7fbeb1400640 1 -- 172.21.15.47:0/2695060851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeac1038c0 msgr2=0x7fbeac103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:20.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.132+0000 7fbeb1400640 1 --2- 172.21.15.47:0/2695060851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeac1038c0 0x7fbeac103ce0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009ca0 tx=0x7fbe9c02c0f0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.132+0000 7fbeb1400640 1 -- 172.21.15.47:0/2695060851 shutdown_connections 2024-03-21T11:36:20.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.132+0000 7fbeb1400640 1 --2- 172.21.15.47:0/2695060851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac105470 0x7fbeac10bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.132+0000 7fbeb1400640 1 --2- 172.21.15.47:0/2695060851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeac104ab0 0x7fbeac104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.132+0000 7fbeb1400640 1 --2- 172.21.15.47:0/2695060851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeac1038c0 0x7fbeac103ce0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.132+0000 7fbeb1400640 1 -- 172.21.15.47:0/2695060851 >> 172.21.15.47:0/2695060851 conn(0x7fbeac0fee50 msgr2=0x7fbeac101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:20.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.132+0000 7fbeb1400640 1 -- 172.21.15.47:0/2695060851 shutdown_connections 2024-03-21T11:36:20.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.132+0000 7fbeb1400640 1 -- 172.21.15.47:0/2695060851 wait complete. 2024-03-21T11:36:20.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.133+0000 7fbeb1400640 1 Processor -- start 2024-03-21T11:36:20.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.133+0000 7fbeb1400640 1 -- start start 2024-03-21T11:36:20.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.133+0000 7fbeb1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac1038c0 0x7fbeac199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.133+0000 7fbeb1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeac104ab0 0x7fbeac19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.133+0000 7fbeb1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeac105470 0x7fbeac1a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.133+0000 7fbeb1400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac10f530 con 0x7fbeac1038c0 2024-03-21T11:36:20.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.133+0000 7fbeb1400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac10f3b0 con 0x7fbeac104ab0 2024-03-21T11:36:20.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.133+0000 7fbeb1400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac10f6b0 con 0x7fbeac105470 2024-03-21T11:36:20.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.134+0000 7fbeab200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeac105470 0x7fbeac1a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.134+0000 7fbea9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeac104ab0 0x7fbeac19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.134+0000 7fbeaa800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac1038c0 0x7fbeac199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.134+0000 7fbeab200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeac105470 0x7fbeac1a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43700/0 (socket says 172.21.15.47:43700) 2024-03-21T11:36:20.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.134+0000 7fbeab200640 1 -- 172.21.15.47:0/2836816468 learned_addr learned my addr 172.21.15.47:0/2836816468 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:20.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.134+0000 7fbea9e00640 1 -- 172.21.15.47:0/2836816468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeac105470 msgr2=0x7fbeac1a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:20.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.134+0000 7fbea9e00640 1 --2- 172.21.15.47:0/2836816468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeac105470 0x7fbeac1a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.134+0000 7fbea9e00640 1 -- 172.21.15.47:0/2836816468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac1038c0 msgr2=0x7fbeac199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:20.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.134+0000 7fbea9e00640 1 --2- 172.21.15.47:0/2836816468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac1038c0 0x7fbeac199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.134+0000 7fbea9e00640 1 -- 172.21.15.47:0/2836816468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac1a1890 con 0x7fbeac104ab0 2024-03-21T11:36:20.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.134+0000 7fbeaa800640 1 --2- 172.21.15.47:0/2836816468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac1038c0 0x7fbeac199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:20.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.135+0000 7fbea9e00640 1 --2- 172.21.15.47:0/2836816468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeac104ab0 0x7fbeac19a1c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400e9f0 tx=0x7fbe9400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:20.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.135+0000 7fbea3400640 1 -- 172.21.15.47:0/2836816468 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9400ccc0 con 0x7fbeac104ab0 2024-03-21T11:36:20.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.135+0000 7fbeb1400640 1 -- 172.21.15.47:0/2836816468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeac1a1b80 con 0x7fbeac104ab0 2024-03-21T11:36:20.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.135+0000 7fbea3400640 1 -- 172.21.15.47:0/2836816468 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe94005020 con 0x7fbeac104ab0 2024-03-21T11:36:20.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.135+0000 7fbea3400640 1 -- 172.21.15.47:0/2836816468 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe940049e0 con 0x7fbeac104ab0 2024-03-21T11:36:20.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.135+0000 7fbeb1400640 1 -- 172.21.15.47:0/2836816468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeac1a20a0 con 0x7fbeac104ab0 2024-03-21T11:36:20.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.139+0000 7fbea3400640 1 -- 172.21.15.47:0/2836816468 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbe94005560 con 0x7fbeac104ab0 2024-03-21T11:36:20.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.139+0000 7fbeb41f6740 1 Processor -- start 2024-03-21T11:36:20.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.139+0000 7fbeb41f6740 1 -- start start 2024-03-21T11:36:20.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.139+0000 7fbeb41f6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70ee80 0x56031c70f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.139+0000 7fbeb41f6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70f870 0x56031c714370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.139+0000 7fbeb41f6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c7149e0 0x56031c716e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.139+0000 7fbeb41f6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031c674250 con 0x56031c7149e0 2024-03-21T11:36:20.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.139+0000 7fbeb41f6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031c61ca30 con 0x56031c70f870 2024-03-21T11:36:20.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.139+0000 7fbeb41f6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031c6702f0 con 0x56031c70ee80 2024-03-21T11:36:20.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.140+0000 7fbeab200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70ee80 0x56031c70f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.140+0000 7fbeab200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70ee80 0x56031c70f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43704/0 (socket says 172.21.15.47:43704) 2024-03-21T11:36:20.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.140+0000 7fbea9e00640 1 --2- 172.21.15.47:0/3376439428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c7149e0 0x56031c716e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.140+0000 7fbeab200640 1 -- 172.21.15.47:0/3376439428 learned_addr learned my addr 172.21.15.47:0/3376439428 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:20.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.140+0000 7fbeaa800640 1 --2- 172.21.15.47:0/3376439428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70f870 0x56031c714370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.140+0000 7fbeab200640 1 -- 172.21.15.47:0/3376439428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70f870 msgr2=0x56031c714370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:20.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.140+0000 7fbeab200640 1 --2- 172.21.15.47:0/3376439428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70f870 0x56031c714370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.140+0000 7fbeab200640 1 -- 172.21.15.47:0/3376439428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c7149e0 msgr2=0x56031c716e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:20.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.140+0000 7fbeab200640 1 --2- 172.21.15.47:0/3376439428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c7149e0 0x56031c716e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.140+0000 7fbeab200640 1 -- 172.21.15.47:0/3376439428 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56031c607690 con 0x56031c70ee80 2024-03-21T11:36:20.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.141+0000 7fbeab200640 1 --2- 172.21.15.47:0/3376439428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70ee80 0x56031c70f2a0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fbea400bd80 tx=0x7fbea40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a13640c55cf68a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:20.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.141+0000 7fbea3400640 1 --2- 172.21.15.47:0/2836816468 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe900a2f20 0x7fbe900a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.141+0000 7fbe9b400640 1 -- 172.21.15.47:0/3376439428 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4011640 con 0x56031c70ee80 2024-03-21T11:36:20.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.141+0000 7fbe9b400640 1 -- 172.21.15.47:0/3376439428 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbea400be70 con 0x56031c70ee80 2024-03-21T11:36:20.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.141+0000 7fbe9b400640 1 -- 172.21.15.47:0/3376439428 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4019c30 con 0x56031c70ee80 2024-03-21T11:36:20.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.141+0000 7fbea3400640 1 -- 172.21.15.47:0/2836816468 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbe940cf7f0 con 0x7fbeac104ab0 2024-03-21T11:36:20.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.141+0000 7fbeaa800640 1 --2- 172.21.15.47:0/2836816468 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe900a2f20 0x7fbe900a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.142+0000 7fbeb41f6740 1 -- 172.21.15.47:0/3376439428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70ee80 msgr2=0x56031c70f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:20.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.142+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/3376439428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70ee80 0x56031c70f2a0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fbea400bd80 tx=0x7fbea40107a0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.142+0000 7fbeaa800640 1 --2- 172.21.15.47:0/2836816468 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe900a2f20 0x7fbe900a53e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c02c630 tx=0x7fbe9c009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:20.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.142+0000 7fbeb41f6740 1 -- 172.21.15.47:0/3376439428 shutdown_connections 2024-03-21T11:36:20.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.142+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/3376439428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c7149e0 0x56031c716e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.142+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/3376439428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70f870 0x56031c714370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.142+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/3376439428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70ee80 0x56031c70f2a0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.142+0000 7fbeb41f6740 1 -- 172.21.15.47:0/3376439428 >> 172.21.15.47:0/3376439428 conn(0x56031c6d7200 msgr2=0x56031c70a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:20.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.142+0000 7fbeb41f6740 1 -- 172.21.15.47:0/3376439428 shutdown_connections 2024-03-21T11:36:20.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.142+0000 7fbeb41f6740 1 -- 172.21.15.47:0/3376439428 wait complete. 2024-03-21T11:36:20.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.143+0000 7fbeb41f6740 1 -- start start 2024-03-21T11:36:20.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.143+0000 7fbeb41f6740 1 Processor -- start 2024-03-21T11:36:20.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbeb41f6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70ee80 0x56031c718ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbeb41f6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c70f870 0x56031c719430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbeb41f6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c7149e0 0x56031c720400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbeb41f6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031c717990 con 0x56031c70f870 2024-03-21T11:36:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbeb41f6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031c62fe80 con 0x56031c7149e0 2024-03-21T11:36:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbeb41f6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031c717b10 con 0x56031c70ee80 2024-03-21T11:36:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbeb41f6740 10 client.0 _ll_register_callbacks cb 0x56031c642130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbea9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c70f870 0x56031c719430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbeab200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c7149e0 0x56031c720400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbeaa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70ee80 0x56031c718ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbeaa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70ee80 0x56031c718ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43708/0 (socket says 172.21.15.47:43708) 2024-03-21T11:36:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbeaa800640 1 -- 172.21.15.47:0/1763322705 learned_addr learned my addr 172.21.15.47:0/1763322705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbea9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c70f870 0x56031c719430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48862/0 (socket says 172.21.15.47:48862) 2024-03-21T11:36:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbeab200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c7149e0 0x56031c720400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33676/0 (socket says 172.21.15.47:33676) 2024-03-21T11:36:20.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbea9e00640 1 -- 172.21.15.47:0/1763322705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70ee80 msgr2=0x56031c718ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:20.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbea9e00640 1 --2- 172.21.15.47:0/1763322705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70ee80 0x56031c718ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.144+0000 7fbea9e00640 1 -- 172.21.15.47:0/1763322705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c7149e0 msgr2=0x56031c720400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:20.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.145+0000 7fbea9e00640 1 --2- 172.21.15.47:0/1763322705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c7149e0 0x56031c720400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.145+0000 7fbea9e00640 1 -- 172.21.15.47:0/1763322705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56031c4527d0 con 0x56031c70f870 2024-03-21T11:36:20.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.145+0000 7fbeaa800640 1 --2- 172.21.15.47:0/1763322705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70ee80 0x56031c718ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:20.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.145+0000 7fbeab200640 1 --2- 172.21.15.47:0/1763322705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c7149e0 0x56031c720400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:20.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.145+0000 7fbea9e00640 1 --2- 172.21.15.47:0/1763322705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c70f870 0x56031c719430 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400e9c0 tx=0x7fbe9409b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed77dff2fc43daf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:20.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.145+0000 7fbe8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:36:20.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.145+0000 7fbe8fe00640 1 -- 172.21.15.47:0/1763322705 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9409ce60 con 0x56031c70f870 2024-03-21T11:36:20.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.145+0000 7fbe8fe00640 1 -- 172.21.15.47:0/1763322705 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbe9409bd90 con 0x56031c70f870 2024-03-21T11:36:20.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.145+0000 7fbeb41f6740 1 -- 172.21.15.47:0/1763322705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbea400be70 con 0x56031c70f870 2024-03-21T11:36:20.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.145+0000 7fbe8fe00640 1 -- 172.21.15.47:0/1763322705 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe940b4e60 con 0x56031c70f870 2024-03-21T11:36:20.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.146+0000 7fbeb41f6740 10 client.19512 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:20.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.146+0000 7fbeb41f6740 1 -- 172.21.15.47:0/1763322705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56031c3883f0 con 0x56031c70f870 2024-03-21T11:36:20.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.146+0000 7fbe8fe00640 1 -- 172.21.15.47:0/1763322705 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe940b1020 con 0x56031c70f870 2024-03-21T11:36:20.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.146+0000 7fbe8fe00640 1 -- 172.21.15.47:0/1763322705 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbe940a40b0 con 0x56031c70f870 2024-03-21T11:36:20.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.146+0000 7fbeb41f6740 10 client.19512 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:20.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.146+0000 7fbeb41f6740 10 client.19512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:20.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.146+0000 7fbeb41f6740 20 client.19512 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:20.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.146+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/1763322705 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56031c724da0 0x56031c727190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.146+0000 7fbeb41f6740 4 client.19512 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:20.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.146+0000 7fbeb41f6740 1 -- 172.21.15.47:0/1763322705 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56031c681790 con 0x56031c724da0 2024-03-21T11:36:20.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.146+0000 7fbeaa800640 1 --2- 172.21.15.47:0/1763322705 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56031c724da0 0x56031c727190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.147+0000 7fbeaa800640 1 --2- 172.21.15.47:0/1763322705 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56031c724da0 0x56031c727190 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=212eecd19ac04e65 server_cookie=f514fedb167f89d5 in_seq=0 out_seq=0 2024-03-21T11:36:20.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.147+0000 7fbe8fe00640 10 client.19512 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:20.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbe8fe00640 1 -- 172.21.15.47:0/1763322705 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56031c681790 con 0x56031c724da0 2024-03-21T11:36:20.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbe8fe00640 10 client.19512 handle_command_reply: tid=0 2024-03-21T11:36:20.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbeb41f6740 1 client.19512 shutdown 2024-03-21T11:36:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbeb41f6740 1 -- 172.21.15.47:0/1763322705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c70f870 msgr2=0x56031c719430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/1763322705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c70f870 0x56031c719430 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400e9c0 tx=0x7fbe9409b900 comp rx=0 tx=0).stop 2024-03-21T11:36:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbeb41f6740 1 -- 172.21.15.47:0/1763322705 shutdown_connections 2024-03-21T11:36:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/1763322705 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56031c724da0 0x56031c727190 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/1763322705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c7149e0 0x56031c720400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/1763322705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c70f870 0x56031c719430 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/1763322705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70ee80 0x56031c718ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbe8fe00640 0 client.19512 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbeb41f6740 1 -- 172.21.15.47:0/1763322705 >> 172.21.15.47:0/1763322705 conn(0x56031c6d7200 msgr2=0x56031c709a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbeb41f6740 1 -- 172.21.15.47:0/1763322705 shutdown_connections 2024-03-21T11:36:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbeb41f6740 1 -- 172.21.15.47:0/1763322705 wait complete. 2024-03-21T11:36:20.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.150+0000 7fbeb41f6740 20 client.19512 trim_cache size 0 max 16384 2024-03-21T11:36:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.174+0000 7fbeb41f6740 1 Processor -- start 2024-03-21T11:36:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.174+0000 7fbeb41f6740 1 -- start start 2024-03-21T11:36:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.174+0000 7fbeb41f6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c6d7200 0x56031c70e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.174+0000 7fbeb41f6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70ee80 0x56031c7203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.174+0000 7fbeb41f6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70f870 0x56031c72ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.174+0000 7fbeb41f6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac1ac270 con 0x56031c6d7200 2024-03-21T11:36:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.174+0000 7fbeb41f6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac1adbf0 con 0x56031c70ee80 2024-03-21T11:36:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.174+0000 7fbeb41f6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac1ac8d0 con 0x56031c70f870 2024-03-21T11:36:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.175+0000 7fbea9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70ee80 0x56031c7203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.175+0000 7fbeab200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70f870 0x56031c72ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.175+0000 7fbea9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70ee80 0x56031c7203c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33678/0 (socket says 172.21.15.47:33678) 2024-03-21T11:36:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.175+0000 7fbea9e00640 1 -- 172.21.15.47:0/2339230986 learned_addr learned my addr 172.21.15.47:0/2339230986 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.175+0000 7fbeaa800640 1 --2- 172.21.15.47:0/2339230986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c6d7200 0x56031c70e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.175+0000 7fbea9e00640 1 -- 172.21.15.47:0/2339230986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70f870 msgr2=0x56031c72ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.175+0000 7fbea9e00640 1 --2- 172.21.15.47:0/2339230986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70f870 0x56031c72ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.175+0000 7fbea9e00640 1 -- 172.21.15.47:0/2339230986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c6d7200 msgr2=0x56031c70e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.175+0000 7fbea9e00640 1 --2- 172.21.15.47:0/2339230986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c6d7200 0x56031c70e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.175+0000 7fbea9e00640 1 -- 172.21.15.47:0/2339230986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56031c3480f0 con 0x56031c70ee80 2024-03-21T11:36:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.175+0000 7fbeab200640 1 --2- 172.21.15.47:0/2339230986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70f870 0x56031c72ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.175+0000 7fbea9e00640 1 --2- 172.21.15.47:0/2339230986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70ee80 0x56031c7203c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbe94098420 tx=0x7fbe9409ba30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc32879cc2045f36 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.175+0000 7fbe9aa00640 1 -- 172.21.15.47:0/2339230986 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9409ce60 con 0x56031c70ee80 2024-03-21T11:36:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.176+0000 7fbe9aa00640 1 -- 172.21.15.47:0/2339230986 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe940b1040 con 0x56031c70ee80 2024-03-21T11:36:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.176+0000 7fbe9aa00640 1 -- 172.21.15.47:0/2339230986 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9401c070 con 0x56031c70ee80 2024-03-21T11:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.176+0000 7fbeb41f6740 1 -- 172.21.15.47:0/2339230986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70ee80 msgr2=0x56031c7203c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.176+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/2339230986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70ee80 0x56031c7203c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbe94098420 tx=0x7fbe9409ba30 comp rx=0 tx=0).stop 2024-03-21T11:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.176+0000 7fbeb41f6740 1 -- 172.21.15.47:0/2339230986 shutdown_connections 2024-03-21T11:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.176+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/2339230986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c70f870 0x56031c72ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.176+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/2339230986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70ee80 0x56031c7203c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.176+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/2339230986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c6d7200 0x56031c70e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.176+0000 7fbeb41f6740 1 -- 172.21.15.47:0/2339230986 >> 172.21.15.47:0/2339230986 conn(0x56031c718ab0 msgr2=0x56031c70da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.177+0000 7fbeb41f6740 1 -- 172.21.15.47:0/2339230986 shutdown_connections 2024-03-21T11:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.177+0000 7fbeb41f6740 1 -- 172.21.15.47:0/2339230986 wait complete. 2024-03-21T11:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.177+0000 7fbeb41f6740 1 -- start start 2024-03-21T11:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.177+0000 7fbeb41f6740 1 Processor -- start 2024-03-21T11:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeb41f6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c6d7200 0x56031c71dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeb41f6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c70ee80 0x56031c71e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeb41f6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70f870 0x56031c71f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeb41f6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac1ad590 con 0x56031c70ee80 2024-03-21T11:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeb41f6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac1acf30 con 0x56031c70f870 2024-03-21T11:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeb41f6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac1aa430 con 0x56031c6d7200 2024-03-21T11:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeb41f6740 10 client.0 _ll_register_callbacks cb 0x7fbe940d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeaa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c6d7200 0x56031c71dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeab200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70f870 0x56031c71f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeaa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c6d7200 0x56031c71dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43720/0 (socket says 172.21.15.47:43720) 2024-03-21T11:36:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeaa800640 1 -- 172.21.15.47:0/3287916500 learned_addr learned my addr 172.21.15.47:0/3287916500 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbea9e00640 1 --2- 172.21.15.47:0/3287916500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c70ee80 0x56031c71e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeaa800640 1 -- 172.21.15.47:0/3287916500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70f870 msgr2=0x56031c71f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeaa800640 1 --2- 172.21.15.47:0/3287916500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70f870 0x56031c71f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeaa800640 1 -- 172.21.15.47:0/3287916500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c70ee80 msgr2=0x56031c71e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeaa800640 1 --2- 172.21.15.47:0/3287916500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c70ee80 0x56031c71e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.178+0000 7fbeaa800640 1 -- 172.21.15.47:0/3287916500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56031c71f7d0 con 0x56031c6d7200 2024-03-21T11:36:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.179+0000 7fbeaa800640 1 --2- 172.21.15.47:0/3287916500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c6d7200 0x56031c71dce0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c037040 tx=0x7fbe9c02c0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78b5c6ffe9624a10 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.179+0000 7fbe8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:20.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.179+0000 7fbe8fe00640 1 -- 172.21.15.47:0/3287916500 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c039040 con 0x56031c6d7200 2024-03-21T11:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.179+0000 7fbeb41f6740 1 -- 172.21.15.47:0/3287916500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe940b1040 con 0x56031c6d7200 2024-03-21T11:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.179+0000 7fbe8fe00640 1 -- 172.21.15.47:0/3287916500 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbe9c0531d0 con 0x56031c6d7200 2024-03-21T11:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.179+0000 7fbe8fe00640 1 -- 172.21.15.47:0/3287916500 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe9c062050 con 0x56031c6d7200 2024-03-21T11:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.179+0000 7fbeb41f6740 10 client.29944 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.179+0000 7fbeb41f6740 1 -- 172.21.15.47:0/3287916500 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56031c780760 con 0x56031c6d7200 2024-03-21T11:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.180+0000 7fbe8fe00640 1 -- 172.21.15.47:0/3287916500 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe940b1040 con 0x56031c6d7200 2024-03-21T11:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.180+0000 7fbe8fe00640 1 -- 172.21.15.47:0/3287916500 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbe9c034b50 con 0x56031c6d7200 2024-03-21T11:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.180+0000 7fbeb41f6740 10 client.29944 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.180+0000 7fbeb41f6740 10 client.29944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.180+0000 7fbeb41f6740 20 client.29944 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.180+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/3287916500 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56031c7149e0 0x56031c783150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.180+0000 7fbeb41f6740 4 client.29944 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.180+0000 7fbeb41f6740 1 -- 172.21.15.47:0/3287916500 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56031c67bae0 con 0x56031c7149e0 2024-03-21T11:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.180+0000 7fbea9e00640 1 --2- 172.21.15.47:0/3287916500 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56031c7149e0 0x56031c783150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.181+0000 7fbea9e00640 1 --2- 172.21.15.47:0/3287916500 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56031c7149e0 0x56031c783150 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a01e70abdc92f93 server_cookie=f5b5d0671036c0f2 in_seq=0 out_seq=0 2024-03-21T11:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.181+0000 7fbe8fe00640 10 client.29944 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.181+0000 7fbe8fe00640 1 -- 172.21.15.47:0/3287916500 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56031c67bae0 con 0x56031c7149e0 2024-03-21T11:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.181+0000 7fbe8fe00640 10 client.29944 handle_command_reply: tid=0 2024-03-21T11:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.181+0000 7fbeb41f6740 1 client.29944 shutdown 2024-03-21T11:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.181+0000 7fbeb41f6740 1 -- 172.21.15.47:0/3287916500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c6d7200 msgr2=0x56031c71dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.181+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/3287916500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c6d7200 0x56031c71dce0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c037040 tx=0x7fbe9c02c0f0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.182+0000 7fbeb41f6740 1 -- 172.21.15.47:0/3287916500 shutdown_connections 2024-03-21T11:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.182+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/3287916500 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56031c7149e0 0x56031c783150 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.182+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/3287916500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56031c70f870 0x56031c71f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.182+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/3287916500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56031c70ee80 0x56031c71e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.182+0000 7fbe8fe00640 0 client.29944 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.182+0000 7fbeb41f6740 1 --2- 172.21.15.47:0/3287916500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56031c6d7200 0x56031c71dce0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.182+0000 7fbeb41f6740 1 -- 172.21.15.47:0/3287916500 >> 172.21.15.47:0/3287916500 conn(0x56031c718ab0 msgr2=0x56031c70e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.182+0000 7fbeb41f6740 1 -- 172.21.15.47:0/3287916500 shutdown_connections 2024-03-21T11:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.182+0000 7fbeb41f6740 1 -- 172.21.15.47:0/3287916500 wait complete. 2024-03-21T11:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.182+0000 7fbeb41f6740 20 client.29944 trim_cache size 0 max 16384 2024-03-21T11:36:20.188 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:36:20.188 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:36:20.188 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:36:20.188 INFO:teuthology.orchestra.run.smithi047.stdout: "1bb9c2fb-9bad-42a3-8780-030156e27250": { 2024-03-21T11:36:20.188 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:36:20.188 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "1bb9c2fb-9bad-42a3-8780-030156e27250", 2024-03-21T11:36:20.188 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:36:20.188 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:36:20.188 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:36:20.188 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:36:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.183+0000 7fbeb1400640 1 -- 172.21.15.47:0/2836816468 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe900a2f20 msgr2=0x7fbe900a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.183+0000 7fbeb1400640 1 --2- 172.21.15.47:0/2836816468 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe900a2f20 0x7fbe900a53e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c02c630 tx=0x7fbe9c009360 comp rx=0 tx=0).stop 2024-03-21T11:36:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.183+0000 7fbeb1400640 1 -- 172.21.15.47:0/2836816468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeac104ab0 msgr2=0x7fbeac19a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.183+0000 7fbeb1400640 1 --2- 172.21.15.47:0/2836816468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeac104ab0 0x7fbeac19a1c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400e9f0 tx=0x7fbe9400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.183+0000 7fbeb1400640 1 -- 172.21.15.47:0/2836816468 shutdown_connections 2024-03-21T11:36:20.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.183+0000 7fbeb1400640 1 --2- 172.21.15.47:0/2836816468 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe900a2f20 0x7fbe900a53e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.183+0000 7fbeb1400640 1 --2- 172.21.15.47:0/2836816468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeac105470 0x7fbeac1a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.183+0000 7fbeb1400640 1 --2- 172.21.15.47:0/2836816468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbeac104ab0 0x7fbeac19a1c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.183+0000 7fbeb1400640 1 --2- 172.21.15.47:0/2836816468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbeac1038c0 0x7fbeac199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:20.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.183+0000 7fbeb1400640 1 -- 172.21.15.47:0/2836816468 >> 172.21.15.47:0/2836816468 conn(0x7fbeac0fee50 msgr2=0x7fbeac107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:20.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.183+0000 7fbeb1400640 1 -- 172.21.15.47:0/2836816468 shutdown_connections 2024-03-21T11:36:20.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:20.183+0000 7fbeb1400640 1 -- 172.21.15.47:0/2836816468 wait complete. 2024-03-21T11:36:20.201 INFO:tasks.cephfs.filesystem:scrub status for tag:1bb9c2fb-9bad-42a3-8780-030156e27250 - {'path': '/', 'tag': '1bb9c2fb-9bad-42a3-8780-030156e27250', 'options': 'recursive,force'} 2024-03-21T11:36:20.579 INFO:tasks.workunit.client.0.smithi047.stdout: 1 990583 7.94 MB/sec execute 544 sec latency 16.057 ms 2024-03-21T11:36:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.019+0000 7fbe50c00640 1 -- 172.21.15.47:0/3420907946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe4c103900 msgr2=0x7fbe4c103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.019+0000 7fbe50c00640 1 --2- 172.21.15.47:0/3420907946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe4c103900 0x7fbe4c103d20 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009900 tx=0x7fbe3c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:36:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.019+0000 7fbe50c00640 1 -- 172.21.15.47:0/3420907946 shutdown_connections 2024-03-21T11:36:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.019+0000 7fbe50c00640 1 --2- 172.21.15.47:0/3420907946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c1054b0 0x7fbe4c10bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.019+0000 7fbe50c00640 1 --2- 172.21.15.47:0/3420907946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe4c104af0 0x7fbe4c104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.019+0000 7fbe50c00640 1 --2- 172.21.15.47:0/3420907946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe4c103900 0x7fbe4c103d20 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.019+0000 7fbe50c00640 1 -- 172.21.15.47:0/3420907946 >> 172.21.15.47:0/3420907946 conn(0x7fbe4c0fee90 msgr2=0x7fbe4c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.019+0000 7fbe50c00640 1 -- 172.21.15.47:0/3420907946 shutdown_connections 2024-03-21T11:36:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.019+0000 7fbe50c00640 1 -- 172.21.15.47:0/3420907946 wait complete. 2024-03-21T11:36:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.020+0000 7fbe50c00640 1 Processor -- start 2024-03-21T11:36:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.021+0000 7fbe50c00640 1 -- start start 2024-03-21T11:36:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.021+0000 7fbe50c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c103900 0x7fbe4c199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.021+0000 7fbe50c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe4c104af0 0x7fbe4c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.021+0000 7fbe50c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe4c1054b0 0x7fbe4c1a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.021+0000 7fbe50c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c10f5a0 con 0x7fbe4c1054b0 2024-03-21T11:36:21.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.021+0000 7fbe50c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c10f420 con 0x7fbe4c103900 2024-03-21T11:36:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.021+0000 7fbe50c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c10f720 con 0x7fbe4c104af0 2024-03-21T11:36:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.021+0000 7fbe49e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c103900 0x7fbe4c199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.021+0000 7fbe49e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c103900 0x7fbe4c199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33708/0 (socket says 172.21.15.47:33708) 2024-03-21T11:36:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.021+0000 7fbe49e00640 1 -- 172.21.15.47:0/3681770612 learned_addr learned my addr 172.21.15.47:0/3681770612 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.021+0000 7fbe49400640 1 --2- 172.21.15.47:0/3681770612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe4c104af0 0x7fbe4c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.021+0000 7fbe4a800640 1 --2- 172.21.15.47:0/3681770612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe4c1054b0 0x7fbe4c1a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.022+0000 7fbe49e00640 1 -- 172.21.15.47:0/3681770612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe4c104af0 msgr2=0x7fbe4c19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.022+0000 7fbe49e00640 1 --2- 172.21.15.47:0/3681770612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe4c104af0 0x7fbe4c19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.022+0000 7fbe49e00640 1 -- 172.21.15.47:0/3681770612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe4c1054b0 msgr2=0x7fbe4c1a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.022+0000 7fbe49e00640 1 --2- 172.21.15.47:0/3681770612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe4c1054b0 0x7fbe4c1a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.022+0000 7fbe49e00640 1 -- 172.21.15.47:0/3681770612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c1a17a0 con 0x7fbe4c103900 2024-03-21T11:36:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.022+0000 7fbe49e00640 1 --2- 172.21.15.47:0/3681770612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c103900 0x7fbe4c199b80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c000c00 tx=0x7fbe3c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.022+0000 7fbe42a00640 1 -- 172.21.15.47:0/3681770612 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c043020 con 0x7fbe4c103900 2024-03-21T11:36:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.023+0000 7fbe42a00640 1 -- 172.21.15.47:0/3681770612 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe3c0054e0 con 0x7fbe4c103900 2024-03-21T11:36:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.023+0000 7fbe42a00640 1 -- 172.21.15.47:0/3681770612 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c003c40 con 0x7fbe4c103900 2024-03-21T11:36:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.023+0000 7fbe50c00640 1 -- 172.21.15.47:0/3681770612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4c1a1a30 con 0x7fbe4c103900 2024-03-21T11:36:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.023+0000 7fbe50c00640 1 -- 172.21.15.47:0/3681770612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4c1a1f30 con 0x7fbe4c103900 2024-03-21T11:36:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.026+0000 7fbe53815740 1 Processor -- start 2024-03-21T11:36:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.026+0000 7fbe53815740 1 -- start start 2024-03-21T11:36:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.026+0000 7fbe42a00640 1 -- 172.21.15.47:0/3681770612 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbe3c046020 con 0x7fbe4c103900 2024-03-21T11:36:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.027+0000 7fbe53815740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581904641e0 0x558190464600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.027+0000 7fbe53815740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190464bd0 0x5581904696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.027+0000 7fbe53815740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190469d40 0x55819046c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.027+0000 7fbe53815740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581904273b0 con 0x558190464bd0 2024-03-21T11:36:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.027+0000 7fbe53815740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581903c9830 con 0x5581904641e0 2024-03-21T11:36:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.027+0000 7fbe53815740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581903725d0 con 0x558190469d40 2024-03-21T11:36:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.027+0000 7fbe49400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190469d40 0x55819046c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.027+0000 7fbe4a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581904641e0 0x558190464600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.027+0000 7fbe49400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190469d40 0x55819046c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43758/0 (socket says 172.21.15.47:43758) 2024-03-21T11:36:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.027+0000 7fbe4a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581904641e0 0x558190464600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33724/0 (socket says 172.21.15.47:33724) 2024-03-21T11:36:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.027+0000 7fbe49400640 1 -- 172.21.15.47:0/1166766568 learned_addr learned my addr 172.21.15.47:0/1166766568 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.027+0000 7fbe49e00640 1 --2- 172.21.15.47:0/1166766568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190464bd0 0x5581904696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.027+0000 7fbe49400640 1 -- 172.21.15.47:0/1166766568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581904641e0 msgr2=0x558190464600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.027+0000 7fbe49400640 1 --2- 172.21.15.47:0/1166766568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581904641e0 0x558190464600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.028+0000 7fbe49400640 1 -- 172.21.15.47:0/1166766568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190464bd0 msgr2=0x5581904696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.028+0000 7fbe49400640 1 --2- 172.21.15.47:0/1166766568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190464bd0 0x5581904696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.028+0000 7fbe49400640 1 -- 172.21.15.47:0/1166766568 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55819038a2c0 con 0x558190469d40 2024-03-21T11:36:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.028+0000 7fbe49400640 1 --2- 172.21.15.47:0/1166766568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190469d40 0x55819046c1d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009c90 tx=0x7fbe38011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98dc43e4e66c3789 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.028+0000 7fbe42a00640 1 --2- 172.21.15.47:0/3681770612 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe300a2fc0 0x7fbe300a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.028+0000 7fbe37000640 1 -- 172.21.15.47:0/1166766568 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3801e070 con 0x558190469d40 2024-03-21T11:36:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.028+0000 7fbe37000640 1 -- 172.21.15.47:0/1166766568 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe38019b10 con 0x558190469d40 2024-03-21T11:36:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.028+0000 7fbe37000640 1 -- 172.21.15.47:0/1166766568 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3800eac0 con 0x558190469d40 2024-03-21T11:36:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.028+0000 7fbe42a00640 1 -- 172.21.15.47:0/3681770612 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbe3c0f1800 con 0x7fbe4c103900 2024-03-21T11:36:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.028+0000 7fbe4a800640 1 --2- 172.21.15.47:0/3681770612 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe300a2fc0 0x7fbe300a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.029+0000 7fbe53815740 1 -- 172.21.15.47:0/1166766568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190469d40 msgr2=0x55819046c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.029+0000 7fbe53815740 1 --2- 172.21.15.47:0/1166766568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190469d40 0x55819046c1d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009c90 tx=0x7fbe38011810 comp rx=0 tx=0).stop 2024-03-21T11:36:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.029+0000 7fbe4a800640 1 --2- 172.21.15.47:0/3681770612 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe300a2fc0 0x7fbe300a5480 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x558190465430 tx=0x7fbe44009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.029+0000 7fbe53815740 1 -- 172.21.15.47:0/1166766568 shutdown_connections 2024-03-21T11:36:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.029+0000 7fbe53815740 1 --2- 172.21.15.47:0/1166766568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190469d40 0x55819046c1d0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.029+0000 7fbe53815740 1 --2- 172.21.15.47:0/1166766568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190464bd0 0x5581904696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.029+0000 7fbe53815740 1 --2- 172.21.15.47:0/1166766568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581904641e0 0x558190464600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.029+0000 7fbe53815740 1 -- 172.21.15.47:0/1166766568 >> 172.21.15.47:0/1166766568 conn(0x55819045f750 msgr2=0x55819045fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.029+0000 7fbe53815740 1 -- 172.21.15.47:0/1166766568 shutdown_connections 2024-03-21T11:36:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.029+0000 7fbe53815740 1 -- 172.21.15.47:0/1166766568 wait complete. 2024-03-21T11:36:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.030+0000 7fbe53815740 1 -- start start 2024-03-21T11:36:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.030+0000 7fbe53815740 1 Processor -- start 2024-03-21T11:36:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe53815740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581904641e0 0x55819046e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe53815740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558190464bd0 0x55819046e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe53815740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190469d40 0x5581904758e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe53815740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558190385a20 con 0x558190469d40 2024-03-21T11:36:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe53815740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581903c58d0 con 0x558190464bd0 2024-03-21T11:36:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe53815740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819046ccf0 con 0x5581904641e0 2024-03-21T11:36:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe53815740 10 client.0 _ll_register_callbacks cb 0x5581903c21c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe49e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581904641e0 0x55819046e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe49400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558190464bd0 0x55819046e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe49e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581904641e0 0x55819046e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43770/0 (socket says 172.21.15.47:43770) 2024-03-21T11:36:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe49e00640 1 -- 172.21.15.47:0/586729432 learned_addr learned my addr 172.21.15.47:0/586729432 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe4a800640 1 --2- 172.21.15.47:0/586729432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190469d40 0x5581904758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe49e00640 1 -- 172.21.15.47:0/586729432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558190464bd0 msgr2=0x55819046e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe49e00640 1 --2- 172.21.15.47:0/586729432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558190464bd0 0x55819046e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe49e00640 1 -- 172.21.15.47:0/586729432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190469d40 msgr2=0x5581904758e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe49e00640 1 --2- 172.21.15.47:0/586729432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190469d40 0x5581904758e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe49e00640 1 -- 172.21.15.47:0/586729432 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581900de210 con 0x5581904641e0 2024-03-21T11:36:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.031+0000 7fbe49400640 1 --2- 172.21.15.47:0/586729432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558190464bd0 0x55819046e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.032+0000 7fbe49e00640 1 --2- 172.21.15.47:0/586729432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581904641e0 0x55819046e360 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c006010 tx=0x7fbe3c0bd9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=697a418a0ce24454 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.032+0000 7fbe2b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.032+0000 7fbe2b400640 1 -- 172.21.15.47:0/586729432 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c0035f0 con 0x5581904641e0 2024-03-21T11:36:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.032+0000 7fbe2b400640 1 -- 172.21.15.47:0/586729432 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbe3c0ef020 con 0x5581904641e0 2024-03-21T11:36:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.032+0000 7fbe53815740 1 -- 172.21.15.47:0/586729432 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe38019b10 con 0x5581904641e0 2024-03-21T11:36:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.032+0000 7fbe2b400640 1 -- 172.21.15.47:0/586729432 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe3c0f7160 con 0x5581904641e0 2024-03-21T11:36:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.032+0000 7fbe53815740 10 client.29956 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.032+0000 7fbe53815740 1 -- 172.21.15.47:0/586729432 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5581901aa620 con 0x5581904641e0 2024-03-21T11:36:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.032+0000 7fbe2b400640 1 -- 172.21.15.47:0/586729432 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe38019b10 con 0x5581904641e0 2024-03-21T11:36:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.033+0000 7fbe2b400640 1 -- 172.21.15.47:0/586729432 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbe3c0c60c0 con 0x5581904641e0 2024-03-21T11:36:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.033+0000 7fbe53815740 10 client.29956 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.033+0000 7fbe53815740 10 client.29956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.033+0000 7fbe53815740 20 client.29956 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.033+0000 7fbe53815740 1 --2- 172.21.15.47:0/586729432 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55819047a110 0x55819047c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.033+0000 7fbe53815740 4 client.29956 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.033+0000 7fbe53815740 1 -- 172.21.15.47:0/586729432 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581903d6d70 con 0x55819047a110 2024-03-21T11:36:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.033+0000 7fbe49400640 1 --2- 172.21.15.47:0/586729432 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55819047a110 0x55819047c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.034+0000 7fbe49400640 1 --2- 172.21.15.47:0/586729432 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55819047a110 0x55819047c5b0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4d46911572dcec5 server_cookie=965f04039687594c in_seq=0 out_seq=0 2024-03-21T11:36:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.034+0000 7fbe2b400640 10 client.29956 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.035+0000 7fbe2b400640 1 -- 172.21.15.47:0/586729432 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5581903d6d70 con 0x55819047a110 2024-03-21T11:36:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.035+0000 7fbe2b400640 10 client.29956 handle_command_reply: tid=0 2024-03-21T11:36:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.035+0000 7fbe53815740 1 client.29956 shutdown 2024-03-21T11:36:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.035+0000 7fbe53815740 1 -- 172.21.15.47:0/586729432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581904641e0 msgr2=0x55819046e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.035+0000 7fbe53815740 1 --2- 172.21.15.47:0/586729432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581904641e0 0x55819046e360 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c006010 tx=0x7fbe3c0bd9e0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.035+0000 7fbe53815740 1 -- 172.21.15.47:0/586729432 shutdown_connections 2024-03-21T11:36:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.035+0000 7fbe53815740 1 --2- 172.21.15.47:0/586729432 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55819047a110 0x55819047c5b0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.035+0000 7fbe53815740 1 --2- 172.21.15.47:0/586729432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190469d40 0x5581904758e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.035+0000 7fbe53815740 1 --2- 172.21.15.47:0/586729432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558190464bd0 0x55819046e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.035+0000 7fbe2b400640 0 client.29956 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.035+0000 7fbe53815740 1 --2- 172.21.15.47:0/586729432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581904641e0 0x55819046e360 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.035+0000 7fbe53815740 1 -- 172.21.15.47:0/586729432 >> 172.21.15.47:0/586729432 conn(0x55819045f750 msgr2=0x55819045e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.036+0000 7fbe53815740 1 -- 172.21.15.47:0/586729432 shutdown_connections 2024-03-21T11:36:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.036+0000 7fbe53815740 1 -- 172.21.15.47:0/586729432 wait complete. 2024-03-21T11:36:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.036+0000 7fbe53815740 20 client.29956 trim_cache size 0 max 16384 2024-03-21T11:36:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.052+0000 7fbe53815740 1 Processor -- start 2024-03-21T11:36:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.052+0000 7fbe53815740 1 -- start start 2024-03-21T11:36:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.052+0000 7fbe53815740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581904641e0 0x55819046e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.052+0000 7fbe53815740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190464bd0 0x5581904758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.052+0000 7fbe53815740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190469d40 0x558190480eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.052+0000 7fbe53815740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c1ac210 con 0x558190464bd0 2024-03-21T11:36:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.052+0000 7fbe53815740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c1adb90 con 0x5581904641e0 2024-03-21T11:36:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.052+0000 7fbe53815740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c1ac870 con 0x558190469d40 2024-03-21T11:36:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.053+0000 7fbe4a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190469d40 0x558190480eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.053+0000 7fbe49e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581904641e0 0x55819046e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.053+0000 7fbe4a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190469d40 0x558190480eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43784/0 (socket says 172.21.15.47:43784) 2024-03-21T11:36:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.053+0000 7fbe4a800640 1 -- 172.21.15.47:0/1718300586 learned_addr learned my addr 172.21.15.47:0/1718300586 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.053+0000 7fbe49400640 1 --2- 172.21.15.47:0/1718300586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190464bd0 0x5581904758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.053+0000 7fbe4a800640 1 -- 172.21.15.47:0/1718300586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581904641e0 msgr2=0x55819046e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.053+0000 7fbe4a800640 1 --2- 172.21.15.47:0/1718300586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581904641e0 0x55819046e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.053+0000 7fbe4a800640 1 -- 172.21.15.47:0/1718300586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190464bd0 msgr2=0x5581904758a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.053+0000 7fbe4a800640 1 --2- 172.21.15.47:0/1718300586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190464bd0 0x5581904758a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.053+0000 7fbe4a800640 1 -- 172.21.15.47:0/1718300586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558190464fb0 con 0x558190469d40 2024-03-21T11:36:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.053+0000 7fbe49400640 1 --2- 172.21.15.47:0/1718300586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190464bd0 0x5581904758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.054+0000 7fbe4a800640 1 --2- 172.21.15.47:0/1718300586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190469d40 0x558190480eb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fbe44003f70 tx=0x7fbe4401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99f1859d17325fd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.054+0000 7fbe36600640 1 -- 172.21.15.47:0/1718300586 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4400f660 con 0x558190469d40 2024-03-21T11:36:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.054+0000 7fbe36600640 1 -- 172.21.15.47:0/1718300586 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe4400fce0 con 0x558190469d40 2024-03-21T11:36:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.054+0000 7fbe36600640 1 -- 172.21.15.47:0/1718300586 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe44029e60 con 0x558190469d40 2024-03-21T11:36:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.055+0000 7fbe53815740 1 -- 172.21.15.47:0/1718300586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190469d40 msgr2=0x558190480eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.055+0000 7fbe53815740 1 --2- 172.21.15.47:0/1718300586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190469d40 0x558190480eb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fbe44003f70 tx=0x7fbe4401a000 comp rx=0 tx=0).stop 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.055+0000 7fbe53815740 1 -- 172.21.15.47:0/1718300586 shutdown_connections 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.055+0000 7fbe53815740 1 --2- 172.21.15.47:0/1718300586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190469d40 0x558190480eb0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.055+0000 7fbe53815740 1 --2- 172.21.15.47:0/1718300586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558190464bd0 0x5581904758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.055+0000 7fbe53815740 1 --2- 172.21.15.47:0/1718300586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581904641e0 0x55819046e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.055+0000 7fbe53815740 1 -- 172.21.15.47:0/1718300586 >> 172.21.15.47:0/1718300586 conn(0x55819045f750 msgr2=0x55819046df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.055+0000 7fbe53815740 1 -- 172.21.15.47:0/1718300586 shutdown_connections 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.055+0000 7fbe53815740 1 -- 172.21.15.47:0/1718300586 wait complete. 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.055+0000 7fbe53815740 1 -- start start 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.055+0000 7fbe53815740 1 Processor -- start 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe53815740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581904641e0 0x5581904757c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe53815740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190464bd0 0x558190473dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe53815740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558190469d40 0x558190474330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe53815740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c1ad530 con 0x5581904641e0 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe53815740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c1aced0 con 0x558190469d40 2024-03-21T11:36:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe53815740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c1aa3d0 con 0x558190464bd0 2024-03-21T11:36:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe53815740 10 client.0 _ll_register_callbacks cb 0x7fbe3c0b9eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe4a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558190469d40 0x558190474330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe49400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190464bd0 0x558190473dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe4a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558190469d40 0x558190474330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33750/0 (socket says 172.21.15.47:33750) 2024-03-21T11:36:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe4a800640 1 -- 172.21.15.47:0/3485810811 learned_addr learned my addr 172.21.15.47:0/3485810811 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe49e00640 1 --2- 172.21.15.47:0/3485810811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581904641e0 0x5581904757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe4a800640 1 -- 172.21.15.47:0/3485810811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190464bd0 msgr2=0x558190473dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe4a800640 1 --2- 172.21.15.47:0/3485810811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190464bd0 0x558190473dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe4a800640 1 -- 172.21.15.47:0/3485810811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581904641e0 msgr2=0x5581904757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe4a800640 1 --2- 172.21.15.47:0/3485810811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581904641e0 0x5581904757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.056+0000 7fbe4a800640 1 -- 172.21.15.47:0/3485810811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558190474b60 con 0x558190469d40 2024-03-21T11:36:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.057+0000 7fbe4a800640 1 --2- 172.21.15.47:0/3485810811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558190469d40 0x558190474330 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fbe440048a0 tx=0x7fbe440297c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=176975a43baf88b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.057+0000 7fbe2b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.057+0000 7fbe2b400640 1 -- 172.21.15.47:0/3485810811 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe44029da0 con 0x558190469d40 2024-03-21T11:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.057+0000 7fbe2b400640 1 -- 172.21.15.47:0/3485810811 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbe44004a40 con 0x558190469d40 2024-03-21T11:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.057+0000 7fbe53815740 1 -- 172.21.15.47:0/3485810811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe4400fce0 con 0x558190469d40 2024-03-21T11:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.057+0000 7fbe2b400640 1 -- 172.21.15.47:0/3485810811 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe4403b050 con 0x558190469d40 2024-03-21T11:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.057+0000 7fbe53815740 10 client.29927 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.057+0000 7fbe53815740 1 -- 172.21.15.47:0/3485810811 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5581904d6b20 con 0x558190469d40 2024-03-21T11:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.058+0000 7fbe2b400640 1 -- 172.21.15.47:0/3485810811 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe4400fce0 con 0x558190469d40 2024-03-21T11:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.058+0000 7fbe2b400640 1 -- 172.21.15.47:0/3485810811 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbe44030020 con 0x558190469d40 2024-03-21T11:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.058+0000 7fbe53815740 10 client.29927 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.058+0000 7fbe53815740 10 client.29927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.058+0000 7fbe53815740 20 client.29927 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.058+0000 7fbe53815740 1 --2- 172.21.15.47:0/3485810811 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5581904d7430 0x5581904d9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.058+0000 7fbe53815740 4 client.29927 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.058+0000 7fbe53815740 1 -- 172.21.15.47:0/3485810811 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5581903d10c0 con 0x5581904d7430 2024-03-21T11:36:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.058+0000 7fbe49e00640 1 --2- 172.21.15.47:0/3485810811 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5581904d7430 0x5581904d9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.059+0000 7fbe49e00640 1 --2- 172.21.15.47:0/3485810811 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5581904d7430 0x5581904d9850 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b83f1ec78f1dbb server_cookie=7994eea2db2c2913 in_seq=0 out_seq=0 2024-03-21T11:36:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.059+0000 7fbe2b400640 10 client.29927 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:21 smithi047 ceph-mon[31351]: pgmap v790: 129 pgs: 129 active+clean; 38 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 303 op/s 2024-03-21T11:36:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:21 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:36:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:21 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:36:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:21 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:36:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:21 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:36:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.333+0000 7fbe2b400640 1 -- 172.21.15.47:0/3485810811 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5581903d10c0 con 0x5581904d7430 2024-03-21T11:36:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.333+0000 7fbe2b400640 10 client.29927 handle_command_reply: tid=0 2024-03-21T11:36:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.333+0000 7fbe53815740 1 client.29927 shutdown 2024-03-21T11:36:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.333+0000 7fbe53815740 1 -- 172.21.15.47:0/3485810811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558190469d40 msgr2=0x558190474330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.333+0000 7fbe53815740 1 --2- 172.21.15.47:0/3485810811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558190469d40 0x558190474330 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fbe440048a0 tx=0x7fbe440297c0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.333+0000 7fbe53815740 1 -- 172.21.15.47:0/3485810811 shutdown_connections 2024-03-21T11:36:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.333+0000 7fbe53815740 1 --2- 172.21.15.47:0/3485810811 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5581904d7430 0x5581904d9850 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.333+0000 7fbe53815740 1 --2- 172.21.15.47:0/3485810811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558190469d40 0x558190474330 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.333+0000 7fbe53815740 1 --2- 172.21.15.47:0/3485810811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558190464bd0 0x558190473dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.333+0000 7fbe53815740 1 --2- 172.21.15.47:0/3485810811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581904641e0 0x5581904757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.333+0000 7fbe2b400640 0 client.29927 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.333+0000 7fbe53815740 1 -- 172.21.15.47:0/3485810811 >> 172.21.15.47:0/3485810811 conn(0x55819045f750 msgr2=0x55819045e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.333+0000 7fbe53815740 1 -- 172.21.15.47:0/3485810811 shutdown_connections 2024-03-21T11:36:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.333+0000 7fbe53815740 1 -- 172.21.15.47:0/3485810811 wait complete. 2024-03-21T11:36:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.334+0000 7fbe53815740 20 client.29927 trim_cache size 0 max 16384 2024-03-21T11:36:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:36:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:36:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:36:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:36:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.334+0000 7fbe50c00640 1 -- 172.21.15.47:0/3681770612 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe300a2fc0 msgr2=0x7fbe300a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.334+0000 7fbe50c00640 1 --2- 172.21.15.47:0/3681770612 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe300a2fc0 0x7fbe300a5480 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x558190465430 tx=0x7fbe44009500 comp rx=0 tx=0).stop 2024-03-21T11:36:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.335+0000 7fbe50c00640 1 -- 172.21.15.47:0/3681770612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c103900 msgr2=0x7fbe4c199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.335+0000 7fbe50c00640 1 --2- 172.21.15.47:0/3681770612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c103900 0x7fbe4c199b80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c000c00 tx=0x7fbe3c02c650 comp rx=0 tx=0).stop 2024-03-21T11:36:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.335+0000 7fbe50c00640 1 -- 172.21.15.47:0/3681770612 shutdown_connections 2024-03-21T11:36:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.335+0000 7fbe50c00640 1 --2- 172.21.15.47:0/3681770612 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbe300a2fc0 0x7fbe300a5480 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.335+0000 7fbe50c00640 1 --2- 172.21.15.47:0/3681770612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe4c1054b0 0x7fbe4c1a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.335+0000 7fbe50c00640 1 --2- 172.21.15.47:0/3681770612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe4c104af0 0x7fbe4c19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.335+0000 7fbe50c00640 1 --2- 172.21.15.47:0/3681770612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe4c103900 0x7fbe4c199b80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.335+0000 7fbe50c00640 1 -- 172.21.15.47:0/3681770612 >> 172.21.15.47:0/3681770612 conn(0x7fbe4c0fee90 msgr2=0x7fbe4c107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.335+0000 7fbe50c00640 1 -- 172.21.15.47:0/3681770612 shutdown_connections 2024-03-21T11:36:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:21.335+0000 7fbe50c00640 1 -- 172.21.15.47:0/3681770612 wait complete. 2024-03-21T11:36:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:21 smithi100 ceph-mon[36245]: pgmap v790: 129 pgs: 129 active+clean; 38 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 303 op/s 2024-03-21T11:36:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:21 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:36:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:21 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:36:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:21 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:36:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:21 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:36:21.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:21 smithi102 ceph-mon[36748]: pgmap v790: 129 pgs: 129 active+clean; 38 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 303 op/s 2024-03-21T11:36:21.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:21 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:36:21.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:21 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:36:21.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:21 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:36:21.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:21 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:36:21.579 INFO:tasks.workunit.client.0.smithi047.stdout: 1 991651 7.94 MB/sec execute 545 sec latency 170.642 ms 2024-03-21T11:36:22.579 INFO:tasks.workunit.client.0.smithi047.stdout: 1 992890 7.93 MB/sec execute 546 sec latency 13.260 ms 2024-03-21T11:36:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:23 smithi047 ceph-mon[31351]: pgmap v791: 129 pgs: 129 active+clean; 38 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.8 MiB/s wr, 250 op/s 2024-03-21T11:36:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:23 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:36:23.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:23 smithi100 ceph-mon[36245]: pgmap v791: 129 pgs: 129 active+clean; 38 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.8 MiB/s wr, 250 op/s 2024-03-21T11:36:23.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:23 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:36:23.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:23 smithi102 ceph-mon[36748]: pgmap v791: 129 pgs: 129 active+clean; 38 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.8 MiB/s wr, 250 op/s 2024-03-21T11:36:23.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:23 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:36:23.579 INFO:tasks.workunit.client.0.smithi047.stdout: 1 994769 7.94 MB/sec execute 547 sec latency 60.681 ms 2024-03-21T11:36:24.579 INFO:tasks.workunit.client.0.smithi047.stdout: 1 995587 7.93 MB/sec execute 548 sec latency 20.597 ms 2024-03-21T11:36:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:25 smithi047 ceph-mon[31351]: pgmap v792: 129 pgs: 129 active+clean; 36 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 329 op/s 2024-03-21T11:36:25.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:25 smithi100 ceph-mon[36245]: pgmap v792: 129 pgs: 129 active+clean; 36 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 329 op/s 2024-03-21T11:36:25.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:25 smithi102 ceph-mon[36748]: pgmap v792: 129 pgs: 129 active+clean; 36 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 329 op/s 2024-03-21T11:36:25.579 INFO:tasks.workunit.client.0.smithi047.stdout: 1 997551 7.94 MB/sec execute 549 sec latency 35.426 ms 2024-03-21T11:36:26.580 INFO:tasks.workunit.client.0.smithi047.stdout: 1 998920 7.94 MB/sec execute 550 sec latency 10.569 ms 2024-03-21T11:36:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:27 smithi047 ceph-mon[31351]: pgmap v793: 129 pgs: 129 active+clean; 36 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-03-21T11:36:27.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:27 smithi100 ceph-mon[36245]: pgmap v793: 129 pgs: 129 active+clean; 36 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-03-21T11:36:27.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:27 smithi102 ceph-mon[36748]: pgmap v793: 129 pgs: 129 active+clean; 36 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-03-21T11:36:27.580 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1000701 7.94 MB/sec execute 551 sec latency 12.149 ms 2024-03-21T11:36:28.580 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1002177 7.94 MB/sec execute 552 sec latency 29.831 ms 2024-03-21T11:36:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:29 smithi100 ceph-mon[36245]: pgmap v794: 129 pgs: 129 active+clean; 36 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2024-03-21T11:36:29.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:29 smithi102 ceph-mon[36748]: pgmap v794: 129 pgs: 129 active+clean; 36 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2024-03-21T11:36:29.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:29 smithi047 ceph-mon[31351]: pgmap v794: 129 pgs: 129 active+clean; 36 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 265 op/s 2024-03-21T11:36:29.580 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1003264 7.94 MB/sec execute 553 sec latency 30.435 ms 2024-03-21T11:36:30.580 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1005375 7.95 MB/sec execute 554 sec latency 14.704 ms 2024-03-21T11:36:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:31 smithi100 ceph-mon[36245]: pgmap v795: 129 pgs: 129 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 303 op/s 2024-03-21T11:36:31.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:31 smithi102 ceph-mon[36748]: pgmap v795: 129 pgs: 129 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 303 op/s 2024-03-21T11:36:31.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:31 smithi047 ceph-mon[31351]: pgmap v795: 129 pgs: 129 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 303 op/s 2024-03-21T11:36:31.580 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1006219 7.94 MB/sec execute 555 sec latency 15.066 ms 2024-03-21T11:36:32.580 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1008023 7.93 MB/sec execute 556 sec latency 20.730 ms 2024-03-21T11:36:33.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:33 smithi100 ceph-mon[36245]: pgmap v796: 129 pgs: 129 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 260 op/s 2024-03-21T11:36:33.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:33 smithi102 ceph-mon[36748]: pgmap v796: 129 pgs: 129 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 260 op/s 2024-03-21T11:36:33.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:33 smithi047 ceph-mon[31351]: pgmap v796: 129 pgs: 129 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 260 op/s 2024-03-21T11:36:33.580 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1009505 7.94 MB/sec execute 557 sec latency 12.684 ms 2024-03-21T11:36:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.487+0000 7fe616000640 1 -- 172.21.15.47:0/390414705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6101073a0 msgr2=0x7fe6101097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.487+0000 7fe616000640 1 --2- 172.21.15.47:0/390414705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6101073a0 0x7fe6101097f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe608009900 tx=0x7fe60802bc40 comp rx=0 tx=0).stop 2024-03-21T11:36:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.488+0000 7fe616000640 1 -- 172.21.15.47:0/390414705 shutdown_connections 2024-03-21T11:36:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.488+0000 7fe616000640 1 --2- 172.21.15.47:0/390414705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6101073a0 0x7fe6101097f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.488+0000 7fe616000640 1 --2- 172.21.15.47:0/390414705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe610104a70 0x7fe610106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.488+0000 7fe616000640 1 --2- 172.21.15.47:0/390414705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe610102140 0x7fe610104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.488+0000 7fe616000640 1 -- 172.21.15.47:0/390414705 >> 172.21.15.47:0/390414705 conn(0x7fe6100fbb30 msgr2=0x7fe6100fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.488+0000 7fe616000640 1 -- 172.21.15.47:0/390414705 shutdown_connections 2024-03-21T11:36:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.488+0000 7fe616000640 1 -- 172.21.15.47:0/390414705 wait complete. 2024-03-21T11:36:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.489+0000 7fe616000640 1 Processor -- start 2024-03-21T11:36:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.489+0000 7fe616000640 1 -- start start 2024-03-21T11:36:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.489+0000 7fe616000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe610102140 0x7fe610199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.489+0000 7fe616000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe610104a70 0x7fe61019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.489+0000 7fe616000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6101073a0 0x7fe6101a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.489+0000 7fe616000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61010d360 con 0x7fe610104a70 2024-03-21T11:36:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.489+0000 7fe616000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61010d1e0 con 0x7fe6101073a0 2024-03-21T11:36:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.489+0000 7fe616000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61010d4e0 con 0x7fe610102140 2024-03-21T11:36:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.489+0000 7fe60f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe610102140 0x7fe610199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.489+0000 7fe60fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6101073a0 0x7fe6101a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.489+0000 7fe60fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6101073a0 0x7fe6101a10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35198/0 (socket says 172.21.15.47:35198) 2024-03-21T11:36:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.489+0000 7fe60f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe610102140 0x7fe610199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51428/0 (socket says 172.21.15.47:51428) 2024-03-21T11:36:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.489+0000 7fe60fe00640 1 -- 172.21.15.47:0/1458032144 learned_addr learned my addr 172.21.15.47:0/1458032144 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.490+0000 7fe60ea00640 1 --2- 172.21.15.47:0/1458032144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe610104a70 0x7fe61019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.490+0000 7fe60fe00640 1 -- 172.21.15.47:0/1458032144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe610102140 msgr2=0x7fe610199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.490+0000 7fe60fe00640 1 --2- 172.21.15.47:0/1458032144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe610102140 0x7fe610199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.490+0000 7fe60fe00640 1 -- 172.21.15.47:0/1458032144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe610104a70 msgr2=0x7fe61019a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.490+0000 7fe60fe00640 1 --2- 172.21.15.47:0/1458032144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe610104a70 0x7fe61019a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.490+0000 7fe60fe00640 1 -- 172.21.15.47:0/1458032144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6101a1750 con 0x7fe6101073a0 2024-03-21T11:36:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.490+0000 7fe60f400640 1 --2- 172.21.15.47:0/1458032144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe610102140 0x7fe610199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.490+0000 7fe60fe00640 1 --2- 172.21.15.47:0/1458032144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6101073a0 0x7fe6101a10e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe60802c7c0 tx=0x7fe608005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.491+0000 7fe607e00640 1 -- 172.21.15.47:0/1458032144 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608041070 con 0x7fe6101073a0 2024-03-21T11:36:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.491+0000 7fe607e00640 1 -- 172.21.15.47:0/1458032144 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6080054e0 con 0x7fe6101073a0 2024-03-21T11:36:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.491+0000 7fe607e00640 1 -- 172.21.15.47:0/1458032144 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608034440 con 0x7fe6101073a0 2024-03-21T11:36:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.491+0000 7fe616000640 1 -- 172.21.15.47:0/1458032144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6101a19e0 con 0x7fe6101073a0 2024-03-21T11:36:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.491+0000 7fe616000640 1 -- 172.21.15.47:0/1458032144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6101a1f20 con 0x7fe6101073a0 2024-03-21T11:36:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.494+0000 7fe607e00640 1 -- 172.21.15.47:0/1458032144 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe608044070 con 0x7fe6101073a0 2024-03-21T11:36:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.494+0000 7fe618c6f740 1 Processor -- start 2024-03-21T11:36:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.494+0000 7fe618c6f740 1 -- start start 2024-03-21T11:36:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe618c6f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab6ff5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe618c6f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ff5bd0 0x555ab6ffa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe618c6f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ffad40 0x555ab6ffd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe618c6f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ab6fb83b0 con 0x555ab6ff5bd0 2024-03-21T11:36:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe618c6f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ab6f5a830 con 0x555ab6ff51e0 2024-03-21T11:36:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe618c6f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ab6f035d0 con 0x555ab6ffad40 2024-03-21T11:36:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe60ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab6ff5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe60ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab6ff5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35210/0 (socket says 172.21.15.47:35210) 2024-03-21T11:36:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe60ea00640 1 -- 172.21.15.47:0/3666669087 learned_addr learned my addr 172.21.15.47:0/3666669087 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe60f400640 1 --2- 172.21.15.47:0/3666669087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ff5bd0 0x555ab6ffa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe60ea00640 1 --2- 172.21.15.47:0/3666669087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ffad40 0x555ab6ffd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe60ea00640 1 -- 172.21.15.47:0/3666669087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ffad40 msgr2=0x555ab6ffd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe60ea00640 1 --2- 172.21.15.47:0/3666669087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ffad40 0x555ab6ffd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe60ea00640 1 -- 172.21.15.47:0/3666669087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ff5bd0 msgr2=0x555ab6ffa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe60ea00640 1 --2- 172.21.15.47:0/3666669087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ff5bd0 0x555ab6ffa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.495+0000 7fe60ea00640 1 -- 172.21.15.47:0/3666669087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ab6f1b2c0 con 0x555ab6ff51e0 2024-03-21T11:36:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.496+0000 7fe607e00640 1 --2- 172.21.15.47:0/1458032144 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe5f40a3040 0x7fe5f40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.496+0000 7fe60ea00640 1 --2- 172.21.15.47:0/3666669087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab6ff5600 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80136f0 tx=0x7fe5f80096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=471330f05d9844e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.496+0000 7fe607e00640 1 -- 172.21.15.47:0/1458032144 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe6080f1360 con 0x7fe6101073a0 2024-03-21T11:36:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.496+0000 7fe60f400640 1 --2- 172.21.15.47:0/1458032144 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe5f40a3040 0x7fe5f40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.496+0000 7fe5ffe00640 1 -- 172.21.15.47:0/3666669087 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8011080 con 0x555ab6ff51e0 2024-03-21T11:36:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.496+0000 7fe5ffe00640 1 -- 172.21.15.47:0/3666669087 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5f8013c70 con 0x555ab6ff51e0 2024-03-21T11:36:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.496+0000 7fe5ffe00640 1 -- 172.21.15.47:0/3666669087 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8011d90 con 0x555ab6ff51e0 2024-03-21T11:36:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.497+0000 7fe618c6f740 1 -- 172.21.15.47:0/3666669087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 msgr2=0x555ab6ff5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.497+0000 7fe60f400640 1 --2- 172.21.15.47:0/1458032144 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe5f40a3040 0x7fe5f40a5500 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x555ab6ff5fd0 tx=0x7fe600009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.497+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3666669087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab6ff5600 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80136f0 tx=0x7fe5f80096c0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.497+0000 7fe618c6f740 1 -- 172.21.15.47:0/3666669087 shutdown_connections 2024-03-21T11:36:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.497+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3666669087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ffad40 0x555ab6ffd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.497+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3666669087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ff5bd0 0x555ab6ffa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.497+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3666669087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab6ff5600 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.497+0000 7fe618c6f740 1 -- 172.21.15.47:0/3666669087 >> 172.21.15.47:0/3666669087 conn(0x555ab6ff0750 msgr2=0x555ab6ff0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.497+0000 7fe618c6f740 1 -- 172.21.15.47:0/3666669087 shutdown_connections 2024-03-21T11:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.497+0000 7fe618c6f740 1 -- 172.21.15.47:0/3666669087 wait complete. 2024-03-21T11:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.498+0000 7fe618c6f740 1 -- start start 2024-03-21T11:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.498+0000 7fe618c6f740 1 Processor -- start 2024-03-21T11:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.498+0000 7fe618c6f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff51e0 0x555ab6fff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.498+0000 7fe618c6f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ff5bd0 0x555ab6fff910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.498+0000 7fe618c6f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ffad40 0x555ab7006950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.498+0000 7fe618c6f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ab6f16a20 con 0x555ab6ff5bd0 2024-03-21T11:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.498+0000 7fe618c6f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ab6f568d0 con 0x555ab6ffad40 2024-03-21T11:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.498+0000 7fe618c6f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ab6ffdcf0 con 0x555ab6ff51e0 2024-03-21T11:36:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.498+0000 7fe618c6f740 10 client.0 _ll_register_callbacks cb 0x555ab6f531c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.499+0000 7fe60f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff51e0 0x555ab6fff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.499+0000 7fe60fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ffad40 0x555ab7006950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.499+0000 7fe60f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff51e0 0x555ab6fff3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51450/0 (socket says 172.21.15.47:51450) 2024-03-21T11:36:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.499+0000 7fe60fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ffad40 0x555ab7006950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35212/0 (socket says 172.21.15.47:35212) 2024-03-21T11:36:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.499+0000 7fe60f400640 1 -- 172.21.15.47:0/3909194936 learned_addr learned my addr 172.21.15.47:0/3909194936 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.499+0000 7fe60ea00640 1 --2- 172.21.15.47:0/3909194936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ff5bd0 0x555ab6fff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.499+0000 7fe60f400640 1 -- 172.21.15.47:0/3909194936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ffad40 msgr2=0x555ab7006950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.499+0000 7fe60f400640 1 --2- 172.21.15.47:0/3909194936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ffad40 0x555ab7006950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.499+0000 7fe60f400640 1 -- 172.21.15.47:0/3909194936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ff5bd0 msgr2=0x555ab6fff910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.499+0000 7fe60f400640 1 --2- 172.21.15.47:0/3909194936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ff5bd0 0x555ab6fff910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.499+0000 7fe60f400640 1 -- 172.21.15.47:0/3909194936 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ab6c6f210 con 0x555ab6ff51e0 2024-03-21T11:36:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.499+0000 7fe60fe00640 1 --2- 172.21.15.47:0/3909194936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ffad40 0x555ab7006950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.499+0000 7fe60f400640 1 --2- 172.21.15.47:0/3909194936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff51e0 0x555ab6fff3d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fe600005f40 tx=0x7fe600004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92f399b2a2132760 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.499+0000 7fe5fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.500+0000 7fe5fcc00640 1 -- 172.21.15.47:0/3909194936 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600003970 con 0x555ab6ff51e0 2024-03-21T11:36:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.500+0000 7fe5fcc00640 1 -- 172.21.15.47:0/3909194936 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe600003b10 con 0x555ab6ff51e0 2024-03-21T11:36:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.500+0000 7fe618c6f740 1 -- 172.21.15.47:0/3909194936 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5f8013c70 con 0x555ab6ff51e0 2024-03-21T11:36:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.500+0000 7fe5fcc00640 1 -- 172.21.15.47:0/3909194936 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe600031050 con 0x555ab6ff51e0 2024-03-21T11:36:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.500+0000 7fe618c6f740 10 client.29974 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.500+0000 7fe618c6f740 1 -- 172.21.15.47:0/3909194936 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555ab6d3b620 con 0x555ab6ff51e0 2024-03-21T11:36:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.500+0000 7fe5fcc00640 1 -- 172.21.15.47:0/3909194936 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5f8013c70 con 0x555ab6ff51e0 2024-03-21T11:36:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.501+0000 7fe5fcc00640 1 -- 172.21.15.47:0/3909194936 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe60001a040 con 0x555ab6ff51e0 2024-03-21T11:36:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.501+0000 7fe618c6f740 10 client.29974 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.501+0000 7fe618c6f740 10 client.29974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.501+0000 7fe618c6f740 20 client.29974 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.501+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3909194936 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ab700b2a0 0x555ab700d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.501+0000 7fe618c6f740 4 client.29974 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.501+0000 7fe618c6f740 1 -- 172.21.15.47:0/3909194936 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ab6f67d70 con 0x555ab700b2a0 2024-03-21T11:36:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.501+0000 7fe60ea00640 1 --2- 172.21.15.47:0/3909194936 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ab700b2a0 0x555ab700d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.501+0000 7fe60ea00640 1 --2- 172.21.15.47:0/3909194936 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ab700b2a0 0x555ab700d6b0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8be852225331c087 server_cookie=a3c2265cafc32167 in_seq=0 out_seq=0 2024-03-21T11:36:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.501+0000 7fe5fcc00640 10 client.29974 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.502+0000 7fe5fcc00640 1 -- 172.21.15.47:0/3909194936 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555ab6f67d70 con 0x555ab700b2a0 2024-03-21T11:36:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.502+0000 7fe5fcc00640 10 client.29974 handle_command_reply: tid=0 2024-03-21T11:36:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.503+0000 7fe618c6f740 1 client.29974 shutdown 2024-03-21T11:36:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.503+0000 7fe618c6f740 1 -- 172.21.15.47:0/3909194936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff51e0 msgr2=0x555ab6fff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.503+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3909194936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff51e0 0x555ab6fff3d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fe600005f40 tx=0x7fe600004a40 comp rx=0 tx=0).stop 2024-03-21T11:36:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.503+0000 7fe618c6f740 1 -- 172.21.15.47:0/3909194936 shutdown_connections 2024-03-21T11:36:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.503+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3909194936 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ab700b2a0 0x555ab700d6b0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.503+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3909194936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ffad40 0x555ab7006950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.503+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3909194936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ff5bd0 0x555ab6fff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.503+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3909194936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff51e0 0x555ab6fff3d0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.503+0000 7fe5fcc00640 0 client.29974 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.503+0000 7fe618c6f740 1 -- 172.21.15.47:0/3909194936 >> 172.21.15.47:0/3909194936 conn(0x555ab6ff0750 msgr2=0x555ab6fef950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.503+0000 7fe618c6f740 1 -- 172.21.15.47:0/3909194936 shutdown_connections 2024-03-21T11:36:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.503+0000 7fe618c6f740 1 -- 172.21.15.47:0/3909194936 wait complete. 2024-03-21T11:36:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.503+0000 7fe618c6f740 20 client.29974 trim_cache size 0 max 16384 2024-03-21T11:36:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.519+0000 7fe618c6f740 1 Processor -- start 2024-03-21T11:36:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.519+0000 7fe618c6f740 1 -- start start 2024-03-21T11:36:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe618c6f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab700d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe618c6f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff5bd0 0x555ab7006910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe618c6f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ffad40 0x555ab6ff39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe618c6f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6101ac270 con 0x555ab6ffad40 2024-03-21T11:36:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe618c6f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6101adbf0 con 0x555ab6ff51e0 2024-03-21T11:36:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe618c6f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6101ac8d0 con 0x555ab6ff5bd0 2024-03-21T11:36:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe60fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ffad40 0x555ab6ff39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe60ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff5bd0 0x555ab7006910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe60fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ffad40 0x555ab6ff39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48618/0 (socket says 172.21.15.47:48618) 2024-03-21T11:36:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe60fe00640 1 -- 172.21.15.47:0/2302550075 learned_addr learned my addr 172.21.15.47:0/2302550075 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe60f400640 1 --2- 172.21.15.47:0/2302550075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab700d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe60ea00640 1 -- 172.21.15.47:0/2302550075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 msgr2=0x555ab700d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe60ea00640 1 --2- 172.21.15.47:0/2302550075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab700d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe60ea00640 1 -- 172.21.15.47:0/2302550075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ffad40 msgr2=0x555ab6ff39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe60ea00640 1 --2- 172.21.15.47:0/2302550075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ffad40 0x555ab6ff39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.520+0000 7fe60ea00640 1 -- 172.21.15.47:0/2302550075 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ab6ffa550 con 0x555ab6ff5bd0 2024-03-21T11:36:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.521+0000 7fe60ea00640 1 --2- 172.21.15.47:0/2302550075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff5bd0 0x555ab7006910 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8005890 tx=0x7fe5f8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5d476e5a87a1eb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.521+0000 7fe5ff400640 1 -- 172.21.15.47:0/2302550075 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8013c70 con 0x555ab6ff5bd0 2024-03-21T11:36:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.521+0000 7fe5ff400640 1 -- 172.21.15.47:0/2302550075 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5f8024030 con 0x555ab6ff5bd0 2024-03-21T11:36:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.521+0000 7fe5ff400640 1 -- 172.21.15.47:0/2302550075 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8024840 con 0x555ab6ff5bd0 2024-03-21T11:36:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.522+0000 7fe618c6f740 1 -- 172.21.15.47:0/2302550075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff5bd0 msgr2=0x555ab7006910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.522+0000 7fe618c6f740 1 --2- 172.21.15.47:0/2302550075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff5bd0 0x555ab7006910 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8005890 tx=0x7fe5f8005f90 comp rx=0 tx=0).stop 2024-03-21T11:36:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.522+0000 7fe618c6f740 1 -- 172.21.15.47:0/2302550075 shutdown_connections 2024-03-21T11:36:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.522+0000 7fe618c6f740 1 --2- 172.21.15.47:0/2302550075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ffad40 0x555ab6ff39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.522+0000 7fe618c6f740 1 --2- 172.21.15.47:0/2302550075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff5bd0 0x555ab7006910 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.522+0000 7fe618c6f740 1 --2- 172.21.15.47:0/2302550075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab700d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.522+0000 7fe618c6f740 1 -- 172.21.15.47:0/2302550075 >> 172.21.15.47:0/2302550075 conn(0x555ab6ff0750 msgr2=0x555ab6ff0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.522+0000 7fe618c6f740 1 -- 172.21.15.47:0/2302550075 shutdown_connections 2024-03-21T11:36:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.522+0000 7fe618c6f740 1 -- 172.21.15.47:0/2302550075 wait complete. 2024-03-21T11:36:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.522+0000 7fe618c6f740 1 -- start start 2024-03-21T11:36:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.522+0000 7fe618c6f740 1 Processor -- start 2024-03-21T11:36:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe618c6f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab7004600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe618c6f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff5bd0 0x555ab7004b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe618c6f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ffad40 0x555ab7005b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe618c6f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6101ad590 con 0x555ab6ffad40 2024-03-21T11:36:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe618c6f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6101acf30 con 0x555ab6ff51e0 2024-03-21T11:36:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe618c6f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6101aa430 con 0x555ab6ff5bd0 2024-03-21T11:36:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe618c6f740 10 client.0 _ll_register_callbacks cb 0x7fe6000115a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe60f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab7004600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe60f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab7004600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35232/0 (socket says 172.21.15.47:35232) 2024-03-21T11:36:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe60f400640 1 -- 172.21.15.47:0/3773002054 learned_addr learned my addr 172.21.15.47:0/3773002054 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe60fe00640 1 --2- 172.21.15.47:0/3773002054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ffad40 0x555ab7005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe60ea00640 1 --2- 172.21.15.47:0/3773002054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff5bd0 0x555ab7004b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe60f400640 1 -- 172.21.15.47:0/3773002054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff5bd0 msgr2=0x555ab7004b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe60f400640 1 --2- 172.21.15.47:0/3773002054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff5bd0 0x555ab7004b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe60f400640 1 -- 172.21.15.47:0/3773002054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ffad40 msgr2=0x555ab7005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe60f400640 1 --2- 172.21.15.47:0/3773002054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ffad40 0x555ab7005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.523+0000 7fe60f400640 1 -- 172.21.15.47:0/3773002054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ab70060f0 con 0x555ab6ff51e0 2024-03-21T11:36:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.524+0000 7fe60f400640 1 --2- 172.21.15.47:0/3773002054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab7004600 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe600006010 tx=0x7fe60001cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b417118fdcc327d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.524+0000 7fe5fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.524+0000 7fe5fcc00640 1 -- 172.21.15.47:0/3773002054 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600039070 con 0x555ab6ff51e0 2024-03-21T11:36:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.524+0000 7fe618c6f740 1 -- 172.21.15.47:0/3773002054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5f8024030 con 0x555ab6ff51e0 2024-03-21T11:36:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.524+0000 7fe5fcc00640 1 -- 172.21.15.47:0/3773002054 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe60001c460 con 0x555ab6ff51e0 2024-03-21T11:36:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.524+0000 7fe5fcc00640 1 -- 172.21.15.47:0/3773002054 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe600045050 con 0x555ab6ff51e0 2024-03-21T11:36:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.524+0000 7fe618c6f740 10 client.29945 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.524+0000 7fe618c6f740 1 -- 172.21.15.47:0/3773002054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555ab706f9b0 con 0x555ab6ff51e0 2024-03-21T11:36:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.525+0000 7fe5fcc00640 1 -- 172.21.15.47:0/3773002054 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5f8024030 con 0x555ab6ff51e0 2024-03-21T11:36:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.525+0000 7fe5fcc00640 1 -- 172.21.15.47:0/3773002054 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe60001a040 con 0x555ab6ff51e0 2024-03-21T11:36:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.525+0000 7fe618c6f740 10 client.29945 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.525+0000 7fe618c6f740 10 client.29945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.525+0000 7fe618c6f740 20 client.29945 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.525+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3773002054 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ab70701f0 0x555ab7072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.525+0000 7fe618c6f740 4 client.29945 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:36:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.525+0000 7fe618c6f740 1 -- 172.21.15.47:0/3773002054 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555ab6f620c0 con 0x555ab70701f0 2024-03-21T11:36:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.525+0000 7fe60ea00640 1 --2- 172.21.15.47:0/3773002054 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ab70701f0 0x555ab7072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.526+0000 7fe60ea00640 1 --2- 172.21.15.47:0/3773002054 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ab70701f0 0x555ab7072610 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=863d4674a8c6506a server_cookie=5c13774813abf981 in_seq=0 out_seq=0 2024-03-21T11:36:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.526+0000 7fe5fcc00640 10 client.29945 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:34.581 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1010579 7.94 MB/sec execute 558 sec latency 60.527 ms 2024-03-21T11:36:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.796+0000 7fe5fcc00640 1 -- 172.21.15.47:0/3773002054 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555ab6f620c0 con 0x555ab70701f0 2024-03-21T11:36:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.796+0000 7fe5fcc00640 10 client.29945 handle_command_reply: tid=0 2024-03-21T11:36:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.796+0000 7fe618c6f740 1 client.29945 shutdown 2024-03-21T11:36:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.796+0000 7fe618c6f740 1 -- 172.21.15.47:0/3773002054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 msgr2=0x555ab7004600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.796+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3773002054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab7004600 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe600006010 tx=0x7fe60001cf40 comp rx=0 tx=0).stop 2024-03-21T11:36:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.797+0000 7fe618c6f740 1 -- 172.21.15.47:0/3773002054 shutdown_connections 2024-03-21T11:36:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.797+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3773002054 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ab70701f0 0x555ab7072610 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.797+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3773002054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ab6ffad40 0x555ab7005b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.797+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3773002054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ab6ff5bd0 0x555ab7004b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.797+0000 7fe618c6f740 1 --2- 172.21.15.47:0/3773002054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ab6ff51e0 0x555ab7004600 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.797+0000 7fe5fcc00640 0 client.29945 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.797+0000 7fe618c6f740 1 -- 172.21.15.47:0/3773002054 >> 172.21.15.47:0/3773002054 conn(0x555ab6ff0750 msgr2=0x555ab6ff39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.797+0000 7fe618c6f740 1 -- 172.21.15.47:0/3773002054 shutdown_connections 2024-03-21T11:36:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.797+0000 7fe618c6f740 1 -- 172.21.15.47:0/3773002054 wait complete. 2024-03-21T11:36:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.797+0000 7fe618c6f740 20 client.29945 trim_cache size 0 max 16384 2024-03-21T11:36:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:36:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:36:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:36:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:36:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.798+0000 7fe616000640 1 -- 172.21.15.47:0/1458032144 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe5f40a3040 msgr2=0x7fe5f40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.798+0000 7fe616000640 1 --2- 172.21.15.47:0/1458032144 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe5f40a3040 0x7fe5f40a5500 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x555ab6ff5fd0 tx=0x7fe600009680 comp rx=0 tx=0).stop 2024-03-21T11:36:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.798+0000 7fe616000640 1 -- 172.21.15.47:0/1458032144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6101073a0 msgr2=0x7fe6101a10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.798+0000 7fe616000640 1 --2- 172.21.15.47:0/1458032144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6101073a0 0x7fe6101a10e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe60802c7c0 tx=0x7fe608005dd0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.798+0000 7fe616000640 1 -- 172.21.15.47:0/1458032144 shutdown_connections 2024-03-21T11:36:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.799+0000 7fe616000640 1 --2- 172.21.15.47:0/1458032144 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe5f40a3040 0x7fe5f40a5500 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.799+0000 7fe616000640 1 --2- 172.21.15.47:0/1458032144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6101073a0 0x7fe6101a10e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.799+0000 7fe616000640 1 --2- 172.21.15.47:0/1458032144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe610104a70 0x7fe61019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.799+0000 7fe616000640 1 --2- 172.21.15.47:0/1458032144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe610102140 0x7fe610199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.799+0000 7fe616000640 1 -- 172.21.15.47:0/1458032144 >> 172.21.15.47:0/1458032144 conn(0x7fe6100fbb30 msgr2=0x7fe6101017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.799+0000 7fe616000640 1 -- 172.21.15.47:0/1458032144 shutdown_connections 2024-03-21T11:36:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:34.799+0000 7fe616000640 1 -- 172.21.15.47:0/1458032144 wait complete. 2024-03-21T11:36:35.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:35 smithi100 ceph-mon[36245]: pgmap v797: 129 pgs: 129 active+clean; 40 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 299 op/s 2024-03-21T11:36:35.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:35 smithi102 ceph-mon[36748]: pgmap v797: 129 pgs: 129 active+clean; 40 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 299 op/s 2024-03-21T11:36:35.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:35 smithi047 ceph-mon[31351]: pgmap v797: 129 pgs: 129 active+clean; 40 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 299 op/s 2024-03-21T11:36:35.580 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1012316 7.94 MB/sec execute 559 sec latency 163.689 ms 2024-03-21T11:36:36.581 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1013237 7.93 MB/sec execute 560 sec latency 25.088 ms 2024-03-21T11:36:37.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:37 smithi100 ceph-mon[36245]: pgmap v798: 129 pgs: 129 active+clean; 42 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-03-21T11:36:37.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:37 smithi102 ceph-mon[36748]: pgmap v798: 129 pgs: 129 active+clean; 42 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-03-21T11:36:37.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:37 smithi047 ceph-mon[31351]: pgmap v798: 129 pgs: 129 active+clean; 42 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-03-21T11:36:37.581 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1014995 7.93 MB/sec execute 561 sec latency 14.927 ms 2024-03-21T11:36:38.581 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1016441 7.94 MB/sec execute 562 sec latency 14.746 ms 2024-03-21T11:36:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:39 smithi100 ceph-mon[36245]: pgmap v799: 129 pgs: 129 active+clean; 44 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 241 op/s 2024-03-21T11:36:39.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:39 smithi102 ceph-mon[36748]: pgmap v799: 129 pgs: 129 active+clean; 44 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 241 op/s 2024-03-21T11:36:39.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:39 smithi047 ceph-mon[31351]: pgmap v799: 129 pgs: 129 active+clean; 44 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 241 op/s 2024-03-21T11:36:39.581 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1017281 7.93 MB/sec execute 563 sec latency 25.717 ms 2024-03-21T11:36:40.582 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1019254 7.93 MB/sec execute 564 sec latency 22.571 ms 2024-03-21T11:36:41.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:41 smithi100 ceph-mon[36245]: pgmap v800: 129 pgs: 129 active+clean; 36 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 281 op/s 2024-03-21T11:36:41.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:41 smithi102 ceph-mon[36748]: pgmap v800: 129 pgs: 129 active+clean; 36 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 281 op/s 2024-03-21T11:36:41.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:41 smithi047 ceph-mon[31351]: pgmap v800: 129 pgs: 129 active+clean; 36 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 281 op/s 2024-03-21T11:36:41.581 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1020281 7.93 MB/sec execute 565 sec latency 13.015 ms 2024-03-21T11:36:42.582 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1021926 7.93 MB/sec execute 566 sec latency 22.077 ms 2024-03-21T11:36:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:43 smithi100 ceph-mon[36245]: pgmap v801: 129 pgs: 129 active+clean; 37 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 257 op/s 2024-03-21T11:36:43.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:43 smithi102 ceph-mon[36748]: pgmap v801: 129 pgs: 129 active+clean; 37 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 257 op/s 2024-03-21T11:36:43.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:43 smithi047 ceph-mon[31351]: pgmap v801: 129 pgs: 129 active+clean; 37 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 257 op/s 2024-03-21T11:36:43.582 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1023387 7.93 MB/sec execute 567 sec latency 15.028 ms 2024-03-21T11:36:44.582 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1024118 7.92 MB/sec execute 568 sec latency 54.049 ms 2024-03-21T11:36:45.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:45 smithi100 ceph-mon[36245]: pgmap v802: 129 pgs: 129 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2024-03-21T11:36:45.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:45 smithi102 ceph-mon[36748]: pgmap v802: 129 pgs: 129 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2024-03-21T11:36:45.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:45 smithi047 ceph-mon[31351]: pgmap v802: 129 pgs: 129 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2024-03-21T11:36:45.582 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1026040 7.93 MB/sec execute 569 sec latency 17.202 ms 2024-03-21T11:36:46.582 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1027222 7.93 MB/sec execute 570 sec latency 13.151 ms 2024-03-21T11:36:47.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:47 smithi100 ceph-mon[36245]: pgmap v803: 129 pgs: 129 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2024-03-21T11:36:47.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:47 smithi102 ceph-mon[36748]: pgmap v803: 129 pgs: 129 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2024-03-21T11:36:47.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:47 smithi047 ceph-mon[31351]: pgmap v803: 129 pgs: 129 active+clean; 39 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2024-03-21T11:36:47.582 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1028562 7.92 MB/sec execute 571 sec latency 15.647 ms 2024-03-21T11:36:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.978+0000 7f93e3200640 1 -- 172.21.15.47:0/2431146692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e4103720 msgr2=0x7f93e4103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.978+0000 7f93e3200640 1 --2- 172.21.15.47:0/2431146692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e4103720 0x7f93e4103b40 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009900 tx=0x7f93d402bc30 comp rx=0 tx=0).stop 2024-03-21T11:36:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.978+0000 7f93e3200640 1 -- 172.21.15.47:0/2431146692 shutdown_connections 2024-03-21T11:36:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.978+0000 7f93e3200640 1 --2- 172.21.15.47:0/2431146692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e41052d0 0x7f93e410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.978+0000 7f93e3200640 1 --2- 172.21.15.47:0/2431146692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e4104910 0x7f93e4104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.978+0000 7f93e3200640 1 --2- 172.21.15.47:0/2431146692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e4103720 0x7f93e4103b40 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.978+0000 7f93e3200640 1 -- 172.21.15.47:0/2431146692 >> 172.21.15.47:0/2431146692 conn(0x7f93e40fecb0 msgr2=0x7f93e41010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.978+0000 7f93e3200640 1 -- 172.21.15.47:0/2431146692 shutdown_connections 2024-03-21T11:36:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.979+0000 7f93e3200640 1 -- 172.21.15.47:0/2431146692 wait complete. 2024-03-21T11:36:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.979+0000 7f93e3200640 1 Processor -- start 2024-03-21T11:36:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.980+0000 7f93e3200640 1 -- start start 2024-03-21T11:36:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.980+0000 7f93e3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e4103720 0x7f93e4199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.980+0000 7f93e3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e4104910 0x7f93e4199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.980+0000 7f93e3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e41052d0 0x7f93e41a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.980+0000 7f93e3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e410f3a0 con 0x7f93e41052d0 2024-03-21T11:36:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.980+0000 7f93e3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e410f220 con 0x7f93e4103720 2024-03-21T11:36:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.980+0000 7f93e3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e410f520 con 0x7f93e4104910 2024-03-21T11:36:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.980+0000 7f93e1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e4103720 0x7f93e4199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.980+0000 7f93e1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e4104910 0x7f93e4199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.980+0000 7f93e1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e4103720 0x7f93e4199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38188/0 (socket says 172.21.15.47:38188) 2024-03-21T11:36:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.980+0000 7f93e1e00640 1 -- 172.21.15.47:0/4152320049 learned_addr learned my addr 172.21.15.47:0/4152320049 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.980+0000 7f93e2800640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e41052d0 0x7f93e41a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.981+0000 7f93e1e00640 1 -- 172.21.15.47:0/4152320049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e4104910 msgr2=0x7f93e4199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.981+0000 7f93e1e00640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e4104910 0x7f93e4199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.981+0000 7f93e1e00640 1 -- 172.21.15.47:0/4152320049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e41052d0 msgr2=0x7f93e41a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.981+0000 7f93e1e00640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e41052d0 0x7f93e41a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.981+0000 7f93e1e00640 1 -- 172.21.15.47:0/4152320049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e41a1640 con 0x7f93e4103720 2024-03-21T11:36:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.981+0000 7f93e2800640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e41052d0 0x7f93e41a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.981+0000 7f93e1400640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e4104910 0x7f93e4199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.981+0000 7f93e1e00640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e4103720 0x7f93e4199a20 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f93d4000c00 tx=0x7f93d402c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.981+0000 7f93daa00640 1 -- 172.21.15.47:0/4152320049 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4041070 con 0x7f93e4103720 2024-03-21T11:36:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.981+0000 7f93e3200640 1 -- 172.21.15.47:0/4152320049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e41a18d0 con 0x7f93e4103720 2024-03-21T11:36:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.981+0000 7f93daa00640 1 -- 172.21.15.47:0/4152320049 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93d4004450 con 0x7f93e4103720 2024-03-21T11:36:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.981+0000 7f93daa00640 1 -- 172.21.15.47:0/4152320049 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4034440 con 0x7f93e4103720 2024-03-21T11:36:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.981+0000 7f93e3200640 1 -- 172.21.15.47:0/4152320049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e41a1dc0 con 0x7f93e4103720 2024-03-21T11:36:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.985+0000 7f93daa00640 1 -- 172.21.15.47:0/4152320049 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f93d4044070 con 0x7f93e4103720 2024-03-21T11:36:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.985+0000 7f93ebdaa740 1 Processor -- start 2024-03-21T11:36:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.985+0000 7f93ebdaa740 1 -- start start 2024-03-21T11:36:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.986+0000 7f93ebdaa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 0x556d52c87810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.986+0000 7f93ebdaa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c8c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.986+0000 7f93ebdaa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c8f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.986+0000 7f93ebdaa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d52c4a3b0 con 0x556d52c873f0 2024-03-21T11:36:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.986+0000 7f93ebdaa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d52bec830 con 0x556d52c87de0 2024-03-21T11:36:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.986+0000 7f93ebdaa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d52b955d0 con 0x556d52c8cf50 2024-03-21T11:36:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.986+0000 7f93e1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c8c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.986+0000 7f93e1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c8f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.986+0000 7f93e1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c8c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38204/0 (socket says 172.21.15.47:38204) 2024-03-21T11:36:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.986+0000 7f93e1e00640 1 -- 172.21.15.47:0/82245553 learned_addr learned my addr 172.21.15.47:0/82245553 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.986+0000 7f93e2800640 1 --2- 172.21.15.47:0/82245553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 0x556d52c87810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.986+0000 7f93daa00640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93c80a2f10 0x7f93c80a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.987+0000 7f93e1e00640 1 -- 172.21.15.47:0/82245553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 msgr2=0x556d52c8f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.987+0000 7f93e1e00640 1 --2- 172.21.15.47:0/82245553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c8f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.987+0000 7f93e1e00640 1 -- 172.21.15.47:0/82245553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 msgr2=0x556d52c87810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.987+0000 7f93e1e00640 1 --2- 172.21.15.47:0/82245553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 0x556d52c87810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.987+0000 7f93e1e00640 1 -- 172.21.15.47:0/82245553 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d52bad2c0 con 0x556d52c87de0 2024-03-21T11:36:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.987+0000 7f93daa00640 1 -- 172.21.15.47:0/4152320049 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f93d40f1360 con 0x7f93e4103720 2024-03-21T11:36:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.987+0000 7f93e2800640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93c80a2f10 0x7f93c80a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.987+0000 7f93e1e00640 1 --2- 172.21.15.47:0/82245553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c8c8e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f93d40b6b50 tx=0x7f93d4031930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc9fa22bb62539fc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.987+0000 7f93cf400640 1 -- 172.21.15.47:0/82245553 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d40bd500 con 0x556d52c87de0 2024-03-21T11:36:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.987+0000 7f93cf400640 1 -- 172.21.15.47:0/82245553 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93d40b6530 con 0x556d52c87de0 2024-03-21T11:36:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.988+0000 7f93cf400640 1 -- 172.21.15.47:0/82245553 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d40bd090 con 0x556d52c87de0 2024-03-21T11:36:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.988+0000 7f93ebdaa740 1 -- 172.21.15.47:0/82245553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 msgr2=0x556d52c8c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.988+0000 7f93e2800640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93c80a2f10 0x7f93c80a53d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x556d52c88640 tx=0x7f93dc004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.988+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/82245553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c8c8e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f93d40b6b50 tx=0x7f93d4031930 comp rx=0 tx=0).stop 2024-03-21T11:36:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.988+0000 7f93ebdaa740 1 -- 172.21.15.47:0/82245553 shutdown_connections 2024-03-21T11:36:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.988+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/82245553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c8f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.988+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/82245553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c8c8e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.988+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/82245553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 0x556d52c87810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.988+0000 7f93ebdaa740 1 -- 172.21.15.47:0/82245553 >> 172.21.15.47:0/82245553 conn(0x556d52c82960 msgr2=0x556d52c82d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.989+0000 7f93ebdaa740 1 -- 172.21.15.47:0/82245553 shutdown_connections 2024-03-21T11:36:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.989+0000 7f93ebdaa740 1 -- 172.21.15.47:0/82245553 wait complete. 2024-03-21T11:36:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.989+0000 7f93ebdaa740 1 -- start start 2024-03-21T11:36:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.989+0000 7f93ebdaa740 1 Processor -- start 2024-03-21T11:36:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93ebdaa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c873f0 0x556d52c911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93ebdaa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c87de0 0x556d52c91700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93ebdaa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c98740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93ebdaa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d52ba8a20 con 0x556d52c87de0 2024-03-21T11:36:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93ebdaa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d52be88d0 con 0x556d52c873f0 2024-03-21T11:36:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93ebdaa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d52c8fcc0 con 0x556d52c8cf50 2024-03-21T11:36:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93ebdaa740 10 client.0 _ll_register_callbacks cb 0x556d52be51c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93e1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c873f0 0x556d52c911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93e1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c873f0 0x556d52c911c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38212/0 (socket says 172.21.15.47:38212) 2024-03-21T11:36:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93e1e00640 1 -- 172.21.15.47:0/1660489736 learned_addr learned my addr 172.21.15.47:0/1660489736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93e2800640 1 --2- 172.21.15.47:0/1660489736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c98740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93e1400640 1 --2- 172.21.15.47:0/1660489736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c87de0 0x556d52c91700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93e1e00640 1 -- 172.21.15.47:0/1660489736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 msgr2=0x556d52c98740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93e1e00640 1 --2- 172.21.15.47:0/1660489736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c98740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93e1e00640 1 -- 172.21.15.47:0/1660489736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c87de0 msgr2=0x556d52c91700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93e1e00640 1 --2- 172.21.15.47:0/1660489736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c87de0 0x556d52c91700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93e1e00640 1 -- 172.21.15.47:0/1660489736 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d52901210 con 0x556d52c873f0 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.990+0000 7f93e1400640 1 --2- 172.21.15.47:0/1660489736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c87de0 0x556d52c91700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.991+0000 7f93e1e00640 1 --2- 172.21.15.47:0/1660489736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c873f0 0x556d52c911c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f93d402cec0 tx=0x7f93d4005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb4e5067b7cb4362 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.991+0000 7f93c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.991+0000 7f93c3e00640 1 -- 172.21.15.47:0/1660489736 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d40bd270 con 0x556d52c873f0 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.991+0000 7f93c3e00640 1 -- 172.21.15.47:0/1660489736 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f93d40a80c0 con 0x556d52c873f0 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.991+0000 7f93ebdaa740 1 -- 172.21.15.47:0/1660489736 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d52bf9d70 con 0x556d52c873f0 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.991+0000 7f93c3e00640 1 -- 172.21.15.47:0/1660489736 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93d40bd090 con 0x556d52c873f0 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.991+0000 7f93ebdaa740 10 client.29957 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.991+0000 7f93ebdaa740 1 -- 172.21.15.47:0/1660489736 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556d529cd620 con 0x556d52c873f0 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.991+0000 7f93c3e00640 1 -- 172.21.15.47:0/1660489736 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556d52bf9d70 con 0x556d52c873f0 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.992+0000 7f93c3e00640 1 -- 172.21.15.47:0/1660489736 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f93d40eeaf0 con 0x556d52c873f0 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.992+0000 7f93ebdaa740 10 client.29957 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.992+0000 7f93ebdaa740 10 client.29957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.992+0000 7f93ebdaa740 20 client.29957 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.992+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/1660489736 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d52c9ccf0 0x556d52c9f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.992+0000 7f93ebdaa740 4 client.29957 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.992+0000 7f93ebdaa740 1 -- 172.21.15.47:0/1660489736 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d52bf40c0 con 0x556d52c9ccf0 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.993+0000 7f93e1400640 1 --2- 172.21.15.47:0/1660489736 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d52c9ccf0 0x556d52c9f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.993+0000 7f93c3e00640 10 client.29957 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.993+0000 7f93e1400640 1 --2- 172.21.15.47:0/1660489736 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d52c9ccf0 0x556d52c9f0e0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dec6d07022268f5 server_cookie=1c7a65c8dd77a5f7 in_seq=0 out_seq=0 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.994+0000 7f93c3e00640 1 -- 172.21.15.47:0/1660489736 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556d52bf40c0 con 0x556d52c9ccf0 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.994+0000 7f93c3e00640 10 client.29957 handle_command_reply: tid=0 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.995+0000 7f93ebdaa740 1 client.29957 shutdown 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.995+0000 7f93ebdaa740 1 -- 172.21.15.47:0/1660489736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c873f0 msgr2=0x556d52c911c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.995+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/1660489736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c873f0 0x556d52c911c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f93d402cec0 tx=0x7f93d4005b70 comp rx=0 tx=0).stop 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.995+0000 7f93ebdaa740 1 -- 172.21.15.47:0/1660489736 shutdown_connections 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.995+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/1660489736 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d52c9ccf0 0x556d52c9f0e0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.995+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/1660489736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c98740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.995+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/1660489736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c87de0 0x556d52c91700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.995+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/1660489736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c873f0 0x556d52c911c0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.995+0000 7f93c3e00640 0 client.29957 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.995+0000 7f93ebdaa740 1 -- 172.21.15.47:0/1660489736 >> 172.21.15.47:0/1660489736 conn(0x556d52c82960 msgr2=0x556d52c81b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.995+0000 7f93ebdaa740 1 -- 172.21.15.47:0/1660489736 shutdown_connections 2024-03-21T11:36:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.995+0000 7f93ebdaa740 1 -- 172.21.15.47:0/1660489736 wait complete. 2024-03-21T11:36:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:47.995+0000 7f93ebdaa740 20 client.29957 trim_cache size 0 max 16384 2024-03-21T11:36:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.011+0000 7f93ebdaa740 1 Processor -- start 2024-03-21T11:36:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.011+0000 7f93ebdaa740 1 -- start start 2024-03-21T11:36:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.011+0000 7f93ebdaa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 0x556d52c90dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.011+0000 7f93ebdaa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c986d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.011+0000 7f93ebdaa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c91310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.011+0000 7f93ebdaa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41ac120 con 0x556d52c873f0 2024-03-21T11:36:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.011+0000 7f93ebdaa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41adaa0 con 0x556d52c87de0 2024-03-21T11:36:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.011+0000 7f93ebdaa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41ac780 con 0x556d52c8cf50 2024-03-21T11:36:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.011+0000 7f93e2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c91310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.011+0000 7f93e1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.011+0000 7f93e1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38224/0 (socket says 172.21.15.47:38224) 2024-03-21T11:36:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.011+0000 7f93e2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c91310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36950/0 (socket says 172.21.15.47:36950) 2024-03-21T11:36:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.011+0000 7f93e1400640 1 -- 172.21.15.47:0/4168783031 learned_addr learned my addr 172.21.15.47:0/4168783031 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.011+0000 7f93e1e00640 1 --2- 172.21.15.47:0/4168783031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 0x556d52c90dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.012+0000 7f93e2800640 1 -- 172.21.15.47:0/4168783031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 msgr2=0x556d52c986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.012+0000 7f93e2800640 1 --2- 172.21.15.47:0/4168783031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.012+0000 7f93e2800640 1 -- 172.21.15.47:0/4168783031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 msgr2=0x556d52c90dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.012+0000 7f93e2800640 1 --2- 172.21.15.47:0/4168783031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 0x556d52c90dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.012+0000 7f93e2800640 1 -- 172.21.15.47:0/4168783031 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d52c7dd10 con 0x556d52c8cf50 2024-03-21T11:36:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.012+0000 7f93e2800640 1 --2- 172.21.15.47:0/4168783031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c91310 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f93dc014010 tx=0x7f93dc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c583a2dff4690fd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.012+0000 7f93cea00640 1 -- 172.21.15.47:0/4168783031 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc011de0 con 0x556d52c8cf50 2024-03-21T11:36:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.013+0000 7f93cea00640 1 -- 172.21.15.47:0/4168783031 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93dc01c0d0 con 0x556d52c8cf50 2024-03-21T11:36:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.013+0000 7f93cea00640 1 -- 172.21.15.47:0/4168783031 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0051a0 con 0x556d52c8cf50 2024-03-21T11:36:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.013+0000 7f93ebdaa740 1 -- 172.21.15.47:0/4168783031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 msgr2=0x556d52c91310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.013+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/4168783031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c91310 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f93dc014010 tx=0x7f93dc017000 comp rx=0 tx=0).stop 2024-03-21T11:36:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.013+0000 7f93ebdaa740 1 -- 172.21.15.47:0/4168783031 shutdown_connections 2024-03-21T11:36:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.013+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/4168783031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c91310 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.013+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/4168783031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.013+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/4168783031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 0x556d52c90dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.013+0000 7f93ebdaa740 1 -- 172.21.15.47:0/4168783031 >> 172.21.15.47:0/4168783031 conn(0x556d52c82960 msgr2=0x556d52c82d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.014+0000 7f93ebdaa740 1 -- 172.21.15.47:0/4168783031 shutdown_connections 2024-03-21T11:36:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.014+0000 7f93ebdaa740 1 -- 172.21.15.47:0/4168783031 wait complete. 2024-03-21T11:36:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.014+0000 7f93ebdaa740 1 -- start start 2024-03-21T11:36:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.014+0000 7f93ebdaa740 1 Processor -- start 2024-03-21T11:36:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.015+0000 7f93ebdaa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 0x556d52c981a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.015+0000 7f93ebdaa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c986e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.015+0000 7f93ebdaa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c96860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.015+0000 7f93ebdaa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41ad440 con 0x556d52c873f0 2024-03-21T11:36:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.015+0000 7f93ebdaa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41acde0 con 0x556d52c87de0 2024-03-21T11:36:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.015+0000 7f93ebdaa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41aa2e0 con 0x556d52c8cf50 2024-03-21T11:36:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.015+0000 7f93ebdaa740 10 client.0 _ll_register_callbacks cb 0x7f93d0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.015+0000 7f93e1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.015+0000 7f93e2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c96860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.015+0000 7f93e1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 0x556d52c981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.015+0000 7f93e1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38240/0 (socket says 172.21.15.47:38240) 2024-03-21T11:36:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.015+0000 7f93e1400640 1 -- 172.21.15.47:0/4269806627 learned_addr learned my addr 172.21.15.47:0/4269806627 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.016+0000 7f93e1400640 1 -- 172.21.15.47:0/4269806627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 msgr2=0x556d52c96860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.016+0000 7f93e1400640 1 --2- 172.21.15.47:0/4269806627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c96860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.016+0000 7f93e1400640 1 -- 172.21.15.47:0/4269806627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 msgr2=0x556d52c981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.016+0000 7f93e1400640 1 --2- 172.21.15.47:0/4269806627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 0x556d52c981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.016+0000 7f93e1400640 1 -- 172.21.15.47:0/4269806627 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d52c97090 con 0x556d52c87de0 2024-03-21T11:36:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.016+0000 7f93e1400640 1 --2- 172.21.15.47:0/4269806627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c986e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f93e419af30 tx=0x7f93d0005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3899276b3b7224ee server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.016+0000 7f93c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.016+0000 7f93c3e00640 1 -- 172.21.15.47:0/4269806627 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0026060 con 0x556d52c87de0 2024-03-21T11:36:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.016+0000 7f93ebdaa740 1 -- 172.21.15.47:0/4269806627 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93dc01c0d0 con 0x556d52c87de0 2024-03-21T11:36:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.016+0000 7f93c3e00640 1 -- 172.21.15.47:0/4269806627 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f93d0004250 con 0x556d52c87de0 2024-03-21T11:36:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.016+0000 7f93c3e00640 1 -- 172.21.15.47:0/4269806627 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93d002d050 con 0x556d52c87de0 2024-03-21T11:36:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.016+0000 7f93ebdaa740 10 client.29963 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.017+0000 7f93ebdaa740 1 -- 172.21.15.47:0/4269806627 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556d52cf9260 con 0x556d52c87de0 2024-03-21T11:36:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.017+0000 7f93c3e00640 1 -- 172.21.15.47:0/4269806627 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93dc01c0d0 con 0x556d52c87de0 2024-03-21T11:36:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.017+0000 7f93c3e00640 1 -- 172.21.15.47:0/4269806627 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f93d001e3f0 con 0x556d52c87de0 2024-03-21T11:36:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.017+0000 7f93ebdaa740 10 client.29963 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.017+0000 7f93ebdaa740 10 client.29963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.017+0000 7f93ebdaa740 20 client.29963 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.017+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/4269806627 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d52cf9ba0 0x556d52cfbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.017+0000 7f93ebdaa740 4 client.29963 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:36:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.017+0000 7f93ebdaa740 1 -- 172.21.15.47:0/4269806627 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556d52b725d0 con 0x556d52cf9ba0 2024-03-21T11:36:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.017+0000 7f93e1e00640 1 --2- 172.21.15.47:0/4269806627 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d52cf9ba0 0x556d52cfbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.018+0000 7f93e1e00640 1 --2- 172.21.15.47:0/4269806627 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d52cf9ba0 0x556d52cfbfc0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44035a20e017fb41 server_cookie=45a85092c3e38fe5 in_seq=0 out_seq=0 2024-03-21T11:36:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.018+0000 7f93c3e00640 10 client.29963 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.301+0000 7f93c3e00640 1 -- 172.21.15.47:0/4269806627 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556d52b725d0 con 0x556d52cf9ba0 2024-03-21T11:36:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.301+0000 7f93c3e00640 10 client.29963 handle_command_reply: tid=0 2024-03-21T11:36:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.301+0000 7f93ebdaa740 1 client.29963 shutdown 2024-03-21T11:36:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.301+0000 7f93ebdaa740 1 -- 172.21.15.47:0/4269806627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 msgr2=0x556d52c986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.301+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/4269806627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c986e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f93e419af30 tx=0x7f93d0005f10 comp rx=0 tx=0).stop 2024-03-21T11:36:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.301+0000 7f93ebdaa740 1 -- 172.21.15.47:0/4269806627 shutdown_connections 2024-03-21T11:36:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.301+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/4269806627 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556d52cf9ba0 0x556d52cfbfc0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.302+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/4269806627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556d52c8cf50 0x556d52c96860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.302+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/4269806627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556d52c87de0 0x556d52c986e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.302+0000 7f93ebdaa740 1 --2- 172.21.15.47:0/4269806627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d52c873f0 0x556d52c981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.302+0000 7f93c3e00640 0 client.29963 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.302+0000 7f93ebdaa740 1 -- 172.21.15.47:0/4269806627 >> 172.21.15.47:0/4269806627 conn(0x556d52c82960 msgr2=0x556d52c90db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.302+0000 7f93ebdaa740 1 -- 172.21.15.47:0/4269806627 shutdown_connections 2024-03-21T11:36:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.302+0000 7f93ebdaa740 1 -- 172.21.15.47:0/4269806627 wait complete. 2024-03-21T11:36:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.302+0000 7f93ebdaa740 20 client.29963 trim_cache size 0 max 16384 2024-03-21T11:36:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:36:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:36:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:36:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:36:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.303+0000 7f93e3200640 1 -- 172.21.15.47:0/4152320049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93c80a2f10 msgr2=0x7f93c80a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.303+0000 7f93e3200640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93c80a2f10 0x7f93c80a53d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x556d52c88640 tx=0x7f93dc004a40 comp rx=0 tx=0).stop 2024-03-21T11:36:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.303+0000 7f93e3200640 1 -- 172.21.15.47:0/4152320049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e4103720 msgr2=0x7f93e4199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.303+0000 7f93e3200640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e4103720 0x7f93e4199a20 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f93d4000c00 tx=0x7f93d402c640 comp rx=0 tx=0).stop 2024-03-21T11:36:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.303+0000 7f93e3200640 1 -- 172.21.15.47:0/4152320049 shutdown_connections 2024-03-21T11:36:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.303+0000 7f93e3200640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93c80a2f10 0x7f93c80a53d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.303+0000 7f93e3200640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e41052d0 0x7f93e41a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.303+0000 7f93e3200640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e4104910 0x7f93e4199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.303+0000 7f93e3200640 1 --2- 172.21.15.47:0/4152320049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e4103720 0x7f93e4199a20 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.303+0000 7f93e3200640 1 -- 172.21.15.47:0/4152320049 >> 172.21.15.47:0/4152320049 conn(0x7f93e40fecb0 msgr2=0x7f93e4107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.303+0000 7f93e3200640 1 -- 172.21.15.47:0/4152320049 shutdown_connections 2024-03-21T11:36:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:36:48.303+0000 7f93e3200640 1 -- 172.21.15.47:0/4152320049 wait complete. 2024-03-21T11:36:48.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:48 smithi102 ceph-mon[36748]: pgmap v804: 129 pgs: 129 active+clean; 38 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 250 op/s 2024-03-21T11:36:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:48 smithi047 ceph-mon[31351]: pgmap v804: 129 pgs: 129 active+clean; 38 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 250 op/s 2024-03-21T11:36:48.582 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1030048 7.92 MB/sec execute 572 sec latency 157.397 ms 2024-03-21T11:36:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:48 smithi100 ceph-mon[36245]: pgmap v804: 129 pgs: 129 active+clean; 38 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 250 op/s 2024-03-21T11:36:49.582 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1030954 7.92 MB/sec execute 573 sec latency 49.041 ms 2024-03-21T11:36:50.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:36:50.203 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:36:50.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.437+0000 7f35c1600640 1 -- 172.21.15.47:0/1220766918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35bc1005a0 msgr2=0x7f35bc1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.437+0000 7f35c1600640 1 --2- 172.21.15.47:0/1220766918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35bc1005a0 0x7f35bc1009c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009900 tx=0x7f35ac02bc50 comp rx=0 tx=0).stop 2024-03-21T11:36:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.437+0000 7f35c1600640 1 -- 172.21.15.47:0/1220766918 shutdown_connections 2024-03-21T11:36:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.437+0000 7f35c1600640 1 --2- 172.21.15.47:0/1220766918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35bc102150 0x7f35bc1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.437+0000 7f35c1600640 1 --2- 172.21.15.47:0/1220766918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35bc101790 0x7f35bc101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.437+0000 7f35c1600640 1 --2- 172.21.15.47:0/1220766918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35bc1005a0 0x7f35bc1009c0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.437+0000 7f35c1600640 1 -- 172.21.15.47:0/1220766918 >> 172.21.15.47:0/1220766918 conn(0x7f35bc0fbb30 msgr2=0x7f35bc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.437+0000 7f35c1600640 1 -- 172.21.15.47:0/1220766918 shutdown_connections 2024-03-21T11:36:50.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.437+0000 7f35c1600640 1 -- 172.21.15.47:0/1220766918 wait complete. 2024-03-21T11:36:50.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.438+0000 7f35c1600640 1 Processor -- start 2024-03-21T11:36:50.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.438+0000 7f35c1600640 1 -- start start 2024-03-21T11:36:50.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35c1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35bc1005a0 0x7f35bc199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35c1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35bc101790 0x7f35bc19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35c1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35bc102150 0x7f35bc1a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35c1600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc10b140 con 0x7f35bc102150 2024-03-21T11:36:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35c1600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc10afc0 con 0x7f35bc101790 2024-03-21T11:36:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35c1600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc10b2c0 con 0x7f35bc1005a0 2024-03-21T11:36:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35baa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35bc1005a0 0x7f35bc199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35baa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35bc1005a0 0x7f35bc199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37930/0 (socket says 172.21.15.47:37930) 2024-03-21T11:36:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35baa00640 1 -- 172.21.15.47:0/1813280414 learned_addr learned my addr 172.21.15.47:0/1813280414 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35b2000640 1 --2- 172.21.15.47:0/1813280414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35bc101790 0x7f35bc19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35bb400640 1 --2- 172.21.15.47:0/1813280414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35bc102150 0x7f35bc1a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35baa00640 1 -- 172.21.15.47:0/1813280414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35bc101790 msgr2=0x7f35bc19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35baa00640 1 --2- 172.21.15.47:0/1813280414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35bc101790 0x7f35bc19a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35baa00640 1 -- 172.21.15.47:0/1813280414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35bc102150 msgr2=0x7f35bc1a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35baa00640 1 --2- 172.21.15.47:0/1813280414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35bc102150 0x7f35bc1a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.439+0000 7f35baa00640 1 -- 172.21.15.47:0/1813280414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc1a18c0 con 0x7f35bc1005a0 2024-03-21T11:36:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.440+0000 7f35baa00640 1 --2- 172.21.15.47:0/1813280414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35bc1005a0 0x7f35bc199c80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f35ac000c00 tx=0x7f35ac005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:50.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.440+0000 7f35b3e00640 1 -- 172.21.15.47:0/1813280414 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac042020 con 0x7f35bc1005a0 2024-03-21T11:36:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.440+0000 7f35b3e00640 1 -- 172.21.15.47:0/1813280414 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35ac0044d0 con 0x7f35bc1005a0 2024-03-21T11:36:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.440+0000 7f35c1600640 1 -- 172.21.15.47:0/1813280414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35bc1a1b50 con 0x7f35bc1005a0 2024-03-21T11:36:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.440+0000 7f35b3e00640 1 -- 172.21.15.47:0/1813280414 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac03e460 con 0x7f35bc1005a0 2024-03-21T11:36:50.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.440+0000 7f35c1600640 1 -- 172.21.15.47:0/1813280414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35bc1a20f0 con 0x7f35bc1005a0 2024-03-21T11:36:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.445+0000 7f35c4201740 1 Processor -- start 2024-03-21T11:36:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.445+0000 7f35b3e00640 1 -- 172.21.15.47:0/1813280414 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f35ac045020 con 0x7f35bc1005a0 2024-03-21T11:36:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.445+0000 7f35c4201740 1 -- start start 2024-03-21T11:36:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.445+0000 7f35c4201740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.445+0000 7f35c4201740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3460cf870 0x55b3460d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.445+0000 7f35c4201740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460d49e0 0x55b3460d6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.445+0000 7f35c4201740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b346034250 con 0x55b3460d49e0 2024-03-21T11:36:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.445+0000 7f35c4201740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b345fdca30 con 0x55b3460cee80 2024-03-21T11:36:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.445+0000 7f35c4201740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3460302f0 con 0x55b3460cf870 2024-03-21T11:36:50.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.446+0000 7f35baa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3460cf870 0x55b3460d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.446+0000 7f35bb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.446+0000 7f35bb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59900/0 (socket says 172.21.15.47:59900) 2024-03-21T11:36:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.446+0000 7f35bb400640 1 -- 172.21.15.47:0/1571070314 learned_addr learned my addr 172.21.15.47:0/1571070314 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.446+0000 7f35baa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3460cf870 0x55b3460d4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37946/0 (socket says 172.21.15.47:37946) 2024-03-21T11:36:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.446+0000 7f35b2000640 1 --2- 172.21.15.47:0/1571070314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460d49e0 0x55b3460d6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.446+0000 7f35bb400640 1 -- 172.21.15.47:0/1571070314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3460cf870 msgr2=0x55b3460d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.446+0000 7f35bb400640 1 --2- 172.21.15.47:0/1571070314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3460cf870 0x55b3460d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.446+0000 7f35bb400640 1 -- 172.21.15.47:0/1571070314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460d49e0 msgr2=0x55b3460d6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.446+0000 7f35bb400640 1 --2- 172.21.15.47:0/1571070314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460d49e0 0x55b3460d6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.446+0000 7f35bb400640 1 -- 172.21.15.47:0/1571070314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b345fc7690 con 0x55b3460cee80 2024-03-21T11:36:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.446+0000 7f35baa00640 1 --2- 172.21.15.47:0/1571070314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3460cf870 0x55b3460d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.446+0000 7f35bb400640 1 --2- 172.21.15.47:0/1571070314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460cf2a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009c10 tx=0x7f35b40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24a68fdc53c3d27c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:50.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.447+0000 7f35ab400640 1 -- 172.21.15.47:0/1571070314 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4012450 con 0x55b3460cee80 2024-03-21T11:36:50.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.447+0000 7f35b3e00640 1 --2- 172.21.15.47:0/1813280414 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35a00a2f80 0x7f35a00a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.447+0000 7f35ab400640 1 -- 172.21.15.47:0/1571070314 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35b4012ad0 con 0x55b3460cee80 2024-03-21T11:36:50.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.447+0000 7f35ab400640 1 -- 172.21.15.47:0/1571070314 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b400fb60 con 0x55b3460cee80 2024-03-21T11:36:50.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.447+0000 7f35b3e00640 1 -- 172.21.15.47:0/1813280414 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f35ac0f17e0 con 0x7f35bc1005a0 2024-03-21T11:36:50.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.447+0000 7f35baa00640 1 --2- 172.21.15.47:0/1813280414 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35a00a2f80 0x7f35a00a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.448+0000 7f35c4201740 1 -- 172.21.15.47:0/1571070314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 msgr2=0x55b3460cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:50.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.448+0000 7f35c4201740 1 --2- 172.21.15.47:0/1571070314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460cf2a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009c10 tx=0x7f35b40117b0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.448+0000 7f35c4201740 1 -- 172.21.15.47:0/1571070314 shutdown_connections 2024-03-21T11:36:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.448+0000 7f35c4201740 1 --2- 172.21.15.47:0/1571070314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460d49e0 0x55b3460d6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.448+0000 7f35c4201740 1 --2- 172.21.15.47:0/1571070314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3460cf870 0x55b3460d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.448+0000 7f35c4201740 1 --2- 172.21.15.47:0/1571070314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460cf2a0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.448+0000 7f35c4201740 1 -- 172.21.15.47:0/1571070314 >> 172.21.15.47:0/1571070314 conn(0x55b346097200 msgr2=0x55b3460ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.448+0000 7f35baa00640 1 --2- 172.21.15.47:0/1813280414 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35a00a2f80 0x7f35a00a5440 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f35ac040000 tx=0x7f35ac0f7530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.448+0000 7f35c4201740 1 -- 172.21.15.47:0/1571070314 shutdown_connections 2024-03-21T11:36:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.448+0000 7f35c4201740 1 -- 172.21.15.47:0/1571070314 wait complete. 2024-03-21T11:36:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.448+0000 7f35c4201740 1 -- start start 2024-03-21T11:36:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.449+0000 7f35c4201740 1 Processor -- start 2024-03-21T11:36:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.449+0000 7f35c4201740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3460cee80 0x55b3460d8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.449+0000 7f35c4201740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cf870 0x55b3460d9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.449+0000 7f35c4201740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460d49e0 0x55b3460e0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.449+0000 7f35c4201740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3460d7990 con 0x55b3460d49e0 2024-03-21T11:36:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.449+0000 7f35c4201740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b345fefe80 con 0x55b3460cf870 2024-03-21T11:36:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.449+0000 7f35c4201740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3460d7b10 con 0x55b3460cee80 2024-03-21T11:36:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.449+0000 7f35c4201740 10 client.0 _ll_register_callbacks cb 0x55b346002130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.449+0000 7f35b2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cf870 0x55b3460d9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.449+0000 7f35b2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cf870 0x55b3460d9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59906/0 (socket says 172.21.15.47:59906) 2024-03-21T11:36:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.449+0000 7f35b2000640 1 -- 172.21.15.47:0/2721699505 learned_addr learned my addr 172.21.15.47:0/2721699505 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.450+0000 7f35bb400640 1 --2- 172.21.15.47:0/2721699505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3460cee80 0x55b3460d8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.450+0000 7f35bb400640 1 --2- 172.21.15.47:0/2721699505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460d49e0 0x55b3460e0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.450+0000 7f35b2000640 1 -- 172.21.15.47:0/2721699505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3460cee80 msgr2=0x55b3460d8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.450+0000 7f35b2000640 1 --2- 172.21.15.47:0/2721699505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3460cee80 0x55b3460d8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.450+0000 7f35b2000640 1 -- 172.21.15.47:0/2721699505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460d49e0 msgr2=0x55b3460e0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.450+0000 7f35b2000640 1 --2- 172.21.15.47:0/2721699505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460d49e0 0x55b3460e0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.450+0000 7f35b2000640 1 -- 172.21.15.47:0/2721699505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b345e127d0 con 0x55b3460cf870 2024-03-21T11:36:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.450+0000 7f35bb400640 1 --2- 172.21.15.47:0/2721699505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460d49e0 0x55b3460e0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:36:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.450+0000 7f35b2000640 1 --2- 172.21.15.47:0/2721699505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cf870 0x55b3460d9310 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f35a4010a70 tx=0x7f35a4010f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca39dc4a10921b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.450+0000 7f359fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.450+0000 7f359fe00640 1 -- 172.21.15.47:0/2721699505 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a400fdd0 con 0x55b3460cf870 2024-03-21T11:36:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.450+0000 7f35c4201740 1 -- 172.21.15.47:0/2721699505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35b4012ad0 con 0x55b3460cf870 2024-03-21T11:36:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.450+0000 7f359fe00640 1 -- 172.21.15.47:0/2721699505 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f35a4004b20 con 0x55b3460cf870 2024-03-21T11:36:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.451+0000 7f359fe00640 1 -- 172.21.15.47:0/2721699505 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a4004d20 con 0x55b3460cf870 2024-03-21T11:36:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.451+0000 7f35c4201740 10 client.29975 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.451+0000 7f35c4201740 1 -- 172.21.15.47:0/2721699505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b345d483f0 con 0x55b3460cf870 2024-03-21T11:36:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.451+0000 7f359fe00640 1 -- 172.21.15.47:0/2721699505 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35b4012ad0 con 0x55b3460cf870 2024-03-21T11:36:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.451+0000 7f359fe00640 1 -- 172.21.15.47:0/2721699505 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f35a4019410 con 0x55b3460cf870 2024-03-21T11:36:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.451+0000 7f35c4201740 10 client.29975 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.451+0000 7f35c4201740 10 client.29975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.451+0000 7f35c4201740 20 client.29975 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.451+0000 7f35c4201740 1 --2- 172.21.15.47:0/2721699505 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b3460e4c10 0x55b3460e7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.451+0000 7f35c4201740 4 client.29975 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.451+0000 7f35c4201740 1 -- 172.21.15.47:0/2721699505 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b346041790 con 0x55b3460e4c10 2024-03-21T11:36:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.452+0000 7f35bb400640 1 --2- 172.21.15.47:0/2721699505 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b3460e4c10 0x55b3460e7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.452+0000 7f35bb400640 1 --2- 172.21.15.47:0/2721699505 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b3460e4c10 0x55b3460e7050 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c84f1d82630c586 server_cookie=a099c4fe9c43defe in_seq=0 out_seq=0 2024-03-21T11:36:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.452+0000 7f359fe00640 10 client.29975 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.453+0000 7f359fe00640 1 -- 172.21.15.47:0/2721699505 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b346041790 con 0x55b3460e4c10 2024-03-21T11:36:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.453+0000 7f359fe00640 10 client.29975 handle_command_reply: tid=0 2024-03-21T11:36:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.453+0000 7f35c4201740 1 client.29975 shutdown 2024-03-21T11:36:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.453+0000 7f35c4201740 1 -- 172.21.15.47:0/2721699505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cf870 msgr2=0x55b3460d9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.453+0000 7f35c4201740 1 --2- 172.21.15.47:0/2721699505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cf870 0x55b3460d9310 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f35a4010a70 tx=0x7f35a4010f30 comp rx=0 tx=0).stop 2024-03-21T11:36:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.453+0000 7f35c4201740 1 -- 172.21.15.47:0/2721699505 shutdown_connections 2024-03-21T11:36:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.453+0000 7f35c4201740 1 --2- 172.21.15.47:0/2721699505 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b3460e4c10 0x55b3460e7050 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.453+0000 7f35c4201740 1 --2- 172.21.15.47:0/2721699505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460d49e0 0x55b3460e0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.453+0000 7f35c4201740 1 --2- 172.21.15.47:0/2721699505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cf870 0x55b3460d9310 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.453+0000 7f35c4201740 1 --2- 172.21.15.47:0/2721699505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3460cee80 0x55b3460d8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.453+0000 7f359fe00640 0 client.29975 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.453+0000 7f35c4201740 1 -- 172.21.15.47:0/2721699505 >> 172.21.15.47:0/2721699505 conn(0x55b346097200 msgr2=0x55b3460c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.454+0000 7f35c4201740 1 -- 172.21.15.47:0/2721699505 shutdown_connections 2024-03-21T11:36:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.454+0000 7f35c4201740 1 -- 172.21.15.47:0/2721699505 wait complete. 2024-03-21T11:36:50.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.454+0000 7f35c4201740 20 client.29975 trim_cache size 0 max 16384 2024-03-21T11:36:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.472+0000 7f35c4201740 1 Processor -- start 2024-03-21T11:36:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.473+0000 7f35c4201740 1 -- start start 2024-03-21T11:36:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.473+0000 7f35c4201740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b346097200 0x55b3460cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.473+0000 7f35c4201740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460e0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.473+0000 7f35c4201740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460cf870 0x55b3460cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.473+0000 7f35c4201740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc1ac330 con 0x55b3460cf870 2024-03-21T11:36:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.473+0000 7f35c4201740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc1adcb0 con 0x55b3460cee80 2024-03-21T11:36:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.473+0000 7f35c4201740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc1ac990 con 0x55b346097200 2024-03-21T11:36:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.473+0000 7f35b2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460e0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.473+0000 7f35b2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460e0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59922/0 (socket says 172.21.15.47:59922) 2024-03-21T11:36:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.473+0000 7f35b2000640 1 -- 172.21.15.47:0/2800108853 learned_addr learned my addr 172.21.15.47:0/2800108853 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.473+0000 7f35bb400640 1 --2- 172.21.15.47:0/2800108853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b346097200 0x55b3460cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.473+0000 7f35bb400640 1 --2- 172.21.15.47:0/2800108853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460cf870 0x55b3460cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.474+0000 7f35b2000640 1 -- 172.21.15.47:0/2800108853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b346097200 msgr2=0x55b3460cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.474+0000 7f35b2000640 1 --2- 172.21.15.47:0/2800108853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b346097200 0x55b3460cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.474+0000 7f35b2000640 1 -- 172.21.15.47:0/2800108853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460cf870 msgr2=0x55b3460cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.474+0000 7f35b2000640 1 --2- 172.21.15.47:0/2800108853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460cf870 0x55b3460cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.474+0000 7f35b2000640 1 -- 172.21.15.47:0/2800108853 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b345d080f0 con 0x55b3460cee80 2024-03-21T11:36:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.474+0000 7f35b2000640 1 --2- 172.21.15.47:0/2800108853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460e0310 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f35a4019850 tx=0x7f35a4010730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d0a5c26b22904e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.474+0000 7f35aaa00640 1 -- 172.21.15.47:0/2800108853 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a400fe90 con 0x55b3460cee80 2024-03-21T11:36:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.474+0000 7f35aaa00640 1 -- 172.21.15.47:0/2800108853 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35a40049e0 con 0x55b3460cee80 2024-03-21T11:36:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.474+0000 7f35aaa00640 1 -- 172.21.15.47:0/2800108853 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4003b90 con 0x55b3460cee80 2024-03-21T11:36:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.475+0000 7f35c4201740 1 -- 172.21.15.47:0/2800108853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 msgr2=0x55b3460e0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.475+0000 7f35c4201740 1 --2- 172.21.15.47:0/2800108853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460e0310 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f35a4019850 tx=0x7f35a4010730 comp rx=0 tx=0).stop 2024-03-21T11:36:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.475+0000 7f35c4201740 1 -- 172.21.15.47:0/2800108853 shutdown_connections 2024-03-21T11:36:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.475+0000 7f35c4201740 1 --2- 172.21.15.47:0/2800108853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460cf870 0x55b3460cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.475+0000 7f35c4201740 1 --2- 172.21.15.47:0/2800108853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460e0310 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.475+0000 7f35c4201740 1 --2- 172.21.15.47:0/2800108853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b346097200 0x55b3460cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.475+0000 7f35c4201740 1 -- 172.21.15.47:0/2800108853 >> 172.21.15.47:0/2800108853 conn(0x55b3460d8990 msgr2=0x55b3460d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.475+0000 7f35c4201740 1 -- 172.21.15.47:0/2800108853 shutdown_connections 2024-03-21T11:36:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.475+0000 7f35c4201740 1 -- 172.21.15.47:0/2800108853 wait complete. 2024-03-21T11:36:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.476+0000 7f35c4201740 1 -- start start 2024-03-21T11:36:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.476+0000 7f35c4201740 1 Processor -- start 2024-03-21T11:36:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.476+0000 7f35c4201740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b346097200 0x55b3460dfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.476+0000 7f35c4201740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460e01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.476+0000 7f35c4201740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460cf870 0x55b3460de270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.476+0000 7f35c4201740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc1ad650 con 0x55b3460cf870 2024-03-21T11:36:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.476+0000 7f35c4201740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc1acff0 con 0x55b3460cee80 2024-03-21T11:36:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.476+0000 7f35c4201740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc1aa4f0 con 0x55b346097200 2024-03-21T11:36:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.476+0000 7f35c4201740 10 client.0 _ll_register_callbacks cb 0x7f35a400aa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.476+0000 7f35b2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460e01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.476+0000 7f35bb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b346097200 0x55b3460dfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.477+0000 7f35b2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460e01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59934/0 (socket says 172.21.15.47:59934) 2024-03-21T11:36:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.477+0000 7f35b2000640 1 -- 172.21.15.47:0/545634192 learned_addr learned my addr 172.21.15.47:0/545634192 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.477+0000 7f35bb400640 1 --2- 172.21.15.47:0/545634192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460cf870 0x55b3460de270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.477+0000 7f35b2000640 1 -- 172.21.15.47:0/545634192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b346097200 msgr2=0x55b3460dfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.477+0000 7f35b2000640 1 --2- 172.21.15.47:0/545634192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b346097200 0x55b3460dfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.477+0000 7f35b2000640 1 -- 172.21.15.47:0/545634192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460cf870 msgr2=0x55b3460de270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.477+0000 7f35b2000640 1 --2- 172.21.15.47:0/545634192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460cf870 0x55b3460de270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.477+0000 7f35b2000640 1 -- 172.21.15.47:0/545634192 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3460de9e0 con 0x55b3460cee80 2024-03-21T11:36:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.477+0000 7f35bb400640 1 --2- 172.21.15.47:0/545634192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460cf870 0x55b3460de270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.477+0000 7f35b2000640 1 --2- 172.21.15.47:0/545634192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460e01a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f35a400afd0 tx=0x7f35a4019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5125676118b7dedf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.477+0000 7f359fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.478+0000 7f359fe00640 1 -- 172.21.15.47:0/545634192 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4038020 con 0x55b3460cee80 2024-03-21T11:36:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.478+0000 7f359fe00640 1 -- 172.21.15.47:0/545634192 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f35a402f040 con 0x55b3460cee80 2024-03-21T11:36:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.478+0000 7f35c4201740 1 -- 172.21.15.47:0/545634192 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35a40049e0 con 0x55b3460cee80 2024-03-21T11:36:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.478+0000 7f359fe00640 1 -- 172.21.15.47:0/545634192 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a400fe90 con 0x55b3460cee80 2024-03-21T11:36:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.478+0000 7f35c4201740 10 client.29981 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.478+0000 7f35c4201740 1 -- 172.21.15.47:0/545634192 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b346140f50 con 0x55b3460cee80 2024-03-21T11:36:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.478+0000 7f359fe00640 1 -- 172.21.15.47:0/545634192 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35a40049e0 con 0x55b3460cee80 2024-03-21T11:36:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.478+0000 7f359fe00640 1 -- 172.21.15.47:0/545634192 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f35a4034450 con 0x55b3460cee80 2024-03-21T11:36:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.478+0000 7f35c4201740 10 client.29981 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.478+0000 7f35c4201740 10 client.29981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.478+0000 7f35c4201740 20 client.29981 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.479+0000 7f35c4201740 1 --2- 172.21.15.47:0/545634192 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b3460d49e0 0x55b346143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.479+0000 7f35c4201740 4 client.29981 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:36:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.479+0000 7f35c4201740 1 -- 172.21.15.47:0/545634192 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b34603bae0 con 0x55b3460d49e0 2024-03-21T11:36:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.479+0000 7f35bb400640 1 --2- 172.21.15.47:0/545634192 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b3460d49e0 0x55b346143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.479+0000 7f35bb400640 1 --2- 172.21.15.47:0/545634192 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b3460d49e0 0x55b346143860 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be9d701229dead90 server_cookie=fca04b8c0d024fbc in_seq=0 out_seq=0 2024-03-21T11:36:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.479+0000 7f359fe00640 10 client.29981 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.480+0000 7f359fe00640 1 -- 172.21.15.47:0/545634192 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b34603bae0 con 0x55b3460d49e0 2024-03-21T11:36:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.480+0000 7f359fe00640 10 client.29981 handle_command_reply: tid=0 2024-03-21T11:36:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.480+0000 7f35c4201740 1 client.29981 shutdown 2024-03-21T11:36:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.480+0000 7f35c4201740 1 -- 172.21.15.47:0/545634192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 msgr2=0x55b3460e01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.480+0000 7f35c4201740 1 --2- 172.21.15.47:0/545634192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460e01a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f35a400afd0 tx=0x7f35a4019640 comp rx=0 tx=0).stop 2024-03-21T11:36:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.480+0000 7f35c4201740 1 -- 172.21.15.47:0/545634192 shutdown_connections 2024-03-21T11:36:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.480+0000 7f35c4201740 1 --2- 172.21.15.47:0/545634192 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b3460d49e0 0x55b346143860 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.480+0000 7f35c4201740 1 --2- 172.21.15.47:0/545634192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b3460cf870 0x55b3460de270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.480+0000 7f35c4201740 1 --2- 172.21.15.47:0/545634192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3460cee80 0x55b3460e01a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.480+0000 7f359fe00640 0 client.29981 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.480+0000 7f35c4201740 1 --2- 172.21.15.47:0/545634192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b346097200 0x55b3460dfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.480+0000 7f35c4201740 1 -- 172.21.15.47:0/545634192 >> 172.21.15.47:0/545634192 conn(0x55b3460d8990 msgr2=0x55b3460c9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.480+0000 7f35c4201740 1 -- 172.21.15.47:0/545634192 shutdown_connections 2024-03-21T11:36:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.480+0000 7f35c4201740 1 -- 172.21.15.47:0/545634192 wait complete. 2024-03-21T11:36:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.481+0000 7f35c4201740 20 client.29981 trim_cache size 0 max 16384 2024-03-21T11:36:50.486 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:36:50.486 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:36:50.486 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:36:50.486 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:36:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.481+0000 7f35c1600640 1 -- 172.21.15.47:0/1813280414 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35a00a2f80 msgr2=0x7f35a00a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.481+0000 7f35c1600640 1 --2- 172.21.15.47:0/1813280414 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35a00a2f80 0x7f35a00a5440 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f35ac040000 tx=0x7f35ac0f7530 comp rx=0 tx=0).stop 2024-03-21T11:36:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.481+0000 7f35c1600640 1 -- 172.21.15.47:0/1813280414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35bc1005a0 msgr2=0x7f35bc199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.481+0000 7f35c1600640 1 --2- 172.21.15.47:0/1813280414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35bc1005a0 0x7f35bc199c80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f35ac000c00 tx=0x7f35ac005d50 comp rx=0 tx=0).stop 2024-03-21T11:36:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.481+0000 7f35c1600640 1 -- 172.21.15.47:0/1813280414 shutdown_connections 2024-03-21T11:36:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.481+0000 7f35c1600640 1 --2- 172.21.15.47:0/1813280414 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35a00a2f80 0x7f35a00a5440 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.481+0000 7f35c1600640 1 --2- 172.21.15.47:0/1813280414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35bc102150 0x7f35bc1a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.481+0000 7f35c1600640 1 --2- 172.21.15.47:0/1813280414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35bc101790 0x7f35bc19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.481+0000 7f35c1600640 1 --2- 172.21.15.47:0/1813280414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35bc1005a0 0x7f35bc199c80 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.481+0000 7f35c1600640 1 -- 172.21.15.47:0/1813280414 >> 172.21.15.47:0/1813280414 conn(0x7f35bc0fbb30 msgr2=0x7f35bc1907e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.482+0000 7f35c1600640 1 -- 172.21.15.47:0/1813280414 shutdown_connections 2024-03-21T11:36:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.482+0000 7f35c1600640 1 -- 172.21.15.47:0/1813280414 wait complete. 2024-03-21T11:36:50.502 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:36:50.502 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:36:50.582 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1032640 7.92 MB/sec execute 574 sec latency 25.231 ms 2024-03-21T11:36:50.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.718+0000 7fd128000640 1 -- 172.21.15.47:0/1769241512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120105490 msgr2=0x7fd12010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:50.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.718+0000 7fd128000640 1 --2- 172.21.15.47:0/1769241512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120105490 0x7fd12010bd20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd118009860 tx=0x7fd11802bb70 comp rx=0 tx=0).stop 2024-03-21T11:36:50.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.718+0000 7fd128000640 1 -- 172.21.15.47:0/1769241512 shutdown_connections 2024-03-21T11:36:50.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.718+0000 7fd128000640 1 --2- 172.21.15.47:0/1769241512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120105490 0x7fd12010bd20 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.718+0000 7fd128000640 1 --2- 172.21.15.47:0/1769241512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd120104ad0 0x7fd120104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.718+0000 7fd128000640 1 --2- 172.21.15.47:0/1769241512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1201038e0 0x7fd120103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.718+0000 7fd128000640 1 -- 172.21.15.47:0/1769241512 >> 172.21.15.47:0/1769241512 conn(0x7fd1200fee50 msgr2=0x7fd120101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:50.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.718+0000 7fd128000640 1 -- 172.21.15.47:0/1769241512 shutdown_connections 2024-03-21T11:36:50.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.718+0000 7fd128000640 1 -- 172.21.15.47:0/1769241512 wait complete. 2024-03-21T11:36:50.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.719+0000 7fd128000640 1 Processor -- start 2024-03-21T11:36:50.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.720+0000 7fd128000640 1 -- start start 2024-03-21T11:36:50.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.720+0000 7fd128000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1201038e0 0x7fd120199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.720+0000 7fd128000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd120104ad0 0x7fd12019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.720+0000 7fd128000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120105490 0x7fd1201a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.720+0000 7fd128000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12010f540 con 0x7fd120104ad0 2024-03-21T11:36:50.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.720+0000 7fd128000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12010f3c0 con 0x7fd120105490 2024-03-21T11:36:50.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.720+0000 7fd128000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12010f6c0 con 0x7fd1201038e0 2024-03-21T11:36:50.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.720+0000 7fd126000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120105490 0x7fd1201a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.720+0000 7fd125600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1201038e0 0x7fd120199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.720+0000 7fd126000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120105490 0x7fd1201a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59954/0 (socket says 172.21.15.47:59954) 2024-03-21T11:36:50.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.720+0000 7fd125600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1201038e0 0x7fd120199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37982/0 (socket says 172.21.15.47:37982) 2024-03-21T11:36:50.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.720+0000 7fd126000640 1 -- 172.21.15.47:0/963730063 learned_addr learned my addr 172.21.15.47:0/963730063 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:50.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.720+0000 7fd124c00640 1 --2- 172.21.15.47:0/963730063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd120104ad0 0x7fd12019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.721+0000 7fd126000640 1 -- 172.21.15.47:0/963730063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1201038e0 msgr2=0x7fd120199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:50.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.721+0000 7fd126000640 1 --2- 172.21.15.47:0/963730063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1201038e0 0x7fd120199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.721+0000 7fd126000640 1 -- 172.21.15.47:0/963730063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd120104ad0 msgr2=0x7fd12019a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:50.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.721+0000 7fd126000640 1 --2- 172.21.15.47:0/963730063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd120104ad0 0x7fd12019a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:50.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.721+0000 7fd126000640 1 -- 172.21.15.47:0/963730063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1201a17a0 con 0x7fd120105490 2024-03-21T11:36:50.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.721+0000 7fd124c00640 1 --2- 172.21.15.47:0/963730063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd120104ad0 0x7fd12019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.721+0000 7fd126000640 1 --2- 172.21.15.47:0/963730063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120105490 0x7fd1201a1130 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd11802c180 tx=0x7fd118032460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.721+0000 7fd11e000640 1 -- 172.21.15.47:0/963730063 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118041020 con 0x7fd120105490 2024-03-21T11:36:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.721+0000 7fd11e000640 1 -- 172.21.15.47:0/963730063 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd11802c9f0 con 0x7fd120105490 2024-03-21T11:36:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.721+0000 7fd11e000640 1 -- 172.21.15.47:0/963730063 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118039040 con 0x7fd120105490 2024-03-21T11:36:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.721+0000 7fd128000640 1 -- 172.21.15.47:0/963730063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1201a1a30 con 0x7fd120105490 2024-03-21T11:36:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.722+0000 7fd128000640 1 -- 172.21.15.47:0/963730063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1201a1ed0 con 0x7fd120105490 2024-03-21T11:36:50.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.723+0000 7fd128000640 1 -- 172.21.15.47:0/963730063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12010aba0 con 0x7fd120105490 2024-03-21T11:36:50.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.725+0000 7fd11e000640 1 -- 172.21.15.47:0/963730063 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd118044020 con 0x7fd120105490 2024-03-21T11:36:50.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.726+0000 7fd11e000640 1 --2- 172.21.15.47:0/963730063 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1080a2fa0 0x7fd1080a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:50.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.726+0000 7fd11e000640 1 -- 172.21.15.47:0/963730063 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd1180f1510 con 0x7fd120105490 2024-03-21T11:36:50.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.727+0000 7fd125600640 1 --2- 172.21.15.47:0/963730063 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1080a2fa0 0x7fd1080a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:50.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.727+0000 7fd125600640 1 --2- 172.21.15.47:0/963730063 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1080a2fa0 0x7fd1080a5460 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd120104930 tx=0x7fd114010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:50.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.732+0000 7fd11e000640 1 -- 172.21.15.47:0/963730063 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd11804b020 con 0x7fd120105490 2024-03-21T11:36:50.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:50 smithi047 ceph-mon[31351]: pgmap v805: 129 pgs: 129 active+clean; 40 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 240 op/s 2024-03-21T11:36:50.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:50 smithi100 ceph-mon[36245]: pgmap v805: 129 pgs: 129 active+clean; 40 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 240 op/s 2024-03-21T11:36:50.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:50 smithi102 ceph-mon[36748]: pgmap v805: 129 pgs: 129 active+clean; 40 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 240 op/s 2024-03-21T11:36:50.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.993+0000 7fd128000640 1 -- 172.21.15.47:0/963730063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd12002d190 con 0x7fd120105490 2024-03-21T11:36:50.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.995+0000 7fd11e000640 1 -- 172.21.15.47:0/963730063 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fd1180be030 con 0x7fd120105490 2024-03-21T11:36:50.996 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:36:50.996 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:23:38.201806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:36:50.996 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 28 2024-03-21T11:36:51.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.998+0000 7fd128000640 1 -- 172.21.15.47:0/963730063 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1080a2fa0 msgr2=0x7fd1080a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.998+0000 7fd128000640 1 --2- 172.21.15.47:0/963730063 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1080a2fa0 0x7fd1080a5460 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd120104930 tx=0x7fd114010040 comp rx=0 tx=0).stop 2024-03-21T11:36:51.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.998+0000 7fd128000640 1 -- 172.21.15.47:0/963730063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120105490 msgr2=0x7fd1201a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.998+0000 7fd128000640 1 --2- 172.21.15.47:0/963730063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120105490 0x7fd1201a1130 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd11802c180 tx=0x7fd118032460 comp rx=0 tx=0).stop 2024-03-21T11:36:51.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.999+0000 7fd128000640 1 -- 172.21.15.47:0/963730063 shutdown_connections 2024-03-21T11:36:51.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.999+0000 7fd128000640 1 --2- 172.21.15.47:0/963730063 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1080a2fa0 0x7fd1080a5460 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.999+0000 7fd128000640 1 --2- 172.21.15.47:0/963730063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd120105490 0x7fd1201a1130 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.999+0000 7fd128000640 1 --2- 172.21.15.47:0/963730063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd120104ad0 0x7fd12019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.999+0000 7fd128000640 1 --2- 172.21.15.47:0/963730063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1201038e0 0x7fd120199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.999+0000 7fd128000640 1 -- 172.21.15.47:0/963730063 >> 172.21.15.47:0/963730063 conn(0x7fd1200fee50 msgr2=0x7fd120107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.999+0000 7fd128000640 1 -- 172.21.15.47:0/963730063 shutdown_connections 2024-03-21T11:36:51.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:50.999+0000 7fd128000640 1 -- 172.21.15.47:0/963730063 wait complete. 2024-03-21T11:36:51.022 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:36:51.022 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:36:51.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.248+0000 7f6de1a00640 1 -- 172.21.15.47:0/476294444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc107bc0 msgr2=0x7f6ddc107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.248+0000 7f6de1a00640 1 --2- 172.21.15.47:0/476294444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc107bc0 0x7f6ddc107fe0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009900 tx=0x7f6dcc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:36:51.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.248+0000 7f6de1a00640 1 -- 172.21.15.47:0/476294444 shutdown_connections 2024-03-21T11:36:51.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.248+0000 7f6de1a00640 1 --2- 172.21.15.47:0/476294444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ddc109770 0x7f6ddc110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.248+0000 7f6de1a00640 1 --2- 172.21.15.47:0/476294444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ddc108db0 0x7f6ddc109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.248+0000 7f6de1a00640 1 --2- 172.21.15.47:0/476294444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc107bc0 0x7f6ddc107fe0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.248+0000 7f6de1a00640 1 -- 172.21.15.47:0/476294444 >> 172.21.15.47:0/476294444 conn(0x7f6ddc076b70 msgr2=0x7f6ddc078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.248+0000 7f6de1a00640 1 -- 172.21.15.47:0/476294444 shutdown_connections 2024-03-21T11:36:51.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.248+0000 7f6de1a00640 1 -- 172.21.15.47:0/476294444 wait complete. 2024-03-21T11:36:51.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.249+0000 7f6de1a00640 1 Processor -- start 2024-03-21T11:36:51.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.249+0000 7f6de1a00640 1 -- start start 2024-03-21T11:36:51.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6de1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ddc107bc0 0x7f6ddc19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6de1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ddc108db0 0x7f6ddc19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6de1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc109770 0x7f6ddc1a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6de1a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc113860 con 0x7f6ddc107bc0 2024-03-21T11:36:51.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6de1a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc1136e0 con 0x7f6ddc109770 2024-03-21T11:36:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6de1a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc1139e0 con 0x7f6ddc108db0 2024-03-21T11:36:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6ddbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ddc107bc0 0x7f6ddc19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6dd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ddc108db0 0x7f6ddc19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6de1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc109770 0x7f6ddc1a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6ddbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ddc107bc0 0x7f6ddc19de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43824/0 (socket says 172.21.15.47:43824) 2024-03-21T11:36:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6dd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ddc108db0 0x7f6ddc19e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37998/0 (socket says 172.21.15.47:37998) 2024-03-21T11:36:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6ddbe00640 1 -- 172.21.15.47:0/1136205587 learned_addr learned my addr 172.21.15.47:0/1136205587 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6de1000640 1 -- 172.21.15.47:0/1136205587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ddc108db0 msgr2=0x7f6ddc19e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6de1000640 1 --2- 172.21.15.47:0/1136205587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ddc108db0 0x7f6ddc19e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6de1000640 1 -- 172.21.15.47:0/1136205587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ddc107bc0 msgr2=0x7f6ddc19de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6de1000640 1 --2- 172.21.15.47:0/1136205587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ddc107bc0 0x7f6ddc19de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.250+0000 7f6de1000640 1 -- 172.21.15.47:0/1136205587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc1a5aa0 con 0x7f6ddc109770 2024-03-21T11:36:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.251+0000 7f6de1000640 1 --2- 172.21.15.47:0/1136205587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc109770 0x7f6ddc1a5430 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009fd0 tx=0x7f6dd400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.251+0000 7f6dd9600640 1 -- 172.21.15.47:0/1136205587 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd400ce20 con 0x7f6ddc109770 2024-03-21T11:36:51.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.251+0000 7f6dd9600640 1 -- 172.21.15.47:0/1136205587 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6dd4005020 con 0x7f6ddc109770 2024-03-21T11:36:51.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.251+0000 7f6de1a00640 1 -- 172.21.15.47:0/1136205587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ddc1a5d90 con 0x7f6ddc109770 2024-03-21T11:36:51.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.251+0000 7f6dd9600640 1 -- 172.21.15.47:0/1136205587 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4010720 con 0x7f6ddc109770 2024-03-21T11:36:51.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.251+0000 7f6de1a00640 1 -- 172.21.15.47:0/1136205587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ddc1a6280 con 0x7f6ddc109770 2024-03-21T11:36:51.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.254+0000 7f6dd9600640 1 -- 172.21.15.47:0/1136205587 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6dd4005540 con 0x7f6ddc109770 2024-03-21T11:36:51.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.255+0000 7f6de5dab740 1 Processor -- start 2024-03-21T11:36:51.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.255+0000 7f6de5dab740 1 -- start start 2024-03-21T11:36:51.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.255+0000 7f6de5dab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b19f0 0x55a6b86b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.255+0000 7f6de5dab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b2bc0 0x55a6b86b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.255+0000 7f6de5dab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b3580 0x55a6b86b9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.255+0000 7f6de5dab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6b86162c0 con 0x55a6b86b2bc0 2024-03-21T11:36:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.255+0000 7f6de5dab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6b85bec90 con 0x55a6b86b3580 2024-03-21T11:36:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.255+0000 7f6de5dab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6b8612360 con 0x55a6b86b19f0 2024-03-21T11:36:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.256+0000 7f6dd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b19f0 0x55a6b86b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.256+0000 7f6dd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b19f0 0x55a6b86b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38014/0 (socket says 172.21.15.47:38014) 2024-03-21T11:36:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.256+0000 7f6dd3e00640 1 -- 172.21.15.47:0/328384349 learned_addr learned my addr 172.21.15.47:0/328384349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.256+0000 7f6ddbe00640 1 --2- 172.21.15.47:0/328384349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b2bc0 0x55a6b86b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.256+0000 7f6dd3e00640 1 --2- 172.21.15.47:0/328384349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b3580 0x55a6b86b9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.256+0000 7f6dd3e00640 1 -- 172.21.15.47:0/328384349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b3580 msgr2=0x55a6b86b9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.256+0000 7f6dd3e00640 1 --2- 172.21.15.47:0/328384349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b3580 0x55a6b86b9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.256+0000 7f6dd3e00640 1 -- 172.21.15.47:0/328384349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b2bc0 msgr2=0x55a6b86b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.256+0000 7f6dd3e00640 1 --2- 172.21.15.47:0/328384349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b2bc0 0x55a6b86b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.256+0000 7f6dd3e00640 1 -- 172.21.15.47:0/328384349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6b85e3080 con 0x55a6b86b19f0 2024-03-21T11:36:51.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.256+0000 7f6dd3e00640 1 --2- 172.21.15.47:0/328384349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b19f0 0x55a6b86b1e10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40093e0 tx=0x7f6dc4009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b20b0ae91ddc203 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.257+0000 7f6dd1600640 1 -- 172.21.15.47:0/328384349 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400f7c0 con 0x55a6b86b19f0 2024-03-21T11:36:51.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.257+0000 7f6dd1600640 1 -- 172.21.15.47:0/328384349 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6dc400fe40 con 0x55a6b86b19f0 2024-03-21T11:36:51.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.257+0000 7f6dd1600640 1 -- 172.21.15.47:0/328384349 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400e3b0 con 0x55a6b86b19f0 2024-03-21T11:36:51.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.257+0000 7f6dd9600640 1 --2- 172.21.15.47:0/1136205587 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dc00a2fa0 0x7f6dc00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.257+0000 7f6dd9600640 1 -- 172.21.15.47:0/1136205587 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6dd40cfe60 con 0x7f6ddc109770 2024-03-21T11:36:51.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.257+0000 7f6ddbe00640 1 --2- 172.21.15.47:0/1136205587 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dc00a2fa0 0x7f6dc00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.257+0000 7f6de5dab740 1 -- 172.21.15.47:0/328384349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b19f0 msgr2=0x55a6b86b1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.257+0000 7f6de5dab740 1 --2- 172.21.15.47:0/328384349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b19f0 0x55a6b86b1e10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40093e0 tx=0x7f6dc4009370 comp rx=0 tx=0).stop 2024-03-21T11:36:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.258+0000 7f6de5dab740 1 -- 172.21.15.47:0/328384349 shutdown_connections 2024-03-21T11:36:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.258+0000 7f6de5dab740 1 --2- 172.21.15.47:0/328384349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b3580 0x55a6b86b9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.258+0000 7f6de5dab740 1 --2- 172.21.15.47:0/328384349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b2bc0 0x55a6b86b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.258+0000 7f6de5dab740 1 --2- 172.21.15.47:0/328384349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b19f0 0x55a6b86b1e10 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.258+0000 7f6de5dab740 1 -- 172.21.15.47:0/328384349 >> 172.21.15.47:0/328384349 conn(0x55a6b86acde0 msgr2=0x55a6b86ad1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.258+0000 7f6de5dab740 1 -- 172.21.15.47:0/328384349 shutdown_connections 2024-03-21T11:36:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.258+0000 7f6de5dab740 1 -- 172.21.15.47:0/328384349 wait complete. 2024-03-21T11:36:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.258+0000 7f6ddbe00640 1 --2- 172.21.15.47:0/1136205587 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dc00a2fa0 0x7f6dc00a5460 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc006010 tx=0x7f6dcc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.258+0000 7f6de5dab740 1 -- start start 2024-03-21T11:36:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.258+0000 7f6de5dab740 1 Processor -- start 2024-03-21T11:36:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.259+0000 7f6de5dab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b19f0 0x55a6b86bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.259+0000 7f6de5dab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b2bc0 0x55a6b86bc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.259+0000 7f6de5dab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b3580 0x55a6b86c3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.259+0000 7f6de5dab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6b86ba590 con 0x55a6b86b19f0 2024-03-21T11:36:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.259+0000 7f6de5dab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6b85d20d0 con 0x55a6b86b2bc0 2024-03-21T11:36:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.259+0000 7f6de5dab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6b86ba710 con 0x55a6b86b3580 2024-03-21T11:36:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.259+0000 7f6de5dab740 10 client.0 _ll_register_callbacks cb 0x55a6b85d6e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.259+0000 7f6dd3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b2bc0 0x55a6b86bc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.259+0000 7f6de1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b3580 0x55a6b86c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.259+0000 7f6dd3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b2bc0 0x55a6b86bc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59970/0 (socket says 172.21.15.47:59970) 2024-03-21T11:36:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.259+0000 7f6dd3e00640 1 -- 172.21.15.47:0/2307140291 learned_addr learned my addr 172.21.15.47:0/2307140291 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.259+0000 7f6ddbe00640 1 --2- 172.21.15.47:0/2307140291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b19f0 0x55a6b86bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.260+0000 7f6dd3e00640 1 -- 172.21.15.47:0/2307140291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b3580 msgr2=0x55a6b86c3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.260+0000 7f6dd3e00640 1 --2- 172.21.15.47:0/2307140291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b3580 0x55a6b86c3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.260+0000 7f6dd3e00640 1 -- 172.21.15.47:0/2307140291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b19f0 msgr2=0x55a6b86bbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.260+0000 7f6dd3e00640 1 --2- 172.21.15.47:0/2307140291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b19f0 0x55a6b86bbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.260+0000 7f6dd3e00640 1 -- 172.21.15.47:0/2307140291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6b832a3f0 con 0x55a6b86b2bc0 2024-03-21T11:36:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.260+0000 7f6de1000640 1 --2- 172.21.15.47:0/2307140291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b3580 0x55a6b86c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.260+0000 7f6dd3e00640 1 --2- 172.21.15.47:0/2307140291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b2bc0 0x55a6b86bc1c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6dc401f7c0 tx=0x7f6dc401f830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a76509ea174a5489 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.260+0000 7f6dca000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.260+0000 7f6dca000640 1 -- 172.21.15.47:0/2307140291 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc402d060 con 0x55a6b86b2bc0 2024-03-21T11:36:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.260+0000 7f6dca000640 1 -- 172.21.15.47:0/2307140291 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6dc4019070 con 0x55a6b86b2bc0 2024-03-21T11:36:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.260+0000 7f6de5dab740 1 -- 172.21.15.47:0/2307140291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6dc400fe40 con 0x55a6b86b2bc0 2024-03-21T11:36:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.261+0000 7f6dca000640 1 -- 172.21.15.47:0/2307140291 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dc4034050 con 0x55a6b86b2bc0 2024-03-21T11:36:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.261+0000 7f6de5dab740 10 client.29999 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.261+0000 7f6de5dab740 1 -- 172.21.15.47:0/2307140291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a6b83f47d0 con 0x55a6b86b2bc0 2024-03-21T11:36:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.261+0000 7f6dca000640 1 -- 172.21.15.47:0/2307140291 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6dc400fe40 con 0x55a6b86b2bc0 2024-03-21T11:36:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.261+0000 7f6dca000640 1 -- 172.21.15.47:0/2307140291 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6dc401d020 con 0x55a6b86b2bc0 2024-03-21T11:36:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.261+0000 7f6de5dab740 10 client.29999 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.261+0000 7f6de5dab740 10 client.29999 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:36:51.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.261+0000 7f6de5dab740 20 client.29999 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:51.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.261+0000 7f6de5dab740 1 --2- 172.21.15.47:0/2307140291 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a6b86c7400 0x55a6b86c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.261+0000 7f6de5dab740 4 client.29999 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:51.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.261+0000 7f6de5dab740 1 -- 172.21.15.47:0/2307140291 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6b8623800 con 0x55a6b86c7400 2024-03-21T11:36:51.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.262+0000 7f6ddbe00640 1 --2- 172.21.15.47:0/2307140291 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a6b86c7400 0x55a6b86c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.262+0000 7f6ddbe00640 1 --2- 172.21.15.47:0/2307140291 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a6b86c7400 0x55a6b86c97f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=41605cbbb5500a5b server_cookie=2bd4fe162997649e in_seq=0 out_seq=0 2024-03-21T11:36:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.262+0000 7f6dca000640 10 client.29999 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:36:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.263+0000 7f6dca000640 1 -- 172.21.15.47:0/2307140291 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a6b8623800 con 0x55a6b86c7400 2024-03-21T11:36:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.263+0000 7f6dca000640 10 client.29999 handle_command_reply: tid=0 2024-03-21T11:36:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.263+0000 7f6de5dab740 1 client.29999 shutdown 2024-03-21T11:36:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.263+0000 7f6de5dab740 1 -- 172.21.15.47:0/2307140291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b2bc0 msgr2=0x55a6b86bc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.263+0000 7f6de5dab740 1 --2- 172.21.15.47:0/2307140291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b2bc0 0x55a6b86bc1c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6dc401f7c0 tx=0x7f6dc401f830 comp rx=0 tx=0).stop 2024-03-21T11:36:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.263+0000 7f6de5dab740 1 -- 172.21.15.47:0/2307140291 shutdown_connections 2024-03-21T11:36:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.264+0000 7f6de5dab740 1 --2- 172.21.15.47:0/2307140291 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a6b86c7400 0x55a6b86c97f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.264+0000 7f6de5dab740 1 --2- 172.21.15.47:0/2307140291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b3580 0x55a6b86c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.264+0000 7f6de5dab740 1 --2- 172.21.15.47:0/2307140291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b2bc0 0x55a6b86bc1c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.264+0000 7f6de5dab740 1 --2- 172.21.15.47:0/2307140291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b19f0 0x55a6b86bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.264+0000 7f6dca000640 0 client.29999 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:36:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.264+0000 7f6de5dab740 1 -- 172.21.15.47:0/2307140291 >> 172.21.15.47:0/2307140291 conn(0x55a6b86acde0 msgr2=0x55a6b86ac160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.264+0000 7f6de5dab740 1 -- 172.21.15.47:0/2307140291 shutdown_connections 2024-03-21T11:36:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.264+0000 7f6de5dab740 1 -- 172.21.15.47:0/2307140291 wait complete. 2024-03-21T11:36:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.264+0000 7f6de5dab740 20 client.29999 trim_cache size 0 max 16384 2024-03-21T11:36:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.278+0000 7f6de5dab740 1 Processor -- start 2024-03-21T11:36:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.279+0000 7f6de5dab740 1 -- start start 2024-03-21T11:36:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.279+0000 7f6de5dab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b19f0 0x55a6b86b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.279+0000 7f6de5dab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b2bc0 0x55a6b86c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.279+0000 7f6de5dab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b3580 0x55a6b86ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.279+0000 7f6de5dab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc1b03f0 con 0x55a6b86b3580 2024-03-21T11:36:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.279+0000 7f6de5dab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc1b1d70 con 0x55a6b86b19f0 2024-03-21T11:36:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.279+0000 7f6de5dab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc1b0a50 con 0x55a6b86b2bc0 2024-03-21T11:36:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.279+0000 7f6ddbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b19f0 0x55a6b86b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.279+0000 7f6dd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b2bc0 0x55a6b86c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.279+0000 7f6ddbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b19f0 0x55a6b86b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59972/0 (socket says 172.21.15.47:59972) 2024-03-21T11:36:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.279+0000 7f6ddbe00640 1 -- 172.21.15.47:0/3523935775 learned_addr learned my addr 172.21.15.47:0/3523935775 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.280+0000 7f6de1000640 1 --2- 172.21.15.47:0/3523935775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b3580 0x55a6b86ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.280+0000 7f6dd3e00640 1 -- 172.21.15.47:0/3523935775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b19f0 msgr2=0x55a6b86b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.280+0000 7f6dd3e00640 1 --2- 172.21.15.47:0/3523935775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b19f0 0x55a6b86b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.280+0000 7f6dd3e00640 1 -- 172.21.15.47:0/3523935775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b3580 msgr2=0x55a6b86ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.280+0000 7f6dd3e00640 1 --2- 172.21.15.47:0/3523935775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b3580 0x55a6b86ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.280+0000 7f6dd3e00640 1 -- 172.21.15.47:0/3523935775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6b86ba860 con 0x55a6b86b2bc0 2024-03-21T11:36:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.280+0000 7f6dd3e00640 1 --2- 172.21.15.47:0/3523935775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b2bc0 0x55a6b86c31c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f6dc401f800 tx=0x7f6dc40059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b22a2a2b13de09c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.280+0000 7f6dd0c00640 1 -- 172.21.15.47:0/3523935775 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc402d040 con 0x55a6b86b2bc0 2024-03-21T11:36:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.280+0000 7f6dd0c00640 1 -- 172.21.15.47:0/3523935775 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6dc4005a70 con 0x55a6b86b2bc0 2024-03-21T11:36:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.280+0000 7f6dd0c00640 1 -- 172.21.15.47:0/3523935775 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4011040 con 0x55a6b86b2bc0 2024-03-21T11:36:51.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.281+0000 7f6de5dab740 1 -- 172.21.15.47:0/3523935775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b2bc0 msgr2=0x55a6b86c31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.281+0000 7f6de5dab740 1 --2- 172.21.15.47:0/3523935775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b2bc0 0x55a6b86c31c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f6dc401f800 tx=0x7f6dc40059b0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.281+0000 7f6de5dab740 1 -- 172.21.15.47:0/3523935775 shutdown_connections 2024-03-21T11:36:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.281+0000 7f6de5dab740 1 --2- 172.21.15.47:0/3523935775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b3580 0x55a6b86ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.281+0000 7f6de5dab740 1 --2- 172.21.15.47:0/3523935775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b2bc0 0x55a6b86c31c0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.281+0000 7f6de5dab740 1 --2- 172.21.15.47:0/3523935775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b19f0 0x55a6b86b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.281+0000 7f6de5dab740 1 -- 172.21.15.47:0/3523935775 >> 172.21.15.47:0/3523935775 conn(0x55a6b86acde0 msgr2=0x55a6b86b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.281+0000 7f6de5dab740 1 -- 172.21.15.47:0/3523935775 shutdown_connections 2024-03-21T11:36:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.281+0000 7f6de5dab740 1 -- 172.21.15.47:0/3523935775 wait complete. 2024-03-21T11:36:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.281+0000 7f6de5dab740 1 -- start start 2024-03-21T11:36:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.282+0000 7f6de5dab740 1 Processor -- start 2024-03-21T11:36:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.282+0000 7f6de5dab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b19f0 0x55a6b86be5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.282+0000 7f6de5dab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b2bc0 0x55a6b86beb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.282+0000 7f6de5dab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b3580 0x55a6b86c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.282+0000 7f6de5dab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc1b1710 con 0x55a6b86b19f0 2024-03-21T11:36:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.282+0000 7f6de5dab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc1b10b0 con 0x55a6b86b2bc0 2024-03-21T11:36:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.282+0000 7f6de5dab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc1ae5b0 con 0x55a6b86b3580 2024-03-21T11:36:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.282+0000 7f6de5dab740 10 client.0 _ll_register_callbacks cb 0x7f6dc4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.282+0000 7f6dd3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b2bc0 0x55a6b86beb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.282+0000 7f6de1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b3580 0x55a6b86c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.282+0000 7f6dd3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b2bc0 0x55a6b86beb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59980/0 (socket says 172.21.15.47:59980) 2024-03-21T11:36:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.282+0000 7f6dd3e00640 1 -- 172.21.15.47:0/3593068664 learned_addr learned my addr 172.21.15.47:0/3593068664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.282+0000 7f6ddbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b19f0 0x55a6b86be5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.283+0000 7f6dd3e00640 1 -- 172.21.15.47:0/3593068664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b3580 msgr2=0x55a6b86c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.283+0000 7f6dd3e00640 1 --2- 172.21.15.47:0/3593068664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b3580 0x55a6b86c1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.283+0000 7f6dd3e00640 1 -- 172.21.15.47:0/3593068664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b19f0 msgr2=0x55a6b86be5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.283+0000 7f6dd3e00640 1 --2- 172.21.15.47:0/3593068664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b19f0 0x55a6b86be5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.283+0000 7f6dd3e00640 1 -- 172.21.15.47:0/3593068664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6b86c2140 con 0x55a6b86b2bc0 2024-03-21T11:36:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.283+0000 7f6de1000640 1 --2- 172.21.15.47:0/3593068664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b3580 0x55a6b86c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.283+0000 7f6dd3e00640 1 --2- 172.21.15.47:0/3593068664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b2bc0 0x55a6b86beb30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4005320 tx=0x7f6dc4011270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64e2e33743dc82a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.283+0000 7f6dca000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.283+0000 7f6dca000640 1 -- 172.21.15.47:0/3593068664 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc402d040 con 0x55a6b86b2bc0 2024-03-21T11:36:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.283+0000 7f6de5dab740 1 -- 172.21.15.47:0/3593068664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6dc4005a70 con 0x55a6b86b2bc0 2024-03-21T11:36:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.284+0000 7f6dca000640 1 -- 172.21.15.47:0/3593068664 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6dc4004020 con 0x55a6b86b2bc0 2024-03-21T11:36:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.284+0000 7f6dca000640 1 -- 172.21.15.47:0/3593068664 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dc404a050 con 0x55a6b86b2bc0 2024-03-21T11:36:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.284+0000 7f6de5dab740 10 client.30005 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.284+0000 7f6de5dab740 1 -- 172.21.15.47:0/3593068664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a6b86c3440 con 0x55a6b86b2bc0 2024-03-21T11:36:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.284+0000 7f6dca000640 1 -- 172.21.15.47:0/3593068664 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6dc4005a70 con 0x55a6b86b2bc0 2024-03-21T11:36:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.284+0000 7f6dca000640 1 -- 172.21.15.47:0/3593068664 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6dc4040030 con 0x55a6b86b2bc0 2024-03-21T11:36:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.284+0000 7f6de5dab740 10 client.30005 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.284+0000 7f6de5dab740 10 client.30005 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:36:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.284+0000 7f6de5dab740 20 client.30005 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.284+0000 7f6de5dab740 1 --2- 172.21.15.47:0/3593068664 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a6b87231c0 0x55a6b87255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.284+0000 7f6de5dab740 4 client.30005 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:36:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.284+0000 7f6de5dab740 1 -- 172.21.15.47:0/3593068664 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a6b861db50 con 0x55a6b87231c0 2024-03-21T11:36:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.285+0000 7f6ddbe00640 1 --2- 172.21.15.47:0/3593068664 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a6b87231c0 0x55a6b87255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.285+0000 7f6ddbe00640 1 --2- 172.21.15.47:0/3593068664 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a6b87231c0 0x55a6b87255b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2a88837aa6be5fd6 server_cookie=a5d279bf2326a57e in_seq=0 out_seq=0 2024-03-21T11:36:51.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.285+0000 7f6dca000640 10 client.30005 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:36:51.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.285+0000 7f6dca000640 1 -- 172.21.15.47:0/3593068664 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a6b861db50 con 0x55a6b87231c0 2024-03-21T11:36:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.285+0000 7f6dca000640 10 client.30005 handle_command_reply: tid=0 2024-03-21T11:36:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.285+0000 7f6de5dab740 1 client.30005 shutdown 2024-03-21T11:36:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.286+0000 7f6de5dab740 1 -- 172.21.15.47:0/3593068664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b2bc0 msgr2=0x55a6b86beb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.286+0000 7f6de5dab740 1 --2- 172.21.15.47:0/3593068664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b2bc0 0x55a6b86beb30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4005320 tx=0x7f6dc4011270 comp rx=0 tx=0).stop 2024-03-21T11:36:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.286+0000 7f6de5dab740 1 -- 172.21.15.47:0/3593068664 shutdown_connections 2024-03-21T11:36:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.286+0000 7f6de5dab740 1 --2- 172.21.15.47:0/3593068664 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a6b87231c0 0x55a6b87255b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.286+0000 7f6de5dab740 1 --2- 172.21.15.47:0/3593068664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6b86b3580 0x55a6b86c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.286+0000 7f6de5dab740 1 --2- 172.21.15.47:0/3593068664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6b86b2bc0 0x55a6b86beb30 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.286+0000 7f6dca000640 0 client.30005 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:36:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.286+0000 7f6de5dab740 1 --2- 172.21.15.47:0/3593068664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6b86b19f0 0x55a6b86be5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.286+0000 7f6de5dab740 1 -- 172.21.15.47:0/3593068664 >> 172.21.15.47:0/3593068664 conn(0x55a6b86acde0 msgr2=0x55a6b86b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.286+0000 7f6de5dab740 1 -- 172.21.15.47:0/3593068664 shutdown_connections 2024-03-21T11:36:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.286+0000 7f6de5dab740 1 -- 172.21.15.47:0/3593068664 wait complete. 2024-03-21T11:36:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.286+0000 7f6de5dab740 20 client.30005 trim_cache size 0 max 16384 2024-03-21T11:36:51.292 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:36:51.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.287+0000 7f6de1a00640 1 -- 172.21.15.47:0/1136205587 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dc00a2fa0 msgr2=0x7f6dc00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.287+0000 7f6de1a00640 1 --2- 172.21.15.47:0/1136205587 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dc00a2fa0 0x7f6dc00a5460 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc006010 tx=0x7f6dcc02df90 comp rx=0 tx=0).stop 2024-03-21T11:36:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.287+0000 7f6de1a00640 1 -- 172.21.15.47:0/1136205587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc109770 msgr2=0x7f6ddc1a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.287+0000 7f6de1a00640 1 --2- 172.21.15.47:0/1136205587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc109770 0x7f6ddc1a5430 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009fd0 tx=0x7f6dd400ef10 comp rx=0 tx=0).stop 2024-03-21T11:36:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.287+0000 7f6de1a00640 1 -- 172.21.15.47:0/1136205587 shutdown_connections 2024-03-21T11:36:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.287+0000 7f6de1a00640 1 --2- 172.21.15.47:0/1136205587 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6dc00a2fa0 0x7f6dc00a5460 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.287+0000 7f6de1a00640 1 --2- 172.21.15.47:0/1136205587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc109770 0x7f6ddc1a5430 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.287+0000 7f6de1a00640 1 --2- 172.21.15.47:0/1136205587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ddc108db0 0x7f6ddc19e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.287+0000 7f6de1a00640 1 --2- 172.21.15.47:0/1136205587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ddc107bc0 0x7f6ddc19de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.287+0000 7f6de1a00640 1 -- 172.21.15.47:0/1136205587 >> 172.21.15.47:0/1136205587 conn(0x7f6ddc076b70 msgr2=0x7f6ddc10c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.287+0000 7f6de1a00640 1 -- 172.21.15.47:0/1136205587 shutdown_connections 2024-03-21T11:36:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.287+0000 7f6de1a00640 1 -- 172.21.15.47:0/1136205587 wait complete. 2024-03-21T11:36:51.306 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:36:51.306 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:36:51.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.538+0000 7fc424e00640 1 -- 172.21.15.47:0/1314198118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc420104990 msgr2=0x7fc420104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.538+0000 7fc424e00640 1 --2- 172.21.15.47:0/1314198118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc420104990 0x7fc420104e10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc408009900 tx=0x7fc40802bc40 comp rx=0 tx=0).stop 2024-03-21T11:36:51.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.538+0000 7fc424e00640 1 -- 172.21.15.47:0/1314198118 shutdown_connections 2024-03-21T11:36:51.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.538+0000 7fc424e00640 1 --2- 172.21.15.47:0/1314198118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc420105350 0x7fc42010bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.538+0000 7fc424e00640 1 --2- 172.21.15.47:0/1314198118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc420104990 0x7fc420104e10 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.538+0000 7fc424e00640 1 --2- 172.21.15.47:0/1314198118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4201037a0 0x7fc420103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.538+0000 7fc424e00640 1 -- 172.21.15.47:0/1314198118 >> 172.21.15.47:0/1314198118 conn(0x7fc4200fed10 msgr2=0x7fc420101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.538+0000 7fc424e00640 1 -- 172.21.15.47:0/1314198118 shutdown_connections 2024-03-21T11:36:51.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.538+0000 7fc424e00640 1 -- 172.21.15.47:0/1314198118 wait complete. 2024-03-21T11:36:51.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.539+0000 7fc424e00640 1 Processor -- start 2024-03-21T11:36:51.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.539+0000 7fc424e00640 1 -- start start 2024-03-21T11:36:51.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc424e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4201037a0 0x7fc420199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc424e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc420104990 0x7fc420199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc424e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc420105350 0x7fc4201a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc424e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42010f470 con 0x7fc420104990 2024-03-21T11:36:51.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc424e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42010f2f0 con 0x7fc420105350 2024-03-21T11:36:51.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc424e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42010f5f0 con 0x7fc4201037a0 2024-03-21T11:36:51.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc41fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc420105350 0x7fc4201a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc41f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4201037a0 0x7fc420199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc41fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc420105350 0x7fc4201a1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59986/0 (socket says 172.21.15.47:59986) 2024-03-21T11:36:51.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc41fe00640 1 -- 172.21.15.47:0/1864629241 learned_addr learned my addr 172.21.15.47:0/1864629241 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc41f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4201037a0 0x7fc420199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38066/0 (socket says 172.21.15.47:38066) 2024-03-21T11:36:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc41ea00640 1 --2- 172.21.15.47:0/1864629241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc420104990 0x7fc420199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc41fe00640 1 -- 172.21.15.47:0/1864629241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4201037a0 msgr2=0x7fc420199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc41fe00640 1 --2- 172.21.15.47:0/1864629241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4201037a0 0x7fc420199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc41fe00640 1 -- 172.21.15.47:0/1864629241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc420104990 msgr2=0x7fc420199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc41fe00640 1 --2- 172.21.15.47:0/1864629241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc420104990 0x7fc420199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc41fe00640 1 -- 172.21.15.47:0/1864629241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4201a16d0 con 0x7fc420105350 2024-03-21T11:36:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.540+0000 7fc41ea00640 1 --2- 172.21.15.47:0/1864629241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc420104990 0x7fc420199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.541+0000 7fc41fe00640 1 --2- 172.21.15.47:0/1864629241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc420105350 0x7fc4201a1060 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc41800d750 tx=0x7fc41800dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.541+0000 7fc417e00640 1 -- 172.21.15.47:0/1864629241 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418017070 con 0x7fc420105350 2024-03-21T11:36:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.541+0000 7fc417e00640 1 -- 172.21.15.47:0/1864629241 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc418005500 con 0x7fc420105350 2024-03-21T11:36:51.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.541+0000 7fc424e00640 1 -- 172.21.15.47:0/1864629241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4201a19c0 con 0x7fc420105350 2024-03-21T11:36:51.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.541+0000 7fc417e00640 1 -- 172.21.15.47:0/1864629241 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418004d90 con 0x7fc420105350 2024-03-21T11:36:51.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.541+0000 7fc424e00640 1 -- 172.21.15.47:0/1864629241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4201a1ee0 con 0x7fc420105350 2024-03-21T11:36:51.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.545+0000 7fc429047740 1 Processor -- start 2024-03-21T11:36:51.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.545+0000 7fc417e00640 1 -- 172.21.15.47:0/1864629241 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc418004f30 con 0x7fc420105350 2024-03-21T11:36:51.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.545+0000 7fc429047740 1 -- start start 2024-03-21T11:36:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.546+0000 7fc429047740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1e6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.546+0000 7fc429047740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1e8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.546+0000 7fc429047740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e8580 0x55c15c1eee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.546+0000 7fc429047740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c14b2c0 con 0x55c15c1e8580 2024-03-21T11:36:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.546+0000 7fc429047740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c0f3c90 con 0x55c15c1e69f0 2024-03-21T11:36:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.546+0000 7fc429047740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c147360 con 0x55c15c1e7bc0 2024-03-21T11:36:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.546+0000 7fc41f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.546+0000 7fc41f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1e8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38080/0 (socket says 172.21.15.47:38080) 2024-03-21T11:36:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.546+0000 7fc41f400640 1 -- 172.21.15.47:0/1773558246 learned_addr learned my addr 172.21.15.47:0/1773558246 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.546+0000 7fc41ea00640 1 --2- 172.21.15.47:0/1773558246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1e6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.546+0000 7fc41ea00640 1 --2- 172.21.15.47:0/1773558246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e8580 0x55c15c1eee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.546+0000 7fc41f400640 1 -- 172.21.15.47:0/1773558246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 msgr2=0x55c15c1e6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.546+0000 7fc41f400640 1 --2- 172.21.15.47:0/1773558246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1e6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.546+0000 7fc41f400640 1 -- 172.21.15.47:0/1773558246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e8580 msgr2=0x55c15c1eee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.547+0000 7fc41f400640 1 --2- 172.21.15.47:0/1773558246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e8580 0x55c15c1eee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.547+0000 7fc41f400640 1 -- 172.21.15.47:0/1773558246 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c15c118080 con 0x55c15c1e7bc0 2024-03-21T11:36:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.547+0000 7fc41ea00640 1 --2- 172.21.15.47:0/1773558246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e8580 0x55c15c1eee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:36:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.547+0000 7fc41f400640 1 --2- 172.21.15.47:0/1773558246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1e8040 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc410011470 tx=0x7fc410011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d9e344ee481e8a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.547+0000 7fc417e00640 1 --2- 172.21.15.47:0/1864629241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc4040a2fa0 0x7fc4040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.548+0000 7fc41ea00640 1 --2- 172.21.15.47:0/1864629241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc4040a2fa0 0x7fc4040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.548+0000 7fc417e00640 1 -- 172.21.15.47:0/1864629241 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc4180cf170 con 0x7fc420105350 2024-03-21T11:36:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.548+0000 7fc41ea00640 1 --2- 172.21.15.47:0/1864629241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc4040a2fa0 0x7fc4040a5460 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc40802c250 tx=0x7fc40802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.549+0000 7fc414c00640 1 -- 172.21.15.47:0/1773558246 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc410003970 con 0x55c15c1e7bc0 2024-03-21T11:36:51.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.549+0000 7fc414c00640 1 -- 172.21.15.47:0/1773558246 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc410004020 con 0x55c15c1e7bc0 2024-03-21T11:36:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.549+0000 7fc414c00640 1 -- 172.21.15.47:0/1773558246 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41000fa00 con 0x55c15c1e7bc0 2024-03-21T11:36:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.550+0000 7fc429047740 1 -- 172.21.15.47:0/1773558246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 msgr2=0x55c15c1e8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.550+0000 7fc429047740 1 --2- 172.21.15.47:0/1773558246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1e8040 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc410011470 tx=0x7fc410011930 comp rx=0 tx=0).stop 2024-03-21T11:36:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.550+0000 7fc429047740 1 -- 172.21.15.47:0/1773558246 shutdown_connections 2024-03-21T11:36:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.550+0000 7fc429047740 1 --2- 172.21.15.47:0/1773558246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e8580 0x55c15c1eee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.550+0000 7fc429047740 1 --2- 172.21.15.47:0/1773558246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1e8040 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.550+0000 7fc429047740 1 --2- 172.21.15.47:0/1773558246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1e6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.550+0000 7fc429047740 1 -- 172.21.15.47:0/1773558246 >> 172.21.15.47:0/1773558246 conn(0x55c15c1e1de0 msgr2=0x55c15c1e21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.550+0000 7fc429047740 1 -- 172.21.15.47:0/1773558246 shutdown_connections 2024-03-21T11:36:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.550+0000 7fc429047740 1 -- 172.21.15.47:0/1773558246 wait complete. 2024-03-21T11:36:51.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.551+0000 7fc429047740 1 -- start start 2024-03-21T11:36:51.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.551+0000 7fc429047740 1 Processor -- start 2024-03-21T11:36:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.552+0000 7fc429047740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.552+0000 7fc429047740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.552+0000 7fc429047740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e8580 0x55c15c1f8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.552+0000 7fc429047740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c1ef590 con 0x55c15c1e8580 2024-03-21T11:36:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.552+0000 7fc429047740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c1070d0 con 0x55c15c1e69f0 2024-03-21T11:36:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.552+0000 7fc429047740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c1ef710 con 0x55c15c1e7bc0 2024-03-21T11:36:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.552+0000 7fc429047740 10 client.0 _ll_register_callbacks cb 0x55c15c10be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.552+0000 7fc41f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.552+0000 7fc41ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.552+0000 7fc41fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e8580 0x55c15c1f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.552+0000 7fc41ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38092/0 (socket says 172.21.15.47:38092) 2024-03-21T11:36:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.552+0000 7fc41ea00640 1 -- 172.21.15.47:0/1682166994 learned_addr learned my addr 172.21.15.47:0/1682166994 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.552+0000 7fc41f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60006/0 (socket says 172.21.15.47:60006) 2024-03-21T11:36:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.553+0000 7fc41f400640 1 -- 172.21.15.47:0/1682166994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 msgr2=0x55c15c1f1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.553+0000 7fc41f400640 1 --2- 172.21.15.47:0/1682166994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.553+0000 7fc41f400640 1 -- 172.21.15.47:0/1682166994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e8580 msgr2=0x55c15c1f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.553+0000 7fc41f400640 1 --2- 172.21.15.47:0/1682166994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e8580 0x55c15c1f8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.553+0000 7fc41f400640 1 -- 172.21.15.47:0/1682166994 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c15be5f3f0 con 0x55c15c1e69f0 2024-03-21T11:36:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.553+0000 7fc41f400640 1 --2- 172.21.15.47:0/1682166994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1f0c00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc41001b040 tx=0x7fc410004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b41f92e5b94eee70 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.553+0000 7fc40d600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.553+0000 7fc40d600640 1 -- 172.21.15.47:0/1682166994 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc410027020 con 0x55c15c1e69f0 2024-03-21T11:36:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.554+0000 7fc40d600640 1 -- 172.21.15.47:0/1682166994 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc410003b80 con 0x55c15c1e69f0 2024-03-21T11:36:51.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.554+0000 7fc429047740 1 -- 172.21.15.47:0/1682166994 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc410004020 con 0x55c15c1e69f0 2024-03-21T11:36:51.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.554+0000 7fc40d600640 1 -- 172.21.15.47:0/1682166994 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc410032050 con 0x55c15c1e69f0 2024-03-21T11:36:51.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.554+0000 7fc429047740 10 client.30017 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:51.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.554+0000 7fc429047740 1 -- 172.21.15.47:0/1682166994 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c15bf297d0 con 0x55c15c1e69f0 2024-03-21T11:36:51.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.554+0000 7fc40d600640 1 -- 172.21.15.47:0/1682166994 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc410004020 con 0x55c15c1e69f0 2024-03-21T11:36:51.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.555+0000 7fc40d600640 1 -- 172.21.15.47:0/1682166994 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc410005040 con 0x55c15c1e69f0 2024-03-21T11:36:51.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.555+0000 7fc429047740 10 client.30017 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:51.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.555+0000 7fc429047740 10 client.30017 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:36:51.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.555+0000 7fc429047740 20 client.30017 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:51.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.555+0000 7fc429047740 1 --2- 172.21.15.47:0/1682166994 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c15c1fc3b0 0x55c15c1fe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.555+0000 7fc429047740 4 client.30017 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:51.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.555+0000 7fc429047740 1 -- 172.21.15.47:0/1682166994 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c15c158800 con 0x55c15c1fc3b0 2024-03-21T11:36:51.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.555+0000 7fc41ea00640 1 --2- 172.21.15.47:0/1682166994 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c15c1fc3b0 0x55c15c1fe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.555+0000 7fc41ea00640 1 --2- 172.21.15.47:0/1682166994 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c15c1fc3b0 0x55c15c1fe810 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=db8d4128bf30428c server_cookie=50525e14ddf54a0f in_seq=0 out_seq=0 2024-03-21T11:36:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.555+0000 7fc40d600640 10 client.30017 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:36:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.556+0000 7fc40d600640 1 -- 172.21.15.47:0/1682166994 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c15c158800 con 0x55c15c1fc3b0 2024-03-21T11:36:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.556+0000 7fc40d600640 10 client.30017 handle_command_reply: tid=0 2024-03-21T11:36:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.556+0000 7fc429047740 1 client.30017 shutdown 2024-03-21T11:36:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.556+0000 7fc429047740 1 -- 172.21.15.47:0/1682166994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 msgr2=0x55c15c1f0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.556+0000 7fc429047740 1 --2- 172.21.15.47:0/1682166994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1f0c00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc41001b040 tx=0x7fc410004c70 comp rx=0 tx=0).stop 2024-03-21T11:36:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.557+0000 7fc429047740 1 -- 172.21.15.47:0/1682166994 shutdown_connections 2024-03-21T11:36:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.557+0000 7fc429047740 1 --2- 172.21.15.47:0/1682166994 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c15c1fc3b0 0x55c15c1fe810 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.557+0000 7fc429047740 1 --2- 172.21.15.47:0/1682166994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e8580 0x55c15c1f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.557+0000 7fc429047740 1 --2- 172.21.15.47:0/1682166994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.557+0000 7fc429047740 1 --2- 172.21.15.47:0/1682166994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1f0c00 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.557+0000 7fc40d600640 0 client.30017 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:36:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.557+0000 7fc429047740 1 -- 172.21.15.47:0/1682166994 >> 172.21.15.47:0/1682166994 conn(0x55c15c1e1de0 msgr2=0x55c15c1e1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.557+0000 7fc429047740 1 -- 172.21.15.47:0/1682166994 shutdown_connections 2024-03-21T11:36:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.557+0000 7fc429047740 1 -- 172.21.15.47:0/1682166994 wait complete. 2024-03-21T11:36:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.557+0000 7fc429047740 20 client.30017 trim_cache size 0 max 16384 2024-03-21T11:36:51.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:51 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/963730063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:36:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.573+0000 7fc429047740 1 Processor -- start 2024-03-21T11:36:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.573+0000 7fc429047740 1 -- start start 2024-03-21T11:36:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.573+0000 7fc429047740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e69f0 0x55c15c1e5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.573+0000 7fc429047740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.573+0000 7fc429047740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e8580 0x55c15c2030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.573+0000 7fc429047740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4201ac010 con 0x55c15c1e69f0 2024-03-21T11:36:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.573+0000 7fc429047740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4201ad990 con 0x55c15c1e8580 2024-03-21T11:36:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.573+0000 7fc429047740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4201ac670 con 0x55c15c1e7bc0 2024-03-21T11:36:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.573+0000 7fc41ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.573+0000 7fc41ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38106/0 (socket says 172.21.15.47:38106) 2024-03-21T11:36:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.573+0000 7fc41fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e8580 0x55c15c2030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.573+0000 7fc41ea00640 1 -- 172.21.15.47:0/3718615336 learned_addr learned my addr 172.21.15.47:0/3718615336 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.573+0000 7fc41f400640 1 --2- 172.21.15.47:0/3718615336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e69f0 0x55c15c1e5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.574+0000 7fc41fe00640 1 -- 172.21.15.47:0/3718615336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 msgr2=0x55c15c1f8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.574+0000 7fc41fe00640 1 --2- 172.21.15.47:0/3718615336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.574+0000 7fc41fe00640 1 -- 172.21.15.47:0/3718615336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e69f0 msgr2=0x55c15c1e5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.574+0000 7fc41fe00640 1 --2- 172.21.15.47:0/3718615336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e69f0 0x55c15c1e5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.574+0000 7fc41fe00640 1 -- 172.21.15.47:0/3718615336 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c15c1dd8b0 con 0x55c15c1e8580 2024-03-21T11:36:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.574+0000 7fc41fe00640 1 --2- 172.21.15.47:0/3718615336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e8580 0x55c15c2030d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc4180cc880 tx=0x7fc41809af20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caed2feb8818bd37 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.574+0000 7fc40fe00640 1 -- 172.21.15.47:0/3718615336 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418020040 con 0x55c15c1e8580 2024-03-21T11:36:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.574+0000 7fc40fe00640 1 -- 172.21.15.47:0/3718615336 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc41809b4b0 con 0x55c15c1e8580 2024-03-21T11:36:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.574+0000 7fc40fe00640 1 -- 172.21.15.47:0/3718615336 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418097b20 con 0x55c15c1e8580 2024-03-21T11:36:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.575+0000 7fc429047740 1 -- 172.21.15.47:0/3718615336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e8580 msgr2=0x55c15c2030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.575+0000 7fc429047740 1 --2- 172.21.15.47:0/3718615336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e8580 0x55c15c2030d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc4180cc880 tx=0x7fc41809af20 comp rx=0 tx=0).stop 2024-03-21T11:36:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.575+0000 7fc429047740 1 -- 172.21.15.47:0/3718615336 shutdown_connections 2024-03-21T11:36:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.575+0000 7fc429047740 1 --2- 172.21.15.47:0/3718615336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e8580 0x55c15c2030d0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.575+0000 7fc429047740 1 --2- 172.21.15.47:0/3718615336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.575+0000 7fc429047740 1 --2- 172.21.15.47:0/3718615336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e69f0 0x55c15c1e5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.575+0000 7fc429047740 1 -- 172.21.15.47:0/3718615336 >> 172.21.15.47:0/3718615336 conn(0x55c15c1e1de0 msgr2=0x55c15c1e1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.575+0000 7fc429047740 1 -- 172.21.15.47:0/3718615336 shutdown_connections 2024-03-21T11:36:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.575+0000 7fc429047740 1 -- 172.21.15.47:0/3718615336 wait complete. 2024-03-21T11:36:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.576+0000 7fc429047740 1 -- start start 2024-03-21T11:36:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.576+0000 7fc429047740 1 Processor -- start 2024-03-21T11:36:51.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.576+0000 7fc429047740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.576+0000 7fc429047740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.576+0000 7fc429047740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e8580 0x55c15c1f7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.576+0000 7fc429047740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4201ad330 con 0x55c15c1e7bc0 2024-03-21T11:36:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.576+0000 7fc429047740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4201accd0 con 0x55c15c1e69f0 2024-03-21T11:36:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.576+0000 7fc429047740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4201aa1d0 con 0x55c15c1e8580 2024-03-21T11:36:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.576+0000 7fc429047740 10 client.0 _ll_register_callbacks cb 0x7fc410009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.577+0000 7fc41f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.577+0000 7fc41f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1f3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60018/0 (socket says 172.21.15.47:60018) 2024-03-21T11:36:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.577+0000 7fc41f400640 1 -- 172.21.15.47:0/118649454 learned_addr learned my addr 172.21.15.47:0/118649454 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.577+0000 7fc41fe00640 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e8580 0x55c15c1f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.577+0000 7fc41ea00640 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.577+0000 7fc41f400640 1 -- 172.21.15.47:0/118649454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e8580 msgr2=0x55c15c1f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.577+0000 7fc41f400640 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e8580 0x55c15c1f7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.577+0000 7fc41f400640 1 -- 172.21.15.47:0/118649454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e7bc0 msgr2=0x55c15c1f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.577+0000 7fc41f400640 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.577+0000 7fc41f400640 1 -- 172.21.15.47:0/118649454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c15c1f7a00 con 0x55c15c1e69f0 2024-03-21T11:36:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.577+0000 7fc41ea00640 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.577+0000 7fc41fe00640 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e8580 0x55c15c1f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.577+0000 7fc41f400640 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1f3eb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc410009560 tx=0x7fc410009cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ab5d6638546a960 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.577+0000 7fc40d600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.578+0000 7fc40d600640 1 -- 172.21.15.47:0/118649454 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc410027040 con 0x55c15c1e69f0 2024-03-21T11:36:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.578+0000 7fc429047740 1 -- 172.21.15.47:0/118649454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc41809b4b0 con 0x55c15c1e69f0 2024-03-21T11:36:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.578+0000 7fc40d600640 1 -- 172.21.15.47:0/118649454 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc410011030 con 0x55c15c1e69f0 2024-03-21T11:36:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.578+0000 7fc40d600640 1 -- 172.21.15.47:0/118649454 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc410043050 con 0x55c15c1e69f0 2024-03-21T11:36:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.578+0000 7fc429047740 10 client.30023 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.578+0000 7fc429047740 1 -- 172.21.15.47:0/118649454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c15c258320 con 0x55c15c1e69f0 2024-03-21T11:36:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.578+0000 7fc40d600640 1 -- 172.21.15.47:0/118649454 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc41809b4b0 con 0x55c15c1e69f0 2024-03-21T11:36:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.578+0000 7fc40d600640 1 -- 172.21.15.47:0/118649454 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc410004e00 con 0x55c15c1e69f0 2024-03-21T11:36:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.579+0000 7fc429047740 10 client.30023 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.579+0000 7fc429047740 10 client.30023 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:36:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.579+0000 7fc429047740 20 client.30023 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.579+0000 7fc429047740 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c15c258b60 0x55c15c25af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.579+0000 7fc429047740 4 client.30023 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:36:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.579+0000 7fc429047740 1 -- 172.21.15.47:0/118649454 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c15c152b50 con 0x55c15c258b60 2024-03-21T11:36:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.579+0000 7fc41ea00640 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c15c258b60 0x55c15c25af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.579+0000 7fc41ea00640 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c15c258b60 0x55c15c25af80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8f02c010bf165481 server_cookie=cc2b3e43a55febed in_seq=0 out_seq=0 2024-03-21T11:36:51.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.579+0000 7fc40d600640 10 client.30023 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:36:51.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc40d600640 1 -- 172.21.15.47:0/118649454 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c15c152b50 con 0x55c15c258b60 2024-03-21T11:36:51.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc40d600640 10 client.30023 handle_command_reply: tid=0 2024-03-21T11:36:51.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc429047740 1 client.30023 shutdown 2024-03-21T11:36:51.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc429047740 1 -- 172.21.15.47:0/118649454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 msgr2=0x55c15c1f3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc429047740 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1f3eb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc410009560 tx=0x7fc410009cf0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc429047740 1 -- 172.21.15.47:0/118649454 shutdown_connections 2024-03-21T11:36:51.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc429047740 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c15c258b60 0x55c15c25af80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc429047740 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c15c1e8580 0x55c15c1f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc429047740 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c15c1e7bc0 0x55c15c1f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc429047740 1 --2- 172.21.15.47:0/118649454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c15c1e69f0 0x55c15c1f3eb0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc40d600640 0 client.30023 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:36:51.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc429047740 1 -- 172.21.15.47:0/118649454 >> 172.21.15.47:0/118649454 conn(0x55c15c1e1de0 msgr2=0x55c15c2030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc429047740 1 -- 172.21.15.47:0/118649454 shutdown_connections 2024-03-21T11:36:51.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc429047740 1 -- 172.21.15.47:0/118649454 wait complete. 2024-03-21T11:36:51.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.580+0000 7fc429047740 20 client.30023 trim_cache size 0 max 16384 2024-03-21T11:36:51.587 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:36:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.581+0000 7fc424e00640 1 -- 172.21.15.47:0/1864629241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc4040a2fa0 msgr2=0x7fc4040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.581+0000 7fc424e00640 1 --2- 172.21.15.47:0/1864629241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc4040a2fa0 0x7fc4040a5460 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc40802c250 tx=0x7fc40802e040 comp rx=0 tx=0).stop 2024-03-21T11:36:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.581+0000 7fc424e00640 1 -- 172.21.15.47:0/1864629241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc420105350 msgr2=0x7fc4201a1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.581+0000 7fc424e00640 1 --2- 172.21.15.47:0/1864629241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc420105350 0x7fc4201a1060 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc41800d750 tx=0x7fc41800dc10 comp rx=0 tx=0).stop 2024-03-21T11:36:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.581+0000 7fc424e00640 1 -- 172.21.15.47:0/1864629241 shutdown_connections 2024-03-21T11:36:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.581+0000 7fc424e00640 1 --2- 172.21.15.47:0/1864629241 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc4040a2fa0 0x7fc4040a5460 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.581+0000 7fc424e00640 1 --2- 172.21.15.47:0/1864629241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc420105350 0x7fc4201a1060 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.581+0000 7fc424e00640 1 --2- 172.21.15.47:0/1864629241 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc420104990 0x7fc420199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.581+0000 7fc424e00640 1 --2- 172.21.15.47:0/1864629241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4201037a0 0x7fc420199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.581+0000 7fc424e00640 1 -- 172.21.15.47:0/1864629241 >> 172.21.15.47:0/1864629241 conn(0x7fc4200fed10 msgr2=0x7fc420107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.581+0000 7fc424e00640 1 -- 172.21.15.47:0/1864629241 shutdown_connections 2024-03-21T11:36:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.581+0000 7fc424e00640 1 -- 172.21.15.47:0/1864629241 wait complete. 2024-03-21T11:36:51.589 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1034186 7.92 MB/sec execute 575 sec latency 11.766 ms 2024-03-21T11:36:51.601 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:36:51.601 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:36:51.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.826+0000 7fa719600640 1 -- 172.21.15.47:0/2297018212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714107bc0 msgr2=0x7fa714107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.826+0000 7fa719600640 1 --2- 172.21.15.47:0/2297018212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714107bc0 0x7fa714107fe0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009860 tx=0x7fa6fc02bb70 comp rx=0 tx=0).stop 2024-03-21T11:36:51.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.826+0000 7fa719600640 1 -- 172.21.15.47:0/2297018212 shutdown_connections 2024-03-21T11:36:51.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.826+0000 7fa719600640 1 --2- 172.21.15.47:0/2297018212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa714109770 0x7fa714110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.826+0000 7fa719600640 1 --2- 172.21.15.47:0/2297018212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa714108db0 0x7fa714109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.826+0000 7fa719600640 1 --2- 172.21.15.47:0/2297018212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714107bc0 0x7fa714107fe0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.826+0000 7fa719600640 1 -- 172.21.15.47:0/2297018212 >> 172.21.15.47:0/2297018212 conn(0x7fa714076b70 msgr2=0x7fa714078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.826+0000 7fa719600640 1 -- 172.21.15.47:0/2297018212 shutdown_connections 2024-03-21T11:36:51.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.826+0000 7fa719600640 1 -- 172.21.15.47:0/2297018212 wait complete. 2024-03-21T11:36:51.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.827+0000 7fa719600640 1 Processor -- start 2024-03-21T11:36:51.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.827+0000 7fa719600640 1 -- start start 2024-03-21T11:36:51.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa719600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714107bc0 0x7fa71419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa719600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa714108db0 0x7fa71419e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa719600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa714109770 0x7fa7141a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa719600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa714113810 con 0x7fa714108db0 2024-03-21T11:36:51.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa719600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa714113690 con 0x7fa714107bc0 2024-03-21T11:36:51.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa719600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa714113990 con 0x7fa714109770 2024-03-21T11:36:51.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa713e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714107bc0 0x7fa71419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa713e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714107bc0 0x7fa71419de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60042/0 (socket says 172.21.15.47:60042) 2024-03-21T11:36:51.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa713e00640 1 -- 172.21.15.47:0/1358255933 learned_addr learned my addr 172.21.15.47:0/1358255933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa718c00640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa714109770 0x7fa7141a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa713400640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa714108db0 0x7fa71419e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa713e00640 1 -- 172.21.15.47:0/1358255933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa714109770 msgr2=0x7fa7141a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa713e00640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa714109770 0x7fa7141a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa713e00640 1 -- 172.21.15.47:0/1358255933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa714108db0 msgr2=0x7fa71419e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa713e00640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa714108db0 0x7fa71419e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa713e00640 1 -- 172.21.15.47:0/1358255933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7141a5aa0 con 0x7fa714107bc0 2024-03-21T11:36:51.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.828+0000 7fa713400640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa714108db0 0x7fa71419e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:51.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.829+0000 7fa713e00640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714107bc0 0x7fa71419de80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0349e0 tx=0x7fa6fc032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.829+0000 7fa710c00640 1 -- 172.21.15.47:0/1358255933 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc041020 con 0x7fa714107bc0 2024-03-21T11:36:51.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.829+0000 7fa710c00640 1 -- 172.21.15.47:0/1358255933 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6fc02c9f0 con 0x7fa714107bc0 2024-03-21T11:36:51.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.829+0000 7fa710c00640 1 -- 172.21.15.47:0/1358255933 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc039040 con 0x7fa714107bc0 2024-03-21T11:36:51.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.829+0000 7fa719600640 1 -- 172.21.15.47:0/1358255933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7141a5d30 con 0x7fa714107bc0 2024-03-21T11:36:51.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.829+0000 7fa719600640 1 -- 172.21.15.47:0/1358255933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7141a61d0 con 0x7fa714107bc0 2024-03-21T11:36:51.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.832+0000 7fa710c00640 1 -- 172.21.15.47:0/1358255933 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa6fc044020 con 0x7fa714107bc0 2024-03-21T11:36:51.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.833+0000 7fa71d9e3740 1 Processor -- start 2024-03-21T11:36:51.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.833+0000 7fa71d9e3740 1 -- start start 2024-03-21T11:36:51.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.833+0000 7fa71d9e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38af9f0 0x5622b38afe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.833+0000 7fa71d9e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 0x5622b38b1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.833+0000 7fa71d9e3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38b1580 0x5622b38b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.833+0000 7fa71d9e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b38142c0 con 0x5622b38b0bc0 2024-03-21T11:36:51.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.833+0000 7fa71d9e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b37bcc90 con 0x5622b38af9f0 2024-03-21T11:36:51.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.833+0000 7fa71d9e3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b3810360 con 0x5622b38b1580 2024-03-21T11:36:51.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.833+0000 7fa718c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38af9f0 0x5622b38afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.833+0000 7fa713400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38b1580 0x5622b38b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.833+0000 7fa718c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38af9f0 0x5622b38afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60046/0 (socket says 172.21.15.47:60046) 2024-03-21T11:36:51.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.833+0000 7fa718c00640 1 -- 172.21.15.47:0/2514255907 learned_addr learned my addr 172.21.15.47:0/2514255907 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.834+0000 7fa713e00640 1 --2- 172.21.15.47:0/2514255907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 0x5622b38b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.834+0000 7fa718c00640 1 -- 172.21.15.47:0/2514255907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38b1580 msgr2=0x5622b38b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.834+0000 7fa718c00640 1 --2- 172.21.15.47:0/2514255907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38b1580 0x5622b38b7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.834+0000 7fa718c00640 1 -- 172.21.15.47:0/2514255907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 msgr2=0x5622b38b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.834+0000 7fa718c00640 1 --2- 172.21.15.47:0/2514255907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 0x5622b38b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.834+0000 7fa718c00640 1 -- 172.21.15.47:0/2514255907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622b37e1080 con 0x5622b38af9f0 2024-03-21T11:36:51.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.834+0000 7fa713e00640 1 --2- 172.21.15.47:0/2514255907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 0x5622b38b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:51.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.834+0000 7fa710c00640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6f80a2fa0 0x7fa6f80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.834+0000 7fa718c00640 1 --2- 172.21.15.47:0/2514255907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38af9f0 0x5622b38afe10 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00e580 tx=0x7fa70c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acbab1cb76f545b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.835+0000 7fa710c00640 1 -- 172.21.15.47:0/1358255933 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa6fc0f1070 con 0x7fa714107bc0 2024-03-21T11:36:51.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.835+0000 7fa713e00640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6f80a2fa0 0x7fa6f80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.835+0000 7fa709200640 1 -- 172.21.15.47:0/2514255907 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c003970 con 0x5622b38af9f0 2024-03-21T11:36:51.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.835+0000 7fa709200640 1 -- 172.21.15.47:0/2514255907 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa70c00d830 con 0x5622b38af9f0 2024-03-21T11:36:51.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.835+0000 7fa709200640 1 -- 172.21.15.47:0/2514255907 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c0045a0 con 0x5622b38af9f0 2024-03-21T11:36:51.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.835+0000 7fa71d9e3740 1 -- 172.21.15.47:0/2514255907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38af9f0 msgr2=0x5622b38afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.835+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/2514255907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38af9f0 0x5622b38afe10 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00e580 tx=0x7fa70c0095c0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.835+0000 7fa713e00640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6f80a2fa0 0x7fa6f80a5460 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0b9d80 tx=0x7fa6fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.836+0000 7fa71d9e3740 1 -- 172.21.15.47:0/2514255907 shutdown_connections 2024-03-21T11:36:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.836+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/2514255907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38b1580 0x5622b38b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.836+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/2514255907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 0x5622b38b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.836+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/2514255907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38af9f0 0x5622b38afe10 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.836+0000 7fa71d9e3740 1 -- 172.21.15.47:0/2514255907 >> 172.21.15.47:0/2514255907 conn(0x5622b38aade0 msgr2=0x5622b38ab1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.836+0000 7fa71d9e3740 1 -- 172.21.15.47:0/2514255907 shutdown_connections 2024-03-21T11:36:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.836+0000 7fa71d9e3740 1 -- 172.21.15.47:0/2514255907 wait complete. 2024-03-21T11:36:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.836+0000 7fa71d9e3740 1 -- start start 2024-03-21T11:36:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.836+0000 7fa71d9e3740 1 Processor -- start 2024-03-21T11:36:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.837+0000 7fa71d9e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38af9f0 0x5622b38b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.837+0000 7fa71d9e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 0x5622b38ba1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.837+0000 7fa71d9e3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38b1580 0x5622b38c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.837+0000 7fa71d9e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b38b8590 con 0x5622b38b0bc0 2024-03-21T11:36:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.837+0000 7fa71d9e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b37d00d0 con 0x5622b38af9f0 2024-03-21T11:36:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.837+0000 7fa71d9e3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b38b8710 con 0x5622b38b1580 2024-03-21T11:36:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.837+0000 7fa71d9e3740 10 client.0 _ll_register_callbacks cb 0x5622b37d4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.837+0000 7fa718c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38af9f0 0x5622b38b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.837+0000 7fa718c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38af9f0 0x5622b38b9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60050/0 (socket says 172.21.15.47:60050) 2024-03-21T11:36:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.837+0000 7fa718c00640 1 -- 172.21.15.47:0/2989180175 learned_addr learned my addr 172.21.15.47:0/2989180175 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.837+0000 7fa713400640 1 --2- 172.21.15.47:0/2989180175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 0x5622b38ba1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.837+0000 7fa718c00640 1 --2- 172.21.15.47:0/2989180175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38b1580 0x5622b38c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.838+0000 7fa713400640 1 -- 172.21.15.47:0/2989180175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38b1580 msgr2=0x5622b38c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.838+0000 7fa713400640 1 --2- 172.21.15.47:0/2989180175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38b1580 0x5622b38c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.838+0000 7fa713400640 1 -- 172.21.15.47:0/2989180175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38af9f0 msgr2=0x5622b38b9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.838+0000 7fa713400640 1 --2- 172.21.15.47:0/2989180175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38af9f0 0x5622b38b9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.838+0000 7fa713400640 1 -- 172.21.15.47:0/2989180175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622b35283f0 con 0x5622b38b0bc0 2024-03-21T11:36:51.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.838+0000 7fa718c00640 1 --2- 172.21.15.47:0/2989180175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38af9f0 0x5622b38b9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:51.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.838+0000 7fa713400640 1 --2- 172.21.15.47:0/2989180175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 0x5622b38ba1d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa71419f390 tx=0x7fa704012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9fac3b254b5046 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.838+0000 7fa701600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:36:51.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.838+0000 7fa701600640 1 -- 172.21.15.47:0/2989180175 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704004320 con 0x5622b38b0bc0 2024-03-21T11:36:51.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.838+0000 7fa701600640 1 -- 172.21.15.47:0/2989180175 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa704011940 con 0x5622b38b0bc0 2024-03-21T11:36:51.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.838+0000 7fa71d9e3740 1 -- 172.21.15.47:0/2989180175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa70c00d830 con 0x5622b38b0bc0 2024-03-21T11:36:51.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.839+0000 7fa701600640 1 -- 172.21.15.47:0/2989180175 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7040044c0 con 0x5622b38b0bc0 2024-03-21T11:36:51.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.839+0000 7fa71d9e3740 10 client.19632 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:51.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.839+0000 7fa71d9e3740 1 -- 172.21.15.47:0/2989180175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5622b35f27d0 con 0x5622b38b0bc0 2024-03-21T11:36:51.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.839+0000 7fa701600640 1 -- 172.21.15.47:0/2989180175 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa70c00d830 con 0x5622b38b0bc0 2024-03-21T11:36:51.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.839+0000 7fa701600640 1 -- 172.21.15.47:0/2989180175 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa70400f440 con 0x5622b38b0bc0 2024-03-21T11:36:51.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.839+0000 7fa71d9e3740 10 client.19632 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:51.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.839+0000 7fa71d9e3740 10 client.19632 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:36:51.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.839+0000 7fa71d9e3740 20 client.19632 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:51.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.839+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/2989180175 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622b38c57e0 0x5622b38c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.839+0000 7fa71d9e3740 4 client.19632 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:51.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.839+0000 7fa71d9e3740 1 -- 172.21.15.47:0/2989180175 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622b3821800 con 0x5622b38c57e0 2024-03-21T11:36:51.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.840+0000 7fa718c00640 1 --2- 172.21.15.47:0/2989180175 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622b38c57e0 0x5622b38c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.840+0000 7fa718c00640 1 --2- 172.21.15.47:0/2989180175 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622b38c57e0 0x5622b38c7c20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=edcdbb7404685198 server_cookie=73a05cef7beb2a94 in_seq=0 out_seq=0 2024-03-21T11:36:51.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.840+0000 7fa701600640 10 client.19632 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:36:51.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.841+0000 7fa701600640 1 -- 172.21.15.47:0/2989180175 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5622b3821800 con 0x5622b38c57e0 2024-03-21T11:36:51.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.841+0000 7fa701600640 10 client.19632 handle_command_reply: tid=0 2024-03-21T11:36:51.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.841+0000 7fa71d9e3740 1 client.19632 shutdown 2024-03-21T11:36:51.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.841+0000 7fa71d9e3740 1 -- 172.21.15.47:0/2989180175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 msgr2=0x5622b38ba1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.841+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/2989180175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 0x5622b38ba1d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa71419f390 tx=0x7fa704012b60 comp rx=0 tx=0).stop 2024-03-21T11:36:51.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.841+0000 7fa71d9e3740 1 -- 172.21.15.47:0/2989180175 shutdown_connections 2024-03-21T11:36:51.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.841+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/2989180175 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622b38c57e0 0x5622b38c7c20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.841+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/2989180175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38b1580 0x5622b38c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.841+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/2989180175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 0x5622b38ba1d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.841+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/2989180175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38af9f0 0x5622b38b9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.841+0000 7fa701600640 0 client.19632 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:36:51.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.841+0000 7fa71d9e3740 1 -- 172.21.15.47:0/2989180175 >> 172.21.15.47:0/2989180175 conn(0x5622b38aade0 msgr2=0x5622b38aa160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.841+0000 7fa71d9e3740 1 -- 172.21.15.47:0/2989180175 shutdown_connections 2024-03-21T11:36:51.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.841+0000 7fa71d9e3740 1 -- 172.21.15.47:0/2989180175 wait complete. 2024-03-21T11:36:51.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.842+0000 7fa71d9e3740 20 client.19632 trim_cache size 0 max 16384 2024-03-21T11:36:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.858+0000 7fa71d9e3740 1 Processor -- start 2024-03-21T11:36:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.858+0000 7fa71d9e3740 1 -- start start 2024-03-21T11:36:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.858+0000 7fa71d9e3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38af9f0 0x5622b38ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.858+0000 7fa71d9e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38b0bc0 0x5622b38c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.859+0000 7fa71d9e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b1580 0x5622b38b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.859+0000 7fa71d9e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141b0500 con 0x5622b38b1580 2024-03-21T11:36:51.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.859+0000 7fa71d9e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141b1e80 con 0x5622b38b0bc0 2024-03-21T11:36:51.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.859+0000 7fa71d9e3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141b0b60 con 0x5622b38af9f0 2024-03-21T11:36:51.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.859+0000 7fa718c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38af9f0 0x5622b38ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.859+0000 7fa713400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38b0bc0 0x5622b38c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.859+0000 7fa713400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38b0bc0 0x5622b38c11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60058/0 (socket says 172.21.15.47:60058) 2024-03-21T11:36:51.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.859+0000 7fa713400640 1 -- 172.21.15.47:0/1398884373 learned_addr learned my addr 172.21.15.47:0/1398884373 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.859+0000 7fa718c00640 1 --2- 172.21.15.47:0/1398884373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b1580 0x5622b38b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.859+0000 7fa713400640 1 -- 172.21.15.47:0/1398884373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38af9f0 msgr2=0x5622b38ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.859+0000 7fa713400640 1 --2- 172.21.15.47:0/1398884373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38af9f0 0x5622b38ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.859+0000 7fa713400640 1 -- 172.21.15.47:0/1398884373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b1580 msgr2=0x5622b38b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.859+0000 7fa713400640 1 --2- 172.21.15.47:0/1398884373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b1580 0x5622b38b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.859+0000 7fa713400640 1 -- 172.21.15.47:0/1398884373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622b38aab50 con 0x5622b38b0bc0 2024-03-21T11:36:51.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.860+0000 7fa713400640 1 --2- 172.21.15.47:0/1398884373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38b0bc0 0x5622b38c11d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa70400f930 tx=0x7fa704009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3d79779aed7aadf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.860+0000 7fa703e00640 1 -- 172.21.15.47:0/1398884373 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70402e020 con 0x5622b38b0bc0 2024-03-21T11:36:51.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.860+0000 7fa703e00640 1 -- 172.21.15.47:0/1398884373 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa70400f440 con 0x5622b38b0bc0 2024-03-21T11:36:51.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.860+0000 7fa703e00640 1 -- 172.21.15.47:0/1398884373 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704011810 con 0x5622b38b0bc0 2024-03-21T11:36:51.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.861+0000 7fa71d9e3740 1 -- 172.21.15.47:0/1398884373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38b0bc0 msgr2=0x5622b38c11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.861+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/1398884373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38b0bc0 0x5622b38c11d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa70400f930 tx=0x7fa704009e50 comp rx=0 tx=0).stop 2024-03-21T11:36:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.861+0000 7fa71d9e3740 1 -- 172.21.15.47:0/1398884373 shutdown_connections 2024-03-21T11:36:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.861+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/1398884373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b1580 0x5622b38b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.861+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/1398884373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38b0bc0 0x5622b38c11d0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.861+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/1398884373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38af9f0 0x5622b38ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.861+0000 7fa71d9e3740 1 -- 172.21.15.47:0/1398884373 >> 172.21.15.47:0/1398884373 conn(0x5622b38aade0 msgr2=0x5622b38ae040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.861+0000 7fa71d9e3740 1 -- 172.21.15.47:0/1398884373 shutdown_connections 2024-03-21T11:36:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.861+0000 7fa71d9e3740 1 -- 172.21.15.47:0/1398884373 wait complete. 2024-03-21T11:36:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.862+0000 7fa71d9e3740 1 -- start start 2024-03-21T11:36:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.862+0000 7fa71d9e3740 1 Processor -- start 2024-03-21T11:36:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa71d9e3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38af9f0 0x5622b38bedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa71d9e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 0x5622b38bf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa71d9e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38b1580 0x5622b38c02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa71d9e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141b1820 con 0x5622b38b0bc0 2024-03-21T11:36:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa71d9e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141b11c0 con 0x5622b38b1580 2024-03-21T11:36:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa71d9e3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141ae6c0 con 0x5622b38af9f0 2024-03-21T11:36:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa71d9e3740 10 client.0 _ll_register_callbacks cb 0x7fa704009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa718c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38af9f0 0x5622b38bedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa718c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38af9f0 0x5622b38bedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38164/0 (socket says 172.21.15.47:38164) 2024-03-21T11:36:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa718c00640 1 -- 172.21.15.47:0/1805239409 learned_addr learned my addr 172.21.15.47:0/1805239409 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa718c00640 1 --2- 172.21.15.47:0/1805239409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38b1580 0x5622b38c02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa713400640 1 --2- 172.21.15.47:0/1805239409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 0x5622b38bf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa718c00640 1 -- 172.21.15.47:0/1805239409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38b1580 msgr2=0x5622b38c02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa718c00640 1 --2- 172.21.15.47:0/1805239409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38b1580 0x5622b38c02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.863+0000 7fa718c00640 1 -- 172.21.15.47:0/1805239409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 msgr2=0x5622b38bf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.864+0000 7fa718c00640 1 --2- 172.21.15.47:0/1805239409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 0x5622b38bf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.864+0000 7fa718c00640 1 -- 172.21.15.47:0/1805239409 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622b38c0890 con 0x5622b38af9f0 2024-03-21T11:36:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.864+0000 7fa718c00640 1 --2- 172.21.15.47:0/1805239409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38af9f0 0x5622b38bedc0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00e580 tx=0x7fa70c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3064d9bf9894a4cb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.864+0000 7fa701600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.864+0000 7fa701600640 1 -- 172.21.15.47:0/1805239409 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c00ddd0 con 0x5622b38af9f0 2024-03-21T11:36:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.864+0000 7fa71d9e3740 1 -- 172.21.15.47:0/1805239409 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa70400f440 con 0x5622b38af9f0 2024-03-21T11:36:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.864+0000 7fa701600640 1 -- 172.21.15.47:0/1805239409 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa70c01c040 con 0x5622b38af9f0 2024-03-21T11:36:51.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.865+0000 7fa701600640 1 -- 172.21.15.47:0/1805239409 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa70c005370 con 0x5622b38af9f0 2024-03-21T11:36:51.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.865+0000 7fa71d9e3740 10 client.30076 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:51.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.865+0000 7fa71d9e3740 1 -- 172.21.15.47:0/1805239409 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5622b3921570 con 0x5622b38af9f0 2024-03-21T11:36:51.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.865+0000 7fa701600640 1 -- 172.21.15.47:0/1805239409 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa70400f440 con 0x5622b38af9f0 2024-03-21T11:36:51.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.865+0000 7fa701600640 1 -- 172.21.15.47:0/1805239409 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa70c00b040 con 0x5622b38af9f0 2024-03-21T11:36:51.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.865+0000 7fa71d9e3740 10 client.30076 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:51.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.865+0000 7fa71d9e3740 10 client.30076 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:36:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.865+0000 7fa71d9e3740 20 client.30076 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.865+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/1805239409 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622b3921cc0 0x5622b39240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.865+0000 7fa71d9e3740 4 client.30076 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:36:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.865+0000 7fa71d9e3740 1 -- 172.21.15.47:0/1805239409 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5622b381bb50 con 0x5622b3921cc0 2024-03-21T11:36:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.866+0000 7fa713400640 1 --2- 172.21.15.47:0/1805239409 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622b3921cc0 0x5622b39240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.866+0000 7fa713400640 1 --2- 172.21.15.47:0/1805239409 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622b3921cc0 0x5622b39240e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d20b0873dc873579 server_cookie=49bd923e52a35228 in_seq=0 out_seq=0 2024-03-21T11:36:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.866+0000 7fa701600640 10 client.30076 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:36:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.867+0000 7fa701600640 1 -- 172.21.15.47:0/1805239409 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5622b381bb50 con 0x5622b3921cc0 2024-03-21T11:36:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.867+0000 7fa701600640 10 client.30076 handle_command_reply: tid=0 2024-03-21T11:36:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.867+0000 7fa71d9e3740 1 client.30076 shutdown 2024-03-21T11:36:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.867+0000 7fa71d9e3740 1 -- 172.21.15.47:0/1805239409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38af9f0 msgr2=0x5622b38bedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.867+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/1805239409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38af9f0 0x5622b38bedc0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00e580 tx=0x7fa70c0095c0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.867+0000 7fa71d9e3740 1 -- 172.21.15.47:0/1805239409 shutdown_connections 2024-03-21T11:36:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.867+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/1805239409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622b38b1580 0x5622b38c02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.867+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/1805239409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622b38b0bc0 0x5622b38bf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.867+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/1805239409 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622b3921cc0 0x5622b39240e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.867+0000 7fa71d9e3740 1 --2- 172.21.15.47:0/1805239409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622b38af9f0 0x5622b38bedc0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.867+0000 7fa71d9e3740 1 -- 172.21.15.47:0/1805239409 >> 172.21.15.47:0/1805239409 conn(0x5622b38aade0 msgr2=0x5622b38ae8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.867+0000 7fa701600640 0 client.30076 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:36:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.867+0000 7fa71d9e3740 1 -- 172.21.15.47:0/1805239409 shutdown_connections 2024-03-21T11:36:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.867+0000 7fa71d9e3740 1 -- 172.21.15.47:0/1805239409 wait complete. 2024-03-21T11:36:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.868+0000 7fa71d9e3740 20 client.30076 trim_cache size 0 max 16384 2024-03-21T11:36:51.871 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:36:51.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.869+0000 7fa719600640 1 -- 172.21.15.47:0/1358255933 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6f80a2fa0 msgr2=0x7fa6f80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.869+0000 7fa719600640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6f80a2fa0 0x7fa6f80a5460 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0b9d80 tx=0x7fa6fc005f90 comp rx=0 tx=0).stop 2024-03-21T11:36:51.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.869+0000 7fa719600640 1 -- 172.21.15.47:0/1358255933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714107bc0 msgr2=0x7fa71419de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:51.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.869+0000 7fa719600640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714107bc0 0x7fa71419de80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0349e0 tx=0x7fa6fc032430 comp rx=0 tx=0).stop 2024-03-21T11:36:51.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.870+0000 7fa719600640 1 -- 172.21.15.47:0/1358255933 shutdown_connections 2024-03-21T11:36:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.870+0000 7fa719600640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6f80a2fa0 0x7fa6f80a5460 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.870+0000 7fa719600640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa714109770 0x7fa7141a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.870+0000 7fa719600640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa714108db0 0x7fa71419e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.870+0000 7fa719600640 1 --2- 172.21.15.47:0/1358255933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714107bc0 0x7fa71419de80 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.870+0000 7fa719600640 1 -- 172.21.15.47:0/1358255933 >> 172.21.15.47:0/1358255933 conn(0x7fa714076b70 msgr2=0x7fa71410c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.870+0000 7fa719600640 1 -- 172.21.15.47:0/1358255933 shutdown_connections 2024-03-21T11:36:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:51.870+0000 7fa719600640 1 -- 172.21.15.47:0/1358255933 wait complete. 2024-03-21T11:36:51.893 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:36:51.893 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:36:51.894 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:51 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/963730063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:36:51.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:51 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/963730063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:36:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.108+0000 7fa3d2a00640 1 -- 172.21.15.47:0/3586789747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3cc104990 msgr2=0x7fa3cc104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.108+0000 7fa3d2a00640 1 --2- 172.21.15.47:0/3586789747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3cc104990 0x7fa3cc104e10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009900 tx=0x7fa3b802bc30 comp rx=0 tx=0).stop 2024-03-21T11:36:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.109+0000 7fa3d2a00640 1 -- 172.21.15.47:0/3586789747 shutdown_connections 2024-03-21T11:36:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.109+0000 7fa3d2a00640 1 --2- 172.21.15.47:0/3586789747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc105350 0x7fa3cc10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.109+0000 7fa3d2a00640 1 --2- 172.21.15.47:0/3586789747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3cc104990 0x7fa3cc104e10 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.109+0000 7fa3d2a00640 1 --2- 172.21.15.47:0/3586789747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3cc1037a0 0x7fa3cc103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.109+0000 7fa3d2a00640 1 -- 172.21.15.47:0/3586789747 >> 172.21.15.47:0/3586789747 conn(0x7fa3cc0fed10 msgr2=0x7fa3cc101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.109+0000 7fa3d2a00640 1 -- 172.21.15.47:0/3586789747 shutdown_connections 2024-03-21T11:36:52.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.109+0000 7fa3d2a00640 1 -- 172.21.15.47:0/3586789747 wait complete. 2024-03-21T11:36:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.110+0000 7fa3d2a00640 1 Processor -- start 2024-03-21T11:36:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.110+0000 7fa3d2a00640 1 -- start start 2024-03-21T11:36:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.110+0000 7fa3d2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3cc1037a0 0x7fa3cc199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.110+0000 7fa3d2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3cc104990 0x7fa3cc199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.110+0000 7fa3d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc105350 0x7fa3cc1a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.110+0000 7fa3d2a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc10f450 con 0x7fa3cc1037a0 2024-03-21T11:36:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.110+0000 7fa3d2a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc10f2d0 con 0x7fa3cc104990 2024-03-21T11:36:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.110+0000 7fa3d2a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc10f5d0 con 0x7fa3cc105350 2024-03-21T11:36:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.111+0000 7fa3d2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc105350 0x7fa3cc1a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.111+0000 7fa3d2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc105350 0x7fa3cc1a1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38186/0 (socket says 172.21.15.47:38186) 2024-03-21T11:36:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.111+0000 7fa3d2000640 1 -- 172.21.15.47:0/2444994882 learned_addr learned my addr 172.21.15.47:0/2444994882 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.111+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3cc104990 0x7fa3cc199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.111+0000 7fa3d1600640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3cc1037a0 0x7fa3cc199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.111+0000 7fa3d2000640 1 -- 172.21.15.47:0/2444994882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3cc104990 msgr2=0x7fa3cc199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.111+0000 7fa3d2000640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3cc104990 0x7fa3cc199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.111+0000 7fa3d2000640 1 -- 172.21.15.47:0/2444994882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3cc1037a0 msgr2=0x7fa3cc199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.111+0000 7fa3d2000640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3cc1037a0 0x7fa3cc199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.111+0000 7fa3d2000640 1 -- 172.21.15.47:0/2444994882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc1a1690 con 0x7fa3cc105350 2024-03-21T11:36:52.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.111+0000 7fa3d1600640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3cc1037a0 0x7fa3cc199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:36:52.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.111+0000 7fa3d2000640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc105350 0x7fa3cc1a1020 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400eb10 tx=0x7fa3c400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:52.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.112+0000 7fa3ca000640 1 -- 172.21.15.47:0/2444994882 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4004b20 con 0x7fa3cc105350 2024-03-21T11:36:52.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.112+0000 7fa3ca000640 1 -- 172.21.15.47:0/2444994882 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3c4004cc0 con 0x7fa3cc105350 2024-03-21T11:36:52.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.112+0000 7fa3ca000640 1 -- 172.21.15.47:0/2444994882 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4010760 con 0x7fa3cc105350 2024-03-21T11:36:52.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.112+0000 7fa3d2a00640 1 -- 172.21.15.47:0/2444994882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc1a1980 con 0x7fa3cc105350 2024-03-21T11:36:52.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.112+0000 7fa3d2a00640 1 -- 172.21.15.47:0/2444994882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc1a1e50 con 0x7fa3cc105350 2024-03-21T11:36:52.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.115+0000 7fa3ca000640 1 -- 172.21.15.47:0/2444994882 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa3c4005020 con 0x7fa3cc105350 2024-03-21T11:36:52.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.115+0000 7fa3d6e0f740 1 Processor -- start 2024-03-21T11:36:52.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.116+0000 7fa3d6e0f740 1 -- start start 2024-03-21T11:36:52.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.116+0000 7fa3d6e0f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 0x55d6ec16ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.116+0000 7fa3d6e0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16dbc0 0x55d6ec16e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.116+0000 7fa3d6e0f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 0x55d6ec174e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.116+0000 7fa3d6e0f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ec0d12c0 con 0x55d6ec16c9f0 2024-03-21T11:36:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.116+0000 7fa3d6e0f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ec079c90 con 0x55d6ec16e580 2024-03-21T11:36:52.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.116+0000 7fa3d6e0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ec0cd360 con 0x55d6ec16dbc0 2024-03-21T11:36:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.116+0000 7fa3d0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 0x55d6ec16ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.116+0000 7fa3d1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16dbc0 0x55d6ec16e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.116+0000 7fa3d0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 0x55d6ec16ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43976/0 (socket says 172.21.15.47:43976) 2024-03-21T11:36:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.116+0000 7fa3d0c00640 1 -- 172.21.15.47:0/3791917842 learned_addr learned my addr 172.21.15.47:0/3791917842 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.116+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/3791917842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 0x55d6ec174e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.117+0000 7fa3ca000640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa3b40a2f20 0x7fa3b40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.117+0000 7fa3d1600640 1 -- 172.21.15.47:0/3791917842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 msgr2=0x55d6ec174e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.117+0000 7fa3d1600640 1 --2- 172.21.15.47:0/3791917842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 0x55d6ec174e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.117+0000 7fa3d1600640 1 -- 172.21.15.47:0/3791917842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 msgr2=0x55d6ec16ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.117+0000 7fa3d1600640 1 --2- 172.21.15.47:0/3791917842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 0x55d6ec16ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.117+0000 7fa3d1600640 1 -- 172.21.15.47:0/3791917842 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6ec09e080 con 0x55d6ec16dbc0 2024-03-21T11:36:52.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.117+0000 7fa3ca000640 1 -- 172.21.15.47:0/2444994882 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa3c40cfde0 con 0x7fa3cc105350 2024-03-21T11:36:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.117+0000 7fa3d2000640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa3b40a2f20 0x7fa3b40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.117+0000 7fa3d1600640 1 --2- 172.21.15.47:0/3791917842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16dbc0 0x55d6ec16e040 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009c30 tx=0x7fa3c0010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c3e934d4f77ed96 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.118+0000 7fa3be600640 1 -- 172.21.15.47:0/3791917842 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0005020 con 0x55d6ec16dbc0 2024-03-21T11:36:52.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.118+0000 7fa3d2000640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa3b40a2f20 0x7fa3b40a53e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40cd4f0 tx=0x7fa3c40058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:52.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.118+0000 7fa3be600640 1 -- 172.21.15.47:0/3791917842 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3c00047a0 con 0x55d6ec16dbc0 2024-03-21T11:36:52.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.118+0000 7fa3be600640 1 -- 172.21.15.47:0/3791917842 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0012740 con 0x55d6ec16dbc0 2024-03-21T11:36:52.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.118+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/3791917842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16dbc0 msgr2=0x55d6ec16e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.118+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/3791917842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16dbc0 0x55d6ec16e040 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009c30 tx=0x7fa3c0010740 comp rx=0 tx=0).stop 2024-03-21T11:36:52.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.119+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/3791917842 shutdown_connections 2024-03-21T11:36:52.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.119+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/3791917842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 0x55d6ec174e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.119+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/3791917842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16dbc0 0x55d6ec16e040 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.119+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/3791917842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 0x55d6ec16ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.119+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/3791917842 >> 172.21.15.47:0/3791917842 conn(0x55d6ec167de0 msgr2=0x55d6ec1681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:52.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.119+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/3791917842 shutdown_connections 2024-03-21T11:36:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.119+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/3791917842 wait complete. 2024-03-21T11:36:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.119+0000 7fa3d6e0f740 1 -- start start 2024-03-21T11:36:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.119+0000 7fa3d6e0f740 1 Processor -- start 2024-03-21T11:36:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.120+0000 7fa3d6e0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16c9f0 0x55d6ec176c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.120+0000 7fa3d6e0f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16dbc0 0x55d6ec1771a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.120+0000 7fa3d6e0f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 0x55d6ec17e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.120+0000 7fa3d6e0f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ec175590 con 0x55d6ec16dbc0 2024-03-21T11:36:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.120+0000 7fa3d6e0f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ec08d0d0 con 0x55d6ec16e580 2024-03-21T11:36:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.120+0000 7fa3d6e0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ec175710 con 0x55d6ec16c9f0 2024-03-21T11:36:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.120+0000 7fa3d6e0f740 10 client.0 _ll_register_callbacks cb 0x55d6ec091e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.120+0000 7fa3d0c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16c9f0 0x55d6ec176c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.120+0000 7fa3d1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16dbc0 0x55d6ec1771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.120+0000 7fa3d0c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16c9f0 0x55d6ec176c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38204/0 (socket says 172.21.15.47:38204) 2024-03-21T11:36:52.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.120+0000 7fa3d0c00640 1 -- 172.21.15.47:0/3547596982 learned_addr learned my addr 172.21.15.47:0/3547596982 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:52.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.120+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/3547596982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 0x55d6ec17e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.121+0000 7fa3d0c00640 1 -- 172.21.15.47:0/3547596982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 msgr2=0x55d6ec17e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.121+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/3547596982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 0x55d6ec17e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.121+0000 7fa3d0c00640 1 -- 172.21.15.47:0/3547596982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16dbc0 msgr2=0x55d6ec1771a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.121+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/3547596982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16dbc0 0x55d6ec1771a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.121+0000 7fa3d0c00640 1 -- 172.21.15.47:0/3547596982 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6ebde53f0 con 0x55d6ec16c9f0 2024-03-21T11:36:52.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.121+0000 7fa3d1600640 1 --2- 172.21.15.47:0/3547596982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16dbc0 0x55d6ec1771a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:52.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.121+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/3547596982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16c9f0 0x55d6ec176c60 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa3b802c8e0 tx=0x7fa3b802df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70e95c8c8dd2622d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:52.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.121+0000 7fa3b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:52.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.121+0000 7fa3b2a00640 1 -- 172.21.15.47:0/3547596982 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80048d0 con 0x55d6ec16c9f0 2024-03-21T11:36:52.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.121+0000 7fa3b2a00640 1 -- 172.21.15.47:0/3547596982 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa3b8004a70 con 0x55d6ec16c9f0 2024-03-21T11:36:52.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.121+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/3547596982 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3c00047a0 con 0x55d6ec16c9f0 2024-03-21T11:36:52.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.122+0000 7fa3b2a00640 1 -- 172.21.15.47:0/3547596982 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3b8004c70 con 0x55d6ec16c9f0 2024-03-21T11:36:52.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.122+0000 7fa3d6e0f740 10 client.30088 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:52.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.122+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/3547596982 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d6ebeaf7d0 con 0x55d6ec16c9f0 2024-03-21T11:36:52.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.122+0000 7fa3b2a00640 1 -- 172.21.15.47:0/3547596982 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3c00047a0 con 0x55d6ec16c9f0 2024-03-21T11:36:52.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.122+0000 7fa3b2a00640 1 -- 172.21.15.47:0/3547596982 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa3b804f070 con 0x55d6ec16c9f0 2024-03-21T11:36:52.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.122+0000 7fa3d6e0f740 10 client.30088 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:52.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.122+0000 7fa3d6e0f740 10 client.30088 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:36:52.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.122+0000 7fa3d6e0f740 20 client.30088 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:52.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.122+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/3547596982 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55d6ec182760 0x55d6ec184bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.122+0000 7fa3d6e0f740 4 client.30088 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:52.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.122+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/3547596982 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6ec0de800 con 0x55d6ec182760 2024-03-21T11:36:52.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.123+0000 7fa3d1600640 1 --2- 172.21.15.47:0/3547596982 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55d6ec182760 0x55d6ec184bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.123+0000 7fa3d1600640 1 --2- 172.21.15.47:0/3547596982 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55d6ec182760 0x55d6ec184bc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ddafd1850d26b83 server_cookie=7b384bf1ed249097 in_seq=0 out_seq=0 2024-03-21T11:36:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.123+0000 7fa3b2a00640 10 client.30088 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:36:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.124+0000 7fa3b2a00640 1 -- 172.21.15.47:0/3547596982 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d6ec0de800 con 0x55d6ec182760 2024-03-21T11:36:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.124+0000 7fa3b2a00640 10 client.30088 handle_command_reply: tid=0 2024-03-21T11:36:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.124+0000 7fa3d6e0f740 1 client.30088 shutdown 2024-03-21T11:36:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.124+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/3547596982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16c9f0 msgr2=0x55d6ec176c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.124+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/3547596982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16c9f0 0x55d6ec176c60 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa3b802c8e0 tx=0x7fa3b802df90 comp rx=0 tx=0).stop 2024-03-21T11:36:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.124+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/3547596982 shutdown_connections 2024-03-21T11:36:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.124+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/3547596982 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55d6ec182760 0x55d6ec184bc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.124+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/3547596982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 0x55d6ec17e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.124+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/3547596982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16dbc0 0x55d6ec1771a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.124+0000 7fa3b2a00640 0 client.30088 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:36:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.124+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/3547596982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16c9f0 0x55d6ec176c60 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.124+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/3547596982 >> 172.21.15.47:0/3547596982 conn(0x55d6ec167de0 msgr2=0x55d6ec167160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:52.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.125+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/3547596982 shutdown_connections 2024-03-21T11:36:52.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.125+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/3547596982 wait complete. 2024-03-21T11:36:52.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.125+0000 7fa3d6e0f740 20 client.30088 trim_cache size 0 max 16384 2024-03-21T11:36:52.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.141+0000 7fa3d6e0f740 1 Processor -- start 2024-03-21T11:36:52.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.141+0000 7fa3d6e0f740 1 -- start start 2024-03-21T11:36:52.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.141+0000 7fa3d6e0f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 0x55d6ec177100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.141+0000 7fa3d6e0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16dbc0 0x55d6ec17e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.141+0000 7fa3d6e0f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 0x55d6ec16b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.142+0000 7fa3d6e0f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc1ac030 con 0x55d6ec16c9f0 2024-03-21T11:36:52.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.142+0000 7fa3d6e0f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc1ad9b0 con 0x55d6ec16e580 2024-03-21T11:36:52.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.142+0000 7fa3d6e0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc1ac690 con 0x55d6ec16dbc0 2024-03-21T11:36:52.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.142+0000 7fa3d0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 0x55d6ec177100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.142+0000 7fa3d0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 0x55d6ec177100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43996/0 (socket says 172.21.15.47:43996) 2024-03-21T11:36:52.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.142+0000 7fa3d0c00640 1 -- 172.21.15.47:0/720312242 learned_addr learned my addr 172.21.15.47:0/720312242 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:52.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.142+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/720312242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 0x55d6ec16b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.143+0000 7fa3d1600640 1 --2- 172.21.15.47:0/720312242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16dbc0 0x55d6ec17e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.143+0000 7fa3d0c00640 1 -- 172.21.15.47:0/720312242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16dbc0 msgr2=0x55d6ec17e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.143+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/720312242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16dbc0 0x55d6ec17e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.143+0000 7fa3d0c00640 1 -- 172.21.15.47:0/720312242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 msgr2=0x55d6ec177100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.143+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/720312242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 0x55d6ec177100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.143+0000 7fa3d0c00640 1 -- 172.21.15.47:0/720312242 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6ec177de0 con 0x55d6ec16e580 2024-03-21T11:36:52.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.144+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/720312242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 0x55d6ec16b750 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa3b802c170 tx=0x7fa3b8009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9781c94f358a466f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:52.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.144+0000 7fa3bdc00640 1 -- 172.21.15.47:0/720312242 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80048d0 con 0x55d6ec16e580 2024-03-21T11:36:52.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.144+0000 7fa3bdc00640 1 -- 172.21.15.47:0/720312242 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3b8047070 con 0x55d6ec16e580 2024-03-21T11:36:52.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.144+0000 7fa3bdc00640 1 -- 172.21.15.47:0/720312242 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80052c0 con 0x55d6ec16e580 2024-03-21T11:36:52.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.145+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/720312242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 msgr2=0x55d6ec16b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.145+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/720312242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 0x55d6ec16b750 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa3b802c170 tx=0x7fa3b8009620 comp rx=0 tx=0).stop 2024-03-21T11:36:52.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.145+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/720312242 shutdown_connections 2024-03-21T11:36:52.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.145+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/720312242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16e580 0x55d6ec16b750 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.145+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/720312242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16dbc0 0x55d6ec17e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.145+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/720312242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 0x55d6ec177100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.145+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/720312242 >> 172.21.15.47:0/720312242 conn(0x55d6ec167de0 msgr2=0x55d6ec16b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:52.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.146+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/720312242 shutdown_connections 2024-03-21T11:36:52.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.146+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/720312242 wait complete. 2024-03-21T11:36:52.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.146+0000 7fa3d6e0f740 1 -- start start 2024-03-21T11:36:52.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.146+0000 7fa3d6e0f740 1 Processor -- start 2024-03-21T11:36:52.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d6e0f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 0x55d6ec179df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d6e0f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16dbc0 0x55d6ec17a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d6e0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16e580 0x55d6ec17d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d6e0f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc1ad350 con 0x55d6ec16c9f0 2024-03-21T11:36:52.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d6e0f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc1accf0 con 0x55d6ec16dbc0 2024-03-21T11:36:52.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d6e0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc1aa1f0 con 0x55d6ec16e580 2024-03-21T11:36:52.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d6e0f740 10 client.0 _ll_register_callbacks cb 0x7fa3b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:52.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16dbc0 0x55d6ec17a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16dbc0 0x55d6ec17a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60136/0 (socket says 172.21.15.47:60136) 2024-03-21T11:36:52.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d1600640 1 -- 172.21.15.47:0/4170051238 learned_addr learned my addr 172.21.15.47:0/4170051238 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:52.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/4170051238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 0x55d6ec179df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/4170051238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16e580 0x55d6ec17d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d1600640 1 -- 172.21.15.47:0/4170051238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16e580 msgr2=0x55d6ec17d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d1600640 1 --2- 172.21.15.47:0/4170051238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16e580 0x55d6ec17d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d1600640 1 -- 172.21.15.47:0/4170051238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 msgr2=0x55d6ec179df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d1600640 1 --2- 172.21.15.47:0/4170051238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 0x55d6ec179df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d1600640 1 -- 172.21.15.47:0/4170051238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6ec17d940 con 0x55d6ec16dbc0 2024-03-21T11:36:52.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.147+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/4170051238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16e580 0x55d6ec17d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:36:52.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.148+0000 7fa3d1600640 1 --2- 172.21.15.47:0/4170051238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16dbc0 0x55d6ec17a330 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x55d6ec168d50 tx=0x7fa3c0005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c18983d351aab8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:52.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.148+0000 7fa3b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:52.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.148+0000 7fa3b2a00640 1 -- 172.21.15.47:0/4170051238 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c002b1d0 con 0x55d6ec16dbc0 2024-03-21T11:36:52.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.148+0000 7fa3b2a00640 1 -- 172.21.15.47:0/4170051238 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa3c0004640 con 0x55d6ec16dbc0 2024-03-21T11:36:52.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.148+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/4170051238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3b8047070 con 0x55d6ec16dbc0 2024-03-21T11:36:52.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.148+0000 7fa3b2a00640 1 -- 172.21.15.47:0/4170051238 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3c0009d20 con 0x55d6ec16dbc0 2024-03-21T11:36:52.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.148+0000 7fa3d6e0f740 10 client.30059 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:52.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.148+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/4170051238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d6ec1de680 con 0x55d6ec16dbc0 2024-03-21T11:36:52.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.149+0000 7fa3b2a00640 1 -- 172.21.15.47:0/4170051238 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3b8047070 con 0x55d6ec16dbc0 2024-03-21T11:36:52.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.149+0000 7fa3b2a00640 1 -- 172.21.15.47:0/4170051238 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa3c000f040 con 0x55d6ec16dbc0 2024-03-21T11:36:52.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.149+0000 7fa3d6e0f740 10 client.30059 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:52.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.149+0000 7fa3d6e0f740 10 client.30059 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:36:52.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.149+0000 7fa3d6e0f740 20 client.30059 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:52.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.149+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/4170051238 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55d6ec1dee90 0x55d6ec1e12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.149+0000 7fa3d6e0f740 4 client.30059 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:36:52.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.149+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/4170051238 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d6ec0d8b50 con 0x55d6ec1dee90 2024-03-21T11:36:52.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.149+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/4170051238 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55d6ec1dee90 0x55d6ec1e12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.150+0000 7fa3d0c00640 1 --2- 172.21.15.47:0/4170051238 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55d6ec1dee90 0x55d6ec1e12b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d172a7c43716393 server_cookie=5ea33216ddcfa752 in_seq=0 out_seq=0 2024-03-21T11:36:52.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.150+0000 7fa3b2a00640 10 client.30059 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:36:52.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.150+0000 7fa3b2a00640 1 -- 172.21.15.47:0/4170051238 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d6ec0d8b50 con 0x55d6ec1dee90 2024-03-21T11:36:52.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.150+0000 7fa3b2a00640 10 client.30059 handle_command_reply: tid=0 2024-03-21T11:36:52.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.150+0000 7fa3d6e0f740 1 client.30059 shutdown 2024-03-21T11:36:52.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.150+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/4170051238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16dbc0 msgr2=0x55d6ec17a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.150+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/4170051238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16dbc0 0x55d6ec17a330 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x55d6ec168d50 tx=0x7fa3c0005db0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.150+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/4170051238 shutdown_connections 2024-03-21T11:36:52.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.150+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/4170051238 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55d6ec1dee90 0x55d6ec1e12b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.150+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/4170051238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d6ec16e580 0x55d6ec17d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.150+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/4170051238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6ec16dbc0 0x55d6ec17a330 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.150+0000 7fa3d6e0f740 1 --2- 172.21.15.47:0/4170051238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d6ec16c9f0 0x55d6ec179df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.150+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/4170051238 >> 172.21.15.47:0/4170051238 conn(0x55d6ec167de0 msgr2=0x55d6ec176f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:52.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.151+0000 7fa3b2a00640 0 client.30059 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:36:52.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.151+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/4170051238 shutdown_connections 2024-03-21T11:36:52.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.151+0000 7fa3d6e0f740 1 -- 172.21.15.47:0/4170051238 wait complete. 2024-03-21T11:36:52.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.151+0000 7fa3d6e0f740 20 client.30059 trim_cache size 0 max 16384 2024-03-21T11:36:52.159 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:36:52.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.152+0000 7fa3d2a00640 1 -- 172.21.15.47:0/2444994882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa3b40a2f20 msgr2=0x7fa3b40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.152+0000 7fa3d2a00640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa3b40a2f20 0x7fa3b40a53e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40cd4f0 tx=0x7fa3c40058d0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.152+0000 7fa3d2a00640 1 -- 172.21.15.47:0/2444994882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc105350 msgr2=0x7fa3cc1a1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.152+0000 7fa3d2a00640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc105350 0x7fa3cc1a1020 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400eb10 tx=0x7fa3c400c2d0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.152+0000 7fa3d2a00640 1 -- 172.21.15.47:0/2444994882 shutdown_connections 2024-03-21T11:36:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.152+0000 7fa3d2a00640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa3b40a2f20 0x7fa3b40a53e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.152+0000 7fa3d2a00640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3cc105350 0x7fa3cc1a1020 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.152+0000 7fa3d2a00640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3cc104990 0x7fa3cc199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.152+0000 7fa3d2a00640 1 --2- 172.21.15.47:0/2444994882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3cc1037a0 0x7fa3cc199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.152+0000 7fa3d2a00640 1 -- 172.21.15.47:0/2444994882 >> 172.21.15.47:0/2444994882 conn(0x7fa3cc0fed10 msgr2=0x7fa3cc107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:52.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.152+0000 7fa3d2a00640 1 -- 172.21.15.47:0/2444994882 shutdown_connections 2024-03-21T11:36:52.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.152+0000 7fa3d2a00640 1 -- 172.21.15.47:0/2444994882 wait complete. 2024-03-21T11:36:52.172 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:36:52.172 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:36:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.384+0000 7f5dc1e00640 1 -- 172.21.15.47:0/322013913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dbc108910 msgr2=0x7f5dbc10ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.384+0000 7f5dc1e00640 1 --2- 172.21.15.47:0/322013913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dbc108910 0x7f5dbc10ad00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009370 tx=0x7f5dac02bc30 comp rx=0 tx=0).stop 2024-03-21T11:36:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.384+0000 7f5dc1e00640 1 -- 172.21.15.47:0/322013913 shutdown_connections 2024-03-21T11:36:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.384+0000 7f5dc1e00640 1 --2- 172.21.15.47:0/322013913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dbc10dd50 0x7f5dbc110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.384+0000 7f5dc1e00640 1 --2- 172.21.15.47:0/322013913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc10b2d0 0x7f5dbc10d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.384+0000 7f5dc1e00640 1 --2- 172.21.15.47:0/322013913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dbc108910 0x7f5dbc10ad00 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.384+0000 7f5dc1e00640 1 -- 172.21.15.47:0/322013913 >> 172.21.15.47:0/322013913 conn(0x7f5dbc06e8a0 msgr2=0x7f5dbc070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.384+0000 7f5dc1e00640 1 -- 172.21.15.47:0/322013913 shutdown_connections 2024-03-21T11:36:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.384+0000 7f5dc1e00640 1 -- 172.21.15.47:0/322013913 wait complete. 2024-03-21T11:36:52.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.385+0000 7f5dc1e00640 1 Processor -- start 2024-03-21T11:36:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.385+0000 7f5dc1e00640 1 -- start start 2024-03-21T11:36:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dc1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dbc108910 0x7f5dbc1a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dc1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dbc10b2d0 0x7f5dbc1a2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dc1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc10dd50 0x7f5dbc1a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dc1e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc113980 con 0x7f5dbc10dd50 2024-03-21T11:36:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dc1e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc113800 con 0x7f5dbc10b2d0 2024-03-21T11:36:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dc1e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc113b00 con 0x7f5dbc108910 2024-03-21T11:36:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dbaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dbc108910 0x7f5dbc1a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dbaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dbc108910 0x7f5dbc1a2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38250/0 (socket says 172.21.15.47:38250) 2024-03-21T11:36:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dbaa00640 1 -- 172.21.15.47:0/3566496206 learned_addr learned my addr 172.21.15.47:0/3566496206 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dba000640 1 --2- 172.21.15.47:0/3566496206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dbc10b2d0 0x7f5dbc1a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dbb400640 1 --2- 172.21.15.47:0/3566496206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc10dd50 0x7f5dbc1a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dbaa00640 1 -- 172.21.15.47:0/3566496206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dbc10b2d0 msgr2=0x7f5dbc1a2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dbaa00640 1 --2- 172.21.15.47:0/3566496206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dbc10b2d0 0x7f5dbc1a2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dbaa00640 1 -- 172.21.15.47:0/3566496206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc10dd50 msgr2=0x7f5dbc1a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dbaa00640 1 --2- 172.21.15.47:0/3566496206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc10dd50 0x7f5dbc1a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.386+0000 7f5dbaa00640 1 -- 172.21.15.47:0/3566496206 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc1aa040 con 0x7f5dbc108910 2024-03-21T11:36:52.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.387+0000 7f5dbaa00640 1 --2- 172.21.15.47:0/3566496206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dbc108910 0x7f5dbc1a2420 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5dac02c240 tx=0x7f5dac02c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:52.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.387+0000 7f5dc0a00640 1 -- 172.21.15.47:0/3566496206 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac042020 con 0x7f5dbc108910 2024-03-21T11:36:52.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.387+0000 7f5dc0a00640 1 -- 172.21.15.47:0/3566496206 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5dac004450 con 0x7f5dbc108910 2024-03-21T11:36:52.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.387+0000 7f5dc0a00640 1 -- 172.21.15.47:0/3566496206 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac03e460 con 0x7f5dbc108910 2024-03-21T11:36:52.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.387+0000 7f5dc1e00640 1 -- 172.21.15.47:0/3566496206 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dbc1aa2d0 con 0x7f5dbc108910 2024-03-21T11:36:52.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.388+0000 7f5dc1e00640 1 -- 172.21.15.47:0/3566496206 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dbc1aa770 con 0x7f5dbc108910 2024-03-21T11:36:52.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.392+0000 7f5dc4be8740 1 Processor -- start 2024-03-21T11:36:52.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.392+0000 7f5dc4be8740 1 -- start start 2024-03-21T11:36:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dc0a00640 1 -- 172.21.15.47:0/3566496206 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5dac045020 con 0x7f5dbc108910 2024-03-21T11:36:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dc4be8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c4361810 0x55c8c4361c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dc4be8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43629e0 0x55c8c4362e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dc4be8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43633a0 0x55c8c4369c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dc4be8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c42c62c0 con 0x55c8c43633a0 2024-03-21T11:36:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dc4be8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c426ec90 con 0x55c8c4361810 2024-03-21T11:36:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dc4be8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c42c2360 con 0x55c8c43629e0 2024-03-21T11:36:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dbaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43629e0 0x55c8c4362e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dbb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c4361810 0x55c8c4361c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dbb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c4361810 0x55c8c4361c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60160/0 (socket says 172.21.15.47:60160) 2024-03-21T11:36:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dbb400640 1 -- 172.21.15.47:0/1890508168 learned_addr learned my addr 172.21.15.47:0/1890508168 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dba000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43633a0 0x55c8c4369c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dbb400640 1 -- 172.21.15.47:0/1890508168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43629e0 msgr2=0x55c8c4362e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dbb400640 1 --2- 172.21.15.47:0/1890508168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43629e0 0x55c8c4362e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dbb400640 1 -- 172.21.15.47:0/1890508168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43633a0 msgr2=0x55c8c4369c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dbb400640 1 --2- 172.21.15.47:0/1890508168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43633a0 0x55c8c4369c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.393+0000 7f5dbb400640 1 -- 172.21.15.47:0/1890508168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8c4293080 con 0x55c8c4361810 2024-03-21T11:36:52.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.394+0000 7f5dbb400640 1 --2- 172.21.15.47:0/1890508168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c4361810 0x55c8c4361c30 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5db400af20 tx=0x7f5db400e8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d466062f1aab40f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:52.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.394+0000 7f5dabe00640 1 -- 172.21.15.47:0/1890508168 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4004b20 con 0x55c8c4361810 2024-03-21T11:36:52.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.394+0000 7f5dabe00640 1 -- 172.21.15.47:0/1890508168 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5db4004cc0 con 0x55c8c4361810 2024-03-21T11:36:52.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.394+0000 7f5dabe00640 1 -- 172.21.15.47:0/1890508168 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db40055e0 con 0x55c8c4361810 2024-03-21T11:36:52.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.394+0000 7f5dc0a00640 1 --2- 172.21.15.47:0/3566496206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5da00a2fa0 0x7f5da00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.394+0000 7f5dc0a00640 1 -- 172.21.15.47:0/3566496206 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5dac0f18b0 con 0x7f5dbc108910 2024-03-21T11:36:52.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.395+0000 7f5dbaa00640 1 --2- 172.21.15.47:0/3566496206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5da00a2fa0 0x7f5da00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.395+0000 7f5dc4be8740 1 -- 172.21.15.47:0/1890508168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c4361810 msgr2=0x55c8c4361c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.395+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/1890508168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c4361810 0x55c8c4361c30 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5db400af20 tx=0x7f5db400e8e0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.395+0000 7f5dc4be8740 1 -- 172.21.15.47:0/1890508168 shutdown_connections 2024-03-21T11:36:52.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.395+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/1890508168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43633a0 0x55c8c4369c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.395+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/1890508168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43629e0 0x55c8c4362e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.395+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/1890508168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c4361810 0x55c8c4361c30 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.395+0000 7f5dc4be8740 1 -- 172.21.15.47:0/1890508168 >> 172.21.15.47:0/1890508168 conn(0x55c8c435cc00 msgr2=0x55c8c435d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:52.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.395+0000 7f5dc4be8740 1 -- 172.21.15.47:0/1890508168 shutdown_connections 2024-03-21T11:36:52.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.395+0000 7f5dc4be8740 1 -- 172.21.15.47:0/1890508168 wait complete. 2024-03-21T11:36:52.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.395+0000 7f5dbaa00640 1 --2- 172.21.15.47:0/3566496206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5da00a2fa0 0x7f5da00a5460 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5dac02c210 tx=0x7f5dac0f7530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:52.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.396+0000 7f5dc4be8740 1 -- start start 2024-03-21T11:36:52.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.396+0000 7f5dc4be8740 1 Processor -- start 2024-03-21T11:36:52.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.396+0000 7f5dc4be8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c4361810 0x55c8c436ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.396+0000 7f5dc4be8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43629e0 0x55c8c436bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.396+0000 7f5dc4be8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c43633a0 0x55c8c4372fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.396+0000 7f5dc4be8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c436a3b0 con 0x55c8c43629e0 2024-03-21T11:36:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.396+0000 7f5dc4be8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c42820d0 con 0x55c8c43633a0 2024-03-21T11:36:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.396+0000 7f5dc4be8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c436a530 con 0x55c8c4361810 2024-03-21T11:36:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.397+0000 7f5dc4be8740 10 client.0 _ll_register_callbacks cb 0x55c8c4286e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.397+0000 7f5dba000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43629e0 0x55c8c436bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.397+0000 7f5dbb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c4361810 0x55c8c436ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.397+0000 7f5dbb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c4361810 0x55c8c436ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38278/0 (socket says 172.21.15.47:38278) 2024-03-21T11:36:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.397+0000 7f5dbb400640 1 -- 172.21.15.47:0/3809416252 learned_addr learned my addr 172.21.15.47:0/3809416252 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.397+0000 7f5dbb400640 1 --2- 172.21.15.47:0/3809416252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c43633a0 0x55c8c4372fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.397+0000 7f5dbb400640 1 -- 172.21.15.47:0/3809416252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c43633a0 msgr2=0x55c8c4372fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.397+0000 7f5dbb400640 1 --2- 172.21.15.47:0/3809416252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c43633a0 0x55c8c4372fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.397+0000 7f5dbb400640 1 -- 172.21.15.47:0/3809416252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43629e0 msgr2=0x55c8c436bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.397+0000 7f5dbb400640 1 --2- 172.21.15.47:0/3809416252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43629e0 0x55c8c436bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.397+0000 7f5dbb400640 1 -- 172.21.15.47:0/3809416252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8c3fda3f0 con 0x55c8c4361810 2024-03-21T11:36:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.397+0000 7f5dba000640 1 --2- 172.21.15.47:0/3809416252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43629e0 0x55c8c436bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.398+0000 7f5dbb400640 1 --2- 172.21.15.47:0/3809416252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c4361810 0x55c8c436ba30 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5db400a8f0 tx=0x7f5db400dfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe2f1bf714f192e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.398+0000 7f5da8c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.398+0000 7f5da8c00640 1 -- 172.21.15.47:0/3809416252 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4005630 con 0x55c8c4361810 2024-03-21T11:36:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.398+0000 7f5da8c00640 1 -- 172.21.15.47:0/3809416252 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5db4003970 con 0x55c8c4361810 2024-03-21T11:36:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.398+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3809416252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5db4004cc0 con 0x55c8c4361810 2024-03-21T11:36:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.398+0000 7f5da8c00640 1 -- 172.21.15.47:0/3809416252 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5db4003b70 con 0x55c8c4361810 2024-03-21T11:36:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.398+0000 7f5dc4be8740 10 client.30103 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.398+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3809416252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c8c40a47d0 con 0x55c8c4361810 2024-03-21T11:36:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.399+0000 7f5da8c00640 1 -- 172.21.15.47:0/3809416252 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5db4004cc0 con 0x55c8c4361810 2024-03-21T11:36:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.399+0000 7f5da8c00640 1 -- 172.21.15.47:0/3809416252 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5db4023040 con 0x55c8c4361810 2024-03-21T11:36:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.399+0000 7f5dc4be8740 10 client.30103 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.399+0000 7f5dc4be8740 10 client.30103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:52.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.399+0000 7f5dc4be8740 20 client.30103 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.399+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/3809416252 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8c4377270 0x55c8c4379660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.399+0000 7f5dc4be8740 4 client.30103 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.399+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3809416252 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8c42d3800 con 0x55c8c4377270 2024-03-21T11:36:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.399+0000 7f5dba000640 1 --2- 172.21.15.47:0/3809416252 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8c4377270 0x55c8c4379660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.400+0000 7f5dba000640 1 --2- 172.21.15.47:0/3809416252 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8c4377270 0x55c8c4379660 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20993407d1ed5e3 server_cookie=2a8b0b5272fd2c49 in_seq=0 out_seq=0 2024-03-21T11:36:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.400+0000 7f5da8c00640 10 client.30103 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.401+0000 7f5da8c00640 1 -- 172.21.15.47:0/3809416252 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c8c42d3800 con 0x55c8c4377270 2024-03-21T11:36:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.401+0000 7f5da8c00640 10 client.30103 handle_command_reply: tid=0 2024-03-21T11:36:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.401+0000 7f5dc4be8740 1 client.30103 shutdown 2024-03-21T11:36:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.401+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3809416252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c4361810 msgr2=0x55c8c436ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.401+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/3809416252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c4361810 0x55c8c436ba30 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5db400a8f0 tx=0x7f5db400dfa0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.401+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3809416252 shutdown_connections 2024-03-21T11:36:52.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.401+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/3809416252 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8c4377270 0x55c8c4379660 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.401+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/3809416252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c43633a0 0x55c8c4372fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.401+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/3809416252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43629e0 0x55c8c436bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.401+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/3809416252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c4361810 0x55c8c436ba30 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.401+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3809416252 >> 172.21.15.47:0/3809416252 conn(0x55c8c435cc00 msgr2=0x55c8c435beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:52.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.401+0000 7f5da8c00640 0 client.30103 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:52.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.402+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3809416252 shutdown_connections 2024-03-21T11:36:52.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.402+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3809416252 wait complete. 2024-03-21T11:36:52.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.402+0000 7f5dc4be8740 20 client.30103 trim_cache size 0 max 16384 2024-03-21T11:36:52.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.419+0000 7f5dc4be8740 1 Processor -- start 2024-03-21T11:36:52.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.419+0000 7f5dc4be8740 1 -- start start 2024-03-21T11:36:52.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.419+0000 7f5dc4be8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c4361810 0x55c8c436bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.419+0000 7f5dc4be8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c43629e0 0x55c8c4372f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.419+0000 7f5dc4be8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43633a0 0x55c8c4360570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.419+0000 7f5dc4be8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc1b4b60 con 0x55c8c4361810 2024-03-21T11:36:52.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.419+0000 7f5dc4be8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc1b64e0 con 0x55c8c43629e0 2024-03-21T11:36:52.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.419+0000 7f5dc4be8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc1b51c0 con 0x55c8c43633a0 2024-03-21T11:36:52.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.420+0000 7f5dba000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c43629e0 0x55c8c4372f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.420+0000 7f5dbb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c4361810 0x55c8c436bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.420+0000 7f5dba000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c43629e0 0x55c8c4372f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60178/0 (socket says 172.21.15.47:60178) 2024-03-21T11:36:52.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.420+0000 7f5dba000640 1 -- 172.21.15.47:0/286422905 learned_addr learned my addr 172.21.15.47:0/286422905 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:52.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.420+0000 7f5dbb400640 1 --2- 172.21.15.47:0/286422905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43633a0 0x55c8c4360570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.420+0000 7f5dba000640 1 -- 172.21.15.47:0/286422905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43633a0 msgr2=0x55c8c4360570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.420+0000 7f5dba000640 1 --2- 172.21.15.47:0/286422905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43633a0 0x55c8c4360570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.420+0000 7f5dba000640 1 -- 172.21.15.47:0/286422905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c4361810 msgr2=0x55c8c436bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.420+0000 7f5dba000640 1 --2- 172.21.15.47:0/286422905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c4361810 0x55c8c436bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.420+0000 7f5dba000640 1 -- 172.21.15.47:0/286422905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8c4360ab0 con 0x55c8c43629e0 2024-03-21T11:36:52.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.420+0000 7f5dbb400640 1 --2- 172.21.15.47:0/286422905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43633a0 0x55c8c4360570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:52.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.420+0000 7f5dba000640 1 --2- 172.21.15.47:0/286422905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c43629e0 0x55c8c4372f40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5da400b740 tx=0x7f5da400b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec3477b0a06daef1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:52.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.421+0000 7f5dab400640 1 -- 172.21.15.47:0/286422905 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4026070 con 0x55c8c43629e0 2024-03-21T11:36:52.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.421+0000 7f5dab400640 1 -- 172.21.15.47:0/286422905 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5da4020c90 con 0x55c8c43629e0 2024-03-21T11:36:52.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.421+0000 7f5dab400640 1 -- 172.21.15.47:0/286422905 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4011860 con 0x55c8c43629e0 2024-03-21T11:36:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.422+0000 7f5dc4be8740 1 -- 172.21.15.47:0/286422905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c43629e0 msgr2=0x55c8c4372f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.422+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/286422905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c43629e0 0x55c8c4372f40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5da400b740 tx=0x7f5da400b390 comp rx=0 tx=0).stop 2024-03-21T11:36:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.422+0000 7f5dc4be8740 1 -- 172.21.15.47:0/286422905 shutdown_connections 2024-03-21T11:36:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.422+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/286422905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43633a0 0x55c8c4360570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.422+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/286422905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c43629e0 0x55c8c4372f40 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.422+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/286422905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c4361810 0x55c8c436bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.422+0000 7f5dc4be8740 1 -- 172.21.15.47:0/286422905 >> 172.21.15.47:0/286422905 conn(0x55c8c435cc00 msgr2=0x55c8c435fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.422+0000 7f5dc4be8740 1 -- 172.21.15.47:0/286422905 shutdown_connections 2024-03-21T11:36:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.422+0000 7f5dc4be8740 1 -- 172.21.15.47:0/286422905 wait complete. 2024-03-21T11:36:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.422+0000 7f5dc4be8740 1 -- start start 2024-03-21T11:36:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.423+0000 7f5dc4be8740 1 Processor -- start 2024-03-21T11:36:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.423+0000 7f5dc4be8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c4361810 0x55c8c4370d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.423+0000 7f5dc4be8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43629e0 0x55c8c43712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.423+0000 7f5dc4be8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43633a0 0x55c8c4372220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.423+0000 7f5dc4be8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc1b5e80 con 0x55c8c43633a0 2024-03-21T11:36:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.423+0000 7f5dc4be8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc1b5820 con 0x55c8c4361810 2024-03-21T11:36:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.423+0000 7f5dc4be8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc1b2d20 con 0x55c8c43629e0 2024-03-21T11:36:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.423+0000 7f5dc4be8740 10 client.0 _ll_register_callbacks cb 0x7f5da400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.423+0000 7f5dbb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c4361810 0x55c8c4370d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.423+0000 7f5dba000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43629e0 0x55c8c43712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.423+0000 7f5dbb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c4361810 0x55c8c4370d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60194/0 (socket says 172.21.15.47:60194) 2024-03-21T11:36:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.423+0000 7f5dba000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43629e0 0x55c8c43712b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38288/0 (socket says 172.21.15.47:38288) 2024-03-21T11:36:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.423+0000 7f5dbb400640 1 -- 172.21.15.47:0/3518728805 learned_addr learned my addr 172.21.15.47:0/3518728805 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.424+0000 7f5dbb400640 1 --2- 172.21.15.47:0/3518728805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43633a0 0x55c8c4372220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.424+0000 7f5dba000640 1 -- 172.21.15.47:0/3518728805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c4361810 msgr2=0x55c8c4370d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.424+0000 7f5dba000640 1 --2- 172.21.15.47:0/3518728805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c4361810 0x55c8c4370d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.424+0000 7f5dba000640 1 -- 172.21.15.47:0/3518728805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43633a0 msgr2=0x55c8c4372220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:52.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.424+0000 7f5dba000640 1 --2- 172.21.15.47:0/3518728805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43633a0 0x55c8c4372220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.424+0000 7f5dba000640 1 -- 172.21.15.47:0/3518728805 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8c4372810 con 0x55c8c43629e0 2024-03-21T11:36:52.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.424+0000 7f5dba000640 1 --2- 172.21.15.47:0/3518728805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43629e0 0x55c8c43712b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x55c8c4375f60 tx=0x7f5da40041d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6c15517a272e81 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:52.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.424+0000 7f5da8c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:52.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.424+0000 7f5da8c00640 1 -- 172.21.15.47:0/3518728805 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4026040 con 0x55c8c43629e0 2024-03-21T11:36:52.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.424+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3518728805 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5da4020c90 con 0x55c8c43629e0 2024-03-21T11:36:52.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.424+0000 7f5da8c00640 1 -- 172.21.15.47:0/3518728805 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5da4005020 con 0x55c8c43629e0 2024-03-21T11:36:52.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.425+0000 7f5da8c00640 1 -- 172.21.15.47:0/3518728805 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5da4005220 con 0x55c8c43629e0 2024-03-21T11:36:52.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.425+0000 7f5dc4be8740 10 client.30109 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:52.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.425+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3518728805 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c8c43d31e0 con 0x55c8c43629e0 2024-03-21T11:36:52.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.425+0000 7f5da8c00640 1 -- 172.21.15.47:0/3518728805 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5da4020c90 con 0x55c8c43629e0 2024-03-21T11:36:52.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.425+0000 7f5da8c00640 1 -- 172.21.15.47:0/3518728805 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5da4004990 con 0x55c8c43629e0 2024-03-21T11:36:52.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.425+0000 7f5dc4be8740 10 client.30109 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:52.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.425+0000 7f5dc4be8740 10 client.30109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:52.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.425+0000 7f5dc4be8740 20 client.30109 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:52.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.425+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/3518728805 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8c43d3ab0 0x55c8c43d5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:52.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.425+0000 7f5dc4be8740 4 client.30109 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:36:52.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.425+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3518728805 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c8c42cdb50 con 0x55c8c43d3ab0 2024-03-21T11:36:52.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.426+0000 7f5dbb400640 1 --2- 172.21.15.47:0/3518728805 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8c43d3ab0 0x55c8c43d5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:52.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.426+0000 7f5dbb400640 1 --2- 172.21.15.47:0/3518728805 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8c43d3ab0 0x55c8c43d5ed0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6849584cad645ed4 server_cookie=44e88836e8ec4ef in_seq=0 out_seq=0 2024-03-21T11:36:52.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.426+0000 7f5da8c00640 10 client.30109 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:52.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.427+0000 7f5da8c00640 1 -- 172.21.15.47:0/3518728805 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c8c42cdb50 con 0x55c8c43d3ab0 2024-03-21T11:36:52.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.427+0000 7f5da8c00640 10 client.30109 handle_command_reply: tid=0 2024-03-21T11:36:52.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.427+0000 7f5dc4be8740 1 client.30109 shutdown 2024-03-21T11:36:52.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.427+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3518728805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43629e0 msgr2=0x55c8c43712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.427+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/3518728805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43629e0 0x55c8c43712b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x55c8c4375f60 tx=0x7f5da40041d0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.428+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3518728805 shutdown_connections 2024-03-21T11:36:52.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.428+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/3518728805 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8c43d3ab0 0x55c8c43d5ed0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.428+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/3518728805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8c43633a0 0x55c8c4372220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.428+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/3518728805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8c43629e0 0x55c8c43712b0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.428+0000 7f5dc4be8740 1 --2- 172.21.15.47:0/3518728805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8c4361810 0x55c8c4370d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.428+0000 7f5da8c00640 0 client.30109 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:52.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.428+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3518728805 >> 172.21.15.47:0/3518728805 conn(0x55c8c435cc00 msgr2=0x55c8c435b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:52.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.428+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3518728805 shutdown_connections 2024-03-21T11:36:52.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.428+0000 7f5dc4be8740 1 -- 172.21.15.47:0/3518728805 wait complete. 2024-03-21T11:36:52.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.428+0000 7f5dc4be8740 20 client.30109 trim_cache size 0 max 16384 2024-03-21T11:36:52.436 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:36:52.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.429+0000 7f5dc1e00640 1 -- 172.21.15.47:0/3566496206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5da00a2fa0 msgr2=0x7f5da00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.429+0000 7f5dc1e00640 1 --2- 172.21.15.47:0/3566496206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5da00a2fa0 0x7f5da00a5460 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5dac02c210 tx=0x7f5dac0f7530 comp rx=0 tx=0).stop 2024-03-21T11:36:52.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.429+0000 7f5dc1e00640 1 -- 172.21.15.47:0/3566496206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dbc108910 msgr2=0x7f5dbc1a2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:52.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.429+0000 7f5dc1e00640 1 --2- 172.21.15.47:0/3566496206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dbc108910 0x7f5dbc1a2420 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5dac02c240 tx=0x7f5dac02c6c0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.429+0000 7f5dc1e00640 1 -- 172.21.15.47:0/3566496206 shutdown_connections 2024-03-21T11:36:52.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.429+0000 7f5dc1e00640 1 --2- 172.21.15.47:0/3566496206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5da00a2fa0 0x7f5da00a5460 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.429+0000 7f5dc1e00640 1 --2- 172.21.15.47:0/3566496206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dbc10dd50 0x7f5dbc1a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.429+0000 7f5dc1e00640 1 --2- 172.21.15.47:0/3566496206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dbc10b2d0 0x7f5dbc1a2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.429+0000 7f5dc1e00640 1 --2- 172.21.15.47:0/3566496206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dbc108910 0x7f5dbc1a2420 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:52.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.429+0000 7f5dc1e00640 1 -- 172.21.15.47:0/3566496206 >> 172.21.15.47:0/3566496206 conn(0x7f5dbc06e8a0 msgr2=0x7f5dbc1091b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:52.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.430+0000 7f5dc1e00640 1 -- 172.21.15.47:0/3566496206 shutdown_connections 2024-03-21T11:36:52.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:52.430+0000 7f5dc1e00640 1 -- 172.21.15.47:0/3566496206 wait complete. 2024-03-21T11:36:52.583 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1035672 7.92 MB/sec execute 576 sec latency 32.362 ms 2024-03-21T11:36:52.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:52 smithi047 ceph-mon[31351]: pgmap v806: 129 pgs: 129 active+clean; 38 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-03-21T11:36:52.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:52 smithi100 ceph-mon[36245]: pgmap v806: 129 pgs: 129 active+clean; 38 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-03-21T11:36:52.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:52 smithi102 ceph-mon[36748]: pgmap v806: 129 pgs: 129 active+clean; 38 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-03-21T11:36:53.448 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:36:53.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:36:53.448 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:36:53.582 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1037440 7.93 MB/sec execute 577 sec latency 20.814 ms 2024-03-21T11:36:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.701+0000 7f851f200640 1 -- 172.21.15.47:0/3956388891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85201049b0 msgr2=0x7f8520104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.701+0000 7f851f200640 1 --2- 172.21.15.47:0/3956388891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85201049b0 0x7f8520104e30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f850c009900 tx=0x7f850c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:36:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.702+0000 7f851f200640 1 -- 172.21.15.47:0/3956388891 shutdown_connections 2024-03-21T11:36:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.702+0000 7f851f200640 1 --2- 172.21.15.47:0/3956388891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8520105370 0x7f852010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.702+0000 7f851f200640 1 --2- 172.21.15.47:0/3956388891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85201049b0 0x7f8520104e30 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.702+0000 7f851f200640 1 --2- 172.21.15.47:0/3956388891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85201037c0 0x7f8520103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.702+0000 7f851f200640 1 -- 172.21.15.47:0/3956388891 >> 172.21.15.47:0/3956388891 conn(0x7f85200fed50 msgr2=0x7f8520101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.702+0000 7f851f200640 1 -- 172.21.15.47:0/3956388891 shutdown_connections 2024-03-21T11:36:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.702+0000 7f851f200640 1 -- 172.21.15.47:0/3956388891 wait complete. 2024-03-21T11:36:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.703+0000 7f851f200640 1 Processor -- start 2024-03-21T11:36:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.703+0000 7f851f200640 1 -- start start 2024-03-21T11:36:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.703+0000 7f851f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85201037c0 0x7f8520199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.703+0000 7f851f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85201049b0 0x7f852019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.703+0000 7f851f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8520105370 0x7f85201a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.703+0000 7f851f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852010f420 con 0x7f8520105370 2024-03-21T11:36:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.703+0000 7f851f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852010f2a0 con 0x7f85201037c0 2024-03-21T11:36:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.703+0000 7f851f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852010f5a0 con 0x7f85201049b0 2024-03-21T11:36:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.704+0000 7f851de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85201037c0 0x7f8520199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.704+0000 7f851de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85201037c0 0x7f8520199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60218/0 (socket says 172.21.15.47:60218) 2024-03-21T11:36:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.704+0000 7f851de00640 1 -- 172.21.15.47:0/1434653557 learned_addr learned my addr 172.21.15.47:0/1434653557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.704+0000 7f851d400640 1 --2- 172.21.15.47:0/1434653557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85201049b0 0x7f852019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.704+0000 7f851e800640 1 --2- 172.21.15.47:0/1434653557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8520105370 0x7f85201a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.704+0000 7f851de00640 1 -- 172.21.15.47:0/1434653557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85201049b0 msgr2=0x7f852019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.704+0000 7f851de00640 1 --2- 172.21.15.47:0/1434653557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85201049b0 0x7f852019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.704+0000 7f851de00640 1 -- 172.21.15.47:0/1434653557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8520105370 msgr2=0x7f85201a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.704+0000 7f851de00640 1 --2- 172.21.15.47:0/1434653557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8520105370 0x7f85201a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.704+0000 7f851de00640 1 -- 172.21.15.47:0/1434653557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85201a16f0 con 0x7f85201037c0 2024-03-21T11:36:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.704+0000 7f851de00640 1 --2- 172.21.15.47:0/1434653557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85201037c0 0x7f8520199ad0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f851000e9c0 tx=0x7f851000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.705+0000 7f8516a00640 1 -- 172.21.15.47:0/1434653557 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851000cd80 con 0x7f85201037c0 2024-03-21T11:36:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.705+0000 7f8516a00640 1 -- 172.21.15.47:0/1434653557 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85100035f0 con 0x7f85201037c0 2024-03-21T11:36:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.705+0000 7f8516a00640 1 -- 172.21.15.47:0/1434653557 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510010690 con 0x7f85201037c0 2024-03-21T11:36:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.705+0000 7f851f200640 1 -- 172.21.15.47:0/1434653557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85201a19e0 con 0x7f85201037c0 2024-03-21T11:36:53.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.705+0000 7f851f200640 1 -- 172.21.15.47:0/1434653557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85201a1f50 con 0x7f85201037c0 2024-03-21T11:36:53.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.708+0000 7f8516a00640 1 -- 172.21.15.47:0/1434653557 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f85100037a0 con 0x7f85201037c0 2024-03-21T11:36:53.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.708+0000 7f852778a740 1 Processor -- start 2024-03-21T11:36:53.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.709+0000 7f852778a740 1 -- start start 2024-03-21T11:36:53.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.709+0000 7f852778a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a7542b0 0x55a73a7546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.709+0000 7f852778a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a754ca0 0x55a73a7597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.709+0000 7f852778a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a75c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.709+0000 7f852778a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73a6b9eb0 con 0x55a73a7542b0 2024-03-21T11:36:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.709+0000 7f852778a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73a662a60 con 0x55a73a759e10 2024-03-21T11:36:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.709+0000 7f852778a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73a6b5f50 con 0x55a73a754ca0 2024-03-21T11:36:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.709+0000 7f851d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a75c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.709+0000 7f851d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a75c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60226/0 (socket says 172.21.15.47:60226) 2024-03-21T11:36:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.709+0000 7f851d400640 1 -- 172.21.15.47:0/1403026514 learned_addr learned my addr 172.21.15.47:0/1403026514 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.709+0000 7f851de00640 1 --2- 172.21.15.47:0/1403026514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a754ca0 0x55a73a7597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.709+0000 7f851e800640 1 --2- 172.21.15.47:0/1403026514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a7542b0 0x55a73a7546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.710+0000 7f851d400640 1 -- 172.21.15.47:0/1403026514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a754ca0 msgr2=0x55a73a7597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.710+0000 7f851d400640 1 --2- 172.21.15.47:0/1403026514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a754ca0 0x55a73a7597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.710+0000 7f851d400640 1 -- 172.21.15.47:0/1403026514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a7542b0 msgr2=0x55a73a7546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.710+0000 7f851d400640 1 --2- 172.21.15.47:0/1403026514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a7542b0 0x55a73a7546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.710+0000 7f851d400640 1 -- 172.21.15.47:0/1403026514 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a73a4967e0 con 0x55a73a759e10 2024-03-21T11:36:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.710+0000 7f851e800640 1 --2- 172.21.15.47:0/1403026514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a7542b0 0x55a73a7546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.710+0000 7f851d400640 1 --2- 172.21.15.47:0/1403026514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a75c2a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f850c009900 tx=0x7f850c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=255ce67356c81fdd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.710+0000 7f8516a00640 1 --2- 172.21.15.47:0/1434653557 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f85040a2fa0 0x7f85040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.710+0000 7f850b000640 1 -- 172.21.15.47:0/1403026514 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c02cb80 con 0x55a73a759e10 2024-03-21T11:36:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.710+0000 7f8516a00640 1 -- 172.21.15.47:0/1434653557 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f85100cfb40 con 0x7f85201037c0 2024-03-21T11:36:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.710+0000 7f850b000640 1 -- 172.21.15.47:0/1403026514 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f850c005040 con 0x55a73a759e10 2024-03-21T11:36:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.710+0000 7f850b000640 1 -- 172.21.15.47:0/1403026514 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c004900 con 0x55a73a759e10 2024-03-21T11:36:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.710+0000 7f851e800640 1 --2- 172.21.15.47:0/1434653557 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f85040a2fa0 0x7f85040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.711+0000 7f852778a740 1 -- 172.21.15.47:0/1403026514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 msgr2=0x55a73a75c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.711+0000 7f852778a740 1 --2- 172.21.15.47:0/1403026514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a75c2a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f850c009900 tx=0x7f850c009560 comp rx=0 tx=0).stop 2024-03-21T11:36:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.711+0000 7f851e800640 1 --2- 172.21.15.47:0/1434653557 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f85040a2fa0 0x7f85040a5460 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x55a73a755500 tx=0x7f8518009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.711+0000 7f852778a740 1 -- 172.21.15.47:0/1403026514 shutdown_connections 2024-03-21T11:36:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.711+0000 7f852778a740 1 --2- 172.21.15.47:0/1403026514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a75c2a0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.711+0000 7f852778a740 1 --2- 172.21.15.47:0/1403026514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a754ca0 0x55a73a7597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.711+0000 7f852778a740 1 --2- 172.21.15.47:0/1403026514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a7542b0 0x55a73a7546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.711+0000 7f852778a740 1 -- 172.21.15.47:0/1403026514 >> 172.21.15.47:0/1403026514 conn(0x55a73a74f820 msgr2=0x55a73a74fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.711+0000 7f852778a740 1 -- 172.21.15.47:0/1403026514 shutdown_connections 2024-03-21T11:36:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.712+0000 7f852778a740 1 -- 172.21.15.47:0/1403026514 wait complete. 2024-03-21T11:36:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.712+0000 7f852778a740 1 -- start start 2024-03-21T11:36:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.712+0000 7f852778a740 1 Processor -- start 2024-03-21T11:36:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f852778a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a7542b0 0x55a73a75e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f852778a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a754ca0 0x55a73a75e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f852778a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a759e10 0x55a73a7659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f852778a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73a75cd60 con 0x55a73a759e10 2024-03-21T11:36:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f852778a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73a675eb0 con 0x55a73a754ca0 2024-03-21T11:36:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f852778a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73a75cee0 con 0x55a73a7542b0 2024-03-21T11:36:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f852778a740 10 client.0 _ll_register_callbacks cb 0x55a73a66a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f851de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a7542b0 0x55a73a75e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f851d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a754ca0 0x55a73a75e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f851de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a7542b0 0x55a73a75e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38324/0 (socket says 172.21.15.47:38324) 2024-03-21T11:36:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f851de00640 1 -- 172.21.15.47:0/4015594797 learned_addr learned my addr 172.21.15.47:0/4015594797 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f851d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a754ca0 0x55a73a75e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60242/0 (socket says 172.21.15.47:60242) 2024-03-21T11:36:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f851e800640 1 --2- 172.21.15.47:0/4015594797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a759e10 0x55a73a7659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f851de00640 1 -- 172.21.15.47:0/4015594797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a754ca0 msgr2=0x55a73a75e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f851de00640 1 --2- 172.21.15.47:0/4015594797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a754ca0 0x55a73a75e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f851de00640 1 -- 172.21.15.47:0/4015594797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a759e10 msgr2=0x55a73a7659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f851de00640 1 --2- 172.21.15.47:0/4015594797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a759e10 0x55a73a7659c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.713+0000 7f851de00640 1 -- 172.21.15.47:0/4015594797 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a73a4f6f70 con 0x55a73a7542b0 2024-03-21T11:36:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.714+0000 7f851e800640 1 --2- 172.21.15.47:0/4015594797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a759e10 0x55a73a7659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.714+0000 7f851de00640 1 --2- 172.21.15.47:0/4015594797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a7542b0 0x55a73a75e440 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f85100982e0 tx=0x7f851009bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ac820d086a687ab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.714+0000 7f84ff400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.714+0000 7f84ff400640 1 -- 172.21.15.47:0/4015594797 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85100bc0b0 con 0x55a73a7542b0 2024-03-21T11:36:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.714+0000 7f84ff400640 1 -- 172.21.15.47:0/4015594797 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f851009b2d0 con 0x55a73a7542b0 2024-03-21T11:36:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.714+0000 7f852778a740 1 -- 172.21.15.47:0/4015594797 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f850c005040 con 0x55a73a7542b0 2024-03-21T11:36:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.714+0000 7f84ff400640 1 -- 172.21.15.47:0/4015594797 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f851009c580 con 0x55a73a7542b0 2024-03-21T11:36:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.714+0000 7f852778a740 10 client.30121 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.714+0000 7f852778a740 1 -- 172.21.15.47:0/4015594797 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a73a38e170 con 0x55a73a7542b0 2024-03-21T11:36:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.715+0000 7f84ff400640 1 -- 172.21.15.47:0/4015594797 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f850c005040 con 0x55a73a7542b0 2024-03-21T11:36:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.715+0000 7f84ff400640 1 -- 172.21.15.47:0/4015594797 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f851009ad40 con 0x55a73a7542b0 2024-03-21T11:36:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.715+0000 7f852778a740 10 client.30121 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.715+0000 7f852778a740 10 client.30121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.715+0000 7f852778a740 20 client.30121 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.715+0000 7f852778a740 1 --2- 172.21.15.47:0/4015594797 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a73a76a320 0x55a73a76c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.715+0000 7f852778a740 4 client.30121 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.715+0000 7f852778a740 1 -- 172.21.15.47:0/4015594797 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a73a6c73f0 con 0x55a73a76a320 2024-03-21T11:36:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.715+0000 7f851d400640 1 --2- 172.21.15.47:0/4015594797 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a73a76a320 0x55a73a76c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.716+0000 7f851d400640 1 --2- 172.21.15.47:0/4015594797 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a73a76a320 0x55a73a76c790 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abb9253b6194f309 server_cookie=d38ba5eb9c48e20e in_seq=0 out_seq=0 2024-03-21T11:36:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.716+0000 7f84ff400640 10 client.30121 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.718+0000 7f84ff400640 1 -- 172.21.15.47:0/4015594797 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a73a6c73f0 con 0x55a73a76a320 2024-03-21T11:36:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.718+0000 7f84ff400640 10 client.30121 handle_command_reply: tid=0 2024-03-21T11:36:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.718+0000 7f852778a740 1 client.30121 shutdown 2024-03-21T11:36:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.718+0000 7f852778a740 1 -- 172.21.15.47:0/4015594797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a7542b0 msgr2=0x55a73a75e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.718+0000 7f852778a740 1 --2- 172.21.15.47:0/4015594797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a7542b0 0x55a73a75e440 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f85100982e0 tx=0x7f851009bc50 comp rx=0 tx=0).stop 2024-03-21T11:36:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.719+0000 7f852778a740 1 -- 172.21.15.47:0/4015594797 shutdown_connections 2024-03-21T11:36:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.719+0000 7f852778a740 1 --2- 172.21.15.47:0/4015594797 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a73a76a320 0x55a73a76c790 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.719+0000 7f852778a740 1 --2- 172.21.15.47:0/4015594797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a759e10 0x55a73a7659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.719+0000 7f852778a740 1 --2- 172.21.15.47:0/4015594797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a754ca0 0x55a73a75e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.719+0000 7f852778a740 1 --2- 172.21.15.47:0/4015594797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a7542b0 0x55a73a75e440 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.719+0000 7f852778a740 1 -- 172.21.15.47:0/4015594797 >> 172.21.15.47:0/4015594797 conn(0x55a73a74f820 msgr2=0x55a73a74ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:53.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.719+0000 7f84ff400640 0 client.30121 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:53.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.719+0000 7f852778a740 1 -- 172.21.15.47:0/4015594797 shutdown_connections 2024-03-21T11:36:53.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.719+0000 7f852778a740 1 -- 172.21.15.47:0/4015594797 wait complete. 2024-03-21T11:36:53.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.719+0000 7f852778a740 20 client.30121 trim_cache size 0 max 16384 2024-03-21T11:36:53.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.738+0000 7f852778a740 1 Processor -- start 2024-03-21T11:36:53.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.738+0000 7f852778a740 1 -- start start 2024-03-21T11:36:53.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f852778a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a7542b0 0x55a73a74e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f852778a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a754ca0 0x55a73a765980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f852778a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a771020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f852778a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85201a38d0 con 0x55a73a7542b0 2024-03-21T11:36:53.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f852778a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85201a5250 con 0x55a73a759e10 2024-03-21T11:36:53.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f852778a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85201a3f30 con 0x55a73a754ca0 2024-03-21T11:36:53.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f851e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a771020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f851d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a754ca0 0x55a73a765980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f851d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a754ca0 0x55a73a765980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38336/0 (socket says 172.21.15.47:38336) 2024-03-21T11:36:53.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f851d400640 1 -- 172.21.15.47:0/3077991585 learned_addr learned my addr 172.21.15.47:0/3077991585 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:53.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f851e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a771020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60250/0 (socket says 172.21.15.47:60250) 2024-03-21T11:36:53.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f851de00640 1 --2- 172.21.15.47:0/3077991585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a7542b0 0x55a73a74e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f851d400640 1 -- 172.21.15.47:0/3077991585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 msgr2=0x55a73a771020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f851d400640 1 --2- 172.21.15.47:0/3077991585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a771020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f851d400640 1 -- 172.21.15.47:0/3077991585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a7542b0 msgr2=0x55a73a74e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f851d400640 1 --2- 172.21.15.47:0/3077991585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a7542b0 0x55a73a74e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.739+0000 7f851d400640 1 -- 172.21.15.47:0/3077991585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a73a750590 con 0x55a73a754ca0 2024-03-21T11:36:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.740+0000 7f851d400640 1 --2- 172.21.15.47:0/3077991585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a754ca0 0x55a73a765980 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f850c044000 tx=0x7f850c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c973deed069516a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.740+0000 7f850a600640 1 -- 172.21.15.47:0/3077991585 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c005d50 con 0x55a73a754ca0 2024-03-21T11:36:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.740+0000 7f850a600640 1 -- 172.21.15.47:0/3077991585 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f850c0510c0 con 0x55a73a754ca0 2024-03-21T11:36:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.740+0000 7f850a600640 1 -- 172.21.15.47:0/3077991585 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c0046c0 con 0x55a73a754ca0 2024-03-21T11:36:53.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.741+0000 7f852778a740 1 -- 172.21.15.47:0/3077991585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a754ca0 msgr2=0x55a73a765980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:53.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.741+0000 7f852778a740 1 --2- 172.21.15.47:0/3077991585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a754ca0 0x55a73a765980 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f850c044000 tx=0x7f850c000bc0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.741+0000 7f852778a740 1 -- 172.21.15.47:0/3077991585 shutdown_connections 2024-03-21T11:36:53.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.741+0000 7f852778a740 1 --2- 172.21.15.47:0/3077991585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a771020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.741+0000 7f852778a740 1 --2- 172.21.15.47:0/3077991585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a754ca0 0x55a73a765980 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.741+0000 7f852778a740 1 --2- 172.21.15.47:0/3077991585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a7542b0 0x55a73a74e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.741+0000 7f852778a740 1 -- 172.21.15.47:0/3077991585 >> 172.21.15.47:0/3077991585 conn(0x55a73a74f820 msgr2=0x55a73a74e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.741+0000 7f852778a740 1 -- 172.21.15.47:0/3077991585 shutdown_connections 2024-03-21T11:36:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.741+0000 7f852778a740 1 -- 172.21.15.47:0/3077991585 wait complete. 2024-03-21T11:36:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.741+0000 7f852778a740 1 -- start start 2024-03-21T11:36:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.742+0000 7f852778a740 1 Processor -- start 2024-03-21T11:36:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.742+0000 7f852778a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a7542b0 0x55a73a7642d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.742+0000 7f852778a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a754ca0 0x55a73a764810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.742+0000 7f852778a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a764d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.742+0000 7f852778a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85201a4bf0 con 0x55a73a754ca0 2024-03-21T11:36:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.742+0000 7f852778a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85201a4590 con 0x55a73a759e10 2024-03-21T11:36:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.742+0000 7f852778a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8520073180 con 0x55a73a7542b0 2024-03-21T11:36:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.742+0000 7f852778a740 10 client.0 _ll_register_callbacks cb 0x7f85100d3270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.742+0000 7f851de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a7542b0 0x55a73a7642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.742+0000 7f851e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a764d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.742+0000 7f851de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a7542b0 0x55a73a7642d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38346/0 (socket says 172.21.15.47:38346) 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.742+0000 7f851de00640 1 -- 172.21.15.47:0/2593818482 learned_addr learned my addr 172.21.15.47:0/2593818482 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.742+0000 7f851d400640 1 --2- 172.21.15.47:0/2593818482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a754ca0 0x55a73a764810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.743+0000 7f851de00640 1 -- 172.21.15.47:0/2593818482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 msgr2=0x55a73a764d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.743+0000 7f851de00640 1 --2- 172.21.15.47:0/2593818482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a764d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.743+0000 7f851de00640 1 -- 172.21.15.47:0/2593818482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a754ca0 msgr2=0x55a73a764810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.743+0000 7f851de00640 1 --2- 172.21.15.47:0/2593818482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a754ca0 0x55a73a764810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.743+0000 7f851de00640 1 -- 172.21.15.47:0/2593818482 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a73a7c5ca0 con 0x55a73a7542b0 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.743+0000 7f851d400640 1 --2- 172.21.15.47:0/2593818482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a754ca0 0x55a73a764810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.743+0000 7f851de00640 1 --2- 172.21.15.47:0/2593818482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a7542b0 0x55a73a7642d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f8510003ee0 tx=0x7f85100d3170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfe9a4b128d393eb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.743+0000 7f84ff400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.743+0000 7f84ff400640 1 -- 172.21.15.47:0/2593818482 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85100bc0b0 con 0x55a73a7542b0 2024-03-21T11:36:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.743+0000 7f84ff400640 1 -- 172.21.15.47:0/2593818482 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8510097bd0 con 0x55a73a7542b0 2024-03-21T11:36:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.743+0000 7f852778a740 1 -- 172.21.15.47:0/2593818482 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f850c0510c0 con 0x55a73a7542b0 2024-03-21T11:36:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.743+0000 7f84ff400640 1 -- 172.21.15.47:0/2593818482 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f851009be80 con 0x55a73a7542b0 2024-03-21T11:36:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.744+0000 7f852778a740 10 client.30127 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.744+0000 7f852778a740 1 -- 172.21.15.47:0/2593818482 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a73a7c7050 con 0x55a73a7542b0 2024-03-21T11:36:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.744+0000 7f84ff400640 1 -- 172.21.15.47:0/2593818482 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f850c0510c0 con 0x55a73a7542b0 2024-03-21T11:36:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.744+0000 7f84ff400640 1 -- 172.21.15.47:0/2593818482 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f85100b8030 con 0x55a73a7542b0 2024-03-21T11:36:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.744+0000 7f852778a740 10 client.30127 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.744+0000 7f852778a740 10 client.30127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.744+0000 7f852778a740 20 client.30127 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.744+0000 7f852778a740 1 --2- 172.21.15.47:0/2593818482 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a73a7c7790 0x55a73a7c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.744+0000 7f852778a740 4 client.30127 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:36:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.744+0000 7f852778a740 1 -- 172.21.15.47:0/2593818482 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a73a6c1740 con 0x55a73a7c7790 2024-03-21T11:36:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.744+0000 7f851d400640 1 --2- 172.21.15.47:0/2593818482 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a73a7c7790 0x55a73a7c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.745+0000 7f851d400640 1 --2- 172.21.15.47:0/2593818482 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a73a7c7790 0x55a73a7c9bb0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40dc027ed4e0bd31 server_cookie=93a10a9f14fed1ef in_seq=0 out_seq=0 2024-03-21T11:36:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.745+0000 7f84ff400640 10 client.30127 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.746+0000 7f84ff400640 1 -- 172.21.15.47:0/2593818482 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a73a6c1740 con 0x55a73a7c7790 2024-03-21T11:36:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.746+0000 7f84ff400640 10 client.30127 handle_command_reply: tid=0 2024-03-21T11:36:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.746+0000 7f852778a740 1 client.30127 shutdown 2024-03-21T11:36:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.746+0000 7f852778a740 1 -- 172.21.15.47:0/2593818482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a7542b0 msgr2=0x55a73a7642d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.746+0000 7f852778a740 1 --2- 172.21.15.47:0/2593818482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a7542b0 0x55a73a7642d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f8510003ee0 tx=0x7f85100d3170 comp rx=0 tx=0).stop 2024-03-21T11:36:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.746+0000 7f852778a740 1 -- 172.21.15.47:0/2593818482 shutdown_connections 2024-03-21T11:36:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.746+0000 7f852778a740 1 --2- 172.21.15.47:0/2593818482 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a73a7c7790 0x55a73a7c9bb0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.746+0000 7f852778a740 1 --2- 172.21.15.47:0/2593818482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a73a759e10 0x55a73a764d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.746+0000 7f852778a740 1 --2- 172.21.15.47:0/2593818482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a73a754ca0 0x55a73a764810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.746+0000 7f852778a740 1 --2- 172.21.15.47:0/2593818482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a73a7542b0 0x55a73a7642d0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.746+0000 7f84ff400640 0 client.30127 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.746+0000 7f852778a740 1 -- 172.21.15.47:0/2593818482 >> 172.21.15.47:0/2593818482 conn(0x55a73a74f820 msgr2=0x55a73a74ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.747+0000 7f852778a740 1 -- 172.21.15.47:0/2593818482 shutdown_connections 2024-03-21T11:36:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.747+0000 7f852778a740 1 -- 172.21.15.47:0/2593818482 wait complete. 2024-03-21T11:36:53.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.747+0000 7f852778a740 20 client.30127 trim_cache size 0 max 16384 2024-03-21T11:36:53.751 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:36:53.751 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:36:53.751 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "f6a3d77a-2ad1-4f3f-8acc-e540cb113340", 2024-03-21T11:36:53.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:36:53.751 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:36:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.747+0000 7f851f200640 1 -- 172.21.15.47:0/1434653557 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f85040a2fa0 msgr2=0x7f85040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.747+0000 7f851f200640 1 --2- 172.21.15.47:0/1434653557 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f85040a2fa0 0x7f85040a5460 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x55a73a755500 tx=0x7f8518009500 comp rx=0 tx=0).stop 2024-03-21T11:36:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.747+0000 7f851f200640 1 -- 172.21.15.47:0/1434653557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85201037c0 msgr2=0x7f8520199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.747+0000 7f851f200640 1 --2- 172.21.15.47:0/1434653557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85201037c0 0x7f8520199ad0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f851000e9c0 tx=0x7f851000ee80 comp rx=0 tx=0).stop 2024-03-21T11:36:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.747+0000 7f851f200640 1 -- 172.21.15.47:0/1434653557 shutdown_connections 2024-03-21T11:36:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.747+0000 7f851f200640 1 --2- 172.21.15.47:0/1434653557 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f85040a2fa0 0x7f85040a5460 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.747+0000 7f851f200640 1 --2- 172.21.15.47:0/1434653557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8520105370 0x7f85201a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.747+0000 7f851f200640 1 --2- 172.21.15.47:0/1434653557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85201049b0 0x7f852019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.747+0000 7f851f200640 1 --2- 172.21.15.47:0/1434653557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85201037c0 0x7f8520199ad0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.748+0000 7f851f200640 1 -- 172.21.15.47:0/1434653557 >> 172.21.15.47:0/1434653557 conn(0x7f85200fed50 msgr2=0x7f8520107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.748+0000 7f851f200640 1 -- 172.21.15.47:0/1434653557 shutdown_connections 2024-03-21T11:36:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.748+0000 7f851f200640 1 -- 172.21.15.47:0/1434653557 wait complete. 2024-03-21T11:36:53.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:36:53.767 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:36:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.988+0000 7f39e7800640 1 -- 172.21.15.47:0/1100145851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e01016b0 msgr2=0x7f39e0101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.988+0000 7f39e7800640 1 --2- 172.21.15.47:0/1100145851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e01016b0 0x7f39e0101ad0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009900 tx=0x7f39d002bc30 comp rx=0 tx=0).stop 2024-03-21T11:36:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.989+0000 7f39e7800640 1 -- 172.21.15.47:0/1100145851 shutdown_connections 2024-03-21T11:36:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.989+0000 7f39e7800640 1 --2- 172.21.15.47:0/1100145851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39e0103260 0x7f39e0109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.989+0000 7f39e7800640 1 --2- 172.21.15.47:0/1100145851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e01028a0 0x7f39e0102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.989+0000 7f39e7800640 1 --2- 172.21.15.47:0/1100145851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e01016b0 0x7f39e0101ad0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.989+0000 7f39e7800640 1 -- 172.21.15.47:0/1100145851 >> 172.21.15.47:0/1100145851 conn(0x7f39e00fcc40 msgr2=0x7f39e00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.989+0000 7f39e7800640 1 -- 172.21.15.47:0/1100145851 shutdown_connections 2024-03-21T11:36:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.989+0000 7f39e7800640 1 -- 172.21.15.47:0/1100145851 wait complete. 2024-03-21T11:36:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.990+0000 7f39e7800640 1 Processor -- start 2024-03-21T11:36:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.990+0000 7f39e7800640 1 -- start start 2024-03-21T11:36:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.990+0000 7f39e7800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e01016b0 0x7f39e01979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.990+0000 7f39e7800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e01028a0 0x7f39e0197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.990+0000 7f39e7800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39e0103260 0x7f39e019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.990+0000 7f39e7800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e010d380 con 0x7f39e01016b0 2024-03-21T11:36:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.990+0000 7f39e7800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e010d200 con 0x7f39e01028a0 2024-03-21T11:36:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.990+0000 7f39e7800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e010d500 con 0x7f39e0103260 2024-03-21T11:36:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.990+0000 7f39e4e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e01016b0 0x7f39e01979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.990+0000 7f39dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e01028a0 0x7f39e0197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.991+0000 7f39e4e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e01016b0 0x7f39e01979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44140/0 (socket says 172.21.15.47:44140) 2024-03-21T11:36:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.991+0000 7f39dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e01028a0 0x7f39e0197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60268/0 (socket says 172.21.15.47:60268) 2024-03-21T11:36:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.991+0000 7f39e4e00640 1 -- 172.21.15.47:0/125325611 learned_addr learned my addr 172.21.15.47:0/125325611 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.991+0000 7f39e5800640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39e0103260 0x7f39e019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.991+0000 7f39dfe00640 1 -- 172.21.15.47:0/125325611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39e0103260 msgr2=0x7f39e019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.991+0000 7f39dfe00640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39e0103260 0x7f39e019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.991+0000 7f39dfe00640 1 -- 172.21.15.47:0/125325611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e01016b0 msgr2=0x7f39e01979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.991+0000 7f39dfe00640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e01016b0 0x7f39e01979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.991+0000 7f39dfe00640 1 -- 172.21.15.47:0/125325611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e019f610 con 0x7f39e01028a0 2024-03-21T11:36:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.991+0000 7f39e4e00640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e01016b0 0x7f39e01979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.991+0000 7f39e5800640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39e0103260 0x7f39e019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.991+0000 7f39dfe00640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e01028a0 0x7f39e0197f30 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f39c8010e10 tx=0x7f39c800b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.992+0000 7f39dd600640 1 -- 172.21.15.47:0/125325611 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c801e070 con 0x7f39e01028a0 2024-03-21T11:36:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.992+0000 7f39dd600640 1 -- 172.21.15.47:0/125325611 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39c8017a00 con 0x7f39e01028a0 2024-03-21T11:36:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.992+0000 7f39e7800640 1 -- 172.21.15.47:0/125325611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e0073930 con 0x7f39e01028a0 2024-03-21T11:36:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.992+0000 7f39dd600640 1 -- 172.21.15.47:0/125325611 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8019a40 con 0x7f39e01028a0 2024-03-21T11:36:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.992+0000 7f39e7800640 1 -- 172.21.15.47:0/125325611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e0073ed0 con 0x7f39e01028a0 2024-03-21T11:36:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.995+0000 7f39dd600640 1 -- 172.21.15.47:0/125325611 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f39c8017bb0 con 0x7f39e01028a0 2024-03-21T11:36:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.996+0000 7f39ea445740 1 Processor -- start 2024-03-21T11:36:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.996+0000 7f39ea445740 1 -- start start 2024-03-21T11:36:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.996+0000 7f39ea445740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc881e80 0x5566cc8822a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.996+0000 7f39ea445740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc882870 0x5566cc887370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.996+0000 7f39ea445740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc8879e0 0x5566cc889e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.996+0000 7f39ea445740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cc7e7250 con 0x5566cc881e80 2024-03-21T11:36:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.996+0000 7f39ea445740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cc78fa30 con 0x5566cc8879e0 2024-03-21T11:36:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.996+0000 7f39ea445740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cc7e32f0 con 0x5566cc882870 2024-03-21T11:36:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.996+0000 7f39e4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc882870 0x5566cc887370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.996+0000 7f39dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc8879e0 0x5566cc889e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.996+0000 7f39e5800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc881e80 0x5566cc8822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.996+0000 7f39e4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc882870 0x5566cc887370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38364/0 (socket says 172.21.15.47:38364) 2024-03-21T11:36:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.996+0000 7f39e4e00640 1 -- 172.21.15.47:0/4169341886 learned_addr learned my addr 172.21.15.47:0/4169341886 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.997+0000 7f39e4e00640 1 -- 172.21.15.47:0/4169341886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc8879e0 msgr2=0x5566cc889e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.997+0000 7f39e4e00640 1 --2- 172.21.15.47:0/4169341886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc8879e0 0x5566cc889e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.997+0000 7f39e4e00640 1 -- 172.21.15.47:0/4169341886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc881e80 msgr2=0x5566cc8822a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.997+0000 7f39e4e00640 1 --2- 172.21.15.47:0/4169341886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc881e80 0x5566cc8822a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.997+0000 7f39e4e00640 1 -- 172.21.15.47:0/4169341886 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566cc77a690 con 0x5566cc882870 2024-03-21T11:36:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.997+0000 7f39dd600640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39c00a2fa0 0x7f39c00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.997+0000 7f39e4e00640 1 --2- 172.21.15.47:0/4169341886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc882870 0x5566cc887370 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f39d002c170 tx=0x7f39d0009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3900fed5fcb1e9a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.997+0000 7f39dd600640 1 -- 172.21.15.47:0/125325611 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f39c80d3380 con 0x7f39e01028a0 2024-03-21T11:36:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.997+0000 7f39e5800640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39c00a2fa0 0x7f39c00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.998+0000 7f39d6000640 1 -- 172.21.15.47:0/4169341886 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0045070 con 0x5566cc882870 2024-03-21T11:36:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.998+0000 7f39d6000640 1 -- 172.21.15.47:0/4169341886 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39d002cd30 con 0x5566cc882870 2024-03-21T11:36:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.998+0000 7f39d6000640 1 -- 172.21.15.47:0/4169341886 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0005200 con 0x5566cc882870 2024-03-21T11:36:54.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.998+0000 7f39ea445740 1 -- 172.21.15.47:0/4169341886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc882870 msgr2=0x5566cc887370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:54.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.998+0000 7f39e5800640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39c00a2fa0 0x7f39c00a5460 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x5566cc8830d0 tx=0x7f39d800b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:54.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.998+0000 7f39ea445740 1 --2- 172.21.15.47:0/4169341886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc882870 0x5566cc887370 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f39d002c170 tx=0x7f39d0009610 comp rx=0 tx=0).stop 2024-03-21T11:36:54.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.999+0000 7f39ea445740 1 -- 172.21.15.47:0/4169341886 shutdown_connections 2024-03-21T11:36:54.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.999+0000 7f39ea445740 1 --2- 172.21.15.47:0/4169341886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc8879e0 0x5566cc889e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.999+0000 7f39ea445740 1 --2- 172.21.15.47:0/4169341886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc882870 0x5566cc887370 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.999+0000 7f39ea445740 1 --2- 172.21.15.47:0/4169341886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc881e80 0x5566cc8822a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.999+0000 7f39ea445740 1 -- 172.21.15.47:0/4169341886 >> 172.21.15.47:0/4169341886 conn(0x5566cc84a200 msgr2=0x5566cc87d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.999+0000 7f39ea445740 1 -- 172.21.15.47:0/4169341886 shutdown_connections 2024-03-21T11:36:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.999+0000 7f39ea445740 1 -- 172.21.15.47:0/4169341886 wait complete. 2024-03-21T11:36:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.999+0000 7f39ea445740 1 -- start start 2024-03-21T11:36:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:53.999+0000 7f39ea445740 1 Processor -- start 2024-03-21T11:36:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.000+0000 7f39ea445740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc881e80 0x5566cc88be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.000+0000 7f39ea445740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc882870 0x5566cc88c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.000+0000 7f39ea445740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc8879e0 0x5566cc893380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.000+0000 7f39ea445740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cc88a930 con 0x5566cc881e80 2024-03-21T11:36:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.000+0000 7f39ea445740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cc7a2e80 con 0x5566cc882870 2024-03-21T11:36:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.000+0000 7f39ea445740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cc88aab0 con 0x5566cc8879e0 2024-03-21T11:36:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.000+0000 7f39ea445740 10 client.0 _ll_register_callbacks cb 0x5566cc7b5130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.000+0000 7f39dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc882870 0x5566cc88c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.000+0000 7f39dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc882870 0x5566cc88c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60292/0 (socket says 172.21.15.47:60292) 2024-03-21T11:36:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.000+0000 7f39dfe00640 1 -- 172.21.15.47:0/2791716790 learned_addr learned my addr 172.21.15.47:0/2791716790 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.000+0000 7f39e5800640 1 --2- 172.21.15.47:0/2791716790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc8879e0 0x5566cc893380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.000+0000 7f39e4e00640 1 --2- 172.21.15.47:0/2791716790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc881e80 0x5566cc88be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.001+0000 7f39dfe00640 1 -- 172.21.15.47:0/2791716790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc8879e0 msgr2=0x5566cc893380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.001+0000 7f39dfe00640 1 --2- 172.21.15.47:0/2791716790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc8879e0 0x5566cc893380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.001+0000 7f39dfe00640 1 -- 172.21.15.47:0/2791716790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc881e80 msgr2=0x5566cc88be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.001+0000 7f39dfe00640 1 --2- 172.21.15.47:0/2791716790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc881e80 0x5566cc88be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.001+0000 7f39dfe00640 1 -- 172.21.15.47:0/2791716790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566cc5c57d0 con 0x5566cc882870 2024-03-21T11:36:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.001+0000 7f39e4e00640 1 --2- 172.21.15.47:0/2791716790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc881e80 0x5566cc88be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.001+0000 7f39dfe00640 1 --2- 172.21.15.47:0/2791716790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc882870 0x5566cc88c340 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f39c800bb90 tx=0x7f39c8005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbdd5e24352d0402 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.001+0000 7f39cea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:36:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.001+0000 7f39cea00640 1 -- 172.21.15.47:0/2791716790 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c809f5a0 con 0x5566cc882870 2024-03-21T11:36:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.001+0000 7f39cea00640 1 -- 172.21.15.47:0/2791716790 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f39c808a0b0 con 0x5566cc882870 2024-03-21T11:36:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.001+0000 7f39ea445740 1 -- 172.21.15.47:0/2791716790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39d002cd30 con 0x5566cc882870 2024-03-21T11:36:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.001+0000 7f39cea00640 1 -- 172.21.15.47:0/2791716790 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39c809f130 con 0x5566cc882870 2024-03-21T11:36:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.001+0000 7f39ea445740 10 client.30107 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.001+0000 7f39ea445740 1 -- 172.21.15.47:0/2791716790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5566cc4fb3f0 con 0x5566cc882870 2024-03-21T11:36:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.002+0000 7f39cea00640 1 -- 172.21.15.47:0/2791716790 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39d002cd30 con 0x5566cc882870 2024-03-21T11:36:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.002+0000 7f39cea00640 1 -- 172.21.15.47:0/2791716790 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f39c809ed40 con 0x5566cc882870 2024-03-21T11:36:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.002+0000 7f39ea445740 10 client.30107 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.002+0000 7f39ea445740 10 client.30107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.002+0000 7f39ea445740 20 client.30107 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.002+0000 7f39ea445740 1 --2- 172.21.15.47:0/2791716790 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5566cc897bc0 0x5566cc89a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.002+0000 7f39ea445740 4 client.30107 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:36:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.002+0000 7f39ea445740 1 -- 172.21.15.47:0/2791716790 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566cc7f4790 con 0x5566cc897bc0 2024-03-21T11:36:54.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.002+0000 7f39e4e00640 1 --2- 172.21.15.47:0/2791716790 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5566cc897bc0 0x5566cc89a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:54.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.003+0000 7f39e4e00640 1 --2- 172.21.15.47:0/2791716790 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5566cc897bc0 0x5566cc89a000 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d403c17937e015 server_cookie=505d514e7b960539 in_seq=0 out_seq=0 2024-03-21T11:36:54.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.003+0000 7f39cea00640 10 client.30107 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.004+0000 7f39cea00640 1 -- 172.21.15.47:0/2791716790 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5566cc7f4790 con 0x5566cc897bc0 2024-03-21T11:36:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.004+0000 7f39cea00640 10 client.30107 handle_command_reply: tid=0 2024-03-21T11:36:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.004+0000 7f39ea445740 1 client.30107 shutdown 2024-03-21T11:36:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.005+0000 7f39ea445740 1 -- 172.21.15.47:0/2791716790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc882870 msgr2=0x5566cc88c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.005+0000 7f39ea445740 1 --2- 172.21.15.47:0/2791716790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc882870 0x5566cc88c340 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f39c800bb90 tx=0x7f39c8005f10 comp rx=0 tx=0).stop 2024-03-21T11:36:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.005+0000 7f39ea445740 1 -- 172.21.15.47:0/2791716790 shutdown_connections 2024-03-21T11:36:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.005+0000 7f39ea445740 1 --2- 172.21.15.47:0/2791716790 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5566cc897bc0 0x5566cc89a000 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.005+0000 7f39ea445740 1 --2- 172.21.15.47:0/2791716790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc8879e0 0x5566cc893380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.005+0000 7f39ea445740 1 --2- 172.21.15.47:0/2791716790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc882870 0x5566cc88c340 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.005+0000 7f39ea445740 1 --2- 172.21.15.47:0/2791716790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc881e80 0x5566cc88be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.005+0000 7f39cea00640 0 client.30107 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.005+0000 7f39ea445740 1 -- 172.21.15.47:0/2791716790 >> 172.21.15.47:0/2791716790 conn(0x5566cc84a200 msgr2=0x5566cc87ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.005+0000 7f39ea445740 1 -- 172.21.15.47:0/2791716790 shutdown_connections 2024-03-21T11:36:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.005+0000 7f39ea445740 1 -- 172.21.15.47:0/2791716790 wait complete. 2024-03-21T11:36:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.005+0000 7f39ea445740 20 client.30107 trim_cache size 0 max 16384 2024-03-21T11:36:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.023+0000 7f39ea445740 1 Processor -- start 2024-03-21T11:36:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.023+0000 7f39ea445740 1 -- start start 2024-03-21T11:36:54.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39ea445740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc881e80 0x5566cc87c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:54.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39ea445740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc882870 0x5566cc893340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:54.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39ea445740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc8879e0 0x5566cc87c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:54.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39ea445740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e01a7e50 con 0x5566cc882870 2024-03-21T11:36:54.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39ea445740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e01a97d0 con 0x5566cc8879e0 2024-03-21T11:36:54.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39ea445740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e01a84b0 con 0x5566cc881e80 2024-03-21T11:36:54.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39e4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc881e80 0x5566cc87c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:54.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39e5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc8879e0 0x5566cc87c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:54.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39e5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc8879e0 0x5566cc87c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60308/0 (socket says 172.21.15.47:60308) 2024-03-21T11:36:54.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39e5800640 1 -- 172.21.15.47:0/539349289 learned_addr learned my addr 172.21.15.47:0/539349289 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:54.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39e4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc881e80 0x5566cc87c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38396/0 (socket says 172.21.15.47:38396) 2024-03-21T11:36:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39dfe00640 1 --2- 172.21.15.47:0/539349289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc882870 0x5566cc893340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39e5800640 1 -- 172.21.15.47:0/539349289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc881e80 msgr2=0x5566cc87c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39e5800640 1 --2- 172.21.15.47:0/539349289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc881e80 0x5566cc87c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39e5800640 1 -- 172.21.15.47:0/539349289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc882870 msgr2=0x5566cc893340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.024+0000 7f39e5800640 1 --2- 172.21.15.47:0/539349289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc882870 0x5566cc893340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.025+0000 7f39e5800640 1 -- 172.21.15.47:0/539349289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566cc4bb0f0 con 0x5566cc8879e0 2024-03-21T11:36:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.025+0000 7f39dfe00640 1 --2- 172.21.15.47:0/539349289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc882870 0x5566cc893340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.025+0000 7f39e5800640 1 --2- 172.21.15.47:0/539349289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc8879e0 0x5566cc87c740 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f39d801e740 tx=0x7f39d8019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a47fca38f16e5567 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.025+0000 7f39d5600640 1 -- 172.21.15.47:0/539349289 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8011500 con 0x5566cc8879e0 2024-03-21T11:36:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.025+0000 7f39d5600640 1 -- 172.21.15.47:0/539349289 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39d8011b80 con 0x5566cc8879e0 2024-03-21T11:36:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.025+0000 7f39d5600640 1 -- 172.21.15.47:0/539349289 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8029c40 con 0x5566cc8879e0 2024-03-21T11:36:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.026+0000 7f39ea445740 1 -- 172.21.15.47:0/539349289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc8879e0 msgr2=0x5566cc87c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.026+0000 7f39ea445740 1 --2- 172.21.15.47:0/539349289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc8879e0 0x5566cc87c740 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f39d801e740 tx=0x7f39d8019000 comp rx=0 tx=0).stop 2024-03-21T11:36:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.026+0000 7f39ea445740 1 -- 172.21.15.47:0/539349289 shutdown_connections 2024-03-21T11:36:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.026+0000 7f39ea445740 1 --2- 172.21.15.47:0/539349289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc8879e0 0x5566cc87c740 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.026+0000 7f39ea445740 1 --2- 172.21.15.47:0/539349289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc882870 0x5566cc893340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.026+0000 7f39ea445740 1 --2- 172.21.15.47:0/539349289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc881e80 0x5566cc87c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.026+0000 7f39ea445740 1 -- 172.21.15.47:0/539349289 >> 172.21.15.47:0/539349289 conn(0x5566cc84a200 msgr2=0x5566cc88be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.026+0000 7f39ea445740 1 -- 172.21.15.47:0/539349289 shutdown_connections 2024-03-21T11:36:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.026+0000 7f39ea445740 1 -- 172.21.15.47:0/539349289 wait complete. 2024-03-21T11:36:54.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.027+0000 7f39ea445740 1 -- start start 2024-03-21T11:36:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.027+0000 7f39ea445740 1 Processor -- start 2024-03-21T11:36:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.027+0000 7f39ea445740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc881e80 0x5566cc8909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.027+0000 7f39ea445740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc882870 0x5566cc890ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.027+0000 7f39ea445740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc8879e0 0x5566cc891ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.027+0000 7f39ea445740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e01a9170 con 0x5566cc8879e0 2024-03-21T11:36:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.027+0000 7f39ea445740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e01a8b10 con 0x5566cc881e80 2024-03-21T11:36:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.027+0000 7f39ea445740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e01a6010 con 0x5566cc882870 2024-03-21T11:36:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.027+0000 7f39ea445740 10 client.0 _ll_register_callbacks cb 0x7f39c80d6b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:36:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.027+0000 7f39dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc882870 0x5566cc890ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39e4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc881e80 0x5566cc8909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc882870 0x5566cc890ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38410/0 (socket says 172.21.15.47:38410) 2024-03-21T11:36:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39e4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc881e80 0x5566cc8909a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60318/0 (socket says 172.21.15.47:60318) 2024-03-21T11:36:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39dfe00640 1 -- 172.21.15.47:0/3248027098 learned_addr learned my addr 172.21.15.47:0/3248027098 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:36:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39e5800640 1 --2- 172.21.15.47:0/3248027098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc8879e0 0x5566cc891ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39dfe00640 1 -- 172.21.15.47:0/3248027098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc881e80 msgr2=0x5566cc8909a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39dfe00640 1 --2- 172.21.15.47:0/3248027098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc881e80 0x5566cc8909a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39dfe00640 1 -- 172.21.15.47:0/3248027098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc8879e0 msgr2=0x5566cc891ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:36:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39dfe00640 1 --2- 172.21.15.47:0/3248027098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc8879e0 0x5566cc891ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39dfe00640 1 -- 172.21.15.47:0/3248027098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566cc8924e0 con 0x5566cc882870 2024-03-21T11:36:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39e5800640 1 --2- 172.21.15.47:0/3248027098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc8879e0 0x5566cc891ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:36:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39e4e00640 1 --2- 172.21.15.47:0/3248027098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc881e80 0x5566cc8909a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:36:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39dfe00640 1 --2- 172.21.15.47:0/3248027098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc882870 0x5566cc890ee0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f39c809b730 tx=0x7f39c809b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=990562e27d2ae57a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:36:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39cea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:36:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39cea00640 1 -- 172.21.15.47:0/3248027098 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c809f310 con 0x5566cc882870 2024-03-21T11:36:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39cea00640 1 -- 172.21.15.47:0/3248027098 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f39c8017520 con 0x5566cc882870 2024-03-21T11:36:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.028+0000 7f39ea445740 1 -- 172.21.15.47:0/3248027098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39d8011b80 con 0x5566cc882870 2024-03-21T11:36:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.029+0000 7f39cea00640 1 -- 172.21.15.47:0/3248027098 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39c809f5a0 con 0x5566cc882870 2024-03-21T11:36:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.029+0000 7f39ea445740 10 client.30145 fetch_fsmap learned FSMap version 28 2024-03-21T11:36:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.029+0000 7f39ea445740 1 -- 172.21.15.47:0/3248027098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5566cc8f3490 con 0x5566cc882870 2024-03-21T11:36:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.029+0000 7f39cea00640 1 -- 172.21.15.47:0/3248027098 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39d8011b80 con 0x5566cc882870 2024-03-21T11:36:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.029+0000 7f39cea00640 1 -- 172.21.15.47:0/3248027098 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f39c80a4a00 con 0x5566cc882870 2024-03-21T11:36:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.029+0000 7f39ea445740 10 client.30145 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:36:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.029+0000 7f39ea445740 10 client.30145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:36:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.029+0000 7f39ea445740 20 client.30145 populate_metadata read hostname 'smithi047' 2024-03-21T11:36:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.029+0000 7f39ea445740 1 --2- 172.21.15.47:0/3248027098 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5566cc8f3cb0 0x5566cc8f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:36:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.029+0000 7f39ea445740 4 client.30145 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:36:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.029+0000 7f39ea445740 1 -- 172.21.15.47:0/3248027098 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5566cc7eeae0 con 0x5566cc8f3cb0 2024-03-21T11:36:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.030+0000 7f39e4e00640 1 --2- 172.21.15.47:0/3248027098 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5566cc8f3cb0 0x5566cc8f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:36:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.030+0000 7f39e4e00640 1 --2- 172.21.15.47:0/3248027098 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5566cc8f3cb0 0x5566cc8f60d0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ced89bfca420be server_cookie=5f00d22661c7daad in_seq=0 out_seq=0 2024-03-21T11:36:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.030+0000 7f39cea00640 10 client.30145 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.032+0000 7f39cea00640 1 -- 172.21.15.47:0/3248027098 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5566cc7eeae0 con 0x5566cc8f3cb0 2024-03-21T11:36:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.032+0000 7f39cea00640 10 client.30145 handle_command_reply: tid=0 2024-03-21T11:36:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.032+0000 7f39ea445740 1 client.30145 shutdown 2024-03-21T11:36:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.032+0000 7f39ea445740 1 -- 172.21.15.47:0/3248027098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc882870 msgr2=0x5566cc890ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.032+0000 7f39ea445740 1 --2- 172.21.15.47:0/3248027098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc882870 0x5566cc890ee0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f39c809b730 tx=0x7f39c809b630 comp rx=0 tx=0).stop 2024-03-21T11:36:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.032+0000 7f39ea445740 1 -- 172.21.15.47:0/3248027098 shutdown_connections 2024-03-21T11:36:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.032+0000 7f39ea445740 1 --2- 172.21.15.47:0/3248027098 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5566cc8f3cb0 0x5566cc8f60d0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.032+0000 7f39ea445740 1 --2- 172.21.15.47:0/3248027098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566cc8879e0 0x5566cc891ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.032+0000 7f39ea445740 1 --2- 172.21.15.47:0/3248027098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566cc882870 0x5566cc890ee0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.032+0000 7f39ea445740 1 --2- 172.21.15.47:0/3248027098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566cc881e80 0x5566cc8909a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.032+0000 7f39cea00640 0 client.30145 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:36:54.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.032+0000 7f39ea445740 1 -- 172.21.15.47:0/3248027098 >> 172.21.15.47:0/3248027098 conn(0x5566cc84a200 msgr2=0x5566cc883310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.032+0000 7f39ea445740 1 -- 172.21.15.47:0/3248027098 shutdown_connections 2024-03-21T11:36:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.032+0000 7f39ea445740 1 -- 172.21.15.47:0/3248027098 wait complete. 2024-03-21T11:36:54.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.033+0000 7f39ea445740 20 client.30145 trim_cache size 0 max 16384 2024-03-21T11:36:54.037 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:36:54.037 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:36:54.037 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:36:54.037 INFO:teuthology.orchestra.run.smithi047.stdout: "f6a3d77a-2ad1-4f3f-8acc-e540cb113340": { 2024-03-21T11:36:54.037 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:36:54.037 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "f6a3d77a-2ad1-4f3f-8acc-e540cb113340", 2024-03-21T11:36:54.037 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:36:54.038 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:36:54.038 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:36:54.038 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:36:54.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.033+0000 7f39e7800640 1 -- 172.21.15.47:0/125325611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39c00a2fa0 msgr2=0x7f39c00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:54.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.033+0000 7f39e7800640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39c00a2fa0 0x7f39c00a5460 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x5566cc8830d0 tx=0x7f39d800b2e0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.033+0000 7f39e7800640 1 -- 172.21.15.47:0/125325611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e01028a0 msgr2=0x7f39e0197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:36:54.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.033+0000 7f39e7800640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e01028a0 0x7f39e0197f30 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f39c8010e10 tx=0x7f39c800b690 comp rx=0 tx=0).stop 2024-03-21T11:36:54.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.033+0000 7f39e7800640 1 -- 172.21.15.47:0/125325611 shutdown_connections 2024-03-21T11:36:54.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.033+0000 7f39e7800640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f39c00a2fa0 0x7f39c00a5460 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.033+0000 7f39e7800640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39e0103260 0x7f39e019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.033+0000 7f39e7800640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e01028a0 0x7f39e0197f30 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.033+0000 7f39e7800640 1 --2- 172.21.15.47:0/125325611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e01016b0 0x7f39e01979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:36:54.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.033+0000 7f39e7800640 1 -- 172.21.15.47:0/125325611 >> 172.21.15.47:0/125325611 conn(0x7f39e00fcc40 msgr2=0x7f39e0105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:36:54.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.034+0000 7f39e7800640 1 -- 172.21.15.47:0/125325611 shutdown_connections 2024-03-21T11:36:54.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:36:54.034+0000 7f39e7800640 1 -- 172.21.15.47:0/125325611 wait complete. 2024-03-21T11:36:54.053 INFO:tasks.cephfs.filesystem:scrub status for tag:f6a3d77a-2ad1-4f3f-8acc-e540cb113340 - {'path': '/', 'tag': 'f6a3d77a-2ad1-4f3f-8acc-e540cb113340', 'options': 'recursive,force'} 2024-03-21T11:36:54.583 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1038212 7.92 MB/sec execute 578 sec latency 27.983 ms 2024-03-21T11:36:54.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:54 smithi047 ceph-mon[31351]: pgmap v807: 129 pgs: 129 active+clean; 40 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 281 op/s 2024-03-21T11:36:54.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:54 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:36:54.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:54 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:36:54.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:54 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:36:54.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:54 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:36:54.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:54 smithi100 ceph-mon[36245]: pgmap v807: 129 pgs: 129 active+clean; 40 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 281 op/s 2024-03-21T11:36:54.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:54 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:36:54.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:54 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:36:54.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:54 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:36:54.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:54 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:36:54.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:54 smithi102 ceph-mon[36748]: pgmap v807: 129 pgs: 129 active+clean; 40 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 281 op/s 2024-03-21T11:36:54.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:54 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:36:54.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:54 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:36:54.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:54 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:36:54.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:54 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:36:55.583 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1039922 7.92 MB/sec execute 579 sec latency 22.350 ms 2024-03-21T11:36:56.583 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1041293 7.92 MB/sec execute 580 sec latency 13.686 ms 2024-03-21T11:36:56.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:56 smithi100 ceph-mon[36245]: pgmap v808: 129 pgs: 129 active+clean; 39 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2024-03-21T11:36:56.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:56 smithi102 ceph-mon[36748]: pgmap v808: 129 pgs: 129 active+clean; 39 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2024-03-21T11:36:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:56 smithi047 ceph-mon[31351]: pgmap v808: 129 pgs: 129 active+clean; 39 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2024-03-21T11:36:57.583 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1042367 7.92 MB/sec execute 581 sec latency 19.225 ms 2024-03-21T11:36:57.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:57 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:36:57.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:57 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:36:58.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:57 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:36:58.583 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1044080 7.92 MB/sec execute 582 sec latency 16.125 ms 2024-03-21T11:36:58.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:36:58 smithi100 ceph-mon[36245]: pgmap v809: 129 pgs: 129 active+clean; 40 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2024-03-21T11:36:58.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:36:58 smithi102 ceph-mon[36748]: pgmap v809: 129 pgs: 129 active+clean; 40 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2024-03-21T11:36:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:36:58 smithi047 ceph-mon[31351]: pgmap v809: 129 pgs: 129 active+clean; 40 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2024-03-21T11:36:59.583 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1044881 7.91 MB/sec execute 583 sec latency 69.650 ms 2024-03-21T11:37:00.583 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1046074 7.91 MB/sec execute 584 sec latency 22.601 ms 2024-03-21T11:37:00.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:00 smithi047 ceph-mon[31351]: pgmap v810: 129 pgs: 129 active+clean; 40 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 290 op/s 2024-03-21T11:37:00.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:00 smithi100 ceph-mon[36245]: pgmap v810: 129 pgs: 129 active+clean; 40 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 290 op/s 2024-03-21T11:37:00.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:00 smithi102 ceph-mon[36748]: pgmap v810: 129 pgs: 129 active+clean; 40 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 290 op/s 2024-03-21T11:37:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.454+0000 7ff3c1800640 1 -- 172.21.15.47:0/1117910549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3bc10dae0 msgr2=0x7ff3bc10fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.454+0000 7ff3c1800640 1 --2- 172.21.15.47:0/1117910549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3bc10dae0 0x7ff3bc10fed0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009ca0 tx=0x7ff3b402c100 comp rx=0 tx=0).stop 2024-03-21T11:37:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.454+0000 7ff3c1800640 1 -- 172.21.15.47:0/1117910549 shutdown_connections 2024-03-21T11:37:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.454+0000 7ff3c1800640 1 --2- 172.21.15.47:0/1117910549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3bc10dae0 0x7ff3bc10fed0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.454+0000 7ff3c1800640 1 --2- 172.21.15.47:0/1117910549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3bc074100 0x7ff3bc074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.454+0000 7ff3c1800640 1 --2- 172.21.15.47:0/1117910549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3bc0756e0 0x7ff3bc073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.454+0000 7ff3c1800640 1 -- 172.21.15.47:0/1117910549 >> 172.21.15.47:0/1117910549 conn(0x7ff3bc0fca80 msgr2=0x7ff3bc0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.454+0000 7ff3c1800640 1 -- 172.21.15.47:0/1117910549 shutdown_connections 2024-03-21T11:37:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.454+0000 7ff3c1800640 1 -- 172.21.15.47:0/1117910549 wait complete. 2024-03-21T11:37:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.455+0000 7ff3c1800640 1 Processor -- start 2024-03-21T11:37:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.455+0000 7ff3c1800640 1 -- start start 2024-03-21T11:37:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.456+0000 7ff3c1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3bc074100 0x7ff3bc19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.456+0000 7ff3c1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3bc0756e0 0x7ff3bc19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.456+0000 7ff3c1800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3bc10dae0 0x7ff3bc1a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.456+0000 7ff3c1800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc113780 con 0x7ff3bc074100 2024-03-21T11:37:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.456+0000 7ff3c1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc113600 con 0x7ff3bc0756e0 2024-03-21T11:37:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.456+0000 7ff3c1800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc113900 con 0x7ff3bc10dae0 2024-03-21T11:37:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.456+0000 7ff3bbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3bc074100 0x7ff3bc19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.456+0000 7ff3bb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3bc0756e0 0x7ff3bc19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.456+0000 7ff3c0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3bc10dae0 0x7ff3bc1a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.456+0000 7ff3bbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3bc074100 0x7ff3bc19ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45176/0 (socket says 172.21.15.47:45176) 2024-03-21T11:37:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.456+0000 7ff3bbe00640 1 -- 172.21.15.47:0/3831851438 learned_addr learned my addr 172.21.15.47:0/3831851438 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.457+0000 7ff3c0e00640 1 -- 172.21.15.47:0/3831851438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3bc0756e0 msgr2=0x7ff3bc19e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.457+0000 7ff3c0e00640 1 --2- 172.21.15.47:0/3831851438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3bc0756e0 0x7ff3bc19e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.457+0000 7ff3c0e00640 1 -- 172.21.15.47:0/3831851438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3bc074100 msgr2=0x7ff3bc19ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.457+0000 7ff3c0e00640 1 --2- 172.21.15.47:0/3831851438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3bc074100 0x7ff3bc19ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.457+0000 7ff3c0e00640 1 -- 172.21.15.47:0/3831851438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc1a59d0 con 0x7ff3bc10dae0 2024-03-21T11:37:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.457+0000 7ff3bb400640 1 --2- 172.21.15.47:0/3831851438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3bc0756e0 0x7ff3bc19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.457+0000 7ff3c0e00640 1 --2- 172.21.15.47:0/3831851438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3bc10dae0 0x7ff3bc1a5330 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4037040 tx=0x7ff3b4034670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.457+0000 7ff3b8c00640 1 -- 172.21.15.47:0/3831851438 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4039050 con 0x7ff3bc10dae0 2024-03-21T11:37:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.458+0000 7ff3b8c00640 1 -- 172.21.15.47:0/3831851438 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3b4045070 con 0x7ff3bc10dae0 2024-03-21T11:37:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.458+0000 7ff3b8c00640 1 -- 172.21.15.47:0/3831851438 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b40047a0 con 0x7ff3bc10dae0 2024-03-21T11:37:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.458+0000 7ff3c1800640 1 -- 172.21.15.47:0/3831851438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3bc1a5c60 con 0x7ff3bc10dae0 2024-03-21T11:37:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.458+0000 7ff3c1800640 1 -- 172.21.15.47:0/3831851438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3bc1a60d0 con 0x7ff3bc10dae0 2024-03-21T11:37:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.462+0000 7ff3c5bcc740 1 Processor -- start 2024-03-21T11:37:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.463+0000 7ff3b8c00640 1 -- 172.21.15.47:0/3831851438 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff3b4003970 con 0x7ff3bc10dae0 2024-03-21T11:37:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.463+0000 7ff3c5bcc740 1 -- start start 2024-03-21T11:37:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.463+0000 7ff3c5bcc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 0x55da3cdc7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.463+0000 7ff3c5bcc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdc7de0 0x55da3cdcc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.463+0000 7ff3c5bcc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdccf50 0x55da3cdcf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.463+0000 7ff3c5bcc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3cd8a3b0 con 0x55da3cdccf50 2024-03-21T11:37:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.463+0000 7ff3c5bcc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3cd2c830 con 0x55da3cdc7de0 2024-03-21T11:37:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.463+0000 7ff3c5bcc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3ccd55d0 con 0x55da3cdc73f0 2024-03-21T11:37:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.463+0000 7ff3bbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdc7de0 0x55da3cdcc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3bbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdc7de0 0x55da3cdcc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46976/0 (socket says 172.21.15.47:46976) 2024-03-21T11:37:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3bbe00640 1 -- 172.21.15.47:0/125578594 learned_addr learned my addr 172.21.15.47:0/125578594 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3bb400640 1 --2- 172.21.15.47:0/125578594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 0x55da3cdc7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3bb400640 1 --2- 172.21.15.47:0/125578594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdccf50 0x55da3cdcf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3bbe00640 1 -- 172.21.15.47:0/125578594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 msgr2=0x55da3cdc7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3bbe00640 1 --2- 172.21.15.47:0/125578594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 0x55da3cdc7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3bbe00640 1 -- 172.21.15.47:0/125578594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdccf50 msgr2=0x55da3cdcf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3bbe00640 1 --2- 172.21.15.47:0/125578594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdccf50 0x55da3cdcf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3bbe00640 1 -- 172.21.15.47:0/125578594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da3cced2c0 con 0x55da3cdc7de0 2024-03-21T11:37:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3bb400640 1 --2- 172.21.15.47:0/125578594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdccf50 0x55da3cdcf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:37:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3bbe00640 1 --2- 172.21.15.47:0/125578594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdc7de0 0x55da3cdcc8e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00bd60 tx=0x7ff3ac0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4078ddc67c6db5f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3b1600640 1 -- 172.21.15.47:0/125578594 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac01c680 con 0x55da3cdc7de0 2024-03-21T11:37:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3b8c00640 1 --2- 172.21.15.47:0/3831851438 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff3a00a2f90 0x7ff3a00a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3b1600640 1 -- 172.21.15.47:0/125578594 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3ac011920 con 0x55da3cdc7de0 2024-03-21T11:37:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.464+0000 7ff3b1600640 1 -- 172.21.15.47:0/125578594 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac01a7e0 con 0x55da3cdc7de0 2024-03-21T11:37:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.465+0000 7ff3b8c00640 1 -- 172.21.15.47:0/3831851438 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff3b40f1fd0 con 0x7ff3bc10dae0 2024-03-21T11:37:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.465+0000 7ff3bb400640 1 --2- 172.21.15.47:0/3831851438 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff3a00a2f90 0x7ff3a00a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.465+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/125578594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdc7de0 msgr2=0x55da3cdcc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.465+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/125578594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdc7de0 0x55da3cdcc8e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00bd60 tx=0x7ff3ac0107a0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.466+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/125578594 shutdown_connections 2024-03-21T11:37:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.466+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/125578594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdccf50 0x55da3cdcf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.466+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/125578594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdc7de0 0x55da3cdcc8e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.466+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/125578594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 0x55da3cdc7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.466+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/125578594 >> 172.21.15.47:0/125578594 conn(0x55da3cdc2960 msgr2=0x55da3cdc2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.466+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/125578594 shutdown_connections 2024-03-21T11:37:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.466+0000 7ff3bb400640 1 --2- 172.21.15.47:0/3831851438 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff3a00a2f90 0x7ff3a00a5450 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55da3cdc8640 tx=0x7ff3a800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.466+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/125578594 wait complete. 2024-03-21T11:37:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.466+0000 7ff3c5bcc740 1 -- start start 2024-03-21T11:37:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.466+0000 7ff3c5bcc740 1 Processor -- start 2024-03-21T11:37:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3c5bcc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdc73f0 0x55da3cdd15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3c5bcc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc7de0 0x55da3cdd1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3c5bcc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdccf50 0x55da3cdd8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3c5bcc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3cce8a20 con 0x55da3cdc73f0 2024-03-21T11:37:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3c5bcc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3cd288d0 con 0x55da3cdccf50 2024-03-21T11:37:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3c5bcc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3cdcff00 con 0x55da3cdc7de0 2024-03-21T11:37:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3bb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc7de0 0x55da3cdd1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3c5bcc740 10 client.0 _ll_register_callbacks cb 0x55da3cd251c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3bbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdc73f0 0x55da3cdd15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3bb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc7de0 0x55da3cdd1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43998/0 (socket says 172.21.15.47:43998) 2024-03-21T11:37:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3bb400640 1 -- 172.21.15.47:0/958931631 learned_addr learned my addr 172.21.15.47:0/958931631 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3c0e00640 1 --2- 172.21.15.47:0/958931631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdccf50 0x55da3cdd8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3c0e00640 1 -- 172.21.15.47:0/958931631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc7de0 msgr2=0x55da3cdd1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3c0e00640 1 --2- 172.21.15.47:0/958931631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc7de0 0x55da3cdd1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3c0e00640 1 -- 172.21.15.47:0/958931631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdc73f0 msgr2=0x55da3cdd15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.467+0000 7ff3c0e00640 1 --2- 172.21.15.47:0/958931631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdc73f0 0x55da3cdd15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.468+0000 7ff3c0e00640 1 -- 172.21.15.47:0/958931631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da3ca41210 con 0x55da3cdccf50 2024-03-21T11:37:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.468+0000 7ff3c0e00640 1 --2- 172.21.15.47:0/958931631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdccf50 0x55da3cdd8b30 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40ef6e0 tx=0x7ff3b40be1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8e94d8ab6220aa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.468+0000 7ff3a6000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:37:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.468+0000 7ff3a6000640 1 -- 172.21.15.47:0/958931631 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b40bea10 con 0x55da3cdccf50 2024-03-21T11:37:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.468+0000 7ff3a6000640 1 -- 172.21.15.47:0/958931631 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff3b40c2160 con 0x55da3cdccf50 2024-03-21T11:37:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.468+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/958931631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3ac011920 con 0x55da3cdccf50 2024-03-21T11:37:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.468+0000 7ff3a6000640 1 -- 172.21.15.47:0/958931631 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3b40be310 con 0x55da3cdccf50 2024-03-21T11:37:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.468+0000 7ff3c5bcc740 10 client.30125 fetch_fsmap learned FSMap version 28 2024-03-21T11:37:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.468+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/958931631 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55da3cb0d620 con 0x55da3cdccf50 2024-03-21T11:37:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.469+0000 7ff3a6000640 1 -- 172.21.15.47:0/958931631 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3ac011920 con 0x55da3cdccf50 2024-03-21T11:37:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.469+0000 7ff3a6000640 1 -- 172.21.15.47:0/958931631 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff3b40c6890 con 0x55da3cdccf50 2024-03-21T11:37:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.469+0000 7ff3c5bcc740 10 client.30125 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:37:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.469+0000 7ff3c5bcc740 10 client.30125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.469+0000 7ff3c5bcc740 20 client.30125 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.469+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/958931631 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55da3cddd320 0x55da3cddf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.469+0000 7ff3c5bcc740 4 client.30125 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:37:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.469+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/958931631 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da3cd39d70 con 0x55da3cddd320 2024-03-21T11:37:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.469+0000 7ff3bbe00640 1 --2- 172.21.15.47:0/958931631 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55da3cddd320 0x55da3cddf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.470+0000 7ff3bbe00640 1 --2- 172.21.15.47:0/958931631 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55da3cddd320 0x55da3cddf750 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c17789dc1832d25 server_cookie=87a1cb657fdc1282 in_seq=0 out_seq=0 2024-03-21T11:37:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.470+0000 7ff3a6000640 10 client.30125 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.471+0000 7ff3a6000640 1 -- 172.21.15.47:0/958931631 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55da3cd39d70 con 0x55da3cddd320 2024-03-21T11:37:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.471+0000 7ff3a6000640 10 client.30125 handle_command_reply: tid=0 2024-03-21T11:37:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.471+0000 7ff3c5bcc740 1 client.30125 shutdown 2024-03-21T11:37:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.471+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/958931631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdccf50 msgr2=0x55da3cdd8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.471+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/958931631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdccf50 0x55da3cdd8b30 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40ef6e0 tx=0x7ff3b40be1a0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.471+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/958931631 shutdown_connections 2024-03-21T11:37:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.471+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/958931631 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55da3cddd320 0x55da3cddf750 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.471+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/958931631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdccf50 0x55da3cdd8b30 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.471+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/958931631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc7de0 0x55da3cdd1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.471+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/958931631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdc73f0 0x55da3cdd15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.471+0000 7ff3a6000640 0 client.30125 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.471+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/958931631 >> 172.21.15.47:0/958931631 conn(0x55da3cdc2960 msgr2=0x55da3cdc1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.471+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/958931631 shutdown_connections 2024-03-21T11:37:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.471+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/958931631 wait complete. 2024-03-21T11:37:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.472+0000 7ff3c5bcc740 20 client.30125 trim_cache size 0 max 16384 2024-03-21T11:37:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.488+0000 7ff3c5bcc740 1 Processor -- start 2024-03-21T11:37:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.488+0000 7ff3c5bcc740 1 -- start start 2024-03-21T11:37:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.488+0000 7ff3c5bcc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 0x55da3cdc9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.488+0000 7ff3c5bcc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdc7de0 0x55da3cdd8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.488+0000 7ff3c5bcc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdccf50 0x55da3cdc9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.488+0000 7ff3c5bcc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc1b03c0 con 0x55da3cdccf50 2024-03-21T11:37:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.488+0000 7ff3c5bcc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc1b1d40 con 0x55da3cdc7de0 2024-03-21T11:37:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.488+0000 7ff3c5bcc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc1b0a20 con 0x55da3cdc73f0 2024-03-21T11:37:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.488+0000 7ff3bb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdc7de0 0x55da3cdd8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.488+0000 7ff3bb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdc7de0 0x55da3cdd8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46990/0 (socket says 172.21.15.47:46990) 2024-03-21T11:37:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.489+0000 7ff3bb400640 1 -- 172.21.15.47:0/963598455 learned_addr learned my addr 172.21.15.47:0/963598455 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.489+0000 7ff3bbe00640 1 --2- 172.21.15.47:0/963598455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 0x55da3cdc9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.489+0000 7ff3c0e00640 1 --2- 172.21.15.47:0/963598455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdccf50 0x55da3cdc9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.489+0000 7ff3bbe00640 1 -- 172.21.15.47:0/963598455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdc7de0 msgr2=0x55da3cdd8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.489+0000 7ff3bbe00640 1 --2- 172.21.15.47:0/963598455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdc7de0 0x55da3cdd8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.489+0000 7ff3bbe00640 1 -- 172.21.15.47:0/963598455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdccf50 msgr2=0x55da3cdc9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.489+0000 7ff3bbe00640 1 --2- 172.21.15.47:0/963598455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdccf50 0x55da3cdc9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.489+0000 7ff3bbe00640 1 -- 172.21.15.47:0/963598455 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da3cdd1e40 con 0x55da3cdc73f0 2024-03-21T11:37:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.489+0000 7ff3bb400640 1 --2- 172.21.15.47:0/963598455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdc7de0 0x55da3cdd8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.489+0000 7ff3bbe00640 1 --2- 172.21.15.47:0/963598455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 0x55da3cdc9140 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x55da3cdc38d0 tx=0x7ff3ac00b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43087e37b90c2de6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.490+0000 7ff3b0c00640 1 -- 172.21.15.47:0/963598455 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac02b310 con 0x55da3cdc73f0 2024-03-21T11:37:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.490+0000 7ff3b0c00640 1 -- 172.21.15.47:0/963598455 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3ac011e00 con 0x55da3cdc73f0 2024-03-21T11:37:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.490+0000 7ff3b0c00640 1 -- 172.21.15.47:0/963598455 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac0053b0 con 0x55da3cdc73f0 2024-03-21T11:37:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.490+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/963598455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 msgr2=0x55da3cdc9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.490+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/963598455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 0x55da3cdc9140 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x55da3cdc38d0 tx=0x7ff3ac00b480 comp rx=0 tx=0).stop 2024-03-21T11:37:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.491+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/963598455 shutdown_connections 2024-03-21T11:37:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.491+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/963598455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdccf50 0x55da3cdc9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.491+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/963598455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdc7de0 0x55da3cdd8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.491+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/963598455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 0x55da3cdc9140 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.491+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/963598455 >> 172.21.15.47:0/963598455 conn(0x55da3cdc2960 msgr2=0x55da3cdc2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.491+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/963598455 shutdown_connections 2024-03-21T11:37:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.491+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/963598455 wait complete. 2024-03-21T11:37:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.491+0000 7ff3c5bcc740 1 -- start start 2024-03-21T11:37:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.491+0000 7ff3c5bcc740 1 Processor -- start 2024-03-21T11:37:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.492+0000 7ff3c5bcc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 0x55da3cdd6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.492+0000 7ff3c5bcc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdc7de0 0x55da3cdd6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.492+0000 7ff3c5bcc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdccf50 0x55da3cdd7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.492+0000 7ff3c5bcc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc1b16e0 con 0x55da3cdc7de0 2024-03-21T11:37:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.492+0000 7ff3c5bcc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc1b1080 con 0x55da3cdccf50 2024-03-21T11:37:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.492+0000 7ff3c5bcc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc1ae580 con 0x55da3cdc73f0 2024-03-21T11:37:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.492+0000 7ff3c5bcc740 10 client.0 _ll_register_callbacks cb 0x7ff3b4009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.492+0000 7ff3bbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 0x55da3cdd6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.492+0000 7ff3bbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 0x55da3cdd6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44020/0 (socket says 172.21.15.47:44020) 2024-03-21T11:37:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.492+0000 7ff3bbe00640 1 -- 172.21.15.47:0/1027435028 learned_addr learned my addr 172.21.15.47:0/1027435028 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.492+0000 7ff3bb400640 1 --2- 172.21.15.47:0/1027435028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdc7de0 0x55da3cdd6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.492+0000 7ff3c0e00640 1 --2- 172.21.15.47:0/1027435028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdccf50 0x55da3cdd7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.493+0000 7ff3c0e00640 1 -- 172.21.15.47:0/1027435028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 msgr2=0x55da3cdd6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.493+0000 7ff3c0e00640 1 --2- 172.21.15.47:0/1027435028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 0x55da3cdd6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.493+0000 7ff3c0e00640 1 -- 172.21.15.47:0/1027435028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdc7de0 msgr2=0x55da3cdd6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.493+0000 7ff3c0e00640 1 --2- 172.21.15.47:0/1027435028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdc7de0 0x55da3cdd6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.493+0000 7ff3c0e00640 1 -- 172.21.15.47:0/1027435028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da3cdd8230 con 0x55da3cdccf50 2024-03-21T11:37:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.493+0000 7ff3bb400640 1 --2- 172.21.15.47:0/1027435028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdc7de0 0x55da3cdd6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.494+0000 7ff3c0e00640 1 --2- 172.21.15.47:0/1027435028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdccf50 0x55da3cdd7c40 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4005e30 tx=0x7ff3b402c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6445faa0cba5ce1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.494+0000 7ff3a6000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:37:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.494+0000 7ff3a6000640 1 -- 172.21.15.47:0/1027435028 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4003e60 con 0x55da3cdccf50 2024-03-21T11:37:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.494+0000 7ff3a6000640 1 -- 172.21.15.47:0/1027435028 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff3b4004b40 con 0x55da3cdccf50 2024-03-21T11:37:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.494+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/1027435028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da3cd340c0 con 0x55da3cdccf50 2024-03-21T11:37:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.494+0000 7ff3a6000640 1 -- 172.21.15.47:0/1027435028 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3b40bea10 con 0x55da3cdccf50 2024-03-21T11:37:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.494+0000 7ff3c5bcc740 10 client.30131 fetch_fsmap learned FSMap version 28 2024-03-21T11:37:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.494+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/1027435028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55da3ce39010 con 0x55da3cdccf50 2024-03-21T11:37:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.495+0000 7ff3a6000640 1 -- 172.21.15.47:0/1027435028 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55da3cd340c0 con 0x55da3cdccf50 2024-03-21T11:37:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.495+0000 7ff3a6000640 1 -- 172.21.15.47:0/1027435028 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff3b40efce0 con 0x55da3cdccf50 2024-03-21T11:37:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.495+0000 7ff3c5bcc740 10 client.30131 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:37:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.495+0000 7ff3c5bcc740 10 client.30131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.495+0000 7ff3c5bcc740 20 client.30131 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.495+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/1027435028 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55da3ce39560 0x55da3ce3b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.495+0000 7ff3c5bcc740 4 client.30131 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:37:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.495+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/1027435028 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55da3ccb25d0 con 0x55da3ce39560 2024-03-21T11:37:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.495+0000 7ff3bbe00640 1 --2- 172.21.15.47:0/1027435028 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55da3ce39560 0x55da3ce3b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.496+0000 7ff3a6000640 10 client.30131 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.498+0000 7ff3bbe00640 1 --2- 172.21.15.47:0/1027435028 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55da3ce39560 0x55da3ce3b980 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e826067a90524f2c server_cookie=1b5af107f1a12206 in_seq=0 out_seq=0 2024-03-21T11:37:01.585 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1047766 7.91 MB/sec execute 585 sec latency 28.729 ms 2024-03-21T11:37:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.766+0000 7ff3a6000640 1 -- 172.21.15.47:0/1027435028 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55da3ccb25d0 con 0x55da3ce39560 2024-03-21T11:37:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.766+0000 7ff3a6000640 10 client.30131 handle_command_reply: tid=0 2024-03-21T11:37:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.766+0000 7ff3c5bcc740 1 client.30131 shutdown 2024-03-21T11:37:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.766+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/1027435028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdccf50 msgr2=0x55da3cdd7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.766+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/1027435028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdccf50 0x55da3cdd7c40 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4005e30 tx=0x7ff3b402c780 comp rx=0 tx=0).stop 2024-03-21T11:37:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.766+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/1027435028 shutdown_connections 2024-03-21T11:37:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.766+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/1027435028 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55da3ce39560 0x55da3ce3b980 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.766+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/1027435028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da3cdccf50 0x55da3cdd7c40 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.766+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/1027435028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da3cdc7de0 0x55da3cdd6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.766+0000 7ff3c5bcc740 1 --2- 172.21.15.47:0/1027435028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55da3cdc73f0 0x55da3cdd6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.766+0000 7ff3a6000640 0 client.30131 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.766+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/1027435028 >> 172.21.15.47:0/1027435028 conn(0x55da3cdc2960 msgr2=0x55da3cdc9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.766+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/1027435028 shutdown_connections 2024-03-21T11:37:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.766+0000 7ff3c5bcc740 1 -- 172.21.15.47:0/1027435028 wait complete. 2024-03-21T11:37:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.767+0000 7ff3c5bcc740 20 client.30131 trim_cache size 0 max 16384 2024-03-21T11:37:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:37:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:37:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:37:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:37:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.767+0000 7ff3c1800640 1 -- 172.21.15.47:0/3831851438 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff3a00a2f90 msgr2=0x7ff3a00a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.767+0000 7ff3c1800640 1 --2- 172.21.15.47:0/3831851438 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff3a00a2f90 0x7ff3a00a5450 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55da3cdc8640 tx=0x7ff3a800b390 comp rx=0 tx=0).stop 2024-03-21T11:37:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.767+0000 7ff3c1800640 1 -- 172.21.15.47:0/3831851438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3bc10dae0 msgr2=0x7ff3bc1a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.767+0000 7ff3c1800640 1 --2- 172.21.15.47:0/3831851438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3bc10dae0 0x7ff3bc1a5330 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4037040 tx=0x7ff3b4034670 comp rx=0 tx=0).stop 2024-03-21T11:37:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.768+0000 7ff3c1800640 1 -- 172.21.15.47:0/3831851438 shutdown_connections 2024-03-21T11:37:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.768+0000 7ff3c1800640 1 --2- 172.21.15.47:0/3831851438 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff3a00a2f90 0x7ff3a00a5450 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.768+0000 7ff3c1800640 1 --2- 172.21.15.47:0/3831851438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3bc10dae0 0x7ff3bc1a5330 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.768+0000 7ff3c1800640 1 --2- 172.21.15.47:0/3831851438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3bc0756e0 0x7ff3bc19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.768+0000 7ff3c1800640 1 --2- 172.21.15.47:0/3831851438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3bc074100 0x7ff3bc19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.768+0000 7ff3c1800640 1 -- 172.21.15.47:0/3831851438 >> 172.21.15.47:0/3831851438 conn(0x7ff3bc0fca80 msgr2=0x7ff3bc100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.768+0000 7ff3c1800640 1 -- 172.21.15.47:0/3831851438 shutdown_connections 2024-03-21T11:37:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:01.768+0000 7ff3c1800640 1 -- 172.21.15.47:0/3831851438 wait complete. 2024-03-21T11:37:02.583 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1048433 7.91 MB/sec execute 586 sec latency 179.582 ms 2024-03-21T11:37:02.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:02 smithi100 ceph-mon[36245]: pgmap v811: 129 pgs: 129 active+clean; 41 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-03-21T11:37:02.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:02 smithi102 ceph-mon[36748]: pgmap v811: 129 pgs: 129 active+clean; 41 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-03-21T11:37:03.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:02 smithi047 ceph-mon[31351]: pgmap v811: 129 pgs: 129 active+clean; 41 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-03-21T11:37:03.584 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1049573 7.90 MB/sec execute 587 sec latency 30.394 ms 2024-03-21T11:37:04.584 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1051138 7.90 MB/sec execute 588 sec latency 56.049 ms 2024-03-21T11:37:04.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:04 smithi100 ceph-mon[36245]: pgmap v812: 129 pgs: 129 active+clean; 34 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 309 op/s 2024-03-21T11:37:04.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:04 smithi102 ceph-mon[36748]: pgmap v812: 129 pgs: 129 active+clean; 34 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 309 op/s 2024-03-21T11:37:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:04 smithi047 ceph-mon[31351]: pgmap v812: 129 pgs: 129 active+clean; 34 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 309 op/s 2024-03-21T11:37:05.584 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1052016 7.90 MB/sec execute 589 sec latency 30.429 ms 2024-03-21T11:37:05.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:05 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:37:05.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:05 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:37:06.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:05 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:37:06.584 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1053402 7.89 MB/sec execute 590 sec latency 14.103 ms 2024-03-21T11:37:06.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:06 smithi100 ceph-mon[36245]: pgmap v813: 129 pgs: 129 active+clean; 34 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 239 op/s 2024-03-21T11:37:06.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:06 smithi102 ceph-mon[36748]: pgmap v813: 129 pgs: 129 active+clean; 34 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 239 op/s 2024-03-21T11:37:07.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:06 smithi047 ceph-mon[31351]: pgmap v813: 129 pgs: 129 active+clean; 34 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 239 op/s 2024-03-21T11:37:07.584 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1055193 7.90 MB/sec execute 591 sec latency 19.699 ms 2024-03-21T11:37:07.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:07 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:37:07.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:07 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:37:07.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:07 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:37:07.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:07 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:37:07.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:07 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:37:07.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:07 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:37:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:07 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:37:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:07 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:37:08.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:07 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:37:08.584 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1056096 7.89 MB/sec execute 592 sec latency 14.760 ms 2024-03-21T11:37:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:08 smithi047 ceph-mon[31351]: pgmap v814: 129 pgs: 129 active+clean; 36 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2024-03-21T11:37:09.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:08 smithi100 ceph-mon[36245]: pgmap v814: 129 pgs: 129 active+clean; 36 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2024-03-21T11:37:09.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:08 smithi102 ceph-mon[36748]: pgmap v814: 129 pgs: 129 active+clean; 36 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2024-03-21T11:37:09.584 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1057985 7.90 MB/sec execute 593 sec latency 45.611 ms 2024-03-21T11:37:10.585 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1059177 7.90 MB/sec execute 594 sec latency 15.226 ms 2024-03-21T11:37:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:10 smithi047 ceph-mon[31351]: pgmap v815: 129 pgs: 129 active+clean; 36 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-03-21T11:37:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:10 smithi100 ceph-mon[36245]: pgmap v815: 129 pgs: 129 active+clean; 36 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-03-21T11:37:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:10 smithi102 ceph-mon[36748]: pgmap v815: 129 pgs: 129 active+clean; 36 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-03-21T11:37:11.585 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1060945 7.89 MB/sec execute 595 sec latency 16.757 ms 2024-03-21T11:37:12.585 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1062446 7.90 MB/sec execute 596 sec latency 14.210 ms 2024-03-21T11:37:13.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:12 smithi047 ceph-mon[31351]: pgmap v816: 129 pgs: 129 active+clean; 38 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-03-21T11:37:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:12 smithi100 ceph-mon[36245]: pgmap v816: 129 pgs: 129 active+clean; 38 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-03-21T11:37:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:12 smithi102 ceph-mon[36748]: pgmap v816: 129 pgs: 129 active+clean; 38 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-03-21T11:37:13.585 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1063548 7.90 MB/sec execute 597 sec latency 12.255 ms 2024-03-21T11:37:14.585 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1065672 7.90 MB/sec execute 598 sec latency 11.801 ms 2024-03-21T11:37:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.944+0000 7f009f800640 1 -- 172.21.15.47:0/1079624970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009806f340 msgr2=0x7f009806fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.944+0000 7f009f800640 1 --2- 172.21.15.47:0/1079624970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009806f340 0x7f009806fbe0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0088009900 tx=0x7f008802bc30 comp rx=0 tx=0).stop 2024-03-21T11:37:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.944+0000 7f009f800640 1 -- 172.21.15.47:0/1079624970 shutdown_connections 2024-03-21T11:37:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.944+0000 7f009f800640 1 --2- 172.21.15.47:0/1079624970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009806f340 0x7f009806fbe0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.944+0000 7f009f800640 1 --2- 172.21.15.47:0/1079624970 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f009806e9a0 0x7f009806ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.944+0000 7f009f800640 1 --2- 172.21.15.47:0/1079624970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009806d760 0x7f009806db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.944+0000 7f009f800640 1 -- 172.21.15.47:0/1079624970 >> 172.21.15.47:0/1079624970 conn(0x7f009806c290 msgr2=0x7f009806c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.944+0000 7f009f800640 1 -- 172.21.15.47:0/1079624970 shutdown_connections 2024-03-21T11:37:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.944+0000 7f009f800640 1 -- 172.21.15.47:0/1079624970 wait complete. 2024-03-21T11:37:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.948+0000 7f009f800640 1 Processor -- start 2024-03-21T11:37:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.949+0000 7f009f800640 1 -- start start 2024-03-21T11:37:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.950+0000 7f009f800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f009806d760 0x7f00981a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.950+0000 7f009f800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009806e9a0 0x7f00981a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.950+0000 7f009f800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009806f340 0x7f00981adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.951+0000 7f009f800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009811c160 con 0x7f009806f340 2024-03-21T11:37:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.951+0000 7f009f800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009811bfe0 con 0x7f009806e9a0 2024-03-21T11:37:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.951+0000 7f009f800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009811c2e0 con 0x7f009806d760 2024-03-21T11:37:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.951+0000 7f009ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f009806d760 0x7f00981a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.951+0000 7f009ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f009806d760 0x7f00981a6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56822/0 (socket says 172.21.15.47:56822) 2024-03-21T11:37:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.951+0000 7f009ce00640 1 -- 172.21.15.47:0/2088585158 learned_addr learned my addr 172.21.15.47:0/2088585158 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.951+0000 7f0097e00640 1 --2- 172.21.15.47:0/2088585158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009806e9a0 0x7f00981a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.952+0000 7f009ce00640 1 -- 172.21.15.47:0/2088585158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009806e9a0 msgr2=0x7f00981a6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.952+0000 7f009ce00640 1 --2- 172.21.15.47:0/2088585158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009806e9a0 0x7f00981a6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.952+0000 7f009ce00640 1 -- 172.21.15.47:0/2088585158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009806f340 msgr2=0x7f00981adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.952+0000 7f009ce00640 1 --2- 172.21.15.47:0/2088585158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009806f340 0x7f00981adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.952+0000 7f009ce00640 1 -- 172.21.15.47:0/2088585158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00981ae210 con 0x7f009806d760 2024-03-21T11:37:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.952+0000 7f0097e00640 1 --2- 172.21.15.47:0/2088585158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009806e9a0 0x7f00981a6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:37:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.952+0000 7f009ce00640 1 --2- 172.21.15.47:0/2088585158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f009806d760 0x7f00981a6640 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f009000cce0 tx=0x7f009000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.952+0000 7f0095600640 1 -- 172.21.15.47:0/2088585158 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009000f650 con 0x7f009806d760 2024-03-21T11:37:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.953+0000 7f009f800640 1 -- 172.21.15.47:0/2088585158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00981ae500 con 0x7f009806d760 2024-03-21T11:37:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.953+0000 7f0095600640 1 -- 172.21.15.47:0/2088585158 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f009000fcd0 con 0x7f009806d760 2024-03-21T11:37:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.953+0000 7f009f800640 1 -- 172.21.15.47:0/2088585158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00981aeaa0 con 0x7f009806d760 2024-03-21T11:37:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.953+0000 7f0095600640 1 -- 172.21.15.47:0/2088585158 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0090018c10 con 0x7f009806d760 2024-03-21T11:37:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.957+0000 7f0095600640 1 -- 172.21.15.47:0/2088585158 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f009000f7f0 con 0x7f009806d760 2024-03-21T11:37:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.958+0000 7f00a2436740 1 Processor -- start 2024-03-21T11:37:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.958+0000 7f00a2436740 1 -- start start 2024-03-21T11:37:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.958+0000 7f00a2436740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed848600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.959+0000 7f00a2436740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed84d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.959+0000 7f00a2436740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed8501d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.959+0000 7f009d800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed848600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.959+0000 7f00a2436740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ed80b3b0 con 0x5560ed8481e0 2024-03-21T11:37:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.959+0000 7f00a2436740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ed7ad830 con 0x5560ed848bd0 2024-03-21T11:37:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.959+0000 7f00a2436740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ed7565d0 con 0x5560ed84dd40 2024-03-21T11:37:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.959+0000 7f0095600640 1 --2- 172.21.15.47:0/2088585158 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f007c0a3010 0x7f007c0a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.959+0000 7f009d800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed848600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59736/0 (socket says 172.21.15.47:59736) 2024-03-21T11:37:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.959+0000 7f009d800640 1 -- 172.21.15.47:0/442582494 learned_addr learned my addr 172.21.15.47:0/442582494 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.960+0000 7f009ce00640 1 --2- 172.21.15.47:0/442582494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed84d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.960+0000 7f0095600640 1 -- 172.21.15.47:0/2088585158 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f00900cfd20 con 0x7f009806d760 2024-03-21T11:37:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.960+0000 7f0097e00640 1 --2- 172.21.15.47:0/442582494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed8501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.960+0000 7f009d800640 1 --2- 172.21.15.47:0/2088585158 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f007c0a3010 0x7f007c0a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.961+0000 7f009d800640 1 -- 172.21.15.47:0/442582494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 msgr2=0x5560ed8501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.961+0000 7f009d800640 1 --2- 172.21.15.47:0/442582494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed8501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.961+0000 7f009d800640 1 -- 172.21.15.47:0/442582494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 msgr2=0x5560ed84d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.961+0000 7f009d800640 1 --2- 172.21.15.47:0/442582494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed84d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.961+0000 7f009d800640 1 -- 172.21.15.47:0/442582494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560ed76e2c0 con 0x5560ed8481e0 2024-03-21T11:37:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.961+0000 7f009d800640 1 --2- 172.21.15.47:0/442582494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed848600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0088005ab0 tx=0x7f008802cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70dc45b7575ac98e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.962+0000 7f009d800640 1 --2- 172.21.15.47:0/2088585158 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f007c0a3010 0x7f007c0a54d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0088044000 tx=0x7f0088047000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.962+0000 7f008e000640 1 -- 172.21.15.47:0/442582494 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00880035f0 con 0x5560ed8481e0 2024-03-21T11:37:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.963+0000 7f008e000640 1 -- 172.21.15.47:0/442582494 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0088003c70 con 0x5560ed8481e0 2024-03-21T11:37:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.963+0000 7f008e000640 1 -- 172.21.15.47:0/442582494 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088042b30 con 0x5560ed8481e0 2024-03-21T11:37:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.964+0000 7f00a2436740 1 -- 172.21.15.47:0/442582494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 msgr2=0x5560ed848600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.964+0000 7f00a2436740 1 --2- 172.21.15.47:0/442582494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed848600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0088005ab0 tx=0x7f008802cb40 comp rx=0 tx=0).stop 2024-03-21T11:37:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.964+0000 7f00a2436740 1 -- 172.21.15.47:0/442582494 shutdown_connections 2024-03-21T11:37:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.964+0000 7f00a2436740 1 --2- 172.21.15.47:0/442582494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed8501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.964+0000 7f00a2436740 1 --2- 172.21.15.47:0/442582494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed84d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.964+0000 7f00a2436740 1 --2- 172.21.15.47:0/442582494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed848600 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.964+0000 7f00a2436740 1 -- 172.21.15.47:0/442582494 >> 172.21.15.47:0/442582494 conn(0x5560ed843750 msgr2=0x5560ed843b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.964+0000 7f00a2436740 1 -- 172.21.15.47:0/442582494 shutdown_connections 2024-03-21T11:37:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.964+0000 7f00a2436740 1 -- 172.21.15.47:0/442582494 wait complete. 2024-03-21T11:37:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.965+0000 7f00a2436740 1 -- start start 2024-03-21T11:37:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.965+0000 7f00a2436740 1 Processor -- start 2024-03-21T11:37:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.966+0000 7f00a2436740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed8481e0 0x5560ed8523a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.966+0000 7f00a2436740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed848bd0 0x5560ed8528e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.966+0000 7f00a2436740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed859920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.966+0000 7f00a2436740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ed769a20 con 0x5560ed848bd0 2024-03-21T11:37:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.966+0000 7f00a2436740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ed7a98d0 con 0x5560ed8481e0 2024-03-21T11:37:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.966+0000 7f00a2436740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ed850c90 con 0x5560ed84dd40 2024-03-21T11:37:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.966+0000 7f00a2436740 10 client.0 _ll_register_callbacks cb 0x5560ed7a61c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.967+0000 7f009ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed8481e0 0x5560ed8523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.967+0000 7f009ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed8481e0 0x5560ed8523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60166/0 (socket says 172.21.15.47:60166) 2024-03-21T11:37:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.967+0000 7f009ce00640 1 -- 172.21.15.47:0/1524574649 learned_addr learned my addr 172.21.15.47:0/1524574649 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.967+0000 7f009d800640 1 --2- 172.21.15.47:0/1524574649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed859920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.967+0000 7f009ce00640 1 -- 172.21.15.47:0/1524574649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 msgr2=0x5560ed859920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.967+0000 7f009ce00640 1 --2- 172.21.15.47:0/1524574649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed859920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.967+0000 7f009ce00640 1 -- 172.21.15.47:0/1524574649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed848bd0 msgr2=0x5560ed8528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.967+0000 7f009ce00640 1 --2- 172.21.15.47:0/1524574649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed848bd0 0x5560ed8528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.967+0000 7f009ce00640 1 -- 172.21.15.47:0/1524574649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560ed4c2210 con 0x5560ed8481e0 2024-03-21T11:37:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.968+0000 7f009ce00640 1 --2- 172.21.15.47:0/1524574649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed8481e0 0x5560ed8523a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f009000ccb0 tx=0x7f0090067000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4a00cc8fcd249c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.968+0000 7f0082a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:37:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.968+0000 7f0082a00640 1 -- 172.21.15.47:0/1524574649 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0090094e60 con 0x5560ed8481e0 2024-03-21T11:37:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.968+0000 7f0082a00640 1 -- 172.21.15.47:0/1524574649 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f009009a3c0 con 0x5560ed8481e0 2024-03-21T11:37:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.968+0000 7f00a2436740 1 -- 172.21.15.47:0/1524574649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0088003c70 con 0x5560ed8481e0 2024-03-21T11:37:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.968+0000 7f0082a00640 1 -- 172.21.15.47:0/1524574649 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f009009c0c0 con 0x5560ed8481e0 2024-03-21T11:37:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.968+0000 7f00a2436740 10 client.30140 fetch_fsmap learned FSMap version 28 2024-03-21T11:37:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.968+0000 7f00a2436740 1 -- 172.21.15.47:0/1524574649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5560ed58e620 con 0x5560ed8481e0 2024-03-21T11:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.969+0000 7f0082a00640 1 -- 172.21.15.47:0/1524574649 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0088003c70 con 0x5560ed8481e0 2024-03-21T11:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.969+0000 7f0082a00640 1 -- 172.21.15.47:0/1524574649 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0090066bd0 con 0x5560ed8481e0 2024-03-21T11:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.970+0000 7f00a2436740 10 client.30140 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.970+0000 7f00a2436740 10 client.30140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.970+0000 7f00a2436740 20 client.30140 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.970+0000 7f00a2436740 1 --2- 172.21.15.47:0/1524574649 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5560ed85e1c0 0x5560ed8605b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.970+0000 7f00a2436740 4 client.30140 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.970+0000 7f00a2436740 1 -- 172.21.15.47:0/1524574649 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560ed7bad70 con 0x5560ed85e1c0 2024-03-21T11:37:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.970+0000 7f0097e00640 1 --2- 172.21.15.47:0/1524574649 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5560ed85e1c0 0x5560ed8605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.974+0000 7f0097e00640 1 --2- 172.21.15.47:0/1524574649 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5560ed85e1c0 0x5560ed8605b0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cbda41639ca05ac server_cookie=9bc6fbeb5a70bfe8 in_seq=0 out_seq=0 2024-03-21T11:37:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.974+0000 7f0082a00640 10 client.30140 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.977+0000 7f0082a00640 1 -- 172.21.15.47:0/1524574649 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5560ed7bad70 con 0x5560ed85e1c0 2024-03-21T11:37:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.977+0000 7f0082a00640 10 client.30140 handle_command_reply: tid=0 2024-03-21T11:37:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.977+0000 7f00a2436740 1 client.30140 shutdown 2024-03-21T11:37:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.977+0000 7f00a2436740 1 -- 172.21.15.47:0/1524574649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed8481e0 msgr2=0x5560ed8523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.977+0000 7f00a2436740 1 --2- 172.21.15.47:0/1524574649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed8481e0 0x5560ed8523a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f009000ccb0 tx=0x7f0090067000 comp rx=0 tx=0).stop 2024-03-21T11:37:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.977+0000 7f00a2436740 1 -- 172.21.15.47:0/1524574649 shutdown_connections 2024-03-21T11:37:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.977+0000 7f00a2436740 1 --2- 172.21.15.47:0/1524574649 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5560ed85e1c0 0x5560ed8605b0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.977+0000 7f00a2436740 1 --2- 172.21.15.47:0/1524574649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed859920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.977+0000 7f00a2436740 1 --2- 172.21.15.47:0/1524574649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed848bd0 0x5560ed8528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.977+0000 7f00a2436740 1 --2- 172.21.15.47:0/1524574649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed8481e0 0x5560ed8523a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.977+0000 7f0082a00640 0 client.30140 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.977+0000 7f00a2436740 1 -- 172.21.15.47:0/1524574649 >> 172.21.15.47:0/1524574649 conn(0x5560ed843750 msgr2=0x5560ed842950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.978+0000 7f00a2436740 1 -- 172.21.15.47:0/1524574649 shutdown_connections 2024-03-21T11:37:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.978+0000 7f00a2436740 1 -- 172.21.15.47:0/1524574649 wait complete. 2024-03-21T11:37:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:14.978+0000 7f00a2436740 20 client.30140 trim_cache size 0 max 16384 2024-03-21T11:37:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.001+0000 7f00a2436740 1 Processor -- start 2024-03-21T11:37:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.001+0000 7f00a2436740 1 -- start start 2024-03-21T11:37:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.001+0000 7f00a2436740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed8470c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.001+0000 7f00a2436740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed8598e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.001+0000 7f00a2436740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed851fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.001+0000 7f00a2436740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00981b8d40 con 0x5560ed8481e0 2024-03-21T11:37:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.001+0000 7f00a2436740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00981ba6c0 con 0x5560ed848bd0 2024-03-21T11:37:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.001+0000 7f00a2436740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00981b93a0 con 0x5560ed84dd40 2024-03-21T11:37:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.002+0000 7f0097e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed8598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.002+0000 7f0097e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed8598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60180/0 (socket says 172.21.15.47:60180) 2024-03-21T11:37:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.002+0000 7f0097e00640 1 -- 172.21.15.47:0/2388902416 learned_addr learned my addr 172.21.15.47:0/2388902416 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.002+0000 7f009ce00640 1 --2- 172.21.15.47:0/2388902416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed8470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.002+0000 7f009d800640 1 --2- 172.21.15.47:0/2388902416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed851fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.003+0000 7f0097e00640 1 -- 172.21.15.47:0/2388902416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 msgr2=0x5560ed851fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.003+0000 7f0097e00640 1 --2- 172.21.15.47:0/2388902416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed851fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.003+0000 7f0097e00640 1 -- 172.21.15.47:0/2388902416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 msgr2=0x5560ed8470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.003+0000 7f0097e00640 1 --2- 172.21.15.47:0/2388902416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed8470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.003+0000 7f0097e00640 1 -- 172.21.15.47:0/2388902416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560ed860920 con 0x5560ed848bd0 2024-03-21T11:37:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.003+0000 7f009ce00640 1 --2- 172.21.15.47:0/2388902416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed8470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:37:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.003+0000 7f0097e00640 1 --2- 172.21.15.47:0/2388902416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed8598e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0084000c00 tx=0x7f0084009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dab55f4e88362905 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.004+0000 7f008d600640 1 -- 172.21.15.47:0/2388902416 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084004250 con 0x5560ed848bd0 2024-03-21T11:37:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.004+0000 7f008d600640 1 -- 172.21.15.47:0/2388902416 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0084020070 con 0x5560ed848bd0 2024-03-21T11:37:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.004+0000 7f008d600640 1 -- 172.21.15.47:0/2388902416 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008401b7f0 con 0x5560ed848bd0 2024-03-21T11:37:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.005+0000 7f00a2436740 1 -- 172.21.15.47:0/2388902416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 msgr2=0x5560ed8598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.005+0000 7f00a2436740 1 --2- 172.21.15.47:0/2388902416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed8598e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0084000c00 tx=0x7f0084009530 comp rx=0 tx=0).stop 2024-03-21T11:37:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.005+0000 7f00a2436740 1 -- 172.21.15.47:0/2388902416 shutdown_connections 2024-03-21T11:37:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.005+0000 7f00a2436740 1 --2- 172.21.15.47:0/2388902416 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed851fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.005+0000 7f00a2436740 1 --2- 172.21.15.47:0/2388902416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed8598e0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.005+0000 7f00a2436740 1 --2- 172.21.15.47:0/2388902416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed8470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.005+0000 7f00a2436740 1 -- 172.21.15.47:0/2388902416 >> 172.21.15.47:0/2388902416 conn(0x5560ed843750 msgr2=0x5560ed8469b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.006+0000 7f00a2436740 1 -- 172.21.15.47:0/2388902416 shutdown_connections 2024-03-21T11:37:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.006+0000 7f00a2436740 1 -- 172.21.15.47:0/2388902416 wait complete. 2024-03-21T11:37:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.007+0000 7f00a2436740 1 -- start start 2024-03-21T11:37:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.007+0000 7f00a2436740 1 Processor -- start 2024-03-21T11:37:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.008+0000 7f00a2436740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed8578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.008+0000 7f00a2436740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed857e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.008+0000 7f00a2436740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed858d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.008+0000 7f00a2436740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00981ba060 con 0x5560ed8481e0 2024-03-21T11:37:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.008+0000 7f00a2436740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00981b9a00 con 0x5560ed848bd0 2024-03-21T11:37:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.008+0000 7f00a2436740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00981b6f00 con 0x5560ed84dd40 2024-03-21T11:37:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.008+0000 7f00a2436740 10 client.0 _ll_register_callbacks cb 0x7f0090098360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.009+0000 7f0097e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed857e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.009+0000 7f0097e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed857e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60182/0 (socket says 172.21.15.47:60182) 2024-03-21T11:37:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.009+0000 7f0097e00640 1 -- 172.21.15.47:0/3054827178 learned_addr learned my addr 172.21.15.47:0/3054827178 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.009+0000 7f009d800640 1 --2- 172.21.15.47:0/3054827178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed858d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.009+0000 7f009ce00640 1 --2- 172.21.15.47:0/3054827178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed8578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.010+0000 7f009d800640 1 -- 172.21.15.47:0/3054827178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 msgr2=0x5560ed857e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.010+0000 7f009d800640 1 --2- 172.21.15.47:0/3054827178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed857e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.010+0000 7f009d800640 1 -- 172.21.15.47:0/3054827178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 msgr2=0x5560ed8578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.011+0000 7f009d800640 1 --2- 172.21.15.47:0/3054827178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed8578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.011+0000 7f009d800640 1 -- 172.21.15.47:0/3054827178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560ed859360 con 0x5560ed84dd40 2024-03-21T11:37:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.011+0000 7f009ce00640 1 --2- 172.21.15.47:0/3054827178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed8578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:37:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.011+0000 7f009d800640 1 --2- 172.21.15.47:0/3054827178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed858d70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f008802c7b0 tx=0x7f008802c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=128917cb02ce1658 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.011+0000 7f0082a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:37:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.012+0000 7f0082a00640 1 -- 172.21.15.47:0/3054827178 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088003a00 con 0x5560ed84dd40 2024-03-21T11:37:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.012+0000 7f0082a00640 1 -- 172.21.15.47:0/3054827178 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0088003ba0 con 0x5560ed84dd40 2024-03-21T11:37:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.013+0000 7f00a2436740 1 -- 172.21.15.47:0/3054827178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0084020070 con 0x5560ed84dd40 2024-03-21T11:37:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.013+0000 7f0082a00640 1 -- 172.21.15.47:0/3054827178 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0088003da0 con 0x5560ed84dd40 2024-03-21T11:37:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.013+0000 7f00a2436740 10 client.30178 fetch_fsmap learned FSMap version 28 2024-03-21T11:37:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.013+0000 7f00a2436740 1 -- 172.21.15.47:0/3054827178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5560ed8ba290 con 0x5560ed84dd40 2024-03-21T11:37:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.014+0000 7f0082a00640 1 -- 172.21.15.47:0/3054827178 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0084020070 con 0x5560ed84dd40 2024-03-21T11:37:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.014+0000 7f0082a00640 1 -- 172.21.15.47:0/3054827178 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0088040040 con 0x5560ed84dd40 2024-03-21T11:37:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.014+0000 7f00a2436740 10 client.30178 fetch_fsmap finished waiting for FSMap version 28 2024-03-21T11:37:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.015+0000 7f00a2436740 10 client.30178 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.015+0000 7f00a2436740 20 client.30178 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.015+0000 7f00a2436740 1 --2- 172.21.15.47:0/3054827178 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5560ed8ba8b0 0x5560ed8bcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:15.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.015+0000 7f00a2436740 4 client.30178 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:37:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.015+0000 7f00a2436740 1 -- 172.21.15.47:0/3054827178 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5560ed7b50c0 con 0x5560ed8ba8b0 2024-03-21T11:37:15.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.015+0000 7f009ce00640 1 --2- 172.21.15.47:0/3054827178 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5560ed8ba8b0 0x5560ed8bcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.016+0000 7f0082a00640 10 client.30178 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.018+0000 7f009ce00640 1 --2- 172.21.15.47:0/3054827178 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5560ed8ba8b0 0x5560ed8bcca0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c2723552a66dfb server_cookie=aa7de75db5cc7599 in_seq=0 out_seq=0 2024-03-21T11:37:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:14 smithi047 ceph-mon[31351]: pgmap v817: 129 pgs: 129 active+clean; 38 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-03-21T11:37:15.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:14 smithi100 ceph-mon[36245]: pgmap v817: 129 pgs: 129 active+clean; 38 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-03-21T11:37:15.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:14 smithi102 ceph-mon[36748]: pgmap v817: 129 pgs: 129 active+clean; 38 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-03-21T11:37:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.323+0000 7f0082a00640 1 -- 172.21.15.47:0/3054827178 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5560ed7b50c0 con 0x5560ed8ba8b0 2024-03-21T11:37:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.323+0000 7f0082a00640 10 client.30178 handle_command_reply: tid=0 2024-03-21T11:37:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.323+0000 7f00a2436740 1 client.30178 shutdown 2024-03-21T11:37:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.324+0000 7f00a2436740 1 -- 172.21.15.47:0/3054827178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 msgr2=0x5560ed858d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.324+0000 7f00a2436740 1 --2- 172.21.15.47:0/3054827178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed858d70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f008802c7b0 tx=0x7f008802c8a0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.324+0000 7f00a2436740 1 -- 172.21.15.47:0/3054827178 shutdown_connections 2024-03-21T11:37:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.324+0000 7f00a2436740 1 --2- 172.21.15.47:0/3054827178 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5560ed8ba8b0 0x5560ed8bcca0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.324+0000 7f00a2436740 1 --2- 172.21.15.47:0/3054827178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560ed84dd40 0x5560ed858d70 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.324+0000 7f0082a00640 0 client.30178 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.324+0000 7f00a2436740 1 --2- 172.21.15.47:0/3054827178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5560ed848bd0 0x5560ed857e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.324+0000 7f00a2436740 1 --2- 172.21.15.47:0/3054827178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5560ed8481e0 0x5560ed8578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.324+0000 7f00a2436740 1 -- 172.21.15.47:0/3054827178 >> 172.21.15.47:0/3054827178 conn(0x5560ed843750 msgr2=0x5560ed851f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.324+0000 7f00a2436740 1 -- 172.21.15.47:0/3054827178 shutdown_connections 2024-03-21T11:37:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.324+0000 7f00a2436740 1 -- 172.21.15.47:0/3054827178 wait complete. 2024-03-21T11:37:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.324+0000 7f00a2436740 20 client.30178 trim_cache size 0 max 16384 2024-03-21T11:37:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:37:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:37:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:37:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:37:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.326+0000 7f009f800640 1 -- 172.21.15.47:0/2088585158 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f007c0a3010 msgr2=0x7f007c0a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.326+0000 7f009f800640 1 --2- 172.21.15.47:0/2088585158 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f007c0a3010 0x7f007c0a54d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0088044000 tx=0x7f0088047000 comp rx=0 tx=0).stop 2024-03-21T11:37:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.326+0000 7f009f800640 1 -- 172.21.15.47:0/2088585158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f009806d760 msgr2=0x7f00981a6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.326+0000 7f009f800640 1 --2- 172.21.15.47:0/2088585158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f009806d760 0x7f00981a6640 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f009000cce0 tx=0x7f009000a620 comp rx=0 tx=0).stop 2024-03-21T11:37:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.326+0000 7f009f800640 1 -- 172.21.15.47:0/2088585158 shutdown_connections 2024-03-21T11:37:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.327+0000 7f009f800640 1 --2- 172.21.15.47:0/2088585158 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f007c0a3010 0x7f007c0a54d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.327+0000 7f009f800640 1 --2- 172.21.15.47:0/2088585158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009806f340 0x7f00981adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.327+0000 7f009f800640 1 --2- 172.21.15.47:0/2088585158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009806e9a0 0x7f00981a6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.327+0000 7f009f800640 1 --2- 172.21.15.47:0/2088585158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f009806d760 0x7f00981a6640 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.327+0000 7f009f800640 1 -- 172.21.15.47:0/2088585158 >> 172.21.15.47:0/2088585158 conn(0x7f009806c290 msgr2=0x7f009810a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.327+0000 7f009f800640 1 -- 172.21.15.47:0/2088585158 shutdown_connections 2024-03-21T11:37:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:15.327+0000 7f009f800640 1 -- 172.21.15.47:0/2088585158 wait complete. 2024-03-21T11:37:15.585 INFO:tasks.workunit.client.0.smithi047.stdout: 1 1066161 7.89 MB/sec execute 599 sec latency 165.173 ms 2024-03-21T11:37:16.586 INFO:tasks.workunit.client.0.smithi047.stdout: 1 cleanup 600 sec 2024-03-21T11:37:16.808 INFO:tasks.workunit.client.0.smithi047.stdout: 0 cleanup 600 sec 2024-03-21T11:37:16.808 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-21T11:37:16.808 INFO:tasks.workunit.client.0.smithi047.stdout: Operation Count AvgLat MaxLat 2024-03-21T11:37:16.808 INFO:tasks.workunit.client.0.smithi047.stdout: ---------------------------------------- 2024-03-21T11:37:16.808 INFO:tasks.workunit.client.0.smithi047.stdout: NTCreateX 150813 1.392 187.273 2024-03-21T11:37:16.808 INFO:tasks.workunit.client.0.smithi047.stdout: Close 110758 0.011 1.083 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: Rename 6386 6.061 259.767 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: Unlink 30475 2.161 151.685 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: Deltree 4 88.375 126.787 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: Mkdir 2 0.043 0.043 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: Qpathinfo 136638 1.078 194.630 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: Qfileinfo 23938 0.005 0.275 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: Qfsinfo 25062 0.445 39.717 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: Sfileinfo 12292 0.094 3.036 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: Find 52815 0.366 118.591 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: WriteX 75197 0.072 72.368 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: ReadX 236183 0.038 126.288 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: LockX 490 0.935 16.502 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: UnlockX 490 0.951 19.525 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: Flush 10572 7.807 216.496 2024-03-21T11:37:16.809 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-21T11:37:16.810 INFO:tasks.workunit.client.0.smithi047.stdout:Throughput 7.89408 MB/sec 1 clients 1 procs max_latency=259.780 ms 2024-03-21T11:37:16.810 INFO:tasks.workunit.client.0.smithi047.stderr:+ dbench 10 2024-03-21T11:37:16.811 INFO:tasks.workunit.client.0.smithi047.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-21T11:37:16.811 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-21T11:37:16.812 INFO:tasks.workunit.client.0.smithi047.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-21T11:37:16.816 INFO:tasks.workunit.client.0.smithi047.stdout:9 of 10 processes prepared for launch 0 sec 2024-03-21T11:37:16.916 INFO:tasks.workunit.client.0.smithi047.stdout:10 of 10 processes prepared for launch 0 sec 2024-03-21T11:37:16.917 INFO:tasks.workunit.client.0.smithi047.stdout:releasing clients 2024-03-21T11:37:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:16 smithi100 ceph-mon[36245]: pgmap v818: 129 pgs: 129 active+clean; 40 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 224 op/s 2024-03-21T11:37:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:16 smithi102 ceph-mon[36748]: pgmap v818: 129 pgs: 129 active+clean; 40 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 224 op/s 2024-03-21T11:37:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:16 smithi047 ceph-mon[31351]: pgmap v818: 129 pgs: 129 active+clean; 40 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 224 op/s 2024-03-21T11:37:17.917 INFO:tasks.workunit.client.0.smithi047.stdout: 10 277 103.65 MB/sec warmup 1 sec latency 109.712 ms 2024-03-21T11:37:18.917 INFO:tasks.workunit.client.0.smithi047.stdout: 10 728 105.49 MB/sec warmup 2 sec latency 214.410 ms 2024-03-21T11:37:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:18 smithi100 ceph-mon[36245]: pgmap v819: 129 pgs: 129 active+clean; 37 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-03-21T11:37:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:18 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:19.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:18 smithi102 ceph-mon[36748]: pgmap v819: 129 pgs: 129 active+clean; 37 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-03-21T11:37:19.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:18 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:18 smithi047 ceph-mon[31351]: pgmap v819: 129 pgs: 129 active+clean; 37 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-03-21T11:37:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:18 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:19.917 INFO:tasks.workunit.client.0.smithi047.stdout: 10 1441 82.47 MB/sec warmup 3 sec latency 126.309 ms 2024-03-21T11:37:20.917 INFO:tasks.workunit.client.0.smithi047.stdout: 10 2326 73.36 MB/sec warmup 4 sec latency 58.962 ms 2024-03-21T11:37:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:20 smithi100 ceph-mon[36245]: pgmap v820: 129 pgs: 129 active+clean; 33 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 253 op/s 2024-03-21T11:37:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:20 smithi102 ceph-mon[36748]: pgmap v820: 129 pgs: 129 active+clean; 33 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 253 op/s 2024-03-21T11:37:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:20 smithi047 ceph-mon[31351]: pgmap v820: 129 pgs: 129 active+clean; 33 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 253 op/s 2024-03-21T11:37:21.917 INFO:tasks.workunit.client.0.smithi047.stdout: 10 3144 68.14 MB/sec warmup 5 sec latency 49.052 ms 2024-03-21T11:37:22.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:21 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:22.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:21 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:22.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:21 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:22.918 INFO:tasks.workunit.client.0.smithi047.stdout: 10 3743 62.58 MB/sec warmup 6 sec latency 53.562 ms 2024-03-21T11:37:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:22 smithi100 ceph-mon[36245]: pgmap v821: 129 pgs: 129 active+clean; 32 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2024-03-21T11:37:23.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:22 smithi102 ceph-mon[36748]: pgmap v821: 129 pgs: 129 active+clean; 32 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2024-03-21T11:37:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:22 smithi047 ceph-mon[31351]: pgmap v821: 129 pgs: 129 active+clean; 32 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 258 op/s 2024-03-21T11:37:23.917 INFO:tasks.workunit.client.0.smithi047.stdout: 10 4410 59.20 MB/sec warmup 7 sec latency 63.320 ms 2024-03-21T11:37:24.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:37:24.054 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:37:24.130 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:23 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:24.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:23 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:24.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:23 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:24.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.307+0000 7f1bec000640 1 -- 172.21.15.47:0/3111626210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1be41038e0 msgr2=0x7f1be4103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:24.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.307+0000 7f1bec000640 1 --2- 172.21.15.47:0/3111626210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1be41038e0 0x7f1be4103d00 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009860 tx=0x7f1bd802bbb0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.307+0000 7f1bec000640 1 -- 172.21.15.47:0/3111626210 shutdown_connections 2024-03-21T11:37:24.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.307+0000 7f1bec000640 1 --2- 172.21.15.47:0/3111626210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1be4105490 0x7f1be410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.307+0000 7f1bec000640 1 --2- 172.21.15.47:0/3111626210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1be4104ad0 0x7f1be4104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.307+0000 7f1bec000640 1 --2- 172.21.15.47:0/3111626210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1be41038e0 0x7f1be4103d00 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.307+0000 7f1bec000640 1 -- 172.21.15.47:0/3111626210 >> 172.21.15.47:0/3111626210 conn(0x7f1be40fee50 msgr2=0x7f1be4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:24.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.307+0000 7f1bec000640 1 -- 172.21.15.47:0/3111626210 shutdown_connections 2024-03-21T11:37:24.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.308+0000 7f1bec000640 1 -- 172.21.15.47:0/3111626210 wait complete. 2024-03-21T11:37:24.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.309+0000 7f1bec000640 1 Processor -- start 2024-03-21T11:37:24.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.310+0000 7f1bec000640 1 -- start start 2024-03-21T11:37:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.311+0000 7f1bec000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1be41038e0 0x7f1be4199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.311+0000 7f1bec000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1be4104ad0 0x7f1be419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.311+0000 7f1bec000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1be4105490 0x7f1be41a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.311+0000 7f1bec000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be410f580 con 0x7f1be4104ad0 2024-03-21T11:37:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.311+0000 7f1bec000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be410f400 con 0x7f1be4105490 2024-03-21T11:37:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.311+0000 7f1bec000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be410f700 con 0x7f1be41038e0 2024-03-21T11:37:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.311+0000 7f1bea000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1be4105490 0x7f1be41a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.311+0000 7f1bea000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1be4105490 0x7f1be41a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41510/0 (socket says 172.21.15.47:41510) 2024-03-21T11:37:24.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.311+0000 7f1bea000640 1 -- 172.21.15.47:0/2765638154 learned_addr learned my addr 172.21.15.47:0/2765638154 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:24.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.312+0000 7f1be8c00640 1 --2- 172.21.15.47:0/2765638154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1be4104ad0 0x7f1be419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.312+0000 7f1bea000640 1 -- 172.21.15.47:0/2765638154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1be41038e0 msgr2=0x7f1be4199c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T11:37:24.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.312+0000 7f1bea000640 1 --2- 172.21.15.47:0/2765638154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1be41038e0 0x7f1be4199c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.312+0000 7f1bea000640 1 -- 172.21.15.47:0/2765638154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1be4104ad0 msgr2=0x7f1be419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:24.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.312+0000 7f1bea000640 1 --2- 172.21.15.47:0/2765638154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1be4104ad0 0x7f1be419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.312+0000 7f1bea000640 1 -- 172.21.15.47:0/2765638154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be41a18c0 con 0x7f1be4105490 2024-03-21T11:37:24.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.313+0000 7f1be8c00640 1 --2- 172.21.15.47:0/2765638154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1be4104ad0 0x7f1be419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:37:24.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.314+0000 7f1bea000640 1 --2- 172.21.15.47:0/2765638154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1be4105490 0x7f1be41a1220 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00eaa0 tx=0x7f1bdc00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:24.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.314+0000 7f1be2000640 1 -- 172.21.15.47:0/2765638154 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc00cdb0 con 0x7f1be4105490 2024-03-21T11:37:24.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.314+0000 7f1be2000640 1 -- 172.21.15.47:0/2765638154 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bdc005020 con 0x7f1be4105490 2024-03-21T11:37:24.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.314+0000 7f1be2000640 1 -- 172.21.15.47:0/2765638154 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0116d0 con 0x7f1be4105490 2024-03-21T11:37:24.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.315+0000 7f1bec000640 1 -- 172.21.15.47:0/2765638154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be4075b40 con 0x7f1be4105490 2024-03-21T11:37:24.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.315+0000 7f1bec000640 1 -- 172.21.15.47:0/2765638154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be4076040 con 0x7f1be4105490 2024-03-21T11:37:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.321+0000 7f1be2000640 1 -- 172.21.15.47:0/2765638154 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1bdc005540 con 0x7f1be4105490 2024-03-21T11:37:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.323+0000 7f1be2000640 1 --2- 172.21.15.47:0/2765638154 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1bcc0a2ff0 0x7f1bcc0a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.323+0000 7f1be2000640 1 -- 172.21.15.47:0/2765638154 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1bdc0cf9f0 con 0x7f1be4105490 2024-03-21T11:37:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.324+0000 7f1be9600640 1 --2- 172.21.15.47:0/2765638154 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1bcc0a2ff0 0x7f1bcc0a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.326+0000 7f1be9600640 1 --2- 172.21.15.47:0/2765638154 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1bcc0a2ff0 0x7f1bcc0a54b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009780 tx=0x7f1bd8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.329+0000 7f1beedbf740 1 Processor -- start 2024-03-21T11:37:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.330+0000 7f1beedbf740 1 -- start start 2024-03-21T11:37:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.330+0000 7f1beedbf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9ae80 0x562af5e9b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.330+0000 7f1beedbf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5e9b870 0x562af5ea0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.331+0000 7f1beedbf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5ea09e0 0x562af5ea2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.332+0000 7f1be9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9ae80 0x562af5e9b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.332+0000 7f1be8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5e9b870 0x562af5ea0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.332+0000 7f1beedbf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5e00250 con 0x562af5ea09e0 2024-03-21T11:37:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.332+0000 7f1beedbf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5da8a30 con 0x562af5e9ae80 2024-03-21T11:37:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.332+0000 7f1beedbf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5dfc2f0 con 0x562af5e9b870 2024-03-21T11:37:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.332+0000 7f1bea000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5ea09e0 0x562af5ea2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.332+0000 7f1be8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5e9b870 0x562af5ea0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56672/0 (socket says 172.21.15.47:56672) 2024-03-21T11:37:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.332+0000 7f1be8c00640 1 -- 172.21.15.47:0/1439578757 learned_addr learned my addr 172.21.15.47:0/1439578757 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.332+0000 7f1be9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9ae80 0x562af5e9b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41514/0 (socket says 172.21.15.47:41514) 2024-03-21T11:37:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.333+0000 7f1bea000640 1 -- 172.21.15.47:0/1439578757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5e9b870 msgr2=0x562af5ea0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.333+0000 7f1bea000640 1 --2- 172.21.15.47:0/1439578757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5e9b870 0x562af5ea0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.333+0000 7f1bea000640 1 -- 172.21.15.47:0/1439578757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9ae80 msgr2=0x562af5e9b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.333+0000 7f1bea000640 1 --2- 172.21.15.47:0/1439578757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9ae80 0x562af5e9b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.334+0000 7f1bea000640 1 -- 172.21.15.47:0/1439578757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562af5d93690 con 0x562af5ea09e0 2024-03-21T11:37:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.334+0000 7f1be8c00640 1 --2- 172.21.15.47:0/1439578757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5e9b870 0x562af5ea0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.334+0000 7f1be9600640 1 --2- 172.21.15.47:0/1439578757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9ae80 0x562af5e9b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.334+0000 7f1bea000640 1 --2- 172.21.15.47:0/1439578757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5ea09e0 0x562af5ea2e70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0cd4d0 tx=0x7f1bdc09be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2515139284e5e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.335+0000 7f1bd6600640 1 -- 172.21.15.47:0/1439578757 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc01f030 con 0x562af5ea09e0 2024-03-21T11:37:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.336+0000 7f1bd6600640 1 -- 172.21.15.47:0/1439578757 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bdc09a090 con 0x562af5ea09e0 2024-03-21T11:37:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.336+0000 7f1bd6600640 1 -- 172.21.15.47:0/1439578757 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc09b440 con 0x562af5ea09e0 2024-03-21T11:37:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.337+0000 7f1beedbf740 1 -- 172.21.15.47:0/1439578757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5ea09e0 msgr2=0x562af5ea2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.337+0000 7f1beedbf740 1 --2- 172.21.15.47:0/1439578757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5ea09e0 0x562af5ea2e70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0cd4d0 tx=0x7f1bdc09be50 comp rx=0 tx=0).stop 2024-03-21T11:37:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.337+0000 7f1beedbf740 1 -- 172.21.15.47:0/1439578757 shutdown_connections 2024-03-21T11:37:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.338+0000 7f1beedbf740 1 --2- 172.21.15.47:0/1439578757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5ea09e0 0x562af5ea2e70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.338+0000 7f1beedbf740 1 --2- 172.21.15.47:0/1439578757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5e9b870 0x562af5ea0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.338+0000 7f1beedbf740 1 --2- 172.21.15.47:0/1439578757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9ae80 0x562af5e9b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.338+0000 7f1beedbf740 1 -- 172.21.15.47:0/1439578757 >> 172.21.15.47:0/1439578757 conn(0x562af5e63200 msgr2=0x562af5e967f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.338+0000 7f1beedbf740 1 -- 172.21.15.47:0/1439578757 shutdown_connections 2024-03-21T11:37:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.338+0000 7f1beedbf740 1 -- 172.21.15.47:0/1439578757 wait complete. 2024-03-21T11:37:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.339+0000 7f1beedbf740 1 -- start start 2024-03-21T11:37:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.340+0000 7f1beedbf740 1 Processor -- start 2024-03-21T11:37:24.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.340+0000 7f1beedbf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9ae80 0x562af5ea4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.340+0000 7f1beedbf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5e9b870 0x562af5ea53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.340+0000 7f1beedbf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5ea09e0 0x562af5eac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.340+0000 7f1beedbf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5ea3930 con 0x562af5e9b870 2024-03-21T11:37:24.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.340+0000 7f1beedbf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5dbbe80 con 0x562af5e9ae80 2024-03-21T11:37:24.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.340+0000 7f1beedbf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5ea3ab0 con 0x562af5ea09e0 2024-03-21T11:37:24.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.340+0000 7f1beedbf740 10 client.0 _ll_register_callbacks cb 0x562af5dce130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:24.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.341+0000 7f1be8c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5e9b870 0x562af5ea53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.341+0000 7f1be8c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5e9b870 0x562af5ea53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58882/0 (socket says 172.21.15.47:58882) 2024-03-21T11:37:24.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.341+0000 7f1be8c00640 1 -- 172.21.15.47:0/4192010967 learned_addr learned my addr 172.21.15.47:0/4192010967 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.341+0000 7f1be9600640 1 --2- 172.21.15.47:0/4192010967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9ae80 0x562af5ea4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.342+0000 7f1bea000640 1 --2- 172.21.15.47:0/4192010967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5ea09e0 0x562af5eac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.342+0000 7f1be9600640 1 -- 172.21.15.47:0/4192010967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5ea09e0 msgr2=0x562af5eac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.342+0000 7f1be9600640 1 --2- 172.21.15.47:0/4192010967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5ea09e0 0x562af5eac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.342+0000 7f1be9600640 1 -- 172.21.15.47:0/4192010967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5e9b870 msgr2=0x562af5ea53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.342+0000 7f1be9600640 1 --2- 172.21.15.47:0/4192010967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5e9b870 0x562af5ea53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.342+0000 7f1be9600640 1 -- 172.21.15.47:0/4192010967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562af5bde7d0 con 0x562af5e9ae80 2024-03-21T11:37:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.343+0000 7f1be9600640 1 --2- 172.21.15.47:0/4192010967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9ae80 0x562af5ea4eb0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8034ab0 tx=0x7f1bd802bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b696102391d302 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.343+0000 7f1bcaa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:37:24.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.345+0000 7f1bcaa00640 1 -- 172.21.15.47:0/4192010967 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8039040 con 0x562af5e9ae80 2024-03-21T11:37:24.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.345+0000 7f1bcaa00640 1 -- 172.21.15.47:0/4192010967 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1bd8004250 con 0x562af5e9ae80 2024-03-21T11:37:24.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.345+0000 7f1beedbf740 1 -- 172.21.15.47:0/4192010967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bdc09a090 con 0x562af5e9ae80 2024-03-21T11:37:24.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.346+0000 7f1bcaa00640 1 -- 172.21.15.47:0/4192010967 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bd8004450 con 0x562af5e9ae80 2024-03-21T11:37:24.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.346+0000 7f1beedbf740 10 client.30158 fetch_fsmap learned FSMap version 32 2024-03-21T11:37:24.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.346+0000 7f1beedbf740 1 -- 172.21.15.47:0/4192010967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562af5b143f0 con 0x562af5e9ae80 2024-03-21T11:37:24.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.346+0000 7f1bcaa00640 1 -- 172.21.15.47:0/4192010967 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bdc09a090 con 0x562af5e9ae80 2024-03-21T11:37:24.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.347+0000 7f1bcaa00640 1 -- 172.21.15.47:0/4192010967 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f1bd803c030 con 0x562af5e9ae80 2024-03-21T11:37:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.347+0000 7f1beedbf740 10 client.30158 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T11:37:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.347+0000 7f1beedbf740 10 client.30158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.347+0000 7f1beedbf740 20 client.30158 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.347+0000 7f1beedbf740 1 --2- 172.21.15.47:0/4192010967 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562af5ec2200 0x562af5ec4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.347+0000 7f1beedbf740 4 client.30158 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:37:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.347+0000 7f1beedbf740 1 -- 172.21.15.47:0/4192010967 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562af5e0d790 con 0x562af5ec2200 2024-03-21T11:37:24.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.347+0000 7f1be8c00640 1 --2- 172.21.15.47:0/4192010967 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562af5ec2200 0x562af5ec4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.350+0000 7f1be8c00640 1 --2- 172.21.15.47:0/4192010967 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562af5ec2200 0x562af5ec4680 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e17e3b97c6ff79ba server_cookie=93b43765ea739889 in_seq=0 out_seq=0 2024-03-21T11:37:24.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.351+0000 7f1bcaa00640 10 client.30158 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:24.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.354+0000 7f1bcaa00640 1 -- 172.21.15.47:0/4192010967 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562af5e0d790 con 0x562af5ec2200 2024-03-21T11:37:24.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.354+0000 7f1bcaa00640 10 client.30158 handle_command_reply: tid=0 2024-03-21T11:37:24.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.354+0000 7f1beedbf740 1 client.30158 shutdown 2024-03-21T11:37:24.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.354+0000 7f1beedbf740 1 -- 172.21.15.47:0/4192010967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9ae80 msgr2=0x562af5ea4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:24.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.354+0000 7f1beedbf740 1 --2- 172.21.15.47:0/4192010967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9ae80 0x562af5ea4eb0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8034ab0 tx=0x7f1bd802bbb0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.354+0000 7f1beedbf740 1 -- 172.21.15.47:0/4192010967 shutdown_connections 2024-03-21T11:37:24.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.354+0000 7f1beedbf740 1 --2- 172.21.15.47:0/4192010967 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562af5ec2200 0x562af5ec4680 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.355+0000 7f1beedbf740 1 --2- 172.21.15.47:0/4192010967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5ea09e0 0x562af5eac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.355+0000 7f1beedbf740 1 --2- 172.21.15.47:0/4192010967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5e9b870 0x562af5ea53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.355+0000 7f1beedbf740 1 --2- 172.21.15.47:0/4192010967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9ae80 0x562af5ea4eb0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.355+0000 7f1beedbf740 1 -- 172.21.15.47:0/4192010967 >> 172.21.15.47:0/4192010967 conn(0x562af5e63200 msgr2=0x562af5e95a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:24.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.355+0000 7f1bcaa00640 0 client.30158 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:24.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.355+0000 7f1beedbf740 1 -- 172.21.15.47:0/4192010967 shutdown_connections 2024-03-21T11:37:24.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.355+0000 7f1beedbf740 1 -- 172.21.15.47:0/4192010967 wait complete. 2024-03-21T11:37:24.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.355+0000 7f1beedbf740 20 client.30158 trim_cache size 0 max 16384 2024-03-21T11:37:24.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.387+0000 7f1beedbf740 1 Processor -- start 2024-03-21T11:37:24.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.388+0000 7f1beedbf740 1 -- start start 2024-03-21T11:37:24.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.389+0000 7f1beedbf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5e9ae80 0x562af5ea51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.389+0000 7f1beedbf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9b870 0x562af5eac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.389+0000 7f1beedbf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5ea09e0 0x562af5e95200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.389+0000 7f1beedbf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be40715c0 con 0x562af5e9ae80 2024-03-21T11:37:24.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.389+0000 7f1beedbf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be4073060 con 0x562af5e9b870 2024-03-21T11:37:24.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.389+0000 7f1beedbf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be4071c20 con 0x562af5ea09e0 2024-03-21T11:37:24.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.389+0000 7f1be8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9b870 0x562af5eac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.389+0000 7f1be8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9b870 0x562af5eac3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41534/0 (socket says 172.21.15.47:41534) 2024-03-21T11:37:24.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.389+0000 7f1be8c00640 1 -- 172.21.15.47:0/1056141830 learned_addr learned my addr 172.21.15.47:0/1056141830 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:24.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.390+0000 7f1be9600640 1 --2- 172.21.15.47:0/1056141830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5e9ae80 0x562af5ea51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.390+0000 7f1be8c00640 1 -- 172.21.15.47:0/1056141830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5ea09e0 msgr2=0x562af5e95200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:24.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.390+0000 7f1bea000640 1 --2- 172.21.15.47:0/1056141830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5ea09e0 0x562af5e95200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.390+0000 7f1be8c00640 1 --2- 172.21.15.47:0/1056141830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5ea09e0 0x562af5e95200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.390+0000 7f1be8c00640 1 -- 172.21.15.47:0/1056141830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5e9ae80 msgr2=0x562af5ea51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:24.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.390+0000 7f1be8c00640 1 --2- 172.21.15.47:0/1056141830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5e9ae80 0x562af5ea51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.390+0000 7f1be8c00640 1 -- 172.21.15.47:0/1056141830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562af5ad40f0 con 0x562af5e9b870 2024-03-21T11:37:24.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.390+0000 7f1be9600640 1 --2- 172.21.15.47:0/1056141830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5e9ae80 0x562af5ea51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:37:24.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.390+0000 7f1bea000640 1 --2- 172.21.15.47:0/1056141830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5ea09e0 0x562af5e95200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:37:24.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.391+0000 7f1be8c00640 1 --2- 172.21.15.47:0/1056141830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9b870 0x562af5eac3c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000b490 tx=0x7f1bd001fb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c6ff505a2c05b90 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:24.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.391+0000 7f1bd5c00640 1 -- 172.21.15.47:0/1056141830 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0013990 con 0x562af5e9b870 2024-03-21T11:37:24.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.391+0000 7f1bd5c00640 1 -- 172.21.15.47:0/1056141830 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bd0013b30 con 0x562af5e9b870 2024-03-21T11:37:24.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.391+0000 7f1bd5c00640 1 -- 172.21.15.47:0/1056141830 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0010630 con 0x562af5e9b870 2024-03-21T11:37:24.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.392+0000 7f1beedbf740 1 -- 172.21.15.47:0/1056141830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9b870 msgr2=0x562af5eac3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:24.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.392+0000 7f1beedbf740 1 --2- 172.21.15.47:0/1056141830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9b870 0x562af5eac3c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000b490 tx=0x7f1bd001fb20 comp rx=0 tx=0).stop 2024-03-21T11:37:24.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.392+0000 7f1beedbf740 1 -- 172.21.15.47:0/1056141830 shutdown_connections 2024-03-21T11:37:24.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.392+0000 7f1beedbf740 1 --2- 172.21.15.47:0/1056141830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5ea09e0 0x562af5e95200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.392+0000 7f1beedbf740 1 --2- 172.21.15.47:0/1056141830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9b870 0x562af5eac3c0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.392+0000 7f1beedbf740 1 --2- 172.21.15.47:0/1056141830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5e9ae80 0x562af5ea51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.392+0000 7f1beedbf740 1 -- 172.21.15.47:0/1056141830 >> 172.21.15.47:0/1056141830 conn(0x562af5e63200 msgr2=0x562af5ea4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:24.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.393+0000 7f1beedbf740 1 -- 172.21.15.47:0/1056141830 shutdown_connections 2024-03-21T11:37:24.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.393+0000 7f1beedbf740 1 -- 172.21.15.47:0/1056141830 wait complete. 2024-03-21T11:37:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.393+0000 7f1beedbf740 1 -- start start 2024-03-21T11:37:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.394+0000 7f1beedbf740 1 Processor -- start 2024-03-21T11:37:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.394+0000 7f1beedbf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5e9ae80 0x562af5ea9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.394+0000 7f1beedbf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9b870 0x562af5ea9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.394+0000 7f1beedbf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5ea09e0 0x562af5f1bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.394+0000 7f1be9600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5e9ae80 0x562af5ea9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.394+0000 7f1be9600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5e9ae80 0x562af5ea9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56694/0 (socket says 172.21.15.47:56694) 2024-03-21T11:37:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.395+0000 7f1beedbf740 1 -- 172.21.15.47:0/2544228654 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be4072940 con 0x562af5ea09e0 2024-03-21T11:37:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.395+0000 7f1beedbf740 1 -- 172.21.15.47:0/2544228654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be40722e0 con 0x562af5e9b870 2024-03-21T11:37:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.395+0000 7f1beedbf740 1 -- 172.21.15.47:0/2544228654 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be406f7a0 con 0x562af5e9ae80 2024-03-21T11:37:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.395+0000 7f1beedbf740 10 client.0 _ll_register_callbacks cb 0x7f1bd80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.395+0000 7f1be9600640 1 -- 172.21.15.47:0/2544228654 learned_addr learned my addr 172.21.15.47:0/2544228654 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.395+0000 7f1be8c00640 1 --2- 172.21.15.47:0/2544228654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9b870 0x562af5ea9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.396+0000 7f1bea000640 1 --2- 172.21.15.47:0/2544228654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5ea09e0 0x562af5f1bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.396+0000 7f1be8c00640 1 -- 172.21.15.47:0/2544228654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5e9ae80 msgr2=0x562af5ea9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.396+0000 7f1be8c00640 1 --2- 172.21.15.47:0/2544228654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5e9ae80 0x562af5ea9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.396+0000 7f1be8c00640 1 -- 172.21.15.47:0/2544228654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5ea09e0 msgr2=0x562af5f1bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.396+0000 7f1be8c00640 1 --2- 172.21.15.47:0/2544228654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5ea09e0 0x562af5f1bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.396+0000 7f1be8c00640 1 -- 172.21.15.47:0/2544228654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562af5eac6c0 con 0x562af5e9b870 2024-03-21T11:37:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.396+0000 7f1bea000640 1 --2- 172.21.15.47:0/2544228654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5ea09e0 0x562af5f1bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:37:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.397+0000 7f1be8c00640 1 --2- 172.21.15.47:0/2544228654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9b870 0x562af5ea9660 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x562af5ec0900 tx=0x7f1bd000bd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=440e8a6f1d0c1d7e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:24.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.397+0000 7f1bcaa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:37:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.397+0000 7f1bcaa00640 1 -- 172.21.15.47:0/2544228654 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0005240 con 0x562af5e9b870 2024-03-21T11:37:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.397+0000 7f1beedbf740 1 -- 172.21.15.47:0/2544228654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bd0013b30 con 0x562af5e9b870 2024-03-21T11:37:24.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.397+0000 7f1bcaa00640 1 -- 172.21.15.47:0/2544228654 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1bd00053e0 con 0x562af5e9b870 2024-03-21T11:37:24.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.399+0000 7f1bcaa00640 1 -- 172.21.15.47:0/2544228654 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bd00055e0 con 0x562af5e9b870 2024-03-21T11:37:24.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.400+0000 7f1beedbf740 10 client.30164 fetch_fsmap learned FSMap version 32 2024-03-21T11:37:24.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.400+0000 7f1beedbf740 1 -- 172.21.15.47:0/2544228654 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562af5f1d0e0 con 0x562af5e9b870 2024-03-21T11:37:24.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.400+0000 7f1bcaa00640 1 -- 172.21.15.47:0/2544228654 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bd0013b30 con 0x562af5e9b870 2024-03-21T11:37:24.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.401+0000 7f1bcaa00640 1 -- 172.21.15.47:0/2544228654 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f1bd0004c10 con 0x562af5e9b870 2024-03-21T11:37:24.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.402+0000 7f1beedbf740 10 client.30164 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T11:37:24.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.402+0000 7f1beedbf740 10 client.30164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:24.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.402+0000 7f1beedbf740 20 client.30164 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:24.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.402+0000 7f1beedbf740 1 --2- 172.21.15.47:0/2544228654 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562af5f1d9e0 0x562af5f1fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.402+0000 7f1beedbf740 4 client.30164 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:37:24.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.402+0000 7f1beedbf740 1 -- 172.21.15.47:0/2544228654 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562af5e07ae0 con 0x562af5f1d9e0 2024-03-21T11:37:24.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.403+0000 7f1be9600640 1 --2- 172.21.15.47:0/2544228654 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562af5f1d9e0 0x562af5f1fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.404+0000 7f1be9600640 1 --2- 172.21.15.47:0/2544228654 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562af5f1d9e0 0x562af5f1fe00 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8adec61c23ff97d server_cookie=a5037713f8f25e3c in_seq=0 out_seq=0 2024-03-21T11:37:24.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.404+0000 7f1bcaa00640 10 client.30164 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:24.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.410+0000 7f1bcaa00640 1 -- 172.21.15.47:0/2544228654 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562af5e07ae0 con 0x562af5f1d9e0 2024-03-21T11:37:24.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.410+0000 7f1bcaa00640 10 client.30164 handle_command_reply: tid=0 2024-03-21T11:37:24.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.410+0000 7f1beedbf740 1 client.30164 shutdown 2024-03-21T11:37:24.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.410+0000 7f1beedbf740 1 -- 172.21.15.47:0/2544228654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9b870 msgr2=0x562af5ea9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:24.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.410+0000 7f1beedbf740 1 --2- 172.21.15.47:0/2544228654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9b870 0x562af5ea9660 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x562af5ec0900 tx=0x7f1bd000bd20 comp rx=0 tx=0).stop 2024-03-21T11:37:24.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.412+0000 7f1beedbf740 1 -- 172.21.15.47:0/2544228654 shutdown_connections 2024-03-21T11:37:24.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.412+0000 7f1beedbf740 1 --2- 172.21.15.47:0/2544228654 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562af5f1d9e0 0x562af5f1fe00 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.412+0000 7f1beedbf740 1 --2- 172.21.15.47:0/2544228654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562af5ea09e0 0x562af5f1bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.412+0000 7f1beedbf740 1 --2- 172.21.15.47:0/2544228654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562af5e9b870 0x562af5ea9660 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.412+0000 7f1beedbf740 1 --2- 172.21.15.47:0/2544228654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562af5e9ae80 0x562af5ea9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.412+0000 7f1bcaa00640 0 client.30164 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.412+0000 7f1beedbf740 1 -- 172.21.15.47:0/2544228654 >> 172.21.15.47:0/2544228654 conn(0x562af5e63200 msgr2=0x562af5e95200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.412+0000 7f1beedbf740 1 -- 172.21.15.47:0/2544228654 shutdown_connections 2024-03-21T11:37:24.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.414+0000 7f1beedbf740 1 -- 172.21.15.47:0/2544228654 wait complete. 2024-03-21T11:37:24.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.414+0000 7f1beedbf740 20 client.30164 trim_cache size 0 max 16384 2024-03-21T11:37:24.415 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:37:24.415 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:37:24.415 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:37:24.415 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:37:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.415+0000 7f1bec000640 1 -- 172.21.15.47:0/2765638154 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1bcc0a2ff0 msgr2=0x7f1bcc0a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.415+0000 7f1bec000640 1 --2- 172.21.15.47:0/2765638154 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1bcc0a2ff0 0x7f1bcc0a54b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009780 tx=0x7f1bd8000bc0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.415+0000 7f1bec000640 1 -- 172.21.15.47:0/2765638154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1be4105490 msgr2=0x7f1be41a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.415+0000 7f1bec000640 1 --2- 172.21.15.47:0/2765638154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1be4105490 0x7f1be41a1220 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00eaa0 tx=0x7f1bdc00ef60 comp rx=0 tx=0).stop 2024-03-21T11:37:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.416+0000 7f1bec000640 1 -- 172.21.15.47:0/2765638154 shutdown_connections 2024-03-21T11:37:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.416+0000 7f1bec000640 1 --2- 172.21.15.47:0/2765638154 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1bcc0a2ff0 0x7f1bcc0a54b0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.416+0000 7f1bec000640 1 --2- 172.21.15.47:0/2765638154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1be4105490 0x7f1be41a1220 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.416+0000 7f1bec000640 1 --2- 172.21.15.47:0/2765638154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1be4104ad0 0x7f1be419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.416+0000 7f1bec000640 1 --2- 172.21.15.47:0/2765638154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1be41038e0 0x7f1be4199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.416+0000 7f1bec000640 1 -- 172.21.15.47:0/2765638154 >> 172.21.15.47:0/2765638154 conn(0x7f1be40fee50 msgr2=0x7f1be4107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.416+0000 7f1bec000640 1 -- 172.21.15.47:0/2765638154 shutdown_connections 2024-03-21T11:37:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.416+0000 7f1bec000640 1 -- 172.21.15.47:0/2765638154 wait complete. 2024-03-21T11:37:24.441 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:37:24.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:37:24.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.693+0000 7ff009a00640 1 -- 172.21.15.47:0/4152234515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff004103260 msgr2=0x7ff004109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:24.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.693+0000 7ff009a00640 1 --2- 172.21.15.47:0/4152234515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff004103260 0x7ff004109af0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7feffc009be0 tx=0x7feffc02bef0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.693+0000 7ff009a00640 1 -- 172.21.15.47:0/4152234515 shutdown_connections 2024-03-21T11:37:24.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.693+0000 7ff009a00640 1 --2- 172.21.15.47:0/4152234515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff004103260 0x7ff004109af0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.693+0000 7ff009a00640 1 --2- 172.21.15.47:0/4152234515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0041028a0 0x7ff004102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.693+0000 7ff009a00640 1 --2- 172.21.15.47:0/4152234515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0041016b0 0x7ff004101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.693+0000 7ff009a00640 1 -- 172.21.15.47:0/4152234515 >> 172.21.15.47:0/4152234515 conn(0x7ff0040fcc80 msgr2=0x7ff0040ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:24.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.693+0000 7ff009a00640 1 -- 172.21.15.47:0/4152234515 shutdown_connections 2024-03-21T11:37:24.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.694+0000 7ff009a00640 1 -- 172.21.15.47:0/4152234515 wait complete. 2024-03-21T11:37:24.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.695+0000 7ff009a00640 1 Processor -- start 2024-03-21T11:37:24.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.695+0000 7ff009a00640 1 -- start start 2024-03-21T11:37:24.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.695+0000 7ff009a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0041016b0 0x7ff004075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.695+0000 7ff009a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0041028a0 0x7ff004073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.695+0000 7ff009a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004103260 0x7ff004073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.695+0000 7ff009a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00410d360 con 0x7ff0041016b0 2024-03-21T11:37:24.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.695+0000 7ff009a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00410d1e0 con 0x7ff0041028a0 2024-03-21T11:37:24.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.695+0000 7ff009a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00410d4e0 con 0x7ff004103260 2024-03-21T11:37:24.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.695+0000 7ff003400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004103260 0x7ff004073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.696+0000 7ff002000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0041028a0 0x7ff004073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.695+0000 7ff002a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0041016b0 0x7ff004075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.696+0000 7ff003400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004103260 0x7ff004073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56708/0 (socket says 172.21.15.47:56708) 2024-03-21T11:37:24.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.696+0000 7ff003400640 1 -- 172.21.15.47:0/3105749636 learned_addr learned my addr 172.21.15.47:0/3105749636 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:24.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.696+0000 7ff003400640 1 -- 172.21.15.47:0/3105749636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0041028a0 msgr2=0x7ff004073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:24.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.696+0000 7ff003400640 1 --2- 172.21.15.47:0/3105749636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0041028a0 0x7ff004073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.696+0000 7ff003400640 1 -- 172.21.15.47:0/3105749636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0041016b0 msgr2=0x7ff004075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:24.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.696+0000 7ff003400640 1 --2- 172.21.15.47:0/3105749636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0041016b0 0x7ff004075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:24.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.696+0000 7ff003400640 1 -- 172.21.15.47:0/3105749636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff004074860 con 0x7ff004103260 2024-03-21T11:37:24.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.696+0000 7ff002a00640 1 --2- 172.21.15.47:0/3105749636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0041016b0 0x7ff004075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:24.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.696+0000 7ff002000640 1 --2- 172.21.15.47:0/3105749636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0041028a0 0x7ff004073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:24.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.696+0000 7ff003400640 1 --2- 172.21.15.47:0/3105749636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004103260 0x7ff004073fd0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7feffc036040 tx=0x7feffc0337b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:24.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.697+0000 7feffb400640 1 -- 172.21.15.47:0/3105749636 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc043020 con 0x7ff004103260 2024-03-21T11:37:24.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.697+0000 7feffb400640 1 -- 172.21.15.47:0/3105749636 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feffc02ca60 con 0x7ff004103260 2024-03-21T11:37:24.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.697+0000 7feffb400640 1 -- 172.21.15.47:0/3105749636 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc034690 con 0x7ff004103260 2024-03-21T11:37:24.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.697+0000 7ff009a00640 1 -- 172.21.15.47:0/3105749636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0041a1a70 con 0x7ff004103260 2024-03-21T11:37:24.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.697+0000 7ff009a00640 1 -- 172.21.15.47:0/3105749636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff004197730 con 0x7ff004103260 2024-03-21T11:37:24.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.698+0000 7ff009a00640 1 -- 172.21.15.47:0/3105749636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff004108970 con 0x7ff004103260 2024-03-21T11:37:24.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.701+0000 7feffb400640 1 -- 172.21.15.47:0/3105749636 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7feffc046020 con 0x7ff004103260 2024-03-21T11:37:24.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.702+0000 7feffb400640 1 --2- 172.21.15.47:0/3105749636 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fefe80a2fa0 0x7fefe80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:24.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.702+0000 7feffb400640 1 -- 172.21.15.47:0/3105749636 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7feffc0f2550 con 0x7ff004103260 2024-03-21T11:37:24.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.702+0000 7ff002a00640 1 --2- 172.21.15.47:0/3105749636 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fefe80a2fa0 0x7fefe80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:24.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.703+0000 7ff002a00640 1 --2- 172.21.15.47:0/3105749636 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fefe80a2fa0 0x7fefe80a5460 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff004102700 tx=0x7feff4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:24.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:24.709+0000 7feffb400640 1 -- 172.21.15.47:0/3105749636 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feffc04d020 con 0x7ff004103260 2024-03-21T11:37:24.917 INFO:tasks.workunit.client.0.smithi047.stdout: 10 4925 55.53 MB/sec warmup 8 sec latency 74.529 ms 2024-03-21T11:37:25.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.125+0000 7ff009a00640 1 -- 172.21.15.47:0/3105749636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff00402d070 con 0x7ff004103260 2024-03-21T11:37:25.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.126+0000 7feffb400640 1 -- 172.21.15.47:0/3105749636 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11483 (secure 0 0 0) 0x7feffc0bef90 con 0x7ff004103260 2024-03-21T11:37:25.127 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:37:25.128 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:37:23.116658+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:37:25.128 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T11:37:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.130+0000 7ff009a00640 1 -- 172.21.15.47:0/3105749636 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fefe80a2fa0 msgr2=0x7fefe80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.130+0000 7ff009a00640 1 --2- 172.21.15.47:0/3105749636 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fefe80a2fa0 0x7fefe80a5460 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff004102700 tx=0x7feff4009260 comp rx=0 tx=0).stop 2024-03-21T11:37:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.130+0000 7ff009a00640 1 -- 172.21.15.47:0/3105749636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004103260 msgr2=0x7ff004073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.130+0000 7ff009a00640 1 --2- 172.21.15.47:0/3105749636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004103260 0x7ff004073fd0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7feffc036040 tx=0x7feffc0337b0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.131+0000 7ff009a00640 1 -- 172.21.15.47:0/3105749636 shutdown_connections 2024-03-21T11:37:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.131+0000 7ff009a00640 1 --2- 172.21.15.47:0/3105749636 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fefe80a2fa0 0x7fefe80a5460 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.131+0000 7ff009a00640 1 --2- 172.21.15.47:0/3105749636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004103260 0x7ff004073fd0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.131+0000 7ff009a00640 1 --2- 172.21.15.47:0/3105749636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0041028a0 0x7ff004073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.131+0000 7ff009a00640 1 --2- 172.21.15.47:0/3105749636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0041016b0 0x7ff004075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.131+0000 7ff009a00640 1 -- 172.21.15.47:0/3105749636 >> 172.21.15.47:0/3105749636 conn(0x7ff0040fcc80 msgr2=0x7ff004105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.132+0000 7ff009a00640 1 -- 172.21.15.47:0/3105749636 shutdown_connections 2024-03-21T11:37:25.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.134+0000 7ff009a00640 1 -- 172.21.15.47:0/3105749636 wait complete. 2024-03-21T11:37:25.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:24 smithi100 ceph-mon[36245]: pgmap v822: 129 pgs: 129 active+clean; 36 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 375 op/s 2024-03-21T11:37:25.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:24 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:25.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:24 smithi047 ceph-mon[31351]: pgmap v822: 129 pgs: 129 active+clean; 36 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 375 op/s 2024-03-21T11:37:25.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:24 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:25.162 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:37:25.162 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:37:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:24 smithi102 ceph-mon[36748]: pgmap v822: 129 pgs: 129 active+clean; 36 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 375 op/s 2024-03-21T11:37:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:24 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:25.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.392+0000 7f4061c00640 1 -- 172.21.15.47:0/4201600937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f405c104930 msgr2=0x7f405c106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.392+0000 7f4061c00640 1 --2- 172.21.15.47:0/4201600937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f405c104930 0x7f405c106d20 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f4044009900 tx=0x7f404402bc40 comp rx=0 tx=0).stop 2024-03-21T11:37:25.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.392+0000 7f4061c00640 1 -- 172.21.15.47:0/4201600937 shutdown_connections 2024-03-21T11:37:25.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.392+0000 7f4061c00640 1 --2- 172.21.15.47:0/4201600937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f405c107260 0x7f405c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.392+0000 7f4061c00640 1 --2- 172.21.15.47:0/4201600937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f405c104930 0x7f405c106d20 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.392+0000 7f4061c00640 1 --2- 172.21.15.47:0/4201600937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f405c102000 0x7f405c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.392+0000 7f4061c00640 1 -- 172.21.15.47:0/4201600937 >> 172.21.15.47:0/4201600937 conn(0x7f405c0fba50 msgr2=0x7f405c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.392+0000 7f4061c00640 1 -- 172.21.15.47:0/4201600937 shutdown_connections 2024-03-21T11:37:25.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.393+0000 7f4061c00640 1 -- 172.21.15.47:0/4201600937 wait complete. 2024-03-21T11:37:25.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.394+0000 7f4061c00640 1 Processor -- start 2024-03-21T11:37:25.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.394+0000 7f4061c00640 1 -- start start 2024-03-21T11:37:25.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.394+0000 7f4061c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f405c102000 0x7f405c101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.394+0000 7f4061c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f405c104930 0x7f405c0fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.394+0000 7f4061c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f405c107260 0x7f405c1004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.394+0000 7f4061c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c10d260 con 0x7f405c102000 2024-03-21T11:37:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.394+0000 7f4061c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c10d0e0 con 0x7f405c107260 2024-03-21T11:37:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.394+0000 7f4061c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c10d3e0 con 0x7f405c104930 2024-03-21T11:37:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.395+0000 7f4053e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f405c104930 0x7f405c0fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.395+0000 7f4061200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f405c107260 0x7f405c1004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.395+0000 7f4053e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f405c104930 0x7f405c0fffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56732/0 (socket says 172.21.15.47:56732) 2024-03-21T11:37:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.395+0000 7f4053e00640 1 -- 172.21.15.47:0/1486049942 learned_addr learned my addr 172.21.15.47:0/1486049942 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:25.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.395+0000 7f405be00640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f405c102000 0x7f405c101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.395+0000 7f4061200640 1 -- 172.21.15.47:0/1486049942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f405c104930 msgr2=0x7f405c0fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.395+0000 7f4061200640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f405c104930 0x7f405c0fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.395+0000 7f4061200640 1 -- 172.21.15.47:0/1486049942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f405c102000 msgr2=0x7f405c101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.395+0000 7f4061200640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f405c102000 0x7f405c101970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.395+0000 7f4061200640 1 -- 172.21.15.47:0/1486049942 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c100d80 con 0x7f405c107260 2024-03-21T11:37:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.395+0000 7f405be00640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f405c102000 0x7f405c101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:37:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.395+0000 7f4053e00640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f405c104930 0x7f405c0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.396+0000 7f4061200640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f405c107260 0x7f405c1004f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f4054009fd0 tx=0x7f405400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.396+0000 7f4059600640 1 -- 172.21.15.47:0/1486049942 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405400ce20 con 0x7f405c107260 2024-03-21T11:37:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.396+0000 7f4059600640 1 -- 172.21.15.47:0/1486049942 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4054005020 con 0x7f405c107260 2024-03-21T11:37:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.396+0000 7f4059600640 1 -- 172.21.15.47:0/1486049942 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054010720 con 0x7f405c107260 2024-03-21T11:37:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.396+0000 7f4061c00640 1 -- 172.21.15.47:0/1486049942 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405c1a19e0 con 0x7f405c107260 2024-03-21T11:37:25.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.396+0000 7f4061c00640 1 -- 172.21.15.47:0/1486049942 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405c1a1eb0 con 0x7f405c107260 2024-03-21T11:37:25.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.400+0000 7f4059600640 1 -- 172.21.15.47:0/1486049942 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4054005540 con 0x7f405c107260 2024-03-21T11:37:25.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.401+0000 7f4065e48740 1 Processor -- start 2024-03-21T11:37:25.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.401+0000 7f4065e48740 1 -- start start 2024-03-21T11:37:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.401+0000 7f4065e48740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823ba9f0 0x55f3823bae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.401+0000 7f4065e48740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823bbbc0 0x55f3823bc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.401+0000 7f4065e48740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 0x55f3823c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.401+0000 7f4065e48740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38231f2c0 con 0x55f3823ba9f0 2024-03-21T11:37:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.401+0000 7f4065e48740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3822c7c90 con 0x55f3823bbbc0 2024-03-21T11:37:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.401+0000 7f4065e48740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38231b360 con 0x55f3823bc580 2024-03-21T11:37:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.401+0000 7f405be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823bbbc0 0x55f3823bc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.401+0000 7f4053e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823ba9f0 0x55f3823bae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.401+0000 7f405be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823bbbc0 0x55f3823bc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41604/0 (socket says 172.21.15.47:41604) 2024-03-21T11:37:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.401+0000 7f405be00640 1 -- 172.21.15.47:0/4218111121 learned_addr learned my addr 172.21.15.47:0/4218111121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.402+0000 7f4053e00640 1 --2- 172.21.15.47:0/4218111121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 0x55f3823c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.402+0000 7f405be00640 1 -- 172.21.15.47:0/4218111121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 msgr2=0x55f3823c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.402+0000 7f405be00640 1 --2- 172.21.15.47:0/4218111121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 0x55f3823c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.402+0000 7f405be00640 1 -- 172.21.15.47:0/4218111121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823ba9f0 msgr2=0x55f3823bae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.402+0000 7f405be00640 1 --2- 172.21.15.47:0/4218111121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823ba9f0 0x55f3823bae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.402+0000 7f405be00640 1 -- 172.21.15.47:0/4218111121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3822ec080 con 0x55f3823bbbc0 2024-03-21T11:37:25.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.402+0000 7f405be00640 1 --2- 172.21.15.47:0/4218111121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823bbbc0 0x55f3823bc040 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f404c009d50 tx=0x7f404c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c27d81927232e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.402+0000 7f4059600640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40400a2fa0 0x7f40400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.403+0000 7f4059600640 1 -- 172.21.15.47:0/1486049942 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f40540cff10 con 0x7f405c107260 2024-03-21T11:37:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.403+0000 7f4053e00640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40400a2fa0 0x7f40400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.403+0000 7f4051600640 1 -- 172.21.15.47:0/4218111121 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c012650 con 0x55f3823bbbc0 2024-03-21T11:37:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.403+0000 7f4051600640 1 -- 172.21.15.47:0/4218111121 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f404c009e40 con 0x55f3823bbbc0 2024-03-21T11:37:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.403+0000 7f4051600640 1 -- 172.21.15.47:0/4218111121 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c00fd10 con 0x55f3823bbbc0 2024-03-21T11:37:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.404+0000 7f4065e48740 1 -- 172.21.15.47:0/4218111121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823bbbc0 msgr2=0x55f3823bc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.404+0000 7f4053e00640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40400a2fa0 0x7f40400a5460 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4044005ab0 tx=0x7f4044000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.404+0000 7f4065e48740 1 --2- 172.21.15.47:0/4218111121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823bbbc0 0x55f3823bc040 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f404c009d50 tx=0x7f404c0117b0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.404+0000 7f4065e48740 1 -- 172.21.15.47:0/4218111121 shutdown_connections 2024-03-21T11:37:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.404+0000 7f4065e48740 1 --2- 172.21.15.47:0/4218111121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 0x55f3823c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.404+0000 7f4065e48740 1 --2- 172.21.15.47:0/4218111121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823bbbc0 0x55f3823bc040 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.404+0000 7f4065e48740 1 --2- 172.21.15.47:0/4218111121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823ba9f0 0x55f3823bae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.404+0000 7f4065e48740 1 -- 172.21.15.47:0/4218111121 >> 172.21.15.47:0/4218111121 conn(0x55f3823b5de0 msgr2=0x55f3823b61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.404+0000 7f4065e48740 1 -- 172.21.15.47:0/4218111121 shutdown_connections 2024-03-21T11:37:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.404+0000 7f4065e48740 1 -- 172.21.15.47:0/4218111121 wait complete. 2024-03-21T11:37:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.404+0000 7f4065e48740 1 -- start start 2024-03-21T11:37:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.405+0000 7f4065e48740 1 Processor -- start 2024-03-21T11:37:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.405+0000 7f4065e48740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.405+0000 7f4065e48740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bbbc0 0x55f3823c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.405+0000 7f4065e48740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bc580 0x55f3823cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.405+0000 7f4065e48740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3823c3590 con 0x55f3823bc580 2024-03-21T11:37:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.405+0000 7f4065e48740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3822db0d0 con 0x55f3823ba9f0 2024-03-21T11:37:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.405+0000 7f4065e48740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3823c3710 con 0x55f3823bbbc0 2024-03-21T11:37:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.405+0000 7f4065e48740 10 client.0 _ll_register_callbacks cb 0x55f3822dfe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.405+0000 7f405be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.405+0000 7f4061200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bc580 0x55f3823cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.405+0000 7f4053e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bbbc0 0x55f3823c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.405+0000 7f405be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41606/0 (socket says 172.21.15.47:41606) 2024-03-21T11:37:25.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.405+0000 7f405be00640 1 -- 172.21.15.47:0/1091871851 learned_addr learned my addr 172.21.15.47:0/1091871851 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:25.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.406+0000 7f4053e00640 1 -- 172.21.15.47:0/1091871851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 msgr2=0x55f3823c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.406+0000 7f4053e00640 1 --2- 172.21.15.47:0/1091871851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.406+0000 7f4053e00640 1 -- 172.21.15.47:0/1091871851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bc580 msgr2=0x55f3823cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.406+0000 7f4053e00640 1 --2- 172.21.15.47:0/1091871851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bc580 0x55f3823cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.406+0000 7f4053e00640 1 -- 172.21.15.47:0/1091871851 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3820333f0 con 0x55f3823bbbc0 2024-03-21T11:37:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.406+0000 7f405be00640 1 --2- 172.21.15.47:0/1091871851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.406+0000 7f4053e00640 1 --2- 172.21.15.47:0/1091871851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bbbc0 0x55f3823c5140 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f404402efd0 tx=0x7f404402cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33cf409a07f832f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.406+0000 7f404a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:37:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.407+0000 7f404a000640 1 -- 172.21.15.47:0/1091871851 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044004d20 con 0x55f3823bbbc0 2024-03-21T11:37:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.407+0000 7f4065e48740 1 -- 172.21.15.47:0/1091871851 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f38232c800 con 0x55f3823bbbc0 2024-03-21T11:37:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.407+0000 7f404a000640 1 -- 172.21.15.47:0/1091871851 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f40440035f0 con 0x55f3823bbbc0 2024-03-21T11:37:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.407+0000 7f404a000640 1 -- 172.21.15.47:0/1091871851 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40440037f0 con 0x55f3823bbbc0 2024-03-21T11:37:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.407+0000 7f4065e48740 10 client.30208 fetch_fsmap learned FSMap version 32 2024-03-21T11:37:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.407+0000 7f4065e48740 1 -- 172.21.15.47:0/1091871851 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f3820fd7d0 con 0x55f3823bbbc0 2024-03-21T11:37:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.407+0000 7f404a000640 1 -- 172.21.15.47:0/1091871851 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f38232c800 con 0x55f3823bbbc0 2024-03-21T11:37:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.408+0000 7f404a000640 1 -- 172.21.15.47:0/1091871851 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f404404eb90 con 0x55f3823bbbc0 2024-03-21T11:37:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.408+0000 7f4065e48740 10 client.30208 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.408+0000 7f4065e48740 10 client.30208 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.408+0000 7f4065e48740 20 client.30208 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.408+0000 7f4065e48740 1 --2- 172.21.15.47:0/1091871851 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f3823d06e0 0x55f3823d2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.408+0000 7f4065e48740 4 client.30208 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.408+0000 7f4065e48740 1 -- 172.21.15.47:0/1091871851 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f382326b50 con 0x55f3823d06e0 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.408+0000 7f405be00640 1 --2- 172.21.15.47:0/1091871851 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f3823d06e0 0x55f3823d2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.409+0000 7f405be00640 1 --2- 172.21.15.47:0/1091871851 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f3823d06e0 0x55f3823d2b40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c57e96e9ddd47710 server_cookie=9b001a415b7985f8 in_seq=0 out_seq=0 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.409+0000 7f404a000640 10 client.30208 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.410+0000 7f404a000640 1 -- 172.21.15.47:0/1091871851 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f382326b50 con 0x55f3823d06e0 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.410+0000 7f404a000640 10 client.30208 handle_command_reply: tid=0 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.410+0000 7f4065e48740 1 client.30208 shutdown 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.410+0000 7f4065e48740 1 -- 172.21.15.47:0/1091871851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bbbc0 msgr2=0x55f3823c5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.410+0000 7f4065e48740 1 --2- 172.21.15.47:0/1091871851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bbbc0 0x55f3823c5140 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f404402efd0 tx=0x7f404402cf90 comp rx=0 tx=0).stop 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.410+0000 7f4065e48740 1 -- 172.21.15.47:0/1091871851 shutdown_connections 2024-03-21T11:37:25.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.410+0000 7f4065e48740 1 --2- 172.21.15.47:0/1091871851 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f3823d06e0 0x55f3823d2b40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.410+0000 7f4065e48740 1 --2- 172.21.15.47:0/1091871851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bc580 0x55f3823cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.410+0000 7f4065e48740 1 --2- 172.21.15.47:0/1091871851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bbbc0 0x55f3823c5140 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.410+0000 7f4065e48740 1 --2- 172.21.15.47:0/1091871851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.410+0000 7f404a000640 0 client.30208 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:37:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.410+0000 7f4065e48740 1 -- 172.21.15.47:0/1091871851 >> 172.21.15.47:0/1091871851 conn(0x55f3823b5de0 msgr2=0x55f3823b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.410+0000 7f4065e48740 1 -- 172.21.15.47:0/1091871851 shutdown_connections 2024-03-21T11:37:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.410+0000 7f4065e48740 1 -- 172.21.15.47:0/1091871851 wait complete. 2024-03-21T11:37:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.411+0000 7f4065e48740 20 client.30208 trim_cache size 0 max 16384 2024-03-21T11:37:25.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.424+0000 7f4065e48740 1 Processor -- start 2024-03-21T11:37:25.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.424+0000 7f4065e48740 1 -- start start 2024-03-21T11:37:25.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f4065e48740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f4065e48740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bbbc0 0x55f3823cc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f4065e48740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 0x55f3823bebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f4065e48740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c1ac080 con 0x55f3823bbbc0 2024-03-21T11:37:25.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f4065e48740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c1adb20 con 0x55f3823ba9f0 2024-03-21T11:37:25.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f4065e48740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c1ac6e0 con 0x55f3823bc580 2024-03-21T11:37:25.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f405be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f4061200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 0x55f3823bebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f405be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41618/0 (socket says 172.21.15.47:41618) 2024-03-21T11:37:25.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f405be00640 1 -- 172.21.15.47:0/2179560592 learned_addr learned my addr 172.21.15.47:0/2179560592 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:25.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f4053e00640 1 --2- 172.21.15.47:0/2179560592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bbbc0 0x55f3823cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f405be00640 1 -- 172.21.15.47:0/2179560592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 msgr2=0x55f3823bebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f405be00640 1 --2- 172.21.15.47:0/2179560592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 0x55f3823bebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f405be00640 1 -- 172.21.15.47:0/2179560592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bbbc0 msgr2=0x55f3823cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f405be00640 1 --2- 172.21.15.47:0/2179560592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bbbc0 0x55f3823cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f405be00640 1 -- 172.21.15.47:0/2179560592 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3823b6850 con 0x55f3823ba9f0 2024-03-21T11:37:25.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f4053e00640 1 --2- 172.21.15.47:0/2179560592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bbbc0 0x55f3823cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:25.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.425+0000 7f4061200640 1 --2- 172.21.15.47:0/2179560592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 0x55f3823bebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:25.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.426+0000 7f405be00640 1 --2- 172.21.15.47:0/2179560592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c4220 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f404c009d20 tx=0x7f404c005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e24247d0d1ec2dfd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.426+0000 7f4050c00640 1 -- 172.21.15.47:0/2179560592 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c0095b0 con 0x55f3823ba9f0 2024-03-21T11:37:25.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.426+0000 7f4050c00640 1 -- 172.21.15.47:0/2179560592 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f404c005520 con 0x55f3823ba9f0 2024-03-21T11:37:25.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.426+0000 7f4050c00640 1 -- 172.21.15.47:0/2179560592 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c00f560 con 0x55f3823ba9f0 2024-03-21T11:37:25.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.427+0000 7f4065e48740 1 -- 172.21.15.47:0/2179560592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 msgr2=0x55f3823c4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.427+0000 7f4065e48740 1 --2- 172.21.15.47:0/2179560592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c4220 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f404c009d20 tx=0x7f404c005c50 comp rx=0 tx=0).stop 2024-03-21T11:37:25.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.427+0000 7f4065e48740 1 -- 172.21.15.47:0/2179560592 shutdown_connections 2024-03-21T11:37:25.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.427+0000 7f4065e48740 1 --2- 172.21.15.47:0/2179560592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 0x55f3823bebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.427+0000 7f4065e48740 1 --2- 172.21.15.47:0/2179560592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bbbc0 0x55f3823cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.427+0000 7f4065e48740 1 --2- 172.21.15.47:0/2179560592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c4220 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.427+0000 7f4065e48740 1 -- 172.21.15.47:0/2179560592 >> 172.21.15.47:0/2179560592 conn(0x55f3823b5de0 msgr2=0x55f3823b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.427+0000 7f4065e48740 1 -- 172.21.15.47:0/2179560592 shutdown_connections 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.427+0000 7f4065e48740 1 -- 172.21.15.47:0/2179560592 wait complete. 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.427+0000 7f4065e48740 1 -- start start 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.428+0000 7f4065e48740 1 Processor -- start 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.428+0000 7f4065e48740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.428+0000 7f4065e48740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bbbc0 0x55f3823c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.428+0000 7f4065e48740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 0x55f3823cb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.428+0000 7f4065e48740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c1ad400 con 0x55f3823bbbc0 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.428+0000 7f4065e48740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c1acda0 con 0x55f3823ba9f0 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.428+0000 7f4065e48740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c1aa240 con 0x55f3823bc580 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.428+0000 7f4065e48740 10 client.0 _ll_register_callbacks cb 0x7f4044009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.428+0000 7f405be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.428+0000 7f405be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41620/0 (socket says 172.21.15.47:41620) 2024-03-21T11:37:25.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.428+0000 7f405be00640 1 -- 172.21.15.47:0/3162747233 learned_addr learned my addr 172.21.15.47:0/3162747233 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.428+0000 7f4061200640 1 --2- 172.21.15.47:0/3162747233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 0x55f3823cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.428+0000 7f4053e00640 1 --2- 172.21.15.47:0/3162747233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bbbc0 0x55f3823c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.429+0000 7f405be00640 1 -- 172.21.15.47:0/3162747233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 msgr2=0x55f3823cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.429+0000 7f405be00640 1 --2- 172.21.15.47:0/3162747233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 0x55f3823cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.429+0000 7f405be00640 1 -- 172.21.15.47:0/3162747233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bbbc0 msgr2=0x55f3823c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.429+0000 7f405be00640 1 --2- 172.21.15.47:0/3162747233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bbbc0 0x55f3823c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.429+0000 7f405be00640 1 -- 172.21.15.47:0/3162747233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3823cbe00 con 0x55f3823ba9f0 2024-03-21T11:37:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.429+0000 7f4061200640 1 --2- 172.21.15.47:0/3162747233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 0x55f3823cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.429+0000 7f405be00640 1 --2- 172.21.15.47:0/3162747233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c82b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f404c005d00 tx=0x7f404c0095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d82f8b94d249b1c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.429+0000 7f404a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:37:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.429+0000 7f404a000640 1 -- 172.21.15.47:0/3162747233 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c003e60 con 0x55f3823ba9f0 2024-03-21T11:37:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.429+0000 7f404a000640 1 -- 172.21.15.47:0/3162747233 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f404c003800 con 0x55f3823ba9f0 2024-03-21T11:37:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.429+0000 7f4065e48740 1 -- 172.21.15.47:0/3162747233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f404c005520 con 0x55f3823ba9f0 2024-03-21T11:37:25.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.430+0000 7f404a000640 1 -- 172.21.15.47:0/3162747233 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f404c03f050 con 0x55f3823ba9f0 2024-03-21T11:37:25.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.430+0000 7f4065e48740 10 client.30188 fetch_fsmap learned FSMap version 32 2024-03-21T11:37:25.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.430+0000 7f4065e48740 1 -- 172.21.15.47:0/3162747233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f38242ca50 con 0x55f3823ba9f0 2024-03-21T11:37:25.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.430+0000 7f404a000640 1 -- 172.21.15.47:0/3162747233 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f404c005520 con 0x55f3823ba9f0 2024-03-21T11:37:25.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.430+0000 7f404a000640 1 -- 172.21.15.47:0/3162747233 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f404c036610 con 0x55f3823ba9f0 2024-03-21T11:37:25.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.430+0000 7f4065e48740 10 client.30188 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T11:37:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.430+0000 7f4065e48740 10 client.30188 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:37:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.430+0000 7f4065e48740 20 client.30188 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.430+0000 7f4065e48740 1 --2- 172.21.15.47:0/3162747233 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f38242d200 0x55f38242f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.430+0000 7f4065e48740 4 client.30188 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:37:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.430+0000 7f4065e48740 1 -- 172.21.15.47:0/3162747233 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f3822a4500 con 0x55f38242d200 2024-03-21T11:37:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.430+0000 7f4053e00640 1 --2- 172.21.15.47:0/3162747233 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f38242d200 0x55f38242f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.431+0000 7f4053e00640 1 --2- 172.21.15.47:0/3162747233 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f38242d200 0x55f38242f620 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9e8937940b49982 server_cookie=112e1fa59187299d in_seq=0 out_seq=0 2024-03-21T11:37:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.431+0000 7f404a000640 10 client.30188 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:37:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.431+0000 7f404a000640 1 -- 172.21.15.47:0/3162747233 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f3822a4500 con 0x55f38242d200 2024-03-21T11:37:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.431+0000 7f404a000640 10 client.30188 handle_command_reply: tid=0 2024-03-21T11:37:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.431+0000 7f4065e48740 1 client.30188 shutdown 2024-03-21T11:37:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.431+0000 7f4065e48740 1 -- 172.21.15.47:0/3162747233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 msgr2=0x55f3823c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.431+0000 7f4065e48740 1 --2- 172.21.15.47:0/3162747233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c82b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f404c005d00 tx=0x7f404c0095b0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.431+0000 7f4065e48740 1 -- 172.21.15.47:0/3162747233 shutdown_connections 2024-03-21T11:37:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.431+0000 7f4065e48740 1 --2- 172.21.15.47:0/3162747233 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55f38242d200 0x55f38242f620 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.431+0000 7f4065e48740 1 --2- 172.21.15.47:0/3162747233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3823bc580 0x55f3823cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.431+0000 7f4065e48740 1 --2- 172.21.15.47:0/3162747233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3823bbbc0 0x55f3823c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.431+0000 7f404a000640 0 client.30188 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:37:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.431+0000 7f4065e48740 1 --2- 172.21.15.47:0/3162747233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3823ba9f0 0x55f3823c82b0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.431+0000 7f4065e48740 1 -- 172.21.15.47:0/3162747233 >> 172.21.15.47:0/3162747233 conn(0x55f3823b5de0 msgr2=0x55f3823b98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.432+0000 7f4065e48740 1 -- 172.21.15.47:0/3162747233 shutdown_connections 2024-03-21T11:37:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.432+0000 7f4065e48740 1 -- 172.21.15.47:0/3162747233 wait complete. 2024-03-21T11:37:25.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.432+0000 7f4065e48740 20 client.30188 trim_cache size 0 max 16384 2024-03-21T11:37:25.437 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:37:25.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.432+0000 7f4061c00640 1 -- 172.21.15.47:0/1486049942 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40400a2fa0 msgr2=0x7f40400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.432+0000 7f4061c00640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40400a2fa0 0x7f40400a5460 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4044005ab0 tx=0x7f4044000bc0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.432+0000 7f4061c00640 1 -- 172.21.15.47:0/1486049942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f405c107260 msgr2=0x7f405c1004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.432+0000 7f4061c00640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f405c107260 0x7f405c1004f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f4054009fd0 tx=0x7f405400ef10 comp rx=0 tx=0).stop 2024-03-21T11:37:25.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.432+0000 7f4061c00640 1 -- 172.21.15.47:0/1486049942 shutdown_connections 2024-03-21T11:37:25.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.432+0000 7f4061c00640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40400a2fa0 0x7f40400a5460 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.432+0000 7f4061c00640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f405c107260 0x7f405c1004f0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.432+0000 7f4061c00640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f405c104930 0x7f405c0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.432+0000 7f4061c00640 1 --2- 172.21.15.47:0/1486049942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f405c102000 0x7f405c101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.432+0000 7f4061c00640 1 -- 172.21.15.47:0/1486049942 >> 172.21.15.47:0/1486049942 conn(0x7f405c0fba50 msgr2=0x7f405c102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.433+0000 7f4061c00640 1 -- 172.21.15.47:0/1486049942 shutdown_connections 2024-03-21T11:37:25.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.433+0000 7f4061c00640 1 -- 172.21.15.47:0/1486049942 wait complete. 2024-03-21T11:37:25.452 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:37:25.452 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:37:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.667+0000 7f72aa600640 1 -- 172.21.15.47:0/1337731690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a4108eb0 msgr2=0x7f72a4109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.667+0000 7f72aa600640 1 --2- 172.21.15.47:0/1337731690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a4108eb0 0x7f72a4109330 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7290009900 tx=0x7f729002bc30 comp rx=0 tx=0).stop 2024-03-21T11:37:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.668+0000 7f72aa600640 1 -- 172.21.15.47:0/1337731690 shutdown_connections 2024-03-21T11:37:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.668+0000 7f72aa600640 1 --2- 172.21.15.47:0/1337731690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72a4109870 0x7f72a4110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.668+0000 7f72aa600640 1 --2- 172.21.15.47:0/1337731690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a4108eb0 0x7f72a4109330 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.668+0000 7f72aa600640 1 --2- 172.21.15.47:0/1337731690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a4107cc0 0x7f72a41080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.668+0000 7f72aa600640 1 -- 172.21.15.47:0/1337731690 >> 172.21.15.47:0/1337731690 conn(0x7f72a4076c60 msgr2=0x7f72a40790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.668+0000 7f72aa600640 1 -- 172.21.15.47:0/1337731690 shutdown_connections 2024-03-21T11:37:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.668+0000 7f72aa600640 1 -- 172.21.15.47:0/1337731690 wait complete. 2024-03-21T11:37:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.669+0000 7f72aa600640 1 Processor -- start 2024-03-21T11:37:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.669+0000 7f72aa600640 1 -- start start 2024-03-21T11:37:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.669+0000 7f72aa600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a4107cc0 0x7f72a419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.669+0000 7f72aa600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72a4108eb0 0x7f72a419e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.669+0000 7f72aa600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a4109870 0x7f72a41a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.669+0000 7f72aa600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a4113940 con 0x7f72a4109870 2024-03-21T11:37:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.669+0000 7f72aa600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a41137c0 con 0x7f72a4108eb0 2024-03-21T11:37:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.669+0000 7f72aa600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a4113ac0 con 0x7f72a4107cc0 2024-03-21T11:37:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.669+0000 7f72a3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a4107cc0 0x7f72a419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.669+0000 7f72a3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a4107cc0 0x7f72a419df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56766/0 (socket says 172.21.15.47:56766) 2024-03-21T11:37:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.669+0000 7f72a3400640 1 -- 172.21.15.47:0/689981709 learned_addr learned my addr 172.21.15.47:0/689981709 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.669+0000 7f72a2a00640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72a4108eb0 0x7f72a419e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.669+0000 7f72a3e00640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a4109870 0x7f72a41a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.670+0000 7f72a2a00640 1 -- 172.21.15.47:0/689981709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a4107cc0 msgr2=0x7f72a419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.670+0000 7f72a2a00640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a4107cc0 0x7f72a419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.670+0000 7f72a2a00640 1 -- 172.21.15.47:0/689981709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a4109870 msgr2=0x7f72a41a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.670+0000 7f72a2a00640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a4109870 0x7f72a41a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.670+0000 7f72a2a00640 1 -- 172.21.15.47:0/689981709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a41a5bb0 con 0x7f72a4108eb0 2024-03-21T11:37:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.670+0000 7f72a3400640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a4107cc0 0x7f72a419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.670+0000 7f72a2a00640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72a4108eb0 0x7f72a419e4d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7290009230 tx=0x7f729002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.670+0000 7f729be00640 1 -- 172.21.15.47:0/689981709 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290041070 con 0x7f72a4108eb0 2024-03-21T11:37:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.671+0000 7f729be00640 1 -- 172.21.15.47:0/689981709 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7290004450 con 0x7f72a4108eb0 2024-03-21T11:37:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.671+0000 7f72aa600640 1 -- 172.21.15.47:0/689981709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a41a5e40 con 0x7f72a4108eb0 2024-03-21T11:37:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.671+0000 7f729be00640 1 -- 172.21.15.47:0/689981709 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290034440 con 0x7f72a4108eb0 2024-03-21T11:37:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.671+0000 7f72aa600640 1 -- 172.21.15.47:0/689981709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a41a6300 con 0x7f72a4108eb0 2024-03-21T11:37:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.675+0000 7f72ad26b740 1 Processor -- start 2024-03-21T11:37:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.675+0000 7f72ad26b740 1 -- start start 2024-03-21T11:37:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.675+0000 7f72ad26b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d6123810 0x5621d6123c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.675+0000 7f72ad26b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d61249e0 0x5621d6124e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.675+0000 7f72ad26b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61253a0 0x5621d612bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.675+0000 7f72ad26b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d60882c0 con 0x5621d6123810 2024-03-21T11:37:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.675+0000 7f72ad26b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d6030c90 con 0x5621d61253a0 2024-03-21T11:37:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.675+0000 7f72ad26b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d6084360 con 0x5621d61249e0 2024-03-21T11:37:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.675+0000 7f72a3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d6123810 0x5621d6123c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.676+0000 7f72a3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d6123810 0x5621d6123c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58996/0 (socket says 172.21.15.47:58996) 2024-03-21T11:37:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.676+0000 7f72a3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d61249e0 0x5621d6124e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.676+0000 7f72a3e00640 1 -- 172.21.15.47:0/735945664 learned_addr learned my addr 172.21.15.47:0/735945664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.676+0000 7f729be00640 1 -- 172.21.15.47:0/689981709 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7290044070 con 0x7f72a4108eb0 2024-03-21T11:37:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.676+0000 7f72a2a00640 1 --2- 172.21.15.47:0/735945664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61253a0 0x5621d612bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.677+0000 7f72a2a00640 1 -- 172.21.15.47:0/735945664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d61249e0 msgr2=0x5621d6124e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.677+0000 7f72a2a00640 1 --2- 172.21.15.47:0/735945664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d61249e0 0x5621d6124e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.677+0000 7f72a2a00640 1 -- 172.21.15.47:0/735945664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d6123810 msgr2=0x5621d6123c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.677+0000 7f72a2a00640 1 --2- 172.21.15.47:0/735945664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d6123810 0x5621d6123c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.677+0000 7f72a2a00640 1 -- 172.21.15.47:0/735945664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621d6055080 con 0x5621d61253a0 2024-03-21T11:37:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.677+0000 7f72a2a00640 1 --2- 172.21.15.47:0/735945664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61253a0 0x5621d612bc30 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f72900ee8e0 tx=0x7f72900f9270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce0545e8c04f9285 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.677+0000 7f7298c00640 1 -- 172.21.15.47:0/735945664 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72900f30d0 con 0x5621d61253a0 2024-03-21T11:37:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.677+0000 7f7298c00640 1 -- 172.21.15.47:0/735945664 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f72900fe030 con 0x5621d61253a0 2024-03-21T11:37:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.678+0000 7f7298c00640 1 -- 172.21.15.47:0/735945664 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72900f1e60 con 0x5621d61253a0 2024-03-21T11:37:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.678+0000 7f729be00640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72880a3020 0x7f72880a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.678+0000 7f729be00640 1 -- 172.21.15.47:0/689981709 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f72900f1230 con 0x7f72a4108eb0 2024-03-21T11:37:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.678+0000 7f72a3e00640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72880a3020 0x7f72880a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.678+0000 7f72ad26b740 1 -- 172.21.15.47:0/735945664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61253a0 msgr2=0x5621d612bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.678+0000 7f72ad26b740 1 --2- 172.21.15.47:0/735945664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61253a0 0x5621d612bc30 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f72900ee8e0 tx=0x7f72900f9270 comp rx=0 tx=0).stop 2024-03-21T11:37:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.679+0000 7f72ad26b740 1 -- 172.21.15.47:0/735945664 shutdown_connections 2024-03-21T11:37:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.679+0000 7f72ad26b740 1 --2- 172.21.15.47:0/735945664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61253a0 0x5621d612bc30 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.679+0000 7f72ad26b740 1 --2- 172.21.15.47:0/735945664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d61249e0 0x5621d6124e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.679+0000 7f72ad26b740 1 --2- 172.21.15.47:0/735945664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d6123810 0x5621d6123c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.679+0000 7f72ad26b740 1 -- 172.21.15.47:0/735945664 >> 172.21.15.47:0/735945664 conn(0x5621d611ec00 msgr2=0x5621d611f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.679+0000 7f72a3e00640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72880a3020 0x7f72880a54e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x5621d61248e0 tx=0x7f729c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.679+0000 7f72ad26b740 1 -- 172.21.15.47:0/735945664 shutdown_connections 2024-03-21T11:37:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.679+0000 7f72ad26b740 1 -- 172.21.15.47:0/735945664 wait complete. 2024-03-21T11:37:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.680+0000 7f72ad26b740 1 -- start start 2024-03-21T11:37:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.680+0000 7f72ad26b740 1 Processor -- start 2024-03-21T11:37:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.680+0000 7f72ad26b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d612d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.680+0000 7f72ad26b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 0x5621d612dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.680+0000 7f72ad26b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 0x5621d6134ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.680+0000 7f72ad26b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d612c170 con 0x5621d61253a0 2024-03-21T11:37:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.680+0000 7f72ad26b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d60440d0 con 0x5621d61249e0 2024-03-21T11:37:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.680+0000 7f72ad26b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d612c2f0 con 0x5621d6123810 2024-03-21T11:37:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.680+0000 7f72ad26b740 10 client.0 _ll_register_callbacks cb 0x5621d6048e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.681+0000 7f72a3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d612d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.681+0000 7f72a2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 0x5621d612dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.681+0000 7f72a3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d612d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56780/0 (socket says 172.21.15.47:56780) 2024-03-21T11:37:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.681+0000 7f72a3400640 1 -- 172.21.15.47:0/2118026954 learned_addr learned my addr 172.21.15.47:0/2118026954 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.681+0000 7f72a3e00640 1 --2- 172.21.15.47:0/2118026954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 0x5621d6134ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.681+0000 7f72a3400640 1 -- 172.21.15.47:0/2118026954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 msgr2=0x5621d612dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.681+0000 7f72a3400640 1 --2- 172.21.15.47:0/2118026954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 0x5621d612dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.681+0000 7f72a3400640 1 -- 172.21.15.47:0/2118026954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 msgr2=0x5621d6134ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.681+0000 7f72a3400640 1 --2- 172.21.15.47:0/2118026954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 0x5621d6134ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.681+0000 7f72a3400640 1 -- 172.21.15.47:0/2118026954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621d5d9c3f0 con 0x5621d6123810 2024-03-21T11:37:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.682+0000 7f72a3400640 1 --2- 172.21.15.47:0/2118026954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d612d720 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f72940118a0 tx=0x7f7294011d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fed076c2550bb33c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.682+0000 7f728d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:37:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.682+0000 7f728d600640 1 -- 172.21.15.47:0/2118026954 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294010c00 con 0x5621d6123810 2024-03-21T11:37:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.682+0000 7f72ad26b740 1 -- 172.21.15.47:0/2118026954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72900fe030 con 0x5621d6123810 2024-03-21T11:37:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.682+0000 7f728d600640 1 -- 172.21.15.47:0/2118026954 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7294010da0 con 0x5621d6123810 2024-03-21T11:37:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.682+0000 7f728d600640 1 -- 172.21.15.47:0/2118026954 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7294024050 con 0x5621d6123810 2024-03-21T11:37:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.682+0000 7f72ad26b740 10 client.30226 fetch_fsmap learned FSMap version 32 2024-03-21T11:37:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.682+0000 7f72ad26b740 1 -- 172.21.15.47:0/2118026954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5621d5e667d0 con 0x5621d6123810 2024-03-21T11:37:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.683+0000 7f728d600640 1 -- 172.21.15.47:0/2118026954 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f72900fe030 con 0x5621d6123810 2024-03-21T11:37:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.683+0000 7f728d600640 1 -- 172.21.15.47:0/2118026954 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f729401a410 con 0x5621d6123810 2024-03-21T11:37:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.683+0000 7f72ad26b740 10 client.30226 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T11:37:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.683+0000 7f72ad26b740 10 client.30226 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:37:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.683+0000 7f72ad26b740 20 client.30226 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.683+0000 7f72ad26b740 1 --2- 172.21.15.47:0/2118026954 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5621d6139240 0x5621d613b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.683+0000 7f72ad26b740 4 client.30226 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:37:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.683+0000 7f72ad26b740 1 -- 172.21.15.47:0/2118026954 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621d6095800 con 0x5621d6139240 2024-03-21T11:37:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.683+0000 7f72a2a00640 1 --2- 172.21.15.47:0/2118026954 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5621d6139240 0x5621d613b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.684+0000 7f72a2a00640 1 --2- 172.21.15.47:0/2118026954 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5621d6139240 0x5621d613b6c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15e5ee827d841156 server_cookie=ee2065f6e98ea4cb in_seq=0 out_seq=0 2024-03-21T11:37:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.684+0000 7f728d600640 10 client.30226 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:37:25.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.685+0000 7f728d600640 1 -- 172.21.15.47:0/2118026954 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5621d6095800 con 0x5621d6139240 2024-03-21T11:37:25.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.685+0000 7f728d600640 10 client.30226 handle_command_reply: tid=0 2024-03-21T11:37:25.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.685+0000 7f72ad26b740 1 client.30226 shutdown 2024-03-21T11:37:25.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.685+0000 7f72ad26b740 1 -- 172.21.15.47:0/2118026954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 msgr2=0x5621d612d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.685+0000 7f72ad26b740 1 --2- 172.21.15.47:0/2118026954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d612d720 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f72940118a0 tx=0x7f7294011d60 comp rx=0 tx=0).stop 2024-03-21T11:37:25.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.685+0000 7f72ad26b740 1 -- 172.21.15.47:0/2118026954 shutdown_connections 2024-03-21T11:37:25.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.685+0000 7f72ad26b740 1 --2- 172.21.15.47:0/2118026954 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5621d6139240 0x5621d613b6c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.685+0000 7f72ad26b740 1 --2- 172.21.15.47:0/2118026954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 0x5621d6134ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.685+0000 7f72ad26b740 1 --2- 172.21.15.47:0/2118026954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 0x5621d612dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.685+0000 7f72ad26b740 1 --2- 172.21.15.47:0/2118026954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d612d720 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.685+0000 7f728d600640 0 client.30226 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:37:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.685+0000 7f72ad26b740 1 -- 172.21.15.47:0/2118026954 >> 172.21.15.47:0/2118026954 conn(0x5621d611ec00 msgr2=0x5621d611deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.685+0000 7f72ad26b740 1 -- 172.21.15.47:0/2118026954 shutdown_connections 2024-03-21T11:37:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.685+0000 7f72ad26b740 1 -- 172.21.15.47:0/2118026954 wait complete. 2024-03-21T11:37:25.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.686+0000 7f72ad26b740 20 client.30226 trim_cache size 0 max 16384 2024-03-21T11:37:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.701+0000 7f72ad26b740 1 Processor -- start 2024-03-21T11:37:25.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.701+0000 7f72ad26b740 1 -- start start 2024-03-21T11:37:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.702+0000 7f72ad26b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d6121e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.702+0000 7f72ad26b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 0x5621d6134c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.702+0000 7f72ad26b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 0x5621d613ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.702+0000 7f72ad26b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a41b0720 con 0x5621d61253a0 2024-03-21T11:37:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.702+0000 7f72ad26b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a41b21c0 con 0x5621d61249e0 2024-03-21T11:37:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.702+0000 7f72ad26b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a41b0d80 con 0x5621d6123810 2024-03-21T11:37:25.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.702+0000 7f72a2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 0x5621d6134c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.702+0000 7f72a3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 0x5621d613ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.702+0000 7f72a2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 0x5621d6134c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41678/0 (socket says 172.21.15.47:41678) 2024-03-21T11:37:25.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.702+0000 7f72a2a00640 1 -- 172.21.15.47:0/1515473991 learned_addr learned my addr 172.21.15.47:0/1515473991 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:25.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.702+0000 7f72a3400640 1 --2- 172.21.15.47:0/1515473991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d6121e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.703+0000 7f72a2a00640 1 -- 172.21.15.47:0/1515473991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 msgr2=0x5621d6121e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.703+0000 7f72a2a00640 1 --2- 172.21.15.47:0/1515473991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d6121e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.703+0000 7f72a2a00640 1 -- 172.21.15.47:0/1515473991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 msgr2=0x5621d613ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.703+0000 7f72a2a00640 1 --2- 172.21.15.47:0/1515473991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 0x5621d613ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.703+0000 7f72a2a00640 1 -- 172.21.15.47:0/1515473991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621d6124460 con 0x5621d61249e0 2024-03-21T11:37:25.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.703+0000 7f72a2a00640 1 --2- 172.21.15.47:0/1515473991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 0x5621d6134c60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f729002c170 tx=0x7f7290005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3469c73747c1f90 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.703+0000 7f728fe00640 1 -- 172.21.15.47:0/1515473991 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729004b040 con 0x5621d61249e0 2024-03-21T11:37:25.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.704+0000 7f728fe00640 1 -- 172.21.15.47:0/1515473991 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f72900fe070 con 0x5621d61249e0 2024-03-21T11:37:25.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.704+0000 7f728fe00640 1 -- 172.21.15.47:0/1515473991 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72900bcf80 con 0x5621d61249e0 2024-03-21T11:37:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.704+0000 7f72ad26b740 1 -- 172.21.15.47:0/1515473991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 msgr2=0x5621d6134c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.704+0000 7f72ad26b740 1 --2- 172.21.15.47:0/1515473991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 0x5621d6134c60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f729002c170 tx=0x7f7290005fd0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.705+0000 7f72ad26b740 1 -- 172.21.15.47:0/1515473991 shutdown_connections 2024-03-21T11:37:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.705+0000 7f72ad26b740 1 --2- 172.21.15.47:0/1515473991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 0x5621d613ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.705+0000 7f72ad26b740 1 --2- 172.21.15.47:0/1515473991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 0x5621d6134c60 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.705+0000 7f72ad26b740 1 --2- 172.21.15.47:0/1515473991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d6121e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.705+0000 7f72ad26b740 1 -- 172.21.15.47:0/1515473991 >> 172.21.15.47:0/1515473991 conn(0x5621d611ec00 msgr2=0x5621d61284a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.705+0000 7f72ad26b740 1 -- 172.21.15.47:0/1515473991 shutdown_connections 2024-03-21T11:37:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.705+0000 7f72ad26b740 1 -- 172.21.15.47:0/1515473991 wait complete. 2024-03-21T11:37:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.705+0000 7f72ad26b740 1 -- start start 2024-03-21T11:37:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.705+0000 7f72ad26b740 1 Processor -- start 2024-03-21T11:37:25.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72ad26b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d6132b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72ad26b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 0x5621d6133060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72ad26b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 0x5621d6134070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72ad26b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a41b1aa0 con 0x5621d61253a0 2024-03-21T11:37:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72ad26b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a41b1440 con 0x5621d61249e0 2024-03-21T11:37:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72ad26b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a41ae8e0 con 0x5621d6123810 2024-03-21T11:37:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72ad26b740 10 client.0 _ll_register_callbacks cb 0x7f729400bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72a3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d6132b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72a3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d6132b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56798/0 (socket says 172.21.15.47:56798) 2024-03-21T11:37:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72a3400640 1 -- 172.21.15.47:0/4157720215 learned_addr learned my addr 172.21.15.47:0/4157720215 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72a2a00640 1 --2- 172.21.15.47:0/4157720215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 0x5621d6133060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72a3e00640 1 --2- 172.21.15.47:0/4157720215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 0x5621d6134070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72a3400640 1 -- 172.21.15.47:0/4157720215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 msgr2=0x5621d6133060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72a3400640 1 --2- 172.21.15.47:0/4157720215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 0x5621d6133060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72a3400640 1 -- 172.21.15.47:0/4157720215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 msgr2=0x5621d6134070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72a3400640 1 --2- 172.21.15.47:0/4157720215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 0x5621d6134070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.706+0000 7f72a3400640 1 -- 172.21.15.47:0/4157720215 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621d6134660 con 0x5621d6123810 2024-03-21T11:37:25.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.707+0000 7f72a3400640 1 --2- 172.21.15.47:0/4157720215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d6132b20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f729401a8b0 tx=0x7f729401a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=134dfbfe7c7e6885 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.707+0000 7f728d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:37:25.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.707+0000 7f728d600640 1 -- 172.21.15.47:0/4157720215 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729401abe0 con 0x5621d6123810 2024-03-21T11:37:25.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.707+0000 7f72ad26b740 1 -- 172.21.15.47:0/4157720215 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72900fe070 con 0x5621d6123810 2024-03-21T11:37:25.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.707+0000 7f728d600640 1 -- 172.21.15.47:0/4157720215 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7294010c00 con 0x5621d6123810 2024-03-21T11:37:25.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.707+0000 7f728d600640 1 -- 172.21.15.47:0/4157720215 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7294011440 con 0x5621d6123810 2024-03-21T11:37:25.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.707+0000 7f72ad26b740 10 client.30232 fetch_fsmap learned FSMap version 32 2024-03-21T11:37:25.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.707+0000 7f72ad26b740 1 -- 172.21.15.47:0/4157720215 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5621d6195310 con 0x5621d6123810 2024-03-21T11:37:25.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.707+0000 7f728d600640 1 -- 172.21.15.47:0/4157720215 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f72900fe070 con 0x5621d6123810 2024-03-21T11:37:25.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.708+0000 7f728d600640 1 -- 172.21.15.47:0/4157720215 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f72940323e0 con 0x5621d6123810 2024-03-21T11:37:25.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.708+0000 7f72ad26b740 10 client.30232 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T11:37:25.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.708+0000 7f72ad26b740 10 client.30232 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:37:25.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.708+0000 7f72ad26b740 20 client.30232 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:25.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.708+0000 7f72ad26b740 1 --2- 172.21.15.47:0/4157720215 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5621d6195c80 0x5621d61980a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.708+0000 7f72ad26b740 4 client.30232 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:37:25.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.708+0000 7f72ad26b740 1 -- 172.21.15.47:0/4157720215 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5621d608fb50 con 0x5621d6195c80 2024-03-21T11:37:25.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.708+0000 7f72a2a00640 1 --2- 172.21.15.47:0/4157720215 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5621d6195c80 0x5621d61980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f72a2a00640 1 --2- 172.21.15.47:0/4157720215 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5621d6195c80 0x5621d61980a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8c83c1de0a4cffd9 server_cookie=e96999610206ba99 in_seq=0 out_seq=0 2024-03-21T11:37:25.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f728d600640 10 client.30232 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:37:25.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f728d600640 1 -- 172.21.15.47:0/4157720215 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5621d608fb50 con 0x5621d6195c80 2024-03-21T11:37:25.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f728d600640 10 client.30232 handle_command_reply: tid=0 2024-03-21T11:37:25.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f72ad26b740 1 client.30232 shutdown 2024-03-21T11:37:25.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f72ad26b740 1 -- 172.21.15.47:0/4157720215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 msgr2=0x5621d6132b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f72ad26b740 1 --2- 172.21.15.47:0/4157720215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d6132b20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f729401a8b0 tx=0x7f729401a610 comp rx=0 tx=0).stop 2024-03-21T11:37:25.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f72ad26b740 1 -- 172.21.15.47:0/4157720215 shutdown_connections 2024-03-21T11:37:25.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f72ad26b740 1 --2- 172.21.15.47:0/4157720215 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5621d6195c80 0x5621d61980a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f72ad26b740 1 --2- 172.21.15.47:0/4157720215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621d61253a0 0x5621d6134070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f72ad26b740 1 --2- 172.21.15.47:0/4157720215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621d61249e0 0x5621d6133060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f728d600640 0 client.30232 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:37:25.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f72ad26b740 1 --2- 172.21.15.47:0/4157720215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621d6123810 0x5621d6132b20 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f72ad26b740 1 -- 172.21.15.47:0/4157720215 >> 172.21.15.47:0/4157720215 conn(0x5621d611ec00 msgr2=0x5621d611d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f72ad26b740 1 -- 172.21.15.47:0/4157720215 shutdown_connections 2024-03-21T11:37:25.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.709+0000 7f72ad26b740 1 -- 172.21.15.47:0/4157720215 wait complete. 2024-03-21T11:37:25.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.710+0000 7f72ad26b740 20 client.30232 trim_cache size 0 max 16384 2024-03-21T11:37:25.714 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:37:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.710+0000 7f72aa600640 1 -- 172.21.15.47:0/689981709 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72880a3020 msgr2=0x7f72880a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.710+0000 7f72aa600640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72880a3020 0x7f72880a54e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x5621d61248e0 tx=0x7f729c009500 comp rx=0 tx=0).stop 2024-03-21T11:37:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.710+0000 7f72aa600640 1 -- 172.21.15.47:0/689981709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72a4108eb0 msgr2=0x7f72a419e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.710+0000 7f72aa600640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72a4108eb0 0x7f72a419e4d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7290009230 tx=0x7f729002c6c0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.710+0000 7f72aa600640 1 -- 172.21.15.47:0/689981709 shutdown_connections 2024-03-21T11:37:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.710+0000 7f72aa600640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f72880a3020 0x7f72880a54e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.710+0000 7f72aa600640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72a4109870 0x7f72a41a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.710+0000 7f72aa600640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72a4108eb0 0x7f72a419e4d0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.710+0000 7f72aa600640 1 --2- 172.21.15.47:0/689981709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a4107cc0 0x7f72a419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.710+0000 7f72aa600640 1 -- 172.21.15.47:0/689981709 >> 172.21.15.47:0/689981709 conn(0x7f72a4076c60 msgr2=0x7f72a410c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.711+0000 7f72aa600640 1 -- 172.21.15.47:0/689981709 shutdown_connections 2024-03-21T11:37:25.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.711+0000 7f72aa600640 1 -- 172.21.15.47:0/689981709 wait complete. 2024-03-21T11:37:25.730 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:37:25.730 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:37:25.917 INFO:tasks.workunit.client.0.smithi047.stdout: 10 5417 51.96 MB/sec warmup 9 sec latency 124.715 ms 2024-03-21T11:37:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.967+0000 7fcbefe00640 1 -- 172.21.15.47:0/15292118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe81038e0 msgr2=0x7fcbe8103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.967+0000 7fcbefe00640 1 --2- 172.21.15.47:0/15292118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe81038e0 0x7fcbe8103d00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009ca0 tx=0x7fcbdc02c0f0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.968+0000 7fcbefe00640 1 -- 172.21.15.47:0/15292118 shutdown_connections 2024-03-21T11:37:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.968+0000 7fcbefe00640 1 --2- 172.21.15.47:0/15292118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbe8105490 0x7fcbe810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.968+0000 7fcbefe00640 1 --2- 172.21.15.47:0/15292118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbe8104ad0 0x7fcbe8104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.968+0000 7fcbefe00640 1 --2- 172.21.15.47:0/15292118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe81038e0 0x7fcbe8103d00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.968+0000 7fcbefe00640 1 -- 172.21.15.47:0/15292118 >> 172.21.15.47:0/15292118 conn(0x7fcbe80fee50 msgr2=0x7fcbe8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.968+0000 7fcbefe00640 1 -- 172.21.15.47:0/15292118 shutdown_connections 2024-03-21T11:37:25.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.969+0000 7fcbefe00640 1 -- 172.21.15.47:0/15292118 wait complete. 2024-03-21T11:37:25.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.970+0000 7fcbefe00640 1 Processor -- start 2024-03-21T11:37:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.971+0000 7fcbefe00640 1 -- start start 2024-03-21T11:37:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.971+0000 7fcbefe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbe81038e0 0x7fcbe8072c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.971+0000 7fcbefe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbe8104ad0 0x7fcbe80731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.971+0000 7fcbefe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe8105490 0x7fcbe80712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.971+0000 7fcbefe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe806bb10 con 0x7fcbe8105490 2024-03-21T11:37:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.971+0000 7fcbefe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe806b990 con 0x7fcbe81038e0 2024-03-21T11:37:25.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.971+0000 7fcbefe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe806bc90 con 0x7fcbe8104ad0 2024-03-21T11:37:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.972+0000 7fcbede00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe8105490 0x7fcbe80712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.972+0000 7fcbed400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbe81038e0 0x7fcbe8072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.973+0000 7fcbede00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe8105490 0x7fcbe80712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59034/0 (socket says 172.21.15.47:59034) 2024-03-21T11:37:25.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.973+0000 7fcbede00640 1 -- 172.21.15.47:0/3711554837 learned_addr learned my addr 172.21.15.47:0/3711554837 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:25.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.974+0000 7fcbede00640 1 -- 172.21.15.47:0/3711554837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbe8104ad0 msgr2=0x7fcbe80731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.974+0000 7fcbede00640 1 --2- 172.21.15.47:0/3711554837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbe8104ad0 0x7fcbe80731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.974+0000 7fcbede00640 1 -- 172.21.15.47:0/3711554837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbe81038e0 msgr2=0x7fcbe8072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.974+0000 7fcbede00640 1 --2- 172.21.15.47:0/3711554837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbe81038e0 0x7fcbe8072c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.974+0000 7fcbede00640 1 -- 172.21.15.47:0/3711554837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe8071aa0 con 0x7fcbe8105490 2024-03-21T11:37:25.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.974+0000 7fcbede00640 1 --2- 172.21.15.47:0/3711554837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe8105490 0x7fcbe80712a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000e970 tx=0x7fcbe000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.975+0000 7fcbe6000640 1 -- 172.21.15.47:0/3711554837 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe000ccc0 con 0x7fcbe8105490 2024-03-21T11:37:25.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.975+0000 7fcbe6000640 1 -- 172.21.15.47:0/3711554837 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbe000ce60 con 0x7fcbe8105490 2024-03-21T11:37:25.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.975+0000 7fcbe6000640 1 -- 172.21.15.47:0/3711554837 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0010490 con 0x7fcbe8105490 2024-03-21T11:37:25.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.975+0000 7fcbefe00640 1 -- 172.21.15.47:0/3711554837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe8071d90 con 0x7fcbe8105490 2024-03-21T11:37:25.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.975+0000 7fcbefe00640 1 -- 172.21.15.47:0/3711554837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe81aaa30 con 0x7fcbe8105490 2024-03-21T11:37:25.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.978+0000 7fcbe6000640 1 -- 172.21.15.47:0/3711554837 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcbe0005020 con 0x7fcbe8105490 2024-03-21T11:37:25.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.979+0000 7fcbf2a50740 1 Processor -- start 2024-03-21T11:37:25.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.979+0000 7fcbf2a50740 1 -- start start 2024-03-21T11:37:25.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.980+0000 7fcbf2a50740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b78810 0x55e599b78c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.980+0000 7fcbf2a50740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b799e0 0x55e599b79e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.980+0000 7fcbf2a50740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b7a3a0 0x55e599b80c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.980+0000 7fcbf2a50740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e599add2c0 con 0x55e599b7a3a0 2024-03-21T11:37:25.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.980+0000 7fcbf2a50740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e599a85c90 con 0x55e599b78810 2024-03-21T11:37:25.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.980+0000 7fcbf2a50740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e599ad9360 con 0x55e599b799e0 2024-03-21T11:37:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.980+0000 7fcbed400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b799e0 0x55e599b79e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.980+0000 7fcbe6000640 1 --2- 172.21.15.47:0/3711554837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd00a2fa0 0x7fcbd00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.980+0000 7fcbed400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b799e0 0x55e599b79e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56830/0 (socket says 172.21.15.47:56830) 2024-03-21T11:37:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.980+0000 7fcbed400640 1 -- 172.21.15.47:0/3201977844 learned_addr learned my addr 172.21.15.47:0/3201977844 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.980+0000 7fcbeca00640 1 --2- 172.21.15.47:0/3201977844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b78810 0x55e599b78c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.980+0000 7fcbeca00640 1 --2- 172.21.15.47:0/3201977844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b7a3a0 0x55e599b80c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.980+0000 7fcbe6000640 1 -- 172.21.15.47:0/3711554837 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcbe00cfd20 con 0x7fcbe8105490 2024-03-21T11:37:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.981+0000 7fcbed400640 1 -- 172.21.15.47:0/3201977844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b78810 msgr2=0x55e599b78c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.981+0000 7fcbed400640 1 --2- 172.21.15.47:0/3201977844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b78810 0x55e599b78c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.981+0000 7fcbed400640 1 -- 172.21.15.47:0/3201977844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b7a3a0 msgr2=0x55e599b80c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.981+0000 7fcbed400640 1 --2- 172.21.15.47:0/3201977844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b7a3a0 0x55e599b80c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.981+0000 7fcbed400640 1 -- 172.21.15.47:0/3201977844 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e599aaa080 con 0x55e599b799e0 2024-03-21T11:37:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.981+0000 7fcbede00640 1 --2- 172.21.15.47:0/3711554837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd00a2fa0 0x7fcbd00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.981+0000 7fcbeca00640 1 --2- 172.21.15.47:0/3201977844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b7a3a0 0x55e599b80c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:37:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.981+0000 7fcbed400640 1 --2- 172.21.15.47:0/3201977844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b799e0 0x55e599b79e60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc02c700 tx=0x7fcbdc005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c68ae564778c18b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.982+0000 7fcbda600640 1 -- 172.21.15.47:0/3201977844 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc054020 con 0x55e599b799e0 2024-03-21T11:37:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.982+0000 7fcbda600640 1 -- 172.21.15.47:0/3201977844 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbdc032c90 con 0x55e599b799e0 2024-03-21T11:37:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.982+0000 7fcbda600640 1 -- 172.21.15.47:0/3201977844 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc039040 con 0x55e599b799e0 2024-03-21T11:37:25.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.983+0000 7fcbede00640 1 --2- 172.21.15.47:0/3711554837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd00a2fa0 0x7fcbd00a5460 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0004a00 tx=0x7fcbe0004950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.984+0000 7fcbf2a50740 1 -- 172.21.15.47:0/3201977844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b799e0 msgr2=0x55e599b79e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.984+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/3201977844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b799e0 0x55e599b79e60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc02c700 tx=0x7fcbdc005ef0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.984+0000 7fcbf2a50740 1 -- 172.21.15.47:0/3201977844 shutdown_connections 2024-03-21T11:37:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.984+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/3201977844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b7a3a0 0x55e599b80c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.984+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/3201977844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b799e0 0x55e599b79e60 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.984+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/3201977844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b78810 0x55e599b78c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.984+0000 7fcbf2a50740 1 -- 172.21.15.47:0/3201977844 >> 172.21.15.47:0/3201977844 conn(0x55e599b73c00 msgr2=0x55e599b74000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.984+0000 7fcbf2a50740 1 -- 172.21.15.47:0/3201977844 shutdown_connections 2024-03-21T11:37:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.984+0000 7fcbf2a50740 1 -- 172.21.15.47:0/3201977844 wait complete. 2024-03-21T11:37:25.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.985+0000 7fcbf2a50740 1 -- start start 2024-03-21T11:37:25.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.985+0000 7fcbf2a50740 1 Processor -- start 2024-03-21T11:37:25.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbf2a50740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 0x55e599b82ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbf2a50740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b799e0 0x55e599b82ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbf2a50740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b7a3a0 0x55e599b8a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbf2a50740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e599b81350 con 0x55e599b7a3a0 2024-03-21T11:37:25.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbf2a50740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e599a990d0 con 0x55e599b799e0 2024-03-21T11:37:25.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbf2a50740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e599b814d0 con 0x55e599b78810 2024-03-21T11:37:25.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbf2a50740 10 client.0 _ll_register_callbacks cb 0x55e599a9de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:25.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbeca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 0x55e599b82ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbeca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 0x55e599b82ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56840/0 (socket says 172.21.15.47:56840) 2024-03-21T11:37:25.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbeca00640 1 -- 172.21.15.47:0/2502504136 learned_addr learned my addr 172.21.15.47:0/2502504136 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:25.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbed400640 1 --2- 172.21.15.47:0/2502504136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b799e0 0x55e599b82ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbeca00640 1 --2- 172.21.15.47:0/2502504136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b7a3a0 0x55e599b8a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbeca00640 1 -- 172.21.15.47:0/2502504136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b799e0 msgr2=0x55e599b82ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbeca00640 1 --2- 172.21.15.47:0/2502504136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b799e0 0x55e599b82ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbeca00640 1 -- 172.21.15.47:0/2502504136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b7a3a0 msgr2=0x55e599b8a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:25.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbeca00640 1 --2- 172.21.15.47:0/2502504136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b7a3a0 0x55e599b8a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.986+0000 7fcbeca00640 1 -- 172.21.15.47:0/2502504136 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5997f13f0 con 0x55e599b78810 2024-03-21T11:37:25.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.987+0000 7fcbeca00640 1 --2- 172.21.15.47:0/2502504136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 0x55e599b82ab0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400a000 tx=0x7fcbd4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c430860f8aa6dae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:25.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.987+0000 7fcbcea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:37:25.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.988+0000 7fcbcea00640 1 -- 172.21.15.47:0/2502504136 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd400eb60 con 0x55e599b78810 2024-03-21T11:37:25.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.988+0000 7fcbf2a50740 1 -- 172.21.15.47:0/2502504136 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbdc032c90 con 0x55e599b78810 2024-03-21T11:37:25.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.988+0000 7fcbcea00640 1 -- 172.21.15.47:0/2502504136 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcbd400ed00 con 0x55e599b78810 2024-03-21T11:37:25.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.988+0000 7fcbcea00640 1 -- 172.21.15.47:0/2502504136 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbd4028050 con 0x55e599b78810 2024-03-21T11:37:25.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.988+0000 7fcbf2a50740 10 client.30241 fetch_fsmap learned FSMap version 33 2024-03-21T11:37:25.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.989+0000 7fcbf2a50740 1 -- 172.21.15.47:0/2502504136 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e5998bb7d0 con 0x55e599b78810 2024-03-21T11:37:25.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.989+0000 7fcbcea00640 1 -- 172.21.15.47:0/2502504136 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbdc032c90 con 0x55e599b78810 2024-03-21T11:37:25.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.989+0000 7fcbcea00640 1 -- 172.21.15.47:0/2502504136 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fcbd4010720 con 0x55e599b78810 2024-03-21T11:37:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.989+0000 7fcbf2a50740 10 client.30241 fetch_fsmap finished waiting for FSMap version 33 2024-03-21T11:37:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.989+0000 7fcbf2a50740 10 client.30241 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:37:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.989+0000 7fcbf2a50740 20 client.30241 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.989+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/2502504136 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e599b8e5a0 0x55e599b90a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.989+0000 7fcbf2a50740 4 client.30241 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:37:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.989+0000 7fcbf2a50740 1 -- 172.21.15.47:0/2502504136 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e599aea800 con 0x55e599b8e5a0 2024-03-21T11:37:25.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.993+0000 7fcbed400640 1 --2- 172.21.15.47:0/2502504136 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e599b8e5a0 0x55e599b90a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.995+0000 7fcbed400640 1 --2- 172.21.15.47:0/2502504136 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e599b8e5a0 0x55e599b90a20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f086a4ca6d53747 server_cookie=4aa89179d72edbb in_seq=0 out_seq=0 2024-03-21T11:37:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.995+0000 7fcbcea00640 10 client.30241 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:37:25.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.996+0000 7fcbcea00640 1 -- 172.21.15.47:0/2502504136 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e599aea800 con 0x55e599b8e5a0 2024-03-21T11:37:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.996+0000 7fcbcea00640 10 client.30241 handle_command_reply: tid=0 2024-03-21T11:37:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.996+0000 7fcbf2a50740 1 client.30241 shutdown 2024-03-21T11:37:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.996+0000 7fcbf2a50740 1 -- 172.21.15.47:0/2502504136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 msgr2=0x55e599b82ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.996+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/2502504136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 0x55e599b82ab0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400a000 tx=0x7fcbd4005f90 comp rx=0 tx=0).stop 2024-03-21T11:37:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.997+0000 7fcbf2a50740 1 -- 172.21.15.47:0/2502504136 shutdown_connections 2024-03-21T11:37:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.997+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/2502504136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b7a3a0 0x55e599b8a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.997+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/2502504136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b799e0 0x55e599b82ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.997+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/2502504136 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e599b8e5a0 0x55e599b90a20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.997+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/2502504136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 0x55e599b82ab0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.997+0000 7fcbf2a50740 1 -- 172.21.15.47:0/2502504136 >> 172.21.15.47:0/2502504136 conn(0x55e599b73c00 msgr2=0x55e599b72eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.997+0000 7fcbcea00640 0 client.30241 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:37:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.997+0000 7fcbf2a50740 1 -- 172.21.15.47:0/2502504136 shutdown_connections 2024-03-21T11:37:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.997+0000 7fcbf2a50740 1 -- 172.21.15.47:0/2502504136 wait complete. 2024-03-21T11:37:25.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:25.997+0000 7fcbf2a50740 20 client.30241 trim_cache size 0 max 16384 2024-03-21T11:37:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.028+0000 7fcbf2a50740 1 Processor -- start 2024-03-21T11:37:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.028+0000 7fcbf2a50740 1 -- start start 2024-03-21T11:37:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.028+0000 7fcbf2a50740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 0x55e599b82f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.028+0000 7fcbf2a50740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b799e0 0x55e599b89ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.028+0000 7fcbf2a50740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b7a3a0 0x55e599b72620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.029+0000 7fcbf2a50740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe81b0760 con 0x55e599b799e0 2024-03-21T11:37:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.029+0000 7fcbf2a50740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe81b2260 con 0x55e599b7a3a0 2024-03-21T11:37:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.029+0000 7fcbf2a50740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe81b0dc0 con 0x55e599b78810 2024-03-21T11:37:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.029+0000 7fcbeca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b7a3a0 0x55e599b72620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.029+0000 7fcbeca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b7a3a0 0x55e599b72620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41720/0 (socket says 172.21.15.47:41720) 2024-03-21T11:37:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.029+0000 7fcbeca00640 1 -- 172.21.15.47:0/4196745614 learned_addr learned my addr 172.21.15.47:0/4196745614 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.029+0000 7fcbed400640 1 --2- 172.21.15.47:0/4196745614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b799e0 0x55e599b89ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.029+0000 7fcbeca00640 1 -- 172.21.15.47:0/4196745614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 msgr2=0x55e599b82f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.029+0000 7fcbeca00640 1 --2- 172.21.15.47:0/4196745614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 0x55e599b82f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.029+0000 7fcbeca00640 1 -- 172.21.15.47:0/4196745614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b799e0 msgr2=0x55e599b89ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.029+0000 7fcbeca00640 1 --2- 172.21.15.47:0/4196745614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b799e0 0x55e599b89ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.029+0000 7fcbeca00640 1 -- 172.21.15.47:0/4196745614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e599b815e0 con 0x55e599b7a3a0 2024-03-21T11:37:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.030+0000 7fcbeca00640 1 --2- 172.21.15.47:0/4196745614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b7a3a0 0x55e599b72620 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4010bc0 tx=0x7fcbd40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bfac071c6f709d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:26.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.031+0000 7fcbd9c00640 1 -- 172.21.15.47:0/4196745614 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4005850 con 0x55e599b7a3a0 2024-03-21T11:37:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.031+0000 7fcbd9c00640 1 -- 172.21.15.47:0/4196745614 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbd4009a80 con 0x55e599b7a3a0 2024-03-21T11:37:26.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.031+0000 7fcbd9c00640 1 -- 172.21.15.47:0/4196745614 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd400e9a0 con 0x55e599b7a3a0 2024-03-21T11:37:26.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.032+0000 7fcbf2a50740 1 -- 172.21.15.47:0/4196745614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b7a3a0 msgr2=0x55e599b72620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.032+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/4196745614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b7a3a0 0x55e599b72620 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4010bc0 tx=0x7fcbd40046c0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.032+0000 7fcbf2a50740 1 -- 172.21.15.47:0/4196745614 shutdown_connections 2024-03-21T11:37:26.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.032+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/4196745614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b7a3a0 0x55e599b72620 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.032+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/4196745614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b799e0 0x55e599b89ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.032+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/4196745614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 0x55e599b82f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.032+0000 7fcbf2a50740 1 -- 172.21.15.47:0/4196745614 >> 172.21.15.47:0/4196745614 conn(0x55e599b73c00 msgr2=0x55e599b81ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.033+0000 7fcbf2a50740 1 -- 172.21.15.47:0/4196745614 shutdown_connections 2024-03-21T11:37:26.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.033+0000 7fcbf2a50740 1 -- 172.21.15.47:0/4196745614 wait complete. 2024-03-21T11:37:26.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.034+0000 7fcbf2a50740 1 -- start start 2024-03-21T11:37:26.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.034+0000 7fcbf2a50740 1 Processor -- start 2024-03-21T11:37:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.035+0000 7fcbf2a50740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 0x55e599b85210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.035+0000 7fcbf2a50740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b799e0 0x55e599b85750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.035+0000 7fcbf2a50740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b7a3a0 0x55e599b88770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.035+0000 7fcbf2a50740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe81b1b40 con 0x55e599b799e0 2024-03-21T11:37:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.035+0000 7fcbf2a50740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe81b14e0 con 0x55e599b7a3a0 2024-03-21T11:37:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.035+0000 7fcbf2a50740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe81ae940 con 0x55e599b78810 2024-03-21T11:37:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.035+0000 7fcbf2a50740 10 client.0 _ll_register_callbacks cb 0x7fcbd40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.036+0000 7fcbeca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b7a3a0 0x55e599b88770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.036+0000 7fcbeca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 0x55e599b85210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.036+0000 7fcbeca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b7a3a0 0x55e599b88770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41726/0 (socket says 172.21.15.47:41726) 2024-03-21T11:37:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.036+0000 7fcbeca00640 1 -- 172.21.15.47:0/68672832 learned_addr learned my addr 172.21.15.47:0/68672832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.036+0000 7fcbed400640 1 --2- 172.21.15.47:0/68672832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b799e0 0x55e599b85750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.037+0000 7fcbeca00640 1 -- 172.21.15.47:0/68672832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 msgr2=0x55e599b85210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.037+0000 7fcbeca00640 1 --2- 172.21.15.47:0/68672832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 0x55e599b85210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.037+0000 7fcbeca00640 1 -- 172.21.15.47:0/68672832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b799e0 msgr2=0x55e599b85750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.037+0000 7fcbeca00640 1 --2- 172.21.15.47:0/68672832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b799e0 0x55e599b85750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.037+0000 7fcbeca00640 1 -- 172.21.15.47:0/68672832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e599b88d60 con 0x55e599b7a3a0 2024-03-21T11:37:26.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.037+0000 7fcbed400640 1 --2- 172.21.15.47:0/68672832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b799e0 0x55e599b85750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:37:26.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.038+0000 7fcbeca00640 1 --2- 172.21.15.47:0/68672832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b7a3a0 0x55e599b88770 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009740 tx=0x7fcbd4013980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b73fed44af04eb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:26.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.038+0000 7fcbcea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:37:26.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.038+0000 7fcbcea00640 1 -- 172.21.15.47:0/68672832 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4010830 con 0x55e599b7a3a0 2024-03-21T11:37:26.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.038+0000 7fcbcea00640 1 -- 172.21.15.47:0/68672832 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcbd400e960 con 0x55e599b7a3a0 2024-03-21T11:37:26.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.038+0000 7fcbf2a50740 1 -- 172.21.15.47:0/68672832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbd4009a80 con 0x55e599b7a3a0 2024-03-21T11:37:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.039+0000 7fcbcea00640 1 -- 172.21.15.47:0/68672832 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbd400eb60 con 0x55e599b7a3a0 2024-03-21T11:37:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.039+0000 7fcbf2a50740 10 client.30224 fetch_fsmap learned FSMap version 33 2024-03-21T11:37:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.039+0000 7fcbf2a50740 1 -- 172.21.15.47:0/68672832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e599b8a060 con 0x55e599b7a3a0 2024-03-21T11:37:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.040+0000 7fcbcea00640 1 -- 172.21.15.47:0/68672832 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbd4009a80 con 0x55e599b7a3a0 2024-03-21T11:37:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.041+0000 7fcbcea00640 1 -- 172.21.15.47:0/68672832 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fcbd40243f0 con 0x55e599b7a3a0 2024-03-21T11:37:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.041+0000 7fcbf2a50740 10 client.30224 fetch_fsmap finished waiting for FSMap version 33 2024-03-21T11:37:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.041+0000 7fcbf2a50740 10 client.30224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:37:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.041+0000 7fcbf2a50740 20 client.30224 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.041+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/68672832 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e599bea2a0 0x55e599bec690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.041+0000 7fcbf2a50740 4 client.30224 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:37:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.041+0000 7fcbf2a50740 1 -- 172.21.15.47:0/68672832 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e599ae4b50 con 0x55e599bea2a0 2024-03-21T11:37:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.041+0000 7fcbed400640 1 --2- 172.21.15.47:0/68672832 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e599bea2a0 0x55e599bec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.042+0000 7fcbcea00640 10 client.30224 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:37:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.042+0000 7fcbed400640 1 --2- 172.21.15.47:0/68672832 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e599bea2a0 0x55e599bec690 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f641302a8aa24d9 server_cookie=8c58ee4293a3423a in_seq=0 out_seq=0 2024-03-21T11:37:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.042+0000 7fcbcea00640 1 -- 172.21.15.47:0/68672832 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e599ae4b50 con 0x55e599bea2a0 2024-03-21T11:37:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.042+0000 7fcbcea00640 10 client.30224 handle_command_reply: tid=0 2024-03-21T11:37:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.043+0000 7fcbf2a50740 1 client.30224 shutdown 2024-03-21T11:37:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.044+0000 7fcbf2a50740 1 -- 172.21.15.47:0/68672832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b7a3a0 msgr2=0x55e599b88770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.044+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/68672832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b7a3a0 0x55e599b88770 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009740 tx=0x7fcbd4013980 comp rx=0 tx=0).stop 2024-03-21T11:37:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.044+0000 7fcbf2a50740 1 -- 172.21.15.47:0/68672832 shutdown_connections 2024-03-21T11:37:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.044+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/68672832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e599b7a3a0 0x55e599b88770 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.044+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/68672832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e599b799e0 0x55e599b85750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.044+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/68672832 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e599bea2a0 0x55e599bec690 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.044+0000 7fcbf2a50740 1 --2- 172.21.15.47:0/68672832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e599b78810 0x55e599b85210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.044+0000 7fcbf2a50740 1 -- 172.21.15.47:0/68672832 >> 172.21.15.47:0/68672832 conn(0x55e599b73c00 msgr2=0x55e599b76e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.044+0000 7fcbcea00640 0 client.30224 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:37:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.044+0000 7fcbf2a50740 1 -- 172.21.15.47:0/68672832 shutdown_connections 2024-03-21T11:37:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.044+0000 7fcbf2a50740 1 -- 172.21.15.47:0/68672832 wait complete. 2024-03-21T11:37:26.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.045+0000 7fcbf2a50740 20 client.30224 trim_cache size 0 max 16384 2024-03-21T11:37:26.049 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:37:26.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.045+0000 7fcbefe00640 1 -- 172.21.15.47:0/3711554837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd00a2fa0 msgr2=0x7fcbd00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.045+0000 7fcbefe00640 1 --2- 172.21.15.47:0/3711554837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd00a2fa0 0x7fcbd00a5460 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0004a00 tx=0x7fcbe0004950 comp rx=0 tx=0).stop 2024-03-21T11:37:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.045+0000 7fcbefe00640 1 -- 172.21.15.47:0/3711554837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe8105490 msgr2=0x7fcbe80712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.045+0000 7fcbefe00640 1 --2- 172.21.15.47:0/3711554837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe8105490 0x7fcbe80712a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000e970 tx=0x7fcbe000ed70 comp rx=0 tx=0).stop 2024-03-21T11:37:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.046+0000 7fcbefe00640 1 -- 172.21.15.47:0/3711554837 shutdown_connections 2024-03-21T11:37:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.046+0000 7fcbefe00640 1 --2- 172.21.15.47:0/3711554837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd00a2fa0 0x7fcbd00a5460 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.046+0000 7fcbefe00640 1 --2- 172.21.15.47:0/3711554837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe8105490 0x7fcbe80712a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.046+0000 7fcbefe00640 1 --2- 172.21.15.47:0/3711554837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbe8104ad0 0x7fcbe80731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.046+0000 7fcbefe00640 1 --2- 172.21.15.47:0/3711554837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbe81038e0 0x7fcbe8072c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.046+0000 7fcbefe00640 1 -- 172.21.15.47:0/3711554837 >> 172.21.15.47:0/3711554837 conn(0x7fcbe80fee50 msgr2=0x7fcbe8107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.047+0000 7fcbefe00640 1 -- 172.21.15.47:0/3711554837 shutdown_connections 2024-03-21T11:37:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.047+0000 7fcbefe00640 1 -- 172.21.15.47:0/3711554837 wait complete. 2024-03-21T11:37:26.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:25 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3105749636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:37:26.074 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:37:26.075 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:37:26.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:25 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3105749636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:37:26.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:25 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3105749636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:37:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.379+0000 7fec5b200640 1 -- 172.21.15.47:0/2330537133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec54105490 msgr2=0x7fec5410bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.379+0000 7fec5b200640 1 --2- 172.21.15.47:0/2330537133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec54105490 0x7fec5410bd20 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009ca0 tx=0x7fec4c02c100 comp rx=0 tx=0).stop 2024-03-21T11:37:26.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.379+0000 7fec5b200640 1 -- 172.21.15.47:0/2330537133 shutdown_connections 2024-03-21T11:37:26.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.379+0000 7fec5b200640 1 --2- 172.21.15.47:0/2330537133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec54105490 0x7fec5410bd20 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.379+0000 7fec5b200640 1 --2- 172.21.15.47:0/2330537133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec54104ad0 0x7fec54104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.379+0000 7fec5b200640 1 --2- 172.21.15.47:0/2330537133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec541038e0 0x7fec54103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.379+0000 7fec5b200640 1 -- 172.21.15.47:0/2330537133 >> 172.21.15.47:0/2330537133 conn(0x7fec540fee50 msgr2=0x7fec54101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.380+0000 7fec5b200640 1 -- 172.21.15.47:0/2330537133 shutdown_connections 2024-03-21T11:37:26.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.380+0000 7fec5b200640 1 -- 172.21.15.47:0/2330537133 wait complete. 2024-03-21T11:37:26.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.381+0000 7fec5b200640 1 Processor -- start 2024-03-21T11:37:26.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.381+0000 7fec5b200640 1 -- start start 2024-03-21T11:37:26.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.382+0000 7fec5b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec541038e0 0x7fec5419e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.382+0000 7fec5b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec54104ad0 0x7fec5419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.383+0000 7fec5b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec54105490 0x7fec541a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.383+0000 7fec5b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5410f5d0 con 0x7fec54105490 2024-03-21T11:37:26.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.383+0000 7fec5b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5410f450 con 0x7fec541038e0 2024-03-21T11:37:26.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.383+0000 7fec5b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5410f750 con 0x7fec54104ad0 2024-03-21T11:37:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.383+0000 7fec59200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec54105490 0x7fec541a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.383+0000 7fec53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec541038e0 0x7fec5419e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.383+0000 7fec59200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec54105490 0x7fec541a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59078/0 (socket says 172.21.15.47:59078) 2024-03-21T11:37:26.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.383+0000 7fec59200640 1 -- 172.21.15.47:0/1576944645 learned_addr learned my addr 172.21.15.47:0/1576944645 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.384+0000 7fec53e00640 1 -- 172.21.15.47:0/1576944645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec54104ad0 msgr2=0x7fec5419e5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T11:37:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.384+0000 7fec53e00640 1 --2- 172.21.15.47:0/1576944645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec54104ad0 0x7fec5419e5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.384+0000 7fec53e00640 1 -- 172.21.15.47:0/1576944645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec54105490 msgr2=0x7fec541a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.384+0000 7fec53e00640 1 --2- 172.21.15.47:0/1576944645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec54105490 0x7fec541a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.384+0000 7fec53e00640 1 -- 172.21.15.47:0/1576944645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec541a5c80 con 0x7fec541038e0 2024-03-21T11:37:26.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.384+0000 7fec59200640 1 --2- 172.21.15.47:0/1576944645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec54105490 0x7fec541a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.385+0000 7fec53e00640 1 --2- 172.21.15.47:0/1576944645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec541038e0 0x7fec5419e060 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fec4400b3c0 tx=0x7fec4400b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:26.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.386+0000 7fec50c00640 1 -- 172.21.15.47:0/1576944645 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec44015020 con 0x7fec541038e0 2024-03-21T11:37:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.386+0000 7fec50c00640 1 -- 172.21.15.47:0/1576944645 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec44011d10 con 0x7fec541038e0 2024-03-21T11:37:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.386+0000 7fec50c00640 1 -- 172.21.15.47:0/1576944645 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4400f2e0 con 0x7fec541038e0 2024-03-21T11:37:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.389+0000 7fec5b200640 1 -- 172.21.15.47:0/1576944645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec541a5f70 con 0x7fec541038e0 2024-03-21T11:37:26.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.389+0000 7fec5b200640 1 -- 172.21.15.47:0/1576944645 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec541a6400 con 0x7fec541038e0 2024-03-21T11:37:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.397+0000 7fec50c00640 1 -- 172.21.15.47:0/1576944645 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fec44018020 con 0x7fec541038e0 2024-03-21T11:37:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.399+0000 7fec5df79740 1 Processor -- start 2024-03-21T11:37:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.400+0000 7fec50c00640 1 --2- 172.21.15.47:0/1576944645 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fec380a3120 0x7fec380a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.401+0000 7fec50c00640 1 -- 172.21.15.47:0/1576944645 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fec440ced40 con 0x7fec541038e0 2024-03-21T11:37:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.401+0000 7fec5df79740 1 -- start start 2024-03-21T11:37:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.402+0000 7fec5df79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b6810 0x55ad075b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.402+0000 7fec59200640 1 --2- 172.21.15.47:0/1576944645 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fec380a3120 0x7fec380a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.402+0000 7fec5df79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 0x55ad075b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.402+0000 7fec53e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b6810 0x55ad075b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.403+0000 7fec53e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b6810 0x55ad075b6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59080/0 (socket says 172.21.15.47:59080) 2024-03-21T11:37:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.403+0000 7fec53e00640 1 -- 172.21.15.47:0/3475942898 learned_addr learned my addr 172.21.15.47:0/3475942898 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.403+0000 7fec5df79740 1 --2- 172.21.15.47:0/3475942898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b83a0 0x55ad075bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.403+0000 7fec5df79740 1 -- 172.21.15.47:0/3475942898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad0751b2c0 con 0x55ad075b6810 2024-03-21T11:37:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.403+0000 7fec5df79740 1 -- 172.21.15.47:0/3475942898 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad074c3c90 con 0x55ad075b79e0 2024-03-21T11:37:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.403+0000 7fec5df79740 1 -- 172.21.15.47:0/3475942898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad07517360 con 0x55ad075b83a0 2024-03-21T11:37:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.403+0000 7fec53400640 1 --2- 172.21.15.47:0/3475942898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 0x55ad075b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.404+0000 7fec53400640 1 -- 172.21.15.47:0/3475942898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b83a0 msgr2=0x55ad075bec30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:37:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.404+0000 7fec53400640 1 --2- 172.21.15.47:0/3475942898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b83a0 0x55ad075bec30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.404+0000 7fec53400640 1 -- 172.21.15.47:0/3475942898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b6810 msgr2=0x55ad075b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.404+0000 7fec53400640 1 --2- 172.21.15.47:0/3475942898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b6810 0x55ad075b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.404+0000 7fec53400640 1 -- 172.21.15.47:0/3475942898 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad074e8080 con 0x55ad075b79e0 2024-03-21T11:37:26.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.404+0000 7fec53400640 1 --2- 172.21.15.47:0/3475942898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 0x55ad075b7e60 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fec4000b9a0 tx=0x7fec4000be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66f23171b1511126 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.404+0000 7fec49600640 1 -- 172.21.15.47:0/3475942898 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40016070 con 0x55ad075b79e0 2024-03-21T11:37:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.404+0000 7fec49600640 1 -- 172.21.15.47:0/3475942898 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec40005500 con 0x55ad075b79e0 2024-03-21T11:37:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.404+0000 7fec49600640 1 -- 172.21.15.47:0/3475942898 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40010460 con 0x55ad075b79e0 2024-03-21T11:37:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.405+0000 7fec5df79740 1 -- 172.21.15.47:0/3475942898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 msgr2=0x55ad075b7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.405+0000 7fec5df79740 1 --2- 172.21.15.47:0/3475942898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 0x55ad075b7e60 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fec4000b9a0 tx=0x7fec4000be60 comp rx=0 tx=0).stop 2024-03-21T11:37:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.406+0000 7fec59200640 1 --2- 172.21.15.47:0/1576944645 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fec380a3120 0x7fec380a55e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fec4c032420 tx=0x7fec4c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.407+0000 7fec5df79740 1 -- 172.21.15.47:0/3475942898 shutdown_connections 2024-03-21T11:37:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.407+0000 7fec5df79740 1 --2- 172.21.15.47:0/3475942898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b83a0 0x55ad075bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.407+0000 7fec5df79740 1 --2- 172.21.15.47:0/3475942898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 0x55ad075b7e60 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.407+0000 7fec5df79740 1 --2- 172.21.15.47:0/3475942898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b6810 0x55ad075b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.407+0000 7fec5df79740 1 -- 172.21.15.47:0/3475942898 >> 172.21.15.47:0/3475942898 conn(0x55ad075b1c00 msgr2=0x55ad075b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.407+0000 7fec5df79740 1 -- 172.21.15.47:0/3475942898 shutdown_connections 2024-03-21T11:37:26.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.407+0000 7fec5df79740 1 -- 172.21.15.47:0/3475942898 wait complete. 2024-03-21T11:37:26.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.408+0000 7fec5df79740 1 -- start start 2024-03-21T11:37:26.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.409+0000 7fec5df79740 1 Processor -- start 2024-03-21T11:37:26.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.410+0000 7fec5df79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b6810 0x55ad075c0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.410+0000 7fec5df79740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b79e0 0x55ad075c0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.410+0000 7fec53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b6810 0x55ad075c0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.410+0000 7fec53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b6810 0x55ad075c0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41780/0 (socket says 172.21.15.47:41780) 2024-03-21T11:37:26.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec5df79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 0x55ad075c8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec53400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b79e0 0x55ad075c0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec5df79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad075bf400 con 0x55ad075b83a0 2024-03-21T11:37:26.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec53400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b79e0 0x55ad075c0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56904/0 (socket says 172.21.15.47:56904) 2024-03-21T11:37:26.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec53400640 1 -- 172.21.15.47:0/660487911 learned_addr learned my addr 172.21.15.47:0/660487911 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:26.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec5df79740 1 -- 172.21.15.47:0/660487911 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad074d70d0 con 0x55ad075b6810 2024-03-21T11:37:26.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec5df79740 1 -- 172.21.15.47:0/660487911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad075bf580 con 0x55ad075b79e0 2024-03-21T11:37:26.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec59200640 1 --2- 172.21.15.47:0/660487911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 0x55ad075c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec53400640 1 -- 172.21.15.47:0/660487911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b6810 msgr2=0x55ad075c0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec53400640 1 --2- 172.21.15.47:0/660487911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b6810 0x55ad075c0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec53400640 1 -- 172.21.15.47:0/660487911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 msgr2=0x55ad075c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec53400640 1 --2- 172.21.15.47:0/660487911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 0x55ad075c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec53400640 1 -- 172.21.15.47:0/660487911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad0722f3f0 con 0x55ad075b79e0 2024-03-21T11:37:26.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec59200640 1 --2- 172.21.15.47:0/660487911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 0x55ad075c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:37:26.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.412+0000 7fec53400640 1 --2- 172.21.15.47:0/660487911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b79e0 0x55ad075c0fc0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fec40005d00 tx=0x7fec40004e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a95e8bc65702619 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:26.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.411+0000 7fec5df79740 10 client.0 _ll_register_callbacks cb 0x55ad074dbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:26.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.412+0000 7fec3e000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:37:26.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.412+0000 7fec5df79740 1 -- 172.21.15.47:0/660487911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec40005500 con 0x55ad075b79e0 2024-03-21T11:37:26.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.412+0000 7fec3e000640 1 -- 172.21.15.47:0/660487911 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40016030 con 0x55ad075b79e0 2024-03-21T11:37:26.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.412+0000 7fec3e000640 1 -- 172.21.15.47:0/660487911 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fec4000d980 con 0x55ad075b79e0 2024-03-21T11:37:26.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.412+0000 7fec3e000640 1 -- 172.21.15.47:0/660487911 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec4000dba0 con 0x55ad075b79e0 2024-03-21T11:37:26.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.412+0000 7fec5df79740 10 client.30250 fetch_fsmap learned FSMap version 33 2024-03-21T11:37:26.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.412+0000 7fec5df79740 1 -- 172.21.15.47:0/660487911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=33}) v3 -- 0x55ad072f97d0 con 0x55ad075b79e0 2024-03-21T11:37:26.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.413+0000 7fec3e000640 1 -- 172.21.15.47:0/660487911 <== mon.1 v2:172.21.15.102:3300/0 4 ==== fsmap(e 33) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fec40022070 con 0x55ad075b79e0 2024-03-21T11:37:26.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.413+0000 7fec5df79740 10 client.30250 fetch_fsmap finished waiting for FSMap version 33 2024-03-21T11:37:26.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.413+0000 7fec5df79740 10 client.30250 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:37:26.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.413+0000 7fec5df79740 20 client.30250 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:26.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.413+0000 7fec5df79740 1 --2- 172.21.15.47:0/660487911 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ad075cc0c0 0x55ad075ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.413+0000 7fec5df79740 4 client.30250 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:37:26.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.413+0000 7fec5df79740 1 -- 172.21.15.47:0/660487911 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad07528800 con 0x55ad075cc0c0 2024-03-21T11:37:26.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.413+0000 7fec53e00640 1 --2- 172.21.15.47:0/660487911 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ad075cc0c0 0x55ad075ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.414+0000 7fec53e00640 1 --2- 172.21.15.47:0/660487911 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ad075cc0c0 0x55ad075ce4b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f10e79101637a9d server_cookie=fee98f2b33ca7cb0 in_seq=0 out_seq=0 2024-03-21T11:37:26.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.414+0000 7fec3e000640 10 client.30250 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:37:26.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.415+0000 7fec3e000640 1 -- 172.21.15.47:0/660487911 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ad07528800 con 0x55ad075cc0c0 2024-03-21T11:37:26.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.415+0000 7fec3e000640 10 client.30250 handle_command_reply: tid=0 2024-03-21T11:37:26.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.416+0000 7fec5df79740 1 client.30250 shutdown 2024-03-21T11:37:26.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.417+0000 7fec5df79740 1 -- 172.21.15.47:0/660487911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b79e0 msgr2=0x55ad075c0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.417+0000 7fec5df79740 1 --2- 172.21.15.47:0/660487911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b79e0 0x55ad075c0fc0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fec40005d00 tx=0x7fec40004e50 comp rx=0 tx=0).stop 2024-03-21T11:37:26.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.417+0000 7fec5df79740 1 -- 172.21.15.47:0/660487911 shutdown_connections 2024-03-21T11:37:26.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.417+0000 7fec5df79740 1 --2- 172.21.15.47:0/660487911 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ad075cc0c0 0x55ad075ce4b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.417+0000 7fec5df79740 1 --2- 172.21.15.47:0/660487911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 0x55ad075c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.417+0000 7fec5df79740 1 --2- 172.21.15.47:0/660487911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b79e0 0x55ad075c0fc0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.417+0000 7fec5df79740 1 --2- 172.21.15.47:0/660487911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b6810 0x55ad075c0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.417+0000 7fec3e000640 0 client.30250 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:37:26.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.417+0000 7fec5df79740 1 -- 172.21.15.47:0/660487911 >> 172.21.15.47:0/660487911 conn(0x55ad075b1c00 msgr2=0x55ad075b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.418+0000 7fec5df79740 1 -- 172.21.15.47:0/660487911 shutdown_connections 2024-03-21T11:37:26.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.418+0000 7fec5df79740 1 -- 172.21.15.47:0/660487911 wait complete. 2024-03-21T11:37:26.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.418+0000 7fec5df79740 20 client.30250 trim_cache size 0 max 16384 2024-03-21T11:37:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.445+0000 7fec5df79740 1 Processor -- start 2024-03-21T11:37:26.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.445+0000 7fec5df79740 1 -- start start 2024-03-21T11:37:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.445+0000 7fec5df79740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b6810 0x55ad075b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.445+0000 7fec5df79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 0x55ad075c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.445+0000 7fec5df79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 0x55ad075b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.445+0000 7fec5df79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec541b0700 con 0x55ad075b83a0 2024-03-21T11:37:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.445+0000 7fec5df79740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec541b2200 con 0x55ad075b79e0 2024-03-21T11:37:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.445+0000 7fec5df79740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec541b0d60 con 0x55ad075b6810 2024-03-21T11:37:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.448+0000 7fec53e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b6810 0x55ad075b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.448+0000 7fec53400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 0x55ad075c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.448+0000 7fec53400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 0x55ad075c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41794/0 (socket says 172.21.15.47:41794) 2024-03-21T11:37:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.448+0000 7fec53e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b6810 0x55ad075b7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56916/0 (socket says 172.21.15.47:56916) 2024-03-21T11:37:26.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.448+0000 7fec53e00640 1 -- 172.21.15.47:0/2236548628 learned_addr learned my addr 172.21.15.47:0/2236548628 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:26.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.449+0000 7fec53e00640 1 -- 172.21.15.47:0/2236548628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 msgr2=0x55ad075c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.449+0000 7fec53e00640 1 --2- 172.21.15.47:0/2236548628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 0x55ad075c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.449+0000 7fec53e00640 1 -- 172.21.15.47:0/2236548628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 msgr2=0x55ad075b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.449+0000 7fec53e00640 1 --2- 172.21.15.47:0/2236548628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 0x55ad075b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.449+0000 7fec53e00640 1 -- 172.21.15.47:0/2236548628 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad075c1a70 con 0x55ad075b6810 2024-03-21T11:37:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.450+0000 7fec53e00640 1 --2- 172.21.15.47:0/2236548628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b6810 0x55ad075b7dc0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fec4409b010 tx=0x7fec440bb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d2a076daff431d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.450+0000 7fec48c00640 1 -- 172.21.15.47:0/2236548628 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec440b4170 con 0x55ad075b6810 2024-03-21T11:37:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.451+0000 7fec48c00640 1 -- 172.21.15.47:0/2236548628 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec440b3070 con 0x55ad075b6810 2024-03-21T11:37:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.451+0000 7fec48c00640 1 -- 172.21.15.47:0/2236548628 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec440bb0b0 con 0x55ad075b6810 2024-03-21T11:37:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.451+0000 7fec5df79740 1 -- 172.21.15.47:0/2236548628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b6810 msgr2=0x55ad075b7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.451+0000 7fec5df79740 1 --2- 172.21.15.47:0/2236548628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b6810 0x55ad075b7dc0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fec4409b010 tx=0x7fec440bb040 comp rx=0 tx=0).stop 2024-03-21T11:37:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.451+0000 7fec5df79740 1 -- 172.21.15.47:0/2236548628 shutdown_connections 2024-03-21T11:37:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.451+0000 7fec5df79740 1 --2- 172.21.15.47:0/2236548628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 0x55ad075b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.451+0000 7fec5df79740 1 --2- 172.21.15.47:0/2236548628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 0x55ad075c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.451+0000 7fec5df79740 1 --2- 172.21.15.47:0/2236548628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b6810 0x55ad075b7dc0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.451+0000 7fec5df79740 1 -- 172.21.15.47:0/2236548628 >> 172.21.15.47:0/2236548628 conn(0x55ad075b1c00 msgr2=0x55ad075b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.452+0000 7fec5df79740 1 -- 172.21.15.47:0/2236548628 shutdown_connections 2024-03-21T11:37:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.452+0000 7fec5df79740 1 -- 172.21.15.47:0/2236548628 wait complete. 2024-03-21T11:37:26.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.453+0000 7fec5df79740 1 -- start start 2024-03-21T11:37:26.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.453+0000 7fec5df79740 1 Processor -- start 2024-03-21T11:37:26.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.453+0000 7fec5df79740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b6810 0x55ad075c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.453+0000 7fec5df79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 0x55ad075c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.453+0000 7fec5df79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 0x55ad075c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.453+0000 7fec5df79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec541b1ae0 con 0x55ad075b83a0 2024-03-21T11:37:26.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.453+0000 7fec5df79740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec541b1480 con 0x55ad075b79e0 2024-03-21T11:37:26.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.453+0000 7fec5df79740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec541ae8c0 con 0x55ad075b6810 2024-03-21T11:37:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.453+0000 7fec5df79740 10 client.0 _ll_register_callbacks cb 0x7fec4000b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.454+0000 7fec53e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b6810 0x55ad075c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.454+0000 7fec53e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b6810 0x55ad075c35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56930/0 (socket says 172.21.15.47:56930) 2024-03-21T11:37:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.454+0000 7fec53e00640 1 -- 172.21.15.47:0/758364456 learned_addr learned my addr 172.21.15.47:0/758364456 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.454+0000 7fec59200640 1 --2- 172.21.15.47:0/758364456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 0x55ad075c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.454+0000 7fec53e00640 1 -- 172.21.15.47:0/758364456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 msgr2=0x55ad075c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.454+0000 7fec53e00640 1 --2- 172.21.15.47:0/758364456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 0x55ad075c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.454+0000 7fec53e00640 1 -- 172.21.15.47:0/758364456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 msgr2=0x55ad075c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.455+0000 7fec53e00640 1 --2- 172.21.15.47:0/758364456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 0x55ad075c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.455+0000 7fec53e00640 1 -- 172.21.15.47:0/758364456 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad075c70e0 con 0x55ad075b6810 2024-03-21T11:37:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.455+0000 7fec59200640 1 --2- 172.21.15.47:0/758364456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 0x55ad075c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:37:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.455+0000 7fec53e00640 1 --2- 172.21.15.47:0/758364456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b6810 0x55ad075c35e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fec440a3560 tx=0x7fec440046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b128ccd6969fbc6a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.455+0000 7fec3e000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:37:26.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.455+0000 7fec3e000640 1 -- 172.21.15.47:0/758364456 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec440a7070 con 0x55ad075b6810 2024-03-21T11:37:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.455+0000 7fec3e000640 1 -- 172.21.15.47:0/758364456 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fec440cc560 con 0x55ad075b6810 2024-03-21T11:37:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.455+0000 7fec5df79740 1 -- 172.21.15.47:0/758364456 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec440b3070 con 0x55ad075b6810 2024-03-21T11:37:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.456+0000 7fec3e000640 1 -- 172.21.15.47:0/758364456 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec4407e050 con 0x55ad075b6810 2024-03-21T11:37:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.456+0000 7fec5df79740 10 client.30256 fetch_fsmap learned FSMap version 33 2024-03-21T11:37:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.456+0000 7fec5df79740 1 -- 172.21.15.47:0/758364456 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ad07627980 con 0x55ad075b6810 2024-03-21T11:37:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.456+0000 7fec3e000640 1 -- 172.21.15.47:0/758364456 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec440b3070 con 0x55ad075b6810 2024-03-21T11:37:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.456+0000 7fec3e000640 1 -- 172.21.15.47:0/758364456 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fec440a3ce0 con 0x55ad075b6810 2024-03-21T11:37:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.457+0000 7fec5df79740 10 client.30256 fetch_fsmap finished waiting for FSMap version 33 2024-03-21T11:37:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.457+0000 7fec5df79740 10 client.30256 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:37:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.457+0000 7fec5df79740 20 client.30256 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.457+0000 7fec5df79740 1 --2- 172.21.15.47:0/758364456 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ad076280a0 0x55ad0762a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.457+0000 7fec5df79740 4 client.30256 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:37:26.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.457+0000 7fec5df79740 1 -- 172.21.15.47:0/758364456 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ad07522b50 con 0x55ad076280a0 2024-03-21T11:37:26.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.458+0000 7fec53400640 1 --2- 172.21.15.47:0/758364456 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ad076280a0 0x55ad0762a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.460+0000 7fec53400640 1 --2- 172.21.15.47:0/758364456 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ad076280a0 0x55ad0762a490 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3aa6829c28f9de8 server_cookie=3cd342122fe1528a in_seq=0 out_seq=0 2024-03-21T11:37:26.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.460+0000 7fec3e000640 10 client.30256 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:37:26.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.460+0000 7fec3e000640 1 -- 172.21.15.47:0/758364456 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ad07522b50 con 0x55ad076280a0 2024-03-21T11:37:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.460+0000 7fec3e000640 10 client.30256 handle_command_reply: tid=0 2024-03-21T11:37:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.460+0000 7fec5df79740 1 client.30256 shutdown 2024-03-21T11:37:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.460+0000 7fec5df79740 1 -- 172.21.15.47:0/758364456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b6810 msgr2=0x55ad075c35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.460+0000 7fec5df79740 1 --2- 172.21.15.47:0/758364456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b6810 0x55ad075c35e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fec440a3560 tx=0x7fec440046c0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.460+0000 7fec5df79740 1 -- 172.21.15.47:0/758364456 shutdown_connections 2024-03-21T11:37:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.460+0000 7fec5df79740 1 --2- 172.21.15.47:0/758364456 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ad076280a0 0x55ad0762a490 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.461+0000 7fec5df79740 1 --2- 172.21.15.47:0/758364456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad075b83a0 0x55ad075c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.461+0000 7fec5df79740 1 --2- 172.21.15.47:0/758364456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad075b79e0 0x55ad075c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.461+0000 7fec5df79740 1 --2- 172.21.15.47:0/758364456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ad075b6810 0x55ad075c35e0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.461+0000 7fec5df79740 1 -- 172.21.15.47:0/758364456 >> 172.21.15.47:0/758364456 conn(0x55ad075b1c00 msgr2=0x55ad075b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.461+0000 7fec3e000640 0 client.30256 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:37:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.461+0000 7fec5df79740 1 -- 172.21.15.47:0/758364456 shutdown_connections 2024-03-21T11:37:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.461+0000 7fec5df79740 1 -- 172.21.15.47:0/758364456 wait complete. 2024-03-21T11:37:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.461+0000 7fec5df79740 20 client.30256 trim_cache size 0 max 16384 2024-03-21T11:37:26.465 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:37:26.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.461+0000 7fec5b200640 1 -- 172.21.15.47:0/1576944645 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fec380a3120 msgr2=0x7fec380a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.462+0000 7fec5b200640 1 --2- 172.21.15.47:0/1576944645 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fec380a3120 0x7fec380a55e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fec4c032420 tx=0x7fec4c004a40 comp rx=0 tx=0).stop 2024-03-21T11:37:26.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.462+0000 7fec5b200640 1 -- 172.21.15.47:0/1576944645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec541038e0 msgr2=0x7fec5419e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.462+0000 7fec5b200640 1 --2- 172.21.15.47:0/1576944645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec541038e0 0x7fec5419e060 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fec4400b3c0 tx=0x7fec4400b880 comp rx=0 tx=0).stop 2024-03-21T11:37:26.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.462+0000 7fec5b200640 1 -- 172.21.15.47:0/1576944645 shutdown_connections 2024-03-21T11:37:26.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.462+0000 7fec5b200640 1 --2- 172.21.15.47:0/1576944645 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fec380a3120 0x7fec380a55e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.462+0000 7fec5b200640 1 --2- 172.21.15.47:0/1576944645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec54105490 0x7fec541a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.462+0000 7fec5b200640 1 --2- 172.21.15.47:0/1576944645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec54104ad0 0x7fec5419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.462+0000 7fec5b200640 1 --2- 172.21.15.47:0/1576944645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec541038e0 0x7fec5419e060 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.462+0000 7fec5b200640 1 -- 172.21.15.47:0/1576944645 >> 172.21.15.47:0/1576944645 conn(0x7fec540fee50 msgr2=0x7fec54077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.462+0000 7fec5b200640 1 -- 172.21.15.47:0/1576944645 shutdown_connections 2024-03-21T11:37:26.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.462+0000 7fec5b200640 1 -- 172.21.15.47:0/1576944645 wait complete. 2024-03-21T11:37:26.487 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:37:26.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:37:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.720+0000 7fe2db000640 1 -- 172.21.15.47:0/174801367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2d4104ad0 msgr2=0x7fe2d4104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.720+0000 7fe2db000640 1 --2- 172.21.15.47:0/174801367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2d4104ad0 0x7fe2d4104f50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009860 tx=0x7fe2c002bb70 comp rx=0 tx=0).stop 2024-03-21T11:37:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.721+0000 7fe2db000640 1 -- 172.21.15.47:0/174801367 shutdown_connections 2024-03-21T11:37:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.721+0000 7fe2db000640 1 --2- 172.21.15.47:0/174801367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d4105490 0x7fe2d410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.721+0000 7fe2db000640 1 --2- 172.21.15.47:0/174801367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2d4104ad0 0x7fe2d4104f50 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.721+0000 7fe2db000640 1 --2- 172.21.15.47:0/174801367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d41038e0 0x7fe2d4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.721+0000 7fe2db000640 1 -- 172.21.15.47:0/174801367 >> 172.21.15.47:0/174801367 conn(0x7fe2d40fee50 msgr2=0x7fe2d4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.721+0000 7fe2db000640 1 -- 172.21.15.47:0/174801367 shutdown_connections 2024-03-21T11:37:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.721+0000 7fe2db000640 1 -- 172.21.15.47:0/174801367 wait complete. 2024-03-21T11:37:26.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.722+0000 7fe2db000640 1 Processor -- start 2024-03-21T11:37:26.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.722+0000 7fe2db000640 1 -- start start 2024-03-21T11:37:26.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.722+0000 7fe2db000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d41038e0 0x7fe2d4199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.722+0000 7fe2db000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d4104ad0 0x7fe2d419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.722+0000 7fe2db000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2d4105490 0x7fe2d41a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.722+0000 7fe2db000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d410f540 con 0x7fe2d41038e0 2024-03-21T11:37:26.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.722+0000 7fe2db000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d410f3c0 con 0x7fe2d4104ad0 2024-03-21T11:37:26.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.722+0000 7fe2db000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d410f6c0 con 0x7fe2d4105490 2024-03-21T11:37:26.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.723+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d41038e0 0x7fe2d4199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.723+0000 7fe2d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d4104ad0 0x7fe2d419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.723+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d41038e0 0x7fe2d4199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59138/0 (socket says 172.21.15.47:59138) 2024-03-21T11:37:26.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.723+0000 7fe2d3e00640 1 -- 172.21.15.47:0/3170892598 learned_addr learned my addr 172.21.15.47:0/3170892598 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.723+0000 7fe2d3e00640 1 -- 172.21.15.47:0/3170892598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2d4105490 msgr2=0x7fe2d41a1140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:37:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.723+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/3170892598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2d4105490 0x7fe2d41a1140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.723+0000 7fe2d3e00640 1 -- 172.21.15.47:0/3170892598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d4104ad0 msgr2=0x7fe2d419a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.723+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/3170892598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d4104ad0 0x7fe2d419a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.723+0000 7fe2d3e00640 1 -- 172.21.15.47:0/3170892598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d41a17b0 con 0x7fe2d41038e0 2024-03-21T11:37:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.724+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/3170892598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d41038e0 0x7fe2d4199b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400e9a0 tx=0x7fe2c400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.724+0000 7fe2d0c00640 1 -- 172.21.15.47:0/3170892598 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c400cd80 con 0x7fe2d41038e0 2024-03-21T11:37:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.724+0000 7fe2db000640 1 -- 172.21.15.47:0/3170892598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d41a1aa0 con 0x7fe2d41038e0 2024-03-21T11:37:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.724+0000 7fe2db000640 1 -- 172.21.15.47:0/3170892598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d41a2010 con 0x7fe2d41038e0 2024-03-21T11:37:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.725+0000 7fe2d0c00640 1 -- 172.21.15.47:0/3170892598 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2c4009460 con 0x7fe2d41038e0 2024-03-21T11:37:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.725+0000 7fe2d0c00640 1 -- 172.21.15.47:0/3170892598 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40187e0 con 0x7fe2d41038e0 2024-03-21T11:37:26.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.728+0000 7fe2d0c00640 1 -- 172.21.15.47:0/3170892598 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe2c4018980 con 0x7fe2d41038e0 2024-03-21T11:37:26.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.729+0000 7fe2ddc13740 1 Processor -- start 2024-03-21T11:37:26.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.729+0000 7fe2ddc13740 1 -- start start 2024-03-21T11:37:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.729+0000 7fe2ddc13740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ab810 0x5571244abc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.729+0000 7fe2ddc13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244ace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.729+0000 7fe2ddc13740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 0x5571244b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.729+0000 7fe2ddc13740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571244102c0 con 0x5571244ab810 2024-03-21T11:37:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.729+0000 7fe2ddc13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571243b8c90 con 0x5571244ac9e0 2024-03-21T11:37:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.729+0000 7fe2ddc13740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712440c360 con 0x5571244ad3a0 2024-03-21T11:37:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.730+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.730+0000 7fe2d9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ab810 0x5571244abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.730+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244ace60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41830/0 (socket says 172.21.15.47:41830) 2024-03-21T11:37:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.730+0000 7fe2d3e00640 1 -- 172.21.15.47:0/827759337 learned_addr learned my addr 172.21.15.47:0/827759337 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.730+0000 7fe2d3400640 1 --2- 172.21.15.47:0/827759337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 0x5571244b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.730+0000 7fe2d0c00640 1 --2- 172.21.15.47:0/3170892598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2b80a2fa0 0x7fe2b80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.730+0000 7fe2d3e00640 1 -- 172.21.15.47:0/827759337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 msgr2=0x5571244b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.730+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/827759337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 0x5571244b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.730+0000 7fe2d3e00640 1 -- 172.21.15.47:0/827759337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ab810 msgr2=0x5571244abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.730+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/827759337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ab810 0x5571244abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.730+0000 7fe2d0c00640 1 -- 172.21.15.47:0/3170892598 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe2c40cfc10 con 0x7fe2d41038e0 2024-03-21T11:37:26.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.730+0000 7fe2d3e00640 1 -- 172.21.15.47:0/827759337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571243dd080 con 0x5571244ac9e0 2024-03-21T11:37:26.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.731+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/827759337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244ace60 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40cd300 tx=0x7fe2c400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4093b3113f54bec0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:26.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.731+0000 7fe2d9000640 1 --2- 172.21.15.47:0/3170892598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2b80a2fa0 0x7fe2b80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.731+0000 7fe2c9600640 1 -- 172.21.15.47:0/827759337 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c409b890 con 0x5571244ac9e0 2024-03-21T11:37:26.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.731+0000 7fe2c9600640 1 -- 172.21.15.47:0/827759337 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2c40d4040 con 0x5571244ac9e0 2024-03-21T11:37:26.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.731+0000 7fe2c9600640 1 -- 172.21.15.47:0/827759337 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4049da0 con 0x5571244ac9e0 2024-03-21T11:37:26.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.732+0000 7fe2ddc13740 1 -- 172.21.15.47:0/827759337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 msgr2=0x5571244ace60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.732+0000 7fe2d9000640 1 --2- 172.21.15.47:0/3170892598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2b80a2fa0 0x7fe2b80a5460 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x5571244ac8e0 tx=0x7fe2cc004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.732+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/827759337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244ace60 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40cd300 tx=0x7fe2c400ef90 comp rx=0 tx=0).stop 2024-03-21T11:37:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.732+0000 7fe2ddc13740 1 -- 172.21.15.47:0/827759337 shutdown_connections 2024-03-21T11:37:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.732+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/827759337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 0x5571244b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.732+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/827759337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244ace60 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.732+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/827759337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ab810 0x5571244abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.732+0000 7fe2ddc13740 1 -- 172.21.15.47:0/827759337 >> 172.21.15.47:0/827759337 conn(0x5571244a6c00 msgr2=0x5571244a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.732+0000 7fe2ddc13740 1 -- 172.21.15.47:0/827759337 shutdown_connections 2024-03-21T11:37:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.732+0000 7fe2ddc13740 1 -- 172.21.15.47:0/827759337 wait complete. 2024-03-21T11:37:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.732+0000 7fe2ddc13740 1 -- start start 2024-03-21T11:37:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.733+0000 7fe2ddc13740 1 Processor -- start 2024-03-21T11:37:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.733+0000 7fe2ddc13740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ab810 0x5571244b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.733+0000 7fe2ddc13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.733+0000 7fe2ddc13740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 0x5571244bcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.733+0000 7fe2ddc13740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571244b42f0 con 0x5571244ab810 2024-03-21T11:37:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.733+0000 7fe2ddc13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571243cc0d0 con 0x5571244ac9e0 2024-03-21T11:37:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.733+0000 7fe2ddc13740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571244b4470 con 0x5571244ad3a0 2024-03-21T11:37:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.733+0000 7fe2d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.733+0000 7fe2ddc13740 10 client.0 _ll_register_callbacks cb 0x5571243d0e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.733+0000 7fe2d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41832/0 (socket says 172.21.15.47:41832) 2024-03-21T11:37:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.733+0000 7fe2d3400640 1 -- 172.21.15.47:0/2672747058 learned_addr learned my addr 172.21.15.47:0/2672747058 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.733+0000 7fe2d9000640 1 --2- 172.21.15.47:0/2672747058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 0x5571244bcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.733+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/2672747058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ab810 0x5571244b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.734+0000 7fe2d3400640 1 -- 172.21.15.47:0/2672747058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 msgr2=0x5571244bcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.734+0000 7fe2d3400640 1 --2- 172.21.15.47:0/2672747058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 0x5571244bcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.734+0000 7fe2d3400640 1 -- 172.21.15.47:0/2672747058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ab810 msgr2=0x5571244b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.734+0000 7fe2d3400640 1 --2- 172.21.15.47:0/2672747058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ab810 0x5571244b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.734+0000 7fe2d3400640 1 -- 172.21.15.47:0/2672747058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571241243f0 con 0x5571244ac9e0 2024-03-21T11:37:26.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.734+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/2672747058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ab810 0x5571244b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:26.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.734+0000 7fe2d3400640 1 --2- 172.21.15.47:0/2672747058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244b5ed0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe2c002c180 tx=0x7fe2c0009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc295cd78e7af195 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.734+0000 7fe2be000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:37:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.735+0000 7fe2be000640 1 -- 172.21.15.47:0/2672747058 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0043070 con 0x5571244ac9e0 2024-03-21T11:37:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.735+0000 7fe2be000640 1 -- 172.21.15.47:0/2672747058 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe2c0004040 con 0x5571244ac9e0 2024-03-21T11:37:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.735+0000 7fe2ddc13740 1 -- 172.21.15.47:0/2672747058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2c40d4040 con 0x5571244ac9e0 2024-03-21T11:37:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.735+0000 7fe2be000640 1 -- 172.21.15.47:0/2672747058 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2c004f050 con 0x5571244ac9e0 2024-03-21T11:37:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.735+0000 7fe2ddc13740 10 client.30251 fetch_fsmap learned FSMap version 33 2024-03-21T11:37:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.735+0000 7fe2ddc13740 1 -- 172.21.15.47:0/2672747058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5571241ee7d0 con 0x5571244ac9e0 2024-03-21T11:37:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.736+0000 7fe2be000640 1 -- 172.21.15.47:0/2672747058 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2c40d4040 con 0x5571244ac9e0 2024-03-21T11:37:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.736+0000 7fe2be000640 1 -- 172.21.15.47:0/2672747058 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fe2c0032640 con 0x5571244ac9e0 2024-03-21T11:37:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.736+0000 7fe2ddc13740 10 client.30251 fetch_fsmap finished waiting for FSMap version 33 2024-03-21T11:37:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.736+0000 7fe2ddc13740 10 client.30251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.736+0000 7fe2ddc13740 20 client.30251 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.736+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/2672747058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5571244c1460 0x5571244c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.736+0000 7fe2ddc13740 4 client.30251 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:37:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.736+0000 7fe2ddc13740 1 -- 172.21.15.47:0/2672747058 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55712441d800 con 0x5571244c1460 2024-03-21T11:37:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.736+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/2672747058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5571244c1460 0x5571244c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.737+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/2672747058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5571244c1460 0x5571244c38c0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e77d94e82d95347 server_cookie=12cb76730c6d6b2c in_seq=0 out_seq=0 2024-03-21T11:37:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.737+0000 7fe2be000640 10 client.30251 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.739+0000 7fe2be000640 1 -- 172.21.15.47:0/2672747058 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55712441d800 con 0x5571244c1460 2024-03-21T11:37:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.739+0000 7fe2be000640 10 client.30251 handle_command_reply: tid=0 2024-03-21T11:37:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.739+0000 7fe2ddc13740 1 client.30251 shutdown 2024-03-21T11:37:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.739+0000 7fe2ddc13740 1 -- 172.21.15.47:0/2672747058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 msgr2=0x5571244b5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.739+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/2672747058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244b5ed0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe2c002c180 tx=0x7fe2c0009490 comp rx=0 tx=0).stop 2024-03-21T11:37:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.739+0000 7fe2ddc13740 1 -- 172.21.15.47:0/2672747058 shutdown_connections 2024-03-21T11:37:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.739+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/2672747058 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5571244c1460 0x5571244c38c0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.739+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/2672747058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 0x5571244bcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.739+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/2672747058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244b5ed0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.739+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/2672747058 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ab810 0x5571244b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.739+0000 7fe2be000640 0 client.30251 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.739+0000 7fe2ddc13740 1 -- 172.21.15.47:0/2672747058 >> 172.21.15.47:0/2672747058 conn(0x5571244a6c00 msgr2=0x5571244a5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.739+0000 7fe2ddc13740 1 -- 172.21.15.47:0/2672747058 shutdown_connections 2024-03-21T11:37:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.740+0000 7fe2ddc13740 1 -- 172.21.15.47:0/2672747058 wait complete. 2024-03-21T11:37:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.740+0000 7fe2ddc13740 20 client.30251 trim_cache size 0 max 16384 2024-03-21T11:37:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.755+0000 7fe2ddc13740 1 Processor -- start 2024-03-21T11:37:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.756+0000 7fe2ddc13740 1 -- start start 2024-03-21T11:37:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.756+0000 7fe2ddc13740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ab810 0x5571244a2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.756+0000 7fe2ddc13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244bced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.756+0000 7fe2ddc13740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ad3a0 0x5571244a6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.756+0000 7fe2ddc13740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b400f9a0 con 0x5571244ad3a0 2024-03-21T11:37:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.756+0000 7fe2ddc13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40114a0 con 0x5571244ac9e0 2024-03-21T11:37:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.756+0000 7fe2ddc13740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b4010000 con 0x5571244ab810 2024-03-21T11:37:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.756+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ab810 0x5571244a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.756+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ab810 0x5571244a2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56982/0 (socket says 172.21.15.47:56982) 2024-03-21T11:37:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.756+0000 7fe2d3e00640 1 -- 172.21.15.47:0/799476450 learned_addr learned my addr 172.21.15.47:0/799476450 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.756+0000 7fe2d9000640 1 --2- 172.21.15.47:0/799476450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ad3a0 0x5571244a6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.757+0000 7fe2d3e00640 1 -- 172.21.15.47:0/799476450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 msgr2=0x5571244bced0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:37:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.757+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/799476450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244bced0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.757+0000 7fe2d3e00640 1 -- 172.21.15.47:0/799476450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ad3a0 msgr2=0x5571244a6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.757+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/799476450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ad3a0 0x5571244a6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.757+0000 7fe2d3e00640 1 -- 172.21.15.47:0/799476450 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571244bfdb0 con 0x5571244ab810 2024-03-21T11:37:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.757+0000 7fe2d9000640 1 --2- 172.21.15.47:0/799476450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ad3a0 0x5571244a6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:37:26.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.757+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/799476450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ab810 0x5571244a2110 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400c6a0 tx=0x7fe2c40d3650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3c1bf23dad4161b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:26.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.757+0000 7fe2c8c00640 1 -- 172.21.15.47:0/799476450 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c409a230 con 0x5571244ab810 2024-03-21T11:37:26.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.757+0000 7fe2c8c00640 1 -- 172.21.15.47:0/799476450 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2c40cd410 con 0x5571244ab810 2024-03-21T11:37:26.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.757+0000 7fe2c8c00640 1 -- 172.21.15.47:0/799476450 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40a44b0 con 0x5571244ab810 2024-03-21T11:37:26.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.758+0000 7fe2ddc13740 1 -- 172.21.15.47:0/799476450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ab810 msgr2=0x5571244a2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.758+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/799476450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ab810 0x5571244a2110 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400c6a0 tx=0x7fe2c40d3650 comp rx=0 tx=0).stop 2024-03-21T11:37:26.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.759+0000 7fe2ddc13740 1 -- 172.21.15.47:0/799476450 shutdown_connections 2024-03-21T11:37:26.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.759+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/799476450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ad3a0 0x5571244a6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.759+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/799476450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ac9e0 0x5571244bced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.759+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/799476450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ab810 0x5571244a2110 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.759+0000 7fe2ddc13740 1 -- 172.21.15.47:0/799476450 >> 172.21.15.47:0/799476450 conn(0x5571244a6c00 msgr2=0x5571244aa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.759+0000 7fe2ddc13740 1 -- 172.21.15.47:0/799476450 shutdown_connections 2024-03-21T11:37:26.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.759+0000 7fe2ddc13740 1 -- 172.21.15.47:0/799476450 wait complete. 2024-03-21T11:37:26.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.759+0000 7fe2ddc13740 1 -- start start 2024-03-21T11:37:26.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.759+0000 7fe2ddc13740 1 Processor -- start 2024-03-21T11:37:26.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.760+0000 7fe2ddc13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ab810 0x5571244bc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.760+0000 7fe2ddc13740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ac9e0 0x5571244bcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.760+0000 7fe2ddc13740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 0x5571244bad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.760+0000 7fe2ddc13740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b4010d80 con 0x5571244ac9e0 2024-03-21T11:37:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.760+0000 7fe2ddc13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b4010720 con 0x5571244ab810 2024-03-21T11:37:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.760+0000 7fe2ddc13740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b400db60 con 0x5571244ad3a0 2024-03-21T11:37:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.760+0000 7fe2ddc13740 10 client.0 _ll_register_callbacks cb 0x7fe2cc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.760+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ab810 0x5571244bc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.760+0000 7fe2d9000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 0x5571244bad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.761+0000 7fe2d9000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 0x5571244bad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56990/0 (socket says 172.21.15.47:56990) 2024-03-21T11:37:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.761+0000 7fe2d9000640 1 -- 172.21.15.47:0/324677620 learned_addr learned my addr 172.21.15.47:0/324677620 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.761+0000 7fe2d3400640 1 --2- 172.21.15.47:0/324677620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ac9e0 0x5571244bcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.761+0000 7fe2d3400640 1 -- 172.21.15.47:0/324677620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 msgr2=0x5571244bad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.761+0000 7fe2d3400640 1 --2- 172.21.15.47:0/324677620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 0x5571244bad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.761+0000 7fe2d3400640 1 -- 172.21.15.47:0/324677620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ab810 msgr2=0x5571244bc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:26.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.761+0000 7fe2d3400640 1 --2- 172.21.15.47:0/324677620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ab810 0x5571244bc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.761+0000 7fe2d3400640 1 -- 172.21.15.47:0/324677620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571244bb300 con 0x5571244ac9e0 2024-03-21T11:37:26.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.762+0000 7fe2d3400640 1 --2- 172.21.15.47:0/324677620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ac9e0 0x5571244bcbf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009780 tx=0x7fe2c0032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c94be556087812 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:26.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.762+0000 7fe2be000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:37:26.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.762+0000 7fe2be000640 1 -- 172.21.15.47:0/324677620 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0058060 con 0x5571244ac9e0 2024-03-21T11:37:26.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.762+0000 7fe2be000640 1 -- 172.21.15.47:0/324677620 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe2c0004040 con 0x5571244ac9e0 2024-03-21T11:37:26.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.762+0000 7fe2ddc13740 1 -- 172.21.15.47:0/324677620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2c40cd410 con 0x5571244ac9e0 2024-03-21T11:37:26.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.762+0000 7fe2be000640 1 -- 172.21.15.47:0/324677620 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2c005f050 con 0x5571244ac9e0 2024-03-21T11:37:26.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.762+0000 7fe2ddc13740 10 client.19824 fetch_fsmap learned FSMap version 33 2024-03-21T11:37:26.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.763+0000 7fe2ddc13740 1 -- 172.21.15.47:0/324677620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55712451d6e0 con 0x5571244ac9e0 2024-03-21T11:37:26.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.763+0000 7fe2be000640 1 -- 172.21.15.47:0/324677620 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2c40cd410 con 0x5571244ac9e0 2024-03-21T11:37:26.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.763+0000 7fe2be000640 1 -- 172.21.15.47:0/324677620 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 33) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fe2c0004260 con 0x5571244ac9e0 2024-03-21T11:37:26.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.763+0000 7fe2ddc13740 10 client.19824 fetch_fsmap finished waiting for FSMap version 33 2024-03-21T11:37:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.763+0000 7fe2ddc13740 10 client.19824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.763+0000 7fe2ddc13740 20 client.19824 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.763+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/324677620 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55712451e130 0x557124520550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.763+0000 7fe2ddc13740 4 client.19824 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:37:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.763+0000 7fe2ddc13740 1 -- 172.21.15.47:0/324677620 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557124417b50 con 0x55712451e130 2024-03-21T11:37:26.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.765+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/324677620 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55712451e130 0x557124520550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:26.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.766+0000 7fe2d3e00640 1 --2- 172.21.15.47:0/324677620 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55712451e130 0x557124520550 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3b38f1c5425b942 server_cookie=2cd113017129f434 in_seq=0 out_seq=0 2024-03-21T11:37:26.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.766+0000 7fe2be000640 10 client.19824 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:26.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.769+0000 7fe2be000640 1 -- 172.21.15.47:0/324677620 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557124417b50 con 0x55712451e130 2024-03-21T11:37:26.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.769+0000 7fe2be000640 10 client.19824 handle_command_reply: tid=0 2024-03-21T11:37:26.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.769+0000 7fe2ddc13740 1 client.19824 shutdown 2024-03-21T11:37:26.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.770+0000 7fe2ddc13740 1 -- 172.21.15.47:0/324677620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ac9e0 msgr2=0x5571244bcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.770+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/324677620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ac9e0 0x5571244bcbf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009780 tx=0x7fe2c0032f80 comp rx=0 tx=0).stop 2024-03-21T11:37:26.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.770+0000 7fe2ddc13740 1 -- 172.21.15.47:0/324677620 shutdown_connections 2024-03-21T11:37:26.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.770+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/324677620 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55712451e130 0x557124520550 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.770+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/324677620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571244ad3a0 0x5571244bad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.770+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/324677620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571244ac9e0 0x5571244bcbf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.770+0000 7fe2be000640 0 client.19824 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:26.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.770+0000 7fe2ddc13740 1 --2- 172.21.15.47:0/324677620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571244ab810 0x5571244bc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.770+0000 7fe2ddc13740 1 -- 172.21.15.47:0/324677620 >> 172.21.15.47:0/324677620 conn(0x5571244a6c00 msgr2=0x5571244aa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.770+0000 7fe2ddc13740 1 -- 172.21.15.47:0/324677620 shutdown_connections 2024-03-21T11:37:26.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.770+0000 7fe2ddc13740 1 -- 172.21.15.47:0/324677620 wait complete. 2024-03-21T11:37:26.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.770+0000 7fe2ddc13740 20 client.19824 trim_cache size 0 max 16384 2024-03-21T11:37:26.772 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:37:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.771+0000 7fe2db000640 1 -- 172.21.15.47:0/3170892598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2b80a2fa0 msgr2=0x7fe2b80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.771+0000 7fe2db000640 1 --2- 172.21.15.47:0/3170892598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2b80a2fa0 0x7fe2b80a5460 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x5571244ac8e0 tx=0x7fe2cc004ab0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.771+0000 7fe2db000640 1 -- 172.21.15.47:0/3170892598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d41038e0 msgr2=0x7fe2d4199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.771+0000 7fe2db000640 1 --2- 172.21.15.47:0/3170892598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d41038e0 0x7fe2d4199b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400e9a0 tx=0x7fe2c400ee60 comp rx=0 tx=0).stop 2024-03-21T11:37:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.771+0000 7fe2db000640 1 -- 172.21.15.47:0/3170892598 shutdown_connections 2024-03-21T11:37:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.771+0000 7fe2db000640 1 --2- 172.21.15.47:0/3170892598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2b80a2fa0 0x7fe2b80a5460 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.771+0000 7fe2db000640 1 --2- 172.21.15.47:0/3170892598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2d4105490 0x7fe2d41a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.771+0000 7fe2db000640 1 --2- 172.21.15.47:0/3170892598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d4104ad0 0x7fe2d419a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.771+0000 7fe2db000640 1 --2- 172.21.15.47:0/3170892598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d41038e0 0x7fe2d4199b90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.771+0000 7fe2db000640 1 -- 172.21.15.47:0/3170892598 >> 172.21.15.47:0/3170892598 conn(0x7fe2d40fee50 msgr2=0x7fe2d4107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:26.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.771+0000 7fe2db000640 1 -- 172.21.15.47:0/3170892598 shutdown_connections 2024-03-21T11:37:26.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:26.771+0000 7fe2db000640 1 -- 172.21.15.47:0/3170892598 wait complete. 2024-03-21T11:37:26.917 INFO:tasks.workunit.client.0.smithi047.stdout: 10 5839 48.81 MB/sec warmup 10 sec latency 89.229 ms 2024-03-21T11:37:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:26 smithi047 ceph-mon[31351]: pgmap v823: 129 pgs: 129 active+clean; 41 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 390 op/s 2024-03-21T11:37:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:26 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:26 smithi100 ceph-mon[36245]: pgmap v823: 129 pgs: 129 active+clean; 41 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 390 op/s 2024-03-21T11:37:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:26 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:27.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:26 smithi102 ceph-mon[36748]: pgmap v823: 129 pgs: 129 active+clean; 41 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 390 op/s 2024-03-21T11:37:27.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:26 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:27.794 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:37:27.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:37:27.795 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:37:27.917 INFO:tasks.workunit.client.0.smithi047.stdout: 10 6529 48.02 MB/sec warmup 11 sec latency 45.365 ms 2024-03-21T11:37:28.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.064+0000 7f2da9200640 1 -- 172.21.15.47:0/3484440676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da4104a70 msgr2=0x7f2da4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.064+0000 7f2da9200640 1 --2- 172.21.15.47:0/3484440676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da4104a70 0x7f2da4106e60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009900 tx=0x7f2d9002bc30 comp rx=0 tx=0).stop 2024-03-21T11:37:28.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.064+0000 7f2da9200640 1 -- 172.21.15.47:0/3484440676 shutdown_connections 2024-03-21T11:37:28.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.064+0000 7f2da9200640 1 --2- 172.21.15.47:0/3484440676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da41073a0 0x7f2da41097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.064+0000 7f2da9200640 1 --2- 172.21.15.47:0/3484440676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da4104a70 0x7f2da4106e60 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.064+0000 7f2da9200640 1 --2- 172.21.15.47:0/3484440676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da4102140 0x7f2da4104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.064+0000 7f2da9200640 1 -- 172.21.15.47:0/3484440676 >> 172.21.15.47:0/3484440676 conn(0x7f2da40fbb50 msgr2=0x7f2da40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.064+0000 7f2da9200640 1 -- 172.21.15.47:0/3484440676 shutdown_connections 2024-03-21T11:37:28.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.064+0000 7f2da9200640 1 -- 172.21.15.47:0/3484440676 wait complete. 2024-03-21T11:37:28.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.065+0000 7f2da9200640 1 Processor -- start 2024-03-21T11:37:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.066+0000 7f2da9200640 1 -- start start 2024-03-21T11:37:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.066+0000 7f2da9200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4102140 0x7f2da4199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.066+0000 7f2da9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da4104a70 0x7f2da419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.066+0000 7f2da9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da41073a0 0x7f2da41a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.066+0000 7f2da9200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da410d380 con 0x7f2da4102140 2024-03-21T11:37:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.066+0000 7f2da9200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da410d200 con 0x7f2da41073a0 2024-03-21T11:37:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.066+0000 7f2da9200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da410d500 con 0x7f2da4104a70 2024-03-21T11:37:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.066+0000 7f2da1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da4104a70 0x7f2da419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.066+0000 7f2da2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da41073a0 0x7f2da41a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.066+0000 7f2da1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da4104a70 0x7f2da419a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57016/0 (socket says 172.21.15.47:57016) 2024-03-21T11:37:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.066+0000 7f2da1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4102140 0x7f2da4199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.066+0000 7f2da2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da41073a0 0x7f2da41a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41864/0 (socket says 172.21.15.47:41864) 2024-03-21T11:37:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.066+0000 7f2da1400640 1 -- 172.21.15.47:0/1365881418 learned_addr learned my addr 172.21.15.47:0/1365881418 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.067+0000 7f2da2800640 1 -- 172.21.15.47:0/1365881418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da4104a70 msgr2=0x7f2da419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.067+0000 7f2da2800640 1 --2- 172.21.15.47:0/1365881418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da4104a70 0x7f2da419a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.067+0000 7f2da2800640 1 -- 172.21.15.47:0/1365881418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4102140 msgr2=0x7f2da4199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.067+0000 7f2da2800640 1 --2- 172.21.15.47:0/1365881418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4102140 0x7f2da4199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.067+0000 7f2da2800640 1 -- 172.21.15.47:0/1365881418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da41a1840 con 0x7f2da41073a0 2024-03-21T11:37:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.067+0000 7f2da2800640 1 --2- 172.21.15.47:0/1365881418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da41073a0 0x7f2da41a11a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009fd0 tx=0x7f2d9c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.067+0000 7f2d9aa00640 1 -- 172.21.15.47:0/1365881418 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c00ce20 con 0x7f2da41073a0 2024-03-21T11:37:28.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.067+0000 7f2da9200640 1 -- 172.21.15.47:0/1365881418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da41a1b30 con 0x7f2da41073a0 2024-03-21T11:37:28.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.067+0000 7f2d9aa00640 1 -- 172.21.15.47:0/1365881418 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d9c005020 con 0x7f2da41073a0 2024-03-21T11:37:28.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.068+0000 7f2d9aa00640 1 -- 172.21.15.47:0/1365881418 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c010720 con 0x7f2da41073a0 2024-03-21T11:37:28.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.068+0000 7f2da9200640 1 -- 172.21.15.47:0/1365881418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da41a2030 con 0x7f2da41073a0 2024-03-21T11:37:28.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.071+0000 7f2d9aa00640 1 -- 172.21.15.47:0/1365881418 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2d9c005540 con 0x7f2da41073a0 2024-03-21T11:37:28.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.072+0000 7f2dac006740 1 Processor -- start 2024-03-21T11:37:28.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.072+0000 7f2dac006740 1 -- start start 2024-03-21T11:37:28.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.072+0000 7f2dac006740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b30a0 0x564dc64b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.072+0000 7f2dac006740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 0x564dc64b8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.072+0000 7f2dac006740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b8c00 0x564dc64bb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.072+0000 7f2dac006740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc6418eb0 con 0x564dc64b3a90 2024-03-21T11:37:28.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.072+0000 7f2dac006740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc63c1a60 con 0x564dc64b8c00 2024-03-21T11:37:28.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.072+0000 7f2dac006740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc6414f50 con 0x564dc64b30a0 2024-03-21T11:37:28.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.073+0000 7f2da1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b30a0 0x564dc64b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.073+0000 7f2da1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b30a0 0x564dc64b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57020/0 (socket says 172.21.15.47:57020) 2024-03-21T11:37:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.073+0000 7f2da1400640 1 -- 172.21.15.47:0/3546955583 learned_addr learned my addr 172.21.15.47:0/3546955583 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.073+0000 7f2da1400640 1 --2- 172.21.15.47:0/3546955583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b8c00 0x564dc64bb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.073+0000 7f2da1400640 1 -- 172.21.15.47:0/3546955583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b8c00 msgr2=0x564dc64bb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.073+0000 7f2da1400640 1 --2- 172.21.15.47:0/3546955583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b8c00 0x564dc64bb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.073+0000 7f2da1400640 1 -- 172.21.15.47:0/3546955583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 msgr2=0x564dc64b8590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:37:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.074+0000 7f2da1e00640 1 --2- 172.21.15.47:0/3546955583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 0x564dc64b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.074+0000 7f2da1400640 1 --2- 172.21.15.47:0/3546955583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 0x564dc64b8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.074+0000 7f2d9aa00640 1 --2- 172.21.15.47:0/1365881418 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d880a2f20 0x7f2d880a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.074+0000 7f2da1400640 1 -- 172.21.15.47:0/3546955583 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc61f57e0 con 0x564dc64b30a0 2024-03-21T11:37:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.074+0000 7f2da1e00640 1 --2- 172.21.15.47:0/3546955583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 0x564dc64b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:37:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.074+0000 7f2d9aa00640 1 -- 172.21.15.47:0/1365881418 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2d9c0cff30 con 0x7f2da41073a0 2024-03-21T11:37:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.074+0000 7f2da1400640 1 --2- 172.21.15.47:0/3546955583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b30a0 0x564dc64b34c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2d90005c40 tx=0x7f2d90005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fb2b5fb3cc8bfc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.075+0000 7f2da1400640 1 --2- 172.21.15.47:0/1365881418 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d880a2f20 0x7f2d880a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.075+0000 7f2d8f400640 1 -- 172.21.15.47:0/3546955583 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90049070 con 0x564dc64b30a0 2024-03-21T11:37:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.075+0000 7f2d8f400640 1 -- 172.21.15.47:0/3546955583 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d900046c0 con 0x564dc64b30a0 2024-03-21T11:37:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.075+0000 7f2d8f400640 1 -- 172.21.15.47:0/3546955583 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d900052f0 con 0x564dc64b30a0 2024-03-21T11:37:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.076+0000 7f2dac006740 1 -- 172.21.15.47:0/3546955583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b30a0 msgr2=0x564dc64b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.076+0000 7f2dac006740 1 --2- 172.21.15.47:0/3546955583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b30a0 0x564dc64b34c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2d90005c40 tx=0x7f2d90005b50 comp rx=0 tx=0).stop 2024-03-21T11:37:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.076+0000 7f2da1400640 1 --2- 172.21.15.47:0/1365881418 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d880a2f20 0x7f2d880a53e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2d9003b000 tx=0x7f2d90005570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.076+0000 7f2dac006740 1 -- 172.21.15.47:0/3546955583 shutdown_connections 2024-03-21T11:37:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.076+0000 7f2dac006740 1 --2- 172.21.15.47:0/3546955583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b8c00 0x564dc64bb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.076+0000 7f2dac006740 1 --2- 172.21.15.47:0/3546955583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 0x564dc64b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.076+0000 7f2dac006740 1 --2- 172.21.15.47:0/3546955583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b30a0 0x564dc64b34c0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.076+0000 7f2dac006740 1 -- 172.21.15.47:0/3546955583 >> 172.21.15.47:0/3546955583 conn(0x564dc64ae610 msgr2=0x564dc64aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.076+0000 7f2dac006740 1 -- 172.21.15.47:0/3546955583 shutdown_connections 2024-03-21T11:37:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.076+0000 7f2dac006740 1 -- 172.21.15.47:0/3546955583 wait complete. 2024-03-21T11:37:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.077+0000 7f2dac006740 1 -- start start 2024-03-21T11:37:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.077+0000 7f2dac006740 1 Processor -- start 2024-03-21T11:37:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2dac006740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b30a0 0x564dc64bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2dac006740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 0x564dc64bd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2dac006740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b8c00 0x564dc64c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2dac006740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc64bbb50 con 0x564dc64b3a90 2024-03-21T11:37:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2dac006740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc63d4eb0 con 0x564dc64b30a0 2024-03-21T11:37:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2dac006740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc64bbcd0 con 0x564dc64b8c00 2024-03-21T11:37:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2dac006740 10 client.0 _ll_register_callbacks cb 0x564dc63c9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2da1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b30a0 0x564dc64bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2da1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b30a0 0x564dc64bd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41894/0 (socket says 172.21.15.47:41894) 2024-03-21T11:37:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2da1e00640 1 -- 172.21.15.47:0/1686576205 learned_addr learned my addr 172.21.15.47:0/1686576205 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2da2800640 1 --2- 172.21.15.47:0/1686576205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b8c00 0x564dc64c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2da1400640 1 --2- 172.21.15.47:0/1686576205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 0x564dc64bd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2da1e00640 1 -- 172.21.15.47:0/1686576205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b8c00 msgr2=0x564dc64c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2da1e00640 1 --2- 172.21.15.47:0/1686576205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b8c00 0x564dc64c47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2da1e00640 1 -- 172.21.15.47:0/1686576205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 msgr2=0x564dc64bd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2da1e00640 1 --2- 172.21.15.47:0/1686576205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 0x564dc64bd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2da1e00640 1 -- 172.21.15.47:0/1686576205 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dc6255f70 con 0x564dc64b30a0 2024-03-21T11:37:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.078+0000 7f2da1400640 1 --2- 172.21.15.47:0/1686576205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 0x564dc64bd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:37:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.079+0000 7f2da1e00640 1 --2- 172.21.15.47:0/1686576205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b30a0 0x564dc64bd290 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2d94013710 tx=0x7f2d94013bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c0cfc880125e9e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.079+0000 7f2d83e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:37:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.079+0000 7f2d83e00640 1 -- 172.21.15.47:0/1686576205 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94004460 con 0x564dc64b30a0 2024-03-21T11:37:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.079+0000 7f2d83e00640 1 -- 172.21.15.47:0/1686576205 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2d940119c0 con 0x564dc64b30a0 2024-03-21T11:37:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.079+0000 7f2dac006740 1 -- 172.21.15.47:0/1686576205 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d900046c0 con 0x564dc64b30a0 2024-03-21T11:37:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.080+0000 7f2d83e00640 1 -- 172.21.15.47:0/1686576205 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d94011bc0 con 0x564dc64b30a0 2024-03-21T11:37:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.080+0000 7f2dac006740 10 client.30266 fetch_fsmap learned FSMap version 33 2024-03-21T11:37:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.080+0000 7f2dac006740 1 -- 172.21.15.47:0/1686576205 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564dc60ed170 con 0x564dc64b30a0 2024-03-21T11:37:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.080+0000 7f2d83e00640 1 -- 172.21.15.47:0/1686576205 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d900046c0 con 0x564dc64b30a0 2024-03-21T11:37:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.080+0000 7f2d83e00640 1 -- 172.21.15.47:0/1686576205 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f2d9401a070 con 0x564dc64b30a0 2024-03-21T11:37:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.080+0000 7f2dac006740 10 client.30266 fetch_fsmap finished waiting for FSMap version 33 2024-03-21T11:37:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.080+0000 7f2dac006740 10 client.30266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.080+0000 7f2dac006740 20 client.30266 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.080+0000 7f2dac006740 1 --2- 172.21.15.47:0/1686576205 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564dc64c9180 0x564dc64cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.080+0000 7f2dac006740 4 client.30266 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:37:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.080+0000 7f2dac006740 1 -- 172.21.15.47:0/1686576205 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564dc64263f0 con 0x564dc64c9180 2024-03-21T11:37:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.081+0000 7f2da1400640 1 --2- 172.21.15.47:0/1686576205 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564dc64c9180 0x564dc64cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.081+0000 7f2da1400640 1 --2- 172.21.15.47:0/1686576205 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564dc64c9180 0x564dc64cb5e0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bf00d00aa0c0612 server_cookie=9ea568ca007216aa in_seq=0 out_seq=0 2024-03-21T11:37:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.081+0000 7f2d83e00640 10 client.30266 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:28.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.086+0000 7f2d83e00640 1 -- 172.21.15.47:0/1686576205 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564dc64263f0 con 0x564dc64c9180 2024-03-21T11:37:28.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.086+0000 7f2d83e00640 10 client.30266 handle_command_reply: tid=0 2024-03-21T11:37:28.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.087+0000 7f2dac006740 1 client.30266 shutdown 2024-03-21T11:37:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.088+0000 7f2dac006740 1 -- 172.21.15.47:0/1686576205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b30a0 msgr2=0x564dc64bd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.088+0000 7f2dac006740 1 --2- 172.21.15.47:0/1686576205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b30a0 0x564dc64bd290 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2d94013710 tx=0x7f2d94013bd0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.088+0000 7f2dac006740 1 -- 172.21.15.47:0/1686576205 shutdown_connections 2024-03-21T11:37:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.088+0000 7f2dac006740 1 --2- 172.21.15.47:0/1686576205 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564dc64c9180 0x564dc64cb5e0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.088+0000 7f2d83e00640 0 client.30266 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.088+0000 7f2dac006740 1 --2- 172.21.15.47:0/1686576205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b8c00 0x564dc64c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.088+0000 7f2dac006740 1 --2- 172.21.15.47:0/1686576205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 0x564dc64bd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.088+0000 7f2dac006740 1 --2- 172.21.15.47:0/1686576205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b30a0 0x564dc64bd290 secure :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2d94013710 tx=0x7f2d94013bd0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.088+0000 7f2dac006740 1 -- 172.21.15.47:0/1686576205 >> 172.21.15.47:0/1686576205 conn(0x564dc64ae610 msgr2=0x564dc64ad830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.089+0000 7f2dac006740 1 -- 172.21.15.47:0/1686576205 shutdown_connections 2024-03-21T11:37:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.089+0000 7f2dac006740 1 -- 172.21.15.47:0/1686576205 wait complete. 2024-03-21T11:37:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.089+0000 7f2dac006740 20 client.30266 trim_cache size 0 max 16384 2024-03-21T11:37:28.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.112+0000 7f2dac006740 1 Processor -- start 2024-03-21T11:37:28.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.112+0000 7f2dac006740 1 -- start start 2024-03-21T11:37:28.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.112+0000 7f2dac006740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b3a90 0x564dc64b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.112+0000 7f2dac006740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b8c00 0x564dc64c4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.112+0000 7f2dac006740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b5330 0x564dc64bc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.113+0000 7f2dac006740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da41ac1c0 con 0x564dc64b5330 2024-03-21T11:37:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.113+0000 7f2dac006740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da41adcc0 con 0x564dc64b8c00 2024-03-21T11:37:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.113+0000 7f2dac006740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da41ac820 con 0x564dc64b3a90 2024-03-21T11:37:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.113+0000 7f2da1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b3a90 0x564dc64b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.113+0000 7f2da1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b8c00 0x564dc64c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.113+0000 7f2da1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b3a90 0x564dc64b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57032/0 (socket says 172.21.15.47:57032) 2024-03-21T11:37:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.113+0000 7f2da1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b8c00 0x564dc64c4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41904/0 (socket says 172.21.15.47:41904) 2024-03-21T11:37:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.113+0000 7f2da1e00640 1 -- 172.21.15.47:0/3116904118 learned_addr learned my addr 172.21.15.47:0/3116904118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.113+0000 7f2da2800640 1 --2- 172.21.15.47:0/3116904118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b5330 0x564dc64bc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.113+0000 7f2da1e00640 1 -- 172.21.15.47:0/3116904118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b8c00 msgr2=0x564dc64c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.113+0000 7f2da1e00640 1 --2- 172.21.15.47:0/3116904118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b8c00 0x564dc64c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.113+0000 7f2da1e00640 1 -- 172.21.15.47:0/3116904118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b5330 msgr2=0x564dc64bc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.113+0000 7f2da1e00640 1 --2- 172.21.15.47:0/3116904118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b5330 0x564dc64bc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.113+0000 7f2da1e00640 1 -- 172.21.15.47:0/3116904118 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc64a9c20 con 0x564dc64b3a90 2024-03-21T11:37:28.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.114+0000 7f2da1e00640 1 --2- 172.21.15.47:0/3116904118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b3a90 0x564dc64b1f80 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2d940092e0 tx=0x7f2d94009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c03c6f20fe591691 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.114+0000 7f2d8ea00640 1 -- 172.21.15.47:0/3116904118 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9402e020 con 0x564dc64b3a90 2024-03-21T11:37:28.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.114+0000 7f2d8ea00640 1 -- 172.21.15.47:0/3116904118 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d94011d60 con 0x564dc64b3a90 2024-03-21T11:37:28.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.114+0000 7f2d8ea00640 1 -- 172.21.15.47:0/3116904118 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d940035f0 con 0x564dc64b3a90 2024-03-21T11:37:28.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.115+0000 7f2dac006740 1 -- 172.21.15.47:0/3116904118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b3a90 msgr2=0x564dc64b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.115+0000 7f2dac006740 1 --2- 172.21.15.47:0/3116904118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b3a90 0x564dc64b1f80 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2d940092e0 tx=0x7f2d94009fd0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.115+0000 7f2dac006740 1 -- 172.21.15.47:0/3116904118 shutdown_connections 2024-03-21T11:37:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.115+0000 7f2dac006740 1 --2- 172.21.15.47:0/3116904118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b5330 0x564dc64bc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.115+0000 7f2dac006740 1 --2- 172.21.15.47:0/3116904118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b8c00 0x564dc64c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.115+0000 7f2dac006740 1 --2- 172.21.15.47:0/3116904118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b3a90 0x564dc64b1f80 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.115+0000 7f2dac006740 1 -- 172.21.15.47:0/3116904118 >> 172.21.15.47:0/3116904118 conn(0x564dc64ae610 msgr2=0x564dc64b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.115+0000 7f2dac006740 1 -- 172.21.15.47:0/3116904118 shutdown_connections 2024-03-21T11:37:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.115+0000 7f2dac006740 1 -- 172.21.15.47:0/3116904118 wait complete. 2024-03-21T11:37:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.116+0000 7f2dac006740 1 -- start start 2024-03-21T11:37:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.116+0000 7f2dac006740 1 Processor -- start 2024-03-21T11:37:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.116+0000 7f2dac006740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 0x564dc64c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.116+0000 7f2dac006740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b5330 0x564dc64c3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.116+0000 7f2dac006740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b8c00 0x564dc64c3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.116+0000 7f2dac006740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da41ad5a0 con 0x564dc64b3a90 2024-03-21T11:37:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.116+0000 7f2dac006740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da41acf40 con 0x564dc64b5330 2024-03-21T11:37:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.116+0000 7f2dac006740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da41aa380 con 0x564dc64b8c00 2024-03-21T11:37:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.116+0000 7f2dac006740 10 client.0 _ll_register_callbacks cb 0x7f2d940097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.117+0000 7f2da2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b8c00 0x564dc64c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.117+0000 7f2da2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b8c00 0x564dc64c3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57048/0 (socket says 172.21.15.47:57048) 2024-03-21T11:37:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.117+0000 7f2da2800640 1 -- 172.21.15.47:0/1505811477 learned_addr learned my addr 172.21.15.47:0/1505811477 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.117+0000 7f2da1400640 1 --2- 172.21.15.47:0/1505811477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b5330 0x564dc64c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.117+0000 7f2da1e00640 1 --2- 172.21.15.47:0/1505811477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 0x564dc64c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.117+0000 7f2da1400640 1 -- 172.21.15.47:0/1505811477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b8c00 msgr2=0x564dc64c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.117+0000 7f2da1400640 1 --2- 172.21.15.47:0/1505811477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b8c00 0x564dc64c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.117+0000 7f2da1400640 1 -- 172.21.15.47:0/1505811477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 msgr2=0x564dc64c2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.117+0000 7f2da1400640 1 --2- 172.21.15.47:0/1505811477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 0x564dc64c2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.117+0000 7f2da1400640 1 -- 172.21.15.47:0/1505811477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dc6524ab0 con 0x564dc64b5330 2024-03-21T11:37:28.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.117+0000 7f2da2800640 1 --2- 172.21.15.47:0/1505811477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b8c00 0x564dc64c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:28.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.120+0000 7f2da1400640 1 --2- 172.21.15.47:0/1505811477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b5330 0x564dc64c3320 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f2d90005c40 tx=0x7f2d90044000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afd3ee18461c1e21 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.120+0000 7f2d83e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:37:28.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.120+0000 7f2d83e00640 1 -- 172.21.15.47:0/1505811477 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90049070 con 0x564dc64b5330 2024-03-21T11:37:28.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.120+0000 7f2d83e00640 1 -- 172.21.15.47:0/1505811477 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2d90004ef0 con 0x564dc64b5330 2024-03-21T11:37:28.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.121+0000 7f2dac006740 1 -- 172.21.15.47:0/1505811477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d94011d60 con 0x564dc64b5330 2024-03-21T11:37:28.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.122+0000 7f2d83e00640 1 -- 172.21.15.47:0/1505811477 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d900050f0 con 0x564dc64b5330 2024-03-21T11:37:28.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.122+0000 7f2dac006740 10 client.30272 fetch_fsmap learned FSMap version 33 2024-03-21T11:37:28.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.122+0000 7f2dac006740 1 -- 172.21.15.47:0/1505811477 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564dc6525cf0 con 0x564dc64b5330 2024-03-21T11:37:28.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.123+0000 7f2d83e00640 1 -- 172.21.15.47:0/1505811477 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d94011d60 con 0x564dc64b5330 2024-03-21T11:37:28.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.123+0000 7f2d83e00640 1 -- 172.21.15.47:0/1505811477 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f2d90003ee0 con 0x564dc64b5330 2024-03-21T11:37:28.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.123+0000 7f2dac006740 10 client.30272 fetch_fsmap finished waiting for FSMap version 33 2024-03-21T11:37:28.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.123+0000 7f2dac006740 10 client.30272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:28.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.123+0000 7f2dac006740 20 client.30272 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:28.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.123+0000 7f2dac006740 1 --2- 172.21.15.47:0/1505811477 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564dc6526320 0x564dc6528710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.123+0000 7f2dac006740 4 client.30272 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:37:28.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.123+0000 7f2dac006740 1 -- 172.21.15.47:0/1505811477 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564dc64b8970 con 0x564dc6526320 2024-03-21T11:37:28.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.124+0000 7f2da1e00640 1 --2- 172.21.15.47:0/1505811477 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564dc6526320 0x564dc6528710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.125+0000 7f2d83e00640 10 client.30272 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:28.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.127+0000 7f2da1e00640 1 --2- 172.21.15.47:0/1505811477 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564dc6526320 0x564dc6528710 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63eaae807923e161 server_cookie=8f661d7f349ee258 in_seq=0 out_seq=0 2024-03-21T11:37:28.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.128+0000 7f2d83e00640 1 -- 172.21.15.47:0/1505811477 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564dc64b8970 con 0x564dc6526320 2024-03-21T11:37:28.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.128+0000 7f2d83e00640 10 client.30272 handle_command_reply: tid=0 2024-03-21T11:37:28.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.128+0000 7f2dac006740 1 client.30272 shutdown 2024-03-21T11:37:28.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.128+0000 7f2dac006740 1 -- 172.21.15.47:0/1505811477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b5330 msgr2=0x564dc64c3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.128+0000 7f2dac006740 1 --2- 172.21.15.47:0/1505811477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b5330 0x564dc64c3320 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f2d90005c40 tx=0x7f2d90044000 comp rx=0 tx=0).stop 2024-03-21T11:37:28.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.128+0000 7f2dac006740 1 -- 172.21.15.47:0/1505811477 shutdown_connections 2024-03-21T11:37:28.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.128+0000 7f2dac006740 1 --2- 172.21.15.47:0/1505811477 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564dc6526320 0x564dc6528710 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.128+0000 7f2dac006740 1 --2- 172.21.15.47:0/1505811477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564dc64b8c00 0x564dc64c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.128+0000 7f2dac006740 1 --2- 172.21.15.47:0/1505811477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dc64b5330 0x564dc64c3320 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.128+0000 7f2dac006740 1 --2- 172.21.15.47:0/1505811477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564dc64b3a90 0x564dc64c2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.128+0000 7f2dac006740 1 -- 172.21.15.47:0/1505811477 >> 172.21.15.47:0/1505811477 conn(0x564dc64ae610 msgr2=0x564dc64b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.129+0000 7f2d83e00640 0 client.30272 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:28.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.129+0000 7f2dac006740 1 -- 172.21.15.47:0/1505811477 shutdown_connections 2024-03-21T11:37:28.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.129+0000 7f2dac006740 1 -- 172.21.15.47:0/1505811477 wait complete. 2024-03-21T11:37:28.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.129+0000 7f2dac006740 20 client.30272 trim_cache size 0 max 16384 2024-03-21T11:37:28.131 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:37:28.131 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:37:28.131 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "bc172c00-5689-41c0-af0b-3efcb5e26fdb", 2024-03-21T11:37:28.131 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:37:28.131 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:37:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.130+0000 7f2da9200640 1 -- 172.21.15.47:0/1365881418 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d880a2f20 msgr2=0x7f2d880a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.130+0000 7f2da9200640 1 --2- 172.21.15.47:0/1365881418 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d880a2f20 0x7f2d880a53e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2d9003b000 tx=0x7f2d90005570 comp rx=0 tx=0).stop 2024-03-21T11:37:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.130+0000 7f2da9200640 1 -- 172.21.15.47:0/1365881418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da41073a0 msgr2=0x7f2da41a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.130+0000 7f2da9200640 1 --2- 172.21.15.47:0/1365881418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da41073a0 0x7f2da41a11a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009fd0 tx=0x7f2d9c00ef10 comp rx=0 tx=0).stop 2024-03-21T11:37:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.131+0000 7f2da9200640 1 -- 172.21.15.47:0/1365881418 shutdown_connections 2024-03-21T11:37:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.131+0000 7f2da9200640 1 --2- 172.21.15.47:0/1365881418 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2d880a2f20 0x7f2d880a53e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.131+0000 7f2da9200640 1 --2- 172.21.15.47:0/1365881418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da41073a0 0x7f2da41a11a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.131+0000 7f2da9200640 1 --2- 172.21.15.47:0/1365881418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2da4104a70 0x7f2da419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.131+0000 7f2da9200640 1 --2- 172.21.15.47:0/1365881418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2da4102140 0x7f2da4199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.131+0000 7f2da9200640 1 -- 172.21.15.47:0/1365881418 >> 172.21.15.47:0/1365881418 conn(0x7f2da40fbb50 msgr2=0x7f2da4101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.131+0000 7f2da9200640 1 -- 172.21.15.47:0/1365881418 shutdown_connections 2024-03-21T11:37:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.131+0000 7f2da9200640 1 -- 172.21.15.47:0/1365881418 wait complete. 2024-03-21T11:37:28.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:37:28.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:37:28.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.397+0000 7f993e600640 1 -- 172.21.15.47:0/1225867431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9938102130 msgr2=0x7f9938104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.397+0000 7f993e600640 1 --2- 172.21.15.47:0/1225867431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9938102130 0x7f9938104520 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9928009900 tx=0x7f992802bc40 comp rx=0 tx=0).stop 2024-03-21T11:37:28.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.397+0000 7f993e600640 1 -- 172.21.15.47:0/1225867431 shutdown_connections 2024-03-21T11:37:28.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.397+0000 7f993e600640 1 --2- 172.21.15.47:0/1225867431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9938107390 0x7f99381097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.397+0000 7f993e600640 1 --2- 172.21.15.47:0/1225867431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9938104a60 0x7f9938106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.397+0000 7f993e600640 1 --2- 172.21.15.47:0/1225867431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9938102130 0x7f9938104520 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.397+0000 7f993e600640 1 -- 172.21.15.47:0/1225867431 >> 172.21.15.47:0/1225867431 conn(0x7f99380fbb40 msgr2=0x7f99380fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.397+0000 7f993e600640 1 -- 172.21.15.47:0/1225867431 shutdown_connections 2024-03-21T11:37:28.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.397+0000 7f993e600640 1 -- 172.21.15.47:0/1225867431 wait complete. 2024-03-21T11:37:28.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.398+0000 7f993e600640 1 Processor -- start 2024-03-21T11:37:28.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.398+0000 7f993e600640 1 -- start start 2024-03-21T11:37:28.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.399+0000 7f993e600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9938102130 0x7f9938101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.399+0000 7f993e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9938104a60 0x7f99381000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.399+0000 7f993e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9938107390 0x7f9938100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.399+0000 7f993e600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993810d380 con 0x7f9938102130 2024-03-21T11:37:28.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.399+0000 7f993e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993810d200 con 0x7f9938107390 2024-03-21T11:37:28.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.399+0000 7f993e600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993810d500 con 0x7f9938104a60 2024-03-21T11:37:28.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.399+0000 7f9937400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9938102130 0x7f9938101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.399+0000 7f9936a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9938104a60 0x7f99381000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.399+0000 7f9937e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9938107390 0x7f9938100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.399+0000 7f9936a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9938104a60 0x7f99381000e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57072/0 (socket says 172.21.15.47:57072) 2024-03-21T11:37:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.399+0000 7f9936a00640 1 -- 172.21.15.47:0/2939271107 learned_addr learned my addr 172.21.15.47:0/2939271107 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.400+0000 7f9937400640 1 -- 172.21.15.47:0/2939271107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9938104a60 msgr2=0x7f99381000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.400+0000 7f9937400640 1 --2- 172.21.15.47:0/2939271107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9938104a60 0x7f99381000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.400+0000 7f9937400640 1 -- 172.21.15.47:0/2939271107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9938107390 msgr2=0x7f9938100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.400+0000 7f9937400640 1 --2- 172.21.15.47:0/2939271107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9938107390 0x7f9938100620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.400+0000 7f9937400640 1 -- 172.21.15.47:0/2939271107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9938100eb0 con 0x7f9938102130 2024-03-21T11:37:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.400+0000 7f9936a00640 1 --2- 172.21.15.47:0/2939271107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9938104a60 0x7f99381000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.400+0000 7f9937e00640 1 --2- 172.21.15.47:0/2939271107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9938107390 0x7f9938100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.400+0000 7f9937400640 1 --2- 172.21.15.47:0/2939271107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9938102130 0x7f9938101aa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9928036040 tx=0x7f992802c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.400+0000 7f992fe00640 1 -- 172.21.15.47:0/2939271107 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928041070 con 0x7f9938102130 2024-03-21T11:37:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.400+0000 7f993e600640 1 -- 172.21.15.47:0/2939271107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99381a1b00 con 0x7f9938102130 2024-03-21T11:37:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.401+0000 7f992fe00640 1 -- 172.21.15.47:0/2939271107 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9928004450 con 0x7f9938102130 2024-03-21T11:37:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.401+0000 7f993e600640 1 -- 172.21.15.47:0/2939271107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99381a2070 con 0x7f9938102130 2024-03-21T11:37:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.401+0000 7f992fe00640 1 -- 172.21.15.47:0/2939271107 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928003df0 con 0x7f9938102130 2024-03-21T11:37:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.404+0000 7f992fe00640 1 -- 172.21.15.47:0/2939271107 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9928044070 con 0x7f9938102130 2024-03-21T11:37:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.406+0000 7f992fe00640 1 --2- 172.21.15.47:0/2939271107 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f991c0a3020 0x7f991c0a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.407+0000 7f9941372740 1 Processor -- start 2024-03-21T11:37:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.407+0000 7f9941372740 1 -- start start 2024-03-21T11:37:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.407+0000 7f9941372740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf5e80 0x55c25ddf62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.407+0000 7f9941372740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddf6870 0x55c25ddfb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.407+0000 7f9941372740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddfb9e0 0x55c25ddfde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.407+0000 7f9941372740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c25dd5b250 con 0x55c25ddf6870 2024-03-21T11:37:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.407+0000 7f9941372740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c25dd03a30 con 0x55c25ddfb9e0 2024-03-21T11:37:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.407+0000 7f9941372740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c25dd572f0 con 0x55c25ddf5e80 2024-03-21T11:37:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.408+0000 7f9937e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddfb9e0 0x55c25ddfde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.408+0000 7f9937e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddfb9e0 0x55c25ddfde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41932/0 (socket says 172.21.15.47:41932) 2024-03-21T11:37:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.408+0000 7f9937e00640 1 -- 172.21.15.47:0/1627487207 learned_addr learned my addr 172.21.15.47:0/1627487207 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.409+0000 7f9936a00640 1 --2- 172.21.15.47:0/2939271107 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f991c0a3020 0x7f991c0a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.409+0000 7f9937e00640 1 -- 172.21.15.47:0/1627487207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf5e80 msgr2=0x55c25ddf62a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:37:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.409+0000 7f9937e00640 1 --2- 172.21.15.47:0/1627487207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf5e80 0x55c25ddf62a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.409+0000 7f992fe00640 1 -- 172.21.15.47:0/2939271107 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f99280f1180 con 0x7f9938102130 2024-03-21T11:37:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.410+0000 7f9937e00640 1 -- 172.21.15.47:0/1627487207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddf6870 msgr2=0x55c25ddfb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.410+0000 7f9937e00640 1 --2- 172.21.15.47:0/1627487207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddf6870 0x55c25ddfb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.410+0000 7f9937e00640 1 -- 172.21.15.47:0/1627487207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c25dcee690 con 0x55c25ddfb9e0 2024-03-21T11:37:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.410+0000 7f9937e00640 1 --2- 172.21.15.47:0/1627487207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddfb9e0 0x55c25ddfde70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9930011810 tx=0x7f9930011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adb478368fb1629f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.410+0000 7f992cc00640 1 -- 172.21.15.47:0/1627487207 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930011310 con 0x55c25ddfb9e0 2024-03-21T11:37:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.411+0000 7f992cc00640 1 -- 172.21.15.47:0/1627487207 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f993000f9a0 con 0x55c25ddfb9e0 2024-03-21T11:37:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.411+0000 7f992cc00640 1 -- 172.21.15.47:0/1627487207 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930005720 con 0x55c25ddfb9e0 2024-03-21T11:37:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.412+0000 7f9941372740 1 -- 172.21.15.47:0/1627487207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddfb9e0 msgr2=0x55c25ddfde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.412+0000 7f9941372740 1 --2- 172.21.15.47:0/1627487207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddfb9e0 0x55c25ddfde70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9930011810 tx=0x7f9930011cd0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.412+0000 7f9936a00640 1 --2- 172.21.15.47:0/2939271107 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f991c0a3020 0x7f991c0a54e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9924009fd0 tx=0x7f9924009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.412+0000 7f9941372740 1 -- 172.21.15.47:0/1627487207 shutdown_connections 2024-03-21T11:37:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.412+0000 7f9941372740 1 --2- 172.21.15.47:0/1627487207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddfb9e0 0x55c25ddfde70 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.412+0000 7f9941372740 1 --2- 172.21.15.47:0/1627487207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddf6870 0x55c25ddfb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.412+0000 7f9941372740 1 --2- 172.21.15.47:0/1627487207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf5e80 0x55c25ddf62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.412+0000 7f9941372740 1 -- 172.21.15.47:0/1627487207 >> 172.21.15.47:0/1627487207 conn(0x55c25ddbe200 msgr2=0x55c25ddf17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.412+0000 7f9941372740 1 -- 172.21.15.47:0/1627487207 shutdown_connections 2024-03-21T11:37:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.412+0000 7f9941372740 1 -- 172.21.15.47:0/1627487207 wait complete. 2024-03-21T11:37:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.413+0000 7f9941372740 1 -- start start 2024-03-21T11:37:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.413+0000 7f9941372740 1 Processor -- start 2024-03-21T11:37:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.414+0000 7f9941372740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 0x55c25ddffe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.414+0000 7f9941372740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf6870 0x55c25de003c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.414+0000 7f9941372740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddfb9e0 0x55c25de07400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.414+0000 7f9941372740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c25ddfe990 con 0x55c25ddfb9e0 2024-03-21T11:37:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.414+0000 7f9941372740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c25dd16e80 con 0x55c25ddf5e80 2024-03-21T11:37:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.414+0000 7f9941372740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c25ddfeb10 con 0x55c25ddf6870 2024-03-21T11:37:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.414+0000 7f9941372740 10 client.0 _ll_register_callbacks cb 0x55c25dd29130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.414+0000 7f9937e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddfb9e0 0x55c25de07400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.414+0000 7f9937e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddfb9e0 0x55c25de07400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59244/0 (socket says 172.21.15.47:59244) 2024-03-21T11:37:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.414+0000 7f9937e00640 1 -- 172.21.15.47:0/315065435 learned_addr learned my addr 172.21.15.47:0/315065435 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.414+0000 7f9936a00640 1 --2- 172.21.15.47:0/315065435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf6870 0x55c25de003c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.414+0000 7f9937400640 1 --2- 172.21.15.47:0/315065435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 0x55c25ddffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.415+0000 7f9937e00640 1 -- 172.21.15.47:0/315065435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf6870 msgr2=0x55c25de003c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.415+0000 7f9937e00640 1 --2- 172.21.15.47:0/315065435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf6870 0x55c25de003c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.415+0000 7f9937e00640 1 -- 172.21.15.47:0/315065435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 msgr2=0x55c25ddffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.415+0000 7f9937e00640 1 --2- 172.21.15.47:0/315065435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 0x55c25ddffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.415+0000 7f9937e00640 1 -- 172.21.15.47:0/315065435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c25db397d0 con 0x55c25ddfb9e0 2024-03-21T11:37:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.415+0000 7f9937e00640 1 --2- 172.21.15.47:0/315065435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddfb9e0 0x55c25de07400 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9930011480 tx=0x7f99300114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c78659536eb05d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.416+0000 7f9921600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:37:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.416+0000 7f9921600640 1 -- 172.21.15.47:0/315065435 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930003d80 con 0x55c25ddfb9e0 2024-03-21T11:37:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.416+0000 7f9921600640 1 -- 172.21.15.47:0/315065435 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f99300054a0 con 0x55c25ddfb9e0 2024-03-21T11:37:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.416+0000 7f9941372740 1 -- 172.21.15.47:0/315065435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f993000f9a0 con 0x55c25ddfb9e0 2024-03-21T11:37:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.417+0000 7f9921600640 1 -- 172.21.15.47:0/315065435 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99300056a0 con 0x55c25ddfb9e0 2024-03-21T11:37:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.417+0000 7f9941372740 10 client.19848 fetch_fsmap learned FSMap version 33 2024-03-21T11:37:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.417+0000 7f9941372740 1 -- 172.21.15.47:0/315065435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c25da6f3f0 con 0x55c25ddfb9e0 2024-03-21T11:37:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.417+0000 7f9921600640 1 -- 172.21.15.47:0/315065435 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f993000f9a0 con 0x55c25ddfb9e0 2024-03-21T11:37:28.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.417+0000 7f9921600640 1 -- 172.21.15.47:0/315065435 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 33) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f9930018070 con 0x55c25ddfb9e0 2024-03-21T11:37:28.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.418+0000 7f9941372740 10 client.19848 fetch_fsmap finished waiting for FSMap version 33 2024-03-21T11:37:28.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.418+0000 7f9941372740 10 client.19848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:28.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.418+0000 7f9941372740 20 client.19848 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:28.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.418+0000 7f9941372740 1 --2- 172.21.15.47:0/315065435 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c25de0bb30 0x55c25de0df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.418+0000 7f9941372740 4 client.19848 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:37:28.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.418+0000 7f9941372740 1 -- 172.21.15.47:0/315065435 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c25dd68790 con 0x55c25de0bb30 2024-03-21T11:37:28.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.418+0000 7f9937400640 1 --2- 172.21.15.47:0/315065435 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c25de0bb30 0x55c25de0df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.419+0000 7f9937400640 1 --2- 172.21.15.47:0/315065435 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c25de0bb30 0x55c25de0df50 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d32ec62ffaf5d38 server_cookie=1e068d62883ee812 in_seq=0 out_seq=0 2024-03-21T11:37:28.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.419+0000 7f9921600640 10 client.19848 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:28.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.421+0000 7f9921600640 1 -- 172.21.15.47:0/315065435 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c25dd68790 con 0x55c25de0bb30 2024-03-21T11:37:28.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.421+0000 7f9921600640 10 client.19848 handle_command_reply: tid=0 2024-03-21T11:37:28.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.421+0000 7f9941372740 1 client.19848 shutdown 2024-03-21T11:37:28.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.421+0000 7f9941372740 1 -- 172.21.15.47:0/315065435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddfb9e0 msgr2=0x55c25de07400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.421+0000 7f9941372740 1 --2- 172.21.15.47:0/315065435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddfb9e0 0x55c25de07400 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9930011480 tx=0x7f99300114b0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.422+0000 7f9941372740 1 -- 172.21.15.47:0/315065435 shutdown_connections 2024-03-21T11:37:28.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.422+0000 7f9941372740 1 --2- 172.21.15.47:0/315065435 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c25de0bb30 0x55c25de0df50 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.422+0000 7f9941372740 1 --2- 172.21.15.47:0/315065435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddfb9e0 0x55c25de07400 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.422+0000 7f9941372740 1 --2- 172.21.15.47:0/315065435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf6870 0x55c25de003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.422+0000 7f9921600640 0 client.19848 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:28.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.422+0000 7f9941372740 1 --2- 172.21.15.47:0/315065435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 0x55c25ddffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.422+0000 7f9941372740 1 -- 172.21.15.47:0/315065435 >> 172.21.15.47:0/315065435 conn(0x55c25ddbe200 msgr2=0x55c25ddf0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.423+0000 7f9941372740 1 -- 172.21.15.47:0/315065435 shutdown_connections 2024-03-21T11:37:28.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.423+0000 7f9941372740 1 -- 172.21.15.47:0/315065435 wait complete. 2024-03-21T11:37:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.423+0000 7f9941372740 20 client.19848 trim_cache size 0 max 16384 2024-03-21T11:37:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.454+0000 7f9941372740 1 Processor -- start 2024-03-21T11:37:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.454+0000 7f9941372740 1 -- start start 2024-03-21T11:37:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.454+0000 7f9941372740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 0x55c25ddf4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.454+0000 7f9941372740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf6870 0x55c25de073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.454+0000 7f9941372740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddfb9e0 0x55c25ddf4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.454+0000 7f9941372740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99381ac310 con 0x55c25ddfb9e0 2024-03-21T11:37:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.454+0000 7f9941372740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99381ade10 con 0x55c25ddf5e80 2024-03-21T11:37:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.454+0000 7f9941372740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99381ac970 con 0x55c25ddf6870 2024-03-21T11:37:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.455+0000 7f9936a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf6870 0x55c25de073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.455+0000 7f9936a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf6870 0x55c25de073c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57092/0 (socket says 172.21.15.47:57092) 2024-03-21T11:37:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.455+0000 7f9936a00640 1 -- 172.21.15.47:0/2337963460 learned_addr learned my addr 172.21.15.47:0/2337963460 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.455+0000 7f9937e00640 1 --2- 172.21.15.47:0/2337963460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddfb9e0 0x55c25ddf4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.455+0000 7f9937400640 1 --2- 172.21.15.47:0/2337963460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 0x55c25ddf4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.455+0000 7f9936a00640 1 -- 172.21.15.47:0/2337963460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 msgr2=0x55c25ddf4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.456+0000 7f9936a00640 1 --2- 172.21.15.47:0/2337963460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 0x55c25ddf4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.456+0000 7f9936a00640 1 -- 172.21.15.47:0/2337963460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddfb9e0 msgr2=0x55c25ddf4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.456+0000 7f9936a00640 1 --2- 172.21.15.47:0/2337963460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddfb9e0 0x55c25ddf4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.456+0000 7f9936a00640 1 -- 172.21.15.47:0/2337963460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c25da2f0f0 con 0x55c25ddf6870 2024-03-21T11:37:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.456+0000 7f9937400640 1 --2- 172.21.15.47:0/2337963460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 0x55c25ddf4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.456+0000 7f9937e00640 1 --2- 172.21.15.47:0/2337963460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddfb9e0 0x55c25ddf4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.456+0000 7f9936a00640 1 --2- 172.21.15.47:0/2337963460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf6870 0x55c25de073c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f9938101810 tx=0x7f9924016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68183203f898342d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.456+0000 7f9923e00640 1 -- 172.21.15.47:0/2337963460 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924005650 con 0x55c25ddf6870 2024-03-21T11:37:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.456+0000 7f9923e00640 1 -- 172.21.15.47:0/2337963460 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9924004c80 con 0x55c25ddf6870 2024-03-21T11:37:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.456+0000 7f9923e00640 1 -- 172.21.15.47:0/2337963460 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992400f5c0 con 0x55c25ddf6870 2024-03-21T11:37:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.458+0000 7f9941372740 1 -- 172.21.15.47:0/2337963460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf6870 msgr2=0x55c25de073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.458+0000 7f9941372740 1 --2- 172.21.15.47:0/2337963460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf6870 0x55c25de073c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f9938101810 tx=0x7f9924016000 comp rx=0 tx=0).stop 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.458+0000 7f9941372740 1 -- 172.21.15.47:0/2337963460 shutdown_connections 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.458+0000 7f9941372740 1 --2- 172.21.15.47:0/2337963460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddfb9e0 0x55c25ddf4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.458+0000 7f9941372740 1 --2- 172.21.15.47:0/2337963460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddf6870 0x55c25de073c0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.458+0000 7f9941372740 1 --2- 172.21.15.47:0/2337963460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 0x55c25ddf4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.458+0000 7f9941372740 1 -- 172.21.15.47:0/2337963460 >> 172.21.15.47:0/2337963460 conn(0x55c25ddbe200 msgr2=0x55c25ddf7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.459+0000 7f9941372740 1 -- 172.21.15.47:0/2337963460 shutdown_connections 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.459+0000 7f9941372740 1 -- 172.21.15.47:0/2337963460 wait complete. 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.459+0000 7f9941372740 1 -- start start 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.460+0000 7f9941372740 1 Processor -- start 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.460+0000 7f9941372740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 0x55c25de03150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.460+0000 7f9941372740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddf6870 0x55c25de03690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.460+0000 7f9941372740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddfb9e0 0x55c25de066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.460+0000 7f9941372740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99381ad6f0 con 0x55c25ddf6870 2024-03-21T11:37:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.460+0000 7f9941372740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99381ad090 con 0x55c25ddf5e80 2024-03-21T11:37:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.460+0000 7f9941372740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99381aa4d0 con 0x55c25ddfb9e0 2024-03-21T11:37:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.460+0000 7f9941372740 10 client.0 _ll_register_callbacks cb 0x7f99300116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.460+0000 7f9937400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 0x55c25de03150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.460+0000 7f9937400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 0x55c25de03150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41960/0 (socket says 172.21.15.47:41960) 2024-03-21T11:37:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.460+0000 7f9937400640 1 -- 172.21.15.47:0/3346194364 learned_addr learned my addr 172.21.15.47:0/3346194364 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.461+0000 7f9936a00640 1 --2- 172.21.15.47:0/3346194364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddf6870 0x55c25de03690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.461+0000 7f9937e00640 1 --2- 172.21.15.47:0/3346194364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddfb9e0 0x55c25de066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.462+0000 7f9937e00640 1 -- 172.21.15.47:0/3346194364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 msgr2=0x55c25de03150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.462+0000 7f9937e00640 1 --2- 172.21.15.47:0/3346194364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 0x55c25de03150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.462+0000 7f9937e00640 1 -- 172.21.15.47:0/3346194364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddf6870 msgr2=0x55c25de03690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.462+0000 7f9937e00640 1 --2- 172.21.15.47:0/3346194364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddf6870 0x55c25de03690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.462+0000 7f9937e00640 1 -- 172.21.15.47:0/3346194364 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c25de06ca0 con 0x55c25ddfb9e0 2024-03-21T11:37:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.462+0000 7f9937e00640 1 --2- 172.21.15.47:0/3346194364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddfb9e0 0x55c25de066b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9930009b80 tx=0x7f9930005af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a2b5e4da1cd442c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.462+0000 7f9921600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:37:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.463+0000 7f9921600640 1 -- 172.21.15.47:0/3346194364 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993003c060 con 0x55c25ddfb9e0 2024-03-21T11:37:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.463+0000 7f9921600640 1 -- 172.21.15.47:0/3346194364 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f99300053e0 con 0x55c25ddfb9e0 2024-03-21T11:37:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.463+0000 7f9941372740 1 -- 172.21.15.47:0/3346194364 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9924004c80 con 0x55c25ddfb9e0 2024-03-21T11:37:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.463+0000 7f9921600640 1 -- 172.21.15.47:0/3346194364 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9930043050 con 0x55c25ddfb9e0 2024-03-21T11:37:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.463+0000 7f9941372740 10 client.30304 fetch_fsmap learned FSMap version 33 2024-03-21T11:37:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.463+0000 7f9941372740 1 -- 172.21.15.47:0/3346194364 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c25de633a0 con 0x55c25ddfb9e0 2024-03-21T11:37:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.464+0000 7f9921600640 1 -- 172.21.15.47:0/3346194364 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9924004c80 con 0x55c25ddfb9e0 2024-03-21T11:37:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.465+0000 7f9921600640 1 -- 172.21.15.47:0/3346194364 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f9930018040 con 0x55c25ddfb9e0 2024-03-21T11:37:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.465+0000 7f9941372740 10 client.30304 fetch_fsmap finished waiting for FSMap version 33 2024-03-21T11:37:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.465+0000 7f9941372740 10 client.30304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.465+0000 7f9941372740 20 client.30304 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.465+0000 7f9941372740 1 --2- 172.21.15.47:0/3346194364 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c25de63d20 0x55c25de66140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.465+0000 7f9941372740 4 client.30304 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:37:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.465+0000 7f9941372740 1 -- 172.21.15.47:0/3346194364 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c25dd62ae0 con 0x55c25de63d20 2024-03-21T11:37:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.465+0000 7f9937400640 1 --2- 172.21.15.47:0/3346194364 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c25de63d20 0x55c25de66140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.466+0000 7f9937400640 1 --2- 172.21.15.47:0/3346194364 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c25de63d20 0x55c25de66140 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78aa006adf5ca3f2 server_cookie=d35d33454a30cbe3 in_seq=0 out_seq=0 2024-03-21T11:37:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.466+0000 7f9921600640 10 client.30304 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.471+0000 7f9921600640 1 -- 172.21.15.47:0/3346194364 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c25dd62ae0 con 0x55c25de63d20 2024-03-21T11:37:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.471+0000 7f9921600640 10 client.30304 handle_command_reply: tid=0 2024-03-21T11:37:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.471+0000 7f9941372740 1 client.30304 shutdown 2024-03-21T11:37:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.472+0000 7f9941372740 1 -- 172.21.15.47:0/3346194364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddfb9e0 msgr2=0x55c25de066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.472+0000 7f9941372740 1 --2- 172.21.15.47:0/3346194364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddfb9e0 0x55c25de066b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9930009b80 tx=0x7f9930005af0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.474+0000 7f9941372740 1 -- 172.21.15.47:0/3346194364 shutdown_connections 2024-03-21T11:37:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.474+0000 7f9941372740 1 --2- 172.21.15.47:0/3346194364 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c25de63d20 0x55c25de66140 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.474+0000 7f9941372740 1 --2- 172.21.15.47:0/3346194364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c25ddfb9e0 0x55c25de066b0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.474+0000 7f9941372740 1 --2- 172.21.15.47:0/3346194364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c25ddf6870 0x55c25de03690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.474+0000 7f9941372740 1 --2- 172.21.15.47:0/3346194364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c25ddf5e80 0x55c25de03150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.474+0000 7f9921600640 0 client.30304 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:28.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.474+0000 7f9941372740 1 -- 172.21.15.47:0/3346194364 >> 172.21.15.47:0/3346194364 conn(0x55c25ddbe200 msgr2=0x55c25ddf0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.477+0000 7f9941372740 1 -- 172.21.15.47:0/3346194364 shutdown_connections 2024-03-21T11:37:28.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.477+0000 7f9941372740 1 -- 172.21.15.47:0/3346194364 wait complete. 2024-03-21T11:37:28.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.478+0000 7f9941372740 20 client.30304 trim_cache size 0 max 16384 2024-03-21T11:37:28.820 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:37:28.821 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:37:28.821 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:37:28.821 INFO:teuthology.orchestra.run.smithi047.stdout: "bc172c00-5689-41c0-af0b-3efcb5e26fdb": { 2024-03-21T11:37:28.821 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:37:28.821 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "bc172c00-5689-41c0-af0b-3efcb5e26fdb", 2024-03-21T11:37:28.821 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:37:28.821 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:37:28.821 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:37:28.821 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:37:28.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.480+0000 7f993e600640 1 -- 172.21.15.47:0/2939271107 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f991c0a3020 msgr2=0x7f991c0a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.480+0000 7f993e600640 1 --2- 172.21.15.47:0/2939271107 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f991c0a3020 0x7f991c0a54e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9924009fd0 tx=0x7f9924009530 comp rx=0 tx=0).stop 2024-03-21T11:37:28.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.480+0000 7f993e600640 1 -- 172.21.15.47:0/2939271107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9938102130 msgr2=0x7f9938101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.480+0000 7f993e600640 1 --2- 172.21.15.47:0/2939271107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9938102130 0x7f9938101aa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9928036040 tx=0x7f992802c660 comp rx=0 tx=0).stop 2024-03-21T11:37:28.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.480+0000 7f993e600640 1 -- 172.21.15.47:0/2939271107 shutdown_connections 2024-03-21T11:37:28.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.480+0000 7f993e600640 1 --2- 172.21.15.47:0/2939271107 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f991c0a3020 0x7f991c0a54e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.480+0000 7f993e600640 1 --2- 172.21.15.47:0/2939271107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9938107390 0x7f9938100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.480+0000 7f993e600640 1 --2- 172.21.15.47:0/2939271107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9938104a60 0x7f99381000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.480+0000 7f993e600640 1 --2- 172.21.15.47:0/2939271107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9938102130 0x7f9938101aa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.480+0000 7f993e600640 1 -- 172.21.15.47:0/2939271107 >> 172.21.15.47:0/2939271107 conn(0x7f99380fbb40 msgr2=0x7f9938102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.481+0000 7f993e600640 1 -- 172.21.15.47:0/2939271107 shutdown_connections 2024-03-21T11:37:28.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:28.481+0000 7f993e600640 1 -- 172.21.15.47:0/2939271107 wait complete. 2024-03-21T11:37:28.824 INFO:tasks.cephfs.filesystem:scrub status for tag:bc172c00-5689-41c0-af0b-3efcb5e26fdb - {'path': '/', 'tag': 'bc172c00-5689-41c0-af0b-3efcb5e26fdb', 'options': 'recursive,force'} 2024-03-21T11:37:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.514+0000 7fb80fe00640 1 -- 172.21.15.47:0/3079907623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808105270 msgr2=0x7fb8081056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.514+0000 7fb80fe00640 1 --2- 172.21.15.47:0/3079907623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808105270 0x7fb8081056f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb800009900 tx=0x7fb80002bc40 comp rx=0 tx=0).stop 2024-03-21T11:37:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.514+0000 7fb80fe00640 1 -- 172.21.15.47:0/3079907623 shutdown_connections 2024-03-21T11:37:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.514+0000 7fb80fe00640 1 --2- 172.21.15.47:0/3079907623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808105270 0x7fb8081056f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.514+0000 7fb80fe00640 1 --2- 172.21.15.47:0/3079907623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8081028c0 0x7fb808104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.514+0000 7fb80fe00640 1 --2- 172.21.15.47:0/3079907623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8080fff50 0x7fb808102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.514+0000 7fb80fe00640 1 -- 172.21.15.47:0/3079907623 >> 172.21.15.47:0/3079907623 conn(0x7fb8080fbb30 msgr2=0x7fb8080fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.514+0000 7fb80fe00640 1 -- 172.21.15.47:0/3079907623 shutdown_connections 2024-03-21T11:37:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.516+0000 7fb80fe00640 1 -- 172.21.15.47:0/3079907623 wait complete. 2024-03-21T11:37:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.517+0000 7fb80fe00640 1 Processor -- start 2024-03-21T11:37:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.517+0000 7fb80fe00640 1 -- start start 2024-03-21T11:37:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.518+0000 7fb80fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8080fff50 0x7fb80819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.518+0000 7fb80fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8081028c0 0x7fb80819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.518+0000 7fb80fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808105270 0x7fb8081a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.518+0000 7fb80fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb808113930 con 0x7fb8081028c0 2024-03-21T11:37:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.518+0000 7fb80fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8081137b0 con 0x7fb808105270 2024-03-21T11:37:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.518+0000 7fb80fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb808113ab0 con 0x7fb8080fff50 2024-03-21T11:37:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.518+0000 7fb80d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8080fff50 0x7fb80819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.518+0000 7fb80d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8080fff50 0x7fb80819de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57112/0 (socket says 172.21.15.47:57112) 2024-03-21T11:37:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.518+0000 7fb80d400640 1 -- 172.21.15.47:0/1017422129 learned_addr learned my addr 172.21.15.47:0/1017422129 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.518+0000 7fb80de00640 1 --2- 172.21.15.47:0/1017422129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808105270 0x7fb8081a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.519+0000 7fb80ca00640 1 --2- 172.21.15.47:0/1017422129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8081028c0 0x7fb80819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.519+0000 7fb80de00640 1 -- 172.21.15.47:0/1017422129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8080fff50 msgr2=0x7fb80819de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.519+0000 7fb80de00640 1 --2- 172.21.15.47:0/1017422129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8080fff50 0x7fb80819de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.519+0000 7fb80de00640 1 -- 172.21.15.47:0/1017422129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8081028c0 msgr2=0x7fb80819e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.519+0000 7fb80de00640 1 --2- 172.21.15.47:0/1017422129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8081028c0 0x7fb80819e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.519+0000 7fb80de00640 1 -- 172.21.15.47:0/1017422129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8081066a0 con 0x7fb808105270 2024-03-21T11:37:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.519+0000 7fb80de00640 1 --2- 172.21.15.47:0/1017422129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808105270 0x7fb8081a5430 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb800009850 tx=0x7fb800005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.519+0000 7fb806000640 1 -- 172.21.15.47:0/1017422129 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800041070 con 0x7fb808105270 2024-03-21T11:37:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.520+0000 7fb80fe00640 1 -- 172.21.15.47:0/1017422129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb808106930 con 0x7fb808105270 2024-03-21T11:37:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.520+0000 7fb806000640 1 -- 172.21.15.47:0/1017422129 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8000044d0 con 0x7fb808105270 2024-03-21T11:37:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.520+0000 7fb806000640 1 -- 172.21.15.47:0/1017422129 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800003e20 con 0x7fb808105270 2024-03-21T11:37:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.520+0000 7fb80fe00640 1 -- 172.21.15.47:0/1017422129 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb808106df0 con 0x7fb808105270 2024-03-21T11:37:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.524+0000 7fb812c10740 1 Processor -- start 2024-03-21T11:37:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.524+0000 7fb806000640 1 -- 172.21.15.47:0/1017422129 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb800044070 con 0x7fb808105270 2024-03-21T11:37:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.524+0000 7fb812c10740 1 -- start start 2024-03-21T11:37:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.525+0000 7fb812c10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3797721e0 0x55d379772600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.525+0000 7fb812c10740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379772bd0 0x55d3797776d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.525+0000 7fb812c10740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 0x55d37977a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.525+0000 7fb812c10740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3797353b0 con 0x55d379772bd0 2024-03-21T11:37:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.525+0000 7fb812c10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3796d7830 con 0x55d3797721e0 2024-03-21T11:37:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.525+0000 7fb812c10740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3796805d0 con 0x55d379777d40 2024-03-21T11:37:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.525+0000 7fb80ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3797721e0 0x55d379772600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.525+0000 7fb80ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3797721e0 0x55d379772600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41988/0 (socket says 172.21.15.47:41988) 2024-03-21T11:37:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.525+0000 7fb80ca00640 1 -- 172.21.15.47:0/2284683949 learned_addr learned my addr 172.21.15.47:0/2284683949 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.525+0000 7fb80ca00640 1 --2- 172.21.15.47:0/2284683949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 0x55d37977a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.525+0000 7fb80d400640 1 --2- 172.21.15.47:0/2284683949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379772bd0 0x55d3797776d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.526+0000 7fb80ca00640 1 -- 172.21.15.47:0/2284683949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 msgr2=0x55d37977a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.526+0000 7fb80ca00640 1 --2- 172.21.15.47:0/2284683949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 0x55d37977a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.526+0000 7fb80ca00640 1 -- 172.21.15.47:0/2284683949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379772bd0 msgr2=0x55d3797776d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.526+0000 7fb80ca00640 1 --2- 172.21.15.47:0/2284683949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379772bd0 0x55d3797776d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.526+0000 7fb80ca00640 1 -- 172.21.15.47:0/2284683949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3796982c0 con 0x55d3797721e0 2024-03-21T11:37:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.526+0000 7fb806000640 1 --2- 172.21.15.47:0/1017422129 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb7f00a2fc0 0x7fb7f00a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.526+0000 7fb806000640 1 -- 172.21.15.47:0/1017422129 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb8000f1330 con 0x7fb808105270 2024-03-21T11:37:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.527+0000 7fb80ca00640 1 --2- 172.21.15.47:0/2284683949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3797721e0 0x55d379772600 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4014870 tx=0x7fb7f4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e3e45bd9bb0c399 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.527+0000 7fb80d400640 1 --2- 172.21.15.47:0/1017422129 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb7f00a2fc0 0x7fb7f00a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.527+0000 7fb7fa600640 1 -- 172.21.15.47:0/2284683949 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4014110 con 0x55d3797721e0 2024-03-21T11:37:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.527+0000 7fb7fa600640 1 -- 172.21.15.47:0/2284683949 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7f40142b0 con 0x55d3797721e0 2024-03-21T11:37:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.527+0000 7fb7fa600640 1 -- 172.21.15.47:0/2284683949 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4004e40 con 0x55d3797721e0 2024-03-21T11:37:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.528+0000 7fb812c10740 1 -- 172.21.15.47:0/2284683949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3797721e0 msgr2=0x55d379772600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.528+0000 7fb812c10740 1 --2- 172.21.15.47:0/2284683949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3797721e0 0x55d379772600 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4014870 tx=0x7fb7f4009f90 comp rx=0 tx=0).stop 2024-03-21T11:37:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.528+0000 7fb80d400640 1 --2- 172.21.15.47:0/1017422129 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb7f00a2fc0 0x7fb7f00a5480 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0099a0 tx=0x7fb7fc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.528+0000 7fb812c10740 1 -- 172.21.15.47:0/2284683949 shutdown_connections 2024-03-21T11:37:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.528+0000 7fb812c10740 1 --2- 172.21.15.47:0/2284683949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 0x55d37977a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.528+0000 7fb812c10740 1 --2- 172.21.15.47:0/2284683949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379772bd0 0x55d3797776d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.528+0000 7fb812c10740 1 --2- 172.21.15.47:0/2284683949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3797721e0 0x55d379772600 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.528+0000 7fb812c10740 1 -- 172.21.15.47:0/2284683949 >> 172.21.15.47:0/2284683949 conn(0x55d37976d750 msgr2=0x55d37976db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.529+0000 7fb812c10740 1 -- 172.21.15.47:0/2284683949 shutdown_connections 2024-03-21T11:37:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.529+0000 7fb812c10740 1 -- 172.21.15.47:0/2284683949 wait complete. 2024-03-21T11:37:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.529+0000 7fb812c10740 1 -- start start 2024-03-21T11:37:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.529+0000 7fb812c10740 1 Processor -- start 2024-03-21T11:37:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.530+0000 7fb812c10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3797721e0 0x55d37977c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.530+0000 7fb812c10740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379772bd0 0x55d37977c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.530+0000 7fb812c10740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 0x55d379783910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.530+0000 7fb812c10740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d379693a20 con 0x55d379772bd0 2024-03-21T11:37:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.530+0000 7fb812c10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3796d38d0 con 0x55d3797721e0 2024-03-21T11:37:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.530+0000 7fb812c10740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d37977acf0 con 0x55d379777d40 2024-03-21T11:37:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.530+0000 7fb812c10740 10 client.0 _ll_register_callbacks cb 0x55d3796d01c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.530+0000 7fb80de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 0x55d379783910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.530+0000 7fb80de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 0x55d379783910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57140/0 (socket says 172.21.15.47:57140) 2024-03-21T11:37:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.530+0000 7fb80de00640 1 -- 172.21.15.47:0/1807993640 learned_addr learned my addr 172.21.15.47:0/1807993640 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.530+0000 7fb80d400640 1 --2- 172.21.15.47:0/1807993640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3797721e0 0x55d37977c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.531+0000 7fb80ca00640 1 --2- 172.21.15.47:0/1807993640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379772bd0 0x55d37977c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.531+0000 7fb80de00640 1 -- 172.21.15.47:0/1807993640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3797721e0 msgr2=0x55d37977c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.531+0000 7fb80de00640 1 --2- 172.21.15.47:0/1807993640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3797721e0 0x55d37977c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.531+0000 7fb80de00640 1 -- 172.21.15.47:0/1807993640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379772bd0 msgr2=0x55d37977c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.531+0000 7fb80de00640 1 --2- 172.21.15.47:0/1807993640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379772bd0 0x55d37977c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.531+0000 7fb80de00640 1 -- 172.21.15.47:0/1807993640 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3793ec210 con 0x55d379777d40 2024-03-21T11:37:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.531+0000 7fb80d400640 1 --2- 172.21.15.47:0/1807993640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3797721e0 0x55d37977c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:37:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.531+0000 7fb80de00640 1 --2- 172.21.15.47:0/1807993640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 0x55d379783910 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb8000b9ad0 tx=0x7fb8000bcfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d13357223e713e8d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.531+0000 7fb7eea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:37:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.532+0000 7fb7eea00640 1 -- 172.21.15.47:0/1807993640 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80004b040 con 0x55d379777d40 2024-03-21T11:37:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.532+0000 7fb7eea00640 1 -- 172.21.15.47:0/1807993640 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb8000035f0 con 0x55d379777d40 2024-03-21T11:37:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.532+0000 7fb812c10740 1 -- 172.21.15.47:0/1807993640 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7f40142b0 con 0x55d379777d40 2024-03-21T11:37:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.533+0000 7fb7eea00640 1 -- 172.21.15.47:0/1807993640 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8000bd670 con 0x55d379777d40 2024-03-21T11:37:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.533+0000 7fb812c10740 10 client.30316 fetch_fsmap learned FSMap version 33 2024-03-21T11:37:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.533+0000 7fb812c10740 1 -- 172.21.15.47:0/1807993640 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d3794b8620 con 0x55d379777d40 2024-03-21T11:37:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.533+0000 7fb7eea00640 1 -- 172.21.15.47:0/1807993640 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7f40142b0 con 0x55d379777d40 2024-03-21T11:37:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.534+0000 7fb7eea00640 1 -- 172.21.15.47:0/1807993640 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fb8000bcac0 con 0x55d379777d40 2024-03-21T11:37:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.534+0000 7fb812c10740 10 client.30316 fetch_fsmap finished waiting for FSMap version 33 2024-03-21T11:37:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.534+0000 7fb812c10740 10 client.30316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.534+0000 7fb812c10740 20 client.30316 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.534+0000 7fb812c10740 1 --2- 172.21.15.47:0/1807993640 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d379788240 0x55d37978a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.534+0000 7fb812c10740 4 client.30316 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:37:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.534+0000 7fb812c10740 1 -- 172.21.15.47:0/1807993640 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3796e4d70 con 0x55d379788240 2024-03-21T11:37:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.534+0000 7fb80d400640 1 --2- 172.21.15.47:0/1807993640 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d379788240 0x55d37978a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.535+0000 7fb80d400640 1 --2- 172.21.15.47:0/1807993640 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d379788240 0x55d37978a6a0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69fb3001ac2e59e3 server_cookie=40dc69468996cb02 in_seq=0 out_seq=0 2024-03-21T11:37:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.535+0000 7fb7eea00640 10 client.30316 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.543+0000 7fb7eea00640 1 -- 172.21.15.47:0/1807993640 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d3796e4d70 con 0x55d379788240 2024-03-21T11:37:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.543+0000 7fb7eea00640 10 client.30316 handle_command_reply: tid=0 2024-03-21T11:37:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.543+0000 7fb812c10740 1 client.30316 shutdown 2024-03-21T11:37:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.543+0000 7fb812c10740 1 -- 172.21.15.47:0/1807993640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 msgr2=0x55d379783910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.543+0000 7fb812c10740 1 --2- 172.21.15.47:0/1807993640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 0x55d379783910 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb8000b9ad0 tx=0x7fb8000bcfd0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.543+0000 7fb812c10740 1 -- 172.21.15.47:0/1807993640 shutdown_connections 2024-03-21T11:37:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.543+0000 7fb812c10740 1 --2- 172.21.15.47:0/1807993640 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d379788240 0x55d37978a6a0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.543+0000 7fb812c10740 1 --2- 172.21.15.47:0/1807993640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 0x55d379783910 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.543+0000 7fb812c10740 1 --2- 172.21.15.47:0/1807993640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379772bd0 0x55d37977c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.543+0000 7fb812c10740 1 --2- 172.21.15.47:0/1807993640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3797721e0 0x55d37977c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.543+0000 7fb812c10740 1 -- 172.21.15.47:0/1807993640 >> 172.21.15.47:0/1807993640 conn(0x55d37976d750 msgr2=0x55d37976c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.543+0000 7fb7eea00640 0 client.30316 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.544+0000 7fb812c10740 1 -- 172.21.15.47:0/1807993640 shutdown_connections 2024-03-21T11:37:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.544+0000 7fb812c10740 1 -- 172.21.15.47:0/1807993640 wait complete. 2024-03-21T11:37:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.544+0000 7fb812c10740 20 client.30316 trim_cache size 0 max 16384 2024-03-21T11:37:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.562+0000 7fb812c10740 1 Processor -- start 2024-03-21T11:37:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.562+0000 7fb812c10740 1 -- start start 2024-03-21T11:37:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb812c10740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3797721e0 0x55d3797709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb812c10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d379772bd0 0x55d3797838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb812c10740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379777d40 0x55d379770ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb812c10740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8081b8cc0 con 0x55d379777d40 2024-03-21T11:37:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb812c10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8081ba7c0 con 0x55d379772bd0 2024-03-21T11:37:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb812c10740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8081b9320 con 0x55d3797721e0 2024-03-21T11:37:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb80d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3797721e0 0x55d3797709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb80ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d379772bd0 0x55d3797838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb80de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379777d40 0x55d379770ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb80ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d379772bd0 0x55d3797838d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42018/0 (socket says 172.21.15.47:42018) 2024-03-21T11:37:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb80d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3797721e0 0x55d3797709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57152/0 (socket says 172.21.15.47:57152) 2024-03-21T11:37:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb80ca00640 1 -- 172.21.15.47:0/2792365277 learned_addr learned my addr 172.21.15.47:0/2792365277 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb80ca00640 1 -- 172.21.15.47:0/2792365277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3797721e0 msgr2=0x55d3797709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb80ca00640 1 --2- 172.21.15.47:0/2792365277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3797721e0 0x55d3797709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb80ca00640 1 -- 172.21.15.47:0/2792365277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379777d40 msgr2=0x55d379770ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.563+0000 7fb80ca00640 1 --2- 172.21.15.47:0/2792365277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379777d40 0x55d379770ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.564+0000 7fb80ca00640 1 -- 172.21.15.47:0/2792365277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d379773460 con 0x55d379772bd0 2024-03-21T11:37:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.564+0000 7fb80ca00640 1 --2- 172.21.15.47:0/2792365277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d379772bd0 0x55d3797838d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4014840 tx=0x7fb7f4009470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a720f3ad2c3aafa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.564+0000 7fb7f9c00640 1 -- 172.21.15.47:0/2792365277 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4014110 con 0x55d379772bd0 2024-03-21T11:37:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.565+0000 7fb7f9c00640 1 -- 172.21.15.47:0/2792365277 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7f4005520 con 0x55d379772bd0 2024-03-21T11:37:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.565+0000 7fb7f9c00640 1 -- 172.21.15.47:0/2792365277 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400fad0 con 0x55d379772bd0 2024-03-21T11:37:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.565+0000 7fb812c10740 1 -- 172.21.15.47:0/2792365277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d379772bd0 msgr2=0x55d3797838d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.565+0000 7fb812c10740 1 --2- 172.21.15.47:0/2792365277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d379772bd0 0x55d3797838d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4014840 tx=0x7fb7f4009470 comp rx=0 tx=0).stop 2024-03-21T11:37:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.566+0000 7fb812c10740 1 -- 172.21.15.47:0/2792365277 shutdown_connections 2024-03-21T11:37:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.566+0000 7fb812c10740 1 --2- 172.21.15.47:0/2792365277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d379777d40 0x55d379770ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.566+0000 7fb812c10740 1 --2- 172.21.15.47:0/2792365277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d379772bd0 0x55d3797838d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.566+0000 7fb812c10740 1 --2- 172.21.15.47:0/2792365277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3797721e0 0x55d3797709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.566+0000 7fb812c10740 1 -- 172.21.15.47:0/2792365277 >> 172.21.15.47:0/2792365277 conn(0x55d37976d750 msgr2=0x55d37977c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.566+0000 7fb812c10740 1 -- 172.21.15.47:0/2792365277 shutdown_connections 2024-03-21T11:37:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.566+0000 7fb812c10740 1 -- 172.21.15.47:0/2792365277 wait complete. 2024-03-21T11:37:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.566+0000 7fb812c10740 1 -- start start 2024-03-21T11:37:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.566+0000 7fb812c10740 1 Processor -- start 2024-03-21T11:37:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.567+0000 7fb812c10740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3797721e0 0x55d379781540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.567+0000 7fb812c10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d379772bd0 0x55d379781a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.567+0000 7fb812c10740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 0x55d379782a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.567+0000 7fb812c10740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8081ba0a0 con 0x55d3797721e0 2024-03-21T11:37:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.567+0000 7fb812c10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8081b9a40 con 0x55d379772bd0 2024-03-21T11:37:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.567+0000 7fb812c10740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80810e960 con 0x55d379777d40 2024-03-21T11:37:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.567+0000 7fb812c10740 10 client.0 _ll_register_callbacks cb 0x7fb800009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.567+0000 7fb80ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d379772bd0 0x55d379781a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.567+0000 7fb80ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d379772bd0 0x55d379781a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42020/0 (socket says 172.21.15.47:42020) 2024-03-21T11:37:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.567+0000 7fb80ca00640 1 -- 172.21.15.47:0/4163769569 learned_addr learned my addr 172.21.15.47:0/4163769569 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.567+0000 7fb80d400640 1 --2- 172.21.15.47:0/4163769569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3797721e0 0x55d379781540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.568+0000 7fb80ca00640 1 -- 172.21.15.47:0/4163769569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 msgr2=0x55d379782a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.568+0000 7fb80ca00640 1 --2- 172.21.15.47:0/4163769569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 0x55d379782a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.568+0000 7fb80ca00640 1 -- 172.21.15.47:0/4163769569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3797721e0 msgr2=0x55d379781540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.568+0000 7fb80ca00640 1 --2- 172.21.15.47:0/4163769569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3797721e0 0x55d379781540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.568+0000 7fb80ca00640 1 -- 172.21.15.47:0/4163769569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d379783010 con 0x55d379772bd0 2024-03-21T11:37:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.568+0000 7fb80ca00640 1 --2- 172.21.15.47:0/4163769569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d379772bd0 0x55d379781a80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009fd0 tx=0x7fb7f40144b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d80144c516921e00 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.568+0000 7fb7eea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:37:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.568+0000 7fb7eea00640 1 -- 172.21.15.47:0/4163769569 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4032070 con 0x55d379772bd0 2024-03-21T11:37:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.568+0000 7fb7eea00640 1 -- 172.21.15.47:0/4163769569 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb7f400f810 con 0x55d379772bd0 2024-03-21T11:37:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.568+0000 7fb812c10740 1 -- 172.21.15.47:0/4163769569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7f4005520 con 0x55d379772bd0 2024-03-21T11:37:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.569+0000 7fb7eea00640 1 -- 172.21.15.47:0/4163769569 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7f403e050 con 0x55d379772bd0 2024-03-21T11:37:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.569+0000 7fb812c10740 10 client.30308 fetch_fsmap learned FSMap version 33 2024-03-21T11:37:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.569+0000 7fb812c10740 1 -- 172.21.15.47:0/4163769569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d3797e3ff0 con 0x55d379772bd0 2024-03-21T11:37:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.569+0000 7fb7eea00640 1 -- 172.21.15.47:0/4163769569 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7f4005520 con 0x55d379772bd0 2024-03-21T11:37:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.570+0000 7fb7eea00640 1 -- 172.21.15.47:0/4163769569 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 33) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fb7f402bbe0 con 0x55d379772bd0 2024-03-21T11:37:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.570+0000 7fb812c10740 10 client.30308 fetch_fsmap finished waiting for FSMap version 33 2024-03-21T11:37:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.570+0000 7fb812c10740 10 client.30308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.570+0000 7fb812c10740 20 client.30308 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.570+0000 7fb812c10740 1 --2- 172.21.15.47:0/4163769569 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3797e46a0 0x55d3797e6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.570+0000 7fb812c10740 4 client.30308 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:37:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.570+0000 7fb812c10740 1 -- 172.21.15.47:0/4163769569 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d3796df0c0 con 0x55d3797e46a0 2024-03-21T11:37:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.570+0000 7fb80d400640 1 --2- 172.21.15.47:0/4163769569 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3797e46a0 0x55d3797e6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.570+0000 7fb80d400640 1 --2- 172.21.15.47:0/4163769569 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3797e46a0 0x55d3797e6b20 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e586b4b6477fd49 server_cookie=7567d61c92963a69 in_seq=0 out_seq=0 2024-03-21T11:37:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:28.570+0000 7fb7eea00640 10 client.30308 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:28.918 INFO:tasks.workunit.client.0.smithi047.stdout: 10 7087 45.36 MB/sec warmup 12 sec latency 263.314 ms 2024-03-21T11:37:29.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: pgmap v824: 129 pgs: 129 active+clean; 43 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 372 op/s 2024-03-21T11:37:29.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:37:29.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:37:29.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:37:29.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:37:29.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:37:29.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:37:29.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:37:29.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:37:29.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:37:29.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:28 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: pgmap v824: 129 pgs: 129 active+clean; 43 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 372 op/s 2024-03-21T11:37:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:37:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:37:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:37:29.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:37:29.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:37:29.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:37:29.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:37:29.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:37:29.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:37:29.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:28 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.359+0000 7fb7eea00640 1 -- 172.21.15.47:0/4163769569 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d3796df0c0 con 0x55d3797e46a0 2024-03-21T11:37:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.359+0000 7fb7eea00640 10 client.30308 handle_command_reply: tid=0 2024-03-21T11:37:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.360+0000 7fb812c10740 1 client.30308 shutdown 2024-03-21T11:37:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.360+0000 7fb812c10740 1 -- 172.21.15.47:0/4163769569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d379772bd0 msgr2=0x55d379781a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.360+0000 7fb812c10740 1 --2- 172.21.15.47:0/4163769569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d379772bd0 0x55d379781a80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009fd0 tx=0x7fb7f40144b0 comp rx=0 tx=0).stop 2024-03-21T11:37:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.360+0000 7fb812c10740 1 -- 172.21.15.47:0/4163769569 shutdown_connections 2024-03-21T11:37:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.360+0000 7fb812c10740 1 --2- 172.21.15.47:0/4163769569 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3797e46a0 0x55d3797e6b20 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.360+0000 7fb812c10740 1 --2- 172.21.15.47:0/4163769569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d379777d40 0x55d379782a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.360+0000 7fb812c10740 1 --2- 172.21.15.47:0/4163769569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d379772bd0 0x55d379781a80 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.360+0000 7fb812c10740 1 --2- 172.21.15.47:0/4163769569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3797721e0 0x55d379781540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.360+0000 7fb812c10740 1 -- 172.21.15.47:0/4163769569 >> 172.21.15.47:0/4163769569 conn(0x55d37976d750 msgr2=0x55d37977c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.360+0000 7fb7eea00640 0 client.30308 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.361+0000 7fb812c10740 1 -- 172.21.15.47:0/4163769569 shutdown_connections 2024-03-21T11:37:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.361+0000 7fb812c10740 1 -- 172.21.15.47:0/4163769569 wait complete. 2024-03-21T11:37:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.361+0000 7fb812c10740 20 client.30308 trim_cache size 0 max 16384 2024-03-21T11:37:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:37:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:37:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:37:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:37:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.362+0000 7fb80fe00640 1 -- 172.21.15.47:0/1017422129 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb7f00a2fc0 msgr2=0x7fb7f00a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.362+0000 7fb80fe00640 1 --2- 172.21.15.47:0/1017422129 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb7f00a2fc0 0x7fb7f00a5480 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0099a0 tx=0x7fb7fc009500 comp rx=0 tx=0).stop 2024-03-21T11:37:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.362+0000 7fb80fe00640 1 -- 172.21.15.47:0/1017422129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808105270 msgr2=0x7fb8081a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.362+0000 7fb80fe00640 1 --2- 172.21.15.47:0/1017422129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808105270 0x7fb8081a5430 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb800009850 tx=0x7fb800005d50 comp rx=0 tx=0).stop 2024-03-21T11:37:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.363+0000 7fb80fe00640 1 -- 172.21.15.47:0/1017422129 shutdown_connections 2024-03-21T11:37:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.363+0000 7fb80fe00640 1 --2- 172.21.15.47:0/1017422129 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb7f00a2fc0 0x7fb7f00a5480 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.363+0000 7fb80fe00640 1 --2- 172.21.15.47:0/1017422129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb808105270 0x7fb8081a5430 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.363+0000 7fb80fe00640 1 --2- 172.21.15.47:0/1017422129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8081028c0 0x7fb80819e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.363+0000 7fb80fe00640 1 --2- 172.21.15.47:0/1017422129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8080fff50 0x7fb80819de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.363+0000 7fb80fe00640 1 -- 172.21.15.47:0/1017422129 >> 172.21.15.47:0/1017422129 conn(0x7fb8080fbb30 msgr2=0x7fb808100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.363+0000 7fb80fe00640 1 -- 172.21.15.47:0/1017422129 shutdown_connections 2024-03-21T11:37:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:29.363+0000 7fb80fe00640 1 -- 172.21.15.47:0/1017422129 wait complete. 2024-03-21T11:37:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: pgmap v824: 129 pgs: 129 active+clean; 43 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 372 op/s 2024-03-21T11:37:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:37:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:37:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:37:29.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:37:29.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:37:29.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:37:29.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:37:29.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:37:29.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:37:29.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:28 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:29.918 INFO:tasks.workunit.client.0.smithi047.stdout: 10 7566 44.42 MB/sec warmup 13 sec latency 607.101 ms 2024-03-21T11:37:30.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:29 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:37:30.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:29 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:30.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:29 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:30.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:29 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:30.218 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:29 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:29 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:37:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:29 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:29 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:29 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:29 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:30.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:29 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:37:30.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:29 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:30.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:29 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:30.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:29 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:37:30.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:29 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:37:30.918 INFO:tasks.workunit.client.0.smithi047.stdout: 10 7806 42.41 MB/sec warmup 14 sec latency 369.155 ms 2024-03-21T11:37:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:30 smithi102 ceph-mon[36748]: pgmap v825: 129 pgs: 129 active+clean; 52 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-03-21T11:37:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:30 smithi047 ceph-mon[31351]: pgmap v825: 129 pgs: 129 active+clean; 52 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-03-21T11:37:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:30 smithi100 ceph-mon[36245]: pgmap v825: 129 pgs: 129 active+clean; 52 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-03-21T11:37:31.918 INFO:tasks.workunit.client.0.smithi047.stdout: 10 8286 41.31 MB/sec warmup 15 sec latency 146.412 ms 2024-03-21T11:37:32.918 INFO:tasks.workunit.client.0.smithi047.stdout: 10 8830 40.84 MB/sec warmup 16 sec latency 70.809 ms 2024-03-21T11:37:33.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:32 smithi102 ceph-mon[36748]: pgmap v826: 129 pgs: 129 active+clean; 56 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 488 op/s 2024-03-21T11:37:33.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:32 smithi047 ceph-mon[31351]: pgmap v826: 129 pgs: 129 active+clean; 56 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 488 op/s 2024-03-21T11:37:33.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:32 smithi100 ceph-mon[36245]: pgmap v826: 129 pgs: 129 active+clean; 56 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 488 op/s 2024-03-21T11:37:33.918 INFO:tasks.workunit.client.0.smithi047.stdout: 10 9478 40.25 MB/sec warmup 17 sec latency 36.805 ms 2024-03-21T11:37:34.920 INFO:tasks.workunit.client.0.smithi047.stdout: 10 10000 39.95 MB/sec warmup 18 sec latency 70.075 ms 2024-03-21T11:37:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:34 smithi102 ceph-mon[36748]: pgmap v827: 129 pgs: 129 active+clean; 102 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 739 op/s 2024-03-21T11:37:35.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:34 smithi047 ceph-mon[31351]: pgmap v827: 129 pgs: 129 active+clean; 102 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 739 op/s 2024-03-21T11:37:35.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:34 smithi100 ceph-mon[36245]: pgmap v827: 129 pgs: 129 active+clean; 102 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 739 op/s 2024-03-21T11:37:35.920 INFO:tasks.workunit.client.0.smithi047.stdout: 10 10273 38.39 MB/sec warmup 19 sec latency 233.420 ms 2024-03-21T11:37:36.920 INFO:tasks.workunit.client.0.smithi047.stdout: 10 10332 36.79 MB/sec warmup 20 sec latency 772.402 ms 2024-03-21T11:37:37.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:36 smithi102 ceph-mon[36748]: pgmap v828: 129 pgs: 129 active+clean; 112 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 695 op/s 2024-03-21T11:37:37.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:36 smithi047 ceph-mon[31351]: pgmap v828: 129 pgs: 129 active+clean; 112 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 695 op/s 2024-03-21T11:37:37.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:36 smithi100 ceph-mon[36245]: pgmap v828: 129 pgs: 129 active+clean; 112 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 695 op/s 2024-03-21T11:37:37.920 INFO:tasks.workunit.client.0.smithi047.stdout: 10 10534 35.32 MB/sec warmup 21 sec latency 511.375 ms 2024-03-21T11:37:38.920 INFO:tasks.workunit.client.0.smithi047.stdout: 10 10888 34.35 MB/sec warmup 22 sec latency 122.362 ms 2024-03-21T11:37:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:38 smithi102 ceph-mon[36748]: pgmap v829: 129 pgs: 129 active+clean; 159 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 15 MiB/s wr, 738 op/s 2024-03-21T11:37:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:38 smithi047 ceph-mon[31351]: pgmap v829: 129 pgs: 129 active+clean; 159 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 15 MiB/s wr, 738 op/s 2024-03-21T11:37:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:38 smithi100 ceph-mon[36245]: pgmap v829: 129 pgs: 129 active+clean; 159 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 15 MiB/s wr, 738 op/s 2024-03-21T11:37:39.920 INFO:tasks.workunit.client.0.smithi047.stdout: 10 11748 34.43 MB/sec warmup 23 sec latency 58.276 ms 2024-03-21T11:37:40.920 INFO:tasks.workunit.client.0.smithi047.stdout: 10 12546 35.48 MB/sec warmup 24 sec latency 64.837 ms 2024-03-21T11:37:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:40 smithi102 ceph-mon[36748]: pgmap v830: 129 pgs: 129 active+clean; 241 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 23 MiB/s wr, 842 op/s 2024-03-21T11:37:41.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:40 smithi047 ceph-mon[31351]: pgmap v830: 129 pgs: 129 active+clean; 241 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 23 MiB/s wr, 842 op/s 2024-03-21T11:37:41.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:40 smithi100 ceph-mon[36245]: pgmap v830: 129 pgs: 129 active+clean; 241 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 23 MiB/s wr, 842 op/s 2024-03-21T11:37:41.920 INFO:tasks.workunit.client.0.smithi047.stdout: 10 13154 35.83 MB/sec warmup 25 sec latency 64.305 ms 2024-03-21T11:37:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.543+0000 7fea91e00640 1 -- 172.21.15.47:0/819319059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea8c0740c0 msgr2=0x7fea8c074540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.543+0000 7fea91e00640 1 --2- 172.21.15.47:0/819319059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea8c0740c0 0x7fea8c074540 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fea74009900 tx=0x7fea7402bc40 comp rx=0 tx=0).stop 2024-03-21T11:37:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.543+0000 7fea91e00640 1 -- 172.21.15.47:0/819319059 shutdown_connections 2024-03-21T11:37:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.543+0000 7fea91e00640 1 --2- 172.21.15.47:0/819319059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8c1095b0 0x7fea8c10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.543+0000 7fea91e00640 1 --2- 172.21.15.47:0/819319059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea8c0740c0 0x7fea8c074540 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.543+0000 7fea91e00640 1 --2- 172.21.15.47:0/819319059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c0756a0 0x7fea8c073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.543+0000 7fea91e00640 1 -- 172.21.15.47:0/819319059 >> 172.21.15.47:0/819319059 conn(0x7fea8c0fcc70 msgr2=0x7fea8c0ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.544+0000 7fea91e00640 1 -- 172.21.15.47:0/819319059 shutdown_connections 2024-03-21T11:37:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.544+0000 7fea91e00640 1 -- 172.21.15.47:0/819319059 wait complete. 2024-03-21T11:37:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.545+0000 7fea91e00640 1 Processor -- start 2024-03-21T11:37:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.546+0000 7fea91e00640 1 -- start start 2024-03-21T11:37:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.546+0000 7fea91e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c0740c0 0x7fea8c199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.546+0000 7fea91e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8c0756a0 0x7fea8c19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.546+0000 7fea91e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea8c1095b0 0x7fea8c1a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.546+0000 7fea91e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c10f540 con 0x7fea8c0756a0 2024-03-21T11:37:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.546+0000 7fea91e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c10f3c0 con 0x7fea8c1095b0 2024-03-21T11:37:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.546+0000 7fea91e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c10f6c0 con 0x7fea8c0740c0 2024-03-21T11:37:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.546+0000 7fea8aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c0740c0 0x7fea8c199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.546+0000 7fea8aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c0740c0 0x7fea8c199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:39856/0 (socket says 172.21.15.47:39856) 2024-03-21T11:37:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.546+0000 7fea8aa00640 1 -- 172.21.15.47:0/2244154331 learned_addr learned my addr 172.21.15.47:0/2244154331 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.546+0000 7fea8b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea8c1095b0 0x7fea8c1a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.547+0000 7fea83e00640 1 --2- 172.21.15.47:0/2244154331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8c0756a0 0x7fea8c19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.547+0000 7fea8aa00640 1 -- 172.21.15.47:0/2244154331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea8c1095b0 msgr2=0x7fea8c1a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.547+0000 7fea8aa00640 1 --2- 172.21.15.47:0/2244154331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea8c1095b0 0x7fea8c1a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.547+0000 7fea8aa00640 1 -- 172.21.15.47:0/2244154331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8c0756a0 msgr2=0x7fea8c19a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.547+0000 7fea8aa00640 1 --2- 172.21.15.47:0/2244154331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8c0756a0 0x7fea8c19a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.547+0000 7fea8aa00640 1 -- 172.21.15.47:0/2244154331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8c1a16d0 con 0x7fea8c0740c0 2024-03-21T11:37:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.547+0000 7fea83e00640 1 --2- 172.21.15.47:0/2244154331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8c0756a0 0x7fea8c19a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:37:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.547+0000 7fea8aa00640 1 --2- 172.21.15.47:0/2244154331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c0740c0 0x7fea8c199b00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00ea60 tx=0x7fea7c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.548+0000 7fea90a00640 1 -- 172.21.15.47:0/2244154331 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c00cde0 con 0x7fea8c0740c0 2024-03-21T11:37:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.548+0000 7fea91e00640 1 -- 172.21.15.47:0/2244154331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8c1a19c0 con 0x7fea8c0740c0 2024-03-21T11:37:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.548+0000 7fea90a00640 1 -- 172.21.15.47:0/2244154331 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea7c0035f0 con 0x7fea8c0740c0 2024-03-21T11:37:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.548+0000 7fea90a00640 1 -- 172.21.15.47:0/2244154331 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c010670 con 0x7fea8c0740c0 2024-03-21T11:37:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.548+0000 7fea91e00640 1 -- 172.21.15.47:0/2244154331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8c1a1f30 con 0x7fea8c0740c0 2024-03-21T11:37:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.552+0000 7fea94bd3740 1 Processor -- start 2024-03-21T11:37:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.553+0000 7fea94bd3740 1 -- start start 2024-03-21T11:37:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.553+0000 7fea94bd3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af50600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.553+0000 7fea94bd3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af556d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.553+0000 7fea94bd3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 0x56425af581d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.553+0000 7fea94bd3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425af133b0 con 0x56425af55d40 2024-03-21T11:37:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.553+0000 7fea94bd3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425aeb5830 con 0x56425af50bd0 2024-03-21T11:37:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.553+0000 7fea94bd3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425ae5e5d0 con 0x56425af501e0 2024-03-21T11:37:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.553+0000 7fea90a00640 1 -- 172.21.15.47:0/2244154331 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fea7c0037a0 con 0x7fea8c0740c0 2024-03-21T11:37:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.553+0000 7fea8b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af50600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.553+0000 7fea8b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af50600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:39866/0 (socket says 172.21.15.47:39866) 2024-03-21T11:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.553+0000 7fea8b400640 1 -- 172.21.15.47:0/3521872177 learned_addr learned my addr 172.21.15.47:0/3521872177 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.554+0000 7fea8aa00640 1 --2- 172.21.15.47:0/3521872177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.554+0000 7fea83e00640 1 --2- 172.21.15.47:0/3521872177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 0x56425af581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.554+0000 7fea8b400640 1 -- 172.21.15.47:0/3521872177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 msgr2=0x56425af556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.554+0000 7fea8b400640 1 --2- 172.21.15.47:0/3521872177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.554+0000 7fea8b400640 1 -- 172.21.15.47:0/3521872177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 msgr2=0x56425af581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.554+0000 7fea8b400640 1 --2- 172.21.15.47:0/3521872177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 0x56425af581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.554+0000 7fea8b400640 1 -- 172.21.15.47:0/3521872177 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56425ae762c0 con 0x56425af501e0 2024-03-21T11:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.554+0000 7fea83e00640 1 --2- 172.21.15.47:0/3521872177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 0x56425af581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.554+0000 7fea8aa00640 1 --2- 172.21.15.47:0/3521872177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.554+0000 7fea8b400640 1 --2- 172.21.15.47:0/3521872177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af50600 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fea84009fd0 tx=0x7fea840118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ca59fed68b91ef7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.555+0000 7fea7be00640 1 -- 172.21.15.47:0/3521872177 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84004d20 con 0x56425af501e0 2024-03-21T11:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.555+0000 7fea7be00640 1 -- 172.21.15.47:0/3521872177 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea84004050 con 0x56425af501e0 2024-03-21T11:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.555+0000 7fea7be00640 1 -- 172.21.15.47:0/3521872177 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84005260 con 0x56425af501e0 2024-03-21T11:37:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.555+0000 7fea90a00640 1 --2- 172.21.15.47:0/2244154331 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fea700a3040 0x7fea700a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.555+0000 7fea90a00640 1 -- 172.21.15.47:0/2244154331 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fea7c0cfaa0 con 0x7fea8c0740c0 2024-03-21T11:37:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.556+0000 7fea8aa00640 1 --2- 172.21.15.47:0/2244154331 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fea700a3040 0x7fea700a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.556+0000 7fea94bd3740 1 -- 172.21.15.47:0/3521872177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 msgr2=0x56425af50600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.556+0000 7fea94bd3740 1 --2- 172.21.15.47:0/3521872177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af50600 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fea84009fd0 tx=0x7fea840118d0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.556+0000 7fea94bd3740 1 -- 172.21.15.47:0/3521872177 shutdown_connections 2024-03-21T11:37:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.556+0000 7fea94bd3740 1 --2- 172.21.15.47:0/3521872177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 0x56425af581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.556+0000 7fea94bd3740 1 --2- 172.21.15.47:0/3521872177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.556+0000 7fea94bd3740 1 --2- 172.21.15.47:0/3521872177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af50600 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.556+0000 7fea94bd3740 1 -- 172.21.15.47:0/3521872177 >> 172.21.15.47:0/3521872177 conn(0x56425af4b750 msgr2=0x56425af4bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.556+0000 7fea94bd3740 1 -- 172.21.15.47:0/3521872177 shutdown_connections 2024-03-21T11:37:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.556+0000 7fea94bd3740 1 -- 172.21.15.47:0/3521872177 wait complete. 2024-03-21T11:37:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.556+0000 7fea8aa00640 1 --2- 172.21.15.47:0/2244154331 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fea700a3040 0x7fea700a5500 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0cd1b0 tx=0x7fea7c00ef90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.557+0000 7fea94bd3740 1 -- start start 2024-03-21T11:37:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.557+0000 7fea94bd3740 1 Processor -- start 2024-03-21T11:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.558+0000 7fea94bd3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af5a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.558+0000 7fea94bd3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af5a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.558+0000 7fea94bd3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 0x56425af61920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.558+0000 7fea94bd3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425ae71a20 con 0x56425af55d40 2024-03-21T11:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.558+0000 7fea94bd3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425aeb18d0 con 0x56425af50bd0 2024-03-21T11:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.558+0000 7fea94bd3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425af58cf0 con 0x56425af501e0 2024-03-21T11:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.558+0000 7fea83e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af5a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.558+0000 7fea8b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af5a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.558+0000 7fea8b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af5a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:39880/0 (socket says 172.21.15.47:39880) 2024-03-21T11:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.558+0000 7fea8b400640 1 -- 172.21.15.47:0/871773086 learned_addr learned my addr 172.21.15.47:0/871773086 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.558+0000 7fea8b400640 1 --2- 172.21.15.47:0/871773086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 0x56425af61920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.558+0000 7fea94bd3740 10 client.0 _ll_register_callbacks cb 0x56425aeae1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.558+0000 7fea83e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af5a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33444/0 (socket says 172.21.15.47:33444) 2024-03-21T11:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.559+0000 7fea8b400640 1 -- 172.21.15.47:0/871773086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 msgr2=0x56425af5a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.559+0000 7fea8b400640 1 --2- 172.21.15.47:0/871773086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af5a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.559+0000 7fea8b400640 1 -- 172.21.15.47:0/871773086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 msgr2=0x56425af61920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.559+0000 7fea8b400640 1 --2- 172.21.15.47:0/871773086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 0x56425af61920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.559+0000 7fea8b400640 1 -- 172.21.15.47:0/871773086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56425abca210 con 0x56425af501e0 2024-03-21T11:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.559+0000 7fea8b400640 1 --2- 172.21.15.47:0/871773086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af5a3a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fea84004730 tx=0x7fea84000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aef44a7094b7d6ab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.559+0000 7fea78c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:37:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.560+0000 7fea78c00640 1 -- 172.21.15.47:0/871773086 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8402c020 con 0x56425af501e0 2024-03-21T11:37:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.560+0000 7fea78c00640 1 -- 172.21.15.47:0/871773086 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fea8400f6f0 con 0x56425af501e0 2024-03-21T11:37:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.560+0000 7fea94bd3740 1 -- 172.21.15.47:0/871773086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea84004050 con 0x56425af501e0 2024-03-21T11:37:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.560+0000 7fea78c00640 1 -- 172.21.15.47:0/871773086 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea8400f8f0 con 0x56425af501e0 2024-03-21T11:37:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.560+0000 7fea94bd3740 10 client.30331 fetch_fsmap learned FSMap version 34 2024-03-21T11:37:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.560+0000 7fea94bd3740 1 -- 172.21.15.47:0/871773086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56425ac96620 con 0x56425af501e0 2024-03-21T11:37:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.561+0000 7fea78c00640 1 -- 172.21.15.47:0/871773086 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea84004050 con 0x56425af501e0 2024-03-21T11:37:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.561+0000 7fea78c00640 1 -- 172.21.15.47:0/871773086 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fea84028b30 con 0x56425af501e0 2024-03-21T11:37:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.561+0000 7fea94bd3740 10 client.30331 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:37:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.561+0000 7fea94bd3740 10 client.30331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.561+0000 7fea94bd3740 20 client.30331 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.561+0000 7fea94bd3740 1 --2- 172.21.15.47:0/871773086 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56425af65f80 0x56425af683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.561+0000 7fea94bd3740 4 client.30331 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:37:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.561+0000 7fea94bd3740 1 -- 172.21.15.47:0/871773086 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56425aec2d70 con 0x56425af65f80 2024-03-21T11:37:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.562+0000 7fea83e00640 1 --2- 172.21.15.47:0/871773086 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56425af65f80 0x56425af683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.562+0000 7fea83e00640 1 --2- 172.21.15.47:0/871773086 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56425af65f80 0x56425af683a0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbb710ff70c3d9a server_cookie=11c0addcbaaaacbf in_seq=0 out_seq=0 2024-03-21T11:37:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.562+0000 7fea78c00640 10 client.30331 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.564+0000 7fea78c00640 1 -- 172.21.15.47:0/871773086 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56425aec2d70 con 0x56425af65f80 2024-03-21T11:37:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.564+0000 7fea78c00640 10 client.30331 handle_command_reply: tid=0 2024-03-21T11:37:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.565+0000 7fea94bd3740 1 client.30331 shutdown 2024-03-21T11:37:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.565+0000 7fea94bd3740 1 -- 172.21.15.47:0/871773086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 msgr2=0x56425af5a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.565+0000 7fea94bd3740 1 --2- 172.21.15.47:0/871773086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af5a3a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fea84004730 tx=0x7fea84000f40 comp rx=0 tx=0).stop 2024-03-21T11:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.565+0000 7fea94bd3740 1 -- 172.21.15.47:0/871773086 shutdown_connections 2024-03-21T11:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.565+0000 7fea94bd3740 1 --2- 172.21.15.47:0/871773086 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56425af65f80 0x56425af683a0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.565+0000 7fea94bd3740 1 --2- 172.21.15.47:0/871773086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 0x56425af61920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.565+0000 7fea94bd3740 1 --2- 172.21.15.47:0/871773086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af5a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.565+0000 7fea94bd3740 1 --2- 172.21.15.47:0/871773086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af5a3a0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.565+0000 7fea94bd3740 1 -- 172.21.15.47:0/871773086 >> 172.21.15.47:0/871773086 conn(0x56425af4b750 msgr2=0x56425af4a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.565+0000 7fea78c00640 0 client.30331 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.565+0000 7fea94bd3740 1 -- 172.21.15.47:0/871773086 shutdown_connections 2024-03-21T11:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.565+0000 7fea94bd3740 1 -- 172.21.15.47:0/871773086 wait complete. 2024-03-21T11:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.565+0000 7fea94bd3740 20 client.30331 trim_cache size 0 max 16384 2024-03-21T11:37:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.585+0000 7fea94bd3740 1 Processor -- start 2024-03-21T11:37:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.586+0000 7fea94bd3740 1 -- start start 2024-03-21T11:37:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.586+0000 7fea94bd3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af51eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.586+0000 7fea94bd3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af50bd0 0x56425af523f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.586+0000 7fea94bd3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af55d40 0x56425af618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.586+0000 7fea94bd3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c1ac220 con 0x56425af50bd0 2024-03-21T11:37:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.586+0000 7fea94bd3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c1add20 con 0x56425af55d40 2024-03-21T11:37:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.586+0000 7fea94bd3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c1ac880 con 0x56425af501e0 2024-03-21T11:37:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.586+0000 7fea8b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af51eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.587+0000 7fea8b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af51eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:39890/0 (socket says 172.21.15.47:39890) 2024-03-21T11:37:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.587+0000 7fea8b400640 1 -- 172.21.15.47:0/2087214938 learned_addr learned my addr 172.21.15.47:0/2087214938 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.587+0000 7fea8b400640 1 --2- 172.21.15.47:0/2087214938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af55d40 0x56425af618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.587+0000 7fea83e00640 1 --2- 172.21.15.47:0/2087214938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af50bd0 0x56425af523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.587+0000 7fea83e00640 1 -- 172.21.15.47:0/2087214938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 msgr2=0x56425af51eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.587+0000 7fea83e00640 1 --2- 172.21.15.47:0/2087214938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af51eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.587+0000 7fea83e00640 1 -- 172.21.15.47:0/2087214938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af55d40 msgr2=0x56425af618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.587+0000 7fea83e00640 1 --2- 172.21.15.47:0/2087214938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af55d40 0x56425af618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.587+0000 7fea83e00640 1 -- 172.21.15.47:0/2087214938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56425af4fe20 con 0x56425af50bd0 2024-03-21T11:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.587+0000 7fea8b400640 1 --2- 172.21.15.47:0/2087214938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af51eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.588+0000 7fea83e00640 1 --2- 172.21.15.47:0/2087214938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af50bd0 0x56425af523f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fea74009230 tx=0x7fea74005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a8a5691d66eb47 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.588+0000 7fea7b400640 1 -- 172.21.15.47:0/2087214938 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7404c070 con 0x56425af50bd0 2024-03-21T11:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.588+0000 7fea7b400640 1 -- 172.21.15.47:0/2087214938 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea7402cd10 con 0x56425af50bd0 2024-03-21T11:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.588+0000 7fea7b400640 1 -- 172.21.15.47:0/2087214938 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74004920 con 0x56425af50bd0 2024-03-21T11:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.589+0000 7fea94bd3740 1 -- 172.21.15.47:0/2087214938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af50bd0 msgr2=0x56425af523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.589+0000 7fea94bd3740 1 --2- 172.21.15.47:0/2087214938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af50bd0 0x56425af523f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fea74009230 tx=0x7fea74005a20 comp rx=0 tx=0).stop 2024-03-21T11:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.589+0000 7fea94bd3740 1 -- 172.21.15.47:0/2087214938 shutdown_connections 2024-03-21T11:37:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.589+0000 7fea94bd3740 1 --2- 172.21.15.47:0/2087214938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af55d40 0x56425af618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.589+0000 7fea94bd3740 1 --2- 172.21.15.47:0/2087214938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af50bd0 0x56425af523f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.589+0000 7fea94bd3740 1 --2- 172.21.15.47:0/2087214938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af51eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.589+0000 7fea94bd3740 1 -- 172.21.15.47:0/2087214938 >> 172.21.15.47:0/2087214938 conn(0x56425af4b750 msgr2=0x56425af55520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.590+0000 7fea94bd3740 1 -- 172.21.15.47:0/2087214938 shutdown_connections 2024-03-21T11:37:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.590+0000 7fea94bd3740 1 -- 172.21.15.47:0/2087214938 wait complete. 2024-03-21T11:37:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.590+0000 7fea94bd3740 1 -- start start 2024-03-21T11:37:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.590+0000 7fea94bd3740 1 Processor -- start 2024-03-21T11:37:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.591+0000 7fea94bd3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af5eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.591+0000 7fea94bd3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af5f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.591+0000 7fea94bd3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 0x56425af60040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.591+0000 7fea94bd3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c1ad600 con 0x56425af55d40 2024-03-21T11:37:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.591+0000 7fea94bd3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c1acfa0 con 0x56425af50bd0 2024-03-21T11:37:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.591+0000 7fea94bd3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c1aa3e0 con 0x56425af501e0 2024-03-21T11:37:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.591+0000 7fea94bd3740 10 client.0 _ll_register_callbacks cb 0x7fea84009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.591+0000 7fea83e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af5f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.591+0000 7fea8b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af5eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.591+0000 7fea83e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af5f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33460/0 (socket says 172.21.15.47:33460) 2024-03-21T11:37:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.591+0000 7fea83e00640 1 -- 172.21.15.47:0/823981901 learned_addr learned my addr 172.21.15.47:0/823981901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.591+0000 7fea8b400640 1 --2- 172.21.15.47:0/823981901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 0x56425af60040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.592+0000 7fea83e00640 1 -- 172.21.15.47:0/823981901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 msgr2=0x56425af5eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.592+0000 7fea83e00640 1 --2- 172.21.15.47:0/823981901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af5eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.592+0000 7fea83e00640 1 -- 172.21.15.47:0/823981901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 msgr2=0x56425af60040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.592+0000 7fea83e00640 1 --2- 172.21.15.47:0/823981901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 0x56425af60040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.592+0000 7fea83e00640 1 -- 172.21.15.47:0/823981901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56425af60630 con 0x56425af50bd0 2024-03-21T11:37:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.592+0000 7fea8b400640 1 --2- 172.21.15.47:0/823981901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 0x56425af60040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:37:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.592+0000 7fea83e00640 1 --2- 172.21.15.47:0/823981901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af5f030 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fea7402c180 tx=0x7fea74009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ed7451b5297d035 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.592+0000 7fea78c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:37:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.592+0000 7fea78c00640 1 -- 172.21.15.47:0/823981901 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7404c040 con 0x56425af50bd0 2024-03-21T11:37:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.592+0000 7fea94bd3740 1 -- 172.21.15.47:0/823981901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea7402cd10 con 0x56425af50bd0 2024-03-21T11:37:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.592+0000 7fea78c00640 1 -- 172.21.15.47:0/823981901 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fea74004040 con 0x56425af50bd0 2024-03-21T11:37:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.593+0000 7fea78c00640 1 -- 172.21.15.47:0/823981901 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea74004240 con 0x56425af50bd0 2024-03-21T11:37:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.593+0000 7fea94bd3740 10 client.30323 fetch_fsmap learned FSMap version 34 2024-03-21T11:37:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.593+0000 7fea94bd3740 1 -- 172.21.15.47:0/823981901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56425af619c0 con 0x56425af50bd0 2024-03-21T11:37:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.593+0000 7fea78c00640 1 -- 172.21.15.47:0/823981901 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea7402cd10 con 0x56425af50bd0 2024-03-21T11:37:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.593+0000 7fea78c00640 1 -- 172.21.15.47:0/823981901 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fea74034680 con 0x56425af50bd0 2024-03-21T11:37:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.593+0000 7fea94bd3740 10 client.30323 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:37:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.593+0000 7fea94bd3740 10 client.30323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.593+0000 7fea94bd3740 20 client.30323 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.593+0000 7fea94bd3740 1 --2- 172.21.15.47:0/823981901 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56425afca590 0x56425afcc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.593+0000 7fea94bd3740 4 client.30323 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:37:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.593+0000 7fea94bd3740 1 -- 172.21.15.47:0/823981901 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56425aebd0c0 con 0x56425afca590 2024-03-21T11:37:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.594+0000 7fea8b400640 1 --2- 172.21.15.47:0/823981901 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56425afca590 0x56425afcc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.594+0000 7fea8b400640 1 --2- 172.21.15.47:0/823981901 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56425afca590 0x56425afcc9b0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cddcc2db79aa9b0 server_cookie=67e4ae960d72ec2c in_seq=0 out_seq=0 2024-03-21T11:37:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:42.594+0000 7fea78c00640 10 client.30323 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:42.920 INFO:tasks.workunit.client.0.smithi047.stdout: 10 13402 34.88 MB/sec warmup 26 sec latency 212.807 ms 2024-03-21T11:37:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea78c00640 1 -- 172.21.15.47:0/823981901 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56425aebd0c0 con 0x56425afca590 2024-03-21T11:37:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea78c00640 10 client.30323 handle_command_reply: tid=0 2024-03-21T11:37:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea94bd3740 1 client.30323 shutdown 2024-03-21T11:37:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea94bd3740 1 -- 172.21.15.47:0/823981901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 msgr2=0x56425af5f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea94bd3740 1 --2- 172.21.15.47:0/823981901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af5f030 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fea7402c180 tx=0x7fea74009870 comp rx=0 tx=0).stop 2024-03-21T11:37:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea94bd3740 1 -- 172.21.15.47:0/823981901 shutdown_connections 2024-03-21T11:37:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea94bd3740 1 --2- 172.21.15.47:0/823981901 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56425afca590 0x56425afcc9b0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea94bd3740 1 --2- 172.21.15.47:0/823981901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56425af55d40 0x56425af60040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea94bd3740 1 --2- 172.21.15.47:0/823981901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56425af50bd0 0x56425af5f030 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea78c00640 0 client.30323 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea94bd3740 1 --2- 172.21.15.47:0/823981901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56425af501e0 0x56425af5eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea94bd3740 1 -- 172.21.15.47:0/823981901 >> 172.21.15.47:0/823981901 conn(0x56425af4b750 msgr2=0x56425af51eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea94bd3740 1 -- 172.21.15.47:0/823981901 shutdown_connections 2024-03-21T11:37:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea94bd3740 1 -- 172.21.15.47:0/823981901 wait complete. 2024-03-21T11:37:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.035+0000 7fea94bd3740 20 client.30323 trim_cache size 0 max 16384 2024-03-21T11:37:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:37:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:37:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:37:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:37:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.036+0000 7fea91e00640 1 -- 172.21.15.47:0/2244154331 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fea700a3040 msgr2=0x7fea700a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.036+0000 7fea91e00640 1 --2- 172.21.15.47:0/2244154331 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fea700a3040 0x7fea700a5500 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0cd1b0 tx=0x7fea7c00ef90 comp rx=0 tx=0).stop 2024-03-21T11:37:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.036+0000 7fea91e00640 1 -- 172.21.15.47:0/2244154331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c0740c0 msgr2=0x7fea8c199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.036+0000 7fea91e00640 1 --2- 172.21.15.47:0/2244154331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c0740c0 0x7fea8c199b00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00ea60 tx=0x7fea7c00ef20 comp rx=0 tx=0).stop 2024-03-21T11:37:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.036+0000 7fea91e00640 1 -- 172.21.15.47:0/2244154331 shutdown_connections 2024-03-21T11:37:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.036+0000 7fea91e00640 1 --2- 172.21.15.47:0/2244154331 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fea700a3040 0x7fea700a5500 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.036+0000 7fea91e00640 1 --2- 172.21.15.47:0/2244154331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea8c1095b0 0x7fea8c1a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.036+0000 7fea91e00640 1 --2- 172.21.15.47:0/2244154331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8c0756a0 0x7fea8c19a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.036+0000 7fea91e00640 1 --2- 172.21.15.47:0/2244154331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea8c0740c0 0x7fea8c199b00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.036+0000 7fea91e00640 1 -- 172.21.15.47:0/2244154331 >> 172.21.15.47:0/2244154331 conn(0x7fea8c0fcc70 msgr2=0x7fea8c105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.037+0000 7fea91e00640 1 -- 172.21.15.47:0/2244154331 shutdown_connections 2024-03-21T11:37:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:43.037+0000 7fea91e00640 1 -- 172.21.15.47:0/2244154331 wait complete. 2024-03-21T11:37:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:42 smithi102 ceph-mon[36748]: pgmap v831: 129 pgs: 129 active+clean; 270 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 25 MiB/s wr, 777 op/s 2024-03-21T11:37:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:42 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:37:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:42 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:37:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:42 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:37:43.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:42 smithi047 ceph-mon[31351]: pgmap v831: 129 pgs: 129 active+clean; 270 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 25 MiB/s wr, 777 op/s 2024-03-21T11:37:43.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:42 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:37:43.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:42 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:37:43.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:42 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:37:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:42 smithi100 ceph-mon[36245]: pgmap v831: 129 pgs: 129 active+clean; 270 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 25 MiB/s wr, 777 op/s 2024-03-21T11:37:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:42 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:37:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:42 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:37:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:42 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:37:43.920 INFO:tasks.workunit.client.0.smithi047.stdout: 10 13843 34.21 MB/sec warmup 27 sec latency 281.294 ms 2024-03-21T11:37:44.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:43 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:37:44.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:43 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:37:44.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:43 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:37:44.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:43 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:37:44.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:43 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:37:44.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:43 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:37:44.921 INFO:tasks.workunit.client.0.smithi047.stdout: 10 14307 33.70 MB/sec warmup 28 sec latency 70.166 ms 2024-03-21T11:37:45.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:44 smithi102 ceph-mon[36748]: pgmap v832: 129 pgs: 129 active+clean; 244 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 909 op/s 2024-03-21T11:37:45.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:44 smithi047 ceph-mon[31351]: pgmap v832: 129 pgs: 129 active+clean; 244 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 909 op/s 2024-03-21T11:37:45.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:44 smithi100 ceph-mon[36245]: pgmap v832: 129 pgs: 129 active+clean; 244 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 25 MiB/s wr, 909 op/s 2024-03-21T11:37:45.921 INFO:tasks.workunit.client.0.smithi047.stdout: 10 14883 33.60 MB/sec warmup 29 sec latency 64.337 ms 2024-03-21T11:37:46.921 INFO:tasks.workunit.client.0.smithi047.stdout: 10 15595 33.71 MB/sec warmup 30 sec latency 49.276 ms 2024-03-21T11:37:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:46 smithi102 ceph-mon[36748]: pgmap v833: 129 pgs: 129 active+clean; 237 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 MiB/s wr, 679 op/s 2024-03-21T11:37:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:46 smithi047 ceph-mon[31351]: pgmap v833: 129 pgs: 129 active+clean; 237 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 MiB/s wr, 679 op/s 2024-03-21T11:37:47.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:46 smithi100 ceph-mon[36245]: pgmap v833: 129 pgs: 129 active+clean; 237 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 MiB/s wr, 679 op/s 2024-03-21T11:37:47.921 INFO:tasks.workunit.client.0.smithi047.stdout: 10 16235 33.72 MB/sec warmup 31 sec latency 93.535 ms 2024-03-21T11:37:48.921 INFO:tasks.workunit.client.0.smithi047.stdout: 10 16920 33.95 MB/sec warmup 32 sec latency 47.998 ms 2024-03-21T11:37:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:48 smithi047 ceph-mon[31351]: pgmap v834: 129 pgs: 129 active+clean; 244 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 704 op/s 2024-03-21T11:37:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:48 smithi100 ceph-mon[36245]: pgmap v834: 129 pgs: 129 active+clean; 244 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 704 op/s 2024-03-21T11:37:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:48 smithi102 ceph-mon[36748]: pgmap v834: 129 pgs: 129 active+clean; 244 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 18 MiB/s wr, 704 op/s 2024-03-21T11:37:49.921 INFO:tasks.workunit.client.0.smithi047.stdout: 10 17570 34.25 MB/sec warmup 33 sec latency 54.302 ms 2024-03-21T11:37:50.921 INFO:tasks.workunit.client.0.smithi047.stdout: 10 18015 33.82 MB/sec warmup 34 sec latency 52.625 ms 2024-03-21T11:37:51.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:51 smithi047 ceph-mon[31351]: pgmap v835: 129 pgs: 129 active+clean; 249 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 727 op/s 2024-03-21T11:37:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:51 smithi100 ceph-mon[36245]: pgmap v835: 129 pgs: 129 active+clean; 249 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 727 op/s 2024-03-21T11:37:51.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:51 smithi102 ceph-mon[36748]: pgmap v835: 129 pgs: 129 active+clean; 249 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 727 op/s 2024-03-21T11:37:51.921 INFO:tasks.workunit.client.0.smithi047.stdout: 10 18425 33.44 MB/sec warmup 35 sec latency 77.635 ms 2024-03-21T11:37:52.921 INFO:tasks.workunit.client.0.smithi047.stdout: 10 18979 33.19 MB/sec warmup 36 sec latency 99.154 ms 2024-03-21T11:37:53.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:53 smithi047 ceph-mon[31351]: pgmap v836: 129 pgs: 129 active+clean; 251 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-03-21T11:37:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:53 smithi100 ceph-mon[36245]: pgmap v836: 129 pgs: 129 active+clean; 251 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-03-21T11:37:53.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:53 smithi102 ceph-mon[36748]: pgmap v836: 129 pgs: 129 active+clean; 251 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.1 MiB/s wr, 601 op/s 2024-03-21T11:37:53.921 INFO:tasks.workunit.client.0.smithi047.stdout: 10 19528 33.06 MB/sec warmup 37 sec latency 105.029 ms 2024-03-21T11:37:54.921 INFO:tasks.workunit.client.0.smithi047.stdout: 10 20260 33.24 MB/sec warmup 38 sec latency 38.031 ms 2024-03-21T11:37:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:55 smithi047 ceph-mon[31351]: pgmap v837: 129 pgs: 129 active+clean; 261 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.8 MiB/s wr, 673 op/s 2024-03-21T11:37:55.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:55 smithi100 ceph-mon[36245]: pgmap v837: 129 pgs: 129 active+clean; 261 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.8 MiB/s wr, 673 op/s 2024-03-21T11:37:55.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:55 smithi102 ceph-mon[36748]: pgmap v837: 129 pgs: 129 active+clean; 261 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.8 MiB/s wr, 673 op/s 2024-03-21T11:37:55.921 INFO:tasks.workunit.client.0.smithi047.stdout: 10 20944 33.34 MB/sec warmup 39 sec latency 115.757 ms 2024-03-21T11:37:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.218+0000 7ff3d0000640 1 -- 172.21.15.47:0/3667471528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3c81021b0 msgr2=0x7ff3c81045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.218+0000 7ff3d0000640 1 --2- 172.21.15.47:0/3667471528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3c81021b0 0x7ff3c81045a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009ca0 tx=0x7ff3bc02c0f0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.219+0000 7ff3d0000640 1 -- 172.21.15.47:0/3667471528 shutdown_connections 2024-03-21T11:37:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.219+0000 7ff3d0000640 1 --2- 172.21.15.47:0/3667471528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c8107410 0x7ff3c8109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.219+0000 7ff3d0000640 1 --2- 172.21.15.47:0/3667471528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3c8104ae0 0x7ff3c8106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.219+0000 7ff3d0000640 1 --2- 172.21.15.47:0/3667471528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3c81021b0 0x7ff3c81045a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.219+0000 7ff3d0000640 1 -- 172.21.15.47:0/3667471528 >> 172.21.15.47:0/3667471528 conn(0x7ff3c80fbba0 msgr2=0x7ff3c80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.219+0000 7ff3d0000640 1 -- 172.21.15.47:0/3667471528 shutdown_connections 2024-03-21T11:37:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.219+0000 7ff3d0000640 1 -- 172.21.15.47:0/3667471528 wait complete. 2024-03-21T11:37:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.220+0000 7ff3d0000640 1 Processor -- start 2024-03-21T11:37:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.220+0000 7ff3d0000640 1 -- start start 2024-03-21T11:37:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.221+0000 7ff3d0000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3c81021b0 0x7ff3c8197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.221+0000 7ff3d0000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c8104ae0 0x7ff3c8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.221+0000 7ff3d0000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3c8107410 0x7ff3c819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.221+0000 7ff3d0000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c810d370 con 0x7ff3c8107410 2024-03-21T11:37:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.221+0000 7ff3d0000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c810d1f0 con 0x7ff3c81021b0 2024-03-21T11:37:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.221+0000 7ff3d0000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c810d4f0 con 0x7ff3c8104ae0 2024-03-21T11:37:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.221+0000 7ff3ce000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3c8107410 0x7ff3c819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.221+0000 7ff3cd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3c81021b0 0x7ff3c8197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.221+0000 7ff3ce000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3c8107410 0x7ff3c819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49062/0 (socket says 172.21.15.47:49062) 2024-03-21T11:37:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.221+0000 7ff3ce000640 1 -- 172.21.15.47:0/1309461837 learned_addr learned my addr 172.21.15.47:0/1309461837 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.221+0000 7ff3cd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3c81021b0 0x7ff3c8197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36776/0 (socket says 172.21.15.47:36776) 2024-03-21T11:37:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.222+0000 7ff3ce000640 1 -- 172.21.15.47:0/1309461837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c8104ae0 msgr2=0x7ff3c8197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.222+0000 7ff3ccc00640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c8104ae0 0x7ff3c8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.222+0000 7ff3ce000640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c8104ae0 0x7ff3c8197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.222+0000 7ff3ce000640 1 -- 172.21.15.47:0/1309461837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3c81021b0 msgr2=0x7ff3c8197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.222+0000 7ff3ce000640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3c81021b0 0x7ff3c8197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.222+0000 7ff3ce000640 1 -- 172.21.15.47:0/1309461837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c819f550 con 0x7ff3c8107410 2024-03-21T11:37:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.222+0000 7ff3cd600640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3c81021b0 0x7ff3c8197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.222+0000 7ff3ccc00640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c8104ae0 0x7ff3c8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:37:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.222+0000 7ff3ce000640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3c8107410 0x7ff3c819eee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000e970 tx=0x7ff3c000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.222+0000 7ff3c6000640 1 -- 172.21.15.47:0/1309461837 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c000ccc0 con 0x7ff3c8107410 2024-03-21T11:37:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.223+0000 7ff3c6000640 1 -- 172.21.15.47:0/1309461837 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3c000ce60 con 0x7ff3c8107410 2024-03-21T11:37:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.223+0000 7ff3d0000640 1 -- 172.21.15.47:0/1309461837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c8100000 con 0x7ff3c8107410 2024-03-21T11:37:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.223+0000 7ff3c6000640 1 -- 172.21.15.47:0/1309461837 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0010490 con 0x7ff3c8107410 2024-03-21T11:37:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.223+0000 7ff3d0000640 1 -- 172.21.15.47:0/1309461837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c81005a0 con 0x7ff3c8107410 2024-03-21T11:37:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.227+0000 7ff3d2dd1740 1 Processor -- start 2024-03-21T11:37:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.227+0000 7ff3c6000640 1 -- 172.21.15.47:0/1309461837 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff3c0005020 con 0x7ff3c8107410 2024-03-21T11:37:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.228+0000 7ff3d2dd1740 1 -- start start 2024-03-21T11:37:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.228+0000 7ff3d2dd1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e39f0 0x562b410e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.228+0000 7ff3d2dd1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e43e0 0x562b410e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.228+0000 7ff3d2dd1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e9550 0x562b410eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.228+0000 7ff3d2dd1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4109e3b0 con 0x562b410e9550 2024-03-21T11:37:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.228+0000 7ff3d2dd1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b41040830 con 0x562b410e39f0 2024-03-21T11:37:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.228+0000 7ff3d2dd1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b40fe95d0 con 0x562b410e43e0 2024-03-21T11:37:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.229+0000 7ff3ccc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e39f0 0x562b410e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.229+0000 7ff3ccc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e39f0 0x562b410e3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36778/0 (socket says 172.21.15.47:36778) 2024-03-21T11:37:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.229+0000 7ff3ccc00640 1 -- 172.21.15.47:0/172739798 learned_addr learned my addr 172.21.15.47:0/172739798 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.229+0000 7ff3c6000640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff3b00a3040 0x7ff3b00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.229+0000 7ff3cd600640 1 --2- 172.21.15.47:0/172739798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e43e0 0x562b410e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.229+0000 7ff3ccc00640 1 --2- 172.21.15.47:0/172739798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e9550 0x562b410eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.229+0000 7ff3c6000640 1 -- 172.21.15.47:0/1309461837 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff3c00cfbe0 con 0x7ff3c8107410 2024-03-21T11:37:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.229+0000 7ff3ce000640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff3b00a3040 0x7ff3b00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.229+0000 7ff3ccc00640 1 -- 172.21.15.47:0/172739798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e43e0 msgr2=0x562b410e8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.229+0000 7ff3ccc00640 1 --2- 172.21.15.47:0/172739798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e43e0 0x562b410e8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.229+0000 7ff3ccc00640 1 -- 172.21.15.47:0/172739798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e9550 msgr2=0x562b410eb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.229+0000 7ff3ccc00640 1 --2- 172.21.15.47:0/172739798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e9550 0x562b410eb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.229+0000 7ff3ccc00640 1 -- 172.21.15.47:0/172739798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b410012c0 con 0x562b410e39f0 2024-03-21T11:37:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.229+0000 7ff3cd600640 1 --2- 172.21.15.47:0/172739798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e43e0 0x562b410e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.230+0000 7ff3ccc00640 1 --2- 172.21.15.47:0/172739798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e39f0 0x562b410e3e10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4014900 tx=0x7ff3b4009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cae817a66b346a40 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.230+0000 7ff3ba600640 1 -- 172.21.15.47:0/172739798 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b40142d0 con 0x562b410e39f0 2024-03-21T11:37:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.230+0000 7ff3ba600640 1 -- 172.21.15.47:0/172739798 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3b400f760 con 0x562b410e39f0 2024-03-21T11:37:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.230+0000 7ff3ba600640 1 -- 172.21.15.47:0/172739798 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4003850 con 0x562b410e39f0 2024-03-21T11:37:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.231+0000 7ff3ce000640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff3b00a3040 0x7ff3b00a5500 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00097d0 tx=0x7ff3c0009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.231+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/172739798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e39f0 msgr2=0x562b410e3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.231+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/172739798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e39f0 0x562b410e3e10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4014900 tx=0x7ff3b4009630 comp rx=0 tx=0).stop 2024-03-21T11:37:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.232+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/172739798 shutdown_connections 2024-03-21T11:37:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.232+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/172739798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e9550 0x562b410eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.232+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/172739798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e43e0 0x562b410e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.232+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/172739798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e39f0 0x562b410e3e10 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.232+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/172739798 >> 172.21.15.47:0/172739798 conn(0x562b410def60 msgr2=0x562b410df360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.232+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/172739798 shutdown_connections 2024-03-21T11:37:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.232+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/172739798 wait complete. 2024-03-21T11:37:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.233+0000 7ff3d2dd1740 1 -- start start 2024-03-21T11:37:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.233+0000 7ff3d2dd1740 1 Processor -- start 2024-03-21T11:37:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.234+0000 7ff3d2dd1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410edb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.234+0000 7ff3d2dd1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 0x562b410ee080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.234+0000 7ff3d2dd1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 0x562b410f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.234+0000 7ff3d2dd1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b40ffca20 con 0x562b410e43e0 2024-03-21T11:37:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.234+0000 7ff3ccc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410edb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.235+0000 7ff3ccc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410edb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41946/0 (socket says 172.21.15.47:41946) 2024-03-21T11:37:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.235+0000 7ff3ccc00640 1 -- 172.21.15.47:0/3811767871 learned_addr learned my addr 172.21.15.47:0/3811767871 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.235+0000 7ff3cd600640 1 --2- 172.21.15.47:0/3811767871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 0x562b410ee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.235+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3811767871 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4103c8d0 con 0x562b410e9550 2024-03-21T11:37:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.235+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3811767871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b410ec4a0 con 0x562b410e39f0 2024-03-21T11:37:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.235+0000 7ff3d2dd1740 10 client.0 _ll_register_callbacks cb 0x562b410391c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.235+0000 7ff3ccc00640 1 --2- 172.21.15.47:0/3811767871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 0x562b410f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.236+0000 7ff3ccc00640 1 -- 172.21.15.47:0/3811767871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 msgr2=0x562b410f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.236+0000 7ff3ccc00640 1 --2- 172.21.15.47:0/3811767871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 0x562b410f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.236+0000 7ff3ccc00640 1 -- 172.21.15.47:0/3811767871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 msgr2=0x562b410ee080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.236+0000 7ff3ccc00640 1 --2- 172.21.15.47:0/3811767871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 0x562b410ee080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.236+0000 7ff3ccc00640 1 -- 172.21.15.47:0/3811767871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b40d55210 con 0x562b410e39f0 2024-03-21T11:37:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.236+0000 7ff3ccc00640 1 --2- 172.21.15.47:0/3811767871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410edb40 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4006010 tx=0x7ff3b400ff80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d803e688703e065 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.236+0000 7ff3aea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:37:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.237+0000 7ff3aea00640 1 -- 172.21.15.47:0/3811767871 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4012450 con 0x562b410e39f0 2024-03-21T11:37:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.237+0000 7ff3aea00640 1 -- 172.21.15.47:0/3811767871 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff3b40125f0 con 0x562b410e39f0 2024-03-21T11:37:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.237+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3811767871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3b400f760 con 0x562b410e39f0 2024-03-21T11:37:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.237+0000 7ff3aea00640 1 -- 172.21.15.47:0/3811767871 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3b40127f0 con 0x562b410e39f0 2024-03-21T11:37:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.237+0000 7ff3d2dd1740 10 client.30346 fetch_fsmap learned FSMap version 34 2024-03-21T11:37:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.237+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3811767871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562b40e21620 con 0x562b410e39f0 2024-03-21T11:37:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.238+0000 7ff3aea00640 1 -- 172.21.15.47:0/3811767871 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3b400f760 con 0x562b410e39f0 2024-03-21T11:37:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.238+0000 7ff3aea00640 1 -- 172.21.15.47:0/3811767871 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7ff3b4022940 con 0x562b410e39f0 2024-03-21T11:37:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.238+0000 7ff3d2dd1740 10 client.30346 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:37:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.238+0000 7ff3d2dd1740 10 client.30346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.238+0000 7ff3d2dd1740 20 client.30346 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.238+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/3811767871 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562b410f96a0 0x562b410fba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.238+0000 7ff3d2dd1740 4 client.30346 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:37:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.238+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3811767871 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b4104dd70 con 0x562b410f96a0 2024-03-21T11:37:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.238+0000 7ff3cd600640 1 --2- 172.21.15.47:0/3811767871 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562b410f96a0 0x562b410fba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.239+0000 7ff3cd600640 1 --2- 172.21.15.47:0/3811767871 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562b410f96a0 0x562b410fba90 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f564c6fe80f98eea server_cookie=c3aa8d6e55af84fd in_seq=0 out_seq=0 2024-03-21T11:37:56.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.239+0000 7ff3aea00640 10 client.30346 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:56.242 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.240+0000 7ff3aea00640 1 -- 172.21.15.47:0/3811767871 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562b4104dd70 con 0x562b410f96a0 2024-03-21T11:37:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.240+0000 7ff3aea00640 10 client.30346 handle_command_reply: tid=0 2024-03-21T11:37:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.241+0000 7ff3d2dd1740 1 client.30346 shutdown 2024-03-21T11:37:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.241+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3811767871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 msgr2=0x562b410edb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.241+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/3811767871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410edb40 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4006010 tx=0x7ff3b400ff80 comp rx=0 tx=0).stop 2024-03-21T11:37:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.241+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3811767871 shutdown_connections 2024-03-21T11:37:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.241+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/3811767871 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562b410f96a0 0x562b410fba90 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.241+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/3811767871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 0x562b410f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.241+0000 7ff3aea00640 0 client.30346 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.241+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/3811767871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 0x562b410ee080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.241+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/3811767871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410edb40 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.241+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3811767871 >> 172.21.15.47:0/3811767871 conn(0x562b410def60 msgr2=0x562b410d5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.242+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3811767871 shutdown_connections 2024-03-21T11:37:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.242+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3811767871 wait complete. 2024-03-21T11:37:56.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.242+0000 7ff3d2dd1740 20 client.30346 trim_cache size 0 max 16384 2024-03-21T11:37:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.268+0000 7ff3d2dd1740 1 Processor -- start 2024-03-21T11:37:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.268+0000 7ff3d2dd1740 1 -- start start 2024-03-21T11:37:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.269+0000 7ff3d2dd1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410e5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.269+0000 7ff3d2dd1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 0x562b410e5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.269+0000 7ff3d2dd1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 0x562b410f5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.269+0000 7ff3d2dd1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c81a7e30 con 0x562b410e43e0 2024-03-21T11:37:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.269+0000 7ff3d2dd1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c81a9930 con 0x562b410e9550 2024-03-21T11:37:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.269+0000 7ff3d2dd1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c81a8490 con 0x562b410e39f0 2024-03-21T11:37:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.269+0000 7ff3ccc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410e5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.269+0000 7ff3ccc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410e5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41960/0 (socket says 172.21.15.47:41960) 2024-03-21T11:37:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.269+0000 7ff3ccc00640 1 -- 172.21.15.47:0/920618102 learned_addr learned my addr 172.21.15.47:0/920618102 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.269+0000 7ff3cd600640 1 --2- 172.21.15.47:0/920618102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 0x562b410e5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.270+0000 7ff3ccc00640 1 --2- 172.21.15.47:0/920618102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 0x562b410f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.270+0000 7ff3cd600640 1 -- 172.21.15.47:0/920618102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 msgr2=0x562b410e5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.270+0000 7ff3cd600640 1 --2- 172.21.15.47:0/920618102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410e5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.270+0000 7ff3cd600640 1 -- 172.21.15.47:0/920618102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 msgr2=0x562b410f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.270+0000 7ff3cd600640 1 --2- 172.21.15.47:0/920618102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 0x562b410f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.270+0000 7ff3cd600640 1 -- 172.21.15.47:0/920618102 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b410e3630 con 0x562b410e43e0 2024-03-21T11:37:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.270+0000 7ff3ccc00640 1 --2- 172.21.15.47:0/920618102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 0x562b410f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:37:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.270+0000 7ff3cd600640 1 --2- 172.21.15.47:0/920618102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 0x562b410e5c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009bf0 tx=0x7ff3bc009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580812cf13f6bdbb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.271+0000 7ff3b9c00640 1 -- 172.21.15.47:0/920618102 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc039040 con 0x562b410e43e0 2024-03-21T11:37:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.271+0000 7ff3b9c00640 1 -- 172.21.15.47:0/920618102 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3bc032c90 con 0x562b410e43e0 2024-03-21T11:37:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.271+0000 7ff3b9c00640 1 -- 172.21.15.47:0/920618102 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc045ae0 con 0x562b410e43e0 2024-03-21T11:37:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.272+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/920618102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 msgr2=0x562b410e5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:56.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.272+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/920618102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 0x562b410e5c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009bf0 tx=0x7ff3bc009990 comp rx=0 tx=0).stop 2024-03-21T11:37:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.272+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/920618102 shutdown_connections 2024-03-21T11:37:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.272+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/920618102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 0x562b410f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.272+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/920618102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 0x562b410e5c80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.272+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/920618102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410e5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.272+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/920618102 >> 172.21.15.47:0/920618102 conn(0x562b410def60 msgr2=0x562b410eb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.273+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/920618102 shutdown_connections 2024-03-21T11:37:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.273+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/920618102 wait complete. 2024-03-21T11:37:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.273+0000 7ff3d2dd1740 1 -- start start 2024-03-21T11:37:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.274+0000 7ff3d2dd1740 1 Processor -- start 2024-03-21T11:37:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.274+0000 7ff3d2dd1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.274+0000 7ff3d2dd1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 0x562b410f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.275+0000 7ff3d2dd1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 0x562b410f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.275 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.275+0000 7ff3d2dd1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c81a9210 con 0x562b410e43e0 2024-03-21T11:37:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.275+0000 7ff3d2dd1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c81a8bb0 con 0x562b410e9550 2024-03-21T11:37:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.275+0000 7ff3d2dd1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c81a5ff0 con 0x562b410e39f0 2024-03-21T11:37:56.276 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.275+0000 7ff3d2dd1740 10 client.0 _ll_register_callbacks cb 0x7ff3b4009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:56.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.275+0000 7ff3ccc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.275+0000 7ff3ccc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410f27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41972/0 (socket says 172.21.15.47:41972) 2024-03-21T11:37:56.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.275+0000 7ff3ccc00640 1 -- 172.21.15.47:0/3688456057 learned_addr learned my addr 172.21.15.47:0/3688456057 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:56.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.275+0000 7ff3cd600640 1 --2- 172.21.15.47:0/3688456057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 0x562b410f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.276+0000 7ff3cd600640 1 -- 172.21.15.47:0/3688456057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 msgr2=0x562b410f27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.276+0000 7ff3cd600640 1 --2- 172.21.15.47:0/3688456057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410f27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.276+0000 7ff3cd600640 1 -- 172.21.15.47:0/3688456057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 msgr2=0x562b410f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.276+0000 7ff3cd600640 1 --2- 172.21.15.47:0/3688456057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 0x562b410f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.276+0000 7ff3cd600640 1 -- 172.21.15.47:0/3688456057 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b410ec5e0 con 0x562b410e43e0 2024-03-21T11:37:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.276+0000 7ff3cd600640 1 --2- 172.21.15.47:0/3688456057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 0x562b410f2d10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc000f80 tx=0x7ff3bc003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681283d78ed629a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.276+0000 7ff3aea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:37:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.277+0000 7ff3aea00640 1 -- 172.21.15.47:0/3688456057 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc039040 con 0x562b410e43e0 2024-03-21T11:37:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.277+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3688456057 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3bc032c90 con 0x562b410e43e0 2024-03-21T11:37:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.277+0000 7ff3aea00640 1 -- 172.21.15.47:0/3688456057 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff3bc003970 con 0x562b410e43e0 2024-03-21T11:37:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.279+0000 7ff3aea00640 1 -- 172.21.15.47:0/3688456057 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3bc003b70 con 0x562b410e43e0 2024-03-21T11:37:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.279+0000 7ff3d2dd1740 10 client.19902 fetch_fsmap learned FSMap version 34 2024-03-21T11:37:56.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.279+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3688456057 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562b410f53c0 con 0x562b410e43e0 2024-03-21T11:37:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.280+0000 7ff3aea00640 1 -- 172.21.15.47:0/3688456057 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3bc032c90 con 0x562b410e43e0 2024-03-21T11:37:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.282+0000 7ff3aea00640 1 -- 172.21.15.47:0/3688456057 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7ff3bc034930 con 0x562b410e43e0 2024-03-21T11:37:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.282+0000 7ff3d2dd1740 10 client.19902 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:37:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.282+0000 7ff3d2dd1740 10 client.19902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.282+0000 7ff3d2dd1740 20 client.19902 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.282+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/3688456057 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562b410f96a0 0x562b411576c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.282+0000 7ff3d2dd1740 4 client.19902 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:37:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.282+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3688456057 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562b410480c0 con 0x562b410f96a0 2024-03-21T11:37:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.283+0000 7ff3ccc00640 1 --2- 172.21.15.47:0/3688456057 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562b410f96a0 0x562b411576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.284+0000 7ff3ccc00640 1 --2- 172.21.15.47:0/3688456057 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562b410f96a0 0x562b411576c0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25c4b4ca13604a6e server_cookie=83a8da9cf6afdafa in_seq=0 out_seq=0 2024-03-21T11:37:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:56.284+0000 7ff3aea00640 10 client.19902 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:56.922 INFO:tasks.workunit.client.0.smithi047.stdout: 10 21261 32.95 MB/sec warmup 40 sec latency 420.387 ms 2024-03-21T11:37:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.021+0000 7ff3aea00640 1 -- 172.21.15.47:0/3688456057 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562b410480c0 con 0x562b410f96a0 2024-03-21T11:37:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.021+0000 7ff3aea00640 10 client.19902 handle_command_reply: tid=0 2024-03-21T11:37:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.021+0000 7ff3d2dd1740 1 client.19902 shutdown 2024-03-21T11:37:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.021+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3688456057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 msgr2=0x562b410f2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.021+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/3688456057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 0x562b410f2d10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc000f80 tx=0x7ff3bc003f90 comp rx=0 tx=0).stop 2024-03-21T11:37:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.023+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3688456057 shutdown_connections 2024-03-21T11:37:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.023+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/3688456057 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562b410f96a0 0x562b411576c0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.023+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/3688456057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b410e9550 0x562b410f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.023+0000 7ff3aea00640 0 client.19902 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.023+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/3688456057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b410e43e0 0x562b410f2d10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.023+0000 7ff3d2dd1740 1 --2- 172.21.15.47:0/3688456057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b410e39f0 0x562b410f27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.023+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3688456057 >> 172.21.15.47:0/3688456057 conn(0x562b410def60 msgr2=0x562b410e5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.023+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3688456057 shutdown_connections 2024-03-21T11:37:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.023+0000 7ff3d2dd1740 1 -- 172.21.15.47:0/3688456057 wait complete. 2024-03-21T11:37:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.023+0000 7ff3d2dd1740 20 client.19902 trim_cache size 0 max 16384 2024-03-21T11:37:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:37:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:37:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:37:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:37:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.024+0000 7ff3d0000640 1 -- 172.21.15.47:0/1309461837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff3b00a3040 msgr2=0x7ff3b00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.024+0000 7ff3d0000640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff3b00a3040 0x7ff3b00a5500 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00097d0 tx=0x7ff3c0009720 comp rx=0 tx=0).stop 2024-03-21T11:37:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.024+0000 7ff3d0000640 1 -- 172.21.15.47:0/1309461837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3c8107410 msgr2=0x7ff3c819eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.024+0000 7ff3d0000640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3c8107410 0x7ff3c819eee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000e970 tx=0x7ff3c000ed70 comp rx=0 tx=0).stop 2024-03-21T11:37:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.024+0000 7ff3d0000640 1 -- 172.21.15.47:0/1309461837 shutdown_connections 2024-03-21T11:37:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.024+0000 7ff3d0000640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff3b00a3040 0x7ff3b00a5500 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.024+0000 7ff3d0000640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3c8107410 0x7ff3c819eee0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.024+0000 7ff3d0000640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c8104ae0 0x7ff3c8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.024+0000 7ff3d0000640 1 --2- 172.21.15.47:0/1309461837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3c81021b0 0x7ff3c8197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.024+0000 7ff3d0000640 1 -- 172.21.15.47:0/1309461837 >> 172.21.15.47:0/1309461837 conn(0x7ff3c80fbba0 msgr2=0x7ff3c8102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.024+0000 7ff3d0000640 1 -- 172.21.15.47:0/1309461837 shutdown_connections 2024-03-21T11:37:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:37:57.024+0000 7ff3d0000640 1 -- 172.21.15.47:0/1309461837 wait complete. 2024-03-21T11:37:57.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:57 smithi047 ceph-mon[31351]: pgmap v838: 129 pgs: 129 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.1 MiB/s wr, 492 op/s 2024-03-21T11:37:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:57 smithi100 ceph-mon[36245]: pgmap v838: 129 pgs: 129 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.1 MiB/s wr, 492 op/s 2024-03-21T11:37:57.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:57 smithi102 ceph-mon[36748]: pgmap v838: 129 pgs: 129 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.1 MiB/s wr, 492 op/s 2024-03-21T11:37:57.922 INFO:tasks.workunit.client.0.smithi047.stdout: 10 21809 33.04 MB/sec warmup 41 sec latency 93.882 ms 2024-03-21T11:37:58.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:37:58.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:37:58.922 INFO:tasks.workunit.client.0.smithi047.stdout: 10 22325 32.97 MB/sec warmup 42 sec latency 116.136 ms 2024-03-21T11:37:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.068+0000 7f1a58000640 1 -- 172.21.15.47:0/853625443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a501038e0 msgr2=0x7f1a50103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.068+0000 7f1a58000640 1 --2- 172.21.15.47:0/853625443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a501038e0 0x7f1a50103d00 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009ca0 tx=0x7f1a4402c0f0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.068+0000 7f1a58000640 1 -- 172.21.15.47:0/853625443 shutdown_connections 2024-03-21T11:37:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.068+0000 7f1a58000640 1 --2- 172.21.15.47:0/853625443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a50105490 0x7f1a5010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.068+0000 7f1a58000640 1 --2- 172.21.15.47:0/853625443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a50104ad0 0x7f1a50104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.068+0000 7f1a58000640 1 --2- 172.21.15.47:0/853625443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a501038e0 0x7f1a50103d00 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.068+0000 7f1a58000640 1 -- 172.21.15.47:0/853625443 >> 172.21.15.47:0/853625443 conn(0x7f1a500fee50 msgr2=0x7f1a50101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.068+0000 7f1a58000640 1 -- 172.21.15.47:0/853625443 shutdown_connections 2024-03-21T11:37:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.069+0000 7f1a58000640 1 -- 172.21.15.47:0/853625443 wait complete. 2024-03-21T11:37:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.070+0000 7f1a58000640 1 Processor -- start 2024-03-21T11:37:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.070+0000 7f1a58000640 1 -- start start 2024-03-21T11:37:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.070+0000 7f1a58000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a501038e0 0x7f1a5019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a58000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a50104ad0 0x7f1a5019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a58000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a50105490 0x7f1a501a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a58000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a50076670 con 0x7f1a50105490 2024-03-21T11:37:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a58000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a500764f0 con 0x7f1a501038e0 2024-03-21T11:37:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a58000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a500767f0 con 0x7f1a50104ad0 2024-03-21T11:37:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a55600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a501038e0 0x7f1a5019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a55600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a501038e0 0x7f1a5019e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48734/0 (socket says 172.21.15.47:48734) 2024-03-21T11:37:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a54c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a50104ad0 0x7f1a5019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a55600640 1 -- 172.21.15.47:0/721210371 learned_addr learned my addr 172.21.15.47:0/721210371 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a56000640 1 --2- 172.21.15.47:0/721210371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a50105490 0x7f1a501a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a54c00640 1 -- 172.21.15.47:0/721210371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a501038e0 msgr2=0x7f1a5019e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a54c00640 1 --2- 172.21.15.47:0/721210371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a501038e0 0x7f1a5019e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a54c00640 1 -- 172.21.15.47:0/721210371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a50105490 msgr2=0x7f1a501a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a54c00640 1 --2- 172.21.15.47:0/721210371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a50105490 0x7f1a501a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a54c00640 1 -- 172.21.15.47:0/721210371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a501a5c30 con 0x7f1a50104ad0 2024-03-21T11:37:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.071+0000 7f1a55600640 1 --2- 172.21.15.47:0/721210371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a501038e0 0x7f1a5019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.072+0000 7f1a54c00640 1 --2- 172.21.15.47:0/721210371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a50104ad0 0x7f1a5019e550 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00e9f0 tx=0x7f1a3c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.072+0000 7f1a4e000640 1 -- 172.21.15.47:0/721210371 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00ccc0 con 0x7f1a50104ad0 2024-03-21T11:37:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.072+0000 7f1a4e000640 1 -- 172.21.15.47:0/721210371 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a3c005020 con 0x7f1a50104ad0 2024-03-21T11:37:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.072+0000 7f1a58000640 1 -- 172.21.15.47:0/721210371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a501a5f20 con 0x7f1a50104ad0 2024-03-21T11:37:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.072+0000 7f1a4e000640 1 -- 172.21.15.47:0/721210371 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0049e0 con 0x7f1a50104ad0 2024-03-21T11:37:59.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.072+0000 7f1a58000640 1 -- 172.21.15.47:0/721210371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a501a63e0 con 0x7f1a50104ad0 2024-03-21T11:37:59.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.077+0000 7f1a5ac78740 1 Processor -- start 2024-03-21T11:37:59.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.077+0000 7f1a5ac78740 1 -- start start 2024-03-21T11:37:59.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.077+0000 7f1a4e000640 1 -- 172.21.15.47:0/721210371 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1a3c005560 con 0x7f1a50104ad0 2024-03-21T11:37:59.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.077+0000 7f1a5ac78740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f19e80 0x55fba4f1a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.077+0000 7f1a5ac78740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 0x55fba4f1f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.078+0000 7f1a5ac78740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f1f9e0 0x55fba4f21e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.078+0000 7f1a5ac78740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba4e7f250 con 0x55fba4f19e80 2024-03-21T11:37:59.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.078+0000 7f1a5ac78740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba4e27a30 con 0x55fba4f1a870 2024-03-21T11:37:59.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.078+0000 7f1a5ac78740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba4e7b2f0 con 0x55fba4f1f9e0 2024-03-21T11:37:59.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.078+0000 7f1a56000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f19e80 0x55fba4f1a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.078+0000 7f1a54c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f1f9e0 0x55fba4f21e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.078+0000 7f1a55600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 0x55fba4f1f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.078+0000 7f1a56000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f19e80 0x55fba4f1a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37914/0 (socket says 172.21.15.47:37914) 2024-03-21T11:37:59.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.078+0000 7f1a56000640 1 -- 172.21.15.47:0/1568909203 learned_addr learned my addr 172.21.15.47:0/1568909203 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:59.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.079+0000 7f1a55600640 1 -- 172.21.15.47:0/1568909203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f1f9e0 msgr2=0x55fba4f21e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:59.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.079+0000 7f1a55600640 1 --2- 172.21.15.47:0/1568909203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f1f9e0 0x55fba4f21e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.079+0000 7f1a55600640 1 -- 172.21.15.47:0/1568909203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f19e80 msgr2=0x55fba4f1a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:59.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.079+0000 7f1a55600640 1 --2- 172.21.15.47:0/1568909203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f19e80 0x55fba4f1a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.079+0000 7f1a55600640 1 -- 172.21.15.47:0/1568909203 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fba4e12690 con 0x55fba4f1a870 2024-03-21T11:37:59.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.079+0000 7f1a56000640 1 --2- 172.21.15.47:0/1568909203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f19e80 0x55fba4f1a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:59.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.079+0000 7f1a4e000640 1 --2- 172.21.15.47:0/721210371 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a380a2fa0 0x7f1a380a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.079+0000 7f1a4e000640 1 -- 172.21.15.47:0/721210371 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1a3c0cf770 con 0x7f1a50104ad0 2024-03-21T11:37:59.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.079+0000 7f1a56000640 1 --2- 172.21.15.47:0/721210371 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a380a2fa0 0x7f1a380a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.080+0000 7f1a55600640 1 --2- 172.21.15.47:0/1568909203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 0x55fba4f1f370 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f1a44000f80 tx=0x7f1a440323e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d24be52da053531 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.080+0000 7f1a42600640 1 -- 172.21.15.47:0/1568909203 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44043070 con 0x55fba4f1a870 2024-03-21T11:37:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.080+0000 7f1a42600640 1 -- 172.21.15.47:0/1568909203 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a44032c90 con 0x55fba4f1a870 2024-03-21T11:37:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.080+0000 7f1a42600640 1 -- 172.21.15.47:0/1568909203 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44039040 con 0x55fba4f1a870 2024-03-21T11:37:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.081+0000 7f1a5ac78740 1 -- 172.21.15.47:0/1568909203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 msgr2=0x55fba4f1f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.081+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/1568909203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 0x55fba4f1f370 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f1a44000f80 tx=0x7f1a440323e0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.081+0000 7f1a56000640 1 --2- 172.21.15.47:0/721210371 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a380a2fa0 0x7f1a380a5460 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55fba4f1b0d0 tx=0x7f1a48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.081+0000 7f1a5ac78740 1 -- 172.21.15.47:0/1568909203 shutdown_connections 2024-03-21T11:37:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.081+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/1568909203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f1f9e0 0x55fba4f21e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.081+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/1568909203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 0x55fba4f1f370 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.081+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/1568909203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f19e80 0x55fba4f1a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.081+0000 7f1a5ac78740 1 -- 172.21.15.47:0/1568909203 >> 172.21.15.47:0/1568909203 conn(0x55fba4ee2200 msgr2=0x55fba4f157f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.081+0000 7f1a5ac78740 1 -- 172.21.15.47:0/1568909203 shutdown_connections 2024-03-21T11:37:59.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.081+0000 7f1a5ac78740 1 -- 172.21.15.47:0/1568909203 wait complete. 2024-03-21T11:37:59.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.082+0000 7f1a5ac78740 1 -- start start 2024-03-21T11:37:59.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.082+0000 7f1a5ac78740 1 Processor -- start 2024-03-21T11:37:59.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.082+0000 7f1a5ac78740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f19e80 0x55fba4f23d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.082+0000 7f1a5ac78740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 0x55fba4f242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.082+0000 7f1a5ac78740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f1f9e0 0x55fba4f2b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.082+0000 7f1a5ac78740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba4f22930 con 0x55fba4f1f9e0 2024-03-21T11:37:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.082+0000 7f1a5ac78740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba4e3ae80 con 0x55fba4f1a870 2024-03-21T11:37:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.082+0000 7f1a5ac78740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba4f22ab0 con 0x55fba4f19e80 2024-03-21T11:37:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.082+0000 7f1a5ac78740 10 client.0 _ll_register_callbacks cb 0x55fba4e4d130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.083+0000 7f1a55600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f19e80 0x55fba4f23d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.083+0000 7f1a55600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f19e80 0x55fba4f23d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43934/0 (socket says 172.21.15.47:43934) 2024-03-21T11:37:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.083+0000 7f1a55600640 1 -- 172.21.15.47:0/2682407396 learned_addr learned my addr 172.21.15.47:0/2682407396 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.083+0000 7f1a54c00640 1 --2- 172.21.15.47:0/2682407396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 0x55fba4f242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.083+0000 7f1a56000640 1 --2- 172.21.15.47:0/2682407396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f1f9e0 0x55fba4f2b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.083+0000 7f1a55600640 1 -- 172.21.15.47:0/2682407396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 msgr2=0x55fba4f242b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.083+0000 7f1a55600640 1 --2- 172.21.15.47:0/2682407396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 0x55fba4f242b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.083+0000 7f1a55600640 1 -- 172.21.15.47:0/2682407396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f1f9e0 msgr2=0x55fba4f2b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.083+0000 7f1a55600640 1 --2- 172.21.15.47:0/2682407396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f1f9e0 0x55fba4f2b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.083+0000 7f1a55600640 1 -- 172.21.15.47:0/2682407396 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fba4c5d7d0 con 0x55fba4f19e80 2024-03-21T11:37:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.083+0000 7f1a56000640 1 --2- 172.21.15.47:0/2682407396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f1f9e0 0x55fba4f2b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:37:59.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.083+0000 7f1a55600640 1 --2- 172.21.15.47:0/2682407396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f19e80 0x55fba4f23d70 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1a4402c700 tx=0x7f1a44032ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76b063b1920267cc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:59.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.083+0000 7f1a36a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:37:59.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.084+0000 7f1a36a00640 1 -- 172.21.15.47:0/2682407396 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4404c030 con 0x55fba4f19e80 2024-03-21T11:37:59.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.084+0000 7f1a5ac78740 1 -- 172.21.15.47:0/2682407396 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a44032c90 con 0x55fba4f19e80 2024-03-21T11:37:59.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.084+0000 7f1a36a00640 1 -- 172.21.15.47:0/2682407396 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1a4402cca0 con 0x55fba4f19e80 2024-03-21T11:37:59.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.084+0000 7f1a36a00640 1 -- 172.21.15.47:0/2682407396 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a44059050 con 0x55fba4f19e80 2024-03-21T11:37:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.084+0000 7f1a5ac78740 10 client.30364 fetch_fsmap learned FSMap version 34 2024-03-21T11:37:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.084+0000 7f1a5ac78740 1 -- 172.21.15.47:0/2682407396 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55fba4b933f0 con 0x55fba4f19e80 2024-03-21T11:37:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.085+0000 7f1a36a00640 1 -- 172.21.15.47:0/2682407396 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a44032c90 con 0x55fba4f19e80 2024-03-21T11:37:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.085+0000 7f1a36a00640 1 -- 172.21.15.47:0/2682407396 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f1a44048920 con 0x55fba4f19e80 2024-03-21T11:37:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.085+0000 7f1a5ac78740 10 client.30364 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:37:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.085+0000 7f1a5ac78740 10 client.30364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.085+0000 7f1a5ac78740 20 client.30364 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.085+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/2682407396 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fba4f2f7e0 0x55fba4f31bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.085+0000 7f1a5ac78740 4 client.30364 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:37:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.085+0000 7f1a5ac78740 1 -- 172.21.15.47:0/2682407396 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fba4e8c790 con 0x55fba4f2f7e0 2024-03-21T11:37:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.086+0000 7f1a54c00640 1 --2- 172.21.15.47:0/2682407396 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fba4f2f7e0 0x55fba4f31bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.086+0000 7f1a36a00640 10 client.30364 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.086+0000 7f1a54c00640 1 --2- 172.21.15.47:0/2682407396 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fba4f2f7e0 0x55fba4f31bd0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a70a3060ffb65792 server_cookie=693c39353cb05c82 in_seq=0 out_seq=0 2024-03-21T11:37:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.090+0000 7f1a36a00640 1 -- 172.21.15.47:0/2682407396 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fba4e8c790 con 0x55fba4f2f7e0 2024-03-21T11:37:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.090+0000 7f1a36a00640 10 client.30364 handle_command_reply: tid=0 2024-03-21T11:37:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.090+0000 7f1a5ac78740 1 client.30364 shutdown 2024-03-21T11:37:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.090+0000 7f1a5ac78740 1 -- 172.21.15.47:0/2682407396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f19e80 msgr2=0x55fba4f23d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:59.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.090+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/2682407396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f19e80 0x55fba4f23d70 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1a4402c700 tx=0x7f1a44032ac0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.091+0000 7f1a5ac78740 1 -- 172.21.15.47:0/2682407396 shutdown_connections 2024-03-21T11:37:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.091+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/2682407396 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fba4f2f7e0 0x55fba4f31bd0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.091+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/2682407396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f1f9e0 0x55fba4f2b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.091+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/2682407396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 0x55fba4f242b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.091+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/2682407396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f19e80 0x55fba4f23d70 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.091+0000 7f1a5ac78740 1 -- 172.21.15.47:0/2682407396 >> 172.21.15.47:0/2682407396 conn(0x55fba4ee2200 msgr2=0x55fba4f14a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.091+0000 7f1a36a00640 0 client.30364 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.091+0000 7f1a5ac78740 1 -- 172.21.15.47:0/2682407396 shutdown_connections 2024-03-21T11:37:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.091+0000 7f1a5ac78740 1 -- 172.21.15.47:0/2682407396 wait complete. 2024-03-21T11:37:59.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.092+0000 7f1a5ac78740 20 client.30364 trim_cache size 0 max 16384 2024-03-21T11:37:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.119+0000 7f1a5ac78740 1 Processor -- start 2024-03-21T11:37:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.119+0000 7f1a5ac78740 1 -- start start 2024-03-21T11:37:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.119+0000 7f1a5ac78740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f19e80 0x55fba4f1b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.119+0000 7f1a5ac78740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f1a870 0x55fba4f2b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.119+0000 7f1a5ac78740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1f9e0 0x55fba4f363f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.119+0000 7f1a5ac78740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a501b0680 con 0x55fba4f1a870 2024-03-21T11:37:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.119+0000 7f1a5ac78740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a501b2180 con 0x55fba4f1f9e0 2024-03-21T11:37:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.119+0000 7f1a5ac78740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a501b0ce0 con 0x55fba4f19e80 2024-03-21T11:37:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.119+0000 7f1a55600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f19e80 0x55fba4f1b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.120+0000 7f1a55600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f19e80 0x55fba4f1b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43936/0 (socket says 172.21.15.47:43936) 2024-03-21T11:37:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.120+0000 7f1a55600640 1 -- 172.21.15.47:0/3105484953 learned_addr learned my addr 172.21.15.47:0/3105484953 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.120+0000 7f1a54c00640 1 --2- 172.21.15.47:0/3105484953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f1a870 0x55fba4f2b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.120+0000 7f1a56000640 1 --2- 172.21.15.47:0/3105484953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1f9e0 0x55fba4f363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.120+0000 7f1a55600640 1 -- 172.21.15.47:0/3105484953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1f9e0 msgr2=0x55fba4f363f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.120+0000 7f1a55600640 1 --2- 172.21.15.47:0/3105484953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1f9e0 0x55fba4f363f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.120+0000 7f1a55600640 1 -- 172.21.15.47:0/3105484953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f1a870 msgr2=0x55fba4f2b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.120+0000 7f1a55600640 1 --2- 172.21.15.47:0/3105484953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f1a870 0x55fba4f2b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.120+0000 7f1a55600640 1 -- 172.21.15.47:0/3105484953 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fba4b530f0 con 0x55fba4f19e80 2024-03-21T11:37:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.120+0000 7f1a55600640 1 --2- 172.21.15.47:0/3105484953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f19e80 0x55fba4f1b2e0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1a44002230 tx=0x7f1a440095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eea1d919cce5182 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.121+0000 7f1a41c00640 1 -- 172.21.15.47:0/3105484953 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44055020 con 0x55fba4f19e80 2024-03-21T11:37:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.121+0000 7f1a41c00640 1 -- 172.21.15.47:0/3105484953 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a44032c90 con 0x55fba4f19e80 2024-03-21T11:37:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.121+0000 7f1a41c00640 1 -- 172.21.15.47:0/3105484953 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4404c050 con 0x55fba4f19e80 2024-03-21T11:37:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.122+0000 7f1a5ac78740 1 -- 172.21.15.47:0/3105484953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f19e80 msgr2=0x55fba4f1b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.122+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/3105484953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f19e80 0x55fba4f1b2e0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1a44002230 tx=0x7f1a440095c0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.122+0000 7f1a5ac78740 1 -- 172.21.15.47:0/3105484953 shutdown_connections 2024-03-21T11:37:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.122+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/3105484953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1f9e0 0x55fba4f363f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.122+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/3105484953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f1a870 0x55fba4f2b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.122+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/3105484953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f19e80 0x55fba4f1b2e0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.122+0000 7f1a5ac78740 1 -- 172.21.15.47:0/3105484953 >> 172.21.15.47:0/3105484953 conn(0x55fba4ee2200 msgr2=0x55fba4f14a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.122+0000 7f1a5ac78740 1 -- 172.21.15.47:0/3105484953 shutdown_connections 2024-03-21T11:37:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.122+0000 7f1a5ac78740 1 -- 172.21.15.47:0/3105484953 wait complete. 2024-03-21T11:37:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.122+0000 7f1a5ac78740 1 -- start start 2024-03-21T11:37:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.123+0000 7f1a5ac78740 1 Processor -- start 2024-03-21T11:37:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.123+0000 7f1a5ac78740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f19e80 0x55fba4f288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.123+0000 7f1a5ac78740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 0x55fba4f28df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.123+0000 7f1a5ac78740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f1f9e0 0x55fba4f29db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.123+0000 7f1a5ac78740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a501b1a60 con 0x55fba4f19e80 2024-03-21T11:37:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.123+0000 7f1a5ac78740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a501b1400 con 0x55fba4f1a870 2024-03-21T11:37:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.123+0000 7f1a5ac78740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a501ae840 con 0x55fba4f1f9e0 2024-03-21T11:37:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.123+0000 7f1a5ac78740 10 client.0 _ll_register_callbacks cb 0x7f1a480096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:37:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.123+0000 7f1a55600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f19e80 0x55fba4f288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.123+0000 7f1a55600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f19e80 0x55fba4f288b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37938/0 (socket says 172.21.15.47:37938) 2024-03-21T11:37:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.123+0000 7f1a55600640 1 -- 172.21.15.47:0/368796131 learned_addr learned my addr 172.21.15.47:0/368796131 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.123+0000 7f1a54c00640 1 --2- 172.21.15.47:0/368796131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 0x55fba4f28df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.124+0000 7f1a56000640 1 --2- 172.21.15.47:0/368796131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f1f9e0 0x55fba4f29db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.124+0000 7f1a56000640 1 -- 172.21.15.47:0/368796131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 msgr2=0x55fba4f28df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.124+0000 7f1a56000640 1 --2- 172.21.15.47:0/368796131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 0x55fba4f28df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.124+0000 7f1a56000640 1 -- 172.21.15.47:0/368796131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f19e80 msgr2=0x55fba4f288b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:37:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.124+0000 7f1a56000640 1 --2- 172.21.15.47:0/368796131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f19e80 0x55fba4f288b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.124+0000 7f1a56000640 1 -- 172.21.15.47:0/368796131 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fba4f2a3a0 con 0x55fba4f1f9e0 2024-03-21T11:37:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.124+0000 7f1a54c00640 1 --2- 172.21.15.47:0/368796131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 0x55fba4f28df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.124+0000 7f1a56000640 1 --2- 172.21.15.47:0/368796131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f1f9e0 0x55fba4f29db0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f1a48005d80 tx=0x7f1a480046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c700b774e4194370 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.124+0000 7f1a36a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:37:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.125+0000 7f1a36a00640 1 -- 172.21.15.47:0/368796131 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48004020 con 0x55fba4f1f9e0 2024-03-21T11:37:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.125+0000 7f1a36a00640 1 -- 172.21.15.47:0/368796131 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1a480041c0 con 0x55fba4f1f9e0 2024-03-21T11:37:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.125+0000 7f1a5ac78740 1 -- 172.21.15.47:0/368796131 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a44032c90 con 0x55fba4f1f9e0 2024-03-21T11:37:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.125+0000 7f1a36a00640 1 -- 172.21.15.47:0/368796131 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a480043c0 con 0x55fba4f1f9e0 2024-03-21T11:37:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.125+0000 7f1a5ac78740 10 client.30370 fetch_fsmap learned FSMap version 34 2024-03-21T11:37:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.125+0000 7f1a5ac78740 1 -- 172.21.15.47:0/368796131 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55fba4f8afc0 con 0x55fba4f1f9e0 2024-03-21T11:37:59.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.125+0000 7f1a36a00640 1 -- 172.21.15.47:0/368796131 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a44032c90 con 0x55fba4f1f9e0 2024-03-21T11:37:59.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.125+0000 7f1a36a00640 1 -- 172.21.15.47:0/368796131 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f1a48011740 con 0x55fba4f1f9e0 2024-03-21T11:37:59.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.126+0000 7f1a5ac78740 10 client.30370 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:37:59.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.126+0000 7f1a5ac78740 10 client.30370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:37:59.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.126+0000 7f1a5ac78740 20 client.30370 populate_metadata read hostname 'smithi047' 2024-03-21T11:37:59.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.126+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/368796131 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fba4f8b7b0 0x55fba4f8dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.126+0000 7f1a5ac78740 4 client.30370 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:37:59.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.126+0000 7f1a5ac78740 1 -- 172.21.15.47:0/368796131 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fba4e86ae0 con 0x55fba4f8b7b0 2024-03-21T11:37:59.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.126+0000 7f1a55600640 1 --2- 172.21.15.47:0/368796131 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fba4f8b7b0 0x55fba4f8dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.126+0000 7f1a36a00640 10 client.30370 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.126+0000 7f1a55600640 1 --2- 172.21.15.47:0/368796131 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fba4f8b7b0 0x55fba4f8dc30 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c570f821e558cb server_cookie=70ae7727596963bf in_seq=0 out_seq=0 2024-03-21T11:37:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.129+0000 7f1a36a00640 1 -- 172.21.15.47:0/368796131 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fba4e86ae0 con 0x55fba4f8b7b0 2024-03-21T11:37:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.129+0000 7f1a36a00640 10 client.30370 handle_command_reply: tid=0 2024-03-21T11:37:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.129+0000 7f1a5ac78740 1 client.30370 shutdown 2024-03-21T11:37:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.129+0000 7f1a5ac78740 1 -- 172.21.15.47:0/368796131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f1f9e0 msgr2=0x55fba4f29db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.129+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/368796131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f1f9e0 0x55fba4f29db0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f1a48005d80 tx=0x7f1a480046c0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.129+0000 7f1a5ac78740 1 -- 172.21.15.47:0/368796131 shutdown_connections 2024-03-21T11:37:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.129+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/368796131 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fba4f8b7b0 0x55fba4f8dc30 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.129+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/368796131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fba4f1f9e0 0x55fba4f29db0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.129+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/368796131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba4f1a870 0x55fba4f28df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.129+0000 7f1a5ac78740 1 --2- 172.21.15.47:0/368796131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fba4f19e80 0x55fba4f288b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.129+0000 7f1a5ac78740 1 -- 172.21.15.47:0/368796131 >> 172.21.15.47:0/368796131 conn(0x55fba4ee2200 msgr2=0x55fba4f14a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.129+0000 7f1a36a00640 0 client.30370 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:37:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.130+0000 7f1a5ac78740 1 -- 172.21.15.47:0/368796131 shutdown_connections 2024-03-21T11:37:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.130+0000 7f1a5ac78740 1 -- 172.21.15.47:0/368796131 wait complete. 2024-03-21T11:37:59.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.130+0000 7f1a5ac78740 20 client.30370 trim_cache size 0 max 16384 2024-03-21T11:37:59.132 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:37:59.132 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:37:59.132 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:37:59.132 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:37:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.131+0000 7f1a58000640 1 -- 172.21.15.47:0/721210371 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a380a2fa0 msgr2=0x7f1a380a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.131+0000 7f1a58000640 1 --2- 172.21.15.47:0/721210371 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a380a2fa0 0x7f1a380a5460 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55fba4f1b0d0 tx=0x7f1a48005f90 comp rx=0 tx=0).stop 2024-03-21T11:37:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.131+0000 7f1a58000640 1 -- 172.21.15.47:0/721210371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a50104ad0 msgr2=0x7f1a5019e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.131+0000 7f1a58000640 1 --2- 172.21.15.47:0/721210371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a50104ad0 0x7f1a5019e550 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00e9f0 tx=0x7f1a3c00eeb0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.131+0000 7f1a58000640 1 -- 172.21.15.47:0/721210371 shutdown_connections 2024-03-21T11:37:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.131+0000 7f1a58000640 1 --2- 172.21.15.47:0/721210371 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a380a2fa0 0x7f1a380a5460 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.131+0000 7f1a58000640 1 --2- 172.21.15.47:0/721210371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a50105490 0x7f1a501a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.131+0000 7f1a58000640 1 --2- 172.21.15.47:0/721210371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a50104ad0 0x7f1a5019e550 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.131+0000 7f1a58000640 1 --2- 172.21.15.47:0/721210371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a501038e0 0x7f1a5019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.131+0000 7f1a58000640 1 -- 172.21.15.47:0/721210371 >> 172.21.15.47:0/721210371 conn(0x7f1a500fee50 msgr2=0x7f1a500791a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.131+0000 7f1a58000640 1 -- 172.21.15.47:0/721210371 shutdown_connections 2024-03-21T11:37:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.131+0000 7f1a58000640 1 -- 172.21.15.47:0/721210371 wait complete. 2024-03-21T11:37:59.151 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:37:59.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:37:59.240 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:37:59 smithi047 ceph-mon[31351]: pgmap v839: 129 pgs: 129 active+clean; 260 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.3 MiB/s wr, 549 op/s 2024-03-21T11:37:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:37:59 smithi100 ceph-mon[36245]: pgmap v839: 129 pgs: 129 active+clean; 260 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.3 MiB/s wr, 549 op/s 2024-03-21T11:37:59.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.430+0000 7f6097e00640 1 -- 172.21.15.47:0/1184471651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60981048a0 msgr2=0x7f6098106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:59.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.430+0000 7f6097e00640 1 --2- 172.21.15.47:0/1184471651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60981048a0 0x7f6098106c90 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f6084009900 tx=0x7f608402bc30 comp rx=0 tx=0).stop 2024-03-21T11:37:59.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.430+0000 7f6097e00640 1 -- 172.21.15.47:0/1184471651 shutdown_connections 2024-03-21T11:37:59.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.430+0000 7f6097e00640 1 --2- 172.21.15.47:0/1184471651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60981071d0 0x7f6098109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.430+0000 7f6097e00640 1 --2- 172.21.15.47:0/1184471651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60981048a0 0x7f6098106c90 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.430+0000 7f6097e00640 1 --2- 172.21.15.47:0/1184471651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6098101f70 0x7f6098104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.430+0000 7f6097e00640 1 -- 172.21.15.47:0/1184471651 >> 172.21.15.47:0/1184471651 conn(0x7f60980fb980 msgr2=0x7f60980fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.430+0000 7f6097e00640 1 -- 172.21.15.47:0/1184471651 shutdown_connections 2024-03-21T11:37:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.430+0000 7f6097e00640 1 -- 172.21.15.47:0/1184471651 wait complete. 2024-03-21T11:37:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.431+0000 7f6097e00640 1 Processor -- start 2024-03-21T11:37:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.431+0000 7f6097e00640 1 -- start start 2024-03-21T11:37:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.432+0000 7f6097e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6098101f70 0x7f60981018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.432+0000 7f6097e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60981048a0 0x7f60980fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.432+0000 7f6097e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60981071d0 0x7f6098100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.432+0000 7f6097e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609810d180 con 0x7f60981071d0 2024-03-21T11:37:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.432+0000 7f6097e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609810d000 con 0x7f60981048a0 2024-03-21T11:37:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.432+0000 7f6097e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609810d300 con 0x7f6098101f70 2024-03-21T11:37:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.432+0000 7f6096a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6098101f70 0x7f60981018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.432+0000 7f6096a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6098101f70 0x7f60981018e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43954/0 (socket says 172.21.15.47:43954) 2024-03-21T11:37:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.432+0000 7f6096000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60981048a0 0x7f60980fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.432+0000 7f6096a00640 1 -- 172.21.15.47:0/1517651050 learned_addr learned my addr 172.21.15.47:0/1517651050 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:37:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.432+0000 7f6097400640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60981071d0 0x7f6098100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.432+0000 7f6096a00640 1 -- 172.21.15.47:0/1517651050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60981048a0 msgr2=0x7f60980fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.432+0000 7f6096a00640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60981048a0 0x7f60980fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.432+0000 7f6096a00640 1 -- 172.21.15.47:0/1517651050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60981071d0 msgr2=0x7f6098100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.433+0000 7f6096a00640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60981071d0 0x7f6098100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.433+0000 7f6096a00640 1 -- 172.21.15.47:0/1517651050 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60981009a0 con 0x7f6098101f70 2024-03-21T11:37:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.433+0000 7f6096000640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60981048a0 0x7f60980fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.433+0000 7f6097400640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60981071d0 0x7f6098100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:37:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.433+0000 7f6096a00640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6098101f70 0x7f60981018e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f6090009d50 tx=0x7f609000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.433+0000 7f608f400640 1 -- 172.21.15.47:0/1517651050 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090003b70 con 0x7f6098101f70 2024-03-21T11:37:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.433+0000 7f6097e00640 1 -- 172.21.15.47:0/1517651050 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6098100c90 con 0x7f6098101f70 2024-03-21T11:37:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.433+0000 7f608f400640 1 -- 172.21.15.47:0/1517651050 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6090003d10 con 0x7f6098101f70 2024-03-21T11:37:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.433+0000 7f608f400640 1 -- 172.21.15.47:0/1517651050 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090010a20 con 0x7f6098101f70 2024-03-21T11:37:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.434+0000 7f6097e00640 1 -- 172.21.15.47:0/1517651050 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60981a62f0 con 0x7f6098101f70 2024-03-21T11:37:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.435+0000 7f6097e00640 1 -- 172.21.15.47:0/1517651050 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60980fd470 con 0x7f6098101f70 2024-03-21T11:37:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.437+0000 7f608f400640 1 -- 172.21.15.47:0/1517651050 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6090004000 con 0x7f6098101f70 2024-03-21T11:37:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.439+0000 7f608f400640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f607c0a3020 0x7f607c0a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:37:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.440+0000 7f608f400640 1 -- 172.21.15.47:0/1517651050 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f60900cfdb0 con 0x7f6098101f70 2024-03-21T11:37:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.440+0000 7f6096000640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f607c0a3020 0x7f607c0a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:37:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.441+0000 7f6096000640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f607c0a3020 0x7f607c0a54e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6084000c00 tx=0x7f6084005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:37:59.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.446+0000 7f608f400640 1 -- 172.21.15.47:0/1517651050 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f609009c7f0 con 0x7f6098101f70 2024-03-21T11:37:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:37:59 smithi102 ceph-mon[36748]: pgmap v839: 129 pgs: 129 active+clean; 260 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.3 MiB/s wr, 549 op/s 2024-03-21T11:37:59.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.796+0000 7f6097e00640 1 -- 172.21.15.47:0/1517651050 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f609802d070 con 0x7f6098101f70 2024-03-21T11:37:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.799+0000 7f608f400640 1 -- 172.21.15.47:0/1517651050 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+11493 (secure 0 0 0) 0x7f609009c0f0 con 0x7f6098101f70 2024-03-21T11:37:59.800 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:37:59.801 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:37:28.116356+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581,24553,24535,24523,24524]},"id":1}]} 2024-03-21T11:37:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2024-03-21T11:37:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.803+0000 7f6097e00640 1 -- 172.21.15.47:0/1517651050 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f607c0a3020 msgr2=0x7f607c0a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.803+0000 7f6097e00640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f607c0a3020 0x7f607c0a54e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6084000c00 tx=0x7f6084005fd0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.803+0000 7f6097e00640 1 -- 172.21.15.47:0/1517651050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6098101f70 msgr2=0x7f60981018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:37:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.803+0000 7f6097e00640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6098101f70 0x7f60981018e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f6090009d50 tx=0x7f609000c6a0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.803+0000 7f6097e00640 1 -- 172.21.15.47:0/1517651050 shutdown_connections 2024-03-21T11:37:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.803+0000 7f6097e00640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f607c0a3020 0x7f607c0a54e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.803+0000 7f6097e00640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60981071d0 0x7f6098100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.803+0000 7f6097e00640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60981048a0 0x7f60980fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.803+0000 7f6097e00640 1 --2- 172.21.15.47:0/1517651050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6098101f70 0x7f60981018e0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:37:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.803+0000 7f6097e00640 1 -- 172.21.15.47:0/1517651050 >> 172.21.15.47:0/1517651050 conn(0x7f60980fb980 msgr2=0x7f60981029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:37:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.803+0000 7f6097e00640 1 -- 172.21.15.47:0/1517651050 shutdown_connections 2024-03-21T11:37:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:37:59.803+0000 7f6097e00640 1 -- 172.21.15.47:0/1517651050 wait complete. 2024-03-21T11:37:59.827 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:37:59.827 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:37:59.922 INFO:tasks.workunit.client.0.smithi047.stdout: 10 22724 32.56 MB/sec warmup 43 sec latency 152.306 ms 2024-03-21T11:38:00.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.076+0000 7f0124400640 1 -- 172.21.15.47:0/2381413004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f011c103260 msgr2=0x7f011c109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.076+0000 7f0124400640 1 --2- 172.21.15.47:0/2381413004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f011c103260 0x7f011c109af0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0114009be0 tx=0x7f011402bef0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.077+0000 7f0124400640 1 -- 172.21.15.47:0/2381413004 shutdown_connections 2024-03-21T11:38:00.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.077+0000 7f0124400640 1 --2- 172.21.15.47:0/2381413004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f011c103260 0x7f011c109af0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.077+0000 7f0124400640 1 --2- 172.21.15.47:0/2381413004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f011c1028a0 0x7f011c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.077+0000 7f0124400640 1 --2- 172.21.15.47:0/2381413004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f011c1016b0 0x7f011c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.077+0000 7f0124400640 1 -- 172.21.15.47:0/2381413004 >> 172.21.15.47:0/2381413004 conn(0x7f011c0fcc40 msgr2=0x7f011c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.077+0000 7f0124400640 1 -- 172.21.15.47:0/2381413004 shutdown_connections 2024-03-21T11:38:00.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.077+0000 7f0124400640 1 -- 172.21.15.47:0/2381413004 wait complete. 2024-03-21T11:38:00.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.078+0000 7f0124400640 1 Processor -- start 2024-03-21T11:38:00.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.078+0000 7f0124400640 1 -- start start 2024-03-21T11:38:00.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.079+0000 7f0124400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f011c1016b0 0x7f011c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.079+0000 7f0124400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f011c1028a0 0x7f011c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.079+0000 7f0124400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f011c103260 0x7f011c1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.079+0000 7f0124400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c10f5a0 con 0x7f011c1028a0 2024-03-21T11:38:00.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.079+0000 7f0124400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c10f420 con 0x7f011c103260 2024-03-21T11:38:00.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.079+0000 7f0124400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c10f720 con 0x7f011c1016b0 2024-03-21T11:38:00.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.079+0000 7f0121000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f011c1028a0 0x7f011c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.079+0000 7f0122400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f011c103260 0x7f011c1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.079+0000 7f0122400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f011c103260 0x7f011c1a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48804/0 (socket says 172.21.15.47:48804) 2024-03-21T11:38:00.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.079+0000 7f0122400640 1 -- 172.21.15.47:0/1359851124 learned_addr learned my addr 172.21.15.47:0/1359851124 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.080+0000 7f0121000640 1 -- 172.21.15.47:0/1359851124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f011c1016b0 msgr2=0x7f011c199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.080+0000 7f0121000640 1 --2- 172.21.15.47:0/1359851124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f011c1016b0 0x7f011c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.080+0000 7f0121000640 1 -- 172.21.15.47:0/1359851124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f011c103260 msgr2=0x7f011c1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.080+0000 7f0121000640 1 --2- 172.21.15.47:0/1359851124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f011c103260 0x7f011c1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.080+0000 7f0121000640 1 -- 172.21.15.47:0/1359851124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c1a1840 con 0x7f011c1028a0 2024-03-21T11:38:00.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.080+0000 7f0121000640 1 --2- 172.21.15.47:0/1359851124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f011c1028a0 0x7f011c19a170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0108005fd0 tx=0x7f010800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.080+0000 7f011a000640 1 -- 172.21.15.47:0/1359851124 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010800cd00 con 0x7f011c1028a0 2024-03-21T11:38:00.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.080+0000 7f011a000640 1 -- 172.21.15.47:0/1359851124 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01080035f0 con 0x7f011c1028a0 2024-03-21T11:38:00.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.080+0000 7f0124400640 1 -- 172.21.15.47:0/1359851124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011c1a1b30 con 0x7f011c1028a0 2024-03-21T11:38:00.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.081+0000 7f011a000640 1 -- 172.21.15.47:0/1359851124 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108010690 con 0x7f011c1028a0 2024-03-21T11:38:00.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.081+0000 7f0124400640 1 -- 172.21.15.47:0/1359851124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011c1a2080 con 0x7f011c1028a0 2024-03-21T11:38:00.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.085+0000 7f012722f740 1 Processor -- start 2024-03-21T11:38:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.085+0000 7f012722f740 1 -- start start 2024-03-21T11:38:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.085+0000 7f012722f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3de020 0x561e9e3de440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.085+0000 7f012722f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 0x561e9e3df670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.086+0000 7f011a000640 1 -- 172.21.15.47:0/1359851124 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f01080037a0 con 0x7f011c1028a0 2024-03-21T11:38:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.086+0000 7f012722f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3dfbb0 0x561e9e3e6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.086+0000 7f012722f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9e33a2c0 con 0x561e9e3de020 2024-03-21T11:38:00.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.086+0000 7f012722f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9e2e2c90 con 0x561e9e3df1f0 2024-03-21T11:38:00.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.086+0000 7f012722f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9e336360 con 0x561e9e3dfbb0 2024-03-21T11:38:00.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.086+0000 7f0122400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3de020 0x561e9e3de440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.087+0000 7f0121000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3dfbb0 0x561e9e3e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.087+0000 7f0121a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 0x561e9e3df670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.087+0000 7f0121000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3dfbb0 0x561e9e3e6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43998/0 (socket says 172.21.15.47:43998) 2024-03-21T11:38:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.087+0000 7f0121000640 1 -- 172.21.15.47:0/2472403902 learned_addr learned my addr 172.21.15.47:0/2472403902 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.087+0000 7f0121a00640 1 -- 172.21.15.47:0/2472403902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3dfbb0 msgr2=0x561e9e3e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.087+0000 7f0121a00640 1 --2- 172.21.15.47:0/2472403902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3dfbb0 0x561e9e3e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.087+0000 7f0121a00640 1 -- 172.21.15.47:0/2472403902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3de020 msgr2=0x561e9e3de440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.088+0000 7f0121a00640 1 --2- 172.21.15.47:0/2472403902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3de020 0x561e9e3de440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.088+0000 7f0121a00640 1 -- 172.21.15.47:0/2472403902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e9e307080 con 0x561e9e3df1f0 2024-03-21T11:38:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.088+0000 7f0122400640 1 --2- 172.21.15.47:0/2472403902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3de020 0x561e9e3de440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:38:00.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.088+0000 7f0121a00640 1 --2- 172.21.15.47:0/2472403902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 0x561e9e3df670 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f011c102700 tx=0x7f0110009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91fd3e594a1e33aa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.088+0000 7f010ea00640 1 -- 172.21.15.47:0/2472403902 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110015020 con 0x561e9e3df1f0 2024-03-21T11:38:00.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.088+0000 7f011a000640 1 --2- 172.21.15.47:0/1359851124 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01040a2fa0 0x7f01040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.088+0000 7f010ea00640 1 -- 172.21.15.47:0/2472403902 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f011000bdb0 con 0x561e9e3df1f0 2024-03-21T11:38:00.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.088+0000 7f010ea00640 1 -- 172.21.15.47:0/2472403902 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110010380 con 0x561e9e3df1f0 2024-03-21T11:38:00.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.089+0000 7f011a000640 1 -- 172.21.15.47:0/1359851124 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f01080cfb30 con 0x7f011c1028a0 2024-03-21T11:38:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.089+0000 7f0122400640 1 --2- 172.21.15.47:0/1359851124 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01040a2fa0 0x7f01040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.090+0000 7f012722f740 1 -- 172.21.15.47:0/2472403902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 msgr2=0x561e9e3df670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.090+0000 7f012722f740 1 --2- 172.21.15.47:0/2472403902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 0x561e9e3df670 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f011c102700 tx=0x7f0110009bb0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.090+0000 7f012722f740 1 -- 172.21.15.47:0/2472403902 shutdown_connections 2024-03-21T11:38:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.090+0000 7f012722f740 1 --2- 172.21.15.47:0/2472403902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3dfbb0 0x561e9e3e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.090+0000 7f012722f740 1 --2- 172.21.15.47:0/2472403902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 0x561e9e3df670 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.090+0000 7f012722f740 1 --2- 172.21.15.47:0/2472403902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3de020 0x561e9e3de440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.090+0000 7f012722f740 1 -- 172.21.15.47:0/2472403902 >> 172.21.15.47:0/2472403902 conn(0x561e9e3d0c00 msgr2=0x561e9e3d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.091+0000 7f012722f740 1 -- 172.21.15.47:0/2472403902 shutdown_connections 2024-03-21T11:38:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.091+0000 7f012722f740 1 -- 172.21.15.47:0/2472403902 wait complete. 2024-03-21T11:38:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.091+0000 7f0122400640 1 --2- 172.21.15.47:0/1359851124 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01040a2fa0 0x7f01040a5460 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f01140099a0 tx=0x7f0114004a80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.091+0000 7f012722f740 1 -- start start 2024-03-21T11:38:00.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.092+0000 7f012722f740 1 Processor -- start 2024-03-21T11:38:00.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.092+0000 7f012722f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3e8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.092+0000 7f012722f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 0x561e9e3e8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.092+0000 7f012722f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3dfbb0 0x561e9e3ef810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.092+0000 7f012722f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9e3e6bc0 con 0x561e9e3dfbb0 2024-03-21T11:38:00.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.092+0000 7f012722f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9e2f60d0 con 0x561e9e3df1f0 2024-03-21T11:38:00.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.092+0000 7f012722f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9e3e6d40 con 0x561e9e3de020 2024-03-21T11:38:00.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.092+0000 7f012722f740 10 client.0 _ll_register_callbacks cb 0x561e9e2fae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:00.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.093+0000 7f0122400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3dfbb0 0x561e9e3ef810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.093+0000 7f0122400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3dfbb0 0x561e9e3ef810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38010/0 (socket says 172.21.15.47:38010) 2024-03-21T11:38:00.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.093+0000 7f0122400640 1 -- 172.21.15.47:0/3037659153 learned_addr learned my addr 172.21.15.47:0/3037659153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.093+0000 7f0121a00640 1 --2- 172.21.15.47:0/3037659153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3e8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.093+0000 7f0122400640 1 -- 172.21.15.47:0/3037659153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 msgr2=0x561e9e3e8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.093+0000 7f0122400640 1 --2- 172.21.15.47:0/3037659153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3e8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.093+0000 7f0122400640 1 -- 172.21.15.47:0/3037659153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 msgr2=0x561e9e3e8840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:38:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.093+0000 7f0122400640 1 --2- 172.21.15.47:0/3037659153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 0x561e9e3e8840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.093+0000 7f0122400640 1 -- 172.21.15.47:0/3037659153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e9e04e3f0 con 0x561e9e3dfbb0 2024-03-21T11:38:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.095+0000 7f0103400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:38:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.095+0000 7f0122400640 1 --2- 172.21.15.47:0/3037659153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3dfbb0 0x561e9e3ef810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0114036040 tx=0x7f011402bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8b745032d5e336 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.095+0000 7f0103400640 1 -- 172.21.15.47:0/3037659153 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114051020 con 0x561e9e3dfbb0 2024-03-21T11:38:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.095+0000 7f0103400640 1 -- 172.21.15.47:0/3037659153 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0114003e00 con 0x561e9e3dfbb0 2024-03-21T11:38:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.095+0000 7f012722f740 1 -- 172.21.15.47:0/3037659153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f011000bdb0 con 0x561e9e3dfbb0 2024-03-21T11:38:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.095+0000 7f0103400640 1 -- 172.21.15.47:0/3037659153 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f011402cde0 con 0x561e9e3dfbb0 2024-03-21T11:38:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.096+0000 7f012722f740 10 client.19935 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.096+0000 7f012722f740 1 -- 172.21.15.47:0/3037659153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561e9e1187d0 con 0x561e9e3dfbb0 2024-03-21T11:38:00.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.096+0000 7f0103400640 1 -- 172.21.15.47:0/3037659153 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f011000bdb0 con 0x561e9e3dfbb0 2024-03-21T11:38:00.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.096+0000 7f0103400640 1 -- 172.21.15.47:0/3037659153 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f01140349b0 con 0x561e9e3dfbb0 2024-03-21T11:38:00.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.096+0000 7f012722f740 10 client.19935 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:00.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.096+0000 7f012722f740 10 client.19935 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:38:00.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.096+0000 7f012722f740 20 client.19935 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:00.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.096+0000 7f012722f740 1 --2- 172.21.15.47:0/3037659153 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561e9e3f3e00 0x561e9e3f6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.096+0000 7f012722f740 4 client.19935 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:00.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.096+0000 7f012722f740 1 -- 172.21.15.47:0/3037659153 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e9e347800 con 0x561e9e3f3e00 2024-03-21T11:38:00.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.097+0000 7f0121a00640 1 --2- 172.21.15.47:0/3037659153 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561e9e3f3e00 0x561e9e3f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.097+0000 7f0121a00640 1 --2- 172.21.15.47:0/3037659153 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561e9e3f3e00 0x561e9e3f6240 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b71921ad49824da8 server_cookie=9785b08035f92d9d in_seq=0 out_seq=0 2024-03-21T11:38:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.097+0000 7f0103400640 10 client.19935 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:38:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.098+0000 7f0103400640 1 -- 172.21.15.47:0/3037659153 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561e9e347800 con 0x561e9e3f3e00 2024-03-21T11:38:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.098+0000 7f0103400640 10 client.19935 handle_command_reply: tid=0 2024-03-21T11:38:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.099+0000 7f012722f740 1 client.19935 shutdown 2024-03-21T11:38:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.099+0000 7f012722f740 1 -- 172.21.15.47:0/3037659153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3dfbb0 msgr2=0x561e9e3ef810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.099+0000 7f012722f740 1 --2- 172.21.15.47:0/3037659153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3dfbb0 0x561e9e3ef810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0114036040 tx=0x7f011402bef0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.099+0000 7f012722f740 1 -- 172.21.15.47:0/3037659153 shutdown_connections 2024-03-21T11:38:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.099+0000 7f012722f740 1 --2- 172.21.15.47:0/3037659153 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561e9e3f3e00 0x561e9e3f6240 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.099+0000 7f012722f740 1 --2- 172.21.15.47:0/3037659153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3dfbb0 0x561e9e3ef810 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.099+0000 7f012722f740 1 --2- 172.21.15.47:0/3037659153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 0x561e9e3e8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.099+0000 7f0103400640 0 client.19935 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:38:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.099+0000 7f012722f740 1 --2- 172.21.15.47:0/3037659153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3e8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.099+0000 7f012722f740 1 -- 172.21.15.47:0/3037659153 >> 172.21.15.47:0/3037659153 conn(0x561e9e3d0c00 msgr2=0x561e9e3cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.099+0000 7f012722f740 1 -- 172.21.15.47:0/3037659153 shutdown_connections 2024-03-21T11:38:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.100+0000 7f012722f740 1 -- 172.21.15.47:0/3037659153 wait complete. 2024-03-21T11:38:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.100+0000 7f012722f740 20 client.19935 trim_cache size 0 max 16384 2024-03-21T11:38:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.123+0000 7f012722f740 1 Processor -- start 2024-03-21T11:38:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.123+0000 7f012722f740 1 -- start start 2024-03-21T11:38:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.123+0000 7f012722f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.123+0000 7f012722f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 0x561e9e3ef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.123+0000 7f012722f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3dfbb0 0x561e9e3fab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.123+0000 7f012722f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c1a7fd0 con 0x561e9e3dfbb0 2024-03-21T11:38:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.123+0000 7f012722f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c1a9ad0 con 0x561e9e3df1f0 2024-03-21T11:38:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.123+0000 7f012722f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c1a8630 con 0x561e9e3de020 2024-03-21T11:38:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.124+0000 7f0121a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.124+0000 7f0121000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 0x561e9e3ef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.124+0000 7f0121a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44030/0 (socket says 172.21.15.47:44030) 2024-03-21T11:38:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.124+0000 7f0121a00640 1 -- 172.21.15.47:0/1047166287 learned_addr learned my addr 172.21.15.47:0/1047166287 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.124+0000 7f0122400640 1 --2- 172.21.15.47:0/1047166287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3dfbb0 0x561e9e3fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.124+0000 7f0121a00640 1 -- 172.21.15.47:0/1047166287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 msgr2=0x561e9e3ef7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.124+0000 7f0121a00640 1 --2- 172.21.15.47:0/1047166287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 0x561e9e3ef7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.124+0000 7f0121a00640 1 -- 172.21.15.47:0/1047166287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3dfbb0 msgr2=0x561e9e3fab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.124+0000 7f0121a00640 1 --2- 172.21.15.47:0/1047166287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3dfbb0 0x561e9e3fab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.124+0000 7f0121a00640 1 -- 172.21.15.47:0/1047166287 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e9e3dec70 con 0x561e9e3de020 2024-03-21T11:38:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.124+0000 7f0121a00640 1 --2- 172.21.15.47:0/1047166287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3cfd30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0110009670 tx=0x7f0110008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce93c726d5a3eaa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.125+0000 7f010e000640 1 -- 172.21.15.47:0/1047166287 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110015040 con 0x561e9e3de020 2024-03-21T11:38:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.125+0000 7f010e000640 1 -- 172.21.15.47:0/1047166287 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0110011420 con 0x561e9e3de020 2024-03-21T11:38:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.125+0000 7f010e000640 1 -- 172.21.15.47:0/1047166287 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011000bd30 con 0x561e9e3de020 2024-03-21T11:38:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.125+0000 7f012722f740 1 -- 172.21.15.47:0/1047166287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 msgr2=0x561e9e3cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.125+0000 7f012722f740 1 --2- 172.21.15.47:0/1047166287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3cfd30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0110009670 tx=0x7f0110008040 comp rx=0 tx=0).stop 2024-03-21T11:38:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.126+0000 7f012722f740 1 -- 172.21.15.47:0/1047166287 shutdown_connections 2024-03-21T11:38:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.126+0000 7f012722f740 1 --2- 172.21.15.47:0/1047166287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3dfbb0 0x561e9e3fab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.126+0000 7f012722f740 1 --2- 172.21.15.47:0/1047166287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3df1f0 0x561e9e3ef7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.126+0000 7f012722f740 1 --2- 172.21.15.47:0/1047166287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3cfd30 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.126+0000 7f012722f740 1 -- 172.21.15.47:0/1047166287 >> 172.21.15.47:0/1047166287 conn(0x561e9e3d0c00 msgr2=0x561e9e3cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.126+0000 7f012722f740 1 -- 172.21.15.47:0/1047166287 shutdown_connections 2024-03-21T11:38:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.127+0000 7f012722f740 1 -- 172.21.15.47:0/1047166287 wait complete. 2024-03-21T11:38:00.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.127+0000 7f012722f740 1 -- start start 2024-03-21T11:38:00.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.127+0000 7f012722f740 1 Processor -- start 2024-03-21T11:38:00.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f012722f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3ec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f012722f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3df1f0 0x561e9e3ec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f012722f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3dfbb0 0x561e9e44f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f012722f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c1a93b0 con 0x561e9e3df1f0 2024-03-21T11:38:00.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f012722f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c1a8d50 con 0x561e9e3dfbb0 2024-03-21T11:38:00.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f012722f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c1a6190 con 0x561e9e3de020 2024-03-21T11:38:00.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f012722f740 10 client.0 _ll_register_callbacks cb 0x7f0114044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:00.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f0121a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3ec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f0121a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3ec2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44042/0 (socket says 172.21.15.47:44042) 2024-03-21T11:38:00.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f0121a00640 1 -- 172.21.15.47:0/2510840008 learned_addr learned my addr 172.21.15.47:0/2510840008 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f0122400640 1 --2- 172.21.15.47:0/2510840008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3dfbb0 0x561e9e44f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f0121000640 1 --2- 172.21.15.47:0/2510840008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3df1f0 0x561e9e3ec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f0121a00640 1 -- 172.21.15.47:0/2510840008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3dfbb0 msgr2=0x561e9e44f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f0121a00640 1 --2- 172.21.15.47:0/2510840008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3dfbb0 0x561e9e44f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f0121a00640 1 -- 172.21.15.47:0/2510840008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3df1f0 msgr2=0x561e9e3ec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f0121a00640 1 --2- 172.21.15.47:0/2510840008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3df1f0 0x561e9e3ec800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f0121a00640 1 -- 172.21.15.47:0/2510840008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e9e3ef8d0 con 0x561e9e3de020 2024-03-21T11:38:00.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.128+0000 7f0121000640 1 --2- 172.21.15.47:0/2510840008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3df1f0 0x561e9e3ec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:00.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.129+0000 7f0121a00640 1 --2- 172.21.15.47:0/2510840008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3ec2c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0110004810 tx=0x7f011000b620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=235275f4e832ddb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.129+0000 7f0103400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:38:00.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.129+0000 7f0103400640 1 -- 172.21.15.47:0/2510840008 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110015040 con 0x561e9e3de020 2024-03-21T11:38:00.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.129+0000 7f0103400640 1 -- 172.21.15.47:0/2510840008 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0110011cd0 con 0x561e9e3de020 2024-03-21T11:38:00.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.129+0000 7f012722f740 1 -- 172.21.15.47:0/2510840008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0110011420 con 0x561e9e3de020 2024-03-21T11:38:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.130+0000 7f0103400640 1 -- 172.21.15.47:0/2510840008 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0110011810 con 0x561e9e3de020 2024-03-21T11:38:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.130+0000 7f012722f740 10 client.30388 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.130+0000 7f012722f740 1 -- 172.21.15.47:0/2510840008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561e9e450ad0 con 0x561e9e3de020 2024-03-21T11:38:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.130+0000 7f0103400640 1 -- 172.21.15.47:0/2510840008 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0110011420 con 0x561e9e3de020 2024-03-21T11:38:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.130+0000 7f0103400640 1 -- 172.21.15.47:0/2510840008 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f0110042480 con 0x561e9e3de020 2024-03-21T11:38:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.131+0000 7f012722f740 10 client.30388 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.131+0000 7f012722f740 10 client.30388 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:38:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.131+0000 7f012722f740 20 client.30388 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.131+0000 7f012722f740 1 --2- 172.21.15.47:0/2510840008 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561e9e4511d0 0x561e9e4535f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.131+0000 7f012722f740 4 client.30388 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:38:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.131+0000 7f012722f740 1 -- 172.21.15.47:0/2510840008 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561e9e3e80f0 con 0x561e9e4511d0 2024-03-21T11:38:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.131+0000 7f0121000640 1 --2- 172.21.15.47:0/2510840008 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561e9e4511d0 0x561e9e4535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.131+0000 7f0121000640 1 --2- 172.21.15.47:0/2510840008 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561e9e4511d0 0x561e9e4535f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=475b7dc471cf3818 server_cookie=63c50203a6fc0e18 in_seq=0 out_seq=0 2024-03-21T11:38:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f0103400640 10 client.30388 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:38:00.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f0103400640 1 -- 172.21.15.47:0/2510840008 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561e9e3e80f0 con 0x561e9e4511d0 2024-03-21T11:38:00.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f0103400640 10 client.30388 handle_command_reply: tid=0 2024-03-21T11:38:00.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f012722f740 1 client.30388 shutdown 2024-03-21T11:38:00.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f012722f740 1 -- 172.21.15.47:0/2510840008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 msgr2=0x561e9e3ec2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f012722f740 1 --2- 172.21.15.47:0/2510840008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3ec2c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0110004810 tx=0x7f011000b620 comp rx=0 tx=0).stop 2024-03-21T11:38:00.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f012722f740 1 -- 172.21.15.47:0/2510840008 shutdown_connections 2024-03-21T11:38:00.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f012722f740 1 --2- 172.21.15.47:0/2510840008 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x561e9e4511d0 0x561e9e4535f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f012722f740 1 --2- 172.21.15.47:0/2510840008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e9e3dfbb0 0x561e9e44f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f012722f740 1 --2- 172.21.15.47:0/2510840008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e9e3df1f0 0x561e9e3ec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f012722f740 1 --2- 172.21.15.47:0/2510840008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e9e3de020 0x561e9e3ec2c0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f0103400640 0 client.30388 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:38:00.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f012722f740 1 -- 172.21.15.47:0/2510840008 >> 172.21.15.47:0/2510840008 conn(0x561e9e3d0c00 msgr2=0x561e9e3cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f012722f740 1 -- 172.21.15.47:0/2510840008 shutdown_connections 2024-03-21T11:38:00.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.132+0000 7f012722f740 1 -- 172.21.15.47:0/2510840008 wait complete. 2024-03-21T11:38:00.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.133+0000 7f012722f740 20 client.30388 trim_cache size 0 max 16384 2024-03-21T11:38:00.137 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:38:00.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.133+0000 7f0124400640 1 -- 172.21.15.47:0/1359851124 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01040a2fa0 msgr2=0x7f01040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.133+0000 7f0124400640 1 --2- 172.21.15.47:0/1359851124 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01040a2fa0 0x7f01040a5460 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f01140099a0 tx=0x7f0114004a80 comp rx=0 tx=0).stop 2024-03-21T11:38:00.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.133+0000 7f0124400640 1 -- 172.21.15.47:0/1359851124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f011c1028a0 msgr2=0x7f011c19a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.133+0000 7f0124400640 1 --2- 172.21.15.47:0/1359851124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f011c1028a0 0x7f011c19a170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0108005fd0 tx=0x7f010800ee50 comp rx=0 tx=0).stop 2024-03-21T11:38:00.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.134+0000 7f0124400640 1 -- 172.21.15.47:0/1359851124 shutdown_connections 2024-03-21T11:38:00.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.134+0000 7f0124400640 1 --2- 172.21.15.47:0/1359851124 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f01040a2fa0 0x7f01040a5460 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.134+0000 7f0124400640 1 --2- 172.21.15.47:0/1359851124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f011c103260 0x7f011c1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.134+0000 7f0124400640 1 --2- 172.21.15.47:0/1359851124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f011c1028a0 0x7f011c19a170 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.134+0000 7f0124400640 1 --2- 172.21.15.47:0/1359851124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f011c1016b0 0x7f011c199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.134+0000 7f0124400640 1 -- 172.21.15.47:0/1359851124 >> 172.21.15.47:0/1359851124 conn(0x7f011c0fcc40 msgr2=0x7f011c105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.134+0000 7f0124400640 1 -- 172.21.15.47:0/1359851124 shutdown_connections 2024-03-21T11:38:00.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.134+0000 7f0124400640 1 -- 172.21.15.47:0/1359851124 wait complete. 2024-03-21T11:38:00.155 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:00 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1517651050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:38:00.161 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:38:00.161 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:38:00.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:00 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1517651050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:38:00.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.411+0000 7f8091000640 1 -- 172.21.15.47:0/3432907922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c10d2a0 msgr2=0x7f808c10d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.411+0000 7f8091000640 1 --2- 172.21.15.47:0/3432907922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c10d2a0 0x7f808c10d720 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8078009860 tx=0x7f807802bb70 comp rx=0 tx=0).stop 2024-03-21T11:38:00.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.412+0000 7f8091000640 1 -- 172.21.15.47:0/3432907922 shutdown_connections 2024-03-21T11:38:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.412+0000 7f8091000640 1 --2- 172.21.15.47:0/3432907922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f808c10dc60 0x7f808c1144f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.412+0000 7f8091000640 1 --2- 172.21.15.47:0/3432907922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c10d2a0 0x7f808c10d720 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.412+0000 7f8091000640 1 --2- 172.21.15.47:0/3432907922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f808c10c0b0 0x7f808c10c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.412+0000 7f8091000640 1 -- 172.21.15.47:0/3432907922 >> 172.21.15.47:0/3432907922 conn(0x7f808c06c060 msgr2=0x7f808c06c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.412+0000 7f8091000640 1 -- 172.21.15.47:0/3432907922 shutdown_connections 2024-03-21T11:38:00.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.413+0000 7f8091000640 1 -- 172.21.15.47:0/3432907922 wait complete. 2024-03-21T11:38:00.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.415+0000 7f8091000640 1 Processor -- start 2024-03-21T11:38:00.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.415+0000 7f8091000640 1 -- start start 2024-03-21T11:38:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.416+0000 7f8091000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f808c10c0b0 0x7f808c1a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.416+0000 7f8091000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f808c10d2a0 0x7f808c1a2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.416+0000 7f8091000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c10dc60 0x7f808c1a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.416+0000 7f8091000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c117d40 con 0x7f808c10dc60 2024-03-21T11:38:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.416+0000 7f8091000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c117bc0 con 0x7f808c10c0b0 2024-03-21T11:38:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.416+0000 7f8091000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c117ec0 con 0x7f808c10d2a0 2024-03-21T11:38:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.416+0000 7f8089400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f808c10d2a0 0x7f808c1a2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.416+0000 7f8089400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f808c10d2a0 0x7f808c1a2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44056/0 (socket says 172.21.15.47:44056) 2024-03-21T11:38:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.416+0000 7f8089400640 1 -- 172.21.15.47:0/910209698 learned_addr learned my addr 172.21.15.47:0/910209698 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.416+0000 7f8089e00640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f808c10c0b0 0x7f808c1a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.417+0000 7f8089400640 1 -- 172.21.15.47:0/910209698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f808c10c0b0 msgr2=0x7f808c1a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.417+0000 7f808a800640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c10dc60 0x7f808c1a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.417+0000 7f8089400640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f808c10c0b0 0x7f808c1a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.417+0000 7f8089400640 1 -- 172.21.15.47:0/910209698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c10dc60 msgr2=0x7f808c1a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.417+0000 7f8089400640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c10dc60 0x7f808c1a9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.417+0000 7f8089400640 1 -- 172.21.15.47:0/910209698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c1aa000 con 0x7f808c10d2a0 2024-03-21T11:38:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.418+0000 7f8089e00640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f808c10c0b0 0x7f808c1a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:38:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.418+0000 7f808a800640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c10dc60 0x7f808c1a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.419+0000 7f8089400640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f808c10d2a0 0x7f808c1a2920 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f80780349e0 tx=0x7f80780042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.420+0000 7f8082a00640 1 -- 172.21.15.47:0/910209698 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078041030 con 0x7f808c10d2a0 2024-03-21T11:38:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.420+0000 7f8082a00640 1 -- 172.21.15.47:0/910209698 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80780347b0 con 0x7f808c10d2a0 2024-03-21T11:38:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.420+0000 7f8082a00640 1 -- 172.21.15.47:0/910209698 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078039040 con 0x7f808c10d2a0 2024-03-21T11:38:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.420+0000 7f8091000640 1 -- 172.21.15.47:0/910209698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808c1aa290 con 0x7f808c10d2a0 2024-03-21T11:38:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.420+0000 7f8091000640 1 -- 172.21.15.47:0/910209698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c1aa710 con 0x7f808c10d2a0 2024-03-21T11:38:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.425+0000 7f8082a00640 1 -- 172.21.15.47:0/910209698 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8078044070 con 0x7f808c10d2a0 2024-03-21T11:38:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.425+0000 7f8093db4740 1 Processor -- start 2024-03-21T11:38:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.426+0000 7f8093db4740 1 -- start start 2024-03-21T11:38:00.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.427+0000 7f8093db4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e88810 0x55a2e6e88c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.427+0000 7f8093db4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e899e0 0x55a2e6e89e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.427+0000 7f8093db4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6e90c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.427+0000 7f8093db4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e6ded2c0 con 0x55a2e6e8a3a0 2024-03-21T11:38:00.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.427+0000 7f8093db4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e6d95c90 con 0x55a2e6e88810 2024-03-21T11:38:00.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.427+0000 7f8093db4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e6de9360 con 0x55a2e6e899e0 2024-03-21T11:38:00.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.428+0000 7f808a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e88810 0x55a2e6e88c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.428+0000 7f8089e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e899e0 0x55a2e6e89e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.428+0000 7f8089e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e899e0 0x55a2e6e89e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44070/0 (socket says 172.21.15.47:44070) 2024-03-21T11:38:00.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.428+0000 7f8089e00640 1 -- 172.21.15.47:0/1580947601 learned_addr learned my addr 172.21.15.47:0/1580947601 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.428+0000 7f8082a00640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80700a2fa0 0x7f80700a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.428+0000 7f8082a00640 1 -- 172.21.15.47:0/910209698 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f80780f0e90 con 0x7f808c10d2a0 2024-03-21T11:38:00.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.428+0000 7f8089400640 1 --2- 172.21.15.47:0/1580947601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6e90c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.430+0000 7f8089e00640 1 -- 172.21.15.47:0/1580947601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e88810 msgr2=0x55a2e6e88c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.430+0000 7f8089e00640 1 --2- 172.21.15.47:0/1580947601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e88810 0x55a2e6e88c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.430+0000 7f8089e00640 1 -- 172.21.15.47:0/1580947601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e8a3a0 msgr2=0x55a2e6e90c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.430+0000 7f8089e00640 1 --2- 172.21.15.47:0/1580947601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6e90c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.430+0000 7f8089e00640 1 -- 172.21.15.47:0/1580947601 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2e6dba080 con 0x55a2e6e899e0 2024-03-21T11:38:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.430+0000 7f808a800640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80700a2fa0 0x7f80700a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.431+0000 7f8089e00640 1 --2- 172.21.15.47:0/1580947601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e899e0 0x55a2e6e89e60 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f807c00bd20 tx=0x7f807c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab9c5cbb9ea97deb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.431+0000 7f8077400640 1 -- 172.21.15.47:0/1580947601 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c015070 con 0x55a2e6e899e0 2024-03-21T11:38:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.431+0000 7f8077400640 1 -- 172.21.15.47:0/1580947601 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f807c00be10 con 0x55a2e6e899e0 2024-03-21T11:38:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.431+0000 7f8077400640 1 -- 172.21.15.47:0/1580947601 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0107c0 con 0x55a2e6e899e0 2024-03-21T11:38:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.432+0000 7f8093db4740 1 -- 172.21.15.47:0/1580947601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e899e0 msgr2=0x55a2e6e89e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.432+0000 7f8093db4740 1 --2- 172.21.15.47:0/1580947601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e899e0 0x55a2e6e89e60 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f807c00bd20 tx=0x7f807c00f730 comp rx=0 tx=0).stop 2024-03-21T11:38:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.433+0000 7f808a800640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80700a2fa0 0x7f80700a5460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x55a2e6e898e0 tx=0x7f8084004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.433+0000 7f8093db4740 1 -- 172.21.15.47:0/1580947601 shutdown_connections 2024-03-21T11:38:00.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.433+0000 7f8093db4740 1 --2- 172.21.15.47:0/1580947601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6e90c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.433+0000 7f8093db4740 1 --2- 172.21.15.47:0/1580947601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e899e0 0x55a2e6e89e60 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.433+0000 7f8093db4740 1 --2- 172.21.15.47:0/1580947601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e88810 0x55a2e6e88c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.433+0000 7f8093db4740 1 -- 172.21.15.47:0/1580947601 >> 172.21.15.47:0/1580947601 conn(0x55a2e6e83c00 msgr2=0x55a2e6e84000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.434+0000 7f8093db4740 1 -- 172.21.15.47:0/1580947601 shutdown_connections 2024-03-21T11:38:00.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.434+0000 7f8093db4740 1 -- 172.21.15.47:0/1580947601 wait complete. 2024-03-21T11:38:00.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.434+0000 7f8093db4740 1 -- start start 2024-03-21T11:38:00.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.435+0000 7f8093db4740 1 Processor -- start 2024-03-21T11:38:00.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.436+0000 7f8093db4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e88810 0x55a2e6e9b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.436+0000 7f8093db4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e899e0 0x55a2e6e9b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.436+0000 7f8093db4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6ea2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.436+0000 7f8093db4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e6e99bc0 con 0x55a2e6e88810 2024-03-21T11:38:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.436+0000 7f8093db4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e6da90d0 con 0x55a2e6e899e0 2024-03-21T11:38:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.436+0000 7f8093db4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e6e99d40 con 0x55a2e6e8a3a0 2024-03-21T11:38:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.436+0000 7f8093db4740 10 client.0 _ll_register_callbacks cb 0x55a2e6dade80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.436+0000 7f808a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6ea2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.436+0000 7f808a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6ea2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44076/0 (socket says 172.21.15.47:44076) 2024-03-21T11:38:00.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.436+0000 7f808a800640 1 -- 172.21.15.47:0/1870493947 learned_addr learned my addr 172.21.15.47:0/1870493947 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.437+0000 7f8089400640 1 --2- 172.21.15.47:0/1870493947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e899e0 0x55a2e6e9b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.437+0000 7f8089e00640 1 --2- 172.21.15.47:0/1870493947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e88810 0x55a2e6e9b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.437+0000 7f808a800640 1 -- 172.21.15.47:0/1870493947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e899e0 msgr2=0x55a2e6e9b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.437+0000 7f808a800640 1 --2- 172.21.15.47:0/1870493947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e899e0 0x55a2e6e9b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.437+0000 7f808a800640 1 -- 172.21.15.47:0/1870493947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e88810 msgr2=0x55a2e6e9b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.437+0000 7f808a800640 1 --2- 172.21.15.47:0/1870493947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e88810 0x55a2e6e9b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.437+0000 7f808a800640 1 -- 172.21.15.47:0/1870493947 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2e6b013f0 con 0x55a2e6e8a3a0 2024-03-21T11:38:00.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.438+0000 7f808a800640 1 --2- 172.21.15.47:0/1870493947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6ea2880 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8084011af0 tx=0x7f8084004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3180ea10c70b11fe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.438+0000 7f806be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:38:00.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.438+0000 7f806be00640 1 -- 172.21.15.47:0/1870493947 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808401d830 con 0x55a2e6e8a3a0 2024-03-21T11:38:00.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.439+0000 7f806be00640 1 -- 172.21.15.47:0/1870493947 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f808401d9d0 con 0x55a2e6e8a3a0 2024-03-21T11:38:00.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.440+0000 7f8093db4740 1 -- 172.21.15.47:0/1870493947 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f807c00be10 con 0x55a2e6e8a3a0 2024-03-21T11:38:00.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.440+0000 7f806be00640 1 -- 172.21.15.47:0/1870493947 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f808401dbd0 con 0x55a2e6e8a3a0 2024-03-21T11:38:00.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.441+0000 7f8093db4740 10 client.30400 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:00.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.441+0000 7f8093db4740 1 -- 172.21.15.47:0/1870493947 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a2e6bcb7d0 con 0x55a2e6e8a3a0 2024-03-21T11:38:00.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.442+0000 7f806be00640 1 -- 172.21.15.47:0/1870493947 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f807c00be10 con 0x55a2e6e8a3a0 2024-03-21T11:38:00.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.443+0000 7f806be00640 1 -- 172.21.15.47:0/1870493947 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f8084033420 con 0x55a2e6e8a3a0 2024-03-21T11:38:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.443+0000 7f8093db4740 10 client.30400 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.443+0000 7f8093db4740 10 client.30400 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:38:00.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.443+0000 7f8093db4740 20 client.30400 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.444+0000 7f8093db4740 1 --2- 172.21.15.47:0/1870493947 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55a2e6ea72f0 0x55a2e6ea9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.444+0000 7f8093db4740 4 client.30400 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.444+0000 7f8093db4740 1 -- 172.21.15.47:0/1870493947 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2e6dfa800 con 0x55a2e6ea72f0 2024-03-21T11:38:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.444+0000 7f8089e00640 1 --2- 172.21.15.47:0/1870493947 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55a2e6ea72f0 0x55a2e6ea9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.445+0000 7f806be00640 10 client.30400 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:38:00.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.445+0000 7f8089e00640 1 --2- 172.21.15.47:0/1870493947 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55a2e6ea72f0 0x55a2e6ea9770 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=646b4f2a70895109 server_cookie=d6378914d736cb4b in_seq=0 out_seq=0 2024-03-21T11:38:00.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.446+0000 7f806be00640 1 -- 172.21.15.47:0/1870493947 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a2e6dfa800 con 0x55a2e6ea72f0 2024-03-21T11:38:00.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.446+0000 7f806be00640 10 client.30400 handle_command_reply: tid=0 2024-03-21T11:38:00.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.446+0000 7f8093db4740 1 client.30400 shutdown 2024-03-21T11:38:00.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.446+0000 7f8093db4740 1 -- 172.21.15.47:0/1870493947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e8a3a0 msgr2=0x55a2e6ea2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.447+0000 7f8093db4740 1 --2- 172.21.15.47:0/1870493947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6ea2880 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8084011af0 tx=0x7f8084004a40 comp rx=0 tx=0).stop 2024-03-21T11:38:00.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.447+0000 7f8093db4740 1 -- 172.21.15.47:0/1870493947 shutdown_connections 2024-03-21T11:38:00.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.447+0000 7f8093db4740 1 --2- 172.21.15.47:0/1870493947 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55a2e6ea72f0 0x55a2e6ea9770 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.447+0000 7f8093db4740 1 --2- 172.21.15.47:0/1870493947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6ea2880 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.447+0000 7f8093db4740 1 --2- 172.21.15.47:0/1870493947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e899e0 0x55a2e6e9b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.447+0000 7f8093db4740 1 --2- 172.21.15.47:0/1870493947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e88810 0x55a2e6e9b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.447+0000 7f8093db4740 1 -- 172.21.15.47:0/1870493947 >> 172.21.15.47:0/1870493947 conn(0x55a2e6e83c00 msgr2=0x55a2e6e82eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.447+0000 7f806be00640 0 client.30400 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:38:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.448+0000 7f8093db4740 1 -- 172.21.15.47:0/1870493947 shutdown_connections 2024-03-21T11:38:00.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.449+0000 7f8093db4740 1 -- 172.21.15.47:0/1870493947 wait complete. 2024-03-21T11:38:00.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.449+0000 7f8093db4740 20 client.30400 trim_cache size 0 max 16384 2024-03-21T11:38:00.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.472+0000 7f8093db4740 1 Processor -- start 2024-03-21T11:38:00.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.472+0000 7f8093db4740 1 -- start start 2024-03-21T11:38:00.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.473+0000 7f8093db4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e88810 0x55a2e6e82620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.473+0000 7f8093db4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e899e0 0x55a2e6e82b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.473+0000 7f8093db4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6e830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.473+0000 7f8093db4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c1b0720 con 0x55a2e6e899e0 2024-03-21T11:38:00.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.473+0000 7f8093db4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c1b2220 con 0x55a2e6e8a3a0 2024-03-21T11:38:00.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.473+0000 7f8093db4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c1b0d80 con 0x55a2e6e88810 2024-03-21T11:38:00.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.473+0000 7f8089400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e899e0 0x55a2e6e82b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.473+0000 7f8089e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e88810 0x55a2e6e82620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.473+0000 7f808a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6e830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.473+0000 7f8089400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e899e0 0x55a2e6e82b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38070/0 (socket says 172.21.15.47:38070) 2024-03-21T11:38:00.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.473+0000 7f8089e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e88810 0x55a2e6e82620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44086/0 (socket says 172.21.15.47:44086) 2024-03-21T11:38:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.473+0000 7f8089400640 1 -- 172.21.15.47:0/327952051 learned_addr learned my addr 172.21.15.47:0/327952051 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.474+0000 7f8089e00640 1 -- 172.21.15.47:0/327952051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e8a3a0 msgr2=0x55a2e6e830a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.474+0000 7f8089e00640 1 --2- 172.21.15.47:0/327952051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6e830a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.474+0000 7f8089e00640 1 -- 172.21.15.47:0/327952051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e899e0 msgr2=0x55a2e6e82b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.474+0000 7f8089e00640 1 --2- 172.21.15.47:0/327952051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e899e0 0x55a2e6e82b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.474+0000 7f8089e00640 1 -- 172.21.15.47:0/327952051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2e6ea6db0 con 0x55a2e6e88810 2024-03-21T11:38:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.474+0000 7f808a800640 1 --2- 172.21.15.47:0/327952051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6e830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.474+0000 7f8089400640 1 --2- 172.21.15.47:0/327952051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e899e0 0x55a2e6e82b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.474+0000 7f8089e00640 1 --2- 172.21.15.47:0/327952051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e88810 0x55a2e6e82620 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f807c000c00 tx=0x7f807c00ff80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=324d0723f4ec334d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.474+0000 7f8076a00640 1 -- 172.21.15.47:0/327952051 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c015040 con 0x55a2e6e88810 2024-03-21T11:38:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.474+0000 7f8076a00640 1 -- 172.21.15.47:0/327952051 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f807c00b920 con 0x55a2e6e88810 2024-03-21T11:38:00.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.474+0000 7f8076a00640 1 -- 172.21.15.47:0/327952051 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0055a0 con 0x55a2e6e88810 2024-03-21T11:38:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.475+0000 7f8093db4740 1 -- 172.21.15.47:0/327952051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e88810 msgr2=0x55a2e6e82620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.475+0000 7f8093db4740 1 --2- 172.21.15.47:0/327952051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e88810 0x55a2e6e82620 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f807c000c00 tx=0x7f807c00ff80 comp rx=0 tx=0).stop 2024-03-21T11:38:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.475+0000 7f8093db4740 1 -- 172.21.15.47:0/327952051 shutdown_connections 2024-03-21T11:38:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.475+0000 7f8093db4740 1 --2- 172.21.15.47:0/327952051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6e830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.475+0000 7f8093db4740 1 --2- 172.21.15.47:0/327952051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e899e0 0x55a2e6e82b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.475+0000 7f8093db4740 1 --2- 172.21.15.47:0/327952051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e88810 0x55a2e6e82620 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.475+0000 7f8093db4740 1 -- 172.21.15.47:0/327952051 >> 172.21.15.47:0/327952051 conn(0x55a2e6e83c00 msgr2=0x55a2e6e83fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.475+0000 7f8093db4740 1 -- 172.21.15.47:0/327952051 shutdown_connections 2024-03-21T11:38:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.475+0000 7f8093db4740 1 -- 172.21.15.47:0/327952051 wait complete. 2024-03-21T11:38:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.476+0000 7f8093db4740 1 -- start start 2024-03-21T11:38:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.476+0000 7f8093db4740 1 Processor -- start 2024-03-21T11:38:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.476+0000 7f8093db4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e88810 0x55a2e6eac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.476+0000 7f8093db4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e899e0 0x55a2e6eace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.476+0000 7f8093db4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6eade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.476+0000 7f8093db4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c1b1b00 con 0x55a2e6e8a3a0 2024-03-21T11:38:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.476+0000 7f8093db4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c1b14a0 con 0x55a2e6e88810 2024-03-21T11:38:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.476+0000 7f8093db4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c1ae900 con 0x55a2e6e899e0 2024-03-21T11:38:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.477+0000 7f8093db4740 10 client.0 _ll_register_callbacks cb 0x7f8084011590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.477+0000 7f8089e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e88810 0x55a2e6eac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.477+0000 7f8089e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e88810 0x55a2e6eac920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48910/0 (socket says 172.21.15.47:48910) 2024-03-21T11:38:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.477+0000 7f8089e00640 1 -- 172.21.15.47:0/821091152 learned_addr learned my addr 172.21.15.47:0/821091152 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.477+0000 7f808a800640 1 --2- 172.21.15.47:0/821091152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6eade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.477+0000 7f8089400640 1 --2- 172.21.15.47:0/821091152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e899e0 0x55a2e6eace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.477+0000 7f8089e00640 1 -- 172.21.15.47:0/821091152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e899e0 msgr2=0x55a2e6eace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.477+0000 7f8089e00640 1 --2- 172.21.15.47:0/821091152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e899e0 0x55a2e6eace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.477+0000 7f8089e00640 1 -- 172.21.15.47:0/821091152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e8a3a0 msgr2=0x55a2e6eade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.477+0000 7f8089e00640 1 --2- 172.21.15.47:0/821091152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6eade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.477+0000 7f8089e00640 1 -- 172.21.15.47:0/821091152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2e6eae460 con 0x55a2e6e88810 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.478+0000 7f8089e00640 1 --2- 172.21.15.47:0/821091152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e88810 0x55a2e6eac920 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f807c00bc70 tx=0x7f807c00be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e624a2074584293a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.478+0000 7f806be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.478+0000 7f806be00640 1 -- 172.21.15.47:0/821091152 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c015030 con 0x55a2e6e88810 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.478+0000 7f806be00640 1 -- 172.21.15.47:0/821091152 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f807c028120 con 0x55a2e6e88810 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.478+0000 7f8093db4740 1 -- 172.21.15.47:0/821091152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f807c00b920 con 0x55a2e6e88810 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.478+0000 7f806be00640 1 -- 172.21.15.47:0/821091152 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f807c005320 con 0x55a2e6e88810 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.478+0000 7f8093db4740 10 client.30392 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.478+0000 7f8093db4740 1 -- 172.21.15.47:0/821091152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a2e6f02ea0 con 0x55a2e6e88810 2024-03-21T11:38:00.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.479+0000 7f806be00640 1 -- 172.21.15.47:0/821091152 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f807c00b920 con 0x55a2e6e88810 2024-03-21T11:38:00.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.479+0000 7f806be00640 1 -- 172.21.15.47:0/821091152 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f807c028be0 con 0x55a2e6e88810 2024-03-21T11:38:00.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.479+0000 7f8093db4740 10 client.30392 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:00.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.479+0000 7f8093db4740 10 client.30392 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:38:00.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.479+0000 7f8093db4740 20 client.30392 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:00.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.479+0000 7f8093db4740 1 --2- 172.21.15.47:0/821091152 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55a2e6f03640 0x55a2e6f05a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.479+0000 7f8093db4740 4 client.30392 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:38:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.479+0000 7f8093db4740 1 -- 172.21.15.47:0/821091152 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a2e6df4b50 con 0x55a2e6f03640 2024-03-21T11:38:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.480+0000 7f8089400640 1 --2- 172.21.15.47:0/821091152 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55a2e6f03640 0x55a2e6f05a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.480+0000 7f8089400640 1 --2- 172.21.15.47:0/821091152 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55a2e6f03640 0x55a2e6f05a60 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=929da6ca8dee37f8 server_cookie=48a3ba8847c8d46b in_seq=0 out_seq=0 2024-03-21T11:38:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.480+0000 7f806be00640 10 client.30392 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:38:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f806be00640 1 -- 172.21.15.47:0/821091152 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a2e6df4b50 con 0x55a2e6f03640 2024-03-21T11:38:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f806be00640 10 client.30392 handle_command_reply: tid=0 2024-03-21T11:38:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f8093db4740 1 client.30392 shutdown 2024-03-21T11:38:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f8093db4740 1 -- 172.21.15.47:0/821091152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e88810 msgr2=0x55a2e6eac920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f8093db4740 1 --2- 172.21.15.47:0/821091152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e88810 0x55a2e6eac920 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f807c00bc70 tx=0x7f807c00be10 comp rx=0 tx=0).stop 2024-03-21T11:38:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f8093db4740 1 -- 172.21.15.47:0/821091152 shutdown_connections 2024-03-21T11:38:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f8093db4740 1 --2- 172.21.15.47:0/821091152 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55a2e6f03640 0x55a2e6f05a60 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f8093db4740 1 --2- 172.21.15.47:0/821091152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2e6e8a3a0 0x55a2e6eade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f8093db4740 1 --2- 172.21.15.47:0/821091152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2e6e899e0 0x55a2e6eace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f8093db4740 1 --2- 172.21.15.47:0/821091152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2e6e88810 0x55a2e6eac920 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f806be00640 0 client.30392 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:38:00.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f8093db4740 1 -- 172.21.15.47:0/821091152 >> 172.21.15.47:0/821091152 conn(0x55a2e6e83c00 msgr2=0x55a2e6e82620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f8093db4740 1 -- 172.21.15.47:0/821091152 shutdown_connections 2024-03-21T11:38:00.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f8093db4740 1 -- 172.21.15.47:0/821091152 wait complete. 2024-03-21T11:38:00.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.481+0000 7f8093db4740 20 client.30392 trim_cache size 0 max 16384 2024-03-21T11:38:00.485 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:38:00.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.482+0000 7f8091000640 1 -- 172.21.15.47:0/910209698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80700a2fa0 msgr2=0x7f80700a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.482+0000 7f8091000640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80700a2fa0 0x7f80700a5460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x55a2e6e898e0 tx=0x7f8084004ab0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.482+0000 7f8091000640 1 -- 172.21.15.47:0/910209698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f808c10d2a0 msgr2=0x7f808c1a2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.482+0000 7f8091000640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f808c10d2a0 0x7f808c1a2920 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f80780349e0 tx=0x7f80780042f0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.483+0000 7f8091000640 1 -- 172.21.15.47:0/910209698 shutdown_connections 2024-03-21T11:38:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.483+0000 7f8091000640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80700a2fa0 0x7f80700a5460 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.483+0000 7f8091000640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f808c10dc60 0x7f808c1a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.483+0000 7f8091000640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f808c10d2a0 0x7f808c1a2920 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.483+0000 7f8091000640 1 --2- 172.21.15.47:0/910209698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f808c10c0b0 0x7f808c1a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.483+0000 7f8091000640 1 -- 172.21.15.47:0/910209698 >> 172.21.15.47:0/910209698 conn(0x7f808c06c060 msgr2=0x7f808c110670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.483+0000 7f8091000640 1 -- 172.21.15.47:0/910209698 shutdown_connections 2024-03-21T11:38:00.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.483+0000 7f8091000640 1 -- 172.21.15.47:0/910209698 wait complete. 2024-03-21T11:38:00.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:00 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1517651050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:38:00.503 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:38:00.504 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:38:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.744+0000 7f7979e00640 1 -- 172.21.15.47:0/2716004173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7974105110 msgr2=0x7f7974105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.744+0000 7f7979e00640 1 --2- 172.21.15.47:0/2716004173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7974105110 0x7f7974105590 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f796c009be0 tx=0x7f796c02bf10 comp rx=0 tx=0).stop 2024-03-21T11:38:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.745+0000 7f7979e00640 1 -- 172.21.15.47:0/2716004173 shutdown_connections 2024-03-21T11:38:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.745+0000 7f7979e00640 1 --2- 172.21.15.47:0/2716004173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7974105110 0x7f7974105590 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.745+0000 7f7979e00640 1 --2- 172.21.15.47:0/2716004173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7974102760 0x7f7974104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.745+0000 7f7979e00640 1 --2- 172.21.15.47:0/2716004173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79740ffdf0 0x7f7974102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.745+0000 7f7979e00640 1 -- 172.21.15.47:0/2716004173 >> 172.21.15.47:0/2716004173 conn(0x7f79740fba10 msgr2=0x7f79740fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.745+0000 7f7979e00640 1 -- 172.21.15.47:0/2716004173 shutdown_connections 2024-03-21T11:38:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.745+0000 7f7979e00640 1 -- 172.21.15.47:0/2716004173 wait complete. 2024-03-21T11:38:00.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.746+0000 7f7979e00640 1 Processor -- start 2024-03-21T11:38:00.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.746+0000 7f7979e00640 1 -- start start 2024-03-21T11:38:00.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.747+0000 7f7979e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79740ffdf0 0x7f797419dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.747+0000 7f7979e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7974102760 0x7f797419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.747+0000 7f7979e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7974105110 0x7f79741a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.747+0000 7f7978a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79740ffdf0 0x7f797419dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.747+0000 7f7978a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79740ffdf0 0x7f797419dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48932/0 (socket says 172.21.15.47:48932) 2024-03-21T11:38:00.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.748+0000 7f7979e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7974113820 con 0x7f7974102760 2024-03-21T11:38:00.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.748+0000 7f7979e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79741136a0 con 0x7f79740ffdf0 2024-03-21T11:38:00.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.748+0000 7f7979e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79741139a0 con 0x7f7974105110 2024-03-21T11:38:00.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.748+0000 7f7978a00640 1 -- 172.21.15.47:0/3088590450 learned_addr learned my addr 172.21.15.47:0/3088590450 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.748+0000 7f7973e00640 1 --2- 172.21.15.47:0/3088590450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7974102760 0x7f797419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.748+0000 7f7979400640 1 --2- 172.21.15.47:0/3088590450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7974105110 0x7f79741a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.748+0000 7f7978a00640 1 -- 172.21.15.47:0/3088590450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7974105110 msgr2=0x7f79741a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.748+0000 7f7978a00640 1 --2- 172.21.15.47:0/3088590450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7974105110 0x7f79741a5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.748+0000 7f7978a00640 1 -- 172.21.15.47:0/3088590450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7974102760 msgr2=0x7f797419e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.748+0000 7f7978a00640 1 --2- 172.21.15.47:0/3088590450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7974102760 0x7f797419e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.748+0000 7f7978a00640 1 -- 172.21.15.47:0/3088590450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79741a5a00 con 0x7f79740ffdf0 2024-03-21T11:38:00.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.749+0000 7f7978a00640 1 --2- 172.21.15.47:0/3088590450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79740ffdf0 0x7f797419dde0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f796800e9c0 tx=0x7f796800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.749+0000 7f7971600640 1 -- 172.21.15.47:0/3088590450 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796800ccc0 con 0x7f79740ffdf0 2024-03-21T11:38:00.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.749+0000 7f7979e00640 1 -- 172.21.15.47:0/3088590450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79741a5cf0 con 0x7f79740ffdf0 2024-03-21T11:38:00.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.749+0000 7f7971600640 1 -- 172.21.15.47:0/3088590450 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7968005500 con 0x7f79740ffdf0 2024-03-21T11:38:00.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.749+0000 7f7971600640 1 -- 172.21.15.47:0/3088590450 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968010430 con 0x7f79740ffdf0 2024-03-21T11:38:00.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.749+0000 7f7979e00640 1 -- 172.21.15.47:0/3088590450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79741a61d0 con 0x7f79740ffdf0 2024-03-21T11:38:00.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.754+0000 7f797dfff740 1 Processor -- start 2024-03-21T11:38:00.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.755+0000 7f797dfff740 1 -- start start 2024-03-21T11:38:00.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.755+0000 7f797dfff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da61e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.755+0000 7f797dfff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da62bc0 0x56550da63040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.755+0000 7f797dfff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da63580 0x56550da69e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.755+0000 7f797dfff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56550d9c62c0 con 0x56550da619f0 2024-03-21T11:38:00.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.755+0000 7f797dfff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56550d96ec90 con 0x56550da63580 2024-03-21T11:38:00.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.755+0000 7f797dfff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56550d9c2360 con 0x56550da62bc0 2024-03-21T11:38:00.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.756+0000 7f7973e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da63580 0x56550da69e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.756+0000 7f7973e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da63580 0x56550da69e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48938/0 (socket says 172.21.15.47:48938) 2024-03-21T11:38:00.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.756+0000 7f7973e00640 1 -- 172.21.15.47:0/303931808 learned_addr learned my addr 172.21.15.47:0/303931808 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.756+0000 7f7979400640 1 --2- 172.21.15.47:0/303931808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da61e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.757+0000 7f7971600640 1 -- 172.21.15.47:0/3088590450 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f79680046c0 con 0x7f79740ffdf0 2024-03-21T11:38:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.757+0000 7f7979400640 1 -- 172.21.15.47:0/303931808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da62bc0 msgr2=0x56550da63040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:38:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.757+0000 7f7979400640 1 --2- 172.21.15.47:0/303931808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da62bc0 0x56550da63040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.757+0000 7f7979400640 1 -- 172.21.15.47:0/303931808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da63580 msgr2=0x56550da69e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.757+0000 7f7979400640 1 --2- 172.21.15.47:0/303931808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da63580 0x56550da69e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.757+0000 7f7979400640 1 -- 172.21.15.47:0/303931808 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56550d993080 con 0x56550da619f0 2024-03-21T11:38:00.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.758+0000 7f7979400640 1 --2- 172.21.15.47:0/303931808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da61e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f796c02c450 tx=0x7f796c005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c16d3c58d2f6ca server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.758+0000 7f7965c00640 1 -- 172.21.15.47:0/303931808 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c044070 con 0x56550da619f0 2024-03-21T11:38:00.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.759+0000 7f7965c00640 1 -- 172.21.15.47:0/303931808 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f796c02c8b0 con 0x56550da619f0 2024-03-21T11:38:00.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.759+0000 7f7965c00640 1 -- 172.21.15.47:0/303931808 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c003bd0 con 0x56550da619f0 2024-03-21T11:38:00.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.760+0000 7f797dfff740 1 -- 172.21.15.47:0/303931808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 msgr2=0x56550da61e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.760+0000 7f797dfff740 1 --2- 172.21.15.47:0/303931808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da61e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f796c02c450 tx=0x7f796c005ec0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.760+0000 7f797dfff740 1 -- 172.21.15.47:0/303931808 shutdown_connections 2024-03-21T11:38:00.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.760+0000 7f797dfff740 1 --2- 172.21.15.47:0/303931808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da63580 0x56550da69e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.760+0000 7f797dfff740 1 --2- 172.21.15.47:0/303931808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da62bc0 0x56550da63040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.760+0000 7f797dfff740 1 --2- 172.21.15.47:0/303931808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da61e10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.760+0000 7f797dfff740 1 -- 172.21.15.47:0/303931808 >> 172.21.15.47:0/303931808 conn(0x56550da5cde0 msgr2=0x56550da5d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.760+0000 7f7971600640 1 --2- 172.21.15.47:0/3088590450 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f795c0a2fa0 0x7f795c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.760+0000 7f797dfff740 1 -- 172.21.15.47:0/303931808 shutdown_connections 2024-03-21T11:38:00.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.760+0000 7f7971600640 1 -- 172.21.15.47:0/3088590450 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f79680cfbf0 con 0x7f79740ffdf0 2024-03-21T11:38:00.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.761+0000 7f797dfff740 1 -- 172.21.15.47:0/303931808 wait complete. 2024-03-21T11:38:00.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.761+0000 7f7973e00640 1 --2- 172.21.15.47:0/3088590450 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f795c0a2fa0 0x7f795c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.762+0000 7f797dfff740 1 -- start start 2024-03-21T11:38:00.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.762+0000 7f797dfff740 1 Processor -- start 2024-03-21T11:38:00.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.762+0000 7f7973e00640 1 --2- 172.21.15.47:0/3088590450 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f795c0a2fa0 0x7f795c0a5460 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x56550da5dd50 tx=0x7f7960005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.763+0000 7f797dfff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da6bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.763+0000 7f797dfff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da62bc0 0x56550da6c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.763+0000 7f797dfff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da63580 0x56550da731b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.763+0000 7f797dfff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56550da6a530 con 0x56550da619f0 2024-03-21T11:38:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.763+0000 7f797dfff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56550d9820d0 con 0x56550da62bc0 2024-03-21T11:38:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.763+0000 7f7973e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da62bc0 0x56550da6c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.763+0000 7f7973e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da62bc0 0x56550da6c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48952/0 (socket says 172.21.15.47:48952) 2024-03-21T11:38:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.763+0000 7f7973e00640 1 -- 172.21.15.47:0/2428126404 learned_addr learned my addr 172.21.15.47:0/2428126404 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.763+0000 7f797dfff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56550da6a6b0 con 0x56550da63580 2024-03-21T11:38:00.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.763+0000 7f797dfff740 10 client.0 _ll_register_callbacks cb 0x56550d986e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:00.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.764+0000 7f7973e00640 1 -- 172.21.15.47:0/2428126404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da63580 msgr2=0x56550da731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.764+0000 7f7979400640 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da63580 0x56550da731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.764+0000 7f7978a00640 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da6bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.764+0000 7f7973e00640 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da63580 0x56550da731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.764+0000 7f7973e00640 1 -- 172.21.15.47:0/2428126404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 msgr2=0x56550da6bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.764+0000 7f7973e00640 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da6bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.764+0000 7f7973e00640 1 -- 172.21.15.47:0/2428126404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56550d6da3f0 con 0x56550da62bc0 2024-03-21T11:38:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.764+0000 7f7978a00640 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da6bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.764+0000 7f7973e00640 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da62bc0 0x56550da6c170 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f796001e450 tx=0x7f79600046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73dc15879ca44ec1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.764+0000 7f795a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.764+0000 7f7979400640 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da63580 0x56550da731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.765+0000 7f795a000640 1 -- 172.21.15.47:0/2428126404 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796002b060 con 0x56550da62bc0 2024-03-21T11:38:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.765+0000 7f795a000640 1 -- 172.21.15.47:0/2428126404 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f796001c730 con 0x56550da62bc0 2024-03-21T11:38:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.765+0000 7f797dfff740 1 -- 172.21.15.47:0/2428126404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f796c02c8b0 con 0x56550da62bc0 2024-03-21T11:38:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.766+0000 7f795a000640 1 -- 172.21.15.47:0/2428126404 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f796001c930 con 0x56550da62bc0 2024-03-21T11:38:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.766+0000 7f797dfff740 10 client.30404 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.766+0000 7f797dfff740 1 -- 172.21.15.47:0/2428126404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56550d7a47d0 con 0x56550da62bc0 2024-03-21T11:38:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.766+0000 7f795a000640 1 -- 172.21.15.47:0/2428126404 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f796c02c8b0 con 0x56550da62bc0 2024-03-21T11:38:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.766+0000 7f795a000640 1 -- 172.21.15.47:0/2428126404 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f796001a040 con 0x56550da62bc0 2024-03-21T11:38:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.766+0000 7f797dfff740 10 client.30404 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.766+0000 7f797dfff740 10 client.30404 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:38:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.767+0000 7f797dfff740 20 client.30404 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.767+0000 7f797dfff740 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56550da776a0 0x56550da79b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.767+0000 7f797dfff740 4 client.30404 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.767+0000 7f797dfff740 1 -- 172.21.15.47:0/2428126404 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56550d9d3800 con 0x56550da776a0 2024-03-21T11:38:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.767+0000 7f7978a00640 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56550da776a0 0x56550da79b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.768+0000 7f7978a00640 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56550da776a0 0x56550da79b00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb542e21b9f3892c server_cookie=f58f18adb501fa51 in_seq=0 out_seq=0 2024-03-21T11:38:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.768+0000 7f795a000640 10 client.30404 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:38:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.770+0000 7f795a000640 1 -- 172.21.15.47:0/2428126404 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56550d9d3800 con 0x56550da776a0 2024-03-21T11:38:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.770+0000 7f795a000640 10 client.30404 handle_command_reply: tid=0 2024-03-21T11:38:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.770+0000 7f797dfff740 1 client.30404 shutdown 2024-03-21T11:38:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.770+0000 7f797dfff740 1 -- 172.21.15.47:0/2428126404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da62bc0 msgr2=0x56550da6c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.770+0000 7f797dfff740 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da62bc0 0x56550da6c170 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f796001e450 tx=0x7f79600046c0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.771+0000 7f797dfff740 1 -- 172.21.15.47:0/2428126404 shutdown_connections 2024-03-21T11:38:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.771+0000 7f797dfff740 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56550da776a0 0x56550da79b00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.771+0000 7f797dfff740 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da63580 0x56550da731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.771+0000 7f797dfff740 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da62bc0 0x56550da6c170 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.771+0000 7f797dfff740 1 --2- 172.21.15.47:0/2428126404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da6bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.771+0000 7f797dfff740 1 -- 172.21.15.47:0/2428126404 >> 172.21.15.47:0/2428126404 conn(0x56550da5cde0 msgr2=0x56550da5c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.771+0000 7f795a000640 0 client.30404 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:38:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.772+0000 7f797dfff740 1 -- 172.21.15.47:0/2428126404 shutdown_connections 2024-03-21T11:38:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.772+0000 7f797dfff740 1 -- 172.21.15.47:0/2428126404 wait complete. 2024-03-21T11:38:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.773+0000 7f797dfff740 20 client.30404 trim_cache size 0 max 16384 2024-03-21T11:38:00.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.793+0000 7f797dfff740 1 Processor -- start 2024-03-21T11:38:00.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.793+0000 7f797dfff740 1 -- start start 2024-03-21T11:38:00.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.793+0000 7f797dfff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da619f0 0x56550da57ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.793+0000 7f797dfff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da62bc0 0x56550da73170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.793+0000 7f797dfff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da63580 0x56550da65c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.793+0000 7f797dfff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79741b0510 con 0x56550da62bc0 2024-03-21T11:38:00.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.793+0000 7f797dfff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79741b2010 con 0x56550da619f0 2024-03-21T11:38:00.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.793+0000 7f797dfff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79741b0b70 con 0x56550da63580 2024-03-21T11:38:00.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.793+0000 7f7978a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da619f0 0x56550da57ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.793+0000 7f7973e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da62bc0 0x56550da73170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.794+0000 7f7979400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da63580 0x56550da65c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.794+0000 7f7978a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da619f0 0x56550da57ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48960/0 (socket says 172.21.15.47:48960) 2024-03-21T11:38:00.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.794+0000 7f7978a00640 1 -- 172.21.15.47:0/2820669891 learned_addr learned my addr 172.21.15.47:0/2820669891 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.794+0000 7f7978a00640 1 -- 172.21.15.47:0/2820669891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da63580 msgr2=0x56550da65c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.794+0000 7f7978a00640 1 --2- 172.21.15.47:0/2820669891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da63580 0x56550da65c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.794+0000 7f7978a00640 1 -- 172.21.15.47:0/2820669891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da62bc0 msgr2=0x56550da73170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.794+0000 7f7978a00640 1 --2- 172.21.15.47:0/2820669891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da62bc0 0x56550da73170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.794+0000 7f7978a00640 1 -- 172.21.15.47:0/2820669891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56550da5d850 con 0x56550da619f0 2024-03-21T11:38:00.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.794+0000 7f7973e00640 1 --2- 172.21.15.47:0/2820669891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da62bc0 0x56550da73170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:00.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.794+0000 7f7979400640 1 --2- 172.21.15.47:0/2820669891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da63580 0x56550da65c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:00.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.794+0000 7f7978a00640 1 --2- 172.21.15.47:0/2820669891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da619f0 0x56550da57ff0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f796800c6a0 tx=0x7f796809bd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e4ee1eda06b8022 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.795+0000 7f7965200640 1 -- 172.21.15.47:0/2820669891 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796809c630 con 0x56550da619f0 2024-03-21T11:38:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.795+0000 7f7965200640 1 -- 172.21.15.47:0/2820669891 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f796809bf30 con 0x56550da619f0 2024-03-21T11:38:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.795+0000 7f7965200640 1 -- 172.21.15.47:0/2820669891 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968016dd0 con 0x56550da619f0 2024-03-21T11:38:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.796+0000 7f797dfff740 1 -- 172.21.15.47:0/2820669891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da619f0 msgr2=0x56550da57ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.796+0000 7f797dfff740 1 --2- 172.21.15.47:0/2820669891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da619f0 0x56550da57ff0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f796800c6a0 tx=0x7f796809bd00 comp rx=0 tx=0).stop 2024-03-21T11:38:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.796+0000 7f797dfff740 1 -- 172.21.15.47:0/2820669891 shutdown_connections 2024-03-21T11:38:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.796+0000 7f797dfff740 1 --2- 172.21.15.47:0/2820669891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da63580 0x56550da65c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.796+0000 7f797dfff740 1 --2- 172.21.15.47:0/2820669891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da62bc0 0x56550da73170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.796+0000 7f797dfff740 1 --2- 172.21.15.47:0/2820669891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da619f0 0x56550da57ff0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.796+0000 7f797dfff740 1 -- 172.21.15.47:0/2820669891 >> 172.21.15.47:0/2820669891 conn(0x56550da5cde0 msgr2=0x56550da608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.796+0000 7f797dfff740 1 -- 172.21.15.47:0/2820669891 shutdown_connections 2024-03-21T11:38:00.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.796+0000 7f797dfff740 1 -- 172.21.15.47:0/2820669891 wait complete. 2024-03-21T11:38:00.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.797+0000 7f797dfff740 1 -- start start 2024-03-21T11:38:00.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.797+0000 7f797dfff740 1 Processor -- start 2024-03-21T11:38:00.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f797dfff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da72bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f797dfff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da62bc0 0x56550da730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f797dfff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da63580 0x56550da6f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f797dfff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79741b18f0 con 0x56550da619f0 2024-03-21T11:38:00.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f797dfff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79741b1290 con 0x56550da63580 2024-03-21T11:38:00.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f797dfff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79741ae6d0 con 0x56550da62bc0 2024-03-21T11:38:00.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f797dfff740 10 client.0 _ll_register_callbacks cb 0x7f7960005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:00.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f7973e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da62bc0 0x56550da730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f7978a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da72bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f7973e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da62bc0 0x56550da730f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44140/0 (socket says 172.21.15.47:44140) 2024-03-21T11:38:00.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f7973e00640 1 -- 172.21.15.47:0/2756668091 learned_addr learned my addr 172.21.15.47:0/2756668091 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:00.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f7973e00640 1 -- 172.21.15.47:0/2756668091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da63580 msgr2=0x56550da6f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f7973e00640 1 --2- 172.21.15.47:0/2756668091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da63580 0x56550da6f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f7973e00640 1 -- 172.21.15.47:0/2756668091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 msgr2=0x56550da72bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:00.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f7973e00640 1 --2- 172.21.15.47:0/2756668091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da72bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f7973e00640 1 -- 172.21.15.47:0/2756668091 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56550da6ff70 con 0x56550da62bc0 2024-03-21T11:38:00.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.798+0000 7f7978a00640 1 --2- 172.21.15.47:0/2756668091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da72bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:00.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.799+0000 7f7973e00640 1 --2- 172.21.15.47:0/2756668091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da62bc0 0x56550da730f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f796001ece0 tx=0x7f79600052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=973ff62af377901e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:00.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.799+0000 7f795a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:38:00.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.799+0000 7f795a000640 1 -- 172.21.15.47:0/2756668091 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960035080 con 0x56550da62bc0 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.799+0000 7f795a000640 1 -- 172.21.15.47:0/2756668091 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f796001b3c0 con 0x56550da62bc0 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.799+0000 7f797dfff740 1 -- 172.21.15.47:0/2756668091 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f796809bf30 con 0x56550da62bc0 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.800+0000 7f795a000640 1 -- 172.21.15.47:0/2756668091 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79600263e0 con 0x56550da62bc0 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.800+0000 7f797dfff740 10 client.30418 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.800+0000 7f797dfff740 1 -- 172.21.15.47:0/2756668091 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56550dad3e30 con 0x56550da62bc0 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.800+0000 7f795a000640 1 -- 172.21.15.47:0/2756668091 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f796809bf30 con 0x56550da62bc0 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.800+0000 7f795a000640 1 -- 172.21.15.47:0/2756668091 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f796001a040 con 0x56550da62bc0 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.800+0000 7f797dfff740 10 client.30418 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.800+0000 7f797dfff740 10 client.30418 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.800+0000 7f797dfff740 20 client.30418 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.800+0000 7f797dfff740 1 --2- 172.21.15.47:0/2756668091 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56550dad4820 0x56550dad6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.800+0000 7f797dfff740 4 client.30418 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.800+0000 7f797dfff740 1 -- 172.21.15.47:0/2756668091 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56550d9cdb50 con 0x56550dad4820 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.801+0000 7f7978a00640 1 --2- 172.21.15.47:0/2756668091 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56550dad4820 0x56550dad6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:00.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.801+0000 7f7978a00640 1 --2- 172.21.15.47:0/2756668091 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56550dad4820 0x56550dad6c40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=93eb5c65b2534ac2 server_cookie=7cc0bad9af55593a in_seq=0 out_seq=0 2024-03-21T11:38:00.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.801+0000 7f795a000640 10 client.30418 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:38:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.802+0000 7f795a000640 1 -- 172.21.15.47:0/2756668091 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56550d9cdb50 con 0x56550dad4820 2024-03-21T11:38:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.802+0000 7f795a000640 10 client.30418 handle_command_reply: tid=0 2024-03-21T11:38:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.802+0000 7f797dfff740 1 client.30418 shutdown 2024-03-21T11:38:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.803+0000 7f797dfff740 1 -- 172.21.15.47:0/2756668091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da62bc0 msgr2=0x56550da730f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.803+0000 7f797dfff740 1 --2- 172.21.15.47:0/2756668091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da62bc0 0x56550da730f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f796001ece0 tx=0x7f79600052f0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.803+0000 7f797dfff740 1 -- 172.21.15.47:0/2756668091 shutdown_connections 2024-03-21T11:38:00.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.803+0000 7f797dfff740 1 --2- 172.21.15.47:0/2756668091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550da63580 0x56550da6f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.803+0000 7f797dfff740 1 --2- 172.21.15.47:0/2756668091 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x56550dad4820 0x56550dad6c40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.803+0000 7f797dfff740 1 --2- 172.21.15.47:0/2756668091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56550da62bc0 0x56550da730f0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.803+0000 7f797dfff740 1 --2- 172.21.15.47:0/2756668091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56550da619f0 0x56550da72bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.803+0000 7f795a000640 0 client.30418 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:38:00.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.803+0000 7f797dfff740 1 -- 172.21.15.47:0/2756668091 >> 172.21.15.47:0/2756668091 conn(0x56550da5cde0 msgr2=0x56550da5b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.803+0000 7f797dfff740 1 -- 172.21.15.47:0/2756668091 shutdown_connections 2024-03-21T11:38:00.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.803+0000 7f797dfff740 1 -- 172.21.15.47:0/2756668091 wait complete. 2024-03-21T11:38:00.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.803+0000 7f797dfff740 20 client.30418 trim_cache size 0 max 16384 2024-03-21T11:38:00.809 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:38:00.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.804+0000 7f7979e00640 1 -- 172.21.15.47:0/3088590450 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f795c0a2fa0 msgr2=0x7f795c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.804+0000 7f7979e00640 1 --2- 172.21.15.47:0/3088590450 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f795c0a2fa0 0x7f795c0a5460 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x56550da5dd50 tx=0x7f7960005f90 comp rx=0 tx=0).stop 2024-03-21T11:38:00.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.804+0000 7f7979e00640 1 -- 172.21.15.47:0/3088590450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79740ffdf0 msgr2=0x7f797419dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:00.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.804+0000 7f7979e00640 1 --2- 172.21.15.47:0/3088590450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79740ffdf0 0x7f797419dde0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f796800e9c0 tx=0x7f796800ee80 comp rx=0 tx=0).stop 2024-03-21T11:38:00.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.804+0000 7f7979e00640 1 -- 172.21.15.47:0/3088590450 shutdown_connections 2024-03-21T11:38:00.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.804+0000 7f7979e00640 1 --2- 172.21.15.47:0/3088590450 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f795c0a2fa0 0x7f795c0a5460 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.804+0000 7f7979e00640 1 --2- 172.21.15.47:0/3088590450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7974105110 0x7f79741a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.804+0000 7f7979e00640 1 --2- 172.21.15.47:0/3088590450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7974102760 0x7f797419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.804+0000 7f7979e00640 1 --2- 172.21.15.47:0/3088590450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79740ffdf0 0x7f797419dde0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:00.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.804+0000 7f7979e00640 1 -- 172.21.15.47:0/3088590450 >> 172.21.15.47:0/3088590450 conn(0x7f79740fba10 msgr2=0x7f7974100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:00.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.805+0000 7f7979e00640 1 -- 172.21.15.47:0/3088590450 shutdown_connections 2024-03-21T11:38:00.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:00.805+0000 7f7979e00640 1 -- 172.21.15.47:0/3088590450 wait complete. 2024-03-21T11:38:00.825 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:38:00.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:38:00.922 INFO:tasks.workunit.client.0.smithi047.stdout: 10 23228 32.52 MB/sec warmup 44 sec latency 102.391 ms 2024-03-21T11:38:01.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.073+0000 7eff0f800640 1 -- 172.21.15.47:0/3501495704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff08104ad0 msgr2=0x7eff08104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.073+0000 7eff0f800640 1 --2- 172.21.15.47:0/3501495704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff08104ad0 0x7eff08104f50 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7efef4009900 tx=0x7efef402bc30 comp rx=0 tx=0).stop 2024-03-21T11:38:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.074+0000 7eff0f800640 1 -- 172.21.15.47:0/3501495704 shutdown_connections 2024-03-21T11:38:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.074+0000 7eff0f800640 1 --2- 172.21.15.47:0/3501495704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff08105490 0x7eff0810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.074+0000 7eff0f800640 1 --2- 172.21.15.47:0/3501495704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff08104ad0 0x7eff08104f50 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.074+0000 7eff0f800640 1 --2- 172.21.15.47:0/3501495704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff081038e0 0x7eff08103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.074+0000 7eff0f800640 1 -- 172.21.15.47:0/3501495704 >> 172.21.15.47:0/3501495704 conn(0x7eff080fee50 msgr2=0x7eff08101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:01.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.074+0000 7eff0f800640 1 -- 172.21.15.47:0/3501495704 shutdown_connections 2024-03-21T11:38:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.074+0000 7eff0f800640 1 -- 172.21.15.47:0/3501495704 wait complete. 2024-03-21T11:38:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.076+0000 7eff0f800640 1 Processor -- start 2024-03-21T11:38:01.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.076+0000 7eff0f800640 1 -- start start 2024-03-21T11:38:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.076+0000 7eff0f800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff081038e0 0x7eff0819dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.076+0000 7eff0f800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff08104ad0 0x7eff0819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.076+0000 7eff0f800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff08105490 0x7eff081a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.076+0000 7eff0f800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0810f540 con 0x7eff081038e0 2024-03-21T11:38:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.076+0000 7eff0f800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0810f3c0 con 0x7eff08104ad0 2024-03-21T11:38:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.076+0000 7eff0f800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0810f6c0 con 0x7eff08105490 2024-03-21T11:38:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.077+0000 7eff0ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff081038e0 0x7eff0819dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.077+0000 7eff0ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff081038e0 0x7eff0819dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38158/0 (socket says 172.21.15.47:38158) 2024-03-21T11:38:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.077+0000 7eff0ce00640 1 -- 172.21.15.47:0/2687285585 learned_addr learned my addr 172.21.15.47:0/2687285585 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.077+0000 7eff0d800640 1 --2- 172.21.15.47:0/2687285585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff08105490 0x7eff081a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.077+0000 7eff07e00640 1 --2- 172.21.15.47:0/2687285585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff08104ad0 0x7eff0819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.077+0000 7eff0ce00640 1 -- 172.21.15.47:0/2687285585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff08105490 msgr2=0x7eff081a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.077+0000 7eff0ce00640 1 --2- 172.21.15.47:0/2687285585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff08105490 0x7eff081a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.077+0000 7eff0ce00640 1 -- 172.21.15.47:0/2687285585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff08104ad0 msgr2=0x7eff0819e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.077+0000 7eff0ce00640 1 --2- 172.21.15.47:0/2687285585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff08104ad0 0x7eff0819e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.077+0000 7eff0ce00640 1 -- 172.21.15.47:0/2687285585 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff081a5c10 con 0x7eff081038e0 2024-03-21T11:38:01.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.078+0000 7eff0ce00640 1 --2- 172.21.15.47:0/2687285585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff081038e0 0x7eff0819dff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efefc00e9c0 tx=0x7efefc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.078+0000 7eff05600640 1 -- 172.21.15.47:0/2687285585 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc00cd80 con 0x7eff081038e0 2024-03-21T11:38:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.078+0000 7eff05600640 1 -- 172.21.15.47:0/2687285585 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efefc010910 con 0x7eff081038e0 2024-03-21T11:38:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.078+0000 7eff05600640 1 -- 172.21.15.47:0/2687285585 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc018950 con 0x7eff081038e0 2024-03-21T11:38:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.078+0000 7eff0f800640 1 -- 172.21.15.47:0/2687285585 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff081a5f00 con 0x7eff081038e0 2024-03-21T11:38:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.078+0000 7eff0f800640 1 -- 172.21.15.47:0/2687285585 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff081a6370 con 0x7eff081038e0 2024-03-21T11:38:01.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.082+0000 7eff05600640 1 -- 172.21.15.47:0/2687285585 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7efefc01f020 con 0x7eff081038e0 2024-03-21T11:38:01.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.082+0000 7eff123f1740 1 Processor -- start 2024-03-21T11:38:01.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.082+0000 7eff123f1740 1 -- start start 2024-03-21T11:38:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.082+0000 7eff123f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 0x55b8760edc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.082+0000 7eff123f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760eee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.082+0000 7eff123f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 0x55b8760f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.082+0000 7eff123f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8760522c0 con 0x55b8760ef3a0 2024-03-21T11:38:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.082+0000 7eff123f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b875ffac90 con 0x55b8760ee9e0 2024-03-21T11:38:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.082+0000 7eff123f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87604e360 con 0x55b8760ed810 2024-03-21T11:38:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.083+0000 7eff0ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.083+0000 7eff0d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 0x55b8760edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.083+0000 7eff0ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760eee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48980/0 (socket says 172.21.15.47:48980) 2024-03-21T11:38:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.083+0000 7eff0ce00640 1 -- 172.21.15.47:0/3252713363 learned_addr learned my addr 172.21.15.47:0/3252713363 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:01.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.083+0000 7eff07e00640 1 --2- 172.21.15.47:0/3252713363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 0x55b8760f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.083+0000 7eff0ce00640 1 -- 172.21.15.47:0/3252713363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 msgr2=0x55b8760edc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.083+0000 7eff0ce00640 1 --2- 172.21.15.47:0/3252713363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 0x55b8760edc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.083+0000 7eff0ce00640 1 -- 172.21.15.47:0/3252713363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 msgr2=0x55b8760f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.083+0000 7eff0ce00640 1 --2- 172.21.15.47:0/3252713363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 0x55b8760f5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.083+0000 7eff0ce00640 1 -- 172.21.15.47:0/3252713363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b87601f080 con 0x55b8760ee9e0 2024-03-21T11:38:01.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.084+0000 7eff0ce00640 1 --2- 172.21.15.47:0/3252713363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760eee60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efefc0d5c20 tx=0x7efefc0d5d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b90d09e729d91e6d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:01.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.084+0000 7eff05600640 1 --2- 172.21.15.47:0/2687285585 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efef00a2fa0 0x7efef00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.084+0000 7efefa000640 1 -- 172.21.15.47:0/3252713363 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc0de070 con 0x55b8760ee9e0 2024-03-21T11:38:01.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.084+0000 7eff0d800640 1 --2- 172.21.15.47:0/2687285585 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efef00a2fa0 0x7efef00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.084+0000 7efefa000640 1 -- 172.21.15.47:0/3252713363 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efefc09c1e0 con 0x55b8760ee9e0 2024-03-21T11:38:01.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.084+0000 7eff05600640 1 -- 172.21.15.47:0/2687285585 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efefc0cf7a0 con 0x7eff081038e0 2024-03-21T11:38:01.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.084+0000 7efefa000640 1 -- 172.21.15.47:0/3252713363 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc09b620 con 0x55b8760ee9e0 2024-03-21T11:38:01.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.086+0000 7eff123f1740 1 -- 172.21.15.47:0/3252713363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 msgr2=0x55b8760eee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.086+0000 7eff0d800640 1 --2- 172.21.15.47:0/2687285585 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efef00a2fa0 0x7efef00a5460 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x55b8760ee8e0 tx=0x7eff00009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.086+0000 7eff123f1740 1 --2- 172.21.15.47:0/3252713363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760eee60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efefc0d5c20 tx=0x7efefc0d5d10 comp rx=0 tx=0).stop 2024-03-21T11:38:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.086+0000 7eff123f1740 1 -- 172.21.15.47:0/3252713363 shutdown_connections 2024-03-21T11:38:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.086+0000 7eff123f1740 1 --2- 172.21.15.47:0/3252713363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 0x55b8760f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.086+0000 7eff123f1740 1 --2- 172.21.15.47:0/3252713363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760eee60 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.086+0000 7eff123f1740 1 --2- 172.21.15.47:0/3252713363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 0x55b8760edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.086+0000 7eff123f1740 1 -- 172.21.15.47:0/3252713363 >> 172.21.15.47:0/3252713363 conn(0x55b8760e8c00 msgr2=0x55b8760e9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.087+0000 7eff123f1740 1 -- 172.21.15.47:0/3252713363 shutdown_connections 2024-03-21T11:38:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.087+0000 7eff123f1740 1 -- 172.21.15.47:0/3252713363 wait complete. 2024-03-21T11:38:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.087+0000 7eff123f1740 1 -- start start 2024-03-21T11:38:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.088+0000 7eff123f1740 1 Processor -- start 2024-03-21T11:38:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.088+0000 7eff123f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 0x55b8760f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.088+0000 7eff123f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.088+0000 7eff123f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 0x55b8760fef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.088+0000 7eff123f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8760f63b0 con 0x55b8760ef3a0 2024-03-21T11:38:01.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.088+0000 7eff123f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87600e0d0 con 0x55b8760ee9e0 2024-03-21T11:38:01.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.088+0000 7eff123f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8760f6530 con 0x55b8760ed810 2024-03-21T11:38:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.089+0000 7eff123f1740 10 client.0 _ll_register_callbacks cb 0x55b876012e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.089+0000 7eff07e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.089+0000 7eff0ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 0x55b8760f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.089+0000 7eff07e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760f7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48984/0 (socket says 172.21.15.47:48984) 2024-03-21T11:38:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.089+0000 7eff07e00640 1 -- 172.21.15.47:0/1364261538 learned_addr learned my addr 172.21.15.47:0/1364261538 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.089+0000 7eff0d800640 1 --2- 172.21.15.47:0/1364261538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 0x55b8760fef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.089+0000 7eff07e00640 1 -- 172.21.15.47:0/1364261538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 msgr2=0x55b8760f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.089+0000 7eff07e00640 1 --2- 172.21.15.47:0/1364261538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 0x55b8760f7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.089+0000 7eff07e00640 1 -- 172.21.15.47:0/1364261538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 msgr2=0x55b8760fef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.089+0000 7eff07e00640 1 --2- 172.21.15.47:0/1364261538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 0x55b8760fef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.089+0000 7eff07e00640 1 -- 172.21.15.47:0/1364261538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b875d663f0 con 0x55b8760ee9e0 2024-03-21T11:38:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.089+0000 7eff0d800640 1 --2- 172.21.15.47:0/1364261538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 0x55b8760fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:38:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.090+0000 7eff07e00640 1 --2- 172.21.15.47:0/1364261538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760f7fa0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7efef4009900 tx=0x7efef4004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=672da4222e3089e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.090+0000 7efeeea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.090+0000 7efeeea00640 1 -- 172.21.15.47:0/1364261538 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4005d50 con 0x55b8760ee9e0 2024-03-21T11:38:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.090+0000 7efeeea00640 1 -- 172.21.15.47:0/1364261538 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efef4005230 con 0x55b8760ee9e0 2024-03-21T11:38:01.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.090+0000 7eff123f1740 1 -- 172.21.15.47:0/1364261538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efefc09c1e0 con 0x55b8760ee9e0 2024-03-21T11:38:01.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.091+0000 7efeeea00640 1 -- 172.21.15.47:0/1364261538 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7efef4005430 con 0x55b8760ee9e0 2024-03-21T11:38:01.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.091+0000 7eff123f1740 10 client.30419 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:01.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.091+0000 7eff123f1740 1 -- 172.21.15.47:0/1364261538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b875e307d0 con 0x55b8760ee9e0 2024-03-21T11:38:01.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.091+0000 7efeeea00640 1 -- 172.21.15.47:0/1364261538 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efefc09c1e0 con 0x55b8760ee9e0 2024-03-21T11:38:01.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.091+0000 7efeeea00640 1 -- 172.21.15.47:0/1364261538 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7efef4003820 con 0x55b8760ee9e0 2024-03-21T11:38:01.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.091+0000 7eff123f1740 10 client.30419 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:01.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.091+0000 7eff123f1740 10 client.30419 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:38:01.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.091+0000 7eff123f1740 20 client.30419 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:01.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.092+0000 7eff123f1740 1 --2- 172.21.15.47:0/1364261538 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b876103420 0x55b876105880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.092+0000 7eff123f1740 4 client.30419 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:01.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.092+0000 7eff123f1740 1 -- 172.21.15.47:0/1364261538 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b87605f800 con 0x55b876103420 2024-03-21T11:38:01.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.092+0000 7eff0ce00640 1 --2- 172.21.15.47:0/1364261538 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b876103420 0x55b876105880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.093+0000 7efeeea00640 10 client.30419 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:38:01.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.093+0000 7eff0ce00640 1 --2- 172.21.15.47:0/1364261538 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b876103420 0x55b876105880 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11e374981069cff0 server_cookie=a57731e0d2fd034f in_seq=0 out_seq=0 2024-03-21T11:38:01.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.094+0000 7efeeea00640 1 -- 172.21.15.47:0/1364261538 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b87605f800 con 0x55b876103420 2024-03-21T11:38:01.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.094+0000 7efeeea00640 10 client.30419 handle_command_reply: tid=0 2024-03-21T11:38:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.094+0000 7eff123f1740 1 client.30419 shutdown 2024-03-21T11:38:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.094+0000 7eff123f1740 1 -- 172.21.15.47:0/1364261538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 msgr2=0x55b8760f7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.094+0000 7eff123f1740 1 --2- 172.21.15.47:0/1364261538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760f7fa0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7efef4009900 tx=0x7efef4004250 comp rx=0 tx=0).stop 2024-03-21T11:38:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.094+0000 7eff123f1740 1 -- 172.21.15.47:0/1364261538 shutdown_connections 2024-03-21T11:38:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.094+0000 7eff123f1740 1 --2- 172.21.15.47:0/1364261538 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b876103420 0x55b876105880 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.094+0000 7eff123f1740 1 --2- 172.21.15.47:0/1364261538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 0x55b8760fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.094+0000 7eff123f1740 1 --2- 172.21.15.47:0/1364261538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760f7fa0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.094+0000 7efeeea00640 0 client.30419 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:38:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.094+0000 7eff123f1740 1 --2- 172.21.15.47:0/1364261538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 0x55b8760f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.094+0000 7eff123f1740 1 -- 172.21.15.47:0/1364261538 >> 172.21.15.47:0/1364261538 conn(0x55b8760e8c00 msgr2=0x55b8760e7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.094+0000 7eff123f1740 1 -- 172.21.15.47:0/1364261538 shutdown_connections 2024-03-21T11:38:01.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.095+0000 7eff123f1740 1 -- 172.21.15.47:0/1364261538 wait complete. 2024-03-21T11:38:01.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.095+0000 7eff123f1740 20 client.30419 trim_cache size 0 max 16384 2024-03-21T11:38:01.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.112+0000 7eff123f1740 1 Processor -- start 2024-03-21T11:38:01.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.112+0000 7eff123f1740 1 -- start start 2024-03-21T11:38:01.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.112+0000 7eff123f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ed810 0x55b8760e7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.112+0000 7eff123f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ee9e0 0x55b8760fef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.112+0000 7eff123f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ef3a0 0x55b8760e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.112+0000 7eff123f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff081b0790 con 0x55b8760ee9e0 2024-03-21T11:38:01.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.112+0000 7eff123f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff081b2290 con 0x55b8760ed810 2024-03-21T11:38:01.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.112+0000 7eff123f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff081b0df0 con 0x55b8760ef3a0 2024-03-21T11:38:01.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.113+0000 7eff07e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ee9e0 0x55b8760fef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.113+0000 7eff0d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ef3a0 0x55b8760e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.113+0000 7eff0d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ef3a0 0x55b8760e7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44190/0 (socket says 172.21.15.47:44190) 2024-03-21T11:38:01.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.113+0000 7eff0d800640 1 -- 172.21.15.47:0/955951560 learned_addr learned my addr 172.21.15.47:0/955951560 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:01.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.113+0000 7eff0ce00640 1 --2- 172.21.15.47:0/955951560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ed810 0x55b8760e7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.113+0000 7eff0d800640 1 -- 172.21.15.47:0/955951560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ed810 msgr2=0x55b8760e7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.113+0000 7eff0d800640 1 --2- 172.21.15.47:0/955951560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ed810 0x55b8760e7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.113+0000 7eff0d800640 1 -- 172.21.15.47:0/955951560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ee9e0 msgr2=0x55b8760fef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.113+0000 7eff0d800640 1 --2- 172.21.15.47:0/955951560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ee9e0 0x55b8760fef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.113+0000 7eff0d800640 1 -- 172.21.15.47:0/955951560 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8760ee460 con 0x55b8760ef3a0 2024-03-21T11:38:01.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.114+0000 7eff0d800640 1 --2- 172.21.15.47:0/955951560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ef3a0 0x55b8760e7b60 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7eff00003e70 tx=0x7eff00017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7eda15ca445b3ae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:01.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.114+0000 7efef9600640 1 -- 172.21.15.47:0/955951560 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0000f4e0 con 0x55b8760ef3a0 2024-03-21T11:38:01.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.114+0000 7efef9600640 1 -- 172.21.15.47:0/955951560 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff0000fb60 con 0x55b8760ef3a0 2024-03-21T11:38:01.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.114+0000 7efef9600640 1 -- 172.21.15.47:0/955951560 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0001eb50 con 0x55b8760ef3a0 2024-03-21T11:38:01.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.115+0000 7eff123f1740 1 -- 172.21.15.47:0/955951560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ef3a0 msgr2=0x55b8760e7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.115+0000 7eff123f1740 1 --2- 172.21.15.47:0/955951560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ef3a0 0x55b8760e7b60 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7eff00003e70 tx=0x7eff00017000 comp rx=0 tx=0).stop 2024-03-21T11:38:01.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.115+0000 7eff123f1740 1 -- 172.21.15.47:0/955951560 shutdown_connections 2024-03-21T11:38:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.115+0000 7eff123f1740 1 --2- 172.21.15.47:0/955951560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ef3a0 0x55b8760e7b60 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.115+0000 7eff123f1740 1 --2- 172.21.15.47:0/955951560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ee9e0 0x55b8760fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.115+0000 7eff123f1740 1 --2- 172.21.15.47:0/955951560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ed810 0x55b8760e7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.115+0000 7eff123f1740 1 -- 172.21.15.47:0/955951560 >> 172.21.15.47:0/955951560 conn(0x55b8760e8c00 msgr2=0x55b8760e8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.115+0000 7eff123f1740 1 -- 172.21.15.47:0/955951560 shutdown_connections 2024-03-21T11:38:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.115+0000 7eff123f1740 1 -- 172.21.15.47:0/955951560 wait complete. 2024-03-21T11:38:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.116+0000 7eff123f1740 1 -- start start 2024-03-21T11:38:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.116+0000 7eff123f1740 1 Processor -- start 2024-03-21T11:38:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff123f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 0x55b8760fb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff123f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760fb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff123f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 0x55b8760fe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff123f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff081b1b70 con 0x55b8760ef3a0 2024-03-21T11:38:01.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff123f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff081b1510 con 0x55b8760ee9e0 2024-03-21T11:38:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff123f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff081ae950 con 0x55b8760ed810 2024-03-21T11:38:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff123f1740 10 client.0 _ll_register_callbacks cb 0x7efef4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff0ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 0x55b8760fb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff07e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760fb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff0ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 0x55b8760fb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44202/0 (socket says 172.21.15.47:44202) 2024-03-21T11:38:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff0ce00640 1 -- 172.21.15.47:0/2116203389 learned_addr learned my addr 172.21.15.47:0/2116203389 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff0d800640 1 --2- 172.21.15.47:0/2116203389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 0x55b8760fe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff07e00640 1 -- 172.21.15.47:0/2116203389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 msgr2=0x55b8760fb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff07e00640 1 --2- 172.21.15.47:0/2116203389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 0x55b8760fb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff07e00640 1 -- 172.21.15.47:0/2116203389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 msgr2=0x55b8760fe5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff07e00640 1 --2- 172.21.15.47:0/2116203389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 0x55b8760fe5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.117+0000 7eff07e00640 1 -- 172.21.15.47:0/2116203389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8760febe0 con 0x55b8760ee9e0 2024-03-21T11:38:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.118+0000 7eff07e00640 1 --2- 172.21.15.47:0/2116203389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760fb620 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7efef4000c00 tx=0x7efef4001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79d0da1195f7c2c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:01.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.118+0000 7efeeea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:01.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.118+0000 7efeeea00640 1 -- 172.21.15.47:0/2116203389 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4005d50 con 0x55b8760ee9e0 2024-03-21T11:38:01.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.118+0000 7efeeea00640 1 -- 172.21.15.47:0/2116203389 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efef4034850 con 0x55b8760ee9e0 2024-03-21T11:38:01.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.118+0000 7eff123f1740 1 -- 172.21.15.47:0/2116203389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff0000fb60 con 0x55b8760ee9e0 2024-03-21T11:38:01.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.119+0000 7efeeea00640 1 -- 172.21.15.47:0/2116203389 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7efef4034a50 con 0x55b8760ee9e0 2024-03-21T11:38:01.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.119+0000 7eff123f1740 10 client.30425 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:01.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.119+0000 7eff123f1740 1 -- 172.21.15.47:0/2116203389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b87615f7a0 con 0x55b8760ee9e0 2024-03-21T11:38:01.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.119+0000 7efeeea00640 1 -- 172.21.15.47:0/2116203389 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff0000fb60 con 0x55b8760ee9e0 2024-03-21T11:38:01.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.120+0000 7efeeea00640 1 -- 172.21.15.47:0/2116203389 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7efef4046c90 con 0x55b8760ee9e0 2024-03-21T11:38:01.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.120+0000 7eff123f1740 10 client.30425 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:01.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.120+0000 7eff123f1740 10 client.30425 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:38:01.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.120+0000 7eff123f1740 20 client.30425 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:01.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.120+0000 7eff123f1740 1 --2- 172.21.15.47:0/2116203389 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b876160090 0x55b8761624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.120+0000 7eff123f1740 4 client.30425 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.120+0000 7eff123f1740 1 -- 172.21.15.47:0/2116203389 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b876059b50 con 0x55b876160090 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.120+0000 7eff0ce00640 1 --2- 172.21.15.47:0/2116203389 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b876160090 0x55b8761624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7eff0ce00640 1 --2- 172.21.15.47:0/2116203389 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b876160090 0x55b8761624b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64d7d22716ee63f5 server_cookie=6eed25f36afa4cea in_seq=0 out_seq=0 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7efeeea00640 10 client.30425 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7efeeea00640 1 -- 172.21.15.47:0/2116203389 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b876059b50 con 0x55b876160090 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7efeeea00640 10 client.30425 handle_command_reply: tid=0 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7eff123f1740 1 client.30425 shutdown 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7eff123f1740 1 -- 172.21.15.47:0/2116203389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 msgr2=0x55b8760fb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7eff123f1740 1 --2- 172.21.15.47:0/2116203389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760fb620 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7efef4000c00 tx=0x7efef4001e50 comp rx=0 tx=0).stop 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7eff123f1740 1 -- 172.21.15.47:0/2116203389 shutdown_connections 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7eff123f1740 1 --2- 172.21.15.47:0/2116203389 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b876160090 0x55b8761624b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7eff123f1740 1 --2- 172.21.15.47:0/2116203389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8760ef3a0 0x55b8760fe5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7eff123f1740 1 --2- 172.21.15.47:0/2116203389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b8760ee9e0 0x55b8760fb620 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7eff123f1740 1 --2- 172.21.15.47:0/2116203389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b8760ed810 0x55b8760fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7efeeea00640 0 client.30425 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:38:01.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7eff123f1740 1 -- 172.21.15.47:0/2116203389 >> 172.21.15.47:0/2116203389 conn(0x55b8760e8c00 msgr2=0x55b8760ebe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:01.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7eff123f1740 1 -- 172.21.15.47:0/2116203389 shutdown_connections 2024-03-21T11:38:01.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.121+0000 7eff123f1740 1 -- 172.21.15.47:0/2116203389 wait complete. 2024-03-21T11:38:01.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.122+0000 7eff123f1740 20 client.30425 trim_cache size 0 max 16384 2024-03-21T11:38:01.125 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:38:01.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.122+0000 7eff0f800640 1 -- 172.21.15.47:0/2687285585 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efef00a2fa0 msgr2=0x7efef00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.122+0000 7eff0f800640 1 --2- 172.21.15.47:0/2687285585 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efef00a2fa0 0x7efef00a5460 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x55b8760ee8e0 tx=0x7eff00009500 comp rx=0 tx=0).stop 2024-03-21T11:38:01.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.122+0000 7eff0f800640 1 -- 172.21.15.47:0/2687285585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff081038e0 msgr2=0x7eff0819dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.122+0000 7eff0f800640 1 --2- 172.21.15.47:0/2687285585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff081038e0 0x7eff0819dff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efefc00e9c0 tx=0x7efefc00edc0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.123+0000 7eff0f800640 1 -- 172.21.15.47:0/2687285585 shutdown_connections 2024-03-21T11:38:01.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.123+0000 7eff0f800640 1 --2- 172.21.15.47:0/2687285585 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efef00a2fa0 0x7efef00a5460 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.123+0000 7eff0f800640 1 --2- 172.21.15.47:0/2687285585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff08105490 0x7eff081a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.123+0000 7eff0f800640 1 --2- 172.21.15.47:0/2687285585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff08104ad0 0x7eff0819e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.123+0000 7eff0f800640 1 --2- 172.21.15.47:0/2687285585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff081038e0 0x7eff0819dff0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.123+0000 7eff0f800640 1 -- 172.21.15.47:0/2687285585 >> 172.21.15.47:0/2687285585 conn(0x7eff080fee50 msgr2=0x7eff08077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:01.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.123+0000 7eff0f800640 1 -- 172.21.15.47:0/2687285585 shutdown_connections 2024-03-21T11:38:01.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.123+0000 7eff0f800640 1 -- 172.21.15.47:0/2687285585 wait complete. 2024-03-21T11:38:01.145 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:01 smithi047 ceph-mon[31351]: pgmap v840: 129 pgs: 129 active+clean; 267 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.4 MiB/s wr, 594 op/s 2024-03-21T11:38:01.146 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:38:01.146 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:38:01.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:01 smithi100 ceph-mon[36245]: pgmap v840: 129 pgs: 129 active+clean; 267 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.4 MiB/s wr, 594 op/s 2024-03-21T11:38:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.433+0000 7ff8b3c00640 1 -- 172.21.15.47:0/226860652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac102170 msgr2=0x7ff8ac108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.433+0000 7ff8b3c00640 1 --2- 172.21.15.47:0/226860652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac102170 0x7ff8ac108a00 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009ca0 tx=0x7ff8a402c100 comp rx=0 tx=0).stop 2024-03-21T11:38:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.434+0000 7ff8b3c00640 1 -- 172.21.15.47:0/226860652 shutdown_connections 2024-03-21T11:38:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.434+0000 7ff8b3c00640 1 --2- 172.21.15.47:0/226860652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac102170 0x7ff8ac108a00 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.434+0000 7ff8b3c00640 1 --2- 172.21.15.47:0/226860652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac1017b0 0x7ff8ac101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.434+0000 7ff8b3c00640 1 --2- 172.21.15.47:0/226860652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac1005c0 0x7ff8ac1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.434+0000 7ff8b3c00640 1 -- 172.21.15.47:0/226860652 >> 172.21.15.47:0/226860652 conn(0x7ff8ac0fbb30 msgr2=0x7ff8ac0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.434+0000 7ff8b3c00640 1 -- 172.21.15.47:0/226860652 shutdown_connections 2024-03-21T11:38:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.434+0000 7ff8b3c00640 1 -- 172.21.15.47:0/226860652 wait complete. 2024-03-21T11:38:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.435+0000 7ff8b3c00640 1 Processor -- start 2024-03-21T11:38:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.436+0000 7ff8b3c00640 1 -- start start 2024-03-21T11:38:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.436+0000 7ff8b3c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac1005c0 0x7ff8ac069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.436+0000 7ff8b3c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac1017b0 0x7ff8ac06a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.436+0000 7ff8b3c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac102170 0x7ff8ac071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.436+0000 7ff8b3c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac10c240 con 0x7ff8ac1005c0 2024-03-21T11:38:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.436+0000 7ff8b3c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac10c0c0 con 0x7ff8ac1017b0 2024-03-21T11:38:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.436+0000 7ff8b3c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac10c3c0 con 0x7ff8ac102170 2024-03-21T11:38:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.436+0000 7ff8b1c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac102170 0x7ff8ac071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.437+0000 7ff8b1c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac102170 0x7ff8ac071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44216/0 (socket says 172.21.15.47:44216) 2024-03-21T11:38:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.437+0000 7ff8b1c00640 1 -- 172.21.15.47:0/3203577644 learned_addr learned my addr 172.21.15.47:0/3203577644 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:01.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.437+0000 7ff8b1200640 1 --2- 172.21.15.47:0/3203577644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac1005c0 0x7ff8ac069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.437+0000 7ff8abe00640 1 --2- 172.21.15.47:0/3203577644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac1017b0 0x7ff8ac06a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.437+0000 7ff8b1c00640 1 -- 172.21.15.47:0/3203577644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac1017b0 msgr2=0x7ff8ac06a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.437+0000 7ff8b1c00640 1 --2- 172.21.15.47:0/3203577644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac1017b0 0x7ff8ac06a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.437+0000 7ff8b1c00640 1 -- 172.21.15.47:0/3203577644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac1005c0 msgr2=0x7ff8ac069b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.437+0000 7ff8b1c00640 1 --2- 172.21.15.47:0/3203577644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac1005c0 0x7ff8ac069b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.437+0000 7ff8b1c00640 1 -- 172.21.15.47:0/3203577644 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac0723a0 con 0x7ff8ac102170 2024-03-21T11:38:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.438+0000 7ff8b1c00640 1 --2- 172.21.15.47:0/3203577644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac102170 0x7ff8ac071e60 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4037040 tx=0x7ff8a4035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.438+0000 7ff8a9600640 1 -- 172.21.15.47:0/3203577644 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4039040 con 0x7ff8ac102170 2024-03-21T11:38:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.438+0000 7ff8b3c00640 1 -- 172.21.15.47:0/3203577644 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ac072630 con 0x7ff8ac102170 2024-03-21T11:38:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.438+0000 7ff8a9600640 1 -- 172.21.15.47:0/3203577644 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8a4044070 con 0x7ff8ac102170 2024-03-21T11:38:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.438+0000 7ff8a9600640 1 -- 172.21.15.47:0/3203577644 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4004380 con 0x7ff8ac102170 2024-03-21T11:38:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.439+0000 7ff8b3c00640 1 -- 172.21.15.47:0/3203577644 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ac072b70 con 0x7ff8ac102170 2024-03-21T11:38:01.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.445+0000 7ff8a9600640 1 -- 172.21.15.47:0/3203577644 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff8a406cb40 con 0x7ff8ac102170 2024-03-21T11:38:01.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.445+0000 7ff8b699d740 1 Processor -- start 2024-03-21T11:38:01.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.446+0000 7ff8b699d740 1 -- start start 2024-03-21T11:38:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.446+0000 7ff8b699d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 0x55a67575cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.446+0000 7ff8b699d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575d9e0 0x55a67575de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.446+0000 7ff8b699d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 0x55a675764c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.446+0000 7ff8b699d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6756c12c0 con 0x55a67575e3a0 2024-03-21T11:38:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.446+0000 7ff8b699d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a675669c90 con 0x55a67575d9e0 2024-03-21T11:38:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.446+0000 7ff8b699d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6756bd360 con 0x55a67575c810 2024-03-21T11:38:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.447+0000 7ff8a9600640 1 --2- 172.21.15.47:0/3203577644 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8940a2f20 0x7ff8940a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.447+0000 7ff8abe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 0x55a67575cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.447+0000 7ff8abe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 0x55a67575cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44222/0 (socket says 172.21.15.47:44222) 2024-03-21T11:38:01.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.447+0000 7ff8abe00640 1 -- 172.21.15.47:0/3399559551 learned_addr learned my addr 172.21.15.47:0/3399559551 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:01.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.447+0000 7ff8abe00640 1 --2- 172.21.15.47:0/3399559551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 0x55a675764c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.447+0000 7ff8b1c00640 1 --2- 172.21.15.47:0/3203577644 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8940a2f20 0x7ff8940a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.447+0000 7ff8a9600640 1 -- 172.21.15.47:0/3203577644 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff8a40f15e0 con 0x7ff8ac102170 2024-03-21T11:38:01.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.447+0000 7ff8b1200640 1 --2- 172.21.15.47:0/3399559551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575d9e0 0x55a67575de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.448+0000 7ff8abe00640 1 -- 172.21.15.47:0/3399559551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575d9e0 msgr2=0x55a67575de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.448+0000 7ff8abe00640 1 --2- 172.21.15.47:0/3399559551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575d9e0 0x55a67575de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.448+0000 7ff8abe00640 1 -- 172.21.15.47:0/3399559551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 msgr2=0x55a675764c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.448+0000 7ff8abe00640 1 --2- 172.21.15.47:0/3399559551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 0x55a675764c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.448+0000 7ff8abe00640 1 -- 172.21.15.47:0/3399559551 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a67568e080 con 0x55a67575c810 2024-03-21T11:38:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.448+0000 7ff8b1c00640 1 --2- 172.21.15.47:0/3203577644 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8940a2f20 0x7ff8940a53e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff8a40eeda0 tx=0x7ff8a402c100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.448+0000 7ff8abe00640 1 --2- 172.21.15.47:0/3399559551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 0x55a67575cc30 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff898012780 tx=0x7ff89800b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9e69ce79ef68ce2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.448+0000 7ff89e000640 1 -- 172.21.15.47:0/3399559551 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff898012dc0 con 0x55a67575c810 2024-03-21T11:38:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.448+0000 7ff89e000640 1 -- 172.21.15.47:0/3399559551 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff898010c50 con 0x55a67575c810 2024-03-21T11:38:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.448+0000 7ff89e000640 1 -- 172.21.15.47:0/3399559551 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89801eb30 con 0x55a67575c810 2024-03-21T11:38:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.449+0000 7ff8b699d740 1 -- 172.21.15.47:0/3399559551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 msgr2=0x55a67575cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.449+0000 7ff8b699d740 1 --2- 172.21.15.47:0/3399559551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 0x55a67575cc30 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff898012780 tx=0x7ff89800b7c0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.450+0000 7ff8b699d740 1 -- 172.21.15.47:0/3399559551 shutdown_connections 2024-03-21T11:38:01.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.450+0000 7ff8b699d740 1 --2- 172.21.15.47:0/3399559551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 0x55a675764c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.450+0000 7ff8b699d740 1 --2- 172.21.15.47:0/3399559551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575d9e0 0x55a67575de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.450+0000 7ff8b699d740 1 --2- 172.21.15.47:0/3399559551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 0x55a67575cc30 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.450+0000 7ff8b699d740 1 -- 172.21.15.47:0/3399559551 >> 172.21.15.47:0/3399559551 conn(0x55a675757c00 msgr2=0x55a675758000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:01.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.450+0000 7ff8b699d740 1 -- 172.21.15.47:0/3399559551 shutdown_connections 2024-03-21T11:38:01.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.450+0000 7ff8b699d740 1 -- 172.21.15.47:0/3399559551 wait complete. 2024-03-21T11:38:01.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.451+0000 7ff8b699d740 1 -- start start 2024-03-21T11:38:01.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.451+0000 7ff8b699d740 1 Processor -- start 2024-03-21T11:38:01.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b699d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 0x55a67576f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b699d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575d9e0 0x55a67576f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b699d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 0x55a675776860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b699d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6757653b0 con 0x55a67575e3a0 2024-03-21T11:38:01.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b699d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67567d0d0 con 0x55a67575d9e0 2024-03-21T11:38:01.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b699d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a675765530 con 0x55a67575c810 2024-03-21T11:38:01.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b699d740 10 client.0 _ll_register_callbacks cb 0x55a675681e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:01.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575d9e0 0x55a67576f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8abe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 0x55a675776860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575d9e0 0x55a67576f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49026/0 (socket says 172.21.15.47:49026) 2024-03-21T11:38:01.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b1200640 1 -- 172.21.15.47:0/3265200921 learned_addr learned my addr 172.21.15.47:0/3265200921 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:01.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8abe00640 1 --2- 172.21.15.47:0/3265200921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 0x55a67576f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b1200640 1 -- 172.21.15.47:0/3265200921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 msgr2=0x55a67576f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b1200640 1 --2- 172.21.15.47:0/3265200921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 0x55a67576f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b1200640 1 -- 172.21.15.47:0/3265200921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 msgr2=0x55a675776860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b1200640 1 --2- 172.21.15.47:0/3265200921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 0x55a675776860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8b1200640 1 -- 172.21.15.47:0/3265200921 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6753d53f0 con 0x55a67575d9e0 2024-03-21T11:38:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.452+0000 7ff8abe00640 1 --2- 172.21.15.47:0/3265200921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 0x55a675776860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.453+0000 7ff8b1200640 1 --2- 172.21.15.47:0/3265200921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575d9e0 0x55a67576f820 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0012700 tx=0x7ff8a0012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=622b904fab72126c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.453+0000 7ff892a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.453+0000 7ff892a00640 1 -- 172.21.15.47:0/3265200921 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a000f920 con 0x55a67575d9e0 2024-03-21T11:38:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.453+0000 7ff892a00640 1 -- 172.21.15.47:0/3265200921 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff8a000fac0 con 0x55a67575d9e0 2024-03-21T11:38:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.453+0000 7ff8b699d740 1 -- 172.21.15.47:0/3265200921 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff898010c50 con 0x55a67575d9e0 2024-03-21T11:38:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.454+0000 7ff892a00640 1 -- 172.21.15.47:0/3265200921 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8a000fcc0 con 0x55a67575d9e0 2024-03-21T11:38:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.454+0000 7ff8b699d740 10 client.30437 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.454+0000 7ff8b699d740 1 -- 172.21.15.47:0/3265200921 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a67549f7d0 con 0x55a67575d9e0 2024-03-21T11:38:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.454+0000 7ff892a00640 1 -- 172.21.15.47:0/3265200921 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff898010c50 con 0x55a67575d9e0 2024-03-21T11:38:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.455+0000 7ff892a00640 1 -- 172.21.15.47:0/3265200921 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7ff8a001c420 con 0x55a67575d9e0 2024-03-21T11:38:01.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.455+0000 7ff8b699d740 10 client.30437 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:01.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.455+0000 7ff8b699d740 10 client.30437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:01.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.455+0000 7ff8b699d740 20 client.30437 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:01.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.455+0000 7ff8b699d740 1 --2- 172.21.15.47:0/3265200921 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a67577ae70 0x55a67577d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.455+0000 7ff8b699d740 4 client.30437 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:01.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.455+0000 7ff8b699d740 1 -- 172.21.15.47:0/3265200921 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6756ce800 con 0x55a67577ae70 2024-03-21T11:38:01.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.456+0000 7ff8abe00640 1 --2- 172.21.15.47:0/3265200921 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a67577ae70 0x55a67577d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.456+0000 7ff8abe00640 1 --2- 172.21.15.47:0/3265200921 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a67577ae70 0x55a67577d290 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=921e8a165d7fcf6e server_cookie=c01dc58bc4b4d260 in_seq=0 out_seq=0 2024-03-21T11:38:01.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.456+0000 7ff892a00640 10 client.30437 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:01.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.458+0000 7ff892a00640 1 -- 172.21.15.47:0/3265200921 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a6756ce800 con 0x55a67577ae70 2024-03-21T11:38:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.458+0000 7ff892a00640 10 client.30437 handle_command_reply: tid=0 2024-03-21T11:38:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.458+0000 7ff8b699d740 1 client.30437 shutdown 2024-03-21T11:38:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.459+0000 7ff8b699d740 1 -- 172.21.15.47:0/3265200921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575d9e0 msgr2=0x55a67576f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.459+0000 7ff8b699d740 1 --2- 172.21.15.47:0/3265200921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575d9e0 0x55a67576f820 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0012700 tx=0x7ff8a0012bc0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.459+0000 7ff8b699d740 1 -- 172.21.15.47:0/3265200921 shutdown_connections 2024-03-21T11:38:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.459+0000 7ff8b699d740 1 --2- 172.21.15.47:0/3265200921 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a67577ae70 0x55a67577d290 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.459+0000 7ff8b699d740 1 --2- 172.21.15.47:0/3265200921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 0x55a675776860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.459+0000 7ff8b699d740 1 --2- 172.21.15.47:0/3265200921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575d9e0 0x55a67576f820 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.459+0000 7ff8b699d740 1 --2- 172.21.15.47:0/3265200921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 0x55a67576f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.459+0000 7ff8b699d740 1 -- 172.21.15.47:0/3265200921 >> 172.21.15.47:0/3265200921 conn(0x55a675757c00 msgr2=0x55a675756eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.459+0000 7ff892a00640 0 client.30437 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:01.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.459+0000 7ff8b699d740 1 -- 172.21.15.47:0/3265200921 shutdown_connections 2024-03-21T11:38:01.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.460+0000 7ff8b699d740 1 -- 172.21.15.47:0/3265200921 wait complete. 2024-03-21T11:38:01.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.461+0000 7ff8b699d740 20 client.30437 trim_cache size 0 max 16384 2024-03-21T11:38:01.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.483+0000 7ff8b699d740 1 Processor -- start 2024-03-21T11:38:01.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.483+0000 7ff8b699d740 1 -- start start 2024-03-21T11:38:01.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.484+0000 7ff8b699d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575c810 0x55a675756620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.484+0000 7ff8b699d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575d9e0 0x55a675776820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.484+0000 7ff8b699d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 0x55a675756b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.484+0000 7ff8b699d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac1b06c0 con 0x55a67575e3a0 2024-03-21T11:38:01.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.484+0000 7ff8b699d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac1b21c0 con 0x55a67575c810 2024-03-21T11:38:01.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.484+0000 7ff8b699d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac1b0d20 con 0x55a67575d9e0 2024-03-21T11:38:01.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.484+0000 7ff8abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575c810 0x55a675756620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.484+0000 7ff8b1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575d9e0 0x55a675776820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.484+0000 7ff8abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575c810 0x55a675756620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49030/0 (socket says 172.21.15.47:49030) 2024-03-21T11:38:01.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.484+0000 7ff8abe00640 1 -- 172.21.15.47:0/1882184311 learned_addr learned my addr 172.21.15.47:0/1882184311 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:01.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.484+0000 7ff8abe00640 1 --2- 172.21.15.47:0/1882184311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 0x55a675756b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.485+0000 7ff8abe00640 1 -- 172.21.15.47:0/1882184311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575d9e0 msgr2=0x55a675776820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.485+0000 7ff8abe00640 1 --2- 172.21.15.47:0/1882184311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575d9e0 0x55a675776820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.485+0000 7ff8abe00640 1 -- 172.21.15.47:0/1882184311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 msgr2=0x55a675756b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.485+0000 7ff8abe00640 1 --2- 172.21.15.47:0/1882184311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 0x55a675756b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.485+0000 7ff8abe00640 1 -- 172.21.15.47:0/1882184311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6757522c0 con 0x55a67575c810 2024-03-21T11:38:01.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.485+0000 7ff8abe00640 1 --2- 172.21.15.47:0/1882184311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575c810 0x55a675756620 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff89800b450 tx=0x7ff89800bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3466ebbe5063599e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:01.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.485+0000 7ff89d600640 1 -- 172.21.15.47:0/1882184311 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff898024040 con 0x55a67575c810 2024-03-21T11:38:01.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.485+0000 7ff89d600640 1 -- 172.21.15.47:0/1882184311 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff898016070 con 0x55a67575c810 2024-03-21T11:38:01.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.485+0000 7ff89d600640 1 -- 172.21.15.47:0/1882184311 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff898010740 con 0x55a67575c810 2024-03-21T11:38:01.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.486+0000 7ff8b699d740 1 -- 172.21.15.47:0/1882184311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575c810 msgr2=0x55a675756620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.486+0000 7ff8b699d740 1 --2- 172.21.15.47:0/1882184311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575c810 0x55a675756620 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff89800b450 tx=0x7ff89800bbc0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.486+0000 7ff8b699d740 1 -- 172.21.15.47:0/1882184311 shutdown_connections 2024-03-21T11:38:01.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.486+0000 7ff8b699d740 1 --2- 172.21.15.47:0/1882184311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575e3a0 0x55a675756b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.486+0000 7ff8b699d740 1 --2- 172.21.15.47:0/1882184311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575d9e0 0x55a675776820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.486+0000 7ff8b699d740 1 --2- 172.21.15.47:0/1882184311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575c810 0x55a675756620 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.486+0000 7ff8b699d740 1 -- 172.21.15.47:0/1882184311 >> 172.21.15.47:0/1882184311 conn(0x55a675757c00 msgr2=0x55a67576f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:01.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.486+0000 7ff8b699d740 1 -- 172.21.15.47:0/1882184311 shutdown_connections 2024-03-21T11:38:01.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.486+0000 7ff8b699d740 1 -- 172.21.15.47:0/1882184311 wait complete. 2024-03-21T11:38:01.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.487+0000 7ff8b699d740 1 -- start start 2024-03-21T11:38:01.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.487+0000 7ff8b699d740 1 Processor -- start 2024-03-21T11:38:01.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.487+0000 7ff8b699d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 0x55a6757720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.487+0000 7ff8b699d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575d9e0 0x55a6757725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.487+0000 7ff8b699d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575e3a0 0x55a675775610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.487+0000 7ff8b699d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac1b1aa0 con 0x55a67575d9e0 2024-03-21T11:38:01.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.487+0000 7ff8b699d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac1b1440 con 0x55a67575e3a0 2024-03-21T11:38:01.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.487+0000 7ff8b699d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac1ae880 con 0x55a67575c810 2024-03-21T11:38:01.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.487+0000 7ff8b699d740 10 client.0 _ll_register_callbacks cb 0x7ff8a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:01.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.488+0000 7ff8b1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575d9e0 0x55a6757725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.488+0000 7ff8abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575e3a0 0x55a675775610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.488+0000 7ff8b1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575d9e0 0x55a6757725f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38256/0 (socket says 172.21.15.47:38256) 2024-03-21T11:38:01.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.488+0000 7ff8b1200640 1 -- 172.21.15.47:0/499034356 learned_addr learned my addr 172.21.15.47:0/499034356 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:01.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.488+0000 7ff8b1200640 1 -- 172.21.15.47:0/499034356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 msgr2=0x55a6757720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.488+0000 7ff8b1200640 1 --2- 172.21.15.47:0/499034356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 0x55a6757720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.488+0000 7ff8b1200640 1 -- 172.21.15.47:0/499034356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575e3a0 msgr2=0x55a675775610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:01.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.488+0000 7ff8b1200640 1 --2- 172.21.15.47:0/499034356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575e3a0 0x55a675775610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.488+0000 7ff8b1200640 1 -- 172.21.15.47:0/499034356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a675775c00 con 0x55a67575d9e0 2024-03-21T11:38:01.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.488+0000 7ff8abe00640 1 --2- 172.21.15.47:0/499034356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575e3a0 0x55a675775610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:01.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.488+0000 7ff8b1200640 1 --2- 172.21.15.47:0/499034356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575d9e0 0x55a6757725f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00092e0 tx=0x7ff8a0012350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6b9b5d519d8a6c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:01.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.488+0000 7ff892a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:38:01.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.489+0000 7ff892a00640 1 -- 172.21.15.47:0/499034356 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a001c630 con 0x55a67575d9e0 2024-03-21T11:38:01.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.489+0000 7ff892a00640 1 -- 172.21.15.47:0/499034356 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff8a0018030 con 0x55a67575d9e0 2024-03-21T11:38:01.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.489+0000 7ff8b699d740 1 -- 172.21.15.47:0/499034356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff898016070 con 0x55a67575d9e0 2024-03-21T11:38:01.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:01 smithi102 ceph-mon[36748]: pgmap v840: 129 pgs: 129 active+clean; 267 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.4 MiB/s wr, 594 op/s 2024-03-21T11:38:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.489+0000 7ff892a00640 1 -- 172.21.15.47:0/499034356 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8a001ca80 con 0x55a67575d9e0 2024-03-21T11:38:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.489+0000 7ff8b699d740 10 client.20001 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.489+0000 7ff8b699d740 1 -- 172.21.15.47:0/499034356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a6757d6850 con 0x55a67575d9e0 2024-03-21T11:38:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.489+0000 7ff892a00640 1 -- 172.21.15.47:0/499034356 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff898016070 con 0x55a67575d9e0 2024-03-21T11:38:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.490+0000 7ff892a00640 1 -- 172.21.15.47:0/499034356 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7ff8a00038c0 con 0x55a67575d9e0 2024-03-21T11:38:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.490+0000 7ff8b699d740 10 client.20001 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.490+0000 7ff8b699d740 10 client.20001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.490+0000 7ff8b699d740 20 client.20001 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.490+0000 7ff8b699d740 1 --2- 172.21.15.47:0/499034356 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a6757d72e0 0x55a6757d9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.490+0000 7ff8b699d740 4 client.20001 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:38:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.490+0000 7ff8b699d740 1 -- 172.21.15.47:0/499034356 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a6756c8b50 con 0x55a6757d72e0 2024-03-21T11:38:01.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.490+0000 7ff8abe00640 1 --2- 172.21.15.47:0/499034356 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a6757d72e0 0x55a6757d9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:01.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.491+0000 7ff8abe00640 1 --2- 172.21.15.47:0/499034356 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a6757d72e0 0x55a6757d9700 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7480e175a0de914 server_cookie=956f0fa3871d9496 in_seq=0 out_seq=0 2024-03-21T11:38:01.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.491+0000 7ff892a00640 10 client.20001 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:01.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff892a00640 1 -- 172.21.15.47:0/499034356 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a6756c8b50 con 0x55a6757d72e0 2024-03-21T11:38:01.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff892a00640 10 client.20001 handle_command_reply: tid=0 2024-03-21T11:38:01.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff8b699d740 1 client.20001 shutdown 2024-03-21T11:38:01.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff8b699d740 1 -- 172.21.15.47:0/499034356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575d9e0 msgr2=0x55a6757725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff8b699d740 1 --2- 172.21.15.47:0/499034356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575d9e0 0x55a6757725f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00092e0 tx=0x7ff8a0012350 comp rx=0 tx=0).stop 2024-03-21T11:38:01.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff8b699d740 1 -- 172.21.15.47:0/499034356 shutdown_connections 2024-03-21T11:38:01.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff8b699d740 1 --2- 172.21.15.47:0/499034356 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a6757d72e0 0x55a6757d9700 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff8b699d740 1 --2- 172.21.15.47:0/499034356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a67575e3a0 0x55a675775610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff8b699d740 1 --2- 172.21.15.47:0/499034356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a67575d9e0 0x55a6757725f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff892a00640 0 client.20001 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:01.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff8b699d740 1 --2- 172.21.15.47:0/499034356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a67575c810 0x55a6757720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff8b699d740 1 -- 172.21.15.47:0/499034356 >> 172.21.15.47:0/499034356 conn(0x55a675757c00 msgr2=0x55a67575ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:01.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff8b699d740 1 -- 172.21.15.47:0/499034356 shutdown_connections 2024-03-21T11:38:01.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff8b699d740 1 -- 172.21.15.47:0/499034356 wait complete. 2024-03-21T11:38:01.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.498+0000 7ff8b699d740 20 client.20001 trim_cache size 0 max 16384 2024-03-21T11:38:01.500 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:38:01.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.499+0000 7ff8b3c00640 1 -- 172.21.15.47:0/3203577644 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8940a2f20 msgr2=0x7ff8940a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.499+0000 7ff8b3c00640 1 --2- 172.21.15.47:0/3203577644 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8940a2f20 0x7ff8940a53e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff8a40eeda0 tx=0x7ff8a402c100 comp rx=0 tx=0).stop 2024-03-21T11:38:01.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.499+0000 7ff8b3c00640 1 -- 172.21.15.47:0/3203577644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac102170 msgr2=0x7ff8ac071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:01.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.499+0000 7ff8b3c00640 1 --2- 172.21.15.47:0/3203577644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac102170 0x7ff8ac071e60 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4037040 tx=0x7ff8a4035d10 comp rx=0 tx=0).stop 2024-03-21T11:38:01.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.499+0000 7ff8b3c00640 1 -- 172.21.15.47:0/3203577644 shutdown_connections 2024-03-21T11:38:01.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.499+0000 7ff8b3c00640 1 --2- 172.21.15.47:0/3203577644 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8940a2f20 0x7ff8940a53e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.499+0000 7ff8b3c00640 1 --2- 172.21.15.47:0/3203577644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac102170 0x7ff8ac071e60 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.499+0000 7ff8b3c00640 1 --2- 172.21.15.47:0/3203577644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac1017b0 0x7ff8ac06a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.499+0000 7ff8b3c00640 1 --2- 172.21.15.47:0/3203577644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac1005c0 0x7ff8ac069b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:01.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.499+0000 7ff8b3c00640 1 -- 172.21.15.47:0/3203577644 >> 172.21.15.47:0/3203577644 conn(0x7ff8ac0fbb30 msgr2=0x7ff8ac104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:01.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.499+0000 7ff8b3c00640 1 -- 172.21.15.47:0/3203577644 shutdown_connections 2024-03-21T11:38:01.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:01.499+0000 7ff8b3c00640 1 -- 172.21.15.47:0/3203577644 wait complete. 2024-03-21T11:38:01.922 INFO:tasks.workunit.client.0.smithi047.stdout: 10 23665 32.11 MB/sec warmup 45 sec latency 263.951 ms 2024-03-21T11:38:02.517 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:38:02.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:38:02.517 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:38:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.826+0000 7ff658e00640 1 -- 172.21.15.47:0/3281643152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff654077c40 msgr2=0x7ff6540784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.826+0000 7ff658e00640 1 --2- 172.21.15.47:0/3281643152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff654077c40 0x7ff6540784e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009ca0 tx=0x7ff64c02c120 comp rx=0 tx=0).stop 2024-03-21T11:38:02.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.827+0000 7ff658e00640 1 -- 172.21.15.47:0/3281643152 shutdown_connections 2024-03-21T11:38:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.827+0000 7ff658e00640 1 --2- 172.21.15.47:0/3281643152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff654077c40 0x7ff6540784e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.827+0000 7ff658e00640 1 --2- 172.21.15.47:0/3281643152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6540772a0 0x7ff654077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.827+0000 7ff658e00640 1 --2- 172.21.15.47:0/3281643152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff654076060 0x7ff654076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.827+0000 7ff658e00640 1 -- 172.21.15.47:0/3281643152 >> 172.21.15.47:0/3281643152 conn(0x7ff6540fee50 msgr2=0x7ff654101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.828+0000 7ff658e00640 1 -- 172.21.15.47:0/3281643152 shutdown_connections 2024-03-21T11:38:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.828+0000 7ff658e00640 1 -- 172.21.15.47:0/3281643152 wait complete. 2024-03-21T11:38:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.830+0000 7ff658e00640 1 Processor -- start 2024-03-21T11:38:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.830+0000 7ff658e00640 1 -- start start 2024-03-21T11:38:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.831+0000 7ff658e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff654076060 0x7ff65410f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.831+0000 7ff658e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6540772a0 0x7ff65410fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.831+0000 7ff658e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff654077c40 0x7ff654110b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.831+0000 7ff658e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff654103d60 con 0x7ff654077c40 2024-03-21T11:38:02.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.831+0000 7ff658e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff654103be0 con 0x7ff6540772a0 2024-03-21T11:38:02.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.831+0000 7ff658e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff654103ee0 con 0x7ff654076060 2024-03-21T11:38:02.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.831+0000 7ff651400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6540772a0 0x7ff65410fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.831+0000 7ff651400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6540772a0 0x7ff65410fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49046/0 (socket says 172.21.15.47:49046) 2024-03-21T11:38:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.831+0000 7ff651400640 1 -- 172.21.15.47:0/2520422342 learned_addr learned my addr 172.21.15.47:0/2520422342 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.831+0000 7ff651e00640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff654076060 0x7ff65410f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.832+0000 7ff652800640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff654077c40 0x7ff654110b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.832+0000 7ff651400640 1 -- 172.21.15.47:0/2520422342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff654076060 msgr2=0x7ff65410f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.832+0000 7ff651400640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff654076060 0x7ff65410f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.832+0000 7ff651400640 1 -- 172.21.15.47:0/2520422342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff654077c40 msgr2=0x7ff654110b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.832+0000 7ff651400640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff654077c40 0x7ff654110b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.832+0000 7ff651400640 1 -- 172.21.15.47:0/2520422342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff654111290 con 0x7ff6540772a0 2024-03-21T11:38:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.832+0000 7ff652800640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff654077c40 0x7ff654110b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.832+0000 7ff651400640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6540772a0 0x7ff65410fcc0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff64000e9f0 tx=0x7ff64000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.833+0000 7ff64aa00640 1 -- 172.21.15.47:0/2520422342 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64000cd80 con 0x7ff6540772a0 2024-03-21T11:38:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.833+0000 7ff658e00640 1 -- 172.21.15.47:0/2520422342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff654111580 con 0x7ff6540772a0 2024-03-21T11:38:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.833+0000 7ff64aa00640 1 -- 172.21.15.47:0/2520422342 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff640005500 con 0x7ff6540772a0 2024-03-21T11:38:02.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.833+0000 7ff64aa00640 1 -- 172.21.15.47:0/2520422342 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640010690 con 0x7ff6540772a0 2024-03-21T11:38:02.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.833+0000 7ff658e00640 1 -- 172.21.15.47:0/2520422342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6541aecf0 con 0x7ff6540772a0 2024-03-21T11:38:02.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.839+0000 7ff64aa00640 1 -- 172.21.15.47:0/2520422342 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff640004eb0 con 0x7ff6540772a0 2024-03-21T11:38:02.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.843+0000 7ff64aa00640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff6380a3020 0x7ff6380a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.844+0000 7ff64aa00640 1 -- 172.21.15.47:0/2520422342 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff6400cfb60 con 0x7ff6540772a0 2024-03-21T11:38:02.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.844+0000 7ff65bc1e740 1 Processor -- start 2024-03-21T11:38:02.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.844+0000 7ff65bc1e740 1 -- start start 2024-03-21T11:38:02.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.844+0000 7ff651e00640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff6380a3020 0x7ff6380a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.845+0000 7ff65bc1e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 0x55e8afa644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.845+0000 7ff65bc1e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa69590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.845+0000 7ff65bc1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa6c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.845+0000 7ff65bc1e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8af9c9eb0 con 0x55e8afa640a0 2024-03-21T11:38:02.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.845+0000 7ff65bc1e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8af972a60 con 0x55e8afa64a90 2024-03-21T11:38:02.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.845+0000 7ff65bc1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8af9c5f50 con 0x55e8afa69c00 2024-03-21T11:38:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.846+0000 7ff652800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa6c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.846+0000 7ff651400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa69590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.846+0000 7ff652800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa6c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44270/0 (socket says 172.21.15.47:44270) 2024-03-21T11:38:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.846+0000 7ff652800640 1 -- 172.21.15.47:0/1564525763 learned_addr learned my addr 172.21.15.47:0/1564525763 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.846+0000 7ff652800640 1 -- 172.21.15.47:0/1564525763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 msgr2=0x55e8afa69590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.846+0000 7ff652800640 1 --2- 172.21.15.47:0/1564525763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa69590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.846+0000 7ff652800640 1 -- 172.21.15.47:0/1564525763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 msgr2=0x55e8afa644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.846+0000 7ff652800640 1 --2- 172.21.15.47:0/1564525763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 0x55e8afa644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.846+0000 7ff652800640 1 -- 172.21.15.47:0/1564525763 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8af7a67e0 con 0x55e8afa69c00 2024-03-21T11:38:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.846+0000 7ff651e00640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff6380a3020 0x7ff6380a54e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff644006fd0 tx=0x7ff6440090e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.846+0000 7ff652800640 1 --2- 172.21.15.47:0/1564525763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa6c090 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0323b0 tx=0x7ff64c0323e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45b8c5f9b4bb83a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.847+0000 7ff63f400640 1 -- 172.21.15.47:0/1564525763 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c039050 con 0x55e8afa69c00 2024-03-21T11:38:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.847+0000 7ff63f400640 1 -- 172.21.15.47:0/1564525763 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff64c045070 con 0x55e8afa69c00 2024-03-21T11:38:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.847+0000 7ff63f400640 1 -- 172.21.15.47:0/1564525763 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c004330 con 0x55e8afa69c00 2024-03-21T11:38:02.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.848+0000 7ff65bc1e740 1 -- 172.21.15.47:0/1564525763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 msgr2=0x55e8afa6c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:02.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.848+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/1564525763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa6c090 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0323b0 tx=0x7ff64c0323e0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.848+0000 7ff65bc1e740 1 -- 172.21.15.47:0/1564525763 shutdown_connections 2024-03-21T11:38:02.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.848+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/1564525763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa6c090 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.848+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/1564525763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa69590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.848+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/1564525763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 0x55e8afa644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.848+0000 7ff65bc1e740 1 -- 172.21.15.47:0/1564525763 >> 172.21.15.47:0/1564525763 conn(0x55e8afa5f610 msgr2=0x55e8afa5fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:02.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.848+0000 7ff65bc1e740 1 -- 172.21.15.47:0/1564525763 shutdown_connections 2024-03-21T11:38:02.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.849+0000 7ff65bc1e740 1 -- 172.21.15.47:0/1564525763 wait complete. 2024-03-21T11:38:02.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.849+0000 7ff65bc1e740 1 -- start start 2024-03-21T11:38:02.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.849+0000 7ff65bc1e740 1 Processor -- start 2024-03-21T11:38:02.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.851+0000 7ff65bc1e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 0x55e8afa6e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.851+0000 7ff65bc1e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa6e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.851+0000 7ff65bc1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa757d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.851+0000 7ff65bc1e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8afa6cb50 con 0x55e8afa640a0 2024-03-21T11:38:02.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.851+0000 7ff65bc1e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8af985eb0 con 0x55e8afa64a90 2024-03-21T11:38:02.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.851+0000 7ff65bc1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8afa6ccd0 con 0x55e8afa69c00 2024-03-21T11:38:02.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.851+0000 7ff65bc1e740 10 client.0 _ll_register_callbacks cb 0x55e8af97a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:02.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.851+0000 7ff652800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.851+0000 7ff652800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa757d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44278/0 (socket says 172.21.15.47:44278) 2024-03-21T11:38:02.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.851+0000 7ff652800640 1 -- 172.21.15.47:0/3555310986 learned_addr learned my addr 172.21.15.47:0/3555310986 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:02.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.852+0000 7ff651e00640 1 --2- 172.21.15.47:0/3555310986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 0x55e8afa6e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.852+0000 7ff652800640 1 -- 172.21.15.47:0/3555310986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 msgr2=0x55e8afa6e790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:38:02.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.853+0000 7ff652800640 1 --2- 172.21.15.47:0/3555310986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa6e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.853+0000 7ff652800640 1 -- 172.21.15.47:0/3555310986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 msgr2=0x55e8afa6e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:02.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.853+0000 7ff652800640 1 --2- 172.21.15.47:0/3555310986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 0x55e8afa6e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.853+0000 7ff652800640 1 -- 172.21.15.47:0/3555310986 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8af806f70 con 0x55e8afa69c00 2024-03-21T11:38:02.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.853+0000 7ff651e00640 1 --2- 172.21.15.47:0/3555310986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 0x55e8afa6e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:38:02.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.854+0000 7ff652800640 1 --2- 172.21.15.47:0/3555310986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa757d0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff64c02c660 tx=0x7ff64c003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2843ad714551a243 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:02.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.854+0000 7ff633e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:38:02.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.855+0000 7ff633e00640 1 -- 172.21.15.47:0/3555310986 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c0040b0 con 0x55e8afa69c00 2024-03-21T11:38:02.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.855+0000 7ff633e00640 1 -- 172.21.15.47:0/3555310986 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff64c004250 con 0x55e8afa69c00 2024-03-21T11:38:02.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.855+0000 7ff65bc1e740 1 -- 172.21.15.47:0/3555310986 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff64c045070 con 0x55e8afa69c00 2024-03-21T11:38:02.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.856+0000 7ff633e00640 1 -- 172.21.15.47:0/3555310986 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff64c004450 con 0x55e8afa69c00 2024-03-21T11:38:02.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.856+0000 7ff65bc1e740 10 client.30460 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:02.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.856+0000 7ff65bc1e740 1 -- 172.21.15.47:0/3555310986 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e8af69e170 con 0x55e8afa69c00 2024-03-21T11:38:02.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.856+0000 7ff633e00640 1 -- 172.21.15.47:0/3555310986 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff64c045070 con 0x55e8afa69c00 2024-03-21T11:38:02.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.858+0000 7ff633e00640 1 -- 172.21.15.47:0/3555310986 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7ff64c02dce0 con 0x55e8afa69c00 2024-03-21T11:38:02.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.858+0000 7ff65bc1e740 10 client.30460 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:02.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.858+0000 7ff65bc1e740 10 client.30460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:02.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.858+0000 7ff65bc1e740 20 client.30460 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:02.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.858+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/3555310986 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e8afa79d60 0x55e8afa7c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.858+0000 7ff65bc1e740 4 client.30460 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:02.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.858+0000 7ff65bc1e740 1 -- 172.21.15.47:0/3555310986 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8af9d73f0 con 0x55e8afa79d60 2024-03-21T11:38:02.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.859+0000 7ff651e00640 1 --2- 172.21.15.47:0/3555310986 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e8afa79d60 0x55e8afa7c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.860+0000 7ff651e00640 1 --2- 172.21.15.47:0/3555310986 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e8afa79d60 0x55e8afa7c1a0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f878bb709ad49106 server_cookie=caf332d99d556009 in_seq=0 out_seq=0 2024-03-21T11:38:02.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.860+0000 7ff633e00640 10 client.30460 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:02.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.863+0000 7ff633e00640 1 -- 172.21.15.47:0/3555310986 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e8af9d73f0 con 0x55e8afa79d60 2024-03-21T11:38:02.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.863+0000 7ff633e00640 10 client.30460 handle_command_reply: tid=0 2024-03-21T11:38:02.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.863+0000 7ff65bc1e740 1 client.30460 shutdown 2024-03-21T11:38:02.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.864+0000 7ff65bc1e740 1 -- 172.21.15.47:0/3555310986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 msgr2=0x55e8afa757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:02.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.864+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/3555310986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa757d0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff64c02c660 tx=0x7ff64c003c20 comp rx=0 tx=0).stop 2024-03-21T11:38:02.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.864+0000 7ff65bc1e740 1 -- 172.21.15.47:0/3555310986 shutdown_connections 2024-03-21T11:38:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.864+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/3555310986 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e8afa79d60 0x55e8afa7c1a0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.864+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/3555310986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa757d0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.864+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/3555310986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa6e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.864+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/3555310986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 0x55e8afa6e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.864+0000 7ff65bc1e740 1 -- 172.21.15.47:0/3555310986 >> 172.21.15.47:0/3555310986 conn(0x55e8afa5f610 msgr2=0x55e8afa5e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.865+0000 7ff633e00640 0 client.30460 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.865+0000 7ff65bc1e740 1 -- 172.21.15.47:0/3555310986 shutdown_connections 2024-03-21T11:38:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.865+0000 7ff65bc1e740 1 -- 172.21.15.47:0/3555310986 wait complete. 2024-03-21T11:38:02.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.865+0000 7ff65bc1e740 20 client.30460 trim_cache size 0 max 16384 2024-03-21T11:38:02.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.895+0000 7ff65bc1e740 1 Processor -- start 2024-03-21T11:38:02.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.895+0000 7ff65bc1e740 1 -- start start 2024-03-21T11:38:02.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.896+0000 7ff65bc1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa640a0 0x55e8afa65320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.896+0000 7ff65bc1e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa75790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.896+0000 7ff65bc1e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa69c00 0x55e8afa65860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.896+0000 7ff65bc1e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541b8d90 con 0x55e8afa69c00 2024-03-21T11:38:02.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.896+0000 7ff65bc1e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541ba890 con 0x55e8afa64a90 2024-03-21T11:38:02.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.896+0000 7ff65bc1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541b93f0 con 0x55e8afa640a0 2024-03-21T11:38:02.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.896+0000 7ff652800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa69c00 0x55e8afa65860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.896+0000 7ff652800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa69c00 0x55e8afa65860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38300/0 (socket says 172.21.15.47:38300) 2024-03-21T11:38:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.896+0000 7ff652800640 1 -- 172.21.15.47:0/2126837990 learned_addr learned my addr 172.21.15.47:0/2126837990 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.896+0000 7ff651400640 1 --2- 172.21.15.47:0/2126837990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa75790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.896+0000 7ff651e00640 1 --2- 172.21.15.47:0/2126837990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa640a0 0x55e8afa65320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.897+0000 7ff651400640 1 -- 172.21.15.47:0/2126837990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa640a0 msgr2=0x55e8afa65320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.897+0000 7ff651400640 1 --2- 172.21.15.47:0/2126837990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa640a0 0x55e8afa65320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.897+0000 7ff651400640 1 -- 172.21.15.47:0/2126837990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa69c00 msgr2=0x55e8afa65860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:02.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.897+0000 7ff651400640 1 --2- 172.21.15.47:0/2126837990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa69c00 0x55e8afa65860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.897+0000 7ff651400640 1 -- 172.21.15.47:0/2126837990 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8afa60380 con 0x55e8afa64a90 2024-03-21T11:38:02.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.897+0000 7ff651400640 1 --2- 172.21.15.47:0/2126837990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa75790 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7ff64000c6a0 tx=0x7ff64009bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70a4a03c4d9702c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.897+0000 7ff63ea00640 1 -- 172.21.15.47:0/2126837990 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6400b4070 con 0x55e8afa64a90 2024-03-21T11:38:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.897+0000 7ff63ea00640 1 -- 172.21.15.47:0/2126837990 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff64009c5a0 con 0x55e8afa64a90 2024-03-21T11:38:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.897+0000 7ff63ea00640 1 -- 172.21.15.47:0/2126837990 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6400cd470 con 0x55e8afa64a90 2024-03-21T11:38:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.898+0000 7ff65bc1e740 1 -- 172.21.15.47:0/2126837990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 msgr2=0x55e8afa75790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:02.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.898+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/2126837990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa75790 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7ff64000c6a0 tx=0x7ff64009bd80 comp rx=0 tx=0).stop 2024-03-21T11:38:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.898+0000 7ff65bc1e740 1 -- 172.21.15.47:0/2126837990 shutdown_connections 2024-03-21T11:38:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.898+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/2126837990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa69c00 0x55e8afa65860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.898+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/2126837990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa75790 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.898+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/2126837990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa640a0 0x55e8afa65320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.898+0000 7ff65bc1e740 1 -- 172.21.15.47:0/2126837990 >> 172.21.15.47:0/2126837990 conn(0x55e8afa5f610 msgr2=0x55e8afa63100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.898+0000 7ff65bc1e740 1 -- 172.21.15.47:0/2126837990 shutdown_connections 2024-03-21T11:38:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.898+0000 7ff65bc1e740 1 -- 172.21.15.47:0/2126837990 wait complete. 2024-03-21T11:38:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.899+0000 7ff65bc1e740 1 -- start start 2024-03-21T11:38:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.899+0000 7ff65bc1e740 1 Processor -- start 2024-03-21T11:38:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.899+0000 7ff65bc1e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 0x55e8afa73750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.899+0000 7ff65bc1e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa73c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.899+0000 7ff65bc1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa74c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.900+0000 7ff65bc1e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541ba170 con 0x55e8afa640a0 2024-03-21T11:38:02.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.900+0000 7ff65bc1e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541b9b10 con 0x55e8afa64a90 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.900+0000 7ff65bc1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541b6f50 con 0x55e8afa69c00 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.900+0000 7ff65bc1e740 10 client.0 _ll_register_callbacks cb 0x7ff64000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.900+0000 7ff651e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 0x55e8afa73750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.900+0000 7ff651400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa73c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.900+0000 7ff651400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa73c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49074/0 (socket says 172.21.15.47:49074) 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.900+0000 7ff651400640 1 -- 172.21.15.47:0/288908794 learned_addr learned my addr 172.21.15.47:0/288908794 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.901+0000 7ff651e00640 1 -- 172.21.15.47:0/288908794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 msgr2=0x55e8afa74c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.901+0000 7ff651e00640 1 --2- 172.21.15.47:0/288908794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa74c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.901+0000 7ff651e00640 1 -- 172.21.15.47:0/288908794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 msgr2=0x55e8afa73c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.901+0000 7ff651e00640 1 --2- 172.21.15.47:0/288908794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa73c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.901+0000 7ff651e00640 1 -- 172.21.15.47:0/288908794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8afa5ac20 con 0x55e8afa640a0 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.901+0000 7ff651e00640 1 --2- 172.21.15.47:0/288908794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 0x55e8afa73750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55e8afa7c9b0 tx=0x7ff64400ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439cde6b512a57e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.901+0000 7ff633e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.901+0000 7ff633e00640 1 -- 172.21.15.47:0/288908794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64401b040 con 0x55e8afa640a0 2024-03-21T11:38:02.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.901+0000 7ff633e00640 1 -- 172.21.15.47:0/288908794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff644015480 con 0x55e8afa640a0 2024-03-21T11:38:02.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.901+0000 7ff65bc1e740 1 -- 172.21.15.47:0/288908794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff64009c5a0 con 0x55e8afa640a0 2024-03-21T11:38:02.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.902+0000 7ff633e00640 1 -- 172.21.15.47:0/288908794 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff644038050 con 0x55e8afa640a0 2024-03-21T11:38:02.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.902+0000 7ff65bc1e740 10 client.20013 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:02.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.902+0000 7ff65bc1e740 1 -- 172.21.15.47:0/288908794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e8afae6d10 con 0x55e8afa640a0 2024-03-21T11:38:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.902+0000 7ff633e00640 1 -- 172.21.15.47:0/288908794 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff64009c5a0 con 0x55e8afa640a0 2024-03-21T11:38:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.902+0000 7ff633e00640 1 -- 172.21.15.47:0/288908794 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7ff64402f6b0 con 0x55e8afa640a0 2024-03-21T11:38:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.903+0000 7ff65bc1e740 10 client.20013 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.903+0000 7ff65bc1e740 10 client.20013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.903+0000 7ff65bc1e740 20 client.20013 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.903+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/288908794 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e8afae7700 0x55e8afae9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.903+0000 7ff65bc1e740 4 client.20013 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:38:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.903+0000 7ff65bc1e740 1 -- 172.21.15.47:0/288908794 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e8af9d1740 con 0x55e8afae7700 2024-03-21T11:38:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.903+0000 7ff651400640 1 --2- 172.21.15.47:0/288908794 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e8afae7700 0x55e8afae9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.904+0000 7ff633e00640 10 client.20013 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:02.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.904+0000 7ff651400640 1 --2- 172.21.15.47:0/288908794 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e8afae7700 0x55e8afae9b80 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f36e284bbc7fb87 server_cookie=d4babc7d8c721f79 in_seq=0 out_seq=0 2024-03-21T11:38:02.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.909+0000 7ff633e00640 1 -- 172.21.15.47:0/288908794 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e8af9d1740 con 0x55e8afae7700 2024-03-21T11:38:02.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.909+0000 7ff633e00640 10 client.20013 handle_command_reply: tid=0 2024-03-21T11:38:02.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.909+0000 7ff65bc1e740 1 client.20013 shutdown 2024-03-21T11:38:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.910+0000 7ff65bc1e740 1 -- 172.21.15.47:0/288908794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 msgr2=0x55e8afa73750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.910+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/288908794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 0x55e8afa73750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55e8afa7c9b0 tx=0x7ff64400ef80 comp rx=0 tx=0).stop 2024-03-21T11:38:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.910+0000 7ff65bc1e740 1 -- 172.21.15.47:0/288908794 shutdown_connections 2024-03-21T11:38:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.910+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/288908794 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e8afae7700 0x55e8afae9b80 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.910+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/288908794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e8afa69c00 0x55e8afa74c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.910+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/288908794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8afa64a90 0x55e8afa73c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.910+0000 7ff65bc1e740 1 --2- 172.21.15.47:0/288908794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8afa640a0 0x55e8afa73750 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.910+0000 7ff65bc1e740 1 -- 172.21.15.47:0/288908794 >> 172.21.15.47:0/288908794 conn(0x55e8afa5f610 msgr2=0x55e8afa5dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:02.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.910+0000 7ff633e00640 0 client.20013 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.912+0000 7ff65bc1e740 1 -- 172.21.15.47:0/288908794 shutdown_connections 2024-03-21T11:38:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.912+0000 7ff65bc1e740 1 -- 172.21.15.47:0/288908794 wait complete. 2024-03-21T11:38:02.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.912+0000 7ff65bc1e740 20 client.20013 trim_cache size 0 max 16384 2024-03-21T11:38:02.913 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:38:02.913 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:38:02.913 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "7b0f3860-e31a-4cb5-96df-dd5cdb5d4f26", 2024-03-21T11:38:02.913 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:38:02.913 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:38:02.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.913+0000 7ff658e00640 1 -- 172.21.15.47:0/2520422342 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff6380a3020 msgr2=0x7ff6380a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:02.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.913+0000 7ff658e00640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff6380a3020 0x7ff6380a54e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff644006fd0 tx=0x7ff6440090e0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.913+0000 7ff658e00640 1 -- 172.21.15.47:0/2520422342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6540772a0 msgr2=0x7ff65410fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:02.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.913+0000 7ff658e00640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6540772a0 0x7ff65410fcc0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff64000e9f0 tx=0x7ff64000edf0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.913+0000 7ff658e00640 1 -- 172.21.15.47:0/2520422342 shutdown_connections 2024-03-21T11:38:02.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.913+0000 7ff658e00640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff6380a3020 0x7ff6380a54e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.913+0000 7ff658e00640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff654077c40 0x7ff654110b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.913+0000 7ff658e00640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6540772a0 0x7ff65410fcc0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.913+0000 7ff658e00640 1 --2- 172.21.15.47:0/2520422342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff654076060 0x7ff65410f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:02.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.913+0000 7ff658e00640 1 -- 172.21.15.47:0/2520422342 >> 172.21.15.47:0/2520422342 conn(0x7ff6540fee50 msgr2=0x7ff654100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:02.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.913+0000 7ff658e00640 1 -- 172.21.15.47:0/2520422342 shutdown_connections 2024-03-21T11:38:02.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:02.913+0000 7ff658e00640 1 -- 172.21.15.47:0/2520422342 wait complete. 2024-03-21T11:38:02.922 INFO:tasks.workunit.client.0.smithi047.stdout: 10 23919 31.60 MB/sec warmup 46 sec latency 400.524 ms 2024-03-21T11:38:02.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:38:02.938 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:38:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.235+0000 7f3c52800640 1 -- 172.21.15.47:0/1599096238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c4c108ed0 msgr2=0x7f3c4c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.235+0000 7f3c52800640 1 --2- 172.21.15.47:0/1599096238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c4c108ed0 0x7f3c4c109350 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009900 tx=0x7f3c3c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:38:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.236+0000 7f3c52800640 1 -- 172.21.15.47:0/1599096238 shutdown_connections 2024-03-21T11:38:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.236+0000 7f3c52800640 1 --2- 172.21.15.47:0/1599096238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c4c109890 0x7f3c4c110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.236+0000 7f3c52800640 1 --2- 172.21.15.47:0/1599096238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c4c108ed0 0x7f3c4c109350 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.236+0000 7f3c52800640 1 --2- 172.21.15.47:0/1599096238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c4c107ce0 0x7f3c4c108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.236+0000 7f3c52800640 1 -- 172.21.15.47:0/1599096238 >> 172.21.15.47:0/1599096238 conn(0x7f3c4c076c70 msgr2=0x7f3c4c0790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.236+0000 7f3c52800640 1 -- 172.21.15.47:0/1599096238 shutdown_connections 2024-03-21T11:38:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.236+0000 7f3c52800640 1 -- 172.21.15.47:0/1599096238 wait complete. 2024-03-21T11:38:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.237+0000 7f3c52800640 1 Processor -- start 2024-03-21T11:38:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.237+0000 7f3c52800640 1 -- start start 2024-03-21T11:38:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.238+0000 7f3c52800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c4c107ce0 0x7f3c4c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.238+0000 7f3c52800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c4c108ed0 0x7f3c4c19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.238+0000 7f3c52800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c4c109890 0x7f3c4c1a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.238+0000 7f3c52800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c113990 con 0x7f3c4c107ce0 2024-03-21T11:38:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.238+0000 7f3c52800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c113810 con 0x7f3c4c108ed0 2024-03-21T11:38:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.238+0000 7f3c52800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c113b10 con 0x7f3c4c109890 2024-03-21T11:38:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.239+0000 7f3c4be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c4c109890 0x7f3c4c1a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.239+0000 7f3c4be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c4c109890 0x7f3c4c1a5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44302/0 (socket says 172.21.15.47:44302) 2024-03-21T11:38:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.239+0000 7f3c4be00640 1 -- 172.21.15.47:0/3915285737 learned_addr learned my addr 172.21.15.47:0/3915285737 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.239+0000 7f3c4b400640 1 --2- 172.21.15.47:0/3915285737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c4c107ce0 0x7f3c4c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.240+0000 7f3c4aa00640 1 --2- 172.21.15.47:0/3915285737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c4c108ed0 0x7f3c4c19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.240+0000 7f3c4be00640 1 -- 172.21.15.47:0/3915285737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c4c108ed0 msgr2=0x7f3c4c19e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.240+0000 7f3c4be00640 1 --2- 172.21.15.47:0/3915285737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c4c108ed0 0x7f3c4c19e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.240+0000 7f3c4be00640 1 -- 172.21.15.47:0/3915285737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c4c107ce0 msgr2=0x7f3c4c19dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.240+0000 7f3c4be00640 1 --2- 172.21.15.47:0/3915285737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c4c107ce0 0x7f3c4c19dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.240+0000 7f3c4be00640 1 -- 172.21.15.47:0/3915285737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c07a020 con 0x7f3c4c109890 2024-03-21T11:38:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.241+0000 7f3c4be00640 1 --2- 172.21.15.47:0/3915285737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c4c109890 0x7f3c4c1a5580 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400d750 tx=0x7f3c3400dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:03.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.241+0000 7f3c43e00640 1 -- 172.21.15.47:0/3915285737 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c34017070 con 0x7f3c4c109890 2024-03-21T11:38:03.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.242+0000 7f3c43e00640 1 -- 172.21.15.47:0/3915285737 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c34005500 con 0x7f3c4c109890 2024-03-21T11:38:03.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.242+0000 7f3c43e00640 1 -- 172.21.15.47:0/3915285737 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c34004d90 con 0x7f3c4c109890 2024-03-21T11:38:03.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.242+0000 7f3c52800640 1 -- 172.21.15.47:0/3915285737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4c07a310 con 0x7f3c4c109890 2024-03-21T11:38:03.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.242+0000 7f3c52800640 1 -- 172.21.15.47:0/3915285737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4c07a7e0 con 0x7f3c4c109890 2024-03-21T11:38:03.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.251+0000 7f3c43e00640 1 -- 172.21.15.47:0/3915285737 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3c34004f30 con 0x7f3c4c109890 2024-03-21T11:38:03.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.253+0000 7f3c43e00640 1 --2- 172.21.15.47:0/3915285737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3c300a2fa0 0x7f3c300a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.253+0000 7f3c43e00640 1 -- 172.21.15.47:0/3915285737 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3c340cf1c0 con 0x7f3c4c109890 2024-03-21T11:38:03.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.253+0000 7f3c4b400640 1 --2- 172.21.15.47:0/3915285737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3c300a2fa0 0x7f3c300a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.254+0000 7f3c5557e740 1 Processor -- start 2024-03-21T11:38:03.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.254+0000 7f3c5557e740 1 -- start start 2024-03-21T11:38:03.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.257+0000 7f3c4b400640 1 --2- 172.21.15.47:0/3915285737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3c300a2fa0 0x7f3c300a5460 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c108d30 tx=0x7f3c44009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:03.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.258+0000 7f3c5557e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743df8e80 0x55c743df92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.258+0000 7f3c5557e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df9870 0x55c743dfe370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.258+0000 7f3c5557e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfe9e0 0x55c743e00e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.258+0000 7f3c5557e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c743d5e250 con 0x55c743dfe9e0 2024-03-21T11:38:03.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.258+0000 7f3c5557e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c743d06a30 con 0x55c743df9870 2024-03-21T11:38:03.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.258+0000 7f3c5557e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c743d5a2f0 con 0x55c743df8e80 2024-03-21T11:38:03.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.258+0000 7f3c4be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfe9e0 0x55c743e00e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.258+0000 7f3c4be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfe9e0 0x55c743e00e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38326/0 (socket says 172.21.15.47:38326) 2024-03-21T11:38:03.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.258+0000 7f3c4be00640 1 -- 172.21.15.47:0/2455237150 learned_addr learned my addr 172.21.15.47:0/2455237150 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:03.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.258+0000 7f3c4aa00640 1 --2- 172.21.15.47:0/2455237150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df9870 0x55c743dfe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.259+0000 7f3c4be00640 1 -- 172.21.15.47:0/2455237150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743df8e80 msgr2=0x55c743df92a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:38:03.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.259+0000 7f3c4be00640 1 --2- 172.21.15.47:0/2455237150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743df8e80 0x55c743df92a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.259+0000 7f3c4be00640 1 -- 172.21.15.47:0/2455237150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df9870 msgr2=0x55c743dfe370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:03.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.259+0000 7f3c4be00640 1 --2- 172.21.15.47:0/2455237150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df9870 0x55c743dfe370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.259+0000 7f3c4be00640 1 -- 172.21.15.47:0/2455237150 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c743cf1690 con 0x55c743dfe9e0 2024-03-21T11:38:03.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.259+0000 7f3c4be00640 1 --2- 172.21.15.47:0/2455237150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfe9e0 0x55c743e00e70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3c3409b260 tx=0x7f3c340056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f7c56ad1a63bfe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:03.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.261+0000 7f3c40c00640 1 -- 172.21.15.47:0/2455237150 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c34020070 con 0x55c743dfe9e0 2024-03-21T11:38:03.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.261+0000 7f3c40c00640 1 -- 172.21.15.47:0/2455237150 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c3409f6b0 con 0x55c743dfe9e0 2024-03-21T11:38:03.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.261+0000 7f3c40c00640 1 -- 172.21.15.47:0/2455237150 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c34097bd0 con 0x55c743dfe9e0 2024-03-21T11:38:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.263+0000 7f3c5557e740 1 -- 172.21.15.47:0/2455237150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfe9e0 msgr2=0x55c743e00e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.263+0000 7f3c5557e740 1 --2- 172.21.15.47:0/2455237150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfe9e0 0x55c743e00e70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3c3409b260 tx=0x7f3c340056b0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.263+0000 7f3c5557e740 1 -- 172.21.15.47:0/2455237150 shutdown_connections 2024-03-21T11:38:03.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.263+0000 7f3c5557e740 1 --2- 172.21.15.47:0/2455237150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfe9e0 0x55c743e00e70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.263+0000 7f3c5557e740 1 --2- 172.21.15.47:0/2455237150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df9870 0x55c743dfe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.263+0000 7f3c5557e740 1 --2- 172.21.15.47:0/2455237150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743df8e80 0x55c743df92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.263+0000 7f3c5557e740 1 -- 172.21.15.47:0/2455237150 >> 172.21.15.47:0/2455237150 conn(0x55c743dc1200 msgr2=0x55c743df47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:03.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.263+0000 7f3c5557e740 1 -- 172.21.15.47:0/2455237150 shutdown_connections 2024-03-21T11:38:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.263+0000 7f3c5557e740 1 -- 172.21.15.47:0/2455237150 wait complete. 2024-03-21T11:38:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.264+0000 7f3c5557e740 1 -- start start 2024-03-21T11:38:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.264+0000 7f3c5557e740 1 Processor -- start 2024-03-21T11:38:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.265+0000 7f3c5557e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743df8e80 0x55c743e02e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.265+0000 7f3c5557e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df9870 0x55c743e033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.265+0000 7f3c5557e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743dfe9e0 0x55c743e0a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.265+0000 7f3c5557e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c743e01930 con 0x55c743df8e80 2024-03-21T11:38:03.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.265+0000 7f3c5557e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c743d19e80 con 0x55c743df9870 2024-03-21T11:38:03.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.265+0000 7f3c5557e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c743e01ab0 con 0x55c743dfe9e0 2024-03-21T11:38:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.265+0000 7f3c5557e740 10 client.0 _ll_register_callbacks cb 0x55c743d2c130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.265+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df9870 0x55c743e033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.265+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df9870 0x55c743e033a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49106/0 (socket says 172.21.15.47:49106) 2024-03-21T11:38:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.265+0000 7f3c4aa00640 1 -- 172.21.15.47:0/504469475 learned_addr learned my addr 172.21.15.47:0/504469475 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.266+0000 7f3c4b400640 1 --2- 172.21.15.47:0/504469475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743df8e80 0x55c743e02e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.266+0000 7f3c4aa00640 1 -- 172.21.15.47:0/504469475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743dfe9e0 msgr2=0x55c743e0a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.266+0000 7f3c4aa00640 1 --2- 172.21.15.47:0/504469475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743dfe9e0 0x55c743e0a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.266+0000 7f3c4aa00640 1 -- 172.21.15.47:0/504469475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743df8e80 msgr2=0x55c743e02e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.266+0000 7f3c4aa00640 1 --2- 172.21.15.47:0/504469475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743df8e80 0x55c743e02e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.266+0000 7f3c4aa00640 1 -- 172.21.15.47:0/504469475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c743b3c7d0 con 0x55c743df9870 2024-03-21T11:38:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.266+0000 7f3c4b400640 1 --2- 172.21.15.47:0/504469475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743df8e80 0x55c743e02e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:03.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.266+0000 7f3c4aa00640 1 --2- 172.21.15.47:0/504469475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df9870 0x55c743e033a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c006010 tx=0x7f3c3c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef52ba5403469fe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.267+0000 7f3c39600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.267+0000 7f3c39600640 1 -- 172.21.15.47:0/504469475 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c042070 con 0x55c743df9870 2024-03-21T11:38:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.267+0000 7f3c39600640 1 -- 172.21.15.47:0/504469475 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3c3c005230 con 0x55c743df9870 2024-03-21T11:38:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.267+0000 7f3c5557e740 1 -- 172.21.15.47:0/504469475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c3409f6b0 con 0x55c743df9870 2024-03-21T11:38:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.267+0000 7f3c39600640 1 -- 172.21.15.47:0/504469475 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c3c04f050 con 0x55c743df9870 2024-03-21T11:38:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.267+0000 7f3c5557e740 10 client.30467 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.267+0000 7f3c5557e740 1 -- 172.21.15.47:0/504469475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c743a723f0 con 0x55c743df9870 2024-03-21T11:38:03.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.268+0000 7f3c39600640 1 -- 172.21.15.47:0/504469475 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c3409f6b0 con 0x55c743df9870 2024-03-21T11:38:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.268+0000 7f3c39600640 1 -- 172.21.15.47:0/504469475 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f3c3c003710 con 0x55c743df9870 2024-03-21T11:38:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.268+0000 7f3c5557e740 10 client.30467 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.268+0000 7f3c5557e740 10 client.30467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.268+0000 7f3c5557e740 20 client.30467 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.268+0000 7f3c5557e740 1 --2- 172.21.15.47:0/504469475 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c743e0ed00 0x55c743e11180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.268+0000 7f3c5557e740 4 client.30467 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:03.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.268+0000 7f3c5557e740 1 -- 172.21.15.47:0/504469475 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c743d6b790 con 0x55c743e0ed00 2024-03-21T11:38:03.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.270+0000 7f3c4b400640 1 --2- 172.21.15.47:0/504469475 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c743e0ed00 0x55c743e11180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.274+0000 7f3c4b400640 1 --2- 172.21.15.47:0/504469475 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c743e0ed00 0x55c743e11180 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f00661baa5de77c0 server_cookie=daf84837d8bcf949 in_seq=0 out_seq=0 2024-03-21T11:38:03.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.274+0000 7f3c39600640 10 client.30467 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:03.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.277+0000 7f3c39600640 1 -- 172.21.15.47:0/504469475 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c743d6b790 con 0x55c743e0ed00 2024-03-21T11:38:03.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.277+0000 7f3c39600640 10 client.30467 handle_command_reply: tid=0 2024-03-21T11:38:03.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.277+0000 7f3c5557e740 1 client.30467 shutdown 2024-03-21T11:38:03.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.278+0000 7f3c5557e740 1 -- 172.21.15.47:0/504469475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df9870 msgr2=0x55c743e033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:03.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.278+0000 7f3c5557e740 1 --2- 172.21.15.47:0/504469475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df9870 0x55c743e033a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c006010 tx=0x7f3c3c004270 comp rx=0 tx=0).stop 2024-03-21T11:38:03.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.278+0000 7f3c5557e740 1 -- 172.21.15.47:0/504469475 shutdown_connections 2024-03-21T11:38:03.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.278+0000 7f3c5557e740 1 --2- 172.21.15.47:0/504469475 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c743e0ed00 0x55c743e11180 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.278+0000 7f3c5557e740 1 --2- 172.21.15.47:0/504469475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743dfe9e0 0x55c743e0a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.278+0000 7f3c5557e740 1 --2- 172.21.15.47:0/504469475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df9870 0x55c743e033a0 secure :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c006010 tx=0x7f3c3c004270 comp rx=0 tx=0).stop 2024-03-21T11:38:03.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.278+0000 7f3c5557e740 1 --2- 172.21.15.47:0/504469475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743df8e80 0x55c743e02e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.278+0000 7f3c39600640 0 client.30467 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:03.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.278+0000 7f3c5557e740 1 -- 172.21.15.47:0/504469475 >> 172.21.15.47:0/504469475 conn(0x55c743dc1200 msgr2=0x55c743df3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:03.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.278+0000 7f3c5557e740 1 -- 172.21.15.47:0/504469475 shutdown_connections 2024-03-21T11:38:03.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.278+0000 7f3c5557e740 1 -- 172.21.15.47:0/504469475 wait complete. 2024-03-21T11:38:03.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.279+0000 7f3c5557e740 20 client.30467 trim_cache size 0 max 16384 2024-03-21T11:38:03.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.304+0000 7f3c5557e740 1 Processor -- start 2024-03-21T11:38:03.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.305+0000 7f3c5557e740 1 -- start start 2024-03-21T11:38:03.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.305+0000 7f3c5557e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df8e80 0x55c743dfdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.305+0000 7f3c5557e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743dfe9e0 0x55c743e0a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.305+0000 7f3c5557e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfa100 0x55c743dfa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.305+0000 7f3c5557e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c06d4c0 con 0x55c743dfa100 2024-03-21T11:38:03.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.305+0000 7f3c5557e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c06efc0 con 0x55c743df8e80 2024-03-21T11:38:03.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.305+0000 7f3c5557e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c06db20 con 0x55c743dfe9e0 2024-03-21T11:38:03.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.305+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df8e80 0x55c743dfdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.305+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df8e80 0x55c743dfdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49118/0 (socket says 172.21.15.47:49118) 2024-03-21T11:38:03.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.305+0000 7f3c4b400640 1 -- 172.21.15.47:0/3042786013 learned_addr learned my addr 172.21.15.47:0/3042786013 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:03.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.306+0000 7f3c4aa00640 1 --2- 172.21.15.47:0/3042786013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743dfe9e0 0x55c743e0a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.306+0000 7f3c4be00640 1 --2- 172.21.15.47:0/3042786013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfa100 0x55c743dfa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.307+0000 7f3c4aa00640 1 -- 172.21.15.47:0/3042786013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df8e80 msgr2=0x55c743dfdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:03.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.307+0000 7f3c4aa00640 1 --2- 172.21.15.47:0/3042786013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df8e80 0x55c743dfdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.307+0000 7f3c4aa00640 1 -- 172.21.15.47:0/3042786013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfa100 msgr2=0x55c743dfa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:03.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.307+0000 7f3c4aa00640 1 --2- 172.21.15.47:0/3042786013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfa100 0x55c743dfa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.307+0000 7f3c4aa00640 1 -- 172.21.15.47:0/3042786013 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c743a320f0 con 0x55c743dfe9e0 2024-03-21T11:38:03.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.307+0000 7f3c4aa00640 1 --2- 172.21.15.47:0/3042786013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743dfe9e0 0x55c743e0a3a0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c040000 tx=0x7f3c3c02bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87d1b69b131270aa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:03.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.308+0000 7f3c4b400640 1 --2- 172.21.15.47:0/3042786013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df8e80 0x55c743dfdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:38:03.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.308+0000 7f3c3be00640 1 -- 172.21.15.47:0/3042786013 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c042040 con 0x55c743dfe9e0 2024-03-21T11:38:03.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.308+0000 7f3c3be00640 1 -- 172.21.15.47:0/3042786013 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c3c005640 con 0x55c743dfe9e0 2024-03-21T11:38:03.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.308+0000 7f3c3be00640 1 -- 172.21.15.47:0/3042786013 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c0349f0 con 0x55c743dfe9e0 2024-03-21T11:38:03.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.309+0000 7f3c5557e740 1 -- 172.21.15.47:0/3042786013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743dfe9e0 msgr2=0x55c743e0a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:03.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.309+0000 7f3c5557e740 1 --2- 172.21.15.47:0/3042786013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743dfe9e0 0x55c743e0a3a0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c040000 tx=0x7f3c3c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:38:03.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.310+0000 7f3c5557e740 1 -- 172.21.15.47:0/3042786013 shutdown_connections 2024-03-21T11:38:03.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.310+0000 7f3c5557e740 1 --2- 172.21.15.47:0/3042786013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfa100 0x55c743dfa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.310+0000 7f3c5557e740 1 --2- 172.21.15.47:0/3042786013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743dfe9e0 0x55c743e0a3a0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.310+0000 7f3c5557e740 1 --2- 172.21.15.47:0/3042786013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743df8e80 0x55c743dfdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.310+0000 7f3c5557e740 1 -- 172.21.15.47:0/3042786013 >> 172.21.15.47:0/3042786013 conn(0x55c743dc1200 msgr2=0x55c743df82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:03.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.310+0000 7f3c5557e740 1 -- 172.21.15.47:0/3042786013 shutdown_connections 2024-03-21T11:38:03.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.310+0000 7f3c5557e740 1 -- 172.21.15.47:0/3042786013 wait complete. 2024-03-21T11:38:03.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.311+0000 7f3c5557e740 1 -- start start 2024-03-21T11:38:03.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.311+0000 7f3c5557e740 1 Processor -- start 2024-03-21T11:38:03.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.312+0000 7f3c5557e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743df8e80 0x55c743e09d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.312+0000 7f3c5557e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfa100 0x55c743e0a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.312+0000 7f3c5557e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743dfe9e0 0x55c743e08420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.312+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743df8e80 0x55c743e09d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.313+0000 7f3c5557e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c06e8a0 con 0x55c743dfa100 2024-03-21T11:38:03.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.313+0000 7f3c5557e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c06e240 con 0x55c743dfe9e0 2024-03-21T11:38:03.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.313+0000 7f3c5557e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c0823b0 con 0x55c743df8e80 2024-03-21T11:38:03.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.313+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfa100 0x55c743e0a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:03 smithi047 ceph-mon[31351]: pgmap v841: 129 pgs: 129 active+clean; 270 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.1 MiB/s wr, 576 op/s 2024-03-21T11:38:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.313+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743df8e80 0x55c743e09d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44336/0 (socket says 172.21.15.47:44336) 2024-03-21T11:38:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.313+0000 7f3c4be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743dfe9e0 0x55c743e08420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.313+0000 7f3c4be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743dfe9e0 0x55c743e08420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49120/0 (socket says 172.21.15.47:49120) 2024-03-21T11:38:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.313+0000 7f3c4be00640 1 -- 172.21.15.47:0/1457202918 learned_addr learned my addr 172.21.15.47:0/1457202918 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.313+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfa100 0x55c743e0a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38344/0 (socket says 172.21.15.47:38344) 2024-03-21T11:38:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.313+0000 7f3c5557e740 10 client.0 _ll_register_callbacks cb 0x7f3c3c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.314+0000 7f3c4b400640 1 -- 172.21.15.47:0/1457202918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743dfe9e0 msgr2=0x55c743e08420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.314+0000 7f3c4b400640 1 --2- 172.21.15.47:0/1457202918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743dfe9e0 0x55c743e08420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.314+0000 7f3c4b400640 1 -- 172.21.15.47:0/1457202918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfa100 msgr2=0x55c743e0a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.314+0000 7f3c4b400640 1 --2- 172.21.15.47:0/1457202918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfa100 0x55c743e0a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.314+0000 7f3c4b400640 1 -- 172.21.15.47:0/1457202918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c743e08c50 con 0x55c743df8e80 2024-03-21T11:38:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.314+0000 7f3c4b400640 1 --2- 172.21.15.47:0/1457202918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743df8e80 0x55c743e09d80 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3c440125a0 tx=0x7f3c44004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=208de04202c53dc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.314+0000 7f3c39600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:38:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.314+0000 7f3c39600640 1 -- 172.21.15.47:0/1457202918 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44026930 con 0x55c743df8e80 2024-03-21T11:38:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.315+0000 7f3c39600640 1 -- 172.21.15.47:0/1457202918 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3c44026ad0 con 0x55c743df8e80 2024-03-21T11:38:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.315+0000 7f3c5557e740 1 -- 172.21.15.47:0/1457202918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c3c005640 con 0x55c743df8e80 2024-03-21T11:38:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.315+0000 7f3c39600640 1 -- 172.21.15.47:0/1457202918 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c44037050 con 0x55c743df8e80 2024-03-21T11:38:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.315+0000 7f3c5557e740 10 client.30472 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.315+0000 7f3c5557e740 1 -- 172.21.15.47:0/1457202918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c743e6b270 con 0x55c743df8e80 2024-03-21T11:38:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.315+0000 7f3c39600640 1 -- 172.21.15.47:0/1457202918 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c3c005640 con 0x55c743df8e80 2024-03-21T11:38:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.316+0000 7f3c39600640 1 -- 172.21.15.47:0/1457202918 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f3c4400f040 con 0x55c743df8e80 2024-03-21T11:38:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.316+0000 7f3c5557e740 10 client.30472 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.316+0000 7f3c5557e740 10 client.30472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.316+0000 7f3c5557e740 20 client.30472 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.316+0000 7f3c5557e740 1 --2- 172.21.15.47:0/1457202918 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c743e6bc40 0x55c743e6e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.316+0000 7f3c5557e740 4 client.30472 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:38:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.316+0000 7f3c5557e740 1 -- 172.21.15.47:0/1457202918 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c743d65ae0 con 0x55c743e6bc40 2024-03-21T11:38:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.317+0000 7f3c4aa00640 1 --2- 172.21.15.47:0/1457202918 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c743e6bc40 0x55c743e6e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.317+0000 7f3c4aa00640 1 --2- 172.21.15.47:0/1457202918 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c743e6bc40 0x55c743e6e030 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2963f88c1f162a39 server_cookie=aa0e3d4ef17fe176 in_seq=0 out_seq=0 2024-03-21T11:38:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.318+0000 7f3c39600640 10 client.30472 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:03.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.321+0000 7f3c39600640 1 -- 172.21.15.47:0/1457202918 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55c743d65ae0 con 0x55c743e6bc40 2024-03-21T11:38:03.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.321+0000 7f3c39600640 10 client.30472 handle_command_reply: tid=0 2024-03-21T11:38:03.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.322+0000 7f3c5557e740 1 client.30472 shutdown 2024-03-21T11:38:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.323+0000 7f3c5557e740 1 -- 172.21.15.47:0/1457202918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743df8e80 msgr2=0x55c743e09d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.323+0000 7f3c5557e740 1 --2- 172.21.15.47:0/1457202918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743df8e80 0x55c743e09d80 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3c440125a0 tx=0x7f3c44004ab0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.323+0000 7f3c5557e740 1 -- 172.21.15.47:0/1457202918 shutdown_connections 2024-03-21T11:38:03.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.323+0000 7f3c5557e740 1 --2- 172.21.15.47:0/1457202918 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c743e6bc40 0x55c743e6e030 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.324+0000 7f3c5557e740 1 --2- 172.21.15.47:0/1457202918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c743dfe9e0 0x55c743e08420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.324+0000 7f3c5557e740 1 --2- 172.21.15.47:0/1457202918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c743dfa100 0x55c743e0a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.324+0000 7f3c5557e740 1 --2- 172.21.15.47:0/1457202918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c743df8e80 0x55c743e09d80 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.324+0000 7f3c5557e740 1 -- 172.21.15.47:0/1457202918 >> 172.21.15.47:0/1457202918 conn(0x55c743dc1200 msgr2=0x55c743dfac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:03.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.324+0000 7f3c39600640 0 client.30472 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:03.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.325+0000 7f3c5557e740 1 -- 172.21.15.47:0/1457202918 shutdown_connections 2024-03-21T11:38:03.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.325+0000 7f3c5557e740 1 -- 172.21.15.47:0/1457202918 wait complete. 2024-03-21T11:38:03.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.325+0000 7f3c5557e740 20 client.30472 trim_cache size 0 max 16384 2024-03-21T11:38:03.327 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:38:03.327 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (76 inodes in the stack)", 2024-03-21T11:38:03.327 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:38:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: "7b0f3860-e31a-4cb5-96df-dd5cdb5d4f26": { 2024-03-21T11:38:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:38:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "7b0f3860-e31a-4cb5-96df-dd5cdb5d4f26", 2024-03-21T11:38:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:38:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:38:03.328 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:38:03.328 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:38:03.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.327+0000 7f3c52800640 1 -- 172.21.15.47:0/3915285737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3c300a2fa0 msgr2=0x7f3c300a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.327+0000 7f3c52800640 1 --2- 172.21.15.47:0/3915285737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3c300a2fa0 0x7f3c300a5460 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c108d30 tx=0x7f3c44009870 comp rx=0 tx=0).stop 2024-03-21T11:38:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.327+0000 7f3c52800640 1 -- 172.21.15.47:0/3915285737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c4c109890 msgr2=0x7f3c4c1a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.327+0000 7f3c52800640 1 --2- 172.21.15.47:0/3915285737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c4c109890 0x7f3c4c1a5580 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400d750 tx=0x7f3c3400dc10 comp rx=0 tx=0).stop 2024-03-21T11:38:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.327+0000 7f3c52800640 1 -- 172.21.15.47:0/3915285737 shutdown_connections 2024-03-21T11:38:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.327+0000 7f3c52800640 1 --2- 172.21.15.47:0/3915285737 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3c300a2fa0 0x7f3c300a5460 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.327+0000 7f3c52800640 1 --2- 172.21.15.47:0/3915285737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c4c109890 0x7f3c4c1a5580 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.327+0000 7f3c52800640 1 --2- 172.21.15.47:0/3915285737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c4c108ed0 0x7f3c4c19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.327+0000 7f3c52800640 1 --2- 172.21.15.47:0/3915285737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c4c107ce0 0x7f3c4c19dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.327+0000 7f3c52800640 1 -- 172.21.15.47:0/3915285737 >> 172.21.15.47:0/3915285737 conn(0x7f3c4c076c70 msgr2=0x7f3c4c10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.327+0000 7f3c52800640 1 -- 172.21.15.47:0/3915285737 shutdown_connections 2024-03-21T11:38:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:03.327+0000 7f3c52800640 1 -- 172.21.15.47:0/3915285737 wait complete. 2024-03-21T11:38:03.359 INFO:tasks.cephfs.filesystem:scrub status for tag:7b0f3860-e31a-4cb5-96df-dd5cdb5d4f26 - {'path': '/', 'tag': '7b0f3860-e31a-4cb5-96df-dd5cdb5d4f26', 'options': 'recursive,force'} 2024-03-21T11:38:03.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:03 smithi100 ceph-mon[36245]: pgmap v841: 129 pgs: 129 active+clean; 270 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.1 MiB/s wr, 576 op/s 2024-03-21T11:38:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:03 smithi102 ceph-mon[36748]: pgmap v841: 129 pgs: 129 active+clean; 270 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.1 MiB/s wr, 576 op/s 2024-03-21T11:38:03.922 INFO:tasks.workunit.client.0.smithi047.stdout: 10 24459 31.56 MB/sec warmup 47 sec latency 123.996 ms 2024-03-21T11:38:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:38:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:04.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:04.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:04.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:04.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:04.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:04.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:04.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:04 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:38:04.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:04.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:04.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:04.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:04.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:04.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:04.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:04.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:04 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:38:04.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:04.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:04.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:04.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:04.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:04.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:04.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:04.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:04 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:04.922 INFO:tasks.workunit.client.0.smithi047.stdout: 10 25280 31.86 MB/sec warmup 48 sec latency 46.376 ms 2024-03-21T11:38:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:05 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:05 smithi047 ceph-mon[31351]: pgmap v842: 129 pgs: 129 active+clean; 285 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.6 MiB/s wr, 702 op/s 2024-03-21T11:38:05.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:05 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:05.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:05 smithi100 ceph-mon[36245]: pgmap v842: 129 pgs: 129 active+clean; 285 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.6 MiB/s wr, 702 op/s 2024-03-21T11:38:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:05 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:05 smithi102 ceph-mon[36748]: pgmap v842: 129 pgs: 129 active+clean; 285 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.6 MiB/s wr, 702 op/s 2024-03-21T11:38:05.922 INFO:tasks.workunit.client.0.smithi047.stdout: 10 25897 32.11 MB/sec warmup 49 sec latency 62.094 ms 2024-03-21T11:38:06.923 INFO:tasks.workunit.client.0.smithi047.stdout: 10 26367 32.00 MB/sec warmup 50 sec latency 130.517 ms 2024-03-21T11:38:07.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:07 smithi047 ceph-mon[31351]: pgmap v843: 129 pgs: 129 active+clean; 287 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 639 op/s 2024-03-21T11:38:07.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:07 smithi100 ceph-mon[36245]: pgmap v843: 129 pgs: 129 active+clean; 287 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 639 op/s 2024-03-21T11:38:07.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:07 smithi102 ceph-mon[36748]: pgmap v843: 129 pgs: 129 active+clean; 287 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 639 op/s 2024-03-21T11:38:07.923 INFO:tasks.workunit.client.0.smithi047.stdout: 10 26374 31.38 MB/sec warmup 51 sec latency 1109.939 ms 2024-03-21T11:38:08.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:08 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:38:08.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:08 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:38:08.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:08 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:38:08.923 INFO:tasks.workunit.client.0.smithi047.stdout: 10 26452 30.86 MB/sec warmup 52 sec latency 1563.084 ms 2024-03-21T11:38:09.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:09 smithi100 ceph-mon[36245]: pgmap v844: 129 pgs: 129 active+clean; 291 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.3 MiB/s wr, 672 op/s 2024-03-21T11:38:09.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:09 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:38:09.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:09 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:38:09.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:09 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:38:09.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:09 smithi102 ceph-mon[36748]: pgmap v844: 129 pgs: 129 active+clean; 291 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.3 MiB/s wr, 672 op/s 2024-03-21T11:38:09.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:09 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:38:09.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:09 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:38:09.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:09 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:38:09.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:09 smithi047 ceph-mon[31351]: pgmap v844: 129 pgs: 129 active+clean; 291 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.3 MiB/s wr, 672 op/s 2024-03-21T11:38:09.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:09 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:38:09.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:09 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:38:09.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:09 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:38:09.923 INFO:tasks.workunit.client.0.smithi047.stdout: 10 26642 30.46 MB/sec warmup 53 sec latency 264.615 ms 2024-03-21T11:38:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.256+0000 7f411e400640 1 -- 172.21.15.47:0/95852543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4118105250 msgr2=0x7f41181056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.256+0000 7f411e400640 1 --2- 172.21.15.47:0/95852543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4118105250 0x7f41181056d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4110009be0 tx=0x7f411002bef0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.257+0000 7f411e400640 1 -- 172.21.15.47:0/95852543 shutdown_connections 2024-03-21T11:38:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.257+0000 7f411e400640 1 --2- 172.21.15.47:0/95852543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4118105250 0x7f41181056d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.257+0000 7f411e400640 1 --2- 172.21.15.47:0/95852543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41181028a0 0x7f4118104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.257+0000 7f411e400640 1 --2- 172.21.15.47:0/95852543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41180fff30 0x7f4118102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.257+0000 7f411e400640 1 -- 172.21.15.47:0/95852543 >> 172.21.15.47:0/95852543 conn(0x7f41180fbb30 msgr2=0x7f41180fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.257+0000 7f411e400640 1 -- 172.21.15.47:0/95852543 shutdown_connections 2024-03-21T11:38:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.257+0000 7f411e400640 1 -- 172.21.15.47:0/95852543 wait complete. 2024-03-21T11:38:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.258+0000 7f411e400640 1 Processor -- start 2024-03-21T11:38:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.260+0000 7f411e400640 1 -- start start 2024-03-21T11:38:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.260+0000 7f411e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41180fff30 0x7f411819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.260+0000 7f411e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41181028a0 0x7f411819e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.260+0000 7f411e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4118105250 0x7f41181a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.260+0000 7f411e400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4118113960 con 0x7f41181028a0 2024-03-21T11:38:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.260+0000 7f411e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41181137e0 con 0x7f41180fff30 2024-03-21T11:38:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.260+0000 7f411e400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4118113ae0 con 0x7f4118105250 2024-03-21T11:38:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.261+0000 7f4117e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4118105250 0x7f41181a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.261+0000 7f4117400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41180fff30 0x7f411819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.261+0000 7f4117400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41180fff30 0x7f411819de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34628/0 (socket says 172.21.15.47:34628) 2024-03-21T11:38:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.261+0000 7f4117e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4118105250 0x7f41181a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58460/0 (socket says 172.21.15.47:58460) 2024-03-21T11:38:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.261+0000 7f4117400640 1 -- 172.21.15.47:0/3991531439 learned_addr learned my addr 172.21.15.47:0/3991531439 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.261+0000 7f4117e00640 1 -- 172.21.15.47:0/3991531439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41180fff30 msgr2=0x7f411819de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.261+0000 7f4117e00640 1 --2- 172.21.15.47:0/3991531439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41180fff30 0x7f411819de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.261+0000 7f4117e00640 1 -- 172.21.15.47:0/3991531439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41181028a0 msgr2=0x7f411819e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.261+0000 7f4117e00640 1 --2- 172.21.15.47:0/3991531439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41181028a0 0x7f411819e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.261+0000 7f4117e00640 1 -- 172.21.15.47:0/3991531439 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411806b1f0 con 0x7f4118105250 2024-03-21T11:38:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.261+0000 7f4117400640 1 --2- 172.21.15.47:0/3991531439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41180fff30 0x7f411819de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.262+0000 7f4117e00640 1 --2- 172.21.15.47:0/3991531439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4118105250 0x7f41181a53e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f411002c430 tx=0x7f41100337b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.262+0000 7f410fe00640 1 -- 172.21.15.47:0/3991531439 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110043020 con 0x7f4118105250 2024-03-21T11:38:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.262+0000 7f410fe00640 1 -- 172.21.15.47:0/3991531439 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4110003b80 con 0x7f4118105250 2024-03-21T11:38:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.262+0000 7f410fe00640 1 -- 172.21.15.47:0/3991531439 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110034690 con 0x7f4118105250 2024-03-21T11:38:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.263+0000 7f411e400640 1 -- 172.21.15.47:0/3991531439 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411806b480 con 0x7f4118105250 2024-03-21T11:38:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.263+0000 7f411e400640 1 -- 172.21.15.47:0/3991531439 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411806b980 con 0x7f4118105250 2024-03-21T11:38:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.268+0000 7f410fe00640 1 -- 172.21.15.47:0/3991531439 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4110046020 con 0x7f4118105250 2024-03-21T11:38:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.270+0000 7f410fe00640 1 --2- 172.21.15.47:0/3991531439 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40fc0a30c0 0x7f40fc0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.271+0000 7f410fe00640 1 -- 172.21.15.47:0/3991531439 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f41100f2e50 con 0x7f4118105250 2024-03-21T11:38:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.271+0000 7f4117400640 1 --2- 172.21.15.47:0/3991531439 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40fc0a30c0 0x7f40fc0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.273+0000 7f4117400640 1 --2- 172.21.15.47:0/3991531439 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40fc0a30c0 0x7f40fc0a5580 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f41080096c0 tx=0x7f4108009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.276+0000 7f41211b0740 1 Processor -- start 2024-03-21T11:38:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.277+0000 7f41211b0740 1 -- start start 2024-03-21T11:38:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.277+0000 7f41211b0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf6d1e0 0x5555edf6d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.277+0000 7f41211b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 0x5555edf726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.277+0000 7f41211b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf72d40 0x5555edf751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.277+0000 7f4116a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 0x5555edf726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.278+0000 7f4117400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf6d1e0 0x5555edf6d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.278+0000 7f4117e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf72d40 0x5555edf751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.278+0000 7f41211b0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555edf303b0 con 0x5555edf6d1e0 2024-03-21T11:38:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.278+0000 7f41211b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555eded2830 con 0x5555edf6dbd0 2024-03-21T11:38:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.278+0000 7f41211b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ede7b5d0 con 0x5555edf72d40 2024-03-21T11:38:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.278+0000 7f4117400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf6d1e0 0x5555edf6d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40940/0 (socket says 172.21.15.47:40940) 2024-03-21T11:38:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.278+0000 7f4117400640 1 -- 172.21.15.47:0/1101482262 learned_addr learned my addr 172.21.15.47:0/1101482262 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.278+0000 7f4117e00640 1 -- 172.21.15.47:0/1101482262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 msgr2=0x5555edf726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.278+0000 7f4117e00640 1 --2- 172.21.15.47:0/1101482262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 0x5555edf726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.279+0000 7f4117e00640 1 -- 172.21.15.47:0/1101482262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf6d1e0 msgr2=0x5555edf6d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.279+0000 7f4117e00640 1 --2- 172.21.15.47:0/1101482262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf6d1e0 0x5555edf6d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.279+0000 7f4117e00640 1 -- 172.21.15.47:0/1101482262 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555ede932c0 con 0x5555edf72d40 2024-03-21T11:38:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.279+0000 7f4117e00640 1 --2- 172.21.15.47:0/1101482262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf72d40 0x5555edf751d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f41100bee90 tx=0x7f411002cac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46ad10ba7d979481 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.280+0000 7f410cc00640 1 -- 172.21.15.47:0/1101482262 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41100bf190 con 0x5555edf72d40 2024-03-21T11:38:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.280+0000 7f410cc00640 1 -- 172.21.15.47:0/1101482262 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41100c32e0 con 0x5555edf72d40 2024-03-21T11:38:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.280+0000 7f410cc00640 1 -- 172.21.15.47:0/1101482262 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411002cb50 con 0x5555edf72d40 2024-03-21T11:38:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.280+0000 7f41211b0740 1 -- 172.21.15.47:0/1101482262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf72d40 msgr2=0x5555edf751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.280+0000 7f41211b0740 1 --2- 172.21.15.47:0/1101482262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf72d40 0x5555edf751d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f41100bee90 tx=0x7f411002cac0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.281+0000 7f41211b0740 1 -- 172.21.15.47:0/1101482262 shutdown_connections 2024-03-21T11:38:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.281+0000 7f41211b0740 1 --2- 172.21.15.47:0/1101482262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf72d40 0x5555edf751d0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.281+0000 7f41211b0740 1 --2- 172.21.15.47:0/1101482262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 0x5555edf726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.281+0000 7f41211b0740 1 --2- 172.21.15.47:0/1101482262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf6d1e0 0x5555edf6d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.281+0000 7f41211b0740 1 -- 172.21.15.47:0/1101482262 >> 172.21.15.47:0/1101482262 conn(0x5555edf68750 msgr2=0x5555edf68b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.282+0000 7f41211b0740 1 -- 172.21.15.47:0/1101482262 shutdown_connections 2024-03-21T11:38:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.282+0000 7f41211b0740 1 -- 172.21.15.47:0/1101482262 wait complete. 2024-03-21T11:38:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.282+0000 7f41211b0740 1 -- start start 2024-03-21T11:38:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.283+0000 7f41211b0740 1 Processor -- start 2024-03-21T11:38:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.283+0000 7f41211b0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf6d1e0 0x5555edf770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.283+0000 7f41211b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6dbd0 0x5555edf775f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.283+0000 7f41211b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf72d40 0x5555edf7e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.283+0000 7f41211b0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ede8ea20 con 0x5555edf6d1e0 2024-03-21T11:38:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.283+0000 7f41211b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555edece8d0 con 0x5555edf72d40 2024-03-21T11:38:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.283+0000 7f41211b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555edf75ab0 con 0x5555edf6dbd0 2024-03-21T11:38:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.283+0000 7f41211b0740 10 client.0 _ll_register_callbacks cb 0x5555edecb1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.284+0000 7f4117e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf72d40 0x5555edf7e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.284+0000 7f4117e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf72d40 0x5555edf7e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34652/0 (socket says 172.21.15.47:34652) 2024-03-21T11:38:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.284+0000 7f4117e00640 1 -- 172.21.15.47:0/4063360106 learned_addr learned my addr 172.21.15.47:0/4063360106 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.284+0000 7f4117400640 1 --2- 172.21.15.47:0/4063360106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf6d1e0 0x5555edf770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.285+0000 7f4117e00640 1 -- 172.21.15.47:0/4063360106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6dbd0 msgr2=0x5555edf775f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.285+0000 7f4117e00640 1 --2- 172.21.15.47:0/4063360106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6dbd0 0x5555edf775f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.285+0000 7f4117e00640 1 -- 172.21.15.47:0/4063360106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf6d1e0 msgr2=0x5555edf770b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.285+0000 7f4117e00640 1 --2- 172.21.15.47:0/4063360106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf6d1e0 0x5555edf770b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.285+0000 7f4117e00640 1 -- 172.21.15.47:0/4063360106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555edbe7210 con 0x5555edf72d40 2024-03-21T11:38:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.287+0000 7f4117e00640 1 --2- 172.21.15.47:0/4063360106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf72d40 0x5555edf7e630 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f41100c7670 tx=0x7f4110034f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=295a3cd62137c593 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.287+0000 7f4101600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.287+0000 7f4101600640 1 -- 172.21.15.47:0/4063360106 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41100d8070 con 0x5555edf72d40 2024-03-21T11:38:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.288+0000 7f4101600640 1 -- 172.21.15.47:0/4063360106 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f41100d8210 con 0x5555edf72d40 2024-03-21T11:38:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.288+0000 7f41211b0740 1 -- 172.21.15.47:0/4063360106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41100c32e0 con 0x5555edf72d40 2024-03-21T11:38:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.289+0000 7f4101600640 1 -- 172.21.15.47:0/4063360106 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41100bf190 con 0x5555edf72d40 2024-03-21T11:38:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.289+0000 7f41211b0740 10 client.30485 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.289+0000 7f41211b0740 1 -- 172.21.15.47:0/4063360106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5555edcb3620 con 0x5555edf72d40 2024-03-21T11:38:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.290+0000 7f4101600640 1 -- 172.21.15.47:0/4063360106 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41100c32e0 con 0x5555edf72d40 2024-03-21T11:38:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.290+0000 7f4101600640 1 -- 172.21.15.47:0/4063360106 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f4110004b40 con 0x5555edf72d40 2024-03-21T11:38:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.290+0000 7f41211b0740 10 client.30485 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.290+0000 7f41211b0740 10 client.30485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.291+0000 7f41211b0740 20 client.30485 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.291+0000 7f41211b0740 1 --2- 172.21.15.47:0/4063360106 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5555edf82e00 0x5555edf85220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.291+0000 7f41211b0740 4 client.30485 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.291+0000 7f41211b0740 1 -- 172.21.15.47:0/4063360106 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555ededfd70 con 0x5555edf82e00 2024-03-21T11:38:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.293+0000 7f4117400640 1 --2- 172.21.15.47:0/4063360106 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5555edf82e00 0x5555edf85220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.299+0000 7f4117400640 1 --2- 172.21.15.47:0/4063360106 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5555edf82e00 0x5555edf85220 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f84a3f69c3ed6376 server_cookie=ae197353d962120e in_seq=0 out_seq=0 2024-03-21T11:38:10.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.300+0000 7f4101600640 10 client.30485 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.302+0000 7f4101600640 1 -- 172.21.15.47:0/4063360106 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5555ededfd70 con 0x5555edf82e00 2024-03-21T11:38:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.302+0000 7f4101600640 10 client.30485 handle_command_reply: tid=0 2024-03-21T11:38:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.302+0000 7f41211b0740 1 client.30485 shutdown 2024-03-21T11:38:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.304+0000 7f41211b0740 1 -- 172.21.15.47:0/4063360106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf72d40 msgr2=0x5555edf7e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.304+0000 7f41211b0740 1 --2- 172.21.15.47:0/4063360106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf72d40 0x5555edf7e630 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f41100c7670 tx=0x7f4110034f90 comp rx=0 tx=0).stop 2024-03-21T11:38:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.305+0000 7f41211b0740 1 -- 172.21.15.47:0/4063360106 shutdown_connections 2024-03-21T11:38:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.305+0000 7f41211b0740 1 --2- 172.21.15.47:0/4063360106 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5555edf82e00 0x5555edf85220 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.305+0000 7f41211b0740 1 --2- 172.21.15.47:0/4063360106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf72d40 0x5555edf7e630 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.305+0000 7f41211b0740 1 --2- 172.21.15.47:0/4063360106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6dbd0 0x5555edf775f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.305+0000 7f41211b0740 1 --2- 172.21.15.47:0/4063360106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf6d1e0 0x5555edf770b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.305+0000 7f4101600640 0 client.30485 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.305+0000 7f41211b0740 1 -- 172.21.15.47:0/4063360106 >> 172.21.15.47:0/4063360106 conn(0x5555edf68750 msgr2=0x5555edf67950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.305+0000 7f41211b0740 1 -- 172.21.15.47:0/4063360106 shutdown_connections 2024-03-21T11:38:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.305+0000 7f41211b0740 1 -- 172.21.15.47:0/4063360106 wait complete. 2024-03-21T11:38:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.305+0000 7f41211b0740 20 client.30485 trim_cache size 0 max 16384 2024-03-21T11:38:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.336+0000 7f41211b0740 1 Processor -- start 2024-03-21T11:38:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.337+0000 7f41211b0740 1 -- start start 2024-03-21T11:38:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.337+0000 7f41211b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6d1e0 0x5555edf770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.337+0000 7f41211b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 0x5555edf7e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.337+0000 7f41211b0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf72d40 0x5555edf775f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.337+0000 7f41211b0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41181b8cb0 con 0x5555edf72d40 2024-03-21T11:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.337+0000 7f4117400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6d1e0 0x5555edf770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.337+0000 7f41211b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41181ba7b0 con 0x5555edf6dbd0 2024-03-21T11:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.337+0000 7f41211b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41181b9310 con 0x5555edf6d1e0 2024-03-21T11:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.337+0000 7f4117400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6d1e0 0x5555edf770b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58494/0 (socket says 172.21.15.47:58494) 2024-03-21T11:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.337+0000 7f4117400640 1 -- 172.21.15.47:0/1668414026 learned_addr learned my addr 172.21.15.47:0/1668414026 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.338+0000 7f4117e00640 1 --2- 172.21.15.47:0/1668414026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf72d40 0x5555edf775f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.338+0000 7f4116a00640 1 --2- 172.21.15.47:0/1668414026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 0x5555edf7e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.338+0000 7f4117400640 1 -- 172.21.15.47:0/1668414026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 msgr2=0x5555edf7e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.338+0000 7f4117400640 1 --2- 172.21.15.47:0/1668414026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 0x5555edf7e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.339+0000 7f4117400640 1 -- 172.21.15.47:0/1668414026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf72d40 msgr2=0x5555edf775f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.339+0000 7f4117400640 1 --2- 172.21.15.47:0/1668414026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf72d40 0x5555edf775f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.339+0000 7f4117400640 1 -- 172.21.15.47:0/1668414026 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555edcb3620 con 0x5555edf6d1e0 2024-03-21T11:38:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.339+0000 7f4117e00640 1 --2- 172.21.15.47:0/1668414026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf72d40 0x5555edf775f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.340+0000 7f4117400640 1 --2- 172.21.15.47:0/1668414026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6d1e0 0x5555edf770b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f4108005c80 tx=0x7f4108014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9010bd3435e14b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.340+0000 7f4103e00640 1 -- 172.21.15.47:0/1668414026 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410802c250 con 0x5555edf6d1e0 2024-03-21T11:38:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.340+0000 7f4103e00640 1 -- 172.21.15.47:0/1668414026 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f410801bd30 con 0x5555edf6d1e0 2024-03-21T11:38:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.340+0000 7f4103e00640 1 -- 172.21.15.47:0/1668414026 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108005650 con 0x5555edf6d1e0 2024-03-21T11:38:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.341+0000 7f41211b0740 1 -- 172.21.15.47:0/1668414026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6d1e0 msgr2=0x5555edf770b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.341+0000 7f41211b0740 1 --2- 172.21.15.47:0/1668414026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6d1e0 0x5555edf770b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f4108005c80 tx=0x7f4108014000 comp rx=0 tx=0).stop 2024-03-21T11:38:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.342+0000 7f41211b0740 1 -- 172.21.15.47:0/1668414026 shutdown_connections 2024-03-21T11:38:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.342+0000 7f41211b0740 1 --2- 172.21.15.47:0/1668414026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf72d40 0x5555edf775f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.342+0000 7f41211b0740 1 --2- 172.21.15.47:0/1668414026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 0x5555edf7e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.342+0000 7f41211b0740 1 --2- 172.21.15.47:0/1668414026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6d1e0 0x5555edf770b0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.342+0000 7f41211b0740 1 -- 172.21.15.47:0/1668414026 >> 172.21.15.47:0/1668414026 conn(0x5555edf68750 msgr2=0x5555edf6eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.342+0000 7f41211b0740 1 -- 172.21.15.47:0/1668414026 shutdown_connections 2024-03-21T11:38:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.342+0000 7f41211b0740 1 -- 172.21.15.47:0/1668414026 wait complete. 2024-03-21T11:38:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.343+0000 7f41211b0740 1 -- start start 2024-03-21T11:38:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.343+0000 7f41211b0740 1 Processor -- start 2024-03-21T11:38:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.343+0000 7f41211b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6d1e0 0x5555edf7c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.343+0000 7f41211b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 0x5555edf7c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.343+0000 7f41211b0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf72d40 0x5555edf7d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.344+0000 7f41211b0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41181ba090 con 0x5555edf72d40 2024-03-21T11:38:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.344+0000 7f41211b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41181b9a30 con 0x5555edf6dbd0 2024-03-21T11:38:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.344+0000 7f41211b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41180734b0 con 0x5555edf6d1e0 2024-03-21T11:38:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.344+0000 7f41211b0740 10 client.0 _ll_register_callbacks cb 0x7f41100c3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.344+0000 7f4117400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6d1e0 0x5555edf7c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.344+0000 7f4117400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6d1e0 0x5555edf7c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58502/0 (socket says 172.21.15.47:58502) 2024-03-21T11:38:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.344+0000 7f4117400640 1 -- 172.21.15.47:0/341947909 learned_addr learned my addr 172.21.15.47:0/341947909 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.344+0000 7f4117e00640 1 --2- 172.21.15.47:0/341947909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf72d40 0x5555edf7d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.344+0000 7f4116a00640 1 --2- 172.21.15.47:0/341947909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 0x5555edf7c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.345+0000 7f4117400640 1 -- 172.21.15.47:0/341947909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 msgr2=0x5555edf7c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.345+0000 7f4117400640 1 --2- 172.21.15.47:0/341947909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 0x5555edf7c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.345+0000 7f4117400640 1 -- 172.21.15.47:0/341947909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf72d40 msgr2=0x5555edf7d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.345+0000 7f4117400640 1 --2- 172.21.15.47:0/341947909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf72d40 0x5555edf7d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.345+0000 7f4117400640 1 -- 172.21.15.47:0/341947909 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555edf7db60 con 0x5555edf6d1e0 2024-03-21T11:38:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.345+0000 7f4117e00640 1 --2- 172.21.15.47:0/341947909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf72d40 0x5555edf7d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:38:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.346+0000 7f4117400640 1 --2- 172.21.15.47:0/341947909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6d1e0 0x5555edf7c0e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f4108011f10 tx=0x7f4108005630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d17e77aa45ce3e2a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.346+0000 7f4101600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:38:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.347+0000 7f4101600640 1 -- 172.21.15.47:0/341947909 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410803b060 con 0x5555edf6d1e0 2024-03-21T11:38:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.348+0000 7f4101600640 1 -- 172.21.15.47:0/341947909 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f410802c0d0 con 0x5555edf6d1e0 2024-03-21T11:38:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.348+0000 7f41211b0740 1 -- 172.21.15.47:0/341947909 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f410801bd30 con 0x5555edf6d1e0 2024-03-21T11:38:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.349+0000 7f4101600640 1 -- 172.21.15.47:0/341947909 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f410802c2d0 con 0x5555edf6d1e0 2024-03-21T11:38:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.349+0000 7f41211b0740 10 client.30484 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.349+0000 7f41211b0740 1 -- 172.21.15.47:0/341947909 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5555edfdea60 con 0x5555edf6d1e0 2024-03-21T11:38:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.352+0000 7f4101600640 1 -- 172.21.15.47:0/341947909 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f410801bd30 con 0x5555edf6d1e0 2024-03-21T11:38:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.352+0000 7f4101600640 1 -- 172.21.15.47:0/341947909 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f4108029630 con 0x5555edf6d1e0 2024-03-21T11:38:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.352+0000 7f41211b0740 10 client.30484 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.352+0000 7f41211b0740 10 client.30484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.352+0000 7f41211b0740 20 client.30484 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.352+0000 7f41211b0740 1 --2- 172.21.15.47:0/341947909 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5555edfdf410 0x5555edfe1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.352+0000 7f41211b0740 4 client.30484 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:38:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.352+0000 7f41211b0740 1 -- 172.21.15.47:0/341947909 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5555ededa0c0 con 0x5555edfdf410 2024-03-21T11:38:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.353+0000 7f4116a00640 1 --2- 172.21.15.47:0/341947909 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5555edfdf410 0x5555edfe1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.354+0000 7f4116a00640 1 --2- 172.21.15.47:0/341947909 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5555edfdf410 0x5555edfe1830 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9fb871518d82abe server_cookie=e486cf900b326c2c in_seq=0 out_seq=0 2024-03-21T11:38:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.354+0000 7f4101600640 10 client.30484 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.868+0000 7f4101600640 1 -- 172.21.15.47:0/341947909 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5555ededa0c0 con 0x5555edfdf410 2024-03-21T11:38:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.868+0000 7f4101600640 10 client.30484 handle_command_reply: tid=0 2024-03-21T11:38:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.868+0000 7f41211b0740 1 client.30484 shutdown 2024-03-21T11:38:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.870+0000 7f41211b0740 1 -- 172.21.15.47:0/341947909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6d1e0 msgr2=0x5555edf7c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.870+0000 7f41211b0740 1 --2- 172.21.15.47:0/341947909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6d1e0 0x5555edf7c0e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f4108011f10 tx=0x7f4108005630 comp rx=0 tx=0).stop 2024-03-21T11:38:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.870+0000 7f41211b0740 1 -- 172.21.15.47:0/341947909 shutdown_connections 2024-03-21T11:38:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.870+0000 7f41211b0740 1 --2- 172.21.15.47:0/341947909 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5555edfdf410 0x5555edfe1830 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.870+0000 7f41211b0740 1 --2- 172.21.15.47:0/341947909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5555edf72d40 0x5555edf7d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.870+0000 7f41211b0740 1 --2- 172.21.15.47:0/341947909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5555edf6dbd0 0x5555edf7c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.870+0000 7f41211b0740 1 --2- 172.21.15.47:0/341947909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5555edf6d1e0 0x5555edf7c0e0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.870+0000 7f41211b0740 1 -- 172.21.15.47:0/341947909 >> 172.21.15.47:0/341947909 conn(0x5555edf68750 msgr2=0x5555edf6eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.870+0000 7f4101600640 0 client.30484 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.871+0000 7f41211b0740 1 -- 172.21.15.47:0/341947909 shutdown_connections 2024-03-21T11:38:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.871+0000 7f41211b0740 1 -- 172.21.15.47:0/341947909 wait complete. 2024-03-21T11:38:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.871+0000 7f41211b0740 20 client.30484 trim_cache size 0 max 16384 2024-03-21T11:38:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:38:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:38:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:38:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:38:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.872+0000 7f411e400640 1 -- 172.21.15.47:0/3991531439 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40fc0a30c0 msgr2=0x7f40fc0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.872+0000 7f411e400640 1 --2- 172.21.15.47:0/3991531439 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40fc0a30c0 0x7f40fc0a5580 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f41080096c0 tx=0x7f4108009260 comp rx=0 tx=0).stop 2024-03-21T11:38:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.872+0000 7f411e400640 1 -- 172.21.15.47:0/3991531439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4118105250 msgr2=0x7f41181a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.872+0000 7f411e400640 1 --2- 172.21.15.47:0/3991531439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4118105250 0x7f41181a53e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f411002c430 tx=0x7f41100337b0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.872+0000 7f411e400640 1 -- 172.21.15.47:0/3991531439 shutdown_connections 2024-03-21T11:38:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.873+0000 7f411e400640 1 --2- 172.21.15.47:0/3991531439 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f40fc0a30c0 0x7f40fc0a5580 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.873+0000 7f411e400640 1 --2- 172.21.15.47:0/3991531439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4118105250 0x7f41181a53e0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.873+0000 7f411e400640 1 --2- 172.21.15.47:0/3991531439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41181028a0 0x7f411819e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.873+0000 7f411e400640 1 --2- 172.21.15.47:0/3991531439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41180fff30 0x7f411819de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.873+0000 7f411e400640 1 -- 172.21.15.47:0/3991531439 >> 172.21.15.47:0/3991531439 conn(0x7f41180fbb30 msgr2=0x7f4118100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.873+0000 7f411e400640 1 -- 172.21.15.47:0/3991531439 shutdown_connections 2024-03-21T11:38:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:10.873+0000 7f411e400640 1 -- 172.21.15.47:0/3991531439 wait complete. 2024-03-21T11:38:10.923 INFO:tasks.workunit.client.0.smithi047.stdout: 10 26812 30.03 MB/sec warmup 54 sec latency 324.818 ms 2024-03-21T11:38:11.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:11 smithi100 ceph-mon[36245]: pgmap v845: 129 pgs: 129 active+clean; 340 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 743 op/s 2024-03-21T11:38:11.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:11 smithi102 ceph-mon[36748]: pgmap v845: 129 pgs: 129 active+clean; 340 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 743 op/s 2024-03-21T11:38:11.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:11 smithi047 ceph-mon[31351]: pgmap v845: 129 pgs: 129 active+clean; 340 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 743 op/s 2024-03-21T11:38:11.923 INFO:tasks.workunit.client.0.smithi047.stdout: 10 27115 29.73 MB/sec warmup 55 sec latency 96.084 ms 2024-03-21T11:38:12.923 INFO:tasks.workunit.client.0.smithi047.stdout: 10 27589 29.54 MB/sec warmup 56 sec latency 68.293 ms 2024-03-21T11:38:13.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:13 smithi100 ceph-mon[36245]: pgmap v846: 129 pgs: 129 active+clean; 344 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 652 op/s 2024-03-21T11:38:13.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:13 smithi102 ceph-mon[36748]: pgmap v846: 129 pgs: 129 active+clean; 344 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 652 op/s 2024-03-21T11:38:13.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:13 smithi047 ceph-mon[31351]: pgmap v846: 129 pgs: 129 active+clean; 344 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 652 op/s 2024-03-21T11:38:13.923 INFO:tasks.workunit.client.0.smithi047.stdout: 10 28262 29.60 MB/sec warmup 57 sec latency 57.133 ms 2024-03-21T11:38:14.923 INFO:tasks.workunit.client.0.smithi047.stdout: 10 28942 29.85 MB/sec warmup 58 sec latency 42.943 ms 2024-03-21T11:38:15.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:15 smithi100 ceph-mon[36245]: pgmap v847: 129 pgs: 129 active+clean; 289 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 852 op/s 2024-03-21T11:38:15.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:15 smithi102 ceph-mon[36748]: pgmap v847: 129 pgs: 129 active+clean; 289 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 852 op/s 2024-03-21T11:38:15.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:15 smithi047 ceph-mon[31351]: pgmap v847: 129 pgs: 129 active+clean; 289 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 852 op/s 2024-03-21T11:38:15.923 INFO:tasks.workunit.client.0.smithi047.stdout: 10 29502 29.91 MB/sec warmup 59 sec latency 54.670 ms 2024-03-21T11:38:16.924 INFO:tasks.workunit.client.0.smithi047.stdout: 10 30108 29.96 MB/sec warmup 60 sec latency 74.969 ms 2024-03-21T11:38:17.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:17 smithi100 ceph-mon[36245]: pgmap v848: 129 pgs: 129 active+clean; 268 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 757 op/s 2024-03-21T11:38:17.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:17 smithi102 ceph-mon[36748]: pgmap v848: 129 pgs: 129 active+clean; 268 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 757 op/s 2024-03-21T11:38:17.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:17 smithi047 ceph-mon[31351]: pgmap v848: 129 pgs: 129 active+clean; 268 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 757 op/s 2024-03-21T11:38:17.923 INFO:tasks.workunit.client.0.smithi047.stdout: 10 30610 29.86 MB/sec warmup 61 sec latency 131.989 ms 2024-03-21T11:38:18.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:18 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:38:18.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:18 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:38:18.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:18 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:38:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:18 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:38:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:18 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:38:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:18 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:38:18.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:18 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:38:18.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:18 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:38:18.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:18 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:38:18.923 INFO:tasks.workunit.client.0.smithi047.stdout: 10 31250 30.00 MB/sec warmup 62 sec latency 45.101 ms 2024-03-21T11:38:19.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:19 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:38:19.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:19 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:38:19.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:19 smithi100 ceph-mon[36245]: pgmap v849: 129 pgs: 129 active+clean; 269 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 11 MiB/s wr, 781 op/s 2024-03-21T11:38:19.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:19 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:38:19.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:19 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:38:19.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:19 smithi102 ceph-mon[36748]: pgmap v849: 129 pgs: 129 active+clean; 269 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 11 MiB/s wr, 781 op/s 2024-03-21T11:38:19.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:19 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:38:19.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:19 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:38:19.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:19 smithi047 ceph-mon[31351]: pgmap v849: 129 pgs: 129 active+clean; 269 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 11 MiB/s wr, 781 op/s 2024-03-21T11:38:19.924 INFO:tasks.workunit.client.0.smithi047.stdout: 10 31749 29.91 MB/sec warmup 63 sec latency 50.843 ms 2024-03-21T11:38:20.924 INFO:tasks.workunit.client.0.smithi047.stdout: 10 32137 29.79 MB/sec warmup 64 sec latency 81.190 ms 2024-03-21T11:38:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:21 smithi100 ceph-mon[36245]: pgmap v850: 129 pgs: 129 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 792 op/s 2024-03-21T11:38:21.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:21 smithi102 ceph-mon[36748]: pgmap v850: 129 pgs: 129 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 792 op/s 2024-03-21T11:38:21.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:21 smithi047 ceph-mon[31351]: pgmap v850: 129 pgs: 129 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 792 op/s 2024-03-21T11:38:21.924 INFO:tasks.workunit.client.0.smithi047.stdout: 10 32731 29.78 MB/sec warmup 65 sec latency 49.546 ms 2024-03-21T11:38:22.924 INFO:tasks.workunit.client.0.smithi047.stdout: 10 33236 29.68 MB/sec warmup 66 sec latency 206.842 ms 2024-03-21T11:38:23.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:23 smithi100 ceph-mon[36245]: pgmap v851: 129 pgs: 129 active+clean; 280 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.3 MiB/s wr, 674 op/s 2024-03-21T11:38:23.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:23 smithi102 ceph-mon[36748]: pgmap v851: 129 pgs: 129 active+clean; 280 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.3 MiB/s wr, 674 op/s 2024-03-21T11:38:23.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:23 smithi047 ceph-mon[31351]: pgmap v851: 129 pgs: 129 active+clean; 280 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.3 MiB/s wr, 674 op/s 2024-03-21T11:38:23.924 INFO:tasks.workunit.client.0.smithi047.stdout: 10 33874 29.77 MB/sec warmup 67 sec latency 70.754 ms 2024-03-21T11:38:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.056+0000 7fdaa6400640 1 -- 172.21.15.47:0/502362673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa0104910 msgr2=0x7fdaa0104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.056+0000 7fdaa6400640 1 --2- 172.21.15.47:0/502362673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa0104910 0x7fdaa0104d90 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fda94009900 tx=0x7fda9402bc60 comp rx=0 tx=0).stop 2024-03-21T11:38:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.056+0000 7fdaa6400640 1 -- 172.21.15.47:0/502362673 shutdown_connections 2024-03-21T11:38:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.056+0000 7fdaa6400640 1 --2- 172.21.15.47:0/502362673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaa01052d0 0x7fdaa010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.056+0000 7fdaa6400640 1 --2- 172.21.15.47:0/502362673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa0104910 0x7fdaa0104d90 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.056+0000 7fdaa6400640 1 --2- 172.21.15.47:0/502362673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaa0103720 0x7fdaa0103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.056+0000 7fdaa6400640 1 -- 172.21.15.47:0/502362673 >> 172.21.15.47:0/502362673 conn(0x7fdaa00fecb0 msgr2=0x7fdaa01010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.056+0000 7fdaa6400640 1 -- 172.21.15.47:0/502362673 shutdown_connections 2024-03-21T11:38:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.056+0000 7fdaa6400640 1 -- 172.21.15.47:0/502362673 wait complete. 2024-03-21T11:38:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.058+0000 7fdaa6400640 1 Processor -- start 2024-03-21T11:38:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.058+0000 7fdaa6400640 1 -- start start 2024-03-21T11:38:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.058+0000 7fdaa6400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaa0103720 0x7fdaa0199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.058+0000 7fdaa6400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa0104910 0x7fdaa0199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.058+0000 7fdaa6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaa01052d0 0x7fdaa01a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.058+0000 7fdaa6400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa010f460 con 0x7fdaa0103720 2024-03-21T11:38:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.058+0000 7fdaa6400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa010f2e0 con 0x7fdaa01052d0 2024-03-21T11:38:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.058+0000 7fdaa6400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa010f5e0 con 0x7fdaa0104910 2024-03-21T11:38:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.059+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa0104910 0x7fdaa0199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.059+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa0104910 0x7fdaa0199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57392/0 (socket says 172.21.15.47:57392) 2024-03-21T11:38:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.059+0000 7fda9fe00640 1 -- 172.21.15.47:0/3463668887 learned_addr learned my addr 172.21.15.47:0/3463668887 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.059+0000 7fdaa5000640 1 --2- 172.21.15.47:0/3463668887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaa0103720 0x7fdaa0199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.059+0000 7fdaa5a00640 1 --2- 172.21.15.47:0/3463668887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaa01052d0 0x7fdaa01a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.059+0000 7fda9fe00640 1 -- 172.21.15.47:0/3463668887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaa01052d0 msgr2=0x7fdaa01a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.059+0000 7fda9fe00640 1 --2- 172.21.15.47:0/3463668887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaa01052d0 0x7fdaa01a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.059+0000 7fda9fe00640 1 -- 172.21.15.47:0/3463668887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaa0103720 msgr2=0x7fdaa0199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.059+0000 7fda9fe00640 1 --2- 172.21.15.47:0/3463668887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaa0103720 0x7fdaa0199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.059+0000 7fda9fe00640 1 -- 172.21.15.47:0/3463668887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa01a1690 con 0x7fdaa0104910 2024-03-21T11:38:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.060+0000 7fda9fe00640 1 --2- 172.21.15.47:0/3463668887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa0104910 0x7fdaa0199fb0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fda94005ab0 tx=0x7fda94005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.060+0000 7fda9d600640 1 -- 172.21.15.47:0/3463668887 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94041070 con 0x7fdaa0104910 2024-03-21T11:38:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.060+0000 7fdaa6400640 1 -- 172.21.15.47:0/3463668887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa01a1920 con 0x7fdaa0104910 2024-03-21T11:38:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.060+0000 7fda9d600640 1 -- 172.21.15.47:0/3463668887 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda940054e0 con 0x7fdaa0104910 2024-03-21T11:38:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.060+0000 7fda9d600640 1 -- 172.21.15.47:0/3463668887 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94034440 con 0x7fdaa0104910 2024-03-21T11:38:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.061+0000 7fdaa6400640 1 -- 172.21.15.47:0/3463668887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa01a1e30 con 0x7fdaa0104910 2024-03-21T11:38:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.065+0000 7fda9d600640 1 -- 172.21.15.47:0/3463668887 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fda94044070 con 0x7fdaa0104910 2024-03-21T11:38:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.067+0000 7fda9d600640 1 --2- 172.21.15.47:0/3463668887 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda880a2f90 0x7fda880a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.067+0000 7fda9d600640 1 -- 172.21.15.47:0/3463668887 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fda940f1430 con 0x7fdaa0104910 2024-03-21T11:38:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.067+0000 7fdaa5000640 1 --2- 172.21.15.47:0/3463668887 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda880a2f90 0x7fda880a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.069+0000 7fdaaa322740 1 Processor -- start 2024-03-21T11:38:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.069+0000 7fdaaa322740 1 -- start start 2024-03-21T11:38:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.069+0000 7fdaa5000640 1 --2- 172.21.15.47:0/3463668887 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda880a2f90 0x7fda880a5450 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0104770 tx=0x7fda8c0090e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.069+0000 7fdaaa322740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb1c00 0x561275bb2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.069+0000 7fdaaa322740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275bb70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.069+0000 7fdaaa322740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb7760 0x561275bb9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.069+0000 7fdaaa322740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561275b6c3b0 con 0x561275bb25f0 2024-03-21T11:38:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.069+0000 7fdaaa322740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561275b0e830 con 0x561275bb1c00 2024-03-21T11:38:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.069+0000 7fdaaa322740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561275ab75d0 con 0x561275bb7760 2024-03-21T11:38:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.069+0000 7fdaa5a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb7760 0x561275bb9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.070+0000 7fdaa5a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb7760 0x561275bb9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57408/0 (socket says 172.21.15.47:57408) 2024-03-21T11:38:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.070+0000 7fdaa5a00640 1 -- 172.21.15.47:0/2750888109 learned_addr learned my addr 172.21.15.47:0/2750888109 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.070+0000 7fdaa5000640 1 --2- 172.21.15.47:0/2750888109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb1c00 0x561275bb2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.070+0000 7fda9fe00640 1 --2- 172.21.15.47:0/2750888109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275bb70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.070+0000 7fdaa5a00640 1 -- 172.21.15.47:0/2750888109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb1c00 msgr2=0x561275bb2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.070+0000 7fdaa5a00640 1 --2- 172.21.15.47:0/2750888109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb1c00 0x561275bb2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.070+0000 7fdaa5a00640 1 -- 172.21.15.47:0/2750888109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 msgr2=0x561275bb70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.070+0000 7fdaa5a00640 1 --2- 172.21.15.47:0/2750888109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275bb70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.070+0000 7fdaa5a00640 1 -- 172.21.15.47:0/2750888109 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561275acf2c0 con 0x561275bb7760 2024-03-21T11:38:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.070+0000 7fda9fe00640 1 --2- 172.21.15.47:0/2750888109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275bb70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:38:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.070+0000 7fdaa5000640 1 --2- 172.21.15.47:0/2750888109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb1c00 0x561275bb2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.071+0000 7fdaa5a00640 1 --2- 172.21.15.47:0/2750888109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb7760 0x561275bb9bf0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fda98010370 tx=0x7fda98010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f323d66d9f6849 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.071+0000 7fda92000640 1 -- 172.21.15.47:0/2750888109 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98016070 con 0x561275bb7760 2024-03-21T11:38:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.071+0000 7fda92000640 1 -- 172.21.15.47:0/2750888109 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda98004d20 con 0x561275bb7760 2024-03-21T11:38:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.071+0000 7fda92000640 1 -- 172.21.15.47:0/2750888109 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98005480 con 0x561275bb7760 2024-03-21T11:38:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.072+0000 7fdaaa322740 1 -- 172.21.15.47:0/2750888109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb7760 msgr2=0x561275bb9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.072+0000 7fdaaa322740 1 --2- 172.21.15.47:0/2750888109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb7760 0x561275bb9bf0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fda98010370 tx=0x7fda98010830 comp rx=0 tx=0).stop 2024-03-21T11:38:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.073+0000 7fdaaa322740 1 -- 172.21.15.47:0/2750888109 shutdown_connections 2024-03-21T11:38:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.073+0000 7fdaaa322740 1 --2- 172.21.15.47:0/2750888109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb7760 0x561275bb9bf0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.073+0000 7fdaaa322740 1 --2- 172.21.15.47:0/2750888109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275bb70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.073+0000 7fdaaa322740 1 --2- 172.21.15.47:0/2750888109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb1c00 0x561275bb2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.073+0000 7fdaaa322740 1 -- 172.21.15.47:0/2750888109 >> 172.21.15.47:0/2750888109 conn(0x561275ba4960 msgr2=0x561275ba4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.073+0000 7fdaaa322740 1 -- 172.21.15.47:0/2750888109 shutdown_connections 2024-03-21T11:38:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.073+0000 7fdaaa322740 1 -- 172.21.15.47:0/2750888109 wait complete. 2024-03-21T11:38:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.074+0000 7fdaaa322740 1 -- start start 2024-03-21T11:38:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.074+0000 7fdaaa322740 1 Processor -- start 2024-03-21T11:38:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.075+0000 7fdaaa322740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb1c00 0x561275bbbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.075+0000 7fdaaa322740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275bbc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.075+0000 7fdaaa322740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb7760 0x561275bc3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.075+0000 7fdaaa322740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561275acaa20 con 0x561275bb25f0 2024-03-21T11:38:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.075+0000 7fdaaa322740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561275b0a8d0 con 0x561275bb7760 2024-03-21T11:38:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.075+0000 7fdaaa322740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561275bba710 con 0x561275bb1c00 2024-03-21T11:38:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.075+0000 7fdaaa322740 10 client.0 _ll_register_callbacks cb 0x561275b071c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.075+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275bbc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.075+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275bbc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51416/0 (socket says 172.21.15.47:51416) 2024-03-21T11:38:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.075+0000 7fda9fe00640 1 -- 172.21.15.47:0/2888271395 learned_addr learned my addr 172.21.15.47:0/2888271395 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.075+0000 7fdaa5000640 1 --2- 172.21.15.47:0/2888271395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb1c00 0x561275bbbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.076+0000 7fda9fe00640 1 -- 172.21.15.47:0/2888271395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb1c00 msgr2=0x561275bbbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.076+0000 7fda9fe00640 1 --2- 172.21.15.47:0/2888271395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb1c00 0x561275bbbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.076+0000 7fda9fe00640 1 -- 172.21.15.47:0/2888271395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb7760 msgr2=0x561275bc3310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:38:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.076+0000 7fda9fe00640 1 --2- 172.21.15.47:0/2888271395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb7760 0x561275bc3310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.076+0000 7fda9fe00640 1 -- 172.21.15.47:0/2888271395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561275823210 con 0x561275bb25f0 2024-03-21T11:38:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.076+0000 7fda9fe00640 1 --2- 172.21.15.47:0/2888271395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275bbc2d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fda940c5c50 tx=0x7fda9402bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d40217345c6681 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.076+0000 7fda86a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:38:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.077+0000 7fda86a00640 1 -- 172.21.15.47:0/2888271395 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9404b040 con 0x561275bb25f0 2024-03-21T11:38:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.077+0000 7fda86a00640 1 -- 172.21.15.47:0/2888271395 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fda940a80b0 con 0x561275bb25f0 2024-03-21T11:38:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.077+0000 7fdaaa322740 1 -- 172.21.15.47:0/2888271395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda98004d20 con 0x561275bb25f0 2024-03-21T11:38:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.077+0000 7fda86a00640 1 -- 172.21.15.47:0/2888271395 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda940b9ac0 con 0x561275bb25f0 2024-03-21T11:38:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.078+0000 7fdaaa322740 10 client.20052 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.078+0000 7fdaaa322740 1 -- 172.21.15.47:0/2888271395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5612758ef620 con 0x561275bb25f0 2024-03-21T11:38:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.078+0000 7fda86a00640 1 -- 172.21.15.47:0/2888271395 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda98004d20 con 0x561275bb25f0 2024-03-21T11:38:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.079+0000 7fda86a00640 1 -- 172.21.15.47:0/2888271395 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fda940bcc70 con 0x561275bb25f0 2024-03-21T11:38:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.079+0000 7fdaaa322740 10 client.20052 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.079+0000 7fdaaa322740 10 client.20052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.079+0000 7fdaaa322740 20 client.20052 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.079+0000 7fdaaa322740 1 --2- 172.21.15.47:0/2888271395 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561275bc7c30 0x561275bca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.079+0000 7fdaaa322740 4 client.20052 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.079+0000 7fdaaa322740 1 -- 172.21.15.47:0/2888271395 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561275b1bd70 con 0x561275bc7c30 2024-03-21T11:38:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.079+0000 7fdaa5000640 1 --2- 172.21.15.47:0/2888271395 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561275bc7c30 0x561275bca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.081+0000 7fdaa5000640 1 --2- 172.21.15.47:0/2888271395 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561275bc7c30 0x561275bca0d0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9fb13725d8a9645 server_cookie=ea49f0b90f9202ef in_seq=0 out_seq=0 2024-03-21T11:38:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.081+0000 7fda86a00640 10 client.20052 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.087+0000 7fda86a00640 1 -- 172.21.15.47:0/2888271395 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561275b1bd70 con 0x561275bc7c30 2024-03-21T11:38:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.087+0000 7fda86a00640 10 client.20052 handle_command_reply: tid=0 2024-03-21T11:38:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.087+0000 7fdaaa322740 1 client.20052 shutdown 2024-03-21T11:38:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.087+0000 7fdaaa322740 1 -- 172.21.15.47:0/2888271395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 msgr2=0x561275bbc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.087+0000 7fdaaa322740 1 --2- 172.21.15.47:0/2888271395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275bbc2d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fda940c5c50 tx=0x7fda9402bc60 comp rx=0 tx=0).stop 2024-03-21T11:38:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.088+0000 7fdaaa322740 1 -- 172.21.15.47:0/2888271395 shutdown_connections 2024-03-21T11:38:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.088+0000 7fdaaa322740 1 --2- 172.21.15.47:0/2888271395 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561275bc7c30 0x561275bca0d0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.088+0000 7fdaaa322740 1 --2- 172.21.15.47:0/2888271395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb7760 0x561275bc3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.088+0000 7fdaaa322740 1 --2- 172.21.15.47:0/2888271395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275bbc2d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.088+0000 7fdaaa322740 1 --2- 172.21.15.47:0/2888271395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb1c00 0x561275bbbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.088+0000 7fda86a00640 0 client.20052 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.088+0000 7fdaaa322740 1 -- 172.21.15.47:0/2888271395 >> 172.21.15.47:0/2888271395 conn(0x561275ba4960 msgr2=0x561275ba3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.088+0000 7fdaaa322740 1 -- 172.21.15.47:0/2888271395 shutdown_connections 2024-03-21T11:38:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.088+0000 7fdaaa322740 1 -- 172.21.15.47:0/2888271395 wait complete. 2024-03-21T11:38:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.089+0000 7fdaaa322740 20 client.20052 trim_cache size 0 max 16384 2024-03-21T11:38:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.114+0000 7fdaaa322740 1 Processor -- start 2024-03-21T11:38:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.114+0000 7fdaaa322740 1 -- start start 2024-03-21T11:38:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.114+0000 7fdaaa322740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275ba4960 0x561275ba7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.114+0000 7fdaaa322740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb1c00 0x561275bc32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.114+0000 7fdaaa322740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275ba8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.114+0000 7fdaaa322740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa01a7d30 con 0x561275bb25f0 2024-03-21T11:38:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.114+0000 7fdaaa322740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa01a9830 con 0x561275ba4960 2024-03-21T11:38:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.114+0000 7fdaaa322740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa01a8390 con 0x561275bb1c00 2024-03-21T11:38:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.115+0000 7fdaa5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275ba4960 0x561275ba7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.115+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb1c00 0x561275bc32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.115+0000 7fdaa5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275ba4960 0x561275ba7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40832/0 (socket says 172.21.15.47:40832) 2024-03-21T11:38:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.115+0000 7fdaa5000640 1 -- 172.21.15.47:0/3755099119 learned_addr learned my addr 172.21.15.47:0/3755099119 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.115+0000 7fdaa5a00640 1 --2- 172.21.15.47:0/3755099119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275ba8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.115+0000 7fdaa5000640 1 -- 172.21.15.47:0/3755099119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb1c00 msgr2=0x561275bc32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.115+0000 7fdaa5000640 1 --2- 172.21.15.47:0/3755099119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb1c00 0x561275bc32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.115+0000 7fdaa5000640 1 -- 172.21.15.47:0/3755099119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 msgr2=0x561275ba8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.115+0000 7fdaa5000640 1 --2- 172.21.15.47:0/3755099119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275ba8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.115+0000 7fdaa5000640 1 -- 172.21.15.47:0/3755099119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561275bba850 con 0x561275ba4960 2024-03-21T11:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.115+0000 7fdaa5000640 1 --2- 172.21.15.47:0/3755099119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275ba4960 0x561275ba7bc0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x561275bca940 tx=0x7fda8c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc1b59819eea97b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.116+0000 7fda91600640 1 -- 172.21.15.47:0/3755099119 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c015b10 con 0x561275ba4960 2024-03-21T11:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.116+0000 7fda91600640 1 -- 172.21.15.47:0/3755099119 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda8c015cb0 con 0x561275ba4960 2024-03-21T11:38:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.116+0000 7fda91600640 1 -- 172.21.15.47:0/3755099119 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c0135d0 con 0x561275ba4960 2024-03-21T11:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.116+0000 7fdaaa322740 1 -- 172.21.15.47:0/3755099119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275ba4960 msgr2=0x561275ba7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.116+0000 7fdaaa322740 1 --2- 172.21.15.47:0/3755099119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275ba4960 0x561275ba7bc0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x561275bca940 tx=0x7fda8c011000 comp rx=0 tx=0).stop 2024-03-21T11:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.117+0000 7fdaaa322740 1 -- 172.21.15.47:0/3755099119 shutdown_connections 2024-03-21T11:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.117+0000 7fdaaa322740 1 --2- 172.21.15.47:0/3755099119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275bb25f0 0x561275ba8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.117+0000 7fdaaa322740 1 --2- 172.21.15.47:0/3755099119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb1c00 0x561275bc32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.117+0000 7fdaaa322740 1 --2- 172.21.15.47:0/3755099119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275ba4960 0x561275ba7bc0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.117+0000 7fdaaa322740 1 -- 172.21.15.47:0/3755099119 >> 172.21.15.47:0/3755099119 conn(0x561275bb31f0 msgr2=0x561275bbbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.117+0000 7fdaaa322740 1 -- 172.21.15.47:0/3755099119 shutdown_connections 2024-03-21T11:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.117+0000 7fdaaa322740 1 -- 172.21.15.47:0/3755099119 wait complete. 2024-03-21T11:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.117+0000 7fdaaa322740 1 -- start start 2024-03-21T11:38:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.117+0000 7fdaaa322740 1 Processor -- start 2024-03-21T11:38:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fdaaa322740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275ba4960 0x561275bc0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fdaaa322740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb1c00 0x561275bc1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fdaaa322740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb25f0 0x561275bc2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fdaaa322740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa01a9110 con 0x561275ba4960 2024-03-21T11:38:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fdaaa322740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa01a8ab0 con 0x561275bb1c00 2024-03-21T11:38:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fdaaa322740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa01a5ef0 con 0x561275bb25f0 2024-03-21T11:38:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fdaaa322740 10 client.0 _ll_register_callbacks cb 0x7fda940b9480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb1c00 0x561275bc1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fdaa5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275ba4960 0x561275bc0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb1c00 0x561275bc1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40836/0 (socket says 172.21.15.47:40836) 2024-03-21T11:38:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fda9fe00640 1 -- 172.21.15.47:0/804494358 learned_addr learned my addr 172.21.15.47:0/804494358 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fdaa5a00640 1 --2- 172.21.15.47:0/804494358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb25f0 0x561275bc2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fda9fe00640 1 -- 172.21.15.47:0/804494358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb25f0 msgr2=0x561275bc2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fda9fe00640 1 --2- 172.21.15.47:0/804494358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb25f0 0x561275bc2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fda9fe00640 1 -- 172.21.15.47:0/804494358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275ba4960 msgr2=0x561275bc0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fda9fe00640 1 --2- 172.21.15.47:0/804494358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275ba4960 0x561275bc0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.118+0000 7fda9fe00640 1 -- 172.21.15.47:0/804494358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561275bc2b30 con 0x561275bb1c00 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.119+0000 7fda9fe00640 1 --2- 172.21.15.47:0/804494358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb1c00 0x561275bc1530 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fda9403d020 tx=0x7fda940eebc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e1559e87f2f4d66 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.119+0000 7fda86a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.119+0000 7fda86a00640 1 -- 172.21.15.47:0/804494358 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9404b050 con 0x561275bb1c00 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.119+0000 7fda86a00640 1 -- 172.21.15.47:0/804494358 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fda94005000 con 0x561275bb1c00 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.119+0000 7fdaaa322740 1 -- 172.21.15.47:0/804494358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda8c015cb0 con 0x561275bb1c00 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.120+0000 7fda86a00640 1 -- 172.21.15.47:0/804494358 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda94005690 con 0x561275bb1c00 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.120+0000 7fdaaa322740 10 client.30500 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.120+0000 7fdaaa322740 1 -- 172.21.15.47:0/804494358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561275c23be0 con 0x561275bb1c00 2024-03-21T11:38:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.120+0000 7fda86a00640 1 -- 172.21.15.47:0/804494358 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda8c015cb0 con 0x561275bb1c00 2024-03-21T11:38:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.120+0000 7fda86a00640 1 -- 172.21.15.47:0/804494358 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fda940bc6e0 con 0x561275bb1c00 2024-03-21T11:38:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.120+0000 7fdaaa322740 10 client.30500 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.120+0000 7fdaaa322740 10 client.30500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.120+0000 7fdaaa322740 20 client.30500 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.120+0000 7fdaaa322740 1 --2- 172.21.15.47:0/804494358 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561275bb7760 0x561275c26420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.120+0000 7fdaaa322740 4 client.30500 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:38:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.120+0000 7fdaaa322740 1 -- 172.21.15.47:0/804494358 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561275b160c0 con 0x561275bb7760 2024-03-21T11:38:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.121+0000 7fdaa5000640 1 --2- 172.21.15.47:0/804494358 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561275bb7760 0x561275c26420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.121+0000 7fdaa5000640 1 --2- 172.21.15.47:0/804494358 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561275bb7760 0x561275c26420 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eb8f921e8f6c7c5 server_cookie=7fa2361de696c9d6 in_seq=0 out_seq=0 2024-03-21T11:38:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.121+0000 7fda86a00640 10 client.30500 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:24.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:24 smithi100 ceph-mon[36245]: pgmap v852: 129 pgs: 129 active+clean; 288 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.4 MiB/s wr, 699 op/s 2024-03-21T11:38:24.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:24 smithi102 ceph-mon[36748]: pgmap v852: 129 pgs: 129 active+clean; 288 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.4 MiB/s wr, 699 op/s 2024-03-21T11:38:24.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:24 smithi047 ceph-mon[31351]: pgmap v852: 129 pgs: 129 active+clean; 288 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.4 MiB/s wr, 699 op/s 2024-03-21T11:38:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.833+0000 7fda86a00640 1 -- 172.21.15.47:0/804494358 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561275b160c0 con 0x561275bb7760 2024-03-21T11:38:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.833+0000 7fda86a00640 10 client.30500 handle_command_reply: tid=0 2024-03-21T11:38:24.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.833+0000 7fdaaa322740 1 client.30500 shutdown 2024-03-21T11:38:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.836+0000 7fdaaa322740 1 -- 172.21.15.47:0/804494358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb1c00 msgr2=0x561275bc1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.836+0000 7fdaaa322740 1 --2- 172.21.15.47:0/804494358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb1c00 0x561275bc1530 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fda9403d020 tx=0x7fda940eebc0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.836+0000 7fdaaa322740 1 -- 172.21.15.47:0/804494358 shutdown_connections 2024-03-21T11:38:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.836+0000 7fdaaa322740 1 --2- 172.21.15.47:0/804494358 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561275bb7760 0x561275c26420 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.836+0000 7fdaaa322740 1 --2- 172.21.15.47:0/804494358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561275bb25f0 0x561275bc2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.836+0000 7fdaaa322740 1 --2- 172.21.15.47:0/804494358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561275bb1c00 0x561275bc1530 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.836+0000 7fdaaa322740 1 --2- 172.21.15.47:0/804494358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561275ba4960 0x561275bc0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.836+0000 7fdaaa322740 1 -- 172.21.15.47:0/804494358 >> 172.21.15.47:0/804494358 conn(0x561275bb31f0 msgr2=0x561275ba7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.836+0000 7fda86a00640 0 client.30500 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.836+0000 7fdaaa322740 1 -- 172.21.15.47:0/804494358 shutdown_connections 2024-03-21T11:38:24.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.836+0000 7fdaaa322740 1 -- 172.21.15.47:0/804494358 wait complete. 2024-03-21T11:38:24.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.837+0000 7fdaaa322740 20 client.30500 trim_cache size 0 max 16384 2024-03-21T11:38:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:38:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:38:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:38:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:38:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.839+0000 7fdaa6400640 1 -- 172.21.15.47:0/3463668887 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda880a2f90 msgr2=0x7fda880a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.839+0000 7fdaa6400640 1 --2- 172.21.15.47:0/3463668887 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda880a2f90 0x7fda880a5450 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0104770 tx=0x7fda8c0090e0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.839+0000 7fdaa6400640 1 -- 172.21.15.47:0/3463668887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa0104910 msgr2=0x7fdaa0199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.839+0000 7fdaa6400640 1 --2- 172.21.15.47:0/3463668887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa0104910 0x7fdaa0199fb0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fda94005ab0 tx=0x7fda94005e40 comp rx=0 tx=0).stop 2024-03-21T11:38:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.839+0000 7fdaa6400640 1 -- 172.21.15.47:0/3463668887 shutdown_connections 2024-03-21T11:38:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.839+0000 7fdaa6400640 1 --2- 172.21.15.47:0/3463668887 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fda880a2f90 0x7fda880a5450 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.839+0000 7fdaa6400640 1 --2- 172.21.15.47:0/3463668887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdaa01052d0 0x7fdaa01a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.839+0000 7fdaa6400640 1 --2- 172.21.15.47:0/3463668887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaa0104910 0x7fdaa0199fb0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.839+0000 7fdaa6400640 1 --2- 172.21.15.47:0/3463668887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdaa0103720 0x7fdaa0199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.839+0000 7fdaa6400640 1 -- 172.21.15.47:0/3463668887 >> 172.21.15.47:0/3463668887 conn(0x7fdaa00fecb0 msgr2=0x7fdaa0107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.839+0000 7fdaa6400640 1 -- 172.21.15.47:0/3463668887 shutdown_connections 2024-03-21T11:38:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:24.839+0000 7fdaa6400640 1 -- 172.21.15.47:0/3463668887 wait complete. 2024-03-21T11:38:24.924 INFO:tasks.workunit.client.0.smithi047.stdout: 10 34188 29.62 MB/sec warmup 68 sec latency 641.445 ms 2024-03-21T11:38:25.924 INFO:tasks.workunit.client.0.smithi047.stdout: 10 34850 29.73 MB/sec warmup 69 sec latency 71.606 ms 2024-03-21T11:38:26.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:26 smithi100 ceph-mon[36245]: pgmap v853: 129 pgs: 129 active+clean; 289 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.8 MiB/s wr, 430 op/s 2024-03-21T11:38:26.924 INFO:tasks.workunit.client.0.smithi047.stdout: 10 35427 29.83 MB/sec warmup 70 sec latency 54.861 ms 2024-03-21T11:38:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:26 smithi102 ceph-mon[36748]: pgmap v853: 129 pgs: 129 active+clean; 289 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.8 MiB/s wr, 430 op/s 2024-03-21T11:38:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:26 smithi047 ceph-mon[31351]: pgmap v853: 129 pgs: 129 active+clean; 289 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.8 MiB/s wr, 430 op/s 2024-03-21T11:38:27.924 INFO:tasks.workunit.client.0.smithi047.stdout: 10 35827 29.73 MB/sec warmup 71 sec latency 151.205 ms 2024-03-21T11:38:28.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:28 smithi100 ceph-mon[36245]: pgmap v854: 129 pgs: 129 active+clean; 286 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.3 MiB/s wr, 523 op/s 2024-03-21T11:38:28.924 INFO:tasks.workunit.client.0.smithi047.stdout: 10 36101 29.49 MB/sec warmup 72 sec latency 348.496 ms 2024-03-21T11:38:28.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:28 smithi102 ceph-mon[36748]: pgmap v854: 129 pgs: 129 active+clean; 286 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.3 MiB/s wr, 523 op/s 2024-03-21T11:38:29.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:28 smithi047 ceph-mon[31351]: pgmap v854: 129 pgs: 129 active+clean; 286 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.3 MiB/s wr, 523 op/s 2024-03-21T11:38:29.924 INFO:tasks.workunit.client.0.smithi047.stdout: 10 36579 29.41 MB/sec warmup 73 sec latency 101.910 ms 2024-03-21T11:38:30.924 INFO:tasks.workunit.client.0.smithi047.stdout: 10 37192 29.30 MB/sec warmup 74 sec latency 84.586 ms 2024-03-21T11:38:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:30 smithi047 ceph-mon[31351]: pgmap v855: 129 pgs: 129 active+clean; 294 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 607 op/s 2024-03-21T11:38:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:30 smithi100 ceph-mon[36245]: pgmap v855: 129 pgs: 129 active+clean; 294 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 607 op/s 2024-03-21T11:38:31.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:30 smithi102 ceph-mon[36748]: pgmap v855: 129 pgs: 129 active+clean; 294 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 607 op/s 2024-03-21T11:38:31.924 INFO:tasks.workunit.client.0.smithi047.stdout: 10 37866 29.48 MB/sec warmup 75 sec latency 94.325 ms 2024-03-21T11:38:32.925 INFO:tasks.workunit.client.0.smithi047.stdout: 10 38551 29.59 MB/sec warmup 76 sec latency 68.885 ms 2024-03-21T11:38:33.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:33 smithi047 ceph-mon[31351]: pgmap v856: 129 pgs: 129 active+clean; 296 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-03-21T11:38:33.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:38:33.360 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:38:33.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:33 smithi100 ceph-mon[36245]: pgmap v856: 129 pgs: 129 active+clean; 296 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-03-21T11:38:33.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:33 smithi102 ceph-mon[36748]: pgmap v856: 129 pgs: 129 active+clean; 296 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-03-21T11:38:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.732+0000 7f2661000640 1 -- 172.21.15.47:0/3929519409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f265c105490 msgr2=0x7f265c10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.732+0000 7f2661000640 1 --2- 172.21.15.47:0/3929519409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f265c105490 0x7f265c10bd20 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2654009ca0 tx=0x7f265402c100 comp rx=0 tx=0).stop 2024-03-21T11:38:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.732+0000 7f2661000640 1 -- 172.21.15.47:0/3929519409 shutdown_connections 2024-03-21T11:38:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.733+0000 7f2661000640 1 --2- 172.21.15.47:0/3929519409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f265c105490 0x7f265c10bd20 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.733+0000 7f2661000640 1 --2- 172.21.15.47:0/3929519409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f265c104ad0 0x7f265c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.733+0000 7f2661000640 1 --2- 172.21.15.47:0/3929519409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f265c1038e0 0x7f265c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.733+0000 7f2661000640 1 -- 172.21.15.47:0/3929519409 >> 172.21.15.47:0/3929519409 conn(0x7f265c0fee50 msgr2=0x7f265c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.733+0000 7f2661000640 1 -- 172.21.15.47:0/3929519409 shutdown_connections 2024-03-21T11:38:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.733+0000 7f2661000640 1 -- 172.21.15.47:0/3929519409 wait complete. 2024-03-21T11:38:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.736+0000 7f2661000640 1 Processor -- start 2024-03-21T11:38:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.736+0000 7f2661000640 1 -- start start 2024-03-21T11:38:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.738+0000 7f2661000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f265c1038e0 0x7f265c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.738+0000 7f2661000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f265c104ad0 0x7f265c19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.738+0000 7f2661000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f265c105490 0x7f265c1a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.738+0000 7f2661000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c10f540 con 0x7f265c1038e0 2024-03-21T11:38:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.738+0000 7f2661000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c10f3c0 con 0x7f265c104ad0 2024-03-21T11:38:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.738+0000 7f2661000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c10f6c0 con 0x7f265c105490 2024-03-21T11:38:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.738+0000 7f265a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f265c105490 0x7f265c1a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.738+0000 7f265a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f265c105490 0x7f265c1a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60186/0 (socket says 172.21.15.47:60186) 2024-03-21T11:38:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.738+0000 7f265a800640 1 -- 172.21.15.47:0/1349246775 learned_addr learned my addr 172.21.15.47:0/1349246775 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.738+0000 7f2659e00640 1 --2- 172.21.15.47:0/1349246775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f265c1038e0 0x7f265c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.739+0000 7f265a800640 1 -- 172.21.15.47:0/1349246775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f265c104ad0 msgr2=0x7f265c19e4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T11:38:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.739+0000 7f265a800640 1 --2- 172.21.15.47:0/1349246775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f265c104ad0 0x7f265c19e4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.739+0000 7f265a800640 1 -- 172.21.15.47:0/1349246775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f265c1038e0 msgr2=0x7f265c19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.739+0000 7f265a800640 1 --2- 172.21.15.47:0/1349246775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f265c1038e0 0x7f265c19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.739+0000 7f265a800640 1 -- 172.21.15.47:0/1349246775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c1a5bb0 con 0x7f265c105490 2024-03-21T11:38:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.739+0000 7f265a800640 1 --2- 172.21.15.47:0/1349246775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f265c105490 0x7f265c1a5540 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f265402c640 tx=0x7f2654035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.741+0000 7f2652a00640 1 -- 172.21.15.47:0/1349246775 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654039040 con 0x7f265c105490 2024-03-21T11:38:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.741+0000 7f2652a00640 1 -- 172.21.15.47:0/1349246775 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2654044070 con 0x7f265c105490 2024-03-21T11:38:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.741+0000 7f2652a00640 1 -- 172.21.15.47:0/1349246775 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540044e0 con 0x7f265c105490 2024-03-21T11:38:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.741+0000 7f2661000640 1 -- 172.21.15.47:0/1349246775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265c1a5e40 con 0x7f265c105490 2024-03-21T11:38:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.741+0000 7f2661000640 1 -- 172.21.15.47:0/1349246775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265c1a62b0 con 0x7f265c105490 2024-03-21T11:38:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.748+0000 7f2652a00640 1 -- 172.21.15.47:0/1349246775 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2654032890 con 0x7f265c105490 2024-03-21T11:38:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.751+0000 7f2663e21740 1 Processor -- start 2024-03-21T11:38:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.751+0000 7f2652a00640 1 --2- 172.21.15.47:0/1349246775 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26400a30f0 0x7f26400a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.751+0000 7f2652a00640 1 -- 172.21.15.47:0/1349246775 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f26540f17b0 con 0x7f265c105490 2024-03-21T11:38:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.752+0000 7f2663e21740 1 -- start start 2024-03-21T11:38:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.752+0000 7f2653e00640 1 --2- 172.21.15.47:0/1349246775 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26400a30f0 0x7f26400a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:33.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.752+0000 7f2663e21740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdeee80 0x55912fdef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.752+0000 7f2663e21740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdef870 0x55912fdf4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.752+0000 7f2663e21740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fdf6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.752+0000 7f2663e21740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55912fd54250 con 0x55912fdef870 2024-03-21T11:38:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.752+0000 7f2663e21740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55912fcfca30 con 0x55912fdeee80 2024-03-21T11:38:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.752+0000 7f2663e21740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55912fd502f0 con 0x55912fdf49e0 2024-03-21T11:38:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.753+0000 7f265a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fdf6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.753+0000 7f2653e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdef870 0x55912fdf4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.753+0000 7f265a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fdf6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60192/0 (socket says 172.21.15.47:60192) 2024-03-21T11:38:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.753+0000 7f265a800640 1 -- 172.21.15.47:0/2226822436 learned_addr learned my addr 172.21.15.47:0/2226822436 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.753+0000 7f265a800640 1 -- 172.21.15.47:0/2226822436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdeee80 msgr2=0x55912fdef2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.753+0000 7f265a800640 1 --2- 172.21.15.47:0/2226822436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdeee80 0x55912fdef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.753+0000 7f265a800640 1 -- 172.21.15.47:0/2226822436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdef870 msgr2=0x55912fdf4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.754+0000 7f265a800640 1 --2- 172.21.15.47:0/2226822436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdef870 0x55912fdf4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.754+0000 7f265a800640 1 -- 172.21.15.47:0/2226822436 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55912fce7690 con 0x55912fdf49e0 2024-03-21T11:38:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.754+0000 7f265a800640 1 --2- 172.21.15.47:0/2226822436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fdf6e70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2654005fa0 tx=0x7f26540bd6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4156e5175a5a4744 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.754+0000 7f2653e00640 1 --2- 172.21.15.47:0/1349246775 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26400a30f0 0x7f26400a55b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2644004650 tx=0x7f2644009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.754+0000 7f264b400640 1 -- 172.21.15.47:0/2226822436 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540c2e60 con 0x55912fdf49e0 2024-03-21T11:38:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.755+0000 7f264b400640 1 -- 172.21.15.47:0/2226822436 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26540bdb80 con 0x55912fdf49e0 2024-03-21T11:38:33.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.755+0000 7f264b400640 1 -- 172.21.15.47:0/2226822436 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540c4070 con 0x55912fdf49e0 2024-03-21T11:38:33.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.756+0000 7f2663e21740 1 -- 172.21.15.47:0/2226822436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 msgr2=0x55912fdf6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:33.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.756+0000 7f2663e21740 1 --2- 172.21.15.47:0/2226822436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fdf6e70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2654005fa0 tx=0x7f26540bd6c0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.756+0000 7f2663e21740 1 -- 172.21.15.47:0/2226822436 shutdown_connections 2024-03-21T11:38:33.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.756+0000 7f2663e21740 1 --2- 172.21.15.47:0/2226822436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fdf6e70 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.756+0000 7f2663e21740 1 --2- 172.21.15.47:0/2226822436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdef870 0x55912fdf4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.756+0000 7f2663e21740 1 --2- 172.21.15.47:0/2226822436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdeee80 0x55912fdef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.756+0000 7f2663e21740 1 -- 172.21.15.47:0/2226822436 >> 172.21.15.47:0/2226822436 conn(0x55912fdb7200 msgr2=0x55912fdea7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:33.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.756+0000 7f2663e21740 1 -- 172.21.15.47:0/2226822436 shutdown_connections 2024-03-21T11:38:33.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.756+0000 7f2663e21740 1 -- 172.21.15.47:0/2226822436 wait complete. 2024-03-21T11:38:33.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.757+0000 7f2663e21740 1 -- start start 2024-03-21T11:38:33.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.757+0000 7f2663e21740 1 Processor -- start 2024-03-21T11:38:33.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.758+0000 7f2663e21740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdeee80 0x55912fdf8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.758+0000 7f2663e21740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdef870 0x55912fdf9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.758+0000 7f2663e21740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdf49e0 0x55912fe001b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.758+0000 7f2663e21740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55912fdf7750 con 0x55912fdf49e0 2024-03-21T11:38:33.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.758+0000 7f2663e21740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55912fd0fe80 con 0x55912fdeee80 2024-03-21T11:38:33.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.758+0000 7f2663e21740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55912fdf78d0 con 0x55912fdef870 2024-03-21T11:38:33.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.758+0000 7f2663e21740 10 client.0 _ll_register_callbacks cb 0x55912fd22130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:33.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.758+0000 7f2659e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdeee80 0x55912fdf8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:33.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.758+0000 7f2653e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdef870 0x55912fdf9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:33.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.758+0000 7f2659e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdeee80 0x55912fdf8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52890/0 (socket says 172.21.15.47:52890) 2024-03-21T11:38:33.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.758+0000 7f2659e00640 1 -- 172.21.15.47:0/1913040563 learned_addr learned my addr 172.21.15.47:0/1913040563 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:33.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.759+0000 7f2653e00640 1 -- 172.21.15.47:0/1913040563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdeee80 msgr2=0x55912fdf8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:33.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.759+0000 7f2653e00640 1 --2- 172.21.15.47:0/1913040563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdeee80 0x55912fdf8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.759+0000 7f2653e00640 1 -- 172.21.15.47:0/1913040563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdf49e0 msgr2=0x55912fe001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:33.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.759+0000 7f2653e00640 1 --2- 172.21.15.47:0/1913040563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdf49e0 0x55912fe001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.759+0000 7f2653e00640 1 -- 172.21.15.47:0/1913040563 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55912fb327d0 con 0x55912fdef870 2024-03-21T11:38:33.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.760+0000 7f2653e00640 1 --2- 172.21.15.47:0/1913040563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdef870 0x55912fdf9170 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x55912fdefc70 tx=0x7f26440046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36e3b2396205acb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:33.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.760+0000 7f263fe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:38:33.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.760+0000 7f263fe00640 1 -- 172.21.15.47:0/1913040563 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644011850 con 0x55912fdef870 2024-03-21T11:38:33.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.760+0000 7f263fe00640 1 -- 172.21.15.47:0/1913040563 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f26440119f0 con 0x55912fdef870 2024-03-21T11:38:33.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.760+0000 7f2663e21740 1 -- 172.21.15.47:0/1913040563 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26540bdb80 con 0x55912fdef870 2024-03-21T11:38:33.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.764+0000 7f263fe00640 1 -- 172.21.15.47:0/1913040563 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f264402d050 con 0x55912fdef870 2024-03-21T11:38:33.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.764+0000 7f2663e21740 10 client.30514 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:33.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.764+0000 7f2663e21740 1 -- 172.21.15.47:0/1913040563 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55912fa683f0 con 0x55912fdef870 2024-03-21T11:38:33.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.765+0000 7f263fe00640 1 -- 172.21.15.47:0/1913040563 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26540bdb80 con 0x55912fdef870 2024-03-21T11:38:33.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.765+0000 7f263fe00640 1 -- 172.21.15.47:0/1913040563 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f264400f030 con 0x55912fdef870 2024-03-21T11:38:33.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.765+0000 7f2663e21740 10 client.30514 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:33.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.765+0000 7f2663e21740 10 client.30514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:33.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.765+0000 7f2663e21740 20 client.30514 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:33.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.765+0000 7f2663e21740 1 --2- 172.21.15.47:0/1913040563 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55912fe04ae0 0x55912fe06f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.765+0000 7f2663e21740 4 client.30514 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:33.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.765+0000 7f2663e21740 1 -- 172.21.15.47:0/1913040563 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55912fd61790 con 0x55912fe04ae0 2024-03-21T11:38:33.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.766+0000 7f2659e00640 1 --2- 172.21.15.47:0/1913040563 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55912fe04ae0 0x55912fe06f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:33.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.770+0000 7f2659e00640 1 --2- 172.21.15.47:0/1913040563 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55912fe04ae0 0x55912fe06f80 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3c5920870d9037d server_cookie=c748f7d071eab0ad in_seq=0 out_seq=0 2024-03-21T11:38:33.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.770+0000 7f263fe00640 10 client.30514 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:33.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.776+0000 7f263fe00640 1 -- 172.21.15.47:0/1913040563 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55912fd61790 con 0x55912fe04ae0 2024-03-21T11:38:33.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.776+0000 7f263fe00640 10 client.30514 handle_command_reply: tid=0 2024-03-21T11:38:33.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.776+0000 7f2663e21740 1 client.30514 shutdown 2024-03-21T11:38:33.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.778+0000 7f2663e21740 1 -- 172.21.15.47:0/1913040563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdef870 msgr2=0x55912fdf9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:33.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.778+0000 7f2663e21740 1 --2- 172.21.15.47:0/1913040563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdef870 0x55912fdf9170 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x55912fdefc70 tx=0x7f26440046c0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.778+0000 7f2663e21740 1 -- 172.21.15.47:0/1913040563 shutdown_connections 2024-03-21T11:38:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.778+0000 7f2663e21740 1 --2- 172.21.15.47:0/1913040563 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55912fe04ae0 0x55912fe06f80 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.778+0000 7f2663e21740 1 --2- 172.21.15.47:0/1913040563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdf49e0 0x55912fe001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.778+0000 7f2663e21740 1 --2- 172.21.15.47:0/1913040563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdef870 0x55912fdf9170 secure :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x55912fdefc70 tx=0x7f26440046c0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.778+0000 7f2663e21740 1 --2- 172.21.15.47:0/1913040563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdeee80 0x55912fdf8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.778+0000 7f263fe00640 0 client.30514 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.778+0000 7f2663e21740 1 -- 172.21.15.47:0/1913040563 >> 172.21.15.47:0/1913040563 conn(0x55912fdb7200 msgr2=0x55912fde9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.780+0000 7f2663e21740 1 -- 172.21.15.47:0/1913040563 shutdown_connections 2024-03-21T11:38:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.780+0000 7f2663e21740 1 -- 172.21.15.47:0/1913040563 wait complete. 2024-03-21T11:38:33.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.780+0000 7f2663e21740 20 client.30514 trim_cache size 0 max 16384 2024-03-21T11:38:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.816+0000 7f2663e21740 1 Processor -- start 2024-03-21T11:38:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.817+0000 7f2663e21740 1 -- start start 2024-03-21T11:38:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.817+0000 7f2663e21740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdeee80 0x55912fdeda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.817+0000 7f2663e21740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fe00170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.817+0000 7f2663e21740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdf92a0 0x55912fdedf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.817+0000 7f2663e21740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c1b0780 con 0x55912fdeee80 2024-03-21T11:38:33.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.817+0000 7f2663e21740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c1b2280 con 0x55912fdf92a0 2024-03-21T11:38:33.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.817+0000 7f2663e21740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c1b0de0 con 0x55912fdf49e0 2024-03-21T11:38:33.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.818+0000 7f2659e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdeee80 0x55912fdeda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:33.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.818+0000 7f2653e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fe00170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:33.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.818+0000 7f2653e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fe00170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60212/0 (socket says 172.21.15.47:60212) 2024-03-21T11:38:33.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.818+0000 7f2653e00640 1 -- 172.21.15.47:0/747731356 learned_addr learned my addr 172.21.15.47:0/747731356 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:33.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.818+0000 7f2653e00640 1 -- 172.21.15.47:0/747731356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdf92a0 msgr2=0x55912fdedf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:33.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.818+0000 7f2653e00640 1 --2- 172.21.15.47:0/747731356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdf92a0 0x55912fdedf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.818+0000 7f2653e00640 1 -- 172.21.15.47:0/747731356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdeee80 msgr2=0x55912fdeda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:33.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.818+0000 7f2653e00640 1 --2- 172.21.15.47:0/747731356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdeee80 0x55912fdeda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.818+0000 7f2653e00640 1 -- 172.21.15.47:0/747731356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55912fa280f0 con 0x55912fdf49e0 2024-03-21T11:38:33.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.819+0000 7f2653e00640 1 --2- 172.21.15.47:0/747731356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fe00170 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2644004870 tx=0x7f26440097e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e70cce27d7358d0d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:33.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.820+0000 7f264aa00640 1 -- 172.21.15.47:0/747731356 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644037070 con 0x55912fdf49e0 2024-03-21T11:38:33.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.820+0000 7f264aa00640 1 -- 172.21.15.47:0/747731356 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2644012700 con 0x55912fdf49e0 2024-03-21T11:38:33.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.820+0000 7f264aa00640 1 -- 172.21.15.47:0/747731356 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644020890 con 0x55912fdf49e0 2024-03-21T11:38:33.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.821+0000 7f2663e21740 1 -- 172.21.15.47:0/747731356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 msgr2=0x55912fe00170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:33.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.821+0000 7f2663e21740 1 --2- 172.21.15.47:0/747731356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fe00170 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2644004870 tx=0x7f26440097e0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.821+0000 7f2663e21740 1 -- 172.21.15.47:0/747731356 shutdown_connections 2024-03-21T11:38:33.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.821+0000 7f2663e21740 1 --2- 172.21.15.47:0/747731356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdf92a0 0x55912fdedf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.821+0000 7f2663e21740 1 --2- 172.21.15.47:0/747731356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fe00170 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.821+0000 7f2663e21740 1 --2- 172.21.15.47:0/747731356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdeee80 0x55912fdeda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.821+0000 7f2663e21740 1 -- 172.21.15.47:0/747731356 >> 172.21.15.47:0/747731356 conn(0x55912fdb7200 msgr2=0x55912fde9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:33.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.822+0000 7f2663e21740 1 -- 172.21.15.47:0/747731356 shutdown_connections 2024-03-21T11:38:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.824+0000 7f2663e21740 1 -- 172.21.15.47:0/747731356 wait complete. 2024-03-21T11:38:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.824+0000 7f2663e21740 1 -- start start 2024-03-21T11:38:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.824+0000 7f2663e21740 1 Processor -- start 2024-03-21T11:38:33.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.826+0000 7f2663e21740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdeee80 0x55912fdfdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.826+0000 7f2663e21740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fdfe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.827+0000 7f2663e21740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdf92a0 0x55912fdff490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.827+0000 7f2653e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fdfe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:33.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.827+0000 7f2653e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fdfe480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60228/0 (socket says 172.21.15.47:60228) 2024-03-21T11:38:33.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.827+0000 7f2653e00640 1 -- 172.21.15.47:0/3553008367 learned_addr learned my addr 172.21.15.47:0/3553008367 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:33.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.827+0000 7f2663e21740 1 -- 172.21.15.47:0/3553008367 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c1b1b60 con 0x55912fdeee80 2024-03-21T11:38:33.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.827+0000 7f2663e21740 1 -- 172.21.15.47:0/3553008367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c1b1500 con 0x55912fdf92a0 2024-03-21T11:38:33.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.827+0000 7f2663e21740 1 -- 172.21.15.47:0/3553008367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c1ae940 con 0x55912fdf49e0 2024-03-21T11:38:33.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.827+0000 7f2663e21740 10 client.0 _ll_register_callbacks cb 0x7f2644014f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:33.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.827+0000 7f2659e00640 1 --2- 172.21.15.47:0/3553008367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdeee80 0x55912fdfdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.828+0000 7f2653e00640 1 -- 172.21.15.47:0/3553008367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdf92a0 msgr2=0x55912fdff490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.828+0000 7f2653e00640 1 --2- 172.21.15.47:0/3553008367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdf92a0 0x55912fdff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.828+0000 7f2653e00640 1 -- 172.21.15.47:0/3553008367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdeee80 msgr2=0x55912fdfdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.828+0000 7f2653e00640 1 --2- 172.21.15.47:0/3553008367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdeee80 0x55912fdfdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.828+0000 7f2653e00640 1 -- 172.21.15.47:0/3553008367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55912fdffa80 con 0x55912fdf49e0 2024-03-21T11:38:33.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.828+0000 7f2659e00640 1 --2- 172.21.15.47:0/3553008367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdeee80 0x55912fdfdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.829+0000 7f2653e00640 1 --2- 172.21.15.47:0/3553008367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fdfe480 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2644005d00 tx=0x7f2644011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d3891c1e4f2d4e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.829+0000 7f263fe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:38:33.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.831+0000 7f263fe00640 1 -- 172.21.15.47:0/3553008367 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644037070 con 0x55912fdf49e0 2024-03-21T11:38:33.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.831+0000 7f2663e21740 1 -- 172.21.15.47:0/3553008367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55912fd5bae0 con 0x55912fdf49e0 2024-03-21T11:38:33.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.831+0000 7f263fe00640 1 -- 172.21.15.47:0/3553008367 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2644020610 con 0x55912fdf49e0 2024-03-21T11:38:33.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.833+0000 7f263fe00640 1 -- 172.21.15.47:0/3553008367 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2644009300 con 0x55912fdf49e0 2024-03-21T11:38:33.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.833+0000 7f2663e21740 10 client.30520 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:33.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.833+0000 7f2663e21740 1 -- 172.21.15.47:0/3553008367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55912fe60b20 con 0x55912fdf49e0 2024-03-21T11:38:33.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.834+0000 7f263fe00640 1 -- 172.21.15.47:0/3553008367 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55912fd5bae0 con 0x55912fdf49e0 2024-03-21T11:38:33.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.834+0000 7f263fe00640 1 -- 172.21.15.47:0/3553008367 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f2644044030 con 0x55912fdf49e0 2024-03-21T11:38:33.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.834+0000 7f2663e21740 10 client.30520 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:33.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.834+0000 7f2663e21740 10 client.30520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:33.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.834+0000 7f2663e21740 20 client.30520 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:33.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.834+0000 7f2663e21740 1 --2- 172.21.15.47:0/3553008367 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55912fe614a0 0x55912fe638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:33.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.834+0000 7f2663e21740 4 client.30520 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:38:33.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.834+0000 7f2663e21740 1 -- 172.21.15.47:0/3553008367 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55912fcd93a0 con 0x55912fe614a0 2024-03-21T11:38:33.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.835+0000 7f2659e00640 1 --2- 172.21.15.47:0/3553008367 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55912fe614a0 0x55912fe638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:33.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.836+0000 7f2659e00640 1 --2- 172.21.15.47:0/3553008367 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55912fe614a0 0x55912fe638c0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b2950cebe4c08d2 server_cookie=390b748959f59c4 in_seq=0 out_seq=0 2024-03-21T11:38:33.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.836+0000 7f263fe00640 10 client.30520 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:33.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.840+0000 7f263fe00640 1 -- 172.21.15.47:0/3553008367 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55912fcd93a0 con 0x55912fe614a0 2024-03-21T11:38:33.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.840+0000 7f263fe00640 10 client.30520 handle_command_reply: tid=0 2024-03-21T11:38:33.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.840+0000 7f2663e21740 1 client.30520 shutdown 2024-03-21T11:38:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.841+0000 7f2663e21740 1 -- 172.21.15.47:0/3553008367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 msgr2=0x55912fdfe480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.841+0000 7f2663e21740 1 --2- 172.21.15.47:0/3553008367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fdfe480 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2644005d00 tx=0x7f2644011980 comp rx=0 tx=0).stop 2024-03-21T11:38:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.842+0000 7f2663e21740 1 -- 172.21.15.47:0/3553008367 shutdown_connections 2024-03-21T11:38:33.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.842+0000 7f2663e21740 1 --2- 172.21.15.47:0/3553008367 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55912fe614a0 0x55912fe638c0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.842+0000 7f2663e21740 1 --2- 172.21.15.47:0/3553008367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55912fdf92a0 0x55912fdff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.842+0000 7f2663e21740 1 --2- 172.21.15.47:0/3553008367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55912fdf49e0 0x55912fdfe480 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.842+0000 7f2663e21740 1 --2- 172.21.15.47:0/3553008367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55912fdeee80 0x55912fdfdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.842+0000 7f2663e21740 1 -- 172.21.15.47:0/3553008367 >> 172.21.15.47:0/3553008367 conn(0x55912fdb7200 msgr2=0x55912fdf02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:33.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.842+0000 7f263fe00640 0 client.30520 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:33.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.843+0000 7f2663e21740 1 -- 172.21.15.47:0/3553008367 shutdown_connections 2024-03-21T11:38:33.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.843+0000 7f2663e21740 1 -- 172.21.15.47:0/3553008367 wait complete. 2024-03-21T11:38:33.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.843+0000 7f2663e21740 20 client.30520 trim_cache size 0 max 16384 2024-03-21T11:38:33.843 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:38:33.844 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:38:33.844 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:38:33.844 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:38:33.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.844+0000 7f2661000640 1 -- 172.21.15.47:0/1349246775 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26400a30f0 msgr2=0x7f26400a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:33.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.844+0000 7f2661000640 1 --2- 172.21.15.47:0/1349246775 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26400a30f0 0x7f26400a55b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2644004650 tx=0x7f2644009590 comp rx=0 tx=0).stop 2024-03-21T11:38:33.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.844+0000 7f2661000640 1 -- 172.21.15.47:0/1349246775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f265c105490 msgr2=0x7f265c1a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:33.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.844+0000 7f2661000640 1 --2- 172.21.15.47:0/1349246775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f265c105490 0x7f265c1a5540 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f265402c640 tx=0x7f2654035d10 comp rx=0 tx=0).stop 2024-03-21T11:38:33.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.844+0000 7f2661000640 1 -- 172.21.15.47:0/1349246775 shutdown_connections 2024-03-21T11:38:33.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.844+0000 7f2661000640 1 --2- 172.21.15.47:0/1349246775 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26400a30f0 0x7f26400a55b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.844+0000 7f2661000640 1 --2- 172.21.15.47:0/1349246775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f265c105490 0x7f265c1a5540 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.844+0000 7f2661000640 1 --2- 172.21.15.47:0/1349246775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f265c104ad0 0x7f265c19e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.844+0000 7f2661000640 1 --2- 172.21.15.47:0/1349246775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f265c1038e0 0x7f265c19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:33.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.844+0000 7f2661000640 1 -- 172.21.15.47:0/1349246775 >> 172.21.15.47:0/1349246775 conn(0x7f265c0fee50 msgr2=0x7f265c077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:33.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.845+0000 7f2661000640 1 -- 172.21.15.47:0/1349246775 shutdown_connections 2024-03-21T11:38:33.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:33.845+0000 7f2661000640 1 -- 172.21.15.47:0/1349246775 wait complete. 2024-03-21T11:38:33.876 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:38:33.877 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:38:33.926 INFO:tasks.workunit.client.0.smithi047.stdout: 10 38587 29.25 MB/sec warmup 77 sec latency 998.952 ms 2024-03-21T11:38:34.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.199+0000 7f98c8e00640 1 -- 172.21.15.47:0/882205313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98c41073a0 msgr2=0x7f98c41097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:34.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.199+0000 7f98c8e00640 1 --2- 172.21.15.47:0/882205313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98c41073a0 0x7f98c41097f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009860 tx=0x7f98b402bb70 comp rx=0 tx=0).stop 2024-03-21T11:38:34.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.199+0000 7f98c8e00640 1 -- 172.21.15.47:0/882205313 shutdown_connections 2024-03-21T11:38:34.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.199+0000 7f98c8e00640 1 --2- 172.21.15.47:0/882205313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98c41073a0 0x7f98c41097f0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.199+0000 7f98c8e00640 1 --2- 172.21.15.47:0/882205313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c4104a70 0x7f98c4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.199+0000 7f98c8e00640 1 --2- 172.21.15.47:0/882205313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98c4102140 0x7f98c4104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.200+0000 7f98c8e00640 1 -- 172.21.15.47:0/882205313 >> 172.21.15.47:0/882205313 conn(0x7f98c40fbb30 msgr2=0x7f98c40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:34.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.200+0000 7f98c8e00640 1 -- 172.21.15.47:0/882205313 shutdown_connections 2024-03-21T11:38:34.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.200+0000 7f98c8e00640 1 -- 172.21.15.47:0/882205313 wait complete. 2024-03-21T11:38:34.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.202+0000 7f98c8e00640 1 Processor -- start 2024-03-21T11:38:34.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.202+0000 7f98c8e00640 1 -- start start 2024-03-21T11:38:34.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.202+0000 7f98c8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98c4102140 0x7f98c4199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.203+0000 7f98c8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98c4104a70 0x7f98c419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.203+0000 7f98c8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c41073a0 0x7f98c41a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.203+0000 7f98c8e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c4100690 con 0x7f98c4102140 2024-03-21T11:38:34.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.203+0000 7f98c8e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c4100510 con 0x7f98c41073a0 2024-03-21T11:38:34.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.203+0000 7f98c8e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c4100810 con 0x7f98c4104a70 2024-03-21T11:38:34.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.203+0000 7f98c1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98c4104a70 0x7f98c419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:34.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.203+0000 7f98c1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98c4104a70 0x7f98c419a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60252/0 (socket says 172.21.15.47:60252) 2024-03-21T11:38:34.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.203+0000 7f98c1400640 1 -- 172.21.15.47:0/510893246 learned_addr learned my addr 172.21.15.47:0/510893246 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:34.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.203+0000 7f98c2800640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c41073a0 0x7f98c41a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:34.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.203+0000 7f98c1e00640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98c4102140 0x7f98c4199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:34.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.204+0000 7f98c2800640 1 -- 172.21.15.47:0/510893246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98c4104a70 msgr2=0x7f98c419a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:34.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.204+0000 7f98c2800640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98c4104a70 0x7f98c419a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.204+0000 7f98c2800640 1 -- 172.21.15.47:0/510893246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98c4102140 msgr2=0x7f98c4199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:34.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.204+0000 7f98c2800640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98c4102140 0x7f98c4199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.204+0000 7f98c2800640 1 -- 172.21.15.47:0/510893246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c41a17a0 con 0x7f98c41073a0 2024-03-21T11:38:34.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.204+0000 7f98c1e00640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98c4102140 0x7f98c4199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:34.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.205+0000 7f98c2800640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c41073a0 0x7f98c41a1130 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009780 tx=0x7f98b4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:34.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.205+0000 7f98baa00640 1 -- 172.21.15.47:0/510893246 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b4041020 con 0x7f98c41073a0 2024-03-21T11:38:34.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.205+0000 7f98baa00640 1 -- 172.21.15.47:0/510893246 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98b40035f0 con 0x7f98c41073a0 2024-03-21T11:38:34.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.205+0000 7f98c8e00640 1 -- 172.21.15.47:0/510893246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c41a1a30 con 0x7f98c41073a0 2024-03-21T11:38:34.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.206+0000 7f98c8e00640 1 -- 172.21.15.47:0/510893246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c41a1ef0 con 0x7f98c41073a0 2024-03-21T11:38:34.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.206+0000 7f98baa00640 1 -- 172.21.15.47:0/510893246 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b4039040 con 0x7f98c41073a0 2024-03-21T11:38:34.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.209+0000 7f98c8e00640 1 -- 172.21.15.47:0/510893246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c41a21e0 con 0x7f98c41073a0 2024-03-21T11:38:34.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.214+0000 7f98baa00640 1 -- 172.21.15.47:0/510893246 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f98b4044020 con 0x7f98c41073a0 2024-03-21T11:38:34.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.218+0000 7f98baa00640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98a80a2f20 0x7f98a80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.219+0000 7f98baa00640 1 -- 172.21.15.47:0/510893246 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f98b40f1550 con 0x7f98c41073a0 2024-03-21T11:38:34.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.219+0000 7f98c1e00640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98a80a2f20 0x7f98a80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:34.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.220+0000 7f98baa00640 1 -- 172.21.15.47:0/510893246 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f98b404b020 con 0x7f98c41073a0 2024-03-21T11:38:34.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.221+0000 7f98c1e00640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98a80a2f20 0x7f98a80a53e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f98bc005fd0 tx=0x7f98bc0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.598+0000 7f98c8e00640 1 -- 172.21.15.47:0/510893246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98c41a2570 con 0x7f98c41073a0 2024-03-21T11:38:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.600+0000 7f98baa00640 1 -- 172.21.15.47:0/510893246 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+11493 (secure 0 0 0) 0x7f98b40d0090 con 0x7f98c41073a0 2024-03-21T11:38:34.600 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:38:34.600 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:37:28.116356+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581,24553,24535,24523,24524]},"id":1}]} 2024-03-21T11:38:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2024-03-21T11:38:34.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.605+0000 7f98c8e00640 1 -- 172.21.15.47:0/510893246 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98a80a2f20 msgr2=0x7f98a80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:34.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.605+0000 7f98c8e00640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98a80a2f20 0x7f98a80a53e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f98bc005fd0 tx=0x7f98bc0095c0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.605+0000 7f98c8e00640 1 -- 172.21.15.47:0/510893246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c41073a0 msgr2=0x7f98c41a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.605+0000 7f98c8e00640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c41073a0 0x7f98c41a1130 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009780 tx=0x7f98b4004270 comp rx=0 tx=0).stop 2024-03-21T11:38:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.605+0000 7f98c8e00640 1 -- 172.21.15.47:0/510893246 shutdown_connections 2024-03-21T11:38:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.605+0000 7f98c8e00640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98a80a2f20 0x7f98a80a53e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.605+0000 7f98c8e00640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98c41073a0 0x7f98c41a1130 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.605+0000 7f98c8e00640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98c4104a70 0x7f98c419a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.605+0000 7f98c8e00640 1 --2- 172.21.15.47:0/510893246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98c4102140 0x7f98c4199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.605+0000 7f98c8e00640 1 -- 172.21.15.47:0/510893246 >> 172.21.15.47:0/510893246 conn(0x7f98c40fbb30 msgr2=0x7f98c4106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.606+0000 7f98c8e00640 1 -- 172.21.15.47:0/510893246 shutdown_connections 2024-03-21T11:38:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.606+0000 7f98c8e00640 1 -- 172.21.15.47:0/510893246 wait complete. 2024-03-21T11:38:34.635 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:38:34.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:38:34.926 INFO:tasks.workunit.client.0.smithi047.stdout: 10 38800 29.06 MB/sec warmup 78 sec latency 1394.023 ms 2024-03-21T11:38:34.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.953+0000 7f9624a00640 1 -- 172.21.15.47:0/3586098547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9620104ab0 msgr2=0x7f9620104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:34.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.953+0000 7f9624a00640 1 --2- 172.21.15.47:0/3586098547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9620104ab0 0x7f9620104f30 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f960c009860 tx=0x7f960c02bb70 comp rx=0 tx=0).stop 2024-03-21T11:38:34.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.954+0000 7f9624a00640 1 -- 172.21.15.47:0/3586098547 shutdown_connections 2024-03-21T11:38:34.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.954+0000 7f9624a00640 1 --2- 172.21.15.47:0/3586098547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9620105470 0x7f962010bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.954+0000 7f9624a00640 1 --2- 172.21.15.47:0/3586098547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9620104ab0 0x7f9620104f30 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.954+0000 7f9624a00640 1 --2- 172.21.15.47:0/3586098547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96201038c0 0x7f9620103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.954+0000 7f9624a00640 1 -- 172.21.15.47:0/3586098547 >> 172.21.15.47:0/3586098547 conn(0x7f96200fee50 msgr2=0x7f9620101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:34.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.954+0000 7f9624a00640 1 -- 172.21.15.47:0/3586098547 shutdown_connections 2024-03-21T11:38:34.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.954+0000 7f9624a00640 1 -- 172.21.15.47:0/3586098547 wait complete. 2024-03-21T11:38:34.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.955+0000 7f9624a00640 1 Processor -- start 2024-03-21T11:38:34.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.956+0000 7f9624a00640 1 -- start start 2024-03-21T11:38:34.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.956+0000 7f9624a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96201038c0 0x7f9620199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.956+0000 7f9624a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9620104ab0 0x7f962019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.956+0000 7f9624a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9620105470 0x7f96201a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.956+0000 7f9624a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962010f590 con 0x7f9620105470 2024-03-21T11:38:34.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.956+0000 7f9624a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962010f410 con 0x7f96201038c0 2024-03-21T11:38:34.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.956+0000 7f9624a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962010f710 con 0x7f9620104ab0 2024-03-21T11:38:34.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.957+0000 7f961d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9620104ab0 0x7f962019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:34.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.957+0000 7f961d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9620104ab0 0x7f962019a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60268/0 (socket says 172.21.15.47:60268) 2024-03-21T11:38:34.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.957+0000 7f961d400640 1 -- 172.21.15.47:0/14289838 learned_addr learned my addr 172.21.15.47:0/14289838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:34.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.957+0000 7f961de00640 1 --2- 172.21.15.47:0/14289838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96201038c0 0x7f9620199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:34.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.957+0000 7f961e800640 1 --2- 172.21.15.47:0/14289838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9620105470 0x7f96201a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:34.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.957+0000 7f961d400640 1 -- 172.21.15.47:0/14289838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96201038c0 msgr2=0x7f9620199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:34.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.957+0000 7f961d400640 1 --2- 172.21.15.47:0/14289838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96201038c0 0x7f9620199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.957+0000 7f961d400640 1 -- 172.21.15.47:0/14289838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9620105470 msgr2=0x7f96201a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:34.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.957+0000 7f961d400640 1 --2- 172.21.15.47:0/14289838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9620105470 0x7f96201a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.957+0000 7f961d400640 1 -- 172.21.15.47:0/14289838 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96201a1860 con 0x7f9620104ab0 2024-03-21T11:38:34.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.958+0000 7f961d400640 1 --2- 172.21.15.47:0/14289838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9620104ab0 0x7f962019a190 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f960c000c00 tx=0x7f960c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:34.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.960+0000 7f9616a00640 1 -- 172.21.15.47:0/14289838 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c039040 con 0x7f9620104ab0 2024-03-21T11:38:34.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.960+0000 7f9616a00640 1 -- 172.21.15.47:0/14289838 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f960c042070 con 0x7f9620104ab0 2024-03-21T11:38:34.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.960+0000 7f9616a00640 1 -- 172.21.15.47:0/14289838 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c004a40 con 0x7f9620104ab0 2024-03-21T11:38:34.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.961+0000 7f9624a00640 1 -- 172.21.15.47:0/14289838 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96201a1af0 con 0x7f9620104ab0 2024-03-21T11:38:34.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.961+0000 7f9624a00640 1 -- 172.21.15.47:0/14289838 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96201a1f90 con 0x7f9620104ab0 2024-03-21T11:38:34.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.969+0000 7f9616a00640 1 -- 172.21.15.47:0/14289838 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f960c005170 con 0x7f9620104ab0 2024-03-21T11:38:34.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.969+0000 7f9627806740 1 Processor -- start 2024-03-21T11:38:34.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.970+0000 7f9627806740 1 -- start start 2024-03-21T11:38:34.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.970+0000 7f9627806740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 0x55e11354ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.970+0000 7f9627806740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e11354be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.970+0000 7f9627806740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354c3a0 0x55e113552c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.970+0000 7f9627806740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1134af2c0 con 0x55e11354b9e0 2024-03-21T11:38:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.970+0000 7f9627806740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e113457c90 con 0x55e11354a810 2024-03-21T11:38:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.970+0000 7f9627806740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1134ab360 con 0x55e11354c3a0 2024-03-21T11:38:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.971+0000 7f961d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354c3a0 0x55e113552c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.971+0000 7f961d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354c3a0 0x55e113552c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60272/0 (socket says 172.21.15.47:60272) 2024-03-21T11:38:34.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.971+0000 7f961d400640 1 -- 172.21.15.47:0/1372308111 learned_addr learned my addr 172.21.15.47:0/1372308111 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:34.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.971+0000 7f961de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e11354be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:34.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.971+0000 7f9616a00640 1 --2- 172.21.15.47:0/14289838 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f96040a30a0 0x7f96040a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.971+0000 7f9616a00640 1 -- 172.21.15.47:0/14289838 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f960c0f1070 con 0x7f9620104ab0 2024-03-21T11:38:34.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.971+0000 7f961e800640 1 --2- 172.21.15.47:0/1372308111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 0x55e11354ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:34.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.971+0000 7f961d400640 1 -- 172.21.15.47:0/1372308111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 msgr2=0x55e11354ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:34.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.971+0000 7f961d400640 1 --2- 172.21.15.47:0/1372308111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 0x55e11354ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.971+0000 7f961d400640 1 -- 172.21.15.47:0/1372308111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 msgr2=0x55e11354be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:34.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.971+0000 7f961d400640 1 --2- 172.21.15.47:0/1372308111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e11354be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.971+0000 7f961d400640 1 -- 172.21.15.47:0/1372308111 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e11347c080 con 0x55e11354c3a0 2024-03-21T11:38:34.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.971+0000 7f961de00640 1 --2- 172.21.15.47:0/1372308111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e11354be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:38:34.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.972+0000 7f961e800640 1 --2- 172.21.15.47:0/14289838 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f96040a30a0 0x7f96040a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:34.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.972+0000 7f961d400640 1 --2- 172.21.15.47:0/1372308111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354c3a0 0x55e113552c30 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f960c0ee7e0 tx=0x7f960c0dd770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc6b57b124eaa065 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:34.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.973+0000 7f960b000640 1 -- 172.21.15.47:0/1372308111 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0bd1e0 con 0x55e11354c3a0 2024-03-21T11:38:34.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.973+0000 7f960b000640 1 -- 172.21.15.47:0/1372308111 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f960c0dd040 con 0x55e11354c3a0 2024-03-21T11:38:34.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.973+0000 7f960b000640 1 -- 172.21.15.47:0/1372308111 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0bb550 con 0x55e11354c3a0 2024-03-21T11:38:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.974+0000 7f9627806740 1 -- 172.21.15.47:0/1372308111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354c3a0 msgr2=0x55e113552c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.974+0000 7f9627806740 1 --2- 172.21.15.47:0/1372308111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354c3a0 0x55e113552c30 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f960c0ee7e0 tx=0x7f960c0dd770 comp rx=0 tx=0).stop 2024-03-21T11:38:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.974+0000 7f9627806740 1 -- 172.21.15.47:0/1372308111 shutdown_connections 2024-03-21T11:38:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.974+0000 7f9627806740 1 --2- 172.21.15.47:0/1372308111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354c3a0 0x55e113552c30 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.974+0000 7f9627806740 1 --2- 172.21.15.47:0/1372308111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e11354be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.974+0000 7f9627806740 1 --2- 172.21.15.47:0/1372308111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 0x55e11354ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.974+0000 7f9627806740 1 -- 172.21.15.47:0/1372308111 >> 172.21.15.47:0/1372308111 conn(0x55e113545c00 msgr2=0x55e113546000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.975+0000 7f961e800640 1 --2- 172.21.15.47:0/14289838 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f96040a30a0 0x7f96040a5560 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x55e11354b8e0 tx=0x7f9618010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.975+0000 7f9627806740 1 -- 172.21.15.47:0/1372308111 shutdown_connections 2024-03-21T11:38:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.975+0000 7f9627806740 1 -- 172.21.15.47:0/1372308111 wait complete. 2024-03-21T11:38:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.977+0000 7f9627806740 1 -- start start 2024-03-21T11:38:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.977+0000 7f9627806740 1 Processor -- start 2024-03-21T11:38:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.977+0000 7f9627806740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 0x55e11355cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.978+0000 7f9627806740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354b9e0 0x55e11355d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.978+0000 7f9627806740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354c3a0 0x55e113564300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.978+0000 7f9627806740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1134ab360 con 0x55e11354c3a0 2024-03-21T11:38:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.978+0000 7f9627806740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e11346b0d0 con 0x55e11354a810 2024-03-21T11:38:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.978+0000 7f9627806740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e113553170 con 0x55e11354b9e0 2024-03-21T11:38:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.978+0000 7f9627806740 10 client.0 _ll_register_callbacks cb 0x55e11346fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.979+0000 7f961de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 0x55e11355cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.979+0000 7f961de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 0x55e11355cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52942/0 (socket says 172.21.15.47:52942) 2024-03-21T11:38:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.979+0000 7f961de00640 1 -- 172.21.15.47:0/1240130488 learned_addr learned my addr 172.21.15.47:0/1240130488 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.979+0000 7f961d400640 1 --2- 172.21.15.47:0/1240130488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354b9e0 0x55e11355d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.979+0000 7f961de00640 1 -- 172.21.15.47:0/1240130488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354b9e0 msgr2=0x55e11355d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.979+0000 7f961de00640 1 --2- 172.21.15.47:0/1240130488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354b9e0 0x55e11355d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.980+0000 7f961de00640 1 -- 172.21.15.47:0/1240130488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354c3a0 msgr2=0x55e113564300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.980+0000 7f961de00640 1 --2- 172.21.15.47:0/1240130488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354c3a0 0x55e113564300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.980+0000 7f961de00640 1 -- 172.21.15.47:0/1240130488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1131c33f0 con 0x55e11354a810 2024-03-21T11:38:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.980+0000 7f95ff400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.980+0000 7f961de00640 1 --2- 172.21.15.47:0/1240130488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 0x55e11355cd80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9610012700 tx=0x7f9610012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27a855cf9a92a875 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:34.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.981+0000 7f95ff400640 1 -- 172.21.15.47:0/1240130488 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961000f920 con 0x55e11354a810 2024-03-21T11:38:34.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.981+0000 7f95ff400640 1 -- 172.21.15.47:0/1240130488 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f961000fac0 con 0x55e11354a810 2024-03-21T11:38:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.981+0000 7f9627806740 1 -- 172.21.15.47:0/1240130488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f960c0dd040 con 0x55e11354a810 2024-03-21T11:38:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.981+0000 7f95ff400640 1 -- 172.21.15.47:0/1240130488 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f961000fcc0 con 0x55e11354a810 2024-03-21T11:38:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.981+0000 7f9627806740 10 client.30521 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.981+0000 7f9627806740 1 -- 172.21.15.47:0/1240130488 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e11328d7d0 con 0x55e11354a810 2024-03-21T11:38:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.982+0000 7f95ff400640 1 -- 172.21.15.47:0/1240130488 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f960c0dd040 con 0x55e11354a810 2024-03-21T11:38:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.982+0000 7f95ff400640 1 -- 172.21.15.47:0/1240130488 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f9610010430 con 0x55e11354a810 2024-03-21T11:38:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.983+0000 7f9627806740 10 client.30521 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.983+0000 7f9627806740 10 client.30521 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:38:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.983+0000 7f9627806740 20 client.30521 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.983+0000 7f9627806740 1 --2- 172.21.15.47:0/1240130488 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e1135687b0 0x55e11356abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.983+0000 7f9627806740 4 client.30521 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.983+0000 7f9627806740 1 -- 172.21.15.47:0/1240130488 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1134bc800 con 0x55e1135687b0 2024-03-21T11:38:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.984+0000 7f961d400640 1 --2- 172.21.15.47:0/1240130488 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e1135687b0 0x55e11356abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:34.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.986+0000 7f961d400640 1 --2- 172.21.15.47:0/1240130488 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e1135687b0 0x55e11356abc0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ffee449267faa281 server_cookie=ed07aaf670b75406 in_seq=0 out_seq=0 2024-03-21T11:38:34.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.986+0000 7f95ff400640 10 client.30521 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:38:34.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.987+0000 7f95ff400640 1 -- 172.21.15.47:0/1240130488 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e1134bc800 con 0x55e1135687b0 2024-03-21T11:38:34.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.987+0000 7f95ff400640 10 client.30521 handle_command_reply: tid=0 2024-03-21T11:38:34.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.987+0000 7f9627806740 1 client.30521 shutdown 2024-03-21T11:38:34.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.988+0000 7f9627806740 1 -- 172.21.15.47:0/1240130488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 msgr2=0x55e11355cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.988+0000 7f9627806740 1 --2- 172.21.15.47:0/1240130488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 0x55e11355cd80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9610012700 tx=0x7f9610012bc0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.988+0000 7f9627806740 1 -- 172.21.15.47:0/1240130488 shutdown_connections 2024-03-21T11:38:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.988+0000 7f9627806740 1 --2- 172.21.15.47:0/1240130488 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e1135687b0 0x55e11356abc0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.988+0000 7f9627806740 1 --2- 172.21.15.47:0/1240130488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354c3a0 0x55e113564300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.988+0000 7f95ff400640 0 client.30521 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:38:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.988+0000 7f9627806740 1 --2- 172.21.15.47:0/1240130488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354b9e0 0x55e11355d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.988+0000 7f9627806740 1 --2- 172.21.15.47:0/1240130488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 0x55e11355cd80 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.988+0000 7f9627806740 1 -- 172.21.15.47:0/1240130488 >> 172.21.15.47:0/1240130488 conn(0x55e113545c00 msgr2=0x55e113544eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.988+0000 7f9627806740 1 -- 172.21.15.47:0/1240130488 shutdown_connections 2024-03-21T11:38:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.988+0000 7f9627806740 1 -- 172.21.15.47:0/1240130488 wait complete. 2024-03-21T11:38:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:34.989+0000 7f9627806740 20 client.30521 trim_cache size 0 max 16384 2024-03-21T11:38:35.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.014+0000 7f9627806740 1 Processor -- start 2024-03-21T11:38:35.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.014+0000 7f9627806740 1 -- start start 2024-03-21T11:38:35.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.015+0000 7f9627806740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 0x55e11355d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.015+0000 7f9627806740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e1135642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.015+0000 7f9627806740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354c3a0 0x55e113549570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.015+0000 7f9627806740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96201a7ef0 con 0x55e11354b9e0 2024-03-21T11:38:35.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.015+0000 7f9627806740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96201a99f0 con 0x55e11354a810 2024-03-21T11:38:35.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.015+0000 7f9627806740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96201a8550 con 0x55e11354c3a0 2024-03-21T11:38:35.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.015+0000 7f961d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e1135642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.015+0000 7f961d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e1135642c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45902/0 (socket says 172.21.15.47:45902) 2024-03-21T11:38:35.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.015+0000 7f961d400640 1 -- 172.21.15.47:0/2563061388 learned_addr learned my addr 172.21.15.47:0/2563061388 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:35.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.015+0000 7f961e800640 1 --2- 172.21.15.47:0/2563061388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354c3a0 0x55e113549570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.016+0000 7f961de00640 1 --2- 172.21.15.47:0/2563061388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 0x55e11355d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.016+0000 7f961e800640 1 -- 172.21.15.47:0/2563061388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 msgr2=0x55e11355d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.016+0000 7f961e800640 1 --2- 172.21.15.47:0/2563061388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 0x55e11355d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.016+0000 7f961e800640 1 -- 172.21.15.47:0/2563061388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 msgr2=0x55e1135642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.016+0000 7f961e800640 1 --2- 172.21.15.47:0/2563061388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e1135642c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.016+0000 7f961e800640 1 -- 172.21.15.47:0/2563061388 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e113546670 con 0x55e11354c3a0 2024-03-21T11:38:35.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.016+0000 7f961e800640 1 --2- 172.21.15.47:0/2563061388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354c3a0 0x55e113549570 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9618005fd0 tx=0x7f9618004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ae9b1561d4e4cd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:35.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.019+0000 7f960a600640 1 -- 172.21.15.47:0/2563061388 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618029070 con 0x55e11354c3a0 2024-03-21T11:38:35.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.019+0000 7f960a600640 1 -- 172.21.15.47:0/2563061388 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9618024400 con 0x55e11354c3a0 2024-03-21T11:38:35.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.019+0000 7f960a600640 1 -- 172.21.15.47:0/2563061388 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618029070 con 0x55e11354c3a0 2024-03-21T11:38:35.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.020+0000 7f9627806740 1 -- 172.21.15.47:0/2563061388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354c3a0 msgr2=0x55e113549570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.020+0000 7f9627806740 1 --2- 172.21.15.47:0/2563061388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354c3a0 0x55e113549570 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9618005fd0 tx=0x7f9618004ab0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.022+0000 7f9627806740 1 -- 172.21.15.47:0/2563061388 shutdown_connections 2024-03-21T11:38:35.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.022+0000 7f9627806740 1 --2- 172.21.15.47:0/2563061388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354c3a0 0x55e113549570 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.022+0000 7f9627806740 1 --2- 172.21.15.47:0/2563061388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e1135642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.022+0000 7f9627806740 1 --2- 172.21.15.47:0/2563061388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354a810 0x55e11355d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.022+0000 7f9627806740 1 -- 172.21.15.47:0/2563061388 >> 172.21.15.47:0/2563061388 conn(0x55e113545c00 msgr2=0x55e113548e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.022+0000 7f9627806740 1 -- 172.21.15.47:0/2563061388 shutdown_connections 2024-03-21T11:38:35.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.022+0000 7f9627806740 1 -- 172.21.15.47:0/2563061388 wait complete. 2024-03-21T11:38:35.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.023+0000 7f9627806740 1 -- start start 2024-03-21T11:38:35.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.023+0000 7f9627806740 1 Processor -- start 2024-03-21T11:38:35.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.024+0000 7f9627806740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354a810 0x55e113562010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.024+0000 7f9627806740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e113562550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.024+0000 7f9627806740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354c3a0 0x55e113563560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.024+0000 7f9627806740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96201a92d0 con 0x55e11354b9e0 2024-03-21T11:38:35.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.024+0000 7f961de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354a810 0x55e113562010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.024+0000 7f961de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354a810 0x55e113562010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60306/0 (socket says 172.21.15.47:60306) 2024-03-21T11:38:35.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.024+0000 7f961de00640 1 -- 172.21.15.47:0/2012246940 learned_addr learned my addr 172.21.15.47:0/2012246940 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:35.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.024+0000 7f9627806740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96201a8c70 con 0x55e11354c3a0 2024-03-21T11:38:35.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.024+0000 7f961d400640 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e113562550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.024+0000 7f9627806740 1 -- 172.21.15.47:0/2012246940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96201a6090 con 0x55e11354a810 2024-03-21T11:38:35.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.024+0000 7f961e800640 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354c3a0 0x55e113563560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.024+0000 7f9627806740 10 client.0 _ll_register_callbacks cb 0x7f9610009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:35.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.025+0000 7f961de00640 1 -- 172.21.15.47:0/2012246940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354c3a0 msgr2=0x55e113563560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.025+0000 7f961de00640 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354c3a0 0x55e113563560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.025+0000 7f961de00640 1 -- 172.21.15.47:0/2012246940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 msgr2=0x55e113562550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.025+0000 7f961de00640 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e113562550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.025+0000 7f961de00640 1 -- 172.21.15.47:0/2012246940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e113563b50 con 0x55e11354a810 2024-03-21T11:38:35.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.025+0000 7f961de00640 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354a810 0x55e113562010 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f9610023690 tx=0x7f961001a3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae288a5116822e3f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:35.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.025+0000 7f95ff400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:38:35.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.026+0000 7f95ff400640 1 -- 172.21.15.47:0/2012246940 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961000fcc0 con 0x55e11354a810 2024-03-21T11:38:35.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.026+0000 7f95ff400640 1 -- 172.21.15.47:0/2012246940 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f961000fac0 con 0x55e11354a810 2024-03-21T11:38:35.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.026+0000 7f961d400640 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e113562550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:35.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.026+0000 7f961e800640 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354c3a0 0x55e113563560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:35.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.026+0000 7f9627806740 1 -- 172.21.15.47:0/2012246940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9618024400 con 0x55e11354a810 2024-03-21T11:38:35.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.026+0000 7f95ff400640 1 -- 172.21.15.47:0/2012246940 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f961000fe60 con 0x55e11354a810 2024-03-21T11:38:35.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.027+0000 7f9627806740 10 client.30544 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:35.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.027+0000 7f9627806740 1 -- 172.21.15.47:0/2012246940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=34}) v3 -- 0x55e1135c45e0 con 0x55e11354a810 2024-03-21T11:38:35.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.027+0000 7f95ff400640 1 -- 172.21.15.47:0/2012246940 <== mon.1 v2:172.21.15.102:3300/0 4 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f9610010430 con 0x55e11354a810 2024-03-21T11:38:35.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.027+0000 7f9627806740 10 client.30544 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:35.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.027+0000 7f9627806740 10 client.30544 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:38:35.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.027+0000 7f9627806740 20 client.30544 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:35.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.028+0000 7f9627806740 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e1135687b0 0x55e1135c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.028+0000 7f9627806740 4 client.30544 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:38:35.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.028+0000 7f9627806740 1 -- 172.21.15.47:0/2012246940 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e1134b6b50 con 0x55e1135687b0 2024-03-21T11:38:35.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.030+0000 7f961d400640 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e1135687b0 0x55e1135c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.031+0000 7f961d400640 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e1135687b0 0x55e1135c6e80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4c28da5beb8d8750 server_cookie=ff9f20dd51d4b0de in_seq=0 out_seq=0 2024-03-21T11:38:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.031+0000 7f95ff400640 10 client.30544 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:38:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.032+0000 7f95ff400640 1 -- 172.21.15.47:0/2012246940 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e1134b6b50 con 0x55e1135687b0 2024-03-21T11:38:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.032+0000 7f95ff400640 10 client.30544 handle_command_reply: tid=0 2024-03-21T11:38:35.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.032+0000 7f9627806740 1 client.30544 shutdown 2024-03-21T11:38:35.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.033+0000 7f9627806740 1 -- 172.21.15.47:0/2012246940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354a810 msgr2=0x55e113562010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.033+0000 7f9627806740 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354a810 0x55e113562010 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f9610023690 tx=0x7f961001a3e0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.034+0000 7f9627806740 1 -- 172.21.15.47:0/2012246940 shutdown_connections 2024-03-21T11:38:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.034+0000 7f9627806740 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e1135687b0 0x55e1135c6e80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.034+0000 7f9627806740 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e11354c3a0 0x55e113563560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.034+0000 7f9627806740 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e11354b9e0 0x55e113562550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.034+0000 7f9627806740 1 --2- 172.21.15.47:0/2012246940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e11354a810 0x55e113562010 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.034+0000 7f95ff400640 0 client.30544 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:38:35.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.034+0000 7f9627806740 1 -- 172.21.15.47:0/2012246940 >> 172.21.15.47:0/2012246940 conn(0x55e113545c00 msgr2=0x55e1135496f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.034+0000 7f9627806740 1 -- 172.21.15.47:0/2012246940 shutdown_connections 2024-03-21T11:38:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.035+0000 7f9627806740 1 -- 172.21.15.47:0/2012246940 wait complete. 2024-03-21T11:38:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.035+0000 7f9627806740 20 client.30544 trim_cache size 0 max 16384 2024-03-21T11:38:35.035 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:38:35.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.036+0000 7f9624a00640 1 -- 172.21.15.47:0/14289838 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f96040a30a0 msgr2=0x7f96040a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.036+0000 7f9624a00640 1 --2- 172.21.15.47:0/14289838 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f96040a30a0 0x7f96040a5560 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x55e11354b8e0 tx=0x7f9618010040 comp rx=0 tx=0).stop 2024-03-21T11:38:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.036+0000 7f9624a00640 1 -- 172.21.15.47:0/14289838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9620104ab0 msgr2=0x7f962019a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.036+0000 7f9624a00640 1 --2- 172.21.15.47:0/14289838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9620104ab0 0x7f962019a190 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f960c000c00 tx=0x7f960c004270 comp rx=0 tx=0).stop 2024-03-21T11:38:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.037+0000 7f9624a00640 1 -- 172.21.15.47:0/14289838 shutdown_connections 2024-03-21T11:38:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.037+0000 7f9624a00640 1 --2- 172.21.15.47:0/14289838 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f96040a30a0 0x7f96040a5560 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.037+0000 7f9624a00640 1 --2- 172.21.15.47:0/14289838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9620105470 0x7f96201a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.037+0000 7f9624a00640 1 --2- 172.21.15.47:0/14289838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9620104ab0 0x7f962019a190 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.037+0000 7f9624a00640 1 --2- 172.21.15.47:0/14289838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96201038c0 0x7f9620199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.037+0000 7f9624a00640 1 -- 172.21.15.47:0/14289838 >> 172.21.15.47:0/14289838 conn(0x7f96200fee50 msgr2=0x7f9620107e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.039+0000 7f9624a00640 1 -- 172.21.15.47:0/14289838 shutdown_connections 2024-03-21T11:38:35.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.040+0000 7f9624a00640 1 -- 172.21.15.47:0/14289838 wait complete. 2024-03-21T11:38:35.071 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:38:35.071 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:38:35.320 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:35 smithi047 ceph-mon[31351]: pgmap v857: 129 pgs: 129 active+clean; 305 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.3 MiB/s wr, 662 op/s 2024-03-21T11:38:35.321 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:35 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/510893246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:38:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.368+0000 7fdd3c600640 1 -- 172.21.15.47:0/315886788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd3410d9c0 msgr2=0x7fdd3410de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.368+0000 7fdd3c600640 1 --2- 172.21.15.47:0/315886788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd3410d9c0 0x7fdd3410de40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009ca0 tx=0x7fdd2c02c0f0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.368+0000 7fdd3c600640 1 -- 172.21.15.47:0/315886788 shutdown_connections 2024-03-21T11:38:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.368+0000 7fdd3c600640 1 --2- 172.21.15.47:0/315886788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd3410d9c0 0x7fdd3410de40 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.368+0000 7fdd3c600640 1 --2- 172.21.15.47:0/315886788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd3410b010 0x7fdd3410d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.368+0000 7fdd3c600640 1 --2- 172.21.15.47:0/315886788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd341086a0 0x7fdd3410aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.368+0000 7fdd3c600640 1 -- 172.21.15.47:0/315886788 >> 172.21.15.47:0/315886788 conn(0x7fdd3406e500 msgr2=0x7fdd34070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.368+0000 7fdd3c600640 1 -- 172.21.15.47:0/315886788 shutdown_connections 2024-03-21T11:38:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.368+0000 7fdd3c600640 1 -- 172.21.15.47:0/315886788 wait complete. 2024-03-21T11:38:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.369+0000 7fdd3c600640 1 Processor -- start 2024-03-21T11:38:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.369+0000 7fdd3c600640 1 -- start start 2024-03-21T11:38:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.370+0000 7fdd3c600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd341086a0 0x7fdd34117e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.370+0000 7fdd3c600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3410b010 0x7fdd34118380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.370+0000 7fdd3c600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd3410d9c0 0x7fdd341193c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.370+0000 7fdd3c600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd341104d0 con 0x7fdd341086a0 2024-03-21T11:38:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.370+0000 7fdd3c600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd34110350 con 0x7fdd3410b010 2024-03-21T11:38:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.370+0000 7fdd3c600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd34110650 con 0x7fdd3410d9c0 2024-03-21T11:38:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.370+0000 7fdd3a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd3410d9c0 0x7fdd341193c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.370+0000 7fdd3a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd3410d9c0 0x7fdd341193c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60328/0 (socket says 172.21.15.47:60328) 2024-03-21T11:38:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.370+0000 7fdd3a600640 1 -- 172.21.15.47:0/3677959836 learned_addr learned my addr 172.21.15.47:0/3677959836 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.370+0000 7fdd39200640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3410b010 0x7fdd34118380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.370+0000 7fdd39c00640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd341086a0 0x7fdd34117e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.371+0000 7fdd3a600640 1 -- 172.21.15.47:0/3677959836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3410b010 msgr2=0x7fdd34118380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.371+0000 7fdd3a600640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3410b010 0x7fdd34118380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.371+0000 7fdd3a600640 1 -- 172.21.15.47:0/3677959836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd341086a0 msgr2=0x7fdd34117e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.371+0000 7fdd3a600640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd341086a0 0x7fdd34117e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.371+0000 7fdd3a600640 1 -- 172.21.15.47:0/3677959836 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd34119a30 con 0x7fdd3410d9c0 2024-03-21T11:38:35.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.371+0000 7fdd3a600640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd3410d9c0 0x7fdd341193c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c02c700 tx=0x7fdd2c032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:35.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.371+0000 7fdd39c00640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd341086a0 0x7fdd34117e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:35.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.371+0000 7fdd32000640 1 -- 172.21.15.47:0/3677959836 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c039040 con 0x7fdd3410d9c0 2024-03-21T11:38:35.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.371+0000 7fdd32000640 1 -- 172.21.15.47:0/3677959836 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd2c044070 con 0x7fdd3410d9c0 2024-03-21T11:38:35.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.371+0000 7fdd32000640 1 -- 172.21.15.47:0/3677959836 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c0042a0 con 0x7fdd3410d9c0 2024-03-21T11:38:35.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.371+0000 7fdd3c600640 1 -- 172.21.15.47:0/3677959836 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd34119cc0 con 0x7fdd3410d9c0 2024-03-21T11:38:35.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.372+0000 7fdd3c600640 1 -- 172.21.15.47:0/3677959836 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3411a160 con 0x7fdd3410d9c0 2024-03-21T11:38:35.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.375+0000 7fdd3f1ff740 1 Processor -- start 2024-03-21T11:38:35.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.375+0000 7fdd32000640 1 -- 172.21.15.47:0/3677959836 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdd2c032690 con 0x7fdd3410d9c0 2024-03-21T11:38:35.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.375+0000 7fdd3f1ff740 1 -- start start 2024-03-21T11:38:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd3f1ff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 0x562663675c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd3f1ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636769e0 0x562663676e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd3f1ff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636773a0 0x56266367dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd3f1ff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626635da2c0 con 0x562663675810 2024-03-21T11:38:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd3f1ff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562663582c90 con 0x5626636773a0 2024-03-21T11:38:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd3f1ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626635d6360 con 0x5626636769e0 2024-03-21T11:38:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd39c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636769e0 0x562663676e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd39200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 0x562663675c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd39c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636769e0 0x562663676e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60332/0 (socket says 172.21.15.47:60332) 2024-03-21T11:38:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd39c00640 1 -- 172.21.15.47:0/960993013 learned_addr learned my addr 172.21.15.47:0/960993013 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd39200640 1 --2- 172.21.15.47:0/960993013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636773a0 0x56266367dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd39c00640 1 -- 172.21.15.47:0/960993013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636773a0 msgr2=0x56266367dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd39c00640 1 --2- 172.21.15.47:0/960993013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636773a0 0x56266367dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd39c00640 1 -- 172.21.15.47:0/960993013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 msgr2=0x562663675c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd39c00640 1 --2- 172.21.15.47:0/960993013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 0x562663675c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd39c00640 1 -- 172.21.15.47:0/960993013 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626635a7080 con 0x5626636769e0 2024-03-21T11:38:35.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.376+0000 7fdd39200640 1 --2- 172.21.15.47:0/960993013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 0x562663675c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:38:35.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.377+0000 7fdd39c00640 1 --2- 172.21.15.47:0/960993013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636769e0 0x562663676e60 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fdd28009c30 tx=0x7fdd28010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c98f35788bc57d92 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:35.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.377+0000 7fdd32000640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd1c0a2fa0 0x7fdd1c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.377+0000 7fdd26a00640 1 -- 172.21.15.47:0/960993013 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28004020 con 0x5626636769e0 2024-03-21T11:38:35.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.377+0000 7fdd26a00640 1 -- 172.21.15.47:0/960993013 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd280041c0 con 0x5626636769e0 2024-03-21T11:38:35.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.377+0000 7fdd26a00640 1 -- 172.21.15.47:0/960993013 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28012670 con 0x5626636769e0 2024-03-21T11:38:35.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.377+0000 7fdd32000640 1 -- 172.21.15.47:0/3677959836 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdd2c0f1b60 con 0x7fdd3410d9c0 2024-03-21T11:38:35.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.377+0000 7fdd39200640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd1c0a2fa0 0x7fdd1c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.378+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/960993013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636769e0 msgr2=0x562663676e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.378+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/960993013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636769e0 0x562663676e60 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fdd28009c30 tx=0x7fdd28010740 comp rx=0 tx=0).stop 2024-03-21T11:38:35.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.378+0000 7fdd39200640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd1c0a2fa0 0x7fdd1c0a5460 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x5626636768e0 tx=0x7fdd20014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:35.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.378+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/960993013 shutdown_connections 2024-03-21T11:38:35.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.378+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/960993013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636773a0 0x56266367dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.378+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/960993013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636769e0 0x562663676e60 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.378+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/960993013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 0x562663675c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.378+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/960993013 >> 172.21.15.47:0/960993013 conn(0x562663670c00 msgr2=0x562663671000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.378+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/960993013 shutdown_connections 2024-03-21T11:38:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.378+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/960993013 wait complete. 2024-03-21T11:38:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.378+0000 7fdd3f1ff740 1 -- start start 2024-03-21T11:38:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.379+0000 7fdd3f1ff740 1 Processor -- start 2024-03-21T11:38:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.379+0000 7fdd3f1ff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 0x56266367fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.379+0000 7fdd3f1ff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636769e0 0x56266367ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.379+0000 7fdd3f1ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636773a0 0x562663686fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.379+0000 7fdd3f1ff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266367e3b0 con 0x562663675810 2024-03-21T11:38:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.379+0000 7fdd3f1ff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626635960d0 con 0x5626636769e0 2024-03-21T11:38:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.379+0000 7fdd3f1ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266367e530 con 0x5626636773a0 2024-03-21T11:38:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.379+0000 7fdd3f1ff740 10 client.0 _ll_register_callbacks cb 0x56266359ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.379+0000 7fdd39c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 0x56266367fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.379+0000 7fdd39200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636769e0 0x56266367ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.379+0000 7fdd3a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636773a0 0x562663686fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.379+0000 7fdd3a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636773a0 0x562663686fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60338/0 (socket says 172.21.15.47:60338) 2024-03-21T11:38:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.379+0000 7fdd3a600640 1 -- 172.21.15.47:0/1511463654 learned_addr learned my addr 172.21.15.47:0/1511463654 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.380+0000 7fdd3a600640 1 -- 172.21.15.47:0/1511463654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636769e0 msgr2=0x56266367ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.380+0000 7fdd3a600640 1 --2- 172.21.15.47:0/1511463654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636769e0 0x56266367ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.380+0000 7fdd3a600640 1 -- 172.21.15.47:0/1511463654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 msgr2=0x56266367fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.380+0000 7fdd3a600640 1 --2- 172.21.15.47:0/1511463654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 0x56266367fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.380+0000 7fdd3a600640 1 -- 172.21.15.47:0/1511463654 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626632ee3f0 con 0x5626636773a0 2024-03-21T11:38:35.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.380+0000 7fdd3a600640 1 --2- 172.21.15.47:0/1511463654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636773a0 0x562663686fb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0c6380 tx=0x7fdd2c0bdd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9d40a0938708f0d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:35.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.380+0000 7fdd1b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:38:35.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.380+0000 7fdd1b400640 1 -- 172.21.15.47:0/1511463654 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c0de0d0 con 0x5626636773a0 2024-03-21T11:38:35.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.380+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/1511463654 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd280041c0 con 0x5626636773a0 2024-03-21T11:38:35.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.380+0000 7fdd1b400640 1 -- 172.21.15.47:0/1511463654 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdd2c0be190 con 0x5626636773a0 2024-03-21T11:38:35.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.381+0000 7fdd1b400640 1 -- 172.21.15.47:0/1511463654 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd2c0be630 con 0x5626636773a0 2024-03-21T11:38:35.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.381+0000 7fdd3f1ff740 10 client.30556 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:35.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.381+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/1511463654 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5626633b87d0 con 0x5626636773a0 2024-03-21T11:38:35.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.381+0000 7fdd1b400640 1 -- 172.21.15.47:0/1511463654 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd280041c0 con 0x5626636773a0 2024-03-21T11:38:35.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.381+0000 7fdd1b400640 1 -- 172.21.15.47:0/1511463654 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fdd2c03c030 con 0x5626636773a0 2024-03-21T11:38:35.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.381+0000 7fdd3f1ff740 10 client.30556 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:35.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.381+0000 7fdd3f1ff740 10 client.30556 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:38:35.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.381+0000 7fdd3f1ff740 20 client.30556 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:35.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.381+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/1511463654 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56266368b670 0x56266368daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.381+0000 7fdd3f1ff740 4 client.30556 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:35.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.381+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/1511463654 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626635e7800 con 0x56266368b670 2024-03-21T11:38:35.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.382+0000 7fdd39c00640 1 --2- 172.21.15.47:0/1511463654 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56266368b670 0x56266368daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.382+0000 7fdd39c00640 1 --2- 172.21.15.47:0/1511463654 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56266368b670 0x56266368daf0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1010dde7466ded8d server_cookie=63a6df0ab26086ff in_seq=0 out_seq=0 2024-03-21T11:38:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.382+0000 7fdd1b400640 10 client.30556 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:38:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.383+0000 7fdd1b400640 1 -- 172.21.15.47:0/1511463654 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5626635e7800 con 0x56266368b670 2024-03-21T11:38:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.383+0000 7fdd1b400640 10 client.30556 handle_command_reply: tid=0 2024-03-21T11:38:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.383+0000 7fdd3f1ff740 1 client.30556 shutdown 2024-03-21T11:38:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.383+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/1511463654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636773a0 msgr2=0x562663686fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.383+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/1511463654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636773a0 0x562663686fb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0c6380 tx=0x7fdd2c0bdd00 comp rx=0 tx=0).stop 2024-03-21T11:38:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.383+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/1511463654 shutdown_connections 2024-03-21T11:38:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.383+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/1511463654 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56266368b670 0x56266368daf0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.383+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/1511463654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636773a0 0x562663686fb0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.383+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/1511463654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636769e0 0x56266367ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.383+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/1511463654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 0x56266367fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.383+0000 7fdd1b400640 0 client.30556 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:38:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.383+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/1511463654 >> 172.21.15.47:0/1511463654 conn(0x562663670c00 msgr2=0x56266366feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.383+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/1511463654 shutdown_connections 2024-03-21T11:38:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.384+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/1511463654 wait complete. 2024-03-21T11:38:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.384+0000 7fdd3f1ff740 20 client.30556 trim_cache size 0 max 16384 2024-03-21T11:38:35.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:35 smithi100 ceph-mon[36245]: pgmap v857: 129 pgs: 129 active+clean; 305 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.3 MiB/s wr, 662 op/s 2024-03-21T11:38:35.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:35 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/510893246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:38:35.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.398+0000 7fdd3f1ff740 1 Processor -- start 2024-03-21T11:38:35.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.398+0000 7fdd3f1ff740 1 -- start start 2024-03-21T11:38:35.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd3f1ff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562663675810 0x562663674570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd3f1ff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626636769e0 0x562663686f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd3f1ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636773a0 0x562663679a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd3f1ff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd341c16b0 con 0x5626636769e0 2024-03-21T11:38:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd3f1ff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd341c31b0 con 0x562663675810 2024-03-21T11:38:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd3f1ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd341c1d10 con 0x5626636773a0 2024-03-21T11:38:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd39c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562663675810 0x562663674570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd3a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636773a0 0x562663679a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd39200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626636769e0 0x562663686f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd39c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562663675810 0x562663674570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53020/0 (socket says 172.21.15.47:53020) 2024-03-21T11:38:35.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd39c00640 1 -- 172.21.15.47:0/697821209 learned_addr learned my addr 172.21.15.47:0/697821209 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:35.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd39c00640 1 -- 172.21.15.47:0/697821209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636773a0 msgr2=0x562663679a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd39c00640 1 --2- 172.21.15.47:0/697821209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636773a0 0x562663679a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd39c00640 1 -- 172.21.15.47:0/697821209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626636769e0 msgr2=0x562663686f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd39c00640 1 --2- 172.21.15.47:0/697821209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626636769e0 0x562663686f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.399+0000 7fdd39c00640 1 -- 172.21.15.47:0/697821209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626636802e0 con 0x562663675810 2024-03-21T11:38:35.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.400+0000 7fdd39c00640 1 --2- 172.21.15.47:0/697821209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562663675810 0x562663674570 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x562663671b70 tx=0x7fdd280099b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3f968afc6ad6abd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:35.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.400+0000 7fdd26000640 1 -- 172.21.15.47:0/697821209 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28031070 con 0x562663675810 2024-03-21T11:38:35.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.400+0000 7fdd26000640 1 -- 172.21.15.47:0/697821209 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd2802a030 con 0x562663675810 2024-03-21T11:38:35.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.400+0000 7fdd26000640 1 -- 172.21.15.47:0/697821209 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28003890 con 0x562663675810 2024-03-21T11:38:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.401+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/697821209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562663675810 msgr2=0x562663674570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.401+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/697821209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562663675810 0x562663674570 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x562663671b70 tx=0x7fdd280099b0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.401+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/697821209 shutdown_connections 2024-03-21T11:38:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.401+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/697821209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636773a0 0x562663679a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.401+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/697821209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626636769e0 0x562663686f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.401+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/697821209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562663675810 0x562663674570 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.401+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/697821209 >> 172.21.15.47:0/697821209 conn(0x562663670c00 msgr2=0x562663673e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.401+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/697821209 shutdown_connections 2024-03-21T11:38:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.401+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/697821209 wait complete. 2024-03-21T11:38:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.401+0000 7fdd3f1ff740 1 -- start start 2024-03-21T11:38:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.401+0000 7fdd3f1ff740 1 Processor -- start 2024-03-21T11:38:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.402+0000 7fdd3f1ff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 0x5626636828a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.402+0000 7fdd3f1ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636769e0 0x562663682de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.402+0000 7fdd3f1ff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636773a0 0x562663685e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.402+0000 7fdd3f1ff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd341c2a90 con 0x562663675810 2024-03-21T11:38:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.402+0000 7fdd3f1ff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd341c2430 con 0x5626636773a0 2024-03-21T11:38:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.402+0000 7fdd3f1ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd341bf870 con 0x5626636769e0 2024-03-21T11:38:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.402+0000 7fdd3f1ff740 10 client.0 _ll_register_callbacks cb 0x7fdd2c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.402+0000 7fdd3a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636773a0 0x562663685e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.402+0000 7fdd39c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 0x5626636828a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.402+0000 7fdd39200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636769e0 0x562663682de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.402+0000 7fdd3a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636773a0 0x562663685e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53036/0 (socket says 172.21.15.47:53036) 2024-03-21T11:38:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.402+0000 7fdd3a600640 1 -- 172.21.15.47:0/2516282551 learned_addr learned my addr 172.21.15.47:0/2516282551 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.403+0000 7fdd3a600640 1 -- 172.21.15.47:0/2516282551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636769e0 msgr2=0x562663682de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.403+0000 7fdd3a600640 1 --2- 172.21.15.47:0/2516282551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636769e0 0x562663682de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.403+0000 7fdd3a600640 1 -- 172.21.15.47:0/2516282551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 msgr2=0x5626636828a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.403+0000 7fdd3a600640 1 --2- 172.21.15.47:0/2516282551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 0x5626636828a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.403+0000 7fdd3a600640 1 -- 172.21.15.47:0/2516282551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626636863f0 con 0x5626636773a0 2024-03-21T11:38:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.403+0000 7fdd3a600640 1 --2- 172.21.15.47:0/2516282551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636773a0 0x562663685e00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c006010 tx=0x7fdd2c003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2631fedaa0076e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.403+0000 7fdd1b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.403+0000 7fdd1b400640 1 -- 172.21.15.47:0/2516282551 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c0bda70 con 0x5626636773a0 2024-03-21T11:38:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.403+0000 7fdd1b400640 1 -- 172.21.15.47:0/2516282551 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdd2c004b40 con 0x5626636773a0 2024-03-21T11:38:35.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.403+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/2516282551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd2802a030 con 0x5626636773a0 2024-03-21T11:38:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.403+0000 7fdd1b400640 1 -- 172.21.15.47:0/2516282551 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd2c0a0020 con 0x5626636773a0 2024-03-21T11:38:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.403+0000 7fdd3f1ff740 10 client.30542 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.403+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/2516282551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5626636e72b0 con 0x5626636773a0 2024-03-21T11:38:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.404+0000 7fdd1b400640 1 -- 172.21.15.47:0/2516282551 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd2802a030 con 0x5626636773a0 2024-03-21T11:38:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.404+0000 7fdd1b400640 1 -- 172.21.15.47:0/2516282551 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fdd2c0bd380 con 0x5626636773a0 2024-03-21T11:38:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.404+0000 7fdd3f1ff740 10 client.30542 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.404+0000 7fdd3f1ff740 10 client.30542 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:38:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.404+0000 7fdd3f1ff740 20 client.30542 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.404+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/2516282551 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5626636e7a60 0x5626636e9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.404+0000 7fdd3f1ff740 4 client.30542 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:38:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.404+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/2516282551 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5626635e1b50 con 0x5626636e7a60 2024-03-21T11:38:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.404+0000 7fdd39c00640 1 --2- 172.21.15.47:0/2516282551 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5626636e7a60 0x5626636e9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd39c00640 1 --2- 172.21.15.47:0/2516282551 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5626636e7a60 0x5626636e9e80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7a4f972a155fc17 server_cookie=f3e6f46c2fd9b789 in_seq=0 out_seq=0 2024-03-21T11:38:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd1b400640 10 client.30542 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:38:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd1b400640 1 -- 172.21.15.47:0/2516282551 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5626635e1b50 con 0x5626636e7a60 2024-03-21T11:38:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd1b400640 10 client.30542 handle_command_reply: tid=0 2024-03-21T11:38:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd3f1ff740 1 client.30542 shutdown 2024-03-21T11:38:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/2516282551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636773a0 msgr2=0x562663685e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/2516282551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636773a0 0x562663685e00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c006010 tx=0x7fdd2c003c30 comp rx=0 tx=0).stop 2024-03-21T11:38:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/2516282551 shutdown_connections 2024-03-21T11:38:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/2516282551 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5626636e7a60 0x5626636e9e80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/2516282551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626636773a0 0x562663685e00 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/2516282551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626636769e0 0x562663682de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd3f1ff740 1 --2- 172.21.15.47:0/2516282551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562663675810 0x5626636828a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/2516282551 >> 172.21.15.47:0/2516282551 conn(0x562663670c00 msgr2=0x5626636746f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd1b400640 0 client.30542 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:38:35.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/2516282551 shutdown_connections 2024-03-21T11:38:35.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd3f1ff740 1 -- 172.21.15.47:0/2516282551 wait complete. 2024-03-21T11:38:35.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.405+0000 7fdd3f1ff740 20 client.30542 trim_cache size 0 max 16384 2024-03-21T11:38:35.413 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:38:35.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.406+0000 7fdd3c600640 1 -- 172.21.15.47:0/3677959836 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd1c0a2fa0 msgr2=0x7fdd1c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.406+0000 7fdd3c600640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd1c0a2fa0 0x7fdd1c0a5460 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x5626636768e0 tx=0x7fdd20014040 comp rx=0 tx=0).stop 2024-03-21T11:38:35.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.406+0000 7fdd3c600640 1 -- 172.21.15.47:0/3677959836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd3410d9c0 msgr2=0x7fdd341193c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.406+0000 7fdd3c600640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd3410d9c0 0x7fdd341193c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c02c700 tx=0x7fdd2c032f20 comp rx=0 tx=0).stop 2024-03-21T11:38:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.406+0000 7fdd3c600640 1 -- 172.21.15.47:0/3677959836 shutdown_connections 2024-03-21T11:38:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.406+0000 7fdd3c600640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd1c0a2fa0 0x7fdd1c0a5460 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.406+0000 7fdd3c600640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd3410d9c0 0x7fdd341193c0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.406+0000 7fdd3c600640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3410b010 0x7fdd34118380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.406+0000 7fdd3c600640 1 --2- 172.21.15.47:0/3677959836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd341086a0 0x7fdd34117e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.406+0000 7fdd3c600640 1 -- 172.21.15.47:0/3677959836 >> 172.21.15.47:0/3677959836 conn(0x7fdd3406e500 msgr2=0x7fdd3410b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.406+0000 7fdd3c600640 1 -- 172.21.15.47:0/3677959836 shutdown_connections 2024-03-21T11:38:35.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.406+0000 7fdd3c600640 1 -- 172.21.15.47:0/3677959836 wait complete. 2024-03-21T11:38:35.421 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:38:35.421 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:38:35.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:35 smithi102 ceph-mon[36748]: pgmap v857: 129 pgs: 129 active+clean; 305 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.3 MiB/s wr, 662 op/s 2024-03-21T11:38:35.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:35 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/510893246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:38:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.710+0000 7fb89ac00640 1 -- 172.21.15.47:0/1338781781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb894103300 msgr2=0x7fb894109b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.710+0000 7fb89ac00640 1 --2- 172.21.15.47:0/1338781781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb894103300 0x7fb894109b90 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009cd0 tx=0x7fb88c02c100 comp rx=0 tx=0).stop 2024-03-21T11:38:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.710+0000 7fb89ac00640 1 -- 172.21.15.47:0/1338781781 shutdown_connections 2024-03-21T11:38:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.710+0000 7fb89ac00640 1 --2- 172.21.15.47:0/1338781781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb894103300 0x7fb894109b90 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.710+0000 7fb89ac00640 1 --2- 172.21.15.47:0/1338781781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb894102940 0x7fb894102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.710+0000 7fb89ac00640 1 --2- 172.21.15.47:0/1338781781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8940ff970 0x7fb8940ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.710+0000 7fb89ac00640 1 -- 172.21.15.47:0/1338781781 >> 172.21.15.47:0/1338781781 conn(0x7fb8940fb2c0 msgr2=0x7fb8940fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.711+0000 7fb89ac00640 1 -- 172.21.15.47:0/1338781781 shutdown_connections 2024-03-21T11:38:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.712+0000 7fb89ac00640 1 -- 172.21.15.47:0/1338781781 wait complete. 2024-03-21T11:38:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.716+0000 7fb89ac00640 1 Processor -- start 2024-03-21T11:38:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.716+0000 7fb89ac00640 1 -- start start 2024-03-21T11:38:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.716+0000 7fb89ac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8940ff970 0x7fb8941022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.716+0000 7fb89ac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb894102940 0x7fb8941008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.716+0000 7fb89ac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb894103300 0x7fb894100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.716+0000 7fb89ac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89410d3a0 con 0x7fb894102940 2024-03-21T11:38:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.716+0000 7fb89ac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89410d220 con 0x7fb8940ff970 2024-03-21T11:38:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.716+0000 7fb89ac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89410d520 con 0x7fb894103300 2024-03-21T11:38:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.717+0000 7fb893e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8940ff970 0x7fb8941022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.717+0000 7fb893e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8940ff970 0x7fb8941022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53056/0 (socket says 172.21.15.47:53056) 2024-03-21T11:38:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.717+0000 7fb893e00640 1 -- 172.21.15.47:0/2926616551 learned_addr learned my addr 172.21.15.47:0/2926616551 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.717+0000 7fb893400640 1 --2- 172.21.15.47:0/2926616551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb894102940 0x7fb8941008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.717+0000 7fb898c00640 1 --2- 172.21.15.47:0/2926616551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb894103300 0x7fb894100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.717+0000 7fb893e00640 1 -- 172.21.15.47:0/2926616551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb894103300 msgr2=0x7fb894100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.717+0000 7fb893e00640 1 --2- 172.21.15.47:0/2926616551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb894103300 0x7fb894100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.717+0000 7fb893e00640 1 -- 172.21.15.47:0/2926616551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb894102940 msgr2=0x7fb8941008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.717+0000 7fb893e00640 1 --2- 172.21.15.47:0/2926616551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb894102940 0x7fb8941008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.717+0000 7fb893e00640 1 -- 172.21.15.47:0/2926616551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8941016c0 con 0x7fb8940ff970 2024-03-21T11:38:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.718+0000 7fb893e00640 1 --2- 172.21.15.47:0/2926616551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8940ff970 0x7fb8941022b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb88400e9c0 tx=0x7fb88400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.719+0000 7fb890c00640 1 -- 172.21.15.47:0/2926616551 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88400cd80 con 0x7fb8940ff970 2024-03-21T11:38:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.719+0000 7fb890c00640 1 -- 172.21.15.47:0/2926616551 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb884010910 con 0x7fb8940ff970 2024-03-21T11:38:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.719+0000 7fb890c00640 1 -- 172.21.15.47:0/2926616551 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884018950 con 0x7fb8940ff970 2024-03-21T11:38:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.719+0000 7fb89ac00640 1 -- 172.21.15.47:0/2926616551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8941a1b90 con 0x7fb8940ff970 2024-03-21T11:38:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.719+0000 7fb89ac00640 1 -- 172.21.15.47:0/2926616551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8941a2090 con 0x7fb8940ff970 2024-03-21T11:38:35.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.727+0000 7fb89d9e5740 1 Processor -- start 2024-03-21T11:38:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.729+0000 7fb890c00640 1 -- 172.21.15.47:0/2926616551 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb884010ac0 con 0x7fb8940ff970 2024-03-21T11:38:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.731+0000 7fb890c00640 1 --2- 172.21.15.47:0/2926616551 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb8780a3070 0x7fb8780a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.731+0000 7fb890c00640 1 -- 172.21.15.47:0/2926616551 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb8840cfc20 con 0x7fb8940ff970 2024-03-21T11:38:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.732+0000 7fb898c00640 1 --2- 172.21.15.47:0/2926616551 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb8780a3070 0x7fb8780a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.732+0000 7fb89d9e5740 1 -- start start 2024-03-21T11:38:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.733+0000 7fb89d9e5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e6020 0x55d3318e6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.733+0000 7fb89d9e5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e71f0 0x55d3318e7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.733+0000 7fb89d9e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 0x55d3318ee440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.733+0000 7fb89d9e5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3318422c0 con 0x55d3318e71f0 2024-03-21T11:38:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.733+0000 7fb89d9e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3317eac90 con 0x55d3318e7bb0 2024-03-21T11:38:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.733+0000 7fb89d9e5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33183e360 con 0x55d3318e6020 2024-03-21T11:38:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.733+0000 7fb898c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 0x55d3318ee440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.734+0000 7fb898c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 0x55d3318ee440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53070/0 (socket says 172.21.15.47:53070) 2024-03-21T11:38:35.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.734+0000 7fb898c00640 1 -- 172.21.15.47:0/2535984639 learned_addr learned my addr 172.21.15.47:0/2535984639 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:35.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.734+0000 7fb893400640 1 --2- 172.21.15.47:0/2535984639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e71f0 0x55d3318e7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.734+0000 7fb893e00640 1 --2- 172.21.15.47:0/2535984639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e6020 0x55d3318e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.735+0000 7fb898c00640 1 --2- 172.21.15.47:0/2926616551 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb8780a3070 0x7fb8780a5530 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb88c037040 tx=0x7fb88c009a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.735+0000 7fb898c00640 1 -- 172.21.15.47:0/2535984639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e6020 msgr2=0x55d3318e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.735+0000 7fb898c00640 1 --2- 172.21.15.47:0/2535984639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e6020 0x55d3318e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.735+0000 7fb898c00640 1 -- 172.21.15.47:0/2535984639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e71f0 msgr2=0x55d3318e7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.735+0000 7fb898c00640 1 --2- 172.21.15.47:0/2535984639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e71f0 0x55d3318e7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.735+0000 7fb898c00640 1 -- 172.21.15.47:0/2535984639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d33180f080 con 0x55d3318e7bb0 2024-03-21T11:38:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.735+0000 7fb893e00640 1 --2- 172.21.15.47:0/2535984639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e6020 0x55d3318e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.735+0000 7fb898c00640 1 --2- 172.21.15.47:0/2535984639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 0x55d3318ee440 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb88c006010 tx=0x7fb88c045000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4611856338538616 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.736+0000 7fb889200640 1 -- 172.21.15.47:0/2535984639 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c047040 con 0x55d3318e7bb0 2024-03-21T11:38:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.736+0000 7fb889200640 1 -- 172.21.15.47:0/2535984639 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb88c003e50 con 0x55d3318e7bb0 2024-03-21T11:38:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.736+0000 7fb889200640 1 -- 172.21.15.47:0/2535984639 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c039040 con 0x55d3318e7bb0 2024-03-21T11:38:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.737+0000 7fb89d9e5740 1 -- 172.21.15.47:0/2535984639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 msgr2=0x55d3318ee440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.737+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/2535984639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 0x55d3318ee440 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb88c006010 tx=0x7fb88c045000 comp rx=0 tx=0).stop 2024-03-21T11:38:35.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.738+0000 7fb89d9e5740 1 -- 172.21.15.47:0/2535984639 shutdown_connections 2024-03-21T11:38:35.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.738+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/2535984639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 0x55d3318ee440 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.738+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/2535984639 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e71f0 0x55d3318e7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.738+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/2535984639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e6020 0x55d3318e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.738+0000 7fb89d9e5740 1 -- 172.21.15.47:0/2535984639 >> 172.21.15.47:0/2535984639 conn(0x55d3318d8c00 msgr2=0x55d3318d9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.740+0000 7fb89d9e5740 1 -- 172.21.15.47:0/2535984639 shutdown_connections 2024-03-21T11:38:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.740+0000 7fb89d9e5740 1 -- 172.21.15.47:0/2535984639 wait complete. 2024-03-21T11:38:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.741+0000 7fb89d9e5740 1 -- start start 2024-03-21T11:38:35.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.742+0000 7fb89d9e5740 1 Processor -- start 2024-03-21T11:38:35.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.745+0000 7fb89d9e5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 0x55d3318deb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.745+0000 7fb89d9e5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e71f0 0x55d3318df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.745+0000 7fb89d9e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 0x55d3318e4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.745+0000 7fb89d9e5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3318dd570 con 0x55d3318e6020 2024-03-21T11:38:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.745+0000 7fb89d9e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3317fe0d0 con 0x55d3318e7bb0 2024-03-21T11:38:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.745+0000 7fb89d9e5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3318dd6f0 con 0x55d3318e71f0 2024-03-21T11:38:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.745+0000 7fb89d9e5740 10 client.0 _ll_register_callbacks cb 0x55d331802e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.745+0000 7fb893400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e71f0 0x55d3318df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.745+0000 7fb893400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e71f0 0x55d3318df090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60384/0 (socket says 172.21.15.47:60384) 2024-03-21T11:38:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.745+0000 7fb893400640 1 -- 172.21.15.47:0/4078436941 learned_addr learned my addr 172.21.15.47:0/4078436941 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:35.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.746+0000 7fb898c00640 1 --2- 172.21.15.47:0/4078436941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 0x55d3318e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.746+0000 7fb893400640 1 -- 172.21.15.47:0/4078436941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 msgr2=0x55d3318e4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.746+0000 7fb893400640 1 --2- 172.21.15.47:0/4078436941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 0x55d3318e4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.746+0000 7fb893400640 1 -- 172.21.15.47:0/4078436941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 msgr2=0x55d3318deb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.746+0000 7fb893400640 1 --2- 172.21.15.47:0/4078436941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 0x55d3318deb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.746+0000 7fb893400640 1 -- 172.21.15.47:0/4078436941 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3315563f0 con 0x55d3318e71f0 2024-03-21T11:38:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.747+0000 7fb893400640 1 --2- 172.21.15.47:0/4078436941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e71f0 0x55d3318df090 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb880011570 tx=0x7fb880009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=583ff8933eb2b642 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.747+0000 7fb87d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:38:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.747+0000 7fb87d600640 1 -- 172.21.15.47:0/4078436941 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880013670 con 0x55d3318e71f0 2024-03-21T11:38:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.747+0000 7fb87d600640 1 -- 172.21.15.47:0/4078436941 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb880013810 con 0x55d3318e71f0 2024-03-21T11:38:35.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.748+0000 7fb89d9e5740 1 -- 172.21.15.47:0/4078436941 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb88c003e50 con 0x55d3318e71f0 2024-03-21T11:38:35.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.748+0000 7fb87d600640 1 -- 172.21.15.47:0/4078436941 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb880013a10 con 0x55d3318e71f0 2024-03-21T11:38:35.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.749+0000 7fb89d9e5740 10 client.30571 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:35.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.749+0000 7fb89d9e5740 1 -- 172.21.15.47:0/4078436941 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d3316207d0 con 0x55d3318e71f0 2024-03-21T11:38:35.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.749+0000 7fb87d600640 1 -- 172.21.15.47:0/4078436941 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb88c003e50 con 0x55d3318e71f0 2024-03-21T11:38:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.750+0000 7fb87d600640 1 -- 172.21.15.47:0/4078436941 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fb880014440 con 0x55d3318e71f0 2024-03-21T11:38:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.750+0000 7fb89d9e5740 10 client.30571 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.750+0000 7fb89d9e5740 10 client.30571 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:38:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.750+0000 7fb89d9e5740 20 client.30571 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.750+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/4078436941 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d331904340 0x55d331906760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.750+0000 7fb89d9e5740 4 client.30571 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.750+0000 7fb89d9e5740 1 -- 172.21.15.47:0/4078436941 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d33184f800 con 0x55d331904340 2024-03-21T11:38:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.750+0000 7fb893e00640 1 --2- 172.21.15.47:0/4078436941 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d331904340 0x55d331906760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.751+0000 7fb893e00640 1 --2- 172.21.15.47:0/4078436941 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d331904340 0x55d331906760 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1861e94d26d874c server_cookie=decd9b091dc82247 in_seq=0 out_seq=0 2024-03-21T11:38:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.753+0000 7fb87d600640 10 client.30571 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:38:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.753+0000 7fb87d600640 1 -- 172.21.15.47:0/4078436941 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d33184f800 con 0x55d331904340 2024-03-21T11:38:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.753+0000 7fb87d600640 10 client.30571 handle_command_reply: tid=0 2024-03-21T11:38:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.753+0000 7fb89d9e5740 1 client.30571 shutdown 2024-03-21T11:38:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.758+0000 7fb89d9e5740 1 -- 172.21.15.47:0/4078436941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e71f0 msgr2=0x55d3318df090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.758+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/4078436941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e71f0 0x55d3318df090 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb880011570 tx=0x7fb880009f90 comp rx=0 tx=0).stop 2024-03-21T11:38:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.758+0000 7fb89d9e5740 1 -- 172.21.15.47:0/4078436941 shutdown_connections 2024-03-21T11:38:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.758+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/4078436941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 0x55d3318e4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.758+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/4078436941 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d331904340 0x55d331906760 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.758+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/4078436941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e71f0 0x55d3318df090 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.758+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/4078436941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 0x55d3318deb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.758+0000 7fb89d9e5740 1 -- 172.21.15.47:0/4078436941 >> 172.21.15.47:0/4078436941 conn(0x55d3318d8c00 msgr2=0x55d3318d7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.758+0000 7fb87d600640 0 client.30571 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:38:35.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.759+0000 7fb89d9e5740 1 -- 172.21.15.47:0/4078436941 shutdown_connections 2024-03-21T11:38:35.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.759+0000 7fb89d9e5740 1 -- 172.21.15.47:0/4078436941 wait complete. 2024-03-21T11:38:35.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.759+0000 7fb89d9e5740 20 client.30571 trim_cache size 0 max 16384 2024-03-21T11:38:35.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.790+0000 7fb89d9e5740 1 Processor -- start 2024-03-21T11:38:35.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.790+0000 7fb89d9e5740 1 -- start start 2024-03-21T11:38:35.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.790+0000 7fb89d9e5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 0x55d3318ea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.790+0000 7fb89d9e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e71f0 0x55d3318ea960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.790+0000 7fb89d9e5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e7bb0 0x55d3318eaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.790+0000 7fb89d9e5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8941a8080 con 0x55d3318e6020 2024-03-21T11:38:35.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.790+0000 7fb89d9e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8941a9b80 con 0x55d3318e71f0 2024-03-21T11:38:35.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.790+0000 7fb89d9e5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8941a86e0 con 0x55d3318e7bb0 2024-03-21T11:38:35.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.791+0000 7fb893e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 0x55d3318ea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.791+0000 7fb893400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e71f0 0x55d3318ea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.791+0000 7fb898c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e7bb0 0x55d3318eaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.791+0000 7fb893400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e71f0 0x55d3318ea960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53084/0 (socket says 172.21.15.47:53084) 2024-03-21T11:38:35.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.791+0000 7fb893400640 1 -- 172.21.15.47:0/3890355923 learned_addr learned my addr 172.21.15.47:0/3890355923 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:35.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.791+0000 7fb893400640 1 -- 172.21.15.47:0/3890355923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e7bb0 msgr2=0x55d3318eaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.791+0000 7fb893400640 1 --2- 172.21.15.47:0/3890355923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e7bb0 0x55d3318eaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.791+0000 7fb893400640 1 -- 172.21.15.47:0/3890355923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 msgr2=0x55d3318ea420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.791+0000 7fb893400640 1 --2- 172.21.15.47:0/3890355923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 0x55d3318ea420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.791+0000 7fb893400640 1 -- 172.21.15.47:0/3890355923 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3318d9970 con 0x55d3318e71f0 2024-03-21T11:38:35.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.791+0000 7fb898c00640 1 --2- 172.21.15.47:0/3890355923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e7bb0 0x55d3318eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:35.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.793+0000 7fb893400640 1 --2- 172.21.15.47:0/3890355923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e71f0 0x55d3318ea960 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb880011540 tx=0x7fb880006ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c7d3c01f1e90f83 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:35.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.793+0000 7fb87fe00640 1 -- 172.21.15.47:0/3890355923 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880009180 con 0x55d3318e71f0 2024-03-21T11:38:35.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.793+0000 7fb87fe00640 1 -- 172.21.15.47:0/3890355923 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb880014440 con 0x55d3318e71f0 2024-03-21T11:38:35.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.793+0000 7fb87fe00640 1 -- 172.21.15.47:0/3890355923 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88001dbc0 con 0x55d3318e71f0 2024-03-21T11:38:35.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.794+0000 7fb89d9e5740 1 -- 172.21.15.47:0/3890355923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e71f0 msgr2=0x55d3318ea960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.794+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/3890355923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e71f0 0x55d3318ea960 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb880011540 tx=0x7fb880006ec0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.794+0000 7fb89d9e5740 1 -- 172.21.15.47:0/3890355923 shutdown_connections 2024-03-21T11:38:35.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.795+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/3890355923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e7bb0 0x55d3318eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.795+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/3890355923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e71f0 0x55d3318ea960 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.795+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/3890355923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 0x55d3318ea420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.795+0000 7fb89d9e5740 1 -- 172.21.15.47:0/3890355923 >> 172.21.15.47:0/3890355923 conn(0x55d3318d8c00 msgr2=0x55d3318dc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.795+0000 7fb89d9e5740 1 -- 172.21.15.47:0/3890355923 shutdown_connections 2024-03-21T11:38:35.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.795+0000 7fb89d9e5740 1 -- 172.21.15.47:0/3890355923 wait complete. 2024-03-21T11:38:35.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.795+0000 7fb89d9e5740 1 -- start start 2024-03-21T11:38:35.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.796+0000 7fb89d9e5740 1 Processor -- start 2024-03-21T11:38:35.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.799+0000 7fb89d9e5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 0x55d331916810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.799+0000 7fb89d9e5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e71f0 0x55d331914e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.799+0000 7fb89d9e5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 0x55d331915360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.799+0000 7fb89d9e5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8941a9460 con 0x55d3318e6020 2024-03-21T11:38:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.799+0000 7fb89d9e5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8941a8e00 con 0x55d3318e7bb0 2024-03-21T11:38:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.799+0000 7fb89d9e5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8941a6240 con 0x55d3318e71f0 2024-03-21T11:38:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.799+0000 7fb89d9e5740 10 client.0 _ll_register_callbacks cb 0x7fb880009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.799+0000 7fb893400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e71f0 0x55d331914e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.799+0000 7fb893400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e71f0 0x55d331914e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60406/0 (socket says 172.21.15.47:60406) 2024-03-21T11:38:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.799+0000 7fb893400640 1 -- 172.21.15.47:0/730385685 learned_addr learned my addr 172.21.15.47:0/730385685 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.799+0000 7fb893e00640 1 --2- 172.21.15.47:0/730385685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 0x55d331916810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.800+0000 7fb893e00640 1 -- 172.21.15.47:0/730385685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e71f0 msgr2=0x55d331914e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.800+0000 7fb893e00640 1 --2- 172.21.15.47:0/730385685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e71f0 0x55d331914e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.800+0000 7fb893e00640 1 -- 172.21.15.47:0/730385685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 msgr2=0x55d331915360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:38:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.800+0000 7fb893e00640 1 --2- 172.21.15.47:0/730385685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 0x55d331915360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.800+0000 7fb893e00640 1 -- 172.21.15.47:0/730385685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d331915b90 con 0x55d3318e6020 2024-03-21T11:38:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.802+0000 7fb893e00640 1 --2- 172.21.15.47:0/730385685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 0x55d331916810 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb88400c6a0 tx=0x7fb884098320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc6c1308451646b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.802+0000 7fb87d600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:38:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.803+0000 7fb87d600640 1 -- 172.21.15.47:0/730385685 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8840a00b0 con 0x55d3318e6020 2024-03-21T11:38:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.803+0000 7fb87d600640 1 -- 172.21.15.47:0/730385685 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb8840cd440 con 0x55d3318e6020 2024-03-21T11:38:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.804+0000 7fb89d9e5740 1 -- 172.21.15.47:0/730385685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb880014440 con 0x55d3318e6020 2024-03-21T11:38:35.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.804+0000 7fb87d600640 1 -- 172.21.15.47:0/730385685 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8840a4970 con 0x55d3318e6020 2024-03-21T11:38:35.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.805+0000 7fb89d9e5740 10 client.20115 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:35.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.805+0000 7fb89d9e5740 1 -- 172.21.15.47:0/730385685 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d3319711c0 con 0x55d3318e6020 2024-03-21T11:38:35.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.805+0000 7fb87d600640 1 -- 172.21.15.47:0/730385685 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb880014440 con 0x55d3318e6020 2024-03-21T11:38:35.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.806+0000 7fb87d600640 1 -- 172.21.15.47:0/730385685 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fb884018af0 con 0x55d3318e6020 2024-03-21T11:38:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.807+0000 7fb89d9e5740 10 client.20115 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.807+0000 7fb89d9e5740 10 client.20115 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:38:35.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.807+0000 7fb89d9e5740 20 client.20115 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:35.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.807+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/730385685 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d331971b10 0x55d331973f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:35.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.807+0000 7fb89d9e5740 4 client.20115 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:38:35.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.807+0000 7fb89d9e5740 1 -- 172.21.15.47:0/730385685 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d331849b50 con 0x55d331971b10 2024-03-21T11:38:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.809+0000 7fb893400640 1 --2- 172.21.15.47:0/730385685 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d331971b10 0x55d331973f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:35.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.811+0000 7fb87d600640 10 client.20115 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:38:35.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.811+0000 7fb893400640 1 --2- 172.21.15.47:0/730385685 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d331971b10 0x55d331973f30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=823ca6a8aed2b757 server_cookie=1cba5f5e2cad4bf0 in_seq=0 out_seq=0 2024-03-21T11:38:35.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.814+0000 7fb87d600640 1 -- 172.21.15.47:0/730385685 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d331849b50 con 0x55d331971b10 2024-03-21T11:38:35.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.814+0000 7fb87d600640 10 client.20115 handle_command_reply: tid=0 2024-03-21T11:38:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.814+0000 7fb89d9e5740 1 client.20115 shutdown 2024-03-21T11:38:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.816+0000 7fb89d9e5740 1 -- 172.21.15.47:0/730385685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 msgr2=0x55d331916810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.816+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/730385685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 0x55d331916810 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb88400c6a0 tx=0x7fb884098320 comp rx=0 tx=0).stop 2024-03-21T11:38:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.817+0000 7fb89d9e5740 1 -- 172.21.15.47:0/730385685 shutdown_connections 2024-03-21T11:38:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.817+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/730385685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3318e7bb0 0x55d331915360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.817+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/730385685 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d331971b10 0x55d331973f30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.817+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/730385685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3318e71f0 0x55d331914e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.817+0000 7fb89d9e5740 1 --2- 172.21.15.47:0/730385685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3318e6020 0x55d331916810 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.817+0000 7fb89d9e5740 1 -- 172.21.15.47:0/730385685 >> 172.21.15.47:0/730385685 conn(0x55d3318d8c00 msgr2=0x55d3318ea420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.817+0000 7fb87d600640 0 client.20115 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:38:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.817+0000 7fb89d9e5740 1 -- 172.21.15.47:0/730385685 shutdown_connections 2024-03-21T11:38:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.817+0000 7fb89d9e5740 1 -- 172.21.15.47:0/730385685 wait complete. 2024-03-21T11:38:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.818+0000 7fb89d9e5740 20 client.20115 trim_cache size 0 max 16384 2024-03-21T11:38:35.818 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:38:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.819+0000 7fb89ac00640 1 -- 172.21.15.47:0/2926616551 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb8780a3070 msgr2=0x7fb8780a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.819+0000 7fb89ac00640 1 --2- 172.21.15.47:0/2926616551 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb8780a3070 0x7fb8780a5530 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb88c037040 tx=0x7fb88c009a20 comp rx=0 tx=0).stop 2024-03-21T11:38:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.819+0000 7fb89ac00640 1 -- 172.21.15.47:0/2926616551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8940ff970 msgr2=0x7fb8941022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.819+0000 7fb89ac00640 1 --2- 172.21.15.47:0/2926616551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8940ff970 0x7fb8941022b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb88400e9c0 tx=0x7fb88400edc0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.819+0000 7fb89ac00640 1 -- 172.21.15.47:0/2926616551 shutdown_connections 2024-03-21T11:38:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.819+0000 7fb89ac00640 1 --2- 172.21.15.47:0/2926616551 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb8780a3070 0x7fb8780a5530 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.819+0000 7fb89ac00640 1 --2- 172.21.15.47:0/2926616551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb894103300 0x7fb894100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.819+0000 7fb89ac00640 1 --2- 172.21.15.47:0/2926616551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb894102940 0x7fb8941008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.819+0000 7fb89ac00640 1 --2- 172.21.15.47:0/2926616551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8940ff970 0x7fb8941022b0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:35.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.819+0000 7fb89ac00640 1 -- 172.21.15.47:0/2926616551 >> 172.21.15.47:0/2926616551 conn(0x7fb8940fb2c0 msgr2=0x7fb894105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:35.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.819+0000 7fb89ac00640 1 -- 172.21.15.47:0/2926616551 shutdown_connections 2024-03-21T11:38:35.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:35.819+0000 7fb89ac00640 1 -- 172.21.15.47:0/2926616551 wait complete. 2024-03-21T11:38:35.849 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:38:35.850 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:38:35.926 INFO:tasks.workunit.client.0.smithi047.stdout: 10 39045 28.87 MB/sec warmup 79 sec latency 251.679 ms 2024-03-21T11:38:36.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.158+0000 7fa6d0000640 1 -- 172.21.15.47:0/522248010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6c810d9c0 msgr2=0x7fa6c810de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.158+0000 7fa6d0000640 1 --2- 172.21.15.47:0/522248010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6c810d9c0 0x7fa6c810de40 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009ca0 tx=0x7fa6c002c100 comp rx=0 tx=0).stop 2024-03-21T11:38:36.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.158+0000 7fa6d0000640 1 -- 172.21.15.47:0/522248010 shutdown_connections 2024-03-21T11:38:36.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.158+0000 7fa6d0000640 1 --2- 172.21.15.47:0/522248010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6c810d9c0 0x7fa6c810de40 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.158+0000 7fa6d0000640 1 --2- 172.21.15.47:0/522248010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c810b010 0x7fa6c810d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.158+0000 7fa6d0000640 1 --2- 172.21.15.47:0/522248010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6c81086a0 0x7fa6c810aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.158+0000 7fa6d0000640 1 -- 172.21.15.47:0/522248010 >> 172.21.15.47:0/522248010 conn(0x7fa6c806e500 msgr2=0x7fa6c8070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:36.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.160+0000 7fa6d0000640 1 -- 172.21.15.47:0/522248010 shutdown_connections 2024-03-21T11:38:36.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.160+0000 7fa6d0000640 1 -- 172.21.15.47:0/522248010 wait complete. 2024-03-21T11:38:36.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.161+0000 7fa6d0000640 1 Processor -- start 2024-03-21T11:38:36.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.162+0000 7fa6d0000640 1 -- start start 2024-03-21T11:38:36.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.163+0000 7fa6d0000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6c81086a0 0x7fa6c81198f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.163+0000 7fa6d0000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c810b010 0x7fa6c8119e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.163+0000 7fa6d0000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6c810d9c0 0x7fa6c8117f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.163+0000 7fa6d0000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c81107c0 con 0x7fa6c81086a0 2024-03-21T11:38:36.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.163+0000 7fa6d0000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c8110640 con 0x7fa6c810b010 2024-03-21T11:38:36.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.163+0000 7fa6d0000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c8110940 con 0x7fa6c810d9c0 2024-03-21T11:38:36.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.163+0000 7fa6ccc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c810b010 0x7fa6c8119e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.163+0000 7fa6ccc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c810b010 0x7fa6c8119e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53108/0 (socket says 172.21.15.47:53108) 2024-03-21T11:38:36.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.163+0000 7fa6ccc00640 1 -- 172.21.15.47:0/1582261207 learned_addr learned my addr 172.21.15.47:0/1582261207 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:36.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.164+0000 7fa6cd600640 1 --2- 172.21.15.47:0/1582261207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6c81086a0 0x7fa6c81198f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.164+0000 7fa6ccc00640 1 -- 172.21.15.47:0/1582261207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6c810d9c0 msgr2=0x7fa6c8117f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.164+0000 7fa6ccc00640 1 --2- 172.21.15.47:0/1582261207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6c810d9c0 0x7fa6c8117f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.164+0000 7fa6ccc00640 1 -- 172.21.15.47:0/1582261207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6c81086a0 msgr2=0x7fa6c81198f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.164+0000 7fa6ccc00640 1 --2- 172.21.15.47:0/1582261207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6c81086a0 0x7fa6c81198f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.164+0000 7fa6ccc00640 1 -- 172.21.15.47:0/1582261207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c81186b0 con 0x7fa6c810b010 2024-03-21T11:38:36.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.165+0000 7fa6ccc00640 1 --2- 172.21.15.47:0/1582261207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c810b010 0x7fa6c8119e30 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00e9f0 tx=0x7fa6bc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:36.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.166+0000 7fa6c6000640 1 -- 172.21.15.47:0/1582261207 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00cd80 con 0x7fa6c810b010 2024-03-21T11:38:36.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.167+0000 7fa6c6000640 1 -- 172.21.15.47:0/1582261207 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6bc005500 con 0x7fa6c810b010 2024-03-21T11:38:36.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.167+0000 7fa6d0000640 1 -- 172.21.15.47:0/1582261207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c81189a0 con 0x7fa6c810b010 2024-03-21T11:38:36.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.167+0000 7fa6d0000640 1 -- 172.21.15.47:0/1582261207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c81b7490 con 0x7fa6c810b010 2024-03-21T11:38:36.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.168+0000 7fa6c6000640 1 -- 172.21.15.47:0/1582261207 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc0186d0 con 0x7fa6c810b010 2024-03-21T11:38:36.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.172+0000 7fa6d2dc3740 1 Processor -- start 2024-03-21T11:38:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.174+0000 7fa6d2dc3740 1 -- start start 2024-03-21T11:38:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.174+0000 7fa6d2dc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 0x55720dcb0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.174+0000 7fa6d2dc3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb19e0 0x55720dcb1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.174+0000 7fa6d2dc3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb23a0 0x55720dcb8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.174+0000 7fa6d2dc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720dc152c0 con 0x55720dcb0810 2024-03-21T11:38:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.174+0000 7fa6d2dc3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720dbbdc90 con 0x55720dcb19e0 2024-03-21T11:38:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.174+0000 7fa6d2dc3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720dc11360 con 0x55720dcb23a0 2024-03-21T11:38:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.175+0000 7fa6ce000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 0x55720dcb0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.175+0000 7fa6cd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb19e0 0x55720dcb1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.175+0000 7fa6cd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb19e0 0x55720dcb1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53122/0 (socket says 172.21.15.47:53122) 2024-03-21T11:38:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.175+0000 7fa6cd600640 1 -- 172.21.15.47:0/2431663886 learned_addr learned my addr 172.21.15.47:0/2431663886 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:36.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.176+0000 7fa6cd600640 1 -- 172.21.15.47:0/2431663886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb23a0 msgr2=0x55720dcb8c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:38:36.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.176+0000 7fa6cd600640 1 --2- 172.21.15.47:0/2431663886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb23a0 0x55720dcb8c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.176+0000 7fa6cd600640 1 -- 172.21.15.47:0/2431663886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 msgr2=0x55720dcb0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:36.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.176+0000 7fa6cd600640 1 --2- 172.21.15.47:0/2431663886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 0x55720dcb0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.176+0000 7fa6cd600640 1 -- 172.21.15.47:0/2431663886 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55720dbe2080 con 0x55720dcb19e0 2024-03-21T11:38:36.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.176+0000 7fa6ce000640 1 --2- 172.21.15.47:0/2431663886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 0x55720dcb0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:38:36.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.176+0000 7fa6cd600640 1 --2- 172.21.15.47:0/2431663886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb19e0 0x55720dcb1e60 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400bcb0 tx=0x7fa6b400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac35182072b458ba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:36.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.176+0000 7fa6c6000640 1 -- 172.21.15.47:0/1582261207 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa6bc018870 con 0x7fa6c810b010 2024-03-21T11:38:36.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.177+0000 7fa6ba600640 1 -- 172.21.15.47:0/2431663886 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4015070 con 0x55720dcb19e0 2024-03-21T11:38:36.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.179+0000 7fa6ba600640 1 -- 172.21.15.47:0/2431663886 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6b401a940 con 0x55720dcb19e0 2024-03-21T11:38:36.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.179+0000 7fa6ba600640 1 -- 172.21.15.47:0/2431663886 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4015070 con 0x55720dcb19e0 2024-03-21T11:38:36.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.180+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/2431663886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb19e0 msgr2=0x55720dcb1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.180+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/2431663886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb19e0 0x55720dcb1e60 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400bcb0 tx=0x7fa6b400f730 comp rx=0 tx=0).stop 2024-03-21T11:38:36.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.180+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/2431663886 shutdown_connections 2024-03-21T11:38:36.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.180+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/2431663886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb23a0 0x55720dcb8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.180+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/2431663886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb19e0 0x55720dcb1e60 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.180+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/2431663886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 0x55720dcb0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.180+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/2431663886 >> 172.21.15.47:0/2431663886 conn(0x55720dcabc00 msgr2=0x55720dcac000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:36.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.181+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/2431663886 shutdown_connections 2024-03-21T11:38:36.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.181+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/2431663886 wait complete. 2024-03-21T11:38:36.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.181+0000 7fa6c6000640 1 --2- 172.21.15.47:0/1582261207 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6b00a2f00 0x7fa6b00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.181+0000 7fa6d2dc3740 1 -- start start 2024-03-21T11:38:36.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.182+0000 7fa6d2dc3740 1 Processor -- start 2024-03-21T11:38:36.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.183+0000 7fa6c6000640 1 -- 172.21.15.47:0/1582261207 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa6bc0cfac0 con 0x7fa6c810b010 2024-03-21T11:38:36.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.183+0000 7fa6ce000640 1 --2- 172.21.15.47:0/1582261207 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6b00a2f00 0x7fa6b00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.184+0000 7fa6ce000640 1 --2- 172.21.15.47:0/1582261207 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6b00a2f00 0x7fa6b00a53c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009bf0 tx=0x7fa6c0032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:36.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.186+0000 7fa6d2dc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 0x55720dcba880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.186+0000 7fa6d2dc3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb19e0 0x55720dcbadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.186+0000 7fa6d2dc3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb23a0 0x55720dcc1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.186+0000 7fa6d2dc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720dcb93b0 con 0x55720dcb0810 2024-03-21T11:38:36.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.186+0000 7fa6d2dc3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720dbd10d0 con 0x55720dcb23a0 2024-03-21T11:38:36.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.186+0000 7fa6d2dc3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720dcb9530 con 0x55720dcb19e0 2024-03-21T11:38:36.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.186+0000 7fa6ccc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb19e0 0x55720dcbadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.186+0000 7fa6ce000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb23a0 0x55720dcc1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.186+0000 7fa6ccc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb19e0 0x55720dcbadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60438/0 (socket says 172.21.15.47:60438) 2024-03-21T11:38:36.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.186+0000 7fa6ccc00640 1 -- 172.21.15.47:0/1704297831 learned_addr learned my addr 172.21.15.47:0/1704297831 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:36.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.186+0000 7fa6d2dc3740 10 client.0 _ll_register_callbacks cb 0x55720dbd5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:36.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.187+0000 7fa6ce000640 1 -- 172.21.15.47:0/1704297831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb19e0 msgr2=0x55720dcbadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:36.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.187+0000 7fa6ce000640 1 --2- 172.21.15.47:0/1704297831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb19e0 0x55720dcbadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.187+0000 7fa6ce000640 1 -- 172.21.15.47:0/1704297831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 msgr2=0x55720dcba880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:36.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.187+0000 7fa6ce000640 1 --2- 172.21.15.47:0/1704297831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 0x55720dcba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.187+0000 7fa6ce000640 1 -- 172.21.15.47:0/1704297831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55720d9293f0 con 0x55720dcb23a0 2024-03-21T11:38:36.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.187+0000 7fa6aea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:36.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.187+0000 7fa6ce000640 1 --2- 172.21.15.47:0/1704297831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb23a0 0x55720dcc1e00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00324c0 tx=0x7fa6c00095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d49cab7da10e355a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:36.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.189+0000 7fa6aea00640 1 -- 172.21.15.47:0/1704297831 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0039040 con 0x55720dcb23a0 2024-03-21T11:38:36.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.189+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/1704297831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6b401a940 con 0x55720dcb23a0 2024-03-21T11:38:36.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.189+0000 7fa6aea00640 1 -- 172.21.15.47:0/1704297831 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa6c00040c0 con 0x55720dcb23a0 2024-03-21T11:38:36.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.190+0000 7fa6aea00640 1 -- 172.21.15.47:0/1704297831 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6c00042c0 con 0x55720dcb23a0 2024-03-21T11:38:36.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.190+0000 7fa6d2dc3740 10 client.30566 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:36.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.190+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/1704297831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55720d9f37d0 con 0x55720dcb23a0 2024-03-21T11:38:36.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.191+0000 7fa6aea00640 1 -- 172.21.15.47:0/1704297831 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6b401a940 con 0x55720dcb23a0 2024-03-21T11:38:36.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.193+0000 7fa6aea00640 1 -- 172.21.15.47:0/1704297831 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fa6c0053030 con 0x55720dcb23a0 2024-03-21T11:38:36.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.193+0000 7fa6d2dc3740 10 client.30566 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:36.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.193+0000 7fa6d2dc3740 10 client.30566 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:38:36.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.193+0000 7fa6d2dc3740 20 client.30566 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:36.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.193+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/1704297831 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720dcc6340 0x55720dcc8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.193+0000 7fa6d2dc3740 4 client.30566 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:36.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.193+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/1704297831 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55720dc22800 con 0x55720dcc6340 2024-03-21T11:38:36.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.194+0000 7fa6cd600640 1 --2- 172.21.15.47:0/1704297831 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720dcc6340 0x55720dcc8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.195+0000 7fa6cd600640 1 --2- 172.21.15.47:0/1704297831 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720dcc6340 0x55720dcc8770 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9706b79e181eecb server_cookie=53317ed360363f92 in_seq=0 out_seq=0 2024-03-21T11:38:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.195+0000 7fa6aea00640 10 client.30566 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:38:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.197+0000 7fa6aea00640 1 -- 172.21.15.47:0/1704297831 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55720dc22800 con 0x55720dcc6340 2024-03-21T11:38:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.197+0000 7fa6aea00640 10 client.30566 handle_command_reply: tid=0 2024-03-21T11:38:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.197+0000 7fa6d2dc3740 1 client.30566 shutdown 2024-03-21T11:38:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.198+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/1704297831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb23a0 msgr2=0x55720dcc1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.198+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/1704297831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb23a0 0x55720dcc1e00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00324c0 tx=0x7fa6c00095c0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.198+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/1704297831 shutdown_connections 2024-03-21T11:38:36.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.198+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/1704297831 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720dcc6340 0x55720dcc8770 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.198+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/1704297831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb23a0 0x55720dcc1e00 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.198+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/1704297831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb19e0 0x55720dcbadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.198+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/1704297831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 0x55720dcba880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.198+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/1704297831 >> 172.21.15.47:0/1704297831 conn(0x55720dcabc00 msgr2=0x55720dcaaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.198+0000 7fa6aea00640 0 client.30566 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:38:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.199+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/1704297831 shutdown_connections 2024-03-21T11:38:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.199+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/1704297831 wait complete. 2024-03-21T11:38:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.200+0000 7fa6d2dc3740 20 client.30566 trim_cache size 0 max 16384 2024-03-21T11:38:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.230+0000 7fa6d2dc3740 1 Processor -- start 2024-03-21T11:38:36.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.230+0000 7fa6d2dc3740 1 -- start start 2024-03-21T11:38:36.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.230+0000 7fa6d2dc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 0x55720dcb86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.230+0000 7fa6d2dc3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb19e0 0x55720dcc1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.230+0000 7fa6d2dc3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb23a0 0x55720dcb47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.230+0000 7fa6d2dc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a800fa60 con 0x55720dcb0810 2024-03-21T11:38:36.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.231+0000 7fa6d2dc3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a8011560 con 0x55720dcb19e0 2024-03-21T11:38:36.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.231+0000 7fa6d2dc3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a80100c0 con 0x55720dcb23a0 2024-03-21T11:38:36.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.231+0000 7fa6ce000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb23a0 0x55720dcb47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.231+0000 7fa6ccc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb19e0 0x55720dcc1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.231+0000 7fa6ccc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb19e0 0x55720dcc1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53134/0 (socket says 172.21.15.47:53134) 2024-03-21T11:38:36.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.231+0000 7fa6ccc00640 1 -- 172.21.15.47:0/2427895153 learned_addr learned my addr 172.21.15.47:0/2427895153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:36.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.231+0000 7fa6ce000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb23a0 0x55720dcb47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60450/0 (socket says 172.21.15.47:60450) 2024-03-21T11:38:36.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.231+0000 7fa6cd600640 1 --2- 172.21.15.47:0/2427895153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 0x55720dcb86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.231+0000 7fa6ce000640 1 -- 172.21.15.47:0/2427895153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb19e0 msgr2=0x55720dcc1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:36.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.231+0000 7fa6ce000640 1 --2- 172.21.15.47:0/2427895153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb19e0 0x55720dcc1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.231+0000 7fa6ce000640 1 -- 172.21.15.47:0/2427895153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 msgr2=0x55720dcb86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:36.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.231+0000 7fa6ce000640 1 --2- 172.21.15.47:0/2427895153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 0x55720dcb86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.231+0000 7fa6ce000640 1 -- 172.21.15.47:0/2427895153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55720dcb9930 con 0x55720dcb23a0 2024-03-21T11:38:36.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.231+0000 7fa6ccc00640 1 --2- 172.21.15.47:0/2427895153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb19e0 0x55720dcc1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:36.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.233+0000 7fa6ce000640 1 --2- 172.21.15.47:0/2427895153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb23a0 0x55720dcb47b0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00324c0 tx=0x7fa6c003f9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be0fc73283813c3e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:36.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.233+0000 7fa6b9c00640 1 -- 172.21.15.47:0/2427895153 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0039040 con 0x55720dcb23a0 2024-03-21T11:38:36.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.233+0000 7fa6b9c00640 1 -- 172.21.15.47:0/2427895153 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6c0047040 con 0x55720dcb23a0 2024-03-21T11:38:36.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.233+0000 7fa6b9c00640 1 -- 172.21.15.47:0/2427895153 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c005d440 con 0x55720dcb23a0 2024-03-21T11:38:36.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.235+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/2427895153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb23a0 msgr2=0x55720dcb47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.235+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/2427895153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb23a0 0x55720dcb47b0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00324c0 tx=0x7fa6c003f9d0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.235+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/2427895153 shutdown_connections 2024-03-21T11:38:36.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.235+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/2427895153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb23a0 0x55720dcb47b0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.235+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/2427895153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb19e0 0x55720dcc1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.235+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/2427895153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb0810 0x55720dcb86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.235+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/2427895153 >> 172.21.15.47:0/2427895153 conn(0x55720dcabc00 msgr2=0x55720dcaf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:36.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.235+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/2427895153 shutdown_connections 2024-03-21T11:38:36.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.235+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/2427895153 wait complete. 2024-03-21T11:38:36.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.235+0000 7fa6d2dc3740 1 -- start start 2024-03-21T11:38:36.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.235+0000 7fa6d2dc3740 1 Processor -- start 2024-03-21T11:38:36.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.236+0000 7fa6d2dc3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb0810 0x55720dcc1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.236+0000 7fa6d2dc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb19e0 0x55720dcbe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.236+0000 7fa6d2dc3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb23a0 0x55720dcbe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.236+0000 7fa6d2dc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a8010e40 con 0x55720dcb19e0 2024-03-21T11:38:36.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.236+0000 7fa6d2dc3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a80107e0 con 0x55720dcb23a0 2024-03-21T11:38:36.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.236+0000 7fa6d2dc3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a800dc20 con 0x55720dcb0810 2024-03-21T11:38:36.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.236+0000 7fa6d2dc3740 10 client.0 _ll_register_callbacks cb 0x7fa6c0037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:36.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.236+0000 7fa6cd600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb0810 0x55720dcc1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.236+0000 7fa6cd600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb0810 0x55720dcc1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60466/0 (socket says 172.21.15.47:60466) 2024-03-21T11:38:36.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.236+0000 7fa6cd600640 1 -- 172.21.15.47:0/252030141 learned_addr learned my addr 172.21.15.47:0/252030141 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:36.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.236+0000 7fa6ce000640 1 --2- 172.21.15.47:0/252030141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb23a0 0x55720dcbe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.237+0000 7fa6ce000640 1 -- 172.21.15.47:0/252030141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb0810 msgr2=0x55720dcc1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:36.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.237+0000 7fa6ce000640 1 --2- 172.21.15.47:0/252030141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb0810 0x55720dcc1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.237+0000 7fa6ce000640 1 -- 172.21.15.47:0/252030141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb19e0 msgr2=0x55720dcbe4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:36.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.237+0000 7fa6ccc00640 1 --2- 172.21.15.47:0/252030141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb19e0 0x55720dcbe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.237+0000 7fa6ce000640 1 --2- 172.21.15.47:0/252030141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb19e0 0x55720dcbe4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.237+0000 7fa6ce000640 1 -- 172.21.15.47:0/252030141 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55720dcc1fc0 con 0x55720dcb23a0 2024-03-21T11:38:36.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.237+0000 7fa6ccc00640 1 --2- 172.21.15.47:0/252030141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb19e0 0x55720dcbe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:36.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.237+0000 7fa6ce000640 1 --2- 172.21.15.47:0/252030141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb23a0 0x55720dcbe9e0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa6c002c710 tx=0x7fa6c003f8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36dcacbfc830eb13 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:36.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.237+0000 7fa6aea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:36.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.238+0000 7fa6aea00640 1 -- 172.21.15.47:0/252030141 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0061070 con 0x55720dcb23a0 2024-03-21T11:38:36.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.238+0000 7fa6aea00640 1 -- 172.21.15.47:0/252030141 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa6c0003f40 con 0x55720dcb23a0 2024-03-21T11:38:36.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.238+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/252030141 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6c0047040 con 0x55720dcb23a0 2024-03-21T11:38:36.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.238+0000 7fa6aea00640 1 -- 172.21.15.47:0/252030141 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6c006b050 con 0x55720dcb23a0 2024-03-21T11:38:36.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.238+0000 7fa6d2dc3740 10 client.30572 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:36.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.238+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/252030141 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55720dd22b00 con 0x55720dcb23a0 2024-03-21T11:38:36.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.239+0000 7fa6aea00640 1 -- 172.21.15.47:0/252030141 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6c0047040 con 0x55720dcb23a0 2024-03-21T11:38:36.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.239+0000 7fa6aea00640 1 -- 172.21.15.47:0/252030141 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fa6c0053070 con 0x55720dcb23a0 2024-03-21T11:38:36.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.239+0000 7fa6d2dc3740 10 client.30572 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:36.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.239+0000 7fa6d2dc3740 10 client.30572 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:38:36.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.239+0000 7fa6d2dc3740 20 client.30572 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:36.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.239+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/252030141 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720dd233f0 0x55720dd25810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.239+0000 7fa6d2dc3740 4 client.30572 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:38:36.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.239+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/252030141 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55720dc1cb50 con 0x55720dd233f0 2024-03-21T11:38:36.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.240+0000 7fa6cd600640 1 --2- 172.21.15.47:0/252030141 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720dd233f0 0x55720dd25810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.241+0000 7fa6cd600640 1 --2- 172.21.15.47:0/252030141 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720dd233f0 0x55720dd25810 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7499c79d659b0be0 server_cookie=ffa6f52d55bd0006 in_seq=0 out_seq=0 2024-03-21T11:38:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.241+0000 7fa6aea00640 10 client.30572 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:38:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.243+0000 7fa6aea00640 1 -- 172.21.15.47:0/252030141 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55720dc1cb50 con 0x55720dd233f0 2024-03-21T11:38:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.243+0000 7fa6aea00640 10 client.30572 handle_command_reply: tid=0 2024-03-21T11:38:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.243+0000 7fa6d2dc3740 1 client.30572 shutdown 2024-03-21T11:38:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.243+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/252030141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb23a0 msgr2=0x55720dcbe9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.243+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/252030141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb23a0 0x55720dcbe9e0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa6c002c710 tx=0x7fa6c003f8e0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.244+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/252030141 shutdown_connections 2024-03-21T11:38:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.244+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/252030141 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55720dd233f0 0x55720dd25810 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.244+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/252030141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55720dcb23a0 0x55720dcbe9e0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.244+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/252030141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55720dcb19e0 0x55720dcbe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.244+0000 7fa6d2dc3740 1 --2- 172.21.15.47:0/252030141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55720dcb0810 0x55720dcc1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.244+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/252030141 >> 172.21.15.47:0/252030141 conn(0x55720dcabc00 msgr2=0x55720dcaf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.244+0000 7fa6aea00640 0 client.30572 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:38:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.244+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/252030141 shutdown_connections 2024-03-21T11:38:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.245+0000 7fa6d2dc3740 1 -- 172.21.15.47:0/252030141 wait complete. 2024-03-21T11:38:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.245+0000 7fa6d2dc3740 20 client.30572 trim_cache size 0 max 16384 2024-03-21T11:38:36.247 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:38:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.246+0000 7fa6d0000640 1 -- 172.21.15.47:0/1582261207 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6b00a2f00 msgr2=0x7fa6b00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.246+0000 7fa6d0000640 1 --2- 172.21.15.47:0/1582261207 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6b00a2f00 0x7fa6b00a53c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009bf0 tx=0x7fa6c0032590 comp rx=0 tx=0).stop 2024-03-21T11:38:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.246+0000 7fa6d0000640 1 -- 172.21.15.47:0/1582261207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c810b010 msgr2=0x7fa6c8119e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.246+0000 7fa6d0000640 1 --2- 172.21.15.47:0/1582261207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c810b010 0x7fa6c8119e30 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00e9f0 tx=0x7fa6bc00edf0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.246+0000 7fa6d0000640 1 -- 172.21.15.47:0/1582261207 shutdown_connections 2024-03-21T11:38:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.246+0000 7fa6d0000640 1 --2- 172.21.15.47:0/1582261207 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa6b00a2f00 0x7fa6b00a53c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.246+0000 7fa6d0000640 1 --2- 172.21.15.47:0/1582261207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6c810d9c0 0x7fa6c8117f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.246+0000 7fa6d0000640 1 --2- 172.21.15.47:0/1582261207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6c810b010 0x7fa6c8119e30 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.246+0000 7fa6d0000640 1 --2- 172.21.15.47:0/1582261207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6c81086a0 0x7fa6c81198f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.246+0000 7fa6d0000640 1 -- 172.21.15.47:0/1582261207 >> 172.21.15.47:0/1582261207 conn(0x7fa6c806e500 msgr2=0x7fa6c810b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.247+0000 7fa6d0000640 1 -- 172.21.15.47:0/1582261207 shutdown_connections 2024-03-21T11:38:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.247+0000 7fa6d0000640 1 -- 172.21.15.47:0/1582261207 wait complete. 2024-03-21T11:38:36.270 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:38:36.270 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:38:36.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.569+0000 7fbeba000640 1 -- 172.21.15.47:0/3196254646 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac02c860 con 0x7fbebc1016d0 2024-03-21T11:38:36.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.571+0000 7fbec2800640 1 -- 172.21.15.47:0/3196254646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbebc1016d0 msgr2=0x7fbebc101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.571+0000 7fbec2800640 1 --2- 172.21.15.47:0/3196254646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbebc1016d0 0x7fbebc101af0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009900 tx=0x7fbeac02bc60 comp rx=0 tx=0).stop 2024-03-21T11:38:36.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.571+0000 7fbec2800640 1 -- 172.21.15.47:0/3196254646 shutdown_connections 2024-03-21T11:38:36.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.571+0000 7fbec2800640 1 --2- 172.21.15.47:0/3196254646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbebc103280 0x7fbebc109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.571+0000 7fbec2800640 1 --2- 172.21.15.47:0/3196254646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc1028c0 0x7fbebc102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.571+0000 7fbec2800640 1 --2- 172.21.15.47:0/3196254646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbebc1016d0 0x7fbebc101af0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.571+0000 7fbec2800640 1 -- 172.21.15.47:0/3196254646 >> 172.21.15.47:0/3196254646 conn(0x7fbebc0fcc40 msgr2=0x7fbebc0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:36.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.572+0000 7fbec2800640 1 -- 172.21.15.47:0/3196254646 shutdown_connections 2024-03-21T11:38:36.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.572+0000 7fbec2800640 1 -- 172.21.15.47:0/3196254646 wait complete. 2024-03-21T11:38:36.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.574+0000 7fbec2800640 1 Processor -- start 2024-03-21T11:38:36.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.576+0000 7fbec2800640 1 -- start start 2024-03-21T11:38:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.576+0000 7fbec2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbebc1016d0 0x7fbebc199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.576+0000 7fbec2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbebc1028c0 0x7fbebc19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.576+0000 7fbec2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc103280 0x7fbebc1a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.576+0000 7fbec2800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc10d470 con 0x7fbebc103280 2024-03-21T11:38:36.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.576+0000 7fbec2800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc10d2f0 con 0x7fbebc1016d0 2024-03-21T11:38:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.576+0000 7fbec2800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc10d5f0 con 0x7fbebc1028c0 2024-03-21T11:38:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.577+0000 7fbebb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbebc1016d0 0x7fbebc199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.577+0000 7fbebb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbebc1016d0 0x7fbebc199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53164/0 (socket says 172.21.15.47:53164) 2024-03-21T11:38:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.577+0000 7fbebb400640 1 -- 172.21.15.47:0/733120647 learned_addr learned my addr 172.21.15.47:0/733120647 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.577+0000 7fbebaa00640 1 --2- 172.21.15.47:0/733120647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbebc1028c0 0x7fbebc19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.577+0000 7fbebbe00640 1 --2- 172.21.15.47:0/733120647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc103280 0x7fbebc1a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.577+0000 7fbebb400640 1 -- 172.21.15.47:0/733120647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbebc1028c0 msgr2=0x7fbebc19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.577+0000 7fbebb400640 1 --2- 172.21.15.47:0/733120647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbebc1028c0 0x7fbebc19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.577+0000 7fbebb400640 1 -- 172.21.15.47:0/733120647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc103280 msgr2=0x7fbebc1a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.577+0000 7fbebb400640 1 --2- 172.21.15.47:0/733120647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc103280 0x7fbebc1a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.577+0000 7fbebb400640 1 -- 172.21.15.47:0/733120647 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc103cd0 con 0x7fbebc1016d0 2024-03-21T11:38:36.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.578+0000 7fbebb400640 1 --2- 172.21.15.47:0/733120647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbebc1016d0 0x7fbebc199d10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009230 tx=0x7fbeac02eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:36.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.578+0000 7fbeb3e00640 1 -- 172.21.15.47:0/733120647 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac041020 con 0x7fbebc1016d0 2024-03-21T11:38:36.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.578+0000 7fbec2800640 1 -- 172.21.15.47:0/733120647 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbebc103f60 con 0x7fbebc1016d0 2024-03-21T11:38:36.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.578+0000 7fbec2800640 1 -- 172.21.15.47:0/733120647 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbebc104520 con 0x7fbebc1016d0 2024-03-21T11:38:36.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.579+0000 7fbeb3e00640 1 -- 172.21.15.47:0/733120647 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbeac004d90 con 0x7fbebc1016d0 2024-03-21T11:38:36.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.579+0000 7fbeb3e00640 1 -- 172.21.15.47:0/733120647 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac02e2b0 con 0x7fbebc1016d0 2024-03-21T11:38:36.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.583+0000 7fbec540c740 1 Processor -- start 2024-03-21T11:38:36.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.584+0000 7fbec540c740 1 -- start start 2024-03-21T11:38:36.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.584+0000 7fbec540c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf2f810 0x561e4cf2fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.585+0000 7fbebbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf2f810 0x561e4cf2fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.585+0000 7fbebbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf2f810 0x561e4cf2fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53176/0 (socket says 172.21.15.47:53176) 2024-03-21T11:38:36.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.585+0000 7fbec540c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf309e0 0x561e4cf30e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.585+0000 7fbec540c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf313a0 0x561e4cf37c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.585+0000 7fbec540c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e4ce942c0 con 0x561e4cf313a0 2024-03-21T11:38:36.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.585+0000 7fbec540c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e4ce3cc90 con 0x561e4cf2f810 2024-03-21T11:38:36.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.585+0000 7fbec540c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e4ce90360 con 0x561e4cf309e0 2024-03-21T11:38:36.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.585+0000 7fbebbe00640 1 -- 172.21.15.47:0/1847792770 learned_addr learned my addr 172.21.15.47:0/1847792770 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:36.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.586+0000 7fbebaa00640 1 --2- 172.21.15.47:0/1847792770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf313a0 0x561e4cf37c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.586+0000 7fbeb3e00640 1 -- 172.21.15.47:0/733120647 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbeac044020 con 0x7fbebc1016d0 2024-03-21T11:38:36.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.587+0000 7fbebbe00640 1 -- 172.21.15.47:0/1847792770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf309e0 msgr2=0x561e4cf30e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:38:36.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.587+0000 7fbebbe00640 1 --2- 172.21.15.47:0/1847792770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf309e0 0x561e4cf30e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.587+0000 7fbebbe00640 1 -- 172.21.15.47:0/1847792770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf313a0 msgr2=0x561e4cf37c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:36.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.587+0000 7fbebbe00640 1 --2- 172.21.15.47:0/1847792770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf313a0 0x561e4cf37c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.587+0000 7fbebbe00640 1 -- 172.21.15.47:0/1847792770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e4ce61080 con 0x561e4cf2f810 2024-03-21T11:38:36.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.588+0000 7fbebbe00640 1 --2- 172.21.15.47:0/1847792770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf2f810 0x561e4cf2fc30 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009f20 tx=0x7fbeb400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acdd29bfa44dce5c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:36.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.589+0000 7fbeb0c00640 1 -- 172.21.15.47:0/1847792770 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb401a070 con 0x561e4cf2f810 2024-03-21T11:38:36.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.590+0000 7fbeb0c00640 1 -- 172.21.15.47:0/1847792770 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbeb40158f0 con 0x561e4cf2f810 2024-03-21T11:38:36.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.590+0000 7fbeb0c00640 1 -- 172.21.15.47:0/1847792770 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb400a900 con 0x561e4cf2f810 2024-03-21T11:38:36.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.591+0000 7fbeb3e00640 1 --2- 172.21.15.47:0/733120647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbea00a3020 0x7fbea00a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.591+0000 7fbeb3e00640 1 -- 172.21.15.47:0/733120647 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbeac0f11c0 con 0x7fbebc1016d0 2024-03-21T11:38:36.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.591+0000 7fbebb400640 1 --2- 172.21.15.47:0/733120647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbea00a3020 0x7fbea00a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.591+0000 7fbec540c740 1 -- 172.21.15.47:0/1847792770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf2f810 msgr2=0x561e4cf2fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.591+0000 7fbec540c740 1 --2- 172.21.15.47:0/1847792770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf2f810 0x561e4cf2fc30 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009f20 tx=0x7fbeb400c8b0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.592+0000 7fbec540c740 1 -- 172.21.15.47:0/1847792770 shutdown_connections 2024-03-21T11:38:36.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.592+0000 7fbec540c740 1 --2- 172.21.15.47:0/1847792770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf313a0 0x561e4cf37c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.592+0000 7fbec540c740 1 --2- 172.21.15.47:0/1847792770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf309e0 0x561e4cf30e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.592+0000 7fbec540c740 1 --2- 172.21.15.47:0/1847792770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf2f810 0x561e4cf2fc30 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.592+0000 7fbec540c740 1 -- 172.21.15.47:0/1847792770 >> 172.21.15.47:0/1847792770 conn(0x561e4cf2ac00 msgr2=0x561e4cf2b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:36.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.592+0000 7fbec540c740 1 -- 172.21.15.47:0/1847792770 shutdown_connections 2024-03-21T11:38:36.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.593+0000 7fbebb400640 1 --2- 172.21.15.47:0/733120647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbea00a3020 0x7fbea00a54e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbeac02e040 tx=0x7fbeac009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:36.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.593+0000 7fbec540c740 1 -- 172.21.15.47:0/1847792770 wait complete. 2024-03-21T11:38:36.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.594+0000 7fbec540c740 1 -- start start 2024-03-21T11:38:36.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.595+0000 7fbec540c740 1 Processor -- start 2024-03-21T11:38:36.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.597+0000 7fbec540c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf2f810 0x561e4cf39af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.598+0000 7fbec540c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf309e0 0x561e4cf3a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.598+0000 7fbec540c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf313a0 0x561e4cf41070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.598+0000 7fbec540c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e4cf383b0 con 0x561e4cf2f810 2024-03-21T11:38:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.598+0000 7fbec540c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e4ce500d0 con 0x561e4cf313a0 2024-03-21T11:38:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.598+0000 7fbec540c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e4cf38530 con 0x561e4cf309e0 2024-03-21T11:38:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.598+0000 7fbec540c740 10 client.0 _ll_register_callbacks cb 0x561e4ce54e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.598+0000 7fbebaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf309e0 0x561e4cf3a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.598+0000 7fbebaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf309e0 0x561e4cf3a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60502/0 (socket says 172.21.15.47:60502) 2024-03-21T11:38:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.599+0000 7fbebaa00640 1 -- 172.21.15.47:0/3896163024 learned_addr learned my addr 172.21.15.47:0/3896163024 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.599+0000 7fbebbe00640 1 --2- 172.21.15.47:0/3896163024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf313a0 0x561e4cf41070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.599+0000 7fbebaa00640 1 -- 172.21.15.47:0/3896163024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf313a0 msgr2=0x561e4cf41070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:36.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.599+0000 7fbebaa00640 1 --2- 172.21.15.47:0/3896163024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf313a0 0x561e4cf41070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.599+0000 7fbebaa00640 1 -- 172.21.15.47:0/3896163024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf2f810 msgr2=0x561e4cf39af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:36.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.599+0000 7fbebaa00640 1 --2- 172.21.15.47:0/3896163024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf2f810 0x561e4cf39af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.600+0000 7fbebaa00640 1 -- 172.21.15.47:0/3896163024 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e4cba83f0 con 0x561e4cf309e0 2024-03-21T11:38:36.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.600+0000 7fbebbe00640 1 --2- 172.21.15.47:0/3896163024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf313a0 0x561e4cf41070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:38:36.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.600+0000 7fbebaa00640 1 --2- 172.21.15.47:0/3896163024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf309e0 0x561e4cf3a030 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fbea4011540 tx=0x7fbea4011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a57b357c61fea0c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:36.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.600+0000 7fbea9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:38:36.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.603+0000 7fbea9600640 1 -- 172.21.15.47:0/3896163024 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4013670 con 0x561e4cf309e0 2024-03-21T11:38:36.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.603+0000 7fbea9600640 1 -- 172.21.15.47:0/3896163024 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbea4013810 con 0x561e4cf309e0 2024-03-21T11:38:36.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.604+0000 7fbec540c740 1 -- 172.21.15.47:0/3896163024 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbeb40158f0 con 0x561e4cf309e0 2024-03-21T11:38:36.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.605+0000 7fbea9600640 1 -- 172.21.15.47:0/3896163024 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbea4013a50 con 0x561e4cf309e0 2024-03-21T11:38:36.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.606+0000 7fbec540c740 10 client.30595 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:36.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.606+0000 7fbec540c740 1 -- 172.21.15.47:0/3896163024 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561e4cc727d0 con 0x561e4cf309e0 2024-03-21T11:38:36.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.607+0000 7fbea9600640 1 -- 172.21.15.47:0/3896163024 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbeb40158f0 con 0x561e4cf309e0 2024-03-21T11:38:36.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.607+0000 7fbea9600640 1 -- 172.21.15.47:0/3896163024 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fbea4027420 con 0x561e4cf309e0 2024-03-21T11:38:36.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.607+0000 7fbec540c740 10 client.30595 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:36.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.607+0000 7fbec540c740 10 client.30595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:36.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.607+0000 7fbec540c740 20 client.30595 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:36.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.607+0000 7fbec540c740 1 --2- 172.21.15.47:0/3896163024 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561e4cf45620 0x561e4cf47a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.607+0000 7fbec540c740 4 client.30595 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:36.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.607+0000 7fbec540c740 1 -- 172.21.15.47:0/3896163024 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e4cea1800 con 0x561e4cf45620 2024-03-21T11:38:36.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.609+0000 7fbebbe00640 1 --2- 172.21.15.47:0/3896163024 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561e4cf45620 0x561e4cf47a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.610+0000 7fbea9600640 10 client.30595 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:36.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.610+0000 7fbebbe00640 1 --2- 172.21.15.47:0/3896163024 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561e4cf45620 0x561e4cf47a60 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce62da883e7c1af server_cookie=e8b11006de65f7d8 in_seq=0 out_seq=0 2024-03-21T11:38:36.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.615+0000 7fbea9600640 1 -- 172.21.15.47:0/3896163024 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561e4cea1800 con 0x561e4cf45620 2024-03-21T11:38:36.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.615+0000 7fbea9600640 10 client.30595 handle_command_reply: tid=0 2024-03-21T11:38:36.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.615+0000 7fbec540c740 1 client.30595 shutdown 2024-03-21T11:38:36.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.615+0000 7fbec540c740 1 -- 172.21.15.47:0/3896163024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf309e0 msgr2=0x561e4cf3a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.615+0000 7fbec540c740 1 --2- 172.21.15.47:0/3896163024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf309e0 0x561e4cf3a030 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fbea4011540 tx=0x7fbea4011a00 comp rx=0 tx=0).stop 2024-03-21T11:38:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.617+0000 7fbec540c740 1 -- 172.21.15.47:0/3896163024 shutdown_connections 2024-03-21T11:38:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.617+0000 7fbec540c740 1 --2- 172.21.15.47:0/3896163024 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561e4cf45620 0x561e4cf47a60 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.617+0000 7fbec540c740 1 --2- 172.21.15.47:0/3896163024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf313a0 0x561e4cf41070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.617+0000 7fbec540c740 1 --2- 172.21.15.47:0/3896163024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf309e0 0x561e4cf3a030 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.617+0000 7fbec540c740 1 --2- 172.21.15.47:0/3896163024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf2f810 0x561e4cf39af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.617+0000 7fbec540c740 1 -- 172.21.15.47:0/3896163024 >> 172.21.15.47:0/3896163024 conn(0x561e4cf2ac00 msgr2=0x561e4cf29eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:36.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.617+0000 7fbea9600640 0 client.30595 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:36.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.618+0000 7fbec540c740 1 -- 172.21.15.47:0/3896163024 shutdown_connections 2024-03-21T11:38:36.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.618+0000 7fbec540c740 1 -- 172.21.15.47:0/3896163024 wait complete. 2024-03-21T11:38:36.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.618+0000 7fbec540c740 20 client.30595 trim_cache size 0 max 16384 2024-03-21T11:38:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.642+0000 7fbec540c740 1 Processor -- start 2024-03-21T11:38:36.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.643+0000 7fbec540c740 1 -- start start 2024-03-21T11:38:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.643+0000 7fbec540c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf2f810 0x561e4cf29620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.643+0000 7fbec540c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf309e0 0x561e4cf41030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.643+0000 7fbec540c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf313a0 0x561e4cf29b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.643+0000 7fbec540c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9800fac0 con 0x561e4cf309e0 2024-03-21T11:38:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.643+0000 7fbec540c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe980115c0 con 0x561e4cf313a0 2024-03-21T11:38:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.643+0000 7fbec540c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe98010120 con 0x561e4cf2f810 2024-03-21T11:38:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.644+0000 7fbebbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf313a0 0x561e4cf29b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.644+0000 7fbebbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf313a0 0x561e4cf29b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53196/0 (socket says 172.21.15.47:53196) 2024-03-21T11:38:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.644+0000 7fbebbe00640 1 -- 172.21.15.47:0/4261058150 learned_addr learned my addr 172.21.15.47:0/4261058150 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:36.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.644+0000 7fbebaa00640 1 --2- 172.21.15.47:0/4261058150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf309e0 0x561e4cf41030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.644+0000 7fbebbe00640 1 --2- 172.21.15.47:0/4261058150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf2f810 0x561e4cf29620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.645+0000 7fbebbe00640 1 -- 172.21.15.47:0/4261058150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf2f810 msgr2=0x561e4cf29620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:36.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.645+0000 7fbebbe00640 1 --2- 172.21.15.47:0/4261058150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf2f810 0x561e4cf29620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.645+0000 7fbebbe00640 1 -- 172.21.15.47:0/4261058150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf309e0 msgr2=0x561e4cf41030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:36.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.645+0000 7fbebbe00640 1 --2- 172.21.15.47:0/4261058150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf309e0 0x561e4cf41030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.646+0000 7fbebbe00640 1 -- 172.21.15.47:0/4261058150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e4cf3a380 con 0x561e4cf313a0 2024-03-21T11:38:36.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.646+0000 7fbebbe00640 1 --2- 172.21.15.47:0/4261058150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf313a0 0x561e4cf29b60 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4018040 tx=0x7fbeb4004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f8c566aeeca1df1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:36.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.647+0000 7fbeabe00640 1 -- 172.21.15.47:0/4261058150 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb401a040 con 0x561e4cf313a0 2024-03-21T11:38:36.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.647+0000 7fbeabe00640 1 -- 172.21.15.47:0/4261058150 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbeb402a030 con 0x561e4cf313a0 2024-03-21T11:38:36.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.647+0000 7fbeabe00640 1 -- 172.21.15.47:0/4261058150 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4026b20 con 0x561e4cf313a0 2024-03-21T11:38:36.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.648+0000 7fbec540c740 1 -- 172.21.15.47:0/4261058150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf313a0 msgr2=0x561e4cf29b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.648+0000 7fbec540c740 1 --2- 172.21.15.47:0/4261058150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf313a0 0x561e4cf29b60 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4018040 tx=0x7fbeb4004b50 comp rx=0 tx=0).stop 2024-03-21T11:38:36.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.648+0000 7fbec540c740 1 -- 172.21.15.47:0/4261058150 shutdown_connections 2024-03-21T11:38:36.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.648+0000 7fbec540c740 1 --2- 172.21.15.47:0/4261058150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf313a0 0x561e4cf29b60 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.648+0000 7fbec540c740 1 --2- 172.21.15.47:0/4261058150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf309e0 0x561e4cf41030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.648+0000 7fbec540c740 1 --2- 172.21.15.47:0/4261058150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf2f810 0x561e4cf29620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.648+0000 7fbec540c740 1 -- 172.21.15.47:0/4261058150 >> 172.21.15.47:0/4261058150 conn(0x561e4cf2ac00 msgr2=0x561e4cf378f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:36.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.649+0000 7fbec540c740 1 -- 172.21.15.47:0/4261058150 shutdown_connections 2024-03-21T11:38:36.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.649+0000 7fbec540c740 1 -- 172.21.15.47:0/4261058150 wait complete. 2024-03-21T11:38:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.650+0000 7fbec540c740 1 -- start start 2024-03-21T11:38:36.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.650+0000 7fbec540c740 1 Processor -- start 2024-03-21T11:38:36.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.651+0000 7fbec540c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf2f810 0x561e4cf3ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.652+0000 7fbec540c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf309e0 0x561e4cf3cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.652+0000 7fbec540c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf313a0 0x561e4cf3ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.652+0000 7fbebbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf2f810 0x561e4cf3ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.652+0000 7fbebbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf2f810 0x561e4cf3ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60514/0 (socket says 172.21.15.47:60514) 2024-03-21T11:38:36.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.652+0000 7fbebbe00640 1 -- 172.21.15.47:0/10836086 learned_addr learned my addr 172.21.15.47:0/10836086 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:36.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.652+0000 7fbec540c740 1 -- 172.21.15.47:0/10836086 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe98010ea0 con 0x561e4cf313a0 2024-03-21T11:38:36.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.652+0000 7fbec540c740 1 -- 172.21.15.47:0/10836086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe98010840 con 0x561e4cf309e0 2024-03-21T11:38:36.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.652+0000 7fbec540c740 1 -- 172.21.15.47:0/10836086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9800dc80 con 0x561e4cf2f810 2024-03-21T11:38:36.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.652+0000 7fbebaa00640 1 --2- 172.21.15.47:0/10836086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf309e0 0x561e4cf3cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.653+0000 7fbec540c740 10 client.0 _ll_register_callbacks cb 0x7fbea4009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:36.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.653+0000 7fbebaa00640 1 -- 172.21.15.47:0/10836086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf2f810 msgr2=0x561e4cf3ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:36.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.653+0000 7fbebaa00640 1 --2- 172.21.15.47:0/10836086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf2f810 0x561e4cf3ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.653+0000 7fbebaa00640 1 -- 172.21.15.47:0/10836086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf313a0 msgr2=0x561e4cf3ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:36.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.653+0000 7fbebaa00640 1 --2- 172.21.15.47:0/10836086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf313a0 0x561e4cf3ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.653+0000 7fbebaa00640 1 -- 172.21.15.47:0/10836086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e4cf405c0 con 0x561e4cf309e0 2024-03-21T11:38:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.656+0000 7fbebaa00640 1 --2- 172.21.15.47:0/10836086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf309e0 0x561e4cf3cfb0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fbea4000c00 tx=0x7fbea40091f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66faad710b38a78a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.656+0000 7fbea9600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.657+0000 7fbea9600640 1 -- 172.21.15.47:0/10836086 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4013870 con 0x561e4cf309e0 2024-03-21T11:38:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.657+0000 7fbea9600640 1 -- 172.21.15.47:0/10836086 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fbea4013bf0 con 0x561e4cf309e0 2024-03-21T11:38:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.657+0000 7fbec540c740 1 -- 172.21.15.47:0/10836086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbeb402a030 con 0x561e4cf309e0 2024-03-21T11:38:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.657+0000 7fbea9600640 1 -- 172.21.15.47:0/10836086 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbea4013e10 con 0x561e4cf309e0 2024-03-21T11:38:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.658+0000 7fbec540c740 10 client.30590 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:36.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.658+0000 7fbec540c740 1 -- 172.21.15.47:0/10836086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561e4cfa1290 con 0x561e4cf309e0 2024-03-21T11:38:36.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.658+0000 7fbea9600640 1 -- 172.21.15.47:0/10836086 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbeb402a030 con 0x561e4cf309e0 2024-03-21T11:38:36.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.658+0000 7fbea9600640 1 -- 172.21.15.47:0/10836086 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fbea4006550 con 0x561e4cf309e0 2024-03-21T11:38:36.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.659+0000 7fbec540c740 10 client.30590 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:36.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.659+0000 7fbec540c740 10 client.30590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:36.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.659+0000 7fbec540c740 20 client.30590 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:36.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.659+0000 7fbec540c740 1 --2- 172.21.15.47:0/10836086 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561e4cfa1ba0 0x561e4cfa3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.659+0000 7fbec540c740 4 client.30590 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:38:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.659+0000 7fbec540c740 1 -- 172.21.15.47:0/10836086 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561e4ce9bb50 con 0x561e4cfa1ba0 2024-03-21T11:38:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.659+0000 7fbebbe00640 1 --2- 172.21.15.47:0/10836086 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561e4cfa1ba0 0x561e4cfa3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.660+0000 7fbebbe00640 1 --2- 172.21.15.47:0/10836086 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561e4cfa1ba0 0x561e4cfa3fc0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcdbc3ba8cb31a90 server_cookie=5cc007bf137aa198 in_seq=0 out_seq=0 2024-03-21T11:38:36.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.660+0000 7fbea9600640 10 client.30590 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.670+0000 7fbea9600640 1 -- 172.21.15.47:0/10836086 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561e4ce9bb50 con 0x561e4cfa1ba0 2024-03-21T11:38:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.670+0000 7fbea9600640 10 client.30590 handle_command_reply: tid=0 2024-03-21T11:38:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.670+0000 7fbec540c740 1 client.30590 shutdown 2024-03-21T11:38:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.672+0000 7fbec540c740 1 -- 172.21.15.47:0/10836086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf309e0 msgr2=0x561e4cf3cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.672+0000 7fbec540c740 1 --2- 172.21.15.47:0/10836086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf309e0 0x561e4cf3cfb0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fbea4000c00 tx=0x7fbea40091f0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.672+0000 7fbec540c740 1 -- 172.21.15.47:0/10836086 shutdown_connections 2024-03-21T11:38:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.672+0000 7fbec540c740 1 --2- 172.21.15.47:0/10836086 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561e4cfa1ba0 0x561e4cfa3fc0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.672+0000 7fbec540c740 1 --2- 172.21.15.47:0/10836086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e4cf313a0 0x561e4cf3ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.672+0000 7fbec540c740 1 --2- 172.21.15.47:0/10836086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561e4cf309e0 0x561e4cf3cfb0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.672+0000 7fbec540c740 1 --2- 172.21.15.47:0/10836086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e4cf2f810 0x561e4cf3ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.672+0000 7fbec540c740 1 -- 172.21.15.47:0/10836086 >> 172.21.15.47:0/10836086 conn(0x561e4cf2ac00 msgr2=0x561e4cf2de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.672+0000 7fbea9600640 0 client.30590 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.673+0000 7fbec540c740 1 -- 172.21.15.47:0/10836086 shutdown_connections 2024-03-21T11:38:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.673+0000 7fbec540c740 1 -- 172.21.15.47:0/10836086 wait complete. 2024-03-21T11:38:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.674+0000 7fbec540c740 20 client.30590 trim_cache size 0 max 16384 2024-03-21T11:38:36.674 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:38:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.675+0000 7fbec2800640 1 -- 172.21.15.47:0/733120647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbea00a3020 msgr2=0x7fbea00a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.675+0000 7fbec2800640 1 --2- 172.21.15.47:0/733120647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbea00a3020 0x7fbea00a54e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbeac02e040 tx=0x7fbeac009680 comp rx=0 tx=0).stop 2024-03-21T11:38:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.675+0000 7fbec2800640 1 -- 172.21.15.47:0/733120647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbebc1016d0 msgr2=0x7fbebc199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.675+0000 7fbec2800640 1 --2- 172.21.15.47:0/733120647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbebc1016d0 0x7fbebc199d10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009230 tx=0x7fbeac02eb90 comp rx=0 tx=0).stop 2024-03-21T11:38:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.675+0000 7fbec2800640 1 -- 172.21.15.47:0/733120647 shutdown_connections 2024-03-21T11:38:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.675+0000 7fbec2800640 1 --2- 172.21.15.47:0/733120647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fbea00a3020 0x7fbea00a54e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.675+0000 7fbec2800640 1 --2- 172.21.15.47:0/733120647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc103280 0x7fbebc1a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.675+0000 7fbec2800640 1 --2- 172.21.15.47:0/733120647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbebc1028c0 0x7fbebc19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.675+0000 7fbec2800640 1 --2- 172.21.15.47:0/733120647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbebc1016d0 0x7fbebc199d10 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.675+0000 7fbec2800640 1 -- 172.21.15.47:0/733120647 >> 172.21.15.47:0/733120647 conn(0x7fbebc0fcc40 msgr2=0x7fbebc108060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.675+0000 7fbec2800640 1 -- 172.21.15.47:0/733120647 shutdown_connections 2024-03-21T11:38:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:36.676+0000 7fbec2800640 1 -- 172.21.15.47:0/733120647 wait complete. 2024-03-21T11:38:36.926 INFO:tasks.workunit.client.0.smithi047.stdout: 10 39315 28.71 MB/sec warmup 80 sec latency 86.776 ms 2024-03-21T11:38:37.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:37 smithi047 ceph-mon[31351]: pgmap v858: 129 pgs: 129 active+clean; 322 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 608 op/s 2024-03-21T11:38:37.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:37 smithi100 ceph-mon[36245]: pgmap v858: 129 pgs: 129 active+clean; 322 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 608 op/s 2024-03-21T11:38:37.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:37 smithi102 ceph-mon[36748]: pgmap v858: 129 pgs: 129 active+clean; 322 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 608 op/s 2024-03-21T11:38:37.705 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:38:37.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:38:37.706 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:38:37.926 INFO:tasks.workunit.client.0.smithi047.stdout: 10 39744 28.62 MB/sec warmup 81 sec latency 53.957 ms 2024-03-21T11:38:37.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.938+0000 7f6265c00640 1 -- 172.21.15.47:0/2750228387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62601037a0 msgr2=0x7f6260103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:37.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.938+0000 7f6265c00640 1 --2- 172.21.15.47:0/2750228387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62601037a0 0x7f6260103bc0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6250009900 tx=0x7f625002bc50 comp rx=0 tx=0).stop 2024-03-21T11:38:37.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.938+0000 7f6265c00640 1 -- 172.21.15.47:0/2750228387 shutdown_connections 2024-03-21T11:38:37.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.938+0000 7f6265c00640 1 --2- 172.21.15.47:0/2750228387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6260105350 0x7f626010bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.938+0000 7f6265c00640 1 --2- 172.21.15.47:0/2750228387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6260104990 0x7f6260104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.938+0000 7f6265c00640 1 --2- 172.21.15.47:0/2750228387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62601037a0 0x7f6260103bc0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.938+0000 7f6265c00640 1 -- 172.21.15.47:0/2750228387 >> 172.21.15.47:0/2750228387 conn(0x7f62600fed10 msgr2=0x7f6260101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.938+0000 7f6265c00640 1 -- 172.21.15.47:0/2750228387 shutdown_connections 2024-03-21T11:38:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.938+0000 7f6265c00640 1 -- 172.21.15.47:0/2750228387 wait complete. 2024-03-21T11:38:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.940+0000 7f6265c00640 1 Processor -- start 2024-03-21T11:38:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.940+0000 7f6265c00640 1 -- start start 2024-03-21T11:38:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.940+0000 7f6265c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62601037a0 0x7f6260199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.940+0000 7f6265c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6260104990 0x7f626019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.940+0000 7f6265c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6260105350 0x7f62601a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.940+0000 7f6265c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626010f490 con 0x7f62601037a0 2024-03-21T11:38:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.940+0000 7f6265c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626010f310 con 0x7f6260105350 2024-03-21T11:38:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.940+0000 7f6265c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626010f610 con 0x7f6260104990 2024-03-21T11:38:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.940+0000 7f6257e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6260104990 0x7f626019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.940+0000 7f6257e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6260104990 0x7f626019a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60544/0 (socket says 172.21.15.47:60544) 2024-03-21T11:38:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.940+0000 7f6257e00640 1 -- 172.21.15.47:0/1432036245 learned_addr learned my addr 172.21.15.47:0/1432036245 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.941+0000 7f625fe00640 1 --2- 172.21.15.47:0/1432036245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62601037a0 0x7f6260199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.941+0000 7f6265200640 1 --2- 172.21.15.47:0/1432036245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6260105350 0x7f62601a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.941+0000 7f6257e00640 1 -- 172.21.15.47:0/1432036245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6260105350 msgr2=0x7f62601a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.941+0000 7f6257e00640 1 --2- 172.21.15.47:0/1432036245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6260105350 0x7f62601a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.941+0000 7f6257e00640 1 -- 172.21.15.47:0/1432036245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62601037a0 msgr2=0x7f6260199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.941+0000 7f6257e00640 1 --2- 172.21.15.47:0/1432036245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62601037a0 0x7f6260199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.941+0000 7f6257e00640 1 -- 172.21.15.47:0/1432036245 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6260075b60 con 0x7f6260104990 2024-03-21T11:38:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.942+0000 7f6257e00640 1 --2- 172.21.15.47:0/1432036245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6260104990 0x7f626019a110 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f624800e9f0 tx=0x7f624800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.942+0000 7f625d600640 1 -- 172.21.15.47:0/1432036245 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624800cd80 con 0x7f6260104990 2024-03-21T11:38:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.942+0000 7f6265c00640 1 -- 172.21.15.47:0/1432036245 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6260075e50 con 0x7f6260104990 2024-03-21T11:38:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.942+0000 7f6265c00640 1 -- 172.21.15.47:0/1432036245 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62600763a0 con 0x7f6260104990 2024-03-21T11:38:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.942+0000 7f625d600640 1 -- 172.21.15.47:0/1432036245 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6248005500 con 0x7f6260104990 2024-03-21T11:38:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.942+0000 7f625d600640 1 -- 172.21.15.47:0/1432036245 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248010690 con 0x7f6260104990 2024-03-21T11:38:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.946+0000 7f6269e0f740 1 Processor -- start 2024-03-21T11:38:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.946+0000 7f6269e0f740 1 -- start start 2024-03-21T11:38:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.947+0000 7f625d600640 1 -- 172.21.15.47:0/1432036245 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6248010830 con 0x7f6260104990 2024-03-21T11:38:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.947+0000 7f6269e0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfca2b0 0x561d8bfca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.947+0000 7f6269e0f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfcf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.948+0000 7f6269e0f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfd22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.948+0000 7f6269e0f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d8bf2feb0 con 0x561d8bfcaca0 2024-03-21T11:38:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.948+0000 7f6269e0f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d8bed8a60 con 0x561d8bfcfe10 2024-03-21T11:38:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.948+0000 7f6269e0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d8bf2bf50 con 0x561d8bfca2b0 2024-03-21T11:38:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.948+0000 7f6265200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfca2b0 0x561d8bfca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.948+0000 7f6257e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfd22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.948+0000 7f6257e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfd22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53224/0 (socket says 172.21.15.47:53224) 2024-03-21T11:38:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.948+0000 7f6257e00640 1 -- 172.21.15.47:0/1306015771 learned_addr learned my addr 172.21.15.47:0/1306015771 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.948+0000 7f625fe00640 1 --2- 172.21.15.47:0/1306015771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfcf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.949+0000 7f625fe00640 1 -- 172.21.15.47:0/1306015771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfca2b0 msgr2=0x561d8bfca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.949+0000 7f625fe00640 1 --2- 172.21.15.47:0/1306015771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfca2b0 0x561d8bfca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.949+0000 7f625fe00640 1 -- 172.21.15.47:0/1306015771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 msgr2=0x561d8bfd22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.949+0000 7f625fe00640 1 --2- 172.21.15.47:0/1306015771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfd22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.949+0000 7f625fe00640 1 -- 172.21.15.47:0/1306015771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d8bd0c7e0 con 0x561d8bfcaca0 2024-03-21T11:38:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.949+0000 7f6257e00640 1 --2- 172.21.15.47:0/1306015771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfd22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.949+0000 7f6265200640 1 --2- 172.21.15.47:0/1306015771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfca2b0 0x561d8bfca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.950+0000 7f625fe00640 1 --2- 172.21.15.47:0/1306015771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfcf7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f625002c190 tx=0x7f6250009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07e02b55fd21922 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.950+0000 7f625d600640 1 --2- 172.21.15.47:0/1432036245 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f62440a2fa0 0x7f62440a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.950+0000 7f625d600640 1 -- 172.21.15.47:0/1432036245 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f62480cf9e0 con 0x7f6260104990 2024-03-21T11:38:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.950+0000 7f6255600640 1 -- 172.21.15.47:0/1306015771 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625002cba0 con 0x561d8bfcaca0 2024-03-21T11:38:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.950+0000 7f6255600640 1 -- 172.21.15.47:0/1306015771 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6250005000 con 0x561d8bfcaca0 2024-03-21T11:38:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.951+0000 7f6269e0f740 1 -- 172.21.15.47:0/1306015771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 msgr2=0x561d8bfcf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.951+0000 7f6269e0f740 1 --2- 172.21.15.47:0/1306015771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfcf7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f625002c190 tx=0x7f6250009610 comp rx=0 tx=0).stop 2024-03-21T11:38:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.952+0000 7f6269e0f740 1 -- 172.21.15.47:0/1306015771 shutdown_connections 2024-03-21T11:38:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.952+0000 7f6269e0f740 1 --2- 172.21.15.47:0/1306015771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfd22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.952+0000 7f6269e0f740 1 --2- 172.21.15.47:0/1306015771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfcf7a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.952+0000 7f6269e0f740 1 --2- 172.21.15.47:0/1306015771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfca2b0 0x561d8bfca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.952+0000 7f6269e0f740 1 -- 172.21.15.47:0/1306015771 >> 172.21.15.47:0/1306015771 conn(0x561d8bfc5820 msgr2=0x561d8bfc5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.952+0000 7f6265200640 1 --2- 172.21.15.47:0/1432036245 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f62440a2fa0 0x7f62440a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.953+0000 7f6269e0f740 1 -- 172.21.15.47:0/1306015771 shutdown_connections 2024-03-21T11:38:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.953+0000 7f6269e0f740 1 -- 172.21.15.47:0/1306015771 wait complete. 2024-03-21T11:38:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.953+0000 7f6269e0f740 1 -- start start 2024-03-21T11:38:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.954+0000 7f6269e0f740 1 Processor -- start 2024-03-21T11:38:37.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.955+0000 7f6265200640 1 --2- 172.21.15.47:0/1432036245 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f62440a2fa0 0x7f62440a5460 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x561d8bfcb500 tx=0x7f6258009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.956+0000 7f6269e0f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfca2b0 0x561d8bfd4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.956+0000 7f6269e0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfcaca0 0x561d8bfd4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.956+0000 7f6269e0f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfdb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.956+0000 7f6269e0f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d8bfd2db0 con 0x561d8bfca2b0 2024-03-21T11:38:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.956+0000 7f6269e0f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d8beebeb0 con 0x561d8bfcfe10 2024-03-21T11:38:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.956+0000 7f6269e0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d8bfd2f30 con 0x561d8bfcaca0 2024-03-21T11:38:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.956+0000 7f6269e0f740 10 client.0 _ll_register_callbacks cb 0x561d8bee0150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.957+0000 7f6265200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfdb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.957+0000 7f6265200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfdb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53234/0 (socket says 172.21.15.47:53234) 2024-03-21T11:38:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.957+0000 7f6265200640 1 -- 172.21.15.47:0/2515634866 learned_addr learned my addr 172.21.15.47:0/2515634866 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.957+0000 7f625fe00640 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfca2b0 0x561d8bfd4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.957+0000 7f6257e00640 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfcaca0 0x561d8bfd4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.957+0000 7f6265200640 1 -- 172.21.15.47:0/2515634866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfcaca0 msgr2=0x561d8bfd4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.958+0000 7f6265200640 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfcaca0 0x561d8bfd4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.958+0000 7f6265200640 1 -- 172.21.15.47:0/2515634866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfca2b0 msgr2=0x561d8bfd4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.958+0000 7f6265200640 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfca2b0 0x561d8bfd4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.958+0000 7f6265200640 1 -- 172.21.15.47:0/2515634866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d8bd6cf70 con 0x561d8bfcfe10 2024-03-21T11:38:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.958+0000 7f625fe00640 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfca2b0 0x561d8bfd4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.958+0000 7f6257e00640 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfcaca0 0x561d8bfd4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:38:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.958+0000 7f6265200640 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfdb9c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6258009b00 tx=0x7f6258004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8eb82b177f766ae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.958+0000 7f624e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.959+0000 7f624e000640 1 -- 172.21.15.47:0/2515634866 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62580168a0 con 0x561d8bfcfe10 2024-03-21T11:38:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.959+0000 7f6269e0f740 1 -- 172.21.15.47:0/2515634866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6250005000 con 0x561d8bfcfe10 2024-03-21T11:38:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.959+0000 7f624e000640 1 -- 172.21.15.47:0/2515634866 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6258016a40 con 0x561d8bfcfe10 2024-03-21T11:38:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.959+0000 7f624e000640 1 -- 172.21.15.47:0/2515634866 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6258016c40 con 0x561d8bfcfe10 2024-03-21T11:38:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.959+0000 7f6269e0f740 10 client.30602 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.959+0000 7f6269e0f740 1 -- 172.21.15.47:0/2515634866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561d8bc04170 con 0x561d8bfcfe10 2024-03-21T11:38:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.960+0000 7f624e000640 1 -- 172.21.15.47:0/2515634866 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6250005000 con 0x561d8bfcfe10 2024-03-21T11:38:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.961+0000 7f624e000640 1 -- 172.21.15.47:0/2515634866 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f6258017410 con 0x561d8bfcfe10 2024-03-21T11:38:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.961+0000 7f6269e0f740 10 client.30602 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.961+0000 7f6269e0f740 10 client.30602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.961+0000 7f6269e0f740 20 client.30602 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.961+0000 7f6269e0f740 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d8bfe0290 0x561d8bfe2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.961+0000 7f6269e0f740 4 client.30602 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.961+0000 7f6269e0f740 1 -- 172.21.15.47:0/2515634866 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d8bf3d3f0 con 0x561d8bfe0290 2024-03-21T11:38:37.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.961+0000 7f625fe00640 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d8bfe0290 0x561d8bfe2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.962+0000 7f625fe00640 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d8bfe0290 0x561d8bfe2710 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=603dd3577506471c server_cookie=c564329b54df4cc4 in_seq=0 out_seq=0 2024-03-21T11:38:37.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.962+0000 7f624e000640 10 client.30602 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:37.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.967+0000 7f624e000640 1 -- 172.21.15.47:0/2515634866 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561d8bf3d3f0 con 0x561d8bfe0290 2024-03-21T11:38:37.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.967+0000 7f624e000640 10 client.30602 handle_command_reply: tid=0 2024-03-21T11:38:37.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.967+0000 7f6269e0f740 1 client.30602 shutdown 2024-03-21T11:38:37.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.968+0000 7f6269e0f740 1 -- 172.21.15.47:0/2515634866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 msgr2=0x561d8bfdb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:37.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.968+0000 7f6269e0f740 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfdb9c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6258009b00 tx=0x7f6258004b50 comp rx=0 tx=0).stop 2024-03-21T11:38:37.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.968+0000 7f6269e0f740 1 -- 172.21.15.47:0/2515634866 shutdown_connections 2024-03-21T11:38:37.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.968+0000 7f6269e0f740 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d8bfe0290 0x561d8bfe2710 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.968+0000 7f6269e0f740 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfdb9c0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.968+0000 7f624e000640 0 client.30602 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.968+0000 7f6269e0f740 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfcaca0 0x561d8bfd4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.968+0000 7f6269e0f740 1 --2- 172.21.15.47:0/2515634866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfca2b0 0x561d8bfd4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.968+0000 7f6269e0f740 1 -- 172.21.15.47:0/2515634866 >> 172.21.15.47:0/2515634866 conn(0x561d8bfc5820 msgr2=0x561d8bfc4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.968+0000 7f6269e0f740 1 -- 172.21.15.47:0/2515634866 shutdown_connections 2024-03-21T11:38:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.968+0000 7f6269e0f740 1 -- 172.21.15.47:0/2515634866 wait complete. 2024-03-21T11:38:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.968+0000 7f6269e0f740 20 client.30602 trim_cache size 0 max 16384 2024-03-21T11:38:37.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.994+0000 7f6269e0f740 1 Processor -- start 2024-03-21T11:38:37.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.994+0000 7f6269e0f740 1 -- start start 2024-03-21T11:38:37.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.995+0000 7f6269e0f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfca2b0 0x561d8bfc9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.995+0000 7f6269e0f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfdb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.995+0000 7f6269e0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfcfe10 0x561d8bfcb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.995+0000 7f6269e0f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c00f9e0 con 0x561d8bfcaca0 2024-03-21T11:38:37.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.995+0000 7f6269e0f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c0114e0 con 0x561d8bfca2b0 2024-03-21T11:38:37.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.995+0000 7f6269e0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c010040 con 0x561d8bfcfe10 2024-03-21T11:38:37.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.995+0000 7f6257e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfdb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:37.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.995+0000 7f625fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfca2b0 0x561d8bfc9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:37.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.995+0000 7f6265200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfcfe10 0x561d8bfcb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:37.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.995+0000 7f6265200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfcfe10 0x561d8bfcb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60552/0 (socket says 172.21.15.47:60552) 2024-03-21T11:38:37.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.995+0000 7f6265200640 1 -- 172.21.15.47:0/4065049144 learned_addr learned my addr 172.21.15.47:0/4065049144 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:37.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.996+0000 7f6265200640 1 -- 172.21.15.47:0/4065049144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfca2b0 msgr2=0x561d8bfc9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:37.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.996+0000 7f6265200640 1 --2- 172.21.15.47:0/4065049144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfca2b0 0x561d8bfc9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.996+0000 7f6265200640 1 -- 172.21.15.47:0/4065049144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 msgr2=0x561d8bfdb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:37.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.996+0000 7f6265200640 1 --2- 172.21.15.47:0/4065049144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfdb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.996+0000 7f6265200640 1 -- 172.21.15.47:0/4065049144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d8bfc6590 con 0x561d8bfcfe10 2024-03-21T11:38:37.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.996+0000 7f6265200640 1 --2- 172.21.15.47:0/4065049144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfcfe10 0x561d8bfcb740 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6258017820 tx=0x7f6258019ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=980b187b20beadb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:37.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.996+0000 7f6254c00640 1 -- 172.21.15.47:0/4065049144 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258036070 con 0x561d8bfcfe10 2024-03-21T11:38:37.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.997+0000 7f6254c00640 1 -- 172.21.15.47:0/4065049144 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6258016de0 con 0x561d8bfcfe10 2024-03-21T11:38:37.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.997+0000 7f6254c00640 1 -- 172.21.15.47:0/4065049144 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258031440 con 0x561d8bfcfe10 2024-03-21T11:38:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.997+0000 7f6269e0f740 1 -- 172.21.15.47:0/4065049144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfcfe10 msgr2=0x561d8bfcb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.997+0000 7f6269e0f740 1 --2- 172.21.15.47:0/4065049144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfcfe10 0x561d8bfcb740 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6258017820 tx=0x7f6258019ab0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.997+0000 7f6269e0f740 1 -- 172.21.15.47:0/4065049144 shutdown_connections 2024-03-21T11:38:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.997+0000 7f6269e0f740 1 --2- 172.21.15.47:0/4065049144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfcfe10 0x561d8bfcb740 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.997+0000 7f6269e0f740 1 --2- 172.21.15.47:0/4065049144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfdb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.997+0000 7f6269e0f740 1 --2- 172.21.15.47:0/4065049144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfca2b0 0x561d8bfc9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.997+0000 7f6269e0f740 1 -- 172.21.15.47:0/4065049144 >> 172.21.15.47:0/4065049144 conn(0x561d8bfc5820 msgr2=0x561d8bfc8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.998+0000 7f6269e0f740 1 -- 172.21.15.47:0/4065049144 shutdown_connections 2024-03-21T11:38:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.998+0000 7f6269e0f740 1 -- 172.21.15.47:0/4065049144 wait complete. 2024-03-21T11:38:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.998+0000 7f6269e0f740 1 -- start start 2024-03-21T11:38:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.998+0000 7f6269e0f740 1 Processor -- start 2024-03-21T11:38:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.999+0000 7f6269e0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfca2b0 0x561d8bfd9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.999+0000 7f6269e0f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfd9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.999+0000 7f6269e0f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfdad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.999+0000 7f6269e0f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c010dc0 con 0x561d8bfcaca0 2024-03-21T11:38:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.999+0000 7f6269e0f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c010760 con 0x561d8bfcfe10 2024-03-21T11:38:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.999+0000 7f6269e0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c00dba0 con 0x561d8bfca2b0 2024-03-21T11:38:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.999+0000 7f6269e0f740 10 client.0 _ll_register_callbacks cb 0x7f6258009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.999+0000 7f6257e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfd9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.999+0000 7f625fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfca2b0 0x561d8bfd9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.999+0000 7f6257e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfd9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46188/0 (socket says 172.21.15.47:46188) 2024-03-21T11:38:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.999+0000 7f6257e00640 1 -- 172.21.15.47:0/361513267 learned_addr learned my addr 172.21.15.47:0/361513267 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:37.999+0000 7f6265200640 1 --2- 172.21.15.47:0/361513267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfdad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.000+0000 7f6265200640 1 -- 172.21.15.47:0/361513267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfca2b0 msgr2=0x561d8bfd9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.000+0000 7f6265200640 1 --2- 172.21.15.47:0/361513267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfca2b0 0x561d8bfd9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.000+0000 7f6265200640 1 -- 172.21.15.47:0/361513267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 msgr2=0x561d8bfd9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.000+0000 7f6265200640 1 --2- 172.21.15.47:0/361513267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfd9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.000+0000 7f6265200640 1 -- 172.21.15.47:0/361513267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d8bfdb360 con 0x561d8bfcfe10 2024-03-21T11:38:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.000+0000 7f6257e00640 1 --2- 172.21.15.47:0/361513267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfd9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:38.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.001+0000 7f6265200640 1 --2- 172.21.15.47:0/361513267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfdad70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f625800ad70 tx=0x7f6258012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93cbdaf509fb0cf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:38.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.001+0000 7f624e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:38.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.001+0000 7f624e000640 1 -- 172.21.15.47:0/361513267 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258036070 con 0x561d8bfcfe10 2024-03-21T11:38:38.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.001+0000 7f6269e0f740 1 -- 172.21.15.47:0/361513267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6258016de0 con 0x561d8bfcfe10 2024-03-21T11:38:38.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.001+0000 7f624e000640 1 -- 172.21.15.47:0/361513267 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6258017c50 con 0x561d8bfcfe10 2024-03-21T11:38:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.001+0000 7f624e000640 1 -- 172.21.15.47:0/361513267 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6258007000 con 0x561d8bfcfe10 2024-03-21T11:38:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.001+0000 7f6269e0f740 10 client.30608 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.001+0000 7f6269e0f740 1 -- 172.21.15.47:0/361513267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561d8c03c3c0 con 0x561d8bfcfe10 2024-03-21T11:38:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.002+0000 7f624e000640 1 -- 172.21.15.47:0/361513267 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6258016de0 con 0x561d8bfcfe10 2024-03-21T11:38:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.002+0000 7f624e000640 1 -- 172.21.15.47:0/361513267 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f6258022400 con 0x561d8bfcfe10 2024-03-21T11:38:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.002+0000 7f6269e0f740 10 client.30608 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.002+0000 7f6269e0f740 10 client.30608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.002+0000 7f6269e0f740 20 client.30608 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.002+0000 7f6269e0f740 1 --2- 172.21.15.47:0/361513267 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d8c03cc90 0x561d8c03f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.002+0000 7f6269e0f740 4 client.30608 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:38:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.002+0000 7f6269e0f740 1 -- 172.21.15.47:0/361513267 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561d8bf37740 con 0x561d8c03cc90 2024-03-21T11:38:38.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.003+0000 7f625fe00640 1 --2- 172.21.15.47:0/361513267 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d8c03cc90 0x561d8c03f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.005+0000 7f625fe00640 1 --2- 172.21.15.47:0/361513267 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d8c03cc90 0x561d8c03f0b0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc90933bbd25b614 server_cookie=e978cf5fcf0d91b9 in_seq=0 out_seq=0 2024-03-21T11:38:38.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.005+0000 7f624e000640 10 client.30608 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:38.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.007+0000 7f624e000640 1 -- 172.21.15.47:0/361513267 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561d8bf37740 con 0x561d8c03cc90 2024-03-21T11:38:38.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.007+0000 7f624e000640 10 client.30608 handle_command_reply: tid=0 2024-03-21T11:38:38.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.007+0000 7f6269e0f740 1 client.30608 shutdown 2024-03-21T11:38:38.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.007+0000 7f6269e0f740 1 -- 172.21.15.47:0/361513267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 msgr2=0x561d8bfdad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.007+0000 7f6269e0f740 1 --2- 172.21.15.47:0/361513267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfdad70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f625800ad70 tx=0x7f6258012000 comp rx=0 tx=0).stop 2024-03-21T11:38:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.008+0000 7f6269e0f740 1 -- 172.21.15.47:0/361513267 shutdown_connections 2024-03-21T11:38:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.008+0000 7f6269e0f740 1 --2- 172.21.15.47:0/361513267 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d8c03cc90 0x561d8c03f0b0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.008+0000 7f6269e0f740 1 --2- 172.21.15.47:0/361513267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d8bfcfe10 0x561d8bfdad70 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.008+0000 7f624e000640 0 client.30608 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.008+0000 7f6269e0f740 1 --2- 172.21.15.47:0/361513267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d8bfcaca0 0x561d8bfd9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.008+0000 7f6269e0f740 1 --2- 172.21.15.47:0/361513267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d8bfca2b0 0x561d8bfd9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.008+0000 7f6269e0f740 1 -- 172.21.15.47:0/361513267 >> 172.21.15.47:0/361513267 conn(0x561d8bfc5820 msgr2=0x561d8bfc9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:38.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.009+0000 7f6269e0f740 1 -- 172.21.15.47:0/361513267 shutdown_connections 2024-03-21T11:38:38.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.009+0000 7f6269e0f740 1 -- 172.21.15.47:0/361513267 wait complete. 2024-03-21T11:38:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.010+0000 7f4e3a600640 1 -- 172.21.15.47:0/183723217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e34077c40 msgr2=0x7f4e340784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.010+0000 7f4e3a600640 1 --2- 172.21.15.47:0/183723217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e34077c40 0x7f4e340784e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009ca0 tx=0x7f4e2c02c100 comp rx=0 tx=0).stop 2024-03-21T11:38:38.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.010+0000 7f6269e0f740 20 client.30608 trim_cache size 0 max 16384 2024-03-21T11:38:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.010+0000 7f4e3a600640 1 -- 172.21.15.47:0/183723217 shutdown_connections 2024-03-21T11:38:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.010+0000 7f4e3a600640 1 --2- 172.21.15.47:0/183723217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e34077c40 0x7f4e340784e0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.010+0000 7f4e3a600640 1 --2- 172.21.15.47:0/183723217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e340772a0 0x7f4e34077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.010+0000 7f4e3a600640 1 --2- 172.21.15.47:0/183723217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e34076060 0x7f4e34076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.010+0000 7f4e3a600640 1 -- 172.21.15.47:0/183723217 >> 172.21.15.47:0/183723217 conn(0x7f4e340fee90 msgr2=0x7f4e341012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:38.011 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:38:38.011 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:38:38.012 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "141c2112-b51d-4e48-96a2-845e628b4583", 2024-03-21T11:38:38.012 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:38:38.012 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:38:38.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.012+0000 7f6265c00640 1 -- 172.21.15.47:0/1432036245 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f62440a2fa0 msgr2=0x7f62440a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.012+0000 7f6265c00640 1 --2- 172.21.15.47:0/1432036245 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f62440a2fa0 0x7f62440a5460 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x561d8bfcb500 tx=0x7f6258009650 comp rx=0 tx=0).stop 2024-03-21T11:38:38.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.012+0000 7f6265c00640 1 -- 172.21.15.47:0/1432036245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6260104990 msgr2=0x7f626019a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.012+0000 7f6265c00640 1 --2- 172.21.15.47:0/1432036245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6260104990 0x7f626019a110 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f624800e9f0 tx=0x7f624800edf0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.012+0000 7f4e3a600640 1 -- 172.21.15.47:0/183723217 shutdown_connections 2024-03-21T11:38:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.012+0000 7f4e3a600640 1 -- 172.21.15.47:0/183723217 wait complete. 2024-03-21T11:38:38.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.012+0000 7f6265c00640 1 -- 172.21.15.47:0/1432036245 shutdown_connections 2024-03-21T11:38:38.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.012+0000 7f6265c00640 1 --2- 172.21.15.47:0/1432036245 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f62440a2fa0 0x7f62440a5460 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.012+0000 7f6265c00640 1 --2- 172.21.15.47:0/1432036245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6260105350 0x7f62601a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.012+0000 7f6265c00640 1 --2- 172.21.15.47:0/1432036245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6260104990 0x7f626019a110 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.012+0000 7f6265c00640 1 --2- 172.21.15.47:0/1432036245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62601037a0 0x7f6260199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.012+0000 7f6265c00640 1 -- 172.21.15.47:0/1432036245 >> 172.21.15.47:0/1432036245 conn(0x7f62600fed10 msgr2=0x7f6260107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:38.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.013+0000 7f6265c00640 1 -- 172.21.15.47:0/1432036245 shutdown_connections 2024-03-21T11:38:38.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.013+0000 7f6265c00640 1 -- 172.21.15.47:0/1432036245 wait complete. 2024-03-21T11:38:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.013+0000 7f4e3a600640 1 Processor -- start 2024-03-21T11:38:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.014+0000 7f4e3a600640 1 -- start start 2024-03-21T11:38:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.014+0000 7f4e3a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e34076060 0x7f4e3419de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.014+0000 7f4e3a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e340772a0 0x7f4e3419e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.014+0000 7f4e3a600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e34077c40 0x7f4e341a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.014+0000 7f4e3a600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e34113990 con 0x7f4e34077c40 2024-03-21T11:38:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.014+0000 7f4e3a600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e34113810 con 0x7f4e34076060 2024-03-21T11:38:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.014+0000 7f4e3a600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e34113b10 con 0x7f4e340772a0 2024-03-21T11:38:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.014+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e340772a0 0x7f4e3419e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.014+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e340772a0 0x7f4e3419e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60580/0 (socket says 172.21.15.47:60580) 2024-03-21T11:38:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.014+0000 7f4e2aa00640 1 -- 172.21.15.47:0/2142154667 learned_addr learned my addr 172.21.15.47:0/2142154667 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.014+0000 7f4e33400640 1 --2- 172.21.15.47:0/2142154667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e34076060 0x7f4e3419de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.015+0000 7f4e33e00640 1 --2- 172.21.15.47:0/2142154667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e34077c40 0x7f4e341a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.015+0000 7f4e2aa00640 1 -- 172.21.15.47:0/2142154667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e34076060 msgr2=0x7f4e3419de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.015+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/2142154667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e34076060 0x7f4e3419de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.015+0000 7f4e2aa00640 1 -- 172.21.15.47:0/2142154667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e34077c40 msgr2=0x7f4e341a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.015+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/2142154667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e34077c40 0x7f4e341a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.015+0000 7f4e2aa00640 1 -- 172.21.15.47:0/2142154667 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e341a5a70 con 0x7f4e340772a0 2024-03-21T11:38:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.016+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/2142154667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e340772a0 0x7f4e3419e390 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00d950 tx=0x7f4e1c00dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.016+0000 7f4e30c00640 1 -- 172.21.15.47:0/2142154667 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c017070 con 0x7f4e340772a0 2024-03-21T11:38:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.017+0000 7f4e30c00640 1 -- 172.21.15.47:0/2142154667 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e1c00de80 con 0x7f4e340772a0 2024-03-21T11:38:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.017+0000 7f4e30c00640 1 -- 172.21.15.47:0/2142154667 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00f620 con 0x7f4e340772a0 2024-03-21T11:38:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.017+0000 7f4e3a600640 1 -- 172.21.15.47:0/2142154667 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e341a5d60 con 0x7f4e340772a0 2024-03-21T11:38:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.017+0000 7f4e3a600640 1 -- 172.21.15.47:0/2142154667 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e341a6200 con 0x7f4e340772a0 2024-03-21T11:38:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.022+0000 7f4e3d24f740 1 Processor -- start 2024-03-21T11:38:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.022+0000 7f4e3d24f740 1 -- start start 2024-03-21T11:38:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.023+0000 7f4e3d24f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c7641f0 0x55df2c764610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.023+0000 7f4e3d24f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c764be0 0x55df2c7696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.023+0000 7f4e3d24f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 0x55df2c76c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.023+0000 7f4e3d24f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2c7163b0 con 0x55df2c769d50 2024-03-21T11:38:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.023+0000 7f4e3d24f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2c6b8830 con 0x55df2c7641f0 2024-03-21T11:38:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.023+0000 7f4e3d24f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2c6615d0 con 0x55df2c764be0 2024-03-21T11:38:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.023+0000 7f4e33400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c764be0 0x55df2c7696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.023+0000 7f4e33400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c764be0 0x55df2c7696e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60594/0 (socket says 172.21.15.47:60594) 2024-03-21T11:38:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.023+0000 7f4e33400640 1 -- 172.21.15.47:0/2489958315 learned_addr learned my addr 172.21.15.47:0/2489958315 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.023+0000 7f4e33e00640 1 --2- 172.21.15.47:0/2489958315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c7641f0 0x55df2c764610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.024+0000 7f4e33e00640 1 -- 172.21.15.47:0/2489958315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c764be0 msgr2=0x55df2c7696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.024+0000 7f4e33e00640 1 --2- 172.21.15.47:0/2489958315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c764be0 0x55df2c7696e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.024+0000 7f4e33e00640 1 -- 172.21.15.47:0/2489958315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 msgr2=0x55df2c76c1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:38:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.024+0000 7f4e33e00640 1 --2- 172.21.15.47:0/2489958315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 0x55df2c76c1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.024+0000 7f4e33e00640 1 -- 172.21.15.47:0/2489958315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df2c6792c0 con 0x55df2c7641f0 2024-03-21T11:38:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.024+0000 7f4e33e00640 1 --2- 172.21.15.47:0/2489958315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c7641f0 0x55df2c764610 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c037040 tx=0x7f4e2c0098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad3a2c08d75ea50 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.025+0000 7f4e30c00640 1 -- 172.21.15.47:0/2142154667 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4e1c010460 con 0x7f4e340772a0 2024-03-21T11:38:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.025+0000 7f4e28c00640 1 -- 172.21.15.47:0/2489958315 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c043020 con 0x55df2c7641f0 2024-03-21T11:38:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.025+0000 7f4e28c00640 1 -- 172.21.15.47:0/2489958315 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e2c003970 con 0x55df2c7641f0 2024-03-21T11:38:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.025+0000 7f4e28c00640 1 -- 172.21.15.47:0/2489958315 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c039040 con 0x55df2c7641f0 2024-03-21T11:38:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.026+0000 7f4e30c00640 1 --2- 172.21.15.47:0/2142154667 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e180a3090 0x7f4e180a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.026+0000 7f4e3d24f740 1 -- 172.21.15.47:0/2489958315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c7641f0 msgr2=0x55df2c764610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.026+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/2489958315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c7641f0 0x55df2c764610 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c037040 tx=0x7f4e2c0098d0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.026+0000 7f4e30c00640 1 -- 172.21.15.47:0/2142154667 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4e1c0cf390 con 0x7f4e340772a0 2024-03-21T11:38:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.026+0000 7f4e3d24f740 1 -- 172.21.15.47:0/2489958315 shutdown_connections 2024-03-21T11:38:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.026+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/2489958315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 0x55df2c76c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.026+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/2489958315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c764be0 0x55df2c7696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.026+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/2489958315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c7641f0 0x55df2c764610 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.026+0000 7f4e3d24f740 1 -- 172.21.15.47:0/2489958315 >> 172.21.15.47:0/2489958315 conn(0x55df2c74e750 msgr2=0x55df2c74eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.027+0000 7f4e33400640 1 --2- 172.21.15.47:0/2142154667 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e180a3090 0x7f4e180a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.027+0000 7f4e3d24f740 1 -- 172.21.15.47:0/2489958315 shutdown_connections 2024-03-21T11:38:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.027+0000 7f4e3d24f740 1 -- 172.21.15.47:0/2489958315 wait complete. 2024-03-21T11:38:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.027+0000 7f4e3d24f740 1 -- start start 2024-03-21T11:38:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.028+0000 7f4e33400640 1 --2- 172.21.15.47:0/2142154667 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e180a3090 0x7f4e180a5550 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55df2c74f6c0 tx=0x7f4e2400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.028+0000 7f4e3d24f740 1 Processor -- start 2024-03-21T11:38:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.029+0000 7f4e3d24f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c7641f0 0x55df2c76e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.029+0000 7f4e3d24f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c764be0 0x55df2c76e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.029+0000 7f4e3d24f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 0x55df2c7759e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.029+0000 7f4e3d24f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2c674a20 con 0x55df2c769d50 2024-03-21T11:38:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.029+0000 7f4e3d24f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2c6b48d0 con 0x55df2c764be0 2024-03-21T11:38:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.029+0000 7f4e3d24f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2c76cca0 con 0x55df2c7641f0 2024-03-21T11:38:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.029+0000 7f4e3d24f740 10 client.0 _ll_register_callbacks cb 0x55df2c6b11c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.029+0000 7f4e33e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 0x55df2c7759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.029+0000 7f4e33e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 0x55df2c7759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46230/0 (socket says 172.21.15.47:46230) 2024-03-21T11:38:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.029+0000 7f4e33e00640 1 -- 172.21.15.47:0/334380645 learned_addr learned my addr 172.21.15.47:0/334380645 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.029+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/334380645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c764be0 0x55df2c76e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.029+0000 7f4e33400640 1 --2- 172.21.15.47:0/334380645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c7641f0 0x55df2c76e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.030+0000 7f4e33e00640 1 -- 172.21.15.47:0/334380645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c7641f0 msgr2=0x55df2c76e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.030+0000 7f4e33e00640 1 --2- 172.21.15.47:0/334380645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c7641f0 0x55df2c76e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.030+0000 7f4e33e00640 1 -- 172.21.15.47:0/334380645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c764be0 msgr2=0x55df2c76e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.030+0000 7f4e33e00640 1 --2- 172.21.15.47:0/334380645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c764be0 0x55df2c76e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.030+0000 7f4e33e00640 1 -- 172.21.15.47:0/334380645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df2c3cd210 con 0x55df2c769d50 2024-03-21T11:38:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.030+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/334380645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c764be0 0x55df2c76e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.030+0000 7f4e33e00640 1 --2- 172.21.15.47:0/334380645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 0x55df2c7759e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009ca0 tx=0x7f4e2c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0553c23766bfcc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.030+0000 7f4e21600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:38:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.033+0000 7f4e21600640 1 -- 172.21.15.47:0/334380645 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c04d060 con 0x55df2c769d50 2024-03-21T11:38:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.033+0000 7f4e21600640 1 -- 172.21.15.47:0/334380645 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4e2c004280 con 0x55df2c769d50 2024-03-21T11:38:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.033+0000 7f4e3d24f740 1 -- 172.21.15.47:0/334380645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e2c003970 con 0x55df2c769d50 2024-03-21T11:38:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.034+0000 7f4e21600640 1 -- 172.21.15.47:0/334380645 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e2c043070 con 0x55df2c769d50 2024-03-21T11:38:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.034+0000 7f4e3d24f740 10 client.20157 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.034+0000 7f4e3d24f740 1 -- 172.21.15.47:0/334380645 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55df2c499620 con 0x55df2c769d50 2024-03-21T11:38:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.035+0000 7f4e21600640 1 -- 172.21.15.47:0/334380645 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e2c003970 con 0x55df2c769d50 2024-03-21T11:38:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.035+0000 7f4e21600640 1 -- 172.21.15.47:0/334380645 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f4e2c03fc10 con 0x55df2c769d50 2024-03-21T11:38:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.036+0000 7f4e3d24f740 10 client.20157 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.036+0000 7f4e3d24f740 10 client.20157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.036+0000 7f4e3d24f740 20 client.20157 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.036+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/334380645 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df2c779ed0 0x55df2c77c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.036+0000 7f4e3d24f740 4 client.20157 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.036+0000 7f4e3d24f740 1 -- 172.21.15.47:0/334380645 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df2c6c5d70 con 0x55df2c779ed0 2024-03-21T11:38:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.037+0000 7f4e33400640 1 --2- 172.21.15.47:0/334380645 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df2c779ed0 0x55df2c77c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.038+0000 7f4e33400640 1 --2- 172.21.15.47:0/334380645 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df2c779ed0 0x55df2c77c2c0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7ce3e71aba17bd server_cookie=ee5261688bac684d in_seq=0 out_seq=0 2024-03-21T11:38:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.038+0000 7f4e21600640 10 client.20157 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:38.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:38:38.044 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:38:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.047+0000 7f4e21600640 1 -- 172.21.15.47:0/334380645 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55df2c6c5d70 con 0x55df2c779ed0 2024-03-21T11:38:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.047+0000 7f4e21600640 10 client.20157 handle_command_reply: tid=0 2024-03-21T11:38:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.047+0000 7f4e3d24f740 1 client.20157 shutdown 2024-03-21T11:38:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.047+0000 7f4e3d24f740 1 -- 172.21.15.47:0/334380645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 msgr2=0x55df2c7759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.047+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/334380645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 0x55df2c7759e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009ca0 tx=0x7f4e2c009650 comp rx=0 tx=0).stop 2024-03-21T11:38:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.048+0000 7f4e3d24f740 1 -- 172.21.15.47:0/334380645 shutdown_connections 2024-03-21T11:38:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.048+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/334380645 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df2c779ed0 0x55df2c77c2c0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.048+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/334380645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 0x55df2c7759e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.048+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/334380645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c764be0 0x55df2c76e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.048+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/334380645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c7641f0 0x55df2c76e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.048+0000 7f4e3d24f740 1 -- 172.21.15.47:0/334380645 >> 172.21.15.47:0/334380645 conn(0x55df2c74e750 msgr2=0x55df2c74d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.048+0000 7f4e21600640 0 client.20157 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.048+0000 7f4e3d24f740 1 -- 172.21.15.47:0/334380645 shutdown_connections 2024-03-21T11:38:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.049+0000 7f4e3d24f740 1 -- 172.21.15.47:0/334380645 wait complete. 2024-03-21T11:38:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.049+0000 7f4e3d24f740 20 client.20157 trim_cache size 0 max 16384 2024-03-21T11:38:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.079+0000 7f4e3d24f740 1 Processor -- start 2024-03-21T11:38:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.079+0000 7f4e3d24f740 1 -- start start 2024-03-21T11:38:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.080+0000 7f4e3d24f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c7641f0 0x55df2c7520c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.080+0000 7f4e3d24f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c764be0 0x55df2c775900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.080+0000 7f4e3d24f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c769d50 0x55df2c780b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.080+0000 7f4e3d24f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e341a7ea0 con 0x55df2c7641f0 2024-03-21T11:38:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.080+0000 7f4e3d24f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e341a99a0 con 0x55df2c769d50 2024-03-21T11:38:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.080+0000 7f4e3d24f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e341a8500 con 0x55df2c764be0 2024-03-21T11:38:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.080+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c764be0 0x55df2c775900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.080+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c764be0 0x55df2c775900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60616/0 (socket says 172.21.15.47:60616) 2024-03-21T11:38:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.080+0000 7f4e33e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c769d50 0x55df2c780b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.080+0000 7f4e2aa00640 1 -- 172.21.15.47:0/1051831882 learned_addr learned my addr 172.21.15.47:0/1051831882 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.081+0000 7f4e33400640 1 --2- 172.21.15.47:0/1051831882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c7641f0 0x55df2c7520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.081+0000 7f4e2aa00640 1 -- 172.21.15.47:0/1051831882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c769d50 msgr2=0x55df2c780b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.081+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/1051831882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c769d50 0x55df2c780b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.081+0000 7f4e2aa00640 1 -- 172.21.15.47:0/1051831882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c7641f0 msgr2=0x55df2c7520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.081+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/1051831882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c7641f0 0x55df2c7520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.081+0000 7f4e2aa00640 1 -- 172.21.15.47:0/1051831882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df2c765440 con 0x55df2c764be0 2024-03-21T11:38:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.082+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/1051831882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c764be0 0x55df2c775900 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009610 tx=0x7f4e1c097390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1db5d553a2ba0669 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.082+0000 7f4e23e00640 1 -- 172.21.15.47:0/1051831882 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0b4070 con 0x55df2c764be0 2024-03-21T11:38:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.082+0000 7f4e23e00640 1 -- 172.21.15.47:0/1051831882 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e1c0a4140 con 0x55df2c764be0 2024-03-21T11:38:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.082+0000 7f4e23e00640 1 -- 172.21.15.47:0/1051831882 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c020070 con 0x55df2c764be0 2024-03-21T11:38:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.083+0000 7f4e3d24f740 1 -- 172.21.15.47:0/1051831882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c764be0 msgr2=0x55df2c775900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.083+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/1051831882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c764be0 0x55df2c775900 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009610 tx=0x7f4e1c097390 comp rx=0 tx=0).stop 2024-03-21T11:38:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.083+0000 7f4e3d24f740 1 -- 172.21.15.47:0/1051831882 shutdown_connections 2024-03-21T11:38:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.083+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/1051831882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c769d50 0x55df2c780b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.084+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/1051831882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c764be0 0x55df2c775900 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.084+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/1051831882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c7641f0 0x55df2c7520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.084+0000 7f4e3d24f740 1 -- 172.21.15.47:0/1051831882 >> 172.21.15.47:0/1051831882 conn(0x55df2c74e750 msgr2=0x55df2c7519b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.084+0000 7f4e3d24f740 1 -- 172.21.15.47:0/1051831882 shutdown_connections 2024-03-21T11:38:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.084+0000 7f4e3d24f740 1 -- 172.21.15.47:0/1051831882 wait complete. 2024-03-21T11:38:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.085+0000 7f4e3d24f740 1 -- start start 2024-03-21T11:38:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.085+0000 7f4e3d24f740 1 Processor -- start 2024-03-21T11:38:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.086+0000 7f4e3d24f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c7641f0 0x55df2c773700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.086+0000 7f4e3d24f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c764be0 0x55df2c773c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.086+0000 7f4e3d24f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 0x55df2c774c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.086+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c764be0 0x55df2c773c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.086+0000 7f4e3d24f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e341a9280 con 0x55df2c769d50 2024-03-21T11:38:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.086+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c764be0 0x55df2c773c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53290/0 (socket says 172.21.15.47:53290) 2024-03-21T11:38:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.086+0000 7f4e2aa00640 1 -- 172.21.15.47:0/3253746235 learned_addr learned my addr 172.21.15.47:0/3253746235 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.087+0000 7f4e3d24f740 1 -- 172.21.15.47:0/3253746235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e341a8c20 con 0x55df2c764be0 2024-03-21T11:38:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.087+0000 7f4e33400640 1 --2- 172.21.15.47:0/3253746235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c7641f0 0x55df2c773700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.087+0000 7f4e3d24f740 1 -- 172.21.15.47:0/3253746235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e34073440 con 0x55df2c7641f0 2024-03-21T11:38:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.087+0000 7f4e3d24f740 10 client.0 _ll_register_callbacks cb 0x7f4e2400a730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.087+0000 7f4e33e00640 1 --2- 172.21.15.47:0/3253746235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 0x55df2c774c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.088+0000 7f4e2aa00640 1 -- 172.21.15.47:0/3253746235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c7641f0 msgr2=0x55df2c773700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.088+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/3253746235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c7641f0 0x55df2c773700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.088+0000 7f4e2aa00640 1 -- 172.21.15.47:0/3253746235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 msgr2=0x55df2c774c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.088+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/3253746235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 0x55df2c774c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.088+0000 7f4e2aa00640 1 -- 172.21.15.47:0/3253746235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df2c7751f0 con 0x55df2c764be0 2024-03-21T11:38:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.088+0000 7f4e33e00640 1 --2- 172.21.15.47:0/3253746235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 0x55df2c774c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.089+0000 7f4e2aa00640 1 --2- 172.21.15.47:0/3253746235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c764be0 0x55df2c773c40 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c086010 tx=0x7f4e1c0ccaa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d234fcf5e0c66938 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.089+0000 7f4e21600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.089+0000 7f4e21600640 1 -- 172.21.15.47:0/3253746235 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0b9060 con 0x55df2c764be0 2024-03-21T11:38:38.089 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.089+0000 7f4e21600640 1 -- 172.21.15.47:0/3253746235 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4e1c0860b0 con 0x55df2c764be0 2024-03-21T11:38:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.089+0000 7f4e3d24f740 1 -- 172.21.15.47:0/3253746235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e1c0a4140 con 0x55df2c764be0 2024-03-21T11:38:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.090+0000 7f4e21600640 1 -- 172.21.15.47:0/3253746235 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e1c020070 con 0x55df2c764be0 2024-03-21T11:38:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.090+0000 7f4e3d24f740 10 client.30623 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.090+0000 7f4e3d24f740 1 -- 172.21.15.47:0/3253746235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55df2c7d5c00 con 0x55df2c764be0 2024-03-21T11:38:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.090+0000 7f4e21600640 1 -- 172.21.15.47:0/3253746235 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e1c0a4140 con 0x55df2c764be0 2024-03-21T11:38:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.091+0000 7f4e21600640 1 -- 172.21.15.47:0/3253746235 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f4e1c09ad20 con 0x55df2c764be0 2024-03-21T11:38:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.091+0000 7f4e3d24f740 10 client.30623 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.091+0000 7f4e3d24f740 10 client.30623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.091+0000 7f4e3d24f740 20 client.30623 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.091+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/3253746235 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df2c7d6590 0x55df2c7d89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.091+0000 7f4e3d24f740 4 client.30623 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:38:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.091+0000 7f4e3d24f740 1 -- 172.21.15.47:0/3253746235 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df2c6c00c0 con 0x55df2c7d6590 2024-03-21T11:38:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.091+0000 7f4e33400640 1 --2- 172.21.15.47:0/3253746235 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df2c7d6590 0x55df2c7d89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.092+0000 7f4e21600640 10 client.30623 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.092+0000 7f4e33400640 1 --2- 172.21.15.47:0/3253746235 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df2c7d6590 0x55df2c7d89b0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4a285f801ae58d server_cookie=c68170ce1e2761a7 in_seq=0 out_seq=0 2024-03-21T11:38:38.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.320+0000 7f3289400640 1 -- 172.21.15.47:0/468553341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3284105490 msgr2=0x7f328410bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.320+0000 7f3289400640 1 --2- 172.21.15.47:0/468553341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3284105490 0x7f328410bd20 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f327c009ca0 tx=0x7f327c02c100 comp rx=0 tx=0).stop 2024-03-21T11:38:38.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.320+0000 7f3289400640 1 -- 172.21.15.47:0/468553341 shutdown_connections 2024-03-21T11:38:38.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.320+0000 7f3289400640 1 --2- 172.21.15.47:0/468553341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3284105490 0x7f328410bd20 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.320+0000 7f3289400640 1 --2- 172.21.15.47:0/468553341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3284104ad0 0x7f3284104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.320+0000 7f3289400640 1 --2- 172.21.15.47:0/468553341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32841038e0 0x7f3284103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.320+0000 7f3289400640 1 -- 172.21.15.47:0/468553341 >> 172.21.15.47:0/468553341 conn(0x7f32840fee50 msgr2=0x7f3284101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:38.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.320+0000 7f3289400640 1 -- 172.21.15.47:0/468553341 shutdown_connections 2024-03-21T11:38:38.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.320+0000 7f3289400640 1 -- 172.21.15.47:0/468553341 wait complete. 2024-03-21T11:38:38.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.322+0000 7f3289400640 1 Processor -- start 2024-03-21T11:38:38.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.322+0000 7f3289400640 1 -- start start 2024-03-21T11:38:38.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.322+0000 7f3289400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32841038e0 0x7f3284199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.322+0000 7f3289400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3284104ad0 0x7f328419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.322+0000 7f3289400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3284105490 0x7f32841a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.322+0000 7f3289400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328410f5c0 con 0x7f3284104ad0 2024-03-21T11:38:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.322+0000 7f3289400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328410f440 con 0x7f3284105490 2024-03-21T11:38:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.322+0000 7f3289400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328410f740 con 0x7f32841038e0 2024-03-21T11:38:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.323+0000 7f3282800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32841038e0 0x7f3284199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.323+0000 7f3282800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32841038e0 0x7f3284199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60650/0 (socket says 172.21.15.47:60650) 2024-03-21T11:38:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.323+0000 7f3282800640 1 -- 172.21.15.47:0/3561026839 learned_addr learned my addr 172.21.15.47:0/3561026839 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.323+0000 7f327be00640 1 --2- 172.21.15.47:0/3561026839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3284104ad0 0x7f328419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.323+0000 7f3283200640 1 --2- 172.21.15.47:0/3561026839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3284105490 0x7f32841a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.323+0000 7f3282800640 1 -- 172.21.15.47:0/3561026839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3284105490 msgr2=0x7f32841a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.323+0000 7f3282800640 1 --2- 172.21.15.47:0/3561026839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3284105490 0x7f32841a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.324+0000 7f3282800640 1 -- 172.21.15.47:0/3561026839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3284104ad0 msgr2=0x7f328419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.324+0000 7f3282800640 1 --2- 172.21.15.47:0/3561026839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3284104ad0 0x7f328419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.324+0000 7f3282800640 1 -- 172.21.15.47:0/3561026839 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32841a17f0 con 0x7f32841038e0 2024-03-21T11:38:38.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.325+0000 7f3282800640 1 --2- 172.21.15.47:0/3561026839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32841038e0 0x7f3284199bd0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f327400c970 tx=0x7f327400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:38.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.325+0000 7f327b400640 1 -- 172.21.15.47:0/3561026839 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274004220 con 0x7f32841038e0 2024-03-21T11:38:38.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.326+0000 7f327b400640 1 -- 172.21.15.47:0/3561026839 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32740043c0 con 0x7f32841038e0 2024-03-21T11:38:38.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.326+0000 7f327b400640 1 -- 172.21.15.47:0/3561026839 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327400f450 con 0x7f32841038e0 2024-03-21T11:38:38.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.326+0000 7f3289400640 1 -- 172.21.15.47:0/3561026839 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32841a1ae0 con 0x7f32841038e0 2024-03-21T11:38:38.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.326+0000 7f3289400640 1 -- 172.21.15.47:0/3561026839 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32841a1fe0 con 0x7f32841038e0 2024-03-21T11:38:38.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.330+0000 7f328c049740 1 Processor -- start 2024-03-21T11:38:38.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.331+0000 7f328c049740 1 -- start start 2024-03-21T11:38:38.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.332+0000 7f328c049740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cae80 0x5627223cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.332+0000 7f328c049740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.332+0000 7f328c049740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223d09e0 0x5627223d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.332+0000 7f328c049740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562722330250 con 0x5627223cae80 2024-03-21T11:38:38.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.332+0000 7f328c049740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627222d8a30 con 0x5627223d09e0 2024-03-21T11:38:38.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.332+0000 7f328c049740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56272232c2f0 con 0x5627223cb870 2024-03-21T11:38:38.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.333+0000 7f327b400640 1 -- 172.21.15.47:0/3561026839 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f32740079b0 con 0x7f32841038e0 2024-03-21T11:38:38.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.333+0000 7f327be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223d09e0 0x5627223d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.333+0000 7f3282800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.333+0000 7f3282800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60652/0 (socket says 172.21.15.47:60652) 2024-03-21T11:38:38.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.333+0000 7f3282800640 1 -- 172.21.15.47:0/798257730 learned_addr learned my addr 172.21.15.47:0/798257730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:38.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.334+0000 7f3282800640 1 -- 172.21.15.47:0/798257730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223d09e0 msgr2=0x5627223d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.334+0000 7f3282800640 1 --2- 172.21.15.47:0/798257730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223d09e0 0x5627223d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.334+0000 7f3282800640 1 -- 172.21.15.47:0/798257730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cae80 msgr2=0x5627223cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.333+0000 7f3283200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cae80 0x5627223cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.336+0000 7f3282800640 1 --2- 172.21.15.47:0/798257730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cae80 0x5627223cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.336+0000 7f3282800640 1 -- 172.21.15.47:0/798257730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627222c3690 con 0x5627223cb870 2024-03-21T11:38:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.337+0000 7f3283200640 1 --2- 172.21.15.47:0/798257730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cae80 0x5627223cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:38:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.337+0000 7f327b400640 1 --2- 172.21.15.47:0/3561026839 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32680a2fa0 0x7f32680a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.338+0000 7f327b400640 1 -- 172.21.15.47:0/3561026839 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f32740cf800 con 0x7f32841038e0 2024-03-21T11:38:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.339+0000 7f3282800640 1 --2- 172.21.15.47:0/798257730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223d0370 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f32740d5c70 tx=0x7f32740d7690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4023829ab6544ec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.339+0000 7f3273400640 1 -- 172.21.15.47:0/798257730 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274097e40 con 0x5627223cb870 2024-03-21T11:38:38.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.340+0000 7f3273400640 1 -- 172.21.15.47:0/798257730 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f327409c2a0 con 0x5627223cb870 2024-03-21T11:38:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.340+0000 7f3283200640 1 --2- 172.21.15.47:0/3561026839 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32680a2fa0 0x7f32680a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.342+0000 7f328c049740 1 -- 172.21.15.47:0/798257730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 msgr2=0x5627223d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.342+0000 7f328c049740 1 --2- 172.21.15.47:0/798257730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223d0370 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f32740d5c70 tx=0x7f32740d7690 comp rx=0 tx=0).stop 2024-03-21T11:38:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.342+0000 7f3273400640 1 -- 172.21.15.47:0/798257730 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327409b500 con 0x5627223cb870 2024-03-21T11:38:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.342+0000 7f328c049740 1 -- 172.21.15.47:0/798257730 shutdown_connections 2024-03-21T11:38:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.342+0000 7f328c049740 1 --2- 172.21.15.47:0/798257730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223d09e0 0x5627223d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.342+0000 7f328c049740 1 --2- 172.21.15.47:0/798257730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223d0370 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.342+0000 7f328c049740 1 --2- 172.21.15.47:0/798257730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cae80 0x5627223cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.342+0000 7f328c049740 1 -- 172.21.15.47:0/798257730 >> 172.21.15.47:0/798257730 conn(0x562722393200 msgr2=0x5627223c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:38.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.343+0000 7f3283200640 1 --2- 172.21.15.47:0/3561026839 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32680a2fa0 0x7f32680a5460 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f327c02c640 tx=0x7f327c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:38.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.343+0000 7f328c049740 1 -- 172.21.15.47:0/798257730 shutdown_connections 2024-03-21T11:38:38.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.344+0000 7f328c049740 1 -- 172.21.15.47:0/798257730 wait complete. 2024-03-21T11:38:38.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.344+0000 7f328c049740 1 -- start start 2024-03-21T11:38:38.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.345+0000 7f328c049740 1 Processor -- start 2024-03-21T11:38:38.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.345+0000 7f328c049740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cae80 0x5627223d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.345+0000 7f328c049740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223d53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.345+0000 7f328c049740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223d09e0 0x5627223dc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.345+0000 7f328c049740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627223d3a30 con 0x5627223cae80 2024-03-21T11:38:38.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.345+0000 7f328c049740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627222ebe80 con 0x5627223d09e0 2024-03-21T11:38:38.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.345+0000 7f328c049740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627223d3bb0 con 0x5627223cb870 2024-03-21T11:38:38.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.346+0000 7f3283200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223d09e0 0x5627223dc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.346+0000 7f3283200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223d09e0 0x5627223dc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53340/0 (socket says 172.21.15.47:53340) 2024-03-21T11:38:38.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.346+0000 7f3283200640 1 -- 172.21.15.47:0/1011202685 learned_addr learned my addr 172.21.15.47:0/1011202685 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:38.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.346+0000 7f328c049740 10 client.0 _ll_register_callbacks cb 0x5627222fe130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:38.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.346+0000 7f327be00640 1 --2- 172.21.15.47:0/1011202685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223d53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.346+0000 7f3282800640 1 --2- 172.21.15.47:0/1011202685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cae80 0x5627223d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.347+0000 7f3283200640 1 -- 172.21.15.47:0/1011202685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 msgr2=0x5627223d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.347+0000 7f3283200640 1 --2- 172.21.15.47:0/1011202685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.347+0000 7f3283200640 1 -- 172.21.15.47:0/1011202685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cae80 msgr2=0x5627223d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.347+0000 7f3283200640 1 --2- 172.21.15.47:0/1011202685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cae80 0x5627223d4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.347+0000 7f3283200640 1 -- 172.21.15.47:0/1011202685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56272210e7d0 con 0x5627223d09e0 2024-03-21T11:38:38.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.347+0000 7f3283200640 1 --2- 172.21.15.47:0/1011202685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223d09e0 0x5627223dc3e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f327c036000 tx=0x7f327c02c100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3612f850568db389 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:38.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.348+0000 7f3267e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:38.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.348+0000 7f3267e00640 1 -- 172.21.15.47:0/1011202685 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c039050 con 0x5627223d09e0 2024-03-21T11:38:38.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.348+0000 7f328c049740 1 -- 172.21.15.47:0/1011202685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f327409c2a0 con 0x5627223d09e0 2024-03-21T11:38:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.349+0000 7f3267e00640 1 -- 172.21.15.47:0/1011202685 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f327c0041b0 con 0x5627223d09e0 2024-03-21T11:38:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.349+0000 7f3267e00640 1 -- 172.21.15.47:0/1011202685 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f327c059050 con 0x5627223d09e0 2024-03-21T11:38:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.349+0000 7f328c049740 10 client.30635 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.349+0000 7f328c049740 1 -- 172.21.15.47:0/1011202685 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5627220443f0 con 0x5627223d09e0 2024-03-21T11:38:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.350+0000 7f3267e00640 1 -- 172.21.15.47:0/1011202685 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f327409c2a0 con 0x5627223d09e0 2024-03-21T11:38:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.351+0000 7f3267e00640 1 -- 172.21.15.47:0/1011202685 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f327c054030 con 0x5627223d09e0 2024-03-21T11:38:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.351+0000 7f328c049740 10 client.30635 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.351+0000 7f328c049740 10 client.30635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.351+0000 7f328c049740 20 client.30635 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.351+0000 7f328c049740 1 --2- 172.21.15.47:0/1011202685 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5627223e0c30 0x5627223e30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.351+0000 7f328c049740 4 client.30635 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.351+0000 7f328c049740 1 -- 172.21.15.47:0/1011202685 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56272233d790 con 0x5627223e0c30 2024-03-21T11:38:38.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.352+0000 7f3282800640 1 --2- 172.21.15.47:0/1011202685 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5627223e0c30 0x5627223e30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.358+0000 7f3282800640 1 --2- 172.21.15.47:0/1011202685 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5627223e0c30 0x5627223e30b0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe49155ce7129ada server_cookie=c1194332beb39aa8 in_seq=0 out_seq=0 2024-03-21T11:38:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.358+0000 7f3267e00640 10 client.30635 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.542+0000 7f4e21600640 1 -- 172.21.15.47:0/3253746235 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df2c6c00c0 con 0x55df2c7d6590 2024-03-21T11:38:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.542+0000 7f4e21600640 10 client.30623 handle_command_reply: tid=0 2024-03-21T11:38:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.542+0000 7f4e3d24f740 1 client.30623 shutdown 2024-03-21T11:38:38.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.542+0000 7f3267e00640 1 -- 172.21.15.47:0/1011202685 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56272233d790 con 0x5627223e0c30 2024-03-21T11:38:38.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.542+0000 7f3267e00640 10 client.30635 handle_command_reply: tid=0 2024-03-21T11:38:38.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.543+0000 7f328c049740 1 client.30635 shutdown 2024-03-21T11:38:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.543+0000 7f4e3d24f740 1 -- 172.21.15.47:0/3253746235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c764be0 msgr2=0x55df2c773c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.543+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/3253746235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c764be0 0x55df2c773c40 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c086010 tx=0x7f4e1c0ccaa0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.543+0000 7f4e3d24f740 1 -- 172.21.15.47:0/3253746235 shutdown_connections 2024-03-21T11:38:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.543+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/3253746235 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df2c7d6590 0x55df2c7d89b0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.543+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/3253746235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df2c769d50 0x55df2c774c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.543+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/3253746235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df2c764be0 0x55df2c773c40 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.543+0000 7f4e3d24f740 1 --2- 172.21.15.47:0/3253746235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df2c7641f0 0x55df2c773700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.543+0000 7f4e3d24f740 1 -- 172.21.15.47:0/3253746235 >> 172.21.15.47:0/3253746235 conn(0x55df2c74e750 msgr2=0x55df2c752240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.543+0000 7f4e21600640 0 client.30623 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.544+0000 7f4e3d24f740 1 -- 172.21.15.47:0/3253746235 shutdown_connections 2024-03-21T11:38:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.544+0000 7f4e3d24f740 1 -- 172.21.15.47:0/3253746235 wait complete. 2024-03-21T11:38:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.544+0000 7f4e3d24f740 20 client.30623 trim_cache size 0 max 16384 2024-03-21T11:38:38.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.546+0000 7f328c049740 1 -- 172.21.15.47:0/1011202685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223d09e0 msgr2=0x5627223dc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.546+0000 7f328c049740 1 --2- 172.21.15.47:0/1011202685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223d09e0 0x5627223dc3e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f327c036000 tx=0x7f327c02c100 comp rx=0 tx=0).stop 2024-03-21T11:38:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:38:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:38:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:38:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:38:38.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.548+0000 7f328c049740 1 -- 172.21.15.47:0/1011202685 shutdown_connections 2024-03-21T11:38:38.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.548+0000 7f328c049740 1 --2- 172.21.15.47:0/1011202685 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5627223e0c30 0x5627223e30b0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.548+0000 7f328c049740 1 --2- 172.21.15.47:0/1011202685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223d09e0 0x5627223dc3e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.548+0000 7f328c049740 1 --2- 172.21.15.47:0/1011202685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223d53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.548+0000 7f328c049740 1 --2- 172.21.15.47:0/1011202685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cae80 0x5627223d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.548+0000 7f328c049740 1 -- 172.21.15.47:0/1011202685 >> 172.21.15.47:0/1011202685 conn(0x562722393200 msgr2=0x5627223c5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:38.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.548+0000 7f3267e00640 0 client.30635 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:38.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.548+0000 7f328c049740 1 -- 172.21.15.47:0/1011202685 shutdown_connections 2024-03-21T11:38:38.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.548+0000 7f328c049740 1 -- 172.21.15.47:0/1011202685 wait complete. 2024-03-21T11:38:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.548+0000 7f4e3a600640 1 -- 172.21.15.47:0/2142154667 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e180a3090 msgr2=0x7f4e180a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.548+0000 7f4e3a600640 1 --2- 172.21.15.47:0/2142154667 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e180a3090 0x7f4e180a5550 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55df2c74f6c0 tx=0x7f4e2400a2d0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.548+0000 7f4e3a600640 1 -- 172.21.15.47:0/2142154667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e340772a0 msgr2=0x7f4e3419e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.548+0000 7f4e3a600640 1 --2- 172.21.15.47:0/2142154667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e340772a0 0x7f4e3419e390 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00d950 tx=0x7f4e1c00dd50 comp rx=0 tx=0).stop 2024-03-21T11:38:38.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.548+0000 7f328c049740 20 client.30635 trim_cache size 0 max 16384 2024-03-21T11:38:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.549+0000 7f4e3a600640 1 -- 172.21.15.47:0/2142154667 shutdown_connections 2024-03-21T11:38:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.549+0000 7f4e3a600640 1 --2- 172.21.15.47:0/2142154667 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e180a3090 0x7f4e180a5550 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.549+0000 7f4e3a600640 1 --2- 172.21.15.47:0/2142154667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e34077c40 0x7f4e341a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.549+0000 7f4e3a600640 1 --2- 172.21.15.47:0/2142154667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e340772a0 0x7f4e3419e390 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.549+0000 7f4e3a600640 1 --2- 172.21.15.47:0/2142154667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e34076060 0x7f4e3419de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.549+0000 7f4e3a600640 1 -- 172.21.15.47:0/2142154667 >> 172.21.15.47:0/2142154667 conn(0x7f4e340fee90 msgr2=0x7f4e34100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.549+0000 7f4e3a600640 1 -- 172.21.15.47:0/2142154667 shutdown_connections 2024-03-21T11:38:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:38.551+0000 7f4e3a600640 1 -- 172.21.15.47:0/2142154667 wait complete. 2024-03-21T11:38:38.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.571+0000 7f328c049740 1 Processor -- start 2024-03-21T11:38:38.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.571+0000 7f328c049740 1 -- start start 2024-03-21T11:38:38.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.571+0000 7f328c049740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223cae80 0x5627223c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.571+0000 7f328c049740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223dc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.571+0000 7f328c049740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223d09e0 0x5627223c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.571+0000 7f328c049740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32841a8080 con 0x5627223d09e0 2024-03-21T11:38:38.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.571+0000 7f328c049740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32841a9b80 con 0x5627223cae80 2024-03-21T11:38:38.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.571+0000 7f328c049740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32841a86e0 con 0x5627223cb870 2024-03-21T11:38:38.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.571+0000 7f327be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223dc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.571+0000 7f327be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223dc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60666/0 (socket says 172.21.15.47:60666) 2024-03-21T11:38:38.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.571+0000 7f327be00640 1 -- 172.21.15.47:0/2298399706 learned_addr learned my addr 172.21.15.47:0/2298399706 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:38.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.571+0000 7f3283200640 1 --2- 172.21.15.47:0/2298399706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223d09e0 0x5627223c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.572+0000 7f327be00640 1 -- 172.21.15.47:0/2298399706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223cae80 msgr2=0x5627223c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.572+0000 7f327be00640 1 --2- 172.21.15.47:0/2298399706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223cae80 0x5627223c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.572+0000 7f327be00640 1 -- 172.21.15.47:0/2298399706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223d09e0 msgr2=0x5627223c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.572+0000 7f327be00640 1 --2- 172.21.15.47:0/2298399706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223d09e0 0x5627223c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.572+0000 7f327be00640 1 -- 172.21.15.47:0/2298399706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627220040f0 con 0x5627223cb870 2024-03-21T11:38:38.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.572+0000 7f3283200640 1 --2- 172.21.15.47:0/2298399706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223d09e0 0x5627223c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:38.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.572+0000 7f327be00640 1 --2- 172.21.15.47:0/2298399706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223dc3a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f326c00b8f0 tx=0x7f326c00bdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50f9d30f3f3ca02 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:38.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.574+0000 7f3272a00640 1 -- 172.21.15.47:0/2298399706 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c019030 con 0x5627223cb870 2024-03-21T11:38:38.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.574+0000 7f3272a00640 1 -- 172.21.15.47:0/2298399706 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f326c012d90 con 0x5627223cb870 2024-03-21T11:38:38.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.574+0000 7f3272a00640 1 -- 172.21.15.47:0/2298399706 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c015da0 con 0x5627223cb870 2024-03-21T11:38:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.575+0000 7f328c049740 1 -- 172.21.15.47:0/2298399706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 msgr2=0x5627223dc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.575+0000 7f328c049740 1 --2- 172.21.15.47:0/2298399706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223dc3a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f326c00b8f0 tx=0x7f326c00bdb0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.575+0000 7f328c049740 1 -- 172.21.15.47:0/2298399706 shutdown_connections 2024-03-21T11:38:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.575+0000 7f328c049740 1 --2- 172.21.15.47:0/2298399706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223d09e0 0x5627223c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.575+0000 7f328c049740 1 --2- 172.21.15.47:0/2298399706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223cb870 0x5627223dc3a0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.575+0000 7f328c049740 1 --2- 172.21.15.47:0/2298399706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223cae80 0x5627223c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.575+0000 7f328c049740 1 -- 172.21.15.47:0/2298399706 >> 172.21.15.47:0/2298399706 conn(0x562722393200 msgr2=0x5627223c6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.575+0000 7f328c049740 1 -- 172.21.15.47:0/2298399706 shutdown_connections 2024-03-21T11:38:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.575+0000 7f328c049740 1 -- 172.21.15.47:0/2298399706 wait complete. 2024-03-21T11:38:38.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.575+0000 7f328c049740 1 -- start start 2024-03-21T11:38:38.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.576+0000 7f328c049740 1 Processor -- start 2024-03-21T11:38:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.576+0000 7f328c049740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223cae80 0x5627223da2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.576+0000 7f328c049740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cb870 0x5627223da7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.576+0000 7f328c049740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223d09e0 0x5627223db7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.576+0000 7f328c049740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32841a9460 con 0x5627223cb870 2024-03-21T11:38:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.576+0000 7f328c049740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32841a8e00 con 0x5627223cae80 2024-03-21T11:38:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.576+0000 7f328c049740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32841a6240 con 0x5627223d09e0 2024-03-21T11:38:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.576+0000 7f328c049740 10 client.0 _ll_register_callbacks cb 0x7f327c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:38.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.577+0000 7f3282800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223cae80 0x5627223da2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.577+0000 7f3283200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223d09e0 0x5627223db7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.577+0000 7f3282800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223cae80 0x5627223da2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53358/0 (socket says 172.21.15.47:53358) 2024-03-21T11:38:38.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.577+0000 7f3283200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223d09e0 0x5627223db7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60672/0 (socket says 172.21.15.47:60672) 2024-03-21T11:38:38.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.577+0000 7f3282800640 1 -- 172.21.15.47:0/3879697946 learned_addr learned my addr 172.21.15.47:0/3879697946 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:38.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.577+0000 7f327be00640 1 --2- 172.21.15.47:0/3879697946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cb870 0x5627223da7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.577+0000 7f3282800640 1 -- 172.21.15.47:0/3879697946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223d09e0 msgr2=0x5627223db7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.577+0000 7f3282800640 1 --2- 172.21.15.47:0/3879697946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223d09e0 0x5627223db7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.577+0000 7f3282800640 1 -- 172.21.15.47:0/3879697946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cb870 msgr2=0x5627223da7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:38.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.577+0000 7f3282800640 1 --2- 172.21.15.47:0/3879697946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cb870 0x5627223da7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.577+0000 7f3282800640 1 -- 172.21.15.47:0/3879697946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627223dbde0 con 0x5627223cae80 2024-03-21T11:38:38.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.578+0000 7f3282800640 1 --2- 172.21.15.47:0/3879697946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223cae80 0x5627223da2a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3274003a00 tx=0x7f32740038e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e34b30a9ada688c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:38.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.578+0000 7f3267e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:38.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.578+0000 7f3267e00640 1 -- 172.21.15.47:0/3879697946 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327409b6e0 con 0x5627223cae80 2024-03-21T11:38:38.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.578+0000 7f328c049740 1 -- 172.21.15.47:0/3879697946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f326c012d90 con 0x5627223cae80 2024-03-21T11:38:38.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.578+0000 7f3267e00640 1 -- 172.21.15.47:0/3879697946 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3274003610 con 0x5627223cae80 2024-03-21T11:38:38.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.578+0000 7f3267e00640 1 -- 172.21.15.47:0/3879697946 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f327409b970 con 0x5627223cae80 2024-03-21T11:38:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.578+0000 7f328c049740 10 client.30638 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.578+0000 7f328c049740 1 -- 172.21.15.47:0/3879697946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56272244de10 con 0x5627223cae80 2024-03-21T11:38:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.579+0000 7f3267e00640 1 -- 172.21.15.47:0/3879697946 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f326c012d90 con 0x5627223cae80 2024-03-21T11:38:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.579+0000 7f3267e00640 1 -- 172.21.15.47:0/3879697946 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f32740d5240 con 0x5627223cae80 2024-03-21T11:38:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.579+0000 7f328c049740 10 client.30638 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.579+0000 7f328c049740 10 client.30638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.579+0000 7f328c049740 20 client.30638 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.579+0000 7f328c049740 1 --2- 172.21.15.47:0/3879697946 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56272244e370 0x562722450760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.579+0000 7f328c049740 4 client.30638 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:38:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.579+0000 7f328c049740 1 -- 172.21.15.47:0/3879697946 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562722337ae0 con 0x56272244e370 2024-03-21T11:38:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.579+0000 7f327be00640 1 --2- 172.21.15.47:0/3879697946 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56272244e370 0x562722450760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.580+0000 7f327be00640 1 --2- 172.21.15.47:0/3879697946 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56272244e370 0x562722450760 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3bbc087c7cea8fd server_cookie=58595495401c54f0 in_seq=0 out_seq=0 2024-03-21T11:38:38.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.580+0000 7f3267e00640 10 client.30638 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:38.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.584+0000 7f3267e00640 1 -- 172.21.15.47:0/3879697946 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x562722337ae0 con 0x56272244e370 2024-03-21T11:38:38.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.584+0000 7f3267e00640 10 client.30638 handle_command_reply: tid=0 2024-03-21T11:38:38.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.584+0000 7f328c049740 1 client.30638 shutdown 2024-03-21T11:38:38.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.584+0000 7f328c049740 1 -- 172.21.15.47:0/3879697946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223cae80 msgr2=0x5627223da2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.584+0000 7f328c049740 1 --2- 172.21.15.47:0/3879697946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223cae80 0x5627223da2a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3274003a00 tx=0x7f32740038e0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.584+0000 7f328c049740 1 -- 172.21.15.47:0/3879697946 shutdown_connections 2024-03-21T11:38:38.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.584+0000 7f328c049740 1 --2- 172.21.15.47:0/3879697946 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56272244e370 0x562722450760 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.585+0000 7f328c049740 1 --2- 172.21.15.47:0/3879697946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627223d09e0 0x5627223db7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.585+0000 7f328c049740 1 --2- 172.21.15.47:0/3879697946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627223cb870 0x5627223da7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.585+0000 7f328c049740 1 --2- 172.21.15.47:0/3879697946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627223cae80 0x5627223da2a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.585+0000 7f328c049740 1 -- 172.21.15.47:0/3879697946 >> 172.21.15.47:0/3879697946 conn(0x562722393200 msgr2=0x5627223c5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:38.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.585+0000 7f3267e00640 0 client.30638 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:38.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.585+0000 7f328c049740 1 -- 172.21.15.47:0/3879697946 shutdown_connections 2024-03-21T11:38:38.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.585+0000 7f328c049740 1 -- 172.21.15.47:0/3879697946 wait complete. 2024-03-21T11:38:38.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.587+0000 7f328c049740 20 client.30638 trim_cache size 0 max 16384 2024-03-21T11:38:38.589 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:38:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (211 inodes in the stack)", 2024-03-21T11:38:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:38:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: "141c2112-b51d-4e48-96a2-845e628b4583": { 2024-03-21T11:38:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:38:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "141c2112-b51d-4e48-96a2-845e628b4583", 2024-03-21T11:38:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:38:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:38:38.589 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:38:38.589 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:38:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.588+0000 7f3289400640 1 -- 172.21.15.47:0/3561026839 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32680a2fa0 msgr2=0x7f32680a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.589+0000 7f3289400640 1 --2- 172.21.15.47:0/3561026839 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32680a2fa0 0x7f32680a5460 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f327c02c640 tx=0x7f327c02e040 comp rx=0 tx=0).stop 2024-03-21T11:38:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.589+0000 7f3289400640 1 -- 172.21.15.47:0/3561026839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32841038e0 msgr2=0x7f3284199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.589+0000 7f3289400640 1 --2- 172.21.15.47:0/3561026839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32841038e0 0x7f3284199bd0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f327400c970 tx=0x7f327400ce30 comp rx=0 tx=0).stop 2024-03-21T11:38:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.589+0000 7f3289400640 1 -- 172.21.15.47:0/3561026839 shutdown_connections 2024-03-21T11:38:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.589+0000 7f3289400640 1 --2- 172.21.15.47:0/3561026839 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32680a2fa0 0x7f32680a5460 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.589+0000 7f3289400640 1 --2- 172.21.15.47:0/3561026839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3284105490 0x7f32841a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.589+0000 7f3289400640 1 --2- 172.21.15.47:0/3561026839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3284104ad0 0x7f328419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.589+0000 7f3289400640 1 --2- 172.21.15.47:0/3561026839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32841038e0 0x7f3284199bd0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.589+0000 7f3289400640 1 -- 172.21.15.47:0/3561026839 >> 172.21.15.47:0/3561026839 conn(0x7f32840fee50 msgr2=0x7f3284107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.589+0000 7f3289400640 1 -- 172.21.15.47:0/3561026839 shutdown_connections 2024-03-21T11:38:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:38:38.589+0000 7f3289400640 1 -- 172.21.15.47:0/3561026839 wait complete. 2024-03-21T11:38:38.611 INFO:tasks.cephfs.filesystem:scrub status for tag:141c2112-b51d-4e48-96a2-845e628b4583 - {'path': '/', 'tag': '141c2112-b51d-4e48-96a2-845e628b4583', 'options': 'recursive,force'} 2024-03-21T11:38:38.926 INFO:tasks.workunit.client.0.smithi047.stdout: 10 40026 28.42 MB/sec warmup 82 sec latency 277.439 ms 2024-03-21T11:38:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: pgmap v859: 129 pgs: 129 active+clean; 324 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.5 MiB/s wr, 702 op/s 2024-03-21T11:38:39.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:38:39.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:39.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:39.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:39.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:39.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:39.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:39.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:39.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:39.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:39 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: pgmap v859: 129 pgs: 129 active+clean; 324 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.5 MiB/s wr, 702 op/s 2024-03-21T11:38:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:38:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:39.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:39.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:39.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:39.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:39.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:39 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: pgmap v859: 129 pgs: 129 active+clean; 324 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.5 MiB/s wr, 702 op/s 2024-03-21T11:38:39.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:38:39.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:39.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:39.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:39.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:39.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:39.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:39.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:39.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:39.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:39 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:39.926 INFO:tasks.workunit.client.0.smithi047.stdout: 10 40525 28.37 MB/sec warmup 83 sec latency 115.028 ms 2024-03-21T11:38:40.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:40 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:40.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:40 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:38:40.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:40 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:40.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:40 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:40.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:40 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:40.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:40 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:38:40.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:40 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:40.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:40 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:40.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:40 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:40.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:40 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:38:40.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:40 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:40.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:40 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:38:40.926 INFO:tasks.workunit.client.0.smithi047.stdout: 10 41153 28.49 MB/sec warmup 84 sec latency 52.219 ms 2024-03-21T11:38:41.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:41 smithi047 ceph-mon[31351]: pgmap v860: 129 pgs: 129 active+clean; 307 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 800 op/s 2024-03-21T11:38:41.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:41 smithi100 ceph-mon[36245]: pgmap v860: 129 pgs: 129 active+clean; 307 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 800 op/s 2024-03-21T11:38:41.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:41 smithi102 ceph-mon[36748]: pgmap v860: 129 pgs: 129 active+clean; 307 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 800 op/s 2024-03-21T11:38:41.927 INFO:tasks.workunit.client.0.smithi047.stdout: 10 41774 28.44 MB/sec warmup 85 sec latency 159.169 ms 2024-03-21T11:38:42.927 INFO:tasks.workunit.client.0.smithi047.stdout: 10 42425 28.55 MB/sec warmup 86 sec latency 69.614 ms 2024-03-21T11:38:43.292 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:43 smithi102 ceph-mon[36748]: pgmap v861: 129 pgs: 129 active+clean; 293 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 684 op/s 2024-03-21T11:38:43.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:43 smithi047 ceph-mon[31351]: pgmap v861: 129 pgs: 129 active+clean; 293 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 684 op/s 2024-03-21T11:38:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:43 smithi100 ceph-mon[36245]: pgmap v861: 129 pgs: 129 active+clean; 293 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 684 op/s 2024-03-21T11:38:43.927 INFO:tasks.workunit.client.0.smithi047.stdout: 10 43225 28.77 MB/sec warmup 87 sec latency 54.339 ms 2024-03-21T11:38:44.927 INFO:tasks.workunit.client.0.smithi047.stdout: 10 43772 28.84 MB/sec warmup 88 sec latency 44.559 ms 2024-03-21T11:38:45.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:45 smithi047 ceph-mon[31351]: pgmap v862: 129 pgs: 129 active+clean; 298 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 840 op/s 2024-03-21T11:38:45.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:45 smithi100 ceph-mon[36245]: pgmap v862: 129 pgs: 129 active+clean; 298 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 840 op/s 2024-03-21T11:38:45.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:45 smithi102 ceph-mon[36748]: pgmap v862: 129 pgs: 129 active+clean; 298 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 840 op/s 2024-03-21T11:38:45.927 INFO:tasks.workunit.client.0.smithi047.stdout: 10 44157 28.74 MB/sec warmup 89 sec latency 84.539 ms 2024-03-21T11:38:46.927 INFO:tasks.workunit.client.0.smithi047.stdout: 10 44574 28.59 MB/sec warmup 90 sec latency 85.001 ms 2024-03-21T11:38:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:47 smithi047 ceph-mon[31351]: pgmap v863: 129 pgs: 129 active+clean; 303 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.8 MiB/s wr, 743 op/s 2024-03-21T11:38:47.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:47 smithi100 ceph-mon[36245]: pgmap v863: 129 pgs: 129 active+clean; 303 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.8 MiB/s wr, 743 op/s 2024-03-21T11:38:47.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:47 smithi102 ceph-mon[36748]: pgmap v863: 129 pgs: 129 active+clean; 303 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.8 MiB/s wr, 743 op/s 2024-03-21T11:38:47.927 INFO:tasks.workunit.client.0.smithi047.stdout: 10 45194 28.62 MB/sec warmup 91 sec latency 62.028 ms 2024-03-21T11:38:48.927 INFO:tasks.workunit.client.0.smithi047.stdout: 10 46040 28.81 MB/sec warmup 92 sec latency 38.745 ms 2024-03-21T11:38:49.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:49 smithi047 ceph-mon[31351]: pgmap v864: 129 pgs: 129 active+clean; 307 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.4 MiB/s wr, 739 op/s 2024-03-21T11:38:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:49 smithi100 ceph-mon[36245]: pgmap v864: 129 pgs: 129 active+clean; 307 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.4 MiB/s wr, 739 op/s 2024-03-21T11:38:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:49 smithi102 ceph-mon[36748]: pgmap v864: 129 pgs: 129 active+clean; 307 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.4 MiB/s wr, 739 op/s 2024-03-21T11:38:49.927 INFO:tasks.workunit.client.0.smithi047.stdout: 10 46697 28.95 MB/sec warmup 93 sec latency 41.608 ms 2024-03-21T11:38:50.933 INFO:tasks.workunit.client.0.smithi047.stdout: 10 47194 28.96 MB/sec warmup 94 sec latency 45.125 ms 2024-03-21T11:38:51.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:51 smithi047 ceph-mon[31351]: pgmap v865: 129 pgs: 129 active+clean; 314 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.9 MiB/s wr, 732 op/s 2024-03-21T11:38:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:51 smithi100 ceph-mon[36245]: pgmap v865: 129 pgs: 129 active+clean; 314 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.9 MiB/s wr, 732 op/s 2024-03-21T11:38:51.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:51 smithi102 ceph-mon[36748]: pgmap v865: 129 pgs: 129 active+clean; 314 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.9 MiB/s wr, 732 op/s 2024-03-21T11:38:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.716+0000 7fc22d000640 1 -- 172.21.15.47:0/639818771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2281012b0 msgr2=0x7fc228101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.716+0000 7fc22d000640 1 --2- 172.21.15.47:0/639818771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2281012b0 0x7fc228101730 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fc220009be0 tx=0x7fc22002bf80 comp rx=0 tx=0).stop 2024-03-21T11:38:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.716+0000 7fc22d000640 1 -- 172.21.15.47:0/639818771 shutdown_connections 2024-03-21T11:38:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.716+0000 7fc22d000640 1 --2- 172.21.15.47:0/639818771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2281012b0 0x7fc228101730 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.716+0000 7fc22d000640 1 --2- 172.21.15.47:0/639818771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc228100910 0x7fc228100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.716+0000 7fc22d000640 1 --2- 172.21.15.47:0/639818771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2280fffb0 0x7fc2281003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.716+0000 7fc22d000640 1 -- 172.21.15.47:0/639818771 >> 172.21.15.47:0/639818771 conn(0x7fc2280fbb30 msgr2=0x7fc2280fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.716+0000 7fc22d000640 1 -- 172.21.15.47:0/639818771 shutdown_connections 2024-03-21T11:38:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.716+0000 7fc22d000640 1 -- 172.21.15.47:0/639818771 wait complete. 2024-03-21T11:38:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.717+0000 7fc22d000640 1 Processor -- start 2024-03-21T11:38:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.718+0000 7fc22d000640 1 -- start start 2024-03-21T11:38:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.718+0000 7fc22d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2280fffb0 0x7fc228199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.718+0000 7fc22d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc228100910 0x7fc228199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.718+0000 7fc22d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2281012b0 0x7fc2281a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.718+0000 7fc22d000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22810f570 con 0x7fc2280fffb0 2024-03-21T11:38:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.718+0000 7fc22d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22810f3f0 con 0x7fc228100910 2024-03-21T11:38:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.718+0000 7fc22d000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22810f6f0 con 0x7fc2281012b0 2024-03-21T11:38:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.718+0000 7fc21fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc228100910 0x7fc228199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.718+0000 7fc21fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc228100910 0x7fc228199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55824/0 (socket says 172.21.15.47:55824) 2024-03-21T11:38:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.718+0000 7fc21fe00640 1 -- 172.21.15.47:0/587992518 learned_addr learned my addr 172.21.15.47:0/587992518 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.718+0000 7fc226800640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2281012b0 0x7fc2281a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.719+0000 7fc225e00640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2280fffb0 0x7fc228199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.719+0000 7fc21fe00640 1 -- 172.21.15.47:0/587992518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2281012b0 msgr2=0x7fc2281a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.719+0000 7fc21fe00640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2281012b0 0x7fc2281a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.719+0000 7fc21fe00640 1 -- 172.21.15.47:0/587992518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2280fffb0 msgr2=0x7fc228199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.719+0000 7fc21fe00640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2280fffb0 0x7fc228199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.719+0000 7fc21fe00640 1 -- 172.21.15.47:0/587992518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2281a16c0 con 0x7fc228100910 2024-03-21T11:38:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.719+0000 7fc226800640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2281012b0 0x7fc2281a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:38:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.720+0000 7fc21fe00640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc228100910 0x7fc228199fe0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc21000e9c0 tx=0x7fc21000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.720+0000 7fc21ea00640 1 -- 172.21.15.47:0/587992518 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21000ccc0 con 0x7fc228100910 2024-03-21T11:38:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.720+0000 7fc21ea00640 1 -- 172.21.15.47:0/587992518 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc210005500 con 0x7fc228100910 2024-03-21T11:38:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.720+0000 7fc22d000640 1 -- 172.21.15.47:0/587992518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2281a19b0 con 0x7fc228100910 2024-03-21T11:38:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.720+0000 7fc21ea00640 1 -- 172.21.15.47:0/587992518 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210010430 con 0x7fc228100910 2024-03-21T11:38:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.720+0000 7fc22d000640 1 -- 172.21.15.47:0/587992518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2281021c0 con 0x7fc228100910 2024-03-21T11:38:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.724+0000 7fc21ea00640 1 -- 172.21.15.47:0/587992518 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc2100046c0 con 0x7fc228100910 2024-03-21T11:38:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.724+0000 7fc22f9fd740 1 Processor -- start 2024-03-21T11:38:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.724+0000 7fc22f9fd740 1 -- start start 2024-03-21T11:38:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc22f9fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b5785600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc22f9fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b578a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc22f9fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 0x55d3b578d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc22f9fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b57483b0 con 0x55d3b578ad40 2024-03-21T11:38:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc22f9fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b56ea830 con 0x55d3b57851e0 2024-03-21T11:38:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc22f9fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b56935d0 con 0x55d3b5785bd0 2024-03-21T11:38:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc225e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b578a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc226800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b5785600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc226800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b5785600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55832/0 (socket says 172.21.15.47:55832) 2024-03-21T11:38:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc226800640 1 -- 172.21.15.47:0/3303236652 learned_addr learned my addr 172.21.15.47:0/3303236652 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc21fe00640 1 --2- 172.21.15.47:0/3303236652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 0x55d3b578d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc225e00640 1 -- 172.21.15.47:0/3303236652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 msgr2=0x55d3b5785600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc225e00640 1 --2- 172.21.15.47:0/3303236652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b5785600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc225e00640 1 -- 172.21.15.47:0/3303236652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 msgr2=0x55d3b578d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc225e00640 1 --2- 172.21.15.47:0/3303236652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 0x55d3b578d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.725+0000 7fc225e00640 1 -- 172.21.15.47:0/3303236652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3b56ab2c0 con 0x55d3b5785bd0 2024-03-21T11:38:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.726+0000 7fc21ea00640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc20c0ab7d0 0x7fc20c0adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.726+0000 7fc225e00640 1 --2- 172.21.15.47:0/3303236652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b578a6d0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc218009b70 tx=0x7fc2180117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beb398d307213871 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.726+0000 7fc21ea00640 1 -- 172.21.15.47:0/587992518 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc2100cfb60 con 0x7fc228100910 2024-03-21T11:38:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.726+0000 7fc226800640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc20c0ab7d0 0x7fc20c0adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.726+0000 7fc217400640 1 -- 172.21.15.47:0/3303236652 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218009e60 con 0x55d3b5785bd0 2024-03-21T11:38:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.727+0000 7fc217400640 1 -- 172.21.15.47:0/3303236652 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc218012930 con 0x55d3b5785bd0 2024-03-21T11:38:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.727+0000 7fc217400640 1 -- 172.21.15.47:0/3303236652 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21800fb50 con 0x55d3b5785bd0 2024-03-21T11:38:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.728+0000 7fc22f9fd740 1 -- 172.21.15.47:0/3303236652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 msgr2=0x55d3b578a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.728+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/3303236652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b578a6d0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc218009b70 tx=0x7fc2180117b0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.728+0000 7fc22f9fd740 1 -- 172.21.15.47:0/3303236652 shutdown_connections 2024-03-21T11:38:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.728+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/3303236652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 0x55d3b578d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.728+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/3303236652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b578a6d0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.728+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/3303236652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b5785600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.728+0000 7fc22f9fd740 1 -- 172.21.15.47:0/3303236652 >> 172.21.15.47:0/3303236652 conn(0x55d3b5780750 msgr2=0x55d3b5780b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.728+0000 7fc226800640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc20c0ab7d0 0x7fc20c0adc90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc2200099a0 tx=0x7fc220004a80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.728+0000 7fc22f9fd740 1 -- 172.21.15.47:0/3303236652 shutdown_connections 2024-03-21T11:38:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.728+0000 7fc22f9fd740 1 -- 172.21.15.47:0/3303236652 wait complete. 2024-03-21T11:38:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.729+0000 7fc22f9fd740 1 -- start start 2024-03-21T11:38:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.729+0000 7fc22f9fd740 1 Processor -- start 2024-03-21T11:38:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.729+0000 7fc22f9fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b578f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.729+0000 7fc22f9fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b578f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.729+0000 7fc22f9fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 0x55d3b5796900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.729+0000 7fc22f9fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b56a6a20 con 0x55d3b578ad40 2024-03-21T11:38:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.729+0000 7fc22f9fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b56e68d0 con 0x55d3b57851e0 2024-03-21T11:38:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.729+0000 7fc22f9fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b578dcf0 con 0x55d3b5785bd0 2024-03-21T11:38:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.730+0000 7fc22f9fd740 10 client.0 _ll_register_callbacks cb 0x55d3b56e31c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.730+0000 7fc21fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b578f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.730+0000 7fc225e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b578f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.730+0000 7fc21fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b578f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44918/0 (socket says 172.21.15.47:44918) 2024-03-21T11:38:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.730+0000 7fc21fe00640 1 -- 172.21.15.47:0/3599312038 learned_addr learned my addr 172.21.15.47:0/3599312038 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.730+0000 7fc226800640 1 --2- 172.21.15.47:0/3599312038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 0x55d3b5796900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.730+0000 7fc21fe00640 1 -- 172.21.15.47:0/3599312038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 msgr2=0x55d3b578f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.730+0000 7fc21fe00640 1 --2- 172.21.15.47:0/3599312038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b578f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.730+0000 7fc21fe00640 1 -- 172.21.15.47:0/3599312038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 msgr2=0x55d3b5796900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.730+0000 7fc21fe00640 1 --2- 172.21.15.47:0/3599312038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 0x55d3b5796900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.730+0000 7fc21fe00640 1 -- 172.21.15.47:0/3599312038 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3b53ff210 con 0x55d3b5785bd0 2024-03-21T11:38:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.731+0000 7fc21fe00640 1 --2- 172.21.15.47:0/3599312038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b578f930 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc210098870 tx=0x7fc2100d3190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9b16b9a575caa5c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.731+0000 7fc20be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:38:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.731+0000 7fc20be00640 1 -- 172.21.15.47:0/3599312038 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21009b800 con 0x55d3b5785bd0 2024-03-21T11:38:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.731+0000 7fc20be00640 1 -- 172.21.15.47:0/3599312038 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc2100d8070 con 0x55d3b5785bd0 2024-03-21T11:38:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.731+0000 7fc22f9fd740 1 -- 172.21.15.47:0/3599312038 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc218012930 con 0x55d3b5785bd0 2024-03-21T11:38:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.731+0000 7fc20be00640 1 -- 172.21.15.47:0/3599312038 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc21009c5a0 con 0x55d3b5785bd0 2024-03-21T11:38:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.731+0000 7fc22f9fd740 10 client.30661 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.731+0000 7fc22f9fd740 1 -- 172.21.15.47:0/3599312038 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d3b54cb620 con 0x55d3b5785bd0 2024-03-21T11:38:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.732+0000 7fc20be00640 1 -- 172.21.15.47:0/3599312038 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc218012930 con 0x55d3b5785bd0 2024-03-21T11:38:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.732+0000 7fc20be00640 1 -- 172.21.15.47:0/3599312038 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fc2100a4420 con 0x55d3b5785bd0 2024-03-21T11:38:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.732+0000 7fc22f9fd740 10 client.30661 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.732+0000 7fc22f9fd740 10 client.30661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.732+0000 7fc22f9fd740 20 client.30661 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.732+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/3599312038 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3b579b260 0x55d3b579d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.732+0000 7fc22f9fd740 4 client.30661 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:38:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.732+0000 7fc22f9fd740 1 -- 172.21.15.47:0/3599312038 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3b56f7d70 con 0x55d3b579b260 2024-03-21T11:38:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.732+0000 7fc225e00640 1 --2- 172.21.15.47:0/3599312038 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3b579b260 0x55d3b579d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.733+0000 7fc225e00640 1 --2- 172.21.15.47:0/3599312038 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3b579b260 0x55d3b579d6c0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd456b24bbe7b708 server_cookie=b39cc4dd2410c44f in_seq=0 out_seq=0 2024-03-21T11:38:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.733+0000 7fc20be00640 10 client.30661 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.735+0000 7fc20be00640 1 -- 172.21.15.47:0/3599312038 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d3b56f7d70 con 0x55d3b579b260 2024-03-21T11:38:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.735+0000 7fc20be00640 10 client.30661 handle_command_reply: tid=0 2024-03-21T11:38:51.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.735+0000 7fc22f9fd740 1 client.30661 shutdown 2024-03-21T11:38:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.735+0000 7fc22f9fd740 1 -- 172.21.15.47:0/3599312038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 msgr2=0x55d3b578f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.735+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/3599312038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b578f930 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc210098870 tx=0x7fc2100d3190 comp rx=0 tx=0).stop 2024-03-21T11:38:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.736+0000 7fc22f9fd740 1 -- 172.21.15.47:0/3599312038 shutdown_connections 2024-03-21T11:38:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.736+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/3599312038 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3b579b260 0x55d3b579d6c0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.736+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/3599312038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 0x55d3b5796900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.742 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.736+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/3599312038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b578f930 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.736+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/3599312038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b578f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.736+0000 7fc20be00640 0 client.30661 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.736+0000 7fc22f9fd740 1 -- 172.21.15.47:0/3599312038 >> 172.21.15.47:0/3599312038 conn(0x55d3b5780750 msgr2=0x55d3b577f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.736+0000 7fc22f9fd740 1 -- 172.21.15.47:0/3599312038 shutdown_connections 2024-03-21T11:38:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.736+0000 7fc22f9fd740 1 -- 172.21.15.47:0/3599312038 wait complete. 2024-03-21T11:38:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.736+0000 7fc22f9fd740 20 client.30661 trim_cache size 0 max 16384 2024-03-21T11:38:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.756+0000 7fc22f9fd740 1 Processor -- start 2024-03-21T11:38:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.756+0000 7fc22f9fd740 1 -- start start 2024-03-21T11:38:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.756+0000 7fc22f9fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b57851e0 0x55d3b578f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.756+0000 7fc22f9fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b5785bd0 0x55d3b57968c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.756+0000 7fc22f9fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b578ad40 0x55d3b578f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.756+0000 7fc22f9fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2281a7e50 con 0x55d3b57851e0 2024-03-21T11:38:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.756+0000 7fc22f9fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2281a9950 con 0x55d3b5785bd0 2024-03-21T11:38:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.756+0000 7fc22f9fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2281a84b0 con 0x55d3b578ad40 2024-03-21T11:38:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.757+0000 7fc21fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b5785bd0 0x55d3b57968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.757+0000 7fc226800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b578ad40 0x55d3b578f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.757+0000 7fc21fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b5785bd0 0x55d3b57968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55844/0 (socket says 172.21.15.47:55844) 2024-03-21T11:38:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.757+0000 7fc21fe00640 1 -- 172.21.15.47:0/1759587649 learned_addr learned my addr 172.21.15.47:0/1759587649 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.757+0000 7fc225e00640 1 --2- 172.21.15.47:0/1759587649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b57851e0 0x55d3b578f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.757+0000 7fc21fe00640 1 -- 172.21.15.47:0/1759587649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b578ad40 msgr2=0x55d3b578f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.757+0000 7fc21fe00640 1 --2- 172.21.15.47:0/1759587649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b578ad40 0x55d3b578f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.757+0000 7fc21fe00640 1 -- 172.21.15.47:0/1759587649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b57851e0 msgr2=0x55d3b578f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.757+0000 7fc21fe00640 1 --2- 172.21.15.47:0/1759587649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b57851e0 0x55d3b578f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.757+0000 7fc21fe00640 1 -- 172.21.15.47:0/1759587649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3b57869e0 con 0x55d3b5785bd0 2024-03-21T11:38:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.757+0000 7fc21fe00640 1 --2- 172.21.15.47:0/1759587649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b5785bd0 0x55d3b57968c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc2100d3200 tx=0x7fc2100057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=412842336f84852c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.758+0000 7fc216a00640 1 -- 172.21.15.47:0/1759587649 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21009b800 con 0x55d3b5785bd0 2024-03-21T11:38:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.758+0000 7fc216a00640 1 -- 172.21.15.47:0/1759587649 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc21009bea0 con 0x55d3b5785bd0 2024-03-21T11:38:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.758+0000 7fc216a00640 1 -- 172.21.15.47:0/1759587649 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21009a1a0 con 0x55d3b5785bd0 2024-03-21T11:38:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.758+0000 7fc22f9fd740 1 -- 172.21.15.47:0/1759587649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b5785bd0 msgr2=0x55d3b57968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.758+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/1759587649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b5785bd0 0x55d3b57968c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc2100d3200 tx=0x7fc2100057b0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.758+0000 7fc22f9fd740 1 -- 172.21.15.47:0/1759587649 shutdown_connections 2024-03-21T11:38:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.758+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/1759587649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b578ad40 0x55d3b578f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.759+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/1759587649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b5785bd0 0x55d3b57968c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.759+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/1759587649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b57851e0 0x55d3b578f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.759+0000 7fc22f9fd740 1 -- 172.21.15.47:0/1759587649 >> 172.21.15.47:0/1759587649 conn(0x55d3b5780750 msgr2=0x55d3b5786ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.759+0000 7fc22f9fd740 1 -- 172.21.15.47:0/1759587649 shutdown_connections 2024-03-21T11:38:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.759+0000 7fc22f9fd740 1 -- 172.21.15.47:0/1759587649 wait complete. 2024-03-21T11:38:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.759+0000 7fc22f9fd740 1 -- start start 2024-03-21T11:38:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.759+0000 7fc22f9fd740 1 Processor -- start 2024-03-21T11:38:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc22f9fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b57945d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc22f9fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b5794b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc22f9fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 0x55d3b5795a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc22f9fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2281a9230 con 0x55d3b578ad40 2024-03-21T11:38:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc22f9fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2281a8bd0 con 0x55d3b57851e0 2024-03-21T11:38:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc22f9fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2281a6010 con 0x55d3b5785bd0 2024-03-21T11:38:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc22f9fd740 10 client.0 _ll_register_callbacks cb 0x7fc2100d3290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:38:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc225e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b57945d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc225e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b57945d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55860/0 (socket says 172.21.15.47:55860) 2024-03-21T11:38:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc21fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b5794b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc225e00640 1 -- 172.21.15.47:0/4073761062 learned_addr learned my addr 172.21.15.47:0/4073761062 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:38:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc226800640 1 --2- 172.21.15.47:0/4073761062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 0x55d3b5795a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc225e00640 1 -- 172.21.15.47:0/4073761062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 msgr2=0x55d3b5794b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc225e00640 1 --2- 172.21.15.47:0/4073761062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b5794b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc225e00640 1 -- 172.21.15.47:0/4073761062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 msgr2=0x55d3b5795a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:38:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc225e00640 1 --2- 172.21.15.47:0/4073761062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 0x55d3b5795a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc225e00640 1 -- 172.21.15.47:0/4073761062 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3b5796070 con 0x55d3b57851e0 2024-03-21T11:38:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.760+0000 7fc226800640 1 --2- 172.21.15.47:0/4073761062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 0x55d3b5795a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:38:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.761+0000 7fc225e00640 1 --2- 172.21.15.47:0/4073761062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b57945d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc21801c040 tx=0x7fc2180096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23e3f43b05df722b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:38:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.761+0000 7fc20be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:38:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.761+0000 7fc20be00640 1 -- 172.21.15.47:0/4073761062 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218009c60 con 0x55d3b57851e0 2024-03-21T11:38:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.761+0000 7fc20be00640 1 -- 172.21.15.47:0/4073761062 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc218005250 con 0x55d3b57851e0 2024-03-21T11:38:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.761+0000 7fc22f9fd740 1 -- 172.21.15.47:0/4073761062 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc21009bea0 con 0x55d3b57851e0 2024-03-21T11:38:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.763+0000 7fc20be00640 1 -- 172.21.15.47:0/4073761062 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc218005450 con 0x55d3b57851e0 2024-03-21T11:38:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.763+0000 7fc22f9fd740 10 client.30656 fetch_fsmap learned FSMap version 34 2024-03-21T11:38:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.763+0000 7fc22f9fd740 1 -- 172.21.15.47:0/4073761062 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d3b57f70d0 con 0x55d3b57851e0 2024-03-21T11:38:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.764+0000 7fc20be00640 1 -- 172.21.15.47:0/4073761062 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc21009bea0 con 0x55d3b57851e0 2024-03-21T11:38:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.764+0000 7fc20be00640 1 -- 172.21.15.47:0/4073761062 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fc21800f620 con 0x55d3b57851e0 2024-03-21T11:38:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.764+0000 7fc22f9fd740 10 client.30656 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:38:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.764+0000 7fc22f9fd740 10 client.30656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:38:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.764+0000 7fc22f9fd740 20 client.30656 populate_metadata read hostname 'smithi047' 2024-03-21T11:38:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.764+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/4073761062 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3b57f7890 0x55d3b57f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:38:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.764+0000 7fc22f9fd740 4 client.30656 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:38:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.764+0000 7fc22f9fd740 1 -- 172.21.15.47:0/4073761062 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d3b56f20c0 con 0x55d3b57f7890 2024-03-21T11:38:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.764+0000 7fc21fe00640 1 --2- 172.21.15.47:0/4073761062 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3b57f7890 0x55d3b57f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:38:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.765+0000 7fc21fe00640 1 --2- 172.21.15.47:0/4073761062 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3b57f7890 0x55d3b57f9cb0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17572724583957a5 server_cookie=78b85f0bbdf0618b in_seq=0 out_seq=0 2024-03-21T11:38:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:51.765+0000 7fc20be00640 10 client.30656 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:51.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 47763 28.96 MB/sec warmup 95 sec latency 141.523 ms 2024-03-21T11:38:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.428+0000 7fc20be00640 1 -- 172.21.15.47:0/4073761062 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d3b56f20c0 con 0x55d3b57f7890 2024-03-21T11:38:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.428+0000 7fc20be00640 10 client.30656 handle_command_reply: tid=0 2024-03-21T11:38:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.428+0000 7fc22f9fd740 1 client.30656 shutdown 2024-03-21T11:38:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.429+0000 7fc22f9fd740 1 -- 172.21.15.47:0/4073761062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 msgr2=0x55d3b57945d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.429+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/4073761062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b57945d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc21801c040 tx=0x7fc2180096c0 comp rx=0 tx=0).stop 2024-03-21T11:38:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.431+0000 7fc22f9fd740 1 -- 172.21.15.47:0/4073761062 shutdown_connections 2024-03-21T11:38:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.431+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/4073761062 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3b57f7890 0x55d3b57f9cb0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.431+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/4073761062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3b578ad40 0x55d3b5795a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.431+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/4073761062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3b5785bd0 0x55d3b5794b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.431+0000 7fc22f9fd740 1 --2- 172.21.15.47:0/4073761062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3b57851e0 0x55d3b57945d0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.431+0000 7fc22f9fd740 1 -- 172.21.15.47:0/4073761062 >> 172.21.15.47:0/4073761062 conn(0x55d3b5780750 msgr2=0x55d3b5786ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.431+0000 7fc20be00640 0 client.30656 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:38:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.432+0000 7fc22f9fd740 1 -- 172.21.15.47:0/4073761062 shutdown_connections 2024-03-21T11:38:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.433+0000 7fc22f9fd740 1 -- 172.21.15.47:0/4073761062 wait complete. 2024-03-21T11:38:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.433+0000 7fc22f9fd740 20 client.30656 trim_cache size 0 max 16384 2024-03-21T11:38:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:38:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:38:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:38:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:38:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.437+0000 7fc22d000640 1 -- 172.21.15.47:0/587992518 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc20c0ab7d0 msgr2=0x7fc20c0adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.437+0000 7fc22d000640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc20c0ab7d0 0x7fc20c0adc90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc2200099a0 tx=0x7fc220004a80 comp rx=0 tx=0).stop 2024-03-21T11:38:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.437+0000 7fc22d000640 1 -- 172.21.15.47:0/587992518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc228100910 msgr2=0x7fc228199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:38:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.437+0000 7fc22d000640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc228100910 0x7fc228199fe0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc21000e9c0 tx=0x7fc21000ee80 comp rx=0 tx=0).stop 2024-03-21T11:38:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.438+0000 7fc22d000640 1 -- 172.21.15.47:0/587992518 shutdown_connections 2024-03-21T11:38:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.438+0000 7fc22d000640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc20c0ab7d0 0x7fc20c0adc90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.438+0000 7fc22d000640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2281012b0 0x7fc2281a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.438+0000 7fc22d000640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc228100910 0x7fc228199fe0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.438+0000 7fc22d000640 1 --2- 172.21.15.47:0/587992518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2280fffb0 0x7fc228199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:38:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.438+0000 7fc22d000640 1 -- 172.21.15.47:0/587992518 >> 172.21.15.47:0/587992518 conn(0x7fc2280fbb30 msgr2=0x7fc2280fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:38:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.438+0000 7fc22d000640 1 -- 172.21.15.47:0/587992518 shutdown_connections 2024-03-21T11:38:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:38:52.438+0000 7fc22d000640 1 -- 172.21.15.47:0/587992518 wait complete. 2024-03-21T11:38:52.927 INFO:tasks.workunit.client.0.smithi047.stdout: 10 48075 28.78 MB/sec warmup 96 sec latency 372.195 ms 2024-03-21T11:38:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:53 smithi047 ceph-mon[31351]: pgmap v866: 129 pgs: 129 active+clean; 316 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.0 MiB/s wr, 532 op/s 2024-03-21T11:38:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:53 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:38:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:53 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:38:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:53 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:38:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:53 smithi100 ceph-mon[36245]: pgmap v866: 129 pgs: 129 active+clean; 316 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.0 MiB/s wr, 532 op/s 2024-03-21T11:38:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:53 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:38:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:53 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:38:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:53 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:38:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:53 smithi102 ceph-mon[36748]: pgmap v866: 129 pgs: 129 active+clean; 316 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.0 MiB/s wr, 532 op/s 2024-03-21T11:38:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:53 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:38:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:53 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:38:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:53 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:38:53.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 48539 28.76 MB/sec warmup 97 sec latency 102.863 ms 2024-03-21T11:38:54.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:54 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:38:54.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:54 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:38:54.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:54 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:38:54.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:54 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:38:54.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:54 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:38:54.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:54 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:38:54.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 49182 28.81 MB/sec warmup 98 sec latency 40.018 ms 2024-03-21T11:38:55.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:55 smithi100 ceph-mon[36245]: pgmap v867: 129 pgs: 129 active+clean; 322 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.9 MiB/s wr, 668 op/s 2024-03-21T11:38:55.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:55 smithi102 ceph-mon[36748]: pgmap v867: 129 pgs: 129 active+clean; 322 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.9 MiB/s wr, 668 op/s 2024-03-21T11:38:55.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:55 smithi047 ceph-mon[31351]: pgmap v867: 129 pgs: 129 active+clean; 322 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.9 MiB/s wr, 668 op/s 2024-03-21T11:38:55.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 49653 28.76 MB/sec warmup 99 sec latency 61.757 ms 2024-03-21T11:38:56.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 50286 28.82 MB/sec warmup 100 sec latency 72.970 ms 2024-03-21T11:38:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:57 smithi100 ceph-mon[36245]: pgmap v868: 129 pgs: 129 active+clean; 319 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.1 MiB/s wr, 534 op/s 2024-03-21T11:38:57.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:57 smithi102 ceph-mon[36748]: pgmap v868: 129 pgs: 129 active+clean; 319 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.1 MiB/s wr, 534 op/s 2024-03-21T11:38:57.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:57 smithi047 ceph-mon[31351]: pgmap v868: 129 pgs: 129 active+clean; 319 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.1 MiB/s wr, 534 op/s 2024-03-21T11:38:57.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 50865 28.84 MB/sec warmup 101 sec latency 76.693 ms 2024-03-21T11:38:58.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 51417 28.90 MB/sec warmup 102 sec latency 153.033 ms 2024-03-21T11:38:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:38:59 smithi100 ceph-mon[36245]: pgmap v869: 129 pgs: 129 active+clean; 318 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.9 MiB/s wr, 547 op/s 2024-03-21T11:38:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:38:59 smithi102 ceph-mon[36748]: pgmap v869: 129 pgs: 129 active+clean; 318 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.9 MiB/s wr, 547 op/s 2024-03-21T11:38:59.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:38:59 smithi047 ceph-mon[31351]: pgmap v869: 129 pgs: 129 active+clean; 318 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.9 MiB/s wr, 547 op/s 2024-03-21T11:38:59.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 51605 28.73 MB/sec warmup 103 sec latency 419.958 ms 2024-03-21T11:39:00.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 51973 28.57 MB/sec warmup 104 sec latency 90.675 ms 2024-03-21T11:39:01.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:01 smithi047 ceph-mon[31351]: pgmap v870: 129 pgs: 129 active+clean; 328 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.7 MiB/s wr, 630 op/s 2024-03-21T11:39:01.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:01 smithi100 ceph-mon[36245]: pgmap v870: 129 pgs: 129 active+clean; 328 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.7 MiB/s wr, 630 op/s 2024-03-21T11:39:01.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:01 smithi102 ceph-mon[36748]: pgmap v870: 129 pgs: 129 active+clean; 328 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.7 MiB/s wr, 630 op/s 2024-03-21T11:39:01.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 52412 28.51 MB/sec warmup 105 sec latency 300.249 ms 2024-03-21T11:39:02.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 53104 28.62 MB/sec warmup 106 sec latency 63.968 ms 2024-03-21T11:39:03.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:03 smithi100 ceph-mon[36245]: pgmap v871: 129 pgs: 129 active+clean; 334 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.9 MiB/s wr, 572 op/s 2024-03-21T11:39:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:03 smithi102 ceph-mon[36748]: pgmap v871: 129 pgs: 129 active+clean; 334 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.9 MiB/s wr, 572 op/s 2024-03-21T11:39:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:03 smithi047 ceph-mon[31351]: pgmap v871: 129 pgs: 129 active+clean; 334 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.9 MiB/s wr, 572 op/s 2024-03-21T11:39:03.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 53763 28.73 MB/sec warmup 107 sec latency 40.548 ms 2024-03-21T11:39:04.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 54118 28.65 MB/sec warmup 108 sec latency 296.984 ms 2024-03-21T11:39:05.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:05 smithi100 ceph-mon[36245]: pgmap v872: 129 pgs: 129 active+clean; 340 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.9 MiB/s wr, 711 op/s 2024-03-21T11:39:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:05 smithi102 ceph-mon[36748]: pgmap v872: 129 pgs: 129 active+clean; 340 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.9 MiB/s wr, 711 op/s 2024-03-21T11:39:05.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:05 smithi047 ceph-mon[31351]: pgmap v872: 129 pgs: 129 active+clean; 340 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.9 MiB/s wr, 711 op/s 2024-03-21T11:39:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.663+0000 7f7fa4c00640 1 -- 172.21.15.47:0/1507207910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fa0104a90 msgr2=0x7f7fa0106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.663+0000 7f7fa4c00640 1 --2- 172.21.15.47:0/1507207910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fa0104a90 0x7f7fa0106e80 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009900 tx=0x7f7f8802bc30 comp rx=0 tx=0).stop 2024-03-21T11:39:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.663+0000 7f7fa4c00640 1 -- 172.21.15.47:0/1507207910 shutdown_connections 2024-03-21T11:39:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.664+0000 7f7fa4c00640 1 --2- 172.21.15.47:0/1507207910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fa01073c0 0x7f7fa0109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.664+0000 7f7fa4c00640 1 --2- 172.21.15.47:0/1507207910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fa0104a90 0x7f7fa0106e80 secure :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009900 tx=0x7f7f8802bc30 comp rx=0 tx=0).stop 2024-03-21T11:39:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.664+0000 7f7fa4c00640 1 --2- 172.21.15.47:0/1507207910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa0102160 0x7f7fa0104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.664+0000 7f7fa4c00640 1 -- 172.21.15.47:0/1507207910 >> 172.21.15.47:0/1507207910 conn(0x7f7fa00fbb50 msgr2=0x7f7fa00fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.665+0000 7f7fa4c00640 1 -- 172.21.15.47:0/1507207910 shutdown_connections 2024-03-21T11:39:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.665+0000 7f7fa4c00640 1 -- 172.21.15.47:0/1507207910 wait complete. 2024-03-21T11:39:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.667+0000 7f7fa4c00640 1 Processor -- start 2024-03-21T11:39:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.667+0000 7f7fa4c00640 1 -- start start 2024-03-21T11:39:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.668+0000 7f7fa4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa0102160 0x7f7fa0195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.668+0000 7f7fa4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fa01073c0 0x7f7fa0195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.668+0000 7f7fa4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fa019af50 0x7f7fa019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.668+0000 7f7fa4c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa010d3c0 con 0x7f7fa019af50 2024-03-21T11:39:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.668+0000 7f7fa4c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa010d240 con 0x7f7fa0102160 2024-03-21T11:39:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.668+0000 7f7fa4c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa010d540 con 0x7f7fa01073c0 2024-03-21T11:39:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.668+0000 7f7f9d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fa01073c0 0x7f7fa0195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.668+0000 7f7f9d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fa01073c0 0x7f7fa0195f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55918/0 (socket says 172.21.15.47:55918) 2024-03-21T11:39:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.668+0000 7f7f9d400640 1 -- 172.21.15.47:0/4204188993 learned_addr learned my addr 172.21.15.47:0/4204188993 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.668+0000 7f7f9de00640 1 --2- 172.21.15.47:0/4204188993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa0102160 0x7f7fa0195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.669+0000 7f7f9d400640 1 -- 172.21.15.47:0/4204188993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa0102160 msgr2=0x7f7fa0195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.669+0000 7f7f9d400640 1 --2- 172.21.15.47:0/4204188993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa0102160 0x7f7fa0195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.669+0000 7f7f9d400640 1 -- 172.21.15.47:0/4204188993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fa019af50 msgr2=0x7f7fa019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.669+0000 7f7f9d400640 1 --2- 172.21.15.47:0/4204188993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fa019af50 0x7f7fa019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.669+0000 7f7f9d400640 1 -- 172.21.15.47:0/4204188993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa019d9d0 con 0x7f7fa01073c0 2024-03-21T11:39:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.669+0000 7f7f9d400640 1 --2- 172.21.15.47:0/4204188993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fa01073c0 0x7f7fa0195f40 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f7f88036040 tx=0x7f7f88005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.673+0000 7f7f96a00640 1 -- 172.21.15.47:0/4204188993 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88043020 con 0x7f7fa01073c0 2024-03-21T11:39:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.673+0000 7f7f96a00640 1 -- 172.21.15.47:0/4204188993 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f880054e0 con 0x7f7fa01073c0 2024-03-21T11:39:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.673+0000 7f7f96a00640 1 -- 172.21.15.47:0/4204188993 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88003bf0 con 0x7f7fa01073c0 2024-03-21T11:39:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.674+0000 7f7fa4c00640 1 -- 172.21.15.47:0/4204188993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa019dc60 con 0x7f7fa01073c0 2024-03-21T11:39:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.674+0000 7f7fa4c00640 1 -- 172.21.15.47:0/4204188993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa019e130 con 0x7f7fa01073c0 2024-03-21T11:39:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.683+0000 7f7f96a00640 1 -- 172.21.15.47:0/4204188993 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7f88046020 con 0x7f7fa01073c0 2024-03-21T11:39:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.685+0000 7f7f96a00640 1 --2- 172.21.15.47:0/4204188993 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f840a31b0 0x7f7f840a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.685+0000 7f7f96a00640 1 -- 172.21.15.47:0/4204188993 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7f880f11b0 con 0x7f7fa01073c0 2024-03-21T11:39:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.687+0000 7f7fa79e3740 1 Processor -- start 2024-03-21T11:39:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.688+0000 7f7f9de00640 1 --2- 172.21.15.47:0/4204188993 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f840a31b0 0x7f7f840a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.689+0000 7f7fa79e3740 1 -- start start 2024-03-21T11:39:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.690+0000 7f7fa79e3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77b1e0 0x55e42a77b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.690+0000 7f7fa79e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a77bbd0 0x55e42a7806d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.690+0000 7f7fa79e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a780d40 0x55e42a7831d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.690+0000 7f7f9d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a77bbd0 0x55e42a7806d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.690+0000 7f7f9d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a77bbd0 0x55e42a7806d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37578/0 (socket says 172.21.15.47:37578) 2024-03-21T11:39:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.690+0000 7f7f9d400640 1 -- 172.21.15.47:0/1409555194 learned_addr learned my addr 172.21.15.47:0/1409555194 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.690+0000 7f7f9e800640 1 --2- 172.21.15.47:0/1409555194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a780d40 0x55e42a7831d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.691+0000 7f7f9de00640 1 --2- 172.21.15.47:0/4204188993 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f840a31b0 0x7f7f840a5670 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7f98005fd0 tx=0x7f7f98009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.692+0000 7f7f9de00640 1 --2- 172.21.15.47:0/1409555194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77b1e0 0x55e42a77b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.692+0000 7f7fa79e3740 1 -- 172.21.15.47:0/1409555194 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42a73e3b0 con 0x55e42a780d40 2024-03-21T11:39:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.692+0000 7f7fa79e3740 1 -- 172.21.15.47:0/1409555194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42a6e0830 con 0x55e42a77bbd0 2024-03-21T11:39:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.692+0000 7f7fa79e3740 1 -- 172.21.15.47:0/1409555194 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42a6895d0 con 0x55e42a77b1e0 2024-03-21T11:39:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.693+0000 7f7f9d400640 1 -- 172.21.15.47:0/1409555194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77b1e0 msgr2=0x55e42a77b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.693+0000 7f7f9d400640 1 --2- 172.21.15.47:0/1409555194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77b1e0 0x55e42a77b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.693+0000 7f7f9d400640 1 -- 172.21.15.47:0/1409555194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a780d40 msgr2=0x55e42a7831d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.693+0000 7f7f9d400640 1 --2- 172.21.15.47:0/1409555194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a780d40 0x55e42a7831d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.693+0000 7f7f9d400640 1 -- 172.21.15.47:0/1409555194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e42a6a12c0 con 0x55e42a77bbd0 2024-03-21T11:39:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.693+0000 7f7f9e800640 1 --2- 172.21.15.47:0/1409555194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a780d40 0x55e42a7831d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:39:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.694+0000 7f7f9d400640 1 --2- 172.21.15.47:0/1409555194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a77bbd0 0x55e42a7806d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f7f880c59d0 tx=0x7f7f880036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b49afa91831e7bc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.694+0000 7f7f8f000640 1 -- 172.21.15.47:0/1409555194 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880d6070 con 0x55e42a77bbd0 2024-03-21T11:39:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.694+0000 7f7f8f000640 1 -- 172.21.15.47:0/1409555194 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f88005690 con 0x55e42a77bbd0 2024-03-21T11:39:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.694+0000 7f7f8f000640 1 -- 172.21.15.47:0/1409555194 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880043e0 con 0x55e42a77bbd0 2024-03-21T11:39:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.697+0000 7f7fa79e3740 1 -- 172.21.15.47:0/1409555194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a77bbd0 msgr2=0x55e42a7806d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.697+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/1409555194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a77bbd0 0x55e42a7806d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f7f880c59d0 tx=0x7f7f880036f0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.698+0000 7f7fa79e3740 1 -- 172.21.15.47:0/1409555194 shutdown_connections 2024-03-21T11:39:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.698+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/1409555194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a780d40 0x55e42a7831d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.698+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/1409555194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a77bbd0 0x55e42a7806d0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.698+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/1409555194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77b1e0 0x55e42a77b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.698+0000 7f7fa79e3740 1 -- 172.21.15.47:0/1409555194 >> 172.21.15.47:0/1409555194 conn(0x55e42a776750 msgr2=0x55e42a776b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.699+0000 7f7fa79e3740 1 -- 172.21.15.47:0/1409555194 shutdown_connections 2024-03-21T11:39:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.699+0000 7f7fa79e3740 1 -- 172.21.15.47:0/1409555194 wait complete. 2024-03-21T11:39:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.700+0000 7f7fa79e3740 1 -- start start 2024-03-21T11:39:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.700+0000 7f7fa79e3740 1 Processor -- start 2024-03-21T11:39:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.701+0000 7f7fa79e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a77b1e0 0x55e42a7851a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.701+0000 7f7fa79e3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77bbd0 0x55e42a7856e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.701+0000 7f7fa79e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a780d40 0x55e42a78c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.701+0000 7f7fa79e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42a69ca20 con 0x55e42a77b1e0 2024-03-21T11:39:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.701+0000 7f7fa79e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42a6dc8d0 con 0x55e42a780d40 2024-03-21T11:39:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.701+0000 7f7fa79e3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42a783ab0 con 0x55e42a77bbd0 2024-03-21T11:39:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.701+0000 7f7fa79e3740 10 client.0 _ll_register_callbacks cb 0x55e42a6d91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.701+0000 7f7f9d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77bbd0 0x55e42a7856e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.701+0000 7f7f9d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77bbd0 0x55e42a7856e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55938/0 (socket says 172.21.15.47:55938) 2024-03-21T11:39:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.701+0000 7f7f9d400640 1 -- 172.21.15.47:0/2057287933 learned_addr learned my addr 172.21.15.47:0/2057287933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.702+0000 7f7f9e800640 1 --2- 172.21.15.47:0/2057287933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a780d40 0x55e42a78c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.702+0000 7f7f9d400640 1 -- 172.21.15.47:0/2057287933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a780d40 msgr2=0x55e42a78c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.702+0000 7f7f9d400640 1 --2- 172.21.15.47:0/2057287933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a780d40 0x55e42a78c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.702+0000 7f7f9d400640 1 -- 172.21.15.47:0/2057287933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a77b1e0 msgr2=0x55e42a7851a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.702+0000 7f7f9d400640 1 --2- 172.21.15.47:0/2057287933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a77b1e0 0x55e42a7851a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.702+0000 7f7f9d400640 1 -- 172.21.15.47:0/2057287933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e42a3f5210 con 0x55e42a77bbd0 2024-03-21T11:39:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.704+0000 7f7f9d400640 1 --2- 172.21.15.47:0/2057287933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77bbd0 0x55e42a7856e0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7f8802c7b0 tx=0x7f7f880ee8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9abae9fe5ff157b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.704+0000 7f7f83400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.705+0000 7f7f83400640 1 -- 172.21.15.47:0/2057287933 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880d6040 con 0x55e42a77bbd0 2024-03-21T11:39:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.705+0000 7f7f83400640 1 -- 172.21.15.47:0/2057287933 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7f880c5a70 con 0x55e42a77bbd0 2024-03-21T11:39:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.705+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2057287933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e42a6edd70 con 0x55e42a77bbd0 2024-03-21T11:39:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.705+0000 7f7f83400640 1 -- 172.21.15.47:0/2057287933 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f880dc020 con 0x55e42a77bbd0 2024-03-21T11:39:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.705+0000 7f7fa79e3740 10 client.30679 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.706+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2057287933 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e42a4c1620 con 0x55e42a77bbd0 2024-03-21T11:39:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.706+0000 7f7f83400640 1 -- 172.21.15.47:0/2057287933 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e42a6edd70 con 0x55e42a77bbd0 2024-03-21T11:39:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.706+0000 7f7f83400640 1 -- 172.21.15.47:0/2057287933 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f7f880d1b40 con 0x55e42a77bbd0 2024-03-21T11:39:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.707+0000 7f7fa79e3740 10 client.30679 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.707+0000 7f7fa79e3740 10 client.30679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.707+0000 7f7fa79e3740 20 client.30679 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.707+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/2057287933 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e42a790e30 0x55e42a793220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.707+0000 7f7fa79e3740 4 client.30679 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.707+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2057287933 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e42a6e80c0 con 0x55e42a790e30 2024-03-21T11:39:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.709+0000 7f7f9de00640 1 --2- 172.21.15.47:0/2057287933 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e42a790e30 0x55e42a793220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.710+0000 7f7f9de00640 1 --2- 172.21.15.47:0/2057287933 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e42a790e30 0x55e42a793220 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a3ec35854572fd server_cookie=72bb532f05b8d684 in_seq=0 out_seq=0 2024-03-21T11:39:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.710+0000 7f7f83400640 10 client.30679 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.713+0000 7f7f83400640 1 -- 172.21.15.47:0/2057287933 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e42a6e80c0 con 0x55e42a790e30 2024-03-21T11:39:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.713+0000 7f7f83400640 10 client.30679 handle_command_reply: tid=0 2024-03-21T11:39:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.713+0000 7f7fa79e3740 1 client.30679 shutdown 2024-03-21T11:39:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.714+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2057287933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77bbd0 msgr2=0x55e42a7856e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.714+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/2057287933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77bbd0 0x55e42a7856e0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7f8802c7b0 tx=0x7f7f880ee8c0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.714+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2057287933 shutdown_connections 2024-03-21T11:39:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.714+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/2057287933 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e42a790e30 0x55e42a793220 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.714+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/2057287933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a780d40 0x55e42a78c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.714+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/2057287933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77bbd0 0x55e42a7856e0 secure :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7f8802c7b0 tx=0x7f7f880ee8c0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.714+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/2057287933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a77b1e0 0x55e42a7851a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.714+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2057287933 >> 172.21.15.47:0/2057287933 conn(0x55e42a776750 msgr2=0x55e42a775950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.714+0000 7f7f83400640 0 client.30679 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.715+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2057287933 shutdown_connections 2024-03-21T11:39:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.716+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2057287933 wait complete. 2024-03-21T11:39:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.716+0000 7f7fa79e3740 20 client.30679 trim_cache size 0 max 16384 2024-03-21T11:39:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.748+0000 7f7fa79e3740 1 Processor -- start 2024-03-21T11:39:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.748+0000 7f7fa79e3740 1 -- start start 2024-03-21T11:39:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.749+0000 7f7fa79e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a77b1e0 0x55e42a77cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.749+0000 7f7fa79e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a780d40 0x55e42a78c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.749+0000 7f7fa79e3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a771800 0x55e42a77d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.749+0000 7f7fa79e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01b9590 con 0x55e42a77b1e0 2024-03-21T11:39:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.749+0000 7f7fa79e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01bb090 con 0x55e42a780d40 2024-03-21T11:39:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.749+0000 7f7fa79e3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01b9bf0 con 0x55e42a771800 2024-03-21T11:39:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.750+0000 7f7f9e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a771800 0x55e42a77d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.750+0000 7f7f9de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a77b1e0 0x55e42a77cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.750+0000 7f7f9d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a780d40 0x55e42a78c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.750+0000 7f7f9de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a77b1e0 0x55e42a77cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59546/0 (socket says 172.21.15.47:59546) 2024-03-21T11:39:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.750+0000 7f7f9de00640 1 -- 172.21.15.47:0/371904830 learned_addr learned my addr 172.21.15.47:0/371904830 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.751+0000 7f7f9d400640 1 -- 172.21.15.47:0/371904830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a771800 msgr2=0x55e42a77d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.751+0000 7f7f9d400640 1 --2- 172.21.15.47:0/371904830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a771800 0x55e42a77d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.751+0000 7f7f9d400640 1 -- 172.21.15.47:0/371904830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a77b1e0 msgr2=0x55e42a77cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.751+0000 7f7f9d400640 1 --2- 172.21.15.47:0/371904830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a77b1e0 0x55e42a77cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.751+0000 7f7f9d400640 1 -- 172.21.15.47:0/371904830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e42a790c10 con 0x55e42a780d40 2024-03-21T11:39:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.751+0000 7f7f9e800640 1 --2- 172.21.15.47:0/371904830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a771800 0x55e42a77d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:39:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.751+0000 7f7f9de00640 1 --2- 172.21.15.47:0/371904830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a77b1e0 0x55e42a77cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.752+0000 7f7f9d400640 1 --2- 172.21.15.47:0/371904830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a780d40 0x55e42a78c6e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7f88001eb0 tx=0x7f7f880ee940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e55dfe6d66610fea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.752+0000 7f7f8e600640 1 -- 172.21.15.47:0/371904830 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880bdbf0 con 0x55e42a780d40 2024-03-21T11:39:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.752+0000 7f7f8e600640 1 -- 172.21.15.47:0/371904830 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f880bc940 con 0x55e42a780d40 2024-03-21T11:39:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.753+0000 7f7fa79e3740 1 -- 172.21.15.47:0/371904830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a780d40 msgr2=0x55e42a78c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.753+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/371904830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a780d40 0x55e42a78c6e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7f88001eb0 tx=0x7f7f880ee940 comp rx=0 tx=0).stop 2024-03-21T11:39:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.753+0000 7f7fa79e3740 1 -- 172.21.15.47:0/371904830 shutdown_connections 2024-03-21T11:39:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.753+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/371904830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a771800 0x55e42a77d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.753+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/371904830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a780d40 0x55e42a78c6e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.753+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/371904830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a77b1e0 0x55e42a77cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.753+0000 7f7fa79e3740 1 -- 172.21.15.47:0/371904830 >> 172.21.15.47:0/371904830 conn(0x55e42a776750 msgr2=0x55e42a7851a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.754+0000 7f7fa79e3740 1 -- 172.21.15.47:0/371904830 shutdown_connections 2024-03-21T11:39:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.754+0000 7f7fa79e3740 1 -- 172.21.15.47:0/371904830 wait complete. 2024-03-21T11:39:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.755+0000 7f7fa79e3740 1 -- start start 2024-03-21T11:39:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.755+0000 7f7fa79e3740 1 Processor -- start 2024-03-21T11:39:05.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.756+0000 7f7fa79e3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a771800 0x55e42a789af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.756+0000 7f7fa79e3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77b1e0 0x55e42a78a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.757+0000 7f7fa79e3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a780d40 0x55e42a78b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.757+0000 7f7f9de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a771800 0x55e42a789af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.757+0000 7f7f9de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a771800 0x55e42a789af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37602/0 (socket says 172.21.15.47:37602) 2024-03-21T11:39:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.757+0000 7f7fa79e3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01ba970 con 0x55e42a780d40 2024-03-21T11:39:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.757+0000 7f7fa79e3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01ba310 con 0x55e42a771800 2024-03-21T11:39:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.757+0000 7f7fa79e3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01b7750 con 0x55e42a77b1e0 2024-03-21T11:39:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.757+0000 7f7f9de00640 1 -- 172.21.15.47:0/2833743293 learned_addr learned my addr 172.21.15.47:0/2833743293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.757+0000 7f7fa79e3740 10 client.0 _ll_register_callbacks cb 0x7f7f880d4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.757+0000 7f7f9d400640 1 --2- 172.21.15.47:0/2833743293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77b1e0 0x55e42a78a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.758+0000 7f7f9e800640 1 --2- 172.21.15.47:0/2833743293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a780d40 0x55e42a78b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.758+0000 7f7f9d400640 1 -- 172.21.15.47:0/2833743293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a771800 msgr2=0x55e42a789af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.758+0000 7f7f9d400640 1 --2- 172.21.15.47:0/2833743293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a771800 0x55e42a789af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.758+0000 7f7f9d400640 1 -- 172.21.15.47:0/2833743293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a780d40 msgr2=0x55e42a78b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.758+0000 7f7f9d400640 1 --2- 172.21.15.47:0/2833743293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a780d40 0x55e42a78b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.758+0000 7f7f9d400640 1 -- 172.21.15.47:0/2833743293 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e42a78b630 con 0x55e42a77b1e0 2024-03-21T11:39:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.759+0000 7f7f9d400640 1 --2- 172.21.15.47:0/2833743293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77b1e0 0x55e42a78a030 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7f880d1f10 tx=0x7f7f880c1380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b04228e8ce1981d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.761+0000 7f7f83400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.761+0000 7f7f83400640 1 -- 172.21.15.47:0/2833743293 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880d6070 con 0x55e42a77b1e0 2024-03-21T11:39:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.761+0000 7f7f83400640 1 -- 172.21.15.47:0/2833743293 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7f880b9240 con 0x55e42a77b1e0 2024-03-21T11:39:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.761+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2833743293 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f880bc940 con 0x55e42a77b1e0 2024-03-21T11:39:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.763+0000 7f7f83400640 1 -- 172.21.15.47:0/2833743293 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f880bdbf0 con 0x55e42a77b1e0 2024-03-21T11:39:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.764+0000 7f7fa79e3740 10 client.30685 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.764+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2833743293 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e42a78b630 con 0x55e42a77b1e0 2024-03-21T11:39:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.765+0000 7f7f83400640 1 -- 172.21.15.47:0/2833743293 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f880bc940 con 0x55e42a77b1e0 2024-03-21T11:39:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.765+0000 7f7f83400640 1 -- 172.21.15.47:0/2833743293 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f7f880c27d0 con 0x55e42a77b1e0 2024-03-21T11:39:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.765+0000 7f7fa79e3740 10 client.30685 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.765+0000 7f7fa79e3740 10 client.30685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.765+0000 7f7fa79e3740 20 client.30685 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.765+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/2833743293 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e42a7ec890 0x55e42a7eecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.765+0000 7f7fa79e3740 4 client.30685 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:39:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.765+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2833743293 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e42a6665d0 con 0x55e42a7ec890 2024-03-21T11:39:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.766+0000 7f7f9de00640 1 --2- 172.21.15.47:0/2833743293 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e42a7ec890 0x55e42a7eecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.767+0000 7f7f9de00640 1 --2- 172.21.15.47:0/2833743293 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e42a7ec890 0x55e42a7eecb0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c19c7f330c5734e server_cookie=bbc26814c177a2db in_seq=0 out_seq=0 2024-03-21T11:39:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:05.767+0000 7f7f83400640 10 client.30685 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:05.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 54130 28.40 MB/sec warmup 109 sec latency 990.338 ms 2024-03-21T11:39:06.928 INFO:tasks.workunit.client.0.smithi047.stdout: 10 54208 28.18 MB/sec warmup 110 sec latency 1620.881 ms 2024-03-21T11:39:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.982+0000 7f7f83400640 1 -- 172.21.15.47:0/2833743293 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e42a6665d0 con 0x55e42a7ec890 2024-03-21T11:39:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.982+0000 7f7f83400640 10 client.30685 handle_command_reply: tid=0 2024-03-21T11:39:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.982+0000 7f7fa79e3740 1 client.30685 shutdown 2024-03-21T11:39:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.983+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2833743293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77b1e0 msgr2=0x55e42a78a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.983+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/2833743293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77b1e0 0x55e42a78a030 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7f880d1f10 tx=0x7f7f880c1380 comp rx=0 tx=0).stop 2024-03-21T11:39:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.983+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2833743293 shutdown_connections 2024-03-21T11:39:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.983+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/2833743293 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e42a7ec890 0x55e42a7eecb0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.983+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/2833743293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e42a780d40 0x55e42a78b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.983+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/2833743293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e42a77b1e0 0x55e42a78a030 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.983+0000 7f7fa79e3740 1 --2- 172.21.15.47:0/2833743293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e42a771800 0x55e42a789af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.983+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2833743293 >> 172.21.15.47:0/2833743293 conn(0x55e42a776750 msgr2=0x55e42a784e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.984+0000 7f7f83400640 0 client.30685 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.984+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2833743293 shutdown_connections 2024-03-21T11:39:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.984+0000 7f7fa79e3740 1 -- 172.21.15.47:0/2833743293 wait complete. 2024-03-21T11:39:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.985+0000 7f7fa79e3740 20 client.30685 trim_cache size 0 max 16384 2024-03-21T11:39:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:39:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:39:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:39:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:39:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.986+0000 7f7fa4c00640 1 -- 172.21.15.47:0/4204188993 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f840a31b0 msgr2=0x7f7f840a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.986+0000 7f7fa4c00640 1 --2- 172.21.15.47:0/4204188993 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f840a31b0 0x7f7f840a5670 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7f98005fd0 tx=0x7f7f98009500 comp rx=0 tx=0).stop 2024-03-21T11:39:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.986+0000 7f7fa4c00640 1 -- 172.21.15.47:0/4204188993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fa01073c0 msgr2=0x7f7fa0195f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.986+0000 7f7fa4c00640 1 --2- 172.21.15.47:0/4204188993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fa01073c0 0x7f7fa0195f40 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f7f88036040 tx=0x7f7f88005dd0 comp rx=0 tx=0).stop 2024-03-21T11:39:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.986+0000 7f7fa4c00640 1 -- 172.21.15.47:0/4204188993 shutdown_connections 2024-03-21T11:39:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.986+0000 7f7fa4c00640 1 --2- 172.21.15.47:0/4204188993 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f840a31b0 0x7f7f840a5670 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.986+0000 7f7fa4c00640 1 --2- 172.21.15.47:0/4204188993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fa019af50 0x7f7fa019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.986+0000 7f7fa4c00640 1 --2- 172.21.15.47:0/4204188993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fa01073c0 0x7f7fa0195f40 secure :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f7f88036040 tx=0x7f7f88005dd0 comp rx=0 tx=0).stop 2024-03-21T11:39:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.986+0000 7f7fa4c00640 1 --2- 172.21.15.47:0/4204188993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa0102160 0x7f7fa0195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.986+0000 7f7fa4c00640 1 -- 172.21.15.47:0/4204188993 >> 172.21.15.47:0/4204188993 conn(0x7f7fa00fbb50 msgr2=0x7f7fa0101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.987+0000 7f7fa4c00640 1 -- 172.21.15.47:0/4204188993 shutdown_connections 2024-03-21T11:39:06.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:06.987+0000 7f7fa4c00640 1 -- 172.21.15.47:0/4204188993 wait complete. 2024-03-21T11:39:07.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:07 smithi100 ceph-mon[36245]: pgmap v873: 129 pgs: 129 active+clean; 340 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.9 MiB/s wr, 545 op/s 2024-03-21T11:39:07.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:07 smithi102 ceph-mon[36748]: pgmap v873: 129 pgs: 129 active+clean; 340 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.9 MiB/s wr, 545 op/s 2024-03-21T11:39:07.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:07 smithi047 ceph-mon[31351]: pgmap v873: 129 pgs: 129 active+clean; 340 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.9 MiB/s wr, 545 op/s 2024-03-21T11:39:07.929 INFO:tasks.workunit.client.0.smithi047.stdout: 10 54370 28.00 MB/sec warmup 111 sec latency 367.187 ms 2024-03-21T11:39:08.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:39:08.613 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:39:08.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.923+0000 7fd068c00640 1 -- 172.21.15.47:0/1757892731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd064109870 msgr2=0x7fd064110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:08.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.923+0000 7fd068c00640 1 --2- 172.21.15.47:0/1757892731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd064109870 0x7fd064110100 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd050009900 tx=0x7fd05002bc60 comp rx=0 tx=0).stop 2024-03-21T11:39:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.924+0000 7fd068c00640 1 -- 172.21.15.47:0/1757892731 shutdown_connections 2024-03-21T11:39:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.924+0000 7fd068c00640 1 --2- 172.21.15.47:0/1757892731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd064109870 0x7fd064110100 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.924+0000 7fd068c00640 1 --2- 172.21.15.47:0/1757892731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd064108eb0 0x7fd064109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.924+0000 7fd068c00640 1 --2- 172.21.15.47:0/1757892731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd064107cc0 0x7fd0641080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.924+0000 7fd068c00640 1 -- 172.21.15.47:0/1757892731 >> 172.21.15.47:0/1757892731 conn(0x7fd064076c60 msgr2=0x7fd0640790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.926+0000 7fd068c00640 1 -- 172.21.15.47:0/1757892731 shutdown_connections 2024-03-21T11:39:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.926+0000 7fd068c00640 1 -- 172.21.15.47:0/1757892731 wait complete. 2024-03-21T11:39:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.928+0000 7fd068c00640 1 Processor -- start 2024-03-21T11:39:08.929 INFO:tasks.workunit.client.0.smithi047.stdout: 10 54604 27.84 MB/sec warmup 112 sec latency 95.337 ms 2024-03-21T11:39:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.928+0000 7fd068c00640 1 -- start start 2024-03-21T11:39:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.928+0000 7fd068c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd064107cc0 0x7fd06419dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.928+0000 7fd068c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd064108eb0 0x7fd06419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.928+0000 7fd068c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd064109870 0x7fd0641a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.928+0000 7fd068c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0641139b0 con 0x7fd064108eb0 2024-03-21T11:39:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.928+0000 7fd068c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd064113830 con 0x7fd064107cc0 2024-03-21T11:39:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.928+0000 7fd068c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd064113b30 con 0x7fd064109870 2024-03-21T11:39:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.929+0000 7fd062800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd064109870 0x7fd0641a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.929+0000 7fd062800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd064109870 0x7fd0641a5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37194/0 (socket says 172.21.15.47:37194) 2024-03-21T11:39:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.929+0000 7fd062800640 1 -- 172.21.15.47:0/1794241178 learned_addr learned my addr 172.21.15.47:0/1794241178 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.929+0000 7fd061e00640 1 --2- 172.21.15.47:0/1794241178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd064107cc0 0x7fd06419dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.929+0000 7fd062800640 1 -- 172.21.15.47:0/1794241178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd064107cc0 msgr2=0x7fd06419dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.929+0000 7fd062800640 1 --2- 172.21.15.47:0/1794241178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd064107cc0 0x7fd06419dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.929+0000 7fd062800640 1 -- 172.21.15.47:0/1794241178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd064108eb0 msgr2=0x7fd06419e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.929+0000 7fd062800640 1 --2- 172.21.15.47:0/1794241178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd064108eb0 0x7fd06419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.929+0000 7fd062800640 1 -- 172.21.15.47:0/1794241178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0641a5bf0 con 0x7fd064109870 2024-03-21T11:39:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.930+0000 7fd062800640 1 --2- 172.21.15.47:0/1794241178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd064109870 0x7fd0641a5580 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd050009850 tx=0x7fd050004ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.930+0000 7fd05aa00640 1 -- 172.21.15.47:0/1794241178 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050043020 con 0x7fd064109870 2024-03-21T11:39:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.931+0000 7fd068c00640 1 -- 172.21.15.47:0/1794241178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0641a5e80 con 0x7fd064109870 2024-03-21T11:39:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.931+0000 7fd05aa00640 1 -- 172.21.15.47:0/1794241178 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0500054e0 con 0x7fd064109870 2024-03-21T11:39:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.931+0000 7fd05aa00640 1 -- 172.21.15.47:0/1794241178 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050003b90 con 0x7fd064109870 2024-03-21T11:39:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.931+0000 7fd068c00640 1 -- 172.21.15.47:0/1794241178 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0641a63a0 con 0x7fd064109870 2024-03-21T11:39:08.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.936+0000 7fd05aa00640 1 -- 172.21.15.47:0/1794241178 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd050046020 con 0x7fd064109870 2024-03-21T11:39:08.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.939+0000 7fd05aa00640 1 --2- 172.21.15.47:0/1794241178 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd0480a2ff0 0x7fd0480a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:08.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.939+0000 7fd06b963740 1 Processor -- start 2024-03-21T11:39:08.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.939+0000 7fd05aa00640 1 -- 172.21.15.47:0/1794241178 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd0500f1a90 con 0x7fd064109870 2024-03-21T11:39:08.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.939+0000 7fd06b963740 1 -- start start 2024-03-21T11:39:08.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.940+0000 7fd06b963740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c77e90 0x5653a4c782b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:08.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.940+0000 7fd061e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c77e90 0x5653a4c782b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.940+0000 7fd061e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c77e90 0x5653a4c782b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35714/0 (socket says 172.21.15.47:35714) 2024-03-21T11:39:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.940+0000 7fd06b963740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c78880 0x5653a4c7d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.940+0000 7fd061e00640 1 -- 172.21.15.47:0/721268949 learned_addr learned my addr 172.21.15.47:0/721268949 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.941+0000 7fd06b963740 1 --2- 172.21.15.47:0/721268949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c7fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.941+0000 7fd06b963740 1 -- 172.21.15.47:0/721268949 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a4bcc250 con 0x5653a4c78880 2024-03-21T11:39:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.941+0000 7fd06b963740 1 -- 172.21.15.47:0/721268949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a4b74a30 con 0x5653a4c77e90 2024-03-21T11:39:08.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.941+0000 7fd06b963740 1 -- 172.21.15.47:0/721268949 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a4bc82f0 con 0x5653a4c7d9f0 2024-03-21T11:39:08.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.941+0000 7fd062800640 1 --2- 172.21.15.47:0/721268949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c7fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:08.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.942+0000 7fd061e00640 1 --2- 172.21.15.47:0/1794241178 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd0480a2ff0 0x7fd0480a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:08.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.942+0000 7fd062800640 1 -- 172.21.15.47:0/721268949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c77e90 msgr2=0x5653a4c782b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:08.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.942+0000 7fd062800640 1 --2- 172.21.15.47:0/721268949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c77e90 0x5653a4c782b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.942+0000 7fd062800640 1 -- 172.21.15.47:0/721268949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c78880 msgr2=0x5653a4c7d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:08.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.942+0000 7fd062800640 1 --2- 172.21.15.47:0/721268949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c78880 0x5653a4c7d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.942+0000 7fd062800640 1 -- 172.21.15.47:0/721268949 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653a4b5f690 con 0x5653a4c7d9f0 2024-03-21T11:39:08.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.943+0000 7fd062800640 1 --2- 172.21.15.47:0/721268949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c7fe80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd0500bdad0 tx=0x7fd050009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d9b8fb0c9809331 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:08.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.943+0000 7fd061e00640 1 --2- 172.21.15.47:0/1794241178 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd0480a2ff0 0x7fd0480a54b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00be40 tx=0x7fd05c009ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:08.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.944+0000 7fd04f000640 1 -- 172.21.15.47:0/721268949 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0500be4d0 con 0x5653a4c7d9f0 2024-03-21T11:39:08.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.944+0000 7fd04f000640 1 -- 172.21.15.47:0/721268949 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0500bddd0 con 0x5653a4c7d9f0 2024-03-21T11:39:08.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.944+0000 7fd04f000640 1 -- 172.21.15.47:0/721268949 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0500ba440 con 0x5653a4c7d9f0 2024-03-21T11:39:08.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.945+0000 7fd06b963740 1 -- 172.21.15.47:0/721268949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 msgr2=0x5653a4c7fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:08.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.945+0000 7fd06b963740 1 --2- 172.21.15.47:0/721268949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c7fe80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd0500bdad0 tx=0x7fd050009400 comp rx=0 tx=0).stop 2024-03-21T11:39:08.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.946+0000 7fd06b963740 1 -- 172.21.15.47:0/721268949 shutdown_connections 2024-03-21T11:39:08.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.946+0000 7fd06b963740 1 --2- 172.21.15.47:0/721268949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c7fe80 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.946+0000 7fd06b963740 1 --2- 172.21.15.47:0/721268949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c78880 0x5653a4c7d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.946+0000 7fd06b963740 1 --2- 172.21.15.47:0/721268949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c77e90 0x5653a4c782b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.946+0000 7fd06b963740 1 -- 172.21.15.47:0/721268949 >> 172.21.15.47:0/721268949 conn(0x5653a4c2f200 msgr2=0x5653a4c627f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:08.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.947+0000 7fd06b963740 1 -- 172.21.15.47:0/721268949 shutdown_connections 2024-03-21T11:39:08.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.947+0000 7fd06b963740 1 -- 172.21.15.47:0/721268949 wait complete. 2024-03-21T11:39:08.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.947+0000 7fd06b963740 1 -- start start 2024-03-21T11:39:08.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.948+0000 7fd06b963740 1 Processor -- start 2024-03-21T11:39:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.948+0000 7fd06b963740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c77e90 0x5653a4c81b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.948+0000 7fd06b963740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c78880 0x5653a4c82090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.948+0000 7fd06b963740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c890d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.948+0000 7fd06b963740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a4bc82f0 con 0x5653a4c77e90 2024-03-21T11:39:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.948+0000 7fd06b963740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a4b87e80 con 0x5653a4c78880 2024-03-21T11:39:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.948+0000 7fd06b963740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a4c80760 con 0x5653a4c7d9f0 2024-03-21T11:39:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.949+0000 7fd062800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c890d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.949+0000 7fd062800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c890d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37224/0 (socket says 172.21.15.47:37224) 2024-03-21T11:39:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.949+0000 7fd062800640 1 -- 172.21.15.47:0/1761777081 learned_addr learned my addr 172.21.15.47:0/1761777081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.949+0000 7fd061e00640 1 --2- 172.21.15.47:0/1761777081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c77e90 0x5653a4c81b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.948+0000 7fd06b963740 10 client.0 _ll_register_callbacks cb 0x5653a4b9a130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.950+0000 7fd062800640 1 -- 172.21.15.47:0/1761777081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c78880 msgr2=0x5653a4c82090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.949+0000 7fd061400640 1 --2- 172.21.15.47:0/1761777081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c78880 0x5653a4c82090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.950+0000 7fd062800640 1 --2- 172.21.15.47:0/1761777081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c78880 0x5653a4c82090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.950+0000 7fd062800640 1 -- 172.21.15.47:0/1761777081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c77e90 msgr2=0x5653a4c81b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.950+0000 7fd062800640 1 --2- 172.21.15.47:0/1761777081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c77e90 0x5653a4c81b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.950+0000 7fd062800640 1 -- 172.21.15.47:0/1761777081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653a49aa7d0 con 0x5653a4c7d9f0 2024-03-21T11:39:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.950+0000 7fd061400640 1 --2- 172.21.15.47:0/1761777081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c78880 0x5653a4c82090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.950+0000 7fd062800640 1 --2- 172.21.15.47:0/1761777081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c890d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd0500c62b0 tx=0x7fd050009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a41897e504d8ccb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.950+0000 7fd06b963740 1 -- 172.21.15.47:0/1761777081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0500bddd0 con 0x5653a4c7d9f0 2024-03-21T11:39:08.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.950+0000 7fd043400640 10 client.30697 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:08.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.951+0000 7fd043400640 1 -- 172.21.15.47:0/1761777081 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0500c6550 con 0x5653a4c7d9f0 2024-03-21T11:39:08.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.951+0000 7fd043400640 1 -- 172.21.15.47:0/1761777081 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd0500dcb40 con 0x5653a4c7d9f0 2024-03-21T11:39:08.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.951+0000 7fd043400640 1 -- 172.21.15.47:0/1761777081 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd050089050 con 0x5653a4c7d9f0 2024-03-21T11:39:08.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.951+0000 7fd06b963740 10 client.30697 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:08.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.951+0000 7fd06b963740 1 -- 172.21.15.47:0/1761777081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=34}) v3 -- 0x5653a48e03f0 con 0x5653a4c7d9f0 2024-03-21T11:39:08.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.952+0000 7fd043400640 1 -- 172.21.15.47:0/1761777081 <== mon.1 v2:172.21.15.102:3300/0 4 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fd0500dc020 con 0x5653a4c7d9f0 2024-03-21T11:39:08.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.952+0000 7fd06b963740 10 client.30697 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:08.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.952+0000 7fd06b963740 10 client.30697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:08.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.952+0000 7fd06b963740 20 client.30697 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:08.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.952+0000 7fd06b963740 1 --2- 172.21.15.47:0/1761777081 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653a4c8d4e0 0x5653a4c8f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:08.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.952+0000 7fd06b963740 4 client.30697 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:08.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.952+0000 7fd06b963740 1 -- 172.21.15.47:0/1761777081 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653a4bd9790 con 0x5653a4c8d4e0 2024-03-21T11:39:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.957+0000 7fd061e00640 1 --2- 172.21.15.47:0/1761777081 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653a4c8d4e0 0x5653a4c8f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.959+0000 7fd061e00640 1 --2- 172.21.15.47:0/1761777081 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653a4c8d4e0 0x5653a4c8f8d0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a416b84c8ee3211e server_cookie=5f3f09a0ca2a246 in_seq=0 out_seq=0 2024-03-21T11:39:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.959+0000 7fd043400640 10 client.30697 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:08.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.975+0000 7fd043400640 1 -- 172.21.15.47:0/1761777081 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5653a4bd9790 con 0x5653a4c8d4e0 2024-03-21T11:39:08.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.975+0000 7fd043400640 10 client.30697 handle_command_reply: tid=0 2024-03-21T11:39:08.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.975+0000 7fd06b963740 1 client.30697 shutdown 2024-03-21T11:39:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.976+0000 7fd06b963740 1 -- 172.21.15.47:0/1761777081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 msgr2=0x5653a4c890d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.976+0000 7fd06b963740 1 --2- 172.21.15.47:0/1761777081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c890d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd0500c62b0 tx=0x7fd050009560 comp rx=0 tx=0).stop 2024-03-21T11:39:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.976+0000 7fd06b963740 1 -- 172.21.15.47:0/1761777081 shutdown_connections 2024-03-21T11:39:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.976+0000 7fd06b963740 1 --2- 172.21.15.47:0/1761777081 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653a4c8d4e0 0x5653a4c8f8d0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.976+0000 7fd06b963740 1 --2- 172.21.15.47:0/1761777081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c890d0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.976+0000 7fd06b963740 1 --2- 172.21.15.47:0/1761777081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c78880 0x5653a4c82090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.976+0000 7fd06b963740 1 --2- 172.21.15.47:0/1761777081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c77e90 0x5653a4c81b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.976+0000 7fd06b963740 1 -- 172.21.15.47:0/1761777081 >> 172.21.15.47:0/1761777081 conn(0x5653a4c2f200 msgr2=0x5653a4c61a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.976+0000 7fd043400640 0 client.30697 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.977+0000 7fd06b963740 1 -- 172.21.15.47:0/1761777081 shutdown_connections 2024-03-21T11:39:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.977+0000 7fd06b963740 1 -- 172.21.15.47:0/1761777081 wait complete. 2024-03-21T11:39:08.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:08.978+0000 7fd06b963740 20 client.30697 trim_cache size 0 max 16384 2024-03-21T11:39:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.025+0000 7fd06b963740 1 Processor -- start 2024-03-21T11:39:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.025+0000 7fd06b963740 1 -- start start 2024-03-21T11:39:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.025+0000 7fd06b963740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c77e90 0x5653a4c79a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.025+0000 7fd06b963740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c78880 0x5653a4c89090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.025+0000 7fd06b963740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c61200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.025+0000 7fd06b963740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0641a7f20 con 0x5653a4c78880 2024-03-21T11:39:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.025+0000 7fd06b963740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0641a9a20 con 0x5653a4c77e90 2024-03-21T11:39:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.025+0000 7fd06b963740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0641a8580 con 0x5653a4c7d9f0 2024-03-21T11:39:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.026+0000 7fd061e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c77e90 0x5653a4c79a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:09.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.026+0000 7fd061e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c77e90 0x5653a4c79a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35736/0 (socket says 172.21.15.47:35736) 2024-03-21T11:39:09.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.026+0000 7fd061e00640 1 -- 172.21.15.47:0/218959506 learned_addr learned my addr 172.21.15.47:0/218959506 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:09.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.026+0000 7fd061400640 1 --2- 172.21.15.47:0/218959506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c78880 0x5653a4c89090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:09.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.026+0000 7fd062800640 1 --2- 172.21.15.47:0/218959506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c61200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:09.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.028+0000 7fd061e00640 1 -- 172.21.15.47:0/218959506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 msgr2=0x5653a4c61200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:09.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.028+0000 7fd061e00640 1 --2- 172.21.15.47:0/218959506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c61200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.028+0000 7fd061e00640 1 -- 172.21.15.47:0/218959506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c78880 msgr2=0x5653a4c89090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:09.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.028+0000 7fd061e00640 1 --2- 172.21.15.47:0/218959506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c78880 0x5653a4c89090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.029+0000 7fd061e00640 1 -- 172.21.15.47:0/218959506 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653a48a00f0 con 0x5653a4c77e90 2024-03-21T11:39:09.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.029+0000 7fd061400640 1 --2- 172.21.15.47:0/218959506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c78880 0x5653a4c89090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:09.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.029+0000 7fd061e00640 1 --2- 172.21.15.47:0/218959506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c77e90 0x5653a4c79a30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009fd0 tx=0x7fd05c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e854dbae9508032 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:09.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.030+0000 7fd04e600640 1 -- 172.21.15.47:0/218959506 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c02c070 con 0x5653a4c77e90 2024-03-21T11:39:09.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.030+0000 7fd04e600640 1 -- 172.21.15.47:0/218959506 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd05c026b50 con 0x5653a4c77e90 2024-03-21T11:39:09.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.030+0000 7fd04e600640 1 -- 172.21.15.47:0/218959506 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c014040 con 0x5653a4c77e90 2024-03-21T11:39:09.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.031+0000 7fd06b963740 1 -- 172.21.15.47:0/218959506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c77e90 msgr2=0x5653a4c79a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:09.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.031+0000 7fd06b963740 1 --2- 172.21.15.47:0/218959506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c77e90 0x5653a4c79a30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009fd0 tx=0x7fd05c004a40 comp rx=0 tx=0).stop 2024-03-21T11:39:09.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.031+0000 7fd06b963740 1 -- 172.21.15.47:0/218959506 shutdown_connections 2024-03-21T11:39:09.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.031+0000 7fd06b963740 1 --2- 172.21.15.47:0/218959506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c7d9f0 0x5653a4c61200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.031+0000 7fd06b963740 1 --2- 172.21.15.47:0/218959506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c78880 0x5653a4c89090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.031+0000 7fd06b963740 1 --2- 172.21.15.47:0/218959506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c77e90 0x5653a4c79a30 secure :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009fd0 tx=0x7fd05c004a40 comp rx=0 tx=0).stop 2024-03-21T11:39:09.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.031+0000 7fd06b963740 1 -- 172.21.15.47:0/218959506 >> 172.21.15.47:0/218959506 conn(0x5653a4c2f200 msgr2=0x5653a4c79320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:09.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.032+0000 7fd06b963740 1 -- 172.21.15.47:0/218959506 shutdown_connections 2024-03-21T11:39:09.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.032+0000 7fd06b963740 1 -- 172.21.15.47:0/218959506 wait complete. 2024-03-21T11:39:09.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.032+0000 7fd06b963740 1 -- start start 2024-03-21T11:39:09.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.033+0000 7fd06b963740 1 Processor -- start 2024-03-21T11:39:09.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.035+0000 7fd06b963740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c78880 0x5653a4c86e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:09.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.035+0000 7fd06b963740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c7d9f0 0x5653a4c87370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:09.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.035+0000 7fd06b963740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c88380 0x5653a4c88800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:09.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.035+0000 7fd06b963740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0641a9300 con 0x5653a4c7d9f0 2024-03-21T11:39:09.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.035+0000 7fd06b963740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0641a8ca0 con 0x5653a4c88380 2024-03-21T11:39:09.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.035+0000 7fd06b963740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd064082260 con 0x5653a4c78880 2024-03-21T11:39:09.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.035+0000 7fd06b963740 10 client.0 _ll_register_callbacks cb 0x7fd05c00b510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:09.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.036+0000 7fd061e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c78880 0x5653a4c86e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:09.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.036+0000 7fd062800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c88380 0x5653a4c88800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:09.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.036+0000 7fd062800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c88380 0x5653a4c88800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35752/0 (socket says 172.21.15.47:35752) 2024-03-21T11:39:09.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.036+0000 7fd062800640 1 -- 172.21.15.47:0/3120115736 learned_addr learned my addr 172.21.15.47:0/3120115736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:09.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.036+0000 7fd061400640 1 --2- 172.21.15.47:0/3120115736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c7d9f0 0x5653a4c87370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:09.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.038+0000 7fd061e00640 1 -- 172.21.15.47:0/3120115736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c88380 msgr2=0x5653a4c88800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:09.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.038+0000 7fd061e00640 1 --2- 172.21.15.47:0/3120115736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c88380 0x5653a4c88800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.038+0000 7fd061e00640 1 -- 172.21.15.47:0/3120115736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c7d9f0 msgr2=0x5653a4c87370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.038+0000 7fd061e00640 1 --2- 172.21.15.47:0/3120115736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c7d9f0 0x5653a4c87370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.038+0000 7fd061e00640 1 -- 172.21.15.47:0/3120115736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653a4c88d40 con 0x5653a4c78880 2024-03-21T11:39:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.040+0000 7fd043400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:09.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.040+0000 7fd061e00640 1 --2- 172.21.15.47:0/3120115736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c78880 0x5653a4c86e30 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd05c01c000 tx=0x7fd05c008dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d55dd758b2563614 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:09.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.041+0000 7fd043400640 1 -- 172.21.15.47:0/3120115736 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c02c030 con 0x5653a4c78880 2024-03-21T11:39:09.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.041+0000 7fd043400640 1 -- 172.21.15.47:0/3120115736 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd05c00a400 con 0x5653a4c78880 2024-03-21T11:39:09.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.041+0000 7fd06b963740 1 -- 172.21.15.47:0/3120115736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd05c026b50 con 0x5653a4c78880 2024-03-21T11:39:09.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.045+0000 7fd043400640 1 -- 172.21.15.47:0/3120115736 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd05c00a620 con 0x5653a4c78880 2024-03-21T11:39:09.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.045+0000 7fd06b963740 10 client.30703 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:09.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.045+0000 7fd06b963740 1 -- 172.21.15.47:0/3120115736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5653a4cfa430 con 0x5653a4c78880 2024-03-21T11:39:09.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.046+0000 7fd043400640 1 -- 172.21.15.47:0/3120115736 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd05c026b50 con 0x5653a4c78880 2024-03-21T11:39:09.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.046+0000 7fd043400640 1 -- 172.21.15.47:0/3120115736 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fd05c01abb0 con 0x5653a4c78880 2024-03-21T11:39:09.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.046+0000 7fd06b963740 10 client.30703 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:09.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.046+0000 7fd06b963740 10 client.30703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:09.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.046+0000 7fd06b963740 20 client.30703 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:09.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.046+0000 7fd06b963740 1 --2- 172.21.15.47:0/3120115736 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653a4cfacc0 0x5653a4cfd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:09.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.046+0000 7fd06b963740 4 client.30703 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:39:09.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.046+0000 7fd06b963740 1 -- 172.21.15.47:0/3120115736 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5653a4bd3ae0 con 0x5653a4cfacc0 2024-03-21T11:39:09.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.047+0000 7fd061400640 1 --2- 172.21.15.47:0/3120115736 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653a4cfacc0 0x5653a4cfd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:09.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.050+0000 7fd061400640 1 --2- 172.21.15.47:0/3120115736 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653a4cfacc0 0x5653a4cfd0e0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37fd6b8a5d49b244 server_cookie=e45fc02f030956fa in_seq=0 out_seq=0 2024-03-21T11:39:09.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.050+0000 7fd043400640 10 client.30703 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:09.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.050+0000 7fd043400640 1 -- 172.21.15.47:0/3120115736 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5653a4bd3ae0 con 0x5653a4cfacc0 2024-03-21T11:39:09.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.050+0000 7fd043400640 10 client.30703 handle_command_reply: tid=0 2024-03-21T11:39:09.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.050+0000 7fd06b963740 1 client.30703 shutdown 2024-03-21T11:39:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.051+0000 7fd06b963740 1 -- 172.21.15.47:0/3120115736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c78880 msgr2=0x5653a4c86e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.051+0000 7fd06b963740 1 --2- 172.21.15.47:0/3120115736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c78880 0x5653a4c86e30 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd05c01c000 tx=0x7fd05c008dd0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.051+0000 7fd06b963740 1 -- 172.21.15.47:0/3120115736 shutdown_connections 2024-03-21T11:39:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.051+0000 7fd06b963740 1 --2- 172.21.15.47:0/3120115736 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5653a4cfacc0 0x5653a4cfd0e0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.051+0000 7fd06b963740 1 --2- 172.21.15.47:0/3120115736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653a4c88380 0x5653a4c88800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.051+0000 7fd06b963740 1 --2- 172.21.15.47:0/3120115736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653a4c7d9f0 0x5653a4c87370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.051+0000 7fd06b963740 1 --2- 172.21.15.47:0/3120115736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653a4c78880 0x5653a4c86e30 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.051+0000 7fd06b963740 1 -- 172.21.15.47:0/3120115736 >> 172.21.15.47:0/3120115736 conn(0x5653a4c2f200 msgr2=0x5653a4c79bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.051+0000 7fd043400640 0 client.30703 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.052+0000 7fd06b963740 1 -- 172.21.15.47:0/3120115736 shutdown_connections 2024-03-21T11:39:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.052+0000 7fd06b963740 1 -- 172.21.15.47:0/3120115736 wait complete. 2024-03-21T11:39:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.052+0000 7fd06b963740 20 client.30703 trim_cache size 0 max 16384 2024-03-21T11:39:09.054 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:39:09.054 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:39:09.054 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:39:09.054 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:39:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.054+0000 7fd068c00640 1 -- 172.21.15.47:0/1794241178 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd0480a2ff0 msgr2=0x7fd0480a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.054+0000 7fd068c00640 1 --2- 172.21.15.47:0/1794241178 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd0480a2ff0 0x7fd0480a54b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00be40 tx=0x7fd05c009ce0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.054+0000 7fd068c00640 1 -- 172.21.15.47:0/1794241178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd064109870 msgr2=0x7fd0641a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.054+0000 7fd068c00640 1 --2- 172.21.15.47:0/1794241178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd064109870 0x7fd0641a5580 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd050009850 tx=0x7fd050004ea0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.055+0000 7fd068c00640 1 -- 172.21.15.47:0/1794241178 shutdown_connections 2024-03-21T11:39:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.055+0000 7fd068c00640 1 --2- 172.21.15.47:0/1794241178 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd0480a2ff0 0x7fd0480a54b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.055+0000 7fd068c00640 1 --2- 172.21.15.47:0/1794241178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd064109870 0x7fd0641a5580 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.055+0000 7fd068c00640 1 --2- 172.21.15.47:0/1794241178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd064108eb0 0x7fd06419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.055+0000 7fd068c00640 1 --2- 172.21.15.47:0/1794241178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd064107cc0 0x7fd06419dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.055+0000 7fd068c00640 1 -- 172.21.15.47:0/1794241178 >> 172.21.15.47:0/1794241178 conn(0x7fd064076c60 msgr2=0x7fd06410c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.055+0000 7fd068c00640 1 -- 172.21.15.47:0/1794241178 shutdown_connections 2024-03-21T11:39:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.056+0000 7fd068c00640 1 -- 172.21.15.47:0/1794241178 wait complete. 2024-03-21T11:39:09.083 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:39:09.083 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:39:09.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:09 smithi100 ceph-mon[36245]: pgmap v874: 129 pgs: 129 active+clean; 360 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.0 MiB/s wr, 573 op/s 2024-03-21T11:39:09.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.404+0000 7fd677c00640 1 -- 172.21.15.47:0/862304741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd670105490 msgr2=0x7fd67010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:09.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.404+0000 7fd677c00640 1 --2- 172.21.15.47:0/862304741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd670105490 0x7fd67010bd20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd668009ca0 tx=0x7fd66802c100 comp rx=0 tx=0).stop 2024-03-21T11:39:09.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.405+0000 7fd677c00640 1 -- 172.21.15.47:0/862304741 shutdown_connections 2024-03-21T11:39:09.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.405+0000 7fd677c00640 1 --2- 172.21.15.47:0/862304741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd670105490 0x7fd67010bd20 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.405+0000 7fd677c00640 1 --2- 172.21.15.47:0/862304741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd670104ad0 0x7fd670104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.405+0000 7fd677c00640 1 --2- 172.21.15.47:0/862304741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6701038e0 0x7fd670103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.405+0000 7fd677c00640 1 -- 172.21.15.47:0/862304741 >> 172.21.15.47:0/862304741 conn(0x7fd6700fee90 msgr2=0x7fd6701012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:09.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.407+0000 7fd677c00640 1 -- 172.21.15.47:0/862304741 shutdown_connections 2024-03-21T11:39:09.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.409+0000 7fd677c00640 1 -- 172.21.15.47:0/862304741 wait complete. 2024-03-21T11:39:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.410+0000 7fd677c00640 1 Processor -- start 2024-03-21T11:39:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.411+0000 7fd677c00640 1 -- start start 2024-03-21T11:39:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.411+0000 7fd677c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6701038e0 0x7fd670199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.411+0000 7fd677c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd670104ad0 0x7fd67019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.411+0000 7fd677c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd670105490 0x7fd6701a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.411+0000 7fd677c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67010f5d0 con 0x7fd6701038e0 2024-03-21T11:39:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.411+0000 7fd677c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67010f450 con 0x7fd670104ad0 2024-03-21T11:39:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.411+0000 7fd677c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67010f750 con 0x7fd670105490 2024-03-21T11:39:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.412+0000 7fd675200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6701038e0 0x7fd670199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.412+0000 7fd675c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd670105490 0x7fd6701a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.412+0000 7fd675c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd670105490 0x7fd6701a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37246/0 (socket says 172.21.15.47:37246) 2024-03-21T11:39:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.412+0000 7fd675c00640 1 -- 172.21.15.47:0/1016581548 learned_addr learned my addr 172.21.15.47:0/1016581548 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.413+0000 7fd66fe00640 1 --2- 172.21.15.47:0/1016581548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd670104ad0 0x7fd67019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.414+0000 7fd675c00640 1 -- 172.21.15.47:0/1016581548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd670104ad0 msgr2=0x7fd67019a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.414+0000 7fd675c00640 1 --2- 172.21.15.47:0/1016581548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd670104ad0 0x7fd67019a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.414+0000 7fd675c00640 1 -- 172.21.15.47:0/1016581548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6701038e0 msgr2=0x7fd670199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.414+0000 7fd675c00640 1 --2- 172.21.15.47:0/1016581548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6701038e0 0x7fd670199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.414+0000 7fd675c00640 1 -- 172.21.15.47:0/1016581548 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6701a1870 con 0x7fd670105490 2024-03-21T11:39:09.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.416+0000 7fd675200640 1 --2- 172.21.15.47:0/1016581548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6701038e0 0x7fd670199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:39:09.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.416+0000 7fd675c00640 1 --2- 172.21.15.47:0/1016581548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd670105490 0x7fd6701a11d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd668009bf0 tx=0x7fd668035e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.416+0000 7fd66d600640 1 -- 172.21.15.47:0/1016581548 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668039040 con 0x7fd670105490 2024-03-21T11:39:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.417+0000 7fd677c00640 1 -- 172.21.15.47:0/1016581548 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6701a1b00 con 0x7fd670105490 2024-03-21T11:39:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.417+0000 7fd677c00640 1 -- 172.21.15.47:0/1016581548 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6701a2040 con 0x7fd670105490 2024-03-21T11:39:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.418+0000 7fd66d600640 1 -- 172.21.15.47:0/1016581548 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd668044070 con 0x7fd670105490 2024-03-21T11:39:09.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.418+0000 7fd66d600640 1 -- 172.21.15.47:0/1016581548 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668004430 con 0x7fd670105490 2024-03-21T11:39:09.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.419+0000 7fd677c00640 1 -- 172.21.15.47:0/1016581548 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6500052c0 con 0x7fd670105490 2024-03-21T11:39:09.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.425+0000 7fd66d600640 1 -- 172.21.15.47:0/1016581548 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd668032890 con 0x7fd670105490 2024-03-21T11:39:09.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.427+0000 7fd66d600640 1 --2- 172.21.15.47:0/1016581548 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd6580a2ff0 0x7fd6580a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:09.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.428+0000 7fd675200640 1 --2- 172.21.15.47:0/1016581548 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd6580a2ff0 0x7fd6580a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:09.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.428+0000 7fd66d600640 1 -- 172.21.15.47:0/1016581548 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd6680f1fb0 con 0x7fd670105490 2024-03-21T11:39:09.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.430+0000 7fd675200640 1 --2- 172.21.15.47:0/1016581548 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd6580a2ff0 0x7fd6580a54b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd67019b130 tx=0x7fd664008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:09.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.433+0000 7fd66d600640 1 -- 172.21.15.47:0/1016581548 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd66804d020 con 0x7fd670105490 2024-03-21T11:39:09.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:09 smithi102 ceph-mon[36748]: pgmap v874: 129 pgs: 129 active+clean; 360 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.0 MiB/s wr, 573 op/s 2024-03-21T11:39:09.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:09 smithi047 ceph-mon[31351]: pgmap v874: 129 pgs: 129 active+clean; 360 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.0 MiB/s wr, 573 op/s 2024-03-21T11:39:09.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.924+0000 7fd677c00640 1 -- 172.21.15.47:0/1016581548 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6500055b0 con 0x7fd670105490 2024-03-21T11:39:09.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.927+0000 7fd66d600640 1 -- 172.21.15.47:0/1016581548 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+11493 (secure 0 0 0) 0x7fd6680bea50 con 0x7fd670105490 2024-03-21T11:39:09.928 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:39:09.928 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:37:28.116356+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24581,24553,24535,24523,24524]},"id":1}]} 2024-03-21T11:39:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 34 2024-03-21T11:39:09.929 INFO:tasks.workunit.client.0.smithi047.stdout: 10 54874 27.71 MB/sec warmup 113 sec latency 171.446 ms 2024-03-21T11:39:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.930+0000 7fd677c00640 1 -- 172.21.15.47:0/1016581548 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd6580a2ff0 msgr2=0x7fd6580a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.930+0000 7fd677c00640 1 --2- 172.21.15.47:0/1016581548 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd6580a2ff0 0x7fd6580a54b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd67019b130 tx=0x7fd664008040 comp rx=0 tx=0).stop 2024-03-21T11:39:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.930+0000 7fd677c00640 1 -- 172.21.15.47:0/1016581548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd670105490 msgr2=0x7fd6701a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.930+0000 7fd677c00640 1 --2- 172.21.15.47:0/1016581548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd670105490 0x7fd6701a11d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd668009bf0 tx=0x7fd668035e50 comp rx=0 tx=0).stop 2024-03-21T11:39:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.930+0000 7fd677c00640 1 -- 172.21.15.47:0/1016581548 shutdown_connections 2024-03-21T11:39:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.930+0000 7fd677c00640 1 --2- 172.21.15.47:0/1016581548 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd6580a2ff0 0x7fd6580a54b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.930+0000 7fd677c00640 1 --2- 172.21.15.47:0/1016581548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd670105490 0x7fd6701a11d0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.930+0000 7fd677c00640 1 --2- 172.21.15.47:0/1016581548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd670104ad0 0x7fd67019a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.930+0000 7fd677c00640 1 --2- 172.21.15.47:0/1016581548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6701038e0 0x7fd670199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.930+0000 7fd677c00640 1 -- 172.21.15.47:0/1016581548 >> 172.21.15.47:0/1016581548 conn(0x7fd6700fee90 msgr2=0x7fd670107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.930+0000 7fd677c00640 1 -- 172.21.15.47:0/1016581548 shutdown_connections 2024-03-21T11:39:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:09.930+0000 7fd677c00640 1 -- 172.21.15.47:0/1016581548 wait complete. 2024-03-21T11:39:09.951 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:39:09.951 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:39:10.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.249+0000 7fd7e6000640 1 -- 172.21.15.47:0/1718821779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e00ffe70 msgr2=0x7fd7e0100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.249+0000 7fd7e6000640 1 --2- 172.21.15.47:0/1718821779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e00ffe70 0x7fd7e0100290 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009900 tx=0x7fd7d402bcd0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.250+0000 7fd7e6000640 1 -- 172.21.15.47:0/1718821779 shutdown_connections 2024-03-21T11:39:10.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.250+0000 7fd7e6000640 1 --2- 172.21.15.47:0/1718821779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7e0101170 0x7fd7e0103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.250+0000 7fd7e6000640 1 --2- 172.21.15.47:0/1718821779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7e01007d0 0x7fd7e0100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.250+0000 7fd7e6000640 1 --2- 172.21.15.47:0/1718821779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e00ffe70 0x7fd7e0100290 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.250+0000 7fd7e6000640 1 -- 172.21.15.47:0/1718821779 >> 172.21.15.47:0/1718821779 conn(0x7fd7e00fba10 msgr2=0x7fd7e00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.250+0000 7fd7e6000640 1 -- 172.21.15.47:0/1718821779 shutdown_connections 2024-03-21T11:39:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.250+0000 7fd7e6000640 1 -- 172.21.15.47:0/1718821779 wait complete. 2024-03-21T11:39:10.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.251+0000 7fd7e6000640 1 Processor -- start 2024-03-21T11:39:10.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.251+0000 7fd7e6000640 1 -- start start 2024-03-21T11:39:10.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.252+0000 7fd7e6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7e00ffe70 0x7fd7e0070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.252+0000 7fd7e6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7e01007d0 0x7fd7e0070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.252+0000 7fd7e6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e0101170 0x7fd7e01adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.252+0000 7fd7e6000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e01138a0 con 0x7fd7e01007d0 2024-03-21T11:39:10.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.252+0000 7fd7e6000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e0113720 con 0x7fd7e00ffe70 2024-03-21T11:39:10.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.252+0000 7fd7e6000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e0113a20 con 0x7fd7e0101170 2024-03-21T11:39:10.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.252+0000 7fd7e4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7e00ffe70 0x7fd7e0070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.252+0000 7fd7e4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7e00ffe70 0x7fd7e0070030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35768/0 (socket says 172.21.15.47:35768) 2024-03-21T11:39:10.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.252+0000 7fd7e4c00640 1 -- 172.21.15.47:0/1681598602 learned_addr learned my addr 172.21.15.47:0/1681598602 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:10.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.253+0000 7fd7e4c00640 1 -- 172.21.15.47:0/1681598602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e0101170 msgr2=0x7fd7e01adf30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T11:39:10.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.253+0000 7fd7e4c00640 1 --2- 172.21.15.47:0/1681598602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e0101170 0x7fd7e01adf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.253+0000 7fd7e4c00640 1 -- 172.21.15.47:0/1681598602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7e01007d0 msgr2=0x7fd7e0070570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:39:10.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.253+0000 7fd7dfe00640 1 --2- 172.21.15.47:0/1681598602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7e01007d0 0x7fd7e0070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.254+0000 7fd7e4c00640 1 --2- 172.21.15.47:0/1681598602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7e01007d0 0x7fd7e0070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.254+0000 7fd7e4c00640 1 -- 172.21.15.47:0/1681598602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e01ae470 con 0x7fd7e00ffe70 2024-03-21T11:39:10.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.254+0000 7fd7dfe00640 1 --2- 172.21.15.47:0/1681598602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7e01007d0 0x7fd7e0070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:10.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.257+0000 7fd7e4c00640 1 --2- 172.21.15.47:0/1681598602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7e00ffe70 0x7fd7e0070030 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4000c00 tx=0x7fd7d4005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.259+0000 7fd7dd600640 1 -- 172.21.15.47:0/1681598602 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4043020 con 0x7fd7e00ffe70 2024-03-21T11:39:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.259+0000 7fd7dd600640 1 -- 172.21.15.47:0/1681598602 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7d40054e0 con 0x7fd7e00ffe70 2024-03-21T11:39:10.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.259+0000 7fd7dd600640 1 -- 172.21.15.47:0/1681598602 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4003cf0 con 0x7fd7e00ffe70 2024-03-21T11:39:10.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.260+0000 7fd7e6000640 1 -- 172.21.15.47:0/1681598602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e01ae640 con 0x7fd7e00ffe70 2024-03-21T11:39:10.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.260+0000 7fd7e6000640 1 -- 172.21.15.47:0/1681598602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e01aea50 con 0x7fd7e00ffe70 2024-03-21T11:39:10.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.264+0000 7fd7ea3fa740 1 Processor -- start 2024-03-21T11:39:10.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.264+0000 7fd7ea3fa740 1 -- start start 2024-03-21T11:39:10.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.266+0000 7fd7dd600640 1 -- 172.21.15.47:0/1681598602 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd7d4046020 con 0x7fd7e00ffe70 2024-03-21T11:39:10.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.266+0000 7fd7ea3fa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93b9f0 0x55acdf93be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.268+0000 7fd7dd600640 1 --2- 172.21.15.47:0/1681598602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7c80a30f0 0x7fd7c80a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.268+0000 7fd7ea3fa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93cbc0 0x55acdf93d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.268+0000 7fd7ea3fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93d580 0x55acdf943e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.268+0000 7fd7dd600640 1 -- 172.21.15.47:0/1681598602 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd7d40f19d0 con 0x7fd7e00ffe70 2024-03-21T11:39:10.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.269+0000 7fd7e5600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93b9f0 0x55acdf93be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.269+0000 7fd7e4c00640 1 --2- 172.21.15.47:0/1681598602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7c80a30f0 0x7fd7c80a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.270+0000 7fd7ea3fa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdf8a02c0 con 0x55acdf93b9f0 2024-03-21T11:39:10.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.270+0000 7fd7ea3fa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdf848c90 con 0x55acdf93cbc0 2024-03-21T11:39:10.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.270+0000 7fd7ea3fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdf89c360 con 0x55acdf93d580 2024-03-21T11:39:10.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.270+0000 7fd7dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93cbc0 0x55acdf93d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.270+0000 7fd7dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93cbc0 0x55acdf93d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35782/0 (socket says 172.21.15.47:35782) 2024-03-21T11:39:10.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.270+0000 7fd7dfe00640 1 -- 172.21.15.47:0/4111107198 learned_addr learned my addr 172.21.15.47:0/4111107198 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:10.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.270+0000 7fd7e4c00640 1 --2- 172.21.15.47:0/1681598602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7c80a30f0 0x7fd7c80a55b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40bdca0 tx=0x7fd7d40de040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:10.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.271+0000 7fd7e5600640 1 --2- 172.21.15.47:0/4111107198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93d580 0x55acdf943e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.271+0000 7fd7dfe00640 1 -- 172.21.15.47:0/4111107198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93d580 msgr2=0x55acdf943e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:10.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.271+0000 7fd7dfe00640 1 --2- 172.21.15.47:0/4111107198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93d580 0x55acdf943e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.271+0000 7fd7dfe00640 1 -- 172.21.15.47:0/4111107198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93b9f0 msgr2=0x55acdf93be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:10.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.271+0000 7fd7dfe00640 1 --2- 172.21.15.47:0/4111107198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93b9f0 0x55acdf93be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.271+0000 7fd7dfe00640 1 -- 172.21.15.47:0/4111107198 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acdf86d080 con 0x55acdf93cbc0 2024-03-21T11:39:10.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.273+0000 7fd7dfe00640 1 --2- 172.21.15.47:0/4111107198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93cbc0 0x55acdf93d040 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009c20 tx=0x7fd7cc00e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25e91ad1be1b9f3e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:10.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.274+0000 7fd7d1c00640 1 -- 172.21.15.47:0/4111107198 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc016070 con 0x55acdf93cbc0 2024-03-21T11:39:10.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.274+0000 7fd7d1c00640 1 -- 172.21.15.47:0/4111107198 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7cc011950 con 0x55acdf93cbc0 2024-03-21T11:39:10.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.274+0000 7fd7d1c00640 1 -- 172.21.15.47:0/4111107198 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc016070 con 0x55acdf93cbc0 2024-03-21T11:39:10.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.277+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/4111107198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93cbc0 msgr2=0x55acdf93d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.277+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/4111107198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93cbc0 0x55acdf93d040 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009c20 tx=0x7fd7cc00e6c0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.277+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/4111107198 shutdown_connections 2024-03-21T11:39:10.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.277+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/4111107198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93d580 0x55acdf943e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.277+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/4111107198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93cbc0 0x55acdf93d040 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.277+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/4111107198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93b9f0 0x55acdf93be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.277+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/4111107198 >> 172.21.15.47:0/4111107198 conn(0x55acdf936de0 msgr2=0x55acdf9371e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:10.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.278+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/4111107198 shutdown_connections 2024-03-21T11:39:10.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.278+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/4111107198 wait complete. 2024-03-21T11:39:10.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.278+0000 7fd7ea3fa740 1 -- start start 2024-03-21T11:39:10.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.279+0000 7fd7ea3fa740 1 Processor -- start 2024-03-21T11:39:10.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.280+0000 7fd7ea3fa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93b9f0 0x55acdf945a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.280+0000 7fd7ea3fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93cbc0 0x55acdf945fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.280+0000 7fd7ea3fa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93d580 0x55acdf94cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.280+0000 7fd7ea3fa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdf944590 con 0x55acdf93b9f0 2024-03-21T11:39:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.280+0000 7fd7ea3fa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdf85c0d0 con 0x55acdf93d580 2024-03-21T11:39:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.280+0000 7fd7ea3fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdf944710 con 0x55acdf93cbc0 2024-03-21T11:39:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.280+0000 7fd7ea3fa740 10 client.0 _ll_register_callbacks cb 0x55acdf860e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.281+0000 7fd7dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93cbc0 0x55acdf945fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.282+0000 7fd7e5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93d580 0x55acdf94cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.282+0000 7fd7dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93cbc0 0x55acdf945fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37262/0 (socket says 172.21.15.47:37262) 2024-03-21T11:39:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.282+0000 7fd7dfe00640 1 -- 172.21.15.47:0/1361231922 learned_addr learned my addr 172.21.15.47:0/1361231922 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:10.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.282+0000 7fd7e5600640 1 --2- 172.21.15.47:0/1361231922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93b9f0 0x55acdf945a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.282+0000 7fd7e5600640 1 -- 172.21.15.47:0/1361231922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93cbc0 msgr2=0x55acdf945fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.282+0000 7fd7e5600640 1 --2- 172.21.15.47:0/1361231922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93cbc0 0x55acdf945fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.282+0000 7fd7e5600640 1 -- 172.21.15.47:0/1361231922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93b9f0 msgr2=0x55acdf945a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.282+0000 7fd7e5600640 1 --2- 172.21.15.47:0/1361231922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93b9f0 0x55acdf945a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.282+0000 7fd7e5600640 1 -- 172.21.15.47:0/1361231922 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acdf5b43f0 con 0x55acdf93d580 2024-03-21T11:39:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.283+0000 7fd7e5600640 1 --2- 172.21.15.47:0/1361231922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93d580 0x55acdf94cff0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8005c40 tx=0x7fd7d8009a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fe3654f9497f79 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:10.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.283+0000 7fd7c6000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:10.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.283+0000 7fd7c6000640 1 -- 172.21.15.47:0/1361231922 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8013c80 con 0x55acdf93d580 2024-03-21T11:39:10.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.283+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/1361231922 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7cc011950 con 0x55acdf93d580 2024-03-21T11:39:10.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.284+0000 7fd7c6000640 1 -- 172.21.15.47:0/1361231922 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd7d8013e20 con 0x55acdf93d580 2024-03-21T11:39:10.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.284+0000 7fd7c6000640 1 -- 172.21.15.47:0/1361231922 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7d8027050 con 0x55acdf93d580 2024-03-21T11:39:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.284+0000 7fd7ea3fa740 10 client.30704 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.284+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/1361231922 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55acdf67e7d0 con 0x55acdf93d580 2024-03-21T11:39:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.285+0000 7fd7c6000640 1 -- 172.21.15.47:0/1361231922 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7cc011950 con 0x55acdf93d580 2024-03-21T11:39:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.285+0000 7fd7c6000640 1 -- 172.21.15.47:0/1361231922 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fd7d800f040 con 0x55acdf93d580 2024-03-21T11:39:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.285+0000 7fd7ea3fa740 10 client.30704 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.285+0000 7fd7ea3fa740 10 client.30704 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:39:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.285+0000 7fd7ea3fa740 20 client.30704 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.285+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/1361231922 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55acdf951470 0x55acdf9538d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.285+0000 7fd7ea3fa740 4 client.30704 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:10.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.285+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/1361231922 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acdf8ad800 con 0x55acdf951470 2024-03-21T11:39:10.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.288+0000 7fd7dfe00640 1 --2- 172.21.15.47:0/1361231922 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55acdf951470 0x55acdf9538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.292+0000 7fd7dfe00640 1 --2- 172.21.15.47:0/1361231922 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55acdf951470 0x55acdf9538d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e303b0ac8280263 server_cookie=3b668bc5252b37d2 in_seq=0 out_seq=0 2024-03-21T11:39:10.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.292+0000 7fd7c6000640 10 client.30704 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:39:10.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.295+0000 7fd7c6000640 1 -- 172.21.15.47:0/1361231922 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55acdf8ad800 con 0x55acdf951470 2024-03-21T11:39:10.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.295+0000 7fd7c6000640 10 client.30704 handle_command_reply: tid=0 2024-03-21T11:39:10.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.295+0000 7fd7ea3fa740 1 client.30704 shutdown 2024-03-21T11:39:10.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.296+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/1361231922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93d580 msgr2=0x55acdf94cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.296+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/1361231922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93d580 0x55acdf94cff0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8005c40 tx=0x7fd7d8009a50 comp rx=0 tx=0).stop 2024-03-21T11:39:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.299+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/1361231922 shutdown_connections 2024-03-21T11:39:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.299+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/1361231922 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55acdf951470 0x55acdf9538d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.299+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/1361231922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93d580 0x55acdf94cff0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.299+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/1361231922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93cbc0 0x55acdf945fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.299+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/1361231922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93b9f0 0x55acdf945a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.299+0000 7fd7c6000640 0 client.30704 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:39:10.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.299+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/1361231922 >> 172.21.15.47:0/1361231922 conn(0x55acdf936de0 msgr2=0x55acdf936160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:10.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.299+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/1361231922 shutdown_connections 2024-03-21T11:39:10.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.299+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/1361231922 wait complete. 2024-03-21T11:39:10.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.299+0000 7fd7ea3fa740 20 client.30704 trim_cache size 0 max 16384 2024-03-21T11:39:10.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:10 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:39:10.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:10 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1016581548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:39:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.330+0000 7fd7ea3fa740 1 Processor -- start 2024-03-21T11:39:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.331+0000 7fd7ea3fa740 1 -- start start 2024-03-21T11:39:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.331+0000 7fd7ea3fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93b9f0 0x55acdf93fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.331+0000 7fd7ea3fa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93cbc0 0x55acdf94cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.331+0000 7fd7ea3fa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93d580 0x55acdf93ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.331+0000 7fd7ea3fa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e01b8cb0 con 0x55acdf93d580 2024-03-21T11:39:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.331+0000 7fd7ea3fa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e01ba7b0 con 0x55acdf93cbc0 2024-03-21T11:39:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.331+0000 7fd7ea3fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e01b9310 con 0x55acdf93b9f0 2024-03-21T11:39:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.331+0000 7fd7e5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93b9f0 0x55acdf93fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.332+0000 7fd7e5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93b9f0 0x55acdf93fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37276/0 (socket says 172.21.15.47:37276) 2024-03-21T11:39:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.332+0000 7fd7e5600640 1 -- 172.21.15.47:0/3111534415 learned_addr learned my addr 172.21.15.47:0/3111534415 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.332+0000 7fd7e5600640 1 --2- 172.21.15.47:0/3111534415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93d580 0x55acdf93ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.332+0000 7fd7e5600640 1 -- 172.21.15.47:0/3111534415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93cbc0 msgr2=0x55acdf94cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.332+0000 7fd7dfe00640 1 --2- 172.21.15.47:0/3111534415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93cbc0 0x55acdf94cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.332+0000 7fd7e5600640 1 --2- 172.21.15.47:0/3111534415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93cbc0 0x55acdf94cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.332+0000 7fd7e5600640 1 -- 172.21.15.47:0/3111534415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93d580 msgr2=0x55acdf93ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.332+0000 7fd7e5600640 1 --2- 172.21.15.47:0/3111534415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93d580 0x55acdf93ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.332+0000 7fd7e5600640 1 -- 172.21.15.47:0/3111534415 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acdf93c640 con 0x55acdf93b9f0 2024-03-21T11:39:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.332+0000 7fd7dfe00640 1 --2- 172.21.15.47:0/3111534415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93cbc0 0x55acdf94cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.333+0000 7fd7e5600640 1 --2- 172.21.15.47:0/3111534415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93b9f0 0x55acdf93fa30 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8005d70 tx=0x7fd7d8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb1066a65748a2e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.334+0000 7fd7d1200640 1 -- 172.21.15.47:0/3111534415 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d801f810 con 0x55acdf93b9f0 2024-03-21T11:39:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.334+0000 7fd7d1200640 1 -- 172.21.15.47:0/3111534415 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7d8013a80 con 0x55acdf93b9f0 2024-03-21T11:39:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.334+0000 7fd7d1200640 1 -- 172.21.15.47:0/3111534415 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d801fe00 con 0x55acdf93b9f0 2024-03-21T11:39:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.335+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/3111534415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93b9f0 msgr2=0x55acdf93fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.335+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/3111534415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93b9f0 0x55acdf93fa30 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8005d70 tx=0x7fd7d8009f90 comp rx=0 tx=0).stop 2024-03-21T11:39:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.335+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/3111534415 shutdown_connections 2024-03-21T11:39:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.336+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/3111534415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93d580 0x55acdf93ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.336+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/3111534415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93cbc0 0x55acdf94cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.336+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/3111534415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93b9f0 0x55acdf93fa30 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.336+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/3111534415 >> 172.21.15.47:0/3111534415 conn(0x55acdf936de0 msgr2=0x55acdf93a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.336+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/3111534415 shutdown_connections 2024-03-21T11:39:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.336+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/3111534415 wait complete. 2024-03-21T11:39:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.337+0000 7fd7ea3fa740 1 -- start start 2024-03-21T11:39:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.337+0000 7fd7ea3fa740 1 Processor -- start 2024-03-21T11:39:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.338+0000 7fd7ea3fa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93b9f0 0x55acdf948f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.338+0000 7fd7ea3fa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93cbc0 0x55acdf949450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.338+0000 7fd7ea3fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93d580 0x55acdf94c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.338+0000 7fd7ea3fa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e01ba090 con 0x55acdf93cbc0 2024-03-21T11:39:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.338+0000 7fd7ea3fa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e01b9a30 con 0x55acdf93b9f0 2024-03-21T11:39:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.338+0000 7fd7ea3fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e01b6e70 con 0x55acdf93d580 2024-03-21T11:39:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.338+0000 7fd7ea3fa740 10 client.0 _ll_register_callbacks cb 0x7fd7d8009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.338+0000 7fd7e5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93b9f0 0x55acdf948f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.339+0000 7fd7e5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93b9f0 0x55acdf948f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35804/0 (socket says 172.21.15.47:35804) 2024-03-21T11:39:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.339+0000 7fd7e5600640 1 -- 172.21.15.47:0/3984335270 learned_addr learned my addr 172.21.15.47:0/3984335270 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.339+0000 7fd7e5600640 1 --2- 172.21.15.47:0/3984335270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93d580 0x55acdf94c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.339+0000 7fd7e5600640 1 -- 172.21.15.47:0/3984335270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93d580 msgr2=0x55acdf94c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.339+0000 7fd7e5600640 1 --2- 172.21.15.47:0/3984335270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93d580 0x55acdf94c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.339+0000 7fd7e5600640 1 -- 172.21.15.47:0/3984335270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93cbc0 msgr2=0x55acdf949450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:39:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.340+0000 7fd7dfe00640 1 --2- 172.21.15.47:0/3984335270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93cbc0 0x55acdf949450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.340+0000 7fd7e5600640 1 --2- 172.21.15.47:0/3984335270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93cbc0 0x55acdf949450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.340+0000 7fd7e5600640 1 -- 172.21.15.47:0/3984335270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acdf94ca60 con 0x55acdf93b9f0 2024-03-21T11:39:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.342+0000 7fd7e5600640 1 --2- 172.21.15.47:0/3984335270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93b9f0 0x55acdf948f10 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd7d801f3f0 tx=0x7fd7d80148c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d518d5594e5ea87 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.342+0000 7fd7c6000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:10.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.345+0000 7fd7c6000640 1 -- 172.21.15.47:0/3984335270 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d803d060 con 0x55acdf93b9f0 2024-03-21T11:39:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.346+0000 7fd7c6000640 1 -- 172.21.15.47:0/3984335270 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd7d801fdc0 con 0x55acdf93b9f0 2024-03-21T11:39:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.346+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/3984335270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7d8013a80 con 0x55acdf93b9f0 2024-03-21T11:39:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.346+0000 7fd7c6000640 1 -- 172.21.15.47:0/3984335270 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7d8003970 con 0x55acdf93b9f0 2024-03-21T11:39:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.346+0000 7fd7ea3fa740 10 client.30707 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.346+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/3984335270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55acdf9ad4f0 con 0x55acdf93b9f0 2024-03-21T11:39:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.347+0000 7fd7c6000640 1 -- 172.21.15.47:0/3984335270 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7d8013a80 con 0x55acdf93b9f0 2024-03-21T11:39:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.347+0000 7fd7c6000640 1 -- 172.21.15.47:0/3984335270 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fd7d800f070 con 0x55acdf93b9f0 2024-03-21T11:39:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.347+0000 7fd7ea3fa740 10 client.30707 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.347+0000 7fd7ea3fa740 10 client.30707 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:39:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.347+0000 7fd7ea3fa740 20 client.30707 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.347+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/3984335270 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55acdf9adda0 0x55acdf9b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.347+0000 7fd7ea3fa740 4 client.30707 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:39:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.347+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/3984335270 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55acdf8a7b50 con 0x55acdf9adda0 2024-03-21T11:39:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.348+0000 7fd7dfe00640 1 --2- 172.21.15.47:0/3984335270 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55acdf9adda0 0x55acdf9b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.349+0000 7fd7c6000640 10 client.30707 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:39:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.350+0000 7fd7dfe00640 1 --2- 172.21.15.47:0/3984335270 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55acdf9adda0 0x55acdf9b01c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=be7ecf1ff4a80fd7 server_cookie=c79d7ee3cb787581 in_seq=0 out_seq=0 2024-03-21T11:39:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.350+0000 7fd7c6000640 1 -- 172.21.15.47:0/3984335270 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55acdf8a7b50 con 0x55acdf9adda0 2024-03-21T11:39:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.350+0000 7fd7c6000640 10 client.30707 handle_command_reply: tid=0 2024-03-21T11:39:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.350+0000 7fd7ea3fa740 1 client.30707 shutdown 2024-03-21T11:39:10.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.351+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/3984335270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93b9f0 msgr2=0x55acdf948f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.351+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/3984335270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93b9f0 0x55acdf948f10 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd7d801f3f0 tx=0x7fd7d80148c0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.351+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/3984335270 shutdown_connections 2024-03-21T11:39:10.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.351+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/3984335270 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55acdf9adda0 0x55acdf9b01c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.351+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/3984335270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acdf93d580 0x55acdf94c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.351+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/3984335270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55acdf93cbc0 0x55acdf949450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.351+0000 7fd7c6000640 0 client.30707 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:39:10.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.351+0000 7fd7ea3fa740 1 --2- 172.21.15.47:0/3984335270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55acdf93b9f0 0x55acdf948f10 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.351+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/3984335270 >> 172.21.15.47:0/3984335270 conn(0x55acdf936de0 msgr2=0x55acdf93fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:10.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.352+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/3984335270 shutdown_connections 2024-03-21T11:39:10.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.352+0000 7fd7ea3fa740 1 -- 172.21.15.47:0/3984335270 wait complete. 2024-03-21T11:39:10.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.353+0000 7fd7ea3fa740 20 client.30707 trim_cache size 0 max 16384 2024-03-21T11:39:10.354 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:39:10.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.354+0000 7fd7e6000640 1 -- 172.21.15.47:0/1681598602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7c80a30f0 msgr2=0x7fd7c80a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.354+0000 7fd7e6000640 1 --2- 172.21.15.47:0/1681598602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7c80a30f0 0x7fd7c80a55b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40bdca0 tx=0x7fd7d40de040 comp rx=0 tx=0).stop 2024-03-21T11:39:10.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.355+0000 7fd7e6000640 1 -- 172.21.15.47:0/1681598602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7e00ffe70 msgr2=0x7fd7e0070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.355+0000 7fd7e6000640 1 --2- 172.21.15.47:0/1681598602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7e00ffe70 0x7fd7e0070030 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4000c00 tx=0x7fd7d4005dd0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.355+0000 7fd7e6000640 1 -- 172.21.15.47:0/1681598602 shutdown_connections 2024-03-21T11:39:10.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.355+0000 7fd7e6000640 1 --2- 172.21.15.47:0/1681598602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7c80a30f0 0x7fd7c80a55b0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.355+0000 7fd7e6000640 1 --2- 172.21.15.47:0/1681598602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7e0101170 0x7fd7e01adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.355+0000 7fd7e6000640 1 --2- 172.21.15.47:0/1681598602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7e01007d0 0x7fd7e0070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.355+0000 7fd7e6000640 1 --2- 172.21.15.47:0/1681598602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7e00ffe70 0x7fd7e0070030 secure :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4000c00 tx=0x7fd7d4005dd0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.355+0000 7fd7e6000640 1 -- 172.21.15.47:0/1681598602 >> 172.21.15.47:0/1681598602 conn(0x7fd7e00fba10 msgr2=0x7fd7e0106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:10.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.357+0000 7fd7e6000640 1 -- 172.21.15.47:0/1681598602 shutdown_connections 2024-03-21T11:39:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.357+0000 7fd7e6000640 1 -- 172.21.15.47:0/1681598602 wait complete. 2024-03-21T11:39:10.386 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:39:10.387 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:39:10.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:10 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:39:10.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:10 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1016581548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:39:10.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:10 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:39:10.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:10 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1016581548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:39:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.679+0000 7f5f2e400640 1 -- 172.21.15.47:0/2489688305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f281086a0 msgr2=0x7f5f28108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.679+0000 7f5f2e400640 1 --2- 172.21.15.47:0/2489688305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f281086a0 0x7f5f28108ac0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009ca0 tx=0x7f5f2002c100 comp rx=0 tx=0).stop 2024-03-21T11:39:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.681+0000 7f5f2e400640 1 -- 172.21.15.47:0/2489688305 shutdown_connections 2024-03-21T11:39:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.681+0000 7f5f2e400640 1 --2- 172.21.15.47:0/2489688305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f2810ff20 0x7f5f28112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.681+0000 7f5f2e400640 1 --2- 172.21.15.47:0/2489688305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f28109000 0x7f5f2810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.681+0000 7f5f2e400640 1 --2- 172.21.15.47:0/2489688305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f281086a0 0x7f5f28108ac0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.681+0000 7f5f2e400640 1 -- 172.21.15.47:0/2489688305 >> 172.21.15.47:0/2489688305 conn(0x7f5f2806e500 msgr2=0x7f5f28070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.681+0000 7f5f2e400640 1 -- 172.21.15.47:0/2489688305 shutdown_connections 2024-03-21T11:39:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.681+0000 7f5f2e400640 1 -- 172.21.15.47:0/2489688305 wait complete. 2024-03-21T11:39:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.683+0000 7f5f2e400640 1 Processor -- start 2024-03-21T11:39:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.683+0000 7f5f2e400640 1 -- start start 2024-03-21T11:39:10.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.683+0000 7f5f2e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f281086a0 0x7f5f281a2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.683+0000 7f5f2e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f28109000 0x7f5f281a2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.683+0000 7f5f2e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2810ff20 0x7f5f281a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.683+0000 7f5f2e400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f28115b00 con 0x7f5f281086a0 2024-03-21T11:39:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.683+0000 7f5f2e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f28115980 con 0x7f5f2810ff20 2024-03-21T11:39:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.683+0000 7f5f2e400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f28115c80 con 0x7f5f28109000 2024-03-21T11:39:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.684+0000 7f5f26a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f28109000 0x7f5f281a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.684+0000 7f5f26a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f28109000 0x7f5f281a2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37294/0 (socket says 172.21.15.47:37294) 2024-03-21T11:39:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.684+0000 7f5f26a00640 1 -- 172.21.15.47:0/3431496744 learned_addr learned my addr 172.21.15.47:0/3431496744 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:10.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.684+0000 7f5f27e00640 1 --2- 172.21.15.47:0/3431496744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2810ff20 0x7f5f281a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.684+0000 7f5f27400640 1 --2- 172.21.15.47:0/3431496744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f281086a0 0x7f5f281a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.685+0000 7f5f27400640 1 -- 172.21.15.47:0/3431496744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f28109000 msgr2=0x7f5f281a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.685+0000 7f5f27400640 1 --2- 172.21.15.47:0/3431496744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f28109000 0x7f5f281a2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.685+0000 7f5f27400640 1 -- 172.21.15.47:0/3431496744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2810ff20 msgr2=0x7f5f281a99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.685+0000 7f5f27400640 1 --2- 172.21.15.47:0/3431496744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2810ff20 0x7f5f281a99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.685+0000 7f5f27400640 1 -- 172.21.15.47:0/3431496744 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f281aa060 con 0x7f5f281086a0 2024-03-21T11:39:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.685+0000 7f5f27400640 1 --2- 172.21.15.47:0/3431496744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f281086a0 0x7f5f281a2440 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5f2002c640 tx=0x7f5f20035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.688+0000 7f5f1fe00640 1 -- 172.21.15.47:0/3431496744 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20039040 con 0x7f5f281086a0 2024-03-21T11:39:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.688+0000 7f5f1fe00640 1 -- 172.21.15.47:0/3431496744 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f20003970 con 0x7f5f281086a0 2024-03-21T11:39:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.688+0000 7f5f1fe00640 1 -- 172.21.15.47:0/3431496744 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20004580 con 0x7f5f281086a0 2024-03-21T11:39:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.688+0000 7f5f2e400640 1 -- 172.21.15.47:0/3431496744 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f281aa2f0 con 0x7f5f281086a0 2024-03-21T11:39:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.688+0000 7f5f2e400640 1 -- 172.21.15.47:0/3431496744 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f281aa790 con 0x7f5f281086a0 2024-03-21T11:39:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.697+0000 7f5f3107b740 1 Processor -- start 2024-03-21T11:39:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.697+0000 7f5f3107b740 1 -- start start 2024-03-21T11:39:10.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.698+0000 7f5f1fe00640 1 -- 172.21.15.47:0/3431496744 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5f20003b20 con 0x7f5f281086a0 2024-03-21T11:39:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.698+0000 7f5f3107b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317b020 0x55e2e317b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.698+0000 7f5f3107b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317c1f0 0x55e2e317c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.698+0000 7f5f3107b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317cbb0 0x55e2e3183440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.698+0000 7f5f3107b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e30d72c0 con 0x55e2e317c1f0 2024-03-21T11:39:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.698+0000 7f5f3107b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e307fc90 con 0x55e2e317b020 2024-03-21T11:39:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.698+0000 7f5f3107b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e30d3360 con 0x55e2e317cbb0 2024-03-21T11:39:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.698+0000 7f5f27e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317b020 0x55e2e317b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.698+0000 7f5f27e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317b020 0x55e2e317b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35832/0 (socket says 172.21.15.47:35832) 2024-03-21T11:39:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.698+0000 7f5f27e00640 1 -- 172.21.15.47:0/425110251 learned_addr learned my addr 172.21.15.47:0/425110251 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.698+0000 7f5f26a00640 1 --2- 172.21.15.47:0/425110251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317cbb0 0x55e2e3183440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.699+0000 7f5f27e00640 1 -- 172.21.15.47:0/425110251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317cbb0 msgr2=0x55e2e3183440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:10.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.699+0000 7f5f27e00640 1 --2- 172.21.15.47:0/425110251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317cbb0 0x55e2e3183440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.699+0000 7f5f27e00640 1 -- 172.21.15.47:0/425110251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317c1f0 msgr2=0x55e2e317c670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:39:10.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.699+0000 7f5f27e00640 1 --2- 172.21.15.47:0/425110251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317c1f0 0x55e2e317c670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.699+0000 7f5f27e00640 1 -- 172.21.15.47:0/425110251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2e30a4080 con 0x55e2e317b020 2024-03-21T11:39:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.700+0000 7f5f1fe00640 1 --2- 172.21.15.47:0/3431496744 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5f0c0a30f0 0x7f5f0c0a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.700+0000 7f5f27e00640 1 --2- 172.21.15.47:0/425110251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317b020 0x55e2e317b440 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400b680 tx=0x7f5f1400bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf6f7f66046f6ac0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.700+0000 7f5f1fe00640 1 -- 172.21.15.47:0/3431496744 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5f200f1580 con 0x7f5f281086a0 2024-03-21T11:39:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.700+0000 7f5f1cc00640 1 -- 172.21.15.47:0/425110251 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14003bd0 con 0x55e2e317b020 2024-03-21T11:39:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.701+0000 7f5f1cc00640 1 -- 172.21.15.47:0/425110251 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f14015070 con 0x55e2e317b020 2024-03-21T11:39:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.701+0000 7f5f1cc00640 1 -- 172.21.15.47:0/425110251 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14003bd0 con 0x55e2e317b020 2024-03-21T11:39:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.702+0000 7f5f3107b740 1 -- 172.21.15.47:0/425110251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317b020 msgr2=0x55e2e317b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.702+0000 7f5f3107b740 1 --2- 172.21.15.47:0/425110251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317b020 0x55e2e317b440 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400b680 tx=0x7f5f1400bb40 comp rx=0 tx=0).stop 2024-03-21T11:39:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.702+0000 7f5f3107b740 1 -- 172.21.15.47:0/425110251 shutdown_connections 2024-03-21T11:39:10.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.703+0000 7f5f3107b740 1 --2- 172.21.15.47:0/425110251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317cbb0 0x55e2e3183440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.703+0000 7f5f27400640 1 --2- 172.21.15.47:0/3431496744 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5f0c0a30f0 0x7f5f0c0a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.703+0000 7f5f3107b740 1 --2- 172.21.15.47:0/425110251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317c1f0 0x55e2e317c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.703+0000 7f5f3107b740 1 --2- 172.21.15.47:0/425110251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317b020 0x55e2e317b440 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.703+0000 7f5f3107b740 1 -- 172.21.15.47:0/425110251 >> 172.21.15.47:0/425110251 conn(0x55e2e316dc00 msgr2=0x55e2e316e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:10.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.703+0000 7f5f3107b740 1 -- 172.21.15.47:0/425110251 shutdown_connections 2024-03-21T11:39:10.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.703+0000 7f5f3107b740 1 -- 172.21.15.47:0/425110251 wait complete. 2024-03-21T11:39:10.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.704+0000 7f5f3107b740 1 -- start start 2024-03-21T11:39:10.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.704+0000 7f5f3107b740 1 Processor -- start 2024-03-21T11:39:10.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.705+0000 7f5f27400640 1 --2- 172.21.15.47:0/3431496744 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5f0c0a30f0 0x7f5f0c0a55b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5f20000f80 tx=0x7f5f200093d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.705+0000 7f5f3107b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317b020 0x55e2e3185100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.706+0000 7f5f3107b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317c1f0 0x55e2e3185640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.706+0000 7f5f3107b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317cbb0 0x55e2e318c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.706+0000 7f5f3107b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e3183bc0 con 0x55e2e317c1f0 2024-03-21T11:39:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.706+0000 7f5f3107b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e30930d0 con 0x55e2e317cbb0 2024-03-21T11:39:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.706+0000 7f5f3107b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e3183d40 con 0x55e2e317b020 2024-03-21T11:39:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.706+0000 7f5f3107b740 10 client.0 _ll_register_callbacks cb 0x55e2e3097e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.706+0000 7f5f26a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317c1f0 0x55e2e3185640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.707+0000 7f5f26a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317c1f0 0x55e2e3185640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52280/0 (socket says 172.21.15.47:52280) 2024-03-21T11:39:10.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.707+0000 7f5f26a00640 1 -- 172.21.15.47:0/3429084575 learned_addr learned my addr 172.21.15.47:0/3429084575 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:10.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.708+0000 7f5f26a00640 1 -- 172.21.15.47:0/3429084575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317b020 msgr2=0x55e2e3185100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:39:10.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.708+0000 7f5f27e00640 1 --2- 172.21.15.47:0/3429084575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317b020 0x55e2e3185100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.709+0000 7f5f26a00640 1 --2- 172.21.15.47:0/3429084575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317b020 0x55e2e3185100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.709+0000 7f5f26a00640 1 -- 172.21.15.47:0/3429084575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317cbb0 msgr2=0x55e2e318c680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:39:10.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.709+0000 7f5f26a00640 1 --2- 172.21.15.47:0/3429084575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317cbb0 0x55e2e318c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.709+0000 7f5f26a00640 1 -- 172.21.15.47:0/3429084575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2e2deb3f0 con 0x55e2e317c1f0 2024-03-21T11:39:10.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.709+0000 7f5f26a00640 1 --2- 172.21.15.47:0/3429084575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317c1f0 0x55e2e3185640 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800bb70 tx=0x7f5f180044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123f10738ba83d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:10.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.710+0000 7f5f11600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:39:10.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.711+0000 7f5f11600640 1 -- 172.21.15.47:0/3429084575 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1801b640 con 0x55e2e317c1f0 2024-03-21T11:39:10.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.711+0000 7f5f3107b740 1 -- 172.21.15.47:0/3429084575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f14015070 con 0x55e2e317c1f0 2024-03-21T11:39:10.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.711+0000 7f5f11600640 1 -- 172.21.15.47:0/3429084575 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5f18011440 con 0x55e2e317c1f0 2024-03-21T11:39:10.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.712+0000 7f5f11600640 1 -- 172.21.15.47:0/3429084575 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f1801b7e0 con 0x55e2e317c1f0 2024-03-21T11:39:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.712+0000 7f5f3107b740 10 client.20235 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.712+0000 7f5f3107b740 1 -- 172.21.15.47:0/3429084575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e2e2eb57d0 con 0x55e2e317c1f0 2024-03-21T11:39:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.712+0000 7f5f11600640 1 -- 172.21.15.47:0/3429084575 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f14015070 con 0x55e2e317c1f0 2024-03-21T11:39:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.713+0000 7f5f11600640 1 -- 172.21.15.47:0/3429084575 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f5f18019020 con 0x55e2e317c1f0 2024-03-21T11:39:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.713+0000 7f5f3107b740 10 client.20235 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.713+0000 7f5f3107b740 10 client.20235 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:39:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.713+0000 7f5f3107b740 20 client.20235 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.713+0000 7f5f27e00640 1 --2- 172.21.15.47:0/3429084575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317b020 0x55e2e3185100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.713+0000 7f5f3107b740 1 --2- 172.21.15.47:0/3429084575 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55e2e3190d30 0x55e2e31931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.713+0000 7f5f3107b740 4 client.20235 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:10.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.713+0000 7f5f3107b740 1 -- 172.21.15.47:0/3429084575 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2e30e4800 con 0x55e2e3190d30 2024-03-21T11:39:10.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.713+0000 7f5f27e00640 1 --2- 172.21.15.47:0/3429084575 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55e2e3190d30 0x55e2e31931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.714+0000 7f5f11600640 10 client.20235 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:39:10.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.714+0000 7f5f27e00640 1 --2- 172.21.15.47:0/3429084575 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55e2e3190d30 0x55e2e31931c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=51592ec7a6b8f635 server_cookie=65b032128e8fa166 in_seq=0 out_seq=0 2024-03-21T11:39:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.715+0000 7f5f11600640 1 -- 172.21.15.47:0/3429084575 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e2e30e4800 con 0x55e2e3190d30 2024-03-21T11:39:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.715+0000 7f5f11600640 10 client.20235 handle_command_reply: tid=0 2024-03-21T11:39:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.715+0000 7f5f3107b740 1 client.20235 shutdown 2024-03-21T11:39:10.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.715+0000 7f5f3107b740 1 -- 172.21.15.47:0/3429084575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317c1f0 msgr2=0x55e2e3185640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.715+0000 7f5f3107b740 1 --2- 172.21.15.47:0/3429084575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317c1f0 0x55e2e3185640 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800bb70 tx=0x7f5f180044a0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.716+0000 7f5f3107b740 1 -- 172.21.15.47:0/3429084575 shutdown_connections 2024-03-21T11:39:10.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.716+0000 7f5f3107b740 1 --2- 172.21.15.47:0/3429084575 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55e2e3190d30 0x55e2e31931c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.716+0000 7f5f3107b740 1 --2- 172.21.15.47:0/3429084575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317cbb0 0x55e2e318c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.716+0000 7f5f3107b740 1 --2- 172.21.15.47:0/3429084575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317c1f0 0x55e2e3185640 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.716+0000 7f5f3107b740 1 --2- 172.21.15.47:0/3429084575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317b020 0x55e2e3185100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.716+0000 7f5f3107b740 1 -- 172.21.15.47:0/3429084575 >> 172.21.15.47:0/3429084575 conn(0x55e2e316dc00 msgr2=0x55e2e316ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.716+0000 7f5f11600640 0 client.20235 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:39:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.716+0000 7f5f3107b740 1 -- 172.21.15.47:0/3429084575 shutdown_connections 2024-03-21T11:39:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.716+0000 7f5f3107b740 1 -- 172.21.15.47:0/3429084575 wait complete. 2024-03-21T11:39:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.717+0000 7f5f3107b740 20 client.20235 trim_cache size 0 max 16384 2024-03-21T11:39:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.737+0000 7f5f3107b740 1 Processor -- start 2024-03-21T11:39:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.737+0000 7f5f3107b740 1 -- start start 2024-03-21T11:39:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.738+0000 7f5f3107b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317b020 0x55e2e3171930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.738+0000 7f5f3107b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317c1f0 0x55e2e318c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.738+0000 7f5f3107b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317cbb0 0x55e2e3197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.738+0000 7f5f3107b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f281b06d0 con 0x55e2e317b020 2024-03-21T11:39:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.738+0000 7f5f3107b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f281b21d0 con 0x55e2e317cbb0 2024-03-21T11:39:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.738+0000 7f5f3107b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f281b0d30 con 0x55e2e317c1f0 2024-03-21T11:39:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.738+0000 7f5f27e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317b020 0x55e2e3171930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.738+0000 7f5f26a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317c1f0 0x55e2e318c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.738+0000 7f5f27e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317b020 0x55e2e3171930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52286/0 (socket says 172.21.15.47:52286) 2024-03-21T11:39:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.738+0000 7f5f27e00640 1 -- 172.21.15.47:0/1107288898 learned_addr learned my addr 172.21.15.47:0/1107288898 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.739+0000 7f5f27e00640 1 -- 172.21.15.47:0/1107288898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317c1f0 msgr2=0x55e2e318c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.739+0000 7f5f27e00640 1 --2- 172.21.15.47:0/1107288898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317c1f0 0x55e2e318c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.739+0000 7f5f27e00640 1 -- 172.21.15.47:0/1107288898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317cbb0 msgr2=0x55e2e3197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.739+0000 7f5f27e00640 1 --2- 172.21.15.47:0/1107288898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317cbb0 0x55e2e3197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.739+0000 7f5f27e00640 1 -- 172.21.15.47:0/1107288898 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2e3182c50 con 0x55e2e317b020 2024-03-21T11:39:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.739+0000 7f5f27e00640 1 --2- 172.21.15.47:0/1107288898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317b020 0x55e2e3171930 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55e2e317c0f0 tx=0x7f5f1400b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9a7cc3888078fa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.740+0000 7f5f13e00640 1 -- 172.21.15.47:0/1107288898 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1402e070 con 0x55e2e317b020 2024-03-21T11:39:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.740+0000 7f5f13e00640 1 -- 172.21.15.47:0/1107288898 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f14015050 con 0x55e2e317b020 2024-03-21T11:39:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.740+0000 7f5f13e00640 1 -- 172.21.15.47:0/1107288898 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14011a30 con 0x55e2e317b020 2024-03-21T11:39:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.741+0000 7f5f3107b740 1 -- 172.21.15.47:0/1107288898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317b020 msgr2=0x55e2e3171930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.741+0000 7f5f3107b740 1 --2- 172.21.15.47:0/1107288898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317b020 0x55e2e3171930 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55e2e317c0f0 tx=0x7f5f1400b2e0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.743+0000 7f5f3107b740 1 -- 172.21.15.47:0/1107288898 shutdown_connections 2024-03-21T11:39:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.743+0000 7f5f3107b740 1 --2- 172.21.15.47:0/1107288898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317cbb0 0x55e2e3197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.743+0000 7f5f3107b740 1 --2- 172.21.15.47:0/1107288898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317c1f0 0x55e2e318c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.743+0000 7f5f3107b740 1 --2- 172.21.15.47:0/1107288898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317b020 0x55e2e3171930 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.743+0000 7f5f3107b740 1 -- 172.21.15.47:0/1107288898 >> 172.21.15.47:0/1107288898 conn(0x55e2e316dc00 msgr2=0x55e2e317fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.744+0000 7f5f3107b740 1 -- 172.21.15.47:0/1107288898 shutdown_connections 2024-03-21T11:39:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.744+0000 7f5f3107b740 1 -- 172.21.15.47:0/1107288898 wait complete. 2024-03-21T11:39:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.744+0000 7f5f3107b740 1 -- start start 2024-03-21T11:39:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.745+0000 7f5f3107b740 1 Processor -- start 2024-03-21T11:39:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.747+0000 7f5f3107b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317b020 0x55e2e318a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.747+0000 7f5f3107b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317c1f0 0x55e2e318a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.748+0000 7f5f27e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317b020 0x55e2e318a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.748+0000 7f5f27e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317b020 0x55e2e318a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35850/0 (socket says 172.21.15.47:35850) 2024-03-21T11:39:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.748+0000 7f5f27e00640 1 -- 172.21.15.47:0/482356492 learned_addr learned my addr 172.21.15.47:0/482356492 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.748+0000 7f5f3107b740 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317cbb0 0x55e2e318b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.748+0000 7f5f3107b740 1 -- 172.21.15.47:0/482356492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f281b1ab0 con 0x55e2e317cbb0 2024-03-21T11:39:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.748+0000 7f5f3107b740 1 -- 172.21.15.47:0/482356492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f281b1450 con 0x55e2e317b020 2024-03-21T11:39:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.748+0000 7f5f3107b740 1 -- 172.21.15.47:0/482356492 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f281ae890 con 0x55e2e317c1f0 2024-03-21T11:39:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.749+0000 7f5f3107b740 10 client.0 _ll_register_callbacks cb 0x7f5f18004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.749+0000 7f5f27e00640 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317cbb0 0x55e2e318b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.749+0000 7f5f26a00640 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317c1f0 0x55e2e318a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.749+0000 7f5f27e00640 1 -- 172.21.15.47:0/482356492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317c1f0 msgr2=0x55e2e318a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.750+0000 7f5f27e00640 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317c1f0 0x55e2e318a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.750+0000 7f5f27e00640 1 -- 172.21.15.47:0/482356492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317cbb0 msgr2=0x55e2e318b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.750+0000 7f5f27e00640 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317cbb0 0x55e2e318b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.750+0000 7f5f27e00640 1 -- 172.21.15.47:0/482356492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2e318be30 con 0x55e2e317b020 2024-03-21T11:39:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.750+0000 7f5f26a00640 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317c1f0 0x55e2e318a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.751+0000 7f5f27e00640 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317b020 0x55e2e318a2f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400b680 tx=0x7f5f1400bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17bbded5ff5645db server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.751+0000 7f5f11600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:10.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.752+0000 7f5f11600640 1 -- 172.21.15.47:0/482356492 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1402e040 con 0x55e2e317b020 2024-03-21T11:39:10.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.753+0000 7f5f3107b740 1 -- 172.21.15.47:0/482356492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f14015050 con 0x55e2e317b020 2024-03-21T11:39:10.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.753+0000 7f5f11600640 1 -- 172.21.15.47:0/482356492 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5f14005040 con 0x55e2e317b020 2024-03-21T11:39:10.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.753+0000 7f5f11600640 1 -- 172.21.15.47:0/482356492 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f14011ce0 con 0x55e2e317b020 2024-03-21T11:39:10.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.753+0000 7f5f3107b740 10 client.30719 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:10.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.753+0000 7f5f3107b740 1 -- 172.21.15.47:0/482356492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e2e3178860 con 0x55e2e317b020 2024-03-21T11:39:10.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.754+0000 7f5f11600640 1 -- 172.21.15.47:0/482356492 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f14015050 con 0x55e2e317b020 2024-03-21T11:39:10.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.754+0000 7f5f11600640 1 -- 172.21.15.47:0/482356492 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f5f14029890 con 0x55e2e317b020 2024-03-21T11:39:10.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.755+0000 7f5f3107b740 10 client.30719 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:10.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.755+0000 7f5f3107b740 10 client.30719 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:39:10.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.755+0000 7f5f3107b740 20 client.30719 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:10.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.755+0000 7f5f3107b740 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55e2e31791b0 0x55e2e31795c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:10.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.755+0000 7f5f3107b740 4 client.30719 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:39:10.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.755+0000 7f5f3107b740 1 -- 172.21.15.47:0/482356492 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e2e30deb50 con 0x55e2e31791b0 2024-03-21T11:39:10.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.756+0000 7f5f26a00640 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55e2e31791b0 0x55e2e31795c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:10.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.758+0000 7f5f26a00640 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55e2e31791b0 0x55e2e31795c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f7c19d30316b11cb server_cookie=8dd9a0915e153b9b in_seq=0 out_seq=0 2024-03-21T11:39:10.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.758+0000 7f5f11600640 10 client.30719 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:39:10.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.758+0000 7f5f11600640 1 -- 172.21.15.47:0/482356492 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e2e30deb50 con 0x55e2e31791b0 2024-03-21T11:39:10.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.758+0000 7f5f11600640 10 client.30719 handle_command_reply: tid=0 2024-03-21T11:39:10.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.758+0000 7f5f3107b740 1 client.30719 shutdown 2024-03-21T11:39:10.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.759+0000 7f5f3107b740 1 -- 172.21.15.47:0/482356492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317b020 msgr2=0x55e2e318a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.759+0000 7f5f3107b740 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317b020 0x55e2e318a2f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400b680 tx=0x7f5f1400bb40 comp rx=0 tx=0).stop 2024-03-21T11:39:10.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.759+0000 7f5f3107b740 1 -- 172.21.15.47:0/482356492 shutdown_connections 2024-03-21T11:39:10.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.759+0000 7f5f3107b740 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55e2e31791b0 0x55e2e31795c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.759+0000 7f5f3107b740 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e2e317cbb0 0x55e2e318b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.759+0000 7f5f3107b740 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e2e317c1f0 0x55e2e318a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.759+0000 7f5f3107b740 1 --2- 172.21.15.47:0/482356492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e2e317b020 0x55e2e318a2f0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.759+0000 7f5f3107b740 1 -- 172.21.15.47:0/482356492 >> 172.21.15.47:0/482356492 conn(0x55e2e316dc00 msgr2=0x55e2e316c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:10.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.759+0000 7f5f11600640 0 client.30719 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:39:10.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.759+0000 7f5f3107b740 1 -- 172.21.15.47:0/482356492 shutdown_connections 2024-03-21T11:39:10.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.761+0000 7f5f3107b740 1 -- 172.21.15.47:0/482356492 wait complete. 2024-03-21T11:39:10.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.761+0000 7f5f3107b740 20 client.30719 trim_cache size 0 max 16384 2024-03-21T11:39:10.762 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:39:10.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.764+0000 7f5f2e400640 1 -- 172.21.15.47:0/3431496744 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5f0c0a30f0 msgr2=0x7f5f0c0a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.764+0000 7f5f2e400640 1 --2- 172.21.15.47:0/3431496744 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5f0c0a30f0 0x7f5f0c0a55b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5f20000f80 tx=0x7f5f200093d0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.764+0000 7f5f2e400640 1 -- 172.21.15.47:0/3431496744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f281086a0 msgr2=0x7f5f281a2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:10.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.764+0000 7f5f2e400640 1 --2- 172.21.15.47:0/3431496744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f281086a0 0x7f5f281a2440 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5f2002c640 tx=0x7f5f20035d90 comp rx=0 tx=0).stop 2024-03-21T11:39:10.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.764+0000 7f5f2e400640 1 -- 172.21.15.47:0/3431496744 shutdown_connections 2024-03-21T11:39:10.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.765+0000 7f5f2e400640 1 --2- 172.21.15.47:0/3431496744 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5f0c0a30f0 0x7f5f0c0a55b0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.765+0000 7f5f2e400640 1 --2- 172.21.15.47:0/3431496744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2810ff20 0x7f5f281a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.765+0000 7f5f2e400640 1 --2- 172.21.15.47:0/3431496744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f28109000 0x7f5f281a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.765+0000 7f5f2e400640 1 --2- 172.21.15.47:0/3431496744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f281086a0 0x7f5f281a2440 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:10.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.765+0000 7f5f2e400640 1 -- 172.21.15.47:0/3431496744 >> 172.21.15.47:0/3431496744 conn(0x7f5f2806e500 msgr2=0x7f5f28070230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:10.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.766+0000 7f5f2e400640 1 -- 172.21.15.47:0/3431496744 shutdown_connections 2024-03-21T11:39:10.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:10.766+0000 7f5f2e400640 1 -- 172.21.15.47:0/3431496744 wait complete. 2024-03-21T11:39:10.801 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:39:10.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:39:10.929 INFO:tasks.workunit.client.0.smithi047.stdout: 10 55231 27.59 MB/sec warmup 114 sec latency 134.896 ms 2024-03-21T11:39:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.116+0000 7fa5c5200640 1 -- 172.21.15.47:0/3111047153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c00756e0 msgr2=0x7fa5c0073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.116+0000 7fa5c5200640 1 --2- 172.21.15.47:0/3111047153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c00756e0 0x7fa5c0073b30 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009900 tx=0x7fa5b002bc30 comp rx=0 tx=0).stop 2024-03-21T11:39:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.116+0000 7fa5c5200640 1 -- 172.21.15.47:0/3111047153 shutdown_connections 2024-03-21T11:39:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.116+0000 7fa5c5200640 1 --2- 172.21.15.47:0/3111047153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5c010dc80 0x7fa5c0110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.116+0000 7fa5c5200640 1 --2- 172.21.15.47:0/3111047153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c0074100 0x7fa5c0074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.116+0000 7fa5c5200640 1 --2- 172.21.15.47:0/3111047153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c00756e0 0x7fa5c0073b30 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.116+0000 7fa5c5200640 1 -- 172.21.15.47:0/3111047153 >> 172.21.15.47:0/3111047153 conn(0x7fa5c00fcc20 msgr2=0x7fa5c00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.117+0000 7fa5c5200640 1 -- 172.21.15.47:0/3111047153 shutdown_connections 2024-03-21T11:39:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.117+0000 7fa5c5200640 1 -- 172.21.15.47:0/3111047153 wait complete. 2024-03-21T11:39:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.118+0000 7fa5c5200640 1 Processor -- start 2024-03-21T11:39:11.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.119+0000 7fa5c5200640 1 -- start start 2024-03-21T11:39:11.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.119+0000 7fa5c5200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5c0074100 0x7fa5c019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.119+0000 7fa5c5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c00756e0 0x7fa5c019e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.119+0000 7fa5c5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c010dc80 0x7fa5c01a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.119+0000 7fa5c5200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c0113900 con 0x7fa5c0074100 2024-03-21T11:39:11.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.119+0000 7fa5c5200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c0113780 con 0x7fa5c00756e0 2024-03-21T11:39:11.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.119+0000 7fa5c5200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c0113a80 con 0x7fa5c010dc80 2024-03-21T11:39:11.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.120+0000 7fa5bde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5c0074100 0x7fa5c019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.120+0000 7fa5bde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5c0074100 0x7fa5c019df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52322/0 (socket says 172.21.15.47:52322) 2024-03-21T11:39:11.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.120+0000 7fa5bde00640 1 -- 172.21.15.47:0/1934166036 learned_addr learned my addr 172.21.15.47:0/1934166036 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.120+0000 7fa5b7e00640 1 --2- 172.21.15.47:0/1934166036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c00756e0 0x7fa5c019e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.120+0000 7fa5be800640 1 --2- 172.21.15.47:0/1934166036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c010dc80 0x7fa5c01a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.121+0000 7fa5b7e00640 1 -- 172.21.15.47:0/1934166036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c010dc80 msgr2=0x7fa5c01a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.121+0000 7fa5b7e00640 1 --2- 172.21.15.47:0/1934166036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c010dc80 0x7fa5c01a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.121+0000 7fa5b7e00640 1 -- 172.21.15.47:0/1934166036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5c0074100 msgr2=0x7fa5c019df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.121+0000 7fa5b7e00640 1 --2- 172.21.15.47:0/1934166036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5c0074100 0x7fa5c019df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.121+0000 7fa5b7e00640 1 -- 172.21.15.47:0/1934166036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c01055a0 con 0x7fa5c00756e0 2024-03-21T11:39:11.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.121+0000 7fa5b7e00640 1 --2- 172.21.15.47:0/1934166036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c00756e0 0x7fa5c019e4d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800e9f0 tx=0x7fa5a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.122+0000 7fa5b6a00640 1 -- 172.21.15.47:0/1934166036 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800ccc0 con 0x7fa5c00756e0 2024-03-21T11:39:11.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.122+0000 7fa5b6a00640 1 -- 172.21.15.47:0/1934166036 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5a8010910 con 0x7fa5c00756e0 2024-03-21T11:39:11.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.122+0000 7fa5b6a00640 1 -- 172.21.15.47:0/1934166036 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8018960 con 0x7fa5c00756e0 2024-03-21T11:39:11.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.122+0000 7fa5c5200640 1 -- 172.21.15.47:0/1934166036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c0105890 con 0x7fa5c00756e0 2024-03-21T11:39:11.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.122+0000 7fa5c5200640 1 -- 172.21.15.47:0/1934166036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c0105d90 con 0x7fa5c00756e0 2024-03-21T11:39:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.128+0000 7fa5c7fbc740 1 Processor -- start 2024-03-21T11:39:11.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.128+0000 7fa5c7fbc740 1 -- start start 2024-03-21T11:39:11.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.129+0000 7fa5c7fbc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bbd810 0x55ec11bbdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.129+0000 7fa5c7fbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11bbee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.129+0000 7fa5c7fbc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbf3a0 0x55ec11bc5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.129+0000 7fa5c7fbc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec11b222c0 con 0x55ec11bbf3a0 2024-03-21T11:39:11.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.129+0000 7fa5c7fbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec11acac90 con 0x55ec11bbe9e0 2024-03-21T11:39:11.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.129+0000 7fa5c7fbc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec11b1e360 con 0x55ec11bbd810 2024-03-21T11:39:11.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.129+0000 7fa5be800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bbd810 0x55ec11bbdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.129+0000 7fa5be800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bbd810 0x55ec11bbdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37358/0 (socket says 172.21.15.47:37358) 2024-03-21T11:39:11.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.130+0000 7fa5be800640 1 -- 172.21.15.47:0/3528068015 learned_addr learned my addr 172.21.15.47:0/3528068015 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.130+0000 7fa5bde00640 1 --2- 172.21.15.47:0/3528068015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11bbee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.130+0000 7fa5be800640 1 -- 172.21.15.47:0/3528068015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 msgr2=0x55ec11bbee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.130+0000 7fa5be800640 1 --2- 172.21.15.47:0/3528068015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11bbee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.130+0000 7fa5be800640 1 -- 172.21.15.47:0/3528068015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbf3a0 msgr2=0x55ec11bc5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.130+0000 7fa5be800640 1 --2- 172.21.15.47:0/3528068015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbf3a0 0x55ec11bc5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.130+0000 7fa5be800640 1 -- 172.21.15.47:0/3528068015 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec11aef080 con 0x55ec11bbd810 2024-03-21T11:39:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.131+0000 7fa5be800640 1 --2- 172.21.15.47:0/3528068015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bbd810 0x55ec11bbdc30 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8007e50 tx=0x7fa5b800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3b6b37d50ab8e40 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.132+0000 7fa5af400640 1 -- 172.21.15.47:0/3528068015 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8014070 con 0x55ec11bbd810 2024-03-21T11:39:11.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.132+0000 7fa5af400640 1 -- 172.21.15.47:0/3528068015 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5b800ee90 con 0x55ec11bbd810 2024-03-21T11:39:11.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.132+0000 7fa5af400640 1 -- 172.21.15.47:0/3528068015 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b80054c0 con 0x55ec11bbd810 2024-03-21T11:39:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.134+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3528068015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bbd810 msgr2=0x55ec11bbdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.134+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/3528068015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bbd810 0x55ec11bbdc30 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8007e50 tx=0x7fa5b800e720 comp rx=0 tx=0).stop 2024-03-21T11:39:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.134+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3528068015 shutdown_connections 2024-03-21T11:39:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.134+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/3528068015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbf3a0 0x55ec11bc5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.134+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/3528068015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11bbee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.134+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/3528068015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bbd810 0x55ec11bbdc30 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.134+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3528068015 >> 172.21.15.47:0/3528068015 conn(0x55ec11bb8c00 msgr2=0x55ec11bb9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.136+0000 7fa5b6a00640 1 -- 172.21.15.47:0/1934166036 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa5a801f020 con 0x7fa5c00756e0 2024-03-21T11:39:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.138+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3528068015 shutdown_connections 2024-03-21T11:39:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.138+0000 7fa5b6a00640 1 --2- 172.21.15.47:0/1934166036 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa5a40a2f80 0x7fa5a40a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.138+0000 7fa5b6a00640 1 -- 172.21.15.47:0/1934166036 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa5a80d1b10 con 0x7fa5c00756e0 2024-03-21T11:39:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.138+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3528068015 wait complete. 2024-03-21T11:39:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.139+0000 7fa5bde00640 1 --2- 172.21.15.47:0/1934166036 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa5a40a2f80 0x7fa5a40a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.140+0000 7fa5c7fbc740 1 -- start start 2024-03-21T11:39:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.140+0000 7fa5c7fbc740 1 Processor -- start 2024-03-21T11:39:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.140+0000 7fa5bde00640 1 --2- 172.21.15.47:0/1934166036 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa5a40a2f80 0x7fa5a40a5440 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa5b002c7e0 tx=0x7fa5b0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.141+0000 7fa5c7fbc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbd810 0x55ec11bc7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.142+0000 7fa5c7fbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11bc8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.142+0000 7fa5c7fbc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bcd140 0x55ec11bcf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.142+0000 7fa5c7fbc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec11bc63b0 con 0x55ec11bbd810 2024-03-21T11:39:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.142+0000 7fa5c7fbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec11ade0d0 con 0x55ec11bbe9e0 2024-03-21T11:39:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.142+0000 7fa5c7fbc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec11bc6530 con 0x55ec11bcd140 2024-03-21T11:39:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.142+0000 7fa5c7fbc740 10 client.0 _ll_register_callbacks cb 0x55ec11ae2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.142+0000 7fa5b7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11bc8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.142+0000 7fa5b7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11bc8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35888/0 (socket says 172.21.15.47:35888) 2024-03-21T11:39:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.142+0000 7fa5b7e00640 1 -- 172.21.15.47:0/1859691835 learned_addr learned my addr 172.21.15.47:0/1859691835 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.143+0000 7fa5b7e00640 1 -- 172.21.15.47:0/1859691835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bcd140 msgr2=0x55ec11bcf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.143+0000 7fa5b7e00640 1 --2- 172.21.15.47:0/1859691835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bcd140 0x55ec11bcf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.143+0000 7fa5b7e00640 1 -- 172.21.15.47:0/1859691835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbd810 msgr2=0x55ec11bc7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.143+0000 7fa5b7e00640 1 --2- 172.21.15.47:0/1859691835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbd810 0x55ec11bc7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.143+0000 7fa5b7e00640 1 -- 172.21.15.47:0/1859691835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec118363f0 con 0x55ec11bbe9e0 2024-03-21T11:39:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.143+0000 7fa5a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.143+0000 7fa5b7e00640 1 --2- 172.21.15.47:0/1859691835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11bc8110 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80a6390 tx=0x7fa5a809d8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e26454ccc86cfd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.144+0000 7fa5a3e00640 1 -- 172.21.15.47:0/1859691835 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80b6070 con 0x55ec11bbe9e0 2024-03-21T11:39:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.144+0000 7fa5a3e00640 1 -- 172.21.15.47:0/1859691835 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa5a809bf50 con 0x55ec11bbe9e0 2024-03-21T11:39:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.144+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/1859691835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5b800ee90 con 0x55ec11bbe9e0 2024-03-21T11:39:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.144+0000 7fa5a3e00640 1 -- 172.21.15.47:0/1859691835 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5a80b6070 con 0x55ec11bbe9e0 2024-03-21T11:39:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.144+0000 7fa5c7fbc740 10 client.30731 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.144+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/1859691835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ec119007d0 con 0x55ec11bbe9e0 2024-03-21T11:39:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.145+0000 7fa5a3e00640 1 -- 172.21.15.47:0/1859691835 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5a80a2000 con 0x55ec11bbe9e0 2024-03-21T11:39:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.145+0000 7fa5a3e00640 1 -- 172.21.15.47:0/1859691835 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fa5a809d300 con 0x55ec11bbe9e0 2024-03-21T11:39:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.145+0000 7fa5c7fbc740 10 client.30731 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.145+0000 7fa5c7fbc740 10 client.30731 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:39:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.145+0000 7fa5c7fbc740 20 client.30731 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.145+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/1859691835 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55ec11bd3f80 0x55ec11bd6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.145+0000 7fa5c7fbc740 4 client.30731 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.145+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/1859691835 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5b800ee90 con 0x55ec11bd3f80 2024-03-21T11:39:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.146+0000 7fa5bde00640 1 --2- 172.21.15.47:0/1859691835 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55ec11bd3f80 0x55ec11bd6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.153+0000 7fa5bde00640 1 --2- 172.21.15.47:0/1859691835 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55ec11bd3f80 0x55ec11bd6420 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28e08d25392368ec server_cookie=f4acac469615acfd in_seq=0 out_seq=0 2024-03-21T11:39:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.153+0000 7fa5a3e00640 10 client.30731 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:39:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.156+0000 7fa5a3e00640 1 -- 172.21.15.47:0/1859691835 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fa5b800ee90 con 0x55ec11bd3f80 2024-03-21T11:39:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.156+0000 7fa5a3e00640 10 client.30731 handle_command_reply: tid=0 2024-03-21T11:39:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.157+0000 7fa5c7fbc740 1 client.30731 shutdown 2024-03-21T11:39:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.157+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/1859691835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 msgr2=0x55ec11bc8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.157+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/1859691835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11bc8110 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80a6390 tx=0x7fa5a809d8d0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.158+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/1859691835 shutdown_connections 2024-03-21T11:39:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.158+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/1859691835 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55ec11bd3f80 0x55ec11bd6420 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.158+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/1859691835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bcd140 0x55ec11bcf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.158+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/1859691835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11bc8110 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.158+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/1859691835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbd810 0x55ec11bc7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.158+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/1859691835 >> 172.21.15.47:0/1859691835 conn(0x55ec11bb8c00 msgr2=0x55ec11bb7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.158+0000 7fa5a3e00640 0 client.30731 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:39:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.159+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/1859691835 shutdown_connections 2024-03-21T11:39:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.159+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/1859691835 wait complete. 2024-03-21T11:39:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.159+0000 7fa5c7fbc740 20 client.30731 trim_cache size 0 max 16384 2024-03-21T11:39:11.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.179+0000 7fa5c7fbc740 1 Processor -- start 2024-03-21T11:39:11.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.180+0000 7fa5c7fbc740 1 -- start start 2024-03-21T11:39:11.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.181+0000 7fa5c7fbc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bbd810 0x55ec11bbc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.181+0000 7fa5c7fbc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbe9e0 0x55ec11bc1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.181+0000 7fa5c7fbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bcd140 0x55ec11bc20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.181+0000 7fa5c7fbc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01b9180 con 0x55ec11bbe9e0 2024-03-21T11:39:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.181+0000 7fa5c7fbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01bac80 con 0x55ec11bcd140 2024-03-21T11:39:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.181+0000 7fa5c7fbc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01b97e0 con 0x55ec11bbd810 2024-03-21T11:39:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.181+0000 7fa5bde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bbd810 0x55ec11bbc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.181+0000 7fa5bde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bbd810 0x55ec11bbc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37380/0 (socket says 172.21.15.47:37380) 2024-03-21T11:39:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.181+0000 7fa5bde00640 1 -- 172.21.15.47:0/3414870647 learned_addr learned my addr 172.21.15.47:0/3414870647 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.181+0000 7fa5b7e00640 1 --2- 172.21.15.47:0/3414870647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbe9e0 0x55ec11bc1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.181+0000 7fa5be800640 1 --2- 172.21.15.47:0/3414870647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bcd140 0x55ec11bc20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.182+0000 7fa5bde00640 1 -- 172.21.15.47:0/3414870647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bcd140 msgr2=0x55ec11bc20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.182+0000 7fa5bde00640 1 --2- 172.21.15.47:0/3414870647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bcd140 0x55ec11bc20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.182+0000 7fa5bde00640 1 -- 172.21.15.47:0/3414870647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbe9e0 msgr2=0x55ec11bc1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.182+0000 7fa5bde00640 1 --2- 172.21.15.47:0/3414870647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbe9e0 0x55ec11bc1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.182+0000 7fa5bde00640 1 -- 172.21.15.47:0/3414870647 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec11bb9210 con 0x55ec11bbd810 2024-03-21T11:39:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.182+0000 7fa5b7e00640 1 --2- 172.21.15.47:0/3414870647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbe9e0 0x55ec11bc1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:11.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.182+0000 7fa5bde00640 1 --2- 172.21.15.47:0/3414870647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bbd810 0x55ec11bbc570 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0000c00 tx=0x7fa5b002e070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4c30bb76f654ea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.183+0000 7fa5aea00640 1 -- 172.21.15.47:0/3414870647 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0059070 con 0x55ec11bbd810 2024-03-21T11:39:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.183+0000 7fa5aea00640 1 -- 172.21.15.47:0/3414870647 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5b0004ef0 con 0x55ec11bbd810 2024-03-21T11:39:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.183+0000 7fa5aea00640 1 -- 172.21.15.47:0/3414870647 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0053860 con 0x55ec11bbd810 2024-03-21T11:39:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.184+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3414870647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bbd810 msgr2=0x55ec11bbc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.184+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/3414870647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bbd810 0x55ec11bbc570 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0000c00 tx=0x7fa5b002e070 comp rx=0 tx=0).stop 2024-03-21T11:39:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.184+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3414870647 shutdown_connections 2024-03-21T11:39:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.184+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/3414870647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bcd140 0x55ec11bc20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.184+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/3414870647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbe9e0 0x55ec11bc1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.184+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/3414870647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bbd810 0x55ec11bbc570 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.184+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3414870647 >> 172.21.15.47:0/3414870647 conn(0x55ec11bb8c00 msgr2=0x55ec11bbbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.184+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3414870647 shutdown_connections 2024-03-21T11:39:11.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.184+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3414870647 wait complete. 2024-03-21T11:39:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.184+0000 7fa5c7fbc740 1 -- start start 2024-03-21T11:39:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.185+0000 7fa5c7fbc740 1 Processor -- start 2024-03-21T11:39:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.185+0000 7fa5c7fbc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbd810 0x55ec11be3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.185+0000 7fa5c7fbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11be39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.185+0000 7fa5c7fbc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bcd140 0x55ec11c2eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.185+0000 7fa5c7fbc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01ba560 con 0x55ec11bbd810 2024-03-21T11:39:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.185+0000 7fa5c7fbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01b9f00 con 0x55ec11bbe9e0 2024-03-21T11:39:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.185+0000 7fa5c7fbc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01b73a0 con 0x55ec11bcd140 2024-03-21T11:39:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.185+0000 7fa5c7fbc740 10 client.0 _ll_register_callbacks cb 0x7fa5a800ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.186+0000 7fa5b7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11be39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.186+0000 7fa5b7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11be39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35900/0 (socket says 172.21.15.47:35900) 2024-03-21T11:39:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.186+0000 7fa5b7e00640 1 -- 172.21.15.47:0/3275972333 learned_addr learned my addr 172.21.15.47:0/3275972333 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.186+0000 7fa5bde00640 1 --2- 172.21.15.47:0/3275972333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbd810 0x55ec11be3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.186+0000 7fa5b7e00640 1 -- 172.21.15.47:0/3275972333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bcd140 msgr2=0x55ec11c2eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.186+0000 7fa5b7e00640 1 --2- 172.21.15.47:0/3275972333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bcd140 0x55ec11c2eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.186+0000 7fa5b7e00640 1 -- 172.21.15.47:0/3275972333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbd810 msgr2=0x55ec11be3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.186+0000 7fa5b7e00640 1 --2- 172.21.15.47:0/3275972333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbd810 0x55ec11be3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.186+0000 7fa5b7e00640 1 -- 172.21.15.47:0/3275972333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec11c2f2e0 con 0x55ec11bbe9e0 2024-03-21T11:39:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.186+0000 7fa5bde00640 1 --2- 172.21.15.47:0/3275972333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbd810 0x55ec11be3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:39:11.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.187+0000 7fa5b7e00640 1 --2- 172.21.15.47:0/3275972333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11be39d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa5a809de40 tx=0x7fa5a8009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90c8dc1da2b51915 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.187+0000 7fa5a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:11.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.187+0000 7fa5a3e00640 1 -- 172.21.15.47:0/3275972333 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80b6070 con 0x55ec11bbe9e0 2024-03-21T11:39:11.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.187+0000 7fa5a3e00640 1 -- 172.21.15.47:0/3275972333 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa5a809ec50 con 0x55ec11bbe9e0 2024-03-21T11:39:11.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.187+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3275972333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5b0004ef0 con 0x55ec11bbe9e0 2024-03-21T11:39:11.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.188+0000 7fa5a3e00640 1 -- 172.21.15.47:0/3275972333 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5a809dc80 con 0x55ec11bbe9e0 2024-03-21T11:39:11.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.188+0000 7fa5c7fbc740 10 client.30737 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:11.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.188+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3275972333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ec11c30580 con 0x55ec11bbe9e0 2024-03-21T11:39:11.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.188+0000 7fa5a3e00640 1 -- 172.21.15.47:0/3275972333 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5b0004ef0 con 0x55ec11bbe9e0 2024-03-21T11:39:11.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.188+0000 7fa5a3e00640 1 -- 172.21.15.47:0/3275972333 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fa5a809d300 con 0x55ec11bbe9e0 2024-03-21T11:39:11.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.189+0000 7fa5c7fbc740 10 client.30737 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:11.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.189+0000 7fa5c7fbc740 10 client.30737 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:39:11.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.189+0000 7fa5c7fbc740 20 client.30737 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:11.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.189+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/3275972333 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55ec11c30de0 0x55ec11c33200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.189+0000 7fa5c7fbc740 4 client.30737 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:39:11.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.189+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3275972333 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ec11b2f800 con 0x55ec11c30de0 2024-03-21T11:39:11.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.191+0000 7fa5bde00640 1 --2- 172.21.15.47:0/3275972333 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55ec11c30de0 0x55ec11c33200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.191+0000 7fa5bde00640 1 --2- 172.21.15.47:0/3275972333 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55ec11c30de0 0x55ec11c33200 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7aff9d465b60504a server_cookie=8d55b6e73761704a in_seq=0 out_seq=0 2024-03-21T11:39:11.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.191+0000 7fa5a3e00640 10 client.30737 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:39:11.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.192+0000 7fa5a3e00640 1 -- 172.21.15.47:0/3275972333 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ec11b2f800 con 0x55ec11c30de0 2024-03-21T11:39:11.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.192+0000 7fa5a3e00640 10 client.30737 handle_command_reply: tid=0 2024-03-21T11:39:11.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.192+0000 7fa5c7fbc740 1 client.30737 shutdown 2024-03-21T11:39:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.193+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3275972333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 msgr2=0x55ec11be39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.193+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/3275972333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11be39d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa5a809de40 tx=0x7fa5a8009700 comp rx=0 tx=0).stop 2024-03-21T11:39:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.193+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3275972333 shutdown_connections 2024-03-21T11:39:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.193+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/3275972333 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55ec11c30de0 0x55ec11c33200 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.193+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/3275972333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec11bcd140 0x55ec11c2eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.193+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/3275972333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec11bbe9e0 0x55ec11be39d0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.193+0000 7fa5c7fbc740 1 --2- 172.21.15.47:0/3275972333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec11bbd810 0x55ec11be3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.193+0000 7fa5a3e00640 0 client.30737 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:39:11.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.193+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3275972333 >> 172.21.15.47:0/3275972333 conn(0x55ec11bb8c00 msgr2=0x55ec11bbc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.194+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3275972333 shutdown_connections 2024-03-21T11:39:11.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.194+0000 7fa5c7fbc740 1 -- 172.21.15.47:0/3275972333 wait complete. 2024-03-21T11:39:11.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.194+0000 7fa5c7fbc740 20 client.30737 trim_cache size 0 max 16384 2024-03-21T11:39:11.195 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:39:11.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.195+0000 7fa5c5200640 1 -- 172.21.15.47:0/1934166036 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa5a40a2f80 msgr2=0x7fa5a40a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.195+0000 7fa5c5200640 1 --2- 172.21.15.47:0/1934166036 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa5a40a2f80 0x7fa5a40a5440 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa5b002c7e0 tx=0x7fa5b0005fd0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.195+0000 7fa5c5200640 1 -- 172.21.15.47:0/1934166036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c00756e0 msgr2=0x7fa5c019e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.195+0000 7fa5c5200640 1 --2- 172.21.15.47:0/1934166036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c00756e0 0x7fa5c019e4d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800e9f0 tx=0x7fa5a800eeb0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.197+0000 7fa5c5200640 1 -- 172.21.15.47:0/1934166036 shutdown_connections 2024-03-21T11:39:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.197+0000 7fa5c5200640 1 --2- 172.21.15.47:0/1934166036 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa5a40a2f80 0x7fa5a40a5440 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.197+0000 7fa5c5200640 1 --2- 172.21.15.47:0/1934166036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5c010dc80 0x7fa5c01a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.197+0000 7fa5c5200640 1 --2- 172.21.15.47:0/1934166036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c00756e0 0x7fa5c019e4d0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.197+0000 7fa5c5200640 1 --2- 172.21.15.47:0/1934166036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5c0074100 0x7fa5c019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.197+0000 7fa5c5200640 1 -- 172.21.15.47:0/1934166036 >> 172.21.15.47:0/1934166036 conn(0x7fa5c00fcc20 msgr2=0x7fa5c0101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.199+0000 7fa5c5200640 1 -- 172.21.15.47:0/1934166036 shutdown_connections 2024-03-21T11:39:11.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.199+0000 7fa5c5200640 1 -- 172.21.15.47:0/1934166036 wait complete. 2024-03-21T11:39:11.223 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:39:11.224 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:39:11.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:11 smithi100 ceph-mon[36245]: pgmap v875: 129 pgs: 129 active+clean; 397 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 14 MiB/s wr, 781 op/s 2024-03-21T11:39:11.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:11 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:39:11.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:11 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:39:11.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:11 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:39:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.468+0000 7f80c7e00640 1 -- 172.21.15.47:0/4267837448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c81037a0 msgr2=0x7f80c8103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.468+0000 7f80c7e00640 1 --2- 172.21.15.47:0/4267837448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c81037a0 0x7f80c8103bc0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009900 tx=0x7f80b402bc30 comp rx=0 tx=0).stop 2024-03-21T11:39:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.469+0000 7f80c7e00640 1 -- 172.21.15.47:0/4267837448 shutdown_connections 2024-03-21T11:39:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.469+0000 7f80c7e00640 1 --2- 172.21.15.47:0/4267837448 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80c8105350 0x7f80c810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.469+0000 7f80c7e00640 1 --2- 172.21.15.47:0/4267837448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c8104990 0x7f80c8104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.469+0000 7f80c7e00640 1 --2- 172.21.15.47:0/4267837448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c81037a0 0x7f80c8103bc0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.469+0000 7f80c7e00640 1 -- 172.21.15.47:0/4267837448 >> 172.21.15.47:0/4267837448 conn(0x7f80c80fed10 msgr2=0x7f80c8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.469+0000 7f80c7e00640 1 -- 172.21.15.47:0/4267837448 shutdown_connections 2024-03-21T11:39:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.469+0000 7f80c7e00640 1 -- 172.21.15.47:0/4267837448 wait complete. 2024-03-21T11:39:11.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.470+0000 7f80c7e00640 1 Processor -- start 2024-03-21T11:39:11.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.470+0000 7f80c7e00640 1 -- start start 2024-03-21T11:39:11.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.470+0000 7f80c7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80c81037a0 0x7f80c8199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.470+0000 7f80c7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8104990 0x7f80c8199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.470+0000 7f80c7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c8105350 0x7f80c81a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.470+0000 7f80c7e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c810f440 con 0x7f80c81037a0 2024-03-21T11:39:11.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.470+0000 7f80c7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c810f2c0 con 0x7f80c8105350 2024-03-21T11:39:11.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.470+0000 7f80c7e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c810f5c0 con 0x7f80c8104990 2024-03-21T11:39:11.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.471+0000 7f80c6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8104990 0x7f80c8199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.471+0000 7f80c6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80c81037a0 0x7f80c8199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.471+0000 7f80c6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8104990 0x7f80c8199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37400/0 (socket says 172.21.15.47:37400) 2024-03-21T11:39:11.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.471+0000 7f80c6000640 1 -- 172.21.15.47:0/4138685869 learned_addr learned my addr 172.21.15.47:0/4138685869 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.471+0000 7f80c7400640 1 --2- 172.21.15.47:0/4138685869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c8105350 0x7f80c81a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.471+0000 7f80c6000640 1 -- 172.21.15.47:0/4138685869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c8105350 msgr2=0x7f80c81a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.471+0000 7f80c6000640 1 --2- 172.21.15.47:0/4138685869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c8105350 0x7f80c81a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.471+0000 7f80c6000640 1 -- 172.21.15.47:0/4138685869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80c81037a0 msgr2=0x7f80c8199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.471+0000 7f80c6000640 1 --2- 172.21.15.47:0/4138685869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80c81037a0 0x7f80c8199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.471+0000 7f80c6000640 1 -- 172.21.15.47:0/4138685869 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c81a16d0 con 0x7f80c8104990 2024-03-21T11:39:11.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.471+0000 7f80c6000640 1 --2- 172.21.15.47:0/4138685869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8104990 0x7f80c8199ff0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f80b800e9f0 tx=0x7f80b800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.472+0000 7f80bf400640 1 -- 172.21.15.47:0/4138685869 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b800ccc0 con 0x7f80c8104990 2024-03-21T11:39:11.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.472+0000 7f80c7e00640 1 -- 172.21.15.47:0/4138685869 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c81a19c0 con 0x7f80c8104990 2024-03-21T11:39:11.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.472+0000 7f80bf400640 1 -- 172.21.15.47:0/4138685869 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80b8010910 con 0x7f80c8104990 2024-03-21T11:39:11.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.472+0000 7f80bf400640 1 -- 172.21.15.47:0/4138685869 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b80188a0 con 0x7f80c8104990 2024-03-21T11:39:11.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.472+0000 7f80c7e00640 1 -- 172.21.15.47:0/4138685869 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c81a1f10 con 0x7f80c8104990 2024-03-21T11:39:11.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.475+0000 7f80bf400640 1 -- 172.21.15.47:0/4138685869 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f80b8010ac0 con 0x7f80c8104990 2024-03-21T11:39:11.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.476+0000 7f80d062d740 1 Processor -- start 2024-03-21T11:39:11.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.476+0000 7f80d062d740 1 -- start start 2024-03-21T11:39:11.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.476+0000 7f80bf400640 1 --2- 172.21.15.47:0/4138685869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80ac0a2f20 0x7f80ac0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.476+0000 7f80d062d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe1349f0 0x55c3fe134e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.476+0000 7f80d062d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe135bc0 0x55c3fe136040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.476+0000 7f80d062d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe136580 0x55c3fe13ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.476+0000 7f80d062d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3fe0992c0 con 0x55c3fe135bc0 2024-03-21T11:39:11.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.476+0000 7f80d062d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3fe041c90 con 0x55c3fe1349f0 2024-03-21T11:39:11.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.476+0000 7f80d062d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3fe095360 con 0x55c3fe136580 2024-03-21T11:39:11.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.477+0000 7f80bf400640 1 -- 172.21.15.47:0/4138685869 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f80b80cfbf0 con 0x7f80c8104990 2024-03-21T11:39:11.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.477+0000 7f80c7400640 1 --2- 172.21.15.47:0/4138685869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80ac0a2f20 0x7f80ac0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.477+0000 7f80c6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe1349f0 0x55c3fe134e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.477+0000 7f80c6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe1349f0 0x55c3fe134e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35930/0 (socket says 172.21.15.47:35930) 2024-03-21T11:39:11.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.477+0000 7f80c6a00640 1 -- 172.21.15.47:0/3690802237 learned_addr learned my addr 172.21.15.47:0/3690802237 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.477+0000 7f80c6000640 1 --2- 172.21.15.47:0/3690802237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe135bc0 0x55c3fe136040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.477+0000 7f80c7400640 1 --2- 172.21.15.47:0/3690802237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe136580 0x55c3fe13ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.477+0000 7f80c6a00640 1 -- 172.21.15.47:0/3690802237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe136580 msgr2=0x55c3fe13ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.477+0000 7f80c6a00640 1 --2- 172.21.15.47:0/3690802237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe136580 0x55c3fe13ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.477+0000 7f80c6a00640 1 -- 172.21.15.47:0/3690802237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe135bc0 msgr2=0x55c3fe136040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.477+0000 7f80c6a00640 1 --2- 172.21.15.47:0/3690802237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe135bc0 0x55c3fe136040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.477+0000 7f80c6a00640 1 -- 172.21.15.47:0/3690802237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3fe066080 con 0x55c3fe1349f0 2024-03-21T11:39:11.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.477+0000 7f80c7400640 1 --2- 172.21.15.47:0/3690802237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe136580 0x55c3fe13ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:39:11.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.478+0000 7f80c7400640 1 --2- 172.21.15.47:0/4138685869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80ac0a2f20 0x7f80ac0a53e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f80c0005fd0 tx=0x7f80c0009900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.478+0000 7f80c6a00640 1 --2- 172.21.15.47:0/3690802237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe1349f0 0x55c3fe134e10 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f80b4000c00 tx=0x7f80b4009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2f007200d0b3a94 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.478+0000 7f80b3e00640 1 -- 172.21.15.47:0/3690802237 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b402c980 con 0x55c3fe1349f0 2024-03-21T11:39:11.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.478+0000 7f80b3e00640 1 -- 172.21.15.47:0/3690802237 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80b402cb20 con 0x55c3fe1349f0 2024-03-21T11:39:11.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.478+0000 7f80b3e00640 1 -- 172.21.15.47:0/3690802237 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b40046c0 con 0x55c3fe1349f0 2024-03-21T11:39:11.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.479+0000 7f80d062d740 1 -- 172.21.15.47:0/3690802237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe1349f0 msgr2=0x55c3fe134e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.479+0000 7f80d062d740 1 --2- 172.21.15.47:0/3690802237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe1349f0 0x55c3fe134e10 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f80b4000c00 tx=0x7f80b4009560 comp rx=0 tx=0).stop 2024-03-21T11:39:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.479+0000 7f80d062d740 1 -- 172.21.15.47:0/3690802237 shutdown_connections 2024-03-21T11:39:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.479+0000 7f80d062d740 1 --2- 172.21.15.47:0/3690802237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe136580 0x55c3fe13ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.479+0000 7f80d062d740 1 --2- 172.21.15.47:0/3690802237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe135bc0 0x55c3fe136040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.479+0000 7f80d062d740 1 --2- 172.21.15.47:0/3690802237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe1349f0 0x55c3fe134e10 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.479+0000 7f80d062d740 1 -- 172.21.15.47:0/3690802237 >> 172.21.15.47:0/3690802237 conn(0x55c3fe12fde0 msgr2=0x55c3fe1301e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.479+0000 7f80d062d740 1 -- 172.21.15.47:0/3690802237 shutdown_connections 2024-03-21T11:39:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.479+0000 7f80d062d740 1 -- 172.21.15.47:0/3690802237 wait complete. 2024-03-21T11:39:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.480+0000 7f80d062d740 1 -- start start 2024-03-21T11:39:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.480+0000 7f80d062d740 1 Processor -- start 2024-03-21T11:39:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.480+0000 7f80d062d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe1349f0 0x55c3fe13ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.480+0000 7f80d062d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe135bc0 0x55c3fe13f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.480+0000 7f80d062d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe136580 0x55c3fe146150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.480+0000 7f80d062d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3fe13d530 con 0x55c3fe135bc0 2024-03-21T11:39:11.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.480+0000 7f80d062d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3fe0550d0 con 0x55c3fe136580 2024-03-21T11:39:11.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.480+0000 7f80d062d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3fe13d6b0 con 0x55c3fe1349f0 2024-03-21T11:39:11.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.480+0000 7f80d062d740 10 client.0 _ll_register_callbacks cb 0x55c3fe059e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:11.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.480+0000 7f80c6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe1349f0 0x55c3fe13ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.480+0000 7f80c6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe1349f0 0x55c3fe13ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37422/0 (socket says 172.21.15.47:37422) 2024-03-21T11:39:11.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.480+0000 7f80c6a00640 1 -- 172.21.15.47:0/2398456739 learned_addr learned my addr 172.21.15.47:0/2398456739 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.480+0000 7f80c7400640 1 --2- 172.21.15.47:0/2398456739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe136580 0x55c3fe146150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.481+0000 7f80c6000640 1 --2- 172.21.15.47:0/2398456739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe135bc0 0x55c3fe13f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.481+0000 7f80c6a00640 1 -- 172.21.15.47:0/2398456739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe136580 msgr2=0x55c3fe146150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.481+0000 7f80c6a00640 1 --2- 172.21.15.47:0/2398456739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe136580 0x55c3fe146150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.481+0000 7f80c6a00640 1 -- 172.21.15.47:0/2398456739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe135bc0 msgr2=0x55c3fe13f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.481+0000 7f80c6a00640 1 --2- 172.21.15.47:0/2398456739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe135bc0 0x55c3fe13f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.481+0000 7f80c6a00640 1 -- 172.21.15.47:0/2398456739 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3fddad3f0 con 0x55c3fe1349f0 2024-03-21T11:39:11.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.481+0000 7f80c6000640 1 --2- 172.21.15.47:0/2398456739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe135bc0 0x55c3fe13f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:39:11.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.481+0000 7f80c6a00640 1 --2- 172.21.15.47:0/2398456739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe1349f0 0x55c3fe13ebd0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f80b402c240 tx=0x7f80b403f960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bae08582e94218a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.481+0000 7f80b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:11.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.481+0000 7f80b0c00640 1 -- 172.21.15.47:0/2398456739 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b402c980 con 0x55c3fe1349f0 2024-03-21T11:39:11.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.482+0000 7f80b0c00640 1 -- 172.21.15.47:0/2398456739 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f80b403fd50 con 0x55c3fe1349f0 2024-03-21T11:39:11.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.482+0000 7f80d062d740 1 -- 172.21.15.47:0/2398456739 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80b402cb20 con 0x55c3fe1349f0 2024-03-21T11:39:11.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.482+0000 7f80b0c00640 1 -- 172.21.15.47:0/2398456739 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80b4054440 con 0x55c3fe1349f0 2024-03-21T11:39:11.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.482+0000 7f80d062d740 10 client.30760 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:11.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.482+0000 7f80d062d740 1 -- 172.21.15.47:0/2398456739 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c3fde777d0 con 0x55c3fe1349f0 2024-03-21T11:39:11.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.482+0000 7f80b0c00640 1 -- 172.21.15.47:0/2398456739 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80b402cb20 con 0x55c3fe1349f0 2024-03-21T11:39:11.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.483+0000 7f80b0c00640 1 -- 172.21.15.47:0/2398456739 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f80b4049480 con 0x55c3fe1349f0 2024-03-21T11:39:11.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.483+0000 7f80d062d740 10 client.30760 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:11.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.483+0000 7f80d062d740 10 client.30760 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:39:11.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.483+0000 7f80d062d740 20 client.30760 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:11.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.483+0000 7f80d062d740 1 --2- 172.21.15.47:0/2398456739 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c3fe14a2b0 0x55c3fe14c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.483+0000 7f80d062d740 4 client.30760 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:11.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.483+0000 7f80d062d740 1 -- 172.21.15.47:0/2398456739 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3fe0a6800 con 0x55c3fe14a2b0 2024-03-21T11:39:11.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.483+0000 7f80c6000640 1 --2- 172.21.15.47:0/2398456739 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c3fe14a2b0 0x55c3fe14c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.484+0000 7f80c6000640 1 --2- 172.21.15.47:0/2398456739 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c3fe14a2b0 0x55c3fe14c6a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44144e4211edbec server_cookie=9e52e0ea2d1077e in_seq=0 out_seq=0 2024-03-21T11:39:11.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.484+0000 7f80b0c00640 10 client.30760 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:39:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.485+0000 7f80b0c00640 1 -- 172.21.15.47:0/2398456739 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c3fe0a6800 con 0x55c3fe14a2b0 2024-03-21T11:39:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.485+0000 7f80b0c00640 10 client.30760 handle_command_reply: tid=0 2024-03-21T11:39:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.485+0000 7f80d062d740 1 client.30760 shutdown 2024-03-21T11:39:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.486+0000 7f80d062d740 1 -- 172.21.15.47:0/2398456739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe1349f0 msgr2=0x55c3fe13ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.486+0000 7f80d062d740 1 --2- 172.21.15.47:0/2398456739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe1349f0 0x55c3fe13ebd0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f80b402c240 tx=0x7f80b403f960 comp rx=0 tx=0).stop 2024-03-21T11:39:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.486+0000 7f80d062d740 1 -- 172.21.15.47:0/2398456739 shutdown_connections 2024-03-21T11:39:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.486+0000 7f80d062d740 1 --2- 172.21.15.47:0/2398456739 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c3fe14a2b0 0x55c3fe14c6a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.486+0000 7f80d062d740 1 --2- 172.21.15.47:0/2398456739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe136580 0x55c3fe146150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.486+0000 7f80d062d740 1 --2- 172.21.15.47:0/2398456739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe135bc0 0x55c3fe13f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.486+0000 7f80d062d740 1 --2- 172.21.15.47:0/2398456739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe1349f0 0x55c3fe13ebd0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.486+0000 7f80b0c00640 0 client.30760 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:39:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.486+0000 7f80d062d740 1 -- 172.21.15.47:0/2398456739 >> 172.21.15.47:0/2398456739 conn(0x55c3fe12fde0 msgr2=0x55c3fe12f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.486+0000 7f80d062d740 1 -- 172.21.15.47:0/2398456739 shutdown_connections 2024-03-21T11:39:11.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.486+0000 7f80d062d740 1 -- 172.21.15.47:0/2398456739 wait complete. 2024-03-21T11:39:11.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.486+0000 7f80d062d740 20 client.30760 trim_cache size 0 max 16384 2024-03-21T11:39:11.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:11 smithi102 ceph-mon[36748]: pgmap v875: 129 pgs: 129 active+clean; 397 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 14 MiB/s wr, 781 op/s 2024-03-21T11:39:11.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:11 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:39:11.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:11 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:39:11.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:11 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:39:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.503+0000 7f80d062d740 1 Processor -- start 2024-03-21T11:39:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.503+0000 7f80d062d740 1 -- start start 2024-03-21T11:39:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.503+0000 7f80d062d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe1349f0 0x55c3fe133b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.503+0000 7f80d062d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe135bc0 0x55c3fe150e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.503+0000 7f80d062d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe136580 0x55c3fe151390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.503+0000 7f80d062d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c81ac110 con 0x55c3fe136580 2024-03-21T11:39:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.503+0000 7f80d062d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c81adc10 con 0x55c3fe1349f0 2024-03-21T11:39:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.503+0000 7f80d062d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c81ac770 con 0x55c3fe135bc0 2024-03-21T11:39:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.503+0000 7f80c6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe1349f0 0x55c3fe133b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.503+0000 7f80c7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe136580 0x55c3fe151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.503+0000 7f80c6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe1349f0 0x55c3fe133b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35948/0 (socket says 172.21.15.47:35948) 2024-03-21T11:39:11.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.503+0000 7f80c6a00640 1 -- 172.21.15.47:0/625623139 learned_addr learned my addr 172.21.15.47:0/625623139 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.504+0000 7f80c6000640 1 --2- 172.21.15.47:0/625623139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe135bc0 0x55c3fe150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.504+0000 7f80c6a00640 1 -- 172.21.15.47:0/625623139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe135bc0 msgr2=0x55c3fe150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.504+0000 7f80c6a00640 1 --2- 172.21.15.47:0/625623139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe135bc0 0x55c3fe150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.504+0000 7f80c6a00640 1 -- 172.21.15.47:0/625623139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe136580 msgr2=0x55c3fe151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.504+0000 7f80c6a00640 1 --2- 172.21.15.47:0/625623139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe136580 0x55c3fe151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.504+0000 7f80c6a00640 1 -- 172.21.15.47:0/625623139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3fe13dff0 con 0x55c3fe1349f0 2024-03-21T11:39:11.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.504+0000 7f80c6000640 1 --2- 172.21.15.47:0/625623139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe135bc0 0x55c3fe150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:11.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.504+0000 7f80c7400640 1 --2- 172.21.15.47:0/625623139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe136580 0x55c3fe151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:11.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.504+0000 7f80c6a00640 1 --2- 172.21.15.47:0/625623139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe1349f0 0x55c3fe133b10 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f80b4049820 tx=0x7f80b4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cab8b8261fa40d6a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.505+0000 7f80b3400640 1 -- 172.21.15.47:0/625623139 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4005d50 con 0x55c3fe1349f0 2024-03-21T11:39:11.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.505+0000 7f80b3400640 1 -- 172.21.15.47:0/625623139 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80b4049a90 con 0x55c3fe1349f0 2024-03-21T11:39:11.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.505+0000 7f80b3400640 1 -- 172.21.15.47:0/625623139 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4054a90 con 0x55c3fe1349f0 2024-03-21T11:39:11.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.506+0000 7f80d062d740 1 -- 172.21.15.47:0/625623139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe1349f0 msgr2=0x55c3fe133b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.506+0000 7f80d062d740 1 --2- 172.21.15.47:0/625623139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe1349f0 0x55c3fe133b10 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f80b4049820 tx=0x7f80b4000bc0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.506+0000 7f80d062d740 1 -- 172.21.15.47:0/625623139 shutdown_connections 2024-03-21T11:39:11.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.506+0000 7f80d062d740 1 --2- 172.21.15.47:0/625623139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe136580 0x55c3fe151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.506+0000 7f80d062d740 1 --2- 172.21.15.47:0/625623139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe135bc0 0x55c3fe150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.506+0000 7f80d062d740 1 --2- 172.21.15.47:0/625623139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe1349f0 0x55c3fe133b10 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.506+0000 7f80d062d740 1 -- 172.21.15.47:0/625623139 >> 172.21.15.47:0/625623139 conn(0x55c3fe12fde0 msgr2=0x55c3fe12f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.506+0000 7f80d062d740 1 -- 172.21.15.47:0/625623139 shutdown_connections 2024-03-21T11:39:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.506+0000 7f80d062d740 1 -- 172.21.15.47:0/625623139 wait complete. 2024-03-21T11:39:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.506+0000 7f80d062d740 1 -- start start 2024-03-21T11:39:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.507+0000 7f80d062d740 1 Processor -- start 2024-03-21T11:39:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.507+0000 7f80d062d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe1349f0 0x55c3fe1436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.507+0000 7f80d062d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe135bc0 0x55c3fe143c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.507+0000 7f80d062d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe136580 0x55c3fe144c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.507+0000 7f80d062d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c81ad4f0 con 0x55c3fe136580 2024-03-21T11:39:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.507+0000 7f80d062d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c81ace90 con 0x55c3fe135bc0 2024-03-21T11:39:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.507+0000 7f80d062d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c81aa2d0 con 0x55c3fe1349f0 2024-03-21T11:39:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.507+0000 7f80d062d740 10 client.0 _ll_register_callbacks cb 0x7f80b8097ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:11.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.507+0000 7f80c6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe135bc0 0x55c3fe143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.507+0000 7f80c6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe1349f0 0x55c3fe1436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.507+0000 7f80c6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe135bc0 0x55c3fe143c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35956/0 (socket says 172.21.15.47:35956) 2024-03-21T11:39:11.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.507+0000 7f80c7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe136580 0x55c3fe144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.507+0000 7f80c6000640 1 -- 172.21.15.47:0/2159407379 learned_addr learned my addr 172.21.15.47:0/2159407379 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.508+0000 7f80c7400640 1 -- 172.21.15.47:0/2159407379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe1349f0 msgr2=0x55c3fe1436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.508+0000 7f80c7400640 1 --2- 172.21.15.47:0/2159407379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe1349f0 0x55c3fe1436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.508+0000 7f80c7400640 1 -- 172.21.15.47:0/2159407379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe135bc0 msgr2=0x55c3fe143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.508+0000 7f80c7400640 1 --2- 172.21.15.47:0/2159407379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe135bc0 0x55c3fe143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.508+0000 7f80c7400640 1 -- 172.21.15.47:0/2159407379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3fe145210 con 0x55c3fe136580 2024-03-21T11:39:11.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.508+0000 7f80c6a00640 1 --2- 172.21.15.47:0/2159407379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe1349f0 0x55c3fe1436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:11.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.508+0000 7f80c7400640 1 --2- 172.21.15.47:0/2159407379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe136580 0x55c3fe144c20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f80c0005ee0 tx=0x7f80c0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb911608963a1941 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.508+0000 7f80b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:39:11.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.509+0000 7f80b0c00640 1 -- 172.21.15.47:0/2159407379 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c00047a0 con 0x55c3fe136580 2024-03-21T11:39:11.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.509+0000 7f80d062d740 1 -- 172.21.15.47:0/2159407379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80b4049a90 con 0x55c3fe136580 2024-03-21T11:39:11.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.509+0000 7f80b0c00640 1 -- 172.21.15.47:0/2159407379 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f80c0003e20 con 0x55c3fe136580 2024-03-21T11:39:11.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.509+0000 7f80b0c00640 1 -- 172.21.15.47:0/2159407379 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80c0022450 con 0x55c3fe136580 2024-03-21T11:39:11.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.509+0000 7f80d062d740 10 client.20268 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:11.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.509+0000 7f80d062d740 1 -- 172.21.15.47:0/2159407379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c3fe1a60f0 con 0x55c3fe136580 2024-03-21T11:39:11.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.509+0000 7f80b0c00640 1 -- 172.21.15.47:0/2159407379 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80b4049a90 con 0x55c3fe136580 2024-03-21T11:39:11.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.509+0000 7f80b0c00640 1 -- 172.21.15.47:0/2159407379 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f80c002a070 con 0x55c3fe136580 2024-03-21T11:39:11.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.509+0000 7f80d062d740 10 client.20268 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:11.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.509+0000 7f80d062d740 10 client.20268 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:39:11.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.510+0000 7f80d062d740 20 client.20268 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:11.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.510+0000 7f80d062d740 1 --2- 172.21.15.47:0/2159407379 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c3fe1a6ad0 0x55c3fe1a8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.510+0000 7f80d062d740 4 client.20268 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:39:11.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.510+0000 7f80d062d740 1 -- 172.21.15.47:0/2159407379 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c3fe0a0b50 con 0x55c3fe1a6ad0 2024-03-21T11:39:11.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.510+0000 7f80c6a00640 1 --2- 172.21.15.47:0/2159407379 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c3fe1a6ad0 0x55c3fe1a8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80b0c00640 10 client.20268 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:39:11.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80c6a00640 1 --2- 172.21.15.47:0/2159407379 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c3fe1a6ad0 0x55c3fe1a8ef0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e89c306d2a05c98 server_cookie=f269343f682963a5 in_seq=0 out_seq=0 2024-03-21T11:39:11.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80b0c00640 1 -- 172.21.15.47:0/2159407379 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c3fe0a0b50 con 0x55c3fe1a6ad0 2024-03-21T11:39:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80b0c00640 10 client.20268 handle_command_reply: tid=0 2024-03-21T11:39:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80d062d740 1 client.20268 shutdown 2024-03-21T11:39:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80d062d740 1 -- 172.21.15.47:0/2159407379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe136580 msgr2=0x55c3fe144c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80d062d740 1 --2- 172.21.15.47:0/2159407379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe136580 0x55c3fe144c20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f80c0005ee0 tx=0x7f80c0009500 comp rx=0 tx=0).stop 2024-03-21T11:39:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80d062d740 1 -- 172.21.15.47:0/2159407379 shutdown_connections 2024-03-21T11:39:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80d062d740 1 --2- 172.21.15.47:0/2159407379 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c3fe1a6ad0 0x55c3fe1a8ef0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80d062d740 1 --2- 172.21.15.47:0/2159407379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3fe136580 0x55c3fe144c20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80d062d740 1 --2- 172.21.15.47:0/2159407379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3fe135bc0 0x55c3fe143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80d062d740 1 --2- 172.21.15.47:0/2159407379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3fe1349f0 0x55c3fe1436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80d062d740 1 -- 172.21.15.47:0/2159407379 >> 172.21.15.47:0/2159407379 conn(0x55c3fe12fde0 msgr2=0x55c3fe12e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80b0c00640 0 client.20268 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:39:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.511+0000 7f80d062d740 1 -- 172.21.15.47:0/2159407379 shutdown_connections 2024-03-21T11:39:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.512+0000 7f80d062d740 1 -- 172.21.15.47:0/2159407379 wait complete. 2024-03-21T11:39:11.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.512+0000 7f80d062d740 20 client.20268 trim_cache size 0 max 16384 2024-03-21T11:39:11.516 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:39:11.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.512+0000 7f80c7e00640 1 -- 172.21.15.47:0/4138685869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80ac0a2f20 msgr2=0x7f80ac0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.512+0000 7f80c7e00640 1 --2- 172.21.15.47:0/4138685869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80ac0a2f20 0x7f80ac0a53e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f80c0005fd0 tx=0x7f80c0009900 comp rx=0 tx=0).stop 2024-03-21T11:39:11.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.512+0000 7f80c7e00640 1 -- 172.21.15.47:0/4138685869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8104990 msgr2=0x7f80c8199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.512+0000 7f80c7e00640 1 --2- 172.21.15.47:0/4138685869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8104990 0x7f80c8199ff0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f80b800e9f0 tx=0x7f80b800edf0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.512+0000 7f80c7e00640 1 -- 172.21.15.47:0/4138685869 shutdown_connections 2024-03-21T11:39:11.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.512+0000 7f80c7e00640 1 --2- 172.21.15.47:0/4138685869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f80ac0a2f20 0x7f80ac0a53e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.512+0000 7f80c7e00640 1 --2- 172.21.15.47:0/4138685869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c8105350 0x7f80c81a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.512+0000 7f80c7e00640 1 --2- 172.21.15.47:0/4138685869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80c8104990 0x7f80c8199ff0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.512+0000 7f80c7e00640 1 --2- 172.21.15.47:0/4138685869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80c81037a0 0x7f80c8199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.512+0000 7f80c7e00640 1 -- 172.21.15.47:0/4138685869 >> 172.21.15.47:0/4138685869 conn(0x7f80c80fed10 msgr2=0x7f80c8107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.513+0000 7f80c7e00640 1 -- 172.21.15.47:0/4138685869 shutdown_connections 2024-03-21T11:39:11.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.513+0000 7f80c7e00640 1 -- 172.21.15.47:0/4138685869 wait complete. 2024-03-21T11:39:11.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:11 smithi047 ceph-mon[31351]: pgmap v875: 129 pgs: 129 active+clean; 397 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 14 MiB/s wr, 781 op/s 2024-03-21T11:39:11.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:11 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:39:11.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:11 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:39:11.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:11 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:39:11.533 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:39:11.533 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:39:11.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.768+0000 7fccb3e00640 1 -- 172.21.15.47:0/1908475841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb4109460 msgr2=0x7fccb410b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.768+0000 7fccb3e00640 1 --2- 172.21.15.47:0/1908475841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb4109460 0x7fccb410b8f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fccac009ca0 tx=0x7fccac02c100 comp rx=0 tx=0).stop 2024-03-21T11:39:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.768+0000 7fccb3e00640 1 -- 172.21.15.47:0/1908475841 shutdown_connections 2024-03-21T11:39:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.768+0000 7fccb3e00640 1 --2- 172.21.15.47:0/1908475841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb4109460 0x7fccb410b8f0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.768+0000 7fccb3e00640 1 --2- 172.21.15.47:0/1908475841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb4074100 0x7fccb4074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.768+0000 7fccb3e00640 1 --2- 172.21.15.47:0/1908475841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccb40756e0 0x7fccb4073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.768+0000 7fccb3e00640 1 -- 172.21.15.47:0/1908475841 >> 172.21.15.47:0/1908475841 conn(0x7fccb40fcb10 msgr2=0x7fccb40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.768+0000 7fccb3e00640 1 -- 172.21.15.47:0/1908475841 shutdown_connections 2024-03-21T11:39:11.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.768+0000 7fccb3e00640 1 -- 172.21.15.47:0/1908475841 wait complete. 2024-03-21T11:39:11.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.770+0000 7fccb3e00640 1 Processor -- start 2024-03-21T11:39:11.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.770+0000 7fccb3e00640 1 -- start start 2024-03-21T11:39:11.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.770+0000 7fccb3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb4074100 0x7fccb4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.770+0000 7fccb3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb40756e0 0x7fccb4199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.770+0000 7fccb3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccb4109460 0x7fccb41a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.770+0000 7fccb3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb410f460 con 0x7fccb40756e0 2024-03-21T11:39:11.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.770+0000 7fccb3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb410f2e0 con 0x7fccb4074100 2024-03-21T11:39:11.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.770+0000 7fccb3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb410f5e0 con 0x7fccb4109460 2024-03-21T11:39:11.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.771+0000 7fccb3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccb4109460 0x7fccb41a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.771+0000 7fccb2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb40756e0 0x7fccb4199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.771+0000 7fccb2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb4074100 0x7fccb4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.771+0000 7fccb3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccb4109460 0x7fccb41a1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37454/0 (socket says 172.21.15.47:37454) 2024-03-21T11:39:11.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.771+0000 7fccb3400640 1 -- 172.21.15.47:0/1016674676 learned_addr learned my addr 172.21.15.47:0/1016674676 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.771+0000 7fccb3400640 1 -- 172.21.15.47:0/1016674676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb4074100 msgr2=0x7fccb4199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.771+0000 7fccb3400640 1 --2- 172.21.15.47:0/1016674676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb4074100 0x7fccb4199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.771+0000 7fccb3400640 1 -- 172.21.15.47:0/1016674676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb40756e0 msgr2=0x7fccb4199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.771+0000 7fccb3400640 1 --2- 172.21.15.47:0/1016674676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb40756e0 0x7fccb4199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.771+0000 7fccb3400640 1 -- 172.21.15.47:0/1016674676 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb41a16d0 con 0x7fccb4109460 2024-03-21T11:39:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.771+0000 7fccb2000640 1 --2- 172.21.15.47:0/1016674676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb40756e0 0x7fccb4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.772+0000 7fccb3400640 1 --2- 172.21.15.47:0/1016674676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccb4109460 0x7fccb41a1060 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fccac02c640 tx=0x7fccac035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.772+0000 7fccb8a00640 1 -- 172.21.15.47:0/1016674676 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac039040 con 0x7fccb4109460 2024-03-21T11:39:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.772+0000 7fccb3e00640 1 -- 172.21.15.47:0/1016674676 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb41a1960 con 0x7fccb4109460 2024-03-21T11:39:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.772+0000 7fccb8a00640 1 -- 172.21.15.47:0/1016674676 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccac044070 con 0x7fccb4109460 2024-03-21T11:39:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.772+0000 7fccb8a00640 1 -- 172.21.15.47:0/1016674676 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac004350 con 0x7fccb4109460 2024-03-21T11:39:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.772+0000 7fccb3e00640 1 -- 172.21.15.47:0/1016674676 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb41a1f20 con 0x7fccb4109460 2024-03-21T11:39:11.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.775+0000 7fccb8a00640 1 -- 172.21.15.47:0/1016674676 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fccac0044f0 con 0x7fccb4109460 2024-03-21T11:39:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.776+0000 7fccbca6d740 1 Processor -- start 2024-03-21T11:39:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.776+0000 7fccbca6d740 1 -- start start 2024-03-21T11:39:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.777+0000 7fccbca6d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe8200 0x5583b9fe8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.777+0000 7fccbca6d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe93d0 0x5583b9fe9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.777+0000 7fccbca6d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe9d90 0x5583b9ff0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.777+0000 7fccbca6d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583b9f442c0 con 0x5583b9fe9d90 2024-03-21T11:39:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.777+0000 7fccbca6d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583b9eecc90 con 0x5583b9fe8200 2024-03-21T11:39:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.777+0000 7fccbca6d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583b9f40360 con 0x5583b9fe93d0 2024-03-21T11:39:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.777+0000 7fccb2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe93d0 0x5583b9fe9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.777+0000 7fccb2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe93d0 0x5583b9fe9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37458/0 (socket says 172.21.15.47:37458) 2024-03-21T11:39:11.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.777+0000 7fccb2a00640 1 -- 172.21.15.47:0/2873045537 learned_addr learned my addr 172.21.15.47:0/2873045537 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.777+0000 7fccb2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe8200 0x5583b9fe8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.777+0000 7fccb8a00640 1 --2- 172.21.15.47:0/1016674676 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc980a2f20 0x7fcc980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.778+0000 7fccb2000640 1 --2- 172.21.15.47:0/2873045537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe9d90 0x5583b9ff0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.778+0000 7fccb8a00640 1 -- 172.21.15.47:0/1016674676 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fccac0f1e00 con 0x7fccb4109460 2024-03-21T11:39:11.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.778+0000 7fccb3400640 1 --2- 172.21.15.47:0/1016674676 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc980a2f20 0x7fcc980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.778+0000 7fccb2a00640 1 -- 172.21.15.47:0/2873045537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe8200 msgr2=0x5583b9fe8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.778+0000 7fccb2a00640 1 --2- 172.21.15.47:0/2873045537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe8200 0x5583b9fe8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.778+0000 7fccb2a00640 1 -- 172.21.15.47:0/2873045537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe9d90 msgr2=0x5583b9ff0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.778+0000 7fccb3400640 1 --2- 172.21.15.47:0/1016674676 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc980a2f20 0x7fcc980a53e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fccac0be0d0 tx=0x7fccac02c100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.779+0000 7fccb2a00640 1 --2- 172.21.15.47:0/2873045537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe9d90 0x5583b9ff0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.779+0000 7fccb2a00640 1 -- 172.21.15.47:0/2873045537 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583b9f11080 con 0x5583b9fe93d0 2024-03-21T11:39:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.779+0000 7fccb2000640 1 --2- 172.21.15.47:0/2873045537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe9d90 0x5583b9ff0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.779+0000 7fccb2a00640 1 --2- 172.21.15.47:0/2873045537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe93d0 0x5583b9fe9850 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009a70 tx=0x7fcca4009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2415a59481fd82bd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.779+0000 7fcc9fe00640 1 -- 172.21.15.47:0/2873045537 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4017070 con 0x5583b9fe93d0 2024-03-21T11:39:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.779+0000 7fcc9fe00640 1 -- 172.21.15.47:0/2873045537 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcca4013470 con 0x5583b9fe93d0 2024-03-21T11:39:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.779+0000 7fcc9fe00640 1 -- 172.21.15.47:0/2873045537 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4012610 con 0x5583b9fe93d0 2024-03-21T11:39:11.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.780+0000 7fccbca6d740 1 -- 172.21.15.47:0/2873045537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe93d0 msgr2=0x5583b9fe9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.780+0000 7fccbca6d740 1 --2- 172.21.15.47:0/2873045537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe93d0 0x5583b9fe9850 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009a70 tx=0x7fcca4009f30 comp rx=0 tx=0).stop 2024-03-21T11:39:11.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.781+0000 7fccbca6d740 1 -- 172.21.15.47:0/2873045537 shutdown_connections 2024-03-21T11:39:11.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.781+0000 7fccbca6d740 1 --2- 172.21.15.47:0/2873045537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe9d90 0x5583b9ff0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.781+0000 7fccbca6d740 1 --2- 172.21.15.47:0/2873045537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe93d0 0x5583b9fe9850 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.781+0000 7fccbca6d740 1 --2- 172.21.15.47:0/2873045537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe8200 0x5583b9fe8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.781+0000 7fccbca6d740 1 -- 172.21.15.47:0/2873045537 >> 172.21.15.47:0/2873045537 conn(0x5583b9fdade0 msgr2=0x5583b9fdb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.781+0000 7fccbca6d740 1 -- 172.21.15.47:0/2873045537 shutdown_connections 2024-03-21T11:39:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.781+0000 7fccbca6d740 1 -- 172.21.15.47:0/2873045537 wait complete. 2024-03-21T11:39:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.781+0000 7fccbca6d740 1 -- start start 2024-03-21T11:39:11.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.782+0000 7fccbca6d740 1 Processor -- start 2024-03-21T11:39:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.782+0000 7fccbca6d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe8200 0x5583b9ff2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.782+0000 7fccbca6d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe93d0 0x5583b9ff29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.782+0000 7fccbca6d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe9d90 0x5583b9ff99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.782+0000 7fccbca6d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583b9ff0da0 con 0x5583b9fe93d0 2024-03-21T11:39:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.782+0000 7fccbca6d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583b9f000d0 con 0x5583b9fe9d90 2024-03-21T11:39:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.782+0000 7fccbca6d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583b9ff0f20 con 0x5583b9fe8200 2024-03-21T11:39:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.782+0000 7fccbca6d740 10 client.0 _ll_register_callbacks cb 0x5583b9f04e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.782+0000 7fccb2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe8200 0x5583b9ff2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.782+0000 7fccb2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe93d0 0x5583b9ff29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.782+0000 7fccb2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe93d0 0x5583b9ff29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52448/0 (socket says 172.21.15.47:52448) 2024-03-21T11:39:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.782+0000 7fccb2a00640 1 -- 172.21.15.47:0/3529316294 learned_addr learned my addr 172.21.15.47:0/3529316294 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.782+0000 7fccb2000640 1 --2- 172.21.15.47:0/3529316294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe8200 0x5583b9ff2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37470/0 (socket says 172.21.15.47:37470) 2024-03-21T11:39:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.782+0000 7fccb2000640 1 --2- 172.21.15.47:0/3529316294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe9d90 0x5583b9ff99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.783+0000 7fccb2a00640 1 -- 172.21.15.47:0/3529316294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe8200 msgr2=0x5583b9ff2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.783+0000 7fccb2a00640 1 --2- 172.21.15.47:0/3529316294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe8200 0x5583b9ff2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.783+0000 7fccb2a00640 1 -- 172.21.15.47:0/3529316294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe9d90 msgr2=0x5583b9ff99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.783+0000 7fccb2a00640 1 --2- 172.21.15.47:0/3529316294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe9d90 0x5583b9ff99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.783+0000 7fccb2a00640 1 -- 172.21.15.47:0/3529316294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583b9c583f0 con 0x5583b9fe93d0 2024-03-21T11:39:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.783+0000 7fccb2000640 1 --2- 172.21.15.47:0/3529316294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe8200 0x5583b9ff2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.783+0000 7fccb2a00640 1 --2- 172.21.15.47:0/3529316294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe93d0 0x5583b9ff29b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcca4013850 tx=0x7fcca4013880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235256b4d82c1534 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.783+0000 7fcc9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:39:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.783+0000 7fcc9cc00640 1 -- 172.21.15.47:0/3529316294 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4017050 con 0x5583b9fe93d0 2024-03-21T11:39:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.784+0000 7fcc9cc00640 1 -- 172.21.15.47:0/3529316294 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcca401c450 con 0x5583b9fe93d0 2024-03-21T11:39:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.784+0000 7fccbca6d740 1 -- 172.21.15.47:0/3529316294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcca4013470 con 0x5583b9fe93d0 2024-03-21T11:39:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.784+0000 7fcc9cc00640 1 -- 172.21.15.47:0/3529316294 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcca402f050 con 0x5583b9fe93d0 2024-03-21T11:39:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.784+0000 7fccbca6d740 10 client.20280 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.784+0000 7fccbca6d740 1 -- 172.21.15.47:0/3529316294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5583b9d227d0 con 0x5583b9fe93d0 2024-03-21T11:39:11.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.784+0000 7fcc9cc00640 1 -- 172.21.15.47:0/3529316294 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcca4013470 con 0x5583b9fe93d0 2024-03-21T11:39:11.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.785+0000 7fcc9cc00640 1 -- 172.21.15.47:0/3529316294 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fcca40266e0 con 0x5583b9fe93d0 2024-03-21T11:39:11.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.785+0000 7fccbca6d740 10 client.20280 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:11.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.785+0000 7fccbca6d740 10 client.20280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:11.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.785+0000 7fccbca6d740 20 client.20280 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:11.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.785+0000 7fccbca6d740 1 --2- 172.21.15.47:0/3529316294 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583b9ffdc00 0x5583b9fffff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.785+0000 7fccbca6d740 4 client.20280 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:11.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.785+0000 7fccbca6d740 1 -- 172.21.15.47:0/3529316294 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583b9f51800 con 0x5583b9ffdc00 2024-03-21T11:39:11.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.785+0000 7fccb2000640 1 --2- 172.21.15.47:0/3529316294 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583b9ffdc00 0x5583b9fffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.786+0000 7fccb2000640 1 --2- 172.21.15.47:0/3529316294 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583b9ffdc00 0x5583b9fffff0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=815c8a8e3fa90748 server_cookie=2c6437f09d98edda in_seq=0 out_seq=0 2024-03-21T11:39:11.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.786+0000 7fcc9cc00640 10 client.20280 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:11.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.791+0000 7fcc9cc00640 1 -- 172.21.15.47:0/3529316294 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5583b9f51800 con 0x5583b9ffdc00 2024-03-21T11:39:11.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.791+0000 7fcc9cc00640 10 client.20280 handle_command_reply: tid=0 2024-03-21T11:39:11.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.791+0000 7fccbca6d740 1 client.20280 shutdown 2024-03-21T11:39:11.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.791+0000 7fccbca6d740 1 -- 172.21.15.47:0/3529316294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe93d0 msgr2=0x5583b9ff29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.791+0000 7fccbca6d740 1 --2- 172.21.15.47:0/3529316294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe93d0 0x5583b9ff29b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcca4013850 tx=0x7fcca4013880 comp rx=0 tx=0).stop 2024-03-21T11:39:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.791+0000 7fccbca6d740 1 -- 172.21.15.47:0/3529316294 shutdown_connections 2024-03-21T11:39:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.791+0000 7fccbca6d740 1 --2- 172.21.15.47:0/3529316294 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583b9ffdc00 0x5583b9fffff0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.791+0000 7fccbca6d740 1 --2- 172.21.15.47:0/3529316294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe9d90 0x5583b9ff99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.791+0000 7fccbca6d740 1 --2- 172.21.15.47:0/3529316294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe93d0 0x5583b9ff29b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.791+0000 7fccbca6d740 1 --2- 172.21.15.47:0/3529316294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe8200 0x5583b9ff2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.791+0000 7fcc9cc00640 0 client.20280 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.791+0000 7fccbca6d740 1 -- 172.21.15.47:0/3529316294 >> 172.21.15.47:0/3529316294 conn(0x5583b9fdade0 msgr2=0x5583b9fda160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.791+0000 7fccbca6d740 1 -- 172.21.15.47:0/3529316294 shutdown_connections 2024-03-21T11:39:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.791+0000 7fccbca6d740 1 -- 172.21.15.47:0/3529316294 wait complete. 2024-03-21T11:39:11.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.792+0000 7fccbca6d740 20 client.20280 trim_cache size 0 max 16384 2024-03-21T11:39:11.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.807+0000 7fccbca6d740 1 Processor -- start 2024-03-21T11:39:11.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.807+0000 7fccbca6d740 1 -- start start 2024-03-21T11:39:11.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.807+0000 7fccbca6d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe8200 0x5583b9fd98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.807+0000 7fccbca6d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe93d0 0x5583b9ff99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.807+0000 7fccbca6d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe9d90 0x5583b9fd9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.807+0000 7fccbca6d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb41a7da0 con 0x5583b9fe8200 2024-03-21T11:39:11.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.807+0000 7fccbca6d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb41a98a0 con 0x5583b9fe93d0 2024-03-21T11:39:11.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.807+0000 7fccbca6d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb41a8400 con 0x5583b9fe9d90 2024-03-21T11:39:11.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.808+0000 7fccb2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe8200 0x5583b9fd98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.808+0000 7fccb2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe93d0 0x5583b9ff99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.808+0000 7fccb2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe8200 0x5583b9fd98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52464/0 (socket says 172.21.15.47:52464) 2024-03-21T11:39:11.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.808+0000 7fccb2000640 1 -- 172.21.15.47:0/2553861975 learned_addr learned my addr 172.21.15.47:0/2553861975 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.808+0000 7fccb2000640 1 --2- 172.21.15.47:0/2553861975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe9d90 0x5583b9fd9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.808+0000 7fccb2a00640 1 -- 172.21.15.47:0/2553861975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe9d90 msgr2=0x5583b9fd9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.808+0000 7fccb2a00640 1 --2- 172.21.15.47:0/2553861975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe9d90 0x5583b9fd9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.808+0000 7fccb2a00640 1 -- 172.21.15.47:0/2553861975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe8200 msgr2=0x5583b9fd98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.808+0000 7fccb2a00640 1 --2- 172.21.15.47:0/2553861975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe8200 0x5583b9fd98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.808+0000 7fccb2a00640 1 -- 172.21.15.47:0/2553861975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583ba0005b0 con 0x5583b9fe93d0 2024-03-21T11:39:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.808+0000 7fccb2a00640 1 --2- 172.21.15.47:0/2553861975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe93d0 0x5583b9ff99b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcca4004700 tx=0x7fcca4009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e02733f85af0a5bf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.809+0000 7fcc9f400640 1 -- 172.21.15.47:0/2553861975 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4017030 con 0x5583b9fe93d0 2024-03-21T11:39:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.809+0000 7fcc9f400640 1 -- 172.21.15.47:0/2553861975 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcca400ed50 con 0x5583b9fe93d0 2024-03-21T11:39:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.809+0000 7fcc9f400640 1 -- 172.21.15.47:0/2553861975 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca40268c0 con 0x5583b9fe93d0 2024-03-21T11:39:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.810+0000 7fccbca6d740 1 -- 172.21.15.47:0/2553861975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe93d0 msgr2=0x5583b9ff99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.810+0000 7fccbca6d740 1 --2- 172.21.15.47:0/2553861975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe93d0 0x5583b9ff99b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcca4004700 tx=0x7fcca4009f30 comp rx=0 tx=0).stop 2024-03-21T11:39:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.810+0000 7fccbca6d740 1 -- 172.21.15.47:0/2553861975 shutdown_connections 2024-03-21T11:39:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.810+0000 7fccbca6d740 1 --2- 172.21.15.47:0/2553861975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe9d90 0x5583b9fd9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.810+0000 7fccbca6d740 1 --2- 172.21.15.47:0/2553861975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe93d0 0x5583b9ff99b0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.810+0000 7fccbca6d740 1 --2- 172.21.15.47:0/2553861975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe8200 0x5583b9fd98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.810+0000 7fccbca6d740 1 -- 172.21.15.47:0/2553861975 >> 172.21.15.47:0/2553861975 conn(0x5583b9fdade0 msgr2=0x5583b9ff2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.810+0000 7fccbca6d740 1 -- 172.21.15.47:0/2553861975 shutdown_connections 2024-03-21T11:39:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.810+0000 7fccbca6d740 1 -- 172.21.15.47:0/2553861975 wait complete. 2024-03-21T11:39:11.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.810+0000 7fccbca6d740 1 -- start start 2024-03-21T11:39:11.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.811+0000 7fccbca6d740 1 Processor -- start 2024-03-21T11:39:11.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.811+0000 7fccbca6d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe8200 0x5583b9ff4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.811+0000 7fccbca6d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe93d0 0x5583b9ff5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.811+0000 7fccbca6d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe9d90 0x5583b9ff8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.811+0000 7fccbca6d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb41a9180 con 0x5583b9fe9d90 2024-03-21T11:39:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.811+0000 7fccbca6d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb41a8b20 con 0x5583b9fe8200 2024-03-21T11:39:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.811+0000 7fccbca6d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb41a5f80 con 0x5583b9fe93d0 2024-03-21T11:39:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.811+0000 7fccbca6d740 10 client.0 _ll_register_callbacks cb 0x7fcca4013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.811+0000 7fccb2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe93d0 0x5583b9ff5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.812+0000 7fccb2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe93d0 0x5583b9ff5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37486/0 (socket says 172.21.15.47:37486) 2024-03-21T11:39:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.812+0000 7fccb2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe8200 0x5583b9ff4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.812+0000 7fccb2a00640 1 -- 172.21.15.47:0/2945990529 learned_addr learned my addr 172.21.15.47:0/2945990529 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.812+0000 7fccb2000640 1 --2- 172.21.15.47:0/2945990529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe9d90 0x5583b9ff8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.812+0000 7fccb2a00640 1 -- 172.21.15.47:0/2945990529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe8200 msgr2=0x5583b9ff4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.812+0000 7fccb2a00640 1 --2- 172.21.15.47:0/2945990529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe8200 0x5583b9ff4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.812+0000 7fccb2a00640 1 -- 172.21.15.47:0/2945990529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe9d90 msgr2=0x5583b9ff8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:11.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.812+0000 7fccb2a00640 1 --2- 172.21.15.47:0/2945990529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe9d90 0x5583b9ff8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.812+0000 7fccb2a00640 1 -- 172.21.15.47:0/2945990529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583b9ff89f0 con 0x5583b9fe93d0 2024-03-21T11:39:11.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.812+0000 7fccb2a00640 1 --2- 172.21.15.47:0/2945990529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe93d0 0x5583b9ff5430 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fcca40138c0 tx=0x7fcca400ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3ec115e2f22d481 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:11.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.812+0000 7fcc9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:11.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.813+0000 7fcc9cc00640 1 -- 172.21.15.47:0/2945990529 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4017040 con 0x5583b9fe93d0 2024-03-21T11:39:11.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.813+0000 7fccbca6d740 1 -- 172.21.15.47:0/2945990529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcca400ed50 con 0x5583b9fe93d0 2024-03-21T11:39:11.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.813+0000 7fcc9cc00640 1 -- 172.21.15.47:0/2945990529 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcca40267f0 con 0x5583b9fe93d0 2024-03-21T11:39:11.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.814+0000 7fcc9cc00640 1 -- 172.21.15.47:0/2945990529 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcca4026c50 con 0x5583b9fe93d0 2024-03-21T11:39:11.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.814+0000 7fccbca6d740 10 client.30778 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:11.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.814+0000 7fccbca6d740 1 -- 172.21.15.47:0/2945990529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5583b9ff9cf0 con 0x5583b9fe93d0 2024-03-21T11:39:11.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.815+0000 7fcc9cc00640 1 -- 172.21.15.47:0/2945990529 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcca400ed50 con 0x5583b9fe93d0 2024-03-21T11:39:11.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.815+0000 7fcc9cc00640 1 -- 172.21.15.47:0/2945990529 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fcca4033430 con 0x5583b9fe93d0 2024-03-21T11:39:11.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.815+0000 7fccbca6d740 10 client.30778 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:11.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.815+0000 7fccbca6d740 10 client.30778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:11.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.815+0000 7fccbca6d740 20 client.30778 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:11.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.815+0000 7fccbca6d740 1 --2- 172.21.15.47:0/2945990529 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583ba059b00 0x5583ba05bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:11.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.815+0000 7fccbca6d740 4 client.30778 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:39:11.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.815+0000 7fccbca6d740 1 -- 172.21.15.47:0/2945990529 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5583b9f4bb50 con 0x5583ba059b00 2024-03-21T11:39:11.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.815+0000 7fccb2000640 1 --2- 172.21.15.47:0/2945990529 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583ba059b00 0x5583ba05bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:11.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.816+0000 7fccb2000640 1 --2- 172.21.15.47:0/2945990529 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583ba059b00 0x5583ba05bf20 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=160d46bc2b094043 server_cookie=a5b70adf917c3a47 in_seq=0 out_seq=0 2024-03-21T11:39:11.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.816+0000 7fcc9cc00640 10 client.30778 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.819+0000 7fcc9cc00640 1 -- 172.21.15.47:0/2945990529 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5583b9f4bb50 con 0x5583ba059b00 2024-03-21T11:39:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.819+0000 7fcc9cc00640 10 client.30778 handle_command_reply: tid=0 2024-03-21T11:39:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.819+0000 7fccbca6d740 1 client.30778 shutdown 2024-03-21T11:39:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.820+0000 7fccbca6d740 1 -- 172.21.15.47:0/2945990529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe93d0 msgr2=0x5583b9ff5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.820+0000 7fccbca6d740 1 --2- 172.21.15.47:0/2945990529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe93d0 0x5583b9ff5430 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fcca40138c0 tx=0x7fcca400ef00 comp rx=0 tx=0).stop 2024-03-21T11:39:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.820+0000 7fccbca6d740 1 -- 172.21.15.47:0/2945990529 shutdown_connections 2024-03-21T11:39:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.820+0000 7fccbca6d740 1 --2- 172.21.15.47:0/2945990529 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5583ba059b00 0x5583ba05bf20 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.820+0000 7fccbca6d740 1 --2- 172.21.15.47:0/2945990529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583b9fe9d90 0x5583b9ff8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.820+0000 7fccbca6d740 1 --2- 172.21.15.47:0/2945990529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5583b9fe93d0 0x5583b9ff5430 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.820+0000 7fcc9cc00640 0 client.30778 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:11.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.820+0000 7fccbca6d740 1 --2- 172.21.15.47:0/2945990529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5583b9fe8200 0x5583b9ff4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.820+0000 7fccbca6d740 1 -- 172.21.15.47:0/2945990529 >> 172.21.15.47:0/2945990529 conn(0x5583b9fdade0 msgr2=0x5583b9fde040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.820+0000 7fccbca6d740 1 -- 172.21.15.47:0/2945990529 shutdown_connections 2024-03-21T11:39:11.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.820+0000 7fccbca6d740 1 -- 172.21.15.47:0/2945990529 wait complete. 2024-03-21T11:39:11.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.820+0000 7fccbca6d740 20 client.30778 trim_cache size 0 max 16384 2024-03-21T11:39:11.822 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:39:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.821+0000 7fccb3e00640 1 -- 172.21.15.47:0/1016674676 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc980a2f20 msgr2=0x7fcc980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.821+0000 7fccb3e00640 1 --2- 172.21.15.47:0/1016674676 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc980a2f20 0x7fcc980a53e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fccac0be0d0 tx=0x7fccac02c100 comp rx=0 tx=0).stop 2024-03-21T11:39:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.821+0000 7fccb3e00640 1 -- 172.21.15.47:0/1016674676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccb4109460 msgr2=0x7fccb41a1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.821+0000 7fccb3e00640 1 --2- 172.21.15.47:0/1016674676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccb4109460 0x7fccb41a1060 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fccac02c640 tx=0x7fccac035d10 comp rx=0 tx=0).stop 2024-03-21T11:39:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.821+0000 7fccb3e00640 1 -- 172.21.15.47:0/1016674676 shutdown_connections 2024-03-21T11:39:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.821+0000 7fccb3e00640 1 --2- 172.21.15.47:0/1016674676 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc980a2f20 0x7fcc980a53e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.821+0000 7fccb3e00640 1 --2- 172.21.15.47:0/1016674676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccb4109460 0x7fccb41a1060 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.821+0000 7fccb3e00640 1 --2- 172.21.15.47:0/1016674676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fccb40756e0 0x7fccb4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.821+0000 7fccb3e00640 1 --2- 172.21.15.47:0/1016674676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb4074100 0x7fccb4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.821+0000 7fccb3e00640 1 -- 172.21.15.47:0/1016674676 >> 172.21.15.47:0/1016674676 conn(0x7fccb40fcb10 msgr2=0x7fccb4105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.821+0000 7fccb3e00640 1 -- 172.21.15.47:0/1016674676 shutdown_connections 2024-03-21T11:39:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:11.821+0000 7fccb3e00640 1 -- 172.21.15.47:0/1016674676 wait complete. 2024-03-21T11:39:11.929 INFO:tasks.workunit.client.0.smithi047.stdout: 10 55811 27.59 MB/sec warmup 115 sec latency 103.525 ms 2024-03-21T11:39:12.841 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:39:12.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:39:12.841 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:39:12.929 INFO:tasks.workunit.client.0.smithi047.stdout: 10 56443 27.73 MB/sec warmup 116 sec latency 80.796 ms 2024-03-21T11:39:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.085+0000 7f8d3fe00640 1 -- 172.21.15.47:0/3733703803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d4010db70 msgr2=0x7f8d4010ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.085+0000 7f8d3fe00640 1 --2- 172.21.15.47:0/3733703803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d4010db70 0x7f8d4010ff60 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009ca0 tx=0x7f8d3802c100 comp rx=0 tx=0).stop 2024-03-21T11:39:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.085+0000 7f8d3fe00640 1 -- 172.21.15.47:0/3733703803 shutdown_connections 2024-03-21T11:39:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.085+0000 7f8d3fe00640 1 --2- 172.21.15.47:0/3733703803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d4010db70 0x7f8d4010ff60 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.085+0000 7f8d3fe00640 1 --2- 172.21.15.47:0/3733703803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d40074100 0x7f8d40074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.085+0000 7f8d3fe00640 1 --2- 172.21.15.47:0/3733703803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d400756e0 0x7f8d40073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.085+0000 7f8d3fe00640 1 -- 172.21.15.47:0/3733703803 >> 172.21.15.47:0/3733703803 conn(0x7f8d400fcb10 msgr2=0x7f8d400fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.085+0000 7f8d3fe00640 1 -- 172.21.15.47:0/3733703803 shutdown_connections 2024-03-21T11:39:13.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.085+0000 7f8d3fe00640 1 -- 172.21.15.47:0/3733703803 wait complete. 2024-03-21T11:39:13.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.086+0000 7f8d3fe00640 1 Processor -- start 2024-03-21T11:39:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.086+0000 7f8d3fe00640 1 -- start start 2024-03-21T11:39:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.087+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d40074100 0x7f8d4019deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.087+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d400756e0 0x7f8d4019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.087+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d4010db70 0x7f8d401a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.087+0000 7f8d3fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40113810 con 0x7f8d40074100 2024-03-21T11:39:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.087+0000 7f8d3fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40113690 con 0x7f8d400756e0 2024-03-21T11:39:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.087+0000 7f8d3fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40113990 con 0x7f8d4010db70 2024-03-21T11:39:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.087+0000 7f8d3ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d40074100 0x7f8d4019deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.087+0000 7f8d3e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d400756e0 0x7f8d4019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.087+0000 7f8d3ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d40074100 0x7f8d4019deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52494/0 (socket says 172.21.15.47:52494) 2024-03-21T11:39:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.087+0000 7f8d3ea00640 1 -- 172.21.15.47:0/1053518213 learned_addr learned my addr 172.21.15.47:0/1053518213 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.087+0000 7f8d3f400640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d4010db70 0x7f8d401a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.088+0000 7f8d3f400640 1 -- 172.21.15.47:0/1053518213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d400756e0 msgr2=0x7f8d4019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.088+0000 7f8d3f400640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d400756e0 0x7f8d4019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.088+0000 7f8d3f400640 1 -- 172.21.15.47:0/1053518213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d40074100 msgr2=0x7f8d4019deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.088+0000 7f8d3f400640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d40074100 0x7f8d4019deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.088+0000 7f8d3f400640 1 -- 172.21.15.47:0/1053518213 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d401a5a80 con 0x7f8d4010db70 2024-03-21T11:39:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.088+0000 7f8d3e000640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d400756e0 0x7f8d4019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.088+0000 7f8d3ea00640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d40074100 0x7f8d4019deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.088+0000 7f8d3f400640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d4010db70 0x7f8d401a5410 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8d3802c640 tx=0x7f8d380346f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:13.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.088+0000 7f8d37400640 1 -- 172.21.15.47:0/1053518213 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38039050 con 0x7f8d4010db70 2024-03-21T11:39:13.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.088+0000 7f8d3fe00640 1 -- 172.21.15.47:0/1053518213 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d401a5d10 con 0x7f8d4010db70 2024-03-21T11:39:13.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.088+0000 7f8d37400640 1 -- 172.21.15.47:0/1053518213 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d38045070 con 0x7f8d4010db70 2024-03-21T11:39:13.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.088+0000 7f8d37400640 1 -- 172.21.15.47:0/1053518213 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d380046a0 con 0x7f8d4010db70 2024-03-21T11:39:13.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.088+0000 7f8d3fe00640 1 -- 172.21.15.47:0/1053518213 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d401a61d0 con 0x7f8d4010db70 2024-03-21T11:39:13.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.092+0000 7f8d37400640 1 -- 172.21.15.47:0/1053518213 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8d3804a460 con 0x7f8d4010db70 2024-03-21T11:39:13.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.092+0000 7f8d48394740 1 Processor -- start 2024-03-21T11:39:13.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.092+0000 7f8d48394740 1 -- start start 2024-03-21T11:39:13.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.092+0000 7f8d48394740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d692b0 0x55b415d696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.092+0000 7f8d48394740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d69ca0 0x55b415d6e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d48394740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d6ee10 0x55b415d712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d48394740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b415cceeb0 con 0x55b415d692b0 2024-03-21T11:39:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d48394740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b415c77a60 con 0x55b415d6ee10 2024-03-21T11:39:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d48394740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b415ccaf50 con 0x55b415d69ca0 2024-03-21T11:39:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d3ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d69ca0 0x55b415d6e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d3ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d69ca0 0x55b415d6e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37520/0 (socket says 172.21.15.47:37520) 2024-03-21T11:39:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d3ea00640 1 -- 172.21.15.47:0/3942049419 learned_addr learned my addr 172.21.15.47:0/3942049419 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d3e000640 1 --2- 172.21.15.47:0/3942049419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d692b0 0x55b415d696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d3e000640 1 --2- 172.21.15.47:0/3942049419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d6ee10 0x55b415d712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d3ea00640 1 -- 172.21.15.47:0/3942049419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d6ee10 msgr2=0x55b415d712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d3ea00640 1 --2- 172.21.15.47:0/3942049419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d6ee10 0x55b415d712a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d3ea00640 1 -- 172.21.15.47:0/3942049419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d692b0 msgr2=0x55b415d696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d3ea00640 1 --2- 172.21.15.47:0/3942049419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d692b0 0x55b415d696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d3ea00640 1 -- 172.21.15.47:0/3942049419 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b415aab7e0 con 0x55b415d69ca0 2024-03-21T11:39:13.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.093+0000 7f8d3e000640 1 --2- 172.21.15.47:0/3942049419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d6ee10 0x55b415d712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.094+0000 7f8d37400640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d240a2fa0 0x7f8d240a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.094+0000 7f8d37400640 1 -- 172.21.15.47:0/1053518213 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8d380f1f00 con 0x7f8d4010db70 2024-03-21T11:39:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.094+0000 7f8d3e000640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d240a2fa0 0x7f8d240a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.094+0000 7f8d3ea00640 1 --2- 172.21.15.47:0/3942049419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d69ca0 0x55b415d6e7a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8d30011650 tx=0x7f8d30009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deb8915253baae4b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.094+0000 7f8d2b400640 1 -- 172.21.15.47:0/3942049419 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3001e070 con 0x55b415d69ca0 2024-03-21T11:39:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.094+0000 7f8d2b400640 1 -- 172.21.15.47:0/3942049419 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d3000fba0 con 0x55b415d69ca0 2024-03-21T11:39:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.094+0000 7f8d2b400640 1 -- 172.21.15.47:0/3942049419 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30019d40 con 0x55b415d69ca0 2024-03-21T11:39:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.095+0000 7f8d48394740 1 -- 172.21.15.47:0/3942049419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d69ca0 msgr2=0x55b415d6e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.095+0000 7f8d48394740 1 --2- 172.21.15.47:0/3942049419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d69ca0 0x55b415d6e7a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8d30011650 tx=0x7f8d30009f90 comp rx=0 tx=0).stop 2024-03-21T11:39:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.095+0000 7f8d48394740 1 -- 172.21.15.47:0/3942049419 shutdown_connections 2024-03-21T11:39:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.095+0000 7f8d48394740 1 --2- 172.21.15.47:0/3942049419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d6ee10 0x55b415d712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.095+0000 7f8d48394740 1 --2- 172.21.15.47:0/3942049419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d69ca0 0x55b415d6e7a0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.096+0000 7f8d48394740 1 --2- 172.21.15.47:0/3942049419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d692b0 0x55b415d696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.096+0000 7f8d48394740 1 -- 172.21.15.47:0/3942049419 >> 172.21.15.47:0/3942049419 conn(0x55b415d64820 msgr2=0x55b415d64c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.096+0000 7f8d3e000640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d240a2fa0 0x7f8d240a5460 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009710 tx=0x7f8d2c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.096+0000 7f8d48394740 1 -- 172.21.15.47:0/3942049419 shutdown_connections 2024-03-21T11:39:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.096+0000 7f8d48394740 1 -- 172.21.15.47:0/3942049419 wait complete. 2024-03-21T11:39:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.096+0000 7f8d48394740 1 -- start start 2024-03-21T11:39:13.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.096+0000 7f8d48394740 1 Processor -- start 2024-03-21T11:39:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.097+0000 7f8d48394740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d692b0 0x55b415d734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.097+0000 7f8d48394740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d69ca0 0x55b415d73a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.097+0000 7f8d48394740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d6ee10 0x55b415d7aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.097+0000 7f8d48394740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b415d71dc0 con 0x55b415d692b0 2024-03-21T11:39:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.097+0000 7f8d3ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d692b0 0x55b415d734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.097+0000 7f8d3e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d69ca0 0x55b415d73a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.097+0000 7f8d3ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d692b0 0x55b415d734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52502/0 (socket says 172.21.15.47:52502) 2024-03-21T11:39:13.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.097+0000 7f8d3e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d69ca0 0x55b415d73a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36050/0 (socket says 172.21.15.47:36050) 2024-03-21T11:39:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.097+0000 7f8d3ea00640 1 -- 172.21.15.47:0/2186479176 learned_addr learned my addr 172.21.15.47:0/2186479176 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.097+0000 7f8d48394740 1 -- 172.21.15.47:0/2186479176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b415c8aeb0 con 0x55b415d69ca0 2024-03-21T11:39:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.097+0000 7f8d48394740 1 -- 172.21.15.47:0/2186479176 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b415d71f40 con 0x55b415d6ee10 2024-03-21T11:39:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.097+0000 7f8d48394740 10 client.0 _ll_register_callbacks cb 0x55b415c7f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.097+0000 7f8d3f400640 1 --2- 172.21.15.47:0/2186479176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d6ee10 0x55b415d7aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.098+0000 7f8d3ea00640 1 -- 172.21.15.47:0/2186479176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d6ee10 msgr2=0x55b415d7aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.098+0000 7f8d3ea00640 1 --2- 172.21.15.47:0/2186479176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d6ee10 0x55b415d7aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.098+0000 7f8d3ea00640 1 -- 172.21.15.47:0/2186479176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d69ca0 msgr2=0x55b415d73a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.098+0000 7f8d3ea00640 1 --2- 172.21.15.47:0/2186479176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d69ca0 0x55b415d73a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.098+0000 7f8d3ea00640 1 -- 172.21.15.47:0/2186479176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b415b0bf70 con 0x55b415d692b0 2024-03-21T11:39:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.098+0000 7f8d48394740 1 -- 172.21.15.47:0/2186479176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d3000fba0 con 0x55b415d692b0 2024-03-21T11:39:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.098+0000 7f8d3ea00640 1 --2- 172.21.15.47:0/2186479176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d692b0 0x55b415d734c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8d30000c00 tx=0x7f8d300196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c75c7a19f488e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.098+0000 7f8d1fe00640 10 client.20298 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:39:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.098+0000 7f8d1fe00640 1 -- 172.21.15.47:0/2186479176 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30027060 con 0x55b415d692b0 2024-03-21T11:39:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.099+0000 7f8d1fe00640 1 -- 172.21.15.47:0/2186479176 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8d30019a80 con 0x55b415d692b0 2024-03-21T11:39:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.099+0000 7f8d1fe00640 1 -- 172.21.15.47:0/2186479176 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d3001e040 con 0x55b415d692b0 2024-03-21T11:39:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.099+0000 7f8d48394740 10 client.20298 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.099+0000 7f8d48394740 1 -- 172.21.15.47:0/2186479176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=34}) v3 -- 0x55b4159a3170 con 0x55b415d692b0 2024-03-21T11:39:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.100+0000 7f8d1fe00640 1 -- 172.21.15.47:0/2186479176 <== mon.0 v2:172.21.15.47:3300/0 4 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f8d30021070 con 0x55b415d692b0 2024-03-21T11:39:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.100+0000 7f8d48394740 10 client.20298 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.100+0000 7f8d48394740 10 client.20298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.100+0000 7f8d48394740 20 client.20298 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.100+0000 7f8d48394740 1 --2- 172.21.15.47:0/2186479176 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b415d7ef80 0x55b415d813a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.100+0000 7f8d48394740 4 client.20298 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.100+0000 7f8d48394740 1 -- 172.21.15.47:0/2186479176 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b415cdc3f0 con 0x55b415d7ef80 2024-03-21T11:39:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.100+0000 7f8d3e000640 1 --2- 172.21.15.47:0/2186479176 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b415d7ef80 0x55b415d813a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.101+0000 7f8d3e000640 1 --2- 172.21.15.47:0/2186479176 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b415d7ef80 0x55b415d813a0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e87e34efc9959c58 server_cookie=8950a876f787efcb in_seq=0 out_seq=0 2024-03-21T11:39:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.101+0000 7f8d1fe00640 10 client.20298 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.102+0000 7f8d1fe00640 1 -- 172.21.15.47:0/2186479176 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b415cdc3f0 con 0x55b415d7ef80 2024-03-21T11:39:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.102+0000 7f8d1fe00640 10 client.20298 handle_command_reply: tid=0 2024-03-21T11:39:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.102+0000 7f8d48394740 1 client.20298 shutdown 2024-03-21T11:39:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.103+0000 7f8d48394740 1 -- 172.21.15.47:0/2186479176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d692b0 msgr2=0x55b415d734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.103+0000 7f8d48394740 1 --2- 172.21.15.47:0/2186479176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d692b0 0x55b415d734c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8d30000c00 tx=0x7f8d300196a0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.103+0000 7f8d48394740 1 -- 172.21.15.47:0/2186479176 shutdown_connections 2024-03-21T11:39:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.103+0000 7f8d48394740 1 --2- 172.21.15.47:0/2186479176 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b415d7ef80 0x55b415d813a0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.103+0000 7f8d48394740 1 --2- 172.21.15.47:0/2186479176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d6ee10 0x55b415d7aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.103+0000 7f8d48394740 1 --2- 172.21.15.47:0/2186479176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d69ca0 0x55b415d73a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.103+0000 7f8d48394740 1 --2- 172.21.15.47:0/2186479176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d692b0 0x55b415d734c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.103+0000 7f8d48394740 1 -- 172.21.15.47:0/2186479176 >> 172.21.15.47:0/2186479176 conn(0x55b415d64820 msgr2=0x55b415d63a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.103+0000 7f8d1fe00640 0 client.20298 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.103+0000 7f8d48394740 1 -- 172.21.15.47:0/2186479176 shutdown_connections 2024-03-21T11:39:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.103+0000 7f8d48394740 1 -- 172.21.15.47:0/2186479176 wait complete. 2024-03-21T11:39:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.103+0000 7f8d48394740 20 client.20298 trim_cache size 0 max 16384 2024-03-21T11:39:13.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.123+0000 7f8d48394740 1 Processor -- start 2024-03-21T11:39:13.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.123+0000 7f8d48394740 1 -- start start 2024-03-21T11:39:13.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.123+0000 7f8d48394740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d69ca0 0x55b415d680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.123+0000 7f8d48394740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d6ee10 0x55b415d72b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.123+0000 7f8d48394740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d86c50 0x55b415d73050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.123+0000 7f8d48394740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d401b03b0 con 0x55b415d6ee10 2024-03-21T11:39:13.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.123+0000 7f8d48394740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d401b1eb0 con 0x55b415d86c50 2024-03-21T11:39:13.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.123+0000 7f8d48394740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d401b0a10 con 0x55b415d69ca0 2024-03-21T11:39:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.123+0000 7f8d3f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d86c50 0x55b415d73050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.123+0000 7f8d3e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d6ee10 0x55b415d72b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.123+0000 7f8d3f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d86c50 0x55b415d73050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36052/0 (socket says 172.21.15.47:36052) 2024-03-21T11:39:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.123+0000 7f8d3ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d69ca0 0x55b415d680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.123+0000 7f8d3f400640 1 -- 172.21.15.47:0/4219748476 learned_addr learned my addr 172.21.15.47:0/4219748476 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.124+0000 7f8d3f400640 1 -- 172.21.15.47:0/4219748476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d69ca0 msgr2=0x55b415d680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.124+0000 7f8d3f400640 1 --2- 172.21.15.47:0/4219748476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d69ca0 0x55b415d680f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.124+0000 7f8d3f400640 1 -- 172.21.15.47:0/4219748476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d6ee10 msgr2=0x55b415d72b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.124+0000 7f8d3f400640 1 --2- 172.21.15.47:0/4219748476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d6ee10 0x55b415d72b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.124+0000 7f8d3f400640 1 -- 172.21.15.47:0/4219748476 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b415d62fa0 con 0x55b415d86c50 2024-03-21T11:39:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.124+0000 7f8d3f400640 1 --2- 172.21.15.47:0/4219748476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d86c50 0x55b415d73050 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f8d380bdf40 tx=0x7f8d38004140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b590e1659f5f07c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.124+0000 7f8d2aa00640 1 -- 172.21.15.47:0/4219748476 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d380be940 con 0x55b415d86c50 2024-03-21T11:39:13.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.124+0000 7f8d2aa00640 1 -- 172.21.15.47:0/4219748476 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d3803a040 con 0x55b415d86c50 2024-03-21T11:39:13.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.124+0000 7f8d2aa00640 1 -- 172.21.15.47:0/4219748476 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d380bc370 con 0x55b415d86c50 2024-03-21T11:39:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.125+0000 7f8d48394740 1 -- 172.21.15.47:0/4219748476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d86c50 msgr2=0x55b415d73050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.125+0000 7f8d48394740 1 --2- 172.21.15.47:0/4219748476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d86c50 0x55b415d73050 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f8d380bdf40 tx=0x7f8d38004140 comp rx=0 tx=0).stop 2024-03-21T11:39:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.125+0000 7f8d48394740 1 -- 172.21.15.47:0/4219748476 shutdown_connections 2024-03-21T11:39:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.125+0000 7f8d48394740 1 --2- 172.21.15.47:0/4219748476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d86c50 0x55b415d73050 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.125+0000 7f8d48394740 1 --2- 172.21.15.47:0/4219748476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d6ee10 0x55b415d72b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.125+0000 7f8d48394740 1 --2- 172.21.15.47:0/4219748476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d69ca0 0x55b415d680f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.125+0000 7f8d48394740 1 -- 172.21.15.47:0/4219748476 >> 172.21.15.47:0/4219748476 conn(0x55b415d64820 msgr2=0x55b415d6afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.125+0000 7f8d48394740 1 -- 172.21.15.47:0/4219748476 shutdown_connections 2024-03-21T11:39:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.126+0000 7f8d48394740 1 -- 172.21.15.47:0/4219748476 wait complete. 2024-03-21T11:39:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.126+0000 7f8d48394740 1 -- start start 2024-03-21T11:39:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.126+0000 7f8d48394740 1 Processor -- start 2024-03-21T11:39:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.127+0000 7f8d48394740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d69ca0 0x55b415d79820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.127+0000 7f8d48394740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d6ee10 0x55b415d77e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.127+0000 7f8d48394740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d86c50 0x55b415d78370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.127+0000 7f8d48394740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d401b1790 con 0x55b415d6ee10 2024-03-21T11:39:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.127+0000 7f8d48394740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d401b1130 con 0x55b415d69ca0 2024-03-21T11:39:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.127+0000 7f8d48394740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d401ae570 con 0x55b415d86c50 2024-03-21T11:39:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.127+0000 7f8d48394740 10 client.0 _ll_register_callbacks cb 0x7f8d3001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.127+0000 7f8d3ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d69ca0 0x55b415d79820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.127+0000 7f8d3ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d69ca0 0x55b415d79820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36060/0 (socket says 172.21.15.47:36060) 2024-03-21T11:39:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.127+0000 7f8d3ea00640 1 -- 172.21.15.47:0/3795804373 learned_addr learned my addr 172.21.15.47:0/3795804373 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.128+0000 7f8d3e000640 1 --2- 172.21.15.47:0/3795804373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d6ee10 0x55b415d77e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.128+0000 7f8d3f400640 1 --2- 172.21.15.47:0/3795804373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d86c50 0x55b415d78370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.128+0000 7f8d3ea00640 1 -- 172.21.15.47:0/3795804373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d86c50 msgr2=0x55b415d78370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.128+0000 7f8d3ea00640 1 --2- 172.21.15.47:0/3795804373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d86c50 0x55b415d78370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.128+0000 7f8d3ea00640 1 -- 172.21.15.47:0/3795804373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d6ee10 msgr2=0x55b415d77e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.128+0000 7f8d3ea00640 1 --2- 172.21.15.47:0/3795804373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d6ee10 0x55b415d77e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.128+0000 7f8d3ea00640 1 -- 172.21.15.47:0/3795804373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b415d78ba0 con 0x55b415d69ca0 2024-03-21T11:39:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.128+0000 7f8d3f400640 1 --2- 172.21.15.47:0/3795804373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d86c50 0x55b415d78370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:13.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.131+0000 7f8d3ea00640 1 --2- 172.21.15.47:0/3795804373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d69ca0 0x55b415d79820 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009e70 tx=0x7f8d30005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=661ebe97c06c4f49 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:13.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.131+0000 7f8d1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:13.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.132+0000 7f8d1fe00640 1 -- 172.21.15.47:0/3795804373 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30039060 con 0x55b415d69ca0 2024-03-21T11:39:13.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.132+0000 7f8d48394740 1 -- 172.21.15.47:0/3795804373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d3803a040 con 0x55b415d69ca0 2024-03-21T11:39:13.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.132+0000 7f8d1fe00640 1 -- 172.21.15.47:0/3795804373 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8d30019a50 con 0x55b415d69ca0 2024-03-21T11:39:13.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.132+0000 7f8d1fe00640 1 -- 172.21.15.47:0/3795804373 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d30027020 con 0x55b415d69ca0 2024-03-21T11:39:13.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.132+0000 7f8d48394740 10 client.30788 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.132+0000 7f8d48394740 1 -- 172.21.15.47:0/3795804373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b415ddb850 con 0x55b415d69ca0 2024-03-21T11:39:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.133+0000 7f8d1fe00640 1 -- 172.21.15.47:0/3795804373 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d3803a040 con 0x55b415d69ca0 2024-03-21T11:39:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.133+0000 7f8d1fe00640 1 -- 172.21.15.47:0/3795804373 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f8d30033070 con 0x55b415d69ca0 2024-03-21T11:39:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.135+0000 7f8d48394740 10 client.30788 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.135+0000 7f8d48394740 10 client.30788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.135+0000 7f8d48394740 20 client.30788 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:13.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.135+0000 7f8d48394740 1 --2- 172.21.15.47:0/3795804373 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b415ddc180 0x55b415dde5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.135+0000 7f8d48394740 4 client.30788 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:39:13.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.135+0000 7f8d48394740 1 -- 172.21.15.47:0/3795804373 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b415cd6740 con 0x55b415ddc180 2024-03-21T11:39:13.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.135+0000 7f8d3e000640 1 --2- 172.21.15.47:0/3795804373 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b415ddc180 0x55b415dde5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.136+0000 7f8d3e000640 1 --2- 172.21.15.47:0/3795804373 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b415ddc180 0x55b415dde5a0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d84adc16d446d200 server_cookie=9de426ec63e0b35d in_seq=0 out_seq=0 2024-03-21T11:39:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.137+0000 7f8d1fe00640 10 client.30788 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:13.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.141+0000 7f8d1fe00640 1 -- 172.21.15.47:0/3795804373 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b415cd6740 con 0x55b415ddc180 2024-03-21T11:39:13.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.141+0000 7f8d1fe00640 10 client.30788 handle_command_reply: tid=0 2024-03-21T11:39:13.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.141+0000 7f8d48394740 1 client.30788 shutdown 2024-03-21T11:39:13.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.142+0000 7f8d48394740 1 -- 172.21.15.47:0/3795804373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d69ca0 msgr2=0x55b415d79820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.142+0000 7f8d48394740 1 --2- 172.21.15.47:0/3795804373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d69ca0 0x55b415d79820 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009e70 tx=0x7f8d30005f90 comp rx=0 tx=0).stop 2024-03-21T11:39:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.142+0000 7f8d48394740 1 -- 172.21.15.47:0/3795804373 shutdown_connections 2024-03-21T11:39:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.142+0000 7f8d48394740 1 --2- 172.21.15.47:0/3795804373 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b415ddc180 0x55b415dde5a0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.142+0000 7f8d48394740 1 --2- 172.21.15.47:0/3795804373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b415d86c50 0x55b415d78370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.142+0000 7f8d48394740 1 --2- 172.21.15.47:0/3795804373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b415d6ee10 0x55b415d77e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.142+0000 7f8d1fe00640 0 client.30788 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.142+0000 7f8d48394740 1 --2- 172.21.15.47:0/3795804373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b415d69ca0 0x55b415d79820 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.142+0000 7f8d48394740 1 -- 172.21.15.47:0/3795804373 >> 172.21.15.47:0/3795804373 conn(0x55b415d64820 msgr2=0x55b415d6a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.142+0000 7f8d48394740 1 -- 172.21.15.47:0/3795804373 shutdown_connections 2024-03-21T11:39:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.142+0000 7f8d48394740 1 -- 172.21.15.47:0/3795804373 wait complete. 2024-03-21T11:39:13.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.142+0000 7f8d48394740 20 client.30788 trim_cache size 0 max 16384 2024-03-21T11:39:13.146 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:39:13.146 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:39:13.146 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "fe01c825-324e-4dce-a94e-a189e7f42d39", 2024-03-21T11:39:13.146 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:39:13.146 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:39:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.143+0000 7f8d3fe00640 1 -- 172.21.15.47:0/1053518213 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d240a2fa0 msgr2=0x7f8d240a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.143+0000 7f8d3fe00640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d240a2fa0 0x7f8d240a5460 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009710 tx=0x7f8d2c005850 comp rx=0 tx=0).stop 2024-03-21T11:39:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.143+0000 7f8d3fe00640 1 -- 172.21.15.47:0/1053518213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d4010db70 msgr2=0x7f8d401a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.143+0000 7f8d3fe00640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d4010db70 0x7f8d401a5410 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8d3802c640 tx=0x7f8d380346f0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.144+0000 7f8d3fe00640 1 -- 172.21.15.47:0/1053518213 shutdown_connections 2024-03-21T11:39:13.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.144+0000 7f8d3fe00640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d240a2fa0 0x7f8d240a5460 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.144+0000 7f8d3fe00640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d4010db70 0x7f8d401a5410 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.144+0000 7f8d3fe00640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d400756e0 0x7f8d4019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.144+0000 7f8d3fe00640 1 --2- 172.21.15.47:0/1053518213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d40074100 0x7f8d4019deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.144+0000 7f8d3fe00640 1 -- 172.21.15.47:0/1053518213 >> 172.21.15.47:0/1053518213 conn(0x7f8d400fcb10 msgr2=0x7f8d40100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:13.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.145+0000 7f8d3fe00640 1 -- 172.21.15.47:0/1053518213 shutdown_connections 2024-03-21T11:39:13.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.145+0000 7f8d3fe00640 1 -- 172.21.15.47:0/1053518213 wait complete. 2024-03-21T11:39:13.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:39:13.168 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:39:13.242 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:13 smithi047 ceph-mon[31351]: pgmap v876: 129 pgs: 129 active+clean; 386 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 738 op/s 2024-03-21T11:39:13.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:13 smithi100 ceph-mon[36245]: pgmap v876: 129 pgs: 129 active+clean; 386 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 738 op/s 2024-03-21T11:39:13.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.400+0000 7fe231e00640 1 -- 172.21.15.47:0/1958188094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c1016b0 msgr2=0x7fe22c101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.400+0000 7fe231e00640 1 --2- 172.21.15.47:0/1958188094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c1016b0 0x7fe22c101ad0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009900 tx=0x7fe21c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:39:13.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.400+0000 7fe231e00640 1 -- 172.21.15.47:0/1958188094 shutdown_connections 2024-03-21T11:39:13.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.401+0000 7fe231e00640 1 --2- 172.21.15.47:0/1958188094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22c103260 0x7fe22c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.401+0000 7fe231e00640 1 --2- 172.21.15.47:0/1958188094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22c1028a0 0x7fe22c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.401+0000 7fe231e00640 1 --2- 172.21.15.47:0/1958188094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c1016b0 0x7fe22c101ad0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.401+0000 7fe231e00640 1 -- 172.21.15.47:0/1958188094 >> 172.21.15.47:0/1958188094 conn(0x7fe22c0fcc40 msgr2=0x7fe22c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:13.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.401+0000 7fe231e00640 1 -- 172.21.15.47:0/1958188094 shutdown_connections 2024-03-21T11:39:13.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.401+0000 7fe231e00640 1 -- 172.21.15.47:0/1958188094 wait complete. 2024-03-21T11:39:13.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.402+0000 7fe231e00640 1 Processor -- start 2024-03-21T11:39:13.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.402+0000 7fe231e00640 1 -- start start 2024-03-21T11:39:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.402+0000 7fe231e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22c1016b0 0x7fe22c199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.402+0000 7fe231e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22c1028a0 0x7fe22c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.402+0000 7fe231e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c103260 0x7fe22c1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.402+0000 7fe231e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c074d50 con 0x7fe22c1016b0 2024-03-21T11:39:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.402+0000 7fe231e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c074bd0 con 0x7fe22c1028a0 2024-03-21T11:39:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.402+0000 7fe231e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c074ed0 con 0x7fe22c103260 2024-03-21T11:39:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.403+0000 7fe22aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22c1016b0 0x7fe22c199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.403+0000 7fe22b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c103260 0x7fe22c1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.403+0000 7fe22a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22c1028a0 0x7fe22c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.403+0000 7fe22b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c103260 0x7fe22c1a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37590/0 (socket says 172.21.15.47:37590) 2024-03-21T11:39:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.403+0000 7fe22aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22c1016b0 0x7fe22c199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52520/0 (socket says 172.21.15.47:52520) 2024-03-21T11:39:13.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.403+0000 7fe22aa00640 1 -- 172.21.15.47:0/373957153 learned_addr learned my addr 172.21.15.47:0/373957153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.403+0000 7fe22b400640 1 -- 172.21.15.47:0/373957153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22c1028a0 msgr2=0x7fe22c19a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.403+0000 7fe22b400640 1 --2- 172.21.15.47:0/373957153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22c1028a0 0x7fe22c19a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.403+0000 7fe22b400640 1 -- 172.21.15.47:0/373957153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22c1016b0 msgr2=0x7fe22c199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.403+0000 7fe22b400640 1 --2- 172.21.15.47:0/373957153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22c1016b0 0x7fe22c199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.403+0000 7fe22b400640 1 -- 172.21.15.47:0/373957153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c1a1810 con 0x7fe22c103260 2024-03-21T11:39:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.403+0000 7fe22aa00640 1 --2- 172.21.15.47:0/373957153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22c1016b0 0x7fe22c199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.404+0000 7fe22b400640 1 --2- 172.21.15.47:0/373957153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c103260 0x7fe22c1a11a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe224009fd0 tx=0x7fe22400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.404+0000 7fe230a00640 1 -- 172.21.15.47:0/373957153 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22400ce20 con 0x7fe22c103260 2024-03-21T11:39:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.404+0000 7fe230a00640 1 -- 172.21.15.47:0/373957153 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe224005020 con 0x7fe22c103260 2024-03-21T11:39:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.404+0000 7fe230a00640 1 -- 172.21.15.47:0/373957153 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224010720 con 0x7fe22c103260 2024-03-21T11:39:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.404+0000 7fe231e00640 1 -- 172.21.15.47:0/373957153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22c1a1b00 con 0x7fe22c103260 2024-03-21T11:39:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.404+0000 7fe231e00640 1 -- 172.21.15.47:0/373957153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22c1a2000 con 0x7fe22c103260 2024-03-21T11:39:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.409+0000 7fe234c0f740 1 Processor -- start 2024-03-21T11:39:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.410+0000 7fe230a00640 1 -- 172.21.15.47:0/373957153 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe224005540 con 0x7fe22c103260 2024-03-21T11:39:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.410+0000 7fe234c0f740 1 -- start start 2024-03-21T11:39:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.411+0000 7fe234c0f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae0e80 0x55893aae12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.411+0000 7fe234c0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aae6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.411+0000 7fe234c0f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae69e0 0x55893aae8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.411+0000 7fe234c0f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893aa46250 con 0x55893aae69e0 2024-03-21T11:39:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.411+0000 7fe234c0f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893a9eea30 con 0x55893aae0e80 2024-03-21T11:39:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.411+0000 7fe234c0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893aa422f0 con 0x55893aae1870 2024-03-21T11:39:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.411+0000 7fe22aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aae6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.411+0000 7fe22a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae0e80 0x55893aae12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.411+0000 7fe22aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aae6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37602/0 (socket says 172.21.15.47:37602) 2024-03-21T11:39:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.411+0000 7fe22aa00640 1 -- 172.21.15.47:0/3056709602 learned_addr learned my addr 172.21.15.47:0/3056709602 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.411+0000 7fe22a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae0e80 0x55893aae12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36088/0 (socket says 172.21.15.47:36088) 2024-03-21T11:39:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.412+0000 7fe22a000640 1 --2- 172.21.15.47:0/3056709602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae69e0 0x55893aae8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.412+0000 7fe230a00640 1 --2- 172.21.15.47:0/373957153 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2100a2f20 0x7fe2100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.412+0000 7fe22a000640 1 -- 172.21.15.47:0/3056709602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 msgr2=0x55893aae6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.412+0000 7fe22a000640 1 --2- 172.21.15.47:0/3056709602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aae6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.412+0000 7fe22a000640 1 -- 172.21.15.47:0/3056709602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae69e0 msgr2=0x55893aae8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.412+0000 7fe22a000640 1 --2- 172.21.15.47:0/3056709602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae69e0 0x55893aae8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.412+0000 7fe22a000640 1 -- 172.21.15.47:0/3056709602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55893a9d9690 con 0x55893aae0e80 2024-03-21T11:39:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.412+0000 7fe230a00640 1 -- 172.21.15.47:0/373957153 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe2240cfe90 con 0x7fe22c103260 2024-03-21T11:39:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.412+0000 7fe22b400640 1 --2- 172.21.15.47:0/373957153 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2100a2f20 0x7fe2100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.412+0000 7fe22a000640 1 --2- 172.21.15.47:0/3056709602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae0e80 0x55893aae12a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe218009730 tx=0x7fe2180096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a5d1da585170181 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.413+0000 7fe217e00640 1 -- 172.21.15.47:0/3056709602 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218004050 con 0x55893aae0e80 2024-03-21T11:39:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.413+0000 7fe217e00640 1 -- 172.21.15.47:0/3056709602 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe218005500 con 0x55893aae0e80 2024-03-21T11:39:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.413+0000 7fe217e00640 1 -- 172.21.15.47:0/3056709602 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218004d60 con 0x55893aae0e80 2024-03-21T11:39:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.414+0000 7fe234c0f740 1 -- 172.21.15.47:0/3056709602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae0e80 msgr2=0x55893aae12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.414+0000 7fe234c0f740 1 --2- 172.21.15.47:0/3056709602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae0e80 0x55893aae12a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe218009730 tx=0x7fe2180096c0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.414+0000 7fe22b400640 1 --2- 172.21.15.47:0/373957153 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2100a2f20 0x7fe2100a53e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe22409c1a0 tx=0x7fe224005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.414+0000 7fe234c0f740 1 -- 172.21.15.47:0/3056709602 shutdown_connections 2024-03-21T11:39:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.414+0000 7fe234c0f740 1 --2- 172.21.15.47:0/3056709602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae69e0 0x55893aae8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.414+0000 7fe234c0f740 1 --2- 172.21.15.47:0/3056709602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aae6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.414+0000 7fe234c0f740 1 --2- 172.21.15.47:0/3056709602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae0e80 0x55893aae12a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.414+0000 7fe234c0f740 1 -- 172.21.15.47:0/3056709602 >> 172.21.15.47:0/3056709602 conn(0x55893aaa9200 msgr2=0x55893aadc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.414+0000 7fe234c0f740 1 -- 172.21.15.47:0/3056709602 shutdown_connections 2024-03-21T11:39:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.414+0000 7fe234c0f740 1 -- 172.21.15.47:0/3056709602 wait complete. 2024-03-21T11:39:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.415+0000 7fe234c0f740 1 -- start start 2024-03-21T11:39:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.415+0000 7fe234c0f740 1 Processor -- start 2024-03-21T11:39:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.416+0000 7fe234c0f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae0e80 0x55893aaeaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.416+0000 7fe234c0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaeb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.416+0000 7fe234c0f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae69e0 0x55893aaf2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.416+0000 7fe234c0f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893aae9990 con 0x55893aae0e80 2024-03-21T11:39:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.416+0000 7fe234c0f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893aa01e80 con 0x55893aae69e0 2024-03-21T11:39:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.416+0000 7fe234c0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893aae9b10 con 0x55893aae1870 2024-03-21T11:39:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.416+0000 7fe234c0f740 10 client.0 _ll_register_callbacks cb 0x55893aa14130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.416+0000 7fe22a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae0e80 0x55893aaeaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.416+0000 7fe22a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae0e80 0x55893aaeaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52538/0 (socket says 172.21.15.47:52538) 2024-03-21T11:39:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.416+0000 7fe22a000640 1 -- 172.21.15.47:0/2452919932 learned_addr learned my addr 172.21.15.47:0/2452919932 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.416+0000 7fe22a000640 1 --2- 172.21.15.47:0/2452919932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae69e0 0x55893aaf2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.417+0000 7fe22aa00640 1 --2- 172.21.15.47:0/2452919932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaeb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.417+0000 7fe22a000640 1 -- 172.21.15.47:0/2452919932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 msgr2=0x55893aaeb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.417+0000 7fe22a000640 1 --2- 172.21.15.47:0/2452919932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaeb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.417+0000 7fe22a000640 1 -- 172.21.15.47:0/2452919932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae69e0 msgr2=0x55893aaf2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.417+0000 7fe22a000640 1 --2- 172.21.15.47:0/2452919932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae69e0 0x55893aaf2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.417+0000 7fe22a000640 1 -- 172.21.15.47:0/2452919932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55893a8247d0 con 0x55893aae0e80 2024-03-21T11:39:13.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.417+0000 7fe22a000640 1 --2- 172.21.15.47:0/2452919932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae0e80 0x55893aaeaeb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe2180054d0 tx=0x7fe2180045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21569c56f536773 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:13.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.417+0000 7fe214c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:39:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.418+0000 7fe214c00640 1 -- 172.21.15.47:0/2452919932 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218004b70 con 0x55893aae0e80 2024-03-21T11:39:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.418+0000 7fe214c00640 1 -- 172.21.15.47:0/2452919932 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe218004d10 con 0x55893aae0e80 2024-03-21T11:39:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.418+0000 7fe234c0f740 1 -- 172.21.15.47:0/2452919932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe218005500 con 0x55893aae0e80 2024-03-21T11:39:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.418+0000 7fe214c00640 1 -- 172.21.15.47:0/2452919932 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe21800f790 con 0x55893aae0e80 2024-03-21T11:39:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.418+0000 7fe234c0f740 10 client.20316 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.418+0000 7fe234c0f740 1 -- 172.21.15.47:0/2452919932 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55893a75a3f0 con 0x55893aae0e80 2024-03-21T11:39:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.419+0000 7fe214c00640 1 -- 172.21.15.47:0/2452919932 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe218005500 con 0x55893aae0e80 2024-03-21T11:39:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.419+0000 7fe214c00640 1 -- 172.21.15.47:0/2452919932 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fe218036020 con 0x55893aae0e80 2024-03-21T11:39:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.419+0000 7fe234c0f740 10 client.20316 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.419+0000 7fe234c0f740 10 client.20316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.419+0000 7fe234c0f740 20 client.20316 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.419+0000 7fe234c0f740 1 --2- 172.21.15.47:0/2452919932 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55893aaf6a50 0x55893aaf8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.419+0000 7fe234c0f740 4 client.20316 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:13.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.419+0000 7fe234c0f740 1 -- 172.21.15.47:0/2452919932 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55893aa53790 con 0x55893aaf6a50 2024-03-21T11:39:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.419+0000 7fe22aa00640 1 --2- 172.21.15.47:0/2452919932 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55893aaf6a50 0x55893aaf8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.420+0000 7fe22aa00640 1 --2- 172.21.15.47:0/2452919932 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55893aaf6a50 0x55893aaf8e70 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95909eb4bd06e448 server_cookie=dbfde432832d81f in_seq=0 out_seq=0 2024-03-21T11:39:13.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.420+0000 7fe214c00640 10 client.20316 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.423+0000 7fe214c00640 1 -- 172.21.15.47:0/2452919932 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55893aa53790 con 0x55893aaf6a50 2024-03-21T11:39:13.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.423+0000 7fe214c00640 10 client.20316 handle_command_reply: tid=0 2024-03-21T11:39:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.423+0000 7fe234c0f740 1 client.20316 shutdown 2024-03-21T11:39:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.423+0000 7fe234c0f740 1 -- 172.21.15.47:0/2452919932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae0e80 msgr2=0x55893aaeaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.423+0000 7fe234c0f740 1 --2- 172.21.15.47:0/2452919932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae0e80 0x55893aaeaeb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe2180054d0 tx=0x7fe2180045e0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.423+0000 7fe234c0f740 1 -- 172.21.15.47:0/2452919932 shutdown_connections 2024-03-21T11:39:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.423+0000 7fe234c0f740 1 --2- 172.21.15.47:0/2452919932 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55893aaf6a50 0x55893aaf8e70 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.423+0000 7fe234c0f740 1 --2- 172.21.15.47:0/2452919932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae69e0 0x55893aaf2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.423+0000 7fe234c0f740 1 --2- 172.21.15.47:0/2452919932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaeb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.423+0000 7fe214c00640 0 client.20316 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.423+0000 7fe234c0f740 1 --2- 172.21.15.47:0/2452919932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae0e80 0x55893aaeaeb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.423+0000 7fe234c0f740 1 -- 172.21.15.47:0/2452919932 >> 172.21.15.47:0/2452919932 conn(0x55893aaa9200 msgr2=0x55893aadba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.424+0000 7fe234c0f740 1 -- 172.21.15.47:0/2452919932 shutdown_connections 2024-03-21T11:39:13.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.424+0000 7fe234c0f740 1 -- 172.21.15.47:0/2452919932 wait complete. 2024-03-21T11:39:13.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.424+0000 7fe234c0f740 20 client.20316 trim_cache size 0 max 16384 2024-03-21T11:39:13.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.450+0000 7fe234c0f740 1 Processor -- start 2024-03-21T11:39:13.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.450+0000 7fe234c0f740 1 -- start start 2024-03-21T11:39:13.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.451+0000 7fe234c0f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae0e80 0x55893aadb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.451+0000 7fe234c0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaf23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.451+0000 7fe234c0f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae69e0 0x55893aadb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.451+0000 7fe234c0f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1ac350 con 0x55893aae69e0 2024-03-21T11:39:13.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.451+0000 7fe234c0f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1ade50 con 0x55893aae0e80 2024-03-21T11:39:13.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.451+0000 7fe234c0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1ac9b0 con 0x55893aae1870 2024-03-21T11:39:13.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.451+0000 7fe22aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaf23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.451+0000 7fe22a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae0e80 0x55893aadb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.451+0000 7fe22aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaf23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37620/0 (socket says 172.21.15.47:37620) 2024-03-21T11:39:13.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.451+0000 7fe22aa00640 1 -- 172.21.15.47:0/2107595807 learned_addr learned my addr 172.21.15.47:0/2107595807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:13.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.451+0000 7fe22a000640 1 --2- 172.21.15.47:0/2107595807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae69e0 0x55893aadb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.452+0000 7fe22aa00640 1 -- 172.21.15.47:0/2107595807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae0e80 msgr2=0x55893aadb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.452+0000 7fe22aa00640 1 --2- 172.21.15.47:0/2107595807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae0e80 0x55893aadb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.452+0000 7fe22aa00640 1 -- 172.21.15.47:0/2107595807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae69e0 msgr2=0x55893aadb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.452+0000 7fe22aa00640 1 --2- 172.21.15.47:0/2107595807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae69e0 0x55893aadb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.452+0000 7fe22aa00640 1 -- 172.21.15.47:0/2107595807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55893a71a0f0 con 0x55893aae1870 2024-03-21T11:39:13.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.452+0000 7fe22a000640 1 --2- 172.21.15.47:0/2107595807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae69e0 0x55893aadb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:39:13.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.452+0000 7fe22aa00640 1 --2- 172.21.15.47:0/2107595807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaf23f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fe21c02c8f0 tx=0x7fe21c02e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=737c2479e890839 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:13.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.453+0000 7fe217400640 1 -- 172.21.15.47:0/2107595807 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c004240 con 0x55893aae1870 2024-03-21T11:39:13.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.453+0000 7fe217400640 1 -- 172.21.15.47:0/2107595807 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe21c0043e0 con 0x55893aae1870 2024-03-21T11:39:13.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.453+0000 7fe217400640 1 -- 172.21.15.47:0/2107595807 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c005580 con 0x55893aae1870 2024-03-21T11:39:13.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.453+0000 7fe234c0f740 1 -- 172.21.15.47:0/2107595807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 msgr2=0x55893aaf23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.453+0000 7fe234c0f740 1 --2- 172.21.15.47:0/2107595807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaf23f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fe21c02c8f0 tx=0x7fe21c02e040 comp rx=0 tx=0).stop 2024-03-21T11:39:13.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.454+0000 7fe234c0f740 1 -- 172.21.15.47:0/2107595807 shutdown_connections 2024-03-21T11:39:13.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.454+0000 7fe234c0f740 1 --2- 172.21.15.47:0/2107595807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae69e0 0x55893aadb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.454+0000 7fe234c0f740 1 --2- 172.21.15.47:0/2107595807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaf23f0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.454+0000 7fe234c0f740 1 --2- 172.21.15.47:0/2107595807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae0e80 0x55893aadb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.454+0000 7fe234c0f740 1 -- 172.21.15.47:0/2107595807 >> 172.21.15.47:0/2107595807 conn(0x55893aaa9200 msgr2=0x55893aaeaac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:13.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.454+0000 7fe234c0f740 1 -- 172.21.15.47:0/2107595807 shutdown_connections 2024-03-21T11:39:13.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.454+0000 7fe234c0f740 1 -- 172.21.15.47:0/2107595807 wait complete. 2024-03-21T11:39:13.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.454+0000 7fe234c0f740 1 -- start start 2024-03-21T11:39:13.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.454+0000 7fe234c0f740 1 Processor -- start 2024-03-21T11:39:13.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.455+0000 7fe234c0f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae0e80 0x55893aaefdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.455+0000 7fe234c0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaf0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.455+0000 7fe234c0f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae69e0 0x55893aaf1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.455+0000 7fe234c0f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1ad730 con 0x55893aae0e80 2024-03-21T11:39:13.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.455+0000 7fe234c0f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1ad0d0 con 0x55893aae69e0 2024-03-21T11:39:13.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.455+0000 7fe234c0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1aa510 con 0x55893aae1870 2024-03-21T11:39:13.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.455+0000 7fe234c0f740 10 client.0 _ll_register_callbacks cb 0x7fe218009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:13.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.455+0000 7fe22aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaf0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.455+0000 7fe22aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaf0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37632/0 (socket says 172.21.15.47:37632) 2024-03-21T11:39:13.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.455+0000 7fe22aa00640 1 -- 172.21.15.47:0/534901343 learned_addr learned my addr 172.21.15.47:0/534901343 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:13.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.455+0000 7fe22a000640 1 --2- 172.21.15.47:0/534901343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae0e80 0x55893aaefdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.456+0000 7fe22a000640 1 --2- 172.21.15.47:0/534901343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae69e0 0x55893aaf1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.456+0000 7fe22aa00640 1 -- 172.21.15.47:0/534901343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae69e0 msgr2=0x55893aaf1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.456+0000 7fe22aa00640 1 --2- 172.21.15.47:0/534901343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae69e0 0x55893aaf1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.456+0000 7fe22aa00640 1 -- 172.21.15.47:0/534901343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae0e80 msgr2=0x55893aaefdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:13.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.456+0000 7fe22aa00640 1 --2- 172.21.15.47:0/534901343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae0e80 0x55893aaefdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.456+0000 7fe22aa00640 1 -- 172.21.15.47:0/534901343 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55893aaf1930 con 0x55893aae1870 2024-03-21T11:39:13.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.456+0000 7fe22a000640 1 --2- 172.21.15.47:0/534901343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae69e0 0x55893aaf1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:13.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.456+0000 7fe22aa00640 1 --2- 172.21.15.47:0/534901343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaf0330 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fe21c005ab0 tx=0x7fe21c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e45adae4806dd6c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:13.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.456+0000 7fe214c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:13.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.457+0000 7fe214c00640 1 -- 172.21.15.47:0/534901343 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c056060 con 0x55893aae1870 2024-03-21T11:39:13.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.457+0000 7fe234c0f740 1 -- 172.21.15.47:0/534901343 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe21c0043e0 con 0x55893aae1870 2024-03-21T11:39:13.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.457+0000 7fe214c00640 1 -- 172.21.15.47:0/534901343 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe21c0046c0 con 0x55893aae1870 2024-03-21T11:39:13.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.457+0000 7fe214c00640 1 -- 172.21.15.47:0/534901343 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe21c05d050 con 0x55893aae1870 2024-03-21T11:39:13.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.457+0000 7fe234c0f740 10 client.30811 fetch_fsmap learned FSMap version 34 2024-03-21T11:39:13.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.457+0000 7fe234c0f740 1 -- 172.21.15.47:0/534901343 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55893ab52670 con 0x55893aae1870 2024-03-21T11:39:13.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.457+0000 7fe214c00640 1 -- 172.21.15.47:0/534901343 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe21c0043e0 con 0x55893aae1870 2024-03-21T11:39:13.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.458+0000 7fe214c00640 1 -- 172.21.15.47:0/534901343 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fe21c049c10 con 0x55893aae1870 2024-03-21T11:39:13.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.458+0000 7fe234c0f740 10 client.30811 fetch_fsmap finished waiting for FSMap version 34 2024-03-21T11:39:13.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.458+0000 7fe234c0f740 10 client.30811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:13.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.458+0000 7fe234c0f740 20 client.30811 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:13.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.458+0000 7fe234c0f740 1 --2- 172.21.15.47:0/534901343 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55893aaf6a50 0x55893ab550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:13.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.458+0000 7fe234c0f740 4 client.30811 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:39:13.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.458+0000 7fe234c0f740 1 -- 172.21.15.47:0/534901343 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55893aa4dae0 con 0x55893aaf6a50 2024-03-21T11:39:13.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.458+0000 7fe22a000640 1 --2- 172.21.15.47:0/534901343 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55893aaf6a50 0x55893ab550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:13.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.459+0000 7fe22a000640 1 --2- 172.21.15.47:0/534901343 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55893aaf6a50 0x55893ab550a0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=888774efc800b351 server_cookie=cd9ce2c4b7677db2 in_seq=0 out_seq=0 2024-03-21T11:39:13.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.459+0000 7fe214c00640 10 client.30811 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:13.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.462+0000 7fe214c00640 1 -- 172.21.15.47:0/534901343 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55893aa4dae0 con 0x55893aaf6a50 2024-03-21T11:39:13.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.462+0000 7fe214c00640 10 client.30811 handle_command_reply: tid=0 2024-03-21T11:39:13.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.462+0000 7fe234c0f740 1 client.30811 shutdown 2024-03-21T11:39:13.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.462+0000 7fe234c0f740 1 -- 172.21.15.47:0/534901343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 msgr2=0x55893aaf0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.462+0000 7fe234c0f740 1 --2- 172.21.15.47:0/534901343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaf0330 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fe21c005ab0 tx=0x7fe21c005fd0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.463+0000 7fe234c0f740 1 -- 172.21.15.47:0/534901343 shutdown_connections 2024-03-21T11:39:13.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.463+0000 7fe234c0f740 1 --2- 172.21.15.47:0/534901343 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55893aaf6a50 0x55893ab550a0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.463+0000 7fe234c0f740 1 --2- 172.21.15.47:0/534901343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55893aae69e0 0x55893aaf1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.463+0000 7fe234c0f740 1 --2- 172.21.15.47:0/534901343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55893aae1870 0x55893aaf0330 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.463+0000 7fe234c0f740 1 --2- 172.21.15.47:0/534901343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55893aae0e80 0x55893aaefdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.463+0000 7fe214c00640 0 client.30811 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:13.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.463+0000 7fe234c0f740 1 -- 172.21.15.47:0/534901343 >> 172.21.15.47:0/534901343 conn(0x55893aaa9200 msgr2=0x55893aae22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:13.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.463+0000 7fe234c0f740 1 -- 172.21.15.47:0/534901343 shutdown_connections 2024-03-21T11:39:13.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.465+0000 7fe234c0f740 1 -- 172.21.15.47:0/534901343 wait complete. 2024-03-21T11:39:13.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.465+0000 7fe234c0f740 20 client.30811 trim_cache size 0 max 16384 2024-03-21T11:39:13.466 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:39:13.466 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:39:13.466 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:39:13.466 INFO:teuthology.orchestra.run.smithi047.stdout: "fe01c825-324e-4dce-a94e-a189e7f42d39": { 2024-03-21T11:39:13.466 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:39:13.466 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "fe01c825-324e-4dce-a94e-a189e7f42d39", 2024-03-21T11:39:13.466 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:39:13.466 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:39:13.466 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:39:13.467 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:39:13.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.466+0000 7fe231e00640 1 -- 172.21.15.47:0/373957153 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2100a2f20 msgr2=0x7fe2100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.466+0000 7fe231e00640 1 --2- 172.21.15.47:0/373957153 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2100a2f20 0x7fe2100a53e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe22409c1a0 tx=0x7fe224005940 comp rx=0 tx=0).stop 2024-03-21T11:39:13.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.466+0000 7fe231e00640 1 -- 172.21.15.47:0/373957153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c103260 msgr2=0x7fe22c1a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:13.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.466+0000 7fe231e00640 1 --2- 172.21.15.47:0/373957153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c103260 0x7fe22c1a11a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe224009fd0 tx=0x7fe22400ef10 comp rx=0 tx=0).stop 2024-03-21T11:39:13.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.466+0000 7fe231e00640 1 -- 172.21.15.47:0/373957153 shutdown_connections 2024-03-21T11:39:13.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.466+0000 7fe231e00640 1 --2- 172.21.15.47:0/373957153 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2100a2f20 0x7fe2100a53e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.466+0000 7fe231e00640 1 --2- 172.21.15.47:0/373957153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe22c103260 0x7fe22c1a11a0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.466+0000 7fe231e00640 1 --2- 172.21.15.47:0/373957153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22c1028a0 0x7fe22c19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.466+0000 7fe231e00640 1 --2- 172.21.15.47:0/373957153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe22c1016b0 0x7fe22c199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:13.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.466+0000 7fe231e00640 1 -- 172.21.15.47:0/373957153 >> 172.21.15.47:0/373957153 conn(0x7fe22c0fcc40 msgr2=0x7fe22c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:13.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.466+0000 7fe231e00640 1 -- 172.21.15.47:0/373957153 shutdown_connections 2024-03-21T11:39:13.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:13.466+0000 7fe231e00640 1 -- 172.21.15.47:0/373957153 wait complete. 2024-03-21T11:39:13.486 INFO:tasks.cephfs.filesystem:scrub status for tag:fe01c825-324e-4dce-a94e-a189e7f42d39 - {'path': '/', 'tag': 'fe01c825-324e-4dce-a94e-a189e7f42d39', 'options': 'recursive,force'} 2024-03-21T11:39:13.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:13 smithi102 ceph-mon[36748]: pgmap v876: 129 pgs: 129 active+clean; 386 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 738 op/s 2024-03-21T11:39:13.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 56934 27.75 MB/sec warmup 117 sec latency 72.329 ms 2024-03-21T11:39:14.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:39:14.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:14.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:14.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:14.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:14.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:14.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:14.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:14.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:14.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:14.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:14 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:39:14.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:14.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:14.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:14.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:14.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:14.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:14.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:14.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:14.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:14.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:14 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:39:14.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:14.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:14.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:14.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:14.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:14.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:14.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:14.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:14.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:14.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:14 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:14.929 INFO:tasks.workunit.client.0.smithi047.stdout: 10 57374 27.69 MB/sec warmup 118 sec latency 68.702 ms 2024-03-21T11:39:15.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:15 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:15.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:15 smithi102 ceph-mon[36748]: pgmap v877: 129 pgs: 129 active+clean; 343 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 13 MiB/s wr, 856 op/s 2024-03-21T11:39:15.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:15 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:15.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:15 smithi047 ceph-mon[31351]: pgmap v877: 129 pgs: 129 active+clean; 343 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 13 MiB/s wr, 856 op/s 2024-03-21T11:39:15.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:15 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:15.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:15 smithi100 ceph-mon[36245]: pgmap v877: 129 pgs: 129 active+clean; 343 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 13 MiB/s wr, 856 op/s 2024-03-21T11:39:15.929 INFO:tasks.workunit.client.0.smithi047.stdout: 10 57896 27.68 MB/sec warmup 119 sec latency 43.459 ms 2024-03-21T11:39:16.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:16 smithi102 ceph-mon[36748]: pgmap v878: 129 pgs: 129 active+clean; 330 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 760 op/s 2024-03-21T11:39:16.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:16 smithi047 ceph-mon[31351]: pgmap v878: 129 pgs: 129 active+clean; 330 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 760 op/s 2024-03-21T11:39:16.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:16 smithi100 ceph-mon[36245]: pgmap v878: 129 pgs: 129 active+clean; 330 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 760 op/s 2024-03-21T11:39:17.929 INFO:tasks.workunit.client.0.smithi047.stdout: 10 59079 37.68 MB/sec execute 1 sec latency 84.663 ms 2024-03-21T11:39:18.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:18 smithi100 ceph-mon[36245]: pgmap v879: 129 pgs: 129 active+clean; 331 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 780 op/s 2024-03-21T11:39:18.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 59521 32.91 MB/sec execute 2 sec latency 77.355 ms 2024-03-21T11:39:18.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:18 smithi102 ceph-mon[36748]: pgmap v879: 129 pgs: 129 active+clean; 331 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 780 op/s 2024-03-21T11:39:19.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:18 smithi047 ceph-mon[31351]: pgmap v879: 129 pgs: 129 active+clean; 331 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 780 op/s 2024-03-21T11:39:19.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:19 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:19.929 INFO:tasks.workunit.client.0.smithi047.stdout: 10 60078 29.77 MB/sec execute 3 sec latency 44.229 ms 2024-03-21T11:39:19.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:19 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:20.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:19 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.165+0000 7f4debe00640 1 -- 172.21.15.47:0/3798488791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4dec075660 msgr2=0x7f4dec073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.165+0000 7f4debe00640 1 --2- 172.21.15.47:0/3798488791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4dec075660 0x7f4dec073ab0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009900 tx=0x7f4ddc02bcd0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.165+0000 7f4debe00640 1 -- 172.21.15.47:0/3798488791 shutdown_connections 2024-03-21T11:39:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.165+0000 7f4debe00640 1 --2- 172.21.15.47:0/3798488791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dec1093d0 0x7f4dec10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.165+0000 7f4debe00640 1 --2- 172.21.15.47:0/3798488791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dec074080 0x7f4dec0744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.165+0000 7f4debe00640 1 --2- 172.21.15.47:0/3798488791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4dec075660 0x7f4dec073ab0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.165+0000 7f4debe00640 1 -- 172.21.15.47:0/3798488791 >> 172.21.15.47:0/3798488791 conn(0x7f4dec0fcab0 msgr2=0x7f4dec0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.165+0000 7f4debe00640 1 -- 172.21.15.47:0/3798488791 shutdown_connections 2024-03-21T11:39:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.165+0000 7f4debe00640 1 -- 172.21.15.47:0/3798488791 wait complete. 2024-03-21T11:39:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.166+0000 7f4debe00640 1 Processor -- start 2024-03-21T11:39:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.166+0000 7f4debe00640 1 -- start start 2024-03-21T11:39:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.166+0000 7f4debe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4dec074080 0x7f4dec199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4debe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dec075660 0x7f4dec199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4debe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dec1093d0 0x7f4dec1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4debe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec10f3f0 con 0x7f4dec075660 2024-03-21T11:39:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4debe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec10f270 con 0x7f4dec1093d0 2024-03-21T11:39:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4debe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec10f570 con 0x7f4dec074080 2024-03-21T11:39:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4deaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4dec074080 0x7f4dec199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4deaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4dec074080 0x7f4dec199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49734/0 (socket says 172.21.15.47:49734) 2024-03-21T11:39:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4deaa00640 1 -- 172.21.15.47:0/3100316565 learned_addr learned my addr 172.21.15.47:0/3100316565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4deb400640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dec1093d0 0x7f4dec1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4dea000640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dec075660 0x7f4dec199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4deaa00640 1 -- 172.21.15.47:0/3100316565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dec1093d0 msgr2=0x7f4dec1a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4deaa00640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dec1093d0 0x7f4dec1a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4deaa00640 1 -- 172.21.15.47:0/3100316565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dec075660 msgr2=0x7f4dec199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4deaa00640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dec075660 0x7f4dec199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.167+0000 7f4deaa00640 1 -- 172.21.15.47:0/3100316565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec1a1620 con 0x7f4dec074080 2024-03-21T11:39:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.168+0000 7f4dea000640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dec075660 0x7f4dec199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.168+0000 7f4deaa00640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4dec074080 0x7f4dec199a00 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc000c00 tx=0x7f4ddc02c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.168+0000 7f4de3400640 1 -- 172.21.15.47:0/3100316565 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc043020 con 0x7f4dec074080 2024-03-21T11:39:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.168+0000 7f4de3400640 1 -- 172.21.15.47:0/3100316565 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ddc0054e0 con 0x7f4dec074080 2024-03-21T11:39:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.168+0000 7f4de3400640 1 -- 172.21.15.47:0/3100316565 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc003c20 con 0x7f4dec074080 2024-03-21T11:39:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.168+0000 7f4debe00640 1 -- 172.21.15.47:0/3100316565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dec1a18b0 con 0x7f4dec074080 2024-03-21T11:39:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.169+0000 7f4debe00640 1 -- 172.21.15.47:0/3100316565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dec1a1db0 con 0x7f4dec074080 2024-03-21T11:39:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.172+0000 7f4df45f9740 1 Processor -- start 2024-03-21T11:39:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.172+0000 7f4df45f9740 1 -- start start 2024-03-21T11:39:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.173+0000 7f4df45f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88b53f0 0x5633b88b5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.173+0000 7f4df45f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b5de0 0x5633b88ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.173+0000 7f4df45f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88baf50 0x5633b88bd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.173+0000 7f4df45f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b88783b0 con 0x5633b88b5de0 2024-03-21T11:39:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.173+0000 7f4df45f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b881a830 con 0x5633b88baf50 2024-03-21T11:39:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.173+0000 7f4df45f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b87c35d0 con 0x5633b88b53f0 2024-03-21T11:39:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.173+0000 7f4de3400640 1 -- 172.21.15.47:0/3100316565 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4ddc046020 con 0x7f4dec074080 2024-03-21T11:39:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.173+0000 7f4deaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b5de0 0x5633b88ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.173+0000 7f4dea000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88baf50 0x5633b88bd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.173+0000 7f4dea000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88baf50 0x5633b88bd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60902/0 (socket says 172.21.15.47:60902) 2024-03-21T11:39:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.173+0000 7f4deaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b5de0 0x5633b88ba8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59920/0 (socket says 172.21.15.47:59920) 2024-03-21T11:39:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.173+0000 7f4dea000640 1 -- 172.21.15.47:0/3083144293 learned_addr learned my addr 172.21.15.47:0/3083144293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.174+0000 7f4deb400640 1 --2- 172.21.15.47:0/3083144293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88b53f0 0x5633b88b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.174+0000 7f4dea000640 1 -- 172.21.15.47:0/3083144293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88b53f0 msgr2=0x5633b88b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.174+0000 7f4dea000640 1 --2- 172.21.15.47:0/3083144293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88b53f0 0x5633b88b5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.174+0000 7f4dea000640 1 -- 172.21.15.47:0/3083144293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b5de0 msgr2=0x5633b88ba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.174+0000 7f4dea000640 1 --2- 172.21.15.47:0/3083144293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b5de0 0x5633b88ba8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.174+0000 7f4dea000640 1 -- 172.21.15.47:0/3083144293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633b87db2c0 con 0x5633b88baf50 2024-03-21T11:39:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.174+0000 7f4deb400640 1 --2- 172.21.15.47:0/3083144293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88b53f0 0x5633b88b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.174+0000 7f4deaa00640 1 --2- 172.21.15.47:0/3083144293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b5de0 0x5633b88ba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.174+0000 7f4dea000640 1 --2- 172.21.15.47:0/3083144293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88baf50 0x5633b88bd3e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009c00 tx=0x7f4dd8010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33aa5ed91dc92e05 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.175+0000 7f4dd7e00640 1 -- 172.21.15.47:0/3083144293 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8004030 con 0x5633b88baf50 2024-03-21T11:39:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.175+0000 7f4de3400640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4dd00a2f90 0x7f4dd00a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.175+0000 7f4dd7e00640 1 -- 172.21.15.47:0/3083144293 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dd80041d0 con 0x5633b88baf50 2024-03-21T11:39:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.175+0000 7f4dd7e00640 1 -- 172.21.15.47:0/3083144293 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8012670 con 0x5633b88baf50 2024-03-21T11:39:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.175+0000 7f4de3400640 1 -- 172.21.15.47:0/3100316565 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4ddc0f1840 con 0x7f4dec074080 2024-03-21T11:39:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.176+0000 7f4deb400640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4dd00a2f90 0x7f4dd00a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.176+0000 7f4df45f9740 1 -- 172.21.15.47:0/3083144293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88baf50 msgr2=0x5633b88bd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.176+0000 7f4df45f9740 1 --2- 172.21.15.47:0/3083144293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88baf50 0x5633b88bd3e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009c00 tx=0x7f4dd8010740 comp rx=0 tx=0).stop 2024-03-21T11:39:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.177+0000 7f4deb400640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4dd00a2f90 0x7f4dd00a5450 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5633b88b6640 tx=0x7f4de4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.177+0000 7f4df45f9740 1 -- 172.21.15.47:0/3083144293 shutdown_connections 2024-03-21T11:39:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.177+0000 7f4df45f9740 1 --2- 172.21.15.47:0/3083144293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88baf50 0x5633b88bd3e0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.177+0000 7f4df45f9740 1 --2- 172.21.15.47:0/3083144293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b5de0 0x5633b88ba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.177+0000 7f4df45f9740 1 --2- 172.21.15.47:0/3083144293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88b53f0 0x5633b88b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.177+0000 7f4df45f9740 1 -- 172.21.15.47:0/3083144293 >> 172.21.15.47:0/3083144293 conn(0x5633b88b0960 msgr2=0x5633b88b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.177+0000 7f4df45f9740 1 -- 172.21.15.47:0/3083144293 shutdown_connections 2024-03-21T11:39:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.177+0000 7f4df45f9740 1 -- 172.21.15.47:0/3083144293 wait complete. 2024-03-21T11:39:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.178+0000 7f4df45f9740 1 -- start start 2024-03-21T11:39:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.178+0000 7f4df45f9740 1 Processor -- start 2024-03-21T11:39:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.179+0000 7f4df45f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b53f0 0x5633b88bf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.179+0000 7f4df45f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b5de0 0x5633b88bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.179+0000 7f4df45f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88baf50 0x5633b88c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.179+0000 7f4df45f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b87d6a20 con 0x5633b88b5de0 2024-03-21T11:39:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.179+0000 7f4df45f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b88168d0 con 0x5633b88b53f0 2024-03-21T11:39:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.179+0000 7f4df45f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b88bdf00 con 0x5633b88baf50 2024-03-21T11:39:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.179+0000 7f4df45f9740 10 client.0 _ll_register_callbacks cb 0x5633b88131c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.179+0000 7f4deaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b53f0 0x5633b88bf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.179+0000 7f4deaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b53f0 0x5633b88bf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60904/0 (socket says 172.21.15.47:60904) 2024-03-21T11:39:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.179+0000 7f4deaa00640 1 -- 172.21.15.47:0/3380964720 learned_addr learned my addr 172.21.15.47:0/3380964720 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.179+0000 7f4dea000640 1 --2- 172.21.15.47:0/3380964720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b5de0 0x5633b88bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.179+0000 7f4deb400640 1 --2- 172.21.15.47:0/3380964720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88baf50 0x5633b88c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.180+0000 7f4deaa00640 1 -- 172.21.15.47:0/3380964720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88baf50 msgr2=0x5633b88c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.180+0000 7f4deaa00640 1 --2- 172.21.15.47:0/3380964720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88baf50 0x5633b88c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.180+0000 7f4deaa00640 1 -- 172.21.15.47:0/3380964720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b5de0 msgr2=0x5633b88bfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.180+0000 7f4deaa00640 1 --2- 172.21.15.47:0/3380964720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b5de0 0x5633b88bfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.180+0000 7f4deaa00640 1 -- 172.21.15.47:0/3380964720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633b852f210 con 0x5633b88b53f0 2024-03-21T11:39:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.180+0000 7f4df45f9740 1 -- 172.21.15.47:0/3380964720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4dd80041d0 con 0x5633b88b53f0 2024-03-21T11:39:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.180+0000 7f4deaa00640 1 --2- 172.21.15.47:0/3380964720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b53f0 0x5633b88bf5a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc005ab0 tx=0x7f4ddc009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b171444040bfb47 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.180+0000 7f4dd4c00640 10 client.30815 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.180+0000 7f4dd4c00640 1 -- 172.21.15.47:0/3380964720 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc0bd6c0 con 0x5633b88b53f0 2024-03-21T11:39:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.180+0000 7f4dd4c00640 1 -- 172.21.15.47:0/3380964720 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4ddc0ef060 con 0x5633b88b53f0 2024-03-21T11:39:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.180+0000 7f4dd4c00640 1 -- 172.21.15.47:0/3380964720 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ddc0bd4e0 con 0x5633b88b53f0 2024-03-21T11:39:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.180+0000 7f4df45f9740 10 client.30815 fetch_fsmap learned FSMap version 35 2024-03-21T11:39:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.180+0000 7f4df45f9740 1 -- 172.21.15.47:0/3380964720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=35}) v3 -- 0x5633b85fb620 con 0x5633b88b53f0 2024-03-21T11:39:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.181+0000 7f4dd4c00640 1 -- 172.21.15.47:0/3380964720 <== mon.2 v2:172.21.15.100:3300/0 4 ==== fsmap(e 35) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f4ddc0c6100 con 0x5633b88b53f0 2024-03-21T11:39:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.181+0000 7f4df45f9740 10 client.30815 fetch_fsmap finished waiting for FSMap version 35 2024-03-21T11:39:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.181+0000 7f4df45f9740 10 client.30815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.181+0000 7f4df45f9740 20 client.30815 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.181+0000 7f4df45f9740 1 --2- 172.21.15.47:0/3380964720 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633b88dc1f0 0x5633b88de5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.181+0000 7f4df45f9740 4 client.30815 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.181+0000 7f4df45f9740 1 -- 172.21.15.47:0/3380964720 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633b8827d70 con 0x5633b88dc1f0 2024-03-21T11:39:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.181+0000 7f4dea000640 1 --2- 172.21.15.47:0/3380964720 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633b88dc1f0 0x5633b88de5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.182+0000 7f4dea000640 1 --2- 172.21.15.47:0/3380964720 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633b88dc1f0 0x5633b88de5e0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f804c93e2460f81c server_cookie=bbc2d4d3f93ccf11 in_seq=0 out_seq=0 2024-03-21T11:39:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.182+0000 7f4dd4c00640 10 client.30815 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.184+0000 7f4dd4c00640 1 -- 172.21.15.47:0/3380964720 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5633b8827d70 con 0x5633b88dc1f0 2024-03-21T11:39:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.185+0000 7f4dd4c00640 10 client.30815 handle_command_reply: tid=0 2024-03-21T11:39:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.185+0000 7f4df45f9740 1 client.30815 shutdown 2024-03-21T11:39:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.185+0000 7f4df45f9740 1 -- 172.21.15.47:0/3380964720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b53f0 msgr2=0x5633b88bf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.185+0000 7f4df45f9740 1 --2- 172.21.15.47:0/3380964720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b53f0 0x5633b88bf5a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc005ab0 tx=0x7f4ddc009560 comp rx=0 tx=0).stop 2024-03-21T11:39:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.186+0000 7f4df45f9740 1 -- 172.21.15.47:0/3380964720 shutdown_connections 2024-03-21T11:39:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.186+0000 7f4df45f9740 1 --2- 172.21.15.47:0/3380964720 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633b88dc1f0 0x5633b88de5e0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.186+0000 7f4df45f9740 1 --2- 172.21.15.47:0/3380964720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88baf50 0x5633b88c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.186+0000 7f4df45f9740 1 --2- 172.21.15.47:0/3380964720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b5de0 0x5633b88bfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.186+0000 7f4dd4c00640 0 client.30815 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.186+0000 7f4df45f9740 1 --2- 172.21.15.47:0/3380964720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b53f0 0x5633b88bf5a0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.186+0000 7f4df45f9740 1 -- 172.21.15.47:0/3380964720 >> 172.21.15.47:0/3380964720 conn(0x5633b88b0960 msgr2=0x5633b88afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.186+0000 7f4df45f9740 1 -- 172.21.15.47:0/3380964720 shutdown_connections 2024-03-21T11:39:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.187+0000 7f4df45f9740 1 -- 172.21.15.47:0/3380964720 wait complete. 2024-03-21T11:39:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.187+0000 7f4df45f9740 20 client.30815 trim_cache size 0 max 16384 2024-03-21T11:39:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.208+0000 7f4df45f9740 1 Processor -- start 2024-03-21T11:39:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.208+0000 7f4df45f9740 1 -- start start 2024-03-21T11:39:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.208+0000 7f4df45f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b53f0 0x5633b88bebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.208+0000 7f4df45f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b5de0 0x5633b88c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.208+0000 7f4df45f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88baf50 0x5633b88bf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.208+0000 7f4df45f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec1a38a0 con 0x5633b88b53f0 2024-03-21T11:39:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.208+0000 7f4df45f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec1a53a0 con 0x5633b88b5de0 2024-03-21T11:39:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.208+0000 7f4df45f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec1a3f00 con 0x5633b88baf50 2024-03-21T11:39:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.209+0000 7f4dea000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b5de0 0x5633b88c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.209+0000 7f4dea000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b5de0 0x5633b88c6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60914/0 (socket says 172.21.15.47:60914) 2024-03-21T11:39:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.209+0000 7f4dea000640 1 -- 172.21.15.47:0/684670056 learned_addr learned my addr 172.21.15.47:0/684670056 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.209+0000 7f4deaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b53f0 0x5633b88bebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.209+0000 7f4deaa00640 1 -- 172.21.15.47:0/684670056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88baf50 msgr2=0x5633b88bf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.209+0000 7f4deaa00640 1 --2- 172.21.15.47:0/684670056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88baf50 0x5633b88bf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.209+0000 7f4deaa00640 1 -- 172.21.15.47:0/684670056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b5de0 msgr2=0x5633b88c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.209+0000 7f4deaa00640 1 --2- 172.21.15.47:0/684670056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b5de0 0x5633b88c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.209+0000 7f4deaa00640 1 -- 172.21.15.47:0/684670056 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633b88aef80 con 0x5633b88b53f0 2024-03-21T11:39:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.209+0000 7f4dea000640 1 --2- 172.21.15.47:0/684670056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b5de0 0x5633b88c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.209+0000 7f4deaa00640 1 --2- 172.21.15.47:0/684670056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b53f0 0x5633b88bebc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0b9fe0 tx=0x7f4ddc0f6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360848cddba7b3a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.210+0000 7f4dd7400640 1 -- 172.21.15.47:0/684670056 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc0bd6c0 con 0x5633b88b53f0 2024-03-21T11:39:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.210+0000 7f4dd7400640 1 -- 172.21.15.47:0/684670056 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dd80041d0 con 0x5633b88b53f0 2024-03-21T11:39:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.210+0000 7f4dd7400640 1 -- 172.21.15.47:0/684670056 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc0c1cd0 con 0x5633b88b53f0 2024-03-21T11:39:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.211+0000 7f4df45f9740 1 -- 172.21.15.47:0/684670056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b53f0 msgr2=0x5633b88bebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.211+0000 7f4df45f9740 1 --2- 172.21.15.47:0/684670056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b53f0 0x5633b88bebc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0b9fe0 tx=0x7f4ddc0f6000 comp rx=0 tx=0).stop 2024-03-21T11:39:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.211+0000 7f4df45f9740 1 -- 172.21.15.47:0/684670056 shutdown_connections 2024-03-21T11:39:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.211+0000 7f4df45f9740 1 --2- 172.21.15.47:0/684670056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88baf50 0x5633b88bf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.211+0000 7f4df45f9740 1 --2- 172.21.15.47:0/684670056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b5de0 0x5633b88c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.211+0000 7f4df45f9740 1 --2- 172.21.15.47:0/684670056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88b53f0 0x5633b88bebc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.211+0000 7f4df45f9740 1 -- 172.21.15.47:0/684670056 >> 172.21.15.47:0/684670056 conn(0x5633b88b0960 msgr2=0x5633b88b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.211+0000 7f4df45f9740 1 -- 172.21.15.47:0/684670056 shutdown_connections 2024-03-21T11:39:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.211+0000 7f4df45f9740 1 -- 172.21.15.47:0/684670056 wait complete. 2024-03-21T11:39:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.211+0000 7f4df45f9740 1 -- start start 2024-03-21T11:39:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.212+0000 7f4df45f9740 1 Processor -- start 2024-03-21T11:39:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.212+0000 7f4df45f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b53f0 0x5633b88c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.212+0000 7f4df45f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88b5de0 0x5633b88c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.212+0000 7f4df45f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88baf50 0x5633b88c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.212+0000 7f4df45f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec1a4c80 con 0x5633b88baf50 2024-03-21T11:39:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.212+0000 7f4df45f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec1a4620 con 0x5633b88b53f0 2024-03-21T11:39:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.212+0000 7f4df45f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec073310 con 0x5633b88b5de0 2024-03-21T11:39:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.212+0000 7f4df45f9740 10 client.0 _ll_register_callbacks cb 0x7f4ddc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.213+0000 7f4dea000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88b5de0 0x5633b88c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.213+0000 7f4dea000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88b5de0 0x5633b88c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49776/0 (socket says 172.21.15.47:49776) 2024-03-21T11:39:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.213+0000 7f4dea000640 1 -- 172.21.15.47:0/2894767391 learned_addr learned my addr 172.21.15.47:0/2894767391 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.213+0000 7f4deaa00640 1 --2- 172.21.15.47:0/2894767391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b53f0 0x5633b88c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.213+0000 7f4deaa00640 1 -- 172.21.15.47:0/2894767391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88b5de0 msgr2=0x5633b88c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.213+0000 7f4deaa00640 1 --2- 172.21.15.47:0/2894767391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88b5de0 0x5633b88c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.213+0000 7f4deaa00640 1 -- 172.21.15.47:0/2894767391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88baf50 msgr2=0x5633b88c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.213+0000 7f4deaa00640 1 --2- 172.21.15.47:0/2894767391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88baf50 0x5633b88c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.213+0000 7f4deaa00640 1 -- 172.21.15.47:0/2894767391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633b88c62a0 con 0x5633b88b53f0 2024-03-21T11:39:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.213+0000 7f4dea000640 1 --2- 172.21.15.47:0/2894767391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88b5de0 0x5633b88c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.213+0000 7f4deaa00640 1 --2- 172.21.15.47:0/2894767391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b53f0 0x5633b88c4760 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0bd880 tx=0x7f4ddc009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c52eb719e73c3a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.213+0000 7f4dd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.214+0000 7f4dd4c00640 1 -- 172.21.15.47:0/2894767391 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc005690 con 0x5633b88b53f0 2024-03-21T11:39:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.214+0000 7f4dd4c00640 1 -- 172.21.15.47:0/2894767391 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4ddc004ef0 con 0x5633b88b53f0 2024-03-21T11:39:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.214+0000 7f4df45f9740 1 -- 172.21.15.47:0/2894767391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4dd80041d0 con 0x5633b88b53f0 2024-03-21T11:39:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.214+0000 7f4dd4c00640 1 -- 172.21.15.47:0/2894767391 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ddc0be280 con 0x5633b88b53f0 2024-03-21T11:39:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.214+0000 7f4df45f9740 10 client.30821 fetch_fsmap learned FSMap version 35 2024-03-21T11:39:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.214+0000 7f4df45f9740 1 -- 172.21.15.47:0/2894767391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5633b8940890 con 0x5633b88b53f0 2024-03-21T11:39:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.215+0000 7f4dd4c00640 1 -- 172.21.15.47:0/2894767391 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dd80041d0 con 0x5633b88b53f0 2024-03-21T11:39:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.215+0000 7f4dd4c00640 1 -- 172.21.15.47:0/2894767391 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 35) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f4ddc09cb10 con 0x5633b88b53f0 2024-03-21T11:39:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.215+0000 7f4df45f9740 10 client.30821 fetch_fsmap finished waiting for FSMap version 35 2024-03-21T11:39:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.215+0000 7f4df45f9740 10 client.30821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.215+0000 7f4df45f9740 20 client.30821 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.215+0000 7f4df45f9740 1 --2- 172.21.15.47:0/2894767391 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633b8940fa0 0x5633b89433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.215+0000 7f4df45f9740 4 client.30821 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:39:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.215+0000 7f4df45f9740 1 -- 172.21.15.47:0/2894767391 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5633b88220c0 con 0x5633b8940fa0 2024-03-21T11:39:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.215+0000 7f4dea000640 1 --2- 172.21.15.47:0/2894767391 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633b8940fa0 0x5633b89433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.216+0000 7f4dea000640 1 --2- 172.21.15.47:0/2894767391 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633b8940fa0 0x5633b89433c0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f637825593d435f server_cookie=c36f18cd57d16b36 in_seq=0 out_seq=0 2024-03-21T11:39:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.216+0000 7f4dd4c00640 10 client.30821 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.824+0000 7f4dd4c00640 1 -- 172.21.15.47:0/2894767391 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5633b88220c0 con 0x5633b8940fa0 2024-03-21T11:39:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.824+0000 7f4dd4c00640 10 client.30821 handle_command_reply: tid=0 2024-03-21T11:39:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.824+0000 7f4df45f9740 1 client.30821 shutdown 2024-03-21T11:39:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.828+0000 7f4df45f9740 1 -- 172.21.15.47:0/2894767391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b53f0 msgr2=0x5633b88c4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.828+0000 7f4df45f9740 1 --2- 172.21.15.47:0/2894767391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b53f0 0x5633b88c4760 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0bd880 tx=0x7f4ddc009560 comp rx=0 tx=0).stop 2024-03-21T11:39:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.828+0000 7f4df45f9740 1 -- 172.21.15.47:0/2894767391 shutdown_connections 2024-03-21T11:39:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.828+0000 7f4df45f9740 1 --2- 172.21.15.47:0/2894767391 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633b8940fa0 0x5633b89433c0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.828+0000 7f4df45f9740 1 --2- 172.21.15.47:0/2894767391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633b88baf50 0x5633b88c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.828+0000 7f4df45f9740 1 --2- 172.21.15.47:0/2894767391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633b88b5de0 0x5633b88c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.828+0000 7f4df45f9740 1 --2- 172.21.15.47:0/2894767391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633b88b53f0 0x5633b88c4760 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.828+0000 7f4df45f9740 1 -- 172.21.15.47:0/2894767391 >> 172.21.15.47:0/2894767391 conn(0x5633b88b0960 msgr2=0x5633b88ba320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.828+0000 7f4dd4c00640 0 client.30821 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.829+0000 7f4df45f9740 1 -- 172.21.15.47:0/2894767391 shutdown_connections 2024-03-21T11:39:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.829+0000 7f4df45f9740 1 -- 172.21.15.47:0/2894767391 wait complete. 2024-03-21T11:39:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.829+0000 7f4df45f9740 20 client.30821 trim_cache size 0 max 16384 2024-03-21T11:39:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:39:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:39:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:39:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:39:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.831+0000 7f4debe00640 1 -- 172.21.15.47:0/3100316565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4dd00a2f90 msgr2=0x7f4dd00a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.831+0000 7f4debe00640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4dd00a2f90 0x7f4dd00a5450 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5633b88b6640 tx=0x7f4de4009500 comp rx=0 tx=0).stop 2024-03-21T11:39:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.831+0000 7f4debe00640 1 -- 172.21.15.47:0/3100316565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4dec074080 msgr2=0x7f4dec199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.831+0000 7f4debe00640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4dec074080 0x7f4dec199a00 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc000c00 tx=0x7f4ddc02c6f0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.832+0000 7f4debe00640 1 -- 172.21.15.47:0/3100316565 shutdown_connections 2024-03-21T11:39:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.832+0000 7f4debe00640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4dd00a2f90 0x7f4dd00a5450 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.832+0000 7f4debe00640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4dec1093d0 0x7f4dec1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.832+0000 7f4debe00640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dec075660 0x7f4dec199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.832+0000 7f4debe00640 1 --2- 172.21.15.47:0/3100316565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4dec074080 0x7f4dec199a00 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.832+0000 7f4debe00640 1 -- 172.21.15.47:0/3100316565 >> 172.21.15.47:0/3100316565 conn(0x7f4dec0fcab0 msgr2=0x7f4dec105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.832+0000 7f4debe00640 1 -- 172.21.15.47:0/3100316565 shutdown_connections 2024-03-21T11:39:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:20.832+0000 7f4debe00640 1 -- 172.21.15.47:0/3100316565 wait complete. 2024-03-21T11:39:20.929 INFO:tasks.workunit.client.0.smithi047.stdout: 10 60377 26.43 MB/sec execute 4 sec latency 431.944 ms 2024-03-21T11:39:20.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:20 smithi102 ceph-mon[36748]: pgmap v880: 129 pgs: 129 active+clean; 338 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.6 MiB/s wr, 798 op/s 2024-03-21T11:39:21.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:20 smithi047 ceph-mon[31351]: pgmap v880: 129 pgs: 129 active+clean; 338 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.6 MiB/s wr, 798 op/s 2024-03-21T11:39:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:20 smithi100 ceph-mon[36245]: pgmap v880: 129 pgs: 129 active+clean; 338 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.6 MiB/s wr, 798 op/s 2024-03-21T11:39:21.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 60973 26.48 MB/sec execute 5 sec latency 82.249 ms 2024-03-21T11:39:22.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 61591 29.37 MB/sec execute 6 sec latency 52.954 ms 2024-03-21T11:39:22.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:22 smithi102 ceph-mon[36748]: pgmap v881: 129 pgs: 129 active+clean; 343 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.2 MiB/s wr, 606 op/s 2024-03-21T11:39:23.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:22 smithi047 ceph-mon[31351]: pgmap v881: 129 pgs: 129 active+clean; 343 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.2 MiB/s wr, 606 op/s 2024-03-21T11:39:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:22 smithi100 ceph-mon[36245]: pgmap v881: 129 pgs: 129 active+clean; 343 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.2 MiB/s wr, 606 op/s 2024-03-21T11:39:23.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 62120 28.91 MB/sec execute 7 sec latency 44.880 ms 2024-03-21T11:39:24.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 62546 27.43 MB/sec execute 8 sec latency 73.546 ms 2024-03-21T11:39:24.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:24 smithi102 ceph-mon[36748]: pgmap v882: 129 pgs: 129 active+clean; 344 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.1 MiB/s wr, 659 op/s 2024-03-21T11:39:25.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:24 smithi047 ceph-mon[31351]: pgmap v882: 129 pgs: 129 active+clean; 344 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.1 MiB/s wr, 659 op/s 2024-03-21T11:39:25.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:24 smithi100 ceph-mon[36245]: pgmap v882: 129 pgs: 129 active+clean; 344 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.1 MiB/s wr, 659 op/s 2024-03-21T11:39:25.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 63175 28.59 MB/sec execute 9 sec latency 67.916 ms 2024-03-21T11:39:26.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 63670 29.14 MB/sec execute 10 sec latency 74.529 ms 2024-03-21T11:39:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:26 smithi102 ceph-mon[36748]: pgmap v883: 129 pgs: 129 active+clean; 341 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.7 MiB/s wr, 546 op/s 2024-03-21T11:39:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:26 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:26 smithi047 ceph-mon[31351]: pgmap v883: 129 pgs: 129 active+clean; 341 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.7 MiB/s wr, 546 op/s 2024-03-21T11:39:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:26 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:26 smithi100 ceph-mon[36245]: pgmap v883: 129 pgs: 129 active+clean; 341 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.7 MiB/s wr, 546 op/s 2024-03-21T11:39:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:26 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:27.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 64261 28.49 MB/sec execute 11 sec latency 51.155 ms 2024-03-21T11:39:27.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:27 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:39:27.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:27 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:39:27.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:27 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:39:28.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:27 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:39:28.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:27 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:39:28.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:27 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:39:28.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:27 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:39:28.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:27 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:39:28.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:27 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:39:28.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 64705 28.32 MB/sec execute 12 sec latency 71.114 ms 2024-03-21T11:39:28.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:28 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:39:28.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:28 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:39:28.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:28 smithi102 ceph-mon[36748]: pgmap v884: 129 pgs: 129 active+clean; 348 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.1 MiB/s wr, 579 op/s 2024-03-21T11:39:29.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:28 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:39:29.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:28 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:39:29.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:28 smithi047 ceph-mon[31351]: pgmap v884: 129 pgs: 129 active+clean; 348 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.1 MiB/s wr, 579 op/s 2024-03-21T11:39:29.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:28 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:39:29.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:28 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:39:29.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:28 smithi100 ceph-mon[36245]: pgmap v884: 129 pgs: 129 active+clean; 348 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.1 MiB/s wr, 579 op/s 2024-03-21T11:39:29.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 65379 28.68 MB/sec execute 13 sec latency 45.547 ms 2024-03-21T11:39:29.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:29 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:30.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:29 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:30.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:29 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:30.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 66014 28.97 MB/sec execute 14 sec latency 175.001 ms 2024-03-21T11:39:30.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:30 smithi102 ceph-mon[36748]: pgmap v885: 129 pgs: 129 active+clean; 354 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.4 MiB/s wr, 597 op/s 2024-03-21T11:39:31.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:30 smithi047 ceph-mon[31351]: pgmap v885: 129 pgs: 129 active+clean; 354 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.4 MiB/s wr, 597 op/s 2024-03-21T11:39:31.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:30 smithi100 ceph-mon[36245]: pgmap v885: 129 pgs: 129 active+clean; 354 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.4 MiB/s wr, 597 op/s 2024-03-21T11:39:31.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 66296 28.47 MB/sec execute 15 sec latency 531.297 ms 2024-03-21T11:39:31.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:31 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:31.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:31 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:32.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:31 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:32.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 66680 27.94 MB/sec execute 16 sec latency 102.681 ms 2024-03-21T11:39:32.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:32 smithi102 ceph-mon[36748]: pgmap v886: 129 pgs: 129 active+clean; 356 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.9 MiB/s wr, 512 op/s 2024-03-21T11:39:33.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:32 smithi047 ceph-mon[31351]: pgmap v886: 129 pgs: 129 active+clean; 356 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.9 MiB/s wr, 512 op/s 2024-03-21T11:39:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:32 smithi100 ceph-mon[36245]: pgmap v886: 129 pgs: 129 active+clean; 356 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.9 MiB/s wr, 512 op/s 2024-03-21T11:39:33.930 INFO:tasks.workunit.client.0.smithi047.stdout: 10 67145 28.21 MB/sec execute 17 sec latency 69.252 ms 2024-03-21T11:39:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.982+0000 7f4625600640 1 -- 172.21.15.47:0/3914804761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4620104af0 msgr2=0x7f4620104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.982+0000 7f4625600640 1 --2- 172.21.15.47:0/3914804761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4620104af0 0x7f4620104f70 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f460c009900 tx=0x7f460c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:39:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.982+0000 7f4625600640 1 -- 172.21.15.47:0/3914804761 shutdown_connections 2024-03-21T11:39:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.982+0000 7f4625600640 1 --2- 172.21.15.47:0/3914804761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46201054b0 0x7f462010bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.982+0000 7f4625600640 1 --2- 172.21.15.47:0/3914804761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4620104af0 0x7f4620104f70 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.982+0000 7f4625600640 1 --2- 172.21.15.47:0/3914804761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4620103900 0x7f4620103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.982+0000 7f4625600640 1 -- 172.21.15.47:0/3914804761 >> 172.21.15.47:0/3914804761 conn(0x7f46200fee90 msgr2=0x7f46201012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.982+0000 7f4625600640 1 -- 172.21.15.47:0/3914804761 shutdown_connections 2024-03-21T11:39:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.982+0000 7f4625600640 1 -- 172.21.15.47:0/3914804761 wait complete. 2024-03-21T11:39:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.983+0000 7f4625600640 1 Processor -- start 2024-03-21T11:39:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.983+0000 7f4625600640 1 -- start start 2024-03-21T11:39:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.983+0000 7f4625600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4620103900 0x7f4620199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.983+0000 7f4625600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4620104af0 0x7f462019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.983+0000 7f4625600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46201054b0 0x7f46201a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.984+0000 7f4625600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462010f5b0 con 0x7f4620103900 2024-03-21T11:39:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.984+0000 7f4625600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462010f430 con 0x7f46201054b0 2024-03-21T11:39:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.984+0000 7f4625600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462010f730 con 0x7f4620104af0 2024-03-21T11:39:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.984+0000 7f461e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4620104af0 0x7f462019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.984+0000 7f461ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4620103900 0x7f4620199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.984+0000 7f461e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4620104af0 0x7f462019a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60052/0 (socket says 172.21.15.47:60052) 2024-03-21T11:39:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.984+0000 7f461e000640 1 -- 172.21.15.47:0/1548791997 learned_addr learned my addr 172.21.15.47:0/1548791997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.984+0000 7f461f400640 1 --2- 172.21.15.47:0/1548791997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46201054b0 0x7f46201a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.984+0000 7f461e000640 1 -- 172.21.15.47:0/1548791997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46201054b0 msgr2=0x7f46201a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.984+0000 7f461e000640 1 --2- 172.21.15.47:0/1548791997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46201054b0 0x7f46201a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.984+0000 7f461e000640 1 -- 172.21.15.47:0/1548791997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4620103900 msgr2=0x7f4620199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.984+0000 7f461e000640 1 --2- 172.21.15.47:0/1548791997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4620103900 0x7f4620199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.984+0000 7f461e000640 1 -- 172.21.15.47:0/1548791997 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46201a1750 con 0x7f4620104af0 2024-03-21T11:39:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.984+0000 7f461e000640 1 --2- 172.21.15.47:0/1548791997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4620104af0 0x7f462019a070 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f460c02c8f0 tx=0x7f460c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.985+0000 7f4617400640 1 -- 172.21.15.47:0/1548791997 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c041070 con 0x7f4620104af0 2024-03-21T11:39:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.985+0000 7f4617400640 1 -- 172.21.15.47:0/1548791997 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f460c0044d0 con 0x7f4620104af0 2024-03-21T11:39:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.985+0000 7f4617400640 1 -- 172.21.15.47:0/1548791997 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c034440 con 0x7f4620104af0 2024-03-21T11:39:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.985+0000 7f4625600640 1 -- 172.21.15.47:0/1548791997 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46201a19e0 con 0x7f4620104af0 2024-03-21T11:39:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.985+0000 7f4625600640 1 -- 172.21.15.47:0/1548791997 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46201a1ed0 con 0x7f4620104af0 2024-03-21T11:39:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.991+0000 7f4628411740 1 Processor -- start 2024-03-21T11:39:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.991+0000 7f4628411740 1 -- start start 2024-03-21T11:39:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.991+0000 7f4617400640 1 -- 172.21.15.47:0/1548791997 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f460c044070 con 0x7f4620104af0 2024-03-21T11:39:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.991+0000 7f4628411740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b64600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.992+0000 7f4628411740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b64bd0 0x55faf4b696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.992+0000 7f4628411740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b6c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.992+0000 7f4628411740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf4b273b0 con 0x55faf4b64bd0 2024-03-21T11:39:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.992+0000 7f4628411740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf4ac9830 con 0x55faf4b641e0 2024-03-21T11:39:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.992+0000 7f4628411740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf4a725d0 con 0x55faf4b69d40 2024-03-21T11:39:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.993+0000 7f461f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b64600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.993+0000 7f461f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b64600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54290/0 (socket says 172.21.15.47:54290) 2024-03-21T11:39:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.993+0000 7f461f400640 1 -- 172.21.15.47:0/4158598563 learned_addr learned my addr 172.21.15.47:0/4158598563 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.993+0000 7f461f400640 1 -- 172.21.15.47:0/4158598563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 msgr2=0x55faf4b6c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.994+0000 7f461e000640 1 --2- 172.21.15.47:0/4158598563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b6c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.994+0000 7f461f400640 1 --2- 172.21.15.47:0/4158598563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b6c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.994+0000 7f461f400640 1 -- 172.21.15.47:0/4158598563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b64bd0 msgr2=0x55faf4b696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.994+0000 7f461f400640 1 --2- 172.21.15.47:0/4158598563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b64bd0 0x55faf4b696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.994+0000 7f461f400640 1 -- 172.21.15.47:0/4158598563 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faf4a8a2c0 con 0x55faf4b641e0 2024-03-21T11:39:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.994+0000 7f461f400640 1 --2- 172.21.15.47:0/4158598563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b64600 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4618009f50 tx=0x7f46180118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=850a9960618ec9c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.995+0000 7f460b400640 1 -- 172.21.15.47:0/4158598563 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618004b20 con 0x55faf4b641e0 2024-03-21T11:39:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.995+0000 7f460b400640 1 -- 172.21.15.47:0/4158598563 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4618004cc0 con 0x55faf4b641e0 2024-03-21T11:39:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.995+0000 7f460b400640 1 -- 172.21.15.47:0/4158598563 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618005020 con 0x55faf4b641e0 2024-03-21T11:39:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.996+0000 7f4628411740 1 -- 172.21.15.47:0/4158598563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 msgr2=0x55faf4b64600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.996+0000 7f4628411740 1 --2- 172.21.15.47:0/4158598563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b64600 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4618009f50 tx=0x7f46180118d0 comp rx=0 tx=0).stop 2024-03-21T11:39:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.996+0000 7f4617400640 1 --2- 172.21.15.47:0/1548791997 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46040a3040 0x7f46040a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.996+0000 7f4617400640 1 -- 172.21.15.47:0/1548791997 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f460c0f1370 con 0x7f4620104af0 2024-03-21T11:39:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.997+0000 7f461f400640 1 --2- 172.21.15.47:0/1548791997 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46040a3040 0x7f46040a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.998+0000 7f4628411740 1 -- 172.21.15.47:0/4158598563 shutdown_connections 2024-03-21T11:39:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.998+0000 7f4628411740 1 --2- 172.21.15.47:0/4158598563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b6c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.998+0000 7f4628411740 1 --2- 172.21.15.47:0/4158598563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b64bd0 0x55faf4b696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.998+0000 7f4628411740 1 --2- 172.21.15.47:0/4158598563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b64600 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.998+0000 7f4628411740 1 -- 172.21.15.47:0/4158598563 >> 172.21.15.47:0/4158598563 conn(0x55faf4b5f750 msgr2=0x55faf4b5fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:33.999+0000 7f4628411740 1 -- 172.21.15.47:0/4158598563 shutdown_connections 2024-03-21T11:39:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.000+0000 7f4628411740 1 -- 172.21.15.47:0/4158598563 wait complete. 2024-03-21T11:39:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.000+0000 7f461f400640 1 --2- 172.21.15.47:0/1548791997 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46040a3040 0x7f46040a5500 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4618009d60 tx=0x7f4618004220 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.001+0000 7f4628411740 1 -- start start 2024-03-21T11:39:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.001+0000 7f4628411740 1 Processor -- start 2024-03-21T11:39:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.002+0000 7f4628411740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b6e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.002+0000 7f4628411740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b64bd0 0x55faf4b6e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.002+0000 7f4628411740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.002+0000 7f4628411740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf4a85a20 con 0x55faf4b64bd0 2024-03-21T11:39:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.002+0000 7f4628411740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf4ac58d0 con 0x55faf4b641e0 2024-03-21T11:39:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.002+0000 7f4628411740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faf4b6ccf0 con 0x55faf4b69d40 2024-03-21T11:39:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.002+0000 7f461f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.002+0000 7f461f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b758a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60066/0 (socket says 172.21.15.47:60066) 2024-03-21T11:39:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.003+0000 7f461f400640 1 -- 172.21.15.47:0/1688308547 learned_addr learned my addr 172.21.15.47:0/1688308547 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.003+0000 7f461ea00640 1 --2- 172.21.15.47:0/1688308547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b6e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.003+0000 7f4628411740 10 client.0 _ll_register_callbacks cb 0x55faf4ac21c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.003+0000 7f461f400640 1 -- 172.21.15.47:0/1688308547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 msgr2=0x55faf4b6e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.003+0000 7f461f400640 1 --2- 172.21.15.47:0/1688308547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b6e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.003+0000 7f461f400640 1 -- 172.21.15.47:0/1688308547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b64bd0 msgr2=0x55faf4b6e860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:39:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.004+0000 7f461f400640 1 --2- 172.21.15.47:0/1688308547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b64bd0 0x55faf4b6e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.004+0000 7f461f400640 1 -- 172.21.15.47:0/1688308547 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55faf47de210 con 0x55faf4b69d40 2024-03-21T11:39:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.005+0000 7f4628411740 1 -- 172.21.15.47:0/1688308547 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4618004cc0 con 0x55faf4b69d40 2024-03-21T11:39:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.005+0000 7f461ea00640 1 --2- 172.21.15.47:0/1688308547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b6e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.005+0000 7f461f400640 1 --2- 172.21.15.47:0/1688308547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b758a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f461801a000 tx=0x7f46180118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af2e0590b0294e26 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.005+0000 7f45ffe00640 10 client.30832 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.006+0000 7f45ffe00640 1 -- 172.21.15.47:0/1688308547 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618032070 con 0x55faf4b69d40 2024-03-21T11:39:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.006+0000 7f45ffe00640 1 -- 172.21.15.47:0/1688308547 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f461800f480 con 0x55faf4b69d40 2024-03-21T11:39:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.006+0000 7f45ffe00640 1 -- 172.21.15.47:0/1688308547 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f461800f680 con 0x55faf4b69d40 2024-03-21T11:39:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.006+0000 7f4628411740 10 client.30832 fetch_fsmap learned FSMap version 38 2024-03-21T11:39:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.006+0000 7f4628411740 1 -- 172.21.15.47:0/1688308547 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=38}) v3 -- 0x55faf48aa620 con 0x55faf4b69d40 2024-03-21T11:39:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.007+0000 7f45ffe00640 1 -- 172.21.15.47:0/1688308547 <== mon.1 v2:172.21.15.102:3300/0 4 ==== fsmap(e 38) v1 ==== 7958+0+0 (secure 0 0 0) 0x7f461802d070 con 0x55faf4b69d40 2024-03-21T11:39:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.007+0000 7f4628411740 10 client.30832 fetch_fsmap finished waiting for FSMap version 38 2024-03-21T11:39:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.007+0000 7f4628411740 10 client.30832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.007+0000 7f4628411740 20 client.30832 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.007+0000 7f4628411740 1 --2- 172.21.15.47:0/1688308547 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55faf4b8aef0 0x55faf4b8d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.007+0000 7f4628411740 4 client.30832 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.007+0000 7f4628411740 1 -- 172.21.15.47:0/1688308547 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55faf4ad6d70 con 0x55faf4b8aef0 2024-03-21T11:39:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.007+0000 7f461ea00640 1 --2- 172.21.15.47:0/1688308547 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55faf4b8aef0 0x55faf4b8d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.008+0000 7f461ea00640 1 --2- 172.21.15.47:0/1688308547 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55faf4b8aef0 0x55faf4b8d370 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e483d0fd8228ffe server_cookie=9fa2e55a0075c94e in_seq=0 out_seq=0 2024-03-21T11:39:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.008+0000 7f45ffe00640 10 client.30832 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.013+0000 7f45ffe00640 1 -- 172.21.15.47:0/1688308547 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55faf4ad6d70 con 0x55faf4b8aef0 2024-03-21T11:39:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.013+0000 7f45ffe00640 10 client.30832 handle_command_reply: tid=0 2024-03-21T11:39:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.013+0000 7f4628411740 1 client.30832 shutdown 2024-03-21T11:39:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.013+0000 7f4628411740 1 -- 172.21.15.47:0/1688308547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 msgr2=0x55faf4b758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.013+0000 7f4628411740 1 --2- 172.21.15.47:0/1688308547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b758a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f461801a000 tx=0x7f46180118d0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.014+0000 7f4628411740 1 -- 172.21.15.47:0/1688308547 shutdown_connections 2024-03-21T11:39:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.014+0000 7f4628411740 1 --2- 172.21.15.47:0/1688308547 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55faf4b8aef0 0x55faf4b8d370 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.014+0000 7f4628411740 1 --2- 172.21.15.47:0/1688308547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b758a0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.014+0000 7f4628411740 1 --2- 172.21.15.47:0/1688308547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b64bd0 0x55faf4b6e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.014+0000 7f4628411740 1 --2- 172.21.15.47:0/1688308547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b6e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.014+0000 7f4628411740 1 -- 172.21.15.47:0/1688308547 >> 172.21.15.47:0/1688308547 conn(0x55faf4b5f750 msgr2=0x55faf4b5e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.014+0000 7f45ffe00640 0 client.30832 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:34.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.014+0000 7f4628411740 1 -- 172.21.15.47:0/1688308547 shutdown_connections 2024-03-21T11:39:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.014+0000 7f4628411740 1 -- 172.21.15.47:0/1688308547 wait complete. 2024-03-21T11:39:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.015+0000 7f4628411740 20 client.30832 trim_cache size 0 max 16384 2024-03-21T11:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.036+0000 7f4628411740 1 Processor -- start 2024-03-21T11:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.036+0000 7f4628411740 1 -- start start 2024-03-21T11:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.036+0000 7f4628411740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b6d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.036+0000 7f4628411740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b64bd0 0x55faf4b75860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.036+0000 7f4628411740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b5e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.036+0000 7f4628411740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a7d90 con 0x55faf4b64bd0 2024-03-21T11:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.036+0000 7f4628411740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a97d0 con 0x55faf4b641e0 2024-03-21T11:39:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.036+0000 7f4628411740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a83f0 con 0x55faf4b69d40 2024-03-21T11:39:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.037+0000 7f461ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b6d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.037+0000 7f461ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b6d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54310/0 (socket says 172.21.15.47:54310) 2024-03-21T11:39:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.037+0000 7f461ea00640 1 -- 172.21.15.47:0/186683909 learned_addr learned my addr 172.21.15.47:0/186683909 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.037+0000 7f461ea00640 1 -- 172.21.15.47:0/186683909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 msgr2=0x55faf4b5e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.037+0000 7f461ea00640 1 --2- 172.21.15.47:0/186683909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b5e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.037+0000 7f461ea00640 1 -- 172.21.15.47:0/186683909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b64bd0 msgr2=0x55faf4b75860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.038+0000 7f461e000640 1 --2- 172.21.15.47:0/186683909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b64bd0 0x55faf4b75860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.038+0000 7f461ea00640 1 --2- 172.21.15.47:0/186683909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b64bd0 0x55faf4b75860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.038+0000 7f461ea00640 1 -- 172.21.15.47:0/186683909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faf4b645c0 con 0x55faf4b641e0 2024-03-21T11:39:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.038+0000 7f461e000640 1 --2- 172.21.15.47:0/186683909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b64bd0 0x55faf4b75860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.038+0000 7f461ea00640 1 --2- 172.21.15.47:0/186683909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b6d7d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x55faf4b65640 tx=0x7f4610004190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=193c9dbc503d6987 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.038+0000 7f460aa00640 1 -- 172.21.15.47:0/186683909 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610003b20 con 0x55faf4b641e0 2024-03-21T11:39:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.038+0000 7f460aa00640 1 -- 172.21.15.47:0/186683909 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46100044b0 con 0x55faf4b641e0 2024-03-21T11:39:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.038+0000 7f460aa00640 1 -- 172.21.15.47:0/186683909 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610004d00 con 0x55faf4b641e0 2024-03-21T11:39:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.039+0000 7f4628411740 1 -- 172.21.15.47:0/186683909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 msgr2=0x55faf4b6d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.039+0000 7f4628411740 1 --2- 172.21.15.47:0/186683909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b6d7d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x55faf4b65640 tx=0x7f4610004190 comp rx=0 tx=0).stop 2024-03-21T11:39:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.040+0000 7f4628411740 1 -- 172.21.15.47:0/186683909 shutdown_connections 2024-03-21T11:39:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.040+0000 7f4628411740 1 --2- 172.21.15.47:0/186683909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b5e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.040+0000 7f4628411740 1 --2- 172.21.15.47:0/186683909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b64bd0 0x55faf4b75860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.040+0000 7f4628411740 1 --2- 172.21.15.47:0/186683909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b641e0 0x55faf4b6d7d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.040+0000 7f4628411740 1 -- 172.21.15.47:0/186683909 >> 172.21.15.47:0/186683909 conn(0x55faf4b5f750 msgr2=0x55faf4b6d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.040+0000 7f4628411740 1 -- 172.21.15.47:0/186683909 shutdown_connections 2024-03-21T11:39:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.040+0000 7f4628411740 1 -- 172.21.15.47:0/186683909 wait complete. 2024-03-21T11:39:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.041+0000 7f4628411740 1 -- start start 2024-03-21T11:39:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.041+0000 7f4628411740 1 Processor -- start 2024-03-21T11:39:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.041+0000 7f4628411740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b641e0 0x55faf4b73170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.041+0000 7f4628411740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b64bd0 0x55faf4b736b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.041+0000 7f4628411740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b74670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.042+0000 7f461e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b64bd0 0x55faf4b736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.042+0000 7f461e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b64bd0 0x55faf4b736b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54322/0 (socket says 172.21.15.47:54322) 2024-03-21T11:39:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.042+0000 7f461ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b641e0 0x55faf4b73170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.042+0000 7f461ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b641e0 0x55faf4b73170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39284/0 (socket says 172.21.15.47:39284) 2024-03-21T11:39:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.042+0000 7f461e000640 1 -- 172.21.15.47:0/3023926201 learned_addr learned my addr 172.21.15.47:0/3023926201 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.042+0000 7f4628411740 1 -- 172.21.15.47:0/3023926201 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a90b0 con 0x55faf4b641e0 2024-03-21T11:39:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.042+0000 7f4628411740 1 -- 172.21.15.47:0/3023926201 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a8a50 con 0x55faf4b64bd0 2024-03-21T11:39:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.042+0000 7f4628411740 1 -- 172.21.15.47:0/3023926201 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a5f50 con 0x55faf4b69d40 2024-03-21T11:39:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.042+0000 7f4628411740 10 client.0 _ll_register_callbacks cb 0x7f4618009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.042+0000 7f461f400640 1 --2- 172.21.15.47:0/3023926201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b74670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.043+0000 7f461e000640 1 -- 172.21.15.47:0/3023926201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 msgr2=0x55faf4b74670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.043+0000 7f461e000640 1 --2- 172.21.15.47:0/3023926201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b74670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.043+0000 7f461e000640 1 -- 172.21.15.47:0/3023926201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b641e0 msgr2=0x55faf4b73170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.043+0000 7f461e000640 1 --2- 172.21.15.47:0/3023926201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b641e0 0x55faf4b73170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.043+0000 7f461e000640 1 -- 172.21.15.47:0/3023926201 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55faf4b74c60 con 0x55faf4b64bd0 2024-03-21T11:39:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.043+0000 7f461e000640 1 --2- 172.21.15.47:0/3023926201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b64bd0 0x55faf4b736b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f460c0ba130 tx=0x7f460c0bd5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=523f3e7d23436218 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.043+0000 7f45ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.043+0000 7f45ffe00640 1 -- 172.21.15.47:0/3023926201 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c04b040 con 0x55faf4b64bd0 2024-03-21T11:39:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.044+0000 7f45ffe00640 1 -- 172.21.15.47:0/3023926201 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f460c0b9dd0 con 0x55faf4b64bd0 2024-03-21T11:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.044+0000 7f4628411740 1 -- 172.21.15.47:0/3023926201 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46100044b0 con 0x55faf4b64bd0 2024-03-21T11:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.044+0000 7f45ffe00640 1 -- 172.21.15.47:0/3023926201 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f460c0bd760 con 0x55faf4b64bd0 2024-03-21T11:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.044+0000 7f4628411740 10 client.30836 fetch_fsmap learned FSMap version 38 2024-03-21T11:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.044+0000 7f4628411740 1 -- 172.21.15.47:0/3023926201 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55faf4be6a50 con 0x55faf4b64bd0 2024-03-21T11:39:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.046+0000 7f45ffe00640 1 -- 172.21.15.47:0/3023926201 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46100044b0 con 0x55faf4b64bd0 2024-03-21T11:39:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.047+0000 7f45ffe00640 1 -- 172.21.15.47:0/3023926201 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 38) v1 ==== 7958+0+0 (secure 0 0 0) 0x7f460c0c5cc0 con 0x55faf4b64bd0 2024-03-21T11:39:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.047+0000 7f4628411740 10 client.30836 fetch_fsmap finished waiting for FSMap version 38 2024-03-21T11:39:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.047+0000 7f4628411740 10 client.30836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.047+0000 7f4628411740 20 client.30836 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.047+0000 7f4628411740 1 --2- 172.21.15.47:0/3023926201 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55faf4be7360 0x55faf4be9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.047+0000 7f4628411740 4 client.30836 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:39:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.047+0000 7f4628411740 1 -- 172.21.15.47:0/3023926201 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55faf4ad10c0 con 0x55faf4be7360 2024-03-21T11:39:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.048+0000 7f461ea00640 1 --2- 172.21.15.47:0/3023926201 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55faf4be7360 0x55faf4be9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.049+0000 7f45ffe00640 10 client.30836 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.049+0000 7f461ea00640 1 --2- 172.21.15.47:0/3023926201 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55faf4be7360 0x55faf4be9780 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79a2ca1ff67e9243 server_cookie=41a4fcbbe7064aa1 in_seq=0 out_seq=0 2024-03-21T11:39:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.788+0000 7f45ffe00640 1 -- 172.21.15.47:0/3023926201 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55faf4ad10c0 con 0x55faf4be7360 2024-03-21T11:39:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.788+0000 7f45ffe00640 10 client.30836 handle_command_reply: tid=0 2024-03-21T11:39:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.788+0000 7f4628411740 1 client.30836 shutdown 2024-03-21T11:39:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.788+0000 7f4628411740 1 -- 172.21.15.47:0/3023926201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b64bd0 msgr2=0x55faf4b736b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.788+0000 7f4628411740 1 --2- 172.21.15.47:0/3023926201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b64bd0 0x55faf4b736b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f460c0ba130 tx=0x7f460c0bd5f0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.791+0000 7f4628411740 1 -- 172.21.15.47:0/3023926201 shutdown_connections 2024-03-21T11:39:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.791+0000 7f4628411740 1 --2- 172.21.15.47:0/3023926201 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55faf4be7360 0x55faf4be9780 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.791+0000 7f4628411740 1 --2- 172.21.15.47:0/3023926201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55faf4b69d40 0x55faf4b74670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.791+0000 7f4628411740 1 --2- 172.21.15.47:0/3023926201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55faf4b64bd0 0x55faf4b736b0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.791+0000 7f4628411740 1 --2- 172.21.15.47:0/3023926201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55faf4b641e0 0x55faf4b73170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.791+0000 7f4628411740 1 -- 172.21.15.47:0/3023926201 >> 172.21.15.47:0/3023926201 conn(0x55faf4b5f750 msgr2=0x55faf4b5e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.791+0000 7f45ffe00640 0 client.30836 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.791+0000 7f4628411740 1 -- 172.21.15.47:0/3023926201 shutdown_connections 2024-03-21T11:39:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.791+0000 7f4628411740 1 -- 172.21.15.47:0/3023926201 wait complete. 2024-03-21T11:39:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.791+0000 7f4628411740 20 client.30836 trim_cache size 0 max 16384 2024-03-21T11:39:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:39:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:39:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:39:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:39:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.792+0000 7f4625600640 1 -- 172.21.15.47:0/1548791997 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46040a3040 msgr2=0x7f46040a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.792+0000 7f4625600640 1 --2- 172.21.15.47:0/1548791997 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46040a3040 0x7f46040a5500 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4618009d60 tx=0x7f4618004220 comp rx=0 tx=0).stop 2024-03-21T11:39:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.792+0000 7f4625600640 1 -- 172.21.15.47:0/1548791997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4620104af0 msgr2=0x7f462019a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.792+0000 7f4625600640 1 --2- 172.21.15.47:0/1548791997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4620104af0 0x7f462019a070 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f460c02c8f0 tx=0x7f460c005d50 comp rx=0 tx=0).stop 2024-03-21T11:39:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.792+0000 7f4625600640 1 -- 172.21.15.47:0/1548791997 shutdown_connections 2024-03-21T11:39:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.792+0000 7f4625600640 1 --2- 172.21.15.47:0/1548791997 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46040a3040 0x7f46040a5500 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.792+0000 7f4625600640 1 --2- 172.21.15.47:0/1548791997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46201054b0 0x7f46201a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.792+0000 7f4625600640 1 --2- 172.21.15.47:0/1548791997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4620104af0 0x7f462019a070 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.792+0000 7f4625600640 1 --2- 172.21.15.47:0/1548791997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4620103900 0x7f4620199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.792+0000 7f4625600640 1 -- 172.21.15.47:0/1548791997 >> 172.21.15.47:0/1548791997 conn(0x7f46200fee90 msgr2=0x7f4620107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.792+0000 7f4625600640 1 -- 172.21.15.47:0/1548791997 shutdown_connections 2024-03-21T11:39:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:34.793+0000 7f4625600640 1 -- 172.21.15.47:0/1548791997 wait complete. 2024-03-21T11:39:34.931 INFO:tasks.workunit.client.0.smithi047.stdout: 10 67298 26.91 MB/sec execute 18 sec latency 586.430 ms 2024-03-21T11:39:34.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:34 smithi102 ceph-mon[36748]: pgmap v887: 129 pgs: 129 active+clean; 365 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.8 MiB/s wr, 603 op/s 2024-03-21T11:39:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:34 smithi047 ceph-mon[31351]: pgmap v887: 129 pgs: 129 active+clean; 365 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.8 MiB/s wr, 603 op/s 2024-03-21T11:39:35.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:34 smithi100 ceph-mon[36245]: pgmap v887: 129 pgs: 129 active+clean; 365 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.8 MiB/s wr, 603 op/s 2024-03-21T11:39:35.932 INFO:tasks.workunit.client.0.smithi047.stdout: 10 67732 26.40 MB/sec execute 19 sec latency 95.085 ms 2024-03-21T11:39:36.932 INFO:tasks.workunit.client.0.smithi047.stdout: 10 67746 25.09 MB/sec execute 20 sec latency 1013.535 ms 2024-03-21T11:39:36.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:36 smithi102 ceph-mon[36748]: pgmap v888: 129 pgs: 129 active+clean; 365 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.5 MiB/s wr, 516 op/s 2024-03-21T11:39:37.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:36 smithi047 ceph-mon[31351]: pgmap v888: 129 pgs: 129 active+clean; 365 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.5 MiB/s wr, 516 op/s 2024-03-21T11:39:37.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:36 smithi100 ceph-mon[36245]: pgmap v888: 129 pgs: 129 active+clean; 365 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.5 MiB/s wr, 516 op/s 2024-03-21T11:39:37.932 INFO:tasks.workunit.client.0.smithi047.stdout: 10 67860 23.99 MB/sec execute 21 sec latency 1188.540 ms 2024-03-21T11:39:38.932 INFO:tasks.workunit.client.0.smithi047.stdout: 10 68172 23.69 MB/sec execute 22 sec latency 143.500 ms 2024-03-21T11:39:38.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:38 smithi102 ceph-mon[36748]: pgmap v889: 129 pgs: 129 active+clean; 376 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.4 MiB/s wr, 613 op/s 2024-03-21T11:39:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:38 smithi047 ceph-mon[31351]: pgmap v889: 129 pgs: 129 active+clean; 376 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.4 MiB/s wr, 613 op/s 2024-03-21T11:39:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:38 smithi100 ceph-mon[36245]: pgmap v889: 129 pgs: 129 active+clean; 376 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.4 MiB/s wr, 613 op/s 2024-03-21T11:39:39.932 INFO:tasks.workunit.client.0.smithi047.stdout: 10 68532 23.48 MB/sec execute 23 sec latency 133.688 ms 2024-03-21T11:39:40.933 INFO:tasks.workunit.client.0.smithi047.stdout: 10 69018 23.57 MB/sec execute 24 sec latency 136.836 ms 2024-03-21T11:39:40.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:40 smithi102 ceph-mon[36748]: pgmap v890: 129 pgs: 129 active+clean; 414 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 735 op/s 2024-03-21T11:39:41.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:40 smithi047 ceph-mon[31351]: pgmap v890: 129 pgs: 129 active+clean; 414 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 735 op/s 2024-03-21T11:39:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:40 smithi100 ceph-mon[36245]: pgmap v890: 129 pgs: 129 active+clean; 414 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 735 op/s 2024-03-21T11:39:41.933 INFO:tasks.workunit.client.0.smithi047.stdout: 10 69656 24.22 MB/sec execute 25 sec latency 65.196 ms 2024-03-21T11:39:42.933 INFO:tasks.workunit.client.0.smithi047.stdout: 10 70127 24.54 MB/sec execute 26 sec latency 113.779 ms 2024-03-21T11:39:42.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:42 smithi102 ceph-mon[36748]: pgmap v891: 129 pgs: 129 active+clean; 411 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 732 op/s 2024-03-21T11:39:43.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:42 smithi047 ceph-mon[31351]: pgmap v891: 129 pgs: 129 active+clean; 411 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 732 op/s 2024-03-21T11:39:43.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:42 smithi100 ceph-mon[36245]: pgmap v891: 129 pgs: 129 active+clean; 411 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 732 op/s 2024-03-21T11:39:43.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:39:43.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:39:43.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.786+0000 7f8936600640 1 -- 172.21.15.47:0/2027907887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8930077c40 msgr2=0x7f89300784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:43.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.786+0000 7f8936600640 1 --2- 172.21.15.47:0/2027907887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8930077c40 0x7f89300784e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f8928009be0 tx=0x7f892802bf10 comp rx=0 tx=0).stop 2024-03-21T11:39:43.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.786+0000 7f8936600640 1 -- 172.21.15.47:0/2027907887 shutdown_connections 2024-03-21T11:39:43.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.786+0000 7f8936600640 1 --2- 172.21.15.47:0/2027907887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8930077c40 0x7f89300784e0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.786+0000 7f8936600640 1 --2- 172.21.15.47:0/2027907887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89300772a0 0x7f8930077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.786+0000 7f8936600640 1 --2- 172.21.15.47:0/2027907887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8930076060 0x7f8930076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.786+0000 7f8936600640 1 -- 172.21.15.47:0/2027907887 >> 172.21.15.47:0/2027907887 conn(0x7f89300fee50 msgr2=0x7f8930101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:43.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.787+0000 7f8936600640 1 -- 172.21.15.47:0/2027907887 shutdown_connections 2024-03-21T11:39:43.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.787+0000 7f8936600640 1 -- 172.21.15.47:0/2027907887 wait complete. 2024-03-21T11:39:43.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.788+0000 7f8936600640 1 Processor -- start 2024-03-21T11:39:43.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.788+0000 7f8936600640 1 -- start start 2024-03-21T11:39:43.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.789+0000 7f8936600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8930076060 0x7f893019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.789+0000 7f8936600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89300772a0 0x7f893019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.789+0000 7f8936600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8930077c40 0x7f89301a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.789+0000 7f8936600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8930113a30 con 0x7f8930077c40 2024-03-21T11:39:43.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.789+0000 7f8936600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89301138b0 con 0x7f89300772a0 2024-03-21T11:39:43.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.789+0000 7f8936600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8930113bb0 con 0x7f8930076060 2024-03-21T11:39:43.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.789+0000 7f892ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89300772a0 0x7f893019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.789+0000 7f892ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89300772a0 0x7f893019e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60272/0 (socket says 172.21.15.47:60272) 2024-03-21T11:39:43.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.789+0000 7f892ea00640 1 -- 172.21.15.47:0/2301389655 learned_addr learned my addr 172.21.15.47:0/2301389655 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:43.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.789+0000 7f892fe00640 1 --2- 172.21.15.47:0/2301389655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8930077c40 0x7f89301a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.790+0000 7f892ea00640 1 -- 172.21.15.47:0/2301389655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8930076060 msgr2=0x7f893019e010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:39:43.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.790+0000 7f892ea00640 1 --2- 172.21.15.47:0/2301389655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8930076060 0x7f893019e010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.790+0000 7f892ea00640 1 -- 172.21.15.47:0/2301389655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8930077c40 msgr2=0x7f89301a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:43.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.790+0000 7f892ea00640 1 --2- 172.21.15.47:0/2301389655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8930077c40 0x7f89301a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.790+0000 7f892ea00640 1 -- 172.21.15.47:0/2301389655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89301a5c30 con 0x7f89300772a0 2024-03-21T11:39:43.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.790+0000 7f892ea00640 1 --2- 172.21.15.47:0/2301389655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89300772a0 0x7f893019e550 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f891c00e9f0 tx=0x7f891c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:43.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.791+0000 7f8927e00640 1 -- 172.21.15.47:0/2301389655 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c00ccc0 con 0x7f89300772a0 2024-03-21T11:39:43.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.791+0000 7f8927e00640 1 -- 172.21.15.47:0/2301389655 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f891c005500 con 0x7f89300772a0 2024-03-21T11:39:43.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.791+0000 7f8936600640 1 -- 172.21.15.47:0/2301389655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89301a5f20 con 0x7f89300772a0 2024-03-21T11:39:43.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.791+0000 7f8927e00640 1 -- 172.21.15.47:0/2301389655 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c010690 con 0x7f89300772a0 2024-03-21T11:39:43.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.791+0000 7f8936600640 1 -- 172.21.15.47:0/2301389655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89301a6410 con 0x7f89300772a0 2024-03-21T11:39:43.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.795+0000 7f8939389740 1 Processor -- start 2024-03-21T11:39:43.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.796+0000 7f8939389740 1 -- start start 2024-03-21T11:39:43.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.796+0000 7f8939389740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d1e90 0x56239c6d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.796+0000 7f8939389740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d2880 0x56239c6d7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.796+0000 7f892fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d1e90 0x56239c6d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.796+0000 7f892fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d1e90 0x56239c6d22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48094/0 (socket says 172.21.15.47:48094) 2024-03-21T11:39:43.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.796+0000 7f892fe00640 1 -- 172.21.15.47:0/3130811437 learned_addr learned my addr 172.21.15.47:0/3130811437 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:43.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.796+0000 7f8939389740 1 --2- 172.21.15.47:0/3130811437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d79f0 0x56239c6d9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.796+0000 7f8939389740 1 -- 172.21.15.47:0/3130811437 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56239c626250 con 0x56239c6d2880 2024-03-21T11:39:43.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.797+0000 7f8939389740 1 -- 172.21.15.47:0/3130811437 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56239c5cea30 con 0x56239c6d79f0 2024-03-21T11:39:43.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.797+0000 7f8939389740 1 -- 172.21.15.47:0/3130811437 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56239c6222f0 con 0x56239c6d1e90 2024-03-21T11:39:43.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.797+0000 7f892f400640 1 --2- 172.21.15.47:0/3130811437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d2880 0x56239c6d7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.797+0000 7f892fe00640 1 -- 172.21.15.47:0/3130811437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d79f0 msgr2=0x56239c6d9e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:39:43.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.797+0000 7f892fe00640 1 --2- 172.21.15.47:0/3130811437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d79f0 0x56239c6d9e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.797+0000 7f892fe00640 1 -- 172.21.15.47:0/3130811437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d2880 msgr2=0x56239c6d7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:43.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.797+0000 7f892fe00640 1 --2- 172.21.15.47:0/3130811437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d2880 0x56239c6d7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.797+0000 7f892fe00640 1 -- 172.21.15.47:0/3130811437 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56239c5b9690 con 0x56239c6d1e90 2024-03-21T11:39:43.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.797+0000 7f892f400640 1 --2- 172.21.15.47:0/3130811437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d2880 0x56239c6d7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:43.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.797+0000 7f8927e00640 1 -- 172.21.15.47:0/2301389655 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f891c0046c0 con 0x7f89300772a0 2024-03-21T11:39:43.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.797+0000 7f892fe00640 1 --2- 172.21.15.47:0/3130811437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d1e90 0x56239c6d22b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f892802c520 tx=0x7f8928009320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f054261507d389b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:43.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.798+0000 7f8924c00640 1 -- 172.21.15.47:0/3130811437 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8928043030 con 0x56239c6d1e90 2024-03-21T11:39:43.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.798+0000 7f8924c00640 1 -- 172.21.15.47:0/3130811437 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8928033da0 con 0x56239c6d1e90 2024-03-21T11:39:43.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.798+0000 7f8924c00640 1 -- 172.21.15.47:0/3130811437 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892802cbb0 con 0x56239c6d1e90 2024-03-21T11:39:43.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.799+0000 7f8939389740 1 -- 172.21.15.47:0/3130811437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d1e90 msgr2=0x56239c6d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:43.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.799+0000 7f8939389740 1 --2- 172.21.15.47:0/3130811437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d1e90 0x56239c6d22b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f892802c520 tx=0x7f8928009320 comp rx=0 tx=0).stop 2024-03-21T11:39:43.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.799+0000 7f8927e00640 1 --2- 172.21.15.47:0/2301389655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f89140a2f20 0x7f89140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.799+0000 7f8927e00640 1 -- 172.21.15.47:0/2301389655 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f891c0cfaf0 con 0x7f89300772a0 2024-03-21T11:39:43.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.800+0000 7f8939389740 1 -- 172.21.15.47:0/3130811437 shutdown_connections 2024-03-21T11:39:43.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.800+0000 7f8939389740 1 --2- 172.21.15.47:0/3130811437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d79f0 0x56239c6d9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.800+0000 7f8939389740 1 --2- 172.21.15.47:0/3130811437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d2880 0x56239c6d7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.800+0000 7f8939389740 1 --2- 172.21.15.47:0/3130811437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d1e90 0x56239c6d22b0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.800+0000 7f8939389740 1 -- 172.21.15.47:0/3130811437 >> 172.21.15.47:0/3130811437 conn(0x56239c689200 msgr2=0x56239c6bc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:43.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.800+0000 7f892fe00640 1 --2- 172.21.15.47:0/2301389655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f89140a2f20 0x7f89140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.801+0000 7f892fe00640 1 --2- 172.21.15.47:0/2301389655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f89140a2f20 0x7f89140a53e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8928009b30 tx=0x7f8928009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:43.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.801+0000 7f8939389740 1 -- 172.21.15.47:0/3130811437 shutdown_connections 2024-03-21T11:39:43.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.801+0000 7f8939389740 1 -- 172.21.15.47:0/3130811437 wait complete. 2024-03-21T11:39:43.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.802+0000 7f8939389740 1 -- start start 2024-03-21T11:39:43.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.802+0000 7f8939389740 1 Processor -- start 2024-03-21T11:39:43.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.803+0000 7f8939389740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d1e90 0x56239c6dbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.803+0000 7f8939389740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d2880 0x56239c6dc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.803+0000 7f8939389740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d79f0 0x56239c6e3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.803+0000 7f8939389740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56239c6da940 con 0x56239c6d2880 2024-03-21T11:39:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.803+0000 7f8939389740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56239c5e1e80 con 0x56239c6d79f0 2024-03-21T11:39:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.803+0000 7f8939389740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56239c6daac0 con 0x56239c6d1e90 2024-03-21T11:39:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.803+0000 7f8939389740 10 client.0 _ll_register_callbacks cb 0x56239c5f4130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.803+0000 7f892f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d1e90 0x56239c6dbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.803+0000 7f892f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d1e90 0x56239c6dbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48108/0 (socket says 172.21.15.47:48108) 2024-03-21T11:39:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.803+0000 7f892f400640 1 -- 172.21.15.47:0/161210156 learned_addr learned my addr 172.21.15.47:0/161210156 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.804+0000 7f892ea00640 1 --2- 172.21.15.47:0/161210156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d2880 0x56239c6dc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.804+0000 7f892fe00640 1 --2- 172.21.15.47:0/161210156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d79f0 0x56239c6e3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.804+0000 7f892f400640 1 -- 172.21.15.47:0/161210156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d79f0 msgr2=0x56239c6e3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.804+0000 7f892f400640 1 --2- 172.21.15.47:0/161210156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d79f0 0x56239c6e3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.804+0000 7f892f400640 1 -- 172.21.15.47:0/161210156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d2880 msgr2=0x56239c6dc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.804+0000 7f892f400640 1 --2- 172.21.15.47:0/161210156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d2880 0x56239c6dc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.804+0000 7f892f400640 1 -- 172.21.15.47:0/161210156 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56239c4047d0 con 0x56239c6d1e90 2024-03-21T11:39:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.804+0000 7f892fe00640 1 --2- 172.21.15.47:0/161210156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d79f0 0x56239c6e3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.804+0000 7f892f400640 1 --2- 172.21.15.47:0/161210156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d1e90 0x56239c6dbe10 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f892000e1d0 tx=0x7f892000e690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=475cdcf825ec177c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.804+0000 7f8919600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.805+0000 7f8919600640 1 -- 172.21.15.47:0/161210156 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8920017070 con 0x56239c6d1e90 2024-03-21T11:39:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.805+0000 7f8919600640 1 -- 172.21.15.47:0/161210156 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8920013680 con 0x56239c6d1e90 2024-03-21T11:39:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.805+0000 7f8939389740 1 -- 172.21.15.47:0/161210156 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8928033da0 con 0x56239c6d1e90 2024-03-21T11:39:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.805+0000 7f8919600640 1 -- 172.21.15.47:0/161210156 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8920021050 con 0x56239c6d1e90 2024-03-21T11:39:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.806+0000 7f8939389740 10 client.30844 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:43.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.806+0000 7f8939389740 1 -- 172.21.15.47:0/161210156 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56239c33a3f0 con 0x56239c6d1e90 2024-03-21T11:39:43.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.806+0000 7f8919600640 1 -- 172.21.15.47:0/161210156 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8928033da0 con 0x56239c6d1e90 2024-03-21T11:39:43.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.806+0000 7f8919600640 1 -- 172.21.15.47:0/161210156 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f892001c950 con 0x56239c6d1e90 2024-03-21T11:39:43.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.807+0000 7f8939389740 10 client.30844 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:43.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.807+0000 7f8939389740 10 client.30844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:43.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.807+0000 7f8939389740 20 client.30844 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:43.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.807+0000 7f8939389740 1 --2- 172.21.15.47:0/161210156 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56239c6f8d80 0x56239c6fb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.807+0000 7f8939389740 4 client.30844 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:43.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.807+0000 7f8939389740 1 -- 172.21.15.47:0/161210156 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56239c633790 con 0x56239c6f8d80 2024-03-21T11:39:43.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.807+0000 7f892ea00640 1 --2- 172.21.15.47:0/161210156 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56239c6f8d80 0x56239c6fb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.808+0000 7f892ea00640 1 --2- 172.21.15.47:0/161210156 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56239c6f8d80 0x56239c6fb200 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d8b462a0f2bd0cf server_cookie=6bb9feb2d11aef7d in_seq=0 out_seq=0 2024-03-21T11:39:43.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.808+0000 7f8919600640 10 client.30844 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.811+0000 7f8919600640 1 -- 172.21.15.47:0/161210156 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56239c633790 con 0x56239c6f8d80 2024-03-21T11:39:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.811+0000 7f8919600640 10 client.30844 handle_command_reply: tid=0 2024-03-21T11:39:43.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.812+0000 7f8939389740 1 client.30844 shutdown 2024-03-21T11:39:43.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.812+0000 7f8939389740 1 -- 172.21.15.47:0/161210156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d1e90 msgr2=0x56239c6dbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:43.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.812+0000 7f8939389740 1 --2- 172.21.15.47:0/161210156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d1e90 0x56239c6dbe10 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f892000e1d0 tx=0x7f892000e690 comp rx=0 tx=0).stop 2024-03-21T11:39:43.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.812+0000 7f8939389740 1 -- 172.21.15.47:0/161210156 shutdown_connections 2024-03-21T11:39:43.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.812+0000 7f8939389740 1 --2- 172.21.15.47:0/161210156 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56239c6f8d80 0x56239c6fb200 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.812+0000 7f8939389740 1 --2- 172.21.15.47:0/161210156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d79f0 0x56239c6e3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.812+0000 7f8939389740 1 --2- 172.21.15.47:0/161210156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d2880 0x56239c6dc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.812+0000 7f8939389740 1 --2- 172.21.15.47:0/161210156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d1e90 0x56239c6dbe10 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.812+0000 7f8919600640 0 client.30844 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:43.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.812+0000 7f8939389740 1 -- 172.21.15.47:0/161210156 >> 172.21.15.47:0/161210156 conn(0x56239c689200 msgr2=0x56239c6bba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:43.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.812+0000 7f8939389740 1 -- 172.21.15.47:0/161210156 shutdown_connections 2024-03-21T11:39:43.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.812+0000 7f8939389740 1 -- 172.21.15.47:0/161210156 wait complete. 2024-03-21T11:39:43.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.813+0000 7f8939389740 20 client.30844 trim_cache size 0 max 16384 2024-03-21T11:39:43.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.844+0000 7f8939389740 1 Processor -- start 2024-03-21T11:39:43.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.844+0000 7f8939389740 1 -- start start 2024-03-21T11:39:43.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.844+0000 7f8939389740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d1e90 0x56239c6c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.844+0000 7f8939389740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d2880 0x56239c6e3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.844+0000 7f8939389740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d79f0 0x56239c6ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.844+0000 7f8939389740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893006c900 con 0x56239c6d79f0 2024-03-21T11:39:43.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.844+0000 7f8939389740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893006e310 con 0x56239c6d1e90 2024-03-21T11:39:43.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.844+0000 7f8939389740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893006cf60 con 0x56239c6d2880 2024-03-21T11:39:43.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.845+0000 7f892ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d2880 0x56239c6e3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.845+0000 7f892f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d1e90 0x56239c6c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.845+0000 7f892f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d1e90 0x56239c6c0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60296/0 (socket says 172.21.15.47:60296) 2024-03-21T11:39:43.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.845+0000 7f892f400640 1 -- 172.21.15.47:0/760812499 learned_addr learned my addr 172.21.15.47:0/760812499 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:43.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.845+0000 7f892fe00640 1 --2- 172.21.15.47:0/760812499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d79f0 0x56239c6ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.846+0000 7f892f400640 1 -- 172.21.15.47:0/760812499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d2880 msgr2=0x56239c6e3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:43.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.846+0000 7f892f400640 1 --2- 172.21.15.47:0/760812499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d2880 0x56239c6e3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.846+0000 7f892f400640 1 -- 172.21.15.47:0/760812499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d79f0 msgr2=0x56239c6ffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:43.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.847+0000 7f892f400640 1 --2- 172.21.15.47:0/760812499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d79f0 0x56239c6ffb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.847+0000 7f892f400640 1 -- 172.21.15.47:0/760812499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56239c2fa0f0 con 0x56239c6d1e90 2024-03-21T11:39:43.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.847+0000 7f892fe00640 1 --2- 172.21.15.47:0/760812499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d79f0 0x56239c6ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:39:43.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.849+0000 7f892f400640 1 --2- 172.21.15.47:0/760812499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d1e90 0x56239c6c0140 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8920009920 tx=0x7f8920004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9fc2032a75ee83e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:43.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.850+0000 7f891be00640 1 -- 172.21.15.47:0/760812499 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8920017070 con 0x56239c6d1e90 2024-03-21T11:39:43.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.850+0000 7f891be00640 1 -- 172.21.15.47:0/760812499 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f892001c920 con 0x56239c6d1e90 2024-03-21T11:39:43.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.850+0000 7f891be00640 1 -- 172.21.15.47:0/760812499 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8920013b50 con 0x56239c6d1e90 2024-03-21T11:39:43.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.851+0000 7f8939389740 1 -- 172.21.15.47:0/760812499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d1e90 msgr2=0x56239c6c0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:43.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.851+0000 7f8939389740 1 --2- 172.21.15.47:0/760812499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d1e90 0x56239c6c0140 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8920009920 tx=0x7f8920004770 comp rx=0 tx=0).stop 2024-03-21T11:39:43.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.852+0000 7f8939389740 1 -- 172.21.15.47:0/760812499 shutdown_connections 2024-03-21T11:39:43.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.852+0000 7f8939389740 1 --2- 172.21.15.47:0/760812499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d79f0 0x56239c6ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.852+0000 7f8939389740 1 --2- 172.21.15.47:0/760812499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d2880 0x56239c6e3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.852+0000 7f8939389740 1 --2- 172.21.15.47:0/760812499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d1e90 0x56239c6c0140 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.852+0000 7f8939389740 1 -- 172.21.15.47:0/760812499 >> 172.21.15.47:0/760812499 conn(0x56239c689200 msgr2=0x56239c6bfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:43.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.854+0000 7f8939389740 1 -- 172.21.15.47:0/760812499 shutdown_connections 2024-03-21T11:39:43.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.854+0000 7f8939389740 1 -- 172.21.15.47:0/760812499 wait complete. 2024-03-21T11:39:43.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.856+0000 7f8939389740 1 -- start start 2024-03-21T11:39:43.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.856+0000 7f8939389740 1 Processor -- start 2024-03-21T11:39:43.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.857+0000 7f8939389740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d1e90 0x56239c6e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.857+0000 7f8939389740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d2880 0x56239c6e1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.857+0000 7f8939389740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d79f0 0x56239c6e26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.857+0000 7f8939389740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893006dc20 con 0x56239c6d1e90 2024-03-21T11:39:43.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.857+0000 7f8939389740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893006d5c0 con 0x56239c6d2880 2024-03-21T11:39:43.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.857+0000 7f8939389740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893010b480 con 0x56239c6d79f0 2024-03-21T11:39:43.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.857+0000 7f8939389740 10 client.0 _ll_register_callbacks cb 0x7f89200092d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:43.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.857+0000 7f892ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d2880 0x56239c6e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.858+0000 7f892ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d2880 0x56239c6e1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60304/0 (socket says 172.21.15.47:60304) 2024-03-21T11:39:43.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.858+0000 7f892ea00640 1 -- 172.21.15.47:0/2464594278 learned_addr learned my addr 172.21.15.47:0/2464594278 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:43.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.858+0000 7f892fe00640 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d79f0 0x56239c6e26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.858+0000 7f892f400640 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d1e90 0x56239c6e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.858+0000 7f892ea00640 1 -- 172.21.15.47:0/2464594278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d79f0 msgr2=0x56239c6e26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.858+0000 7f892ea00640 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d79f0 0x56239c6e26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.859+0000 7f892ea00640 1 -- 172.21.15.47:0/2464594278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d1e90 msgr2=0x56239c6e11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.859+0000 7f892ea00640 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d1e90 0x56239c6e11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.859+0000 7f892ea00640 1 -- 172.21.15.47:0/2464594278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56239c6e2cb0 con 0x56239c6d2880 2024-03-21T11:39:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.859+0000 7f892f400640 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d1e90 0x56239c6e11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.859+0000 7f892fe00640 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d79f0 0x56239c6e26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.859+0000 7f892ea00640 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d2880 0x56239c6e1720 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f891c00c6a0 tx=0x7f891c009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=115fdd2f93e591b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.859+0000 7f8919600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.860+0000 7f8919600640 1 -- 172.21.15.47:0/2464594278 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c0a4840 con 0x56239c6d2880 2024-03-21T11:39:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.860+0000 7f8919600640 1 -- 172.21.15.47:0/2464594278 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f891c09a130 con 0x56239c6d2880 2024-03-21T11:39:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.860+0000 7f8939389740 1 -- 172.21.15.47:0/2464594278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f892001c920 con 0x56239c6d2880 2024-03-21T11:39:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.860+0000 7f8919600640 1 -- 172.21.15.47:0/2464594278 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f891c0984a0 con 0x56239c6d2880 2024-03-21T11:39:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.860+0000 7f8939389740 10 client.30848 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.860+0000 7f8939389740 1 -- 172.21.15.47:0/2464594278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56239c754e70 con 0x56239c6d2880 2024-03-21T11:39:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.860+0000 7f8919600640 1 -- 172.21.15.47:0/2464594278 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f892001c920 con 0x56239c6d2880 2024-03-21T11:39:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.861+0000 7f8919600640 1 -- 172.21.15.47:0/2464594278 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f891c09b280 con 0x56239c6d2880 2024-03-21T11:39:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.861+0000 7f8939389740 10 client.30848 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.861+0000 7f8939389740 10 client.30848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.861+0000 7f8939389740 20 client.30848 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:43.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.861+0000 7f8939389740 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56239c755810 0x56239c757c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.861+0000 7f8939389740 4 client.30848 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:39:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.861+0000 7f8939389740 1 -- 172.21.15.47:0/2464594278 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56239c62dae0 con 0x56239c755810 2024-03-21T11:39:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.861+0000 7f892f400640 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56239c755810 0x56239c757c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:43.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.862+0000 7f892f400640 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56239c755810 0x56239c757c30 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5471378458243704 server_cookie=bcef6e3f22a32bd9 in_seq=0 out_seq=0 2024-03-21T11:39:43.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.862+0000 7f8919600640 10 client.30848 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:43.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.864+0000 7f8919600640 1 -- 172.21.15.47:0/2464594278 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56239c62dae0 con 0x56239c755810 2024-03-21T11:39:43.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.864+0000 7f8919600640 10 client.30848 handle_command_reply: tid=0 2024-03-21T11:39:43.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.864+0000 7f8939389740 1 client.30848 shutdown 2024-03-21T11:39:43.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.865+0000 7f8939389740 1 -- 172.21.15.47:0/2464594278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d2880 msgr2=0x56239c6e1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:43.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.865+0000 7f8939389740 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d2880 0x56239c6e1720 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f891c00c6a0 tx=0x7f891c009f50 comp rx=0 tx=0).stop 2024-03-21T11:39:43.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.865+0000 7f8939389740 1 -- 172.21.15.47:0/2464594278 shutdown_connections 2024-03-21T11:39:43.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.865+0000 7f8939389740 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56239c755810 0x56239c757c30 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.866+0000 7f8939389740 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56239c6d79f0 0x56239c6e26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.866+0000 7f8919600640 0 client.30848 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:43.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.866+0000 7f8939389740 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56239c6d2880 0x56239c6e1720 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.866+0000 7f8939389740 1 --2- 172.21.15.47:0/2464594278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56239c6d1e90 0x56239c6e11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.866+0000 7f8939389740 1 -- 172.21.15.47:0/2464594278 >> 172.21.15.47:0/2464594278 conn(0x56239c689200 msgr2=0x56239c6c02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:43.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.866+0000 7f8939389740 1 -- 172.21.15.47:0/2464594278 shutdown_connections 2024-03-21T11:39:43.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.866+0000 7f8939389740 1 -- 172.21.15.47:0/2464594278 wait complete. 2024-03-21T11:39:43.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.866+0000 7f8939389740 20 client.30848 trim_cache size 0 max 16384 2024-03-21T11:39:43.868 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:39:43.868 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:39:43.868 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:39:43.868 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:39:43.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.868+0000 7f8936600640 1 -- 172.21.15.47:0/2301389655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f89140a2f20 msgr2=0x7f89140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:43.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.868+0000 7f8936600640 1 --2- 172.21.15.47:0/2301389655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f89140a2f20 0x7f89140a53e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8928009b30 tx=0x7f8928009500 comp rx=0 tx=0).stop 2024-03-21T11:39:43.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.868+0000 7f8936600640 1 -- 172.21.15.47:0/2301389655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89300772a0 msgr2=0x7f893019e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:43.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.868+0000 7f8936600640 1 --2- 172.21.15.47:0/2301389655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89300772a0 0x7f893019e550 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f891c00e9f0 tx=0x7f891c00eeb0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.868+0000 7f8936600640 1 -- 172.21.15.47:0/2301389655 shutdown_connections 2024-03-21T11:39:43.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.868+0000 7f8936600640 1 --2- 172.21.15.47:0/2301389655 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f89140a2f20 0x7f89140a53e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.868+0000 7f8936600640 1 --2- 172.21.15.47:0/2301389655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8930077c40 0x7f89301a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.868+0000 7f8936600640 1 --2- 172.21.15.47:0/2301389655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89300772a0 0x7f893019e550 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.868+0000 7f8936600640 1 --2- 172.21.15.47:0/2301389655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8930076060 0x7f893019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:43.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.868+0000 7f8936600640 1 -- 172.21.15.47:0/2301389655 >> 172.21.15.47:0/2301389655 conn(0x7f89300fee50 msgr2=0x7f89301004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:43.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.869+0000 7f8936600640 1 -- 172.21.15.47:0/2301389655 shutdown_connections 2024-03-21T11:39:43.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:43.869+0000 7f8936600640 1 -- 172.21.15.47:0/2301389655 wait complete. 2024-03-21T11:39:43.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:43 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:43.895 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:39:43.896 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:39:43.933 INFO:tasks.workunit.client.0.smithi047.stdout: 10 70532 24.30 MB/sec execute 27 sec latency 84.961 ms 2024-03-21T11:39:44.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:43 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:44.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.162+0000 7f9ffc400640 1 -- 172.21.15.47:0/2031806055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ff40772a0 msgr2=0x7f9ff4077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:44.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.162+0000 7f9ffc400640 1 --2- 172.21.15.47:0/2031806055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ff40772a0 0x7f9ff4077700 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009900 tx=0x7f9fe002bc30 comp rx=0 tx=0).stop 2024-03-21T11:39:44.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.162+0000 7f9ffc400640 1 -- 172.21.15.47:0/2031806055 shutdown_connections 2024-03-21T11:39:44.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.162+0000 7f9ffc400640 1 --2- 172.21.15.47:0/2031806055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff4077c40 0x7f9ff40784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.162+0000 7f9ffc400640 1 --2- 172.21.15.47:0/2031806055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ff40772a0 0x7f9ff4077700 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.162+0000 7f9ffc400640 1 --2- 172.21.15.47:0/2031806055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ff4076060 0x7f9ff4076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.162+0000 7f9ffc400640 1 -- 172.21.15.47:0/2031806055 >> 172.21.15.47:0/2031806055 conn(0x7f9ff40fee50 msgr2=0x7f9ff4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:44.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.162+0000 7f9ffc400640 1 -- 172.21.15.47:0/2031806055 shutdown_connections 2024-03-21T11:39:44.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.162+0000 7f9ffc400640 1 -- 172.21.15.47:0/2031806055 wait complete. 2024-03-21T11:39:44.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.164+0000 7f9ffc400640 1 Processor -- start 2024-03-21T11:39:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.164+0000 7f9ffc400640 1 -- start start 2024-03-21T11:39:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.164+0000 7f9ffc400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ff4076060 0x7f9ff4070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.164+0000 7f9ffc400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff40772a0 0x7f9ff4070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.164+0000 7f9ffc400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ff4077c40 0x7f9ff41ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.164+0000 7f9ffc400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff41139f0 con 0x7f9ff40772a0 2024-03-21T11:39:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.164+0000 7f9ffc400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff4113870 con 0x7f9ff4077c40 2024-03-21T11:39:44.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.164+0000 7f9ffc400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff4113b70 con 0x7f9ff4076060 2024-03-21T11:39:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.165+0000 7f9ffa400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ff4077c40 0x7f9ff41ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.165+0000 7f9ffa400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ff4077c40 0x7f9ff41ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60334/0 (socket says 172.21.15.47:60334) 2024-03-21T11:39:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.165+0000 7f9ffa400640 1 -- 172.21.15.47:0/2775542030 learned_addr learned my addr 172.21.15.47:0/2775542030 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.165+0000 7f9ffa400640 1 -- 172.21.15.47:0/2775542030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ff4076060 msgr2=0x7f9ff4070040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:39:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.165+0000 7f9ffa400640 1 --2- 172.21.15.47:0/2775542030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ff4076060 0x7f9ff4070040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.165+0000 7f9ffa400640 1 -- 172.21.15.47:0/2775542030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff40772a0 msgr2=0x7f9ff4070580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:39:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.165+0000 7f9ffa400640 1 --2- 172.21.15.47:0/2775542030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff40772a0 0x7f9ff4070580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.165+0000 7f9ffa400640 1 -- 172.21.15.47:0/2775542030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff4070c60 con 0x7f9ff4077c40 2024-03-21T11:39:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.166+0000 7f9ffa400640 1 --2- 172.21.15.47:0/2775542030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ff4077c40 0x7f9ff41ae100 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b970 tx=0x7f9fec00be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:44.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.167+0000 7f9ff2000640 1 -- 172.21.15.47:0/2775542030 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec019070 con 0x7f9ff4077c40 2024-03-21T11:39:44.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.167+0000 7f9ff2000640 1 -- 172.21.15.47:0/2775542030 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fec00cb40 con 0x7f9ff4077c40 2024-03-21T11:39:44.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.167+0000 7f9ff2000640 1 -- 172.21.15.47:0/2775542030 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec007650 con 0x7f9ff4077c40 2024-03-21T11:39:44.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.167+0000 7f9ffc400640 1 -- 172.21.15.47:0/2775542030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff41ae760 con 0x7f9ff4077c40 2024-03-21T11:39:44.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.167+0000 7f9ffc400640 1 -- 172.21.15.47:0/2775542030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff41aec00 con 0x7f9ff4077c40 2024-03-21T11:39:44.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.168+0000 7f9ffc400640 1 -- 172.21.15.47:0/2775542030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff4110250 con 0x7f9ff4077c40 2024-03-21T11:39:44.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.171+0000 7f9ff2000640 1 -- 172.21.15.47:0/2775542030 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9fec01c070 con 0x7f9ff4077c40 2024-03-21T11:39:44.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.174+0000 7f9ff2000640 1 --2- 172.21.15.47:0/2775542030 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9fd80a2ff0 0x7f9fd80a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.174+0000 7f9ff2000640 1 -- 172.21.15.47:0/2775542030 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9fec0cf500 con 0x7f9ff4077c40 2024-03-21T11:39:44.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.174+0000 7f9ff9a00640 1 --2- 172.21.15.47:0/2775542030 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9fd80a2ff0 0x7f9fd80a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.175+0000 7f9ff9a00640 1 --2- 172.21.15.47:0/2775542030 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9fd80a2ff0 0x7f9fd80a54b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009a60 tx=0x7f9fe4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:44.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.177+0000 7f9ff2000640 1 -- 172.21.15.47:0/2775542030 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9fec09bfa0 con 0x7f9ff4077c40 2024-03-21T11:39:44.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:43 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.515+0000 7f9ffc400640 1 -- 172.21.15.47:0/2775542030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ff402d320 con 0x7f9ff4077c40 2024-03-21T11:39:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.517+0000 7f9ff2000640 1 -- 172.21.15.47:0/2775542030 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+11477 (secure 0 0 0) 0x7f9fec09b8a0 con 0x7f9ff4077c40 2024-03-21T11:39:44.518 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:39:44.518 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:39:42.302664+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24524,24523,24535,24581,24553]},"id":1}]} 2024-03-21T11:39:44.518 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 39 2024-03-21T11:39:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.519+0000 7f9ffc400640 1 -- 172.21.15.47:0/2775542030 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9fd80a2ff0 msgr2=0x7f9fd80a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.519+0000 7f9ffc400640 1 --2- 172.21.15.47:0/2775542030 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9fd80a2ff0 0x7f9fd80a54b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009a60 tx=0x7f9fe4005f90 comp rx=0 tx=0).stop 2024-03-21T11:39:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.519+0000 7f9ffc400640 1 -- 172.21.15.47:0/2775542030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ff4077c40 msgr2=0x7f9ff41ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.519+0000 7f9ffc400640 1 --2- 172.21.15.47:0/2775542030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ff4077c40 0x7f9ff41ae100 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b970 tx=0x7f9fec00be30 comp rx=0 tx=0).stop 2024-03-21T11:39:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.520+0000 7f9ffc400640 1 -- 172.21.15.47:0/2775542030 shutdown_connections 2024-03-21T11:39:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.520+0000 7f9ffc400640 1 --2- 172.21.15.47:0/2775542030 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9fd80a2ff0 0x7f9fd80a54b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.520+0000 7f9ffc400640 1 --2- 172.21.15.47:0/2775542030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ff4077c40 0x7f9ff41ae100 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.520+0000 7f9ffc400640 1 --2- 172.21.15.47:0/2775542030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff40772a0 0x7f9ff4070580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.520+0000 7f9ffc400640 1 --2- 172.21.15.47:0/2775542030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ff4076060 0x7f9ff4070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.520+0000 7f9ffc400640 1 -- 172.21.15.47:0/2775542030 >> 172.21.15.47:0/2775542030 conn(0x7f9ff40fee50 msgr2=0x7f9ff4100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.520+0000 7f9ffc400640 1 -- 172.21.15.47:0/2775542030 shutdown_connections 2024-03-21T11:39:44.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.520+0000 7f9ffc400640 1 -- 172.21.15.47:0/2775542030 wait complete. 2024-03-21T11:39:44.540 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:39:44.540 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:39:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.817+0000 7fede6800640 1 -- 172.21.15.47:0/4182963221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fede0104ad0 msgr2=0x7fede0106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.817+0000 7fede6800640 1 --2- 172.21.15.47:0/4182963221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fede0104ad0 0x7fede0106ec0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009900 tx=0x7fedcc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:39:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.817+0000 7fede6800640 1 -- 172.21.15.47:0/4182963221 shutdown_connections 2024-03-21T11:39:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.817+0000 7fede6800640 1 --2- 172.21.15.47:0/4182963221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0107400 0x7fede01097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.817+0000 7fede6800640 1 --2- 172.21.15.47:0/4182963221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fede0104ad0 0x7fede0106ec0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.817+0000 7fede6800640 1 --2- 172.21.15.47:0/4182963221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fede01021a0 0x7fede0104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.817+0000 7fede6800640 1 -- 172.21.15.47:0/4182963221 >> 172.21.15.47:0/4182963221 conn(0x7fede00fbb50 msgr2=0x7fede00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:44.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.817+0000 7fede6800640 1 -- 172.21.15.47:0/4182963221 shutdown_connections 2024-03-21T11:39:44.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.817+0000 7fede6800640 1 -- 172.21.15.47:0/4182963221 wait complete. 2024-03-21T11:39:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.818+0000 7fede6800640 1 Processor -- start 2024-03-21T11:39:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.818+0000 7fede6800640 1 -- start start 2024-03-21T11:39:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.818+0000 7fede6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fede01021a0 0x7fede0199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.818+0000 7fede6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0104ad0 0x7fede019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.818+0000 7fede6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fede0107400 0x7fede01a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.818+0000 7fede6800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede0100460 con 0x7fede01021a0 2024-03-21T11:39:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.818+0000 7fede6800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede01002e0 con 0x7fede0107400 2024-03-21T11:39:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.818+0000 7fede6800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede01005e0 con 0x7fede0104ad0 2024-03-21T11:39:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.819+0000 7feddfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fede0107400 0x7fede01a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.819+0000 7feddfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fede0107400 0x7fede01a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60348/0 (socket says 172.21.15.47:60348) 2024-03-21T11:39:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.819+0000 7feddfe00640 1 -- 172.21.15.47:0/3692616197 learned_addr learned my addr 172.21.15.47:0/3692616197 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.819+0000 7feddf400640 1 --2- 172.21.15.47:0/3692616197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fede01021a0 0x7fede0199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.819+0000 7feddfe00640 1 -- 172.21.15.47:0/3692616197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0104ad0 msgr2=0x7fede019a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.819+0000 7feddfe00640 1 --2- 172.21.15.47:0/3692616197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0104ad0 0x7fede019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.819+0000 7feddfe00640 1 -- 172.21.15.47:0/3692616197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fede01021a0 msgr2=0x7fede0199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.819+0000 7feddfe00640 1 --2- 172.21.15.47:0/3692616197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fede01021a0 0x7fede0199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.819+0000 7feddfe00640 1 -- 172.21.15.47:0/3692616197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede01a1870 con 0x7fede0107400 2024-03-21T11:39:44.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.820+0000 7feddfe00640 1 --2- 172.21.15.47:0/3692616197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fede0107400 0x7fede01a11d0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fedd800efd0 tx=0x7fedd800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:44.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.820+0000 7fedd7e00640 1 -- 172.21.15.47:0/3692616197 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8003b70 con 0x7fede0107400 2024-03-21T11:39:44.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.820+0000 7fede6800640 1 -- 172.21.15.47:0/3692616197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede01a1b60 con 0x7fede0107400 2024-03-21T11:39:44.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.820+0000 7fede6800640 1 -- 172.21.15.47:0/3692616197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede01a20b0 con 0x7fede0107400 2024-03-21T11:39:44.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.820+0000 7fedd7e00640 1 -- 172.21.15.47:0/3692616197 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedd8003d10 con 0x7fede0107400 2024-03-21T11:39:44.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.820+0000 7fedd7e00640 1 -- 172.21.15.47:0/3692616197 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8010a30 con 0x7fede0107400 2024-03-21T11:39:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.824+0000 7fedd7e00640 1 -- 172.21.15.47:0/3692616197 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fedd8003fe0 con 0x7fede0107400 2024-03-21T11:39:44.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.825+0000 7fede95e7740 1 Processor -- start 2024-03-21T11:39:44.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.825+0000 7fede95e7740 1 -- start start 2024-03-21T11:39:44.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.826+0000 7fede95e7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c37810 0x564d05c37c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.826+0000 7fede95e7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c389e0 0x564d05c38e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.826+0000 7fede95e7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c393a0 0x564d05c3fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.826+0000 7fede95e7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d05b9c2c0 con 0x564d05c389e0 2024-03-21T11:39:44.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.826+0000 7fede95e7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d05b44c90 con 0x564d05c393a0 2024-03-21T11:39:44.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.826+0000 7fede95e7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d05b98360 con 0x564d05c37810 2024-03-21T11:39:44.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.826+0000 7feddea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c37810 0x564d05c37c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.826+0000 7feddea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c37810 0x564d05c37c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48158/0 (socket says 172.21.15.47:48158) 2024-03-21T11:39:44.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.826+0000 7feddea00640 1 -- 172.21.15.47:0/3671011911 learned_addr learned my addr 172.21.15.47:0/3671011911 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:44.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.826+0000 7feddea00640 1 --2- 172.21.15.47:0/3671011911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c393a0 0x564d05c3fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.826+0000 7feddf400640 1 --2- 172.21.15.47:0/3671011911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c389e0 0x564d05c38e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.827+0000 7fedd7e00640 1 --2- 172.21.15.47:0/3692616197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fedc40a2ff0 0x7fedc40a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.827+0000 7fedd7e00640 1 -- 172.21.15.47:0/3692616197 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fedd80cfee0 con 0x7fede0107400 2024-03-21T11:39:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.827+0000 7feddf400640 1 -- 172.21.15.47:0/3671011911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c37810 msgr2=0x564d05c37c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.827+0000 7feddf400640 1 --2- 172.21.15.47:0/3671011911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c37810 0x564d05c37c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.827+0000 7feddf400640 1 -- 172.21.15.47:0/3671011911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c393a0 msgr2=0x564d05c3fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.827+0000 7feddf400640 1 --2- 172.21.15.47:0/3671011911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c393a0 0x564d05c3fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.827+0000 7feddf400640 1 -- 172.21.15.47:0/3671011911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d05b69080 con 0x564d05c389e0 2024-03-21T11:39:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.827+0000 7feddea00640 1 --2- 172.21.15.47:0/3671011911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c37810 0x564d05c37c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.827+0000 7feddfe00640 1 --2- 172.21.15.47:0/3692616197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fedc40a2ff0 0x7fedc40a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.828+0000 7feddf400640 1 --2- 172.21.15.47:0/3671011911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c389e0 0x564d05c38e60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fedd000bbc0 tx=0x7fedd00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fe7c6f4d3f15f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.828+0000 7fedd4c00640 1 -- 172.21.15.47:0/3671011911 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd0004030 con 0x564d05c389e0 2024-03-21T11:39:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.828+0000 7fedd4c00640 1 -- 172.21.15.47:0/3671011911 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedd00041d0 con 0x564d05c389e0 2024-03-21T11:39:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.828+0000 7fedd4c00640 1 -- 172.21.15.47:0/3671011911 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd00116a0 con 0x564d05c389e0 2024-03-21T11:39:44.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.829+0000 7fede95e7740 1 -- 172.21.15.47:0/3671011911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c389e0 msgr2=0x564d05c38e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:44.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.829+0000 7fede95e7740 1 --2- 172.21.15.47:0/3671011911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c389e0 0x564d05c38e60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fedd000bbc0 tx=0x7fedd00107a0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.829+0000 7feddfe00640 1 --2- 172.21.15.47:0/3692616197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fedc40a2ff0 0x7fedc40a54b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fedd801e000 tx=0x7fedd80baf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.829+0000 7fede95e7740 1 -- 172.21.15.47:0/3671011911 shutdown_connections 2024-03-21T11:39:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.829+0000 7fede95e7740 1 --2- 172.21.15.47:0/3671011911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c393a0 0x564d05c3fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.829+0000 7fede95e7740 1 --2- 172.21.15.47:0/3671011911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c389e0 0x564d05c38e60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.829+0000 7fede95e7740 1 --2- 172.21.15.47:0/3671011911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c37810 0x564d05c37c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.829+0000 7fede95e7740 1 -- 172.21.15.47:0/3671011911 >> 172.21.15.47:0/3671011911 conn(0x564d05c32c00 msgr2=0x564d05c33000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.829+0000 7fede95e7740 1 -- 172.21.15.47:0/3671011911 shutdown_connections 2024-03-21T11:39:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.829+0000 7fede95e7740 1 -- 172.21.15.47:0/3671011911 wait complete. 2024-03-21T11:39:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.830+0000 7fede95e7740 1 -- start start 2024-03-21T11:39:44.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.830+0000 7fede95e7740 1 Processor -- start 2024-03-21T11:39:44.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.830+0000 7fede95e7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c37810 0x564d05c41ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.831+0000 7fede95e7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c389e0 0x564d05c42000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.831+0000 7fede95e7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c393a0 0x564d05c49040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.831+0000 7fede95e7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d05c403b0 con 0x564d05c37810 2024-03-21T11:39:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.831+0000 7fede95e7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d05b580d0 con 0x564d05c393a0 2024-03-21T11:39:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.831+0000 7fede95e7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d05c40530 con 0x564d05c389e0 2024-03-21T11:39:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.831+0000 7fede95e7740 10 client.0 _ll_register_callbacks cb 0x564d05b5ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.831+0000 7feddf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c389e0 0x564d05c42000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.831+0000 7feddf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c389e0 0x564d05c42000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48172/0 (socket says 172.21.15.47:48172) 2024-03-21T11:39:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.831+0000 7feddf400640 1 -- 172.21.15.47:0/3277368597 learned_addr learned my addr 172.21.15.47:0/3277368597 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.831+0000 7feddf400640 1 -- 172.21.15.47:0/3277368597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c393a0 msgr2=0x564d05c49040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.831+0000 7feddf400640 1 --2- 172.21.15.47:0/3277368597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c393a0 0x564d05c49040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.831+0000 7feddf400640 1 -- 172.21.15.47:0/3277368597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c37810 msgr2=0x564d05c41ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.831+0000 7feddea00640 1 --2- 172.21.15.47:0/3277368597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c37810 0x564d05c41ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.832+0000 7feddf400640 1 --2- 172.21.15.47:0/3277368597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c37810 0x564d05c41ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.832+0000 7feddf400640 1 -- 172.21.15.47:0/3277368597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d058b03f0 con 0x564d05c389e0 2024-03-21T11:39:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.832+0000 7feddea00640 1 --2- 172.21.15.47:0/3277368597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c37810 0x564d05c41ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.835+0000 7feddf400640 1 --2- 172.21.15.47:0/3277368597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c389e0 0x564d05c42000 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fedd0011010 tx=0x7fedd0004160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1f05bc52e73bdc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.835+0000 7fedc9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.836+0000 7fedc9600640 1 -- 172.21.15.47:0/3277368597 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd00035f0 con 0x564d05c389e0 2024-03-21T11:39:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.837+0000 7fede95e7740 1 -- 172.21.15.47:0/3277368597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedd00041d0 con 0x564d05c389e0 2024-03-21T11:39:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.837+0000 7fedc9600640 1 -- 172.21.15.47:0/3277368597 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fedd0003790 con 0x564d05c389e0 2024-03-21T11:39:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.837+0000 7fedc9600640 1 -- 172.21.15.47:0/3277368597 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedd0032050 con 0x564d05c389e0 2024-03-21T11:39:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.837+0000 7fede95e7740 10 client.30859 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.837+0000 7fede95e7740 1 -- 172.21.15.47:0/3277368597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564d0597a7d0 con 0x564d05c389e0 2024-03-21T11:39:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.838+0000 7fedc9600640 1 -- 172.21.15.47:0/3277368597 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedd00041d0 con 0x564d05c389e0 2024-03-21T11:39:44.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.838+0000 7fedc9600640 1 -- 172.21.15.47:0/3277368597 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fedd00049c0 con 0x564d05c389e0 2024-03-21T11:39:44.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.839+0000 7fede95e7740 10 client.30859 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:44.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.839+0000 7fede95e7740 10 client.30859 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:39:44.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.839+0000 7fede95e7740 20 client.30859 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:44.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.839+0000 7fede95e7740 1 --2- 172.21.15.47:0/3277368597 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564d05c4d1f0 0x564d05c4f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.839+0000 7fede95e7740 4 client.30859 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:44.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.839+0000 7fede95e7740 1 -- 172.21.15.47:0/3277368597 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d05ba9800 con 0x564d05c4d1f0 2024-03-21T11:39:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.842+0000 7feddea00640 1 --2- 172.21.15.47:0/3277368597 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564d05c4d1f0 0x564d05c4f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.843+0000 7feddea00640 1 --2- 172.21.15.47:0/3277368597 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564d05c4d1f0 0x564d05c4f5e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=22540508eadd7477 server_cookie=44ccd8fb364076b9 in_seq=0 out_seq=0 2024-03-21T11:39:44.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.843+0000 7fedc9600640 10 client.30859 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:39:44.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.847+0000 7fedc9600640 1 -- 172.21.15.47:0/3277368597 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564d05ba9800 con 0x564d05c4d1f0 2024-03-21T11:39:44.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.847+0000 7fedc9600640 10 client.30859 handle_command_reply: tid=0 2024-03-21T11:39:44.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.847+0000 7fede95e7740 1 client.30859 shutdown 2024-03-21T11:39:44.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.847+0000 7fede95e7740 1 -- 172.21.15.47:0/3277368597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c389e0 msgr2=0x564d05c42000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:44.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.847+0000 7fede95e7740 1 --2- 172.21.15.47:0/3277368597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c389e0 0x564d05c42000 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fedd0011010 tx=0x7fedd0004160 comp rx=0 tx=0).stop 2024-03-21T11:39:44.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.847+0000 7fede95e7740 1 -- 172.21.15.47:0/3277368597 shutdown_connections 2024-03-21T11:39:44.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.847+0000 7fede95e7740 1 --2- 172.21.15.47:0/3277368597 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564d05c4d1f0 0x564d05c4f5e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.847+0000 7fede95e7740 1 --2- 172.21.15.47:0/3277368597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c393a0 0x564d05c49040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.847+0000 7fede95e7740 1 --2- 172.21.15.47:0/3277368597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c389e0 0x564d05c42000 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.847+0000 7fede95e7740 1 --2- 172.21.15.47:0/3277368597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c37810 0x564d05c41ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.847+0000 7fedc9600640 0 client.30859 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:39:44.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.847+0000 7fede95e7740 1 -- 172.21.15.47:0/3277368597 >> 172.21.15.47:0/3277368597 conn(0x564d05c32c00 msgr2=0x564d05c31eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:44.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.847+0000 7fede95e7740 1 -- 172.21.15.47:0/3277368597 shutdown_connections 2024-03-21T11:39:44.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.847+0000 7fede95e7740 1 -- 172.21.15.47:0/3277368597 wait complete. 2024-03-21T11:39:44.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.848+0000 7fede95e7740 20 client.30859 trim_cache size 0 max 16384 2024-03-21T11:39:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.862+0000 7fede95e7740 1 Processor -- start 2024-03-21T11:39:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.862+0000 7fede95e7740 1 -- start start 2024-03-21T11:39:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.862+0000 7fede95e7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c37810 0x564d05c36570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.862+0000 7fede95e7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c389e0 0x564d05c49000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.862+0000 7fede95e7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c393a0 0x564d05c3ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.862+0000 7fede95e7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc000f9a0 con 0x564d05c393a0 2024-03-21T11:39:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.862+0000 7fede95e7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00113b0 con 0x564d05c37810 2024-03-21T11:39:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.862+0000 7fede95e7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc0010000 con 0x564d05c389e0 2024-03-21T11:39:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.863+0000 7feddf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c389e0 0x564d05c49000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.863+0000 7feddf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c389e0 0x564d05c49000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48188/0 (socket says 172.21.15.47:48188) 2024-03-21T11:39:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.863+0000 7feddf400640 1 -- 172.21.15.47:0/808335321 learned_addr learned my addr 172.21.15.47:0/808335321 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.863+0000 7feddea00640 1 --2- 172.21.15.47:0/808335321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c37810 0x564d05c36570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.863+0000 7feddea00640 1 --2- 172.21.15.47:0/808335321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c393a0 0x564d05c3ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.863+0000 7feddf400640 1 -- 172.21.15.47:0/808335321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c37810 msgr2=0x564d05c36570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.863+0000 7feddf400640 1 --2- 172.21.15.47:0/808335321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c37810 0x564d05c36570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.863+0000 7feddf400640 1 -- 172.21.15.47:0/808335321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c393a0 msgr2=0x564d05c3ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.863+0000 7feddf400640 1 --2- 172.21.15.47:0/808335321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c393a0 0x564d05c3ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.863+0000 7feddf400640 1 -- 172.21.15.47:0/808335321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d05c40ec0 con 0x564d05c389e0 2024-03-21T11:39:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.864+0000 7feddf400640 1 --2- 172.21.15.47:0/808335321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c389e0 0x564d05c49000 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fedd0000c00 tx=0x7fedd00107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ea58bfb127d90e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.864+0000 7fedcbe00640 1 -- 172.21.15.47:0/808335321 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd003a030 con 0x564d05c389e0 2024-03-21T11:39:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.864+0000 7fedcbe00640 1 -- 172.21.15.47:0/808335321 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedd0029070 con 0x564d05c389e0 2024-03-21T11:39:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.864+0000 7fedcbe00640 1 -- 172.21.15.47:0/808335321 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd00047f0 con 0x564d05c389e0 2024-03-21T11:39:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.865+0000 7fede95e7740 1 -- 172.21.15.47:0/808335321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c389e0 msgr2=0x564d05c49000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.865+0000 7fede95e7740 1 --2- 172.21.15.47:0/808335321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c389e0 0x564d05c49000 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fedd0000c00 tx=0x7fedd00107a0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.865+0000 7fede95e7740 1 -- 172.21.15.47:0/808335321 shutdown_connections 2024-03-21T11:39:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.865+0000 7fede95e7740 1 --2- 172.21.15.47:0/808335321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c393a0 0x564d05c3ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.865+0000 7fede95e7740 1 --2- 172.21.15.47:0/808335321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c389e0 0x564d05c49000 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.865+0000 7fede95e7740 1 --2- 172.21.15.47:0/808335321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c37810 0x564d05c36570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.865+0000 7fede95e7740 1 -- 172.21.15.47:0/808335321 >> 172.21.15.47:0/808335321 conn(0x564d05c32c00 msgr2=0x564d05c35e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.865+0000 7fede95e7740 1 -- 172.21.15.47:0/808335321 shutdown_connections 2024-03-21T11:39:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.865+0000 7fede95e7740 1 -- 172.21.15.47:0/808335321 wait complete. 2024-03-21T11:39:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.865+0000 7fede95e7740 1 -- start start 2024-03-21T11:39:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.865+0000 7fede95e7740 1 Processor -- start 2024-03-21T11:39:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.866+0000 7fede95e7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c37810 0x564d05c44c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.866+0000 7fede95e7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c389e0 0x564d05c45150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.866+0000 7fede95e7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c393a0 0x564d05c48120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.866+0000 7fede95e7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc0010cc0 con 0x564d05c393a0 2024-03-21T11:39:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.866+0000 7fede95e7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc0010660 con 0x564d05c389e0 2024-03-21T11:39:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.866+0000 7fede95e7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc000db60 con 0x564d05c37810 2024-03-21T11:39:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.866+0000 7fede95e7740 10 client.0 _ll_register_callbacks cb 0x7fedcc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.866+0000 7feddea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c393a0 0x564d05c48120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.866+0000 7feddea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c393a0 0x564d05c48120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45782/0 (socket says 172.21.15.47:45782) 2024-03-21T11:39:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.866+0000 7feddea00640 1 -- 172.21.15.47:0/1906544569 learned_addr learned my addr 172.21.15.47:0/1906544569 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.867+0000 7feddea00640 1 -- 172.21.15.47:0/1906544569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c37810 msgr2=0x564d05c44c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.867+0000 7feddea00640 1 --2- 172.21.15.47:0/1906544569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c37810 0x564d05c44c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.867+0000 7feddea00640 1 -- 172.21.15.47:0/1906544569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c389e0 msgr2=0x564d05c45150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.867+0000 7feddea00640 1 --2- 172.21.15.47:0/1906544569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c389e0 0x564d05c45150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.867+0000 7feddea00640 1 -- 172.21.15.47:0/1906544569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d05c48710 con 0x564d05c393a0 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.867+0000 7feddea00640 1 --2- 172.21.15.47:0/1906544569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c393a0 0x564d05c48120 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009850 tx=0x7fedcc005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c828a50ea84c37e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.867+0000 7fedc9600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.867+0000 7fedc9600640 1 -- 172.21.15.47:0/1906544569 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc02cd10 con 0x564d05c393a0 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.867+0000 7fede95e7740 1 -- 172.21.15.47:0/1906544569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedd0029070 con 0x564d05c393a0 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.867+0000 7fedc9600640 1 -- 172.21.15.47:0/1906544569 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fedcc004040 con 0x564d05c393a0 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.867+0000 7fedc9600640 1 -- 172.21.15.47:0/1906544569 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedcc004240 con 0x564d05c393a0 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.868+0000 7fede95e7740 10 client.20370 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.868+0000 7fede95e7740 1 -- 172.21.15.47:0/1906544569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564d05ca8fc0 con 0x564d05c393a0 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.868+0000 7fedc9600640 1 -- 172.21.15.47:0/1906544569 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedd0029070 con 0x564d05c393a0 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.868+0000 7fedc9600640 1 -- 172.21.15.47:0/1906544569 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fedcc040040 con 0x564d05c393a0 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.868+0000 7fede95e7740 10 client.20370 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.868+0000 7fede95e7740 10 client.20370 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.868+0000 7fede95e7740 20 client.20370 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.868+0000 7fede95e7740 1 --2- 172.21.15.47:0/1906544569 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564d05ca9970 0x564d05cabe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:44.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.868+0000 7fede95e7740 4 client.20370 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:39:44.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.868+0000 7fede95e7740 1 -- 172.21.15.47:0/1906544569 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564d05ba3b50 con 0x564d05ca9970 2024-03-21T11:39:44.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.869+0000 7feddf400640 1 --2- 172.21.15.47:0/1906544569 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564d05ca9970 0x564d05cabe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:44.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.869+0000 7fedc9600640 10 client.20370 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:39:44.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.869+0000 7feddf400640 1 --2- 172.21.15.47:0/1906544569 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564d05ca9970 0x564d05cabe10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e1c755c4448a2c7 server_cookie=e44546f1ef58143d in_seq=0 out_seq=0 2024-03-21T11:39:44.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.870+0000 7fedc9600640 1 -- 172.21.15.47:0/1906544569 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564d05ba3b50 con 0x564d05ca9970 2024-03-21T11:39:44.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.870+0000 7fedc9600640 10 client.20370 handle_command_reply: tid=0 2024-03-21T11:39:44.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.870+0000 7fede95e7740 1 client.20370 shutdown 2024-03-21T11:39:44.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.870+0000 7fede95e7740 1 -- 172.21.15.47:0/1906544569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c393a0 msgr2=0x564d05c48120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:44.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.870+0000 7fede95e7740 1 --2- 172.21.15.47:0/1906544569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c393a0 0x564d05c48120 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009850 tx=0x7fedcc005e70 comp rx=0 tx=0).stop 2024-03-21T11:39:44.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.870+0000 7fede95e7740 1 -- 172.21.15.47:0/1906544569 shutdown_connections 2024-03-21T11:39:44.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.870+0000 7fede95e7740 1 --2- 172.21.15.47:0/1906544569 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564d05ca9970 0x564d05cabe10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.870+0000 7fede95e7740 1 --2- 172.21.15.47:0/1906544569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d05c393a0 0x564d05c48120 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.870+0000 7fede95e7740 1 --2- 172.21.15.47:0/1906544569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d05c389e0 0x564d05c45150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.870+0000 7fede95e7740 1 --2- 172.21.15.47:0/1906544569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d05c37810 0x564d05c44c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.871+0000 7fede95e7740 1 -- 172.21.15.47:0/1906544569 >> 172.21.15.47:0/1906544569 conn(0x564d05c32c00 msgr2=0x564d05c366f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:44.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.871+0000 7fedc9600640 0 client.20370 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:39:44.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.871+0000 7fede95e7740 1 -- 172.21.15.47:0/1906544569 shutdown_connections 2024-03-21T11:39:44.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.871+0000 7fede95e7740 1 -- 172.21.15.47:0/1906544569 wait complete. 2024-03-21T11:39:44.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.871+0000 7fede95e7740 20 client.20370 trim_cache size 0 max 16384 2024-03-21T11:39:44.875 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:39:44.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.872+0000 7fede6800640 1 -- 172.21.15.47:0/3692616197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fedc40a2ff0 msgr2=0x7fedc40a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:44.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.872+0000 7fede6800640 1 --2- 172.21.15.47:0/3692616197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fedc40a2ff0 0x7fedc40a54b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fedd801e000 tx=0x7fedd80baf90 comp rx=0 tx=0).stop 2024-03-21T11:39:44.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.872+0000 7fede6800640 1 -- 172.21.15.47:0/3692616197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fede0107400 msgr2=0x7fede01a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:44.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.872+0000 7fede6800640 1 --2- 172.21.15.47:0/3692616197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fede0107400 0x7fede01a11d0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fedd800efd0 tx=0x7fedd800c6a0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.872+0000 7fede6800640 1 -- 172.21.15.47:0/3692616197 shutdown_connections 2024-03-21T11:39:44.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.872+0000 7fede6800640 1 --2- 172.21.15.47:0/3692616197 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fedc40a2ff0 0x7fedc40a54b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.872+0000 7fede6800640 1 --2- 172.21.15.47:0/3692616197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fede0107400 0x7fede01a11d0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.872+0000 7fede6800640 1 --2- 172.21.15.47:0/3692616197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0104ad0 0x7fede019a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.872+0000 7fede6800640 1 --2- 172.21.15.47:0/3692616197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fede01021a0 0x7fede0199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:44.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.872+0000 7fede6800640 1 -- 172.21.15.47:0/3692616197 >> 172.21.15.47:0/3692616197 conn(0x7fede00fbb50 msgr2=0x7fede0104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:44.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.873+0000 7fede6800640 1 -- 172.21.15.47:0/3692616197 shutdown_connections 2024-03-21T11:39:44.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:44.873+0000 7fede6800640 1 -- 172.21.15.47:0/3692616197 wait complete. 2024-03-21T11:39:44.893 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:39:44.893 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:39:44.934 INFO:tasks.workunit.client.0.smithi047.stdout: 10 71060 24.33 MB/sec execute 28 sec latency 60.241 ms 2024-03-21T11:39:45.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.197+0000 7f9427e00640 1 -- 172.21.15.47:0/2484919593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9428104930 msgr2=0x7f9428106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.197+0000 7f9427e00640 1 --2- 172.21.15.47:0/2484919593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9428104930 0x7f9428106d20 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9414009900 tx=0x7f941402bc30 comp rx=0 tx=0).stop 2024-03-21T11:39:45.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.197+0000 7f9427e00640 1 -- 172.21.15.47:0/2484919593 shutdown_connections 2024-03-21T11:39:45.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.197+0000 7f9427e00640 1 --2- 172.21.15.47:0/2484919593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9428107260 0x7f94281096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.197+0000 7f9427e00640 1 --2- 172.21.15.47:0/2484919593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9428104930 0x7f9428106d20 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.197+0000 7f9427e00640 1 --2- 172.21.15.47:0/2484919593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9428102000 0x7f94281043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.197+0000 7f9427e00640 1 -- 172.21.15.47:0/2484919593 >> 172.21.15.47:0/2484919593 conn(0x7f94280fba10 msgr2=0x7f94280fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.197+0000 7f9427e00640 1 -- 172.21.15.47:0/2484919593 shutdown_connections 2024-03-21T11:39:45.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.197+0000 7f9427e00640 1 -- 172.21.15.47:0/2484919593 wait complete. 2024-03-21T11:39:45.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.198+0000 7f9427e00640 1 Processor -- start 2024-03-21T11:39:45.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.198+0000 7f9427e00640 1 -- start start 2024-03-21T11:39:45.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9427e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9428102000 0x7f9428195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9427e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9428104930 0x7f9428195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9427e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9428107260 0x7f942819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9427e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942810b050 con 0x7f9428104930 2024-03-21T11:39:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9427e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942810aed0 con 0x7f9428107260 2024-03-21T11:39:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9427e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942810b1d0 con 0x7f9428102000 2024-03-21T11:39:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9426a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9428102000 0x7f9428195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9426a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9428102000 0x7f9428195680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48206/0 (socket says 172.21.15.47:48206) 2024-03-21T11:39:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9426a00640 1 -- 172.21.15.47:0/52852265 learned_addr learned my addr 172.21.15.47:0/52852265 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9426000640 1 --2- 172.21.15.47:0/52852265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9428104930 0x7f9428195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9427400640 1 --2- 172.21.15.47:0/52852265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9428107260 0x7f942819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9426a00640 1 -- 172.21.15.47:0/52852265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9428107260 msgr2=0x7f942819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9426a00640 1 --2- 172.21.15.47:0/52852265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9428107260 0x7f942819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9426a00640 1 -- 172.21.15.47:0/52852265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9428104930 msgr2=0x7f9428195bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.199+0000 7f9426a00640 1 --2- 172.21.15.47:0/52852265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9428104930 0x7f9428195bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.200+0000 7f9426a00640 1 -- 172.21.15.47:0/52852265 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942819d2a0 con 0x7f9428102000 2024-03-21T11:39:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.200+0000 7f9426a00640 1 --2- 172.21.15.47:0/52852265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9428102000 0x7f9428195680 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f941800d950 tx=0x7f941800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.200+0000 7f941f400640 1 -- 172.21.15.47:0/52852265 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418017030 con 0x7f9428102000 2024-03-21T11:39:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.200+0000 7f9427e00640 1 -- 172.21.15.47:0/52852265 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942819d590 con 0x7f9428102000 2024-03-21T11:39:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.200+0000 7f941f400640 1 -- 172.21.15.47:0/52852265 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9418004020 con 0x7f9428102000 2024-03-21T11:39:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.200+0000 7f941f400640 1 -- 172.21.15.47:0/52852265 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94180047a0 con 0x7f9428102000 2024-03-21T11:39:45.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.200+0000 7f9427e00640 1 -- 172.21.15.47:0/52852265 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942819db00 con 0x7f9428102000 2024-03-21T11:39:45.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.204+0000 7f9430464740 1 Processor -- start 2024-03-21T11:39:45.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.204+0000 7f9430464740 1 -- start start 2024-03-21T11:39:45.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.204+0000 7f9430464740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52d200 0x56184a52d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.204+0000 7f9430464740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52e3d0 0x56184a52e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.204+0000 7f9430464740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52ed90 0x56184a535620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.204+0000 7f9430464740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184a4892c0 con 0x56184a52d200 2024-03-21T11:39:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.204+0000 7f9430464740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184a431c90 con 0x56184a52e3d0 2024-03-21T11:39:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.204+0000 7f9430464740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184a485360 con 0x56184a52ed90 2024-03-21T11:39:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.205+0000 7f9427400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52d200 0x56184a52d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.205+0000 7f9427400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52d200 0x56184a52d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45808/0 (socket says 172.21.15.47:45808) 2024-03-21T11:39:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.205+0000 7f9427400640 1 -- 172.21.15.47:0/2125143243 learned_addr learned my addr 172.21.15.47:0/2125143243 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.205+0000 7f9426000640 1 --2- 172.21.15.47:0/2125143243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52ed90 0x56184a535620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.205+0000 7f9427400640 1 -- 172.21.15.47:0/2125143243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52ed90 msgr2=0x56184a535620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.205+0000 7f9427400640 1 --2- 172.21.15.47:0/2125143243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52ed90 0x56184a535620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.205+0000 7f9427400640 1 -- 172.21.15.47:0/2125143243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52e3d0 msgr2=0x56184a52e850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:39:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.205+0000 7f9427400640 1 --2- 172.21.15.47:0/2125143243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52e3d0 0x56184a52e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.205+0000 7f9427400640 1 -- 172.21.15.47:0/2125143243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56184a456080 con 0x56184a52d200 2024-03-21T11:39:45.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.205+0000 7f9426000640 1 --2- 172.21.15.47:0/2125143243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52ed90 0x56184a535620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:39:45.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.205+0000 7f941f400640 1 -- 172.21.15.47:0/52852265 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f94180041d0 con 0x7f9428102000 2024-03-21T11:39:45.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.206+0000 7f9427400640 1 --2- 172.21.15.47:0/2125143243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52d200 0x56184a52d620 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9420009990 tx=0x7f94200118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d79a14c22b4e0a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.206+0000 7f9413e00640 1 -- 172.21.15.47:0/2125143243 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420004d20 con 0x56184a52d200 2024-03-21T11:39:45.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.206+0000 7f9413e00640 1 -- 172.21.15.47:0/2125143243 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9420004050 con 0x56184a52d200 2024-03-21T11:39:45.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.206+0000 7f9413e00640 1 -- 172.21.15.47:0/2125143243 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420005220 con 0x56184a52d200 2024-03-21T11:39:45.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.207+0000 7f9430464740 1 -- 172.21.15.47:0/2125143243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52d200 msgr2=0x56184a52d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.207+0000 7f9430464740 1 --2- 172.21.15.47:0/2125143243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52d200 0x56184a52d620 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9420009990 tx=0x7f94200118d0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.207+0000 7f9430464740 1 -- 172.21.15.47:0/2125143243 shutdown_connections 2024-03-21T11:39:45.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.207+0000 7f941f400640 1 --2- 172.21.15.47:0/52852265 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f940c0a2f10 0x7f940c0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.207+0000 7f9430464740 1 --2- 172.21.15.47:0/2125143243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52ed90 0x56184a535620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.207+0000 7f9430464740 1 --2- 172.21.15.47:0/2125143243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52e3d0 0x56184a52e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.207+0000 7f9430464740 1 --2- 172.21.15.47:0/2125143243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52d200 0x56184a52d620 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.207+0000 7f9430464740 1 -- 172.21.15.47:0/2125143243 >> 172.21.15.47:0/2125143243 conn(0x56184a51fde0 msgr2=0x56184a5201e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.207+0000 7f941f400640 1 -- 172.21.15.47:0/52852265 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f94180cf160 con 0x7f9428102000 2024-03-21T11:39:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.207+0000 7f9427400640 1 --2- 172.21.15.47:0/52852265 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f940c0a2f10 0x7f940c0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.207+0000 7f9430464740 1 -- 172.21.15.47:0/2125143243 shutdown_connections 2024-03-21T11:39:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.208+0000 7f9430464740 1 -- 172.21.15.47:0/2125143243 wait complete. 2024-03-21T11:39:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.208+0000 7f9430464740 1 -- start start 2024-03-21T11:39:45.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.208+0000 7f9430464740 1 Processor -- start 2024-03-21T11:39:45.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.209+0000 7f9427400640 1 --2- 172.21.15.47:0/52852265 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f940c0a2f10 0x7f940c0a53d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9420009d60 tx=0x7f9420004870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.209+0000 7f9430464740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52d200 0x56184a537430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.209+0000 7f9430464740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52e3d0 0x56184a537970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.209+0000 7f9430464740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52ed90 0x56184a53e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.210+0000 7f9426a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52d200 0x56184a537430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.210+0000 7f9426a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52d200 0x56184a537430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48226/0 (socket says 172.21.15.47:48226) 2024-03-21T11:39:45.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.210+0000 7f9426a00640 1 -- 172.21.15.47:0/2082034135 learned_addr learned my addr 172.21.15.47:0/2082034135 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:45.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.210+0000 7f9430464740 1 -- 172.21.15.47:0/2082034135 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184a535da0 con 0x56184a52e3d0 2024-03-21T11:39:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.210+0000 7f9430464740 1 -- 172.21.15.47:0/2082034135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184a4450d0 con 0x56184a52ed90 2024-03-21T11:39:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.210+0000 7f9426000640 1 --2- 172.21.15.47:0/2082034135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52e3d0 0x56184a537970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.210+0000 7f9430464740 1 -- 172.21.15.47:0/2082034135 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56184a535f20 con 0x56184a52d200 2024-03-21T11:39:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.210+0000 7f9427400640 1 --2- 172.21.15.47:0/2082034135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52ed90 0x56184a53e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.210+0000 7f9430464740 10 client.0 _ll_register_callbacks cb 0x56184a449e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.210+0000 7f9426a00640 1 -- 172.21.15.47:0/2082034135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52ed90 msgr2=0x56184a53e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.210+0000 7f9426a00640 1 --2- 172.21.15.47:0/2082034135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52ed90 0x56184a53e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.210+0000 7f9426a00640 1 -- 172.21.15.47:0/2082034135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52e3d0 msgr2=0x56184a537970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.210+0000 7f9426a00640 1 --2- 172.21.15.47:0/2082034135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52e3d0 0x56184a537970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.210+0000 7f9426a00640 1 -- 172.21.15.47:0/2082034135 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56184a19d3f0 con 0x56184a52d200 2024-03-21T11:39:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.211+0000 7f9426a00640 1 --2- 172.21.15.47:0/2082034135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52d200 0x56184a537430 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f94180a39e0 tx=0x7f941809b100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=407af70e8ff91e6e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.211+0000 7f9410c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.211+0000 7f9410c00640 1 -- 172.21.15.47:0/2082034135 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941809f650 con 0x56184a52d200 2024-03-21T11:39:45.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.211+0000 7f9430464740 1 -- 172.21.15.47:0/2082034135 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9420004050 con 0x56184a52d200 2024-03-21T11:39:45.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.211+0000 7f9410c00640 1 -- 172.21.15.47:0/2082034135 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f941809f160 con 0x56184a52d200 2024-03-21T11:39:45.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.211+0000 7f9426000640 1 --2- 172.21.15.47:0/2082034135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52e3d0 0x56184a537970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:45.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.211+0000 7f9410c00640 1 -- 172.21.15.47:0/2082034135 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94180a3f10 con 0x56184a52d200 2024-03-21T11:39:45.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.211+0000 7f9430464740 10 client.30874 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:45.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.211+0000 7f9430464740 1 -- 172.21.15.47:0/2082034135 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56184a2677d0 con 0x56184a52d200 2024-03-21T11:39:45.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.211+0000 7f9410c00640 1 -- 172.21.15.47:0/2082034135 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9420004050 con 0x56184a52d200 2024-03-21T11:39:45.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.212+0000 7f9410c00640 1 -- 172.21.15.47:0/2082034135 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f941809a3c0 con 0x56184a52d200 2024-03-21T11:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.212+0000 7f9430464740 10 client.30874 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.212+0000 7f9430464740 10 client.30874 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.212+0000 7f9430464740 20 client.30874 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.212+0000 7f9430464740 1 --2- 172.21.15.47:0/2082034135 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56184a542e80 0x56184a5452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.212+0000 7f9430464740 4 client.30874 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.212+0000 7f9430464740 1 -- 172.21.15.47:0/2082034135 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56184a496800 con 0x56184a542e80 2024-03-21T11:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.212+0000 7f9426000640 1 --2- 172.21.15.47:0/2082034135 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56184a542e80 0x56184a5452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.212+0000 7f9426000640 1 --2- 172.21.15.47:0/2082034135 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56184a542e80 0x56184a5452b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=162395511bb1f4c6 server_cookie=3665c4a4cd0c01cb in_seq=0 out_seq=0 2024-03-21T11:39:45.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.212+0000 7f9410c00640 10 client.30874 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:39:45.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.214+0000 7f9410c00640 1 -- 172.21.15.47:0/2082034135 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56184a496800 con 0x56184a542e80 2024-03-21T11:39:45.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.214+0000 7f9410c00640 10 client.30874 handle_command_reply: tid=0 2024-03-21T11:39:45.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.214+0000 7f9430464740 1 client.30874 shutdown 2024-03-21T11:39:45.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.215+0000 7f9430464740 1 -- 172.21.15.47:0/2082034135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52d200 msgr2=0x56184a537430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.215+0000 7f9430464740 1 --2- 172.21.15.47:0/2082034135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52d200 0x56184a537430 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f94180a39e0 tx=0x7f941809b100 comp rx=0 tx=0).stop 2024-03-21T11:39:45.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.215+0000 7f9430464740 1 -- 172.21.15.47:0/2082034135 shutdown_connections 2024-03-21T11:39:45.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.215+0000 7f9430464740 1 --2- 172.21.15.47:0/2082034135 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56184a542e80 0x56184a5452b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.215+0000 7f9430464740 1 --2- 172.21.15.47:0/2082034135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52ed90 0x56184a53e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.215+0000 7f9430464740 1 --2- 172.21.15.47:0/2082034135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52e3d0 0x56184a537970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.215+0000 7f9410c00640 0 client.30874 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:39:45.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.215+0000 7f9430464740 1 --2- 172.21.15.47:0/2082034135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52d200 0x56184a537430 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.215+0000 7f9430464740 1 -- 172.21.15.47:0/2082034135 >> 172.21.15.47:0/2082034135 conn(0x56184a51fde0 msgr2=0x56184a51f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.215+0000 7f9430464740 1 -- 172.21.15.47:0/2082034135 shutdown_connections 2024-03-21T11:39:45.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.215+0000 7f9430464740 1 -- 172.21.15.47:0/2082034135 wait complete. 2024-03-21T11:39:45.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.216+0000 7f9430464740 20 client.30874 trim_cache size 0 max 16384 2024-03-21T11:39:45.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.238+0000 7f9430464740 1 Processor -- start 2024-03-21T11:39:45.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.238+0000 7f9430464740 1 -- start start 2024-03-21T11:39:45.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.239+0000 7f9430464740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52d200 0x56184a531470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.239+0000 7f9430464740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52e3d0 0x56184a53e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.239+0000 7f9430464740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52ed90 0x56184a5319b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.239+0000 7f9430464740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942819f4c0 con 0x56184a52ed90 2024-03-21T11:39:45.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.239+0000 7f9430464740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94281a0ed0 con 0x56184a52d200 2024-03-21T11:39:45.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.239+0000 7f9430464740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942819fb20 con 0x56184a52e3d0 2024-03-21T11:39:45.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.239+0000 7f9426a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52d200 0x56184a531470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.239+0000 7f9426a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52d200 0x56184a531470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60406/0 (socket says 172.21.15.47:60406) 2024-03-21T11:39:45.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.239+0000 7f9426a00640 1 -- 172.21.15.47:0/3954975724 learned_addr learned my addr 172.21.15.47:0/3954975724 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:45.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.239+0000 7f9427400640 1 --2- 172.21.15.47:0/3954975724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52ed90 0x56184a5319b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.239+0000 7f9426000640 1 --2- 172.21.15.47:0/3954975724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52e3d0 0x56184a53e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.240+0000 7f9426a00640 1 -- 172.21.15.47:0/3954975724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52e3d0 msgr2=0x56184a53e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.240+0000 7f9426a00640 1 --2- 172.21.15.47:0/3954975724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52e3d0 0x56184a53e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.240+0000 7f9426a00640 1 -- 172.21.15.47:0/3954975724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52ed90 msgr2=0x56184a5319b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.240+0000 7f9426a00640 1 --2- 172.21.15.47:0/3954975724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52ed90 0x56184a5319b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.240+0000 7f9426a00640 1 -- 172.21.15.47:0/3954975724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56184a534f60 con 0x56184a52d200 2024-03-21T11:39:45.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.240+0000 7f9426000640 1 --2- 172.21.15.47:0/3954975724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52e3d0 0x56184a53e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:45.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.240+0000 7f9426a00640 1 --2- 172.21.15.47:0/3954975724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52d200 0x56184a531470 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9418097ed0 tx=0x7f941809b170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=453935e2c91dc893 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.241+0000 7f9413400640 1 -- 172.21.15.47:0/3954975724 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941809f650 con 0x56184a52d200 2024-03-21T11:39:45.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.241+0000 7f9413400640 1 -- 172.21.15.47:0/3954975724 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f941809a950 con 0x56184a52d200 2024-03-21T11:39:45.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.241+0000 7f9413400640 1 -- 172.21.15.47:0/3954975724 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941809f160 con 0x56184a52d200 2024-03-21T11:39:45.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.242+0000 7f9430464740 1 -- 172.21.15.47:0/3954975724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52d200 msgr2=0x56184a531470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.242+0000 7f9430464740 1 --2- 172.21.15.47:0/3954975724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52d200 0x56184a531470 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9418097ed0 tx=0x7f941809b170 comp rx=0 tx=0).stop 2024-03-21T11:39:45.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.243+0000 7f9430464740 1 -- 172.21.15.47:0/3954975724 shutdown_connections 2024-03-21T11:39:45.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.243+0000 7f9430464740 1 --2- 172.21.15.47:0/3954975724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52ed90 0x56184a5319b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.243+0000 7f9430464740 1 --2- 172.21.15.47:0/3954975724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52e3d0 0x56184a53e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.243+0000 7f9430464740 1 --2- 172.21.15.47:0/3954975724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52d200 0x56184a531470 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.243+0000 7f9430464740 1 -- 172.21.15.47:0/3954975724 >> 172.21.15.47:0/3954975724 conn(0x56184a51fde0 msgr2=0x56184a5238d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.243+0000 7f9430464740 1 -- 172.21.15.47:0/3954975724 shutdown_connections 2024-03-21T11:39:45.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.243+0000 7f9430464740 1 -- 172.21.15.47:0/3954975724 wait complete. 2024-03-21T11:39:45.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.244+0000 7f9430464740 1 -- start start 2024-03-21T11:39:45.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.244+0000 7f9430464740 1 Processor -- start 2024-03-21T11:39:45.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.245+0000 7f9430464740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52d200 0x56184a53a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.245+0000 7f9430464740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52e3d0 0x56184a53acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.245+0000 7f9430464740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52ed90 0x56184a53dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.245+0000 7f9430464740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94281a07e0 con 0x56184a52e3d0 2024-03-21T11:39:45.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.245+0000 7f9430464740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94281a0180 con 0x56184a52ed90 2024-03-21T11:39:45.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.245+0000 7f9430464740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9428073130 con 0x56184a52d200 2024-03-21T11:39:45.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.245+0000 7f9430464740 10 client.0 _ll_register_callbacks cb 0x7f941809b2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.245+0000 7f9426a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52d200 0x56184a53a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.245+0000 7f9427400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52ed90 0x56184a53dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.245+0000 7f9427400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52ed90 0x56184a53dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60410/0 (socket says 172.21.15.47:60410) 2024-03-21T11:39:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.246+0000 7f9427400640 1 -- 172.21.15.47:0/1969924452 learned_addr learned my addr 172.21.15.47:0/1969924452 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.246+0000 7f9426000640 1 --2- 172.21.15.47:0/1969924452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52e3d0 0x56184a53acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.246+0000 7f9427400640 1 -- 172.21.15.47:0/1969924452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52d200 msgr2=0x56184a53a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.246+0000 7f9427400640 1 --2- 172.21.15.47:0/1969924452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52d200 0x56184a53a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.246+0000 7f9427400640 1 -- 172.21.15.47:0/1969924452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52e3d0 msgr2=0x56184a53acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.246+0000 7f9427400640 1 --2- 172.21.15.47:0/1969924452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52e3d0 0x56184a53acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.246+0000 7f9427400640 1 -- 172.21.15.47:0/1969924452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56184a53e2c0 con 0x56184a52ed90 2024-03-21T11:39:45.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.247+0000 7f9427400640 1 --2- 172.21.15.47:0/1969924452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52ed90 0x56184a53dcd0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9420009fb0 tx=0x7f94200118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=262f391533dce989 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.247+0000 7f9410c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:45.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.248+0000 7f9410c00640 1 -- 172.21.15.47:0/1969924452 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420049070 con 0x56184a52ed90 2024-03-21T11:39:45.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.248+0000 7f9410c00640 1 -- 172.21.15.47:0/1969924452 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9420005250 con 0x56184a52ed90 2024-03-21T11:39:45.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.248+0000 7f9430464740 1 -- 172.21.15.47:0/1969924452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f941809a950 con 0x56184a52ed90 2024-03-21T11:39:45.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.250+0000 7f9410c00640 1 -- 172.21.15.47:0/1969924452 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9420053050 con 0x56184a52ed90 2024-03-21T11:39:45.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.250+0000 7f9430464740 10 client.30881 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:45.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.250+0000 7f9430464740 1 -- 172.21.15.47:0/1969924452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56184a59ed60 con 0x56184a52ed90 2024-03-21T11:39:45.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.250+0000 7f9410c00640 1 -- 172.21.15.47:0/1969924452 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f941809a950 con 0x56184a52ed90 2024-03-21T11:39:45.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.250+0000 7f9410c00640 1 -- 172.21.15.47:0/1969924452 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f9420043040 con 0x56184a52ed90 2024-03-21T11:39:45.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.250+0000 7f9430464740 10 client.30881 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:45.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.250+0000 7f9430464740 10 client.30881 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:39:45.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.251+0000 7f9430464740 20 client.30881 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:45.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.251+0000 7f9430464740 1 --2- 172.21.15.47:0/1969924452 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56184a59f470 0x56184a5a1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.251+0000 7f9430464740 4 client.30881 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:39:45.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.251+0000 7f9430464740 1 -- 172.21.15.47:0/1969924452 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56184a490b50 con 0x56184a59f470 2024-03-21T11:39:45.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.251+0000 7f9426a00640 1 --2- 172.21.15.47:0/1969924452 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56184a59f470 0x56184a5a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.252+0000 7f9426a00640 1 --2- 172.21.15.47:0/1969924452 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56184a59f470 0x56184a5a1890 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=590d4f2c0d5dc8a9 server_cookie=78279b237fd7c641 in_seq=0 out_seq=0 2024-03-21T11:39:45.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.252+0000 7f9410c00640 10 client.30881 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:39:45.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.252+0000 7f9410c00640 1 -- 172.21.15.47:0/1969924452 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56184a490b50 con 0x56184a59f470 2024-03-21T11:39:45.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.252+0000 7f9410c00640 10 client.30881 handle_command_reply: tid=0 2024-03-21T11:39:45.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.252+0000 7f9430464740 1 client.30881 shutdown 2024-03-21T11:39:45.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.253+0000 7f9430464740 1 -- 172.21.15.47:0/1969924452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52ed90 msgr2=0x56184a53dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.253+0000 7f9430464740 1 --2- 172.21.15.47:0/1969924452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52ed90 0x56184a53dcd0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9420009fb0 tx=0x7f94200118d0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.253+0000 7f9430464740 1 -- 172.21.15.47:0/1969924452 shutdown_connections 2024-03-21T11:39:45.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.253+0000 7f9430464740 1 --2- 172.21.15.47:0/1969924452 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56184a59f470 0x56184a5a1890 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.253+0000 7f9430464740 1 --2- 172.21.15.47:0/1969924452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56184a52ed90 0x56184a53dcd0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.253+0000 7f9430464740 1 --2- 172.21.15.47:0/1969924452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56184a52e3d0 0x56184a53acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.253+0000 7f9430464740 1 --2- 172.21.15.47:0/1969924452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56184a52d200 0x56184a53a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.253+0000 7f9430464740 1 -- 172.21.15.47:0/1969924452 >> 172.21.15.47:0/1969924452 conn(0x56184a51fde0 msgr2=0x56184a51e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.253+0000 7f9410c00640 0 client.30881 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:39:45.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.253+0000 7f9430464740 1 -- 172.21.15.47:0/1969924452 shutdown_connections 2024-03-21T11:39:45.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.253+0000 7f9430464740 1 -- 172.21.15.47:0/1969924452 wait complete. 2024-03-21T11:39:45.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.253+0000 7f9430464740 20 client.30881 trim_cache size 0 max 16384 2024-03-21T11:39:45.255 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:39:45.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.254+0000 7f9427e00640 1 -- 172.21.15.47:0/52852265 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f940c0a2f10 msgr2=0x7f940c0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.254+0000 7f9427e00640 1 --2- 172.21.15.47:0/52852265 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f940c0a2f10 0x7f940c0a53d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9420009d60 tx=0x7f9420004870 comp rx=0 tx=0).stop 2024-03-21T11:39:45.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.254+0000 7f9427e00640 1 -- 172.21.15.47:0/52852265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9428102000 msgr2=0x7f9428195680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.254+0000 7f9427e00640 1 --2- 172.21.15.47:0/52852265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9428102000 0x7f9428195680 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f941800d950 tx=0x7f941800de10 comp rx=0 tx=0).stop 2024-03-21T11:39:45.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.255+0000 7f9427e00640 1 -- 172.21.15.47:0/52852265 shutdown_connections 2024-03-21T11:39:45.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.255+0000 7f9427e00640 1 --2- 172.21.15.47:0/52852265 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f940c0a2f10 0x7f940c0a53d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.255+0000 7f9427e00640 1 --2- 172.21.15.47:0/52852265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9428107260 0x7f942819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.255+0000 7f9427e00640 1 --2- 172.21.15.47:0/52852265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9428104930 0x7f9428195bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.255+0000 7f9427e00640 1 --2- 172.21.15.47:0/52852265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9428102000 0x7f9428195680 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.255+0000 7f9427e00640 1 -- 172.21.15.47:0/52852265 >> 172.21.15.47:0/52852265 conn(0x7f94280fba10 msgr2=0x7f9428102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.255+0000 7f9427e00640 1 -- 172.21.15.47:0/52852265 shutdown_connections 2024-03-21T11:39:45.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.255+0000 7f9427e00640 1 -- 172.21.15.47:0/52852265 wait complete. 2024-03-21T11:39:45.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:45 smithi047 ceph-mon[31351]: pgmap v892: 129 pgs: 129 active+clean; 369 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 13 MiB/s wr, 914 op/s 2024-03-21T11:39:45.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:45 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2775542030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:39:45.282 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:39:45.282 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:39:45.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:45 smithi100 ceph-mon[36245]: pgmap v892: 129 pgs: 129 active+clean; 369 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 13 MiB/s wr, 914 op/s 2024-03-21T11:39:45.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:45 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2775542030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:39:45.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:45 smithi102 ceph-mon[36748]: pgmap v892: 129 pgs: 129 active+clean; 369 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 13 MiB/s wr, 914 op/s 2024-03-21T11:39:45.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:45 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2775542030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:39:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.611+0000 7fe1ada00640 1 -- 172.21.15.47:0/2013790447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a8101030 msgr2=0x7fe1a81018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.611+0000 7fe1ada00640 1 --2- 172.21.15.47:0/2013790447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a8101030 0x7fe1a81018d0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009be0 tx=0x7fe1a002bef0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.611+0000 7fe1ada00640 1 -- 172.21.15.47:0/2013790447 shutdown_connections 2024-03-21T11:39:45.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.611+0000 7fe1ada00640 1 --2- 172.21.15.47:0/2013790447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a8101030 0x7fe1a81018d0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.611+0000 7fe1ada00640 1 --2- 172.21.15.47:0/2013790447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a8100690 0x7fe1a8100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.611+0000 7fe1ada00640 1 --2- 172.21.15.47:0/2013790447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a80ff560 0x7fe1a80ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.611+0000 7fe1ada00640 1 -- 172.21.15.47:0/2013790447 >> 172.21.15.47:0/2013790447 conn(0x7fe1a80fae60 msgr2=0x7fe1a80fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.612+0000 7fe1ada00640 1 -- 172.21.15.47:0/2013790447 shutdown_connections 2024-03-21T11:39:45.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.612+0000 7fe1ada00640 1 -- 172.21.15.47:0/2013790447 wait complete. 2024-03-21T11:39:45.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.613+0000 7fe1ada00640 1 Processor -- start 2024-03-21T11:39:45.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.613+0000 7fe1ada00640 1 -- start start 2024-03-21T11:39:45.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.614+0000 7fe1ada00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a80ff560 0x7fe1a8199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.614+0000 7fe1ada00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a8100690 0x7fe1a819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.614+0000 7fe1ada00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a8101030 0x7fe1a81a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.614+0000 7fe1ada00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a810f5e0 con 0x7fe1a8100690 2024-03-21T11:39:45.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.614+0000 7fe1ada00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a810f460 con 0x7fe1a80ff560 2024-03-21T11:39:45.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.614+0000 7fe1ada00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a810f760 con 0x7fe1a8101030 2024-03-21T11:39:45.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.614+0000 7fe1a7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a8101030 0x7fe1a81a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.614+0000 7fe1a6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a8100690 0x7fe1a819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.614+0000 7fe1a7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a8101030 0x7fe1a81a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48264/0 (socket says 172.21.15.47:48264) 2024-03-21T11:39:45.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.614+0000 7fe1a7400640 1 -- 172.21.15.47:0/2110250078 learned_addr learned my addr 172.21.15.47:0/2110250078 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.614+0000 7fe1a6a00640 1 --2- 172.21.15.47:0/2110250078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a80ff560 0x7fe1a8199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.615+0000 7fe1a7400640 1 -- 172.21.15.47:0/2110250078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a80ff560 msgr2=0x7fe1a8199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.615+0000 7fe1a7400640 1 --2- 172.21.15.47:0/2110250078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a80ff560 0x7fe1a8199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.615+0000 7fe1a7400640 1 -- 172.21.15.47:0/2110250078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a8100690 msgr2=0x7fe1a819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.615+0000 7fe1a7400640 1 --2- 172.21.15.47:0/2110250078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a8100690 0x7fe1a819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.615+0000 7fe1a7400640 1 -- 172.21.15.47:0/2110250078 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a81a1830 con 0x7fe1a8101030 2024-03-21T11:39:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.615+0000 7fe1a7400640 1 --2- 172.21.15.47:0/2110250078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a8101030 0x7fe1a81a1190 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0036040 tx=0x7fe1a002cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.616+0000 7fe19f400640 1 -- 172.21.15.47:0/2110250078 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0043020 con 0x7fe1a8101030 2024-03-21T11:39:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.616+0000 7fe19f400640 1 -- 172.21.15.47:0/2110250078 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1a00338b0 con 0x7fe1a8101030 2024-03-21T11:39:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.616+0000 7fe19f400640 1 -- 172.21.15.47:0/2110250078 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a00347b0 con 0x7fe1a8101030 2024-03-21T11:39:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.616+0000 7fe1ada00640 1 -- 172.21.15.47:0/2110250078 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a8102d70 con 0x7fe1a8101030 2024-03-21T11:39:45.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.616+0000 7fe1ada00640 1 -- 172.21.15.47:0/2110250078 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a81032f0 con 0x7fe1a8101030 2024-03-21T11:39:45.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.620+0000 7fe1b065e740 1 Processor -- start 2024-03-21T11:39:45.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.620+0000 7fe19f400640 1 -- 172.21.15.47:0/2110250078 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe1a0046020 con 0x7fe1a8101030 2024-03-21T11:39:45.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.620+0000 7fe1b065e740 1 -- start start 2024-03-21T11:39:45.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.621+0000 7fe1b065e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5d810 0x55950de5dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.621+0000 7fe1b065e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5e9e0 0x55950de5ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.621+0000 7fe1a6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5d810 0x55950de5dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.621+0000 7fe1a6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5d810 0x55950de5dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60436/0 (socket says 172.21.15.47:60436) 2024-03-21T11:39:45.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.621+0000 7fe1a6000640 1 -- 172.21.15.47:0/4278612161 learned_addr learned my addr 172.21.15.47:0/4278612161 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:45.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.621+0000 7fe1b065e740 1 --2- 172.21.15.47:0/4278612161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5f3a0 0x55950de65c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.621+0000 7fe1b065e740 1 -- 172.21.15.47:0/4278612161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950ddc22c0 con 0x55950de5e9e0 2024-03-21T11:39:45.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.621+0000 7fe1b065e740 1 -- 172.21.15.47:0/4278612161 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950dd6ac90 con 0x55950de5d810 2024-03-21T11:39:45.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.621+0000 7fe1b065e740 1 -- 172.21.15.47:0/4278612161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950ddbe360 con 0x55950de5f3a0 2024-03-21T11:39:45.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.621+0000 7fe19f400640 1 --2- 172.21.15.47:0/2110250078 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe18c0a2fa0 0x7fe18c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.621+0000 7fe1a6a00640 1 --2- 172.21.15.47:0/4278612161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5e9e0 0x55950de5ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.622+0000 7fe19f400640 1 -- 172.21.15.47:0/2110250078 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe1a00f2110 con 0x7fe1a8101030 2024-03-21T11:39:45.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.622+0000 7fe1a6000640 1 --2- 172.21.15.47:0/4278612161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5f3a0 0x55950de65c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.622+0000 7fe1a6000640 1 -- 172.21.15.47:0/4278612161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5f3a0 msgr2=0x55950de65c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.622+0000 7fe1a6000640 1 --2- 172.21.15.47:0/4278612161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5f3a0 0x55950de65c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.622+0000 7fe1a6000640 1 -- 172.21.15.47:0/4278612161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5e9e0 msgr2=0x55950de5ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.622+0000 7fe1a6000640 1 --2- 172.21.15.47:0/4278612161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5e9e0 0x55950de5ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.622+0000 7fe1a7400640 1 --2- 172.21.15.47:0/2110250078 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe18c0a2fa0 0x7fe18c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.622+0000 7fe1a6000640 1 -- 172.21.15.47:0/4278612161 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55950dd8f080 con 0x55950de5d810 2024-03-21T11:39:45.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.622+0000 7fe1a6a00640 1 --2- 172.21.15.47:0/4278612161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5e9e0 0x55950de5ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:39:45.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.623+0000 7fe1a6000640 1 --2- 172.21.15.47:0/4278612161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5d810 0x55950de5dc30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe1940145b0 tx=0x7fe19400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60322531f788bfb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.623+0000 7fe1a7400640 1 --2- 172.21.15.47:0/2110250078 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe18c0a2fa0 0x7fe18c0a5460 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00be3e0 tx=0x7fe1a002bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.623+0000 7fe193e00640 1 -- 172.21.15.47:0/4278612161 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19401a030 con 0x55950de5d810 2024-03-21T11:39:45.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.623+0000 7fe193e00640 1 -- 172.21.15.47:0/4278612161 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe194005500 con 0x55950de5d810 2024-03-21T11:39:45.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.623+0000 7fe193e00640 1 -- 172.21.15.47:0/4278612161 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194010590 con 0x55950de5d810 2024-03-21T11:39:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.624+0000 7fe1b065e740 1 -- 172.21.15.47:0/4278612161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5d810 msgr2=0x55950de5dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.624+0000 7fe1b065e740 1 --2- 172.21.15.47:0/4278612161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5d810 0x55950de5dc30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe1940145b0 tx=0x7fe19400b960 comp rx=0 tx=0).stop 2024-03-21T11:39:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.624+0000 7fe1b065e740 1 -- 172.21.15.47:0/4278612161 shutdown_connections 2024-03-21T11:39:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.624+0000 7fe1b065e740 1 --2- 172.21.15.47:0/4278612161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5f3a0 0x55950de65c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.624+0000 7fe1b065e740 1 --2- 172.21.15.47:0/4278612161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5e9e0 0x55950de5ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.624+0000 7fe1b065e740 1 --2- 172.21.15.47:0/4278612161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5d810 0x55950de5dc30 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.624+0000 7fe1b065e740 1 -- 172.21.15.47:0/4278612161 >> 172.21.15.47:0/4278612161 conn(0x55950de58c00 msgr2=0x55950de59000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.624+0000 7fe1b065e740 1 -- 172.21.15.47:0/4278612161 shutdown_connections 2024-03-21T11:39:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.624+0000 7fe1b065e740 1 -- 172.21.15.47:0/4278612161 wait complete. 2024-03-21T11:39:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.624+0000 7fe1b065e740 1 -- start start 2024-03-21T11:39:45.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.625+0000 7fe1b065e740 1 Processor -- start 2024-03-21T11:39:45.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.625+0000 7fe1b065e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5d810 0x55950de67b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.625+0000 7fe1b065e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5e9e0 0x55950de68050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.625+0000 7fe1a6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5d810 0x55950de67b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.625+0000 7fe1a6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5d810 0x55950de67b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60446/0 (socket says 172.21.15.47:60446) 2024-03-21T11:39:45.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.625+0000 7fe1a6000640 1 -- 172.21.15.47:0/366693838 learned_addr learned my addr 172.21.15.47:0/366693838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:45.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.626+0000 7fe1b065e740 1 --2- 172.21.15.47:0/366693838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5f3a0 0x55950de6f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.626+0000 7fe1b065e740 1 -- 172.21.15.47:0/366693838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950de663b0 con 0x55950de5f3a0 2024-03-21T11:39:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.626+0000 7fe1b065e740 1 -- 172.21.15.47:0/366693838 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950dd7e0d0 con 0x55950de5d810 2024-03-21T11:39:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.626+0000 7fe1b065e740 1 -- 172.21.15.47:0/366693838 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950de66530 con 0x55950de5e9e0 2024-03-21T11:39:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.626+0000 7fe1b065e740 10 client.0 _ll_register_callbacks cb 0x55950dd82e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.626+0000 7fe1a6000640 1 -- 172.21.15.47:0/366693838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5e9e0 msgr2=0x55950de68050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:39:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.626+0000 7fe1a6000640 1 --2- 172.21.15.47:0/366693838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5e9e0 0x55950de68050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.626+0000 7fe1a6000640 1 -- 172.21.15.47:0/366693838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5f3a0 msgr2=0x55950de6f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.626+0000 7fe1a6000640 1 --2- 172.21.15.47:0/366693838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5f3a0 0x55950de6f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.626+0000 7fe1a6000640 1 -- 172.21.15.47:0/366693838 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55950dad63f0 con 0x55950de5d810 2024-03-21T11:39:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.626+0000 7fe1b065e740 1 -- 172.21.15.47:0/366693838 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe194005500 con 0x55950de5d810 2024-03-21T11:39:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.627+0000 7fe1a6000640 1 --2- 172.21.15.47:0/366693838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5d810 0x55950de67b10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe194014f00 tx=0x7fe194014380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f237785666ba7bf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.627+0000 7fe190c00640 10 client.30893 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.627+0000 7fe190c00640 1 -- 172.21.15.47:0/366693838 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19402c060 con 0x55950de5d810 2024-03-21T11:39:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.627+0000 7fe190c00640 1 -- 172.21.15.47:0/366693838 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe194024070 con 0x55950de5d810 2024-03-21T11:39:45.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.628+0000 7fe190c00640 1 -- 172.21.15.47:0/366693838 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe19401a030 con 0x55950de5d810 2024-03-21T11:39:45.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.628+0000 7fe1b065e740 10 client.30893 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:45.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.628+0000 7fe1b065e740 1 -- 172.21.15.47:0/366693838 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=39}) v3 -- 0x55950dba07d0 con 0x55950de5d810 2024-03-21T11:39:45.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.628+0000 7fe190c00640 1 -- 172.21.15.47:0/366693838 <== mon.2 v2:172.21.15.100:3300/0 4 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fe194004040 con 0x55950de5d810 2024-03-21T11:39:45.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.629+0000 7fe1b065e740 10 client.30893 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:45.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.629+0000 7fe1b065e740 10 client.30893 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:39:45.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.629+0000 7fe1b065e740 20 client.30893 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:45.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.629+0000 7fe1b065e740 1 --2- 172.21.15.47:0/366693838 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55950de731a0 0x55950de75590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.629+0000 7fe1b065e740 4 client.30893 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:45.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.629+0000 7fe1b065e740 1 -- 172.21.15.47:0/366693838 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55950ddcf800 con 0x55950de731a0 2024-03-21T11:39:45.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.629+0000 7fe1a6a00640 1 --2- 172.21.15.47:0/366693838 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55950de731a0 0x55950de75590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.630+0000 7fe1a6a00640 1 --2- 172.21.15.47:0/366693838 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55950de731a0 0x55950de75590 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28530393a2d42fcc server_cookie=80fa3359509bb291 in_seq=0 out_seq=0 2024-03-21T11:39:45.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.630+0000 7fe190c00640 10 client.30893 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:39:45.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.631+0000 7fe190c00640 1 -- 172.21.15.47:0/366693838 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55950ddcf800 con 0x55950de731a0 2024-03-21T11:39:45.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.631+0000 7fe190c00640 10 client.30893 handle_command_reply: tid=0 2024-03-21T11:39:45.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.632+0000 7fe1b065e740 1 client.30893 shutdown 2024-03-21T11:39:45.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.633+0000 7fe1b065e740 1 -- 172.21.15.47:0/366693838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5d810 msgr2=0x55950de67b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.633+0000 7fe1b065e740 1 --2- 172.21.15.47:0/366693838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5d810 0x55950de67b10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe194014f00 tx=0x7fe194014380 comp rx=0 tx=0).stop 2024-03-21T11:39:45.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.636+0000 7fe1b065e740 1 -- 172.21.15.47:0/366693838 shutdown_connections 2024-03-21T11:39:45.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.636+0000 7fe1b065e740 1 --2- 172.21.15.47:0/366693838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5f3a0 0x55950de6f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.636+0000 7fe1b065e740 1 --2- 172.21.15.47:0/366693838 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55950de731a0 0x55950de75590 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.636+0000 7fe1b065e740 1 --2- 172.21.15.47:0/366693838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5e9e0 0x55950de68050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.636+0000 7fe1b065e740 1 --2- 172.21.15.47:0/366693838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5d810 0x55950de67b10 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.636+0000 7fe1b065e740 1 -- 172.21.15.47:0/366693838 >> 172.21.15.47:0/366693838 conn(0x55950de58c00 msgr2=0x55950de57eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.636+0000 7fe190c00640 0 client.30893 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:39:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.636+0000 7fe1b065e740 1 -- 172.21.15.47:0/366693838 shutdown_connections 2024-03-21T11:39:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.636+0000 7fe1b065e740 1 -- 172.21.15.47:0/366693838 wait complete. 2024-03-21T11:39:45.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.637+0000 7fe1b065e740 20 client.30893 trim_cache size 0 max 16384 2024-03-21T11:39:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.660+0000 7fe1b065e740 1 Processor -- start 2024-03-21T11:39:45.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.660+0000 7fe1b065e740 1 -- start start 2024-03-21T11:39:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.660+0000 7fe1b065e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5d810 0x55950de67fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.660+0000 7fe1b065e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5e9e0 0x55950de6f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.660+0000 7fe1b065e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5f3a0 0x55950de668d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.660+0000 7fe1b065e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81b0730 con 0x55950de5d810 2024-03-21T11:39:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.660+0000 7fe1b065e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81b2140 con 0x55950de5e9e0 2024-03-21T11:39:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.660+0000 7fe1b065e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81b0d90 con 0x55950de5f3a0 2024-03-21T11:39:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.661+0000 7fe1a6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5f3a0 0x55950de668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.661+0000 7fe1a6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5f3a0 0x55950de668d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48298/0 (socket says 172.21.15.47:48298) 2024-03-21T11:39:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.661+0000 7fe1a6000640 1 -- 172.21.15.47:0/3979970106 learned_addr learned my addr 172.21.15.47:0/3979970106 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.661+0000 7fe1a6a00640 1 --2- 172.21.15.47:0/3979970106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5e9e0 0x55950de6f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.661+0000 7fe1a6000640 1 -- 172.21.15.47:0/3979970106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5e9e0 msgr2=0x55950de6f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.661+0000 7fe1a6000640 1 --2- 172.21.15.47:0/3979970106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5e9e0 0x55950de6f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.661+0000 7fe1a6000640 1 -- 172.21.15.47:0/3979970106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5d810 msgr2=0x55950de67fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.661+0000 7fe1a6000640 1 --2- 172.21.15.47:0/3979970106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5d810 0x55950de67fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.661+0000 7fe1a6000640 1 -- 172.21.15.47:0/3979970106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55950de65600 con 0x55950de5f3a0 2024-03-21T11:39:45.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.662+0000 7fe1a6000640 1 --2- 172.21.15.47:0/3979970106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5f3a0 0x55950de668d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe194000c00 tx=0x7fe194021040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ef6bf5ef7a15a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.662+0000 7fe193400640 1 -- 172.21.15.47:0/3979970106 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194005c50 con 0x55950de5f3a0 2024-03-21T11:39:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.662+0000 7fe193400640 1 -- 172.21.15.47:0/3979970106 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe194005500 con 0x55950de5f3a0 2024-03-21T11:39:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.662+0000 7fe193400640 1 -- 172.21.15.47:0/3979970106 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194011800 con 0x55950de5f3a0 2024-03-21T11:39:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.663+0000 7fe1b065e740 1 -- 172.21.15.47:0/3979970106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5f3a0 msgr2=0x55950de668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.663+0000 7fe1b065e740 1 --2- 172.21.15.47:0/3979970106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5f3a0 0x55950de668d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe194000c00 tx=0x7fe194021040 comp rx=0 tx=0).stop 2024-03-21T11:39:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.664+0000 7fe1b065e740 1 -- 172.21.15.47:0/3979970106 shutdown_connections 2024-03-21T11:39:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.664+0000 7fe1b065e740 1 --2- 172.21.15.47:0/3979970106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5f3a0 0x55950de668d0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.664+0000 7fe1b065e740 1 --2- 172.21.15.47:0/3979970106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5e9e0 0x55950de6f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.664+0000 7fe1b065e740 1 --2- 172.21.15.47:0/3979970106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5d810 0x55950de67fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.664+0000 7fe1b065e740 1 -- 172.21.15.47:0/3979970106 >> 172.21.15.47:0/3979970106 conn(0x55950de58c00 msgr2=0x55950de5be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.664+0000 7fe1b065e740 1 -- 172.21.15.47:0/3979970106 shutdown_connections 2024-03-21T11:39:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.664+0000 7fe1b065e740 1 -- 172.21.15.47:0/3979970106 wait complete. 2024-03-21T11:39:45.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.665+0000 7fe1b065e740 1 -- start start 2024-03-21T11:39:45.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.665+0000 7fe1b065e740 1 Processor -- start 2024-03-21T11:39:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.665+0000 7fe1b065e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5d810 0x55950de6a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.665+0000 7fe1b065e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5e9e0 0x55950de6aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.665+0000 7fe1b065e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5f3a0 0x55950de6daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.665+0000 7fe1b065e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81b1a50 con 0x55950de5d810 2024-03-21T11:39:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.665+0000 7fe1b065e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81b13f0 con 0x55950de5f3a0 2024-03-21T11:39:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.665+0000 7fe1b065e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81ae8f0 con 0x55950de5e9e0 2024-03-21T11:39:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.665+0000 7fe1b065e740 10 client.0 _ll_register_callbacks cb 0x7fe194014490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.666+0000 7fe1a6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5d810 0x55950de6a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.666+0000 7fe1a6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5e9e0 0x55950de6aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.666+0000 7fe1a6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5d810 0x55950de6a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45900/0 (socket says 172.21.15.47:45900) 2024-03-21T11:39:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.666+0000 7fe1a6000640 1 -- 172.21.15.47:0/2536275677 learned_addr learned my addr 172.21.15.47:0/2536275677 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:45.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.666+0000 7fe1a6000640 1 --2- 172.21.15.47:0/2536275677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5f3a0 0x55950de6daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.666+0000 7fe1a6a00640 1 -- 172.21.15.47:0/2536275677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5f3a0 msgr2=0x55950de6daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.666+0000 7fe1a6a00640 1 --2- 172.21.15.47:0/2536275677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5f3a0 0x55950de6daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.666+0000 7fe1a6a00640 1 -- 172.21.15.47:0/2536275677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5d810 msgr2=0x55950de6a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.666+0000 7fe1a6a00640 1 --2- 172.21.15.47:0/2536275677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5d810 0x55950de6a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.666+0000 7fe1a6a00640 1 -- 172.21.15.47:0/2536275677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55950de6e090 con 0x55950de5e9e0 2024-03-21T11:39:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.667+0000 7fe1a6a00640 1 --2- 172.21.15.47:0/2536275677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5e9e0 0x55950de6aaf0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x55950de75d30 tx=0x7fe19800b740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e36929c88ccc6f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.667+0000 7fe190c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.668+0000 7fe190c00640 1 -- 172.21.15.47:0/2536275677 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198019310 con 0x55950de5e9e0 2024-03-21T11:39:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.668+0000 7fe190c00640 1 -- 172.21.15.47:0/2536275677 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe19800bd90 con 0x55950de5e9e0 2024-03-21T11:39:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.668+0000 7fe1b065e740 1 -- 172.21.15.47:0/2536275677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe194005500 con 0x55950de5e9e0 2024-03-21T11:39:45.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.668+0000 7fe190c00640 1 -- 172.21.15.47:0/2536275677 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe198019880 con 0x55950de5e9e0 2024-03-21T11:39:45.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.669+0000 7fe1b065e740 10 client.30892 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:45.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.669+0000 7fe1b065e740 1 -- 172.21.15.47:0/2536275677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55950dedf9c0 con 0x55950de5e9e0 2024-03-21T11:39:45.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.669+0000 7fe190c00640 1 -- 172.21.15.47:0/2536275677 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe194005500 con 0x55950de5e9e0 2024-03-21T11:39:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.671+0000 7fe190c00640 1 -- 172.21.15.47:0/2536275677 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fe198019a20 con 0x55950de5e9e0 2024-03-21T11:39:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.671+0000 7fe1b065e740 10 client.30892 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.671+0000 7fe1b065e740 10 client.30892 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:39:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.671+0000 7fe1b065e740 20 client.30892 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.671+0000 7fe1b065e740 1 --2- 172.21.15.47:0/2536275677 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55950dee0100 0x55950dee24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.671+0000 7fe1b065e740 4 client.30892 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:39:45.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.671+0000 7fe1b065e740 1 -- 172.21.15.47:0/2536275677 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55950ddc9b50 con 0x55950dee0100 2024-03-21T11:39:45.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.672+0000 7fe1a6000640 1 --2- 172.21.15.47:0/2536275677 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55950dee0100 0x55950dee24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.673+0000 7fe1a6000640 1 --2- 172.21.15.47:0/2536275677 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55950dee0100 0x55950dee24f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df917e5793b90341 server_cookie=b72be9c96b5679d3 in_seq=0 out_seq=0 2024-03-21T11:39:45.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.673+0000 7fe190c00640 10 client.30892 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:39:45.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.674+0000 7fe190c00640 1 -- 172.21.15.47:0/2536275677 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55950ddc9b50 con 0x55950dee0100 2024-03-21T11:39:45.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.675+0000 7fe190c00640 10 client.30892 handle_command_reply: tid=0 2024-03-21T11:39:45.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.675+0000 7fe1b065e740 1 client.30892 shutdown 2024-03-21T11:39:45.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.675+0000 7fe1b065e740 1 -- 172.21.15.47:0/2536275677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5e9e0 msgr2=0x55950de6aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.675+0000 7fe1b065e740 1 --2- 172.21.15.47:0/2536275677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5e9e0 0x55950de6aaf0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x55950de75d30 tx=0x7fe19800b740 comp rx=0 tx=0).stop 2024-03-21T11:39:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.675+0000 7fe1b065e740 1 -- 172.21.15.47:0/2536275677 shutdown_connections 2024-03-21T11:39:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.675+0000 7fe1b065e740 1 --2- 172.21.15.47:0/2536275677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55950de5f3a0 0x55950de6daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.675+0000 7fe1b065e740 1 --2- 172.21.15.47:0/2536275677 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55950dee0100 0x55950dee24f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.675+0000 7fe1b065e740 1 --2- 172.21.15.47:0/2536275677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55950de5e9e0 0x55950de6aaf0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.675+0000 7fe1b065e740 1 --2- 172.21.15.47:0/2536275677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55950de5d810 0x55950de6a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.675+0000 7fe1b065e740 1 -- 172.21.15.47:0/2536275677 >> 172.21.15.47:0/2536275677 conn(0x55950de58c00 msgr2=0x55950de5c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.675+0000 7fe190c00640 0 client.30892 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:39:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.676+0000 7fe1b065e740 1 -- 172.21.15.47:0/2536275677 shutdown_connections 2024-03-21T11:39:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.676+0000 7fe1b065e740 1 -- 172.21.15.47:0/2536275677 wait complete. 2024-03-21T11:39:45.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.676+0000 7fe1b065e740 20 client.30892 trim_cache size 0 max 16384 2024-03-21T11:39:45.678 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:39:45.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.679+0000 7fe1ada00640 1 -- 172.21.15.47:0/2110250078 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe18c0a2fa0 msgr2=0x7fe18c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.679+0000 7fe1ada00640 1 --2- 172.21.15.47:0/2110250078 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe18c0a2fa0 0x7fe18c0a5460 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00be3e0 tx=0x7fe1a002bef0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.679+0000 7fe1ada00640 1 -- 172.21.15.47:0/2110250078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a8101030 msgr2=0x7fe1a81a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.679+0000 7fe1ada00640 1 --2- 172.21.15.47:0/2110250078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a8101030 0x7fe1a81a1190 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0036040 tx=0x7fe1a002cbc0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.679+0000 7fe1ada00640 1 -- 172.21.15.47:0/2110250078 shutdown_connections 2024-03-21T11:39:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.679+0000 7fe1ada00640 1 --2- 172.21.15.47:0/2110250078 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe18c0a2fa0 0x7fe18c0a5460 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.679+0000 7fe1ada00640 1 --2- 172.21.15.47:0/2110250078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1a8101030 0x7fe1a81a1190 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.679+0000 7fe1ada00640 1 --2- 172.21.15.47:0/2110250078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1a8100690 0x7fe1a819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.679+0000 7fe1ada00640 1 --2- 172.21.15.47:0/2110250078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1a80ff560 0x7fe1a8199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.679+0000 7fe1ada00640 1 -- 172.21.15.47:0/2110250078 >> 172.21.15.47:0/2110250078 conn(0x7fe1a80fae60 msgr2=0x7fe1a80fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.679+0000 7fe1ada00640 1 -- 172.21.15.47:0/2110250078 shutdown_connections 2024-03-21T11:39:45.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.679+0000 7fe1ada00640 1 -- 172.21.15.47:0/2110250078 wait complete. 2024-03-21T11:39:45.707 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:39:45.708 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:39:45.934 INFO:tasks.workunit.client.0.smithi047.stdout: 10 71651 24.50 MB/sec execute 29 sec latency 68.603 ms 2024-03-21T11:39:45.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.964+0000 7f17a1800640 1 -- 172.21.15.47:0/2372932867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c105250 msgr2=0x7f179c1056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.964+0000 7f17a1800640 1 --2- 172.21.15.47:0/2372932867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c105250 0x7f179c1056d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f1794009ca0 tx=0x7f179402c0f0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.965+0000 7f17a1800640 1 -- 172.21.15.47:0/2372932867 shutdown_connections 2024-03-21T11:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.965+0000 7f17a1800640 1 --2- 172.21.15.47:0/2372932867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c105250 0x7f179c1056d0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.965+0000 7f17a1800640 1 --2- 172.21.15.47:0/2372932867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c1028a0 0x7f179c104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.965+0000 7f17a1800640 1 --2- 172.21.15.47:0/2372932867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c0fff30 0x7f179c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.965+0000 7f17a1800640 1 -- 172.21.15.47:0/2372932867 >> 172.21.15.47:0/2372932867 conn(0x7f179c0fbb30 msgr2=0x7f179c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.966+0000 7f17a1800640 1 -- 172.21.15.47:0/2372932867 shutdown_connections 2024-03-21T11:39:45.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.966+0000 7f17a1800640 1 -- 172.21.15.47:0/2372932867 wait complete. 2024-03-21T11:39:45.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.967+0000 7f17a1800640 1 Processor -- start 2024-03-21T11:39:45.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.967+0000 7f17a1800640 1 -- start start 2024-03-21T11:39:45.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.968+0000 7f17a1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c0fff30 0x7f179c19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.968+0000 7f17a1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c1028a0 0x7f179c19e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.968+0000 7f17a1800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c105250 0x7f179c1a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.968+0000 7f17a1800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c113930 con 0x7f179c0fff30 2024-03-21T11:39:45.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.968+0000 7f17a1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1137b0 con 0x7f179c1028a0 2024-03-21T11:39:45.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.968+0000 7f17a1800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c113ab0 con 0x7f179c105250 2024-03-21T11:39:45.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.968+0000 7f179b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c105250 0x7f179c1a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.969+0000 7f179b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c105250 0x7f179c1a55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48338/0 (socket says 172.21.15.47:48338) 2024-03-21T11:39:45.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.969+0000 7f179b400640 1 -- 172.21.15.47:0/2696571079 learned_addr learned my addr 172.21.15.47:0/2696571079 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:45.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.969+0000 7f179a000640 1 --2- 172.21.15.47:0/2696571079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c1028a0 0x7f179c19e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.969+0000 7f179aa00640 1 --2- 172.21.15.47:0/2696571079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c0fff30 0x7f179c19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.969+0000 7f179b400640 1 -- 172.21.15.47:0/2696571079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c1028a0 msgr2=0x7f179c19e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.969+0000 7f179b400640 1 --2- 172.21.15.47:0/2696571079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c1028a0 0x7f179c19e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.969+0000 7f179b400640 1 -- 172.21.15.47:0/2696571079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c0fff30 msgr2=0x7f179c19e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.969+0000 7f179b400640 1 --2- 172.21.15.47:0/2696571079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c0fff30 0x7f179c19e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.969+0000 7f179b400640 1 -- 172.21.15.47:0/2696571079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c106680 con 0x7f179c105250 2024-03-21T11:39:45.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.970+0000 7f179b400640 1 --2- 172.21.15.47:0/2696571079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c105250 0x7f179c1a55b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1794037040 tx=0x7f1794032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.971+0000 7f1793400640 1 -- 172.21.15.47:0/2696571079 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794041020 con 0x7f179c105250 2024-03-21T11:39:45.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.971+0000 7f1793400640 1 -- 172.21.15.47:0/2696571079 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1794032340 con 0x7f179c105250 2024-03-21T11:39:45.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.971+0000 7f1793400640 1 -- 172.21.15.47:0/2696571079 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794039040 con 0x7f179c105250 2024-03-21T11:39:45.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.971+0000 7f17a1800640 1 -- 172.21.15.47:0/2696571079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179c106910 con 0x7f179c105250 2024-03-21T11:39:45.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.971+0000 7f17a1800640 1 -- 172.21.15.47:0/2696571079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179c106db0 con 0x7f179c105250 2024-03-21T11:39:45.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.975+0000 7f1793400640 1 -- 172.21.15.47:0/2696571079 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1794044020 con 0x7f179c105250 2024-03-21T11:39:45.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.976+0000 7f17a43fb740 1 Processor -- start 2024-03-21T11:39:45.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.976+0000 7f17a43fb740 1 -- start start 2024-03-21T11:39:45.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.976+0000 7f17a43fb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df8810 0x557162df8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.976+0000 7f17a43fb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162df99e0 0x557162df9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.976+0000 7f17a43fb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162dfa3a0 0x557162e00c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.976+0000 7f17a43fb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557162d5d2c0 con 0x557162dfa3a0 2024-03-21T11:39:45.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.976+0000 7f17a43fb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557162d05c90 con 0x557162df8810 2024-03-21T11:39:45.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.976+0000 7f17a43fb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557162d59360 con 0x557162df99e0 2024-03-21T11:39:45.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.977+0000 7f179aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162df99e0 0x557162df9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.977+0000 7f179aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162df99e0 0x557162df9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48350/0 (socket says 172.21.15.47:48350) 2024-03-21T11:39:45.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.977+0000 7f179aa00640 1 -- 172.21.15.47:0/1134146807 learned_addr learned my addr 172.21.15.47:0/1134146807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:45.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.977+0000 7f1793400640 1 --2- 172.21.15.47:0/2696571079 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17800a3070 0x7f17800a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.977+0000 7f179a000640 1 --2- 172.21.15.47:0/1134146807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df8810 0x557162df8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.977+0000 7f1793400640 1 -- 172.21.15.47:0/2696571079 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f17940f1600 con 0x7f179c105250 2024-03-21T11:39:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.978+0000 7f179aa00640 1 -- 172.21.15.47:0/1134146807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df8810 msgr2=0x557162df8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.978+0000 7f179aa00640 1 --2- 172.21.15.47:0/1134146807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df8810 0x557162df8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.978+0000 7f179aa00640 1 -- 172.21.15.47:0/1134146807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162dfa3a0 msgr2=0x557162e00c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.978+0000 7f179aa00640 1 --2- 172.21.15.47:0/1134146807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162dfa3a0 0x557162e00c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.978+0000 7f179aa00640 1 -- 172.21.15.47:0/1134146807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557162d2a080 con 0x557162df99e0 2024-03-21T11:39:45.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.978+0000 7f179b400640 1 --2- 172.21.15.47:0/2696571079 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17800a3070 0x7f17800a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.978+0000 7f179aa00640 1 --2- 172.21.15.47:0/1134146807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162df99e0 0x557162df9e60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f178c009c80 tx=0x7f178c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1af07dfada30cad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.979+0000 7f1787e00640 1 -- 172.21.15.47:0/1134146807 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c012450 con 0x557162df99e0 2024-03-21T11:39:45.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.979+0000 7f1787e00640 1 -- 172.21.15.47:0/1134146807 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f178c012ad0 con 0x557162df99e0 2024-03-21T11:39:45.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.979+0000 7f1787e00640 1 -- 172.21.15.47:0/1134146807 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c00fc00 con 0x557162df99e0 2024-03-21T11:39:45.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.979+0000 7f17a43fb740 1 -- 172.21.15.47:0/1134146807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162df99e0 msgr2=0x557162df9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.979+0000 7f179b400640 1 --2- 172.21.15.47:0/2696571079 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17800a3070 0x7f17800a5530 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f17940ba310 tx=0x7f1794009a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.979+0000 7f17a43fb740 1 --2- 172.21.15.47:0/1134146807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162df99e0 0x557162df9e60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f178c009c80 tx=0x7f178c0117b0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.980+0000 7f17a43fb740 1 -- 172.21.15.47:0/1134146807 shutdown_connections 2024-03-21T11:39:45.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.980+0000 7f17a43fb740 1 --2- 172.21.15.47:0/1134146807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162dfa3a0 0x557162e00c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.980+0000 7f17a43fb740 1 --2- 172.21.15.47:0/1134146807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162df99e0 0x557162df9e60 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.980+0000 7f17a43fb740 1 --2- 172.21.15.47:0/1134146807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df8810 0x557162df8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.980+0000 7f17a43fb740 1 -- 172.21.15.47:0/1134146807 >> 172.21.15.47:0/1134146807 conn(0x557162df3c00 msgr2=0x557162df4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.980+0000 7f17a43fb740 1 -- 172.21.15.47:0/1134146807 shutdown_connections 2024-03-21T11:39:45.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.980+0000 7f17a43fb740 1 -- 172.21.15.47:0/1134146807 wait complete. 2024-03-21T11:39:45.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.980+0000 7f17a43fb740 1 -- start start 2024-03-21T11:39:45.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.980+0000 7f17a43fb740 1 Processor -- start 2024-03-21T11:39:45.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f17a43fb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162df8810 0x557162e02ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f17a43fb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df99e0 0x557162e03010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f17a43fb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162dfa3a0 0x557162e0a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f17a43fb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557162e01400 con 0x557162df99e0 2024-03-21T11:39:45.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f17a43fb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557162d190d0 con 0x557162dfa3a0 2024-03-21T11:39:45.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f17a43fb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557162e01580 con 0x557162df8810 2024-03-21T11:39:45.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f17a43fb740 10 client.0 _ll_register_callbacks cb 0x557162d1de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:45.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f179a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162df8810 0x557162e02ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f179a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162df8810 0x557162e02ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48354/0 (socket says 172.21.15.47:48354) 2024-03-21T11:39:45.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f179a000640 1 -- 172.21.15.47:0/4216906151 learned_addr learned my addr 172.21.15.47:0/4216906151 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:45.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f179a000640 1 --2- 172.21.15.47:0/4216906151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162dfa3a0 0x557162e0a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f179a000640 1 -- 172.21.15.47:0/4216906151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162dfa3a0 msgr2=0x557162e0a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f179a000640 1 --2- 172.21.15.47:0/4216906151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162dfa3a0 0x557162e0a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f179a000640 1 -- 172.21.15.47:0/4216906151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df99e0 msgr2=0x557162e03010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:45.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f179a000640 1 --2- 172.21.15.47:0/4216906151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df99e0 0x557162e03010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.981+0000 7f179a000640 1 -- 172.21.15.47:0/4216906151 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557162a713f0 con 0x557162df8810 2024-03-21T11:39:45.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.982+0000 7f179a000640 1 --2- 172.21.15.47:0/4216906151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162df8810 0x557162e02ad0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f1788012c30 tx=0x7f1788009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d56b0b0977b6efa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:45.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.982+0000 7f1784c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:45.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.982+0000 7f1784c00640 1 -- 172.21.15.47:0/4216906151 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788004020 con 0x557162df8810 2024-03-21T11:39:45.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.982+0000 7f1784c00640 1 -- 172.21.15.47:0/4216906151 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f17880041c0 con 0x557162df8810 2024-03-21T11:39:45.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.982+0000 7f17a43fb740 1 -- 172.21.15.47:0/4216906151 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f178c012ad0 con 0x557162df8810 2024-03-21T11:39:45.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.983+0000 7f1784c00640 1 -- 172.21.15.47:0/4216906151 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17880043c0 con 0x557162df8810 2024-03-21T11:39:45.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.983+0000 7f17a43fb740 10 client.30904 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:45.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.983+0000 7f17a43fb740 1 -- 172.21.15.47:0/4216906151 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x557162b3b7d0 con 0x557162df8810 2024-03-21T11:39:45.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.984+0000 7f1784c00640 1 -- 172.21.15.47:0/4216906151 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f178c012ad0 con 0x557162df8810 2024-03-21T11:39:45.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.984+0000 7f1784c00640 1 -- 172.21.15.47:0/4216906151 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f17880205b0 con 0x557162df8810 2024-03-21T11:39:45.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.984+0000 7f17a43fb740 10 client.30904 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:45.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.984+0000 7f17a43fb740 10 client.30904 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:39:45.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.984+0000 7f17a43fb740 20 client.30904 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:45.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.984+0000 7f17a43fb740 1 --2- 172.21.15.47:0/4216906151 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557162e0e5a0 0x557162e10a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:45.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.984+0000 7f17a43fb740 4 client.30904 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.984+0000 7f17a43fb740 1 -- 172.21.15.47:0/4216906151 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557162d6a800 con 0x557162e0e5a0 2024-03-21T11:39:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.984+0000 7f179aa00640 1 --2- 172.21.15.47:0/4216906151 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557162e0e5a0 0x557162e10a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.985+0000 7f179aa00640 1 --2- 172.21.15.47:0/4216906151 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557162e0e5a0 0x557162e10a40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=972e00083d54cfd server_cookie=dec96b6c783507e0 in_seq=0 out_seq=0 2024-03-21T11:39:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.985+0000 7f1784c00640 10 client.30904 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:39:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.986+0000 7f1784c00640 1 -- 172.21.15.47:0/4216906151 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557162d6a800 con 0x557162e0e5a0 2024-03-21T11:39:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.986+0000 7f1784c00640 10 client.30904 handle_command_reply: tid=0 2024-03-21T11:39:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.986+0000 7f17a43fb740 1 client.30904 shutdown 2024-03-21T11:39:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.987+0000 7f17a43fb740 1 -- 172.21.15.47:0/4216906151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162df8810 msgr2=0x557162e02ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.987+0000 7f17a43fb740 1 --2- 172.21.15.47:0/4216906151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162df8810 0x557162e02ad0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f1788012c30 tx=0x7f1788009800 comp rx=0 tx=0).stop 2024-03-21T11:39:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.987+0000 7f17a43fb740 1 -- 172.21.15.47:0/4216906151 shutdown_connections 2024-03-21T11:39:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.987+0000 7f17a43fb740 1 --2- 172.21.15.47:0/4216906151 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557162e0e5a0 0x557162e10a40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.987+0000 7f17a43fb740 1 --2- 172.21.15.47:0/4216906151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162dfa3a0 0x557162e0a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.987+0000 7f17a43fb740 1 --2- 172.21.15.47:0/4216906151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df99e0 0x557162e03010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.987+0000 7f1784c00640 0 client.30904 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:39:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.987+0000 7f17a43fb740 1 --2- 172.21.15.47:0/4216906151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162df8810 0x557162e02ad0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.987+0000 7f17a43fb740 1 -- 172.21.15.47:0/4216906151 >> 172.21.15.47:0/4216906151 conn(0x557162df3c00 msgr2=0x557162df2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.987+0000 7f17a43fb740 1 -- 172.21.15.47:0/4216906151 shutdown_connections 2024-03-21T11:39:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.987+0000 7f17a43fb740 1 -- 172.21.15.47:0/4216906151 wait complete. 2024-03-21T11:39:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:45.987+0000 7f17a43fb740 20 client.30904 trim_cache size 0 max 16384 2024-03-21T11:39:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.005+0000 7f17a43fb740 1 Processor -- start 2024-03-21T11:39:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.006+0000 7f17a43fb740 1 -- start start 2024-03-21T11:39:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.006+0000 7f17a43fb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df8810 0x557162dfca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.006+0000 7f17a43fb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df99e0 0x557162e0a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.006+0000 7f17a43fb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162dfa3a0 0x557162dfcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.006+0000 7f17a43fb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1b8f00 con 0x557162df8810 2024-03-21T11:39:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.006+0000 7f17a43fb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1ba910 con 0x557162df99e0 2024-03-21T11:39:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.006+0000 7f17a43fb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1b9560 con 0x557162dfa3a0 2024-03-21T11:39:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.006+0000 7f179aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df99e0 0x557162e0a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.007+0000 7f179aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df99e0 0x557162e0a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60494/0 (socket says 172.21.15.47:60494) 2024-03-21T11:39:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.007+0000 7f179aa00640 1 -- 172.21.15.47:0/1820486017 learned_addr learned my addr 172.21.15.47:0/1820486017 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.007+0000 7f179a000640 1 --2- 172.21.15.47:0/1820486017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df8810 0x557162dfca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.007+0000 7f179a000640 1 --2- 172.21.15.47:0/1820486017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162dfa3a0 0x557162dfcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.007+0000 7f179aa00640 1 -- 172.21.15.47:0/1820486017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162dfa3a0 msgr2=0x557162dfcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.007+0000 7f179aa00640 1 --2- 172.21.15.47:0/1820486017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162dfa3a0 0x557162dfcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.007+0000 7f179aa00640 1 -- 172.21.15.47:0/1820486017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df8810 msgr2=0x557162dfca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.007+0000 7f179aa00640 1 --2- 172.21.15.47:0/1820486017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df8810 0x557162dfca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.007+0000 7f179aa00640 1 -- 172.21.15.47:0/1820486017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557162df4970 con 0x557162df99e0 2024-03-21T11:39:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.008+0000 7f179aa00640 1 --2- 172.21.15.47:0/1820486017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df99e0 0x557162e0a010 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f178c011cb0 tx=0x7f178c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=672960bcfe785fb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.008+0000 7f1787400640 1 -- 172.21.15.47:0/1820486017 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c02f070 con 0x557162df99e0 2024-03-21T11:39:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.008+0000 7f1787400640 1 -- 172.21.15.47:0/1820486017 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f178c005590 con 0x557162df99e0 2024-03-21T11:39:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.008+0000 7f1787400640 1 -- 172.21.15.47:0/1820486017 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c029c70 con 0x557162df99e0 2024-03-21T11:39:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.009+0000 7f17a43fb740 1 -- 172.21.15.47:0/1820486017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df99e0 msgr2=0x557162e0a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.009+0000 7f17a43fb740 1 --2- 172.21.15.47:0/1820486017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df99e0 0x557162e0a010 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f178c011cb0 tx=0x7f178c005f90 comp rx=0 tx=0).stop 2024-03-21T11:39:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.009+0000 7f17a43fb740 1 -- 172.21.15.47:0/1820486017 shutdown_connections 2024-03-21T11:39:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.009+0000 7f17a43fb740 1 --2- 172.21.15.47:0/1820486017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162dfa3a0 0x557162dfcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.009+0000 7f17a43fb740 1 --2- 172.21.15.47:0/1820486017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df99e0 0x557162e0a010 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.009+0000 7f17a43fb740 1 --2- 172.21.15.47:0/1820486017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df8810 0x557162dfca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.009+0000 7f17a43fb740 1 -- 172.21.15.47:0/1820486017 >> 172.21.15.47:0/1820486017 conn(0x557162df3c00 msgr2=0x557162df76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.009+0000 7f17a43fb740 1 -- 172.21.15.47:0/1820486017 shutdown_connections 2024-03-21T11:39:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.009+0000 7f17a43fb740 1 -- 172.21.15.47:0/1820486017 wait complete. 2024-03-21T11:39:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.010+0000 7f17a43fb740 1 -- start start 2024-03-21T11:39:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.010+0000 7f17a43fb740 1 Processor -- start 2024-03-21T11:39:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.010+0000 7f17a43fb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df8810 0x557162e09c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.010+0000 7f17a43fb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df99e0 0x557162e066f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.010+0000 7f17a43fb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162dfa3a0 0x557162e06c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.010+0000 7f17a43fb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1ba220 con 0x557162df8810 2024-03-21T11:39:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.010+0000 7f17a43fb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1b9bc0 con 0x557162df99e0 2024-03-21T11:39:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.010+0000 7f17a43fb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c10ea10 con 0x557162dfa3a0 2024-03-21T11:39:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.010+0000 7f17a43fb740 10 client.0 _ll_register_callbacks cb 0x7f1788009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.010+0000 7f179aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df99e0 0x557162e066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.010+0000 7f179aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df99e0 0x557162e066f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60508/0 (socket says 172.21.15.47:60508) 2024-03-21T11:39:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.010+0000 7f179aa00640 1 -- 172.21.15.47:0/4065628515 learned_addr learned my addr 172.21.15.47:0/4065628515 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.011+0000 7f179a000640 1 --2- 172.21.15.47:0/4065628515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df8810 0x557162e09c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.011+0000 7f179a000640 1 --2- 172.21.15.47:0/4065628515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162dfa3a0 0x557162e06c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.011+0000 7f179aa00640 1 -- 172.21.15.47:0/4065628515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162dfa3a0 msgr2=0x557162e06c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:46.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.011+0000 7f179aa00640 1 --2- 172.21.15.47:0/4065628515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162dfa3a0 0x557162e06c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.011+0000 7f179aa00640 1 -- 172.21.15.47:0/4065628515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df8810 msgr2=0x557162e09c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:46.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.011+0000 7f179aa00640 1 --2- 172.21.15.47:0/4065628515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df8810 0x557162e09c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.011+0000 7f179aa00640 1 -- 172.21.15.47:0/4065628515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557162e0a280 con 0x557162df99e0 2024-03-21T11:39:46.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.011+0000 7f179a000640 1 --2- 172.21.15.47:0/4065628515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df8810 0x557162e09c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:39:46.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.011+0000 7f179aa00640 1 --2- 172.21.15.47:0/4065628515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df99e0 0x557162e066f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f178c005930 tx=0x7f178c029290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61ca3c07412e9b93 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:46.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.011+0000 7f1784c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:46.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.011+0000 7f1784c00640 1 -- 172.21.15.47:0/4065628515 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c02f070 con 0x557162df99e0 2024-03-21T11:39:46.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.011+0000 7f1784c00640 1 -- 172.21.15.47:0/4065628515 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f178c0052b0 con 0x557162df99e0 2024-03-21T11:39:46.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.012+0000 7f17a43fb740 1 -- 172.21.15.47:0/4065628515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f178c005590 con 0x557162df99e0 2024-03-21T11:39:46.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.012+0000 7f1784c00640 1 -- 172.21.15.47:0/4065628515 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f178c009e30 con 0x557162df99e0 2024-03-21T11:39:46.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.012+0000 7f17a43fb740 10 client.30914 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:46.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.012+0000 7f17a43fb740 1 -- 172.21.15.47:0/4065628515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x557162e6add0 con 0x557162df99e0 2024-03-21T11:39:46.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.012+0000 7f1784c00640 1 -- 172.21.15.47:0/4065628515 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f178c005590 con 0x557162df99e0 2024-03-21T11:39:46.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.012+0000 7f1784c00640 1 -- 172.21.15.47:0/4065628515 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f178c003ca0 con 0x557162df99e0 2024-03-21T11:39:46.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.013+0000 7f17a43fb740 10 client.30914 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:46.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.013+0000 7f17a43fb740 10 client.30914 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:39:46.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.013+0000 7f17a43fb740 20 client.30914 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:46.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.013+0000 7f17a43fb740 1 --2- 172.21.15.47:0/4065628515 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557162e6b560 0x557162e6d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.013+0000 7f17a43fb740 4 client.30914 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:39:46.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.013+0000 7f17a43fb740 1 -- 172.21.15.47:0/4065628515 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557162d64b50 con 0x557162e6b560 2024-03-21T11:39:46.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.013+0000 7f179a000640 1 --2- 172.21.15.47:0/4065628515 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557162e6b560 0x557162e6d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.013+0000 7f179a000640 1 --2- 172.21.15.47:0/4065628515 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557162e6b560 0x557162e6d980 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc39c1daf6e7e891 server_cookie=b3659696576f5036 in_seq=0 out_seq=0 2024-03-21T11:39:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.013+0000 7f1784c00640 10 client.30914 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:39:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.014+0000 7f1784c00640 1 -- 172.21.15.47:0/4065628515 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557162d64b50 con 0x557162e6b560 2024-03-21T11:39:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.014+0000 7f1784c00640 10 client.30914 handle_command_reply: tid=0 2024-03-21T11:39:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.014+0000 7f17a43fb740 1 client.30914 shutdown 2024-03-21T11:39:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.014+0000 7f17a43fb740 1 -- 172.21.15.47:0/4065628515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df99e0 msgr2=0x557162e066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.014+0000 7f17a43fb740 1 --2- 172.21.15.47:0/4065628515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df99e0 0x557162e066f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f178c005930 tx=0x7f178c029290 comp rx=0 tx=0).stop 2024-03-21T11:39:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.014+0000 7f17a43fb740 1 -- 172.21.15.47:0/4065628515 shutdown_connections 2024-03-21T11:39:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.014+0000 7f17a43fb740 1 --2- 172.21.15.47:0/4065628515 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557162e6b560 0x557162e6d980 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.014+0000 7f17a43fb740 1 --2- 172.21.15.47:0/4065628515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557162dfa3a0 0x557162e06c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.014+0000 7f17a43fb740 1 --2- 172.21.15.47:0/4065628515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557162df99e0 0x557162e066f0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.014+0000 7f17a43fb740 1 --2- 172.21.15.47:0/4065628515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557162df8810 0x557162e09c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.014+0000 7f1784c00640 0 client.30914 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:39:46.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.014+0000 7f17a43fb740 1 -- 172.21.15.47:0/4065628515 >> 172.21.15.47:0/4065628515 conn(0x557162df3c00 msgr2=0x557162df2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:46.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.014+0000 7f17a43fb740 1 -- 172.21.15.47:0/4065628515 shutdown_connections 2024-03-21T11:39:46.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.014+0000 7f17a43fb740 1 -- 172.21.15.47:0/4065628515 wait complete. 2024-03-21T11:39:46.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.015+0000 7f17a43fb740 20 client.30914 trim_cache size 0 max 16384 2024-03-21T11:39:46.020 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:39:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.015+0000 7f17a1800640 1 -- 172.21.15.47:0/2696571079 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17800a3070 msgr2=0x7f17800a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.015+0000 7f17a1800640 1 --2- 172.21.15.47:0/2696571079 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17800a3070 0x7f17800a5530 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f17940ba310 tx=0x7f1794009a20 comp rx=0 tx=0).stop 2024-03-21T11:39:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.015+0000 7f17a1800640 1 -- 172.21.15.47:0/2696571079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c105250 msgr2=0x7f179c1a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.015+0000 7f17a1800640 1 --2- 172.21.15.47:0/2696571079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c105250 0x7f179c1a55b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1794037040 tx=0x7f1794032ea0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.016+0000 7f17a1800640 1 -- 172.21.15.47:0/2696571079 shutdown_connections 2024-03-21T11:39:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.016+0000 7f17a1800640 1 --2- 172.21.15.47:0/2696571079 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17800a3070 0x7f17800a5530 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.016+0000 7f17a1800640 1 --2- 172.21.15.47:0/2696571079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f179c105250 0x7f179c1a55b0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.016+0000 7f17a1800640 1 --2- 172.21.15.47:0/2696571079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c1028a0 0x7f179c19e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.016+0000 7f17a1800640 1 --2- 172.21.15.47:0/2696571079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c0fff30 0x7f179c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.016+0000 7f17a1800640 1 -- 172.21.15.47:0/2696571079 >> 172.21.15.47:0/2696571079 conn(0x7f179c0fbb30 msgr2=0x7f179c1008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.016+0000 7f17a1800640 1 -- 172.21.15.47:0/2696571079 shutdown_connections 2024-03-21T11:39:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.016+0000 7f17a1800640 1 -- 172.21.15.47:0/2696571079 wait complete. 2024-03-21T11:39:46.040 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:39:46.040 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:39:46.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.279+0000 7f4cadc00640 1 -- 172.21.15.47:0/1676459348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ca8104990 msgr2=0x7f4ca8104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:46.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.279+0000 7f4cadc00640 1 --2- 172.21.15.47:0/1676459348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ca8104990 0x7f4ca8104e10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009900 tx=0x7f4c9402bc40 comp rx=0 tx=0).stop 2024-03-21T11:39:46.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.279+0000 7f4cadc00640 1 -- 172.21.15.47:0/1676459348 shutdown_connections 2024-03-21T11:39:46.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.279+0000 7f4cadc00640 1 --2- 172.21.15.47:0/1676459348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca8105350 0x7f4ca810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.279+0000 7f4cadc00640 1 --2- 172.21.15.47:0/1676459348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ca8104990 0x7f4ca8104e10 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.279+0000 7f4cadc00640 1 --2- 172.21.15.47:0/1676459348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca81037a0 0x7f4ca8103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.279+0000 7f4cadc00640 1 -- 172.21.15.47:0/1676459348 >> 172.21.15.47:0/1676459348 conn(0x7f4ca80fed10 msgr2=0x7f4ca8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:46.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.281+0000 7f4cadc00640 1 -- 172.21.15.47:0/1676459348 shutdown_connections 2024-03-21T11:39:46.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.282+0000 7f4cadc00640 1 -- 172.21.15.47:0/1676459348 wait complete. 2024-03-21T11:39:46.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.283+0000 7f4cadc00640 1 Processor -- start 2024-03-21T11:39:46.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.283+0000 7f4cadc00640 1 -- start start 2024-03-21T11:39:46.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.284+0000 7f4cadc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca81037a0 0x7f4ca8199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.284+0000 7f4cadc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ca8104990 0x7f4ca8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.284+0000 7f4cadc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca8105350 0x7f4ca81a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.284+0000 7f4cadc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca810f450 con 0x7f4ca8105350 2024-03-21T11:39:46.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.284+0000 7f4cadc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca810f2d0 con 0x7f4ca81037a0 2024-03-21T11:39:46.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.284+0000 7f4cadc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca810f5d0 con 0x7f4ca8104990 2024-03-21T11:39:46.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.284+0000 7f4ca7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca81037a0 0x7f4ca8199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.284+0000 7f4ca7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ca8104990 0x7f4ca8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.284+0000 7f4ca7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ca8104990 0x7f4ca8199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48390/0 (socket says 172.21.15.47:48390) 2024-03-21T11:39:46.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.284+0000 7f4ca7400640 1 -- 172.21.15.47:0/1080566698 learned_addr learned my addr 172.21.15.47:0/1080566698 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:46.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.284+0000 7f4cad200640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca8105350 0x7f4ca81a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.285+0000 7f4ca7400640 1 -- 172.21.15.47:0/1080566698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca81037a0 msgr2=0x7f4ca8199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:46.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.286+0000 7f4ca7400640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca81037a0 0x7f4ca8199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.286+0000 7f4ca7400640 1 -- 172.21.15.47:0/1080566698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca8105350 msgr2=0x7f4ca81a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:46.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.286+0000 7f4ca7400640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca8105350 0x7f4ca81a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.286+0000 7f4ca7400640 1 -- 172.21.15.47:0/1080566698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca81a1680 con 0x7f4ca8104990 2024-03-21T11:39:46.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.286+0000 7f4cad200640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca8105350 0x7f4ca81a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:46.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.286+0000 7f4ca7e00640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca81037a0 0x7f4ca8199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:46.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.286+0000 7f4ca7400640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ca8104990 0x7f4ca8199fa0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f4c9402c8f0 tx=0x7f4c9402c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:46.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.286+0000 7f4ca4c00640 1 -- 172.21.15.47:0/1080566698 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94041070 con 0x7f4ca8104990 2024-03-21T11:39:46.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.286+0000 7f4ca4c00640 1 -- 172.21.15.47:0/1080566698 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c94004450 con 0x7f4ca8104990 2024-03-21T11:39:46.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.286+0000 7f4cadc00640 1 -- 172.21.15.47:0/1080566698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca81a1910 con 0x7f4ca8104990 2024-03-21T11:39:46.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.286+0000 7f4ca4c00640 1 -- 172.21.15.47:0/1080566698 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94034440 con 0x7f4ca8104990 2024-03-21T11:39:46.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.287+0000 7f4cadc00640 1 -- 172.21.15.47:0/1080566698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca81a1e50 con 0x7f4ca8104990 2024-03-21T11:39:46.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.291+0000 7f4ca4c00640 1 -- 172.21.15.47:0/1080566698 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4c94044070 con 0x7f4ca8104990 2024-03-21T11:39:46.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.291+0000 7f4cb1fdd740 1 Processor -- start 2024-03-21T11:39:46.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.291+0000 7f4cb1fdd740 1 -- start start 2024-03-21T11:39:46.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.292+0000 7f4cb1fdd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c6200 0x55d3bb8c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.292+0000 7f4cb1fdd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.292+0000 7f4cb1fdd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.292+0000 7f4cb1fdd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3bb8222c0 con 0x55d3bb8c73d0 2024-03-21T11:39:46.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.292+0000 7f4cb1fdd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3bb7cac90 con 0x55d3bb8c6200 2024-03-21T11:39:46.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.292+0000 7f4cb1fdd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3bb81e360 con 0x55d3bb8c7d90 2024-03-21T11:39:46.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.292+0000 7f4ca4c00640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c8c0a2fa0 0x7f4c8c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.292+0000 7f4ca4c00640 1 -- 172.21.15.47:0/1080566698 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4c940f12c0 con 0x7f4ca8104990 2024-03-21T11:39:46.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.293+0000 7f4cad200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c6200 0x55d3bb8c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.293+0000 7f4ca7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.293+0000 7f4ca7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48406/0 (socket says 172.21.15.47:48406) 2024-03-21T11:39:46.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.293+0000 7f4ca7400640 1 -- 172.21.15.47:0/348923023 learned_addr learned my addr 172.21.15.47:0/348923023 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:46.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.293+0000 7f4ca7e00640 1 --2- 172.21.15.47:0/348923023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.293+0000 7f4cad200640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c8c0a2fa0 0x7f4c8c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.294+0000 7f4ca7e00640 1 -- 172.21.15.47:0/348923023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c7d90 msgr2=0x55d3bb8ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:46.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.294+0000 7f4ca7e00640 1 --2- 172.21.15.47:0/348923023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.294+0000 7f4ca7e00640 1 -- 172.21.15.47:0/348923023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c6200 msgr2=0x55d3bb8c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:46.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.294+0000 7f4ca7e00640 1 --2- 172.21.15.47:0/348923023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c6200 0x55d3bb8c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.294+0000 7f4ca7e00640 1 -- 172.21.15.47:0/348923023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3bb7ef080 con 0x55d3bb8c73d0 2024-03-21T11:39:46.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.294+0000 7f4cad200640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c8c0a2fa0 0x7f4c8c0a5460 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009fd0 tx=0x7f4ca0009e00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:46.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.295+0000 7f4ca7e00640 1 --2- 172.21.15.47:0/348923023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8c7850 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009c80 tx=0x7f4c98010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab25dd53d962ce9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:46.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.295+0000 7f4c9d600640 1 -- 172.21.15.47:0/348923023 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98005020 con 0x55d3bb8c73d0 2024-03-21T11:39:46.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.295+0000 7f4c9d600640 1 -- 172.21.15.47:0/348923023 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c980047a0 con 0x55d3bb8c73d0 2024-03-21T11:39:46.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.295+0000 7f4c9d600640 1 -- 172.21.15.47:0/348923023 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980126b0 con 0x55d3bb8c73d0 2024-03-21T11:39:46.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.296+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/348923023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c73d0 msgr2=0x55d3bb8c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:46.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.296+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/348923023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8c7850 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009c80 tx=0x7f4c98010740 comp rx=0 tx=0).stop 2024-03-21T11:39:46.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.296+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/348923023 shutdown_connections 2024-03-21T11:39:46.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.296+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/348923023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.296+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/348923023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8c7850 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.296+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/348923023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c6200 0x55d3bb8c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.296+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/348923023 >> 172.21.15.47:0/348923023 conn(0x55d3bb8c15f0 msgr2=0x55d3bb8c19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:46.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.296+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/348923023 shutdown_connections 2024-03-21T11:39:46.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.296+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/348923023 wait complete. 2024-03-21T11:39:46.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.297+0000 7f4cb1fdd740 1 -- start start 2024-03-21T11:39:46.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.297+0000 7f4cb1fdd740 1 Processor -- start 2024-03-21T11:39:46.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.298+0000 7f4cb1fdd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c6200 0x55d3bb8d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.298+0000 7f4cb1fdd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.298+0000 7f4cb1fdd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8d79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.298+0000 7f4cb1fdd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3bb8ceda0 con 0x55d3bb8c7d90 2024-03-21T11:39:46.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.298+0000 7f4cb1fdd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3bb7de0d0 con 0x55d3bb8c73d0 2024-03-21T11:39:46.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.298+0000 7f4cb1fdd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3bb8cef20 con 0x55d3bb8c6200 2024-03-21T11:39:46.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.298+0000 7f4cb1fdd740 10 client.0 _ll_register_callbacks cb 0x55d3bb7e2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:46.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.298+0000 7f4ca7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.298+0000 7f4ca7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8d09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60544/0 (socket says 172.21.15.47:60544) 2024-03-21T11:39:46.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.298+0000 7f4ca7400640 1 -- 172.21.15.47:0/1415463969 learned_addr learned my addr 172.21.15.47:0/1415463969 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:46.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.298+0000 7f4cad200640 1 --2- 172.21.15.47:0/1415463969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8d79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.298+0000 7f4ca7e00640 1 --2- 172.21.15.47:0/1415463969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c6200 0x55d3bb8d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.299+0000 7f4ca7e00640 1 -- 172.21.15.47:0/1415463969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c73d0 msgr2=0x55d3bb8d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.299+0000 7f4ca7e00640 1 --2- 172.21.15.47:0/1415463969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8d09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.299+0000 7f4ca7e00640 1 -- 172.21.15.47:0/1415463969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c7d90 msgr2=0x55d3bb8d79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.299+0000 7f4ca7e00640 1 --2- 172.21.15.47:0/1415463969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8d79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.299+0000 7f4ca7e00640 1 -- 172.21.15.47:0/1415463969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3bb5363f0 con 0x55d3bb8c6200 2024-03-21T11:39:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.299+0000 7f4ca7400640 1 --2- 172.21.15.47:0/1415463969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.299+0000 7f4ca7e00640 1 --2- 172.21.15.47:0/1415463969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c6200 0x55d3bb8d0470 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f4c98000c00 tx=0x7f4c98009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e53e8ce4c63164d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.299+0000 7f4c92000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.300+0000 7f4c92000640 1 -- 172.21.15.47:0/1415463969 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9802a060 con 0x55d3bb8c6200 2024-03-21T11:39:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.300+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1415463969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c980047a0 con 0x55d3bb8c6200 2024-03-21T11:39:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.300+0000 7f4c92000640 1 -- 172.21.15.47:0/1415463969 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4c98005430 con 0x55d3bb8c6200 2024-03-21T11:39:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.300+0000 7f4c92000640 1 -- 172.21.15.47:0/1415463969 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c98031050 con 0x55d3bb8c6200 2024-03-21T11:39:46.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.300+0000 7f4cb1fdd740 10 client.30922 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:46.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.300+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1415463969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55d3bb6007d0 con 0x55d3bb8c6200 2024-03-21T11:39:46.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.301+0000 7f4c92000640 1 -- 172.21.15.47:0/1415463969 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c980047a0 con 0x55d3bb8c6200 2024-03-21T11:39:46.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.301+0000 7f4c92000640 1 -- 172.21.15.47:0/1415463969 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f4c9800f040 con 0x55d3bb8c6200 2024-03-21T11:39:46.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.301+0000 7f4cb1fdd740 10 client.30922 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:46.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.301+0000 7f4cb1fdd740 10 client.30922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:46.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.301+0000 7f4cb1fdd740 20 client.30922 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:46.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.301+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/1415463969 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3bb8ecbb0 0x55d3bb8ef020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.301+0000 7f4cb1fdd740 4 client.30922 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:46.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.301+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1415463969 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3bb82f800 con 0x55d3bb8ecbb0 2024-03-21T11:39:46.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.301+0000 7f4ca7400640 1 --2- 172.21.15.47:0/1415463969 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3bb8ecbb0 0x55d3bb8ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.302+0000 7f4ca7400640 1 --2- 172.21.15.47:0/1415463969 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3bb8ecbb0 0x55d3bb8ef020 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9ddbcd031bb1835 server_cookie=b60d01574774f1b9 in_seq=0 out_seq=0 2024-03-21T11:39:46.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.302+0000 7f4c92000640 10 client.30922 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:46.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.303+0000 7f4c92000640 1 -- 172.21.15.47:0/1415463969 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d3bb82f800 con 0x55d3bb8ecbb0 2024-03-21T11:39:46.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.303+0000 7f4c92000640 10 client.30922 handle_command_reply: tid=0 2024-03-21T11:39:46.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.303+0000 7f4cb1fdd740 1 client.30922 shutdown 2024-03-21T11:39:46.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.304+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1415463969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c6200 msgr2=0x55d3bb8d0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:46.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.304+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/1415463969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c6200 0x55d3bb8d0470 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f4c98000c00 tx=0x7f4c98009fa0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.304+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1415463969 shutdown_connections 2024-03-21T11:39:46.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.304+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/1415463969 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3bb8ecbb0 0x55d3bb8ef020 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.304+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/1415463969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8d79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.304+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/1415463969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.304+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/1415463969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c6200 0x55d3bb8d0470 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.304+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1415463969 >> 172.21.15.47:0/1415463969 conn(0x55d3bb8c15f0 msgr2=0x55d3bb8b8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:46.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.304+0000 7f4c92000640 0 client.30922 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:46.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.304+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1415463969 shutdown_connections 2024-03-21T11:39:46.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.304+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1415463969 wait complete. 2024-03-21T11:39:46.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.304+0000 7f4cb1fdd740 20 client.30922 trim_cache size 0 max 16384 2024-03-21T11:39:46.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.328+0000 7f4cb1fdd740 1 Processor -- start 2024-03-21T11:39:46.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.328+0000 7f4cb1fdd740 1 -- start start 2024-03-21T11:39:46.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.328+0000 7f4cb1fdd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c6200 0x55d3bb8ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.328+0000 7f4cb1fdd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.328+0000 7f4cb1fdd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.328+0000 7f4cb1fdd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca8070d40 con 0x55d3bb8c7d90 2024-03-21T11:39:46.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.328+0000 7f4cb1fdd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca8072750 con 0x55d3bb8c6200 2024-03-21T11:39:46.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.328+0000 7f4cb1fdd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca80713a0 con 0x55d3bb8c73d0 2024-03-21T11:39:46.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.328+0000 7f4ca7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.328+0000 7f4cad200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.328+0000 7f4ca7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8ca470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48430/0 (socket says 172.21.15.47:48430) 2024-03-21T11:39:46.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.328+0000 7f4ca7400640 1 -- 172.21.15.47:0/37850893 learned_addr learned my addr 172.21.15.47:0/37850893 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:46.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.328+0000 7f4ca7e00640 1 --2- 172.21.15.47:0/37850893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c6200 0x55d3bb8ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.329+0000 7f4ca7400640 1 -- 172.21.15.47:0/37850893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c6200 msgr2=0x55d3bb8ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:46.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.329+0000 7f4ca7400640 1 --2- 172.21.15.47:0/37850893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c6200 0x55d3bb8ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.329+0000 7f4ca7400640 1 -- 172.21.15.47:0/37850893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c7d90 msgr2=0x55d3bb8ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.329+0000 7f4ca7400640 1 --2- 172.21.15.47:0/37850893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.329+0000 7f4ca7400640 1 -- 172.21.15.47:0/37850893 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3bb8c6e50 con 0x55d3bb8c73d0 2024-03-21T11:39:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.329+0000 7f4ca7400640 1 --2- 172.21.15.47:0/37850893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8ca470 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f4c940b9fe0 tx=0x7f4c940a8040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3c3a5f1547bf604 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.329+0000 7f4c9cc00640 1 -- 172.21.15.47:0/37850893 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9404b040 con 0x55d3bb8c73d0 2024-03-21T11:39:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.329+0000 7f4c9cc00640 1 -- 172.21.15.47:0/37850893 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c94004c60 con 0x55d3bb8c73d0 2024-03-21T11:39:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.329+0000 7f4c9cc00640 1 -- 172.21.15.47:0/37850893 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c940eeb40 con 0x55d3bb8c73d0 2024-03-21T11:39:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.330+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/37850893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c73d0 msgr2=0x55d3bb8ca470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.330+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/37850893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8ca470 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f4c940b9fe0 tx=0x7f4c940a8040 comp rx=0 tx=0).stop 2024-03-21T11:39:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.330+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/37850893 shutdown_connections 2024-03-21T11:39:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.330+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/37850893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.330+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/37850893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8ca470 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.330+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/37850893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c6200 0x55d3bb8ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.330+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/37850893 >> 172.21.15.47:0/37850893 conn(0x55d3bb8c15f0 msgr2=0x55d3bb8b8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.330+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/37850893 shutdown_connections 2024-03-21T11:39:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.330+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/37850893 wait complete. 2024-03-21T11:39:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.331+0000 7f4cb1fdd740 1 -- start start 2024-03-21T11:39:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.331+0000 7f4cb1fdd740 1 Processor -- start 2024-03-21T11:39:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.331+0000 7f4cb1fdd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c6200 0x55d3bb8d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.331+0000 7f4cb1fdd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.331+0000 7f4cb1fdd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.331+0000 7f4cb1fdd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca8072060 con 0x55d3bb8c6200 2024-03-21T11:39:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.331+0000 7f4cb1fdd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca8071a00 con 0x55d3bb8c73d0 2024-03-21T11:39:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.331+0000 7f4cb1fdd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca806ef00 con 0x55d3bb8c7d90 2024-03-21T11:39:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.332+0000 7f4cb1fdd740 10 client.0 _ll_register_callbacks cb 0x7f4c94005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.332+0000 7f4ca7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.332+0000 7f4ca7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8d2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60568/0 (socket says 172.21.15.47:60568) 2024-03-21T11:39:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.332+0000 7f4ca7400640 1 -- 172.21.15.47:0/1384228245 learned_addr learned my addr 172.21.15.47:0/1384228245 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.332+0000 7f4ca7e00640 1 --2- 172.21.15.47:0/1384228245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c6200 0x55d3bb8d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.332+0000 7f4cad200640 1 --2- 172.21.15.47:0/1384228245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8d5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.332+0000 7f4ca7400640 1 -- 172.21.15.47:0/1384228245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c7d90 msgr2=0x55d3bb8d5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.332+0000 7f4ca7400640 1 --2- 172.21.15.47:0/1384228245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8d5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.332+0000 7f4ca7400640 1 -- 172.21.15.47:0/1384228245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c6200 msgr2=0x55d3bb8d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.332+0000 7f4ca7400640 1 --2- 172.21.15.47:0/1384228245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c6200 0x55d3bb8d2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.332+0000 7f4ca7400640 1 -- 172.21.15.47:0/1384228245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3bb8d62b0 con 0x55d3bb8c73d0 2024-03-21T11:39:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.332+0000 7f4ca7400640 1 --2- 172.21.15.47:0/1384228245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8d2ca0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4c940bd5f0 tx=0x7f4c9402cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5452b4ff32886d73 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.332+0000 7f4c92000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.333+0000 7f4c92000640 1 -- 172.21.15.47:0/1384228245 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9404b070 con 0x55d3bb8c73d0 2024-03-21T11:39:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.333+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1384228245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c94004c60 con 0x55d3bb8c73d0 2024-03-21T11:39:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.333+0000 7f4c92000640 1 -- 172.21.15.47:0/1384228245 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4c940b9040 con 0x55d3bb8c73d0 2024-03-21T11:39:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.333+0000 7f4c92000640 1 -- 172.21.15.47:0/1384228245 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c940bd660 con 0x55d3bb8c73d0 2024-03-21T11:39:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.333+0000 7f4cb1fdd740 10 client.30929 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.333+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1384228245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55d3bb948520 con 0x55d3bb8c73d0 2024-03-21T11:39:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.333+0000 7f4c92000640 1 -- 172.21.15.47:0/1384228245 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c94004c60 con 0x55d3bb8c73d0 2024-03-21T11:39:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.334+0000 7f4c92000640 1 -- 172.21.15.47:0/1384228245 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f4c940c5be0 con 0x55d3bb8c73d0 2024-03-21T11:39:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.334+0000 7f4cb1fdd740 10 client.30929 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:46.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.334+0000 7f4cb1fdd740 10 client.30929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:46.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.334+0000 7f4cb1fdd740 20 client.30929 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:46.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.334+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/1384228245 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3bb948e60 0x55d3bb94b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:46.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.334+0000 7f4cb1fdd740 4 client.30929 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:39:46.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.334+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1384228245 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d3bb829b50 con 0x55d3bb948e60 2024-03-21T11:39:46.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.334+0000 7f4ca7e00640 1 --2- 172.21.15.47:0/1384228245 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3bb948e60 0x55d3bb94b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:46.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.335+0000 7f4ca7e00640 1 --2- 172.21.15.47:0/1384228245 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3bb948e60 0x55d3bb94b280 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ed504e92b18b257 server_cookie=1fe71e027d2a5699 in_seq=0 out_seq=0 2024-03-21T11:39:46.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.335+0000 7f4c92000640 10 client.30929 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:46.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.338+0000 7f4c92000640 1 -- 172.21.15.47:0/1384228245 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d3bb829b50 con 0x55d3bb948e60 2024-03-21T11:39:46.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.338+0000 7f4c92000640 10 client.30929 handle_command_reply: tid=0 2024-03-21T11:39:46.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.338+0000 7f4cb1fdd740 1 client.30929 shutdown 2024-03-21T11:39:46.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.339+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1384228245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c73d0 msgr2=0x55d3bb8d2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.339+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/1384228245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8d2ca0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4c940bd5f0 tx=0x7f4c9402cf90 comp rx=0 tx=0).stop 2024-03-21T11:39:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.339+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1384228245 shutdown_connections 2024-03-21T11:39:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.339+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/1384228245 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d3bb948e60 0x55d3bb94b280 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.339+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/1384228245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3bb8c7d90 0x55d3bb8d5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.339+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/1384228245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3bb8c73d0 0x55d3bb8d2ca0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.339+0000 7f4cb1fdd740 1 --2- 172.21.15.47:0/1384228245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3bb8c6200 0x55d3bb8d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.339+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1384228245 >> 172.21.15.47:0/1384228245 conn(0x55d3bb8c15f0 msgr2=0x55d3bb8c4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.339+0000 7f4c92000640 0 client.30929 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.339+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1384228245 shutdown_connections 2024-03-21T11:39:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.339+0000 7f4cb1fdd740 1 -- 172.21.15.47:0/1384228245 wait complete. 2024-03-21T11:39:46.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.339+0000 7f4cb1fdd740 20 client.30929 trim_cache size 0 max 16384 2024-03-21T11:39:46.343 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:39:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.340+0000 7f4cadc00640 1 -- 172.21.15.47:0/1080566698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c8c0a2fa0 msgr2=0x7f4c8c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.340+0000 7f4cadc00640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c8c0a2fa0 0x7f4c8c0a5460 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009fd0 tx=0x7f4ca0009e00 comp rx=0 tx=0).stop 2024-03-21T11:39:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.340+0000 7f4cadc00640 1 -- 172.21.15.47:0/1080566698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ca8104990 msgr2=0x7f4ca8199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.340+0000 7f4cadc00640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ca8104990 0x7f4ca8199fa0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f4c9402c8f0 tx=0x7f4c9402c660 comp rx=0 tx=0).stop 2024-03-21T11:39:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.341+0000 7f4cadc00640 1 -- 172.21.15.47:0/1080566698 shutdown_connections 2024-03-21T11:39:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.341+0000 7f4cadc00640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4c8c0a2fa0 0x7f4c8c0a5460 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.341+0000 7f4cadc00640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca8105350 0x7f4ca81a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.341+0000 7f4cadc00640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ca8104990 0x7f4ca8199fa0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.341+0000 7f4cadc00640 1 --2- 172.21.15.47:0/1080566698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca81037a0 0x7f4ca8199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.341+0000 7f4cadc00640 1 -- 172.21.15.47:0/1080566698 >> 172.21.15.47:0/1080566698 conn(0x7f4ca80fed10 msgr2=0x7f4ca8107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:46.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.341+0000 7f4cadc00640 1 -- 172.21.15.47:0/1080566698 shutdown_connections 2024-03-21T11:39:46.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:46.341+0000 7f4cadc00640 1 -- 172.21.15.47:0/1080566698 wait complete. 2024-03-21T11:39:46.934 INFO:tasks.workunit.client.0.smithi047.stdout: 10 72305 25.12 MB/sec execute 30 sec latency 56.004 ms 2024-03-21T11:39:47.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:47 smithi047 ceph-mon[31351]: pgmap v893: 129 pgs: 129 active+clean; 352 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 777 op/s 2024-03-21T11:39:47.363 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:39:47.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:39:47.364 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:39:47.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:47 smithi100 ceph-mon[36245]: pgmap v893: 129 pgs: 129 active+clean; 352 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 777 op/s 2024-03-21T11:39:47.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:47 smithi102 ceph-mon[36748]: pgmap v893: 129 pgs: 129 active+clean; 352 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 777 op/s 2024-03-21T11:39:47.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.659+0000 7f786d600640 1 -- 172.21.15.47:0/3680650369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7868105110 msgr2=0x7f7868105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:47.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.659+0000 7f786d600640 1 --2- 172.21.15.47:0/3680650369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7868105110 0x7f7868105590 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f7860009be0 tx=0x7f786002bef0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.659+0000 7f786d600640 1 -- 172.21.15.47:0/3680650369 shutdown_connections 2024-03-21T11:39:47.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.659+0000 7f786d600640 1 --2- 172.21.15.47:0/3680650369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7868105110 0x7f7868105590 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.659+0000 7f786d600640 1 --2- 172.21.15.47:0/3680650369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7868102760 0x7f7868104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.659+0000 7f786d600640 1 --2- 172.21.15.47:0/3680650369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78680ffdf0 0x7f7868102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.659+0000 7f786d600640 1 -- 172.21.15.47:0/3680650369 >> 172.21.15.47:0/3680650369 conn(0x7f78680fba10 msgr2=0x7f78680fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:47.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.660+0000 7f786d600640 1 -- 172.21.15.47:0/3680650369 shutdown_connections 2024-03-21T11:39:47.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.661+0000 7f786d600640 1 -- 172.21.15.47:0/3680650369 wait complete. 2024-03-21T11:39:47.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.662+0000 7f786d600640 1 Processor -- start 2024-03-21T11:39:47.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.663+0000 7f786d600640 1 -- start start 2024-03-21T11:39:47.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.663+0000 7f786d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78680ffdf0 0x7f786810b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.663+0000 7f786d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7868102760 0x7f786810b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.663+0000 7f786d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7868105110 0x7f78681adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.663+0000 7f786d600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7868113820 con 0x7f7868105110 2024-03-21T11:39:47.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.663+0000 7f786d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78681136a0 con 0x7f7868102760 2024-03-21T11:39:47.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.663+0000 7f786d600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78681139a0 con 0x7f78680ffdf0 2024-03-21T11:39:47.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.664+0000 7f7867400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7868102760 0x7f786810b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.664+0000 7f7867400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7868102760 0x7f786810b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60590/0 (socket says 172.21.15.47:60590) 2024-03-21T11:39:47.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.664+0000 7f7867400640 1 -- 172.21.15.47:0/4094557199 learned_addr learned my addr 172.21.15.47:0/4094557199 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:47.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.665+0000 7f786cc00640 1 --2- 172.21.15.47:0/4094557199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7868105110 0x7f78681adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.665+0000 7f7867e00640 1 --2- 172.21.15.47:0/4094557199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78680ffdf0 0x7f786810b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.665+0000 7f786cc00640 1 -- 172.21.15.47:0/4094557199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78680ffdf0 msgr2=0x7f786810b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:47.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.666+0000 7f786cc00640 1 --2- 172.21.15.47:0/4094557199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78680ffdf0 0x7f786810b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.666+0000 7f786cc00640 1 -- 172.21.15.47:0/4094557199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7868102760 msgr2=0x7f786810b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:47.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.666+0000 7f786cc00640 1 --2- 172.21.15.47:0/4094557199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7868102760 0x7f786810b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.666+0000 7f786cc00640 1 -- 172.21.15.47:0/4094557199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786810e9c0 con 0x7f7868105110 2024-03-21T11:39:47.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.667+0000 7f786cc00640 1 --2- 172.21.15.47:0/4094557199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7868105110 0x7f78681adf30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f786002c430 tx=0x7f786002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:47.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.669+0000 7f7864c00640 1 -- 172.21.15.47:0/4094557199 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860043020 con 0x7f7868105110 2024-03-21T11:39:47.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.669+0000 7f7864c00640 1 -- 172.21.15.47:0/4094557199 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78600047b0 con 0x7f7868105110 2024-03-21T11:39:47.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.669+0000 7f7864c00640 1 -- 172.21.15.47:0/4094557199 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78600337e0 con 0x7f7868105110 2024-03-21T11:39:47.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.669+0000 7f786d600640 1 -- 172.21.15.47:0/4094557199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78681ae4d0 con 0x7f7868105110 2024-03-21T11:39:47.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.670+0000 7f786d600640 1 -- 172.21.15.47:0/4094557199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78681ae9a0 con 0x7f7868105110 2024-03-21T11:39:47.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.676+0000 7f7864c00640 1 -- 172.21.15.47:0/4094557199 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7860046020 con 0x7f7868105110 2024-03-21T11:39:47.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.679+0000 7f7864c00640 1 --2- 172.21.15.47:0/4094557199 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f784c0a2fa0 0x7f784c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.679+0000 7f7864c00640 1 -- 172.21.15.47:0/4094557199 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f78600f1d20 con 0x7f7868105110 2024-03-21T11:39:47.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.680+0000 7f78717f3740 1 Processor -- start 2024-03-21T11:39:47.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.681+0000 7f7867e00640 1 --2- 172.21.15.47:0/4094557199 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f784c0a2fa0 0x7f784c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.682+0000 7f7867e00640 1 --2- 172.21.15.47:0/4094557199 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f784c0a2fa0 0x7f784c0a5460 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7850005fd0 tx=0x7f7850005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:47.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.682+0000 7f78717f3740 1 -- start start 2024-03-21T11:39:47.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.682+0000 7f78717f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 0x55df460d66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.683+0000 7f78717f3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460d6ca0 0x55df460db7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.683+0000 7f78717f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460dbe10 0x55df460de2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.683+0000 7f78717f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df4603beb0 con 0x55df460d62b0 2024-03-21T11:39:47.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.683+0000 7f78717f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df45fe4a60 con 0x55df460dbe10 2024-03-21T11:39:47.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.683+0000 7f78717f3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df46037f50 con 0x55df460d6ca0 2024-03-21T11:39:47.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.684+0000 7f786cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460dbe10 0x55df460de2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.684+0000 7f786cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460dbe10 0x55df460de2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60604/0 (socket says 172.21.15.47:60604) 2024-03-21T11:39:47.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.684+0000 7f786cc00640 1 -- 172.21.15.47:0/979896255 learned_addr learned my addr 172.21.15.47:0/979896255 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:47.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.684+0000 7f786cc00640 1 -- 172.21.15.47:0/979896255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460d6ca0 msgr2=0x55df460db7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:39:47.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.685+0000 7f786cc00640 1 --2- 172.21.15.47:0/979896255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460d6ca0 0x55df460db7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.685+0000 7f786cc00640 1 -- 172.21.15.47:0/979896255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 msgr2=0x55df460d66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:47.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.685+0000 7f786cc00640 1 --2- 172.21.15.47:0/979896255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 0x55df460d66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.685+0000 7f786cc00640 1 -- 172.21.15.47:0/979896255 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df45e187e0 con 0x55df460dbe10 2024-03-21T11:39:47.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.685+0000 7f786cc00640 1 --2- 172.21.15.47:0/979896255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460dbe10 0x55df460de2a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7860009be0 tx=0x7f7860009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eec6055a5a09eb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:47.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.686+0000 7f785d200640 1 -- 172.21.15.47:0/979896255 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78600be150 con 0x55df460dbe10 2024-03-21T11:39:47.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.686+0000 7f785d200640 1 -- 172.21.15.47:0/979896255 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78600c22a0 con 0x55df460dbe10 2024-03-21T11:39:47.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.686+0000 7f785d200640 1 -- 172.21.15.47:0/979896255 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78600de2d0 con 0x55df460dbe10 2024-03-21T11:39:47.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.687+0000 7f78717f3740 1 -- 172.21.15.47:0/979896255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460dbe10 msgr2=0x55df460de2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:47.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.687+0000 7f78717f3740 1 --2- 172.21.15.47:0/979896255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460dbe10 0x55df460de2a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7860009be0 tx=0x7f7860009500 comp rx=0 tx=0).stop 2024-03-21T11:39:47.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.687+0000 7f78717f3740 1 -- 172.21.15.47:0/979896255 shutdown_connections 2024-03-21T11:39:47.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.687+0000 7f78717f3740 1 --2- 172.21.15.47:0/979896255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460dbe10 0x55df460de2a0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.687+0000 7f78717f3740 1 --2- 172.21.15.47:0/979896255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460d6ca0 0x55df460db7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.687+0000 7f78717f3740 1 --2- 172.21.15.47:0/979896255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 0x55df460d66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.687+0000 7f78717f3740 1 -- 172.21.15.47:0/979896255 >> 172.21.15.47:0/979896255 conn(0x55df460d1820 msgr2=0x55df460d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:47.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.687+0000 7f78717f3740 1 -- 172.21.15.47:0/979896255 shutdown_connections 2024-03-21T11:39:47.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.687+0000 7f78717f3740 1 -- 172.21.15.47:0/979896255 wait complete. 2024-03-21T11:39:47.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.688+0000 7f78717f3740 1 -- start start 2024-03-21T11:39:47.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.688+0000 7f78717f3740 1 Processor -- start 2024-03-21T11:39:47.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.689+0000 7f78717f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 0x55df460e02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.690+0000 7f78717f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d6ca0 0x55df460e0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.690+0000 7f78717f3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460dbe10 0x55df460e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.690+0000 7f7867e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 0x55df460e02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.690+0000 7f7867e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 0x55df460e02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46068/0 (socket says 172.21.15.47:46068) 2024-03-21T11:39:47.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.690+0000 7f7867400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d6ca0 0x55df460e0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.690+0000 7f7867e00640 1 -- 172.21.15.47:0/451326928 learned_addr learned my addr 172.21.15.47:0/451326928 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:47.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.690+0000 7f78717f3740 1 -- 172.21.15.47:0/451326928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df460deb80 con 0x55df460d62b0 2024-03-21T11:39:47.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.690+0000 7f78717f3740 1 -- 172.21.15.47:0/451326928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df45ff7eb0 con 0x55df460d6ca0 2024-03-21T11:39:47.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.690+0000 7f78717f3740 1 -- 172.21.15.47:0/451326928 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df460ded00 con 0x55df460dbe10 2024-03-21T11:39:47.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.691+0000 7f78717f3740 10 client.0 _ll_register_callbacks cb 0x55df45fec150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:47.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.691+0000 7f786cc00640 1 --2- 172.21.15.47:0/451326928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460dbe10 0x55df460e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.691+0000 7f7867400640 1 -- 172.21.15.47:0/451326928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460dbe10 msgr2=0x55df460e77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:47.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.691+0000 7f7867400640 1 --2- 172.21.15.47:0/451326928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460dbe10 0x55df460e77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.691+0000 7f7867400640 1 -- 172.21.15.47:0/451326928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 msgr2=0x55df460e02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:47.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.691+0000 7f7867400640 1 --2- 172.21.15.47:0/451326928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 0x55df460e02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.691+0000 7f7867400640 1 -- 172.21.15.47:0/451326928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df45e78f70 con 0x55df460d6ca0 2024-03-21T11:39:47.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.692+0000 7f786cc00640 1 --2- 172.21.15.47:0/451326928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460dbe10 0x55df460e77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:47.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.692+0000 7f7867400640 1 --2- 172.21.15.47:0/451326928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d6ca0 0x55df460e0810 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x55df460d70a0 tx=0x7f7858011ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4a46108d52f74ae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:47.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.692+0000 7f7855600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:47.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.693+0000 7f7855600640 1 -- 172.21.15.47:0/451326928 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858005020 con 0x55df460d6ca0 2024-03-21T11:39:47.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.693+0000 7f7855600640 1 -- 172.21.15.47:0/451326928 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f78580051c0 con 0x55df460d6ca0 2024-03-21T11:39:47.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.693+0000 7f78717f3740 1 -- 172.21.15.47:0/451326928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78600c22a0 con 0x55df460d6ca0 2024-03-21T11:39:47.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.693+0000 7f7855600640 1 -- 172.21.15.47:0/451326928 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7858024050 con 0x55df460d6ca0 2024-03-21T11:39:47.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.693+0000 7f78717f3740 10 client.30941 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:47.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.693+0000 7f78717f3740 1 -- 172.21.15.47:0/451326928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55df45d10170 con 0x55df460d6ca0 2024-03-21T11:39:47.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.694+0000 7f7855600640 1 -- 172.21.15.47:0/451326928 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78600c22a0 con 0x55df460d6ca0 2024-03-21T11:39:47.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.695+0000 7f7855600640 1 -- 172.21.15.47:0/451326928 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f7858004ae0 con 0x55df460d6ca0 2024-03-21T11:39:47.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.695+0000 7f78717f3740 10 client.30941 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:47.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.695+0000 7f78717f3740 10 client.30941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:47.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.695+0000 7f78717f3740 20 client.30941 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:47.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.695+0000 7f78717f3740 1 --2- 172.21.15.47:0/451326928 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df460ec120 0x55df460ee560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.695+0000 7f78717f3740 4 client.30941 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:47.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.695+0000 7f78717f3740 1 -- 172.21.15.47:0/451326928 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df460493f0 con 0x55df460ec120 2024-03-21T11:39:47.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.697+0000 7f7867e00640 1 --2- 172.21.15.47:0/451326928 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df460ec120 0x55df460ee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.700+0000 7f7855600640 10 client.30941 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:47.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.700+0000 7f7867e00640 1 --2- 172.21.15.47:0/451326928 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df460ec120 0x55df460ee560 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69a77918bd623f33 server_cookie=ff5292870edd89b4 in_seq=0 out_seq=0 2024-03-21T11:39:47.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.709+0000 7f7855600640 1 -- 172.21.15.47:0/451326928 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55df460493f0 con 0x55df460ec120 2024-03-21T11:39:47.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.709+0000 7f7855600640 10 client.30941 handle_command_reply: tid=0 2024-03-21T11:39:47.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.709+0000 7f78717f3740 1 client.30941 shutdown 2024-03-21T11:39:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.710+0000 7f78717f3740 1 -- 172.21.15.47:0/451326928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d6ca0 msgr2=0x55df460e0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.710+0000 7f78717f3740 1 --2- 172.21.15.47:0/451326928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d6ca0 0x55df460e0810 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x55df460d70a0 tx=0x7f7858011ae0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.711+0000 7f78717f3740 1 -- 172.21.15.47:0/451326928 shutdown_connections 2024-03-21T11:39:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.711+0000 7f78717f3740 1 --2- 172.21.15.47:0/451326928 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df460ec120 0x55df460ee560 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.711+0000 7f78717f3740 1 --2- 172.21.15.47:0/451326928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460dbe10 0x55df460e77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.711+0000 7f78717f3740 1 --2- 172.21.15.47:0/451326928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d6ca0 0x55df460e0810 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.711+0000 7f78717f3740 1 --2- 172.21.15.47:0/451326928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 0x55df460e02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.711+0000 7f78717f3740 1 -- 172.21.15.47:0/451326928 >> 172.21.15.47:0/451326928 conn(0x55df460d1820 msgr2=0x55df460d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.711+0000 7f7855600640 0 client.30941 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.712+0000 7f78717f3740 1 -- 172.21.15.47:0/451326928 shutdown_connections 2024-03-21T11:39:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.712+0000 7f78717f3740 1 -- 172.21.15.47:0/451326928 wait complete. 2024-03-21T11:39:47.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.712+0000 7f78717f3740 20 client.30941 trim_cache size 0 max 16384 2024-03-21T11:39:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.741+0000 7f78717f3740 1 Processor -- start 2024-03-21T11:39:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.742+0000 7f78717f3740 1 -- start start 2024-03-21T11:39:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.742+0000 7f78717f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d1820 0x55df460d5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.742+0000 7f78717f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d62b0 0x55df460e77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.742+0000 7f78717f3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460d6ca0 0x55df460d7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.742+0000 7f78717f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784400fa60 con 0x55df460d1820 2024-03-21T11:39:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.742+0000 7f78717f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7844011470 con 0x55df460d62b0 2024-03-21T11:39:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.742+0000 7f78717f3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78440100c0 con 0x55df460d6ca0 2024-03-21T11:39:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.743+0000 7f7867400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d62b0 0x55df460e77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.743+0000 7f7867400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d62b0 0x55df460e77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60634/0 (socket says 172.21.15.47:60634) 2024-03-21T11:39:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.743+0000 7f7867400640 1 -- 172.21.15.47:0/51250516 learned_addr learned my addr 172.21.15.47:0/51250516 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:47.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.743+0000 7f7867e00640 1 --2- 172.21.15.47:0/51250516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d1820 0x55df460d5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.743+0000 7f7867400640 1 -- 172.21.15.47:0/51250516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460d6ca0 msgr2=0x55df460d7710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:39:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.743+0000 7f7867400640 1 --2- 172.21.15.47:0/51250516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460d6ca0 0x55df460d7710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.743+0000 7f7867400640 1 -- 172.21.15.47:0/51250516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d1820 msgr2=0x55df460d5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.743+0000 7f7867400640 1 --2- 172.21.15.47:0/51250516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d1820 0x55df460d5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.744+0000 7f7867400640 1 -- 172.21.15.47:0/51250516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df460db220 con 0x55df460d62b0 2024-03-21T11:39:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.744+0000 7f7867400640 1 --2- 172.21.15.47:0/51250516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d62b0 0x55df460e77a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f78580095a0 tx=0x7f7858004cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=998e8143cf5992b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.745+0000 7f7857e00640 1 -- 172.21.15.47:0/51250516 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785802c070 con 0x55df460d62b0 2024-03-21T11:39:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.745+0000 7f7857e00640 1 -- 172.21.15.47:0/51250516 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7858004820 con 0x55df460d62b0 2024-03-21T11:39:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.745+0000 7f7857e00640 1 -- 172.21.15.47:0/51250516 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858000e60 con 0x55df460d62b0 2024-03-21T11:39:47.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.746+0000 7f78717f3740 1 -- 172.21.15.47:0/51250516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d62b0 msgr2=0x55df460e77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:47.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.746+0000 7f78717f3740 1 --2- 172.21.15.47:0/51250516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d62b0 0x55df460e77a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f78580095a0 tx=0x7f7858004cb0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.746+0000 7f78717f3740 1 -- 172.21.15.47:0/51250516 shutdown_connections 2024-03-21T11:39:47.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.746+0000 7f78717f3740 1 --2- 172.21.15.47:0/51250516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460d6ca0 0x55df460d7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.746+0000 7f78717f3740 1 --2- 172.21.15.47:0/51250516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d62b0 0x55df460e77a0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.746+0000 7f78717f3740 1 --2- 172.21.15.47:0/51250516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d1820 0x55df460d5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.746+0000 7f78717f3740 1 -- 172.21.15.47:0/51250516 >> 172.21.15.47:0/51250516 conn(0x55df460db910 msgr2=0x55df460d4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:47.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.746+0000 7f78717f3740 1 -- 172.21.15.47:0/51250516 shutdown_connections 2024-03-21T11:39:47.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.746+0000 7f78717f3740 1 -- 172.21.15.47:0/51250516 wait complete. 2024-03-21T11:39:47.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.747+0000 7f78717f3740 1 -- start start 2024-03-21T11:39:47.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.747+0000 7f78717f3740 1 Processor -- start 2024-03-21T11:39:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.748+0000 7f78717f3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460d1820 0x55df460e5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.748+0000 7f78717f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 0x55df460e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.748+0000 7f78717f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d6ca0 0x55df460e6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.748+0000 7f78717f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7844010d80 con 0x55df460d62b0 2024-03-21T11:39:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.748+0000 7f78717f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7844010720 con 0x55df460d6ca0 2024-03-21T11:39:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.748+0000 7f78717f3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784400dc20 con 0x55df460d1820 2024-03-21T11:39:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.748+0000 7f78717f3740 10 client.0 _ll_register_callbacks cb 0x7f7858009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.749+0000 7f7867400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 0x55df460e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.749+0000 7f7867400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 0x55df460e5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46098/0 (socket says 172.21.15.47:46098) 2024-03-21T11:39:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.749+0000 7f7867400640 1 -- 172.21.15.47:0/2311480955 learned_addr learned my addr 172.21.15.47:0/2311480955 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.749+0000 7f7867e00640 1 --2- 172.21.15.47:0/2311480955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460d1820 0x55df460e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.749+0000 7f786cc00640 1 --2- 172.21.15.47:0/2311480955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d6ca0 0x55df460e6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.750+0000 7f7867e00640 1 -- 172.21.15.47:0/2311480955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d6ca0 msgr2=0x55df460e6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.750+0000 7f7867e00640 1 --2- 172.21.15.47:0/2311480955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d6ca0 0x55df460e6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.750+0000 7f7867e00640 1 -- 172.21.15.47:0/2311480955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 msgr2=0x55df460e5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.750+0000 7f7867e00640 1 --2- 172.21.15.47:0/2311480955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 0x55df460e5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.750+0000 7f7867e00640 1 -- 172.21.15.47:0/2311480955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df460e71a0 con 0x55df460d1820 2024-03-21T11:39:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.750+0000 7f7867400640 1 --2- 172.21.15.47:0/2311480955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 0x55df460e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:39:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.750+0000 7f7867e00640 1 --2- 172.21.15.47:0/2311480955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460d1820 0x55df460e5660 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f785000bfd0 tx=0x7f78500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9bbe5ff770599c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.750+0000 7f7855600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.751+0000 7f7855600640 1 -- 172.21.15.47:0/2311480955 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7850011850 con 0x55df460d1820 2024-03-21T11:39:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.751+0000 7f7855600640 1 -- 172.21.15.47:0/2311480955 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f78500119f0 con 0x55df460d1820 2024-03-21T11:39:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.751+0000 7f78717f3740 1 -- 172.21.15.47:0/2311480955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7858004820 con 0x55df460d1820 2024-03-21T11:39:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.752+0000 7f7855600640 1 -- 172.21.15.47:0/2311480955 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7850036050 con 0x55df460d1820 2024-03-21T11:39:47.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.753+0000 7f78717f3740 10 client.30937 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:47.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.753+0000 7f78717f3740 1 -- 172.21.15.47:0/2311480955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55df46148300 con 0x55df460d1820 2024-03-21T11:39:47.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.755+0000 7f7855600640 1 -- 172.21.15.47:0/2311480955 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7858004820 con 0x55df460d1820 2024-03-21T11:39:47.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.755+0000 7f7855600640 1 -- 172.21.15.47:0/2311480955 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f785001f430 con 0x55df460d1820 2024-03-21T11:39:47.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.755+0000 7f78717f3740 10 client.30937 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:47.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.755+0000 7f78717f3740 10 client.30937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:47.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.755+0000 7f78717f3740 20 client.30937 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:47.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.755+0000 7f78717f3740 1 --2- 172.21.15.47:0/2311480955 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df460dbe10 0x55df4614ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.755+0000 7f78717f3740 4 client.30937 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:39:47.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.755+0000 7f78717f3740 1 -- 172.21.15.47:0/2311480955 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55df46043740 con 0x55df460dbe10 2024-03-21T11:39:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.758+0000 7f7867400640 1 --2- 172.21.15.47:0/2311480955 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df460dbe10 0x55df4614ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.759+0000 7f7867400640 1 --2- 172.21.15.47:0/2311480955 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df460dbe10 0x55df4614ad30 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7023cc3abbdcb3ae server_cookie=b2c4444e1bb580f in_seq=0 out_seq=0 2024-03-21T11:39:47.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.759+0000 7f7855600640 10 client.30937 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:47.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.760+0000 7f7855600640 1 -- 172.21.15.47:0/2311480955 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55df46043740 con 0x55df460dbe10 2024-03-21T11:39:47.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.760+0000 7f7855600640 10 client.30937 handle_command_reply: tid=0 2024-03-21T11:39:47.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.761+0000 7f78717f3740 1 client.30937 shutdown 2024-03-21T11:39:47.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.761+0000 7f78717f3740 1 -- 172.21.15.47:0/2311480955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460d1820 msgr2=0x55df460e5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:47.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.762+0000 7f78717f3740 1 --2- 172.21.15.47:0/2311480955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460d1820 0x55df460e5660 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f785000bfd0 tx=0x7f78500046c0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.762+0000 7f78717f3740 1 -- 172.21.15.47:0/2311480955 shutdown_connections 2024-03-21T11:39:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.762+0000 7f78717f3740 1 --2- 172.21.15.47:0/2311480955 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55df460dbe10 0x55df4614ad30 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.762+0000 7f78717f3740 1 --2- 172.21.15.47:0/2311480955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df460d6ca0 0x55df460e6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.763+0000 7f78717f3740 1 --2- 172.21.15.47:0/2311480955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df460d62b0 0x55df460e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.763+0000 7f78717f3740 1 --2- 172.21.15.47:0/2311480955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df460d1820 0x55df460e5660 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.763+0000 7f78717f3740 1 -- 172.21.15.47:0/2311480955 >> 172.21.15.47:0/2311480955 conn(0x55df460db910 msgr2=0x55df460d5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.763+0000 7f7855600640 0 client.30937 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.763+0000 7f78717f3740 1 -- 172.21.15.47:0/2311480955 shutdown_connections 2024-03-21T11:39:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.763+0000 7f78717f3740 1 -- 172.21.15.47:0/2311480955 wait complete. 2024-03-21T11:39:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.763+0000 7f78717f3740 20 client.30937 trim_cache size 0 max 16384 2024-03-21T11:39:47.765 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:39:47.765 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:39:47.765 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "f6ad4206-317d-420f-8914-31e9aadc9937", 2024-03-21T11:39:47.765 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:39:47.766 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:39:47.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.764+0000 7f786d600640 1 -- 172.21.15.47:0/4094557199 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f784c0a2fa0 msgr2=0x7f784c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:47.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.764+0000 7f786d600640 1 --2- 172.21.15.47:0/4094557199 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f784c0a2fa0 0x7f784c0a5460 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7850005fd0 tx=0x7f7850005850 comp rx=0 tx=0).stop 2024-03-21T11:39:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.764+0000 7f786d600640 1 -- 172.21.15.47:0/4094557199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7868105110 msgr2=0x7f78681adf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.764+0000 7f786d600640 1 --2- 172.21.15.47:0/4094557199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7868105110 0x7f78681adf30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f786002c430 tx=0x7f786002cb30 comp rx=0 tx=0).stop 2024-03-21T11:39:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.765+0000 7f786d600640 1 -- 172.21.15.47:0/4094557199 shutdown_connections 2024-03-21T11:39:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.765+0000 7f786d600640 1 --2- 172.21.15.47:0/4094557199 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f784c0a2fa0 0x7f784c0a5460 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.765+0000 7f786d600640 1 --2- 172.21.15.47:0/4094557199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7868105110 0x7f78681adf30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.765+0000 7f786d600640 1 --2- 172.21.15.47:0/4094557199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7868102760 0x7f786810b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.765+0000 7f786d600640 1 --2- 172.21.15.47:0/4094557199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78680ffdf0 0x7f786810b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.765+0000 7f786d600640 1 -- 172.21.15.47:0/4094557199 >> 172.21.15.47:0/4094557199 conn(0x7f78680fba10 msgr2=0x7f78681006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.765+0000 7f786d600640 1 -- 172.21.15.47:0/4094557199 shutdown_connections 2024-03-21T11:39:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:47.765+0000 7f786d600640 1 -- 172.21.15.47:0/4094557199 wait complete. 2024-03-21T11:39:47.786 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:39:47.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:39:47.934 INFO:tasks.workunit.client.0.smithi047.stdout: 10 72718 24.93 MB/sec execute 31 sec latency 130.470 ms 2024-03-21T11:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.936+0000 7fa8b5e00640 1 -- 172.21.15.47:0/1421192921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8b0101f70 msgr2=0x7fa8b0104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.936+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/1421192921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8b0101f70 0x7fa8b0104360 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009900 tx=0x7fa8a402bc30 comp rx=0 tx=0).stop 2024-03-21T11:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.936+0000 7fa8b5e00640 1 -- 172.21.15.47:0/1421192921 shutdown_connections 2024-03-21T11:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.936+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/1421192921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b01071d0 0x7fa8b0109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.936+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/1421192921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b01048a0 0x7fa8b0106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.936+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/1421192921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8b0101f70 0x7fa8b0104360 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.936+0000 7fa8b5e00640 1 -- 172.21.15.47:0/1421192921 >> 172.21.15.47:0/1421192921 conn(0x7fa8b00fb980 msgr2=0x7fa8b00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.936+0000 7fa8b5e00640 1 -- 172.21.15.47:0/1421192921 shutdown_connections 2024-03-21T11:39:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.936+0000 7fa8b5e00640 1 -- 172.21.15.47:0/1421192921 wait complete. 2024-03-21T11:39:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.937+0000 7fa8b5e00640 1 Processor -- start 2024-03-21T11:39:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.937+0000 7fa8b5e00640 1 -- start start 2024-03-21T11:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.938+0000 7fa8b5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8b0101f70 0x7fa8b0199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.938+0000 7fa8b5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b01048a0 0x7fa8b0199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.938+0000 7fa8b5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b01071d0 0x7fa8b01a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.938+0000 7fa8b5e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b010d1d0 con 0x7fa8b01048a0 2024-03-21T11:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.938+0000 7fa8b5e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b010d050 con 0x7fa8b01071d0 2024-03-21T11:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.938+0000 7fa8b5e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b010d350 con 0x7fa8b0101f70 2024-03-21T11:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.938+0000 7fa8b4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8b0101f70 0x7fa8b0199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.938+0000 7fa8b4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8b0101f70 0x7fa8b0199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48502/0 (socket says 172.21.15.47:48502) 2024-03-21T11:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.938+0000 7fa8b4a00640 1 -- 172.21.15.47:0/139035273 learned_addr learned my addr 172.21.15.47:0/139035273 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.938+0000 7fa8afe00640 1 --2- 172.21.15.47:0/139035273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b01048a0 0x7fa8b0199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.938+0000 7fa8b5400640 1 --2- 172.21.15.47:0/139035273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b01071d0 0x7fa8b01a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.939+0000 7fa8b4a00640 1 -- 172.21.15.47:0/139035273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b01071d0 msgr2=0x7fa8b01a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.939+0000 7fa8b4a00640 1 --2- 172.21.15.47:0/139035273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b01071d0 0x7fa8b01a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.939+0000 7fa8b4a00640 1 -- 172.21.15.47:0/139035273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b01048a0 msgr2=0x7fa8b0199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.939+0000 7fa8b4a00640 1 --2- 172.21.15.47:0/139035273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b01048a0 0x7fa8b0199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.939+0000 7fa8b4a00640 1 -- 172.21.15.47:0/139035273 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b01a1620 con 0x7fa8b0101f70 2024-03-21T11:39:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.939+0000 7fa8b4a00640 1 --2- 172.21.15.47:0/139035273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8b0101f70 0x7fa8b0199a00 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4000c00 tx=0x7fa8a402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.939+0000 7fa8ad600640 1 -- 172.21.15.47:0/139035273 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4043020 con 0x7fa8b0101f70 2024-03-21T11:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.939+0000 7fa8b5e00640 1 -- 172.21.15.47:0/139035273 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b01a18b0 con 0x7fa8b0101f70 2024-03-21T11:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.939+0000 7fa8ad600640 1 -- 172.21.15.47:0/139035273 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8a40054e0 con 0x7fa8b0101f70 2024-03-21T11:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.939+0000 7fa8ad600640 1 -- 172.21.15.47:0/139035273 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4003c40 con 0x7fa8b0101f70 2024-03-21T11:39:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.940+0000 7fa8b5e00640 1 -- 172.21.15.47:0/139035273 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b01a1e50 con 0x7fa8b0101f70 2024-03-21T11:39:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.945+0000 7fa8ba19d740 1 Processor -- start 2024-03-21T11:39:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.947+0000 7fa8ba19d740 1 -- start start 2024-03-21T11:39:47.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.947+0000 7fa8ad600640 1 -- 172.21.15.47:0/139035273 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa8a4046020 con 0x7fa8b0101f70 2024-03-21T11:39:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.948+0000 7fa8ba19d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75d33f0 0x55fac75d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.948+0000 7fa8ba19d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d3de0 0x55fac75d88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.948+0000 7fa8ba19d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d8f50 0x55fac75db3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.948+0000 7fa8ba19d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac75963b0 con 0x55fac75d3de0 2024-03-21T11:39:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.948+0000 7fa8ba19d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac7538830 con 0x55fac75d33f0 2024-03-21T11:39:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.948+0000 7fa8ba19d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac74e15d0 con 0x55fac75d8f50 2024-03-21T11:39:47.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.949+0000 7fa8afe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d8f50 0x55fac75db3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.949+0000 7fa8afe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d8f50 0x55fac75db3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48506/0 (socket says 172.21.15.47:48506) 2024-03-21T11:39:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.949+0000 7fa8afe00640 1 -- 172.21.15.47:0/1703679576 learned_addr learned my addr 172.21.15.47:0/1703679576 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.949+0000 7fa8b5400640 1 --2- 172.21.15.47:0/1703679576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75d33f0 0x55fac75d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.950+0000 7fa8b4a00640 1 --2- 172.21.15.47:0/1703679576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d3de0 0x55fac75d88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.950+0000 7fa8ad600640 1 --2- 172.21.15.47:0/139035273 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8980a2f10 0x7fa8980a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.951+0000 7fa8afe00640 1 -- 172.21.15.47:0/1703679576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75d33f0 msgr2=0x55fac75d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.951+0000 7fa8afe00640 1 --2- 172.21.15.47:0/1703679576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75d33f0 0x55fac75d3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.951+0000 7fa8afe00640 1 -- 172.21.15.47:0/1703679576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d3de0 msgr2=0x55fac75d88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.951+0000 7fa8afe00640 1 --2- 172.21.15.47:0/1703679576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d3de0 0x55fac75d88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.951+0000 7fa8afe00640 1 -- 172.21.15.47:0/1703679576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fac74f92c0 con 0x55fac75d8f50 2024-03-21T11:39:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.951+0000 7fa8ad600640 1 -- 172.21.15.47:0/139035273 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa8a40f1830 con 0x7fa8b0101f70 2024-03-21T11:39:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.951+0000 7fa8b5400640 1 --2- 172.21.15.47:0/139035273 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8980a2f10 0x7fa8980a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.952+0000 7fa8afe00640 1 --2- 172.21.15.47:0/1703679576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d8f50 0x55fac75db3e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009c60 tx=0x7fa89c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a96ed172af38dd95 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.954+0000 7fa8a1c00640 1 -- 172.21.15.47:0/1703679576 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c004250 con 0x55fac75d8f50 2024-03-21T11:39:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.954+0000 7fa8a1c00640 1 -- 172.21.15.47:0/1703679576 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa89c009e10 con 0x55fac75d8f50 2024-03-21T11:39:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.954+0000 7fa8a1c00640 1 -- 172.21.15.47:0/1703679576 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c004dc0 con 0x55fac75d8f50 2024-03-21T11:39:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.955+0000 7fa8b5400640 1 --2- 172.21.15.47:0/139035273 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8980a2f10 0x7fa8980a53d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009fd0 tx=0x7fa8a8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.955+0000 7fa8ba19d740 1 -- 172.21.15.47:0/1703679576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d8f50 msgr2=0x55fac75db3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.955+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/1703679576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d8f50 0x55fac75db3e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009c60 tx=0x7fa89c011810 comp rx=0 tx=0).stop 2024-03-21T11:39:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.955+0000 7fa8ba19d740 1 -- 172.21.15.47:0/1703679576 shutdown_connections 2024-03-21T11:39:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.955+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/1703679576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d8f50 0x55fac75db3e0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.955+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/1703679576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d3de0 0x55fac75d88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.955+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/1703679576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75d33f0 0x55fac75d3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.955+0000 7fa8ba19d740 1 -- 172.21.15.47:0/1703679576 >> 172.21.15.47:0/1703679576 conn(0x55fac75ce960 msgr2=0x55fac75ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.956+0000 7fa8ba19d740 1 -- 172.21.15.47:0/1703679576 shutdown_connections 2024-03-21T11:39:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.956+0000 7fa8ba19d740 1 -- 172.21.15.47:0/1703679576 wait complete. 2024-03-21T11:39:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.957+0000 7fa8ba19d740 1 -- start start 2024-03-21T11:39:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.957+0000 7fa8ba19d740 1 Processor -- start 2024-03-21T11:39:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.958+0000 7fa8ba19d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 0x55fac75dd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.958+0000 7fa8ba19d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75d3de0 0x55fac75dd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.958+0000 7fa8ba19d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 0x55fac75e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.958+0000 7fa8ba19d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac74f4a20 con 0x55fac75d8f50 2024-03-21T11:39:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.958+0000 7fa8ba19d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac75348d0 con 0x55fac75d3de0 2024-03-21T11:39:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.958+0000 7fa8ba19d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fac75dbdc0 con 0x55fac75d33f0 2024-03-21T11:39:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.959+0000 7fa8ba19d740 10 client.0 _ll_register_callbacks cb 0x55fac75311c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.959+0000 7fa8afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75d3de0 0x55fac75dd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.960+0000 7fa8afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75d3de0 0x55fac75dd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60696/0 (socket says 172.21.15.47:60696) 2024-03-21T11:39:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.960+0000 7fa8afe00640 1 -- 172.21.15.47:0/3021392815 learned_addr learned my addr 172.21.15.47:0/3021392815 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.960+0000 7fa8afe00640 1 -- 172.21.15.47:0/3021392815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 msgr2=0x55fac75dd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.961+0000 7fa8b4a00640 1 --2- 172.21.15.47:0/3021392815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 0x55fac75dd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.961+0000 7fa8afe00640 1 --2- 172.21.15.47:0/3021392815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 0x55fac75dd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.961+0000 7fa8afe00640 1 -- 172.21.15.47:0/3021392815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 msgr2=0x55fac75e2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.961+0000 7fa8afe00640 1 --2- 172.21.15.47:0/3021392815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 0x55fac75e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.961+0000 7fa8afe00640 1 -- 172.21.15.47:0/3021392815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fac724d210 con 0x55fac75d3de0 2024-03-21T11:39:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.961+0000 7fa8b4a00640 1 --2- 172.21.15.47:0/3021392815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 0x55fac75dd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.963+0000 7fa8afe00640 1 --2- 172.21.15.47:0/3021392815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75d3de0 0x55fac75dd9e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa89c02c040 tx=0x7fa89c0045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1562fd0ed61c2aed server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.963+0000 7fa896000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.966+0000 7fa896000640 1 -- 172.21.15.47:0/3021392815 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c03a070 con 0x55fac75d3de0 2024-03-21T11:39:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.966+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3021392815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa89c009e10 con 0x55fac75d3de0 2024-03-21T11:39:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.966+0000 7fa896000640 1 -- 172.21.15.47:0/3021392815 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa89c003a00 con 0x55fac75d3de0 2024-03-21T11:39:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.968+0000 7fa896000640 1 -- 172.21.15.47:0/3021392815 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa89c044050 con 0x55fac75d3de0 2024-03-21T11:39:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.968+0000 7fa8ba19d740 10 client.30959 fetch_fsmap learned FSMap version 39 2024-03-21T11:39:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.968+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3021392815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55fac7319620 con 0x55fac75d3de0 2024-03-21T11:39:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.970+0000 7fa896000640 1 -- 172.21.15.47:0/3021392815 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa89c009e10 con 0x55fac75d3de0 2024-03-21T11:39:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.971+0000 7fa896000640 1 -- 172.21.15.47:0/3021392815 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 39) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fa89c0053a0 con 0x55fac75d3de0 2024-03-21T11:39:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.971+0000 7fa8ba19d740 10 client.30959 fetch_fsmap finished waiting for FSMap version 39 2024-03-21T11:39:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.971+0000 7fa8ba19d740 10 client.30959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.971+0000 7fa8ba19d740 20 client.30959 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.971+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/3021392815 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fac75e9160 0x55fac75eb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.971+0000 7fa8ba19d740 4 client.30959 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.971+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3021392815 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fac7545d70 con 0x55fac75e9160 2024-03-21T11:39:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.972+0000 7fa8b4a00640 1 --2- 172.21.15.47:0/3021392815 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fac75e9160 0x55fac75eb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.973+0000 7fa8b4a00640 1 --2- 172.21.15.47:0/3021392815 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fac75e9160 0x55fac75eb550 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=511819cd462cf7af server_cookie=de89b4ffcb70c6c2 in_seq=0 out_seq=0 2024-03-21T11:39:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.973+0000 7fa896000640 10 client.30959 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.980+0000 7fa896000640 1 -- 172.21.15.47:0/3021392815 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fac7545d70 con 0x55fac75e9160 2024-03-21T11:39:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.980+0000 7fa896000640 10 client.30959 handle_command_reply: tid=0 2024-03-21T11:39:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.980+0000 7fa8ba19d740 1 client.30959 shutdown 2024-03-21T11:39:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.982+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3021392815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75d3de0 msgr2=0x55fac75dd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.982+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/3021392815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75d3de0 0x55fac75dd9e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa89c02c040 tx=0x7fa89c0045a0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.982+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3021392815 shutdown_connections 2024-03-21T11:39:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.982+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/3021392815 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fac75e9160 0x55fac75eb550 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.982+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/3021392815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 0x55fac75e2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.982+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/3021392815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75d3de0 0x55fac75dd9e0 secure :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa89c02c040 tx=0x7fa89c0045a0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.982+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/3021392815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 0x55fac75dd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.982+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3021392815 >> 172.21.15.47:0/3021392815 conn(0x55fac75ce960 msgr2=0x55fac75cdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.985+0000 7fa896000640 0 client.30959 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.987+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3021392815 shutdown_connections 2024-03-21T11:39:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.988+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3021392815 wait complete. 2024-03-21T11:39:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:47.989+0000 7fa8ba19d740 20 client.30959 trim_cache size 0 max 16384 2024-03-21T11:39:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.030+0000 7fa8ba19d740 1 Processor -- start 2024-03-21T11:39:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.030+0000 7fa8ba19d740 1 -- start start 2024-03-21T11:39:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.030+0000 7fa8ba19d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 0x55fac75d8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.030+0000 7fa8ba19d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 0x55fac75dd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.030+0000 7fa8ba19d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75dd9e0 0x55fac75d5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.031+0000 7fa8ba19d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b01a7d50 con 0x55fac75d8f50 2024-03-21T11:39:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.031+0000 7fa8ba19d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b01a9760 con 0x55fac75dd9e0 2024-03-21T11:39:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.031+0000 7fa8ba19d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b01a83b0 con 0x55fac75d33f0 2024-03-21T11:39:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.031+0000 7fa8afe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 0x55fac75dd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.032+0000 7fa8afe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 0x55fac75dd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46160/0 (socket says 172.21.15.47:46160) 2024-03-21T11:39:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.032+0000 7fa8afe00640 1 -- 172.21.15.47:0/882596122 learned_addr learned my addr 172.21.15.47:0/882596122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.032+0000 7fa8b4a00640 1 --2- 172.21.15.47:0/882596122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 0x55fac75d8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.032+0000 7fa8afe00640 1 -- 172.21.15.47:0/882596122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 msgr2=0x55fac75d8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.032+0000 7fa8afe00640 1 --2- 172.21.15.47:0/882596122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 0x55fac75d8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.032+0000 7fa8afe00640 1 -- 172.21.15.47:0/882596122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75dd9e0 msgr2=0x55fac75d5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.032+0000 7fa8afe00640 1 --2- 172.21.15.47:0/882596122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75dd9e0 0x55fac75d5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.032+0000 7fa8afe00640 1 -- 172.21.15.47:0/882596122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fac75d4670 con 0x55fac75d8f50 2024-03-21T11:39:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.035+0000 7fa8afe00640 1 --2- 172.21.15.47:0/882596122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 0x55fac75dd4a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55fac75de980 tx=0x7fa89c004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157ee7ab273bd021 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:48.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.037+0000 7f77df800640 1 -- 172.21.15.47:0/3295836901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8108880 msgr2=0x7f77d810c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:48.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.037+0000 7f77df800640 1 --2- 172.21.15.47:0/3295836901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8108880 0x7f77d810c230 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009900 tx=0x7f77cc02bc40 comp rx=0 tx=0).stop 2024-03-21T11:39:48.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.037+0000 7f77df800640 1 -- 172.21.15.47:0/3295836901 shutdown_connections 2024-03-21T11:39:48.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.037+0000 7f77df800640 1 --2- 172.21.15.47:0/3295836901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d8109640 0x7f77d8109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.037+0000 7f77df800640 1 --2- 172.21.15.47:0/3295836901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77d8108ca0 0x7f77d8109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.037+0000 7f77df800640 1 --2- 172.21.15.47:0/3295836901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8108880 0x7f77d810c230 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.037+0000 7f77df800640 1 -- 172.21.15.47:0/3295836901 >> 172.21.15.47:0/3295836901 conn(0x7f77d806e500 msgr2=0x7f77d8070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:48.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.037+0000 7f77df800640 1 -- 172.21.15.47:0/3295836901 shutdown_connections 2024-03-21T11:39:48.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.037+0000 7f77df800640 1 -- 172.21.15.47:0/3295836901 wait complete. 2024-03-21T11:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.038+0000 7fa8a1200640 1 -- 172.21.15.47:0/882596122 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c03a070 con 0x55fac75d8f50 2024-03-21T11:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.038+0000 7fa8a1200640 1 -- 172.21.15.47:0/882596122 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa89c011d10 con 0x55fac75d8f50 2024-03-21T11:39:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.038+0000 7fa8a1200640 1 -- 172.21.15.47:0/882596122 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c036dd0 con 0x55fac75d8f50 2024-03-21T11:39:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.039+0000 7fa8ba19d740 1 -- 172.21.15.47:0/882596122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 msgr2=0x55fac75dd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.039+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/882596122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 0x55fac75dd4a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55fac75de980 tx=0x7fa89c004440 comp rx=0 tx=0).stop 2024-03-21T11:39:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.041+0000 7fa8ba19d740 1 -- 172.21.15.47:0/882596122 shutdown_connections 2024-03-21T11:39:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.041+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/882596122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75dd9e0 0x55fac75d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.041+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/882596122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 0x55fac75dd4a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.041+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/882596122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 0x55fac75d8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.041+0000 7fa8ba19d740 1 -- 172.21.15.47:0/882596122 >> 172.21.15.47:0/882596122 conn(0x55fac75ce960 msgr2=0x55fac75ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:48.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.039+0000 7f77df800640 1 Processor -- start 2024-03-21T11:39:48.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.040+0000 7f77df800640 1 -- start start 2024-03-21T11:39:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.042+0000 7fa8ba19d740 1 -- 172.21.15.47:0/882596122 shutdown_connections 2024-03-21T11:39:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.042+0000 7fa8ba19d740 1 -- 172.21.15.47:0/882596122 wait complete. 2024-03-21T11:39:48.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.040+0000 7f77df800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77d8108880 0x7f77d819d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.040+0000 7f77df800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d8108ca0 0x7f77d819ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.040+0000 7f77df800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8109640 0x7f77d81a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.041+0000 7f77df800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d8117dc0 con 0x7f77d8108880 2024-03-21T11:39:48.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.041+0000 7f77df800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d8117c40 con 0x7f77d8108ca0 2024-03-21T11:39:48.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.041+0000 7f77df800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d8117f40 con 0x7f77d8109640 2024-03-21T11:39:48.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.041+0000 7f77dd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8109640 0x7f77d81a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.041+0000 7f77dd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8109640 0x7f77d81a4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48538/0 (socket says 172.21.15.47:48538) 2024-03-21T11:39:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.041+0000 7f77dd800640 1 -- 172.21.15.47:0/939614832 learned_addr learned my addr 172.21.15.47:0/939614832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.041+0000 7f77d7e00640 1 --2- 172.21.15.47:0/939614832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d8108ca0 0x7f77d819ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.041+0000 7f77dce00640 1 --2- 172.21.15.47:0/939614832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77d8108880 0x7f77d819d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.042+0000 7f77dd800640 1 -- 172.21.15.47:0/939614832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d8108ca0 msgr2=0x7f77d819ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.042+0000 7f77dd800640 1 --2- 172.21.15.47:0/939614832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d8108ca0 0x7f77d819ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.042+0000 7f77dd800640 1 -- 172.21.15.47:0/939614832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77d8108880 msgr2=0x7f77d819d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.042+0000 7f77dd800640 1 --2- 172.21.15.47:0/939614832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77d8108880 0x7f77d819d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.042+0000 7f77dd800640 1 -- 172.21.15.47:0/939614832 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d81a54a0 con 0x7f77d8109640 2024-03-21T11:39:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.042+0000 7f77dd800640 1 --2- 172.21.15.47:0/939614832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8109640 0x7f77d81a4e30 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f77d000b9e0 tx=0x7f77d000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.043+0000 7f77d5600640 1 -- 172.21.15.47:0/939614832 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0019070 con 0x7f77d8109640 2024-03-21T11:39:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.043+0000 7f77d5600640 1 -- 172.21.15.47:0/939614832 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77d0014440 con 0x7f77d8109640 2024-03-21T11:39:48.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.043+0000 7f77d5600640 1 -- 172.21.15.47:0/939614832 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d00125e0 con 0x7f77d8109640 2024-03-21T11:39:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.042+0000 7fa8ba19d740 1 -- start start 2024-03-21T11:39:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.043+0000 7fa8ba19d740 1 Processor -- start 2024-03-21T11:39:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.044+0000 7fa8ba19d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 0x55fac75f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.044+0000 7fa8ba19d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 0x55fac75fad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.044+0000 7fa8ba19d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75dd9e0 0x55fac75fb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.045+0000 7fa8ba19d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b01a9070 con 0x55fac75d8f50 2024-03-21T11:39:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.045+0000 7fa8ba19d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b01a8a10 con 0x55fac75dd9e0 2024-03-21T11:39:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.045+0000 7fa8ba19d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b01a5f10 con 0x55fac75d33f0 2024-03-21T11:39:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.045+0000 7fa8ba19d740 10 client.0 _ll_register_callbacks cb 0x7fa8a8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:48.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.045+0000 7f77df800640 1 -- 172.21.15.47:0/939614832 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d81a5790 con 0x7f77d8109640 2024-03-21T11:39:48.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.045+0000 7f77df800640 1 -- 172.21.15.47:0/939614832 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d81b7460 con 0x7f77d8109640 2024-03-21T11:39:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.045+0000 7fa8b5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75dd9e0 0x55fac75fb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.045+0000 7fa8b5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75dd9e0 0x55fac75fb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60722/0 (socket says 172.21.15.47:60722) 2024-03-21T11:39:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.045+0000 7fa8b5400640 1 -- 172.21.15.47:0/3432247795 learned_addr learned my addr 172.21.15.47:0/3432247795 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.046+0000 7fa8b4a00640 1 --2- 172.21.15.47:0/3432247795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 0x55fac75f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.046+0000 7fa8afe00640 1 --2- 172.21.15.47:0/3432247795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 0x55fac75fad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.046+0000 7fa8b5400640 1 -- 172.21.15.47:0/3432247795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 msgr2=0x55fac75f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.046+0000 7fa8b5400640 1 --2- 172.21.15.47:0/3432247795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 0x55fac75f87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.046+0000 7fa8b5400640 1 -- 172.21.15.47:0/3432247795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 msgr2=0x55fac75fad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.046+0000 7fa8b5400640 1 --2- 172.21.15.47:0/3432247795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 0x55fac75fad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.046+0000 7fa8b5400640 1 -- 172.21.15.47:0/3432247795 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fac75fb7a0 con 0x55fac75dd9e0 2024-03-21T11:39:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.046+0000 7fa8afe00640 1 --2- 172.21.15.47:0/3432247795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 0x55fac75fad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.047+0000 7fa8b5400640 1 --2- 172.21.15.47:0/3432247795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75dd9e0 0x55fac75fb260 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800faf0 tx=0x7fa8a800fef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fbe317220495bd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.047+0000 7fa896000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:39:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.048+0000 7fa896000640 1 -- 172.21.15.47:0/3432247795 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a801ea40 con 0x55fac75dd9e0 2024-03-21T11:39:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.048+0000 7fa896000640 1 -- 172.21.15.47:0/3432247795 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa8a801ebe0 con 0x55fac75dd9e0 2024-03-21T11:39:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.048+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3432247795 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa89c011d10 con 0x55fac75dd9e0 2024-03-21T11:39:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.048+0000 7fa896000640 1 -- 172.21.15.47:0/3432247795 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8a801ede0 con 0x55fac75dd9e0 2024-03-21T11:39:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.048+0000 7fa8ba19d740 10 client.30965 fetch_fsmap learned FSMap version 40 2024-03-21T11:39:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.048+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3432247795 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55fac7655dc0 con 0x55fac75dd9e0 2024-03-21T11:39:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.052+0000 7fa896000640 1 -- 172.21.15.47:0/3432247795 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa89c011d10 con 0x55fac75dd9e0 2024-03-21T11:39:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.052+0000 7fa896000640 1 -- 172.21.15.47:0/3432247795 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa8a801c540 con 0x55fac75dd9e0 2024-03-21T11:39:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.052+0000 7fa8ba19d740 10 client.30965 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:39:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.053+0000 7fa8ba19d740 10 client.30965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.053+0000 7fa8ba19d740 20 client.30965 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.053+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/3432247795 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fac7656780 0x55fac7658ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.053+0000 7fa8ba19d740 4 client.30965 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:39:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.053+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3432247795 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fac75400c0 con 0x55fac7656780 2024-03-21T11:39:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.055+0000 7fa8b4a00640 1 --2- 172.21.15.47:0/3432247795 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fac7656780 0x55fac7658ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.053+0000 7f77d5600640 1 -- 172.21.15.47:0/939614832 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f77d001c070 con 0x7f77d8109640 2024-03-21T11:39:48.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.055+0000 7f77e2586740 1 Processor -- start 2024-03-21T11:39:48.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.055+0000 7f77e2586740 1 -- start start 2024-03-21T11:39:48.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.055+0000 7f77e2586740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf344e80 0x558bbf3452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.055+0000 7f77e2586740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 0x558bbf34a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.056+0000 7f77e2586740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf34a9e0 0x558bbf34ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.056+0000 7f77e2586740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbf2aa250 con 0x558bbf345870 2024-03-21T11:39:48.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.056+0000 7f77e2586740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbf252a30 con 0x558bbf34a9e0 2024-03-21T11:39:48.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.056+0000 7f77e2586740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbf2a62f0 con 0x558bbf344e80 2024-03-21T11:39:48.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.056+0000 7f77d7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf344e80 0x558bbf3452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.056+0000 7f77d7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf344e80 0x558bbf3452a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48542/0 (socket says 172.21.15.47:48542) 2024-03-21T11:39:48.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.056+0000 7f77d7e00640 1 -- 172.21.15.47:0/968060488 learned_addr learned my addr 172.21.15.47:0/968060488 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:48.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.056+0000 7f77d7e00640 1 --2- 172.21.15.47:0/968060488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf34a9e0 0x558bbf34ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.057+0000 7f77d7e00640 1 -- 172.21.15.47:0/968060488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf34a9e0 msgr2=0x558bbf34ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.057+0000 7f77d7e00640 1 --2- 172.21.15.47:0/968060488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf34a9e0 0x558bbf34ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.057+0000 7f77d7e00640 1 -- 172.21.15.47:0/968060488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 msgr2=0x558bbf34a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.057+0000 7f77d7e00640 1 --2- 172.21.15.47:0/968060488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 0x558bbf34a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.057+0000 7f77d7e00640 1 -- 172.21.15.47:0/968060488 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bbf23d690 con 0x558bbf344e80 2024-03-21T11:39:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.057+0000 7f77d7e00640 1 --2- 172.21.15.47:0/968060488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf344e80 0x558bbf3452a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f77c40058c0 tx=0x7f77c4005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ff102802ff2ed79 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:48.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.057+0000 7f77d5600640 1 --2- 172.21.15.47:0/939614832 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f77c00a2fa0 0x7f77c00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.058+0000 7fa8b4a00640 1 --2- 172.21.15.47:0/3432247795 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fac7656780 0x55fac7658ba0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af603c30f0a13a3f server_cookie=d7136a5e9b0e87eb in_seq=0 out_seq=0 2024-03-21T11:39:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.058+0000 7fa896000640 10 client.30965 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.058+0000 7f77ca000640 1 -- 172.21.15.47:0/968060488 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4004030 con 0x558bbf344e80 2024-03-21T11:39:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.058+0000 7f77ca000640 1 -- 172.21.15.47:0/968060488 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77c4005500 con 0x558bbf344e80 2024-03-21T11:39:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.058+0000 7f77ca000640 1 -- 172.21.15.47:0/968060488 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4004ce0 con 0x558bbf344e80 2024-03-21T11:39:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.059+0000 7f77dce00640 1 --2- 172.21.15.47:0/939614832 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f77c00a2fa0 0x7f77c00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.059+0000 7f77e2586740 1 -- 172.21.15.47:0/968060488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf344e80 msgr2=0x558bbf3452a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.059+0000 7f77e2586740 1 --2- 172.21.15.47:0/968060488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf344e80 0x558bbf3452a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f77c40058c0 tx=0x7f77c4005850 comp rx=0 tx=0).stop 2024-03-21T11:39:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.059+0000 7f77d5600640 1 -- 172.21.15.47:0/939614832 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f77d00cfd20 con 0x7f77d8109640 2024-03-21T11:39:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.059+0000 7f77e2586740 1 -- 172.21.15.47:0/968060488 shutdown_connections 2024-03-21T11:39:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.059+0000 7f77e2586740 1 --2- 172.21.15.47:0/968060488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf34a9e0 0x558bbf34ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.059+0000 7f77e2586740 1 --2- 172.21.15.47:0/968060488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 0x558bbf34a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.059+0000 7f77e2586740 1 --2- 172.21.15.47:0/968060488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf344e80 0x558bbf3452a0 secure :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f77c40058c0 tx=0x7f77c4005850 comp rx=0 tx=0).stop 2024-03-21T11:39:48.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.059+0000 7f77e2586740 1 -- 172.21.15.47:0/968060488 >> 172.21.15.47:0/968060488 conn(0x558bbf30d200 msgr2=0x558bbf3407f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:48.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.061+0000 7f77dce00640 1 --2- 172.21.15.47:0/939614832 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f77c00a2fa0 0x7f77c00a5460 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f77cc02c250 tx=0x7f77cc02cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:48.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.062+0000 7f77e2586740 1 -- 172.21.15.47:0/968060488 shutdown_connections 2024-03-21T11:39:48.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.062+0000 7f77e2586740 1 -- 172.21.15.47:0/968060488 wait complete. 2024-03-21T11:39:48.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.062+0000 7f77e2586740 1 -- start start 2024-03-21T11:39:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.063+0000 7f77e2586740 1 Processor -- start 2024-03-21T11:39:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.064+0000 7f77e2586740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 0x558bbf34f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.064+0000 7f77e2586740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf34a9e0 0x558bbf34f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.064+0000 7f77e2586740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf354570 0x558bbf356960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.064+0000 7f77e2586740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbf34d990 con 0x558bbf345870 2024-03-21T11:39:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.064+0000 7f77e2586740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbf265e80 con 0x558bbf34a9e0 2024-03-21T11:39:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.064+0000 7f77e2586740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbf34db10 con 0x558bbf354570 2024-03-21T11:39:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.064+0000 7f77e2586740 10 client.0 _ll_register_callbacks cb 0x558bbf278130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.065+0000 7f77dd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf354570 0x558bbf356960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.065+0000 7f77dd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf354570 0x558bbf356960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48544/0 (socket says 172.21.15.47:48544) 2024-03-21T11:39:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.065+0000 7f77dd800640 1 -- 172.21.15.47:0/1114089052 learned_addr learned my addr 172.21.15.47:0/1114089052 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.065+0000 7f77dce00640 1 --2- 172.21.15.47:0/1114089052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 0x558bbf34f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.065+0000 7f77d7e00640 1 --2- 172.21.15.47:0/1114089052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf34a9e0 0x558bbf34f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.066+0000 7f77dd800640 1 -- 172.21.15.47:0/1114089052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf34a9e0 msgr2=0x558bbf34f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.066+0000 7f77dd800640 1 --2- 172.21.15.47:0/1114089052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf34a9e0 0x558bbf34f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.066+0000 7f77dd800640 1 -- 172.21.15.47:0/1114089052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 msgr2=0x558bbf34f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.066+0000 7f77dd800640 1 --2- 172.21.15.47:0/1114089052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 0x558bbf34f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.066+0000 7f77dd800640 1 -- 172.21.15.47:0/1114089052 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bbf0887d0 con 0x558bbf354570 2024-03-21T11:39:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.067+0000 7f77dd800640 1 --2- 172.21.15.47:0/1114089052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf354570 0x558bbf356960 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f77d00a4540 tx=0x7f77d00a02f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d87091742b017a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.067+0000 7f77bea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.067+0000 7f77bea00640 1 -- 172.21.15.47:0/1114089052 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d00bc070 con 0x558bbf354570 2024-03-21T11:39:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.067+0000 7f77bea00640 1 -- 172.21.15.47:0/1114089052 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f77d00bc210 con 0x558bbf354570 2024-03-21T11:39:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.067+0000 7f77e2586740 1 -- 172.21.15.47:0/1114089052 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77c4005500 con 0x558bbf354570 2024-03-21T11:39:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.068+0000 7f77bea00640 1 -- 172.21.15.47:0/1114089052 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77d00a45e0 con 0x558bbf354570 2024-03-21T11:39:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.068+0000 7f77e2586740 10 client.30961 fetch_fsmap learned FSMap version 40 2024-03-21T11:39:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.068+0000 7f77e2586740 1 -- 172.21.15.47:0/1114089052 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558bbefbe3f0 con 0x558bbf354570 2024-03-21T11:39:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.068+0000 7f77bea00640 1 -- 172.21.15.47:0/1114089052 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77c4005500 con 0x558bbf354570 2024-03-21T11:39:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.069+0000 7f77bea00640 1 -- 172.21.15.47:0/1114089052 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f77d009b560 con 0x558bbf354570 2024-03-21T11:39:48.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.069+0000 7f77e2586740 10 client.30961 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:39:48.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.069+0000 7f77e2586740 10 client.30961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:48.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.069+0000 7f77e2586740 20 client.30961 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:48.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.069+0000 7f77e2586740 1 --2- 172.21.15.47:0/1114089052 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558bbf35b220 0x558bbf35d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.069+0000 7f77e2586740 4 client.30961 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:39:48.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.069+0000 7f77e2586740 1 -- 172.21.15.47:0/1114089052 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bbf2b7790 con 0x558bbf35b220 2024-03-21T11:39:48.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.070+0000 7f77dce00640 1 --2- 172.21.15.47:0/1114089052 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558bbf35b220 0x558bbf35d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.071+0000 7f77dce00640 1 --2- 172.21.15.47:0/1114089052 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558bbf35b220 0x558bbf35d650 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5463afa767b4da14 server_cookie=1627cd54c59431f3 in_seq=0 out_seq=0 2024-03-21T11:39:48.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.071+0000 7f77bea00640 10 client.30961 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.690+0000 7fa896000640 1 -- 172.21.15.47:0/3432247795 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fac75400c0 con 0x55fac7656780 2024-03-21T11:39:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.690+0000 7fa896000640 10 client.30965 handle_command_reply: tid=0 2024-03-21T11:39:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.690+0000 7fa8ba19d740 1 client.30965 shutdown 2024-03-21T11:39:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.690+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3432247795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75dd9e0 msgr2=0x55fac75fb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.690+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/3432247795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75dd9e0 0x55fac75fb260 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800faf0 tx=0x7fa8a800fef0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.690+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3432247795 shutdown_connections 2024-03-21T11:39:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.690+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/3432247795 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fac7656780 0x55fac7658ba0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.690+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/3432247795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fac75dd9e0 0x55fac75fb260 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.690+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/3432247795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fac75d8f50 0x55fac75fad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.690+0000 7fa8ba19d740 1 --2- 172.21.15.47:0/3432247795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fac75d33f0 0x55fac75f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.690+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3432247795 >> 172.21.15.47:0/3432247795 conn(0x55fac75ce960 msgr2=0x55fac75dca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.691+0000 7fa896000640 0 client.30965 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:48.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.691+0000 7f77bea00640 1 -- 172.21.15.47:0/1114089052 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558bbf2b7790 con 0x558bbf35b220 2024-03-21T11:39:48.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.691+0000 7f77bea00640 10 client.30961 handle_command_reply: tid=0 2024-03-21T11:39:48.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.691+0000 7f77e2586740 1 client.30961 shutdown 2024-03-21T11:39:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.691+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3432247795 shutdown_connections 2024-03-21T11:39:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.691+0000 7fa8ba19d740 1 -- 172.21.15.47:0/3432247795 wait complete. 2024-03-21T11:39:48.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.691+0000 7f77e2586740 1 -- 172.21.15.47:0/1114089052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf354570 msgr2=0x558bbf356960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:48.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.691+0000 7f77e2586740 1 --2- 172.21.15.47:0/1114089052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf354570 0x558bbf356960 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f77d00a4540 tx=0x7f77d00a02f0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.692+0000 7fa8ba19d740 20 client.30965 trim_cache size 0 max 16384 2024-03-21T11:39:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.692+0000 7f77e2586740 1 -- 172.21.15.47:0/1114089052 shutdown_connections 2024-03-21T11:39:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.692+0000 7f77e2586740 1 --2- 172.21.15.47:0/1114089052 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558bbf35b220 0x558bbf35d650 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.692+0000 7f77e2586740 1 --2- 172.21.15.47:0/1114089052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf354570 0x558bbf356960 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.692+0000 7f77e2586740 1 --2- 172.21.15.47:0/1114089052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf34a9e0 0x558bbf34f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.692+0000 7f77e2586740 1 --2- 172.21.15.47:0/1114089052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 0x558bbf34f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.692+0000 7f77e2586740 1 -- 172.21.15.47:0/1114089052 >> 172.21.15.47:0/1114089052 conn(0x558bbf30d200 msgr2=0x558bbf33fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.692+0000 7f77bea00640 0 client.30961 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.692+0000 7f77e2586740 1 -- 172.21.15.47:0/1114089052 shutdown_connections 2024-03-21T11:39:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.692+0000 7f77e2586740 1 -- 172.21.15.47:0/1114089052 wait complete. 2024-03-21T11:39:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.692+0000 7f77e2586740 20 client.30961 trim_cache size 0 max 16384 2024-03-21T11:39:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:39:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:39:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:39:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:39:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.693+0000 7fa8b5e00640 1 -- 172.21.15.47:0/139035273 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8980a2f10 msgr2=0x7fa8980a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.693+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/139035273 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8980a2f10 0x7fa8980a53d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009fd0 tx=0x7fa8a8004a40 comp rx=0 tx=0).stop 2024-03-21T11:39:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.694+0000 7fa8b5e00640 1 -- 172.21.15.47:0/139035273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8b0101f70 msgr2=0x7fa8b0199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.694+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/139035273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8b0101f70 0x7fa8b0199a00 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4000c00 tx=0x7fa8a402c650 comp rx=0 tx=0).stop 2024-03-21T11:39:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.694+0000 7fa8b5e00640 1 -- 172.21.15.47:0/139035273 shutdown_connections 2024-03-21T11:39:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.694+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/139035273 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa8980a2f10 0x7fa8980a53d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.694+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/139035273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b01071d0 0x7fa8b01a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.694+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/139035273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b01048a0 0x7fa8b0199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.694+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/139035273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8b0101f70 0x7fa8b0199a00 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.694+0000 7fa8b5e00640 1 -- 172.21.15.47:0/139035273 >> 172.21.15.47:0/139035273 conn(0x7fa8b00fb980 msgr2=0x7fa8b01016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.694+0000 7fa8b5e00640 1 -- 172.21.15.47:0/139035273 shutdown_connections 2024-03-21T11:39:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:39:48.695+0000 7fa8b5e00640 1 -- 172.21.15.47:0/139035273 wait complete. 2024-03-21T11:39:48.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.731+0000 7f77e2586740 1 Processor -- start 2024-03-21T11:39:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.731+0000 7f77e2586740 1 -- start start 2024-03-21T11:39:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.731+0000 7f77e2586740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 0x558bbf34e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.731+0000 7f77e2586740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf34a9e0 0x558bbf3462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.731+0000 7f77e2586740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf354570 0x558bbf346820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.732+0000 7f77e2586740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81c1720 con 0x558bbf345870 2024-03-21T11:39:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.732+0000 7f77e2586740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81c30a0 con 0x558bbf354570 2024-03-21T11:39:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.732+0000 7f77e2586740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81c1d80 con 0x558bbf34a9e0 2024-03-21T11:39:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.732+0000 7f77dce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 0x558bbf34e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.732+0000 7f77dce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 0x558bbf34e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46214/0 (socket says 172.21.15.47:46214) 2024-03-21T11:39:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.732+0000 7f77dce00640 1 -- 172.21.15.47:0/1768674321 learned_addr learned my addr 172.21.15.47:0/1768674321 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.732+0000 7f77d7e00640 1 --2- 172.21.15.47:0/1768674321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf34a9e0 0x558bbf3462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.732+0000 7f77dce00640 1 -- 172.21.15.47:0/1768674321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf34a9e0 msgr2=0x558bbf3462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.732+0000 7f77dce00640 1 --2- 172.21.15.47:0/1768674321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf34a9e0 0x558bbf3462e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.732+0000 7f77dce00640 1 -- 172.21.15.47:0/1768674321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf354570 msgr2=0x558bbf346820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.732+0000 7f77dce00640 1 --2- 172.21.15.47:0/1768674321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf354570 0x558bbf346820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.732+0000 7f77dce00640 1 -- 172.21.15.47:0/1768674321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bbef7e0f0 con 0x558bbf345870 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.733+0000 7f77dce00640 1 --2- 172.21.15.47:0/1768674321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 0x558bbf34e5c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f77cc02cca0 tx=0x7f77cc03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbeae6bca5fc89a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.733+0000 7f77c9600640 1 -- 172.21.15.47:0/1768674321 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc059070 con 0x558bbf345870 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.733+0000 7f77c9600640 1 -- 172.21.15.47:0/1768674321 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77cc0046c0 con 0x558bbf345870 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.733+0000 7f77c9600640 1 -- 172.21.15.47:0/1768674321 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc003850 con 0x558bbf345870 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.734+0000 7f77e2586740 1 -- 172.21.15.47:0/1768674321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 msgr2=0x558bbf34e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.734+0000 7f77e2586740 1 --2- 172.21.15.47:0/1768674321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 0x558bbf34e5c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f77cc02cca0 tx=0x7f77cc03e000 comp rx=0 tx=0).stop 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.734+0000 7f77e2586740 1 -- 172.21.15.47:0/1768674321 shutdown_connections 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.734+0000 7f77e2586740 1 --2- 172.21.15.47:0/1768674321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf354570 0x558bbf346820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.734+0000 7f77e2586740 1 --2- 172.21.15.47:0/1768674321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf34a9e0 0x558bbf3462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.734+0000 7f77e2586740 1 --2- 172.21.15.47:0/1768674321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf345870 0x558bbf34e5c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.734+0000 7f77e2586740 1 -- 172.21.15.47:0/1768674321 >> 172.21.15.47:0/1768674321 conn(0x558bbf30d200 msgr2=0x558bbf34deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:48.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.734+0000 7f77e2586740 1 -- 172.21.15.47:0/1768674321 shutdown_connections 2024-03-21T11:39:48.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.734+0000 7f77e2586740 1 -- 172.21.15.47:0/1768674321 wait complete. 2024-03-21T11:39:48.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.735+0000 7f77e2586740 1 -- start start 2024-03-21T11:39:48.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.735+0000 7f77e2586740 1 Processor -- start 2024-03-21T11:39:48.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.738+0000 7f77e2586740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf34a9e0 0x558bbf36d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.738+0000 7f77e2586740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf354570 0x558bbf36bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.738+0000 7f77e2586740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf36c1b0 0x558bbf36c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.738+0000 7f77e2586740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81c2a40 con 0x558bbf354570 2024-03-21T11:39:48.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.738+0000 7f77e2586740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81c23e0 con 0x558bbf36c1b0 2024-03-21T11:39:48.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.738+0000 7f77e2586740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81bf8e0 con 0x558bbf34a9e0 2024-03-21T11:39:48.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.738+0000 7f77e2586740 10 client.0 _ll_register_callbacks cb 0x7f77d0098430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:39:48.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.738+0000 7f77dce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf34a9e0 0x558bbf36d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.739+0000 7f77dce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf34a9e0 0x558bbf36d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48570/0 (socket says 172.21.15.47:48570) 2024-03-21T11:39:48.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.739+0000 7f77dce00640 1 -- 172.21.15.47:0/3577870925 learned_addr learned my addr 172.21.15.47:0/3577870925 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:39:48.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.738+0000 7f77d7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf354570 0x558bbf36bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.738+0000 7f77dd800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf36c1b0 0x558bbf36c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.739+0000 7f77dce00640 1 -- 172.21.15.47:0/3577870925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf36c1b0 msgr2=0x558bbf36c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:48.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.739+0000 7f77dce00640 1 --2- 172.21.15.47:0/3577870925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf36c1b0 0x558bbf36c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.739+0000 7f77dce00640 1 -- 172.21.15.47:0/3577870925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf354570 msgr2=0x558bbf36bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:39:48.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.739+0000 7f77dce00640 1 --2- 172.21.15.47:0/3577870925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf354570 0x558bbf36bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.739+0000 7f77dce00640 1 -- 172.21.15.47:0/3577870925 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bbf3b6040 con 0x558bbf34a9e0 2024-03-21T11:39:48.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.739+0000 7f77dd800640 1 --2- 172.21.15.47:0/3577870925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf36c1b0 0x558bbf36c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:39:48.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.740+0000 7f77dce00640 1 --2- 172.21.15.47:0/3577870925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf34a9e0 0x558bbf36d660 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f77cc005ec0 tx=0x7f77cc0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbb3f6ad79437817 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:39:48.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.740+0000 7f77bea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:39:48.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.740+0000 7f77bea00640 1 -- 172.21.15.47:0/3577870925 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc059070 con 0x558bbf34a9e0 2024-03-21T11:39:48.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.741+0000 7f77bea00640 1 -- 172.21.15.47:0/3577870925 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f77cc005200 con 0x558bbf34a9e0 2024-03-21T11:39:48.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.741+0000 7f77e2586740 1 -- 172.21.15.47:0/3577870925 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77cc0046c0 con 0x558bbf34a9e0 2024-03-21T11:39:48.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.741+0000 7f77bea00640 1 -- 172.21.15.47:0/3577870925 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77cc067050 con 0x558bbf34a9e0 2024-03-21T11:39:48.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.741+0000 7f77e2586740 10 client.30967 fetch_fsmap learned FSMap version 40 2024-03-21T11:39:48.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.741+0000 7f77e2586740 1 -- 172.21.15.47:0/3577870925 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558bbf3b7330 con 0x558bbf34a9e0 2024-03-21T11:39:48.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.742+0000 7f77bea00640 1 -- 172.21.15.47:0/3577870925 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77cc0046c0 con 0x558bbf34a9e0 2024-03-21T11:39:48.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.742+0000 7f77bea00640 1 -- 172.21.15.47:0/3577870925 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f77cc053070 con 0x558bbf34a9e0 2024-03-21T11:39:48.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.742+0000 7f77e2586740 10 client.30967 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:39:48.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.743+0000 7f77e2586740 10 client.30967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:39:48.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.743+0000 7f77e2586740 20 client.30967 populate_metadata read hostname 'smithi047' 2024-03-21T11:39:48.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.743+0000 7f77e2586740 1 --2- 172.21.15.47:0/3577870925 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558bbf3b7a40 0x558bbf3b9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:39:48.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.743+0000 7f77e2586740 4 client.30967 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:39:48.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.743+0000 7f77e2586740 1 -- 172.21.15.47:0/3577870925 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558bbf2b1ae0 con 0x558bbf3b7a40 2024-03-21T11:39:48.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.743+0000 7f77d7e00640 1 --2- 172.21.15.47:0/3577870925 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558bbf3b7a40 0x558bbf3b9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:39:48.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.746+0000 7f77d7e00640 1 --2- 172.21.15.47:0/3577870925 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558bbf3b7a40 0x558bbf3b9e60 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df6a5882000b913c server_cookie=9a93b84f46065f95 in_seq=0 out_seq=0 2024-03-21T11:39:48.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.746+0000 7f77bea00640 10 client.30967 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:48.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.751+0000 7f77bea00640 1 -- 172.21.15.47:0/3577870925 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558bbf2b1ae0 con 0x558bbf3b7a40 2024-03-21T11:39:48.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.751+0000 7f77bea00640 10 client.30967 handle_command_reply: tid=0 2024-03-21T11:39:48.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.751+0000 7f77e2586740 1 client.30967 shutdown 2024-03-21T11:39:48.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.752+0000 7f77e2586740 1 -- 172.21.15.47:0/3577870925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf34a9e0 msgr2=0x558bbf36d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:48.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.752+0000 7f77e2586740 1 --2- 172.21.15.47:0/3577870925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf34a9e0 0x558bbf36d660 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f77cc005ec0 tx=0x7f77cc0048f0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.752+0000 7f77e2586740 1 -- 172.21.15.47:0/3577870925 shutdown_connections 2024-03-21T11:39:48.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.752+0000 7f77e2586740 1 --2- 172.21.15.47:0/3577870925 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558bbf3b7a40 0x558bbf3b9e60 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.752+0000 7f77e2586740 1 --2- 172.21.15.47:0/3577870925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558bbf36c1b0 0x558bbf36c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.752+0000 7f77e2586740 1 --2- 172.21.15.47:0/3577870925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558bbf354570 0x558bbf36bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.752+0000 7f77e2586740 1 --2- 172.21.15.47:0/3577870925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558bbf34a9e0 0x558bbf36d660 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.752+0000 7f77bea00640 0 client.30967 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:39:48.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.752+0000 7f77e2586740 1 -- 172.21.15.47:0/3577870925 >> 172.21.15.47:0/3577870925 conn(0x558bbf30d200 msgr2=0x558bbf33f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:48.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.752+0000 7f77e2586740 1 -- 172.21.15.47:0/3577870925 shutdown_connections 2024-03-21T11:39:48.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.752+0000 7f77e2586740 1 -- 172.21.15.47:0/3577870925 wait complete. 2024-03-21T11:39:48.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.753+0000 7f77e2586740 20 client.30967 trim_cache size 0 max 16384 2024-03-21T11:39:48.754 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:39:48.755 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T11:39:48.755 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:39:48.755 INFO:teuthology.orchestra.run.smithi047.stdout: "f6ad4206-317d-420f-8914-31e9aadc9937": { 2024-03-21T11:39:48.755 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:39:48.755 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "f6ad4206-317d-420f-8914-31e9aadc9937", 2024-03-21T11:39:48.755 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:39:48.755 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:39:48.755 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:39:48.755 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:39:48.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.755+0000 7f77df800640 1 -- 172.21.15.47:0/939614832 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f77c00a2fa0 msgr2=0x7f77c00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:48.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.755+0000 7f77df800640 1 --2- 172.21.15.47:0/939614832 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f77c00a2fa0 0x7f77c00a5460 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f77cc02c250 tx=0x7f77cc02cf90 comp rx=0 tx=0).stop 2024-03-21T11:39:48.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.755+0000 7f77df800640 1 -- 172.21.15.47:0/939614832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8109640 msgr2=0x7f77d81a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:39:48.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.755+0000 7f77df800640 1 --2- 172.21.15.47:0/939614832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8109640 0x7f77d81a4e30 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f77d000b9e0 tx=0x7f77d000bea0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.755+0000 7f77df800640 1 -- 172.21.15.47:0/939614832 shutdown_connections 2024-03-21T11:39:48.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.755+0000 7f77df800640 1 --2- 172.21.15.47:0/939614832 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f77c00a2fa0 0x7f77c00a5460 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.755+0000 7f77df800640 1 --2- 172.21.15.47:0/939614832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8109640 0x7f77d81a4e30 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.755+0000 7f77df800640 1 --2- 172.21.15.47:0/939614832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77d8108ca0 0x7f77d819ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.755+0000 7f77df800640 1 --2- 172.21.15.47:0/939614832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77d8108880 0x7f77d819d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:39:48.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.755+0000 7f77df800640 1 -- 172.21.15.47:0/939614832 >> 172.21.15.47:0/939614832 conn(0x7f77d806e500 msgr2=0x7f77d810e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:39:48.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.755+0000 7f77df800640 1 -- 172.21.15.47:0/939614832 shutdown_connections 2024-03-21T11:39:48.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:39:48.755+0000 7f77df800640 1 -- 172.21.15.47:0/939614832 wait complete. 2024-03-21T11:39:48.771 INFO:tasks.cephfs.filesystem:scrub status for tag:f6ad4206-317d-420f-8914-31e9aadc9937 - {'path': '/', 'tag': 'f6ad4206-317d-420f-8914-31e9aadc9937', 'options': 'recursive,force'} 2024-03-21T11:39:48.935 INFO:tasks.workunit.client.0.smithi047.stdout: 10 72952 24.47 MB/sec execute 32 sec latency 424.773 ms 2024-03-21T11:39:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: pgmap v894: 129 pgs: 129 active+clean; 358 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 792 op/s 2024-03-21T11:39:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:39:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:49.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:49.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:49.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:49.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:49.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:49.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:49.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:49.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:39:49.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:49.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:49 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: pgmap v894: 129 pgs: 129 active+clean; 358 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 792 op/s 2024-03-21T11:39:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:39:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:49.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:39:49.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:49.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:49 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: pgmap v894: 129 pgs: 129 active+clean; 358 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 792 op/s 2024-03-21T11:39:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:39:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:49.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:49.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:39:49.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:39:49.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:49 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:49.935 INFO:tasks.workunit.client.0.smithi047.stdout: 10 73461 24.55 MB/sec execute 33 sec latency 98.888 ms 2024-03-21T11:39:50.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:50 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:50.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:50 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:50.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:50 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:39:50.935 INFO:tasks.workunit.client.0.smithi047.stdout: 10 74094 24.78 MB/sec execute 34 sec latency 76.363 ms 2024-03-21T11:39:51.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:51 smithi047 ceph-mon[31351]: pgmap v895: 129 pgs: 129 active+clean; 365 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.8 MiB/s wr, 781 op/s 2024-03-21T11:39:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:51 smithi100 ceph-mon[36245]: pgmap v895: 129 pgs: 129 active+clean; 365 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.8 MiB/s wr, 781 op/s 2024-03-21T11:39:51.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:51 smithi102 ceph-mon[36748]: pgmap v895: 129 pgs: 129 active+clean; 365 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.8 MiB/s wr, 781 op/s 2024-03-21T11:39:51.936 INFO:tasks.workunit.client.0.smithi047.stdout: 10 74587 24.98 MB/sec execute 35 sec latency 144.103 ms 2024-03-21T11:39:52.937 INFO:tasks.workunit.client.0.smithi047.stdout: 10 75162 25.07 MB/sec execute 36 sec latency 158.029 ms 2024-03-21T11:39:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:53 smithi047 ceph-mon[31351]: pgmap v896: 129 pgs: 129 active+clean; 366 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.9 MiB/s wr, 593 op/s 2024-03-21T11:39:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:53 smithi100 ceph-mon[36245]: pgmap v896: 129 pgs: 129 active+clean; 366 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.9 MiB/s wr, 593 op/s 2024-03-21T11:39:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:53 smithi102 ceph-mon[36748]: pgmap v896: 129 pgs: 129 active+clean; 366 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.9 MiB/s wr, 593 op/s 2024-03-21T11:39:53.937 INFO:tasks.workunit.client.0.smithi047.stdout: 10 75703 25.33 MB/sec execute 37 sec latency 186.300 ms 2024-03-21T11:39:54.937 INFO:tasks.workunit.client.0.smithi047.stdout: 10 76140 25.24 MB/sec execute 38 sec latency 99.873 ms 2024-03-21T11:39:55.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:55 smithi100 ceph-mon[36245]: pgmap v897: 129 pgs: 129 active+clean; 365 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.2 MiB/s wr, 687 op/s 2024-03-21T11:39:55.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:55 smithi102 ceph-mon[36748]: pgmap v897: 129 pgs: 129 active+clean; 365 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.2 MiB/s wr, 687 op/s 2024-03-21T11:39:55.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:55 smithi047 ceph-mon[31351]: pgmap v897: 129 pgs: 129 active+clean; 365 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.2 MiB/s wr, 687 op/s 2024-03-21T11:39:55.937 INFO:tasks.workunit.client.0.smithi047.stdout: 10 76626 25.35 MB/sec execute 39 sec latency 45.785 ms 2024-03-21T11:39:56.937 INFO:tasks.workunit.client.0.smithi047.stdout: 10 77049 25.14 MB/sec execute 40 sec latency 100.542 ms 2024-03-21T11:39:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:57 smithi100 ceph-mon[36245]: pgmap v898: 129 pgs: 129 active+clean; 368 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.8 MiB/s wr, 531 op/s 2024-03-21T11:39:57.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:57 smithi102 ceph-mon[36748]: pgmap v898: 129 pgs: 129 active+clean; 368 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.8 MiB/s wr, 531 op/s 2024-03-21T11:39:57.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:57 smithi047 ceph-mon[31351]: pgmap v898: 129 pgs: 129 active+clean; 368 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.8 MiB/s wr, 531 op/s 2024-03-21T11:39:57.937 INFO:tasks.workunit.client.0.smithi047.stdout: 10 77577 25.09 MB/sec execute 41 sec latency 95.624 ms 2024-03-21T11:39:58.937 INFO:tasks.workunit.client.0.smithi047.stdout: 10 78221 25.32 MB/sec execute 42 sec latency 63.389 ms 2024-03-21T11:39:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:39:59 smithi100 ceph-mon[36245]: pgmap v899: 129 pgs: 129 active+clean; 369 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 508 op/s 2024-03-21T11:39:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:39:59 smithi102 ceph-mon[36748]: pgmap v899: 129 pgs: 129 active+clean; 369 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 508 op/s 2024-03-21T11:39:59.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:39:59 smithi047 ceph-mon[31351]: pgmap v899: 129 pgs: 129 active+clean; 369 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 508 op/s 2024-03-21T11:39:59.937 INFO:tasks.workunit.client.0.smithi047.stdout: 10 78917 25.61 MB/sec execute 43 sec latency 87.264 ms 2024-03-21T11:40:00.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:00 smithi100 ceph-mon[36245]: overall HEALTH_OK 2024-03-21T11:40:00.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:00 smithi102 ceph-mon[36748]: overall HEALTH_OK 2024-03-21T11:40:00.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:00 smithi047 ceph-mon[31351]: overall HEALTH_OK 2024-03-21T11:40:00.937 INFO:tasks.workunit.client.0.smithi047.stdout: 10 79509 25.97 MB/sec execute 44 sec latency 94.363 ms 2024-03-21T11:40:01.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:01 smithi100 ceph-mon[36245]: pgmap v900: 129 pgs: 129 active+clean; 381 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.4 MiB/s wr, 585 op/s 2024-03-21T11:40:01.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:01 smithi047 ceph-mon[31351]: pgmap v900: 129 pgs: 129 active+clean; 381 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.4 MiB/s wr, 585 op/s 2024-03-21T11:40:01.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:01 smithi102 ceph-mon[36748]: pgmap v900: 129 pgs: 129 active+clean; 381 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.4 MiB/s wr, 585 op/s 2024-03-21T11:40:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.880+0000 7f842ee00640 1 -- 172.21.15.47:0/3351505319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8428101510 msgr2=0x7f8428101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.880+0000 7f842ee00640 1 --2- 172.21.15.47:0/3351505319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8428101510 0x7f8428101930 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f841c009900 tx=0x7f841c02bc50 comp rx=0 tx=0).stop 2024-03-21T11:40:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.881+0000 7f842ee00640 1 -- 172.21.15.47:0/3351505319 shutdown_connections 2024-03-21T11:40:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.881+0000 7f842ee00640 1 --2- 172.21.15.47:0/3351505319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84281030c0 0x7f8428109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.881+0000 7f842ee00640 1 --2- 172.21.15.47:0/3351505319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8428102700 0x7f8428102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.881+0000 7f842ee00640 1 --2- 172.21.15.47:0/3351505319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8428101510 0x7f8428101930 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.881+0000 7f842ee00640 1 -- 172.21.15.47:0/3351505319 >> 172.21.15.47:0/3351505319 conn(0x7f84280fca80 msgr2=0x7f84280feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.881+0000 7f842ee00640 1 -- 172.21.15.47:0/3351505319 shutdown_connections 2024-03-21T11:40:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.881+0000 7f842ee00640 1 -- 172.21.15.47:0/3351505319 wait complete. 2024-03-21T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.882+0000 7f842ee00640 1 Processor -- start 2024-03-21T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.882+0000 7f842ee00640 1 -- start start 2024-03-21T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.883+0000 7f842ee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8428101510 0x7f8428199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.883+0000 7f842ee00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8428102700 0x7f8428199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.883+0000 7f842ee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84281030c0 0x7f84281a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.883+0000 7f842ee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842810d1f0 con 0x7f8428101510 2024-03-21T11:40:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.883+0000 7f842ee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842810d070 con 0x7f84281030c0 2024-03-21T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.883+0000 7f842ee00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842810d370 con 0x7f8428102700 2024-03-21T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.883+0000 7f842d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8428102700 0x7f8428199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.883+0000 7f842e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84281030c0 0x7f84281a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.883+0000 7f842d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8428102700 0x7f8428199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38348/0 (socket says 172.21.15.47:38348) 2024-03-21T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.883+0000 7f842d000640 1 -- 172.21.15.47:0/192638042 learned_addr learned my addr 172.21.15.47:0/192638042 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.883+0000 7f842da00640 1 --2- 172.21.15.47:0/192638042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8428101510 0x7f8428199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.884+0000 7f842d000640 1 -- 172.21.15.47:0/192638042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84281030c0 msgr2=0x7f84281a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.884+0000 7f842d000640 1 --2- 172.21.15.47:0/192638042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84281030c0 0x7f84281a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.884+0000 7f842d000640 1 -- 172.21.15.47:0/192638042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8428101510 msgr2=0x7f8428199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.884+0000 7f842d000640 1 --2- 172.21.15.47:0/192638042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8428101510 0x7f8428199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.884+0000 7f842d000640 1 -- 172.21.15.47:0/192638042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84281a1660 con 0x7f8428102700 2024-03-21T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.884+0000 7f842da00640 1 --2- 172.21.15.47:0/192638042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8428101510 0x7f8428199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.884+0000 7f842d000640 1 --2- 172.21.15.47:0/192638042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8428102700 0x7f8428199f80 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f841400d950 tx=0x7f841400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.885+0000 7f8426000640 1 -- 172.21.15.47:0/192638042 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414017030 con 0x7f8428102700 2024-03-21T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.885+0000 7f842ee00640 1 -- 172.21.15.47:0/192638042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84281a1950 con 0x7f8428102700 2024-03-21T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.885+0000 7f8426000640 1 -- 172.21.15.47:0/192638042 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8414004020 con 0x7f8428102700 2024-03-21T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.885+0000 7f8426000640 1 -- 172.21.15.47:0/192638042 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84140047a0 con 0x7f8428102700 2024-03-21T11:40:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.885+0000 7f842ee00640 1 -- 172.21.15.47:0/192638042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84281a1ef0 con 0x7f8428102700 2024-03-21T11:40:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.888+0000 7f8433222740 1 Processor -- start 2024-03-21T11:40:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.889+0000 7f8433222740 1 -- start start 2024-03-21T11:40:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.889+0000 7f8433222740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1d4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.889+0000 7f8433222740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1d90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.889+0000 7f8433222740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d9760 0x55ca7e1dbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.889+0000 7f8433222740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7e18e3b0 con 0x55ca7e1d9760 2024-03-21T11:40:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.889+0000 7f8433222740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7e130830 con 0x55ca7e1d3c00 2024-03-21T11:40:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.889+0000 7f8433222740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7e0d95d0 con 0x55ca7e1d45f0 2024-03-21T11:40:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.890+0000 7f842e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1d4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.890+0000 7f842e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1d4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48780/0 (socket says 172.21.15.47:48780) 2024-03-21T11:40:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.890+0000 7f842da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1d90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.890+0000 7f842e400640 1 -- 172.21.15.47:0/4029124804 learned_addr learned my addr 172.21.15.47:0/4029124804 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.890+0000 7f842e400640 1 -- 172.21.15.47:0/4029124804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d45f0 msgr2=0x55ca7e1d90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.890+0000 7f842e400640 1 --2- 172.21.15.47:0/4029124804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1d90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.890+0000 7f842e400640 1 -- 172.21.15.47:0/4029124804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d9760 msgr2=0x55ca7e1dbbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:40:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.890+0000 7f842e400640 1 --2- 172.21.15.47:0/4029124804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d9760 0x55ca7e1dbbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.890+0000 7f842e400640 1 -- 172.21.15.47:0/4029124804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca7e0f12c0 con 0x55ca7e1d3c00 2024-03-21T11:40:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.890+0000 7f8426000640 1 -- 172.21.15.47:0/192638042 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f84140041d0 con 0x7f8428102700 2024-03-21T11:40:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.890+0000 7f842e400640 1 --2- 172.21.15.47:0/4029124804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1d4020 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8420009f50 tx=0x7f84200118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c77a560745c824 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.891+0000 7f841aa00640 1 -- 172.21.15.47:0/4029124804 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420004d20 con 0x55ca7e1d3c00 2024-03-21T11:40:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.891+0000 7f841aa00640 1 -- 172.21.15.47:0/4029124804 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8420004050 con 0x55ca7e1d3c00 2024-03-21T11:40:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.891+0000 7f841aa00640 1 -- 172.21.15.47:0/4029124804 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420005280 con 0x55ca7e1d3c00 2024-03-21T11:40:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.892+0000 7f8433222740 1 -- 172.21.15.47:0/4029124804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d3c00 msgr2=0x55ca7e1d4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.892+0000 7f8433222740 1 --2- 172.21.15.47:0/4029124804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1d4020 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8420009f50 tx=0x7f84200118d0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.892+0000 7f8433222740 1 -- 172.21.15.47:0/4029124804 shutdown_connections 2024-03-21T11:40:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.892+0000 7f8433222740 1 --2- 172.21.15.47:0/4029124804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d9760 0x55ca7e1dbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.892+0000 7f8433222740 1 --2- 172.21.15.47:0/4029124804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1d90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.892+0000 7f8433222740 1 --2- 172.21.15.47:0/4029124804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1d4020 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.892+0000 7f8433222740 1 -- 172.21.15.47:0/4029124804 >> 172.21.15.47:0/4029124804 conn(0x55ca7e1c6960 msgr2=0x55ca7e1c6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.892+0000 7f8433222740 1 -- 172.21.15.47:0/4029124804 shutdown_connections 2024-03-21T11:40:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.892+0000 7f8426000640 1 --2- 172.21.15.47:0/192638042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84100a2e20 0x7f84100a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.893+0000 7f8426000640 1 -- 172.21.15.47:0/192638042 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f84140cf210 con 0x7f8428102700 2024-03-21T11:40:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.893+0000 7f8433222740 1 -- 172.21.15.47:0/4029124804 wait complete. 2024-03-21T11:40:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.893+0000 7f842da00640 1 --2- 172.21.15.47:0/192638042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84100a2e20 0x7f84100a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.893+0000 7f8433222740 1 -- start start 2024-03-21T11:40:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.894+0000 7f8433222740 1 Processor -- start 2024-03-21T11:40:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.894+0000 7f8433222740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1dde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.894+0000 7f8433222740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1de3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.894+0000 7f8433222740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d9760 0x55ca7e1e53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.894+0000 7f8433222740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7e0eca20 con 0x55ca7e1d3c00 2024-03-21T11:40:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.894+0000 7f8433222740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7e12c8d0 con 0x55ca7e1d9760 2024-03-21T11:40:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.894+0000 7f8433222740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7e1dc710 con 0x55ca7e1d45f0 2024-03-21T11:40:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.895+0000 7f8433222740 10 client.0 _ll_register_callbacks cb 0x55ca7e1291c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.895+0000 7f842e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d9760 0x55ca7e1e53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.895+0000 7f842e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d9760 0x55ca7e1e53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48790/0 (socket says 172.21.15.47:48790) 2024-03-21T11:40:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.895+0000 7f842da00640 1 --2- 172.21.15.47:0/192638042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84100a2e20 0x7f84100a52e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f841c02c190 tx=0x7f841c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.895+0000 7f842e400640 1 -- 172.21.15.47:0/2912718558 learned_addr learned my addr 172.21.15.47:0/2912718558 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.895+0000 7f842d000640 1 --2- 172.21.15.47:0/2912718558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1de3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.895+0000 7f842da00640 1 --2- 172.21.15.47:0/2912718558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1dde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.895+0000 7f842d000640 1 -- 172.21.15.47:0/2912718558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d9760 msgr2=0x55ca7e1e53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.895+0000 7f842d000640 1 --2- 172.21.15.47:0/2912718558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d9760 0x55ca7e1e53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.895+0000 7f842d000640 1 -- 172.21.15.47:0/2912718558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d3c00 msgr2=0x55ca7e1dde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.895+0000 7f842d000640 1 --2- 172.21.15.47:0/2912718558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1dde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.895+0000 7f842d000640 1 -- 172.21.15.47:0/2912718558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca7de45210 con 0x55ca7e1d45f0 2024-03-21T11:40:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.896+0000 7f842d000640 1 --2- 172.21.15.47:0/2912718558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1de3b0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f84140cc980 tx=0x7f841409b440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cdf8e537e75a6f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.896+0000 7f840f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:40:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.896+0000 7f840f400640 1 -- 172.21.15.47:0/2912718558 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841409b5b0 con 0x55ca7e1d45f0 2024-03-21T11:40:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.896+0000 7f840f400640 1 -- 172.21.15.47:0/2912718558 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8414084b40 con 0x55ca7e1d45f0 2024-03-21T11:40:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.896+0000 7f8433222740 1 -- 172.21.15.47:0/2912718558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8420004050 con 0x55ca7e1d45f0 2024-03-21T11:40:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.896+0000 7f840f400640 1 -- 172.21.15.47:0/2912718558 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f841409f700 con 0x55ca7e1d45f0 2024-03-21T11:40:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.896+0000 7f8433222740 10 client.30979 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.896+0000 7f8433222740 1 -- 172.21.15.47:0/2912718558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ca7df11620 con 0x55ca7e1d45f0 2024-03-21T11:40:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.897+0000 7f840f400640 1 -- 172.21.15.47:0/2912718558 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8420004050 con 0x55ca7e1d45f0 2024-03-21T11:40:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.897+0000 7f840f400640 1 -- 172.21.15.47:0/2912718558 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f841409f160 con 0x55ca7e1d45f0 2024-03-21T11:40:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.897+0000 7f8433222740 10 client.30979 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.897+0000 7f8433222740 10 client.30979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.897+0000 7f8433222740 20 client.30979 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.897+0000 7f8433222740 1 --2- 172.21.15.47:0/2912718558 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca7e1e9d30 0x55ca7e1ec120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.897+0000 7f8433222740 4 client.30979 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.897+0000 7f8433222740 1 -- 172.21.15.47:0/2912718558 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca7e13dd70 con 0x55ca7e1e9d30 2024-03-21T11:40:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.897+0000 7f842da00640 1 --2- 172.21.15.47:0/2912718558 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca7e1e9d30 0x55ca7e1ec120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.898+0000 7f842da00640 1 --2- 172.21.15.47:0/2912718558 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca7e1e9d30 0x55ca7e1ec120 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8329006d65143896 server_cookie=5f309c856426f754 in_seq=0 out_seq=0 2024-03-21T11:40:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.898+0000 7f840f400640 10 client.30979 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.903+0000 7f840f400640 1 -- 172.21.15.47:0/2912718558 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ca7e13dd70 con 0x55ca7e1e9d30 2024-03-21T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.903+0000 7f840f400640 10 client.30979 handle_command_reply: tid=0 2024-03-21T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.903+0000 7f8433222740 1 client.30979 shutdown 2024-03-21T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.904+0000 7f8433222740 1 -- 172.21.15.47:0/2912718558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d45f0 msgr2=0x55ca7e1de3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.904+0000 7f8433222740 1 --2- 172.21.15.47:0/2912718558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1de3b0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f84140cc980 tx=0x7f841409b440 comp rx=0 tx=0).stop 2024-03-21T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.904+0000 7f8433222740 1 -- 172.21.15.47:0/2912718558 shutdown_connections 2024-03-21T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.904+0000 7f8433222740 1 --2- 172.21.15.47:0/2912718558 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca7e1e9d30 0x55ca7e1ec120 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.904+0000 7f8433222740 1 --2- 172.21.15.47:0/2912718558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d9760 0x55ca7e1e53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.904+0000 7f8433222740 1 --2- 172.21.15.47:0/2912718558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1de3b0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.904+0000 7f8433222740 1 --2- 172.21.15.47:0/2912718558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1dde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.904+0000 7f840f400640 0 client.30979 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.904+0000 7f8433222740 1 -- 172.21.15.47:0/2912718558 >> 172.21.15.47:0/2912718558 conn(0x55ca7e1c6960 msgr2=0x55ca7e1c5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.904+0000 7f8433222740 1 -- 172.21.15.47:0/2912718558 shutdown_connections 2024-03-21T11:40:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.904+0000 7f8433222740 1 -- 172.21.15.47:0/2912718558 wait complete. 2024-03-21T11:40:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.904+0000 7f8433222740 20 client.30979 trim_cache size 0 max 16384 2024-03-21T11:40:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.925+0000 7f8433222740 1 Processor -- start 2024-03-21T11:40:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.925+0000 7f8433222740 1 -- start start 2024-03-21T11:40:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.925+0000 7f8433222740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1d59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.925+0000 7f8433222740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.925+0000 7f8433222740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d9760 0x55ca7e1e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.925+0000 7f8433222740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281a7d30 con 0x55ca7e1d45f0 2024-03-21T11:40:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.925+0000 7f8433222740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281a96b0 con 0x55ca7e1d3c00 2024-03-21T11:40:01.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.925+0000 7f8433222740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281a8390 con 0x55ca7e1d9760 2024-03-21T11:40:01.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.925+0000 7f842da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1d59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.925+0000 7f842e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d9760 0x55ca7e1e53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.925+0000 7f842da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1d59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48800/0 (socket says 172.21.15.47:48800) 2024-03-21T11:40:01.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.925+0000 7f842da00640 1 -- 172.21.15.47:0/3036066674 learned_addr learned my addr 172.21.15.47:0/3036066674 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:01.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.925+0000 7f842d000640 1 --2- 172.21.15.47:0/3036066674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.926+0000 7f842da00640 1 -- 172.21.15.47:0/3036066674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d9760 msgr2=0x55ca7e1e53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.926+0000 7f842da00640 1 --2- 172.21.15.47:0/3036066674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d9760 0x55ca7e1e53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.926+0000 7f842da00640 1 -- 172.21.15.47:0/3036066674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d45f0 msgr2=0x55ca7e1d5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.926+0000 7f842da00640 1 --2- 172.21.15.47:0/3036066674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1d5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.926+0000 7f842da00640 1 -- 172.21.15.47:0/3036066674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca7e1d49d0 con 0x55ca7e1d3c00 2024-03-21T11:40:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.926+0000 7f842da00640 1 --2- 172.21.15.47:0/3036066674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1d59f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f841c000c00 tx=0x7f841c02e050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5811329e421840b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.926+0000 7f841a000640 1 -- 172.21.15.47:0/3036066674 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c052310 con 0x55ca7e1d3c00 2024-03-21T11:40:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.926+0000 7f841a000640 1 -- 172.21.15.47:0/3036066674 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f841c057070 con 0x55ca7e1d3c00 2024-03-21T11:40:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.926+0000 7f841a000640 1 -- 172.21.15.47:0/3036066674 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c0035f0 con 0x55ca7e1d3c00 2024-03-21T11:40:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.927+0000 7f8433222740 1 -- 172.21.15.47:0/3036066674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d3c00 msgr2=0x55ca7e1d59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.927+0000 7f8433222740 1 --2- 172.21.15.47:0/3036066674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1d59f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f841c000c00 tx=0x7f841c02e050 comp rx=0 tx=0).stop 2024-03-21T11:40:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.927+0000 7f8433222740 1 -- 172.21.15.47:0/3036066674 shutdown_connections 2024-03-21T11:40:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.927+0000 7f8433222740 1 --2- 172.21.15.47:0/3036066674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d9760 0x55ca7e1e53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.927+0000 7f8433222740 1 --2- 172.21.15.47:0/3036066674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1d5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.927+0000 7f8433222740 1 --2- 172.21.15.47:0/3036066674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1d59f0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.927+0000 7f8433222740 1 -- 172.21.15.47:0/3036066674 >> 172.21.15.47:0/3036066674 conn(0x55ca7e1c6960 msgr2=0x55ca7e1c6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.928+0000 7f8433222740 1 -- 172.21.15.47:0/3036066674 shutdown_connections 2024-03-21T11:40:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.928+0000 7f8433222740 1 -- 172.21.15.47:0/3036066674 wait complete. 2024-03-21T11:40:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.928+0000 7f8433222740 1 -- start start 2024-03-21T11:40:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.928+0000 7f8433222740 1 Processor -- start 2024-03-21T11:40:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.929+0000 7f8433222740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1e2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.929+0000 7f8433222740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1e3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.929+0000 7f8433222740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d9760 0x55ca7e1e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.929+0000 7f8433222740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281a9050 con 0x55ca7e1d45f0 2024-03-21T11:40:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.929+0000 7f8433222740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281a89f0 con 0x55ca7e1d9760 2024-03-21T11:40:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.929+0000 7f8433222740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281a5f10 con 0x55ca7e1d3c00 2024-03-21T11:40:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.929+0000 7f8433222740 10 client.0 _ll_register_callbacks cb 0x7f84140046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.929+0000 7f842d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1e3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.929+0000 7f842e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d9760 0x55ca7e1e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.929+0000 7f842d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1e3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47184/0 (socket says 172.21.15.47:47184) 2024-03-21T11:40:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.929+0000 7f842d000640 1 -- 172.21.15.47:0/3940414308 learned_addr learned my addr 172.21.15.47:0/3940414308 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.930+0000 7f842da00640 1 --2- 172.21.15.47:0/3940414308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1e2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.930+0000 7f842e400640 1 -- 172.21.15.47:0/3940414308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d3c00 msgr2=0x55ca7e1e2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.930+0000 7f842e400640 1 --2- 172.21.15.47:0/3940414308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1e2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.930+0000 7f842e400640 1 -- 172.21.15.47:0/3940414308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d45f0 msgr2=0x55ca7e1e3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.930+0000 7f842e400640 1 --2- 172.21.15.47:0/3940414308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1e3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.930+0000 7f842e400640 1 -- 172.21.15.47:0/3940414308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca7e1e4a90 con 0x55ca7e1d9760 2024-03-21T11:40:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.930+0000 7f842da00640 1 --2- 172.21.15.47:0/3940414308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1e2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.930+0000 7f842d000640 1 --2- 172.21.15.47:0/3940414308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1e3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.930+0000 7f842e400640 1 --2- 172.21.15.47:0/3940414308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d9760 0x55ca7e1e44a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8420009250 tx=0x7f84200052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b622b682e97092e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.930+0000 7f840f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.930+0000 7f840f400640 1 -- 172.21.15.47:0/3940414308 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420005630 con 0x55ca7e1d9760 2024-03-21T11:40:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.930+0000 7f8433222740 1 -- 172.21.15.47:0/3940414308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f841c057070 con 0x55ca7e1d9760 2024-03-21T11:40:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.931+0000 7f840f400640 1 -- 172.21.15.47:0/3940414308 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f84200057d0 con 0x55ca7e1d9760 2024-03-21T11:40:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.931+0000 7f840f400640 1 -- 172.21.15.47:0/3940414308 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84200059f0 con 0x55ca7e1d9760 2024-03-21T11:40:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.931+0000 7f8433222740 10 client.30989 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.931+0000 7f8433222740 1 -- 172.21.15.47:0/3940414308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ca7e245a60 con 0x55ca7e1d9760 2024-03-21T11:40:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.931+0000 7f840f400640 1 -- 172.21.15.47:0/3940414308 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f841c057070 con 0x55ca7e1d9760 2024-03-21T11:40:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.931+0000 7f840f400640 1 -- 172.21.15.47:0/3940414308 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f84200195c0 con 0x55ca7e1d9760 2024-03-21T11:40:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.931+0000 7f8433222740 10 client.30989 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.931+0000 7f8433222740 10 client.30989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.931+0000 7f8433222740 20 client.30989 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.932+0000 7f8433222740 1 --2- 172.21.15.47:0/3940414308 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca7e246170 0x55ca7e248590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.932+0000 7f8433222740 4 client.30989 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:40:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.932+0000 7f8433222740 1 -- 172.21.15.47:0/3940414308 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca7e1380c0 con 0x55ca7e246170 2024-03-21T11:40:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.932+0000 7f842da00640 1 --2- 172.21.15.47:0/3940414308 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca7e246170 0x55ca7e248590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.933+0000 7f840f400640 10 client.30989 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:01.933+0000 7f842da00640 1 --2- 172.21.15.47:0/3940414308 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca7e246170 0x55ca7e248590 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=874ff9ab09e302c5 server_cookie=fd3aa583b5aa3a0b in_seq=0 out_seq=0 2024-03-21T11:40:01.937 INFO:tasks.workunit.client.0.smithi047.stdout: 10 79964 26.01 MB/sec execute 45 sec latency 67.446 ms 2024-03-21T11:40:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.835+0000 7f840f400640 1 -- 172.21.15.47:0/3940414308 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca7e1380c0 con 0x55ca7e246170 2024-03-21T11:40:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.835+0000 7f840f400640 10 client.30989 handle_command_reply: tid=0 2024-03-21T11:40:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.835+0000 7f8433222740 1 client.30989 shutdown 2024-03-21T11:40:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.835+0000 7f8433222740 1 -- 172.21.15.47:0/3940414308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d9760 msgr2=0x55ca7e1e44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.835+0000 7f8433222740 1 --2- 172.21.15.47:0/3940414308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d9760 0x55ca7e1e44a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8420009250 tx=0x7f84200052f0 comp rx=0 tx=0).stop 2024-03-21T11:40:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.835+0000 7f8433222740 1 -- 172.21.15.47:0/3940414308 shutdown_connections 2024-03-21T11:40:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.835+0000 7f8433222740 1 --2- 172.21.15.47:0/3940414308 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ca7e246170 0x55ca7e248590 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.835+0000 7f8433222740 1 --2- 172.21.15.47:0/3940414308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca7e1d9760 0x55ca7e1e44a0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.835+0000 7f8433222740 1 --2- 172.21.15.47:0/3940414308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca7e1d45f0 0x55ca7e1e3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.835+0000 7f8433222740 1 --2- 172.21.15.47:0/3940414308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ca7e1d3c00 0x55ca7e1e2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.835+0000 7f840f400640 0 client.30989 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.835+0000 7f8433222740 1 -- 172.21.15.47:0/3940414308 >> 172.21.15.47:0/3940414308 conn(0x55ca7e1c6960 msgr2=0x55ca7e1dde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.836+0000 7f8433222740 1 -- 172.21.15.47:0/3940414308 shutdown_connections 2024-03-21T11:40:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.836+0000 7f8433222740 1 -- 172.21.15.47:0/3940414308 wait complete. 2024-03-21T11:40:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.836+0000 7f8433222740 20 client.30989 trim_cache size 0 max 16384 2024-03-21T11:40:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:40:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:40:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:40:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:40:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.836+0000 7f842ee00640 1 -- 172.21.15.47:0/192638042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84100a2e20 msgr2=0x7f84100a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.836+0000 7f842ee00640 1 --2- 172.21.15.47:0/192638042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84100a2e20 0x7f84100a52e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f841c02c190 tx=0x7f841c005a20 comp rx=0 tx=0).stop 2024-03-21T11:40:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.836+0000 7f842ee00640 1 -- 172.21.15.47:0/192638042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8428102700 msgr2=0x7f8428199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.836+0000 7f842ee00640 1 --2- 172.21.15.47:0/192638042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8428102700 0x7f8428199f80 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f841400d950 tx=0x7f841400de10 comp rx=0 tx=0).stop 2024-03-21T11:40:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.837+0000 7f842ee00640 1 -- 172.21.15.47:0/192638042 shutdown_connections 2024-03-21T11:40:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.837+0000 7f842ee00640 1 --2- 172.21.15.47:0/192638042 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84100a2e20 0x7f84100a52e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.837+0000 7f842ee00640 1 --2- 172.21.15.47:0/192638042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84281030c0 0x7f84281a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.837+0000 7f842ee00640 1 --2- 172.21.15.47:0/192638042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8428102700 0x7f8428199f80 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.837+0000 7f842ee00640 1 --2- 172.21.15.47:0/192638042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8428101510 0x7f8428199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.837+0000 7f842ee00640 1 -- 172.21.15.47:0/192638042 >> 172.21.15.47:0/192638042 conn(0x7f84280fca80 msgr2=0x7f84280738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.837+0000 7f842ee00640 1 -- 172.21.15.47:0/192638042 shutdown_connections 2024-03-21T11:40:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:02.837+0000 7f842ee00640 1 -- 172.21.15.47:0/192638042 wait complete. 2024-03-21T11:40:02.937 INFO:tasks.workunit.client.0.smithi047.stdout: 10 80080 25.56 MB/sec execute 46 sec latency 676.713 ms 2024-03-21T11:40:03.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:03 smithi100 ceph-mon[36245]: pgmap v901: 129 pgs: 129 active+clean; 382 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.7 MiB/s wr, 458 op/s 2024-03-21T11:40:03.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:03 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:40:03.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:03 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:40:03.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:03 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:40:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:03 smithi102 ceph-mon[36748]: pgmap v901: 129 pgs: 129 active+clean; 382 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.7 MiB/s wr, 458 op/s 2024-03-21T11:40:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:03 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:40:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:03 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:40:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:03 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:40:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:03 smithi047 ceph-mon[31351]: pgmap v901: 129 pgs: 129 active+clean; 382 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.7 MiB/s wr, 458 op/s 2024-03-21T11:40:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:03 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:40:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:03 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:40:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:03 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:40:03.937 INFO:tasks.workunit.client.0.smithi047.stdout: 10 80440 25.37 MB/sec execute 47 sec latency 77.543 ms 2024-03-21T11:40:04.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:04 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:40:04.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:04 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:40:04.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:04 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:40:04.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:04 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:40:04.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:04 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:40:04.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:04 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:40:04.938 INFO:tasks.workunit.client.0.smithi047.stdout: 10 80925 25.25 MB/sec execute 48 sec latency 86.187 ms 2024-03-21T11:40:05.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:05 smithi100 ceph-mon[36245]: pgmap v902: 129 pgs: 129 active+clean; 391 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.5 MiB/s wr, 592 op/s 2024-03-21T11:40:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:05 smithi102 ceph-mon[36748]: pgmap v902: 129 pgs: 129 active+clean; 391 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.5 MiB/s wr, 592 op/s 2024-03-21T11:40:05.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:05 smithi047 ceph-mon[31351]: pgmap v902: 129 pgs: 129 active+clean; 391 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.5 MiB/s wr, 592 op/s 2024-03-21T11:40:05.938 INFO:tasks.workunit.client.0.smithi047.stdout: 10 81442 25.30 MB/sec execute 49 sec latency 61.732 ms 2024-03-21T11:40:06.938 INFO:tasks.workunit.client.0.smithi047.stdout: 10 82182 25.43 MB/sec execute 50 sec latency 49.991 ms 2024-03-21T11:40:07.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:07 smithi102 ceph-mon[36748]: pgmap v903: 129 pgs: 129 active+clean; 388 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.2 MiB/s wr, 491 op/s 2024-03-21T11:40:07.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:07 smithi047 ceph-mon[31351]: pgmap v903: 129 pgs: 129 active+clean; 388 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.2 MiB/s wr, 491 op/s 2024-03-21T11:40:07.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:07 smithi100 ceph-mon[36245]: pgmap v903: 129 pgs: 129 active+clean; 388 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.2 MiB/s wr, 491 op/s 2024-03-21T11:40:07.938 INFO:tasks.workunit.client.0.smithi047.stdout: 10 82641 25.61 MB/sec execute 51 sec latency 253.348 ms 2024-03-21T11:40:08.938 INFO:tasks.workunit.client.0.smithi047.stdout: 10 83114 25.71 MB/sec execute 52 sec latency 82.761 ms 2024-03-21T11:40:09.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:09 smithi102 ceph-mon[36748]: pgmap v904: 129 pgs: 129 active+clean; 393 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.6 MiB/s wr, 541 op/s 2024-03-21T11:40:09.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:09 smithi047 ceph-mon[31351]: pgmap v904: 129 pgs: 129 active+clean; 393 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.6 MiB/s wr, 541 op/s 2024-03-21T11:40:09.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:09 smithi100 ceph-mon[36245]: pgmap v904: 129 pgs: 129 active+clean; 393 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.6 MiB/s wr, 541 op/s 2024-03-21T11:40:09.938 INFO:tasks.workunit.client.0.smithi047.stdout: 10 83597 25.69 MB/sec execute 53 sec latency 55.247 ms 2024-03-21T11:40:10.939 INFO:tasks.workunit.client.0.smithi047.stdout: 10 84043 25.61 MB/sec execute 54 sec latency 58.430 ms 2024-03-21T11:40:11.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:11 smithi102 ceph-mon[36748]: pgmap v905: 129 pgs: 129 active+clean; 398 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.2 MiB/s wr, 640 op/s 2024-03-21T11:40:11.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:11 smithi047 ceph-mon[31351]: pgmap v905: 129 pgs: 129 active+clean; 398 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.2 MiB/s wr, 640 op/s 2024-03-21T11:40:11.646 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:11 smithi100 ceph-mon[36245]: pgmap v905: 129 pgs: 129 active+clean; 398 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.2 MiB/s wr, 640 op/s 2024-03-21T11:40:11.938 INFO:tasks.workunit.client.0.smithi047.stdout: 10 84568 25.57 MB/sec execute 55 sec latency 77.156 ms 2024-03-21T11:40:12.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:12 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:40:12.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:12 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:40:12.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:12 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:40:12.939 INFO:tasks.workunit.client.0.smithi047.stdout: 10 84800 25.40 MB/sec execute 56 sec latency 379.018 ms 2024-03-21T11:40:13.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: pgmap v906: 129 pgs: 129 active+clean; 400 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.7 MiB/s wr, 545 op/s 2024-03-21T11:40:13.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi100 to 174.1M 2024-03-21T11:40:13.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: Unable to set osd_memory_target on smithi100 to 182575411: error parsing value: Value '182575411' is below minimum 939524096 2024-03-21T11:40:13.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi047 to 177.8M 2024-03-21T11:40:13.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: Unable to set osd_memory_target on smithi047 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T11:40:13.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:40:13.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:40:13.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:40:13.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: pgmap v906: 129 pgs: 129 active+clean; 400 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.7 MiB/s wr, 545 op/s 2024-03-21T11:40:13.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi100 to 174.1M 2024-03-21T11:40:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: Unable to set osd_memory_target on smithi100 to 182575411: error parsing value: Value '182575411' is below minimum 939524096 2024-03-21T11:40:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi047 to 177.8M 2024-03-21T11:40:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: Unable to set osd_memory_target on smithi047 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T11:40:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:40:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:40:13.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:40:13.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: pgmap v906: 129 pgs: 129 active+clean; 400 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.7 MiB/s wr, 545 op/s 2024-03-21T11:40:13.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi100 to 174.1M 2024-03-21T11:40:13.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: Unable to set osd_memory_target on smithi100 to 182575411: error parsing value: Value '182575411' is below minimum 939524096 2024-03-21T11:40:13.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:40:13.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi047 to 177.8M 2024-03-21T11:40:13.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: Unable to set osd_memory_target on smithi047 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T11:40:13.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:40:13.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:40:13.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:40:13.939 INFO:tasks.workunit.client.0.smithi047.stdout: 10 85192 25.35 MB/sec execute 57 sec latency 81.436 ms 2024-03-21T11:40:14.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:14 smithi102 ceph-mon[36748]: pgmap v907: 129 pgs: 129 active+clean; 409 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.0 MiB/s wr, 634 op/s 2024-03-21T11:40:14.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:14 smithi047 ceph-mon[31351]: pgmap v907: 129 pgs: 129 active+clean; 409 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.0 MiB/s wr, 634 op/s 2024-03-21T11:40:14.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:14 smithi100 ceph-mon[36245]: pgmap v907: 129 pgs: 129 active+clean; 409 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.0 MiB/s wr, 634 op/s 2024-03-21T11:40:14.940 INFO:tasks.workunit.client.0.smithi047.stdout: 10 85789 25.43 MB/sec execute 58 sec latency 43.747 ms 2024-03-21T11:40:15.939 INFO:tasks.workunit.client.0.smithi047.stdout: 10 86303 25.34 MB/sec execute 59 sec latency 77.479 ms 2024-03-21T11:40:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.025+0000 7f0365e00640 1 -- 172.21.15.47:0/2792029222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03601016f0 msgr2=0x7f0360101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.025+0000 7f0365e00640 1 --2- 172.21.15.47:0/2792029222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03601016f0 0x7f0360101b10 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0350009860 tx=0x7f035002bb70 comp rx=0 tx=0).stop 2024-03-21T11:40:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.025+0000 7f0365e00640 1 -- 172.21.15.47:0/2792029222 shutdown_connections 2024-03-21T11:40:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.025+0000 7f0365e00640 1 --2- 172.21.15.47:0/2792029222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03601032a0 0x7f0360109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.025+0000 7f0365e00640 1 --2- 172.21.15.47:0/2792029222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03601028e0 0x7f0360102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.025+0000 7f0365e00640 1 --2- 172.21.15.47:0/2792029222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03601016f0 0x7f0360101b10 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.025+0000 7f0365e00640 1 -- 172.21.15.47:0/2792029222 >> 172.21.15.47:0/2792029222 conn(0x7f03600fcc60 msgr2=0x7f03600ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.026+0000 7f0365e00640 1 -- 172.21.15.47:0/2792029222 shutdown_connections 2024-03-21T11:40:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.026+0000 7f0365e00640 1 -- 172.21.15.47:0/2792029222 wait complete. 2024-03-21T11:40:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.027+0000 7f0365e00640 1 Processor -- start 2024-03-21T11:40:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.029+0000 7f0365e00640 1 -- start start 2024-03-21T11:40:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.029+0000 7f0365e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03601016f0 0x7f03601979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.029+0000 7f0365e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03601028e0 0x7f0360197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.029+0000 7f0365e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03601032a0 0x7f036019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.029+0000 7f0365e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036010d340 con 0x7f03601028e0 2024-03-21T11:40:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.029+0000 7f0365e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036010d1c0 con 0x7f03601032a0 2024-03-21T11:40:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.029+0000 7f0365e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036010d4c0 con 0x7f03601016f0 2024-03-21T11:40:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.029+0000 7f035ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03601016f0 0x7f03601979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.029+0000 7f035f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03601032a0 0x7f036019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.030+0000 7f035ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03601016f0 0x7f03601979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44526/0 (socket says 172.21.15.47:44526) 2024-03-21T11:40:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.030+0000 7f035ea00640 1 -- 172.21.15.47:0/1676511305 learned_addr learned my addr 172.21.15.47:0/1676511305 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.030+0000 7f035f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03601032a0 0x7f036019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48020/0 (socket says 172.21.15.47:48020) 2024-03-21T11:40:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.030+0000 7f035e000640 1 --2- 172.21.15.47:0/1676511305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03601028e0 0x7f0360197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.030+0000 7f035ea00640 1 -- 172.21.15.47:0/1676511305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03601032a0 msgr2=0x7f036019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.030+0000 7f035ea00640 1 --2- 172.21.15.47:0/1676511305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03601032a0 0x7f036019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.030+0000 7f035ea00640 1 -- 172.21.15.47:0/1676511305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03601028e0 msgr2=0x7f0360197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.030+0000 7f035ea00640 1 --2- 172.21.15.47:0/1676511305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03601028e0 0x7f0360197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.030+0000 7f035ea00640 1 -- 172.21.15.47:0/1676511305 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03600738d0 con 0x7f03601016f0 2024-03-21T11:40:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.031+0000 7f035ea00640 1 --2- 172.21.15.47:0/1676511305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03601016f0 0x7f03601979c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f03500349e0 tx=0x7f03500042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.032+0000 7f0364a00640 1 -- 172.21.15.47:0/1676511305 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350041020 con 0x7f03601016f0 2024-03-21T11:40:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.032+0000 7f0365e00640 1 -- 172.21.15.47:0/1676511305 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0360073b60 con 0x7f03601016f0 2024-03-21T11:40:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.032+0000 7f0364a00640 1 -- 172.21.15.47:0/1676511305 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03500323b0 con 0x7f03601016f0 2024-03-21T11:40:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.032+0000 7f0364a00640 1 -- 172.21.15.47:0/1676511305 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350039040 con 0x7f03601016f0 2024-03-21T11:40:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.032+0000 7f0365e00640 1 -- 172.21.15.47:0/1676511305 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03600740a0 con 0x7f03601016f0 2024-03-21T11:40:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.036+0000 7f0364a00640 1 -- 172.21.15.47:0/1676511305 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0350044020 con 0x7f03601016f0 2024-03-21T11:40:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.036+0000 7f0368a7f740 1 Processor -- start 2024-03-21T11:40:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.037+0000 7f0368a7f740 1 -- start start 2024-03-21T11:40:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.037+0000 7f0368a7f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ee9f0 0x55a98c4eee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.037+0000 7f035f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ee9f0 0x55a98c4eee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.037+0000 7f035f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ee9f0 0x55a98c4eee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48026/0 (socket says 172.21.15.47:48026) 2024-03-21T11:40:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.037+0000 7f0368a7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ef3e0 0x55a98c4f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.038+0000 7f035f400640 1 -- 172.21.15.47:0/1610520439 learned_addr learned my addr 172.21.15.47:0/1610520439 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.038+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1610520439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4f4550 0x55a98c4f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.038+0000 7f0368a7f740 1 -- 172.21.15.47:0/1610520439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98c4a93b0 con 0x55a98c4ef3e0 2024-03-21T11:40:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.038+0000 7f0368a7f740 1 -- 172.21.15.47:0/1610520439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98c44b830 con 0x55a98c4ee9f0 2024-03-21T11:40:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.038+0000 7f0368a7f740 1 -- 172.21.15.47:0/1610520439 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98c3f45d0 con 0x55a98c4f4550 2024-03-21T11:40:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.038+0000 7f035ea00640 1 --2- 172.21.15.47:0/1610520439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ef3e0 0x55a98c4f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.038+0000 7f035e000640 1 --2- 172.21.15.47:0/1610520439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4f4550 0x55a98c4f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.038+0000 7f035f400640 1 -- 172.21.15.47:0/1610520439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4f4550 msgr2=0x55a98c4f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.038+0000 7f035f400640 1 --2- 172.21.15.47:0/1610520439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4f4550 0x55a98c4f69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.038+0000 7f035f400640 1 -- 172.21.15.47:0/1610520439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ef3e0 msgr2=0x55a98c4f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.038+0000 7f035f400640 1 --2- 172.21.15.47:0/1610520439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ef3e0 0x55a98c4f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.039+0000 7f035f400640 1 -- 172.21.15.47:0/1610520439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a98c40c2c0 con 0x55a98c4ee9f0 2024-03-21T11:40:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.039+0000 7f035e000640 1 --2- 172.21.15.47:0/1610520439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4f4550 0x55a98c4f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.039+0000 7f035f400640 1 --2- 172.21.15.47:0/1610520439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ee9f0 0x55a98c4eee10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0358008dc0 tx=0x7f035800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77c88feb41a0dfb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.039+0000 7f034be00640 1 -- 172.21.15.47:0/1610520439 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358014070 con 0x55a98c4ee9f0 2024-03-21T11:40:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.040+0000 7f034be00640 1 -- 172.21.15.47:0/1610520439 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0358003b70 con 0x55a98c4ee9f0 2024-03-21T11:40:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.040+0000 7f034be00640 1 -- 172.21.15.47:0/1610520439 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358010790 con 0x55a98c4ee9f0 2024-03-21T11:40:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.040+0000 7f0368a7f740 1 -- 172.21.15.47:0/1610520439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ee9f0 msgr2=0x55a98c4eee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.040+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1610520439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ee9f0 0x55a98c4eee10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0358008dc0 tx=0x7f035800e720 comp rx=0 tx=0).stop 2024-03-21T11:40:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.041+0000 7f0368a7f740 1 -- 172.21.15.47:0/1610520439 shutdown_connections 2024-03-21T11:40:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.041+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1610520439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4f4550 0x55a98c4f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.041+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1610520439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ef3e0 0x55a98c4f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.041+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1610520439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ee9f0 0x55a98c4eee10 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.041+0000 7f0368a7f740 1 -- 172.21.15.47:0/1610520439 >> 172.21.15.47:0/1610520439 conn(0x55a98c4e1750 msgr2=0x55a98c4e1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.041+0000 7f0364a00640 1 --2- 172.21.15.47:0/1676511305 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f03440a2fc0 0x7f03440a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.041+0000 7f0368a7f740 1 -- 172.21.15.47:0/1610520439 shutdown_connections 2024-03-21T11:40:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.041+0000 7f0368a7f740 1 -- 172.21.15.47:0/1610520439 wait complete. 2024-03-21T11:40:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.041+0000 7f035e000640 1 --2- 172.21.15.47:0/1676511305 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f03440a2fc0 0x7f03440a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.041+0000 7f0364a00640 1 -- 172.21.15.47:0/1676511305 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f03500f1140 con 0x7f03601016f0 2024-03-21T11:40:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.042+0000 7f0368a7f740 1 -- start start 2024-03-21T11:40:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.042+0000 7f0368a7f740 1 Processor -- start 2024-03-21T11:40:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.042+0000 7f035e000640 1 --2- 172.21.15.47:0/1676511305 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f03440a2fc0 0x7f03440a5480 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x55a98c4e26c0 tx=0x7f034c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.042+0000 7f0368a7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ee9f0 0x55a98c4f8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.042+0000 7f0368a7f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ef3e0 0x55a98c4f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.042+0000 7f0368a7f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4f4550 0x55a98c500100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.042+0000 7f0368a7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98c407a20 con 0x55a98c4ee9f0 2024-03-21T11:40:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.042+0000 7f0368a7f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98c4478d0 con 0x55a98c4ef3e0 2024-03-21T11:40:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.042+0000 7f0368a7f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98c4f7500 con 0x55a98c4f4550 2024-03-21T11:40:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.043+0000 7f035e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ef3e0 0x55a98c4f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.043+0000 7f0368a7f740 10 client.0 _ll_register_callbacks cb 0x55a98c4441c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.043+0000 7f035e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ef3e0 0x55a98c4f90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48040/0 (socket says 172.21.15.47:48040) 2024-03-21T11:40:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.043+0000 7f035e000640 1 -- 172.21.15.47:0/1350828213 learned_addr learned my addr 172.21.15.47:0/1350828213 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.043+0000 7f035f400640 1 --2- 172.21.15.47:0/1350828213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4f4550 0x55a98c500100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.043+0000 7f035e000640 1 -- 172.21.15.47:0/1350828213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4f4550 msgr2=0x55a98c500100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.043+0000 7f035e000640 1 --2- 172.21.15.47:0/1350828213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4f4550 0x55a98c500100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.043+0000 7f035e000640 1 -- 172.21.15.47:0/1350828213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ee9f0 msgr2=0x55a98c4f8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.043+0000 7f035e000640 1 --2- 172.21.15.47:0/1350828213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ee9f0 0x55a98c4f8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.043+0000 7f035e000640 1 -- 172.21.15.47:0/1350828213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a98c160210 con 0x55a98c4ef3e0 2024-03-21T11:40:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.044+0000 7f035e000640 1 --2- 172.21.15.47:0/1350828213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ef3e0 0x55a98c4f90c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f034c009670 tx=0x7f034c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9b46b1dc0fac079 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.044+0000 7f0348c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.044+0000 7f0348c00640 1 -- 172.21.15.47:0/1350828213 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c010850 con 0x55a98c4ef3e0 2024-03-21T11:40:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.044+0000 7f0348c00640 1 -- 172.21.15.47:0/1350828213 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f034c0109f0 con 0x55a98c4ef3e0 2024-03-21T11:40:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.044+0000 7f0368a7f740 1 -- 172.21.15.47:0/1350828213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0358003b70 con 0x55a98c4ef3e0 2024-03-21T11:40:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.044+0000 7f0348c00640 1 -- 172.21.15.47:0/1350828213 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f034c010bf0 con 0x55a98c4ef3e0 2024-03-21T11:40:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.044+0000 7f0368a7f740 10 client.31001 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.044+0000 7f0368a7f740 1 -- 172.21.15.47:0/1350828213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a98c22c620 con 0x55a98c4ef3e0 2024-03-21T11:40:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.045+0000 7f0348c00640 1 -- 172.21.15.47:0/1350828213 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0358003b70 con 0x55a98c4ef3e0 2024-03-21T11:40:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.046+0000 7f0348c00640 1 -- 172.21.15.47:0/1350828213 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f034c027400 con 0x55a98c4ef3e0 2024-03-21T11:40:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.046+0000 7f0368a7f740 10 client.31001 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.046+0000 7f0368a7f740 10 client.31001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.046+0000 7f0368a7f740 20 client.31001 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.046+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1350828213 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a98c504980 0x55a98c506db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.046+0000 7f0368a7f740 4 client.31001 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.046+0000 7f0368a7f740 1 -- 172.21.15.47:0/1350828213 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a98c458d70 con 0x55a98c504980 2024-03-21T11:40:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.046+0000 7f035ea00640 1 --2- 172.21.15.47:0/1350828213 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a98c504980 0x55a98c506db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.047+0000 7f035ea00640 1 --2- 172.21.15.47:0/1350828213 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a98c504980 0x55a98c506db0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5feee2da7924cba9 server_cookie=32908f8197397719 in_seq=0 out_seq=0 2024-03-21T11:40:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.047+0000 7f0348c00640 10 client.31001 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.051+0000 7f0348c00640 1 -- 172.21.15.47:0/1350828213 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a98c458d70 con 0x55a98c504980 2024-03-21T11:40:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.051+0000 7f0348c00640 10 client.31001 handle_command_reply: tid=0 2024-03-21T11:40:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.051+0000 7f0368a7f740 1 client.31001 shutdown 2024-03-21T11:40:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.051+0000 7f0368a7f740 1 -- 172.21.15.47:0/1350828213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ef3e0 msgr2=0x55a98c4f90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.051+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1350828213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ef3e0 0x55a98c4f90c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f034c009670 tx=0x7f034c0046c0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.051+0000 7f0368a7f740 1 -- 172.21.15.47:0/1350828213 shutdown_connections 2024-03-21T11:40:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.051+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1350828213 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a98c504980 0x55a98c506db0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.051+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1350828213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4f4550 0x55a98c500100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.051+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1350828213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ef3e0 0x55a98c4f90c0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.051+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1350828213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ee9f0 0x55a98c4f8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.051+0000 7f0368a7f740 1 -- 172.21.15.47:0/1350828213 >> 172.21.15.47:0/1350828213 conn(0x55a98c4e1750 msgr2=0x55a98c4e0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.051+0000 7f0348c00640 0 client.31001 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.051+0000 7f0368a7f740 1 -- 172.21.15.47:0/1350828213 shutdown_connections 2024-03-21T11:40:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.051+0000 7f0368a7f740 1 -- 172.21.15.47:0/1350828213 wait complete. 2024-03-21T11:40:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.052+0000 7f0368a7f740 20 client.31001 trim_cache size 0 max 16384 2024-03-21T11:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.079+0000 7f0368a7f740 1 Processor -- start 2024-03-21T11:40:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.079+0000 7f0368a7f740 1 -- start start 2024-03-21T11:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.079+0000 7f0368a7f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4e1750 0x55a98c4e00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.079+0000 7f0368a7f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ee9f0 0x55a98c5000c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.079+0000 7f0368a7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ef3e0 0x55a98c4e0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.081+0000 7f0368a7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03601a7fe0 con 0x55a98c4ef3e0 2024-03-21T11:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.081+0000 7f0368a7f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03601a9960 con 0x55a98c4ee9f0 2024-03-21T11:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.081+0000 7f0368a7f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03601a8640 con 0x55a98c4e1750 2024-03-21T11:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.081+0000 7f035e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ee9f0 0x55a98c5000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.081+0000 7f035e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ee9f0 0x55a98c5000c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48054/0 (socket says 172.21.15.47:48054) 2024-03-21T11:40:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.081+0000 7f035e000640 1 -- 172.21.15.47:0/3345678056 learned_addr learned my addr 172.21.15.47:0/3345678056 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.081+0000 7f035e000640 1 -- 172.21.15.47:0/3345678056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4e1750 msgr2=0x55a98c4e00c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:40:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.081+0000 7f035e000640 1 --2- 172.21.15.47:0/3345678056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4e1750 0x55a98c4e00c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.081+0000 7f035e000640 1 -- 172.21.15.47:0/3345678056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ef3e0 msgr2=0x55a98c4e0600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:40:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.081+0000 7f035e000640 1 --2- 172.21.15.47:0/3345678056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ef3e0 0x55a98c4e0600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.081+0000 7f035e000640 1 -- 172.21.15.47:0/3345678056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a98c4e24c0 con 0x55a98c4ee9f0 2024-03-21T11:40:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.082+0000 7f035e000640 1 --2- 172.21.15.47:0/3345678056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ee9f0 0x55a98c5000c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f034c0277a0 tx=0x7f034c003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eaca156f476ebbc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.082+0000 7f034b400640 1 -- 172.21.15.47:0/3345678056 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c03f070 con 0x55a98c4ee9f0 2024-03-21T11:40:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.082+0000 7f034b400640 1 -- 172.21.15.47:0/3345678056 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f034c027400 con 0x55a98c4ee9f0 2024-03-21T11:40:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.082+0000 7f034b400640 1 -- 172.21.15.47:0/3345678056 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c036040 con 0x55a98c4ee9f0 2024-03-21T11:40:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.084+0000 7f0368a7f740 1 -- 172.21.15.47:0/3345678056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ee9f0 msgr2=0x55a98c5000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.084+0000 7f0368a7f740 1 --2- 172.21.15.47:0/3345678056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ee9f0 0x55a98c5000c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f034c0277a0 tx=0x7f034c003e90 comp rx=0 tx=0).stop 2024-03-21T11:40:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.085+0000 7f0368a7f740 1 -- 172.21.15.47:0/3345678056 shutdown_connections 2024-03-21T11:40:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.085+0000 7f0368a7f740 1 --2- 172.21.15.47:0/3345678056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ef3e0 0x55a98c4e0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.085+0000 7f0368a7f740 1 --2- 172.21.15.47:0/3345678056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ee9f0 0x55a98c5000c0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.085+0000 7f0368a7f740 1 --2- 172.21.15.47:0/3345678056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4e1750 0x55a98c4e00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.085+0000 7f0368a7f740 1 -- 172.21.15.47:0/3345678056 >> 172.21.15.47:0/3345678056 conn(0x55a98c4f4090 msgr2=0x55a98c4f0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.085+0000 7f0368a7f740 1 -- 172.21.15.47:0/3345678056 shutdown_connections 2024-03-21T11:40:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.085+0000 7f0368a7f740 1 -- 172.21.15.47:0/3345678056 wait complete. 2024-03-21T11:40:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.086+0000 7f0368a7f740 1 -- start start 2024-03-21T11:40:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.088+0000 7f0368a7f740 1 Processor -- start 2024-03-21T11:40:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.089+0000 7f0368a7f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4e1750 0x55a98c4fdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.090+0000 7f0368a7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ee9f0 0x55a98c4fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.090+0000 7f0368a7f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ef3e0 0x55a98c4ff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.090+0000 7f0368a7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03601a9300 con 0x55a98c4ee9f0 2024-03-21T11:40:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.090+0000 7f0368a7f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03601a8ca0 con 0x55a98c4ef3e0 2024-03-21T11:40:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.090+0000 7f0368a7f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03601a61a0 con 0x55a98c4e1750 2024-03-21T11:40:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.090+0000 7f0368a7f740 10 client.0 _ll_register_callbacks cb 0x7f034c009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.091+0000 7f035f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ef3e0 0x55a98c4ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.091+0000 7f035f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ef3e0 0x55a98c4ff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48060/0 (socket says 172.21.15.47:48060) 2024-03-21T11:40:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.091+0000 7f035f400640 1 -- 172.21.15.47:0/1218342801 learned_addr learned my addr 172.21.15.47:0/1218342801 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.091+0000 7f035ea00640 1 --2- 172.21.15.47:0/1218342801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4e1750 0x55a98c4fdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.092+0000 7f035e000640 1 --2- 172.21.15.47:0/1218342801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ee9f0 0x55a98c4fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.092+0000 7f035f400640 1 -- 172.21.15.47:0/1218342801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4e1750 msgr2=0x55a98c4fdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.092+0000 7f035f400640 1 --2- 172.21.15.47:0/1218342801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4e1750 0x55a98c4fdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.092+0000 7f035f400640 1 -- 172.21.15.47:0/1218342801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ee9f0 msgr2=0x55a98c4fe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.092+0000 7f035f400640 1 --2- 172.21.15.47:0/1218342801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ee9f0 0x55a98c4fe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.092+0000 7f035f400640 1 -- 172.21.15.47:0/1218342801 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a98c4ff860 con 0x55a98c4ef3e0 2024-03-21T11:40:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.092+0000 7f035e000640 1 --2- 172.21.15.47:0/1218342801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ee9f0 0x55a98c4fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:40:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.093+0000 7f035f400640 1 --2- 172.21.15.47:0/1218342801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ef3e0 0x55a98c4ff270 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0358008dc0 tx=0x7f0358010bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54cca26aba451183 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.093+0000 7f0348c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.093+0000 7f0348c00640 1 -- 172.21.15.47:0/1218342801 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358014040 con 0x55a98c4ef3e0 2024-03-21T11:40:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.093+0000 7f0368a7f740 1 -- 172.21.15.47:0/1218342801 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f034c027400 con 0x55a98c4ef3e0 2024-03-21T11:40:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.093+0000 7f0348c00640 1 -- 172.21.15.47:0/1218342801 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0358004630 con 0x55a98c4ef3e0 2024-03-21T11:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.094+0000 7f0348c00640 1 -- 172.21.15.47:0/1218342801 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0358030050 con 0x55a98c4ef3e0 2024-03-21T11:40:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.094+0000 7f0368a7f740 10 client.31007 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.094+0000 7f0368a7f740 1 -- 172.21.15.47:0/1218342801 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a98c560870 con 0x55a98c4ef3e0 2024-03-21T11:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.097+0000 7f0348c00640 1 -- 172.21.15.47:0/1218342801 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f034c027400 con 0x55a98c4ef3e0 2024-03-21T11:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.098+0000 7f0348c00640 1 -- 172.21.15.47:0/1218342801 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f035802b030 con 0x55a98c4ef3e0 2024-03-21T11:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.098+0000 7f0368a7f740 10 client.31007 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.098+0000 7f0368a7f740 10 client.31007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.098+0000 7f0368a7f740 20 client.31007 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.098+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1218342801 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a98c4f4550 0x55a98c562f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.098+0000 7f0368a7f740 4 client.31007 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.098+0000 7f0368a7f740 1 -- 172.21.15.47:0/1218342801 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a98c4530c0 con 0x55a98c4f4550 2024-03-21T11:40:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.099+0000 7f035ea00640 1 --2- 172.21.15.47:0/1218342801 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a98c4f4550 0x55a98c562f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.099+0000 7f035ea00640 1 --2- 172.21.15.47:0/1218342801 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a98c4f4550 0x55a98c562f80 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3fd6608927993f4 server_cookie=2973ce34bdff267e in_seq=0 out_seq=0 2024-03-21T11:40:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.099+0000 7f0348c00640 10 client.31007 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.842+0000 7f0348c00640 1 -- 172.21.15.47:0/1218342801 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a98c4530c0 con 0x55a98c4f4550 2024-03-21T11:40:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.842+0000 7f0348c00640 10 client.31007 handle_command_reply: tid=0 2024-03-21T11:40:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.842+0000 7f0368a7f740 1 client.31007 shutdown 2024-03-21T11:40:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.843+0000 7f0368a7f740 1 -- 172.21.15.47:0/1218342801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ef3e0 msgr2=0x55a98c4ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.843+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1218342801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ef3e0 0x55a98c4ff270 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0358008dc0 tx=0x7f0358010bf0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.843+0000 7f0368a7f740 1 -- 172.21.15.47:0/1218342801 shutdown_connections 2024-03-21T11:40:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.843+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1218342801 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a98c4f4550 0x55a98c562f80 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.843+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1218342801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a98c4ef3e0 0x55a98c4ff270 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.843+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1218342801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a98c4ee9f0 0x55a98c4fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.843+0000 7f0368a7f740 1 --2- 172.21.15.47:0/1218342801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a98c4e1750 0x55a98c4fdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.843+0000 7f0368a7f740 1 -- 172.21.15.47:0/1218342801 >> 172.21.15.47:0/1218342801 conn(0x55a98c4f4090 msgr2=0x55a98c4e49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.843+0000 7f0348c00640 0 client.31007 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.843+0000 7f0368a7f740 1 -- 172.21.15.47:0/1218342801 shutdown_connections 2024-03-21T11:40:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.843+0000 7f0368a7f740 1 -- 172.21.15.47:0/1218342801 wait complete. 2024-03-21T11:40:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.844+0000 7f0368a7f740 20 client.31007 trim_cache size 0 max 16384 2024-03-21T11:40:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:40:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:40:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:40:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:40:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.844+0000 7f0365e00640 1 -- 172.21.15.47:0/1676511305 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f03440a2fc0 msgr2=0x7f03440a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.844+0000 7f0365e00640 1 --2- 172.21.15.47:0/1676511305 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f03440a2fc0 0x7f03440a5480 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x55a98c4e26c0 tx=0x7f034c009260 comp rx=0 tx=0).stop 2024-03-21T11:40:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.844+0000 7f0365e00640 1 -- 172.21.15.47:0/1676511305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03601016f0 msgr2=0x7f03601979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.844+0000 7f0365e00640 1 --2- 172.21.15.47:0/1676511305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03601016f0 0x7f03601979c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f03500349e0 tx=0x7f03500042f0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.845+0000 7f0365e00640 1 -- 172.21.15.47:0/1676511305 shutdown_connections 2024-03-21T11:40:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.845+0000 7f0365e00640 1 --2- 172.21.15.47:0/1676511305 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f03440a2fc0 0x7f03440a5480 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.845+0000 7f0365e00640 1 --2- 172.21.15.47:0/1676511305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03601032a0 0x7f036019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.845+0000 7f0365e00640 1 --2- 172.21.15.47:0/1676511305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03601028e0 0x7f0360197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.845+0000 7f0365e00640 1 --2- 172.21.15.47:0/1676511305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03601016f0 0x7f03601979c0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.845+0000 7f0365e00640 1 -- 172.21.15.47:0/1676511305 >> 172.21.15.47:0/1676511305 conn(0x7f03600fcc60 msgr2=0x7f0360105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.845+0000 7f0365e00640 1 -- 172.21.15.47:0/1676511305 shutdown_connections 2024-03-21T11:40:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:16.845+0000 7f0365e00640 1 -- 172.21.15.47:0/1676511305 wait complete. 2024-03-21T11:40:16.939 INFO:tasks.workunit.client.0.smithi047.stdout: 10 86593 25.21 MB/sec execute 60 sec latency 561.053 ms 2024-03-21T11:40:16.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:16 smithi102 ceph-mon[36748]: pgmap v908: 129 pgs: 129 active+clean; 411 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.7 MiB/s wr, 504 op/s 2024-03-21T11:40:17.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:16 smithi047 ceph-mon[31351]: pgmap v908: 129 pgs: 129 active+clean; 411 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.7 MiB/s wr, 504 op/s 2024-03-21T11:40:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:16 smithi100 ceph-mon[36245]: pgmap v908: 129 pgs: 129 active+clean; 411 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.7 MiB/s wr, 504 op/s 2024-03-21T11:40:17.939 INFO:tasks.workunit.client.0.smithi047.stdout: 10 87005 25.24 MB/sec execute 61 sec latency 365.395 ms 2024-03-21T11:40:18.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:40:18.773 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:40:18.941 INFO:tasks.workunit.client.0.smithi047.stdout: 10 87426 25.21 MB/sec execute 62 sec latency 130.517 ms 2024-03-21T11:40:18.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:18 smithi102 ceph-mon[36748]: pgmap v909: 129 pgs: 129 active+clean; 412 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.6 MiB/s wr, 461 op/s 2024-03-21T11:40:19.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:18 smithi047 ceph-mon[31351]: pgmap v909: 129 pgs: 129 active+clean; 412 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.6 MiB/s wr, 461 op/s 2024-03-21T11:40:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.070+0000 7faae8e00640 1 -- 172.21.15.47:0/1976653924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4101210 msgr2=0x7faae4101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:19.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.070+0000 7faae8e00640 1 --2- 172.21.15.47:0/1976653924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4101210 0x7faae4101690 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7faad0009900 tx=0x7faad002bc30 comp rx=0 tx=0).stop 2024-03-21T11:40:19.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.072+0000 7faae8e00640 1 -- 172.21.15.47:0/1976653924 shutdown_connections 2024-03-21T11:40:19.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.072+0000 7faae8e00640 1 --2- 172.21.15.47:0/1976653924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4101210 0x7faae4101690 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.072+0000 7faae8e00640 1 --2- 172.21.15.47:0/1976653924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faae4100870 0x7faae4100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.072+0000 7faae8e00640 1 --2- 172.21.15.47:0/1976653924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae40fff10 0x7faae4100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.072+0000 7faae8e00640 1 -- 172.21.15.47:0/1976653924 >> 172.21.15.47:0/1976653924 conn(0x7faae40fbb30 msgr2=0x7faae40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:19.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.073+0000 7faae8e00640 1 -- 172.21.15.47:0/1976653924 shutdown_connections 2024-03-21T11:40:19.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.073+0000 7faae8e00640 1 -- 172.21.15.47:0/1976653924 wait complete. 2024-03-21T11:40:19.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.075+0000 7faae8e00640 1 Processor -- start 2024-03-21T11:40:19.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.075+0000 7faae8e00640 1 -- start start 2024-03-21T11:40:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.076+0000 7faae8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faae40fff10 0x7faae419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.076+0000 7faae8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae4100870 0x7faae419e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.076+0000 7faae8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4101210 0x7faae41a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.076+0000 7faae8e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae4103710 con 0x7faae40fff10 2024-03-21T11:40:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.076+0000 7faae8e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae4103590 con 0x7faae4100870 2024-03-21T11:40:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.076+0000 7faae8e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae4103890 con 0x7faae4101210 2024-03-21T11:40:19.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.076+0000 7faae1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae4100870 0x7faae419e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.076+0000 7faae1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae4100870 0x7faae419e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55018/0 (socket says 172.21.15.47:55018) 2024-03-21T11:40:19.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.076+0000 7faae1400640 1 -- 172.21.15.47:0/1815226602 learned_addr learned my addr 172.21.15.47:0/1815226602 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:19.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.076+0000 7faae2800640 1 --2- 172.21.15.47:0/1815226602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4101210 0x7faae41a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.077+0000 7faae1400640 1 -- 172.21.15.47:0/1815226602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4101210 msgr2=0x7faae41a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:19.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.077+0000 7faae1400640 1 --2- 172.21.15.47:0/1815226602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4101210 0x7faae41a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.077+0000 7faae1400640 1 -- 172.21.15.47:0/1815226602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faae40fff10 msgr2=0x7faae419dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:19.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.077+0000 7faae1400640 1 --2- 172.21.15.47:0/1815226602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faae40fff10 0x7faae419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.077+0000 7faae1400640 1 -- 172.21.15.47:0/1815226602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae41a5bd0 con 0x7faae4100870 2024-03-21T11:40:19.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.077+0000 7faae1400640 1 --2- 172.21.15.47:0/1815226602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae4100870 0x7faae419e4f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7faad400e9f0 tx=0x7faad400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:19.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.077+0000 7faadaa00640 1 -- 172.21.15.47:0/1815226602 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad400cd80 con 0x7faae4100870 2024-03-21T11:40:19.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.078+0000 7faae8e00640 1 -- 172.21.15.47:0/1815226602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae41a5ec0 con 0x7faae4100870 2024-03-21T11:40:19.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.078+0000 7faae8e00640 1 -- 172.21.15.47:0/1815226602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae41a63d0 con 0x7faae4100870 2024-03-21T11:40:19.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.078+0000 7faadaa00640 1 -- 172.21.15.47:0/1815226602 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faad4010910 con 0x7faae4100870 2024-03-21T11:40:19.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.078+0000 7faadaa00640 1 -- 172.21.15.47:0/1815226602 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad40188f0 con 0x7faae4100870 2024-03-21T11:40:19.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.082+0000 7faaeba83740 1 Processor -- start 2024-03-21T11:40:19.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.083+0000 7faaeba83740 1 -- start start 2024-03-21T11:40:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.084+0000 7faadaa00640 1 -- 172.21.15.47:0/1815226602 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7faad4018a90 con 0x7faae4100870 2024-03-21T11:40:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.085+0000 7faaeba83740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d8e80 0x559e510d92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.085+0000 7faaeba83740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d9870 0x559e510de370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.086+0000 7faaeba83740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 0x559e510e0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.086+0000 7faaeba83740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e5103e250 con 0x559e510de9e0 2024-03-21T11:40:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.086+0000 7faaeba83740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e50fe6a30 con 0x559e510d8e80 2024-03-21T11:40:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.086+0000 7faaeba83740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e5103a2f0 con 0x559e510d9870 2024-03-21T11:40:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.086+0000 7faae1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d9870 0x559e510de370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.086+0000 7faae1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d9870 0x559e510de370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46756/0 (socket says 172.21.15.47:46756) 2024-03-21T11:40:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.086+0000 7faae1e00640 1 -- 172.21.15.47:0/2887307598 learned_addr learned my addr 172.21.15.47:0/2887307598 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.086+0000 7faae2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d8e80 0x559e510d92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.087+0000 7faae1e00640 1 -- 172.21.15.47:0/2887307598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d8e80 msgr2=0x559e510d92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.087+0000 7faae1e00640 1 --2- 172.21.15.47:0/2887307598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d8e80 0x559e510d92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.087+0000 7faae1e00640 1 -- 172.21.15.47:0/2887307598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 msgr2=0x559e510e0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.087+0000 7faae1e00640 1 --2- 172.21.15.47:0/2887307598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 0x559e510e0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.087+0000 7faae1e00640 1 -- 172.21.15.47:0/2887307598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e50fd1690 con 0x559e510d9870 2024-03-21T11:40:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.088+0000 7faae1e00640 1 --2- 172.21.15.47:0/2887307598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d9870 0x559e510de370 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7faadc009ea0 tx=0x7faadc009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=495b5d2e75190cc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.088+0000 7faacf400640 1 -- 172.21.15.47:0/2887307598 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc004d20 con 0x559e510d9870 2024-03-21T11:40:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.088+0000 7faadaa00640 1 --2- 172.21.15.47:0/1815226602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faac80a3020 0x7faac80a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.089+0000 7faadaa00640 1 -- 172.21.15.47:0/1815226602 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7faad40cfac0 con 0x7faae4100870 2024-03-21T11:40:19.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.089+0000 7faacf400640 1 -- 172.21.15.47:0/2887307598 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faadc005500 con 0x559e510d9870 2024-03-21T11:40:19.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.089+0000 7faacf400640 1 -- 172.21.15.47:0/2887307598 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc010970 con 0x559e510d9870 2024-03-21T11:40:19.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.089+0000 7faae2800640 1 --2- 172.21.15.47:0/1815226602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faac80a3020 0x7faac80a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.090+0000 7faaeba83740 1 -- 172.21.15.47:0/2887307598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d9870 msgr2=0x559e510de370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:19.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.090+0000 7faaeba83740 1 --2- 172.21.15.47:0/2887307598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d9870 0x559e510de370 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7faadc009ea0 tx=0x7faadc009f90 comp rx=0 tx=0).stop 2024-03-21T11:40:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.090+0000 7faae2800640 1 --2- 172.21.15.47:0/1815226602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faac80a3020 0x7faac80a54e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7faad0005ab0 tx=0x7faad002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.091+0000 7faaeba83740 1 -- 172.21.15.47:0/2887307598 shutdown_connections 2024-03-21T11:40:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.091+0000 7faaeba83740 1 --2- 172.21.15.47:0/2887307598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 0x559e510e0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.091+0000 7faaeba83740 1 --2- 172.21.15.47:0/2887307598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d9870 0x559e510de370 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.091+0000 7faaeba83740 1 --2- 172.21.15.47:0/2887307598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d8e80 0x559e510d92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.091+0000 7faaeba83740 1 -- 172.21.15.47:0/2887307598 >> 172.21.15.47:0/2887307598 conn(0x559e510a1200 msgr2=0x559e510d47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.092+0000 7faaeba83740 1 -- 172.21.15.47:0/2887307598 shutdown_connections 2024-03-21T11:40:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.092+0000 7faaeba83740 1 -- 172.21.15.47:0/2887307598 wait complete. 2024-03-21T11:40:19.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.092+0000 7faaeba83740 1 -- start start 2024-03-21T11:40:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.092+0000 7faaeba83740 1 Processor -- start 2024-03-21T11:40:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.093+0000 7faaeba83740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d8e80 0x559e510e2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.093+0000 7faaeba83740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510e3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.093+0000 7faaeba83740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 0x559e510ea460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.093+0000 7faaeba83740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e510e1990 con 0x559e510de9e0 2024-03-21T11:40:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.093+0000 7faaeba83740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e50ff9e80 con 0x559e510d9870 2024-03-21T11:40:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.093+0000 7faaeba83740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e510e1b10 con 0x559e510d8e80 2024-03-21T11:40:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.093+0000 7faaeba83740 10 client.0 _ll_register_callbacks cb 0x559e5100c130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.094+0000 7faae1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d8e80 0x559e510e2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.094+0000 7faae2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 0x559e510ea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.094+0000 7faae2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 0x559e510ea460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59760/0 (socket says 172.21.15.47:59760) 2024-03-21T11:40:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.094+0000 7faae2800640 1 -- 172.21.15.47:0/1135756089 learned_addr learned my addr 172.21.15.47:0/1135756089 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:19.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.094+0000 7faae1400640 1 --2- 172.21.15.47:0/1135756089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.094+0000 7faae1400640 1 -- 172.21.15.47:0/1135756089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d8e80 msgr2=0x559e510e2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.094+0000 7faae1400640 1 --2- 172.21.15.47:0/1135756089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d8e80 0x559e510e2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.094+0000 7faae1400640 1 -- 172.21.15.47:0/1135756089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 msgr2=0x559e510ea460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.094+0000 7faae1400640 1 --2- 172.21.15.47:0/1135756089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 0x559e510ea460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.094+0000 7faae1400640 1 -- 172.21.15.47:0/1135756089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e50e1c7d0 con 0x559e510d9870 2024-03-21T11:40:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.095+0000 7faae1400640 1 --2- 172.21.15.47:0/1135756089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510e3420 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7faad4098260 tx=0x7faad409b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64f09dc6ce4bdea9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.095+0000 7faac3e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.095+0000 7faac3e00640 1 -- 172.21.15.47:0/1135756089 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad409ff50 con 0x559e510d9870 2024-03-21T11:40:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.095+0000 7faaeba83740 1 -- 172.21.15.47:0/1135756089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faadc005500 con 0x559e510d9870 2024-03-21T11:40:19.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.095+0000 7faac3e00640 1 -- 172.21.15.47:0/1135756089 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7faad40bab70 con 0x559e510d9870 2024-03-21T11:40:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.096+0000 7faac3e00640 1 -- 172.21.15.47:0/1135756089 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7faad40a4810 con 0x559e510d9870 2024-03-21T11:40:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.096+0000 7faaeba83740 10 client.31019 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.096+0000 7faaeba83740 1 -- 172.21.15.47:0/1135756089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559e50d523f0 con 0x559e510d9870 2024-03-21T11:40:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.096+0000 7faac3e00640 1 -- 172.21.15.47:0/1135756089 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faadc005500 con 0x559e510d9870 2024-03-21T11:40:19.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.096+0000 7faac3e00640 1 -- 172.21.15.47:0/1135756089 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7faad4097bc0 con 0x559e510d9870 2024-03-21T11:40:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.097+0000 7faaeba83740 10 client.31019 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.097+0000 7faaeba83740 10 client.31019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.097+0000 7faaeba83740 20 client.31019 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.097+0000 7faaeba83740 1 --2- 172.21.15.47:0/1135756089 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e510eed10 0x559e510f1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.097+0000 7faaeba83740 4 client.31019 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.097+0000 7faaeba83740 1 -- 172.21.15.47:0/1135756089 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e5104b790 con 0x559e510eed10 2024-03-21T11:40:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.097+0000 7faae1e00640 1 --2- 172.21.15.47:0/1135756089 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e510eed10 0x559e510f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.098+0000 7faae1e00640 1 --2- 172.21.15.47:0/1135756089 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e510eed10 0x559e510f1130 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=181c5069f555d706 server_cookie=946938c4be795a2 in_seq=0 out_seq=0 2024-03-21T11:40:19.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.098+0000 7faac3e00640 10 client.31019 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:19.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.100+0000 7faac3e00640 1 -- 172.21.15.47:0/1135756089 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559e5104b790 con 0x559e510eed10 2024-03-21T11:40:19.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.100+0000 7faac3e00640 10 client.31019 handle_command_reply: tid=0 2024-03-21T11:40:19.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.101+0000 7faaeba83740 1 client.31019 shutdown 2024-03-21T11:40:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.101+0000 7faaeba83740 1 -- 172.21.15.47:0/1135756089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 msgr2=0x559e510e3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:19.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.101+0000 7faaeba83740 1 --2- 172.21.15.47:0/1135756089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510e3420 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7faad4098260 tx=0x7faad409b820 comp rx=0 tx=0).stop 2024-03-21T11:40:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.103+0000 7faaeba83740 1 -- 172.21.15.47:0/1135756089 shutdown_connections 2024-03-21T11:40:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.103+0000 7faaeba83740 1 --2- 172.21.15.47:0/1135756089 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e510eed10 0x559e510f1130 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.103+0000 7faaeba83740 1 --2- 172.21.15.47:0/1135756089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 0x559e510ea460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.103+0000 7faaeba83740 1 --2- 172.21.15.47:0/1135756089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510e3420 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.103+0000 7faaeba83740 1 --2- 172.21.15.47:0/1135756089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d8e80 0x559e510e2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.103+0000 7faaeba83740 1 -- 172.21.15.47:0/1135756089 >> 172.21.15.47:0/1135756089 conn(0x559e510a1200 msgr2=0x559e510d3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.103+0000 7faac3e00640 0 client.31019 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.103+0000 7faaeba83740 1 -- 172.21.15.47:0/1135756089 shutdown_connections 2024-03-21T11:40:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.103+0000 7faaeba83740 1 -- 172.21.15.47:0/1135756089 wait complete. 2024-03-21T11:40:19.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.103+0000 7faaeba83740 20 client.31019 trim_cache size 0 max 16384 2024-03-21T11:40:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.137+0000 7faaeba83740 1 Processor -- start 2024-03-21T11:40:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.137+0000 7faaeba83740 1 -- start start 2024-03-21T11:40:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.138+0000 7faaeba83740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510d8e80 0x559e510daa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.138+0000 7faaeba83740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510ea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.138+0000 7faaeba83740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510de9e0 0x559e510d7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.138+0000 7faaeba83740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac400f9d0 con 0x559e510d8e80 2024-03-21T11:40:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.138+0000 7faaeba83740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac4011350 con 0x559e510d9870 2024-03-21T11:40:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.138+0000 7faaeba83740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac4010030 con 0x559e510de9e0 2024-03-21T11:40:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.138+0000 7faae2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510de9e0 0x559e510d7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.138+0000 7faae1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510ea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.138+0000 7faae2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510de9e0 0x559e510d7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46770/0 (socket says 172.21.15.47:46770) 2024-03-21T11:40:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.138+0000 7faae2800640 1 -- 172.21.15.47:0/1814721016 learned_addr learned my addr 172.21.15.47:0/1814721016 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:19.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.138+0000 7faae1e00640 1 --2- 172.21.15.47:0/1814721016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510d8e80 0x559e510daa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.139+0000 7faae2800640 1 -- 172.21.15.47:0/1814721016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 msgr2=0x559e510ea420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.139+0000 7faae2800640 1 --2- 172.21.15.47:0/1814721016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510ea420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.139+0000 7faae2800640 1 -- 172.21.15.47:0/1814721016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510d8e80 msgr2=0x559e510daa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.139+0000 7faae2800640 1 --2- 172.21.15.47:0/1814721016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510d8e80 0x559e510daa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.139+0000 7faae2800640 1 -- 172.21.15.47:0/1814721016 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e50d120f0 con 0x559e510de9e0 2024-03-21T11:40:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.139+0000 7faae2800640 1 --2- 172.21.15.47:0/1814721016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510de9e0 0x559e510d7a30 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7faad0036040 tx=0x7faad0041070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d2685640367173c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.140+0000 7faacea00640 1 -- 172.21.15.47:0/1814721016 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0004b40 con 0x559e510de9e0 2024-03-21T11:40:19.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.140+0000 7faacea00640 1 -- 172.21.15.47:0/1814721016 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faad0004ce0 con 0x559e510de9e0 2024-03-21T11:40:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.140+0000 7faacea00640 1 -- 172.21.15.47:0/1814721016 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0003850 con 0x559e510de9e0 2024-03-21T11:40:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.141+0000 7faaeba83740 1 -- 172.21.15.47:0/1814721016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510de9e0 msgr2=0x559e510d7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.141+0000 7faaeba83740 1 --2- 172.21.15.47:0/1814721016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510de9e0 0x559e510d7a30 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7faad0036040 tx=0x7faad0041070 comp rx=0 tx=0).stop 2024-03-21T11:40:19.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.142+0000 7faaeba83740 1 -- 172.21.15.47:0/1814721016 shutdown_connections 2024-03-21T11:40:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.142+0000 7faaeba83740 1 --2- 172.21.15.47:0/1814721016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510de9e0 0x559e510d7a30 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.142+0000 7faaeba83740 1 --2- 172.21.15.47:0/1814721016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510ea420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.142+0000 7faaeba83740 1 --2- 172.21.15.47:0/1814721016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510d8e80 0x559e510daa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.142+0000 7faaeba83740 1 -- 172.21.15.47:0/1814721016 >> 172.21.15.47:0/1814721016 conn(0x559e510a1200 msgr2=0x559e510da310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.142+0000 7faaeba83740 1 -- 172.21.15.47:0/1814721016 shutdown_connections 2024-03-21T11:40:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.142+0000 7faaeba83740 1 -- 172.21.15.47:0/1814721016 wait complete. 2024-03-21T11:40:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:18 smithi100 ceph-mon[36245]: pgmap v909: 129 pgs: 129 active+clean; 412 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.6 MiB/s wr, 461 op/s 2024-03-21T11:40:19.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.142+0000 7faaeba83740 1 -- start start 2024-03-21T11:40:19.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.143+0000 7faaeba83740 1 Processor -- start 2024-03-21T11:40:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.145+0000 7faaeba83740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d8e80 0x559e510e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.145+0000 7faaeba83740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.145+0000 7faaeba83740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 0x559e510e9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.145+0000 7faaeba83740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac4010cf0 con 0x559e510de9e0 2024-03-21T11:40:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.145+0000 7faaeba83740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac4010690 con 0x559e510d9870 2024-03-21T11:40:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.145+0000 7faaeba83740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac400db90 con 0x559e510d8e80 2024-03-21T11:40:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.145+0000 7faaeba83740 10 client.0 _ll_register_callbacks cb 0x7faad409b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.145+0000 7faae1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.145+0000 7faae1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510e82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55060/0 (socket says 172.21.15.47:55060) 2024-03-21T11:40:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.145+0000 7faae1400640 1 -- 172.21.15.47:0/929269495 learned_addr learned my addr 172.21.15.47:0/929269495 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.146+0000 7faae1e00640 1 --2- 172.21.15.47:0/929269495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d8e80 0x559e510e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.146+0000 7faae2800640 1 --2- 172.21.15.47:0/929269495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 0x559e510e9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.146+0000 7faae1400640 1 -- 172.21.15.47:0/929269495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d8e80 msgr2=0x559e510e7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:19.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.146+0000 7faae1400640 1 --2- 172.21.15.47:0/929269495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d8e80 0x559e510e7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.146+0000 7faae1400640 1 -- 172.21.15.47:0/929269495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 msgr2=0x559e510e9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.146+0000 7faae1400640 1 --2- 172.21.15.47:0/929269495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 0x559e510e9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.146+0000 7faae1400640 1 -- 172.21.15.47:0/929269495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e510e98f0 con 0x559e510d9870 2024-03-21T11:40:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.146+0000 7faae1400640 1 --2- 172.21.15.47:0/929269495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510e82f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7faad4097f30 tx=0x7faad409b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=284da2ade0b0bcf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.146+0000 7faac3e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.147+0000 7faac3e00640 1 -- 172.21.15.47:0/929269495 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad407e020 con 0x559e510d9870 2024-03-21T11:40:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.147+0000 7faac3e00640 1 -- 172.21.15.47:0/929269495 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7faad409b460 con 0x559e510d9870 2024-03-21T11:40:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.147+0000 7faaeba83740 1 -- 172.21.15.47:0/929269495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faad0004ce0 con 0x559e510d9870 2024-03-21T11:40:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.147+0000 7faac3e00640 1 -- 172.21.15.47:0/929269495 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7faad4010c80 con 0x559e510d9870 2024-03-21T11:40:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.147+0000 7faaeba83740 10 client.31025 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.147+0000 7faaeba83740 1 -- 172.21.15.47:0/929269495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559e5114a8b0 con 0x559e510d9870 2024-03-21T11:40:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.148+0000 7faac3e00640 1 -- 172.21.15.47:0/929269495 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faad0004ce0 con 0x559e510d9870 2024-03-21T11:40:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.148+0000 7faac3e00640 1 -- 172.21.15.47:0/929269495 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7faad40a10e0 con 0x559e510d9870 2024-03-21T11:40:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.148+0000 7faaeba83740 10 client.31025 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.148+0000 7faaeba83740 10 client.31025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.148+0000 7faaeba83740 20 client.31025 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.148+0000 7faaeba83740 1 --2- 172.21.15.47:0/929269495 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e5114b1a0 0x559e5114d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.148+0000 7faaeba83740 4 client.31025 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:40:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.148+0000 7faaeba83740 1 -- 172.21.15.47:0/929269495 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559e51045ae0 con 0x559e5114b1a0 2024-03-21T11:40:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.149+0000 7faae1e00640 1 --2- 172.21.15.47:0/929269495 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e5114b1a0 0x559e5114d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.149+0000 7faae1e00640 1 --2- 172.21.15.47:0/929269495 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e5114b1a0 0x559e5114d5c0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b18e32b717111137 server_cookie=7867362b9e8ff078 in_seq=0 out_seq=0 2024-03-21T11:40:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.149+0000 7faac3e00640 10 client.31025 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.151+0000 7faac3e00640 1 -- 172.21.15.47:0/929269495 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559e51045ae0 con 0x559e5114b1a0 2024-03-21T11:40:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.151+0000 7faac3e00640 10 client.31025 handle_command_reply: tid=0 2024-03-21T11:40:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.152+0000 7faaeba83740 1 client.31025 shutdown 2024-03-21T11:40:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.152+0000 7faaeba83740 1 -- 172.21.15.47:0/929269495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 msgr2=0x559e510e82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.152+0000 7faaeba83740 1 --2- 172.21.15.47:0/929269495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510e82f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7faad4097f30 tx=0x7faad409b890 comp rx=0 tx=0).stop 2024-03-21T11:40:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.152+0000 7faaeba83740 1 -- 172.21.15.47:0/929269495 shutdown_connections 2024-03-21T11:40:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.152+0000 7faaeba83740 1 --2- 172.21.15.47:0/929269495 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559e5114b1a0 0x559e5114d5c0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.152+0000 7faaeba83740 1 --2- 172.21.15.47:0/929269495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e510de9e0 0x559e510e9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.152+0000 7faaeba83740 1 --2- 172.21.15.47:0/929269495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e510d9870 0x559e510e82f0 secure :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7faad4097f30 tx=0x7faad409b890 comp rx=0 tx=0).stop 2024-03-21T11:40:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.152+0000 7faaeba83740 1 --2- 172.21.15.47:0/929269495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e510d8e80 0x559e510e7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.152+0000 7faaeba83740 1 -- 172.21.15.47:0/929269495 >> 172.21.15.47:0/929269495 conn(0x559e510a1200 msgr2=0x559e510daba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.152+0000 7faac3e00640 0 client.31025 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.153+0000 7faaeba83740 1 -- 172.21.15.47:0/929269495 shutdown_connections 2024-03-21T11:40:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.154+0000 7faaeba83740 1 -- 172.21.15.47:0/929269495 wait complete. 2024-03-21T11:40:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.154+0000 7faaeba83740 20 client.31025 trim_cache size 0 max 16384 2024-03-21T11:40:19.155 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:40:19.155 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:40:19.155 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:40:19.155 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:40:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.155+0000 7faae8e00640 1 -- 172.21.15.47:0/1815226602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faac80a3020 msgr2=0x7faac80a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.155+0000 7faae8e00640 1 --2- 172.21.15.47:0/1815226602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faac80a3020 0x7faac80a54e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7faad0005ab0 tx=0x7faad002e040 comp rx=0 tx=0).stop 2024-03-21T11:40:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.155+0000 7faae8e00640 1 -- 172.21.15.47:0/1815226602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae4100870 msgr2=0x7faae419e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.155+0000 7faae8e00640 1 --2- 172.21.15.47:0/1815226602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae4100870 0x7faae419e4f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7faad400e9f0 tx=0x7faad400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.155+0000 7faae8e00640 1 -- 172.21.15.47:0/1815226602 shutdown_connections 2024-03-21T11:40:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.155+0000 7faae8e00640 1 --2- 172.21.15.47:0/1815226602 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faac80a3020 0x7faac80a54e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.155+0000 7faae8e00640 1 --2- 172.21.15.47:0/1815226602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faae4101210 0x7faae41a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.155+0000 7faae8e00640 1 --2- 172.21.15.47:0/1815226602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae4100870 0x7faae419e4f0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.155+0000 7faae8e00640 1 --2- 172.21.15.47:0/1815226602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faae40fff10 0x7faae419dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.155+0000 7faae8e00640 1 -- 172.21.15.47:0/1815226602 >> 172.21.15.47:0/1815226602 conn(0x7faae40fbb30 msgr2=0x7faae40fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.156+0000 7faae8e00640 1 -- 172.21.15.47:0/1815226602 shutdown_connections 2024-03-21T11:40:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.156+0000 7faae8e00640 1 -- 172.21.15.47:0/1815226602 wait complete. 2024-03-21T11:40:19.182 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:40:19.182 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:40:19.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.421+0000 7f4ff5e00640 1 -- 172.21.15.47:0/2905932536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ff01071d0 msgr2=0x7f4ff0109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.421+0000 7f4ff5e00640 1 --2- 172.21.15.47:0/2905932536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ff01071d0 0x7f4ff0109620 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009ca0 tx=0x7f4fe802c100 comp rx=0 tx=0).stop 2024-03-21T11:40:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.422+0000 7f4ff5e00640 1 -- 172.21.15.47:0/2905932536 shutdown_connections 2024-03-21T11:40:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.422+0000 7f4ff5e00640 1 --2- 172.21.15.47:0/2905932536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ff01071d0 0x7f4ff0109620 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.422+0000 7f4ff5e00640 1 --2- 172.21.15.47:0/2905932536 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff01048a0 0x7f4ff0106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.422+0000 7f4ff5e00640 1 --2- 172.21.15.47:0/2905932536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff0101f70 0x7f4ff0104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.422+0000 7f4ff5e00640 1 -- 172.21.15.47:0/2905932536 >> 172.21.15.47:0/2905932536 conn(0x7f4ff00fb980 msgr2=0x7f4ff00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:19.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.424+0000 7f4ff5e00640 1 -- 172.21.15.47:0/2905932536 shutdown_connections 2024-03-21T11:40:19.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.424+0000 7f4ff5e00640 1 -- 172.21.15.47:0/2905932536 wait complete. 2024-03-21T11:40:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.425+0000 7f4ff5e00640 1 Processor -- start 2024-03-21T11:40:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.426+0000 7f4ff5e00640 1 -- start start 2024-03-21T11:40:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.426+0000 7f4ff5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff0101f70 0x7f4ff0195500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.426+0000 7f4ff5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff01048a0 0x7f4ff0195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.426+0000 7f4ff5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ff01071d0 0x7f4ff01a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.426+0000 7f4ff5e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff010d180 con 0x7f4ff01048a0 2024-03-21T11:40:19.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.426+0000 7f4ff5e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff010d000 con 0x7f4ff0101f70 2024-03-21T11:40:19.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.426+0000 7f4ff5e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff010d300 con 0x7f4ff01071d0 2024-03-21T11:40:19.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.427+0000 7f4fefe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff01048a0 0x7f4ff0195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.427+0000 7f4fefe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff01048a0 0x7f4ff0195a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59792/0 (socket says 172.21.15.47:59792) 2024-03-21T11:40:19.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.427+0000 7f4fefe00640 1 -- 172.21.15.47:0/1781662927 learned_addr learned my addr 172.21.15.47:0/1781662927 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:19.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.427+0000 7f4ff4a00640 1 --2- 172.21.15.47:0/1781662927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff0101f70 0x7f4ff0195500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.427+0000 7f4ff5400640 1 --2- 172.21.15.47:0/1781662927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ff01071d0 0x7f4ff01a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.427+0000 7f4fefe00640 1 -- 172.21.15.47:0/1781662927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ff01071d0 msgr2=0x7f4ff01a5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:19.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.427+0000 7f4fefe00640 1 --2- 172.21.15.47:0/1781662927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ff01071d0 0x7f4ff01a5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.427+0000 7f4fefe00640 1 -- 172.21.15.47:0/1781662927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff0101f70 msgr2=0x7f4ff0195500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:19.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.427+0000 7f4fefe00640 1 --2- 172.21.15.47:0/1781662927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff0101f70 0x7f4ff0195500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.427+0000 7f4fefe00640 1 -- 172.21.15.47:0/1781662927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff01a59d0 con 0x7f4ff01048a0 2024-03-21T11:40:19.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.427+0000 7f4fefe00640 1 --2- 172.21.15.47:0/1781662927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff01048a0 0x7f4ff0195a40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc005fd0 tx=0x7f4fdc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:19.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.428+0000 7f4fed600640 1 -- 172.21.15.47:0/1781662927 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc017070 con 0x7f4ff01048a0 2024-03-21T11:40:19.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.428+0000 7f4ff5e00640 1 -- 172.21.15.47:0/1781662927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff01a5c60 con 0x7f4ff01048a0 2024-03-21T11:40:19.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.428+0000 7f4ff5e00640 1 -- 172.21.15.47:0/1781662927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff01a61d0 con 0x7f4ff01048a0 2024-03-21T11:40:19.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.428+0000 7f4fed600640 1 -- 172.21.15.47:0/1781662927 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fdc009c30 con 0x7f4ff01048a0 2024-03-21T11:40:19.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.428+0000 7f4fed600640 1 -- 172.21.15.47:0/1781662927 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc003e20 con 0x7f4ff01048a0 2024-03-21T11:40:19.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.429+0000 7f4ff5e00640 1 -- 172.21.15.47:0/1781662927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd00052c0 con 0x7f4ff01048a0 2024-03-21T11:40:19.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.432+0000 7f4fed600640 1 -- 172.21.15.47:0/1781662927 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4fdc0035f0 con 0x7f4ff01048a0 2024-03-21T11:40:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.435+0000 7f4fed600640 1 --2- 172.21.15.47:0/1781662927 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4fd40a2fa0 0x7f4fd40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:19.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.435+0000 7f4fed600640 1 -- 172.21.15.47:0/1781662927 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4fdc0cf380 con 0x7f4ff01048a0 2024-03-21T11:40:19.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.435+0000 7f4ff4a00640 1 --2- 172.21.15.47:0/1781662927 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4fd40a2fa0 0x7f4fd40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.436+0000 7f4ff4a00640 1 --2- 172.21.15.47:0/1781662927 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4fd40a2fa0 0x7f4fd40a5460 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0005e10 tx=0x7f4fe0005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:19.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.440+0000 7f4fed600640 1 -- 172.21.15.47:0/1781662927 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4fdc020030 con 0x7f4ff01048a0 2024-03-21T11:40:19.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.767+0000 7f4ff5e00640 1 -- 172.21.15.47:0/1781662927 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fd00055b0 con 0x7f4ff01048a0 2024-03-21T11:40:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.768+0000 7f4fed600640 1 -- 172.21.15.47:0/1781662927 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f4fdc009de0 con 0x7f4ff01048a0 2024-03-21T11:40:19.769 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:40:19.769 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:39:47.303461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:40:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-21T11:40:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.771+0000 7f4ff5e00640 1 -- 172.21.15.47:0/1781662927 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4fd40a2fa0 msgr2=0x7f4fd40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.771+0000 7f4ff5e00640 1 --2- 172.21.15.47:0/1781662927 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4fd40a2fa0 0x7f4fd40a5460 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0005e10 tx=0x7f4fe0005da0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.771+0000 7f4ff5e00640 1 -- 172.21.15.47:0/1781662927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff01048a0 msgr2=0x7f4ff0195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.771+0000 7f4ff5e00640 1 --2- 172.21.15.47:0/1781662927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff01048a0 0x7f4ff0195a40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc005fd0 tx=0x7f4fdc00dde0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.771+0000 7f4ff5e00640 1 -- 172.21.15.47:0/1781662927 shutdown_connections 2024-03-21T11:40:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.771+0000 7f4ff5e00640 1 --2- 172.21.15.47:0/1781662927 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4fd40a2fa0 0x7f4fd40a5460 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.771+0000 7f4ff5e00640 1 --2- 172.21.15.47:0/1781662927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ff01071d0 0x7f4ff01a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.771+0000 7f4ff5e00640 1 --2- 172.21.15.47:0/1781662927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ff01048a0 0x7f4ff0195a40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.771+0000 7f4ff5e00640 1 --2- 172.21.15.47:0/1781662927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff0101f70 0x7f4ff0195500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.771+0000 7f4ff5e00640 1 -- 172.21.15.47:0/1781662927 >> 172.21.15.47:0/1781662927 conn(0x7f4ff00fb980 msgr2=0x7f4ff01015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.772+0000 7f4ff5e00640 1 -- 172.21.15.47:0/1781662927 shutdown_connections 2024-03-21T11:40:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:19.772+0000 7f4ff5e00640 1 -- 172.21.15.47:0/1781662927 wait complete. 2024-03-21T11:40:19.791 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:40:19.791 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:40:19.941 INFO:tasks.workunit.client.0.smithi047.stdout: 10 87946 25.35 MB/sec execute 63 sec latency 75.158 ms 2024-03-21T11:40:20.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.007+0000 7fe2dec00640 1 -- 172.21.15.47:0/3441159318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d81028c0 msgr2=0x7fe2d8102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.007+0000 7fe2dec00640 1 --2- 172.21.15.47:0/3441159318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d81028c0 0x7fe2d8102d40 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009900 tx=0x7fe2c402bc30 comp rx=0 tx=0).stop 2024-03-21T11:40:20.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.008+0000 7fe2dec00640 1 -- 172.21.15.47:0/3441159318 shutdown_connections 2024-03-21T11:40:20.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.008+0000 7fe2dec00640 1 --2- 172.21.15.47:0/3441159318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d8103280 0x7fe2d8109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.008+0000 7fe2dec00640 1 --2- 172.21.15.47:0/3441159318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d81028c0 0x7fe2d8102d40 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.008+0000 7fe2dec00640 1 --2- 172.21.15.47:0/3441159318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2d81016d0 0x7fe2d8101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.008+0000 7fe2dec00640 1 -- 172.21.15.47:0/3441159318 >> 172.21.15.47:0/3441159318 conn(0x7fe2d80fcc40 msgr2=0x7fe2d80ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.008+0000 7fe2dec00640 1 -- 172.21.15.47:0/3441159318 shutdown_connections 2024-03-21T11:40:20.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.008+0000 7fe2dec00640 1 -- 172.21.15.47:0/3441159318 wait complete. 2024-03-21T11:40:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.009+0000 7fe2dec00640 1 Processor -- start 2024-03-21T11:40:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.009+0000 7fe2dec00640 1 -- start start 2024-03-21T11:40:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.009+0000 7fe2dec00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d81016d0 0x7fe2d8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.009+0000 7fe2dec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2d81028c0 0x7fe2d8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.009+0000 7fe2dec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d8103280 0x7fe2d819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.009+0000 7fe2dec00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d810d3a0 con 0x7fe2d81016d0 2024-03-21T11:40:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.009+0000 7fe2dec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d810d220 con 0x7fe2d8103280 2024-03-21T11:40:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.009+0000 7fe2dec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d810d520 con 0x7fe2d81028c0 2024-03-21T11:40:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.009+0000 7fe2d7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2d81028c0 0x7fe2d8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.010+0000 7fe2dcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d8103280 0x7fe2d819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.009+0000 7fe2d7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d81016d0 0x7fe2d8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.010+0000 7fe2d7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d81016d0 0x7fe2d8197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59814/0 (socket says 172.21.15.47:59814) 2024-03-21T11:40:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.010+0000 7fe2dcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d8103280 0x7fe2d819f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55092/0 (socket says 172.21.15.47:55092) 2024-03-21T11:40:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.010+0000 7fe2d7e00640 1 -- 172.21.15.47:0/1959779730 learned_addr learned my addr 172.21.15.47:0/1959779730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.010+0000 7fe2d7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2d81028c0 0x7fe2d8197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46814/0 (socket says 172.21.15.47:46814) 2024-03-21T11:40:20.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.010+0000 7fe2d7e00640 1 -- 172.21.15.47:0/1959779730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2d81028c0 msgr2=0x7fe2d8197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.010+0000 7fe2d7e00640 1 --2- 172.21.15.47:0/1959779730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2d81028c0 0x7fe2d8197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.010+0000 7fe2d7e00640 1 -- 172.21.15.47:0/1959779730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d8103280 msgr2=0x7fe2d819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.010+0000 7fe2d7e00640 1 --2- 172.21.15.47:0/1959779730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d8103280 0x7fe2d819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.010+0000 7fe2d7e00640 1 -- 172.21.15.47:0/1959779730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d819f670 con 0x7fe2d81016d0 2024-03-21T11:40:20.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.011+0000 7fe2d7e00640 1 --2- 172.21.15.47:0/1959779730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d81016d0 0x7fe2d8197a50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800e9c0 tx=0x7fe2c800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.011+0000 7fe2d4c00640 1 -- 172.21.15.47:0/1959779730 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c800cd80 con 0x7fe2d81016d0 2024-03-21T11:40:20.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.012+0000 7fe2d4c00640 1 -- 172.21.15.47:0/1959779730 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2c80035f0 con 0x7fe2d81016d0 2024-03-21T11:40:20.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.012+0000 7fe2dec00640 1 -- 172.21.15.47:0/1959779730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d819f960 con 0x7fe2d81016d0 2024-03-21T11:40:20.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.012+0000 7fe2d4c00640 1 -- 172.21.15.47:0/1959779730 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8010690 con 0x7fe2d81016d0 2024-03-21T11:40:20.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.012+0000 7fe2dec00640 1 -- 172.21.15.47:0/1959779730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d819fe30 con 0x7fe2d81016d0 2024-03-21T11:40:20.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.016+0000 7fe2d4c00640 1 -- 172.21.15.47:0/1959779730 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe2c80037a0 con 0x7fe2d81016d0 2024-03-21T11:40:20.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.017+0000 7fe2e1998740 1 Processor -- start 2024-03-21T11:40:20.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.018+0000 7fe2d4c00640 1 --2- 172.21.15.47:0/1959779730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2bc0a2f20 0x7fe2bc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.019+0000 7fe2d4c00640 1 -- 172.21.15.47:0/1959779730 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe2c80cfae0 con 0x7fe2d81016d0 2024-03-21T11:40:20.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.019+0000 7fe2e1998740 1 -- start start 2024-03-21T11:40:20.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.019+0000 7fe2e1998740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a0c20 0x55c6f06a1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.019+0000 7fe2e1998740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a1df0 0x55c6f06a2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.019+0000 7fe2e1998740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a27b0 0x55c6f06a9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.020+0000 7fe2d7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a1df0 0x55c6f06a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.020+0000 7fe2d7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a1df0 0x55c6f06a2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46820/0 (socket says 172.21.15.47:46820) 2024-03-21T11:40:20.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.020+0000 7fe2d7400640 1 -- 172.21.15.47:0/248854758 learned_addr learned my addr 172.21.15.47:0/248854758 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.020+0000 7fe2d7e00640 1 --2- 172.21.15.47:0/248854758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a0c20 0x55c6f06a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.020+0000 7fe2dcc00640 1 --2- 172.21.15.47:0/1959779730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2bc0a2f20 0x7fe2bc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.021+0000 7fe2e1998740 1 -- 172.21.15.47:0/248854758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f06012c0 con 0x55c6f06a0c20 2024-03-21T11:40:20.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.021+0000 7fe2e1998740 1 -- 172.21.15.47:0/248854758 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f05a9c90 con 0x55c6f06a27b0 2024-03-21T11:40:20.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.021+0000 7fe2e1998740 1 -- 172.21.15.47:0/248854758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f05fd360 con 0x55c6f06a1df0 2024-03-21T11:40:20.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.021+0000 7fe2d7e00640 1 -- 172.21.15.47:0/248854758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a1df0 msgr2=0x55c6f06a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.021+0000 7fe2d7e00640 1 --2- 172.21.15.47:0/248854758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a1df0 0x55c6f06a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.021+0000 7fe2d7e00640 1 -- 172.21.15.47:0/248854758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a27b0 msgr2=0x55c6f06a9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.021+0000 7fe2d7e00640 1 --2- 172.21.15.47:0/248854758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a27b0 0x55c6f06a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.021+0000 7fe2d7e00640 1 -- 172.21.15.47:0/248854758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6f05ce080 con 0x55c6f06a0c20 2024-03-21T11:40:20.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.022+0000 7fe2d7e00640 1 --2- 172.21.15.47:0/248854758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a0c20 0x55c6f06a1040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80cd2a0 tx=0x7fe2c809c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569d2089f1136555 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.022+0000 7fe2cd200640 1 -- 172.21.15.47:0/248854758 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c809ffd0 con 0x55c6f06a0c20 2024-03-21T11:40:20.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.022+0000 7fe2dcc00640 1 --2- 172.21.15.47:0/1959779730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2bc0a2f20 0x7fe2bc0a53e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0005fd0 tx=0x7fe2d0010740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.022+0000 7fe2cd200640 1 -- 172.21.15.47:0/248854758 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2c80a4890 con 0x55c6f06a0c20 2024-03-21T11:40:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.022+0000 7fe2cd200640 1 -- 172.21.15.47:0/248854758 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80a4660 con 0x55c6f06a0c20 2024-03-21T11:40:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.023+0000 7fe2e1998740 1 -- 172.21.15.47:0/248854758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a0c20 msgr2=0x55c6f06a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.023+0000 7fe2e1998740 1 --2- 172.21.15.47:0/248854758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a0c20 0x55c6f06a1040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80cd2a0 tx=0x7fe2c809c580 comp rx=0 tx=0).stop 2024-03-21T11:40:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.023+0000 7fe2e1998740 1 -- 172.21.15.47:0/248854758 shutdown_connections 2024-03-21T11:40:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.023+0000 7fe2e1998740 1 --2- 172.21.15.47:0/248854758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a27b0 0x55c6f06a9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.023+0000 7fe2e1998740 1 --2- 172.21.15.47:0/248854758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a1df0 0x55c6f06a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.023+0000 7fe2e1998740 1 --2- 172.21.15.47:0/248854758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a0c20 0x55c6f06a1040 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.023+0000 7fe2e1998740 1 -- 172.21.15.47:0/248854758 >> 172.21.15.47:0/248854758 conn(0x55c6f0697c00 msgr2=0x55c6f0698000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.023+0000 7fe2e1998740 1 -- 172.21.15.47:0/248854758 shutdown_connections 2024-03-21T11:40:20.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.023+0000 7fe2e1998740 1 -- 172.21.15.47:0/248854758 wait complete. 2024-03-21T11:40:20.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.024+0000 7fe2e1998740 1 -- start start 2024-03-21T11:40:20.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.024+0000 7fe2e1998740 1 Processor -- start 2024-03-21T11:40:20.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.024+0000 7fe2e1998740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a0c20 0x55c6f06aae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.024+0000 7fe2e1998740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a1df0 0x55c6f06ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.024+0000 7fe2e1998740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a27b0 0x55c6f06b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.024+0000 7fe2e1998740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f06a9760 con 0x55c6f06a1df0 2024-03-21T11:40:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.024+0000 7fe2e1998740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f05bd0d0 con 0x55c6f06a27b0 2024-03-21T11:40:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.025+0000 7fe2e1998740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6f06a98e0 con 0x55c6f06a0c20 2024-03-21T11:40:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.025+0000 7fe2d7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a0c20 0x55c6f06aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.025+0000 7fe2d7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a0c20 0x55c6f06aae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46834/0 (socket says 172.21.15.47:46834) 2024-03-21T11:40:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.025+0000 7fe2d7e00640 1 -- 172.21.15.47:0/2310894940 learned_addr learned my addr 172.21.15.47:0/2310894940 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.025+0000 7fe2e1998740 10 client.0 _ll_register_callbacks cb 0x55c6f05c1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.025+0000 7fe2d7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a1df0 0x55c6f06ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.025+0000 7fe2dcc00640 1 --2- 172.21.15.47:0/2310894940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a27b0 0x55c6f06b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.025+0000 7fe2d7e00640 1 -- 172.21.15.47:0/2310894940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a27b0 msgr2=0x55c6f06b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.025+0000 7fe2d7e00640 1 --2- 172.21.15.47:0/2310894940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a27b0 0x55c6f06b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.025+0000 7fe2d7e00640 1 -- 172.21.15.47:0/2310894940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a1df0 msgr2=0x55c6f06ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.025+0000 7fe2d7e00640 1 --2- 172.21.15.47:0/2310894940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a1df0 0x55c6f06ab350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.025+0000 7fe2d7e00640 1 -- 172.21.15.47:0/2310894940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6f03153f0 con 0x55c6f06a0c20 2024-03-21T11:40:20.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.025+0000 7fe2d7400640 1 --2- 172.21.15.47:0/2310894940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a1df0 0x55c6f06ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:40:20.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.026+0000 7fe2d7e00640 1 --2- 172.21.15.47:0/2310894940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a0c20 0x55c6f06aae10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800c6a0 tx=0x7fe2c8098740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deb7a7e05c01dd56 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.026+0000 7fe2c1600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:40:20.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.026+0000 7fe2c1600640 1 -- 172.21.15.47:0/2310894940 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c809b2d0 con 0x55c6f06a0c20 2024-03-21T11:40:20.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.026+0000 7fe2e1998740 1 -- 172.21.15.47:0/2310894940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2c80a4890 con 0x55c6f06a0c20 2024-03-21T11:40:20.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.026+0000 7fe2c1600640 1 -- 172.21.15.47:0/2310894940 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe2c809b470 con 0x55c6f06a0c20 2024-03-21T11:40:20.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.026+0000 7fe2c1600640 1 -- 172.21.15.47:0/2310894940 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2c8004000 con 0x55c6f06a0c20 2024-03-21T11:40:20.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.026+0000 7fe2e1998740 10 client.31027 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:20.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.026+0000 7fe2e1998740 1 -- 172.21.15.47:0/2310894940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c6f03df7d0 con 0x55c6f06a0c20 2024-03-21T11:40:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.027+0000 7fe2c1600640 1 -- 172.21.15.47:0/2310894940 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2c80a4890 con 0x55c6f06a0c20 2024-03-21T11:40:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.027+0000 7fe2c1600640 1 -- 172.21.15.47:0/2310894940 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe2c80b8070 con 0x55c6f06a0c20 2024-03-21T11:40:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.027+0000 7fe2e1998740 10 client.31027 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.027+0000 7fe2e1998740 10 client.31027 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:40:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.027+0000 7fe2e1998740 20 client.31027 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.027+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2310894940 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c6f06c74e0 0x55c6f06c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.027+0000 7fe2e1998740 4 client.31027 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.027+0000 7fe2e1998740 1 -- 172.21.15.47:0/2310894940 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6f060e800 con 0x55c6f06c74e0 2024-03-21T11:40:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.028+0000 7fe2d7400640 1 --2- 172.21.15.47:0/2310894940 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c6f06c74e0 0x55c6f06c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.028+0000 7fe2d7400640 1 --2- 172.21.15.47:0/2310894940 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c6f06c74e0 0x55c6f06c98d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a386b066b136260e server_cookie=1cf7651af842b91 in_seq=0 out_seq=0 2024-03-21T11:40:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.028+0000 7fe2c1600640 10 client.31027 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:40:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.029+0000 7fe2c1600640 1 -- 172.21.15.47:0/2310894940 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c6f060e800 con 0x55c6f06c74e0 2024-03-21T11:40:20.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.029+0000 7fe2c1600640 10 client.31027 handle_command_reply: tid=0 2024-03-21T11:40:20.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.030+0000 7fe2e1998740 1 client.31027 shutdown 2024-03-21T11:40:20.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.030+0000 7fe2e1998740 1 -- 172.21.15.47:0/2310894940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a0c20 msgr2=0x55c6f06aae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.030+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2310894940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a0c20 0x55c6f06aae10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800c6a0 tx=0x7fe2c8098740 comp rx=0 tx=0).stop 2024-03-21T11:40:20.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.030+0000 7fe2e1998740 1 -- 172.21.15.47:0/2310894940 shutdown_connections 2024-03-21T11:40:20.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.030+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2310894940 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c6f06c74e0 0x55c6f06c98d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.030+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2310894940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a27b0 0x55c6f06b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.030+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2310894940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a1df0 0x55c6f06ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.030+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2310894940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a0c20 0x55c6f06aae10 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.030+0000 7fe2e1998740 1 -- 172.21.15.47:0/2310894940 >> 172.21.15.47:0/2310894940 conn(0x55c6f0697c00 msgr2=0x55c6f0696eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.031+0000 7fe2c1600640 0 client.31027 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:40:20.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.031+0000 7fe2e1998740 1 -- 172.21.15.47:0/2310894940 shutdown_connections 2024-03-21T11:40:20.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.031+0000 7fe2e1998740 1 -- 172.21.15.47:0/2310894940 wait complete. 2024-03-21T11:40:20.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.031+0000 7fe2e1998740 20 client.31027 trim_cache size 0 max 16384 2024-03-21T11:40:20.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.049+0000 7fe2e1998740 1 Processor -- start 2024-03-21T11:40:20.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.049+0000 7fe2e1998740 1 -- start start 2024-03-21T11:40:20.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.049+0000 7fe2e1998740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a0c20 0x55c6f06a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.049+0000 7fe2e1998740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a1df0 0x55c6f06b2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.049+0000 7fe2e1998740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a27b0 0x55c6f06a5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.049+0000 7fe2e1998740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d80751d0 con 0x55c6f06a27b0 2024-03-21T11:40:20.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.049+0000 7fe2e1998740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d81a1080 con 0x55c6f06a1df0 2024-03-21T11:40:20.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.049+0000 7fe2e1998740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d8075830 con 0x55c6f06a0c20 2024-03-21T11:40:20.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.050+0000 7fe2d7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a1df0 0x55c6f06b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.050+0000 7fe2d7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a0c20 0x55c6f06a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.050+0000 7fe2d7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a1df0 0x55c6f06b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55114/0 (socket says 172.21.15.47:55114) 2024-03-21T11:40:20.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.050+0000 7fe2dcc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a27b0 0x55c6f06a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.050+0000 7fe2d7400640 1 -- 172.21.15.47:0/2452537964 learned_addr learned my addr 172.21.15.47:0/2452537964 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.050+0000 7fe2d7400640 1 -- 172.21.15.47:0/2452537964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a0c20 msgr2=0x55c6f06a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.050+0000 7fe2d7400640 1 --2- 172.21.15.47:0/2452537964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a0c20 0x55c6f06a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.050+0000 7fe2d7400640 1 -- 172.21.15.47:0/2452537964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a27b0 msgr2=0x55c6f06a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.050+0000 7fe2d7400640 1 --2- 172.21.15.47:0/2452537964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a27b0 0x55c6f06a5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.050+0000 7fe2d7400640 1 -- 172.21.15.47:0/2452537964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6f06a8970 con 0x55c6f06a1df0 2024-03-21T11:40:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.050+0000 7fe2dcc00640 1 --2- 172.21.15.47:0/2452537964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a27b0 0x55c6f06a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.050+0000 7fe2d7400640 1 --2- 172.21.15.47:0/2452537964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a1df0 0x55c6f06b2350 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40096c0 tx=0x7fe2c402df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c11e6268df274249 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.051+0000 7fe2c3e00640 1 -- 172.21.15.47:0/2452537964 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c402ca80 con 0x55c6f06a1df0 2024-03-21T11:40:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.051+0000 7fe2c3e00640 1 -- 172.21.15.47:0/2452537964 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2c402cc20 con 0x55c6f06a1df0 2024-03-21T11:40:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.051+0000 7fe2c3e00640 1 -- 172.21.15.47:0/2452537964 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4005150 con 0x55c6f06a1df0 2024-03-21T11:40:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.051+0000 7fe2e1998740 1 -- 172.21.15.47:0/2452537964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a1df0 msgr2=0x55c6f06b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.051+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2452537964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a1df0 0x55c6f06b2350 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40096c0 tx=0x7fe2c402df90 comp rx=0 tx=0).stop 2024-03-21T11:40:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.051+0000 7fe2e1998740 1 -- 172.21.15.47:0/2452537964 shutdown_connections 2024-03-21T11:40:20.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.052+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2452537964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a27b0 0x55c6f06a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.052+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2452537964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a1df0 0x55c6f06b2350 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.052+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2452537964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a0c20 0x55c6f06a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.052+0000 7fe2e1998740 1 -- 172.21.15.47:0/2452537964 >> 172.21.15.47:0/2452537964 conn(0x55c6f0697c00 msgr2=0x55c6f069b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.052+0000 7fe2e1998740 1 -- 172.21.15.47:0/2452537964 shutdown_connections 2024-03-21T11:40:20.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.052+0000 7fe2e1998740 1 -- 172.21.15.47:0/2452537964 wait complete. 2024-03-21T11:40:20.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.052+0000 7fe2e1998740 1 -- start start 2024-03-21T11:40:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.052+0000 7fe2e1998740 1 Processor -- start 2024-03-21T11:40:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.053+0000 7fe2e1998740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a0c20 0x55c6f06ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.053+0000 7fe2e1998740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a1df0 0x55c6f06ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.053+0000 7fe2e1998740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a27b0 0x55c6f06b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.053+0000 7fe2e1998740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d81a0a20 con 0x55c6f06a0c20 2024-03-21T11:40:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.053+0000 7fe2e1998740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d81a03c0 con 0x55c6f06a1df0 2024-03-21T11:40:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.053+0000 7fe2e1998740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d8073390 con 0x55c6f06a27b0 2024-03-21T11:40:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.053+0000 7fe2e1998740 10 client.0 _ll_register_callbacks cb 0x7fe2c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.053+0000 7fe2d7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a1df0 0x55c6f06ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.053+0000 7fe2dcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a27b0 0x55c6f06b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.053+0000 7fe2d7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a1df0 0x55c6f06ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55120/0 (socket says 172.21.15.47:55120) 2024-03-21T11:40:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.053+0000 7fe2dcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a27b0 0x55c6f06b19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46848/0 (socket says 172.21.15.47:46848) 2024-03-21T11:40:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.053+0000 7fe2d7400640 1 -- 172.21.15.47:0/2088557041 learned_addr learned my addr 172.21.15.47:0/2088557041 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.053+0000 7fe2d7e00640 1 --2- 172.21.15.47:0/2088557041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a0c20 0x55c6f06ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.054+0000 7fe2d7400640 1 -- 172.21.15.47:0/2088557041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a27b0 msgr2=0x55c6f06b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.054+0000 7fe2d7400640 1 --2- 172.21.15.47:0/2088557041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a27b0 0x55c6f06b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.054+0000 7fe2d7400640 1 -- 172.21.15.47:0/2088557041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a0c20 msgr2=0x55c6f06ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.054+0000 7fe2d7400640 1 --2- 172.21.15.47:0/2088557041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a0c20 0x55c6f06ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.054+0000 7fe2d7400640 1 -- 172.21.15.47:0/2088557041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6f06b1fd0 con 0x55c6f06a1df0 2024-03-21T11:40:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.054+0000 7fe2d7400640 1 --2- 172.21.15.47:0/2088557041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a1df0 0x55c6f06ae9c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe2c402c240 tx=0x7fe2c4005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2682190db49ab1b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.054+0000 7fe2c1600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.054+0000 7fe2c1600640 1 -- 172.21.15.47:0/2088557041 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4059030 con 0x55c6f06a1df0 2024-03-21T11:40:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.054+0000 7fe2c1600640 1 -- 172.21.15.47:0/2088557041 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe2c40046c0 con 0x55c6f06a1df0 2024-03-21T11:40:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.055+0000 7fe2e1998740 1 -- 172.21.15.47:0/2088557041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2c402cc20 con 0x55c6f06a1df0 2024-03-21T11:40:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.055+0000 7fe2c1600640 1 -- 172.21.15.47:0/2088557041 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2c4004900 con 0x55c6f06a1df0 2024-03-21T11:40:20.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.055+0000 7fe2e1998740 10 client.31043 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:20.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.055+0000 7fe2e1998740 1 -- 172.21.15.47:0/2088557041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c6f07235f0 con 0x55c6f06a1df0 2024-03-21T11:40:20.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.055+0000 7fe2c1600640 1 -- 172.21.15.47:0/2088557041 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2c402cc20 con 0x55c6f06a1df0 2024-03-21T11:40:20.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.055+0000 7fe2c1600640 1 -- 172.21.15.47:0/2088557041 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe2c404ca00 con 0x55c6f06a1df0 2024-03-21T11:40:20.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.056+0000 7fe2e1998740 10 client.31043 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:20.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.056+0000 7fe2e1998740 10 client.31043 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:40:20.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.056+0000 7fe2e1998740 20 client.31043 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:20.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.056+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2088557041 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c6f0723f10 0x55c6f0726330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.056+0000 7fe2e1998740 4 client.31043 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:40:20.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.056+0000 7fe2e1998740 1 -- 172.21.15.47:0/2088557041 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6f0608b50 con 0x55c6f0723f10 2024-03-21T11:40:20.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.056+0000 7fe2d7e00640 1 --2- 172.21.15.47:0/2088557041 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c6f0723f10 0x55c6f0726330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.057+0000 7fe2d7e00640 1 --2- 172.21.15.47:0/2088557041 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c6f0723f10 0x55c6f0726330 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=51071b420f0200ca server_cookie=27979e8207f1037d in_seq=0 out_seq=0 2024-03-21T11:40:20.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.057+0000 7fe2c1600640 10 client.31043 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:40:20.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.057+0000 7fe2c1600640 1 -- 172.21.15.47:0/2088557041 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6f0608b50 con 0x55c6f0723f10 2024-03-21T11:40:20.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.058+0000 7fe2c1600640 10 client.31043 handle_command_reply: tid=0 2024-03-21T11:40:20.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.058+0000 7fe2e1998740 1 client.31043 shutdown 2024-03-21T11:40:20.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.058+0000 7fe2e1998740 1 -- 172.21.15.47:0/2088557041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a1df0 msgr2=0x55c6f06ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.058+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2088557041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a1df0 0x55c6f06ae9c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe2c402c240 tx=0x7fe2c4005b50 comp rx=0 tx=0).stop 2024-03-21T11:40:20.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.058+0000 7fe2e1998740 1 -- 172.21.15.47:0/2088557041 shutdown_connections 2024-03-21T11:40:20.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.058+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2088557041 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55c6f0723f10 0x55c6f0726330 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.058+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2088557041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c6f06a27b0 0x55c6f06b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.058+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2088557041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6f06a1df0 0x55c6f06ae9c0 secure :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe2c402c240 tx=0x7fe2c4005b50 comp rx=0 tx=0).stop 2024-03-21T11:40:20.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.058+0000 7fe2e1998740 1 --2- 172.21.15.47:0/2088557041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6f06a0c20 0x55c6f06ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.058+0000 7fe2c1600640 0 client.31043 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:40:20.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.058+0000 7fe2e1998740 1 -- 172.21.15.47:0/2088557041 >> 172.21.15.47:0/2088557041 conn(0x55c6f0697c00 msgr2=0x55c6f0696620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.058+0000 7fe2e1998740 1 -- 172.21.15.47:0/2088557041 shutdown_connections 2024-03-21T11:40:20.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.058+0000 7fe2e1998740 1 -- 172.21.15.47:0/2088557041 wait complete. 2024-03-21T11:40:20.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.058+0000 7fe2e1998740 20 client.31043 trim_cache size 0 max 16384 2024-03-21T11:40:20.063 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:40:20.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.059+0000 7fe2dec00640 1 -- 172.21.15.47:0/1959779730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2bc0a2f20 msgr2=0x7fe2bc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.059+0000 7fe2dec00640 1 --2- 172.21.15.47:0/1959779730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2bc0a2f20 0x7fe2bc0a53e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0005fd0 tx=0x7fe2d0010740 comp rx=0 tx=0).stop 2024-03-21T11:40:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.059+0000 7fe2dec00640 1 -- 172.21.15.47:0/1959779730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d81016d0 msgr2=0x7fe2d8197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.059+0000 7fe2dec00640 1 --2- 172.21.15.47:0/1959779730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d81016d0 0x7fe2d8197a50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800e9c0 tx=0x7fe2c800ee80 comp rx=0 tx=0).stop 2024-03-21T11:40:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.059+0000 7fe2dec00640 1 -- 172.21.15.47:0/1959779730 shutdown_connections 2024-03-21T11:40:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.059+0000 7fe2dec00640 1 --2- 172.21.15.47:0/1959779730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe2bc0a2f20 0x7fe2bc0a53e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.059+0000 7fe2dec00640 1 --2- 172.21.15.47:0/1959779730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2d8103280 0x7fe2d819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.059+0000 7fe2dec00640 1 --2- 172.21.15.47:0/1959779730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2d81028c0 0x7fe2d8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.059+0000 7fe2dec00640 1 --2- 172.21.15.47:0/1959779730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2d81016d0 0x7fe2d8197a50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.059+0000 7fe2dec00640 1 -- 172.21.15.47:0/1959779730 >> 172.21.15.47:0/1959779730 conn(0x7fe2d80fcc40 msgr2=0x7fe2d8105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.059+0000 7fe2dec00640 1 -- 172.21.15.47:0/1959779730 shutdown_connections 2024-03-21T11:40:20.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.059+0000 7fe2dec00640 1 -- 172.21.15.47:0/1959779730 wait complete. 2024-03-21T11:40:20.078 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:40:20.078 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:40:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.321+0000 7f2e95200640 1 -- 172.21.15.47:0/892024948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e901016c0 msgr2=0x7f2e90101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.321+0000 7f2e95200640 1 --2- 172.21.15.47:0/892024948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e901016c0 0x7f2e90101ae0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009900 tx=0x7f2e8002bc40 comp rx=0 tx=0).stop 2024-03-21T11:40:20.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.321+0000 7f2e95200640 1 -- 172.21.15.47:0/892024948 shutdown_connections 2024-03-21T11:40:20.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.321+0000 7f2e95200640 1 --2- 172.21.15.47:0/892024948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e90103270 0x7f2e90109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.321+0000 7f2e95200640 1 --2- 172.21.15.47:0/892024948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e901028b0 0x7f2e90102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.321+0000 7f2e95200640 1 --2- 172.21.15.47:0/892024948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e901016c0 0x7f2e90101ae0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.321+0000 7f2e95200640 1 -- 172.21.15.47:0/892024948 >> 172.21.15.47:0/892024948 conn(0x7f2e900fcc30 msgr2=0x7f2e900ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.321+0000 7f2e95200640 1 -- 172.21.15.47:0/892024948 shutdown_connections 2024-03-21T11:40:20.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.321+0000 7f2e95200640 1 -- 172.21.15.47:0/892024948 wait complete. 2024-03-21T11:40:20.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.322+0000 7f2e95200640 1 Processor -- start 2024-03-21T11:40:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.323+0000 7f2e95200640 1 -- start start 2024-03-21T11:40:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.323+0000 7f2e95200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e901016c0 0x7f2e90199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.323+0000 7f2e95200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e901028b0 0x7f2e9019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.323+0000 7f2e95200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e90103270 0x7f2e901a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.323+0000 7f2e95200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9010d360 con 0x7f2e901028b0 2024-03-21T11:40:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.323+0000 7f2e95200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9010d1e0 con 0x7f2e901016c0 2024-03-21T11:40:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.323+0000 7f2e95200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9010d4e0 con 0x7f2e90103270 2024-03-21T11:40:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.323+0000 7f2e87e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e901028b0 0x7f2e9019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.323+0000 7f2e8de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e901016c0 0x7f2e90199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.323+0000 7f2e87e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e901028b0 0x7f2e9019a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59872/0 (socket says 172.21.15.47:59872) 2024-03-21T11:40:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.323+0000 7f2e87e00640 1 -- 172.21.15.47:0/1515942697 learned_addr learned my addr 172.21.15.47:0/1515942697 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.323+0000 7f2e8e800640 1 --2- 172.21.15.47:0/1515942697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e90103270 0x7f2e901a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.324+0000 7f2e87e00640 1 -- 172.21.15.47:0/1515942697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e90103270 msgr2=0x7f2e901a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.324+0000 7f2e87e00640 1 --2- 172.21.15.47:0/1515942697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e90103270 0x7f2e901a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.324+0000 7f2e87e00640 1 -- 172.21.15.47:0/1515942697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e901016c0 msgr2=0x7f2e90199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.324+0000 7f2e87e00640 1 --2- 172.21.15.47:0/1515942697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e901016c0 0x7f2e90199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.324+0000 7f2e87e00640 1 -- 172.21.15.47:0/1515942697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e901a1830 con 0x7f2e901028b0 2024-03-21T11:40:20.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.324+0000 7f2e87e00640 1 --2- 172.21.15.47:0/1515942697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e901028b0 0x7f2e9019a1b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800e9f0 tx=0x7f2e7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.324+0000 7f2e86a00640 1 -- 172.21.15.47:0/1515942697 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7800ccc0 con 0x7f2e901028b0 2024-03-21T11:40:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.325+0000 7f2e95200640 1 -- 172.21.15.47:0/1515942697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e901a1b20 con 0x7f2e901028b0 2024-03-21T11:40:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.325+0000 7f2e86a00640 1 -- 172.21.15.47:0/1515942697 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e78010910 con 0x7f2e901028b0 2024-03-21T11:40:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.325+0000 7f2e86a00640 1 -- 172.21.15.47:0/1515942697 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e780188a0 con 0x7f2e901028b0 2024-03-21T11:40:20.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.325+0000 7f2e95200640 1 -- 172.21.15.47:0/1515942697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e901a2090 con 0x7f2e901028b0 2024-03-21T11:40:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.329+0000 7f2e97e0a740 1 Processor -- start 2024-03-21T11:40:20.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.329+0000 7f2e86a00640 1 -- 172.21.15.47:0/1515942697 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2e78010ac0 con 0x7f2e901028b0 2024-03-21T11:40:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.331+0000 7f2e97e0a740 1 -- start start 2024-03-21T11:40:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.331+0000 7f2e86a00640 1 --2- 172.21.15.47:0/1515942697 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2e740a2fa0 0x7f2e740a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.331+0000 7f2e97e0a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657c9020 0x5578657c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.331+0000 7f2e97e0a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657ca1f0 0x5578657ca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.331+0000 7f2e86a00640 1 -- 172.21.15.47:0/1515942697 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2e780cfbe0 con 0x7f2e901028b0 2024-03-21T11:40:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.331+0000 7f2e97e0a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657d1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.331+0000 7f2e97e0a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578657252c0 con 0x5578657cabb0 2024-03-21T11:40:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.331+0000 7f2e97e0a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578656cdc90 con 0x5578657c9020 2024-03-21T11:40:20.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.331+0000 7f2e97e0a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557865721360 con 0x5578657ca1f0 2024-03-21T11:40:20.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.331+0000 7f2e87e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657ca1f0 0x5578657ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.331+0000 7f2e8de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657c9020 0x5578657c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.331+0000 7f2e87e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657ca1f0 0x5578657ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46870/0 (socket says 172.21.15.47:46870) 2024-03-21T11:40:20.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.331+0000 7f2e87e00640 1 -- 172.21.15.47:0/3905726803 learned_addr learned my addr 172.21.15.47:0/3905726803 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.332+0000 7f2e8e800640 1 --2- 172.21.15.47:0/1515942697 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2e740a2fa0 0x7f2e740a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.332+0000 7f2e8e800640 1 --2- 172.21.15.47:0/3905726803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657d1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.332+0000 7f2e87e00640 1 -- 172.21.15.47:0/3905726803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657c9020 msgr2=0x5578657c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.332+0000 7f2e87e00640 1 --2- 172.21.15.47:0/3905726803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657c9020 0x5578657c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.332+0000 7f2e87e00640 1 -- 172.21.15.47:0/3905726803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 msgr2=0x5578657d1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.332+0000 7f2e87e00640 1 --2- 172.21.15.47:0/3905726803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657d1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.332+0000 7f2e87e00640 1 -- 172.21.15.47:0/3905726803 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578656f2080 con 0x5578657ca1f0 2024-03-21T11:40:20.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.332+0000 7f2e8de00640 1 --2- 172.21.15.47:0/3905726803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657c9020 0x5578657c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:20.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.333+0000 7f2e87e00640 1 --2- 172.21.15.47:0/3905726803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657ca1f0 0x5578657ca670 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f2e78098330 tx=0x7f2e7800ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a3ba7d65c230d21 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.333+0000 7f2e8e800640 1 --2- 172.21.15.47:0/1515942697 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2e740a2fa0 0x7f2e740a5460 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2e88005fd0 tx=0x7f2e88009c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.333+0000 7f2e7f400640 1 -- 172.21.15.47:0/3905726803 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e780a4970 con 0x5578657ca1f0 2024-03-21T11:40:20.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.333+0000 7f2e7f400640 1 -- 172.21.15.47:0/3905726803 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e780bc070 con 0x5578657ca1f0 2024-03-21T11:40:20.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.334+0000 7f2e97e0a740 1 -- 172.21.15.47:0/3905726803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657ca1f0 msgr2=0x5578657ca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.334+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/3905726803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657ca1f0 0x5578657ca670 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f2e78098330 tx=0x7f2e7800ce60 comp rx=0 tx=0).stop 2024-03-21T11:40:20.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.334+0000 7f2e97e0a740 1 -- 172.21.15.47:0/3905726803 shutdown_connections 2024-03-21T11:40:20.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.334+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/3905726803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657d1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.334+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/3905726803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657ca1f0 0x5578657ca670 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.334+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/3905726803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657c9020 0x5578657c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.334+0000 7f2e97e0a740 1 -- 172.21.15.47:0/3905726803 >> 172.21.15.47:0/3905726803 conn(0x5578657c4410 msgr2=0x5578657c4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.334+0000 7f2e97e0a740 1 -- 172.21.15.47:0/3905726803 shutdown_connections 2024-03-21T11:40:20.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.334+0000 7f2e97e0a740 1 -- 172.21.15.47:0/3905726803 wait complete. 2024-03-21T11:40:20.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.335+0000 7f2e97e0a740 1 -- start start 2024-03-21T11:40:20.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.335+0000 7f2e97e0a740 1 Processor -- start 2024-03-21T11:40:20.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.335+0000 7f2e97e0a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657c9020 0x5578657d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.335+0000 7f2e97e0a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657ca1f0 0x5578657d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.336+0000 7f2e97e0a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657da790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.336+0000 7f2e97e0a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578657d1bb0 con 0x5578657cabb0 2024-03-21T11:40:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.336+0000 7f2e97e0a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578656e10d0 con 0x5578657c9020 2024-03-21T11:40:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.336+0000 7f2e97e0a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578657d1d30 con 0x5578657ca1f0 2024-03-21T11:40:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.336+0000 7f2e97e0a740 10 client.0 _ll_register_callbacks cb 0x5578656e5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.336+0000 7f2e8de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657c9020 0x5578657d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.336+0000 7f2e8de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657c9020 0x5578657d3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55150/0 (socket says 172.21.15.47:55150) 2024-03-21T11:40:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.336+0000 7f2e8de00640 1 -- 172.21.15.47:0/2785086392 learned_addr learned my addr 172.21.15.47:0/2785086392 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.336+0000 7f2e8de00640 1 -- 172.21.15.47:0/2785086392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657ca1f0 msgr2=0x5578657d3750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:40:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.336+0000 7f2e8de00640 1 --2- 172.21.15.47:0/2785086392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657ca1f0 0x5578657d3750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.336+0000 7f2e8de00640 1 -- 172.21.15.47:0/2785086392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 msgr2=0x5578657da790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:40:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.337+0000 7f2e8e800640 1 --2- 172.21.15.47:0/2785086392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657da790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.337+0000 7f2e8de00640 1 --2- 172.21.15.47:0/2785086392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657da790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.337+0000 7f2e8de00640 1 -- 172.21.15.47:0/2785086392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578654393f0 con 0x5578657c9020 2024-03-21T11:40:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.337+0000 7f2e8e800640 1 --2- 172.21.15.47:0/2785086392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657da790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.337+0000 7f2e8de00640 1 --2- 172.21.15.47:0/2785086392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657c9020 0x5578657d3210 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2e8002c250 tx=0x7f2e80004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=536dc824cb2825df server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.337+0000 7f2e73e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.338+0000 7f2e73e00640 1 -- 172.21.15.47:0/2785086392 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80005d50 con 0x5578657c9020 2024-03-21T11:40:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.338+0000 7f2e97e0a740 1 -- 172.21.15.47:0/2785086392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e780bc070 con 0x5578657c9020 2024-03-21T11:40:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.338+0000 7f2e73e00640 1 -- 172.21.15.47:0/2785086392 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2e80005050 con 0x5578657c9020 2024-03-21T11:40:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.338+0000 7f2e73e00640 1 -- 172.21.15.47:0/2785086392 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e80005250 con 0x5578657c9020 2024-03-21T11:40:20.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.338+0000 7f2e97e0a740 10 client.31055 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:20.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.338+0000 7f2e97e0a740 1 -- 172.21.15.47:0/2785086392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5578655037d0 con 0x5578657c9020 2024-03-21T11:40:20.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.338+0000 7f2e73e00640 1 -- 172.21.15.47:0/2785086392 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e780bc070 con 0x5578657c9020 2024-03-21T11:40:20.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.339+0000 7f2e73e00640 1 -- 172.21.15.47:0/2785086392 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2e800035f0 con 0x5578657c9020 2024-03-21T11:40:20.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.339+0000 7f2e97e0a740 10 client.31055 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:20.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.339+0000 7f2e97e0a740 10 client.31055 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:40:20.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.339+0000 7f2e97e0a740 20 client.31055 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:20.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.339+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/2785086392 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5578657ded50 0x5578657e11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.339+0000 7f2e97e0a740 4 client.31055 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:20.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.339+0000 7f2e97e0a740 1 -- 172.21.15.47:0/2785086392 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557865732800 con 0x5578657ded50 2024-03-21T11:40:20.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.339+0000 7f2e87e00640 1 --2- 172.21.15.47:0/2785086392 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5578657ded50 0x5578657e11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.340+0000 7f2e87e00640 1 --2- 172.21.15.47:0/2785086392 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5578657ded50 0x5578657e11f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=29d2372ca900668f server_cookie=5e4ea922034ab129 in_seq=0 out_seq=0 2024-03-21T11:40:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.340+0000 7f2e73e00640 10 client.31055 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:40:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.341+0000 7f2e73e00640 1 -- 172.21.15.47:0/2785086392 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557865732800 con 0x5578657ded50 2024-03-21T11:40:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.341+0000 7f2e73e00640 10 client.31055 handle_command_reply: tid=0 2024-03-21T11:40:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.341+0000 7f2e97e0a740 1 client.31055 shutdown 2024-03-21T11:40:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.342+0000 7f2e97e0a740 1 -- 172.21.15.47:0/2785086392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657c9020 msgr2=0x5578657d3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.342+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/2785086392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657c9020 0x5578657d3210 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2e8002c250 tx=0x7f2e80004250 comp rx=0 tx=0).stop 2024-03-21T11:40:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.342+0000 7f2e97e0a740 1 -- 172.21.15.47:0/2785086392 shutdown_connections 2024-03-21T11:40:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.342+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/2785086392 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5578657ded50 0x5578657e11f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.342+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/2785086392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657da790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.342+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/2785086392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657ca1f0 0x5578657d3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.342+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/2785086392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657c9020 0x5578657d3210 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.342+0000 7f2e97e0a740 1 -- 172.21.15.47:0/2785086392 >> 172.21.15.47:0/2785086392 conn(0x5578657c4410 msgr2=0x5578657baeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.342+0000 7f2e73e00640 0 client.31055 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:40:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.342+0000 7f2e97e0a740 1 -- 172.21.15.47:0/2785086392 shutdown_connections 2024-03-21T11:40:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.342+0000 7f2e97e0a740 1 -- 172.21.15.47:0/2785086392 wait complete. 2024-03-21T11:40:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.342+0000 7f2e97e0a740 20 client.31055 trim_cache size 0 max 16384 2024-03-21T11:40:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.360+0000 7f2e97e0a740 1 Processor -- start 2024-03-21T11:40:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.360+0000 7f2e97e0a740 1 -- start start 2024-03-21T11:40:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.360+0000 7f2e97e0a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657c9020 0x5578657c8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.360+0000 7f2e97e0a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657ca1f0 0x5578657da750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.360+0000 7f2e97e0a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657cabb0 0x5578657e5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.360+0000 7f2e97e0a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e901a7f40 con 0x5578657ca1f0 2024-03-21T11:40:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.360+0000 7f2e97e0a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e901a98c0 con 0x5578657cabb0 2024-03-21T11:40:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.360+0000 7f2e97e0a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e901a85a0 con 0x5578657c9020 2024-03-21T11:40:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.360+0000 7f2e87e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657ca1f0 0x5578657da750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.360+0000 7f2e8e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657cabb0 0x5578657e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.360+0000 7f2e87e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657ca1f0 0x5578657da750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59900/0 (socket says 172.21.15.47:59900) 2024-03-21T11:40:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.360+0000 7f2e87e00640 1 -- 172.21.15.47:0/3053215125 learned_addr learned my addr 172.21.15.47:0/3053215125 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.361+0000 7f2e87e00640 1 -- 172.21.15.47:0/3053215125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657c9020 msgr2=0x5578657c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.361+0000 7f2e87e00640 1 --2- 172.21.15.47:0/3053215125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657c9020 0x5578657c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.361+0000 7f2e87e00640 1 -- 172.21.15.47:0/3053215125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657cabb0 msgr2=0x5578657e5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.361+0000 7f2e87e00640 1 --2- 172.21.15.47:0/3053215125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657cabb0 0x5578657e5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.361+0000 7f2e87e00640 1 -- 172.21.15.47:0/3053215125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578657b7510 con 0x5578657ca1f0 2024-03-21T11:40:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.361+0000 7f2e8e800640 1 --2- 172.21.15.47:0/3053215125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657cabb0 0x5578657e5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:20.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.361+0000 7f2e87e00640 1 --2- 172.21.15.47:0/3053215125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657ca1f0 0x5578657da750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2e7809bef0 tx=0x7f2e780a0230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239aeb1111124dbc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.362+0000 7f2e7ea00640 1 -- 172.21.15.47:0/3053215125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e780b5070 con 0x5578657ca1f0 2024-03-21T11:40:20.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.362+0000 7f2e7ea00640 1 -- 172.21.15.47:0/3053215125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e7809b420 con 0x5578657ca1f0 2024-03-21T11:40:20.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.362+0000 7f2e7ea00640 1 -- 172.21.15.47:0/3053215125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7809ae20 con 0x5578657ca1f0 2024-03-21T11:40:20.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.364+0000 7f2e97e0a740 1 -- 172.21.15.47:0/3053215125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657ca1f0 msgr2=0x5578657da750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.364+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/3053215125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657ca1f0 0x5578657da750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2e7809bef0 tx=0x7f2e780a0230 comp rx=0 tx=0).stop 2024-03-21T11:40:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.364+0000 7f2e97e0a740 1 -- 172.21.15.47:0/3053215125 shutdown_connections 2024-03-21T11:40:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.364+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/3053215125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657cabb0 0x5578657e5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.364+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/3053215125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657ca1f0 0x5578657da750 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.364+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/3053215125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657c9020 0x5578657c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.364+0000 7f2e97e0a740 1 -- 172.21.15.47:0/3053215125 >> 172.21.15.47:0/3053215125 conn(0x5578657c4410 msgr2=0x5578657baeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.364+0000 7f2e97e0a740 1 -- 172.21.15.47:0/3053215125 shutdown_connections 2024-03-21T11:40:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.364+0000 7f2e97e0a740 1 -- 172.21.15.47:0/3053215125 wait complete. 2024-03-21T11:40:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.365+0000 7f2e97e0a740 1 -- start start 2024-03-21T11:40:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.366+0000 7f2e97e0a740 1 Processor -- start 2024-03-21T11:40:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.366+0000 7f2e97e0a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657c9020 0x5578657d8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.366+0000 7f2e97e0a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657ca1f0 0x5578657d8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.366+0000 7f2e97e0a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657d9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.366+0000 7f2e97e0a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e901a9260 con 0x5578657cabb0 2024-03-21T11:40:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.366+0000 7f2e97e0a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e901a8c00 con 0x5578657ca1f0 2024-03-21T11:40:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.366+0000 7f2e97e0a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e901a6120 con 0x5578657c9020 2024-03-21T11:40:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.366+0000 7f2e97e0a740 10 client.0 _ll_register_callbacks cb 0x7f2e80009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.367+0000 7f2e8e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657d9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.367+0000 7f2e8e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657d9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59908/0 (socket says 172.21.15.47:59908) 2024-03-21T11:40:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.367+0000 7f2e8e800640 1 -- 172.21.15.47:0/1980610903 learned_addr learned my addr 172.21.15.47:0/1980610903 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.368+0000 7f2e8e800640 1 -- 172.21.15.47:0/1980610903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657c9020 msgr2=0x5578657d8310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:40:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.368+0000 7f2e8e800640 1 --2- 172.21.15.47:0/1980610903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657c9020 0x5578657d8310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.368+0000 7f2e8e800640 1 -- 172.21.15.47:0/1980610903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657ca1f0 msgr2=0x5578657d8850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:40:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.368+0000 7f2e8e800640 1 --2- 172.21.15.47:0/1980610903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657ca1f0 0x5578657d8850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.368+0000 7f2e8e800640 1 -- 172.21.15.47:0/1980610903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578657d9e50 con 0x5578657cabb0 2024-03-21T11:40:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.369+0000 7f2e8e800640 1 --2- 172.21.15.47:0/1980610903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657d9860 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800fa40 tx=0x7f2e88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261aef51cc408c71 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.369+0000 7f2e73e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:40:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.370+0000 7f2e73e00640 1 -- 172.21.15.47:0/1980610903 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88052060 con 0x5578657cabb0 2024-03-21T11:40:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.370+0000 7f2e73e00640 1 -- 172.21.15.47:0/1980610903 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2e88021ca0 con 0x5578657cabb0 2024-03-21T11:40:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.370+0000 7f2e97e0a740 1 -- 172.21.15.47:0/1980610903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e7809b420 con 0x5578657cabb0 2024-03-21T11:40:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.371+0000 7f2e73e00640 1 -- 172.21.15.47:0/1980610903 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e88052060 con 0x5578657cabb0 2024-03-21T11:40:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.371+0000 7f2e97e0a740 10 client.20538 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.371+0000 7f2e97e0a740 1 -- 172.21.15.47:0/1980610903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55786583a780 con 0x5578657cabb0 2024-03-21T11:40:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.372+0000 7f2e73e00640 1 -- 172.21.15.47:0/1980610903 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2e7809b420 con 0x5578657cabb0 2024-03-21T11:40:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.372+0000 7f2e73e00640 1 -- 172.21.15.47:0/1980610903 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2e8801e420 con 0x5578657cabb0 2024-03-21T11:40:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.372+0000 7f2e97e0a740 10 client.20538 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:20.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.372+0000 7f2e97e0a740 10 client.20538 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:40:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.372+0000 7f2e97e0a740 20 client.20538 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.372+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/1980610903 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55786583b110 0x55786583d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.372+0000 7f2e97e0a740 4 client.20538 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:40:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.372+0000 7f2e97e0a740 1 -- 172.21.15.47:0/1980610903 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55786572cb50 con 0x55786583b110 2024-03-21T11:40:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.372+0000 7f2e8de00640 1 --2- 172.21.15.47:0/1980610903 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55786583b110 0x55786583d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.374+0000 7f2e8de00640 1 --2- 172.21.15.47:0/1980610903 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55786583b110 0x55786583d530 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6fbd3ec223d324f0 server_cookie=3a4fe22d3bd568c4 in_seq=0 out_seq=0 2024-03-21T11:40:20.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.374+0000 7f2e73e00640 10 client.20538 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:40:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.376+0000 7f2e73e00640 1 -- 172.21.15.47:0/1980610903 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55786572cb50 con 0x55786583b110 2024-03-21T11:40:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.376+0000 7f2e73e00640 10 client.20538 handle_command_reply: tid=0 2024-03-21T11:40:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.376+0000 7f2e97e0a740 1 client.20538 shutdown 2024-03-21T11:40:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.376+0000 7f2e97e0a740 1 -- 172.21.15.47:0/1980610903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 msgr2=0x5578657d9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.376+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/1980610903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657d9860 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800fa40 tx=0x7f2e88004a40 comp rx=0 tx=0).stop 2024-03-21T11:40:20.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.376+0000 7f2e97e0a740 1 -- 172.21.15.47:0/1980610903 shutdown_connections 2024-03-21T11:40:20.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.376+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/1980610903 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55786583b110 0x55786583d530 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.376+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/1980610903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5578657cabb0 0x5578657d9860 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.376+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/1980610903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578657ca1f0 0x5578657d8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.376+0000 7f2e97e0a740 1 --2- 172.21.15.47:0/1980610903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578657c9020 0x5578657d8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.376+0000 7f2e97e0a740 1 -- 172.21.15.47:0/1980610903 >> 172.21.15.47:0/1980610903 conn(0x5578657c4410 msgr2=0x5578657cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.376+0000 7f2e73e00640 0 client.20538 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:40:20.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.379+0000 7f2e97e0a740 1 -- 172.21.15.47:0/1980610903 shutdown_connections 2024-03-21T11:40:20.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.379+0000 7f2e97e0a740 1 -- 172.21.15.47:0/1980610903 wait complete. 2024-03-21T11:40:20.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.379+0000 7f2e97e0a740 20 client.20538 trim_cache size 0 max 16384 2024-03-21T11:40:20.380 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:40:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.380+0000 7f2e95200640 1 -- 172.21.15.47:0/1515942697 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2e740a2fa0 msgr2=0x7f2e740a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.380+0000 7f2e95200640 1 --2- 172.21.15.47:0/1515942697 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2e740a2fa0 0x7f2e740a5460 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2e88005fd0 tx=0x7f2e88009c10 comp rx=0 tx=0).stop 2024-03-21T11:40:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.380+0000 7f2e95200640 1 -- 172.21.15.47:0/1515942697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e901028b0 msgr2=0x7f2e9019a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.380+0000 7f2e95200640 1 --2- 172.21.15.47:0/1515942697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e901028b0 0x7f2e9019a1b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800e9f0 tx=0x7f2e7800edf0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.380+0000 7f2e95200640 1 -- 172.21.15.47:0/1515942697 shutdown_connections 2024-03-21T11:40:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.380+0000 7f2e95200640 1 --2- 172.21.15.47:0/1515942697 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2e740a2fa0 0x7f2e740a5460 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.380+0000 7f2e95200640 1 --2- 172.21.15.47:0/1515942697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e90103270 0x7f2e901a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.380+0000 7f2e95200640 1 --2- 172.21.15.47:0/1515942697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e901028b0 0x7f2e9019a1b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.380+0000 7f2e95200640 1 --2- 172.21.15.47:0/1515942697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e901016c0 0x7f2e90199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.380+0000 7f2e95200640 1 -- 172.21.15.47:0/1515942697 >> 172.21.15.47:0/1515942697 conn(0x7f2e900fcc30 msgr2=0x7f2e900ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.380+0000 7f2e95200640 1 -- 172.21.15.47:0/1515942697 shutdown_connections 2024-03-21T11:40:20.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.380+0000 7f2e95200640 1 -- 172.21.15.47:0/1515942697 wait complete. 2024-03-21T11:40:20.407 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:40:20.408 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:40:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.668+0000 7f1ef6a00640 1 -- 172.21.15.47:0/4076180632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef01028a0 msgr2=0x7f1ef0102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.668+0000 7f1ef6a00640 1 --2- 172.21.15.47:0/4076180632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef01028a0 0x7f1ef0102d20 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009900 tx=0x7f1edc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:40:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.668+0000 7f1ef6a00640 1 -- 172.21.15.47:0/4076180632 shutdown_connections 2024-03-21T11:40:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.668+0000 7f1ef6a00640 1 --2- 172.21.15.47:0/4076180632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef0103260 0x7f1ef0109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.668+0000 7f1ef6a00640 1 --2- 172.21.15.47:0/4076180632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef01028a0 0x7f1ef0102d20 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.668+0000 7f1ef6a00640 1 --2- 172.21.15.47:0/4076180632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ef01016b0 0x7f1ef0101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.668+0000 7f1ef6a00640 1 -- 172.21.15.47:0/4076180632 >> 172.21.15.47:0/4076180632 conn(0x7f1ef00fcc20 msgr2=0x7f1ef00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.668+0000 7f1ef6a00640 1 -- 172.21.15.47:0/4076180632 shutdown_connections 2024-03-21T11:40:20.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.668+0000 7f1ef6a00640 1 -- 172.21.15.47:0/4076180632 wait complete. 2024-03-21T11:40:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.669+0000 7f1ef6a00640 1 Processor -- start 2024-03-21T11:40:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.669+0000 7f1ef6a00640 1 -- start start 2024-03-21T11:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.669+0000 7f1ef6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ef01016b0 0x7f1ef0075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.669+0000 7f1ef6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef01028a0 0x7f1ef0073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.669+0000 7f1ef6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef0103260 0x7f1ef0073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.669+0000 7f1ef6a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef010d330 con 0x7f1ef01028a0 2024-03-21T11:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.669+0000 7f1ef6a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef010d1b0 con 0x7f1ef0103260 2024-03-21T11:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.669+0000 7f1ef6a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef010d4b0 con 0x7f1ef01016b0 2024-03-21T11:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.669+0000 7f1eefe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ef01016b0 0x7f1ef0075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.669+0000 7f1ef4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef0103260 0x7f1ef0073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.669+0000 7f1eef400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef01028a0 0x7f1ef0073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.669+0000 7f1eefe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ef01016b0 0x7f1ef0075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46890/0 (socket says 172.21.15.47:46890) 2024-03-21T11:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.669+0000 7f1eefe00640 1 -- 172.21.15.47:0/3699409074 learned_addr learned my addr 172.21.15.47:0/3699409074 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.670+0000 7f1eef400640 1 -- 172.21.15.47:0/3699409074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ef01016b0 msgr2=0x7f1ef0075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.670+0000 7f1eef400640 1 --2- 172.21.15.47:0/3699409074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ef01016b0 0x7f1ef0075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.670+0000 7f1eef400640 1 -- 172.21.15.47:0/3699409074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef0103260 msgr2=0x7f1ef0073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.670+0000 7f1eef400640 1 --2- 172.21.15.47:0/3699409074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef0103260 0x7f1ef0073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.670+0000 7f1eef400640 1 -- 172.21.15.47:0/3699409074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef0074860 con 0x7f1ef01028a0 2024-03-21T11:40:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.670+0000 7f1eefe00640 1 --2- 172.21.15.47:0/3699409074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ef01016b0 0x7f1ef0075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.670+0000 7f1ef4a00640 1 --2- 172.21.15.47:0/3699409074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef0103260 0x7f1ef0073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.670+0000 7f1eef400640 1 --2- 172.21.15.47:0/3699409074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef01028a0 0x7f1ef0073a90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009230 tx=0x7f1edc02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.670+0000 7f1eecc00640 1 -- 172.21.15.47:0/3699409074 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc041070 con 0x7f1ef01028a0 2024-03-21T11:40:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.670+0000 7f1ef6a00640 1 -- 172.21.15.47:0/3699409074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef01a1ac0 con 0x7f1ef01028a0 2024-03-21T11:40:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.671+0000 7f1eecc00640 1 -- 172.21.15.47:0/3699409074 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1edc004450 con 0x7f1ef01028a0 2024-03-21T11:40:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.671+0000 7f1eecc00640 1 -- 172.21.15.47:0/3699409074 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc003df0 con 0x7f1ef01028a0 2024-03-21T11:40:20.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.671+0000 7f1ef6a00640 1 -- 172.21.15.47:0/3699409074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef01a2060 con 0x7f1ef01028a0 2024-03-21T11:40:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.674+0000 7f1eecc00640 1 -- 172.21.15.47:0/3699409074 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1edc044070 con 0x7f1ef01028a0 2024-03-21T11:40:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.676+0000 7f1eecc00640 1 --2- 172.21.15.47:0/3699409074 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1ed40a2fa0 0x7f1ed40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.676+0000 7f1eecc00640 1 -- 172.21.15.47:0/3699409074 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1edc0f1160 con 0x7f1ef01028a0 2024-03-21T11:40:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.676+0000 7f1eefe00640 1 --2- 172.21.15.47:0/3699409074 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1ed40a2fa0 0x7f1ed40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.677+0000 7f1eefe00640 1 --2- 172.21.15.47:0/3699409074 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1ed40a2fa0 0x7f1ed40a5460 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00058c0 tx=0x7f1ee0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.678+0000 7f1ef983a740 1 Processor -- start 2024-03-21T11:40:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.679+0000 7f1ef983a740 1 -- start start 2024-03-21T11:40:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.679+0000 7f1ef983a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 0x55d934d43c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.679+0000 7f1ef983a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d449e0 0x55d934d44e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.679+0000 7f1ef983a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d453a0 0x55d934d4bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.679+0000 7f1ef983a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d934ca82c0 con 0x55d934d449e0 2024-03-21T11:40:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.679+0000 7f1ef983a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d934c50c90 con 0x55d934d43810 2024-03-21T11:40:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.679+0000 7f1ef983a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d934ca4360 con 0x55d934d453a0 2024-03-21T11:40:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.679+0000 7f1ef4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d453a0 0x55d934d4bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.680+0000 7f1ef4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d453a0 0x55d934d4bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46898/0 (socket says 172.21.15.47:46898) 2024-03-21T11:40:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.680+0000 7f1ef4a00640 1 -- 172.21.15.47:0/2342213555 learned_addr learned my addr 172.21.15.47:0/2342213555 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.680+0000 7f1eefe00640 1 --2- 172.21.15.47:0/2342213555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 0x55d934d43c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.680+0000 7f1eef400640 1 --2- 172.21.15.47:0/2342213555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d449e0 0x55d934d44e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.680+0000 7f1ef4a00640 1 -- 172.21.15.47:0/2342213555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 msgr2=0x55d934d43c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.680+0000 7f1ef4a00640 1 --2- 172.21.15.47:0/2342213555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 0x55d934d43c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.680+0000 7f1ef4a00640 1 -- 172.21.15.47:0/2342213555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d449e0 msgr2=0x55d934d44e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.680+0000 7f1ef4a00640 1 --2- 172.21.15.47:0/2342213555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d449e0 0x55d934d44e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.680+0000 7f1ef4a00640 1 -- 172.21.15.47:0/2342213555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d934c75080 con 0x55d934d453a0 2024-03-21T11:40:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.682+0000 7f1ef4a00640 1 --2- 172.21.15.47:0/2342213555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d453a0 0x55d934d4bc30 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80117e0 tx=0x7f1ee8011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=544b0cfdc559187c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.683+0000 7f1ee5200640 1 -- 172.21.15.47:0/2342213555 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8011310 con 0x55d934d453a0 2024-03-21T11:40:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.683+0000 7f1ee5200640 1 -- 172.21.15.47:0/2342213555 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ee8003e50 con 0x55d934d453a0 2024-03-21T11:40:20.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.683+0000 7f1ee5200640 1 -- 172.21.15.47:0/2342213555 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8004620 con 0x55d934d453a0 2024-03-21T11:40:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.684+0000 7f1ef983a740 1 -- 172.21.15.47:0/2342213555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d453a0 msgr2=0x55d934d4bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.684+0000 7f1ef983a740 1 --2- 172.21.15.47:0/2342213555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d453a0 0x55d934d4bc30 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80117e0 tx=0x7f1ee8011ca0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.685+0000 7f1ef983a740 1 -- 172.21.15.47:0/2342213555 shutdown_connections 2024-03-21T11:40:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.685+0000 7f1ef983a740 1 --2- 172.21.15.47:0/2342213555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d453a0 0x55d934d4bc30 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.685+0000 7f1ef983a740 1 --2- 172.21.15.47:0/2342213555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d449e0 0x55d934d44e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.685+0000 7f1ef983a740 1 --2- 172.21.15.47:0/2342213555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 0x55d934d43c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.685+0000 7f1ef983a740 1 -- 172.21.15.47:0/2342213555 >> 172.21.15.47:0/2342213555 conn(0x55d934d3ec00 msgr2=0x55d934d3f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.686+0000 7f1ef983a740 1 -- 172.21.15.47:0/2342213555 shutdown_connections 2024-03-21T11:40:20.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.686+0000 7f1ef983a740 1 -- 172.21.15.47:0/2342213555 wait complete. 2024-03-21T11:40:20.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.687+0000 7f1ef983a740 1 -- start start 2024-03-21T11:40:20.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.687+0000 7f1ef983a740 1 Processor -- start 2024-03-21T11:40:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.688+0000 7f1ef983a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d43810 0x55d934d4db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.688+0000 7f1ef983a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d449e0 0x55d934d4e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.688+0000 7f1ef983a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d453a0 0x55d934d55090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.688+0000 7f1eef400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d449e0 0x55d934d4e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.688+0000 7f1eef400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d449e0 0x55d934d4e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46910/0 (socket says 172.21.15.47:46910) 2024-03-21T11:40:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.688+0000 7f1ef983a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d934d4c3b0 con 0x55d934d43810 2024-03-21T11:40:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.689+0000 7f1ef983a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d934c640d0 con 0x55d934d453a0 2024-03-21T11:40:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.689+0000 7f1ef983a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d934d4c530 con 0x55d934d449e0 2024-03-21T11:40:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.689+0000 7f1ef983a740 10 client.0 _ll_register_callbacks cb 0x55d934c68e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.689+0000 7f1eefe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d43810 0x55d934d4db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.689+0000 7f1eefe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d43810 0x55d934d4db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59944/0 (socket says 172.21.15.47:59944) 2024-03-21T11:40:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.689+0000 7f1eefe00640 1 -- 172.21.15.47:0/685682851 learned_addr learned my addr 172.21.15.47:0/685682851 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.690+0000 7f1eefe00640 1 -- 172.21.15.47:0/685682851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d449e0 msgr2=0x55d934d4e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.690+0000 7f1eefe00640 1 --2- 172.21.15.47:0/685682851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d449e0 0x55d934d4e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.690+0000 7f1eefe00640 1 -- 172.21.15.47:0/685682851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d453a0 msgr2=0x55d934d55090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.690+0000 7f1eefe00640 1 --2- 172.21.15.47:0/685682851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d453a0 0x55d934d55090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.690+0000 7f1eefe00640 1 -- 172.21.15.47:0/685682851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9349bc3f0 con 0x55d934d43810 2024-03-21T11:40:20.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.690+0000 7f1eef400640 1 --2- 172.21.15.47:0/685682851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d449e0 0x55d934d4e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-21T11:40:20.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.691+0000 7f1eefe00640 1 --2- 172.21.15.47:0/685682851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d43810 0x55d934d4db10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000fc50 tx=0x7f1ee0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d71f2e2d2648b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.691+0000 7f1ed9600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:40:20.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.691+0000 7f1ed9600640 1 -- 172.21.15.47:0/685682851 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee001cca0 con 0x55d934d43810 2024-03-21T11:40:20.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.691+0000 7f1ed9600640 1 -- 172.21.15.47:0/685682851 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1ee0005020 con 0x55d934d43810 2024-03-21T11:40:20.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.692+0000 7f1ef983a740 1 -- 172.21.15.47:0/685682851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ee8003e50 con 0x55d934d43810 2024-03-21T11:40:20.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.692+0000 7f1ed9600640 1 -- 172.21.15.47:0/685682851 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ee0032050 con 0x55d934d43810 2024-03-21T11:40:20.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.693+0000 7f1ef983a740 10 client.20547 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.693+0000 7f1ef983a740 1 -- 172.21.15.47:0/685682851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d934a867d0 con 0x55d934d43810 2024-03-21T11:40:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.693+0000 7f1ed9600640 1 -- 172.21.15.47:0/685682851 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ee8003e50 con 0x55d934d43810 2024-03-21T11:40:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.693+0000 7f1ed9600640 1 -- 172.21.15.47:0/685682851 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1ee0004ce0 con 0x55d934d43810 2024-03-21T11:40:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.694+0000 7f1ef983a740 10 client.20547 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.694+0000 7f1ef983a740 10 client.20547 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:40:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.694+0000 7f1ef983a740 20 client.20547 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.694+0000 7f1ef983a740 1 --2- 172.21.15.47:0/685682851 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d934d59a70 0x55d934d5bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.694+0000 7f1ef983a740 4 client.20547 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:20.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.694+0000 7f1ef983a740 1 -- 172.21.15.47:0/685682851 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d934cb5800 con 0x55d934d59a70 2024-03-21T11:40:20.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.696+0000 7f1eef400640 1 --2- 172.21.15.47:0/685682851 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d934d59a70 0x55d934d5bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.697+0000 7f1eef400640 1 --2- 172.21.15.47:0/685682851 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d934d59a70 0x55d934d5bf30 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=617def199fc2d61e server_cookie=19d88ae147acf9b in_seq=0 out_seq=0 2024-03-21T11:40:20.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.697+0000 7f1ed9600640 10 client.20547 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:40:20.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.702+0000 7f1ed9600640 1 -- 172.21.15.47:0/685682851 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d934cb5800 con 0x55d934d59a70 2024-03-21T11:40:20.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.702+0000 7f1ed9600640 10 client.20547 handle_command_reply: tid=0 2024-03-21T11:40:20.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.702+0000 7f1ef983a740 1 client.20547 shutdown 2024-03-21T11:40:20.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.702+0000 7f1ef983a740 1 -- 172.21.15.47:0/685682851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d43810 msgr2=0x55d934d4db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.703+0000 7f1ef983a740 1 --2- 172.21.15.47:0/685682851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d43810 0x55d934d4db10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000fc50 tx=0x7f1ee0013040 comp rx=0 tx=0).stop 2024-03-21T11:40:20.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.703+0000 7f1ef983a740 1 -- 172.21.15.47:0/685682851 shutdown_connections 2024-03-21T11:40:20.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.703+0000 7f1ef983a740 1 --2- 172.21.15.47:0/685682851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d453a0 0x55d934d55090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.703+0000 7f1ef983a740 1 --2- 172.21.15.47:0/685682851 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d934d59a70 0x55d934d5bf30 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.703+0000 7f1ef983a740 1 --2- 172.21.15.47:0/685682851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d449e0 0x55d934d4e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.703+0000 7f1ed9600640 0 client.20547 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:40:20.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.703+0000 7f1ef983a740 1 --2- 172.21.15.47:0/685682851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d43810 0x55d934d4db10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.703+0000 7f1ef983a740 1 -- 172.21.15.47:0/685682851 >> 172.21.15.47:0/685682851 conn(0x55d934d3ec00 msgr2=0x55d934d3deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.705+0000 7f1ef983a740 1 -- 172.21.15.47:0/685682851 shutdown_connections 2024-03-21T11:40:20.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.706+0000 7f1ef983a740 1 -- 172.21.15.47:0/685682851 wait complete. 2024-03-21T11:40:20.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.707+0000 7f1ef983a740 20 client.20547 trim_cache size 0 max 16384 2024-03-21T11:40:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.730+0000 7f1ef983a740 1 Processor -- start 2024-03-21T11:40:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.730+0000 7f1ef983a740 1 -- start start 2024-03-21T11:40:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.730+0000 7f1ef983a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 0x55d934d4dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.730+0000 7f1ef983a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d449e0 0x55d934d42570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.731+0000 7f1ef983a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d453a0 0x55d934d47a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.731+0000 7f1ef983a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef01ac450 con 0x55d934d453a0 2024-03-21T11:40:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.731+0000 7f1ef983a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef01addd0 con 0x55d934d43810 2024-03-21T11:40:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.731+0000 7f1ef983a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef01acab0 con 0x55d934d449e0 2024-03-21T11:40:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.731+0000 7f1eef400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d449e0 0x55d934d42570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.731+0000 7f1eef400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d449e0 0x55d934d42570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46916/0 (socket says 172.21.15.47:46916) 2024-03-21T11:40:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.731+0000 7f1eef400640 1 -- 172.21.15.47:0/42588969 learned_addr learned my addr 172.21.15.47:0/42588969 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.731+0000 7f1eefe00640 1 --2- 172.21.15.47:0/42588969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 0x55d934d4dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.732+0000 7f1ef4a00640 1 --2- 172.21.15.47:0/42588969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d453a0 0x55d934d47a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.732+0000 7f1ef4a00640 1 -- 172.21.15.47:0/42588969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d449e0 msgr2=0x55d934d42570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.732+0000 7f1ef4a00640 1 --2- 172.21.15.47:0/42588969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d449e0 0x55d934d42570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.732+0000 7f1ef4a00640 1 -- 172.21.15.47:0/42588969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 msgr2=0x55d934d4dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.732+0000 7f1ef4a00640 1 --2- 172.21.15.47:0/42588969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 0x55d934d4dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.732+0000 7f1ef4a00640 1 -- 172.21.15.47:0/42588969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d934d3f970 con 0x55d934d453a0 2024-03-21T11:40:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.733+0000 7f1ef4a00640 1 --2- 172.21.15.47:0/42588969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d453a0 0x55d934d47a80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55d934d3fb40 tx=0x7f1ee8011050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a429166afa5bf182 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.734+0000 7f1edbe00640 1 -- 172.21.15.47:0/42588969 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8011140 con 0x55d934d453a0 2024-03-21T11:40:20.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.734+0000 7f1edbe00640 1 -- 172.21.15.47:0/42588969 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ee8003e50 con 0x55d934d453a0 2024-03-21T11:40:20.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.734+0000 7f1edbe00640 1 -- 172.21.15.47:0/42588969 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80052c0 con 0x55d934d453a0 2024-03-21T11:40:20.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.735+0000 7f1ef983a740 1 -- 172.21.15.47:0/42588969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d453a0 msgr2=0x55d934d47a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.735+0000 7f1ef983a740 1 --2- 172.21.15.47:0/42588969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d453a0 0x55d934d47a80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55d934d3fb40 tx=0x7f1ee8011050 comp rx=0 tx=0).stop 2024-03-21T11:40:20.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.736+0000 7f1ef983a740 1 -- 172.21.15.47:0/42588969 shutdown_connections 2024-03-21T11:40:20.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.736+0000 7f1ef983a740 1 --2- 172.21.15.47:0/42588969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d453a0 0x55d934d47a80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.736+0000 7f1ef983a740 1 --2- 172.21.15.47:0/42588969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d449e0 0x55d934d42570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.736+0000 7f1ef983a740 1 --2- 172.21.15.47:0/42588969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 0x55d934d4dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.736+0000 7f1ef983a740 1 -- 172.21.15.47:0/42588969 >> 172.21.15.47:0/42588969 conn(0x55d934d3ec00 msgr2=0x55d934d41e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.736+0000 7f1ef983a740 1 -- 172.21.15.47:0/42588969 shutdown_connections 2024-03-21T11:40:20.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.737+0000 7f1ef983a740 1 -- 172.21.15.47:0/42588969 wait complete. 2024-03-21T11:40:20.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.737+0000 7f1ef983a740 1 -- start start 2024-03-21T11:40:20.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.738+0000 7f1ef983a740 1 Processor -- start 2024-03-21T11:40:20.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.738+0000 7f1ef983a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 0x55d934d5f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.739+0000 7f1ef983a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d449e0 0x55d934d5f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.739+0000 7f1ef983a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d453a0 0x55d934d60830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.739+0000 7f1ef983a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef01ad770 con 0x55d934d449e0 2024-03-21T11:40:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.739+0000 7f1ef983a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef01ad110 con 0x55d934d43810 2024-03-21T11:40:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.739+0000 7f1ef983a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef01aa610 con 0x55d934d453a0 2024-03-21T11:40:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.739+0000 7f1ef983a740 10 client.0 _ll_register_callbacks cb 0x7f1ee000f5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.739+0000 7f1ef4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d453a0 0x55d934d60830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.739+0000 7f1eefe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 0x55d934d5f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.739+0000 7f1eefe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 0x55d934d5f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55222/0 (socket says 172.21.15.47:55222) 2024-03-21T11:40:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.739+0000 7f1ef4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d453a0 0x55d934d60830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46930/0 (socket says 172.21.15.47:46930) 2024-03-21T11:40:20.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.739+0000 7f1eefe00640 1 -- 172.21.15.47:0/1761645475 learned_addr learned my addr 172.21.15.47:0/1761645475 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:20.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.740+0000 7f1eef400640 1 --2- 172.21.15.47:0/1761645475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d449e0 0x55d934d5f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.740+0000 7f1ef4a00640 1 -- 172.21.15.47:0/1761645475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 msgr2=0x55d934d5f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.740+0000 7f1ef4a00640 1 --2- 172.21.15.47:0/1761645475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 0x55d934d5f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.740+0000 7f1ef4a00640 1 -- 172.21.15.47:0/1761645475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d449e0 msgr2=0x55d934d5f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:20.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.740+0000 7f1ef4a00640 1 --2- 172.21.15.47:0/1761645475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d449e0 0x55d934d5f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.740+0000 7f1ef4a00640 1 -- 172.21.15.47:0/1761645475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d934d60e20 con 0x55d934d453a0 2024-03-21T11:40:20.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.741+0000 7f1eef400640 1 --2- 172.21.15.47:0/1761645475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d449e0 0x55d934d5f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:20.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.741+0000 7f1ef4a00640 1 --2- 172.21.15.47:0/1761645475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d453a0 0x55d934d60830 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x55d934d4e7e0 tx=0x7f1ee800fa10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bbfd41f8f01ca7f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:20.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.741+0000 7f1ed9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:40:20.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.742+0000 7f1ed9600640 1 -- 172.21.15.47:0/1761645475 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8011350 con 0x55d934d453a0 2024-03-21T11:40:20.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.742+0000 7f1ed9600640 1 -- 172.21.15.47:0/1761645475 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1ee8005240 con 0x55d934d453a0 2024-03-21T11:40:20.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.743+0000 7f1ef983a740 1 -- 172.21.15.47:0/1761645475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ee8003e50 con 0x55d934d453a0 2024-03-21T11:40:20.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.743+0000 7f1ed9600640 1 -- 172.21.15.47:0/1761645475 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ee8035050 con 0x55d934d453a0 2024-03-21T11:40:20.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.743+0000 7f1ef983a740 10 client.31057 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:20.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.743+0000 7f1ef983a740 1 -- 172.21.15.47:0/1761645475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d934db5860 con 0x55d934d453a0 2024-03-21T11:40:20.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.744+0000 7f1ed9600640 1 -- 172.21.15.47:0/1761645475 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ee8003e50 con 0x55d934d453a0 2024-03-21T11:40:20.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.744+0000 7f1ed9600640 1 -- 172.21.15.47:0/1761645475 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1ee8018020 con 0x55d934d453a0 2024-03-21T11:40:20.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.745+0000 7f1ef983a740 10 client.31057 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:20.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.745+0000 7f1ef983a740 10 client.31057 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:40:20.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.745+0000 7f1ef983a740 20 client.31057 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:20.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.746+0000 7f1ef983a740 1 --2- 172.21.15.47:0/1761645475 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d934d59a70 0x55d934db5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:20.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.746+0000 7f1ef983a740 4 client.31057 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:40:20.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.746+0000 7f1ef983a740 1 -- 172.21.15.47:0/1761645475 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d934cafb50 con 0x55d934d59a70 2024-03-21T11:40:20.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.748+0000 7f1eefe00640 1 --2- 172.21.15.47:0/1761645475 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d934d59a70 0x55d934db5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:20.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.749+0000 7f1ed9600640 10 client.31057 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:40:20.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.749+0000 7f1eefe00640 1 --2- 172.21.15.47:0/1761645475 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d934d59a70 0x55d934db5f80 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8fd1a3efd9a9e27 server_cookie=9c5411d0356d43ab in_seq=0 out_seq=0 2024-03-21T11:40:20.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.749+0000 7f1ed9600640 1 -- 172.21.15.47:0/1761645475 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d934cafb50 con 0x55d934d59a70 2024-03-21T11:40:20.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.749+0000 7f1ed9600640 10 client.31057 handle_command_reply: tid=0 2024-03-21T11:40:20.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.749+0000 7f1ef983a740 1 client.31057 shutdown 2024-03-21T11:40:20.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.750+0000 7f1ef983a740 1 -- 172.21.15.47:0/1761645475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d453a0 msgr2=0x55d934d60830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.750+0000 7f1ef983a740 1 --2- 172.21.15.47:0/1761645475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d453a0 0x55d934d60830 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x55d934d4e7e0 tx=0x7f1ee800fa10 comp rx=0 tx=0).stop 2024-03-21T11:40:20.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.750+0000 7f1ef983a740 1 -- 172.21.15.47:0/1761645475 shutdown_connections 2024-03-21T11:40:20.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.750+0000 7f1ef983a740 1 --2- 172.21.15.47:0/1761645475 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55d934d59a70 0x55d934db5f80 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.750+0000 7f1ed9600640 0 client.31057 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:40:20.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.750+0000 7f1ef983a740 1 --2- 172.21.15.47:0/1761645475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d934d453a0 0x55d934d60830 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.750+0000 7f1ef983a740 1 --2- 172.21.15.47:0/1761645475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d934d449e0 0x55d934d5f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.750+0000 7f1ef983a740 1 --2- 172.21.15.47:0/1761645475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d934d43810 0x55d934d5f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.750+0000 7f1ef983a740 1 -- 172.21.15.47:0/1761645475 >> 172.21.15.47:0/1761645475 conn(0x55d934d3ec00 msgr2=0x55d934d4dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.750+0000 7f1ef983a740 1 -- 172.21.15.47:0/1761645475 shutdown_connections 2024-03-21T11:40:20.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.751+0000 7f1ef983a740 1 -- 172.21.15.47:0/1761645475 wait complete. 2024-03-21T11:40:20.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.751+0000 7f1ef983a740 20 client.31057 trim_cache size 0 max 16384 2024-03-21T11:40:20.752 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:40:20.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.752+0000 7f1ef6a00640 1 -- 172.21.15.47:0/3699409074 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1ed40a2fa0 msgr2=0x7f1ed40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.752+0000 7f1ef6a00640 1 --2- 172.21.15.47:0/3699409074 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1ed40a2fa0 0x7f1ed40a5460 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00058c0 tx=0x7f1ee0005f90 comp rx=0 tx=0).stop 2024-03-21T11:40:20.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.752+0000 7f1ef6a00640 1 -- 172.21.15.47:0/3699409074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef01028a0 msgr2=0x7f1ef0073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.752+0000 7f1ef6a00640 1 --2- 172.21.15.47:0/3699409074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef01028a0 0x7f1ef0073a90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009230 tx=0x7f1edc02c650 comp rx=0 tx=0).stop 2024-03-21T11:40:20.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.752+0000 7f1ef6a00640 1 -- 172.21.15.47:0/3699409074 shutdown_connections 2024-03-21T11:40:20.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.752+0000 7f1ef6a00640 1 --2- 172.21.15.47:0/3699409074 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1ed40a2fa0 0x7f1ed40a5460 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.752+0000 7f1ef6a00640 1 --2- 172.21.15.47:0/3699409074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ef0103260 0x7f1ef0073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.752+0000 7f1ef6a00640 1 --2- 172.21.15.47:0/3699409074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef01028a0 0x7f1ef0073a90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.752+0000 7f1ef6a00640 1 --2- 172.21.15.47:0/3699409074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ef01016b0 0x7f1ef0075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.752+0000 7f1ef6a00640 1 -- 172.21.15.47:0/3699409074 >> 172.21.15.47:0/3699409074 conn(0x7f1ef00fcc20 msgr2=0x7f1ef0105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.754+0000 7f1ef6a00640 1 -- 172.21.15.47:0/3699409074 shutdown_connections 2024-03-21T11:40:20.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.754+0000 7f1ef6a00640 1 -- 172.21.15.47:0/3699409074 wait complete. 2024-03-21T11:40:20.779 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:40:20.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:40:20.941 INFO:tasks.workunit.client.0.smithi047.stdout: 10 88310 25.24 MB/sec execute 64 sec latency 87.424 ms 2024-03-21T11:40:20.945 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:20 smithi100 ceph-mon[36245]: pgmap v910: 129 pgs: 129 active+clean; 419 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.4 MiB/s wr, 655 op/s 2024-03-21T11:40:20.945 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:20 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1781662927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:40:20.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:20 smithi102 ceph-mon[36748]: pgmap v910: 129 pgs: 129 active+clean; 419 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.4 MiB/s wr, 655 op/s 2024-03-21T11:40:20.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:20 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1781662927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:40:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.997+0000 7f6a66c00640 1 -- 172.21.15.47:0/4128720999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a601038e0 msgr2=0x7f6a60103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.997+0000 7f6a66c00640 1 --2- 172.21.15.47:0/4128720999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a601038e0 0x7f6a60103d00 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009900 tx=0x7f6a5002bc30 comp rx=0 tx=0).stop 2024-03-21T11:40:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.997+0000 7f6a66c00640 1 -- 172.21.15.47:0/4128720999 shutdown_connections 2024-03-21T11:40:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.997+0000 7f6a66c00640 1 --2- 172.21.15.47:0/4128720999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a60105490 0x7f6a6010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.997+0000 7f6a66c00640 1 --2- 172.21.15.47:0/4128720999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a60104ad0 0x7f6a60104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.998+0000 7f6a66c00640 1 --2- 172.21.15.47:0/4128720999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a601038e0 0x7f6a60103d00 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.998+0000 7f6a66c00640 1 -- 172.21.15.47:0/4128720999 >> 172.21.15.47:0/4128720999 conn(0x7f6a600fee50 msgr2=0x7f6a60101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.998+0000 7f6a66c00640 1 -- 172.21.15.47:0/4128720999 shutdown_connections 2024-03-21T11:40:20.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.998+0000 7f6a66c00640 1 -- 172.21.15.47:0/4128720999 wait complete. 2024-03-21T11:40:21.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.999+0000 7f6a66c00640 1 Processor -- start 2024-03-21T11:40:21.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.999+0000 7f6a66c00640 1 -- start start 2024-03-21T11:40:21.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.999+0000 7f6a66c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a601038e0 0x7f6a60199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.999+0000 7f6a66c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a60104ad0 0x7f6a6019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.999+0000 7f6a66c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a60105490 0x7f6a601a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.999+0000 7f6a66c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6010f5b0 con 0x7f6a601038e0 2024-03-21T11:40:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.999+0000 7f6a66c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6010f430 con 0x7f6a60105490 2024-03-21T11:40:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.999+0000 7f6a66c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6010f730 con 0x7f6a60104ad0 2024-03-21T11:40:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.999+0000 7f6a5fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a601038e0 0x7f6a60199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.999+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a60104ad0 0x7f6a6019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.999+0000 7f6a5fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a601038e0 0x7f6a60199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59976/0 (socket says 172.21.15.47:59976) 2024-03-21T11:40:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.999+0000 7f6a5fe00640 1 -- 172.21.15.47:0/3464125853 learned_addr learned my addr 172.21.15.47:0/3464125853 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:20.999+0000 7f6a64c00640 1 --2- 172.21.15.47:0/3464125853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a60105490 0x7f6a601a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.000+0000 7f6a5fe00640 1 -- 172.21.15.47:0/3464125853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a60104ad0 msgr2=0x7f6a6019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.000+0000 7f6a5fe00640 1 --2- 172.21.15.47:0/3464125853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a60104ad0 0x7f6a6019a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.000+0000 7f6a5fe00640 1 -- 172.21.15.47:0/3464125853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a60105490 msgr2=0x7f6a601a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.000+0000 7f6a5fe00640 1 --2- 172.21.15.47:0/3464125853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a60105490 0x7f6a601a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.000+0000 7f6a5fe00640 1 -- 172.21.15.47:0/3464125853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a601a18c0 con 0x7f6a601038e0 2024-03-21T11:40:21.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.000+0000 7f6a5f400640 1 --2- 172.21.15.47:0/3464125853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a60104ad0 0x7f6a6019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:21.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.000+0000 7f6a5fe00640 1 --2- 172.21.15.47:0/3464125853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a601038e0 0x7f6a60199c80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6a50000c00 tx=0x7f6a50005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:21.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.000+0000 7f6a5cc00640 1 -- 172.21.15.47:0/3464125853 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50041070 con 0x7f6a601038e0 2024-03-21T11:40:21.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.000+0000 7f6a66c00640 1 -- 172.21.15.47:0/3464125853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a601a1b50 con 0x7f6a601038e0 2024-03-21T11:40:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.000+0000 7f6a5cc00640 1 -- 172.21.15.47:0/3464125853 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a500044d0 con 0x7f6a601038e0 2024-03-21T11:40:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.000+0000 7f6a5cc00640 1 -- 172.21.15.47:0/3464125853 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50034440 con 0x7f6a601038e0 2024-03-21T11:40:21.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.001+0000 7f6a66c00640 1 -- 172.21.15.47:0/3464125853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a601a2040 con 0x7f6a601038e0 2024-03-21T11:40:21.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.004+0000 7f6a5cc00640 1 -- 172.21.15.47:0/3464125853 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6a50044070 con 0x7f6a601038e0 2024-03-21T11:40:21.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.006+0000 7f6a69812740 1 Processor -- start 2024-03-21T11:40:21.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.006+0000 7f6a69812740 1 -- start start 2024-03-21T11:40:21.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.006+0000 7f6a69812740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d5020 0x555e944d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.007+0000 7f6a69812740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e944d6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.007+0000 7f6a69812740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d6bb0 0x555e944dd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.007+0000 7f6a69812740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e944312c0 con 0x555e944d61f0 2024-03-21T11:40:21.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.007+0000 7f6a69812740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e943d9c90 con 0x555e944d5020 2024-03-21T11:40:21.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.007+0000 7f6a69812740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9442d360 con 0x555e944d6bb0 2024-03-21T11:40:21.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.007+0000 7f6a5fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e944d6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.007+0000 7f6a5fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e944d6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59984/0 (socket says 172.21.15.47:59984) 2024-03-21T11:40:21.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.007+0000 7f6a5fe00640 1 -- 172.21.15.47:0/1328741552 learned_addr learned my addr 172.21.15.47:0/1328741552 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:21.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.007+0000 7f6a5cc00640 1 --2- 172.21.15.47:0/3464125853 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a440a2fa0 0x7f6a440a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.007+0000 7f6a64c00640 1 --2- 172.21.15.47:0/1328741552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d5020 0x555e944d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.008+0000 7f6a5fe00640 1 -- 172.21.15.47:0/1328741552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d6bb0 msgr2=0x555e944dd440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:40:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.008+0000 7f6a5fe00640 1 --2- 172.21.15.47:0/1328741552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d6bb0 0x555e944dd440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.008+0000 7f6a5fe00640 1 -- 172.21.15.47:0/1328741552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d5020 msgr2=0x555e944d5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.008+0000 7f6a5fe00640 1 --2- 172.21.15.47:0/1328741552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d5020 0x555e944d5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.008+0000 7f6a5fe00640 1 -- 172.21.15.47:0/1328741552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e943fe080 con 0x555e944d61f0 2024-03-21T11:40:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.008+0000 7f6a5fe00640 1 --2- 172.21.15.47:0/1328741552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e944d6670 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6a500bd450 tx=0x7f6a500bd0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8753ab786ef090b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.008+0000 7f6a64c00640 1 --2- 172.21.15.47:0/3464125853 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a440a2fa0 0x7f6a440a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.009+0000 7f6a55200640 1 -- 172.21.15.47:0/1328741552 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a500ca250 con 0x555e944d61f0 2024-03-21T11:40:21.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.009+0000 7f6a55200640 1 -- 172.21.15.47:0/1328741552 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a500c9a80 con 0x555e944d61f0 2024-03-21T11:40:21.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.009+0000 7f6a55200640 1 -- 172.21.15.47:0/1328741552 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5003c3b0 con 0x555e944d61f0 2024-03-21T11:40:21.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.009+0000 7f6a5cc00640 1 -- 172.21.15.47:0/3464125853 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6a500f1360 con 0x7f6a601038e0 2024-03-21T11:40:21.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.011+0000 7f6a69812740 1 -- 172.21.15.47:0/1328741552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 msgr2=0x555e944d6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.011+0000 7f6a69812740 1 --2- 172.21.15.47:0/1328741552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e944d6670 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6a500bd450 tx=0x7f6a500bd0b0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.011+0000 7f6a64c00640 1 --2- 172.21.15.47:0/3464125853 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a440a2fa0 0x7f6a440a5460 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x555e944d60f0 tx=0x7f6a58004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:21.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.011+0000 7f6a69812740 1 -- 172.21.15.47:0/1328741552 shutdown_connections 2024-03-21T11:40:21.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.011+0000 7f6a69812740 1 --2- 172.21.15.47:0/1328741552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d6bb0 0x555e944dd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.011+0000 7f6a69812740 1 --2- 172.21.15.47:0/1328741552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e944d6670 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.011+0000 7f6a69812740 1 --2- 172.21.15.47:0/1328741552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d5020 0x555e944d5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.011+0000 7f6a69812740 1 -- 172.21.15.47:0/1328741552 >> 172.21.15.47:0/1328741552 conn(0x555e944c7c00 msgr2=0x555e944c8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:21.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.012+0000 7f6a69812740 1 -- 172.21.15.47:0/1328741552 shutdown_connections 2024-03-21T11:40:21.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.012+0000 7f6a69812740 1 -- 172.21.15.47:0/1328741552 wait complete. 2024-03-21T11:40:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.012+0000 7f6a69812740 1 -- start start 2024-03-21T11:40:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.012+0000 7f6a69812740 1 Processor -- start 2024-03-21T11:40:21.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.013+0000 7f6a69812740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d5020 0x555e944cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.013+0000 7f6a69812740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d61f0 0x555e944cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.013+0000 7f6a69812740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d6bb0 0x555e944d2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.013+0000 7f6a69812740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e944312c0 con 0x555e944d6bb0 2024-03-21T11:40:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.013+0000 7f6a69812740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e943ed0d0 con 0x555e944d5020 2024-03-21T11:40:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.013+0000 7f6a69812740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e944cc390 con 0x555e944d61f0 2024-03-21T11:40:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.013+0000 7f6a69812740 10 client.0 _ll_register_callbacks cb 0x555e943f1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.013+0000 7f6a5fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d5020 0x555e944cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.013+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d61f0 0x555e944cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.013+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d61f0 0x555e944cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46976/0 (socket says 172.21.15.47:46976) 2024-03-21T11:40:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.013+0000 7f6a5fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d5020 0x555e944cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55268/0 (socket says 172.21.15.47:55268) 2024-03-21T11:40:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.013+0000 7f6a5f400640 1 -- 172.21.15.47:0/2006638769 learned_addr learned my addr 172.21.15.47:0/2006638769 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.014+0000 7f6a5f400640 1 -- 172.21.15.47:0/2006638769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d5020 msgr2=0x555e944cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.014+0000 7f6a5f400640 1 --2- 172.21.15.47:0/2006638769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d5020 0x555e944cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.014+0000 7f6a5f400640 1 -- 172.21.15.47:0/2006638769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d6bb0 msgr2=0x555e944d2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.014+0000 7f6a5f400640 1 --2- 172.21.15.47:0/2006638769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d6bb0 0x555e944d2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.014+0000 7f6a5f400640 1 -- 172.21.15.47:0/2006638769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e941453f0 con 0x555e944d61f0 2024-03-21T11:40:21.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.014+0000 7f6a5fe00640 1 --2- 172.21.15.47:0/2006638769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d5020 0x555e944cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:21.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.014+0000 7f6a5f400640 1 --2- 172.21.15.47:0/2006638769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d61f0 0x555e944cdeb0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c011970 tx=0x7f6a4c011e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49579bb408016fdc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:21.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.014+0000 7f6a49600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:40:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.014+0000 7f6a49600640 1 -- 172.21.15.47:0/2006638769 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00fc50 con 0x555e944d61f0 2024-03-21T11:40:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.014+0000 7f6a69812740 1 -- 172.21.15.47:0/2006638769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e9443e800 con 0x555e944d61f0 2024-03-21T11:40:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.014+0000 7f6a49600640 1 -- 172.21.15.47:0/2006638769 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6a4c00fdf0 con 0x555e944d61f0 2024-03-21T11:40:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.014+0000 7f6a49600640 1 -- 172.21.15.47:0/2006638769 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a4c023050 con 0x555e944d61f0 2024-03-21T11:40:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.014+0000 7f6a69812740 10 client.31066 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.014+0000 7f6a69812740 1 -- 172.21.15.47:0/2006638769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555e9420f7d0 con 0x555e944d61f0 2024-03-21T11:40:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.015+0000 7f6a49600640 1 -- 172.21.15.47:0/2006638769 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555e9443e800 con 0x555e944d61f0 2024-03-21T11:40:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.015+0000 7f6a49600640 1 -- 172.21.15.47:0/2006638769 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6a4c019410 con 0x555e944d61f0 2024-03-21T11:40:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.015+0000 7f6a69812740 10 client.31066 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:21.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.015+0000 7f6a69812740 10 client.31066 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:40:21.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.015+0000 7f6a69812740 20 client.31066 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:21.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.015+0000 7f6a69812740 1 --2- 172.21.15.47:0/2006638769 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x555e94504270 0x555e945066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.015+0000 7f6a69812740 4 client.31066 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:21.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.015+0000 7f6a69812740 1 -- 172.21.15.47:0/2006638769 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e94438b50 con 0x555e94504270 2024-03-21T11:40:21.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.015+0000 7f6a5fe00640 1 --2- 172.21.15.47:0/2006638769 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x555e94504270 0x555e945066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.016+0000 7f6a49600640 10 client.31066 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:40:21.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.016+0000 7f6a5fe00640 1 --2- 172.21.15.47:0/2006638769 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x555e94504270 0x555e945066b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6bef1d9a0f461379 server_cookie=324b9567b74e1ef4 in_seq=0 out_seq=0 2024-03-21T11:40:21.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.017+0000 7f6a49600640 1 -- 172.21.15.47:0/2006638769 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555e94438b50 con 0x555e94504270 2024-03-21T11:40:21.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.017+0000 7f6a49600640 10 client.31066 handle_command_reply: tid=0 2024-03-21T11:40:21.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.017+0000 7f6a69812740 1 client.31066 shutdown 2024-03-21T11:40:21.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.017+0000 7f6a69812740 1 -- 172.21.15.47:0/2006638769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d61f0 msgr2=0x555e944cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.017+0000 7f6a69812740 1 --2- 172.21.15.47:0/2006638769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d61f0 0x555e944cdeb0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c011970 tx=0x7f6a4c011e30 comp rx=0 tx=0).stop 2024-03-21T11:40:21.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.017+0000 7f6a69812740 1 -- 172.21.15.47:0/2006638769 shutdown_connections 2024-03-21T11:40:21.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.017+0000 7f6a69812740 1 --2- 172.21.15.47:0/2006638769 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x555e94504270 0x555e945066b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.017+0000 7f6a69812740 1 --2- 172.21.15.47:0/2006638769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d6bb0 0x555e944d2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.017+0000 7f6a69812740 1 --2- 172.21.15.47:0/2006638769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d61f0 0x555e944cdeb0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.017+0000 7f6a69812740 1 --2- 172.21.15.47:0/2006638769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d5020 0x555e944cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.017+0000 7f6a69812740 1 -- 172.21.15.47:0/2006638769 >> 172.21.15.47:0/2006638769 conn(0x555e944c7c00 msgr2=0x555e944c6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:21.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.017+0000 7f6a49600640 0 client.31066 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:40:21.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.018+0000 7f6a69812740 1 -- 172.21.15.47:0/2006638769 shutdown_connections 2024-03-21T11:40:21.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.018+0000 7f6a69812740 1 -- 172.21.15.47:0/2006638769 wait complete. 2024-03-21T11:40:21.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.018+0000 7f6a69812740 20 client.31066 trim_cache size 0 max 16384 2024-03-21T11:40:21.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.036+0000 7f6a69812740 1 Processor -- start 2024-03-21T11:40:21.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.036+0000 7f6a69812740 1 -- start start 2024-03-21T11:40:21.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.036+0000 7f6a69812740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d5020 0x555e944c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.036+0000 7f6a69812740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e944cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a69812740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d6bb0 0x555e944d9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a69812740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a60070dd0 con 0x555e944d61f0 2024-03-21T11:40:21.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a69812740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a60072750 con 0x555e944d6bb0 2024-03-21T11:40:21.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a69812740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a60071430 con 0x555e944d5020 2024-03-21T11:40:21.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e944cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a5fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d5020 0x555e944c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e944cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60008/0 (socket says 172.21.15.47:60008) 2024-03-21T11:40:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a5f400640 1 -- 172.21.15.47:0/1750940 learned_addr learned my addr 172.21.15.47:0/1750940 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a64c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d6bb0 0x555e944d9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a5fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d5020 0x555e944c2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46978/0 (socket says 172.21.15.47:46978) 2024-03-21T11:40:21.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a5f400640 1 -- 172.21.15.47:0/1750940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d5020 msgr2=0x555e944c2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a5f400640 1 --2- 172.21.15.47:0/1750940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d5020 0x555e944c2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a5f400640 1 -- 172.21.15.47:0/1750940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d6bb0 msgr2=0x555e944d9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a5f400640 1 --2- 172.21.15.47:0/1750940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d6bb0 0x555e944d9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.037+0000 7f6a5f400640 1 -- 172.21.15.47:0/1750940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e944c3510 con 0x555e944d61f0 2024-03-21T11:40:21.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.038+0000 7f6a5f400640 1 --2- 172.21.15.47:0/1750940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e944cb570 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c019900 tx=0x7f6a4c0115e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fdfda77238c6ad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:21.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.038+0000 7f6a4be00640 1 -- 172.21.15.47:0/1750940 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c019610 con 0x555e944d61f0 2024-03-21T11:40:21.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.038+0000 7f6a4be00640 1 -- 172.21.15.47:0/1750940 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a4c00fc50 con 0x555e944d61f0 2024-03-21T11:40:21.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.038+0000 7f6a4be00640 1 -- 172.21.15.47:0/1750940 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c004290 con 0x555e944d61f0 2024-03-21T11:40:21.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.039+0000 7f6a69812740 1 -- 172.21.15.47:0/1750940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 msgr2=0x555e944cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.039+0000 7f6a69812740 1 --2- 172.21.15.47:0/1750940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e944cb570 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c019900 tx=0x7f6a4c0115e0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.039+0000 7f6a69812740 1 -- 172.21.15.47:0/1750940 shutdown_connections 2024-03-21T11:40:21.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.039+0000 7f6a69812740 1 --2- 172.21.15.47:0/1750940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d6bb0 0x555e944d9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.039+0000 7f6a69812740 1 --2- 172.21.15.47:0/1750940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e944cb570 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.039+0000 7f6a69812740 1 --2- 172.21.15.47:0/1750940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d5020 0x555e944c2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.039+0000 7f6a69812740 1 -- 172.21.15.47:0/1750940 >> 172.21.15.47:0/1750940 conn(0x555e944c7c00 msgr2=0x555e944cae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:21.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.040+0000 7f6a69812740 1 -- 172.21.15.47:0/1750940 shutdown_connections 2024-03-21T11:40:21.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.040+0000 7f6a69812740 1 -- 172.21.15.47:0/1750940 wait complete. 2024-03-21T11:40:21.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.040+0000 7f6a69812740 1 -- start start 2024-03-21T11:40:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.040+0000 7f6a69812740 1 Processor -- start 2024-03-21T11:40:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a69812740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d5020 0x555e94516670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a69812740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e94513110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a69812740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d6bb0 0x555e94513650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a69812740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600720f0 con 0x555e944d61f0 2024-03-21T11:40:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a69812740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a60071a90 con 0x555e944d6bb0 2024-03-21T11:40:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a69812740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6006ef90 con 0x555e944d5020 2024-03-21T11:40:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a69812740 10 client.0 _ll_register_callbacks cb 0x7f6a4c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e94513110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e94513110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60010/0 (socket says 172.21.15.47:60010) 2024-03-21T11:40:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a5f400640 1 -- 172.21.15.47:0/105682487 learned_addr learned my addr 172.21.15.47:0/105682487 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a5fe00640 1 --2- 172.21.15.47:0/105682487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d5020 0x555e94516670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a64c00640 1 --2- 172.21.15.47:0/105682487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d6bb0 0x555e94513650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a5fe00640 1 -- 172.21.15.47:0/105682487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d6bb0 msgr2=0x555e94513650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a5fe00640 1 --2- 172.21.15.47:0/105682487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d6bb0 0x555e94513650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a5fe00640 1 -- 172.21.15.47:0/105682487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 msgr2=0x555e94513110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a5fe00640 1 --2- 172.21.15.47:0/105682487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e94513110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a5fe00640 1 -- 172.21.15.47:0/105682487 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e94513cf0 con 0x555e944d5020 2024-03-21T11:40:21.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.041+0000 7f6a64c00640 1 --2- 172.21.15.47:0/105682487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d6bb0 0x555e94513650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:40:21.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.042+0000 7f6a5fe00640 1 --2- 172.21.15.47:0/105682487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d5020 0x555e94516670 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f6a50043000 tx=0x7f6a50003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f64327998cbc997 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:21.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.042+0000 7f6a49600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:40:21.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.042+0000 7f6a49600640 1 -- 172.21.15.47:0/105682487 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a500ca050 con 0x555e944d5020 2024-03-21T11:40:21.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.042+0000 7f6a49600640 1 -- 172.21.15.47:0/105682487 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6a500036f0 con 0x555e944d5020 2024-03-21T11:40:21.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.042+0000 7f6a69812740 1 -- 172.21.15.47:0/105682487 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a4c00fc50 con 0x555e944d5020 2024-03-21T11:40:21.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.043+0000 7f6a49600640 1 -- 172.21.15.47:0/105682487 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a5004a390 con 0x555e944d5020 2024-03-21T11:40:21.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.045+0000 7f6a69812740 10 client.31072 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:21.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.045+0000 7f6a69812740 1 -- 172.21.15.47:0/105682487 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555e94560220 con 0x555e944d5020 2024-03-21T11:40:21.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.045+0000 7f6a49600640 1 -- 172.21.15.47:0/105682487 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a4c00fc50 con 0x555e944d5020 2024-03-21T11:40:21.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.046+0000 7f6a49600640 1 -- 172.21.15.47:0/105682487 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6a500a6ce0 con 0x555e944d5020 2024-03-21T11:40:21.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.046+0000 7f6a69812740 10 client.31072 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:21.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.046+0000 7f6a69812740 10 client.31072 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:40:21.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.046+0000 7f6a69812740 20 client.31072 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:21.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.046+0000 7f6a69812740 1 --2- 172.21.15.47:0/105682487 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x555e94560890 0x555e94562c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.046+0000 7f6a69812740 4 client.31072 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:40:21.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.046+0000 7f6a69812740 1 -- 172.21.15.47:0/105682487 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555e943b6500 con 0x555e94560890 2024-03-21T11:40:21.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.047+0000 7f6a5f400640 1 --2- 172.21.15.47:0/105682487 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x555e94560890 0x555e94562c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.047+0000 7f6a5f400640 1 --2- 172.21.15.47:0/105682487 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x555e94560890 0x555e94562c80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=643d0026b0b76db1 server_cookie=6a149eb937304188 in_seq=0 out_seq=0 2024-03-21T11:40:21.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.047+0000 7f6a49600640 10 client.31072 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:40:21.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a49600640 1 -- 172.21.15.47:0/105682487 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555e943b6500 con 0x555e94560890 2024-03-21T11:40:21.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a49600640 10 client.31072 handle_command_reply: tid=0 2024-03-21T11:40:21.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a69812740 1 client.31072 shutdown 2024-03-21T11:40:21.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a69812740 1 -- 172.21.15.47:0/105682487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d5020 msgr2=0x555e94516670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a69812740 1 --2- 172.21.15.47:0/105682487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d5020 0x555e94516670 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f6a50043000 tx=0x7f6a50003f90 comp rx=0 tx=0).stop 2024-03-21T11:40:21.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a69812740 1 -- 172.21.15.47:0/105682487 shutdown_connections 2024-03-21T11:40:21.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a69812740 1 --2- 172.21.15.47:0/105682487 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x555e94560890 0x555e94562c80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a69812740 1 --2- 172.21.15.47:0/105682487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e944d6bb0 0x555e94513650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a69812740 1 --2- 172.21.15.47:0/105682487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e944d61f0 0x555e94513110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a69812740 1 --2- 172.21.15.47:0/105682487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e944d5020 0x555e94516670 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a49600640 0 client.31072 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:40:21.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a69812740 1 -- 172.21.15.47:0/105682487 >> 172.21.15.47:0/105682487 conn(0x555e944c7c00 msgr2=0x555e944cb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:21.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a69812740 1 -- 172.21.15.47:0/105682487 shutdown_connections 2024-03-21T11:40:21.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a69812740 1 -- 172.21.15.47:0/105682487 wait complete. 2024-03-21T11:40:21.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.048+0000 7f6a69812740 20 client.31072 trim_cache size 0 max 16384 2024-03-21T11:40:21.052 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:40:21.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.049+0000 7f6a66c00640 1 -- 172.21.15.47:0/3464125853 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a440a2fa0 msgr2=0x7f6a440a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.049+0000 7f6a66c00640 1 --2- 172.21.15.47:0/3464125853 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a440a2fa0 0x7f6a440a5460 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x555e944d60f0 tx=0x7f6a58004a40 comp rx=0 tx=0).stop 2024-03-21T11:40:21.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.049+0000 7f6a66c00640 1 -- 172.21.15.47:0/3464125853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a601038e0 msgr2=0x7f6a60199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.049+0000 7f6a66c00640 1 --2- 172.21.15.47:0/3464125853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a601038e0 0x7f6a60199c80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6a50000c00 tx=0x7f6a50005d50 comp rx=0 tx=0).stop 2024-03-21T11:40:21.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.050+0000 7f6a66c00640 1 -- 172.21.15.47:0/3464125853 shutdown_connections 2024-03-21T11:40:21.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.050+0000 7f6a66c00640 1 --2- 172.21.15.47:0/3464125853 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a440a2fa0 0x7f6a440a5460 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.050+0000 7f6a66c00640 1 --2- 172.21.15.47:0/3464125853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a60105490 0x7f6a601a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.050+0000 7f6a66c00640 1 --2- 172.21.15.47:0/3464125853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a60104ad0 0x7f6a6019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.050+0000 7f6a66c00640 1 --2- 172.21.15.47:0/3464125853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a601038e0 0x7f6a60199c80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.050+0000 7f6a66c00640 1 -- 172.21.15.47:0/3464125853 >> 172.21.15.47:0/3464125853 conn(0x7f6a600fee50 msgr2=0x7f6a60107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:21.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.050+0000 7f6a66c00640 1 -- 172.21.15.47:0/3464125853 shutdown_connections 2024-03-21T11:40:21.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.050+0000 7f6a66c00640 1 -- 172.21.15.47:0/3464125853 wait complete. 2024-03-21T11:40:21.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:20 smithi047 ceph-mon[31351]: pgmap v910: 129 pgs: 129 active+clean; 419 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.4 MiB/s wr, 655 op/s 2024-03-21T11:40:21.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:20 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1781662927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:40:21.071 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:40:21.071 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:40:21.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.323+0000 7f7f79a00640 1 -- 172.21.15.47:0/4156282339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f74103260 msgr2=0x7f7f74109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.323+0000 7f7f79a00640 1 --2- 172.21.15.47:0/4156282339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f74103260 0x7f7f74109af0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009ca0 tx=0x7f7f6c02c100 comp rx=0 tx=0).stop 2024-03-21T11:40:21.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.323+0000 7f7f79a00640 1 -- 172.21.15.47:0/4156282339 shutdown_connections 2024-03-21T11:40:21.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.323+0000 7f7f79a00640 1 --2- 172.21.15.47:0/4156282339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f74103260 0x7f7f74109af0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.323+0000 7f7f79a00640 1 --2- 172.21.15.47:0/4156282339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f741028a0 0x7f7f74102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.323+0000 7f7f79a00640 1 --2- 172.21.15.47:0/4156282339 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f741016b0 0x7f7f74101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.323+0000 7f7f79a00640 1 -- 172.21.15.47:0/4156282339 >> 172.21.15.47:0/4156282339 conn(0x7f7f740fcc40 msgr2=0x7f7f740ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:21.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.324+0000 7f7f79a00640 1 -- 172.21.15.47:0/4156282339 shutdown_connections 2024-03-21T11:40:21.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.324+0000 7f7f79a00640 1 -- 172.21.15.47:0/4156282339 wait complete. 2024-03-21T11:40:21.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.325+0000 7f7f79a00640 1 Processor -- start 2024-03-21T11:40:21.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.325+0000 7f7f79a00640 1 -- start start 2024-03-21T11:40:21.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.326+0000 7f7f79a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f741016b0 0x7f7f74197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.326+0000 7f7f79a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f741028a0 0x7f7f74197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.326+0000 7f7f79a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f74103260 0x7f7f7419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.326+0000 7f7f79a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7410d310 con 0x7f7f74103260 2024-03-21T11:40:21.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.326+0000 7f7f79a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7410d190 con 0x7f7f741028a0 2024-03-21T11:40:21.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.326+0000 7f7f79a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7410d490 con 0x7f7f741016b0 2024-03-21T11:40:21.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.326+0000 7f7f72a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f741016b0 0x7f7f74197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.326+0000 7f7f72a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f741016b0 0x7f7f74197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47006/0 (socket says 172.21.15.47:47006) 2024-03-21T11:40:21.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.326+0000 7f7f72a00640 1 -- 172.21.15.47:0/3970222122 learned_addr learned my addr 172.21.15.47:0/3970222122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:21.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.326+0000 7f7f72000640 1 --2- 172.21.15.47:0/3970222122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f741028a0 0x7f7f74197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.327+0000 7f7f72a00640 1 -- 172.21.15.47:0/3970222122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f741028a0 msgr2=0x7f7f74197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.327+0000 7f7f72a00640 1 --2- 172.21.15.47:0/3970222122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f741028a0 0x7f7f74197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.327+0000 7f7f72a00640 1 -- 172.21.15.47:0/3970222122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f74103260 msgr2=0x7f7f7419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.327+0000 7f7f72a00640 1 --2- 172.21.15.47:0/3970222122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f74103260 0x7f7f7419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.327+0000 7f7f72a00640 1 -- 172.21.15.47:0/3970222122 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7419f570 con 0x7f7f741016b0 2024-03-21T11:40:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.327+0000 7f7f72000640 1 --2- 172.21.15.47:0/3970222122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f741028a0 0x7f7f74197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.327+0000 7f7f72a00640 1 --2- 172.21.15.47:0/3970222122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f741016b0 0x7f7f74197950 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00c970 tx=0x7f7f5c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.328+0000 7f7f6b400640 1 -- 172.21.15.47:0/3970222122 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c007c10 con 0x7f7f741016b0 2024-03-21T11:40:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.328+0000 7f7f6b400640 1 -- 172.21.15.47:0/3970222122 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c007db0 con 0x7f7f741016b0 2024-03-21T11:40:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.328+0000 7f7f79a00640 1 -- 172.21.15.47:0/3970222122 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7419f860 con 0x7f7f741016b0 2024-03-21T11:40:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.328+0000 7f7f6b400640 1 -- 172.21.15.47:0/3970222122 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00f870 con 0x7f7f741016b0 2024-03-21T11:40:21.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.328+0000 7f7f79a00640 1 -- 172.21.15.47:0/3970222122 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7419fe00 con 0x7f7f741016b0 2024-03-21T11:40:21.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.336+0000 7f7f7c7de740 1 Processor -- start 2024-03-21T11:40:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.339+0000 7f7f7c7de740 1 -- start start 2024-03-21T11:40:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.339+0000 7f7f7c7de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989c05e0 0x5595989c0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.339+0000 7f7f7c7de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989bf2a0 0x5595989bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.339+0000 7f7f7c7de740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 0x5595989c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.339+0000 7f7f7c7de740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55959891b2c0 con 0x5595989bfc60 2024-03-21T11:40:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.339+0000 7f7f7c7de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595988c3c90 con 0x5595989c05e0 2024-03-21T11:40:21.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.339+0000 7f7f7c7de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559598917360 con 0x5595989bf2a0 2024-03-21T11:40:21.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.340+0000 7f7f73400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989c05e0 0x5595989c0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.340+0000 7f7f73400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989c05e0 0x5595989c0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55290/0 (socket says 172.21.15.47:55290) 2024-03-21T11:40:21.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.340+0000 7f7f73400640 1 -- 172.21.15.47:0/1554524693 learned_addr learned my addr 172.21.15.47:0/1554524693 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:21.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.341+0000 7f7f6b400640 1 -- 172.21.15.47:0/3970222122 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7f5c00fad0 con 0x7f7f741016b0 2024-03-21T11:40:21.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.341+0000 7f7f72000640 1 --2- 172.21.15.47:0/1554524693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 0x5595989c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.342+0000 7f7f72000640 1 -- 172.21.15.47:0/1554524693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989bf2a0 msgr2=0x5595989bf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.342+0000 7f7f72000640 1 --2- 172.21.15.47:0/1554524693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989bf2a0 0x5595989bf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.342+0000 7f7f72000640 1 -- 172.21.15.47:0/1554524693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989c05e0 msgr2=0x5595989c0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.342+0000 7f7f72000640 1 --2- 172.21.15.47:0/1554524693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989c05e0 0x5595989c0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.342+0000 7f7f72000640 1 -- 172.21.15.47:0/1554524693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595988e8080 con 0x5595989bfc60 2024-03-21T11:40:21.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.344+0000 7f7f72000640 1 --2- 172.21.15.47:0/1554524693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 0x5595989c7500 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400bfb0 tx=0x7f7f64010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca910ee0c88e8eaa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:21.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.345+0000 7f7f6b400640 1 --2- 172.21.15.47:0/3970222122 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f580a2ff0 0x7f7f580a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.345+0000 7f7f6b400640 1 -- 172.21.15.47:0/3970222122 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7f5c0cfac0 con 0x7f7f741016b0 2024-03-21T11:40:21.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.345+0000 7f7f63e00640 1 -- 172.21.15.47:0/1554524693 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f64004220 con 0x5595989bfc60 2024-03-21T11:40:21.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.345+0000 7f7f63e00640 1 -- 172.21.15.47:0/1554524693 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f640043c0 con 0x5595989bfc60 2024-03-21T11:40:21.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.345+0000 7f7f63e00640 1 -- 172.21.15.47:0/1554524693 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f64011a80 con 0x5595989bfc60 2024-03-21T11:40:21.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.346+0000 7f7f73400640 1 --2- 172.21.15.47:0/3970222122 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f580a2ff0 0x7f7f580a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.347+0000 7f7f7c7de740 1 -- 172.21.15.47:0/1554524693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 msgr2=0x5595989c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.347+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/1554524693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 0x5595989c7500 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400bfb0 tx=0x7f7f64010860 comp rx=0 tx=0).stop 2024-03-21T11:40:21.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.347+0000 7f7f7c7de740 1 -- 172.21.15.47:0/1554524693 shutdown_connections 2024-03-21T11:40:21.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.347+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/1554524693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 0x5595989c7500 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.347+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/1554524693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989bf2a0 0x5595989bf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.347+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/1554524693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989c05e0 0x5595989c0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.347+0000 7f7f7c7de740 1 -- 172.21.15.47:0/1554524693 >> 172.21.15.47:0/1554524693 conn(0x5595989b1b80 msgr2=0x5595989b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:21.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.348+0000 7f7f73400640 1 --2- 172.21.15.47:0/3970222122 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f580a2ff0 0x7f7f580a54b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009a60 tx=0x7f7f6c005f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:21.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.348+0000 7f7f7c7de740 1 -- 172.21.15.47:0/1554524693 shutdown_connections 2024-03-21T11:40:21.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.348+0000 7f7f7c7de740 1 -- 172.21.15.47:0/1554524693 wait complete. 2024-03-21T11:40:21.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.349+0000 7f7f7c7de740 1 -- start start 2024-03-21T11:40:21.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.349+0000 7f7f7c7de740 1 Processor -- start 2024-03-21T11:40:21.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.350+0000 7f7f7c7de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989bf2a0 0x5595989c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.350+0000 7f7f7c7de740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 0x5595989c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.350+0000 7f7f7c7de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989c05e0 0x5595989d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.350+0000 7f7f7c7de740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595989c7c80 con 0x5595989bfc60 2024-03-21T11:40:21.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.350+0000 7f7f7c7de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595988d70d0 con 0x5595989c05e0 2024-03-21T11:40:21.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.350+0000 7f7f7c7de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595989c7e00 con 0x5595989bf2a0 2024-03-21T11:40:21.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.350+0000 7f7f7c7de740 10 client.0 _ll_register_callbacks cb 0x5595988dbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:21.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.350+0000 7f7f73400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989c05e0 0x5595989d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.350+0000 7f7f73400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989c05e0 0x5595989d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55292/0 (socket says 172.21.15.47:55292) 2024-03-21T11:40:21.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.350+0000 7f7f73400640 1 -- 172.21.15.47:0/2058034288 learned_addr learned my addr 172.21.15.47:0/2058034288 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:21.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.351+0000 7f7f72000640 1 --2- 172.21.15.47:0/2058034288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 0x5595989c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.350+0000 7f7f72a00640 1 --2- 172.21.15.47:0/2058034288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989bf2a0 0x5595989c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.351+0000 7f7f73400640 1 -- 172.21.15.47:0/2058034288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989bf2a0 msgr2=0x5595989c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.351+0000 7f7f73400640 1 --2- 172.21.15.47:0/2058034288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989bf2a0 0x5595989c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.351+0000 7f7f73400640 1 -- 172.21.15.47:0/2058034288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 msgr2=0x5595989c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.351+0000 7f7f73400640 1 --2- 172.21.15.47:0/2058034288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 0x5595989c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.352+0000 7f7f73400640 1 -- 172.21.15.47:0/2058034288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55959862f3f0 con 0x5595989c05e0 2024-03-21T11:40:21.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.353+0000 7f7f73400640 1 --2- 172.21.15.47:0/2058034288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989c05e0 0x5595989d08b0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009ca0 tx=0x7f7f6c032590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9a2ca15967aca08 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:21.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.353+0000 7f7f60c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:21.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.353+0000 7f7f60c00640 1 -- 172.21.15.47:0/2058034288 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c050060 con 0x5595989c05e0 2024-03-21T11:40:21.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.354+0000 7f7f7c7de740 1 -- 172.21.15.47:0/2058034288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f640043c0 con 0x5595989c05e0 2024-03-21T11:40:21.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.354+0000 7f7f60c00640 1 -- 172.21.15.47:0/2058034288 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7f6c003e40 con 0x5595989c05e0 2024-03-21T11:40:21.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.354+0000 7f7f60c00640 1 -- 172.21.15.47:0/2058034288 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f6c057050 con 0x5595989c05e0 2024-03-21T11:40:21.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.354+0000 7f7f7c7de740 10 client.31100 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:21.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.355+0000 7f7f7c7de740 1 -- 172.21.15.47:0/2058034288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5595986f97d0 con 0x5595989c05e0 2024-03-21T11:40:21.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.355+0000 7f7f60c00640 1 -- 172.21.15.47:0/2058034288 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f640043c0 con 0x5595989c05e0 2024-03-21T11:40:21.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.355+0000 7f7f60c00640 1 -- 172.21.15.47:0/2058034288 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7f6c03fa70 con 0x5595989c05e0 2024-03-21T11:40:21.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.355+0000 7f7f7c7de740 10 client.31100 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:21.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.355+0000 7f7f7c7de740 10 client.31100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:21.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.355+0000 7f7f7c7de740 20 client.31100 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:21.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.355+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/2058034288 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5595989d4cf0 0x5595989d7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.356+0000 7f7f7c7de740 4 client.31100 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:21.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.356+0000 7f7f7c7de740 1 -- 172.21.15.47:0/2058034288 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559598928800 con 0x5595989d4cf0 2024-03-21T11:40:21.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.356+0000 7f7f72a00640 1 --2- 172.21.15.47:0/2058034288 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5595989d4cf0 0x5595989d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.358+0000 7f7f72a00640 1 --2- 172.21.15.47:0/2058034288 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5595989d4cf0 0x5595989d7140 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0247969f4afcd0b server_cookie=e530d9ffbfb07f95 in_seq=0 out_seq=0 2024-03-21T11:40:21.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.359+0000 7f7f60c00640 10 client.31100 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:21.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.361+0000 7f7f60c00640 1 -- 172.21.15.47:0/2058034288 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559598928800 con 0x5595989d4cf0 2024-03-21T11:40:21.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.361+0000 7f7f60c00640 10 client.31100 handle_command_reply: tid=0 2024-03-21T11:40:21.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.361+0000 7f7f7c7de740 1 client.31100 shutdown 2024-03-21T11:40:21.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.361+0000 7f7f7c7de740 1 -- 172.21.15.47:0/2058034288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989c05e0 msgr2=0x5595989d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.361+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/2058034288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989c05e0 0x5595989d08b0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009ca0 tx=0x7f7f6c032590 comp rx=0 tx=0).stop 2024-03-21T11:40:21.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.361+0000 7f7f7c7de740 1 -- 172.21.15.47:0/2058034288 shutdown_connections 2024-03-21T11:40:21.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.361+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/2058034288 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5595989d4cf0 0x5595989d7140 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.362+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/2058034288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989c05e0 0x5595989d08b0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.362+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/2058034288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 0x5595989c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.362+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/2058034288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989bf2a0 0x5595989c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.362+0000 7f7f7c7de740 1 -- 172.21.15.47:0/2058034288 >> 172.21.15.47:0/2058034288 conn(0x5595989b1b80 msgr2=0x5595989beb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:21.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.362+0000 7f7f60c00640 0 client.31100 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:21.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.362+0000 7f7f7c7de740 1 -- 172.21.15.47:0/2058034288 shutdown_connections 2024-03-21T11:40:21.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.363+0000 7f7f7c7de740 1 -- 172.21.15.47:0/2058034288 wait complete. 2024-03-21T11:40:21.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.363+0000 7f7f7c7de740 20 client.31100 trim_cache size 0 max 16384 2024-03-21T11:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.388+0000 7f7f7c7de740 1 Processor -- start 2024-03-21T11:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.388+0000 7f7f7c7de740 1 -- start start 2024-03-21T11:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.388+0000 7f7f7c7de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989b1b80 0x5595989dba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.388+0000 7f7f7c7de740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bf2a0 0x5595989d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.388+0000 7f7f7c7de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989bfc60 0x5595989dbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.388+0000 7f7f7c7de740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f741a7f70 con 0x5595989bf2a0 2024-03-21T11:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.388+0000 7f7f7c7de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f741a98f0 con 0x5595989b1b80 2024-03-21T11:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.388+0000 7f7f7c7de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f741a85d0 con 0x5595989bfc60 2024-03-21T11:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.389+0000 7f7f72a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989b1b80 0x5595989dba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.389+0000 7f7f72a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989b1b80 0x5595989dba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55300/0 (socket says 172.21.15.47:55300) 2024-03-21T11:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.389+0000 7f7f72a00640 1 -- 172.21.15.47:0/3843154502 learned_addr learned my addr 172.21.15.47:0/3843154502 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.389+0000 7f7f73400640 1 --2- 172.21.15.47:0/3843154502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989bfc60 0x5595989dbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.389+0000 7f7f72000640 1 --2- 172.21.15.47:0/3843154502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bf2a0 0x5595989d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.389+0000 7f7f72a00640 1 -- 172.21.15.47:0/3843154502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989bfc60 msgr2=0x5595989dbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.389+0000 7f7f72a00640 1 --2- 172.21.15.47:0/3843154502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989bfc60 0x5595989dbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.389+0000 7f7f72a00640 1 -- 172.21.15.47:0/3843154502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bf2a0 msgr2=0x5595989d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.389+0000 7f7f72a00640 1 --2- 172.21.15.47:0/3843154502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bf2a0 0x5595989d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.389+0000 7f7f72a00640 1 -- 172.21.15.47:0/3843154502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595989b56d0 con 0x5595989b1b80 2024-03-21T11:40:21.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.390+0000 7f7f72a00640 1 --2- 172.21.15.47:0/3843154502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989b1b80 0x5595989dba50 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c09bb90 tx=0x7f7f5c018ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59956017d55b716d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.390+0000 7f7f63400640 1 -- 172.21.15.47:0/3843154502 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0a2040 con 0x5595989b1b80 2024-03-21T11:40:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.390+0000 7f7f63400640 1 -- 172.21.15.47:0/3843154502 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c0a4810 con 0x5595989b1b80 2024-03-21T11:40:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.390+0000 7f7f63400640 1 -- 172.21.15.47:0/3843154502 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c098500 con 0x5595989b1b80 2024-03-21T11:40:21.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.391+0000 7f7f7c7de740 1 -- 172.21.15.47:0/3843154502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989b1b80 msgr2=0x5595989dba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.391+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/3843154502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989b1b80 0x5595989dba50 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c09bb90 tx=0x7f7f5c018ec0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.391+0000 7f7f7c7de740 1 -- 172.21.15.47:0/3843154502 shutdown_connections 2024-03-21T11:40:21.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.391+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/3843154502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989bfc60 0x5595989dbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.391+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/3843154502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bf2a0 0x5595989d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.391+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/3843154502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989b1b80 0x5595989dba50 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.391+0000 7f7f7c7de740 1 -- 172.21.15.47:0/3843154502 >> 172.21.15.47:0/3843154502 conn(0x5595989c75f0 msgr2=0x5595989beb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:21.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.392+0000 7f7f7c7de740 1 -- 172.21.15.47:0/3843154502 shutdown_connections 2024-03-21T11:40:21.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.392+0000 7f7f7c7de740 1 -- 172.21.15.47:0/3843154502 wait complete. 2024-03-21T11:40:21.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.393+0000 7f7f7c7de740 1 -- start start 2024-03-21T11:40:21.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.393+0000 7f7f7c7de740 1 Processor -- start 2024-03-21T11:40:21.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.394+0000 7f7f7c7de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989b1b80 0x5595989cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.394+0000 7f7f7c7de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989bf2a0 0x5595989ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.394+0000 7f7f7c7de740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 0x5595989cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.394+0000 7f7f7c7de740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f741a9290 con 0x5595989bfc60 2024-03-21T11:40:21.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.394+0000 7f7f7c7de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f741a8c30 con 0x5595989bf2a0 2024-03-21T11:40:21.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.394+0000 7f7f7c7de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f741a6130 con 0x5595989b1b80 2024-03-21T11:40:21.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.394+0000 7f7f7c7de740 10 client.0 _ll_register_callbacks cb 0x7f7f6c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:21.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.394+0000 7f7f73400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 0x5595989cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.394+0000 7f7f73400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 0x5595989cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60066/0 (socket says 172.21.15.47:60066) 2024-03-21T11:40:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.394+0000 7f7f73400640 1 -- 172.21.15.47:0/944388619 learned_addr learned my addr 172.21.15.47:0/944388619 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.394+0000 7f7f72000640 1 --2- 172.21.15.47:0/944388619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989bf2a0 0x5595989ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.394+0000 7f7f72a00640 1 --2- 172.21.15.47:0/944388619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989b1b80 0x5595989cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.395+0000 7f7f72000640 1 -- 172.21.15.47:0/944388619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989b1b80 msgr2=0x5595989cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.395+0000 7f7f72000640 1 --2- 172.21.15.47:0/944388619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989b1b80 0x5595989cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.395+0000 7f7f72000640 1 -- 172.21.15.47:0/944388619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 msgr2=0x5595989cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.395+0000 7f7f72000640 1 --2- 172.21.15.47:0/944388619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 0x5595989cfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.395+0000 7f7f72000640 1 -- 172.21.15.47:0/944388619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595989d00b0 con 0x5595989bf2a0 2024-03-21T11:40:21.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.395+0000 7f7f72000640 1 --2- 172.21.15.47:0/944388619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989bf2a0 0x5595989ccb60 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7f64010d60 tx=0x7f7f64005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b42468aab429754 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:21.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.395+0000 7f7f60c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.396+0000 7f7f60c00640 1 -- 172.21.15.47:0/944388619 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6402a030 con 0x5595989bf2a0 2024-03-21T11:40:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.396+0000 7f7f60c00640 1 -- 172.21.15.47:0/944388619 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7f640055f0 con 0x5595989bf2a0 2024-03-21T11:40:21.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.396+0000 7f7f7c7de740 1 -- 172.21.15.47:0/944388619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f5c0a4810 con 0x5595989bf2a0 2024-03-21T11:40:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.396+0000 7f7f60c00640 1 -- 172.21.15.47:0/944388619 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f64034050 con 0x5595989bf2a0 2024-03-21T11:40:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.397+0000 7f7f7c7de740 10 client.31106 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.397+0000 7f7f7c7de740 1 -- 172.21.15.47:0/944388619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559598a30cc0 con 0x5595989bf2a0 2024-03-21T11:40:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.397+0000 7f7f60c00640 1 -- 172.21.15.47:0/944388619 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c0a4810 con 0x5595989bf2a0 2024-03-21T11:40:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.397+0000 7f7f60c00640 1 -- 172.21.15.47:0/944388619 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7f640119e0 con 0x5595989bf2a0 2024-03-21T11:40:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.398+0000 7f7f7c7de740 10 client.31106 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.398+0000 7f7f7c7de740 10 client.31106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.398+0000 7f7f7c7de740 20 client.31106 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.398+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/944388619 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5595989c05e0 0x559598a333e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.398+0000 7f7f7c7de740 4 client.31106 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:40:21.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.398+0000 7f7f7c7de740 1 -- 172.21.15.47:0/944388619 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5595989b2500 con 0x5595989c05e0 2024-03-21T11:40:21.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.399+0000 7f7f72a00640 1 --2- 172.21.15.47:0/944388619 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5595989c05e0 0x559598a333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:21.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.402+0000 7f7f72a00640 1 --2- 172.21.15.47:0/944388619 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5595989c05e0 0x559598a333e0 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc2b173f11606af server_cookie=3fa4af162b33e496 in_seq=0 out_seq=0 2024-03-21T11:40:21.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.402+0000 7f7f60c00640 10 client.31106 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:21.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.408+0000 7f7f60c00640 1 -- 172.21.15.47:0/944388619 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5595989b2500 con 0x5595989c05e0 2024-03-21T11:40:21.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.408+0000 7f7f60c00640 10 client.31106 handle_command_reply: tid=0 2024-03-21T11:40:21.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.408+0000 7f7f7c7de740 1 client.31106 shutdown 2024-03-21T11:40:21.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.409+0000 7f7f7c7de740 1 -- 172.21.15.47:0/944388619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989bf2a0 msgr2=0x5595989ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.409+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/944388619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989bf2a0 0x5595989ccb60 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7f64010d60 tx=0x7f7f64005270 comp rx=0 tx=0).stop 2024-03-21T11:40:21.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.409+0000 7f7f7c7de740 1 -- 172.21.15.47:0/944388619 shutdown_connections 2024-03-21T11:40:21.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.409+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/944388619 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5595989c05e0 0x559598a333e0 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.410+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/944388619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5595989bfc60 0x5595989cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.410+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/944388619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5595989bf2a0 0x5595989ccb60 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.410+0000 7f7f60c00640 0 client.31106 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:21.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.410+0000 7f7f7c7de740 1 --2- 172.21.15.47:0/944388619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5595989b1b80 0x5595989cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.410+0000 7f7f7c7de740 1 -- 172.21.15.47:0/944388619 >> 172.21.15.47:0/944388619 conn(0x5595989c75f0 msgr2=0x5595989c34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:21.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.410+0000 7f7f7c7de740 1 -- 172.21.15.47:0/944388619 shutdown_connections 2024-03-21T11:40:21.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.410+0000 7f7f7c7de740 1 -- 172.21.15.47:0/944388619 wait complete. 2024-03-21T11:40:21.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.410+0000 7f7f7c7de740 20 client.31106 trim_cache size 0 max 16384 2024-03-21T11:40:21.412 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:40:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.411+0000 7f7f79a00640 1 -- 172.21.15.47:0/3970222122 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f580a2ff0 msgr2=0x7f7f580a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.411+0000 7f7f79a00640 1 --2- 172.21.15.47:0/3970222122 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f580a2ff0 0x7f7f580a54b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009a60 tx=0x7f7f6c005f80 comp rx=0 tx=0).stop 2024-03-21T11:40:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.411+0000 7f7f79a00640 1 -- 172.21.15.47:0/3970222122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f741016b0 msgr2=0x7f7f74197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.411+0000 7f7f79a00640 1 --2- 172.21.15.47:0/3970222122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f741016b0 0x7f7f74197950 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00c970 tx=0x7f7f5c00ce30 comp rx=0 tx=0).stop 2024-03-21T11:40:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.412+0000 7f7f79a00640 1 -- 172.21.15.47:0/3970222122 shutdown_connections 2024-03-21T11:40:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.412+0000 7f7f79a00640 1 --2- 172.21.15.47:0/3970222122 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f580a2ff0 0x7f7f580a54b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.412+0000 7f7f79a00640 1 --2- 172.21.15.47:0/3970222122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f74103260 0x7f7f7419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.412+0000 7f7f79a00640 1 --2- 172.21.15.47:0/3970222122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f741028a0 0x7f7f74197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.412+0000 7f7f79a00640 1 --2- 172.21.15.47:0/3970222122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f741016b0 0x7f7f74197950 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.412+0000 7f7f79a00640 1 -- 172.21.15.47:0/3970222122 >> 172.21.15.47:0/3970222122 conn(0x7f7f740fcc40 msgr2=0x7f7f74105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.412+0000 7f7f79a00640 1 -- 172.21.15.47:0/3970222122 shutdown_connections 2024-03-21T11:40:21.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:21.412+0000 7f7f79a00640 1 -- 172.21.15.47:0/3970222122 wait complete. 2024-03-21T11:40:21.941 INFO:tasks.workunit.client.0.smithi047.stdout: 10 88723 25.19 MB/sec execute 65 sec latency 93.676 ms 2024-03-21T11:40:22.441 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:40:22.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:40:22.442 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:40:22.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.800+0000 7efc02000640 1 -- 172.21.15.47:0/3625300279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efbfc103260 msgr2=0x7efbfc109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.800+0000 7efc02000640 1 --2- 172.21.15.47:0/3625300279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efbfc103260 0x7efbfc109af0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009ca0 tx=0x7efbf402c100 comp rx=0 tx=0).stop 2024-03-21T11:40:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.800+0000 7efc02000640 1 -- 172.21.15.47:0/3625300279 shutdown_connections 2024-03-21T11:40:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.800+0000 7efc02000640 1 --2- 172.21.15.47:0/3625300279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efbfc103260 0x7efbfc109af0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.800+0000 7efc02000640 1 --2- 172.21.15.47:0/3625300279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbfc1028a0 0x7efbfc102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.800+0000 7efc02000640 1 --2- 172.21.15.47:0/3625300279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efbfc1016b0 0x7efbfc101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.800+0000 7efc02000640 1 -- 172.21.15.47:0/3625300279 >> 172.21.15.47:0/3625300279 conn(0x7efbfc0fcc40 msgr2=0x7efbfc0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.800+0000 7efc02000640 1 -- 172.21.15.47:0/3625300279 shutdown_connections 2024-03-21T11:40:22.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.800+0000 7efc02000640 1 -- 172.21.15.47:0/3625300279 wait complete. 2024-03-21T11:40:22.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.802+0000 7efc02000640 1 Processor -- start 2024-03-21T11:40:22.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.802+0000 7efc02000640 1 -- start start 2024-03-21T11:40:22.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.804+0000 7efc02000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbfc1016b0 0x7efbfc199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.804+0000 7efc02000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efbfc1028a0 0x7efbfc19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.804+0000 7efc02000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efbfc103260 0x7efbfc1a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.804+0000 7efc02000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc10d310 con 0x7efbfc1016b0 2024-03-21T11:40:22.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.804+0000 7efc02000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc10d190 con 0x7efbfc1028a0 2024-03-21T11:40:22.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.804+0000 7efc02000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc10d490 con 0x7efbfc103260 2024-03-21T11:40:22.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.804+0000 7efbfbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efbfc103260 0x7efbfc1a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.805+0000 7efbfbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efbfc103260 0x7efbfc1a1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47060/0 (socket says 172.21.15.47:47060) 2024-03-21T11:40:22.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.805+0000 7efbfbe00640 1 -- 172.21.15.47:0/3938285127 learned_addr learned my addr 172.21.15.47:0/3938285127 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:22.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.805+0000 7efbfaa00640 1 --2- 172.21.15.47:0/3938285127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efbfc1028a0 0x7efbfc19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.805+0000 7efbfbe00640 1 -- 172.21.15.47:0/3938285127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efbfc1028a0 msgr2=0x7efbfc19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.805+0000 7efbfbe00640 1 --2- 172.21.15.47:0/3938285127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efbfc1028a0 0x7efbfc19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.805+0000 7efbfbe00640 1 -- 172.21.15.47:0/3938285127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbfc1016b0 msgr2=0x7efbfc199bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:40:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.805+0000 7efbfbe00640 1 --2- 172.21.15.47:0/3938285127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbfc1016b0 0x7efbfc199bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.805+0000 7efbfbe00640 1 -- 172.21.15.47:0/3938285127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc1a17e0 con 0x7efbfc103260 2024-03-21T11:40:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.805+0000 7efbfbe00640 1 --2- 172.21.15.47:0/3938285127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efbfc103260 0x7efbfc1a1170 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009bf0 tx=0x7efbf4035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.806+0000 7efbf3e00640 1 -- 172.21.15.47:0/3938285127 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4039040 con 0x7efbfc103260 2024-03-21T11:40:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.806+0000 7efc02000640 1 -- 172.21.15.47:0/3938285127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbfc103ac0 con 0x7efbfc103260 2024-03-21T11:40:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.806+0000 7efbf3e00640 1 -- 172.21.15.47:0/3938285127 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efbf4044070 con 0x7efbfc103260 2024-03-21T11:40:22.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.806+0000 7efbf3e00640 1 -- 172.21.15.47:0/3938285127 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40044e0 con 0x7efbfc103260 2024-03-21T11:40:22.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.806+0000 7efc02000640 1 -- 172.21.15.47:0/3938285127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbfc104000 con 0x7efbfc103260 2024-03-21T11:40:22.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.813+0000 7efc04e1c740 1 Processor -- start 2024-03-21T11:40:22.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.813+0000 7efbf3e00640 1 -- 172.21.15.47:0/3938285127 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7efbf4004680 con 0x7efbfc103260 2024-03-21T11:40:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.816+0000 7efbf3e00640 1 --2- 172.21.15.47:0/3938285127 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efbe00ab880 0x7efbe00add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.816+0000 7efc04e1c740 1 -- start start 2024-03-21T11:40:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.816+0000 7efbf3e00640 1 -- 172.21.15.47:0/3938285127 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efbf40f1890 con 0x7efbfc103260 2024-03-21T11:40:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.816+0000 7efc04e1c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 0x55f8204584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.816+0000 7efc04e1c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f820458a90 0x55f82045d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.816+0000 7efc04e1c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f82045dc00 0x55f820460090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.816+0000 7efc04e1c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8203bdeb0 con 0x55f820458a90 2024-03-21T11:40:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.816+0000 7efc04e1c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f820366a60 con 0x55f82045dc00 2024-03-21T11:40:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.816+0000 7efc04e1c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8203b9f50 con 0x55f8204580a0 2024-03-21T11:40:22.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.817+0000 7efbfbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f82045dc00 0x55f820460090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.817+0000 7efbfbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f82045dc00 0x55f820460090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55346/0 (socket says 172.21.15.47:55346) 2024-03-21T11:40:22.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.817+0000 7efbfbe00640 1 -- 172.21.15.47:0/2573979623 learned_addr learned my addr 172.21.15.47:0/2573979623 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.817+0000 7efbfb400640 1 --2- 172.21.15.47:0/2573979623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 0x55f8204584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.817+0000 7efbfaa00640 1 --2- 172.21.15.47:0/3938285127 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efbe00ab880 0x7efbe00add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.817+0000 7efbfaa00640 1 --2- 172.21.15.47:0/2573979623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f820458a90 0x55f82045d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.817+0000 7efbfbe00640 1 -- 172.21.15.47:0/2573979623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 msgr2=0x55f8204584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.817+0000 7efbfbe00640 1 --2- 172.21.15.47:0/2573979623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 0x55f8204584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.817+0000 7efbfbe00640 1 -- 172.21.15.47:0/2573979623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f820458a90 msgr2=0x55f82045d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.817+0000 7efbfbe00640 1 --2- 172.21.15.47:0/2573979623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f820458a90 0x55f82045d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.817+0000 7efbfbe00640 1 -- 172.21.15.47:0/2573979623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f82019a7e0 con 0x55f82045dc00 2024-03-21T11:40:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.818+0000 7efbfbe00640 1 --2- 172.21.15.47:0/2573979623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f82045dc00 0x55f820460090 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7efbf40bd8d0 tx=0x7efbf40bd7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=199cef7f35ebd548 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:22.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.818+0000 7efbe7e00640 1 -- 172.21.15.47:0/2573979623 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40bdbd0 con 0x55f82045dc00 2024-03-21T11:40:22.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.819+0000 7efbe7e00640 1 -- 172.21.15.47:0/2573979623 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efbf40c1d20 con 0x55f82045dc00 2024-03-21T11:40:22.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.819+0000 7efbe7e00640 1 -- 172.21.15.47:0/2573979623 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4003b90 con 0x55f82045dc00 2024-03-21T11:40:22.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.820+0000 7efc04e1c740 1 -- 172.21.15.47:0/2573979623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f82045dc00 msgr2=0x55f820460090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.820+0000 7efbfaa00640 1 --2- 172.21.15.47:0/3938285127 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efbe00ab880 0x7efbe00add40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x55f820458e90 tx=0x7efbec004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.820+0000 7efc04e1c740 1 --2- 172.21.15.47:0/2573979623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f82045dc00 0x55f820460090 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7efbf40bd8d0 tx=0x7efbf40bd7d0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.820+0000 7efc04e1c740 1 -- 172.21.15.47:0/2573979623 shutdown_connections 2024-03-21T11:40:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.820+0000 7efc04e1c740 1 --2- 172.21.15.47:0/2573979623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f82045dc00 0x55f820460090 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.820+0000 7efc04e1c740 1 --2- 172.21.15.47:0/2573979623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f820458a90 0x55f82045d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.820+0000 7efc04e1c740 1 --2- 172.21.15.47:0/2573979623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 0x55f8204584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.820+0000 7efc04e1c740 1 -- 172.21.15.47:0/2573979623 >> 172.21.15.47:0/2573979623 conn(0x55f820453610 msgr2=0x55f820453a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.820+0000 7efc04e1c740 1 -- 172.21.15.47:0/2573979623 shutdown_connections 2024-03-21T11:40:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.820+0000 7efc04e1c740 1 -- 172.21.15.47:0/2573979623 wait complete. 2024-03-21T11:40:22.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.821+0000 7efc04e1c740 1 -- start start 2024-03-21T11:40:22.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.821+0000 7efc04e1c740 1 Processor -- start 2024-03-21T11:40:22.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.822+0000 7efc04e1c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f8204580a0 0x55f820461f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.822+0000 7efc04e1c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f820458a90 0x55f8204624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.822+0000 7efc04e1c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f82045dc00 0x55f8204694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.822+0000 7efc04e1c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f820460970 con 0x55f8204580a0 2024-03-21T11:40:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.822+0000 7efc04e1c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f820379eb0 con 0x55f82045dc00 2024-03-21T11:40:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.822+0000 7efc04e1c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f820460af0 con 0x55f820458a90 2024-03-21T11:40:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.822+0000 7efc04e1c740 10 client.0 _ll_register_callbacks cb 0x55f82036e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.822+0000 7efbfb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f8204580a0 0x55f820461f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.823+0000 7efbfb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f8204580a0 0x55f820461f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60082/0 (socket says 172.21.15.47:60082) 2024-03-21T11:40:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.823+0000 7efbfb400640 1 -- 172.21.15.47:0/56447312 learned_addr learned my addr 172.21.15.47:0/56447312 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.823+0000 7efbfaa00640 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f820458a90 0x55f8204624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.824+0000 7efbfbe00640 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f82045dc00 0x55f8204694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.824+0000 7efbfb400640 1 -- 172.21.15.47:0/56447312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f820458a90 msgr2=0x55f8204624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.824+0000 7efbfb400640 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f820458a90 0x55f8204624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.824+0000 7efbfb400640 1 -- 172.21.15.47:0/56447312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f82045dc00 msgr2=0x55f8204694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.824+0000 7efbfb400640 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f82045dc00 0x55f8204694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.824+0000 7efbfb400640 1 -- 172.21.15.47:0/56447312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8201faf70 con 0x55f8204580a0 2024-03-21T11:40:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.824+0000 7efbfbe00640 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f82045dc00 0x55f8204694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.824+0000 7efbfaa00640 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f820458a90 0x55f8204624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:22.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.826+0000 7efbfb400640 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f8204580a0 0x55f820461f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009f60 tx=0x7efbe8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5080be0726b1ddf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:22.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.826+0000 7efbe4c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:40:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.827+0000 7efbe4c00640 1 -- 172.21.15.47:0/56447312 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe8016070 con 0x55f8204580a0 2024-03-21T11:40:22.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.828+0000 7efbe4c00640 1 -- 172.21.15.47:0/56447312 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efbe8011880 con 0x55f8204580a0 2024-03-21T11:40:22.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.828+0000 7efc04e1c740 1 -- 172.21.15.47:0/56447312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efbf40c1d20 con 0x55f8204580a0 2024-03-21T11:40:22.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.830+0000 7efbe4c00640 1 -- 172.21.15.47:0/56447312 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbe8020050 con 0x55f8204580a0 2024-03-21T11:40:22.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.830+0000 7efc04e1c740 10 client.20586 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.830+0000 7efc04e1c740 1 -- 172.21.15.47:0/56447312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f820092170 con 0x55f8204580a0 2024-03-21T11:40:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.832+0000 7efbe4c00640 1 -- 172.21.15.47:0/56447312 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efbf40c1d20 con 0x55f8204580a0 2024-03-21T11:40:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.832+0000 7efbe4c00640 1 -- 172.21.15.47:0/56447312 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efbe800f040 con 0x55f8204580a0 2024-03-21T11:40:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.832+0000 7efc04e1c740 10 client.20586 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.832+0000 7efc04e1c740 10 client.20586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.833+0000 7efc04e1c740 20 client.20586 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.833+0000 7efc04e1c740 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f82046de10 0x55f8204702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.833+0000 7efc04e1c740 4 client.20586 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:22.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.833+0000 7efc04e1c740 1 -- 172.21.15.47:0/56447312 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8203cb3f0 con 0x55f82046de10 2024-03-21T11:40:22.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.833+0000 7efbfaa00640 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f82046de10 0x55f8204702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.835+0000 7efbfaa00640 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f82046de10 0x55f8204702b0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96a254069bee036f server_cookie=421f8b9ec191c1bd in_seq=0 out_seq=0 2024-03-21T11:40:22.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.835+0000 7efbe4c00640 10 client.20586 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:22.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.836+0000 7efbe4c00640 1 -- 172.21.15.47:0/56447312 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f8203cb3f0 con 0x55f82046de10 2024-03-21T11:40:22.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.836+0000 7efbe4c00640 10 client.20586 handle_command_reply: tid=0 2024-03-21T11:40:22.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.838+0000 7efc04e1c740 1 client.20586 shutdown 2024-03-21T11:40:22.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.840+0000 7efc04e1c740 1 -- 172.21.15.47:0/56447312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f8204580a0 msgr2=0x55f820461f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:22.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.840+0000 7efc04e1c740 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f8204580a0 0x55f820461f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009f60 tx=0x7efbe8009f90 comp rx=0 tx=0).stop 2024-03-21T11:40:22.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.843+0000 7efc04e1c740 1 -- 172.21.15.47:0/56447312 shutdown_connections 2024-03-21T11:40:22.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.843+0000 7efc04e1c740 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f82046de10 0x55f8204702b0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.843+0000 7efc04e1c740 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f82045dc00 0x55f8204694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.843+0000 7efc04e1c740 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f820458a90 0x55f8204624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.843+0000 7efc04e1c740 1 --2- 172.21.15.47:0/56447312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f8204580a0 0x55f820461f70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.843+0000 7efc04e1c740 1 -- 172.21.15.47:0/56447312 >> 172.21.15.47:0/56447312 conn(0x55f820453610 msgr2=0x55f820452830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:22.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.843+0000 7efbe4c00640 0 client.20586 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:22.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.843+0000 7efc04e1c740 1 -- 172.21.15.47:0/56447312 shutdown_connections 2024-03-21T11:40:22.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.844+0000 7efc04e1c740 1 -- 172.21.15.47:0/56447312 wait complete. 2024-03-21T11:40:22.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.844+0000 7efc04e1c740 20 client.20586 trim_cache size 0 max 16384 2024-03-21T11:40:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.879+0000 7efc04e1c740 1 Processor -- start 2024-03-21T11:40:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.879+0000 7efc04e1c740 1 -- start start 2024-03-21T11:40:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.880+0000 7efc04e1c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 0x55f820474bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.880+0000 7efc04e1c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f820458a90 0x55f8204694b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.880+0000 7efc04e1c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f82045dc00 0x55f8204750f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.880+0000 7efc04e1c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc1a8010 con 0x55f82045dc00 2024-03-21T11:40:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.880+0000 7efc04e1c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc1a9990 con 0x55f820458a90 2024-03-21T11:40:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.880+0000 7efc04e1c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc1a8670 con 0x55f8204580a0 2024-03-21T11:40:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.880+0000 7efbfaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f820458a90 0x55f8204694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.880+0000 7efbfb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 0x55f820474bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.880+0000 7efbfaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f820458a90 0x55f8204694b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55370/0 (socket says 172.21.15.47:55370) 2024-03-21T11:40:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.880+0000 7efbfaa00640 1 -- 172.21.15.47:0/1272978817 learned_addr learned my addr 172.21.15.47:0/1272978817 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.880+0000 7efbfbe00640 1 --2- 172.21.15.47:0/1272978817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f82045dc00 0x55f8204750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.881+0000 7efbfaa00640 1 -- 172.21.15.47:0/1272978817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 msgr2=0x55f820474bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.881+0000 7efbfaa00640 1 --2- 172.21.15.47:0/1272978817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 0x55f820474bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.881+0000 7efbfaa00640 1 -- 172.21.15.47:0/1272978817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f82045dc00 msgr2=0x55f8204750f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:22.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.881+0000 7efbfaa00640 1 --2- 172.21.15.47:0/1272978817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f82045dc00 0x55f8204750f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.881+0000 7efbfaa00640 1 -- 172.21.15.47:0/1272978817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f82044ec20 con 0x55f820458a90 2024-03-21T11:40:22.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.881+0000 7efbfaa00640 1 --2- 172.21.15.47:0/1272978817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f820458a90 0x55f8204694b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x55f820459500 tx=0x7efbec012070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b99bf4327523892 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:22.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.881+0000 7efbe7400640 1 -- 172.21.15.47:0/1272978817 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec019040 con 0x55f820458a90 2024-03-21T11:40:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.881+0000 7efbe7400640 1 -- 172.21.15.47:0/1272978817 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efbec005500 con 0x55f820458a90 2024-03-21T11:40:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.881+0000 7efbe7400640 1 -- 172.21.15.47:0/1272978817 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec004d70 con 0x55f820458a90 2024-03-21T11:40:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.883+0000 7efc04e1c740 1 -- 172.21.15.47:0/1272978817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f820458a90 msgr2=0x55f8204694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.883+0000 7efc04e1c740 1 --2- 172.21.15.47:0/1272978817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f820458a90 0x55f8204694b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x55f820459500 tx=0x7efbec012070 comp rx=0 tx=0).stop 2024-03-21T11:40:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.884+0000 7efc04e1c740 1 -- 172.21.15.47:0/1272978817 shutdown_connections 2024-03-21T11:40:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.884+0000 7efc04e1c740 1 --2- 172.21.15.47:0/1272978817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f82045dc00 0x55f8204750f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.884+0000 7efc04e1c740 1 --2- 172.21.15.47:0/1272978817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f820458a90 0x55f8204694b0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.884+0000 7efc04e1c740 1 --2- 172.21.15.47:0/1272978817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 0x55f820474bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.884+0000 7efc04e1c740 1 -- 172.21.15.47:0/1272978817 >> 172.21.15.47:0/1272978817 conn(0x55f820453610 msgr2=0x55f820452830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.885+0000 7efc04e1c740 1 -- 172.21.15.47:0/1272978817 shutdown_connections 2024-03-21T11:40:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.885+0000 7efc04e1c740 1 -- 172.21.15.47:0/1272978817 wait complete. 2024-03-21T11:40:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.885+0000 7efc04e1c740 1 -- start start 2024-03-21T11:40:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.886+0000 7efc04e1c740 1 Processor -- start 2024-03-21T11:40:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.886+0000 7efc04e1c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 0x55f820466f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.886+0000 7efc04e1c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f820458a90 0x55f820467470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.886+0000 7efc04e1c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f82045dc00 0x55f820468480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.886+0000 7efc04e1c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc1a9330 con 0x55f82045dc00 2024-03-21T11:40:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.886+0000 7efc04e1c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc1a8cd0 con 0x55f820458a90 2024-03-21T11:40:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.886+0000 7efc04e1c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc1a61d0 con 0x55f8204580a0 2024-03-21T11:40:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.886+0000 7efc04e1c740 10 client.0 _ll_register_callbacks cb 0x7efbe8009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.886+0000 7efbfbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f82045dc00 0x55f820468480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.886+0000 7efbfb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 0x55f820466f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.886+0000 7efbfb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 0x55f820466f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47100/0 (socket says 172.21.15.47:47100) 2024-03-21T11:40:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.887+0000 7efbfb400640 1 -- 172.21.15.47:0/882899075 learned_addr learned my addr 172.21.15.47:0/882899075 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.887+0000 7efbfaa00640 1 --2- 172.21.15.47:0/882899075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f820458a90 0x55f820467470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.887+0000 7efbfb400640 1 -- 172.21.15.47:0/882899075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f820458a90 msgr2=0x55f820467470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.887+0000 7efbfb400640 1 --2- 172.21.15.47:0/882899075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f820458a90 0x55f820467470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.887+0000 7efbfb400640 1 -- 172.21.15.47:0/882899075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f82045dc00 msgr2=0x55f820468480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.887+0000 7efbfb400640 1 --2- 172.21.15.47:0/882899075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f82045dc00 0x55f820468480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.887+0000 7efbfb400640 1 -- 172.21.15.47:0/882899075 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f820468a70 con 0x55f8204580a0 2024-03-21T11:40:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.887+0000 7efbfb400640 1 --2- 172.21.15.47:0/882899075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 0x55f820466f30 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7efbe8001eb0 tx=0x7efbe800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d93f69be3f7cf25 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.887+0000 7efbe4c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:40:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.888+0000 7efbe4c00640 1 -- 172.21.15.47:0/882899075 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe8016070 con 0x55f8204580a0 2024-03-21T11:40:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.888+0000 7efbe4c00640 1 -- 172.21.15.47:0/882899075 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efbe801b850 con 0x55f8204580a0 2024-03-21T11:40:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.889+0000 7efc04e1c740 1 -- 172.21.15.47:0/882899075 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efbec005500 con 0x55f8204580a0 2024-03-21T11:40:22.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.889+0000 7efbe4c00640 1 -- 172.21.15.47:0/882899075 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbe8034050 con 0x55f8204580a0 2024-03-21T11:40:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.890+0000 7efc04e1c740 10 client.31102 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.890+0000 7efc04e1c740 1 -- 172.21.15.47:0/882899075 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f8204c9bb0 con 0x55f8204580a0 2024-03-21T11:40:22.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.895+0000 7efbe4c00640 1 -- 172.21.15.47:0/882899075 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efbec005500 con 0x55f8204580a0 2024-03-21T11:40:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.895+0000 7efbe4c00640 1 -- 172.21.15.47:0/882899075 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efbe80047c0 con 0x55f8204580a0 2024-03-21T11:40:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.895+0000 7efc04e1c740 10 client.31102 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.895+0000 7efc04e1c740 10 client.31102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.895+0000 7efc04e1c740 20 client.31102 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.895+0000 7efc04e1c740 1 --2- 172.21.15.47:0/882899075 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f8204ca470 0x55f8204cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.895+0000 7efc04e1c740 4 client.31102 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:40:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.895+0000 7efc04e1c740 1 -- 172.21.15.47:0/882899075 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f8203c5740 con 0x55f8204ca470 2024-03-21T11:40:22.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.895+0000 7efbfaa00640 1 --2- 172.21.15.47:0/882899075 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f8204ca470 0x55f8204cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.896+0000 7efbfaa00640 1 --2- 172.21.15.47:0/882899075 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f8204ca470 0x55f8204cc8d0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ae26e24c40a215 server_cookie=f03f3c21ce109066 in_seq=0 out_seq=0 2024-03-21T11:40:22.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.896+0000 7efbe4c00640 10 client.31102 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.912+0000 7efbe4c00640 1 -- 172.21.15.47:0/882899075 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f8203c5740 con 0x55f8204ca470 2024-03-21T11:40:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.913+0000 7efbe4c00640 10 client.31102 handle_command_reply: tid=0 2024-03-21T11:40:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.913+0000 7efc04e1c740 1 client.31102 shutdown 2024-03-21T11:40:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.913+0000 7efc04e1c740 1 -- 172.21.15.47:0/882899075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 msgr2=0x55f820466f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.913+0000 7efc04e1c740 1 --2- 172.21.15.47:0/882899075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 0x55f820466f30 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7efbe8001eb0 tx=0x7efbe800ece0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.913+0000 7efc04e1c740 1 -- 172.21.15.47:0/882899075 shutdown_connections 2024-03-21T11:40:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.913+0000 7efc04e1c740 1 --2- 172.21.15.47:0/882899075 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f8204ca470 0x55f8204cc8d0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.913+0000 7efc04e1c740 1 --2- 172.21.15.47:0/882899075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f82045dc00 0x55f820468480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.913+0000 7efc04e1c740 1 --2- 172.21.15.47:0/882899075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f820458a90 0x55f820467470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.913+0000 7efc04e1c740 1 --2- 172.21.15.47:0/882899075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f8204580a0 0x55f820466f30 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.913+0000 7efbe4c00640 0 client.31102 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.913+0000 7efc04e1c740 1 -- 172.21.15.47:0/882899075 >> 172.21.15.47:0/882899075 conn(0x55f820453610 msgr2=0x55f820459530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.913+0000 7efc04e1c740 1 -- 172.21.15.47:0/882899075 shutdown_connections 2024-03-21T11:40:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.913+0000 7efc04e1c740 1 -- 172.21.15.47:0/882899075 wait complete. 2024-03-21T11:40:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.913+0000 7efc04e1c740 20 client.31102 trim_cache size 0 max 16384 2024-03-21T11:40:22.916 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:40:22.916 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:40:22.916 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "939550e2-576c-4604-86a6-3a4f4923bdd5", 2024-03-21T11:40:22.916 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:40:22.916 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:40:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.914+0000 7efc02000640 1 -- 172.21.15.47:0/3938285127 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efbe00ab880 msgr2=0x7efbe00add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.914+0000 7efc02000640 1 --2- 172.21.15.47:0/3938285127 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efbe00ab880 0x7efbe00add40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x55f820458e90 tx=0x7efbec004730 comp rx=0 tx=0).stop 2024-03-21T11:40:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.914+0000 7efc02000640 1 -- 172.21.15.47:0/3938285127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efbfc103260 msgr2=0x7efbfc1a1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.914+0000 7efc02000640 1 --2- 172.21.15.47:0/3938285127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efbfc103260 0x7efbfc1a1170 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009bf0 tx=0x7efbf4035d10 comp rx=0 tx=0).stop 2024-03-21T11:40:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.914+0000 7efc02000640 1 -- 172.21.15.47:0/3938285127 shutdown_connections 2024-03-21T11:40:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.914+0000 7efc02000640 1 --2- 172.21.15.47:0/3938285127 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efbe00ab880 0x7efbe00add40 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.914+0000 7efc02000640 1 --2- 172.21.15.47:0/3938285127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efbfc103260 0x7efbfc1a1170 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.914+0000 7efc02000640 1 --2- 172.21.15.47:0/3938285127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efbfc1028a0 0x7efbfc19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.914+0000 7efc02000640 1 --2- 172.21.15.47:0/3938285127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbfc1016b0 0x7efbfc199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.914+0000 7efc02000640 1 -- 172.21.15.47:0/3938285127 >> 172.21.15.47:0/3938285127 conn(0x7efbfc0fcc40 msgr2=0x7efbfc0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.914+0000 7efc02000640 1 -- 172.21.15.47:0/3938285127 shutdown_connections 2024-03-21T11:40:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:22.914+0000 7efc02000640 1 -- 172.21.15.47:0/3938285127 wait complete. 2024-03-21T11:40:22.931 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:22 smithi047 ceph-mon[31351]: pgmap v911: 129 pgs: 129 active+clean; 415 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 595 op/s 2024-03-21T11:40:22.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:40:22.933 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:40:22.941 INFO:tasks.workunit.client.0.smithi047.stdout: 10 89024 25.04 MB/sec execute 66 sec latency 387.484 ms 2024-03-21T11:40:22.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:22 smithi102 ceph-mon[36748]: pgmap v911: 129 pgs: 129 active+clean; 415 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 595 op/s 2024-03-21T11:40:23.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:22 smithi100 ceph-mon[36245]: pgmap v911: 129 pgs: 129 active+clean; 415 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 595 op/s 2024-03-21T11:40:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.162+0000 7fe4d8c00640 1 -- 172.21.15.47:0/1861075341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d4105370 msgr2=0x7fe4d410bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.162+0000 7fe4d8c00640 1 --2- 172.21.15.47:0/1861075341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d4105370 0x7fe4d410bc00 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009be0 tx=0x7fe4c402bef0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.162+0000 7fe4d8c00640 1 -- 172.21.15.47:0/1861075341 shutdown_connections 2024-03-21T11:40:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.162+0000 7fe4d8c00640 1 --2- 172.21.15.47:0/1861075341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d4105370 0x7fe4d410bc00 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.162+0000 7fe4d8c00640 1 --2- 172.21.15.47:0/1861075341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d41049b0 0x7fe4d4104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.162+0000 7fe4d8c00640 1 --2- 172.21.15.47:0/1861075341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d41037c0 0x7fe4d4103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.162+0000 7fe4d8c00640 1 -- 172.21.15.47:0/1861075341 >> 172.21.15.47:0/1861075341 conn(0x7fe4d40fed50 msgr2=0x7fe4d4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:23.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.162+0000 7fe4d8c00640 1 -- 172.21.15.47:0/1861075341 shutdown_connections 2024-03-21T11:40:23.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.162+0000 7fe4d8c00640 1 -- 172.21.15.47:0/1861075341 wait complete. 2024-03-21T11:40:23.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.163+0000 7fe4d8c00640 1 Processor -- start 2024-03-21T11:40:23.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.163+0000 7fe4d8c00640 1 -- start start 2024-03-21T11:40:23.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.164+0000 7fe4d8c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d41037c0 0x7fe4d4199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.164+0000 7fe4d8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d41049b0 0x7fe4d4199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.164+0000 7fe4d8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d4105370 0x7fe4d41a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.164+0000 7fe4d8c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d410f420 con 0x7fe4d41037c0 2024-03-21T11:40:23.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.164+0000 7fe4d8c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d410f2a0 con 0x7fe4d4105370 2024-03-21T11:40:23.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.164+0000 7fe4d8c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d410f5a0 con 0x7fe4d41049b0 2024-03-21T11:40:23.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.164+0000 7fe4d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d41049b0 0x7fe4d4199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.164+0000 7fe4d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d41049b0 0x7fe4d4199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47114/0 (socket says 172.21.15.47:47114) 2024-03-21T11:40:23.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.164+0000 7fe4d2a00640 1 -- 172.21.15.47:0/331411895 learned_addr learned my addr 172.21.15.47:0/331411895 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:23.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.164+0000 7fe4d3e00640 1 --2- 172.21.15.47:0/331411895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d4105370 0x7fe4d41a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.164+0000 7fe4d3400640 1 --2- 172.21.15.47:0/331411895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d41037c0 0x7fe4d4199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.165+0000 7fe4d2a00640 1 -- 172.21.15.47:0/331411895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d4105370 msgr2=0x7fe4d41a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:23.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.165+0000 7fe4d2a00640 1 --2- 172.21.15.47:0/331411895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d4105370 0x7fe4d41a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.165+0000 7fe4d2a00640 1 -- 172.21.15.47:0/331411895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d41037c0 msgr2=0x7fe4d4199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:23.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.165+0000 7fe4d2a00640 1 --2- 172.21.15.47:0/331411895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d41037c0 0x7fe4d4199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.165+0000 7fe4d2a00640 1 -- 172.21.15.47:0/331411895 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d41a1660 con 0x7fe4d41049b0 2024-03-21T11:40:23.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.165+0000 7fe4d2a00640 1 --2- 172.21.15.47:0/331411895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d41049b0 0x7fe4d4199f80 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00ea60 tx=0x7fe4cc00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:23.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.166+0000 7fe4cbe00640 1 -- 172.21.15.47:0/331411895 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00cd20 con 0x7fe4d41049b0 2024-03-21T11:40:23.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.166+0000 7fe4d8c00640 1 -- 172.21.15.47:0/331411895 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d41a1950 con 0x7fe4d41049b0 2024-03-21T11:40:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.166+0000 7fe4cbe00640 1 -- 172.21.15.47:0/331411895 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4cc0035f0 con 0x7fe4d41049b0 2024-03-21T11:40:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.166+0000 7fe4cbe00640 1 -- 172.21.15.47:0/331411895 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc010690 con 0x7fe4d41049b0 2024-03-21T11:40:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.166+0000 7fe4d8c00640 1 -- 172.21.15.47:0/331411895 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d41a1ef0 con 0x7fe4d41049b0 2024-03-21T11:40:23.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.170+0000 7fe4dcfd4740 1 Processor -- start 2024-03-21T11:40:23.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.170+0000 7fe4dcfd4740 1 -- start start 2024-03-21T11:40:23.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.171+0000 7fe4dcfd4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75050 0x55a774b75470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.171+0000 7fe4dcfd4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b75a40 0x55a774b7a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.171+0000 7fe4dcfd4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b7abb0 0x55a774b7d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.171+0000 7fe4dcfd4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a774ada250 con 0x55a774b75050 2024-03-21T11:40:23.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.171+0000 7fe4dcfd4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a774a82a30 con 0x55a774b75a40 2024-03-21T11:40:23.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.171+0000 7fe4dcfd4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a774ad62f0 con 0x55a774b7abb0 2024-03-21T11:40:23.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.171+0000 7fe4d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b75a40 0x55a774b7a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.171+0000 7fe4d3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75050 0x55a774b75470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.171+0000 7fe4d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b75a40 0x55a774b7a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55406/0 (socket says 172.21.15.47:55406) 2024-03-21T11:40:23.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.171+0000 7fe4d3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75050 0x55a774b75470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60138/0 (socket says 172.21.15.47:60138) 2024-03-21T11:40:23.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.171+0000 7fe4d3400640 1 -- 172.21.15.47:0/2043699386 learned_addr learned my addr 172.21.15.47:0/2043699386 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:23.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.171+0000 7fe4cbe00640 1 -- 172.21.15.47:0/331411895 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe4cc004000 con 0x7fe4d41049b0 2024-03-21T11:40:23.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.172+0000 7fe4d3e00640 1 -- 172.21.15.47:0/2043699386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b7abb0 msgr2=0x55a774b7d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:23.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.172+0000 7fe4d3e00640 1 --2- 172.21.15.47:0/2043699386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b7abb0 0x55a774b7d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.172+0000 7fe4d3e00640 1 -- 172.21.15.47:0/2043699386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b75a40 msgr2=0x55a774b7a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:23.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.172+0000 7fe4d3e00640 1 --2- 172.21.15.47:0/2043699386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b75a40 0x55a774b7a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.172+0000 7fe4d3e00640 1 -- 172.21.15.47:0/2043699386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a774a6d690 con 0x55a774b75050 2024-03-21T11:40:23.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.172+0000 7fe4d3e00640 1 --2- 172.21.15.47:0/2043699386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75050 0x55a774b75470 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4034fd0 tx=0x7fe4c4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261b3dc8747e1fd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:23.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.173+0000 7fe4c3e00640 1 -- 172.21.15.47:0/2043699386 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4043030 con 0x55a774b75050 2024-03-21T11:40:23.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.173+0000 7fe4c3e00640 1 -- 172.21.15.47:0/2043699386 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4c4033b90 con 0x55a774b75050 2024-03-21T11:40:23.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.173+0000 7fe4c3e00640 1 -- 172.21.15.47:0/2043699386 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c402c890 con 0x55a774b75050 2024-03-21T11:40:23.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.173+0000 7fe4cbe00640 1 --2- 172.21.15.47:0/331411895 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4b80a2fa0 0x7fe4b80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.173+0000 7fe4cbe00640 1 -- 172.21.15.47:0/331411895 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe4cc0cf910 con 0x7fe4d41049b0 2024-03-21T11:40:23.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.174+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/2043699386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75050 msgr2=0x55a774b75470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:23.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.174+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/2043699386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75050 0x55a774b75470 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4034fd0 tx=0x7fe4c4009310 comp rx=0 tx=0).stop 2024-03-21T11:40:23.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.174+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/2043699386 shutdown_connections 2024-03-21T11:40:23.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.174+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/2043699386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b7abb0 0x55a774b7d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.174+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/2043699386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b75a40 0x55a774b7a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.174+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/2043699386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75050 0x55a774b75470 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.174+0000 7fe4d3400640 1 --2- 172.21.15.47:0/331411895 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4b80a2fa0 0x7fe4b80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.174+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/2043699386 >> 172.21.15.47:0/2043699386 conn(0x55a774b3d200 msgr2=0x55a774b709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:23.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.174+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/2043699386 shutdown_connections 2024-03-21T11:40:23.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.174+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/2043699386 wait complete. 2024-03-21T11:40:23.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.175+0000 7fe4dcfd4740 1 -- start start 2024-03-21T11:40:23.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.175+0000 7fe4d3400640 1 --2- 172.21.15.47:0/331411895 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4b80a2fa0 0x7fe4b80a5460 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc005fd0 tx=0x7fe4bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:23.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.175+0000 7fe4dcfd4740 1 Processor -- start 2024-03-21T11:40:23.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.175+0000 7fe4dcfd4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b75050 0x55a774b7f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.175+0000 7fe4dcfd4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75a40 0x55a774b7f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.175+0000 7fe4dcfd4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b7abb0 0x55a774b86590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.175+0000 7fe4dcfd4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a774b7db00 con 0x55a774b75a40 2024-03-21T11:40:23.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.175+0000 7fe4dcfd4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a774a95e80 con 0x55a774b75050 2024-03-21T11:40:23.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.175+0000 7fe4dcfd4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a774b7dc80 con 0x55a774b7abb0 2024-03-21T11:40:23.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.175+0000 7fe4dcfd4740 10 client.0 _ll_register_callbacks cb 0x55a774aa8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:23.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.175+0000 7fe4d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b75050 0x55a774b7f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.176+0000 7fe4d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b75050 0x55a774b7f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55410/0 (socket says 172.21.15.47:55410) 2024-03-21T11:40:23.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.176+0000 7fe4d3400640 1 -- 172.21.15.47:0/1539022162 learned_addr learned my addr 172.21.15.47:0/1539022162 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:23.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.176+0000 7fe4d2a00640 1 --2- 172.21.15.47:0/1539022162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75a40 0x55a774b7f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.176+0000 7fe4d3e00640 1 --2- 172.21.15.47:0/1539022162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b7abb0 0x55a774b86590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.176+0000 7fe4d3400640 1 -- 172.21.15.47:0/1539022162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b7abb0 msgr2=0x55a774b86590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:23.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.176+0000 7fe4d3400640 1 --2- 172.21.15.47:0/1539022162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b7abb0 0x55a774b86590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.176+0000 7fe4d3400640 1 -- 172.21.15.47:0/1539022162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75a40 msgr2=0x55a774b7f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:23.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.176+0000 7fe4d3400640 1 --2- 172.21.15.47:0/1539022162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75a40 0x55a774b7f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.176+0000 7fe4d3400640 1 -- 172.21.15.47:0/1539022162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7748b87d0 con 0x55a774b75050 2024-03-21T11:40:23.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.176+0000 7fe4d3400640 1 --2- 172.21.15.47:0/1539022162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b75050 0x55a774b7f010 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc01da00 tx=0x7fe4bc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22eee943da6d9f24 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:23.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.176+0000 7fe4c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:23.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.177+0000 7fe4c0c00640 1 -- 172.21.15.47:0/1539022162 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc02d060 con 0x55a774b75050 2024-03-21T11:40:23.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.177+0000 7fe4c0c00640 1 -- 172.21.15.47:0/1539022162 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe4bc017030 con 0x55a774b75050 2024-03-21T11:40:23.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.177+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/1539022162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4c4033b90 con 0x55a774b75050 2024-03-21T11:40:23.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.177+0000 7fe4c0c00640 1 -- 172.21.15.47:0/1539022162 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4bc034050 con 0x55a774b75050 2024-03-21T11:40:23.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.177+0000 7fe4dcfd4740 10 client.31133 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:23.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.177+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/1539022162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a7747ee3f0 con 0x55a774b75050 2024-03-21T11:40:23.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.178+0000 7fe4c0c00640 1 -- 172.21.15.47:0/1539022162 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4c4033b90 con 0x55a774b75050 2024-03-21T11:40:23.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.178+0000 7fe4c0c00640 1 -- 172.21.15.47:0/1539022162 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe4bc01b540 con 0x55a774b75050 2024-03-21T11:40:23.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.178+0000 7fe4dcfd4740 10 client.31133 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:23.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.178+0000 7fe4dcfd4740 10 client.31133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:23.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.178+0000 7fe4dcfd4740 20 client.31133 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:23.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.178+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/1539022162 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a774b93680 0x55a774b95ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.178+0000 7fe4dcfd4740 4 client.31133 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:23.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.178+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/1539022162 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a774ae7790 con 0x55a774b93680 2024-03-21T11:40:23.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.178+0000 7fe4d2a00640 1 --2- 172.21.15.47:0/1539022162 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a774b93680 0x55a774b95ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.179+0000 7fe4c0c00640 10 client.31133 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:23.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.179+0000 7fe4d2a00640 1 --2- 172.21.15.47:0/1539022162 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a774b93680 0x55a774b95ae0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2343de61eca50a6 server_cookie=a7c93369757a924b in_seq=0 out_seq=0 2024-03-21T11:40:23.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.183+0000 7fe4c0c00640 1 -- 172.21.15.47:0/1539022162 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a774ae7790 con 0x55a774b93680 2024-03-21T11:40:23.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.183+0000 7fe4c0c00640 10 client.31133 handle_command_reply: tid=0 2024-03-21T11:40:23.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.183+0000 7fe4dcfd4740 1 client.31133 shutdown 2024-03-21T11:40:23.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.183+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/1539022162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b75050 msgr2=0x55a774b7f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:23.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.183+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/1539022162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b75050 0x55a774b7f010 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc01da00 tx=0x7fe4bc004730 comp rx=0 tx=0).stop 2024-03-21T11:40:23.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.183+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/1539022162 shutdown_connections 2024-03-21T11:40:23.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.183+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/1539022162 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a774b93680 0x55a774b95ae0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.183+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/1539022162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b7abb0 0x55a774b86590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.183+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/1539022162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75a40 0x55a774b7f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.183+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/1539022162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b75050 0x55a774b7f010 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.183+0000 7fe4c0c00640 0 client.31133 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:23.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.183+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/1539022162 >> 172.21.15.47:0/1539022162 conn(0x55a774b3d200 msgr2=0x55a774b6fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:23.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.184+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/1539022162 shutdown_connections 2024-03-21T11:40:23.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.184+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/1539022162 wait complete. 2024-03-21T11:40:23.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.184+0000 7fe4dcfd4740 20 client.31133 trim_cache size 0 max 16384 2024-03-21T11:40:23.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.203+0000 7fe4dcfd4740 1 Processor -- start 2024-03-21T11:40:23.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.204+0000 7fe4dcfd4740 1 -- start start 2024-03-21T11:40:23.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.204+0000 7fe4dcfd4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75050 0x55a774b7f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.204+0000 7fe4dcfd4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b75a40 0x55a774b86520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.204+0000 7fe4dcfd4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b7abb0 0x55a774b6fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.204+0000 7fe4dcfd4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b000f970 con 0x55a774b75050 2024-03-21T11:40:23.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.204+0000 7fe4dcfd4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b00112f0 con 0x55a774b7abb0 2024-03-21T11:40:23.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.204+0000 7fe4dcfd4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b000ffd0 con 0x55a774b75a40 2024-03-21T11:40:23.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.204+0000 7fe4d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b7abb0 0x55a774b6fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.204+0000 7fe4d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b75a40 0x55a774b86520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.204+0000 7fe4d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b7abb0 0x55a774b6fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55426/0 (socket says 172.21.15.47:55426) 2024-03-21T11:40:23.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.204+0000 7fe4d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b75a40 0x55a774b86520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47142/0 (socket says 172.21.15.47:47142) 2024-03-21T11:40:23.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.204+0000 7fe4d3e00640 1 -- 172.21.15.47:0/2682694118 learned_addr learned my addr 172.21.15.47:0/2682694118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:23.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.204+0000 7fe4d3400640 1 --2- 172.21.15.47:0/2682694118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75050 0x55a774b7f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.205+0000 7fe4d3e00640 1 -- 172.21.15.47:0/2682694118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b75a40 msgr2=0x55a774b86520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:23.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.205+0000 7fe4d3e00640 1 --2- 172.21.15.47:0/2682694118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b75a40 0x55a774b86520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.205+0000 7fe4d3e00640 1 -- 172.21.15.47:0/2682694118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75050 msgr2=0x55a774b7f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:23.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.205+0000 7fe4d3e00640 1 --2- 172.21.15.47:0/2682694118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75050 0x55a774b7f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.205+0000 7fe4d3e00640 1 -- 172.21.15.47:0/2682694118 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7747ae0f0 con 0x55a774b7abb0 2024-03-21T11:40:23.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.205+0000 7fe4d3400640 1 --2- 172.21.15.47:0/2682694118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75050 0x55a774b7f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:23.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.205+0000 7fe4d3e00640 1 --2- 172.21.15.47:0/2682694118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b7abb0 0x55a774b6fa40 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009be0 tx=0x7fe4c4009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cf6dc57721d39a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:23.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.205+0000 7fe4c3400640 1 -- 172.21.15.47:0/2682694118 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c404c070 con 0x55a774b7abb0 2024-03-21T11:40:23.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.206+0000 7fe4c3400640 1 -- 172.21.15.47:0/2682694118 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4c4003970 con 0x55a774b7abb0 2024-03-21T11:40:23.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.206+0000 7fe4c3400640 1 -- 172.21.15.47:0/2682694118 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4043040 con 0x55a774b7abb0 2024-03-21T11:40:23.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.206+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/2682694118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b7abb0 msgr2=0x55a774b6fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:23.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.206+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/2682694118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b7abb0 0x55a774b6fa40 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009be0 tx=0x7fe4c4009940 comp rx=0 tx=0).stop 2024-03-21T11:40:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.206+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/2682694118 shutdown_connections 2024-03-21T11:40:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.206+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/2682694118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b7abb0 0x55a774b6fa40 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.206+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/2682694118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b75a40 0x55a774b86520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.206+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/2682694118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75050 0x55a774b7f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.206+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/2682694118 >> 172.21.15.47:0/2682694118 conn(0x55a774b3d200 msgr2=0x55a774b6f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.206+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/2682694118 shutdown_connections 2024-03-21T11:40:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.206+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/2682694118 wait complete. 2024-03-21T11:40:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.207+0000 7fe4dcfd4740 1 -- start start 2024-03-21T11:40:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.207+0000 7fe4dcfd4740 1 Processor -- start 2024-03-21T11:40:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.207+0000 7fe4dcfd4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b75050 0x55a774b84150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4dcfd4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75a40 0x55a774b84690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4dcfd4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b7abb0 0x55a774b85600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4dcfd4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b0010c90 con 0x55a774b75a40 2024-03-21T11:40:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4dcfd4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b0010630 con 0x55a774b7abb0 2024-03-21T11:40:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4dcfd4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b000db30 con 0x55a774b75050 2024-03-21T11:40:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4dcfd4740 10 client.0 _ll_register_callbacks cb 0x7fe4bc01d890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b7abb0 0x55a774b85600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4d2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75a40 0x55a774b84690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b7abb0 0x55a774b85600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55436/0 (socket says 172.21.15.47:55436) 2024-03-21T11:40:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4d3e00640 1 -- 172.21.15.47:0/103145059 learned_addr learned my addr 172.21.15.47:0/103145059 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4d3400640 1 --2- 172.21.15.47:0/103145059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b75050 0x55a774b84150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4d3e00640 1 -- 172.21.15.47:0/103145059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b75050 msgr2=0x55a774b84150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4d3e00640 1 --2- 172.21.15.47:0/103145059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b75050 0x55a774b84150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4d3e00640 1 -- 172.21.15.47:0/103145059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75a40 msgr2=0x55a774b84690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:23.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4d3e00640 1 --2- 172.21.15.47:0/103145059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75a40 0x55a774b84690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.208+0000 7fe4d3e00640 1 -- 172.21.15.47:0/103145059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a774b85bf0 con 0x55a774b7abb0 2024-03-21T11:40:23.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.209+0000 7fe4d3e00640 1 --2- 172.21.15.47:0/103145059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b7abb0 0x55a774b85600 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40339b0 tx=0x7fe4c402c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a3b2ec1fe0978b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:23.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.209+0000 7fe4c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:23.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.209+0000 7fe4c0c00640 1 -- 172.21.15.47:0/103145059 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c404c040 con 0x55a774b7abb0 2024-03-21T11:40:23.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.209+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/103145059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4c4003970 con 0x55a774b7abb0 2024-03-21T11:40:23.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.209+0000 7fe4c0c00640 1 -- 172.21.15.47:0/103145059 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe4c4034430 con 0x55a774b7abb0 2024-03-21T11:40:23.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.209+0000 7fe4c0c00640 1 -- 172.21.15.47:0/103145059 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4c4060050 con 0x55a774b7abb0 2024-03-21T11:40:23.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.209+0000 7fe4dcfd4740 10 client.31139 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:23.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.209+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/103145059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a774bef3f0 con 0x55a774b7abb0 2024-03-21T11:40:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.210+0000 7fe4c0c00640 1 -- 172.21.15.47:0/103145059 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4c4003970 con 0x55a774b7abb0 2024-03-21T11:40:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.210+0000 7fe4c0c00640 1 -- 172.21.15.47:0/103145059 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe4c4051890 con 0x55a774b7abb0 2024-03-21T11:40:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.210+0000 7fe4dcfd4740 10 client.31139 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.210+0000 7fe4dcfd4740 10 client.31139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.210+0000 7fe4dcfd4740 20 client.31139 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.210+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/103145059 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a774bef8c0 0x55a774bf1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.210+0000 7fe4dcfd4740 4 client.31139 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:40:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.210+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/103145059 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a774ae1ae0 con 0x55a774bef8c0 2024-03-21T11:40:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.210+0000 7fe4d3400640 1 --2- 172.21.15.47:0/103145059 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a774bef8c0 0x55a774bf1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.211+0000 7fe4d3400640 1 --2- 172.21.15.47:0/103145059 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a774bef8c0 0x55a774bf1ce0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=481b039a2aa82913 server_cookie=cf48f2f93f6f0c16 in_seq=0 out_seq=0 2024-03-21T11:40:23.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.211+0000 7fe4c0c00640 10 client.31139 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.215+0000 7fe4c0c00640 1 -- 172.21.15.47:0/103145059 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55a774ae1ae0 con 0x55a774bef8c0 2024-03-21T11:40:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.215+0000 7fe4c0c00640 10 client.31139 handle_command_reply: tid=0 2024-03-21T11:40:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.215+0000 7fe4dcfd4740 1 client.31139 shutdown 2024-03-21T11:40:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.215+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/103145059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b7abb0 msgr2=0x55a774b85600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.215+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/103145059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b7abb0 0x55a774b85600 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40339b0 tx=0x7fe4c402c4b0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.215+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/103145059 shutdown_connections 2024-03-21T11:40:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.215+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/103145059 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a774bef8c0 0x55a774bf1ce0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.215+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/103145059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a774b7abb0 0x55a774b85600 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.215+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/103145059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a774b75a40 0x55a774b84690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.215+0000 7fe4c0c00640 0 client.31139 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.215+0000 7fe4dcfd4740 1 --2- 172.21.15.47:0/103145059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a774b75050 0x55a774b84150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.215+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/103145059 >> 172.21.15.47:0/103145059 conn(0x55a774b3d200 msgr2=0x55a774b7f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:23.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.218+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/103145059 shutdown_connections 2024-03-21T11:40:23.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.218+0000 7fe4dcfd4740 1 -- 172.21.15.47:0/103145059 wait complete. 2024-03-21T11:40:23.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.218+0000 7fe4dcfd4740 20 client.31139 trim_cache size 0 max 16384 2024-03-21T11:40:23.219 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:40:23.219 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (182 inodes in the stack)", 2024-03-21T11:40:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:40:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: "939550e2-576c-4604-86a6-3a4f4923bdd5": { 2024-03-21T11:40:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:40:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "939550e2-576c-4604-86a6-3a4f4923bdd5", 2024-03-21T11:40:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:40:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:40:23.220 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:40:23.220 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:40:23.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.219+0000 7fe4d8c00640 1 -- 172.21.15.47:0/331411895 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4b80a2fa0 msgr2=0x7fe4b80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:23.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.219+0000 7fe4d8c00640 1 --2- 172.21.15.47:0/331411895 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4b80a2fa0 0x7fe4b80a5460 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc005fd0 tx=0x7fe4bc009260 comp rx=0 tx=0).stop 2024-03-21T11:40:23.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.219+0000 7fe4d8c00640 1 -- 172.21.15.47:0/331411895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d41049b0 msgr2=0x7fe4d4199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:23.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.219+0000 7fe4d8c00640 1 --2- 172.21.15.47:0/331411895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d41049b0 0x7fe4d4199f80 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00ea60 tx=0x7fe4cc00ee60 comp rx=0 tx=0).stop 2024-03-21T11:40:23.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.221+0000 7fe4d8c00640 1 -- 172.21.15.47:0/331411895 shutdown_connections 2024-03-21T11:40:23.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.221+0000 7fe4d8c00640 1 --2- 172.21.15.47:0/331411895 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4b80a2fa0 0x7fe4b80a5460 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.221+0000 7fe4d8c00640 1 --2- 172.21.15.47:0/331411895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d4105370 0x7fe4d41a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.221+0000 7fe4d8c00640 1 --2- 172.21.15.47:0/331411895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4d41049b0 0x7fe4d4199f80 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.221+0000 7fe4d8c00640 1 --2- 172.21.15.47:0/331411895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d41037c0 0x7fe4d4199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:23.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.221+0000 7fe4d8c00640 1 -- 172.21.15.47:0/331411895 >> 172.21.15.47:0/331411895 conn(0x7fe4d40fed50 msgr2=0x7fe4d4107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:23.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.221+0000 7fe4d8c00640 1 -- 172.21.15.47:0/331411895 shutdown_connections 2024-03-21T11:40:23.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:23.221+0000 7fe4d8c00640 1 -- 172.21.15.47:0/331411895 wait complete. 2024-03-21T11:40:23.243 INFO:tasks.cephfs.filesystem:scrub status for tag:939550e2-576c-4604-86a6-3a4f4923bdd5 - {'path': '/', 'tag': '939550e2-576c-4604-86a6-3a4f4923bdd5', 'options': 'recursive,force'} 2024-03-21T11:40:23.942 INFO:tasks.workunit.client.0.smithi047.stdout: 10 89398 24.84 MB/sec execute 67 sec latency 157.123 ms 2024-03-21T11:40:23.993 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:40:23.993 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:23.993 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:23.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:23.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:23.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:23.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:23.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:23.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:23.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:23.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:23.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:23.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:23.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:23.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:23.994 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:23.995 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:23.995 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:23.995 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:23 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:40:24.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:24.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:24.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:24.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:24.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:24.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:24.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:24.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:24.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:24.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:23 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:40:24.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:24.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:24.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:24.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:24.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:24.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:24.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:24.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:24.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:24.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:23 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.942 INFO:tasks.workunit.client.0.smithi047.stdout: 10 89933 25.00 MB/sec execute 68 sec latency 116.549 ms 2024-03-21T11:40:24.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:24 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:24.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:24 smithi102 ceph-mon[36748]: pgmap v912: 129 pgs: 129 active+clean; 418 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.0 MiB/s wr, 681 op/s 2024-03-21T11:40:25.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:24 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:25.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:24 smithi047 ceph-mon[31351]: pgmap v912: 129 pgs: 129 active+clean; 418 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.0 MiB/s wr, 681 op/s 2024-03-21T11:40:25.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:24 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:25.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:24 smithi100 ceph-mon[36245]: pgmap v912: 129 pgs: 129 active+clean; 418 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.0 MiB/s wr, 681 op/s 2024-03-21T11:40:25.942 INFO:tasks.workunit.client.0.smithi047.stdout: 10 90510 25.05 MB/sec execute 69 sec latency 78.445 ms 2024-03-21T11:40:26.942 INFO:tasks.workunit.client.0.smithi047.stdout: 10 90790 24.89 MB/sec execute 70 sec latency 208.021 ms 2024-03-21T11:40:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:26 smithi102 ceph-mon[36748]: pgmap v913: 129 pgs: 129 active+clean; 418 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 640 op/s 2024-03-21T11:40:27.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:26 smithi047 ceph-mon[31351]: pgmap v913: 129 pgs: 129 active+clean; 418 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 640 op/s 2024-03-21T11:40:27.067 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:26 smithi100 ceph-mon[36245]: pgmap v913: 129 pgs: 129 active+clean; 418 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.2 MiB/s wr, 640 op/s 2024-03-21T11:40:27.942 INFO:tasks.workunit.client.0.smithi047.stdout: 10 91145 24.86 MB/sec execute 71 sec latency 235.468 ms 2024-03-21T11:40:28.942 INFO:tasks.workunit.client.0.smithi047.stdout: 10 91502 24.77 MB/sec execute 72 sec latency 146.144 ms 2024-03-21T11:40:28.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:28 smithi102 ceph-mon[36748]: pgmap v914: 129 pgs: 129 active+clean; 420 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 637 op/s 2024-03-21T11:40:29.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:28 smithi047 ceph-mon[31351]: pgmap v914: 129 pgs: 129 active+clean; 420 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 637 op/s 2024-03-21T11:40:29.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:28 smithi100 ceph-mon[36245]: pgmap v914: 129 pgs: 129 active+clean; 420 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 637 op/s 2024-03-21T11:40:29.942 INFO:tasks.workunit.client.0.smithi047.stdout: 10 92117 24.83 MB/sec execute 73 sec latency 163.327 ms 2024-03-21T11:40:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:29.999+0000 7fc620000640 1 -- 172.21.15.47:0/2505468337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc618103900 msgr2=0x7fc618103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:29.999+0000 7fc620000640 1 --2- 172.21.15.47:0/2505468337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc618103900 0x7fc618103d20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009900 tx=0x7fc60c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:40:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:29.999+0000 7fc620000640 1 -- 172.21.15.47:0/2505468337 shutdown_connections 2024-03-21T11:40:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:29.999+0000 7fc620000640 1 --2- 172.21.15.47:0/2505468337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6181054b0 0x7fc61810bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:29.999+0000 7fc620000640 1 --2- 172.21.15.47:0/2505468337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc618104af0 0x7fc618104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:29.999+0000 7fc620000640 1 --2- 172.21.15.47:0/2505468337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc618103900 0x7fc618103d20 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:29.999+0000 7fc620000640 1 -- 172.21.15.47:0/2505468337 >> 172.21.15.47:0/2505468337 conn(0x7fc6180fee90 msgr2=0x7fc6181012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:29.999+0000 7fc620000640 1 -- 172.21.15.47:0/2505468337 shutdown_connections 2024-03-21T11:40:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:29.999+0000 7fc620000640 1 -- 172.21.15.47:0/2505468337 wait complete. 2024-03-21T11:40:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.000+0000 7fc620000640 1 Processor -- start 2024-03-21T11:40:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.000+0000 7fc620000640 1 -- start start 2024-03-21T11:40:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.000+0000 7fc620000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc618103900 0x7fc618199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.000+0000 7fc620000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc618104af0 0x7fc61819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.000+0000 7fc620000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6181054b0 0x7fc6181a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.000+0000 7fc620000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61810f5c0 con 0x7fc618103900 2024-03-21T11:40:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.000+0000 7fc620000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61810f440 con 0x7fc6181054b0 2024-03-21T11:40:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.000+0000 7fc620000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61810f740 con 0x7fc618104af0 2024-03-21T11:40:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.001+0000 7fc61cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc618104af0 0x7fc61819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.001+0000 7fc61d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc618103900 0x7fc618199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.001+0000 7fc61e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6181054b0 0x7fc6181a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.001+0000 7fc61cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc618104af0 0x7fc61819a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57370/0 (socket says 172.21.15.47:57370) 2024-03-21T11:40:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.001+0000 7fc61cc00640 1 -- 172.21.15.47:0/2568568369 learned_addr learned my addr 172.21.15.47:0/2568568369 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.001+0000 7fc61cc00640 1 -- 172.21.15.47:0/2568568369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6181054b0 msgr2=0x7fc6181a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.001+0000 7fc61cc00640 1 --2- 172.21.15.47:0/2568568369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6181054b0 0x7fc6181a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.001+0000 7fc61cc00640 1 -- 172.21.15.47:0/2568568369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc618103900 msgr2=0x7fc618199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.001+0000 7fc61cc00640 1 --2- 172.21.15.47:0/2568568369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc618103900 0x7fc618199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.001+0000 7fc61cc00640 1 -- 172.21.15.47:0/2568568369 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6181a1810 con 0x7fc618104af0 2024-03-21T11:40:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.001+0000 7fc61cc00640 1 --2- 172.21.15.47:0/2568568369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc618104af0 0x7fc61819a130 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc60800e9f0 tx=0x7fc60800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.002+0000 7fc616000640 1 -- 172.21.15.47:0/2568568369 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60800cd80 con 0x7fc618104af0 2024-03-21T11:40:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.002+0000 7fc616000640 1 -- 172.21.15.47:0/2568568369 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc608005500 con 0x7fc618104af0 2024-03-21T11:40:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.002+0000 7fc620000640 1 -- 172.21.15.47:0/2568568369 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6181a1b00 con 0x7fc618104af0 2024-03-21T11:40:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.002+0000 7fc616000640 1 -- 172.21.15.47:0/2568568369 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc608010690 con 0x7fc618104af0 2024-03-21T11:40:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.002+0000 7fc620000640 1 -- 172.21.15.47:0/2568568369 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6181a2050 con 0x7fc618104af0 2024-03-21T11:40:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.005+0000 7fc622dab740 1 Processor -- start 2024-03-21T11:40:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.005+0000 7fc622dab740 1 -- start start 2024-03-21T11:40:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.005+0000 7fc622dab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0b5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc622dab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc622dab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0bad40 0x55e7ed0bd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc616000640 1 -- 172.21.15.47:0/2568568369 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc6080046c0 con 0x7fc618104af0 2024-03-21T11:40:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc622dab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ed0783b0 con 0x55e7ed0bad40 2024-03-21T11:40:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc622dab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ed01a830 con 0x55e7ed0b5bd0 2024-03-21T11:40:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc622dab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ecfc35d0 con 0x55e7ed0b51e0 2024-03-21T11:40:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc61e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc61d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc61e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57384/0 (socket says 172.21.15.47:57384) 2024-03-21T11:40:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc61d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0ba6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35842/0 (socket says 172.21.15.47:35842) 2024-03-21T11:40:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc61e000640 1 -- 172.21.15.47:0/1677629690 learned_addr learned my addr 172.21.15.47:0/1677629690 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc61cc00640 1 --2- 172.21.15.47:0/1677629690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0bad40 0x55e7ed0bd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc61d600640 1 -- 172.21.15.47:0/1677629690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 msgr2=0x55e7ed0b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc61d600640 1 --2- 172.21.15.47:0/1677629690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc61d600640 1 -- 172.21.15.47:0/1677629690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0bad40 msgr2=0x55e7ed0bd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc61d600640 1 --2- 172.21.15.47:0/1677629690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0bad40 0x55e7ed0bd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.006+0000 7fc61d600640 1 -- 172.21.15.47:0/1677629690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7ecfdb2c0 con 0x55e7ed0b5bd0 2024-03-21T11:40:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.007+0000 7fc61d600640 1 --2- 172.21.15.47:0/1677629690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0ba6d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009230 tx=0x7fc60c02c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d42ad0fd3d5f8c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.007+0000 7fc606600640 1 -- 172.21.15.47:0/1677629690 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c02c980 con 0x55e7ed0b5bd0 2024-03-21T11:40:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.007+0000 7fc606600640 1 -- 172.21.15.47:0/1677629690 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc60c02cb20 con 0x55e7ed0b5bd0 2024-03-21T11:40:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.007+0000 7fc606600640 1 -- 172.21.15.47:0/1677629690 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c004920 con 0x55e7ed0b5bd0 2024-03-21T11:40:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.007+0000 7fc616000640 1 --2- 172.21.15.47:0/2568568369 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc6000a3040 0x7fc6000a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.008+0000 7fc616000640 1 -- 172.21.15.47:0/2568568369 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc6080cfb40 con 0x7fc618104af0 2024-03-21T11:40:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.008+0000 7fc622dab740 1 -- 172.21.15.47:0/1677629690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b5bd0 msgr2=0x55e7ed0ba6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.008+0000 7fc622dab740 1 --2- 172.21.15.47:0/1677629690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0ba6d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009230 tx=0x7fc60c02c670 comp rx=0 tx=0).stop 2024-03-21T11:40:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.008+0000 7fc61e000640 1 --2- 172.21.15.47:0/2568568369 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc6000a3040 0x7fc6000a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.008+0000 7fc622dab740 1 -- 172.21.15.47:0/1677629690 shutdown_connections 2024-03-21T11:40:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.008+0000 7fc622dab740 1 --2- 172.21.15.47:0/1677629690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0bad40 0x55e7ed0bd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.008+0000 7fc622dab740 1 --2- 172.21.15.47:0/1677629690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0ba6d0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.008+0000 7fc622dab740 1 --2- 172.21.15.47:0/1677629690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.008+0000 7fc622dab740 1 -- 172.21.15.47:0/1677629690 >> 172.21.15.47:0/1677629690 conn(0x55e7ed0b0750 msgr2=0x55e7ed0b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.008+0000 7fc622dab740 1 -- 172.21.15.47:0/1677629690 shutdown_connections 2024-03-21T11:40:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.009+0000 7fc622dab740 1 -- 172.21.15.47:0/1677629690 wait complete. 2024-03-21T11:40:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.009+0000 7fc61e000640 1 --2- 172.21.15.47:0/2568568369 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc6000a3040 0x7fc6000a5500 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x55e7ed0b6430 tx=0x7fc610009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.009+0000 7fc622dab740 1 -- start start 2024-03-21T11:40:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.009+0000 7fc622dab740 1 Processor -- start 2024-03-21T11:40:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.010+0000 7fc622dab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0bf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.010+0000 7fc622dab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.010+0000 7fc622dab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0bad40 0x55e7ed0c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.010+0000 7fc622dab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ecfd6a20 con 0x55e7ed0b5bd0 2024-03-21T11:40:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.010+0000 7fc622dab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ed0168d0 con 0x55e7ed0bad40 2024-03-21T11:40:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.010+0000 7fc622dab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ed0bdc90 con 0x55e7ed0b51e0 2024-03-21T11:40:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.010+0000 7fc622dab740 10 client.0 _ll_register_callbacks cb 0x55e7ed0131c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.010+0000 7fc61d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0bf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.010+0000 7fc61d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0bf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57398/0 (socket says 172.21.15.47:57398) 2024-03-21T11:40:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.010+0000 7fc61d600640 1 -- 172.21.15.47:0/2511670565 learned_addr learned my addr 172.21.15.47:0/2511670565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.010+0000 7fc61e000640 1 --2- 172.21.15.47:0/2511670565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0bad40 0x55e7ed0c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.010+0000 7fc61cc00640 1 --2- 172.21.15.47:0/2511670565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.010+0000 7fc61d600640 1 -- 172.21.15.47:0/2511670565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0bad40 msgr2=0x55e7ed0c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.010+0000 7fc61d600640 1 --2- 172.21.15.47:0/2511670565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0bad40 0x55e7ed0c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.011+0000 7fc61d600640 1 -- 172.21.15.47:0/2511670565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0b5bd0 msgr2=0x55e7ed0bf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.011+0000 7fc61d600640 1 --2- 172.21.15.47:0/2511670565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0bf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.011+0000 7fc61d600640 1 -- 172.21.15.47:0/2511670565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7ecd2f210 con 0x55e7ed0b51e0 2024-03-21T11:40:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.011+0000 7fc61d600640 1 --2- 172.21.15.47:0/2511670565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0bf390 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009900 tx=0x7fc60c02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b478cf87fd246f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.011+0000 7fc5faa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:40:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.011+0000 7fc5faa00640 1 -- 172.21.15.47:0/2511670565 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c050070 con 0x55e7ed0b51e0 2024-03-21T11:40:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.011+0000 7fc5faa00640 1 -- 172.21.15.47:0/2511670565 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc60c003a00 con 0x55e7ed0b51e0 2024-03-21T11:40:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.011+0000 7fc622dab740 1 -- 172.21.15.47:0/2511670565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc60c02cb20 con 0x55e7ed0b51e0 2024-03-21T11:40:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.012+0000 7fc5faa00640 1 -- 172.21.15.47:0/2511670565 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc60c05c050 con 0x55e7ed0b51e0 2024-03-21T11:40:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.012+0000 7fc622dab740 10 client.31129 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.012+0000 7fc622dab740 1 -- 172.21.15.47:0/2511670565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e7ecdfb620 con 0x55e7ed0b51e0 2024-03-21T11:40:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.012+0000 7fc5faa00640 1 -- 172.21.15.47:0/2511670565 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc60c02cb20 con 0x55e7ed0b51e0 2024-03-21T11:40:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.012+0000 7fc5faa00640 1 -- 172.21.15.47:0/2511670565 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc60c03fa00 con 0x55e7ed0b51e0 2024-03-21T11:40:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.012+0000 7fc622dab740 10 client.31129 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.012+0000 7fc622dab740 10 client.31129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.012+0000 7fc622dab740 20 client.31129 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.013+0000 7fc622dab740 1 --2- 172.21.15.47:0/2511670565 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e7ed0cae00 0x55e7ed0cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.013+0000 7fc622dab740 4 client.31129 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.013+0000 7fc622dab740 1 -- 172.21.15.47:0/2511670565 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7ed027d70 con 0x55e7ed0cae00 2024-03-21T11:40:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.013+0000 7fc61cc00640 1 --2- 172.21.15.47:0/2511670565 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e7ed0cae00 0x55e7ed0cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.013+0000 7fc61cc00640 1 --2- 172.21.15.47:0/2511670565 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e7ed0cae00 0x55e7ed0cd1f0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54c6646706bb1854 server_cookie=84ac9c22adce23c in_seq=0 out_seq=0 2024-03-21T11:40:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.013+0000 7fc5faa00640 10 client.31129 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.017+0000 7fc5faa00640 1 -- 172.21.15.47:0/2511670565 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e7ed027d70 con 0x55e7ed0cae00 2024-03-21T11:40:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.017+0000 7fc5faa00640 10 client.31129 handle_command_reply: tid=0 2024-03-21T11:40:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.017+0000 7fc622dab740 1 client.31129 shutdown 2024-03-21T11:40:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.018+0000 7fc622dab740 1 -- 172.21.15.47:0/2511670565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 msgr2=0x55e7ed0bf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.018+0000 7fc622dab740 1 --2- 172.21.15.47:0/2511670565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0bf390 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009900 tx=0x7fc60c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:40:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.018+0000 7fc622dab740 1 -- 172.21.15.47:0/2511670565 shutdown_connections 2024-03-21T11:40:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.018+0000 7fc622dab740 1 --2- 172.21.15.47:0/2511670565 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e7ed0cae00 0x55e7ed0cd1f0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.018+0000 7fc622dab740 1 --2- 172.21.15.47:0/2511670565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0bad40 0x55e7ed0c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.018+0000 7fc622dab740 1 --2- 172.21.15.47:0/2511670565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0bf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.018+0000 7fc622dab740 1 --2- 172.21.15.47:0/2511670565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0bf390 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.018+0000 7fc5faa00640 0 client.31129 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.018+0000 7fc622dab740 1 -- 172.21.15.47:0/2511670565 >> 172.21.15.47:0/2511670565 conn(0x55e7ed0b0750 msgr2=0x55e7ed0af950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.018+0000 7fc622dab740 1 -- 172.21.15.47:0/2511670565 shutdown_connections 2024-03-21T11:40:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.018+0000 7fc622dab740 1 -- 172.21.15.47:0/2511670565 wait complete. 2024-03-21T11:40:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.018+0000 7fc622dab740 20 client.31129 trim_cache size 0 max 16384 2024-03-21T11:40:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.038+0000 7fc622dab740 1 Processor -- start 2024-03-21T11:40:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.038+0000 7fc622dab740 1 -- start start 2024-03-21T11:40:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.039+0000 7fc622dab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.039+0000 7fc622dab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.039+0000 7fc622dab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0bad40 0x55e7ed0afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.039+0000 7fc622dab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6181ac1b0 con 0x55e7ed0bad40 2024-03-21T11:40:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.039+0000 7fc622dab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6181adb30 con 0x55e7ed0b5bd0 2024-03-21T11:40:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.039+0000 7fc622dab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6181ac810 con 0x55e7ed0b51e0 2024-03-21T11:40:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.039+0000 7fc61d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.039+0000 7fc61d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57414/0 (socket says 172.21.15.47:57414) 2024-03-21T11:40:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.039+0000 7fc61d600640 1 -- 172.21.15.47:0/2631459660 learned_addr learned my addr 172.21.15.47:0/2631459660 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.039+0000 7fc61cc00640 1 --2- 172.21.15.47:0/2631459660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.039+0000 7fc61e000640 1 --2- 172.21.15.47:0/2631459660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0bad40 0x55e7ed0afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.040+0000 7fc61d600640 1 -- 172.21.15.47:0/2631459660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b5bd0 msgr2=0x55e7ed0af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.040+0000 7fc61d600640 1 --2- 172.21.15.47:0/2631459660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0af600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.040+0000 7fc61d600640 1 -- 172.21.15.47:0/2631459660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0bad40 msgr2=0x55e7ed0afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.040+0000 7fc61d600640 1 --2- 172.21.15.47:0/2631459660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0bad40 0x55e7ed0afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.040+0000 7fc61d600640 1 -- 172.21.15.47:0/2631459660 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7ed0b0080 con 0x55e7ed0b51e0 2024-03-21T11:40:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.040+0000 7fc61e000640 1 --2- 172.21.15.47:0/2631459660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0bad40 0x55e7ed0afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:40:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.040+0000 7fc61cc00640 1 --2- 172.21.15.47:0/2631459660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.040+0000 7fc61d600640 1 --2- 172.21.15.47:0/2631459660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0af0c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc60c02c240 tx=0x7fc60c009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdc14952e2680e89 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.040+0000 7fc605c00640 1 -- 172.21.15.47:0/2631459660 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c050070 con 0x55e7ed0b51e0 2024-03-21T11:40:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.040+0000 7fc605c00640 1 -- 172.21.15.47:0/2631459660 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc60c034440 con 0x55e7ed0b51e0 2024-03-21T11:40:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.040+0000 7fc605c00640 1 -- 172.21.15.47:0/2631459660 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c004c80 con 0x55e7ed0b51e0 2024-03-21T11:40:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.041+0000 7fc622dab740 1 -- 172.21.15.47:0/2631459660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 msgr2=0x55e7ed0af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.041+0000 7fc622dab740 1 --2- 172.21.15.47:0/2631459660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0af0c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc60c02c240 tx=0x7fc60c009520 comp rx=0 tx=0).stop 2024-03-21T11:40:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.042+0000 7fc622dab740 1 -- 172.21.15.47:0/2631459660 shutdown_connections 2024-03-21T11:40:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.042+0000 7fc622dab740 1 --2- 172.21.15.47:0/2631459660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0bad40 0x55e7ed0afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.042+0000 7fc622dab740 1 --2- 172.21.15.47:0/2631459660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.042+0000 7fc622dab740 1 --2- 172.21.15.47:0/2631459660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0af0c0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.042+0000 7fc622dab740 1 -- 172.21.15.47:0/2631459660 >> 172.21.15.47:0/2631459660 conn(0x55e7ed0b0750 msgr2=0x55e7ed0b6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.042+0000 7fc622dab740 1 -- 172.21.15.47:0/2631459660 shutdown_connections 2024-03-21T11:40:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.042+0000 7fc622dab740 1 -- 172.21.15.47:0/2631459660 wait complete. 2024-03-21T11:40:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.042+0000 7fc622dab740 1 -- start start 2024-03-21T11:40:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.043+0000 7fc622dab740 1 Processor -- start 2024-03-21T11:40:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.043+0000 7fc622dab740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.043+0000 7fc622dab740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0c4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.043+0000 7fc622dab740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0bad40 0x55e7ed0c5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.043+0000 7fc622dab740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6181ad4d0 con 0x55e7ed0b5bd0 2024-03-21T11:40:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.044+0000 7fc622dab740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6181ace70 con 0x55e7ed0b51e0 2024-03-21T11:40:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.044+0000 7fc622dab740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6181aa370 con 0x55e7ed0bad40 2024-03-21T11:40:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.044+0000 7fc622dab740 10 client.0 _ll_register_callbacks cb 0x7fc610009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.044+0000 7fc61d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.044+0000 7fc61e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0bad40 0x55e7ed0c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.044+0000 7fc61d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0c3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35860/0 (socket says 172.21.15.47:35860) 2024-03-21T11:40:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.044+0000 7fc61d600640 1 -- 172.21.15.47:0/1511615151 learned_addr learned my addr 172.21.15.47:0/1511615151 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.044+0000 7fc61cc00640 1 --2- 172.21.15.47:0/1511615151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.044+0000 7fc61d600640 1 -- 172.21.15.47:0/1511615151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0bad40 msgr2=0x55e7ed0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.044+0000 7fc61d600640 1 --2- 172.21.15.47:0/1511615151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0bad40 0x55e7ed0c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.044+0000 7fc61d600640 1 -- 172.21.15.47:0/1511615151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0b5bd0 msgr2=0x55e7ed0c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.044+0000 7fc61d600640 1 --2- 172.21.15.47:0/1511615151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.044+0000 7fc61d600640 1 -- 172.21.15.47:0/1511615151 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7ed0c5810 con 0x55e7ed0b51e0 2024-03-21T11:40:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.045+0000 7fc61d600640 1 --2- 172.21.15.47:0/1511615151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0c3cd0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009900 tx=0x7fc60c009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8c0e66ffdb0a6aa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.045+0000 7fc5faa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.045+0000 7fc5faa00640 1 -- 172.21.15.47:0/1511615151 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c050070 con 0x55e7ed0b51e0 2024-03-21T11:40:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.045+0000 7fc5faa00640 1 -- 172.21.15.47:0/1511615151 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc60c003800 con 0x55e7ed0b51e0 2024-03-21T11:40:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.045+0000 7fc622dab740 1 -- 172.21.15.47:0/1511615151 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc60c034440 con 0x55e7ed0b51e0 2024-03-21T11:40:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.045+0000 7fc5faa00640 1 -- 172.21.15.47:0/1511615151 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc60c004e10 con 0x55e7ed0b51e0 2024-03-21T11:40:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.045+0000 7fc622dab740 10 client.31157 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.045+0000 7fc622dab740 1 -- 172.21.15.47:0/1511615151 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e7ed126ac0 con 0x55e7ed0b51e0 2024-03-21T11:40:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.046+0000 7fc5faa00640 1 -- 172.21.15.47:0/1511615151 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc60c034440 con 0x55e7ed0b51e0 2024-03-21T11:40:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.046+0000 7fc5faa00640 1 -- 172.21.15.47:0/1511615151 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc60c034670 con 0x55e7ed0b51e0 2024-03-21T11:40:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.046+0000 7fc622dab740 10 client.31157 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.046+0000 7fc622dab740 10 client.31157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.046+0000 7fc622dab740 20 client.31157 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.046+0000 7fc622dab740 1 --2- 172.21.15.47:0/1511615151 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e7ed1270a0 0x55e7ed129490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.046+0000 7fc622dab740 4 client.31157 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:40:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.046+0000 7fc622dab740 1 -- 172.21.15.47:0/1511615151 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e7ed0220c0 con 0x55e7ed1270a0 2024-03-21T11:40:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.046+0000 7fc61cc00640 1 --2- 172.21.15.47:0/1511615151 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e7ed1270a0 0x55e7ed129490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.047+0000 7fc61cc00640 1 --2- 172.21.15.47:0/1511615151 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e7ed1270a0 0x55e7ed129490 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d585cc2b302ab296 server_cookie=7741314bd513e68a in_seq=0 out_seq=0 2024-03-21T11:40:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.047+0000 7fc5faa00640 10 client.31157 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.735+0000 7fc5faa00640 1 -- 172.21.15.47:0/1511615151 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e7ed0220c0 con 0x55e7ed1270a0 2024-03-21T11:40:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.735+0000 7fc5faa00640 10 client.31157 handle_command_reply: tid=0 2024-03-21T11:40:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.735+0000 7fc622dab740 1 client.31157 shutdown 2024-03-21T11:40:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.735+0000 7fc622dab740 1 -- 172.21.15.47:0/1511615151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b51e0 msgr2=0x55e7ed0c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.735+0000 7fc622dab740 1 --2- 172.21.15.47:0/1511615151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0c3cd0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009900 tx=0x7fc60c009680 comp rx=0 tx=0).stop 2024-03-21T11:40:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.735+0000 7fc622dab740 1 -- 172.21.15.47:0/1511615151 shutdown_connections 2024-03-21T11:40:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.735+0000 7fc622dab740 1 --2- 172.21.15.47:0/1511615151 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e7ed1270a0 0x55e7ed129490 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.735+0000 7fc622dab740 1 --2- 172.21.15.47:0/1511615151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ed0bad40 0x55e7ed0c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.735+0000 7fc622dab740 1 --2- 172.21.15.47:0/1511615151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e7ed0b5bd0 0x55e7ed0c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.735+0000 7fc622dab740 1 --2- 172.21.15.47:0/1511615151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e7ed0b51e0 0x55e7ed0c3cd0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.735+0000 7fc5faa00640 0 client.31157 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.735+0000 7fc622dab740 1 -- 172.21.15.47:0/1511615151 >> 172.21.15.47:0/1511615151 conn(0x55e7ed0b0750 msgr2=0x55e7ed0af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.736+0000 7fc622dab740 1 -- 172.21.15.47:0/1511615151 shutdown_connections 2024-03-21T11:40:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.736+0000 7fc622dab740 1 -- 172.21.15.47:0/1511615151 wait complete. 2024-03-21T11:40:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.736+0000 7fc622dab740 20 client.31157 trim_cache size 0 max 16384 2024-03-21T11:40:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:40:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:40:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:40:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:40:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.737+0000 7fc620000640 1 -- 172.21.15.47:0/2568568369 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc6000a3040 msgr2=0x7fc6000a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.737+0000 7fc620000640 1 --2- 172.21.15.47:0/2568568369 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc6000a3040 0x7fc6000a5500 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x55e7ed0b6430 tx=0x7fc610009500 comp rx=0 tx=0).stop 2024-03-21T11:40:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.737+0000 7fc620000640 1 -- 172.21.15.47:0/2568568369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc618104af0 msgr2=0x7fc61819a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.737+0000 7fc620000640 1 --2- 172.21.15.47:0/2568568369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc618104af0 0x7fc61819a130 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc60800e9f0 tx=0x7fc60800edf0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.737+0000 7fc620000640 1 -- 172.21.15.47:0/2568568369 shutdown_connections 2024-03-21T11:40:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.737+0000 7fc620000640 1 --2- 172.21.15.47:0/2568568369 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc6000a3040 0x7fc6000a5500 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.737+0000 7fc620000640 1 --2- 172.21.15.47:0/2568568369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6181054b0 0x7fc6181a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.737+0000 7fc620000640 1 --2- 172.21.15.47:0/2568568369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc618104af0 0x7fc61819a130 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.737+0000 7fc620000640 1 --2- 172.21.15.47:0/2568568369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc618103900 0x7fc618199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.737+0000 7fc620000640 1 -- 172.21.15.47:0/2568568369 >> 172.21.15.47:0/2568568369 conn(0x7fc6180fee90 msgr2=0x7fc618107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.738+0000 7fc620000640 1 -- 172.21.15.47:0/2568568369 shutdown_connections 2024-03-21T11:40:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:30.738+0000 7fc620000640 1 -- 172.21.15.47:0/2568568369 wait complete. 2024-03-21T11:40:30.942 INFO:tasks.workunit.client.0.smithi047.stdout: 10 92368 24.76 MB/sec execute 74 sec latency 354.701 ms 2024-03-21T11:40:30.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:30 smithi102 ceph-mon[36748]: pgmap v915: 129 pgs: 129 active+clean; 425 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 770 op/s 2024-03-21T11:40:31.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:30 smithi047 ceph-mon[31351]: pgmap v915: 129 pgs: 129 active+clean; 425 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 770 op/s 2024-03-21T11:40:31.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:30 smithi100 ceph-mon[36245]: pgmap v915: 129 pgs: 129 active+clean; 425 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 770 op/s 2024-03-21T11:40:31.943 INFO:tasks.workunit.client.0.smithi047.stdout: 10 92903 24.81 MB/sec execute 75 sec latency 56.428 ms 2024-03-21T11:40:32.943 INFO:tasks.workunit.client.0.smithi047.stdout: 10 93213 24.71 MB/sec execute 76 sec latency 241.785 ms 2024-03-21T11:40:32.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:32 smithi102 ceph-mon[36748]: pgmap v916: 129 pgs: 129 active+clean; 427 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 545 op/s 2024-03-21T11:40:33.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:32 smithi047 ceph-mon[31351]: pgmap v916: 129 pgs: 129 active+clean; 427 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 545 op/s 2024-03-21T11:40:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:32 smithi100 ceph-mon[36245]: pgmap v916: 129 pgs: 129 active+clean; 427 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 545 op/s 2024-03-21T11:40:33.943 INFO:tasks.workunit.client.0.smithi047.stdout: 10 93414 24.53 MB/sec execute 77 sec latency 344.404 ms 2024-03-21T11:40:34.943 INFO:tasks.workunit.client.0.smithi047.stdout: 10 93903 24.59 MB/sec execute 78 sec latency 123.701 ms 2024-03-21T11:40:34.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:34 smithi102 ceph-mon[36748]: pgmap v917: 129 pgs: 129 active+clean; 425 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.9 MiB/s wr, 679 op/s 2024-03-21T11:40:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:34 smithi047 ceph-mon[31351]: pgmap v917: 129 pgs: 129 active+clean; 425 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.9 MiB/s wr, 679 op/s 2024-03-21T11:40:35.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:34 smithi100 ceph-mon[36245]: pgmap v917: 129 pgs: 129 active+clean; 425 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.9 MiB/s wr, 679 op/s 2024-03-21T11:40:35.943 INFO:tasks.workunit.client.0.smithi047.stdout: 10 94626 24.57 MB/sec execute 79 sec latency 60.315 ms 2024-03-21T11:40:36.944 INFO:tasks.workunit.client.0.smithi047.stdout: 10 95212 24.76 MB/sec execute 80 sec latency 55.237 ms 2024-03-21T11:40:36.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:36 smithi102 ceph-mon[36748]: pgmap v918: 129 pgs: 129 active+clean; 427 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 622 op/s 2024-03-21T11:40:37.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:36 smithi047 ceph-mon[31351]: pgmap v918: 129 pgs: 129 active+clean; 427 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 622 op/s 2024-03-21T11:40:37.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:36 smithi100 ceph-mon[36245]: pgmap v918: 129 pgs: 129 active+clean; 427 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 622 op/s 2024-03-21T11:40:37.946 INFO:tasks.workunit.client.0.smithi047.stdout: 10 95726 24.94 MB/sec execute 81 sec latency 129.589 ms 2024-03-21T11:40:37.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:37 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:40:37.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:37 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:40:37.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:37 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:40:38.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:37 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:40:38.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:37 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:40:38.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:37 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:40:38.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:37 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:40:38.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:37 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:40:38.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:37 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:40:38.944 INFO:tasks.workunit.client.0.smithi047.stdout: 10 95965 24.75 MB/sec execute 82 sec latency 309.124 ms 2024-03-21T11:40:38.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:38 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:40:38.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:38 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:40:38.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:38 smithi102 ceph-mon[36748]: pgmap v919: 129 pgs: 129 active+clean; 427 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 593 op/s 2024-03-21T11:40:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:38 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:40:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:38 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:40:39.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:38 smithi047 ceph-mon[31351]: pgmap v919: 129 pgs: 129 active+clean; 427 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 593 op/s 2024-03-21T11:40:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:38 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:40:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:38 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:40:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:38 smithi100 ceph-mon[36245]: pgmap v919: 129 pgs: 129 active+clean; 427 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 593 op/s 2024-03-21T11:40:39.944 INFO:tasks.workunit.client.0.smithi047.stdout: 10 96331 24.63 MB/sec execute 83 sec latency 122.011 ms 2024-03-21T11:40:40.944 INFO:tasks.workunit.client.0.smithi047.stdout: 10 96890 24.67 MB/sec execute 84 sec latency 45.425 ms 2024-03-21T11:40:41.021 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:40 smithi102 ceph-mon[36748]: pgmap v920: 129 pgs: 129 active+clean; 430 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.0 MiB/s wr, 754 op/s 2024-03-21T11:40:41.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:40 smithi047 ceph-mon[31351]: pgmap v920: 129 pgs: 129 active+clean; 430 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.0 MiB/s wr, 754 op/s 2024-03-21T11:40:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:40 smithi100 ceph-mon[36245]: pgmap v920: 129 pgs: 129 active+clean; 430 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.0 MiB/s wr, 754 op/s 2024-03-21T11:40:41.944 INFO:tasks.workunit.client.0.smithi047.stdout: 10 97537 24.81 MB/sec execute 85 sec latency 65.853 ms 2024-03-21T11:40:42.944 INFO:tasks.workunit.client.0.smithi047.stdout: 10 98023 24.90 MB/sec execute 86 sec latency 71.384 ms 2024-03-21T11:40:43.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:42 smithi047 ceph-mon[31351]: pgmap v921: 129 pgs: 129 active+clean; 431 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.7 MiB/s wr, 636 op/s 2024-03-21T11:40:43.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:42 smithi100 ceph-mon[36245]: pgmap v921: 129 pgs: 129 active+clean; 431 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.7 MiB/s wr, 636 op/s 2024-03-21T11:40:43.181 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:42 smithi102 ceph-mon[36748]: pgmap v921: 129 pgs: 129 active+clean; 431 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.7 MiB/s wr, 636 op/s 2024-03-21T11:40:43.944 INFO:tasks.workunit.client.0.smithi047.stdout: 10 98218 24.72 MB/sec execute 87 sec latency 416.871 ms 2024-03-21T11:40:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.946+0000 7ff281800640 1 -- 172.21.15.47:0/691489399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff27c1016d0 msgr2=0x7ff27c101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.946+0000 7ff281800640 1 --2- 172.21.15.47:0/691489399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff27c1016d0 0x7ff27c101af0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff264009900 tx=0x7ff26402bc40 comp rx=0 tx=0).stop 2024-03-21T11:40:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.947+0000 7ff281800640 1 -- 172.21.15.47:0/691489399 shutdown_connections 2024-03-21T11:40:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.947+0000 7ff281800640 1 --2- 172.21.15.47:0/691489399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff27c103280 0x7ff27c109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.947+0000 7ff281800640 1 --2- 172.21.15.47:0/691489399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff27c1028c0 0x7ff27c102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.947+0000 7ff281800640 1 --2- 172.21.15.47:0/691489399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff27c1016d0 0x7ff27c101af0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.947+0000 7ff281800640 1 -- 172.21.15.47:0/691489399 >> 172.21.15.47:0/691489399 conn(0x7ff27c0fcc60 msgr2=0x7ff27c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.947+0000 7ff281800640 1 -- 172.21.15.47:0/691489399 shutdown_connections 2024-03-21T11:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.947+0000 7ff281800640 1 -- 172.21.15.47:0/691489399 wait complete. 2024-03-21T11:40:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.949+0000 7ff281800640 1 Processor -- start 2024-03-21T11:40:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.949+0000 7ff281800640 1 -- start start 2024-03-21T11:40:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.950+0000 7ff281800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff27c1016d0 0x7ff27c19de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.950+0000 7ff281800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff27c1028c0 0x7ff27c19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.950+0000 7ff281800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff27c103280 0x7ff27c1a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.950+0000 7ff281800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c075150 con 0x7ff27c103280 2024-03-21T11:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.951+0000 7ff27aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff27c1016d0 0x7ff27c19de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.951+0000 7ff27aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff27c1016d0 0x7ff27c19de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59140/0 (socket says 172.21.15.47:59140) 2024-03-21T11:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.951+0000 7ff27aa00640 1 -- 172.21.15.47:0/4035244459 learned_addr learned my addr 172.21.15.47:0/4035244459 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.951+0000 7ff281800640 1 -- 172.21.15.47:0/4035244459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c074fd0 con 0x7ff27c1028c0 2024-03-21T11:40:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.951+0000 7ff281800640 1 -- 172.21.15.47:0/4035244459 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0752d0 con 0x7ff27c1016d0 2024-03-21T11:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.951+0000 7ff27b400640 1 --2- 172.21.15.47:0/4035244459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff27c103280 0x7ff27c1a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.952+0000 7ff27a000640 1 --2- 172.21.15.47:0/4035244459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff27c1028c0 0x7ff27c19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.952+0000 7ff27b400640 1 -- 172.21.15.47:0/4035244459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff27c1016d0 msgr2=0x7ff27c19de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.952+0000 7ff27b400640 1 --2- 172.21.15.47:0/4035244459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff27c1016d0 0x7ff27c19de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.952+0000 7ff27b400640 1 -- 172.21.15.47:0/4035244459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff27c1028c0 msgr2=0x7ff27c19e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.952+0000 7ff27b400640 1 --2- 172.21.15.47:0/4035244459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff27c1028c0 0x7ff27c19e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.952+0000 7ff27b400640 1 -- 172.21.15.47:0/4035244459 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c110060 con 0x7ff27c103280 2024-03-21T11:40:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.953+0000 7ff27b400640 1 --2- 172.21.15.47:0/4035244459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff27c103280 0x7ff27c1a5440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff274009fd0 tx=0x7ff27400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.953+0000 7ff273400640 1 -- 172.21.15.47:0/4035244459 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27400ce20 con 0x7ff27c103280 2024-03-21T11:40:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.953+0000 7ff273400640 1 -- 172.21.15.47:0/4035244459 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff274005020 con 0x7ff27c103280 2024-03-21T11:40:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.953+0000 7ff273400640 1 -- 172.21.15.47:0/4035244459 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274010720 con 0x7ff27c103280 2024-03-21T11:40:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.953+0000 7ff281800640 1 -- 172.21.15.47:0/4035244459 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27c110350 con 0x7ff27c103280 2024-03-21T11:40:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.954+0000 7ff281800640 1 -- 172.21.15.47:0/4035244459 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27c1108f0 con 0x7ff27c103280 2024-03-21T11:40:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.959+0000 7ff284471740 1 Processor -- start 2024-03-21T11:40:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.959+0000 7ff273400640 1 -- 172.21.15.47:0/4035244459 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff274005540 con 0x7ff27c103280 2024-03-21T11:40:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.959+0000 7ff284471740 1 -- start start 2024-03-21T11:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.960+0000 7ff284471740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5ed1e0 0x563e4c5ed600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.960+0000 7ff284471740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.960+0000 7ff284471740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5f2d40 0x563e4c5f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.960+0000 7ff284471740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e4c5b03b0 con 0x563e4c5ed1e0 2024-03-21T11:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.960+0000 7ff284471740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e4c552830 con 0x563e4c5f2d40 2024-03-21T11:40:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.960+0000 7ff284471740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e4c4fb5d0 con 0x563e4c5edbd0 2024-03-21T11:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.960+0000 7ff27a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5ed1e0 0x563e4c5ed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.961+0000 7ff27a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5ed1e0 0x563e4c5ed600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37542/0 (socket says 172.21.15.47:37542) 2024-03-21T11:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.961+0000 7ff27a000640 1 -- 172.21.15.47:0/3342733971 learned_addr learned my addr 172.21.15.47:0/3342733971 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.961+0000 7ff27aa00640 1 --2- 172.21.15.47:0/3342733971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.961+0000 7ff27a000640 1 --2- 172.21.15.47:0/3342733971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5f2d40 0x563e4c5f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.961+0000 7ff27a000640 1 -- 172.21.15.47:0/3342733971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 msgr2=0x563e4c5f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.961+0000 7ff27a000640 1 --2- 172.21.15.47:0/3342733971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5f26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.961+0000 7ff27a000640 1 -- 172.21.15.47:0/3342733971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5ed1e0 msgr2=0x563e4c5ed600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.961+0000 7ff27a000640 1 --2- 172.21.15.47:0/3342733971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5ed1e0 0x563e4c5ed600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.961+0000 7ff27a000640 1 -- 172.21.15.47:0/3342733971 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e4c5132c0 con 0x563e4c5f2d40 2024-03-21T11:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.961+0000 7ff27a000640 1 --2- 172.21.15.47:0/3342733971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5f2d40 0x563e4c5f51d0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7ff26c005e10 tx=0x7ff26c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2ecd6fb4a7b49dd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.962+0000 7ff273400640 1 --2- 172.21.15.47:0/4035244459 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2600a3040 0x7ff2600a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.962+0000 7ff26be00640 1 -- 172.21.15.47:0/3342733971 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0109b0 con 0x563e4c5f2d40 2024-03-21T11:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.962+0000 7ff273400640 1 -- 172.21.15.47:0/4035244459 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff2740cfd80 con 0x7ff27c103280 2024-03-21T11:40:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.962+0000 7ff26be00640 1 -- 172.21.15.47:0/3342733971 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff26c005500 con 0x563e4c5f2d40 2024-03-21T11:40:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.962+0000 7ff26be00640 1 -- 172.21.15.47:0/3342733971 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c004a10 con 0x563e4c5f2d40 2024-03-21T11:40:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.963+0000 7ff284471740 1 -- 172.21.15.47:0/3342733971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5f2d40 msgr2=0x563e4c5f51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.963+0000 7ff284471740 1 --2- 172.21.15.47:0/3342733971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5f2d40 0x563e4c5f51d0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7ff26c005e10 tx=0x7ff26c0046c0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.963+0000 7ff27aa00640 1 --2- 172.21.15.47:0/4035244459 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2600a3040 0x7ff2600a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.963+0000 7ff284471740 1 -- 172.21.15.47:0/3342733971 shutdown_connections 2024-03-21T11:40:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.963+0000 7ff284471740 1 --2- 172.21.15.47:0/3342733971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5f2d40 0x563e4c5f51d0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.963+0000 7ff284471740 1 --2- 172.21.15.47:0/3342733971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.963+0000 7ff284471740 1 --2- 172.21.15.47:0/3342733971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5ed1e0 0x563e4c5ed600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.963+0000 7ff284471740 1 -- 172.21.15.47:0/3342733971 >> 172.21.15.47:0/3342733971 conn(0x563e4c5e8750 msgr2=0x563e4c5e8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.965+0000 7ff284471740 1 -- 172.21.15.47:0/3342733971 shutdown_connections 2024-03-21T11:40:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.966+0000 7ff27aa00640 1 --2- 172.21.15.47:0/4035244459 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2600a3040 0x7ff2600a5500 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff264005ab0 tx=0x7ff26402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.966+0000 7ff284471740 1 -- 172.21.15.47:0/3342733971 wait complete. 2024-03-21T11:40:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.966+0000 7ff284471740 1 -- start start 2024-03-21T11:40:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.967+0000 7ff284471740 1 Processor -- start 2024-03-21T11:40:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.967+0000 7ff284471740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5f7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.967+0000 7ff284471740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5f7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.967+0000 7ff284471740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 0x563e4c5fe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.967+0000 7ff284471740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e4c50ea20 con 0x563e4c5f2d40 2024-03-21T11:40:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.967+0000 7ff284471740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e4c54e8d0 con 0x563e4c5ed1e0 2024-03-21T11:40:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.967+0000 7ff284471740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e4c5f5cf0 con 0x563e4c5edbd0 2024-03-21T11:40:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.968+0000 7ff27a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.968+0000 7ff27aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.968+0000 7ff27a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5f7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59164/0 (socket says 172.21.15.47:59164) 2024-03-21T11:40:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.968+0000 7ff27a000640 1 -- 172.21.15.47:0/1890860643 learned_addr learned my addr 172.21.15.47:0/1890860643 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.968+0000 7ff27a000640 1 -- 172.21.15.47:0/1890860643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 msgr2=0x563e4c5f7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.968+0000 7ff27a000640 1 --2- 172.21.15.47:0/1890860643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5f7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.968+0000 7ff27a000640 1 -- 172.21.15.47:0/1890860643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 msgr2=0x563e4c5fe880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.968+0000 7ff27a000640 1 --2- 172.21.15.47:0/1890860643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 0x563e4c5fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.968+0000 7ff27a000640 1 -- 172.21.15.47:0/1890860643 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e4c267210 con 0x563e4c5edbd0 2024-03-21T11:40:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.969+0000 7ff27a000640 1 --2- 172.21.15.47:0/1890860643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5f7840 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff26c006010 tx=0x7ff26c013340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c3a5d37069754e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.969+0000 7ff268c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:40:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.969+0000 7ff268c00640 1 -- 172.21.15.47:0/1890860643 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c027650 con 0x563e4c5edbd0 2024-03-21T11:40:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.969+0000 7ff268c00640 1 -- 172.21.15.47:0/1890860643 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff26c0277f0 con 0x563e4c5edbd0 2024-03-21T11:40:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.969+0000 7ff268c00640 10 client.0 inactive, discarding osd_map(95..95 src has 1..95) v4 2024-03-21T11:40:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.969+0000 7ff284471740 10 client.0 _ll_register_callbacks cb 0x563e4c54b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.969+0000 7ff284471740 1 -- 172.21.15.47:0/1890860643 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff26c005500 con 0x563e4c5edbd0 2024-03-21T11:40:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.970+0000 7ff268c00640 1 -- 172.21.15.47:0/1890860643 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff26c010c40 con 0x563e4c5edbd0 2024-03-21T11:40:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.971+0000 7ff284471740 10 client.31147 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.971+0000 7ff284471740 1 -- 172.21.15.47:0/1890860643 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=40}) v3 -- 0x563e4c333620 con 0x563e4c5edbd0 2024-03-21T11:40:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.972+0000 7ff268c00640 1 -- 172.21.15.47:0/1890860643 <== mon.1 v2:172.21.15.102:3300/0 4 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff26c0279f0 con 0x563e4c5edbd0 2024-03-21T11:40:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.972+0000 7ff284471740 10 client.31147 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.972+0000 7ff284471740 10 client.31147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.972+0000 7ff284471740 20 client.31147 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.972+0000 7ff284471740 1 --2- 172.21.15.47:0/1890860643 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e4c602ba0 0x563e4c604f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.972+0000 7ff284471740 4 client.31147 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.972+0000 7ff284471740 1 -- 172.21.15.47:0/1890860643 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e4c55fd70 con 0x563e4c602ba0 2024-03-21T11:40:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.974+0000 7ff27aa00640 1 --2- 172.21.15.47:0/1890860643 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e4c602ba0 0x563e4c604f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.975+0000 7ff27aa00640 1 --2- 172.21.15.47:0/1890860643 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e4c602ba0 0x563e4c604f90 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49cd99508ae3d512 server_cookie=1b7fe12c63313a39 in_seq=0 out_seq=0 2024-03-21T11:40:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.976+0000 7ff268c00640 10 client.31147 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.977+0000 7ff268c00640 1 -- 172.21.15.47:0/1890860643 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563e4c55fd70 con 0x563e4c602ba0 2024-03-21T11:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.977+0000 7ff268c00640 10 client.31147 handle_command_reply: tid=0 2024-03-21T11:40:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.977+0000 7ff284471740 1 client.31147 shutdown 2024-03-21T11:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.978+0000 7ff284471740 1 -- 172.21.15.47:0/1890860643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 msgr2=0x563e4c5f7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.978+0000 7ff284471740 1 --2- 172.21.15.47:0/1890860643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5f7840 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff26c006010 tx=0x7ff26c013340 comp rx=0 tx=0).stop 2024-03-21T11:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.978+0000 7ff284471740 1 -- 172.21.15.47:0/1890860643 shutdown_connections 2024-03-21T11:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.978+0000 7ff284471740 1 --2- 172.21.15.47:0/1890860643 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e4c602ba0 0x563e4c604f90 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.978+0000 7ff284471740 1 --2- 172.21.15.47:0/1890860643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 0x563e4c5fe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.978+0000 7ff284471740 1 --2- 172.21.15.47:0/1890860643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5f7840 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.978+0000 7ff284471740 1 --2- 172.21.15.47:0/1890860643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5f7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.978+0000 7ff284471740 1 -- 172.21.15.47:0/1890860643 >> 172.21.15.47:0/1890860643 conn(0x563e4c5e8750 msgr2=0x563e4c5e7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.978+0000 7ff268c00640 0 client.31147 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.979+0000 7ff284471740 1 -- 172.21.15.47:0/1890860643 shutdown_connections 2024-03-21T11:40:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.979+0000 7ff284471740 1 -- 172.21.15.47:0/1890860643 wait complete. 2024-03-21T11:40:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:43.979+0000 7ff284471740 20 client.31147 trim_cache size 0 max 16384 2024-03-21T11:40:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.013+0000 7ff284471740 1 Processor -- start 2024-03-21T11:40:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.013+0000 7ff284471740 1 -- start start 2024-03-21T11:40:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.014+0000 7ff284471740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5eb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.014+0000 7ff284471740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5fe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.014+0000 7ff284471740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 0x563e4c5ebef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.014+0000 7ff284471740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c1b8d80 con 0x563e4c5f2d40 2024-03-21T11:40:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.014+0000 7ff284471740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c1ba700 con 0x563e4c5ed1e0 2024-03-21T11:40:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.014+0000 7ff284471740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c1b93e0 con 0x563e4c5edbd0 2024-03-21T11:40:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.014+0000 7ff27a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.014+0000 7ff27a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5fe840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59166/0 (socket says 172.21.15.47:59166) 2024-03-21T11:40:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.014+0000 7ff27a000640 1 -- 172.21.15.47:0/367052703 learned_addr learned my addr 172.21.15.47:0/367052703 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.015+0000 7ff27aa00640 1 --2- 172.21.15.47:0/367052703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5eb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.015+0000 7ff27a000640 1 -- 172.21.15.47:0/367052703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 msgr2=0x563e4c5eb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.015+0000 7ff27a000640 1 --2- 172.21.15.47:0/367052703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5eb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.015+0000 7ff27a000640 1 -- 172.21.15.47:0/367052703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 msgr2=0x563e4c5ebef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.015+0000 7ff27b400640 1 --2- 172.21.15.47:0/367052703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 0x563e4c5ebef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.016+0000 7ff27a000640 1 --2- 172.21.15.47:0/367052703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 0x563e4c5ebef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.016+0000 7ff27a000640 1 -- 172.21.15.47:0/367052703 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e4c605300 con 0x563e4c5edbd0 2024-03-21T11:40:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.016+0000 7ff27aa00640 1 --2- 172.21.15.47:0/367052703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5eb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.016+0000 7ff27b400640 1 --2- 172.21.15.47:0/367052703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 0x563e4c5ebef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.021+0000 7ff27a000640 1 --2- 172.21.15.47:0/367052703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5fe840 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0057a0 tx=0x7ff26c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=912b67be2ca4975f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.021+0000 7ff26b400640 1 -- 172.21.15.47:0/367052703 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c010de0 con 0x563e4c5edbd0 2024-03-21T11:40:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.022+0000 7ff26b400640 1 -- 172.21.15.47:0/367052703 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff26c005500 con 0x563e4c5edbd0 2024-03-21T11:40:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.022+0000 7ff26b400640 1 -- 172.21.15.47:0/367052703 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0276b0 con 0x563e4c5edbd0 2024-03-21T11:40:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.023+0000 7ff284471740 1 -- 172.21.15.47:0/367052703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 msgr2=0x563e4c5fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.023+0000 7ff284471740 1 --2- 172.21.15.47:0/367052703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5fe840 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0057a0 tx=0x7ff26c020000 comp rx=0 tx=0).stop 2024-03-21T11:40:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.025+0000 7ff284471740 1 -- 172.21.15.47:0/367052703 shutdown_connections 2024-03-21T11:40:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.025+0000 7ff284471740 1 --2- 172.21.15.47:0/367052703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 0x563e4c5ebef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.025+0000 7ff284471740 1 --2- 172.21.15.47:0/367052703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5fe840 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.025+0000 7ff284471740 1 --2- 172.21.15.47:0/367052703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5eb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.025+0000 7ff284471740 1 -- 172.21.15.47:0/367052703 >> 172.21.15.47:0/367052703 conn(0x563e4c5e8750 msgr2=0x563e4c5ee2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.027+0000 7ff284471740 1 -- 172.21.15.47:0/367052703 shutdown_connections 2024-03-21T11:40:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.027+0000 7ff284471740 1 -- 172.21.15.47:0/367052703 wait complete. 2024-03-21T11:40:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.028+0000 7ff284471740 1 -- start start 2024-03-21T11:40:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.028+0000 7ff284471740 1 Processor -- start 2024-03-21T11:40:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.029+0000 7ff284471740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5fbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.029+0000 7ff284471740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5fc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.029+0000 7ff284471740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 0x563e4c5fd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.030+0000 7ff27a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5fc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.030+0000 7ff284471740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c1ba0a0 con 0x563e4c5f2d40 2024-03-21T11:40:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.030+0000 7ff284471740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c1b9a40 con 0x563e4c5ed1e0 2024-03-21T11:40:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.030+0000 7ff284471740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c1184a0 con 0x563e4c5edbd0 2024-03-21T11:40:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.030+0000 7ff27aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5fbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.030+0000 7ff284471740 10 client.0 _ll_register_callbacks cb 0x7ff26c00ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.030+0000 7ff27aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5fbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56514/0 (socket says 172.21.15.47:56514) 2024-03-21T11:40:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.030+0000 7ff27aa00640 1 -- 172.21.15.47:0/766264308 learned_addr learned my addr 172.21.15.47:0/766264308 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.030+0000 7ff27b400640 1 --2- 172.21.15.47:0/766264308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 0x563e4c5fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.031+0000 7ff27aa00640 1 -- 172.21.15.47:0/766264308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 msgr2=0x563e4c5fc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.031+0000 7ff27aa00640 1 --2- 172.21.15.47:0/766264308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5fc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.031+0000 7ff27aa00640 1 -- 172.21.15.47:0/766264308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 msgr2=0x563e4c5fd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.031+0000 7ff27aa00640 1 --2- 172.21.15.47:0/766264308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 0x563e4c5fd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.031+0000 7ff27aa00640 1 -- 172.21.15.47:0/766264308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e4c5fd940 con 0x563e4c5ed1e0 2024-03-21T11:40:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.032+0000 7ff27a000640 1 --2- 172.21.15.47:0/766264308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5fc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.032+0000 7ff27aa00640 1 --2- 172.21.15.47:0/766264308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5fbe00 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7ff26402e010 tx=0x7ff26402cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26720abf787b9d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.032+0000 7ff268c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.034+0000 7ff268c00640 1 -- 172.21.15.47:0/766264308 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264004ef0 con 0x563e4c5ed1e0 2024-03-21T11:40:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.034+0000 7ff268c00640 1 -- 172.21.15.47:0/766264308 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff264005090 con 0x563e4c5ed1e0 2024-03-21T11:40:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.034+0000 7ff284471740 1 -- 172.21.15.47:0/766264308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff26c005500 con 0x563e4c5ed1e0 2024-03-21T11:40:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.034+0000 7ff268c00640 1 -- 172.21.15.47:0/766264308 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff264066050 con 0x563e4c5ed1e0 2024-03-21T11:40:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.034+0000 7ff284471740 10 client.31175 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.035+0000 7ff284471740 1 -- 172.21.15.47:0/766264308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563e4c65e460 con 0x563e4c5ed1e0 2024-03-21T11:40:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.036+0000 7ff268c00640 1 -- 172.21.15.47:0/766264308 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff26c005500 con 0x563e4c5ed1e0 2024-03-21T11:40:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.036+0000 7ff268c00640 1 -- 172.21.15.47:0/766264308 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff26c0134e0 con 0x563e4c5ed1e0 2024-03-21T11:40:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.036+0000 7ff284471740 10 client.31175 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.036+0000 7ff284471740 10 client.31175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.036+0000 7ff284471740 20 client.31175 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.036+0000 7ff284471740 1 --2- 172.21.15.47:0/766264308 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e4c65ec40 0x563e4c661030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.036+0000 7ff284471740 4 client.31175 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:40:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.036+0000 7ff284471740 1 -- 172.21.15.47:0/766264308 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563e4c5f2a40 con 0x563e4c65ec40 2024-03-21T11:40:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.037+0000 7ff27a000640 1 --2- 172.21.15.47:0/766264308 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e4c65ec40 0x563e4c661030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.038+0000 7ff27a000640 1 --2- 172.21.15.47:0/766264308 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e4c65ec40 0x563e4c661030 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58961b056b39018e server_cookie=e6a279f64f63e1d1 in_seq=0 out_seq=0 2024-03-21T11:40:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:44.038+0000 7ff268c00640 10 client.31175 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:44.944 INFO:tasks.workunit.client.0.smithi047.stdout: 10 98367 24.53 MB/sec execute 88 sec latency 553.542 ms 2024-03-21T11:40:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.001+0000 7ff268c00640 1 -- 172.21.15.47:0/766264308 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563e4c5f2a40 con 0x563e4c65ec40 2024-03-21T11:40:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.001+0000 7ff268c00640 10 client.31175 handle_command_reply: tid=0 2024-03-21T11:40:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.001+0000 7ff284471740 1 client.31175 shutdown 2024-03-21T11:40:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.002+0000 7ff284471740 1 -- 172.21.15.47:0/766264308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 msgr2=0x563e4c5fbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.002+0000 7ff284471740 1 --2- 172.21.15.47:0/766264308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5fbe00 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7ff26402e010 tx=0x7ff26402cf90 comp rx=0 tx=0).stop 2024-03-21T11:40:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.002+0000 7ff284471740 1 -- 172.21.15.47:0/766264308 shutdown_connections 2024-03-21T11:40:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.002+0000 7ff284471740 1 --2- 172.21.15.47:0/766264308 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e4c65ec40 0x563e4c661030 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.002+0000 7ff284471740 1 --2- 172.21.15.47:0/766264308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e4c5f2d40 0x563e4c5fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.002+0000 7ff284471740 1 --2- 172.21.15.47:0/766264308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e4c5edbd0 0x563e4c5fc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.002+0000 7ff284471740 1 --2- 172.21.15.47:0/766264308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e4c5ed1e0 0x563e4c5fbe00 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.002+0000 7ff284471740 1 -- 172.21.15.47:0/766264308 >> 172.21.15.47:0/766264308 conn(0x563e4c5e8750 msgr2=0x563e4c5eb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.002+0000 7ff268c00640 0 client.31175 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.002+0000 7ff284471740 1 -- 172.21.15.47:0/766264308 shutdown_connections 2024-03-21T11:40:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.003+0000 7ff284471740 1 -- 172.21.15.47:0/766264308 wait complete. 2024-03-21T11:40:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.003+0000 7ff284471740 20 client.31175 trim_cache size 0 max 16384 2024-03-21T11:40:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:40:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:40:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:40:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:40:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.004+0000 7ff281800640 1 -- 172.21.15.47:0/4035244459 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2600a3040 msgr2=0x7ff2600a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.004+0000 7ff281800640 1 --2- 172.21.15.47:0/4035244459 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2600a3040 0x7ff2600a5500 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff264005ab0 tx=0x7ff26402e040 comp rx=0 tx=0).stop 2024-03-21T11:40:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.004+0000 7ff281800640 1 -- 172.21.15.47:0/4035244459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff27c103280 msgr2=0x7ff27c1a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.004+0000 7ff281800640 1 --2- 172.21.15.47:0/4035244459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff27c103280 0x7ff27c1a5440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff274009fd0 tx=0x7ff27400ef10 comp rx=0 tx=0).stop 2024-03-21T11:40:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.005+0000 7ff281800640 1 -- 172.21.15.47:0/4035244459 shutdown_connections 2024-03-21T11:40:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.005+0000 7ff281800640 1 --2- 172.21.15.47:0/4035244459 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff2600a3040 0x7ff2600a5500 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.005+0000 7ff281800640 1 --2- 172.21.15.47:0/4035244459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff27c103280 0x7ff27c1a5440 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.005+0000 7ff281800640 1 --2- 172.21.15.47:0/4035244459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff27c1028c0 0x7ff27c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.005+0000 7ff281800640 1 --2- 172.21.15.47:0/4035244459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff27c1016d0 0x7ff27c19de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.005+0000 7ff281800640 1 -- 172.21.15.47:0/4035244459 >> 172.21.15.47:0/4035244459 conn(0x7ff27c0fcc60 msgr2=0x7ff27c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.005+0000 7ff281800640 1 -- 172.21.15.47:0/4035244459 shutdown_connections 2024-03-21T11:40:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:45.005+0000 7ff281800640 1 -- 172.21.15.47:0/4035244459 wait complete. 2024-03-21T11:40:45.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:44 smithi047 ceph-mon[31351]: pgmap v922: 129 pgs: 129 active+clean; 434 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 656 op/s 2024-03-21T11:40:45.096 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:44 smithi102 ceph-mon[36748]: pgmap v922: 129 pgs: 129 active+clean; 434 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 656 op/s 2024-03-21T11:40:45.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:44 smithi100 ceph-mon[36245]: pgmap v922: 129 pgs: 129 active+clean; 434 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 656 op/s 2024-03-21T11:40:45.945 INFO:tasks.workunit.client.0.smithi047.stdout: 10 98735 24.38 MB/sec execute 89 sec latency 169.525 ms 2024-03-21T11:40:46.946 INFO:tasks.workunit.client.0.smithi047.stdout: 10 99300 24.44 MB/sec execute 90 sec latency 371.295 ms 2024-03-21T11:40:47.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:46 smithi047 ceph-mon[31351]: pgmap v923: 129 pgs: 129 active+clean; 437 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 534 op/s 2024-03-21T11:40:47.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:46 smithi100 ceph-mon[36245]: pgmap v923: 129 pgs: 129 active+clean; 437 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 534 op/s 2024-03-21T11:40:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:46 smithi102 ceph-mon[36748]: pgmap v923: 129 pgs: 129 active+clean; 437 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 534 op/s 2024-03-21T11:40:47.944 INFO:tasks.workunit.client.0.smithi047.stdout: 10 99952 24.59 MB/sec execute 91 sec latency 64.299 ms 2024-03-21T11:40:48.944 INFO:tasks.workunit.client.0.smithi047.stdout: 10 100180 24.50 MB/sec execute 92 sec latency 493.891 ms 2024-03-21T11:40:49.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:48 smithi047 ceph-mon[31351]: pgmap v924: 129 pgs: 129 active+clean; 441 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 533 op/s 2024-03-21T11:40:49.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:48 smithi100 ceph-mon[36245]: pgmap v924: 129 pgs: 129 active+clean; 441 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 533 op/s 2024-03-21T11:40:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:48 smithi102 ceph-mon[36748]: pgmap v924: 129 pgs: 129 active+clean; 441 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.6 MiB/s wr, 533 op/s 2024-03-21T11:40:49.945 INFO:tasks.workunit.client.0.smithi047.stdout: 10 100658 24.50 MB/sec execute 93 sec latency 128.499 ms 2024-03-21T11:40:50.945 INFO:tasks.workunit.client.0.smithi047.stdout: 10 101132 24.49 MB/sec execute 94 sec latency 112.028 ms 2024-03-21T11:40:51.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:50 smithi047 ceph-mon[31351]: pgmap v925: 129 pgs: 129 active+clean; 438 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.3 MiB/s wr, 744 op/s 2024-03-21T11:40:51.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:50 smithi100 ceph-mon[36245]: pgmap v925: 129 pgs: 129 active+clean; 438 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.3 MiB/s wr, 744 op/s 2024-03-21T11:40:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:50 smithi102 ceph-mon[36748]: pgmap v925: 129 pgs: 129 active+clean; 438 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.3 MiB/s wr, 744 op/s 2024-03-21T11:40:51.945 INFO:tasks.workunit.client.0.smithi047.stdout: 10 101531 24.51 MB/sec execute 95 sec latency 154.378 ms 2024-03-21T11:40:52.945 INFO:tasks.workunit.client.0.smithi047.stdout: 10 102013 24.53 MB/sec execute 96 sec latency 98.144 ms 2024-03-21T11:40:53.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:52 smithi047 ceph-mon[31351]: pgmap v926: 129 pgs: 129 active+clean; 439 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 585 op/s 2024-03-21T11:40:53.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:52 smithi100 ceph-mon[36245]: pgmap v926: 129 pgs: 129 active+clean; 439 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 585 op/s 2024-03-21T11:40:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:52 smithi102 ceph-mon[36748]: pgmap v926: 129 pgs: 129 active+clean; 439 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 585 op/s 2024-03-21T11:40:53.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:40:53.244 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:40:53.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.602+0000 7f6af8800640 1 -- 172.21.15.47:0/3745247787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af0102880 msgr2=0x7f6af0104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:53.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.602+0000 7f6af8800640 1 --2- 172.21.15.47:0/3745247787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af0102880 0x7f6af0104cf0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009900 tx=0x7f6ae002bc40 comp rx=0 tx=0).stop 2024-03-21T11:40:53.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.603+0000 7f6af8800640 1 -- 172.21.15.47:0/3745247787 shutdown_connections 2024-03-21T11:40:53.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.603+0000 7f6af8800640 1 --2- 172.21.15.47:0/3745247787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af0105230 0x7f6af01056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.603+0000 7f6af8800640 1 --2- 172.21.15.47:0/3745247787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af0102880 0x7f6af0104cf0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.603+0000 7f6af8800640 1 --2- 172.21.15.47:0/3745247787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af00fff10 0x7f6af0102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.603+0000 7f6af8800640 1 -- 172.21.15.47:0/3745247787 >> 172.21.15.47:0/3745247787 conn(0x7f6af00fbb30 msgr2=0x7f6af00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:53.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.604+0000 7f6af8800640 1 -- 172.21.15.47:0/3745247787 shutdown_connections 2024-03-21T11:40:53.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.604+0000 7f6af8800640 1 -- 172.21.15.47:0/3745247787 wait complete. 2024-03-21T11:40:53.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.606+0000 7f6af8800640 1 Processor -- start 2024-03-21T11:40:53.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.607+0000 7f6af8800640 1 -- start start 2024-03-21T11:40:53.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.608+0000 7f6af8800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af00fff10 0x7f6af019dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.608+0000 7f6af8800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af0102880 0x7f6af019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.609+0000 7f6af8800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af0105230 0x7f6af01a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.609+0000 7f6af8800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af0113970 con 0x7f6af00fff10 2024-03-21T11:40:53.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.609+0000 7f6af8800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01137f0 con 0x7f6af0102880 2024-03-21T11:40:53.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.609+0000 7f6af8800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af0113af0 con 0x7f6af0105230 2024-03-21T11:40:53.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.609+0000 7f6af5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af0102880 0x7f6af019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.609+0000 7f6af5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af00fff10 0x7f6af019dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.609+0000 7f6af5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af0102880 0x7f6af019e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46956/0 (socket says 172.21.15.47:46956) 2024-03-21T11:40:53.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.609+0000 7f6af5400640 1 -- 172.21.15.47:0/656547324 learned_addr learned my addr 172.21.15.47:0/656547324 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:53.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.609+0000 7f6af6800640 1 --2- 172.21.15.47:0/656547324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af0105230 0x7f6af01a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.609+0000 7f6af5400640 1 -- 172.21.15.47:0/656547324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af0105230 msgr2=0x7f6af01a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:53.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.609+0000 7f6af5400640 1 --2- 172.21.15.47:0/656547324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af0105230 0x7f6af01a5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.609+0000 7f6af5400640 1 -- 172.21.15.47:0/656547324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af00fff10 msgr2=0x7f6af019dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:53.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.612+0000 7f6af5400640 1 --2- 172.21.15.47:0/656547324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af00fff10 0x7f6af019dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.612+0000 7f6af5400640 1 -- 172.21.15.47:0/656547324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af01a5bd0 con 0x7f6af0102880 2024-03-21T11:40:53.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.612+0000 7f6af5e00640 1 --2- 172.21.15.47:0/656547324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af00fff10 0x7f6af019dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:53.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.613+0000 7f6af5400640 1 --2- 172.21.15.47:0/656547324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af0102880 0x7f6af019e4f0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009230 tx=0x7f6ae0004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:53.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.614+0000 7f6aeaa00640 1 -- 172.21.15.47:0/656547324 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0041070 con 0x7f6af0102880 2024-03-21T11:40:53.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.614+0000 7f6aeaa00640 1 -- 172.21.15.47:0/656547324 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ae00054e0 con 0x7f6af0102880 2024-03-21T11:40:53.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.614+0000 7f6af8800640 1 -- 172.21.15.47:0/656547324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af01a5e60 con 0x7f6af0102880 2024-03-21T11:40:53.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.614+0000 7f6aeaa00640 1 -- 172.21.15.47:0/656547324 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0034440 con 0x7f6af0102880 2024-03-21T11:40:53.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.614+0000 7f6af8800640 1 -- 172.21.15.47:0/656547324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af01a62d0 con 0x7f6af0102880 2024-03-21T11:40:53.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.618+0000 7f6afb5da740 1 Processor -- start 2024-03-21T11:40:53.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.618+0000 7f6afb5da740 1 -- start start 2024-03-21T11:40:53.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.619+0000 7f6afb5da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f4fe80 0x557dc8f502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.619+0000 7f6afb5da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f50870 0x557dc8f55370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.619+0000 7f6afb5da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f559e0 0x557dc8f57e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.619+0000 7f6afb5da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc8eb5250 con 0x557dc8f559e0 2024-03-21T11:40:53.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.619+0000 7f6afb5da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc8e5da30 con 0x557dc8f50870 2024-03-21T11:40:53.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.619+0000 7f6afb5da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc8eb12f0 con 0x557dc8f4fe80 2024-03-21T11:40:53.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.621+0000 7f6aeaa00640 1 -- 172.21.15.47:0/656547324 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6ae0044070 con 0x7f6af0102880 2024-03-21T11:40:53.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.622+0000 7f6af6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f4fe80 0x557dc8f502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.622+0000 7f6af6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f4fe80 0x557dc8f502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41922/0 (socket says 172.21.15.47:41922) 2024-03-21T11:40:53.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.622+0000 7f6af6800640 1 -- 172.21.15.47:0/1839665640 learned_addr learned my addr 172.21.15.47:0/1839665640 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:53.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.622+0000 7f6af5e00640 1 --2- 172.21.15.47:0/1839665640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f50870 0x557dc8f55370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.623+0000 7f6af6800640 1 -- 172.21.15.47:0/1839665640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f50870 msgr2=0x557dc8f55370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.623+0000 7f6af6800640 1 --2- 172.21.15.47:0/1839665640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f50870 0x557dc8f55370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.623+0000 7f6af6800640 1 -- 172.21.15.47:0/1839665640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f559e0 msgr2=0x557dc8f57e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.623+0000 7f6af5400640 1 --2- 172.21.15.47:0/1839665640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f559e0 0x557dc8f57e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.623+0000 7f6af6800640 1 --2- 172.21.15.47:0/1839665640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f559e0 0x557dc8f57e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.624+0000 7f6af6800640 1 -- 172.21.15.47:0/1839665640 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dc8e48690 con 0x557dc8f4fe80 2024-03-21T11:40:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.624+0000 7f6af5400640 1 --2- 172.21.15.47:0/1839665640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f559e0 0x557dc8f57e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.625+0000 7f6aeaa00640 1 --2- 172.21.15.47:0/656547324 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ad80a2fa0 0x7f6ad80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.625+0000 7f6af5e00640 1 --2- 172.21.15.47:0/656547324 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ad80a2fa0 0x7f6ad80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.625+0000 7f6af6800640 1 --2- 172.21.15.47:0/1839665640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f4fe80 0x557dc8f502a0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6aec007d90 tx=0x7f6aec00c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f8215314cad5f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:53.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.626+0000 7f6aeaa00640 1 -- 172.21.15.47:0/656547324 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6ae00f1500 con 0x7f6af0102880 2024-03-21T11:40:53.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.626+0000 7f6adf000640 1 -- 172.21.15.47:0/1839665640 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec015020 con 0x557dc8f4fe80 2024-03-21T11:40:53.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.626+0000 7f6adf000640 1 -- 172.21.15.47:0/1839665640 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6aec011b10 con 0x557dc8f4fe80 2024-03-21T11:40:53.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.626+0000 7f6adf000640 1 -- 172.21.15.47:0/1839665640 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec010b10 con 0x557dc8f4fe80 2024-03-21T11:40:53.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.627+0000 7f6af5e00640 1 --2- 172.21.15.47:0/656547324 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ad80a2fa0 0x7f6ad80a5460 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x557dc8f50c70 tx=0x7f6ae400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:53.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.627+0000 7f6afb5da740 1 -- 172.21.15.47:0/1839665640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f4fe80 msgr2=0x557dc8f502a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:53.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.627+0000 7f6afb5da740 1 --2- 172.21.15.47:0/1839665640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f4fe80 0x557dc8f502a0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6aec007d90 tx=0x7f6aec00c560 comp rx=0 tx=0).stop 2024-03-21T11:40:53.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.628+0000 7f6afb5da740 1 -- 172.21.15.47:0/1839665640 shutdown_connections 2024-03-21T11:40:53.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.628+0000 7f6afb5da740 1 --2- 172.21.15.47:0/1839665640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f559e0 0x557dc8f57e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.628+0000 7f6afb5da740 1 --2- 172.21.15.47:0/1839665640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f50870 0x557dc8f55370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.628+0000 7f6afb5da740 1 --2- 172.21.15.47:0/1839665640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f4fe80 0x557dc8f502a0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.628+0000 7f6afb5da740 1 -- 172.21.15.47:0/1839665640 >> 172.21.15.47:0/1839665640 conn(0x557dc8f18200 msgr2=0x557dc8f4b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:53.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.629+0000 7f6afb5da740 1 -- 172.21.15.47:0/1839665640 shutdown_connections 2024-03-21T11:40:53.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.629+0000 7f6afb5da740 1 -- 172.21.15.47:0/1839665640 wait complete. 2024-03-21T11:40:53.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.629+0000 7f6afb5da740 1 -- start start 2024-03-21T11:40:53.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.630+0000 7f6afb5da740 1 Processor -- start 2024-03-21T11:40:53.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.631+0000 7f6afb5da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f4fe80 0x557dc8f59eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.631+0000 7f6afb5da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f50870 0x557dc8f5a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.631+0000 7f6afb5da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f559e0 0x557dc8f61430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.631+0000 7f6afb5da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc8f58990 con 0x557dc8f4fe80 2024-03-21T11:40:53.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.631+0000 7f6afb5da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc8e70e80 con 0x557dc8f50870 2024-03-21T11:40:53.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.631+0000 7f6afb5da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc8f58b10 con 0x557dc8f559e0 2024-03-21T11:40:53.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.632+0000 7f6afb5da740 10 client.0 _ll_register_callbacks cb 0x557dc8e83130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:53.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.632+0000 7f6af5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f4fe80 0x557dc8f59eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.632+0000 7f6af5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f4fe80 0x557dc8f59eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39074/0 (socket says 172.21.15.47:39074) 2024-03-21T11:40:53.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.632+0000 7f6af5e00640 1 -- 172.21.15.47:0/2383062776 learned_addr learned my addr 172.21.15.47:0/2383062776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:53.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.633+0000 7f6af6800640 1 --2- 172.21.15.47:0/2383062776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f559e0 0x557dc8f61430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.633+0000 7f6af5e00640 1 -- 172.21.15.47:0/2383062776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f559e0 msgr2=0x557dc8f61430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:53.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.633+0000 7f6af5e00640 1 --2- 172.21.15.47:0/2383062776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f559e0 0x557dc8f61430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.633+0000 7f6af5e00640 1 -- 172.21.15.47:0/2383062776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f50870 msgr2=0x557dc8f5a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:53.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.633+0000 7f6af5e00640 1 --2- 172.21.15.47:0/2383062776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f50870 0x557dc8f5a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.633+0000 7f6af5e00640 1 -- 172.21.15.47:0/2383062776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557dc8c937d0 con 0x557dc8f4fe80 2024-03-21T11:40:53.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.633+0000 7f6af6800640 1 --2- 172.21.15.47:0/2383062776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f559e0 0x557dc8f61430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:53.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.634+0000 7f6af5e00640 1 --2- 172.21.15.47:0/2383062776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f4fe80 0x557dc8f59eb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4013fd0 tx=0x7f6ae40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69024b0b871247c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:53.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.634+0000 7f6ad3400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:40:53.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.635+0000 7f6ad3400640 1 -- 172.21.15.47:0/2383062776 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae401de60 con 0x557dc8f4fe80 2024-03-21T11:40:53.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.635+0000 7f6ad3400640 1 -- 172.21.15.47:0/2383062776 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6ae4005020 con 0x557dc8f4fe80 2024-03-21T11:40:53.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.635+0000 7f6afb5da740 1 -- 172.21.15.47:0/2383062776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6aec011b10 con 0x557dc8f4fe80 2024-03-21T11:40:53.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.635+0000 7f6ad3400640 1 -- 172.21.15.47:0/2383062776 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ae4032050 con 0x557dc8f4fe80 2024-03-21T11:40:53.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.635+0000 7f6afb5da740 10 client.20643 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:53.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.635+0000 7f6afb5da740 1 -- 172.21.15.47:0/2383062776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557dc8bc93f0 con 0x557dc8f4fe80 2024-03-21T11:40:53.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.636+0000 7f6ad3400640 1 -- 172.21.15.47:0/2383062776 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6aec011b10 con 0x557dc8f4fe80 2024-03-21T11:40:53.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.637+0000 7f6ad3400640 1 -- 172.21.15.47:0/2383062776 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6ae40274b0 con 0x557dc8f4fe80 2024-03-21T11:40:53.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.637+0000 7f6afb5da740 10 client.20643 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:53.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.637+0000 7f6afb5da740 10 client.20643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:53.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.637+0000 7f6afb5da740 20 client.20643 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:53.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.637+0000 7f6afb5da740 1 --2- 172.21.15.47:0/2383062776 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557dc8f65d40 0x557dc8f681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.637+0000 7f6afb5da740 4 client.20643 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:53.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.637+0000 7f6afb5da740 1 -- 172.21.15.47:0/2383062776 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557dc8ec2790 con 0x557dc8f65d40 2024-03-21T11:40:53.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.637+0000 7f6af5400640 1 --2- 172.21.15.47:0/2383062776 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557dc8f65d40 0x557dc8f681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.638+0000 7f6af5400640 1 --2- 172.21.15.47:0/2383062776 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557dc8f65d40 0x557dc8f681c0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea27a5a7793e5b3d server_cookie=65b81fdb9f883888 in_seq=0 out_seq=0 2024-03-21T11:40:53.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.638+0000 7f6ad3400640 10 client.20643 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:53.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.642+0000 7f6ad3400640 1 -- 172.21.15.47:0/2383062776 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557dc8ec2790 con 0x557dc8f65d40 2024-03-21T11:40:53.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.642+0000 7f6ad3400640 10 client.20643 handle_command_reply: tid=0 2024-03-21T11:40:53.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.643+0000 7f6afb5da740 1 client.20643 shutdown 2024-03-21T11:40:53.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.644+0000 7f6afb5da740 1 -- 172.21.15.47:0/2383062776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f4fe80 msgr2=0x557dc8f59eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:53.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.644+0000 7f6afb5da740 1 --2- 172.21.15.47:0/2383062776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f4fe80 0x557dc8f59eb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4013fd0 tx=0x7f6ae40046c0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.644+0000 7f6afb5da740 1 -- 172.21.15.47:0/2383062776 shutdown_connections 2024-03-21T11:40:53.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.644+0000 7f6afb5da740 1 --2- 172.21.15.47:0/2383062776 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557dc8f65d40 0x557dc8f681c0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.644+0000 7f6afb5da740 1 --2- 172.21.15.47:0/2383062776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f559e0 0x557dc8f61430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.644+0000 7f6afb5da740 1 --2- 172.21.15.47:0/2383062776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f50870 0x557dc8f5a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.644+0000 7f6afb5da740 1 --2- 172.21.15.47:0/2383062776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f4fe80 0x557dc8f59eb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.644+0000 7f6ad3400640 0 client.20643 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:53.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.644+0000 7f6afb5da740 1 -- 172.21.15.47:0/2383062776 >> 172.21.15.47:0/2383062776 conn(0x557dc8f18200 msgr2=0x557dc8f4aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:53.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.646+0000 7f6afb5da740 1 -- 172.21.15.47:0/2383062776 shutdown_connections 2024-03-21T11:40:53.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.647+0000 7f6afb5da740 1 -- 172.21.15.47:0/2383062776 wait complete. 2024-03-21T11:40:53.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.647+0000 7f6afb5da740 20 client.20643 trim_cache size 0 max 16384 2024-03-21T11:40:53.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.684+0000 7f6afb5da740 1 Processor -- start 2024-03-21T11:40:53.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.684+0000 7f6afb5da740 1 -- start start 2024-03-21T11:40:53.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.685+0000 7f6afb5da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f4fe80 0x557dc8f59180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.685+0000 7f6afb5da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f50870 0x557dc8f613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.685+0000 7f6afb5da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f559e0 0x557dc8f519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.685+0000 7f6afb5da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01b0670 con 0x557dc8f50870 2024-03-21T11:40:53.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.685+0000 7f6afb5da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01b1ff0 con 0x557dc8f559e0 2024-03-21T11:40:53.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.685+0000 7f6afb5da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01b0cd0 con 0x557dc8f4fe80 2024-03-21T11:40:53.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.685+0000 7f6af5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f4fe80 0x557dc8f59180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.685+0000 7f6af5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f4fe80 0x557dc8f59180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41934/0 (socket says 172.21.15.47:41934) 2024-03-21T11:40:53.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.685+0000 7f6af5e00640 1 -- 172.21.15.47:0/2869876671 learned_addr learned my addr 172.21.15.47:0/2869876671 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:53.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.686+0000 7f6af5400640 1 --2- 172.21.15.47:0/2869876671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f50870 0x557dc8f613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.686+0000 7f6af6800640 1 --2- 172.21.15.47:0/2869876671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f559e0 0x557dc8f519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.687+0000 7f6af5400640 1 -- 172.21.15.47:0/2869876671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f4fe80 msgr2=0x557dc8f59180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.687+0000 7f6af5400640 1 --2- 172.21.15.47:0/2869876671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f4fe80 0x557dc8f59180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.687+0000 7f6af5400640 1 -- 172.21.15.47:0/2869876671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f559e0 msgr2=0x557dc8f519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.687+0000 7f6af5400640 1 --2- 172.21.15.47:0/2869876671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f559e0 0x557dc8f519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.687+0000 7f6af5400640 1 -- 172.21.15.47:0/2869876671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dc8b890f0 con 0x557dc8f50870 2024-03-21T11:40:53.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.687+0000 7f6af6800640 1 --2- 172.21.15.47:0/2869876671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f559e0 0x557dc8f519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:53.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.688+0000 7f6af5400640 1 --2- 172.21.15.47:0/2869876671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f50870 0x557dc8f613c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00eecc0 tx=0x7f6ae00eeb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f361f773fbf5af51 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:53.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.688+0000 7f6ade600640 1 -- 172.21.15.47:0/2869876671 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae004b040 con 0x557dc8f50870 2024-03-21T11:40:53.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.689+0000 7f6ade600640 1 -- 172.21.15.47:0/2869876671 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ae00bd8a0 con 0x557dc8f50870 2024-03-21T11:40:53.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.689+0000 7f6ade600640 1 -- 172.21.15.47:0/2869876671 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00f1250 con 0x557dc8f50870 2024-03-21T11:40:53.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.690+0000 7f6afb5da740 1 -- 172.21.15.47:0/2869876671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f50870 msgr2=0x557dc8f613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.690+0000 7f6afb5da740 1 --2- 172.21.15.47:0/2869876671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f50870 0x557dc8f613c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00eecc0 tx=0x7f6ae00eeb40 comp rx=0 tx=0).stop 2024-03-21T11:40:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.690+0000 7f6afb5da740 1 -- 172.21.15.47:0/2869876671 shutdown_connections 2024-03-21T11:40:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.691+0000 7f6afb5da740 1 --2- 172.21.15.47:0/2869876671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f559e0 0x557dc8f519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.691+0000 7f6afb5da740 1 --2- 172.21.15.47:0/2869876671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f50870 0x557dc8f613c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.691+0000 7f6afb5da740 1 --2- 172.21.15.47:0/2869876671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f4fe80 0x557dc8f59180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.691+0000 7f6afb5da740 1 -- 172.21.15.47:0/2869876671 >> 172.21.15.47:0/2869876671 conn(0x557dc8f18200 msgr2=0x557dc8f51290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:53.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.691+0000 7f6afb5da740 1 -- 172.21.15.47:0/2869876671 shutdown_connections 2024-03-21T11:40:53.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.691+0000 7f6afb5da740 1 -- 172.21.15.47:0/2869876671 wait complete. 2024-03-21T11:40:53.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.692+0000 7f6afb5da740 1 -- start start 2024-03-21T11:40:53.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.692+0000 7f6afb5da740 1 Processor -- start 2024-03-21T11:40:53.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.693+0000 7f6afb5da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f4fe80 0x557dc8f5cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.694+0000 7f6afb5da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f50870 0x557dc8f5d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.694+0000 7f6afb5da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f559e0 0x557dc8f60500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.694+0000 7f6af5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f50870 0x557dc8f5d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.694+0000 7f6af5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f50870 0x557dc8f5d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41944/0 (socket says 172.21.15.47:41944) 2024-03-21T11:40:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.694+0000 7f6afb5da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01b1990 con 0x557dc8f559e0 2024-03-21T11:40:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.694+0000 7f6afb5da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01b1330 con 0x557dc8f4fe80 2024-03-21T11:40:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.694+0000 7f6afb5da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01ae830 con 0x557dc8f50870 2024-03-21T11:40:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.695+0000 7f6afb5da740 10 client.0 _ll_register_callbacks cb 0x7f6ae400ad80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.695+0000 7f6af6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f559e0 0x557dc8f60500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.695+0000 7f6af5400640 1 -- 172.21.15.47:0/251969504 learned_addr learned my addr 172.21.15.47:0/251969504 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:53.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.695+0000 7f6af5e00640 1 --2- 172.21.15.47:0/251969504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f4fe80 0x557dc8f5cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.696+0000 7f6af5400640 1 -- 172.21.15.47:0/251969504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f4fe80 msgr2=0x557dc8f5cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.696+0000 7f6af5400640 1 --2- 172.21.15.47:0/251969504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f4fe80 0x557dc8f5cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.696+0000 7f6af5400640 1 -- 172.21.15.47:0/251969504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f559e0 msgr2=0x557dc8f60500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.696+0000 7f6af5400640 1 --2- 172.21.15.47:0/251969504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f559e0 0x557dc8f60500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.696+0000 7f6af5400640 1 -- 172.21.15.47:0/251969504 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557dc8f60af0 con 0x557dc8f50870 2024-03-21T11:40:53.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.697+0000 7f6af5400640 1 --2- 172.21.15.47:0/251969504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f50870 0x557dc8f5d4e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00b9d00 tx=0x7f6ae0005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d6a44a4e1d8a685 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:53.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.697+0000 7f6ad3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:40:53.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.697+0000 7f6ad3400640 1 -- 172.21.15.47:0/251969504 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae004b070 con 0x557dc8f50870 2024-03-21T11:40:53.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.697+0000 7f6ad3400640 1 -- 172.21.15.47:0/251969504 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6ae00bedd0 con 0x557dc8f50870 2024-03-21T11:40:53.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.697+0000 7f6afb5da740 1 -- 172.21.15.47:0/251969504 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ae00bd8a0 con 0x557dc8f50870 2024-03-21T11:40:53.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.698+0000 7f6ad3400640 1 -- 172.21.15.47:0/251969504 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ae00eed80 con 0x557dc8f50870 2024-03-21T11:40:53.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.698+0000 7f6afb5da740 10 client.31165 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:53.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.698+0000 7f6afb5da740 1 -- 172.21.15.47:0/251969504 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557dc8fd2b10 con 0x557dc8f50870 2024-03-21T11:40:53.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.699+0000 7f6ad3400640 1 -- 172.21.15.47:0/251969504 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ae00bd8a0 con 0x557dc8f50870 2024-03-21T11:40:53.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.702+0000 7f6ad3400640 1 -- 172.21.15.47:0/251969504 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6ae00f1050 con 0x557dc8f50870 2024-03-21T11:40:53.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.702+0000 7f6afb5da740 10 client.31165 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:53.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.702+0000 7f6afb5da740 10 client.31165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:53.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.702+0000 7f6afb5da740 20 client.31165 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:53.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.702+0000 7f6afb5da740 1 --2- 172.21.15.47:0/251969504 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557dc8fd3230 0x557dc8fd5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:53.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.702+0000 7f6afb5da740 4 client.31165 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:40:53.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.702+0000 7f6afb5da740 1 -- 172.21.15.47:0/251969504 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557dc8ebcae0 con 0x557dc8fd3230 2024-03-21T11:40:53.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.702+0000 7f6af5e00640 1 --2- 172.21.15.47:0/251969504 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557dc8fd3230 0x557dc8fd5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.705+0000 7f6af5e00640 1 --2- 172.21.15.47:0/251969504 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557dc8fd3230 0x557dc8fd5650 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f06aef1e374c96e server_cookie=30a39ae1c78fb11 in_seq=0 out_seq=0 2024-03-21T11:40:53.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.705+0000 7f6ad3400640 10 client.31165 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:53.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.709+0000 7f6ad3400640 1 -- 172.21.15.47:0/251969504 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557dc8ebcae0 con 0x557dc8fd3230 2024-03-21T11:40:53.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.710+0000 7f6ad3400640 10 client.31165 handle_command_reply: tid=0 2024-03-21T11:40:53.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.710+0000 7f6afb5da740 1 client.31165 shutdown 2024-03-21T11:40:53.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.710+0000 7f6afb5da740 1 -- 172.21.15.47:0/251969504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f50870 msgr2=0x557dc8f5d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.710+0000 7f6afb5da740 1 --2- 172.21.15.47:0/251969504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f50870 0x557dc8f5d4e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00b9d00 tx=0x7f6ae0005be0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.711+0000 7f6afb5da740 1 -- 172.21.15.47:0/251969504 shutdown_connections 2024-03-21T11:40:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.711+0000 7f6afb5da740 1 --2- 172.21.15.47:0/251969504 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557dc8fd3230 0x557dc8fd5650 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.711+0000 7f6afb5da740 1 --2- 172.21.15.47:0/251969504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557dc8f559e0 0x557dc8f60500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.711+0000 7f6afb5da740 1 --2- 172.21.15.47:0/251969504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dc8f50870 0x557dc8f5d4e0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.711+0000 7f6afb5da740 1 --2- 172.21.15.47:0/251969504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557dc8f4fe80 0x557dc8f5cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.711+0000 7f6afb5da740 1 -- 172.21.15.47:0/251969504 >> 172.21.15.47:0/251969504 conn(0x557dc8f18200 msgr2=0x557dc8f51b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.711+0000 7f6ad3400640 0 client.31165 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:53.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.711+0000 7f6afb5da740 1 -- 172.21.15.47:0/251969504 shutdown_connections 2024-03-21T11:40:53.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.712+0000 7f6afb5da740 1 -- 172.21.15.47:0/251969504 wait complete. 2024-03-21T11:40:53.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.712+0000 7f6afb5da740 20 client.31165 trim_cache size 0 max 16384 2024-03-21T11:40:53.712 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:40:53.713 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:40:53.713 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:40:53.713 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:40:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.714+0000 7f6af8800640 1 -- 172.21.15.47:0/656547324 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ad80a2fa0 msgr2=0x7f6ad80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.714+0000 7f6af8800640 1 --2- 172.21.15.47:0/656547324 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ad80a2fa0 0x7f6ad80a5460 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x557dc8f50c70 tx=0x7f6ae400a2d0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.714+0000 7f6af8800640 1 -- 172.21.15.47:0/656547324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af0102880 msgr2=0x7f6af019e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.714+0000 7f6af8800640 1 --2- 172.21.15.47:0/656547324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af0102880 0x7f6af019e4f0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009230 tx=0x7f6ae0004040 comp rx=0 tx=0).stop 2024-03-21T11:40:53.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.715+0000 7f6af8800640 1 -- 172.21.15.47:0/656547324 shutdown_connections 2024-03-21T11:40:53.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.715+0000 7f6af8800640 1 --2- 172.21.15.47:0/656547324 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ad80a2fa0 0x7f6ad80a5460 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.715+0000 7f6af8800640 1 --2- 172.21.15.47:0/656547324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af0105230 0x7f6af01a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.715+0000 7f6af8800640 1 --2- 172.21.15.47:0/656547324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af0102880 0x7f6af019e4f0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.715+0000 7f6af8800640 1 --2- 172.21.15.47:0/656547324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6af00fff10 0x7f6af019dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:53.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.715+0000 7f6af8800640 1 -- 172.21.15.47:0/656547324 >> 172.21.15.47:0/656547324 conn(0x7f6af00fbb30 msgr2=0x7f6af0100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:53.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.715+0000 7f6af8800640 1 -- 172.21.15.47:0/656547324 shutdown_connections 2024-03-21T11:40:53.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:53.715+0000 7f6af8800640 1 -- 172.21.15.47:0/656547324 wait complete. 2024-03-21T11:40:53.750 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:40:53.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:40:53.945 INFO:tasks.workunit.client.0.smithi047.stdout: 10 102303 24.36 MB/sec execute 97 sec latency 259.019 ms 2024-03-21T11:40:54.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.074+0000 7f3053200640 1 -- 172.21.15.47:0/66140716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304c107b00 msgr2=0x7f304c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:54.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.074+0000 7f3053200640 1 --2- 172.21.15.47:0/66140716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304c107b00 0x7f304c107f20 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3040009900 tx=0x7f304002bc30 comp rx=0 tx=0).stop 2024-03-21T11:40:54.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.075+0000 7f3053200640 1 -- 172.21.15.47:0/66140716 shutdown_connections 2024-03-21T11:40:54.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.075+0000 7f3053200640 1 --2- 172.21.15.47:0/66140716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f304c1096b0 0x7f304c10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.075+0000 7f3053200640 1 --2- 172.21.15.47:0/66140716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f304c108cf0 0x7f304c109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.075+0000 7f3053200640 1 --2- 172.21.15.47:0/66140716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304c107b00 0x7f304c107f20 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.075+0000 7f3053200640 1 -- 172.21.15.47:0/66140716 >> 172.21.15.47:0/66140716 conn(0x7f304c076aa0 msgr2=0x7f304c078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:54.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.075+0000 7f3053200640 1 -- 172.21.15.47:0/66140716 shutdown_connections 2024-03-21T11:40:54.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.075+0000 7f3053200640 1 -- 172.21.15.47:0/66140716 wait complete. 2024-03-21T11:40:54.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.077+0000 7f3053200640 1 Processor -- start 2024-03-21T11:40:54.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.077+0000 7f3053200640 1 -- start start 2024-03-21T11:40:54.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.077+0000 7f3053200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304c107b00 0x7f304c0700b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.077+0000 7f3053200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f304c108cf0 0x7f304c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.077+0000 7f3053200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f304c1096b0 0x7f304c1adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.077+0000 7f3053200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1137d0 con 0x7f304c108cf0 2024-03-21T11:40:54.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.077+0000 7f3053200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c113650 con 0x7f304c107b00 2024-03-21T11:40:54.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.077+0000 7f3053200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c113950 con 0x7f304c1096b0 2024-03-21T11:40:54.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.077+0000 7f3051e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304c107b00 0x7f304c0700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.078+0000 7f3051e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304c107b00 0x7f304c0700b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47002/0 (socket says 172.21.15.47:47002) 2024-03-21T11:40:54.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.078+0000 7f3051e00640 1 -- 172.21.15.47:0/3151082150 learned_addr learned my addr 172.21.15.47:0/3151082150 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:54.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.078+0000 7f3052800640 1 --2- 172.21.15.47:0/3151082150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f304c1096b0 0x7f304c1adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.078+0000 7f3051400640 1 --2- 172.21.15.47:0/3151082150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f304c108cf0 0x7f304c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.079+0000 7f3051e00640 1 -- 172.21.15.47:0/3151082150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f304c1096b0 msgr2=0x7f304c1adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:54.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.079+0000 7f3051e00640 1 --2- 172.21.15.47:0/3151082150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f304c1096b0 0x7f304c1adee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.079+0000 7f3051e00640 1 -- 172.21.15.47:0/3151082150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f304c108cf0 msgr2=0x7f304c0705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:54.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.079+0000 7f3051e00640 1 --2- 172.21.15.47:0/3151082150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f304c108cf0 0x7f304c0705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.079+0000 7f3051e00640 1 -- 172.21.15.47:0/3151082150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c1ae420 con 0x7f304c107b00 2024-03-21T11:40:54.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.080+0000 7f3051e00640 1 --2- 172.21.15.47:0/3151082150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304c107b00 0x7f304c0700b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3040005ab0 tx=0x7f3040005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:54.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.080+0000 7f3046a00640 1 -- 172.21.15.47:0/3151082150 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040041070 con 0x7f304c107b00 2024-03-21T11:40:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.081+0000 7f3053200640 1 -- 172.21.15.47:0/3151082150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c1ae5f0 con 0x7f304c107b00 2024-03-21T11:40:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.081+0000 7f3053200640 1 -- 172.21.15.47:0/3151082150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c1aeab0 con 0x7f304c107b00 2024-03-21T11:40:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.081+0000 7f3046a00640 1 -- 172.21.15.47:0/3151082150 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30400054e0 con 0x7f304c107b00 2024-03-21T11:40:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.081+0000 7f3046a00640 1 -- 172.21.15.47:0/3151082150 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040041070 con 0x7f304c107b00 2024-03-21T11:40:54.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.082+0000 7f3053200640 1 -- 172.21.15.47:0/3151082150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304c10ee20 con 0x7f304c107b00 2024-03-21T11:40:54.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.088+0000 7f3046a00640 1 -- 172.21.15.47:0/3151082150 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3040044070 con 0x7f304c107b00 2024-03-21T11:40:54.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.094+0000 7f3046a00640 1 --2- 172.21.15.47:0/3151082150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f30340a31a0 0x7f30340a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.094+0000 7f3046a00640 1 -- 172.21.15.47:0/3151082150 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f30400f1320 con 0x7f304c107b00 2024-03-21T11:40:54.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.094+0000 7f3046a00640 1 -- 172.21.15.47:0/3151082150 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f30400f1840 con 0x7f304c107b00 2024-03-21T11:40:54.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.095+0000 7f3051400640 1 --2- 172.21.15.47:0/3151082150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f30340a31a0 0x7f30340a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.097+0000 7f3051400640 1 --2- 172.21.15.47:0/3151082150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f30340a31a0 0x7f30340a5660 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f304c0715c0 tx=0x7f303c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.449+0000 7f3053200640 1 -- 172.21.15.47:0/3151082150 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f304c02d070 con 0x7f304c107b00 2024-03-21T11:40:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.450+0000 7f3046a00640 1 -- 172.21.15.47:0/3151082150 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f30400bddc0 con 0x7f304c107b00 2024-03-21T11:40:54.453 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:40:54.453 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:39:47.303461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:40:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-21T11:40:54.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.455+0000 7f3053200640 1 -- 172.21.15.47:0/3151082150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f30340a31a0 msgr2=0x7f30340a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:54.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.455+0000 7f3053200640 1 --2- 172.21.15.47:0/3151082150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f30340a31a0 0x7f30340a5660 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f304c0715c0 tx=0x7f303c008040 comp rx=0 tx=0).stop 2024-03-21T11:40:54.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.455+0000 7f3053200640 1 -- 172.21.15.47:0/3151082150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304c107b00 msgr2=0x7f304c0700b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:54.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.455+0000 7f3053200640 1 --2- 172.21.15.47:0/3151082150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304c107b00 0x7f304c0700b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3040005ab0 tx=0x7f3040005dc0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.456+0000 7f3053200640 1 -- 172.21.15.47:0/3151082150 shutdown_connections 2024-03-21T11:40:54.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.456+0000 7f3053200640 1 --2- 172.21.15.47:0/3151082150 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f30340a31a0 0x7f30340a5660 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.456+0000 7f3053200640 1 --2- 172.21.15.47:0/3151082150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f304c1096b0 0x7f304c1adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.456+0000 7f3053200640 1 --2- 172.21.15.47:0/3151082150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f304c108cf0 0x7f304c0705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.456+0000 7f3053200640 1 --2- 172.21.15.47:0/3151082150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f304c107b00 0x7f304c0700b0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.456+0000 7f3053200640 1 -- 172.21.15.47:0/3151082150 >> 172.21.15.47:0/3151082150 conn(0x7f304c076aa0 msgr2=0x7f304c10bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:54.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.456+0000 7f3053200640 1 -- 172.21.15.47:0/3151082150 shutdown_connections 2024-03-21T11:40:54.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.456+0000 7f3053200640 1 -- 172.21.15.47:0/3151082150 wait complete. 2024-03-21T11:40:54.484 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:40:54.484 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:40:54.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:54 smithi047 ceph-mon[31351]: pgmap v927: 129 pgs: 129 active+clean; 442 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.0 MiB/s wr, 652 op/s 2024-03-21T11:40:54.783 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:54 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3151082150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:40:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.790+0000 7f17d4600640 1 -- 172.21.15.47:0/3040630699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17cc101270 msgr2=0x7f17cc1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:54.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.790+0000 7f17d4600640 1 --2- 172.21.15.47:0/3040630699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17cc101270 0x7f17cc1016f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009ca0 tx=0x7f17c402c120 comp rx=0 tx=0).stop 2024-03-21T11:40:54.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.792+0000 7f17d4600640 1 -- 172.21.15.47:0/3040630699 shutdown_connections 2024-03-21T11:40:54.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.792+0000 7f17d4600640 1 --2- 172.21.15.47:0/3040630699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17cc101270 0x7f17cc1016f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.792+0000 7f17d4600640 1 --2- 172.21.15.47:0/3040630699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17cc1008d0 0x7f17cc100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.792+0000 7f17d4600640 1 --2- 172.21.15.47:0/3040630699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17cc0fff70 0x7f17cc100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.792+0000 7f17d4600640 1 -- 172.21.15.47:0/3040630699 >> 172.21.15.47:0/3040630699 conn(0x7f17cc0fbb30 msgr2=0x7f17cc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:54.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.792+0000 7f17d4600640 1 -- 172.21.15.47:0/3040630699 shutdown_connections 2024-03-21T11:40:54.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.792+0000 7f17d4600640 1 -- 172.21.15.47:0/3040630699 wait complete. 2024-03-21T11:40:54.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.795+0000 7f17d4600640 1 Processor -- start 2024-03-21T11:40:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.795+0000 7f17d4600640 1 -- start start 2024-03-21T11:40:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.795+0000 7f17d4600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17cc0fff70 0x7f17cc105af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.795+0000 7f17d4600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17cc1008d0 0x7f17cc106030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.795+0000 7f17d4600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17cc101270 0x7f17cc102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.795+0000 7f17d4600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc10f4a0 con 0x7f17cc0fff70 2024-03-21T11:40:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.795+0000 7f17d4600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc10f320 con 0x7f17cc1008d0 2024-03-21T11:40:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.795+0000 7f17d4600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc10f620 con 0x7f17cc101270 2024-03-21T11:40:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.796+0000 7f17d1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17cc0fff70 0x7f17cc105af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.796+0000 7f17d2600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17cc101270 0x7f17cc102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.796+0000 7f17d2600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17cc101270 0x7f17cc102660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41956/0 (socket says 172.21.15.47:41956) 2024-03-21T11:40:54.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.796+0000 7f17d2600640 1 -- 172.21.15.47:0/1417176574 learned_addr learned my addr 172.21.15.47:0/1417176574 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.796+0000 7f17d1200640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17cc1008d0 0x7f17cc106030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.796+0000 7f17d2600640 1 -- 172.21.15.47:0/1417176574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17cc1008d0 msgr2=0x7f17cc106030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.796+0000 7f17d2600640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17cc1008d0 0x7f17cc106030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.796+0000 7f17d2600640 1 -- 172.21.15.47:0/1417176574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17cc0fff70 msgr2=0x7f17cc105af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.796+0000 7f17d2600640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17cc0fff70 0x7f17cc105af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.796+0000 7f17d2600640 1 -- 172.21.15.47:0/1417176574 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc102f20 con 0x7f17cc101270 2024-03-21T11:40:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.796+0000 7f17d1200640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17cc1008d0 0x7f17cc106030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.796+0000 7f17d1c00640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17cc0fff70 0x7f17cc105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:54.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.797+0000 7f17d2600640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17cc101270 0x7f17cc102660 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009430 tx=0x7f17c40328c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.804+0000 7f17ca000640 1 -- 172.21.15.47:0/1417176574 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c4039040 con 0x7f17cc101270 2024-03-21T11:40:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.804+0000 7f17ca000640 1 -- 172.21.15.47:0/1417176574 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17c4044070 con 0x7f17cc101270 2024-03-21T11:40:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.804+0000 7f17ca000640 1 -- 172.21.15.47:0/1417176574 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c4004850 con 0x7f17cc101270 2024-03-21T11:40:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.804+0000 7f17d4600640 1 -- 172.21.15.47:0/1417176574 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17cc199590 con 0x7f17cc101270 2024-03-21T11:40:54.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.805+0000 7f17d4600640 1 -- 172.21.15.47:0/1417176574 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc199a30 con 0x7f17cc101270 2024-03-21T11:40:54.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.816+0000 7f17ca000640 1 -- 172.21.15.47:0/1417176574 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f17c4049460 con 0x7f17cc101270 2024-03-21T11:40:54.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.816+0000 7f17d7399740 1 Processor -- start 2024-03-21T11:40:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.817+0000 7f17d7399740 1 -- start start 2024-03-21T11:40:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.818+0000 7f17d7399740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fa7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.818+0000 7f17d7399740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa89e0 0x55cb95fa8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.818+0000 7f17d7399740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 0x55cb95fafc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.818+0000 7f17d7399740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb95f0c2c0 con 0x55cb95fa93a0 2024-03-21T11:40:54.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.818+0000 7f17d7399740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb95eb4c90 con 0x55cb95fa7810 2024-03-21T11:40:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.818+0000 7f17d7399740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb95f08360 con 0x55cb95fa89e0 2024-03-21T11:40:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.818+0000 7f17d1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fa7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.818+0000 7f17d1c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa89e0 0x55cb95fa8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.818+0000 7f17d1c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa89e0 0x55cb95fa8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41968/0 (socket says 172.21.15.47:41968) 2024-03-21T11:40:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.818+0000 7f17d1c00640 1 -- 172.21.15.47:0/964230822 learned_addr learned my addr 172.21.15.47:0/964230822 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.818+0000 7f17d1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fa7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47024/0 (socket says 172.21.15.47:47024) 2024-03-21T11:40:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.819+0000 7f17d1200640 1 --2- 172.21.15.47:0/964230822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 0x55cb95fafc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.819+0000 7f17d1200640 1 -- 172.21.15.47:0/964230822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa89e0 msgr2=0x55cb95fa8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.819+0000 7f17d1200640 1 --2- 172.21.15.47:0/964230822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa89e0 0x55cb95fa8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.819+0000 7f17d1200640 1 -- 172.21.15.47:0/964230822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 msgr2=0x55cb95fafc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.819+0000 7f17d1200640 1 --2- 172.21.15.47:0/964230822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 0x55cb95fafc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.819+0000 7f17d1200640 1 -- 172.21.15.47:0/964230822 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb95ed9080 con 0x55cb95fa7810 2024-03-21T11:40:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.820+0000 7f17d1200640 1 --2- 172.21.15.47:0/964230822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fa7c30 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f17c00058c0 tx=0x7f17c000d360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d80b31522d98db2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.821+0000 7f17bea00640 1 -- 172.21.15.47:0/964230822 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c000a850 con 0x55cb95fa7810 2024-03-21T11:40:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.821+0000 7f17ca000640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17b40a3110 0x7f17b40a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.821+0000 7f17bea00640 1 -- 172.21.15.47:0/964230822 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17c00103d0 con 0x55cb95fa7810 2024-03-21T11:40:54.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.821+0000 7f17bea00640 1 -- 172.21.15.47:0/964230822 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c000f620 con 0x55cb95fa7810 2024-03-21T11:40:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.821+0000 7f17ca000640 1 -- 172.21.15.47:0/1417176574 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f17c40f2370 con 0x7f17cc101270 2024-03-21T11:40:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.822+0000 7f17d1c00640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17b40a3110 0x7f17b40a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.822+0000 7f17d7399740 1 -- 172.21.15.47:0/964230822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 msgr2=0x55cb95fa7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.822+0000 7f17d7399740 1 --2- 172.21.15.47:0/964230822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fa7c30 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f17c00058c0 tx=0x7f17c000d360 comp rx=0 tx=0).stop 2024-03-21T11:40:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.822+0000 7f17d7399740 1 -- 172.21.15.47:0/964230822 shutdown_connections 2024-03-21T11:40:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.822+0000 7f17d7399740 1 --2- 172.21.15.47:0/964230822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 0x55cb95fafc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.822+0000 7f17d7399740 1 --2- 172.21.15.47:0/964230822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa89e0 0x55cb95fa8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.822+0000 7f17d7399740 1 --2- 172.21.15.47:0/964230822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fa7c30 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.822+0000 7f17d7399740 1 -- 172.21.15.47:0/964230822 >> 172.21.15.47:0/964230822 conn(0x55cb95fa2c00 msgr2=0x55cb95fa3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.823+0000 7f17d7399740 1 -- 172.21.15.47:0/964230822 shutdown_connections 2024-03-21T11:40:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.823+0000 7f17d1c00640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17b40a3110 0x7f17b40a55d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x55cb95fa3b70 tx=0x7f17b800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.823+0000 7f17d7399740 1 -- 172.21.15.47:0/964230822 wait complete. 2024-03-21T11:40:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.824+0000 7f17d7399740 1 -- start start 2024-03-21T11:40:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.824+0000 7f17d7399740 1 Processor -- start 2024-03-21T11:40:54.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.825+0000 7f17d7399740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fb1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.825+0000 7f17d7399740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa89e0 0x55cb95fb1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.825+0000 7f17d7399740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 0x55cb95fb9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.825+0000 7f17d7399740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb95fb03b0 con 0x55cb95fa93a0 2024-03-21T11:40:54.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.826+0000 7f17d7399740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb95ec80d0 con 0x55cb95fa7810 2024-03-21T11:40:54.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.826+0000 7f17d7399740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb95fb0530 con 0x55cb95fa89e0 2024-03-21T11:40:54.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.826+0000 7f17d7399740 10 client.0 _ll_register_callbacks cb 0x55cb95ecce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:54.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.826+0000 7f17d1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa89e0 0x55cb95fb1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.826+0000 7f17d1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa89e0 0x55cb95fb1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41980/0 (socket says 172.21.15.47:41980) 2024-03-21T11:40:54.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.826+0000 7f17d1200640 1 -- 172.21.15.47:0/1571609807 learned_addr learned my addr 172.21.15.47:0/1571609807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:54.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.826+0000 7f17d1c00640 1 --2- 172.21.15.47:0/1571609807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fb1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.826+0000 7f17d2600640 1 --2- 172.21.15.47:0/1571609807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 0x55cb95fb9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.826+0000 7f17d1200640 1 -- 172.21.15.47:0/1571609807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 msgr2=0x55cb95fb1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:54.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.826+0000 7f17d1200640 1 --2- 172.21.15.47:0/1571609807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fb1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.826+0000 7f17d1200640 1 -- 172.21.15.47:0/1571609807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 msgr2=0x55cb95fb9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:54.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.826+0000 7f17d1200640 1 --2- 172.21.15.47:0/1571609807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 0x55cb95fb9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.826+0000 7f17d1200640 1 -- 172.21.15.47:0/1571609807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb95c203f0 con 0x55cb95fa89e0 2024-03-21T11:40:54.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.827+0000 7f17d1200640 1 --2- 172.21.15.47:0/1571609807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa89e0 0x55cb95fb1ff0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f17c000d330 tx=0x7f17c0005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93193a746e36e59b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:54.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.827+0000 7f17b3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:40:54.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.828+0000 7f17b3400640 1 -- 172.21.15.47:0/1571609807 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c002a060 con 0x55cb95fa89e0 2024-03-21T11:40:54.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.829+0000 7f17b3400640 1 -- 172.21.15.47:0/1571609807 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f17c0012250 con 0x55cb95fa89e0 2024-03-21T11:40:54.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.829+0000 7f17d7399740 1 -- 172.21.15.47:0/1571609807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17c00103d0 con 0x55cb95fa89e0 2024-03-21T11:40:54.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.830+0000 7f17b3400640 1 -- 172.21.15.47:0/1571609807 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17c0005d10 con 0x55cb95fa89e0 2024-03-21T11:40:54.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.830+0000 7f17d7399740 10 client.31180 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:54.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.830+0000 7f17d7399740 1 -- 172.21.15.47:0/1571609807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55cb95cea7d0 con 0x55cb95fa89e0 2024-03-21T11:40:54.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.830+0000 7f17b3400640 1 -- 172.21.15.47:0/1571609807 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17c00103d0 con 0x55cb95fa89e0 2024-03-21T11:40:54.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.831+0000 7f17b3400640 1 -- 172.21.15.47:0/1571609807 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f17c000abd0 con 0x55cb95fa89e0 2024-03-21T11:40:54.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.831+0000 7f17d7399740 10 client.31180 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:54.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.831+0000 7f17d7399740 10 client.31180 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:40:54.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.831+0000 7f17d7399740 20 client.31180 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:54.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.831+0000 7f17d7399740 1 --2- 172.21.15.47:0/1571609807 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55cb95fbd240 0x55cb95fbf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.831+0000 7f17d7399740 4 client.31180 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:54.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.831+0000 7f17d7399740 1 -- 172.21.15.47:0/1571609807 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb95f19800 con 0x55cb95fbd240 2024-03-21T11:40:54.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.833+0000 7f17d1c00640 1 --2- 172.21.15.47:0/1571609807 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55cb95fbd240 0x55cb95fbf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.834+0000 7f17d1c00640 1 --2- 172.21.15.47:0/1571609807 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55cb95fbd240 0x55cb95fbf660 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5f72ccad55ed5b28 server_cookie=7b94e8256bd81259 in_seq=0 out_seq=0 2024-03-21T11:40:54.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.834+0000 7f17b3400640 10 client.31180 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:40:54.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.835+0000 7f17b3400640 1 -- 172.21.15.47:0/1571609807 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cb95f19800 con 0x55cb95fbd240 2024-03-21T11:40:54.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.835+0000 7f17b3400640 10 client.31180 handle_command_reply: tid=0 2024-03-21T11:40:54.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.835+0000 7f17d7399740 1 client.31180 shutdown 2024-03-21T11:40:54.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.835+0000 7f17d7399740 1 -- 172.21.15.47:0/1571609807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa89e0 msgr2=0x55cb95fb1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:54.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.835+0000 7f17d7399740 1 --2- 172.21.15.47:0/1571609807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa89e0 0x55cb95fb1ff0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f17c000d330 tx=0x7f17c0005b20 comp rx=0 tx=0).stop 2024-03-21T11:40:54.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.836+0000 7f17d7399740 1 -- 172.21.15.47:0/1571609807 shutdown_connections 2024-03-21T11:40:54.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.836+0000 7f17d7399740 1 --2- 172.21.15.47:0/1571609807 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55cb95fbd240 0x55cb95fbf660 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.836+0000 7f17d7399740 1 --2- 172.21.15.47:0/1571609807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 0x55cb95fb9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.836+0000 7f17d7399740 1 --2- 172.21.15.47:0/1571609807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa89e0 0x55cb95fb1ff0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.836+0000 7f17d7399740 1 --2- 172.21.15.47:0/1571609807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fb1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.836+0000 7f17d7399740 1 -- 172.21.15.47:0/1571609807 >> 172.21.15.47:0/1571609807 conn(0x55cb95fa2c00 msgr2=0x55cb95fa1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:54.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.836+0000 7f17b3400640 0 client.31180 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:40:54.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.836+0000 7f17d7399740 1 -- 172.21.15.47:0/1571609807 shutdown_connections 2024-03-21T11:40:54.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.837+0000 7f17d7399740 1 -- 172.21.15.47:0/1571609807 wait complete. 2024-03-21T11:40:54.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.837+0000 7f17d7399740 20 client.31180 trim_cache size 0 max 16384 2024-03-21T11:40:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.876+0000 7f17d7399740 1 Processor -- start 2024-03-21T11:40:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.876+0000 7f17d7399740 1 -- start start 2024-03-21T11:40:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.877+0000 7f17d7399740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fa6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.877+0000 7f17d7399740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa89e0 0x55cb95fb8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.877+0000 7f17d7399740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa93a0 0x55cb95faba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.877+0000 7f17d7399740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc06cfa0 con 0x55cb95fa89e0 2024-03-21T11:40:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.877+0000 7f17d7399740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc06e920 con 0x55cb95fa7810 2024-03-21T11:40:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.877+0000 7f17d7399740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc06d600 con 0x55cb95fa93a0 2024-03-21T11:40:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.878+0000 7f17d2600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa93a0 0x55cb95faba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.878+0000 7f17d1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fa6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.878+0000 7f17d1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fa6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47038/0 (socket says 172.21.15.47:47038) 2024-03-21T11:40:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.878+0000 7f17d1c00640 1 -- 172.21.15.47:0/2778568391 learned_addr learned my addr 172.21.15.47:0/2778568391 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.878+0000 7f17d1200640 1 --2- 172.21.15.47:0/2778568391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa89e0 0x55cb95fb8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.878+0000 7f17d1c00640 1 -- 172.21.15.47:0/2778568391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa93a0 msgr2=0x55cb95faba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.878+0000 7f17d1c00640 1 --2- 172.21.15.47:0/2778568391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa93a0 0x55cb95faba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.878+0000 7f17d1c00640 1 -- 172.21.15.47:0/2778568391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa89e0 msgr2=0x55cb95fb8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.878+0000 7f17d1c00640 1 --2- 172.21.15.47:0/2778568391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa89e0 0x55cb95fb8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.878+0000 7f17d1c00640 1 -- 172.21.15.47:0/2778568391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb95faf570 con 0x55cb95fa7810 2024-03-21T11:40:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.879+0000 7f17d1c00640 1 --2- 172.21.15.47:0/2778568391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fa6570 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f17b800fb70 tx=0x7f17b8015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4be2adfa6e8ece59 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.880+0000 7f17be000640 1 -- 172.21.15.47:0/2778568391 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b8039c70 con 0x55cb95fa7810 2024-03-21T11:40:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.880+0000 7f17be000640 1 -- 172.21.15.47:0/2778568391 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17b8039e10 con 0x55cb95fa7810 2024-03-21T11:40:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.880+0000 7f17be000640 1 -- 172.21.15.47:0/2778568391 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b8038610 con 0x55cb95fa7810 2024-03-21T11:40:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.881+0000 7f17d7399740 1 -- 172.21.15.47:0/2778568391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 msgr2=0x55cb95fa6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.881+0000 7f17d7399740 1 --2- 172.21.15.47:0/2778568391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fa6570 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f17b800fb70 tx=0x7f17b8015000 comp rx=0 tx=0).stop 2024-03-21T11:40:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.881+0000 7f17d7399740 1 -- 172.21.15.47:0/2778568391 shutdown_connections 2024-03-21T11:40:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.881+0000 7f17d7399740 1 --2- 172.21.15.47:0/2778568391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa93a0 0x55cb95faba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.881+0000 7f17d7399740 1 --2- 172.21.15.47:0/2778568391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa89e0 0x55cb95fb8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.881+0000 7f17d7399740 1 --2- 172.21.15.47:0/2778568391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa7810 0x55cb95fa6570 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.882+0000 7f17d7399740 1 -- 172.21.15.47:0/2778568391 >> 172.21.15.47:0/2778568391 conn(0x55cb95fa2c00 msgr2=0x55cb95fa5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.882+0000 7f17d7399740 1 -- 172.21.15.47:0/2778568391 shutdown_connections 2024-03-21T11:40:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.882+0000 7f17d7399740 1 -- 172.21.15.47:0/2778568391 wait complete. 2024-03-21T11:40:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.883+0000 7f17d7399740 1 -- start start 2024-03-21T11:40:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.884+0000 7f17d7399740 1 Processor -- start 2024-03-21T11:40:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.885+0000 7f17d7399740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa7810 0x55cb95fb45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.885+0000 7f17d7399740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa89e0 0x55cb95fb4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.885+0000 7f17d7399740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 0x55cb95fb7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.886+0000 7f17d7399740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc06e2c0 con 0x55cb95fa93a0 2024-03-21T11:40:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.886+0000 7f17d7399740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc06dc60 con 0x55cb95fa89e0 2024-03-21T11:40:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.886+0000 7f17d7399740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc06b180 con 0x55cb95fa7810 2024-03-21T11:40:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.886+0000 7f17d7399740 10 client.0 _ll_register_callbacks cb 0x7f17c000dd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.886+0000 7f17d1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa89e0 0x55cb95fb4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.886+0000 7f17d1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa89e0 0x55cb95fb4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47044/0 (socket says 172.21.15.47:47044) 2024-03-21T11:40:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.886+0000 7f17d1200640 1 -- 172.21.15.47:0/2218893512 learned_addr learned my addr 172.21.15.47:0/2218893512 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.886+0000 7f17d1c00640 1 --2- 172.21.15.47:0/2218893512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa7810 0x55cb95fb45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.886+0000 7f17d2600640 1 --2- 172.21.15.47:0/2218893512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 0x55cb95fb7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.887+0000 7f17d1200640 1 -- 172.21.15.47:0/2218893512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa7810 msgr2=0x55cb95fb45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.887+0000 7f17d1200640 1 --2- 172.21.15.47:0/2218893512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa7810 0x55cb95fb45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.887+0000 7f17d1200640 1 -- 172.21.15.47:0/2218893512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 msgr2=0x55cb95fb7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.887+0000 7f17d1200640 1 --2- 172.21.15.47:0/2218893512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 0x55cb95fb7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.887+0000 7f17d1200640 1 -- 172.21.15.47:0/2218893512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb95fb8040 con 0x55cb95fa89e0 2024-03-21T11:40:54.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.887+0000 7f17d1c00640 1 --2- 172.21.15.47:0/2218893512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa7810 0x55cb95fb45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.889+0000 7f17d1200640 1 --2- 172.21.15.47:0/2218893512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa89e0 0x55cb95fb4af0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f17c000daa0 tx=0x7f17c0012480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=552ac0e3ff11cbdd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.889+0000 7f17b3400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.890+0000 7f17b3400640 1 -- 172.21.15.47:0/2218893512 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c002a050 con 0x55cb95fa89e0 2024-03-21T11:40:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.890+0000 7f17b3400640 1 -- 172.21.15.47:0/2218893512 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f17c0011df0 con 0x55cb95fa89e0 2024-03-21T11:40:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.890+0000 7f17d7399740 1 -- 172.21.15.47:0/2218893512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17b8039e10 con 0x55cb95fa89e0 2024-03-21T11:40:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.891+0000 7f17b3400640 1 -- 172.21.15.47:0/2218893512 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17c0034080 con 0x55cb95fa89e0 2024-03-21T11:40:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.891+0000 7f17d7399740 10 client.31208 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.891+0000 7f17d7399740 1 -- 172.21.15.47:0/2218893512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55cb96018af0 con 0x55cb95fa89e0 2024-03-21T11:40:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.891+0000 7f17b3400640 1 -- 172.21.15.47:0/2218893512 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17b8039e10 con 0x55cb95fa89e0 2024-03-21T11:40:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.892+0000 7f17b3400640 1 -- 172.21.15.47:0/2218893512 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f17c00047c0 con 0x55cb95fa89e0 2024-03-21T11:40:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.892+0000 7f17d7399740 10 client.31208 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.892+0000 7f17d7399740 10 client.31208 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:40:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.892+0000 7f17d7399740 20 client.31208 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.892+0000 7f17d7399740 1 --2- 172.21.15.47:0/2218893512 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55cb960191f0 0x55cb9601b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.892+0000 7f17d7399740 4 client.31208 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:40:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.892+0000 7f17d7399740 1 -- 172.21.15.47:0/2218893512 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb95f13b50 con 0x55cb960191f0 2024-03-21T11:40:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.892+0000 7f17d1c00640 1 --2- 172.21.15.47:0/2218893512 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55cb960191f0 0x55cb9601b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:54.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.893+0000 7f17d1c00640 1 --2- 172.21.15.47:0/2218893512 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55cb960191f0 0x55cb9601b5e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=93c998a361642ce6 server_cookie=c51197cc77c6e13f in_seq=0 out_seq=0 2024-03-21T11:40:54.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.893+0000 7f17b3400640 10 client.31208 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:40:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.895+0000 7f17b3400640 1 -- 172.21.15.47:0/2218893512 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb95f13b50 con 0x55cb960191f0 2024-03-21T11:40:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.895+0000 7f17b3400640 10 client.31208 handle_command_reply: tid=0 2024-03-21T11:40:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.895+0000 7f17d7399740 1 client.31208 shutdown 2024-03-21T11:40:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.896+0000 7f17d7399740 1 -- 172.21.15.47:0/2218893512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa89e0 msgr2=0x55cb95fb4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.896+0000 7f17d7399740 1 --2- 172.21.15.47:0/2218893512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa89e0 0x55cb95fb4af0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f17c000daa0 tx=0x7f17c0012480 comp rx=0 tx=0).stop 2024-03-21T11:40:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.897+0000 7f17d7399740 1 -- 172.21.15.47:0/2218893512 shutdown_connections 2024-03-21T11:40:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.897+0000 7f17d7399740 1 --2- 172.21.15.47:0/2218893512 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55cb960191f0 0x55cb9601b5e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.897+0000 7f17d7399740 1 --2- 172.21.15.47:0/2218893512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb95fa93a0 0x55cb95fb7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.897+0000 7f17b3400640 0 client.31208 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:40:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.897+0000 7f17d7399740 1 --2- 172.21.15.47:0/2218893512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cb95fa89e0 0x55cb95fb4af0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.897+0000 7f17d7399740 1 --2- 172.21.15.47:0/2218893512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cb95fa7810 0x55cb95fb45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.897+0000 7f17d7399740 1 -- 172.21.15.47:0/2218893512 >> 172.21.15.47:0/2218893512 conn(0x55cb95fa2c00 msgr2=0x55cb95fa66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.899+0000 7f17d7399740 1 -- 172.21.15.47:0/2218893512 shutdown_connections 2024-03-21T11:40:54.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.899+0000 7f17d7399740 1 -- 172.21.15.47:0/2218893512 wait complete. 2024-03-21T11:40:54.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.899+0000 7f17d7399740 20 client.31208 trim_cache size 0 max 16384 2024-03-21T11:40:54.900 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:40:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.900+0000 7f17d4600640 1 -- 172.21.15.47:0/1417176574 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17b40a3110 msgr2=0x7f17b40a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.900+0000 7f17d4600640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17b40a3110 0x7f17b40a55d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x55cb95fa3b70 tx=0x7f17b800a2d0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.900+0000 7f17d4600640 1 -- 172.21.15.47:0/1417176574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17cc101270 msgr2=0x7f17cc102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.900+0000 7f17d4600640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17cc101270 0x7f17cc102660 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009430 tx=0x7f17c40328c0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.901+0000 7f17d4600640 1 -- 172.21.15.47:0/1417176574 shutdown_connections 2024-03-21T11:40:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.901+0000 7f17d4600640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f17b40a3110 0x7f17b40a55d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.901+0000 7f17d4600640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17cc101270 0x7f17cc102660 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.901+0000 7f17d4600640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17cc1008d0 0x7f17cc106030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.901+0000 7f17d4600640 1 --2- 172.21.15.47:0/1417176574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17cc0fff70 0x7f17cc105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:54.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.901+0000 7f17d4600640 1 -- 172.21.15.47:0/1417176574 >> 172.21.15.47:0/1417176574 conn(0x7f17cc0fbb30 msgr2=0x7f17cc0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:54.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.901+0000 7f17d4600640 1 -- 172.21.15.47:0/1417176574 shutdown_connections 2024-03-21T11:40:54.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:54.901+0000 7f17d4600640 1 -- 172.21.15.47:0/1417176574 wait complete. 2024-03-21T11:40:54.924 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:40:54.925 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:40:54.945 INFO:tasks.workunit.client.0.smithi047.stdout: 10 102698 24.32 MB/sec execute 98 sec latency 159.008 ms 2024-03-21T11:40:55.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:54 smithi100 ceph-mon[36245]: pgmap v927: 129 pgs: 129 active+clean; 442 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.0 MiB/s wr, 652 op/s 2024-03-21T11:40:55.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:54 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3151082150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:40:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.157+0000 7f0863200640 1 -- 172.21.15.47:0/1585486156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0864103150 msgr2=0x7f08641099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.157+0000 7f0863200640 1 --2- 172.21.15.47:0/1585486156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0864103150 0x7f08641099e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f085c009be0 tx=0x7f085c02bef0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.158+0000 7f0863200640 1 -- 172.21.15.47:0/1585486156 shutdown_connections 2024-03-21T11:40:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.158+0000 7f0863200640 1 --2- 172.21.15.47:0/1585486156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0864103150 0x7f08641099e0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.158+0000 7f0863200640 1 --2- 172.21.15.47:0/1585486156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0864102790 0x7f0864102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.158+0000 7f0863200640 1 --2- 172.21.15.47:0/1585486156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08641015a0 0x7f08641019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.158+0000 7f0863200640 1 -- 172.21.15.47:0/1585486156 >> 172.21.15.47:0/1585486156 conn(0x7f08640fcb10 msgr2=0x7f08640fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.158+0000 7f0863200640 1 -- 172.21.15.47:0/1585486156 shutdown_connections 2024-03-21T11:40:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.158+0000 7f0863200640 1 -- 172.21.15.47:0/1585486156 wait complete. 2024-03-21T11:40:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.159+0000 7f0863200640 1 Processor -- start 2024-03-21T11:40:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.160+0000 7f0863200640 1 -- start start 2024-03-21T11:40:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.160+0000 7f0863200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08641015a0 0x7f086419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.160+0000 7f0863200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864102790 0x7f086419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.160+0000 7f0863200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0864103150 0x7f08641a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.160+0000 7f0861e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08641015a0 0x7f086419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.160+0000 7f0861e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08641015a0 0x7f086419de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47066/0 (socket says 172.21.15.47:47066) 2024-03-21T11:40:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.160+0000 7f0861400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864102790 0x7f086419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.160+0000 7f0861400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864102790 0x7f086419e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42028/0 (socket says 172.21.15.47:42028) 2024-03-21T11:40:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.161+0000 7f0863200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086410aff0 con 0x7f0864103150 2024-03-21T11:40:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.161+0000 7f0863200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086410ae70 con 0x7f08641015a0 2024-03-21T11:40:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.161+0000 7f0863200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086410b170 con 0x7f0864102790 2024-03-21T11:40:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.161+0000 7f0861e00640 1 -- 172.21.15.47:0/395333469 learned_addr learned my addr 172.21.15.47:0/395333469 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.161+0000 7f0862800640 1 --2- 172.21.15.47:0/395333469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0864103150 0x7f08641a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.161+0000 7f0861400640 1 -- 172.21.15.47:0/395333469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08641015a0 msgr2=0x7f086419de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.161+0000 7f0861400640 1 --2- 172.21.15.47:0/395333469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08641015a0 0x7f086419de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.161+0000 7f0861400640 1 -- 172.21.15.47:0/395333469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0864103150 msgr2=0x7f08641a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.161+0000 7f0861400640 1 --2- 172.21.15.47:0/395333469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0864103150 0x7f08641a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.161+0000 7f0861400640 1 -- 172.21.15.47:0/395333469 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08641a5a80 con 0x7f0864102790 2024-03-21T11:40:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.161+0000 7f0862800640 1 --2- 172.21.15.47:0/395333469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0864103150 0x7f08641a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.162+0000 7f0861400640 1 --2- 172.21.15.47:0/395333469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864102790 0x7f086419e3a0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f085400e9f0 tx=0x7f085400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.162+0000 7f085aa00640 1 -- 172.21.15.47:0/395333469 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085400cd80 con 0x7f0864102790 2024-03-21T11:40:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.163+0000 7f085aa00640 1 -- 172.21.15.47:0/395333469 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08540035f0 con 0x7f0864102790 2024-03-21T11:40:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.163+0000 7f085aa00640 1 -- 172.21.15.47:0/395333469 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854010690 con 0x7f0864102790 2024-03-21T11:40:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.163+0000 7f0863200640 1 -- 172.21.15.47:0/395333469 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08641a5d70 con 0x7f0864102790 2024-03-21T11:40:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.163+0000 7f0863200640 1 -- 172.21.15.47:0/395333469 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08641a6210 con 0x7f0864102790 2024-03-21T11:40:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.167+0000 7f085aa00640 1 -- 172.21.15.47:0/395333469 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0854010830 con 0x7f0864102790 2024-03-21T11:40:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.169+0000 7f086b602740 1 Processor -- start 2024-03-21T11:40:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.169+0000 7f086b602740 1 -- start start 2024-03-21T11:40:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.170+0000 7f086b602740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed80ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.170+0000 7f086b602740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed80fbc0 0x560eed810040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.170+0000 7f086b602740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed810580 0x560eed816e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.170+0000 7f086b602740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eed7732c0 con 0x560eed80fbc0 2024-03-21T11:40:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.170+0000 7f086b602740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eed71bc90 con 0x560eed810580 2024-03-21T11:40:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.170+0000 7f086b602740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eed76f360 con 0x560eed80e9f0 2024-03-21T11:40:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.170+0000 7f0862800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed80ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.170+0000 7f0862800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed80ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42034/0 (socket says 172.21.15.47:42034) 2024-03-21T11:40:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.170+0000 7f0862800640 1 -- 172.21.15.47:0/3516882155 learned_addr learned my addr 172.21.15.47:0/3516882155 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.170+0000 7f0861e00640 1 --2- 172.21.15.47:0/3516882155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed80fbc0 0x560eed810040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.171+0000 7f0861400640 1 --2- 172.21.15.47:0/3516882155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed810580 0x560eed816e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.171+0000 7f0861e00640 1 -- 172.21.15.47:0/3516882155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 msgr2=0x560eed80ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.171+0000 7f0861e00640 1 --2- 172.21.15.47:0/3516882155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed80ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.171+0000 7f0861e00640 1 -- 172.21.15.47:0/3516882155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed810580 msgr2=0x560eed816e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.171+0000 7f0861e00640 1 --2- 172.21.15.47:0/3516882155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed810580 0x560eed816e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.171+0000 7f085aa00640 1 --2- 172.21.15.47:0/395333469 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f08480a2fa0 0x7f08480a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.171+0000 7f0861e00640 1 -- 172.21.15.47:0/3516882155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560eed740080 con 0x560eed80fbc0 2024-03-21T11:40:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.171+0000 7f0862800640 1 --2- 172.21.15.47:0/3516882155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed80ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:40:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.172+0000 7f0861e00640 1 --2- 172.21.15.47:0/3516882155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed80fbc0 0x560eed810040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f085000bca0 tx=0x7f08500107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950c3f44d9ecfd60 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.172+0000 7f085aa00640 1 -- 172.21.15.47:0/395333469 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f08540cfa70 con 0x7f0864102790 2024-03-21T11:40:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.173+0000 7f0862800640 1 --2- 172.21.15.47:0/395333469 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f08480a2fa0 0x7f08480a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.174+0000 7f084f000640 1 -- 172.21.15.47:0/3516882155 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850004030 con 0x560eed80fbc0 2024-03-21T11:40:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.174+0000 7f084f000640 1 -- 172.21.15.47:0/3516882155 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08500041d0 con 0x560eed80fbc0 2024-03-21T11:40:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.174+0000 7f084f000640 1 -- 172.21.15.47:0/3516882155 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08500116c0 con 0x560eed80fbc0 2024-03-21T11:40:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.175+0000 7f086b602740 1 -- 172.21.15.47:0/3516882155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed80fbc0 msgr2=0x560eed810040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.175+0000 7f086b602740 1 --2- 172.21.15.47:0/3516882155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed80fbc0 0x560eed810040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f085000bca0 tx=0x7f08500107a0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.175+0000 7f0862800640 1 --2- 172.21.15.47:0/395333469 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f08480a2fa0 0x7f08480a5460 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f085c0099a0 tx=0x7f085c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.175+0000 7f086b602740 1 -- 172.21.15.47:0/3516882155 shutdown_connections 2024-03-21T11:40:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.175+0000 7f086b602740 1 --2- 172.21.15.47:0/3516882155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed810580 0x560eed816e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.175+0000 7f086b602740 1 --2- 172.21.15.47:0/3516882155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed80fbc0 0x560eed810040 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.175+0000 7f086b602740 1 --2- 172.21.15.47:0/3516882155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed80ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.175+0000 7f086b602740 1 -- 172.21.15.47:0/3516882155 >> 172.21.15.47:0/3516882155 conn(0x560eed809de0 msgr2=0x560eed80a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.175+0000 7f086b602740 1 -- 172.21.15.47:0/3516882155 shutdown_connections 2024-03-21T11:40:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.175+0000 7f086b602740 1 -- 172.21.15.47:0/3516882155 wait complete. 2024-03-21T11:40:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.176+0000 7f086b602740 1 -- start start 2024-03-21T11:40:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.176+0000 7f086b602740 1 Processor -- start 2024-03-21T11:40:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.177+0000 7f086b602740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed818ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.177+0000 7f086b602740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80fbc0 0x560eed819020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.177+0000 7f086b602740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed810580 0x560eed820060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.178+0000 7f0861400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80fbc0 0x560eed819020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.178+0000 7f0861e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed818ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.178+0000 7f0861e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed818ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42048/0 (socket says 172.21.15.47:42048) 2024-03-21T11:40:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.178+0000 7f0861400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80fbc0 0x560eed819020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47084/0 (socket says 172.21.15.47:47084) 2024-03-21T11:40:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.178+0000 7f086b602740 1 -- 172.21.15.47:0/3401446209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eed8175e0 con 0x560eed810580 2024-03-21T11:40:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.178+0000 7f086b602740 1 -- 172.21.15.47:0/3401446209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eed72f0d0 con 0x560eed80fbc0 2024-03-21T11:40:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.178+0000 7f086b602740 1 -- 172.21.15.47:0/3401446209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eed817760 con 0x560eed80e9f0 2024-03-21T11:40:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.178+0000 7f0861e00640 1 -- 172.21.15.47:0/3401446209 learned_addr learned my addr 172.21.15.47:0/3401446209 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.178+0000 7f086b602740 10 client.0 _ll_register_callbacks cb 0x560eed733e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.178+0000 7f0862800640 1 --2- 172.21.15.47:0/3401446209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed810580 0x560eed820060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.178+0000 7f0861400640 1 -- 172.21.15.47:0/3401446209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 msgr2=0x560eed818ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.178+0000 7f0861400640 1 --2- 172.21.15.47:0/3401446209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed818ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.178+0000 7f0861400640 1 -- 172.21.15.47:0/3401446209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed810580 msgr2=0x560eed820060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.178+0000 7f0861400640 1 --2- 172.21.15.47:0/3401446209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed810580 0x560eed820060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.179+0000 7f0861400640 1 -- 172.21.15.47:0/3401446209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560eed4873f0 con 0x560eed80fbc0 2024-03-21T11:40:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.179+0000 7f0861e00640 1 --2- 172.21.15.47:0/3401446209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed818ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.179+0000 7f0862800640 1 --2- 172.21.15.47:0/3401446209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed810580 0x560eed820060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:40:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.179+0000 7f0861400640 1 --2- 172.21.15.47:0/3401446209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80fbc0 0x560eed819020 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f08540cd230 tx=0x7f0854009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dc73d408ff15e95 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.179+0000 7f0843400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.179+0000 7f0843400640 1 -- 172.21.15.47:0/3401446209 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08540a0f50 con 0x560eed80fbc0 2024-03-21T11:40:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.179+0000 7f0843400640 1 -- 172.21.15.47:0/3401446209 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f08540037a0 con 0x560eed80fbc0 2024-03-21T11:40:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.180+0000 7f086b602740 1 -- 172.21.15.47:0/3401446209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08500041d0 con 0x560eed80fbc0 2024-03-21T11:40:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.180+0000 7f0843400640 1 -- 172.21.15.47:0/3401446209 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08540a5810 con 0x560eed80fbc0 2024-03-21T11:40:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.180+0000 7f086b602740 10 client.31217 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.180+0000 7f086b602740 1 -- 172.21.15.47:0/3401446209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560eed5517d0 con 0x560eed80fbc0 2024-03-21T11:40:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.180+0000 7f0843400640 1 -- 172.21.15.47:0/3401446209 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08500041d0 con 0x560eed80fbc0 2024-03-21T11:40:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.181+0000 7f0843400640 1 -- 172.21.15.47:0/3401446209 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f085409c270 con 0x560eed80fbc0 2024-03-21T11:40:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.181+0000 7f086b602740 10 client.31217 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.181+0000 7f086b602740 10 client.31217 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:40:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.181+0000 7f086b602740 20 client.31217 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.181+0000 7f086b602740 1 --2- 172.21.15.47:0/3401446209 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560eed824580 0x560eed8269d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.181+0000 7f086b602740 4 client.31217 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.181+0000 7f086b602740 1 -- 172.21.15.47:0/3401446209 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560eed780800 con 0x560eed824580 2024-03-21T11:40:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.182+0000 7f0861e00640 1 --2- 172.21.15.47:0/3401446209 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560eed824580 0x560eed8269d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.182+0000 7f0861e00640 1 --2- 172.21.15.47:0/3401446209 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560eed824580 0x560eed8269d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a9cad6cb608ef4ae server_cookie=24fb0e0028bcb53b in_seq=0 out_seq=0 2024-03-21T11:40:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.182+0000 7f0843400640 10 client.31217 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:40:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.183+0000 7f0843400640 1 -- 172.21.15.47:0/3401446209 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560eed780800 con 0x560eed824580 2024-03-21T11:40:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.183+0000 7f0843400640 10 client.31217 handle_command_reply: tid=0 2024-03-21T11:40:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.183+0000 7f086b602740 1 client.31217 shutdown 2024-03-21T11:40:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.184+0000 7f086b602740 1 -- 172.21.15.47:0/3401446209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80fbc0 msgr2=0x560eed819020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.184+0000 7f086b602740 1 --2- 172.21.15.47:0/3401446209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80fbc0 0x560eed819020 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f08540cd230 tx=0x7f0854009590 comp rx=0 tx=0).stop 2024-03-21T11:40:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.184+0000 7f086b602740 1 -- 172.21.15.47:0/3401446209 shutdown_connections 2024-03-21T11:40:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.184+0000 7f086b602740 1 --2- 172.21.15.47:0/3401446209 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560eed824580 0x560eed8269d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.184+0000 7f086b602740 1 --2- 172.21.15.47:0/3401446209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed810580 0x560eed820060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.184+0000 7f086b602740 1 --2- 172.21.15.47:0/3401446209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80fbc0 0x560eed819020 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.184+0000 7f086b602740 1 --2- 172.21.15.47:0/3401446209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed818ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.184+0000 7f086b602740 1 -- 172.21.15.47:0/3401446209 >> 172.21.15.47:0/3401446209 conn(0x560eed809de0 msgr2=0x560eed809160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.184+0000 7f0843400640 0 client.31217 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:40:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.184+0000 7f086b602740 1 -- 172.21.15.47:0/3401446209 shutdown_connections 2024-03-21T11:40:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.184+0000 7f086b602740 1 -- 172.21.15.47:0/3401446209 wait complete. 2024-03-21T11:40:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.184+0000 7f086b602740 20 client.31217 trim_cache size 0 max 16384 2024-03-21T11:40:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.204+0000 7f086b602740 1 Processor -- start 2024-03-21T11:40:55.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.204+0000 7f086b602740 1 -- start start 2024-03-21T11:40:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.204+0000 7f086b602740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80e9f0 0x560eed80d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.204+0000 7f086b602740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed80fbc0 0x560eed820020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.204+0000 7f086b602740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed810580 0x560eed82b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.204+0000 7f086b602740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08641b0470 con 0x560eed80fbc0 2024-03-21T11:40:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.204+0000 7f086b602740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08641b1df0 con 0x560eed80e9f0 2024-03-21T11:40:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.204+0000 7f086b602740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08641b0ad0 con 0x560eed810580 2024-03-21T11:40:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.205+0000 7f0861e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80e9f0 0x560eed80d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.205+0000 7f0861e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80e9f0 0x560eed80d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47092/0 (socket says 172.21.15.47:47092) 2024-03-21T11:40:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.205+0000 7f0861e00640 1 -- 172.21.15.47:0/2956492824 learned_addr learned my addr 172.21.15.47:0/2956492824 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.205+0000 7f0862800640 1 --2- 172.21.15.47:0/2956492824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed810580 0x560eed82b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.205+0000 7f0861400640 1 --2- 172.21.15.47:0/2956492824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed80fbc0 0x560eed820020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.205+0000 7f0861e00640 1 -- 172.21.15.47:0/2956492824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed810580 msgr2=0x560eed82b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.205+0000 7f0861e00640 1 --2- 172.21.15.47:0/2956492824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed810580 0x560eed82b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.205+0000 7f0861e00640 1 -- 172.21.15.47:0/2956492824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed80fbc0 msgr2=0x560eed820020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.205+0000 7f0861e00640 1 --2- 172.21.15.47:0/2956492824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed80fbc0 0x560eed820020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.205+0000 7f0861e00640 1 -- 172.21.15.47:0/2956492824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560eed80f640 con 0x560eed80e9f0 2024-03-21T11:40:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.205+0000 7f0861400640 1 --2- 172.21.15.47:0/2956492824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed80fbc0 0x560eed820020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:40:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.205+0000 7f0861e00640 1 --2- 172.21.15.47:0/2956492824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80e9f0 0x560eed80d750 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0850010ca0 tx=0x7f085000b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=416efbd034f183d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.206+0000 7f084e600640 1 -- 172.21.15.47:0/2956492824 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850005040 con 0x560eed80e9f0 2024-03-21T11:40:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.206+0000 7f084e600640 1 -- 172.21.15.47:0/2956492824 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f085002a7f0 con 0x560eed80e9f0 2024-03-21T11:40:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.206+0000 7f084e600640 1 -- 172.21.15.47:0/2956492824 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08500038e0 con 0x560eed80e9f0 2024-03-21T11:40:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.207+0000 7f086b602740 1 -- 172.21.15.47:0/2956492824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80e9f0 msgr2=0x560eed80d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.207+0000 7f086b602740 1 --2- 172.21.15.47:0/2956492824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80e9f0 0x560eed80d750 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0850010ca0 tx=0x7f085000b750 comp rx=0 tx=0).stop 2024-03-21T11:40:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.207+0000 7f086b602740 1 -- 172.21.15.47:0/2956492824 shutdown_connections 2024-03-21T11:40:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.207+0000 7f086b602740 1 --2- 172.21.15.47:0/2956492824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed810580 0x560eed82b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.207+0000 7f086b602740 1 --2- 172.21.15.47:0/2956492824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed80fbc0 0x560eed820020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.207+0000 7f086b602740 1 --2- 172.21.15.47:0/2956492824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80e9f0 0x560eed80d750 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.207+0000 7f086b602740 1 -- 172.21.15.47:0/2956492824 >> 172.21.15.47:0/2956492824 conn(0x560eed809de0 msgr2=0x560eed80d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.207+0000 7f086b602740 1 -- 172.21.15.47:0/2956492824 shutdown_connections 2024-03-21T11:40:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.207+0000 7f086b602740 1 -- 172.21.15.47:0/2956492824 wait complete. 2024-03-21T11:40:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.207+0000 7f086b602740 1 -- start start 2024-03-21T11:40:55.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.208+0000 7f086b602740 1 Processor -- start 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.208+0000 7f086b602740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed81fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.208+0000 7f086b602740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80fbc0 0x560eed81e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.208+0000 7f086b602740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed810580 0x560eed81e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.208+0000 7f086b602740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08641b1790 con 0x560eed810580 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.208+0000 7f086b602740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08641b1130 con 0x560eed80fbc0 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.208+0000 7f086b602740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08641ae630 con 0x560eed80e9f0 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.208+0000 7f086b602740 10 client.0 _ll_register_callbacks cb 0x7f085409c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.208+0000 7f0861e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed81fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.208+0000 7f0861e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed81fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42074/0 (socket says 172.21.15.47:42074) 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.208+0000 7f0861e00640 1 -- 172.21.15.47:0/1243409577 learned_addr learned my addr 172.21.15.47:0/1243409577 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.209+0000 7f0861e00640 1 -- 172.21.15.47:0/1243409577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80fbc0 msgr2=0x560eed81e4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.209+0000 7f0861e00640 1 --2- 172.21.15.47:0/1243409577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80fbc0 0x560eed81e4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.209+0000 7f0861e00640 1 -- 172.21.15.47:0/1243409577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed810580 msgr2=0x560eed81e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.209+0000 7f0861e00640 1 --2- 172.21.15.47:0/1243409577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed810580 0x560eed81e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.209+0000 7f0861e00640 1 -- 172.21.15.47:0/1243409577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560eed81f1a0 con 0x560eed80e9f0 2024-03-21T11:40:55.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.209+0000 7f0861e00640 1 --2- 172.21.15.47:0/1243409577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed81fea0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x560eed826e20 tx=0x7f085002a240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce65582a856c9ffc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.209+0000 7f0843400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:40:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.209+0000 7f0843400640 1 -- 172.21.15.47:0/1243409577 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085002adb0 con 0x560eed80e9f0 2024-03-21T11:40:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.209+0000 7f0843400640 1 -- 172.21.15.47:0/1243409577 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f08500050b0 con 0x560eed80e9f0 2024-03-21T11:40:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.209+0000 7f086b602740 1 -- 172.21.15.47:0/1243409577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f085002a7f0 con 0x560eed80e9f0 2024-03-21T11:40:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.210+0000 7f0843400640 1 -- 172.21.15.47:0/1243409577 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08500052f0 con 0x560eed80e9f0 2024-03-21T11:40:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.210+0000 7f086b602740 10 client.31204 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.210+0000 7f086b602740 1 -- 172.21.15.47:0/1243409577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560eed891e50 con 0x560eed80e9f0 2024-03-21T11:40:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.210+0000 7f0843400640 1 -- 172.21.15.47:0/1243409577 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f085002a7f0 con 0x560eed80e9f0 2024-03-21T11:40:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.210+0000 7f0843400640 1 -- 172.21.15.47:0/1243409577 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f08500115b0 con 0x560eed80e9f0 2024-03-21T11:40:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.210+0000 7f086b602740 10 client.31204 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.210+0000 7f086b602740 10 client.31204 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:40:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.211+0000 7f086b602740 20 client.31204 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.211+0000 7f086b602740 1 --2- 172.21.15.47:0/1243409577 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560eed892600 0x560eed894a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.211+0000 7f086b602740 4 client.31204 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:40:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.211+0000 7f086b602740 1 -- 172.21.15.47:0/1243409577 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560eed77ab50 con 0x560eed892600 2024-03-21T11:40:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.211+0000 7f0861400640 1 --2- 172.21.15.47:0/1243409577 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560eed892600 0x560eed894a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.212+0000 7f0843400640 10 client.31204 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:40:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.212+0000 7f0861400640 1 --2- 172.21.15.47:0/1243409577 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560eed892600 0x560eed894a20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8cccf13592ae75ef server_cookie=46490123acb215c in_seq=0 out_seq=0 2024-03-21T11:40:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f0843400640 1 -- 172.21.15.47:0/1243409577 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560eed77ab50 con 0x560eed892600 2024-03-21T11:40:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f0843400640 10 client.31204 handle_command_reply: tid=0 2024-03-21T11:40:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f086b602740 1 client.31204 shutdown 2024-03-21T11:40:55.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f086b602740 1 -- 172.21.15.47:0/1243409577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 msgr2=0x560eed81fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f086b602740 1 --2- 172.21.15.47:0/1243409577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed81fea0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x560eed826e20 tx=0x7f085002a240 comp rx=0 tx=0).stop 2024-03-21T11:40:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f086b602740 1 -- 172.21.15.47:0/1243409577 shutdown_connections 2024-03-21T11:40:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f086b602740 1 --2- 172.21.15.47:0/1243409577 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560eed892600 0x560eed894a20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f086b602740 1 --2- 172.21.15.47:0/1243409577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560eed810580 0x560eed81e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f086b602740 1 --2- 172.21.15.47:0/1243409577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560eed80fbc0 0x560eed81e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f086b602740 1 --2- 172.21.15.47:0/1243409577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560eed80e9f0 0x560eed81fea0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f086b602740 1 -- 172.21.15.47:0/1243409577 >> 172.21.15.47:0/1243409577 conn(0x560eed809de0 msgr2=0x560eed80d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f0843400640 0 client.31204 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:40:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f086b602740 1 -- 172.21.15.47:0/1243409577 shutdown_connections 2024-03-21T11:40:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f086b602740 1 -- 172.21.15.47:0/1243409577 wait complete. 2024-03-21T11:40:55.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.213+0000 7f086b602740 20 client.31204 trim_cache size 0 max 16384 2024-03-21T11:40:55.217 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:40:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.214+0000 7f0863200640 1 -- 172.21.15.47:0/395333469 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f08480a2fa0 msgr2=0x7f08480a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.214+0000 7f0863200640 1 --2- 172.21.15.47:0/395333469 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f08480a2fa0 0x7f08480a5460 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f085c0099a0 tx=0x7f085c02e040 comp rx=0 tx=0).stop 2024-03-21T11:40:55.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.214+0000 7f0863200640 1 -- 172.21.15.47:0/395333469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864102790 msgr2=0x7f086419e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.214+0000 7f0863200640 1 --2- 172.21.15.47:0/395333469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864102790 0x7f086419e3a0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f085400e9f0 tx=0x7f085400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.214+0000 7f0863200640 1 -- 172.21.15.47:0/395333469 shutdown_connections 2024-03-21T11:40:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.214+0000 7f0863200640 1 --2- 172.21.15.47:0/395333469 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f08480a2fa0 0x7f08480a5460 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.214+0000 7f0863200640 1 --2- 172.21.15.47:0/395333469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0864103150 0x7f08641a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.214+0000 7f0863200640 1 --2- 172.21.15.47:0/395333469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864102790 0x7f086419e3a0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.214+0000 7f0863200640 1 --2- 172.21.15.47:0/395333469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08641015a0 0x7f086419de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.214+0000 7f0863200640 1 -- 172.21.15.47:0/395333469 >> 172.21.15.47:0/395333469 conn(0x7f08640fcb10 msgr2=0x7f086410daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.215+0000 7f0863200640 1 -- 172.21.15.47:0/395333469 shutdown_connections 2024-03-21T11:40:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.215+0000 7f0863200640 1 -- 172.21.15.47:0/395333469 wait complete. 2024-03-21T11:40:55.236 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:40:55.236 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:40:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:54 smithi102 ceph-mon[36748]: pgmap v927: 129 pgs: 129 active+clean; 442 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.0 MiB/s wr, 652 op/s 2024-03-21T11:40:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:54 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3151082150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:40:55.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.494+0000 7fa07e200640 1 -- 172.21.15.47:0/1367028382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa078105490 msgr2=0x7fa07810bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.494+0000 7fa07e200640 1 --2- 172.21.15.47:0/1367028382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa078105490 0x7fa07810bd20 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fa070009be0 tx=0x7fa07002bef0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.494+0000 7fa07e200640 1 -- 172.21.15.47:0/1367028382 shutdown_connections 2024-03-21T11:40:55.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.494+0000 7fa07e200640 1 --2- 172.21.15.47:0/1367028382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa078105490 0x7fa07810bd20 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.494+0000 7fa07e200640 1 --2- 172.21.15.47:0/1367028382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa078104ad0 0x7fa078104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.494+0000 7fa07e200640 1 --2- 172.21.15.47:0/1367028382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0781038e0 0x7fa078103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.494+0000 7fa07e200640 1 -- 172.21.15.47:0/1367028382 >> 172.21.15.47:0/1367028382 conn(0x7fa0780fee50 msgr2=0x7fa078101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.494+0000 7fa07e200640 1 -- 172.21.15.47:0/1367028382 shutdown_connections 2024-03-21T11:40:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.494+0000 7fa07e200640 1 -- 172.21.15.47:0/1367028382 wait complete. 2024-03-21T11:40:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.495+0000 7fa07e200640 1 Processor -- start 2024-03-21T11:40:55.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.495+0000 7fa07e200640 1 -- start start 2024-03-21T11:40:55.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.496+0000 7fa07e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0781038e0 0x7fa07819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.496+0000 7fa07e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa078104ad0 0x7fa07819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.496+0000 7fa07e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa078105490 0x7fa0781a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.497+0000 7fa07e200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07810f560 con 0x7fa0781038e0 2024-03-21T11:40:55.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.497+0000 7fa07e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07810f3e0 con 0x7fa078105490 2024-03-21T11:40:55.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.497+0000 7fa07e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07810f6e0 con 0x7fa078104ad0 2024-03-21T11:40:55.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.497+0000 7fa076a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa078104ad0 0x7fa07819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.497+0000 7fa076a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa078104ad0 0x7fa07819e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42094/0 (socket says 172.21.15.47:42094) 2024-03-21T11:40:55.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.497+0000 7fa076a00640 1 -- 172.21.15.47:0/1592633373 learned_addr learned my addr 172.21.15.47:0/1592633373 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.497+0000 7fa077e00640 1 --2- 172.21.15.47:0/1592633373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa078105490 0x7fa0781a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.497+0000 7fa077400640 1 --2- 172.21.15.47:0/1592633373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0781038e0 0x7fa07819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.498+0000 7fa076a00640 1 -- 172.21.15.47:0/1592633373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa078105490 msgr2=0x7fa0781a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.498+0000 7fa076a00640 1 --2- 172.21.15.47:0/1592633373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa078105490 0x7fa0781a5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.498+0000 7fa076a00640 1 -- 172.21.15.47:0/1592633373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0781038e0 msgr2=0x7fa07819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.498+0000 7fa076a00640 1 --2- 172.21.15.47:0/1592633373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0781038e0 0x7fa07819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.498+0000 7fa076a00640 1 -- 172.21.15.47:0/1592633373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0781a5bf0 con 0x7fa078104ad0 2024-03-21T11:40:55.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.498+0000 7fa076a00640 1 --2- 172.21.15.47:0/1592633373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa078104ad0 0x7fa07819e510 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fa06400e9f0 tx=0x7fa06400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.499+0000 7fa06fe00640 1 -- 172.21.15.47:0/1592633373 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06400ccc0 con 0x7fa078104ad0 2024-03-21T11:40:55.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.499+0000 7fa06fe00640 1 -- 172.21.15.47:0/1592633373 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa064005500 con 0x7fa078104ad0 2024-03-21T11:40:55.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.499+0000 7fa06fe00640 1 -- 172.21.15.47:0/1592633373 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064010690 con 0x7fa078104ad0 2024-03-21T11:40:55.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.499+0000 7fa07e200640 1 -- 172.21.15.47:0/1592633373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0781a5ee0 con 0x7fa078104ad0 2024-03-21T11:40:55.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.500+0000 7fa07e200640 1 -- 172.21.15.47:0/1592633373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0781a6350 con 0x7fa078104ad0 2024-03-21T11:40:55.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.505+0000 7fa080fc3740 1 Processor -- start 2024-03-21T11:40:55.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.505+0000 7fa080fc3740 1 -- start start 2024-03-21T11:40:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.505+0000 7fa080fc3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b0016c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.505+0000 7fa080fc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b00179e0 0x55f0b0017e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.505+0000 7fa080fc3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b00183a0 0x55f0b001ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.505+0000 7fa080fc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0aff7b2c0 con 0x55f0b00179e0 2024-03-21T11:40:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.505+0000 7fa080fc3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0aff23c90 con 0x55f0b0016810 2024-03-21T11:40:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.505+0000 7fa080fc3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0aff77360 con 0x55f0b00183a0 2024-03-21T11:40:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.505+0000 7fa077e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b0016c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.505+0000 7fa077e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b0016c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47114/0 (socket says 172.21.15.47:47114) 2024-03-21T11:40:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.505+0000 7fa077e00640 1 -- 172.21.15.47:0/1388807904 learned_addr learned my addr 172.21.15.47:0/1388807904 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.506+0000 7fa077400640 1 --2- 172.21.15.47:0/1388807904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b00179e0 0x55f0b0017e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.506+0000 7fa06fe00640 1 -- 172.21.15.47:0/1592633373 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa0640046c0 con 0x7fa078104ad0 2024-03-21T11:40:55.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.506+0000 7fa077e00640 1 -- 172.21.15.47:0/1388807904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b00183a0 msgr2=0x55f0b001ec30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:40:55.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.506+0000 7fa077e00640 1 --2- 172.21.15.47:0/1388807904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b00183a0 0x55f0b001ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.506+0000 7fa077e00640 1 -- 172.21.15.47:0/1388807904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b00179e0 msgr2=0x55f0b0017e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.506+0000 7fa077e00640 1 --2- 172.21.15.47:0/1388807904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b00179e0 0x55f0b0017e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.506+0000 7fa077e00640 1 -- 172.21.15.47:0/1388807904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0aff48080 con 0x55f0b0016810 2024-03-21T11:40:55.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.506+0000 7fa077400640 1 --2- 172.21.15.47:0/1388807904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b00179e0 0x55f0b0017e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:55.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.509+0000 7fa06fe00640 1 --2- 172.21.15.47:0/1592633373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa05c0a3070 0x7fa05c0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.509+0000 7fa077e00640 1 --2- 172.21.15.47:0/1388807904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b0016c30 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa070034fd0 tx=0x7fa070009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d0289400f32f85 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.509+0000 7fa06fe00640 1 -- 172.21.15.47:0/1592633373 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa0640cf9e0 con 0x7fa078104ad0 2024-03-21T11:40:55.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.509+0000 7fa077400640 1 --2- 172.21.15.47:0/1592633373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa05c0a3070 0x7fa05c0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.510+0000 7fa063e00640 1 -- 172.21.15.47:0/1388807904 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070042070 con 0x55f0b0016810 2024-03-21T11:40:55.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.511+0000 7fa063e00640 1 -- 172.21.15.47:0/1388807904 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa070033d90 con 0x55f0b0016810 2024-03-21T11:40:55.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.511+0000 7fa063e00640 1 -- 172.21.15.47:0/1388807904 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07002cb80 con 0x55f0b0016810 2024-03-21T11:40:55.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.511+0000 7fa080fc3740 1 -- 172.21.15.47:0/1388807904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 msgr2=0x55f0b0016c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.511+0000 7fa080fc3740 1 --2- 172.21.15.47:0/1388807904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b0016c30 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa070034fd0 tx=0x7fa070009280 comp rx=0 tx=0).stop 2024-03-21T11:40:55.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.512+0000 7fa077400640 1 --2- 172.21.15.47:0/1592633373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa05c0a3070 0x7fa05c0a5530 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa068005fd0 tx=0x7fa06800d5a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.512+0000 7fa080fc3740 1 -- 172.21.15.47:0/1388807904 shutdown_connections 2024-03-21T11:40:55.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.512+0000 7fa080fc3740 1 --2- 172.21.15.47:0/1388807904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b00183a0 0x55f0b001ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.512+0000 7fa080fc3740 1 --2- 172.21.15.47:0/1388807904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b00179e0 0x55f0b0017e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.512+0000 7fa080fc3740 1 --2- 172.21.15.47:0/1388807904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b0016c30 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.512+0000 7fa080fc3740 1 -- 172.21.15.47:0/1388807904 >> 172.21.15.47:0/1388807904 conn(0x55f0b0011c00 msgr2=0x55f0b0012000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.512+0000 7fa080fc3740 1 -- 172.21.15.47:0/1388807904 shutdown_connections 2024-03-21T11:40:55.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.512+0000 7fa080fc3740 1 -- 172.21.15.47:0/1388807904 wait complete. 2024-03-21T11:40:55.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.512+0000 7fa080fc3740 1 -- start start 2024-03-21T11:40:55.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.513+0000 7fa080fc3740 1 Processor -- start 2024-03-21T11:40:55.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.513+0000 7fa080fc3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b0020b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.513+0000 7fa080fc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b00179e0 0x55f0b0021080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.513+0000 7fa080fc3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b00183a0 0x55f0b0028050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.513+0000 7fa080fc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b001f350 con 0x55f0b00179e0 2024-03-21T11:40:55.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.513+0000 7fa080fc3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0aff370d0 con 0x55f0b0016810 2024-03-21T11:40:55.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.513+0000 7fa080fc3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b001f4d0 con 0x55f0b00183a0 2024-03-21T11:40:55.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.513+0000 7fa080fc3740 10 client.0 _ll_register_callbacks cb 0x55f0aff3be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:55.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.514+0000 7fa076a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b00179e0 0x55f0b0021080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.514+0000 7fa076a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b00179e0 0x55f0b0021080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39248/0 (socket says 172.21.15.47:39248) 2024-03-21T11:40:55.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.514+0000 7fa076a00640 1 -- 172.21.15.47:0/3225762704 learned_addr learned my addr 172.21.15.47:0/3225762704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.515+0000 7fa077400640 1 --2- 172.21.15.47:0/3225762704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b0020b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.515+0000 7fa076a00640 1 -- 172.21.15.47:0/3225762704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b00183a0 msgr2=0x55f0b0028050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.515+0000 7fa076a00640 1 --2- 172.21.15.47:0/3225762704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b00183a0 0x55f0b0028050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.515+0000 7fa076a00640 1 -- 172.21.15.47:0/3225762704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 msgr2=0x55f0b0020b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.515+0000 7fa076a00640 1 --2- 172.21.15.47:0/3225762704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b0020b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.515+0000 7fa076a00640 1 -- 172.21.15.47:0/3225762704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0afc8f3f0 con 0x55f0b00179e0 2024-03-21T11:40:55.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.515+0000 7fa076a00640 1 --2- 172.21.15.47:0/3225762704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b00179e0 0x55f0b0021080 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa06409bd10 tx=0x7fa06409b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f317321327076f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.515+0000 7fa060c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:40:55.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.515+0000 7fa060c00640 1 -- 172.21.15.47:0/3225762704 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06409bd80 con 0x55f0b00179e0 2024-03-21T11:40:55.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.516+0000 7fa060c00640 1 -- 172.21.15.47:0/3225762704 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa064016d90 con 0x55f0b00179e0 2024-03-21T11:40:55.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.516+0000 7fa080fc3740 1 -- 172.21.15.47:0/3225762704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa070033d90 con 0x55f0b00179e0 2024-03-21T11:40:55.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.516+0000 7fa060c00640 1 -- 172.21.15.47:0/3225762704 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa06409bd80 con 0x55f0b00179e0 2024-03-21T11:40:55.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.516+0000 7fa080fc3740 10 client.20685 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:55.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.516+0000 7fa080fc3740 1 -- 172.21.15.47:0/3225762704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f0afd597d0 con 0x55f0b00179e0 2024-03-21T11:40:55.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.517+0000 7fa060c00640 1 -- 172.21.15.47:0/3225762704 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa070033d90 con 0x55f0b00179e0 2024-03-21T11:40:55.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.517+0000 7fa060c00640 1 -- 172.21.15.47:0/3225762704 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa06409b1d0 con 0x55f0b00179e0 2024-03-21T11:40:55.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.517+0000 7fa080fc3740 10 client.20685 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:55.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.517+0000 7fa080fc3740 10 client.20685 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:40:55.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.517+0000 7fa080fc3740 20 client.20685 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:55.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.517+0000 7fa080fc3740 1 --2- 172.21.15.47:0/3225762704 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f0b002c5d0 0x55f0b002e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.517+0000 7fa080fc3740 4 client.20685 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:55.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.517+0000 7fa080fc3740 1 -- 172.21.15.47:0/3225762704 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0aff88800 con 0x55f0b002c5d0 2024-03-21T11:40:55.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.518+0000 7fa077400640 1 --2- 172.21.15.47:0/3225762704 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f0b002c5d0 0x55f0b002e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.519+0000 7fa077400640 1 --2- 172.21.15.47:0/3225762704 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f0b002c5d0 0x55f0b002e9f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13dc702f8b4a1b server_cookie=f3e8ed57b2cc6fc4 in_seq=0 out_seq=0 2024-03-21T11:40:55.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.519+0000 7fa060c00640 10 client.20685 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:40:55.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.523+0000 7fa060c00640 1 -- 172.21.15.47:0/3225762704 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f0aff88800 con 0x55f0b002c5d0 2024-03-21T11:40:55.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.523+0000 7fa060c00640 10 client.20685 handle_command_reply: tid=0 2024-03-21T11:40:55.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.523+0000 7fa080fc3740 1 client.20685 shutdown 2024-03-21T11:40:55.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.525+0000 7fa080fc3740 1 -- 172.21.15.47:0/3225762704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b00179e0 msgr2=0x55f0b0021080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.525+0000 7fa080fc3740 1 --2- 172.21.15.47:0/3225762704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b00179e0 0x55f0b0021080 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa06409bd10 tx=0x7fa06409b6e0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.527+0000 7fa080fc3740 1 -- 172.21.15.47:0/3225762704 shutdown_connections 2024-03-21T11:40:55.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.527+0000 7fa080fc3740 1 --2- 172.21.15.47:0/3225762704 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f0b002c5d0 0x55f0b002e9f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.527+0000 7fa080fc3740 1 --2- 172.21.15.47:0/3225762704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b00183a0 0x55f0b0028050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.527+0000 7fa080fc3740 1 --2- 172.21.15.47:0/3225762704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b00179e0 0x55f0b0021080 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.527+0000 7fa080fc3740 1 --2- 172.21.15.47:0/3225762704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b0020b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.527+0000 7fa080fc3740 1 -- 172.21.15.47:0/3225762704 >> 172.21.15.47:0/3225762704 conn(0x55f0b0011c00 msgr2=0x55f0b0010eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.528+0000 7fa060c00640 0 client.20685 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:40:55.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.529+0000 7fa080fc3740 1 -- 172.21.15.47:0/3225762704 shutdown_connections 2024-03-21T11:40:55.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.529+0000 7fa080fc3740 1 -- 172.21.15.47:0/3225762704 wait complete. 2024-03-21T11:40:55.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.529+0000 7fa080fc3740 20 client.20685 trim_cache size 0 max 16384 2024-03-21T11:40:55.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.549+0000 7fa080fc3740 1 Processor -- start 2024-03-21T11:40:55.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.550+0000 7fa080fc3740 1 -- start start 2024-03-21T11:40:55.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.550+0000 7fa080fc3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b001ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.550+0000 7fa080fc3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b00183a0 0x55f0b0028010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.550+0000 7fa080fc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b0021250 0x55f0b001b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.550+0000 7fa080fc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0781b0660 con 0x55f0b0021250 2024-03-21T11:40:55.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.550+0000 7fa080fc3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0781b1fe0 con 0x55f0b0016810 2024-03-21T11:40:55.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.550+0000 7fa080fc3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0781b0cc0 con 0x55f0b00183a0 2024-03-21T11:40:55.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.550+0000 7fa076a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b00183a0 0x55f0b0028010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.550+0000 7fa077400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b001ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.550+0000 7fa077400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b001ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47120/0 (socket says 172.21.15.47:47120) 2024-03-21T11:40:55.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.550+0000 7fa077400640 1 -- 172.21.15.47:0/3214459475 learned_addr learned my addr 172.21.15.47:0/3214459475 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.550+0000 7fa077e00640 1 --2- 172.21.15.47:0/3214459475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b0021250 0x55f0b001b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.551+0000 7fa077400640 1 -- 172.21.15.47:0/3214459475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b00183a0 msgr2=0x55f0b0028010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.551+0000 7fa077400640 1 --2- 172.21.15.47:0/3214459475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b00183a0 0x55f0b0028010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.551+0000 7fa077400640 1 -- 172.21.15.47:0/3214459475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b0021250 msgr2=0x55f0b001b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.551+0000 7fa077400640 1 --2- 172.21.15.47:0/3214459475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b0021250 0x55f0b001b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.551+0000 7fa077400640 1 -- 172.21.15.47:0/3214459475 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0b0012650 con 0x55f0b0016810 2024-03-21T11:40:55.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.551+0000 7fa077400640 1 --2- 172.21.15.47:0/3214459475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b001ab20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x55f0b0012b70 tx=0x7fa068015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71e35e2944ea3190 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.552+0000 7fa063400640 1 -- 172.21.15.47:0/3214459475 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068010d20 con 0x55f0b0016810 2024-03-21T11:40:55.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.552+0000 7fa063400640 1 -- 172.21.15.47:0/3214459475 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa06802d070 con 0x55f0b0016810 2024-03-21T11:40:55.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.552+0000 7fa063400640 1 -- 172.21.15.47:0/3214459475 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06801e670 con 0x55f0b0016810 2024-03-21T11:40:55.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.553+0000 7fa080fc3740 1 -- 172.21.15.47:0/3214459475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 msgr2=0x55f0b001ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.553+0000 7fa080fc3740 1 --2- 172.21.15.47:0/3214459475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b001ab20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x55f0b0012b70 tx=0x7fa068015000 comp rx=0 tx=0).stop 2024-03-21T11:40:55.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.553+0000 7fa080fc3740 1 -- 172.21.15.47:0/3214459475 shutdown_connections 2024-03-21T11:40:55.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.553+0000 7fa080fc3740 1 --2- 172.21.15.47:0/3214459475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b0021250 0x55f0b001b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.553+0000 7fa080fc3740 1 --2- 172.21.15.47:0/3214459475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b00183a0 0x55f0b0028010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.553+0000 7fa080fc3740 1 --2- 172.21.15.47:0/3214459475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b0016810 0x55f0b001ab20 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.553+0000 7fa080fc3740 1 -- 172.21.15.47:0/3214459475 >> 172.21.15.47:0/3214459475 conn(0x55f0b0011c00 msgr2=0x55f0b0020b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.553+0000 7fa080fc3740 1 -- 172.21.15.47:0/3214459475 shutdown_connections 2024-03-21T11:40:55.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.553+0000 7fa080fc3740 1 -- 172.21.15.47:0/3214459475 wait complete. 2024-03-21T11:40:55.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.554+0000 7fa080fc3740 1 -- start start 2024-03-21T11:40:55.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.554+0000 7fa080fc3740 1 Processor -- start 2024-03-21T11:40:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.554+0000 7fa080fc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b0016810 0x55f0b0023960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.554+0000 7fa080fc3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b00183a0 0x55f0b0023ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.554+0000 7fa080fc3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b0021250 0x55f0b0026ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.554+0000 7fa080fc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0781b1980 con 0x55f0b0016810 2024-03-21T11:40:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.554+0000 7fa080fc3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0781b1320 con 0x55f0b00183a0 2024-03-21T11:40:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.554+0000 7fa080fc3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0781ae820 con 0x55f0b0021250 2024-03-21T11:40:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.554+0000 7fa080fc3740 10 client.0 _ll_register_callbacks cb 0x7fa06409bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.554+0000 7fa077400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b0016810 0x55f0b0023960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.554+0000 7fa076a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b00183a0 0x55f0b0023ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.554+0000 7fa077400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b0016810 0x55f0b0023960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39270/0 (socket says 172.21.15.47:39270) 2024-03-21T11:40:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.554+0000 7fa077e00640 1 --2- 172.21.15.47:0/296466308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b0021250 0x55f0b0026ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.554+0000 7fa077400640 1 -- 172.21.15.47:0/296466308 learned_addr learned my addr 172.21.15.47:0/296466308 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.555+0000 7fa077400640 1 -- 172.21.15.47:0/296466308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b0021250 msgr2=0x55f0b0026ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.555+0000 7fa077400640 1 --2- 172.21.15.47:0/296466308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b0021250 0x55f0b0026ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.555+0000 7fa077400640 1 -- 172.21.15.47:0/296466308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b00183a0 msgr2=0x55f0b0023ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.555+0000 7fa077400640 1 --2- 172.21.15.47:0/296466308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b00183a0 0x55f0b0023ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.555+0000 7fa077400640 1 -- 172.21.15.47:0/296466308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0b00274b0 con 0x55f0b0016810 2024-03-21T11:40:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.555+0000 7fa076a00640 1 --2- 172.21.15.47:0/296466308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b00183a0 0x55f0b0023ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.555+0000 7fa077400640 1 --2- 172.21.15.47:0/296466308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b0016810 0x55f0b0023960 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa068005fa0 tx=0x7fa068029f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cae047dd6b6de2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.555+0000 7fa060c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:40:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.555+0000 7fa060c00640 1 -- 172.21.15.47:0/296466308 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068035020 con 0x55f0b0016810 2024-03-21T11:40:55.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.556+0000 7fa080fc3740 1 -- 172.21.15.47:0/296466308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa06802d070 con 0x55f0b0016810 2024-03-21T11:40:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.556+0000 7fa060c00640 1 -- 172.21.15.47:0/296466308 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa068005040 con 0x55f0b0016810 2024-03-21T11:40:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.556+0000 7fa060c00640 1 -- 172.21.15.47:0/296466308 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa068040050 con 0x55f0b0016810 2024-03-21T11:40:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.556+0000 7fa080fc3740 10 client.20691 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.556+0000 7fa080fc3740 1 -- 172.21.15.47:0/296466308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f0b0088110 con 0x55f0b0016810 2024-03-21T11:40:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.556+0000 7fa060c00640 1 -- 172.21.15.47:0/296466308 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa06802d070 con 0x55f0b0016810 2024-03-21T11:40:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.556+0000 7fa060c00640 1 -- 172.21.15.47:0/296466308 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa068004750 con 0x55f0b0016810 2024-03-21T11:40:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.556+0000 7fa080fc3740 10 client.20691 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.556+0000 7fa080fc3740 10 client.20691 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:40:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.556+0000 7fa080fc3740 20 client.20691 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.556+0000 7fa080fc3740 1 --2- 172.21.15.47:0/296466308 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f0b0088a40 0x55f0b008ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.556+0000 7fa080fc3740 4 client.20691 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:40:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.556+0000 7fa080fc3740 1 -- 172.21.15.47:0/296466308 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f0aff82b50 con 0x55f0b0088a40 2024-03-21T11:40:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.557+0000 7fa076a00640 1 --2- 172.21.15.47:0/296466308 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f0b0088a40 0x55f0b008ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.557+0000 7fa076a00640 1 --2- 172.21.15.47:0/296466308 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f0b0088a40 0x55f0b008ae60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc3136889525b36d server_cookie=360f38a96a65d83f in_seq=0 out_seq=0 2024-03-21T11:40:55.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.557+0000 7fa060c00640 10 client.20691 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:40:55.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa060c00640 1 -- 172.21.15.47:0/296466308 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f0aff82b50 con 0x55f0b0088a40 2024-03-21T11:40:55.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa060c00640 10 client.20691 handle_command_reply: tid=0 2024-03-21T11:40:55.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa080fc3740 1 client.20691 shutdown 2024-03-21T11:40:55.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa080fc3740 1 -- 172.21.15.47:0/296466308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b0016810 msgr2=0x55f0b0023960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa080fc3740 1 --2- 172.21.15.47:0/296466308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b0016810 0x55f0b0023960 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa068005fa0 tx=0x7fa068029f80 comp rx=0 tx=0).stop 2024-03-21T11:40:55.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa080fc3740 1 -- 172.21.15.47:0/296466308 shutdown_connections 2024-03-21T11:40:55.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa080fc3740 1 --2- 172.21.15.47:0/296466308 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f0b0088a40 0x55f0b008ae60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa080fc3740 1 --2- 172.21.15.47:0/296466308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0b0021250 0x55f0b0026ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa080fc3740 1 --2- 172.21.15.47:0/296466308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0b00183a0 0x55f0b0023ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa080fc3740 1 --2- 172.21.15.47:0/296466308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f0b0016810 0x55f0b0023960 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa060c00640 0 client.20691 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:40:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa080fc3740 1 -- 172.21.15.47:0/296466308 >> 172.21.15.47:0/296466308 conn(0x55f0b0011c00 msgr2=0x55f0b0014e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa080fc3740 1 -- 172.21.15.47:0/296466308 shutdown_connections 2024-03-21T11:40:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa080fc3740 1 -- 172.21.15.47:0/296466308 wait complete. 2024-03-21T11:40:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.560+0000 7fa080fc3740 20 client.20691 trim_cache size 0 max 16384 2024-03-21T11:40:55.563 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:40:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.561+0000 7fa07e200640 1 -- 172.21.15.47:0/1592633373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa05c0a3070 msgr2=0x7fa05c0a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.561+0000 7fa07e200640 1 --2- 172.21.15.47:0/1592633373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa05c0a3070 0x7fa05c0a5530 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa068005fd0 tx=0x7fa06800d5a0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.561+0000 7fa07e200640 1 -- 172.21.15.47:0/1592633373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa078104ad0 msgr2=0x7fa07819e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.561+0000 7fa07e200640 1 --2- 172.21.15.47:0/1592633373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa078104ad0 0x7fa07819e510 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fa06400e9f0 tx=0x7fa06400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.561+0000 7fa07e200640 1 -- 172.21.15.47:0/1592633373 shutdown_connections 2024-03-21T11:40:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.561+0000 7fa07e200640 1 --2- 172.21.15.47:0/1592633373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa05c0a3070 0x7fa05c0a5530 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.561+0000 7fa07e200640 1 --2- 172.21.15.47:0/1592633373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa078105490 0x7fa0781a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.561+0000 7fa07e200640 1 --2- 172.21.15.47:0/1592633373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa078104ad0 0x7fa07819e510 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.561+0000 7fa07e200640 1 --2- 172.21.15.47:0/1592633373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0781038e0 0x7fa07819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.561+0000 7fa07e200640 1 -- 172.21.15.47:0/1592633373 >> 172.21.15.47:0/1592633373 conn(0x7fa0780fee50 msgr2=0x7fa078077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.561+0000 7fa07e200640 1 -- 172.21.15.47:0/1592633373 shutdown_connections 2024-03-21T11:40:55.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.561+0000 7fa07e200640 1 -- 172.21.15.47:0/1592633373 wait complete. 2024-03-21T11:40:55.579 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:40:55.579 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:40:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.798+0000 7f3fc3e00640 1 -- 172.21.15.47:0/3280049503 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3fbc104ad0 msgr2=0x7f3fbc104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.798+0000 7f3fc3e00640 1 --2- 172.21.15.47:0/3280049503 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3fbc104ad0 0x7f3fbc104f50 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009860 tx=0x7f3fa802bb70 comp rx=0 tx=0).stop 2024-03-21T11:40:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.799+0000 7f3fc3e00640 1 -- 172.21.15.47:0/3280049503 shutdown_connections 2024-03-21T11:40:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.799+0000 7f3fc3e00640 1 --2- 172.21.15.47:0/3280049503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fbc105490 0x7f3fbc10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.799+0000 7f3fc3e00640 1 --2- 172.21.15.47:0/3280049503 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3fbc104ad0 0x7f3fbc104f50 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.799+0000 7f3fc3e00640 1 --2- 172.21.15.47:0/3280049503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fbc1038e0 0x7f3fbc103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.799+0000 7f3fc3e00640 1 -- 172.21.15.47:0/3280049503 >> 172.21.15.47:0/3280049503 conn(0x7f3fbc0fee50 msgr2=0x7f3fbc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.799+0000 7f3fc3e00640 1 -- 172.21.15.47:0/3280049503 shutdown_connections 2024-03-21T11:40:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.799+0000 7f3fc3e00640 1 -- 172.21.15.47:0/3280049503 wait complete. 2024-03-21T11:40:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.800+0000 7f3fc3e00640 1 Processor -- start 2024-03-21T11:40:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.800+0000 7f3fc3e00640 1 -- start start 2024-03-21T11:40:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.800+0000 7f3fc3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3fbc1038e0 0x7f3fbc199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.800+0000 7f3fc3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fbc104ad0 0x7f3fbc19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.800+0000 7f3fc3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fbc105490 0x7f3fbc1a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.800+0000 7f3fc3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc10f540 con 0x7f3fbc105490 2024-03-21T11:40:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.800+0000 7f3fc3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc10f3c0 con 0x7f3fbc104ad0 2024-03-21T11:40:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.800+0000 7f3fc3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc10f6c0 con 0x7f3fbc1038e0 2024-03-21T11:40:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.800+0000 7f3fc1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3fbc1038e0 0x7f3fbc199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.800+0000 7f3fc1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3fbc1038e0 0x7f3fbc199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42156/0 (socket says 172.21.15.47:42156) 2024-03-21T11:40:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.800+0000 7f3fc1400640 1 -- 172.21.15.47:0/2664461029 learned_addr learned my addr 172.21.15.47:0/2664461029 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.800+0000 7f3fc0a00640 1 --2- 172.21.15.47:0/2664461029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fbc104ad0 0x7f3fbc19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.800+0000 7f3fc1e00640 1 --2- 172.21.15.47:0/2664461029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fbc105490 0x7f3fbc1a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.801+0000 7f3fc1400640 1 -- 172.21.15.47:0/2664461029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fbc104ad0 msgr2=0x7f3fbc19a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.801+0000 7f3fc1400640 1 --2- 172.21.15.47:0/2664461029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fbc104ad0 0x7f3fbc19a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.801+0000 7f3fc1400640 1 -- 172.21.15.47:0/2664461029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fbc105490 msgr2=0x7f3fbc1a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.801+0000 7f3fc1400640 1 --2- 172.21.15.47:0/2664461029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fbc105490 0x7f3fbc1a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.801+0000 7f3fc1400640 1 -- 172.21.15.47:0/2664461029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc1a17f0 con 0x7f3fbc1038e0 2024-03-21T11:40:55.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.801+0000 7f3fc1400640 1 --2- 172.21.15.47:0/2664461029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3fbc1038e0 0x7f3fbc199c70 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000e9c0 tx=0x7f3fb000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.801+0000 7f3fba000640 1 -- 172.21.15.47:0/2664461029 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb000cd80 con 0x7f3fbc1038e0 2024-03-21T11:40:55.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.801+0000 7f3fba000640 1 -- 172.21.15.47:0/2664461029 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fb00035f0 con 0x7f3fbc1038e0 2024-03-21T11:40:55.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.801+0000 7f3fba000640 1 -- 172.21.15.47:0/2664461029 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0010690 con 0x7f3fbc1038e0 2024-03-21T11:40:55.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.801+0000 7f3fc3e00640 1 -- 172.21.15.47:0/2664461029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc1a1ae0 con 0x7f3fbc1038e0 2024-03-21T11:40:55.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.802+0000 7f3fc3e00640 1 -- 172.21.15.47:0/2664461029 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc1a2000 con 0x7f3fbc1038e0 2024-03-21T11:40:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.805+0000 7f3fc6b6f740 1 Processor -- start 2024-03-21T11:40:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.805+0000 7f3fc6b6f740 1 -- start start 2024-03-21T11:40:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.806+0000 7f3fba000640 1 -- 172.21.15.47:0/2664461029 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3fb00037a0 con 0x7f3fbc1038e0 2024-03-21T11:40:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.806+0000 7f3fc6b6f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x562876393c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.806+0000 7f3fc6b6f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763949e0 0x562876394e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.806+0000 7f3fc6b6f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763953a0 0x56287639bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.806+0000 7f3fc6b6f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628762f82c0 con 0x562876393810 2024-03-21T11:40:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.806+0000 7f3fc6b6f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628762a0c90 con 0x5628763949e0 2024-03-21T11:40:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.806+0000 7f3fc6b6f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628762f4360 con 0x5628763953a0 2024-03-21T11:40:55.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.806+0000 7f3fc1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x562876393c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.806+0000 7f3fc0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763953a0 0x56287639bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.806+0000 7f3fc1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x562876393c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39304/0 (socket says 172.21.15.47:39304) 2024-03-21T11:40:55.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.806+0000 7f3fc1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763949e0 0x562876394e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.806+0000 7f3fc1e00640 1 -- 172.21.15.47:0/3602395562 learned_addr learned my addr 172.21.15.47:0/3602395562 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.807+0000 7f3fc0a00640 1 -- 172.21.15.47:0/3602395562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763949e0 msgr2=0x562876394e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.807+0000 7f3fc0a00640 1 --2- 172.21.15.47:0/3602395562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763949e0 0x562876394e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.807+0000 7f3fc0a00640 1 -- 172.21.15.47:0/3602395562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 msgr2=0x562876393c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.807+0000 7f3fc0a00640 1 --2- 172.21.15.47:0/3602395562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x562876393c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.807+0000 7f3fc0a00640 1 -- 172.21.15.47:0/3602395562 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628762c5080 con 0x5628763953a0 2024-03-21T11:40:55.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.807+0000 7f3fc1e00640 1 --2- 172.21.15.47:0/3602395562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x562876393c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:55.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.807+0000 7f3fc0a00640 1 --2- 172.21.15.47:0/3602395562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763953a0 0x56287639bc30 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80349e0 tx=0x7f3fa8009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=113f48695e593e57 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.807+0000 7f3fae600640 1 -- 172.21.15.47:0/3602395562 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8039040 con 0x5628763953a0 2024-03-21T11:40:55.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.807+0000 7f3fae600640 1 -- 172.21.15.47:0/3602395562 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fa8034db0 con 0x5628763953a0 2024-03-21T11:40:55.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.807+0000 7f3fae600640 1 -- 172.21.15.47:0/3602395562 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa802c890 con 0x5628763953a0 2024-03-21T11:40:55.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.808+0000 7f3fba000640 1 --2- 172.21.15.47:0/2664461029 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3fa40a2f20 0x7f3fa40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.808+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/3602395562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763953a0 msgr2=0x56287639bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.808+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/3602395562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763953a0 0x56287639bc30 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80349e0 tx=0x7f3fa8009490 comp rx=0 tx=0).stop 2024-03-21T11:40:55.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.808+0000 7f3fba000640 1 -- 172.21.15.47:0/2664461029 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3fb00cfbb0 con 0x7f3fbc1038e0 2024-03-21T11:40:55.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.808+0000 7f3fc1e00640 1 --2- 172.21.15.47:0/2664461029 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3fa40a2f20 0x7f3fa40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.808+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/3602395562 shutdown_connections 2024-03-21T11:40:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.808+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/3602395562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763953a0 0x56287639bc30 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.808+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/3602395562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763949e0 0x562876394e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.808+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/3602395562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x562876393c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.808+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/3602395562 >> 172.21.15.47:0/3602395562 conn(0x56287638ec00 msgr2=0x56287638f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.809+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/3602395562 shutdown_connections 2024-03-21T11:40:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.809+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/3602395562 wait complete. 2024-03-21T11:40:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.809+0000 7f3fc6b6f740 1 -- start start 2024-03-21T11:40:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.809+0000 7f3fc1e00640 1 --2- 172.21.15.47:0/2664461029 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3fa40a2f20 0x7f3fa40a53e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009fd0 tx=0x7f3fb4010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.809+0000 7f3fc6b6f740 1 Processor -- start 2024-03-21T11:40:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.810+0000 7f3fc6b6f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562876393810 0x5628763a6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.810+0000 7f3fc6b6f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763949e0 0x5628763a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.810+0000 7f3fc6b6f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628763953a0 0x5628763ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.810+0000 7f3fc6b6f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56287639c350 con 0x5628763953a0 2024-03-21T11:40:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.810+0000 7f3fc6b6f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628762b40d0 con 0x562876393810 2024-03-21T11:40:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.810+0000 7f3fc6b6f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56287639c4d0 con 0x5628763949e0 2024-03-21T11:40:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.810+0000 7f3fc6b6f740 10 client.0 _ll_register_callbacks cb 0x5628762b8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.810+0000 7f3fc1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562876393810 0x5628763a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.810+0000 7f3fc1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628763953a0 0x5628763ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.810+0000 7f3fc0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763949e0 0x5628763a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.810+0000 7f3fc1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628763953a0 0x5628763ad6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39312/0 (socket says 172.21.15.47:39312) 2024-03-21T11:40:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.810+0000 7f3fc1e00640 1 -- 172.21.15.47:0/449431051 learned_addr learned my addr 172.21.15.47:0/449431051 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.810+0000 7f3fc1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562876393810 0x5628763a6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47164/0 (socket says 172.21.15.47:47164) 2024-03-21T11:40:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.811+0000 7f3fc1e00640 1 -- 172.21.15.47:0/449431051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763949e0 msgr2=0x5628763a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.811+0000 7f3fc1e00640 1 --2- 172.21.15.47:0/449431051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763949e0 0x5628763a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.811+0000 7f3fc1e00640 1 -- 172.21.15.47:0/449431051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562876393810 msgr2=0x5628763a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.811+0000 7f3fc1e00640 1 --2- 172.21.15.47:0/449431051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562876393810 0x5628763a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.811+0000 7f3fc1e00640 1 -- 172.21.15.47:0/449431051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56287600c3f0 con 0x5628763953a0 2024-03-21T11:40:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.811+0000 7f3fc1400640 1 --2- 172.21.15.47:0/449431051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562876393810 0x5628763a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.811+0000 7f3fc1e00640 1 --2- 172.21.15.47:0/449431051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628763953a0 0x5628763ad6f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400ffd0 tx=0x7f3fb4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f5576782a6febc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.811+0000 7f3fa2a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:40:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.812+0000 7f3fa2a00640 1 -- 172.21.15.47:0/449431051 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4018630 con 0x5628763953a0 2024-03-21T11:40:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.812+0000 7f3fa2a00640 1 -- 172.21.15.47:0/449431051 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3fb40187d0 con 0x5628763953a0 2024-03-21T11:40:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.812+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/449431051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fa8034db0 con 0x5628763953a0 2024-03-21T11:40:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.812+0000 7f3fa2a00640 1 -- 172.21.15.47:0/449431051 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fb40189d0 con 0x5628763953a0 2024-03-21T11:40:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.812+0000 7f3fc6b6f740 10 client.20703 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.812+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/449431051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5628760d67d0 con 0x5628763953a0 2024-03-21T11:40:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.813+0000 7f3fa2a00640 1 -- 172.21.15.47:0/449431051 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fa8034db0 con 0x5628763953a0 2024-03-21T11:40:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.813+0000 7f3fa2a00640 1 -- 172.21.15.47:0/449431051 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3fb4024070 con 0x5628763953a0 2024-03-21T11:40:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.813+0000 7f3fc6b6f740 10 client.20703 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.813+0000 7f3fc6b6f740 10 client.20703 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:40:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.813+0000 7f3fc6b6f740 20 client.20703 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.813+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/449431051 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5628763b1ce0 0x5628763b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.813+0000 7f3fc6b6f740 4 client.20703 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.813+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/449431051 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562876305800 con 0x5628763b1ce0 2024-03-21T11:40:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.814+0000 7f3fc1400640 1 --2- 172.21.15.47:0/449431051 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5628763b1ce0 0x5628763b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.814+0000 7f3fc1400640 1 --2- 172.21.15.47:0/449431051 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5628763b1ce0 0x5628763b4150 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4d354f129914b75 server_cookie=1e50b6b610d49501 in_seq=0 out_seq=0 2024-03-21T11:40:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.814+0000 7f3fa2a00640 10 client.20703 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:40:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.815+0000 7f3fa2a00640 1 -- 172.21.15.47:0/449431051 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562876305800 con 0x5628763b1ce0 2024-03-21T11:40:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.815+0000 7f3fa2a00640 10 client.20703 handle_command_reply: tid=0 2024-03-21T11:40:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.815+0000 7f3fc6b6f740 1 client.20703 shutdown 2024-03-21T11:40:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.815+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/449431051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628763953a0 msgr2=0x5628763ad6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.815+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/449431051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628763953a0 0x5628763ad6f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400ffd0 tx=0x7f3fb4004ab0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.816+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/449431051 shutdown_connections 2024-03-21T11:40:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.816+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/449431051 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5628763b1ce0 0x5628763b4150 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.816+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/449431051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628763953a0 0x5628763ad6f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.816+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/449431051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763949e0 0x5628763a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.816+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/449431051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562876393810 0x5628763a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.816+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/449431051 >> 172.21.15.47:0/449431051 conn(0x56287638ec00 msgr2=0x56287638deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.816+0000 7f3fa2a00640 0 client.20703 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:40:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.816+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/449431051 shutdown_connections 2024-03-21T11:40:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.816+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/449431051 wait complete. 2024-03-21T11:40:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.817+0000 7f3fc6b6f740 20 client.20703 trim_cache size 0 max 16384 2024-03-21T11:40:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.833+0000 7f3fc6b6f740 1 Processor -- start 2024-03-21T11:40:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.833+0000 7f3fc6b6f740 1 -- start start 2024-03-21T11:40:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc6b6f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x56287638d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc6b6f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763949e0 0x5628763ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc6b6f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763953a0 0x56287638db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc6b6f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc1a7e60 con 0x562876393810 2024-03-21T11:40:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc6b6f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc1a97e0 con 0x5628763949e0 2024-03-21T11:40:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc6b6f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc1a84c0 con 0x5628763953a0 2024-03-21T11:40:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc0a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763949e0 0x5628763ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc0a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763949e0 0x5628763ad6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47176/0 (socket says 172.21.15.47:47176) 2024-03-21T11:40:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc0a00640 1 -- 172.21.15.47:0/1254338263 learned_addr learned my addr 172.21.15.47:0/1254338263 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc1e00640 1 --2- 172.21.15.47:0/1254338263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763953a0 0x56287638db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x56287638d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc0a00640 1 -- 172.21.15.47:0/1254338263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763953a0 msgr2=0x56287638db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc0a00640 1 --2- 172.21.15.47:0/1254338263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763953a0 0x56287638db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc0a00640 1 -- 172.21.15.47:0/1254338263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 msgr2=0x56287638d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc0a00640 1 --2- 172.21.15.47:0/1254338263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x56287638d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.834+0000 7f3fc0a00640 1 -- 172.21.15.47:0/1254338263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628763b44c0 con 0x5628763949e0 2024-03-21T11:40:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.835+0000 7f3fc0a00640 1 --2- 172.21.15.47:0/1254338263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763949e0 0x5628763ad6b0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009860 tx=0x7f3fa8005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da4b34121565b39b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.835+0000 7f3fadc00640 1 -- 172.21.15.47:0/1254338263 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa804f070 con 0x5628763949e0 2024-03-21T11:40:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.835+0000 7f3fadc00640 1 -- 172.21.15.47:0/1254338263 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fa8034db0 con 0x5628763949e0 2024-03-21T11:40:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.835+0000 7f3fadc00640 1 -- 172.21.15.47:0/1254338263 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8039040 con 0x5628763949e0 2024-03-21T11:40:55.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.836+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/1254338263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763949e0 msgr2=0x5628763ad6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.836+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/1254338263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763949e0 0x5628763ad6b0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009860 tx=0x7f3fa8005780 comp rx=0 tx=0).stop 2024-03-21T11:40:55.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.836+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/1254338263 shutdown_connections 2024-03-21T11:40:55.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.836+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/1254338263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763953a0 0x56287638db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.836+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/1254338263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763949e0 0x5628763ad6b0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.836+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/1254338263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x56287638d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.836+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/1254338263 >> 172.21.15.47:0/1254338263 conn(0x56287638ec00 msgr2=0x5628763926f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.836+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/1254338263 shutdown_connections 2024-03-21T11:40:55.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.836+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/1254338263 wait complete. 2024-03-21T11:40:55.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.836+0000 7f3fc6b6f740 1 -- start start 2024-03-21T11:40:55.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.837+0000 7f3fc6b6f740 1 Processor -- start 2024-03-21T11:40:55.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.837+0000 7f3fc6b6f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x5628763aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.837+0000 7f3fc6b6f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763949e0 0x5628763ab510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.837+0000 7f3fc6b6f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763953a0 0x5628763ac520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.837+0000 7f3fc6b6f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc1a9180 con 0x562876393810 2024-03-21T11:40:55.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.837+0000 7f3fc6b6f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc1a8b20 con 0x5628763953a0 2024-03-21T11:40:55.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.837+0000 7f3fc6b6f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc1a6020 con 0x5628763949e0 2024-03-21T11:40:55.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.837+0000 7f3fc6b6f740 10 client.0 _ll_register_callbacks cb 0x7f3fb400d4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.837+0000 7f3fc1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x5628763aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.837+0000 7f3fc1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763953a0 0x5628763ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.837+0000 7f3fc0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763949e0 0x5628763ab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.837+0000 7f3fc1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x5628763aafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39328/0 (socket says 172.21.15.47:39328) 2024-03-21T11:40:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.837+0000 7f3fc1400640 1 -- 172.21.15.47:0/3864654747 learned_addr learned my addr 172.21.15.47:0/3864654747 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.838+0000 7f3fc1e00640 1 -- 172.21.15.47:0/3864654747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763949e0 msgr2=0x5628763ab510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.838+0000 7f3fc1e00640 1 --2- 172.21.15.47:0/3864654747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763949e0 0x5628763ab510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.838+0000 7f3fc1e00640 1 -- 172.21.15.47:0/3864654747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 msgr2=0x5628763aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.838+0000 7f3fc1e00640 1 --2- 172.21.15.47:0/3864654747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x5628763aafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.838+0000 7f3fc1e00640 1 -- 172.21.15.47:0/3864654747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628763acb10 con 0x5628763953a0 2024-03-21T11:40:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.838+0000 7f3fc1400640 1 --2- 172.21.15.47:0/3864654747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x5628763aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.838+0000 7f3fc0a00640 1 --2- 172.21.15.47:0/3864654747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763949e0 0x5628763ab510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.838+0000 7f3fc1e00640 1 --2- 172.21.15.47:0/3864654747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763953a0 0x5628763ac520 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400ee30 tx=0x7f3fb40188e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9121cbd43f518e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.838+0000 7f3fa2a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.838+0000 7f3fa2a00640 1 -- 172.21.15.47:0/3864654747 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb401acb0 con 0x5628763953a0 2024-03-21T11:40:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.838+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/3864654747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fa8034db0 con 0x5628763953a0 2024-03-21T11:40:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.839+0000 7f3fa2a00640 1 -- 172.21.15.47:0/3864654747 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3fb4018d80 con 0x5628763953a0 2024-03-21T11:40:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.839+0000 7f3fa2a00640 1 -- 172.21.15.47:0/3864654747 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fb4048050 con 0x5628763953a0 2024-03-21T11:40:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.839+0000 7f3fc6b6f740 10 client.31253 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.839+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/3864654747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56287640d830 con 0x5628763953a0 2024-03-21T11:40:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.839+0000 7f3fa2a00640 1 -- 172.21.15.47:0/3864654747 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fa8034db0 con 0x5628763953a0 2024-03-21T11:40:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.839+0000 7f3fa2a00640 1 -- 172.21.15.47:0/3864654747 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3fb400b040 con 0x5628763953a0 2024-03-21T11:40:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.839+0000 7f3fc6b6f740 10 client.31253 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.839+0000 7f3fc6b6f740 10 client.31253 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:40:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.839+0000 7f3fc6b6f740 20 client.31253 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.839+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/3864654747 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56287640e120 0x562876410540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.839+0000 7f3fc6b6f740 4 client.31253 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:40:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.839+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/3864654747 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5628762ffb50 con 0x56287640e120 2024-03-21T11:40:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.840+0000 7f3fc1400640 1 --2- 172.21.15.47:0/3864654747 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56287640e120 0x562876410540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.840+0000 7f3fc1400640 1 --2- 172.21.15.47:0/3864654747 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56287640e120 0x562876410540 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8caefa9cb4d4e296 server_cookie=e6de7d669689b5 in_seq=0 out_seq=0 2024-03-21T11:40:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.840+0000 7f3fa2a00640 10 client.31253 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:40:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.841+0000 7f3fa2a00640 1 -- 172.21.15.47:0/3864654747 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5628762ffb50 con 0x56287640e120 2024-03-21T11:40:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.841+0000 7f3fa2a00640 10 client.31253 handle_command_reply: tid=0 2024-03-21T11:40:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.841+0000 7f3fc6b6f740 1 client.31253 shutdown 2024-03-21T11:40:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.841+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/3864654747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763953a0 msgr2=0x5628763ac520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.841+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/3864654747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763953a0 0x5628763ac520 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400ee30 tx=0x7f3fb40188e0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.841+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/3864654747 shutdown_connections 2024-03-21T11:40:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.841+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/3864654747 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56287640e120 0x562876410540 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.841+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/3864654747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628763953a0 0x5628763ac520 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.841+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/3864654747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628763949e0 0x5628763ab510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.841+0000 7f3fc6b6f740 1 --2- 172.21.15.47:0/3864654747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562876393810 0x5628763aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.841+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/3864654747 >> 172.21.15.47:0/3864654747 conn(0x56287638ec00 msgr2=0x56287638d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.841+0000 7f3fa2a00640 0 client.31253 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:40:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.841+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/3864654747 shutdown_connections 2024-03-21T11:40:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.841+0000 7f3fc6b6f740 1 -- 172.21.15.47:0/3864654747 wait complete. 2024-03-21T11:40:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.842+0000 7f3fc6b6f740 20 client.31253 trim_cache size 0 max 16384 2024-03-21T11:40:55.850 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:40:55.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.842+0000 7f3fc3e00640 1 -- 172.21.15.47:0/2664461029 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3fa40a2f20 msgr2=0x7f3fa40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.842+0000 7f3fc3e00640 1 --2- 172.21.15.47:0/2664461029 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3fa40a2f20 0x7f3fa40a53e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009fd0 tx=0x7f3fb4010040 comp rx=0 tx=0).stop 2024-03-21T11:40:55.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.842+0000 7f3fc3e00640 1 -- 172.21.15.47:0/2664461029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3fbc1038e0 msgr2=0x7f3fbc199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:55.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.842+0000 7f3fc3e00640 1 --2- 172.21.15.47:0/2664461029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3fbc1038e0 0x7f3fbc199c70 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000e9c0 tx=0x7f3fb000ee80 comp rx=0 tx=0).stop 2024-03-21T11:40:55.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.842+0000 7f3fc3e00640 1 -- 172.21.15.47:0/2664461029 shutdown_connections 2024-03-21T11:40:55.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.842+0000 7f3fc3e00640 1 --2- 172.21.15.47:0/2664461029 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3fa40a2f20 0x7f3fa40a53e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.842+0000 7f3fc3e00640 1 --2- 172.21.15.47:0/2664461029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fbc105490 0x7f3fbc1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.842+0000 7f3fc3e00640 1 --2- 172.21.15.47:0/2664461029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fbc104ad0 0x7f3fbc19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.842+0000 7f3fc3e00640 1 --2- 172.21.15.47:0/2664461029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3fbc1038e0 0x7f3fbc199c70 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:55.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.842+0000 7f3fc3e00640 1 -- 172.21.15.47:0/2664461029 >> 172.21.15.47:0/2664461029 conn(0x7f3fbc0fee50 msgr2=0x7f3fbc107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:55.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.843+0000 7f3fc3e00640 1 -- 172.21.15.47:0/2664461029 shutdown_connections 2024-03-21T11:40:55.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:55.843+0000 7f3fc3e00640 1 -- 172.21.15.47:0/2664461029 wait complete. 2024-03-21T11:40:55.862 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:40:55.862 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:40:55.945 INFO:tasks.workunit.client.0.smithi047.stdout: 10 103380 24.46 MB/sec execute 99 sec latency 64.453 ms 2024-03-21T11:40:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.072+0000 7fc7aec00640 1 -- 172.21.15.47:0/1433888245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7a8109870 msgr2=0x7fc7a8110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.072+0000 7fc7aec00640 1 --2- 172.21.15.47:0/1433888245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7a8109870 0x7fc7a8110100 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009ca0 tx=0x7fc7a002c0f0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.072+0000 7fc7aec00640 1 -- 172.21.15.47:0/1433888245 shutdown_connections 2024-03-21T11:40:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.072+0000 7fc7aec00640 1 --2- 172.21.15.47:0/1433888245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7a8109870 0x7fc7a8110100 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.072+0000 7fc7aec00640 1 --2- 172.21.15.47:0/1433888245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7a8108eb0 0x7fc7a8109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.072+0000 7fc7aec00640 1 --2- 172.21.15.47:0/1433888245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7a8107cc0 0x7fc7a81080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.072+0000 7fc7aec00640 1 -- 172.21.15.47:0/1433888245 >> 172.21.15.47:0/1433888245 conn(0x7fc7a8076c60 msgr2=0x7fc7a80790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.073+0000 7fc7aec00640 1 -- 172.21.15.47:0/1433888245 shutdown_connections 2024-03-21T11:40:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.073+0000 7fc7aec00640 1 -- 172.21.15.47:0/1433888245 wait complete. 2024-03-21T11:40:56.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.074+0000 7fc7aec00640 1 Processor -- start 2024-03-21T11:40:56.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.074+0000 7fc7aec00640 1 -- start start 2024-03-21T11:40:56.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.074+0000 7fc7aec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7a8107cc0 0x7fc7a819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.074+0000 7fc7aec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7a8108eb0 0x7fc7a819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.074+0000 7fc7aec00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7a8109870 0x7fc7a81a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.074+0000 7fc7aec00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a8113940 con 0x7fc7a8109870 2024-03-21T11:40:56.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.074+0000 7fc7aec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a81137c0 con 0x7fc7a8107cc0 2024-03-21T11:40:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.074+0000 7fc7aec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a8113ac0 con 0x7fc7a8108eb0 2024-03-21T11:40:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.074+0000 7fc79f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7a8108eb0 0x7fc7a819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.075+0000 7fc79f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7a8108eb0 0x7fc7a819e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42212/0 (socket says 172.21.15.47:42212) 2024-03-21T11:40:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.075+0000 7fc79f400640 1 -- 172.21.15.47:0/2493817027 learned_addr learned my addr 172.21.15.47:0/2493817027 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.075+0000 7fc7a7e00640 1 --2- 172.21.15.47:0/2493817027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7a8107cc0 0x7fc7a819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.075+0000 7fc7acc00640 1 --2- 172.21.15.47:0/2493817027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7a8109870 0x7fc7a81a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.075+0000 7fc79f400640 1 -- 172.21.15.47:0/2493817027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7a8107cc0 msgr2=0x7fc7a819df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.075+0000 7fc79f400640 1 --2- 172.21.15.47:0/2493817027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7a8107cc0 0x7fc7a819df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.075+0000 7fc79f400640 1 -- 172.21.15.47:0/2493817027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7a8109870 msgr2=0x7fc7a81a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.075+0000 7fc79f400640 1 --2- 172.21.15.47:0/2493817027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7a8109870 0x7fc7a81a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.075+0000 7fc79f400640 1 -- 172.21.15.47:0/2493817027 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a81a5b70 con 0x7fc7a8108eb0 2024-03-21T11:40:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.075+0000 7fc79f400640 1 --2- 172.21.15.47:0/2493817027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7a8108eb0 0x7fc7a819e490 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc79000e9f0 tx=0x7fc79000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:56.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.076+0000 7fc7a5600640 1 -- 172.21.15.47:0/2493817027 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79000cd80 con 0x7fc7a8108eb0 2024-03-21T11:40:56.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.076+0000 7fc7a5600640 1 -- 172.21.15.47:0/2493817027 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc790005500 con 0x7fc7a8108eb0 2024-03-21T11:40:56.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.076+0000 7fc7aec00640 1 -- 172.21.15.47:0/2493817027 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a81a5e60 con 0x7fc7a8108eb0 2024-03-21T11:40:56.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.076+0000 7fc7a5600640 1 -- 172.21.15.47:0/2493817027 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc790010690 con 0x7fc7a8108eb0 2024-03-21T11:40:56.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.076+0000 7fc7aec00640 1 -- 172.21.15.47:0/2493817027 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a81a6320 con 0x7fc7a8108eb0 2024-03-21T11:40:56.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.080+0000 7fc7a5600640 1 -- 172.21.15.47:0/2493817027 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc790004f10 con 0x7fc7a8108eb0 2024-03-21T11:40:56.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.080+0000 7fc7b19b7740 1 Processor -- start 2024-03-21T11:40:56.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.080+0000 7fc7b19b7740 1 -- start start 2024-03-21T11:40:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.081+0000 7fc7b19b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616aa810 0x55fe616aac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.081+0000 7fc7b19b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616ab9e0 0x55fe616abe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.081+0000 7fc7b19b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.081+0000 7fc7b19b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6160f2c0 con 0x55fe616ab9e0 2024-03-21T11:40:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.081+0000 7fc7b19b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe615b7c90 con 0x55fe616aa810 2024-03-21T11:40:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.081+0000 7fc7b19b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6160b360 con 0x55fe616ac3a0 2024-03-21T11:40:56.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.081+0000 7fc7acc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616aa810 0x55fe616aac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.081+0000 7fc7acc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616aa810 0x55fe616aac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47208/0 (socket says 172.21.15.47:47208) 2024-03-21T11:40:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.081+0000 7fc7acc00640 1 -- 172.21.15.47:0/2443571393 learned_addr learned my addr 172.21.15.47:0/2443571393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.081+0000 7fc79f400640 1 --2- 172.21.15.47:0/2443571393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.082+0000 7fc7a7e00640 1 --2- 172.21.15.47:0/2443571393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616ab9e0 0x55fe616abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.082+0000 7fc7a5600640 1 --2- 172.21.15.47:0/2493817027 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc78c0a2fa0 0x7fc78c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.082+0000 7fc79f400640 1 -- 172.21.15.47:0/2443571393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616aa810 msgr2=0x55fe616aac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.082+0000 7fc79f400640 1 --2- 172.21.15.47:0/2443571393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616aa810 0x55fe616aac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.082+0000 7fc79f400640 1 -- 172.21.15.47:0/2443571393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616ab9e0 msgr2=0x55fe616abe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.082+0000 7fc79f400640 1 --2- 172.21.15.47:0/2443571393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616ab9e0 0x55fe616abe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.082+0000 7fc7a5600640 1 -- 172.21.15.47:0/2493817027 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc7900cf930 con 0x7fc7a8108eb0 2024-03-21T11:40:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.082+0000 7fc79f400640 1 -- 172.21.15.47:0/2443571393 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe615dc080 con 0x55fe616ac3a0 2024-03-21T11:40:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.082+0000 7fc7a7e00640 1 --2- 172.21.15.47:0/2443571393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616ab9e0 0x55fe616abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.083+0000 7fc7acc00640 1 --2- 172.21.15.47:0/2493817027 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc78c0a2fa0 0x7fc78c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.083+0000 7fc79f400640 1 --2- 172.21.15.47:0/2443571393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616b2c30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc7900cd070 tx=0x7fc790009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de353099f3424eef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.083+0000 7fc79d200640 1 -- 172.21.15.47:0/2443571393 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79009c3d0 con 0x55fe616ac3a0 2024-03-21T11:40:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.083+0000 7fc79d200640 1 -- 172.21.15.47:0/2443571393 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc79009bcd0 con 0x55fe616ac3a0 2024-03-21T11:40:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.084+0000 7fc7b19b7740 1 -- 172.21.15.47:0/2443571393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 msgr2=0x55fe616b2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.084+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/2443571393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616b2c30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc7900cd070 tx=0x7fc790009780 comp rx=0 tx=0).stop 2024-03-21T11:40:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.084+0000 7fc7acc00640 1 --2- 172.21.15.47:0/2493817027 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc78c0a2fa0 0x7fc78c0a5460 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0006010 tx=0x7fc7a002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.084+0000 7fc7b19b7740 1 -- 172.21.15.47:0/2443571393 shutdown_connections 2024-03-21T11:40:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.084+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/2443571393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616b2c30 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.084+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/2443571393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616ab9e0 0x55fe616abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.084+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/2443571393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616aa810 0x55fe616aac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.084+0000 7fc7b19b7740 1 -- 172.21.15.47:0/2443571393 >> 172.21.15.47:0/2443571393 conn(0x55fe616a5c00 msgr2=0x55fe616a6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.085+0000 7fc7b19b7740 1 -- 172.21.15.47:0/2443571393 shutdown_connections 2024-03-21T11:40:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.085+0000 7fc7b19b7740 1 -- 172.21.15.47:0/2443571393 wait complete. 2024-03-21T11:40:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.085+0000 7fc7b19b7740 1 -- start start 2024-03-21T11:40:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.085+0000 7fc7b19b7740 1 Processor -- start 2024-03-21T11:40:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.086+0000 7fc7b19b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 0x55fe616b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.086+0000 7fc7b19b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ab9e0 0x55fe616b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.086+0000 7fc7b19b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616bbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.086+0000 7fc7b19b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe616b32f0 con 0x55fe616aa810 2024-03-21T11:40:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.086+0000 7fc7b19b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe615cb0d0 con 0x55fe616ab9e0 2024-03-21T11:40:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.086+0000 7fc7b19b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe616b3470 con 0x55fe616ac3a0 2024-03-21T11:40:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.086+0000 7fc7b19b7740 10 client.0 _ll_register_callbacks cb 0x55fe615cfe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.086+0000 7fc7acc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616bbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.086+0000 7fc79f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ab9e0 0x55fe616b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.087+0000 7fc79f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ab9e0 0x55fe616b4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47212/0 (socket says 172.21.15.47:47212) 2024-03-21T11:40:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.087+0000 7fc79f400640 1 -- 172.21.15.47:0/3170775714 learned_addr learned my addr 172.21.15.47:0/3170775714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.087+0000 7fc7a7e00640 1 --2- 172.21.15.47:0/3170775714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 0x55fe616b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.087+0000 7fc7acc00640 1 -- 172.21.15.47:0/3170775714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ab9e0 msgr2=0x55fe616b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:56.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.087+0000 7fc7acc00640 1 --2- 172.21.15.47:0/3170775714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ab9e0 0x55fe616b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.087+0000 7fc7acc00640 1 -- 172.21.15.47:0/3170775714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 msgr2=0x55fe616b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:56.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.087+0000 7fc7acc00640 1 --2- 172.21.15.47:0/3170775714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 0x55fe616b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.087+0000 7fc7acc00640 1 -- 172.21.15.47:0/3170775714 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe613233f0 con 0x55fe616ac3a0 2024-03-21T11:40:56.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.088+0000 7fc7acc00640 1 --2- 172.21.15.47:0/3170775714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616bbf10 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x55fe616ab8e0 tx=0x7fc7a0032590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d73d621da0f6e24c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:56.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.088+0000 7fc795600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:40:56.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.088+0000 7fc795600640 1 -- 172.21.15.47:0/3170775714 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0039040 con 0x55fe616ac3a0 2024-03-21T11:40:56.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.088+0000 7fc795600640 1 -- 172.21.15.47:0/3170775714 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc7a00417f0 con 0x55fe616ac3a0 2024-03-21T11:40:56.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.088+0000 7fc7b19b7740 1 -- 172.21.15.47:0/3170775714 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc79009bcd0 con 0x55fe616ac3a0 2024-03-21T11:40:56.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.088+0000 7fc795600640 1 -- 172.21.15.47:0/3170775714 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7a00419f0 con 0x55fe616ac3a0 2024-03-21T11:40:56.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.088+0000 7fc7b19b7740 10 client.31246 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:56.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.088+0000 7fc7b19b7740 1 -- 172.21.15.47:0/3170775714 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55fe613ed7d0 con 0x55fe616ac3a0 2024-03-21T11:40:56.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.089+0000 7fc795600640 1 -- 172.21.15.47:0/3170775714 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc79009bcd0 con 0x55fe616ac3a0 2024-03-21T11:40:56.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.089+0000 7fc795600640 1 -- 172.21.15.47:0/3170775714 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc7a003f040 con 0x55fe616ac3a0 2024-03-21T11:40:56.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.089+0000 7fc7b19b7740 10 client.31246 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:56.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.089+0000 7fc7b19b7740 10 client.31246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:56.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.089+0000 7fc7b19b7740 20 client.31246 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:56.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.089+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/3170775714 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe616c04b0 0x55fe616c2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.089+0000 7fc7b19b7740 4 client.31246 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:56.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.089+0000 7fc7b19b7740 1 -- 172.21.15.47:0/3170775714 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe6161c800 con 0x55fe616c04b0 2024-03-21T11:40:56.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.090+0000 7fc7a7e00640 1 --2- 172.21.15.47:0/3170775714 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe616c04b0 0x55fe616c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.091+0000 7fc7a7e00640 1 --2- 172.21.15.47:0/3170775714 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe616c04b0 0x55fe616c2930 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=202fbc663b354bcc server_cookie=6075aae398777b73 in_seq=0 out_seq=0 2024-03-21T11:40:56.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.091+0000 7fc795600640 10 client.31246 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:56.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.097+0000 7fc795600640 1 -- 172.21.15.47:0/3170775714 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fe6161c800 con 0x55fe616c04b0 2024-03-21T11:40:56.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.097+0000 7fc795600640 10 client.31246 handle_command_reply: tid=0 2024-03-21T11:40:56.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.097+0000 7fc7b19b7740 1 client.31246 shutdown 2024-03-21T11:40:56.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.097+0000 7fc7b19b7740 1 -- 172.21.15.47:0/3170775714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 msgr2=0x55fe616bbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.097+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/3170775714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616bbf10 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x55fe616ab8e0 tx=0x7fc7a0032590 comp rx=0 tx=0).stop 2024-03-21T11:40:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.097+0000 7fc7b19b7740 1 -- 172.21.15.47:0/3170775714 shutdown_connections 2024-03-21T11:40:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.097+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/3170775714 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe616c04b0 0x55fe616c2930 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.097+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/3170775714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616bbf10 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.097+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/3170775714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ab9e0 0x55fe616b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.097+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/3170775714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 0x55fe616b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.097+0000 7fc795600640 0 client.31246 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.097+0000 7fc7b19b7740 1 -- 172.21.15.47:0/3170775714 >> 172.21.15.47:0/3170775714 conn(0x55fe616a5c00 msgr2=0x55fe616a4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.098+0000 7fc7b19b7740 1 -- 172.21.15.47:0/3170775714 shutdown_connections 2024-03-21T11:40:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.098+0000 7fc7b19b7740 1 -- 172.21.15.47:0/3170775714 wait complete. 2024-03-21T11:40:56.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.098+0000 7fc7b19b7740 20 client.31246 trim_cache size 0 max 16384 2024-03-21T11:40:56.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.116+0000 7fc7b19b7740 1 Processor -- start 2024-03-21T11:40:56.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.116+0000 7fc7b19b7740 1 -- start start 2024-03-21T11:40:56.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.117+0000 7fc7b19b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 0x55fe616a9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.117+0000 7fc7b19b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ab9e0 0x55fe616bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.117+0000 7fc7b19b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.117+0000 7fc7b19b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78400f9e0 con 0x55fe616aa810 2024-03-21T11:40:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.117+0000 7fc7b19b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc784011360 con 0x55fe616ab9e0 2024-03-21T11:40:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.117+0000 7fc7b19b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc784010040 con 0x55fe616ac3a0 2024-03-21T11:40:56.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.117+0000 7fc7a7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 0x55fe616a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.117+0000 7fc7a7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 0x55fe616a9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39388/0 (socket says 172.21.15.47:39388) 2024-03-21T11:40:56.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.117+0000 7fc7a7e00640 1 -- 172.21.15.47:0/1446325913 learned_addr learned my addr 172.21.15.47:0/1446325913 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:56.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.117+0000 7fc79f400640 1 --2- 172.21.15.47:0/1446325913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ab9e0 0x55fe616bbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.117+0000 7fc7acc00640 1 --2- 172.21.15.47:0/1446325913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.118+0000 7fc79f400640 1 -- 172.21.15.47:0/1446325913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 msgr2=0x55fe616c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.118+0000 7fc79f400640 1 --2- 172.21.15.47:0/1446325913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.118+0000 7fc79f400640 1 -- 172.21.15.47:0/1446325913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 msgr2=0x55fe616a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.118+0000 7fc79f400640 1 --2- 172.21.15.47:0/1446325913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 0x55fe616a9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.118+0000 7fc79f400640 1 -- 172.21.15.47:0/1446325913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe616ab460 con 0x55fe616ab9e0 2024-03-21T11:40:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.118+0000 7fc7a7e00640 1 --2- 172.21.15.47:0/1446325913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 0x55fe616a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.118+0000 7fc7acc00640 1 --2- 172.21.15.47:0/1446325913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:40:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.118+0000 7fc79f400640 1 --2- 172.21.15.47:0/1446325913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ab9e0 0x55fe616bbed0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc7900cd0f0 tx=0x7fc79009b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d56eb2275750c68 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.118+0000 7fc797e00640 1 -- 172.21.15.47:0/1446325913 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79009bcd0 con 0x55fe616ab9e0 2024-03-21T11:40:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.118+0000 7fc797e00640 1 -- 172.21.15.47:0/1446325913 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc79009c3d0 con 0x55fe616ab9e0 2024-03-21T11:40:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.118+0000 7fc797e00640 1 -- 172.21.15.47:0/1446325913 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7900d50d0 con 0x55fe616ab9e0 2024-03-21T11:40:56.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.119+0000 7fc7b19b7740 1 -- 172.21.15.47:0/1446325913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ab9e0 msgr2=0x55fe616bbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.119+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/1446325913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ab9e0 0x55fe616bbed0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc7900cd0f0 tx=0x7fc79009b6b0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.119+0000 7fc7b19b7740 1 -- 172.21.15.47:0/1446325913 shutdown_connections 2024-03-21T11:40:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.119+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/1446325913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ac3a0 0x55fe616c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.119+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/1446325913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ab9e0 0x55fe616bbed0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.119+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/1446325913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 0x55fe616a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.119+0000 7fc7b19b7740 1 -- 172.21.15.47:0/1446325913 >> 172.21.15.47:0/1446325913 conn(0x55fe616a5c00 msgr2=0x55fe616a8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.119+0000 7fc7b19b7740 1 -- 172.21.15.47:0/1446325913 shutdown_connections 2024-03-21T11:40:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.119+0000 7fc7b19b7740 1 -- 172.21.15.47:0/1446325913 wait complete. 2024-03-21T11:40:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.120+0000 7fc7b19b7740 1 -- start start 2024-03-21T11:40:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.120+0000 7fc7b19b7740 1 Processor -- start 2024-03-21T11:40:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc7b19b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 0x55fe616ba890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc7b19b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ab9e0 0x55fe616badd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc7b19b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ac3a0 0x55fe616bb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc7b19b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc784010d00 con 0x55fe616aa810 2024-03-21T11:40:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc7b19b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840106a0 con 0x55fe616ac3a0 2024-03-21T11:40:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc7b19b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78400dba0 con 0x55fe616ab9e0 2024-03-21T11:40:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc7b19b7740 10 client.0 _ll_register_callbacks cb 0x7fc790005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc79f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ab9e0 0x55fe616badd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc7acc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ac3a0 0x55fe616bb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc79f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ab9e0 0x55fe616badd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42258/0 (socket says 172.21.15.47:42258) 2024-03-21T11:40:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc79f400640 1 -- 172.21.15.47:0/2056771073 learned_addr learned my addr 172.21.15.47:0/2056771073 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc7a7e00640 1 --2- 172.21.15.47:0/2056771073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 0x55fe616ba890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc79f400640 1 -- 172.21.15.47:0/2056771073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ac3a0 msgr2=0x55fe616bb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc79f400640 1 --2- 172.21.15.47:0/2056771073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ac3a0 0x55fe616bb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc79f400640 1 -- 172.21.15.47:0/2056771073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 msgr2=0x55fe616ba890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc79f400640 1 --2- 172.21.15.47:0/2056771073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 0x55fe616ba890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.121+0000 7fc79f400640 1 -- 172.21.15.47:0/2056771073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe6171bfc0 con 0x55fe616ab9e0 2024-03-21T11:40:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.122+0000 7fc79f400640 1 --2- 172.21.15.47:0/2056771073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ab9e0 0x55fe616badd0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fc790098130 tx=0x7fc790005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1039bdb2bcc37f64 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.122+0000 7fc795600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:40:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.122+0000 7fc795600640 1 -- 172.21.15.47:0/2056771073 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79009bcd0 con 0x55fe616ab9e0 2024-03-21T11:40:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.122+0000 7fc7b19b7740 1 -- 172.21.15.47:0/2056771073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc79009c3d0 con 0x55fe616ab9e0 2024-03-21T11:40:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.122+0000 7fc795600640 1 -- 172.21.15.47:0/2056771073 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc7900d50d0 con 0x55fe616ab9e0 2024-03-21T11:40:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.122+0000 7fc795600640 1 -- 172.21.15.47:0/2056771073 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc79009fe20 con 0x55fe616ab9e0 2024-03-21T11:40:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.122+0000 7fc7b19b7740 10 client.31252 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.122+0000 7fc7b19b7740 1 -- 172.21.15.47:0/2056771073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55fe6171d2c0 con 0x55fe616ab9e0 2024-03-21T11:40:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.122+0000 7fc795600640 1 -- 172.21.15.47:0/2056771073 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc79009c3d0 con 0x55fe616ab9e0 2024-03-21T11:40:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.122+0000 7fc795600640 1 -- 172.21.15.47:0/2056771073 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc7900a0fb0 con 0x55fe616ab9e0 2024-03-21T11:40:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.123+0000 7fc7b19b7740 10 client.31252 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.123+0000 7fc7b19b7740 10 client.31252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.123+0000 7fc7b19b7740 20 client.31252 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:56.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.123+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/2056771073 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe6171d710 0x55fe6171fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:56.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.123+0000 7fc7b19b7740 4 client.31252 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:40:56.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.123+0000 7fc7b19b7740 1 -- 172.21.15.47:0/2056771073 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fe61616b50 con 0x55fe6171d710 2024-03-21T11:40:56.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.123+0000 7fc7a7e00640 1 --2- 172.21.15.47:0/2056771073 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe6171d710 0x55fe6171fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:56.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.123+0000 7fc7a7e00640 1 --2- 172.21.15.47:0/2056771073 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe6171d710 0x55fe6171fb00 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f9f8a80e7ffc708 server_cookie=5dd0f9ed0e4e6fb in_seq=0 out_seq=0 2024-03-21T11:40:56.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.123+0000 7fc795600640 10 client.31252 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:56.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc795600640 1 -- 172.21.15.47:0/2056771073 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fe61616b50 con 0x55fe6171d710 2024-03-21T11:40:56.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc795600640 10 client.31252 handle_command_reply: tid=0 2024-03-21T11:40:56.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc7b19b7740 1 client.31252 shutdown 2024-03-21T11:40:56.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc7b19b7740 1 -- 172.21.15.47:0/2056771073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ab9e0 msgr2=0x55fe616badd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:56.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/2056771073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ab9e0 0x55fe616badd0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fc790098130 tx=0x7fc790005b00 comp rx=0 tx=0).stop 2024-03-21T11:40:56.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc7b19b7740 1 -- 172.21.15.47:0/2056771073 shutdown_connections 2024-03-21T11:40:56.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/2056771073 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fe6171d710 0x55fe6171fb00 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/2056771073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fe616ac3a0 0x55fe616bb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/2056771073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe616ab9e0 0x55fe616badd0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc7b19b7740 1 --2- 172.21.15.47:0/2056771073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fe616aa810 0x55fe616ba890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc7b19b7740 1 -- 172.21.15.47:0/2056771073 >> 172.21.15.47:0/2056771073 conn(0x55fe616a5c00 msgr2=0x55fe616a96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:56.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc795600640 0 client.31252 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:56.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc7b19b7740 1 -- 172.21.15.47:0/2056771073 shutdown_connections 2024-03-21T11:40:56.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc7b19b7740 1 -- 172.21.15.47:0/2056771073 wait complete. 2024-03-21T11:40:56.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.127+0000 7fc7b19b7740 20 client.31252 trim_cache size 0 max 16384 2024-03-21T11:40:56.131 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:40:56.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.128+0000 7fc7aec00640 1 -- 172.21.15.47:0/2493817027 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc78c0a2fa0 msgr2=0x7fc78c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:56.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.128+0000 7fc7aec00640 1 --2- 172.21.15.47:0/2493817027 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc78c0a2fa0 0x7fc78c0a5460 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0006010 tx=0x7fc7a002e040 comp rx=0 tx=0).stop 2024-03-21T11:40:56.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.128+0000 7fc7aec00640 1 -- 172.21.15.47:0/2493817027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7a8108eb0 msgr2=0x7fc7a819e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.128+0000 7fc7aec00640 1 --2- 172.21.15.47:0/2493817027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7a8108eb0 0x7fc7a819e490 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc79000e9f0 tx=0x7fc79000edf0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.128+0000 7fc7aec00640 1 -- 172.21.15.47:0/2493817027 shutdown_connections 2024-03-21T11:40:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.128+0000 7fc7aec00640 1 --2- 172.21.15.47:0/2493817027 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc78c0a2fa0 0x7fc78c0a5460 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.128+0000 7fc7aec00640 1 --2- 172.21.15.47:0/2493817027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7a8109870 0x7fc7a81a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.128+0000 7fc7aec00640 1 --2- 172.21.15.47:0/2493817027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7a8108eb0 0x7fc7a819e490 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.128+0000 7fc7aec00640 1 --2- 172.21.15.47:0/2493817027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7a8107cc0 0x7fc7a819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.128+0000 7fc7aec00640 1 -- 172.21.15.47:0/2493817027 >> 172.21.15.47:0/2493817027 conn(0x7fc7a8076c60 msgr2=0x7fc7a810c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.128+0000 7fc7aec00640 1 -- 172.21.15.47:0/2493817027 shutdown_connections 2024-03-21T11:40:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:56.128+0000 7fc7aec00640 1 -- 172.21.15.47:0/2493817027 wait complete. 2024-03-21T11:40:56.946 INFO:tasks.workunit.client.0.smithi047.stdout: 10 103896 24.51 MB/sec execute 100 sec latency 211.892 ms 2024-03-21T11:40:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:56 smithi047 ceph-mon[31351]: pgmap v928: 129 pgs: 129 active+clean; 442 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 611 op/s 2024-03-21T11:40:57.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:56 smithi100 ceph-mon[36245]: pgmap v928: 129 pgs: 129 active+clean; 442 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 611 op/s 2024-03-21T11:40:57.146 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:40:57.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:40:57.146 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:40:57.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:56 smithi102 ceph-mon[36748]: pgmap v928: 129 pgs: 129 active+clean; 442 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 611 op/s 2024-03-21T11:40:57.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.393+0000 7fc7df000640 1 -- 172.21.15.47:0/2979875937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7d8108eb0 msgr2=0x7fc7d8109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.393+0000 7fc7df000640 1 --2- 172.21.15.47:0/2979875937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7d8108eb0 0x7fc7d8109330 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009900 tx=0x7fc7c002bc40 comp rx=0 tx=0).stop 2024-03-21T11:40:57.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.393+0000 7fc7df000640 1 -- 172.21.15.47:0/2979875937 shutdown_connections 2024-03-21T11:40:57.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.393+0000 7fc7df000640 1 --2- 172.21.15.47:0/2979875937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7d8109870 0x7fc7d8110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.393+0000 7fc7df000640 1 --2- 172.21.15.47:0/2979875937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7d8108eb0 0x7fc7d8109330 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.393+0000 7fc7df000640 1 --2- 172.21.15.47:0/2979875937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d8107cc0 0x7fc7d81080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.393+0000 7fc7df000640 1 -- 172.21.15.47:0/2979875937 >> 172.21.15.47:0/2979875937 conn(0x7fc7d8076c60 msgr2=0x7fc7d80790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:57.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.393+0000 7fc7df000640 1 -- 172.21.15.47:0/2979875937 shutdown_connections 2024-03-21T11:40:57.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.393+0000 7fc7df000640 1 -- 172.21.15.47:0/2979875937 wait complete. 2024-03-21T11:40:57.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.394+0000 7fc7df000640 1 Processor -- start 2024-03-21T11:40:57.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.394+0000 7fc7df000640 1 -- start start 2024-03-21T11:40:57.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7df000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7d8107cc0 0x7fc7d819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7df000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d8108eb0 0x7fc7d819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7df000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7d8109870 0x7fc7d81a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7df000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8113920 con 0x7fc7d8109870 2024-03-21T11:40:57.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7df000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d81137a0 con 0x7fc7d8108eb0 2024-03-21T11:40:57.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7df000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8113aa0 con 0x7fc7d8107cc0 2024-03-21T11:40:57.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7d7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7d8107cc0 0x7fc7d819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d8108eb0 0x7fc7d819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7d7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7d8107cc0 0x7fc7d819df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42270/0 (socket says 172.21.15.47:42270) 2024-03-21T11:40:57.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7d7e00640 1 -- 172.21.15.47:0/2270144813 learned_addr learned my addr 172.21.15.47:0/2270144813 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:57.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d8108eb0 0x7fc7d819e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47252/0 (socket says 172.21.15.47:47252) 2024-03-21T11:40:57.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7dd000640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7d8109870 0x7fc7d81a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7d7e00640 1 -- 172.21.15.47:0/2270144813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d8108eb0 msgr2=0x7fc7d819e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7d7e00640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d8108eb0 0x7fc7d819e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7d7e00640 1 -- 172.21.15.47:0/2270144813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7d8109870 msgr2=0x7fc7d81a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7d7e00640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7d8109870 0x7fc7d81a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7d7e00640 1 -- 172.21.15.47:0/2270144813 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d81a5bb0 con 0x7fc7d8107cc0 2024-03-21T11:40:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.395+0000 7fc7cfe00640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d8108eb0 0x7fc7d819e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.396+0000 7fc7dd000640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7d8109870 0x7fc7d81a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:40:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.396+0000 7fc7d7e00640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7d8107cc0 0x7fc7d819df90 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800e9c0 tx=0x7fc7c800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.396+0000 7fc7d5600640 1 -- 172.21.15.47:0/2270144813 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c800cd80 con 0x7fc7d8107cc0 2024-03-21T11:40:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.396+0000 7fc7d5600640 1 -- 172.21.15.47:0/2270144813 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7c80035f0 con 0x7fc7d8107cc0 2024-03-21T11:40:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.396+0000 7fc7d5600640 1 -- 172.21.15.47:0/2270144813 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8010690 con 0x7fc7d8107cc0 2024-03-21T11:40:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.396+0000 7fc7df000640 1 -- 172.21.15.47:0/2270144813 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d81a5ea0 con 0x7fc7d8107cc0 2024-03-21T11:40:57.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.396+0000 7fc7df000640 1 -- 172.21.15.47:0/2270144813 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d81a6250 con 0x7fc7d8107cc0 2024-03-21T11:40:57.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.399+0000 7fc7d5600640 1 -- 172.21.15.47:0/2270144813 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc7c80037a0 con 0x7fc7d8107cc0 2024-03-21T11:40:57.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.399+0000 7fc7e19fd740 1 Processor -- start 2024-03-21T11:40:57.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.400+0000 7fc7e19fd740 1 -- start start 2024-03-21T11:40:57.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.400+0000 7fc7e19fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc516440a0 0x55fc516444c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.400+0000 7fc7e19fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51644a90 0x55fc51649590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.400+0000 7fc7e19fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc51649c00 0x55fc5164c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.400+0000 7fc7e19fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc515a9eb0 con 0x55fc516440a0 2024-03-21T11:40:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.400+0000 7fc7e19fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc51552a60 con 0x55fc51644a90 2024-03-21T11:40:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.400+0000 7fc7e19fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc515a5f50 con 0x55fc51649c00 2024-03-21T11:40:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.400+0000 7fc7cfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc51649c00 0x55fc5164c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.400+0000 7fc7dd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc516440a0 0x55fc516444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.400+0000 7fc7d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51644a90 0x55fc51649590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.400+0000 7fc7cfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc51649c00 0x55fc5164c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42276/0 (socket says 172.21.15.47:42276) 2024-03-21T11:40:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.400+0000 7fc7cfe00640 1 -- 172.21.15.47:0/925249002 learned_addr learned my addr 172.21.15.47:0/925249002 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.401+0000 7fc7cfe00640 1 -- 172.21.15.47:0/925249002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51644a90 msgr2=0x55fc51649590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.401+0000 7fc7cfe00640 1 --2- 172.21.15.47:0/925249002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51644a90 0x55fc51649590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.401+0000 7fc7cfe00640 1 -- 172.21.15.47:0/925249002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc516440a0 msgr2=0x55fc516444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.401+0000 7fc7cfe00640 1 --2- 172.21.15.47:0/925249002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc516440a0 0x55fc516444c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.401+0000 7fc7cfe00640 1 -- 172.21.15.47:0/925249002 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc513867e0 con 0x55fc51649c00 2024-03-21T11:40:57.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.401+0000 7fc7cfe00640 1 --2- 172.21.15.47:0/925249002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc51649c00 0x55fc5164c090 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0036040 tx=0x7fc7c002cd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75abc064e71b0d24 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.401+0000 7fc7cd600640 1 -- 172.21.15.47:0/925249002 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0005d50 con 0x55fc51649c00 2024-03-21T11:40:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.401+0000 7fc7d5600640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7bc0a2fa0 0x7fc7bc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.401+0000 7fc7cd600640 1 -- 172.21.15.47:0/925249002 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7c0005520 con 0x55fc51649c00 2024-03-21T11:40:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.401+0000 7fc7cd600640 1 -- 172.21.15.47:0/925249002 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0004d20 con 0x55fc51649c00 2024-03-21T11:40:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.401+0000 7fc7d5600640 1 -- 172.21.15.47:0/2270144813 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc7c80cfbf0 con 0x7fc7d8107cc0 2024-03-21T11:40:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.402+0000 7fc7dd000640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7bc0a2fa0 0x7fc7bc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.402+0000 7fc7e19fd740 1 -- 172.21.15.47:0/925249002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc51649c00 msgr2=0x55fc5164c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.402+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/925249002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc51649c00 0x55fc5164c090 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0036040 tx=0x7fc7c002cd40 comp rx=0 tx=0).stop 2024-03-21T11:40:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.402+0000 7fc7e19fd740 1 -- 172.21.15.47:0/925249002 shutdown_connections 2024-03-21T11:40:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.402+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/925249002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc51649c00 0x55fc5164c090 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.402+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/925249002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51644a90 0x55fc51649590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.402+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/925249002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc516440a0 0x55fc516444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.402+0000 7fc7e19fd740 1 -- 172.21.15.47:0/925249002 >> 172.21.15.47:0/925249002 conn(0x55fc5163f610 msgr2=0x55fc5163fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.402+0000 7fc7e19fd740 1 -- 172.21.15.47:0/925249002 shutdown_connections 2024-03-21T11:40:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.402+0000 7fc7dd000640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7bc0a2fa0 0x7fc7bc0a5460 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x55fc516452f0 tx=0x7fc7d0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.402+0000 7fc7e19fd740 1 -- 172.21.15.47:0/925249002 wait complete. 2024-03-21T11:40:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.403+0000 7fc7e19fd740 1 -- start start 2024-03-21T11:40:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.403+0000 7fc7e19fd740 1 Processor -- start 2024-03-21T11:40:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.403+0000 7fc7e19fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc5164e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.403+0000 7fc7e19fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 0x55fc5164e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.403+0000 7fc7e19fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc516557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.403+0000 7fc7e19fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc5164cb50 con 0x55fc51644a90 2024-03-21T11:40:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.403+0000 7fc7e19fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc51565eb0 con 0x55fc51649c00 2024-03-21T11:40:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.403+0000 7fc7e19fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc5164ccd0 con 0x55fc516440a0 2024-03-21T11:40:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.404+0000 7fc7e19fd740 10 client.0 _ll_register_callbacks cb 0x55fc5155a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.404+0000 7fc7dd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc516557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.404+0000 7fc7d7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc5164e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.404+0000 7fc7dd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc516557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47268/0 (socket says 172.21.15.47:47268) 2024-03-21T11:40:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.404+0000 7fc7dd000640 1 -- 172.21.15.47:0/3258383842 learned_addr learned my addr 172.21.15.47:0/3258383842 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.404+0000 7fc7d7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc5164e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42278/0 (socket says 172.21.15.47:42278) 2024-03-21T11:40:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.404+0000 7fc7cfe00640 1 --2- 172.21.15.47:0/3258383842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 0x55fc5164e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.404+0000 7fc7dd000640 1 -- 172.21.15.47:0/3258383842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 msgr2=0x55fc5164e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.404+0000 7fc7dd000640 1 --2- 172.21.15.47:0/3258383842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc5164e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.404+0000 7fc7dd000640 1 -- 172.21.15.47:0/3258383842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 msgr2=0x55fc5164e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.404+0000 7fc7dd000640 1 --2- 172.21.15.47:0/3258383842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 0x55fc5164e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.404+0000 7fc7dd000640 1 -- 172.21.15.47:0/3258383842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc513e6f70 con 0x55fc51649c00 2024-03-21T11:40:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.404+0000 7fc7dd000640 1 --2- 172.21.15.47:0/3258383842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc516557b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0010b40 tx=0x7fc7d0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77b848ede898a1d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.404+0000 7fc7c6000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.405+0000 7fc7c6000640 1 -- 172.21.15.47:0/3258383842 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d002c060 con 0x55fc51649c00 2024-03-21T11:40:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.405+0000 7fc7e19fd740 1 -- 172.21.15.47:0/3258383842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7c0005520 con 0x55fc51649c00 2024-03-21T11:40:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.405+0000 7fc7c6000640 1 -- 172.21.15.47:0/3258383842 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc7d0017030 con 0x55fc51649c00 2024-03-21T11:40:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.405+0000 7fc7c6000640 1 -- 172.21.15.47:0/3258383842 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7d0033050 con 0x55fc51649c00 2024-03-21T11:40:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.405+0000 7fc7e19fd740 10 client.31283 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.405+0000 7fc7e19fd740 1 -- 172.21.15.47:0/3258383842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55fc5127e170 con 0x55fc51649c00 2024-03-21T11:40:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.405+0000 7fc7c6000640 1 -- 172.21.15.47:0/3258383842 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7c0005520 con 0x55fc51649c00 2024-03-21T11:40:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.405+0000 7fc7c6000640 1 -- 172.21.15.47:0/3258383842 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc7d001c3f0 con 0x55fc51649c00 2024-03-21T11:40:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.405+0000 7fc7e19fd740 10 client.31283 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.405+0000 7fc7e19fd740 10 client.31283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.406+0000 7fc7e19fd740 20 client.31283 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.406+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/3258383842 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc5165a130 0x55fc5165c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.406+0000 7fc7e19fd740 4 client.31283 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.406+0000 7fc7e19fd740 1 -- 172.21.15.47:0/3258383842 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc515b73f0 con 0x55fc5165a130 2024-03-21T11:40:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.406+0000 7fc7d7e00640 1 --2- 172.21.15.47:0/3258383842 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc5165a130 0x55fc5165c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.406+0000 7fc7d7e00640 1 --2- 172.21.15.47:0/3258383842 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc5165a130 0x55fc5165c570 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ebd721034238695 server_cookie=8df847711655d6d5 in_seq=0 out_seq=0 2024-03-21T11:40:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.406+0000 7fc7c6000640 10 client.31283 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.407+0000 7fc7c6000640 1 -- 172.21.15.47:0/3258383842 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fc515b73f0 con 0x55fc5165a130 2024-03-21T11:40:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.407+0000 7fc7c6000640 10 client.31283 handle_command_reply: tid=0 2024-03-21T11:40:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.407+0000 7fc7e19fd740 1 client.31283 shutdown 2024-03-21T11:40:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.407+0000 7fc7e19fd740 1 -- 172.21.15.47:0/3258383842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 msgr2=0x55fc516557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.408+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/3258383842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc516557b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0010b40 tx=0x7fc7d0004ab0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.408+0000 7fc7e19fd740 1 -- 172.21.15.47:0/3258383842 shutdown_connections 2024-03-21T11:40:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.408+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/3258383842 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc5165a130 0x55fc5165c570 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.408+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/3258383842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc516557b0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.408+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/3258383842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 0x55fc5164e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.408+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/3258383842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc5164e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.408+0000 7fc7c6000640 0 client.31283 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.408+0000 7fc7e19fd740 1 -- 172.21.15.47:0/3258383842 >> 172.21.15.47:0/3258383842 conn(0x55fc5163f610 msgr2=0x55fc5163e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.408+0000 7fc7e19fd740 1 -- 172.21.15.47:0/3258383842 shutdown_connections 2024-03-21T11:40:57.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.408+0000 7fc7e19fd740 1 -- 172.21.15.47:0/3258383842 wait complete. 2024-03-21T11:40:57.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.408+0000 7fc7e19fd740 20 client.31283 trim_cache size 0 max 16384 2024-03-21T11:40:57.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.426+0000 7fc7e19fd740 1 Processor -- start 2024-03-21T11:40:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.426+0000 7fc7e19fd740 1 -- start start 2024-03-21T11:40:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7e19fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc51645530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7e19fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 0x55fc51655770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7e19fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc51645a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7e19fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d81b04e0 con 0x55fc51644a90 2024-03-21T11:40:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7e19fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d81b1e60 con 0x55fc51649c00 2024-03-21T11:40:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7e19fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d81b0b40 con 0x55fc516440a0 2024-03-21T11:40:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7d7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc51645530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7d7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc51645530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42282/0 (socket says 172.21.15.47:42282) 2024-03-21T11:40:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7d7e00640 1 -- 172.21.15.47:0/3655524778 learned_addr learned my addr 172.21.15.47:0/3655524778 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7dd000640 1 --2- 172.21.15.47:0/3655524778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc51645a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7cfe00640 1 --2- 172.21.15.47:0/3655524778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 0x55fc51655770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7d7e00640 1 -- 172.21.15.47:0/3655524778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 msgr2=0x55fc51645a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7d7e00640 1 --2- 172.21.15.47:0/3655524778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc51645a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7d7e00640 1 -- 172.21.15.47:0/3655524778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 msgr2=0x55fc51655770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7d7e00640 1 --2- 172.21.15.47:0/3655524778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 0x55fc51655770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.427+0000 7fc7d7e00640 1 -- 172.21.15.47:0/3655524778 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc51640380 con 0x55fc516440a0 2024-03-21T11:40:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.428+0000 7fc7d7e00640 1 --2- 172.21.15.47:0/3655524778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc51645530 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8098900 tx=0x7fc7c80d3630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f2ca56e3b917a33 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.428+0000 7fc7ccc00640 1 -- 172.21.15.47:0/3655524778 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c809b890 con 0x55fc516440a0 2024-03-21T11:40:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.428+0000 7fc7ccc00640 1 -- 172.21.15.47:0/3655524778 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7c809c630 con 0x55fc516440a0 2024-03-21T11:40:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.428+0000 7fc7ccc00640 1 -- 172.21.15.47:0/3655524778 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c80bc070 con 0x55fc516440a0 2024-03-21T11:40:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.428+0000 7fc7e19fd740 1 -- 172.21.15.47:0/3655524778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 msgr2=0x55fc51645530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.428+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/3655524778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc51645530 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8098900 tx=0x7fc7c80d3630 comp rx=0 tx=0).stop 2024-03-21T11:40:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.428+0000 7fc7e19fd740 1 -- 172.21.15.47:0/3655524778 shutdown_connections 2024-03-21T11:40:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.428+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/3655524778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc51645a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.429+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/3655524778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 0x55fc51655770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.429+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/3655524778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc51645530 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.429+0000 7fc7e19fd740 1 -- 172.21.15.47:0/3655524778 >> 172.21.15.47:0/3655524778 conn(0x55fc5163f610 msgr2=0x55fc5164d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.429+0000 7fc7e19fd740 1 -- 172.21.15.47:0/3655524778 shutdown_connections 2024-03-21T11:40:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.429+0000 7fc7e19fd740 1 -- 172.21.15.47:0/3655524778 wait complete. 2024-03-21T11:40:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.429+0000 7fc7e19fd740 1 -- start start 2024-03-21T11:40:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.429+0000 7fc7e19fd740 1 Processor -- start 2024-03-21T11:40:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7e19fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc51655440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7e19fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 0x55fc51653aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7e19fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc51653fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7e19fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d81b1800 con 0x55fc51644a90 2024-03-21T11:40:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7e19fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d81b11a0 con 0x55fc51649c00 2024-03-21T11:40:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7e19fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d81ae6a0 con 0x55fc516440a0 2024-03-21T11:40:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7e19fd740 10 client.0 _ll_register_callbacks cb 0x7fc7d0009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7d7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc51655440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7dd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc51653fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7dd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc51653fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47282/0 (socket says 172.21.15.47:47282) 2024-03-21T11:40:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7dd000640 1 -- 172.21.15.47:0/2494101829 learned_addr learned my addr 172.21.15.47:0/2494101829 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7d7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc51655440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42290/0 (socket says 172.21.15.47:42290) 2024-03-21T11:40:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7cfe00640 1 --2- 172.21.15.47:0/2494101829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 0x55fc51653aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7dd000640 1 -- 172.21.15.47:0/2494101829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 msgr2=0x55fc51655440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7dd000640 1 --2- 172.21.15.47:0/2494101829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc51655440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7dd000640 1 -- 172.21.15.47:0/2494101829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 msgr2=0x55fc51653aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7dd000640 1 --2- 172.21.15.47:0/2494101829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 0x55fc51653aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7dd000640 1 -- 172.21.15.47:0/2494101829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc51654570 con 0x55fc51649c00 2024-03-21T11:40:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7dd000640 1 --2- 172.21.15.47:0/2494101829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc51653fe0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0010b40 tx=0x7fc7d001c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=833c037e2524ce94 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.430+0000 7fc7c6000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.431+0000 7fc7c6000640 1 -- 172.21.15.47:0/2494101829 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d003b020 con 0x55fc51649c00 2024-03-21T11:40:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.431+0000 7fc7c6000640 1 -- 172.21.15.47:0/2494101829 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc7d0028680 con 0x55fc51649c00 2024-03-21T11:40:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.431+0000 7fc7e19fd740 1 -- 172.21.15.47:0/2494101829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7c809c630 con 0x55fc51649c00 2024-03-21T11:40:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.431+0000 7fc7c6000640 1 -- 172.21.15.47:0/2494101829 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7d0046050 con 0x55fc51649c00 2024-03-21T11:40:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.431+0000 7fc7e19fd740 10 client.31289 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.431+0000 7fc7e19fd740 1 -- 172.21.15.47:0/2494101829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55fc516b6840 con 0x55fc51649c00 2024-03-21T11:40:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.431+0000 7fc7c6000640 1 -- 172.21.15.47:0/2494101829 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7c809c630 con 0x55fc51649c00 2024-03-21T11:40:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.431+0000 7fc7c6000640 1 -- 172.21.15.47:0/2494101829 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc7d00376d0 con 0x55fc51649c00 2024-03-21T11:40:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.431+0000 7fc7e19fd740 10 client.31289 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.431+0000 7fc7e19fd740 10 client.31289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.431+0000 7fc7e19fd740 20 client.31289 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.431+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/2494101829 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc516b71f0 0x55fc516b95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.431+0000 7fc7e19fd740 4 client.31289 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:40:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.431+0000 7fc7e19fd740 1 -- 172.21.15.47:0/2494101829 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fc515b1740 con 0x55fc516b71f0 2024-03-21T11:40:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.432+0000 7fc7d7e00640 1 --2- 172.21.15.47:0/2494101829 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc516b71f0 0x55fc516b95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.432+0000 7fc7d7e00640 1 --2- 172.21.15.47:0/2494101829 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc516b71f0 0x55fc516b95e0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0f184b33cbbffac server_cookie=7e181b3048e3771b in_seq=0 out_seq=0 2024-03-21T11:40:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.432+0000 7fc7c6000640 10 client.31289 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.433+0000 7fc7c6000640 1 -- 172.21.15.47:0/2494101829 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fc515b1740 con 0x55fc516b71f0 2024-03-21T11:40:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.433+0000 7fc7c6000640 10 client.31289 handle_command_reply: tid=0 2024-03-21T11:40:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.433+0000 7fc7e19fd740 1 client.31289 shutdown 2024-03-21T11:40:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.433+0000 7fc7e19fd740 1 -- 172.21.15.47:0/2494101829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 msgr2=0x55fc51653fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.433+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/2494101829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc51653fe0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0010b40 tx=0x7fc7d001c6a0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.434+0000 7fc7e19fd740 1 -- 172.21.15.47:0/2494101829 shutdown_connections 2024-03-21T11:40:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.434+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/2494101829 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc516b71f0 0x55fc516b95e0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.434+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/2494101829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc51649c00 0x55fc51653fe0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.434+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/2494101829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc51644a90 0x55fc51653aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.434+0000 7fc7e19fd740 1 --2- 172.21.15.47:0/2494101829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc516440a0 0x55fc51655440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.434+0000 7fc7e19fd740 1 -- 172.21.15.47:0/2494101829 >> 172.21.15.47:0/2494101829 conn(0x55fc5163f610 msgr2=0x55fc5163dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.434+0000 7fc7c6000640 0 client.31289 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.434+0000 7fc7e19fd740 1 -- 172.21.15.47:0/2494101829 shutdown_connections 2024-03-21T11:40:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.434+0000 7fc7e19fd740 1 -- 172.21.15.47:0/2494101829 wait complete. 2024-03-21T11:40:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.434+0000 7fc7e19fd740 20 client.31289 trim_cache size 0 max 16384 2024-03-21T11:40:57.439 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:40:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:40:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "f383a0ac-4d76-4c44-8141-64eb5cb7eab0", 2024-03-21T11:40:57.439 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:40:57.439 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:40:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.435+0000 7fc7df000640 1 -- 172.21.15.47:0/2270144813 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7bc0a2fa0 msgr2=0x7fc7bc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.435+0000 7fc7df000640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7bc0a2fa0 0x7fc7bc0a5460 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x55fc516452f0 tx=0x7fc7d0009500 comp rx=0 tx=0).stop 2024-03-21T11:40:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.435+0000 7fc7df000640 1 -- 172.21.15.47:0/2270144813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7d8107cc0 msgr2=0x7fc7d819df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.435+0000 7fc7df000640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7d8107cc0 0x7fc7d819df90 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800e9c0 tx=0x7fc7c800ee80 comp rx=0 tx=0).stop 2024-03-21T11:40:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.435+0000 7fc7df000640 1 -- 172.21.15.47:0/2270144813 shutdown_connections 2024-03-21T11:40:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.435+0000 7fc7df000640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7bc0a2fa0 0x7fc7bc0a5460 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.435+0000 7fc7df000640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7d8109870 0x7fc7d81a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.435+0000 7fc7df000640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7d8108eb0 0x7fc7d819e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.435+0000 7fc7df000640 1 --2- 172.21.15.47:0/2270144813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7d8107cc0 0x7fc7d819df90 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.435+0000 7fc7df000640 1 -- 172.21.15.47:0/2270144813 >> 172.21.15.47:0/2270144813 conn(0x7fc7d8076c60 msgr2=0x7fc7d810c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.435+0000 7fc7df000640 1 -- 172.21.15.47:0/2270144813 shutdown_connections 2024-03-21T11:40:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.435+0000 7fc7df000640 1 -- 172.21.15.47:0/2270144813 wait complete. 2024-03-21T11:40:57.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:40:57.463 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:40:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.735+0000 7f4423000640 1 -- 172.21.15.47:0/1629429903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f441c0772a0 msgr2=0x7f441c077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.735+0000 7f4423000640 1 --2- 172.21.15.47:0/1629429903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f441c0772a0 0x7f441c077700 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4408009860 tx=0x7f440802bb90 comp rx=0 tx=0).stop 2024-03-21T11:40:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.735+0000 7f4423000640 1 -- 172.21.15.47:0/1629429903 shutdown_connections 2024-03-21T11:40:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.735+0000 7f4423000640 1 --2- 172.21.15.47:0/1629429903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c077c40 0x7f441c0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.735+0000 7f4423000640 1 --2- 172.21.15.47:0/1629429903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f441c0772a0 0x7f441c077700 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.735+0000 7f4423000640 1 --2- 172.21.15.47:0/1629429903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f441c076060 0x7f441c076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.735+0000 7f4423000640 1 -- 172.21.15.47:0/1629429903 >> 172.21.15.47:0/1629429903 conn(0x7f441c0fed10 msgr2=0x7f441c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.735+0000 7f4423000640 1 -- 172.21.15.47:0/1629429903 shutdown_connections 2024-03-21T11:40:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.735+0000 7f4423000640 1 -- 172.21.15.47:0/1629429903 wait complete. 2024-03-21T11:40:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.737+0000 7f4423000640 1 Processor -- start 2024-03-21T11:40:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.737+0000 7f4423000640 1 -- start start 2024-03-21T11:40:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.737+0000 7f4423000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f441c076060 0x7f441c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.737+0000 7f4423000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f441c0772a0 0x7f441c19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.737+0000 7f4423000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c077c40 0x7f441c1a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.737+0000 7f4423000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c113830 con 0x7f441c0772a0 2024-03-21T11:40:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.737+0000 7f4423000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1136b0 con 0x7f441c076060 2024-03-21T11:40:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.737+0000 7f4423000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1139b0 con 0x7f441c077c40 2024-03-21T11:40:57.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.738+0000 7f4421200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f441c0772a0 0x7f441c19e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.738+0000 7f4421200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f441c0772a0 0x7f441c19e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39470/0 (socket says 172.21.15.47:39470) 2024-03-21T11:40:57.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.738+0000 7f4421200640 1 -- 172.21.15.47:0/1157232113 learned_addr learned my addr 172.21.15.47:0/1157232113 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:57.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.739+0000 7f4421200640 1 -- 172.21.15.47:0/1157232113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c077c40 msgr2=0x7f441c1a5420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:40:57.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.739+0000 7f4421200640 1 --2- 172.21.15.47:0/1157232113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c077c40 0x7f441c1a5420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.739+0000 7f4421200640 1 -- 172.21.15.47:0/1157232113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f441c076060 msgr2=0x7f441c19de70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T11:40:57.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.739+0000 7f4421200640 1 --2- 172.21.15.47:0/1157232113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f441c076060 0x7f441c19de70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.739+0000 7f4421200640 1 -- 172.21.15.47:0/1157232113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c1a5a90 con 0x7f441c0772a0 2024-03-21T11:40:57.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.741+0000 7f4421200640 1 --2- 172.21.15.47:0/1157232113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f441c0772a0 0x7f441c19e3b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f44080349e0 tx=0x7f4408032450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:57.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.741+0000 7f441a000640 1 -- 172.21.15.47:0/1157232113 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408041030 con 0x7f441c0772a0 2024-03-21T11:40:57.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.742+0000 7f441a000640 1 -- 172.21.15.47:0/1157232113 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f440802c510 con 0x7f441c0772a0 2024-03-21T11:40:57.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.742+0000 7f441a000640 1 -- 172.21.15.47:0/1157232113 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408039040 con 0x7f441c0772a0 2024-03-21T11:40:57.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.743+0000 7f4423000640 1 -- 172.21.15.47:0/1157232113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441c1a5d20 con 0x7f441c0772a0 2024-03-21T11:40:57.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.743+0000 7f4423000640 1 -- 172.21.15.47:0/1157232113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c1a61c0 con 0x7f441c0772a0 2024-03-21T11:40:57.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.749+0000 7f44273b7740 1 Processor -- start 2024-03-21T11:40:57.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.749+0000 7f44273b7740 1 -- start start 2024-03-21T11:40:57.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.750+0000 7f44273b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc64d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.750+0000 7f44273b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc64da40 0x55bfbc652540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.750+0000 7f44273b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc652bb0 0x55bfbc655040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.750+0000 7f44273b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc5b2250 con 0x55bfbc64d050 2024-03-21T11:40:57.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.750+0000 7f44273b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc55aa30 con 0x55bfbc652bb0 2024-03-21T11:40:57.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.750+0000 7f44273b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc5ae2f0 con 0x55bfbc64da40 2024-03-21T11:40:57.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.750+0000 7f4421c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc64da40 0x55bfbc652540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.750+0000 7f4421c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc64da40 0x55bfbc652540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42314/0 (socket says 172.21.15.47:42314) 2024-03-21T11:40:57.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.750+0000 7f4421c00640 1 -- 172.21.15.47:0/2287191293 learned_addr learned my addr 172.21.15.47:0/2287191293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:57.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.751+0000 7f4421c00640 1 -- 172.21.15.47:0/2287191293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc652bb0 msgr2=0x55bfbc655040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:40:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.751+0000 7f441a000640 1 -- 172.21.15.47:0/1157232113 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4408044070 con 0x7f441c0772a0 2024-03-21T11:40:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.751+0000 7f4422600640 1 --2- 172.21.15.47:0/2287191293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc64d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.752+0000 7f4421200640 1 --2- 172.21.15.47:0/2287191293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc652bb0 0x55bfbc655040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.752+0000 7f4421c00640 1 --2- 172.21.15.47:0/2287191293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc652bb0 0x55bfbc655040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.752+0000 7f4421c00640 1 -- 172.21.15.47:0/2287191293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 msgr2=0x55bfbc64d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.752+0000 7f4421c00640 1 --2- 172.21.15.47:0/2287191293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc64d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.752+0000 7f4421c00640 1 -- 172.21.15.47:0/2287191293 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfbc545690 con 0x55bfbc64da40 2024-03-21T11:40:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.752+0000 7f4422600640 1 --2- 172.21.15.47:0/2287191293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc64d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.752+0000 7f4421200640 1 --2- 172.21.15.47:0/2287191293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc652bb0 0x55bfbc655040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.754+0000 7f441a000640 1 --2- 172.21.15.47:0/1157232113 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44040a3020 0x7f44040a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.754+0000 7f4422600640 1 --2- 172.21.15.47:0/1157232113 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44040a3020 0x7f44040a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.754+0000 7f441a000640 1 -- 172.21.15.47:0/1157232113 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f44080f0de0 con 0x7f441c0772a0 2024-03-21T11:40:57.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.755+0000 7f4421c00640 1 --2- 172.21.15.47:0/2287191293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc64da40 0x55bfbc652540 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f4410009ae0 tx=0x7f441000e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=568a0d6a28c6bfa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:57.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.756+0000 7f4422600640 1 --2- 172.21.15.47:0/1157232113 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44040a3020 0x7f44040a54e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55bfbc64e2a0 tx=0x7f4414009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.757+0000 7f440ea00640 1 -- 172.21.15.47:0/2287191293 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441000ed20 con 0x55bfbc64da40 2024-03-21T11:40:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.757+0000 7f440ea00640 1 -- 172.21.15.47:0/2287191293 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44100035f0 con 0x55bfbc64da40 2024-03-21T11:40:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.757+0000 7f440ea00640 1 -- 172.21.15.47:0/2287191293 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410010670 con 0x55bfbc64da40 2024-03-21T11:40:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.758+0000 7f44273b7740 1 -- 172.21.15.47:0/2287191293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc64da40 msgr2=0x55bfbc652540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.758+0000 7f44273b7740 1 --2- 172.21.15.47:0/2287191293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc64da40 0x55bfbc652540 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f4410009ae0 tx=0x7f441000e350 comp rx=0 tx=0).stop 2024-03-21T11:40:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.759+0000 7f44273b7740 1 -- 172.21.15.47:0/2287191293 shutdown_connections 2024-03-21T11:40:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.759+0000 7f44273b7740 1 --2- 172.21.15.47:0/2287191293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc652bb0 0x55bfbc655040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.759+0000 7f44273b7740 1 --2- 172.21.15.47:0/2287191293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc64da40 0x55bfbc652540 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.759+0000 7f44273b7740 1 --2- 172.21.15.47:0/2287191293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc64d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.759+0000 7f44273b7740 1 -- 172.21.15.47:0/2287191293 >> 172.21.15.47:0/2287191293 conn(0x55bfbc615200 msgr2=0x55bfbc6489c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.759+0000 7f44273b7740 1 -- 172.21.15.47:0/2287191293 shutdown_connections 2024-03-21T11:40:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.759+0000 7f44273b7740 1 -- 172.21.15.47:0/2287191293 wait complete. 2024-03-21T11:40:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.759+0000 7f44273b7740 1 -- start start 2024-03-21T11:40:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.760+0000 7f44273b7740 1 Processor -- start 2024-03-21T11:40:57.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.760+0000 7f44273b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc668010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.761+0000 7f4421c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc668010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.761+0000 7f4421c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc668010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39496/0 (socket says 172.21.15.47:39496) 2024-03-21T11:40:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.761+0000 7f4421c00640 1 -- 172.21.15.47:0/2819402127 learned_addr learned my addr 172.21.15.47:0/2819402127 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.761+0000 7f44273b7740 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64da40 0x55bfbc668550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.761+0000 7f44273b7740 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc66f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.761+0000 7f44273b7740 1 -- 172.21.15.47:0/2819402127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc666b70 con 0x55bfbc64d050 2024-03-21T11:40:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.761+0000 7f44273b7740 1 -- 172.21.15.47:0/2819402127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc56de80 con 0x55bfbc64da40 2024-03-21T11:40:57.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.761+0000 7f44273b7740 1 -- 172.21.15.47:0/2819402127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc666cf0 con 0x55bfbc652bb0 2024-03-21T11:40:57.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.762+0000 7f4421200640 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64da40 0x55bfbc668550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.762+0000 7f44273b7740 10 client.0 _ll_register_callbacks cb 0x55bfbc580130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:57.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.762+0000 7f4422600640 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc66f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.762+0000 7f4421c00640 1 -- 172.21.15.47:0/2819402127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 msgr2=0x55bfbc66f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.762+0000 7f4421c00640 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc66f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.762+0000 7f4421c00640 1 -- 172.21.15.47:0/2819402127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64da40 msgr2=0x55bfbc668550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.762+0000 7f4421c00640 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64da40 0x55bfbc668550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.762+0000 7f4421c00640 1 -- 172.21.15.47:0/2819402127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfbc3907d0 con 0x55bfbc64d050 2024-03-21T11:40:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.763+0000 7f4421200640 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64da40 0x55bfbc668550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.763+0000 7f4422600640 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc66f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.763+0000 7f4421c00640 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc668010 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4410009ab0 tx=0x7f441000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7403ce2b7bd4050 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.763+0000 7f4403400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:40:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.764+0000 7f4403400640 1 -- 172.21.15.47:0/2819402127 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441001d040 con 0x55bfbc64d050 2024-03-21T11:40:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.764+0000 7f4403400640 1 -- 172.21.15.47:0/2819402127 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f44100037a0 con 0x55bfbc64d050 2024-03-21T11:40:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.764+0000 7f44273b7740 1 -- 172.21.15.47:0/2819402127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44100035f0 con 0x55bfbc64d050 2024-03-21T11:40:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.764+0000 7f4403400640 1 -- 172.21.15.47:0/2819402127 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4410030050 con 0x55bfbc64d050 2024-03-21T11:40:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.764+0000 7f44273b7740 10 client.20754 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.765+0000 7f44273b7740 1 -- 172.21.15.47:0/2819402127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55bfbc2c63f0 con 0x55bfbc64d050 2024-03-21T11:40:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.765+0000 7f4403400640 1 -- 172.21.15.47:0/2819402127 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44100035f0 con 0x55bfbc64d050 2024-03-21T11:40:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.765+0000 7f4403400640 1 -- 172.21.15.47:0/2819402127 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4410004930 con 0x55bfbc64d050 2024-03-21T11:40:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.766+0000 7f44273b7740 10 client.20754 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.766+0000 7f44273b7740 10 client.20754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.766+0000 7f44273b7740 20 client.20754 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.766+0000 7f44273b7740 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bfbc673a80 0x55bfbc675e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.766+0000 7f44273b7740 4 client.20754 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.766+0000 7f44273b7740 1 -- 172.21.15.47:0/2819402127 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfbc5bf790 con 0x55bfbc673a80 2024-03-21T11:40:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.766+0000 7f4421200640 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bfbc673a80 0x55bfbc675e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.767+0000 7f4403400640 10 client.20754 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:57.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.767+0000 7f4421200640 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bfbc673a80 0x55bfbc675e70 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4ef719e1461b3b server_cookie=c57157589278e419 in_seq=0 out_seq=0 2024-03-21T11:40:57.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.773+0000 7f4403400640 1 -- 172.21.15.47:0/2819402127 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bfbc5bf790 con 0x55bfbc673a80 2024-03-21T11:40:57.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.773+0000 7f4403400640 10 client.20754 handle_command_reply: tid=0 2024-03-21T11:40:57.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.773+0000 7f44273b7740 1 client.20754 shutdown 2024-03-21T11:40:57.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.774+0000 7f44273b7740 1 -- 172.21.15.47:0/2819402127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 msgr2=0x55bfbc668010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.774+0000 7f44273b7740 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc668010 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4410009ab0 tx=0x7f441000ed50 comp rx=0 tx=0).stop 2024-03-21T11:40:57.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.774+0000 7f44273b7740 1 -- 172.21.15.47:0/2819402127 shutdown_connections 2024-03-21T11:40:57.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.774+0000 7f44273b7740 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bfbc673a80 0x55bfbc675e70 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.774+0000 7f44273b7740 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc66f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.774+0000 7f44273b7740 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64da40 0x55bfbc668550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.774+0000 7f44273b7740 1 --2- 172.21.15.47:0/2819402127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc668010 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.774+0000 7f44273b7740 1 -- 172.21.15.47:0/2819402127 >> 172.21.15.47:0/2819402127 conn(0x55bfbc615200 msgr2=0x55bfbc647bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:57.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.774+0000 7f4403400640 0 client.20754 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.774+0000 7f44273b7740 1 -- 172.21.15.47:0/2819402127 shutdown_connections 2024-03-21T11:40:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.775+0000 7f44273b7740 1 -- 172.21.15.47:0/2819402127 wait complete. 2024-03-21T11:40:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.775+0000 7f44273b7740 20 client.20754 trim_cache size 0 max 16384 2024-03-21T11:40:57.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f44273b7740 1 Processor -- start 2024-03-21T11:40:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f44273b7740 1 -- start start 2024-03-21T11:40:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f44273b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc64ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f44273b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64da40 0x55bfbc66f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f44273b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc667630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f44273b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1a7cd0 con 0x55bfbc64d050 2024-03-21T11:40:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f44273b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1a9650 con 0x55bfbc64da40 2024-03-21T11:40:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f44273b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1a8330 con 0x55bfbc652bb0 2024-03-21T11:40:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f4422600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc667630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f4421c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc64ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f4421200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64da40 0x55bfbc66f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f4422600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc667630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42336/0 (socket says 172.21.15.47:42336) 2024-03-21T11:40:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f4421c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc64ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39504/0 (socket says 172.21.15.47:39504) 2024-03-21T11:40:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f4422600640 1 -- 172.21.15.47:0/2796841956 learned_addr learned my addr 172.21.15.47:0/2796841956 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.803+0000 7f4421200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64da40 0x55bfbc66f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47320/0 (socket says 172.21.15.47:47320) 2024-03-21T11:40:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.804+0000 7f4421200640 1 -- 172.21.15.47:0/2796841956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 msgr2=0x55bfbc667630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.804+0000 7f4421200640 1 --2- 172.21.15.47:0/2796841956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc667630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.804+0000 7f4421200640 1 -- 172.21.15.47:0/2796841956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 msgr2=0x55bfbc64ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.804+0000 7f4421200640 1 --2- 172.21.15.47:0/2796841956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc64ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.804+0000 7f4421200640 1 -- 172.21.15.47:0/2796841956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfbc2860f0 con 0x55bfbc64da40 2024-03-21T11:40:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.804+0000 7f4422600640 1 --2- 172.21.15.47:0/2796841956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc667630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.804+0000 7f4421200640 1 --2- 172.21.15.47:0/2796841956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64da40 0x55bfbc66f550 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x55bfbc6694f0 tx=0x7f44080f0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92316b818c4eb96f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.804+0000 7f440e000640 1 -- 172.21.15.47:0/2796841956 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44080bd820 con 0x55bfbc64da40 2024-03-21T11:40:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.805+0000 7f440e000640 1 -- 172.21.15.47:0/2796841956 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44080bd120 con 0x55bfbc64da40 2024-03-21T11:40:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.805+0000 7f440e000640 1 -- 172.21.15.47:0/2796841956 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44080ee600 con 0x55bfbc64da40 2024-03-21T11:40:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.805+0000 7f44273b7740 1 -- 172.21.15.47:0/2796841956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64da40 msgr2=0x55bfbc66f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.805+0000 7f44273b7740 1 --2- 172.21.15.47:0/2796841956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64da40 0x55bfbc66f550 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x55bfbc6694f0 tx=0x7f44080f0040 comp rx=0 tx=0).stop 2024-03-21T11:40:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.805+0000 7f44273b7740 1 -- 172.21.15.47:0/2796841956 shutdown_connections 2024-03-21T11:40:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.805+0000 7f44273b7740 1 --2- 172.21.15.47:0/2796841956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc667630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.805+0000 7f44273b7740 1 --2- 172.21.15.47:0/2796841956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64da40 0x55bfbc66f550 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.805+0000 7f44273b7740 1 --2- 172.21.15.47:0/2796841956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64d050 0x55bfbc64ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.805+0000 7f44273b7740 1 -- 172.21.15.47:0/2796841956 >> 172.21.15.47:0/2796841956 conn(0x55bfbc615200 msgr2=0x55bfbc64c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.806+0000 7f44273b7740 1 -- 172.21.15.47:0/2796841956 shutdown_connections 2024-03-21T11:40:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.806+0000 7f44273b7740 1 -- 172.21.15.47:0/2796841956 wait complete. 2024-03-21T11:40:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.806+0000 7f44273b7740 1 -- start start 2024-03-21T11:40:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.806+0000 7f44273b7740 1 Processor -- start 2024-03-21T11:40:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f44273b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64d050 0x55bfbc66cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f44273b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64da40 0x55bfbc66d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f44273b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc66e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f44273b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1a8ff0 con 0x55bfbc64da40 2024-03-21T11:40:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f44273b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1a8990 con 0x55bfbc64d050 2024-03-21T11:40:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f44273b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c10b4e0 con 0x55bfbc652bb0 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f44273b7740 10 client.0 _ll_register_callbacks cb 0x7f44100099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f4421c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64d050 0x55bfbc66cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f4421200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64da40 0x55bfbc66d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f4421c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64d050 0x55bfbc66cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47322/0 (socket says 172.21.15.47:47322) 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f4421c00640 1 -- 172.21.15.47:0/2153345624 learned_addr learned my addr 172.21.15.47:0/2153345624 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f4422600640 1 --2- 172.21.15.47:0/2153345624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc66e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f4421c00640 1 -- 172.21.15.47:0/2153345624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 msgr2=0x55bfbc66e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f4421c00640 1 --2- 172.21.15.47:0/2153345624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc66e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f4421c00640 1 -- 172.21.15.47:0/2153345624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64da40 msgr2=0x55bfbc66d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f4421c00640 1 --2- 172.21.15.47:0/2153345624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64da40 0x55bfbc66d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.807+0000 7f4421c00640 1 -- 172.21.15.47:0/2153345624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfbc66e6d0 con 0x55bfbc64d050 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.808+0000 7f4421c00640 1 --2- 172.21.15.47:0/2153345624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64d050 0x55bfbc66cb90 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4410004d80 tx=0x7f4410010610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eed53c9827b082e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.808+0000 7f4403400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.808+0000 7f4403400640 1 -- 172.21.15.47:0/2153345624 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410005300 con 0x55bfbc64d050 2024-03-21T11:40:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.808+0000 7f44273b7740 1 -- 172.21.15.47:0/2153345624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44080bd120 con 0x55bfbc64d050 2024-03-21T11:40:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.808+0000 7f4403400640 1 -- 172.21.15.47:0/2153345624 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f44100054a0 con 0x55bfbc64d050 2024-03-21T11:40:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.808+0000 7f4403400640 1 -- 172.21.15.47:0/2153345624 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4410045050 con 0x55bfbc64d050 2024-03-21T11:40:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.808+0000 7f44273b7740 10 client.31298 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.808+0000 7f44273b7740 1 -- 172.21.15.47:0/2153345624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55bfbc6cf230 con 0x55bfbc64d050 2024-03-21T11:40:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.809+0000 7f4403400640 1 -- 172.21.15.47:0/2153345624 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44080bd120 con 0x55bfbc64d050 2024-03-21T11:40:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.809+0000 7f4403400640 1 -- 172.21.15.47:0/2153345624 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4410004650 con 0x55bfbc64d050 2024-03-21T11:40:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.809+0000 7f44273b7740 10 client.31298 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.809+0000 7f44273b7740 10 client.31298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.809+0000 7f44273b7740 20 client.31298 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.809+0000 7f44273b7740 1 --2- 172.21.15.47:0/2153345624 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bfbc6cfac0 0x55bfbc6d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.809+0000 7f44273b7740 4 client.31298 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:40:57.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.809+0000 7f44273b7740 1 -- 172.21.15.47:0/2153345624 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bfbc5b9ae0 con 0x55bfbc6cfac0 2024-03-21T11:40:57.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.809+0000 7f4421200640 1 --2- 172.21.15.47:0/2153345624 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bfbc6cfac0 0x55bfbc6d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:57.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.810+0000 7f4421200640 1 --2- 172.21.15.47:0/2153345624 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bfbc6cfac0 0x55bfbc6d1eb0 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d4be59d0e70ba34 server_cookie=1cccb4719382de26 in_seq=0 out_seq=0 2024-03-21T11:40:57.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.810+0000 7f4403400640 10 client.31298 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f4403400640 1 -- 172.21.15.47:0/2153345624 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55bfbc5b9ae0 con 0x55bfbc6cfac0 2024-03-21T11:40:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f4403400640 10 client.31298 handle_command_reply: tid=0 2024-03-21T11:40:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f44273b7740 1 client.31298 shutdown 2024-03-21T11:40:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f44273b7740 1 -- 172.21.15.47:0/2153345624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64d050 msgr2=0x55bfbc66cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f44273b7740 1 --2- 172.21.15.47:0/2153345624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64d050 0x55bfbc66cb90 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4410004d80 tx=0x7f4410010610 comp rx=0 tx=0).stop 2024-03-21T11:40:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f44273b7740 1 -- 172.21.15.47:0/2153345624 shutdown_connections 2024-03-21T11:40:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f44273b7740 1 --2- 172.21.15.47:0/2153345624 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55bfbc6cfac0 0x55bfbc6d1eb0 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f44273b7740 1 --2- 172.21.15.47:0/2153345624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bfbc652bb0 0x55bfbc66e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f44273b7740 1 --2- 172.21.15.47:0/2153345624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfbc64da40 0x55bfbc66d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f44273b7740 1 --2- 172.21.15.47:0/2153345624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfbc64d050 0x55bfbc66cb90 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f44273b7740 1 -- 172.21.15.47:0/2153345624 >> 172.21.15.47:0/2153345624 conn(0x55bfbc615200 msgr2=0x55bfbc64c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:57.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f4403400640 0 client.31298 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:57.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f44273b7740 1 -- 172.21.15.47:0/2153345624 shutdown_connections 2024-03-21T11:40:57.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f44273b7740 1 -- 172.21.15.47:0/2153345624 wait complete. 2024-03-21T11:40:57.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.813+0000 7f44273b7740 20 client.31298 trim_cache size 0 max 16384 2024-03-21T11:40:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:40:57.819 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (66 inodes in the stack)", 2024-03-21T11:40:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:40:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: "f383a0ac-4d76-4c44-8141-64eb5cb7eab0": { 2024-03-21T11:40:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:40:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "f383a0ac-4d76-4c44-8141-64eb5cb7eab0", 2024-03-21T11:40:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:40:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:40:57.820 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:40:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:40:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.814+0000 7f4423000640 1 -- 172.21.15.47:0/1157232113 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44040a3020 msgr2=0x7f44040a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.814+0000 7f4423000640 1 --2- 172.21.15.47:0/1157232113 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44040a3020 0x7f44040a54e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55bfbc64e2a0 tx=0x7f4414009500 comp rx=0 tx=0).stop 2024-03-21T11:40:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.814+0000 7f4423000640 1 -- 172.21.15.47:0/1157232113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f441c0772a0 msgr2=0x7f441c19e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.814+0000 7f4423000640 1 --2- 172.21.15.47:0/1157232113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f441c0772a0 0x7f441c19e3b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f44080349e0 tx=0x7f4408032450 comp rx=0 tx=0).stop 2024-03-21T11:40:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.814+0000 7f4423000640 1 -- 172.21.15.47:0/1157232113 shutdown_connections 2024-03-21T11:40:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.814+0000 7f4423000640 1 --2- 172.21.15.47:0/1157232113 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44040a3020 0x7f44040a54e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.814+0000 7f4423000640 1 --2- 172.21.15.47:0/1157232113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f441c077c40 0x7f441c1a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.814+0000 7f4423000640 1 --2- 172.21.15.47:0/1157232113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f441c0772a0 0x7f441c19e3b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.814+0000 7f4423000640 1 --2- 172.21.15.47:0/1157232113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f441c076060 0x7f441c19de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.814+0000 7f4423000640 1 -- 172.21.15.47:0/1157232113 >> 172.21.15.47:0/1157232113 conn(0x7f441c0fed10 msgr2=0x7f441c100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.815+0000 7f4423000640 1 -- 172.21.15.47:0/1157232113 shutdown_connections 2024-03-21T11:40:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:40:57.815+0000 7f4423000640 1 -- 172.21.15.47:0/1157232113 wait complete. 2024-03-21T11:40:57.836 INFO:tasks.cephfs.filesystem:scrub status for tag:f383a0ac-4d76-4c44-8141-64eb5cb7eab0 - {'path': '/', 'tag': 'f383a0ac-4d76-4c44-8141-64eb5cb7eab0', 'options': 'recursive,force'} 2024-03-21T11:40:57.946 INFO:tasks.workunit.client.0.smithi047.stdout: 10 104142 24.41 MB/sec execute 101 sec latency 740.454 ms 2024-03-21T11:40:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.186+0000 7f3b93200640 1 -- 172.21.15.47:0/1937019003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b94074100 msgr2=0x7f3b94074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.186+0000 7f3b93200640 1 --2- 172.21.15.47:0/1937019003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b94074100 0x7f3b94074580 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009900 tx=0x7f3b8002bc30 comp rx=0 tx=0).stop 2024-03-21T11:40:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.186+0000 7f3b93200640 1 -- 172.21.15.47:0/1937019003 shutdown_connections 2024-03-21T11:40:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.186+0000 7f3b93200640 1 --2- 172.21.15.47:0/1937019003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b9410dae0 0x7f3b9410fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.186+0000 7f3b93200640 1 --2- 172.21.15.47:0/1937019003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b94074100 0x7f3b94074580 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.186+0000 7f3b93200640 1 --2- 172.21.15.47:0/1937019003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b940756e0 0x7f3b94073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.186+0000 7f3b93200640 1 -- 172.21.15.47:0/1937019003 >> 172.21.15.47:0/1937019003 conn(0x7f3b940fca80 msgr2=0x7f3b940feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.186+0000 7f3b93200640 1 -- 172.21.15.47:0/1937019003 shutdown_connections 2024-03-21T11:40:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.186+0000 7f3b93200640 1 -- 172.21.15.47:0/1937019003 wait complete. 2024-03-21T11:40:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.187+0000 7f3b93200640 1 Processor -- start 2024-03-21T11:40:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.188+0000 7f3b93200640 1 -- start start 2024-03-21T11:40:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.188+0000 7f3b93200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b94074100 0x7f3b9419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.188+0000 7f3b93200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b940756e0 0x7f3b9419e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.188+0000 7f3b93200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b9410dae0 0x7f3b941a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.188+0000 7f3b93200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941137b0 con 0x7f3b94074100 2024-03-21T11:40:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.188+0000 7f3b93200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b94113630 con 0x7f3b940756e0 2024-03-21T11:40:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.188+0000 7f3b93200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b94113930 con 0x7f3b9410dae0 2024-03-21T11:40:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.188+0000 7f3b92800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b9410dae0 0x7f3b941a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.188+0000 7f3b91e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b94074100 0x7f3b9419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.188+0000 7f3b92800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b9410dae0 0x7f3b941a53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42356/0 (socket says 172.21.15.47:42356) 2024-03-21T11:40:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.188+0000 7f3b92800640 1 -- 172.21.15.47:0/1225096009 learned_addr learned my addr 172.21.15.47:0/1225096009 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.189+0000 7f3b91400640 1 --2- 172.21.15.47:0/1225096009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b940756e0 0x7f3b9419e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.189+0000 7f3b92800640 1 -- 172.21.15.47:0/1225096009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b940756e0 msgr2=0x7f3b9419e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.189+0000 7f3b92800640 1 --2- 172.21.15.47:0/1225096009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b940756e0 0x7f3b9419e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.189+0000 7f3b92800640 1 -- 172.21.15.47:0/1225096009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b94074100 msgr2=0x7f3b9419de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.189+0000 7f3b92800640 1 --2- 172.21.15.47:0/1225096009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b94074100 0x7f3b9419de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.189+0000 7f3b92800640 1 -- 172.21.15.47:0/1225096009 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b941a5a20 con 0x7f3b9410dae0 2024-03-21T11:40:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.189+0000 7f3b92800640 1 --2- 172.21.15.47:0/1225096009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b9410dae0 0x7f3b941a53b0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009fd0 tx=0x7f3b8c00b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.190+0000 7f3b8aa00640 1 -- 172.21.15.47:0/1225096009 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c017070 con 0x7f3b9410dae0 2024-03-21T11:40:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.190+0000 7f3b93200640 1 -- 172.21.15.47:0/1225096009 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b941a5d10 con 0x7f3b9410dae0 2024-03-21T11:40:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.190+0000 7f3b8aa00640 1 -- 172.21.15.47:0/1225096009 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b8c003e50 con 0x7f3b9410dae0 2024-03-21T11:40:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.190+0000 7f3b8aa00640 1 -- 172.21.15.47:0/1225096009 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0106c0 con 0x7f3b9410dae0 2024-03-21T11:40:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.190+0000 7f3b93200640 1 -- 172.21.15.47:0/1225096009 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b941a6180 con 0x7f3b9410dae0 2024-03-21T11:40:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.193+0000 7f3b8aa00640 1 -- 172.21.15.47:0/1225096009 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3b8c003970 con 0x7f3b9410dae0 2024-03-21T11:40:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.194+0000 7f3b9b7cb740 1 Processor -- start 2024-03-21T11:40:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.194+0000 7f3b9b7cb740 1 -- start start 2024-03-21T11:40:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.194+0000 7f3b9b7cb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998b93f0 0x5571998b9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.194+0000 7f3b9b7cb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998be8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.194+0000 7f3b9b7cb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998bef50 0x5571998c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.194+0000 7f3b9b7cb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55719987c3b0 con 0x5571998b93f0 2024-03-21T11:40:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.194+0000 7f3b9b7cb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55719981e830 con 0x5571998b9de0 2024-03-21T11:40:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.194+0000 7f3b9b7cb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571997c75d0 con 0x5571998bef50 2024-03-21T11:40:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.194+0000 7f3b91400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998b93f0 0x5571998b9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.194+0000 7f3b91e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998be8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.195+0000 7f3b91400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998b93f0 0x5571998b9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39526/0 (socket says 172.21.15.47:39526) 2024-03-21T11:40:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.195+0000 7f3b91400640 1 -- 172.21.15.47:0/2986546975 learned_addr learned my addr 172.21.15.47:0/2986546975 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.195+0000 7f3b91400640 1 --2- 172.21.15.47:0/2986546975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998bef50 0x5571998c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.195+0000 7f3b8aa00640 1 --2- 172.21.15.47:0/1225096009 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b780a2f10 0x7f3b780a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.195+0000 7f3b8aa00640 1 -- 172.21.15.47:0/1225096009 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3b8c0cf5a0 con 0x7f3b9410dae0 2024-03-21T11:40:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.195+0000 7f3b91e00640 1 -- 172.21.15.47:0/2986546975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998bef50 msgr2=0x5571998c13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.195+0000 7f3b91e00640 1 --2- 172.21.15.47:0/2986546975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998bef50 0x5571998c13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.195+0000 7f3b91e00640 1 -- 172.21.15.47:0/2986546975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998b93f0 msgr2=0x5571998b9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.195+0000 7f3b91e00640 1 --2- 172.21.15.47:0/2986546975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998b93f0 0x5571998b9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.195+0000 7f3b91e00640 1 -- 172.21.15.47:0/2986546975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571997df2c0 con 0x5571998b9de0 2024-03-21T11:40:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.195+0000 7f3b91400640 1 --2- 172.21.15.47:0/2986546975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998b93f0 0x5571998b9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:40:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.195+0000 7f3b92800640 1 --2- 172.21.15.47:0/1225096009 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b780a2f10 0x7f3b780a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.196+0000 7f3b91e00640 1 --2- 172.21.15.47:0/2986546975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998be8e0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400bbf0 tx=0x7f3b840107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c607aec7acabf7c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.196+0000 7f3b92800640 1 --2- 172.21.15.47:0/1225096009 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b780a2f10 0x7f3b780a53d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00dfd0 tx=0x7f3b8c004020 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.196+0000 7f3b7f000640 1 -- 172.21.15.47:0/2986546975 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b84004030 con 0x5571998b9de0 2024-03-21T11:40:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.196+0000 7f3b7f000640 1 -- 172.21.15.47:0/2986546975 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b840041d0 con 0x5571998b9de0 2024-03-21T11:40:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.196+0000 7f3b7f000640 1 -- 172.21.15.47:0/2986546975 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b840116a0 con 0x5571998b9de0 2024-03-21T11:40:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.197+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/2986546975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 msgr2=0x5571998be8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.197+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/2986546975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998be8e0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400bbf0 tx=0x7f3b840107a0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.197+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/2986546975 shutdown_connections 2024-03-21T11:40:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.197+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/2986546975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998bef50 0x5571998c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.197+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/2986546975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998be8e0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.197+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/2986546975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998b93f0 0x5571998b9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.197+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/2986546975 >> 172.21.15.47:0/2986546975 conn(0x5571998b4960 msgr2=0x5571998b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.197+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/2986546975 shutdown_connections 2024-03-21T11:40:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.197+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/2986546975 wait complete. 2024-03-21T11:40:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.198+0000 7f3b9b7cb740 1 -- start start 2024-03-21T11:40:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.198+0000 7f3b9b7cb740 1 Processor -- start 2024-03-21T11:40:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.198+0000 7f3b9b7cb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998b93f0 0x5571998c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.198+0000 7f3b9b7cb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.198+0000 7f3b9b7cb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998bef50 0x5571998cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.198+0000 7f3b9b7cb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571997daa20 con 0x5571998b93f0 2024-03-21T11:40:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.199+0000 7f3b9b7cb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55719981a8d0 con 0x5571998b9de0 2024-03-21T11:40:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.199+0000 7f3b9b7cb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571998c1f00 con 0x5571998bef50 2024-03-21T11:40:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.199+0000 7f3b9b7cb740 10 client.0 _ll_register_callbacks cb 0x5571998171c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.199+0000 7f3b91400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998b93f0 0x5571998c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.199+0000 7f3b91e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.199+0000 7f3b91400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998b93f0 0x5571998c35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39542/0 (socket says 172.21.15.47:39542) 2024-03-21T11:40:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.199+0000 7f3b91400640 1 -- 172.21.15.47:0/603254470 learned_addr learned my addr 172.21.15.47:0/603254470 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.199+0000 7f3b91400640 1 --2- 172.21.15.47:0/603254470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998bef50 0x5571998cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.199+0000 7f3b91e00640 1 -- 172.21.15.47:0/603254470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998bef50 msgr2=0x5571998cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.199+0000 7f3b91e00640 1 --2- 172.21.15.47:0/603254470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998bef50 0x5571998cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.199+0000 7f3b91e00640 1 -- 172.21.15.47:0/603254470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998b93f0 msgr2=0x5571998c35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.199+0000 7f3b91e00640 1 --2- 172.21.15.47:0/603254470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998b93f0 0x5571998c35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.199+0000 7f3b91e00640 1 -- 172.21.15.47:0/603254470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557199533210 con 0x5571998b9de0 2024-03-21T11:40:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.199+0000 7f3b91400640 1 --2- 172.21.15.47:0/603254470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998bef50 0x5571998cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:40:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.200+0000 7f3b91e00640 1 --2- 172.21.15.47:0/603254470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998c3b20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x5571998b58d0 tx=0x7f3b8400b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5722a20ae8efd828 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.200+0000 7f3b73400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.200+0000 7f3b73400640 1 -- 172.21.15.47:0/603254470 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8402a060 con 0x5571998b9de0 2024-03-21T11:40:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.200+0000 7f3b73400640 1 -- 172.21.15.47:0/603254470 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3b84004380 con 0x5571998b9de0 2024-03-21T11:40:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.200+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/603254470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b840041d0 con 0x5571998b9de0 2024-03-21T11:40:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.201+0000 7f3b73400640 1 -- 172.21.15.47:0/603254470 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b84031050 con 0x5571998b9de0 2024-03-21T11:40:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.201+0000 7f3b9b7cb740 10 client.31310 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.201+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/603254470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5571995ff620 con 0x5571998b9de0 2024-03-21T11:40:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.201+0000 7f3b73400640 1 -- 172.21.15.47:0/603254470 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b840041d0 con 0x5571998b9de0 2024-03-21T11:40:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.201+0000 7f3b73400640 1 -- 172.21.15.47:0/603254470 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3b84004740 con 0x5571998b9de0 2024-03-21T11:40:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.201+0000 7f3b9b7cb740 10 client.31310 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.201+0000 7f3b9b7cb740 10 client.31310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.202+0000 7f3b9b7cb740 20 client.31310 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.202+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/603254470 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5571998cf150 0x5571998d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.202+0000 7f3b9b7cb740 4 client.31310 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:40:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.202+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/603254470 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55719982bd70 con 0x5571998cf150 2024-03-21T11:40:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.202+0000 7f3b91400640 1 --2- 172.21.15.47:0/603254470 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5571998cf150 0x5571998d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.203+0000 7f3b91400640 1 --2- 172.21.15.47:0/603254470 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5571998cf150 0x5571998d1540 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea6d0e2791f19b9 server_cookie=365e3df0f0f0f5bb in_seq=0 out_seq=0 2024-03-21T11:40:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.203+0000 7f3b73400640 10 client.31310 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.206+0000 7f3b73400640 1 -- 172.21.15.47:0/603254470 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55719982bd70 con 0x5571998cf150 2024-03-21T11:40:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.206+0000 7f3b73400640 10 client.31310 handle_command_reply: tid=0 2024-03-21T11:40:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.206+0000 7f3b9b7cb740 1 client.31310 shutdown 2024-03-21T11:40:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.206+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/603254470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 msgr2=0x5571998c3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.206+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/603254470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998c3b20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x5571998b58d0 tx=0x7f3b8400b820 comp rx=0 tx=0).stop 2024-03-21T11:40:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.206+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/603254470 shutdown_connections 2024-03-21T11:40:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.206+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/603254470 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5571998cf150 0x5571998d1540 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.206+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/603254470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998bef50 0x5571998cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.206+0000 7f3b73400640 0 client.31310 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.206+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/603254470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998c3b20 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.206+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/603254470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998b93f0 0x5571998c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.206+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/603254470 >> 172.21.15.47:0/603254470 conn(0x5571998b4960 msgr2=0x5571998b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.206+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/603254470 shutdown_connections 2024-03-21T11:40:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.207+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/603254470 wait complete. 2024-03-21T11:40:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.207+0000 7f3b9b7cb740 20 client.31310 trim_cache size 0 max 16384 2024-03-21T11:40:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.229+0000 7f3b9b7cb740 1 Processor -- start 2024-03-21T11:40:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.229+0000 7f3b9b7cb740 1 -- start start 2024-03-21T11:40:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.230+0000 7f3b9b7cb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b93f0 0x5571998be790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.230+0000 7f3b9b7cb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998b9de0 0x5571998cab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.230+0000 7f3b9b7cb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998bef50 0x5571998b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.230+0000 7f3b9b7cb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941b0490 con 0x5571998bef50 2024-03-21T11:40:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.230+0000 7f3b9b7cb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941b1e10 con 0x5571998b93f0 2024-03-21T11:40:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.230+0000 7f3b9b7cb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941b0af0 con 0x5571998b9de0 2024-03-21T11:40:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.230+0000 7f3b91e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998b9de0 0x5571998cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.230+0000 7f3b91e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998b9de0 0x5571998cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42372/0 (socket says 172.21.15.47:42372) 2024-03-21T11:40:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.230+0000 7f3b91e00640 1 -- 172.21.15.47:0/889785561 learned_addr learned my addr 172.21.15.47:0/889785561 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.231+0000 7f3b91400640 1 --2- 172.21.15.47:0/889785561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b93f0 0x5571998be790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.231+0000 7f3b91400640 1 --2- 172.21.15.47:0/889785561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998bef50 0x5571998b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.231+0000 7f3b91e00640 1 -- 172.21.15.47:0/889785561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b93f0 msgr2=0x5571998be790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.231+0000 7f3b91e00640 1 --2- 172.21.15.47:0/889785561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b93f0 0x5571998be790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.231+0000 7f3b91e00640 1 -- 172.21.15.47:0/889785561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998bef50 msgr2=0x5571998b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.231+0000 7f3b91e00640 1 --2- 172.21.15.47:0/889785561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998bef50 0x5571998b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.231+0000 7f3b91e00640 1 -- 172.21.15.47:0/889785561 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571998be410 con 0x5571998b9de0 2024-03-21T11:40:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.232+0000 7f3b91e00640 1 --2- 172.21.15.47:0/889785561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998b9de0 0x5571998cab20 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f3b84010ca0 tx=0x7f3b84010cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc6fea22fd2d09b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.232+0000 7f3b7e600640 1 -- 172.21.15.47:0/889785561 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8401f070 con 0x5571998b9de0 2024-03-21T11:40:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.232+0000 7f3b7e600640 1 -- 172.21.15.47:0/889785561 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b840041b0 con 0x5571998b9de0 2024-03-21T11:40:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.232+0000 7f3b7e600640 1 -- 172.21.15.47:0/889785561 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8402a070 con 0x5571998b9de0 2024-03-21T11:40:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.233+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/889785561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998b9de0 msgr2=0x5571998cab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.233+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/889785561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998b9de0 0x5571998cab20 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f3b84010ca0 tx=0x7f3b84010cd0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.233+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/889785561 shutdown_connections 2024-03-21T11:40:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.233+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/889785561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998bef50 0x5571998b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.233+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/889785561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998b9de0 0x5571998cab20 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.233+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/889785561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b93f0 0x5571998be790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.233+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/889785561 >> 172.21.15.47:0/889785561 conn(0x5571998b4960 msgr2=0x5571998d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.233+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/889785561 shutdown_connections 2024-03-21T11:40:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.233+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/889785561 wait complete. 2024-03-21T11:40:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.234+0000 7f3b9b7cb740 1 -- start start 2024-03-21T11:40:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.234+0000 7f3b9b7cb740 1 Processor -- start 2024-03-21T11:40:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.234+0000 7f3b9b7cb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998b93f0 0x5571998c8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.234+0000 7f3b9b7cb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.234+0000 7f3b9b7cb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998bef50 0x5571998c9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.234+0000 7f3b9b7cb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941b17b0 con 0x5571998bef50 2024-03-21T11:40:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.234+0000 7f3b9b7cb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941b1150 con 0x5571998b9de0 2024-03-21T11:40:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.234+0000 7f3b9b7cb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941ae650 con 0x5571998b93f0 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.234+0000 7f3b9b7cb740 10 client.0 _ll_register_callbacks cb 0x7f3b80009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.235+0000 7f3b91e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.235+0000 7f3b91400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998b93f0 0x5571998c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.235+0000 7f3b91e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47354/0 (socket says 172.21.15.47:47354) 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.235+0000 7f3b91e00640 1 -- 172.21.15.47:0/2785786858 learned_addr learned my addr 172.21.15.47:0/2785786858 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.235+0000 7f3b91400640 1 --2- 172.21.15.47:0/2785786858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998bef50 0x5571998c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.235+0000 7f3b91e00640 1 -- 172.21.15.47:0/2785786858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998b93f0 msgr2=0x5571998c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.235+0000 7f3b91e00640 1 --2- 172.21.15.47:0/2785786858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998b93f0 0x5571998c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.235+0000 7f3b91e00640 1 -- 172.21.15.47:0/2785786858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998bef50 msgr2=0x5571998c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.235+0000 7f3b91e00640 1 --2- 172.21.15.47:0/2785786858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998bef50 0x5571998c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.235+0000 7f3b91e00640 1 -- 172.21.15.47:0/2785786858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571998c9c30 con 0x5571998b9de0 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.236+0000 7f3b91e00640 1 --2- 172.21.15.47:0/2785786858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998c8680 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f3b84004b10 tx=0x7f3b840038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1b35e6d072dc590 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.236+0000 7f3b73400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.236+0000 7f3b73400640 1 -- 172.21.15.47:0/2785786858 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8402a030 con 0x5571998b9de0 2024-03-21T11:40:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.236+0000 7f3b73400640 1 -- 172.21.15.47:0/2785786858 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3b84003c90 con 0x5571998b9de0 2024-03-21T11:40:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.236+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/2785786858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b840041b0 con 0x5571998b9de0 2024-03-21T11:40:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.237+0000 7f3b73400640 1 -- 172.21.15.47:0/2785786858 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b84048050 con 0x5571998b9de0 2024-03-21T11:40:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.237+0000 7f3b9b7cb740 10 client.31316 fetch_fsmap learned FSMap version 40 2024-03-21T11:40:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.237+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/2785786858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55719992a890 con 0x5571998b9de0 2024-03-21T11:40:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.237+0000 7f3b73400640 1 -- 172.21.15.47:0/2785786858 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b840041b0 con 0x5571998b9de0 2024-03-21T11:40:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.237+0000 7f3b73400640 1 -- 172.21.15.47:0/2785786858 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3b84011aa0 con 0x5571998b9de0 2024-03-21T11:40:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.237+0000 7f3b9b7cb740 10 client.31316 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:40:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.237+0000 7f3b9b7cb740 10 client.31316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:40:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.237+0000 7f3b9b7cb740 20 client.31316 populate_metadata read hostname 'smithi047' 2024-03-21T11:40:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.237+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/2785786858 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55719992afe0 0x55719992d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:40:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.237+0000 7f3b9b7cb740 4 client.31316 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:40:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.237+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/2785786858 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5571998260c0 con 0x55719992afe0 2024-03-21T11:40:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.238+0000 7f3b91400640 1 --2- 172.21.15.47:0/2785786858 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55719992afe0 0x55719992d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:40:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.238+0000 7f3b91400640 1 --2- 172.21.15.47:0/2785786858 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55719992afe0 0x55719992d400 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4150c942b8587e51 server_cookie=7a6ca1cefa3578a8 in_seq=0 out_seq=0 2024-03-21T11:40:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:58.238+0000 7f3b73400640 10 client.31316 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:58.946 INFO:tasks.workunit.client.0.smithi047.stdout: 10 104353 24.32 MB/sec execute 102 sec latency 594.232 ms 2024-03-21T11:40:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:40:59.089 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.090 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:59.090 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:59.090 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:59.090 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:59.090 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.090 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:59.090 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.090 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.090 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:59.090 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.090 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:59.091 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.091 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:40:59.091 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.091 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: pgmap v929: 129 pgs: 129 active+clean; 446 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 609 op/s 2024-03-21T11:40:59.091 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.091 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:40:58 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:40:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:59.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:59.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:59.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:40:59.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: pgmap v929: 129 pgs: 129 active+clean; 446 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 609 op/s 2024-03-21T11:40:59.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:40:58 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:40:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:40:59.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: pgmap v929: 129 pgs: 129 active+clean; 446 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 609 op/s 2024-03-21T11:40:59.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:40:58 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:40:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.462+0000 7f3b73400640 1 -- 172.21.15.47:0/2785786858 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5571998260c0 con 0x55719992afe0 2024-03-21T11:40:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.462+0000 7f3b73400640 10 client.31316 handle_command_reply: tid=0 2024-03-21T11:40:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.464+0000 7f3b9b7cb740 1 client.31316 shutdown 2024-03-21T11:40:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.465+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/2785786858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 msgr2=0x5571998c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.465+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/2785786858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998c8680 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f3b84004b10 tx=0x7f3b840038a0 comp rx=0 tx=0).stop 2024-03-21T11:40:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.465+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/2785786858 shutdown_connections 2024-03-21T11:40:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.465+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/2785786858 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55719992afe0 0x55719992d400 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.465+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/2785786858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5571998bef50 0x5571998c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.465+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/2785786858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5571998b9de0 0x5571998c8680 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.465+0000 7f3b9b7cb740 1 --2- 172.21.15.47:0/2785786858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571998b93f0 0x5571998c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.465+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/2785786858 >> 172.21.15.47:0/2785786858 conn(0x5571998b4960 msgr2=0x5571998b4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.466+0000 7f3b73400640 0 client.31316 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:40:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.466+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/2785786858 shutdown_connections 2024-03-21T11:40:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.466+0000 7f3b9b7cb740 1 -- 172.21.15.47:0/2785786858 wait complete. 2024-03-21T11:40:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.467+0000 7f3b9b7cb740 20 client.31316 trim_cache size 0 max 16384 2024-03-21T11:40:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:40:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:40:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:40:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:40:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.469+0000 7f3b93200640 1 -- 172.21.15.47:0/1225096009 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b780a2f10 msgr2=0x7f3b780a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.469+0000 7f3b93200640 1 --2- 172.21.15.47:0/1225096009 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b780a2f10 0x7f3b780a53d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00dfd0 tx=0x7f3b8c004020 comp rx=0 tx=0).stop 2024-03-21T11:40:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.469+0000 7f3b93200640 1 -- 172.21.15.47:0/1225096009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b9410dae0 msgr2=0x7f3b941a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:40:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.469+0000 7f3b93200640 1 --2- 172.21.15.47:0/1225096009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b9410dae0 0x7f3b941a53b0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009fd0 tx=0x7f3b8c00b260 comp rx=0 tx=0).stop 2024-03-21T11:40:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.470+0000 7f3b93200640 1 -- 172.21.15.47:0/1225096009 shutdown_connections 2024-03-21T11:40:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.470+0000 7f3b93200640 1 --2- 172.21.15.47:0/1225096009 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3b780a2f10 0x7f3b780a53d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.470+0000 7f3b93200640 1 --2- 172.21.15.47:0/1225096009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b9410dae0 0x7f3b941a53b0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.470+0000 7f3b93200640 1 --2- 172.21.15.47:0/1225096009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b940756e0 0x7f3b9419e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.470+0000 7f3b93200640 1 --2- 172.21.15.47:0/1225096009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b94074100 0x7f3b9419de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:40:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.470+0000 7f3b93200640 1 -- 172.21.15.47:0/1225096009 >> 172.21.15.47:0/1225096009 conn(0x7f3b940fca80 msgr2=0x7f3b94100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:40:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.470+0000 7f3b93200640 1 -- 172.21.15.47:0/1225096009 shutdown_connections 2024-03-21T11:40:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:40:59.470+0000 7f3b93200640 1 -- 172.21.15.47:0/1225096009 wait complete. 2024-03-21T11:40:59.946 INFO:tasks.workunit.client.0.smithi047.stdout: 10 104568 24.21 MB/sec execute 103 sec latency 1010.072 ms 2024-03-21T11:41:00.946 INFO:tasks.workunit.client.0.smithi047.stdout: 10 105019 24.19 MB/sec execute 104 sec latency 98.032 ms 2024-03-21T11:41:00.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:00 smithi102 ceph-mon[36748]: pgmap v930: 129 pgs: 129 active+clean; 456 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 722 op/s 2024-03-21T11:41:01.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:00 smithi047 ceph-mon[31351]: pgmap v930: 129 pgs: 129 active+clean; 456 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 722 op/s 2024-03-21T11:41:01.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:00 smithi100 ceph-mon[36245]: pgmap v930: 129 pgs: 129 active+clean; 456 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 722 op/s 2024-03-21T11:41:01.946 INFO:tasks.workunit.client.0.smithi047.stdout: 10 105465 24.19 MB/sec execute 105 sec latency 78.076 ms 2024-03-21T11:41:02.946 INFO:tasks.workunit.client.0.smithi047.stdout: 10 105924 24.21 MB/sec execute 106 sec latency 73.994 ms 2024-03-21T11:41:03.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:02 smithi047 ceph-mon[31351]: pgmap v931: 129 pgs: 129 active+clean; 454 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.6 MiB/s wr, 570 op/s 2024-03-21T11:41:03.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:02 smithi100 ceph-mon[36245]: pgmap v931: 129 pgs: 129 active+clean; 454 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.6 MiB/s wr, 570 op/s 2024-03-21T11:41:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:02 smithi102 ceph-mon[36748]: pgmap v931: 129 pgs: 129 active+clean; 454 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.6 MiB/s wr, 570 op/s 2024-03-21T11:41:03.946 INFO:tasks.workunit.client.0.smithi047.stdout: 10 106259 24.08 MB/sec execute 107 sec latency 276.638 ms 2024-03-21T11:41:04.946 INFO:tasks.workunit.client.0.smithi047.stdout: 10 106693 24.07 MB/sec execute 108 sec latency 160.525 ms 2024-03-21T11:41:05.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:04 smithi047 ceph-mon[31351]: pgmap v932: 129 pgs: 129 active+clean; 454 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.2 MiB/s wr, 571 op/s 2024-03-21T11:41:05.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:04 smithi100 ceph-mon[36245]: pgmap v932: 129 pgs: 129 active+clean; 454 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.2 MiB/s wr, 571 op/s 2024-03-21T11:41:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:04 smithi102 ceph-mon[36748]: pgmap v932: 129 pgs: 129 active+clean; 454 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.2 MiB/s wr, 571 op/s 2024-03-21T11:41:05.946 INFO:tasks.workunit.client.0.smithi047.stdout: 10 107261 24.09 MB/sec execute 109 sec latency 65.736 ms 2024-03-21T11:41:06.946 INFO:tasks.workunit.client.0.smithi047.stdout: 10 107866 24.26 MB/sec execute 110 sec latency 56.275 ms 2024-03-21T11:41:07.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:06 smithi100 ceph-mon[36245]: pgmap v933: 129 pgs: 129 active+clean; 454 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 646 op/s 2024-03-21T11:41:07.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:06 smithi102 ceph-mon[36748]: pgmap v933: 129 pgs: 129 active+clean; 454 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 646 op/s 2024-03-21T11:41:07.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:06 smithi047 ceph-mon[31351]: pgmap v933: 129 pgs: 129 active+clean; 454 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 646 op/s 2024-03-21T11:41:07.947 INFO:tasks.workunit.client.0.smithi047.stdout: 10 108465 24.32 MB/sec execute 111 sec latency 42.495 ms 2024-03-21T11:41:08.947 INFO:tasks.workunit.client.0.smithi047.stdout: 10 108740 24.25 MB/sec execute 112 sec latency 364.981 ms 2024-03-21T11:41:09.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:08 smithi100 ceph-mon[36245]: pgmap v934: 129 pgs: 129 active+clean; 454 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 618 op/s 2024-03-21T11:41:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:08 smithi102 ceph-mon[36748]: pgmap v934: 129 pgs: 129 active+clean; 454 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 618 op/s 2024-03-21T11:41:09.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:08 smithi047 ceph-mon[31351]: pgmap v934: 129 pgs: 129 active+clean; 454 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 618 op/s 2024-03-21T11:41:09.947 INFO:tasks.workunit.client.0.smithi047.stdout: 10 109066 24.23 MB/sec execute 113 sec latency 231.304 ms 2024-03-21T11:41:10.947 INFO:tasks.workunit.client.0.smithi047.stdout: 10 109388 24.14 MB/sec execute 114 sec latency 257.638 ms 2024-03-21T11:41:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:10 smithi100 ceph-mon[36245]: pgmap v935: 129 pgs: 129 active+clean; 460 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.2 MiB/s wr, 731 op/s 2024-03-21T11:41:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:10 smithi102 ceph-mon[36748]: pgmap v935: 129 pgs: 129 active+clean; 460 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.2 MiB/s wr, 731 op/s 2024-03-21T11:41:11.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:10 smithi047 ceph-mon[31351]: pgmap v935: 129 pgs: 129 active+clean; 460 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.2 MiB/s wr, 731 op/s 2024-03-21T11:41:11.947 INFO:tasks.workunit.client.0.smithi047.stdout: 10 109919 24.13 MB/sec execute 115 sec latency 136.553 ms 2024-03-21T11:41:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.679+0000 7f0d08800640 1 -- 172.21.15.47:0/3541386210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d00104a90 msgr2=0x7f0d00106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.679+0000 7f0d08800640 1 --2- 172.21.15.47:0/3541386210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d00104a90 0x7f0d00106e80 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009900 tx=0x7f0cf002bc40 comp rx=0 tx=0).stop 2024-03-21T11:41:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.679+0000 7f0d08800640 1 -- 172.21.15.47:0/3541386210 shutdown_connections 2024-03-21T11:41:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.679+0000 7f0d08800640 1 --2- 172.21.15.47:0/3541386210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d001073c0 0x7f0d00109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.679+0000 7f0d08800640 1 --2- 172.21.15.47:0/3541386210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d00104a90 0x7f0d00106e80 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.679+0000 7f0d08800640 1 --2- 172.21.15.47:0/3541386210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d00102160 0x7f0d00104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.679+0000 7f0d08800640 1 -- 172.21.15.47:0/3541386210 >> 172.21.15.47:0/3541386210 conn(0x7f0d000fbb50 msgr2=0x7f0d000fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.680+0000 7f0d08800640 1 -- 172.21.15.47:0/3541386210 shutdown_connections 2024-03-21T11:41:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.680+0000 7f0d08800640 1 -- 172.21.15.47:0/3541386210 wait complete. 2024-03-21T11:41:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.681+0000 7f0d08800640 1 Processor -- start 2024-03-21T11:41:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.681+0000 7f0d08800640 1 -- start start 2024-03-21T11:41:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.682+0000 7f0d08800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d00102160 0x7f0d00199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.682+0000 7f0d08800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d00104a90 0x7f0d0019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.682+0000 7f0d08800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d001073c0 0x7f0d001a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.682+0000 7f0d08800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0010d3b0 con 0x7f0d00104a90 2024-03-21T11:41:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.682+0000 7f0d05e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d00102160 0x7f0d00199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.682+0000 7f0d05e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d00102160 0x7f0d00199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60092/0 (socket says 172.21.15.47:60092) 2024-03-21T11:41:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.682+0000 7f0d05e00640 1 -- 172.21.15.47:0/1959447965 learned_addr learned my addr 172.21.15.47:0/1959447965 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.682+0000 7f0d08800640 1 -- 172.21.15.47:0/1959447965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0010d230 con 0x7f0d001073c0 2024-03-21T11:41:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.682+0000 7f0d08800640 1 -- 172.21.15.47:0/1959447965 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0010d530 con 0x7f0d00102160 2024-03-21T11:41:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.682+0000 7f0d05400640 1 --2- 172.21.15.47:0/1959447965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d00104a90 0x7f0d0019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.683+0000 7f0d06800640 1 --2- 172.21.15.47:0/1959447965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d001073c0 0x7f0d001a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.683+0000 7f0d05e00640 1 -- 172.21.15.47:0/1959447965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d001073c0 msgr2=0x7f0d001a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.683+0000 7f0d05e00640 1 --2- 172.21.15.47:0/1959447965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d001073c0 0x7f0d001a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.683+0000 7f0d05e00640 1 -- 172.21.15.47:0/1959447965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d00104a90 msgr2=0x7f0d0019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.683+0000 7f0d05e00640 1 --2- 172.21.15.47:0/1959447965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d00104a90 0x7f0d0019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.683+0000 7f0d05e00640 1 -- 172.21.15.47:0/1959447965 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d001a17e0 con 0x7f0d00102160 2024-03-21T11:41:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.683+0000 7f0d05e00640 1 --2- 172.21.15.47:0/1959447965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d00102160 0x7f0d00199bc0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400e9f0 tx=0x7f0cf400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.684+0000 7f0cfaa00640 1 -- 172.21.15.47:0/1959447965 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400cd80 con 0x7f0d00102160 2024-03-21T11:41:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.684+0000 7f0cfaa00640 1 -- 172.21.15.47:0/1959447965 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cf4010910 con 0x7f0d00102160 2024-03-21T11:41:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.684+0000 7f0cfaa00640 1 -- 172.21.15.47:0/1959447965 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4018990 con 0x7f0d00102160 2024-03-21T11:41:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.684+0000 7f0d08800640 1 -- 172.21.15.47:0/1959447965 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d001a1ad0 con 0x7f0d00102160 2024-03-21T11:41:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.684+0000 7f0d08800640 1 -- 172.21.15.47:0/1959447965 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d001a1fa0 con 0x7f0d00102160 2024-03-21T11:41:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.688+0000 7f0d0b614740 1 Processor -- start 2024-03-21T11:41:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.689+0000 7f0d0b614740 1 -- start start 2024-03-21T11:41:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.689+0000 7f0d0b614740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3e1e0 0x564504a3e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.689+0000 7f0d0b614740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3ebd0 0x564504a436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.689+0000 7f0d0b614740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a43d40 0x564504a461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.689+0000 7f0d0b614740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564504a013b0 con 0x564504a3e1e0 2024-03-21T11:41:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.689+0000 7f0d0b614740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645049a3830 con 0x564504a43d40 2024-03-21T11:41:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.689+0000 7f0d0b614740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450494c5d0 con 0x564504a3ebd0 2024-03-21T11:41:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.689+0000 7f0cfaa00640 1 -- 172.21.15.47:0/1959447965 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0cf4010ac0 con 0x7f0d00102160 2024-03-21T11:41:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.689+0000 7f0d05400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a43d40 0x564504a461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.689+0000 7f0d05400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a43d40 0x564504a461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60850/0 (socket says 172.21.15.47:60850) 2024-03-21T11:41:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.689+0000 7f0d05400640 1 -- 172.21.15.47:0/2960408467 learned_addr learned my addr 172.21.15.47:0/2960408467 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.689+0000 7f0d05e00640 1 --2- 172.21.15.47:0/2960408467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3ebd0 0x564504a436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.690+0000 7f0d06800640 1 --2- 172.21.15.47:0/2960408467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3e1e0 0x564504a3e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.690+0000 7f0d05400640 1 -- 172.21.15.47:0/2960408467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3ebd0 msgr2=0x564504a436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.690+0000 7f0d05400640 1 --2- 172.21.15.47:0/2960408467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3ebd0 0x564504a436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.690+0000 7f0d05400640 1 -- 172.21.15.47:0/2960408467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3e1e0 msgr2=0x564504a3e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.690+0000 7f0d05400640 1 --2- 172.21.15.47:0/2960408467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3e1e0 0x564504a3e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.690+0000 7f0d05400640 1 -- 172.21.15.47:0/2960408467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645049642c0 con 0x564504a43d40 2024-03-21T11:41:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.690+0000 7f0d06800640 1 --2- 172.21.15.47:0/2960408467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3e1e0 0x564504a3e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:41:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.690+0000 7f0d05400640 1 --2- 172.21.15.47:0/2960408467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a43d40 0x564504a461d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0cf002efd0 tx=0x7f0cf0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362476b44f4ab0d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.691+0000 7f0cef000640 1 -- 172.21.15.47:0/2960408467 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0042070 con 0x564504a43d40 2024-03-21T11:41:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.691+0000 7f0cef000640 1 -- 172.21.15.47:0/2960408467 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cf002cb90 con 0x564504a43d40 2024-03-21T11:41:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.691+0000 7f0cfaa00640 1 --2- 172.21.15.47:0/1959447965 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0ce80a3040 0x7f0ce80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.691+0000 7f0cef000640 1 -- 172.21.15.47:0/2960408467 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0004920 con 0x564504a43d40 2024-03-21T11:41:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.692+0000 7f0cfaa00640 1 -- 172.21.15.47:0/1959447965 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0cf40cfaf0 con 0x7f0d00102160 2024-03-21T11:41:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.692+0000 7f0d06800640 1 --2- 172.21.15.47:0/1959447965 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0ce80a3040 0x7f0ce80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.692+0000 7f0d0b614740 1 -- 172.21.15.47:0/2960408467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a43d40 msgr2=0x564504a461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.692+0000 7f0d0b614740 1 --2- 172.21.15.47:0/2960408467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a43d40 0x564504a461d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0cf002efd0 tx=0x7f0cf0000bc0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.693+0000 7f0d06800640 1 --2- 172.21.15.47:0/1959447965 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0ce80a3040 0x7f0ce80a5500 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x564504a3f430 tx=0x7f0cfc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.693+0000 7f0d0b614740 1 -- 172.21.15.47:0/2960408467 shutdown_connections 2024-03-21T11:41:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.693+0000 7f0d0b614740 1 --2- 172.21.15.47:0/2960408467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a43d40 0x564504a461d0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.693+0000 7f0d0b614740 1 --2- 172.21.15.47:0/2960408467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3ebd0 0x564504a436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.693+0000 7f0d0b614740 1 --2- 172.21.15.47:0/2960408467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3e1e0 0x564504a3e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.693+0000 7f0d0b614740 1 -- 172.21.15.47:0/2960408467 >> 172.21.15.47:0/2960408467 conn(0x564504a39750 msgr2=0x564504a39b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.694+0000 7f0d0b614740 1 -- 172.21.15.47:0/2960408467 shutdown_connections 2024-03-21T11:41:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.694+0000 7f0d0b614740 1 -- 172.21.15.47:0/2960408467 wait complete. 2024-03-21T11:41:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.694+0000 7f0d0b614740 1 -- start start 2024-03-21T11:41:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.694+0000 7f0d0b614740 1 Processor -- start 2024-03-21T11:41:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.695+0000 7f0d0b614740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3e1e0 0x564504a483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.695+0000 7f0d0b614740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a3ebd0 0x564504a48900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.695+0000 7f0d0b614740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a43d40 0x564504a4f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.696+0000 7f0d0b614740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450495fa20 con 0x564504a43d40 2024-03-21T11:41:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.696+0000 7f0d0b614740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450499f8d0 con 0x564504a3ebd0 2024-03-21T11:41:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.696+0000 7f0d0b614740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564504a46c90 con 0x564504a3e1e0 2024-03-21T11:41:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.696+0000 7f0d05e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3e1e0 0x564504a483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.696+0000 7f0d0b614740 10 client.0 _ll_register_callbacks cb 0x56450499c1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.696+0000 7f0d05e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3e1e0 0x564504a483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60106/0 (socket says 172.21.15.47:60106) 2024-03-21T11:41:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.696+0000 7f0d05e00640 1 -- 172.21.15.47:0/4252846929 learned_addr learned my addr 172.21.15.47:0/4252846929 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.696+0000 7f0d06800640 1 --2- 172.21.15.47:0/4252846929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a43d40 0x564504a4f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.696+0000 7f0d05e00640 1 -- 172.21.15.47:0/4252846929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a3ebd0 msgr2=0x564504a48900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:41:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.696+0000 7f0d05e00640 1 --2- 172.21.15.47:0/4252846929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a3ebd0 0x564504a48900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.696+0000 7f0d05e00640 1 -- 172.21.15.47:0/4252846929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a43d40 msgr2=0x564504a4f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.696+0000 7f0d05e00640 1 --2- 172.21.15.47:0/4252846929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a43d40 0x564504a4f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.696+0000 7f0d05e00640 1 -- 172.21.15.47:0/4252846929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645046b8210 con 0x564504a3e1e0 2024-03-21T11:41:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.697+0000 7f0d05e00640 1 --2- 172.21.15.47:0/4252846929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3e1e0 0x564504a483c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4000c00 tx=0x7f0cf409b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=571980793a1eaf90 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.697+0000 7f0ce3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:41:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.697+0000 7f0ce3400640 1 -- 172.21.15.47:0/4252846929 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf409ff80 con 0x564504a3e1e0 2024-03-21T11:41:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.697+0000 7f0d0b614740 1 -- 172.21.15.47:0/4252846929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cf002cb90 con 0x564504a3e1e0 2024-03-21T11:41:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.697+0000 7f0ce3400640 1 -- 172.21.15.47:0/4252846929 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0cf40baba0 con 0x564504a3e1e0 2024-03-21T11:41:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.698+0000 7f0ce3400640 1 -- 172.21.15.47:0/4252846929 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cf40e7020 con 0x564504a3e1e0 2024-03-21T11:41:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.698+0000 7f0d0b614740 10 client.31309 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.698+0000 7f0d0b614740 1 -- 172.21.15.47:0/4252846929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564504784620 con 0x564504a3e1e0 2024-03-21T11:41:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.698+0000 7f0ce3400640 1 -- 172.21.15.47:0/4252846929 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cf002cb90 con 0x564504a3e1e0 2024-03-21T11:41:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.698+0000 7f0ce3400640 1 -- 172.21.15.47:0/4252846929 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0cf4018c30 con 0x564504a3e1e0 2024-03-21T11:41:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.698+0000 7f0d0b614740 10 client.31309 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.698+0000 7f0d0b614740 10 client.31309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.698+0000 7f0d0b614740 20 client.31309 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.698+0000 7f0d0b614740 1 --2- 172.21.15.47:0/4252846929 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564504a54150 0x564504a565a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.698+0000 7f0d0b614740 4 client.31309 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:41:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.698+0000 7f0d0b614740 1 -- 172.21.15.47:0/4252846929 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645049b0d70 con 0x564504a54150 2024-03-21T11:41:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.699+0000 7f0d05400640 1 --2- 172.21.15.47:0/4252846929 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564504a54150 0x564504a565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.699+0000 7f0d05400640 1 --2- 172.21.15.47:0/4252846929 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564504a54150 0x564504a565a0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86b1d4bf6f47d390 server_cookie=1c1192931b163795 in_seq=0 out_seq=0 2024-03-21T11:41:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.700+0000 7f0ce3400640 10 client.31309 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.702+0000 7f0ce3400640 1 -- 172.21.15.47:0/4252846929 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5645049b0d70 con 0x564504a54150 2024-03-21T11:41:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.702+0000 7f0ce3400640 10 client.31309 handle_command_reply: tid=0 2024-03-21T11:41:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.702+0000 7f0d0b614740 1 client.31309 shutdown 2024-03-21T11:41:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.702+0000 7f0d0b614740 1 -- 172.21.15.47:0/4252846929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3e1e0 msgr2=0x564504a483c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.702+0000 7f0d0b614740 1 --2- 172.21.15.47:0/4252846929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3e1e0 0x564504a483c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4000c00 tx=0x7f0cf409b790 comp rx=0 tx=0).stop 2024-03-21T11:41:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.702+0000 7f0d0b614740 1 -- 172.21.15.47:0/4252846929 shutdown_connections 2024-03-21T11:41:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.702+0000 7f0d0b614740 1 --2- 172.21.15.47:0/4252846929 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564504a54150 0x564504a565a0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.702+0000 7f0d0b614740 1 --2- 172.21.15.47:0/4252846929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a43d40 0x564504a4f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.702+0000 7f0d0b614740 1 --2- 172.21.15.47:0/4252846929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a3ebd0 0x564504a48900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.702+0000 7f0ce3400640 0 client.31309 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.702+0000 7f0d0b614740 1 --2- 172.21.15.47:0/4252846929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3e1e0 0x564504a483c0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.702+0000 7f0d0b614740 1 -- 172.21.15.47:0/4252846929 >> 172.21.15.47:0/4252846929 conn(0x564504a39750 msgr2=0x564504a38950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.702+0000 7f0d0b614740 1 -- 172.21.15.47:0/4252846929 shutdown_connections 2024-03-21T11:41:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.703+0000 7f0d0b614740 1 -- 172.21.15.47:0/4252846929 wait complete. 2024-03-21T11:41:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.703+0000 7f0d0b614740 20 client.31309 trim_cache size 0 max 16384 2024-03-21T11:41:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.724+0000 7f0d0b614740 1 Processor -- start 2024-03-21T11:41:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.724+0000 7f0d0b614740 1 -- start start 2024-03-21T11:41:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d0b614740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3e1e0 0x564504a3d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d0b614740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a3ebd0 0x564504a4f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d0b614740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a43d40 0x564504a47fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d0b614740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d001a3a50 con 0x564504a3e1e0 2024-03-21T11:41:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d0b614740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d001a53d0 con 0x564504a3ebd0 2024-03-21T11:41:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d0b614740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d001a40b0 con 0x564504a43d40 2024-03-21T11:41:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d05400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a3ebd0 0x564504a4f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d05400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a3ebd0 0x564504a4f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60852/0 (socket says 172.21.15.47:60852) 2024-03-21T11:41:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d05400640 1 -- 172.21.15.47:0/1806134828 learned_addr learned my addr 172.21.15.47:0/1806134828 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d05e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3e1e0 0x564504a3d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d06800640 1 --2- 172.21.15.47:0/1806134828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a43d40 0x564504a47fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d05400640 1 -- 172.21.15.47:0/1806134828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a43d40 msgr2=0x564504a47fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d05400640 1 --2- 172.21.15.47:0/1806134828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a43d40 0x564504a47fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d05400640 1 -- 172.21.15.47:0/1806134828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3e1e0 msgr2=0x564504a3d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d05400640 1 --2- 172.21.15.47:0/1806134828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3e1e0 0x564504a3d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d05400640 1 -- 172.21.15.47:0/1806134828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564504a3efb0 con 0x564504a3ebd0 2024-03-21T11:41:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.725+0000 7f0d05e00640 1 --2- 172.21.15.47:0/1806134828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3e1e0 0x564504a3d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:41:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.726+0000 7f0d05400640 1 --2- 172.21.15.47:0/1806134828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a3ebd0 0x564504a4f890 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0cf002c250 tx=0x7f0cf002bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14cb74179f37ec46 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.726+0000 7f0cee600640 1 -- 172.21.15.47:0/1806134828 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0042040 con 0x564504a3ebd0 2024-03-21T11:41:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.726+0000 7f0cee600640 1 -- 172.21.15.47:0/1806134828 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cf002cb90 con 0x564504a3ebd0 2024-03-21T11:41:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.726+0000 7f0cee600640 1 -- 172.21.15.47:0/1806134828 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0004b50 con 0x564504a3ebd0 2024-03-21T11:41:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.727+0000 7f0d0b614740 1 -- 172.21.15.47:0/1806134828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a3ebd0 msgr2=0x564504a4f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.727+0000 7f0d0b614740 1 --2- 172.21.15.47:0/1806134828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a3ebd0 0x564504a4f890 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0cf002c250 tx=0x7f0cf002bc40 comp rx=0 tx=0).stop 2024-03-21T11:41:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.727+0000 7f0d0b614740 1 -- 172.21.15.47:0/1806134828 shutdown_connections 2024-03-21T11:41:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.727+0000 7f0d0b614740 1 --2- 172.21.15.47:0/1806134828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a43d40 0x564504a47fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.727+0000 7f0d0b614740 1 --2- 172.21.15.47:0/1806134828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a3ebd0 0x564504a4f890 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.727+0000 7f0d0b614740 1 --2- 172.21.15.47:0/1806134828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3e1e0 0x564504a3d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.727+0000 7f0d0b614740 1 -- 172.21.15.47:0/1806134828 >> 172.21.15.47:0/1806134828 conn(0x564504a39750 msgr2=0x564504a3c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.727+0000 7f0d0b614740 1 -- 172.21.15.47:0/1806134828 shutdown_connections 2024-03-21T11:41:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.727+0000 7f0d0b614740 1 -- 172.21.15.47:0/1806134828 wait complete. 2024-03-21T11:41:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.728+0000 7f0d0b614740 1 -- start start 2024-03-21T11:41:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.728+0000 7f0d0b614740 1 Processor -- start 2024-03-21T11:41:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.728+0000 7f0d0b614740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3e1e0 0x564504a4d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.728+0000 7f0d0b614740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3ebd0 0x564504a4dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.728+0000 7f0d0b614740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a43d40 0x564504a4ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.728+0000 7f0d0b614740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d001a4d70 con 0x564504a3ebd0 2024-03-21T11:41:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.728+0000 7f0d0b614740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d001a4710 con 0x564504a43d40 2024-03-21T11:41:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.728+0000 7f0d0b614740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d00073240 con 0x564504a3e1e0 2024-03-21T11:41:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.729+0000 7f0d0b614740 10 client.0 _ll_register_callbacks cb 0x7f0cf4009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.729+0000 7f0d06800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a43d40 0x564504a4ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.729+0000 7f0d06800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a43d40 0x564504a4ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60860/0 (socket says 172.21.15.47:60860) 2024-03-21T11:41:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.729+0000 7f0d06800640 1 -- 172.21.15.47:0/225440671 learned_addr learned my addr 172.21.15.47:0/225440671 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.729+0000 7f0d05e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3e1e0 0x564504a4d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.729+0000 7f0d06800640 1 -- 172.21.15.47:0/225440671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3e1e0 msgr2=0x564504a4d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.729+0000 7f0d06800640 1 --2- 172.21.15.47:0/225440671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3e1e0 0x564504a4d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.729+0000 7f0d06800640 1 -- 172.21.15.47:0/225440671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3ebd0 msgr2=0x564504a4dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.729+0000 7f0d06800640 1 --2- 172.21.15.47:0/225440671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3ebd0 0x564504a4dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.729+0000 7f0d06800640 1 -- 172.21.15.47:0/225440671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564504a4f300 con 0x564504a43d40 2024-03-21T11:41:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.731+0000 7f0d06800640 1 --2- 172.21.15.47:0/225440671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a43d40 0x564504a4ed10 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc012960 tx=0x7f0cfc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80e76b9caef88081 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.731+0000 7f0ce3400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:41:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.732+0000 7f0ce3400640 1 -- 172.21.15.47:0/225440671 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc035060 con 0x564504a43d40 2024-03-21T11:41:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.732+0000 7f0ce3400640 1 -- 172.21.15.47:0/225440671 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0cfc00fce0 con 0x564504a43d40 2024-03-21T11:41:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.732+0000 7f0d0b614740 1 -- 172.21.15.47:0/225440671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cf002cb90 con 0x564504a43d40 2024-03-21T11:41:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.732+0000 7f0ce3400640 1 -- 172.21.15.47:0/225440671 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cfc03c050 con 0x564504a43d40 2024-03-21T11:41:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.732+0000 7f0d0b614740 10 client.31331 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.732+0000 7f0d0b614740 1 -- 172.21.15.47:0/225440671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564504ab0260 con 0x564504a43d40 2024-03-21T11:41:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.733+0000 7f0ce3400640 1 -- 172.21.15.47:0/225440671 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cf002cb90 con 0x564504a43d40 2024-03-21T11:41:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.733+0000 7f0ce3400640 1 -- 172.21.15.47:0/225440671 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0cfc02f070 con 0x564504a43d40 2024-03-21T11:41:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.733+0000 7f0d0b614740 10 client.31331 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.733+0000 7f0d0b614740 10 client.31331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.733+0000 7f0d0b614740 20 client.31331 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.733+0000 7f0d0b614740 1 --2- 172.21.15.47:0/225440671 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564504ab0c70 0x564504ab3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.733+0000 7f0d0b614740 4 client.31331 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:41:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.733+0000 7f0d0b614740 1 -- 172.21.15.47:0/225440671 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5645049ab0c0 con 0x564504ab0c70 2024-03-21T11:41:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.733+0000 7f0d05e00640 1 --2- 172.21.15.47:0/225440671 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564504ab0c70 0x564504ab3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.734+0000 7f0d05e00640 1 --2- 172.21.15.47:0/225440671 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564504ab0c70 0x564504ab3090 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b4cc9a7af63a038 server_cookie=f66224627100bad6 in_seq=0 out_seq=0 2024-03-21T11:41:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:12.734+0000 7f0ce3400640 10 client.31331 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:12.947 INFO:tasks.workunit.client.0.smithi047.stdout: 10 110433 24.14 MB/sec execute 116 sec latency 229.919 ms 2024-03-21T11:41:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:12 smithi100 ceph-mon[36245]: pgmap v936: 129 pgs: 129 active+clean; 461 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 561 op/s 2024-03-21T11:41:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:12 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:41:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:12 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:41:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:12 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:41:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:12 smithi102 ceph-mon[36748]: pgmap v936: 129 pgs: 129 active+clean; 461 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 561 op/s 2024-03-21T11:41:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:12 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:41:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:12 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:41:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:12 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:41:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:12 smithi047 ceph-mon[31351]: pgmap v936: 129 pgs: 129 active+clean; 461 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 561 op/s 2024-03-21T11:41:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:12 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:41:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:12 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:41:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:12 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:41:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.319+0000 7f0ce3400640 1 -- 172.21.15.47:0/225440671 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5645049ab0c0 con 0x564504ab0c70 2024-03-21T11:41:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.319+0000 7f0ce3400640 10 client.31331 handle_command_reply: tid=0 2024-03-21T11:41:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.319+0000 7f0d0b614740 1 client.31331 shutdown 2024-03-21T11:41:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.319+0000 7f0d0b614740 1 -- 172.21.15.47:0/225440671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a43d40 msgr2=0x564504a4ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.319+0000 7f0d0b614740 1 --2- 172.21.15.47:0/225440671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a43d40 0x564504a4ed10 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc012960 tx=0x7f0cfc004a40 comp rx=0 tx=0).stop 2024-03-21T11:41:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.319+0000 7f0d0b614740 1 -- 172.21.15.47:0/225440671 shutdown_connections 2024-03-21T11:41:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.319+0000 7f0d0b614740 1 --2- 172.21.15.47:0/225440671 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564504ab0c70 0x564504ab3090 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.319+0000 7f0d0b614740 1 --2- 172.21.15.47:0/225440671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564504a43d40 0x564504a4ed10 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.319+0000 7f0d0b614740 1 --2- 172.21.15.47:0/225440671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564504a3ebd0 0x564504a4dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.319+0000 7f0d0b614740 1 --2- 172.21.15.47:0/225440671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564504a3e1e0 0x564504a4d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.319+0000 7f0d0b614740 1 -- 172.21.15.47:0/225440671 >> 172.21.15.47:0/225440671 conn(0x564504a39750 msgr2=0x564504a47fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.320+0000 7f0ce3400640 0 client.31331 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.320+0000 7f0d0b614740 1 -- 172.21.15.47:0/225440671 shutdown_connections 2024-03-21T11:41:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.320+0000 7f0d0b614740 1 -- 172.21.15.47:0/225440671 wait complete. 2024-03-21T11:41:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.320+0000 7f0d0b614740 20 client.31331 trim_cache size 0 max 16384 2024-03-21T11:41:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:41:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:41:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:41:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:41:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.321+0000 7f0d08800640 1 -- 172.21.15.47:0/1959447965 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0ce80a3040 msgr2=0x7f0ce80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.321+0000 7f0d08800640 1 --2- 172.21.15.47:0/1959447965 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0ce80a3040 0x7f0ce80a5500 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x564504a3f430 tx=0x7f0cfc009500 comp rx=0 tx=0).stop 2024-03-21T11:41:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.321+0000 7f0d08800640 1 -- 172.21.15.47:0/1959447965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d00102160 msgr2=0x7f0d00199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.321+0000 7f0d08800640 1 --2- 172.21.15.47:0/1959447965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d00102160 0x7f0d00199bc0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400e9f0 tx=0x7f0cf400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:41:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.321+0000 7f0d08800640 1 -- 172.21.15.47:0/1959447965 shutdown_connections 2024-03-21T11:41:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.321+0000 7f0d08800640 1 --2- 172.21.15.47:0/1959447965 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0ce80a3040 0x7f0ce80a5500 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.321+0000 7f0d08800640 1 --2- 172.21.15.47:0/1959447965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d001073c0 0x7f0d001a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.321+0000 7f0d08800640 1 --2- 172.21.15.47:0/1959447965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d00104a90 0x7f0d0019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.321+0000 7f0d08800640 1 --2- 172.21.15.47:0/1959447965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d00102160 0x7f0d00199bc0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.321+0000 7f0d08800640 1 -- 172.21.15.47:0/1959447965 >> 172.21.15.47:0/1959447965 conn(0x7f0d000fbb50 msgr2=0x7f0d001018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.322+0000 7f0d08800640 1 -- 172.21.15.47:0/1959447965 shutdown_connections 2024-03-21T11:41:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:13.322+0000 7f0d08800640 1 -- 172.21.15.47:0/1959447965 wait complete. 2024-03-21T11:41:13.947 INFO:tasks.workunit.client.0.smithi047.stdout: 10 110674 24.04 MB/sec execute 117 sec latency 565.311 ms 2024-03-21T11:41:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:13 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:41:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:13 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:41:14.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:13 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:41:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:13 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:41:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:13 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:41:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:13 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:41:14.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:13 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:41:14.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:13 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:41:14.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:13 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:41:14.947 INFO:tasks.workunit.client.0.smithi047.stdout: 10 111096 24.06 MB/sec execute 118 sec latency 310.547 ms 2024-03-21T11:41:15.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:14 smithi100 ceph-mon[36245]: pgmap v937: 129 pgs: 129 active+clean; 462 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.3 MiB/s wr, 505 op/s 2024-03-21T11:41:15.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:14 smithi102 ceph-mon[36748]: pgmap v937: 129 pgs: 129 active+clean; 462 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.3 MiB/s wr, 505 op/s 2024-03-21T11:41:15.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:14 smithi047 ceph-mon[31351]: pgmap v937: 129 pgs: 129 active+clean; 462 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.3 MiB/s wr, 505 op/s 2024-03-21T11:41:15.947 INFO:tasks.workunit.client.0.smithi047.stdout: 10 111451 24.02 MB/sec execute 119 sec latency 130.094 ms 2024-03-21T11:41:16.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:15 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:41:16.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:15 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:41:16.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:15 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:41:16.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:15 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:41:16.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:15 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:41:16.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:15 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:41:16.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:15 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:41:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:15 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:41:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:15 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:41:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:15 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:41:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:15 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:41:16.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:15 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:41:16.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:15 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:41:16.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:15 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:41:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:15 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:41:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:15 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:41:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:15 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:41:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:15 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T11:41:16.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:15 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:41:16.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:15 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:41:16.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:15 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:41:16.947 INFO:tasks.workunit.client.0.smithi047.stdout: 10 112050 24.10 MB/sec execute 120 sec latency 55.743 ms 2024-03-21T11:41:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:16 smithi100 ceph-mon[36245]: pgmap v938: 129 pgs: 129 active+clean; 462 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.7 MiB/s wr, 687 op/s 2024-03-21T11:41:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:16 smithi100 ceph-mon[36245]: Adjusting osd_memory_target on smithi102 to 174.1M 2024-03-21T11:41:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:16 smithi100 ceph-mon[36245]: Unable to set osd_memory_target on smithi102 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-03-21T11:41:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:16 smithi102 ceph-mon[36748]: pgmap v938: 129 pgs: 129 active+clean; 462 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.7 MiB/s wr, 687 op/s 2024-03-21T11:41:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:16 smithi102 ceph-mon[36748]: Adjusting osd_memory_target on smithi102 to 174.1M 2024-03-21T11:41:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:16 smithi102 ceph-mon[36748]: Unable to set osd_memory_target on smithi102 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-03-21T11:41:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:16 smithi047 ceph-mon[31351]: pgmap v938: 129 pgs: 129 active+clean; 462 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.7 MiB/s wr, 687 op/s 2024-03-21T11:41:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:16 smithi047 ceph-mon[31351]: Adjusting osd_memory_target on smithi102 to 174.1M 2024-03-21T11:41:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:16 smithi047 ceph-mon[31351]: Unable to set osd_memory_target on smithi102 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-03-21T11:41:17.947 INFO:tasks.workunit.client.0.smithi047.stdout: 10 112555 24.12 MB/sec execute 121 sec latency 47.261 ms 2024-03-21T11:41:18.947 INFO:tasks.workunit.client.0.smithi047.stdout: 10 112964 24.13 MB/sec execute 122 sec latency 163.420 ms 2024-03-21T11:41:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:18 smithi100 ceph-mon[36245]: pgmap v939: 129 pgs: 129 active+clean; 460 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.0 MiB/s wr, 563 op/s 2024-03-21T11:41:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:18 smithi102 ceph-mon[36748]: pgmap v939: 129 pgs: 129 active+clean; 460 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.0 MiB/s wr, 563 op/s 2024-03-21T11:41:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:18 smithi047 ceph-mon[31351]: pgmap v939: 129 pgs: 129 active+clean; 460 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.0 MiB/s wr, 563 op/s 2024-03-21T11:41:19.947 INFO:tasks.workunit.client.0.smithi047.stdout: 10 113364 24.11 MB/sec execute 123 sec latency 260.243 ms 2024-03-21T11:41:20.948 INFO:tasks.workunit.client.0.smithi047.stdout: 10 113711 24.03 MB/sec execute 124 sec latency 98.681 ms 2024-03-21T11:41:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:20 smithi100 ceph-mon[36245]: pgmap v940: 129 pgs: 129 active+clean; 459 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.7 MiB/s wr, 691 op/s 2024-03-21T11:41:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:20 smithi102 ceph-mon[36748]: pgmap v940: 129 pgs: 129 active+clean; 459 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.7 MiB/s wr, 691 op/s 2024-03-21T11:41:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:20 smithi047 ceph-mon[31351]: pgmap v940: 129 pgs: 129 active+clean; 459 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.7 MiB/s wr, 691 op/s 2024-03-21T11:41:21.948 INFO:tasks.workunit.client.0.smithi047.stdout: 10 114357 24.11 MB/sec execute 125 sec latency 82.293 ms 2024-03-21T11:41:22.948 INFO:tasks.workunit.client.0.smithi047.stdout: 10 114876 24.17 MB/sec execute 126 sec latency 98.429 ms 2024-03-21T11:41:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:22 smithi100 ceph-mon[36245]: pgmap v941: 129 pgs: 129 active+clean; 461 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 559 op/s 2024-03-21T11:41:23.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:22 smithi102 ceph-mon[36748]: pgmap v941: 129 pgs: 129 active+clean; 461 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 559 op/s 2024-03-21T11:41:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:22 smithi047 ceph-mon[31351]: pgmap v941: 129 pgs: 129 active+clean; 461 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.3 MiB/s wr, 559 op/s 2024-03-21T11:41:23.948 INFO:tasks.workunit.client.0.smithi047.stdout: 10 115407 24.21 MB/sec execute 127 sec latency 83.960 ms 2024-03-21T11:41:24.948 INFO:tasks.workunit.client.0.smithi047.stdout: 10 115675 24.14 MB/sec execute 128 sec latency 286.026 ms 2024-03-21T11:41:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:24 smithi102 ceph-mon[36748]: pgmap v942: 129 pgs: 129 active+clean; 463 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 574 op/s 2024-03-21T11:41:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:24 smithi047 ceph-mon[31351]: pgmap v942: 129 pgs: 129 active+clean; 463 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 574 op/s 2024-03-21T11:41:25.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:24 smithi100 ceph-mon[36245]: pgmap v942: 129 pgs: 129 active+clean; 463 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 6.2 MiB/s wr, 574 op/s 2024-03-21T11:41:25.948 INFO:tasks.workunit.client.0.smithi047.stdout: 10 116108 24.10 MB/sec execute 129 sec latency 153.029 ms 2024-03-21T11:41:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.508+0000 7ffb3ae00640 1 -- 172.21.15.47:0/2900485570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb34109380 msgr2=0x7ffb3410b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.508+0000 7ffb3ae00640 1 --2- 172.21.15.47:0/2900485570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb34109380 0x7ffb3410b820 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009ca0 tx=0x7ffb2c02c180 comp rx=0 tx=0).stop 2024-03-21T11:41:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.508+0000 7ffb3ae00640 1 -- 172.21.15.47:0/2900485570 shutdown_connections 2024-03-21T11:41:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.508+0000 7ffb3ae00640 1 --2- 172.21.15.47:0/2900485570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb34109380 0x7ffb3410b820 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.508+0000 7ffb3ae00640 1 --2- 172.21.15.47:0/2900485570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb340740a0 0x7ffb34074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.508+0000 7ffb3ae00640 1 --2- 172.21.15.47:0/2900485570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb34075680 0x7ffb34073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.508+0000 7ffb3ae00640 1 -- 172.21.15.47:0/2900485570 >> 172.21.15.47:0/2900485570 conn(0x7ffb340fca80 msgr2=0x7ffb340feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.508+0000 7ffb3ae00640 1 -- 172.21.15.47:0/2900485570 shutdown_connections 2024-03-21T11:41:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.508+0000 7ffb3ae00640 1 -- 172.21.15.47:0/2900485570 wait complete. 2024-03-21T11:41:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.510+0000 7ffb3ae00640 1 Processor -- start 2024-03-21T11:41:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.510+0000 7ffb3ae00640 1 -- start start 2024-03-21T11:41:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.510+0000 7ffb3ae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb340740a0 0x7ffb34199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.510+0000 7ffb3ae00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb34075680 0x7ffb34199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.510+0000 7ffb3ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb34109380 0x7ffb341a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.510+0000 7ffb3ae00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3410f350 con 0x7ffb34075680 2024-03-21T11:41:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.510+0000 7ffb3ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3410f1d0 con 0x7ffb34109380 2024-03-21T11:41:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.510+0000 7ffb3ae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3410f4d0 con 0x7ffb340740a0 2024-03-21T11:41:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.511+0000 7ffb39a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb340740a0 0x7ffb34199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.511+0000 7ffb39a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb340740a0 0x7ffb34199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51188/0 (socket says 172.21.15.47:51188) 2024-03-21T11:41:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.511+0000 7ffb39a00640 1 -- 172.21.15.47:0/1185860764 learned_addr learned my addr 172.21.15.47:0/1185860764 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.511+0000 7ffb3a400640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb34109380 0x7ffb341a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.511+0000 7ffb39000640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb34075680 0x7ffb34199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.512+0000 7ffb3a400640 1 -- 172.21.15.47:0/1185860764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb340740a0 msgr2=0x7ffb34199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.512+0000 7ffb3a400640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb340740a0 0x7ffb34199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.512+0000 7ffb3a400640 1 -- 172.21.15.47:0/1185860764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb34075680 msgr2=0x7ffb34199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.512+0000 7ffb3a400640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb34075680 0x7ffb34199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.512+0000 7ffb3a400640 1 -- 172.21.15.47:0/1185860764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb341a15a0 con 0x7ffb34109380 2024-03-21T11:41:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.512+0000 7ffb39a00640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb340740a0 0x7ffb34199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:41:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.513+0000 7ffb3a400640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb34109380 0x7ffb341a0f30 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c02c6c0 tx=0x7ffb2c003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.513+0000 7ffb32000640 1 -- 172.21.15.47:0/1185860764 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c039040 con 0x7ffb34109380 2024-03-21T11:41:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.513+0000 7ffb32000640 1 -- 172.21.15.47:0/1185860764 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb2c044070 con 0x7ffb34109380 2024-03-21T11:41:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.513+0000 7ffb32000640 1 -- 172.21.15.47:0/1185860764 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0043a0 con 0x7ffb34109380 2024-03-21T11:41:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.514+0000 7ffb3ae00640 1 -- 172.21.15.47:0/1185860764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb341a1830 con 0x7ffb34109380 2024-03-21T11:41:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.514+0000 7ffb3ae00640 1 -- 172.21.15.47:0/1185860764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb341a1ca0 con 0x7ffb34109380 2024-03-21T11:41:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.519+0000 7ffb32000640 1 -- 172.21.15.47:0/1185860764 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ffb2c004560 con 0x7ffb34109380 2024-03-21T11:41:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.521+0000 7ffb3f070740 1 Processor -- start 2024-03-21T11:41:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.521+0000 7ffb3f070740 1 -- start start 2024-03-21T11:41:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.521+0000 7ffb32000640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb1c0a3010 0x7ffb1c0a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.522+0000 7ffb3f070740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d993f0 0x562207d99810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.522+0000 7ffb32000640 1 -- 172.21.15.47:0/1185860764 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffb2c0f1e70 con 0x7ffb34109380 2024-03-21T11:41:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.522+0000 7ffb3f070740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d99de0 0x562207d9e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.522+0000 7ffb3f070740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d9ef50 0x562207da13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.522+0000 7ffb3f070740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562207d5c3b0 con 0x562207d9ef50 2024-03-21T11:41:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.522+0000 7ffb3f070740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562207cfe830 con 0x562207d99de0 2024-03-21T11:41:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.522+0000 7ffb3f070740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562207ca75d0 con 0x562207d993f0 2024-03-21T11:41:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.522+0000 7ffb39a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d993f0 0x562207d99810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.522+0000 7ffb39a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d993f0 0x562207d99810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51200/0 (socket says 172.21.15.47:51200) 2024-03-21T11:41:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.522+0000 7ffb39a00640 1 -- 172.21.15.47:0/2759183239 learned_addr learned my addr 172.21.15.47:0/2759183239 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.522+0000 7ffb3a400640 1 --2- 172.21.15.47:0/2759183239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d9ef50 0x562207da13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.523+0000 7ffb39000640 1 --2- 172.21.15.47:0/2759183239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d99de0 0x562207d9e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.523+0000 7ffb39a00640 1 -- 172.21.15.47:0/2759183239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d99de0 msgr2=0x562207d9e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.523+0000 7ffb39a00640 1 --2- 172.21.15.47:0/2759183239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d99de0 0x562207d9e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.523+0000 7ffb39000640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb1c0a3010 0x7ffb1c0a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.523+0000 7ffb39a00640 1 -- 172.21.15.47:0/2759183239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d9ef50 msgr2=0x562207da13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.523+0000 7ffb39a00640 1 --2- 172.21.15.47:0/2759183239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d9ef50 0x562207da13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.523+0000 7ffb39a00640 1 -- 172.21.15.47:0/2759183239 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562207cbf2c0 con 0x562207d993f0 2024-03-21T11:41:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.523+0000 7ffb39a00640 1 --2- 172.21.15.47:0/2759183239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d993f0 0x562207d99810 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ffb28010580 tx=0x7ffb28010a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35a68b033b47b3d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.524+0000 7ffb26a00640 1 -- 172.21.15.47:0/2759183239 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2800e890 con 0x562207d993f0 2024-03-21T11:41:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.524+0000 7ffb39000640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb1c0a3010 0x7ffb1c0a54d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x562207d9a1e0 tx=0x7ffb20005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.524+0000 7ffb26a00640 1 -- 172.21.15.47:0/2759183239 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb2801c480 con 0x562207d993f0 2024-03-21T11:41:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.524+0000 7ffb26a00640 1 -- 172.21.15.47:0/2759183239 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280116a0 con 0x562207d993f0 2024-03-21T11:41:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.525+0000 7ffb3f070740 1 -- 172.21.15.47:0/2759183239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d993f0 msgr2=0x562207d99810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.525+0000 7ffb3f070740 1 --2- 172.21.15.47:0/2759183239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d993f0 0x562207d99810 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ffb28010580 tx=0x7ffb28010a40 comp rx=0 tx=0).stop 2024-03-21T11:41:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.525+0000 7ffb3f070740 1 -- 172.21.15.47:0/2759183239 shutdown_connections 2024-03-21T11:41:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.525+0000 7ffb3f070740 1 --2- 172.21.15.47:0/2759183239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d9ef50 0x562207da13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.525+0000 7ffb3f070740 1 --2- 172.21.15.47:0/2759183239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d99de0 0x562207d9e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.525+0000 7ffb3f070740 1 --2- 172.21.15.47:0/2759183239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d993f0 0x562207d99810 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.525+0000 7ffb3f070740 1 -- 172.21.15.47:0/2759183239 >> 172.21.15.47:0/2759183239 conn(0x562207d94960 msgr2=0x562207d94d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.525+0000 7ffb3f070740 1 -- 172.21.15.47:0/2759183239 shutdown_connections 2024-03-21T11:41:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.525+0000 7ffb3f070740 1 -- 172.21.15.47:0/2759183239 wait complete. 2024-03-21T11:41:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.526+0000 7ffb3f070740 1 -- start start 2024-03-21T11:41:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.526+0000 7ffb3f070740 1 Processor -- start 2024-03-21T11:41:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.527+0000 7ffb3f070740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d993f0 0x562207da3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.527+0000 7ffb3f070740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d99de0 0x562207da3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.527+0000 7ffb3f070740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d9ef50 0x562207daab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.527+0000 7ffb3f070740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562207cbaa20 con 0x562207d99de0 2024-03-21T11:41:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.527+0000 7ffb3f070740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562207cfa8d0 con 0x562207d9ef50 2024-03-21T11:41:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.527+0000 7ffb3f070740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562207da1f00 con 0x562207d993f0 2024-03-21T11:41:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.527+0000 7ffb3f070740 10 client.0 _ll_register_callbacks cb 0x562207cf71c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.528+0000 7ffb39000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d99de0 0x562207da3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.528+0000 7ffb39000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d99de0 0x562207da3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54130/0 (socket says 172.21.15.47:54130) 2024-03-21T11:41:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.528+0000 7ffb39000640 1 -- 172.21.15.47:0/70683145 learned_addr learned my addr 172.21.15.47:0/70683145 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.529+0000 7ffb39a00640 1 --2- 172.21.15.47:0/70683145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d993f0 0x562207da3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.529+0000 7ffb39000640 1 -- 172.21.15.47:0/70683145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d993f0 msgr2=0x562207da3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.529+0000 7ffb39000640 1 --2- 172.21.15.47:0/70683145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d993f0 0x562207da3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.529+0000 7ffb39000640 1 -- 172.21.15.47:0/70683145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d9ef50 msgr2=0x562207daab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.529+0000 7ffb3a400640 1 --2- 172.21.15.47:0/70683145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d9ef50 0x562207daab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.529+0000 7ffb39000640 1 --2- 172.21.15.47:0/70683145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d9ef50 0x562207daab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.529+0000 7ffb39000640 1 -- 172.21.15.47:0/70683145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562207a13210 con 0x562207d99de0 2024-03-21T11:41:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.529+0000 7ffb3a400640 1 --2- 172.21.15.47:0/70683145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d9ef50 0x562207daab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:41:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.531+0000 7ffb39000640 1 --2- 172.21.15.47:0/70683145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d99de0 0x562207da3b50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000fc40 tx=0x7ffb20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bafac178dd7fa4f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.531+0000 7ffb1b400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:41:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.531+0000 7ffb1b400640 1 -- 172.21.15.47:0/70683145 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20023070 con 0x562207d99de0 2024-03-21T11:41:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.531+0000 7ffb1b400640 1 -- 172.21.15.47:0/70683145 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffb2001f400 con 0x562207d99de0 2024-03-21T11:41:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.532+0000 7ffb3f070740 1 -- 172.21.15.47:0/70683145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb2801c480 con 0x562207d99de0 2024-03-21T11:41:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.532+0000 7ffb1b400640 1 -- 172.21.15.47:0/70683145 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb2002d050 con 0x562207d99de0 2024-03-21T11:41:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.532+0000 7ffb3f070740 10 client.20802 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.532+0000 7ffb3f070740 1 -- 172.21.15.47:0/70683145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562207adf620 con 0x562207d99de0 2024-03-21T11:41:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.532+0000 7ffb1b400640 1 -- 172.21.15.47:0/70683145 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb2801c480 con 0x562207d99de0 2024-03-21T11:41:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.533+0000 7ffb1b400640 1 -- 172.21.15.47:0/70683145 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffb2001e040 con 0x562207d99de0 2024-03-21T11:41:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.533+0000 7ffb3f070740 10 client.20802 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.533+0000 7ffb3f070740 10 client.20802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.533+0000 7ffb3f070740 20 client.20802 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.533+0000 7ffb3f070740 1 --2- 172.21.15.47:0/70683145 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562207daf4d0 0x562207db1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.533+0000 7ffb3f070740 4 client.20802 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:41:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.533+0000 7ffb3f070740 1 -- 172.21.15.47:0/70683145 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562207d0bd70 con 0x562207daf4d0 2024-03-21T11:41:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.533+0000 7ffb39a00640 1 --2- 172.21.15.47:0/70683145 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562207daf4d0 0x562207db1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.534+0000 7ffb39a00640 1 --2- 172.21.15.47:0/70683145 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562207daf4d0 0x562207db1900 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185441a0a5f0b626 server_cookie=2108aaf37bfcaa12 in_seq=0 out_seq=0 2024-03-21T11:41:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.534+0000 7ffb1b400640 10 client.20802 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.537+0000 7ffb1b400640 1 -- 172.21.15.47:0/70683145 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562207d0bd70 con 0x562207daf4d0 2024-03-21T11:41:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.537+0000 7ffb1b400640 10 client.20802 handle_command_reply: tid=0 2024-03-21T11:41:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.537+0000 7ffb3f070740 1 client.20802 shutdown 2024-03-21T11:41:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.538+0000 7ffb3f070740 1 -- 172.21.15.47:0/70683145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d99de0 msgr2=0x562207da3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.538+0000 7ffb3f070740 1 --2- 172.21.15.47:0/70683145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d99de0 0x562207da3b50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000fc40 tx=0x7ffb20004730 comp rx=0 tx=0).stop 2024-03-21T11:41:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.538+0000 7ffb3f070740 1 -- 172.21.15.47:0/70683145 shutdown_connections 2024-03-21T11:41:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.538+0000 7ffb3f070740 1 --2- 172.21.15.47:0/70683145 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562207daf4d0 0x562207db1900 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.538+0000 7ffb3f070740 1 --2- 172.21.15.47:0/70683145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d9ef50 0x562207daab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.538+0000 7ffb3f070740 1 --2- 172.21.15.47:0/70683145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d99de0 0x562207da3b50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.538+0000 7ffb3f070740 1 --2- 172.21.15.47:0/70683145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d993f0 0x562207da3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.538+0000 7ffb3f070740 1 -- 172.21.15.47:0/70683145 >> 172.21.15.47:0/70683145 conn(0x562207d94960 msgr2=0x562207d93b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.538+0000 7ffb1b400640 0 client.20802 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.538+0000 7ffb3f070740 1 -- 172.21.15.47:0/70683145 shutdown_connections 2024-03-21T11:41:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.538+0000 7ffb3f070740 1 -- 172.21.15.47:0/70683145 wait complete. 2024-03-21T11:41:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.538+0000 7ffb3f070740 20 client.20802 trim_cache size 0 max 16384 2024-03-21T11:41:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.559+0000 7ffb3f070740 1 Processor -- start 2024-03-21T11:41:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.560+0000 7ffb3f070740 1 -- start start 2024-03-21T11:41:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.560+0000 7ffb3f070740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d993f0 0x562207d9b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.560+0000 7ffb3f070740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d99de0 0x562207d9b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.560+0000 7ffb3f070740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d9ef50 0x562207daab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.560+0000 7ffb3f070740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb341ac050 con 0x562207d993f0 2024-03-21T11:41:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.560+0000 7ffb3f070740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb341ad9d0 con 0x562207d9ef50 2024-03-21T11:41:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.560+0000 7ffb3f070740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb341ac6b0 con 0x562207d99de0 2024-03-21T11:41:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.560+0000 7ffb39000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d99de0 0x562207d9b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.560+0000 7ffb3a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d9ef50 0x562207daab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.560+0000 7ffb39a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d993f0 0x562207d9b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.560+0000 7ffb39000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d99de0 0x562207d9b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51222/0 (socket says 172.21.15.47:51222) 2024-03-21T11:41:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.560+0000 7ffb39000640 1 -- 172.21.15.47:0/611318342 learned_addr learned my addr 172.21.15.47:0/611318342 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.561+0000 7ffb39000640 1 -- 172.21.15.47:0/611318342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d9ef50 msgr2=0x562207daab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.561+0000 7ffb39000640 1 --2- 172.21.15.47:0/611318342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d9ef50 0x562207daab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.561+0000 7ffb39000640 1 -- 172.21.15.47:0/611318342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d993f0 msgr2=0x562207d9b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.561+0000 7ffb39000640 1 --2- 172.21.15.47:0/611318342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d993f0 0x562207d9b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.561+0000 7ffb39000640 1 -- 172.21.15.47:0/611318342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562207d997d0 con 0x562207d99de0 2024-03-21T11:41:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.561+0000 7ffb39a00640 1 --2- 172.21.15.47:0/611318342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d993f0 0x562207d9b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:41:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.561+0000 7ffb39000640 1 --2- 172.21.15.47:0/611318342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d99de0 0x562207d9b680 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000c640 tx=0x7ffb2000c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67596c3aa23ceaaa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.562+0000 7ffb26000640 1 -- 172.21.15.47:0/611318342 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20023050 con 0x562207d99de0 2024-03-21T11:41:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.562+0000 7ffb26000640 1 -- 172.21.15.47:0/611318342 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb20028890 con 0x562207d99de0 2024-03-21T11:41:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.562+0000 7ffb26000640 1 -- 172.21.15.47:0/611318342 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20018040 con 0x562207d99de0 2024-03-21T11:41:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.563+0000 7ffb3f070740 1 -- 172.21.15.47:0/611318342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d99de0 msgr2=0x562207d9b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.563+0000 7ffb3f070740 1 --2- 172.21.15.47:0/611318342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d99de0 0x562207d9b680 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000c640 tx=0x7ffb2000c790 comp rx=0 tx=0).stop 2024-03-21T11:41:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.564+0000 7ffb3f070740 1 -- 172.21.15.47:0/611318342 shutdown_connections 2024-03-21T11:41:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.564+0000 7ffb3f070740 1 --2- 172.21.15.47:0/611318342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d9ef50 0x562207daab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.564+0000 7ffb3f070740 1 --2- 172.21.15.47:0/611318342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d99de0 0x562207d9b680 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.564+0000 7ffb3f070740 1 --2- 172.21.15.47:0/611318342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d993f0 0x562207d9b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.564+0000 7ffb3f070740 1 -- 172.21.15.47:0/611318342 >> 172.21.15.47:0/611318342 conn(0x562207d94960 msgr2=0x562207d94d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.564+0000 7ffb3f070740 1 -- 172.21.15.47:0/611318342 shutdown_connections 2024-03-21T11:41:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.564+0000 7ffb3f070740 1 -- 172.21.15.47:0/611318342 wait complete. 2024-03-21T11:41:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.565+0000 7ffb3f070740 1 -- start start 2024-03-21T11:41:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.565+0000 7ffb3f070740 1 Processor -- start 2024-03-21T11:41:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.565+0000 7ffb3f070740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d993f0 0x562207da8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.566+0000 7ffb3f070740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d99de0 0x562207da87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.566+0000 7ffb3f070740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d9ef50 0x562207da97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.566+0000 7ffb3f070740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb341ad370 con 0x562207d993f0 2024-03-21T11:41:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.566+0000 7ffb3f070740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb341acd10 con 0x562207d99de0 2024-03-21T11:41:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.566+0000 7ffb3f070740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb341aa210 con 0x562207d9ef50 2024-03-21T11:41:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.566+0000 7ffb3f070740 10 client.0 _ll_register_callbacks cb 0x7ffb2000c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.566+0000 7ffb39a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d993f0 0x562207da8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.566+0000 7ffb39000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d99de0 0x562207da87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.566+0000 7ffb39000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d99de0 0x562207da87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47308/0 (socket says 172.21.15.47:47308) 2024-03-21T11:41:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.567+0000 7ffb39000640 1 -- 172.21.15.47:0/557127845 learned_addr learned my addr 172.21.15.47:0/557127845 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.567+0000 7ffb3a400640 1 --2- 172.21.15.47:0/557127845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d9ef50 0x562207da97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.567+0000 7ffb39000640 1 -- 172.21.15.47:0/557127845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d9ef50 msgr2=0x562207da97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.567+0000 7ffb39000640 1 --2- 172.21.15.47:0/557127845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d9ef50 0x562207da97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.567+0000 7ffb39000640 1 -- 172.21.15.47:0/557127845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d993f0 msgr2=0x562207da8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.567+0000 7ffb39000640 1 --2- 172.21.15.47:0/557127845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d993f0 0x562207da8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.567+0000 7ffb39000640 1 -- 172.21.15.47:0/557127845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562207da9dd0 con 0x562207d99de0 2024-03-21T11:41:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.568+0000 7ffb39000640 1 --2- 172.21.15.47:0/557127845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d99de0 0x562207da87d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ffb20028860 tx=0x7ffb200284b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30179a2afab7460f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.568+0000 7ffb1b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:41:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.569+0000 7ffb1b400640 1 -- 172.21.15.47:0/557127845 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20018030 con 0x562207d99de0 2024-03-21T11:41:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.569+0000 7ffb1b400640 1 -- 172.21.15.47:0/557127845 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffb20005040 con 0x562207d99de0 2024-03-21T11:41:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.569+0000 7ffb3f070740 1 -- 172.21.15.47:0/557127845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb20028890 con 0x562207d99de0 2024-03-21T11:41:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.569+0000 7ffb1b400640 1 -- 172.21.15.47:0/557127845 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb20023040 con 0x562207d99de0 2024-03-21T11:41:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.570+0000 7ffb3f070740 10 client.31346 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.570+0000 7ffb3f070740 1 -- 172.21.15.47:0/557127845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562207e0add0 con 0x562207d99de0 2024-03-21T11:41:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.571+0000 7ffb1b400640 1 -- 172.21.15.47:0/557127845 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb20028890 con 0x562207d99de0 2024-03-21T11:41:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.571+0000 7ffb1b400640 1 -- 172.21.15.47:0/557127845 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffb2001e040 con 0x562207d99de0 2024-03-21T11:41:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.571+0000 7ffb3f070740 10 client.31346 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.571+0000 7ffb3f070740 10 client.31346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.571+0000 7ffb3f070740 20 client.31346 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.571+0000 7ffb3f070740 1 --2- 172.21.15.47:0/557127845 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562207e0b520 0x562207e0d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.571+0000 7ffb3f070740 4 client.31346 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:41:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.571+0000 7ffb3f070740 1 -- 172.21.15.47:0/557127845 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562207d060c0 con 0x562207e0b520 2024-03-21T11:41:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.572+0000 7ffb39a00640 1 --2- 172.21.15.47:0/557127845 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562207e0b520 0x562207e0d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.572+0000 7ffb39a00640 1 --2- 172.21.15.47:0/557127845 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562207e0b520 0x562207e0d940 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d38b38b56459ea2 server_cookie=10f29677891ef814 in_seq=0 out_seq=0 2024-03-21T11:41:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:26.572+0000 7ffb1b400640 10 client.31346 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:26.948 INFO:tasks.workunit.client.0.smithi047.stdout: 10 116499 24.13 MB/sec execute 130 sec latency 277.331 ms 2024-03-21T11:41:27.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:26 smithi102 ceph-mon[36748]: pgmap v943: 129 pgs: 129 active+clean; 468 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.9 MiB/s wr, 693 op/s 2024-03-21T11:41:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.255+0000 7ffb1b400640 1 -- 172.21.15.47:0/557127845 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562207d060c0 con 0x562207e0b520 2024-03-21T11:41:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.255+0000 7ffb1b400640 10 client.31346 handle_command_reply: tid=0 2024-03-21T11:41:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.255+0000 7ffb3f070740 1 client.31346 shutdown 2024-03-21T11:41:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.257+0000 7ffb3f070740 1 -- 172.21.15.47:0/557127845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d99de0 msgr2=0x562207da87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.257+0000 7ffb3f070740 1 --2- 172.21.15.47:0/557127845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d99de0 0x562207da87d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ffb20028860 tx=0x7ffb200284b0 comp rx=0 tx=0).stop 2024-03-21T11:41:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.257+0000 7ffb3f070740 1 -- 172.21.15.47:0/557127845 shutdown_connections 2024-03-21T11:41:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.257+0000 7ffb3f070740 1 --2- 172.21.15.47:0/557127845 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562207e0b520 0x562207e0d940 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.257+0000 7ffb3f070740 1 --2- 172.21.15.47:0/557127845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562207d9ef50 0x562207da97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.257+0000 7ffb3f070740 1 --2- 172.21.15.47:0/557127845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562207d99de0 0x562207da87d0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.257+0000 7ffb3f070740 1 --2- 172.21.15.47:0/557127845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562207d993f0 0x562207da8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.257+0000 7ffb1b400640 0 client.31346 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.257+0000 7ffb3f070740 1 -- 172.21.15.47:0/557127845 >> 172.21.15.47:0/557127845 conn(0x562207d94960 msgr2=0x562207da35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.258+0000 7ffb3f070740 1 -- 172.21.15.47:0/557127845 shutdown_connections 2024-03-21T11:41:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.258+0000 7ffb3f070740 1 -- 172.21.15.47:0/557127845 wait complete. 2024-03-21T11:41:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.258+0000 7ffb3f070740 20 client.31346 trim_cache size 0 max 16384 2024-03-21T11:41:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:41:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:41:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:41:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:41:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.260+0000 7ffb3ae00640 1 -- 172.21.15.47:0/1185860764 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb1c0a3010 msgr2=0x7ffb1c0a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.260+0000 7ffb3ae00640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb1c0a3010 0x7ffb1c0a54d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x562207d9a1e0 tx=0x7ffb20005850 comp rx=0 tx=0).stop 2024-03-21T11:41:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.260+0000 7ffb3ae00640 1 -- 172.21.15.47:0/1185860764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb34109380 msgr2=0x7ffb341a0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.260+0000 7ffb3ae00640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb34109380 0x7ffb341a0f30 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c02c6c0 tx=0x7ffb2c003c20 comp rx=0 tx=0).stop 2024-03-21T11:41:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.261+0000 7ffb3ae00640 1 -- 172.21.15.47:0/1185860764 shutdown_connections 2024-03-21T11:41:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.261+0000 7ffb3ae00640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb1c0a3010 0x7ffb1c0a54d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.261+0000 7ffb3ae00640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb34109380 0x7ffb341a0f30 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.261+0000 7ffb3ae00640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb34075680 0x7ffb34199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.261+0000 7ffb3ae00640 1 --2- 172.21.15.47:0/1185860764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb340740a0 0x7ffb34199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.261+0000 7ffb3ae00640 1 -- 172.21.15.47:0/1185860764 >> 172.21.15.47:0/1185860764 conn(0x7ffb340fca80 msgr2=0x7ffb34105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.261+0000 7ffb3ae00640 1 -- 172.21.15.47:0/1185860764 shutdown_connections 2024-03-21T11:41:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:27.261+0000 7ffb3ae00640 1 -- 172.21.15.47:0/1185860764 wait complete. 2024-03-21T11:41:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:26 smithi047 ceph-mon[31351]: pgmap v943: 129 pgs: 129 active+clean; 468 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.9 MiB/s wr, 693 op/s 2024-03-21T11:41:27.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:26 smithi100 ceph-mon[36245]: pgmap v943: 129 pgs: 129 active+clean; 468 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.9 MiB/s wr, 693 op/s 2024-03-21T11:41:27.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:41:27.837 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:41:27.948 INFO:tasks.workunit.client.0.smithi047.stdout: 10 117025 24.14 MB/sec execute 131 sec latency 488.476 ms 2024-03-21T11:41:28.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.055+0000 7fb45bc00640 1 -- 172.21.15.47:0/1858503661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4541038c0 msgr2=0x7fb454103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.055+0000 7fb45bc00640 1 --2- 172.21.15.47:0/1858503661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4541038c0 0x7fb454103ce0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fb448009900 tx=0x7fb44802bc30 comp rx=0 tx=0).stop 2024-03-21T11:41:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.055+0000 7fb45bc00640 1 -- 172.21.15.47:0/1858503661 shutdown_connections 2024-03-21T11:41:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.055+0000 7fb45bc00640 1 --2- 172.21.15.47:0/1858503661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb454105470 0x7fb45410bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.055+0000 7fb45bc00640 1 --2- 172.21.15.47:0/1858503661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb454104ab0 0x7fb454104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.055+0000 7fb45bc00640 1 --2- 172.21.15.47:0/1858503661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4541038c0 0x7fb454103ce0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.055+0000 7fb45bc00640 1 -- 172.21.15.47:0/1858503661 >> 172.21.15.47:0/1858503661 conn(0x7fb4540fee50 msgr2=0x7fb454101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.055+0000 7fb45bc00640 1 -- 172.21.15.47:0/1858503661 shutdown_connections 2024-03-21T11:41:28.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.056+0000 7fb45bc00640 1 -- 172.21.15.47:0/1858503661 wait complete. 2024-03-21T11:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.056+0000 7fb45bc00640 1 Processor -- start 2024-03-21T11:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.057+0000 7fb45bc00640 1 -- start start 2024-03-21T11:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.057+0000 7fb45bc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4541038c0 0x7fb454199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.057+0000 7fb45bc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb454104ab0 0x7fb45419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.057+0000 7fb45bc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb454105470 0x7fb4541a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.057+0000 7fb45bc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45410f590 con 0x7fb454105470 2024-03-21T11:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.057+0000 7fb45bc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45410f410 con 0x7fb454104ab0 2024-03-21T11:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.057+0000 7fb45bc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45410f710 con 0x7fb4541038c0 2024-03-21T11:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.057+0000 7fb459200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4541038c0 0x7fb454199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.057+0000 7fb459200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4541038c0 0x7fb454199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51252/0 (socket says 172.21.15.47:51252) 2024-03-21T11:41:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.057+0000 7fb459200640 1 -- 172.21.15.47:0/127438485 learned_addr learned my addr 172.21.15.47:0/127438485 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.057+0000 7fb453e00640 1 --2- 172.21.15.47:0/127438485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb454104ab0 0x7fb45419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.057+0000 7fb459c00640 1 --2- 172.21.15.47:0/127438485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb454105470 0x7fb4541a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.058+0000 7fb459200640 1 -- 172.21.15.47:0/127438485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb454104ab0 msgr2=0x7fb45419a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.058+0000 7fb459200640 1 --2- 172.21.15.47:0/127438485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb454104ab0 0x7fb45419a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.058+0000 7fb459200640 1 -- 172.21.15.47:0/127438485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb454105470 msgr2=0x7fb4541a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.058+0000 7fb459200640 1 --2- 172.21.15.47:0/127438485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb454105470 0x7fb4541a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.058+0000 7fb459200640 1 -- 172.21.15.47:0/127438485 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4541a17d0 con 0x7fb4541038c0 2024-03-21T11:41:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.058+0000 7fb459200640 1 --2- 172.21.15.47:0/127438485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4541038c0 0x7fb454199bb0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb448000c00 tx=0x7fb44802c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.059+0000 7fb451600640 1 -- 172.21.15.47:0/127438485 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448043020 con 0x7fb4541038c0 2024-03-21T11:41:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.059+0000 7fb451600640 1 -- 172.21.15.47:0/127438485 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4480054e0 con 0x7fb4541038c0 2024-03-21T11:41:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.059+0000 7fb451600640 1 -- 172.21.15.47:0/127438485 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448003ce0 con 0x7fb4541038c0 2024-03-21T11:41:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.060+0000 7fb45bc00640 1 -- 172.21.15.47:0/127438485 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4541a1a60 con 0x7fb4541038c0 2024-03-21T11:41:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.060+0000 7fb45bc00640 1 -- 172.21.15.47:0/127438485 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4541a1f30 con 0x7fb4541038c0 2024-03-21T11:41:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.063+0000 7fb451600640 1 -- 172.21.15.47:0/127438485 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb448046020 con 0x7fb4541038c0 2024-03-21T11:41:28.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.064+0000 7fb45e977740 1 Processor -- start 2024-03-21T11:41:28.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.064+0000 7fb45e977740 1 -- start start 2024-03-21T11:41:28.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.064+0000 7fb45e977740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cae80 0x55fc114cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.064+0000 7fb45e977740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cb870 0x55fc114d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.064+0000 7fb45e977740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114d09e0 0x55fc114d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.064+0000 7fb45e977740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc11430250 con 0x55fc114d09e0 2024-03-21T11:41:28.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.064+0000 7fb45e977740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc113d8a30 con 0x55fc114cae80 2024-03-21T11:41:28.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.064+0000 7fb45e977740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1142c2f0 con 0x55fc114cb870 2024-03-21T11:41:28.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.065+0000 7fb459c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cae80 0x55fc114cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.065+0000 7fb459c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cae80 0x55fc114cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47334/0 (socket says 172.21.15.47:47334) 2024-03-21T11:41:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.065+0000 7fb459c00640 1 -- 172.21.15.47:0/3711894495 learned_addr learned my addr 172.21.15.47:0/3711894495 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.065+0000 7fb459200640 1 --2- 172.21.15.47:0/3711894495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cb870 0x55fc114d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.065+0000 7fb459c00640 1 -- 172.21.15.47:0/3711894495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cb870 msgr2=0x55fc114d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.065+0000 7fb459c00640 1 --2- 172.21.15.47:0/3711894495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cb870 0x55fc114d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.065+0000 7fb459c00640 1 -- 172.21.15.47:0/3711894495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114d09e0 msgr2=0x55fc114d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.065+0000 7fb459c00640 1 --2- 172.21.15.47:0/3711894495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114d09e0 0x55fc114d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.065+0000 7fb459c00640 1 -- 172.21.15.47:0/3711894495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc113c3690 con 0x55fc114cae80 2024-03-21T11:41:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.066+0000 7fb459200640 1 --2- 172.21.15.47:0/3711894495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cb870 0x55fc114d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:41:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.066+0000 7fb459c00640 1 --2- 172.21.15.47:0/3711894495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cae80 0x55fc114cb2a0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009f50 tx=0x7fb44c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af3ac055b76d6fa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.066+0000 7fb451600640 1 --2- 172.21.15.47:0/127438485 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb43c0a3070 0x7fb43c0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.066+0000 7fb446000640 1 -- 172.21.15.47:0/3711894495 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c004d20 con 0x55fc114cae80 2024-03-21T11:41:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.066+0000 7fb446000640 1 -- 172.21.15.47:0/3711894495 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb44c003970 con 0x55fc114cae80 2024-03-21T11:41:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.066+0000 7fb446000640 1 -- 172.21.15.47:0/3711894495 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c004490 con 0x55fc114cae80 2024-03-21T11:41:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.066+0000 7fb451600640 1 -- 172.21.15.47:0/127438485 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb4480f11c0 con 0x7fb4541038c0 2024-03-21T11:41:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.067+0000 7fb459200640 1 --2- 172.21.15.47:0/127438485 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb43c0a3070 0x7fb43c0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.067+0000 7fb45e977740 1 -- 172.21.15.47:0/3711894495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cae80 msgr2=0x55fc114cb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:28.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.067+0000 7fb45e977740 1 --2- 172.21.15.47:0/3711894495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cae80 0x55fc114cb2a0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009f50 tx=0x7fb44c0118d0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.067+0000 7fb45e977740 1 -- 172.21.15.47:0/3711894495 shutdown_connections 2024-03-21T11:41:28.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.067+0000 7fb45e977740 1 --2- 172.21.15.47:0/3711894495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114d09e0 0x55fc114d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.067+0000 7fb45e977740 1 --2- 172.21.15.47:0/3711894495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cb870 0x55fc114d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.067+0000 7fb45e977740 1 --2- 172.21.15.47:0/3711894495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cae80 0x55fc114cb2a0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.067+0000 7fb45e977740 1 -- 172.21.15.47:0/3711894495 >> 172.21.15.47:0/3711894495 conn(0x55fc11493200 msgr2=0x55fc114c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.067+0000 7fb45e977740 1 -- 172.21.15.47:0/3711894495 shutdown_connections 2024-03-21T11:41:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.068+0000 7fb459200640 1 --2- 172.21.15.47:0/127438485 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb43c0a3070 0x7fb43c0a5530 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb4480bd490 tx=0x7fb448009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.068+0000 7fb45e977740 1 -- 172.21.15.47:0/3711894495 wait complete. 2024-03-21T11:41:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.068+0000 7fb45e977740 1 -- start start 2024-03-21T11:41:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.068+0000 7fb45e977740 1 Processor -- start 2024-03-21T11:41:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.069+0000 7fb45e977740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114cae80 0x55fc114d4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.069+0000 7fb45e977740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cb870 0x55fc114d53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.069+0000 7fb45e977740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114d09e0 0x55fc114dc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.069+0000 7fb45e977740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc114d3990 con 0x55fc114cae80 2024-03-21T11:41:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.069+0000 7fb45e977740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc113ebe80 con 0x55fc114cb870 2024-03-21T11:41:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.069+0000 7fb45e977740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc114d3b10 con 0x55fc114d09e0 2024-03-21T11:41:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.069+0000 7fb45e977740 10 client.0 _ll_register_callbacks cb 0x55fc113fe130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.069+0000 7fb453e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cb870 0x55fc114d53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.069+0000 7fb453e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cb870 0x55fc114d53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47350/0 (socket says 172.21.15.47:47350) 2024-03-21T11:41:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.069+0000 7fb453e00640 1 -- 172.21.15.47:0/1773872748 learned_addr learned my addr 172.21.15.47:0/1773872748 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.070+0000 7fb453e00640 1 -- 172.21.15.47:0/1773872748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114d09e0 msgr2=0x55fc114dc3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:41:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.070+0000 7fb459c00640 1 --2- 172.21.15.47:0/1773872748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114d09e0 0x55fc114dc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.070+0000 7fb453e00640 1 --2- 172.21.15.47:0/1773872748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114d09e0 0x55fc114dc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.070+0000 7fb453e00640 1 -- 172.21.15.47:0/1773872748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114cae80 msgr2=0x55fc114d4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.070+0000 7fb453e00640 1 --2- 172.21.15.47:0/1773872748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114cae80 0x55fc114d4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.070+0000 7fb453e00640 1 -- 172.21.15.47:0/1773872748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc1120e7d0 con 0x55fc114cb870 2024-03-21T11:41:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.070+0000 7fb459c00640 1 --2- 172.21.15.47:0/1773872748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114d09e0 0x55fc114dc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:41:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.070+0000 7fb453e00640 1 --2- 172.21.15.47:0/1773872748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cb870 0x55fc114d53f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb44000bed0 tx=0x7fb4400107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f41ff7aced23975 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.070+0000 7fb43aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:41:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.070+0000 7fb43aa00640 1 -- 172.21.15.47:0/1773872748 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440011850 con 0x55fc114cb870 2024-03-21T11:41:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.071+0000 7fb45e977740 1 -- 172.21.15.47:0/1773872748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb44c003970 con 0x55fc114cb870 2024-03-21T11:41:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.071+0000 7fb43aa00640 1 -- 172.21.15.47:0/1773872748 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb4400119f0 con 0x55fc114cb870 2024-03-21T11:41:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.071+0000 7fb43aa00640 1 -- 172.21.15.47:0/1773872748 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb440025050 con 0x55fc114cb870 2024-03-21T11:41:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.071+0000 7fb45e977740 10 client.31358 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.071+0000 7fb45e977740 1 -- 172.21.15.47:0/1773872748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55fc111443f0 con 0x55fc114cb870 2024-03-21T11:41:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.071+0000 7fb43aa00640 1 -- 172.21.15.47:0/1773872748 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb44c003970 con 0x55fc114cb870 2024-03-21T11:41:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.071+0000 7fb43aa00640 1 -- 172.21.15.47:0/1773872748 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb44001a380 con 0x55fc114cb870 2024-03-21T11:41:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.072+0000 7fb45e977740 10 client.31358 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.072+0000 7fb45e977740 10 client.31358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.072+0000 7fb45e977740 20 client.31358 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.072+0000 7fb45e977740 1 --2- 172.21.15.47:0/1773872748 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc114e0bd0 0x55fc114e3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.072+0000 7fb45e977740 4 client.31358 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:41:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.072+0000 7fb45e977740 1 -- 172.21.15.47:0/1773872748 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc1143d790 con 0x55fc114e0bd0 2024-03-21T11:41:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.072+0000 7fb459c00640 1 --2- 172.21.15.47:0/1773872748 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc114e0bd0 0x55fc114e3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.073+0000 7fb459c00640 1 --2- 172.21.15.47:0/1773872748 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc114e0bd0 0x55fc114e3050 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0b1513e6fd78c10 server_cookie=3abfa6080d5cd5df in_seq=0 out_seq=0 2024-03-21T11:41:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.073+0000 7fb43aa00640 10 client.31358 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.077+0000 7fb43aa00640 1 -- 172.21.15.47:0/1773872748 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fc1143d790 con 0x55fc114e0bd0 2024-03-21T11:41:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.077+0000 7fb43aa00640 10 client.31358 handle_command_reply: tid=0 2024-03-21T11:41:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.077+0000 7fb45e977740 1 client.31358 shutdown 2024-03-21T11:41:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.077+0000 7fb45e977740 1 -- 172.21.15.47:0/1773872748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cb870 msgr2=0x55fc114d53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.077+0000 7fb45e977740 1 --2- 172.21.15.47:0/1773872748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cb870 0x55fc114d53f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb44000bed0 tx=0x7fb4400107a0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.077+0000 7fb45e977740 1 -- 172.21.15.47:0/1773872748 shutdown_connections 2024-03-21T11:41:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.077+0000 7fb45e977740 1 --2- 172.21.15.47:0/1773872748 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc114e0bd0 0x55fc114e3050 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.077+0000 7fb45e977740 1 --2- 172.21.15.47:0/1773872748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114d09e0 0x55fc114dc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.077+0000 7fb45e977740 1 --2- 172.21.15.47:0/1773872748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cb870 0x55fc114d53f0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.077+0000 7fb45e977740 1 --2- 172.21.15.47:0/1773872748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114cae80 0x55fc114d4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.077+0000 7fb45e977740 1 -- 172.21.15.47:0/1773872748 >> 172.21.15.47:0/1773872748 conn(0x55fc11493200 msgr2=0x55fc114c5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.077+0000 7fb43aa00640 0 client.31358 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.077+0000 7fb45e977740 1 -- 172.21.15.47:0/1773872748 shutdown_connections 2024-03-21T11:41:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.077+0000 7fb45e977740 1 -- 172.21.15.47:0/1773872748 wait complete. 2024-03-21T11:41:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.078+0000 7fb45e977740 20 client.31358 trim_cache size 0 max 16384 2024-03-21T11:41:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.099+0000 7fb45e977740 1 Processor -- start 2024-03-21T11:41:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.100+0000 7fb45e977740 1 -- start start 2024-03-21T11:41:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.100+0000 7fb45e977740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cae80 0x55fc114cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.100+0000 7fb45e977740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cb870 0x55fc114dc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.100+0000 7fb45e977740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114d09e0 0x55fc114c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.100+0000 7fb45e977740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4541a7e40 con 0x55fc114d09e0 2024-03-21T11:41:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.100+0000 7fb45e977740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4541a97c0 con 0x55fc114cb870 2024-03-21T11:41:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.100+0000 7fb45e977740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4541a84a0 con 0x55fc114cae80 2024-03-21T11:41:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.100+0000 7fb453e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cb870 0x55fc114dc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.100+0000 7fb459c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cae80 0x55fc114cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.100+0000 7fb453e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cb870 0x55fc114dc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47356/0 (socket says 172.21.15.47:47356) 2024-03-21T11:41:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.100+0000 7fb453e00640 1 -- 172.21.15.47:0/471575215 learned_addr learned my addr 172.21.15.47:0/471575215 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.100+0000 7fb459c00640 1 --2- 172.21.15.47:0/471575215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114d09e0 0x55fc114c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.101+0000 7fb453e00640 1 -- 172.21.15.47:0/471575215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cae80 msgr2=0x55fc114cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.101+0000 7fb453e00640 1 --2- 172.21.15.47:0/471575215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cae80 0x55fc114cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.101+0000 7fb453e00640 1 -- 172.21.15.47:0/471575215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114d09e0 msgr2=0x55fc114c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.101+0000 7fb453e00640 1 --2- 172.21.15.47:0/471575215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114d09e0 0x55fc114c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.101+0000 7fb453e00640 1 -- 172.21.15.47:0/471575215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc111040f0 con 0x55fc114cb870 2024-03-21T11:41:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.101+0000 7fb453e00640 1 --2- 172.21.15.47:0/471575215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cb870 0x55fc114dc380 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb44001a7a0 tx=0x7fb44000b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94f74b009adf53fd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.101+0000 7fb445600640 1 -- 172.21.15.47:0/471575215 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440022040 con 0x55fc114cb870 2024-03-21T11:41:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.101+0000 7fb445600640 1 -- 172.21.15.47:0/471575215 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb44001a380 con 0x55fc114cb870 2024-03-21T11:41:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.101+0000 7fb445600640 1 -- 172.21.15.47:0/471575215 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440000e60 con 0x55fc114cb870 2024-03-21T11:41:28.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.102+0000 7fb45e977740 1 -- 172.21.15.47:0/471575215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cb870 msgr2=0x55fc114dc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:28.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.102+0000 7fb45e977740 1 --2- 172.21.15.47:0/471575215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cb870 0x55fc114dc380 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb44001a7a0 tx=0x7fb44000b410 comp rx=0 tx=0).stop 2024-03-21T11:41:28.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.102+0000 7fb45e977740 1 -- 172.21.15.47:0/471575215 shutdown_connections 2024-03-21T11:41:28.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.102+0000 7fb45e977740 1 --2- 172.21.15.47:0/471575215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114d09e0 0x55fc114c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.102+0000 7fb45e977740 1 --2- 172.21.15.47:0/471575215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cb870 0x55fc114dc380 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.102+0000 7fb45e977740 1 --2- 172.21.15.47:0/471575215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cae80 0x55fc114cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.102+0000 7fb45e977740 1 -- 172.21.15.47:0/471575215 >> 172.21.15.47:0/471575215 conn(0x55fc11493200 msgr2=0x55fc114cc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:28.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.102+0000 7fb45e977740 1 -- 172.21.15.47:0/471575215 shutdown_connections 2024-03-21T11:41:28.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.102+0000 7fb45e977740 1 -- 172.21.15.47:0/471575215 wait complete. 2024-03-21T11:41:28.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.103+0000 7fb45e977740 1 -- start start 2024-03-21T11:41:28.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.103+0000 7fb45e977740 1 Processor -- start 2024-03-21T11:41:28.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.103+0000 7fb45e977740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cae80 0x55fc114d9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.103+0000 7fb45e977740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cb870 0x55fc114d9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.103+0000 7fb45e977740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114d09e0 0x55fc114dacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.103+0000 7fb45e977740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4541a9160 con 0x55fc114d09e0 2024-03-21T11:41:28.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.103+0000 7fb45e977740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4541a8b00 con 0x55fc114cae80 2024-03-21T11:41:28.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.103+0000 7fb45e977740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4541a6000 con 0x55fc114cb870 2024-03-21T11:41:28.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.103+0000 7fb45e977740 10 client.0 _ll_register_callbacks cb 0x7fb44000b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:28.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.103+0000 7fb453e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cb870 0x55fc114d9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.103+0000 7fb459c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cae80 0x55fc114d9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.103+0000 7fb453e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cb870 0x55fc114d9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51292/0 (socket says 172.21.15.47:51292) 2024-03-21T11:41:28.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.103+0000 7fb453e00640 1 -- 172.21.15.47:0/3106115796 learned_addr learned my addr 172.21.15.47:0/3106115796 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:28.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.103+0000 7fb459c00640 1 --2- 172.21.15.47:0/3106115796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114d09e0 0x55fc114dacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.104+0000 7fb453e00640 1 -- 172.21.15.47:0/3106115796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cae80 msgr2=0x55fc114d9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:28.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.104+0000 7fb453e00640 1 --2- 172.21.15.47:0/3106115796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cae80 0x55fc114d9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.104+0000 7fb453e00640 1 -- 172.21.15.47:0/3106115796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114d09e0 msgr2=0x55fc114dacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:28.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.104+0000 7fb453e00640 1 --2- 172.21.15.47:0/3106115796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114d09e0 0x55fc114dacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.104+0000 7fb453e00640 1 -- 172.21.15.47:0/3106115796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc114db2a0 con 0x55fc114cb870 2024-03-21T11:41:28.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.104+0000 7fb459c00640 1 --2- 172.21.15.47:0/3106115796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cae80 0x55fc114d9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:41:28.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.104+0000 7fb453e00640 1 --2- 172.21.15.47:0/3106115796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cb870 0x55fc114d9db0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb440005d80 tx=0x7fb44001aaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9ca15b83d8acea9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:28.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.104+0000 7fb43aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:41:28.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.105+0000 7fb43aa00640 1 -- 172.21.15.47:0/3106115796 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440000e60 con 0x55fc114cb870 2024-03-21T11:41:28.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.105+0000 7fb45e977740 1 -- 172.21.15.47:0/3106115796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb44001a380 con 0x55fc114cb870 2024-03-21T11:41:28.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.105+0000 7fb43aa00640 1 -- 172.21.15.47:0/3106115796 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb440011650 con 0x55fc114cb870 2024-03-21T11:41:28.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.105+0000 7fb43aa00640 1 -- 172.21.15.47:0/3106115796 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb440042050 con 0x55fc114cb870 2024-03-21T11:41:28.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.105+0000 7fb45e977740 10 client.31348 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:28.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.105+0000 7fb45e977740 1 -- 172.21.15.47:0/3106115796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55fc114dc5e0 con 0x55fc114cb870 2024-03-21T11:41:28.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.105+0000 7fb43aa00640 1 -- 172.21.15.47:0/3106115796 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb44001a380 con 0x55fc114cb870 2024-03-21T11:41:28.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.106+0000 7fb43aa00640 1 -- 172.21.15.47:0/3106115796 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb4400035f0 con 0x55fc114cb870 2024-03-21T11:41:28.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.106+0000 7fb45e977740 10 client.31348 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:28.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.106+0000 7fb45e977740 10 client.31348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:28.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.106+0000 7fb45e977740 20 client.31348 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:28.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.106+0000 7fb45e977740 1 --2- 172.21.15.47:0/3106115796 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc1153ca80 0x55fc1153eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.106+0000 7fb45e977740 4 client.31348 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:41:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.106+0000 7fb45e977740 1 -- 172.21.15.47:0/3106115796 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fc114d4ca0 con 0x55fc1153ca80 2024-03-21T11:41:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.107+0000 7fb459c00640 1 --2- 172.21.15.47:0/3106115796 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc1153ca80 0x55fc1153eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.107+0000 7fb459c00640 1 --2- 172.21.15.47:0/3106115796 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc1153ca80 0x55fc1153eea0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed987aa5b513dec server_cookie=8cd4d3781dbde4b0 in_seq=0 out_seq=0 2024-03-21T11:41:28.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.107+0000 7fb43aa00640 10 client.31348 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:28.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb43aa00640 1 -- 172.21.15.47:0/3106115796 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fc114d4ca0 con 0x55fc1153ca80 2024-03-21T11:41:28.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb43aa00640 10 client.31348 handle_command_reply: tid=0 2024-03-21T11:41:28.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb45e977740 1 client.31348 shutdown 2024-03-21T11:41:28.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb45e977740 1 -- 172.21.15.47:0/3106115796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cb870 msgr2=0x55fc114d9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:28.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb45e977740 1 --2- 172.21.15.47:0/3106115796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cb870 0x55fc114d9db0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb440005d80 tx=0x7fb44001aaf0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb45e977740 1 -- 172.21.15.47:0/3106115796 shutdown_connections 2024-03-21T11:41:28.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb45e977740 1 --2- 172.21.15.47:0/3106115796 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc1153ca80 0x55fc1153eea0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb45e977740 1 --2- 172.21.15.47:0/3106115796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc114d09e0 0x55fc114dacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb45e977740 1 --2- 172.21.15.47:0/3106115796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc114cb870 0x55fc114d9db0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb45e977740 1 --2- 172.21.15.47:0/3106115796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc114cae80 0x55fc114d9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb45e977740 1 -- 172.21.15.47:0/3106115796 >> 172.21.15.47:0/3106115796 conn(0x55fc11493200 msgr2=0x55fc114ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb43aa00640 0 client.31348 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb45e977740 1 -- 172.21.15.47:0/3106115796 shutdown_connections 2024-03-21T11:41:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb45e977740 1 -- 172.21.15.47:0/3106115796 wait complete. 2024-03-21T11:41:28.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.109+0000 7fb45e977740 20 client.31348 trim_cache size 0 max 16384 2024-03-21T11:41:28.119 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:41:28.119 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:41:28.119 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:41:28.119 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:41:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.110+0000 7fb45bc00640 1 -- 172.21.15.47:0/127438485 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb43c0a3070 msgr2=0x7fb43c0a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.110+0000 7fb45bc00640 1 --2- 172.21.15.47:0/127438485 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb43c0a3070 0x7fb43c0a5530 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb4480bd490 tx=0x7fb448009260 comp rx=0 tx=0).stop 2024-03-21T11:41:28.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.110+0000 7fb45bc00640 1 -- 172.21.15.47:0/127438485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4541038c0 msgr2=0x7fb454199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:28.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.110+0000 7fb45bc00640 1 --2- 172.21.15.47:0/127438485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4541038c0 0x7fb454199bb0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb448000c00 tx=0x7fb44802c6c0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.110+0000 7fb45bc00640 1 -- 172.21.15.47:0/127438485 shutdown_connections 2024-03-21T11:41:28.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.110+0000 7fb45bc00640 1 --2- 172.21.15.47:0/127438485 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb43c0a3070 0x7fb43c0a5530 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.110+0000 7fb45bc00640 1 --2- 172.21.15.47:0/127438485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb454105470 0x7fb4541a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.110+0000 7fb45bc00640 1 --2- 172.21.15.47:0/127438485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb454104ab0 0x7fb45419a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.110+0000 7fb45bc00640 1 --2- 172.21.15.47:0/127438485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4541038c0 0x7fb454199bb0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.110+0000 7fb45bc00640 1 -- 172.21.15.47:0/127438485 >> 172.21.15.47:0/127438485 conn(0x7fb4540fee50 msgr2=0x7fb454107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:28.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.110+0000 7fb45bc00640 1 -- 172.21.15.47:0/127438485 shutdown_connections 2024-03-21T11:41:28.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.110+0000 7fb45bc00640 1 -- 172.21.15.47:0/127438485 wait complete. 2024-03-21T11:41:28.129 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:41:28.129 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:41:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.351+0000 7fab54600640 1 -- 172.21.15.47:0/1661890717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab4c1028a0 msgr2=0x7fab4c104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.351+0000 7fab54600640 1 --2- 172.21.15.47:0/1661890717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab4c1028a0 0x7fab4c104d10 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fab38009900 tx=0x7fab3802bc30 comp rx=0 tx=0).stop 2024-03-21T11:41:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.352+0000 7fab54600640 1 -- 172.21.15.47:0/1661890717 shutdown_connections 2024-03-21T11:41:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.352+0000 7fab54600640 1 --2- 172.21.15.47:0/1661890717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab4c105250 0x7fab4c1056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.352+0000 7fab54600640 1 --2- 172.21.15.47:0/1661890717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab4c1028a0 0x7fab4c104d10 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.352+0000 7fab54600640 1 --2- 172.21.15.47:0/1661890717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab4c0fff30 0x7fab4c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.352+0000 7fab54600640 1 -- 172.21.15.47:0/1661890717 >> 172.21.15.47:0/1661890717 conn(0x7fab4c0fbb30 msgr2=0x7fab4c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.352+0000 7fab54600640 1 -- 172.21.15.47:0/1661890717 shutdown_connections 2024-03-21T11:41:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.352+0000 7fab54600640 1 -- 172.21.15.47:0/1661890717 wait complete. 2024-03-21T11:41:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.353+0000 7fab54600640 1 Processor -- start 2024-03-21T11:41:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.354+0000 7fab54600640 1 -- start start 2024-03-21T11:41:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.354+0000 7fab54600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab4c0fff30 0x7fab4c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.354+0000 7fab54600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab4c1028a0 0x7fab4c19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.354+0000 7fab54600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab4c105250 0x7fab4c1a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.354+0000 7fab54600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c113930 con 0x7fab4c1028a0 2024-03-21T11:41:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.354+0000 7fab54600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c1137b0 con 0x7fab4c105250 2024-03-21T11:41:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.354+0000 7fab54600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c113ab0 con 0x7fab4c0fff30 2024-03-21T11:41:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.354+0000 7fab51c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab4c0fff30 0x7fab4c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.354+0000 7fab51c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab4c0fff30 0x7fab4c19dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51314/0 (socket says 172.21.15.47:51314) 2024-03-21T11:41:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.354+0000 7fab51200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab4c1028a0 0x7fab4c19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.354+0000 7fab52600640 1 --2- 172.21.15.47:0/2395875708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab4c105250 0x7fab4c1a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.354+0000 7fab51c00640 1 -- 172.21.15.47:0/2395875708 learned_addr learned my addr 172.21.15.47:0/2395875708 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.355+0000 7fab52600640 1 -- 172.21.15.47:0/2395875708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab4c0fff30 msgr2=0x7fab4c19dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.355+0000 7fab52600640 1 --2- 172.21.15.47:0/2395875708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab4c0fff30 0x7fab4c19dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.355+0000 7fab52600640 1 -- 172.21.15.47:0/2395875708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab4c1028a0 msgr2=0x7fab4c19e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.355+0000 7fab52600640 1 --2- 172.21.15.47:0/2395875708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab4c1028a0 0x7fab4c19e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.355+0000 7fab52600640 1 -- 172.21.15.47:0/2395875708 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c1a5ba0 con 0x7fab4c105250 2024-03-21T11:41:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.355+0000 7fab51200640 1 --2- 172.21.15.47:0/2395875708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab4c1028a0 0x7fab4c19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:41:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.355+0000 7fab52600640 1 --2- 172.21.15.47:0/2395875708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab4c105250 0x7fab4c1a5530 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fab44009fd0 tx=0x7fab4400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.355+0000 7fab4a000640 1 -- 172.21.15.47:0/2395875708 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4400ce20 con 0x7fab4c105250 2024-03-21T11:41:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.356+0000 7fab4a000640 1 -- 172.21.15.47:0/2395875708 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab44005020 con 0x7fab4c105250 2024-03-21T11:41:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.356+0000 7fab54600640 1 -- 172.21.15.47:0/2395875708 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4c1a5e90 con 0x7fab4c105250 2024-03-21T11:41:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.356+0000 7fab4a000640 1 -- 172.21.15.47:0/2395875708 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44010720 con 0x7fab4c105250 2024-03-21T11:41:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.356+0000 7fab54600640 1 -- 172.21.15.47:0/2395875708 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4c1a6350 con 0x7fab4c105250 2024-03-21T11:41:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.357+0000 7fab54600640 1 -- 172.21.15.47:0/2395875708 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4c0fd620 con 0x7fab4c105250 2024-03-21T11:41:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.360+0000 7fab4a000640 1 -- 172.21.15.47:0/2395875708 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fab44005540 con 0x7fab4c105250 2024-03-21T11:41:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.362+0000 7fab4a000640 1 --2- 172.21.15.47:0/2395875708 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab340a2fa0 0x7fab340a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.362+0000 7fab4a000640 1 -- 172.21.15.47:0/2395875708 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fab440cfec0 con 0x7fab4c105250 2024-03-21T11:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.362+0000 7fab51c00640 1 --2- 172.21.15.47:0/2395875708 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab340a2fa0 0x7fab340a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.363+0000 7fab51c00640 1 --2- 172.21.15.47:0/2395875708 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab340a2fa0 0x7fab340a5460 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fab40009cd0 tx=0x7fab40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:28.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.364+0000 7fab4a000640 1 -- 172.21.15.47:0/2395875708 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fab4409c960 con 0x7fab4c105250 2024-03-21T11:41:28.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.749+0000 7fab54600640 1 -- 172.21.15.47:0/2395875708 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab4c02d070 con 0x7fab4c105250 2024-03-21T11:41:28.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.750+0000 7fab4a000640 1 -- 172.21.15.47:0/2395875708 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fab4409c260 con 0x7fab4c105250 2024-03-21T11:41:28.751 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:41:28.752 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:39:47.303461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:41:28.752 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-21T11:41:28.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.754+0000 7fab54600640 1 -- 172.21.15.47:0/2395875708 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab340a2fa0 msgr2=0x7fab340a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:28.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.754+0000 7fab54600640 1 --2- 172.21.15.47:0/2395875708 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab340a2fa0 0x7fab340a5460 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fab40009cd0 tx=0x7fab40005f90 comp rx=0 tx=0).stop 2024-03-21T11:41:28.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.754+0000 7fab54600640 1 -- 172.21.15.47:0/2395875708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab4c105250 msgr2=0x7fab4c1a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:28.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.754+0000 7fab54600640 1 --2- 172.21.15.47:0/2395875708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab4c105250 0x7fab4c1a5530 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fab44009fd0 tx=0x7fab4400ef10 comp rx=0 tx=0).stop 2024-03-21T11:41:28.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.754+0000 7fab54600640 1 -- 172.21.15.47:0/2395875708 shutdown_connections 2024-03-21T11:41:28.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.754+0000 7fab54600640 1 --2- 172.21.15.47:0/2395875708 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab340a2fa0 0x7fab340a5460 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.754+0000 7fab54600640 1 --2- 172.21.15.47:0/2395875708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab4c105250 0x7fab4c1a5530 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.754+0000 7fab54600640 1 --2- 172.21.15.47:0/2395875708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab4c1028a0 0x7fab4c19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.754+0000 7fab54600640 1 --2- 172.21.15.47:0/2395875708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab4c0fff30 0x7fab4c19dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:28.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.754+0000 7fab54600640 1 -- 172.21.15.47:0/2395875708 >> 172.21.15.47:0/2395875708 conn(0x7fab4c0fbb30 msgr2=0x7fab4c1009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:28.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.754+0000 7fab54600640 1 -- 172.21.15.47:0/2395875708 shutdown_connections 2024-03-21T11:41:28.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:28.754+0000 7fab54600640 1 -- 172.21.15.47:0/2395875708 wait complete. 2024-03-21T11:41:28.778 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:41:28.778 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:41:28.948 INFO:tasks.workunit.client.0.smithi047.stdout: 10 117449 24.13 MB/sec execute 132 sec latency 72.975 ms 2024-03-21T11:41:29.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.059+0000 7f9308e00640 1 -- 172.21.15.47:0/4108742087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f930410dc80 msgr2=0x7f9304110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.059+0000 7f9308e00640 1 --2- 172.21.15.47:0/4108742087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f930410dc80 0x7f9304110070 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009c10 tx=0x7f92fc02bf00 comp rx=0 tx=0).stop 2024-03-21T11:41:29.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.059+0000 7f9308e00640 1 -- 172.21.15.47:0/4108742087 shutdown_connections 2024-03-21T11:41:29.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.059+0000 7f9308e00640 1 --2- 172.21.15.47:0/4108742087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f930410dc80 0x7f9304110070 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.059+0000 7f9308e00640 1 --2- 172.21.15.47:0/4108742087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9304074100 0x7f9304074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.059+0000 7f9308e00640 1 --2- 172.21.15.47:0/4108742087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93040756e0 0x7f9304073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.059+0000 7f9308e00640 1 -- 172.21.15.47:0/4108742087 >> 172.21.15.47:0/4108742087 conn(0x7f93040fcc20 msgr2=0x7f93040ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.060+0000 7f9308e00640 1 -- 172.21.15.47:0/4108742087 shutdown_connections 2024-03-21T11:41:29.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.060+0000 7f9308e00640 1 -- 172.21.15.47:0/4108742087 wait complete. 2024-03-21T11:41:29.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.062+0000 7f9308e00640 1 Processor -- start 2024-03-21T11:41:29.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.063+0000 7f9308e00640 1 -- start start 2024-03-21T11:41:29.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.063+0000 7f9308e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9304074100 0x7f930419dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:28 smithi047 ceph-mon[31351]: pgmap v944: 129 pgs: 129 active+clean; 469 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.4 MiB/s wr, 482 op/s 2024-03-21T11:41:29.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:28 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2395875708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:41:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.064+0000 7f9308e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93040756e0 0x7f930419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.064+0000 7f9308e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930410dc80 0x7f93041a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.064+0000 7f9308e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9304113950 con 0x7f93040756e0 2024-03-21T11:41:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.064+0000 7f9308e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041137d0 con 0x7f930410dc80 2024-03-21T11:41:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.064+0000 7f9308e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9304113ad0 con 0x7f9304074100 2024-03-21T11:41:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.064+0000 7f9301e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9304074100 0x7f930419dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.064+0000 7f9301e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9304074100 0x7f930419dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34674/0 (socket says 172.21.15.47:34674) 2024-03-21T11:41:29.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.064+0000 7f9301e00640 1 -- 172.21.15.47:0/423295400 learned_addr learned my addr 172.21.15.47:0/423295400 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.064+0000 7f9302800640 1 --2- 172.21.15.47:0/423295400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930410dc80 0x7f93041a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.065+0000 7f9301e00640 1 -- 172.21.15.47:0/423295400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930410dc80 msgr2=0x7f93041a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.065+0000 7f9301e00640 1 --2- 172.21.15.47:0/423295400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930410dc80 0x7f93041a5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.065+0000 7f9301e00640 1 -- 172.21.15.47:0/423295400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93040756e0 msgr2=0x7f930419e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.065+0000 7f9301e00640 1 --2- 172.21.15.47:0/423295400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93040756e0 0x7f930419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.065+0000 7f9301e00640 1 -- 172.21.15.47:0/423295400 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93041a5bf0 con 0x7f9304074100 2024-03-21T11:41:29.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.065+0000 7f9301e00640 1 --2- 172.21.15.47:0/423295400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9304074100 0x7f930419dfd0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f92f400e9f0 tx=0x7f92f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.066+0000 7f92fb400640 1 -- 172.21.15.47:0/423295400 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f400cd80 con 0x7f9304074100 2024-03-21T11:41:29.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.066+0000 7f92fb400640 1 -- 172.21.15.47:0/423295400 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92f4010910 con 0x7f9304074100 2024-03-21T11:41:29.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.066+0000 7f92fb400640 1 -- 172.21.15.47:0/423295400 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4018980 con 0x7f9304074100 2024-03-21T11:41:29.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.066+0000 7f9308e00640 1 -- 172.21.15.47:0/423295400 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93041a5ee0 con 0x7f9304074100 2024-03-21T11:41:29.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.066+0000 7f9308e00640 1 -- 172.21.15.47:0/423295400 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93041a6350 con 0x7f9304074100 2024-03-21T11:41:29.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.075+0000 7f930bc20740 1 Processor -- start 2024-03-21T11:41:29.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.076+0000 7f930bc20740 1 -- start start 2024-03-21T11:41:29.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.077+0000 7f930bc20740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e7810 0x5569267e7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.077+0000 7f930bc20740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e89e0 0x5569267e8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.077+0000 7f930bc20740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267efc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.077+0000 7f930bc20740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692674c2c0 con 0x5569267e89e0 2024-03-21T11:41:29.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.077+0000 7f930bc20740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569266f4c90 con 0x5569267e93a0 2024-03-21T11:41:29.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.077+0000 7f930bc20740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556926748360 con 0x5569267e7810 2024-03-21T11:41:29.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.077+0000 7f92fb400640 1 -- 172.21.15.47:0/423295400 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f92f4010ac0 con 0x7f9304074100 2024-03-21T11:41:29.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.077+0000 7f9302800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e7810 0x5569267e7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.077+0000 7f9302800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e7810 0x5569267e7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34682/0 (socket says 172.21.15.47:34682) 2024-03-21T11:41:29.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.078+0000 7f9302800640 1 -- 172.21.15.47:0/1490517751 learned_addr learned my addr 172.21.15.47:0/1490517751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.078+0000 7f92f9400640 1 --2- 172.21.15.47:0/1490517751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267efc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.078+0000 7f9302800640 1 -- 172.21.15.47:0/1490517751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 msgr2=0x5569267efc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.078+0000 7f9302800640 1 --2- 172.21.15.47:0/1490517751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267efc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.078+0000 7f9302800640 1 -- 172.21.15.47:0/1490517751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e89e0 msgr2=0x5569267e8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.078+0000 7f9302800640 1 --2- 172.21.15.47:0/1490517751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e89e0 0x5569267e8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.078+0000 7f9302800640 1 -- 172.21.15.47:0/1490517751 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556926719080 con 0x5569267e7810 2024-03-21T11:41:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.079+0000 7f9302800640 1 --2- 172.21.15.47:0/1490517751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e7810 0x5569267e7c30 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f92fc02c440 tx=0x7f92fc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35ecb396b652e943 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.079+0000 7f92f3400640 1 -- 172.21.15.47:0/1490517751 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc042070 con 0x5569267e7810 2024-03-21T11:41:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.079+0000 7f92f3400640 1 -- 172.21.15.47:0/1490517751 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92fc0344d0 con 0x5569267e7810 2024-03-21T11:41:29.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.080+0000 7f930bc20740 1 -- 172.21.15.47:0/1490517751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e7810 msgr2=0x5569267e7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.080+0000 7f930bc20740 1 --2- 172.21.15.47:0/1490517751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e7810 0x5569267e7c30 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f92fc02c440 tx=0x7f92fc009310 comp rx=0 tx=0).stop 2024-03-21T11:41:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.081+0000 7f92fb400640 1 --2- 172.21.15.47:0/423295400 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92e80a2fa0 0x7f92e80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.082+0000 7f92fb400640 1 -- 172.21.15.47:0/423295400 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f92f40cfc60 con 0x7f9304074100 2024-03-21T11:41:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.082+0000 7f9302800640 1 --2- 172.21.15.47:0/423295400 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92e80a2fa0 0x7f92e80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.084+0000 7f930bc20740 1 -- 172.21.15.47:0/1490517751 shutdown_connections 2024-03-21T11:41:29.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.084+0000 7f930bc20740 1 --2- 172.21.15.47:0/1490517751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267efc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.084+0000 7f930bc20740 1 --2- 172.21.15.47:0/1490517751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e89e0 0x5569267e8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.084+0000 7f930bc20740 1 --2- 172.21.15.47:0/1490517751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e7810 0x5569267e7c30 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.084+0000 7f930bc20740 1 -- 172.21.15.47:0/1490517751 >> 172.21.15.47:0/1490517751 conn(0x5569267e2c00 msgr2=0x5569267e3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.084+0000 7f930bc20740 1 -- 172.21.15.47:0/1490517751 shutdown_connections 2024-03-21T11:41:29.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.084+0000 7f930bc20740 1 -- 172.21.15.47:0/1490517751 wait complete. 2024-03-21T11:41:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.085+0000 7f930bc20740 1 -- start start 2024-03-21T11:41:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.086+0000 7f9302800640 1 --2- 172.21.15.47:0/423295400 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92e80a2fa0 0x7f92e80a5460 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009540 tx=0x7f92fc02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.086+0000 7f930bc20740 1 Processor -- start 2024-03-21T11:41:29.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.087+0000 7f930bc20740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e7810 0x5569267f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.087+0000 7f930bc20740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e89e0 0x5569267f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.087+0000 7f930bc20740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.087+0000 7f930bc20740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569267f03f0 con 0x5569267e89e0 2024-03-21T11:41:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.087+0000 7f930bc20740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569267080d0 con 0x5569267e93a0 2024-03-21T11:41:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.087+0000 7f930bc20740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569267f0570 con 0x5569267e7810 2024-03-21T11:41:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.087+0000 7f930bc20740 10 client.0 _ll_register_callbacks cb 0x55692670ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.088+0000 7f9302800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.088+0000 7f9302800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267f8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38186/0 (socket says 172.21.15.47:38186) 2024-03-21T11:41:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.088+0000 7f9302800640 1 -- 172.21.15.47:0/1511280234 learned_addr learned my addr 172.21.15.47:0/1511280234 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.088+0000 7f9301e00640 1 --2- 172.21.15.47:0/1511280234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e7810 0x5569267f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.088+0000 7f92f9400640 1 --2- 172.21.15.47:0/1511280234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e89e0 0x5569267f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.088+0000 7f9302800640 1 -- 172.21.15.47:0/1511280234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e7810 msgr2=0x5569267f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.088+0000 7f9302800640 1 --2- 172.21.15.47:0/1511280234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e7810 0x5569267f1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.088+0000 7f9302800640 1 -- 172.21.15.47:0/1511280234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e89e0 msgr2=0x5569267f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.088+0000 7f9302800640 1 --2- 172.21.15.47:0/1511280234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e89e0 0x5569267f1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.088+0000 7f9302800640 1 -- 172.21.15.47:0/1511280234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569264603f0 con 0x5569267e93a0 2024-03-21T11:41:29.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.089+0000 7f9302800640 1 --2- 172.21.15.47:0/1511280234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267f8fa0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f92fc036010 tx=0x7f92fc005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aebe0f2444b28cd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.089+0000 7f92e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:41:29.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.090+0000 7f92e7e00640 1 -- 172.21.15.47:0/1511280234 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc04d070 con 0x5569267e93a0 2024-03-21T11:41:29.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.092+0000 7f92e7e00640 1 -- 172.21.15.47:0/1511280234 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f92fc047660 con 0x5569267e93a0 2024-03-21T11:41:29.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.094+0000 7f930bc20740 1 -- 172.21.15.47:0/1511280234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92fc0344d0 con 0x5569267e93a0 2024-03-21T11:41:29.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.094+0000 7f92e7e00640 1 -- 172.21.15.47:0/1511280234 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92fc047860 con 0x5569267e93a0 2024-03-21T11:41:29.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.095+0000 7f930bc20740 10 client.31379 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:29.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.095+0000 7f930bc20740 1 -- 172.21.15.47:0/1511280234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55692652a7d0 con 0x5569267e93a0 2024-03-21T11:41:29.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.095+0000 7f92e7e00640 1 -- 172.21.15.47:0/1511280234 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92fc0344d0 con 0x5569267e93a0 2024-03-21T11:41:29.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.096+0000 7f92e7e00640 1 -- 172.21.15.47:0/1511280234 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f92fc047a00 con 0x5569267e93a0 2024-03-21T11:41:29.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.097+0000 7f930bc20740 10 client.31379 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:29.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.097+0000 7f930bc20740 10 client.31379 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:41:29.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.097+0000 7f930bc20740 20 client.31379 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:29.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.097+0000 7f930bc20740 1 --2- 172.21.15.47:0/1511280234 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5569267fd100 0x5569267ff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.097+0000 7f930bc20740 4 client.31379 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:41:29.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.097+0000 7f930bc20740 1 -- 172.21.15.47:0/1511280234 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556926759800 con 0x5569267fd100 2024-03-21T11:41:29.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.098+0000 7f9301e00640 1 --2- 172.21.15.47:0/1511280234 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5569267fd100 0x5569267ff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.099+0000 7f9301e00640 1 --2- 172.21.15.47:0/1511280234 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5569267fd100 0x5569267ff4f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=87c710081cc11121 server_cookie=8994edb3fab22ab3 in_seq=0 out_seq=0 2024-03-21T11:41:29.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.099+0000 7f92e7e00640 10 client.31379 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:41:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.100+0000 7f92e7e00640 1 -- 172.21.15.47:0/1511280234 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556926759800 con 0x5569267fd100 2024-03-21T11:41:29.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.100+0000 7f92e7e00640 10 client.31379 handle_command_reply: tid=0 2024-03-21T11:41:29.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.100+0000 7f930bc20740 1 client.31379 shutdown 2024-03-21T11:41:29.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.100+0000 7f930bc20740 1 -- 172.21.15.47:0/1511280234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 msgr2=0x5569267f8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.100+0000 7f930bc20740 1 --2- 172.21.15.47:0/1511280234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267f8fa0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f92fc036010 tx=0x7f92fc005fd0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.101+0000 7f930bc20740 1 -- 172.21.15.47:0/1511280234 shutdown_connections 2024-03-21T11:41:29.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.101+0000 7f930bc20740 1 --2- 172.21.15.47:0/1511280234 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5569267fd100 0x5569267ff4f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.101+0000 7f930bc20740 1 --2- 172.21.15.47:0/1511280234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267f8fa0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.101+0000 7f930bc20740 1 --2- 172.21.15.47:0/1511280234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e89e0 0x5569267f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.101+0000 7f930bc20740 1 --2- 172.21.15.47:0/1511280234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e7810 0x5569267f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.101+0000 7f930bc20740 1 -- 172.21.15.47:0/1511280234 >> 172.21.15.47:0/1511280234 conn(0x5569267e2c00 msgr2=0x5569267e1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.101+0000 7f92e7e00640 0 client.31379 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:41:29.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.101+0000 7f930bc20740 1 -- 172.21.15.47:0/1511280234 shutdown_connections 2024-03-21T11:41:29.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.101+0000 7f930bc20740 1 -- 172.21.15.47:0/1511280234 wait complete. 2024-03-21T11:41:29.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.101+0000 7f930bc20740 20 client.31379 trim_cache size 0 max 16384 2024-03-21T11:41:29.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.127+0000 7f930bc20740 1 Processor -- start 2024-03-21T11:41:29.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.127+0000 7f930bc20740 1 -- start start 2024-03-21T11:41:29.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.128+0000 7f930bc20740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e7810 0x5569267ebc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.128+0000 7f930bc20740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e89e0 0x5569267f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.128+0000 7f930bc20740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e93a0 0x5569267ec150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.128+0000 7f930bc20740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041b0770 con 0x5569267e89e0 2024-03-21T11:41:29.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.128+0000 7f930bc20740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041b20f0 con 0x5569267e7810 2024-03-21T11:41:29.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.128+0000 7f930bc20740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041b0dd0 con 0x5569267e93a0 2024-03-21T11:41:29.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.128+0000 7f9302800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e93a0 0x5569267ec150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.128+0000 7f9302800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e93a0 0x5569267ec150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34704/0 (socket says 172.21.15.47:34704) 2024-03-21T11:41:29.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.128+0000 7f9302800640 1 -- 172.21.15.47:0/1872045771 learned_addr learned my addr 172.21.15.47:0/1872045771 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.129+0000 7f9302800640 1 -- 172.21.15.47:0/1872045771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e7810 msgr2=0x5569267ebc10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:41:29.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.129+0000 7f9302800640 1 --2- 172.21.15.47:0/1872045771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e7810 0x5569267ebc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.129+0000 7f9302800640 1 -- 172.21.15.47:0/1872045771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e89e0 msgr2=0x5569267f8f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:41:29.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.129+0000 7f9302800640 1 --2- 172.21.15.47:0/1872045771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e89e0 0x5569267f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.129+0000 7f9302800640 1 -- 172.21.15.47:0/1872045771 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569267effa0 con 0x5569267e93a0 2024-03-21T11:41:29.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.130+0000 7f9302800640 1 --2- 172.21.15.47:0/1872045771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e93a0 0x5569267ec150 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f92fc02c440 tx=0x7f92fc02bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8483d186c254b7d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.130+0000 7f92f2a00640 1 -- 172.21.15.47:0/1872045771 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc04d040 con 0x5569267e93a0 2024-03-21T11:41:29.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.130+0000 7f92f2a00640 1 -- 172.21.15.47:0/1872045771 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92fc047660 con 0x5569267e93a0 2024-03-21T11:41:29.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.130+0000 7f92f2a00640 1 -- 172.21.15.47:0/1872045771 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc047a00 con 0x5569267e93a0 2024-03-21T11:41:29.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.131+0000 7f930bc20740 1 -- 172.21.15.47:0/1872045771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e93a0 msgr2=0x5569267ec150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.131+0000 7f930bc20740 1 --2- 172.21.15.47:0/1872045771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e93a0 0x5569267ec150 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f92fc02c440 tx=0x7f92fc02bf00 comp rx=0 tx=0).stop 2024-03-21T11:41:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.131+0000 7f930bc20740 1 -- 172.21.15.47:0/1872045771 shutdown_connections 2024-03-21T11:41:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.133+0000 7f930bc20740 1 --2- 172.21.15.47:0/1872045771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e93a0 0x5569267ec150 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.133+0000 7f930bc20740 1 --2- 172.21.15.47:0/1872045771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e89e0 0x5569267f8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.133+0000 7f930bc20740 1 --2- 172.21.15.47:0/1872045771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e7810 0x5569267ebc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.133+0000 7f930bc20740 1 -- 172.21.15.47:0/1872045771 >> 172.21.15.47:0/1872045771 conn(0x5569267e2c00 msgr2=0x5569267e1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.133+0000 7f930bc20740 1 -- 172.21.15.47:0/1872045771 shutdown_connections 2024-03-21T11:41:29.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.134+0000 7f930bc20740 1 -- 172.21.15.47:0/1872045771 wait complete. 2024-03-21T11:41:29.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.135+0000 7f930bc20740 1 -- start start 2024-03-21T11:41:29.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.136+0000 7f930bc20740 1 Processor -- start 2024-03-21T11:41:29.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.137+0000 7f930bc20740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e7810 0x5569267f6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.137+0000 7f930bc20740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e89e0 0x5569267f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.137+0000 7f930bc20740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.137+0000 7f92f9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e89e0 0x5569267f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.137+0000 7f92f9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e89e0 0x5569267f6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34708/0 (socket says 172.21.15.47:34708) 2024-03-21T11:41:29.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.138+0000 7f9302800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.138+0000 7f9302800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267f7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38204/0 (socket says 172.21.15.47:38204) 2024-03-21T11:41:29.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.138+0000 7f9302800640 1 -- 172.21.15.47:0/271662411 learned_addr learned my addr 172.21.15.47:0/271662411 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.138+0000 7f930bc20740 1 -- 172.21.15.47:0/271662411 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041b1a90 con 0x5569267e7810 2024-03-21T11:41:29.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.138+0000 7f930bc20740 1 -- 172.21.15.47:0/271662411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041b1430 con 0x5569267e93a0 2024-03-21T11:41:29.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.138+0000 7f930bc20740 1 -- 172.21.15.47:0/271662411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041ae930 con 0x5569267e89e0 2024-03-21T11:41:29.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.138+0000 7f930bc20740 10 client.0 _ll_register_callbacks cb 0x7f92f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:29.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.139+0000 7f9301e00640 1 --2- 172.21.15.47:0/271662411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e7810 0x5569267f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.139+0000 7f92f9400640 1 -- 172.21.15.47:0/271662411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 msgr2=0x5569267f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.139+0000 7f92f9400640 1 --2- 172.21.15.47:0/271662411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.139+0000 7f92f9400640 1 -- 172.21.15.47:0/271662411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e7810 msgr2=0x5569267f6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.139+0000 7f92f9400640 1 --2- 172.21.15.47:0/271662411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e7810 0x5569267f6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.139+0000 7f92f9400640 1 -- 172.21.15.47:0/271662411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569267f8450 con 0x5569267e89e0 2024-03-21T11:41:29.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.140+0000 7f92f9400640 1 --2- 172.21.15.47:0/271662411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e89e0 0x5569267f6e50 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x5569267f2f00 tx=0x7f92ec013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2faa428f6747602c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.140+0000 7f92e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:41:29.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.141+0000 7f92e7e00640 1 -- 172.21.15.47:0/271662411 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec005220 con 0x5569267e89e0 2024-03-21T11:41:29.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.141+0000 7f92e7e00640 1 -- 172.21.15.47:0/271662411 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f92ec0053c0 con 0x5569267e89e0 2024-03-21T11:41:29.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.141+0000 7f930bc20740 1 -- 172.21.15.47:0/271662411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92fc047660 con 0x5569267e89e0 2024-03-21T11:41:29.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.141+0000 7f92e7e00640 1 -- 172.21.15.47:0/271662411 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92ec0055c0 con 0x5569267e89e0 2024-03-21T11:41:29.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.142+0000 7f930bc20740 10 client.31372 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:29.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.142+0000 7f930bc20740 1 -- 172.21.15.47:0/271662411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556926858ae0 con 0x5569267e89e0 2024-03-21T11:41:29.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.142+0000 7f92e7e00640 1 -- 172.21.15.47:0/271662411 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92fc047660 con 0x5569267e89e0 2024-03-21T11:41:29.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.143+0000 7f92e7e00640 1 -- 172.21.15.47:0/271662411 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f92ec01a070 con 0x5569267e89e0 2024-03-21T11:41:29.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.143+0000 7f930bc20740 10 client.31372 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:29.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.143+0000 7f930bc20740 10 client.31372 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:41:29.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.143+0000 7f930bc20740 20 client.31372 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:29.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.143+0000 7f930bc20740 1 --2- 172.21.15.47:0/271662411 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5569268594b0 0x55692685b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.143+0000 7f930bc20740 4 client.31372 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:41:29.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.143+0000 7f930bc20740 1 -- 172.21.15.47:0/271662411 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556926753b50 con 0x5569268594b0 2024-03-21T11:41:29.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.145+0000 7f9301e00640 1 --2- 172.21.15.47:0/271662411 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5569268594b0 0x55692685b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.145+0000 7f92e7e00640 10 client.31372 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:41:29.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.145+0000 7f9301e00640 1 --2- 172.21.15.47:0/271662411 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5569268594b0 0x55692685b8d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8554f5b7044e0035 server_cookie=1516698a907cb8d1 in_seq=0 out_seq=0 2024-03-21T11:41:29.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.146+0000 7f92e7e00640 1 -- 172.21.15.47:0/271662411 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556926753b50 con 0x5569268594b0 2024-03-21T11:41:29.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.146+0000 7f92e7e00640 10 client.31372 handle_command_reply: tid=0 2024-03-21T11:41:29.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.146+0000 7f930bc20740 1 client.31372 shutdown 2024-03-21T11:41:29.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.148+0000 7f930bc20740 1 -- 172.21.15.47:0/271662411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e89e0 msgr2=0x5569267f6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.148+0000 7f930bc20740 1 --2- 172.21.15.47:0/271662411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e89e0 0x5569267f6e50 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x5569267f2f00 tx=0x7f92ec013830 comp rx=0 tx=0).stop 2024-03-21T11:41:29.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.148+0000 7f930bc20740 1 -- 172.21.15.47:0/271662411 shutdown_connections 2024-03-21T11:41:29.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.148+0000 7f930bc20740 1 --2- 172.21.15.47:0/271662411 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5569268594b0 0x55692685b8d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.149+0000 7f930bc20740 1 --2- 172.21.15.47:0/271662411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569267e93a0 0x5569267f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.149+0000 7f92e7e00640 0 client.31372 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:41:29.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.149+0000 7f930bc20740 1 --2- 172.21.15.47:0/271662411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5569267e89e0 0x5569267f6e50 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.149+0000 7f930bc20740 1 --2- 172.21.15.47:0/271662411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569267e7810 0x5569267f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.149+0000 7f930bc20740 1 -- 172.21.15.47:0/271662411 >> 172.21.15.47:0/271662411 conn(0x5569267e2c00 msgr2=0x5569267ebc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.150+0000 7f930bc20740 1 -- 172.21.15.47:0/271662411 shutdown_connections 2024-03-21T11:41:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.150+0000 7f930bc20740 1 -- 172.21.15.47:0/271662411 wait complete. 2024-03-21T11:41:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.150+0000 7f930bc20740 20 client.31372 trim_cache size 0 max 16384 2024-03-21T11:41:29.151 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:41:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.151+0000 7f9308e00640 1 -- 172.21.15.47:0/423295400 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92e80a2fa0 msgr2=0x7f92e80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.151+0000 7f9308e00640 1 --2- 172.21.15.47:0/423295400 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92e80a2fa0 0x7f92e80a5460 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009540 tx=0x7f92fc02e040 comp rx=0 tx=0).stop 2024-03-21T11:41:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.151+0000 7f9308e00640 1 -- 172.21.15.47:0/423295400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9304074100 msgr2=0x7f930419dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.151+0000 7f9308e00640 1 --2- 172.21.15.47:0/423295400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9304074100 0x7f930419dfd0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f92f400e9f0 tx=0x7f92f400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.151+0000 7f9308e00640 1 -- 172.21.15.47:0/423295400 shutdown_connections 2024-03-21T11:41:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.151+0000 7f9308e00640 1 --2- 172.21.15.47:0/423295400 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92e80a2fa0 0x7f92e80a5460 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.151+0000 7f9308e00640 1 --2- 172.21.15.47:0/423295400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930410dc80 0x7f93041a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.151+0000 7f9308e00640 1 --2- 172.21.15.47:0/423295400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93040756e0 0x7f930419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.151+0000 7f9308e00640 1 --2- 172.21.15.47:0/423295400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9304074100 0x7f930419dfd0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.151+0000 7f9308e00640 1 -- 172.21.15.47:0/423295400 >> 172.21.15.47:0/423295400 conn(0x7f93040fcc20 msgr2=0x7f9304101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.152+0000 7f9308e00640 1 -- 172.21.15.47:0/423295400 shutdown_connections 2024-03-21T11:41:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.152+0000 7f9308e00640 1 -- 172.21.15.47:0/423295400 wait complete. 2024-03-21T11:41:29.184 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:41:29.184 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:41:29.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:28 smithi102 ceph-mon[36748]: pgmap v944: 129 pgs: 129 active+clean; 469 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.4 MiB/s wr, 482 op/s 2024-03-21T11:41:29.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:28 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2395875708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:41:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:28 smithi100 ceph-mon[36245]: pgmap v944: 129 pgs: 129 active+clean; 469 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.4 MiB/s wr, 482 op/s 2024-03-21T11:41:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:28 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2395875708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:41:29.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.488+0000 7fd099a00640 1 -- 172.21.15.47:0/988444070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd094104ad0 msgr2=0x7fd094104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.488+0000 7fd099a00640 1 --2- 172.21.15.47:0/988444070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd094104ad0 0x7fd094104f50 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd080009860 tx=0x7fd08002bb70 comp rx=0 tx=0).stop 2024-03-21T11:41:29.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.488+0000 7fd099a00640 1 -- 172.21.15.47:0/988444070 shutdown_connections 2024-03-21T11:41:29.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.488+0000 7fd099a00640 1 --2- 172.21.15.47:0/988444070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd094105490 0x7fd09410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.488+0000 7fd099a00640 1 --2- 172.21.15.47:0/988444070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd094104ad0 0x7fd094104f50 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.488+0000 7fd099a00640 1 --2- 172.21.15.47:0/988444070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0941038e0 0x7fd094103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.488+0000 7fd099a00640 1 -- 172.21.15.47:0/988444070 >> 172.21.15.47:0/988444070 conn(0x7fd0940fee50 msgr2=0x7fd094101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.488+0000 7fd099a00640 1 -- 172.21.15.47:0/988444070 shutdown_connections 2024-03-21T11:41:29.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.488+0000 7fd099a00640 1 -- 172.21.15.47:0/988444070 wait complete. 2024-03-21T11:41:29.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.489+0000 7fd099a00640 1 Processor -- start 2024-03-21T11:41:29.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.489+0000 7fd099a00640 1 -- start start 2024-03-21T11:41:29.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd099a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0941038e0 0x7fd094199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd099a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd094104ad0 0x7fd09419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd099a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd094105490 0x7fd0941a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd099a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09410f540 con 0x7fd094105490 2024-03-21T11:41:29.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd099a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09410f3c0 con 0x7fd094104ad0 2024-03-21T11:41:29.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd099a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09410f6c0 con 0x7fd0941038e0 2024-03-21T11:41:29.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd092a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0941038e0 0x7fd094199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd092000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd094104ad0 0x7fd09419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd092a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0941038e0 0x7fd094199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34728/0 (socket says 172.21.15.47:34728) 2024-03-21T11:41:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd092000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd094104ad0 0x7fd09419a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38216/0 (socket says 172.21.15.47:38216) 2024-03-21T11:41:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd092a00640 1 -- 172.21.15.47:0/673745918 learned_addr learned my addr 172.21.15.47:0/673745918 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd093400640 1 --2- 172.21.15.47:0/673745918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd094105490 0x7fd0941a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd092000640 1 -- 172.21.15.47:0/673745918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0941038e0 msgr2=0x7fd094199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd092000640 1 --2- 172.21.15.47:0/673745918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0941038e0 0x7fd094199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd092000640 1 -- 172.21.15.47:0/673745918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd094105490 msgr2=0x7fd0941a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd092000640 1 --2- 172.21.15.47:0/673745918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd094105490 0x7fd0941a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd092000640 1 -- 172.21.15.47:0/673745918 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0941a17d0 con 0x7fd094104ad0 2024-03-21T11:41:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd093400640 1 --2- 172.21.15.47:0/673745918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd094105490 0x7fd0941a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:41:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.490+0000 7fd092000640 1 --2- 172.21.15.47:0/673745918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd094104ad0 0x7fd09419a0f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd08002c0b0 tx=0x7fd080032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.491+0000 7fd08b400640 1 -- 172.21.15.47:0/673745918 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080041020 con 0x7fd094104ad0 2024-03-21T11:41:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.491+0000 7fd099a00640 1 -- 172.21.15.47:0/673745918 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0941a1a60 con 0x7fd094104ad0 2024-03-21T11:41:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.491+0000 7fd08b400640 1 -- 172.21.15.47:0/673745918 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd08002c9f0 con 0x7fd094104ad0 2024-03-21T11:41:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.491+0000 7fd08b400640 1 -- 172.21.15.47:0/673745918 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080039040 con 0x7fd094104ad0 2024-03-21T11:41:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.491+0000 7fd099a00640 1 -- 172.21.15.47:0/673745918 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0941a1fa0 con 0x7fd094104ad0 2024-03-21T11:41:29.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.494+0000 7fd09c78d740 1 Processor -- start 2024-03-21T11:41:29.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.494+0000 7fd09c78d740 1 -- start start 2024-03-21T11:41:29.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.494+0000 7fd08b400640 1 -- 172.21.15.47:0/673745918 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd080044020 con 0x7fd094104ad0 2024-03-21T11:41:29.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd09c78d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c464810 0x55c72c464c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd09c78d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c465e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd09c78d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c4663a0 0x55c72c46cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd09c78d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72c3c92c0 con 0x55c72c464810 2024-03-21T11:41:29.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd09c78d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72c371c90 con 0x55c72c4663a0 2024-03-21T11:41:29.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd09c78d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72c3c5360 con 0x55c72c4659e0 2024-03-21T11:41:29.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd093400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c464810 0x55c72c464c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd092a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c465e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd092000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c4663a0 0x55c72c46cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd092a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c465e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34744/0 (socket says 172.21.15.47:34744) 2024-03-21T11:41:29.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd093400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c464810 0x55c72c464c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48626/0 (socket says 172.21.15.47:48626) 2024-03-21T11:41:29.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd092a00640 1 -- 172.21.15.47:0/1923826343 learned_addr learned my addr 172.21.15.47:0/1923826343 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd092000640 1 -- 172.21.15.47:0/1923826343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 msgr2=0x55c72c465e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd092000640 1 --2- 172.21.15.47:0/1923826343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c465e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd092000640 1 -- 172.21.15.47:0/1923826343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c464810 msgr2=0x55c72c464c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd092000640 1 --2- 172.21.15.47:0/1923826343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c464810 0x55c72c464c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.495+0000 7fd092000640 1 -- 172.21.15.47:0/1923826343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c72c396080 con 0x55c72c4663a0 2024-03-21T11:41:29.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.496+0000 7fd092000640 1 --2- 172.21.15.47:0/1923826343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c4663a0 0x55c72c46cc30 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd0800ee6d0 tx=0x7fd0800f90e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaeaaa4bfcfcda7a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.496+0000 7fd07fe00640 1 -- 172.21.15.47:0/1923826343 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0800f9ae0 con 0x55c72c4663a0 2024-03-21T11:41:29.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.496+0000 7fd07fe00640 1 -- 172.21.15.47:0/1923826343 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0800fe030 con 0x55c72c4663a0 2024-03-21T11:41:29.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.496+0000 7fd07fe00640 1 -- 172.21.15.47:0/1923826343 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0801023d0 con 0x55c72c4663a0 2024-03-21T11:41:29.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.496+0000 7fd08b400640 1 --2- 172.21.15.47:0/673745918 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd0780a2fa0 0x7fd0780a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.496+0000 7fd08b400640 1 -- 172.21.15.47:0/673745918 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd0800f0fe0 con 0x7fd094104ad0 2024-03-21T11:41:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.497+0000 7fd09c78d740 1 -- 172.21.15.47:0/1923826343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c4663a0 msgr2=0x55c72c46cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.497+0000 7fd09c78d740 1 --2- 172.21.15.47:0/1923826343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c4663a0 0x55c72c46cc30 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd0800ee6d0 tx=0x7fd0800f90e0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.497+0000 7fd093400640 1 --2- 172.21.15.47:0/673745918 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd0780a2fa0 0x7fd0780a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.497+0000 7fd09c78d740 1 -- 172.21.15.47:0/1923826343 shutdown_connections 2024-03-21T11:41:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.497+0000 7fd09c78d740 1 --2- 172.21.15.47:0/1923826343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c4663a0 0x55c72c46cc30 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.497+0000 7fd09c78d740 1 --2- 172.21.15.47:0/1923826343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c465e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.497+0000 7fd09c78d740 1 --2- 172.21.15.47:0/1923826343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c464810 0x55c72c464c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.497+0000 7fd09c78d740 1 -- 172.21.15.47:0/1923826343 >> 172.21.15.47:0/1923826343 conn(0x55c72c45fc00 msgr2=0x55c72c460000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.497+0000 7fd09c78d740 1 -- 172.21.15.47:0/1923826343 shutdown_connections 2024-03-21T11:41:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.497+0000 7fd09c78d740 1 -- 172.21.15.47:0/1923826343 wait complete. 2024-03-21T11:41:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.497+0000 7fd093400640 1 --2- 172.21.15.47:0/673745918 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd0780a2fa0 0x7fd0780a5460 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x55c72c4658e0 tx=0x7fd08c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.497+0000 7fd09c78d740 1 -- start start 2024-03-21T11:41:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.498+0000 7fd09c78d740 1 Processor -- start 2024-03-21T11:41:29.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.498+0000 7fd09c78d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c464810 0x55c72c46e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.498+0000 7fd09c78d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c46ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.498+0000 7fd09c78d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c4663a0 0x55c72c475d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.498+0000 7fd09c78d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72c46d170 con 0x55c72c4663a0 2024-03-21T11:41:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.498+0000 7fd09c78d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72c3850d0 con 0x55c72c464810 2024-03-21T11:41:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.498+0000 7fd09c78d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72c46d2f0 con 0x55c72c4659e0 2024-03-21T11:41:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.498+0000 7fd09c78d740 10 client.0 _ll_register_callbacks cb 0x55c72c389e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.498+0000 7fd092a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c464810 0x55c72c46e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.498+0000 7fd092000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c46ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.498+0000 7fd092a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c464810 0x55c72c46e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38234/0 (socket says 172.21.15.47:38234) 2024-03-21T11:41:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.498+0000 7fd092a00640 1 -- 172.21.15.47:0/2515418463 learned_addr learned my addr 172.21.15.47:0/2515418463 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.499+0000 7fd093400640 1 --2- 172.21.15.47:0/2515418463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c4663a0 0x55c72c475d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.499+0000 7fd092000640 1 -- 172.21.15.47:0/2515418463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c464810 msgr2=0x55c72c46e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.499+0000 7fd092000640 1 --2- 172.21.15.47:0/2515418463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c464810 0x55c72c46e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.499+0000 7fd092000640 1 -- 172.21.15.47:0/2515418463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c4663a0 msgr2=0x55c72c475d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.499+0000 7fd092000640 1 --2- 172.21.15.47:0/2515418463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c4663a0 0x55c72c475d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.499+0000 7fd092000640 1 -- 172.21.15.47:0/2515418463 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c72c0dd3f0 con 0x55c72c4659e0 2024-03-21T11:41:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.499+0000 7fd092a00640 1 --2- 172.21.15.47:0/2515418463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c464810 0x55c72c46e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:41:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.499+0000 7fd092000640 1 --2- 172.21.15.47:0/2515418463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c46ed40 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd080000c00 tx=0x7fd08002c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82694af693ce2011 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.499+0000 7fd07cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:41:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.499+0000 7fd07cc00640 1 -- 172.21.15.47:0/2515418463 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0800b9600 con 0x55c72c4659e0 2024-03-21T11:41:29.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.499+0000 7fd09c78d740 1 -- 172.21.15.47:0/2515418463 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0800fe030 con 0x55c72c4659e0 2024-03-21T11:41:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.499+0000 7fd07cc00640 1 -- 172.21.15.47:0/2515418463 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd0800ee840 con 0x55c72c4659e0 2024-03-21T11:41:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.500+0000 7fd07cc00640 1 -- 172.21.15.47:0/2515418463 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd080032640 con 0x55c72c4659e0 2024-03-21T11:41:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.500+0000 7fd09c78d740 10 client.31384 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.500+0000 7fd09c78d740 1 -- 172.21.15.47:0/2515418463 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c72c1a77d0 con 0x55c72c4659e0 2024-03-21T11:41:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.500+0000 7fd07cc00640 1 -- 172.21.15.47:0/2515418463 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0800fe030 con 0x55c72c4659e0 2024-03-21T11:41:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.500+0000 7fd07cc00640 1 -- 172.21.15.47:0/2515418463 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd08003c030 con 0x55c72c4659e0 2024-03-21T11:41:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.500+0000 7fd09c78d740 10 client.31384 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.500+0000 7fd09c78d740 10 client.31384 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:41:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.500+0000 7fd09c78d740 20 client.31384 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.500+0000 7fd09c78d740 1 --2- 172.21.15.47:0/2515418463 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c72c479ff0 0x55c72c47c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.500+0000 7fd09c78d740 4 client.31384 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:41:29.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.500+0000 7fd09c78d740 1 -- 172.21.15.47:0/2515418463 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c72c3d6800 con 0x55c72c479ff0 2024-03-21T11:41:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.501+0000 7fd092a00640 1 --2- 172.21.15.47:0/2515418463 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c72c479ff0 0x55c72c47c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.501+0000 7fd092a00640 1 --2- 172.21.15.47:0/2515418463 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c72c479ff0 0x55c72c47c410 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6a80015dc9b4f303 server_cookie=3a95c76f945a741f in_seq=0 out_seq=0 2024-03-21T11:41:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.501+0000 7fd07cc00640 10 client.31384 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:41:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.502+0000 7fd07cc00640 1 -- 172.21.15.47:0/2515418463 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c72c3d6800 con 0x55c72c479ff0 2024-03-21T11:41:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.502+0000 7fd07cc00640 10 client.31384 handle_command_reply: tid=0 2024-03-21T11:41:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.502+0000 7fd09c78d740 1 client.31384 shutdown 2024-03-21T11:41:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.502+0000 7fd09c78d740 1 -- 172.21.15.47:0/2515418463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 msgr2=0x55c72c46ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.502+0000 7fd09c78d740 1 --2- 172.21.15.47:0/2515418463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c46ed40 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd080000c00 tx=0x7fd08002c390 comp rx=0 tx=0).stop 2024-03-21T11:41:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.502+0000 7fd09c78d740 1 -- 172.21.15.47:0/2515418463 shutdown_connections 2024-03-21T11:41:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.502+0000 7fd09c78d740 1 --2- 172.21.15.47:0/2515418463 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c72c479ff0 0x55c72c47c410 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.502+0000 7fd09c78d740 1 --2- 172.21.15.47:0/2515418463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c4663a0 0x55c72c475d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.502+0000 7fd09c78d740 1 --2- 172.21.15.47:0/2515418463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c46ed40 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.502+0000 7fd07cc00640 0 client.31384 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:41:29.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.502+0000 7fd09c78d740 1 --2- 172.21.15.47:0/2515418463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c464810 0x55c72c46e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.502+0000 7fd09c78d740 1 -- 172.21.15.47:0/2515418463 >> 172.21.15.47:0/2515418463 conn(0x55c72c45fc00 msgr2=0x55c72c45eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.503+0000 7fd09c78d740 1 -- 172.21.15.47:0/2515418463 shutdown_connections 2024-03-21T11:41:29.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.503+0000 7fd09c78d740 1 -- 172.21.15.47:0/2515418463 wait complete. 2024-03-21T11:41:29.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.503+0000 7fd09c78d740 20 client.31384 trim_cache size 0 max 16384 2024-03-21T11:41:29.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.518+0000 7fd09c78d740 1 Processor -- start 2024-03-21T11:41:29.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.518+0000 7fd09c78d740 1 -- start start 2024-03-21T11:41:29.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd09c78d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c464810 0x55c72c46de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd09c78d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c475d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd09c78d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c4663a0 0x55c72c480cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd09c78d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0941ac1c0 con 0x55c72c464810 2024-03-21T11:41:29.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd09c78d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0941adb40 con 0x55c72c4663a0 2024-03-21T11:41:29.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd09c78d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0941ac820 con 0x55c72c4659e0 2024-03-21T11:41:29.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd092000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c475d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd092a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c464810 0x55c72c46de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd092000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c475d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34766/0 (socket says 172.21.15.47:34766) 2024-03-21T11:41:29.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd092000640 1 -- 172.21.15.47:0/1362357284 learned_addr learned my addr 172.21.15.47:0/1362357284 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd093400640 1 --2- 172.21.15.47:0/1362357284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c4663a0 0x55c72c480cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd092000640 1 -- 172.21.15.47:0/1362357284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c4663a0 msgr2=0x55c72c480cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd092000640 1 --2- 172.21.15.47:0/1362357284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c4663a0 0x55c72c480cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd092000640 1 -- 172.21.15.47:0/1362357284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c464810 msgr2=0x55c72c46de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd092000640 1 --2- 172.21.15.47:0/1362357284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c464810 0x55c72c46de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.519+0000 7fd092000640 1 -- 172.21.15.47:0/1362357284 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c72c45b700 con 0x55c72c4659e0 2024-03-21T11:41:29.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.520+0000 7fd092000640 1 --2- 172.21.15.47:0/1362357284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c475d40 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd08002c420 tx=0x7fd0800f48f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c35d5b1dbe9fc7e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.520+0000 7fd07f400640 1 -- 172.21.15.47:0/1362357284 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0800bccc0 con 0x55c72c4659e0 2024-03-21T11:41:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.520+0000 7fd07f400640 1 -- 172.21.15.47:0/1362357284 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0800fe030 con 0x55c72c4659e0 2024-03-21T11:41:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.520+0000 7fd07f400640 1 -- 172.21.15.47:0/1362357284 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0800bcea0 con 0x55c72c4659e0 2024-03-21T11:41:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.521+0000 7fd09c78d740 1 -- 172.21.15.47:0/1362357284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 msgr2=0x55c72c475d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.521+0000 7fd09c78d740 1 --2- 172.21.15.47:0/1362357284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c475d40 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd08002c420 tx=0x7fd0800f48f0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.521+0000 7fd09c78d740 1 -- 172.21.15.47:0/1362357284 shutdown_connections 2024-03-21T11:41:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.521+0000 7fd09c78d740 1 --2- 172.21.15.47:0/1362357284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c4663a0 0x55c72c480cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.521+0000 7fd09c78d740 1 --2- 172.21.15.47:0/1362357284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4659e0 0x55c72c475d40 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.521+0000 7fd09c78d740 1 --2- 172.21.15.47:0/1362357284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c464810 0x55c72c46de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.521+0000 7fd09c78d740 1 -- 172.21.15.47:0/1362357284 >> 172.21.15.47:0/1362357284 conn(0x55c72c45fc00 msgr2=0x55c72c4636f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.521+0000 7fd09c78d740 1 -- 172.21.15.47:0/1362357284 shutdown_connections 2024-03-21T11:41:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.521+0000 7fd09c78d740 1 -- 172.21.15.47:0/1362357284 wait complete. 2024-03-21T11:41:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.521+0000 7fd09c78d740 1 -- start start 2024-03-21T11:41:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.522+0000 7fd09c78d740 1 Processor -- start 2024-03-21T11:41:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.522+0000 7fd09c78d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c464810 0x55c72c471970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.522+0000 7fd09c78d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c4659e0 0x55c72c471eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.522+0000 7fd09c78d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4663a0 0x55c72c474e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.522+0000 7fd09c78d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0941ad4e0 con 0x55c72c4659e0 2024-03-21T11:41:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.522+0000 7fd09c78d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0941ace80 con 0x55c72c464810 2024-03-21T11:41:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.522+0000 7fd09c78d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0941aa380 con 0x55c72c4663a0 2024-03-21T11:41:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.522+0000 7fd09c78d740 10 client.0 _ll_register_callbacks cb 0x7fd08c0099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.522+0000 7fd092a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c464810 0x55c72c471970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.522+0000 7fd092a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c464810 0x55c72c471970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38248/0 (socket says 172.21.15.47:38248) 2024-03-21T11:41:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.522+0000 7fd092a00640 1 -- 172.21.15.47:0/3744231759 learned_addr learned my addr 172.21.15.47:0/3744231759 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.522+0000 7fd092000640 1 --2- 172.21.15.47:0/3744231759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c4659e0 0x55c72c471eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.522+0000 7fd093400640 1 --2- 172.21.15.47:0/3744231759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4663a0 0x55c72c474e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.523+0000 7fd092a00640 1 -- 172.21.15.47:0/3744231759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4663a0 msgr2=0x55c72c474e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.523+0000 7fd092a00640 1 --2- 172.21.15.47:0/3744231759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4663a0 0x55c72c474e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.523+0000 7fd092a00640 1 -- 172.21.15.47:0/3744231759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c4659e0 msgr2=0x55c72c471eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.523+0000 7fd092a00640 1 --2- 172.21.15.47:0/3744231759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c4659e0 0x55c72c471eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.523+0000 7fd092a00640 1 -- 172.21.15.47:0/3744231759 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c72c475470 con 0x55c72c464810 2024-03-21T11:41:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.523+0000 7fd092a00640 1 --2- 172.21.15.47:0/3744231759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c464810 0x55c72c471970 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x55c72c47cc30 tx=0x7fd084012f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3e99e76eda22dab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.523+0000 7fd07cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:41:29.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.523+0000 7fd07cc00640 1 -- 172.21.15.47:0/3744231759 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0840222d0 con 0x55c72c464810 2024-03-21T11:41:29.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.523+0000 7fd07cc00640 1 -- 172.21.15.47:0/3744231759 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd084005020 con 0x55c72c464810 2024-03-21T11:41:29.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.523+0000 7fd09c78d740 1 -- 172.21.15.47:0/3744231759 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0800fe030 con 0x55c72c464810 2024-03-21T11:41:29.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.524+0000 7fd07cc00640 1 -- 172.21.15.47:0/3744231759 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd084031050 con 0x55c72c464810 2024-03-21T11:41:29.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.524+0000 7fd09c78d740 10 client.31400 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:29.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.524+0000 7fd09c78d740 1 -- 172.21.15.47:0/3744231759 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c72c4d5de0 con 0x55c72c464810 2024-03-21T11:41:29.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.524+0000 7fd07cc00640 1 -- 172.21.15.47:0/3744231759 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0800fe030 con 0x55c72c464810 2024-03-21T11:41:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.524+0000 7fd07cc00640 1 -- 172.21.15.47:0/3744231759 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd084010430 con 0x55c72c464810 2024-03-21T11:41:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.524+0000 7fd09c78d740 10 client.31400 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.524+0000 7fd09c78d740 10 client.31400 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:41:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.524+0000 7fd09c78d740 20 client.31400 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.524+0000 7fd09c78d740 1 --2- 172.21.15.47:0/3744231759 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c72c4d6600 0x55c72c4d8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.524+0000 7fd09c78d740 4 client.31400 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:41:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.524+0000 7fd09c78d740 1 -- 172.21.15.47:0/3744231759 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c72c46e5f0 con 0x55c72c4d6600 2024-03-21T11:41:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd092000640 1 --2- 172.21.15.47:0/3744231759 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c72c4d6600 0x55c72c4d8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd092000640 1 --2- 172.21.15.47:0/3744231759 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c72c4d6600 0x55c72c4d8ac0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ab614446503538a server_cookie=b8a58b2b961447aa in_seq=0 out_seq=0 2024-03-21T11:41:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd07cc00640 10 client.31400 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:41:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd07cc00640 1 -- 172.21.15.47:0/3744231759 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c72c46e5f0 con 0x55c72c4d6600 2024-03-21T11:41:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd07cc00640 10 client.31400 handle_command_reply: tid=0 2024-03-21T11:41:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd09c78d740 1 client.31400 shutdown 2024-03-21T11:41:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd09c78d740 1 -- 172.21.15.47:0/3744231759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c464810 msgr2=0x55c72c471970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd09c78d740 1 --2- 172.21.15.47:0/3744231759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c464810 0x55c72c471970 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x55c72c47cc30 tx=0x7fd084012f30 comp rx=0 tx=0).stop 2024-03-21T11:41:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd09c78d740 1 -- 172.21.15.47:0/3744231759 shutdown_connections 2024-03-21T11:41:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd09c78d740 1 --2- 172.21.15.47:0/3744231759 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c72c4d6600 0x55c72c4d8ac0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd09c78d740 1 --2- 172.21.15.47:0/3744231759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c72c4663a0 0x55c72c474e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd09c78d740 1 --2- 172.21.15.47:0/3744231759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c72c4659e0 0x55c72c471eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd07cc00640 0 client.31400 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:41:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd09c78d740 1 --2- 172.21.15.47:0/3744231759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c72c464810 0x55c72c471970 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.525+0000 7fd09c78d740 1 -- 172.21.15.47:0/3744231759 >> 172.21.15.47:0/3744231759 conn(0x55c72c45fc00 msgr2=0x55c72c4636f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.526+0000 7fd09c78d740 1 -- 172.21.15.47:0/3744231759 shutdown_connections 2024-03-21T11:41:29.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.526+0000 7fd09c78d740 1 -- 172.21.15.47:0/3744231759 wait complete. 2024-03-21T11:41:29.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.526+0000 7fd09c78d740 20 client.31400 trim_cache size 0 max 16384 2024-03-21T11:41:29.536 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:41:29.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.526+0000 7fd099a00640 1 -- 172.21.15.47:0/673745918 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd0780a2fa0 msgr2=0x7fd0780a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.526+0000 7fd099a00640 1 --2- 172.21.15.47:0/673745918 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd0780a2fa0 0x7fd0780a5460 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x55c72c4658e0 tx=0x7fd08c009500 comp rx=0 tx=0).stop 2024-03-21T11:41:29.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.526+0000 7fd099a00640 1 -- 172.21.15.47:0/673745918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd094104ad0 msgr2=0x7fd09419a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.526+0000 7fd099a00640 1 --2- 172.21.15.47:0/673745918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd094104ad0 0x7fd09419a0f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd08002c0b0 tx=0x7fd080032430 comp rx=0 tx=0).stop 2024-03-21T11:41:29.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.526+0000 7fd099a00640 1 -- 172.21.15.47:0/673745918 shutdown_connections 2024-03-21T11:41:29.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.526+0000 7fd099a00640 1 --2- 172.21.15.47:0/673745918 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd0780a2fa0 0x7fd0780a5460 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.526+0000 7fd099a00640 1 --2- 172.21.15.47:0/673745918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd094105490 0x7fd0941a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.526+0000 7fd099a00640 1 --2- 172.21.15.47:0/673745918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd094104ad0 0x7fd09419a0f0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.527+0000 7fd099a00640 1 --2- 172.21.15.47:0/673745918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0941038e0 0x7fd094199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.527+0000 7fd099a00640 1 -- 172.21.15.47:0/673745918 >> 172.21.15.47:0/673745918 conn(0x7fd0940fee50 msgr2=0x7fd094107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.527+0000 7fd099a00640 1 -- 172.21.15.47:0/673745918 shutdown_connections 2024-03-21T11:41:29.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.527+0000 7fd099a00640 1 -- 172.21.15.47:0/673745918 wait complete. 2024-03-21T11:41:29.546 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:41:29.547 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:41:29.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.818+0000 7f7066c00640 1 -- 172.21.15.47:0/871624363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7060072ce0 msgr2=0x7f7060073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.818+0000 7f7066c00640 1 --2- 172.21.15.47:0/871624363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7060072ce0 0x7f7060073160 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f7054009860 tx=0x7f705402bb70 comp rx=0 tx=0).stop 2024-03-21T11:41:29.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.818+0000 7f7066c00640 1 -- 172.21.15.47:0/871624363 shutdown_connections 2024-03-21T11:41:29.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.818+0000 7f7066c00640 1 --2- 172.21.15.47:0/871624363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f706010fa70 0x7f7060111ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.818+0000 7f7066c00640 1 --2- 172.21.15.47:0/871624363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7060072ce0 0x7f7060073160 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.818+0000 7f7066c00640 1 --2- 172.21.15.47:0/871624363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7060109340 0x7f7060072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.818+0000 7f7066c00640 1 -- 172.21.15.47:0/871624363 >> 172.21.15.47:0/871624363 conn(0x7f706006e580 msgr2=0x7f70600709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.820+0000 7f7066c00640 1 -- 172.21.15.47:0/871624363 shutdown_connections 2024-03-21T11:41:29.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.820+0000 7f7066c00640 1 -- 172.21.15.47:0/871624363 wait complete. 2024-03-21T11:41:29.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.821+0000 7f7066c00640 1 Processor -- start 2024-03-21T11:41:29.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.821+0000 7f7066c00640 1 -- start start 2024-03-21T11:41:29.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.822+0000 7f7066c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7060072ce0 0x7f7060119190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.822+0000 7f7066c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7060109340 0x7f70601196d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.822+0000 7f7066c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706010fa70 0x7f706011a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.822+0000 7f7066c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70601137f0 con 0x7f7060072ce0 2024-03-21T11:41:29.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.822+0000 7f7066c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7060113670 con 0x7f706010fa70 2024-03-21T11:41:29.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.822+0000 7f7066c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7060113970 con 0x7f7060109340 2024-03-21T11:41:29.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.822+0000 7f7064e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7060109340 0x7f70601196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.823+0000 7f7064e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7060109340 0x7f70601196d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34798/0 (socket says 172.21.15.47:34798) 2024-03-21T11:41:29.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.823+0000 7f7064e00640 1 -- 172.21.15.47:0/700073350 learned_addr learned my addr 172.21.15.47:0/700073350 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.823+0000 7f7066200640 1 --2- 172.21.15.47:0/700073350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706010fa70 0x7f706011a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.823+0000 7f7065800640 1 --2- 172.21.15.47:0/700073350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7060072ce0 0x7f7060119190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.824+0000 7f7065800640 1 -- 172.21.15.47:0/700073350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7060109340 msgr2=0x7f70601196d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.824+0000 7f7065800640 1 --2- 172.21.15.47:0/700073350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7060109340 0x7f70601196d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.824+0000 7f7065800640 1 -- 172.21.15.47:0/700073350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706010fa70 msgr2=0x7f706011a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.824+0000 7f7065800640 1 --2- 172.21.15.47:0/700073350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706010fa70 0x7f706011a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.824+0000 7f7065800640 1 -- 172.21.15.47:0/700073350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706011acd0 con 0x7f7060072ce0 2024-03-21T11:41:29.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.825+0000 7f7065800640 1 --2- 172.21.15.47:0/700073350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7060072ce0 0x7f7060119190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f704c005fd0 tx=0x7f704c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.828+0000 7f705e000640 1 -- 172.21.15.47:0/700073350 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c00ccc0 con 0x7f7060072ce0 2024-03-21T11:41:29.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.828+0000 7f705e000640 1 -- 172.21.15.47:0/700073350 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f704c00ee70 con 0x7f7060072ce0 2024-03-21T11:41:29.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.828+0000 7f705e000640 1 -- 172.21.15.47:0/700073350 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c010430 con 0x7f7060072ce0 2024-03-21T11:41:29.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.828+0000 7f7066c00640 1 -- 172.21.15.47:0/700073350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706011afc0 con 0x7f7060072ce0 2024-03-21T11:41:29.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.828+0000 7f7066c00640 1 -- 172.21.15.47:0/700073350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70601b7390 con 0x7f7060072ce0 2024-03-21T11:41:29.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.835+0000 7f706af6a740 1 Processor -- start 2024-03-21T11:41:29.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.835+0000 7f706af6a740 1 -- start start 2024-03-21T11:41:29.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.835+0000 7f705e000640 1 -- 172.21.15.47:0/700073350 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f704c003ef0 con 0x7f7060072ce0 2024-03-21T11:41:29.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.835+0000 7f706af6a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c348899f0 0x564c34889e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.835+0000 7f706af6a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488abc0 0x564c3488b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.835+0000 7f706af6a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c3488b580 0x564c34891e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.835+0000 7f706af6a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c347ee2c0 con 0x564c3488b580 2024-03-21T11:41:29.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.835+0000 7f706af6a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c34796c90 con 0x564c3488abc0 2024-03-21T11:41:29.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.835+0000 7f706af6a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c347ea360 con 0x564c348899f0 2024-03-21T11:41:29.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.836+0000 7f7066200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c348899f0 0x564c34889e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.836+0000 7f7066200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c348899f0 0x564c34889e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34808/0 (socket says 172.21.15.47:34808) 2024-03-21T11:41:29.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.836+0000 7f7066200640 1 -- 172.21.15.47:0/2637203611 learned_addr learned my addr 172.21.15.47:0/2637203611 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.836+0000 7f7065800640 1 --2- 172.21.15.47:0/2637203611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488abc0 0x564c3488b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.837+0000 7f7064e00640 1 --2- 172.21.15.47:0/2637203611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c3488b580 0x564c34891e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.837+0000 7f7066200640 1 -- 172.21.15.47:0/2637203611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488abc0 msgr2=0x564c3488b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.837+0000 7f7066200640 1 --2- 172.21.15.47:0/2637203611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488abc0 0x564c3488b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.837+0000 7f7066200640 1 -- 172.21.15.47:0/2637203611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c3488b580 msgr2=0x564c34891e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.837+0000 7f7066200640 1 --2- 172.21.15.47:0/2637203611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c3488b580 0x564c34891e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.837+0000 7f7066200640 1 -- 172.21.15.47:0/2637203611 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c347bb080 con 0x564c348899f0 2024-03-21T11:41:29.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.838+0000 7f7064e00640 1 --2- 172.21.15.47:0/2637203611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c3488b580 0x564c34891e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:41:29.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.838+0000 7f7066200640 1 --2- 172.21.15.47:0/2637203611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c348899f0 0x564c34889e10 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f7058009eb0 tx=0x7f7058010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=461b4aeabb3af5cb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.839+0000 7f7052a00640 1 -- 172.21.15.47:0/2637203611 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058016070 con 0x564c348899f0 2024-03-21T11:41:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.839+0000 7f7052a00640 1 -- 172.21.15.47:0/2637203611 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7058004d20 con 0x564c348899f0 2024-03-21T11:41:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.839+0000 7f7052a00640 1 -- 172.21.15.47:0/2637203611 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70580128f0 con 0x564c348899f0 2024-03-21T11:41:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.840+0000 7f706af6a740 1 -- 172.21.15.47:0/2637203611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c348899f0 msgr2=0x564c34889e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.840+0000 7f706af6a740 1 --2- 172.21.15.47:0/2637203611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c348899f0 0x564c34889e10 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f7058009eb0 tx=0x7f7058010740 comp rx=0 tx=0).stop 2024-03-21T11:41:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.840+0000 7f706af6a740 1 -- 172.21.15.47:0/2637203611 shutdown_connections 2024-03-21T11:41:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.840+0000 7f705e000640 1 --2- 172.21.15.47:0/700073350 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f70480a3050 0x7f70480a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.840+0000 7f706af6a740 1 --2- 172.21.15.47:0/2637203611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c3488b580 0x564c34891e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.840+0000 7f706af6a740 1 --2- 172.21.15.47:0/2637203611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488abc0 0x564c3488b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.840+0000 7f706af6a740 1 --2- 172.21.15.47:0/2637203611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c348899f0 0x564c34889e10 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.841+0000 7f706af6a740 1 -- 172.21.15.47:0/2637203611 >> 172.21.15.47:0/2637203611 conn(0x564c34884de0 msgr2=0x564c348851e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.841+0000 7f706af6a740 1 -- 172.21.15.47:0/2637203611 shutdown_connections 2024-03-21T11:41:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.841+0000 7f706af6a740 1 -- 172.21.15.47:0/2637203611 wait complete. 2024-03-21T11:41:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.841+0000 7f7066200640 1 --2- 172.21.15.47:0/700073350 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f70480a3050 0x7f70480a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.842+0000 7f706af6a740 1 -- start start 2024-03-21T11:41:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.842+0000 7f706af6a740 1 Processor -- start 2024-03-21T11:41:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.842+0000 7f705e000640 1 -- 172.21.15.47:0/700073350 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f704c0d3060 con 0x7f7060072ce0 2024-03-21T11:41:29.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.843+0000 7f706af6a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c348899f0 0x564c34893c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.843+0000 7f706af6a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488abc0 0x564c348941c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.844+0000 7f706af6a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c3488b580 0x564c3489b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.844+0000 7f706af6a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c34892590 con 0x564c3488b580 2024-03-21T11:41:29.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.844+0000 7f706af6a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c347aa0d0 con 0x564c3488abc0 2024-03-21T11:41:29.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.844+0000 7f706af6a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c34892710 con 0x564c348899f0 2024-03-21T11:41:29.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.844+0000 7f706af6a740 10 client.0 _ll_register_callbacks cb 0x564c347aee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:29.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.844+0000 7f7064e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488abc0 0x564c348941c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.844+0000 7f7064e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488abc0 0x564c348941c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38268/0 (socket says 172.21.15.47:38268) 2024-03-21T11:41:29.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.844+0000 7f7064e00640 1 -- 172.21.15.47:0/4116176640 learned_addr learned my addr 172.21.15.47:0/4116176640 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.845+0000 7f7066200640 1 --2- 172.21.15.47:0/4116176640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c3488b580 0x564c3489b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.845+0000 7f7066200640 1 --2- 172.21.15.47:0/700073350 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f70480a3050 0x7f70480a5510 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7058010c40 tx=0x7f7058005780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.845+0000 7f7064e00640 1 -- 172.21.15.47:0/4116176640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c348899f0 msgr2=0x564c34893c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.845+0000 7f7064e00640 1 --2- 172.21.15.47:0/4116176640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c348899f0 0x564c34893c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.845+0000 7f7064e00640 1 -- 172.21.15.47:0/4116176640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c3488b580 msgr2=0x564c3489b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.845+0000 7f7064e00640 1 --2- 172.21.15.47:0/4116176640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c3488b580 0x564c3489b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.845+0000 7f7064e00640 1 -- 172.21.15.47:0/4116176640 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c345023f0 con 0x564c3488abc0 2024-03-21T11:41:29.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.846+0000 7f7064e00640 1 --2- 172.21.15.47:0/4116176640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488abc0 0x564c348941c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f7054009830 tx=0x7f7054005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a64990d5b67ec31 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.846+0000 7f7047400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:41:29.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.846+0000 7f7047400640 1 -- 172.21.15.47:0/4116176640 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054039070 con 0x564c3488abc0 2024-03-21T11:41:29.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.846+0000 7f7047400640 1 -- 172.21.15.47:0/4116176640 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7054004040 con 0x564c3488abc0 2024-03-21T11:41:29.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.847+0000 7f706af6a740 1 -- 172.21.15.47:0/4116176640 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7058004d20 con 0x564c3488abc0 2024-03-21T11:41:29.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.848+0000 7f7047400640 1 -- 172.21.15.47:0/4116176640 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7054004240 con 0x564c3488abc0 2024-03-21T11:41:29.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.848+0000 7f706af6a740 10 client.31412 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:29.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.848+0000 7f706af6a740 1 -- 172.21.15.47:0/4116176640 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564c345cc7d0 con 0x564c3488abc0 2024-03-21T11:41:29.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.849+0000 7f7047400640 1 -- 172.21.15.47:0/4116176640 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7058004d20 con 0x564c3488abc0 2024-03-21T11:41:29.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.849+0000 7f7047400640 1 -- 172.21.15.47:0/4116176640 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7054047030 con 0x564c3488abc0 2024-03-21T11:41:29.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.849+0000 7f706af6a740 10 client.31412 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:29.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.849+0000 7f706af6a740 10 client.31412 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:41:29.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.849+0000 7f706af6a740 20 client.31412 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:29.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.849+0000 7f706af6a740 1 --2- 172.21.15.47:0/4116176640 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x564c3489f700 0x564c348a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.849+0000 7f706af6a740 4 client.31412 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:41:29.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.849+0000 7f706af6a740 1 -- 172.21.15.47:0/4116176640 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c347fb800 con 0x564c3489f700 2024-03-21T11:41:29.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.853+0000 7f7065800640 1 --2- 172.21.15.47:0/4116176640 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x564c3489f700 0x564c348a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.854+0000 7f7065800640 1 --2- 172.21.15.47:0/4116176640 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x564c3489f700 0x564c348a1b30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=817c39a57f6aaab4 server_cookie=a7b413c4a02dfd83 in_seq=0 out_seq=0 2024-03-21T11:41:29.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.854+0000 7f7047400640 10 client.31412 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:41:29.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.855+0000 7f7047400640 1 -- 172.21.15.47:0/4116176640 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564c347fb800 con 0x564c3489f700 2024-03-21T11:41:29.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.856+0000 7f7047400640 10 client.31412 handle_command_reply: tid=0 2024-03-21T11:41:29.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.856+0000 7f706af6a740 1 client.31412 shutdown 2024-03-21T11:41:29.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.858+0000 7f706af6a740 1 -- 172.21.15.47:0/4116176640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488abc0 msgr2=0x564c348941c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.858+0000 7f706af6a740 1 --2- 172.21.15.47:0/4116176640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488abc0 0x564c348941c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f7054009830 tx=0x7f7054005d20 comp rx=0 tx=0).stop 2024-03-21T11:41:29.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.859+0000 7f706af6a740 1 -- 172.21.15.47:0/4116176640 shutdown_connections 2024-03-21T11:41:29.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.859+0000 7f706af6a740 1 --2- 172.21.15.47:0/4116176640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c3488b580 0x564c3489b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.859+0000 7f706af6a740 1 --2- 172.21.15.47:0/4116176640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488abc0 0x564c348941c0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.859+0000 7f706af6a740 1 --2- 172.21.15.47:0/4116176640 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x564c3489f700 0x564c348a1b30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.859+0000 7f706af6a740 1 --2- 172.21.15.47:0/4116176640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c348899f0 0x564c34893c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.859+0000 7f7047400640 0 client.31412 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:41:29.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.859+0000 7f706af6a740 1 -- 172.21.15.47:0/4116176640 >> 172.21.15.47:0/4116176640 conn(0x564c34884de0 msgr2=0x564c34884160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.860+0000 7f706af6a740 1 -- 172.21.15.47:0/4116176640 shutdown_connections 2024-03-21T11:41:29.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.860+0000 7f706af6a740 1 -- 172.21.15.47:0/4116176640 wait complete. 2024-03-21T11:41:29.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.860+0000 7f706af6a740 20 client.31412 trim_cache size 0 max 16384 2024-03-21T11:41:29.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.891+0000 7f706af6a740 1 Processor -- start 2024-03-21T11:41:29.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.891+0000 7f706af6a740 1 -- start start 2024-03-21T11:41:29.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.891+0000 7f706af6a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c348899f0 0x564c34883fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.892+0000 7f706af6a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c3488abc0 0x564c3489b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.892+0000 7f706af6a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488b580 0x564c348a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.892+0000 7f706af6a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70601c15c0 con 0x564c348899f0 2024-03-21T11:41:29.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.892+0000 7f706af6a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70601c2f40 con 0x564c3488b580 2024-03-21T11:41:29.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.892+0000 7f706af6a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70601c1c20 con 0x564c3488abc0 2024-03-21T11:41:29.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.893+0000 7f7066200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488b580 0x564c348a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.893+0000 7f7064e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c3488abc0 0x564c3489b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.893+0000 7f7064e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c3488abc0 0x564c3489b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34826/0 (socket says 172.21.15.47:34826) 2024-03-21T11:41:29.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.893+0000 7f7066200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488b580 0x564c348a63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38284/0 (socket says 172.21.15.47:38284) 2024-03-21T11:41:29.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.893+0000 7f7064e00640 1 -- 172.21.15.47:0/2553753505 learned_addr learned my addr 172.21.15.47:0/2553753505 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.893+0000 7f7065800640 1 --2- 172.21.15.47:0/2553753505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c348899f0 0x564c34883fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.894+0000 7f7066200640 1 -- 172.21.15.47:0/2553753505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c3488abc0 msgr2=0x564c3489b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.894+0000 7f7066200640 1 --2- 172.21.15.47:0/2553753505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c3488abc0 0x564c3489b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.894+0000 7f7066200640 1 -- 172.21.15.47:0/2553753505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c348899f0 msgr2=0x564c34883fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.894+0000 7f7066200640 1 --2- 172.21.15.47:0/2553753505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c348899f0 0x564c34883fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.894+0000 7f7066200640 1 -- 172.21.15.47:0/2553753505 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c34885d50 con 0x564c3488b580 2024-03-21T11:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.894+0000 7f7066200640 1 --2- 172.21.15.47:0/2553753505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488b580 0x564c348a63f0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f7058009e80 tx=0x7f70580097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f204a73bd6efd5bb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.895+0000 7f7052000640 1 -- 172.21.15.47:0/2553753505 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058036070 con 0x564c3488b580 2024-03-21T11:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.895+0000 7f7052000640 1 -- 172.21.15.47:0/2553753505 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70580094a0 con 0x564c3488b580 2024-03-21T11:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.895+0000 7f7052000640 1 -- 172.21.15.47:0/2553753505 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058016040 con 0x564c3488b580 2024-03-21T11:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.896+0000 7f706af6a740 1 -- 172.21.15.47:0/2553753505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488b580 msgr2=0x564c348a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.896+0000 7f706af6a740 1 --2- 172.21.15.47:0/2553753505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488b580 0x564c348a63f0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f7058009e80 tx=0x7f70580097a0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.897+0000 7f706af6a740 1 -- 172.21.15.47:0/2553753505 shutdown_connections 2024-03-21T11:41:29.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.898+0000 7f706af6a740 1 --2- 172.21.15.47:0/2553753505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488b580 0x564c348a63f0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.898+0000 7f706af6a740 1 --2- 172.21.15.47:0/2553753505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c3488abc0 0x564c3489b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.898+0000 7f706af6a740 1 --2- 172.21.15.47:0/2553753505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c348899f0 0x564c34883fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.898+0000 7f706af6a740 1 -- 172.21.15.47:0/2553753505 >> 172.21.15.47:0/2553753505 conn(0x564c34884de0 msgr2=0x564c348838d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.898+0000 7f706af6a740 1 -- 172.21.15.47:0/2553753505 shutdown_connections 2024-03-21T11:41:29.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.898+0000 7f706af6a740 1 -- 172.21.15.47:0/2553753505 wait complete. 2024-03-21T11:41:29.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.899+0000 7f706af6a740 1 -- start start 2024-03-21T11:41:29.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.899+0000 7f706af6a740 1 Processor -- start 2024-03-21T11:41:29.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.900+0000 7f706af6a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c348899f0 0x564c34896fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.900+0000 7f706af6a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c3488abc0 0x564c34897510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.900+0000 7f706af6a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488b580 0x564c3489a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.900+0000 7f706af6a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70601c28e0 con 0x564c348899f0 2024-03-21T11:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.900+0000 7f706af6a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70601c2280 con 0x564c3488b580 2024-03-21T11:41:29.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.900+0000 7f706af6a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70601bf780 con 0x564c3488abc0 2024-03-21T11:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.900+0000 7f706af6a740 10 client.0 _ll_register_callbacks cb 0x7f7054037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.901+0000 7f7066200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488b580 0x564c3489a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.901+0000 7f7066200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488b580 0x564c3489a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38286/0 (socket says 172.21.15.47:38286) 2024-03-21T11:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.901+0000 7f7066200640 1 -- 172.21.15.47:0/2950096831 learned_addr learned my addr 172.21.15.47:0/2950096831 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.901+0000 7f7064e00640 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c3488abc0 0x564c34897510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.901+0000 7f7065800640 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c348899f0 0x564c34896fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.901+0000 7f7066200640 1 -- 172.21.15.47:0/2950096831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c3488abc0 msgr2=0x564c34897510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.901+0000 7f7066200640 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c3488abc0 0x564c34897510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.901+0000 7f7066200640 1 -- 172.21.15.47:0/2950096831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c348899f0 msgr2=0x564c34896fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:29.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.901+0000 7f7066200640 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c348899f0 0x564c34896fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.902+0000 7f7066200640 1 -- 172.21.15.47:0/2950096831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c3489ab20 con 0x564c3488b580 2024-03-21T11:41:29.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.902+0000 7f7065800640 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c348899f0 0x564c34896fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:41:29.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.902+0000 7f7064e00640 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c3488abc0 0x564c34897510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:41:29.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.902+0000 7f7066200640 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488b580 0x564c3489a530 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f7058005ae0 tx=0x7f705801cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94976c9ff05daa36 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:29.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.902+0000 7f7047400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:41:29.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.903+0000 7f7047400640 1 -- 172.21.15.47:0/2950096831 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058036070 con 0x564c3488b580 2024-03-21T11:41:29.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.903+0000 7f706af6a740 1 -- 172.21.15.47:0/2950096831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70580094a0 con 0x564c3488b580 2024-03-21T11:41:29.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.903+0000 7f7047400640 1 -- 172.21.15.47:0/2950096831 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7058013ad0 con 0x564c3488b580 2024-03-21T11:41:29.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.906+0000 7f7047400640 1 -- 172.21.15.47:0/2950096831 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7058013cd0 con 0x564c3488b580 2024-03-21T11:41:29.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.906+0000 7f706af6a740 10 client.31418 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:29.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.906+0000 7f706af6a740 1 -- 172.21.15.47:0/2950096831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564c348fb680 con 0x564c3488b580 2024-03-21T11:41:29.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.907+0000 7f7047400640 1 -- 172.21.15.47:0/2950096831 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70580094a0 con 0x564c3488b580 2024-03-21T11:41:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.907+0000 7f7047400640 1 -- 172.21.15.47:0/2950096831 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f705800f040 con 0x564c3488b580 2024-03-21T11:41:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.908+0000 7f706af6a740 10 client.31418 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.908+0000 7f706af6a740 10 client.31418 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:41:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.908+0000 7f706af6a740 20 client.31418 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.908+0000 7f706af6a740 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x564c348fbe20 0x564c348fe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.908+0000 7f706af6a740 4 client.31418 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:41:29.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.908+0000 7f706af6a740 1 -- 172.21.15.47:0/2950096831 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c347f5b50 con 0x564c348fbe20 2024-03-21T11:41:29.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.910+0000 7f7065800640 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x564c348fbe20 0x564c348fe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:29.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.910+0000 7f7047400640 10 client.31418 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:41:29.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.911+0000 7f7065800640 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x564c348fbe20 0x564c348fe240 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=247711c878856629 server_cookie=77faafb2f661260e in_seq=0 out_seq=0 2024-03-21T11:41:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.915+0000 7f7047400640 1 -- 172.21.15.47:0/2950096831 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c347f5b50 con 0x564c348fbe20 2024-03-21T11:41:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.915+0000 7f7047400640 10 client.31418 handle_command_reply: tid=0 2024-03-21T11:41:29.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.915+0000 7f706af6a740 1 client.31418 shutdown 2024-03-21T11:41:29.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.915+0000 7f706af6a740 1 -- 172.21.15.47:0/2950096831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488b580 msgr2=0x564c3489a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.915+0000 7f706af6a740 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488b580 0x564c3489a530 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f7058005ae0 tx=0x7f705801cb40 comp rx=0 tx=0).stop 2024-03-21T11:41:29.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.916+0000 7f706af6a740 1 -- 172.21.15.47:0/2950096831 shutdown_connections 2024-03-21T11:41:29.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.916+0000 7f706af6a740 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c3488b580 0x564c3489a530 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.916+0000 7f706af6a740 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x564c348fbe20 0x564c348fe240 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.916+0000 7f706af6a740 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c3488abc0 0x564c34897510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.916+0000 7f706af6a740 1 --2- 172.21.15.47:0/2950096831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c348899f0 0x564c34896fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.916+0000 7f7047400640 0 client.31418 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:41:29.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.916+0000 7f706af6a740 1 -- 172.21.15.47:0/2950096831 >> 172.21.15.47:0/2950096831 conn(0x564c34884de0 msgr2=0x564c34884160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.916+0000 7f706af6a740 1 -- 172.21.15.47:0/2950096831 shutdown_connections 2024-03-21T11:41:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.916+0000 7f706af6a740 1 -- 172.21.15.47:0/2950096831 wait complete. 2024-03-21T11:41:29.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.916+0000 7f706af6a740 20 client.31418 trim_cache size 0 max 16384 2024-03-21T11:41:29.920 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:41:29.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.920+0000 7f7066c00640 1 -- 172.21.15.47:0/700073350 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f70480a3050 msgr2=0x7f70480a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.920+0000 7f7066c00640 1 --2- 172.21.15.47:0/700073350 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f70480a3050 0x7f70480a5510 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7058010c40 tx=0x7f7058005780 comp rx=0 tx=0).stop 2024-03-21T11:41:29.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.920+0000 7f7066c00640 1 -- 172.21.15.47:0/700073350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7060072ce0 msgr2=0x7f7060119190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:29.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.920+0000 7f7066c00640 1 --2- 172.21.15.47:0/700073350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7060072ce0 0x7f7060119190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f704c005fd0 tx=0x7f704c00ec20 comp rx=0 tx=0).stop 2024-03-21T11:41:29.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.920+0000 7f7066c00640 1 -- 172.21.15.47:0/700073350 shutdown_connections 2024-03-21T11:41:29.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.920+0000 7f7066c00640 1 --2- 172.21.15.47:0/700073350 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f70480a3050 0x7f70480a5510 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.920+0000 7f7066c00640 1 --2- 172.21.15.47:0/700073350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706010fa70 0x7f706011a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.920+0000 7f7066c00640 1 --2- 172.21.15.47:0/700073350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7060109340 0x7f70601196d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.920+0000 7f7066c00640 1 --2- 172.21.15.47:0/700073350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7060072ce0 0x7f7060119190 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:29.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.920+0000 7f7066c00640 1 -- 172.21.15.47:0/700073350 >> 172.21.15.47:0/700073350 conn(0x7f706006e580 msgr2=0x7f706010b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:29.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.920+0000 7f7066c00640 1 -- 172.21.15.47:0/700073350 shutdown_connections 2024-03-21T11:41:29.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:29.921+0000 7f7066c00640 1 -- 172.21.15.47:0/700073350 wait complete. 2024-03-21T11:41:29.949 INFO:tasks.workunit.client.0.smithi047.stdout: 10 117668 24.02 MB/sec execute 133 sec latency 333.678 ms 2024-03-21T11:41:29.951 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:41:29.951 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:41:30.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.270+0000 7f53e6800640 1 -- 172.21.15.47:0/1646132768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e0108910 msgr2=0x7f53e010ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.270+0000 7f53e6800640 1 --2- 172.21.15.47:0/1646132768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e0108910 0x7f53e010ad00 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009900 tx=0x7f53d802bdc0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.271+0000 7f53e6800640 1 -- 172.21.15.47:0/1646132768 shutdown_connections 2024-03-21T11:41:30.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.271+0000 7f53e6800640 1 --2- 172.21.15.47:0/1646132768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53e010dd50 0x7f53e0110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.272+0000 7f53e6800640 1 --2- 172.21.15.47:0/1646132768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e010b2d0 0x7f53e010d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.272+0000 7f53e6800640 1 --2- 172.21.15.47:0/1646132768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e0108910 0x7f53e010ad00 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.272+0000 7f53e6800640 1 -- 172.21.15.47:0/1646132768 >> 172.21.15.47:0/1646132768 conn(0x7f53e006e8a0 msgr2=0x7f53e0070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.272+0000 7f53e6800640 1 -- 172.21.15.47:0/1646132768 shutdown_connections 2024-03-21T11:41:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.272+0000 7f53e6800640 1 -- 172.21.15.47:0/1646132768 wait complete. 2024-03-21T11:41:30.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.274+0000 7f53e6800640 1 Processor -- start 2024-03-21T11:41:30.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.275+0000 7f53e6800640 1 -- start start 2024-03-21T11:41:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.275+0000 7f53e6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53e0108910 0x7f53e0117680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.275+0000 7f53e6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e010b2d0 0x7f53e0117bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.275+0000 7f53e6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e010dd50 0x7f53e01b6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.275+0000 7f53e6800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e011c140 con 0x7f53e0108910 2024-03-21T11:41:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.275+0000 7f53e6800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e011bfc0 con 0x7f53e010b2d0 2024-03-21T11:41:30.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.275+0000 7f53e6800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e011c2c0 con 0x7f53e010dd50 2024-03-21T11:41:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.276+0000 7f53df400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53e0108910 0x7f53e0117680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.276+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e010dd50 0x7f53e01b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.276+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e010dd50 0x7f53e01b6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34848/0 (socket says 172.21.15.47:34848) 2024-03-21T11:41:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.276+0000 7f53dfe00640 1 -- 172.21.15.47:0/3736608626 learned_addr learned my addr 172.21.15.47:0/3736608626 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.276+0000 7f53dea00640 1 --2- 172.21.15.47:0/3736608626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e010b2d0 0x7f53e0117bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.276+0000 7f53dfe00640 1 -- 172.21.15.47:0/3736608626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e010b2d0 msgr2=0x7f53e0117bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.276+0000 7f53dfe00640 1 --2- 172.21.15.47:0/3736608626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e010b2d0 0x7f53e0117bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.276+0000 7f53dfe00640 1 -- 172.21.15.47:0/3736608626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53e0108910 msgr2=0x7f53e0117680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.276+0000 7f53dfe00640 1 --2- 172.21.15.47:0/3736608626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53e0108910 0x7f53e0117680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.276+0000 7f53dfe00640 1 -- 172.21.15.47:0/3736608626 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e01b6d70 con 0x7f53e010dd50 2024-03-21T11:41:30.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.277+0000 7f53dfe00640 1 --2- 172.21.15.47:0/3736608626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e010dd50 0x7f53e01b6830 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f53c8005fd0 tx=0x7f53c800ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:30.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.282+0000 7f53d7e00640 1 -- 172.21.15.47:0/3736608626 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c800cbe0 con 0x7f53e010dd50 2024-03-21T11:41:30.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.282+0000 7f53d7e00640 1 -- 172.21.15.47:0/3736608626 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53c800cd80 con 0x7f53e010dd50 2024-03-21T11:41:30.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.282+0000 7f53d7e00640 1 -- 172.21.15.47:0/3736608626 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8015670 con 0x7f53e010dd50 2024-03-21T11:41:30.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.282+0000 7f53e6800640 1 -- 172.21.15.47:0/3736608626 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e01b6fa0 con 0x7f53e010dd50 2024-03-21T11:41:30.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.282+0000 7f53e6800640 1 -- 172.21.15.47:0/3736608626 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e01b7410 con 0x7f53e010dd50 2024-03-21T11:41:30.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.289+0000 7f53e95e0740 1 Processor -- start 2024-03-21T11:41:30.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.292+0000 7f53d7e00640 1 -- 172.21.15.47:0/3736608626 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f53c8003e50 con 0x7f53e010dd50 2024-03-21T11:41:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.293+0000 7f53e95e0740 1 -- start start 2024-03-21T11:41:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.294+0000 7f53e95e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca989810 0x5612ca989c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.294+0000 7f53e95e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca98a9e0 0x5612ca98ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.294+0000 7f53e95e0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca98b3a0 0x5612ca991c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.294+0000 7f53e95e0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca8ee2c0 con 0x5612ca98b3a0 2024-03-21T11:41:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.294+0000 7f53e95e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca896c90 con 0x5612ca98a9e0 2024-03-21T11:41:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.294+0000 7f53e95e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca8ea360 con 0x5612ca989810 2024-03-21T11:41:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.294+0000 7f53df400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca98a9e0 0x5612ca98ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.294+0000 7f53df400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca98a9e0 0x5612ca98ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38310/0 (socket says 172.21.15.47:38310) 2024-03-21T11:41:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.294+0000 7f53df400640 1 -- 172.21.15.47:0/2675124170 learned_addr learned my addr 172.21.15.47:0/2675124170 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:30.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.295+0000 7f53dea00640 1 --2- 172.21.15.47:0/2675124170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca989810 0x5612ca989c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.295+0000 7f53df400640 1 -- 172.21.15.47:0/2675124170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca989810 msgr2=0x5612ca989c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.295+0000 7f53df400640 1 --2- 172.21.15.47:0/2675124170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca989810 0x5612ca989c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.295+0000 7f53df400640 1 -- 172.21.15.47:0/2675124170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca98b3a0 msgr2=0x5612ca991c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.295+0000 7f53df400640 1 --2- 172.21.15.47:0/2675124170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca98b3a0 0x5612ca991c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.295+0000 7f53df400640 1 -- 172.21.15.47:0/2675124170 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612ca8bb080 con 0x5612ca98a9e0 2024-03-21T11:41:30.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.295+0000 7f53df400640 1 --2- 172.21.15.47:0/2675124170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca98a9e0 0x5612ca98ae60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f53d8000f80 tx=0x7f53d80094a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28bf4f10acc2f95d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:30.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.296+0000 7f53d4c00640 1 -- 172.21.15.47:0/2675124170 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8043030 con 0x5612ca98a9e0 2024-03-21T11:41:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.297+0000 7f53d7e00640 1 --2- 172.21.15.47:0/3736608626 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53c40a3070 0x7f53c40a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.297+0000 7f53d4c00640 1 -- 172.21.15.47:0/2675124170 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53d803a040 con 0x5612ca98a9e0 2024-03-21T11:41:30.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.297+0000 7f53d4c00640 1 -- 172.21.15.47:0/2675124170 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8039070 con 0x5612ca98a9e0 2024-03-21T11:41:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.298+0000 7f53d7e00640 1 -- 172.21.15.47:0/3736608626 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f53c80cedc0 con 0x7f53e010dd50 2024-03-21T11:41:30.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.299+0000 7f53e95e0740 1 -- 172.21.15.47:0/2675124170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca98a9e0 msgr2=0x5612ca98ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.299+0000 7f53e95e0740 1 --2- 172.21.15.47:0/2675124170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca98a9e0 0x5612ca98ae60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f53d8000f80 tx=0x7f53d80094a0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.299+0000 7f53dea00640 1 --2- 172.21.15.47:0/3736608626 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53c40a3070 0x7f53c40a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.300+0000 7f53e95e0740 1 -- 172.21.15.47:0/2675124170 shutdown_connections 2024-03-21T11:41:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.300+0000 7f53e95e0740 1 --2- 172.21.15.47:0/2675124170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca98b3a0 0x5612ca991c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.300+0000 7f53e95e0740 1 --2- 172.21.15.47:0/2675124170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca98a9e0 0x5612ca98ae60 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.300+0000 7f53e95e0740 1 --2- 172.21.15.47:0/2675124170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca989810 0x5612ca989c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.300+0000 7f53e95e0740 1 -- 172.21.15.47:0/2675124170 >> 172.21.15.47:0/2675124170 conn(0x5612ca984c00 msgr2=0x5612ca985000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.301+0000 7f53dea00640 1 --2- 172.21.15.47:0/3736608626 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53c40a3070 0x7f53c40a5530 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f53d0005bc0 tx=0x7f53d0005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.301+0000 7f53e95e0740 1 -- 172.21.15.47:0/2675124170 shutdown_connections 2024-03-21T11:41:30.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.301+0000 7f53e95e0740 1 -- 172.21.15.47:0/2675124170 wait complete. 2024-03-21T11:41:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.302+0000 7f53e95e0740 1 -- start start 2024-03-21T11:41:30.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.302+0000 7f53e95e0740 1 Processor -- start 2024-03-21T11:41:30.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.302+0000 7f53e95e0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca989810 0x5612ca993a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.302+0000 7f53e95e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 0x5612ca993f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.303+0000 7f53e95e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca98b3a0 0x5612ca99afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.304+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca98b3a0 0x5612ca99afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.304+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca98b3a0 0x5612ca99afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38324/0 (socket says 172.21.15.47:38324) 2024-03-21T11:41:30.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.304+0000 7f53dfe00640 1 -- 172.21.15.47:0/204727316 learned_addr learned my addr 172.21.15.47:0/204727316 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:30.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.303+0000 7f53e95e0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca992350 con 0x5612ca989810 2024-03-21T11:41:30.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.304+0000 7f53df400640 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca989810 0x5612ca993a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.304+0000 7f53e95e0740 1 -- 172.21.15.47:0/204727316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca8aa0d0 con 0x5612ca98b3a0 2024-03-21T11:41:30.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.304+0000 7f53e95e0740 1 -- 172.21.15.47:0/204727316 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca9924d0 con 0x5612ca98a9e0 2024-03-21T11:41:30.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.304+0000 7f53e95e0740 10 client.0 _ll_register_callbacks cb 0x5612ca8aee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.304+0000 7f53dea00640 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 0x5612ca993f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.305+0000 7f53dfe00640 1 -- 172.21.15.47:0/204727316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 msgr2=0x5612ca993f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.305+0000 7f53dfe00640 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 0x5612ca993f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.305+0000 7f53dfe00640 1 -- 172.21.15.47:0/204727316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca989810 msgr2=0x5612ca993a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.305+0000 7f53dfe00640 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca989810 0x5612ca993a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.305+0000 7f53dfe00640 1 -- 172.21.15.47:0/204727316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612ca6023f0 con 0x5612ca98b3a0 2024-03-21T11:41:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.305+0000 7f53df400640 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca989810 0x5612ca993a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:41:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.305+0000 7f53dea00640 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 0x5612ca993f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:41:30.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.305+0000 7f53dfe00640 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca98b3a0 0x5612ca99afd0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f53c80975c0 tx=0x7f53c809b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cd019dbd1647365 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:30.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.305+0000 7f53cd600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:41:30.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.305+0000 7f53cd600640 1 -- 172.21.15.47:0/204727316 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c80a3b70 con 0x5612ca98b3a0 2024-03-21T11:41:30.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.306+0000 7f53cd600640 1 -- 172.21.15.47:0/204727316 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f53c8004140 con 0x5612ca98b3a0 2024-03-21T11:41:30.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.306+0000 7f53e95e0740 1 -- 172.21.15.47:0/204727316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53d803a040 con 0x5612ca98b3a0 2024-03-21T11:41:30.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.306+0000 7f53cd600640 1 -- 172.21.15.47:0/204727316 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53c8004340 con 0x5612ca98b3a0 2024-03-21T11:41:30.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.307+0000 7f53e95e0740 10 client.31430 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:30.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.307+0000 7f53e95e0740 1 -- 172.21.15.47:0/204727316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5612ca6cc7d0 con 0x5612ca98b3a0 2024-03-21T11:41:30.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.307+0000 7f53cd600640 1 -- 172.21.15.47:0/204727316 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53d803a040 con 0x5612ca98b3a0 2024-03-21T11:41:30.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.307+0000 7f53cd600640 1 -- 172.21.15.47:0/204727316 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f53c809a5b0 con 0x5612ca98b3a0 2024-03-21T11:41:30.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.307+0000 7f53e95e0740 10 client.31430 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:30.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.307+0000 7f53e95e0740 10 client.31430 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:41:30.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.307+0000 7f53e95e0740 20 client.31430 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:30.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.307+0000 7f53e95e0740 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5612ca99f440 0x5612ca9a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.308+0000 7f53e95e0740 4 client.31430 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:41:30.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.308+0000 7f53e95e0740 1 -- 172.21.15.47:0/204727316 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612ca8fb800 con 0x5612ca99f440 2024-03-21T11:41:30.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.308+0000 7f53df400640 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5612ca99f440 0x5612ca9a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.310+0000 7f53df400640 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5612ca99f440 0x5612ca9a18e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7963ba591f64b8e server_cookie=fa02996c9946f318 in_seq=0 out_seq=0 2024-03-21T11:41:30.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.312+0000 7f53cd600640 10 client.31430 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:41:30.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.314+0000 7f53cd600640 1 -- 172.21.15.47:0/204727316 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5612ca8fb800 con 0x5612ca99f440 2024-03-21T11:41:30.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.314+0000 7f53cd600640 10 client.31430 handle_command_reply: tid=0 2024-03-21T11:41:30.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.314+0000 7f53e95e0740 1 client.31430 shutdown 2024-03-21T11:41:30.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.315+0000 7f53e95e0740 1 -- 172.21.15.47:0/204727316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca98b3a0 msgr2=0x5612ca99afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.315+0000 7f53e95e0740 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca98b3a0 0x5612ca99afd0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f53c80975c0 tx=0x7f53c809b920 comp rx=0 tx=0).stop 2024-03-21T11:41:30.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.315+0000 7f53e95e0740 1 -- 172.21.15.47:0/204727316 shutdown_connections 2024-03-21T11:41:30.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.315+0000 7f53e95e0740 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5612ca99f440 0x5612ca9a18e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.315+0000 7f53e95e0740 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca98b3a0 0x5612ca99afd0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.315+0000 7f53e95e0740 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 0x5612ca993f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.315+0000 7f53e95e0740 1 --2- 172.21.15.47:0/204727316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca989810 0x5612ca993a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.315+0000 7f53e95e0740 1 -- 172.21.15.47:0/204727316 >> 172.21.15.47:0/204727316 conn(0x5612ca984c00 msgr2=0x5612ca983eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:30.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.315+0000 7f53cd600640 0 client.31430 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:41:30.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.315+0000 7f53e95e0740 1 -- 172.21.15.47:0/204727316 shutdown_connections 2024-03-21T11:41:30.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.316+0000 7f53e95e0740 1 -- 172.21.15.47:0/204727316 wait complete. 2024-03-21T11:41:30.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.317+0000 7f53e95e0740 20 client.31430 trim_cache size 0 max 16384 2024-03-21T11:41:30.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.345+0000 7f53e95e0740 1 Processor -- start 2024-03-21T11:41:30.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.345+0000 7f53e95e0740 1 -- start start 2024-03-21T11:41:30.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.346+0000 7f53e95e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca989810 0x5612ca988570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.347+0000 7f53e95e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 0x5612ca99af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.347+0000 7f53e95e0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca98b3a0 0x5612ca98da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.347+0000 7f53e95e0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01c17c0 con 0x5612ca98b3a0 2024-03-21T11:41:30.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.347+0000 7f53e95e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01c3140 con 0x5612ca989810 2024-03-21T11:41:30.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.347+0000 7f53e95e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01c1e20 con 0x5612ca98a9e0 2024-03-21T11:41:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.347+0000 7f53df400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca989810 0x5612ca988570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.347+0000 7f53df400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca989810 0x5612ca988570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38328/0 (socket says 172.21.15.47:38328) 2024-03-21T11:41:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.347+0000 7f53df400640 1 -- 172.21.15.47:0/122607872 learned_addr learned my addr 172.21.15.47:0/122607872 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.347+0000 7f53dea00640 1 --2- 172.21.15.47:0/122607872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 0x5612ca99af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.348+0000 7f53dfe00640 1 --2- 172.21.15.47:0/122607872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca98b3a0 0x5612ca98da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.348+0000 7f53df400640 1 -- 172.21.15.47:0/122607872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 msgr2=0x5612ca99af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.348+0000 7f53df400640 1 --2- 172.21.15.47:0/122607872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 0x5612ca99af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.348+0000 7f53df400640 1 -- 172.21.15.47:0/122607872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca98b3a0 msgr2=0x5612ca98da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.348+0000 7f53df400640 1 --2- 172.21.15.47:0/122607872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca98b3a0 0x5612ca98da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.348+0000 7f53df400640 1 -- 172.21.15.47:0/122607872 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612ca98a460 con 0x5612ca989810 2024-03-21T11:41:30.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.348+0000 7f53df400640 1 --2- 172.21.15.47:0/122607872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca989810 0x5612ca988570 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009850 tx=0x7f53d8004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93e05b6b8da4d2cd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:30.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.349+0000 7f53cfe00640 1 -- 172.21.15.47:0/122607872 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d804c070 con 0x5612ca989810 2024-03-21T11:41:30.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.349+0000 7f53cfe00640 1 -- 172.21.15.47:0/122607872 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53d8003e50 con 0x5612ca989810 2024-03-21T11:41:30.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.349+0000 7f53cfe00640 1 -- 172.21.15.47:0/122607872 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8043040 con 0x5612ca989810 2024-03-21T11:41:30.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.350+0000 7f53e95e0740 1 -- 172.21.15.47:0/122607872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca989810 msgr2=0x5612ca988570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.350+0000 7f53e95e0740 1 --2- 172.21.15.47:0/122607872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca989810 0x5612ca988570 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009850 tx=0x7f53d8004a60 comp rx=0 tx=0).stop 2024-03-21T11:41:30.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.351+0000 7f53e95e0740 1 -- 172.21.15.47:0/122607872 shutdown_connections 2024-03-21T11:41:30.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.351+0000 7f53e95e0740 1 --2- 172.21.15.47:0/122607872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca98b3a0 0x5612ca98da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.351+0000 7f53e95e0740 1 --2- 172.21.15.47:0/122607872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 0x5612ca99af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.351+0000 7f53e95e0740 1 --2- 172.21.15.47:0/122607872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca989810 0x5612ca988570 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.351+0000 7f53e95e0740 1 -- 172.21.15.47:0/122607872 >> 172.21.15.47:0/122607872 conn(0x5612ca984c00 msgr2=0x5612ca987e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:30.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.351+0000 7f53e95e0740 1 -- 172.21.15.47:0/122607872 shutdown_connections 2024-03-21T11:41:30.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.351+0000 7f53e95e0740 1 -- 172.21.15.47:0/122607872 wait complete. 2024-03-21T11:41:30.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.352+0000 7f53e95e0740 1 -- start start 2024-03-21T11:41:30.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.352+0000 7f53e95e0740 1 Processor -- start 2024-03-21T11:41:30.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.353+0000 7f53e95e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca989810 0x5612ca997090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.353+0000 7f53e95e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 0x5612ca9975d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.353+0000 7f53e95e0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca98b3a0 0x5612ca99a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.353+0000 7f53e95e0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01c2ae0 con 0x5612ca98b3a0 2024-03-21T11:41:30.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.353+0000 7f53e95e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01c2480 con 0x5612ca989810 2024-03-21T11:41:30.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.353+0000 7f53e95e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01bf980 con 0x5612ca98a9e0 2024-03-21T11:41:30.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.353+0000 7f53e95e0740 10 client.0 _ll_register_callbacks cb 0x7f53c8003a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:30.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.353+0000 7f53df400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca989810 0x5612ca997090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.353+0000 7f53df400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca989810 0x5612ca997090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38332/0 (socket says 172.21.15.47:38332) 2024-03-21T11:41:30.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.353+0000 7f53df400640 1 -- 172.21.15.47:0/115580655 learned_addr learned my addr 172.21.15.47:0/115580655 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:30.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.354+0000 7f53dea00640 1 --2- 172.21.15.47:0/115580655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 0x5612ca9975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.354+0000 7f53dfe00640 1 --2- 172.21.15.47:0/115580655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca98b3a0 0x5612ca99a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.354+0000 7f53df400640 1 -- 172.21.15.47:0/115580655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 msgr2=0x5612ca9975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:30.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.354+0000 7f53df400640 1 --2- 172.21.15.47:0/115580655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 0x5612ca9975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.354+0000 7f53df400640 1 -- 172.21.15.47:0/115580655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca98b3a0 msgr2=0x5612ca99a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:30.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.354+0000 7f53df400640 1 --2- 172.21.15.47:0/115580655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca98b3a0 0x5612ca99a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.354+0000 7f53df400640 1 -- 172.21.15.47:0/115580655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612ca99abe0 con 0x5612ca989810 2024-03-21T11:41:30.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.355+0000 7f53df400640 1 --2- 172.21.15.47:0/115580655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca989810 0x5612ca997090 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f53d8004af0 tx=0x7f53d8034f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=947ff77b68a42011 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:30.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.355+0000 7f53cd600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:41:30.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.355+0000 7f53cd600640 1 -- 172.21.15.47:0/115580655 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8043050 con 0x5612ca989810 2024-03-21T11:41:30.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.355+0000 7f53e95e0740 1 -- 172.21.15.47:0/115580655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53d8003e50 con 0x5612ca989810 2024-03-21T11:41:30.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.355+0000 7f53cd600640 1 -- 172.21.15.47:0/115580655 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f53d8032bb0 con 0x5612ca989810 2024-03-21T11:41:30.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.356+0000 7f53cd600640 1 -- 172.21.15.47:0/115580655 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53d805e050 con 0x5612ca989810 2024-03-21T11:41:30.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.356+0000 7f53e95e0740 10 client.31436 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:30.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.356+0000 7f53e95e0740 1 -- 172.21.15.47:0/115580655 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5612ca9fb690 con 0x5612ca989810 2024-03-21T11:41:30.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.356+0000 7f53cd600640 1 -- 172.21.15.47:0/115580655 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53d8003e50 con 0x5612ca989810 2024-03-21T11:41:30.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.356+0000 7f53cd600640 1 -- 172.21.15.47:0/115580655 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f53d8034440 con 0x5612ca989810 2024-03-21T11:41:30.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.356+0000 7f53e95e0740 10 client.31436 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:30.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.356+0000 7f53e95e0740 10 client.31436 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:41:30.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.356+0000 7f53e95e0740 20 client.31436 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:30.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.356+0000 7f53e95e0740 1 --2- 172.21.15.47:0/115580655 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5612ca9fba20 0x5612ca9fde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.357+0000 7f53e95e0740 4 client.31436 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:41:30.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.357+0000 7f53e95e0740 1 -- 172.21.15.47:0/115580655 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5612ca8f5b50 con 0x5612ca9fba20 2024-03-21T11:41:30.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.358+0000 7f53dea00640 1 --2- 172.21.15.47:0/115580655 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5612ca9fba20 0x5612ca9fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.359+0000 7f53dea00640 1 --2- 172.21.15.47:0/115580655 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5612ca9fba20 0x5612ca9fde10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7aa864b9a03a11d4 server_cookie=90f3477e63edda2 in_seq=0 out_seq=0 2024-03-21T11:41:30.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.359+0000 7f53cd600640 10 client.31436 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:41:30.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.359+0000 7f53cd600640 1 -- 172.21.15.47:0/115580655 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5612ca8f5b50 con 0x5612ca9fba20 2024-03-21T11:41:30.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.359+0000 7f53cd600640 10 client.31436 handle_command_reply: tid=0 2024-03-21T11:41:30.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.359+0000 7f53e95e0740 1 client.31436 shutdown 2024-03-21T11:41:30.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.360+0000 7f53e95e0740 1 -- 172.21.15.47:0/115580655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca989810 msgr2=0x5612ca997090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.360+0000 7f53e95e0740 1 --2- 172.21.15.47:0/115580655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca989810 0x5612ca997090 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f53d8004af0 tx=0x7f53d8034f50 comp rx=0 tx=0).stop 2024-03-21T11:41:30.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.360+0000 7f53e95e0740 1 -- 172.21.15.47:0/115580655 shutdown_connections 2024-03-21T11:41:30.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.360+0000 7f53e95e0740 1 --2- 172.21.15.47:0/115580655 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5612ca9fba20 0x5612ca9fde10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.360+0000 7f53e95e0740 1 --2- 172.21.15.47:0/115580655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612ca98b3a0 0x5612ca99a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.360+0000 7f53e95e0740 1 --2- 172.21.15.47:0/115580655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5612ca98a9e0 0x5612ca9975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.360+0000 7f53e95e0740 1 --2- 172.21.15.47:0/115580655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612ca989810 0x5612ca997090 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.360+0000 7f53e95e0740 1 -- 172.21.15.47:0/115580655 >> 172.21.15.47:0/115580655 conn(0x5612ca984c00 msgr2=0x5612ca9886f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.360+0000 7f53cd600640 0 client.31436 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:41:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.360+0000 7f53e95e0740 1 -- 172.21.15.47:0/115580655 shutdown_connections 2024-03-21T11:41:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.360+0000 7f53e95e0740 1 -- 172.21.15.47:0/115580655 wait complete. 2024-03-21T11:41:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.361+0000 7f53e95e0740 20 client.31436 trim_cache size 0 max 16384 2024-03-21T11:41:30.362 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:41:30.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.361+0000 7f53e6800640 1 -- 172.21.15.47:0/3736608626 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53c40a3070 msgr2=0x7f53c40a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.362+0000 7f53e6800640 1 --2- 172.21.15.47:0/3736608626 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53c40a3070 0x7f53c40a5530 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f53d0005bc0 tx=0x7f53d0005ab0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.362+0000 7f53e6800640 1 -- 172.21.15.47:0/3736608626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e010dd50 msgr2=0x7f53e01b6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.362+0000 7f53e6800640 1 --2- 172.21.15.47:0/3736608626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e010dd50 0x7f53e01b6830 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f53c8005fd0 tx=0x7f53c800ded0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.362+0000 7f53e6800640 1 -- 172.21.15.47:0/3736608626 shutdown_connections 2024-03-21T11:41:30.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.362+0000 7f53e6800640 1 --2- 172.21.15.47:0/3736608626 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f53c40a3070 0x7f53c40a5530 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.362+0000 7f53e6800640 1 --2- 172.21.15.47:0/3736608626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53e010dd50 0x7f53e01b6830 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.362+0000 7f53e6800640 1 --2- 172.21.15.47:0/3736608626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e010b2d0 0x7f53e0117bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.362+0000 7f53e6800640 1 --2- 172.21.15.47:0/3736608626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53e0108910 0x7f53e0117680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.362+0000 7f53e6800640 1 -- 172.21.15.47:0/3736608626 >> 172.21.15.47:0/3736608626 conn(0x7f53e006e8a0 msgr2=0x7f53e0109330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:30.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.362+0000 7f53e6800640 1 -- 172.21.15.47:0/3736608626 shutdown_connections 2024-03-21T11:41:30.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.363+0000 7f53e6800640 1 -- 172.21.15.47:0/3736608626 wait complete. 2024-03-21T11:41:30.385 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:41:30.385 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:41:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.638+0000 7fcbf3e00640 1 -- 172.21.15.47:0/715241237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf4075660 msgr2=0x7fcbf4073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.638+0000 7fcbf3e00640 1 --2- 172.21.15.47:0/715241237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf4075660 0x7fcbf4073ab0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009ca0 tx=0x7fcbe402c190 comp rx=0 tx=0).stop 2024-03-21T11:41:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.639+0000 7fcbf3e00640 1 -- 172.21.15.47:0/715241237 shutdown_connections 2024-03-21T11:41:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.639+0000 7fcbf3e00640 1 --2- 172.21.15.47:0/715241237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf4109450 0x7fcbf410b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.639+0000 7fcbf3e00640 1 --2- 172.21.15.47:0/715241237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf4074080 0x7fcbf40744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.639+0000 7fcbf3e00640 1 --2- 172.21.15.47:0/715241237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf4075660 0x7fcbf4073ab0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.639+0000 7fcbf3e00640 1 -- 172.21.15.47:0/715241237 >> 172.21.15.47:0/715241237 conn(0x7fcbf40fcb10 msgr2=0x7fcbf40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.639+0000 7fcbf3e00640 1 -- 172.21.15.47:0/715241237 shutdown_connections 2024-03-21T11:41:30.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.639+0000 7fcbf3e00640 1 -- 172.21.15.47:0/715241237 wait complete. 2024-03-21T11:41:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.640+0000 7fcbf3e00640 1 Processor -- start 2024-03-21T11:41:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.640+0000 7fcbf3e00640 1 -- start start 2024-03-21T11:41:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.640+0000 7fcbf3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf4074080 0x7fcbf4199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.640+0000 7fcbf3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf4075660 0x7fcbf4199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.640+0000 7fcbf3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf4109450 0x7fcbf41a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.640+0000 7fcbf3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf410f400 con 0x7fcbf4075660 2024-03-21T11:41:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.640+0000 7fcbf3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf410f280 con 0x7fcbf4109450 2024-03-21T11:41:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.640+0000 7fcbf3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf410f580 con 0x7fcbf4074080 2024-03-21T11:41:30.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.641+0000 7fcbf2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf4075660 0x7fcbf4199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.641+0000 7fcbf3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf4109450 0x7fcbf41a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.641+0000 7fcbf2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf4075660 0x7fcbf4199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48776/0 (socket says 172.21.15.47:48776) 2024-03-21T11:41:30.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.641+0000 7fcbf2000640 1 -- 172.21.15.47:0/3810722414 learned_addr learned my addr 172.21.15.47:0/3810722414 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:30.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.641+0000 7fcbf3400640 1 -- 172.21.15.47:0/3810722414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf4074080 msgr2=0x7fcbf4199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.641+0000 7fcbf3400640 1 --2- 172.21.15.47:0/3810722414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf4074080 0x7fcbf4199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.641+0000 7fcbf3400640 1 -- 172.21.15.47:0/3810722414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf4075660 msgr2=0x7fcbf4199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.641+0000 7fcbf3400640 1 --2- 172.21.15.47:0/3810722414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf4075660 0x7fcbf4199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.641+0000 7fcbf3400640 1 -- 172.21.15.47:0/3810722414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf41a1670 con 0x7fcbf4109450 2024-03-21T11:41:30.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.642+0000 7fcbf3400640 1 --2- 172.21.15.47:0/3810722414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf4109450 0x7fcbf41a1000 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00e970 tx=0x7fcbec00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:30.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.642+0000 7fcbf8a00640 1 -- 172.21.15.47:0/3810722414 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec00ccc0 con 0x7fcbf4109450 2024-03-21T11:41:30.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.642+0000 7fcbf8a00640 1 -- 172.21.15.47:0/3810722414 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbec005020 con 0x7fcbf4109450 2024-03-21T11:41:30.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.642+0000 7fcbf8a00640 1 -- 172.21.15.47:0/3810722414 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec004a00 con 0x7fcbf4109450 2024-03-21T11:41:30.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.642+0000 7fcbf3e00640 1 -- 172.21.15.47:0/3810722414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf41a1960 con 0x7fcbf4109450 2024-03-21T11:41:30.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.642+0000 7fcbf3e00640 1 -- 172.21.15.47:0/3810722414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf41a1e60 con 0x7fcbf4109450 2024-03-21T11:41:30.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.646+0000 7fcbfcbe5740 1 Processor -- start 2024-03-21T11:41:30.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.647+0000 7fcbf8a00640 1 -- 172.21.15.47:0/3810722414 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcbec0051d0 con 0x7fcbf4109450 2024-03-21T11:41:30.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.647+0000 7fcbfcbe5740 1 -- start start 2024-03-21T11:41:30.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.648+0000 7fcbfcbe5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc3200 0x557334fc3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.648+0000 7fcbfcbe5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fc4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.648+0000 7fcbfcbe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 0x557334fcb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.648+0000 7fcbfcbe5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557334f1f2c0 con 0x557334fc43d0 2024-03-21T11:41:30.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.648+0000 7fcbfcbe5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557334ec7c90 con 0x557334fc3200 2024-03-21T11:41:30.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.648+0000 7fcbfcbe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557334f1b360 con 0x557334fc4d90 2024-03-21T11:41:30.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.648+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fc4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.648+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fc4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48792/0 (socket says 172.21.15.47:48792) 2024-03-21T11:41:30.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.648+0000 7fcbf2a00640 1 -- 172.21.15.47:0/622659252 learned_addr learned my addr 172.21.15.47:0/622659252 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:30.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.649+0000 7fcbf2000640 1 --2- 172.21.15.47:0/622659252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc3200 0x557334fc3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.649+0000 7fcbf2000640 1 --2- 172.21.15.47:0/622659252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 0x557334fcb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.649+0000 7fcbf2a00640 1 -- 172.21.15.47:0/622659252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 msgr2=0x557334fcb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:30.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.649+0000 7fcbf2a00640 1 --2- 172.21.15.47:0/622659252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 0x557334fcb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.649+0000 7fcbf2a00640 1 -- 172.21.15.47:0/622659252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc3200 msgr2=0x557334fc3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:30.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.649+0000 7fcbf2a00640 1 --2- 172.21.15.47:0/622659252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc3200 0x557334fc3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.649+0000 7fcbf2a00640 1 -- 172.21.15.47:0/622659252 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557334eec080 con 0x557334fc43d0 2024-03-21T11:41:30.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.649+0000 7fcbf2000640 1 --2- 172.21.15.47:0/622659252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 0x557334fcb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:41:30.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.649+0000 7fcbf8a00640 1 --2- 172.21.15.47:0/3810722414 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd80a3020 0x7fcbd80a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.649+0000 7fcbf8a00640 1 -- 172.21.15.47:0/3810722414 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcbec0cf7e0 con 0x7fcbf4109450 2024-03-21T11:41:30.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.649+0000 7fcbf2a00640 1 --2- 172.21.15.47:0/622659252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fc4850 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4005e30 tx=0x7fcbe4032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd45e3a8f9b61930 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:30.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.649+0000 7fcbf2000640 1 --2- 172.21.15.47:0/3810722414 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd80a3020 0x7fcbd80a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.650+0000 7fcbdfe00640 1 -- 172.21.15.47:0/622659252 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4041020 con 0x557334fc43d0 2024-03-21T11:41:30.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.650+0000 7fcbdfe00640 1 -- 172.21.15.47:0/622659252 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbe4003e50 con 0x557334fc43d0 2024-03-21T11:41:30.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.650+0000 7fcbdfe00640 1 -- 172.21.15.47:0/622659252 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4039040 con 0x557334fc43d0 2024-03-21T11:41:30.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.651+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/622659252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 msgr2=0x557334fc4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.651+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/622659252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fc4850 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4005e30 tx=0x7fcbe4032340 comp rx=0 tx=0).stop 2024-03-21T11:41:30.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.651+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/622659252 shutdown_connections 2024-03-21T11:41:30.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.651+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/622659252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 0x557334fcb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.651+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/622659252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fc4850 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.651+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/622659252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc3200 0x557334fc3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.651+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/622659252 >> 172.21.15.47:0/622659252 conn(0x557334fb5de0 msgr2=0x557334fb61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:30.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.651+0000 7fcbf2000640 1 --2- 172.21.15.47:0/3810722414 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd80a3020 0x7fcbd80a54e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009620 tx=0x7fcbe0009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:30.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.651+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/622659252 shutdown_connections 2024-03-21T11:41:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.651+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/622659252 wait complete. 2024-03-21T11:41:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.651+0000 7fcbfcbe5740 1 -- start start 2024-03-21T11:41:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.652+0000 7fcbfcbe5740 1 Processor -- start 2024-03-21T11:41:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.652+0000 7fcbfcbe5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc3200 0x557334fbbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.652+0000 7fcbfcbe5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fbc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.652+0000 7fcbfcbe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 0x557334fc13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.652+0000 7fcbfcbe5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557334fba730 con 0x557334fc43d0 2024-03-21T11:41:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.652+0000 7fcbfcbe5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557334edb0d0 con 0x557334fc3200 2024-03-21T11:41:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.652+0000 7fcbfcbe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557334fba8b0 con 0x557334fc4d90 2024-03-21T11:41:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.652+0000 7fcbfcbe5740 10 client.0 _ll_register_callbacks cb 0x557334edfe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.653+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc3200 0x557334fbbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.653+0000 7fcbf2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fbc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.653+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc3200 0x557334fbbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38362/0 (socket says 172.21.15.47:38362) 2024-03-21T11:41:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.653+0000 7fcbf2a00640 1 -- 172.21.15.47:0/4181476601 learned_addr learned my addr 172.21.15.47:0/4181476601 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.653+0000 7fcbf3400640 1 --2- 172.21.15.47:0/4181476601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 0x557334fc13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.653+0000 7fcbf3400640 1 -- 172.21.15.47:0/4181476601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc3200 msgr2=0x557334fbbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.653+0000 7fcbf3400640 1 --2- 172.21.15.47:0/4181476601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc3200 0x557334fbbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.653+0000 7fcbf3400640 1 -- 172.21.15.47:0/4181476601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 msgr2=0x557334fbc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.653+0000 7fcbf3400640 1 --2- 172.21.15.47:0/4181476601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fbc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.653+0000 7fcbf3400640 1 -- 172.21.15.47:0/4181476601 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557334c333f0 con 0x557334fc4d90 2024-03-21T11:41:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.653+0000 7fcbf2000640 1 --2- 172.21.15.47:0/4181476601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fbc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:41:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.654+0000 7fcbf3400640 1 --2- 172.21.15.47:0/4181476601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 0x557334fc13d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fcbec09b820 tx=0x7fcbec0055e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=822278fa41e1374b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.654+0000 7fcbdcc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:41:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.654+0000 7fcbdcc00640 1 -- 172.21.15.47:0/4181476601 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0b1020 con 0x557334fc4d90 2024-03-21T11:41:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.654+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/4181476601 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbe4003e50 con 0x557334fc4d90 2024-03-21T11:41:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.654+0000 7fcbdcc00640 1 -- 172.21.15.47:0/4181476601 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcbec0a42a0 con 0x557334fc4d90 2024-03-21T11:41:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.654+0000 7fcbdcc00640 1 -- 172.21.15.47:0/4181476601 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbec01c040 con 0x557334fc4d90 2024-03-21T11:41:30.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.655+0000 7fcbfcbe5740 10 client.31423 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:30.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.655+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/4181476601 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557334cfd7d0 con 0x557334fc4d90 2024-03-21T11:41:30.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.655+0000 7fcbdcc00640 1 -- 172.21.15.47:0/4181476601 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbe4003e50 con 0x557334fc4d90 2024-03-21T11:41:30.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.655+0000 7fcbdcc00640 1 -- 172.21.15.47:0/4181476601 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcbec09b090 con 0x557334fc4d90 2024-03-21T11:41:30.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.655+0000 7fcbfcbe5740 10 client.31423 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:30.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.655+0000 7fcbfcbe5740 10 client.31423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:30.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.656+0000 7fcbfcbe5740 20 client.31423 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:30.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.656+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/4181476601 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557334fe1670 0x557334fe3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.656+0000 7fcbfcbe5740 4 client.31423 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:41:30.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.656+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/4181476601 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557334f2c800 con 0x557334fe1670 2024-03-21T11:41:30.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.657+0000 7fcbf2a00640 1 --2- 172.21.15.47:0/4181476601 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557334fe1670 0x557334fe3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.658+0000 7fcbdcc00640 10 client.31423 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:30.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.658+0000 7fcbf2a00640 1 --2- 172.21.15.47:0/4181476601 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557334fe1670 0x557334fe3a80 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4031ede79b2d83dd server_cookie=863072bb94b971e3 in_seq=0 out_seq=0 2024-03-21T11:41:30.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.665+0000 7fcbdcc00640 1 -- 172.21.15.47:0/4181476601 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557334f2c800 con 0x557334fe1670 2024-03-21T11:41:30.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.665+0000 7fcbdcc00640 10 client.31423 handle_command_reply: tid=0 2024-03-21T11:41:30.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.665+0000 7fcbfcbe5740 1 client.31423 shutdown 2024-03-21T11:41:30.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.665+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/4181476601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 msgr2=0x557334fc13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.665+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/4181476601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 0x557334fc13d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fcbec09b820 tx=0x7fcbec0055e0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.665+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/4181476601 shutdown_connections 2024-03-21T11:41:30.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.665+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/4181476601 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557334fe1670 0x557334fe3a80 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.665+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/4181476601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 0x557334fc13d0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.665+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/4181476601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fbc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.665+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/4181476601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc3200 0x557334fbbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.665+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/4181476601 >> 172.21.15.47:0/4181476601 conn(0x557334fb5de0 msgr2=0x557334fb5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:30.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.665+0000 7fcbdcc00640 0 client.31423 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:30.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.666+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/4181476601 shutdown_connections 2024-03-21T11:41:30.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.666+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/4181476601 wait complete. 2024-03-21T11:41:30.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.667+0000 7fcbfcbe5740 20 client.31423 trim_cache size 0 max 16384 2024-03-21T11:41:30.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.693+0000 7fcbfcbe5740 1 Processor -- start 2024-03-21T11:41:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.693+0000 7fcbfcbe5740 1 -- start start 2024-03-21T11:41:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.694+0000 7fcbfcbe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc3200 0x557334fb48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.694+0000 7fcbfcbe5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fb4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.694+0000 7fcbfcbe5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc4d90 0x557334fb5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.694+0000 7fcbfcbe5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf41a7c30 con 0x557334fc43d0 2024-03-21T11:41:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.694+0000 7fcbfcbe5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf41a95b0 con 0x557334fc4d90 2024-03-21T11:41:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.694+0000 7fcbfcbe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf41a8290 con 0x557334fc3200 2024-03-21T11:41:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.695+0000 7fcbf2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fb4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.695+0000 7fcbf3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc4d90 0x557334fb5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.695+0000 7fcbf3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc4d90 0x557334fb5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38364/0 (socket says 172.21.15.47:38364) 2024-03-21T11:41:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.695+0000 7fcbf3400640 1 -- 172.21.15.47:0/2325255421 learned_addr learned my addr 172.21.15.47:0/2325255421 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.695+0000 7fcbf2a00640 1 --2- 172.21.15.47:0/2325255421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc3200 0x557334fb48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.695+0000 7fcbf3400640 1 -- 172.21.15.47:0/2325255421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc3200 msgr2=0x557334fb48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.696+0000 7fcbf3400640 1 --2- 172.21.15.47:0/2325255421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc3200 0x557334fb48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.696+0000 7fcbf3400640 1 -- 172.21.15.47:0/2325255421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 msgr2=0x557334fb4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.696+0000 7fcbf3400640 1 --2- 172.21.15.47:0/2325255421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fb4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.696+0000 7fcbf3400640 1 -- 172.21.15.47:0/2325255421 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557334fc3e50 con 0x557334fc4d90 2024-03-21T11:41:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.696+0000 7fcbf2a00640 1 --2- 172.21.15.47:0/2325255421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc3200 0x557334fb48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:41:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.696+0000 7fcbf3400640 1 --2- 172.21.15.47:0/2325255421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc4d90 0x557334fb5350 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00e940 tx=0x7fcbec09bca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10907a24b8444eb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.696+0000 7fcbdf400640 1 -- 172.21.15.47:0/2325255421 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0b1070 con 0x557334fc4d90 2024-03-21T11:41:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.696+0000 7fcbdf400640 1 -- 172.21.15.47:0/2325255421 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbec0b4e50 con 0x557334fc4d90 2024-03-21T11:41:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.696+0000 7fcbdf400640 1 -- 172.21.15.47:0/2325255421 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec09af70 con 0x557334fc4d90 2024-03-21T11:41:30.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.698+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/2325255421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc4d90 msgr2=0x557334fb5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.698+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/2325255421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc4d90 0x557334fb5350 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00e940 tx=0x7fcbec09bca0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.698+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/2325255421 shutdown_connections 2024-03-21T11:41:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.698+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/2325255421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc4d90 0x557334fb5350 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.698+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/2325255421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc43d0 0x557334fb4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.698+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/2325255421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc3200 0x557334fb48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.698+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/2325255421 >> 172.21.15.47:0/2325255421 conn(0x557334fb5de0 msgr2=0x557334fbbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:30.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.699+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/2325255421 shutdown_connections 2024-03-21T11:41:30.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.699+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/2325255421 wait complete. 2024-03-21T11:41:30.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.700+0000 7fcbfcbe5740 1 -- start start 2024-03-21T11:41:30.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.700+0000 7fcbfcbe5740 1 Processor -- start 2024-03-21T11:41:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.700+0000 7fcbfcbe5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc3200 0x557334ff3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.701+0000 7fcbfcbe5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc43d0 0x557334ff3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.701+0000 7fcbfcbe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 0x557334ff0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.701+0000 7fcbfcbe5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf41a8f50 con 0x557334fc3200 2024-03-21T11:41:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.701+0000 7fcbfcbe5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf41a88f0 con 0x557334fc43d0 2024-03-21T11:41:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.701+0000 7fcbfcbe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf41a5df0 con 0x557334fc4d90 2024-03-21T11:41:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.701+0000 7fcbfcbe5740 10 client.0 _ll_register_callbacks cb 0x7fcbec005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:30.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.701+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc3200 0x557334ff3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.701+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc3200 0x557334ff3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48830/0 (socket says 172.21.15.47:48830) 2024-03-21T11:41:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.701+0000 7fcbf2a00640 1 -- 172.21.15.47:0/3324371998 learned_addr learned my addr 172.21.15.47:0/3324371998 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.702+0000 7fcbf2a00640 1 -- 172.21.15.47:0/3324371998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 msgr2=0x557334ff0020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:41:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.702+0000 7fcbf2a00640 1 --2- 172.21.15.47:0/3324371998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 0x557334ff0020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.702+0000 7fcbf2a00640 1 -- 172.21.15.47:0/3324371998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc43d0 msgr2=0x557334ff3a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:41:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.702+0000 7fcbf2a00640 1 --2- 172.21.15.47:0/3324371998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc43d0 0x557334ff3a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.702+0000 7fcbf2a00640 1 -- 172.21.15.47:0/3324371998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557334ff0790 con 0x557334fc3200 2024-03-21T11:41:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.702+0000 7fcbf2a00640 1 --2- 172.21.15.47:0/3324371998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc3200 0x557334ff3530 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcbe402e010 tx=0x7fcbe40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70340d4891d9abcb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:30.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.703+0000 7fcbdcc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:41:30.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.703+0000 7fcbdcc00640 1 -- 172.21.15.47:0/3324371998 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4039040 con 0x557334fc3200 2024-03-21T11:41:30.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.703+0000 7fcbdcc00640 1 -- 172.21.15.47:0/3324371998 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcbe4032970 con 0x557334fc3200 2024-03-21T11:41:30.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.704+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/3324371998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbec0b4e50 con 0x557334fc3200 2024-03-21T11:41:30.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.704+0000 7fcbdcc00640 1 -- 172.21.15.47:0/3324371998 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbe4041040 con 0x557334fc3200 2024-03-21T11:41:30.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.704+0000 7fcbfcbe5740 10 client.20898 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:30.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.705+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/3324371998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55733503d050 con 0x557334fc3200 2024-03-21T11:41:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.705+0000 7fcbdcc00640 1 -- 172.21.15.47:0/3324371998 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbec0b4e50 con 0x557334fc3200 2024-03-21T11:41:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.705+0000 7fcbdcc00640 1 -- 172.21.15.47:0/3324371998 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcbe403c030 con 0x557334fc3200 2024-03-21T11:41:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.705+0000 7fcbfcbe5740 10 client.20898 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.705+0000 7fcbfcbe5740 10 client.20898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.706+0000 7fcbfcbe5740 20 client.20898 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:30.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.706+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/3324371998 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55733503d550 0x55733503f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.706+0000 7fcbfcbe5740 4 client.20898 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:41:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.706+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/3324371998 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557334f26b50 con 0x55733503d550 2024-03-21T11:41:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.707+0000 7fcbf2000640 1 --2- 172.21.15.47:0/3324371998 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55733503d550 0x55733503f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.708+0000 7fcbdcc00640 10 client.20898 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:30.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.709+0000 7fcbf2000640 1 --2- 172.21.15.47:0/3324371998 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55733503d550 0x55733503f940 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=698dc6c28fb2b03d server_cookie=c0ada9db1021806f in_seq=0 out_seq=0 2024-03-21T11:41:30.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.711+0000 7fcbdcc00640 1 -- 172.21.15.47:0/3324371998 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557334f26b50 con 0x55733503d550 2024-03-21T11:41:30.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.711+0000 7fcbdcc00640 10 client.20898 handle_command_reply: tid=0 2024-03-21T11:41:30.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.712+0000 7fcbfcbe5740 1 client.20898 shutdown 2024-03-21T11:41:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.712+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/3324371998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc3200 msgr2=0x557334ff3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.712+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/3324371998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc3200 0x557334ff3530 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcbe402e010 tx=0x7fcbe40095c0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.712+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/3324371998 shutdown_connections 2024-03-21T11:41:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.712+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/3324371998 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55733503d550 0x55733503f940 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.712+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/3324371998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557334fc4d90 0x557334ff0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.712+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/3324371998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557334fc43d0 0x557334ff3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.712+0000 7fcbfcbe5740 1 --2- 172.21.15.47:0/3324371998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557334fc3200 0x557334ff3530 secure :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcbe402e010 tx=0x7fcbe40095c0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.712+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/3324371998 >> 172.21.15.47:0/3324371998 conn(0x557334fb5de0 msgr2=0x557334fb48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.713+0000 7fcbdcc00640 0 client.20898 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.713+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/3324371998 shutdown_connections 2024-03-21T11:41:30.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.714+0000 7fcbfcbe5740 1 -- 172.21.15.47:0/3324371998 wait complete. 2024-03-21T11:41:30.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.714+0000 7fcbfcbe5740 20 client.20898 trim_cache size 0 max 16384 2024-03-21T11:41:30.715 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:41:30.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.715+0000 7fcbf3e00640 1 -- 172.21.15.47:0/3810722414 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd80a3020 msgr2=0x7fcbd80a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.715+0000 7fcbf3e00640 1 --2- 172.21.15.47:0/3810722414 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd80a3020 0x7fcbd80a54e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009620 tx=0x7fcbe0009530 comp rx=0 tx=0).stop 2024-03-21T11:41:30.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.715+0000 7fcbf3e00640 1 -- 172.21.15.47:0/3810722414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf4109450 msgr2=0x7fcbf41a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:30.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.715+0000 7fcbf3e00640 1 --2- 172.21.15.47:0/3810722414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf4109450 0x7fcbf41a1000 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00e970 tx=0x7fcbec00ee30 comp rx=0 tx=0).stop 2024-03-21T11:41:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.715+0000 7fcbf3e00640 1 -- 172.21.15.47:0/3810722414 shutdown_connections 2024-03-21T11:41:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.715+0000 7fcbf3e00640 1 --2- 172.21.15.47:0/3810722414 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcbd80a3020 0x7fcbd80a54e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.715+0000 7fcbf3e00640 1 --2- 172.21.15.47:0/3810722414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf4109450 0x7fcbf41a1000 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.715+0000 7fcbf3e00640 1 --2- 172.21.15.47:0/3810722414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf4075660 0x7fcbf4199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.715+0000 7fcbf3e00640 1 --2- 172.21.15.47:0/3810722414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbf4074080 0x7fcbf4199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.716+0000 7fcbf3e00640 1 -- 172.21.15.47:0/3810722414 >> 172.21.15.47:0/3810722414 conn(0x7fcbf40fcb10 msgr2=0x7fcbf41053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.716+0000 7fcbf3e00640 1 -- 172.21.15.47:0/3810722414 shutdown_connections 2024-03-21T11:41:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:30.716+0000 7fcbf3e00640 1 -- 172.21.15.47:0/3810722414 wait complete. 2024-03-21T11:41:30.949 INFO:tasks.workunit.client.0.smithi047.stdout: 10 118000 23.99 MB/sec execute 134 sec latency 207.591 ms 2024-03-21T11:41:31.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:30 smithi047 ceph-mon[31351]: pgmap v945: 129 pgs: 129 active+clean; 473 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.1 MiB/s wr, 698 op/s 2024-03-21T11:41:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:30 smithi102 ceph-mon[36748]: pgmap v945: 129 pgs: 129 active+clean; 473 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.1 MiB/s wr, 698 op/s 2024-03-21T11:41:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:30 smithi100 ceph-mon[36245]: pgmap v945: 129 pgs: 129 active+clean; 473 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.1 MiB/s wr, 698 op/s 2024-03-21T11:41:31.743 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:41:31.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:41:31.744 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:41:31.950 INFO:tasks.workunit.client.0.smithi047.stdout: 10 118505 23.98 MB/sec execute 135 sec latency 133.227 ms 2024-03-21T11:41:31.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.985+0000 7f75ac200640 1 -- 172.21.15.47:0/779790844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a41038e0 msgr2=0x7f75a4103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:31.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.985+0000 7f75ac200640 1 --2- 172.21.15.47:0/779790844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a41038e0 0x7f75a4103d00 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7598009900 tx=0x7f759802bc30 comp rx=0 tx=0).stop 2024-03-21T11:41:31.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.985+0000 7f75ac200640 1 -- 172.21.15.47:0/779790844 shutdown_connections 2024-03-21T11:41:31.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.985+0000 7f75ac200640 1 --2- 172.21.15.47:0/779790844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75a4105490 0x7f75a410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:31.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.985+0000 7f75ac200640 1 --2- 172.21.15.47:0/779790844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a4104ad0 0x7f75a4104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:31.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.985+0000 7f75ac200640 1 --2- 172.21.15.47:0/779790844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a41038e0 0x7f75a4103d00 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:31.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.985+0000 7f75ac200640 1 -- 172.21.15.47:0/779790844 >> 172.21.15.47:0/779790844 conn(0x7f75a40fee50 msgr2=0x7f75a4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:31.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.985+0000 7f75ac200640 1 -- 172.21.15.47:0/779790844 shutdown_connections 2024-03-21T11:41:31.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.986+0000 7f75ac200640 1 -- 172.21.15.47:0/779790844 wait complete. 2024-03-21T11:41:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.987+0000 7f75ac200640 1 Processor -- start 2024-03-21T11:41:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.987+0000 7f75ac200640 1 -- start start 2024-03-21T11:41:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.987+0000 7f75ac200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75a41038e0 0x7f75a4199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.987+0000 7f75ac200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a4104ad0 0x7f75a419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.987+0000 7f75ac200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a4105490 0x7f75a41a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.987+0000 7f75ac200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a410f600 con 0x7f75a41038e0 2024-03-21T11:41:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.987+0000 7f75ac200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a410f480 con 0x7f75a4104ad0 2024-03-21T11:41:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.987+0000 7f75ac200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a410f780 con 0x7f75a4105490 2024-03-21T11:41:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.988+0000 7f75a8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a4104ad0 0x7f75a419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.988+0000 7f75aa200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a4105490 0x7f75a41a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.988+0000 7f75a8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a4104ad0 0x7f75a419a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38390/0 (socket says 172.21.15.47:38390) 2024-03-21T11:41:31.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.988+0000 7f75a8e00640 1 -- 172.21.15.47:0/316389385 learned_addr learned my addr 172.21.15.47:0/316389385 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:31.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.988+0000 7f75a9800640 1 --2- 172.21.15.47:0/316389385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75a41038e0 0x7f75a4199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:31.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.988+0000 7f75aa200640 1 -- 172.21.15.47:0/316389385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a4104ad0 msgr2=0x7f75a419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:31.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.988+0000 7f75aa200640 1 --2- 172.21.15.47:0/316389385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a4104ad0 0x7f75a419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:31.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.988+0000 7f75aa200640 1 -- 172.21.15.47:0/316389385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75a41038e0 msgr2=0x7f75a4199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:31.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.988+0000 7f75aa200640 1 --2- 172.21.15.47:0/316389385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75a41038e0 0x7f75a4199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:31.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.988+0000 7f75aa200640 1 -- 172.21.15.47:0/316389385 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a41a18b0 con 0x7f75a4105490 2024-03-21T11:41:31.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.989+0000 7f75aa200640 1 --2- 172.21.15.47:0/316389385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a4105490 0x7f75a41a1210 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f759c00cab0 tx=0x7f759c00cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:31.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.989+0000 7f75a2000640 1 -- 172.21.15.47:0/316389385 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c00ae20 con 0x7f75a4105490 2024-03-21T11:41:31.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.989+0000 7f75ac200640 1 -- 172.21.15.47:0/316389385 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a41a1ba0 con 0x7f75a4105490 2024-03-21T11:41:31.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.989+0000 7f75a2000640 1 -- 172.21.15.47:0/316389385 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f759c00f930 con 0x7f75a4105490 2024-03-21T11:41:31.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.989+0000 7f75a2000640 1 -- 172.21.15.47:0/316389385 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c018b30 con 0x7f75a4105490 2024-03-21T11:41:31.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.989+0000 7f75ac200640 1 -- 172.21.15.47:0/316389385 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a41a2070 con 0x7f75a4105490 2024-03-21T11:41:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.993+0000 7f75aee10740 1 Processor -- start 2024-03-21T11:41:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.994+0000 7f75a2000640 1 -- 172.21.15.47:0/316389385 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f759c00fae0 con 0x7f75a4105490 2024-03-21T11:41:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.994+0000 7f75aee10740 1 -- start start 2024-03-21T11:41:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.994+0000 7f75aee10740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4fe90a0 0x555ea4fe94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.994+0000 7f75aee10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe9a90 0x555ea4fee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.994+0000 7f75aee10740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4feec00 0x555ea4ff1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.994+0000 7f75aee10740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea4f4eeb0 con 0x555ea4fe90a0 2024-03-21T11:41:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.994+0000 7f75aee10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea4ef7a60 con 0x555ea4fe9a90 2024-03-21T11:41:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.994+0000 7f75aee10740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea4f4af50 con 0x555ea4feec00 2024-03-21T11:41:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.995+0000 7f75a8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4fe90a0 0x555ea4fe94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:31.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.995+0000 7f75a8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4fe90a0 0x555ea4fe94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48870/0 (socket says 172.21.15.47:48870) 2024-03-21T11:41:31.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.995+0000 7f75a8e00640 1 -- 172.21.15.47:0/2822645248 learned_addr learned my addr 172.21.15.47:0/2822645248 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:31.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.995+0000 7f75a8e00640 1 --2- 172.21.15.47:0/2822645248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4feec00 0x555ea4ff1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:31.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.995+0000 7f75a9800640 1 --2- 172.21.15.47:0/2822645248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe9a90 0x555ea4fee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:31.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.995+0000 7f75a8e00640 1 -- 172.21.15.47:0/2822645248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe9a90 msgr2=0x555ea4fee590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:31.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.995+0000 7f75a8e00640 1 --2- 172.21.15.47:0/2822645248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe9a90 0x555ea4fee590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:31.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.995+0000 7f75a8e00640 1 -- 172.21.15.47:0/2822645248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4fe90a0 msgr2=0x555ea4fe94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:31.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.995+0000 7f75a8e00640 1 --2- 172.21.15.47:0/2822645248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4fe90a0 0x555ea4fe94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:31.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.995+0000 7f75a8e00640 1 -- 172.21.15.47:0/2822645248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ea4d2b7e0 con 0x555ea4feec00 2024-03-21T11:41:31.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.996+0000 7f75a2000640 1 --2- 172.21.15.47:0/316389385 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f758c0a2f20 0x7f758c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:31.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.996+0000 7f75a8e00640 1 --2- 172.21.15.47:0/2822645248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4feec00 0x555ea4ff1090 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f7590009fd0 tx=0x7f75900046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b21e636a89ab002e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:31.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.996+0000 7f75a2000640 1 -- 172.21.15.47:0/316389385 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f759c0cfe70 con 0x7f75a4105490 2024-03-21T11:41:31.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.996+0000 7f75a9800640 1 --2- 172.21.15.47:0/316389385 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f758c0a2f20 0x7f758c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:31.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.996+0000 7f7596a00640 1 -- 172.21.15.47:0/2822645248 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759000f040 con 0x555ea4feec00 2024-03-21T11:41:31.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.996+0000 7f7596a00640 1 -- 172.21.15.47:0/2822645248 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f759001d070 con 0x555ea4feec00 2024-03-21T11:41:31.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.996+0000 7f7596a00640 1 -- 172.21.15.47:0/2822645248 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590018890 con 0x555ea4feec00 2024-03-21T11:41:31.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.997+0000 7f75a9800640 1 --2- 172.21.15.47:0/316389385 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f758c0a2f20 0x7f758c0a53e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f759802c170 tx=0x7f759802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:31.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.997+0000 7f75aee10740 1 -- 172.21.15.47:0/2822645248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4feec00 msgr2=0x555ea4ff1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:31.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.997+0000 7f75aee10740 1 --2- 172.21.15.47:0/2822645248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4feec00 0x555ea4ff1090 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f7590009fd0 tx=0x7f75900046c0 comp rx=0 tx=0).stop 2024-03-21T11:41:31.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.997+0000 7f75aee10740 1 -- 172.21.15.47:0/2822645248 shutdown_connections 2024-03-21T11:41:31.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.997+0000 7f75aee10740 1 --2- 172.21.15.47:0/2822645248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4feec00 0x555ea4ff1090 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.997+0000 7f75aee10740 1 --2- 172.21.15.47:0/2822645248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe9a90 0x555ea4fee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.997+0000 7f75aee10740 1 --2- 172.21.15.47:0/2822645248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4fe90a0 0x555ea4fe94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.997+0000 7f75aee10740 1 -- 172.21.15.47:0/2822645248 >> 172.21.15.47:0/2822645248 conn(0x555ea4fe4610 msgr2=0x555ea4fe4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:32.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.998+0000 7f75aee10740 1 -- 172.21.15.47:0/2822645248 shutdown_connections 2024-03-21T11:41:32.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.998+0000 7f75aee10740 1 -- 172.21.15.47:0/2822645248 wait complete. 2024-03-21T11:41:32.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.998+0000 7f75aee10740 1 -- start start 2024-03-21T11:41:32.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.998+0000 7f75aee10740 1 Processor -- start 2024-03-21T11:41:32.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.999+0000 7f75aee10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe90a0 0x555ea4ff3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.999+0000 7f75aee10740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4fe9a90 0x555ea4ff3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.999+0000 7f75aee10740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4feec00 0x555ea4ffa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.999+0000 7f75aee10740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea4ff1b50 con 0x555ea4feec00 2024-03-21T11:41:32.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.999+0000 7f75aee10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea4f0aeb0 con 0x555ea4fe90a0 2024-03-21T11:41:32.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.999+0000 7f75aee10740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea4ff1cd0 con 0x555ea4fe9a90 2024-03-21T11:41:32.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.999+0000 7f75aee10740 10 client.0 _ll_register_callbacks cb 0x555ea4eff150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:32.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.999+0000 7f75a8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4fe9a90 0x555ea4ff3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.999+0000 7f75a9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe90a0 0x555ea4ff3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.999+0000 7f75a8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4fe9a90 0x555ea4ff3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34976/0 (socket says 172.21.15.47:34976) 2024-03-21T11:41:32.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.999+0000 7f75a8e00640 1 -- 172.21.15.47:0/1649443992 learned_addr learned my addr 172.21.15.47:0/1649443992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:32.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:31.999+0000 7f75aa200640 1 --2- 172.21.15.47:0/1649443992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4feec00 0x555ea4ffa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.000+0000 7f75a8e00640 1 -- 172.21.15.47:0/1649443992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe90a0 msgr2=0x555ea4ff3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:32.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.000+0000 7f75a8e00640 1 --2- 172.21.15.47:0/1649443992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe90a0 0x555ea4ff3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.000+0000 7f75a8e00640 1 -- 172.21.15.47:0/1649443992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4feec00 msgr2=0x555ea4ffa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:32.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.000+0000 7f75a8e00640 1 --2- 172.21.15.47:0/1649443992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4feec00 0x555ea4ffa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.000+0000 7f75a8e00640 1 -- 172.21.15.47:0/1649443992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ea4d8bf70 con 0x555ea4fe9a90 2024-03-21T11:41:32.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.000+0000 7f75aa200640 1 --2- 172.21.15.47:0/1649443992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4feec00 0x555ea4ffa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:41:32.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.000+0000 7f75a9800640 1 --2- 172.21.15.47:0/1649443992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe90a0 0x555ea4ff3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:41:32.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.001+0000 7f75a8e00640 1 --2- 172.21.15.47:0/1649443992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4fe9a90 0x555ea4ff3760 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f7590009b80 tx=0x7f75900159e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da00f6ec82a5d5f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:32.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.001+0000 7f758b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:41:32.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.002+0000 7f758b400640 1 -- 172.21.15.47:0/1649443992 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590025070 con 0x555ea4fe9a90 2024-03-21T11:41:32.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.002+0000 7f758b400640 1 -- 172.21.15.47:0/1649443992 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7590018a20 con 0x555ea4fe9a90 2024-03-21T11:41:32.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.002+0000 7f75aee10740 1 -- 172.21.15.47:0/1649443992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f759001d070 con 0x555ea4fe9a90 2024-03-21T11:41:32.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.002+0000 7f758b400640 1 -- 172.21.15.47:0/1649443992 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f759002f050 con 0x555ea4fe9a90 2024-03-21T11:41:32.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.002+0000 7f75aee10740 10 client.31435 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:32.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.002+0000 7f75aee10740 1 -- 172.21.15.47:0/1649443992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555ea4c23170 con 0x555ea4fe9a90 2024-03-21T11:41:32.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.003+0000 7f758b400640 1 -- 172.21.15.47:0/1649443992 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f759001d070 con 0x555ea4fe9a90 2024-03-21T11:41:32.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.003+0000 7f758b400640 1 -- 172.21.15.47:0/1649443992 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f759002a920 con 0x555ea4fe9a90 2024-03-21T11:41:32.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.003+0000 7f75aee10740 10 client.31435 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:32.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.003+0000 7f75aee10740 10 client.31435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:32.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.003+0000 7f75aee10740 20 client.31435 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:32.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.003+0000 7f75aee10740 1 --2- 172.21.15.47:0/1649443992 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ea500feb0 0x555ea50122f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.003+0000 7f75aee10740 4 client.31435 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:41:32.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.003+0000 7f75aee10740 1 -- 172.21.15.47:0/1649443992 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ea4f5c3f0 con 0x555ea500feb0 2024-03-21T11:41:32.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.004+0000 7f75a9800640 1 --2- 172.21.15.47:0/1649443992 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ea500feb0 0x555ea50122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.005+0000 7f75a9800640 1 --2- 172.21.15.47:0/1649443992 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ea500feb0 0x555ea50122f0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c64ca5108abb641 server_cookie=420e3abfe02e4435 in_seq=0 out_seq=0 2024-03-21T11:41:32.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.005+0000 7f758b400640 10 client.31435 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:32.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.007+0000 7f758b400640 1 -- 172.21.15.47:0/1649443992 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555ea4f5c3f0 con 0x555ea500feb0 2024-03-21T11:41:32.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.007+0000 7f758b400640 10 client.31435 handle_command_reply: tid=0 2024-03-21T11:41:32.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.007+0000 7f75aee10740 1 client.31435 shutdown 2024-03-21T11:41:32.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.008+0000 7f75aee10740 1 -- 172.21.15.47:0/1649443992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4fe9a90 msgr2=0x555ea4ff3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:32.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.008+0000 7f75aee10740 1 --2- 172.21.15.47:0/1649443992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4fe9a90 0x555ea4ff3760 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f7590009b80 tx=0x7f75900159e0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.008+0000 7f75aee10740 1 -- 172.21.15.47:0/1649443992 shutdown_connections 2024-03-21T11:41:32.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.008+0000 7f75aee10740 1 --2- 172.21.15.47:0/1649443992 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ea500feb0 0x555ea50122f0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.008+0000 7f75aee10740 1 --2- 172.21.15.47:0/1649443992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4feec00 0x555ea4ffa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.008+0000 7f75aee10740 1 --2- 172.21.15.47:0/1649443992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4fe9a90 0x555ea4ff3760 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.008+0000 7f75aee10740 1 --2- 172.21.15.47:0/1649443992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe90a0 0x555ea4ff3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.008+0000 7f758b400640 0 client.31435 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:32.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.008+0000 7f75aee10740 1 -- 172.21.15.47:0/1649443992 >> 172.21.15.47:0/1649443992 conn(0x555ea4fe4610 msgr2=0x555ea4fe3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:32.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.008+0000 7f75aee10740 1 -- 172.21.15.47:0/1649443992 shutdown_connections 2024-03-21T11:41:32.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.008+0000 7f75aee10740 1 -- 172.21.15.47:0/1649443992 wait complete. 2024-03-21T11:41:32.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.008+0000 7f75aee10740 20 client.31435 trim_cache size 0 max 16384 2024-03-21T11:41:32.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.031+0000 7f75aee10740 1 Processor -- start 2024-03-21T11:41:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.031+0000 7f75aee10740 1 -- start start 2024-03-21T11:41:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.031+0000 7f75aee10740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4fe90a0 0x555ea5016bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.031+0000 7f75aee10740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4feec00 0x555ea4ffa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.031+0000 7f75aee10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe4e50 0x555ea5017130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.031+0000 7f75aee10740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a41a3b00 con 0x555ea4feec00 2024-03-21T11:41:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.031+0000 7f75aee10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a41a5480 con 0x555ea4fe4e50 2024-03-21T11:41:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.031+0000 7f75aee10740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a41a4160 con 0x555ea4fe90a0 2024-03-21T11:41:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.032+0000 7f75a9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4fe90a0 0x555ea5016bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.032+0000 7f75a9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4fe90a0 0x555ea5016bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34978/0 (socket says 172.21.15.47:34978) 2024-03-21T11:41:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.032+0000 7f75a9800640 1 -- 172.21.15.47:0/563105631 learned_addr learned my addr 172.21.15.47:0/563105631 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.032+0000 7f75aa200640 1 --2- 172.21.15.47:0/563105631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe4e50 0x555ea5017130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.032+0000 7f75a8e00640 1 --2- 172.21.15.47:0/563105631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4feec00 0x555ea4ffa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.032+0000 7f75a9800640 1 -- 172.21.15.47:0/563105631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe4e50 msgr2=0x555ea5017130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.032+0000 7f75a9800640 1 --2- 172.21.15.47:0/563105631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe4e50 0x555ea5017130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.032+0000 7f75a9800640 1 -- 172.21.15.47:0/563105631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4feec00 msgr2=0x555ea4ffa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.032+0000 7f75a9800640 1 --2- 172.21.15.47:0/563105631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4feec00 0x555ea4ffa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.032+0000 7f75a9800640 1 -- 172.21.15.47:0/563105631 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ea4ff2640 con 0x555ea4fe90a0 2024-03-21T11:41:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.032+0000 7f75a8e00640 1 --2- 172.21.15.47:0/563105631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4feec00 0x555ea4ffa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:41:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.032+0000 7f75aa200640 1 --2- 172.21.15.47:0/563105631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe4e50 0x555ea5017130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:41:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.032+0000 7f75a9800640 1 --2- 172.21.15.47:0/563105631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4fe90a0 0x555ea5016bf0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f7598004f50 tx=0x7f7598046000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be3ae1d01e95c817 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.033+0000 7f7596000640 1 -- 172.21.15.47:0/563105631 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759805a020 con 0x555ea4fe90a0 2024-03-21T11:41:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.033+0000 7f7596000640 1 -- 172.21.15.47:0/563105631 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7598004460 con 0x555ea4fe90a0 2024-03-21T11:41:32.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.033+0000 7f7596000640 1 -- 172.21.15.47:0/563105631 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598046070 con 0x555ea4fe90a0 2024-03-21T11:41:32.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.034+0000 7f75aee10740 1 -- 172.21.15.47:0/563105631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4fe90a0 msgr2=0x555ea5016bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.034+0000 7f75aee10740 1 --2- 172.21.15.47:0/563105631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4fe90a0 0x555ea5016bf0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f7598004f50 tx=0x7f7598046000 comp rx=0 tx=0).stop 2024-03-21T11:41:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.034+0000 7f75aee10740 1 -- 172.21.15.47:0/563105631 shutdown_connections 2024-03-21T11:41:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.034+0000 7f75aee10740 1 --2- 172.21.15.47:0/563105631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe4e50 0x555ea5017130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.034+0000 7f75aee10740 1 --2- 172.21.15.47:0/563105631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4feec00 0x555ea4ffa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.034+0000 7f75aee10740 1 --2- 172.21.15.47:0/563105631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4fe90a0 0x555ea5016bf0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.034+0000 7f75aee10740 1 -- 172.21.15.47:0/563105631 >> 172.21.15.47:0/563105631 conn(0x555ea4fe4610 msgr2=0x555ea4fe8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.034+0000 7f75aee10740 1 -- 172.21.15.47:0/563105631 shutdown_connections 2024-03-21T11:41:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.034+0000 7f75aee10740 1 -- 172.21.15.47:0/563105631 wait complete. 2024-03-21T11:41:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.034+0000 7f75aee10740 1 -- start start 2024-03-21T11:41:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.034+0000 7f75aee10740 1 Processor -- start 2024-03-21T11:41:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.035+0000 7f75aee10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe4e50 0x555ea4ffa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.035+0000 7f75aee10740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4fe90a0 0x555ea4ffa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.035+0000 7f75aee10740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4feec00 0x555ea4ff8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.035+0000 7f75aee10740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a41a4e20 con 0x555ea4fe90a0 2024-03-21T11:41:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.035+0000 7f75aee10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a41a47c0 con 0x555ea4fe4e50 2024-03-21T11:41:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.035+0000 7f75aee10740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a40732f0 con 0x555ea4feec00 2024-03-21T11:41:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.035+0000 7f75aee10740 10 client.0 _ll_register_callbacks cb 0x7f7590009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.035+0000 7f75a9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe4e50 0x555ea4ffa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.035+0000 7f75aa200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4feec00 0x555ea4ff8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.035+0000 7f75a9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe4e50 0x555ea4ffa1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38440/0 (socket says 172.21.15.47:38440) 2024-03-21T11:41:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.035+0000 7f75a9800640 1 -- 172.21.15.47:0/350293200 learned_addr learned my addr 172.21.15.47:0/350293200 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.035+0000 7f75a8e00640 1 --2- 172.21.15.47:0/350293200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4fe90a0 0x555ea4ffa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.036+0000 7f75aa200640 1 -- 172.21.15.47:0/350293200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe4e50 msgr2=0x555ea4ffa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:32.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.036+0000 7f75aa200640 1 --2- 172.21.15.47:0/350293200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe4e50 0x555ea4ffa1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.036+0000 7f75aa200640 1 -- 172.21.15.47:0/350293200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4fe90a0 msgr2=0x555ea4ffa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.036+0000 7f75aa200640 1 --2- 172.21.15.47:0/350293200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4fe90a0 0x555ea4ffa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.036+0000 7f75aa200640 1 -- 172.21.15.47:0/350293200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ea4ff90c0 con 0x555ea4feec00 2024-03-21T11:41:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.036+0000 7f75a9800640 1 --2- 172.21.15.47:0/350293200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe4e50 0x555ea4ffa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:41:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.036+0000 7f75aa200640 1 --2- 172.21.15.47:0/350293200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4feec00 0x555ea4ff8890 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f759c00a6e0 tx=0x7f759c0184c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c09c58bf2e9eba45 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.036+0000 7f758b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:41:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.036+0000 7f758b400640 1 -- 172.21.15.47:0/350293200 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c0a0300 con 0x555ea4feec00 2024-03-21T11:41:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.036+0000 7f75aee10740 1 -- 172.21.15.47:0/350293200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7598004460 con 0x555ea4feec00 2024-03-21T11:41:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.036+0000 7f758b400640 1 -- 172.21.15.47:0/350293200 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f759c0a4730 con 0x555ea4feec00 2024-03-21T11:41:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.036+0000 7f758b400640 1 -- 172.21.15.47:0/350293200 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f759c0a4bc0 con 0x555ea4feec00 2024-03-21T11:41:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.036+0000 7f75aee10740 10 client.31441 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.036+0000 7f75aee10740 1 -- 172.21.15.47:0/350293200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555ea506c4a0 con 0x555ea4feec00 2024-03-21T11:41:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.037+0000 7f758b400640 1 -- 172.21.15.47:0/350293200 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7598004460 con 0x555ea4feec00 2024-03-21T11:41:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.037+0000 7f758b400640 1 -- 172.21.15.47:0/350293200 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f759c0bc0b0 con 0x555ea4feec00 2024-03-21T11:41:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.037+0000 7f75aee10740 10 client.31441 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.037+0000 7f75aee10740 10 client.31441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.037+0000 7f75aee10740 20 client.31441 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.037+0000 7f75aee10740 1 --2- 172.21.15.47:0/350293200 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ea506cd50 0x555ea506f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.037+0000 7f75aee10740 4 client.31441 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:41:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.037+0000 7f75aee10740 1 -- 172.21.15.47:0/350293200 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555ea4f56740 con 0x555ea506cd50 2024-03-21T11:41:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.038+0000 7f75a9800640 1 --2- 172.21.15.47:0/350293200 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ea506cd50 0x555ea506f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.038+0000 7f75a9800640 1 --2- 172.21.15.47:0/350293200 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ea506cd50 0x555ea506f170 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34f174515c520daa server_cookie=cfc5c59b434d1710 in_seq=0 out_seq=0 2024-03-21T11:41:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.038+0000 7f758b400640 10 client.31441 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.040+0000 7f758b400640 1 -- 172.21.15.47:0/350293200 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555ea4f56740 con 0x555ea506cd50 2024-03-21T11:41:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.040+0000 7f758b400640 10 client.31441 handle_command_reply: tid=0 2024-03-21T11:41:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.040+0000 7f75aee10740 1 client.31441 shutdown 2024-03-21T11:41:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.040+0000 7f75aee10740 1 -- 172.21.15.47:0/350293200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4feec00 msgr2=0x555ea4ff8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:32.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.040+0000 7f75aee10740 1 --2- 172.21.15.47:0/350293200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4feec00 0x555ea4ff8890 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f759c00a6e0 tx=0x7f759c0184c0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.040+0000 7f75aee10740 1 -- 172.21.15.47:0/350293200 shutdown_connections 2024-03-21T11:41:32.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.040+0000 7f75aee10740 1 --2- 172.21.15.47:0/350293200 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ea506cd50 0x555ea506f170 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.040+0000 7f75aee10740 1 --2- 172.21.15.47:0/350293200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ea4feec00 0x555ea4ff8890 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.040+0000 7f75aee10740 1 --2- 172.21.15.47:0/350293200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea4fe90a0 0x555ea4ffa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.040+0000 7f75aee10740 1 --2- 172.21.15.47:0/350293200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea4fe4e50 0x555ea4ffa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.040+0000 7f75aee10740 1 -- 172.21.15.47:0/350293200 >> 172.21.15.47:0/350293200 conn(0x555ea4fe4610 msgr2=0x555ea4fe2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.040+0000 7f758b400640 0 client.31441 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.041+0000 7f75aee10740 1 -- 172.21.15.47:0/350293200 shutdown_connections 2024-03-21T11:41:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.041+0000 7f75aee10740 1 -- 172.21.15.47:0/350293200 wait complete. 2024-03-21T11:41:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.041+0000 7f75aee10740 20 client.31441 trim_cache size 0 max 16384 2024-03-21T11:41:32.046 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:41:32.046 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:41:32.047 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "8e7e8575-f150-4ec1-bfd0-d717af683f04", 2024-03-21T11:41:32.047 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:41:32.047 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:41:32.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.042+0000 7f75ac200640 1 -- 172.21.15.47:0/316389385 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f758c0a2f20 msgr2=0x7f758c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:32.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.042+0000 7f75ac200640 1 --2- 172.21.15.47:0/316389385 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f758c0a2f20 0x7f758c0a53e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f759802c170 tx=0x7f759802df90 comp rx=0 tx=0).stop 2024-03-21T11:41:32.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.042+0000 7f75ac200640 1 -- 172.21.15.47:0/316389385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a4105490 msgr2=0x7f75a41a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:32.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.042+0000 7f75ac200640 1 --2- 172.21.15.47:0/316389385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a4105490 0x7f75a41a1210 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f759c00cab0 tx=0x7f759c00cf70 comp rx=0 tx=0).stop 2024-03-21T11:41:32.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.042+0000 7f75ac200640 1 -- 172.21.15.47:0/316389385 shutdown_connections 2024-03-21T11:41:32.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.042+0000 7f75ac200640 1 --2- 172.21.15.47:0/316389385 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f758c0a2f20 0x7f758c0a53e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.042+0000 7f75ac200640 1 --2- 172.21.15.47:0/316389385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75a4105490 0x7f75a41a1210 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.042+0000 7f75ac200640 1 --2- 172.21.15.47:0/316389385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75a4104ad0 0x7f75a419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.042+0000 7f75ac200640 1 --2- 172.21.15.47:0/316389385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75a41038e0 0x7f75a4199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.042+0000 7f75ac200640 1 -- 172.21.15.47:0/316389385 >> 172.21.15.47:0/316389385 conn(0x7f75a40fee50 msgr2=0x7f75a4107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.042+0000 7f75ac200640 1 -- 172.21.15.47:0/316389385 shutdown_connections 2024-03-21T11:41:32.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.042+0000 7f75ac200640 1 -- 172.21.15.47:0/316389385 wait complete. 2024-03-21T11:41:32.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:41:32.060 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:41:32.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.284+0000 7f2a4c400640 1 -- 172.21.15.47:0/1730765950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a441038e0 msgr2=0x7f2a44103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:32.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.284+0000 7f2a4c400640 1 --2- 172.21.15.47:0/1730765950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a441038e0 0x7f2a44103d00 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009900 tx=0x7f2a3802bc30 comp rx=0 tx=0).stop 2024-03-21T11:41:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.284+0000 7f2a4c400640 1 -- 172.21.15.47:0/1730765950 shutdown_connections 2024-03-21T11:41:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.284+0000 7f2a4c400640 1 --2- 172.21.15.47:0/1730765950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a44105490 0x7f2a4410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.284+0000 7f2a4c400640 1 --2- 172.21.15.47:0/1730765950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a44104ad0 0x7f2a44104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.284+0000 7f2a4c400640 1 --2- 172.21.15.47:0/1730765950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a441038e0 0x7f2a44103d00 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.284+0000 7f2a4c400640 1 -- 172.21.15.47:0/1730765950 >> 172.21.15.47:0/1730765950 conn(0x7f2a440fee50 msgr2=0x7f2a44101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.284+0000 7f2a4c400640 1 -- 172.21.15.47:0/1730765950 shutdown_connections 2024-03-21T11:41:32.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.284+0000 7f2a4c400640 1 -- 172.21.15.47:0/1730765950 wait complete. 2024-03-21T11:41:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.285+0000 7f2a4c400640 1 Processor -- start 2024-03-21T11:41:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.285+0000 7f2a4c400640 1 -- start start 2024-03-21T11:41:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.286+0000 7f2a4c400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a441038e0 0x7f2a44199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.286+0000 7f2a4c400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a44104ad0 0x7f2a4419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.286+0000 7f2a4c400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a44105490 0x7f2a441a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.286+0000 7f2a4c400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4410f580 con 0x7f2a44104ad0 2024-03-21T11:41:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.286+0000 7f2a4c400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4410f400 con 0x7f2a44105490 2024-03-21T11:41:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.286+0000 7f2a4c400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4410f700 con 0x7f2a441038e0 2024-03-21T11:41:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.286+0000 7f2a49a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a441038e0 0x7f2a44199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.286+0000 7f2a49a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a441038e0 0x7f2a44199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35012/0 (socket says 172.21.15.47:35012) 2024-03-21T11:41:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.286+0000 7f2a49a00640 1 -- 172.21.15.47:0/1711630457 learned_addr learned my addr 172.21.15.47:0/1711630457 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.286+0000 7f2a4a400640 1 --2- 172.21.15.47:0/1711630457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a44105490 0x7f2a441a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.286+0000 7f2a49000640 1 --2- 172.21.15.47:0/1711630457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a44104ad0 0x7f2a4419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.287+0000 7f2a49a00640 1 -- 172.21.15.47:0/1711630457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a44105490 msgr2=0x7f2a441a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.287+0000 7f2a49a00640 1 --2- 172.21.15.47:0/1711630457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a44105490 0x7f2a441a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.287+0000 7f2a49a00640 1 -- 172.21.15.47:0/1711630457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a44104ad0 msgr2=0x7f2a4419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.287+0000 7f2a49a00640 1 --2- 172.21.15.47:0/1711630457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a44104ad0 0x7f2a4419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.287+0000 7f2a49a00640 1 -- 172.21.15.47:0/1711630457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a441a1880 con 0x7f2a441038e0 2024-03-21T11:41:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.287+0000 7f2a49a00640 1 --2- 172.21.15.47:0/1711630457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a441038e0 0x7f2a44199c70 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f2a38000c00 tx=0x7f2a38005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:32.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.287+0000 7f2a42000640 1 -- 172.21.15.47:0/1711630457 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38043020 con 0x7f2a441038e0 2024-03-21T11:41:32.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.287+0000 7f2a4c400640 1 -- 172.21.15.47:0/1711630457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a441a1b10 con 0x7f2a441038e0 2024-03-21T11:41:32.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.288+0000 7f2a42000640 1 -- 172.21.15.47:0/1711630457 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a380054e0 con 0x7f2a441038e0 2024-03-21T11:41:32.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.288+0000 7f2a42000640 1 -- 172.21.15.47:0/1711630457 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38003e60 con 0x7f2a441038e0 2024-03-21T11:41:32.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.288+0000 7f2a4c400640 1 -- 172.21.15.47:0/1711630457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a441a20b0 con 0x7f2a441038e0 2024-03-21T11:41:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.291+0000 7f2a4f184740 1 Processor -- start 2024-03-21T11:41:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.291+0000 7f2a42000640 1 -- 172.21.15.47:0/1711630457 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2a38046020 con 0x7f2a441038e0 2024-03-21T11:41:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.291+0000 7f2a4f184740 1 -- start start 2024-03-21T11:41:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.291+0000 7f2a4f184740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab7e80 0x5618c0ab82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a4f184740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0ab8870 0x5618c0abd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a4f184740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0abd9e0 0x5618c0abfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a4f184740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c0a1d250 con 0x5618c0ab7e80 2024-03-21T11:41:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a4f184740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c09c5a30 con 0x5618c0ab8870 2024-03-21T11:41:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a4f184740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c0a192f0 con 0x5618c0abd9e0 2024-03-21T11:41:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a49a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0ab8870 0x5618c0abd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a4a400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab7e80 0x5618c0ab82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a49a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0ab8870 0x5618c0abd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38462/0 (socket says 172.21.15.47:38462) 2024-03-21T11:41:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a49a00640 1 -- 172.21.15.47:0/3352270673 learned_addr learned my addr 172.21.15.47:0/3352270673 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a49000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0abd9e0 0x5618c0abfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a49000640 1 -- 172.21.15.47:0/3352270673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0ab8870 msgr2=0x5618c0abd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a49000640 1 --2- 172.21.15.47:0/3352270673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0ab8870 0x5618c0abd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a49000640 1 -- 172.21.15.47:0/3352270673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab7e80 msgr2=0x5618c0ab82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a49000640 1 --2- 172.21.15.47:0/3352270673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab7e80 0x5618c0ab82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a49000640 1 -- 172.21.15.47:0/3352270673 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618c09b0690 con 0x5618c0abd9e0 2024-03-21T11:41:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.292+0000 7f2a4a400640 1 --2- 172.21.15.47:0/3352270673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab7e80 0x5618c0ab82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:41:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.293+0000 7f2a49000640 1 --2- 172.21.15.47:0/3352270673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0abd9e0 0x5618c0abfe70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000bbf0 tx=0x7f2a300107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=853b96d6816bb1b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.293+0000 7f2a42000640 1 --2- 172.21.15.47:0/1711630457 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2a2c0a2f20 0x7f2a2c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.293+0000 7f2a36a00640 1 -- 172.21.15.47:0/3352270673 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30004030 con 0x5618c0abd9e0 2024-03-21T11:41:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.293+0000 7f2a36a00640 1 -- 172.21.15.47:0/3352270673 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a300041d0 con 0x5618c0abd9e0 2024-03-21T11:41:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.293+0000 7f2a36a00640 1 -- 172.21.15.47:0/3352270673 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a300116c0 con 0x5618c0abd9e0 2024-03-21T11:41:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.293+0000 7f2a42000640 1 -- 172.21.15.47:0/1711630457 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2a380f19f0 con 0x7f2a441038e0 2024-03-21T11:41:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.293+0000 7f2a4a400640 1 --2- 172.21.15.47:0/1711630457 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2a2c0a2f20 0x7f2a2c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.294+0000 7f2a4f184740 1 -- 172.21.15.47:0/3352270673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0abd9e0 msgr2=0x5618c0abfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.294+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3352270673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0abd9e0 0x5618c0abfe70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000bbf0 tx=0x7f2a300107a0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.294+0000 7f2a4a400640 1 --2- 172.21.15.47:0/1711630457 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2a2c0a2f20 0x7f2a2c0a53e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x5618c0ab90d0 tx=0x7f2a3c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.295+0000 7f2a4f184740 1 -- 172.21.15.47:0/3352270673 shutdown_connections 2024-03-21T11:41:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.295+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3352270673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0abd9e0 0x5618c0abfe70 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.295+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3352270673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0ab8870 0x5618c0abd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.295+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3352270673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab7e80 0x5618c0ab82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.295+0000 7f2a4f184740 1 -- 172.21.15.47:0/3352270673 >> 172.21.15.47:0/3352270673 conn(0x5618c0a80200 msgr2=0x5618c0ab37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.295+0000 7f2a4f184740 1 -- 172.21.15.47:0/3352270673 shutdown_connections 2024-03-21T11:41:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.295+0000 7f2a4f184740 1 -- 172.21.15.47:0/3352270673 wait complete. 2024-03-21T11:41:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.295+0000 7f2a4f184740 1 -- start start 2024-03-21T11:41:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.296+0000 7f2a4f184740 1 Processor -- start 2024-03-21T11:41:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.296+0000 7f2a4f184740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0ab7e80 0x5618c0aca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.296+0000 7f2a4f184740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab8870 0x5618c0acab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.296+0000 7f2a4f184740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0abd9e0 0x5618c0ad1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.296+0000 7f2a4f184740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c0ac0990 con 0x5618c0ab8870 2024-03-21T11:41:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.296+0000 7f2a4f184740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c09d8e80 con 0x5618c0abd9e0 2024-03-21T11:41:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.296+0000 7f2a4f184740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c0ac0b10 con 0x5618c0ab7e80 2024-03-21T11:41:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.296+0000 7f2a4f184740 10 client.0 _ll_register_callbacks cb 0x5618c09eb130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.296+0000 7f2a49a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0ab7e80 0x5618c0aca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.296+0000 7f2a4a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0abd9e0 0x5618c0ad1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.296+0000 7f2a49a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0ab7e80 0x5618c0aca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35036/0 (socket says 172.21.15.47:35036) 2024-03-21T11:41:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.296+0000 7f2a49a00640 1 -- 172.21.15.47:0/3265323474 learned_addr learned my addr 172.21.15.47:0/3265323474 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.297+0000 7f2a49000640 1 --2- 172.21.15.47:0/3265323474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab8870 0x5618c0acab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.297+0000 7f2a49a00640 1 -- 172.21.15.47:0/3265323474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0abd9e0 msgr2=0x5618c0ad1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.297+0000 7f2a49a00640 1 --2- 172.21.15.47:0/3265323474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0abd9e0 0x5618c0ad1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.297+0000 7f2a49a00640 1 -- 172.21.15.47:0/3265323474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab8870 msgr2=0x5618c0acab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.297+0000 7f2a49a00640 1 --2- 172.21.15.47:0/3265323474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab8870 0x5618c0acab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.297+0000 7f2a49a00640 1 -- 172.21.15.47:0/3265323474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618c07fb7d0 con 0x5618c0ab7e80 2024-03-21T11:41:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.297+0000 7f2a49a00640 1 --2- 172.21.15.47:0/3265323474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0ab7e80 0x5618c0aca630 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f2a380f1c90 tx=0x7f2a380bbd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40c0f35b744f3252 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.297+0000 7f2a2b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:41:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.298+0000 7f2a2b400640 1 -- 172.21.15.47:0/3265323474 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a380bdb00 con 0x5618c0ab7e80 2024-03-21T11:41:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.298+0000 7f2a2b400640 1 -- 172.21.15.47:0/3265323474 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2a380c62b0 con 0x5618c0ab7e80 2024-03-21T11:41:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.298+0000 7f2a4f184740 1 -- 172.21.15.47:0/3265323474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a300041d0 con 0x5618c0ab7e80 2024-03-21T11:41:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.298+0000 7f2a2b400640 1 -- 172.21.15.47:0/3265323474 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a380bbe70 con 0x5618c0ab7e80 2024-03-21T11:41:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.298+0000 7f2a4f184740 10 client.31453 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.298+0000 7f2a4f184740 1 -- 172.21.15.47:0/3265323474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5618c07313f0 con 0x5618c0ab7e80 2024-03-21T11:41:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.298+0000 7f2a2b400640 1 -- 172.21.15.47:0/3265323474 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a300041d0 con 0x5618c0ab7e80 2024-03-21T11:41:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.299+0000 7f2a2b400640 1 -- 172.21.15.47:0/3265323474 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2a380bd180 con 0x5618c0ab7e80 2024-03-21T11:41:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.299+0000 7f2a4f184740 10 client.31453 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.299+0000 7f2a4f184740 10 client.31453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.299+0000 7f2a4f184740 20 client.31453 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.299+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3265323474 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5618c0ad6430 0x5618c0ad88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.299+0000 7f2a4f184740 4 client.31453 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:41:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.299+0000 7f2a4f184740 1 -- 172.21.15.47:0/3265323474 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618c0a2a790 con 0x5618c0ad6430 2024-03-21T11:41:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.299+0000 7f2a49000640 1 --2- 172.21.15.47:0/3265323474 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5618c0ad6430 0x5618c0ad88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.300+0000 7f2a49000640 1 --2- 172.21.15.47:0/3265323474 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5618c0ad6430 0x5618c0ad88a0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e1c6a8e9ec3ad2 server_cookie=471e5b9217a37dc0 in_seq=0 out_seq=0 2024-03-21T11:41:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.300+0000 7f2a2b400640 10 client.31453 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.302+0000 7f2a2b400640 1 -- 172.21.15.47:0/3265323474 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5618c0a2a790 con 0x5618c0ad6430 2024-03-21T11:41:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.303+0000 7f2a2b400640 10 client.31453 handle_command_reply: tid=0 2024-03-21T11:41:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.303+0000 7f2a4f184740 1 client.31453 shutdown 2024-03-21T11:41:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.303+0000 7f2a4f184740 1 -- 172.21.15.47:0/3265323474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0ab7e80 msgr2=0x5618c0aca630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.303+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3265323474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0ab7e80 0x5618c0aca630 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f2a380f1c90 tx=0x7f2a380bbd40 comp rx=0 tx=0).stop 2024-03-21T11:41:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.303+0000 7f2a4f184740 1 -- 172.21.15.47:0/3265323474 shutdown_connections 2024-03-21T11:41:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.303+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3265323474 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5618c0ad6430 0x5618c0ad88a0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.303+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3265323474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0abd9e0 0x5618c0ad1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.303+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3265323474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab8870 0x5618c0acab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.303+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3265323474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0ab7e80 0x5618c0aca630 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.303+0000 7f2a2b400640 0 client.31453 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.303+0000 7f2a4f184740 1 -- 172.21.15.47:0/3265323474 >> 172.21.15.47:0/3265323474 conn(0x5618c0a80200 msgr2=0x5618c0ab2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.303+0000 7f2a4f184740 1 -- 172.21.15.47:0/3265323474 shutdown_connections 2024-03-21T11:41:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.303+0000 7f2a4f184740 1 -- 172.21.15.47:0/3265323474 wait complete. 2024-03-21T11:41:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.304+0000 7f2a4f184740 20 client.31453 trim_cache size 0 max 16384 2024-03-21T11:41:32.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.324+0000 7f2a4f184740 1 Processor -- start 2024-03-21T11:41:32.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.324+0000 7f2a4f184740 1 -- start start 2024-03-21T11:41:32.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.325+0000 7f2a4f184740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab7e80 0x5618c0ab9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.325+0000 7f2a4f184740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0ab8870 0x5618c0ad1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.325+0000 7f2a4f184740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0abd9e0 0x5618c0ab6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.325+0000 7f2a4f184740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a441a7fa0 con 0x5618c0ab7e80 2024-03-21T11:41:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.325+0000 7f2a4f184740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a441a9920 con 0x5618c0ab8870 2024-03-21T11:41:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.325+0000 7f2a4f184740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a441a8600 con 0x5618c0abd9e0 2024-03-21T11:41:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.325+0000 7f2a49000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0ab8870 0x5618c0ad1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.325+0000 7f2a49a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab7e80 0x5618c0ab9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.325+0000 7f2a49000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0ab8870 0x5618c0ad1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38486/0 (socket says 172.21.15.47:38486) 2024-03-21T11:41:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.325+0000 7f2a49a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab7e80 0x5618c0ab9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48942/0 (socket says 172.21.15.47:48942) 2024-03-21T11:41:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.325+0000 7f2a4a400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0abd9e0 0x5618c0ab6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.325+0000 7f2a49000640 1 -- 172.21.15.47:0/2381867048 learned_addr learned my addr 172.21.15.47:0/2381867048 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.326+0000 7f2a49000640 1 -- 172.21.15.47:0/2381867048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0abd9e0 msgr2=0x5618c0ab6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.326+0000 7f2a49000640 1 --2- 172.21.15.47:0/2381867048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0abd9e0 0x5618c0ab6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.326+0000 7f2a49000640 1 -- 172.21.15.47:0/2381867048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab7e80 msgr2=0x5618c0ab9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.326+0000 7f2a49000640 1 --2- 172.21.15.47:0/2381867048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab7e80 0x5618c0ab9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.326+0000 7f2a49000640 1 -- 172.21.15.47:0/2381867048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618c06f10f0 con 0x5618c0ab8870 2024-03-21T11:41:32.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.326+0000 7f2a49000640 1 --2- 172.21.15.47:0/2381867048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0ab8870 0x5618c0ad1b70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000b7f0 tx=0x7f2a30013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9e0ef7d5596453a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:32.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.326+0000 7f2a36000640 1 -- 172.21.15.47:0/2381867048 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a300052d0 con 0x5618c0ab8870 2024-03-21T11:41:32.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.326+0000 7f2a36000640 1 -- 172.21.15.47:0/2381867048 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a30030070 con 0x5618c0ab8870 2024-03-21T11:41:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.326+0000 7f2a36000640 1 -- 172.21.15.47:0/2381867048 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a300035f0 con 0x5618c0ab8870 2024-03-21T11:41:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.327+0000 7f2a4f184740 1 -- 172.21.15.47:0/2381867048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0ab8870 msgr2=0x5618c0ad1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.327+0000 7f2a4f184740 1 --2- 172.21.15.47:0/2381867048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0ab8870 0x5618c0ad1b70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000b7f0 tx=0x7f2a30013040 comp rx=0 tx=0).stop 2024-03-21T11:41:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.327+0000 7f2a4f184740 1 -- 172.21.15.47:0/2381867048 shutdown_connections 2024-03-21T11:41:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.327+0000 7f2a4f184740 1 --2- 172.21.15.47:0/2381867048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0abd9e0 0x5618c0ab6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.327+0000 7f2a4f184740 1 --2- 172.21.15.47:0/2381867048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0ab8870 0x5618c0ad1b70 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.327+0000 7f2a4f184740 1 --2- 172.21.15.47:0/2381867048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab7e80 0x5618c0ab9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.327+0000 7f2a4f184740 1 -- 172.21.15.47:0/2381867048 >> 172.21.15.47:0/2381867048 conn(0x5618c0a80200 msgr2=0x5618c0ab9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.327+0000 7f2a4f184740 1 -- 172.21.15.47:0/2381867048 shutdown_connections 2024-03-21T11:41:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.327+0000 7f2a4f184740 1 -- 172.21.15.47:0/2381867048 wait complete. 2024-03-21T11:41:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.328+0000 7f2a4f184740 1 -- start start 2024-03-21T11:41:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.328+0000 7f2a4f184740 1 Processor -- start 2024-03-21T11:41:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.328+0000 7f2a4f184740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0ab7e80 0x5618c0acf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a4f184740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab8870 0x5618c0acfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a4f184740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0abd9e0 0x5618c0ad0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a4f184740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a441a92c0 con 0x5618c0ab8870 2024-03-21T11:41:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a4f184740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a441a8c60 con 0x5618c0abd9e0 2024-03-21T11:41:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a4f184740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a441a6160 con 0x5618c0ab7e80 2024-03-21T11:41:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a4f184740 10 client.0 _ll_register_callbacks cb 0x7f2a380ef040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a49a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0ab7e80 0x5618c0acf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a49000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab8870 0x5618c0acfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a49a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0ab7e80 0x5618c0acf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35056/0 (socket says 172.21.15.47:35056) 2024-03-21T11:41:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a49a00640 1 -- 172.21.15.47:0/3546135816 learned_addr learned my addr 172.21.15.47:0/3546135816 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:32.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a4a400640 1 --2- 172.21.15.47:0/3546135816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0abd9e0 0x5618c0ad0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a49000640 1 -- 172.21.15.47:0/3546135816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0ab7e80 msgr2=0x5618c0acf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a49000640 1 --2- 172.21.15.47:0/3546135816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0ab7e80 0x5618c0acf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a49000640 1 -- 172.21.15.47:0/3546135816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0abd9e0 msgr2=0x5618c0ad0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a49000640 1 --2- 172.21.15.47:0/3546135816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0abd9e0 0x5618c0ad0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a49000640 1 -- 172.21.15.47:0/3546135816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618c0ad1050 con 0x5618c0ab8870 2024-03-21T11:41:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.329+0000 7f2a49a00640 1 --2- 172.21.15.47:0/3546135816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0ab7e80 0x5618c0acf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:41:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.330+0000 7f2a49000640 1 --2- 172.21.15.47:0/3546135816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab8870 0x5618c0acfa50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2a3002d040 tx=0x7f2a3000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816a7fd1683335b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.330+0000 7f2a2b400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:41:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.330+0000 7f2a2b400640 1 -- 172.21.15.47:0/3546135816 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3002a9f0 con 0x5618c0ab8870 2024-03-21T11:41:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.330+0000 7f2a2b400640 1 -- 172.21.15.47:0/3546135816 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2a300052d0 con 0x5618c0ab8870 2024-03-21T11:41:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.330+0000 7f2a4f184740 1 -- 172.21.15.47:0/3546135816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a30030070 con 0x5618c0ab8870 2024-03-21T11:41:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.330+0000 7f2a2b400640 1 -- 172.21.15.47:0/3546135816 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a30045050 con 0x5618c0ab8870 2024-03-21T11:41:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.330+0000 7f2a4f184740 10 client.20931 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.330+0000 7f2a4f184740 1 -- 172.21.15.47:0/3546135816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5618c0b31f60 con 0x5618c0ab8870 2024-03-21T11:41:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.331+0000 7f2a2b400640 1 -- 172.21.15.47:0/3546135816 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a30030070 con 0x5618c0ab8870 2024-03-21T11:41:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.331+0000 7f2a2b400640 1 -- 172.21.15.47:0/3546135816 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2a30003b60 con 0x5618c0ab8870 2024-03-21T11:41:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.331+0000 7f2a4f184740 10 client.20931 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.331+0000 7f2a4f184740 10 client.20931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.331+0000 7f2a4f184740 20 client.20931 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.331+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3546135816 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5618c0b32660 0x5618c0b34a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.331+0000 7f2a4f184740 4 client.20931 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:41:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.331+0000 7f2a4f184740 1 -- 172.21.15.47:0/3546135816 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5618c0a24ae0 con 0x5618c0b32660 2024-03-21T11:41:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.331+0000 7f2a49a00640 1 --2- 172.21.15.47:0/3546135816 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5618c0b32660 0x5618c0b34a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.332+0000 7f2a49a00640 1 --2- 172.21.15.47:0/3546135816 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5618c0b32660 0x5618c0b34a80 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e302b92005d6b0 server_cookie=dacbe28d13e114ad in_seq=0 out_seq=0 2024-03-21T11:41:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.332+0000 7f2a2b400640 10 client.20931 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:32.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a2b400640 1 -- 172.21.15.47:0/3546135816 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x5618c0a24ae0 con 0x5618c0b32660 2024-03-21T11:41:32.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a2b400640 10 client.20931 handle_command_reply: tid=0 2024-03-21T11:41:32.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a4f184740 1 client.20931 shutdown 2024-03-21T11:41:32.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a4f184740 1 -- 172.21.15.47:0/3546135816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab8870 msgr2=0x5618c0acfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:32.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3546135816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab8870 0x5618c0acfa50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2a3002d040 tx=0x7f2a3000b410 comp rx=0 tx=0).stop 2024-03-21T11:41:32.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a4f184740 1 -- 172.21.15.47:0/3546135816 shutdown_connections 2024-03-21T11:41:32.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3546135816 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5618c0b32660 0x5618c0b34a80 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3546135816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5618c0abd9e0 0x5618c0ad0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3546135816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5618c0ab8870 0x5618c0acfa50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a4f184740 1 --2- 172.21.15.47:0/3546135816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618c0ab7e80 0x5618c0acf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a2b400640 0 client.20931 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:32.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a4f184740 1 -- 172.21.15.47:0/3546135816 >> 172.21.15.47:0/3546135816 conn(0x5618c0a80200 msgr2=0x5618c0acaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:32.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a4f184740 1 -- 172.21.15.47:0/3546135816 shutdown_connections 2024-03-21T11:41:32.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a4f184740 1 -- 172.21.15.47:0/3546135816 wait complete. 2024-03-21T11:41:32.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.335+0000 7f2a4f184740 20 client.20931 trim_cache size 0 max 16384 2024-03-21T11:41:32.339 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:41:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (59 inodes in the stack)", 2024-03-21T11:41:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:41:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "8e7e8575-f150-4ec1-bfd0-d717af683f04": { 2024-03-21T11:41:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:41:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "8e7e8575-f150-4ec1-bfd0-d717af683f04", 2024-03-21T11:41:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:41:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:41:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:41:32.339 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:41:32.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.336+0000 7f2a4c400640 1 -- 172.21.15.47:0/1711630457 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2a2c0a2f20 msgr2=0x7f2a2c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:32.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.336+0000 7f2a4c400640 1 --2- 172.21.15.47:0/1711630457 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2a2c0a2f20 0x7f2a2c0a53e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x5618c0ab90d0 tx=0x7f2a3c009600 comp rx=0 tx=0).stop 2024-03-21T11:41:32.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.336+0000 7f2a4c400640 1 -- 172.21.15.47:0/1711630457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a441038e0 msgr2=0x7f2a44199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:32.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.336+0000 7f2a4c400640 1 --2- 172.21.15.47:0/1711630457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a441038e0 0x7f2a44199c70 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f2a38000c00 tx=0x7f2a38005dd0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.336+0000 7f2a4c400640 1 -- 172.21.15.47:0/1711630457 shutdown_connections 2024-03-21T11:41:32.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.336+0000 7f2a4c400640 1 --2- 172.21.15.47:0/1711630457 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2a2c0a2f20 0x7f2a2c0a53e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.336+0000 7f2a4c400640 1 --2- 172.21.15.47:0/1711630457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a44105490 0x7f2a441a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.336+0000 7f2a4c400640 1 --2- 172.21.15.47:0/1711630457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a44104ad0 0x7f2a4419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.336+0000 7f2a4c400640 1 --2- 172.21.15.47:0/1711630457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2a441038e0 0x7f2a44199c70 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:32.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.336+0000 7f2a4c400640 1 -- 172.21.15.47:0/1711630457 >> 172.21.15.47:0/1711630457 conn(0x7f2a440fee50 msgr2=0x7f2a44107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:32.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.337+0000 7f2a4c400640 1 -- 172.21.15.47:0/1711630457 shutdown_connections 2024-03-21T11:41:32.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:41:32.337+0000 7f2a4c400640 1 -- 172.21.15.47:0/1711630457 wait complete. 2024-03-21T11:41:32.356 INFO:tasks.cephfs.filesystem:scrub status for tag:8e7e8575-f150-4ec1-bfd0-d717af683f04 - {'path': '/', 'tag': '8e7e8575-f150-4ec1-bfd0-d717af683f04', 'options': 'recursive,force'} 2024-03-21T11:41:32.950 INFO:tasks.workunit.client.0.smithi047.stdout: 10 118926 23.96 MB/sec execute 136 sec latency 107.288 ms 2024-03-21T11:41:33.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: pgmap v946: 129 pgs: 129 active+clean; 469 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.5 MiB/s wr, 613 op/s 2024-03-21T11:41:33.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:41:33.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:41:33.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:41:33.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:41:33.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:41:33.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:41:33.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:41:33.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:41:33.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:41:33.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:41:33.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:32 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: pgmap v946: 129 pgs: 129 active+clean; 469 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.5 MiB/s wr, 613 op/s 2024-03-21T11:41:33.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:41:33.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:41:33.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:41:33.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:41:33.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:41:33.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:41:33.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:41:33.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:41:33.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:41:33.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:41:33.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:32 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: pgmap v946: 129 pgs: 129 active+clean; 469 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.5 MiB/s wr, 613 op/s 2024-03-21T11:41:33.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:41:33.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:41:33.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:41:33.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:41:33.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:41:33.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:41:33.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:41:33.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:41:33.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:41:33.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:41:33.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:32 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:41:33.950 INFO:tasks.workunit.client.0.smithi047.stdout: 10 119463 23.99 MB/sec execute 137 sec latency 84.032 ms 2024-03-21T11:41:34.950 INFO:tasks.workunit.client.0.smithi047.stdout: 10 119846 24.01 MB/sec execute 138 sec latency 242.413 ms 2024-03-21T11:41:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:34 smithi102 ceph-mon[36748]: pgmap v947: 129 pgs: 129 active+clean; 471 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.2 MiB/s wr, 604 op/s 2024-03-21T11:41:35.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:34 smithi047 ceph-mon[31351]: pgmap v947: 129 pgs: 129 active+clean; 471 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.2 MiB/s wr, 604 op/s 2024-03-21T11:41:35.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:34 smithi100 ceph-mon[36245]: pgmap v947: 129 pgs: 129 active+clean; 471 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.2 MiB/s wr, 604 op/s 2024-03-21T11:41:35.950 INFO:tasks.workunit.client.0.smithi047.stdout: 10 120353 24.00 MB/sec execute 139 sec latency 125.910 ms 2024-03-21T11:41:36.950 INFO:tasks.workunit.client.0.smithi047.stdout: 10 120849 24.06 MB/sec execute 140 sec latency 90.949 ms 2024-03-21T11:41:37.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:36 smithi102 ceph-mon[36748]: pgmap v948: 129 pgs: 129 active+clean; 472 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.9 MiB/s wr, 719 op/s 2024-03-21T11:41:37.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:36 smithi047 ceph-mon[31351]: pgmap v948: 129 pgs: 129 active+clean; 472 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.9 MiB/s wr, 719 op/s 2024-03-21T11:41:37.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:36 smithi100 ceph-mon[36245]: pgmap v948: 129 pgs: 129 active+clean; 472 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.9 MiB/s wr, 719 op/s 2024-03-21T11:41:37.950 INFO:tasks.workunit.client.0.smithi047.stdout: 10 121341 24.09 MB/sec execute 141 sec latency 86.220 ms 2024-03-21T11:41:38.950 INFO:tasks.workunit.client.0.smithi047.stdout: 10 121744 24.08 MB/sec execute 142 sec latency 83.795 ms 2024-03-21T11:41:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:38 smithi102 ceph-mon[36748]: pgmap v949: 129 pgs: 129 active+clean; 476 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 618 op/s 2024-03-21T11:41:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:38 smithi047 ceph-mon[31351]: pgmap v949: 129 pgs: 129 active+clean; 476 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 618 op/s 2024-03-21T11:41:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:38 smithi100 ceph-mon[36245]: pgmap v949: 129 pgs: 129 active+clean; 476 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 618 op/s 2024-03-21T11:41:39.950 INFO:tasks.workunit.client.0.smithi047.stdout: 10 121984 24.01 MB/sec execute 143 sec latency 382.529 ms 2024-03-21T11:41:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.441+0000 7f38b1600640 1 -- 172.21.15.47:0/1457096148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38ac104a90 msgr2=0x7f38ac106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.441+0000 7f38b1600640 1 --2- 172.21.15.47:0/1457096148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38ac104a90 0x7f38ac106e80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f3898009900 tx=0x7f389802bc30 comp rx=0 tx=0).stop 2024-03-21T11:41:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.441+0000 7f38b1600640 1 -- 172.21.15.47:0/1457096148 shutdown_connections 2024-03-21T11:41:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.441+0000 7f38b1600640 1 --2- 172.21.15.47:0/1457096148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38ac1073c0 0x7f38ac109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.441+0000 7f38b1600640 1 --2- 172.21.15.47:0/1457096148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38ac104a90 0x7f38ac106e80 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.441+0000 7f38b1600640 1 --2- 172.21.15.47:0/1457096148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38ac102160 0x7f38ac104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.441+0000 7f38b1600640 1 -- 172.21.15.47:0/1457096148 >> 172.21.15.47:0/1457096148 conn(0x7f38ac0fbb50 msgr2=0x7f38ac0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.441+0000 7f38b1600640 1 -- 172.21.15.47:0/1457096148 shutdown_connections 2024-03-21T11:41:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.441+0000 7f38b1600640 1 -- 172.21.15.47:0/1457096148 wait complete. 2024-03-21T11:41:40.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.443+0000 7f38b1600640 1 Processor -- start 2024-03-21T11:41:40.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.443+0000 7f38b1600640 1 -- start start 2024-03-21T11:41:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.444+0000 7f38b1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38ac102160 0x7f38ac1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.444+0000 7f38b1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38ac104a90 0x7f38ac197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.444+0000 7f38b1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38ac1073c0 0x7f38ac1a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.444+0000 7f38b1600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac10d3b0 con 0x7f38ac1073c0 2024-03-21T11:41:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.444+0000 7f38b1600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac10d230 con 0x7f38ac104a90 2024-03-21T11:41:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.444+0000 7f38b1600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac10d530 con 0x7f38ac102160 2024-03-21T11:41:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.444+0000 7f38aa000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38ac104a90 0x7f38ac197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.444+0000 7f38aa000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38ac104a90 0x7f38ac197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44048/0 (socket says 172.21.15.47:44048) 2024-03-21T11:41:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.444+0000 7f38aa000640 1 -- 172.21.15.47:0/2291415051 learned_addr learned my addr 172.21.15.47:0/2291415051 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:40.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.444+0000 7f38aaa00640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38ac102160 0x7f38ac1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.445+0000 7f38aa000640 1 -- 172.21.15.47:0/2291415051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38ac102160 msgr2=0x7f38ac1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.445+0000 7f38aa000640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38ac102160 0x7f38ac1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.445+0000 7f38aa000640 1 -- 172.21.15.47:0/2291415051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38ac1073c0 msgr2=0x7f38ac1a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.445+0000 7f38ab400640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38ac1073c0 0x7f38ac1a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.445+0000 7f38aa000640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38ac1073c0 0x7f38ac1a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.445+0000 7f38aa000640 1 -- 172.21.15.47:0/2291415051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac19af80 con 0x7f38ac104a90 2024-03-21T11:41:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.445+0000 7f38ab400640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38ac1073c0 0x7f38ac1a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:41:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.445+0000 7f38aa000640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38ac104a90 0x7f38ac197f30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3898000c00 tx=0x7f3898005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.446+0000 7f38a3400640 1 -- 172.21.15.47:0/2291415051 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898041070 con 0x7f38ac104a90 2024-03-21T11:41:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.446+0000 7f38a3400640 1 -- 172.21.15.47:0/2291415051 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38980044d0 con 0x7f38ac104a90 2024-03-21T11:41:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.446+0000 7f38a3400640 1 -- 172.21.15.47:0/2291415051 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898034440 con 0x7f38ac104a90 2024-03-21T11:41:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.446+0000 7f38b1600640 1 -- 172.21.15.47:0/2291415051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ac19b210 con 0x7f38ac104a90 2024-03-21T11:41:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.446+0000 7f38b1600640 1 -- 172.21.15.47:0/2291415051 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ac19b750 con 0x7f38ac104a90 2024-03-21T11:41:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.451+0000 7f38b43b4740 1 Processor -- start 2024-03-21T11:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.451+0000 7f38b43b4740 1 -- start start 2024-03-21T11:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.451+0000 7f38b43b4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f82600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.451+0000 7f38b43b4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f876d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.451+0000 7f38b43b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 0x561d65f8a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.451+0000 7f38b43b4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65f453b0 con 0x561d65f82bd0 2024-03-21T11:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.451+0000 7f38b43b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65ee7830 con 0x561d65f87d40 2024-03-21T11:41:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.451+0000 7f38b43b4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65e905d0 con 0x561d65f821e0 2024-03-21T11:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.451+0000 7f38ab400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f82600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.451+0000 7f38ab400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f82600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55404/0 (socket says 172.21.15.47:55404) 2024-03-21T11:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.451+0000 7f38ab400640 1 -- 172.21.15.47:0/735120273 learned_addr learned my addr 172.21.15.47:0/735120273 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.452+0000 7f38aaa00640 1 --2- 172.21.15.47:0/735120273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f876d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.452+0000 7f38ab400640 1 -- 172.21.15.47:0/735120273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 msgr2=0x561d65f8a1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.452+0000 7f38ab400640 1 --2- 172.21.15.47:0/735120273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 0x561d65f8a1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.452+0000 7f38ab400640 1 -- 172.21.15.47:0/735120273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 msgr2=0x561d65f876d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.452+0000 7f38ab400640 1 --2- 172.21.15.47:0/735120273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f876d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.452+0000 7f38ab400640 1 -- 172.21.15.47:0/735120273 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d65ea82c0 con 0x561d65f821e0 2024-03-21T11:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.452+0000 7f38a3400640 1 -- 172.21.15.47:0/2291415051 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3898044070 con 0x7f38ac104a90 2024-03-21T11:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.453+0000 7f38ab400640 1 --2- 172.21.15.47:0/735120273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f82600 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f38a4007f30 tx=0x7f38a400c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da2ea1e36e55a68d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.453+0000 7f3897400640 1 -- 172.21.15.47:0/735120273 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4014070 con 0x561d65f821e0 2024-03-21T11:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.453+0000 7f3897400640 1 -- 172.21.15.47:0/735120273 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38a400fd30 con 0x561d65f821e0 2024-03-21T11:41:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.453+0000 7f3897400640 1 -- 172.21.15.47:0/735120273 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a400ed30 con 0x561d65f821e0 2024-03-21T11:41:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.453+0000 7f38a3400640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f38900a3040 0x7f38900a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.453+0000 7f38a3400640 1 -- 172.21.15.47:0/2291415051 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f38980f1320 con 0x7f38ac104a90 2024-03-21T11:41:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.454+0000 7f38b43b4740 1 -- 172.21.15.47:0/735120273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 msgr2=0x561d65f82600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.454+0000 7f38b43b4740 1 --2- 172.21.15.47:0/735120273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f82600 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f38a4007f30 tx=0x7f38a400c750 comp rx=0 tx=0).stop 2024-03-21T11:41:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.454+0000 7f38aaa00640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f38900a3040 0x7f38900a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.454+0000 7f38b43b4740 1 -- 172.21.15.47:0/735120273 shutdown_connections 2024-03-21T11:41:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.454+0000 7f38b43b4740 1 --2- 172.21.15.47:0/735120273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 0x561d65f8a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.454+0000 7f38b43b4740 1 --2- 172.21.15.47:0/735120273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f876d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.454+0000 7f38b43b4740 1 --2- 172.21.15.47:0/735120273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f82600 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.454+0000 7f38b43b4740 1 -- 172.21.15.47:0/735120273 >> 172.21.15.47:0/735120273 conn(0x561d65f7d750 msgr2=0x561d65f7db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.454+0000 7f38b43b4740 1 -- 172.21.15.47:0/735120273 shutdown_connections 2024-03-21T11:41:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.454+0000 7f38b43b4740 1 -- 172.21.15.47:0/735120273 wait complete. 2024-03-21T11:41:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.455+0000 7f38aaa00640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f38900a3040 0x7f38900a5500 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x561d65f7e6c0 tx=0x7f389c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.455+0000 7f38b43b4740 1 -- start start 2024-03-21T11:41:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.455+0000 7f38b43b4740 1 Processor -- start 2024-03-21T11:41:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.455+0000 7f38b43b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f821e0 0x561d65f8c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.455+0000 7f38b43b4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f8c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.455+0000 7f38b43b4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f87d40 0x561d65f93920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.455+0000 7f38b43b4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65ea3a20 con 0x561d65f82bd0 2024-03-21T11:41:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.455+0000 7f38b43b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65ee38d0 con 0x561d65f821e0 2024-03-21T11:41:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.455+0000 7f38b43b4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d65f8acf0 con 0x561d65f87d40 2024-03-21T11:41:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.455+0000 7f38b43b4740 10 client.0 _ll_register_callbacks cb 0x561d65ee01c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.455+0000 7f38aaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f821e0 0x561d65f8c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.456+0000 7f38aaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f821e0 0x561d65f8c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44052/0 (socket says 172.21.15.47:44052) 2024-03-21T11:41:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.456+0000 7f38aaa00640 1 -- 172.21.15.47:0/4196696680 learned_addr learned my addr 172.21.15.47:0/4196696680 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.456+0000 7f38ab400640 1 --2- 172.21.15.47:0/4196696680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f87d40 0x561d65f93920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.456+0000 7f38aa000640 1 --2- 172.21.15.47:0/4196696680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f8c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.456+0000 7f38aaa00640 1 -- 172.21.15.47:0/4196696680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f87d40 msgr2=0x561d65f93920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.456+0000 7f38aaa00640 1 --2- 172.21.15.47:0/4196696680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f87d40 0x561d65f93920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.456+0000 7f38aaa00640 1 -- 172.21.15.47:0/4196696680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 msgr2=0x561d65f8c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.456+0000 7f38aaa00640 1 --2- 172.21.15.47:0/4196696680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f8c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.456+0000 7f38aaa00640 1 -- 172.21.15.47:0/4196696680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d65bfc210 con 0x561d65f821e0 2024-03-21T11:41:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.456+0000 7f38aa000640 1 --2- 172.21.15.47:0/4196696680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f8c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:41:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.456+0000 7f38aaa00640 1 --2- 172.21.15.47:0/4196696680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f821e0 0x561d65f8c3a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f389c004610 tx=0x7f389c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e225fcc8cf761e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.456+0000 7f388be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:41:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.457+0000 7f388be00640 1 -- 172.21.15.47:0/4196696680 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c005020 con 0x561d65f821e0 2024-03-21T11:41:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.457+0000 7f388be00640 1 -- 172.21.15.47:0/4196696680 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f389c0051c0 con 0x561d65f821e0 2024-03-21T11:41:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.457+0000 7f38b43b4740 1 -- 172.21.15.47:0/4196696680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38a400fd30 con 0x561d65f821e0 2024-03-21T11:41:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.458+0000 7f388be00640 1 -- 172.21.15.47:0/4196696680 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f389c0053c0 con 0x561d65f821e0 2024-03-21T11:41:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.458+0000 7f38b43b4740 10 client.31493 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.458+0000 7f38b43b4740 1 -- 172.21.15.47:0/4196696680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561d65cc8620 con 0x561d65f821e0 2024-03-21T11:41:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.458+0000 7f388be00640 1 -- 172.21.15.47:0/4196696680 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38a400fd30 con 0x561d65f821e0 2024-03-21T11:41:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.458+0000 7f388be00640 1 -- 172.21.15.47:0/4196696680 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f389c027070 con 0x561d65f821e0 2024-03-21T11:41:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.458+0000 7f38b43b4740 10 client.31493 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.458+0000 7f38b43b4740 10 client.31493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.458+0000 7f38b43b4740 20 client.31493 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.458+0000 7f38b43b4740 1 --2- 172.21.15.47:0/4196696680 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d65f981b0 0x561d65f9a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.458+0000 7f38b43b4740 4 client.31493 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:41:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.458+0000 7f38b43b4740 1 -- 172.21.15.47:0/4196696680 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d65ef4d70 con 0x561d65f981b0 2024-03-21T11:41:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.459+0000 7f38aa000640 1 --2- 172.21.15.47:0/4196696680 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d65f981b0 0x561d65f9a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.461+0000 7f38aa000640 1 --2- 172.21.15.47:0/4196696680 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d65f981b0 0x561d65f9a630 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37f3b60f629d4bac server_cookie=4ebe06a10053a4e8 in_seq=0 out_seq=0 2024-03-21T11:41:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.461+0000 7f388be00640 10 client.31493 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f388be00640 1 -- 172.21.15.47:0/4196696680 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561d65ef4d70 con 0x561d65f981b0 2024-03-21T11:41:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f388be00640 10 client.31493 handle_command_reply: tid=0 2024-03-21T11:41:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f38b43b4740 1 client.31493 shutdown 2024-03-21T11:41:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f38b43b4740 1 -- 172.21.15.47:0/4196696680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f821e0 msgr2=0x561d65f8c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f38b43b4740 1 --2- 172.21.15.47:0/4196696680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f821e0 0x561d65f8c3a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f389c004610 tx=0x7f389c004730 comp rx=0 tx=0).stop 2024-03-21T11:41:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f38b43b4740 1 -- 172.21.15.47:0/4196696680 shutdown_connections 2024-03-21T11:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f38b43b4740 1 --2- 172.21.15.47:0/4196696680 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d65f981b0 0x561d65f9a630 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f38b43b4740 1 --2- 172.21.15.47:0/4196696680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f87d40 0x561d65f93920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f38b43b4740 1 --2- 172.21.15.47:0/4196696680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f8c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f38b43b4740 1 --2- 172.21.15.47:0/4196696680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f821e0 0x561d65f8c3a0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f38b43b4740 1 -- 172.21.15.47:0/4196696680 >> 172.21.15.47:0/4196696680 conn(0x561d65f7d750 msgr2=0x561d65f7c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f388be00640 0 client.31493 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f38b43b4740 1 -- 172.21.15.47:0/4196696680 shutdown_connections 2024-03-21T11:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f38b43b4740 1 -- 172.21.15.47:0/4196696680 wait complete. 2024-03-21T11:41:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.467+0000 7f38b43b4740 20 client.31493 trim_cache size 0 max 16384 2024-03-21T11:41:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.486+0000 7f38b43b4740 1 Processor -- start 2024-03-21T11:41:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.486+0000 7f38b43b4740 1 -- start start 2024-03-21T11:41:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.486+0000 7f38b43b4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f8c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.486+0000 7f38b43b4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f938e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.486+0000 7f38b43b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 0x561d65f7c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.486+0000 7f38b43b4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac1b0750 con 0x561d65f82bd0 2024-03-21T11:41:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.486+0000 7f38b43b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac1b20d0 con 0x561d65f87d40 2024-03-21T11:41:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.486+0000 7f38b43b4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac1b0db0 con 0x561d65f821e0 2024-03-21T11:41:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.486+0000 7f38aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f8c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.486+0000 7f38aa000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.487+0000 7f38aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f8c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55424/0 (socket says 172.21.15.47:55424) 2024-03-21T11:41:40.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.487+0000 7f38aaa00640 1 -- 172.21.15.47:0/1577472166 learned_addr learned my addr 172.21.15.47:0/1577472166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.487+0000 7f38ab400640 1 --2- 172.21.15.47:0/1577472166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 0x561d65f7c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.487+0000 7f38aaa00640 1 -- 172.21.15.47:0/1577472166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 msgr2=0x561d65f7c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.487+0000 7f38aaa00640 1 --2- 172.21.15.47:0/1577472166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 0x561d65f7c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.487+0000 7f38aaa00640 1 -- 172.21.15.47:0/1577472166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 msgr2=0x561d65f938e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.487+0000 7f38aaa00640 1 --2- 172.21.15.47:0/1577472166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f938e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.487+0000 7f38aaa00640 1 -- 172.21.15.47:0/1577472166 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d65f825c0 con 0x561d65f821e0 2024-03-21T11:41:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.487+0000 7f38aa000640 1 --2- 172.21.15.47:0/1577472166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f938e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:41:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.487+0000 7f38aaa00640 1 --2- 172.21.15.47:0/1577472166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f8c6c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f389c000c00 tx=0x7f389c003e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cecd3428c91c049 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.488+0000 7f3896a00640 1 -- 172.21.15.47:0/1577472166 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c037030 con 0x561d65f821e0 2024-03-21T11:41:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.488+0000 7f3896a00640 1 -- 172.21.15.47:0/1577472166 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f389c005560 con 0x561d65f821e0 2024-03-21T11:41:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.488+0000 7f3896a00640 1 -- 172.21.15.47:0/1577472166 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c01d400 con 0x561d65f821e0 2024-03-21T11:41:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.488+0000 7f38b43b4740 1 -- 172.21.15.47:0/1577472166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 msgr2=0x561d65f8c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.488+0000 7f38b43b4740 1 --2- 172.21.15.47:0/1577472166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f8c6c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f389c000c00 tx=0x7f389c003e50 comp rx=0 tx=0).stop 2024-03-21T11:41:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.488+0000 7f38b43b4740 1 -- 172.21.15.47:0/1577472166 shutdown_connections 2024-03-21T11:41:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.488+0000 7f38b43b4740 1 --2- 172.21.15.47:0/1577472166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 0x561d65f7c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.488+0000 7f38b43b4740 1 --2- 172.21.15.47:0/1577472166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f938e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.488+0000 7f38b43b4740 1 --2- 172.21.15.47:0/1577472166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f8c6c0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.488+0000 7f38b43b4740 1 -- 172.21.15.47:0/1577472166 >> 172.21.15.47:0/1577472166 conn(0x561d65f7d750 msgr2=0x561d65f8bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.489+0000 7f38b43b4740 1 -- 172.21.15.47:0/1577472166 shutdown_connections 2024-03-21T11:41:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.489+0000 7f38b43b4740 1 -- 172.21.15.47:0/1577472166 wait complete. 2024-03-21T11:41:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.489+0000 7f38b43b4740 1 -- start start 2024-03-21T11:41:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.489+0000 7f38b43b4740 1 Processor -- start 2024-03-21T11:41:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.489+0000 7f38b43b4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f91790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.490+0000 7f38b43b4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f91cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.490+0000 7f38b43b4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 0x561d65f92bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.490+0000 7f38b43b4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac1b1a70 con 0x561d65f82bd0 2024-03-21T11:41:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.490+0000 7f38b43b4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac1b1410 con 0x561d65f87d40 2024-03-21T11:41:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.490+0000 7f38b43b4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac1ae910 con 0x561d65f821e0 2024-03-21T11:41:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.490+0000 7f38b43b4740 10 client.0 _ll_register_callbacks cb 0x7f389c009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.490+0000 7f38aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f91790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.490+0000 7f38aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f91790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55430/0 (socket says 172.21.15.47:55430) 2024-03-21T11:41:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.490+0000 7f38aaa00640 1 -- 172.21.15.47:0/1954311294 learned_addr learned my addr 172.21.15.47:0/1954311294 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.490+0000 7f38ab400640 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 0x561d65f92bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.490+0000 7f38aa000640 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f91cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.491+0000 7f38ab400640 1 -- 172.21.15.47:0/1954311294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 msgr2=0x561d65f91790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.491+0000 7f38ab400640 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f91790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.491+0000 7f38ab400640 1 -- 172.21.15.47:0/1954311294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 msgr2=0x561d65f91cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.491+0000 7f38ab400640 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f91cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.491+0000 7f38ab400640 1 -- 172.21.15.47:0/1954311294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d65f931c0 con 0x561d65f87d40 2024-03-21T11:41:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.491+0000 7f38aaa00640 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f91790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:41:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.491+0000 7f38aa000640 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f91cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:41:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.491+0000 7f38ab400640 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 0x561d65f92bd0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f38a4004a80 tx=0x7f38a400fe30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f98c1e74d9eedd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.491+0000 7f388be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:41:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.492+0000 7f388be00640 1 -- 172.21.15.47:0/1954311294 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4029020 con 0x561d65f87d40 2024-03-21T11:41:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.492+0000 7f388be00640 1 -- 172.21.15.47:0/1954311294 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f38a4019650 con 0x561d65f87d40 2024-03-21T11:41:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.492+0000 7f38b43b4740 1 -- 172.21.15.47:0/1954311294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f389c005560 con 0x561d65f87d40 2024-03-21T11:41:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.493+0000 7f388be00640 1 -- 172.21.15.47:0/1954311294 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38a4014040 con 0x561d65f87d40 2024-03-21T11:41:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.493+0000 7f38b43b4740 10 client.31499 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.493+0000 7f38b43b4740 1 -- 172.21.15.47:0/1954311294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561d65ff4100 con 0x561d65f87d40 2024-03-21T11:41:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.493+0000 7f388be00640 1 -- 172.21.15.47:0/1954311294 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f389c005560 con 0x561d65f87d40 2024-03-21T11:41:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.494+0000 7f388be00640 1 -- 172.21.15.47:0/1954311294 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f38a4022070 con 0x561d65f87d40 2024-03-21T11:41:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.494+0000 7f38b43b4740 10 client.31499 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.494+0000 7f38b43b4740 10 client.31499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.494+0000 7f38b43b4740 20 client.31499 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.494+0000 7f38b43b4740 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d65ff4820 0x561d65ff6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.494+0000 7f38b43b4740 4 client.31499 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:41:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.494+0000 7f38b43b4740 1 -- 172.21.15.47:0/1954311294 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561d65eef0c0 con 0x561d65ff4820 2024-03-21T11:41:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.494+0000 7f38aaa00640 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d65ff4820 0x561d65ff6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.495+0000 7f38aaa00640 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d65ff4820 0x561d65ff6c40 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e49806abe2f5fe server_cookie=4e646cec8cc5d8df in_seq=0 out_seq=0 2024-03-21T11:41:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:40.495+0000 7f388be00640 10 client.31499 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:40.951 INFO:tasks.workunit.client.0.smithi047.stdout: 10 122185 23.89 MB/sec execute 144 sec latency 438.640 ms 2024-03-21T11:41:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:40 smithi102 ceph-mon[36748]: pgmap v950: 129 pgs: 129 active+clean; 481 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.8 MiB/s wr, 736 op/s 2024-03-21T11:41:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.261+0000 7f388be00640 1 -- 172.21.15.47:0/1954311294 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561d65eef0c0 con 0x561d65ff4820 2024-03-21T11:41:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.261+0000 7f388be00640 10 client.31499 handle_command_reply: tid=0 2024-03-21T11:41:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.261+0000 7f38b43b4740 1 client.31499 shutdown 2024-03-21T11:41:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.261+0000 7f38b43b4740 1 -- 172.21.15.47:0/1954311294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 msgr2=0x561d65f92bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.261+0000 7f38b43b4740 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 0x561d65f92bd0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f38a4004a80 tx=0x7f38a400fe30 comp rx=0 tx=0).stop 2024-03-21T11:41:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.261+0000 7f38b43b4740 1 -- 172.21.15.47:0/1954311294 shutdown_connections 2024-03-21T11:41:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.261+0000 7f38b43b4740 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d65ff4820 0x561d65ff6c40 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.261+0000 7f38b43b4740 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d65f87d40 0x561d65f92bd0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.262+0000 7f38b43b4740 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d65f82bd0 0x561d65f91cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.262+0000 7f38b43b4740 1 --2- 172.21.15.47:0/1954311294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d65f821e0 0x561d65f91790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.262+0000 7f388be00640 0 client.31499 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.262+0000 7f38b43b4740 1 -- 172.21.15.47:0/1954311294 >> 172.21.15.47:0/1954311294 conn(0x561d65f7d750 msgr2=0x561d65f8c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.262+0000 7f38b43b4740 1 -- 172.21.15.47:0/1954311294 shutdown_connections 2024-03-21T11:41:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.262+0000 7f38b43b4740 1 -- 172.21.15.47:0/1954311294 wait complete. 2024-03-21T11:41:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.263+0000 7f38b43b4740 20 client.31499 trim_cache size 0 max 16384 2024-03-21T11:41:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:41:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:41:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:41:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:41:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.264+0000 7f38b1600640 1 -- 172.21.15.47:0/2291415051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f38900a3040 msgr2=0x7f38900a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.264+0000 7f38b1600640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f38900a3040 0x7f38900a5500 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x561d65f7e6c0 tx=0x7f389c005f90 comp rx=0 tx=0).stop 2024-03-21T11:41:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.264+0000 7f38b1600640 1 -- 172.21.15.47:0/2291415051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38ac104a90 msgr2=0x7f38ac197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.264+0000 7f38b1600640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38ac104a90 0x7f38ac197f30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3898000c00 tx=0x7f3898005d50 comp rx=0 tx=0).stop 2024-03-21T11:41:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.264+0000 7f38b1600640 1 -- 172.21.15.47:0/2291415051 shutdown_connections 2024-03-21T11:41:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.264+0000 7f38b1600640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f38900a3040 0x7f38900a5500 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.264+0000 7f38b1600640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38ac1073c0 0x7f38ac1a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.264+0000 7f38b1600640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38ac104a90 0x7f38ac197f30 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.264+0000 7f38b1600640 1 --2- 172.21.15.47:0/2291415051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38ac102160 0x7f38ac1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.264+0000 7f38b1600640 1 -- 172.21.15.47:0/2291415051 >> 172.21.15.47:0/2291415051 conn(0x7f38ac0fbb50 msgr2=0x7f38ac102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.264+0000 7f38b1600640 1 -- 172.21.15.47:0/2291415051 shutdown_connections 2024-03-21T11:41:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:41.264+0000 7f38b1600640 1 -- 172.21.15.47:0/2291415051 wait complete. 2024-03-21T11:41:41.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:40 smithi047 ceph-mon[31351]: pgmap v950: 129 pgs: 129 active+clean; 481 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.8 MiB/s wr, 736 op/s 2024-03-21T11:41:41.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:40 smithi100 ceph-mon[36245]: pgmap v950: 129 pgs: 129 active+clean; 481 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.8 MiB/s wr, 736 op/s 2024-03-21T11:41:41.951 INFO:tasks.workunit.client.0.smithi047.stdout: 10 122480 23.80 MB/sec execute 145 sec latency 349.200 ms 2024-03-21T11:41:42.951 INFO:tasks.workunit.client.0.smithi047.stdout: 10 123126 23.87 MB/sec execute 146 sec latency 41.267 ms 2024-03-21T11:41:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:42 smithi102 ceph-mon[36748]: pgmap v951: 129 pgs: 129 active+clean; 487 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 519 op/s 2024-03-21T11:41:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:42 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:41:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:42 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:41:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:42 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:41:43.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:42 smithi047 ceph-mon[31351]: pgmap v951: 129 pgs: 129 active+clean; 487 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 519 op/s 2024-03-21T11:41:43.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:42 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:41:43.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:42 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:41:43.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:42 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:41:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:42 smithi100 ceph-mon[36245]: pgmap v951: 129 pgs: 129 active+clean; 487 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 519 op/s 2024-03-21T11:41:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:42 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:41:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:42 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:41:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:42 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:41:43.951 INFO:tasks.workunit.client.0.smithi047.stdout: 10 123892 23.96 MB/sec execute 147 sec latency 48.190 ms 2024-03-21T11:41:44.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:43 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:41:44.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:43 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:41:44.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:43 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:41:44.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:43 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:41:44.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:43 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:41:44.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:43 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:41:44.951 INFO:tasks.workunit.client.0.smithi047.stdout: 10 124256 24.00 MB/sec execute 148 sec latency 259.881 ms 2024-03-21T11:41:45.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:44 smithi102 ceph-mon[36748]: pgmap v952: 129 pgs: 129 active+clean; 489 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 511 op/s 2024-03-21T11:41:45.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:44 smithi047 ceph-mon[31351]: pgmap v952: 129 pgs: 129 active+clean; 489 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 511 op/s 2024-03-21T11:41:45.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:44 smithi100 ceph-mon[36245]: pgmap v952: 129 pgs: 129 active+clean; 489 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 511 op/s 2024-03-21T11:41:45.951 INFO:tasks.workunit.client.0.smithi047.stdout: 10 124648 24.01 MB/sec execute 149 sec latency 153.547 ms 2024-03-21T11:41:46.951 INFO:tasks.workunit.client.0.smithi047.stdout: 10 125005 24.00 MB/sec execute 150 sec latency 102.448 ms 2024-03-21T11:41:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:46 smithi102 ceph-mon[36748]: pgmap v953: 129 pgs: 129 active+clean; 487 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.2 MiB/s wr, 682 op/s 2024-03-21T11:41:47.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:46 smithi047 ceph-mon[31351]: pgmap v953: 129 pgs: 129 active+clean; 487 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.2 MiB/s wr, 682 op/s 2024-03-21T11:41:47.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:46 smithi100 ceph-mon[36245]: pgmap v953: 129 pgs: 129 active+clean; 487 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.2 MiB/s wr, 682 op/s 2024-03-21T11:41:47.951 INFO:tasks.workunit.client.0.smithi047.stdout: 10 125400 23.92 MB/sec execute 151 sec latency 97.366 ms 2024-03-21T11:41:48.951 INFO:tasks.workunit.client.0.smithi047.stdout: 10 125774 23.89 MB/sec execute 152 sec latency 98.337 ms 2024-03-21T11:41:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:48 smithi102 ceph-mon[36748]: pgmap v954: 129 pgs: 129 active+clean; 481 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 567 op/s 2024-03-21T11:41:49.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:48 smithi047 ceph-mon[31351]: pgmap v954: 129 pgs: 129 active+clean; 481 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 567 op/s 2024-03-21T11:41:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:48 smithi100 ceph-mon[36245]: pgmap v954: 129 pgs: 129 active+clean; 481 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 567 op/s 2024-03-21T11:41:49.951 INFO:tasks.workunit.client.0.smithi047.stdout: 10 126089 23.85 MB/sec execute 153 sec latency 256.088 ms 2024-03-21T11:41:50.951 INFO:tasks.workunit.client.0.smithi047.stdout: 10 126529 23.83 MB/sec execute 154 sec latency 210.364 ms 2024-03-21T11:41:51.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:50 smithi047 ceph-mon[31351]: pgmap v955: 129 pgs: 129 active+clean; 484 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.9 MiB/s wr, 582 op/s 2024-03-21T11:41:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:50 smithi100 ceph-mon[36245]: pgmap v955: 129 pgs: 129 active+clean; 484 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.9 MiB/s wr, 582 op/s 2024-03-21T11:41:51.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:50 smithi102 ceph-mon[36748]: pgmap v955: 129 pgs: 129 active+clean; 484 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.9 MiB/s wr, 582 op/s 2024-03-21T11:41:51.952 INFO:tasks.workunit.client.0.smithi047.stdout: 10 126931 23.78 MB/sec execute 155 sec latency 210.523 ms 2024-03-21T11:41:52.952 INFO:tasks.workunit.client.0.smithi047.stdout: 10 127555 23.89 MB/sec execute 156 sec latency 42.028 ms 2024-03-21T11:41:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:52 smithi047 ceph-mon[31351]: pgmap v956: 129 pgs: 129 active+clean; 490 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 574 op/s 2024-03-21T11:41:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:53 smithi100 ceph-mon[36245]: pgmap v956: 129 pgs: 129 active+clean; 490 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 574 op/s 2024-03-21T11:41:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:52 smithi102 ceph-mon[36748]: pgmap v956: 129 pgs: 129 active+clean; 490 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 574 op/s 2024-03-21T11:41:53.952 INFO:tasks.workunit.client.0.smithi047.stdout: 10 128084 23.94 MB/sec execute 157 sec latency 85.346 ms 2024-03-21T11:41:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.419+0000 7f498ba00640 1 -- 172.21.15.47:0/268218564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4984103900 msgr2=0x7f4984103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.419+0000 7f498ba00640 1 --2- 172.21.15.47:0/268218564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4984103900 0x7f4984103d20 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4974009900 tx=0x7f497402bc30 comp rx=0 tx=0).stop 2024-03-21T11:41:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.420+0000 7f498ba00640 1 -- 172.21.15.47:0/268218564 shutdown_connections 2024-03-21T11:41:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.420+0000 7f498ba00640 1 --2- 172.21.15.47:0/268218564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49841054b0 0x7f498410bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.420+0000 7f498ba00640 1 --2- 172.21.15.47:0/268218564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4984104af0 0x7f4984104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.420+0000 7f498ba00640 1 --2- 172.21.15.47:0/268218564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4984103900 0x7f4984103d20 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.420+0000 7f498ba00640 1 -- 172.21.15.47:0/268218564 >> 172.21.15.47:0/268218564 conn(0x7f49840fee90 msgr2=0x7f49841012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.420+0000 7f498ba00640 1 -- 172.21.15.47:0/268218564 shutdown_connections 2024-03-21T11:41:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.420+0000 7f498ba00640 1 -- 172.21.15.47:0/268218564 wait complete. 2024-03-21T11:41:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.421+0000 7f498ba00640 1 Processor -- start 2024-03-21T11:41:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.421+0000 7f498ba00640 1 -- start start 2024-03-21T11:41:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.421+0000 7f498ba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4984103900 0x7f4984199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.421+0000 7f498ba00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4984104af0 0x7f498419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.421+0000 7f498ba00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49841054b0 0x7f49841a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.421+0000 7f498ba00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498410f580 con 0x7f4984104af0 2024-03-21T11:41:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.421+0000 7f498ba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498410f400 con 0x7f4984103900 2024-03-21T11:41:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.421+0000 7f498ba00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498410f700 con 0x7f49841054b0 2024-03-21T11:41:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.422+0000 7f4989000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4984103900 0x7f4984199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.422+0000 7f4989000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4984103900 0x7f4984199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36478/0 (socket says 172.21.15.47:36478) 2024-03-21T11:41:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.422+0000 7f4989000640 1 -- 172.21.15.47:0/1437657068 learned_addr learned my addr 172.21.15.47:0/1437657068 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.422+0000 7f497be00640 1 --2- 172.21.15.47:0/1437657068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4984104af0 0x7f498419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.422+0000 7f4989a00640 1 --2- 172.21.15.47:0/1437657068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49841054b0 0x7f49841a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.422+0000 7f4989000640 1 -- 172.21.15.47:0/1437657068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49841054b0 msgr2=0x7f49841a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.422+0000 7f4989000640 1 --2- 172.21.15.47:0/1437657068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49841054b0 0x7f49841a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.422+0000 7f4989000640 1 -- 172.21.15.47:0/1437657068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4984104af0 msgr2=0x7f498419a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.422+0000 7f4989000640 1 --2- 172.21.15.47:0/1437657068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4984104af0 0x7f498419a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.422+0000 7f4989000640 1 -- 172.21.15.47:0/1437657068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49841a1730 con 0x7f4984103900 2024-03-21T11:41:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.423+0000 7f4989000640 1 --2- 172.21.15.47:0/1437657068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4984103900 0x7f4984199b10 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4974000c00 tx=0x7f497402c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.423+0000 7f4982000640 1 -- 172.21.15.47:0/1437657068 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974041070 con 0x7f4984103900 2024-03-21T11:41:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.423+0000 7f498ba00640 1 -- 172.21.15.47:0/1437657068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49841a19c0 con 0x7f4984103900 2024-03-21T11:41:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.423+0000 7f498ba00640 1 -- 172.21.15.47:0/1437657068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49841a1f00 con 0x7f4984103900 2024-03-21T11:41:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.423+0000 7f4982000640 1 -- 172.21.15.47:0/1437657068 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4974004450 con 0x7f4984103900 2024-03-21T11:41:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.423+0000 7f4982000640 1 -- 172.21.15.47:0/1437657068 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497404b060 con 0x7f4984103900 2024-03-21T11:41:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.427+0000 7f498e7c6740 1 Processor -- start 2024-03-21T11:41:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.427+0000 7f4982000640 1 -- 172.21.15.47:0/1437657068 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4974044070 con 0x7f4984103900 2024-03-21T11:41:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.427+0000 7f498e7c6740 1 -- start start 2024-03-21T11:41:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.427+0000 7f498e7c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae671e0 0x55ff7ae67600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.427+0000 7f498e7c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae6c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.427+0000 7f498e7c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae6f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.427+0000 7f498e7c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7ae2a3b0 con 0x55ff7ae67bd0 2024-03-21T11:41:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.427+0000 7f498e7c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7adcc830 con 0x55ff7ae671e0 2024-03-21T11:41:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.427+0000 7f498e7c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7ad755d0 con 0x55ff7ae6cd40 2024-03-21T11:41:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.428+0000 7f4989a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae671e0 0x55ff7ae67600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.428+0000 7f497be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae6f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.428+0000 7f4989a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae671e0 0x55ff7ae67600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36486/0 (socket says 172.21.15.47:36486) 2024-03-21T11:41:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.428+0000 7f4989a00640 1 -- 172.21.15.47:0/904215032 learned_addr learned my addr 172.21.15.47:0/904215032 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.428+0000 7f497be00640 1 -- 172.21.15.47:0/904215032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae671e0 msgr2=0x55ff7ae67600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.428+0000 7f497be00640 1 --2- 172.21.15.47:0/904215032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae671e0 0x55ff7ae67600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.428+0000 7f497be00640 1 -- 172.21.15.47:0/904215032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 msgr2=0x55ff7ae6c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.429+0000 7f4989000640 1 --2- 172.21.15.47:0/904215032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae6c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.429+0000 7f497be00640 1 --2- 172.21.15.47:0/904215032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae6c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.429+0000 7f497be00640 1 -- 172.21.15.47:0/904215032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff7ad8d2c0 con 0x55ff7ae6cd40 2024-03-21T11:41:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.429+0000 7f4989a00640 1 --2- 172.21.15.47:0/904215032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae671e0 0x55ff7ae67600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:41:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.429+0000 7f4989000640 1 --2- 172.21.15.47:0/904215032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae6c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:41:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.429+0000 7f497be00640 1 --2- 172.21.15.47:0/904215032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae6f1d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f496c009bf0 tx=0x7f496c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfe4e1a717a4c2f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.429+0000 7f4982000640 1 --2- 172.21.15.47:0/1437657068 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f49680a3040 0x7f49680a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.429+0000 7f4982000640 1 -- 172.21.15.47:0/1437657068 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f49740f0d60 con 0x7f4984103900 2024-03-21T11:41:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.429+0000 7f497a000640 1 -- 172.21.15.47:0/904215032 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c009da0 con 0x55ff7ae6cd40 2024-03-21T11:41:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.430+0000 7f497a000640 1 -- 172.21.15.47:0/904215032 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f496c005020 con 0x55ff7ae6cd40 2024-03-21T11:41:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.430+0000 7f497a000640 1 -- 172.21.15.47:0/904215032 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c004bc0 con 0x55ff7ae6cd40 2024-03-21T11:41:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.430+0000 7f4989a00640 1 --2- 172.21.15.47:0/1437657068 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f49680a3040 0x7f49680a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.430+0000 7f498e7c6740 1 -- 172.21.15.47:0/904215032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae6cd40 msgr2=0x55ff7ae6f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.430+0000 7f498e7c6740 1 --2- 172.21.15.47:0/904215032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae6f1d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f496c009bf0 tx=0x7f496c011810 comp rx=0 tx=0).stop 2024-03-21T11:41:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.431+0000 7f498e7c6740 1 -- 172.21.15.47:0/904215032 shutdown_connections 2024-03-21T11:41:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.431+0000 7f498e7c6740 1 --2- 172.21.15.47:0/904215032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae6f1d0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.431+0000 7f498e7c6740 1 --2- 172.21.15.47:0/904215032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae6c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.431+0000 7f498e7c6740 1 --2- 172.21.15.47:0/904215032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae671e0 0x55ff7ae67600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.431+0000 7f4989a00640 1 --2- 172.21.15.47:0/1437657068 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f49680a3040 0x7f49680a5500 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x55ff7ae68430 tx=0x7f497c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.431+0000 7f498e7c6740 1 -- 172.21.15.47:0/904215032 >> 172.21.15.47:0/904215032 conn(0x55ff7ae62750 msgr2=0x55ff7ae62b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.431+0000 7f498e7c6740 1 -- 172.21.15.47:0/904215032 shutdown_connections 2024-03-21T11:41:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.431+0000 7f498e7c6740 1 -- 172.21.15.47:0/904215032 wait complete. 2024-03-21T11:41:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.431+0000 7f498e7c6740 1 -- start start 2024-03-21T11:41:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.432+0000 7f498e7c6740 1 Processor -- start 2024-03-21T11:41:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.432+0000 7f498e7c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae671e0 0x55ff7ae71370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.432+0000 7f498e7c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.432+0000 7f498e7c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae788f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.432+0000 7f498e7c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7ad88a20 con 0x55ff7ae67bd0 2024-03-21T11:41:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.432+0000 7f498e7c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7adc88d0 con 0x55ff7ae6cd40 2024-03-21T11:41:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.432+0000 7f498e7c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff7ae6fcf0 con 0x55ff7ae671e0 2024-03-21T11:41:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.432+0000 7f498e7c6740 10 client.0 _ll_register_callbacks cb 0x55ff7adc51c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.432+0000 7f4989000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae671e0 0x55ff7ae71370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.433+0000 7f4989000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae671e0 0x55ff7ae71370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38814/0 (socket says 172.21.15.47:38814) 2024-03-21T11:41:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.433+0000 7f4989000640 1 -- 172.21.15.47:0/2740819937 learned_addr learned my addr 172.21.15.47:0/2740819937 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.433+0000 7f4989a00640 1 --2- 172.21.15.47:0/2740819937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae788f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.433+0000 7f497be00640 1 --2- 172.21.15.47:0/2740819937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.433+0000 7f4989000640 1 -- 172.21.15.47:0/2740819937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae6cd40 msgr2=0x55ff7ae788f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.433+0000 7f4989000640 1 --2- 172.21.15.47:0/2740819937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae788f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.433+0000 7f4989000640 1 -- 172.21.15.47:0/2740819937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 msgr2=0x55ff7ae718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.433+0000 7f4989000640 1 --2- 172.21.15.47:0/2740819937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae718b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.433+0000 7f4989000640 1 -- 172.21.15.47:0/2740819937 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff7aae1210 con 0x55ff7ae671e0 2024-03-21T11:41:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.433+0000 7f4989000640 1 --2- 172.21.15.47:0/2740819937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae671e0 0x55ff7ae71370 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f497402cfd0 tx=0x7f49740bce40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1962d36868bff166 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.433+0000 7f4972a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:41:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.434+0000 7f4972a00640 1 -- 172.21.15.47:0/2740819937 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49740c5b10 con 0x55ff7ae671e0 2024-03-21T11:41:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.434+0000 7f498e7c6740 1 -- 172.21.15.47:0/2740819937 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f496c005020 con 0x55ff7ae671e0 2024-03-21T11:41:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.434+0000 7f4972a00640 1 -- 172.21.15.47:0/2740819937 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f49740dd0b0 con 0x55ff7ae671e0 2024-03-21T11:41:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.434+0000 7f4972a00640 1 -- 172.21.15.47:0/2740819937 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49740dd2b0 con 0x55ff7ae671e0 2024-03-21T11:41:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.434+0000 7f498e7c6740 10 client.31489 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.434+0000 7f498e7c6740 1 -- 172.21.15.47:0/2740819937 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ff7abad620 con 0x55ff7ae671e0 2024-03-21T11:41:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.434+0000 7f4972a00640 1 -- 172.21.15.47:0/2740819937 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f496c005020 con 0x55ff7ae671e0 2024-03-21T11:41:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.435+0000 7f4972a00640 1 -- 172.21.15.47:0/2740819937 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f49740c5680 con 0x55ff7ae671e0 2024-03-21T11:41:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.435+0000 7f498e7c6740 10 client.31489 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.435+0000 7f498e7c6740 10 client.31489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.435+0000 7f498e7c6740 20 client.31489 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.435+0000 7f498e7c6740 1 --2- 172.21.15.47:0/2740819937 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff7ae7d140 0x55ff7ae7f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.435+0000 7f498e7c6740 4 client.31489 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:41:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.435+0000 7f498e7c6740 1 -- 172.21.15.47:0/2740819937 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff7add9d70 con 0x55ff7ae7d140 2024-03-21T11:41:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.435+0000 7f497be00640 1 --2- 172.21.15.47:0/2740819937 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff7ae7d140 0x55ff7ae7f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.436+0000 7f497be00640 1 --2- 172.21.15.47:0/2740819937 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff7ae7d140 0x55ff7ae7f5e0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ac195e0abac607 server_cookie=7609e349ae2c52b7 in_seq=0 out_seq=0 2024-03-21T11:41:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.436+0000 7f4972a00640 10 client.31489 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.439+0000 7f4972a00640 1 -- 172.21.15.47:0/2740819937 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ff7add9d70 con 0x55ff7ae7d140 2024-03-21T11:41:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.439+0000 7f4972a00640 10 client.31489 handle_command_reply: tid=0 2024-03-21T11:41:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.439+0000 7f498e7c6740 1 client.31489 shutdown 2024-03-21T11:41:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.439+0000 7f498e7c6740 1 -- 172.21.15.47:0/2740819937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae671e0 msgr2=0x55ff7ae71370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.439+0000 7f498e7c6740 1 --2- 172.21.15.47:0/2740819937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae671e0 0x55ff7ae71370 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f497402cfd0 tx=0x7f49740bce40 comp rx=0 tx=0).stop 2024-03-21T11:41:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.439+0000 7f498e7c6740 1 -- 172.21.15.47:0/2740819937 shutdown_connections 2024-03-21T11:41:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.439+0000 7f498e7c6740 1 --2- 172.21.15.47:0/2740819937 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff7ae7d140 0x55ff7ae7f5e0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.440+0000 7f498e7c6740 1 --2- 172.21.15.47:0/2740819937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae788f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.440+0000 7f498e7c6740 1 --2- 172.21.15.47:0/2740819937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.440+0000 7f498e7c6740 1 --2- 172.21.15.47:0/2740819937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae671e0 0x55ff7ae71370 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.440+0000 7f4972a00640 0 client.31489 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.440+0000 7f498e7c6740 1 -- 172.21.15.47:0/2740819937 >> 172.21.15.47:0/2740819937 conn(0x55ff7ae62750 msgr2=0x55ff7ae61950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.440+0000 7f498e7c6740 1 -- 172.21.15.47:0/2740819937 shutdown_connections 2024-03-21T11:41:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.440+0000 7f498e7c6740 1 -- 172.21.15.47:0/2740819937 wait complete. 2024-03-21T11:41:54.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.440+0000 7f498e7c6740 20 client.31489 trim_cache size 0 max 16384 2024-03-21T11:41:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.459+0000 7f498e7c6740 1 Processor -- start 2024-03-21T11:41:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.459+0000 7f498e7c6740 1 -- start start 2024-03-21T11:41:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.459+0000 7f498e7c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae671e0 0x55ff7ae659b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.459+0000 7f498e7c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.459+0000 7f498e7c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae65ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.459+0000 7f498e7c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496000f9a0 con 0x55ff7ae67bd0 2024-03-21T11:41:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.459+0000 7f498e7c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4960011320 con 0x55ff7ae6cd40 2024-03-21T11:41:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.459+0000 7f498e7c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4960010000 con 0x55ff7ae671e0 2024-03-21T11:41:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.459+0000 7f4989000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae671e0 0x55ff7ae659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.459+0000 7f4989000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae671e0 0x55ff7ae659b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:38818/0 (socket says 172.21.15.47:38818) 2024-03-21T11:41:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.459+0000 7f4989000640 1 -- 172.21.15.47:0/566500019 learned_addr learned my addr 172.21.15.47:0/566500019 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.459+0000 7f497be00640 1 --2- 172.21.15.47:0/566500019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.460+0000 7f4989000640 1 -- 172.21.15.47:0/566500019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae6cd40 msgr2=0x55ff7ae65ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.460+0000 7f4989000640 1 --2- 172.21.15.47:0/566500019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae65ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.460+0000 7f4989000640 1 -- 172.21.15.47:0/566500019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 msgr2=0x55ff7ae788b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.460+0000 7f4989000640 1 --2- 172.21.15.47:0/566500019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae788b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.460+0000 7f4989000640 1 -- 172.21.15.47:0/566500019 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff7ae620a0 con 0x55ff7ae671e0 2024-03-21T11:41:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.460+0000 7f4989000640 1 --2- 172.21.15.47:0/566500019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae671e0 0x55ff7ae659b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f49740bd1d0 tx=0x7f49740bceb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3418e455cc47c3f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.460+0000 7f4979600640 1 -- 172.21.15.47:0/566500019 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49740c5b10 con 0x55ff7ae671e0 2024-03-21T11:41:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.460+0000 7f4979600640 1 -- 172.21.15.47:0/566500019 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49740dd0c0 con 0x55ff7ae671e0 2024-03-21T11:41:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.460+0000 7f4979600640 1 -- 172.21.15.47:0/566500019 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49740bd730 con 0x55ff7ae671e0 2024-03-21T11:41:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.461+0000 7f498e7c6740 1 -- 172.21.15.47:0/566500019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae671e0 msgr2=0x55ff7ae659b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.461+0000 7f498e7c6740 1 --2- 172.21.15.47:0/566500019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae671e0 0x55ff7ae659b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f49740bd1d0 tx=0x7f49740bceb0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.462+0000 7f498e7c6740 1 -- 172.21.15.47:0/566500019 shutdown_connections 2024-03-21T11:41:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.462+0000 7f498e7c6740 1 --2- 172.21.15.47:0/566500019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae65ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.462+0000 7f498e7c6740 1 --2- 172.21.15.47:0/566500019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae788b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.462+0000 7f498e7c6740 1 --2- 172.21.15.47:0/566500019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae671e0 0x55ff7ae659b0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.462+0000 7f498e7c6740 1 -- 172.21.15.47:0/566500019 >> 172.21.15.47:0/566500019 conn(0x55ff7ae62750 msgr2=0x55ff7ae68e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.462+0000 7f498e7c6740 1 -- 172.21.15.47:0/566500019 shutdown_connections 2024-03-21T11:41:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.462+0000 7f498e7c6740 1 -- 172.21.15.47:0/566500019 wait complete. 2024-03-21T11:41:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.462+0000 7f498e7c6740 1 -- start start 2024-03-21T11:41:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.463+0000 7f498e7c6740 1 Processor -- start 2024-03-21T11:41:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.463+0000 7f498e7c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae671e0 0x55ff7ae76070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.463+0000 7f498e7c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae765b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.463+0000 7f498e7c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae77500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.463+0000 7f498e7c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4960010cc0 con 0x55ff7ae671e0 2024-03-21T11:41:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.463+0000 7f498e7c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4960010660 con 0x55ff7ae67bd0 2024-03-21T11:41:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.463+0000 7f498e7c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496000db60 con 0x55ff7ae6cd40 2024-03-21T11:41:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.463+0000 7f498e7c6740 10 client.0 _ll_register_callbacks cb 0x7f49740bd010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:41:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.463+0000 7f4989000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae671e0 0x55ff7ae76070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.463+0000 7f4989000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae671e0 0x55ff7ae76070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55786/0 (socket says 172.21.15.47:55786) 2024-03-21T11:41:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.463+0000 7f4989000640 1 -- 172.21.15.47:0/4013167634 learned_addr learned my addr 172.21.15.47:0/4013167634 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:41:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.464+0000 7f497be00640 1 --2- 172.21.15.47:0/4013167634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.464+0000 7f4989a00640 1 --2- 172.21.15.47:0/4013167634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae77500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.464+0000 7f4989000640 1 -- 172.21.15.47:0/4013167634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae6cd40 msgr2=0x55ff7ae77500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.464+0000 7f4989000640 1 --2- 172.21.15.47:0/4013167634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae77500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.464+0000 7f4989000640 1 -- 172.21.15.47:0/4013167634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae67bd0 msgr2=0x55ff7ae765b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:41:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.464+0000 7f4989000640 1 --2- 172.21.15.47:0/4013167634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae765b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.464+0000 7f4989000640 1 -- 172.21.15.47:0/4013167634 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff7ae77af0 con 0x55ff7ae671e0 2024-03-21T11:41:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.464+0000 7f4989000640 1 --2- 172.21.15.47:0/4013167634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae671e0 0x55ff7ae76070 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f49740c55e0 tx=0x7f49740092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6866906caafdae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:41:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.464+0000 7f4972a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:41:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.465+0000 7f4972a00640 1 -- 172.21.15.47:0/4013167634 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49740c5b10 con 0x55ff7ae671e0 2024-03-21T11:41:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.465+0000 7f4972a00640 1 -- 172.21.15.47:0/4013167634 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f49740bc730 con 0x55ff7ae671e0 2024-03-21T11:41:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.465+0000 7f498e7c6740 1 -- 172.21.15.47:0/4013167634 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49740dd0c0 con 0x55ff7ae671e0 2024-03-21T11:41:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.465+0000 7f4972a00640 1 -- 172.21.15.47:0/4013167634 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49740bc930 con 0x55ff7ae671e0 2024-03-21T11:41:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.465+0000 7f498e7c6740 10 client.20958 fetch_fsmap learned FSMap version 40 2024-03-21T11:41:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.465+0000 7f498e7c6740 1 -- 172.21.15.47:0/4013167634 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ff7ae77af0 con 0x55ff7ae671e0 2024-03-21T11:41:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.465+0000 7f4972a00640 1 -- 172.21.15.47:0/4013167634 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49740dd0c0 con 0x55ff7ae671e0 2024-03-21T11:41:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.466+0000 7f4972a00640 1 -- 172.21.15.47:0/4013167634 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f49740ee990 con 0x55ff7ae671e0 2024-03-21T11:41:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.466+0000 7f498e7c6740 10 client.20958 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:41:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.466+0000 7f498e7c6740 10 client.20958 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:41:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.466+0000 7f498e7c6740 20 client.20958 populate_metadata read hostname 'smithi047' 2024-03-21T11:41:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.466+0000 7f498e7c6740 1 --2- 172.21.15.47:0/4013167634 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff7aee1790 0x55ff7aee3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:41:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.466+0000 7f498e7c6740 4 client.20958 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:41:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.466+0000 7f498e7c6740 1 -- 172.21.15.47:0/4013167634 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ff7add40c0 con 0x55ff7aee1790 2024-03-21T11:41:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.466+0000 7f497be00640 1 --2- 172.21.15.47:0/4013167634 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff7aee1790 0x55ff7aee3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:41:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.467+0000 7f497be00640 1 --2- 172.21.15.47:0/4013167634 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff7aee1790 0x55ff7aee3bb0 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6371649e45459deb server_cookie=5508b61a89b590fb in_seq=0 out_seq=0 2024-03-21T11:41:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:54.467+0000 7f4972a00640 10 client.20958 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:54.952 INFO:tasks.workunit.client.0.smithi047.stdout: 10 128396 23.90 MB/sec execute 158 sec latency 396.643 ms 2024-03-21T11:41:55.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:55 smithi047 ceph-mon[31351]: pgmap v957: 129 pgs: 129 active+clean; 493 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.7 MiB/s wr, 555 op/s 2024-03-21T11:41:55.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:55 smithi100 ceph-mon[36245]: pgmap v957: 129 pgs: 129 active+clean; 493 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.7 MiB/s wr, 555 op/s 2024-03-21T11:41:55.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:55 smithi102 ceph-mon[36748]: pgmap v957: 129 pgs: 129 active+clean; 493 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.7 MiB/s wr, 555 op/s 2024-03-21T11:41:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.575+0000 7f4972a00640 1 -- 172.21.15.47:0/4013167634 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ff7add40c0 con 0x55ff7aee1790 2024-03-21T11:41:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.575+0000 7f4972a00640 10 client.20958 handle_command_reply: tid=0 2024-03-21T11:41:55.576 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.575+0000 7f498e7c6740 1 client.20958 shutdown 2024-03-21T11:41:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.576+0000 7f498e7c6740 1 -- 172.21.15.47:0/4013167634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae671e0 msgr2=0x55ff7ae76070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.576+0000 7f498e7c6740 1 --2- 172.21.15.47:0/4013167634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae671e0 0x55ff7ae76070 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f49740c55e0 tx=0x7f49740092b0 comp rx=0 tx=0).stop 2024-03-21T11:41:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.577+0000 7f498e7c6740 1 -- 172.21.15.47:0/4013167634 shutdown_connections 2024-03-21T11:41:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.577+0000 7f498e7c6740 1 --2- 172.21.15.47:0/4013167634 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff7aee1790 0x55ff7aee3bb0 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.577+0000 7f498e7c6740 1 --2- 172.21.15.47:0/4013167634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff7ae6cd40 0x55ff7ae77500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.577+0000 7f498e7c6740 1 --2- 172.21.15.47:0/4013167634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff7ae67bd0 0x55ff7ae765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.577+0000 7f4972a00640 0 client.20958 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:41:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.577+0000 7f498e7c6740 1 --2- 172.21.15.47:0/4013167634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff7ae671e0 0x55ff7ae76070 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.577+0000 7f498e7c6740 1 -- 172.21.15.47:0/4013167634 >> 172.21.15.47:0/4013167634 conn(0x55ff7ae62750 msgr2=0x55ff7ae68e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.577+0000 7f498e7c6740 1 -- 172.21.15.47:0/4013167634 shutdown_connections 2024-03-21T11:41:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.577+0000 7f498e7c6740 1 -- 172.21.15.47:0/4013167634 wait complete. 2024-03-21T11:41:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.577+0000 7f498e7c6740 20 client.20958 trim_cache size 0 max 16384 2024-03-21T11:41:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:41:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:41:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:41:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:41:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.578+0000 7f498ba00640 1 -- 172.21.15.47:0/1437657068 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f49680a3040 msgr2=0x7f49680a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.578+0000 7f498ba00640 1 --2- 172.21.15.47:0/1437657068 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f49680a3040 0x7f49680a5500 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x55ff7ae68430 tx=0x7f497c009500 comp rx=0 tx=0).stop 2024-03-21T11:41:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.578+0000 7f498ba00640 1 -- 172.21.15.47:0/1437657068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4984103900 msgr2=0x7f4984199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:41:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.578+0000 7f498ba00640 1 --2- 172.21.15.47:0/1437657068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4984103900 0x7f4984199b10 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4974000c00 tx=0x7f497402c640 comp rx=0 tx=0).stop 2024-03-21T11:41:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.579+0000 7f498ba00640 1 -- 172.21.15.47:0/1437657068 shutdown_connections 2024-03-21T11:41:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.579+0000 7f498ba00640 1 --2- 172.21.15.47:0/1437657068 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f49680a3040 0x7f49680a5500 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.579+0000 7f498ba00640 1 --2- 172.21.15.47:0/1437657068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49841054b0 0x7f49841a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.579+0000 7f498ba00640 1 --2- 172.21.15.47:0/1437657068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4984104af0 0x7f498419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.579+0000 7f498ba00640 1 --2- 172.21.15.47:0/1437657068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4984103900 0x7f4984199b10 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:41:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.579+0000 7f498ba00640 1 -- 172.21.15.47:0/1437657068 >> 172.21.15.47:0/1437657068 conn(0x7f49840fee90 msgr2=0x7f4984107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:41:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.579+0000 7f498ba00640 1 -- 172.21.15.47:0/1437657068 shutdown_connections 2024-03-21T11:41:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:41:55.579+0000 7f498ba00640 1 -- 172.21.15.47:0/1437657068 wait complete. 2024-03-21T11:41:55.952 INFO:tasks.workunit.client.0.smithi047.stdout: 10 128488 23.78 MB/sec execute 159 sec latency 996.979 ms 2024-03-21T11:41:56.952 INFO:tasks.workunit.client.0.smithi047.stdout: 10 128776 23.75 MB/sec execute 160 sec latency 143.247 ms 2024-03-21T11:41:57.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:57 smithi047 ceph-mon[31351]: pgmap v958: 129 pgs: 129 active+clean; 491 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.9 MiB/s wr, 664 op/s 2024-03-21T11:41:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:57 smithi100 ceph-mon[36245]: pgmap v958: 129 pgs: 129 active+clean; 491 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.9 MiB/s wr, 664 op/s 2024-03-21T11:41:57.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:57 smithi102 ceph-mon[36748]: pgmap v958: 129 pgs: 129 active+clean; 491 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.9 MiB/s wr, 664 op/s 2024-03-21T11:41:57.952 INFO:tasks.workunit.client.0.smithi047.stdout: 10 129233 23.74 MB/sec execute 161 sec latency 147.466 ms 2024-03-21T11:41:58.952 INFO:tasks.workunit.client.0.smithi047.stdout: 10 129611 23.70 MB/sec execute 162 sec latency 140.902 ms 2024-03-21T11:41:59.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:41:59 smithi047 ceph-mon[31351]: pgmap v959: 129 pgs: 129 active+clean; 491 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.3 MiB/s wr, 508 op/s 2024-03-21T11:41:59.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:41:59 smithi100 ceph-mon[36245]: pgmap v959: 129 pgs: 129 active+clean; 491 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.3 MiB/s wr, 508 op/s 2024-03-21T11:41:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:41:59 smithi102 ceph-mon[36748]: pgmap v959: 129 pgs: 129 active+clean; 491 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.3 MiB/s wr, 508 op/s 2024-03-21T11:41:59.952 INFO:tasks.workunit.client.0.smithi047.stdout: 10 130041 23.69 MB/sec execute 163 sec latency 212.107 ms 2024-03-21T11:42:00.953 INFO:tasks.workunit.client.0.smithi047.stdout: 10 130407 23.66 MB/sec execute 164 sec latency 478.091 ms 2024-03-21T11:42:01.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:01 smithi047 ceph-mon[31351]: pgmap v960: 129 pgs: 129 active+clean; 494 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.0 MiB/s wr, 568 op/s 2024-03-21T11:42:01.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:01 smithi100 ceph-mon[36245]: pgmap v960: 129 pgs: 129 active+clean; 494 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.0 MiB/s wr, 568 op/s 2024-03-21T11:42:01.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:01 smithi102 ceph-mon[36748]: pgmap v960: 129 pgs: 129 active+clean; 494 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.0 MiB/s wr, 568 op/s 2024-03-21T11:42:01.952 INFO:tasks.workunit.client.0.smithi047.stdout: 10 130776 23.61 MB/sec execute 165 sec latency 179.399 ms 2024-03-21T11:42:02.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:42:02.357 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:42:02.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.593+0000 7f2236a00640 1 -- 172.21.15.47:0/1702399898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22300740c0 msgr2=0x7f2230074540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:02.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.593+0000 7f2236a00640 1 --2- 172.21.15.47:0/1702399898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22300740c0 0x7f2230074540 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2218009900 tx=0x7f221802bc40 comp rx=0 tx=0).stop 2024-03-21T11:42:02.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.593+0000 7f2236a00640 1 -- 172.21.15.47:0/1702399898 shutdown_connections 2024-03-21T11:42:02.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.593+0000 7f2236a00640 1 --2- 172.21.15.47:0/1702399898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2230109560 0x7f223010b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.593+0000 7f2236a00640 1 --2- 172.21.15.47:0/1702399898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22300740c0 0x7f2230074540 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.593+0000 7f2236a00640 1 --2- 172.21.15.47:0/1702399898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22300756a0 0x7f2230073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.593+0000 7f2236a00640 1 -- 172.21.15.47:0/1702399898 >> 172.21.15.47:0/1702399898 conn(0x7f22300fcc40 msgr2=0x7f22300ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:02.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.593+0000 7f2236a00640 1 -- 172.21.15.47:0/1702399898 shutdown_connections 2024-03-21T11:42:02.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.593+0000 7f2236a00640 1 -- 172.21.15.47:0/1702399898 wait complete. 2024-03-21T11:42:02.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.595+0000 7f2236a00640 1 Processor -- start 2024-03-21T11:42:02.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.595+0000 7f2236a00640 1 -- start start 2024-03-21T11:42:02.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.595+0000 7f2236a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22300740c0 0x7f2230199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.595+0000 7f2236a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22300756a0 0x7f223019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.595+0000 7f2236a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2230109560 0x7f22301a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.595+0000 7f2236a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223010f4f0 con 0x7f22300756a0 2024-03-21T11:42:02.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.595+0000 7f2236a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223010f370 con 0x7f22300740c0 2024-03-21T11:42:02.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.595+0000 7f2236a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223010f670 con 0x7f2230109560 2024-03-21T11:42:02.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.595+0000 7f222fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22300740c0 0x7f2230199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.596+0000 7f2227e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22300756a0 0x7f223019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.596+0000 7f222fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22300740c0 0x7f2230199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56798/0 (socket says 172.21.15.47:56798) 2024-03-21T11:42:02.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.596+0000 7f222fe00640 1 -- 172.21.15.47:0/3457905052 learned_addr learned my addr 172.21.15.47:0/3457905052 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:02.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.596+0000 7f2234a00640 1 --2- 172.21.15.47:0/3457905052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2230109560 0x7f22301a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.596+0000 7f222fe00640 1 -- 172.21.15.47:0/3457905052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2230109560 msgr2=0x7f22301a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:02.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.596+0000 7f222fe00640 1 --2- 172.21.15.47:0/3457905052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2230109560 0x7f22301a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.596+0000 7f222fe00640 1 -- 172.21.15.47:0/3457905052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22300756a0 msgr2=0x7f223019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:02.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.596+0000 7f222fe00640 1 --2- 172.21.15.47:0/3457905052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22300756a0 0x7f223019a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.596+0000 7f222fe00640 1 -- 172.21.15.47:0/3457905052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22301a1800 con 0x7f22300740c0 2024-03-21T11:42:02.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.596+0000 7f222fe00640 1 --2- 172.21.15.47:0/3457905052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22300740c0 0x7f2230199be0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f222000ea60 tx=0x7f222000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:02.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.597+0000 7f2227e00640 1 --2- 172.21.15.47:0/3457905052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22300756a0 0x7f223019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:02.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.597+0000 7f222d600640 1 -- 172.21.15.47:0/3457905052 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222000cd20 con 0x7f22300740c0 2024-03-21T11:42:02.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.597+0000 7f222d600640 1 -- 172.21.15.47:0/3457905052 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22200035f0 con 0x7f22300740c0 2024-03-21T11:42:02.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.597+0000 7f222d600640 1 -- 172.21.15.47:0/3457905052 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220010690 con 0x7f22300740c0 2024-03-21T11:42:02.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.597+0000 7f2236a00640 1 -- 172.21.15.47:0/3457905052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22301a1af0 con 0x7f22300740c0 2024-03-21T11:42:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.597+0000 7f2236a00640 1 -- 172.21.15.47:0/3457905052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22301a1fc0 con 0x7f22300740c0 2024-03-21T11:42:02.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.601+0000 7f222d600640 1 -- 172.21.15.47:0/3457905052 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f22200037a0 con 0x7f22300740c0 2024-03-21T11:42:02.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.603+0000 7f2239776740 1 Processor -- start 2024-03-21T11:42:02.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.603+0000 7f2239776740 1 -- start start 2024-03-21T11:42:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.604+0000 7f222d600640 1 --2- 172.21.15.47:0/3457905052 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22140a2fa0 0x7f22140a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.604+0000 7f2239776740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed41141e80 0x55ed411422a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.604+0000 7f2239776740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41142870 0x55ed41147370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.604+0000 7f2239776740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed411479e0 0x55ed41149e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.604+0000 7f2239776740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed410a7250 con 0x55ed411479e0 2024-03-21T11:42:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.604+0000 7f2239776740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4104fa30 con 0x55ed41142870 2024-03-21T11:42:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.604+0000 7f2239776740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed410a32f0 con 0x55ed41141e80 2024-03-21T11:42:02.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.604+0000 7f222d600640 1 -- 172.21.15.47:0/3457905052 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f22200cfa00 con 0x7f22300740c0 2024-03-21T11:42:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.604+0000 7f222fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed41141e80 0x55ed411422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.604+0000 7f222fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed41141e80 0x55ed411422a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58262/0 (socket says 172.21.15.47:58262) 2024-03-21T11:42:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.604+0000 7f222fe00640 1 -- 172.21.15.47:0/3837085702 learned_addr learned my addr 172.21.15.47:0/3837085702 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:02.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.605+0000 7f2234a00640 1 --2- 172.21.15.47:0/3457905052 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22140a2fa0 0x7f22140a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.605+0000 7f222fe00640 1 -- 172.21.15.47:0/3837085702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41142870 msgr2=0x55ed41147370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.605+0000 7f222fe00640 1 --2- 172.21.15.47:0/3837085702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41142870 0x55ed41147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.605+0000 7f222fe00640 1 -- 172.21.15.47:0/3837085702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed411479e0 msgr2=0x55ed41149e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.605+0000 7f222fe00640 1 --2- 172.21.15.47:0/3837085702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed411479e0 0x55ed41149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.605+0000 7f222fe00640 1 -- 172.21.15.47:0/3837085702 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed4103a690 con 0x55ed41141e80 2024-03-21T11:42:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.605+0000 7f222fe00640 1 --2- 172.21.15.47:0/3837085702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed41141e80 0x55ed411422a0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f22200a4220 tx=0x7f222009b6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=546880e1602a02dd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.606+0000 7f2225200640 1 -- 172.21.15.47:0/3837085702 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220066f80 con 0x55ed41141e80 2024-03-21T11:42:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.606+0000 7f2225200640 1 -- 172.21.15.47:0/3837085702 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2220067b50 con 0x55ed41141e80 2024-03-21T11:42:02.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.606+0000 7f2225200640 1 -- 172.21.15.47:0/3837085702 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220076500 con 0x55ed41141e80 2024-03-21T11:42:02.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.607+0000 7f2239776740 1 -- 172.21.15.47:0/3837085702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed41141e80 msgr2=0x55ed411422a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:02.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.607+0000 7f2239776740 1 --2- 172.21.15.47:0/3837085702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed41141e80 0x55ed411422a0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f22200a4220 tx=0x7f222009b6a0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.607+0000 7f2234a00640 1 --2- 172.21.15.47:0/3457905052 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22140a2fa0 0x7f22140a5460 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2228005fd0 tx=0x7f2228009cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:02.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.608+0000 7f2239776740 1 -- 172.21.15.47:0/3837085702 shutdown_connections 2024-03-21T11:42:02.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.608+0000 7f2239776740 1 --2- 172.21.15.47:0/3837085702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed411479e0 0x55ed41149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.608+0000 7f2239776740 1 --2- 172.21.15.47:0/3837085702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41142870 0x55ed41147370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.608+0000 7f2239776740 1 --2- 172.21.15.47:0/3837085702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed41141e80 0x55ed411422a0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.608+0000 7f2239776740 1 -- 172.21.15.47:0/3837085702 >> 172.21.15.47:0/3837085702 conn(0x55ed4110a200 msgr2=0x55ed4113d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:02.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.608+0000 7f2239776740 1 -- 172.21.15.47:0/3837085702 shutdown_connections 2024-03-21T11:42:02.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.608+0000 7f2239776740 1 -- 172.21.15.47:0/3837085702 wait complete. 2024-03-21T11:42:02.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.609+0000 7f2239776740 1 -- start start 2024-03-21T11:42:02.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.610+0000 7f2239776740 1 Processor -- start 2024-03-21T11:42:02.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.611+0000 7f2239776740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41141e80 0x55ed4114be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.612+0000 7f2239776740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41142870 0x55ed4114c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.612+0000 7f2239776740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed411479e0 0x55ed411533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.612+0000 7f2239776740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4114a990 con 0x55ed41141e80 2024-03-21T11:42:02.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.612+0000 7f2239776740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed41062e80 con 0x55ed41142870 2024-03-21T11:42:02.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.612+0000 7f2239776740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4114ab10 con 0x55ed411479e0 2024-03-21T11:42:02.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.612+0000 7f2239776740 10 client.0 _ll_register_callbacks cb 0x55ed41075130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:02.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.612+0000 7f2227e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41142870 0x55ed4114c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.612+0000 7f2227e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41142870 0x55ed4114c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56820/0 (socket says 172.21.15.47:56820) 2024-03-21T11:42:02.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.613+0000 7f2227e00640 1 -- 172.21.15.47:0/1371070834 learned_addr learned my addr 172.21.15.47:0/1371070834 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:02.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.613+0000 7f2234a00640 1 --2- 172.21.15.47:0/1371070834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed411479e0 0x55ed411533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.613+0000 7f222fe00640 1 --2- 172.21.15.47:0/1371070834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41141e80 0x55ed4114be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.613+0000 7f2227e00640 1 -- 172.21.15.47:0/1371070834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed411479e0 msgr2=0x55ed411533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:02.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.613+0000 7f2227e00640 1 --2- 172.21.15.47:0/1371070834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed411479e0 0x55ed411533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.613+0000 7f2227e00640 1 -- 172.21.15.47:0/1371070834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41141e80 msgr2=0x55ed4114be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:02.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.614+0000 7f2227e00640 1 --2- 172.21.15.47:0/1371070834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41141e80 0x55ed4114be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.614+0000 7f2227e00640 1 -- 172.21.15.47:0/1371070834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed40e857d0 con 0x55ed41142870 2024-03-21T11:42:02.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.614+0000 7f222fe00640 1 --2- 172.21.15.47:0/1371070834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41141e80 0x55ed4114be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:02.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.614+0000 7f2227e00640 1 --2- 172.21.15.47:0/1371070834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41142870 0x55ed4114c360 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2218009900 tx=0x7f2218005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52d7a6294c5ea012 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:02.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.614+0000 7f221d600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:02.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.614+0000 7f221d600640 1 -- 172.21.15.47:0/1371070834 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218005d50 con 0x55ed41142870 2024-03-21T11:42:02.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.615+0000 7f221d600640 1 -- 172.21.15.47:0/1371070834 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f22180046c0 con 0x55ed41142870 2024-03-21T11:42:02.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.615+0000 7f2239776740 1 -- 172.21.15.47:0/1371070834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2220067b50 con 0x55ed41142870 2024-03-21T11:42:02.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.615+0000 7f221d600640 1 -- 172.21.15.47:0/1371070834 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f221804f050 con 0x55ed41142870 2024-03-21T11:42:02.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.615+0000 7f2239776740 10 client.31523 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:02.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.615+0000 7f2239776740 1 -- 172.21.15.47:0/1371070834 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ed40dbb3f0 con 0x55ed41142870 2024-03-21T11:42:02.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.615+0000 7f221d600640 1 -- 172.21.15.47:0/1371070834 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2218004da0 con 0x55ed41142870 2024-03-21T11:42:02.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.616+0000 7f221d600640 1 -- 172.21.15.47:0/1371070834 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f22180039d0 con 0x55ed41142870 2024-03-21T11:42:02.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.616+0000 7f2239776740 10 client.31523 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:02.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.616+0000 7f2239776740 10 client.31523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:02.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.616+0000 7f2239776740 20 client.31523 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:02.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.616+0000 7f2239776740 1 --2- 172.21.15.47:0/1371070834 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed41157bb0 0x55ed41159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.616+0000 7f2239776740 4 client.31523 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:02.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.616+0000 7f2239776740 1 -- 172.21.15.47:0/1371070834 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2220067b50 con 0x55ed41157bb0 2024-03-21T11:42:02.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.617+0000 7f222fe00640 1 --2- 172.21.15.47:0/1371070834 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed41157bb0 0x55ed41159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.617+0000 7f222fe00640 1 --2- 172.21.15.47:0/1371070834 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed41157bb0 0x55ed41159ff0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6936ce16754eca2c server_cookie=4d5d92043ec5971c in_seq=0 out_seq=0 2024-03-21T11:42:02.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.617+0000 7f221d600640 10 client.31523 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.622+0000 7f221d600640 1 -- 172.21.15.47:0/1371070834 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f2220067b50 con 0x55ed41157bb0 2024-03-21T11:42:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.622+0000 7f221d600640 10 client.31523 handle_command_reply: tid=0 2024-03-21T11:42:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.622+0000 7f2239776740 1 client.31523 shutdown 2024-03-21T11:42:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.623+0000 7f2239776740 1 -- 172.21.15.47:0/1371070834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41142870 msgr2=0x55ed4114c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.623+0000 7f2239776740 1 --2- 172.21.15.47:0/1371070834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41142870 0x55ed4114c360 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2218009900 tx=0x7f2218005270 comp rx=0 tx=0).stop 2024-03-21T11:42:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.623+0000 7f2239776740 1 -- 172.21.15.47:0/1371070834 shutdown_connections 2024-03-21T11:42:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.623+0000 7f2239776740 1 --2- 172.21.15.47:0/1371070834 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed41157bb0 0x55ed41159ff0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.623+0000 7f2239776740 1 --2- 172.21.15.47:0/1371070834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed411479e0 0x55ed411533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.623+0000 7f2239776740 1 --2- 172.21.15.47:0/1371070834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41142870 0x55ed4114c360 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.623+0000 7f2239776740 1 --2- 172.21.15.47:0/1371070834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41141e80 0x55ed4114be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.623+0000 7f221d600640 0 client.31523 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:02.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.623+0000 7f2239776740 1 -- 172.21.15.47:0/1371070834 >> 172.21.15.47:0/1371070834 conn(0x55ed4110a200 msgr2=0x55ed4113ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:02.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.623+0000 7f2239776740 1 -- 172.21.15.47:0/1371070834 shutdown_connections 2024-03-21T11:42:02.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.623+0000 7f2239776740 1 -- 172.21.15.47:0/1371070834 wait complete. 2024-03-21T11:42:02.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.623+0000 7f2239776740 20 client.31523 trim_cache size 0 max 16384 2024-03-21T11:42:02.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.645+0000 7f2239776740 1 Processor -- start 2024-03-21T11:42:02.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.645+0000 7f2239776740 1 -- start start 2024-03-21T11:42:02.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.645+0000 7f2239776740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41141e80 0x55ed41141140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.645+0000 7f2239776740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41142870 0x55ed41153360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.645+0000 7f2239776740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed411479e0 0x55ed41143100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.645+0000 7f2239776740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22301ac2f0 con 0x55ed41142870 2024-03-21T11:42:02.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.645+0000 7f2239776740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22301adc70 con 0x55ed41141e80 2024-03-21T11:42:02.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.645+0000 7f2239776740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22301ac950 con 0x55ed411479e0 2024-03-21T11:42:02.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.646+0000 7f222fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41141e80 0x55ed41141140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.646+0000 7f2234a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed411479e0 0x55ed41143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.646+0000 7f2227e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41142870 0x55ed41153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.646+0000 7f222fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41141e80 0x55ed41141140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56824/0 (socket says 172.21.15.47:56824) 2024-03-21T11:42:02.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.646+0000 7f222fe00640 1 -- 172.21.15.47:0/1595904588 learned_addr learned my addr 172.21.15.47:0/1595904588 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:02.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.646+0000 7f222fe00640 1 -- 172.21.15.47:0/1595904588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed411479e0 msgr2=0x55ed41143100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:02.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.646+0000 7f222fe00640 1 --2- 172.21.15.47:0/1595904588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed411479e0 0x55ed41143100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.646+0000 7f222fe00640 1 -- 172.21.15.47:0/1595904588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41142870 msgr2=0x55ed41153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:02.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.646+0000 7f222fe00640 1 --2- 172.21.15.47:0/1595904588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41142870 0x55ed41153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.646+0000 7f222fe00640 1 -- 172.21.15.47:0/1595904588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed40d7b0f0 con 0x55ed41141e80 2024-03-21T11:42:02.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.647+0000 7f222fe00640 1 --2- 172.21.15.47:0/1595904588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41141e80 0x55ed41141140 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f222009ba40 tx=0x7f222000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f1ecfe84381899 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:02.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.647+0000 7f221fe00640 1 -- 172.21.15.47:0/1595904588 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22200ba070 con 0x55ed41141e80 2024-03-21T11:42:02.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.647+0000 7f221fe00640 1 -- 172.21.15.47:0/1595904588 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f222009fe90 con 0x55ed41141e80 2024-03-21T11:42:02.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.647+0000 7f221fe00640 1 -- 172.21.15.47:0/1595904588 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220076750 con 0x55ed41141e80 2024-03-21T11:42:02.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.648+0000 7f2239776740 1 -- 172.21.15.47:0/1595904588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41141e80 msgr2=0x55ed41141140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:02.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.648+0000 7f2239776740 1 --2- 172.21.15.47:0/1595904588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41141e80 0x55ed41141140 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f222009ba40 tx=0x7f222000ef90 comp rx=0 tx=0).stop 2024-03-21T11:42:02.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.648+0000 7f2239776740 1 -- 172.21.15.47:0/1595904588 shutdown_connections 2024-03-21T11:42:02.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.648+0000 7f2239776740 1 --2- 172.21.15.47:0/1595904588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed411479e0 0x55ed41143100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.648+0000 7f2239776740 1 --2- 172.21.15.47:0/1595904588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41142870 0x55ed41153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.648+0000 7f2239776740 1 --2- 172.21.15.47:0/1595904588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41141e80 0x55ed41141140 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.648+0000 7f2239776740 1 -- 172.21.15.47:0/1595904588 >> 172.21.15.47:0/1595904588 conn(0x55ed4110a200 msgr2=0x55ed41140a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:02.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.648+0000 7f2239776740 1 -- 172.21.15.47:0/1595904588 shutdown_connections 2024-03-21T11:42:02.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.649+0000 7f2239776740 1 -- 172.21.15.47:0/1595904588 wait complete. 2024-03-21T11:42:02.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.649+0000 7f2239776740 1 -- start start 2024-03-21T11:42:02.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.649+0000 7f2239776740 1 Processor -- start 2024-03-21T11:42:02.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.650+0000 7f2239776740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41141e80 0x55ed41150c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.650+0000 7f2239776740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41142870 0x55ed411511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.650+0000 7f2239776740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed411479e0 0x55ed411521d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.650+0000 7f222fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41141e80 0x55ed41150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.650+0000 7f222fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41141e80 0x55ed41150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56832/0 (socket says 172.21.15.47:56832) 2024-03-21T11:42:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.650+0000 7f2239776740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22301ad610 con 0x55ed41142870 2024-03-21T11:42:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.650+0000 7f2239776740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22301acfb0 con 0x55ed41141e80 2024-03-21T11:42:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.650+0000 7f2239776740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22301aa4b0 con 0x55ed411479e0 2024-03-21T11:42:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.650+0000 7f2239776740 10 client.0 _ll_register_callbacks cb 0x7f22280093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.650+0000 7f222fe00640 1 -- 172.21.15.47:0/4073907735 learned_addr learned my addr 172.21.15.47:0/4073907735 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.651+0000 7f2227e00640 1 --2- 172.21.15.47:0/4073907735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41142870 0x55ed411511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.651+0000 7f222fe00640 1 -- 172.21.15.47:0/4073907735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed411479e0 msgr2=0x55ed411521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.651+0000 7f222fe00640 1 --2- 172.21.15.47:0/4073907735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed411479e0 0x55ed411521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.651+0000 7f222fe00640 1 -- 172.21.15.47:0/4073907735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41142870 msgr2=0x55ed411511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.651+0000 7f222fe00640 1 --2- 172.21.15.47:0/4073907735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41142870 0x55ed411511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.651+0000 7f222fe00640 1 -- 172.21.15.47:0/4073907735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed411527c0 con 0x55ed41141e80 2024-03-21T11:42:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.651+0000 7f2227e00640 1 --2- 172.21.15.47:0/4073907735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41142870 0x55ed411511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.651+0000 7f222fe00640 1 --2- 172.21.15.47:0/4073907735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41141e80 0x55ed41150c80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f222009bcd0 tx=0x7f222009b6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c442ab3b58e809db server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.651+0000 7f221d600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.652+0000 7f221d600640 1 -- 172.21.15.47:0/4073907735 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22200ba040 con 0x55ed41141e80 2024-03-21T11:42:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.652+0000 7f221d600640 1 -- 172.21.15.47:0/4073907735 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f22200cd220 con 0x55ed41141e80 2024-03-21T11:42:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.652+0000 7f2239776740 1 -- 172.21.15.47:0/4073907735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f222009fe90 con 0x55ed41141e80 2024-03-21T11:42:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.652+0000 7f221d600640 1 -- 172.21.15.47:0/4073907735 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22200764d0 con 0x55ed41141e80 2024-03-21T11:42:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.653+0000 7f2239776740 10 client.31529 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:02.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.653+0000 7f2239776740 1 -- 172.21.15.47:0/4073907735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ed411b3670 con 0x55ed41141e80 2024-03-21T11:42:02.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.653+0000 7f221d600640 1 -- 172.21.15.47:0/4073907735 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f222009fe90 con 0x55ed41141e80 2024-03-21T11:42:02.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.653+0000 7f221d600640 1 -- 172.21.15.47:0/4073907735 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2220097b70 con 0x55ed41141e80 2024-03-21T11:42:02.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.653+0000 7f2239776740 10 client.31529 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:02.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.653+0000 7f2239776740 10 client.31529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:02.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.653+0000 7f2239776740 20 client.31529 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.653+0000 7f2239776740 1 --2- 172.21.15.47:0/4073907735 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed411b38a0 0x55ed411b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.653+0000 7f2239776740 4 client.31529 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:42:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.653+0000 7f2239776740 1 -- 172.21.15.47:0/4073907735 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ed410b4790 con 0x55ed411b38a0 2024-03-21T11:42:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.653+0000 7f2227e00640 1 --2- 172.21.15.47:0/4073907735 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed411b38a0 0x55ed411b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.654+0000 7f2227e00640 1 --2- 172.21.15.47:0/4073907735 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed411b38a0 0x55ed411b5cc0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1523790943313095 server_cookie=f2c00d9e7281e3b in_seq=0 out_seq=0 2024-03-21T11:42:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.654+0000 7f221d600640 10 client.31529 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.655+0000 7f221d600640 1 -- 172.21.15.47:0/4073907735 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ed410b4790 con 0x55ed411b38a0 2024-03-21T11:42:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.655+0000 7f221d600640 10 client.31529 handle_command_reply: tid=0 2024-03-21T11:42:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.655+0000 7f2239776740 1 client.31529 shutdown 2024-03-21T11:42:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.655+0000 7f2239776740 1 -- 172.21.15.47:0/4073907735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41141e80 msgr2=0x55ed41150c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.655+0000 7f2239776740 1 --2- 172.21.15.47:0/4073907735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41141e80 0x55ed41150c80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f222009bcd0 tx=0x7f222009b6a0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.655+0000 7f2239776740 1 -- 172.21.15.47:0/4073907735 shutdown_connections 2024-03-21T11:42:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.655+0000 7f2239776740 1 --2- 172.21.15.47:0/4073907735 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ed411b38a0 0x55ed411b5cc0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.655+0000 7f2239776740 1 --2- 172.21.15.47:0/4073907735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed411479e0 0x55ed411521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.655+0000 7f2239776740 1 --2- 172.21.15.47:0/4073907735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ed41142870 0x55ed411511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.655+0000 7f2239776740 1 --2- 172.21.15.47:0/4073907735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ed41141e80 0x55ed41150c80 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.655+0000 7f2239776740 1 -- 172.21.15.47:0/4073907735 >> 172.21.15.47:0/4073907735 conn(0x55ed4110a200 msgr2=0x55ed411412c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.655+0000 7f221d600640 0 client.31529 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.655+0000 7f2239776740 1 -- 172.21.15.47:0/4073907735 shutdown_connections 2024-03-21T11:42:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.656+0000 7f2239776740 1 -- 172.21.15.47:0/4073907735 wait complete. 2024-03-21T11:42:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.656+0000 7f2239776740 20 client.31529 trim_cache size 0 max 16384 2024-03-21T11:42:02.658 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:42:02.659 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:42:02.659 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:42:02.659 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:42:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.656+0000 7f2236a00640 1 -- 172.21.15.47:0/3457905052 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22140a2fa0 msgr2=0x7f22140a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.656+0000 7f2236a00640 1 --2- 172.21.15.47:0/3457905052 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22140a2fa0 0x7f22140a5460 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2228005fd0 tx=0x7f2228009cb0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.656+0000 7f2236a00640 1 -- 172.21.15.47:0/3457905052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22300740c0 msgr2=0x7f2230199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.656+0000 7f2236a00640 1 --2- 172.21.15.47:0/3457905052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22300740c0 0x7f2230199be0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f222000ea60 tx=0x7f222000ee60 comp rx=0 tx=0).stop 2024-03-21T11:42:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.657+0000 7f2236a00640 1 -- 172.21.15.47:0/3457905052 shutdown_connections 2024-03-21T11:42:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.657+0000 7f2236a00640 1 --2- 172.21.15.47:0/3457905052 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22140a2fa0 0x7f22140a5460 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.657+0000 7f2236a00640 1 --2- 172.21.15.47:0/3457905052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2230109560 0x7f22301a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.657+0000 7f2236a00640 1 --2- 172.21.15.47:0/3457905052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22300756a0 0x7f223019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.657+0000 7f2236a00640 1 --2- 172.21.15.47:0/3457905052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22300740c0 0x7f2230199be0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.657+0000 7f2236a00640 1 -- 172.21.15.47:0/3457905052 >> 172.21.15.47:0/3457905052 conn(0x7f22300fcc40 msgr2=0x7f22301054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.657+0000 7f2236a00640 1 -- 172.21.15.47:0/3457905052 shutdown_connections 2024-03-21T11:42:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.657+0000 7f2236a00640 1 -- 172.21.15.47:0/3457905052 wait complete. 2024-03-21T11:42:02.678 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:42:02.678 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:42:02.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.931+0000 7f22f2200640 1 -- 172.21.15.47:0/368282279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec107ce0 msgr2=0x7f22ec108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:02.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.931+0000 7f22f2200640 1 --2- 172.21.15.47:0/368282279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec107ce0 0x7f22ec108100 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009900 tx=0x7f22dc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:42:02.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.933+0000 7f22f2200640 1 -- 172.21.15.47:0/368282279 shutdown_connections 2024-03-21T11:42:02.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.933+0000 7f22f2200640 1 --2- 172.21.15.47:0/368282279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec109890 0x7f22ec110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.933+0000 7f22f2200640 1 --2- 172.21.15.47:0/368282279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22ec108ed0 0x7f22ec109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.933+0000 7f22f2200640 1 --2- 172.21.15.47:0/368282279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec107ce0 0x7f22ec108100 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.933+0000 7f22f2200640 1 -- 172.21.15.47:0/368282279 >> 172.21.15.47:0/368282279 conn(0x7f22ec076cb0 msgr2=0x7f22ec0790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:02.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.933+0000 7f22f2200640 1 -- 172.21.15.47:0/368282279 shutdown_connections 2024-03-21T11:42:02.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.933+0000 7f22f2200640 1 -- 172.21.15.47:0/368282279 wait complete. 2024-03-21T11:42:02.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.934+0000 7f22f2200640 1 Processor -- start 2024-03-21T11:42:02.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.934+0000 7f22f2200640 1 -- start start 2024-03-21T11:42:02.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.935+0000 7f22f2200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec107ce0 0x7f22ec19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.935+0000 7f22f2200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec108ed0 0x7f22ec19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.935+0000 7f22f2200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22ec109890 0x7f22ec1a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.935+0000 7f22f2200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec113980 con 0x7f22ec108ed0 2024-03-21T11:42:02.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.935+0000 7f22f2200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec113800 con 0x7f22ec109890 2024-03-21T11:42:02.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.935+0000 7f22f2200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec113b00 con 0x7f22ec107ce0 2024-03-21T11:42:02.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.935+0000 7f22eb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec107ce0 0x7f22ec19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.935+0000 7f22eb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec107ce0 0x7f22ec19dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58310/0 (socket says 172.21.15.47:58310) 2024-03-21T11:42:02.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.935+0000 7f22eb400640 1 -- 172.21.15.47:0/702035593 learned_addr learned my addr 172.21.15.47:0/702035593 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:02.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.936+0000 7f22ebe00640 1 --2- 172.21.15.47:0/702035593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22ec109890 0x7f22ec1a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.936+0000 7f22eaa00640 1 --2- 172.21.15.47:0/702035593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec108ed0 0x7f22ec19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.936+0000 7f22eb400640 1 -- 172.21.15.47:0/702035593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22ec109890 msgr2=0x7f22ec1a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:02.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.936+0000 7f22eb400640 1 --2- 172.21.15.47:0/702035593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22ec109890 0x7f22ec1a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.936+0000 7f22eb400640 1 -- 172.21.15.47:0/702035593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec108ed0 msgr2=0x7f22ec19e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:02.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.936+0000 7f22eb400640 1 --2- 172.21.15.47:0/702035593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec108ed0 0x7f22ec19e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:02.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.936+0000 7f22eb400640 1 -- 172.21.15.47:0/702035593 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec1a5be0 con 0x7f22ec107ce0 2024-03-21T11:42:02.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.936+0000 7f22eb400640 1 --2- 172.21.15.47:0/702035593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec107ce0 0x7f22ec19dfc0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f22dc02c240 tx=0x7f22dc004e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:02.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.937+0000 7f22e3e00640 1 -- 172.21.15.47:0/702035593 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc043020 con 0x7f22ec107ce0 2024-03-21T11:42:02.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.937+0000 7f22e3e00640 1 -- 172.21.15.47:0/702035593 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22dc0054e0 con 0x7f22ec107ce0 2024-03-21T11:42:02.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.937+0000 7f22e3e00640 1 -- 172.21.15.47:0/702035593 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc003c60 con 0x7f22ec107ce0 2024-03-21T11:42:02.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.937+0000 7f22f2200640 1 -- 172.21.15.47:0/702035593 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec1a5e70 con 0x7f22ec107ce0 2024-03-21T11:42:02.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.938+0000 7f22f2200640 1 -- 172.21.15.47:0/702035593 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec1a6280 con 0x7f22ec107ce0 2024-03-21T11:42:02.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.940+0000 7f22f2200640 1 -- 172.21.15.47:0/702035593 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ec10efa0 con 0x7f22ec107ce0 2024-03-21T11:42:02.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.944+0000 7f22e3e00640 1 -- 172.21.15.47:0/702035593 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f22dc046020 con 0x7f22ec107ce0 2024-03-21T11:42:02.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.946+0000 7f22e3e00640 1 --2- 172.21.15.47:0/702035593 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22d00a2f70 0x7f22d00a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:02.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.946+0000 7f22e3e00640 1 -- 172.21.15.47:0/702035593 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f22dc0f1590 con 0x7f22ec107ce0 2024-03-21T11:42:02.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.947+0000 7f22eaa00640 1 --2- 172.21.15.47:0/702035593 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22d00a2f70 0x7f22d00a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:02.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.948+0000 7f22eaa00640 1 --2- 172.21.15.47:0/702035593 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22d00a2f70 0x7f22d00a5430 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f22ec19f4d0 tx=0x7f22d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:02.952 INFO:tasks.workunit.client.0.smithi047.stdout: 10 131314 23.68 MB/sec execute 166 sec latency 67.115 ms 2024-03-21T11:42:02.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:02.953+0000 7f22e3e00640 1 -- 172.21.15.47:0/702035593 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f22dc04d030 con 0x7f22ec107ce0 2024-03-21T11:42:03.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.304+0000 7f22f2200640 1 -- 172.21.15.47:0/702035593 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22ec02d070 con 0x7f22ec107ce0 2024-03-21T11:42:03.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.305+0000 7f22e3e00640 1 -- 172.21.15.47:0/702035593 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f22dc0be030 con 0x7f22ec107ce0 2024-03-21T11:42:03.306 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:42:03.306 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:39:47.303461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:42:03.306 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-21T11:42:03.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.308+0000 7f22f2200640 1 -- 172.21.15.47:0/702035593 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22d00a2f70 msgr2=0x7f22d00a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.308+0000 7f22f2200640 1 --2- 172.21.15.47:0/702035593 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22d00a2f70 0x7f22d00a5430 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f22ec19f4d0 tx=0x7f22d8005f90 comp rx=0 tx=0).stop 2024-03-21T11:42:03.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.308+0000 7f22f2200640 1 -- 172.21.15.47:0/702035593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec107ce0 msgr2=0x7f22ec19dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.308+0000 7f22f2200640 1 --2- 172.21.15.47:0/702035593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec107ce0 0x7f22ec19dfc0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f22dc02c240 tx=0x7f22dc004e30 comp rx=0 tx=0).stop 2024-03-21T11:42:03.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.308+0000 7f22f2200640 1 -- 172.21.15.47:0/702035593 shutdown_connections 2024-03-21T11:42:03.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.308+0000 7f22f2200640 1 --2- 172.21.15.47:0/702035593 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22d00a2f70 0x7f22d00a5430 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.308+0000 7f22f2200640 1 --2- 172.21.15.47:0/702035593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22ec109890 0x7f22ec1a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.308+0000 7f22f2200640 1 --2- 172.21.15.47:0/702035593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22ec108ed0 0x7f22ec19e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.308+0000 7f22f2200640 1 --2- 172.21.15.47:0/702035593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec107ce0 0x7f22ec19dfc0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.309+0000 7f22f2200640 1 -- 172.21.15.47:0/702035593 >> 172.21.15.47:0/702035593 conn(0x7f22ec076cb0 msgr2=0x7f22ec10c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:03.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.309+0000 7f22f2200640 1 -- 172.21.15.47:0/702035593 shutdown_connections 2024-03-21T11:42:03.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.309+0000 7f22f2200640 1 -- 172.21.15.47:0/702035593 wait complete. 2024-03-21T11:42:03.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:03 smithi047 ceph-mon[31351]: pgmap v961: 129 pgs: 129 active+clean; 499 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.8 MiB/s wr, 686 op/s 2024-03-21T11:42:03.330 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:42:03.330 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:42:03.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:03 smithi100 ceph-mon[36245]: pgmap v961: 129 pgs: 129 active+clean; 499 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.8 MiB/s wr, 686 op/s 2024-03-21T11:42:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:03 smithi102 ceph-mon[36748]: pgmap v961: 129 pgs: 129 active+clean; 499 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.8 MiB/s wr, 686 op/s 2024-03-21T11:42:03.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.565+0000 7f6c54800640 1 -- 172.21.15.47:0/1900261251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c107cc0 msgr2=0x7f6c4c1080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.565+0000 7f6c54800640 1 --2- 172.21.15.47:0/1900261251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c107cc0 0x7f6c4c1080e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009900 tx=0x7f6c4002bc50 comp rx=0 tx=0).stop 2024-03-21T11:42:03.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.565+0000 7f6c54800640 1 -- 172.21.15.47:0/1900261251 shutdown_connections 2024-03-21T11:42:03.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.565+0000 7f6c54800640 1 --2- 172.21.15.47:0/1900261251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c4c109870 0x7f6c4c110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.565+0000 7f6c54800640 1 --2- 172.21.15.47:0/1900261251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c4c108eb0 0x7f6c4c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.565+0000 7f6c54800640 1 --2- 172.21.15.47:0/1900261251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c107cc0 0x7f6c4c1080e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.565+0000 7f6c54800640 1 -- 172.21.15.47:0/1900261251 >> 172.21.15.47:0/1900261251 conn(0x7f6c4c076c60 msgr2=0x7f6c4c0790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:03.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.565+0000 7f6c54800640 1 -- 172.21.15.47:0/1900261251 shutdown_connections 2024-03-21T11:42:03.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.565+0000 7f6c54800640 1 -- 172.21.15.47:0/1900261251 wait complete. 2024-03-21T11:42:03.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.566+0000 7f6c54800640 1 Processor -- start 2024-03-21T11:42:03.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.567+0000 7f6c54800640 1 -- start start 2024-03-21T11:42:03.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.567+0000 7f6c54800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c4c107cc0 0x7f6c4c19e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.567+0000 7f6c54800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c108eb0 0x7f6c4c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.567+0000 7f6c54800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c4c109870 0x7f6c4c1a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.567+0000 7f6c54800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c113910 con 0x7f6c4c108eb0 2024-03-21T11:42:03.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.567+0000 7f6c54800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c113790 con 0x7f6c4c109870 2024-03-21T11:42:03.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.567+0000 7f6c54800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c113a90 con 0x7f6c4c107cc0 2024-03-21T11:42:03.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.567+0000 7f6c51e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c4c107cc0 0x7f6c4c19e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.567+0000 7f6c51e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c4c107cc0 0x7f6c4c19e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58318/0 (socket says 172.21.15.47:58318) 2024-03-21T11:42:03.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.567+0000 7f6c51e00640 1 -- 172.21.15.47:0/511353214 learned_addr learned my addr 172.21.15.47:0/511353214 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:03.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.567+0000 7f6c51400640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c108eb0 0x7f6c4c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.567+0000 7f6c52800640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c4c109870 0x7f6c4c1a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.568+0000 7f6c51e00640 1 -- 172.21.15.47:0/511353214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c4c109870 msgr2=0x7f6c4c1a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.568+0000 7f6c51e00640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c4c109870 0x7f6c4c1a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.568+0000 7f6c51e00640 1 -- 172.21.15.47:0/511353214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c108eb0 msgr2=0x7f6c4c19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.568+0000 7f6c51e00640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c108eb0 0x7f6c4c19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.568+0000 7f6c51e00640 1 -- 172.21.15.47:0/511353214 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c1a5ca0 con 0x7f6c4c107cc0 2024-03-21T11:42:03.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.568+0000 7f6c51400640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c108eb0 0x7f6c4c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:03.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.568+0000 7f6c51e00640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c4c107cc0 0x7f6c4c19e080 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f6c40000c00 tx=0x7f6c4002c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:03.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.569+0000 7f6c46a00640 1 -- 172.21.15.47:0/511353214 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40042020 con 0x7f6c4c107cc0 2024-03-21T11:42:03.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.569+0000 7f6c54800640 1 -- 172.21.15.47:0/511353214 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4c1a5f30 con 0x7f6c4c107cc0 2024-03-21T11:42:03.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.569+0000 7f6c54800640 1 -- 172.21.15.47:0/511353214 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4c1a63f0 con 0x7f6c4c107cc0 2024-03-21T11:42:03.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.569+0000 7f6c46a00640 1 -- 172.21.15.47:0/511353214 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c40004450 con 0x7f6c4c107cc0 2024-03-21T11:42:03.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.569+0000 7f6c46a00640 1 -- 172.21.15.47:0/511353214 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4003e460 con 0x7f6c4c107cc0 2024-03-21T11:42:03.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.573+0000 7f6c57604740 1 Processor -- start 2024-03-21T11:42:03.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.573+0000 7f6c46a00640 1 -- 172.21.15.47:0/511353214 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6c40045020 con 0x7f6c4c107cc0 2024-03-21T11:42:03.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.573+0000 7f6c57604740 1 -- start start 2024-03-21T11:42:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.574+0000 7f6c57604740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e2f810 0x5610e9e2fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.574+0000 7f6c57604740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e309e0 0x5610e9e30e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.574+0000 7f6c57604740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e313a0 0x5610e9e37c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.574+0000 7f6c57604740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e9d942c0 con 0x5610e9e2f810 2024-03-21T11:42:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.574+0000 7f6c57604740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e9d3cc90 con 0x5610e9e313a0 2024-03-21T11:42:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.574+0000 7f6c57604740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e9d90360 con 0x5610e9e309e0 2024-03-21T11:42:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.574+0000 7f6c51e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e309e0 0x5610e9e30e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.574+0000 7f6c51400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e313a0 0x5610e9e37c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.574+0000 7f6c52800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e2f810 0x5610e9e2fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.574+0000 7f6c51400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e313a0 0x5610e9e37c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56856/0 (socket says 172.21.15.47:56856) 2024-03-21T11:42:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.574+0000 7f6c51400640 1 -- 172.21.15.47:0/276183325 learned_addr learned my addr 172.21.15.47:0/276183325 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.575+0000 7f6c51e00640 1 -- 172.21.15.47:0/276183325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e313a0 msgr2=0x5610e9e37c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.575+0000 7f6c51e00640 1 --2- 172.21.15.47:0/276183325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e313a0 0x5610e9e37c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.575+0000 7f6c51e00640 1 -- 172.21.15.47:0/276183325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e2f810 msgr2=0x5610e9e2fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.575+0000 7f6c51e00640 1 --2- 172.21.15.47:0/276183325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e2f810 0x5610e9e2fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.575+0000 7f6c51e00640 1 -- 172.21.15.47:0/276183325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610e9d61080 con 0x5610e9e309e0 2024-03-21T11:42:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.575+0000 7f6c51e00640 1 --2- 172.21.15.47:0/276183325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e309e0 0x5610e9e30e60 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6c400eed60 tx=0x7f6c400f7b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91ec4a1c983c62f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.575+0000 7f6c3b000640 1 -- 172.21.15.47:0/276183325 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40100070 con 0x5610e9e309e0 2024-03-21T11:42:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.575+0000 7f6c3b000640 1 -- 172.21.15.47:0/276183325 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c400f9d50 con 0x5610e9e309e0 2024-03-21T11:42:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.575+0000 7f6c46a00640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6c340a2fa0 0x7f6c340a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.575+0000 7f6c3b000640 1 -- 172.21.15.47:0/276183325 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400f3870 con 0x5610e9e309e0 2024-03-21T11:42:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.576+0000 7f6c46a00640 1 -- 172.21.15.47:0/511353214 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6c400f1620 con 0x7f6c4c107cc0 2024-03-21T11:42:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.576+0000 7f6c52800640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6c340a2fa0 0x7f6c340a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.576+0000 7f6c57604740 1 -- 172.21.15.47:0/276183325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e309e0 msgr2=0x5610e9e30e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.576+0000 7f6c57604740 1 --2- 172.21.15.47:0/276183325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e309e0 0x5610e9e30e60 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6c400eed60 tx=0x7f6c400f7b40 comp rx=0 tx=0).stop 2024-03-21T11:42:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.576+0000 7f6c57604740 1 -- 172.21.15.47:0/276183325 shutdown_connections 2024-03-21T11:42:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.576+0000 7f6c57604740 1 --2- 172.21.15.47:0/276183325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e313a0 0x5610e9e37c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.576+0000 7f6c57604740 1 --2- 172.21.15.47:0/276183325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e309e0 0x5610e9e30e60 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.576+0000 7f6c57604740 1 --2- 172.21.15.47:0/276183325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e2f810 0x5610e9e2fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.576+0000 7f6c57604740 1 -- 172.21.15.47:0/276183325 >> 172.21.15.47:0/276183325 conn(0x5610e9e2ac00 msgr2=0x5610e9e2b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.577+0000 7f6c57604740 1 -- 172.21.15.47:0/276183325 shutdown_connections 2024-03-21T11:42:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.577+0000 7f6c52800640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6c340a2fa0 0x7f6c340a5460 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x5610e9e308e0 tx=0x7f6c48009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.577+0000 7f6c57604740 1 -- 172.21.15.47:0/276183325 wait complete. 2024-03-21T11:42:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.577+0000 7f6c57604740 1 -- start start 2024-03-21T11:42:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.577+0000 7f6c57604740 1 Processor -- start 2024-03-21T11:42:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.578+0000 7f6c57604740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e2f810 0x5610e9e39740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.578+0000 7f6c57604740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e309e0 0x5610e9e39c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.578+0000 7f6c57604740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e313a0 0x5610e9e40cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.578+0000 7f6c57604740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e9e38170 con 0x5610e9e309e0 2024-03-21T11:42:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.578+0000 7f6c57604740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e9d500d0 con 0x5610e9e2f810 2024-03-21T11:42:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.578+0000 7f6c57604740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610e9e382f0 con 0x5610e9e313a0 2024-03-21T11:42:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.578+0000 7f6c57604740 10 client.0 _ll_register_callbacks cb 0x5610e9d54e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.578+0000 7f6c51e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e2f810 0x5610e9e39740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.578+0000 7f6c51e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e2f810 0x5610e9e39740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56862/0 (socket says 172.21.15.47:56862) 2024-03-21T11:42:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.578+0000 7f6c51e00640 1 -- 172.21.15.47:0/1333192139 learned_addr learned my addr 172.21.15.47:0/1333192139 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.578+0000 7f6c52800640 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e313a0 0x5610e9e40cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.579+0000 7f6c51400640 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e309e0 0x5610e9e39c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.579+0000 7f6c51e00640 1 -- 172.21.15.47:0/1333192139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e313a0 msgr2=0x5610e9e40cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.579+0000 7f6c51e00640 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e313a0 0x5610e9e40cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.579+0000 7f6c51e00640 1 -- 172.21.15.47:0/1333192139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e309e0 msgr2=0x5610e9e39c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.579+0000 7f6c51e00640 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e309e0 0x5610e9e39c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.579+0000 7f6c51e00640 1 -- 172.21.15.47:0/1333192139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610e9aa83f0 con 0x5610e9e2f810 2024-03-21T11:42:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.579+0000 7f6c51400640 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e309e0 0x5610e9e39c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:42:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.579+0000 7f6c52800640 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e313a0 0x5610e9e40cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.579+0000 7f6c51e00640 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e2f810 0x5610e9e39740 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f6c400b9eb0 tx=0x7f6c40003990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=146ca9b48240c247 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.579+0000 7f6c2f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.579+0000 7f6c2f400640 1 -- 172.21.15.47:0/1333192139 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40100070 con 0x5610e9e2f810 2024-03-21T11:42:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.579+0000 7f6c57604740 1 -- 172.21.15.47:0/1333192139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c400f9d50 con 0x5610e9e2f810 2024-03-21T11:42:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.580+0000 7f6c2f400640 1 -- 172.21.15.47:0/1333192139 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6c400bbd50 con 0x5610e9e2f810 2024-03-21T11:42:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.580+0000 7f6c2f400640 1 -- 172.21.15.47:0/1333192139 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c400b9cf0 con 0x5610e9e2f810 2024-03-21T11:42:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.580+0000 7f6c57604740 10 client.31547 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.580+0000 7f6c57604740 1 -- 172.21.15.47:0/1333192139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5610e9b727d0 con 0x5610e9e2f810 2024-03-21T11:42:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.580+0000 7f6c2f400640 1 -- 172.21.15.47:0/1333192139 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c400f9d50 con 0x5610e9e2f810 2024-03-21T11:42:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.581+0000 7f6c2f400640 1 -- 172.21.15.47:0/1333192139 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6c400035f0 con 0x5610e9e2f810 2024-03-21T11:42:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.581+0000 7f6c57604740 10 client.31547 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:03.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.581+0000 7f6c57604740 10 client.31547 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:42:03.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.581+0000 7f6c57604740 20 client.31547 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:03.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.581+0000 7f6c57604740 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5610e9e44ee0 0x5610e9e47330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.581+0000 7f6c57604740 4 client.31547 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:03.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.581+0000 7f6c57604740 1 -- 172.21.15.47:0/1333192139 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610e9da1800 con 0x5610e9e44ee0 2024-03-21T11:42:03.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.581+0000 7f6c51400640 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5610e9e44ee0 0x5610e9e47330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.582+0000 7f6c51400640 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5610e9e44ee0 0x5610e9e47330 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1da192dc3edba949 server_cookie=9134ec7b8e51eeb0 in_seq=0 out_seq=0 2024-03-21T11:42:03.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.582+0000 7f6c2f400640 10 client.31547 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:42:03.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c2f400640 1 -- 172.21.15.47:0/1333192139 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5610e9da1800 con 0x5610e9e44ee0 2024-03-21T11:42:03.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c2f400640 10 client.31547 handle_command_reply: tid=0 2024-03-21T11:42:03.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c57604740 1 client.31547 shutdown 2024-03-21T11:42:03.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c57604740 1 -- 172.21.15.47:0/1333192139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e2f810 msgr2=0x5610e9e39740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c57604740 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e2f810 0x5610e9e39740 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f6c400b9eb0 tx=0x7f6c40003990 comp rx=0 tx=0).stop 2024-03-21T11:42:03.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c57604740 1 -- 172.21.15.47:0/1333192139 shutdown_connections 2024-03-21T11:42:03.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c57604740 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5610e9e44ee0 0x5610e9e47330 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c57604740 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e313a0 0x5610e9e40cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c57604740 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e309e0 0x5610e9e39c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c57604740 1 --2- 172.21.15.47:0/1333192139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e2f810 0x5610e9e39740 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c2f400640 0 client.31547 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:42:03.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c57604740 1 -- 172.21.15.47:0/1333192139 >> 172.21.15.47:0/1333192139 conn(0x5610e9e2ac00 msgr2=0x5610e9e29eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:03.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c57604740 1 -- 172.21.15.47:0/1333192139 shutdown_connections 2024-03-21T11:42:03.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c57604740 1 -- 172.21.15.47:0/1333192139 wait complete. 2024-03-21T11:42:03.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.584+0000 7f6c57604740 20 client.31547 trim_cache size 0 max 16384 2024-03-21T11:42:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.600+0000 7f6c57604740 1 Processor -- start 2024-03-21T11:42:03.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.600+0000 7f6c57604740 1 -- start start 2024-03-21T11:42:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.600+0000 7f6c57604740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e2ac00 0x5610e9e339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.600+0000 7f6c57604740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e2f810 0x5610e9e40c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.600+0000 7f6c57604740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e309e0 0x5610e9e33f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.600+0000 7f6c57604740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c1b0770 con 0x5610e9e309e0 2024-03-21T11:42:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.600+0000 7f6c57604740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c1b20f0 con 0x5610e9e2ac00 2024-03-21T11:42:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.600+0000 7f6c57604740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c1b0dd0 con 0x5610e9e2f810 2024-03-21T11:42:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.600+0000 7f6c51e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e2ac00 0x5610e9e339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.600+0000 7f6c51e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e2ac00 0x5610e9e339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56864/0 (socket says 172.21.15.47:56864) 2024-03-21T11:42:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.601+0000 7f6c51e00640 1 -- 172.21.15.47:0/484837136 learned_addr learned my addr 172.21.15.47:0/484837136 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:03.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.601+0000 7f6c51400640 1 --2- 172.21.15.47:0/484837136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e2f810 0x5610e9e40c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.601+0000 7f6c52800640 1 --2- 172.21.15.47:0/484837136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e309e0 0x5610e9e33f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.601+0000 7f6c51e00640 1 -- 172.21.15.47:0/484837136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e2f810 msgr2=0x5610e9e40c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.601+0000 7f6c51e00640 1 --2- 172.21.15.47:0/484837136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e2f810 0x5610e9e40c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.601+0000 7f6c51e00640 1 -- 172.21.15.47:0/484837136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e309e0 msgr2=0x5610e9e33f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.601+0000 7f6c51e00640 1 --2- 172.21.15.47:0/484837136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e309e0 0x5610e9e33f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.601+0000 7f6c51e00640 1 -- 172.21.15.47:0/484837136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610e9e2b970 con 0x5610e9e2ac00 2024-03-21T11:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.601+0000 7f6c52800640 1 --2- 172.21.15.47:0/484837136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e309e0 0x5610e9e33f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:03.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.601+0000 7f6c51e00640 1 --2- 172.21.15.47:0/484837136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e2ac00 0x5610e9e339f0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f6c400bd9e0 tx=0x7f6c400eede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c81f94820222d158 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:03.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.602+0000 7f6c3a600640 1 -- 172.21.15.47:0/484837136 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40100070 con 0x5610e9e2ac00 2024-03-21T11:42:03.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.602+0000 7f6c3a600640 1 -- 172.21.15.47:0/484837136 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c400c1ba0 con 0x5610e9e2ac00 2024-03-21T11:42:03.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.602+0000 7f6c3a600640 1 -- 172.21.15.47:0/484837136 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400bd3b0 con 0x5610e9e2ac00 2024-03-21T11:42:03.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.603+0000 7f6c57604740 1 -- 172.21.15.47:0/484837136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e2ac00 msgr2=0x5610e9e339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.603+0000 7f6c57604740 1 --2- 172.21.15.47:0/484837136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e2ac00 0x5610e9e339f0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f6c400bd9e0 tx=0x7f6c400eede0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.603+0000 7f6c57604740 1 -- 172.21.15.47:0/484837136 shutdown_connections 2024-03-21T11:42:03.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.603+0000 7f6c57604740 1 --2- 172.21.15.47:0/484837136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e309e0 0x5610e9e33f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.603+0000 7f6c57604740 1 --2- 172.21.15.47:0/484837136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e2f810 0x5610e9e40c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.603+0000 7f6c57604740 1 --2- 172.21.15.47:0/484837136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e2ac00 0x5610e9e339f0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.603+0000 7f6c57604740 1 -- 172.21.15.47:0/484837136 >> 172.21.15.47:0/484837136 conn(0x5610e9e38950 msgr2=0x5610e9e2e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:03.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.603+0000 7f6c57604740 1 -- 172.21.15.47:0/484837136 shutdown_connections 2024-03-21T11:42:03.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.603+0000 7f6c57604740 1 -- 172.21.15.47:0/484837136 wait complete. 2024-03-21T11:42:03.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.604+0000 7f6c57604740 1 -- start start 2024-03-21T11:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.604+0000 7f6c57604740 1 Processor -- start 2024-03-21T11:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.604+0000 7f6c57604740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e2ac00 0x5610e9e3cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.604+0000 7f6c57604740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e2f810 0x5610e9e3d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.604+0000 7f6c57604740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e309e0 0x5610e9e402b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c57604740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c1b1a90 con 0x5610e9e2f810 2024-03-21T11:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c57604740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c1b1430 con 0x5610e9e309e0 2024-03-21T11:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c57604740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c1ae930 con 0x5610e9e2ac00 2024-03-21T11:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c57604740 10 client.0 _ll_register_callbacks cb 0x7f6c48009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c52800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e309e0 0x5610e9e402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c51400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e2f810 0x5610e9e3d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c51e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e2ac00 0x5610e9e3cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c51e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e2ac00 0x5610e9e3cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58346/0 (socket says 172.21.15.47:58346) 2024-03-21T11:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c52800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e309e0 0x5610e9e402b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56880/0 (socket says 172.21.15.47:56880) 2024-03-21T11:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c51e00640 1 -- 172.21.15.47:0/3299668658 learned_addr learned my addr 172.21.15.47:0/3299668658 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c51e00640 1 -- 172.21.15.47:0/3299668658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e309e0 msgr2=0x5610e9e402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c51e00640 1 --2- 172.21.15.47:0/3299668658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e309e0 0x5610e9e402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c51e00640 1 -- 172.21.15.47:0/3299668658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e2f810 msgr2=0x5610e9e3d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c51e00640 1 --2- 172.21.15.47:0/3299668658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e2f810 0x5610e9e3d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c51e00640 1 -- 172.21.15.47:0/3299668658 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610e9e408a0 con 0x5610e9e2ac00 2024-03-21T11:42:03.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.605+0000 7f6c51400640 1 --2- 172.21.15.47:0/3299668658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e2f810 0x5610e9e3d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:03.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.606+0000 7f6c51e00640 1 --2- 172.21.15.47:0/3299668658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e2ac00 0x5610e9e3cd50 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f6c40004e00 tx=0x7f6c4002c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8229f7851a040ef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:03.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.606+0000 7f6c2f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:42:03.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.606+0000 7f6c2f400640 1 -- 172.21.15.47:0/3299668658 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400da070 con 0x5610e9e2ac00 2024-03-21T11:42:03.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.606+0000 7f6c2f400640 1 -- 172.21.15.47:0/3299668658 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6c400035f0 con 0x5610e9e2ac00 2024-03-21T11:42:03.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.606+0000 7f6c57604740 1 -- 172.21.15.47:0/3299668658 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c400c1ba0 con 0x5610e9e2ac00 2024-03-21T11:42:03.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.606+0000 7f6c2f400640 1 -- 172.21.15.47:0/3299668658 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c40100070 con 0x5610e9e2ac00 2024-03-21T11:42:03.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.606+0000 7f6c57604740 10 client.31531 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:03.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.606+0000 7f6c57604740 1 -- 172.21.15.47:0/3299668658 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5610e9e408a0 con 0x5610e9e2ac00 2024-03-21T11:42:03.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.607+0000 7f6c2f400640 1 -- 172.21.15.47:0/3299668658 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c400c1ba0 con 0x5610e9e2ac00 2024-03-21T11:42:03.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.607+0000 7f6c2f400640 1 -- 172.21.15.47:0/3299668658 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6c400dc2f0 con 0x5610e9e2ac00 2024-03-21T11:42:03.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.607+0000 7f6c57604740 10 client.31531 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:03.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.607+0000 7f6c57604740 10 client.31531 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:42:03.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.607+0000 7f6c57604740 20 client.31531 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:03.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.607+0000 7f6c57604740 1 --2- 172.21.15.47:0/3299668658 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5610e9e313a0 0x5610e9ea36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.607+0000 7f6c57604740 4 client.31531 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:42:03.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.607+0000 7f6c57604740 1 -- 172.21.15.47:0/3299668658 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5610e9d9bb50 con 0x5610e9e313a0 2024-03-21T11:42:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.609+0000 7f6c51400640 1 --2- 172.21.15.47:0/3299668658 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5610e9e313a0 0x5610e9ea36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.610+0000 7f6c51400640 1 --2- 172.21.15.47:0/3299668658 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5610e9e313a0 0x5610e9ea36a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2527dd3023b62949 server_cookie=8142dac21346d236 in_seq=0 out_seq=0 2024-03-21T11:42:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.610+0000 7f6c2f400640 10 client.31531 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:42:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.610+0000 7f6c2f400640 1 -- 172.21.15.47:0/3299668658 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5610e9d9bb50 con 0x5610e9e313a0 2024-03-21T11:42:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.610+0000 7f6c2f400640 10 client.31531 handle_command_reply: tid=0 2024-03-21T11:42:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.610+0000 7f6c57604740 1 client.31531 shutdown 2024-03-21T11:42:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.611+0000 7f6c57604740 1 -- 172.21.15.47:0/3299668658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e2ac00 msgr2=0x5610e9e3cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.611+0000 7f6c57604740 1 --2- 172.21.15.47:0/3299668658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e2ac00 0x5610e9e3cd50 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f6c40004e00 tx=0x7f6c4002c8a0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.611+0000 7f6c57604740 1 -- 172.21.15.47:0/3299668658 shutdown_connections 2024-03-21T11:42:03.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.611+0000 7f6c57604740 1 --2- 172.21.15.47:0/3299668658 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5610e9e313a0 0x5610e9ea36a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.611+0000 7f6c57604740 1 --2- 172.21.15.47:0/3299668658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5610e9e309e0 0x5610e9e402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.611+0000 7f6c57604740 1 --2- 172.21.15.47:0/3299668658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5610e9e2f810 0x5610e9e3d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.611+0000 7f6c57604740 1 --2- 172.21.15.47:0/3299668658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5610e9e2ac00 0x5610e9e3cd50 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.611+0000 7f6c57604740 1 -- 172.21.15.47:0/3299668658 >> 172.21.15.47:0/3299668658 conn(0x5610e9e38950 msgr2=0x5610e9e29620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:03.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.611+0000 7f6c2f400640 0 client.31531 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:42:03.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.611+0000 7f6c57604740 1 -- 172.21.15.47:0/3299668658 shutdown_connections 2024-03-21T11:42:03.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.611+0000 7f6c57604740 1 -- 172.21.15.47:0/3299668658 wait complete. 2024-03-21T11:42:03.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.611+0000 7f6c57604740 20 client.31531 trim_cache size 0 max 16384 2024-03-21T11:42:03.613 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:42:03.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.614+0000 7f6c54800640 1 -- 172.21.15.47:0/511353214 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6c340a2fa0 msgr2=0x7f6c340a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.614+0000 7f6c54800640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6c340a2fa0 0x7f6c340a5460 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x5610e9e308e0 tx=0x7f6c48009500 comp rx=0 tx=0).stop 2024-03-21T11:42:03.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.614+0000 7f6c54800640 1 -- 172.21.15.47:0/511353214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c4c107cc0 msgr2=0x7f6c4c19e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.614+0000 7f6c54800640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c4c107cc0 0x7f6c4c19e080 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f6c40000c00 tx=0x7f6c4002c660 comp rx=0 tx=0).stop 2024-03-21T11:42:03.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.614+0000 7f6c54800640 1 -- 172.21.15.47:0/511353214 shutdown_connections 2024-03-21T11:42:03.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.614+0000 7f6c54800640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6c340a2fa0 0x7f6c340a5460 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.614+0000 7f6c54800640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c4c109870 0x7f6c4c1a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.614+0000 7f6c54800640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4c108eb0 0x7f6c4c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.614+0000 7f6c54800640 1 --2- 172.21.15.47:0/511353214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c4c107cc0 0x7f6c4c19e080 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.614+0000 7f6c54800640 1 -- 172.21.15.47:0/511353214 >> 172.21.15.47:0/511353214 conn(0x7f6c4c076c60 msgr2=0x7f6c4c10c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:03.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.614+0000 7f6c54800640 1 -- 172.21.15.47:0/511353214 shutdown_connections 2024-03-21T11:42:03.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.614+0000 7f6c54800640 1 -- 172.21.15.47:0/511353214 wait complete. 2024-03-21T11:42:03.637 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:42:03.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:42:03.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.919+0000 7fdcb2800640 1 -- 172.21.15.47:0/2070659141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac1016b0 msgr2=0x7fdcac101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.919+0000 7fdcb2800640 1 --2- 172.21.15.47:0/2070659141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac1016b0 0x7fdcac101ad0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009be0 tx=0x7fdca402bef0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.920+0000 7fdcb2800640 1 -- 172.21.15.47:0/2070659141 shutdown_connections 2024-03-21T11:42:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.920+0000 7fdcb2800640 1 --2- 172.21.15.47:0/2070659141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcac103260 0x7fdcac109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.920+0000 7fdcb2800640 1 --2- 172.21.15.47:0/2070659141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcac1028a0 0x7fdcac102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.920+0000 7fdcb2800640 1 --2- 172.21.15.47:0/2070659141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac1016b0 0x7fdcac101ad0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.920+0000 7fdcb2800640 1 -- 172.21.15.47:0/2070659141 >> 172.21.15.47:0/2070659141 conn(0x7fdcac0fcc40 msgr2=0x7fdcac0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:03.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.922+0000 7fdcb2800640 1 -- 172.21.15.47:0/2070659141 shutdown_connections 2024-03-21T11:42:03.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.922+0000 7fdcb2800640 1 -- 172.21.15.47:0/2070659141 wait complete. 2024-03-21T11:42:03.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.924+0000 7fdcb2800640 1 Processor -- start 2024-03-21T11:42:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.924+0000 7fdcb2800640 1 -- start start 2024-03-21T11:42:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.925+0000 7fdcb2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac1016b0 0x7fdcac19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.925+0000 7fdcb2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcac1028a0 0x7fdcac19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.925+0000 7fdcb2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcac103260 0x7fdcac1a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.925+0000 7fdcb2800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac075930 con 0x7fdcac1028a0 2024-03-21T11:42:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.925+0000 7fdcb2800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac0757b0 con 0x7fdcac1016b0 2024-03-21T11:42:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.925+0000 7fdcb2800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac10b2f0 con 0x7fdcac103260 2024-03-21T11:42:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.925+0000 7fdcabe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcac103260 0x7fdcac1a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.925+0000 7fdcab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac1016b0 0x7fdcac19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.925+0000 7fdcabe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcac103260 0x7fdcac1a5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58362/0 (socket says 172.21.15.47:58362) 2024-03-21T11:42:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.925+0000 7fdcabe00640 1 -- 172.21.15.47:0/171234330 learned_addr learned my addr 172.21.15.47:0/171234330 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:03.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.926+0000 7fdcaaa00640 1 --2- 172.21.15.47:0/171234330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcac1028a0 0x7fdcac19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.926+0000 7fdcabe00640 1 -- 172.21.15.47:0/171234330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac1016b0 msgr2=0x7fdcac19dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.926+0000 7fdcabe00640 1 --2- 172.21.15.47:0/171234330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac1016b0 0x7fdcac19dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.926+0000 7fdcabe00640 1 -- 172.21.15.47:0/171234330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcac1028a0 msgr2=0x7fdcac19e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.926+0000 7fdcabe00640 1 --2- 172.21.15.47:0/171234330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcac1028a0 0x7fdcac19e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.926+0000 7fdcabe00640 1 -- 172.21.15.47:0/171234330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcac1a5bc0 con 0x7fdcac103260 2024-03-21T11:42:03.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.926+0000 7fdcabe00640 1 --2- 172.21.15.47:0/171234330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcac103260 0x7fdcac1a5550 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00e970 tx=0x7fdc9c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:03.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.927+0000 7fdca3e00640 1 -- 172.21.15.47:0/171234330 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00ccc0 con 0x7fdcac103260 2024-03-21T11:42:03.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.927+0000 7fdca3e00640 1 -- 172.21.15.47:0/171234330 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c005500 con 0x7fdcac103260 2024-03-21T11:42:03.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.927+0000 7fdca3e00640 1 -- 172.21.15.47:0/171234330 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c010690 con 0x7fdcac103260 2024-03-21T11:42:03.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.927+0000 7fdcb2800640 1 -- 172.21.15.47:0/171234330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcac1a5eb0 con 0x7fdcac103260 2024-03-21T11:42:03.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.927+0000 7fdcb2800640 1 -- 172.21.15.47:0/171234330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcac1a6320 con 0x7fdcac103260 2024-03-21T11:42:03.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.937+0000 7fdca3e00640 1 -- 172.21.15.47:0/171234330 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdc9c004e50 con 0x7fdcac103260 2024-03-21T11:42:03.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.938+0000 7fdcb5615740 1 Processor -- start 2024-03-21T11:42:03.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.939+0000 7fdcb5615740 1 -- start start 2024-03-21T11:42:03.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.940+0000 7fdcb5615740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f4e810 0x560b43f4ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.940+0000 7fdcb5615740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4f9e0 0x560b43f4fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.941+0000 7fdcb5615740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f503a0 0x560b43f56c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.941+0000 7fdcb5615740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b43eb32c0 con 0x560b43f4e810 2024-03-21T11:42:03.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.941+0000 7fdcb5615740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b43e5bc90 con 0x560b43f503a0 2024-03-21T11:42:03.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.941+0000 7fdcb5615740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b43eaf360 con 0x560b43f4f9e0 2024-03-21T11:42:03.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.941+0000 7fdca3e00640 1 --2- 172.21.15.47:0/171234330 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc900a2fa0 0x7fdc900a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.942+0000 7fdca3e00640 1 -- 172.21.15.47:0/171234330 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdc9c0cfa20 con 0x7fdcac103260 2024-03-21T11:42:03.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.942+0000 7fdcaaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f4e810 0x560b43f4ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.942+0000 7fdcabe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f503a0 0x560b43f56c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.943+0000 7fdcabe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f503a0 0x560b43f56c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56906/0 (socket says 172.21.15.47:56906) 2024-03-21T11:42:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.943+0000 7fdcabe00640 1 -- 172.21.15.47:0/1708364191 learned_addr learned my addr 172.21.15.47:0/1708364191 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.943+0000 7fdcab400640 1 --2- 172.21.15.47:0/1708364191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4f9e0 0x560b43f4fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.943+0000 7fdcaaa00640 1 --2- 172.21.15.47:0/171234330 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc900a2fa0 0x7fdc900a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.943+0000 7fdcabe00640 1 -- 172.21.15.47:0/1708364191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4f9e0 msgr2=0x560b43f4fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.944+0000 7fdcabe00640 1 --2- 172.21.15.47:0/1708364191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4f9e0 0x560b43f4fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.944+0000 7fdcabe00640 1 -- 172.21.15.47:0/1708364191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f4e810 msgr2=0x560b43f4ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.944+0000 7fdcabe00640 1 --2- 172.21.15.47:0/1708364191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f4e810 0x560b43f4ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.944+0000 7fdcabe00640 1 -- 172.21.15.47:0/1708364191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b43e80080 con 0x560b43f503a0 2024-03-21T11:42:03.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.945+0000 7fdcabe00640 1 --2- 172.21.15.47:0/1708364191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f503a0 0x560b43f56c30 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c098220 tx=0x7fdc9c004930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf498c5f0e1a9593 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:03.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.945+0000 7fdca0c00640 1 -- 172.21.15.47:0/1708364191 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c09bdc0 con 0x560b43f503a0 2024-03-21T11:42:03.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.945+0000 7fdca0c00640 1 -- 172.21.15.47:0/1708364191 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c09ff10 con 0x560b43f503a0 2024-03-21T11:42:03.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.946+0000 7fdca0c00640 1 -- 172.21.15.47:0/1708364191 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0a4340 con 0x560b43f503a0 2024-03-21T11:42:03.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.947+0000 7fdcb5615740 1 -- 172.21.15.47:0/1708364191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f503a0 msgr2=0x560b43f56c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.947+0000 7fdcb5615740 1 --2- 172.21.15.47:0/1708364191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f503a0 0x560b43f56c30 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c098220 tx=0x7fdc9c004930 comp rx=0 tx=0).stop 2024-03-21T11:42:03.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.948+0000 7fdcaaa00640 1 --2- 172.21.15.47:0/171234330 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc900a2fa0 0x7fdc900a5460 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x560b43f4f8e0 tx=0x7fdc94004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:03.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.948+0000 7fdcb5615740 1 -- 172.21.15.47:0/1708364191 shutdown_connections 2024-03-21T11:42:03.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.948+0000 7fdcb5615740 1 --2- 172.21.15.47:0/1708364191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f503a0 0x560b43f56c30 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.948+0000 7fdcb5615740 1 --2- 172.21.15.47:0/1708364191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4f9e0 0x560b43f4fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.948+0000 7fdcb5615740 1 --2- 172.21.15.47:0/1708364191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f4e810 0x560b43f4ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.948+0000 7fdcb5615740 1 -- 172.21.15.47:0/1708364191 >> 172.21.15.47:0/1708364191 conn(0x560b43f49c00 msgr2=0x560b43f4a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:03.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.949+0000 7fdcb5615740 1 -- 172.21.15.47:0/1708364191 shutdown_connections 2024-03-21T11:42:03.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.950+0000 7fdcb5615740 1 -- 172.21.15.47:0/1708364191 wait complete. 2024-03-21T11:42:03.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.951+0000 7fdcb5615740 1 -- start start 2024-03-21T11:42:03.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.951+0000 7fdcb5615740 1 Processor -- start 2024-03-21T11:42:03.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.952+0000 7fdcb5615740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f4e810 0x560b43f58800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.952+0000 7fdcb5615740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4f9e0 0x560b43f58d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.952+0000 7fdcb5615740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f503a0 0x560b43f5fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.952+0000 7fdcb5615740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b43f573a0 con 0x560b43f4e810 2024-03-21T11:42:03.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.952+0000 7fdcb5615740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b43e6f0d0 con 0x560b43f503a0 2024-03-21T11:42:03.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.952+0000 7fdcb5615740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b43f57520 con 0x560b43f4f9e0 2024-03-21T11:42:03.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.952+0000 7fdcb5615740 10 client.0 _ll_register_callbacks cb 0x560b43e73e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:03.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.952+0000 7fdcaaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4f9e0 0x560b43f58d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.952+0000 7fdcaaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4f9e0 0x560b43f58d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58380/0 (socket says 172.21.15.47:58380) 2024-03-21T11:42:03.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.952+0000 7fdcaaa00640 1 -- 172.21.15.47:0/3030011522 learned_addr learned my addr 172.21.15.47:0/3030011522 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:03.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 131888 23.74 MB/sec execute 167 sec latency 65.811 ms 2024-03-21T11:42:03.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.952+0000 7fdcab400640 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f4e810 0x560b43f58800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.953+0000 7fdcabe00640 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f503a0 0x560b43f5fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.954+0000 7fdcaaa00640 1 -- 172.21.15.47:0/3030011522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f503a0 msgr2=0x560b43f5fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.954+0000 7fdcaaa00640 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f503a0 0x560b43f5fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.954+0000 7fdcaaa00640 1 -- 172.21.15.47:0/3030011522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f4e810 msgr2=0x560b43f58800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.954+0000 7fdcaaa00640 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f4e810 0x560b43f58800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.954+0000 7fdcaaa00640 1 -- 172.21.15.47:0/3030011522 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b43bc73f0 con 0x560b43f4f9e0 2024-03-21T11:42:03.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.954+0000 7fdcabe00640 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f503a0 0x560b43f5fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:03.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.954+0000 7fdcab400640 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f4e810 0x560b43f58800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:03.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.954+0000 7fdcaaa00640 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4f9e0 0x560b43f58d40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fdc94011b00 tx=0x7fdc940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9446db871ed3eda9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:03.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.954+0000 7fdc99600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:42:03.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.955+0000 7fdc99600640 1 -- 172.21.15.47:0/3030011522 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94021070 con 0x560b43f4f9e0 2024-03-21T11:42:03.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.955+0000 7fdc99600640 1 -- 172.21.15.47:0/3030011522 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdc9401d420 con 0x560b43f4f9e0 2024-03-21T11:42:03.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.955+0000 7fdcb5615740 1 -- 172.21.15.47:0/3030011522 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc9c09ff10 con 0x560b43f4f9e0 2024-03-21T11:42:03.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.956+0000 7fdc99600640 1 -- 172.21.15.47:0/3030011522 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc9402b050 con 0x560b43f4f9e0 2024-03-21T11:42:03.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.956+0000 7fdcb5615740 10 client.31543 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:03.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.956+0000 7fdcb5615740 1 -- 172.21.15.47:0/3030011522 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560b43c917d0 con 0x560b43f4f9e0 2024-03-21T11:42:03.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.957+0000 7fdc99600640 1 -- 172.21.15.47:0/3030011522 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c09ff10 con 0x560b43f4f9e0 2024-03-21T11:42:03.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.957+0000 7fdc99600640 1 -- 172.21.15.47:0/3030011522 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdc940266e0 con 0x560b43f4f9e0 2024-03-21T11:42:03.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.959+0000 7fdcb5615740 10 client.31543 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:03.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.959+0000 7fdcb5615740 10 client.31543 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:42:03.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.959+0000 7fdcb5615740 20 client.31543 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:03.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.959+0000 7fdcb5615740 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560b43f75370 0x560b43f777b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.959+0000 7fdcb5615740 4 client.31543 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:03.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.959+0000 7fdcb5615740 1 -- 172.21.15.47:0/3030011522 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b43ec0800 con 0x560b43f75370 2024-03-21T11:42:03.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.959+0000 7fdcab400640 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560b43f75370 0x560b43f777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.961+0000 7fdcab400640 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560b43f75370 0x560b43f777b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=37d64376c87621f9 server_cookie=7f06cca4b2b1613d in_seq=0 out_seq=0 2024-03-21T11:42:03.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.961+0000 7fdc99600640 10 client.31543 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:42:03.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.962+0000 7fdc99600640 1 -- 172.21.15.47:0/3030011522 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560b43ec0800 con 0x560b43f75370 2024-03-21T11:42:03.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.962+0000 7fdc99600640 10 client.31543 handle_command_reply: tid=0 2024-03-21T11:42:03.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.963+0000 7fdcb5615740 1 client.31543 shutdown 2024-03-21T11:42:03.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.963+0000 7fdcb5615740 1 -- 172.21.15.47:0/3030011522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4f9e0 msgr2=0x560b43f58d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.963+0000 7fdcb5615740 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4f9e0 0x560b43f58d40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fdc94011b00 tx=0x7fdc940046c0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.963+0000 7fdcb5615740 1 -- 172.21.15.47:0/3030011522 shutdown_connections 2024-03-21T11:42:03.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.963+0000 7fdcb5615740 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560b43f75370 0x560b43f777b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.963+0000 7fdcb5615740 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f503a0 0x560b43f5fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.963+0000 7fdcb5615740 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4f9e0 0x560b43f58d40 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.963+0000 7fdcb5615740 1 --2- 172.21.15.47:0/3030011522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f4e810 0x560b43f58800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.963+0000 7fdc99600640 0 client.31543 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:42:03.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.963+0000 7fdcb5615740 1 -- 172.21.15.47:0/3030011522 >> 172.21.15.47:0/3030011522 conn(0x560b43f49c00 msgr2=0x560b43f48eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:03.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.963+0000 7fdcb5615740 1 -- 172.21.15.47:0/3030011522 shutdown_connections 2024-03-21T11:42:03.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.963+0000 7fdcb5615740 1 -- 172.21.15.47:0/3030011522 wait complete. 2024-03-21T11:42:03.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.964+0000 7fdcb5615740 20 client.31543 trim_cache size 0 max 16384 2024-03-21T11:42:03.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.985+0000 7fdcb5615740 1 Processor -- start 2024-03-21T11:42:03.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.986+0000 7fdcb5615740 1 -- start start 2024-03-21T11:42:03.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.986+0000 7fdcb5615740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4e810 0x560b43f534a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.986+0000 7fdcb5615740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f4f9e0 0x560b43f5fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.986+0000 7fdcb5615740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f503a0 0x560b43f4d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.986+0000 7fdcb5615740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac1a7f70 con 0x560b43f503a0 2024-03-21T11:42:03.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.986+0000 7fdcb5615740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac1a98f0 con 0x560b43f4f9e0 2024-03-21T11:42:03.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.986+0000 7fdcb5615740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac1a85d0 con 0x560b43f4e810 2024-03-21T11:42:03.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.987+0000 7fdcab400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4e810 0x560b43f534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.987+0000 7fdcab400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4e810 0x560b43f534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58388/0 (socket says 172.21.15.47:58388) 2024-03-21T11:42:03.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.987+0000 7fdcab400640 1 -- 172.21.15.47:0/2198624832 learned_addr learned my addr 172.21.15.47:0/2198624832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:03.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.987+0000 7fdcaaa00640 1 --2- 172.21.15.47:0/2198624832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f4f9e0 0x560b43f5fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.987+0000 7fdcabe00640 1 --2- 172.21.15.47:0/2198624832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f503a0 0x560b43f4d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.987+0000 7fdcaaa00640 1 -- 172.21.15.47:0/2198624832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4e810 msgr2=0x560b43f534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.987+0000 7fdcaaa00640 1 --2- 172.21.15.47:0/2198624832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4e810 0x560b43f534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.988+0000 7fdcaaa00640 1 -- 172.21.15.47:0/2198624832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f503a0 msgr2=0x560b43f4d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.988+0000 7fdcaaa00640 1 --2- 172.21.15.47:0/2198624832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f503a0 0x560b43f4d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.988+0000 7fdcaaa00640 1 -- 172.21.15.47:0/2198624832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b43f75130 con 0x560b43f4f9e0 2024-03-21T11:42:03.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.988+0000 7fdcaaa00640 1 --2- 172.21.15.47:0/2198624832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f4f9e0 0x560b43f5fd40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fdc94026bd0 tx=0x7fdc9400c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a05e5fa8a650cd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:03.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.988+0000 7fdc9be00640 1 -- 172.21.15.47:0/2198624832 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94021040 con 0x560b43f4f9e0 2024-03-21T11:42:03.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.988+0000 7fdc9be00640 1 -- 172.21.15.47:0/2198624832 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc94034070 con 0x560b43f4f9e0 2024-03-21T11:42:03.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.988+0000 7fdc9be00640 1 -- 172.21.15.47:0/2198624832 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94030a70 con 0x560b43f4f9e0 2024-03-21T11:42:03.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.989+0000 7fdcb5615740 1 -- 172.21.15.47:0/2198624832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f4f9e0 msgr2=0x560b43f5fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:03.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.989+0000 7fdcb5615740 1 --2- 172.21.15.47:0/2198624832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f4f9e0 0x560b43f5fd40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fdc94026bd0 tx=0x7fdc9400c4e0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.990+0000 7fdcb5615740 1 -- 172.21.15.47:0/2198624832 shutdown_connections 2024-03-21T11:42:03.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.990+0000 7fdcb5615740 1 --2- 172.21.15.47:0/2198624832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f503a0 0x560b43f4d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.990+0000 7fdcb5615740 1 --2- 172.21.15.47:0/2198624832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f4f9e0 0x560b43f5fd40 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.990+0000 7fdcb5615740 1 --2- 172.21.15.47:0/2198624832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4e810 0x560b43f534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.990+0000 7fdcb5615740 1 -- 172.21.15.47:0/2198624832 >> 172.21.15.47:0/2198624832 conn(0x560b43f49c00 msgr2=0x560b43f4ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:03.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.990+0000 7fdcb5615740 1 -- 172.21.15.47:0/2198624832 shutdown_connections 2024-03-21T11:42:03.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.990+0000 7fdcb5615740 1 -- 172.21.15.47:0/2198624832 wait complete. 2024-03-21T11:42:03.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.990+0000 7fdcb5615740 1 -- start start 2024-03-21T11:42:03.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.991+0000 7fdcb5615740 1 Processor -- start 2024-03-21T11:42:03.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.991+0000 7fdcb5615740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4e810 0x560b43f5f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.991+0000 7fdcb5615740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f4f9e0 0x560b43f5df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.991+0000 7fdcb5615740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f503a0 0x560b43f5e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.991+0000 7fdcb5615740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac1a9290 con 0x560b43f503a0 2024-03-21T11:42:03.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.991+0000 7fdcb5615740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac1a8c30 con 0x560b43f4f9e0 2024-03-21T11:42:03.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.991+0000 7fdcb5615740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac117a00 con 0x560b43f4e810 2024-03-21T11:42:03.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.991+0000 7fdcb5615740 10 client.0 _ll_register_callbacks cb 0x7fdc9400c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:03.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.992+0000 7fdcab400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4e810 0x560b43f5f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.992+0000 7fdcab400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4e810 0x560b43f5f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58390/0 (socket says 172.21.15.47:58390) 2024-03-21T11:42:03.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.992+0000 7fdcab400640 1 -- 172.21.15.47:0/28577817 learned_addr learned my addr 172.21.15.47:0/28577817 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:03.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.992+0000 7fdcaaa00640 1 --2- 172.21.15.47:0/28577817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f4f9e0 0x560b43f5df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.993+0000 7fdcaaa00640 1 -- 172.21.15.47:0/28577817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4e810 msgr2=0x560b43f5f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.993+0000 7fdcaaa00640 1 --2- 172.21.15.47:0/28577817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4e810 0x560b43f5f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.993+0000 7fdcaaa00640 1 -- 172.21.15.47:0/28577817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f503a0 msgr2=0x560b43f5e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:03.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.993+0000 7fdcabe00640 1 --2- 172.21.15.47:0/28577817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f503a0 0x560b43f5e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:03.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.993+0000 7fdcaaa00640 1 --2- 172.21.15.47:0/28577817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f503a0 0x560b43f5e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:03.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.993+0000 7fdcaaa00640 1 -- 172.21.15.47:0/28577817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b43f5e9b0 con 0x560b43f4f9e0 2024-03-21T11:42:03.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.994+0000 7fdcabe00640 1 --2- 172.21.15.47:0/28577817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f503a0 0x560b43f5e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:42:03.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.994+0000 7fdcaaa00640 1 --2- 172.21.15.47:0/28577817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f4f9e0 0x560b43f5df30 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fdc94006010 tx=0x7fdc94010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1eb9d1abf83a29f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:03.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.994+0000 7fdc99600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:03.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.995+0000 7fdc99600640 1 -- 172.21.15.47:0/28577817 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94021040 con 0x560b43f4f9e0 2024-03-21T11:42:03.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.995+0000 7fdc99600640 1 -- 172.21.15.47:0/28577817 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdc94005220 con 0x560b43f4f9e0 2024-03-21T11:42:03.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.995+0000 7fdcb5615740 1 -- 172.21.15.47:0/28577817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc94034070 con 0x560b43f4f9e0 2024-03-21T11:42:03.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.995+0000 7fdc99600640 1 -- 172.21.15.47:0/28577817 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc94030440 con 0x560b43f4f9e0 2024-03-21T11:42:03.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.995+0000 7fdcb5615740 10 client.31571 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:03.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.995+0000 7fdcb5615740 1 -- 172.21.15.47:0/28577817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560b43fd1920 con 0x560b43f4f9e0 2024-03-21T11:42:03.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.996+0000 7fdc99600640 1 -- 172.21.15.47:0/28577817 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc94034070 con 0x560b43f4f9e0 2024-03-21T11:42:03.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.996+0000 7fdc99600640 1 -- 172.21.15.47:0/28577817 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdc940047a0 con 0x560b43f4f9e0 2024-03-21T11:42:03.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.996+0000 7fdcb5615740 10 client.31571 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:03.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.996+0000 7fdcb5615740 10 client.31571 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:42:03.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.996+0000 7fdcb5615740 20 client.31571 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:03.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.996+0000 7fdcb5615740 1 --2- 172.21.15.47:0/28577817 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560b43fd2140 0x560b43fd4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:03.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.996+0000 7fdcb5615740 4 client.31571 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:42:03.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.996+0000 7fdcb5615740 1 -- 172.21.15.47:0/28577817 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560b43ebab50 con 0x560b43fd2140 2024-03-21T11:42:03.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.997+0000 7fdcab400640 1 --2- 172.21.15.47:0/28577817 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560b43fd2140 0x560b43fd4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.998+0000 7fdcab400640 1 --2- 172.21.15.47:0/28577817 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560b43fd2140 0x560b43fd4560 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d5e2c1f6dcd0e248 server_cookie=b7c1cfed4b72c25 in_seq=0 out_seq=0 2024-03-21T11:42:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.998+0000 7fdc99600640 10 client.31571 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:42:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.998+0000 7fdc99600640 1 -- 172.21.15.47:0/28577817 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560b43ebab50 con 0x560b43fd2140 2024-03-21T11:42:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.998+0000 7fdc99600640 10 client.31571 handle_command_reply: tid=0 2024-03-21T11:42:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.998+0000 7fdcb5615740 1 client.31571 shutdown 2024-03-21T11:42:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.998+0000 7fdcb5615740 1 -- 172.21.15.47:0/28577817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f4f9e0 msgr2=0x560b43f5df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.998+0000 7fdcb5615740 1 --2- 172.21.15.47:0/28577817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f4f9e0 0x560b43f5df30 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fdc94006010 tx=0x7fdc94010fd0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.999+0000 7fdcb5615740 1 -- 172.21.15.47:0/28577817 shutdown_connections 2024-03-21T11:42:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.999+0000 7fdcb5615740 1 --2- 172.21.15.47:0/28577817 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x560b43fd2140 0x560b43fd4560 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.999+0000 7fdcb5615740 1 --2- 172.21.15.47:0/28577817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b43f503a0 0x560b43f5e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.999+0000 7fdcb5615740 1 --2- 172.21.15.47:0/28577817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b43f4f9e0 0x560b43f5df30 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.999+0000 7fdcb5615740 1 --2- 172.21.15.47:0/28577817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b43f4e810 0x560b43f5f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.999+0000 7fdcb5615740 1 -- 172.21.15.47:0/28577817 >> 172.21.15.47:0/28577817 conn(0x560b43f49c00 msgr2=0x560b43f53320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.999+0000 7fdc99600640 0 client.31571 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:42:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.999+0000 7fdcb5615740 1 -- 172.21.15.47:0/28577817 shutdown_connections 2024-03-21T11:42:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.999+0000 7fdcb5615740 1 -- 172.21.15.47:0/28577817 wait complete. 2024-03-21T11:42:04.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:03.999+0000 7fdcb5615740 20 client.31571 trim_cache size 0 max 16384 2024-03-21T11:42:04.001 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:42:04.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.000+0000 7fdcb2800640 1 -- 172.21.15.47:0/171234330 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc900a2fa0 msgr2=0x7fdc900a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.000+0000 7fdcb2800640 1 --2- 172.21.15.47:0/171234330 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc900a2fa0 0x7fdc900a5460 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x560b43f4f8e0 tx=0x7fdc94004730 comp rx=0 tx=0).stop 2024-03-21T11:42:04.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.000+0000 7fdcb2800640 1 -- 172.21.15.47:0/171234330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcac103260 msgr2=0x7fdcac1a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.000+0000 7fdcb2800640 1 --2- 172.21.15.47:0/171234330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcac103260 0x7fdcac1a5550 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00e970 tx=0x7fdc9c00ee30 comp rx=0 tx=0).stop 2024-03-21T11:42:04.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.000+0000 7fdcb2800640 1 -- 172.21.15.47:0/171234330 shutdown_connections 2024-03-21T11:42:04.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.000+0000 7fdcb2800640 1 --2- 172.21.15.47:0/171234330 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc900a2fa0 0x7fdc900a5460 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.000+0000 7fdcb2800640 1 --2- 172.21.15.47:0/171234330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcac103260 0x7fdcac1a5550 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.000+0000 7fdcb2800640 1 --2- 172.21.15.47:0/171234330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcac1028a0 0x7fdcac19e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.000+0000 7fdcb2800640 1 --2- 172.21.15.47:0/171234330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdcac1016b0 0x7fdcac19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.000+0000 7fdcb2800640 1 -- 172.21.15.47:0/171234330 >> 172.21.15.47:0/171234330 conn(0x7fdcac0fcc40 msgr2=0x7fdcac10dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.000+0000 7fdcb2800640 1 -- 172.21.15.47:0/171234330 shutdown_connections 2024-03-21T11:42:04.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.001+0000 7fdcb2800640 1 -- 172.21.15.47:0/171234330 wait complete. 2024-03-21T11:42:04.021 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:42:04.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:42:04.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.246+0000 7f6b23a00640 1 -- 172.21.15.47:0/185685290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c104ad0 msgr2=0x7f6b1c104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.246+0000 7f6b23a00640 1 --2- 172.21.15.47:0/185685290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c104ad0 0x7f6b1c104f50 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009900 tx=0x7f6b0802bc30 comp rx=0 tx=0).stop 2024-03-21T11:42:04.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.247+0000 7f6b23a00640 1 -- 172.21.15.47:0/185685290 shutdown_connections 2024-03-21T11:42:04.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.247+0000 7f6b23a00640 1 --2- 172.21.15.47:0/185685290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b1c105490 0x7f6b1c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.247+0000 7f6b23a00640 1 --2- 172.21.15.47:0/185685290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c104ad0 0x7f6b1c104f50 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.247+0000 7f6b23a00640 1 --2- 172.21.15.47:0/185685290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b1c1038e0 0x7f6b1c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.247+0000 7f6b23a00640 1 -- 172.21.15.47:0/185685290 >> 172.21.15.47:0/185685290 conn(0x7f6b1c0fee50 msgr2=0x7f6b1c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.247+0000 7f6b23a00640 1 -- 172.21.15.47:0/185685290 shutdown_connections 2024-03-21T11:42:04.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.247+0000 7f6b23a00640 1 -- 172.21.15.47:0/185685290 wait complete. 2024-03-21T11:42:04.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.248+0000 7f6b23a00640 1 Processor -- start 2024-03-21T11:42:04.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.248+0000 7f6b23a00640 1 -- start start 2024-03-21T11:42:04.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.249+0000 7f6b23a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c1038e0 0x7f6b1c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.249+0000 7f6b23a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b1c104ad0 0x7f6b1c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.249+0000 7f6b23a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b1c105490 0x7f6b1c1a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.249+0000 7f6b23a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c10f5b0 con 0x7f6b1c104ad0 2024-03-21T11:42:04.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.249+0000 7f6b23a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c10f430 con 0x7f6b1c105490 2024-03-21T11:42:04.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.249+0000 7f6b23a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c10f730 con 0x7f6b1c1038e0 2024-03-21T11:42:04.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.249+0000 7f6b21a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b1c105490 0x7f6b1c1a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.249+0000 7f6b21000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c1038e0 0x7f6b1c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.249+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b1c104ad0 0x7f6b1c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.249+0000 7f6b21a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b1c105490 0x7f6b1c1a1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56966/0 (socket says 172.21.15.47:56966) 2024-03-21T11:42:04.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.249+0000 7f6b21000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c1038e0 0x7f6b1c199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58408/0 (socket says 172.21.15.47:58408) 2024-03-21T11:42:04.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.249+0000 7f6b21a00640 1 -- 172.21.15.47:0/3863008845 learned_addr learned my addr 172.21.15.47:0/3863008845 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.250+0000 7f6b21a00640 1 -- 172.21.15.47:0/3863008845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c1038e0 msgr2=0x7f6b1c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.250+0000 7f6b21a00640 1 --2- 172.21.15.47:0/3863008845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c1038e0 0x7f6b1c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.250+0000 7f6b21a00640 1 -- 172.21.15.47:0/3863008845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b1c104ad0 msgr2=0x7f6b1c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.250+0000 7f6b21a00640 1 --2- 172.21.15.47:0/3863008845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b1c104ad0 0x7f6b1c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.250+0000 7f6b21a00640 1 -- 172.21.15.47:0/3863008845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c1a17e0 con 0x7f6b1c105490 2024-03-21T11:42:04.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.250+0000 7f6b21a00640 1 --2- 172.21.15.47:0/3863008845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b1c105490 0x7f6b1c1a1170 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400eb10 tx=0x7f6b1400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.251+0000 7f6b19600640 1 -- 172.21.15.47:0/3863008845 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14004b20 con 0x7f6b1c105490 2024-03-21T11:42:04.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.251+0000 7f6b19600640 1 -- 172.21.15.47:0/3863008845 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b14004cc0 con 0x7f6b1c105490 2024-03-21T11:42:04.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.251+0000 7f6b23a00640 1 -- 172.21.15.47:0/3863008845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c1a1ad0 con 0x7f6b1c105490 2024-03-21T11:42:04.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.251+0000 7f6b19600640 1 -- 172.21.15.47:0/3863008845 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14010760 con 0x7f6b1c105490 2024-03-21T11:42:04.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.251+0000 7f6b23a00640 1 -- 172.21.15.47:0/3863008845 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c1a2040 con 0x7f6b1c105490 2024-03-21T11:42:04.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.255+0000 7f6b19600640 1 -- 172.21.15.47:0/3863008845 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6b14005020 con 0x7f6b1c105490 2024-03-21T11:42:04.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.255+0000 7f6b267d5740 1 Processor -- start 2024-03-21T11:42:04.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.255+0000 7f6b267d5740 1 -- start start 2024-03-21T11:42:04.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.255+0000 7f6b267d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86dac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.255+0000 7f6b267d5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86dbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.255+0000 7f6b267d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 0x5572a86e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.255+0000 7f6b267d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a863f2c0 con 0x5572a86dc3a0 2024-03-21T11:42:04.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.255+0000 7f6b267d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a85e7c90 con 0x5572a86da810 2024-03-21T11:42:04.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.255+0000 7f6b267d5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a863b360 con 0x5572a86db9e0 2024-03-21T11:42:04.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.256+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86dac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.256+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86dac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56980/0 (socket says 172.21.15.47:56980) 2024-03-21T11:42:04.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.256+0000 7f6b1be00640 1 -- 172.21.15.47:0/1185229218 learned_addr learned my addr 172.21.15.47:0/1185229218 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.256+0000 7f6b1be00640 1 --2- 172.21.15.47:0/1185229218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 0x5572a86e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.257+0000 7f6b1be00640 1 -- 172.21.15.47:0/1185229218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 msgr2=0x5572a86dbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.257+0000 7f6b1be00640 1 --2- 172.21.15.47:0/1185229218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86dbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.257+0000 7f6b1be00640 1 -- 172.21.15.47:0/1185229218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 msgr2=0x5572a86dac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.257+0000 7f6b1be00640 1 --2- 172.21.15.47:0/1185229218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86dac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.257+0000 7f6b1be00640 1 -- 172.21.15.47:0/1185229218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572a860c080 con 0x5572a86dc3a0 2024-03-21T11:42:04.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.257+0000 7f6b19600640 1 --2- 172.21.15.47:0/3863008845 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6b040a2fa0 0x7f6b040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.257+0000 7f6b1be00640 1 --2- 172.21.15.47:0/1185229218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 0x5572a86e2c30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6b0802c710 tx=0x7f6b0802df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd06a6d70b69366 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.257+0000 7f6b19600640 1 -- 172.21.15.47:0/3863008845 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6b140cfda0 con 0x7f6b1c105490 2024-03-21T11:42:04.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.257+0000 7f6b21000640 1 --2- 172.21.15.47:0/3863008845 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6b040a2fa0 0x7f6b040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.257+0000 7f6b0e000640 1 -- 172.21.15.47:0/1185229218 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08004040 con 0x5572a86dc3a0 2024-03-21T11:42:04.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.257+0000 7f6b0e000640 1 -- 172.21.15.47:0/1185229218 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b08004ba0 con 0x5572a86dc3a0 2024-03-21T11:42:04.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.257+0000 7f6b0e000640 1 -- 172.21.15.47:0/1185229218 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08005110 con 0x5572a86dc3a0 2024-03-21T11:42:04.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.258+0000 7f6b267d5740 1 -- 172.21.15.47:0/1185229218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 msgr2=0x5572a86e2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.258+0000 7f6b267d5740 1 --2- 172.21.15.47:0/1185229218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 0x5572a86e2c30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6b0802c710 tx=0x7f6b0802df90 comp rx=0 tx=0).stop 2024-03-21T11:42:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.259+0000 7f6b21000640 1 --2- 172.21.15.47:0/3863008845 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6b040a2fa0 0x7f6b040a5460 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c104930 tx=0x7f6b10009840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.259+0000 7f6b267d5740 1 -- 172.21.15.47:0/1185229218 shutdown_connections 2024-03-21T11:42:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.259+0000 7f6b267d5740 1 --2- 172.21.15.47:0/1185229218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 0x5572a86e2c30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.259+0000 7f6b267d5740 1 --2- 172.21.15.47:0/1185229218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86dbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.259+0000 7f6b267d5740 1 --2- 172.21.15.47:0/1185229218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86dac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.259+0000 7f6b267d5740 1 -- 172.21.15.47:0/1185229218 >> 172.21.15.47:0/1185229218 conn(0x5572a86d5c00 msgr2=0x5572a86d6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.259+0000 7f6b267d5740 1 -- 172.21.15.47:0/1185229218 shutdown_connections 2024-03-21T11:42:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.259+0000 7f6b267d5740 1 -- 172.21.15.47:0/1185229218 wait complete. 2024-03-21T11:42:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.259+0000 7f6b267d5740 1 -- start start 2024-03-21T11:42:04.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.260+0000 7f6b267d5740 1 Processor -- start 2024-03-21T11:42:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.261+0000 7f6b267d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86e4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.261+0000 7f6b267d5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.261+0000 7f6b267d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 0x5572a86ec000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.261+0000 7f6b267d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a86e3350 con 0x5572a86dc3a0 2024-03-21T11:42:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.261+0000 7f6b267d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a85fb0d0 con 0x5572a86da810 2024-03-21T11:42:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.261+0000 7f6b267d5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a86e34d0 con 0x5572a86db9e0 2024-03-21T11:42:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.261+0000 7f6b267d5740 10 client.0 _ll_register_callbacks cb 0x5572a85ffe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.261+0000 7f6b21000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86e4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.261+0000 7f6b21000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86e4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56986/0 (socket says 172.21.15.47:56986) 2024-03-21T11:42:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.261+0000 7f6b21000640 1 -- 172.21.15.47:0/2976953411 learned_addr learned my addr 172.21.15.47:0/2976953411 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.262+0000 7f6b1be00640 1 --2- 172.21.15.47:0/2976953411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.262+0000 7f6b21a00640 1 --2- 172.21.15.47:0/2976953411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 0x5572a86ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.262+0000 7f6b21000640 1 -- 172.21.15.47:0/2976953411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 msgr2=0x5572a86e4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.262+0000 7f6b21000640 1 --2- 172.21.15.47:0/2976953411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86e4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.262+0000 7f6b21000640 1 -- 172.21.15.47:0/2976953411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 msgr2=0x5572a86ec000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.262+0000 7f6b21000640 1 --2- 172.21.15.47:0/2976953411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 0x5572a86ec000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.262+0000 7f6b21000640 1 -- 172.21.15.47:0/2976953411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572a83533f0 con 0x5572a86da810 2024-03-21T11:42:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.263+0000 7f6b21000640 1 --2- 172.21.15.47:0/2976953411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86e4a80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f6b1001cad0 tx=0x7f6b10005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=379fe03ae3f5be69 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.263+0000 7f6b02a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.263+0000 7f6b02a00640 1 -- 172.21.15.47:0/2976953411 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1001d820 con 0x5572a86da810 2024-03-21T11:42:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.263+0000 7f6b02a00640 1 -- 172.21.15.47:0/2976953411 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6b1001d9c0 con 0x5572a86da810 2024-03-21T11:42:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.263+0000 7f6b267d5740 1 -- 172.21.15.47:0/2976953411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b08004ba0 con 0x5572a86da810 2024-03-21T11:42:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.264+0000 7f6b02a00640 1 -- 172.21.15.47:0/2976953411 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b1001dbc0 con 0x5572a86da810 2024-03-21T11:42:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.264+0000 7f6b267d5740 10 client.31583 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.264+0000 7f6b267d5740 1 -- 172.21.15.47:0/2976953411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5572a841d7d0 con 0x5572a86da810 2024-03-21T11:42:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.264+0000 7f6b02a00640 1 -- 172.21.15.47:0/2976953411 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b08004ba0 con 0x5572a86da810 2024-03-21T11:42:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.264+0000 7f6b02a00640 1 -- 172.21.15.47:0/2976953411 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6b1001a390 con 0x5572a86da810 2024-03-21T11:42:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.264+0000 7f6b267d5740 10 client.31583 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.264+0000 7f6b267d5740 10 client.31583 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:42:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.264+0000 7f6b267d5740 20 client.31583 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.264+0000 7f6b267d5740 1 --2- 172.21.15.47:0/2976953411 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5572a86f0480 0x5572a86f2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.264+0000 7f6b267d5740 4 client.31583 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.265+0000 7f6b267d5740 1 -- 172.21.15.47:0/2976953411 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572a864c800 con 0x5572a86f0480 2024-03-21T11:42:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.265+0000 7f6b1be00640 1 --2- 172.21.15.47:0/2976953411 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5572a86f0480 0x5572a86f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.266+0000 7f6b1be00640 1 --2- 172.21.15.47:0/2976953411 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5572a86f0480 0x5572a86f2920 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ecb73eba322be77 server_cookie=26f3910c65a1fe5e in_seq=0 out_seq=0 2024-03-21T11:42:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.266+0000 7f6b02a00640 10 client.31583 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:42:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.267+0000 7f6b02a00640 1 -- 172.21.15.47:0/2976953411 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5572a864c800 con 0x5572a86f0480 2024-03-21T11:42:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.267+0000 7f6b02a00640 10 client.31583 handle_command_reply: tid=0 2024-03-21T11:42:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.268+0000 7f6b267d5740 1 client.31583 shutdown 2024-03-21T11:42:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.268+0000 7f6b267d5740 1 -- 172.21.15.47:0/2976953411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 msgr2=0x5572a86e4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.268+0000 7f6b267d5740 1 --2- 172.21.15.47:0/2976953411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86e4a80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f6b1001cad0 tx=0x7f6b10005f90 comp rx=0 tx=0).stop 2024-03-21T11:42:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.268+0000 7f6b267d5740 1 -- 172.21.15.47:0/2976953411 shutdown_connections 2024-03-21T11:42:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.268+0000 7f6b267d5740 1 --2- 172.21.15.47:0/2976953411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 0x5572a86ec000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.268+0000 7f6b267d5740 1 --2- 172.21.15.47:0/2976953411 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5572a86f0480 0x5572a86f2920 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.268+0000 7f6b267d5740 1 --2- 172.21.15.47:0/2976953411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86e4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.268+0000 7f6b267d5740 1 --2- 172.21.15.47:0/2976953411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86e4a80 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.268+0000 7f6b02a00640 0 client.31583 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:42:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.268+0000 7f6b267d5740 1 -- 172.21.15.47:0/2976953411 >> 172.21.15.47:0/2976953411 conn(0x5572a86d5c00 msgr2=0x5572a86d4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.268+0000 7f6b267d5740 1 -- 172.21.15.47:0/2976953411 shutdown_connections 2024-03-21T11:42:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.269+0000 7f6b267d5740 1 -- 172.21.15.47:0/2976953411 wait complete. 2024-03-21T11:42:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.269+0000 7f6b267d5740 20 client.31583 trim_cache size 0 max 16384 2024-03-21T11:42:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.283+0000 7f6b267d5740 1 Processor -- start 2024-03-21T11:42:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.283+0000 7f6b267d5740 1 -- start start 2024-03-21T11:42:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.283+0000 7f6b267d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86da810 0x5572a86e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.283+0000 7f6b267d5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.283+0000 7f6b267d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86dc3a0 0x5572a86d9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.283+0000 7f6b267d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1ac300 con 0x5572a86da810 2024-03-21T11:42:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.283+0000 7f6b267d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1adc80 con 0x5572a86dc3a0 2024-03-21T11:42:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.283+0000 7f6b267d5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1ac960 con 0x5572a86db9e0 2024-03-21T11:42:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.284+0000 7f6b21000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86da810 0x5572a86e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.284+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.284+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86ebfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58450/0 (socket says 172.21.15.47:58450) 2024-03-21T11:42:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.284+0000 7f6b1be00640 1 -- 172.21.15.47:0/4223240559 learned_addr learned my addr 172.21.15.47:0/4223240559 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.284+0000 7f6b21a00640 1 --2- 172.21.15.47:0/4223240559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86dc3a0 0x5572a86d9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.284+0000 7f6b21000640 1 -- 172.21.15.47:0/4223240559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 msgr2=0x5572a86ebfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.284+0000 7f6b21000640 1 --2- 172.21.15.47:0/4223240559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86ebfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.284+0000 7f6b21000640 1 -- 172.21.15.47:0/4223240559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86dc3a0 msgr2=0x5572a86d9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.284+0000 7f6b21000640 1 --2- 172.21.15.47:0/4223240559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86dc3a0 0x5572a86d9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.284+0000 7f6b21000640 1 -- 172.21.15.47:0/4223240559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572a86d6650 con 0x5572a86da810 2024-03-21T11:42:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.284+0000 7f6b21a00640 1 --2- 172.21.15.47:0/4223240559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86dc3a0 0x5572a86d9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.284+0000 7f6b21000640 1 --2- 172.21.15.47:0/4223240559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86da810 0x5572a86e4f20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6b1001a830 tx=0x7f6b1001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37703a7d745fdcf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.285+0000 7f6b0d600640 1 -- 172.21.15.47:0/4223240559 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1003b070 con 0x5572a86da810 2024-03-21T11:42:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.285+0000 7f6b0d600640 1 -- 172.21.15.47:0/4223240559 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b1001a9a0 con 0x5572a86da810 2024-03-21T11:42:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.285+0000 7f6b0d600640 1 -- 172.21.15.47:0/4223240559 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b100055f0 con 0x5572a86da810 2024-03-21T11:42:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.286+0000 7f6b267d5740 1 -- 172.21.15.47:0/4223240559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86da810 msgr2=0x5572a86e4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.286+0000 7f6b267d5740 1 --2- 172.21.15.47:0/4223240559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86da810 0x5572a86e4f20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6b1001a830 tx=0x7f6b1001c730 comp rx=0 tx=0).stop 2024-03-21T11:42:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.286+0000 7f6b267d5740 1 -- 172.21.15.47:0/4223240559 shutdown_connections 2024-03-21T11:42:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.286+0000 7f6b267d5740 1 --2- 172.21.15.47:0/4223240559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86dc3a0 0x5572a86d9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.286+0000 7f6b267d5740 1 --2- 172.21.15.47:0/4223240559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86ebfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.286+0000 7f6b267d5740 1 --2- 172.21.15.47:0/4223240559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86da810 0x5572a86e4f20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.286+0000 7f6b267d5740 1 -- 172.21.15.47:0/4223240559 >> 172.21.15.47:0/4223240559 conn(0x5572a86d5c00 msgr2=0x5572a86d8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.286+0000 7f6b267d5740 1 -- 172.21.15.47:0/4223240559 shutdown_connections 2024-03-21T11:42:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.286+0000 7f6b267d5740 1 -- 172.21.15.47:0/4223240559 wait complete. 2024-03-21T11:42:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.286+0000 7f6b267d5740 1 -- start start 2024-03-21T11:42:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.286+0000 7f6b267d5740 1 Processor -- start 2024-03-21T11:42:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.287+0000 7f6b267d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86e7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.287+0000 7f6b267d5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86e7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.287+0000 7f6b267d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 0x5572a86eac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.287+0000 7f6b267d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1ad620 con 0x5572a86dc3a0 2024-03-21T11:42:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.287+0000 7f6b267d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1acfc0 con 0x5572a86da810 2024-03-21T11:42:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.287+0000 7f6b267d5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1aa4c0 con 0x5572a86db9e0 2024-03-21T11:42:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.287+0000 7f6b267d5740 10 client.0 _ll_register_callbacks cb 0x7f6b10009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.287+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86e7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.287+0000 7f6b21a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 0x5572a86eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.287+0000 7f6b21000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86e7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.287+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86e7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58462/0 (socket says 172.21.15.47:58462) 2024-03-21T11:42:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.287+0000 7f6b1be00640 1 -- 172.21.15.47:0/2578359236 learned_addr learned my addr 172.21.15.47:0/2578359236 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.288+0000 7f6b21a00640 1 -- 172.21.15.47:0/2578359236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 msgr2=0x5572a86e7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.288+0000 7f6b21a00640 1 --2- 172.21.15.47:0/2578359236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86e7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.288+0000 7f6b21a00640 1 -- 172.21.15.47:0/2578359236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 msgr2=0x5572a86e7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.288+0000 7f6b21a00640 1 --2- 172.21.15.47:0/2578359236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86e7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.288+0000 7f6b21a00640 1 -- 172.21.15.47:0/2578359236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572a86eb270 con 0x5572a86dc3a0 2024-03-21T11:42:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.288+0000 7f6b21000640 1 --2- 172.21.15.47:0/2578359236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.288+0000 7f6b21a00640 1 --2- 172.21.15.47:0/2578359236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 0x5572a86eac80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6b140cd880 tx=0x7f6b1409bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6198ab5755d0b880 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.288+0000 7f6b02a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:42:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.288+0000 7f6b02a00640 1 -- 172.21.15.47:0/2578359236 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b140bc2b0 con 0x5572a86dc3a0 2024-03-21T11:42:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.288+0000 7f6b02a00640 1 -- 172.21.15.47:0/2578359236 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6b14005310 con 0x5572a86dc3a0 2024-03-21T11:42:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.288+0000 7f6b267d5740 1 -- 172.21.15.47:0/2578359236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b1001a9a0 con 0x5572a86dc3a0 2024-03-21T11:42:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.288+0000 7f6b02a00640 1 -- 172.21.15.47:0/2578359236 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b140b5240 con 0x5572a86dc3a0 2024-03-21T11:42:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.288+0000 7f6b267d5740 10 client.21018 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.289+0000 7f6b267d5740 1 -- 172.21.15.47:0/2578359236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5572a874be20 con 0x5572a86dc3a0 2024-03-21T11:42:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.289+0000 7f6b02a00640 1 -- 172.21.15.47:0/2578359236 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b1001a9a0 con 0x5572a86dc3a0 2024-03-21T11:42:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.289+0000 7f6b02a00640 1 -- 172.21.15.47:0/2578359236 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6b1409b590 con 0x5572a86dc3a0 2024-03-21T11:42:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.289+0000 7f6b267d5740 10 client.21018 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.289+0000 7f6b267d5740 10 client.21018 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:42:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.289+0000 7f6b267d5740 20 client.21018 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.289+0000 7f6b267d5740 1 --2- 172.21.15.47:0/2578359236 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5572a874c6b0 0x5572a874eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.289+0000 7f6b267d5740 4 client.21018 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:42:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.289+0000 7f6b267d5740 1 -- 172.21.15.47:0/2578359236 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5572a8646b50 con 0x5572a874c6b0 2024-03-21T11:42:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.291+0000 7f6b21000640 1 --2- 172.21.15.47:0/2578359236 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5572a874c6b0 0x5572a874eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.291+0000 7f6b21000640 1 --2- 172.21.15.47:0/2578359236 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5572a874c6b0 0x5572a874eb30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2297f0a3400f9ee6 server_cookie=20107c2b344e899e in_seq=0 out_seq=0 2024-03-21T11:42:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.291+0000 7f6b02a00640 10 client.21018 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:42:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.292+0000 7f6b02a00640 1 -- 172.21.15.47:0/2578359236 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5572a8646b50 con 0x5572a874c6b0 2024-03-21T11:42:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.292+0000 7f6b02a00640 10 client.21018 handle_command_reply: tid=0 2024-03-21T11:42:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.292+0000 7f6b267d5740 1 client.21018 shutdown 2024-03-21T11:42:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.293+0000 7f6b267d5740 1 -- 172.21.15.47:0/2578359236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 msgr2=0x5572a86eac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.293+0000 7f6b267d5740 1 --2- 172.21.15.47:0/2578359236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 0x5572a86eac80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6b140cd880 tx=0x7f6b1409bd90 comp rx=0 tx=0).stop 2024-03-21T11:42:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.293+0000 7f6b267d5740 1 -- 172.21.15.47:0/2578359236 shutdown_connections 2024-03-21T11:42:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.293+0000 7f6b267d5740 1 --2- 172.21.15.47:0/2578359236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572a86dc3a0 0x5572a86eac80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.293+0000 7f6b267d5740 1 --2- 172.21.15.47:0/2578359236 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5572a874c6b0 0x5572a874eb30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.293+0000 7f6b267d5740 1 --2- 172.21.15.47:0/2578359236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572a86db9e0 0x5572a86e7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.293+0000 7f6b267d5740 1 --2- 172.21.15.47:0/2578359236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572a86da810 0x5572a86e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.293+0000 7f6b267d5740 1 -- 172.21.15.47:0/2578359236 >> 172.21.15.47:0/2578359236 conn(0x5572a86d5c00 msgr2=0x5572a86d96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.293+0000 7f6b02a00640 0 client.21018 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:42:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.293+0000 7f6b267d5740 1 -- 172.21.15.47:0/2578359236 shutdown_connections 2024-03-21T11:42:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.293+0000 7f6b267d5740 1 -- 172.21.15.47:0/2578359236 wait complete. 2024-03-21T11:42:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.293+0000 7f6b267d5740 20 client.21018 trim_cache size 0 max 16384 2024-03-21T11:42:04.295 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:42:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.294+0000 7f6b23a00640 1 -- 172.21.15.47:0/3863008845 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6b040a2fa0 msgr2=0x7f6b040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.294+0000 7f6b23a00640 1 --2- 172.21.15.47:0/3863008845 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6b040a2fa0 0x7f6b040a5460 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c104930 tx=0x7f6b10009840 comp rx=0 tx=0).stop 2024-03-21T11:42:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.294+0000 7f6b23a00640 1 -- 172.21.15.47:0/3863008845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b1c105490 msgr2=0x7f6b1c1a1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.294+0000 7f6b23a00640 1 --2- 172.21.15.47:0/3863008845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b1c105490 0x7f6b1c1a1170 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400eb10 tx=0x7f6b1400c2d0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.294+0000 7f6b23a00640 1 -- 172.21.15.47:0/3863008845 shutdown_connections 2024-03-21T11:42:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.294+0000 7f6b23a00640 1 --2- 172.21.15.47:0/3863008845 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6b040a2fa0 0x7f6b040a5460 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.294+0000 7f6b23a00640 1 --2- 172.21.15.47:0/3863008845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b1c105490 0x7f6b1c1a1170 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.294+0000 7f6b23a00640 1 --2- 172.21.15.47:0/3863008845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b1c104ad0 0x7f6b1c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.294+0000 7f6b23a00640 1 --2- 172.21.15.47:0/3863008845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b1c1038e0 0x7f6b1c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.294+0000 7f6b23a00640 1 -- 172.21.15.47:0/3863008845 >> 172.21.15.47:0/3863008845 conn(0x7f6b1c0fee50 msgr2=0x7f6b1c107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.294+0000 7f6b23a00640 1 -- 172.21.15.47:0/3863008845 shutdown_connections 2024-03-21T11:42:04.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.294+0000 7f6b23a00640 1 -- 172.21.15.47:0/3863008845 wait complete. 2024-03-21T11:42:04.310 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:04 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/702035593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:42:04.314 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:42:04.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:42:04.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:04 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/702035593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:42:04.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:04 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/702035593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.529+0000 7f91ebe00640 1 -- 172.21.15.47:0/3168852566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91ec102050 msgr2=0x7f91ec1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.529+0000 7f91ebe00640 1 --2- 172.21.15.47:0/3168852566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91ec102050 0x7f91ec1088e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009be0 tx=0x7f91e402bef0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.530+0000 7f91ebe00640 1 -- 172.21.15.47:0/3168852566 shutdown_connections 2024-03-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.530+0000 7f91ebe00640 1 --2- 172.21.15.47:0/3168852566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91ec102050 0x7f91ec1088e0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.530+0000 7f91ebe00640 1 --2- 172.21.15.47:0/3168852566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91ec101690 0x7f91ec101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.530+0000 7f91ebe00640 1 --2- 172.21.15.47:0/3168852566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91ec1004a0 0x7f91ec1008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.530+0000 7f91ebe00640 1 -- 172.21.15.47:0/3168852566 >> 172.21.15.47:0/3168852566 conn(0x7f91ec0fba10 msgr2=0x7f91ec0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.530+0000 7f91ebe00640 1 -- 172.21.15.47:0/3168852566 shutdown_connections 2024-03-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.530+0000 7f91ebe00640 1 -- 172.21.15.47:0/3168852566 wait complete. 2024-03-21T11:42:04.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.531+0000 7f91ebe00640 1 Processor -- start 2024-03-21T11:42:04.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.531+0000 7f91ebe00640 1 -- start start 2024-03-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.531+0000 7f91ebe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91ec1004a0 0x7f91ec199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.531+0000 7f91ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91ec101690 0x7f91ec199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.531+0000 7f91ebe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91ec102050 0x7f91ec1a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.531+0000 7f91ebe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec10c180 con 0x7f91ec102050 2024-03-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.531+0000 7f91ebe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec10c000 con 0x7f91ec1004a0 2024-03-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.531+0000 7f91ebe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec10c300 con 0x7f91ec101690 2024-03-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.531+0000 7f91eaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91ec1004a0 0x7f91ec199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.531+0000 7f91eaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91ec1004a0 0x7f91ec199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57038/0 (socket says 172.21.15.47:57038) 2024-03-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.531+0000 7f91eaa00640 1 -- 172.21.15.47:0/2626833538 learned_addr learned my addr 172.21.15.47:0/2626833538 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.532+0000 7f91eb400640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91ec102050 0x7f91ec1a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.532+0000 7f91ea000640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91ec101690 0x7f91ec199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.532+0000 7f91eaa00640 1 -- 172.21.15.47:0/2626833538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91ec101690 msgr2=0x7f91ec199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.532+0000 7f91eaa00640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91ec101690 0x7f91ec199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.532+0000 7f91eaa00640 1 -- 172.21.15.47:0/2626833538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91ec102050 msgr2=0x7f91ec1a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.532+0000 7f91eaa00640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91ec102050 0x7f91ec1a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.532+0000 7f91eaa00640 1 -- 172.21.15.47:0/2626833538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ec1a16c0 con 0x7f91ec1004a0 2024-03-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.532+0000 7f91eb400640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91ec102050 0x7f91ec1a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.532+0000 7f91ea000640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91ec101690 0x7f91ec199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.532+0000 7f91eaa00640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91ec1004a0 0x7f91ec199aa0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00e9a0 tx=0x7f91dc00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.532+0000 7f91e3400640 1 -- 172.21.15.47:0/2626833538 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc00cd80 con 0x7f91ec1004a0 2024-03-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.533+0000 7f91ebe00640 1 -- 172.21.15.47:0/2626833538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ec1a19b0 con 0x7f91ec1004a0 2024-03-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.533+0000 7f91e3400640 1 -- 172.21.15.47:0/2626833538 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91dc010910 con 0x7f91ec1004a0 2024-03-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.533+0000 7f91e3400640 1 -- 172.21.15.47:0/2626833538 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc018a40 con 0x7f91ec1004a0 2024-03-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.533+0000 7f91ebe00640 1 -- 172.21.15.47:0/2626833538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ec1a1f50 con 0x7f91ec1004a0 2024-03-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.536+0000 7f91f45b6740 1 Processor -- start 2024-03-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.536+0000 7f91f45b6740 1 -- start start 2024-03-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.536+0000 7f91e3400640 1 -- 172.21.15.47:0/2626833538 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f91dc010ac0 con 0x7f91ec1004a0 2024-03-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.537+0000 7f91f45b6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd8ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.537+0000 7f91f45b6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8fbc0 0x56145cd90040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.537+0000 7f91f45b6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 0x56145cd96e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.537+0000 7f91f45b6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145ccf32c0 con 0x56145cd90580 2024-03-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.537+0000 7f91f45b6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145cc9bc90 con 0x56145cd8e9f0 2024-03-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.537+0000 7f91f45b6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145ccef360 con 0x56145cd8fbc0 2024-03-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.537+0000 7f91eaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8fbc0 0x56145cd90040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.537+0000 7f91eb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd8ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.537+0000 7f91eb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd8ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57052/0 (socket says 172.21.15.47:57052) 2024-03-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.537+0000 7f91eb400640 1 -- 172.21.15.47:0/2660457396 learned_addr learned my addr 172.21.15.47:0/2660457396 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.537+0000 7f91ea000640 1 --2- 172.21.15.47:0/2660457396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 0x56145cd96e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.538+0000 7f91eaa00640 1 -- 172.21.15.47:0/2660457396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 msgr2=0x56145cd8ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.538+0000 7f91eaa00640 1 --2- 172.21.15.47:0/2660457396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd8ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.538+0000 7f91eaa00640 1 -- 172.21.15.47:0/2660457396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 msgr2=0x56145cd96e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.538+0000 7f91eaa00640 1 --2- 172.21.15.47:0/2660457396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 0x56145cd96e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.538+0000 7f91eaa00640 1 -- 172.21.15.47:0/2660457396 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56145ccc0080 con 0x56145cd8fbc0 2024-03-21T11:42:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.538+0000 7f91ea000640 1 --2- 172.21.15.47:0/2660457396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 0x56145cd96e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.538+0000 7f91eaa00640 1 --2- 172.21.15.47:0/2660457396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8fbc0 0x56145cd90040 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0cd330 tx=0x7f91dc0d86c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=712665a8db209a29 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.538+0000 7f91e3400640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f91d00a2f20 0x7f91d00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.539+0000 7f91e3400640 1 -- 172.21.15.47:0/2626833538 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f91dc0cfbf0 con 0x7f91ec1004a0 2024-03-21T11:42:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.539+0000 7f91eb400640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f91d00a2f20 0x7f91d00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.539+0000 7f91d7e00640 1 -- 172.21.15.47:0/2660457396 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0df070 con 0x56145cd8fbc0 2024-03-21T11:42:04.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.539+0000 7f91d7e00640 1 -- 172.21.15.47:0/2660457396 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91dc09c690 con 0x56145cd8fbc0 2024-03-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.539+0000 7f91d7e00640 1 -- 172.21.15.47:0/2660457396 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc09bd60 con 0x56145cd8fbc0 2024-03-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.540+0000 7f91f45b6740 1 -- 172.21.15.47:0/2660457396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8fbc0 msgr2=0x56145cd90040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.540+0000 7f91f45b6740 1 --2- 172.21.15.47:0/2660457396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8fbc0 0x56145cd90040 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0cd330 tx=0x7f91dc0d86c0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.540+0000 7f91eb400640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f91d00a2f20 0x7f91d00a53e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f91e4004af0 tx=0x7f91e4009840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.540+0000 7f91f45b6740 1 -- 172.21.15.47:0/2660457396 shutdown_connections 2024-03-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.540+0000 7f91f45b6740 1 --2- 172.21.15.47:0/2660457396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 0x56145cd96e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.540+0000 7f91f45b6740 1 --2- 172.21.15.47:0/2660457396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8fbc0 0x56145cd90040 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.540+0000 7f91f45b6740 1 --2- 172.21.15.47:0/2660457396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd8ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.540+0000 7f91f45b6740 1 -- 172.21.15.47:0/2660457396 >> 172.21.15.47:0/2660457396 conn(0x56145cd89de0 msgr2=0x56145cd8a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.540+0000 7f91f45b6740 1 -- 172.21.15.47:0/2660457396 shutdown_connections 2024-03-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.540+0000 7f91f45b6740 1 -- 172.21.15.47:0/2660457396 wait complete. 2024-03-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.541+0000 7f91f45b6740 1 -- start start 2024-03-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.541+0000 7f91f45b6740 1 Processor -- start 2024-03-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.541+0000 7f91f45b6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8e9f0 0x56145cd98d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.541+0000 7f91f45b6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8fbc0 0x56145cd99260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.541+0000 7f91f45b6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 0x56145cda0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.541+0000 7f91f45b6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145cd97590 con 0x56145cd90580 2024-03-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.541+0000 7f91f45b6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145ccaf0d0 con 0x56145cd8fbc0 2024-03-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.541+0000 7f91f45b6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145cd97710 con 0x56145cd8e9f0 2024-03-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.541+0000 7f91f45b6740 10 client.0 _ll_register_callbacks cb 0x56145ccb3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.542+0000 7f91ea000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8fbc0 0x56145cd99260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.542+0000 7f91eaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8e9f0 0x56145cd98d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.542+0000 7f91ea000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8fbc0 0x56145cd99260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57062/0 (socket says 172.21.15.47:57062) 2024-03-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.542+0000 7f91ea000640 1 -- 172.21.15.47:0/1291715393 learned_addr learned my addr 172.21.15.47:0/1291715393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.542+0000 7f91eb400640 1 --2- 172.21.15.47:0/1291715393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 0x56145cda0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.542+0000 7f91ea000640 1 -- 172.21.15.47:0/1291715393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8e9f0 msgr2=0x56145cd98d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.542+0000 7f91ea000640 1 --2- 172.21.15.47:0/1291715393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8e9f0 0x56145cd98d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.542+0000 7f91ea000640 1 -- 172.21.15.47:0/1291715393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 msgr2=0x56145cda0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.542+0000 7f91ea000640 1 --2- 172.21.15.47:0/1291715393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 0x56145cda0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.542+0000 7f91ea000640 1 -- 172.21.15.47:0/1291715393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56145ca073f0 con 0x56145cd8fbc0 2024-03-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.543+0000 7f91ea000640 1 --2- 172.21.15.47:0/1291715393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8fbc0 0x56145cd99260 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f91d8013710 tx=0x7f91d8013bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5dcc4660a54c5d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.543+0000 7f91d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.543+0000 7f91d4c00640 1 -- 172.21.15.47:0/1291715393 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8020060 con 0x56145cd8fbc0 2024-03-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.543+0000 7f91f45b6740 1 -- 172.21.15.47:0/1291715393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91dc09c690 con 0x56145cd8fbc0 2024-03-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.543+0000 7f91d4c00640 1 -- 172.21.15.47:0/1291715393 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f91d8011dd0 con 0x56145cd8fbc0 2024-03-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.544+0000 7f91d4c00640 1 -- 172.21.15.47:0/1291715393 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91d8027050 con 0x56145cd8fbc0 2024-03-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.544+0000 7f91f45b6740 10 client.31601 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.544+0000 7f91f45b6740 1 -- 172.21.15.47:0/1291715393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56145cad17d0 con 0x56145cd8fbc0 2024-03-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.544+0000 7f91d4c00640 1 -- 172.21.15.47:0/1291715393 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91dc09c690 con 0x56145cd8fbc0 2024-03-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.544+0000 7f91d4c00640 1 -- 172.21.15.47:0/1291715393 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f91d801a030 con 0x56145cd8fbc0 2024-03-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.544+0000 7f91f45b6740 10 client.31601 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.544+0000 7f91f45b6740 10 client.31601 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.544+0000 7f91f45b6740 20 client.31601 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.544+0000 7f91f45b6740 1 --2- 172.21.15.47:0/1291715393 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56145cda4770 0x56145cda6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.544+0000 7f91f45b6740 4 client.31601 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.544+0000 7f91f45b6740 1 -- 172.21.15.47:0/1291715393 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56145cd00800 con 0x56145cda4770 2024-03-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.545+0000 7f91eaa00640 1 --2- 172.21.15.47:0/1291715393 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56145cda4770 0x56145cda6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.545+0000 7f91d4c00640 10 client.31601 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.545+0000 7f91eaa00640 1 --2- 172.21.15.47:0/1291715393 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56145cda4770 0x56145cda6bb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68491282099992cb server_cookie=9e205772c21b8dfc in_seq=0 out_seq=0 2024-03-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.548+0000 7f91d4c00640 1 -- 172.21.15.47:0/1291715393 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56145cd00800 con 0x56145cda4770 2024-03-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.548+0000 7f91d4c00640 10 client.31601 handle_command_reply: tid=0 2024-03-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.548+0000 7f91f45b6740 1 client.31601 shutdown 2024-03-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.549+0000 7f91f45b6740 1 -- 172.21.15.47:0/1291715393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8fbc0 msgr2=0x56145cd99260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.549+0000 7f91f45b6740 1 --2- 172.21.15.47:0/1291715393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8fbc0 0x56145cd99260 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f91d8013710 tx=0x7f91d8013bd0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.549+0000 7f91f45b6740 1 -- 172.21.15.47:0/1291715393 shutdown_connections 2024-03-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.549+0000 7f91f45b6740 1 --2- 172.21.15.47:0/1291715393 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56145cda4770 0x56145cda6bb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.549+0000 7f91f45b6740 1 --2- 172.21.15.47:0/1291715393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 0x56145cda0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.549+0000 7f91f45b6740 1 --2- 172.21.15.47:0/1291715393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8fbc0 0x56145cd99260 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.549+0000 7f91f45b6740 1 --2- 172.21.15.47:0/1291715393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8e9f0 0x56145cd98d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.549+0000 7f91f45b6740 1 -- 172.21.15.47:0/1291715393 >> 172.21.15.47:0/1291715393 conn(0x56145cd89de0 msgr2=0x56145cd89160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.549+0000 7f91d4c00640 0 client.31601 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.550+0000 7f91f45b6740 1 -- 172.21.15.47:0/1291715393 shutdown_connections 2024-03-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.550+0000 7f91f45b6740 1 -- 172.21.15.47:0/1291715393 wait complete. 2024-03-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.550+0000 7f91f45b6740 20 client.31601 trim_cache size 0 max 16384 2024-03-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.565+0000 7f91f45b6740 1 Processor -- start 2024-03-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.565+0000 7f91f45b6740 1 -- start start 2024-03-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.565+0000 7f91f45b6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd8d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.565+0000 7f91f45b6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd8fbc0 0x56145cda01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.565+0000 7f91f45b6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd90580 0x56145cd92c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.565+0000 7f91f45b6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec1ac1b0 con 0x56145cd8fbc0 2024-03-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.565+0000 7f91f45b6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec1adb30 con 0x56145cd8e9f0 2024-03-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.565+0000 7f91f45b6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec1ac810 con 0x56145cd90580 2024-03-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.565+0000 7f91eb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd90580 0x56145cd92c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.565+0000 7f91ea000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd8fbc0 0x56145cda01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.565+0000 7f91eaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd8d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.565+0000 7f91eb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd90580 0x56145cd92c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58500/0 (socket says 172.21.15.47:58500) 2024-03-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.565+0000 7f91eaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd8d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57074/0 (socket says 172.21.15.47:57074) 2024-03-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.565+0000 7f91eb400640 1 -- 172.21.15.47:0/2413153359 learned_addr learned my addr 172.21.15.47:0/2413153359 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.566+0000 7f91ea000640 1 -- 172.21.15.47:0/2413153359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd90580 msgr2=0x56145cd92c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.566+0000 7f91ea000640 1 --2- 172.21.15.47:0/2413153359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd90580 0x56145cd92c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.566+0000 7f91ea000640 1 -- 172.21.15.47:0/2413153359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 msgr2=0x56145cd8d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.566+0000 7f91ea000640 1 --2- 172.21.15.47:0/2413153359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd8d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.566+0000 7f91ea000640 1 -- 172.21.15.47:0/2413153359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56145cd8f640 con 0x56145cd8fbc0 2024-03-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.566+0000 7f91ea000640 1 --2- 172.21.15.47:0/2413153359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd8fbc0 0x56145cda01f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f91d800e6d0 tx=0x7f91d80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661774860369f1b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.566+0000 7f91d7400640 1 -- 172.21.15.47:0/2413153359 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8030070 con 0x56145cd8fbc0 2024-03-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.567+0000 7f91d7400640 1 -- 172.21.15.47:0/2413153359 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91d8005510 con 0x56145cd8fbc0 2024-03-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.567+0000 7f91d7400640 1 -- 172.21.15.47:0/2413153359 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8011bd0 con 0x56145cd8fbc0 2024-03-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.567+0000 7f91f45b6740 1 -- 172.21.15.47:0/2413153359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd8fbc0 msgr2=0x56145cda01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.567+0000 7f91f45b6740 1 --2- 172.21.15.47:0/2413153359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd8fbc0 0x56145cda01f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f91d800e6d0 tx=0x7f91d80117b0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.567+0000 7f91f45b6740 1 -- 172.21.15.47:0/2413153359 shutdown_connections 2024-03-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.567+0000 7f91f45b6740 1 --2- 172.21.15.47:0/2413153359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd90580 0x56145cd92c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.567+0000 7f91f45b6740 1 --2- 172.21.15.47:0/2413153359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd8fbc0 0x56145cda01f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.567+0000 7f91f45b6740 1 --2- 172.21.15.47:0/2413153359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd8d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.567+0000 7f91f45b6740 1 -- 172.21.15.47:0/2413153359 >> 172.21.15.47:0/2413153359 conn(0x56145cd89de0 msgr2=0x56145cd8d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.567+0000 7f91f45b6740 1 -- 172.21.15.47:0/2413153359 shutdown_connections 2024-03-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.567+0000 7f91f45b6740 1 -- 172.21.15.47:0/2413153359 wait complete. 2024-03-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.568+0000 7f91f45b6740 1 -- start start 2024-03-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.568+0000 7f91f45b6740 1 Processor -- start 2024-03-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.568+0000 7f91f45b6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd9b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.568+0000 7f91f45b6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8fbc0 0x56145cd9bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.568+0000 7f91f45b6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 0x56145cd9ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.568+0000 7f91f45b6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec1ad4d0 con 0x56145cd90580 2024-03-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.568+0000 7f91f45b6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec1ace70 con 0x56145cd8e9f0 2024-03-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.568+0000 7f91f45b6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec1aa370 con 0x56145cd8fbc0 2024-03-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.568+0000 7f91f45b6740 10 client.0 _ll_register_callbacks cb 0x7f91d80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.569+0000 7f91eaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd9b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.569+0000 7f91eaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd9b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57086/0 (socket says 172.21.15.47:57086) 2024-03-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.569+0000 7f91eaa00640 1 -- 172.21.15.47:0/3768290866 learned_addr learned my addr 172.21.15.47:0/3768290866 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.569+0000 7f91ea000640 1 --2- 172.21.15.47:0/3768290866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8fbc0 0x56145cd9bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.569+0000 7f91eb400640 1 --2- 172.21.15.47:0/3768290866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 0x56145cd9ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.569+0000 7f91eaa00640 1 -- 172.21.15.47:0/3768290866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8fbc0 msgr2=0x56145cd9bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.569+0000 7f91eaa00640 1 --2- 172.21.15.47:0/3768290866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8fbc0 0x56145cd9bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.569+0000 7f91eaa00640 1 -- 172.21.15.47:0/3768290866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 msgr2=0x56145cd9ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.569+0000 7f91eaa00640 1 --2- 172.21.15.47:0/3768290866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 0x56145cd9ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.569+0000 7f91eaa00640 1 -- 172.21.15.47:0/3768290866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56145cd9f520 con 0x56145cd8e9f0 2024-03-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.569+0000 7f91eb400640 1 --2- 172.21.15.47:0/3768290866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 0x56145cd9ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.569+0000 7f91eaa00640 1 --2- 172.21.15.47:0/3768290866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd9b9d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f91dc098960 tx=0x7f91dc00cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9322e5d4e3b4d10a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.569+0000 7f91d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.570+0000 7f91d4c00640 1 -- 172.21.15.47:0/3768290866 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0df040 con 0x56145cd8e9f0 2024-03-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.570+0000 7f91d4c00640 1 -- 172.21.15.47:0/3768290866 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f91dc09a290 con 0x56145cd8e9f0 2024-03-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.570+0000 7f91f45b6740 1 -- 172.21.15.47:0/3768290866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91d8005510 con 0x56145cd8e9f0 2024-03-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.570+0000 7f91d4c00640 1 -- 172.21.15.47:0/3768290866 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91dc09b8f0 con 0x56145cd8e9f0 2024-03-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.570+0000 7f91f45b6740 10 client.31607 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.570+0000 7f91f45b6740 1 -- 172.21.15.47:0/3768290866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56145ce00100 con 0x56145cd8e9f0 2024-03-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.571+0000 7f91d4c00640 1 -- 172.21.15.47:0/3768290866 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91d8005510 con 0x56145cd8e9f0 2024-03-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.571+0000 7f91d4c00640 1 -- 172.21.15.47:0/3768290866 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f91dc0d9040 con 0x56145cd8e9f0 2024-03-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.571+0000 7f91f45b6740 10 client.31607 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.571+0000 7f91f45b6740 10 client.31607 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.571+0000 7f91f45b6740 20 client.31607 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.571+0000 7f91f45b6740 1 --2- 172.21.15.47:0/3768290866 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56145ce00610 0x56145ce02a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.571+0000 7f91f45b6740 4 client.31607 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.571+0000 7f91f45b6740 1 -- 172.21.15.47:0/3768290866 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56145ccfab50 con 0x56145ce00610 2024-03-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.572+0000 7f91ea000640 1 --2- 172.21.15.47:0/3768290866 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56145ce00610 0x56145ce02a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.572+0000 7f91ea000640 1 --2- 172.21.15.47:0/3768290866 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56145ce00610 0x56145ce02a00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6e51479d7b5c62f server_cookie=315e6acf95f3c46b in_seq=0 out_seq=0 2024-03-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.572+0000 7f91d4c00640 10 client.31607 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.573+0000 7f91d4c00640 1 -- 172.21.15.47:0/3768290866 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56145ccfab50 con 0x56145ce00610 2024-03-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.573+0000 7f91d4c00640 10 client.31607 handle_command_reply: tid=0 2024-03-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.573+0000 7f91f45b6740 1 client.31607 shutdown 2024-03-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.574+0000 7f91f45b6740 1 -- 172.21.15.47:0/3768290866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 msgr2=0x56145cd9b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.574+0000 7f91f45b6740 1 --2- 172.21.15.47:0/3768290866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd9b9d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f91dc098960 tx=0x7f91dc00cf20 comp rx=0 tx=0).stop 2024-03-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.574+0000 7f91f45b6740 1 -- 172.21.15.47:0/3768290866 shutdown_connections 2024-03-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.574+0000 7f91f45b6740 1 --2- 172.21.15.47:0/3768290866 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56145ce00610 0x56145ce02a00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.574+0000 7f91f45b6740 1 --2- 172.21.15.47:0/3768290866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56145cd90580 0x56145cd9ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.574+0000 7f91f45b6740 1 --2- 172.21.15.47:0/3768290866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56145cd8fbc0 0x56145cd9bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.574+0000 7f91d4c00640 0 client.31607 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.574+0000 7f91f45b6740 1 --2- 172.21.15.47:0/3768290866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56145cd8e9f0 0x56145cd9b9d0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.574+0000 7f91f45b6740 1 -- 172.21.15.47:0/3768290866 >> 172.21.15.47:0/3768290866 conn(0x56145cd89de0 msgr2=0x56145cd990c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.574+0000 7f91f45b6740 1 -- 172.21.15.47:0/3768290866 shutdown_connections 2024-03-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.574+0000 7f91f45b6740 1 -- 172.21.15.47:0/3768290866 wait complete. 2024-03-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.574+0000 7f91f45b6740 20 client.31607 trim_cache size 0 max 16384 2024-03-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.575+0000 7f91ebe00640 1 -- 172.21.15.47:0/2626833538 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f91d00a2f20 msgr2=0x7f91d00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.575+0000 7f91ebe00640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f91d00a2f20 0x7f91d00a53e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f91e4004af0 tx=0x7f91e4009840 comp rx=0 tx=0).stop 2024-03-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.575+0000 7f91ebe00640 1 -- 172.21.15.47:0/2626833538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91ec1004a0 msgr2=0x7f91ec199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.575+0000 7f91ebe00640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91ec1004a0 0x7f91ec199aa0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00e9a0 tx=0x7f91dc00ee60 comp rx=0 tx=0).stop 2024-03-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.575+0000 7f91ebe00640 1 -- 172.21.15.47:0/2626833538 shutdown_connections 2024-03-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.575+0000 7f91ebe00640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f91d00a2f20 0x7f91d00a53e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.575+0000 7f91ebe00640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91ec102050 0x7f91ec1a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.575+0000 7f91ebe00640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91ec101690 0x7f91ec199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.575+0000 7f91ebe00640 1 --2- 172.21.15.47:0/2626833538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91ec1004a0 0x7f91ec199aa0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.575+0000 7f91ebe00640 1 -- 172.21.15.47:0/2626833538 >> 172.21.15.47:0/2626833538 conn(0x7f91ec0fba10 msgr2=0x7f91ec0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.575+0000 7f91ebe00640 1 -- 172.21.15.47:0/2626833538 shutdown_connections 2024-03-21T11:42:04.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.575+0000 7f91ebe00640 1 -- 172.21.15.47:0/2626833538 wait complete. 2024-03-21T11:42:04.592 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:42:04.592 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:42:04.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.828+0000 7fab49e00640 1 -- 172.21.15.47:0/1937692730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab441086a0 msgr2=0x7fab4410aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.828+0000 7fab49e00640 1 --2- 172.21.15.47:0/1937692730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab441086a0 0x7fab4410aad0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fab34009900 tx=0x7fab3402bc40 comp rx=0 tx=0).stop 2024-03-21T11:42:04.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.828+0000 7fab49e00640 1 -- 172.21.15.47:0/1937692730 shutdown_connections 2024-03-21T11:42:04.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.828+0000 7fab49e00640 1 --2- 172.21.15.47:0/1937692730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab4410b490 0x7fab44114330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.828+0000 7fab49e00640 1 --2- 172.21.15.47:0/1937692730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab4410b010 0x7fab44111de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.828+0000 7fab49e00640 1 --2- 172.21.15.47:0/1937692730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab441086a0 0x7fab4410aad0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.828+0000 7fab49e00640 1 -- 172.21.15.47:0/1937692730 >> 172.21.15.47:0/1937692730 conn(0x7fab4406e8a0 msgr2=0x7fab44070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.828+0000 7fab49e00640 1 -- 172.21.15.47:0/1937692730 shutdown_connections 2024-03-21T11:42:04.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.828+0000 7fab49e00640 1 -- 172.21.15.47:0/1937692730 wait complete. 2024-03-21T11:42:04.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.829+0000 7fab49e00640 1 Processor -- start 2024-03-21T11:42:04.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.829+0000 7fab49e00640 1 -- start start 2024-03-21T11:42:04.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.829+0000 7fab49e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab441086a0 0x7fab441a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.829+0000 7fab49e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab4410b010 0x7fab441a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.829+0000 7fab49e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab4410b490 0x7fab441a9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.829+0000 7fab49e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44117d80 con 0x7fab4410b490 2024-03-21T11:42:04.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.829+0000 7fab49e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44117c00 con 0x7fab4410b010 2024-03-21T11:42:04.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.829+0000 7fab49e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44117f00 con 0x7fab441086a0 2024-03-21T11:42:04.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.829+0000 7fab42000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab4410b010 0x7fab441a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.830+0000 7fab42a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab441086a0 0x7fab441a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.830+0000 7fab42000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab4410b010 0x7fab441a29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57100/0 (socket says 172.21.15.47:57100) 2024-03-21T11:42:04.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.830+0000 7fab42000640 1 -- 172.21.15.47:0/2646867433 learned_addr learned my addr 172.21.15.47:0/2646867433 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.830+0000 7fab43400640 1 --2- 172.21.15.47:0/2646867433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab4410b490 0x7fab441a9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.830+0000 7fab42000640 1 -- 172.21.15.47:0/2646867433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab441086a0 msgr2=0x7fab441a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.830+0000 7fab42000640 1 --2- 172.21.15.47:0/2646867433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab441086a0 0x7fab441a24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.830+0000 7fab42000640 1 -- 172.21.15.47:0/2646867433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab4410b490 msgr2=0x7fab441a9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.830+0000 7fab42000640 1 --2- 172.21.15.47:0/2646867433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab4410b490 0x7fab441a9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.830+0000 7fab42000640 1 -- 172.21.15.47:0/2646867433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab441aa0c0 con 0x7fab4410b010 2024-03-21T11:42:04.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.830+0000 7fab43400640 1 --2- 172.21.15.47:0/2646867433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab4410b490 0x7fab441a9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:04.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.831+0000 7fab42000640 1 --2- 172.21.15.47:0/2646867433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab4410b010 0x7fab441a29e0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fab3000b4a0 tx=0x7fab3000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.831+0000 7fab48a00640 1 -- 172.21.15.47:0/2646867433 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30015020 con 0x7fab4410b010 2024-03-21T11:42:04.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.831+0000 7fab48a00640 1 -- 172.21.15.47:0/2646867433 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab30011d10 con 0x7fab4410b010 2024-03-21T11:42:04.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.831+0000 7fab48a00640 1 -- 172.21.15.47:0/2646867433 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3000f2e0 con 0x7fab4410b010 2024-03-21T11:42:04.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.831+0000 7fab49e00640 1 -- 172.21.15.47:0/2646867433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab441aa3b0 con 0x7fab4410b010 2024-03-21T11:42:04.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.831+0000 7fab49e00640 1 -- 172.21.15.47:0/2646867433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab441aa8b0 con 0x7fab4410b010 2024-03-21T11:42:04.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.836+0000 7fab4ca03740 1 Processor -- start 2024-03-21T11:42:04.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.836+0000 7fab4ca03740 1 -- start start 2024-03-21T11:42:04.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.837+0000 7fab4ca03740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e4020 0x562d8c6e4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.837+0000 7fab4ca03740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e51f0 0x562d8c6e5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.837+0000 7fab4ca03740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e5bb0 0x562d8c6ec440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.837+0000 7fab4ca03740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8c6402c0 con 0x562d8c6e5bb0 2024-03-21T11:42:04.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.837+0000 7fab4ca03740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8c5e8c90 con 0x562d8c6e51f0 2024-03-21T11:42:04.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.837+0000 7fab4ca03740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8c63c360 con 0x562d8c6e4020 2024-03-21T11:42:04.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.837+0000 7fab42a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e51f0 0x562d8c6e5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.837+0000 7fab42a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e51f0 0x562d8c6e5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57104/0 (socket says 172.21.15.47:57104) 2024-03-21T11:42:04.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.837+0000 7fab43400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e4020 0x562d8c6e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.837+0000 7fab42a00640 1 -- 172.21.15.47:0/1701667169 learned_addr learned my addr 172.21.15.47:0/1701667169 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.838+0000 7fab43400640 1 -- 172.21.15.47:0/1701667169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e51f0 msgr2=0x562d8c6e5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.838+0000 7fab43400640 1 --2- 172.21.15.47:0/1701667169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e51f0 0x562d8c6e5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.838+0000 7fab43400640 1 -- 172.21.15.47:0/1701667169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e5bb0 msgr2=0x562d8c6ec440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:42:04.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.838+0000 7fab43400640 1 --2- 172.21.15.47:0/1701667169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e5bb0 0x562d8c6ec440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.838+0000 7fab43400640 1 -- 172.21.15.47:0/1701667169 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d8c60d080 con 0x562d8c6e4020 2024-03-21T11:42:04.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.838+0000 7fab43400640 1 --2- 172.21.15.47:0/1701667169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e4020 0x562d8c6e4440 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fab3c011760 tx=0x7fab3c011c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6168bec4c2191e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.839+0000 7fab2fe00640 1 -- 172.21.15.47:0/1701667169 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c004d20 con 0x562d8c6e4020 2024-03-21T11:42:04.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.839+0000 7fab2fe00640 1 -- 172.21.15.47:0/1701667169 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab3c003970 con 0x562d8c6e4020 2024-03-21T11:42:04.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.839+0000 7fab2fe00640 1 -- 172.21.15.47:0/1701667169 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c004670 con 0x562d8c6e4020 2024-03-21T11:42:04.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.840+0000 7fab4ca03740 1 -- 172.21.15.47:0/1701667169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e4020 msgr2=0x562d8c6e4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.840+0000 7fab4ca03740 1 --2- 172.21.15.47:0/1701667169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e4020 0x562d8c6e4440 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fab3c011760 tx=0x7fab3c011c20 comp rx=0 tx=0).stop 2024-03-21T11:42:04.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.841+0000 7fab4ca03740 1 -- 172.21.15.47:0/1701667169 shutdown_connections 2024-03-21T11:42:04.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.841+0000 7fab4ca03740 1 --2- 172.21.15.47:0/1701667169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e5bb0 0x562d8c6ec440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.841+0000 7fab4ca03740 1 --2- 172.21.15.47:0/1701667169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e51f0 0x562d8c6e5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.841+0000 7fab48a00640 1 -- 172.21.15.47:0/2646867433 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fab30018020 con 0x7fab4410b010 2024-03-21T11:42:04.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.841+0000 7fab4ca03740 1 --2- 172.21.15.47:0/1701667169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e4020 0x562d8c6e4440 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.841+0000 7fab4ca03740 1 -- 172.21.15.47:0/1701667169 >> 172.21.15.47:0/1701667169 conn(0x562d8c6d6c00 msgr2=0x562d8c6d7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.842+0000 7fab4ca03740 1 -- 172.21.15.47:0/1701667169 shutdown_connections 2024-03-21T11:42:04.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.842+0000 7fab4ca03740 1 -- 172.21.15.47:0/1701667169 wait complete. 2024-03-21T11:42:04.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.843+0000 7fab4ca03740 1 -- start start 2024-03-21T11:42:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.843+0000 7fab4ca03740 1 Processor -- start 2024-03-21T11:42:04.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.844+0000 7fab48a00640 1 --2- 172.21.15.47:0/2646867433 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab280a2e30 0x7fab280a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.844+0000 7fab4ca03740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e4020 0x562d8c6ee320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.844+0000 7fab4ca03740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e51f0 0x562d8c6ee860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.844+0000 7fab4ca03740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e5bb0 0x562d8c6f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.844+0000 7fab4ca03740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8c6ecbc0 con 0x562d8c6e5bb0 2024-03-21T11:42:04.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.844+0000 7fab4ca03740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8c5fc0d0 con 0x562d8c6e51f0 2024-03-21T11:42:04.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.844+0000 7fab4ca03740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8c6ecd40 con 0x562d8c6e4020 2024-03-21T11:42:04.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.844+0000 7fab48a00640 1 -- 172.21.15.47:0/2646867433 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fab300cef90 con 0x7fab4410b010 2024-03-21T11:42:04.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.845+0000 7fab4ca03740 10 client.0 _ll_register_callbacks cb 0x562d8c600e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:04.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.845+0000 7fab42000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e51f0 0x562d8c6ee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.845+0000 7fab42000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e51f0 0x562d8c6ee860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57108/0 (socket says 172.21.15.47:57108) 2024-03-21T11:42:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.845+0000 7fab42000640 1 -- 172.21.15.47:0/2077310594 learned_addr learned my addr 172.21.15.47:0/2077310594 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.845+0000 7fab42a00640 1 --2- 172.21.15.47:0/2077310594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e4020 0x562d8c6ee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.846+0000 7fab42000640 1 -- 172.21.15.47:0/2077310594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e4020 msgr2=0x562d8c6ee320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.846+0000 7fab42000640 1 --2- 172.21.15.47:0/2077310594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e4020 0x562d8c6ee320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.846+0000 7fab42000640 1 -- 172.21.15.47:0/2077310594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e5bb0 msgr2=0x562d8c6f58a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:42:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.846+0000 7fab42000640 1 --2- 172.21.15.47:0/2077310594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e5bb0 0x562d8c6f58a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.846+0000 7fab42000640 1 -- 172.21.15.47:0/2077310594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d8c3543f0 con 0x562d8c6e51f0 2024-03-21T11:42:04.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.846+0000 7fab43400640 1 --2- 172.21.15.47:0/2646867433 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab280a2e30 0x7fab280a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.847+0000 7fab42000640 1 --2- 172.21.15.47:0/2077310594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e51f0 0x562d8c6ee860 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fab3009b030 tx=0x7fab3009ac90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e68f7c90123dbf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.847+0000 7fab2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:04.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.848+0000 7fab2cc00640 1 -- 172.21.15.47:0/2077310594 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab300ae180 con 0x562d8c6e51f0 2024-03-21T11:42:04.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.848+0000 7fab2cc00640 1 -- 172.21.15.47:0/2077310594 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fab30010510 con 0x562d8c6e51f0 2024-03-21T11:42:04.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.848+0000 7fab43400640 1 --2- 172.21.15.47:0/2646867433 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab280a2e30 0x7fab280a52f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0115a0 tx=0x7fab3c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.848+0000 7fab4ca03740 1 -- 172.21.15.47:0/2077310594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab3c003970 con 0x562d8c6e51f0 2024-03-21T11:42:04.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.849+0000 7fab2cc00640 1 -- 172.21.15.47:0/2077310594 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab300092b0 con 0x562d8c6e51f0 2024-03-21T11:42:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.849+0000 7fab4ca03740 10 client.31619 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.849+0000 7fab4ca03740 1 -- 172.21.15.47:0/2077310594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562d8c41e7d0 con 0x562d8c6e51f0 2024-03-21T11:42:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.849+0000 7fab2cc00640 1 -- 172.21.15.47:0/2077310594 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab3c003970 con 0x562d8c6e51f0 2024-03-21T11:42:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.849+0000 7fab2cc00640 1 -- 172.21.15.47:0/2077310594 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fab300979a0 con 0x562d8c6e51f0 2024-03-21T11:42:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.849+0000 7fab4ca03740 10 client.31619 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.849+0000 7fab4ca03740 10 client.31619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.849+0000 7fab4ca03740 20 client.31619 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.849+0000 7fab4ca03740 1 --2- 172.21.15.47:0/2077310594 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562d8c6f9cc0 0x562d8c6fc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.849+0000 7fab4ca03740 4 client.31619 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.849+0000 7fab4ca03740 1 -- 172.21.15.47:0/2077310594 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d8c64d800 con 0x562d8c6f9cc0 2024-03-21T11:42:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.850+0000 7fab42a00640 1 --2- 172.21.15.47:0/2077310594 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562d8c6f9cc0 0x562d8c6fc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.852+0000 7fab42a00640 1 --2- 172.21.15.47:0/2077310594 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562d8c6f9cc0 0x562d8c6fc0b0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6bc97d1b437d5df server_cookie=8487cd3c4d495ad4 in_seq=0 out_seq=0 2024-03-21T11:42:04.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.852+0000 7fab2cc00640 10 client.31619 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:04.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.856+0000 7fab2cc00640 1 -- 172.21.15.47:0/2077310594 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562d8c64d800 con 0x562d8c6f9cc0 2024-03-21T11:42:04.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.856+0000 7fab2cc00640 10 client.31619 handle_command_reply: tid=0 2024-03-21T11:42:04.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.856+0000 7fab4ca03740 1 client.31619 shutdown 2024-03-21T11:42:04.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.856+0000 7fab4ca03740 1 -- 172.21.15.47:0/2077310594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e51f0 msgr2=0x562d8c6ee860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.856+0000 7fab4ca03740 1 --2- 172.21.15.47:0/2077310594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e51f0 0x562d8c6ee860 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fab3009b030 tx=0x7fab3009ac90 comp rx=0 tx=0).stop 2024-03-21T11:42:04.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.856+0000 7fab4ca03740 1 -- 172.21.15.47:0/2077310594 shutdown_connections 2024-03-21T11:42:04.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.856+0000 7fab4ca03740 1 --2- 172.21.15.47:0/2077310594 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562d8c6f9cc0 0x562d8c6fc0b0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.856+0000 7fab4ca03740 1 --2- 172.21.15.47:0/2077310594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e5bb0 0x562d8c6f58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.856+0000 7fab4ca03740 1 --2- 172.21.15.47:0/2077310594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e51f0 0x562d8c6ee860 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.856+0000 7fab4ca03740 1 --2- 172.21.15.47:0/2077310594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e4020 0x562d8c6ee320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.856+0000 7fab4ca03740 1 -- 172.21.15.47:0/2077310594 >> 172.21.15.47:0/2077310594 conn(0x562d8c6d6c00 msgr2=0x562d8c6d5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.856+0000 7fab2cc00640 0 client.31619 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:04.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.857+0000 7fab4ca03740 1 -- 172.21.15.47:0/2077310594 shutdown_connections 2024-03-21T11:42:04.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.857+0000 7fab4ca03740 1 -- 172.21.15.47:0/2077310594 wait complete. 2024-03-21T11:42:04.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.857+0000 7fab4ca03740 20 client.31619 trim_cache size 0 max 16384 2024-03-21T11:42:04.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.909+0000 7fab4ca03740 1 Processor -- start 2024-03-21T11:42:04.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.909+0000 7fab4ca03740 1 -- start start 2024-03-21T11:42:04.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.910+0000 7fab4ca03740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e4020 0x562d8c6da570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.910+0000 7fab4ca03740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e51f0 0x562d8c6f5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.910+0000 7fab4ca03740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e5bb0 0x562d8c6e8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.910+0000 7fab4ca03740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab441b08c0 con 0x562d8c6e4020 2024-03-21T11:42:04.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.910+0000 7fab4ca03740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab441b2240 con 0x562d8c6e5bb0 2024-03-21T11:42:04.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.910+0000 7fab4ca03740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab441b0f20 con 0x562d8c6e51f0 2024-03-21T11:42:04.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.911+0000 7fab42000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e51f0 0x562d8c6f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.911+0000 7fab43400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e5bb0 0x562d8c6e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.911+0000 7fab43400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e5bb0 0x562d8c6e8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57116/0 (socket says 172.21.15.47:57116) 2024-03-21T11:42:04.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.911+0000 7fab43400640 1 -- 172.21.15.47:0/3031623910 learned_addr learned my addr 172.21.15.47:0/3031623910 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.911+0000 7fab42a00640 1 --2- 172.21.15.47:0/3031623910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e4020 0x562d8c6da570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.912+0000 7fab43400640 1 -- 172.21.15.47:0/3031623910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e51f0 msgr2=0x562d8c6f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.912+0000 7fab43400640 1 --2- 172.21.15.47:0/3031623910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e51f0 0x562d8c6f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.912+0000 7fab43400640 1 -- 172.21.15.47:0/3031623910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e4020 msgr2=0x562d8c6da570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.912+0000 7fab43400640 1 --2- 172.21.15.47:0/3031623910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e4020 0x562d8c6da570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.912+0000 7fab43400640 1 -- 172.21.15.47:0/3031623910 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d8c6d7670 con 0x562d8c6e5bb0 2024-03-21T11:42:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.912+0000 7fab42000640 1 --2- 172.21.15.47:0/3031623910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e51f0 0x562d8c6f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:04.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.912+0000 7fab42a00640 1 --2- 172.21.15.47:0/3031623910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e4020 0x562d8c6da570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:04.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.914+0000 7fab43400640 1 --2- 172.21.15.47:0/3031623910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e5bb0 0x562d8c6e8290 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fab3c009970 tx=0x7fab3c022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5582a5211244fcd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.918+0000 7fab2f400640 1 -- 172.21.15.47:0/3031623910 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c035070 con 0x562d8c6e5bb0 2024-03-21T11:42:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.919+0000 7fab2f400640 1 -- 172.21.15.47:0/3031623910 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab3c02c040 con 0x562d8c6e5bb0 2024-03-21T11:42:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.920+0000 7fab2f400640 1 -- 172.21.15.47:0/3031623910 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c02b050 con 0x562d8c6e5bb0 2024-03-21T11:42:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.921+0000 7fab4ca03740 1 -- 172.21.15.47:0/3031623910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e5bb0 msgr2=0x562d8c6e8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.921+0000 7fab4ca03740 1 --2- 172.21.15.47:0/3031623910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e5bb0 0x562d8c6e8290 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fab3c009970 tx=0x7fab3c022000 comp rx=0 tx=0).stop 2024-03-21T11:42:04.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.921+0000 7fab4ca03740 1 -- 172.21.15.47:0/3031623910 shutdown_connections 2024-03-21T11:42:04.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.921+0000 7fab4ca03740 1 --2- 172.21.15.47:0/3031623910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e5bb0 0x562d8c6e8290 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.921+0000 7fab4ca03740 1 --2- 172.21.15.47:0/3031623910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e51f0 0x562d8c6f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.921+0000 7fab4ca03740 1 --2- 172.21.15.47:0/3031623910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e4020 0x562d8c6da570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.921+0000 7fab4ca03740 1 -- 172.21.15.47:0/3031623910 >> 172.21.15.47:0/3031623910 conn(0x562d8c6d6c00 msgr2=0x562d8c6d9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.923+0000 7fab4ca03740 1 -- 172.21.15.47:0/3031623910 shutdown_connections 2024-03-21T11:42:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.923+0000 7fab4ca03740 1 -- 172.21.15.47:0/3031623910 wait complete. 2024-03-21T11:42:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.923+0000 7fab4ca03740 1 -- start start 2024-03-21T11:42:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.924+0000 7fab4ca03740 1 Processor -- start 2024-03-21T11:42:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.924+0000 7fab4ca03740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e4020 0x562d8c6f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.924+0000 7fab4ca03740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e51f0 0x562d8c6f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.924+0000 7fab4ca03740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e5bb0 0x562d8c6f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.925+0000 7fab43400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e5bb0 0x562d8c6f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.925+0000 7fab43400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e5bb0 0x562d8c6f44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57126/0 (socket says 172.21.15.47:57126) 2024-03-21T11:42:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.925+0000 7fab43400640 1 -- 172.21.15.47:0/1623679074 learned_addr learned my addr 172.21.15.47:0/1623679074 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.924+0000 7fab4ca03740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab441b1be0 con 0x562d8c6e4020 2024-03-21T11:42:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.925+0000 7fab4ca03740 1 -- 172.21.15.47:0/1623679074 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab441b1580 con 0x562d8c6e5bb0 2024-03-21T11:42:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.925+0000 7fab4ca03740 1 -- 172.21.15.47:0/1623679074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab441aeaa0 con 0x562d8c6e51f0 2024-03-21T11:42:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.925+0000 7fab4ca03740 10 client.0 _ll_register_callbacks cb 0x7fab3009b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.926+0000 7fab42000640 1 --2- 172.21.15.47:0/1623679074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e51f0 0x562d8c6f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.926+0000 7fab42a00640 1 --2- 172.21.15.47:0/1623679074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e4020 0x562d8c6f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.928+0000 7fab42a00640 1 -- 172.21.15.47:0/1623679074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e51f0 msgr2=0x562d8c6f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.928+0000 7fab42a00640 1 --2- 172.21.15.47:0/1623679074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e51f0 0x562d8c6f14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.928+0000 7fab42a00640 1 -- 172.21.15.47:0/1623679074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e5bb0 msgr2=0x562d8c6f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.928+0000 7fab42a00640 1 --2- 172.21.15.47:0/1623679074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e5bb0 0x562d8c6f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.928+0000 7fab42a00640 1 -- 172.21.15.47:0/1623679074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d8c6f4ab0 con 0x562d8c6e4020 2024-03-21T11:42:04.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.929+0000 7fab42a00640 1 --2- 172.21.15.47:0/1623679074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e4020 0x562d8c6f0f60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x562d8c6d7b70 tx=0x7fab34005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c618bd3e6c010b41 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:04.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.929+0000 7fab2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:42:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.931+0000 7fab2cc00640 1 -- 172.21.15.47:0/1623679074 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34004450 con 0x562d8c6e4020 2024-03-21T11:42:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.932+0000 7fab2cc00640 1 -- 172.21.15.47:0/1623679074 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fab34005000 con 0x562d8c6e4020 2024-03-21T11:42:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.932+0000 7fab4ca03740 1 -- 172.21.15.47:0/1623679074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab3c02c040 con 0x562d8c6e4020 2024-03-21T11:42:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.932+0000 7fab2cc00640 1 -- 172.21.15.47:0/1623679074 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab34055050 con 0x562d8c6e4020 2024-03-21T11:42:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.932+0000 7fab4ca03740 10 client.21045 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.932+0000 7fab4ca03740 1 -- 172.21.15.47:0/1623679074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562d8c755560 con 0x562d8c6e4020 2024-03-21T11:42:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.933+0000 7fab2cc00640 1 -- 172.21.15.47:0/1623679074 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab3c02c040 con 0x562d8c6e4020 2024-03-21T11:42:04.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.934+0000 7fab2cc00640 1 -- 172.21.15.47:0/1623679074 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fab340035f0 con 0x562d8c6e4020 2024-03-21T11:42:04.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.934+0000 7fab4ca03740 10 client.21045 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:04.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.934+0000 7fab4ca03740 10 client.21045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:04.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.934+0000 7fab4ca03740 20 client.21045 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:04.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.934+0000 7fab4ca03740 1 --2- 172.21.15.47:0/1623679074 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562d8c755f10 0x562d8c758330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:04.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.934+0000 7fab4ca03740 4 client.21045 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:42:04.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.934+0000 7fab4ca03740 1 -- 172.21.15.47:0/1623679074 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562d8c647b50 con 0x562d8c755f10 2024-03-21T11:42:04.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.934+0000 7fab42000640 1 --2- 172.21.15.47:0/1623679074 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562d8c755f10 0x562d8c758330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:04.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.935+0000 7fab2cc00640 10 client.21045 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:04.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.936+0000 7fab42000640 1 --2- 172.21.15.47:0/1623679074 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562d8c755f10 0x562d8c758330 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4572d25854d52345 server_cookie=818c2bde2cf99899 in_seq=0 out_seq=0 2024-03-21T11:42:04.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.937+0000 7fab2cc00640 1 -- 172.21.15.47:0/1623679074 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562d8c647b50 con 0x562d8c755f10 2024-03-21T11:42:04.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.937+0000 7fab2cc00640 10 client.21045 handle_command_reply: tid=0 2024-03-21T11:42:04.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.937+0000 7fab4ca03740 1 client.21045 shutdown 2024-03-21T11:42:04.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.937+0000 7fab4ca03740 1 -- 172.21.15.47:0/1623679074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e4020 msgr2=0x562d8c6f0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.937+0000 7fab4ca03740 1 --2- 172.21.15.47:0/1623679074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e4020 0x562d8c6f0f60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x562d8c6d7b70 tx=0x7fab34005fd0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.937+0000 7fab4ca03740 1 -- 172.21.15.47:0/1623679074 shutdown_connections 2024-03-21T11:42:04.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.937+0000 7fab4ca03740 1 --2- 172.21.15.47:0/1623679074 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562d8c755f10 0x562d8c758330 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.937+0000 7fab4ca03740 1 --2- 172.21.15.47:0/1623679074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8c6e5bb0 0x562d8c6f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.937+0000 7fab4ca03740 1 --2- 172.21.15.47:0/1623679074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d8c6e51f0 0x562d8c6f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.937+0000 7fab2cc00640 0 client.21045 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:04.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.937+0000 7fab4ca03740 1 --2- 172.21.15.47:0/1623679074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562d8c6e4020 0x562d8c6f0f60 secure :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0x562d8c6d7b70 tx=0x7fab34005fd0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.937+0000 7fab4ca03740 1 -- 172.21.15.47:0/1623679074 >> 172.21.15.47:0/1623679074 conn(0x562d8c6d6c00 msgr2=0x562d8c6da6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.938+0000 7fab4ca03740 1 -- 172.21.15.47:0/1623679074 shutdown_connections 2024-03-21T11:42:04.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.938+0000 7fab4ca03740 1 -- 172.21.15.47:0/1623679074 wait complete. 2024-03-21T11:42:04.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.938+0000 7fab4ca03740 20 client.21045 trim_cache size 0 max 16384 2024-03-21T11:42:04.941 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:42:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.940+0000 7fab49e00640 1 -- 172.21.15.47:0/2646867433 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab280a2e30 msgr2=0x7fab280a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.940+0000 7fab49e00640 1 --2- 172.21.15.47:0/2646867433 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab280a2e30 0x7fab280a52f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0115a0 tx=0x7fab3c005fd0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.940+0000 7fab49e00640 1 -- 172.21.15.47:0/2646867433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab4410b010 msgr2=0x7fab441a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:04.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.940+0000 7fab49e00640 1 --2- 172.21.15.47:0/2646867433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab4410b010 0x7fab441a29e0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fab3000b4a0 tx=0x7fab3000b8a0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.941+0000 7fab49e00640 1 -- 172.21.15.47:0/2646867433 shutdown_connections 2024-03-21T11:42:04.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.941+0000 7fab49e00640 1 --2- 172.21.15.47:0/2646867433 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fab280a2e30 0x7fab280a52f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.941+0000 7fab49e00640 1 --2- 172.21.15.47:0/2646867433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab4410b490 0x7fab441a9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.941+0000 7fab49e00640 1 --2- 172.21.15.47:0/2646867433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab4410b010 0x7fab441a29e0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.941+0000 7fab49e00640 1 --2- 172.21.15.47:0/2646867433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab441086a0 0x7fab441a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:04.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.941+0000 7fab49e00640 1 -- 172.21.15.47:0/2646867433 >> 172.21.15.47:0/2646867433 conn(0x7fab4406e8a0 msgr2=0x7fab4406f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:04.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.943+0000 7fab49e00640 1 -- 172.21.15.47:0/2646867433 shutdown_connections 2024-03-21T11:42:04.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:04.943+0000 7fab49e00640 1 -- 172.21.15.47:0/2646867433 wait complete. 2024-03-21T11:42:04.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 132321 23.72 MB/sec execute 168 sec latency 105.454 ms 2024-03-21T11:42:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:05 smithi047 ceph-mon[31351]: pgmap v962: 129 pgs: 129 active+clean; 502 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 597 op/s 2024-03-21T11:42:05.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:05 smithi100 ceph-mon[36245]: pgmap v962: 129 pgs: 129 active+clean; 502 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 597 op/s 2024-03-21T11:42:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:05 smithi102 ceph-mon[36748]: pgmap v962: 129 pgs: 129 active+clean; 502 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.2 MiB/s wr, 597 op/s 2024-03-21T11:42:05.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 132581 23.69 MB/sec execute 169 sec latency 347.126 ms 2024-03-21T11:42:05.980 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:42:05.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:42:05.981 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:42:06.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.322+0000 7f07f4c00640 1 -- 172.21.15.47:0/445608767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07f0067c00 msgr2=0x7f07f0101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.322+0000 7f07f4c00640 1 --2- 172.21.15.47:0/445608767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07f0067c00 0x7f07f0101370 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009900 tx=0x7f07dc02bcb0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.322+0000 7f07f4c00640 1 -- 172.21.15.47:0/445608767 shutdown_connections 2024-03-21T11:42:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.322+0000 7f07f4c00640 1 --2- 172.21.15.47:0/445608767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07f0100ae0 0x7f07f0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.322+0000 7f07f4c00640 1 --2- 172.21.15.47:0/445608767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07f0067c00 0x7f07f0101370 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.322+0000 7f07f4c00640 1 --2- 172.21.15.47:0/445608767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f00ff8f0 0x7f07f00ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.322+0000 7f07f4c00640 1 -- 172.21.15.47:0/445608767 >> 172.21.15.47:0/445608767 conn(0x7f07f00fae60 msgr2=0x7f07f00fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.323+0000 7f07f4c00640 1 -- 172.21.15.47:0/445608767 shutdown_connections 2024-03-21T11:42:06.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.323+0000 7f07f4c00640 1 -- 172.21.15.47:0/445608767 wait complete. 2024-03-21T11:42:06.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.324+0000 7f07f4c00640 1 Processor -- start 2024-03-21T11:42:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.325+0000 7f07f4c00640 1 -- start start 2024-03-21T11:42:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.325+0000 7f07f4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07f0067c00 0x7f07f0070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.325+0000 7f07f4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07f00ff8f0 0x7f07f0070b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.325+0000 7f07f4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f0100ae0 0x7f07f0071ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.326+0000 7f07f4c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f010b1b0 con 0x7f07f00ff8f0 2024-03-21T11:42:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.326+0000 7f07f4c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f010b030 con 0x7f07f0100ae0 2024-03-21T11:42:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.326+0000 7f07f4c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f010b330 con 0x7f07f0067c00 2024-03-21T11:42:06.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.326+0000 7f07ee800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f0100ae0 0x7f07f0071ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.326+0000 7f07ee800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f0100ae0 0x7f07f0071ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57152/0 (socket says 172.21.15.47:57152) 2024-03-21T11:42:06.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.326+0000 7f07ee800640 1 -- 172.21.15.47:0/3360016428 learned_addr learned my addr 172.21.15.47:0/3360016428 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:06.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.326+0000 7f07ed400640 1 --2- 172.21.15.47:0/3360016428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07f00ff8f0 0x7f07f0070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.327+0000 7f07ede00640 1 --2- 172.21.15.47:0/3360016428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07f0067c00 0x7f07f0070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.327+0000 7f07ee800640 1 -- 172.21.15.47:0/3360016428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07f0067c00 msgr2=0x7f07f0070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.327+0000 7f07ee800640 1 --2- 172.21.15.47:0/3360016428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07f0067c00 0x7f07f0070620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.327+0000 7f07ee800640 1 -- 172.21.15.47:0/3360016428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07f00ff8f0 msgr2=0x7f07f0070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.327+0000 7f07ee800640 1 --2- 172.21.15.47:0/3360016428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07f00ff8f0 0x7f07f0070b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.327+0000 7f07ee800640 1 -- 172.21.15.47:0/3360016428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f010f6e0 con 0x7f07f0100ae0 2024-03-21T11:42:06.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.331+0000 7f07ee800640 1 --2- 172.21.15.47:0/3360016428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f0100ae0 0x7f07f0071ba0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f07e800efd0 tx=0x7f07e800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:06.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.331+0000 7f07e6a00640 1 -- 172.21.15.47:0/3360016428 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8003b70 con 0x7f07f0100ae0 2024-03-21T11:42:06.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.331+0000 7f07e6a00640 1 -- 172.21.15.47:0/3360016428 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07e8003d10 con 0x7f07f0100ae0 2024-03-21T11:42:06.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.331+0000 7f07e6a00640 1 -- 172.21.15.47:0/3360016428 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8003b70 con 0x7f07f0100ae0 2024-03-21T11:42:06.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.331+0000 7f07f4c00640 1 -- 172.21.15.47:0/3360016428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f0072140 con 0x7f07f0100ae0 2024-03-21T11:42:06.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.332+0000 7f07f4c00640 1 -- 172.21.15.47:0/3360016428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f00725b0 con 0x7f07f0100ae0 2024-03-21T11:42:06.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.338+0000 7f07e6a00640 1 -- 172.21.15.47:0/3360016428 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f07e8003fc0 con 0x7f07f0100ae0 2024-03-21T11:42:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.344+0000 7f07e6a00640 1 --2- 172.21.15.47:0/3360016428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07d40a31a0 0x7f07d40a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.344+0000 7f07e6a00640 1 -- 172.21.15.47:0/3360016428 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f07e80cfbf0 con 0x7f07f0100ae0 2024-03-21T11:42:06.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.346+0000 7f07ede00640 1 --2- 172.21.15.47:0/3360016428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07d40a31a0 0x7f07d40a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.347+0000 7f07ede00640 1 --2- 172.21.15.47:0/3360016428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07d40a31a0 0x7f07d40a5660 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f07e0005fd0 tx=0x7f07e0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.347+0000 7f07f7989740 1 Processor -- start 2024-03-21T11:42:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.348+0000 7f07f7989740 1 -- start start 2024-03-21T11:42:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.349+0000 7f07f7989740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c00a0 0x564c742c04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.350+0000 7f07f7989740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c0a90 0x564c742c5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.350+0000 7f07f7989740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c5c00 0x564c742c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.350+0000 7f07f7989740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c74225eb0 con 0x564c742c0a90 2024-03-21T11:42:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.350+0000 7f07f7989740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c741cea60 con 0x564c742c00a0 2024-03-21T11:42:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.350+0000 7f07f7989740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c74221f50 con 0x564c742c5c00 2024-03-21T11:42:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.350+0000 7f07ede00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c00a0 0x564c742c04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.350+0000 7f07ede00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c00a0 0x564c742c04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57162/0 (socket says 172.21.15.47:57162) 2024-03-21T11:42:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.350+0000 7f07ede00640 1 -- 172.21.15.47:0/2009437597 learned_addr learned my addr 172.21.15.47:0/2009437597 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.350+0000 7f07ed400640 1 --2- 172.21.15.47:0/2009437597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c0a90 0x564c742c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.351+0000 7f07ede00640 1 -- 172.21.15.47:0/2009437597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c5c00 msgr2=0x564c742c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.351+0000 7f07ede00640 1 --2- 172.21.15.47:0/2009437597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c5c00 0x564c742c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.351+0000 7f07ede00640 1 -- 172.21.15.47:0/2009437597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c0a90 msgr2=0x564c742c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.351+0000 7f07ede00640 1 --2- 172.21.15.47:0/2009437597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c0a90 0x564c742c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.351+0000 7f07ede00640 1 -- 172.21.15.47:0/2009437597 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c740027e0 con 0x564c742c00a0 2024-03-21T11:42:06.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.352+0000 7f07ede00640 1 --2- 172.21.15.47:0/2009437597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c00a0 0x564c742c04c0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f07e000fe80 tx=0x7f07e0012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22f0e1fcdd462fa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:06.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.357+0000 7f07db000640 1 -- 172.21.15.47:0/2009437597 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0022070 con 0x564c742c00a0 2024-03-21T11:42:06.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.357+0000 7f07db000640 1 -- 172.21.15.47:0/2009437597 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07e0011e50 con 0x564c742c00a0 2024-03-21T11:42:06.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.357+0000 7f07db000640 1 -- 172.21.15.47:0/2009437597 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0010a00 con 0x564c742c00a0 2024-03-21T11:42:06.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.359+0000 7f07f7989740 1 -- 172.21.15.47:0/2009437597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c00a0 msgr2=0x564c742c04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.359+0000 7f07f7989740 1 --2- 172.21.15.47:0/2009437597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c00a0 0x564c742c04c0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f07e000fe80 tx=0x7f07e0012000 comp rx=0 tx=0).stop 2024-03-21T11:42:06.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.359+0000 7f07f7989740 1 -- 172.21.15.47:0/2009437597 shutdown_connections 2024-03-21T11:42:06.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.359+0000 7f07f7989740 1 --2- 172.21.15.47:0/2009437597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c5c00 0x564c742c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.359+0000 7f07f7989740 1 --2- 172.21.15.47:0/2009437597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c0a90 0x564c742c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.359+0000 7f07f7989740 1 --2- 172.21.15.47:0/2009437597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c00a0 0x564c742c04c0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.359+0000 7f07f7989740 1 -- 172.21.15.47:0/2009437597 >> 172.21.15.47:0/2009437597 conn(0x564c742bb610 msgr2=0x564c742bba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:06.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.359+0000 7f07f7989740 1 -- 172.21.15.47:0/2009437597 shutdown_connections 2024-03-21T11:42:06.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.359+0000 7f07f7989740 1 -- 172.21.15.47:0/2009437597 wait complete. 2024-03-21T11:42:06.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.360+0000 7f07f7989740 1 -- start start 2024-03-21T11:42:06.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.360+0000 7f07f7989740 1 Processor -- start 2024-03-21T11:42:06.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.361+0000 7f07f7989740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c00a0 0x564c742ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.361+0000 7f07f7989740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c0a90 0x564c742ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.361+0000 7f07f7989740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c5c00 0x564c742d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.362+0000 7f07ede00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c00a0 0x564c742ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.362+0000 7f07ede00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c00a0 0x564c742ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46812/0 (socket says 172.21.15.47:46812) 2024-03-21T11:42:06.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.363+0000 7f07ede00640 1 -- 172.21.15.47:0/2911869957 learned_addr learned my addr 172.21.15.47:0/2911869957 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:06.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.363+0000 7f07f7989740 1 -- 172.21.15.47:0/2911869957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c742c8bb0 con 0x564c742c00a0 2024-03-21T11:42:06.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.363+0000 7f07f7989740 1 -- 172.21.15.47:0/2911869957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c741e1eb0 con 0x564c742c0a90 2024-03-21T11:42:06.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.363+0000 7f07f7989740 1 -- 172.21.15.47:0/2911869957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c742c8d30 con 0x564c742c5c00 2024-03-21T11:42:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.363+0000 7f07f7989740 10 client.0 _ll_register_callbacks cb 0x564c741d6150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.363+0000 7f07ee800640 1 --2- 172.21.15.47:0/2911869957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c5c00 0x564c742d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.363+0000 7f07ede00640 1 -- 172.21.15.47:0/2911869957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c5c00 msgr2=0x564c742d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.363+0000 7f07ede00640 1 --2- 172.21.15.47:0/2911869957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c5c00 0x564c742d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.363+0000 7f07ede00640 1 -- 172.21.15.47:0/2911869957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c0a90 msgr2=0x564c742ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.363+0000 7f07ede00640 1 --2- 172.21.15.47:0/2911869957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c0a90 0x564c742ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.363+0000 7f07ede00640 1 -- 172.21.15.47:0/2911869957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c74062f70 con 0x564c742c00a0 2024-03-21T11:42:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.364+0000 7f07ede00640 1 --2- 172.21.15.47:0/2911869957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c00a0 0x564c742ca2b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f07e000bfd0 tx=0x7f07e0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838aa722f99d8b97 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.365+0000 7f07cf400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:42:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.365+0000 7f07cf400640 1 -- 172.21.15.47:0/2911869957 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0022070 con 0x564c742c00a0 2024-03-21T11:42:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.365+0000 7f07f7989740 1 -- 172.21.15.47:0/2911869957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07e0011e50 con 0x564c742c00a0 2024-03-21T11:42:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.365+0000 7f07cf400640 1 -- 172.21.15.47:0/2911869957 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f07e0010780 con 0x564c742c00a0 2024-03-21T11:42:06.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.366+0000 7f07cf400640 1 -- 172.21.15.47:0/2911869957 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07e0010980 con 0x564c742c00a0 2024-03-21T11:42:06.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.366+0000 7f07f7989740 10 client.21057 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:06.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.366+0000 7f07f7989740 1 -- 172.21.15.47:0/2911869957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564c73efa170 con 0x564c742c00a0 2024-03-21T11:42:06.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.368+0000 7f07cf400640 1 -- 172.21.15.47:0/2911869957 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07e0011e50 con 0x564c742c00a0 2024-03-21T11:42:06.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.368+0000 7f07cf400640 1 -- 172.21.15.47:0/2911869957 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f07e001e450 con 0x564c742c00a0 2024-03-21T11:42:06.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.369+0000 7f07f7989740 10 client.21057 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:06.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.369+0000 7f07f7989740 10 client.21057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:06.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.369+0000 7f07f7989740 20 client.21057 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:06.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.369+0000 7f07f7989740 1 --2- 172.21.15.47:0/2911869957 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564c742d5dc0 0x564c742d8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.369+0000 7f07f7989740 4 client.21057 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:06.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.369+0000 7f07f7989740 1 -- 172.21.15.47:0/2911869957 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c742333f0 con 0x564c742d5dc0 2024-03-21T11:42:06.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.370+0000 7f07ed400640 1 --2- 172.21.15.47:0/2911869957 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564c742d5dc0 0x564c742d8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.372+0000 7f07ed400640 1 --2- 172.21.15.47:0/2911869957 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564c742d5dc0 0x564c742d8210 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a056b8c3399fe306 server_cookie=4d623a63a247213c in_seq=0 out_seq=0 2024-03-21T11:42:06.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.372+0000 7f07cf400640 10 client.21057 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:06.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.377+0000 7f07cf400640 1 -- 172.21.15.47:0/2911869957 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564c742333f0 con 0x564c742d5dc0 2024-03-21T11:42:06.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.377+0000 7f07cf400640 10 client.21057 handle_command_reply: tid=0 2024-03-21T11:42:06.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.378+0000 7f07f7989740 1 client.21057 shutdown 2024-03-21T11:42:06.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.379+0000 7f07f7989740 1 -- 172.21.15.47:0/2911869957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c00a0 msgr2=0x564c742ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.379+0000 7f07f7989740 1 --2- 172.21.15.47:0/2911869957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c00a0 0x564c742ca2b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f07e000bfd0 tx=0x7f07e0005270 comp rx=0 tx=0).stop 2024-03-21T11:42:06.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.379+0000 7f07f7989740 1 -- 172.21.15.47:0/2911869957 shutdown_connections 2024-03-21T11:42:06.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.379+0000 7f07f7989740 1 --2- 172.21.15.47:0/2911869957 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564c742d5dc0 0x564c742d8210 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.379+0000 7f07f7989740 1 --2- 172.21.15.47:0/2911869957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c5c00 0x564c742d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.379+0000 7f07f7989740 1 --2- 172.21.15.47:0/2911869957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c0a90 0x564c742ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.379+0000 7f07f7989740 1 --2- 172.21.15.47:0/2911869957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c00a0 0x564c742ca2b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.379+0000 7f07cf400640 0 client.21057 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:06.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.379+0000 7f07f7989740 1 -- 172.21.15.47:0/2911869957 >> 172.21.15.47:0/2911869957 conn(0x564c742bb610 msgr2=0x564c742ba830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:06.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.379+0000 7f07f7989740 1 -- 172.21.15.47:0/2911869957 shutdown_connections 2024-03-21T11:42:06.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.382+0000 7f07f7989740 1 -- 172.21.15.47:0/2911869957 wait complete. 2024-03-21T11:42:06.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.382+0000 7f07f7989740 20 client.21057 trim_cache size 0 max 16384 2024-03-21T11:42:06.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.419+0000 7f07f7989740 1 Processor -- start 2024-03-21T11:42:06.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.420+0000 7f07f7989740 1 -- start start 2024-03-21T11:42:06.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.420+0000 7f07f7989740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c00a0 0x564c742ba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.420+0000 7f07f7989740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c0a90 0x564c742babf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.420+0000 7f07f7989740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c5c00 0x564c742d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.420+0000 7f07f7989740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f01b8f10 con 0x564c742c5c00 2024-03-21T11:42:06.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.420+0000 7f07f7989740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f01ba890 con 0x564c742c0a90 2024-03-21T11:42:06.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.420+0000 7f07f7989740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f01b9570 con 0x564c742c00a0 2024-03-21T11:42:06.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.421+0000 7f07ed400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c0a90 0x564c742babf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.422+0000 7f07ed400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c0a90 0x564c742babf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57190/0 (socket says 172.21.15.47:57190) 2024-03-21T11:42:06.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.421+0000 7f07ede00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c00a0 0x564c742ba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.422+0000 7f07ede00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c00a0 0x564c742ba6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58608/0 (socket says 172.21.15.47:58608) 2024-03-21T11:42:06.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.422+0000 7f07ede00640 1 -- 172.21.15.47:0/57125781 learned_addr learned my addr 172.21.15.47:0/57125781 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:06.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.422+0000 7f07ee800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c5c00 0x564c742d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.423+0000 7f07ede00640 1 -- 172.21.15.47:0/57125781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c0a90 msgr2=0x564c742babf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.423+0000 7f07ede00640 1 --2- 172.21.15.47:0/57125781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c0a90 0x564c742babf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.423+0000 7f07ede00640 1 -- 172.21.15.47:0/57125781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c5c00 msgr2=0x564c742d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.423+0000 7f07ede00640 1 --2- 172.21.15.47:0/57125781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c5c00 0x564c742d17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.423+0000 7f07ede00640 1 -- 172.21.15.47:0/57125781 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c742bfce0 con 0x564c742c00a0 2024-03-21T11:42:06.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.424+0000 7f07ede00640 1 --2- 172.21.15.47:0/57125781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c00a0 0x564c742ba6b0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f07e001e840 tx=0x7f07e000fee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d45dbef35bd33736 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.424+0000 7f07da600640 1 -- 172.21.15.47:0/57125781 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0015070 con 0x564c742c00a0 2024-03-21T11:42:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.424+0000 7f07da600640 1 -- 172.21.15.47:0/57125781 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07e001eab0 con 0x564c742c00a0 2024-03-21T11:42:06.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.424+0000 7f07da600640 1 -- 172.21.15.47:0/57125781 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0022040 con 0x564c742c00a0 2024-03-21T11:42:06.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.426+0000 7f07f7989740 1 -- 172.21.15.47:0/57125781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c00a0 msgr2=0x564c742ba6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.426+0000 7f07f7989740 1 --2- 172.21.15.47:0/57125781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c00a0 0x564c742ba6b0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f07e001e840 tx=0x7f07e000fee0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.426+0000 7f07f7989740 1 -- 172.21.15.47:0/57125781 shutdown_connections 2024-03-21T11:42:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.426+0000 7f07f7989740 1 --2- 172.21.15.47:0/57125781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c5c00 0x564c742d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.426+0000 7f07f7989740 1 --2- 172.21.15.47:0/57125781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c0a90 0x564c742babf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.426+0000 7f07f7989740 1 --2- 172.21.15.47:0/57125781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c00a0 0x564c742ba6b0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.426+0000 7f07f7989740 1 -- 172.21.15.47:0/57125781 >> 172.21.15.47:0/57125781 conn(0x564c742bb610 msgr2=0x564c742b9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.426+0000 7f07f7989740 1 -- 172.21.15.47:0/57125781 shutdown_connections 2024-03-21T11:42:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.427+0000 7f07f7989740 1 -- 172.21.15.47:0/57125781 wait complete. 2024-03-21T11:42:06.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.427+0000 7f07f7989740 1 -- start start 2024-03-21T11:42:06.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.429+0000 7f07f7989740 1 Processor -- start 2024-03-21T11:42:06.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.430+0000 7f07f7989740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c00a0 0x564c742d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.431+0000 7f07f7989740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c0a90 0x564c742d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.431+0000 7f07ed400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c0a90 0x564c742d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.432+0000 7f07f7989740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c5c00 0x564c742cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.431+0000 7f07ede00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c00a0 0x564c742d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.432+0000 7f07f7989740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f01ba230 con 0x564c742c00a0 2024-03-21T11:42:06.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.432+0000 7f07ede00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c00a0 0x564c742d1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46820/0 (socket says 172.21.15.47:46820) 2024-03-21T11:42:06.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.432+0000 7f07ede00640 1 -- 172.21.15.47:0/2578505552 learned_addr learned my addr 172.21.15.47:0/2578505552 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:06.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.432+0000 7f07ee800640 1 --2- 172.21.15.47:0/2578505552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c5c00 0x564c742cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.432+0000 7f07f7989740 1 -- 172.21.15.47:0/2578505552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f01b9bd0 con 0x564c742c5c00 2024-03-21T11:42:06.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.432+0000 7f07f7989740 1 -- 172.21.15.47:0/2578505552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f01b70d0 con 0x564c742c0a90 2024-03-21T11:42:06.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.433+0000 7f07f7989740 10 client.0 _ll_register_callbacks cb 0x7f07e80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:06.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.435+0000 7f07ee800640 1 -- 172.21.15.47:0/2578505552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c0a90 msgr2=0x564c742d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.435+0000 7f07ee800640 1 --2- 172.21.15.47:0/2578505552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c0a90 0x564c742d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.435+0000 7f07ee800640 1 -- 172.21.15.47:0/2578505552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c00a0 msgr2=0x564c742d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.435+0000 7f07ee800640 1 --2- 172.21.15.47:0/2578505552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c00a0 0x564c742d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.435+0000 7f07ee800640 1 -- 172.21.15.47:0/2578505552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c742b6c20 con 0x564c742c5c00 2024-03-21T11:42:06.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.435+0000 7f07ede00640 1 --2- 172.21.15.47:0/2578505552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c00a0 0x564c742d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:06.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.436+0000 7f07ee800640 1 --2- 172.21.15.47:0/2578505552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c5c00 0x564c742cdbb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f07e800cba0 tx=0x7f07e809b9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e730028e6684e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:06.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.436+0000 7f07cf400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:06.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.438+0000 7f07cf400640 1 -- 172.21.15.47:0/2578505552 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e80b4070 con 0x564c742c5c00 2024-03-21T11:42:06.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.438+0000 7f07cf400640 1 -- 172.21.15.47:0/2578505552 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f07e809c1f0 con 0x564c742c5c00 2024-03-21T11:42:06.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.438+0000 7f07f7989740 1 -- 172.21.15.47:0/2578505552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07e001eab0 con 0x564c742c5c00 2024-03-21T11:42:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.439+0000 7f07cf400640 1 -- 172.21.15.47:0/2578505552 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07e80a00e0 con 0x564c742c5c00 2024-03-21T11:42:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.439+0000 7f07f7989740 10 client.31640 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.439+0000 7f07f7989740 1 -- 172.21.15.47:0/2578505552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564c743431c0 con 0x564c742c5c00 2024-03-21T11:42:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.439+0000 7f07cf400640 1 -- 172.21.15.47:0/2578505552 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07e80a49a0 con 0x564c742c5c00 2024-03-21T11:42:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.439+0000 7f07cf400640 1 -- 172.21.15.47:0/2578505552 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f07e809b400 con 0x564c742c5c00 2024-03-21T11:42:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.440+0000 7f07f7989740 10 client.31640 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.440+0000 7f07f7989740 10 client.31640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.440+0000 7f07f7989740 20 client.31640 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.440+0000 7f07f7989740 1 --2- 172.21.15.47:0/2578505552 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564c74343a50 0x564c74345e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.440+0000 7f07f7989740 4 client.31640 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:42:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.440+0000 7f07f7989740 1 -- 172.21.15.47:0/2578505552 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f07e001eab0 con 0x564c74343a50 2024-03-21T11:42:06.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.440+0000 7f07ede00640 1 --2- 172.21.15.47:0/2578505552 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564c74343a50 0x564c74345e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.442+0000 7f07ede00640 1 --2- 172.21.15.47:0/2578505552 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564c74343a50 0x564c74345e70 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e062a6915b2173 server_cookie=6086e8c1f01bf7b0 in_seq=0 out_seq=0 2024-03-21T11:42:06.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.442+0000 7f07cf400640 10 client.31640 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:06.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.451+0000 7f07cf400640 1 -- 172.21.15.47:0/2578505552 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f07e0046080 con 0x564c74343a50 2024-03-21T11:42:06.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.451+0000 7f07cf400640 10 client.31640 handle_command_reply: tid=0 2024-03-21T11:42:06.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.451+0000 7f07f7989740 1 client.31640 shutdown 2024-03-21T11:42:06.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.451+0000 7f07f7989740 1 -- 172.21.15.47:0/2578505552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c5c00 msgr2=0x564c742cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.451+0000 7f07f7989740 1 --2- 172.21.15.47:0/2578505552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c5c00 0x564c742cdbb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f07e800cba0 tx=0x7f07e809b9e0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.451+0000 7f07f7989740 1 -- 172.21.15.47:0/2578505552 shutdown_connections 2024-03-21T11:42:06.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.451+0000 7f07f7989740 1 --2- 172.21.15.47:0/2578505552 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564c74343a50 0x564c74345e70 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.452+0000 7f07f7989740 1 --2- 172.21.15.47:0/2578505552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c742c5c00 0x564c742cdbb0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.452+0000 7f07f7989740 1 --2- 172.21.15.47:0/2578505552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c742c0a90 0x564c742d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.452+0000 7f07f7989740 1 --2- 172.21.15.47:0/2578505552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c742c00a0 0x564c742d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.452+0000 7f07f7989740 1 -- 172.21.15.47:0/2578505552 >> 172.21.15.47:0/2578505552 conn(0x564c742bb610 msgr2=0x564c742ba830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:06.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.452+0000 7f07cf400640 0 client.31640 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:06.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.452+0000 7f07f7989740 1 -- 172.21.15.47:0/2578505552 shutdown_connections 2024-03-21T11:42:06.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.452+0000 7f07f7989740 1 -- 172.21.15.47:0/2578505552 wait complete. 2024-03-21T11:42:06.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.452+0000 7f07f7989740 20 client.31640 trim_cache size 0 max 16384 2024-03-21T11:42:06.454 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:42:06.454 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:42:06.454 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "16895d3f-ac0e-4979-a84f-d883fd77eb21", 2024-03-21T11:42:06.454 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:42:06.454 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:42:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.454+0000 7f07f4c00640 1 -- 172.21.15.47:0/3360016428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07d40a31a0 msgr2=0x7f07d40a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.454+0000 7f07f4c00640 1 --2- 172.21.15.47:0/3360016428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07d40a31a0 0x7f07d40a5660 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f07e0005fd0 tx=0x7f07e0005850 comp rx=0 tx=0).stop 2024-03-21T11:42:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.454+0000 7f07f4c00640 1 -- 172.21.15.47:0/3360016428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f0100ae0 msgr2=0x7f07f0071ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.454+0000 7f07f4c00640 1 --2- 172.21.15.47:0/3360016428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f0100ae0 0x7f07f0071ba0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f07e800efd0 tx=0x7f07e800c6a0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.454+0000 7f07f4c00640 1 -- 172.21.15.47:0/3360016428 shutdown_connections 2024-03-21T11:42:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.454+0000 7f07f4c00640 1 --2- 172.21.15.47:0/3360016428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f07d40a31a0 0x7f07d40a5660 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.454+0000 7f07f4c00640 1 --2- 172.21.15.47:0/3360016428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f0100ae0 0x7f07f0071ba0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.454+0000 7f07f4c00640 1 --2- 172.21.15.47:0/3360016428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07f00ff8f0 0x7f07f0070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.454+0000 7f07f4c00640 1 --2- 172.21.15.47:0/3360016428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07f0067c00 0x7f07f0070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.454+0000 7f07f4c00640 1 -- 172.21.15.47:0/3360016428 >> 172.21.15.47:0/3360016428 conn(0x7f07f00fae60 msgr2=0x7f07f0101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.454+0000 7f07f4c00640 1 -- 172.21.15.47:0/3360016428 shutdown_connections 2024-03-21T11:42:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.454+0000 7f07f4c00640 1 -- 172.21.15.47:0/3360016428 wait complete. 2024-03-21T11:42:06.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:42:06.481 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:42:06.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.774+0000 7f31fa000640 1 -- 172.21.15.47:0/2442043879 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f402ccb0 con 0x7f31fc10dc80 2024-03-21T11:42:06.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.774+0000 7f3201c00640 1 -- 172.21.15.47:0/2442043879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31fc10dc80 msgr2=0x7f31fc110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.774+0000 7f3201c00640 1 --2- 172.21.15.47:0/2442043879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31fc10dc80 0x7f31fc110070 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f31f4009900 tx=0x7f31f402bc30 comp rx=0 tx=0).stop 2024-03-21T11:42:06.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.774+0000 7f3201c00640 1 -- 172.21.15.47:0/2442043879 shutdown_connections 2024-03-21T11:42:06.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.774+0000 7f3201c00640 1 --2- 172.21.15.47:0/2442043879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31fc10dc80 0x7f31fc110070 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.774+0000 7f3201c00640 1 --2- 172.21.15.47:0/2442043879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31fc074100 0x7f31fc074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.774+0000 7f3201c00640 1 --2- 172.21.15.47:0/2442043879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0756e0 0x7f31fc073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.774+0000 7f3201c00640 1 -- 172.21.15.47:0/2442043879 >> 172.21.15.47:0/2442043879 conn(0x7f31fc0fcc20 msgr2=0x7f31fc0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:06.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.775+0000 7f3201c00640 1 -- 172.21.15.47:0/2442043879 shutdown_connections 2024-03-21T11:42:06.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.775+0000 7f3201c00640 1 -- 172.21.15.47:0/2442043879 wait complete. 2024-03-21T11:42:06.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.776+0000 7f3201c00640 1 Processor -- start 2024-03-21T11:42:06.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.776+0000 7f3201c00640 1 -- start start 2024-03-21T11:42:06.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.776+0000 7f3201c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31fc074100 0x7f31fc19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.776+0000 7f3201c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0756e0 0x7f31fc19e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.776+0000 7f3201c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31fc10dc80 0x7f31fc1a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.776+0000 7f3201c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc113970 con 0x7f31fc0756e0 2024-03-21T11:42:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.776+0000 7f3201c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc1137f0 con 0x7f31fc10dc80 2024-03-21T11:42:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.776+0000 7f3201c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc113af0 con 0x7f31fc074100 2024-03-21T11:42:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.777+0000 7f31faa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31fc074100 0x7f31fc19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.777+0000 7f31faa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31fc074100 0x7f31fc19df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58640/0 (socket says 172.21.15.47:58640) 2024-03-21T11:42:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.777+0000 7f31faa00640 1 -- 172.21.15.47:0/1539673804 learned_addr learned my addr 172.21.15.47:0/1539673804 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.777+0000 7f31fb400640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31fc10dc80 0x7f31fc1a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.777+0000 7f31f2000640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0756e0 0x7f31fc19e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.777+0000 7f31faa00640 1 -- 172.21.15.47:0/1539673804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31fc10dc80 msgr2=0x7f31fc1a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.777+0000 7f31faa00640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31fc10dc80 0x7f31fc1a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.777+0000 7f31faa00640 1 -- 172.21.15.47:0/1539673804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0756e0 msgr2=0x7f31fc19e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.777+0000 7f31faa00640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0756e0 0x7f31fc19e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.777+0000 7f31faa00640 1 -- 172.21.15.47:0/1539673804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc1a5ba0 con 0x7f31fc074100 2024-03-21T11:42:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.777+0000 7f31f2000640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0756e0 0x7f31fc19e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.777+0000 7f31faa00640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31fc074100 0x7f31fc19df80 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f31e400e9f0 tx=0x7f31e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:06.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.778+0000 7f31f3e00640 1 -- 172.21.15.47:0/1539673804 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e400cd80 con 0x7f31fc074100 2024-03-21T11:42:06.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.778+0000 7f31f3e00640 1 -- 172.21.15.47:0/1539673804 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31e4010910 con 0x7f31fc074100 2024-03-21T11:42:06.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.778+0000 7f31f3e00640 1 -- 172.21.15.47:0/1539673804 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4018980 con 0x7f31fc074100 2024-03-21T11:42:06.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.778+0000 7f3201c00640 1 -- 172.21.15.47:0/1539673804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31fc1a5e90 con 0x7f31fc074100 2024-03-21T11:42:06.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.778+0000 7f3201c00640 1 -- 172.21.15.47:0/1539673804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc1a62a0 con 0x7f31fc074100 2024-03-21T11:42:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.781+0000 7f3204824740 1 Processor -- start 2024-03-21T11:42:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.781+0000 7f3204824740 1 -- start start 2024-03-21T11:42:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f3204824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b2e80 0x555ffe5b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f3204824740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b3870 0x555ffe5b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f3204824740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 0x555ffe5bae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f3204824740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ffe518250 con 0x555ffe5b3870 2024-03-21T11:42:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f3204824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ffe4c0a30 con 0x555ffe5b2e80 2024-03-21T11:42:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f3204824740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ffe5142f0 con 0x555ffe5b89e0 2024-03-21T11:42:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f31fb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b2e80 0x555ffe5b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f31fb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b2e80 0x555ffe5b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57236/0 (socket says 172.21.15.47:57236) 2024-03-21T11:42:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f31fb400640 1 -- 172.21.15.47:0/788954081 learned_addr learned my addr 172.21.15.47:0/788954081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f31f3e00640 1 -- 172.21.15.47:0/1539673804 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f31e4010ac0 con 0x7f31fc074100 2024-03-21T11:42:06.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f31f2000640 1 --2- 172.21.15.47:0/788954081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 0x555ffe5bae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f31fb400640 1 -- 172.21.15.47:0/788954081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 msgr2=0x555ffe5bae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f31fb400640 1 --2- 172.21.15.47:0/788954081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 0x555ffe5bae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f31fb400640 1 -- 172.21.15.47:0/788954081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b3870 msgr2=0x555ffe5b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f31fb400640 1 --2- 172.21.15.47:0/788954081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b3870 0x555ffe5b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.782+0000 7f31fb400640 1 -- 172.21.15.47:0/788954081 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ffe4ab690 con 0x555ffe5b2e80 2024-03-21T11:42:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.783+0000 7f31fb400640 1 --2- 172.21.15.47:0/788954081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b2e80 0x555ffe5b32a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f31f4005ab0 tx=0x7f31f40095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec558fce995601bf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.783+0000 7f31ebe00640 1 -- 172.21.15.47:0/788954081 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4042070 con 0x555ffe5b2e80 2024-03-21T11:42:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.783+0000 7f31ebe00640 1 -- 172.21.15.47:0/788954081 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31f402cbb0 con 0x555ffe5b2e80 2024-03-21T11:42:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.783+0000 7f31ebe00640 1 -- 172.21.15.47:0/788954081 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4004f10 con 0x555ffe5b2e80 2024-03-21T11:42:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.784+0000 7f31f3e00640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31e00a2fa0 0x7f31e00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.784+0000 7f31f3e00640 1 -- 172.21.15.47:0/1539673804 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f31e40cfa90 con 0x7f31fc074100 2024-03-21T11:42:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.784+0000 7f3204824740 1 -- 172.21.15.47:0/788954081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b2e80 msgr2=0x555ffe5b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.784+0000 7f3204824740 1 --2- 172.21.15.47:0/788954081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b2e80 0x555ffe5b32a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f31f4005ab0 tx=0x7f31f40095f0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.785+0000 7f31faa00640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31e00a2fa0 0x7f31e00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.786+0000 7f31faa00640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31e00a2fa0 0x7f31e00a5460 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f31e409bdc0 tx=0x7f31e4097a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.787+0000 7f3204824740 1 -- 172.21.15.47:0/788954081 shutdown_connections 2024-03-21T11:42:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.787+0000 7f3204824740 1 --2- 172.21.15.47:0/788954081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 0x555ffe5bae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.787+0000 7f3204824740 1 --2- 172.21.15.47:0/788954081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b3870 0x555ffe5b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.787+0000 7f3204824740 1 --2- 172.21.15.47:0/788954081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b2e80 0x555ffe5b32a0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.787+0000 7f3204824740 1 -- 172.21.15.47:0/788954081 >> 172.21.15.47:0/788954081 conn(0x555ffe57b200 msgr2=0x555ffe5ae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.787+0000 7f3204824740 1 -- 172.21.15.47:0/788954081 shutdown_connections 2024-03-21T11:42:06.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.787+0000 7f3204824740 1 -- 172.21.15.47:0/788954081 wait complete. 2024-03-21T11:42:06.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.788+0000 7f3204824740 1 -- start start 2024-03-21T11:42:06.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.788+0000 7f3204824740 1 Processor -- start 2024-03-21T11:42:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.789+0000 7f3204824740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b2e80 0x555ffe5bcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.789+0000 7f3204824740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b3870 0x555ffe5bd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.789+0000 7f3204824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b89e0 0x555ffe5c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.789+0000 7f3204824740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ffe5bb930 con 0x555ffe5b3870 2024-03-21T11:42:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.789+0000 7f3204824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ffe4d3e80 con 0x555ffe5b89e0 2024-03-21T11:42:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.789+0000 7f3204824740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ffe5bbab0 con 0x555ffe5b2e80 2024-03-21T11:42:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.789+0000 7f3204824740 10 client.0 _ll_register_callbacks cb 0x555ffe4e6130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.789+0000 7f31fb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b2e80 0x555ffe5bcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.789+0000 7f31fb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b2e80 0x555ffe5bcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58670/0 (socket says 172.21.15.47:58670) 2024-03-21T11:42:06.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.789+0000 7f31fb400640 1 -- 172.21.15.47:0/2227927126 learned_addr learned my addr 172.21.15.47:0/2227927126 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.789+0000 7f31fb400640 1 --2- 172.21.15.47:0/2227927126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b89e0 0x555ffe5c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.789+0000 7f31f2000640 1 --2- 172.21.15.47:0/2227927126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b3870 0x555ffe5bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.790+0000 7f31fb400640 1 -- 172.21.15.47:0/2227927126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b89e0 msgr2=0x555ffe5c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.790+0000 7f31fb400640 1 --2- 172.21.15.47:0/2227927126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b89e0 0x555ffe5c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.790+0000 7f31fb400640 1 -- 172.21.15.47:0/2227927126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b3870 msgr2=0x555ffe5bd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.790+0000 7f31fb400640 1 --2- 172.21.15.47:0/2227927126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b3870 0x555ffe5bd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.790+0000 7f31fb400640 1 -- 172.21.15.47:0/2227927126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ffe2f67d0 con 0x555ffe5b2e80 2024-03-21T11:42:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.790+0000 7f31f2000640 1 --2- 172.21.15.47:0/2227927126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b3870 0x555ffe5bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.790+0000 7f31fb400640 1 --2- 172.21.15.47:0/2227927126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b2e80 0x555ffe5bcdf0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f31f4006010 tx=0x7f31f402e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f4e43737924f1d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:06.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.790+0000 7f31e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:42:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.791+0000 7f31e8c00640 1 -- 172.21.15.47:0/2227927126 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4042070 con 0x555ffe5b2e80 2024-03-21T11:42:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.791+0000 7f31e8c00640 1 -- 172.21.15.47:0/2227927126 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f31f40042e0 con 0x555ffe5b2e80 2024-03-21T11:42:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.791+0000 7f3204824740 1 -- 172.21.15.47:0/2227927126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31f402cbb0 con 0x555ffe5b2e80 2024-03-21T11:42:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.791+0000 7f31e8c00640 1 -- 172.21.15.47:0/2227927126 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31f405d050 con 0x555ffe5b2e80 2024-03-21T11:42:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.791+0000 7f3204824740 10 client.31624 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.791+0000 7f3204824740 1 -- 172.21.15.47:0/2227927126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555ffe22c3f0 con 0x555ffe5b2e80 2024-03-21T11:42:06.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.791+0000 7f31e8c00640 1 -- 172.21.15.47:0/2227927126 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31f402cbb0 con 0x555ffe5b2e80 2024-03-21T11:42:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.792+0000 7f31e8c00640 1 -- 172.21.15.47:0/2227927126 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f31f40039b0 con 0x555ffe5b2e80 2024-03-21T11:42:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.792+0000 7f3204824740 10 client.31624 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.792+0000 7f3204824740 10 client.31624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.792+0000 7f3204824740 20 client.31624 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.792+0000 7f3204824740 1 --2- 172.21.15.47:0/2227927126 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ffe5c8840 0x555ffe5cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.792+0000 7f3204824740 4 client.31624 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.792+0000 7f3204824740 1 -- 172.21.15.47:0/2227927126 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ffe525790 con 0x555ffe5c8840 2024-03-21T11:42:06.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.792+0000 7f31f2000640 1 --2- 172.21.15.47:0/2227927126 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ffe5c8840 0x555ffe5cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.794+0000 7f31f2000640 1 --2- 172.21.15.47:0/2227927126 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ffe5c8840 0x555ffe5cac30 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d738a84eb4848117 server_cookie=63caafee109f5367 in_seq=0 out_seq=0 2024-03-21T11:42:06.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.794+0000 7f31e8c00640 10 client.31624 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.800+0000 7f31e8c00640 1 -- 172.21.15.47:0/2227927126 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555ffe525790 con 0x555ffe5c8840 2024-03-21T11:42:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.800+0000 7f31e8c00640 10 client.31624 handle_command_reply: tid=0 2024-03-21T11:42:06.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.800+0000 7f3204824740 1 client.31624 shutdown 2024-03-21T11:42:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.800+0000 7f3204824740 1 -- 172.21.15.47:0/2227927126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b2e80 msgr2=0x555ffe5bcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.800+0000 7f3204824740 1 --2- 172.21.15.47:0/2227927126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b2e80 0x555ffe5bcdf0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f31f4006010 tx=0x7f31f402e040 comp rx=0 tx=0).stop 2024-03-21T11:42:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.800+0000 7f3204824740 1 -- 172.21.15.47:0/2227927126 shutdown_connections 2024-03-21T11:42:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.800+0000 7f3204824740 1 --2- 172.21.15.47:0/2227927126 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ffe5c8840 0x555ffe5cac30 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.800+0000 7f3204824740 1 --2- 172.21.15.47:0/2227927126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b89e0 0x555ffe5c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.800+0000 7f3204824740 1 --2- 172.21.15.47:0/2227927126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b3870 0x555ffe5bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.800+0000 7f3204824740 1 --2- 172.21.15.47:0/2227927126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b2e80 0x555ffe5bcdf0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.800+0000 7f3204824740 1 -- 172.21.15.47:0/2227927126 >> 172.21.15.47:0/2227927126 conn(0x555ffe57b200 msgr2=0x555ffe5ada90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.800+0000 7f31e8c00640 0 client.31624 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.801+0000 7f3204824740 1 -- 172.21.15.47:0/2227927126 shutdown_connections 2024-03-21T11:42:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.801+0000 7f3204824740 1 -- 172.21.15.47:0/2227927126 wait complete. 2024-03-21T11:42:06.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.801+0000 7f3204824740 20 client.31624 trim_cache size 0 max 16384 2024-03-21T11:42:06.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.830+0000 7f3204824740 1 Processor -- start 2024-03-21T11:42:06.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.830+0000 7f3204824740 1 -- start start 2024-03-21T11:42:06.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.830+0000 7f3204824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b2e80 0x555ffe5b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.830+0000 7f3204824740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b3870 0x555ffe5c4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.830+0000 7f3204824740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 0x555ffe5b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.830+0000 7f3204824740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc1b0670 con 0x555ffe5b3870 2024-03-21T11:42:06.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.830+0000 7f3204824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc1b1ff0 con 0x555ffe5b2e80 2024-03-21T11:42:06.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.830+0000 7f3204824740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc1b0cd0 con 0x555ffe5b89e0 2024-03-21T11:42:06.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.831+0000 7f31fb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 0x555ffe5b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.831+0000 7f31fb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 0x555ffe5b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58676/0 (socket says 172.21.15.47:58676) 2024-03-21T11:42:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.831+0000 7f31fb400640 1 -- 172.21.15.47:0/1604130515 learned_addr learned my addr 172.21.15.47:0/1604130515 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.831+0000 7f31fb400640 1 --2- 172.21.15.47:0/1604130515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b2e80 0x555ffe5b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.831+0000 7f31fb400640 1 -- 172.21.15.47:0/1604130515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b2e80 msgr2=0x555ffe5b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.831+0000 7f31fb400640 1 --2- 172.21.15.47:0/1604130515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b2e80 0x555ffe5b7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.831+0000 7f31fb400640 1 -- 172.21.15.47:0/1604130515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b3870 msgr2=0x555ffe5c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.831+0000 7f31fb400640 1 --2- 172.21.15.47:0/1604130515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b3870 0x555ffe5c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.831+0000 7f31fb400640 1 -- 172.21.15.47:0/1604130515 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ffe1ec0f0 con 0x555ffe5b89e0 2024-03-21T11:42:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.832+0000 7f31fb400640 1 --2- 172.21.15.47:0/1604130515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 0x555ffe5b4a20 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f31f4003fd0 tx=0x7f31f40095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a212177e1b56753 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.832+0000 7f31eb400640 1 -- 172.21.15.47:0/1604130515 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4042070 con 0x555ffe5b89e0 2024-03-21T11:42:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.832+0000 7f31eb400640 1 -- 172.21.15.47:0/1604130515 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31f40049d0 con 0x555ffe5b89e0 2024-03-21T11:42:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.832+0000 7f31eb400640 1 -- 172.21.15.47:0/1604130515 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f40042e0 con 0x555ffe5b89e0 2024-03-21T11:42:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.833+0000 7f3204824740 1 -- 172.21.15.47:0/1604130515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 msgr2=0x555ffe5b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.833+0000 7f3204824740 1 --2- 172.21.15.47:0/1604130515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 0x555ffe5b4a20 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f31f4003fd0 tx=0x7f31f40095f0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.834+0000 7f3204824740 1 -- 172.21.15.47:0/1604130515 shutdown_connections 2024-03-21T11:42:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.834+0000 7f3204824740 1 --2- 172.21.15.47:0/1604130515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 0x555ffe5b4a20 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.834+0000 7f3204824740 1 --2- 172.21.15.47:0/1604130515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b3870 0x555ffe5c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.834+0000 7f3204824740 1 --2- 172.21.15.47:0/1604130515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b2e80 0x555ffe5b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.834+0000 7f3204824740 1 -- 172.21.15.47:0/1604130515 >> 172.21.15.47:0/1604130515 conn(0x555ffe57b200 msgr2=0x555ffe5b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:06.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.834+0000 7f3204824740 1 -- 172.21.15.47:0/1604130515 shutdown_connections 2024-03-21T11:42:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.834+0000 7f3204824740 1 -- 172.21.15.47:0/1604130515 wait complete. 2024-03-21T11:42:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.834+0000 7f3204824740 1 -- start start 2024-03-21T11:42:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.835+0000 7f3204824740 1 Processor -- start 2024-03-21T11:42:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.835+0000 7f3204824740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b2e80 0x555ffe5c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.835+0000 7f3204824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b3870 0x555ffe5c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.835+0000 7f3204824740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 0x555ffe5c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.835+0000 7f3204824740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc1b1990 con 0x555ffe5b2e80 2024-03-21T11:42:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.835+0000 7f3204824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc1b1330 con 0x555ffe5b3870 2024-03-21T11:42:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.835+0000 7f3204824740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc1ae830 con 0x555ffe5b89e0 2024-03-21T11:42:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.835+0000 7f3204824740 10 client.0 _ll_register_callbacks cb 0x7f31ec0099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:06.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.836+0000 7f31f2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b3870 0x555ffe5c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.836+0000 7f31f2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b3870 0x555ffe5c1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57266/0 (socket says 172.21.15.47:57266) 2024-03-21T11:42:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.836+0000 7f31f2000640 1 -- 172.21.15.47:0/1732139680 learned_addr learned my addr 172.21.15.47:0/1732139680 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.836+0000 7f31f2000640 1 -- 172.21.15.47:0/1732139680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 msgr2=0x555ffe5c2f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:42:06.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.836+0000 7f31f2000640 1 --2- 172.21.15.47:0/1732139680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 0x555ffe5c2f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.836+0000 7f31f2000640 1 -- 172.21.15.47:0/1732139680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b2e80 msgr2=0x555ffe5c1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.836+0000 7f31f2000640 1 --2- 172.21.15.47:0/1732139680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b2e80 0x555ffe5c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.836+0000 7f31f2000640 1 -- 172.21.15.47:0/1732139680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ffe5c3530 con 0x555ffe5b3870 2024-03-21T11:42:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.837+0000 7f31f2000640 1 --2- 172.21.15.47:0/1732139680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b3870 0x555ffe5c1f80 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f31ec005fd0 tx=0x7f31ec011ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34b2f4f5cf156a77 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.837+0000 7f31e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.837+0000 7f31e8c00640 1 -- 172.21.15.47:0/1732139680 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec004b20 con 0x555ffe5b3870 2024-03-21T11:42:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.837+0000 7f31e8c00640 1 -- 172.21.15.47:0/1732139680 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f31ec004cc0 con 0x555ffe5b3870 2024-03-21T11:42:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.837+0000 7f3204824740 1 -- 172.21.15.47:0/1732139680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31f40049d0 con 0x555ffe5b3870 2024-03-21T11:42:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.838+0000 7f31e8c00640 1 -- 172.21.15.47:0/1732139680 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31ec02f050 con 0x555ffe5b3870 2024-03-21T11:42:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.838+0000 7f3204824740 10 client.31658 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:06.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.838+0000 7f3204824740 1 -- 172.21.15.47:0/1732139680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555ffe634fe0 con 0x555ffe5b3870 2024-03-21T11:42:06.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.838+0000 7f31e8c00640 1 -- 172.21.15.47:0/1732139680 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31f40049d0 con 0x555ffe5b3870 2024-03-21T11:42:06.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.839+0000 7f31e8c00640 1 -- 172.21.15.47:0/1732139680 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f31ec01d480 con 0x555ffe5b3870 2024-03-21T11:42:06.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.839+0000 7f3204824740 10 client.31658 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:06.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.839+0000 7f3204824740 10 client.31658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:06.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.839+0000 7f3204824740 20 client.31658 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:06.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.839+0000 7f3204824740 1 --2- 172.21.15.47:0/1732139680 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ffe635930 0x555ffe637d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:06.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.839+0000 7f3204824740 4 client.31658 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:42:06.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.839+0000 7f3204824740 1 -- 172.21.15.47:0/1732139680 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555ffe51fae0 con 0x555ffe635930 2024-03-21T11:42:06.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.840+0000 7f31fb400640 1 --2- 172.21.15.47:0/1732139680 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ffe635930 0x555ffe637d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:06.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.840+0000 7f31fb400640 1 --2- 172.21.15.47:0/1732139680 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ffe635930 0x555ffe637d50 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fecc90f65553e240 server_cookie=5576f5a49418525 in_seq=0 out_seq=0 2024-03-21T11:42:06.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.840+0000 7f31e8c00640 10 client.31658 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:06.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.846+0000 7f31e8c00640 1 -- 172.21.15.47:0/1732139680 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x555ffe51fae0 con 0x555ffe635930 2024-03-21T11:42:06.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.846+0000 7f31e8c00640 10 client.31658 handle_command_reply: tid=0 2024-03-21T11:42:06.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.846+0000 7f3204824740 1 client.31658 shutdown 2024-03-21T11:42:06.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.846+0000 7f3204824740 1 -- 172.21.15.47:0/1732139680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b3870 msgr2=0x555ffe5c1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.846+0000 7f3204824740 1 --2- 172.21.15.47:0/1732139680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b3870 0x555ffe5c1f80 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f31ec005fd0 tx=0x7f31ec011ec0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.847+0000 7f3204824740 1 -- 172.21.15.47:0/1732139680 shutdown_connections 2024-03-21T11:42:06.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.847+0000 7f3204824740 1 --2- 172.21.15.47:0/1732139680 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555ffe635930 0x555ffe637d50 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.847+0000 7f3204824740 1 --2- 172.21.15.47:0/1732139680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555ffe5b89e0 0x555ffe5c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.847+0000 7f3204824740 1 --2- 172.21.15.47:0/1732139680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ffe5b3870 0x555ffe5c1f80 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.847+0000 7f3204824740 1 --2- 172.21.15.47:0/1732139680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ffe5b2e80 0x555ffe5c1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.847+0000 7f3204824740 1 -- 172.21.15.47:0/1732139680 >> 172.21.15.47:0/1732139680 conn(0x555ffe57b200 msgr2=0x555ffe5b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:06.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.847+0000 7f31e8c00640 0 client.31658 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:06.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.847+0000 7f3204824740 1 -- 172.21.15.47:0/1732139680 shutdown_connections 2024-03-21T11:42:06.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.848+0000 7f3204824740 1 -- 172.21.15.47:0/1732139680 wait complete. 2024-03-21T11:42:06.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.848+0000 7f3204824740 20 client.31658 trim_cache size 0 max 16384 2024-03-21T11:42:06.849 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:42:06.849 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (174 inodes in the stack)", 2024-03-21T11:42:06.850 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:42:06.850 INFO:teuthology.orchestra.run.smithi047.stdout: "16895d3f-ac0e-4979-a84f-d883fd77eb21": { 2024-03-21T11:42:06.850 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:42:06.850 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "16895d3f-ac0e-4979-a84f-d883fd77eb21", 2024-03-21T11:42:06.850 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:42:06.850 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:42:06.850 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:42:06.850 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:42:06.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.851+0000 7f3201c00640 1 -- 172.21.15.47:0/1539673804 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31e00a2fa0 msgr2=0x7f31e00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.851+0000 7f3201c00640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31e00a2fa0 0x7f31e00a5460 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f31e409bdc0 tx=0x7f31e4097a30 comp rx=0 tx=0).stop 2024-03-21T11:42:06.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.851+0000 7f3201c00640 1 -- 172.21.15.47:0/1539673804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31fc074100 msgr2=0x7f31fc19df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:06.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.851+0000 7f3201c00640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31fc074100 0x7f31fc19df80 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f31e400e9f0 tx=0x7f31e400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.851+0000 7f3201c00640 1 -- 172.21.15.47:0/1539673804 shutdown_connections 2024-03-21T11:42:06.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.851+0000 7f3201c00640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31e00a2fa0 0x7f31e00a5460 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.851+0000 7f3201c00640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31fc10dc80 0x7f31fc1a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.851+0000 7f3201c00640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31fc0756e0 0x7f31fc19e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.851+0000 7f3201c00640 1 --2- 172.21.15.47:0/1539673804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31fc074100 0x7f31fc19df80 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:06.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.851+0000 7f3201c00640 1 -- 172.21.15.47:0/1539673804 >> 172.21.15.47:0/1539673804 conn(0x7f31fc0fcc20 msgr2=0x7f31fc101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:06.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.851+0000 7f3201c00640 1 -- 172.21.15.47:0/1539673804 shutdown_connections 2024-03-21T11:42:06.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:06.851+0000 7f3201c00640 1 -- 172.21.15.47:0/1539673804 wait complete. 2024-03-21T11:42:06.874 INFO:tasks.cephfs.filesystem:scrub status for tag:16895d3f-ac0e-4979-a84f-d883fd77eb21 - {'path': '/', 'tag': '16895d3f-ac0e-4979-a84f-d883fd77eb21', 'options': 'recursive,force'} 2024-03-21T11:42:06.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 132847 23.62 MB/sec execute 170 sec latency 113.877 ms 2024-03-21T11:42:07.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:07 smithi047 ceph-mon[31351]: pgmap v963: 129 pgs: 129 active+clean; 499 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.2 MiB/s wr, 636 op/s 2024-03-21T11:42:07.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:07 smithi100 ceph-mon[36245]: pgmap v963: 129 pgs: 129 active+clean; 499 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.2 MiB/s wr, 636 op/s 2024-03-21T11:42:07.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:07 smithi102 ceph-mon[36748]: pgmap v963: 129 pgs: 129 active+clean; 499 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.2 MiB/s wr, 636 op/s 2024-03-21T11:42:07.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 133326 23.62 MB/sec execute 171 sec latency 99.231 ms 2024-03-21T11:42:08.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:42:08.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:08.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:08.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:08.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:08.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:08.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:08.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:08.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:08.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:08 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:42:08.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:08.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:08.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:08.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:08.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:08.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:08.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:08.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:08.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:08 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:42:08.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:08.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:08.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:08.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:08.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:08.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:08.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:08.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:08.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:08 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.771+0000 7fd5dec00640 1 -- 172.21.15.47:0/537127863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5d810ff20 msgr2=0x7fd5d8112310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.771+0000 7fd5dec00640 1 --2- 172.21.15.47:0/537127863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5d810ff20 0x7fd5d8112310 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009ca0 tx=0x7fd5d002c100 comp rx=0 tx=0).stop 2024-03-21T11:42:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.772+0000 7fd5dec00640 1 -- 172.21.15.47:0/537127863 shutdown_connections 2024-03-21T11:42:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.772+0000 7fd5dec00640 1 --2- 172.21.15.47:0/537127863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5d810ff20 0x7fd5d8112310 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.772+0000 7fd5dec00640 1 --2- 172.21.15.47:0/537127863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d8109000 0x7fd5d810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.772+0000 7fd5dec00640 1 --2- 172.21.15.47:0/537127863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d81086a0 0x7fd5d8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.772+0000 7fd5dec00640 1 -- 172.21.15.47:0/537127863 >> 172.21.15.47:0/537127863 conn(0x7fd5d806e6e0 msgr2=0x7fd5d8070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.773+0000 7fd5dec00640 1 -- 172.21.15.47:0/537127863 shutdown_connections 2024-03-21T11:42:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.774+0000 7fd5dec00640 1 -- 172.21.15.47:0/537127863 wait complete. 2024-03-21T11:42:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.775+0000 7fd5dec00640 1 Processor -- start 2024-03-21T11:42:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.776+0000 7fd5dec00640 1 -- start start 2024-03-21T11:42:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.776+0000 7fd5dec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d81086a0 0x7fd5d819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.776+0000 7fd5dec00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d8109000 0x7fd5d819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.776+0000 7fd5dec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5d810ff20 0x7fd5d81b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.776+0000 7fd5dec00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d8115c00 con 0x7fd5d8109000 2024-03-21T11:42:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.776+0000 7fd5dec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d8115a80 con 0x7fd5d81086a0 2024-03-21T11:42:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.776+0000 7fd5dec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d8115d80 con 0x7fd5d810ff20 2024-03-21T11:42:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.776+0000 7fd5d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d81086a0 0x7fd5d819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.776+0000 7fd5d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d81086a0 0x7fd5d819e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57294/0 (socket says 172.21.15.47:57294) 2024-03-21T11:42:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.776+0000 7fd5d7e00640 1 -- 172.21.15.47:0/2633758450 learned_addr learned my addr 172.21.15.47:0/2633758450 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.776+0000 7fd5d7400640 1 --2- 172.21.15.47:0/2633758450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d8109000 0x7fd5d819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.777+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/2633758450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5d810ff20 0x7fd5d81b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.777+0000 7fd5d7e00640 1 -- 172.21.15.47:0/2633758450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5d810ff20 msgr2=0x7fd5d81b6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.777+0000 7fd5d7e00640 1 --2- 172.21.15.47:0/2633758450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5d810ff20 0x7fd5d81b6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.777+0000 7fd5d7e00640 1 -- 172.21.15.47:0/2633758450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d8109000 msgr2=0x7fd5d819e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.777+0000 7fd5d7e00640 1 --2- 172.21.15.47:0/2633758450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d8109000 0x7fd5d819e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.777+0000 7fd5d7e00640 1 -- 172.21.15.47:0/2633758450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d81a36f0 con 0x7fd5d81086a0 2024-03-21T11:42:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.778+0000 7fd5d7e00640 1 --2- 172.21.15.47:0/2633758450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d81086a0 0x7fd5d819e020 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800b4a0 tx=0x7fd5c800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.780+0000 7fd5d4c00640 1 -- 172.21.15.47:0/2633758450 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8015020 con 0x7fd5d81086a0 2024-03-21T11:42:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.781+0000 7fd5d4c00640 1 -- 172.21.15.47:0/2633758450 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5c8011d10 con 0x7fd5d81086a0 2024-03-21T11:42:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.781+0000 7fd5d4c00640 1 -- 172.21.15.47:0/2633758450 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800f2e0 con 0x7fd5d81086a0 2024-03-21T11:42:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.781+0000 7fd5dec00640 1 -- 172.21.15.47:0/2633758450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d81b7050 con 0x7fd5d81086a0 2024-03-21T11:42:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.781+0000 7fd5dec00640 1 -- 172.21.15.47:0/2633758450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d81b74e0 con 0x7fd5d81086a0 2024-03-21T11:42:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.786+0000 7fd5d4c00640 1 -- 172.21.15.47:0/2633758450 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd5c8018020 con 0x7fd5d81086a0 2024-03-21T11:42:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.788+0000 7fd5e1a06740 1 Processor -- start 2024-03-21T11:42:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.788+0000 7fd5e1a06740 1 -- start start 2024-03-21T11:42:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.789+0000 7fd5e1a06740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff801e0 0x559f9ff80600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.789+0000 7fd5e1a06740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff80bd0 0x559f9ff856d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.789+0000 7fd5e1a06740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff85d40 0x559f9ff881d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.789+0000 7fd5d4c00640 1 --2- 172.21.15.47:0/2633758450 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5bc0a31b0 0x7fd5bc0a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.789+0000 7fd5e1a06740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9ff433b0 con 0x559f9ff85d40 2024-03-21T11:42:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.789+0000 7fd5e1a06740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9fee5830 con 0x559f9ff80bd0 2024-03-21T11:42:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.789+0000 7fd5e1a06740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9fe8e5d0 con 0x559f9ff801e0 2024-03-21T11:42:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.789+0000 7fd5d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff80bd0 0x559f9ff856d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.789+0000 7fd5dcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff801e0 0x559f9ff80600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.789+0000 7fd5d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff80bd0 0x559f9ff856d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57304/0 (socket says 172.21.15.47:57304) 2024-03-21T11:42:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.789+0000 7fd5d7e00640 1 -- 172.21.15.47:0/4136457951 learned_addr learned my addr 172.21.15.47:0/4136457951 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.789+0000 7fd5d4c00640 1 -- 172.21.15.47:0/2633758450 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd5c80cef90 con 0x7fd5d81086a0 2024-03-21T11:42:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.789+0000 7fd5d7400640 1 --2- 172.21.15.47:0/4136457951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff85d40 0x559f9ff881d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.790+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/2633758450 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5bc0a31b0 0x7fd5bc0a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.790+0000 7fd5d7e00640 1 -- 172.21.15.47:0/4136457951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff801e0 msgr2=0x559f9ff80600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.790+0000 7fd5d7e00640 1 --2- 172.21.15.47:0/4136457951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff801e0 0x559f9ff80600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.790+0000 7fd5d7e00640 1 -- 172.21.15.47:0/4136457951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff85d40 msgr2=0x559f9ff881d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.790+0000 7fd5d7e00640 1 --2- 172.21.15.47:0/4136457951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff85d40 0x559f9ff881d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.790+0000 7fd5d7e00640 1 -- 172.21.15.47:0/4136457951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f9fea62c0 con 0x559f9ff80bd0 2024-03-21T11:42:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.790+0000 7fd5d7e00640 1 --2- 172.21.15.47:0/4136457951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff80bd0 0x559f9ff856d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8097730 tx=0x7fd5c801fea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=229e219eecea7a1f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.791+0000 7fd5cd200640 1 -- 172.21.15.47:0/4136457951 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c809ac30 con 0x559f9ff80bd0 2024-03-21T11:42:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.791+0000 7fd5cd200640 1 -- 172.21.15.47:0/4136457951 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5c809b9d0 con 0x559f9ff80bd0 2024-03-21T11:42:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.791+0000 7fd5cd200640 1 -- 172.21.15.47:0/4136457951 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c80092b0 con 0x559f9ff80bd0 2024-03-21T11:42:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.791+0000 7fd5e1a06740 1 -- 172.21.15.47:0/4136457951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff80bd0 msgr2=0x559f9ff856d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.791+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/4136457951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff80bd0 0x559f9ff856d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8097730 tx=0x7fd5c801fea0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.792+0000 7fd5e1a06740 1 -- 172.21.15.47:0/4136457951 shutdown_connections 2024-03-21T11:42:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.792+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/4136457951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff85d40 0x559f9ff881d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.792+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/4136457951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff80bd0 0x559f9ff856d0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.792+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/4136457951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff801e0 0x559f9ff80600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.792+0000 7fd5e1a06740 1 -- 172.21.15.47:0/4136457951 >> 172.21.15.47:0/4136457951 conn(0x559f9ff7b750 msgr2=0x559f9ff7bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.792+0000 7fd5e1a06740 1 -- 172.21.15.47:0/4136457951 shutdown_connections 2024-03-21T11:42:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.793+0000 7fd5e1a06740 1 -- 172.21.15.47:0/4136457951 wait complete. 2024-03-21T11:42:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.793+0000 7fd5e1a06740 1 -- start start 2024-03-21T11:42:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.794+0000 7fd5e1a06740 1 Processor -- start 2024-03-21T11:42:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.795+0000 7fd5e1a06740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff801e0 0x559f9ff8a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.795+0000 7fd5e1a06740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff80bd0 0x559f9ff8a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.795+0000 7fd5e1a06740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff85d40 0x559f9ff918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.795+0000 7fd5e1a06740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9fea1a20 con 0x559f9ff801e0 2024-03-21T11:42:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.795+0000 7fd5e1a06740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9fee18d0 con 0x559f9ff85d40 2024-03-21T11:42:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.795+0000 7fd5e1a06740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9ff88c30 con 0x559f9ff80bd0 2024-03-21T11:42:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.795+0000 7fd5e1a06740 10 client.0 _ll_register_callbacks cb 0x559f9fede1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.795+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/2633758450 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5bc0a31b0 0x7fd5bc0a5670 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd5d002c640 tx=0x7fd5d002c100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.795+0000 7fd5dcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff85d40 0x559f9ff918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.795+0000 7fd5dcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff85d40 0x559f9ff918b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57318/0 (socket says 172.21.15.47:57318) 2024-03-21T11:42:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.795+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff80bd0 0x559f9ff8a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.795+0000 7fd5dcc00640 1 -- 172.21.15.47:0/3842150360 learned_addr learned my addr 172.21.15.47:0/3842150360 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.796+0000 7fd5dcc00640 1 -- 172.21.15.47:0/3842150360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff80bd0 msgr2=0x559f9ff8a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.796+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/3842150360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff80bd0 0x559f9ff8a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.796+0000 7fd5dcc00640 1 -- 172.21.15.47:0/3842150360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff801e0 msgr2=0x559f9ff8a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.796+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/3842150360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff801e0 0x559f9ff8a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.796+0000 7fd5dcc00640 1 -- 172.21.15.47:0/3842150360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f9fbfa210 con 0x559f9ff85d40 2024-03-21T11:42:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.796+0000 7fd5d7400640 1 --2- 172.21.15.47:0/3842150360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff80bd0 0x559f9ff8a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.796+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/3842150360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff85d40 0x559f9ff918b0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009ca0 tx=0x7fd5d00095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9fb64974729e66 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.796+0000 7fd5c1600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.798+0000 7fd5c1600640 1 -- 172.21.15.47:0/3842150360 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0053060 con 0x559f9ff85d40 2024-03-21T11:42:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.799+0000 7fd5c1600640 1 -- 172.21.15.47:0/3842150360 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd5d0004790 con 0x559f9ff85d40 2024-03-21T11:42:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.799+0000 7fd5e1a06740 1 -- 172.21.15.47:0/3842150360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5c809b9d0 con 0x559f9ff85d40 2024-03-21T11:42:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.800+0000 7fd5c1600640 1 -- 172.21.15.47:0/3842150360 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5d0039070 con 0x559f9ff85d40 2024-03-21T11:42:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.800+0000 7fd5e1a06740 10 client.31667 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.801+0000 7fd5e1a06740 1 -- 172.21.15.47:0/3842150360 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559f9fcc6620 con 0x559f9ff85d40 2024-03-21T11:42:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.801+0000 7fd5c1600640 1 -- 172.21.15.47:0/3842150360 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5c809b9d0 con 0x559f9ff85d40 2024-03-21T11:42:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.801+0000 7fd5c1600640 1 -- 172.21.15.47:0/3842150360 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd5d00435d0 con 0x559f9ff85d40 2024-03-21T11:42:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.802+0000 7fd5e1a06740 10 client.31667 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.802+0000 7fd5e1a06740 10 client.31667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.802+0000 7fd5e1a06740 20 client.31667 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.802+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/3842150360 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f9ff961f0 0x559f9ff98600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.802+0000 7fd5e1a06740 4 client.31667 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.802+0000 7fd5e1a06740 1 -- 172.21.15.47:0/3842150360 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f9fef2d70 con 0x559f9ff961f0 2024-03-21T11:42:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.804+0000 7fd5d7e00640 1 --2- 172.21.15.47:0/3842150360 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f9ff961f0 0x559f9ff98600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.805+0000 7fd5d7e00640 1 --2- 172.21.15.47:0/3842150360 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f9ff961f0 0x559f9ff98600 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce703df91c77a2 server_cookie=53a8ef6fa15ca124 in_seq=0 out_seq=0 2024-03-21T11:42:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.805+0000 7fd5c1600640 10 client.31667 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.811+0000 7fd5c1600640 1 -- 172.21.15.47:0/3842150360 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559f9fef2d70 con 0x559f9ff961f0 2024-03-21T11:42:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.811+0000 7fd5c1600640 10 client.31667 handle_command_reply: tid=0 2024-03-21T11:42:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.811+0000 7fd5e1a06740 1 client.31667 shutdown 2024-03-21T11:42:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.811+0000 7fd5e1a06740 1 -- 172.21.15.47:0/3842150360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff85d40 msgr2=0x559f9ff918b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.811+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/3842150360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff85d40 0x559f9ff918b0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009ca0 tx=0x7fd5d00095c0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.812+0000 7fd5e1a06740 1 -- 172.21.15.47:0/3842150360 shutdown_connections 2024-03-21T11:42:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.812+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/3842150360 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f9ff961f0 0x559f9ff98600 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.812+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/3842150360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff85d40 0x559f9ff918b0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.812+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/3842150360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff80bd0 0x559f9ff8a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.812+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/3842150360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff801e0 0x559f9ff8a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.812+0000 7fd5e1a06740 1 -- 172.21.15.47:0/3842150360 >> 172.21.15.47:0/3842150360 conn(0x559f9ff7b750 msgr2=0x559f9ff7a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.812+0000 7fd5c1600640 0 client.31667 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.812+0000 7fd5e1a06740 1 -- 172.21.15.47:0/3842150360 shutdown_connections 2024-03-21T11:42:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.812+0000 7fd5e1a06740 1 -- 172.21.15.47:0/3842150360 wait complete. 2024-03-21T11:42:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.812+0000 7fd5e1a06740 20 client.31667 trim_cache size 0 max 16384 2024-03-21T11:42:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.839+0000 7fd5e1a06740 1 Processor -- start 2024-03-21T11:42:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.839+0000 7fd5e1a06740 1 -- start start 2024-03-21T11:42:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.839+0000 7fd5e1a06740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff801e0 0x559f9ff89d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.839+0000 7fd5e1a06740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff80bd0 0x559f9ff91870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.839+0000 7fd5e1a06740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff85d40 0x559f9ff763a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.839+0000 7fd5e1a06740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d81c17c0 con 0x559f9ff801e0 2024-03-21T11:42:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.839+0000 7fd5e1a06740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d81c3140 con 0x559f9ff85d40 2024-03-21T11:42:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.839+0000 7fd5e1a06740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d81c1e20 con 0x559f9ff80bd0 2024-03-21T11:42:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.840+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff80bd0 0x559f9ff91870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.840+0000 7fd5d7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff80bd0 0x559f9ff91870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58726/0 (socket says 172.21.15.47:58726) 2024-03-21T11:42:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.840+0000 7fd5d7400640 1 -- 172.21.15.47:0/59835077 learned_addr learned my addr 172.21.15.47:0/59835077 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.840+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/59835077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff85d40 0x559f9ff763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.841+0000 7fd5d7400640 1 -- 172.21.15.47:0/59835077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff85d40 msgr2=0x559f9ff763a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.841+0000 7fd5d7400640 1 --2- 172.21.15.47:0/59835077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff85d40 0x559f9ff763a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.841+0000 7fd5d7400640 1 -- 172.21.15.47:0/59835077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff801e0 msgr2=0x559f9ff89d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.841+0000 7fd5d7400640 1 --2- 172.21.15.47:0/59835077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff801e0 0x559f9ff89d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.841+0000 7fd5d7400640 1 -- 172.21.15.47:0/59835077 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f9ff88d70 con 0x559f9ff80bd0 2024-03-21T11:42:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.841+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/59835077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff85d40 0x559f9ff763a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.841+0000 7fd5d7400640 1 --2- 172.21.15.47:0/59835077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff80bd0 0x559f9ff91870 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40099c0 tx=0x7fd5c4009110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7531a5b0c825389f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.842+0000 7fd5c3e00640 1 -- 172.21.15.47:0/59835077 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4019030 con 0x559f9ff80bd0 2024-03-21T11:42:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.843+0000 7fd5c3e00640 1 -- 172.21.15.47:0/59835077 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5c4012df0 con 0x559f9ff80bd0 2024-03-21T11:42:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.843+0000 7fd5c3e00640 1 -- 172.21.15.47:0/59835077 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400f040 con 0x559f9ff80bd0 2024-03-21T11:42:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.844+0000 7fd5e1a06740 1 -- 172.21.15.47:0/59835077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff80bd0 msgr2=0x559f9ff91870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.844+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/59835077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff80bd0 0x559f9ff91870 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40099c0 tx=0x7fd5c4009110 comp rx=0 tx=0).stop 2024-03-21T11:42:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.844+0000 7fd5e1a06740 1 -- 172.21.15.47:0/59835077 shutdown_connections 2024-03-21T11:42:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.844+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/59835077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff85d40 0x559f9ff763a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.844+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/59835077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff80bd0 0x559f9ff91870 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.844+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/59835077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff801e0 0x559f9ff89d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.844+0000 7fd5e1a06740 1 -- 172.21.15.47:0/59835077 >> 172.21.15.47:0/59835077 conn(0x559f9ff7b750 msgr2=0x559f9ff89600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.844+0000 7fd5e1a06740 1 -- 172.21.15.47:0/59835077 shutdown_connections 2024-03-21T11:42:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.844+0000 7fd5e1a06740 1 -- 172.21.15.47:0/59835077 wait complete. 2024-03-21T11:42:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.845+0000 7fd5e1a06740 1 -- start start 2024-03-21T11:42:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.845+0000 7fd5e1a06740 1 Processor -- start 2024-03-21T11:42:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.846+0000 7fd5e1a06740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff801e0 0x559f9ff8efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.846+0000 7fd5e1a06740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff80bd0 0x559f9ff8f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.846+0000 7fd5e1a06740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff85d40 0x559f9ff90530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.846+0000 7fd5e1a06740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d81c2ae0 con 0x559f9ff801e0 2024-03-21T11:42:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.846+0000 7fd5e1a06740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d81c2480 con 0x559f9ff80bd0 2024-03-21T11:42:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.846+0000 7fd5e1a06740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d81bf980 con 0x559f9ff85d40 2024-03-21T11:42:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.846+0000 7fd5e1a06740 10 client.0 _ll_register_callbacks cb 0x7fd5d0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.847+0000 7fd5dcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff85d40 0x559f9ff90530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.847+0000 7fd5dcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff85d40 0x559f9ff90530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:58740/0 (socket says 172.21.15.47:58740) 2024-03-21T11:42:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.847+0000 7fd5dcc00640 1 -- 172.21.15.47:0/266156930 learned_addr learned my addr 172.21.15.47:0/266156930 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.847+0000 7fd5d7e00640 1 --2- 172.21.15.47:0/266156930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff801e0 0x559f9ff8efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.847+0000 7fd5dcc00640 1 -- 172.21.15.47:0/266156930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff80bd0 msgr2=0x559f9ff8f520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:42:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.847+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/266156930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff80bd0 0x559f9ff8f520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.847+0000 7fd5dcc00640 1 -- 172.21.15.47:0/266156930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff801e0 msgr2=0x559f9ff8efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.848+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/266156930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff801e0 0x559f9ff8efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.848+0000 7fd5dcc00640 1 -- 172.21.15.47:0/266156930 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f9ff90b20 con 0x559f9ff85d40 2024-03-21T11:42:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.848+0000 7fd5d7e00640 1 --2- 172.21.15.47:0/266156930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff801e0 0x559f9ff8efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:42:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.848+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/266156930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff85d40 0x559f9ff90530 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0037040 tx=0x7fd5d00437a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e331c98f268c2b84 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.848+0000 7fd5c1600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:42:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.849+0000 7fd5c1600640 1 -- 172.21.15.47:0/266156930 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0066060 con 0x559f9ff85d40 2024-03-21T11:42:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.849+0000 7fd5c1600640 1 -- 172.21.15.47:0/266156930 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd5d0004620 con 0x559f9ff85d40 2024-03-21T11:42:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.849+0000 7fd5e1a06740 1 -- 172.21.15.47:0/266156930 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5c4012df0 con 0x559f9ff85d40 2024-03-21T11:42:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.849+0000 7fd5c1600640 1 -- 172.21.15.47:0/266156930 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5d0053040 con 0x559f9ff85d40 2024-03-21T11:42:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.849+0000 7fd5e1a06740 10 client.31645 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.849+0000 7fd5e1a06740 1 -- 172.21.15.47:0/266156930 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559f9fff1c00 con 0x559f9ff85d40 2024-03-21T11:42:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.850+0000 7fd5c1600640 1 -- 172.21.15.47:0/266156930 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5c4012df0 con 0x559f9ff85d40 2024-03-21T11:42:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.850+0000 7fd5c1600640 1 -- 172.21.15.47:0/266156930 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd5d0034440 con 0x559f9ff85d40 2024-03-21T11:42:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.850+0000 7fd5e1a06740 10 client.31645 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.850+0000 7fd5e1a06740 10 client.31645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.850+0000 7fd5e1a06740 20 client.31645 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.850+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/266156930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f9fff2560 0x559f9fff49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.850+0000 7fd5e1a06740 4 client.31645 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:42:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.850+0000 7fd5e1a06740 1 -- 172.21.15.47:0/266156930 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559f9ff76a80 con 0x559f9fff2560 2024-03-21T11:42:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.853+0000 7fd5d7e00640 1 --2- 172.21.15.47:0/266156930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f9fff2560 0x559f9fff49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.854+0000 7fd5d7e00640 1 --2- 172.21.15.47:0/266156930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f9fff2560 0x559f9fff49a0 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6841a7f13bec91d9 server_cookie=646c7947096c12c7 in_seq=0 out_seq=0 2024-03-21T11:42:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:08.854+0000 7fd5c1600640 10 client.31645 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:08.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 133810 23.63 MB/sec execute 172 sec latency 111.391 ms 2024-03-21T11:42:09.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:09 smithi100 ceph-mon[36245]: pgmap v964: 129 pgs: 129 active+clean; 501 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 538 op/s 2024-03-21T11:42:09.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:09 smithi102 ceph-mon[36748]: pgmap v964: 129 pgs: 129 active+clean; 501 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 538 op/s 2024-03-21T11:42:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.505+0000 7fd5c1600640 1 -- 172.21.15.47:0/266156930 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559f9ff76a80 con 0x559f9fff2560 2024-03-21T11:42:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.505+0000 7fd5c1600640 10 client.31645 handle_command_reply: tid=0 2024-03-21T11:42:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.505+0000 7fd5e1a06740 1 client.31645 shutdown 2024-03-21T11:42:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.505+0000 7fd5e1a06740 1 -- 172.21.15.47:0/266156930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff85d40 msgr2=0x559f9ff90530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.505+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/266156930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff85d40 0x559f9ff90530 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0037040 tx=0x7fd5d00437a0 comp rx=0 tx=0).stop 2024-03-21T11:42:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.505+0000 7fd5e1a06740 1 -- 172.21.15.47:0/266156930 shutdown_connections 2024-03-21T11:42:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.505+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/266156930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f9fff2560 0x559f9fff49a0 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.505+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/266156930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f9ff85d40 0x559f9ff90530 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.505+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/266156930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9ff80bd0 0x559f9ff8f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.505+0000 7fd5e1a06740 1 --2- 172.21.15.47:0/266156930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9ff801e0 0x559f9ff8efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.505+0000 7fd5e1a06740 1 -- 172.21.15.47:0/266156930 >> 172.21.15.47:0/266156930 conn(0x559f9ff7b750 msgr2=0x559f9ff7e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.506+0000 7fd5c1600640 0 client.31645 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.506+0000 7fd5e1a06740 1 -- 172.21.15.47:0/266156930 shutdown_connections 2024-03-21T11:42:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.506+0000 7fd5e1a06740 1 -- 172.21.15.47:0/266156930 wait complete. 2024-03-21T11:42:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.506+0000 7fd5e1a06740 20 client.31645 trim_cache size 0 max 16384 2024-03-21T11:42:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:42:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:42:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:42:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:42:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.507+0000 7fd5dec00640 1 -- 172.21.15.47:0/2633758450 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5bc0a31b0 msgr2=0x7fd5bc0a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.507+0000 7fd5dec00640 1 --2- 172.21.15.47:0/2633758450 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5bc0a31b0 0x7fd5bc0a5670 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd5d002c640 tx=0x7fd5d002c100 comp rx=0 tx=0).stop 2024-03-21T11:42:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.507+0000 7fd5dec00640 1 -- 172.21.15.47:0/2633758450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d81086a0 msgr2=0x7fd5d819e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.507+0000 7fd5dec00640 1 --2- 172.21.15.47:0/2633758450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d81086a0 0x7fd5d819e020 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800b4a0 tx=0x7fd5c800b8a0 comp rx=0 tx=0).stop 2024-03-21T11:42:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.507+0000 7fd5dec00640 1 -- 172.21.15.47:0/2633758450 shutdown_connections 2024-03-21T11:42:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.507+0000 7fd5dec00640 1 --2- 172.21.15.47:0/2633758450 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5bc0a31b0 0x7fd5bc0a5670 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.507+0000 7fd5dec00640 1 --2- 172.21.15.47:0/2633758450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5d810ff20 0x7fd5d81b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.507+0000 7fd5dec00640 1 --2- 172.21.15.47:0/2633758450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d8109000 0x7fd5d819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.507+0000 7fd5dec00640 1 --2- 172.21.15.47:0/2633758450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d81086a0 0x7fd5d819e020 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.507+0000 7fd5dec00640 1 -- 172.21.15.47:0/2633758450 >> 172.21.15.47:0/2633758450 conn(0x7fd5d806e6e0 msgr2=0x7fd5d81097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.507+0000 7fd5dec00640 1 -- 172.21.15.47:0/2633758450 shutdown_connections 2024-03-21T11:42:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:09.507+0000 7fd5dec00640 1 -- 172.21.15.47:0/2633758450 wait complete. 2024-03-21T11:42:09.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:09 smithi047 ceph-mon[31351]: pgmap v964: 129 pgs: 129 active+clean; 501 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 538 op/s 2024-03-21T11:42:09.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 134051 23.56 MB/sec execute 173 sec latency 444.120 ms 2024-03-21T11:42:10.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 134366 23.56 MB/sec execute 174 sec latency 381.971 ms 2024-03-21T11:42:11.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:11 smithi100 ceph-mon[36245]: pgmap v965: 129 pgs: 129 active+clean; 507 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-03-21T11:42:11.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:11 smithi102 ceph-mon[36748]: pgmap v965: 129 pgs: 129 active+clean; 507 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-03-21T11:42:11.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:11 smithi047 ceph-mon[31351]: pgmap v965: 129 pgs: 129 active+clean; 507 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-03-21T11:42:11.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 134748 23.51 MB/sec execute 175 sec latency 161.240 ms 2024-03-21T11:42:12.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 135285 23.55 MB/sec execute 176 sec latency 67.419 ms 2024-03-21T11:42:13.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:13 smithi100 ceph-mon[36245]: pgmap v966: 129 pgs: 129 active+clean; 509 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 706 op/s 2024-03-21T11:42:13.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:13 smithi102 ceph-mon[36748]: pgmap v966: 129 pgs: 129 active+clean; 509 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 706 op/s 2024-03-21T11:42:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:13 smithi047 ceph-mon[31351]: pgmap v966: 129 pgs: 129 active+clean; 509 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 706 op/s 2024-03-21T11:42:13.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 135788 23.59 MB/sec execute 177 sec latency 56.849 ms 2024-03-21T11:42:14.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 136300 23.57 MB/sec execute 178 sec latency 55.171 ms 2024-03-21T11:42:15.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:15 smithi100 ceph-mon[36245]: pgmap v967: 129 pgs: 129 active+clean; 507 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.0 MiB/s wr, 584 op/s 2024-03-21T11:42:15.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:15 smithi102 ceph-mon[36748]: pgmap v967: 129 pgs: 129 active+clean; 507 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.0 MiB/s wr, 584 op/s 2024-03-21T11:42:15.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:15 smithi047 ceph-mon[31351]: pgmap v967: 129 pgs: 129 active+clean; 507 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.0 MiB/s wr, 584 op/s 2024-03-21T11:42:15.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 136571 23.56 MB/sec execute 179 sec latency 461.993 ms 2024-03-21T11:42:16.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:16 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:42:16.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:16 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:42:16.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:16 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:42:16.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 136771 23.46 MB/sec execute 180 sec latency 242.370 ms 2024-03-21T11:42:17.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:17 smithi102 ceph-mon[36748]: pgmap v968: 129 pgs: 129 active+clean; 506 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.5 MiB/s wr, 678 op/s 2024-03-21T11:42:17.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:17 smithi047 ceph-mon[31351]: pgmap v968: 129 pgs: 129 active+clean; 506 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.5 MiB/s wr, 678 op/s 2024-03-21T11:42:17.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:17 smithi100 ceph-mon[36245]: pgmap v968: 129 pgs: 129 active+clean; 506 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.5 MiB/s wr, 678 op/s 2024-03-21T11:42:17.954 INFO:tasks.workunit.client.0.smithi047.stdout: 10 137242 23.48 MB/sec execute 181 sec latency 124.787 ms 2024-03-21T11:42:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:18 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:42:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:18 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:42:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:18 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:42:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:18 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:42:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:18 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:42:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:18 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:42:18.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:18 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:42:18.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:18 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:42:18.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:18 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:42:18.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:18 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:42:18.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:18 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:42:18.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:18 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:42:18.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:18 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:42:18.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:18 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:42:18.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:18 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:42:18.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:18 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:42:18.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:18 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:42:18.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:18 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:42:18.955 INFO:tasks.workunit.client.0.smithi047.stdout: 10 137625 23.47 MB/sec execute 182 sec latency 273.734 ms 2024-03-21T11:42:19.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:19 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:42:19.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:19 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:42:19.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:19 smithi102 ceph-mon[36748]: pgmap v969: 129 pgs: 129 active+clean; 507 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.4 MiB/s wr, 627 op/s 2024-03-21T11:42:19.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:19 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:42:19.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:19 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:42:19.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:19 smithi047 ceph-mon[31351]: pgmap v969: 129 pgs: 129 active+clean; 507 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.4 MiB/s wr, 627 op/s 2024-03-21T11:42:19.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:19 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:42:19.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:19 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:42:19.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:19 smithi100 ceph-mon[36245]: pgmap v969: 129 pgs: 129 active+clean; 507 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.4 MiB/s wr, 627 op/s 2024-03-21T11:42:19.955 INFO:tasks.workunit.client.0.smithi047.stdout: 10 138159 23.52 MB/sec execute 183 sec latency 68.846 ms 2024-03-21T11:42:20.955 INFO:tasks.workunit.client.0.smithi047.stdout: 10 138497 23.47 MB/sec execute 184 sec latency 237.161 ms 2024-03-21T11:42:21.445 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:21 smithi100 ceph-mon[36245]: pgmap v970: 129 pgs: 129 active+clean; 510 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.0 MiB/s wr, 646 op/s 2024-03-21T11:42:21.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:21 smithi102 ceph-mon[36748]: pgmap v970: 129 pgs: 129 active+clean; 510 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.0 MiB/s wr, 646 op/s 2024-03-21T11:42:21.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:21 smithi047 ceph-mon[31351]: pgmap v970: 129 pgs: 129 active+clean; 510 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.0 MiB/s wr, 646 op/s 2024-03-21T11:42:21.955 INFO:tasks.workunit.client.0.smithi047.stdout: 10 138729 23.42 MB/sec execute 185 sec latency 246.665 ms 2024-03-21T11:42:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.712+0000 7fe21c000640 1 -- 172.21.15.47:0/3821462415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe214104ae0 msgr2=0x7fe214104f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.712+0000 7fe21c000640 1 --2- 172.21.15.47:0/3821462415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe214104ae0 0x7fe214104f60 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe200009900 tx=0x7fe20002bc30 comp rx=0 tx=0).stop 2024-03-21T11:42:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.712+0000 7fe21c000640 1 -- 172.21.15.47:0/3821462415 shutdown_connections 2024-03-21T11:42:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.712+0000 7fe21c000640 1 --2- 172.21.15.47:0/3821462415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2141054a0 0x7fe21410bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.712+0000 7fe21c000640 1 --2- 172.21.15.47:0/3821462415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe214104ae0 0x7fe214104f60 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.712+0000 7fe21c000640 1 --2- 172.21.15.47:0/3821462415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe2141038f0 0x7fe214103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.712+0000 7fe21c000640 1 -- 172.21.15.47:0/3821462415 >> 172.21.15.47:0/3821462415 conn(0x7fe2140fee60 msgr2=0x7fe2141012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.712+0000 7fe21c000640 1 -- 172.21.15.47:0/3821462415 shutdown_connections 2024-03-21T11:42:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.712+0000 7fe21c000640 1 -- 172.21.15.47:0/3821462415 wait complete. 2024-03-21T11:42:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.713+0000 7fe21c000640 1 Processor -- start 2024-03-21T11:42:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.714+0000 7fe21c000640 1 -- start start 2024-03-21T11:42:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.714+0000 7fe21c000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2141038f0 0x7fe214199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.714+0000 7fe21c000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe214104ae0 0x7fe21419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.714+0000 7fe21c000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2141054a0 0x7fe2141a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.714+0000 7fe21c000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21410f570 con 0x7fe214104ae0 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.714+0000 7fe21c000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21410f3f0 con 0x7fe2141054a0 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.714+0000 7fe21c000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21410f6f0 con 0x7fe2141038f0 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.714+0000 7fe219600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2141038f0 0x7fe214199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.714+0000 7fe21a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2141054a0 0x7fe2141a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.714+0000 7fe21a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2141054a0 0x7fe2141a10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34320/0 (socket says 172.21.15.47:34320) 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.714+0000 7fe21a000640 1 -- 172.21.15.47:0/65769344 learned_addr learned my addr 172.21.15.47:0/65769344 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.714+0000 7fe218c00640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe214104ae0 0x7fe21419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.715+0000 7fe21a000640 1 -- 172.21.15.47:0/65769344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2141038f0 msgr2=0x7fe214199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.715+0000 7fe21a000640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2141038f0 0x7fe214199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.715+0000 7fe21a000640 1 -- 172.21.15.47:0/65769344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe214104ae0 msgr2=0x7fe21419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.715+0000 7fe21a000640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe214104ae0 0x7fe21419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.715+0000 7fe21a000640 1 -- 172.21.15.47:0/65769344 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2141a1750 con 0x7fe2141054a0 2024-03-21T11:42:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.715+0000 7fe219600640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2141038f0 0x7fe214199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.715+0000 7fe218c00640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe214104ae0 0x7fe21419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.715+0000 7fe21a000640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2141054a0 0x7fe2141a10e0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00e970 tx=0x7fe20c00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.715+0000 7fe212000640 1 -- 172.21.15.47:0/65769344 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c00ccc0 con 0x7fe2141054a0 2024-03-21T11:42:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.715+0000 7fe212000640 1 -- 172.21.15.47:0/65769344 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe20c00ce60 con 0x7fe2141054a0 2024-03-21T11:42:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.715+0000 7fe212000640 1 -- 172.21.15.47:0/65769344 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c010490 con 0x7fe2141054a0 2024-03-21T11:42:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.715+0000 7fe21c000640 1 -- 172.21.15.47:0/65769344 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2141a1a40 con 0x7fe2141054a0 2024-03-21T11:42:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.716+0000 7fe21c000640 1 -- 172.21.15.47:0/65769344 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2141a1f10 con 0x7fe2141054a0 2024-03-21T11:42:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.719+0000 7fe21ec19740 1 Processor -- start 2024-03-21T11:42:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.721+0000 7fe212000640 1 -- 172.21.15.47:0/65769344 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe20c005020 con 0x7fe2141054a0 2024-03-21T11:42:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.721+0000 7fe21ec19740 1 -- start start 2024-03-21T11:42:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.721+0000 7fe21ec19740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2529f0 0x55c4cb252e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.721+0000 7fe21ec19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2533e0 0x55c4cb257ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.721+0000 7fe21ec19740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 0x55c4cb25a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.721+0000 7fe21ec19740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4cb20d3b0 con 0x55c4cb258550 2024-03-21T11:42:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.721+0000 7fe21ec19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4cb1af830 con 0x55c4cb2533e0 2024-03-21T11:42:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.721+0000 7fe21ec19740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4cb1585d0 con 0x55c4cb2529f0 2024-03-21T11:42:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.722+0000 7fe218c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2529f0 0x55c4cb252e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.722+0000 7fe219600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2533e0 0x55c4cb257ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.722+0000 7fe218c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2529f0 0x55c4cb252e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53420/0 (socket says 172.21.15.47:53420) 2024-03-21T11:42:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.722+0000 7fe218c00640 1 -- 172.21.15.47:0/4127640719 learned_addr learned my addr 172.21.15.47:0/4127640719 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.723+0000 7fe219600640 1 -- 172.21.15.47:0/4127640719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2529f0 msgr2=0x55c4cb252e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.723+0000 7fe219600640 1 --2- 172.21.15.47:0/4127640719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2529f0 0x55c4cb252e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.723+0000 7fe219600640 1 -- 172.21.15.47:0/4127640719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 msgr2=0x55c4cb25a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.723+0000 7fe219600640 1 --2- 172.21.15.47:0/4127640719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 0x55c4cb25a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.723+0000 7fe219600640 1 -- 172.21.15.47:0/4127640719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4cb1702c0 con 0x55c4cb2533e0 2024-03-21T11:42:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.723+0000 7fe218c00640 1 --2- 172.21.15.47:0/4127640719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2529f0 0x55c4cb252e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.723+0000 7fe212000640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe1f80a3040 0x7fe1f80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.723+0000 7fe219600640 1 --2- 172.21.15.47:0/4127640719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2533e0 0x55c4cb257ee0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fe204005fd0 tx=0x7fe204011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1006996f826cbad2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.723+0000 7fe20a600640 1 -- 172.21.15.47:0/4127640719 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204011e60 con 0x55c4cb2533e0 2024-03-21T11:42:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.724+0000 7fe218c00640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe1f80a3040 0x7fe1f80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.724+0000 7fe20a600640 1 -- 172.21.15.47:0/4127640719 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe204003e50 con 0x55c4cb2533e0 2024-03-21T11:42:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.724+0000 7fe20a600640 1 -- 172.21.15.47:0/4127640719 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2040047f0 con 0x55c4cb2533e0 2024-03-21T11:42:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.724+0000 7fe212000640 1 -- 172.21.15.47:0/65769344 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe20c0cfc10 con 0x7fe2141054a0 2024-03-21T11:42:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.725+0000 7fe21ec19740 1 -- 172.21.15.47:0/4127640719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2533e0 msgr2=0x55c4cb257ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.725+0000 7fe21ec19740 1 --2- 172.21.15.47:0/4127640719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2533e0 0x55c4cb257ee0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fe204005fd0 tx=0x7fe204011c70 comp rx=0 tx=0).stop 2024-03-21T11:42:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.725+0000 7fe218c00640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe1f80a3040 0x7fe1f80a5500 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe200005ab0 tx=0x7fe20002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.725+0000 7fe21ec19740 1 -- 172.21.15.47:0/4127640719 shutdown_connections 2024-03-21T11:42:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.725+0000 7fe21ec19740 1 --2- 172.21.15.47:0/4127640719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 0x55c4cb25a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.725+0000 7fe21ec19740 1 --2- 172.21.15.47:0/4127640719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2533e0 0x55c4cb257ee0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.725+0000 7fe21ec19740 1 --2- 172.21.15.47:0/4127640719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2529f0 0x55c4cb252e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.725+0000 7fe21ec19740 1 -- 172.21.15.47:0/4127640719 >> 172.21.15.47:0/4127640719 conn(0x55c4cb245750 msgr2=0x55c4cb245b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:22.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.726+0000 7fe21ec19740 1 -- 172.21.15.47:0/4127640719 shutdown_connections 2024-03-21T11:42:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.727+0000 7fe21ec19740 1 -- 172.21.15.47:0/4127640719 wait complete. 2024-03-21T11:42:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.727+0000 7fe21ec19740 1 -- start start 2024-03-21T11:42:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.727+0000 7fe21ec19740 1 Processor -- start 2024-03-21T11:42:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.728+0000 7fe21ec19740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2529f0 0x55c4cb25cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.728+0000 7fe21ec19740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb2533e0 0x55c4cb25d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.728+0000 7fe21ec19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb258550 0x55c4cb264180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.728+0000 7fe21ec19740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4cb16ba20 con 0x55c4cb2533e0 2024-03-21T11:42:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.728+0000 7fe21ec19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4cb1ab8d0 con 0x55c4cb258550 2024-03-21T11:42:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.728+0000 7fe21ec19740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4cb25b500 con 0x55c4cb2529f0 2024-03-21T11:42:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.728+0000 7fe21ec19740 10 client.0 _ll_register_callbacks cb 0x55c4cb1a81c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.728+0000 7fe219600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2529f0 0x55c4cb25cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.729+0000 7fe219600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2529f0 0x55c4cb25cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53436/0 (socket says 172.21.15.47:53436) 2024-03-21T11:42:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.729+0000 7fe219600640 1 -- 172.21.15.47:0/158215996 learned_addr learned my addr 172.21.15.47:0/158215996 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.729+0000 7fe218c00640 1 --2- 172.21.15.47:0/158215996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb2533e0 0x55c4cb25d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.729+0000 7fe219600640 1 -- 172.21.15.47:0/158215996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb258550 msgr2=0x55c4cb264180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.729+0000 7fe219600640 1 --2- 172.21.15.47:0/158215996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb258550 0x55c4cb264180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.729+0000 7fe219600640 1 -- 172.21.15.47:0/158215996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb2533e0 msgr2=0x55c4cb25d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.729+0000 7fe219600640 1 --2- 172.21.15.47:0/158215996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb2533e0 0x55c4cb25d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.729+0000 7fe219600640 1 -- 172.21.15.47:0/158215996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4caec4210 con 0x55c4cb2529f0 2024-03-21T11:42:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.729+0000 7fe218c00640 1 --2- 172.21.15.47:0/158215996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb2533e0 0x55c4cb25d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:22.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.730+0000 7fe219600640 1 --2- 172.21.15.47:0/158215996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2529f0 0x55c4cb25cc00 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fe204005c10 tx=0x7fe204004230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ff38cc21d6efc9f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:22.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.730+0000 7fe1fea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:42:22.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.730+0000 7fe1fea00640 1 -- 172.21.15.47:0/158215996 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204027070 con 0x55c4cb2529f0 2024-03-21T11:42:22.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.730+0000 7fe1fea00640 1 -- 172.21.15.47:0/158215996 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe204011250 con 0x55c4cb2529f0 2024-03-21T11:42:22.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.730+0000 7fe21ec19740 1 -- 172.21.15.47:0/158215996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe204003e50 con 0x55c4cb2529f0 2024-03-21T11:42:22.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.731+0000 7fe1fea00640 1 -- 172.21.15.47:0/158215996 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe204031050 con 0x55c4cb2529f0 2024-03-21T11:42:22.733 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.731+0000 7fe21ec19740 10 client.31657 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.731+0000 7fe21ec19740 1 -- 172.21.15.47:0/158215996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c4caf90620 con 0x55c4cb2529f0 2024-03-21T11:42:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.731+0000 7fe1fea00640 1 -- 172.21.15.47:0/158215996 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe204003e50 con 0x55c4cb2529f0 2024-03-21T11:42:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.731+0000 7fe1fea00640 1 -- 172.21.15.47:0/158215996 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe2040051e0 con 0x55c4cb2529f0 2024-03-21T11:42:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.731+0000 7fe21ec19740 10 client.31657 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.731+0000 7fe21ec19740 10 client.31657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.731+0000 7fe21ec19740 20 client.31657 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.731+0000 7fe21ec19740 1 --2- 172.21.15.47:0/158215996 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c4cb2688b0 0x55c4cb26aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.731+0000 7fe21ec19740 4 client.31657 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.732+0000 7fe21ec19740 1 -- 172.21.15.47:0/158215996 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4cb1bcd70 con 0x55c4cb2688b0 2024-03-21T11:42:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.732+0000 7fe218c00640 1 --2- 172.21.15.47:0/158215996 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c4cb2688b0 0x55c4cb26aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.734+0000 7fe218c00640 1 --2- 172.21.15.47:0/158215996 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c4cb2688b0 0x55c4cb26aca0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e26d24c14d5f1d2e server_cookie=90dc26f207937816 in_seq=0 out_seq=0 2024-03-21T11:42:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.734+0000 7fe1fea00640 10 client.31657 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.736+0000 7fe1fea00640 1 -- 172.21.15.47:0/158215996 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c4cb1bcd70 con 0x55c4cb2688b0 2024-03-21T11:42:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.736+0000 7fe1fea00640 10 client.31657 handle_command_reply: tid=0 2024-03-21T11:42:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.736+0000 7fe21ec19740 1 client.31657 shutdown 2024-03-21T11:42:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.736+0000 7fe21ec19740 1 -- 172.21.15.47:0/158215996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2529f0 msgr2=0x55c4cb25cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.736+0000 7fe21ec19740 1 --2- 172.21.15.47:0/158215996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2529f0 0x55c4cb25cc00 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fe204005c10 tx=0x7fe204004230 comp rx=0 tx=0).stop 2024-03-21T11:42:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.737+0000 7fe21ec19740 1 -- 172.21.15.47:0/158215996 shutdown_connections 2024-03-21T11:42:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.737+0000 7fe21ec19740 1 --2- 172.21.15.47:0/158215996 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c4cb2688b0 0x55c4cb26aca0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.737+0000 7fe21ec19740 1 --2- 172.21.15.47:0/158215996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb258550 0x55c4cb264180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.737+0000 7fe21ec19740 1 --2- 172.21.15.47:0/158215996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb2533e0 0x55c4cb25d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.737+0000 7fe21ec19740 1 --2- 172.21.15.47:0/158215996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2529f0 0x55c4cb25cc00 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.737+0000 7fe21ec19740 1 -- 172.21.15.47:0/158215996 >> 172.21.15.47:0/158215996 conn(0x55c4cb245750 msgr2=0x55c4cb244950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.737+0000 7fe1fea00640 0 client.31657 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.737+0000 7fe21ec19740 1 -- 172.21.15.47:0/158215996 shutdown_connections 2024-03-21T11:42:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.737+0000 7fe21ec19740 1 -- 172.21.15.47:0/158215996 wait complete. 2024-03-21T11:42:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.737+0000 7fe21ec19740 20 client.31657 trim_cache size 0 max 16384 2024-03-21T11:42:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.756+0000 7fe21ec19740 1 Processor -- start 2024-03-21T11:42:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.756+0000 7fe21ec19740 1 -- start start 2024-03-21T11:42:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.757+0000 7fe21ec19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2529f0 0x55c4cb2440c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.757+0000 7fe21ec19740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2533e0 0x55c4cb244600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.757+0000 7fe21ec19740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 0x55c4cb244b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.757+0000 7fe21ec19740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141a7f10 con 0x55c4cb258550 2024-03-21T11:42:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.757+0000 7fe21ec19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141a9890 con 0x55c4cb2529f0 2024-03-21T11:42:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.757+0000 7fe21ec19740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141a8570 con 0x55c4cb2533e0 2024-03-21T11:42:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.757+0000 7fe219600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2529f0 0x55c4cb2440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.757+0000 7fe218c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2533e0 0x55c4cb244600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.757+0000 7fe219600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2529f0 0x55c4cb2440c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34346/0 (socket says 172.21.15.47:34346) 2024-03-21T11:42:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.757+0000 7fe218c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2533e0 0x55c4cb244600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53452/0 (socket says 172.21.15.47:53452) 2024-03-21T11:42:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.757+0000 7fe219600640 1 -- 172.21.15.47:0/1886312633 learned_addr learned my addr 172.21.15.47:0/1886312633 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.757+0000 7fe21a000640 1 --2- 172.21.15.47:0/1886312633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 0x55c4cb244b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.758+0000 7fe218c00640 1 -- 172.21.15.47:0/1886312633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2529f0 msgr2=0x55c4cb2440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.758+0000 7fe218c00640 1 --2- 172.21.15.47:0/1886312633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2529f0 0x55c4cb2440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.758+0000 7fe218c00640 1 -- 172.21.15.47:0/1886312633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 msgr2=0x55c4cb244b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.758+0000 7fe218c00640 1 --2- 172.21.15.47:0/1886312633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 0x55c4cb244b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.758+0000 7fe218c00640 1 -- 172.21.15.47:0/1886312633 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4cb245080 con 0x55c4cb2533e0 2024-03-21T11:42:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.758+0000 7fe218c00640 1 --2- 172.21.15.47:0/1886312633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2533e0 0x55c4cb244600 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x55c4cb253c40 tx=0x7fe20003f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cdc751947a6558a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.758+0000 7fe209c00640 1 -- 172.21.15.47:0/1886312633 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe200056070 con 0x55c4cb2533e0 2024-03-21T11:42:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.758+0000 7fe209c00640 1 -- 172.21.15.47:0/1886312633 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe200045040 con 0x55c4cb2533e0 2024-03-21T11:42:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.759+0000 7fe209c00640 1 -- 172.21.15.47:0/1886312633 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe200005310 con 0x55c4cb2533e0 2024-03-21T11:42:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.759+0000 7fe21ec19740 1 -- 172.21.15.47:0/1886312633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2533e0 msgr2=0x55c4cb244600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.759+0000 7fe21ec19740 1 --2- 172.21.15.47:0/1886312633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2533e0 0x55c4cb244600 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x55c4cb253c40 tx=0x7fe20003f000 comp rx=0 tx=0).stop 2024-03-21T11:42:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.759+0000 7fe21ec19740 1 -- 172.21.15.47:0/1886312633 shutdown_connections 2024-03-21T11:42:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.760+0000 7fe21ec19740 1 --2- 172.21.15.47:0/1886312633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 0x55c4cb244b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.760+0000 7fe21ec19740 1 --2- 172.21.15.47:0/1886312633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2533e0 0x55c4cb244600 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.760+0000 7fe21ec19740 1 --2- 172.21.15.47:0/1886312633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2529f0 0x55c4cb2440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.760+0000 7fe21ec19740 1 -- 172.21.15.47:0/1886312633 >> 172.21.15.47:0/1886312633 conn(0x55c4cb245750 msgr2=0x55c4cb254740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.760+0000 7fe21ec19740 1 -- 172.21.15.47:0/1886312633 shutdown_connections 2024-03-21T11:42:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.760+0000 7fe21ec19740 1 -- 172.21.15.47:0/1886312633 wait complete. 2024-03-21T11:42:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.760+0000 7fe21ec19740 1 -- start start 2024-03-21T11:42:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.760+0000 7fe21ec19740 1 Processor -- start 2024-03-21T11:42:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.761+0000 7fe21ec19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2529f0 0x55c4cb261230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.761+0000 7fe21ec19740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2533e0 0x55c4cb261770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.761+0000 7fe21ec19740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 0x55c4cb262780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.761+0000 7fe21ec19740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141a9230 con 0x55c4cb258550 2024-03-21T11:42:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.761+0000 7fe21ec19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141a8bd0 con 0x55c4cb2529f0 2024-03-21T11:42:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.761+0000 7fe21ec19740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141a60d0 con 0x55c4cb2533e0 2024-03-21T11:42:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.761+0000 7fe21ec19740 10 client.0 _ll_register_callbacks cb 0x7fe204011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.761+0000 7fe218c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2533e0 0x55c4cb261770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.761+0000 7fe219600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2529f0 0x55c4cb261230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.761+0000 7fe218c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2533e0 0x55c4cb261770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53466/0 (socket says 172.21.15.47:53466) 2024-03-21T11:42:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.761+0000 7fe219600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2529f0 0x55c4cb261230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34358/0 (socket says 172.21.15.47:34358) 2024-03-21T11:42:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.761+0000 7fe218c00640 1 -- 172.21.15.47:0/3284123583 learned_addr learned my addr 172.21.15.47:0/3284123583 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.761+0000 7fe21a000640 1 --2- 172.21.15.47:0/3284123583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 0x55c4cb262780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.762+0000 7fe219600640 1 -- 172.21.15.47:0/3284123583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2533e0 msgr2=0x55c4cb261770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.762+0000 7fe219600640 1 --2- 172.21.15.47:0/3284123583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2533e0 0x55c4cb261770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.762+0000 7fe219600640 1 -- 172.21.15.47:0/3284123583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 msgr2=0x55c4cb262780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.762+0000 7fe219600640 1 --2- 172.21.15.47:0/3284123583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 0x55c4cb262780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.762+0000 7fe219600640 1 -- 172.21.15.47:0/3284123583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4cb262d70 con 0x55c4cb2529f0 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.762+0000 7fe21a000640 1 --2- 172.21.15.47:0/3284123583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 0x55c4cb262780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.762+0000 7fe219600640 1 --2- 172.21.15.47:0/3284123583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2529f0 0x55c4cb261230 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe204009370 tx=0x7fe204009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29995e37018c0ede server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.762+0000 7fe1fea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.762+0000 7fe1fea00640 1 -- 172.21.15.47:0/3284123583 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20403a070 con 0x55c4cb2529f0 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.762+0000 7fe1fea00640 1 -- 172.21.15.47:0/3284123583 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe20400f9a0 con 0x55c4cb2529f0 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.762+0000 7fe21ec19740 1 -- 172.21.15.47:0/3284123583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe200045040 con 0x55c4cb2529f0 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.763+0000 7fe1fea00640 1 -- 172.21.15.47:0/3284123583 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe204046050 con 0x55c4cb2529f0 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.763+0000 7fe21ec19740 10 client.31685 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.763+0000 7fe21ec19740 1 -- 172.21.15.47:0/3284123583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c4cb2c42e0 con 0x55c4cb2529f0 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.763+0000 7fe1fea00640 1 -- 172.21.15.47:0/3284123583 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe200045040 con 0x55c4cb2529f0 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.763+0000 7fe1fea00640 1 -- 172.21.15.47:0/3284123583 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe20402cb20 con 0x55c4cb2529f0 2024-03-21T11:42:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.763+0000 7fe21ec19740 10 client.31685 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.763+0000 7fe21ec19740 10 client.31685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.763+0000 7fe21ec19740 20 client.31685 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.763+0000 7fe21ec19740 1 --2- 172.21.15.47:0/3284123583 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c4cb2c4b20 0x55c4cb2c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.763+0000 7fe21ec19740 4 client.31685 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:42:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.763+0000 7fe21ec19740 1 -- 172.21.15.47:0/3284123583 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c4cb1b70c0 con 0x55c4cb2c4b20 2024-03-21T11:42:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.763+0000 7fe218c00640 1 --2- 172.21.15.47:0/3284123583 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c4cb2c4b20 0x55c4cb2c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.765+0000 7fe218c00640 1 --2- 172.21.15.47:0/3284123583 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c4cb2c4b20 0x55c4cb2c6f40 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=948a30b5a6e158df server_cookie=27cfffd17208d332 in_seq=0 out_seq=0 2024-03-21T11:42:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:22.765+0000 7fe1fea00640 10 client.31685 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:22.955 INFO:tasks.workunit.client.0.smithi047.stdout: 10 139233 23.43 MB/sec execute 186 sec latency 166.380 ms 2024-03-21T11:42:23.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:23 smithi102 ceph-mon[36748]: pgmap v971: 129 pgs: 129 active+clean; 514 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.6 MiB/s wr, 650 op/s 2024-03-21T11:42:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.541+0000 7fe1fea00640 1 -- 172.21.15.47:0/3284123583 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c4cb1b70c0 con 0x55c4cb2c4b20 2024-03-21T11:42:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.541+0000 7fe1fea00640 10 client.31685 handle_command_reply: tid=0 2024-03-21T11:42:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.542+0000 7fe21ec19740 1 client.31685 shutdown 2024-03-21T11:42:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.543+0000 7fe21ec19740 1 -- 172.21.15.47:0/3284123583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2529f0 msgr2=0x55c4cb261230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.543+0000 7fe21ec19740 1 --2- 172.21.15.47:0/3284123583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2529f0 0x55c4cb261230 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe204009370 tx=0x7fe204009500 comp rx=0 tx=0).stop 2024-03-21T11:42:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.543+0000 7fe21ec19740 1 -- 172.21.15.47:0/3284123583 shutdown_connections 2024-03-21T11:42:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.543+0000 7fe21ec19740 1 --2- 172.21.15.47:0/3284123583 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c4cb2c4b20 0x55c4cb2c6f40 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.543+0000 7fe21ec19740 1 --2- 172.21.15.47:0/3284123583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4cb258550 0x55c4cb262780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.543+0000 7fe21ec19740 1 --2- 172.21.15.47:0/3284123583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4cb2533e0 0x55c4cb261770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.543+0000 7fe21ec19740 1 --2- 172.21.15.47:0/3284123583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c4cb2529f0 0x55c4cb261230 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.543+0000 7fe21ec19740 1 -- 172.21.15.47:0/3284123583 >> 172.21.15.47:0/3284123583 conn(0x55c4cb245750 msgr2=0x55c4cb2440c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.543+0000 7fe1fea00640 0 client.31685 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.543+0000 7fe21ec19740 1 -- 172.21.15.47:0/3284123583 shutdown_connections 2024-03-21T11:42:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.544+0000 7fe21ec19740 1 -- 172.21.15.47:0/3284123583 wait complete. 2024-03-21T11:42:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.547+0000 7fe21ec19740 20 client.31685 trim_cache size 0 max 16384 2024-03-21T11:42:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:42:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:42:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:42:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:42:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.548+0000 7fe21c000640 1 -- 172.21.15.47:0/65769344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe1f80a3040 msgr2=0x7fe1f80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.548+0000 7fe21c000640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe1f80a3040 0x7fe1f80a5500 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe200005ab0 tx=0x7fe20002e040 comp rx=0 tx=0).stop 2024-03-21T11:42:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.548+0000 7fe21c000640 1 -- 172.21.15.47:0/65769344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2141054a0 msgr2=0x7fe2141a10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.548+0000 7fe21c000640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2141054a0 0x7fe2141a10e0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00e970 tx=0x7fe20c00ed70 comp rx=0 tx=0).stop 2024-03-21T11:42:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.548+0000 7fe21c000640 1 -- 172.21.15.47:0/65769344 shutdown_connections 2024-03-21T11:42:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.548+0000 7fe21c000640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe1f80a3040 0x7fe1f80a5500 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.548+0000 7fe21c000640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2141054a0 0x7fe2141a10e0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.548+0000 7fe21c000640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe214104ae0 0x7fe21419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.548+0000 7fe21c000640 1 --2- 172.21.15.47:0/65769344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2141038f0 0x7fe214199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.548+0000 7fe21c000640 1 -- 172.21.15.47:0/65769344 >> 172.21.15.47:0/65769344 conn(0x7fe2140fee60 msgr2=0x7fe214107d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.548+0000 7fe21c000640 1 -- 172.21.15.47:0/65769344 shutdown_connections 2024-03-21T11:42:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:23.548+0000 7fe21c000640 1 -- 172.21.15.47:0/65769344 wait complete. 2024-03-21T11:42:23.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:23 smithi047 ceph-mon[31351]: pgmap v971: 129 pgs: 129 active+clean; 514 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.6 MiB/s wr, 650 op/s 2024-03-21T11:42:23.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:23 smithi100 ceph-mon[36245]: pgmap v971: 129 pgs: 129 active+clean; 514 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.6 MiB/s wr, 650 op/s 2024-03-21T11:42:23.955 INFO:tasks.workunit.client.0.smithi047.stdout: 10 139445 23.38 MB/sec execute 187 sec latency 618.718 ms 2024-03-21T11:42:24.955 INFO:tasks.workunit.client.0.smithi047.stdout: 10 140103 23.40 MB/sec execute 188 sec latency 106.364 ms 2024-03-21T11:42:25.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:25 smithi102 ceph-mon[36748]: pgmap v972: 129 pgs: 129 active+clean; 515 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.2 MiB/s wr, 483 op/s 2024-03-21T11:42:25.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:25 smithi047 ceph-mon[31351]: pgmap v972: 129 pgs: 129 active+clean; 515 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.2 MiB/s wr, 483 op/s 2024-03-21T11:42:25.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:25 smithi100 ceph-mon[36245]: pgmap v972: 129 pgs: 129 active+clean; 515 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.2 MiB/s wr, 483 op/s 2024-03-21T11:42:25.956 INFO:tasks.workunit.client.0.smithi047.stdout: 10 140415 23.38 MB/sec execute 189 sec latency 391.908 ms 2024-03-21T11:42:26.955 INFO:tasks.workunit.client.0.smithi047.stdout: 10 140715 23.34 MB/sec execute 190 sec latency 116.857 ms 2024-03-21T11:42:27.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:27 smithi102 ceph-mon[36748]: pgmap v973: 129 pgs: 129 active+clean; 515 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.1 MiB/s wr, 540 op/s 2024-03-21T11:42:27.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:27 smithi047 ceph-mon[31351]: pgmap v973: 129 pgs: 129 active+clean; 515 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.1 MiB/s wr, 540 op/s 2024-03-21T11:42:27.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:27 smithi100 ceph-mon[36245]: pgmap v973: 129 pgs: 129 active+clean; 515 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.1 MiB/s wr, 540 op/s 2024-03-21T11:42:27.956 INFO:tasks.workunit.client.0.smithi047.stdout: 10 141368 23.42 MB/sec execute 191 sec latency 85.856 ms 2024-03-21T11:42:28.956 INFO:tasks.workunit.client.0.smithi047.stdout: 10 141850 23.46 MB/sec execute 192 sec latency 192.643 ms 2024-03-21T11:42:29.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:29 smithi102 ceph-mon[36748]: pgmap v974: 129 pgs: 129 active+clean; 519 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.6 MiB/s wr, 562 op/s 2024-03-21T11:42:29.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:29 smithi047 ceph-mon[31351]: pgmap v974: 129 pgs: 129 active+clean; 519 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.6 MiB/s wr, 562 op/s 2024-03-21T11:42:29.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:29 smithi100 ceph-mon[36245]: pgmap v974: 129 pgs: 129 active+clean; 519 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.6 MiB/s wr, 562 op/s 2024-03-21T11:42:29.956 INFO:tasks.workunit.client.0.smithi047.stdout: 10 142288 23.46 MB/sec execute 193 sec latency 210.833 ms 2024-03-21T11:42:30.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:30 smithi047 ceph-mon[31351]: pgmap v975: 129 pgs: 129 active+clean; 516 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 580 op/s 2024-03-21T11:42:30.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:30 smithi100 ceph-mon[36245]: pgmap v975: 129 pgs: 129 active+clean; 516 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 580 op/s 2024-03-21T11:42:30.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:30 smithi102 ceph-mon[36748]: pgmap v975: 129 pgs: 129 active+clean; 516 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 580 op/s 2024-03-21T11:42:30.956 INFO:tasks.workunit.client.0.smithi047.stdout: 10 142562 23.41 MB/sec execute 194 sec latency 268.294 ms 2024-03-21T11:42:31.956 INFO:tasks.workunit.client.0.smithi047.stdout: 10 142791 23.32 MB/sec execute 195 sec latency 294.213 ms 2024-03-21T11:42:32.956 INFO:tasks.workunit.client.0.smithi047.stdout: 10 143304 23.34 MB/sec execute 196 sec latency 83.005 ms 2024-03-21T11:42:32.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:32 smithi102 ceph-mon[36748]: pgmap v976: 129 pgs: 129 active+clean; 517 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.8 MiB/s wr, 632 op/s 2024-03-21T11:42:33.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:32 smithi047 ceph-mon[31351]: pgmap v976: 129 pgs: 129 active+clean; 517 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.8 MiB/s wr, 632 op/s 2024-03-21T11:42:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:32 smithi100 ceph-mon[36245]: pgmap v976: 129 pgs: 129 active+clean; 517 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.8 MiB/s wr, 632 op/s 2024-03-21T11:42:33.956 INFO:tasks.workunit.client.0.smithi047.stdout: 10 143904 23.40 MB/sec execute 197 sec latency 77.261 ms 2024-03-21T11:42:34.957 INFO:tasks.workunit.client.0.smithi047.stdout: 10 144572 23.41 MB/sec execute 198 sec latency 73.689 ms 2024-03-21T11:42:34.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:34 smithi102 ceph-mon[36748]: pgmap v977: 129 pgs: 129 active+clean; 520 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.7 MiB/s wr, 549 op/s 2024-03-21T11:42:35.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:34 smithi047 ceph-mon[31351]: pgmap v977: 129 pgs: 129 active+clean; 520 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.7 MiB/s wr, 549 op/s 2024-03-21T11:42:35.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:34 smithi100 ceph-mon[36245]: pgmap v977: 129 pgs: 129 active+clean; 520 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.7 MiB/s wr, 549 op/s 2024-03-21T11:42:35.956 INFO:tasks.workunit.client.0.smithi047.stdout: 10 144945 23.43 MB/sec execute 199 sec latency 322.187 ms 2024-03-21T11:42:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.715+0000 7f9703e00640 1 -- 172.21.15.47:0/3213850207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc107440 msgr2=0x7f96fc109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.715+0000 7f9703e00640 1 --2- 172.21.15.47:0/3213850207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc107440 0x7f96fc109830 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009be0 tx=0x7f96f402bef0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.715+0000 7f9703e00640 1 -- 172.21.15.47:0/3213850207 shutdown_connections 2024-03-21T11:42:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.715+0000 7f9703e00640 1 --2- 172.21.15.47:0/3213850207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc107440 0x7f96fc109830 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.715+0000 7f9703e00640 1 --2- 172.21.15.47:0/3213850207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96fc104b10 0x7f96fc106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.715+0000 7f9703e00640 1 --2- 172.21.15.47:0/3213850207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96fc1021e0 0x7f96fc1045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.715+0000 7f9703e00640 1 -- 172.21.15.47:0/3213850207 >> 172.21.15.47:0/3213850207 conn(0x7f96fc0fbb30 msgr2=0x7f96fc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.715+0000 7f9703e00640 1 -- 172.21.15.47:0/3213850207 shutdown_connections 2024-03-21T11:42:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.715+0000 7f9703e00640 1 -- 172.21.15.47:0/3213850207 wait complete. 2024-03-21T11:42:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.716+0000 7f9703e00640 1 Processor -- start 2024-03-21T11:42:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.716+0000 7f9703e00640 1 -- start start 2024-03-21T11:42:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.717+0000 7f9703e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc1021e0 0x7f96fc1954a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.717+0000 7f9703e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96fc104b10 0x7f96fc1959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.717+0000 7f9703e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96fc107440 0x7f96fc1a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.717+0000 7f9703e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc10d390 con 0x7f96fc104b10 2024-03-21T11:42:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.717+0000 7f9703e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc10d210 con 0x7f96fc1021e0 2024-03-21T11:42:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.717+0000 7f9703e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc10d510 con 0x7f96fc107440 2024-03-21T11:42:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.717+0000 7f9701400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc1021e0 0x7f96fc1954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.717+0000 7f9700a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96fc104b10 0x7f96fc1959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.717+0000 7f9701400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc1021e0 0x7f96fc1954a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41160/0 (socket says 172.21.15.47:41160) 2024-03-21T11:42:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.717+0000 7f9701400640 1 -- 172.21.15.47:0/2680939285 learned_addr learned my addr 172.21.15.47:0/2680939285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.717+0000 7f9701e00640 1 --2- 172.21.15.47:0/2680939285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96fc107440 0x7f96fc1a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.718+0000 7f9701400640 1 -- 172.21.15.47:0/2680939285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96fc107440 msgr2=0x7f96fc1a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.718+0000 7f9701400640 1 --2- 172.21.15.47:0/2680939285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96fc107440 0x7f96fc1a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.718+0000 7f9701400640 1 -- 172.21.15.47:0/2680939285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96fc104b10 msgr2=0x7f96fc1959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.718+0000 7f9701400640 1 --2- 172.21.15.47:0/2680939285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96fc104b10 0x7f96fc1959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.718+0000 7f9701400640 1 -- 172.21.15.47:0/2680939285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc1a5c00 con 0x7f96fc1021e0 2024-03-21T11:42:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.718+0000 7f9700a00640 1 --2- 172.21.15.47:0/2680939285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96fc104b10 0x7f96fc1959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.719+0000 7f9701400640 1 --2- 172.21.15.47:0/2680939285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc1021e0 0x7f96fc1954a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f96f000e9f0 tx=0x7f96f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.719+0000 7f96fa000640 1 -- 172.21.15.47:0/2680939285 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f000cd80 con 0x7f96fc1021e0 2024-03-21T11:42:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.719+0000 7f96fa000640 1 -- 172.21.15.47:0/2680939285 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96f0010910 con 0x7f96fc1021e0 2024-03-21T11:42:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.719+0000 7f96fa000640 1 -- 172.21.15.47:0/2680939285 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0018990 con 0x7f96fc1021e0 2024-03-21T11:42:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.719+0000 7f9703e00640 1 -- 172.21.15.47:0/2680939285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96fc1a5e30 con 0x7f96fc1021e0 2024-03-21T11:42:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.719+0000 7f9703e00640 1 -- 172.21.15.47:0/2680939285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96fc1a62a0 con 0x7f96fc1021e0 2024-03-21T11:42:36.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.723+0000 7f96fa000640 1 -- 172.21.15.47:0/2680939285 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f96f0010ac0 con 0x7f96fc1021e0 2024-03-21T11:42:36.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.724+0000 7f9706bdb740 1 Processor -- start 2024-03-21T11:42:36.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.724+0000 7f9706bdb740 1 -- start start 2024-03-21T11:42:36.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.724+0000 7f9706bdb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0bb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.724+0000 7f9706bdb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0c06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.724+0000 7f9706bdb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.724+0000 7f9706bdb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a07e3b0 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.724+0000 7f9706bdb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a020830 con 0x55fc1a0c0d40 2024-03-21T11:42:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.724+0000 7f9706bdb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc19fc95d0 con 0x55fc1a0bb1e0 2024-03-21T11:42:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.725+0000 7f9701e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0bb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.725+0000 7f9701e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0bb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53634/0 (socket says 172.21.15.47:53634) 2024-03-21T11:42:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.725+0000 7f9701e00640 1 -- 172.21.15.47:0/3174844081 learned_addr learned my addr 172.21.15.47:0/3174844081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.725+0000 7f9701400640 1 --2- 172.21.15.47:0/3174844081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0c06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.725+0000 7f9700a00640 1 --2- 172.21.15.47:0/3174844081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.725+0000 7f9701e00640 1 -- 172.21.15.47:0/3174844081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0c0d40 msgr2=0x55fc1a0c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.725+0000 7f9701e00640 1 --2- 172.21.15.47:0/3174844081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.725+0000 7f9701e00640 1 -- 172.21.15.47:0/3174844081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0bbbd0 msgr2=0x55fc1a0c06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.725+0000 7f9701e00640 1 --2- 172.21.15.47:0/3174844081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0c06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.725+0000 7f9701e00640 1 -- 172.21.15.47:0/3174844081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc19fe12c0 con 0x55fc1a0bb1e0 2024-03-21T11:42:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.725+0000 7f9701400640 1 --2- 172.21.15.47:0/3174844081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0c06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.725+0000 7f9701e00640 1 --2- 172.21.15.47:0/3174844081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0bb600 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f96f402c500 tx=0x7f96f4009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0bc7af8e96b6765 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.725+0000 7f96fa000640 1 --2- 172.21.15.47:0/2680939285 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f96e40a3130 0x7f96e40a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.726+0000 7f96fa000640 1 -- 172.21.15.47:0/2680939285 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f96f00cfa90 con 0x7f96fc1021e0 2024-03-21T11:42:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.726+0000 7f9701400640 1 --2- 172.21.15.47:0/2680939285 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f96e40a3130 0x7f96e40a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.727+0000 7f9701400640 1 --2- 172.21.15.47:0/2680939285 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f96e40a3130 0x7f96e40a55f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f96f009bd60 tx=0x7f96f0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.729+0000 7f96ee600640 1 -- 172.21.15.47:0/3174844081 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4043030 con 0x55fc1a0bb1e0 2024-03-21T11:42:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.729+0000 7f96ee600640 1 -- 172.21.15.47:0/3174844081 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96f4033580 con 0x55fc1a0bb1e0 2024-03-21T11:42:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.729+0000 7f96ee600640 1 -- 172.21.15.47:0/3174844081 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f402c890 con 0x55fc1a0bb1e0 2024-03-21T11:42:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.730+0000 7f9706bdb740 1 -- 172.21.15.47:0/3174844081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bb1e0 msgr2=0x55fc1a0bb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.730+0000 7f9706bdb740 1 --2- 172.21.15.47:0/3174844081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0bb600 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f96f402c500 tx=0x7f96f4009330 comp rx=0 tx=0).stop 2024-03-21T11:42:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.730+0000 7f9706bdb740 1 -- 172.21.15.47:0/3174844081 shutdown_connections 2024-03-21T11:42:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.730+0000 7f9706bdb740 1 --2- 172.21.15.47:0/3174844081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.730+0000 7f9706bdb740 1 --2- 172.21.15.47:0/3174844081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0c06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.730+0000 7f9706bdb740 1 --2- 172.21.15.47:0/3174844081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0bb600 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.730+0000 7f9706bdb740 1 -- 172.21.15.47:0/3174844081 >> 172.21.15.47:0/3174844081 conn(0x55fc1a0b6750 msgr2=0x55fc1a0b6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.730+0000 7f9706bdb740 1 -- 172.21.15.47:0/3174844081 shutdown_connections 2024-03-21T11:42:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.731+0000 7f9706bdb740 1 -- 172.21.15.47:0/3174844081 wait complete. 2024-03-21T11:42:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.731+0000 7f9706bdb740 1 -- start start 2024-03-21T11:42:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.731+0000 7f9706bdb740 1 Processor -- start 2024-03-21T11:42:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.732+0000 7f9706bdb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.732+0000 7f9706bdb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.732+0000 7f9706bdb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.732+0000 7f9706bdb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc19fdca20 con 0x55fc1a0c0d40 2024-03-21T11:42:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.732+0000 7f9706bdb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a01c8d0 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.732+0000 7f9706bdb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a0c3c90 con 0x55fc1a0bb1e0 2024-03-21T11:42:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.732+0000 7f9706bdb740 10 client.0 _ll_register_callbacks cb 0x55fc1a0191c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.732+0000 7f9701e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.732+0000 7f9701e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53646/0 (socket says 172.21.15.47:53646) 2024-03-21T11:42:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.732+0000 7f9700a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.732+0000 7f9701e00640 1 -- 172.21.15.47:0/3646763525 learned_addr learned my addr 172.21.15.47:0/3646763525 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.732+0000 7f9701e00640 1 --2- 172.21.15.47:0/3646763525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.733+0000 7f9700a00640 1 -- 172.21.15.47:0/3646763525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bb1e0 msgr2=0x55fc1a0c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.733+0000 7f9700a00640 1 --2- 172.21.15.47:0/3646763525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.733+0000 7f9700a00640 1 -- 172.21.15.47:0/3646763525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0c0d40 msgr2=0x55fc1a0cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.733+0000 7f9700a00640 1 --2- 172.21.15.47:0/3646763525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.733+0000 7f9700a00640 1 -- 172.21.15.47:0/3646763525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc19d35210 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.733+0000 7f9701e00640 1 --2- 172.21.15.47:0/3646763525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.733+0000 7f9700a00640 1 --2- 172.21.15.47:0/3646763525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0c58d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f96fc196bd0 tx=0x7f96e8013bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4236e5a33b8fba1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.733+0000 7f96e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.733+0000 7f96e2a00640 1 -- 172.21.15.47:0/3646763525 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8020060 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.733+0000 7f9706bdb740 1 -- 172.21.15.47:0/3646763525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96f4033580 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.734+0000 7f96e2a00640 1 -- 172.21.15.47:0/3646763525 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f96e8011da0 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.734+0000 7f96e2a00640 1 -- 172.21.15.47:0/3646763525 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96e800f450 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.734+0000 7f9706bdb740 10 client.31697 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.734+0000 7f9706bdb740 1 -- 172.21.15.47:0/3646763525 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55fc19e01620 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.736+0000 7f96e2a00640 1 -- 172.21.15.47:0/3646763525 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96f4033580 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.736+0000 7f96e2a00640 1 -- 172.21.15.47:0/3646763525 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f96e801a030 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.737+0000 7f9706bdb740 10 client.31697 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.737+0000 7f9706bdb740 10 client.31697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.737+0000 7f9706bdb740 20 client.31697 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.737+0000 7f9706bdb740 1 --2- 172.21.15.47:0/3646763525 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc1a0d11f0 0x55fc1a0d35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.737+0000 7f9706bdb740 4 client.31697 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.737+0000 7f9706bdb740 1 -- 172.21.15.47:0/3646763525 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc1a02dd70 con 0x55fc1a0d11f0 2024-03-21T11:42:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.737+0000 7f9701e00640 1 --2- 172.21.15.47:0/3646763525 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc1a0d11f0 0x55fc1a0d35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.737+0000 7f9701e00640 1 --2- 172.21.15.47:0/3646763525 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc1a0d11f0 0x55fc1a0d35e0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7717f0b9ad1c862c server_cookie=acbfe86a236a2b07 in_seq=0 out_seq=0 2024-03-21T11:42:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.737+0000 7f96e2a00640 10 client.31697 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.738+0000 7f96e2a00640 1 -- 172.21.15.47:0/3646763525 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fc1a02dd70 con 0x55fc1a0d11f0 2024-03-21T11:42:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.738+0000 7f96e2a00640 10 client.31697 handle_command_reply: tid=0 2024-03-21T11:42:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.739+0000 7f9706bdb740 1 client.31697 shutdown 2024-03-21T11:42:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.739+0000 7f9706bdb740 1 -- 172.21.15.47:0/3646763525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0bbbd0 msgr2=0x55fc1a0c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.739+0000 7f9706bdb740 1 --2- 172.21.15.47:0/3646763525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0c58d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f96fc196bd0 tx=0x7f96e8013bf0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.739+0000 7f9706bdb740 1 -- 172.21.15.47:0/3646763525 shutdown_connections 2024-03-21T11:42:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.739+0000 7f9706bdb740 1 --2- 172.21.15.47:0/3646763525 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc1a0d11f0 0x55fc1a0d35e0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.739+0000 7f9706bdb740 1 --2- 172.21.15.47:0/3646763525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.739+0000 7f9706bdb740 1 --2- 172.21.15.47:0/3646763525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0c58d0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.739+0000 7f96e2a00640 0 client.31697 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.739+0000 7f9706bdb740 1 --2- 172.21.15.47:0/3646763525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.739+0000 7f9706bdb740 1 -- 172.21.15.47:0/3646763525 >> 172.21.15.47:0/3646763525 conn(0x55fc1a0b6750 msgr2=0x55fc1a0b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.739+0000 7f9706bdb740 1 -- 172.21.15.47:0/3646763525 shutdown_connections 2024-03-21T11:42:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.739+0000 7f9706bdb740 1 -- 172.21.15.47:0/3646763525 wait complete. 2024-03-21T11:42:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.739+0000 7f9706bdb740 20 client.31697 trim_cache size 0 max 16384 2024-03-21T11:42:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.757+0000 7f9706bdb740 1 Processor -- start 2024-03-21T11:42:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.757+0000 7f9706bdb740 1 -- start start 2024-03-21T11:42:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9706bdb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0bcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9706bdb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9706bdb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0bd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9706bdb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc1b0560 con 0x55fc1a0c0d40 2024-03-21T11:42:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9706bdb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc1b1ee0 con 0x55fc1a0bb1e0 2024-03-21T11:42:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9706bdb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc1b0bc0 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9700a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9700a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53650/0 (socket says 172.21.15.47:53650) 2024-03-21T11:42:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9700a00640 1 -- 172.21.15.47:0/4171788237 learned_addr learned my addr 172.21.15.47:0/4171788237 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9701e00640 1 --2- 172.21.15.47:0/4171788237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0bcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9701e00640 1 --2- 172.21.15.47:0/4171788237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0bd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9700a00640 1 -- 172.21.15.47:0/4171788237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0bb1e0 msgr2=0x55fc1a0bcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9700a00640 1 --2- 172.21.15.47:0/4171788237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0bcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9700a00640 1 -- 172.21.15.47:0/4171788237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0c0d40 msgr2=0x55fc1a0bd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9700a00640 1 --2- 172.21.15.47:0/4171788237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0bd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.758+0000 7f9700a00640 1 -- 172.21.15.47:0/4171788237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc1a0bc9e0 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.759+0000 7f9700a00640 1 --2- 172.21.15.47:0/4171788237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0cc8d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009ee0 tx=0x7f96e8009d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb5318802b92aa94 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.759+0000 7f96edc00640 1 -- 172.21.15.47:0/4171788237 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e802a080 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.759+0000 7f96edc00640 1 -- 172.21.15.47:0/4171788237 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96e801bca0 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.759+0000 7f96edc00640 1 -- 172.21.15.47:0/4171788237 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e800f5f0 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.760+0000 7f9706bdb740 1 -- 172.21.15.47:0/4171788237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bbbd0 msgr2=0x55fc1a0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.760+0000 7f9706bdb740 1 --2- 172.21.15.47:0/4171788237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0cc8d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009ee0 tx=0x7f96e8009d70 comp rx=0 tx=0).stop 2024-03-21T11:42:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.760+0000 7f9706bdb740 1 -- 172.21.15.47:0/4171788237 shutdown_connections 2024-03-21T11:42:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.760+0000 7f9706bdb740 1 --2- 172.21.15.47:0/4171788237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0bd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.760+0000 7f9706bdb740 1 --2- 172.21.15.47:0/4171788237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0cc8d0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.760+0000 7f9706bdb740 1 --2- 172.21.15.47:0/4171788237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0bcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.760+0000 7f9706bdb740 1 -- 172.21.15.47:0/4171788237 >> 172.21.15.47:0/4171788237 conn(0x55fc1a0b6750 msgr2=0x55fc1a0c5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.760+0000 7f9706bdb740 1 -- 172.21.15.47:0/4171788237 shutdown_connections 2024-03-21T11:42:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.761+0000 7f9706bdb740 1 -- 172.21.15.47:0/4171788237 wait complete. 2024-03-21T11:42:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.761+0000 7f9706bdb740 1 -- start start 2024-03-21T11:42:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.761+0000 7f9706bdb740 1 Processor -- start 2024-03-21T11:42:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.762+0000 7f9706bdb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.762+0000 7f9706bdb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.762+0000 7f9706bdb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.762+0000 7f9706bdb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc1b1880 con 0x55fc1a0bb1e0 2024-03-21T11:42:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.762+0000 7f9706bdb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc1b1220 con 0x55fc1a0c0d40 2024-03-21T11:42:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.762+0000 7f9706bdb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc1ae720 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.762+0000 7f9706bdb740 10 client.0 _ll_register_callbacks cb 0x7f96e80096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.762+0000 7f9701e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.762+0000 7f9701e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54082/0 (socket says 172.21.15.47:54082) 2024-03-21T11:42:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.762+0000 7f9700a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.762+0000 7f9701e00640 1 -- 172.21.15.47:0/4038494984 learned_addr learned my addr 172.21.15.47:0/4038494984 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.762+0000 7f9701e00640 1 --2- 172.21.15.47:0/4038494984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.763+0000 7f9700a00640 1 -- 172.21.15.47:0/4038494984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0c0d40 msgr2=0x55fc1a0cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.763+0000 7f9700a00640 1 --2- 172.21.15.47:0/4038494984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.763+0000 7f9700a00640 1 -- 172.21.15.47:0/4038494984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0bb1e0 msgr2=0x55fc1a0ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.763+0000 7f9700a00640 1 --2- 172.21.15.47:0/4038494984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.763+0000 7f9700a00640 1 -- 172.21.15.47:0/4038494984 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc1a0cc030 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.763+0000 7f9700a00640 1 --2- 172.21.15.47:0/4038494984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0caa30 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f96e801b930 tx=0x7f96e8005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4df8e782686c09cd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.763+0000 7f96e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:42:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.763+0000 7f96e2a00640 1 -- 172.21.15.47:0/4038494984 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e803c060 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.763+0000 7f96e2a00640 1 -- 172.21.15.47:0/4038494984 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f96e801b3f0 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.763+0000 7f9706bdb740 1 -- 172.21.15.47:0/4038494984 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96e801bca0 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.764+0000 7f96e2a00640 1 -- 172.21.15.47:0/4038494984 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96e8043050 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.764+0000 7f9706bdb740 10 client.31681 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.764+0000 7f9706bdb740 1 -- 172.21.15.47:0/4038494984 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55fc1a12cfd0 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.764+0000 7f96e2a00640 1 -- 172.21.15.47:0/4038494984 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96e801bca0 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.764+0000 7f96e2a00640 1 -- 172.21.15.47:0/4038494984 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f96e801a050 con 0x55fc1a0bbbd0 2024-03-21T11:42:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.764+0000 7f9706bdb740 10 client.31681 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.764+0000 7f9706bdb740 10 client.31681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.764+0000 7f9706bdb740 20 client.31681 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.764+0000 7f9706bdb740 1 --2- 172.21.15.47:0/4038494984 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc1a12d7a0 0x55fc1a12fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.764+0000 7f9706bdb740 4 client.31681 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:42:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.764+0000 7f9706bdb740 1 -- 172.21.15.47:0/4038494984 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fc1a0280c0 con 0x55fc1a12d7a0 2024-03-21T11:42:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.765+0000 7f9701e00640 1 --2- 172.21.15.47:0/4038494984 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc1a12d7a0 0x55fc1a12fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.765+0000 7f9701e00640 1 --2- 172.21.15.47:0/4038494984 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc1a12d7a0 0x55fc1a12fc00 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=772f8cacc8bcf382 server_cookie=87ffbf322ec6317f in_seq=0 out_seq=0 2024-03-21T11:42:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:36.765+0000 7f96e2a00640 10 client.31681 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:36.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:42:36.876 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:42:36.956 INFO:tasks.workunit.client.0.smithi047.stdout: 10 145314 23.44 MB/sec execute 200 sec latency 315.452 ms 2024-03-21T11:42:36.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:36 smithi102 ceph-mon[36748]: pgmap v978: 129 pgs: 129 active+clean; 523 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 563 op/s 2024-03-21T11:42:36.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:36 smithi047 ceph-mon[31351]: pgmap v978: 129 pgs: 129 active+clean; 523 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 563 op/s 2024-03-21T11:42:37.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:36 smithi100 ceph-mon[36245]: pgmap v978: 129 pgs: 129 active+clean; 523 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.1 MiB/s wr, 563 op/s 2024-03-21T11:42:37.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.219+0000 7f2a01800640 1 -- 172.21.15.47:0/849531857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29fc105490 msgr2=0x7f29fc10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:37.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.219+0000 7f2a01800640 1 --2- 172.21.15.47:0/849531857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29fc105490 0x7f29fc10bd20 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009ca0 tx=0x7f29f402c100 comp rx=0 tx=0).stop 2024-03-21T11:42:37.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.222+0000 7f2a01800640 1 -- 172.21.15.47:0/849531857 shutdown_connections 2024-03-21T11:42:37.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.222+0000 7f2a01800640 1 --2- 172.21.15.47:0/849531857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29fc105490 0x7f29fc10bd20 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.222+0000 7f2a01800640 1 --2- 172.21.15.47:0/849531857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29fc104ad0 0x7f29fc104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.222+0000 7f2a01800640 1 --2- 172.21.15.47:0/849531857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29fc1038e0 0x7f29fc103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.222+0000 7f2a01800640 1 -- 172.21.15.47:0/849531857 >> 172.21.15.47:0/849531857 conn(0x7f29fc0fee50 msgr2=0x7f29fc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:37.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.222+0000 7f2a01800640 1 -- 172.21.15.47:0/849531857 shutdown_connections 2024-03-21T11:42:37.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.222+0000 7f2a01800640 1 -- 172.21.15.47:0/849531857 wait complete. 2024-03-21T11:42:37.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.224+0000 7f2a01800640 1 Processor -- start 2024-03-21T11:42:37.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.224+0000 7f2a01800640 1 -- start start 2024-03-21T11:42:37.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.225+0000 7f2a01800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29fc1038e0 0x7f29fc070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.225+0000 7f2a01800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29fc104ad0 0x7f29fc070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.225+0000 7f2a01800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29fc105490 0x7f29fc1a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.225+0000 7f2a01800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc10f540 con 0x7f29fc104ad0 2024-03-21T11:42:37.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.225+0000 7f2a01800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc10f3c0 con 0x7f29fc1038e0 2024-03-21T11:42:37.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.225+0000 7f2a01800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc10f6c0 con 0x7f29fc105490 2024-03-21T11:42:37.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.226+0000 7f29faa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29fc1038e0 0x7f29fc070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.226+0000 7f29faa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29fc1038e0 0x7f29fc070030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41208/0 (socket says 172.21.15.47:41208) 2024-03-21T11:42:37.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.226+0000 7f29faa00640 1 -- 172.21.15.47:0/26240647 learned_addr learned my addr 172.21.15.47:0/26240647 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:37.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.226+0000 7f29fb400640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29fc105490 0x7f29fc1a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.226+0000 7f29f3e00640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29fc104ad0 0x7f29fc070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.227+0000 7f29fb400640 1 -- 172.21.15.47:0/26240647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29fc1038e0 msgr2=0x7f29fc070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:37.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.227+0000 7f29fb400640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29fc1038e0 0x7f29fc070030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.227+0000 7f29fb400640 1 -- 172.21.15.47:0/26240647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29fc104ad0 msgr2=0x7f29fc070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:37.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.227+0000 7f29fb400640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29fc104ad0 0x7f29fc070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.227+0000 7f29fb400640 1 -- 172.21.15.47:0/26240647 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc070c10 con 0x7f29fc105490 2024-03-21T11:42:37.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.227+0000 7f29faa00640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29fc1038e0 0x7f29fc070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:37.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.228+0000 7f29fb400640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29fc105490 0x7f29fc1a9c50 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f29f402c640 tx=0x7f29f4035d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:37.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.229+0000 7f29f3400640 1 -- 172.21.15.47:0/26240647 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4039040 con 0x7f29fc105490 2024-03-21T11:42:37.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.229+0000 7f29f3400640 1 -- 172.21.15.47:0/26240647 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29f4044070 con 0x7f29fc105490 2024-03-21T11:42:37.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.229+0000 7f2a01800640 1 -- 172.21.15.47:0/26240647 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29fc1aa250 con 0x7f29fc105490 2024-03-21T11:42:37.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.229+0000 7f29f3400640 1 -- 172.21.15.47:0/26240647 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40043e0 con 0x7f29fc105490 2024-03-21T11:42:37.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.230+0000 7f2a01800640 1 -- 172.21.15.47:0/26240647 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29fc1aa760 con 0x7f29fc105490 2024-03-21T11:42:37.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.238+0000 7f2a04637740 1 Processor -- start 2024-03-21T11:42:37.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.238+0000 7f29f3400640 1 -- 172.21.15.47:0/26240647 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f29f4004580 con 0x7f29fc105490 2024-03-21T11:42:37.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.238+0000 7f2a04637740 1 -- start start 2024-03-21T11:42:37.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.239+0000 7f2a04637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 0x555f892a72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.239+0000 7f2a04637740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892ac370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.240+0000 7f29f3400640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f29e00a2fa0 0x7f29e00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.240+0000 7f2a04637740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892ac9e0 0x555f892aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.240+0000 7f2a04637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8920c250 con 0x555f892a6e80 2024-03-21T11:42:37.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.240+0000 7f2a04637740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f891b4a30 con 0x555f892a7870 2024-03-21T11:42:37.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.240+0000 7f2a04637740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f892082f0 con 0x555f892ac9e0 2024-03-21T11:42:37.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.240+0000 7f29f3400640 1 -- 172.21.15.47:0/26240647 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f29f40f1d40 con 0x7f29fc105490 2024-03-21T11:42:37.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.241+0000 7f29fb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892ac9e0 0x555f892aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.241+0000 7f29faa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.241+0000 7f29faa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892ac370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41218/0 (socket says 172.21.15.47:41218) 2024-03-21T11:42:37.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.241+0000 7f29faa00640 1 -- 172.21.15.47:0/1943944186 learned_addr learned my addr 172.21.15.47:0/1943944186 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:37.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.243+0000 7f29f3e00640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f29e00a2fa0 0x7f29e00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.243+0000 7f29f3e00640 1 --2- 172.21.15.47:0/1943944186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 0x555f892a72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.244+0000 7f29fb400640 1 -- 172.21.15.47:0/1943944186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 msgr2=0x555f892ac370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:37.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.244+0000 7f29fb400640 1 --2- 172.21.15.47:0/1943944186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892ac370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.244+0000 7f29fb400640 1 -- 172.21.15.47:0/1943944186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 msgr2=0x555f892a72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:37.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.244+0000 7f29fb400640 1 --2- 172.21.15.47:0/1943944186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 0x555f892a72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.244+0000 7f29fb400640 1 -- 172.21.15.47:0/1943944186 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f8919f690 con 0x555f892ac9e0 2024-03-21T11:42:37.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.244+0000 7f29faa00640 1 --2- 172.21.15.47:0/1943944186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:37.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.244+0000 7f29fb400640 1 --2- 172.21.15.47:0/1943944186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892ac9e0 0x555f892aee70 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f29f40324c0 tx=0x7f29f40095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db6c3dc0d0e70e5a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:37.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.245+0000 7f29ebe00640 1 -- 172.21.15.47:0/1943944186 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40bdc20 con 0x555f892ac9e0 2024-03-21T11:42:37.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.245+0000 7f29ebe00640 1 -- 172.21.15.47:0/1943944186 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29f40dce50 con 0x555f892ac9e0 2024-03-21T11:42:37.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.245+0000 7f29ebe00640 1 -- 172.21.15.47:0/1943944186 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40bda40 con 0x555f892ac9e0 2024-03-21T11:42:37.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.246+0000 7f2a04637740 1 -- 172.21.15.47:0/1943944186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892ac9e0 msgr2=0x555f892aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:37.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.246+0000 7f2a04637740 1 --2- 172.21.15.47:0/1943944186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892ac9e0 0x555f892aee70 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f29f40324c0 tx=0x7f29f40095c0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.247+0000 7f2a04637740 1 -- 172.21.15.47:0/1943944186 shutdown_connections 2024-03-21T11:42:37.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.247+0000 7f2a04637740 1 --2- 172.21.15.47:0/1943944186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892ac9e0 0x555f892aee70 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.247+0000 7f2a04637740 1 --2- 172.21.15.47:0/1943944186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.247+0000 7f2a04637740 1 --2- 172.21.15.47:0/1943944186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 0x555f892a72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.247+0000 7f2a04637740 1 -- 172.21.15.47:0/1943944186 >> 172.21.15.47:0/1943944186 conn(0x555f8926f200 msgr2=0x555f892a27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:37.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.248+0000 7f29f3e00640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f29e00a2fa0 0x7f29e00a5460 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0714f0 tx=0x7f29e4009b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.248+0000 7f2a04637740 1 -- 172.21.15.47:0/1943944186 shutdown_connections 2024-03-21T11:42:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.248+0000 7f2a04637740 1 -- 172.21.15.47:0/1943944186 wait complete. 2024-03-21T11:42:37.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.248+0000 7f2a04637740 1 -- start start 2024-03-21T11:42:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.249+0000 7f2a04637740 1 Processor -- start 2024-03-21T11:42:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.249+0000 7f2a04637740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892a6e80 0x555f892b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.249+0000 7f2a04637740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.249+0000 7f2a04637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892ac9e0 0x555f892b8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.250+0000 7f29f3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.250+0000 7f29f3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41222/0 (socket says 172.21.15.47:41222) 2024-03-21T11:42:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.250+0000 7f29faa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892a6e80 0x555f892b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.250+0000 7f29faa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892a6e80 0x555f892b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53704/0 (socket says 172.21.15.47:53704) 2024-03-21T11:42:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.250+0000 7f2a04637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f892af750 con 0x555f892ac9e0 2024-03-21T11:42:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.250+0000 7f2a04637740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f891c7e80 con 0x555f892a7870 2024-03-21T11:42:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.250+0000 7f2a04637740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f892af8d0 con 0x555f892a6e80 2024-03-21T11:42:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.250+0000 7f2a04637740 10 client.0 _ll_register_callbacks cb 0x555f891da130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.251+0000 7f29f3e00640 1 -- 172.21.15.47:0/1412210522 learned_addr learned my addr 172.21.15.47:0/1412210522 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.251+0000 7f29fb400640 1 --2- 172.21.15.47:0/1412210522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892ac9e0 0x555f892b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.251+0000 7f29faa00640 1 -- 172.21.15.47:0/1412210522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 msgr2=0x555f892b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.251+0000 7f29faa00640 1 --2- 172.21.15.47:0/1412210522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.252+0000 7f29faa00640 1 -- 172.21.15.47:0/1412210522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892ac9e0 msgr2=0x555f892b8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.252+0000 7f29faa00640 1 --2- 172.21.15.47:0/1412210522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892ac9e0 0x555f892b8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.252+0000 7f29faa00640 1 -- 172.21.15.47:0/1412210522 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f88fea7d0 con 0x555f892a6e80 2024-03-21T11:42:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.252+0000 7f29faa00640 1 --2- 172.21.15.47:0/1412210522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892a6e80 0x555f892b0bb0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x555f892a7c70 tx=0x7f29ec00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c8850390076c795 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.252+0000 7f29e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:42:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.252+0000 7f29e8c00640 1 -- 172.21.15.47:0/1412210522 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec00fe50 con 0x555f892a6e80 2024-03-21T11:42:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.252+0000 7f2a04637740 1 -- 172.21.15.47:0/1412210522 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29f40dce50 con 0x555f892a6e80 2024-03-21T11:42:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.252+0000 7f29e8c00640 1 -- 172.21.15.47:0/1412210522 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f29ec010430 con 0x555f892a6e80 2024-03-21T11:42:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.253+0000 7f29e8c00640 1 -- 172.21.15.47:0/1412210522 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29ec010630 con 0x555f892a6e80 2024-03-21T11:42:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.253+0000 7f2a04637740 10 client.31693 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.253+0000 7f2a04637740 1 -- 172.21.15.47:0/1412210522 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555f88f203f0 con 0x555f892a6e80 2024-03-21T11:42:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.253+0000 7f29e8c00640 1 -- 172.21.15.47:0/1412210522 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29f40dce50 con 0x555f892a6e80 2024-03-21T11:42:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.256+0000 7f29e8c00640 1 -- 172.21.15.47:0/1412210522 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f29ec01b760 con 0x555f892a6e80 2024-03-21T11:42:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.256+0000 7f2a04637740 10 client.31693 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.256+0000 7f2a04637740 10 client.31693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.256+0000 7f2a04637740 20 client.31693 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.256+0000 7f2a04637740 1 --2- 172.21.15.47:0/1412210522 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f892bc980 0x555f892bee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.256+0000 7f2a04637740 4 client.31693 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.256+0000 7f2a04637740 1 -- 172.21.15.47:0/1412210522 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f89219790 con 0x555f892bc980 2024-03-21T11:42:37.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.258+0000 7f29f3e00640 1 --2- 172.21.15.47:0/1412210522 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f892bc980 0x555f892bee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.260+0000 7f29f3e00640 1 --2- 172.21.15.47:0/1412210522 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f892bc980 0x555f892bee10 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a00eb5a7cb4ba94 server_cookie=ae8ea446f34e91c4 in_seq=0 out_seq=0 2024-03-21T11:42:37.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.260+0000 7f29e8c00640 10 client.31693 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:37.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.788+0000 7f96e2a00640 1 -- 172.21.15.47:0/4038494984 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fc1a0280c0 con 0x55fc1a12d7a0 2024-03-21T11:42:37.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.788+0000 7f96e2a00640 10 client.31681 handle_command_reply: tid=0 2024-03-21T11:42:37.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.788+0000 7f9706bdb740 1 client.31681 shutdown 2024-03-21T11:42:37.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.788+0000 7f9706bdb740 1 -- 172.21.15.47:0/4038494984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bbbd0 msgr2=0x55fc1a0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:37.789 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.788+0000 7f9706bdb740 1 --2- 172.21.15.47:0/4038494984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0caa30 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f96e801b930 tx=0x7f96e8005700 comp rx=0 tx=0).stop 2024-03-21T11:42:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.788+0000 7f29e8c00640 1 -- 172.21.15.47:0/1412210522 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555f89219790 con 0x555f892bc980 2024-03-21T11:42:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.789+0000 7f29e8c00640 10 client.31693 handle_command_reply: tid=0 2024-03-21T11:42:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.789+0000 7f2a04637740 1 client.31693 shutdown 2024-03-21T11:42:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.790+0000 7f9706bdb740 1 -- 172.21.15.47:0/4038494984 shutdown_connections 2024-03-21T11:42:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.790+0000 7f9706bdb740 1 --2- 172.21.15.47:0/4038494984 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fc1a12d7a0 0x55fc1a12fc00 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.790+0000 7f9706bdb740 1 --2- 172.21.15.47:0/4038494984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc1a0c0d40 0x55fc1a0cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.790+0000 7f9706bdb740 1 --2- 172.21.15.47:0/4038494984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc1a0bbbd0 0x55fc1a0caa30 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.790+0000 7f9706bdb740 1 --2- 172.21.15.47:0/4038494984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc1a0bb1e0 0x55fc1a0ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.790+0000 7f9706bdb740 1 -- 172.21.15.47:0/4038494984 >> 172.21.15.47:0/4038494984 conn(0x55fc1a0b6750 msgr2=0x55fc1a0c5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.790+0000 7f2a04637740 1 -- 172.21.15.47:0/1412210522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892a6e80 msgr2=0x555f892b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.790+0000 7f2a04637740 1 --2- 172.21.15.47:0/1412210522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892a6e80 0x555f892b0bb0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x555f892a7c70 tx=0x7f29ec00f790 comp rx=0 tx=0).stop 2024-03-21T11:42:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.790+0000 7f96e2a00640 0 client.31681 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.790+0000 7f2a04637740 1 -- 172.21.15.47:0/1412210522 shutdown_connections 2024-03-21T11:42:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.791+0000 7f9706bdb740 1 -- 172.21.15.47:0/4038494984 shutdown_connections 2024-03-21T11:42:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.791+0000 7f9706bdb740 1 -- 172.21.15.47:0/4038494984 wait complete. 2024-03-21T11:42:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.791+0000 7f2a04637740 1 --2- 172.21.15.47:0/1412210522 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f892bc980 0x555f892bee10 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.791+0000 7f2a04637740 1 --2- 172.21.15.47:0/1412210522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892ac9e0 0x555f892b8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.791+0000 7f2a04637740 1 --2- 172.21.15.47:0/1412210522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.791+0000 7f2a04637740 1 --2- 172.21.15.47:0/1412210522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892a6e80 0x555f892b0bb0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.791+0000 7f2a04637740 1 -- 172.21.15.47:0/1412210522 >> 172.21.15.47:0/1412210522 conn(0x555f8926f200 msgr2=0x555f892a1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:42:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:42:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:42:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:42:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.791+0000 7f9706bdb740 20 client.31681 trim_cache size 0 max 16384 2024-03-21T11:42:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.791+0000 7f29e8c00640 0 client.31693 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.792+0000 7f2a04637740 1 -- 172.21.15.47:0/1412210522 shutdown_connections 2024-03-21T11:42:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.792+0000 7f9703e00640 1 -- 172.21.15.47:0/2680939285 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f96e40a3130 msgr2=0x7f96e40a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.792+0000 7f9703e00640 1 --2- 172.21.15.47:0/2680939285 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f96e40a3130 0x7f96e40a55f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f96f009bd60 tx=0x7f96f0000bc0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.792+0000 7f2a04637740 1 -- 172.21.15.47:0/1412210522 wait complete. 2024-03-21T11:42:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.793+0000 7f2a04637740 20 client.31693 trim_cache size 0 max 16384 2024-03-21T11:42:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.792+0000 7f9703e00640 1 -- 172.21.15.47:0/2680939285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc1021e0 msgr2=0x7f96fc1954a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.792+0000 7f9703e00640 1 --2- 172.21.15.47:0/2680939285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc1021e0 0x7f96fc1954a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f96f000e9f0 tx=0x7f96f000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.793+0000 7f9703e00640 1 -- 172.21.15.47:0/2680939285 shutdown_connections 2024-03-21T11:42:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.793+0000 7f9703e00640 1 --2- 172.21.15.47:0/2680939285 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f96e40a3130 0x7f96e40a55f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.793+0000 7f9703e00640 1 --2- 172.21.15.47:0/2680939285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96fc107440 0x7f96fc1a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.793+0000 7f9703e00640 1 --2- 172.21.15.47:0/2680939285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96fc104b10 0x7f96fc1959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.793+0000 7f9703e00640 1 --2- 172.21.15.47:0/2680939285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc1021e0 0x7f96fc1954a0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.793+0000 7f9703e00640 1 -- 172.21.15.47:0/2680939285 >> 172.21.15.47:0/2680939285 conn(0x7f96fc0fbb30 msgr2=0x7f96fc101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.794+0000 7f9703e00640 1 -- 172.21.15.47:0/2680939285 shutdown_connections 2024-03-21T11:42:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:37.796+0000 7f9703e00640 1 -- 172.21.15.47:0/2680939285 wait complete. 2024-03-21T11:42:37.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.822+0000 7f2a04637740 1 Processor -- start 2024-03-21T11:42:37.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.822+0000 7f2a04637740 1 -- start start 2024-03-21T11:42:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.822+0000 7f2a04637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 0x555f892a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.822+0000 7f2a04637740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892a7870 0x555f892b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.822+0000 7f2a04637740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892ac9e0 0x555f892a5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.822+0000 7f2a04637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc1b4aa0 con 0x555f892a6e80 2024-03-21T11:42:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.822+0000 7f2a04637740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc1b6420 con 0x555f892ac9e0 2024-03-21T11:42:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.822+0000 7f2a04637740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc1b5100 con 0x555f892a7870 2024-03-21T11:42:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.823+0000 7f29f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892a7870 0x555f892b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.823+0000 7f29fb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892ac9e0 0x555f892a5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.823+0000 7f29faa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 0x555f892a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.823+0000 7f29f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892a7870 0x555f892b80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53720/0 (socket says 172.21.15.47:53720) 2024-03-21T11:42:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.823+0000 7f29f3e00640 1 -- 172.21.15.47:0/4186292024 learned_addr learned my addr 172.21.15.47:0/4186292024 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:37.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.823+0000 7f29fb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892ac9e0 0x555f892a5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41236/0 (socket says 172.21.15.47:41236) 2024-03-21T11:42:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.823+0000 7f29f3e00640 1 -- 172.21.15.47:0/4186292024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892ac9e0 msgr2=0x555f892a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.823+0000 7f29f3e00640 1 --2- 172.21.15.47:0/4186292024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892ac9e0 0x555f892a5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.823+0000 7f29f3e00640 1 -- 172.21.15.47:0/4186292024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 msgr2=0x555f892a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.823+0000 7f29f3e00640 1 --2- 172.21.15.47:0/4186292024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 0x555f892a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.823+0000 7f29f3e00640 1 -- 172.21.15.47:0/4186292024 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f88ee00f0 con 0x555f892a7870 2024-03-21T11:42:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.823+0000 7f29fb400640 1 --2- 172.21.15.47:0/4186292024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892ac9e0 0x555f892a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.823+0000 7f29faa00640 1 --2- 172.21.15.47:0/4186292024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 0x555f892a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.824+0000 7f29f3e00640 1 --2- 172.21.15.47:0/4186292024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892a7870 0x555f892b80f0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f29e4004700 tx=0x7f29e4011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf04aa9b3aaa06a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.824+0000 7f29eb400640 1 -- 172.21.15.47:0/4186292024 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e400f5e0 con 0x555f892a7870 2024-03-21T11:42:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.824+0000 7f29eb400640 1 -- 172.21.15.47:0/4186292024 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29e400fd90 con 0x555f892a7870 2024-03-21T11:42:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.824+0000 7f29eb400640 1 -- 172.21.15.47:0/4186292024 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4005610 con 0x555f892a7870 2024-03-21T11:42:37.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.825+0000 7f2a04637740 1 -- 172.21.15.47:0/4186292024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892a7870 msgr2=0x555f892b80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.825+0000 7f2a04637740 1 --2- 172.21.15.47:0/4186292024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892a7870 0x555f892b80f0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f29e4004700 tx=0x7f29e4011f90 comp rx=0 tx=0).stop 2024-03-21T11:42:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.826+0000 7f2a04637740 1 -- 172.21.15.47:0/4186292024 shutdown_connections 2024-03-21T11:42:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.826+0000 7f2a04637740 1 --2- 172.21.15.47:0/4186292024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892ac9e0 0x555f892a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.826+0000 7f2a04637740 1 --2- 172.21.15.47:0/4186292024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892a7870 0x555f892b80f0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.826+0000 7f2a04637740 1 --2- 172.21.15.47:0/4186292024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 0x555f892a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.826+0000 7f2a04637740 1 -- 172.21.15.47:0/4186292024 >> 172.21.15.47:0/4186292024 conn(0x555f8926f200 msgr2=0x555f892a8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:37.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.827+0000 7f2a04637740 1 -- 172.21.15.47:0/4186292024 shutdown_connections 2024-03-21T11:42:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.827+0000 7f2a04637740 1 -- 172.21.15.47:0/4186292024 wait complete. 2024-03-21T11:42:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.827+0000 7f2a04637740 1 -- start start 2024-03-21T11:42:37.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.828+0000 7f2a04637740 1 Processor -- start 2024-03-21T11:42:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.828+0000 7f2a04637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 0x555f892b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.828+0000 7f2a04637740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.828+0000 7f2a04637740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892ac9e0 0x555f892b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.828+0000 7f2a04637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc1b5dc0 con 0x555f892a6e80 2024-03-21T11:42:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.828+0000 7f2a04637740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc1b5760 con 0x555f892a7870 2024-03-21T11:42:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.828+0000 7f2a04637740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc1b2c60 con 0x555f892ac9e0 2024-03-21T11:42:37.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.828+0000 7f2a04637740 10 client.0 _ll_register_callbacks cb 0x7f29ec012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:37.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.829+0000 7f29f3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.829+0000 7f29fb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892ac9e0 0x555f892b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.829+0000 7f29f3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892b5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41242/0 (socket says 172.21.15.47:41242) 2024-03-21T11:42:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.829+0000 7f29f3e00640 1 -- 172.21.15.47:0/1089488504 learned_addr learned my addr 172.21.15.47:0/1089488504 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.829+0000 7f29faa00640 1 --2- 172.21.15.47:0/1089488504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 0x555f892b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.829+0000 7f29f3e00640 1 -- 172.21.15.47:0/1089488504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892ac9e0 msgr2=0x555f892b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.829+0000 7f29f3e00640 1 --2- 172.21.15.47:0/1089488504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892ac9e0 0x555f892b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.829+0000 7f29f3e00640 1 -- 172.21.15.47:0/1089488504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 msgr2=0x555f892b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.829+0000 7f29f3e00640 1 --2- 172.21.15.47:0/1089488504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 0x555f892b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.829+0000 7f29f3e00640 1 -- 172.21.15.47:0/1089488504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f892b7060 con 0x555f892a7870 2024-03-21T11:42:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.829+0000 7f29fb400640 1 --2- 172.21.15.47:0/1089488504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892ac9e0 0x555f892b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.829+0000 7f29faa00640 1 --2- 172.21.15.47:0/1089488504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 0x555f892b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:42:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.829+0000 7f29f3e00640 1 --2- 172.21.15.47:0/1089488504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892b5a60 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f29e4005190 tx=0x7f29e400f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6205a85fbacd36e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:37.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.829+0000 7f29e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.830+0000 7f29e8c00640 1 -- 172.21.15.47:0/1089488504 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4039060 con 0x555f892a7870 2024-03-21T11:42:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.830+0000 7f29e8c00640 1 -- 172.21.15.47:0/1089488504 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f29e402a240 con 0x555f892a7870 2024-03-21T11:42:37.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.830+0000 7f2a04637740 1 -- 172.21.15.47:0/1089488504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29e400fd90 con 0x555f892a7870 2024-03-21T11:42:37.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.830+0000 7f29e8c00640 1 -- 172.21.15.47:0/1089488504 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29e401a040 con 0x555f892a7870 2024-03-21T11:42:37.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.831+0000 7f2a04637740 10 client.31721 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:37.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.831+0000 7f2a04637740 1 -- 172.21.15.47:0/1089488504 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555f892b83a0 con 0x555f892a7870 2024-03-21T11:42:37.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.831+0000 7f29e8c00640 1 -- 172.21.15.47:0/1089488504 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29e400fd90 con 0x555f892a7870 2024-03-21T11:42:37.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.832+0000 7f29e8c00640 1 -- 172.21.15.47:0/1089488504 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f29e401e070 con 0x555f892a7870 2024-03-21T11:42:37.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.832+0000 7f2a04637740 10 client.31721 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:37.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.832+0000 7f2a04637740 10 client.31721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:37.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.832+0000 7f2a04637740 20 client.31721 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:37.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.832+0000 7f2a04637740 1 --2- 172.21.15.47:0/1089488504 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f89321110 0x555f89323530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:37.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.832+0000 7f2a04637740 4 client.31721 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:42:37.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.832+0000 7f2a04637740 1 -- 172.21.15.47:0/1089488504 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555f89213ae0 con 0x555f89321110 2024-03-21T11:42:37.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.832+0000 7f29faa00640 1 --2- 172.21.15.47:0/1089488504 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f89321110 0x555f89323530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:37.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.833+0000 7f29faa00640 1 --2- 172.21.15.47:0/1089488504 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f89321110 0x555f89323530 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c08367bbe41444a8 server_cookie=b940881b925035a9 in_seq=0 out_seq=0 2024-03-21T11:42:37.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.833+0000 7f29e8c00640 10 client.31721 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:37.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.833+0000 7f29e8c00640 1 -- 172.21.15.47:0/1089488504 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555f89213ae0 con 0x555f89321110 2024-03-21T11:42:37.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.833+0000 7f29e8c00640 10 client.31721 handle_command_reply: tid=0 2024-03-21T11:42:37.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.833+0000 7f2a04637740 1 client.31721 shutdown 2024-03-21T11:42:37.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.833+0000 7f2a04637740 1 -- 172.21.15.47:0/1089488504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 msgr2=0x555f892b5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:37.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.833+0000 7f2a04637740 1 --2- 172.21.15.47:0/1089488504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892b5a60 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f29e4005190 tx=0x7f29e400f710 comp rx=0 tx=0).stop 2024-03-21T11:42:37.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.834+0000 7f2a04637740 1 -- 172.21.15.47:0/1089488504 shutdown_connections 2024-03-21T11:42:37.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.834+0000 7f2a04637740 1 --2- 172.21.15.47:0/1089488504 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555f89321110 0x555f89323530 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.834+0000 7f2a04637740 1 --2- 172.21.15.47:0/1089488504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555f892ac9e0 0x555f892b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.834+0000 7f2a04637740 1 --2- 172.21.15.47:0/1089488504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555f892a7870 0x555f892b5a60 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.834+0000 7f2a04637740 1 --2- 172.21.15.47:0/1089488504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555f892a6e80 0x555f892b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.834+0000 7f2a04637740 1 -- 172.21.15.47:0/1089488504 >> 172.21.15.47:0/1089488504 conn(0x555f8926f200 msgr2=0x555f892a1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:37.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.834+0000 7f29e8c00640 0 client.31721 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:37.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.834+0000 7f2a04637740 1 -- 172.21.15.47:0/1089488504 shutdown_connections 2024-03-21T11:42:37.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.834+0000 7f2a04637740 1 -- 172.21.15.47:0/1089488504 wait complete. 2024-03-21T11:42:37.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.834+0000 7f2a04637740 20 client.31721 trim_cache size 0 max 16384 2024-03-21T11:42:37.836 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:42:37.836 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:42:37.836 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:42:37.836 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:42:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.835+0000 7f2a01800640 1 -- 172.21.15.47:0/26240647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f29e00a2fa0 msgr2=0x7f29e00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.835+0000 7f2a01800640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f29e00a2fa0 0x7f29e00a5460 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0714f0 tx=0x7f29e4009b40 comp rx=0 tx=0).stop 2024-03-21T11:42:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.835+0000 7f2a01800640 1 -- 172.21.15.47:0/26240647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29fc105490 msgr2=0x7f29fc1a9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.835+0000 7f2a01800640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29fc105490 0x7f29fc1a9c50 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f29f402c640 tx=0x7f29f4035d90 comp rx=0 tx=0).stop 2024-03-21T11:42:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.836+0000 7f2a01800640 1 -- 172.21.15.47:0/26240647 shutdown_connections 2024-03-21T11:42:37.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.836+0000 7f2a01800640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f29e00a2fa0 0x7f29e00a5460 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.836+0000 7f2a01800640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29fc105490 0x7f29fc1a9c50 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.836+0000 7f2a01800640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29fc104ad0 0x7f29fc070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.836+0000 7f2a01800640 1 --2- 172.21.15.47:0/26240647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29fc1038e0 0x7f29fc070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:37.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.836+0000 7f2a01800640 1 -- 172.21.15.47:0/26240647 >> 172.21.15.47:0/26240647 conn(0x7f29fc0fee50 msgr2=0x7f29fc107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:37.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.836+0000 7f2a01800640 1 -- 172.21.15.47:0/26240647 shutdown_connections 2024-03-21T11:42:37.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:37.837+0000 7f2a01800640 1 -- 172.21.15.47:0/26240647 wait complete. 2024-03-21T11:42:37.867 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:42:37.867 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:42:37.957 INFO:tasks.workunit.client.0.smithi047.stdout: 10 145450 23.35 MB/sec execute 201 sec latency 749.552 ms 2024-03-21T11:42:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.148+0000 7f8b10000640 1 -- 172.21.15.47:0/2324064494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b08100ae0 msgr2=0x7f8b08100f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:38.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.148+0000 7f8b10000640 1 --2- 172.21.15.47:0/2324064494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b08100ae0 0x7f8b08100f60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f8af401a910 tx=0x7f8af403cc70 comp rx=0 tx=0).stop 2024-03-21T11:42:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.149+0000 7f8b10000640 1 -- 172.21.15.47:0/2324064494 shutdown_connections 2024-03-21T11:42:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.149+0000 7f8b10000640 1 --2- 172.21.15.47:0/2324064494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b081014a0 0x7f8b08107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.149+0000 7f8b10000640 1 --2- 172.21.15.47:0/2324064494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b08100ae0 0x7f8b08100f60 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.149+0000 7f8b10000640 1 --2- 172.21.15.47:0/2324064494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b080ff8f0 0x7f8b080ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.149+0000 7f8b10000640 1 -- 172.21.15.47:0/2324064494 >> 172.21.15.47:0/2324064494 conn(0x7f8b080faea0 msgr2=0x7f8b080fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:38.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.150+0000 7f8b10000640 1 -- 172.21.15.47:0/2324064494 shutdown_connections 2024-03-21T11:42:38.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.150+0000 7f8b10000640 1 -- 172.21.15.47:0/2324064494 wait complete. 2024-03-21T11:42:38.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.152+0000 7f8b10000640 1 Processor -- start 2024-03-21T11:42:38.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.152+0000 7f8b10000640 1 -- start start 2024-03-21T11:42:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.153+0000 7f8b10000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b080ff8f0 0x7f8b0818f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.153+0000 7f8b10000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08100ae0 0x7f8b08191dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.153+0000 7f8b10000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b081014a0 0x7f8b08194320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.153+0000 7f8b10000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0806d8d0 con 0x7f8b080ff8f0 2024-03-21T11:42:38.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.153+0000 7f8b10000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0806d750 con 0x7f8b081014a0 2024-03-21T11:42:38.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.153+0000 7f8b10000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0806da50 con 0x7f8b08100ae0 2024-03-21T11:42:38.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.153+0000 7f8b0e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b081014a0 0x7f8b08194320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.153+0000 7f8b0d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b080ff8f0 0x7f8b0818f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.153+0000 7f8b0e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b081014a0 0x7f8b08194320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41246/0 (socket says 172.21.15.47:41246) 2024-03-21T11:42:38.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.154+0000 7f8b0e000640 1 -- 172.21.15.47:0/3387332491 learned_addr learned my addr 172.21.15.47:0/3387332491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:38.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.154+0000 7f8b0cc00640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08100ae0 0x7f8b08191dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.155+0000 7f8b0e000640 1 -- 172.21.15.47:0/3387332491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08100ae0 msgr2=0x7f8b08191dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.155+0000 7f8b0e000640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08100ae0 0x7f8b08191dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.155+0000 7f8b0e000640 1 -- 172.21.15.47:0/3387332491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b080ff8f0 msgr2=0x7f8b0818f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.155+0000 7f8b0e000640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b080ff8f0 0x7f8b0818f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.155+0000 7f8b0e000640 1 -- 172.21.15.47:0/3387332491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b08194b90 con 0x7f8b081014a0 2024-03-21T11:42:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.155+0000 7f8b0cc00640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08100ae0 0x7f8b08191dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:38.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.155+0000 7f8b0d600640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b080ff8f0 0x7f8b0818f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:38.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.155+0000 7f8b0e000640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b081014a0 0x7f8b08194320 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000ec90 tx=0x7f8b0000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:38.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.156+0000 7f8b06000640 1 -- 172.21.15.47:0/3387332491 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00004d20 con 0x7f8b081014a0 2024-03-21T11:42:38.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.156+0000 7f8b10000640 1 -- 172.21.15.47:0/3387332491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b08194e80 con 0x7f8b081014a0 2024-03-21T11:42:38.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.156+0000 7f8b10000640 1 -- 172.21.15.47:0/3387332491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b08195340 con 0x7f8b081014a0 2024-03-21T11:42:38.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.156+0000 7f8b06000640 1 -- 172.21.15.47:0/3387332491 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b000168d0 con 0x7f8b081014a0 2024-03-21T11:42:38.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.157+0000 7f8b06000640 1 -- 172.21.15.47:0/3387332491 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00024060 con 0x7f8b081014a0 2024-03-21T11:42:38.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.159+0000 7f8b10000640 1 -- 172.21.15.47:0/3387332491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae80052c0 con 0x7f8b081014a0 2024-03-21T11:42:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.165+0000 7f8b06000640 1 -- 172.21.15.47:0/3387332491 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8b000163f0 con 0x7f8b081014a0 2024-03-21T11:42:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.167+0000 7f8b06000640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8af00a7380 0x7f8af00a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.167+0000 7f8b06000640 1 -- 172.21.15.47:0/3387332491 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8b000cf570 con 0x7f8b081014a0 2024-03-21T11:42:38.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.167+0000 7f8b0d600640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8af00a7380 0x7f8af00a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.170+0000 7f8b0d600640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8af00a7380 0x7f8af00a9840 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8b08108040 tx=0x7f8afc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:38.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.177+0000 7f8b06000640 1 -- 172.21.15.47:0/3387332491 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8b00013070 con 0x7f8b081014a0 2024-03-21T11:42:38.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.578+0000 7f8b10000640 1 -- 172.21.15.47:0/3387332491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ae80055b0 con 0x7f8b081014a0 2024-03-21T11:42:38.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.579+0000 7f8b06000640 1 -- 172.21.15.47:0/3387332491 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f8b0009b9c0 con 0x7f8b081014a0 2024-03-21T11:42:38.580 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:42:38.580 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:39:47.303461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:42:38.581 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-21T11:42:38.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.582+0000 7f8b10000640 1 -- 172.21.15.47:0/3387332491 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8af00a7380 msgr2=0x7f8af00a9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:38.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.582+0000 7f8b10000640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8af00a7380 0x7f8af00a9840 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8b08108040 tx=0x7f8afc00b390 comp rx=0 tx=0).stop 2024-03-21T11:42:38.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.582+0000 7f8b10000640 1 -- 172.21.15.47:0/3387332491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b081014a0 msgr2=0x7f8b08194320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:38.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.582+0000 7f8b10000640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b081014a0 0x7f8b08194320 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000ec90 tx=0x7f8b0000c6a0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.582+0000 7f8b10000640 1 -- 172.21.15.47:0/3387332491 shutdown_connections 2024-03-21T11:42:38.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.582+0000 7f8b10000640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8af00a7380 0x7f8af00a9840 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.582+0000 7f8b10000640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b081014a0 0x7f8b08194320 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.582+0000 7f8b10000640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08100ae0 0x7f8b08191dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.582+0000 7f8b10000640 1 --2- 172.21.15.47:0/3387332491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b080ff8f0 0x7f8b0818f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.582+0000 7f8b10000640 1 -- 172.21.15.47:0/3387332491 >> 172.21.15.47:0/3387332491 conn(0x7f8b080faea0 msgr2=0x7f8b080702e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:38.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.582+0000 7f8b10000640 1 -- 172.21.15.47:0/3387332491 shutdown_connections 2024-03-21T11:42:38.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.582+0000 7f8b10000640 1 -- 172.21.15.47:0/3387332491 wait complete. 2024-03-21T11:42:38.602 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:42:38.602 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:42:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.840+0000 7f6d5ca00640 1 -- 172.21.15.47:0/2475935716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d58107cc0 msgr2=0x7f6d581080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.840+0000 7f6d5ca00640 1 --2- 172.21.15.47:0/2475935716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d58107cc0 0x7f6d581080e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009ca0 tx=0x7f6d5002c100 comp rx=0 tx=0).stop 2024-03-21T11:42:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.840+0000 7f6d5ca00640 1 -- 172.21.15.47:0/2475935716 shutdown_connections 2024-03-21T11:42:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.840+0000 7f6d5ca00640 1 --2- 172.21.15.47:0/2475935716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d58109870 0x7f6d58110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.840+0000 7f6d5ca00640 1 --2- 172.21.15.47:0/2475935716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d58108eb0 0x7f6d58109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.840+0000 7f6d5ca00640 1 --2- 172.21.15.47:0/2475935716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d58107cc0 0x7f6d581080e0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.840+0000 7f6d5ca00640 1 -- 172.21.15.47:0/2475935716 >> 172.21.15.47:0/2475935716 conn(0x7f6d58076c60 msgr2=0x7f6d580790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.840+0000 7f6d5ca00640 1 -- 172.21.15.47:0/2475935716 shutdown_connections 2024-03-21T11:42:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.840+0000 7f6d5ca00640 1 -- 172.21.15.47:0/2475935716 wait complete. 2024-03-21T11:42:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.842+0000 7f6d5ca00640 1 Processor -- start 2024-03-21T11:42:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.843+0000 7f6d5ca00640 1 -- start start 2024-03-21T11:42:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.843+0000 7f6d5ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d58107cc0 0x7f6d5819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.843+0000 7f6d5ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d58108eb0 0x7f6d5819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.843+0000 7f6d5ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d58109870 0x7f6d581a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.843+0000 7f6d5ca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d58113910 con 0x7f6d58107cc0 2024-03-21T11:42:38.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.843+0000 7f6d55400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d58108eb0 0x7f6d5819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.843+0000 7f6d55400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d58108eb0 0x7f6d5819e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53762/0 (socket says 172.21.15.47:53762) 2024-03-21T11:42:38.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.843+0000 7f6d55400640 1 -- 172.21.15.47:0/3959736037 learned_addr learned my addr 172.21.15.47:0/3959736037 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:38.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.844+0000 7f6d55e00640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d58107cc0 0x7f6d5819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.844+0000 7f6d56800640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d58109870 0x7f6d581a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.844+0000 7f6d5ca00640 1 -- 172.21.15.47:0/3959736037 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d58113790 con 0x7f6d58109870 2024-03-21T11:42:38.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.844+0000 7f6d5ca00640 1 -- 172.21.15.47:0/3959736037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d58113a90 con 0x7f6d58108eb0 2024-03-21T11:42:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.844+0000 7f6d55e00640 1 -- 172.21.15.47:0/3959736037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d58108eb0 msgr2=0x7f6d5819e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.844+0000 7f6d55e00640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d58108eb0 0x7f6d5819e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.844+0000 7f6d55e00640 1 -- 172.21.15.47:0/3959736037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d58109870 msgr2=0x7f6d581a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.844+0000 7f6d55e00640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d58109870 0x7f6d581a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.844+0000 7f6d55e00640 1 -- 172.21.15.47:0/3959736037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5807a020 con 0x7f6d58107cc0 2024-03-21T11:42:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.844+0000 7f6d56800640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d58109870 0x7f6d581a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.844+0000 7f6d55400640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d58108eb0 0x7f6d5819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.845+0000 7f6d55e00640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d58107cc0 0x7f6d5819e010 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009bf0 tx=0x7f6d50035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.845+0000 7f6d4ea00640 1 -- 172.21.15.47:0/3959736037 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50039040 con 0x7f6d58107cc0 2024-03-21T11:42:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.845+0000 7f6d4ea00640 1 -- 172.21.15.47:0/3959736037 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d50003970 con 0x7f6d58107cc0 2024-03-21T11:42:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.845+0000 7f6d4ea00640 1 -- 172.21.15.47:0/3959736037 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d500042a0 con 0x7f6d58107cc0 2024-03-21T11:42:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.845+0000 7f6d5ca00640 1 -- 172.21.15.47:0/3959736037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5807a2b0 con 0x7f6d58107cc0 2024-03-21T11:42:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.845+0000 7f6d5ca00640 1 -- 172.21.15.47:0/3959736037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5807a720 con 0x7f6d58107cc0 2024-03-21T11:42:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.849+0000 7f6d5f662740 1 Processor -- start 2024-03-21T11:42:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.849+0000 7f6d5f662740 1 -- start start 2024-03-21T11:42:38.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.849+0000 7f6d5f662740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56b810 0x55e4ed56bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.849+0000 7f6d5f662740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56c9e0 0x55e4ed56ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.849+0000 7f6d5f662740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56d3a0 0x55e4ed573c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.849+0000 7f6d5f662740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ed4d02c0 con 0x55e4ed56c9e0 2024-03-21T11:42:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.849+0000 7f6d5f662740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ed478c90 con 0x55e4ed56b810 2024-03-21T11:42:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.849+0000 7f6d5f662740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ed4cc360 con 0x55e4ed56d3a0 2024-03-21T11:42:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.850+0000 7f6d56800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56b810 0x55e4ed56bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.850+0000 7f6d56800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56b810 0x55e4ed56bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41260/0 (socket says 172.21.15.47:41260) 2024-03-21T11:42:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.850+0000 7f6d56800640 1 -- 172.21.15.47:0/3193390720 learned_addr learned my addr 172.21.15.47:0/3193390720 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.850+0000 7f6d55400640 1 --2- 172.21.15.47:0/3193390720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56d3a0 0x55e4ed573c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.850+0000 7f6d56800640 1 -- 172.21.15.47:0/3193390720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56d3a0 msgr2=0x55e4ed573c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.850+0000 7f6d56800640 1 --2- 172.21.15.47:0/3193390720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56d3a0 0x55e4ed573c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.850+0000 7f6d56800640 1 -- 172.21.15.47:0/3193390720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56c9e0 msgr2=0x55e4ed56ce60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:42:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.850+0000 7f6d56800640 1 --2- 172.21.15.47:0/3193390720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56c9e0 0x55e4ed56ce60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.850+0000 7f6d56800640 1 -- 172.21.15.47:0/3193390720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4ed49d080 con 0x55e4ed56b810 2024-03-21T11:42:38.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.850+0000 7f6d4ea00640 1 -- 172.21.15.47:0/3959736037 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6d50046030 con 0x7f6d58107cc0 2024-03-21T11:42:38.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.850+0000 7f6d56800640 1 --2- 172.21.15.47:0/3193390720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56b810 0x55e4ed56bc30 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f6d48011570 tx=0x7f6d48011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=343b25c45d6d312d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:38.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.851+0000 7f6d43000640 1 -- 172.21.15.47:0/3193390720 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4801e070 con 0x55e4ed56b810 2024-03-21T11:42:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.851+0000 7f6d43000640 1 -- 172.21.15.47:0/3193390720 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d4800faa0 con 0x55e4ed56b810 2024-03-21T11:42:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.851+0000 7f6d43000640 1 -- 172.21.15.47:0/3193390720 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d48019c00 con 0x55e4ed56b810 2024-03-21T11:42:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.852+0000 7f6d4ea00640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d3c0a2fa0 0x7f6d3c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.852+0000 7f6d4ea00640 1 -- 172.21.15.47:0/3959736037 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6d500f1050 con 0x7f6d58107cc0 2024-03-21T11:42:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.852+0000 7f6d5f662740 1 -- 172.21.15.47:0/3193390720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56b810 msgr2=0x55e4ed56bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.852+0000 7f6d5f662740 1 --2- 172.21.15.47:0/3193390720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56b810 0x55e4ed56bc30 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f6d48011570 tx=0x7f6d48011a30 comp rx=0 tx=0).stop 2024-03-21T11:42:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.852+0000 7f6d55e00640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d3c0a2fa0 0x7f6d3c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.852+0000 7f6d5f662740 1 -- 172.21.15.47:0/3193390720 shutdown_connections 2024-03-21T11:42:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.852+0000 7f6d5f662740 1 --2- 172.21.15.47:0/3193390720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56d3a0 0x55e4ed573c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.852+0000 7f6d5f662740 1 --2- 172.21.15.47:0/3193390720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56c9e0 0x55e4ed56ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.852+0000 7f6d5f662740 1 --2- 172.21.15.47:0/3193390720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56b810 0x55e4ed56bc30 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.852+0000 7f6d5f662740 1 -- 172.21.15.47:0/3193390720 >> 172.21.15.47:0/3193390720 conn(0x55e4ed566c00 msgr2=0x55e4ed567000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.853+0000 7f6d5f662740 1 -- 172.21.15.47:0/3193390720 shutdown_connections 2024-03-21T11:42:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.853+0000 7f6d5f662740 1 -- 172.21.15.47:0/3193390720 wait complete. 2024-03-21T11:42:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.853+0000 7f6d55e00640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d3c0a2fa0 0x7f6d3c0a5460 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6d500b9e50 tx=0x7f6d50003db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.853+0000 7f6d5f662740 1 -- start start 2024-03-21T11:42:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.854+0000 7f6d5f662740 1 Processor -- start 2024-03-21T11:42:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.854+0000 7f6d5f662740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56b810 0x55e4ed575ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.854+0000 7f6d5f662740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56c9e0 0x55e4ed576000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.854+0000 7f6d5f662740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 0x55e4ed57d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.854+0000 7f6d5f662740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ed5743b0 con 0x55e4ed56d3a0 2024-03-21T11:42:38.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.854+0000 7f6d5f662740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ed48c0d0 con 0x55e4ed56c9e0 2024-03-21T11:42:38.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.854+0000 7f6d5f662740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ed574530 con 0x55e4ed56b810 2024-03-21T11:42:38.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.854+0000 7f6d5f662740 10 client.0 _ll_register_callbacks cb 0x55e4ed490e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:38.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.854+0000 7f6d56800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56b810 0x55e4ed575ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.854+0000 7f6d55400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56c9e0 0x55e4ed576000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.854+0000 7f6d56800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56b810 0x55e4ed575ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53782/0 (socket says 172.21.15.47:53782) 2024-03-21T11:42:38.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.854+0000 7f6d56800640 1 -- 172.21.15.47:0/648013136 learned_addr learned my addr 172.21.15.47:0/648013136 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.855+0000 7f6d56800640 1 --2- 172.21.15.47:0/648013136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 0x55e4ed57d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.855+0000 7f6d56800640 1 -- 172.21.15.47:0/648013136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56c9e0 msgr2=0x55e4ed576000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.855+0000 7f6d56800640 1 --2- 172.21.15.47:0/648013136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56c9e0 0x55e4ed576000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.855+0000 7f6d56800640 1 -- 172.21.15.47:0/648013136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 msgr2=0x55e4ed57d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.855+0000 7f6d56800640 1 --2- 172.21.15.47:0/648013136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 0x55e4ed57d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.855+0000 7f6d56800640 1 -- 172.21.15.47:0/648013136 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4ed1e43f0 con 0x55e4ed56b810 2024-03-21T11:42:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.855+0000 7f6d56800640 1 --2- 172.21.15.47:0/648013136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56b810 0x55e4ed575ac0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f6d48011540 tx=0x7f6d48009cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4706d95b09635a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.855+0000 7f6d37400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:42:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.856+0000 7f6d37400640 1 -- 172.21.15.47:0/648013136 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4802c060 con 0x55e4ed56b810 2024-03-21T11:42:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.856+0000 7f6d5f662740 1 -- 172.21.15.47:0/648013136 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d4800faa0 con 0x55e4ed56b810 2024-03-21T11:42:38.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.856+0000 7f6d37400640 1 -- 172.21.15.47:0/648013136 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6d48019b50 con 0x55e4ed56b810 2024-03-21T11:42:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.856+0000 7f6d37400640 1 -- 172.21.15.47:0/648013136 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d48033050 con 0x55e4ed56b810 2024-03-21T11:42:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.856+0000 7f6d5f662740 10 client.31717 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.856+0000 7f6d5f662740 1 -- 172.21.15.47:0/648013136 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e4ed2ae7d0 con 0x55e4ed56b810 2024-03-21T11:42:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.856+0000 7f6d37400640 1 -- 172.21.15.47:0/648013136 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d4800faa0 con 0x55e4ed56b810 2024-03-21T11:42:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.857+0000 7f6d37400640 1 -- 172.21.15.47:0/648013136 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6d48004990 con 0x55e4ed56b810 2024-03-21T11:42:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.857+0000 7f6d5f662740 10 client.31717 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.857+0000 7f6d5f662740 10 client.31717 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:42:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.857+0000 7f6d5f662740 20 client.31717 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.857+0000 7f6d5f662740 1 --2- 172.21.15.47:0/648013136 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e4ed5812b0 0x55e4ed5836c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.857+0000 7f6d5f662740 4 client.31717 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.857+0000 7f6d5f662740 1 -- 172.21.15.47:0/648013136 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4ed4dd800 con 0x55e4ed5812b0 2024-03-21T11:42:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.857+0000 7f6d55400640 1 --2- 172.21.15.47:0/648013136 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e4ed5812b0 0x55e4ed5836c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.858+0000 7f6d55400640 1 --2- 172.21.15.47:0/648013136 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e4ed5812b0 0x55e4ed5836c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4688211e8c4d7d43 server_cookie=9182439bb8427e3f in_seq=0 out_seq=0 2024-03-21T11:42:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.858+0000 7f6d37400640 10 client.31717 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:42:38.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.861+0000 7f6d37400640 1 -- 172.21.15.47:0/648013136 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e4ed4dd800 con 0x55e4ed5812b0 2024-03-21T11:42:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.861+0000 7f6d37400640 10 client.31717 handle_command_reply: tid=0 2024-03-21T11:42:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.861+0000 7f6d5f662740 1 client.31717 shutdown 2024-03-21T11:42:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.861+0000 7f6d5f662740 1 -- 172.21.15.47:0/648013136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56b810 msgr2=0x55e4ed575ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.861+0000 7f6d5f662740 1 --2- 172.21.15.47:0/648013136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56b810 0x55e4ed575ac0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f6d48011540 tx=0x7f6d48009cb0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.861+0000 7f6d5f662740 1 -- 172.21.15.47:0/648013136 shutdown_connections 2024-03-21T11:42:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.861+0000 7f6d5f662740 1 --2- 172.21.15.47:0/648013136 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e4ed5812b0 0x55e4ed5836c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.861+0000 7f6d5f662740 1 --2- 172.21.15.47:0/648013136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 0x55e4ed57d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.861+0000 7f6d5f662740 1 --2- 172.21.15.47:0/648013136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56c9e0 0x55e4ed576000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.861+0000 7f6d5f662740 1 --2- 172.21.15.47:0/648013136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56b810 0x55e4ed575ac0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.861+0000 7f6d5f662740 1 -- 172.21.15.47:0/648013136 >> 172.21.15.47:0/648013136 conn(0x55e4ed566c00 msgr2=0x55e4ed565eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:38.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.861+0000 7f6d37400640 0 client.31717 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:42:38.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.862+0000 7f6d5f662740 1 -- 172.21.15.47:0/648013136 shutdown_connections 2024-03-21T11:42:38.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.862+0000 7f6d5f662740 1 -- 172.21.15.47:0/648013136 wait complete. 2024-03-21T11:42:38.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.862+0000 7f6d5f662740 20 client.31717 trim_cache size 0 max 16384 2024-03-21T11:42:38.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.878+0000 7f6d5f662740 1 Processor -- start 2024-03-21T11:42:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.879+0000 7f6d5f662740 1 -- start start 2024-03-21T11:42:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.879+0000 7f6d5f662740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56b810 0x55e4ed587f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.879+0000 7f6d5f662740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56c9e0 0x55e4ed5884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.879+0000 7f6d5f662740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 0x55e4ed588a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.879+0000 7f6d5f662740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d581b8e50 con 0x55e4ed56d3a0 2024-03-21T11:42:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.879+0000 7f6d5f662740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d581ba7d0 con 0x55e4ed56c9e0 2024-03-21T11:42:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.879+0000 7f6d5f662740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d581b94b0 con 0x55e4ed56b810 2024-03-21T11:42:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.879+0000 7f6d56800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56b810 0x55e4ed587f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.879+0000 7f6d55400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56c9e0 0x55e4ed5884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.879+0000 7f6d55400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56c9e0 0x55e4ed5884c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60366/0 (socket says 172.21.15.47:60366) 2024-03-21T11:42:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.879+0000 7f6d56800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56b810 0x55e4ed587f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34972/0 (socket says 172.21.15.47:34972) 2024-03-21T11:42:38.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.879+0000 7f6d55400640 1 -- 172.21.15.47:0/3464848877 learned_addr learned my addr 172.21.15.47:0/3464848877 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:38.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.879+0000 7f6d56800640 1 --2- 172.21.15.47:0/3464848877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 0x55e4ed588a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.880+0000 7f6d56800640 1 -- 172.21.15.47:0/3464848877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56c9e0 msgr2=0x55e4ed5884c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:38.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.880+0000 7f6d56800640 1 --2- 172.21.15.47:0/3464848877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56c9e0 0x55e4ed5884c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.880+0000 7f6d56800640 1 -- 172.21.15.47:0/3464848877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 msgr2=0x55e4ed588a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:38.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.880+0000 7f6d56800640 1 --2- 172.21.15.47:0/3464848877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 0x55e4ed588a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.880+0000 7f6d56800640 1 -- 172.21.15.47:0/3464848877 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4ed567970 con 0x55e4ed56b810 2024-03-21T11:42:38.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.880+0000 7f6d56800640 1 --2- 172.21.15.47:0/3464848877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56b810 0x55e4ed587f80 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f6d48011570 tx=0x7f6d48005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c01312cb4824a530 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:38.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.880+0000 7f6d42600640 1 -- 172.21.15.47:0/3464848877 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4802c040 con 0x55e4ed56b810 2024-03-21T11:42:38.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.880+0000 7f6d42600640 1 -- 172.21.15.47:0/3464848877 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d4800fa40 con 0x55e4ed56b810 2024-03-21T11:42:38.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.880+0000 7f6d42600640 1 -- 172.21.15.47:0/3464848877 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4801e040 con 0x55e4ed56b810 2024-03-21T11:42:38.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.881+0000 7f6d5f662740 1 -- 172.21.15.47:0/3464848877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56b810 msgr2=0x55e4ed587f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:38.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.881+0000 7f6d5f662740 1 --2- 172.21.15.47:0/3464848877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56b810 0x55e4ed587f80 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f6d48011570 tx=0x7f6d48005970 comp rx=0 tx=0).stop 2024-03-21T11:42:38.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.881+0000 7f6d5f662740 1 -- 172.21.15.47:0/3464848877 shutdown_connections 2024-03-21T11:42:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.881+0000 7f6d5f662740 1 --2- 172.21.15.47:0/3464848877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 0x55e4ed588a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.881+0000 7f6d5f662740 1 --2- 172.21.15.47:0/3464848877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56c9e0 0x55e4ed5884c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.881+0000 7f6d5f662740 1 --2- 172.21.15.47:0/3464848877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56b810 0x55e4ed587f80 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.881+0000 7f6d5f662740 1 -- 172.21.15.47:0/3464848877 >> 172.21.15.47:0/3464848877 conn(0x55e4ed566c00 msgr2=0x55e4ed56a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.881+0000 7f6d5f662740 1 -- 172.21.15.47:0/3464848877 shutdown_connections 2024-03-21T11:42:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.881+0000 7f6d5f662740 1 -- 172.21.15.47:0/3464848877 wait complete. 2024-03-21T11:42:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.882+0000 7f6d5f662740 1 -- start start 2024-03-21T11:42:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.882+0000 7f6d5f662740 1 Processor -- start 2024-03-21T11:42:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.882+0000 7f6d5f662740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56b810 0x55e4ed578340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.882+0000 7f6d5f662740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56c9e0 0x55e4ed578880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.882+0000 7f6d5f662740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 0x55e4ed57b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.882+0000 7f6d5f662740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d581ba170 con 0x55e4ed56d3a0 2024-03-21T11:42:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.882+0000 7f6d5f662740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d581b9b10 con 0x55e4ed56b810 2024-03-21T11:42:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.882+0000 7f6d5f662740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d580822e0 con 0x55e4ed56c9e0 2024-03-21T11:42:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.882+0000 7f6d5f662740 10 client.0 _ll_register_callbacks cb 0x7f6d4801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.882+0000 7f6d56800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56b810 0x55e4ed578340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.882+0000 7f6d55400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56c9e0 0x55e4ed578880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.883+0000 7f6d55400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56c9e0 0x55e4ed578880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34976/0 (socket says 172.21.15.47:34976) 2024-03-21T11:42:38.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.883+0000 7f6d56800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56b810 0x55e4ed578340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60370/0 (socket says 172.21.15.47:60370) 2024-03-21T11:42:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.883+0000 7f6d55400640 1 -- 172.21.15.47:0/38314289 learned_addr learned my addr 172.21.15.47:0/38314289 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.883+0000 7f6d56800640 1 --2- 172.21.15.47:0/38314289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 0x55e4ed57b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.883+0000 7f6d56800640 1 -- 172.21.15.47:0/38314289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56c9e0 msgr2=0x55e4ed578880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.883+0000 7f6d56800640 1 --2- 172.21.15.47:0/38314289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56c9e0 0x55e4ed578880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.883+0000 7f6d56800640 1 -- 172.21.15.47:0/38314289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 msgr2=0x55e4ed57b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.883+0000 7f6d56800640 1 --2- 172.21.15.47:0/38314289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 0x55e4ed57b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.883+0000 7f6d56800640 1 -- 172.21.15.47:0/38314289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4ed57bdf0 con 0x55e4ed56b810 2024-03-21T11:42:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.883+0000 7f6d56800640 1 --2- 172.21.15.47:0/38314289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56b810 0x55e4ed578340 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x55e4ed576d90 tx=0x7f6d48009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1607e0f56664ffb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.883+0000 7f6d37400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.884+0000 7f6d37400640 1 -- 172.21.15.47:0/38314289 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4802c030 con 0x55e4ed56b810 2024-03-21T11:42:38.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.884+0000 7f6d37400640 1 -- 172.21.15.47:0/38314289 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6d4800f840 con 0x55e4ed56b810 2024-03-21T11:42:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.884+0000 7f6d5f662740 1 -- 172.21.15.47:0/38314289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d4800fa40 con 0x55e4ed56b810 2024-03-21T11:42:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.884+0000 7f6d37400640 1 -- 172.21.15.47:0/38314289 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d4801e040 con 0x55e4ed56b810 2024-03-21T11:42:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.884+0000 7f6d5f662740 10 client.31745 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.884+0000 7f6d5f662740 1 -- 172.21.15.47:0/38314289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e4ed5dcf10 con 0x55e4ed56b810 2024-03-21T11:42:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.884+0000 7f6d37400640 1 -- 172.21.15.47:0/38314289 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d4800fa40 con 0x55e4ed56b810 2024-03-21T11:42:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.885+0000 7f6d37400640 1 -- 172.21.15.47:0/38314289 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6d480051c0 con 0x55e4ed56b810 2024-03-21T11:42:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.885+0000 7f6d5f662740 10 client.31745 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.885+0000 7f6d5f662740 10 client.31745 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:42:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.885+0000 7f6d5f662740 20 client.31745 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.885+0000 7f6d5f662740 1 --2- 172.21.15.47:0/38314289 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e4ed5dd6e0 0x55e4ed5dfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.885+0000 7f6d5f662740 4 client.31745 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:42:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.885+0000 7f6d5f662740 1 -- 172.21.15.47:0/38314289 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e4ed4d7b50 con 0x55e4ed5dd6e0 2024-03-21T11:42:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.885+0000 7f6d55400640 1 --2- 172.21.15.47:0/38314289 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e4ed5dd6e0 0x55e4ed5dfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d55400640 1 --2- 172.21.15.47:0/38314289 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e4ed5dd6e0 0x55e4ed5dfb20 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e0d6876ec5546a34 server_cookie=175a5391f8e30a0 in_seq=0 out_seq=0 2024-03-21T11:42:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d37400640 10 client.31745 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:42:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d37400640 1 -- 172.21.15.47:0/38314289 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e4ed4d7b50 con 0x55e4ed5dd6e0 2024-03-21T11:42:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d37400640 10 client.31745 handle_command_reply: tid=0 2024-03-21T11:42:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d5f662740 1 client.31745 shutdown 2024-03-21T11:42:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d5f662740 1 -- 172.21.15.47:0/38314289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56b810 msgr2=0x55e4ed578340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d5f662740 1 --2- 172.21.15.47:0/38314289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56b810 0x55e4ed578340 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x55e4ed576d90 tx=0x7f6d48009260 comp rx=0 tx=0).stop 2024-03-21T11:42:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d5f662740 1 -- 172.21.15.47:0/38314289 shutdown_connections 2024-03-21T11:42:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d5f662740 1 --2- 172.21.15.47:0/38314289 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55e4ed5dd6e0 0x55e4ed5dfb20 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d5f662740 1 --2- 172.21.15.47:0/38314289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4ed56d3a0 0x55e4ed57b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d5f662740 1 --2- 172.21.15.47:0/38314289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4ed56c9e0 0x55e4ed578880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d5f662740 1 --2- 172.21.15.47:0/38314289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e4ed56b810 0x55e4ed578340 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d5f662740 1 -- 172.21.15.47:0/38314289 >> 172.21.15.47:0/38314289 conn(0x55e4ed566c00 msgr2=0x55e4ed56fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d37400640 0 client.31745 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:42:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.886+0000 7f6d5f662740 1 -- 172.21.15.47:0/38314289 shutdown_connections 2024-03-21T11:42:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.887+0000 7f6d5f662740 1 -- 172.21.15.47:0/38314289 wait complete. 2024-03-21T11:42:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.887+0000 7f6d5f662740 20 client.31745 trim_cache size 0 max 16384 2024-03-21T11:42:38.893 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:42:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.887+0000 7f6d5ca00640 1 -- 172.21.15.47:0/3959736037 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d3c0a2fa0 msgr2=0x7f6d3c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.887+0000 7f6d5ca00640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d3c0a2fa0 0x7f6d3c0a5460 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6d500b9e50 tx=0x7f6d50003db0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.887+0000 7f6d5ca00640 1 -- 172.21.15.47:0/3959736037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d58107cc0 msgr2=0x7f6d5819e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.887+0000 7f6d5ca00640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d58107cc0 0x7f6d5819e010 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009bf0 tx=0x7f6d50035d10 comp rx=0 tx=0).stop 2024-03-21T11:42:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.887+0000 7f6d5ca00640 1 -- 172.21.15.47:0/3959736037 shutdown_connections 2024-03-21T11:42:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.887+0000 7f6d5ca00640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d3c0a2fa0 0x7f6d3c0a5460 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.887+0000 7f6d5ca00640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d58109870 0x7f6d581a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.887+0000 7f6d5ca00640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d58108eb0 0x7f6d5819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.887+0000 7f6d5ca00640 1 --2- 172.21.15.47:0/3959736037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d58107cc0 0x7f6d5819e010 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.887+0000 7f6d5ca00640 1 -- 172.21.15.47:0/3959736037 >> 172.21.15.47:0/3959736037 conn(0x7f6d58076c60 msgr2=0x7f6d5810c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:38.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.888+0000 7f6d5ca00640 1 -- 172.21.15.47:0/3959736037 shutdown_connections 2024-03-21T11:42:38.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:38.888+0000 7f6d5ca00640 1 -- 172.21.15.47:0/3959736037 wait complete. 2024-03-21T11:42:38.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:38 smithi047 ceph-mon[31351]: pgmap v979: 129 pgs: 129 active+clean; 529 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.4 MiB/s wr, 553 op/s 2024-03-21T11:42:38.908 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:38 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3387332491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:42:38.913 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:42:38.913 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:42:38.958 INFO:tasks.workunit.client.0.smithi047.stdout: 10 145961 23.40 MB/sec execute 202 sec latency 126.503 ms 2024-03-21T11:42:38.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:38 smithi102 ceph-mon[36748]: pgmap v979: 129 pgs: 129 active+clean; 529 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.4 MiB/s wr, 553 op/s 2024-03-21T11:42:38.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:38 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3387332491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:42:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:38 smithi100 ceph-mon[36245]: pgmap v979: 129 pgs: 129 active+clean; 529 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.4 MiB/s wr, 553 op/s 2024-03-21T11:42:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:38 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3387332491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:42:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.179+0000 7fec19400640 1 -- 172.21.15.47:0/3424777022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec141008e0 msgr2=0x7fec14100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.179+0000 7fec19400640 1 --2- 172.21.15.47:0/3424777022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec141008e0 0x7fec14100d40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fec00009900 tx=0x7fec0002bc30 comp rx=0 tx=0).stop 2024-03-21T11:42:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.179+0000 7fec19400640 1 -- 172.21.15.47:0/3424777022 shutdown_connections 2024-03-21T11:42:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.179+0000 7fec19400640 1 --2- 172.21.15.47:0/3424777022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14101280 0x7fec14101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.179+0000 7fec19400640 1 --2- 172.21.15.47:0/3424777022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec141008e0 0x7fec14100d40 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.179+0000 7fec19400640 1 --2- 172.21.15.47:0/3424777022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec140fff80 0x7fec141003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.179+0000 7fec19400640 1 -- 172.21.15.47:0/3424777022 >> 172.21.15.47:0/3424777022 conn(0x7fec140fbb40 msgr2=0x7fec140fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.179+0000 7fec19400640 1 -- 172.21.15.47:0/3424777022 shutdown_connections 2024-03-21T11:42:39.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.179+0000 7fec19400640 1 -- 172.21.15.47:0/3424777022 wait complete. 2024-03-21T11:42:39.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.180+0000 7fec19400640 1 Processor -- start 2024-03-21T11:42:39.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.180+0000 7fec19400640 1 -- start start 2024-03-21T11:42:39.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec19400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec140fff80 0x7fec14199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec19400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec141008e0 0x7fec1419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec19400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec14101280 0x7fec141a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec19400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1410f520 con 0x7fec140fff80 2024-03-21T11:42:39.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec19400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1410f3a0 con 0x7fec141008e0 2024-03-21T11:42:39.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec19400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1410f6a0 con 0x7fec14101280 2024-03-21T11:42:39.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec12800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec140fff80 0x7fec14199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec12800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec140fff80 0x7fec14199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43538/0 (socket says 172.21.15.47:43538) 2024-03-21T11:42:39.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec11e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec141008e0 0x7fec1419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec12800640 1 -- 172.21.15.47:0/3528843398 learned_addr learned my addr 172.21.15.47:0/3528843398 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec13200640 1 --2- 172.21.15.47:0/3528843398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec14101280 0x7fec141a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec12800640 1 -- 172.21.15.47:0/3528843398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec14101280 msgr2=0x7fec141a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec12800640 1 --2- 172.21.15.47:0/3528843398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec14101280 0x7fec141a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec12800640 1 -- 172.21.15.47:0/3528843398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec141008e0 msgr2=0x7fec1419a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec12800640 1 --2- 172.21.15.47:0/3528843398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec141008e0 0x7fec1419a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec12800640 1 -- 172.21.15.47:0/3528843398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec141a1810 con 0x7fec140fff80 2024-03-21T11:42:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.181+0000 7fec13200640 1 --2- 172.21.15.47:0/3528843398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec14101280 0x7fec141a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:42:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.182+0000 7fec12800640 1 --2- 172.21.15.47:0/3528843398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec140fff80 0x7fec14199bf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fec0400e9f0 tx=0x7fec0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.182+0000 7fec0b400640 1 -- 172.21.15.47:0/3528843398 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0400cd80 con 0x7fec140fff80 2024-03-21T11:42:39.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.182+0000 7fec0b400640 1 -- 172.21.15.47:0/3528843398 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec04010910 con 0x7fec140fff80 2024-03-21T11:42:39.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.183+0000 7fec19400640 1 -- 172.21.15.47:0/3528843398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec141a1b00 con 0x7fec140fff80 2024-03-21T11:42:39.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.183+0000 7fec19400640 1 -- 172.21.15.47:0/3528843398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec141a2020 con 0x7fec140fff80 2024-03-21T11:42:39.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.184+0000 7fec0b400640 1 -- 172.21.15.47:0/3528843398 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec04018990 con 0x7fec140fff80 2024-03-21T11:42:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.188+0000 7fec1c1de740 1 Processor -- start 2024-03-21T11:42:39.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.189+0000 7fec1c1de740 1 -- start start 2024-03-21T11:42:39.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.189+0000 7fec0b400640 1 -- 172.21.15.47:0/3528843398 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fec04018c10 con 0x7fec140fff80 2024-03-21T11:42:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.189+0000 7fec1c1de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afd020 0x559659afd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.189+0000 7fec1c1de740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afe1f0 0x559659afe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.189+0000 7fec1c1de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afebb0 0x559659b05440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.189+0000 7fec1c1de740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559659a592c0 con 0x559659afe1f0 2024-03-21T11:42:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.189+0000 7fec1c1de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559659a01c90 con 0x559659afd020 2024-03-21T11:42:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.189+0000 7fec1c1de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559659a55360 con 0x559659afebb0 2024-03-21T11:42:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.190+0000 7fec13200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afd020 0x559659afd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.190+0000 7fec11e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afebb0 0x559659b05440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.190+0000 7fec11e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afebb0 0x559659b05440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34998/0 (socket says 172.21.15.47:34998) 2024-03-21T11:42:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.190+0000 7fec11e00640 1 -- 172.21.15.47:0/1585426605 learned_addr learned my addr 172.21.15.47:0/1585426605 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.190+0000 7fec13200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afd020 0x559659afd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60400/0 (socket says 172.21.15.47:60400) 2024-03-21T11:42:39.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.190+0000 7fec13200640 1 -- 172.21.15.47:0/1585426605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afebb0 msgr2=0x559659b05440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.191+0000 7fec12800640 1 --2- 172.21.15.47:0/1585426605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afe1f0 0x559659afe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.192+0000 7fec13200640 1 --2- 172.21.15.47:0/1585426605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afebb0 0x559659b05440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.192+0000 7fec13200640 1 -- 172.21.15.47:0/1585426605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afe1f0 msgr2=0x559659afe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.192+0000 7fec13200640 1 --2- 172.21.15.47:0/1585426605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afe1f0 0x559659afe670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.192+0000 7fec0b400640 1 --2- 172.21.15.47:0/3528843398 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7febf80a3070 0x7febf80a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.192+0000 7fec13200640 1 -- 172.21.15.47:0/1585426605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559659a26080 con 0x559659afd020 2024-03-21T11:42:39.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.192+0000 7fec12800640 1 --2- 172.21.15.47:0/1585426605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afe1f0 0x559659afe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:42:39.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.192+0000 7fec0b400640 1 -- 172.21.15.47:0/3528843398 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fec040cf9b0 con 0x7fec140fff80 2024-03-21T11:42:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.192+0000 7fec13200640 1 --2- 172.21.15.47:0/3528843398 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7febf80a3070 0x7febf80a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.192+0000 7fec13200640 1 --2- 172.21.15.47:0/1585426605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afd020 0x559659afd440 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009ec0 tx=0x7fec0c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c7e08143a933f7e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.193+0000 7febff400640 1 -- 172.21.15.47:0/1585426605 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c011a20 con 0x559659afd020 2024-03-21T11:42:39.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.193+0000 7febff400640 1 -- 172.21.15.47:0/1585426605 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec0c011c40 con 0x559659afd020 2024-03-21T11:42:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.193+0000 7febff400640 1 -- 172.21.15.47:0/1585426605 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c01c3c0 con 0x559659afd020 2024-03-21T11:42:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.194+0000 7fec13200640 1 --2- 172.21.15.47:0/3528843398 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7febf80a3070 0x7febf80a5530 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fec0c01e000 tx=0x7fec0c021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.194+0000 7fec1c1de740 1 -- 172.21.15.47:0/1585426605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afd020 msgr2=0x559659afd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.195+0000 7fec1c1de740 1 --2- 172.21.15.47:0/1585426605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afd020 0x559659afd440 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009ec0 tx=0x7fec0c00f730 comp rx=0 tx=0).stop 2024-03-21T11:42:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.195+0000 7fec1c1de740 1 -- 172.21.15.47:0/1585426605 shutdown_connections 2024-03-21T11:42:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.195+0000 7fec1c1de740 1 --2- 172.21.15.47:0/1585426605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afebb0 0x559659b05440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.195+0000 7fec1c1de740 1 --2- 172.21.15.47:0/1585426605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afe1f0 0x559659afe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.195+0000 7fec1c1de740 1 --2- 172.21.15.47:0/1585426605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afd020 0x559659afd440 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.195+0000 7fec1c1de740 1 -- 172.21.15.47:0/1585426605 >> 172.21.15.47:0/1585426605 conn(0x559659af8410 msgr2=0x559659af8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.195+0000 7fec1c1de740 1 -- 172.21.15.47:0/1585426605 shutdown_connections 2024-03-21T11:42:39.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.195+0000 7fec1c1de740 1 -- 172.21.15.47:0/1585426605 wait complete. 2024-03-21T11:42:39.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.196+0000 7fec1c1de740 1 -- start start 2024-03-21T11:42:39.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.197+0000 7fec1c1de740 1 Processor -- start 2024-03-21T11:42:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.197+0000 7fec1c1de740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afd020 0x559659af0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.197+0000 7fec1c1de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afe1f0 0x559659af1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.197+0000 7fec1c1de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afebb0 0x559659af6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.197+0000 7fec1c1de740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559659aef8b0 con 0x559659afd020 2024-03-21T11:42:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.197+0000 7fec1c1de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559659a150d0 con 0x559659afebb0 2024-03-21T11:42:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.197+0000 7fec1c1de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559659aefa30 con 0x559659afe1f0 2024-03-21T11:42:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.198+0000 7fec1c1de740 10 client.0 _ll_register_callbacks cb 0x559659a19e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.198+0000 7fec11e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afe1f0 0x559659af1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.198+0000 7fec11e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afe1f0 0x559659af1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35012/0 (socket says 172.21.15.47:35012) 2024-03-21T11:42:39.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.198+0000 7fec11e00640 1 -- 172.21.15.47:0/807333888 learned_addr learned my addr 172.21.15.47:0/807333888 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.198+0000 7fec13200640 1 --2- 172.21.15.47:0/807333888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afebb0 0x559659af6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.198+0000 7fec12800640 1 --2- 172.21.15.47:0/807333888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afd020 0x559659af0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.199+0000 7fec11e00640 1 -- 172.21.15.47:0/807333888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afebb0 msgr2=0x559659af6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.199+0000 7fec11e00640 1 --2- 172.21.15.47:0/807333888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afebb0 0x559659af6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.199+0000 7fec11e00640 1 -- 172.21.15.47:0/807333888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afd020 msgr2=0x559659af0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.199+0000 7fec11e00640 1 --2- 172.21.15.47:0/807333888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afd020 0x559659af0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.199+0000 7fec11e00640 1 -- 172.21.15.47:0/807333888 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55965976d3f0 con 0x559659afe1f0 2024-03-21T11:42:39.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.200+0000 7fec12800640 1 --2- 172.21.15.47:0/807333888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afd020 0x559659af0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:39.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.200+0000 7fec11e00640 1 --2- 172.21.15.47:0/807333888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afe1f0 0x559659af1530 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fec0002c170 tx=0x7fec00004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48587faf32ecc664 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.200+0000 7febf3e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:42:39.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.202+0000 7febf3e00640 1 -- 172.21.15.47:0/807333888 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec00005d50 con 0x559659afe1f0 2024-03-21T11:42:39.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.202+0000 7febf3e00640 1 -- 172.21.15.47:0/807333888 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fec00005230 con 0x559659afe1f0 2024-03-21T11:42:39.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.202+0000 7fec1c1de740 1 -- 172.21.15.47:0/807333888 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec0c011c40 con 0x559659afe1f0 2024-03-21T11:42:39.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.202+0000 7febf3e00640 1 -- 172.21.15.47:0/807333888 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec00005430 con 0x559659afe1f0 2024-03-21T11:42:39.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.202+0000 7fec1c1de740 10 client.31732 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:39.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.203+0000 7fec1c1de740 1 -- 172.21.15.47:0/807333888 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5596598377d0 con 0x559659afe1f0 2024-03-21T11:42:39.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.203+0000 7febf3e00640 1 -- 172.21.15.47:0/807333888 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec000055d0 con 0x559659afe1f0 2024-03-21T11:42:39.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.203+0000 7febf3e00640 1 -- 172.21.15.47:0/807333888 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fec000039c0 con 0x559659afe1f0 2024-03-21T11:42:39.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.203+0000 7fec1c1de740 10 client.31732 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:39.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.203+0000 7fec1c1de740 10 client.31732 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:42:39.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.203+0000 7fec1c1de740 20 client.31732 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:39.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.203+0000 7fec1c1de740 1 --2- 172.21.15.47:0/807333888 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x559659b1b570 0x559659b1d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.203+0000 7fec1c1de740 4 client.31732 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:39.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.203+0000 7fec1c1de740 1 -- 172.21.15.47:0/807333888 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec0c011c40 con 0x559659b1b570 2024-03-21T11:42:39.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.204+0000 7fec12800640 1 --2- 172.21.15.47:0/807333888 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x559659b1b570 0x559659b1d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.204+0000 7fec12800640 1 --2- 172.21.15.47:0/807333888 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x559659b1b570 0x559659b1d9d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9a97ccd9ddb72f22 server_cookie=b3024f416d3c323e in_seq=0 out_seq=0 2024-03-21T11:42:39.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.204+0000 7febf3e00640 10 client.31732 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:42:39.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.205+0000 7febf3e00640 1 -- 172.21.15.47:0/807333888 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fec0c011c40 con 0x559659b1b570 2024-03-21T11:42:39.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.205+0000 7febf3e00640 10 client.31732 handle_command_reply: tid=0 2024-03-21T11:42:39.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.205+0000 7fec1c1de740 1 client.31732 shutdown 2024-03-21T11:42:39.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.205+0000 7fec1c1de740 1 -- 172.21.15.47:0/807333888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afe1f0 msgr2=0x559659af1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.205+0000 7fec1c1de740 1 --2- 172.21.15.47:0/807333888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afe1f0 0x559659af1530 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fec0002c170 tx=0x7fec00004250 comp rx=0 tx=0).stop 2024-03-21T11:42:39.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.205+0000 7fec1c1de740 1 -- 172.21.15.47:0/807333888 shutdown_connections 2024-03-21T11:42:39.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.206+0000 7fec1c1de740 1 --2- 172.21.15.47:0/807333888 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x559659b1b570 0x559659b1d9d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.206+0000 7fec1c1de740 1 --2- 172.21.15.47:0/807333888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afebb0 0x559659af6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.206+0000 7fec1c1de740 1 --2- 172.21.15.47:0/807333888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afe1f0 0x559659af1530 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.206+0000 7febf3e00640 0 client.31732 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:42:39.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.206+0000 7fec1c1de740 1 --2- 172.21.15.47:0/807333888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afd020 0x559659af0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.206+0000 7fec1c1de740 1 -- 172.21.15.47:0/807333888 >> 172.21.15.47:0/807333888 conn(0x559659af8410 msgr2=0x559659aeeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.206+0000 7fec1c1de740 1 -- 172.21.15.47:0/807333888 shutdown_connections 2024-03-21T11:42:39.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.206+0000 7fec1c1de740 1 -- 172.21.15.47:0/807333888 wait complete. 2024-03-21T11:42:39.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.206+0000 7fec1c1de740 20 client.31732 trim_cache size 0 max 16384 2024-03-21T11:42:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.226+0000 7fec1c1de740 1 Processor -- start 2024-03-21T11:42:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.226+0000 7fec1c1de740 1 -- start start 2024-03-21T11:42:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.226+0000 7fec1c1de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afd020 0x559659b05100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.226+0000 7fec1c1de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afe1f0 0x559659af87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.226+0000 7fec1c1de740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afebb0 0x559659b01290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.226+0000 7fec1c1de740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf400f990 con 0x559659afebb0 2024-03-21T11:42:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.226+0000 7fec1c1de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf4011310 con 0x559659afd020 2024-03-21T11:42:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.226+0000 7fec1c1de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf400fff0 con 0x559659afe1f0 2024-03-21T11:42:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.226+0000 7fec12800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afd020 0x559659b05100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.226+0000 7fec11e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afe1f0 0x559659af87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.226+0000 7fec12800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afd020 0x559659b05100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60418/0 (socket says 172.21.15.47:60418) 2024-03-21T11:42:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.226+0000 7fec12800640 1 -- 172.21.15.47:0/2158555327 learned_addr learned my addr 172.21.15.47:0/2158555327 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.226+0000 7fec13200640 1 --2- 172.21.15.47:0/2158555327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afebb0 0x559659b01290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.227+0000 7fec11e00640 1 -- 172.21.15.47:0/2158555327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afd020 msgr2=0x559659b05100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.227+0000 7fec11e00640 1 --2- 172.21.15.47:0/2158555327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afd020 0x559659b05100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.227+0000 7fec11e00640 1 -- 172.21.15.47:0/2158555327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afebb0 msgr2=0x559659b01290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.227+0000 7fec11e00640 1 --2- 172.21.15.47:0/2158555327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afebb0 0x559659b01290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.227+0000 7fec11e00640 1 -- 172.21.15.47:0/2158555327 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559659af1880 con 0x559659afe1f0 2024-03-21T11:42:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.227+0000 7fec11e00640 1 --2- 172.21.15.47:0/2158555327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afe1f0 0x559659af87f0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fec00003d30 tx=0x7fec00001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bdd463028e5422 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.228+0000 7febfea00640 1 -- 172.21.15.47:0/2158555327 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec00005d50 con 0x559659afe1f0 2024-03-21T11:42:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.228+0000 7febfea00640 1 -- 172.21.15.47:0/2158555327 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec00058070 con 0x559659afe1f0 2024-03-21T11:42:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.228+0000 7fec1c1de740 1 -- 172.21.15.47:0/2158555327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afe1f0 msgr2=0x559659af87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.228+0000 7fec1c1de740 1 --2- 172.21.15.47:0/2158555327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afe1f0 0x559659af87f0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fec00003d30 tx=0x7fec00001e70 comp rx=0 tx=0).stop 2024-03-21T11:42:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.228+0000 7febfea00640 1 -- 172.21.15.47:0/2158555327 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0002cd90 con 0x559659afe1f0 2024-03-21T11:42:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.229+0000 7fec1c1de740 1 -- 172.21.15.47:0/2158555327 shutdown_connections 2024-03-21T11:42:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.229+0000 7fec1c1de740 1 --2- 172.21.15.47:0/2158555327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afebb0 0x559659b01290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.229+0000 7fec1c1de740 1 --2- 172.21.15.47:0/2158555327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afe1f0 0x559659af87f0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.229+0000 7fec1c1de740 1 --2- 172.21.15.47:0/2158555327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afd020 0x559659b05100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.229+0000 7fec1c1de740 1 -- 172.21.15.47:0/2158555327 >> 172.21.15.47:0/2158555327 conn(0x559659af8410 msgr2=0x559659afbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.229+0000 7fec1c1de740 1 -- 172.21.15.47:0/2158555327 shutdown_connections 2024-03-21T11:42:39.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.229+0000 7fec1c1de740 1 -- 172.21.15.47:0/2158555327 wait complete. 2024-03-21T11:42:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.229+0000 7fec1c1de740 1 -- start start 2024-03-21T11:42:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.229+0000 7fec1c1de740 1 Processor -- start 2024-03-21T11:42:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.230+0000 7fec1c1de740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afd020 0x559659b2da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.230+0000 7fec1c1de740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afe1f0 0x559659b2a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.230+0000 7fec1c1de740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afebb0 0x559659b2aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.230+0000 7fec1c1de740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf4010cb0 con 0x559659afebb0 2024-03-21T11:42:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.230+0000 7fec1c1de740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf4010650 con 0x559659afe1f0 2024-03-21T11:42:39.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.230+0000 7fec1c1de740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf400db50 con 0x559659afd020 2024-03-21T11:42:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.230+0000 7fec1c1de740 10 client.0 _ll_register_callbacks cb 0x7fec0c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.230+0000 7fec12800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afd020 0x559659b2da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.231+0000 7fec11e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afe1f0 0x559659b2a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.231+0000 7fec12800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afd020 0x559659b2da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35034/0 (socket says 172.21.15.47:35034) 2024-03-21T11:42:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.231+0000 7fec12800640 1 -- 172.21.15.47:0/47205483 learned_addr learned my addr 172.21.15.47:0/47205483 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.231+0000 7fec13200640 1 --2- 172.21.15.47:0/47205483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afebb0 0x559659b2aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.231+0000 7fec11e00640 1 -- 172.21.15.47:0/47205483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afd020 msgr2=0x559659b2da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.231+0000 7fec11e00640 1 --2- 172.21.15.47:0/47205483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afd020 0x559659b2da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.231+0000 7fec11e00640 1 -- 172.21.15.47:0/47205483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afebb0 msgr2=0x559659b2aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.231+0000 7fec11e00640 1 --2- 172.21.15.47:0/47205483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afebb0 0x559659b2aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.231+0000 7fec11e00640 1 -- 172.21.15.47:0/47205483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559659b76310 con 0x559659afe1f0 2024-03-21T11:42:39.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.231+0000 7fec13200640 1 --2- 172.21.15.47:0/47205483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afebb0 0x559659b2aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:42:39.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.232+0000 7fec11e00640 1 --2- 172.21.15.47:0/47205483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afe1f0 0x559659b2a530 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fec00004010 tx=0x7fec00003c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e09c0e80c88cb1d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.232+0000 7febf3e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.233+0000 7febf3e00640 1 -- 172.21.15.47:0/47205483 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0002cd90 con 0x559659afe1f0 2024-03-21T11:42:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.233+0000 7febf3e00640 1 -- 172.21.15.47:0/47205483 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fec00034720 con 0x559659afe1f0 2024-03-21T11:42:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.233+0000 7fec1c1de740 1 -- 172.21.15.47:0/47205483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec00058070 con 0x559659afe1f0 2024-03-21T11:42:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.233+0000 7febf3e00640 1 -- 172.21.15.47:0/47205483 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec00005d50 con 0x559659afe1f0 2024-03-21T11:42:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.233+0000 7fec1c1de740 10 client.31763 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.233+0000 7fec1c1de740 1 -- 172.21.15.47:0/47205483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559659b775c0 con 0x559659afe1f0 2024-03-21T11:42:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.234+0000 7febf3e00640 1 -- 172.21.15.47:0/47205483 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec00058070 con 0x559659afe1f0 2024-03-21T11:42:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.234+0000 7febf3e00640 1 -- 172.21.15.47:0/47205483 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fec00053a80 con 0x559659afe1f0 2024-03-21T11:42:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.234+0000 7fec1c1de740 10 client.31763 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.234+0000 7fec1c1de740 10 client.31763 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:42:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.234+0000 7fec1c1de740 20 client.31763 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.234+0000 7fec1c1de740 1 --2- 172.21.15.47:0/47205483 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x559659b77d00 0x559659b7a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.235+0000 7fec1c1de740 4 client.31763 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:42:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.235+0000 7fec1c1de740 1 -- 172.21.15.47:0/47205483 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559659a66800 con 0x559659b77d00 2024-03-21T11:42:39.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.235+0000 7fec12800640 1 --2- 172.21.15.47:0/47205483 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x559659b77d00 0x559659b7a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.236+0000 7fec12800640 1 --2- 172.21.15.47:0/47205483 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x559659b77d00 0x559659b7a120 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5c9b7b5ed514ac0d server_cookie=4f83316b9e37a3bb in_seq=0 out_seq=0 2024-03-21T11:42:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.236+0000 7febf3e00640 10 client.31763 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:42:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.236+0000 7febf3e00640 1 -- 172.21.15.47:0/47205483 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559659a66800 con 0x559659b77d00 2024-03-21T11:42:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.236+0000 7febf3e00640 10 client.31763 handle_command_reply: tid=0 2024-03-21T11:42:39.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.236+0000 7fec1c1de740 1 client.31763 shutdown 2024-03-21T11:42:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.237+0000 7fec1c1de740 1 -- 172.21.15.47:0/47205483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afe1f0 msgr2=0x559659b2a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.237+0000 7fec1c1de740 1 --2- 172.21.15.47:0/47205483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afe1f0 0x559659b2a530 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fec00004010 tx=0x7fec00003c60 comp rx=0 tx=0).stop 2024-03-21T11:42:39.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.237+0000 7fec1c1de740 1 -- 172.21.15.47:0/47205483 shutdown_connections 2024-03-21T11:42:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.237+0000 7fec1c1de740 1 --2- 172.21.15.47:0/47205483 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x559659b77d00 0x559659b7a120 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.237+0000 7fec1c1de740 1 --2- 172.21.15.47:0/47205483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559659afebb0 0x559659b2aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.237+0000 7fec1c1de740 1 --2- 172.21.15.47:0/47205483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559659afe1f0 0x559659b2a530 secure :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fec00004010 tx=0x7fec00003c60 comp rx=0 tx=0).stop 2024-03-21T11:42:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.237+0000 7fec1c1de740 1 --2- 172.21.15.47:0/47205483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559659afd020 0x559659b2da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.237+0000 7fec1c1de740 1 -- 172.21.15.47:0/47205483 >> 172.21.15.47:0/47205483 conn(0x559659af8410 msgr2=0x559659afb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.237+0000 7febf3e00640 0 client.31763 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:42:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.237+0000 7fec1c1de740 1 -- 172.21.15.47:0/47205483 shutdown_connections 2024-03-21T11:42:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.238+0000 7fec1c1de740 1 -- 172.21.15.47:0/47205483 wait complete. 2024-03-21T11:42:39.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.238+0000 7fec1c1de740 20 client.31763 trim_cache size 0 max 16384 2024-03-21T11:42:39.240 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:42:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.240+0000 7fec19400640 1 -- 172.21.15.47:0/3528843398 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7febf80a3070 msgr2=0x7febf80a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.240+0000 7fec19400640 1 --2- 172.21.15.47:0/3528843398 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7febf80a3070 0x7febf80a5530 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fec0c01e000 tx=0x7fec0c021000 comp rx=0 tx=0).stop 2024-03-21T11:42:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.240+0000 7fec19400640 1 -- 172.21.15.47:0/3528843398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec140fff80 msgr2=0x7fec14199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.240+0000 7fec19400640 1 --2- 172.21.15.47:0/3528843398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec140fff80 0x7fec14199bf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fec0400e9f0 tx=0x7fec0400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.240+0000 7fec19400640 1 -- 172.21.15.47:0/3528843398 shutdown_connections 2024-03-21T11:42:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.240+0000 7fec19400640 1 --2- 172.21.15.47:0/3528843398 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7febf80a3070 0x7febf80a5530 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.240+0000 7fec19400640 1 --2- 172.21.15.47:0/3528843398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec14101280 0x7fec141a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.240+0000 7fec19400640 1 --2- 172.21.15.47:0/3528843398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec141008e0 0x7fec1419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.240+0000 7fec19400640 1 --2- 172.21.15.47:0/3528843398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec140fff80 0x7fec14199bf0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.240+0000 7fec19400640 1 -- 172.21.15.47:0/3528843398 >> 172.21.15.47:0/3528843398 conn(0x7fec140fbb40 msgr2=0x7fec140fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.240+0000 7fec19400640 1 -- 172.21.15.47:0/3528843398 shutdown_connections 2024-03-21T11:42:39.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.240+0000 7fec19400640 1 -- 172.21.15.47:0/3528843398 wait complete. 2024-03-21T11:42:39.262 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:42:39.263 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:42:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.540+0000 7f92b3e00640 1 -- 172.21.15.47:0/1160687658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92b4104990 msgr2=0x7f92b4104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.540+0000 7f92b3e00640 1 --2- 172.21.15.47:0/1160687658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92b4104990 0x7f92b4104e10 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009900 tx=0x7f92a002bc40 comp rx=0 tx=0).stop 2024-03-21T11:42:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.540+0000 7f92b3e00640 1 -- 172.21.15.47:0/1160687658 shutdown_connections 2024-03-21T11:42:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.540+0000 7f92b3e00640 1 --2- 172.21.15.47:0/1160687658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b4105350 0x7f92b410bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.540+0000 7f92b3e00640 1 --2- 172.21.15.47:0/1160687658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92b4104990 0x7f92b4104e10 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.540+0000 7f92b3e00640 1 --2- 172.21.15.47:0/1160687658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92b41037a0 0x7f92b4103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.540+0000 7f92b3e00640 1 -- 172.21.15.47:0/1160687658 >> 172.21.15.47:0/1160687658 conn(0x7f92b40fed10 msgr2=0x7f92b4101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.541+0000 7f92b3e00640 1 -- 172.21.15.47:0/1160687658 shutdown_connections 2024-03-21T11:42:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.541+0000 7f92b3e00640 1 -- 172.21.15.47:0/1160687658 wait complete. 2024-03-21T11:42:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.542+0000 7f92b3e00640 1 Processor -- start 2024-03-21T11:42:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.542+0000 7f92b3e00640 1 -- start start 2024-03-21T11:42:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.542+0000 7f92b3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b41037a0 0x7f92b4199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.542+0000 7f92b3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92b4104990 0x7f92b4199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.542+0000 7f92b3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92b4105350 0x7f92b41a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.542+0000 7f92b3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b410f450 con 0x7f92b4104990 2024-03-21T11:42:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.542+0000 7f92b3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b410f2d0 con 0x7f92b41037a0 2024-03-21T11:42:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.542+0000 7f92b3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b410f5d0 con 0x7f92b4105350 2024-03-21T11:42:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.542+0000 7f92b2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b41037a0 0x7f92b4199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.542+0000 7f92b2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92b4104990 0x7f92b4199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.542+0000 7f92b2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b41037a0 0x7f92b4199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60444/0 (socket says 172.21.15.47:60444) 2024-03-21T11:42:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.542+0000 7f92b2a00640 1 -- 172.21.15.47:0/3176622224 learned_addr learned my addr 172.21.15.47:0/3176622224 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.543+0000 7f92b3400640 1 --2- 172.21.15.47:0/3176622224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92b4105350 0x7f92b41a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.543+0000 7f92b2a00640 1 -- 172.21.15.47:0/3176622224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92b4105350 msgr2=0x7f92b41a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.543+0000 7f92b2a00640 1 --2- 172.21.15.47:0/3176622224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92b4105350 0x7f92b41a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.543+0000 7f92b2a00640 1 -- 172.21.15.47:0/3176622224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92b4104990 msgr2=0x7f92b4199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.543+0000 7f92b2a00640 1 --2- 172.21.15.47:0/3176622224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92b4104990 0x7f92b4199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.543+0000 7f92b2a00640 1 -- 172.21.15.47:0/3176622224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b41a16b0 con 0x7f92b41037a0 2024-03-21T11:42:39.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.543+0000 7f92b2a00640 1 --2- 172.21.15.47:0/3176622224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b41037a0 0x7f92b4199a90 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f92a400e9c0 tx=0x7f92a400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.543+0000 7f92ab400640 1 -- 172.21.15.47:0/3176622224 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a400cd80 con 0x7f92b41037a0 2024-03-21T11:42:39.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.544+0000 7f92b3e00640 1 -- 172.21.15.47:0/3176622224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b41a19a0 con 0x7f92b41037a0 2024-03-21T11:42:39.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.544+0000 7f92b3e00640 1 -- 172.21.15.47:0/3176622224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b41a1f10 con 0x7f92b41037a0 2024-03-21T11:42:39.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.544+0000 7f92ab400640 1 -- 172.21.15.47:0/3176622224 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92a4010910 con 0x7f92b41037a0 2024-03-21T11:42:39.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.544+0000 7f92ab400640 1 -- 172.21.15.47:0/3176622224 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4018950 con 0x7f92b41037a0 2024-03-21T11:42:39.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.547+0000 7f92bc202740 1 Processor -- start 2024-03-21T11:42:39.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.547+0000 7f92bc202740 1 -- start start 2024-03-21T11:42:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.547+0000 7f92ab400640 1 -- 172.21.15.47:0/3176622224 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f92a401f020 con 0x7f92b41037a0 2024-03-21T11:42:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.547+0000 7f92bc202740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc771e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.547+0000 7f92bc202740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc772bc0 0x5619dc773040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.547+0000 7f92bc202740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 0x5619dc779e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.547+0000 7f92bc202740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619dc6d62c0 con 0x5619dc772bc0 2024-03-21T11:42:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.547+0000 7f92bc202740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619dc67ec90 con 0x5619dc773580 2024-03-21T11:42:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.547+0000 7f92bc202740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619dc6d2360 con 0x5619dc7719f0 2024-03-21T11:42:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.548+0000 7f92b3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc771e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.548+0000 7f92b2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 0x5619dc779e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.548+0000 7f92b3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc771e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35054/0 (socket says 172.21.15.47:35054) 2024-03-21T11:42:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.548+0000 7f92b3400640 1 -- 172.21.15.47:0/4089231947 learned_addr learned my addr 172.21.15.47:0/4089231947 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.548+0000 7f92b2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc772bc0 0x5619dc773040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.548+0000 7f92b3400640 1 -- 172.21.15.47:0/4089231947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 msgr2=0x5619dc779e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.548+0000 7f92b3400640 1 --2- 172.21.15.47:0/4089231947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 0x5619dc779e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.548+0000 7f92b3400640 1 -- 172.21.15.47:0/4089231947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc772bc0 msgr2=0x5619dc773040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.548+0000 7f92b3400640 1 --2- 172.21.15.47:0/4089231947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc772bc0 0x5619dc773040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.548+0000 7f92b3400640 1 -- 172.21.15.47:0/4089231947 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619dc6a3080 con 0x5619dc7719f0 2024-03-21T11:42:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.548+0000 7f92b2000640 1 --2- 172.21.15.47:0/4089231947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 0x5619dc779e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.549+0000 7f92b3400640 1 --2- 172.21.15.47:0/4089231947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc771e10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f92ac011440 tx=0x7f92ac011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dee9c28a0e333be server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.549+0000 7f929f400640 1 -- 172.21.15.47:0/4089231947 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac003c30 con 0x5619dc7719f0 2024-03-21T11:42:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.549+0000 7f929f400640 1 -- 172.21.15.47:0/4089231947 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92ac003dd0 con 0x5619dc7719f0 2024-03-21T11:42:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.549+0000 7f929f400640 1 -- 172.21.15.47:0/4089231947 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00fb50 con 0x5619dc7719f0 2024-03-21T11:42:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.549+0000 7f92ab400640 1 --2- 172.21.15.47:0/3176622224 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92980a2fa0 0x7f92980a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.549+0000 7f92ab400640 1 -- 172.21.15.47:0/3176622224 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f92a40cf750 con 0x7f92b41037a0 2024-03-21T11:42:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.549+0000 7f92b2a00640 1 --2- 172.21.15.47:0/3176622224 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92980a2fa0 0x7f92980a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.550+0000 7f92bc202740 1 -- 172.21.15.47:0/4089231947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 msgr2=0x5619dc771e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.550+0000 7f92bc202740 1 --2- 172.21.15.47:0/4089231947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc771e10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f92ac011440 tx=0x7f92ac011900 comp rx=0 tx=0).stop 2024-03-21T11:42:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.550+0000 7f92bc202740 1 -- 172.21.15.47:0/4089231947 shutdown_connections 2024-03-21T11:42:39.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.550+0000 7f92bc202740 1 --2- 172.21.15.47:0/4089231947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 0x5619dc779e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.550+0000 7f92bc202740 1 --2- 172.21.15.47:0/4089231947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc772bc0 0x5619dc773040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.550+0000 7f92bc202740 1 --2- 172.21.15.47:0/4089231947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc771e10 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.550+0000 7f92bc202740 1 -- 172.21.15.47:0/4089231947 >> 172.21.15.47:0/4089231947 conn(0x5619dc76cde0 msgr2=0x5619dc76d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.550+0000 7f92bc202740 1 -- 172.21.15.47:0/4089231947 shutdown_connections 2024-03-21T11:42:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.550+0000 7f92bc202740 1 -- 172.21.15.47:0/4089231947 wait complete. 2024-03-21T11:42:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.550+0000 7f92b2a00640 1 --2- 172.21.15.47:0/3176622224 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92980a2fa0 0x7f92980a5460 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f92a4098460 tx=0x7f92a40d5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.550+0000 7f92bc202740 1 -- start start 2024-03-21T11:42:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.551+0000 7f92bc202740 1 Processor -- start 2024-03-21T11:42:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.551+0000 7f92bc202740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc77bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.551+0000 7f92bc202740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc772bc0 0x5619dc77c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.551+0000 7f92bc202740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 0x5619dc7831e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.551+0000 7f92bc202740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619dc77a590 con 0x5619dc772bc0 2024-03-21T11:42:39.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.551+0000 7f92bc202740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619dc6920d0 con 0x5619dc773580 2024-03-21T11:42:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.551+0000 7f92bc202740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619dc77a710 con 0x5619dc7719f0 2024-03-21T11:42:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.551+0000 7f92bc202740 10 client.0 _ll_register_callbacks cb 0x5619dc696e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.551+0000 7f92b2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc772bc0 0x5619dc77c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.552+0000 7f92b2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc772bc0 0x5619dc77c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43592/0 (socket says 172.21.15.47:43592) 2024-03-21T11:42:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.552+0000 7f92b2000640 1 -- 172.21.15.47:0/3337019472 learned_addr learned my addr 172.21.15.47:0/3337019472 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.552+0000 7f92b3400640 1 --2- 172.21.15.47:0/3337019472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc77bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.552+0000 7f92b3400640 1 --2- 172.21.15.47:0/3337019472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 0x5619dc7831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.552+0000 7f92b2000640 1 -- 172.21.15.47:0/3337019472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 msgr2=0x5619dc77bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.552+0000 7f92b2000640 1 --2- 172.21.15.47:0/3337019472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc77bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.552+0000 7f92b2000640 1 -- 172.21.15.47:0/3337019472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 msgr2=0x5619dc7831e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.552+0000 7f92b2000640 1 --2- 172.21.15.47:0/3337019472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 0x5619dc7831e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.552+0000 7f92b2000640 1 -- 172.21.15.47:0/3337019472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619dc3ea3f0 con 0x5619dc772bc0 2024-03-21T11:42:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.553+0000 7f92b2000640 1 --2- 172.21.15.47:0/3337019472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc772bc0 0x5619dc77c1a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f92a0006010 tx=0x7f92a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181c991e9a9f38b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.553+0000 7f9293e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:42:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.553+0000 7f9293e00640 1 -- 172.21.15.47:0/3337019472 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a0005d50 con 0x5619dc772bc0 2024-03-21T11:42:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.553+0000 7f9293e00640 1 -- 172.21.15.47:0/3337019472 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f92a00046c0 con 0x5619dc772bc0 2024-03-21T11:42:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.553+0000 7f92bc202740 1 -- 172.21.15.47:0/3337019472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92ac003dd0 con 0x5619dc772bc0 2024-03-21T11:42:39.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.553+0000 7f9293e00640 1 -- 172.21.15.47:0/3337019472 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92a00048c0 con 0x5619dc772bc0 2024-03-21T11:42:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.553+0000 7f92bc202740 10 client.21165 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.553+0000 7f92bc202740 1 -- 172.21.15.47:0/3337019472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5619dc4b47d0 con 0x5619dc772bc0 2024-03-21T11:42:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.554+0000 7f9293e00640 1 -- 172.21.15.47:0/3337019472 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92ac003dd0 con 0x5619dc772bc0 2024-03-21T11:42:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.554+0000 7f9293e00640 1 -- 172.21.15.47:0/3337019472 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f92a00035f0 con 0x5619dc772bc0 2024-03-21T11:42:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.554+0000 7f92bc202740 10 client.21165 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.554+0000 7f92bc202740 10 client.21165 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:42:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.554+0000 7f92bc202740 20 client.21165 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.554+0000 7f92bc202740 1 --2- 172.21.15.47:0/3337019472 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5619dc787800 0x5619dc789c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.554+0000 7f92bc202740 4 client.21165 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.554+0000 7f92bc202740 1 -- 172.21.15.47:0/3337019472 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619dc6e3800 con 0x5619dc787800 2024-03-21T11:42:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.554+0000 7f92b3400640 1 --2- 172.21.15.47:0/3337019472 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5619dc787800 0x5619dc789c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.556+0000 7f92b3400640 1 --2- 172.21.15.47:0/3337019472 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5619dc787800 0x5619dc789c20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e700eee2408bd51 server_cookie=96a3b89252c8b7e9 in_seq=0 out_seq=0 2024-03-21T11:42:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.556+0000 7f9293e00640 10 client.21165 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:42:39.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.560+0000 7f9293e00640 1 -- 172.21.15.47:0/3337019472 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5619dc6e3800 con 0x5619dc787800 2024-03-21T11:42:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.560+0000 7f9293e00640 10 client.21165 handle_command_reply: tid=0 2024-03-21T11:42:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.560+0000 7f92bc202740 1 client.21165 shutdown 2024-03-21T11:42:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.560+0000 7f92bc202740 1 -- 172.21.15.47:0/3337019472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc772bc0 msgr2=0x5619dc77c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.560+0000 7f92bc202740 1 --2- 172.21.15.47:0/3337019472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc772bc0 0x5619dc77c1a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f92a0006010 tx=0x7f92a0004270 comp rx=0 tx=0).stop 2024-03-21T11:42:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.560+0000 7f92bc202740 1 -- 172.21.15.47:0/3337019472 shutdown_connections 2024-03-21T11:42:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.560+0000 7f92bc202740 1 --2- 172.21.15.47:0/3337019472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 0x5619dc7831e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.560+0000 7f92bc202740 1 --2- 172.21.15.47:0/3337019472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc772bc0 0x5619dc77c1a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.560+0000 7f92bc202740 1 --2- 172.21.15.47:0/3337019472 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5619dc787800 0x5619dc789c20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.560+0000 7f92bc202740 1 --2- 172.21.15.47:0/3337019472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc77bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.560+0000 7f9293e00640 0 client.21165 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:42:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.560+0000 7f92bc202740 1 -- 172.21.15.47:0/3337019472 >> 172.21.15.47:0/3337019472 conn(0x5619dc76cde0 msgr2=0x5619dc76c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.561+0000 7f92bc202740 1 -- 172.21.15.47:0/3337019472 shutdown_connections 2024-03-21T11:42:39.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.561+0000 7f92bc202740 1 -- 172.21.15.47:0/3337019472 wait complete. 2024-03-21T11:42:39.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.561+0000 7f92bc202740 20 client.21165 trim_cache size 0 max 16384 2024-03-21T11:42:39.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.579+0000 7f92bc202740 1 Processor -- start 2024-03-21T11:42:39.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.579+0000 7f92bc202740 1 -- start start 2024-03-21T11:42:39.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.579+0000 7f92bc202740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc770750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.580+0000 7f92bc202740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 0x5619dc7831a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.580+0000 7f92bc202740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc775df0 0x5619dc776270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.580+0000 7f92bc202740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929400f9a0 con 0x5619dc775df0 2024-03-21T11:42:39.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.580+0000 7f92bc202740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9294011320 con 0x5619dc773580 2024-03-21T11:42:39.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.580+0000 7f92bc202740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9294010000 con 0x5619dc7719f0 2024-03-21T11:42:39.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.580+0000 7f92b3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc770750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.580+0000 7f92b3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc770750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35070/0 (socket says 172.21.15.47:35070) 2024-03-21T11:42:39.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.580+0000 7f92b3400640 1 -- 172.21.15.47:0/621134680 learned_addr learned my addr 172.21.15.47:0/621134680 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.580+0000 7f92b3400640 1 --2- 172.21.15.47:0/621134680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc775df0 0x5619dc776270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.580+0000 7f92b2000640 1 --2- 172.21.15.47:0/621134680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 0x5619dc7831a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.581+0000 7f92b3400640 1 -- 172.21.15.47:0/621134680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 msgr2=0x5619dc7831a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.581+0000 7f92b3400640 1 --2- 172.21.15.47:0/621134680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 0x5619dc7831a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.581+0000 7f92b3400640 1 -- 172.21.15.47:0/621134680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc775df0 msgr2=0x5619dc776270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.581+0000 7f92b3400640 1 --2- 172.21.15.47:0/621134680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc775df0 0x5619dc776270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.581+0000 7f92b3400640 1 -- 172.21.15.47:0/621134680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619dc77bc60 con 0x5619dc7719f0 2024-03-21T11:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.581+0000 7f92b2000640 1 --2- 172.21.15.47:0/621134680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 0x5619dc7831a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.581+0000 7f92b3400640 1 --2- 172.21.15.47:0/621134680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc770750 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009fa0 tx=0x7f92ac009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=272e6e5815e7301 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.582+0000 7f929ea00640 1 -- 172.21.15.47:0/621134680 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac004ac0 con 0x5619dc7719f0 2024-03-21T11:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.582+0000 7f929ea00640 1 -- 172.21.15.47:0/621134680 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92ac004c60 con 0x5619dc7719f0 2024-03-21T11:42:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.582+0000 7f929ea00640 1 -- 172.21.15.47:0/621134680 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00f820 con 0x5619dc7719f0 2024-03-21T11:42:39.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.583+0000 7f92bc202740 1 -- 172.21.15.47:0/621134680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 msgr2=0x5619dc770750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.583+0000 7f92bc202740 1 --2- 172.21.15.47:0/621134680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc770750 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009fa0 tx=0x7f92ac009500 comp rx=0 tx=0).stop 2024-03-21T11:42:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.583+0000 7f92bc202740 1 -- 172.21.15.47:0/621134680 shutdown_connections 2024-03-21T11:42:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.583+0000 7f92bc202740 1 --2- 172.21.15.47:0/621134680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc775df0 0x5619dc776270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.583+0000 7f92bc202740 1 --2- 172.21.15.47:0/621134680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc773580 0x5619dc7831a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.583+0000 7f92bc202740 1 --2- 172.21.15.47:0/621134680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc7719f0 0x5619dc770750 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.583+0000 7f92bc202740 1 -- 172.21.15.47:0/621134680 >> 172.21.15.47:0/621134680 conn(0x5619dc76cde0 msgr2=0x5619dc770040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.583+0000 7f92bc202740 1 -- 172.21.15.47:0/621134680 shutdown_connections 2024-03-21T11:42:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.583+0000 7f92bc202740 1 -- 172.21.15.47:0/621134680 wait complete. 2024-03-21T11:42:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.583+0000 7f92bc202740 1 -- start start 2024-03-21T11:42:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.584+0000 7f92bc202740 1 Processor -- start 2024-03-21T11:42:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.584+0000 7f92bc202740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc7719f0 0x5619dc782c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.584+0000 7f92bc202740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc773580 0x5619dc783160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.584+0000 7f92bc202740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc775df0 0x5619dc781230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.584+0000 7f92bc202740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9294010cc0 con 0x5619dc773580 2024-03-21T11:42:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.584+0000 7f92bc202740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9294010660 con 0x5619dc7719f0 2024-03-21T11:42:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.584+0000 7f92bc202740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929400db60 con 0x5619dc775df0 2024-03-21T11:42:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.584+0000 7f92bc202740 10 client.0 _ll_register_callbacks cb 0x7f92a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.584+0000 7f92b3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc775df0 0x5619dc781230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.585+0000 7f92b3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc775df0 0x5619dc781230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35082/0 (socket says 172.21.15.47:35082) 2024-03-21T11:42:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.585+0000 7f92b3400640 1 -- 172.21.15.47:0/704134841 learned_addr learned my addr 172.21.15.47:0/704134841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.585+0000 7f92b2000640 1 --2- 172.21.15.47:0/704134841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc773580 0x5619dc783160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.585+0000 7f92b3400640 1 --2- 172.21.15.47:0/704134841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc7719f0 0x5619dc782c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.585+0000 7f92b3400640 1 -- 172.21.15.47:0/704134841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc7719f0 msgr2=0x5619dc782c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.585+0000 7f92b3400640 1 --2- 172.21.15.47:0/704134841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc7719f0 0x5619dc782c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.585+0000 7f92b3400640 1 -- 172.21.15.47:0/704134841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc773580 msgr2=0x5619dc783160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.585+0000 7f92b3400640 1 --2- 172.21.15.47:0/704134841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc773580 0x5619dc783160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.585+0000 7f92b3400640 1 -- 172.21.15.47:0/704134841 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619dc781a60 con 0x5619dc775df0 2024-03-21T11:42:39.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.585+0000 7f92b2000640 1 --2- 172.21.15.47:0/704134841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc773580 0x5619dc783160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:39.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.586+0000 7f92b3400640 1 --2- 172.21.15.47:0/704134841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc775df0 0x5619dc781230 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f92ac011440 tx=0x7f92ac02a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfcd011d98d822ba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.586+0000 7f9293e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:42:39.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.586+0000 7f9293e00640 1 -- 172.21.15.47:0/704134841 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac004a40 con 0x5619dc775df0 2024-03-21T11:42:39.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.586+0000 7f9293e00640 1 -- 172.21.15.47:0/704134841 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f92ac004050 con 0x5619dc775df0 2024-03-21T11:42:39.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.586+0000 7f92bc202740 1 -- 172.21.15.47:0/704134841 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92ac004c60 con 0x5619dc775df0 2024-03-21T11:42:39.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.587+0000 7f9293e00640 1 -- 172.21.15.47:0/704134841 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92ac004250 con 0x5619dc775df0 2024-03-21T11:42:39.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.587+0000 7f92bc202740 10 client.31756 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:39.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.587+0000 7f92bc202740 1 -- 172.21.15.47:0/704134841 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5619dc7e3bb0 con 0x5619dc775df0 2024-03-21T11:42:39.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.587+0000 7f9293e00640 1 -- 172.21.15.47:0/704134841 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92ac004c60 con 0x5619dc775df0 2024-03-21T11:42:39.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.587+0000 7f9293e00640 1 -- 172.21.15.47:0/704134841 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f92ac025040 con 0x5619dc775df0 2024-03-21T11:42:39.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.587+0000 7f92bc202740 10 client.31756 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:39.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.587+0000 7f92bc202740 10 client.31756 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:42:39.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.587+0000 7f92bc202740 20 client.31756 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:39.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.587+0000 7f92bc202740 1 --2- 172.21.15.47:0/704134841 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5619dc7e4320 0x5619dc7e6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.587+0000 7f92bc202740 4 client.31756 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:42:39.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.587+0000 7f92bc202740 1 -- 172.21.15.47:0/704134841 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5619dc6ddb50 con 0x5619dc7e4320 2024-03-21T11:42:39.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.588+0000 7f92b2000640 1 --2- 172.21.15.47:0/704134841 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5619dc7e4320 0x5619dc7e6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.589+0000 7f92b2000640 1 --2- 172.21.15.47:0/704134841 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5619dc7e4320 0x5619dc7e6740 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30bd924318a545a9 server_cookie=d5d9330df5ec57e4 in_seq=0 out_seq=0 2024-03-21T11:42:39.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.589+0000 7f9293e00640 10 client.31756 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:42:39.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.590+0000 7f9293e00640 1 -- 172.21.15.47:0/704134841 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5619dc6ddb50 con 0x5619dc7e4320 2024-03-21T11:42:39.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.590+0000 7f9293e00640 10 client.31756 handle_command_reply: tid=0 2024-03-21T11:42:39.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.590+0000 7f92bc202740 1 client.31756 shutdown 2024-03-21T11:42:39.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.590+0000 7f92bc202740 1 -- 172.21.15.47:0/704134841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc775df0 msgr2=0x5619dc781230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.590+0000 7f92bc202740 1 --2- 172.21.15.47:0/704134841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc775df0 0x5619dc781230 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f92ac011440 tx=0x7f92ac02a040 comp rx=0 tx=0).stop 2024-03-21T11:42:39.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.591+0000 7f92bc202740 1 -- 172.21.15.47:0/704134841 shutdown_connections 2024-03-21T11:42:39.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.591+0000 7f92bc202740 1 --2- 172.21.15.47:0/704134841 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5619dc7e4320 0x5619dc7e6740 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.591+0000 7f92bc202740 1 --2- 172.21.15.47:0/704134841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619dc775df0 0x5619dc781230 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.591+0000 7f92bc202740 1 --2- 172.21.15.47:0/704134841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5619dc773580 0x5619dc783160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.591+0000 7f92bc202740 1 --2- 172.21.15.47:0/704134841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619dc7719f0 0x5619dc782c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.591+0000 7f92bc202740 1 -- 172.21.15.47:0/704134841 >> 172.21.15.47:0/704134841 conn(0x5619dc76cde0 msgr2=0x5619dc770040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.591+0000 7f9293e00640 0 client.31756 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:42:39.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.591+0000 7f92bc202740 1 -- 172.21.15.47:0/704134841 shutdown_connections 2024-03-21T11:42:39.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.591+0000 7f92bc202740 1 -- 172.21.15.47:0/704134841 wait complete. 2024-03-21T11:42:39.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.591+0000 7f92bc202740 20 client.31756 trim_cache size 0 max 16384 2024-03-21T11:42:39.593 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:42:39.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.592+0000 7f92b3e00640 1 -- 172.21.15.47:0/3176622224 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92980a2fa0 msgr2=0x7f92980a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.592+0000 7f92b3e00640 1 --2- 172.21.15.47:0/3176622224 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92980a2fa0 0x7f92980a5460 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f92a4098460 tx=0x7f92a40d5530 comp rx=0 tx=0).stop 2024-03-21T11:42:39.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.592+0000 7f92b3e00640 1 -- 172.21.15.47:0/3176622224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b41037a0 msgr2=0x7f92b4199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.592+0000 7f92b3e00640 1 --2- 172.21.15.47:0/3176622224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b41037a0 0x7f92b4199a90 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f92a400e9c0 tx=0x7f92a400edc0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.592+0000 7f92b3e00640 1 -- 172.21.15.47:0/3176622224 shutdown_connections 2024-03-21T11:42:39.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.592+0000 7f92b3e00640 1 --2- 172.21.15.47:0/3176622224 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92980a2fa0 0x7f92980a5460 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.592+0000 7f92b3e00640 1 --2- 172.21.15.47:0/3176622224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92b4105350 0x7f92b41a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.592+0000 7f92b3e00640 1 --2- 172.21.15.47:0/3176622224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92b4104990 0x7f92b4199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.592+0000 7f92b3e00640 1 --2- 172.21.15.47:0/3176622224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92b41037a0 0x7f92b4199a90 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.592+0000 7f92b3e00640 1 -- 172.21.15.47:0/3176622224 >> 172.21.15.47:0/3176622224 conn(0x7f92b40fed10 msgr2=0x7f92b4107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.592+0000 7f92b3e00640 1 -- 172.21.15.47:0/3176622224 shutdown_connections 2024-03-21T11:42:39.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.592+0000 7f92b3e00640 1 -- 172.21.15.47:0/3176622224 wait complete. 2024-03-21T11:42:39.611 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:42:39.611 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:42:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.825+0000 7fb9e3200640 1 -- 172.21.15.47:0/430649494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9e4107260 msgr2=0x7fb9e41096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.825+0000 7fb9e3200640 1 --2- 172.21.15.47:0/430649494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9e4107260 0x7fb9e41096b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009be0 tx=0x7fb9dc02bef0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.825+0000 7fb9e3200640 1 -- 172.21.15.47:0/430649494 shutdown_connections 2024-03-21T11:42:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.825+0000 7fb9e3200640 1 --2- 172.21.15.47:0/430649494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9e4107260 0x7fb9e41096b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.825+0000 7fb9e3200640 1 --2- 172.21.15.47:0/430649494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e4104930 0x7fb9e4106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.825+0000 7fb9e3200640 1 --2- 172.21.15.47:0/430649494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9e4102000 0x7fb9e41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.825+0000 7fb9e3200640 1 -- 172.21.15.47:0/430649494 >> 172.21.15.47:0/430649494 conn(0x7fb9e40fba10 msgr2=0x7fb9e40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.825+0000 7fb9e3200640 1 -- 172.21.15.47:0/430649494 shutdown_connections 2024-03-21T11:42:39.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.825+0000 7fb9e3200640 1 -- 172.21.15.47:0/430649494 wait complete. 2024-03-21T11:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.826+0000 7fb9e3200640 1 Processor -- start 2024-03-21T11:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.826+0000 7fb9e3200640 1 -- start start 2024-03-21T11:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e4102000 0x7fb9e4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9e4104930 0x7fb9e4199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9e4107260 0x7fb9e41a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e410d260 con 0x7fb9e4107260 2024-03-21T11:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e410d0e0 con 0x7fb9e4102000 2024-03-21T11:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e410d3e0 con 0x7fb9e4104930 2024-03-21T11:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e4102000 0x7fb9e4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9e4104930 0x7fb9e4199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e4102000 0x7fb9e4199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60500/0 (socket says 172.21.15.47:60500) 2024-03-21T11:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e1e00640 1 -- 172.21.15.47:0/4008385890 learned_addr learned my addr 172.21.15.47:0/4008385890 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e2800640 1 --2- 172.21.15.47:0/4008385890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9e4107260 0x7fb9e41a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e1e00640 1 -- 172.21.15.47:0/4008385890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9e4104930 msgr2=0x7fb9e4199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e1e00640 1 --2- 172.21.15.47:0/4008385890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9e4104930 0x7fb9e4199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e1e00640 1 -- 172.21.15.47:0/4008385890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9e4107260 msgr2=0x7fb9e41a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e1e00640 1 --2- 172.21.15.47:0/4008385890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9e4107260 0x7fb9e41a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.827+0000 7fb9e1e00640 1 -- 172.21.15.47:0/4008385890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e41a1650 con 0x7fb9e4102000 2024-03-21T11:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.828+0000 7fb9e1e00640 1 --2- 172.21.15.47:0/4008385890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e4102000 0x7fb9e4199a30 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400e9f0 tx=0x7fb9d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.828+0000 7fb9daa00640 1 -- 172.21.15.47:0/4008385890 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d400ccc0 con 0x7fb9e4102000 2024-03-21T11:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.828+0000 7fb9e3200640 1 -- 172.21.15.47:0/4008385890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e41a1940 con 0x7fb9e4102000 2024-03-21T11:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.828+0000 7fb9daa00640 1 -- 172.21.15.47:0/4008385890 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9d4005500 con 0x7fb9e4102000 2024-03-21T11:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.828+0000 7fb9daa00640 1 -- 172.21.15.47:0/4008385890 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4010690 con 0x7fb9e4102000 2024-03-21T11:42:39.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.828+0000 7fb9e3200640 1 -- 172.21.15.47:0/4008385890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e41a1e90 con 0x7fb9e4102000 2024-03-21T11:42:39.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.831+0000 7fb9eb601740 1 Processor -- start 2024-03-21T11:42:39.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.831+0000 7fb9eb601740 1 -- start start 2024-03-21T11:42:39.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9daa00640 1 -- 172.21.15.47:0/4008385890 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb9d40046c0 con 0x7fb9e4102000 2024-03-21T11:42:39.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9eb601740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2ce9f0 0x5577db2cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9eb601740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2cfbc0 0x5577db2d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9eb601740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2d0580 0x5577db2d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9eb601740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577db2332c0 con 0x5577db2d0580 2024-03-21T11:42:39.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9eb601740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577db1dbc90 con 0x5577db2cfbc0 2024-03-21T11:42:39.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9eb601740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577db22f360 con 0x5577db2ce9f0 2024-03-21T11:42:39.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9e2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2ce9f0 0x5577db2cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9e2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2ce9f0 0x5577db2cee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35098/0 (socket says 172.21.15.47:35098) 2024-03-21T11:42:39.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9e2800640 1 -- 172.21.15.47:0/4137638383 learned_addr learned my addr 172.21.15.47:0/4137638383 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9e1e00640 1 --2- 172.21.15.47:0/4137638383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2cfbc0 0x5577db2d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9e1400640 1 --2- 172.21.15.47:0/4137638383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2d0580 0x5577db2d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9e2800640 1 -- 172.21.15.47:0/4137638383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2cfbc0 msgr2=0x5577db2d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9e2800640 1 --2- 172.21.15.47:0/4137638383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2cfbc0 0x5577db2d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9e2800640 1 -- 172.21.15.47:0/4137638383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2d0580 msgr2=0x5577db2d6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9e2800640 1 --2- 172.21.15.47:0/4137638383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2d0580 0x5577db2d6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.832+0000 7fb9e2800640 1 -- 172.21.15.47:0/4137638383 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577db200080 con 0x5577db2ce9f0 2024-03-21T11:42:39.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.833+0000 7fb9e2800640 1 --2- 172.21.15.47:0/4137638383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2ce9f0 0x5577db2cee10 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009be0 tx=0x7fb9dc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cfd7d77387adad8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.833+0000 7fb9cf000640 1 -- 172.21.15.47:0/4137638383 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc043070 con 0x5577db2ce9f0 2024-03-21T11:42:39.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.833+0000 7fb9cf000640 1 -- 172.21.15.47:0/4137638383 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9dc033bc0 con 0x5577db2ce9f0 2024-03-21T11:42:39.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.833+0000 7fb9cf000640 1 -- 172.21.15.47:0/4137638383 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc02caf0 con 0x5577db2ce9f0 2024-03-21T11:42:39.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.834+0000 7fb9daa00640 1 --2- 172.21.15.47:0/4008385890 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9c80a2f20 0x7fb9c80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.834+0000 7fb9daa00640 1 -- 172.21.15.47:0/4008385890 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb9d40cfa90 con 0x7fb9e4102000 2024-03-21T11:42:39.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.834+0000 7fb9eb601740 1 -- 172.21.15.47:0/4137638383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2ce9f0 msgr2=0x5577db2cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.834+0000 7fb9eb601740 1 --2- 172.21.15.47:0/4137638383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2ce9f0 0x5577db2cee10 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009be0 tx=0x7fb9dc009310 comp rx=0 tx=0).stop 2024-03-21T11:42:39.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.834+0000 7fb9e1e00640 1 --2- 172.21.15.47:0/4008385890 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9c80a2f20 0x7fb9c80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.835+0000 7fb9e1e00640 1 --2- 172.21.15.47:0/4008385890 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9c80a2f20 0x7fb9c80a53e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb9d409bdc0 tx=0x7fb9d40d34c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.836+0000 7fb9eb601740 1 -- 172.21.15.47:0/4137638383 shutdown_connections 2024-03-21T11:42:39.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.836+0000 7fb9eb601740 1 --2- 172.21.15.47:0/4137638383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2d0580 0x5577db2d6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.836+0000 7fb9eb601740 1 --2- 172.21.15.47:0/4137638383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2cfbc0 0x5577db2d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.836+0000 7fb9eb601740 1 --2- 172.21.15.47:0/4137638383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2ce9f0 0x5577db2cee10 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.836+0000 7fb9eb601740 1 -- 172.21.15.47:0/4137638383 >> 172.21.15.47:0/4137638383 conn(0x5577db2c9de0 msgr2=0x5577db2ca1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.836+0000 7fb9eb601740 1 -- 172.21.15.47:0/4137638383 shutdown_connections 2024-03-21T11:42:39.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.836+0000 7fb9eb601740 1 -- 172.21.15.47:0/4137638383 wait complete. 2024-03-21T11:42:39.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.837+0000 7fb9eb601740 1 -- start start 2024-03-21T11:42:39.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.837+0000 7fb9eb601740 1 Processor -- start 2024-03-21T11:42:39.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.838+0000 7fb9eb601740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2ce9f0 0x5577db2e14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.838+0000 7fb9eb601740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2cfbc0 0x5577db2e19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.838+0000 7fb9eb601740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2d0580 0x5577db2e89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.838+0000 7fb9eb601740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577db2d7530 con 0x5577db2d0580 2024-03-21T11:42:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.838+0000 7fb9eb601740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577db1ef0d0 con 0x5577db2cfbc0 2024-03-21T11:42:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.838+0000 7fb9eb601740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577db2d76b0 con 0x5577db2ce9f0 2024-03-21T11:42:39.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.838+0000 7fb9eb601740 10 client.0 _ll_register_callbacks cb 0x5577db1f3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:39.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.838+0000 7fb9e1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2cfbc0 0x5577db2e19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.838+0000 7fb9e1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2cfbc0 0x5577db2e19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60510/0 (socket says 172.21.15.47:60510) 2024-03-21T11:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.838+0000 7fb9e1400640 1 -- 172.21.15.47:0/1054049413 learned_addr learned my addr 172.21.15.47:0/1054049413 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.838+0000 7fb9e2800640 1 --2- 172.21.15.47:0/1054049413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2d0580 0x5577db2e89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.839+0000 7fb9e2800640 1 --2- 172.21.15.47:0/1054049413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2ce9f0 0x5577db2e14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.839+0000 7fb9e1400640 1 -- 172.21.15.47:0/1054049413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2ce9f0 msgr2=0x5577db2e14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.839+0000 7fb9e1400640 1 --2- 172.21.15.47:0/1054049413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2ce9f0 0x5577db2e14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.839+0000 7fb9e1400640 1 -- 172.21.15.47:0/1054049413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2d0580 msgr2=0x5577db2e89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.839+0000 7fb9e1400640 1 --2- 172.21.15.47:0/1054049413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2d0580 0x5577db2e89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.839+0000 7fb9e1400640 1 -- 172.21.15.47:0/1054049413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577daf473f0 con 0x5577db2cfbc0 2024-03-21T11:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.839+0000 7fb9e2800640 1 --2- 172.21.15.47:0/1054049413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2ce9f0 0x5577db2e14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.839+0000 7fb9e1400640 1 --2- 172.21.15.47:0/1054049413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2cfbc0 0x5577db2e19e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0012700 tx=0x7fb9d0012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb29f1f1d12a1ea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.839+0000 7fb9c3400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.840+0000 7fb9c3400640 1 -- 172.21.15.47:0/1054049413 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0004030 con 0x5577db2cfbc0 2024-03-21T11:42:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.840+0000 7fb9c3400640 1 -- 172.21.15.47:0/1054049413 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb9d00041d0 con 0x5577db2cfbc0 2024-03-21T11:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.840+0000 7fb9eb601740 1 -- 172.21.15.47:0/1054049413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9dc033bc0 con 0x5577db2cfbc0 2024-03-21T11:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.840+0000 7fb9c3400640 1 -- 172.21.15.47:0/1054049413 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9d00043d0 con 0x5577db2cfbc0 2024-03-21T11:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.840+0000 7fb9eb601740 10 client.31793 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.840+0000 7fb9eb601740 1 -- 172.21.15.47:0/1054049413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5577db0117d0 con 0x5577db2cfbc0 2024-03-21T11:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.841+0000 7fb9c3400640 1 -- 172.21.15.47:0/1054049413 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9dc033bc0 con 0x5577db2cfbc0 2024-03-21T11:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.841+0000 7fb9c3400640 1 -- 172.21.15.47:0/1054049413 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb9d0003b90 con 0x5577db2cfbc0 2024-03-21T11:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.841+0000 7fb9eb601740 10 client.31793 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.841+0000 7fb9eb601740 10 client.31793 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.841+0000 7fb9eb601740 20 client.31793 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.841+0000 7fb9eb601740 1 --2- 172.21.15.47:0/1054049413 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5577db2ecf50 0x5577db2ef370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.841+0000 7fb9eb601740 4 client.31793 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:39.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.841+0000 7fb9eb601740 1 -- 172.21.15.47:0/1054049413 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577db240800 con 0x5577db2ecf50 2024-03-21T11:42:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.841+0000 7fb9e2800640 1 --2- 172.21.15.47:0/1054049413 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5577db2ecf50 0x5577db2ef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.843+0000 7fb9e2800640 1 --2- 172.21.15.47:0/1054049413 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5577db2ecf50 0x5577db2ef370 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d8c11f937b03e95 server_cookie=be831f062aca1bfd in_seq=0 out_seq=0 2024-03-21T11:42:39.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.843+0000 7fb9c3400640 10 client.31793 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:42:39.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.845+0000 7fb9c3400640 1 -- 172.21.15.47:0/1054049413 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5577db240800 con 0x5577db2ecf50 2024-03-21T11:42:39.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.845+0000 7fb9c3400640 10 client.31793 handle_command_reply: tid=0 2024-03-21T11:42:39.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.845+0000 7fb9eb601740 1 client.31793 shutdown 2024-03-21T11:42:39.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.846+0000 7fb9eb601740 1 -- 172.21.15.47:0/1054049413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2cfbc0 msgr2=0x5577db2e19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.846+0000 7fb9eb601740 1 --2- 172.21.15.47:0/1054049413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2cfbc0 0x5577db2e19e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0012700 tx=0x7fb9d0012bc0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.846+0000 7fb9eb601740 1 -- 172.21.15.47:0/1054049413 shutdown_connections 2024-03-21T11:42:39.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.846+0000 7fb9eb601740 1 --2- 172.21.15.47:0/1054049413 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5577db2ecf50 0x5577db2ef370 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.846+0000 7fb9eb601740 1 --2- 172.21.15.47:0/1054049413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2d0580 0x5577db2e89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.846+0000 7fb9eb601740 1 --2- 172.21.15.47:0/1054049413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2cfbc0 0x5577db2e19e0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.846+0000 7fb9eb601740 1 --2- 172.21.15.47:0/1054049413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2ce9f0 0x5577db2e14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.846+0000 7fb9eb601740 1 -- 172.21.15.47:0/1054049413 >> 172.21.15.47:0/1054049413 conn(0x5577db2c9de0 msgr2=0x5577db2c9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.846+0000 7fb9c3400640 0 client.31793 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:42:39.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.846+0000 7fb9eb601740 1 -- 172.21.15.47:0/1054049413 shutdown_connections 2024-03-21T11:42:39.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.847+0000 7fb9eb601740 1 -- 172.21.15.47:0/1054049413 wait complete. 2024-03-21T11:42:39.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.847+0000 7fb9eb601740 20 client.31793 trim_cache size 0 max 16384 2024-03-21T11:42:39.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.870+0000 7fb9eb601740 1 Processor -- start 2024-03-21T11:42:39.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.870+0000 7fb9eb601740 1 -- start start 2024-03-21T11:42:39.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.870+0000 7fb9eb601740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2ce9f0 0x5577db2cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.870+0000 7fb9eb601740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2d0580 0x5577db2e8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.870+0000 7fb9eb601740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2d2d00 0x5577db2d3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.870+0000 7fb9eb601740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e41a7d00 con 0x5577db2ce9f0 2024-03-21T11:42:39.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.870+0000 7fb9eb601740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e41a9680 con 0x5577db2d2d00 2024-03-21T11:42:39.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.870+0000 7fb9eb601740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e41a8360 con 0x5577db2d0580 2024-03-21T11:42:39.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.871+0000 7fb9e2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2ce9f0 0x5577db2cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.871+0000 7fb9e2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2ce9f0 0x5577db2cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43642/0 (socket says 172.21.15.47:43642) 2024-03-21T11:42:39.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.871+0000 7fb9e2800640 1 -- 172.21.15.47:0/1687318095 learned_addr learned my addr 172.21.15.47:0/1687318095 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.871+0000 7fb9e1400640 1 --2- 172.21.15.47:0/1687318095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2d0580 0x5577db2e8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.872+0000 7fb9e2800640 1 --2- 172.21.15.47:0/1687318095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2d2d00 0x5577db2d3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.872+0000 7fb9e1400640 1 -- 172.21.15.47:0/1687318095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2d2d00 msgr2=0x5577db2d3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.872+0000 7fb9e1400640 1 --2- 172.21.15.47:0/1687318095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2d2d00 0x5577db2d3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.872+0000 7fb9e1400640 1 -- 172.21.15.47:0/1687318095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2ce9f0 msgr2=0x5577db2cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.873+0000 7fb9e1400640 1 --2- 172.21.15.47:0/1687318095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2ce9f0 0x5577db2cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.873+0000 7fb9e1400640 1 -- 172.21.15.47:0/1687318095 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577db2cad50 con 0x5577db2d0580 2024-03-21T11:42:39.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.873+0000 7fb9e2800640 1 --2- 172.21.15.47:0/1687318095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2ce9f0 0x5577db2cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:39.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.873+0000 7fb9e1400640 1 --2- 172.21.15.47:0/1687318095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2d0580 0x5577db2e8970 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0004000 tx=0x7fb9d0009dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bf836f9d9e29314 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.874+0000 7fb9ce600640 1 -- 172.21.15.47:0/1687318095 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0010620 con 0x5577db2d0580 2024-03-21T11:42:39.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.874+0000 7fb9ce600640 1 -- 172.21.15.47:0/1687318095 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9d0004030 con 0x5577db2d0580 2024-03-21T11:42:39.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.874+0000 7fb9ce600640 1 -- 172.21.15.47:0/1687318095 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000fdd0 con 0x5577db2d0580 2024-03-21T11:42:39.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.875+0000 7fb9eb601740 1 -- 172.21.15.47:0/1687318095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2d0580 msgr2=0x5577db2e8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.875+0000 7fb9eb601740 1 --2- 172.21.15.47:0/1687318095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2d0580 0x5577db2e8970 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0004000 tx=0x7fb9d0009dd0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.876+0000 7fb9eb601740 1 -- 172.21.15.47:0/1687318095 shutdown_connections 2024-03-21T11:42:39.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.876+0000 7fb9eb601740 1 --2- 172.21.15.47:0/1687318095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2d2d00 0x5577db2d3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.876+0000 7fb9eb601740 1 --2- 172.21.15.47:0/1687318095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2d0580 0x5577db2e8970 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.876+0000 7fb9eb601740 1 --2- 172.21.15.47:0/1687318095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2ce9f0 0x5577db2cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.876+0000 7fb9eb601740 1 -- 172.21.15.47:0/1687318095 >> 172.21.15.47:0/1687318095 conn(0x5577db2c9de0 msgr2=0x5577db2cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.876+0000 7fb9eb601740 1 -- 172.21.15.47:0/1687318095 shutdown_connections 2024-03-21T11:42:39.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.876+0000 7fb9eb601740 1 -- 172.21.15.47:0/1687318095 wait complete. 2024-03-21T11:42:39.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.876+0000 7fb9eb601740 1 -- start start 2024-03-21T11:42:39.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.877+0000 7fb9eb601740 1 Processor -- start 2024-03-21T11:42:39.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.877+0000 7fb9eb601740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2ce9f0 0x5577db2e8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.877+0000 7fb9eb601740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2d0580 0x5577db2e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.877+0000 7fb9eb601740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2d2d00 0x5577db2e4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.877+0000 7fb9eb601740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e41a9020 con 0x5577db2ce9f0 2024-03-21T11:42:39.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.877+0000 7fb9eb601740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e41a89c0 con 0x5577db2d2d00 2024-03-21T11:42:39.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.877+0000 7fb9eb601740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e41a5ec0 con 0x5577db2d0580 2024-03-21T11:42:39.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.877+0000 7fb9eb601740 10 client.0 _ll_register_callbacks cb 0x7fb9d0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:39.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.877+0000 7fb9e2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2ce9f0 0x5577db2e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.877+0000 7fb9e1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2d0580 0x5577db2e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.877+0000 7fb9e1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2d0580 0x5577db2e86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35128/0 (socket says 172.21.15.47:35128) 2024-03-21T11:42:39.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.877+0000 7fb9e1400640 1 -- 172.21.15.47:0/533165405 learned_addr learned my addr 172.21.15.47:0/533165405 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:39.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.878+0000 7fb9e2800640 1 --2- 172.21.15.47:0/533165405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2d2d00 0x5577db2e4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.878+0000 7fb9e2800640 1 -- 172.21.15.47:0/533165405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2d0580 msgr2=0x5577db2e86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.878+0000 7fb9e2800640 1 --2- 172.21.15.47:0/533165405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2d0580 0x5577db2e86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.878+0000 7fb9e2800640 1 -- 172.21.15.47:0/533165405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2d2d00 msgr2=0x5577db2e4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:39.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.878+0000 7fb9e2800640 1 --2- 172.21.15.47:0/533165405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2d2d00 0x5577db2e4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.878+0000 7fb9e2800640 1 -- 172.21.15.47:0/533165405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577db2e5290 con 0x5577db2ce9f0 2024-03-21T11:42:39.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.878+0000 7fb9e2800640 1 --2- 172.21.15.47:0/533165405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2ce9f0 0x5577db2e8170 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc02c500 tx=0x7fb9dc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7863e2f9ea30142 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:39.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.878+0000 7fb9c3400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:42:39.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.878+0000 7fb9c3400640 1 -- 172.21.15.47:0/533165405 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc043040 con 0x5577db2ce9f0 2024-03-21T11:42:39.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.879+0000 7fb9c3400640 1 -- 172.21.15.47:0/533165405 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb9dc033780 con 0x5577db2ce9f0 2024-03-21T11:42:39.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.879+0000 7fb9eb601740 1 -- 172.21.15.47:0/533165405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9d0004030 con 0x5577db2ce9f0 2024-03-21T11:42:39.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.879+0000 7fb9c3400640 1 -- 172.21.15.47:0/533165405 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9dc004b40 con 0x5577db2ce9f0 2024-03-21T11:42:39.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.879+0000 7fb9eb601740 10 client.21189 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:39.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.879+0000 7fb9eb601740 1 -- 172.21.15.47:0/533165405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5577db349400 con 0x5577db2ce9f0 2024-03-21T11:42:39.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.879+0000 7fb9c3400640 1 -- 172.21.15.47:0/533165405 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9d0004030 con 0x5577db2ce9f0 2024-03-21T11:42:39.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.879+0000 7fb9c3400640 1 -- 172.21.15.47:0/533165405 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb9dc052ad0 con 0x5577db2ce9f0 2024-03-21T11:42:39.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.879+0000 7fb9eb601740 10 client.21189 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:39.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.879+0000 7fb9eb601740 10 client.21189 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:42:39.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.879+0000 7fb9eb601740 20 client.21189 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:39.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.879+0000 7fb9eb601740 1 --2- 172.21.15.47:0/533165405 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5577db3499c0 0x5577db34bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:39.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.879+0000 7fb9eb601740 4 client.21189 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:42:39.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.879+0000 7fb9eb601740 1 -- 172.21.15.47:0/533165405 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5577db23ab50 con 0x5577db3499c0 2024-03-21T11:42:39.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.881+0000 7fb9e1400640 1 --2- 172.21.15.47:0/533165405 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5577db3499c0 0x5577db34bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:39.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.881+0000 7fb9e1400640 1 --2- 172.21.15.47:0/533165405 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5577db3499c0 0x5577db34bdb0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c85e10a08db64de9 server_cookie=67595e889b396fc in_seq=0 out_seq=0 2024-03-21T11:42:39.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.881+0000 7fb9c3400640 10 client.21189 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:42:39.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.884+0000 7fb9c3400640 1 -- 172.21.15.47:0/533165405 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5577db23ab50 con 0x5577db3499c0 2024-03-21T11:42:39.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.884+0000 7fb9c3400640 10 client.21189 handle_command_reply: tid=0 2024-03-21T11:42:39.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.884+0000 7fb9eb601740 1 client.21189 shutdown 2024-03-21T11:42:39.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.884+0000 7fb9eb601740 1 -- 172.21.15.47:0/533165405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2ce9f0 msgr2=0x5577db2e8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.884+0000 7fb9eb601740 1 --2- 172.21.15.47:0/533165405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2ce9f0 0x5577db2e8170 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc02c500 tx=0x7fb9dc009310 comp rx=0 tx=0).stop 2024-03-21T11:42:39.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.884+0000 7fb9eb601740 1 -- 172.21.15.47:0/533165405 shutdown_connections 2024-03-21T11:42:39.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.884+0000 7fb9eb601740 1 --2- 172.21.15.47:0/533165405 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5577db3499c0 0x5577db34bdb0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.884+0000 7fb9eb601740 1 --2- 172.21.15.47:0/533165405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577db2d2d00 0x5577db2e4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.884+0000 7fb9eb601740 1 --2- 172.21.15.47:0/533165405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5577db2d0580 0x5577db2e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.884+0000 7fb9eb601740 1 --2- 172.21.15.47:0/533165405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5577db2ce9f0 0x5577db2e8170 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.884+0000 7fb9c3400640 0 client.21189 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:42:39.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.884+0000 7fb9eb601740 1 -- 172.21.15.47:0/533165405 >> 172.21.15.47:0/533165405 conn(0x5577db2c9de0 msgr2=0x5577db2cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.885+0000 7fb9eb601740 1 -- 172.21.15.47:0/533165405 shutdown_connections 2024-03-21T11:42:39.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.885+0000 7fb9eb601740 1 -- 172.21.15.47:0/533165405 wait complete. 2024-03-21T11:42:39.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.885+0000 7fb9eb601740 20 client.21189 trim_cache size 0 max 16384 2024-03-21T11:42:39.887 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:42:39.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.885+0000 7fb9e3200640 1 -- 172.21.15.47:0/4008385890 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9c80a2f20 msgr2=0x7fb9c80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.885+0000 7fb9e3200640 1 --2- 172.21.15.47:0/4008385890 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9c80a2f20 0x7fb9c80a53e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb9d409bdc0 tx=0x7fb9d40d34c0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.885+0000 7fb9e3200640 1 -- 172.21.15.47:0/4008385890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e4102000 msgr2=0x7fb9e4199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:39.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.885+0000 7fb9e3200640 1 --2- 172.21.15.47:0/4008385890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e4102000 0x7fb9e4199a30 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400e9f0 tx=0x7fb9d400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.886+0000 7fb9e3200640 1 -- 172.21.15.47:0/4008385890 shutdown_connections 2024-03-21T11:42:39.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.886+0000 7fb9e3200640 1 --2- 172.21.15.47:0/4008385890 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9c80a2f20 0x7fb9c80a53e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.886+0000 7fb9e3200640 1 --2- 172.21.15.47:0/4008385890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9e4107260 0x7fb9e41a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.886+0000 7fb9e3200640 1 --2- 172.21.15.47:0/4008385890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9e4104930 0x7fb9e4199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.886+0000 7fb9e3200640 1 --2- 172.21.15.47:0/4008385890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e4102000 0x7fb9e4199a30 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:39.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.886+0000 7fb9e3200640 1 -- 172.21.15.47:0/4008385890 >> 172.21.15.47:0/4008385890 conn(0x7fb9e40fba10 msgr2=0x7fb9e4101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:39.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.886+0000 7fb9e3200640 1 -- 172.21.15.47:0/4008385890 shutdown_connections 2024-03-21T11:42:39.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:39.886+0000 7fb9e3200640 1 -- 172.21.15.47:0/4008385890 wait complete. 2024-03-21T11:42:39.906 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:42:39.906 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:42:39.958 INFO:tasks.workunit.client.0.smithi047.stdout: 10 146457 23.41 MB/sec execute 203 sec latency 42.207 ms 2024-03-21T11:42:40.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.181+0000 7fa73cc00640 1 -- 172.21.15.47:0/141932309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa738074100 msgr2=0x7fa738074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:40.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.181+0000 7fa73cc00640 1 --2- 172.21.15.47:0/141932309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa738074100 0x7fa738074580 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa724009900 tx=0x7fa72402bc30 comp rx=0 tx=0).stop 2024-03-21T11:42:40.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.181+0000 7fa73cc00640 1 -- 172.21.15.47:0/141932309 shutdown_connections 2024-03-21T11:42:40.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.181+0000 7fa73cc00640 1 --2- 172.21.15.47:0/141932309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73810dc80 0x7fa738110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.181+0000 7fa73cc00640 1 --2- 172.21.15.47:0/141932309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa738074100 0x7fa738074580 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.181+0000 7fa73cc00640 1 --2- 172.21.15.47:0/141932309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7380756e0 0x7fa738073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.181+0000 7fa73cc00640 1 -- 172.21.15.47:0/141932309 >> 172.21.15.47:0/141932309 conn(0x7fa7380fcc20 msgr2=0x7fa7380ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:40.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.181+0000 7fa73cc00640 1 -- 172.21.15.47:0/141932309 shutdown_connections 2024-03-21T11:42:40.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.182+0000 7fa73cc00640 1 -- 172.21.15.47:0/141932309 wait complete. 2024-03-21T11:42:40.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.183+0000 7fa73cc00640 1 Processor -- start 2024-03-21T11:42:40.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.183+0000 7fa73cc00640 1 -- start start 2024-03-21T11:42:40.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.184+0000 7fa73cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa738074100 0x7fa73810a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.184+0000 7fa73cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7380756e0 0x7fa73810a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.185+0000 7fa73cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa73810dc80 0x7fa7381ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.185+0000 7fa73cc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa738113950 con 0x7fa73810dc80 2024-03-21T11:42:40.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.185+0000 7fa73cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7381137d0 con 0x7fa738074100 2024-03-21T11:42:40.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.185+0000 7fa73cc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa738113ad0 con 0x7fa7380756e0 2024-03-21T11:42:40.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.185+0000 7fa735400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7380756e0 0x7fa73810a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.185+0000 7fa735e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa738074100 0x7fa73810a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.185+0000 7fa735e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa738074100 0x7fa73810a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60542/0 (socket says 172.21.15.47:60542) 2024-03-21T11:42:40.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.185+0000 7fa735e00640 1 -- 172.21.15.47:0/2826571004 learned_addr learned my addr 172.21.15.47:0/2826571004 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:40.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.186+0000 7fa736800640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa73810dc80 0x7fa7381ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.186+0000 7fa736800640 1 -- 172.21.15.47:0/2826571004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7380756e0 msgr2=0x7fa73810a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:40.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.187+0000 7fa736800640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7380756e0 0x7fa73810a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.187+0000 7fa736800640 1 -- 172.21.15.47:0/2826571004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa738074100 msgr2=0x7fa73810a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:40.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.187+0000 7fa736800640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa738074100 0x7fa73810a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.187+0000 7fa736800640 1 -- 172.21.15.47:0/2826571004 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7381ae5a0 con 0x7fa73810dc80 2024-03-21T11:42:40.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.187+0000 7fa735400640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7380756e0 0x7fa73810a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:42:40.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.187+0000 7fa735e00640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa738074100 0x7fa73810a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:40.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.187+0000 7fa736800640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa73810dc80 0x7fa7381ae060 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa73000e940 tx=0x7fa73000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:40.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.188+0000 7fa72ea00640 1 -- 172.21.15.47:0/2826571004 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa730004d20 con 0x7fa73810dc80 2024-03-21T11:42:40.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.188+0000 7fa72ea00640 1 -- 172.21.15.47:0/2826571004 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa730005020 con 0x7fa73810dc80 2024-03-21T11:42:40.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.188+0000 7fa72ea00640 1 -- 172.21.15.47:0/2826571004 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7300116b0 con 0x7fa73810dc80 2024-03-21T11:42:40.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.188+0000 7fa73cc00640 1 -- 172.21.15.47:0/2826571004 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7381ae770 con 0x7fa73810dc80 2024-03-21T11:42:40.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.188+0000 7fa73cc00640 1 -- 172.21.15.47:0/2826571004 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7381aec70 con 0x7fa73810dc80 2024-03-21T11:42:40.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.192+0000 7fa73f806740 1 Processor -- start 2024-03-21T11:42:40.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.194+0000 7fa72ea00640 1 -- 172.21.15.47:0/2826571004 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa73000cb20 con 0x7fa73810dc80 2024-03-21T11:42:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.195+0000 7fa73f806740 1 -- start start 2024-03-21T11:42:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.195+0000 7fa73f806740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c0142810 0x5585c0142c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.195+0000 7fa73f806740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c01439e0 0x5585c0143e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.195+0000 7fa73f806740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01443a0 0x5585c014ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.195+0000 7fa73f806740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c00a72c0 con 0x5585c01439e0 2024-03-21T11:42:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.195+0000 7fa73f806740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c004fc90 con 0x5585c0142810 2024-03-21T11:42:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.196+0000 7fa73f806740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c00a3360 con 0x5585c01443a0 2024-03-21T11:42:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.196+0000 7fa735400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c0142810 0x5585c0142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.196+0000 7fa735400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c0142810 0x5585c0142c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60554/0 (socket says 172.21.15.47:60554) 2024-03-21T11:42:40.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.196+0000 7fa735400640 1 -- 172.21.15.47:0/2763168190 learned_addr learned my addr 172.21.15.47:0/2763168190 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:40.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.196+0000 7fa735e00640 1 --2- 172.21.15.47:0/2763168190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c01439e0 0x5585c0143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.197+0000 7fa735e00640 1 -- 172.21.15.47:0/2763168190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01443a0 msgr2=0x5585c014ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:40.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.196+0000 7fa735400640 1 --2- 172.21.15.47:0/2763168190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01443a0 0x5585c014ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.197+0000 7fa735e00640 1 --2- 172.21.15.47:0/2763168190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01443a0 0x5585c014ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.197+0000 7fa735e00640 1 -- 172.21.15.47:0/2763168190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c0142810 msgr2=0x5585c0142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.197+0000 7fa735e00640 1 --2- 172.21.15.47:0/2763168190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c0142810 0x5585c0142c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.197+0000 7fa735e00640 1 -- 172.21.15.47:0/2763168190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585c0074080 con 0x5585c01439e0 2024-03-21T11:42:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.197+0000 7fa735400640 1 --2- 172.21.15.47:0/2763168190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01443a0 0x5585c014ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:42:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.198+0000 7fa735e00640 1 --2- 172.21.15.47:0/2763168190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c01439e0 0x5585c0143e60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa728009cf0 tx=0x7fa7280117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8946214a9179189 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.198+0000 7fa72ea00640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa71c0a3020 0x7fa71c0a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.198+0000 7fa723000640 1 -- 172.21.15.47:0/2763168190 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728012450 con 0x5585c01439e0 2024-03-21T11:42:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.198+0000 7fa72ea00640 1 -- 172.21.15.47:0/2826571004 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa7300cfb10 con 0x7fa73810dc80 2024-03-21T11:42:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.199+0000 7fa723000640 1 -- 172.21.15.47:0/2763168190 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa728012ad0 con 0x5585c01439e0 2024-03-21T11:42:40.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.199+0000 7fa723000640 1 -- 172.21.15.47:0/2763168190 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72800fcc0 con 0x5585c01439e0 2024-03-21T11:42:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.200+0000 7fa735400640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa71c0a3020 0x7fa71c0a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.200+0000 7fa73f806740 1 -- 172.21.15.47:0/2763168190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c01439e0 msgr2=0x5585c0143e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:40.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.200+0000 7fa73f806740 1 --2- 172.21.15.47:0/2763168190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c01439e0 0x5585c0143e60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa728009cf0 tx=0x7fa7280117b0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.202+0000 7fa735400640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa71c0a3020 0x7fa71c0a54e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa72402c170 tx=0x7fa724000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:40.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.202+0000 7fa73f806740 1 -- 172.21.15.47:0/2763168190 shutdown_connections 2024-03-21T11:42:40.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.202+0000 7fa73f806740 1 --2- 172.21.15.47:0/2763168190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01443a0 0x5585c014ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.202+0000 7fa73f806740 1 --2- 172.21.15.47:0/2763168190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c01439e0 0x5585c0143e60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.202+0000 7fa73f806740 1 --2- 172.21.15.47:0/2763168190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c0142810 0x5585c0142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.202+0000 7fa73f806740 1 -- 172.21.15.47:0/2763168190 >> 172.21.15.47:0/2763168190 conn(0x5585c013dc00 msgr2=0x5585c013e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:40.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.203+0000 7fa73f806740 1 -- 172.21.15.47:0/2763168190 shutdown_connections 2024-03-21T11:42:40.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.203+0000 7fa73f806740 1 -- 172.21.15.47:0/2763168190 wait complete. 2024-03-21T11:42:40.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.204+0000 7fa73f806740 1 -- start start 2024-03-21T11:42:40.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.204+0000 7fa73f806740 1 Processor -- start 2024-03-21T11:42:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.205+0000 7fa73f806740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c0142810 0x5585c015da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.205+0000 7fa73f806740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01439e0 0x5585c015df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.205+0000 7fa73f806740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c01443a0 0x5585c0164f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.205+0000 7fa73f806740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c015c3c0 con 0x5585c01443a0 2024-03-21T11:42:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.205+0000 7fa73f806740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c00630d0 con 0x5585c0142810 2024-03-21T11:42:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.205+0000 7fa73f806740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585c015c540 con 0x5585c01439e0 2024-03-21T11:42:40.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.205+0000 7fa73f806740 10 client.0 _ll_register_callbacks cb 0x5585c0067e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.205+0000 7fa735e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c0142810 0x5585c015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.205+0000 7fa735e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c0142810 0x5585c015da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60564/0 (socket says 172.21.15.47:60564) 2024-03-21T11:42:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.205+0000 7fa735e00640 1 -- 172.21.15.47:0/2089085301 learned_addr learned my addr 172.21.15.47:0/2089085301 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.206+0000 7fa736800640 1 --2- 172.21.15.47:0/2089085301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c01443a0 0x5585c0164f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.206+0000 7fa735400640 1 --2- 172.21.15.47:0/2089085301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01439e0 0x5585c015df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.206+0000 7fa735400640 1 -- 172.21.15.47:0/2089085301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c0142810 msgr2=0x5585c015da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.206+0000 7fa735400640 1 --2- 172.21.15.47:0/2089085301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c0142810 0x5585c015da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.206+0000 7fa735400640 1 -- 172.21.15.47:0/2089085301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c01443a0 msgr2=0x5585c0164f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.206+0000 7fa735400640 1 --2- 172.21.15.47:0/2089085301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c01443a0 0x5585c0164f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.207+0000 7fa735400640 1 -- 172.21.15.47:0/2089085301 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585bfdbb3f0 con 0x5585c01439e0 2024-03-21T11:42:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.207+0000 7fa736800640 1 --2- 172.21.15.47:0/2089085301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c01443a0 0x5585c0164f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.207+0000 7fa735400640 1 --2- 172.21.15.47:0/2089085301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01439e0 0x5585c015df40 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x5585c01438e0 tx=0x7fa724005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2460dc4314b69cfb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:40.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.207+0000 7fa717400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:42:40.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.207+0000 7fa717400640 1 -- 172.21.15.47:0/2089085301 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724050020 con 0x5585c01439e0 2024-03-21T11:42:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.207+0000 7fa717400640 1 -- 172.21.15.47:0/2089085301 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa724003800 con 0x5585c01439e0 2024-03-21T11:42:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.208+0000 7fa73f806740 1 -- 172.21.15.47:0/2089085301 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa728012ad0 con 0x5585c01439e0 2024-03-21T11:42:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.208+0000 7fa717400640 1 -- 172.21.15.47:0/2089085301 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa724003a00 con 0x5585c01439e0 2024-03-21T11:42:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.208+0000 7fa73f806740 10 client.31777 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.208+0000 7fa73f806740 1 -- 172.21.15.47:0/2089085301 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5585bfe857d0 con 0x5585c01439e0 2024-03-21T11:42:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.208+0000 7fa717400640 1 -- 172.21.15.47:0/2089085301 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa728012ad0 con 0x5585c01439e0 2024-03-21T11:42:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.209+0000 7fa717400640 1 -- 172.21.15.47:0/2089085301 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa72403f440 con 0x5585c01439e0 2024-03-21T11:42:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.209+0000 7fa73f806740 10 client.31777 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.209+0000 7fa73f806740 10 client.31777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.209+0000 7fa73f806740 20 client.31777 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.209+0000 7fa73f806740 1 --2- 172.21.15.47:0/2089085301 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585c01695d0 0x5585c016ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.209+0000 7fa73f806740 4 client.31777 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:40.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.209+0000 7fa73f806740 1 -- 172.21.15.47:0/2089085301 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585c00b4800 con 0x5585c01695d0 2024-03-21T11:42:40.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.209+0000 7fa735e00640 1 --2- 172.21.15.47:0/2089085301 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585c01695d0 0x5585c016ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.210+0000 7fa717400640 10 client.31777 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:40.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.212+0000 7fa735e00640 1 --2- 172.21.15.47:0/2089085301 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585c01695d0 0x5585c016ba30 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ef1fcaee64251a2 server_cookie=8da747330bcc0a96 in_seq=0 out_seq=0 2024-03-21T11:42:40.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.215+0000 7fa717400640 1 -- 172.21.15.47:0/2089085301 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5585c00b4800 con 0x5585c01695d0 2024-03-21T11:42:40.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.215+0000 7fa717400640 10 client.31777 handle_command_reply: tid=0 2024-03-21T11:42:40.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.215+0000 7fa73f806740 1 client.31777 shutdown 2024-03-21T11:42:40.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.216+0000 7fa73f806740 1 -- 172.21.15.47:0/2089085301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01439e0 msgr2=0x5585c015df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:40.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.216+0000 7fa73f806740 1 --2- 172.21.15.47:0/2089085301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01439e0 0x5585c015df40 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x5585c01438e0 tx=0x7fa724005fd0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.216+0000 7fa73f806740 1 -- 172.21.15.47:0/2089085301 shutdown_connections 2024-03-21T11:42:40.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.216+0000 7fa73f806740 1 --2- 172.21.15.47:0/2089085301 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585c01695d0 0x5585c016ba30 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.216+0000 7fa73f806740 1 --2- 172.21.15.47:0/2089085301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c01443a0 0x5585c0164f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.216+0000 7fa73f806740 1 --2- 172.21.15.47:0/2089085301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01439e0 0x5585c015df40 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.216+0000 7fa73f806740 1 --2- 172.21.15.47:0/2089085301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c0142810 0x5585c015da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.216+0000 7fa73f806740 1 -- 172.21.15.47:0/2089085301 >> 172.21.15.47:0/2089085301 conn(0x5585c013dc00 msgr2=0x5585c013ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:40.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.216+0000 7fa717400640 0 client.31777 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:40.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.217+0000 7fa73f806740 1 -- 172.21.15.47:0/2089085301 shutdown_connections 2024-03-21T11:42:40.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.217+0000 7fa73f806740 1 -- 172.21.15.47:0/2089085301 wait complete. 2024-03-21T11:42:40.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.217+0000 7fa73f806740 20 client.31777 trim_cache size 0 max 16384 2024-03-21T11:42:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.241+0000 7fa73f806740 1 Processor -- start 2024-03-21T11:42:40.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.241+0000 7fa73f806740 1 -- start start 2024-03-21T11:42:40.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.241+0000 7fa73f806740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c0142810 0x5585c0140e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.241+0000 7fa73f806740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c01439e0 0x5585c0164f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.241+0000 7fa73f806740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01443a0 0x5585c013c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.241+0000 7fa73f806740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71800f9a0 con 0x5585c0142810 2024-03-21T11:42:40.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.241+0000 7fa73f806740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa718011320 con 0x5585c01439e0 2024-03-21T11:42:40.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.241+0000 7fa73f806740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa718010000 con 0x5585c01443a0 2024-03-21T11:42:40.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.242+0000 7fa735400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c01439e0 0x5585c0164f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.242+0000 7fa735400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c01439e0 0x5585c0164f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60578/0 (socket says 172.21.15.47:60578) 2024-03-21T11:42:40.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.242+0000 7fa735400640 1 -- 172.21.15.47:0/3464512221 learned_addr learned my addr 172.21.15.47:0/3464512221 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:40.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.242+0000 7fa736800640 1 --2- 172.21.15.47:0/3464512221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01443a0 0x5585c013c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.242+0000 7fa735e00640 1 --2- 172.21.15.47:0/3464512221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c0142810 0x5585c0140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.242+0000 7fa735400640 1 -- 172.21.15.47:0/3464512221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01443a0 msgr2=0x5585c013c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:40.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.242+0000 7fa735400640 1 --2- 172.21.15.47:0/3464512221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01443a0 0x5585c013c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.242+0000 7fa735400640 1 -- 172.21.15.47:0/3464512221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c0142810 msgr2=0x5585c0140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:40.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.242+0000 7fa735400640 1 --2- 172.21.15.47:0/3464512221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c0142810 0x5585c0140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.242+0000 7fa735400640 1 -- 172.21.15.47:0/3464512221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585c013e670 con 0x5585c01439e0 2024-03-21T11:42:40.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.242+0000 7fa735e00640 1 --2- 172.21.15.47:0/3464512221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c0142810 0x5585c0140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:40.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.243+0000 7fa735400640 1 --2- 172.21.15.47:0/3464512221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c01439e0 0x5585c0164f40 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa72402c210 tx=0x7fa72402ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd200682aee00f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:40.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.243+0000 7fa722600640 1 -- 172.21.15.47:0/3464512221 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724064070 con 0x5585c01439e0 2024-03-21T11:42:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.243+0000 7fa722600640 1 -- 172.21.15.47:0/3464512221 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa724004330 con 0x5585c01439e0 2024-03-21T11:42:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.243+0000 7fa722600640 1 -- 172.21.15.47:0/3464512221 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724050070 con 0x5585c01439e0 2024-03-21T11:42:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.244+0000 7fa73f806740 1 -- 172.21.15.47:0/3464512221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c01439e0 msgr2=0x5585c0164f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.244+0000 7fa73f806740 1 --2- 172.21.15.47:0/3464512221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c01439e0 0x5585c0164f40 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa72402c210 tx=0x7fa72402ca30 comp rx=0 tx=0).stop 2024-03-21T11:42:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.244+0000 7fa73f806740 1 -- 172.21.15.47:0/3464512221 shutdown_connections 2024-03-21T11:42:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.244+0000 7fa73f806740 1 --2- 172.21.15.47:0/3464512221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01443a0 0x5585c013c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.244+0000 7fa73f806740 1 --2- 172.21.15.47:0/3464512221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c01439e0 0x5585c0164f40 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.244+0000 7fa73f806740 1 --2- 172.21.15.47:0/3464512221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c0142810 0x5585c0140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.244+0000 7fa73f806740 1 -- 172.21.15.47:0/3464512221 >> 172.21.15.47:0/3464512221 conn(0x5585c013dc00 msgr2=0x5585c015d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.244+0000 7fa73f806740 1 -- 172.21.15.47:0/3464512221 shutdown_connections 2024-03-21T11:42:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.244+0000 7fa73f806740 1 -- 172.21.15.47:0/3464512221 wait complete. 2024-03-21T11:42:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.245+0000 7fa73f806740 1 -- start start 2024-03-21T11:42:40.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.245+0000 7fa73f806740 1 Processor -- start 2024-03-21T11:42:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.245+0000 7fa73f806740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c0142810 0x5585c0162c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.245+0000 7fa73f806740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01439e0 0x5585c0163160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.245+0000 7fa73f806740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c01443a0 0x5585c0164170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.245+0000 7fa73f806740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa718010cc0 con 0x5585c0142810 2024-03-21T11:42:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.245+0000 7fa73f806740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa718010660 con 0x5585c01443a0 2024-03-21T11:42:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.245+0000 7fa73f806740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71800db60 con 0x5585c01439e0 2024-03-21T11:42:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.245+0000 7fa73f806740 10 client.0 _ll_register_callbacks cb 0x7fa728005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.246+0000 7fa736800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c01443a0 0x5585c0164170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.246+0000 7fa736800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c01443a0 0x5585c0164170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60588/0 (socket says 172.21.15.47:60588) 2024-03-21T11:42:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.246+0000 7fa736800640 1 -- 172.21.15.47:0/1520628686 learned_addr learned my addr 172.21.15.47:0/1520628686 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.246+0000 7fa735e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c0142810 0x5585c0162c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.246+0000 7fa735400640 1 --2- 172.21.15.47:0/1520628686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01439e0 0x5585c0163160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.246+0000 7fa736800640 1 -- 172.21.15.47:0/1520628686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01439e0 msgr2=0x5585c0163160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:40.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.246+0000 7fa736800640 1 --2- 172.21.15.47:0/1520628686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01439e0 0x5585c0163160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.246+0000 7fa736800640 1 -- 172.21.15.47:0/1520628686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c0142810 msgr2=0x5585c0162c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.246+0000 7fa736800640 1 --2- 172.21.15.47:0/1520628686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c0142810 0x5585c0162c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.246+0000 7fa736800640 1 -- 172.21.15.47:0/1520628686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585c0164760 con 0x5585c01443a0 2024-03-21T11:42:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.246+0000 7fa735e00640 1 --2- 172.21.15.47:0/1520628686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c0142810 0x5585c0162c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.246+0000 7fa735400640 1 --2- 172.21.15.47:0/1520628686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01439e0 0x5585c0163160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.247+0000 7fa736800640 1 --2- 172.21.15.47:0/1520628686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c01443a0 0x5585c0164170 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa73009bde0 tx=0x7fa73009ba60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51cb40e726f85245 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.247+0000 7fa717400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.247+0000 7fa717400640 1 -- 172.21.15.47:0/1520628686 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73009b4a0 con 0x5585c01443a0 2024-03-21T11:42:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.247+0000 7fa717400640 1 -- 172.21.15.47:0/1520628686 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa730097ba0 con 0x5585c01443a0 2024-03-21T11:42:40.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.247+0000 7fa73f806740 1 -- 172.21.15.47:0/1520628686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa724004330 con 0x5585c01443a0 2024-03-21T11:42:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.247+0000 7fa717400640 1 -- 172.21.15.47:0/1520628686 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7300ac050 con 0x5585c01443a0 2024-03-21T11:42:40.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.247+0000 7fa73f806740 10 client.31817 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:40.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.247+0000 7fa73f806740 1 -- 172.21.15.47:0/1520628686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5585c01c5520 con 0x5585c01443a0 2024-03-21T11:42:40.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.248+0000 7fa717400640 1 -- 172.21.15.47:0/1520628686 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa724004330 con 0x5585c01443a0 2024-03-21T11:42:40.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.248+0000 7fa717400640 1 -- 172.21.15.47:0/1520628686 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa73000f040 con 0x5585c01443a0 2024-03-21T11:42:40.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.248+0000 7fa73f806740 10 client.31817 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:40.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.248+0000 7fa73f806740 10 client.31817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:40.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.248+0000 7fa73f806740 20 client.31817 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:40.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.248+0000 7fa73f806740 1 --2- 172.21.15.47:0/1520628686 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585c01c5ec0 0x5585c01c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:40.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.248+0000 7fa73f806740 4 client.31817 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:42:40.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.248+0000 7fa73f806740 1 -- 172.21.15.47:0/1520628686 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5585c00aeb50 con 0x5585c01c5ec0 2024-03-21T11:42:40.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.248+0000 7fa735e00640 1 --2- 172.21.15.47:0/1520628686 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585c01c5ec0 0x5585c01c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:40.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.249+0000 7fa717400640 10 client.31817 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:40.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.249+0000 7fa735e00640 1 --2- 172.21.15.47:0/1520628686 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585c01c5ec0 0x5585c01c82e0 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb071e9354ce015 server_cookie=39a2350b2a08f63e in_seq=0 out_seq=0 2024-03-21T11:42:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.255+0000 7fa717400640 1 -- 172.21.15.47:0/1520628686 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5585c00aeb50 con 0x5585c01c5ec0 2024-03-21T11:42:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.255+0000 7fa717400640 10 client.31817 handle_command_reply: tid=0 2024-03-21T11:42:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.255+0000 7fa73f806740 1 client.31817 shutdown 2024-03-21T11:42:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.255+0000 7fa73f806740 1 -- 172.21.15.47:0/1520628686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c01443a0 msgr2=0x5585c0164170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.255+0000 7fa73f806740 1 --2- 172.21.15.47:0/1520628686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c01443a0 0x5585c0164170 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa73009bde0 tx=0x7fa73009ba60 comp rx=0 tx=0).stop 2024-03-21T11:42:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.255+0000 7fa73f806740 1 -- 172.21.15.47:0/1520628686 shutdown_connections 2024-03-21T11:42:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.255+0000 7fa73f806740 1 --2- 172.21.15.47:0/1520628686 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585c01c5ec0 0x5585c01c82e0 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.255+0000 7fa73f806740 1 --2- 172.21.15.47:0/1520628686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585c01443a0 0x5585c0164170 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.255+0000 7fa73f806740 1 --2- 172.21.15.47:0/1520628686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585c01439e0 0x5585c0163160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.255+0000 7fa73f806740 1 --2- 172.21.15.47:0/1520628686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585c0142810 0x5585c0162c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.255+0000 7fa717400640 0 client.31817 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.255+0000 7fa73f806740 1 -- 172.21.15.47:0/1520628686 >> 172.21.15.47:0/1520628686 conn(0x5585c013dc00 msgr2=0x5585c015c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:40.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.255+0000 7fa73f806740 1 -- 172.21.15.47:0/1520628686 shutdown_connections 2024-03-21T11:42:40.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.256+0000 7fa73f806740 1 -- 172.21.15.47:0/1520628686 wait complete. 2024-03-21T11:42:40.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.256+0000 7fa73f806740 20 client.31817 trim_cache size 0 max 16384 2024-03-21T11:42:40.258 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:42:40.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.256+0000 7fa73cc00640 1 -- 172.21.15.47:0/2826571004 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa71c0a3020 msgr2=0x7fa71c0a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:40.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.256+0000 7fa73cc00640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa71c0a3020 0x7fa71c0a54e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa72402c170 tx=0x7fa724000bc0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.256+0000 7fa73cc00640 1 -- 172.21.15.47:0/2826571004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa73810dc80 msgr2=0x7fa7381ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.256+0000 7fa73cc00640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa73810dc80 0x7fa7381ae060 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa73000e940 tx=0x7fa73000ee00 comp rx=0 tx=0).stop 2024-03-21T11:42:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.257+0000 7fa73cc00640 1 -- 172.21.15.47:0/2826571004 shutdown_connections 2024-03-21T11:42:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.257+0000 7fa73cc00640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa71c0a3020 0x7fa71c0a54e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.257+0000 7fa73cc00640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa73810dc80 0x7fa7381ae060 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.257+0000 7fa73cc00640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7380756e0 0x7fa73810a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.257+0000 7fa73cc00640 1 --2- 172.21.15.47:0/2826571004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa738074100 0x7fa73810a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.257+0000 7fa73cc00640 1 -- 172.21.15.47:0/2826571004 >> 172.21.15.47:0/2826571004 conn(0x7fa7380fcc20 msgr2=0x7fa738101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.257+0000 7fa73cc00640 1 -- 172.21.15.47:0/2826571004 shutdown_connections 2024-03-21T11:42:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:40.257+0000 7fa73cc00640 1 -- 172.21.15.47:0/2826571004 wait complete. 2024-03-21T11:42:40.958 INFO:tasks.workunit.client.0.smithi047.stdout: 10 146861 23.37 MB/sec execute 204 sec latency 237.042 ms 2024-03-21T11:42:40.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:40 smithi102 ceph-mon[36748]: pgmap v980: 129 pgs: 129 active+clean; 527 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.1 MiB/s wr, 503 op/s 2024-03-21T11:42:41.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:40 smithi047 ceph-mon[31351]: pgmap v980: 129 pgs: 129 active+clean; 527 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.1 MiB/s wr, 503 op/s 2024-03-21T11:42:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:40 smithi100 ceph-mon[36245]: pgmap v980: 129 pgs: 129 active+clean; 527 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.1 MiB/s wr, 503 op/s 2024-03-21T11:42:41.277 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:42:41.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:42:41.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:42:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.538+0000 7fdfbea00640 1 -- 172.21.15.47:0/2760304581 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8032a00 con 0x7fdfc0109870 2024-03-21T11:42:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.539+0000 7fdfc6a00640 1 -- 172.21.15.47:0/2760304581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc0109870 msgr2=0x7fdfc0110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.539+0000 7fdfc6a00640 1 --2- 172.21.15.47:0/2760304581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc0109870 0x7fdfc0110100 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009ca0 tx=0x7fdfb802c120 comp rx=0 tx=0).stop 2024-03-21T11:42:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.539+0000 7fdfc6a00640 1 -- 172.21.15.47:0/2760304581 shutdown_connections 2024-03-21T11:42:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.539+0000 7fdfc6a00640 1 --2- 172.21.15.47:0/2760304581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc0109870 0x7fdfc0110100 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.539+0000 7fdfc6a00640 1 --2- 172.21.15.47:0/2760304581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfc0108eb0 0x7fdfc0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.539+0000 7fdfc6a00640 1 --2- 172.21.15.47:0/2760304581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfc0107cc0 0x7fdfc01080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.539+0000 7fdfc6a00640 1 -- 172.21.15.47:0/2760304581 >> 172.21.15.47:0/2760304581 conn(0x7fdfc0076c60 msgr2=0x7fdfc00790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.540+0000 7fdfc6a00640 1 -- 172.21.15.47:0/2760304581 shutdown_connections 2024-03-21T11:42:41.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.540+0000 7fdfc6a00640 1 -- 172.21.15.47:0/2760304581 wait complete. 2024-03-21T11:42:41.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.541+0000 7fdfc6a00640 1 Processor -- start 2024-03-21T11:42:41.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.541+0000 7fdfc6a00640 1 -- start start 2024-03-21T11:42:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.542+0000 7fdfc6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfc0107cc0 0x7fdfc019e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.542+0000 7fdfc6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc0108eb0 0x7fdfc019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.542+0000 7fdfc6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfc0109870 0x7fdfc01a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.542+0000 7fdfc6a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc0113a10 con 0x7fdfc0107cc0 2024-03-21T11:42:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.542+0000 7fdfc6a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc0113890 con 0x7fdfc0108eb0 2024-03-21T11:42:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.542+0000 7fdfc6a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc0113b90 con 0x7fdfc0109870 2024-03-21T11:42:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.542+0000 7fdfbfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfc0107cc0 0x7fdfc019e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.542+0000 7fdfbf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc0108eb0 0x7fdfc019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.542+0000 7fdfbf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc0108eb0 0x7fdfc019e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60610/0 (socket says 172.21.15.47:60610) 2024-03-21T11:42:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.542+0000 7fdfbf400640 1 -- 172.21.15.47:0/3708442206 learned_addr learned my addr 172.21.15.47:0/3708442206 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.542+0000 7fdfc4a00640 1 --2- 172.21.15.47:0/3708442206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfc0109870 0x7fdfc01a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.543+0000 7fdfbf400640 1 -- 172.21.15.47:0/3708442206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfc0109870 msgr2=0x7fdfc01a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.543+0000 7fdfbf400640 1 --2- 172.21.15.47:0/3708442206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfc0109870 0x7fdfc01a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.543+0000 7fdfbf400640 1 -- 172.21.15.47:0/3708442206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfc0107cc0 msgr2=0x7fdfc019e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.543+0000 7fdfbf400640 1 --2- 172.21.15.47:0/3708442206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfc0107cc0 0x7fdfc019e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.543+0000 7fdfbf400640 1 -- 172.21.15.47:0/3708442206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc01a5c90 con 0x7fdfc0108eb0 2024-03-21T11:42:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.543+0000 7fdfbf400640 1 --2- 172.21.15.47:0/3708442206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc0108eb0 0x7fdfc019e5b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00b450 tx=0x7fdfac00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.544+0000 7fdfbcc00640 1 -- 172.21.15.47:0/3708442206 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac015020 con 0x7fdfc0108eb0 2024-03-21T11:42:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.544+0000 7fdfbcc00640 1 -- 172.21.15.47:0/3708442206 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfac011d70 con 0x7fdfc0108eb0 2024-03-21T11:42:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.544+0000 7fdfbcc00640 1 -- 172.21.15.47:0/3708442206 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00f340 con 0x7fdfc0108eb0 2024-03-21T11:42:41.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.544+0000 7fdfc6a00640 1 -- 172.21.15.47:0/3708442206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc01a5f80 con 0x7fdfc0108eb0 2024-03-21T11:42:41.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.544+0000 7fdfc6a00640 1 -- 172.21.15.47:0/3708442206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc01a6410 con 0x7fdfc0108eb0 2024-03-21T11:42:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.558+0000 7fdfbcc00640 1 -- 172.21.15.47:0/3708442206 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdfac018020 con 0x7fdfc0108eb0 2024-03-21T11:42:41.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.558+0000 7fdfc97b7740 1 Processor -- start 2024-03-21T11:42:41.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.559+0000 7fdfc97b7740 1 -- start start 2024-03-21T11:42:41.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.559+0000 7fdfc97b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf690a0 0x5575daf694c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.559+0000 7fdfc97b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf69a90 0x5575daf6e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.563+0000 7fdfc97b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 0x5575daf71090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.564+0000 7fdfc97b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575daeceeb0 con 0x5575daf690a0 2024-03-21T11:42:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.564+0000 7fdfc97b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575dae77a60 con 0x5575daf69a90 2024-03-21T11:42:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.564+0000 7fdfc97b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575daecaf50 con 0x5575daf6ec00 2024-03-21T11:42:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.564+0000 7fdfbf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 0x5575daf71090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.564+0000 7fdfbf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 0x5575daf71090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35198/0 (socket says 172.21.15.47:35198) 2024-03-21T11:42:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.564+0000 7fdfbf400640 1 -- 172.21.15.47:0/3124708664 learned_addr learned my addr 172.21.15.47:0/3124708664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.564+0000 7fdfbfe00640 1 --2- 172.21.15.47:0/3124708664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf69a90 0x5575daf6e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.565+0000 7fdfbfe00640 1 -- 172.21.15.47:0/3124708664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 msgr2=0x5575daf71090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.565+0000 7fdfbfe00640 1 --2- 172.21.15.47:0/3124708664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 0x5575daf71090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.565+0000 7fdfbfe00640 1 -- 172.21.15.47:0/3124708664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf690a0 msgr2=0x5575daf694c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.565+0000 7fdfbfe00640 1 --2- 172.21.15.47:0/3124708664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf690a0 0x5575daf694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.565+0000 7fdfbfe00640 1 -- 172.21.15.47:0/3124708664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575dacab7e0 con 0x5575daf69a90 2024-03-21T11:42:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.566+0000 7fdfbfe00640 1 --2- 172.21.15.47:0/3124708664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf69a90 0x5575daf6e590 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000bbc0 tx=0x7fdfb00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1b659678e78817c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.566+0000 7fdfbcc00640 1 --2- 172.21.15.47:0/3708442206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdfa40a2f20 0x7fdfa40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.566+0000 7fdfbcc00640 1 -- 172.21.15.47:0/3708442206 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdfac0cf180 con 0x7fdfc0108eb0 2024-03-21T11:42:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.566+0000 7fdfb5200640 1 -- 172.21.15.47:0/3124708664 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0004030 con 0x5575daf69a90 2024-03-21T11:42:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.567+0000 7fdfb5200640 1 -- 172.21.15.47:0/3124708664 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfb0005500 con 0x5575daf69a90 2024-03-21T11:42:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.567+0000 7fdfb5200640 1 -- 172.21.15.47:0/3124708664 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0004d80 con 0x5575daf69a90 2024-03-21T11:42:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.567+0000 7fdfc4a00640 1 --2- 172.21.15.47:0/3708442206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdfa40a2f20 0x7fdfa40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.568+0000 7fdfc97b7740 1 -- 172.21.15.47:0/3124708664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf69a90 msgr2=0x5575daf6e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.568+0000 7fdfc4a00640 1 --2- 172.21.15.47:0/3708442206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdfa40a2f20 0x7fdfa40a53e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8006010 tx=0x7fdfb8009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.568+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/3124708664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf69a90 0x5575daf6e590 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000bbc0 tx=0x7fdfb00107a0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.569+0000 7fdfc97b7740 1 -- 172.21.15.47:0/3124708664 shutdown_connections 2024-03-21T11:42:41.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.569+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/3124708664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 0x5575daf71090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.569+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/3124708664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf69a90 0x5575daf6e590 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.569+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/3124708664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf690a0 0x5575daf694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.569+0000 7fdfc97b7740 1 -- 172.21.15.47:0/3124708664 >> 172.21.15.47:0/3124708664 conn(0x5575daf64610 msgr2=0x5575daf64a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:41.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.569+0000 7fdfc97b7740 1 -- 172.21.15.47:0/3124708664 shutdown_connections 2024-03-21T11:42:41.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.569+0000 7fdfc97b7740 1 -- 172.21.15.47:0/3124708664 wait complete. 2024-03-21T11:42:41.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.570+0000 7fdfc97b7740 1 -- start start 2024-03-21T11:42:41.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.570+0000 7fdfc97b7740 1 Processor -- start 2024-03-21T11:42:41.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.571+0000 7fdfc97b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf690a0 0x5575daf732b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.571+0000 7fdfc97b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf69a90 0x5575daf737f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.571+0000 7fdfc97b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 0x5575daf7a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.572+0000 7fdfbf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf69a90 0x5575daf737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.572+0000 7fdfc97b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575daf71bb0 con 0x5575daf690a0 2024-03-21T11:42:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.572+0000 7fdfc97b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575dae8aeb0 con 0x5575daf69a90 2024-03-21T11:42:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.572+0000 7fdfc97b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575daf71d30 con 0x5575daf6ec00 2024-03-21T11:42:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.572+0000 7fdfbf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf69a90 0x5575daf737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60628/0 (socket says 172.21.15.47:60628) 2024-03-21T11:42:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.572+0000 7fdfbf400640 1 -- 172.21.15.47:0/3351182289 learned_addr learned my addr 172.21.15.47:0/3351182289 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.572+0000 7fdfc97b7740 10 client.0 _ll_register_callbacks cb 0x5575dae7f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.572+0000 7fdfc4a00640 1 --2- 172.21.15.47:0/3351182289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 0x5575daf7a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.572+0000 7fdfbfe00640 1 --2- 172.21.15.47:0/3351182289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf690a0 0x5575daf732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.572+0000 7fdfbf400640 1 -- 172.21.15.47:0/3351182289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 msgr2=0x5575daf7a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.573+0000 7fdfbf400640 1 --2- 172.21.15.47:0/3351182289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 0x5575daf7a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.573+0000 7fdfbf400640 1 -- 172.21.15.47:0/3351182289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf690a0 msgr2=0x5575daf732b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.573+0000 7fdfbf400640 1 --2- 172.21.15.47:0/3351182289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf690a0 0x5575daf732b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.573+0000 7fdfbf400640 1 -- 172.21.15.47:0/3351182289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575dad0bf70 con 0x5575daf69a90 2024-03-21T11:42:41.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.573+0000 7fdfc4a00640 1 --2- 172.21.15.47:0/3351182289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 0x5575daf7a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.574+0000 7fdfbf400640 1 --2- 172.21.15.47:0/3351182289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf69a90 0x5575daf737f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009fd0 tx=0x7fdfac097100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c10c53fb1cb3021 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.574+0000 7fdfa9600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:41.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.576+0000 7fdfa9600640 1 -- 172.21.15.47:0/3351182289 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac09b060 con 0x5575daf69a90 2024-03-21T11:42:41.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.576+0000 7fdfa9600640 1 -- 172.21.15.47:0/3351182289 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdfac099820 con 0x5575daf69a90 2024-03-21T11:42:41.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.577+0000 7fdfc97b7740 1 -- 172.21.15.47:0/3351182289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdfb0005500 con 0x5575daf69a90 2024-03-21T11:42:41.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.578+0000 7fdfa9600640 1 -- 172.21.15.47:0/3351182289 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfac09ae80 con 0x5575daf69a90 2024-03-21T11:42:41.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.578+0000 7fdfc97b7740 10 client.31829 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:41.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.578+0000 7fdfc97b7740 1 -- 172.21.15.47:0/3351182289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5575daba3170 con 0x5575daf69a90 2024-03-21T11:42:41.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.578+0000 7fdfa9600640 1 -- 172.21.15.47:0/3351182289 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfb0005500 con 0x5575daf69a90 2024-03-21T11:42:41.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.578+0000 7fdfa9600640 1 -- 172.21.15.47:0/3351182289 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdfac09f150 con 0x5575daf69a90 2024-03-21T11:42:41.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.578+0000 7fdfc97b7740 10 client.31829 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:41.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.578+0000 7fdfc97b7740 10 client.31829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:41.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.578+0000 7fdfc97b7740 20 client.31829 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:41.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.578+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/3351182289 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5575daf7f070 0x5575daf81530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.579+0000 7fdfc97b7740 4 client.31829 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:41.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.579+0000 7fdfc97b7740 1 -- 172.21.15.47:0/3351182289 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575daedc3f0 con 0x5575daf7f070 2024-03-21T11:42:41.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.580+0000 7fdfbfe00640 1 --2- 172.21.15.47:0/3351182289 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5575daf7f070 0x5575daf81530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.583+0000 7fdfbfe00640 1 --2- 172.21.15.47:0/3351182289 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5575daf7f070 0x5575daf81530 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8daf563fadd3c9bb server_cookie=dd9f39168a40b978 in_seq=0 out_seq=0 2024-03-21T11:42:41.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.583+0000 7fdfa9600640 10 client.31829 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:41.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.586+0000 7fdfa9600640 1 -- 172.21.15.47:0/3351182289 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5575daedc3f0 con 0x5575daf7f070 2024-03-21T11:42:41.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.586+0000 7fdfa9600640 10 client.31829 handle_command_reply: tid=0 2024-03-21T11:42:41.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.586+0000 7fdfc97b7740 1 client.31829 shutdown 2024-03-21T11:42:41.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.587+0000 7fdfc97b7740 1 -- 172.21.15.47:0/3351182289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf69a90 msgr2=0x5575daf737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.587+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/3351182289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf69a90 0x5575daf737f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009fd0 tx=0x7fdfac097100 comp rx=0 tx=0).stop 2024-03-21T11:42:41.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.588+0000 7fdfc97b7740 1 -- 172.21.15.47:0/3351182289 shutdown_connections 2024-03-21T11:42:41.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.588+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/3351182289 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5575daf7f070 0x5575daf81530 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.588+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/3351182289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 0x5575daf7a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.588+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/3351182289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf69a90 0x5575daf737f0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.588+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/3351182289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf690a0 0x5575daf732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.588+0000 7fdfc97b7740 1 -- 172.21.15.47:0/3351182289 >> 172.21.15.47:0/3351182289 conn(0x5575daf64610 msgr2=0x5575daf63830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:41.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.588+0000 7fdfa9600640 0 client.31829 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:41.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.588+0000 7fdfc97b7740 1 -- 172.21.15.47:0/3351182289 shutdown_connections 2024-03-21T11:42:41.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.588+0000 7fdfc97b7740 1 -- 172.21.15.47:0/3351182289 wait complete. 2024-03-21T11:42:41.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.588+0000 7fdfc97b7740 20 client.31829 trim_cache size 0 max 16384 2024-03-21T11:42:41.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.626+0000 7fdfc97b7740 1 Processor -- start 2024-03-21T11:42:41.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.626+0000 7fdfc97b7740 1 -- start start 2024-03-21T11:42:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.627+0000 7fdfc97b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf690a0 0x5575daf636b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.627+0000 7fdfc97b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf69a90 0x5575daf63bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.627+0000 7fdfc97b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf6ec00 0x5575daf7a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.627+0000 7fdfc97b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc01b0800 con 0x5575daf6ec00 2024-03-21T11:42:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.627+0000 7fdfc97b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc01b2180 con 0x5575daf690a0 2024-03-21T11:42:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.627+0000 7fdfc97b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc01b0e60 con 0x5575daf69a90 2024-03-21T11:42:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.627+0000 7fdfbf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf69a90 0x5575daf63bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.627+0000 7fdfc4a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf6ec00 0x5575daf7a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.627+0000 7fdfbfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf690a0 0x5575daf636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.627+0000 7fdfbfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf690a0 0x5575daf636b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60644/0 (socket says 172.21.15.47:60644) 2024-03-21T11:42:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.627+0000 7fdfbfe00640 1 -- 172.21.15.47:0/1986200544 learned_addr learned my addr 172.21.15.47:0/1986200544 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.628+0000 7fdfc4a00640 1 -- 172.21.15.47:0/1986200544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf69a90 msgr2=0x5575daf63bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.628+0000 7fdfc4a00640 1 --2- 172.21.15.47:0/1986200544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf69a90 0x5575daf63bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.628+0000 7fdfc4a00640 1 -- 172.21.15.47:0/1986200544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf690a0 msgr2=0x5575daf636b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.628+0000 7fdfc4a00640 1 --2- 172.21.15.47:0/1986200544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf690a0 0x5575daf636b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.628+0000 7fdfc4a00640 1 -- 172.21.15.47:0/1986200544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575daf5fc20 con 0x5575daf6ec00 2024-03-21T11:42:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.628+0000 7fdfbf400640 1 --2- 172.21.15.47:0/1986200544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf69a90 0x5575daf63bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.629+0000 7fdfc4a00640 1 --2- 172.21.15.47:0/1986200544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf6ec00 0x5575daf7a7f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdfb802c730 tx=0x7fdfb803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452ecf6ef3af10a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.630+0000 7fdfabe00640 1 -- 172.21.15.47:0/1986200544 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8039040 con 0x5575daf6ec00 2024-03-21T11:42:41.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.630+0000 7fdfabe00640 1 -- 172.21.15.47:0/1986200544 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfb8051070 con 0x5575daf6ec00 2024-03-21T11:42:41.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.630+0000 7fdfabe00640 1 -- 172.21.15.47:0/1986200544 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb804ca90 con 0x5575daf6ec00 2024-03-21T11:42:41.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.631+0000 7fdfc97b7740 1 -- 172.21.15.47:0/1986200544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf6ec00 msgr2=0x5575daf7a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.631+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/1986200544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf6ec00 0x5575daf7a7f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdfb802c730 tx=0x7fdfb803e000 comp rx=0 tx=0).stop 2024-03-21T11:42:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.631+0000 7fdfc97b7740 1 -- 172.21.15.47:0/1986200544 shutdown_connections 2024-03-21T11:42:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.631+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/1986200544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf6ec00 0x5575daf7a7f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.631+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/1986200544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf69a90 0x5575daf63bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.631+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/1986200544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf690a0 0x5575daf636b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.631+0000 7fdfc97b7740 1 -- 172.21.15.47:0/1986200544 >> 172.21.15.47:0/1986200544 conn(0x5575daf64610 msgr2=0x5575daf62fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:41.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.631+0000 7fdfc97b7740 1 -- 172.21.15.47:0/1986200544 shutdown_connections 2024-03-21T11:42:41.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.632+0000 7fdfc97b7740 1 -- 172.21.15.47:0/1986200544 wait complete. 2024-03-21T11:42:41.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.632+0000 7fdfc97b7740 1 -- start start 2024-03-21T11:42:41.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.632+0000 7fdfc97b7740 1 Processor -- start 2024-03-21T11:42:41.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.633+0000 7fdfc97b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf690a0 0x5575daf78fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.633+0000 7fdfc97b7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf69a90 0x5575daf794e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.633+0000 7fdfc97b7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 0x5575daf79a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.633+0000 7fdfc97b7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc01b1b20 con 0x5575daf69a90 2024-03-21T11:42:41.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.633+0000 7fdfc97b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc01b14c0 con 0x5575daf690a0 2024-03-21T11:42:41.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.633+0000 7fdfc97b7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc01ae9c0 con 0x5575daf6ec00 2024-03-21T11:42:41.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.633+0000 7fdfc97b7740 10 client.0 _ll_register_callbacks cb 0x7fdfac0046e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:41.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.634+0000 7fdfbf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf69a90 0x5575daf794e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.634+0000 7fdfbf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf69a90 0x5575daf794e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43784/0 (socket says 172.21.15.47:43784) 2024-03-21T11:42:41.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.634+0000 7fdfbf400640 1 -- 172.21.15.47:0/1511469894 learned_addr learned my addr 172.21.15.47:0/1511469894 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:41.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.634+0000 7fdfbfe00640 1 --2- 172.21.15.47:0/1511469894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf690a0 0x5575daf78fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.634+0000 7fdfbf400640 1 -- 172.21.15.47:0/1511469894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 msgr2=0x5575daf79a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.634+0000 7fdfbf400640 1 --2- 172.21.15.47:0/1511469894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 0x5575daf79a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.634+0000 7fdfbf400640 1 -- 172.21.15.47:0/1511469894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf690a0 msgr2=0x5575daf78fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.634+0000 7fdfbf400640 1 --2- 172.21.15.47:0/1511469894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf690a0 0x5575daf78fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.634+0000 7fdfbf400640 1 -- 172.21.15.47:0/1511469894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575dafda9c0 con 0x5575daf69a90 2024-03-21T11:42:41.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.634+0000 7fdfbfe00640 1 --2- 172.21.15.47:0/1511469894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf690a0 0x5575daf78fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:42:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.635+0000 7fdfbf400640 1 --2- 172.21.15.47:0/1511469894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf69a90 0x5575daf794e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0997d0 tx=0x7fdfac09b110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d69e6c011b6a59 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.635+0000 7fdfa9600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:42:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.635+0000 7fdfa9600640 1 -- 172.21.15.47:0/1511469894 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0b3070 con 0x5575daf69a90 2024-03-21T11:42:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.636+0000 7fdfa9600640 1 -- 172.21.15.47:0/1511469894 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdfac0afd80 con 0x5575daf69a90 2024-03-21T11:42:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.636+0000 7fdfc97b7740 1 -- 172.21.15.47:0/1511469894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdfb8051070 con 0x5575daf69a90 2024-03-21T11:42:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.636+0000 7fdfa9600640 1 -- 172.21.15.47:0/1511469894 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfac09bc20 con 0x5575daf69a90 2024-03-21T11:42:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.636+0000 7fdfc97b7740 10 client.21219 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.636+0000 7fdfc97b7740 1 -- 172.21.15.47:0/1511469894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5575dafdbcf0 con 0x5575daf69a90 2024-03-21T11:42:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.637+0000 7fdfa9600640 1 -- 172.21.15.47:0/1511469894 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfb8051070 con 0x5575daf69a90 2024-03-21T11:42:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.637+0000 7fdfa9600640 1 -- 172.21.15.47:0/1511469894 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdfac097270 con 0x5575daf69a90 2024-03-21T11:42:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.638+0000 7fdfc97b7740 10 client.21219 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.638+0000 7fdfc97b7740 10 client.21219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.638+0000 7fdfc97b7740 20 client.21219 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.638+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/1511469894 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5575dafdc650 0x5575dafdea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.638+0000 7fdfc97b7740 4 client.21219 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:42:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.638+0000 7fdfc97b7740 1 -- 172.21.15.47:0/1511469894 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5575daed6740 con 0x5575dafdc650 2024-03-21T11:42:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.639+0000 7fdfbfe00640 1 --2- 172.21.15.47:0/1511469894 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5575dafdc650 0x5575dafdea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.640+0000 7fdfbfe00640 1 --2- 172.21.15.47:0/1511469894 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5575dafdc650 0x5575dafdea70 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a402e114e20542d9 server_cookie=50cfa8791f3df777 in_seq=0 out_seq=0 2024-03-21T11:42:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.640+0000 7fdfa9600640 10 client.21219 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.646+0000 7fdfa9600640 1 -- 172.21.15.47:0/1511469894 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5575daed6740 con 0x5575dafdc650 2024-03-21T11:42:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.646+0000 7fdfa9600640 10 client.21219 handle_command_reply: tid=0 2024-03-21T11:42:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.646+0000 7fdfc97b7740 1 client.21219 shutdown 2024-03-21T11:42:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.647+0000 7fdfc97b7740 1 -- 172.21.15.47:0/1511469894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf69a90 msgr2=0x5575daf794e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.647+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/1511469894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf69a90 0x5575daf794e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0997d0 tx=0x7fdfac09b110 comp rx=0 tx=0).stop 2024-03-21T11:42:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.647+0000 7fdfc97b7740 1 -- 172.21.15.47:0/1511469894 shutdown_connections 2024-03-21T11:42:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.647+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/1511469894 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5575dafdc650 0x5575dafdea70 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.648+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/1511469894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575daf6ec00 0x5575daf79a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.648+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/1511469894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575daf69a90 0x5575daf794e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.648+0000 7fdfc97b7740 1 --2- 172.21.15.47:0/1511469894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5575daf690a0 0x5575daf78fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.648+0000 7fdfc97b7740 1 -- 172.21.15.47:0/1511469894 >> 172.21.15.47:0/1511469894 conn(0x5575daf64610 msgr2=0x5575daf63830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.648+0000 7fdfa9600640 0 client.21219 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.648+0000 7fdfc97b7740 1 -- 172.21.15.47:0/1511469894 shutdown_connections 2024-03-21T11:42:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.649+0000 7fdfc97b7740 1 -- 172.21.15.47:0/1511469894 wait complete. 2024-03-21T11:42:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.649+0000 7fdfc97b7740 20 client.21219 trim_cache size 0 max 16384 2024-03-21T11:42:41.651 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:42:41.652 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:42:41.652 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "bcb7aefa-b809-4b87-b178-6f11dddeeef4", 2024-03-21T11:42:41.652 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:42:41.652 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:42:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.652+0000 7fdfc6a00640 1 -- 172.21.15.47:0/3708442206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdfa40a2f20 msgr2=0x7fdfa40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.652+0000 7fdfc6a00640 1 --2- 172.21.15.47:0/3708442206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdfa40a2f20 0x7fdfa40a53e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8006010 tx=0x7fdfb8009360 comp rx=0 tx=0).stop 2024-03-21T11:42:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.652+0000 7fdfc6a00640 1 -- 172.21.15.47:0/3708442206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc0108eb0 msgr2=0x7fdfc019e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.652+0000 7fdfc6a00640 1 --2- 172.21.15.47:0/3708442206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc0108eb0 0x7fdfc019e5b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00b450 tx=0x7fdfac00b910 comp rx=0 tx=0).stop 2024-03-21T11:42:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.652+0000 7fdfc6a00640 1 -- 172.21.15.47:0/3708442206 shutdown_connections 2024-03-21T11:42:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.652+0000 7fdfc6a00640 1 --2- 172.21.15.47:0/3708442206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdfa40a2f20 0x7fdfa40a53e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.652+0000 7fdfc6a00640 1 --2- 172.21.15.47:0/3708442206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdfc0109870 0x7fdfc01a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.652+0000 7fdfc6a00640 1 --2- 172.21.15.47:0/3708442206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc0108eb0 0x7fdfc019e5b0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.652+0000 7fdfc6a00640 1 --2- 172.21.15.47:0/3708442206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfc0107cc0 0x7fdfc019e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.652+0000 7fdfc6a00640 1 -- 172.21.15.47:0/3708442206 >> 172.21.15.47:0/3708442206 conn(0x7fdfc0076c60 msgr2=0x7fdfc010c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:41.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.653+0000 7fdfc6a00640 1 -- 172.21.15.47:0/3708442206 shutdown_connections 2024-03-21T11:42:41.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.653+0000 7fdfc6a00640 1 -- 172.21.15.47:0/3708442206 wait complete. 2024-03-21T11:42:41.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:42:41.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:42:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.906+0000 7f8b61c00640 1 -- 172.21.15.47:0/920127458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b5c104ad0 msgr2=0x7f8b5c104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.906+0000 7f8b61c00640 1 --2- 172.21.15.47:0/920127458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b5c104ad0 0x7f8b5c104f50 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009900 tx=0x7f8b4802bc30 comp rx=0 tx=0).stop 2024-03-21T11:42:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.906+0000 7f8b61c00640 1 -- 172.21.15.47:0/920127458 shutdown_connections 2024-03-21T11:42:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.906+0000 7f8b61c00640 1 --2- 172.21.15.47:0/920127458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b5c105490 0x7f8b5c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.906+0000 7f8b61c00640 1 --2- 172.21.15.47:0/920127458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b5c104ad0 0x7f8b5c104f50 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.906+0000 7f8b61c00640 1 --2- 172.21.15.47:0/920127458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5c1038e0 0x7f8b5c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.906+0000 7f8b61c00640 1 -- 172.21.15.47:0/920127458 >> 172.21.15.47:0/920127458 conn(0x7f8b5c0fee50 msgr2=0x7f8b5c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.907+0000 7f8b61c00640 1 -- 172.21.15.47:0/920127458 shutdown_connections 2024-03-21T11:42:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.907+0000 7f8b61c00640 1 -- 172.21.15.47:0/920127458 wait complete. 2024-03-21T11:42:41.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.908+0000 7f8b61c00640 1 Processor -- start 2024-03-21T11:42:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.908+0000 7f8b61c00640 1 -- start start 2024-03-21T11:42:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.908+0000 7f8b61c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5c1038e0 0x7f8b5c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.908+0000 7f8b61c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b5c104ad0 0x7f8b5c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.908+0000 7f8b61c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b5c105490 0x7f8b5c1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.908+0000 7f8b61c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c10f540 con 0x7f8b5c105490 2024-03-21T11:42:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.908+0000 7f8b61c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c10f3c0 con 0x7f8b5c1038e0 2024-03-21T11:42:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.908+0000 7f8b61c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c10f6c0 con 0x7f8b5c104ad0 2024-03-21T11:42:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.909+0000 7f8b5aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5c1038e0 0x7f8b5c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.909+0000 7f8b5a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b5c104ad0 0x7f8b5c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.909+0000 7f8b5aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5c1038e0 0x7f8b5c199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60672/0 (socket says 172.21.15.47:60672) 2024-03-21T11:42:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.909+0000 7f8b5aa00640 1 -- 172.21.15.47:0/1683868555 learned_addr learned my addr 172.21.15.47:0/1683868555 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.909+0000 7f8b5a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b5c104ad0 0x7f8b5c19a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35244/0 (socket says 172.21.15.47:35244) 2024-03-21T11:42:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.909+0000 7f8b5b400640 1 --2- 172.21.15.47:0/1683868555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b5c105490 0x7f8b5c1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.909+0000 7f8b5aa00640 1 -- 172.21.15.47:0/1683868555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b5c104ad0 msgr2=0x7f8b5c19a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.909+0000 7f8b5aa00640 1 --2- 172.21.15.47:0/1683868555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b5c104ad0 0x7f8b5c19a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.909+0000 7f8b5aa00640 1 -- 172.21.15.47:0/1683868555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b5c105490 msgr2=0x7f8b5c1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.909+0000 7f8b5aa00640 1 --2- 172.21.15.47:0/1683868555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b5c105490 0x7f8b5c1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.909+0000 7f8b5aa00640 1 -- 172.21.15.47:0/1683868555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c1a1840 con 0x7f8b5c1038e0 2024-03-21T11:42:41.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.910+0000 7f8b5aa00640 1 --2- 172.21.15.47:0/1683868555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5c1038e0 0x7f8b5c199c30 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00e9c0 tx=0x7f8b4c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:41.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.910+0000 7f8b53400640 1 -- 172.21.15.47:0/1683868555 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c00ccc0 con 0x7f8b5c1038e0 2024-03-21T11:42:41.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.910+0000 7f8b53400640 1 -- 172.21.15.47:0/1683868555 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b4c0035f0 con 0x7f8b5c1038e0 2024-03-21T11:42:41.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.910+0000 7f8b53400640 1 -- 172.21.15.47:0/1683868555 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c00ce60 con 0x7f8b5c1038e0 2024-03-21T11:42:41.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.910+0000 7f8b61c00640 1 -- 172.21.15.47:0/1683868555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5c1a1b30 con 0x7f8b5c1038e0 2024-03-21T11:42:41.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.911+0000 7f8b61c00640 1 -- 172.21.15.47:0/1683868555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c1a2030 con 0x7f8b5c1038e0 2024-03-21T11:42:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.916+0000 7f8b6481f740 1 Processor -- start 2024-03-21T11:42:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.917+0000 7f8b6481f740 1 -- start start 2024-03-21T11:42:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.917+0000 7f8b6481f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782ce80 0x5633a782d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.917+0000 7f8b6481f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782d870 0x5633a7832370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.917+0000 7f8b6481f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a7834e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.917+0000 7f8b6481f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a7792250 con 0x5633a78329e0 2024-03-21T11:42:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.917+0000 7f8b6481f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a773aa30 con 0x5633a782ce80 2024-03-21T11:42:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.917+0000 7f8b6481f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a778e2f0 con 0x5633a782d870 2024-03-21T11:42:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.917+0000 7f8b5a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a7834e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.918+0000 7f8b5a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a7834e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43816/0 (socket says 172.21.15.47:43816) 2024-03-21T11:42:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.918+0000 7f8b5a000640 1 -- 172.21.15.47:0/3557858018 learned_addr learned my addr 172.21.15.47:0/3557858018 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.918+0000 7f8b5a000640 1 -- 172.21.15.47:0/3557858018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782d870 msgr2=0x5633a7832370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:42:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.918+0000 7f8b5a000640 1 --2- 172.21.15.47:0/3557858018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782d870 0x5633a7832370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.918+0000 7f8b5a000640 1 -- 172.21.15.47:0/3557858018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782ce80 msgr2=0x5633a782d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.918+0000 7f8b5a000640 1 --2- 172.21.15.47:0/3557858018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782ce80 0x5633a782d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.918+0000 7f8b5a000640 1 -- 172.21.15.47:0/3557858018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633a7725690 con 0x5633a78329e0 2024-03-21T11:42:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.918+0000 7f8b53400640 1 -- 172.21.15.47:0/1683868555 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8b4c0037a0 con 0x7f8b5c1038e0 2024-03-21T11:42:41.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.918+0000 7f8b5a000640 1 --2- 172.21.15.47:0/3557858018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a7834e70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009900 tx=0x7f8b48009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6eb4671c512c41 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:41.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.919+0000 7f8b47e00640 1 -- 172.21.15.47:0/3557858018 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4802cb80 con 0x5633a78329e0 2024-03-21T11:42:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.919+0000 7f8b47e00640 1 -- 172.21.15.47:0/3557858018 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b48005040 con 0x5633a78329e0 2024-03-21T11:42:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.919+0000 7f8b47e00640 1 -- 172.21.15.47:0/3557858018 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48004900 con 0x5633a78329e0 2024-03-21T11:42:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.919+0000 7f8b53400640 1 --2- 172.21.15.47:0/1683868555 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8b400a2fa0 0x7f8b400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.920+0000 7f8b53400640 1 -- 172.21.15.47:0/1683868555 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8b4c0cfad0 con 0x7f8b5c1038e0 2024-03-21T11:42:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.920+0000 7f8b6481f740 1 -- 172.21.15.47:0/3557858018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 msgr2=0x5633a7834e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.920+0000 7f8b6481f740 1 --2- 172.21.15.47:0/3557858018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a7834e70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009900 tx=0x7f8b48009560 comp rx=0 tx=0).stop 2024-03-21T11:42:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.920+0000 7f8b5b400640 1 --2- 172.21.15.47:0/1683868555 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8b400a2fa0 0x7f8b400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.920+0000 7f8b6481f740 1 -- 172.21.15.47:0/3557858018 shutdown_connections 2024-03-21T11:42:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.920+0000 7f8b6481f740 1 --2- 172.21.15.47:0/3557858018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a7834e70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.920+0000 7f8b6481f740 1 --2- 172.21.15.47:0/3557858018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782d870 0x5633a7832370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.920+0000 7f8b6481f740 1 --2- 172.21.15.47:0/3557858018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782ce80 0x5633a782d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.920+0000 7f8b6481f740 1 -- 172.21.15.47:0/3557858018 >> 172.21.15.47:0/3557858018 conn(0x5633a77f5200 msgr2=0x5633a78287f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.920+0000 7f8b6481f740 1 -- 172.21.15.47:0/3557858018 shutdown_connections 2024-03-21T11:42:41.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.921+0000 7f8b6481f740 1 -- 172.21.15.47:0/3557858018 wait complete. 2024-03-21T11:42:41.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.921+0000 7f8b5b400640 1 --2- 172.21.15.47:0/1683868555 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8b400a2fa0 0x7f8b400a5460 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8b540099a0 tx=0x7f8b54004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:41.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.921+0000 7f8b6481f740 1 -- start start 2024-03-21T11:42:41.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.921+0000 7f8b6481f740 1 Processor -- start 2024-03-21T11:42:41.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.922+0000 7f8b6481f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782ce80 0x5633a7836df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.922+0000 7f8b6481f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a782d870 0x5633a7837330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.922+0000 7f8b6481f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a78329e0 0x5633a783e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.922+0000 7f8b6481f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a7835930 con 0x5633a782d870 2024-03-21T11:42:41.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.922+0000 7f8b6481f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a774de80 con 0x5633a782ce80 2024-03-21T11:42:41.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.922+0000 7f8b6481f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633a7835ab0 con 0x5633a78329e0 2024-03-21T11:42:41.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.922+0000 7f8b6481f740 10 client.0 _ll_register_callbacks cb 0x5633a7760130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:41.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.922+0000 7f8b5a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a782d870 0x5633a7837330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.922+0000 7f8b5b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a78329e0 0x5633a783e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.922+0000 7f8b5a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a782d870 0x5633a7837330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43832/0 (socket says 172.21.15.47:43832) 2024-03-21T11:42:41.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.922+0000 7f8b5a000640 1 -- 172.21.15.47:0/2131701878 learned_addr learned my addr 172.21.15.47:0/2131701878 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:41.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.923+0000 7f8b5aa00640 1 --2- 172.21.15.47:0/2131701878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782ce80 0x5633a7836df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.923+0000 7f8b5b400640 1 -- 172.21.15.47:0/2131701878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782ce80 msgr2=0x5633a7836df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.923+0000 7f8b5b400640 1 --2- 172.21.15.47:0/2131701878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782ce80 0x5633a7836df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.923+0000 7f8b5b400640 1 -- 172.21.15.47:0/2131701878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a782d870 msgr2=0x5633a7837330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.923+0000 7f8b5b400640 1 --2- 172.21.15.47:0/2131701878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a782d870 0x5633a7837330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.923+0000 7f8b5b400640 1 -- 172.21.15.47:0/2131701878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633a75707d0 con 0x5633a78329e0 2024-03-21T11:42:41.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.924+0000 7f8b5b400640 1 --2- 172.21.15.47:0/2131701878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a78329e0 0x5633a783e370 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8b54005fd0 tx=0x7f8b54009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=153e4ecb865540c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:41.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.924+0000 7f8b44c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:42:41.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.924+0000 7f8b44c00640 1 -- 172.21.15.47:0/2131701878 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b54011e60 con 0x5633a78329e0 2024-03-21T11:42:41.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.924+0000 7f8b44c00640 1 -- 172.21.15.47:0/2131701878 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8b54004d20 con 0x5633a78329e0 2024-03-21T11:42:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.924+0000 7f8b6481f740 1 -- 172.21.15.47:0/2131701878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b48005040 con 0x5633a78329e0 2024-03-21T11:42:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.925+0000 7f8b44c00640 1 -- 172.21.15.47:0/2131701878 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b5402f050 con 0x5633a78329e0 2024-03-21T11:42:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.925+0000 7f8b6481f740 10 client.31801 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.925+0000 7f8b6481f740 1 -- 172.21.15.47:0/2131701878 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5633a74a63f0 con 0x5633a78329e0 2024-03-21T11:42:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.927+0000 7f8b44c00640 1 -- 172.21.15.47:0/2131701878 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b48005040 con 0x5633a78329e0 2024-03-21T11:42:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.927+0000 7f8b44c00640 1 -- 172.21.15.47:0/2131701878 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8b540055a0 con 0x5633a78329e0 2024-03-21T11:42:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.927+0000 7f8b6481f740 10 client.31801 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:41.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.927+0000 7f8b6481f740 10 client.31801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:41.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.927+0000 7f8b6481f740 20 client.31801 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:41.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.927+0000 7f8b6481f740 1 --2- 172.21.15.47:0/2131701878 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633a7842c90 0x5633a78450f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.927+0000 7f8b6481f740 4 client.31801 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:41.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.927+0000 7f8b6481f740 1 -- 172.21.15.47:0/2131701878 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633a779f790 con 0x5633a7842c90 2024-03-21T11:42:41.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.929+0000 7f8b5aa00640 1 --2- 172.21.15.47:0/2131701878 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633a7842c90 0x5633a78450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.930+0000 7f8b5aa00640 1 --2- 172.21.15.47:0/2131701878 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633a7842c90 0x5633a78450f0 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2001a6dd98de38a8 server_cookie=d235289bf728f934 in_seq=0 out_seq=0 2024-03-21T11:42:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.930+0000 7f8b44c00640 10 client.31801 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.934+0000 7f8b44c00640 1 -- 172.21.15.47:0/2131701878 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5633a779f790 con 0x5633a7842c90 2024-03-21T11:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.934+0000 7f8b44c00640 10 client.31801 handle_command_reply: tid=0 2024-03-21T11:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.934+0000 7f8b6481f740 1 client.31801 shutdown 2024-03-21T11:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.935+0000 7f8b6481f740 1 -- 172.21.15.47:0/2131701878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a78329e0 msgr2=0x5633a783e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.935+0000 7f8b6481f740 1 --2- 172.21.15.47:0/2131701878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a78329e0 0x5633a783e370 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8b54005fd0 tx=0x7f8b54009500 comp rx=0 tx=0).stop 2024-03-21T11:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.935+0000 7f8b6481f740 1 -- 172.21.15.47:0/2131701878 shutdown_connections 2024-03-21T11:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.935+0000 7f8b6481f740 1 --2- 172.21.15.47:0/2131701878 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633a7842c90 0x5633a78450f0 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.935+0000 7f8b6481f740 1 --2- 172.21.15.47:0/2131701878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a78329e0 0x5633a783e370 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.935+0000 7f8b6481f740 1 --2- 172.21.15.47:0/2131701878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a782d870 0x5633a7837330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.935+0000 7f8b6481f740 1 --2- 172.21.15.47:0/2131701878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782ce80 0x5633a7836df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.935+0000 7f8b44c00640 0 client.31801 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.935+0000 7f8b6481f740 1 -- 172.21.15.47:0/2131701878 >> 172.21.15.47:0/2131701878 conn(0x5633a77f5200 msgr2=0x5633a7827a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.935+0000 7f8b6481f740 1 -- 172.21.15.47:0/2131701878 shutdown_connections 2024-03-21T11:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.935+0000 7f8b6481f740 1 -- 172.21.15.47:0/2131701878 wait complete. 2024-03-21T11:42:41.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.935+0000 7f8b6481f740 20 client.31801 trim_cache size 0 max 16384 2024-03-21T11:42:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.956+0000 7f8b6481f740 1 Processor -- start 2024-03-21T11:42:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.956+0000 7f8b6481f740 1 -- start start 2024-03-21T11:42:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.957+0000 7f8b6481f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782ce80 0x5633a782ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.957+0000 7f8b6481f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782d870 0x5633a783e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.957+0000 7f8b6481f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a7827200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.957+0000 7f8b6481f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c1ac340 con 0x5633a78329e0 2024-03-21T11:42:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.957+0000 7f8b6481f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c1adcc0 con 0x5633a782d870 2024-03-21T11:42:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.957+0000 7f8b6481f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c1ac9a0 con 0x5633a782ce80 2024-03-21T11:42:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.957+0000 7f8b5b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a7827200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.957+0000 7f8b5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782ce80 0x5633a782ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.957+0000 7f8b5a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782d870 0x5633a783e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.957+0000 7f8b5a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782d870 0x5633a783e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60690/0 (socket says 172.21.15.47:60690) 2024-03-21T11:42:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.957+0000 7f8b5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782ce80 0x5633a782ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35262/0 (socket says 172.21.15.47:35262) 2024-03-21T11:42:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.957+0000 7f8b5a000640 1 -- 172.21.15.47:0/3900507793 learned_addr learned my addr 172.21.15.47:0/3900507793 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:41.959 INFO:tasks.workunit.client.0.smithi047.stdout: 10 147079 23.30 MB/sec execute 205 sec latency 265.143 ms 2024-03-21T11:42:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.958+0000 7f8b5a000640 1 -- 172.21.15.47:0/3900507793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782ce80 msgr2=0x5633a782ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.958+0000 7f8b5a000640 1 --2- 172.21.15.47:0/3900507793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782ce80 0x5633a782ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.958+0000 7f8b5a000640 1 -- 172.21.15.47:0/3900507793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 msgr2=0x5633a7827200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.958+0000 7f8b5a000640 1 --2- 172.21.15.47:0/3900507793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a7827200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.958+0000 7f8b5a000640 1 -- 172.21.15.47:0/3900507793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633a7832730 con 0x5633a782d870 2024-03-21T11:42:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.958+0000 7f8b5a000640 1 --2- 172.21.15.47:0/3900507793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782d870 0x5633a783e330 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8b48005920 tx=0x7f8b480056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aad26aa40d677bd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.958+0000 7f8b47400640 1 -- 172.21.15.47:0/3900507793 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4804c020 con 0x5633a782d870 2024-03-21T11:42:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.959+0000 7f8b47400640 1 -- 172.21.15.47:0/3900507793 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b4802ccc0 con 0x5633a782d870 2024-03-21T11:42:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.959+0000 7f8b47400640 1 -- 172.21.15.47:0/3900507793 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48034440 con 0x5633a782d870 2024-03-21T11:42:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.959+0000 7f8b6481f740 1 -- 172.21.15.47:0/3900507793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782d870 msgr2=0x5633a783e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.959+0000 7f8b6481f740 1 --2- 172.21.15.47:0/3900507793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782d870 0x5633a783e330 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8b48005920 tx=0x7f8b480056d0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.960+0000 7f8b6481f740 1 -- 172.21.15.47:0/3900507793 shutdown_connections 2024-03-21T11:42:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.960+0000 7f8b6481f740 1 --2- 172.21.15.47:0/3900507793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a7827200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.960+0000 7f8b6481f740 1 --2- 172.21.15.47:0/3900507793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782d870 0x5633a783e330 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.960+0000 7f8b6481f740 1 --2- 172.21.15.47:0/3900507793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782ce80 0x5633a782ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.960+0000 7f8b6481f740 1 -- 172.21.15.47:0/3900507793 >> 172.21.15.47:0/3900507793 conn(0x5633a77f5200 msgr2=0x5633a782e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.960+0000 7f8b6481f740 1 -- 172.21.15.47:0/3900507793 shutdown_connections 2024-03-21T11:42:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.960+0000 7f8b6481f740 1 -- 172.21.15.47:0/3900507793 wait complete. 2024-03-21T11:42:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.961+0000 7f8b6481f740 1 -- start start 2024-03-21T11:42:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.961+0000 7f8b6481f740 1 Processor -- start 2024-03-21T11:42:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b6481f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782ce80 0x5633a783c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b6481f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782d870 0x5633a783c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b6481f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a783d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b6481f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c1ad660 con 0x5633a78329e0 2024-03-21T11:42:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b6481f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c1ad000 con 0x5633a782d870 2024-03-21T11:42:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b6481f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c1aa500 con 0x5633a782ce80 2024-03-21T11:42:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b6481f740 10 client.0 _ll_register_callbacks cb 0x7f8b54009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782ce80 0x5633a783c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b5a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782d870 0x5633a783c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b5b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a783d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b5a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782d870 0x5633a783c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60700/0 (socket says 172.21.15.47:60700) 2024-03-21T11:42:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b5a000640 1 -- 172.21.15.47:0/831477765 learned_addr learned my addr 172.21.15.47:0/831477765 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b5a000640 1 -- 172.21.15.47:0/831477765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782ce80 msgr2=0x5633a783c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b5a000640 1 --2- 172.21.15.47:0/831477765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782ce80 0x5633a783c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b5a000640 1 -- 172.21.15.47:0/831477765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 msgr2=0x5633a783d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b5a000640 1 --2- 172.21.15.47:0/831477765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a783d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b5a000640 1 -- 172.21.15.47:0/831477765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633a74660f0 con 0x5633a782d870 2024-03-21T11:42:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.962+0000 7f8b5b400640 1 --2- 172.21.15.47:0/831477765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a783d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.963+0000 7f8b5a000640 1 --2- 172.21.15.47:0/831477765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782d870 0x5633a783c610 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8b4802c170 tx=0x7f8b4802c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec4c41d353e47d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.963+0000 7f8b44c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.963+0000 7f8b44c00640 1 -- 172.21.15.47:0/831477765 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48057020 con 0x5633a782d870 2024-03-21T11:42:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.963+0000 7f8b44c00640 1 -- 172.21.15.47:0/831477765 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8b48004210 con 0x5633a782d870 2024-03-21T11:42:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.963+0000 7f8b6481f740 1 -- 172.21.15.47:0/831477765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b4802ccc0 con 0x5633a782d870 2024-03-21T11:42:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.964+0000 7f8b44c00640 1 -- 172.21.15.47:0/831477765 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b4805f050 con 0x5633a782d870 2024-03-21T11:42:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.964+0000 7f8b6481f740 10 client.31847 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.964+0000 7f8b6481f740 1 -- 172.21.15.47:0/831477765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5633a78a7220 con 0x5633a782d870 2024-03-21T11:42:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.968+0000 7f8b44c00640 1 -- 172.21.15.47:0/831477765 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b4802ccc0 con 0x5633a782d870 2024-03-21T11:42:41.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.968+0000 7f8b44c00640 1 -- 172.21.15.47:0/831477765 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8b48034560 con 0x5633a782d870 2024-03-21T11:42:41.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.968+0000 7f8b6481f740 10 client.31847 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:41.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.968+0000 7f8b6481f740 10 client.31847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:41.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.968+0000 7f8b6481f740 20 client.31847 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:41.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.968+0000 7f8b6481f740 1 --2- 172.21.15.47:0/831477765 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633a78a7790 0x5633a78a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:41.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.968+0000 7f8b6481f740 4 client.31847 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:42:41.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.968+0000 7f8b6481f740 1 -- 172.21.15.47:0/831477765 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5633a7799ae0 con 0x5633a78a7790 2024-03-21T11:42:41.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.970+0000 7f8b5aa00640 1 --2- 172.21.15.47:0/831477765 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633a78a7790 0x5633a78a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:41.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.971+0000 7f8b5aa00640 1 --2- 172.21.15.47:0/831477765 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633a78a7790 0x5633a78a9b80 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e802f0c61212ab5b server_cookie=70481ecb5c9cc4be in_seq=0 out_seq=0 2024-03-21T11:42:41.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.971+0000 7f8b44c00640 10 client.31847 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.978+0000 7f8b44c00640 1 -- 172.21.15.47:0/831477765 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5633a7799ae0 con 0x5633a78a7790 2024-03-21T11:42:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.978+0000 7f8b44c00640 10 client.31847 handle_command_reply: tid=0 2024-03-21T11:42:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.978+0000 7f8b6481f740 1 client.31847 shutdown 2024-03-21T11:42:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.979+0000 7f8b6481f740 1 -- 172.21.15.47:0/831477765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782d870 msgr2=0x5633a783c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.979+0000 7f8b6481f740 1 --2- 172.21.15.47:0/831477765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782d870 0x5633a783c610 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8b4802c170 tx=0x7f8b4802c330 comp rx=0 tx=0).stop 2024-03-21T11:42:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.979+0000 7f8b6481f740 1 -- 172.21.15.47:0/831477765 shutdown_connections 2024-03-21T11:42:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.979+0000 7f8b6481f740 1 --2- 172.21.15.47:0/831477765 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633a78a7790 0x5633a78a9b80 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.979+0000 7f8b6481f740 1 --2- 172.21.15.47:0/831477765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633a78329e0 0x5633a783d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.979+0000 7f8b6481f740 1 --2- 172.21.15.47:0/831477765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633a782d870 0x5633a783c610 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.979+0000 7f8b6481f740 1 --2- 172.21.15.47:0/831477765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633a782ce80 0x5633a783c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.979+0000 7f8b6481f740 1 -- 172.21.15.47:0/831477765 >> 172.21.15.47:0/831477765 conn(0x5633a77f5200 msgr2=0x5633a782eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.979+0000 7f8b44c00640 0 client.31847 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.979+0000 7f8b6481f740 1 -- 172.21.15.47:0/831477765 shutdown_connections 2024-03-21T11:42:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.979+0000 7f8b6481f740 1 -- 172.21.15.47:0/831477765 wait complete. 2024-03-21T11:42:41.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.979+0000 7f8b6481f740 20 client.31847 trim_cache size 0 max 16384 2024-03-21T11:42:41.981 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:42:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (115 inodes in the stack)", 2024-03-21T11:42:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:42:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "bcb7aefa-b809-4b87-b178-6f11dddeeef4": { 2024-03-21T11:42:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:42:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "bcb7aefa-b809-4b87-b178-6f11dddeeef4", 2024-03-21T11:42:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:42:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:42:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:42:41.982 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:42:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.980+0000 7f8b61c00640 1 -- 172.21.15.47:0/1683868555 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8b400a2fa0 msgr2=0x7f8b400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.980+0000 7f8b61c00640 1 --2- 172.21.15.47:0/1683868555 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8b400a2fa0 0x7f8b400a5460 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8b540099a0 tx=0x7f8b54004a40 comp rx=0 tx=0).stop 2024-03-21T11:42:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.980+0000 7f8b61c00640 1 -- 172.21.15.47:0/1683868555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5c1038e0 msgr2=0x7f8b5c199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.980+0000 7f8b61c00640 1 --2- 172.21.15.47:0/1683868555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5c1038e0 0x7f8b5c199c30 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00e9c0 tx=0x7f8b4c00edc0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.980+0000 7f8b61c00640 1 -- 172.21.15.47:0/1683868555 shutdown_connections 2024-03-21T11:42:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.980+0000 7f8b61c00640 1 --2- 172.21.15.47:0/1683868555 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8b400a2fa0 0x7f8b400a5460 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.980+0000 7f8b61c00640 1 --2- 172.21.15.47:0/1683868555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b5c105490 0x7f8b5c1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.980+0000 7f8b61c00640 1 --2- 172.21.15.47:0/1683868555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b5c104ad0 0x7f8b5c19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.980+0000 7f8b61c00640 1 --2- 172.21.15.47:0/1683868555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5c1038e0 0x7f8b5c199c30 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.980+0000 7f8b61c00640 1 -- 172.21.15.47:0/1683868555 >> 172.21.15.47:0/1683868555 conn(0x7f8b5c0fee50 msgr2=0x7f8b5c107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.981+0000 7f8b61c00640 1 -- 172.21.15.47:0/1683868555 shutdown_connections 2024-03-21T11:42:41.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:42:41.981+0000 7f8b61c00640 1 -- 172.21.15.47:0/1683868555 wait complete. 2024-03-21T11:42:42.005 INFO:tasks.cephfs.filesystem:scrub status for tag:bcb7aefa-b809-4b87-b178-6f11dddeeef4 - {'path': '/', 'tag': 'bcb7aefa-b809-4b87-b178-6f11dddeeef4', 'options': 'recursive,force'} 2024-03-21T11:42:42.958 INFO:tasks.workunit.client.0.smithi047.stdout: 10 147367 23.28 MB/sec execute 206 sec latency 150.668 ms 2024-03-21T11:42:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:42:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: pgmap v981: 129 pgs: 129 active+clean; 525 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.8 MiB/s wr, 678 op/s 2024-03-21T11:42:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:43.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:43.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:42:43.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:42 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:42:43.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:43.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: pgmap v981: 129 pgs: 129 active+clean; 525 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.8 MiB/s wr, 678 op/s 2024-03-21T11:42:43.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:43.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:43.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:43.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:43.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:43.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:43.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:42:43.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:42 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:42:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:43.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: pgmap v981: 129 pgs: 129 active+clean; 525 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.8 MiB/s wr, 678 op/s 2024-03-21T11:42:43.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:43.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:43.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:43.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:43.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:43.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:43.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:42:43.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:42 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:42:43.958 INFO:tasks.workunit.client.0.smithi047.stdout: 10 147977 23.35 MB/sec execute 207 sec latency 58.974 ms 2024-03-21T11:42:44.958 INFO:tasks.workunit.client.0.smithi047.stdout: 10 148614 23.34 MB/sec execute 208 sec latency 81.186 ms 2024-03-21T11:42:45.042 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:44 smithi102 ceph-mon[36748]: pgmap v982: 129 pgs: 129 active+clean; 527 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-03-21T11:42:45.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:44 smithi047 ceph-mon[31351]: pgmap v982: 129 pgs: 129 active+clean; 527 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-03-21T11:42:45.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:44 smithi100 ceph-mon[36245]: pgmap v982: 129 pgs: 129 active+clean; 527 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-03-21T11:42:45.959 INFO:tasks.workunit.client.0.smithi047.stdout: 10 148968 23.34 MB/sec execute 209 sec latency 144.839 ms 2024-03-21T11:42:46.959 INFO:tasks.workunit.client.0.smithi047.stdout: 10 149300 23.34 MB/sec execute 210 sec latency 441.511 ms 2024-03-21T11:42:47.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:46 smithi047 ceph-mon[31351]: pgmap v983: 129 pgs: 129 active+clean; 528 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.4 MiB/s wr, 575 op/s 2024-03-21T11:42:47.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:46 smithi100 ceph-mon[36245]: pgmap v983: 129 pgs: 129 active+clean; 528 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.4 MiB/s wr, 575 op/s 2024-03-21T11:42:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:46 smithi102 ceph-mon[36748]: pgmap v983: 129 pgs: 129 active+clean; 528 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.4 MiB/s wr, 575 op/s 2024-03-21T11:42:47.959 INFO:tasks.workunit.client.0.smithi047.stdout: 10 149634 23.31 MB/sec execute 211 sec latency 232.906 ms 2024-03-21T11:42:48.959 INFO:tasks.workunit.client.0.smithi047.stdout: 10 150189 23.37 MB/sec execute 212 sec latency 68.742 ms 2024-03-21T11:42:49.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:48 smithi047 ceph-mon[31351]: pgmap v984: 129 pgs: 129 active+clean; 533 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.2 MiB/s wr, 690 op/s 2024-03-21T11:42:49.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:48 smithi100 ceph-mon[36245]: pgmap v984: 129 pgs: 129 active+clean; 533 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.2 MiB/s wr, 690 op/s 2024-03-21T11:42:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:48 smithi102 ceph-mon[36748]: pgmap v984: 129 pgs: 129 active+clean; 533 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.2 MiB/s wr, 690 op/s 2024-03-21T11:42:49.959 INFO:tasks.workunit.client.0.smithi047.stdout: 10 150671 23.37 MB/sec execute 213 sec latency 54.285 ms 2024-03-21T11:42:50.964 INFO:tasks.workunit.client.0.smithi047.stdout: 10 151052 23.34 MB/sec execute 214 sec latency 107.516 ms 2024-03-21T11:42:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.004+0000 7ff4d6600640 1 -- 172.21.15.47:0/4016817982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0102160 msgr2=0x7ff4d0104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.004+0000 7ff4d6600640 1 --2- 172.21.15.47:0/4016817982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0102160 0x7ff4d0104550 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009ca0 tx=0x7ff4c802c120 comp rx=0 tx=0).stop 2024-03-21T11:42:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.005+0000 7ff4d6600640 1 -- 172.21.15.47:0/4016817982 shutdown_connections 2024-03-21T11:42:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.005+0000 7ff4d6600640 1 --2- 172.21.15.47:0/4016817982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d01073c0 0x7ff4d0109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.005+0000 7ff4d6600640 1 --2- 172.21.15.47:0/4016817982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4d0104a90 0x7ff4d0106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.005+0000 7ff4d6600640 1 --2- 172.21.15.47:0/4016817982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0102160 0x7ff4d0104550 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.005+0000 7ff4d6600640 1 -- 172.21.15.47:0/4016817982 >> 172.21.15.47:0/4016817982 conn(0x7ff4d00fbb50 msgr2=0x7ff4d00fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.005+0000 7ff4d6600640 1 -- 172.21.15.47:0/4016817982 shutdown_connections 2024-03-21T11:42:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.006+0000 7ff4d6600640 1 -- 172.21.15.47:0/4016817982 wait complete. 2024-03-21T11:42:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.008+0000 7ff4d6600640 1 Processor -- start 2024-03-21T11:42:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.009+0000 7ff4d6600640 1 -- start start 2024-03-21T11:42:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.009+0000 7ff4d6600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d0102160 0x7ff4d0101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.009+0000 7ff4d6600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0104a90 0x7ff4d0100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.009+0000 7ff4d6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4d01073c0 0x7ff4d0100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.009+0000 7ff4d6600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d010d3d0 con 0x7ff4d0102160 2024-03-21T11:42:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.009+0000 7ff4d6600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d010d250 con 0x7ff4d0104a90 2024-03-21T11:42:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.009+0000 7ff4d6600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d010d550 con 0x7ff4d01073c0 2024-03-21T11:42:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.010+0000 7ff4cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0104a90 0x7ff4d0100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.010+0000 7ff4cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0104a90 0x7ff4d0100110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36102/0 (socket says 172.21.15.47:36102) 2024-03-21T11:42:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.011+0000 7ff4cfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4d01073c0 0x7ff4d0100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.011+0000 7ff4cea00640 1 -- 172.21.15.47:0/903337532 learned_addr learned my addr 172.21.15.47:0/903337532 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.011+0000 7ff4cf400640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d0102160 0x7ff4d0101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.012+0000 7ff4cea00640 1 -- 172.21.15.47:0/903337532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4d01073c0 msgr2=0x7ff4d0100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.013+0000 7ff4cea00640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4d01073c0 0x7ff4d0100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.013+0000 7ff4cea00640 1 -- 172.21.15.47:0/903337532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d0102160 msgr2=0x7ff4d0101ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.013+0000 7ff4cea00640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d0102160 0x7ff4d0101ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.013+0000 7ff4cea00640 1 -- 172.21.15.47:0/903337532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d0100ee0 con 0x7ff4d0104a90 2024-03-21T11:42:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.014+0000 7ff4cf400640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d0102160 0x7ff4d0101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:42:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.014+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4d01073c0 0x7ff4d0100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:42:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.014+0000 7ff4cea00640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0104a90 0x7ff4d0100110 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800e9a0 tx=0x7ff4b800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.015+0000 7ff4c7e00640 1 -- 172.21.15.47:0/903337532 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b800cd80 con 0x7ff4d0104a90 2024-03-21T11:42:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.015+0000 7ff4c7e00640 1 -- 172.21.15.47:0/903337532 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4b80035f0 con 0x7ff4d0104a90 2024-03-21T11:42:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.015+0000 7ff4c7e00640 1 -- 172.21.15.47:0/903337532 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8010670 con 0x7ff4d0104a90 2024-03-21T11:42:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.015+0000 7ff4d6600640 1 -- 172.21.15.47:0/903337532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d0197670 con 0x7ff4d0104a90 2024-03-21T11:42:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.015+0000 7ff4d6600640 1 -- 172.21.15.47:0/903337532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d0197c90 con 0x7ff4d0104a90 2024-03-21T11:42:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.024+0000 7ff4c7e00640 1 -- 172.21.15.47:0/903337532 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff4b8010810 con 0x7ff4d0104a90 2024-03-21T11:42:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.027+0000 7ff4d93cc740 1 Processor -- start 2024-03-21T11:42:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.027+0000 7ff4d93cc740 1 -- start start 2024-03-21T11:42:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.027+0000 7ff4d93cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dc3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.027+0000 7ff4d93cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dc86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.027+0000 7ff4d93cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc8d50 0x55b9a6dcb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.027+0000 7ff4d93cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a6d753b0 con 0x55b9a6dc3be0 2024-03-21T11:42:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.027+0000 7ff4d93cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a6d17830 con 0x55b9a6dc31f0 2024-03-21T11:42:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.027+0000 7ff4d93cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a6cc05d0 con 0x55b9a6dc8d50 2024-03-21T11:42:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.028+0000 7ff4cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dc3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.028+0000 7ff4c7e00640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff4b40a3040 0x7ff4b40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.028+0000 7ff4cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dc3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36118/0 (socket says 172.21.15.47:36118) 2024-03-21T11:42:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.028+0000 7ff4cfe00640 1 -- 172.21.15.47:0/3015115653 learned_addr learned my addr 172.21.15.47:0/3015115653 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.028+0000 7ff4c7e00640 1 -- 172.21.15.47:0/903337532 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff4b80cfac0 con 0x7ff4d0104a90 2024-03-21T11:42:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.028+0000 7ff4cf400640 1 --2- 172.21.15.47:0/3015115653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dc86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.029+0000 7ff4cfe00640 1 -- 172.21.15.47:0/3015115653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc8d50 msgr2=0x55b9a6dcb1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:42:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.029+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/3015115653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc8d50 0x55b9a6dcb1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.029+0000 7ff4cfe00640 1 -- 172.21.15.47:0/3015115653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc3be0 msgr2=0x55b9a6dc86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.029+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/3015115653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dc86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.029+0000 7ff4cfe00640 1 -- 172.21.15.47:0/3015115653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9a6cd82c0 con 0x55b9a6dc31f0 2024-03-21T11:42:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.029+0000 7ff4cf400640 1 --2- 172.21.15.47:0/3015115653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dc86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:42:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.029+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff4b40a3040 0x7ff4b40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.033+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/3015115653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dc3610 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000aaf0 tx=0x7ff4c000e170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e8ae0f76545159e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.033+0000 7ff4c4c00640 1 -- 172.21.15.47:0/3015115653 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c000f040 con 0x55b9a6dc31f0 2024-03-21T11:42:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.033+0000 7ff4c4c00640 1 -- 172.21.15.47:0/3015115653 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4c00138f0 con 0x55b9a6dc31f0 2024-03-21T11:42:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.033+0000 7ff4c4c00640 1 -- 172.21.15.47:0/3015115653 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c001dac0 con 0x55b9a6dc31f0 2024-03-21T11:42:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.035+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff4b40a3040 0x7ff4b40a5500 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff4c001e000 tx=0x7ff4c0012740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.035+0000 7ff4d93cc740 1 -- 172.21.15.47:0/3015115653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc31f0 msgr2=0x55b9a6dc3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.035+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/3015115653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dc3610 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000aaf0 tx=0x7ff4c000e170 comp rx=0 tx=0).stop 2024-03-21T11:42:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.037+0000 7ff4d93cc740 1 -- 172.21.15.47:0/3015115653 shutdown_connections 2024-03-21T11:42:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.037+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/3015115653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc8d50 0x55b9a6dcb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.037+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/3015115653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dc86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.037+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/3015115653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dc3610 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.037+0000 7ff4d93cc740 1 -- 172.21.15.47:0/3015115653 >> 172.21.15.47:0/3015115653 conn(0x55b9a6dad750 msgr2=0x55b9a6dadb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.037+0000 7ff4d93cc740 1 -- 172.21.15.47:0/3015115653 shutdown_connections 2024-03-21T11:42:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.037+0000 7ff4d93cc740 1 -- 172.21.15.47:0/3015115653 wait complete. 2024-03-21T11:42:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.038+0000 7ff4d93cc740 1 -- start start 2024-03-21T11:42:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.038+0000 7ff4d93cc740 1 Processor -- start 2024-03-21T11:42:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.039+0000 7ff4d93cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dcd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.040+0000 7ff4d93cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dcd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.040+0000 7ff4cf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dcd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.040+0000 7ff4cf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dcd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51654/0 (socket says 172.21.15.47:51654) 2024-03-21T11:42:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.041+0000 7ff4cf400640 1 -- 172.21.15.47:0/2290418064 learned_addr learned my addr 172.21.15.47:0/2290418064 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.041+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2290418064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc8d50 0x55b9a6dd4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.041+0000 7ff4cea00640 1 --2- 172.21.15.47:0/2290418064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dcd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.042+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2290418064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a6cd3a20 con 0x55b9a6dc8d50 2024-03-21T11:42:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.042+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2290418064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a6d138d0 con 0x55b9a6dc3be0 2024-03-21T11:42:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.042+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2290418064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a6dcbd00 con 0x55b9a6dc31f0 2024-03-21T11:42:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.042+0000 7ff4d93cc740 10 client.0 _ll_register_callbacks cb 0x55b9a6d101c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.042+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/2290418064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc8d50 0x55b9a6dd4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.042+0000 7ff4cf400640 1 -- 172.21.15.47:0/2290418064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc3be0 msgr2=0x55b9a6dcd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.042+0000 7ff4cf400640 1 --2- 172.21.15.47:0/2290418064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dcd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.042+0000 7ff4cf400640 1 -- 172.21.15.47:0/2290418064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc8d50 msgr2=0x55b9a6dd4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.043+0000 7ff4cf400640 1 --2- 172.21.15.47:0/2290418064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc8d50 0x55b9a6dd4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.043+0000 7ff4cf400640 1 -- 172.21.15.47:0/2290418064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9a6a2c210 con 0x55b9a6dc31f0 2024-03-21T11:42:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.043+0000 7ff4cf400640 1 --2- 172.21.15.47:0/2290418064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dcd3d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8000f80 tx=0x7ff4c8009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaf5cc3c3293807d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.043+0000 7ff4bd600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:42:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.043+0000 7ff4bd600640 1 -- 172.21.15.47:0/2290418064 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8045070 con 0x55b9a6dc31f0 2024-03-21T11:42:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.044+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2290418064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4c00138f0 con 0x55b9a6dc31f0 2024-03-21T11:42:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.044+0000 7ff4bd600640 1 -- 172.21.15.47:0/2290418064 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff4c80347f0 con 0x55b9a6dc31f0 2024-03-21T11:42:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.044+0000 7ff4bd600640 1 -- 172.21.15.47:0/2290418064 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c8051050 con 0x55b9a6dc31f0 2024-03-21T11:42:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.044+0000 7ff4d93cc740 10 client.31813 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.044+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2290418064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b9a6af8620 con 0x55b9a6dc31f0 2024-03-21T11:42:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.045+0000 7ff4bd600640 1 -- 172.21.15.47:0/2290418064 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4c803f450 con 0x55b9a6dc31f0 2024-03-21T11:42:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.046+0000 7ff4bd600640 1 -- 172.21.15.47:0/2290418064 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff4c803e3b0 con 0x55b9a6dc31f0 2024-03-21T11:42:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.046+0000 7ff4d93cc740 10 client.31813 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.046+0000 7ff4d93cc740 10 client.31813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.046+0000 7ff4d93cc740 20 client.31813 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.046+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2290418064 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b9a6dd9020 0x55b9a6ddb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.046+0000 7ff4d93cc740 4 client.31813 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:42:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.046+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2290418064 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4c00138f0 con 0x55b9a6dd9020 2024-03-21T11:42:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.050+0000 7ff4cea00640 1 --2- 172.21.15.47:0/2290418064 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b9a6dd9020 0x55b9a6ddb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.055+0000 7ff4bd600640 10 client.31813 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.055+0000 7ff4cea00640 1 --2- 172.21.15.47:0/2290418064 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b9a6dd9020 0x55b9a6ddb410 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37af9766f33c354 server_cookie=7be00857f30938bf in_seq=0 out_seq=0 2024-03-21T11:42:51.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:50 smithi047 ceph-mon[31351]: pgmap v985: 129 pgs: 129 active+clean; 535 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 642 op/s 2024-03-21T11:42:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.069+0000 7ff4bd600640 1 -- 172.21.15.47:0/2290418064 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7ff4c00138f0 con 0x55b9a6dd9020 2024-03-21T11:42:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.069+0000 7ff4bd600640 10 client.31813 handle_command_reply: tid=0 2024-03-21T11:42:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.069+0000 7ff4d93cc740 1 client.31813 shutdown 2024-03-21T11:42:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.070+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2290418064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc31f0 msgr2=0x55b9a6dcd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.070+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2290418064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dcd3d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8000f80 tx=0x7ff4c8009960 comp rx=0 tx=0).stop 2024-03-21T11:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.070+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2290418064 shutdown_connections 2024-03-21T11:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.070+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2290418064 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b9a6dd9020 0x55b9a6ddb410 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.070+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2290418064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc8d50 0x55b9a6dd4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.070+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2290418064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dcd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.070+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2290418064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dcd3d0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.070+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2290418064 >> 172.21.15.47:0/2290418064 conn(0x55b9a6dad750 msgr2=0x55b9a6dac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.070+0000 7ff4bd600640 0 client.31813 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.071+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2290418064 shutdown_connections 2024-03-21T11:42:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.071+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2290418064 wait complete. 2024-03-21T11:42:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.071+0000 7ff4d93cc740 20 client.31813 trim_cache size 0 max 16384 2024-03-21T11:42:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.111+0000 7ff4d93cc740 1 Processor -- start 2024-03-21T11:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.111+0000 7ff4d93cc740 1 -- start start 2024-03-21T11:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.111+0000 7ff4d93cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc31f0 0x55b9a6da85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.111+0000 7ff4d93cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dd4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.112+0000 7ff4d93cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc8d50 0x55b9a6db10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.112+0000 7ff4d93cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0071580 con 0x55b9a6dc8d50 2024-03-21T11:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.112+0000 7ff4d93cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0072f00 con 0x55b9a6dc31f0 2024-03-21T11:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.112+0000 7ff4d93cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0071be0 con 0x55b9a6dc3be0 2024-03-21T11:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.112+0000 7ff4cf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc31f0 0x55b9a6da85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.112+0000 7ff4cf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc31f0 0x55b9a6da85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36132/0 (socket says 172.21.15.47:36132) 2024-03-21T11:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.112+0000 7ff4cf400640 1 -- 172.21.15.47:0/2184995202 learned_addr learned my addr 172.21.15.47:0/2184995202 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.113+0000 7ff4cea00640 1 --2- 172.21.15.47:0/2184995202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dd4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.113+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/2184995202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc8d50 0x55b9a6db10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.113+0000 7ff4cfe00640 1 -- 172.21.15.47:0/2184995202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc3be0 msgr2=0x55b9a6dd4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.113+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/2184995202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dd4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.113+0000 7ff4cfe00640 1 -- 172.21.15.47:0/2184995202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc31f0 msgr2=0x55b9a6da85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.113+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/2184995202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc31f0 0x55b9a6da85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.113+0000 7ff4cfe00640 1 -- 172.21.15.47:0/2184995202 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9a6dae1c0 con 0x55b9a6dc8d50 2024-03-21T11:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.114+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/2184995202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc8d50 0x55b9a6db10c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0004700 tx=0x7ff4c000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a0ab3b0e11fd0c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.117+0000 7ff4bfe00640 1 -- 172.21.15.47:0/2184995202 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0013410 con 0x55b9a6dc8d50 2024-03-21T11:42:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.117+0000 7ff4bfe00640 1 -- 172.21.15.47:0/2184995202 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4c00135b0 con 0x55b9a6dc8d50 2024-03-21T11:42:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.117+0000 7ff4bfe00640 1 -- 172.21.15.47:0/2184995202 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c00278c0 con 0x55b9a6dc8d50 2024-03-21T11:42:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.118+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2184995202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc8d50 msgr2=0x55b9a6db10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.118+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2184995202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc8d50 0x55b9a6db10c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0004700 tx=0x7ff4c000d000 comp rx=0 tx=0).stop 2024-03-21T11:42:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.121+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2184995202 shutdown_connections 2024-03-21T11:42:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.121+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2184995202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc8d50 0x55b9a6db10c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.121+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2184995202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dd4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.121+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2184995202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc31f0 0x55b9a6da85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.121+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2184995202 >> 172.21.15.47:0/2184995202 conn(0x55b9a6dad750 msgr2=0x55b9a6db09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.122+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2184995202 shutdown_connections 2024-03-21T11:42:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.122+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2184995202 wait complete. 2024-03-21T11:42:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.123+0000 7ff4d93cc740 1 -- start start 2024-03-21T11:42:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.123+0000 7ff4d93cc740 1 Processor -- start 2024-03-21T11:42:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.124+0000 7ff4d93cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dd1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.124+0000 7ff4d93cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dd2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.124+0000 7ff4d93cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc8d50 0x55b9a6dd3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.125+0000 7ff4d93cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d00728a0 con 0x55b9a6dc31f0 2024-03-21T11:42:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.125+0000 7ff4d93cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0072240 con 0x55b9a6dc8d50 2024-03-21T11:42:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.125+0000 7ff4d93cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d006f740 con 0x55b9a6dc3be0 2024-03-21T11:42:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.125+0000 7ff4cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc8d50 0x55b9a6dd3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.125+0000 7ff4cfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc8d50 0x55b9a6dd3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36144/0 (socket says 172.21.15.47:36144) 2024-03-21T11:42:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.125+0000 7ff4cfe00640 1 -- 172.21.15.47:0/2197599915 learned_addr learned my addr 172.21.15.47:0/2197599915 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:42:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.125+0000 7ff4d93cc740 10 client.0 _ll_register_callbacks cb 0x7ff4b8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:42:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.125+0000 7ff4cea00640 1 --2- 172.21.15.47:0/2197599915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dd2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.127 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.126+0000 7ff4cfe00640 1 -- 172.21.15.47:0/2197599915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc3be0 msgr2=0x55b9a6dd2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.126+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/2197599915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dd2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.126+0000 7ff4cfe00640 1 -- 172.21.15.47:0/2197599915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc31f0 msgr2=0x55b9a6dd1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:42:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.126+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/2197599915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dd1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.126+0000 7ff4cfe00640 1 -- 172.21.15.47:0/2197599915 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9a6dd3950 con 0x55b9a6dc8d50 2024-03-21T11:42:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.128+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/2197599915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc8d50 0x55b9a6dd3360 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0015010 tx=0x7ff4c000ea60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dcc8beee8bd78f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:42:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.128+0000 7ff4bd600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:42:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.129+0000 7ff4bd600640 1 -- 172.21.15.47:0/2197599915 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0013410 con 0x55b9a6dc8d50 2024-03-21T11:42:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.130+0000 7ff4bd600640 1 -- 172.21.15.47:0/2197599915 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff4c0027a50 con 0x55b9a6dc8d50 2024-03-21T11:42:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.130+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2197599915 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4c00135b0 con 0x55b9a6dc8d50 2024-03-21T11:42:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.130+0000 7ff4bd600640 1 -- 172.21.15.47:0/2197599915 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c0013df0 con 0x55b9a6dc8d50 2024-03-21T11:42:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.130+0000 7ff4d93cc740 10 client.31865 fetch_fsmap learned FSMap version 40 2024-03-21T11:42:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.130+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2197599915 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b9a6dd4c60 con 0x55b9a6dc8d50 2024-03-21T11:42:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.133+0000 7ff4bd600640 1 -- 172.21.15.47:0/2197599915 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4c00135b0 con 0x55b9a6dc8d50 2024-03-21T11:42:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.133+0000 7ff4bd600640 1 -- 172.21.15.47:0/2197599915 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff4c0032070 con 0x55b9a6dc8d50 2024-03-21T11:42:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.133+0000 7ff4d93cc740 10 client.31865 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:42:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.133+0000 7ff4d93cc740 10 client.31865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:42:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.133+0000 7ff4d93cc740 20 client.31865 populate_metadata read hostname 'smithi047' 2024-03-21T11:42:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.134+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2197599915 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b9a6e34da0 0x55b9a6e371c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:42:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.134+0000 7ff4d93cc740 4 client.31865 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:42:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.134+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2197599915 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b9a6d24d70 con 0x55b9a6e34da0 2024-03-21T11:42:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.135+0000 7ff4cf400640 1 --2- 172.21.15.47:0/2197599915 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b9a6e34da0 0x55b9a6e371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:42:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.136+0000 7ff4cf400640 1 --2- 172.21.15.47:0/2197599915 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b9a6e34da0 0x55b9a6e371c0 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f2b3f9b440fb66 server_cookie=4ccf2a8bb6a90435 in_seq=0 out_seq=0 2024-03-21T11:42:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:51.136+0000 7ff4bd600640 10 client.31865 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:51.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:50 smithi100 ceph-mon[36245]: pgmap v985: 129 pgs: 129 active+clean; 535 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 642 op/s 2024-03-21T11:42:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:50 smithi102 ceph-mon[36748]: pgmap v985: 129 pgs: 129 active+clean; 535 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.7 MiB/s wr, 642 op/s 2024-03-21T11:42:51.960 INFO:tasks.workunit.client.0.smithi047.stdout: 10 151164 23.27 MB/sec execute 215 sec latency 471.264 ms 2024-03-21T11:42:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4bd600640 1 -- 172.21.15.47:0/2197599915 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b9a6d24d70 con 0x55b9a6e34da0 2024-03-21T11:42:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4bd600640 10 client.31865 handle_command_reply: tid=0 2024-03-21T11:42:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4d93cc740 1 client.31865 shutdown 2024-03-21T11:42:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2197599915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc8d50 msgr2=0x55b9a6dd3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2197599915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc8d50 0x55b9a6dd3360 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0015010 tx=0x7ff4c000ea60 comp rx=0 tx=0).stop 2024-03-21T11:42:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2197599915 shutdown_connections 2024-03-21T11:42:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2197599915 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55b9a6e34da0 0x55b9a6e371c0 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2197599915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b9a6dc8d50 0x55b9a6dd3360 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2197599915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b9a6dc3be0 0x55b9a6dd2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4d93cc740 1 --2- 172.21.15.47:0/2197599915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b9a6dc31f0 0x55b9a6dd1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4bd600640 0 client.31865 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:42:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2197599915 >> 172.21.15.47:0/2197599915 conn(0x55b9a6dad750 msgr2=0x55b9a6db1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2197599915 shutdown_connections 2024-03-21T11:42:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4d93cc740 1 -- 172.21.15.47:0/2197599915 wait complete. 2024-03-21T11:42:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.587+0000 7ff4d93cc740 20 client.31865 trim_cache size 0 max 16384 2024-03-21T11:42:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:42:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:42:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:42:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:42:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.588+0000 7ff4d6600640 1 -- 172.21.15.47:0/903337532 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff4b40a3040 msgr2=0x7ff4b40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.588+0000 7ff4d6600640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff4b40a3040 0x7ff4b40a5500 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff4c001e000 tx=0x7ff4c0012740 comp rx=0 tx=0).stop 2024-03-21T11:42:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.588+0000 7ff4d6600640 1 -- 172.21.15.47:0/903337532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0104a90 msgr2=0x7ff4d0100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:42:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.588+0000 7ff4d6600640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0104a90 0x7ff4d0100110 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800e9a0 tx=0x7ff4b800ee60 comp rx=0 tx=0).stop 2024-03-21T11:42:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.588+0000 7ff4d6600640 1 -- 172.21.15.47:0/903337532 shutdown_connections 2024-03-21T11:42:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.588+0000 7ff4d6600640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff4b40a3040 0x7ff4b40a5500 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.588+0000 7ff4d6600640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4d01073c0 0x7ff4d0100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.588+0000 7ff4d6600640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0104a90 0x7ff4d0100110 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.588+0000 7ff4d6600640 1 --2- 172.21.15.47:0/903337532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d0102160 0x7ff4d0101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:42:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.588+0000 7ff4d6600640 1 -- 172.21.15.47:0/903337532 >> 172.21.15.47:0/903337532 conn(0x7ff4d00fbb50 msgr2=0x7ff4d0102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:42:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.588+0000 7ff4d6600640 1 -- 172.21.15.47:0/903337532 shutdown_connections 2024-03-21T11:42:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:42:52.588+0000 7ff4d6600640 1 -- 172.21.15.47:0/903337532 wait complete. 2024-03-21T11:42:52.961 INFO:tasks.workunit.client.0.smithi047.stdout: 10 151322 23.21 MB/sec execute 216 sec latency 580.498 ms 2024-03-21T11:42:53.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:52 smithi047 ceph-mon[31351]: pgmap v986: 129 pgs: 129 active+clean; 541 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.5 MiB/s wr, 673 op/s 2024-03-21T11:42:53.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:52 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:42:53.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:52 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:42:53.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:52 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:42:53.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:52 smithi100 ceph-mon[36245]: pgmap v986: 129 pgs: 129 active+clean; 541 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.5 MiB/s wr, 673 op/s 2024-03-21T11:42:53.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:52 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:42:53.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:52 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:42:53.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:52 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:42:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:52 smithi102 ceph-mon[36748]: pgmap v986: 129 pgs: 129 active+clean; 541 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.5 MiB/s wr, 673 op/s 2024-03-21T11:42:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:52 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:42:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:52 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:42:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:52 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:42:53.961 INFO:tasks.workunit.client.0.smithi047.stdout: 10 151764 23.20 MB/sec execute 217 sec latency 135.001 ms 2024-03-21T11:42:54.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:53 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:42:54.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:53 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:42:54.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:53 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:42:54.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:53 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:42:54.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:53 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:42:54.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:53 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:42:54.961 INFO:tasks.workunit.client.0.smithi047.stdout: 10 152234 23.20 MB/sec execute 218 sec latency 81.197 ms 2024-03-21T11:42:55.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:54 smithi047 ceph-mon[31351]: pgmap v987: 129 pgs: 129 active+clean; 539 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.2 MiB/s wr, 493 op/s 2024-03-21T11:42:55.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:54 smithi100 ceph-mon[36245]: pgmap v987: 129 pgs: 129 active+clean; 539 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.2 MiB/s wr, 493 op/s 2024-03-21T11:42:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:54 smithi102 ceph-mon[36748]: pgmap v987: 129 pgs: 129 active+clean; 539 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.2 MiB/s wr, 493 op/s 2024-03-21T11:42:55.961 INFO:tasks.workunit.client.0.smithi047.stdout: 10 152804 23.19 MB/sec execute 219 sec latency 97.920 ms 2024-03-21T11:42:56.961 INFO:tasks.workunit.client.0.smithi047.stdout: 10 153185 23.23 MB/sec execute 220 sec latency 336.749 ms 2024-03-21T11:42:57.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:56 smithi047 ceph-mon[31351]: pgmap v988: 129 pgs: 129 active+clean; 539 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.5 MiB/s wr, 510 op/s 2024-03-21T11:42:57.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:56 smithi100 ceph-mon[36245]: pgmap v988: 129 pgs: 129 active+clean; 539 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.5 MiB/s wr, 510 op/s 2024-03-21T11:42:57.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:56 smithi102 ceph-mon[36748]: pgmap v988: 129 pgs: 129 active+clean; 539 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.5 MiB/s wr, 510 op/s 2024-03-21T11:42:57.961 INFO:tasks.workunit.client.0.smithi047.stdout: 10 153519 23.18 MB/sec execute 221 sec latency 133.036 ms 2024-03-21T11:42:58.962 INFO:tasks.workunit.client.0.smithi047.stdout: 10 154046 23.24 MB/sec execute 222 sec latency 101.938 ms 2024-03-21T11:42:59.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:42:58 smithi047 ceph-mon[31351]: pgmap v989: 129 pgs: 129 active+clean; 539 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.5 MiB/s wr, 660 op/s 2024-03-21T11:42:59.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:42:58 smithi100 ceph-mon[36245]: pgmap v989: 129 pgs: 129 active+clean; 539 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.5 MiB/s wr, 660 op/s 2024-03-21T11:42:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:42:58 smithi102 ceph-mon[36748]: pgmap v989: 129 pgs: 129 active+clean; 539 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.5 MiB/s wr, 660 op/s 2024-03-21T11:42:59.961 INFO:tasks.workunit.client.0.smithi047.stdout: 10 154483 23.28 MB/sec execute 223 sec latency 83.645 ms 2024-03-21T11:43:00.962 INFO:tasks.workunit.client.0.smithi047.stdout: 10 154902 23.24 MB/sec execute 224 sec latency 197.570 ms 2024-03-21T11:43:01.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:00 smithi047 ceph-mon[31351]: pgmap v990: 129 pgs: 129 active+clean; 541 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.6 MiB/s wr, 549 op/s 2024-03-21T11:43:01.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:00 smithi100 ceph-mon[36245]: pgmap v990: 129 pgs: 129 active+clean; 541 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.6 MiB/s wr, 549 op/s 2024-03-21T11:43:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:00 smithi102 ceph-mon[36748]: pgmap v990: 129 pgs: 129 active+clean; 541 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.6 MiB/s wr, 549 op/s 2024-03-21T11:43:01.962 INFO:tasks.workunit.client.0.smithi047.stdout: 10 155101 23.17 MB/sec execute 225 sec latency 328.638 ms 2024-03-21T11:43:02.962 INFO:tasks.workunit.client.0.smithi047.stdout: 10 155356 23.12 MB/sec execute 226 sec latency 122.545 ms 2024-03-21T11:43:03.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:02 smithi047 ceph-mon[31351]: pgmap v991: 129 pgs: 129 active+clean; 549 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-03-21T11:43:03.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:02 smithi100 ceph-mon[36245]: pgmap v991: 129 pgs: 129 active+clean; 549 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-03-21T11:43:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:02 smithi102 ceph-mon[36748]: pgmap v991: 129 pgs: 129 active+clean; 549 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-03-21T11:43:03.962 INFO:tasks.workunit.client.0.smithi047.stdout: 10 155822 23.13 MB/sec execute 227 sec latency 108.446 ms 2024-03-21T11:43:04.963 INFO:tasks.workunit.client.0.smithi047.stdout: 10 156326 23.17 MB/sec execute 228 sec latency 96.254 ms 2024-03-21T11:43:05.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:04 smithi100 ceph-mon[36245]: pgmap v992: 129 pgs: 129 active+clean; 550 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.2 MiB/s wr, 522 op/s 2024-03-21T11:43:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:04 smithi102 ceph-mon[36748]: pgmap v992: 129 pgs: 129 active+clean; 550 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.2 MiB/s wr, 522 op/s 2024-03-21T11:43:05.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:04 smithi047 ceph-mon[31351]: pgmap v992: 129 pgs: 129 active+clean; 550 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.2 MiB/s wr, 522 op/s 2024-03-21T11:43:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.753+0000 7f5fb3a00640 1 -- 172.21.15.47:0/1344211036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fac101bd0 msgr2=0x7f5fac101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.753+0000 7f5fb3a00640 1 --2- 172.21.15.47:0/1344211036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fac101bd0 0x7f5fac101ff0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009900 tx=0x7f5fa002bc40 comp rx=0 tx=0).stop 2024-03-21T11:43:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.753+0000 7f5fb3a00640 1 -- 172.21.15.47:0/1344211036 shutdown_connections 2024-03-21T11:43:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.753+0000 7f5fb3a00640 1 --2- 172.21.15.47:0/1344211036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fac100110 0x7f5fac100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.753+0000 7f5fb3a00640 1 --2- 172.21.15.47:0/1344211036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fac0ff750 0x7f5fac0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.753+0000 7f5fb3a00640 1 --2- 172.21.15.47:0/1344211036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fac101bd0 0x7f5fac101ff0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.753+0000 7f5fb3a00640 1 -- 172.21.15.47:0/1344211036 >> 172.21.15.47:0/1344211036 conn(0x7f5fac0fb2f0 msgr2=0x7f5fac0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.753+0000 7f5fb3a00640 1 -- 172.21.15.47:0/1344211036 shutdown_connections 2024-03-21T11:43:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.753+0000 7f5fb3a00640 1 -- 172.21.15.47:0/1344211036 wait complete. 2024-03-21T11:43:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.754+0000 7f5fb3a00640 1 Processor -- start 2024-03-21T11:43:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.755+0000 7f5fb3a00640 1 -- start start 2024-03-21T11:43:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.755+0000 7f5fb3a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fac0ff750 0x7f5fac1063a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.755+0000 7f5fb3a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fac100110 0x7f5fac1068e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.755+0000 7f5fb3a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fac101bd0 0x7f5fac102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.755+0000 7f5fb3a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac10f530 con 0x7f5fac100110 2024-03-21T11:43:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.755+0000 7f5fb3a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac10f3b0 con 0x7f5fac101bd0 2024-03-21T11:43:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.755+0000 7f5fb3a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac10f6b0 con 0x7f5fac0ff750 2024-03-21T11:43:05.757 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.755+0000 7f5fb1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fac0ff750 0x7f5fac1063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.755+0000 7f5fb1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fac0ff750 0x7f5fac1063a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34820/0 (socket says 172.21.15.47:34820) 2024-03-21T11:43:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.755+0000 7f5fb1000640 1 -- 172.21.15.47:0/2766453581 learned_addr learned my addr 172.21.15.47:0/2766453581 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.755+0000 7f5fb1a00640 1 --2- 172.21.15.47:0/2766453581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fac101bd0 0x7f5fac102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.756+0000 7f5fabe00640 1 --2- 172.21.15.47:0/2766453581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fac100110 0x7f5fac1068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.756+0000 7f5fb1a00640 1 -- 172.21.15.47:0/2766453581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fac0ff750 msgr2=0x7f5fac1063a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.756+0000 7f5fb1a00640 1 --2- 172.21.15.47:0/2766453581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fac0ff750 0x7f5fac1063a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.756+0000 7f5fb1a00640 1 -- 172.21.15.47:0/2766453581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fac100110 msgr2=0x7f5fac1068e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.756+0000 7f5fb1a00640 1 --2- 172.21.15.47:0/2766453581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fac100110 0x7f5fac1068e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.756+0000 7f5fb1a00640 1 -- 172.21.15.47:0/2766453581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac1037d0 con 0x7f5fac101bd0 2024-03-21T11:43:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.756+0000 7f5fb1a00640 1 --2- 172.21.15.47:0/2766453581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fac101bd0 0x7f5fac102f10 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400e940 tx=0x7f5fa400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.757+0000 7f5fa9600640 1 -- 172.21.15.47:0/2766453581 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4004d20 con 0x7f5fac101bd0 2024-03-21T11:43:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.757+0000 7f5fa9600640 1 -- 172.21.15.47:0/2766453581 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fa4005020 con 0x7f5fac101bd0 2024-03-21T11:43:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.757+0000 7f5fa9600640 1 -- 172.21.15.47:0/2766453581 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40116b0 con 0x7f5fac101bd0 2024-03-21T11:43:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.757+0000 7f5fb3a00640 1 -- 172.21.15.47:0/2766453581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fac1a5fc0 con 0x7f5fac101bd0 2024-03-21T11:43:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.757+0000 7f5fb3a00640 1 -- 172.21.15.47:0/2766453581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fac1a6370 con 0x7f5fac101bd0 2024-03-21T11:43:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.761+0000 7f5fa9600640 1 -- 172.21.15.47:0/2766453581 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5fa40054d0 con 0x7f5fac101bd0 2024-03-21T11:43:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.763+0000 7f5fa9600640 1 --2- 172.21.15.47:0/2766453581 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5f940a30c0 0x7f5f940a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.764+0000 7f5fa9600640 1 -- 172.21.15.47:0/2766453581 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5fa40cf830 con 0x7f5fac101bd0 2024-03-21T11:43:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.764+0000 7f5fb6654740 1 Processor -- start 2024-03-21T11:43:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.765+0000 7f5fb6654740 1 -- start start 2024-03-21T11:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.765+0000 7f5fb6654740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86be1e0 0x5594c86be600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.765+0000 7f5fb6654740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86bebd0 0x5594c86c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.765+0000 7f5fb6654740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86c3d40 0x5594c86c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.766+0000 7f5fb6654740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c86813b0 con 0x5594c86be1e0 2024-03-21T11:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.766+0000 7f5fb6654740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c8623830 con 0x5594c86c3d40 2024-03-21T11:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.766+0000 7f5fb6654740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c85cc5d0 con 0x5594c86bebd0 2024-03-21T11:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.766+0000 7f5fb1000640 1 --2- 172.21.15.47:0/2766453581 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5f940a30c0 0x7f5f940a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.766+0000 7f5fabe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86bebd0 0x5594c86c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.766+0000 7f5fabe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86bebd0 0x5594c86c36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34830/0 (socket says 172.21.15.47:34830) 2024-03-21T11:43:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.766+0000 7f5fabe00640 1 -- 172.21.15.47:0/1043485345 learned_addr learned my addr 172.21.15.47:0/1043485345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.766+0000 7f5fb1a00640 1 --2- 172.21.15.47:0/1043485345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86c3d40 0x5594c86c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.766+0000 7f5fabe00640 1 -- 172.21.15.47:0/1043485345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86c3d40 msgr2=0x5594c86c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.766+0000 7f5fabe00640 1 --2- 172.21.15.47:0/1043485345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86c3d40 0x5594c86c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.766+0000 7f5fabe00640 1 -- 172.21.15.47:0/1043485345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86be1e0 msgr2=0x5594c86be600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:43:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.768+0000 7f5fb1000640 1 --2- 172.21.15.47:0/1043485345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86be1e0 0x5594c86be600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.768+0000 7f5fabe00640 1 --2- 172.21.15.47:0/1043485345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86be1e0 0x5594c86be600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.768+0000 7f5fabe00640 1 -- 172.21.15.47:0/1043485345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594c85e42c0 con 0x5594c86bebd0 2024-03-21T11:43:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.768+0000 7f5fb1000640 1 --2- 172.21.15.47:0/1043485345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86be1e0 0x5594c86be600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:43:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.769+0000 7f5fabe00640 1 --2- 172.21.15.47:0/1043485345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86bebd0 0x5594c86c36d0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009c10 tx=0x7f5f9c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3c1538de3a6969f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.770+0000 7f5f9a000640 1 -- 172.21.15.47:0/1043485345 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c012450 con 0x5594c86bebd0 2024-03-21T11:43:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.770+0000 7f5fb1000640 1 --2- 172.21.15.47:0/2766453581 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5f940a30c0 0x7f5f940a5580 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5fa002c8f0 tx=0x7f5fa002cbb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.770+0000 7f5f9a000640 1 -- 172.21.15.47:0/1043485345 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f9c012ad0 con 0x5594c86bebd0 2024-03-21T11:43:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.770+0000 7f5f9a000640 1 -- 172.21.15.47:0/1043485345 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c00fbd0 con 0x5594c86bebd0 2024-03-21T11:43:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.771+0000 7f5fb6654740 1 -- 172.21.15.47:0/1043485345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86bebd0 msgr2=0x5594c86c36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.771+0000 7f5fb6654740 1 --2- 172.21.15.47:0/1043485345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86bebd0 0x5594c86c36d0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009c10 tx=0x7f5f9c0117b0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.772+0000 7f5fb6654740 1 -- 172.21.15.47:0/1043485345 shutdown_connections 2024-03-21T11:43:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.772+0000 7f5fb6654740 1 --2- 172.21.15.47:0/1043485345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86c3d40 0x5594c86c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.772+0000 7f5fb6654740 1 --2- 172.21.15.47:0/1043485345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86bebd0 0x5594c86c36d0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.772+0000 7f5fb6654740 1 --2- 172.21.15.47:0/1043485345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86be1e0 0x5594c86be600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.772+0000 7f5fb6654740 1 -- 172.21.15.47:0/1043485345 >> 172.21.15.47:0/1043485345 conn(0x5594c86b9750 msgr2=0x5594c86b9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.772+0000 7f5fb6654740 1 -- 172.21.15.47:0/1043485345 shutdown_connections 2024-03-21T11:43:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.772+0000 7f5fb6654740 1 -- 172.21.15.47:0/1043485345 wait complete. 2024-03-21T11:43:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.773+0000 7f5fb6654740 1 -- start start 2024-03-21T11:43:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.773+0000 7f5fb6654740 1 Processor -- start 2024-03-21T11:43:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.774+0000 7f5fb6654740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86be1e0 0x5594c86c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.774+0000 7f5fb6654740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86bebd0 0x5594c86c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.775+0000 7f5fb1000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86be1e0 0x5594c86c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.775+0000 7f5fabe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86bebd0 0x5594c86c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.775+0000 7f5fb1000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86be1e0 0x5594c86c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43152/0 (socket says 172.21.15.47:43152) 2024-03-21T11:43:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.775+0000 7f5fabe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86bebd0 0x5594c86c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34844/0 (socket says 172.21.15.47:34844) 2024-03-21T11:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.775+0000 7f5fb6654740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86c3d40 0x5594c86cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.775+0000 7f5fb6654740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c85dfa20 con 0x5594c86be1e0 2024-03-21T11:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.775+0000 7f5fb6654740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c861f8d0 con 0x5594c86c3d40 2024-03-21T11:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.775+0000 7f5fb6654740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c86c6cf0 con 0x5594c86bebd0 2024-03-21T11:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.775+0000 7f5fb1000640 1 -- 172.21.15.47:0/1725094584 learned_addr learned my addr 172.21.15.47:0/1725094584 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.775+0000 7f5fb1a00640 1 --2- 172.21.15.47:0/1725094584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86c3d40 0x5594c86cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.776+0000 7f5fabe00640 1 -- 172.21.15.47:0/1725094584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86c3d40 msgr2=0x5594c86cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.776+0000 7f5fabe00640 1 --2- 172.21.15.47:0/1725094584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86c3d40 0x5594c86cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.776+0000 7f5fabe00640 1 -- 172.21.15.47:0/1725094584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86be1e0 msgr2=0x5594c86c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.776+0000 7f5fabe00640 1 --2- 172.21.15.47:0/1725094584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86be1e0 0x5594c86c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.776+0000 7f5fabe00640 1 -- 172.21.15.47:0/1725094584 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594c8338210 con 0x5594c86bebd0 2024-03-21T11:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.775+0000 7f5fb6654740 10 client.0 _ll_register_callbacks cb 0x5594c861c1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.776+0000 7f5fb6654740 1 -- 172.21.15.47:0/1725094584 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f9c012ad0 con 0x5594c86bebd0 2024-03-21T11:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.776+0000 7f5fabe00640 1 --2- 172.21.15.47:0/1725094584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86bebd0 0x5594c86c8930 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x5594c86ba6c0 tx=0x7f5f9c009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4fb88a167a297f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.776+0000 7f5f8ea00640 10 client.31831 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.777+0000 7f5f8ea00640 1 -- 172.21.15.47:0/1725094584 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c028070 con 0x5594c86bebd0 2024-03-21T11:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.777+0000 7f5f8ea00640 1 -- 172.21.15.47:0/1725094584 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5f9c012860 con 0x5594c86bebd0 2024-03-21T11:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.777+0000 7f5f8ea00640 1 -- 172.21.15.47:0/1725094584 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f9c004020 con 0x5594c86bebd0 2024-03-21T11:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.777+0000 7f5fb6654740 10 client.31831 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.777+0000 7f5fb6654740 1 -- 172.21.15.47:0/1725094584 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=40}) v3 -- 0x5594c8404620 con 0x5594c86bebd0 2024-03-21T11:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.778+0000 7f5f8ea00640 1 -- 172.21.15.47:0/1725094584 <== mon.1 v2:172.21.15.102:3300/0 4 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5f9c0041c0 con 0x5594c86bebd0 2024-03-21T11:43:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.778+0000 7f5fb6654740 10 client.31831 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.778+0000 7f5fb6654740 10 client.31831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.778+0000 7f5fb6654740 20 client.31831 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.778+0000 7f5fb6654740 1 --2- 172.21.15.47:0/1725094584 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5594c86d3d70 0x5594c86d6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.778+0000 7f5fb6654740 4 client.31831 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.778+0000 7f5fb6654740 1 -- 172.21.15.47:0/1725094584 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594c8630d70 con 0x5594c86d3d70 2024-03-21T11:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.778+0000 7f5fb1000640 1 --2- 172.21.15.47:0/1725094584 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5594c86d3d70 0x5594c86d6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.779+0000 7f5fb1000640 1 --2- 172.21.15.47:0/1725094584 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5594c86d3d70 0x5594c86d6160 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3be860493aa6413 server_cookie=c1623bbd2afa527c in_seq=0 out_seq=0 2024-03-21T11:43:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.779+0000 7f5f8ea00640 10 client.31831 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.783+0000 7f5f8ea00640 1 -- 172.21.15.47:0/1725094584 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5594c8630d70 con 0x5594c86d3d70 2024-03-21T11:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.783+0000 7f5f8ea00640 10 client.31831 handle_command_reply: tid=0 2024-03-21T11:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.783+0000 7f5fb6654740 1 client.31831 shutdown 2024-03-21T11:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.783+0000 7f5fb6654740 1 -- 172.21.15.47:0/1725094584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86bebd0 msgr2=0x5594c86c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.783+0000 7f5fb6654740 1 --2- 172.21.15.47:0/1725094584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86bebd0 0x5594c86c8930 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x5594c86ba6c0 tx=0x7f5f9c009830 comp rx=0 tx=0).stop 2024-03-21T11:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.783+0000 7f5fb6654740 1 -- 172.21.15.47:0/1725094584 shutdown_connections 2024-03-21T11:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.783+0000 7f5fb6654740 1 --2- 172.21.15.47:0/1725094584 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5594c86d3d70 0x5594c86d6160 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.783+0000 7f5fb6654740 1 --2- 172.21.15.47:0/1725094584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86c3d40 0x5594c86cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.783+0000 7f5fb6654740 1 --2- 172.21.15.47:0/1725094584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86bebd0 0x5594c86c8930 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.783+0000 7f5fb6654740 1 --2- 172.21.15.47:0/1725094584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86be1e0 0x5594c86c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.783+0000 7f5f8ea00640 0 client.31831 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.783+0000 7f5fb6654740 1 -- 172.21.15.47:0/1725094584 >> 172.21.15.47:0/1725094584 conn(0x5594c86b9750 msgr2=0x5594c86b8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.783+0000 7f5fb6654740 1 -- 172.21.15.47:0/1725094584 shutdown_connections 2024-03-21T11:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.784+0000 7f5fb6654740 1 -- 172.21.15.47:0/1725094584 wait complete. 2024-03-21T11:43:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.784+0000 7f5fb6654740 20 client.31831 trim_cache size 0 max 16384 2024-03-21T11:43:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.804+0000 7f5fb6654740 1 Processor -- start 2024-03-21T11:43:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.804+0000 7f5fb6654740 1 -- start start 2024-03-21T11:43:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb6654740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86be1e0 0x5594c86b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb6654740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86bebd0 0x5594c86cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb6654740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86c3d40 0x5594c86bd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb6654740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac1b0640 con 0x5594c86bebd0 2024-03-21T11:43:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb6654740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac1b1fc0 con 0x5594c86be1e0 2024-03-21T11:43:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb6654740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac1b0ca0 con 0x5594c86c3d40 2024-03-21T11:43:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86be1e0 0x5594c86b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fabe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86bebd0 0x5594c86cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86be1e0 0x5594c86b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46916/0 (socket says 172.21.15.47:46916) 2024-03-21T11:43:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb1000640 1 -- 172.21.15.47:0/258724154 learned_addr learned my addr 172.21.15.47:0/258724154 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb1a00640 1 --2- 172.21.15.47:0/258724154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86c3d40 0x5594c86bd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb1000640 1 -- 172.21.15.47:0/258724154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86c3d40 msgr2=0x5594c86bd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb1000640 1 --2- 172.21.15.47:0/258724154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86c3d40 0x5594c86bd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb1000640 1 -- 172.21.15.47:0/258724154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86bebd0 msgr2=0x5594c86cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb1000640 1 --2- 172.21.15.47:0/258724154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86bebd0 0x5594c86cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb1000640 1 -- 172.21.15.47:0/258724154 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594c86c9520 con 0x5594c86be1e0 2024-03-21T11:43:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb1a00640 1 --2- 172.21.15.47:0/258724154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86c3d40 0x5594c86bd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5fb1000640 1 --2- 172.21.15.47:0/258724154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86be1e0 0x5594c86b43a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5fa002c7c0 tx=0x7f5fa002e050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c411f738deb6b970 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5f99600640 1 -- 172.21.15.47:0/258724154 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0058070 con 0x5594c86be1e0 2024-03-21T11:43:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5f99600640 1 -- 172.21.15.47:0/258724154 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fa0004ba0 con 0x5594c86be1e0 2024-03-21T11:43:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.805+0000 7f5f99600640 1 -- 172.21.15.47:0/258724154 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00052f0 con 0x5594c86be1e0 2024-03-21T11:43:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 -- 172.21.15.47:0/258724154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86be1e0 msgr2=0x5594c86b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 --2- 172.21.15.47:0/258724154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86be1e0 0x5594c86b43a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5fa002c7c0 tx=0x7f5fa002e050 comp rx=0 tx=0).stop 2024-03-21T11:43:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 -- 172.21.15.47:0/258724154 shutdown_connections 2024-03-21T11:43:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 --2- 172.21.15.47:0/258724154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86c3d40 0x5594c86bd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 --2- 172.21.15.47:0/258724154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86bebd0 0x5594c86cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 --2- 172.21.15.47:0/258724154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86be1e0 0x5594c86b43a0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 -- 172.21.15.47:0/258724154 >> 172.21.15.47:0/258724154 conn(0x5594c86b9750 msgr2=0x5594c86bc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 -- 172.21.15.47:0/258724154 shutdown_connections 2024-03-21T11:43:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 -- 172.21.15.47:0/258724154 wait complete. 2024-03-21T11:43:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 -- start start 2024-03-21T11:43:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 Processor -- start 2024-03-21T11:43:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86be1e0 0x5594c86cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86bebd0 0x5594c86cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86c3d40 0x5594c86cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac1b1960 con 0x5594c86bebd0 2024-03-21T11:43:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac1b1300 con 0x5594c86be1e0 2024-03-21T11:43:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac1ae800 con 0x5594c86c3d40 2024-03-21T11:43:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb6654740 10 client.0 _ll_register_callbacks cb 0x7f5f9c009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86be1e0 0x5594c86cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86c3d40 0x5594c86cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fabe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86bebd0 0x5594c86cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86be1e0 0x5594c86cf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46918/0 (socket says 172.21.15.47:46918) 2024-03-21T11:43:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb1000640 1 -- 172.21.15.47:0/2596684545 learned_addr learned my addr 172.21.15.47:0/2596684545 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.807+0000 7f5fb1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86c3d40 0x5594c86cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:34854/0 (socket says 172.21.15.47:34854) 2024-03-21T11:43:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.808+0000 7f5fb1000640 1 -- 172.21.15.47:0/2596684545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86c3d40 msgr2=0x5594c86cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.808+0000 7f5fb1000640 1 --2- 172.21.15.47:0/2596684545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86c3d40 0x5594c86cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.808+0000 7f5fb1000640 1 -- 172.21.15.47:0/2596684545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86bebd0 msgr2=0x5594c86cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.808+0000 7f5fb1000640 1 --2- 172.21.15.47:0/2596684545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86bebd0 0x5594c86cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.808+0000 7f5fb1000640 1 -- 172.21.15.47:0/2596684545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594c86ce520 con 0x5594c86be1e0 2024-03-21T11:43:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.808+0000 7f5fb1000640 1 --2- 172.21.15.47:0/2596684545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86be1e0 0x5594c86cf490 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0005c40 tx=0x7f5fa0005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928b12b586c8aad3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.808+0000 7f5f8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:43:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.808+0000 7f5f8ea00640 1 -- 172.21.15.47:0/2596684545 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0058070 con 0x5594c86be1e0 2024-03-21T11:43:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.810+0000 7f5f8ea00640 1 -- 172.21.15.47:0/2596684545 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5fa00046c0 con 0x5594c86be1e0 2024-03-21T11:43:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.810+0000 7f5fb6654740 1 -- 172.21.15.47:0/2596684545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fa0004ba0 con 0x5594c86be1e0 2024-03-21T11:43:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.810+0000 7f5f8ea00640 1 -- 172.21.15.47:0/2596684545 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fa0067050 con 0x5594c86be1e0 2024-03-21T11:43:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.810+0000 7f5fb6654740 10 client.31883 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.810+0000 7f5fb6654740 1 -- 172.21.15.47:0/2596684545 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5594c87300d0 con 0x5594c86be1e0 2024-03-21T11:43:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.811+0000 7f5f8ea00640 1 -- 172.21.15.47:0/2596684545 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fa0004ba0 con 0x5594c86be1e0 2024-03-21T11:43:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.811+0000 7f5f8ea00640 1 -- 172.21.15.47:0/2596684545 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5fa00037b0 con 0x5594c86be1e0 2024-03-21T11:43:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.811+0000 7f5fb6654740 10 client.31883 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.811+0000 7f5fb6654740 10 client.31883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.811+0000 7f5fb6654740 20 client.31883 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.811+0000 7f5fb6654740 1 --2- 172.21.15.47:0/2596684545 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5594c8730a30 0x5594c8732e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.811+0000 7f5fb6654740 4 client.31883 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:43:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.811+0000 7f5fb6654740 1 -- 172.21.15.47:0/2596684545 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5594c86b4a80 con 0x5594c8730a30 2024-03-21T11:43:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.811+0000 7f5fabe00640 1 --2- 172.21.15.47:0/2596684545 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5594c8730a30 0x5594c8732e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.812+0000 7f5fabe00640 1 --2- 172.21.15.47:0/2596684545 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5594c8730a30 0x5594c8732e50 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9c80698ef5db4b server_cookie=8bc35318d9fe2b22 in_seq=0 out_seq=0 2024-03-21T11:43:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:05.812+0000 7f5f8ea00640 10 client.31883 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:05.962 INFO:tasks.workunit.client.0.smithi047.stdout: 10 156954 23.16 MB/sec execute 229 sec latency 96.469 ms 2024-03-21T11:43:06.962 INFO:tasks.workunit.client.0.smithi047.stdout: 10 157019 23.08 MB/sec execute 230 sec latency 1064.916 ms 2024-03-21T11:43:07.146 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:06 smithi100 ceph-mon[36245]: pgmap v993: 129 pgs: 129 active+clean; 550 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.6 MiB/s wr, 500 op/s 2024-03-21T11:43:07.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:06 smithi102 ceph-mon[36748]: pgmap v993: 129 pgs: 129 active+clean; 550 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.6 MiB/s wr, 500 op/s 2024-03-21T11:43:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.241+0000 7f5f8ea00640 1 -- 172.21.15.47:0/2596684545 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5594c86b4a80 con 0x5594c8730a30 2024-03-21T11:43:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.241+0000 7f5f8ea00640 10 client.31883 handle_command_reply: tid=0 2024-03-21T11:43:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.241+0000 7f5fb6654740 1 client.31883 shutdown 2024-03-21T11:43:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.242+0000 7f5fb6654740 1 -- 172.21.15.47:0/2596684545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86be1e0 msgr2=0x5594c86cf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.242+0000 7f5fb6654740 1 --2- 172.21.15.47:0/2596684545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86be1e0 0x5594c86cf490 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0005c40 tx=0x7f5fa0005b50 comp rx=0 tx=0).stop 2024-03-21T11:43:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.243+0000 7f5fb6654740 1 -- 172.21.15.47:0/2596684545 shutdown_connections 2024-03-21T11:43:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.243+0000 7f5fb6654740 1 --2- 172.21.15.47:0/2596684545 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5594c8730a30 0x5594c8732e50 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.243+0000 7f5fb6654740 1 --2- 172.21.15.47:0/2596684545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5594c86c3d40 0x5594c86cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.243+0000 7f5fb6654740 1 --2- 172.21.15.47:0/2596684545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594c86bebd0 0x5594c86cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.243+0000 7f5fb6654740 1 --2- 172.21.15.47:0/2596684545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594c86be1e0 0x5594c86cf490 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.243+0000 7f5fb6654740 1 -- 172.21.15.47:0/2596684545 >> 172.21.15.47:0/2596684545 conn(0x5594c86b9750 msgr2=0x5594c86bd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.243+0000 7f5f8ea00640 0 client.31883 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.243+0000 7f5fb6654740 1 -- 172.21.15.47:0/2596684545 shutdown_connections 2024-03-21T11:43:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.243+0000 7f5fb6654740 1 -- 172.21.15.47:0/2596684545 wait complete. 2024-03-21T11:43:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.243+0000 7f5fb6654740 20 client.31883 trim_cache size 0 max 16384 2024-03-21T11:43:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:43:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:43:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:43:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:43:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.244+0000 7f5fb3a00640 1 -- 172.21.15.47:0/2766453581 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5f940a30c0 msgr2=0x7f5f940a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.244+0000 7f5fb3a00640 1 --2- 172.21.15.47:0/2766453581 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5f940a30c0 0x7f5f940a5580 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5fa002c8f0 tx=0x7f5fa002cbb0 comp rx=0 tx=0).stop 2024-03-21T11:43:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.244+0000 7f5fb3a00640 1 -- 172.21.15.47:0/2766453581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fac101bd0 msgr2=0x7f5fac102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.244+0000 7f5fb3a00640 1 --2- 172.21.15.47:0/2766453581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fac101bd0 0x7f5fac102f10 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400e940 tx=0x7f5fa400ee00 comp rx=0 tx=0).stop 2024-03-21T11:43:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.246+0000 7f5fb3a00640 1 -- 172.21.15.47:0/2766453581 shutdown_connections 2024-03-21T11:43:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.246+0000 7f5fb3a00640 1 --2- 172.21.15.47:0/2766453581 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5f940a30c0 0x7f5f940a5580 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.246+0000 7f5fb3a00640 1 --2- 172.21.15.47:0/2766453581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fac101bd0 0x7f5fac102f10 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.246+0000 7f5fb3a00640 1 --2- 172.21.15.47:0/2766453581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fac100110 0x7f5fac1068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.246+0000 7f5fb3a00640 1 --2- 172.21.15.47:0/2766453581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fac0ff750 0x7f5fac1063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.246+0000 7f5fb3a00640 1 -- 172.21.15.47:0/2766453581 >> 172.21.15.47:0/2766453581 conn(0x7f5fac0fb2f0 msgr2=0x7f5fac0fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.246+0000 7f5fb3a00640 1 -- 172.21.15.47:0/2766453581 shutdown_connections 2024-03-21T11:43:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:07.246+0000 7f5fb3a00640 1 -- 172.21.15.47:0/2766453581 wait complete. 2024-03-21T11:43:07.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:06 smithi047 ceph-mon[31351]: pgmap v993: 129 pgs: 129 active+clean; 550 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.6 MiB/s wr, 500 op/s 2024-03-21T11:43:07.962 INFO:tasks.workunit.client.0.smithi047.stdout: 10 157347 23.04 MB/sec execute 231 sec latency 1256.349 ms 2024-03-21T11:43:08.962 INFO:tasks.workunit.client.0.smithi047.stdout: 10 157830 23.10 MB/sec execute 232 sec latency 89.589 ms 2024-03-21T11:43:09.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:08 smithi100 ceph-mon[36245]: pgmap v994: 129 pgs: 129 active+clean; 550 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.6 MiB/s wr, 659 op/s 2024-03-21T11:43:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:08 smithi102 ceph-mon[36748]: pgmap v994: 129 pgs: 129 active+clean; 550 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.6 MiB/s wr, 659 op/s 2024-03-21T11:43:09.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:08 smithi047 ceph-mon[31351]: pgmap v994: 129 pgs: 129 active+clean; 550 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.6 MiB/s wr, 659 op/s 2024-03-21T11:43:09.962 INFO:tasks.workunit.client.0.smithi047.stdout: 10 158453 23.20 MB/sec execute 233 sec latency 54.186 ms 2024-03-21T11:43:10.962 INFO:tasks.workunit.client.0.smithi047.stdout: 10 158866 23.16 MB/sec execute 234 sec latency 72.118 ms 2024-03-21T11:43:11.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:10 smithi100 ceph-mon[36245]: pgmap v995: 129 pgs: 129 active+clean; 551 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.7 MiB/s wr, 550 op/s 2024-03-21T11:43:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:10 smithi102 ceph-mon[36748]: pgmap v995: 129 pgs: 129 active+clean; 551 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.7 MiB/s wr, 550 op/s 2024-03-21T11:43:11.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:10 smithi047 ceph-mon[31351]: pgmap v995: 129 pgs: 129 active+clean; 551 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.7 MiB/s wr, 550 op/s 2024-03-21T11:43:11.962 INFO:tasks.workunit.client.0.smithi047.stdout: 10 159110 23.11 MB/sec execute 235 sec latency 279.118 ms 2024-03-21T11:43:12.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:43:12.006 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:43:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.331+0000 7f3157a00640 1 -- 172.21.15.47:0/1739134425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3150105470 msgr2=0x7f315010bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.331+0000 7f3157a00640 1 --2- 172.21.15.47:0/1739134425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3150105470 0x7f315010bd00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f3140009900 tx=0x7f314002bc30 comp rx=0 tx=0).stop 2024-03-21T11:43:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.331+0000 7f3157a00640 1 -- 172.21.15.47:0/1739134425 shutdown_connections 2024-03-21T11:43:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.331+0000 7f3157a00640 1 --2- 172.21.15.47:0/1739134425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3150105470 0x7f315010bd00 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.331+0000 7f3157a00640 1 --2- 172.21.15.47:0/1739134425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3150104ab0 0x7f3150104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.331+0000 7f3157a00640 1 --2- 172.21.15.47:0/1739134425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31501038c0 0x7f3150103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.331+0000 7f3157a00640 1 -- 172.21.15.47:0/1739134425 >> 172.21.15.47:0/1739134425 conn(0x7f31500fee50 msgr2=0x7f3150101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.333+0000 7f3157a00640 1 -- 172.21.15.47:0/1739134425 shutdown_connections 2024-03-21T11:43:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.333+0000 7f3157a00640 1 -- 172.21.15.47:0/1739134425 wait complete. 2024-03-21T11:43:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.334+0000 7f3157a00640 1 Processor -- start 2024-03-21T11:43:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.334+0000 7f3157a00640 1 -- start start 2024-03-21T11:43:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.335+0000 7f3157a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31501038c0 0x7f3150070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.335+0000 7f3157a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3150104ab0 0x7f3150070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.335+0000 7f3157a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3150105470 0x7f3150198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.335+0000 7f3157a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315010f630 con 0x7f3150104ab0 2024-03-21T11:43:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.335+0000 7f3157a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315010f4b0 con 0x7f3150105470 2024-03-21T11:43:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.335+0000 7f3155000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31501038c0 0x7f3150070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.335+0000 7f3157a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315010f7b0 con 0x7f31501038c0 2024-03-21T11:43:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.335+0000 7f3155000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31501038c0 0x7f3150070020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50632/0 (socket says 172.21.15.47:50632) 2024-03-21T11:43:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.335+0000 7f3155000640 1 -- 172.21.15.47:0/592239232 learned_addr learned my addr 172.21.15.47:0/592239232 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.335+0000 7f314fe00640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3150104ab0 0x7f3150070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.335+0000 7f3155a00640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3150105470 0x7f3150198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.336+0000 7f3155000640 1 -- 172.21.15.47:0/592239232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3150105470 msgr2=0x7f3150198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.336+0000 7f3155000640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3150105470 0x7f3150198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.336+0000 7f3155000640 1 -- 172.21.15.47:0/592239232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3150104ab0 msgr2=0x7f3150070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.336+0000 7f3155000640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3150104ab0 0x7f3150070560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.336+0000 7f3155000640 1 -- 172.21.15.47:0/592239232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3150199270 con 0x7f31501038c0 2024-03-21T11:43:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.336+0000 7f314fe00640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3150104ab0 0x7f3150070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:12.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.337+0000 7f3155000640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31501038c0 0x7f3150070020 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f314800b9c0 tx=0x7f314800be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:12.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.338+0000 7f314d600640 1 -- 172.21.15.47:0/592239232 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3148019070 con 0x7f31501038c0 2024-03-21T11:43:12.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.338+0000 7f314d600640 1 -- 172.21.15.47:0/592239232 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f314800cba0 con 0x7f31501038c0 2024-03-21T11:43:12.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.338+0000 7f314d600640 1 -- 172.21.15.47:0/592239232 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3148012640 con 0x7f31501038c0 2024-03-21T11:43:12.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.338+0000 7f3157a00640 1 -- 172.21.15.47:0/592239232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3150199560 con 0x7f31501038c0 2024-03-21T11:43:12.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.338+0000 7f3157a00640 1 -- 172.21.15.47:0/592239232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3150199a60 con 0x7f31501038c0 2024-03-21T11:43:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.344+0000 7f314d600640 1 -- 172.21.15.47:0/592239232 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f314801c070 con 0x7f31501038c0 2024-03-21T11:43:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.345+0000 7f315a602740 1 Processor -- start 2024-03-21T11:43:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.347+0000 7f315a602740 1 -- start start 2024-03-21T11:43:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.348+0000 7f315a602740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbce80 0x562213fbd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.348+0000 7f315a602740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fbd870 0x562213fc2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.348+0000 7f315a602740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fc29e0 0x562213fc4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.348+0000 7f315a602740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562213f22250 con 0x562213fbce80 2024-03-21T11:43:12.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.348+0000 7f315a602740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562213ecaa30 con 0x562213fbd870 2024-03-21T11:43:12.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.348+0000 7f315a602740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562213f1e2f0 con 0x562213fc29e0 2024-03-21T11:43:12.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.348+0000 7f3155a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbce80 0x562213fbd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.348+0000 7f314fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fc29e0 0x562213fc4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.348+0000 7f314d600640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31340a3070 0x7f31340a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.349+0000 7f3155000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fbd870 0x562213fc2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.349+0000 7f3155a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbce80 0x562213fbd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33808/0 (socket says 172.21.15.47:33808) 2024-03-21T11:43:12.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.349+0000 7f3155a00640 1 -- 172.21.15.47:0/1587312979 learned_addr learned my addr 172.21.15.47:0/1587312979 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:12.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.349+0000 7f3155a00640 1 -- 172.21.15.47:0/1587312979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fc29e0 msgr2=0x562213fc4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:12.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.349+0000 7f3155a00640 1 --2- 172.21.15.47:0/1587312979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fc29e0 0x562213fc4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.349+0000 7f3155a00640 1 -- 172.21.15.47:0/1587312979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fbd870 msgr2=0x562213fc2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:12.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.349+0000 7f3155a00640 1 --2- 172.21.15.47:0/1587312979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fbd870 0x562213fc2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.349+0000 7f3155a00640 1 -- 172.21.15.47:0/1587312979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562213eb5690 con 0x562213fbce80 2024-03-21T11:43:12.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.350+0000 7f3155a00640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31340a3070 0x7f31340a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.350+0000 7f3155a00640 1 --2- 172.21.15.47:0/1587312979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbce80 0x562213fbd2a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3140005ab0 tx=0x7f314002ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d1f977895f4d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:12.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.350+0000 7f3146000640 1 -- 172.21.15.47:0/1587312979 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140044070 con 0x562213fbce80 2024-03-21T11:43:12.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.350+0000 7f3146000640 1 -- 172.21.15.47:0/1587312979 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3140004ba0 con 0x562213fbce80 2024-03-21T11:43:12.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.351+0000 7f3146000640 1 -- 172.21.15.47:0/1587312979 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140003b40 con 0x562213fbce80 2024-03-21T11:43:12.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.352+0000 7f315a602740 1 -- 172.21.15.47:0/1587312979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbce80 msgr2=0x562213fbd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:12.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.352+0000 7f315a602740 1 --2- 172.21.15.47:0/1587312979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbce80 0x562213fbd2a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3140005ab0 tx=0x7f314002ca30 comp rx=0 tx=0).stop 2024-03-21T11:43:12.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.352+0000 7f314d600640 1 -- 172.21.15.47:0/592239232 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f31480cebd0 con 0x7f31501038c0 2024-03-21T11:43:12.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.352+0000 7f3155a00640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31340a3070 0x7f31340a5530 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f314002cfd0 tx=0x7f31400056a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:12.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.352+0000 7f315a602740 1 -- 172.21.15.47:0/1587312979 shutdown_connections 2024-03-21T11:43:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.353+0000 7f315a602740 1 --2- 172.21.15.47:0/1587312979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fc29e0 0x562213fc4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.353+0000 7f315a602740 1 --2- 172.21.15.47:0/1587312979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fbd870 0x562213fc2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.353+0000 7f315a602740 1 --2- 172.21.15.47:0/1587312979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbce80 0x562213fbd2a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.353+0000 7f315a602740 1 -- 172.21.15.47:0/1587312979 >> 172.21.15.47:0/1587312979 conn(0x562213f85200 msgr2=0x562213fb87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.353+0000 7f315a602740 1 -- 172.21.15.47:0/1587312979 shutdown_connections 2024-03-21T11:43:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.353+0000 7f315a602740 1 -- 172.21.15.47:0/1587312979 wait complete. 2024-03-21T11:43:12.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.354+0000 7f315a602740 1 -- start start 2024-03-21T11:43:12.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.354+0000 7f315a602740 1 Processor -- start 2024-03-21T11:43:12.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.356+0000 7f315a602740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fbce80 0x562213fcf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.356+0000 7f315a602740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbd870 0x562213fcfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.356+0000 7f315a602740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fc29e0 0x562213fd6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.356+0000 7f315a602740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562213fc5980 con 0x562213fbd870 2024-03-21T11:43:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.356+0000 7f3155000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fbce80 0x562213fcf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.356+0000 7f3155000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fbce80 0x562213fcf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57966/0 (socket says 172.21.15.47:57966) 2024-03-21T11:43:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.356+0000 7f3155000640 1 -- 172.21.15.47:0/785119570 learned_addr learned my addr 172.21.15.47:0/785119570 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.356+0000 7f3155a00640 1 --2- 172.21.15.47:0/785119570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fc29e0 0x562213fd6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.356+0000 7f315a602740 1 -- 172.21.15.47:0/785119570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562213edde80 con 0x562213fbce80 2024-03-21T11:43:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.356+0000 7f315a602740 1 -- 172.21.15.47:0/785119570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562213fc5b00 con 0x562213fc29e0 2024-03-21T11:43:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.357+0000 7f315a602740 10 client.0 _ll_register_callbacks cb 0x562213ef0130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.357+0000 7f3155a00640 1 -- 172.21.15.47:0/785119570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fbce80 msgr2=0x562213fcf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.357+0000 7f3155a00640 1 --2- 172.21.15.47:0/785119570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fbce80 0x562213fcf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.357+0000 7f3155a00640 1 -- 172.21.15.47:0/785119570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbd870 msgr2=0x562213fcfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.357+0000 7f3155a00640 1 --2- 172.21.15.47:0/785119570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbd870 0x562213fcfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.357+0000 7f3155a00640 1 -- 172.21.15.47:0/785119570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562213d007d0 con 0x562213fc29e0 2024-03-21T11:43:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.358+0000 7f3155a00640 1 --2- 172.21.15.47:0/785119570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fc29e0 0x562213fd6af0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f314002c7b0 tx=0x7f3140044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f78b0f2336f55ad1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.358+0000 7f313aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.358+0000 7f313aa00640 1 -- 172.21.15.47:0/785119570 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140004990 con 0x562213fc29e0 2024-03-21T11:43:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.359+0000 7f313aa00640 1 -- 172.21.15.47:0/785119570 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f314003d450 con 0x562213fc29e0 2024-03-21T11:43:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.359+0000 7f315a602740 1 -- 172.21.15.47:0/785119570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3140004ba0 con 0x562213fc29e0 2024-03-21T11:43:12.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.359+0000 7f313aa00640 1 -- 172.21.15.47:0/785119570 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f314003d650 con 0x562213fc29e0 2024-03-21T11:43:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.359+0000 7f315a602740 10 client.31846 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.359+0000 7f315a602740 1 -- 172.21.15.47:0/785119570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562213c363f0 con 0x562213fc29e0 2024-03-21T11:43:12.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.360+0000 7f313aa00640 1 -- 172.21.15.47:0/785119570 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3140004ba0 con 0x562213fc29e0 2024-03-21T11:43:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.360+0000 7f313aa00640 1 -- 172.21.15.47:0/785119570 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3140047070 con 0x562213fc29e0 2024-03-21T11:43:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.360+0000 7f315a602740 10 client.31846 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.360+0000 7f315a602740 10 client.31846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.361+0000 7f315a602740 20 client.31846 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.361+0000 7f315a602740 1 --2- 172.21.15.47:0/785119570 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562213fdafe0 0x562213fdd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.361+0000 7f315a602740 4 client.31846 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.361+0000 7f315a602740 1 -- 172.21.15.47:0/785119570 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562213f2f790 con 0x562213fdafe0 2024-03-21T11:43:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.364+0000 7f3155000640 1 --2- 172.21.15.47:0/785119570 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562213fdafe0 0x562213fdd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.367+0000 7f3155000640 1 --2- 172.21.15.47:0/785119570 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562213fdafe0 0x562213fdd3d0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88edd74949ecee19 server_cookie=f76325733ed6bdc8 in_seq=0 out_seq=0 2024-03-21T11:43:12.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.367+0000 7f313aa00640 10 client.31846 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:12.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.375+0000 7f313aa00640 1 -- 172.21.15.47:0/785119570 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562213f2f790 con 0x562213fdafe0 2024-03-21T11:43:12.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.375+0000 7f313aa00640 10 client.31846 handle_command_reply: tid=0 2024-03-21T11:43:12.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.375+0000 7f315a602740 1 client.31846 shutdown 2024-03-21T11:43:12.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.376+0000 7f315a602740 1 -- 172.21.15.47:0/785119570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fc29e0 msgr2=0x562213fd6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:12.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.376+0000 7f315a602740 1 --2- 172.21.15.47:0/785119570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fc29e0 0x562213fd6af0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f314002c7b0 tx=0x7f3140044000 comp rx=0 tx=0).stop 2024-03-21T11:43:12.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.377+0000 7f315a602740 1 -- 172.21.15.47:0/785119570 shutdown_connections 2024-03-21T11:43:12.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.377+0000 7f315a602740 1 --2- 172.21.15.47:0/785119570 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562213fdafe0 0x562213fdd3d0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.377+0000 7f315a602740 1 --2- 172.21.15.47:0/785119570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fc29e0 0x562213fd6af0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.377+0000 7f313aa00640 0 client.31846 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:12.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.377+0000 7f315a602740 1 --2- 172.21.15.47:0/785119570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbd870 0x562213fcfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.377+0000 7f315a602740 1 --2- 172.21.15.47:0/785119570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fbce80 0x562213fcf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.377+0000 7f315a602740 1 -- 172.21.15.47:0/785119570 >> 172.21.15.47:0/785119570 conn(0x562213f85200 msgr2=0x562213fb7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:12.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.377+0000 7f315a602740 1 -- 172.21.15.47:0/785119570 shutdown_connections 2024-03-21T11:43:12.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.377+0000 7f315a602740 1 -- 172.21.15.47:0/785119570 wait complete. 2024-03-21T11:43:12.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.377+0000 7f315a602740 20 client.31846 trim_cache size 0 max 16384 2024-03-21T11:43:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.412+0000 7f315a602740 1 Processor -- start 2024-03-21T11:43:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.412+0000 7f315a602740 1 -- start start 2024-03-21T11:43:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.412+0000 7f315a602740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbce80 0x562213fbba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.412+0000 7f315a602740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fbd870 0x562213fd6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.412+0000 7f315a602740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fc29e0 0x562213fbbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.412+0000 7f315a602740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315019f680 con 0x562213fbce80 2024-03-21T11:43:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.412+0000 7f315a602740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31501a1000 con 0x562213fc29e0 2024-03-21T11:43:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.412+0000 7f315a602740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315019fce0 con 0x562213fbd870 2024-03-21T11:43:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.413+0000 7f314fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fbd870 0x562213fd6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.413+0000 7f314fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fbd870 0x562213fd6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50664/0 (socket says 172.21.15.47:50664) 2024-03-21T11:43:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.413+0000 7f314fe00640 1 -- 172.21.15.47:0/301217437 learned_addr learned my addr 172.21.15.47:0/301217437 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.413+0000 7f3155a00640 1 --2- 172.21.15.47:0/301217437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fc29e0 0x562213fbbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.414+0000 7f314fe00640 1 -- 172.21.15.47:0/301217437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fc29e0 msgr2=0x562213fbbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.414+0000 7f314fe00640 1 --2- 172.21.15.47:0/301217437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fc29e0 0x562213fbbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.414+0000 7f314fe00640 1 -- 172.21.15.47:0/301217437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbce80 msgr2=0x562213fbba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.414+0000 7f314fe00640 1 --2- 172.21.15.47:0/301217437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbce80 0x562213fbba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.414+0000 7f314fe00640 1 -- 172.21.15.47:0/301217437 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562213bf60f0 con 0x562213fbd870 2024-03-21T11:43:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.414+0000 7f314fe00640 1 --2- 172.21.15.47:0/301217437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fbd870 0x562213fd6ab0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f313c00d6c0 tx=0x7f313c00db80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35e08d5a7b3ee639 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.415+0000 7f3145600640 1 -- 172.21.15.47:0/301217437 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c019020 con 0x562213fbd870 2024-03-21T11:43:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.415+0000 7f3145600640 1 -- 172.21.15.47:0/301217437 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f313c00fca0 con 0x562213fbd870 2024-03-21T11:43:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.415+0000 7f3145600640 1 -- 172.21.15.47:0/301217437 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c014e10 con 0x562213fbd870 2024-03-21T11:43:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.416+0000 7f315a602740 1 -- 172.21.15.47:0/301217437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fbd870 msgr2=0x562213fd6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.416+0000 7f315a602740 1 --2- 172.21.15.47:0/301217437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fbd870 0x562213fd6ab0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f313c00d6c0 tx=0x7f313c00db80 comp rx=0 tx=0).stop 2024-03-21T11:43:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.416+0000 7f315a602740 1 -- 172.21.15.47:0/301217437 shutdown_connections 2024-03-21T11:43:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.416+0000 7f315a602740 1 --2- 172.21.15.47:0/301217437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fc29e0 0x562213fbbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.416+0000 7f315a602740 1 --2- 172.21.15.47:0/301217437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fbd870 0x562213fd6ab0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.416+0000 7f315a602740 1 --2- 172.21.15.47:0/301217437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbce80 0x562213fbba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.416+0000 7f315a602740 1 -- 172.21.15.47:0/301217437 >> 172.21.15.47:0/301217437 conn(0x562213f85200 msgr2=0x562213fbe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.416+0000 7f315a602740 1 -- 172.21.15.47:0/301217437 shutdown_connections 2024-03-21T11:43:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.416+0000 7f315a602740 1 -- 172.21.15.47:0/301217437 wait complete. 2024-03-21T11:43:12.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.417+0000 7f315a602740 1 -- start start 2024-03-21T11:43:12.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.417+0000 7f315a602740 1 Processor -- start 2024-03-21T11:43:12.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.418+0000 7f315a602740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fbce80 0x562213fd4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.418+0000 7f315a602740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbd870 0x562213fd4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.418+0000 7f315a602740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fc29e0 0x562213fd5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.418+0000 7f315a602740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31501a09a0 con 0x562213fbd870 2024-03-21T11:43:12.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.418+0000 7f315a602740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31501a0340 con 0x562213fc29e0 2024-03-21T11:43:12.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.418+0000 7f315a602740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315019d840 con 0x562213fbce80 2024-03-21T11:43:12.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.418+0000 7f315a602740 10 client.0 _ll_register_callbacks cb 0x7f314002c980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:12.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.419+0000 7f3155a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fc29e0 0x562213fd5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.419+0000 7f3155a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fc29e0 0x562213fd5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57982/0 (socket says 172.21.15.47:57982) 2024-03-21T11:43:12.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.419+0000 7f3155a00640 1 -- 172.21.15.47:0/1674053276 learned_addr learned my addr 172.21.15.47:0/1674053276 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:12.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.419+0000 7f314fe00640 1 --2- 172.21.15.47:0/1674053276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbd870 0x562213fd4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.420+0000 7f314fe00640 1 -- 172.21.15.47:0/1674053276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fbce80 msgr2=0x562213fd4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.420+0000 7f314fe00640 1 --2- 172.21.15.47:0/1674053276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fbce80 0x562213fd4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.420+0000 7f314fe00640 1 -- 172.21.15.47:0/1674053276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fc29e0 msgr2=0x562213fd5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.420+0000 7f314fe00640 1 --2- 172.21.15.47:0/1674053276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fc29e0 0x562213fd5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.420+0000 7f314fe00640 1 -- 172.21.15.47:0/1674053276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562213fd6030 con 0x562213fbd870 2024-03-21T11:43:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.421+0000 7f314fe00640 1 --2- 172.21.15.47:0/1674053276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbd870 0x562213fd4a80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x562213fd0570 tx=0x7f313c014690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbf5c0025675769 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.421+0000 7f313aa00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:43:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.421+0000 7f313aa00640 1 -- 172.21.15.47:0/1674053276 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c019030 con 0x562213fbd870 2024-03-21T11:43:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.422+0000 7f313aa00640 1 -- 172.21.15.47:0/1674053276 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f313c014d10 con 0x562213fbd870 2024-03-21T11:43:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.422+0000 7f315a602740 1 -- 172.21.15.47:0/1674053276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f313c00fca0 con 0x562213fbd870 2024-03-21T11:43:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.422+0000 7f313aa00640 1 -- 172.21.15.47:0/1674053276 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f313c030050 con 0x562213fbd870 2024-03-21T11:43:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.423+0000 7f315a602740 10 client.21273 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.423+0000 7f315a602740 1 -- 172.21.15.47:0/1674053276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562214036b60 con 0x562213fbd870 2024-03-21T11:43:12.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.424+0000 7f313aa00640 1 -- 172.21.15.47:0/1674053276 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f313c00fca0 con 0x562213fbd870 2024-03-21T11:43:12.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.424+0000 7f313aa00640 1 -- 172.21.15.47:0/1674053276 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f313c01e420 con 0x562213fbd870 2024-03-21T11:43:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.424+0000 7f315a602740 10 client.21273 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.424+0000 7f315a602740 10 client.21273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.424+0000 7f315a602740 20 client.21273 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.425+0000 7f315a602740 1 --2- 172.21.15.47:0/1674053276 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5622140374b0 0x5622140398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.425+0000 7f315a602740 4 client.21273 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:43:12.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.425+0000 7f315a602740 1 -- 172.21.15.47:0/1674053276 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562213f29ae0 con 0x5622140374b0 2024-03-21T11:43:12.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.428+0000 7f3155000640 1 --2- 172.21.15.47:0/1674053276 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5622140374b0 0x5622140398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.429+0000 7f3155000640 1 --2- 172.21.15.47:0/1674053276 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5622140374b0 0x5622140398d0 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d69443f5e5ca5dd server_cookie=b4a46977d981ab2c in_seq=0 out_seq=0 2024-03-21T11:43:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.429+0000 7f313aa00640 10 client.21273 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.430+0000 7f313aa00640 1 -- 172.21.15.47:0/1674053276 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562213f29ae0 con 0x5622140374b0 2024-03-21T11:43:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.430+0000 7f313aa00640 10 client.21273 handle_command_reply: tid=0 2024-03-21T11:43:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.430+0000 7f315a602740 1 client.21273 shutdown 2024-03-21T11:43:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.431+0000 7f315a602740 1 -- 172.21.15.47:0/1674053276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbd870 msgr2=0x562213fd4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:12.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.431+0000 7f315a602740 1 --2- 172.21.15.47:0/1674053276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbd870 0x562213fd4a80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x562213fd0570 tx=0x7f313c014690 comp rx=0 tx=0).stop 2024-03-21T11:43:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.431+0000 7f315a602740 1 -- 172.21.15.47:0/1674053276 shutdown_connections 2024-03-21T11:43:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.431+0000 7f315a602740 1 --2- 172.21.15.47:0/1674053276 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5622140374b0 0x5622140398d0 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.431+0000 7f315a602740 1 --2- 172.21.15.47:0/1674053276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562213fc29e0 0x562213fd5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.431+0000 7f315a602740 1 --2- 172.21.15.47:0/1674053276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562213fbd870 0x562213fd4a80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.431+0000 7f315a602740 1 --2- 172.21.15.47:0/1674053276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562213fbce80 0x562213fd4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.431+0000 7f315a602740 1 -- 172.21.15.47:0/1674053276 >> 172.21.15.47:0/1674053276 conn(0x562213f85200 msgr2=0x562213fb7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:12.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.433+0000 7f313aa00640 0 client.21273 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:12.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.433+0000 7f315a602740 1 -- 172.21.15.47:0/1674053276 shutdown_connections 2024-03-21T11:43:12.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.433+0000 7f315a602740 1 -- 172.21.15.47:0/1674053276 wait complete. 2024-03-21T11:43:12.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.433+0000 7f315a602740 20 client.21273 trim_cache size 0 max 16384 2024-03-21T11:43:12.435 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:43:12.435 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:43:12.435 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:43:12.435 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:43:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.435+0000 7f3157a00640 1 -- 172.21.15.47:0/592239232 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31340a3070 msgr2=0x7f31340a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.435+0000 7f3157a00640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31340a3070 0x7f31340a5530 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f314002cfd0 tx=0x7f31400056a0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.435+0000 7f3157a00640 1 -- 172.21.15.47:0/592239232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31501038c0 msgr2=0x7f3150070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.435+0000 7f3157a00640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31501038c0 0x7f3150070020 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f314800b9c0 tx=0x7f314800be80 comp rx=0 tx=0).stop 2024-03-21T11:43:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.435+0000 7f3157a00640 1 -- 172.21.15.47:0/592239232 shutdown_connections 2024-03-21T11:43:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.435+0000 7f3157a00640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f31340a3070 0x7f31340a5530 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.435+0000 7f3157a00640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3150105470 0x7f3150198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.435+0000 7f3157a00640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3150104ab0 0x7f3150070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.435+0000 7f3157a00640 1 --2- 172.21.15.47:0/592239232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31501038c0 0x7f3150070020 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.435+0000 7f3157a00640 1 -- 172.21.15.47:0/592239232 >> 172.21.15.47:0/592239232 conn(0x7f31500fee50 msgr2=0x7f3150107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:12.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.436+0000 7f3157a00640 1 -- 172.21.15.47:0/592239232 shutdown_connections 2024-03-21T11:43:12.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.436+0000 7f3157a00640 1 -- 172.21.15.47:0/592239232 wait complete. 2024-03-21T11:43:12.470 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:43:12.470 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:43:12.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.761+0000 7fdc7dc00640 1 -- 172.21.15.47:0/831868956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc78107460 msgr2=0x7fdc78109850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:12.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.761+0000 7fdc7dc00640 1 --2- 172.21.15.47:0/831868956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc78107460 0x7fdc78109850 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009ca0 tx=0x7fdc7002c140 comp rx=0 tx=0).stop 2024-03-21T11:43:12.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.761+0000 7fdc7dc00640 1 -- 172.21.15.47:0/831868956 shutdown_connections 2024-03-21T11:43:12.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.761+0000 7fdc7dc00640 1 --2- 172.21.15.47:0/831868956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc78107460 0x7fdc78109850 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.761+0000 7fdc7dc00640 1 --2- 172.21.15.47:0/831868956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc78104b30 0x7fdc78106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.761+0000 7fdc7dc00640 1 --2- 172.21.15.47:0/831868956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc78102200 0x7fdc781045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.761+0000 7fdc7dc00640 1 -- 172.21.15.47:0/831868956 >> 172.21.15.47:0/831868956 conn(0x7fdc780fbb50 msgr2=0x7fdc780fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:12.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.762+0000 7fdc7dc00640 1 -- 172.21.15.47:0/831868956 shutdown_connections 2024-03-21T11:43:12.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.762+0000 7fdc7dc00640 1 -- 172.21.15.47:0/831868956 wait complete. 2024-03-21T11:43:12.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.764+0000 7fdc7dc00640 1 Processor -- start 2024-03-21T11:43:12.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.764+0000 7fdc7dc00640 1 -- start start 2024-03-21T11:43:12.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.765+0000 7fdc7dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc78102200 0x7fdc78101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.765+0000 7fdc7dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc78104b30 0x7fdc781001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.765+0000 7fdc7dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc78107460 0x7fdc781006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.765+0000 7fdc7dc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7810d3d0 con 0x7fdc78102200 2024-03-21T11:43:12.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.765+0000 7fdc7dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7810d250 con 0x7fdc78107460 2024-03-21T11:43:12.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.765+0000 7fdc7dc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7810d550 con 0x7fdc78104b30 2024-03-21T11:43:12.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.765+0000 7fdc76a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc78102200 0x7fdc78101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.765+0000 7fdc77400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc78107460 0x7fdc781006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.765+0000 7fdc77400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc78107460 0x7fdc781006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57998/0 (socket says 172.21.15.47:57998) 2024-03-21T11:43:12.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.765+0000 7fdc77400640 1 -- 172.21.15.47:0/3746068995 learned_addr learned my addr 172.21.15.47:0/3746068995 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:12.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.766+0000 7fdc77400640 1 -- 172.21.15.47:0/3746068995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc78104b30 msgr2=0x7fdc781001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:12.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.766+0000 7fdc77400640 1 --2- 172.21.15.47:0/3746068995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc78104b30 0x7fdc781001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.766+0000 7fdc77400640 1 -- 172.21.15.47:0/3746068995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc78102200 msgr2=0x7fdc78101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:12.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.766+0000 7fdc77400640 1 --2- 172.21.15.47:0/3746068995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc78102200 0x7fdc78101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:12.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.766+0000 7fdc77400640 1 -- 172.21.15.47:0/3746068995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc78100f80 con 0x7fdc78107460 2024-03-21T11:43:12.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.767+0000 7fdc77400640 1 --2- 172.21.15.47:0/3746068995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc78107460 0x7fdc781006f0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fdc700325e0 tx=0x7fdc70034d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:12.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.767+0000 7fdc6f400640 1 -- 172.21.15.47:0/3746068995 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70041030 con 0x7fdc78107460 2024-03-21T11:43:12.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.767+0000 7fdc7dc00640 1 -- 172.21.15.47:0/3746068995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc781a1b00 con 0x7fdc78107460 2024-03-21T11:43:12.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.767+0000 7fdc6f400640 1 -- 172.21.15.47:0/3746068995 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc70003970 con 0x7fdc78107460 2024-03-21T11:43:12.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.767+0000 7fdc6f400640 1 -- 172.21.15.47:0/3746068995 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70039040 con 0x7fdc78107460 2024-03-21T11:43:12.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.767+0000 7fdc7dc00640 1 -- 172.21.15.47:0/3746068995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc781a20a0 con 0x7fdc78107460 2024-03-21T11:43:12.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.771+0000 7fdc7dc00640 1 -- 172.21.15.47:0/3746068995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc78067c00 con 0x7fdc78107460 2024-03-21T11:43:12.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.773+0000 7fdc6f400640 1 -- 172.21.15.47:0/3746068995 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdc70044070 con 0x7fdc78107460 2024-03-21T11:43:12.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.775+0000 7fdc6f400640 1 --2- 172.21.15.47:0/3746068995 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc5c0a2fa0 0x7fdc5c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:12.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.776+0000 7fdc6f400640 1 -- 172.21.15.47:0/3746068995 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdc700f1910 con 0x7fdc78107460 2024-03-21T11:43:12.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.776+0000 7fdc76a00640 1 --2- 172.21.15.47:0/3746068995 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc5c0a2fa0 0x7fdc5c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:12.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.778+0000 7fdc76a00640 1 --2- 172.21.15.47:0/3746068995 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc5c0a2fa0 0x7fdc5c0a5460 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdc68005d90 tx=0x7fdc68005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:12.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:12.788+0000 7fdc6f400640 1 -- 172.21.15.47:0/3746068995 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdc700be350 con 0x7fdc78107460 2024-03-21T11:43:12.962 INFO:tasks.workunit.client.0.smithi047.stdout: 10 159355 23.07 MB/sec execute 236 sec latency 171.898 ms 2024-03-21T11:43:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:12 smithi100 ceph-mon[36245]: pgmap v996: 129 pgs: 129 active+clean; 555 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.4 MiB/s wr, 597 op/s 2024-03-21T11:43:13.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.229+0000 7fdc7dc00640 1 -- 172.21.15.47:0/3746068995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc780622a0 con 0x7fdc78107460 2024-03-21T11:43:13.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.231+0000 7fdc6f400640 1 -- 172.21.15.47:0/3746068995 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fdc700bdc50 con 0x7fdc78107460 2024-03-21T11:43:13.232 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:43:13.232 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:39:47.303461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:43:13.232 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-21T11:43:13.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.234+0000 7fdc7dc00640 1 -- 172.21.15.47:0/3746068995 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc5c0a2fa0 msgr2=0x7fdc5c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.234+0000 7fdc7dc00640 1 --2- 172.21.15.47:0/3746068995 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc5c0a2fa0 0x7fdc5c0a5460 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdc68005d90 tx=0x7fdc68005d20 comp rx=0 tx=0).stop 2024-03-21T11:43:13.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.234+0000 7fdc7dc00640 1 -- 172.21.15.47:0/3746068995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc78107460 msgr2=0x7fdc781006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.234+0000 7fdc7dc00640 1 --2- 172.21.15.47:0/3746068995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc78107460 0x7fdc781006f0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fdc700325e0 tx=0x7fdc70034d80 comp rx=0 tx=0).stop 2024-03-21T11:43:13.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.234+0000 7fdc7dc00640 1 -- 172.21.15.47:0/3746068995 shutdown_connections 2024-03-21T11:43:13.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.234+0000 7fdc7dc00640 1 --2- 172.21.15.47:0/3746068995 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc5c0a2fa0 0x7fdc5c0a5460 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.234+0000 7fdc7dc00640 1 --2- 172.21.15.47:0/3746068995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc78107460 0x7fdc781006f0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.234+0000 7fdc7dc00640 1 --2- 172.21.15.47:0/3746068995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc78104b30 0x7fdc781001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.234+0000 7fdc7dc00640 1 --2- 172.21.15.47:0/3746068995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc78102200 0x7fdc78101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.234+0000 7fdc7dc00640 1 -- 172.21.15.47:0/3746068995 >> 172.21.15.47:0/3746068995 conn(0x7fdc780fbb50 msgr2=0x7fdc78102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:13.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.235+0000 7fdc7dc00640 1 -- 172.21.15.47:0/3746068995 shutdown_connections 2024-03-21T11:43:13.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.235+0000 7fdc7dc00640 1 -- 172.21.15.47:0/3746068995 wait complete. 2024-03-21T11:43:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:12 smithi102 ceph-mon[36748]: pgmap v996: 129 pgs: 129 active+clean; 555 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.4 MiB/s wr, 597 op/s 2024-03-21T11:43:13.258 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:12 smithi047 ceph-mon[31351]: pgmap v996: 129 pgs: 129 active+clean; 555 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.4 MiB/s wr, 597 op/s 2024-03-21T11:43:13.262 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:43:13.262 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:43:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.510+0000 7f7d5dc00640 1 -- 172.21.15.47:0/3086187258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d58104930 msgr2=0x7f7d58106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.510+0000 7f7d5dc00640 1 --2- 172.21.15.47:0/3086187258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d58104930 0x7f7d58106d20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009860 tx=0x7f7d4402bb70 comp rx=0 tx=0).stop 2024-03-21T11:43:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.511+0000 7f7d5dc00640 1 -- 172.21.15.47:0/3086187258 shutdown_connections 2024-03-21T11:43:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.511+0000 7f7d5dc00640 1 --2- 172.21.15.47:0/3086187258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d58107260 0x7f7d581096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.511+0000 7f7d5dc00640 1 --2- 172.21.15.47:0/3086187258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d58104930 0x7f7d58106d20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.511+0000 7f7d5dc00640 1 --2- 172.21.15.47:0/3086187258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d58102000 0x7f7d581043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.511+0000 7f7d5dc00640 1 -- 172.21.15.47:0/3086187258 >> 172.21.15.47:0/3086187258 conn(0x7f7d580fba10 msgr2=0x7f7d580fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.511+0000 7f7d5dc00640 1 -- 172.21.15.47:0/3086187258 shutdown_connections 2024-03-21T11:43:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.511+0000 7f7d5dc00640 1 -- 172.21.15.47:0/3086187258 wait complete. 2024-03-21T11:43:13.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.512+0000 7f7d5dc00640 1 Processor -- start 2024-03-21T11:43:13.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.512+0000 7f7d5dc00640 1 -- start start 2024-03-21T11:43:13.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.515+0000 7f7d5dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d58102000 0x7f7d58199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.515+0000 7f7d5dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d58104930 0x7f7d58199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.515+0000 7f7d5dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d58107260 0x7f7d581a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.515+0000 7f7d5dc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5810d210 con 0x7f7d58107260 2024-03-21T11:43:13.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.515+0000 7f7d5dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5810d090 con 0x7f7d58102000 2024-03-21T11:43:13.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.515+0000 7f7d5dc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5810d390 con 0x7f7d58104930 2024-03-21T11:43:13.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.515+0000 7f7d57400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d58104930 0x7f7d58199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.515+0000 7f7d57e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d58102000 0x7f7d58199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.515+0000 7f7d57400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d58104930 0x7f7d58199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50684/0 (socket says 172.21.15.47:50684) 2024-03-21T11:43:13.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.515+0000 7f7d57400640 1 -- 172.21.15.47:0/1984458489 learned_addr learned my addr 172.21.15.47:0/1984458489 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.515+0000 7f7d5d200640 1 --2- 172.21.15.47:0/1984458489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d58107260 0x7f7d581a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.516+0000 7f7d5d200640 1 -- 172.21.15.47:0/1984458489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d58104930 msgr2=0x7f7d58199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.516+0000 7f7d5d200640 1 --2- 172.21.15.47:0/1984458489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d58104930 0x7f7d58199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.516+0000 7f7d5d200640 1 -- 172.21.15.47:0/1984458489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d58102000 msgr2=0x7f7d58199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.516+0000 7f7d5d200640 1 --2- 172.21.15.47:0/1984458489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d58102000 0x7f7d58199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.516+0000 7f7d5d200640 1 -- 172.21.15.47:0/1984458489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d581a16c0 con 0x7f7d58107260 2024-03-21T11:43:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.516+0000 7f7d57400640 1 --2- 172.21.15.47:0/1984458489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d58104930 0x7f7d58199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.516+0000 7f7d5d200640 1 --2- 172.21.15.47:0/1984458489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d58107260 0x7f7d581a1050 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009fd0 tx=0x7f7d5000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.517+0000 7f7d54c00640 1 -- 172.21.15.47:0/1984458489 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5000ce20 con 0x7f7d58107260 2024-03-21T11:43:13.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.517+0000 7f7d5dc00640 1 -- 172.21.15.47:0/1984458489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d581a19b0 con 0x7f7d58107260 2024-03-21T11:43:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.517+0000 7f7d5dc00640 1 -- 172.21.15.47:0/1984458489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d581a1ed0 con 0x7f7d58107260 2024-03-21T11:43:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.517+0000 7f7d54c00640 1 -- 172.21.15.47:0/1984458489 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d50005020 con 0x7f7d58107260 2024-03-21T11:43:13.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.517+0000 7f7d54c00640 1 -- 172.21.15.47:0/1984458489 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50010720 con 0x7f7d58107260 2024-03-21T11:43:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.520+0000 7f7d61f77740 1 Processor -- start 2024-03-21T11:43:13.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.521+0000 7f7d54c00640 1 -- 172.21.15.47:0/1984458489 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7d500108e0 con 0x7f7d58107260 2024-03-21T11:43:13.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.521+0000 7f7d61f77740 1 -- start start 2024-03-21T11:43:13.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d61f77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d61f77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5cb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d61f77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 0x55d17b5d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d61f77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17b5262c0 con 0x55d17b5cb3d0 2024-03-21T11:43:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d61f77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17b4cec90 con 0x55d17b5cbd90 2024-03-21T11:43:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d61f77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17b522360 con 0x55d17b5ca200 2024-03-21T11:43:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d57400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d57400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50686/0 (socket says 172.21.15.47:50686) 2024-03-21T11:43:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d57400640 1 -- 172.21.15.47:0/997786663 learned_addr learned my addr 172.21.15.47:0/997786663 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d57400640 1 --2- 172.21.15.47:0/997786663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 0x55d17b5d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d57e00640 1 --2- 172.21.15.47:0/997786663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d57400640 1 -- 172.21.15.47:0/997786663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 msgr2=0x55d17b5d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d57400640 1 --2- 172.21.15.47:0/997786663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 0x55d17b5d2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d57400640 1 -- 172.21.15.47:0/997786663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 msgr2=0x55d17b5cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d57400640 1 --2- 172.21.15.47:0/997786663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d57400640 1 -- 172.21.15.47:0/997786663 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d17b4f3080 con 0x55d17b5ca200 2024-03-21T11:43:13.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d54c00640 1 --2- 172.21.15.47:0/1984458489 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d3c0a2fa0 0x7f7d3c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.522+0000 7f7d57e00640 1 --2- 172.21.15.47:0/997786663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:43:13.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.523+0000 7f7d54c00640 1 -- 172.21.15.47:0/1984458489 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7d500cfd10 con 0x7f7d58107260 2024-03-21T11:43:13.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.523+0000 7f7d57400640 1 --2- 172.21.15.47:0/1984458489 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d3c0a2fa0 0x7f7d3c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.523+0000 7f7d57400640 1 --2- 172.21.15.47:0/997786663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5ca620 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f7d440349e0 tx=0x7f7d44035f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8315b550004b64a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:13.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.523+0000 7f7d4d600640 1 -- 172.21.15.47:0/997786663 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d44039040 con 0x55d17b5ca200 2024-03-21T11:43:13.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.524+0000 7f7d4d600640 1 -- 172.21.15.47:0/997786663 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d4402cab0 con 0x55d17b5ca200 2024-03-21T11:43:13.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.524+0000 7f7d4d600640 1 -- 172.21.15.47:0/997786663 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d44032e80 con 0x55d17b5ca200 2024-03-21T11:43:13.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.524+0000 7f7d61f77740 1 -- 172.21.15.47:0/997786663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 msgr2=0x55d17b5ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.524+0000 7f7d61f77740 1 --2- 172.21.15.47:0/997786663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5ca620 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f7d440349e0 tx=0x7f7d44035f70 comp rx=0 tx=0).stop 2024-03-21T11:43:13.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.524+0000 7f7d57400640 1 --2- 172.21.15.47:0/1984458489 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d3c0a2fa0 0x7f7d3c0a5460 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7d44032540 tx=0x7f7d44048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:13.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.524+0000 7f7d61f77740 1 -- 172.21.15.47:0/997786663 shutdown_connections 2024-03-21T11:43:13.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.524+0000 7f7d61f77740 1 --2- 172.21.15.47:0/997786663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 0x55d17b5d2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.524+0000 7f7d61f77740 1 --2- 172.21.15.47:0/997786663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.524+0000 7f7d61f77740 1 --2- 172.21.15.47:0/997786663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5ca620 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.524+0000 7f7d61f77740 1 -- 172.21.15.47:0/997786663 >> 172.21.15.47:0/997786663 conn(0x55d17b5c55f0 msgr2=0x55d17b5c59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:13.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.525+0000 7f7d61f77740 1 -- 172.21.15.47:0/997786663 shutdown_connections 2024-03-21T11:43:13.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.525+0000 7f7d61f77740 1 -- 172.21.15.47:0/997786663 wait complete. 2024-03-21T11:43:13.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.525+0000 7f7d61f77740 1 -- start start 2024-03-21T11:43:13.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.525+0000 7f7d61f77740 1 Processor -- start 2024-03-21T11:43:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.526+0000 7f7d61f77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5d4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.526+0000 7f7d61f77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.526+0000 7f7d61f77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 0x55d17b5db9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.526+0000 7f7d61f77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17b5d2da0 con 0x55d17b5cb3d0 2024-03-21T11:43:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.526+0000 7f7d61f77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17b4e20d0 con 0x55d17b5cbd90 2024-03-21T11:43:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.526+0000 7f7d61f77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17b5d2f20 con 0x55d17b5ca200 2024-03-21T11:43:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.526+0000 7f7d57e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5d4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.526+0000 7f7d61f77740 10 client.0 _ll_register_callbacks cb 0x55d17b4e6e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.526+0000 7f7d57e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5d4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50696/0 (socket says 172.21.15.47:50696) 2024-03-21T11:43:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.526+0000 7f7d57e00640 1 -- 172.21.15.47:0/858021789 learned_addr learned my addr 172.21.15.47:0/858021789 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.526+0000 7f7d5d200640 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 0x55d17b5db9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.526+0000 7f7d57400640 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.527+0000 7f7d57e00640 1 -- 172.21.15.47:0/858021789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 msgr2=0x55d17b5db9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.527+0000 7f7d57e00640 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 0x55d17b5db9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.527+0000 7f7d57e00640 1 -- 172.21.15.47:0/858021789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 msgr2=0x55d17b5d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.527+0000 7f7d57e00640 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5d4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.527+0000 7f7d57e00640 1 -- 172.21.15.47:0/858021789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d17b23a3f0 con 0x55d17b5ca200 2024-03-21T11:43:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.527+0000 7f7d5d200640 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 0x55d17b5db9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.527+0000 7f7d57400640 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.527+0000 7f7d57e00640 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5d4440 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f7d48013710 tx=0x7f7d48013bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d59e942b65ac82af server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.527+0000 7f7d42000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.527+0000 7f7d42000640 1 -- 172.21.15.47:0/858021789 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d48011e30 con 0x55d17b5ca200 2024-03-21T11:43:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.527+0000 7f7d42000640 1 -- 172.21.15.47:0/858021789 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7d48004020 con 0x55d17b5ca200 2024-03-21T11:43:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.528+0000 7f7d61f77740 1 -- 172.21.15.47:0/858021789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d4402cab0 con 0x55d17b5ca200 2024-03-21T11:43:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.528+0000 7f7d42000640 1 -- 172.21.15.47:0/858021789 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d48004220 con 0x55d17b5ca200 2024-03-21T11:43:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.528+0000 7f7d61f77740 10 client.31861 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:13.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.528+0000 7f7d61f77740 1 -- 172.21.15.47:0/858021789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d17b3047d0 con 0x55d17b5ca200 2024-03-21T11:43:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.528+0000 7f7d42000640 1 -- 172.21.15.47:0/858021789 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d4402cab0 con 0x55d17b5ca200 2024-03-21T11:43:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.528+0000 7f7d42000640 1 -- 172.21.15.47:0/858021789 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7d4801b4c0 con 0x55d17b5ca200 2024-03-21T11:43:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.528+0000 7f7d61f77740 10 client.31861 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.528+0000 7f7d61f77740 10 client.31861 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:43:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.528+0000 7f7d61f77740 20 client.31861 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.528+0000 7f7d61f77740 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55d17b5dff60 0x55d17b5e23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.528+0000 7f7d61f77740 4 client.31861 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.528+0000 7f7d61f77740 1 -- 172.21.15.47:0/858021789 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d17b533800 con 0x55d17b5dff60 2024-03-21T11:43:13.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.529+0000 7f7d57400640 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55d17b5dff60 0x55d17b5e23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.529+0000 7f7d57400640 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55d17b5dff60 0x55d17b5e23e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5beb34b2edf639a0 server_cookie=bda2929502036dc7 in_seq=0 out_seq=0 2024-03-21T11:43:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.529+0000 7f7d42000640 10 client.31861 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:43:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.530+0000 7f7d42000640 1 -- 172.21.15.47:0/858021789 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d17b533800 con 0x55d17b5dff60 2024-03-21T11:43:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.530+0000 7f7d42000640 10 client.31861 handle_command_reply: tid=0 2024-03-21T11:43:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.531+0000 7f7d61f77740 1 client.31861 shutdown 2024-03-21T11:43:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.531+0000 7f7d61f77740 1 -- 172.21.15.47:0/858021789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 msgr2=0x55d17b5d4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.531+0000 7f7d61f77740 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5d4440 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f7d48013710 tx=0x7f7d48013bd0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.531+0000 7f7d61f77740 1 -- 172.21.15.47:0/858021789 shutdown_connections 2024-03-21T11:43:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.531+0000 7f7d61f77740 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55d17b5dff60 0x55d17b5e23e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.531+0000 7f7d61f77740 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 0x55d17b5db9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.531+0000 7f7d61f77740 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.531+0000 7f7d42000640 0 client.31861 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:43:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.531+0000 7f7d61f77740 1 --2- 172.21.15.47:0/858021789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5d4440 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.531+0000 7f7d61f77740 1 -- 172.21.15.47:0/858021789 >> 172.21.15.47:0/858021789 conn(0x55d17b5c55f0 msgr2=0x55d17b5c8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.531+0000 7f7d61f77740 1 -- 172.21.15.47:0/858021789 shutdown_connections 2024-03-21T11:43:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.531+0000 7f7d61f77740 1 -- 172.21.15.47:0/858021789 wait complete. 2024-03-21T11:43:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.531+0000 7f7d61f77740 20 client.31861 trim_cache size 0 max 16384 2024-03-21T11:43:13.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.547+0000 7f7d61f77740 1 Processor -- start 2024-03-21T11:43:13.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.547+0000 7f7d61f77740 1 -- start start 2024-03-21T11:43:13.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.547+0000 7f7d61f77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5ca200 0x55d17b5e6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.547+0000 7f7d61f77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5cb3d0 0x55d17b5db980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.547+0000 7f7d61f77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cbd90 0x55d17b5e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.547+0000 7f7d61f77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3800f9a0 con 0x55d17b5cbd90 2024-03-21T11:43:13.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.547+0000 7f7d61f77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d38011320 con 0x55d17b5ca200 2024-03-21T11:43:13.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.547+0000 7f7d61f77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d38010000 con 0x55d17b5cb3d0 2024-03-21T11:43:13.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.547+0000 7f7d57e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5ca200 0x55d17b5e6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.547+0000 7f7d57400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5cb3d0 0x55d17b5db980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.547+0000 7f7d57e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5ca200 0x55d17b5e6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58046/0 (socket says 172.21.15.47:58046) 2024-03-21T11:43:13.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.547+0000 7f7d5d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cbd90 0x55d17b5e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.547+0000 7f7d57e00640 1 -- 172.21.15.47:0/2079566774 learned_addr learned my addr 172.21.15.47:0/2079566774 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:13.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.548+0000 7f7d57400640 1 -- 172.21.15.47:0/2079566774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5ca200 msgr2=0x55d17b5e6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.548+0000 7f7d57400640 1 --2- 172.21.15.47:0/2079566774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5ca200 0x55d17b5e6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.548+0000 7f7d57400640 1 -- 172.21.15.47:0/2079566774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cbd90 msgr2=0x55d17b5e71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.548+0000 7f7d57400640 1 --2- 172.21.15.47:0/2079566774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cbd90 0x55d17b5e71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.548+0000 7f7d57400640 1 -- 172.21.15.47:0/2079566774 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d17b5cae50 con 0x55d17b5cb3d0 2024-03-21T11:43:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.548+0000 7f7d5d200640 1 --2- 172.21.15.47:0/2079566774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cbd90 0x55d17b5e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.548+0000 7f7d57400640 1 --2- 172.21.15.47:0/2079566774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5cb3d0 0x55d17b5db980 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x55d17b5d5920 tx=0x7f7d44053000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a8829f5f8d6b8a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.548+0000 7f7d4cc00640 1 -- 172.21.15.47:0/2079566774 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d44039040 con 0x55d17b5cb3d0 2024-03-21T11:43:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.548+0000 7f7d4cc00640 1 -- 172.21.15.47:0/2079566774 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d44061030 con 0x55d17b5cb3d0 2024-03-21T11:43:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.548+0000 7f7d4cc00640 1 -- 172.21.15.47:0/2079566774 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d440037f0 con 0x55d17b5cb3d0 2024-03-21T11:43:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.549+0000 7f7d61f77740 1 -- 172.21.15.47:0/2079566774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5cb3d0 msgr2=0x55d17b5db980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.549+0000 7f7d61f77740 1 --2- 172.21.15.47:0/2079566774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5cb3d0 0x55d17b5db980 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x55d17b5d5920 tx=0x7f7d44053000 comp rx=0 tx=0).stop 2024-03-21T11:43:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.549+0000 7f7d61f77740 1 -- 172.21.15.47:0/2079566774 shutdown_connections 2024-03-21T11:43:13.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.549+0000 7f7d61f77740 1 --2- 172.21.15.47:0/2079566774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cbd90 0x55d17b5e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.549+0000 7f7d61f77740 1 --2- 172.21.15.47:0/2079566774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5cb3d0 0x55d17b5db980 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.549+0000 7f7d61f77740 1 --2- 172.21.15.47:0/2079566774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5ca200 0x55d17b5e6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.549+0000 7f7d61f77740 1 -- 172.21.15.47:0/2079566774 >> 172.21.15.47:0/2079566774 conn(0x55d17b5c55f0 msgr2=0x55d17b5c90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.549+0000 7f7d61f77740 1 -- 172.21.15.47:0/2079566774 shutdown_connections 2024-03-21T11:43:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.549+0000 7f7d61f77740 1 -- 172.21.15.47:0/2079566774 wait complete. 2024-03-21T11:43:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.550+0000 7f7d61f77740 1 -- start start 2024-03-21T11:43:13.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.550+0000 7f7d61f77740 1 Processor -- start 2024-03-21T11:43:13.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d61f77740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5d96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d61f77740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5d9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d61f77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 0x55d17b5dabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d61f77740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d38010cc0 con 0x55d17b5cb3d0 2024-03-21T11:43:13.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d61f77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d38010660 con 0x55d17b5cbd90 2024-03-21T11:43:13.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d61f77740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3800db60 con 0x55d17b5ca200 2024-03-21T11:43:13.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d61f77740 10 client.0 _ll_register_callbacks cb 0x7f7d48009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:13.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d57e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5d96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d57e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5d96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50710/0 (socket says 172.21.15.47:50710) 2024-03-21T11:43:13.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d57e00640 1 -- 172.21.15.47:0/1681791523 learned_addr learned my addr 172.21.15.47:0/1681791523 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:13.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d5d200640 1 --2- 172.21.15.47:0/1681791523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 0x55d17b5dabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d57400640 1 --2- 172.21.15.47:0/1681791523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5d9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d57e00640 1 -- 172.21.15.47:0/1681791523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 msgr2=0x55d17b5dabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d57e00640 1 --2- 172.21.15.47:0/1681791523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 0x55d17b5dabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d57e00640 1 -- 172.21.15.47:0/1681791523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 msgr2=0x55d17b5d9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d57e00640 1 --2- 172.21.15.47:0/1681791523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5d9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d57e00640 1 -- 172.21.15.47:0/1681791523 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d17b5db1e0 con 0x55d17b5ca200 2024-03-21T11:43:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.551+0000 7f7d57400640 1 --2- 172.21.15.47:0/1681791523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5d9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.552+0000 7f7d57e00640 1 --2- 172.21.15.47:0/1681791523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5d96a0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f7d48013710 tx=0x7f7d48004450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbdb95aa4b12a2bc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.552+0000 7f7d42000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.552+0000 7f7d42000640 1 -- 172.21.15.47:0/1681791523 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4801b430 con 0x55d17b5ca200 2024-03-21T11:43:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.552+0000 7f7d61f77740 1 -- 172.21.15.47:0/1681791523 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d44061030 con 0x55d17b5ca200 2024-03-21T11:43:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.552+0000 7f7d42000640 1 -- 172.21.15.47:0/1681791523 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7d4801b5d0 con 0x55d17b5ca200 2024-03-21T11:43:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.553+0000 7f7d42000640 1 -- 172.21.15.47:0/1681791523 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d4803e050 con 0x55d17b5ca200 2024-03-21T11:43:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.553+0000 7f7d61f77740 10 client.31867 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:13.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.553+0000 7f7d61f77740 1 -- 172.21.15.47:0/1681791523 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d17b63be10 con 0x55d17b5ca200 2024-03-21T11:43:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.553+0000 7f7d42000640 1 -- 172.21.15.47:0/1681791523 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d44061030 con 0x55d17b5ca200 2024-03-21T11:43:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.554+0000 7f7d42000640 1 -- 172.21.15.47:0/1681791523 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7d48004660 con 0x55d17b5ca200 2024-03-21T11:43:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.554+0000 7f7d61f77740 10 client.31867 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.554+0000 7f7d61f77740 10 client.31867 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:43:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.554+0000 7f7d61f77740 20 client.31867 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.554+0000 7f7d61f77740 1 --2- 172.21.15.47:0/1681791523 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55d17b63c780 0x55d17b63eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.554+0000 7f7d61f77740 4 client.31867 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:43:13.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.554+0000 7f7d61f77740 1 -- 172.21.15.47:0/1681791523 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d17b52db50 con 0x55d17b63c780 2024-03-21T11:43:13.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.555+0000 7f7d57400640 1 --2- 172.21.15.47:0/1681791523 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55d17b63c780 0x55d17b63eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.555+0000 7f7d57400640 1 --2- 172.21.15.47:0/1681791523 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55d17b63c780 0x55d17b63eba0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ef44e4c36afdf294 server_cookie=73f61c22f58bca4f in_seq=0 out_seq=0 2024-03-21T11:43:13.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.555+0000 7f7d42000640 10 client.31867 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:43:13.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.555+0000 7f7d42000640 1 -- 172.21.15.47:0/1681791523 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d17b52db50 con 0x55d17b63c780 2024-03-21T11:43:13.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.555+0000 7f7d42000640 10 client.31867 handle_command_reply: tid=0 2024-03-21T11:43:13.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.555+0000 7f7d61f77740 1 client.31867 shutdown 2024-03-21T11:43:13.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.556+0000 7f7d61f77740 1 -- 172.21.15.47:0/1681791523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 msgr2=0x55d17b5d96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.556+0000 7f7d61f77740 1 --2- 172.21.15.47:0/1681791523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5d96a0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f7d48013710 tx=0x7f7d48004450 comp rx=0 tx=0).stop 2024-03-21T11:43:13.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.556+0000 7f7d61f77740 1 -- 172.21.15.47:0/1681791523 shutdown_connections 2024-03-21T11:43:13.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.556+0000 7f7d61f77740 1 --2- 172.21.15.47:0/1681791523 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55d17b63c780 0x55d17b63eba0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.556+0000 7f7d61f77740 1 --2- 172.21.15.47:0/1681791523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d17b5cbd90 0x55d17b5dabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.556+0000 7f7d61f77740 1 --2- 172.21.15.47:0/1681791523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d17b5cb3d0 0x55d17b5d9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.556+0000 7f7d61f77740 1 --2- 172.21.15.47:0/1681791523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d17b5ca200 0x55d17b5d96a0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.556+0000 7f7d42000640 0 client.31867 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:43:13.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.556+0000 7f7d61f77740 1 -- 172.21.15.47:0/1681791523 >> 172.21.15.47:0/1681791523 conn(0x55d17b5c55f0 msgr2=0x55d17b5ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:13.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.556+0000 7f7d61f77740 1 -- 172.21.15.47:0/1681791523 shutdown_connections 2024-03-21T11:43:13.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.556+0000 7f7d61f77740 1 -- 172.21.15.47:0/1681791523 wait complete. 2024-03-21T11:43:13.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.556+0000 7f7d61f77740 20 client.31867 trim_cache size 0 max 16384 2024-03-21T11:43:13.560 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:43:13.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.557+0000 7f7d5dc00640 1 -- 172.21.15.47:0/1984458489 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d3c0a2fa0 msgr2=0x7f7d3c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.557+0000 7f7d5dc00640 1 --2- 172.21.15.47:0/1984458489 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d3c0a2fa0 0x7f7d3c0a5460 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7d44032540 tx=0x7f7d44048000 comp rx=0 tx=0).stop 2024-03-21T11:43:13.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.557+0000 7f7d5dc00640 1 -- 172.21.15.47:0/1984458489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d58107260 msgr2=0x7f7d581a1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.557+0000 7f7d5dc00640 1 --2- 172.21.15.47:0/1984458489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d58107260 0x7f7d581a1050 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009fd0 tx=0x7f7d5000ef10 comp rx=0 tx=0).stop 2024-03-21T11:43:13.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.557+0000 7f7d5dc00640 1 -- 172.21.15.47:0/1984458489 shutdown_connections 2024-03-21T11:43:13.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.557+0000 7f7d5dc00640 1 --2- 172.21.15.47:0/1984458489 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7d3c0a2fa0 0x7f7d3c0a5460 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.557+0000 7f7d5dc00640 1 --2- 172.21.15.47:0/1984458489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d58107260 0x7f7d581a1050 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.557+0000 7f7d5dc00640 1 --2- 172.21.15.47:0/1984458489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d58104930 0x7f7d58199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.557+0000 7f7d5dc00640 1 --2- 172.21.15.47:0/1984458489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d58102000 0x7f7d58199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.557+0000 7f7d5dc00640 1 -- 172.21.15.47:0/1984458489 >> 172.21.15.47:0/1984458489 conn(0x7f7d580fba10 msgr2=0x7f7d58101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:13.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.557+0000 7f7d5dc00640 1 -- 172.21.15.47:0/1984458489 shutdown_connections 2024-03-21T11:43:13.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.557+0000 7f7d5dc00640 1 -- 172.21.15.47:0/1984458489 wait complete. 2024-03-21T11:43:13.576 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:43:13.576 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:43:13.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.803+0000 7fccada00640 1 -- 172.21.15.47:0/1438214673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcca8104ad0 msgr2=0x7fcca8104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.803+0000 7fccada00640 1 --2- 172.21.15.47:0/1438214673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcca8104ad0 0x7fcca8104f50 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009900 tx=0x7fcc9402bc40 comp rx=0 tx=0).stop 2024-03-21T11:43:13.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.805+0000 7fccada00640 1 -- 172.21.15.47:0/1438214673 shutdown_connections 2024-03-21T11:43:13.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.805+0000 7fccada00640 1 --2- 172.21.15.47:0/1438214673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcca8105490 0x7fcca810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.805+0000 7fccada00640 1 --2- 172.21.15.47:0/1438214673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcca8104ad0 0x7fcca8104f50 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.805+0000 7fccada00640 1 --2- 172.21.15.47:0/1438214673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcca81038e0 0x7fcca8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.805+0000 7fccada00640 1 -- 172.21.15.47:0/1438214673 >> 172.21.15.47:0/1438214673 conn(0x7fcca80fee50 msgr2=0x7fcca8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:13.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.805+0000 7fccada00640 1 -- 172.21.15.47:0/1438214673 shutdown_connections 2024-03-21T11:43:13.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.805+0000 7fccada00640 1 -- 172.21.15.47:0/1438214673 wait complete. 2024-03-21T11:43:13.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.810+0000 7fccada00640 1 Processor -- start 2024-03-21T11:43:13.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.810+0000 7fccada00640 1 -- start start 2024-03-21T11:43:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.812+0000 7fccada00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcca81038e0 0x7fcca8199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.812+0000 7fccada00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcca8104ad0 0x7fcca819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.812+0000 7fccada00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcca8105490 0x7fcca81a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.812+0000 7fccada00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca810f590 con 0x7fcca81038e0 2024-03-21T11:43:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.812+0000 7fccada00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca810f410 con 0x7fcca8105490 2024-03-21T11:43:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.812+0000 7fccada00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca810f710 con 0x7fcca8104ad0 2024-03-21T11:43:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.812+0000 7fcca6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcca8104ad0 0x7fcca819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.812+0000 7fcca6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcca8104ad0 0x7fcca819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50716/0 (socket says 172.21.15.47:50716) 2024-03-21T11:43:13.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.812+0000 7fcca6000640 1 -- 172.21.15.47:0/3912148562 learned_addr learned my addr 172.21.15.47:0/3912148562 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:13.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.813+0000 7fcca7400640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcca8105490 0x7fcca81a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.813+0000 7fcca6a00640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcca81038e0 0x7fcca8199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.813+0000 7fcca6000640 1 -- 172.21.15.47:0/3912148562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcca8105490 msgr2=0x7fcca81a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.813+0000 7fcca6000640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcca8105490 0x7fcca81a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.813+0000 7fcca6000640 1 -- 172.21.15.47:0/3912148562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcca81038e0 msgr2=0x7fcca8199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.813+0000 7fcca6000640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcca81038e0 0x7fcca8199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.813+0000 7fcca6000640 1 -- 172.21.15.47:0/3912148562 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca81a17f0 con 0x7fcca8104ad0 2024-03-21T11:43:13.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.813+0000 7fcca6a00640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcca81038e0 0x7fcca8199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:43:13.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.813+0000 7fcca6000640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcca8104ad0 0x7fcca819a110 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fcc9402c250 tx=0x7fcc9402c5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:13.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.814+0000 7fcc9f400640 1 -- 172.21.15.47:0/3912148562 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94041070 con 0x7fcca8104ad0 2024-03-21T11:43:13.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.814+0000 7fcc9f400640 1 -- 172.21.15.47:0/3912148562 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc940054e0 con 0x7fcca8104ad0 2024-03-21T11:43:13.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.814+0000 7fcc9f400640 1 -- 172.21.15.47:0/3912148562 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94034440 con 0x7fcca8104ad0 2024-03-21T11:43:13.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.814+0000 7fccada00640 1 -- 172.21.15.47:0/3912148562 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca81a1a80 con 0x7fcca8104ad0 2024-03-21T11:43:13.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.814+0000 7fccada00640 1 -- 172.21.15.47:0/3912148562 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca81a1f20 con 0x7fcca8104ad0 2024-03-21T11:43:13.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.821+0000 7fcc9f400640 1 -- 172.21.15.47:0/3912148562 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcc94044070 con 0x7fcca8104ad0 2024-03-21T11:43:13.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.824+0000 7fcc9f400640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc8c0a3070 0x7fcc8c0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.824+0000 7fcc9f400640 1 -- 172.21.15.47:0/3912148562 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcc940f1380 con 0x7fcca8104ad0 2024-03-21T11:43:13.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.826+0000 7fcca6a00640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc8c0a3070 0x7fcc8c0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.826+0000 7fccb0793740 1 Processor -- start 2024-03-21T11:43:13.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.826+0000 7fccb0793740 1 -- start start 2024-03-21T11:43:13.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.827+0000 7fccb0793740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe4020 0x561ba0fe4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.827+0000 7fccb0793740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe51f0 0x561ba0fe5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.827+0000 7fccb0793740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0fec440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.827+0000 7fccb0793740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ba0f402c0 con 0x561ba0fe4020 2024-03-21T11:43:13.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.827+0000 7fccb0793740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ba0ee8c90 con 0x561ba0fe51f0 2024-03-21T11:43:13.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.827+0000 7fccb0793740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ba0f3c360 con 0x561ba0fe5bb0 2024-03-21T11:43:13.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.827+0000 7fcca7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0fec440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.827+0000 7fcca7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0fec440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50732/0 (socket says 172.21.15.47:50732) 2024-03-21T11:43:13.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.827+0000 7fcca7400640 1 -- 172.21.15.47:0/1208341625 learned_addr learned my addr 172.21.15.47:0/1208341625 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:13.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.827+0000 7fcca6000640 1 --2- 172.21.15.47:0/1208341625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe51f0 0x561ba0fe5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.827+0000 7fcca6a00640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc8c0a3070 0x7fcc8c0a5530 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fcca8104930 tx=0x7fcc98005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:13.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.827+0000 7fcca6a00640 1 --2- 172.21.15.47:0/1208341625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe4020 0x561ba0fe4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.828+0000 7fcca6000640 1 -- 172.21.15.47:0/1208341625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 msgr2=0x561ba0fec440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:13.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.828+0000 7fcca6000640 1 --2- 172.21.15.47:0/1208341625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0fec440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.828+0000 7fcca6000640 1 -- 172.21.15.47:0/1208341625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe4020 msgr2=0x561ba0fe4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:13.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.828+0000 7fcca6000640 1 --2- 172.21.15.47:0/1208341625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe4020 0x561ba0fe4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.828+0000 7fcca6000640 1 -- 172.21.15.47:0/1208341625 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ba0f0d080 con 0x561ba0fe51f0 2024-03-21T11:43:13.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.828+0000 7fcca7400640 1 --2- 172.21.15.47:0/1208341625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0fec440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:13.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.828+0000 7fcca6000640 1 --2- 172.21.15.47:0/1208341625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe51f0 0x561ba0fe5670 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fcc940eea90 tx=0x7fcc9402ce80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb6a5ce89a4dea61 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:13.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.828+0000 7fcc93e00640 1 -- 172.21.15.47:0/1208341625 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9404b040 con 0x561ba0fe51f0 2024-03-21T11:43:13.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.828+0000 7fcc93e00640 1 -- 172.21.15.47:0/1208341625 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc94005690 con 0x561ba0fe51f0 2024-03-21T11:43:13.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.829+0000 7fcc93e00640 1 -- 172.21.15.47:0/1208341625 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc940eeba0 con 0x561ba0fe51f0 2024-03-21T11:43:13.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.829+0000 7fccb0793740 1 -- 172.21.15.47:0/1208341625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe51f0 msgr2=0x561ba0fe5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.829+0000 7fccb0793740 1 --2- 172.21.15.47:0/1208341625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe51f0 0x561ba0fe5670 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fcc940eea90 tx=0x7fcc9402ce80 comp rx=0 tx=0).stop 2024-03-21T11:43:13.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.829+0000 7fccb0793740 1 -- 172.21.15.47:0/1208341625 shutdown_connections 2024-03-21T11:43:13.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.829+0000 7fccb0793740 1 --2- 172.21.15.47:0/1208341625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0fec440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.829+0000 7fccb0793740 1 --2- 172.21.15.47:0/1208341625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe51f0 0x561ba0fe5670 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.829+0000 7fccb0793740 1 --2- 172.21.15.47:0/1208341625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe4020 0x561ba0fe4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.829+0000 7fccb0793740 1 -- 172.21.15.47:0/1208341625 >> 172.21.15.47:0/1208341625 conn(0x561ba0fd6c00 msgr2=0x561ba0fd7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:13.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.830+0000 7fccb0793740 1 -- 172.21.15.47:0/1208341625 shutdown_connections 2024-03-21T11:43:13.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.830+0000 7fccb0793740 1 -- 172.21.15.47:0/1208341625 wait complete. 2024-03-21T11:43:13.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.830+0000 7fccb0793740 1 -- start start 2024-03-21T11:43:13.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.831+0000 7fccb0793740 1 Processor -- start 2024-03-21T11:43:13.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.831+0000 7fccb0793740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe4020 0x561ba0fedfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.831+0000 7fccb0793740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe51f0 0x561ba0fee500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.831+0000 7fccb0793740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0ff54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.831+0000 7fccb0793740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ba0fec980 con 0x561ba0fe51f0 2024-03-21T11:43:13.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.831+0000 7fccb0793740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ba0efc0d0 con 0x561ba0fe4020 2024-03-21T11:43:13.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.831+0000 7fccb0793740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ba0fecb00 con 0x561ba0fe5bb0 2024-03-21T11:43:13.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.831+0000 7fccb0793740 10 client.0 _ll_register_callbacks cb 0x561ba0f00e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:13.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.832+0000 7fcca7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0ff54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.832+0000 7fcca6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe4020 0x561ba0fedfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.832+0000 7fcca7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0ff54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50748/0 (socket says 172.21.15.47:50748) 2024-03-21T11:43:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.832+0000 7fcca7400640 1 -- 172.21.15.47:0/743379562 learned_addr learned my addr 172.21.15.47:0/743379562 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.833+0000 7fcca6000640 1 --2- 172.21.15.47:0/743379562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe51f0 0x561ba0fee500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.833+0000 7fcca7400640 1 -- 172.21.15.47:0/743379562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe4020 msgr2=0x561ba0fedfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.833+0000 7fcca7400640 1 --2- 172.21.15.47:0/743379562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe4020 0x561ba0fedfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.833+0000 7fcca7400640 1 -- 172.21.15.47:0/743379562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe51f0 msgr2=0x561ba0fee500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.833+0000 7fcca7400640 1 --2- 172.21.15.47:0/743379562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe51f0 0x561ba0fee500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.833+0000 7fcca7400640 1 -- 172.21.15.47:0/743379562 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ba0c543f0 con 0x561ba0fe5bb0 2024-03-21T11:43:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.833+0000 7fcca7400640 1 --2- 172.21.15.47:0/743379562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0ff54d0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fcca00139b0 tx=0x7fcca0013e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a3d1161358cfbc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.833+0000 7fcc90c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.834+0000 7fcc90c00640 1 -- 172.21.15.47:0/743379562 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0004b20 con 0x561ba0fe5bb0 2024-03-21T11:43:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.834+0000 7fcc90c00640 1 -- 172.21.15.47:0/743379562 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcca0004cc0 con 0x561ba0fe5bb0 2024-03-21T11:43:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.834+0000 7fccb0793740 1 -- 172.21.15.47:0/743379562 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ba0f4d800 con 0x561ba0fe5bb0 2024-03-21T11:43:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.835+0000 7fcc90c00640 1 -- 172.21.15.47:0/743379562 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcca0003970 con 0x561ba0fe5bb0 2024-03-21T11:43:13.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.835+0000 7fccb0793740 10 client.31879 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:13.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.835+0000 7fccb0793740 1 -- 172.21.15.47:0/743379562 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561ba0d1e7d0 con 0x561ba0fe5bb0 2024-03-21T11:43:13.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.835+0000 7fcc90c00640 1 -- 172.21.15.47:0/743379562 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561ba0f4d800 con 0x561ba0fe5bb0 2024-03-21T11:43:13.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.836+0000 7fcc90c00640 1 -- 172.21.15.47:0/743379562 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcca001a070 con 0x561ba0fe5bb0 2024-03-21T11:43:13.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.836+0000 7fccb0793740 10 client.31879 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:13.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.836+0000 7fccb0793740 10 client.31879 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:43:13.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.836+0000 7fccb0793740 20 client.31879 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:13.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.836+0000 7fccb0793740 1 --2- 172.21.15.47:0/743379562 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561ba0ff9b90 0x561ba0ffbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.837+0000 7fccb0793740 4 client.31879 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:13.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.837+0000 7fccb0793740 1 -- 172.21.15.47:0/743379562 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ba0f47b50 con 0x561ba0ff9b90 2024-03-21T11:43:13.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.839+0000 7fcca6a00640 1 --2- 172.21.15.47:0/743379562 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561ba0ff9b90 0x561ba0ffbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.840+0000 7fcca6a00640 1 --2- 172.21.15.47:0/743379562 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561ba0ff9b90 0x561ba0ffbfb0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7838a0453a63a827 server_cookie=2cc575d27c5f7792 in_seq=0 out_seq=0 2024-03-21T11:43:13.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.840+0000 7fcc90c00640 10 client.31879 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:43:13.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.842+0000 7fcc90c00640 1 -- 172.21.15.47:0/743379562 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561ba0f47b50 con 0x561ba0ff9b90 2024-03-21T11:43:13.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.842+0000 7fcc90c00640 10 client.31879 handle_command_reply: tid=0 2024-03-21T11:43:13.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.842+0000 7fccb0793740 1 client.31879 shutdown 2024-03-21T11:43:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.843+0000 7fccb0793740 1 -- 172.21.15.47:0/743379562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 msgr2=0x561ba0ff54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.844+0000 7fccb0793740 1 --2- 172.21.15.47:0/743379562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0ff54d0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fcca00139b0 tx=0x7fcca0013e70 comp rx=0 tx=0).stop 2024-03-21T11:43:13.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.844+0000 7fccb0793740 1 -- 172.21.15.47:0/743379562 shutdown_connections 2024-03-21T11:43:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.845+0000 7fccb0793740 1 --2- 172.21.15.47:0/743379562 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561ba0ff9b90 0x561ba0ffbfb0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.845+0000 7fccb0793740 1 --2- 172.21.15.47:0/743379562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0ff54d0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.845+0000 7fccb0793740 1 --2- 172.21.15.47:0/743379562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe51f0 0x561ba0fee500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.845+0000 7fccb0793740 1 --2- 172.21.15.47:0/743379562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe4020 0x561ba0fedfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.845+0000 7fccb0793740 1 -- 172.21.15.47:0/743379562 >> 172.21.15.47:0/743379562 conn(0x561ba0fd6c00 msgr2=0x561ba0fd5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.845+0000 7fcc90c00640 0 client.31879 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:43:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.845+0000 7fccb0793740 1 -- 172.21.15.47:0/743379562 shutdown_connections 2024-03-21T11:43:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.845+0000 7fccb0793740 1 -- 172.21.15.47:0/743379562 wait complete. 2024-03-21T11:43:13.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.845+0000 7fccb0793740 20 client.31879 trim_cache size 0 max 16384 2024-03-21T11:43:13.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.873+0000 7fccb0793740 1 Processor -- start 2024-03-21T11:43:13.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.873+0000 7fccb0793740 1 -- start start 2024-03-21T11:43:13.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.873+0000 7fccb0793740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe4020 0x561ba0fe82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.873+0000 7fccb0793740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe51f0 0x561ba0ff5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.874+0000 7fccb0793740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe5bb0 0x561ba0fe87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.874+0000 7fccb0793740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca81a7f70 con 0x561ba0fe4020 2024-03-21T11:43:13.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.874+0000 7fccb0793740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca81a98f0 con 0x561ba0fe5bb0 2024-03-21T11:43:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.874+0000 7fccb0793740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca81a85d0 con 0x561ba0fe51f0 2024-03-21T11:43:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.874+0000 7fcca7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe5bb0 0x561ba0fe87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.874+0000 7fcca7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe5bb0 0x561ba0fe87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58088/0 (socket says 172.21.15.47:58088) 2024-03-21T11:43:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.874+0000 7fcca7400640 1 -- 172.21.15.47:0/3656321464 learned_addr learned my addr 172.21.15.47:0/3656321464 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.874+0000 7fcca6000640 1 --2- 172.21.15.47:0/3656321464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe51f0 0x561ba0ff5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.875+0000 7fcca7400640 1 -- 172.21.15.47:0/3656321464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe51f0 msgr2=0x561ba0ff5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.875+0000 7fcca7400640 1 --2- 172.21.15.47:0/3656321464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe51f0 0x561ba0ff5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.875+0000 7fcca7400640 1 -- 172.21.15.47:0/3656321464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe4020 msgr2=0x561ba0fe82a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:43:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.875+0000 7fcca7400640 1 --2- 172.21.15.47:0/3656321464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe4020 0x561ba0fe82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.875+0000 7fcca7400640 1 -- 172.21.15.47:0/3656321464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ba0fd2510 con 0x561ba0fe5bb0 2024-03-21T11:43:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.875+0000 7fcca6000640 1 --2- 172.21.15.47:0/3656321464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe51f0 0x561ba0ff5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.875+0000 7fcca7400640 1 --2- 172.21.15.47:0/3656321464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe5bb0 0x561ba0fe87e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fcca0013980 tx=0x7fcca00096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d02899eef2853218 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.876+0000 7fcc93400640 1 -- 172.21.15.47:0/3656321464 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0033070 con 0x561ba0fe5bb0 2024-03-21T11:43:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.876+0000 7fcc93400640 1 -- 172.21.15.47:0/3656321464 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcca0011cf0 con 0x561ba0fe5bb0 2024-03-21T11:43:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.876+0000 7fcc93400640 1 -- 172.21.15.47:0/3656321464 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0004320 con 0x561ba0fe5bb0 2024-03-21T11:43:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.877+0000 7fccb0793740 1 -- 172.21.15.47:0/3656321464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe5bb0 msgr2=0x561ba0fe87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.877+0000 7fccb0793740 1 --2- 172.21.15.47:0/3656321464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe5bb0 0x561ba0fe87e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fcca0013980 tx=0x7fcca00096f0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.877+0000 7fccb0793740 1 -- 172.21.15.47:0/3656321464 shutdown_connections 2024-03-21T11:43:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.877+0000 7fccb0793740 1 --2- 172.21.15.47:0/3656321464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe5bb0 0x561ba0fe87e0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.877+0000 7fccb0793740 1 --2- 172.21.15.47:0/3656321464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe51f0 0x561ba0ff5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.877+0000 7fccb0793740 1 --2- 172.21.15.47:0/3656321464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe4020 0x561ba0fe82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.877+0000 7fccb0793740 1 -- 172.21.15.47:0/3656321464 >> 172.21.15.47:0/3656321464 conn(0x561ba0fd6c00 msgr2=0x561ba0fda6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.878+0000 7fccb0793740 1 -- 172.21.15.47:0/3656321464 shutdown_connections 2024-03-21T11:43:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.878+0000 7fccb0793740 1 -- 172.21.15.47:0/3656321464 wait complete. 2024-03-21T11:43:13.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.879+0000 7fccb0793740 1 -- start start 2024-03-21T11:43:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.879+0000 7fccb0793740 1 Processor -- start 2024-03-21T11:43:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.879+0000 7fccb0793740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe4020 0x561ba0ff0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.879+0000 7fccb0793740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe51f0 0x561ba0ff1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.879+0000 7fccb0793740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0ff4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.879+0000 7fccb0793740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca81a9290 con 0x561ba0fe51f0 2024-03-21T11:43:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.879+0000 7fccb0793740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca81a8c30 con 0x561ba0fe4020 2024-03-21T11:43:13.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.880+0000 7fccb0793740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca81a6130 con 0x561ba0fe5bb0 2024-03-21T11:43:13.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.880+0000 7fccb0793740 10 client.0 _ll_register_callbacks cb 0x7fcca0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:13.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.881+0000 7fcca6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe51f0 0x561ba0ff1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.882+0000 7fcca6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe51f0 0x561ba0ff1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33964/0 (socket says 172.21.15.47:33964) 2024-03-21T11:43:13.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.882+0000 7fcca6000640 1 -- 172.21.15.47:0/1738373005 learned_addr learned my addr 172.21.15.47:0/1738373005 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:13.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.882+0000 7fcca7400640 1 --2- 172.21.15.47:0/1738373005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0ff4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.882+0000 7fcca6000640 1 -- 172.21.15.47:0/1738373005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 msgr2=0x561ba0ff4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:13.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.882+0000 7fcca6000640 1 --2- 172.21.15.47:0/1738373005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0ff4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.882+0000 7fcca6000640 1 -- 172.21.15.47:0/1738373005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe4020 msgr2=0x561ba0ff0ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:43:13.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.882+0000 7fcca6000640 1 --2- 172.21.15.47:0/1738373005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe4020 0x561ba0ff0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.882+0000 7fcca6000640 1 -- 172.21.15.47:0/1738373005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ba0feddb0 con 0x561ba0fe51f0 2024-03-21T11:43:13.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.882+0000 7fcca7400640 1 --2- 172.21.15.47:0/1738373005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0ff4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:43:13.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.882+0000 7fcca6000640 1 --2- 172.21.15.47:0/1738373005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe51f0 0x561ba0ff1420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcc940bb970 tx=0x7fcc94041fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8df6e5946107d32 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:13.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.883+0000 7fcc90c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:43:13.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.883+0000 7fcc90c00640 1 -- 172.21.15.47:0/1738373005 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9404b040 con 0x561ba0fe51f0 2024-03-21T11:43:13.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.883+0000 7fccb0793740 1 -- 172.21.15.47:0/1738373005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcca0011cf0 con 0x561ba0fe51f0 2024-03-21T11:43:13.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.883+0000 7fcc90c00640 1 -- 172.21.15.47:0/1738373005 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcc94003680 con 0x561ba0fe51f0 2024-03-21T11:43:13.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.883+0000 7fcc90c00640 1 -- 172.21.15.47:0/1738373005 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc940bddc0 con 0x561ba0fe51f0 2024-03-21T11:43:13.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.883+0000 7fccb0793740 10 client.21303 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:13.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.883+0000 7fccb0793740 1 -- 172.21.15.47:0/1738373005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561ba1066610 con 0x561ba0fe51f0 2024-03-21T11:43:13.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.884+0000 7fcc90c00640 1 -- 172.21.15.47:0/1738373005 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcca0011cf0 con 0x561ba0fe51f0 2024-03-21T11:43:13.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.884+0000 7fcc90c00640 1 -- 172.21.15.47:0/1738373005 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcc940c5c40 con 0x561ba0fe51f0 2024-03-21T11:43:13.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.884+0000 7fccb0793740 10 client.21303 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:13.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.884+0000 7fccb0793740 10 client.21303 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:43:13.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.884+0000 7fccb0793740 20 client.21303 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:13.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.884+0000 7fccb0793740 1 --2- 172.21.15.47:0/1738373005 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561ba1066d20 0x561ba1069140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:13.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.884+0000 7fccb0793740 4 client.21303 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:43:13.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.884+0000 7fccb0793740 1 -- 172.21.15.47:0/1738373005 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561ba0ec5500 con 0x561ba1066d20 2024-03-21T11:43:13.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.885+0000 7fcca6a00640 1 --2- 172.21.15.47:0/1738373005 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561ba1066d20 0x561ba1069140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:13.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.886+0000 7fcca6a00640 1 --2- 172.21.15.47:0/1738373005 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561ba1066d20 0x561ba1069140 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=61ff3888edb5ddb8 server_cookie=8a8fd165a5bbd183 in_seq=0 out_seq=0 2024-03-21T11:43:13.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.886+0000 7fcc90c00640 10 client.21303 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:43:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.886+0000 7fcc90c00640 1 -- 172.21.15.47:0/1738373005 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561ba0ec5500 con 0x561ba1066d20 2024-03-21T11:43:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.886+0000 7fcc90c00640 10 client.21303 handle_command_reply: tid=0 2024-03-21T11:43:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.886+0000 7fccb0793740 1 client.21303 shutdown 2024-03-21T11:43:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.886+0000 7fccb0793740 1 -- 172.21.15.47:0/1738373005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe51f0 msgr2=0x561ba0ff1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.886+0000 7fccb0793740 1 --2- 172.21.15.47:0/1738373005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe51f0 0x561ba0ff1420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcc940bb970 tx=0x7fcc94041fb0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.886+0000 7fccb0793740 1 -- 172.21.15.47:0/1738373005 shutdown_connections 2024-03-21T11:43:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.886+0000 7fccb0793740 1 --2- 172.21.15.47:0/1738373005 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x561ba1066d20 0x561ba1069140 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.886+0000 7fccb0793740 1 --2- 172.21.15.47:0/1738373005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561ba0fe5bb0 0x561ba0ff4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.886+0000 7fccb0793740 1 --2- 172.21.15.47:0/1738373005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ba0fe51f0 0x561ba0ff1420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.886+0000 7fccb0793740 1 --2- 172.21.15.47:0/1738373005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ba0fe4020 0x561ba0ff0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.887+0000 7fccb0793740 1 -- 172.21.15.47:0/1738373005 >> 172.21.15.47:0/1738373005 conn(0x561ba0fd6c00 msgr2=0x561ba0fd5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.887+0000 7fcc90c00640 0 client.21303 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:43:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.887+0000 7fccb0793740 1 -- 172.21.15.47:0/1738373005 shutdown_connections 2024-03-21T11:43:13.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.887+0000 7fccb0793740 1 -- 172.21.15.47:0/1738373005 wait complete. 2024-03-21T11:43:13.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.887+0000 7fccb0793740 20 client.21303 trim_cache size 0 max 16384 2024-03-21T11:43:13.889 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:43:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.888+0000 7fccada00640 1 -- 172.21.15.47:0/3912148562 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc8c0a3070 msgr2=0x7fcc8c0a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.888+0000 7fccada00640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc8c0a3070 0x7fcc8c0a5530 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fcca8104930 tx=0x7fcc98005e70 comp rx=0 tx=0).stop 2024-03-21T11:43:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.888+0000 7fccada00640 1 -- 172.21.15.47:0/3912148562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcca8104ad0 msgr2=0x7fcca819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.888+0000 7fccada00640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcca8104ad0 0x7fcca819a110 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fcc9402c250 tx=0x7fcc9402c5e0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.888+0000 7fccada00640 1 -- 172.21.15.47:0/3912148562 shutdown_connections 2024-03-21T11:43:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.888+0000 7fccada00640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc8c0a3070 0x7fcc8c0a5530 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.888+0000 7fccada00640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcca8105490 0x7fcca81a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.888+0000 7fccada00640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcca8104ad0 0x7fcca819a110 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.888+0000 7fccada00640 1 --2- 172.21.15.47:0/3912148562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcca81038e0 0x7fcca8199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.888+0000 7fccada00640 1 -- 172.21.15.47:0/3912148562 >> 172.21.15.47:0/3912148562 conn(0x7fcca80fee50 msgr2=0x7fcca8107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.889+0000 7fccada00640 1 -- 172.21.15.47:0/3912148562 shutdown_connections 2024-03-21T11:43:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:13.889+0000 7fccada00640 1 -- 172.21.15.47:0/3912148562 wait complete. 2024-03-21T11:43:13.912 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:43:13.912 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:43:13.962 INFO:tasks.workunit.client.0.smithi047.stdout: 10 159778 23.06 MB/sec execute 237 sec latency 123.348 ms 2024-03-21T11:43:14.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.132+0000 7fc816c00640 1 -- 172.21.15.47:0/2201807533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc810108f30 msgr2=0x7fc81010b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.132+0000 7fc816c00640 1 --2- 172.21.15.47:0/2201807533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc810108f30 0x7fc81010b320 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc800009900 tx=0x7fc80002bc60 comp rx=0 tx=0).stop 2024-03-21T11:43:14.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.132+0000 7fc816c00640 1 -- 172.21.15.47:0/2201807533 shutdown_connections 2024-03-21T11:43:14.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.132+0000 7fc816c00640 1 --2- 172.21.15.47:0/2201807533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc81010e1e0 0x7fc810110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.132+0000 7fc816c00640 1 --2- 172.21.15.47:0/2201807533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc81010b860 0x7fc81010dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.132+0000 7fc816c00640 1 --2- 172.21.15.47:0/2201807533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc810108f30 0x7fc81010b320 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.132+0000 7fc816c00640 1 -- 172.21.15.47:0/2201807533 >> 172.21.15.47:0/2201807533 conn(0x7fc81006e500 msgr2=0x7fc810070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.132+0000 7fc816c00640 1 -- 172.21.15.47:0/2201807533 shutdown_connections 2024-03-21T11:43:14.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.132+0000 7fc816c00640 1 -- 172.21.15.47:0/2201807533 wait complete. 2024-03-21T11:43:14.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.133+0000 7fc816c00640 1 Processor -- start 2024-03-21T11:43:14.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.133+0000 7fc816c00640 1 -- start start 2024-03-21T11:43:14.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.134+0000 7fc816c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc810108f30 0x7fc8101a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.134+0000 7fc816c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc81010b860 0x7fc8101a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.134+0000 7fc816c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc81010e1e0 0x7fc8101a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.134+0000 7fc816c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810115bc0 con 0x7fc810108f30 2024-03-21T11:43:14.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.134+0000 7fc816c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810115a40 con 0x7fc81010b860 2024-03-21T11:43:14.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.134+0000 7fc816c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810115d40 con 0x7fc81010e1e0 2024-03-21T11:43:14.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.134+0000 7fc814c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc81010e1e0 0x7fc8101a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.134+0000 7fc80fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc810108f30 0x7fc8101a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.134+0000 7fc80f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc81010b860 0x7fc8101a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.134+0000 7fc814c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc81010e1e0 0x7fc8101a9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50790/0 (socket says 172.21.15.47:50790) 2024-03-21T11:43:14.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.134+0000 7fc814c00640 1 -- 172.21.15.47:0/1729672976 learned_addr learned my addr 172.21.15.47:0/1729672976 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.135+0000 7fc814c00640 1 -- 172.21.15.47:0/1729672976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc81010b860 msgr2=0x7fc8101a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.135+0000 7fc814c00640 1 --2- 172.21.15.47:0/1729672976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc81010b860 0x7fc8101a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.135+0000 7fc814c00640 1 -- 172.21.15.47:0/1729672976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc810108f30 msgr2=0x7fc8101a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.135+0000 7fc814c00640 1 --2- 172.21.15.47:0/1729672976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc810108f30 0x7fc8101a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.135+0000 7fc814c00640 1 -- 172.21.15.47:0/1729672976 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8101aa0b0 con 0x7fc81010e1e0 2024-03-21T11:43:14.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.135+0000 7fc814c00640 1 --2- 172.21.15.47:0/1729672976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc81010e1e0 0x7fc8101a9a40 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fc80800c570 tx=0x7fc80800ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.135+0000 7fc80cc00640 1 -- 172.21.15.47:0/1729672976 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808004b20 con 0x7fc81010e1e0 2024-03-21T11:43:14.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.135+0000 7fc816c00640 1 -- 172.21.15.47:0/1729672976 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81019b300 con 0x7fc81010e1e0 2024-03-21T11:43:14.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.135+0000 7fc80cc00640 1 -- 172.21.15.47:0/1729672976 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc808004cc0 con 0x7fc81010e1e0 2024-03-21T11:43:14.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.135+0000 7fc816c00640 1 -- 172.21.15.47:0/1729672976 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81019b850 con 0x7fc81010e1e0 2024-03-21T11:43:14.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.135+0000 7fc80cc00640 1 -- 172.21.15.47:0/1729672976 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80801e890 con 0x7fc81010e1e0 2024-03-21T11:43:14.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.139+0000 7fc80cc00640 1 -- 172.21.15.47:0/1729672976 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc80801ea30 con 0x7fc81010e1e0 2024-03-21T11:43:14.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.139+0000 7fc81986e740 1 Processor -- start 2024-03-21T11:43:14.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.139+0000 7fc81986e740 1 -- start start 2024-03-21T11:43:14.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.140+0000 7fc81986e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 0x5622e8ca9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.140+0000 7fc81986e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8caae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.140+0000 7fc81986e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8cab3a0 0x5622e8cb1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.140+0000 7fc81986e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8c0e2c0 con 0x5622e8ca9810 2024-03-21T11:43:14.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.140+0000 7fc81986e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8bb6c90 con 0x5622e8cab3a0 2024-03-21T11:43:14.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.140+0000 7fc81986e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8c0a360 con 0x5622e8caa9e0 2024-03-21T11:43:14.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.140+0000 7fc80fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8caae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.140+0000 7fc80fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8caae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50802/0 (socket says 172.21.15.47:50802) 2024-03-21T11:43:14.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.140+0000 7fc80fe00640 1 -- 172.21.15.47:0/1275447087 learned_addr learned my addr 172.21.15.47:0/1275447087 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.140+0000 7fc80f400640 1 --2- 172.21.15.47:0/1275447087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8cab3a0 0x5622e8cb1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.140+0000 7fc80f400640 1 --2- 172.21.15.47:0/1275447087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 0x5622e8ca9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.141+0000 7fc80fe00640 1 -- 172.21.15.47:0/1275447087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8cab3a0 msgr2=0x5622e8cb1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.141+0000 7fc80fe00640 1 --2- 172.21.15.47:0/1275447087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8cab3a0 0x5622e8cb1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.141+0000 7fc80fe00640 1 -- 172.21.15.47:0/1275447087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 msgr2=0x5622e8ca9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.141+0000 7fc80fe00640 1 --2- 172.21.15.47:0/1275447087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 0x5622e8ca9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.141+0000 7fc80fe00640 1 -- 172.21.15.47:0/1275447087 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622e8bdb080 con 0x5622e8caa9e0 2024-03-21T11:43:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.141+0000 7fc80f400640 1 --2- 172.21.15.47:0/1275447087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 0x5622e8ca9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:43:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.141+0000 7fc80fe00640 1 --2- 172.21.15.47:0/1275447087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8caae60 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc800004bc0 tx=0x7fc800005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6888e094427e5a40 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.141+0000 7fc80cc00640 1 --2- 172.21.15.47:0/1729672976 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7f40a2f20 0x7fc7f40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.141+0000 7fc805200640 1 -- 172.21.15.47:0/1275447087 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800043070 con 0x5622e8caa9e0 2024-03-21T11:43:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.141+0000 7fc805200640 1 -- 172.21.15.47:0/1275447087 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc800004840 con 0x5622e8caa9e0 2024-03-21T11:43:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.141+0000 7fc805200640 1 -- 172.21.15.47:0/1275447087 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8000052f0 con 0x5622e8caa9e0 2024-03-21T11:43:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.141+0000 7fc80cc00640 1 -- 172.21.15.47:0/1729672976 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc8080d7950 con 0x7fc81010e1e0 2024-03-21T11:43:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.141+0000 7fc80f400640 1 --2- 172.21.15.47:0/1729672976 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7f40a2f20 0x7fc7f40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.142+0000 7fc81986e740 1 -- 172.21.15.47:0/1275447087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 msgr2=0x5622e8caae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.142+0000 7fc80f400640 1 --2- 172.21.15.47:0/1729672976 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7f40a2f20 0x7fc7f40a53e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x5622e8caa8e0 tx=0x7fc7fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.142+0000 7fc81986e740 1 --2- 172.21.15.47:0/1275447087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8caae60 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc800004bc0 tx=0x7fc800005be0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.142+0000 7fc81986e740 1 -- 172.21.15.47:0/1275447087 shutdown_connections 2024-03-21T11:43:14.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.142+0000 7fc81986e740 1 --2- 172.21.15.47:0/1275447087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8cab3a0 0x5622e8cb1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.142+0000 7fc81986e740 1 --2- 172.21.15.47:0/1275447087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8caae60 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.142+0000 7fc81986e740 1 --2- 172.21.15.47:0/1275447087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 0x5622e8ca9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.142+0000 7fc81986e740 1 -- 172.21.15.47:0/1275447087 >> 172.21.15.47:0/1275447087 conn(0x5622e8ca4c00 msgr2=0x5622e8ca5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.142+0000 7fc81986e740 1 -- 172.21.15.47:0/1275447087 shutdown_connections 2024-03-21T11:43:14.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.142+0000 7fc81986e740 1 -- 172.21.15.47:0/1275447087 wait complete. 2024-03-21T11:43:14.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.143+0000 7fc81986e740 1 -- start start 2024-03-21T11:43:14.146 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:13 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3746068995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:43:14.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.143+0000 7fc81986e740 1 Processor -- start 2024-03-21T11:43:14.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.143+0000 7fc81986e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 0x5622e8cb39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.143+0000 7fc81986e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8caa9e0 0x5622e8cb3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.143+0000 7fc81986e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8cab3a0 0x5622e8cbaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.143+0000 7fc81986e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8cb2350 con 0x5622e8ca9810 2024-03-21T11:43:14.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.143+0000 7fc81986e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8bca0d0 con 0x5622e8caa9e0 2024-03-21T11:43:14.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.143+0000 7fc81986e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e8cb24d0 con 0x5622e8cab3a0 2024-03-21T11:43:14.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.144+0000 7fc81986e740 10 client.0 _ll_register_callbacks cb 0x5622e8bcee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:14.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.144+0000 7fc80fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 0x5622e8cb39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.144+0000 7fc80f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8caa9e0 0x5622e8cb3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.144+0000 7fc80fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 0x5622e8cb39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34004/0 (socket says 172.21.15.47:34004) 2024-03-21T11:43:14.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.144+0000 7fc80fe00640 1 -- 172.21.15.47:0/3348567730 learned_addr learned my addr 172.21.15.47:0/3348567730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.144+0000 7fc814c00640 1 --2- 172.21.15.47:0/3348567730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8cab3a0 0x5622e8cbaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.144+0000 7fc80f400640 1 -- 172.21.15.47:0/3348567730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8cab3a0 msgr2=0x5622e8cbaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.144+0000 7fc80f400640 1 --2- 172.21.15.47:0/3348567730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8cab3a0 0x5622e8cbaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.144+0000 7fc80f400640 1 -- 172.21.15.47:0/3348567730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 msgr2=0x5622e8cb39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.144+0000 7fc80f400640 1 --2- 172.21.15.47:0/3348567730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 0x5622e8cb39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.144+0000 7fc80f400640 1 -- 172.21.15.47:0/3348567730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622e89223f0 con 0x5622e8caa9e0 2024-03-21T11:43:14.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.145+0000 7fc80f400640 1 --2- 172.21.15.47:0/3348567730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8caa9e0 0x5622e8cb3f20 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fc8101a39a0 tx=0x7fc7fc005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58df7abc0c6a16c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.145+0000 7fc7f9600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:43:14.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.145+0000 7fc7f9600640 1 -- 172.21.15.47:0/3348567730 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00cab0 con 0x5622e8caa9e0 2024-03-21T11:43:14.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.145+0000 7fc7f9600640 1 -- 172.21.15.47:0/3348567730 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc7fc00cc50 con 0x5622e8caa9e0 2024-03-21T11:43:14.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.145+0000 7fc81986e740 1 -- 172.21.15.47:0/3348567730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc800004840 con 0x5622e8caa9e0 2024-03-21T11:43:14.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.145+0000 7fc7f9600640 1 -- 172.21.15.47:0/3348567730 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7fc033050 con 0x5622e8caa9e0 2024-03-21T11:43:14.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.145+0000 7fc81986e740 10 client.31952 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:14.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.145+0000 7fc81986e740 1 -- 172.21.15.47:0/3348567730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5622e89ec7d0 con 0x5622e8caa9e0 2024-03-21T11:43:14.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.146+0000 7fc7f9600640 1 -- 172.21.15.47:0/3348567730 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc800004840 con 0x5622e8caa9e0 2024-03-21T11:43:14.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.146+0000 7fc7f9600640 1 -- 172.21.15.47:0/3348567730 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc7fc01a070 con 0x5622e8caa9e0 2024-03-21T11:43:14.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.146+0000 7fc81986e740 10 client.31952 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:14.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.146+0000 7fc81986e740 10 client.31952 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:43:14.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.146+0000 7fc81986e740 20 client.31952 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:14.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.146+0000 7fc81986e740 1 --2- 172.21.15.47:0/3348567730 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622e8cbf3d0 0x5622e8cc1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.146+0000 7fc81986e740 4 client.31952 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:14.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.146+0000 7fc81986e740 1 -- 172.21.15.47:0/3348567730 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622e8c1b800 con 0x5622e8cbf3d0 2024-03-21T11:43:14.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.146+0000 7fc80fe00640 1 --2- 172.21.15.47:0/3348567730 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622e8cbf3d0 0x5622e8cc1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.147+0000 7fc80fe00640 1 --2- 172.21.15.47:0/3348567730 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622e8cbf3d0 0x5622e8cc1870 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a9a28c7737039e2 server_cookie=3797d2b642b70894 in_seq=0 out_seq=0 2024-03-21T11:43:14.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.147+0000 7fc7f9600640 10 client.31952 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:43:14.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc7f9600640 1 -- 172.21.15.47:0/3348567730 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5622e8c1b800 con 0x5622e8cbf3d0 2024-03-21T11:43:14.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc7f9600640 10 client.31952 handle_command_reply: tid=0 2024-03-21T11:43:14.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc81986e740 1 client.31952 shutdown 2024-03-21T11:43:14.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc81986e740 1 -- 172.21.15.47:0/3348567730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8caa9e0 msgr2=0x5622e8cb3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc81986e740 1 --2- 172.21.15.47:0/3348567730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8caa9e0 0x5622e8cb3f20 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fc8101a39a0 tx=0x7fc7fc005850 comp rx=0 tx=0).stop 2024-03-21T11:43:14.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc81986e740 1 -- 172.21.15.47:0/3348567730 shutdown_connections 2024-03-21T11:43:14.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc81986e740 1 --2- 172.21.15.47:0/3348567730 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622e8cbf3d0 0x5622e8cc1870 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc81986e740 1 --2- 172.21.15.47:0/3348567730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8cab3a0 0x5622e8cbaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc81986e740 1 --2- 172.21.15.47:0/3348567730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8caa9e0 0x5622e8cb3f20 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc7f9600640 0 client.31952 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:43:14.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc81986e740 1 --2- 172.21.15.47:0/3348567730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 0x5622e8cb39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc81986e740 1 -- 172.21.15.47:0/3348567730 >> 172.21.15.47:0/3348567730 conn(0x5622e8ca4c00 msgr2=0x5622e8ca3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc81986e740 1 -- 172.21.15.47:0/3348567730 shutdown_connections 2024-03-21T11:43:14.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc81986e740 1 -- 172.21.15.47:0/3348567730 wait complete. 2024-03-21T11:43:14.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.148+0000 7fc81986e740 20 client.31952 trim_cache size 0 max 16384 2024-03-21T11:43:14.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.168+0000 7fc81986e740 1 Processor -- start 2024-03-21T11:43:14.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.169+0000 7fc81986e740 1 -- start start 2024-03-21T11:43:14.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.169+0000 7fc81986e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 0x5622e8ca8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.169+0000 7fc81986e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8cbaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.169+0000 7fc81986e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8cab3a0 0x5622e8cad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.169+0000 7fc81986e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f000f9a0 con 0x5622e8ca9810 2024-03-21T11:43:14.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.169+0000 7fc81986e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f0011320 con 0x5622e8cab3a0 2024-03-21T11:43:14.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.169+0000 7fc81986e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f0010000 con 0x5622e8caa9e0 2024-03-21T11:43:14.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.170+0000 7fc80f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8cbaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.170+0000 7fc80f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8cbaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50816/0 (socket says 172.21.15.47:50816) 2024-03-21T11:43:14.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.170+0000 7fc80f400640 1 -- 172.21.15.47:0/2543611056 learned_addr learned my addr 172.21.15.47:0/2543611056 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.170+0000 7fc80fe00640 1 --2- 172.21.15.47:0/2543611056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 0x5622e8ca8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.170+0000 7fc814c00640 1 --2- 172.21.15.47:0/2543611056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8cab3a0 0x5622e8cad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.172+0000 7fc80f400640 1 -- 172.21.15.47:0/2543611056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8cab3a0 msgr2=0x5622e8cad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.172+0000 7fc80f400640 1 --2- 172.21.15.47:0/2543611056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8cab3a0 0x5622e8cad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.172+0000 7fc80f400640 1 -- 172.21.15.47:0/2543611056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 msgr2=0x5622e8ca8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.172+0000 7fc80f400640 1 --2- 172.21.15.47:0/2543611056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 0x5622e8ca8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.172+0000 7fc80f400640 1 -- 172.21.15.47:0/2543611056 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622e8cb14d0 con 0x5622e8caa9e0 2024-03-21T11:43:14.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.173+0000 7fc814c00640 1 --2- 172.21.15.47:0/2543611056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8cab3a0 0x5622e8cad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:14.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.173+0000 7fc80f400640 1 --2- 172.21.15.47:0/2543611056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8cbaeb0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0057c0 tx=0x7fc7fc00a2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db1150acaf4d4a35 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.174+0000 7fc7fbe00640 1 -- 172.21.15.47:0/2543611056 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00cab0 con 0x5622e8caa9e0 2024-03-21T11:43:14.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.174+0000 7fc7fbe00640 1 -- 172.21.15.47:0/2543611056 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7fc02a070 con 0x5622e8caa9e0 2024-03-21T11:43:14.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.175+0000 7fc7fbe00640 1 -- 172.21.15.47:0/2543611056 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc005590 con 0x5622e8caa9e0 2024-03-21T11:43:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.177+0000 7fc81986e740 1 -- 172.21.15.47:0/2543611056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 msgr2=0x5622e8cbaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.177+0000 7fc81986e740 1 --2- 172.21.15.47:0/2543611056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8cbaeb0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0057c0 tx=0x7fc7fc00a2d0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.178+0000 7fc81986e740 1 -- 172.21.15.47:0/2543611056 shutdown_connections 2024-03-21T11:43:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.178+0000 7fc81986e740 1 --2- 172.21.15.47:0/2543611056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8cab3a0 0x5622e8cad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.178+0000 7fc81986e740 1 --2- 172.21.15.47:0/2543611056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8cbaeb0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.178+0000 7fc81986e740 1 --2- 172.21.15.47:0/2543611056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8ca9810 0x5622e8ca8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.178+0000 7fc81986e740 1 -- 172.21.15.47:0/2543611056 >> 172.21.15.47:0/2543611056 conn(0x5622e8ca4c00 msgr2=0x5622e8ca7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.178+0000 7fc81986e740 1 -- 172.21.15.47:0/2543611056 shutdown_connections 2024-03-21T11:43:14.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.178+0000 7fc81986e740 1 -- 172.21.15.47:0/2543611056 wait complete. 2024-03-21T11:43:14.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.179+0000 7fc81986e740 1 -- start start 2024-03-21T11:43:14.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.179+0000 7fc81986e740 1 Processor -- start 2024-03-21T11:43:14.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.179+0000 7fc81986e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8ca9810 0x5622e8cb6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.179+0000 7fc81986e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8cb7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc81986e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8cab3a0 0x5622e8cba2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc81986e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f0010cc0 con 0x5622e8cab3a0 2024-03-21T11:43:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc81986e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f0010660 con 0x5622e8ca9810 2024-03-21T11:43:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc81986e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f000db60 con 0x5622e8caa9e0 2024-03-21T11:43:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc81986e740 10 client.0 _ll_register_callbacks cb 0x7fc7fc00aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc80f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8cb7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc80f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8cb7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50832/0 (socket says 172.21.15.47:50832) 2024-03-21T11:43:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc80f400640 1 -- 172.21.15.47:0/2664839824 learned_addr learned my addr 172.21.15.47:0/2664839824 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc814c00640 1 --2- 172.21.15.47:0/2664839824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8cab3a0 0x5622e8cba2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc80fe00640 1 --2- 172.21.15.47:0/2664839824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8ca9810 0x5622e8cb6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc80f400640 1 -- 172.21.15.47:0/2664839824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8ca9810 msgr2=0x5622e8cb6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc80f400640 1 --2- 172.21.15.47:0/2664839824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8ca9810 0x5622e8cb6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc80f400640 1 -- 172.21.15.47:0/2664839824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8cab3a0 msgr2=0x5622e8cba2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc80f400640 1 --2- 172.21.15.47:0/2664839824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8cab3a0 0x5622e8cba2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.180+0000 7fc80f400640 1 -- 172.21.15.47:0/2664839824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622e8cba890 con 0x5622e8caa9e0 2024-03-21T11:43:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.181+0000 7fc80f400640 1 --2- 172.21.15.47:0/2664839824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8cb7280 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00ad40 tx=0x7fc7fc016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=706223f3b4c5ba68 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.181+0000 7fc7f9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.181+0000 7fc7f9600640 1 -- 172.21.15.47:0/2664839824 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc005510 con 0x5622e8caa9e0 2024-03-21T11:43:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.181+0000 7fc81986e740 1 -- 172.21.15.47:0/2664839824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7fc02a070 con 0x5622e8caa9e0 2024-03-21T11:43:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.181+0000 7fc7f9600640 1 -- 172.21.15.47:0/2664839824 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc7fc004020 con 0x5622e8caa9e0 2024-03-21T11:43:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.182+0000 7fc7f9600640 1 -- 172.21.15.47:0/2664839824 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7fc04c050 con 0x5622e8caa9e0 2024-03-21T11:43:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.182+0000 7fc81986e740 10 client.31900 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.182+0000 7fc81986e740 1 -- 172.21.15.47:0/2664839824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5622e8d1b460 con 0x5622e8caa9e0 2024-03-21T11:43:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.182+0000 7fc7f9600640 1 -- 172.21.15.47:0/2664839824 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7fc02a070 con 0x5622e8caa9e0 2024-03-21T11:43:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.182+0000 7fc7f9600640 1 -- 172.21.15.47:0/2664839824 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc7fc047030 con 0x5622e8caa9e0 2024-03-21T11:43:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.182+0000 7fc81986e740 10 client.31900 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.182+0000 7fc81986e740 10 client.31900 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:43:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.182+0000 7fc81986e740 20 client.31900 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.182+0000 7fc81986e740 1 --2- 172.21.15.47:0/2664839824 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622e8cbf3d0 0x5622e8d1ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.182+0000 7fc81986e740 4 client.31900 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:43:14.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.182+0000 7fc81986e740 1 -- 172.21.15.47:0/2664839824 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5622e8c15b50 con 0x5622e8cbf3d0 2024-03-21T11:43:14.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.185+0000 7fc80fe00640 1 --2- 172.21.15.47:0/2664839824 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622e8cbf3d0 0x5622e8d1ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.186+0000 7fc80fe00640 1 --2- 172.21.15.47:0/2664839824 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622e8cbf3d0 0x5622e8d1ddc0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a55a3bdb78b956c0 server_cookie=bc730bb15840044d in_seq=0 out_seq=0 2024-03-21T11:43:14.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.186+0000 7fc7f9600640 10 client.31900 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:43:14.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.186+0000 7fc7f9600640 1 -- 172.21.15.47:0/2664839824 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5622e8c15b50 con 0x5622e8cbf3d0 2024-03-21T11:43:14.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.186+0000 7fc7f9600640 10 client.31900 handle_command_reply: tid=0 2024-03-21T11:43:14.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.186+0000 7fc81986e740 1 client.31900 shutdown 2024-03-21T11:43:14.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.186+0000 7fc81986e740 1 -- 172.21.15.47:0/2664839824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 msgr2=0x5622e8cb7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.186+0000 7fc81986e740 1 --2- 172.21.15.47:0/2664839824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8cb7280 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00ad40 tx=0x7fc7fc016000 comp rx=0 tx=0).stop 2024-03-21T11:43:14.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.187+0000 7fc81986e740 1 -- 172.21.15.47:0/2664839824 shutdown_connections 2024-03-21T11:43:14.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.187+0000 7fc81986e740 1 --2- 172.21.15.47:0/2664839824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622e8cab3a0 0x5622e8cba2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.187+0000 7fc81986e740 1 --2- 172.21.15.47:0/2664839824 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5622e8cbf3d0 0x5622e8d1ddc0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.187+0000 7fc81986e740 1 --2- 172.21.15.47:0/2664839824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5622e8caa9e0 0x5622e8cb7280 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.187+0000 7fc81986e740 1 --2- 172.21.15.47:0/2664839824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622e8ca9810 0x5622e8cb6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.187+0000 7fc81986e740 1 -- 172.21.15.47:0/2664839824 >> 172.21.15.47:0/2664839824 conn(0x5622e8ca4c00 msgr2=0x5622e8ca86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.187+0000 7fc7f9600640 0 client.31900 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:43:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.187+0000 7fc81986e740 1 -- 172.21.15.47:0/2664839824 shutdown_connections 2024-03-21T11:43:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.187+0000 7fc81986e740 1 -- 172.21.15.47:0/2664839824 wait complete. 2024-03-21T11:43:14.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.189+0000 7fc81986e740 20 client.31900 trim_cache size 0 max 16384 2024-03-21T11:43:14.191 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:43:14.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.191+0000 7fc816c00640 1 -- 172.21.15.47:0/1729672976 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7f40a2f20 msgr2=0x7fc7f40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.191+0000 7fc816c00640 1 --2- 172.21.15.47:0/1729672976 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7f40a2f20 0x7fc7f40a53e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x5622e8caa8e0 tx=0x7fc7fc005f90 comp rx=0 tx=0).stop 2024-03-21T11:43:14.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.191+0000 7fc816c00640 1 -- 172.21.15.47:0/1729672976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc81010e1e0 msgr2=0x7fc8101a9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.191+0000 7fc816c00640 1 --2- 172.21.15.47:0/1729672976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc81010e1e0 0x7fc8101a9a40 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fc80800c570 tx=0x7fc80800ca30 comp rx=0 tx=0).stop 2024-03-21T11:43:14.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.191+0000 7fc816c00640 1 -- 172.21.15.47:0/1729672976 shutdown_connections 2024-03-21T11:43:14.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.191+0000 7fc816c00640 1 --2- 172.21.15.47:0/1729672976 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc7f40a2f20 0x7fc7f40a53e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.191+0000 7fc816c00640 1 --2- 172.21.15.47:0/1729672976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc81010e1e0 0x7fc8101a9a40 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.191+0000 7fc816c00640 1 --2- 172.21.15.47:0/1729672976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc81010b860 0x7fc8101a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.191+0000 7fc816c00640 1 --2- 172.21.15.47:0/1729672976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc810108f30 0x7fc8101a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.191+0000 7fc816c00640 1 -- 172.21.15.47:0/1729672976 >> 172.21.15.47:0/1729672976 conn(0x7fc81006e500 msgr2=0x7fc81010a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.193+0000 7fc816c00640 1 -- 172.21.15.47:0/1729672976 shutdown_connections 2024-03-21T11:43:14.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.195+0000 7fc816c00640 1 -- 172.21.15.47:0/1729672976 wait complete. 2024-03-21T11:43:14.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:13 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3746068995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:43:14.222 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:43:14.222 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:43:14.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:13 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3746068995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:43:14.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.435+0000 7f985a400640 1 -- 172.21.15.47:0/3716127031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854104ab0 msgr2=0x7f9854104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.435+0000 7f985a400640 1 --2- 172.21.15.47:0/3716127031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854104ab0 0x7f9854104f30 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f983c009900 tx=0x7f983c02bc50 comp rx=0 tx=0).stop 2024-03-21T11:43:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.435+0000 7f985a400640 1 -- 172.21.15.47:0/3716127031 shutdown_connections 2024-03-21T11:43:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.435+0000 7f985a400640 1 --2- 172.21.15.47:0/3716127031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9854105470 0x7f985410bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.435+0000 7f985a400640 1 --2- 172.21.15.47:0/3716127031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854104ab0 0x7f9854104f30 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.435+0000 7f985a400640 1 --2- 172.21.15.47:0/3716127031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98541038c0 0x7f9854103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.435+0000 7f985a400640 1 -- 172.21.15.47:0/3716127031 >> 172.21.15.47:0/3716127031 conn(0x7f98540fee50 msgr2=0x7f9854101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.435+0000 7f985a400640 1 -- 172.21.15.47:0/3716127031 shutdown_connections 2024-03-21T11:43:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.435+0000 7f985a400640 1 -- 172.21.15.47:0/3716127031 wait complete. 2024-03-21T11:43:14.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.436+0000 7f985a400640 1 Processor -- start 2024-03-21T11:43:14.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.436+0000 7f985a400640 1 -- start start 2024-03-21T11:43:14.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.437+0000 7f985a400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98541038c0 0x7f9854079450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.437+0000 7f985a400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9854104ab0 0x7f9854079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.437+0000 7f985a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854105470 0x7f9854075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.437+0000 7f985a400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985410f520 con 0x7f98541038c0 2024-03-21T11:43:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.437+0000 7f985a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985410f3a0 con 0x7f9854105470 2024-03-21T11:43:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.437+0000 7f985a400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985410f6a0 con 0x7f9854104ab0 2024-03-21T11:43:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.437+0000 7f9853400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98541038c0 0x7f9854079450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.437+0000 7f984be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9854104ab0 0x7f9854079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.437+0000 7f984be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9854104ab0 0x7f9854079990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50848/0 (socket says 172.21.15.47:50848) 2024-03-21T11:43:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.437+0000 7f984be00640 1 -- 172.21.15.47:0/1686525598 learned_addr learned my addr 172.21.15.47:0/1686525598 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.437+0000 7f9853e00640 1 --2- 172.21.15.47:0/1686525598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854105470 0x7f9854075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.438+0000 7f984be00640 1 -- 172.21.15.47:0/1686525598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854105470 msgr2=0x7f9854075fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.438+0000 7f984be00640 1 --2- 172.21.15.47:0/1686525598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854105470 0x7f9854075fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.438+0000 7f984be00640 1 -- 172.21.15.47:0/1686525598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98541038c0 msgr2=0x7f9854079450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.438+0000 7f984be00640 1 --2- 172.21.15.47:0/1686525598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98541038c0 0x7f9854079450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.438+0000 7f984be00640 1 -- 172.21.15.47:0/1686525598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9854076880 con 0x7f9854104ab0 2024-03-21T11:43:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.438+0000 7f9853e00640 1 --2- 172.21.15.47:0/1686525598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854105470 0x7f9854075fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.438+0000 7f984be00640 1 --2- 172.21.15.47:0/1686525598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9854104ab0 0x7f9854079990 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f983c009230 tx=0x7f983c02c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.438+0000 7f9850c00640 1 -- 172.21.15.47:0/1686525598 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c042020 con 0x7f9854104ab0 2024-03-21T11:43:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.438+0000 7f985a400640 1 -- 172.21.15.47:0/1686525598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98541a60b0 con 0x7f9854104ab0 2024-03-21T11:43:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.438+0000 7f9850c00640 1 -- 172.21.15.47:0/1686525598 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f983c004450 con 0x7f9854104ab0 2024-03-21T11:43:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.438+0000 7f9850c00640 1 -- 172.21.15.47:0/1686525598 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c003e20 con 0x7f9854104ab0 2024-03-21T11:43:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.439+0000 7f985a400640 1 -- 172.21.15.47:0/1686525598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98541a64b0 con 0x7f9854104ab0 2024-03-21T11:43:14.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.442+0000 7f9850c00640 1 -- 172.21.15.47:0/1686525598 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f983c045020 con 0x7f9854104ab0 2024-03-21T11:43:14.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.442+0000 7f985d1e9740 1 Processor -- start 2024-03-21T11:43:14.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.443+0000 7f985d1e9740 1 -- start start 2024-03-21T11:43:14.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.443+0000 7f985d1e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cd810 0x562f328cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.443+0000 7f985d1e9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328ce9e0 0x562f328cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.443+0000 7f985d1e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328cf3a0 0x562f328d5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.443+0000 7f985d1e9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f328322c0 con 0x562f328ce9e0 2024-03-21T11:43:14.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.443+0000 7f985d1e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f327dac90 con 0x562f328cf3a0 2024-03-21T11:43:14.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.443+0000 7f985d1e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f3282e360 con 0x562f328cd810 2024-03-21T11:43:14.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.444+0000 7f984be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328cf3a0 0x562f328d5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.444+0000 7f984be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328cf3a0 0x562f328d5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58168/0 (socket says 172.21.15.47:58168) 2024-03-21T11:43:14.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.444+0000 7f984be00640 1 -- 172.21.15.47:0/271249977 learned_addr learned my addr 172.21.15.47:0/271249977 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.444+0000 7f9853e00640 1 --2- 172.21.15.47:0/271249977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cd810 0x562f328cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.444+0000 7f9850c00640 1 --2- 172.21.15.47:0/1686525598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98380a2fa0 0x7f98380a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.444+0000 7f9853400640 1 --2- 172.21.15.47:0/271249977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328ce9e0 0x562f328cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.444+0000 7f9850c00640 1 -- 172.21.15.47:0/1686525598 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f983c0f1720 con 0x7f9854104ab0 2024-03-21T11:43:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.444+0000 7f984be00640 1 -- 172.21.15.47:0/271249977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cd810 msgr2=0x562f328cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.444+0000 7f984be00640 1 --2- 172.21.15.47:0/271249977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cd810 0x562f328cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.444+0000 7f984be00640 1 -- 172.21.15.47:0/271249977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328ce9e0 msgr2=0x562f328cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.444+0000 7f984be00640 1 --2- 172.21.15.47:0/271249977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328ce9e0 0x562f328cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.445+0000 7f984be00640 1 -- 172.21.15.47:0/271249977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f327ff080 con 0x562f328cf3a0 2024-03-21T11:43:14.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.445+0000 7f9853e00640 1 --2- 172.21.15.47:0/1686525598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98380a2fa0 0x7f98380a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.445+0000 7f9853400640 1 --2- 172.21.15.47:0/271249977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328ce9e0 0x562f328cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:43:14.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.445+0000 7f984be00640 1 --2- 172.21.15.47:0/271249977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328cf3a0 0x562f328d5c30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f983c0bd9f0 tx=0x7f983c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46f7e36b5d08cd52 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.445+0000 7f9848c00640 1 -- 172.21.15.47:0/271249977 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0bd5a0 con 0x562f328cf3a0 2024-03-21T11:43:14.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.445+0000 7f9848c00640 1 -- 172.21.15.47:0/271249977 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f983c004c60 con 0x562f328cf3a0 2024-03-21T11:43:14.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.445+0000 7f9848c00640 1 -- 172.21.15.47:0/271249977 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0bd3c0 con 0x562f328cf3a0 2024-03-21T11:43:14.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.446+0000 7f985d1e9740 1 -- 172.21.15.47:0/271249977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328cf3a0 msgr2=0x562f328d5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.446+0000 7f985d1e9740 1 --2- 172.21.15.47:0/271249977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328cf3a0 0x562f328d5c30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f983c0bd9f0 tx=0x7f983c009260 comp rx=0 tx=0).stop 2024-03-21T11:43:14.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.446+0000 7f9853e00640 1 --2- 172.21.15.47:0/1686525598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98380a2fa0 0x7f98380a5460 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x562f328ce8e0 tx=0x7f984c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.446+0000 7f985d1e9740 1 -- 172.21.15.47:0/271249977 shutdown_connections 2024-03-21T11:43:14.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.446+0000 7f985d1e9740 1 --2- 172.21.15.47:0/271249977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328cf3a0 0x562f328d5c30 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.446+0000 7f985d1e9740 1 --2- 172.21.15.47:0/271249977 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328ce9e0 0x562f328cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.446+0000 7f985d1e9740 1 --2- 172.21.15.47:0/271249977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cd810 0x562f328cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.446+0000 7f985d1e9740 1 -- 172.21.15.47:0/271249977 >> 172.21.15.47:0/271249977 conn(0x562f328c8c00 msgr2=0x562f328c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.446+0000 7f985d1e9740 1 -- 172.21.15.47:0/271249977 shutdown_connections 2024-03-21T11:43:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.446+0000 7f985d1e9740 1 -- 172.21.15.47:0/271249977 wait complete. 2024-03-21T11:43:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.447+0000 7f985d1e9740 1 -- start start 2024-03-21T11:43:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.447+0000 7f985d1e9740 1 Processor -- start 2024-03-21T11:43:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.447+0000 7f985d1e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cd810 0x562f328d76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.447+0000 7f985d1e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328ce9e0 0x562f328d7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f985d1e9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cf3a0 0x562f328dec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f985d1e9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f328d6170 con 0x562f328cf3a0 2024-03-21T11:43:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f985d1e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f327ee0d0 con 0x562f328ce9e0 2024-03-21T11:43:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f985d1e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f328d62f0 con 0x562f328cd810 2024-03-21T11:43:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f985d1e9740 10 client.0 _ll_register_callbacks cb 0x562f327f2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f9853400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cd810 0x562f328d76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f984be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328ce9e0 0x562f328d7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f9853400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cd810 0x562f328d76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50862/0 (socket says 172.21.15.47:50862) 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f9853400640 1 -- 172.21.15.47:0/1097095776 learned_addr learned my addr 172.21.15.47:0/1097095776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f9853e00640 1 --2- 172.21.15.47:0/1097095776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cf3a0 0x562f328dec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f9853400640 1 -- 172.21.15.47:0/1097095776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328ce9e0 msgr2=0x562f328d7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f9853400640 1 --2- 172.21.15.47:0/1097095776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328ce9e0 0x562f328d7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f9853400640 1 -- 172.21.15.47:0/1097095776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cf3a0 msgr2=0x562f328dec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f9853400640 1 --2- 172.21.15.47:0/1097095776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cf3a0 0x562f328dec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f9853400640 1 -- 172.21.15.47:0/1097095776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f325463f0 con 0x562f328cd810 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f9853e00640 1 --2- 172.21.15.47:0/1097095776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cf3a0 0x562f328dec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.448+0000 7f984be00640 1 --2- 172.21.15.47:0/1097095776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328ce9e0 0x562f328d7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.449+0000 7f9853400640 1 --2- 172.21.15.47:0/1097095776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cd810 0x562f328d76f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f98440136e0 tx=0x7f9844013ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51efd19f67989140 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.449+0000 7f9841600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:14.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.449+0000 7f9841600640 1 -- 172.21.15.47:0/1097095776 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844020060 con 0x562f328cd810 2024-03-21T11:43:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.449+0000 7f9841600640 1 -- 172.21.15.47:0/1097095776 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9844011d70 con 0x562f328cd810 2024-03-21T11:43:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.449+0000 7f985d1e9740 1 -- 172.21.15.47:0/1097095776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f983c004c60 con 0x562f328cd810 2024-03-21T11:43:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.450+0000 7f9841600640 1 -- 172.21.15.47:0/1097095776 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f984401b430 con 0x562f328cd810 2024-03-21T11:43:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.450+0000 7f985d1e9740 10 client.31912 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.450+0000 7f985d1e9740 1 -- 172.21.15.47:0/1097095776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562f326107d0 con 0x562f328cd810 2024-03-21T11:43:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.450+0000 7f9841600640 1 -- 172.21.15.47:0/1097095776 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f983c004c60 con 0x562f328cd810 2024-03-21T11:43:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.450+0000 7f9841600640 1 -- 172.21.15.47:0/1097095776 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f984400e570 con 0x562f328cd810 2024-03-21T11:43:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.450+0000 7f985d1e9740 10 client.31912 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.450+0000 7f985d1e9740 10 client.31912 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:43:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.450+0000 7f985d1e9740 20 client.31912 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.450+0000 7f985d1e9740 1 --2- 172.21.15.47:0/1097095776 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562f328e30c0 0x562f328e5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.450+0000 7f985d1e9740 4 client.31912 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.450+0000 7f985d1e9740 1 -- 172.21.15.47:0/1097095776 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f3283f800 con 0x562f328e30c0 2024-03-21T11:43:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.452+0000 7f984be00640 1 --2- 172.21.15.47:0/1097095776 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562f328e30c0 0x562f328e5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.454+0000 7f984be00640 1 --2- 172.21.15.47:0/1097095776 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562f328e30c0 0x562f328e5530 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=996c85cee1da03c1 server_cookie=88d513c096b58f0b in_seq=0 out_seq=0 2024-03-21T11:43:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.454+0000 7f9841600640 10 client.31912 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:43:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.455+0000 7f9841600640 1 -- 172.21.15.47:0/1097095776 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562f3283f800 con 0x562f328e30c0 2024-03-21T11:43:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.455+0000 7f9841600640 10 client.31912 handle_command_reply: tid=0 2024-03-21T11:43:14.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.455+0000 7f985d1e9740 1 client.31912 shutdown 2024-03-21T11:43:14.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.456+0000 7f985d1e9740 1 -- 172.21.15.47:0/1097095776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cd810 msgr2=0x562f328d76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.456+0000 7f985d1e9740 1 --2- 172.21.15.47:0/1097095776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cd810 0x562f328d76f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f98440136e0 tx=0x7f9844013ba0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.456+0000 7f985d1e9740 1 -- 172.21.15.47:0/1097095776 shutdown_connections 2024-03-21T11:43:14.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.456+0000 7f985d1e9740 1 --2- 172.21.15.47:0/1097095776 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562f328e30c0 0x562f328e5530 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.456+0000 7f985d1e9740 1 --2- 172.21.15.47:0/1097095776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cf3a0 0x562f328dec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.456+0000 7f985d1e9740 1 --2- 172.21.15.47:0/1097095776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328ce9e0 0x562f328d7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.456+0000 7f985d1e9740 1 --2- 172.21.15.47:0/1097095776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cd810 0x562f328d76f0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.456+0000 7f9841600640 0 client.31912 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:43:14.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.456+0000 7f985d1e9740 1 -- 172.21.15.47:0/1097095776 >> 172.21.15.47:0/1097095776 conn(0x562f328c8c00 msgr2=0x562f328c7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.456+0000 7f985d1e9740 1 -- 172.21.15.47:0/1097095776 shutdown_connections 2024-03-21T11:43:14.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.456+0000 7f985d1e9740 1 -- 172.21.15.47:0/1097095776 wait complete. 2024-03-21T11:43:14.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.457+0000 7f985d1e9740 20 client.31912 trim_cache size 0 max 16384 2024-03-21T11:43:14.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.474+0000 7f985d1e9740 1 Processor -- start 2024-03-21T11:43:14.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.475+0000 7f985d1e9740 1 -- start start 2024-03-21T11:43:14.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.475+0000 7f985d1e9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cd810 0x562f328e9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.475+0000 7f985d1e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328ce9e0 0x562f328deb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.475+0000 7f985d1e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cf3a0 0x562f328ea330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.475+0000 7f985d1e9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98541b07f0 con 0x562f328cd810 2024-03-21T11:43:14.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.475+0000 7f985d1e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98541b2170 con 0x562f328ce9e0 2024-03-21T11:43:14.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.475+0000 7f985d1e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98541b0e50 con 0x562f328cf3a0 2024-03-21T11:43:14.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.475+0000 7f984be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328ce9e0 0x562f328deb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.475+0000 7f9853400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cd810 0x562f328e9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.475+0000 7f9853e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cf3a0 0x562f328ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.475+0000 7f9853400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cd810 0x562f328e9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34040/0 (socket says 172.21.15.47:34040) 2024-03-21T11:43:14.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.475+0000 7f9853e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cf3a0 0x562f328ea330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50864/0 (socket says 172.21.15.47:50864) 2024-03-21T11:43:14.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.475+0000 7f9853400640 1 -- 172.21.15.47:0/2486309996 learned_addr learned my addr 172.21.15.47:0/2486309996 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.475+0000 7f984be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328ce9e0 0x562f328deb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58182/0 (socket says 172.21.15.47:58182) 2024-03-21T11:43:14.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.476+0000 7f984be00640 1 -- 172.21.15.47:0/2486309996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cf3a0 msgr2=0x562f328ea330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.476+0000 7f984be00640 1 --2- 172.21.15.47:0/2486309996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cf3a0 0x562f328ea330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.476+0000 7f984be00640 1 -- 172.21.15.47:0/2486309996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cd810 msgr2=0x562f328e9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.476+0000 7f984be00640 1 --2- 172.21.15.47:0/2486309996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cd810 0x562f328e9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.476+0000 7f984be00640 1 -- 172.21.15.47:0/2486309996 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f328d6150 con 0x562f328ce9e0 2024-03-21T11:43:14.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.476+0000 7f984be00640 1 --2- 172.21.15.47:0/2486309996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328ce9e0 0x562f328deb90 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f983c0f2000 tx=0x7f983c0f7530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5144c20aad2cd07e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.477+0000 7f9843e00640 1 -- 172.21.15.47:0/2486309996 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0bd3c0 con 0x562f328ce9e0 2024-03-21T11:43:14.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.477+0000 7f9843e00640 1 -- 172.21.15.47:0/2486309996 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f983c004c60 con 0x562f328ce9e0 2024-03-21T11:43:14.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.477+0000 7f9843e00640 1 -- 172.21.15.47:0/2486309996 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0f7220 con 0x562f328ce9e0 2024-03-21T11:43:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.478+0000 7f985d1e9740 1 -- 172.21.15.47:0/2486309996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328ce9e0 msgr2=0x562f328deb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.478+0000 7f985d1e9740 1 --2- 172.21.15.47:0/2486309996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328ce9e0 0x562f328deb90 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f983c0f2000 tx=0x7f983c0f7530 comp rx=0 tx=0).stop 2024-03-21T11:43:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.478+0000 7f985d1e9740 1 -- 172.21.15.47:0/2486309996 shutdown_connections 2024-03-21T11:43:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.478+0000 7f985d1e9740 1 --2- 172.21.15.47:0/2486309996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328cf3a0 0x562f328ea330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.478+0000 7f985d1e9740 1 --2- 172.21.15.47:0/2486309996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328ce9e0 0x562f328deb90 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.478+0000 7f985d1e9740 1 --2- 172.21.15.47:0/2486309996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cd810 0x562f328e9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.478+0000 7f985d1e9740 1 -- 172.21.15.47:0/2486309996 >> 172.21.15.47:0/2486309996 conn(0x562f328c8c00 msgr2=0x562f328c7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.478+0000 7f985d1e9740 1 -- 172.21.15.47:0/2486309996 shutdown_connections 2024-03-21T11:43:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.478+0000 7f985d1e9740 1 -- 172.21.15.47:0/2486309996 wait complete. 2024-03-21T11:43:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.479+0000 7f985d1e9740 1 -- start start 2024-03-21T11:43:14.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.479+0000 7f985d1e9740 1 Processor -- start 2024-03-21T11:43:14.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.480+0000 7f985d1e9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cd810 0x562f328dc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.480+0000 7f985d1e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328ce9e0 0x562f328dcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.480+0000 7f985d1e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328cf3a0 0x562f328dde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.480+0000 7f985d1e9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98541b1b10 con 0x562f328cd810 2024-03-21T11:43:14.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.480+0000 7f985d1e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98541b14b0 con 0x562f328cf3a0 2024-03-21T11:43:14.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.480+0000 7f985d1e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98541ae9b0 con 0x562f328ce9e0 2024-03-21T11:43:14.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.480+0000 7f985d1e9740 10 client.0 _ll_register_callbacks cb 0x7f9844009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:14.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.480+0000 7f984be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328ce9e0 0x562f328dcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.480+0000 7f984be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328ce9e0 0x562f328dcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50866/0 (socket says 172.21.15.47:50866) 2024-03-21T11:43:14.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.480+0000 7f984be00640 1 -- 172.21.15.47:0/2555099337 learned_addr learned my addr 172.21.15.47:0/2555099337 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.481+0000 7f984be00640 1 -- 172.21.15.47:0/2555099337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328cf3a0 msgr2=0x562f328dde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.481+0000 7f984be00640 1 --2- 172.21.15.47:0/2555099337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328cf3a0 0x562f328dde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.481+0000 7f984be00640 1 -- 172.21.15.47:0/2555099337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cd810 msgr2=0x562f328dc8b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:43:14.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.481+0000 7f984be00640 1 --2- 172.21.15.47:0/2555099337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cd810 0x562f328dc8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.481+0000 7f984be00640 1 -- 172.21.15.47:0/2555099337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f328c9670 con 0x562f328ce9e0 2024-03-21T11:43:14.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.482+0000 7f984be00640 1 --2- 172.21.15.47:0/2555099337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328ce9e0 0x562f328dcdf0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f983c000c00 tx=0x7f983c0f1a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=470ca355813d46b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.482+0000 7f9841600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:14.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.482+0000 7f9841600640 1 -- 172.21.15.47:0/2555099337 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0c1bb0 con 0x562f328ce9e0 2024-03-21T11:43:14.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.483+0000 7f9841600640 1 -- 172.21.15.47:0/2555099337 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f983c0f1430 con 0x562f328ce9e0 2024-03-21T11:43:14.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.483+0000 7f985d1e9740 1 -- 172.21.15.47:0/2555099337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f983c004c60 con 0x562f328ce9e0 2024-03-21T11:43:14.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.483+0000 7f9841600640 1 -- 172.21.15.47:0/2555099337 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f983c0c6470 con 0x562f328ce9e0 2024-03-21T11:43:14.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.483+0000 7f985d1e9740 10 client.31918 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:14.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.483+0000 7f985d1e9740 1 -- 172.21.15.47:0/2555099337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562f3293edd0 con 0x562f328ce9e0 2024-03-21T11:43:14.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.484+0000 7f9841600640 1 -- 172.21.15.47:0/2555099337 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f983c004c60 con 0x562f328ce9e0 2024-03-21T11:43:14.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.484+0000 7f9841600640 1 -- 172.21.15.47:0/2555099337 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f983c0f0070 con 0x562f328ce9e0 2024-03-21T11:43:14.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.484+0000 7f985d1e9740 10 client.31918 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:14.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.484+0000 7f985d1e9740 10 client.31918 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:43:14.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.484+0000 7f985d1e9740 20 client.31918 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:14.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.484+0000 7f985d1e9740 1 --2- 172.21.15.47:0/2555099337 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562f3293f380 0x562f329417d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.484+0000 7f985d1e9740 4 client.31918 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:43:14.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.485+0000 7f985d1e9740 1 -- 172.21.15.47:0/2555099337 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562f32839b50 con 0x562f3293f380 2024-03-21T11:43:14.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.486+0000 7f9853400640 1 --2- 172.21.15.47:0/2555099337 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562f3293f380 0x562f329417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.488+0000 7f9853400640 1 --2- 172.21.15.47:0/2555099337 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562f3293f380 0x562f329417d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7867ea04facbf832 server_cookie=ee659958e6daa566 in_seq=0 out_seq=0 2024-03-21T11:43:14.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.488+0000 7f9841600640 10 client.31918 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:43:14.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.488+0000 7f9841600640 1 -- 172.21.15.47:0/2555099337 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562f32839b50 con 0x562f3293f380 2024-03-21T11:43:14.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.488+0000 7f9841600640 10 client.31918 handle_command_reply: tid=0 2024-03-21T11:43:14.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.488+0000 7f985d1e9740 1 client.31918 shutdown 2024-03-21T11:43:14.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.489+0000 7f985d1e9740 1 -- 172.21.15.47:0/2555099337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328ce9e0 msgr2=0x562f328dcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.489+0000 7f985d1e9740 1 --2- 172.21.15.47:0/2555099337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328ce9e0 0x562f328dcdf0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f983c000c00 tx=0x7f983c0f1a40 comp rx=0 tx=0).stop 2024-03-21T11:43:14.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.489+0000 7f985d1e9740 1 -- 172.21.15.47:0/2555099337 shutdown_connections 2024-03-21T11:43:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.489+0000 7f985d1e9740 1 --2- 172.21.15.47:0/2555099337 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x562f3293f380 0x562f329417d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.489+0000 7f985d1e9740 1 --2- 172.21.15.47:0/2555099337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f328cf3a0 0x562f328dde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.489+0000 7f985d1e9740 1 --2- 172.21.15.47:0/2555099337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f328ce9e0 0x562f328dcdf0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.489+0000 7f985d1e9740 1 --2- 172.21.15.47:0/2555099337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562f328cd810 0x562f328dc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.489+0000 7f985d1e9740 1 -- 172.21.15.47:0/2555099337 >> 172.21.15.47:0/2555099337 conn(0x562f328c8c00 msgr2=0x562f328c7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.489+0000 7f9841600640 0 client.31918 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:43:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.489+0000 7f985d1e9740 1 -- 172.21.15.47:0/2555099337 shutdown_connections 2024-03-21T11:43:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.489+0000 7f985d1e9740 1 -- 172.21.15.47:0/2555099337 wait complete. 2024-03-21T11:43:14.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.489+0000 7f985d1e9740 20 client.31918 trim_cache size 0 max 16384 2024-03-21T11:43:14.492 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:43:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.490+0000 7f985a400640 1 -- 172.21.15.47:0/1686525598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98380a2fa0 msgr2=0x7f98380a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.490+0000 7f985a400640 1 --2- 172.21.15.47:0/1686525598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98380a2fa0 0x7f98380a5460 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x562f328ce8e0 tx=0x7f984c004ab0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.490+0000 7f985a400640 1 -- 172.21.15.47:0/1686525598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9854104ab0 msgr2=0x7f9854079990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.490+0000 7f985a400640 1 --2- 172.21.15.47:0/1686525598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9854104ab0 0x7f9854079990 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f983c009230 tx=0x7f983c02c670 comp rx=0 tx=0).stop 2024-03-21T11:43:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.491+0000 7f985a400640 1 -- 172.21.15.47:0/1686525598 shutdown_connections 2024-03-21T11:43:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.491+0000 7f985a400640 1 --2- 172.21.15.47:0/1686525598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98380a2fa0 0x7f98380a5460 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.491+0000 7f985a400640 1 --2- 172.21.15.47:0/1686525598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9854105470 0x7f9854075fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.491+0000 7f985a400640 1 --2- 172.21.15.47:0/1686525598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9854104ab0 0x7f9854079990 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.491+0000 7f985a400640 1 --2- 172.21.15.47:0/1686525598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98541038c0 0x7f9854079450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.491+0000 7f985a400640 1 -- 172.21.15.47:0/1686525598 >> 172.21.15.47:0/1686525598 conn(0x7f98540fee50 msgr2=0x7f9854107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.491+0000 7f985a400640 1 -- 172.21.15.47:0/1686525598 shutdown_connections 2024-03-21T11:43:14.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.491+0000 7f985a400640 1 -- 172.21.15.47:0/1686525598 wait complete. 2024-03-21T11:43:14.512 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:43:14.512 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:43:14.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.720+0000 7fd12f200640 1 -- 172.21.15.47:0/1053499239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd130104990 msgr2=0x7fd130104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.720+0000 7fd12f200640 1 --2- 172.21.15.47:0/1053499239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd130104990 0x7fd130104e10 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fd118009900 tx=0x7fd11802bc30 comp rx=0 tx=0).stop 2024-03-21T11:43:14.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.720+0000 7fd12f200640 1 -- 172.21.15.47:0/1053499239 shutdown_connections 2024-03-21T11:43:14.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.720+0000 7fd12f200640 1 --2- 172.21.15.47:0/1053499239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd130105350 0x7fd13010bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.720+0000 7fd12f200640 1 --2- 172.21.15.47:0/1053499239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd130104990 0x7fd130104e10 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.720+0000 7fd12f200640 1 --2- 172.21.15.47:0/1053499239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1301037a0 0x7fd130103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.720+0000 7fd12f200640 1 -- 172.21.15.47:0/1053499239 >> 172.21.15.47:0/1053499239 conn(0x7fd1300fed10 msgr2=0x7fd130101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.720+0000 7fd12f200640 1 -- 172.21.15.47:0/1053499239 shutdown_connections 2024-03-21T11:43:14.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.720+0000 7fd12f200640 1 -- 172.21.15.47:0/1053499239 wait complete. 2024-03-21T11:43:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.721+0000 7fd12f200640 1 Processor -- start 2024-03-21T11:43:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.722+0000 7fd12f200640 1 -- start start 2024-03-21T11:43:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.722+0000 7fd12f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1301037a0 0x7fd130199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.722+0000 7fd12f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd130104990 0x7fd13019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.722+0000 7fd12f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd130105350 0x7fd1301a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.722+0000 7fd12f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13010f470 con 0x7fd1301037a0 2024-03-21T11:43:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.722+0000 7fd12f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13010f2f0 con 0x7fd130105350 2024-03-21T11:43:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.722+0000 7fd12f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13010f5f0 con 0x7fd130104990 2024-03-21T11:43:14.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.722+0000 7fd12de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1301037a0 0x7fd130199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.722+0000 7fd12de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1301037a0 0x7fd130199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34066/0 (socket says 172.21.15.47:34066) 2024-03-21T11:43:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.722+0000 7fd12de00640 1 -- 172.21.15.47:0/822884343 learned_addr learned my addr 172.21.15.47:0/822884343 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.722+0000 7fd125400640 1 --2- 172.21.15.47:0/822884343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd130104990 0x7fd13019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.722+0000 7fd12e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd130105350 0x7fd1301a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.723+0000 7fd12de00640 1 -- 172.21.15.47:0/822884343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd130104990 msgr2=0x7fd13019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.723+0000 7fd12de00640 1 --2- 172.21.15.47:0/822884343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd130104990 0x7fd13019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.723+0000 7fd12de00640 1 -- 172.21.15.47:0/822884343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd130105350 msgr2=0x7fd1301a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.723+0000 7fd12de00640 1 --2- 172.21.15.47:0/822884343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd130105350 0x7fd1301a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.723+0000 7fd12de00640 1 -- 172.21.15.47:0/822884343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1301a16f0 con 0x7fd1301037a0 2024-03-21T11:43:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.723+0000 7fd125400640 1 --2- 172.21.15.47:0/822884343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd130104990 0x7fd13019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:14.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.723+0000 7fd12de00640 1 --2- 172.21.15.47:0/822884343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1301037a0 0x7fd130199ad0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd12000e9c0 tx=0x7fd12000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.723+0000 7fd127400640 1 -- 172.21.15.47:0/822884343 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12000cd80 con 0x7fd1301037a0 2024-03-21T11:43:14.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.723+0000 7fd12f200640 1 -- 172.21.15.47:0/822884343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd130075b40 con 0x7fd1301037a0 2024-03-21T11:43:14.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.724+0000 7fd12f200640 1 -- 172.21.15.47:0/822884343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1300760b0 con 0x7fd1301037a0 2024-03-21T11:43:14.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.724+0000 7fd127400640 1 -- 172.21.15.47:0/822884343 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1200035f0 con 0x7fd1301037a0 2024-03-21T11:43:14.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.724+0000 7fd127400640 1 -- 172.21.15.47:0/822884343 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd120010690 con 0x7fd1301037a0 2024-03-21T11:43:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.726+0000 7fd137dd4740 1 Processor -- start 2024-03-21T11:43:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.727+0000 7fd137dd4740 1 -- start start 2024-03-21T11:43:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.727+0000 7fd127400640 1 -- 172.21.15.47:0/822884343 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd120010830 con 0x7fd1301037a0 2024-03-21T11:43:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.727+0000 7fd137dd4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.727+0000 7fd137dd4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a3bc0 0x559b269a4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.727+0000 7fd137dd4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a4580 0x559b269aae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.727+0000 7fd137dd4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b269072c0 con 0x559b269a3bc0 2024-03-21T11:43:14.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.727+0000 7fd137dd4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b268afc90 con 0x559b269a4580 2024-03-21T11:43:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.727+0000 7fd137dd4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b26903360 con 0x559b269a29f0 2024-03-21T11:43:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.727+0000 7fd12e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.727+0000 7fd125400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a4580 0x559b269aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.727+0000 7fd12e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269a2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50886/0 (socket says 172.21.15.47:50886) 2024-03-21T11:43:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.727+0000 7fd12e800640 1 -- 172.21.15.47:0/60314787 learned_addr learned my addr 172.21.15.47:0/60314787 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.727+0000 7fd12de00640 1 --2- 172.21.15.47:0/60314787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a3bc0 0x559b269a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.728+0000 7fd12e800640 1 -- 172.21.15.47:0/60314787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a4580 msgr2=0x559b269aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.728+0000 7fd12e800640 1 --2- 172.21.15.47:0/60314787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a4580 0x559b269aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.728+0000 7fd12e800640 1 -- 172.21.15.47:0/60314787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a3bc0 msgr2=0x559b269a4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.728+0000 7fd12e800640 1 --2- 172.21.15.47:0/60314787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a3bc0 0x559b269a4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.728+0000 7fd12e800640 1 -- 172.21.15.47:0/60314787 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b268d4080 con 0x559b269a29f0 2024-03-21T11:43:14.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.728+0000 7fd12e800640 1 --2- 172.21.15.47:0/60314787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269a2e10 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fd128009fd0 tx=0x7fd1280118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed25314e14feefef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.728+0000 7fd11f400640 1 -- 172.21.15.47:0/60314787 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd128004d20 con 0x559b269a29f0 2024-03-21T11:43:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.728+0000 7fd11f400640 1 -- 172.21.15.47:0/60314787 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd128004050 con 0x559b269a29f0 2024-03-21T11:43:14.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.728+0000 7fd11f400640 1 -- 172.21.15.47:0/60314787 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd128005260 con 0x559b269a29f0 2024-03-21T11:43:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.729+0000 7fd127400640 1 --2- 172.21.15.47:0/822884343 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1140a2fa0 0x7fd1140a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.729+0000 7fd127400640 1 -- 172.21.15.47:0/822884343 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd1200cfac0 con 0x7fd1301037a0 2024-03-21T11:43:14.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.729+0000 7fd12de00640 1 --2- 172.21.15.47:0/822884343 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1140a2fa0 0x7fd1140a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.729+0000 7fd137dd4740 1 -- 172.21.15.47:0/60314787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 msgr2=0x559b269a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.729+0000 7fd137dd4740 1 --2- 172.21.15.47:0/60314787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269a2e10 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fd128009fd0 tx=0x7fd1280118d0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.729+0000 7fd137dd4740 1 -- 172.21.15.47:0/60314787 shutdown_connections 2024-03-21T11:43:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.729+0000 7fd137dd4740 1 --2- 172.21.15.47:0/60314787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a4580 0x559b269aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.729+0000 7fd137dd4740 1 --2- 172.21.15.47:0/60314787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a3bc0 0x559b269a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.729+0000 7fd137dd4740 1 --2- 172.21.15.47:0/60314787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269a2e10 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.729+0000 7fd137dd4740 1 -- 172.21.15.47:0/60314787 >> 172.21.15.47:0/60314787 conn(0x559b2699dde0 msgr2=0x559b2699e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.729+0000 7fd137dd4740 1 -- 172.21.15.47:0/60314787 shutdown_connections 2024-03-21T11:43:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.729+0000 7fd137dd4740 1 -- 172.21.15.47:0/60314787 wait complete. 2024-03-21T11:43:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.730+0000 7fd137dd4740 1 -- start start 2024-03-21T11:43:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.730+0000 7fd12de00640 1 --2- 172.21.15.47:0/822884343 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1140a2fa0 0x7fd1140a5460 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd1200a42e0 tx=0x7fd120017000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.730+0000 7fd137dd4740 1 Processor -- start 2024-03-21T11:43:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd137dd4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd137dd4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a3bc0 0x559b269ad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd137dd4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a4580 0x559b269b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd137dd4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b269ab590 con 0x559b269a3bc0 2024-03-21T11:43:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd137dd4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b268c30d0 con 0x559b269a4580 2024-03-21T11:43:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd137dd4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b269ab710 con 0x559b269a29f0 2024-03-21T11:43:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd137dd4740 10 client.0 _ll_register_callbacks cb 0x559b268c7e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd125400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a3bc0 0x559b269ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd12e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd12e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269acbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50890/0 (socket says 172.21.15.47:50890) 2024-03-21T11:43:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd12e800640 1 -- 172.21.15.47:0/3238848969 learned_addr learned my addr 172.21.15.47:0/3238848969 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd12e800640 1 --2- 172.21.15.47:0/3238848969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a4580 0x559b269b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd125400640 1 -- 172.21.15.47:0/3238848969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 msgr2=0x559b269acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd125400640 1 --2- 172.21.15.47:0/3238848969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd125400640 1 -- 172.21.15.47:0/3238848969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a4580 msgr2=0x559b269b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd125400640 1 --2- 172.21.15.47:0/3238848969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a4580 0x559b269b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.731+0000 7fd125400640 1 -- 172.21.15.47:0/3238848969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b2661b3f0 con 0x559b269a3bc0 2024-03-21T11:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.732+0000 7fd125400640 1 --2- 172.21.15.47:0/3238848969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a3bc0 0x559b269ad120 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd11802c170 tx=0x7fd1180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464ca0d588750cec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.732+0000 7fd113e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.732+0000 7fd113e00640 1 -- 172.21.15.47:0/3238848969 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118005d50 con 0x559b269a3bc0 2024-03-21T11:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.732+0000 7fd113e00640 1 -- 172.21.15.47:0/3238848969 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd118004ed0 con 0x559b269a3bc0 2024-03-21T11:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.732+0000 7fd137dd4740 1 -- 172.21.15.47:0/3238848969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd128004050 con 0x559b269a3bc0 2024-03-21T11:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.732+0000 7fd113e00640 1 -- 172.21.15.47:0/3238848969 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd11802cd30 con 0x559b269a3bc0 2024-03-21T11:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.732+0000 7fd137dd4740 10 client.21342 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.732+0000 7fd137dd4740 1 -- 172.21.15.47:0/3238848969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559b266e57d0 con 0x559b269a3bc0 2024-03-21T11:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.732+0000 7fd113e00640 1 -- 172.21.15.47:0/3238848969 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd128004050 con 0x559b269a3bc0 2024-03-21T11:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.733+0000 7fd113e00640 1 -- 172.21.15.47:0/3238848969 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd1180039d0 con 0x559b269a3bc0 2024-03-21T11:43:14.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.733+0000 7fd137dd4740 10 client.21342 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.733+0000 7fd137dd4740 10 client.21342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.733+0000 7fd137dd4740 20 client.21342 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.733+0000 7fd137dd4740 1 --2- 172.21.15.47:0/3238848969 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b269b86e0 0x559b269baad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.733+0000 7fd137dd4740 4 client.21342 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.733+0000 7fd137dd4740 1 -- 172.21.15.47:0/3238848969 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b26914800 con 0x559b269b86e0 2024-03-21T11:43:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.733+0000 7fd12e800640 1 --2- 172.21.15.47:0/3238848969 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b269b86e0 0x559b269baad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.734+0000 7fd12e800640 1 --2- 172.21.15.47:0/3238848969 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b269b86e0 0x559b269baad0 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52446a44ca61495 server_cookie=465f403595653ebf in_seq=0 out_seq=0 2024-03-21T11:43:14.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.734+0000 7fd113e00640 10 client.21342 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.736+0000 7fd113e00640 1 -- 172.21.15.47:0/3238848969 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559b26914800 con 0x559b269b86e0 2024-03-21T11:43:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.736+0000 7fd113e00640 10 client.21342 handle_command_reply: tid=0 2024-03-21T11:43:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.736+0000 7fd137dd4740 1 client.21342 shutdown 2024-03-21T11:43:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.736+0000 7fd137dd4740 1 -- 172.21.15.47:0/3238848969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a3bc0 msgr2=0x559b269ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.736+0000 7fd137dd4740 1 --2- 172.21.15.47:0/3238848969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a3bc0 0x559b269ad120 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd11802c170 tx=0x7fd1180048f0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.736+0000 7fd137dd4740 1 -- 172.21.15.47:0/3238848969 shutdown_connections 2024-03-21T11:43:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.736+0000 7fd137dd4740 1 --2- 172.21.15.47:0/3238848969 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b269b86e0 0x559b269baad0 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.736+0000 7fd137dd4740 1 --2- 172.21.15.47:0/3238848969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a4580 0x559b269b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.736+0000 7fd137dd4740 1 --2- 172.21.15.47:0/3238848969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a3bc0 0x559b269ad120 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.736+0000 7fd113e00640 0 client.21342 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.736+0000 7fd137dd4740 1 --2- 172.21.15.47:0/3238848969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.736+0000 7fd137dd4740 1 -- 172.21.15.47:0/3238848969 >> 172.21.15.47:0/3238848969 conn(0x559b2699dde0 msgr2=0x559b2699d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.736+0000 7fd137dd4740 1 -- 172.21.15.47:0/3238848969 shutdown_connections 2024-03-21T11:43:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.737+0000 7fd137dd4740 1 -- 172.21.15.47:0/3238848969 wait complete. 2024-03-21T11:43:14.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.737+0000 7fd137dd4740 20 client.21342 trim_cache size 0 max 16384 2024-03-21T11:43:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.751+0000 7fd137dd4740 1 Processor -- start 2024-03-21T11:43:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.751+0000 7fd137dd4740 1 -- start start 2024-03-21T11:43:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd137dd4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a29f0 0x559b2699cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd137dd4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a3bc0 0x559b269b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd137dd4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a4580 0x559b269a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd137dd4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c00f9a0 con 0x559b269a4580 2024-03-21T11:43:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd137dd4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c011320 con 0x559b269a29f0 2024-03-21T11:43:14.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd137dd4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c010000 con 0x559b269a3bc0 2024-03-21T11:43:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd12e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a29f0 0x559b2699cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd125400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a3bc0 0x559b269b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd12e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a29f0 0x559b2699cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58232/0 (socket says 172.21.15.47:58232) 2024-03-21T11:43:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd12e800640 1 -- 172.21.15.47:0/3211519323 learned_addr learned my addr 172.21.15.47:0/3211519323 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd12e800640 1 --2- 172.21.15.47:0/3211519323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a4580 0x559b269a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd125400640 1 -- 172.21.15.47:0/3211519323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a29f0 msgr2=0x559b2699cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd125400640 1 --2- 172.21.15.47:0/3211519323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a29f0 0x559b2699cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd125400640 1 -- 172.21.15.47:0/3211519323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a4580 msgr2=0x559b269a6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd125400640 1 --2- 172.21.15.47:0/3211519323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a4580 0x559b269a6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.752+0000 7fd125400640 1 -- 172.21.15.47:0/3211519323 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b269aa650 con 0x559b269a3bc0 2024-03-21T11:43:14.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.753+0000 7fd125400640 1 --2- 172.21.15.47:0/3211519323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a3bc0 0x559b269b4120 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x559b269a3580 tx=0x7fd11802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=272deff5d512a8f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.753+0000 7fd11ea00640 1 -- 172.21.15.47:0/3211519323 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11805d070 con 0x559b269a3bc0 2024-03-21T11:43:14.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.753+0000 7fd11ea00640 1 -- 172.21.15.47:0/3211519323 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd11802cd30 con 0x559b269a3bc0 2024-03-21T11:43:14.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.753+0000 7fd11ea00640 1 -- 172.21.15.47:0/3211519323 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11804b9f0 con 0x559b269a3bc0 2024-03-21T11:43:14.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.754+0000 7fd137dd4740 1 -- 172.21.15.47:0/3211519323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a3bc0 msgr2=0x559b269b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.754+0000 7fd137dd4740 1 --2- 172.21.15.47:0/3211519323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a3bc0 0x559b269b4120 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x559b269a3580 tx=0x7fd11802bc30 comp rx=0 tx=0).stop 2024-03-21T11:43:14.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.754+0000 7fd137dd4740 1 -- 172.21.15.47:0/3211519323 shutdown_connections 2024-03-21T11:43:14.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.754+0000 7fd137dd4740 1 --2- 172.21.15.47:0/3211519323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a4580 0x559b269a6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.754+0000 7fd137dd4740 1 --2- 172.21.15.47:0/3211519323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a3bc0 0x559b269b4120 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.754+0000 7fd137dd4740 1 --2- 172.21.15.47:0/3211519323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a29f0 0x559b2699cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.754+0000 7fd137dd4740 1 -- 172.21.15.47:0/3211519323 >> 172.21.15.47:0/3211519323 conn(0x559b2699dde0 msgr2=0x559b2699c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.754+0000 7fd137dd4740 1 -- 172.21.15.47:0/3211519323 shutdown_connections 2024-03-21T11:43:14.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.754+0000 7fd137dd4740 1 -- 172.21.15.47:0/3211519323 wait complete. 2024-03-21T11:43:14.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.754+0000 7fd137dd4740 1 -- start start 2024-03-21T11:43:14.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.755+0000 7fd137dd4740 1 Processor -- start 2024-03-21T11:43:14.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.755+0000 7fd137dd4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.755+0000 7fd137dd4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a3bc0 0x559b269b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.755+0000 7fd137dd4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a4580 0x559b269b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.755+0000 7fd137dd4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c010cc0 con 0x559b269a4580 2024-03-21T11:43:14.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.755+0000 7fd137dd4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c010660 con 0x559b269a3bc0 2024-03-21T11:43:14.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.755+0000 7fd137dd4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c00db60 con 0x559b269a29f0 2024-03-21T11:43:14.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.755+0000 7fd137dd4740 10 client.0 _ll_register_callbacks cb 0x7fd118009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:14.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.755+0000 7fd125400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a3bc0 0x559b269b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.755+0000 7fd12e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.755+0000 7fd12e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a4580 0x559b269b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.755+0000 7fd125400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a3bc0 0x559b269b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58234/0 (socket says 172.21.15.47:58234) 2024-03-21T11:43:14.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.755+0000 7fd125400640 1 -- 172.21.15.47:0/2754872764 learned_addr learned my addr 172.21.15.47:0/2754872764 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:14.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.756+0000 7fd125400640 1 -- 172.21.15.47:0/2754872764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 msgr2=0x559b269b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.756+0000 7fd125400640 1 --2- 172.21.15.47:0/2754872764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.756+0000 7fd125400640 1 -- 172.21.15.47:0/2754872764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a4580 msgr2=0x559b269b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:14.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.756+0000 7fd125400640 1 --2- 172.21.15.47:0/2754872764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a4580 0x559b269b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.756+0000 7fd125400640 1 -- 172.21.15.47:0/2754872764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b269b24b0 con 0x559b269a3bc0 2024-03-21T11:43:14.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.756+0000 7fd12e800640 1 --2- 172.21.15.47:0/2754872764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a4580 0x559b269b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:14.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.756+0000 7fd125400640 1 --2- 172.21.15.47:0/2754872764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a3bc0 0x559b269b3df0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd11802c8e0 tx=0x7fd118003c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1d398a88e4d1aa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:14.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.756+0000 7fd113e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:43:14.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.757+0000 7fd113e00640 1 -- 172.21.15.47:0/2754872764 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11805d040 con 0x559b269a3bc0 2024-03-21T11:43:14.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.757+0000 7fd113e00640 1 -- 172.21.15.47:0/2754872764 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd11804b750 con 0x559b269a3bc0 2024-03-21T11:43:14.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.757+0000 7fd137dd4740 1 -- 172.21.15.47:0/2754872764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd11802cd30 con 0x559b269a3bc0 2024-03-21T11:43:14.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.757+0000 7fd113e00640 1 -- 172.21.15.47:0/2754872764 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1180048d0 con 0x559b269a3bc0 2024-03-21T11:43:14.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.757+0000 7fd137dd4740 10 client.31991 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:14.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.757+0000 7fd137dd4740 1 -- 172.21.15.47:0/2754872764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559b26a14880 con 0x559b269a3bc0 2024-03-21T11:43:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.757+0000 7fd113e00640 1 -- 172.21.15.47:0/2754872764 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd11802cd30 con 0x559b269a3bc0 2024-03-21T11:43:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.757+0000 7fd113e00640 1 -- 172.21.15.47:0/2754872764 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd118060020 con 0x559b269a3bc0 2024-03-21T11:43:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.757+0000 7fd137dd4740 10 client.31991 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.757+0000 7fd137dd4740 10 client.31991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.757+0000 7fd137dd4740 20 client.31991 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.758+0000 7fd137dd4740 1 --2- 172.21.15.47:0/2754872764 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b26a15200 0x559b26a17620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.758+0000 7fd137dd4740 4 client.31991 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:43:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.758+0000 7fd137dd4740 1 -- 172.21.15.47:0/2754872764 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559b2690eb50 con 0x559b26a15200 2024-03-21T11:43:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.758+0000 7fd12e800640 1 --2- 172.21.15.47:0/2754872764 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b26a15200 0x559b26a17620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.759+0000 7fd12e800640 1 --2- 172.21.15.47:0/2754872764 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b26a15200 0x559b26a17620 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77ad45c8c2cdca1b server_cookie=12bba469ef601982 in_seq=0 out_seq=0 2024-03-21T11:43:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.759+0000 7fd113e00640 10 client.31991 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.761+0000 7fd113e00640 1 -- 172.21.15.47:0/2754872764 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559b2690eb50 con 0x559b26a15200 2024-03-21T11:43:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.762+0000 7fd113e00640 10 client.31991 handle_command_reply: tid=0 2024-03-21T11:43:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.762+0000 7fd137dd4740 1 client.31991 shutdown 2024-03-21T11:43:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.762+0000 7fd137dd4740 1 -- 172.21.15.47:0/2754872764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a3bc0 msgr2=0x559b269b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.762+0000 7fd137dd4740 1 --2- 172.21.15.47:0/2754872764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a3bc0 0x559b269b3df0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd11802c8e0 tx=0x7fd118003c70 comp rx=0 tx=0).stop 2024-03-21T11:43:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.762+0000 7fd137dd4740 1 -- 172.21.15.47:0/2754872764 shutdown_connections 2024-03-21T11:43:14.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.762+0000 7fd137dd4740 1 --2- 172.21.15.47:0/2754872764 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b26a15200 0x559b26a17620 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.762+0000 7fd137dd4740 1 --2- 172.21.15.47:0/2754872764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b269a4580 0x559b269b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.762+0000 7fd137dd4740 1 --2- 172.21.15.47:0/2754872764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b269a3bc0 0x559b269b3df0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.762+0000 7fd137dd4740 1 --2- 172.21.15.47:0/2754872764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b269a29f0 0x559b269b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.762+0000 7fd137dd4740 1 -- 172.21.15.47:0/2754872764 >> 172.21.15.47:0/2754872764 conn(0x559b2699dde0 msgr2=0x559b2699d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.762+0000 7fd113e00640 0 client.31991 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:14.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.762+0000 7fd137dd4740 1 -- 172.21.15.47:0/2754872764 shutdown_connections 2024-03-21T11:43:14.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.762+0000 7fd137dd4740 1 -- 172.21.15.47:0/2754872764 wait complete. 2024-03-21T11:43:14.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.762+0000 7fd137dd4740 20 client.31991 trim_cache size 0 max 16384 2024-03-21T11:43:14.766 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:43:14.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.763+0000 7fd12f200640 1 -- 172.21.15.47:0/822884343 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1140a2fa0 msgr2=0x7fd1140a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.763+0000 7fd12f200640 1 --2- 172.21.15.47:0/822884343 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1140a2fa0 0x7fd1140a5460 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd1200a42e0 tx=0x7fd120017000 comp rx=0 tx=0).stop 2024-03-21T11:43:14.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.763+0000 7fd12f200640 1 -- 172.21.15.47:0/822884343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1301037a0 msgr2=0x7fd130199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:14.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.763+0000 7fd12f200640 1 --2- 172.21.15.47:0/822884343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1301037a0 0x7fd130199ad0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd12000e9c0 tx=0x7fd12000ee80 comp rx=0 tx=0).stop 2024-03-21T11:43:14.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.763+0000 7fd12f200640 1 -- 172.21.15.47:0/822884343 shutdown_connections 2024-03-21T11:43:14.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.763+0000 7fd12f200640 1 --2- 172.21.15.47:0/822884343 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1140a2fa0 0x7fd1140a5460 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.763+0000 7fd12f200640 1 --2- 172.21.15.47:0/822884343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd130105350 0x7fd1301a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.763+0000 7fd12f200640 1 --2- 172.21.15.47:0/822884343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd130104990 0x7fd13019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.763+0000 7fd12f200640 1 --2- 172.21.15.47:0/822884343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1301037a0 0x7fd130199ad0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:14.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.763+0000 7fd12f200640 1 -- 172.21.15.47:0/822884343 >> 172.21.15.47:0/822884343 conn(0x7fd1300fed10 msgr2=0x7fd130107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:14.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.763+0000 7fd12f200640 1 -- 172.21.15.47:0/822884343 shutdown_connections 2024-03-21T11:43:14.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:14.763+0000 7fd12f200640 1 -- 172.21.15.47:0/822884343 wait complete. 2024-03-21T11:43:14.963 INFO:tasks.workunit.client.0.smithi047.stdout: 10 160285 23.08 MB/sec execute 238 sec latency 104.634 ms 2024-03-21T11:43:15.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:14 smithi047 ceph-mon[31351]: pgmap v997: 129 pgs: 129 active+clean; 554 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 563 op/s 2024-03-21T11:43:15.141 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:14 smithi102 ceph-mon[36748]: pgmap v997: 129 pgs: 129 active+clean; 554 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 563 op/s 2024-03-21T11:43:15.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:14 smithi100 ceph-mon[36245]: pgmap v997: 129 pgs: 129 active+clean; 554 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.8 MiB/s wr, 563 op/s 2024-03-21T11:43:15.781 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:43:15.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:43:15.782 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:43:15.963 INFO:tasks.workunit.client.0.smithi047.stdout: 10 160881 23.09 MB/sec execute 239 sec latency 82.789 ms 2024-03-21T11:43:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.019+0000 7fac91e00640 1 -- 172.21.15.47:0/690417518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac8c1016d0 msgr2=0x7fac8c101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.019+0000 7fac91e00640 1 --2- 172.21.15.47:0/690417518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac8c1016d0 0x7fac8c101af0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009ca0 tx=0x7fac7c02c100 comp rx=0 tx=0).stop 2024-03-21T11:43:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.019+0000 7fac91e00640 1 -- 172.21.15.47:0/690417518 shutdown_connections 2024-03-21T11:43:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.019+0000 7fac91e00640 1 --2- 172.21.15.47:0/690417518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac8c103280 0x7fac8c109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.019+0000 7fac91e00640 1 --2- 172.21.15.47:0/690417518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac8c1028c0 0x7fac8c102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.019+0000 7fac91e00640 1 --2- 172.21.15.47:0/690417518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac8c1016d0 0x7fac8c101af0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.019+0000 7fac91e00640 1 -- 172.21.15.47:0/690417518 >> 172.21.15.47:0/690417518 conn(0x7fac8c0fcc40 msgr2=0x7fac8c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.019+0000 7fac91e00640 1 -- 172.21.15.47:0/690417518 shutdown_connections 2024-03-21T11:43:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.019+0000 7fac91e00640 1 -- 172.21.15.47:0/690417518 wait complete. 2024-03-21T11:43:16.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.020+0000 7fac91e00640 1 Processor -- start 2024-03-21T11:43:16.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.020+0000 7fac91e00640 1 -- start start 2024-03-21T11:43:16.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.021+0000 7fac91e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac8c1016d0 0x7fac8c195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.021+0000 7fac91e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac8c1028c0 0x7fac8c195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.021+0000 7fac91e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac8c103280 0x7fac8c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.021+0000 7fac91e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c10b170 con 0x7fac8c1028c0 2024-03-21T11:43:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.021+0000 7fac91e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c10aff0 con 0x7fac8c103280 2024-03-21T11:43:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.021+0000 7fac91e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c10b2f0 con 0x7fac8c1016d0 2024-03-21T11:43:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.021+0000 7fac8aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac8c1016d0 0x7fac8c195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.021+0000 7fac8aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac8c1016d0 0x7fac8c195810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50916/0 (socket says 172.21.15.47:50916) 2024-03-21T11:43:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.021+0000 7fac8aa00640 1 -- 172.21.15.47:0/247010985 learned_addr learned my addr 172.21.15.47:0/247010985 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.021+0000 7fac82000640 1 --2- 172.21.15.47:0/247010985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac8c1028c0 0x7fac8c195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.021+0000 7fac8b400640 1 --2- 172.21.15.47:0/247010985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac8c103280 0x7fac8c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.022+0000 7fac8aa00640 1 -- 172.21.15.47:0/247010985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac8c103280 msgr2=0x7fac8c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.022+0000 7fac8aa00640 1 --2- 172.21.15.47:0/247010985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac8c103280 0x7fac8c19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.022+0000 7fac8aa00640 1 -- 172.21.15.47:0/247010985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac8c1028c0 msgr2=0x7fac8c195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.022+0000 7fac8aa00640 1 --2- 172.21.15.47:0/247010985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac8c1028c0 0x7fac8c195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.022+0000 7fac8aa00640 1 -- 172.21.15.47:0/247010985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8c19d430 con 0x7fac8c1016d0 2024-03-21T11:43:16.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.022+0000 7fac8aa00640 1 --2- 172.21.15.47:0/247010985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac8c1016d0 0x7fac8c195810 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0324c0 tx=0x7fac7c035d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:16.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.023+0000 7fac90a00640 1 -- 172.21.15.47:0/247010985 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c039040 con 0x7fac8c1016d0 2024-03-21T11:43:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.025+0000 7fac90a00640 1 -- 172.21.15.47:0/247010985 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac7c044070 con 0x7fac8c1016d0 2024-03-21T11:43:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.025+0000 7fac90a00640 1 -- 172.21.15.47:0/247010985 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c004460 con 0x7fac8c1016d0 2024-03-21T11:43:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.025+0000 7fac91e00640 1 -- 172.21.15.47:0/247010985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac8c19d6c0 con 0x7fac8c1016d0 2024-03-21T11:43:16.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.025+0000 7fac91e00640 1 -- 172.21.15.47:0/247010985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac8c19db60 con 0x7fac8c1016d0 2024-03-21T11:43:16.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.029+0000 7fac90a00640 1 -- 172.21.15.47:0/247010985 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fac7c004600 con 0x7fac8c1016d0 2024-03-21T11:43:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.029+0000 7fac94bf4740 1 Processor -- start 2024-03-21T11:43:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.029+0000 7fac94bf4740 1 -- start start 2024-03-21T11:43:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.030+0000 7fac94bf4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e702860a0 0x558e702864c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.030+0000 7fac94bf4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e70286a90 0x558e7028b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.030+0000 7fac94bf4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e7028e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.030+0000 7fac94bf4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e701ebeb0 con 0x558e7028bc00 2024-03-21T11:43:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.030+0000 7fac94bf4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e70194a60 con 0x558e70286a90 2024-03-21T11:43:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.030+0000 7fac94bf4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e701e7f50 con 0x558e702860a0 2024-03-21T11:43:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.030+0000 7fac82000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e7028e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.030+0000 7fac82000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e7028e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34114/0 (socket says 172.21.15.47:34114) 2024-03-21T11:43:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.030+0000 7fac82000640 1 -- 172.21.15.47:0/1848112880 learned_addr learned my addr 172.21.15.47:0/1848112880 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.031+0000 7fac8aa00640 1 --2- 172.21.15.47:0/1848112880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e70286a90 0x558e7028b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.031+0000 7fac90a00640 1 --2- 172.21.15.47:0/247010985 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac700a3070 0x7fac700a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.031+0000 7fac8aa00640 1 -- 172.21.15.47:0/1848112880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e702860a0 msgr2=0x558e702864c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.031+0000 7fac8aa00640 1 --2- 172.21.15.47:0/1848112880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e702860a0 0x558e702864c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.031+0000 7fac8aa00640 1 -- 172.21.15.47:0/1848112880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 msgr2=0x558e7028e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.031+0000 7fac8aa00640 1 --2- 172.21.15.47:0/1848112880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e7028e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.031+0000 7fac8aa00640 1 -- 172.21.15.47:0/1848112880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e6ffc87e0 con 0x558e70286a90 2024-03-21T11:43:16.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.031+0000 7fac82000640 1 --2- 172.21.15.47:0/1848112880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e7028e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:16.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.031+0000 7fac8b400640 1 --2- 172.21.15.47:0/247010985 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac700a3070 0x7fac700a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.032+0000 7fac90a00640 1 -- 172.21.15.47:0/247010985 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fac7c0f1d50 con 0x7fac8c1016d0 2024-03-21T11:43:16.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.032+0000 7fac8aa00640 1 --2- 172.21.15.47:0/1848112880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e70286a90 0x558e7028b590 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0ef430 tx=0x7fac7c0b3570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b169460da749eb92 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:16.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.032+0000 7fac7be00640 1 -- 172.21.15.47:0/1848112880 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c0bdbd0 con 0x558e70286a90 2024-03-21T11:43:16.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.032+0000 7fac7be00640 1 -- 172.21.15.47:0/1848112880 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac7c0be790 con 0x558e70286a90 2024-03-21T11:43:16.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.032+0000 7fac7be00640 1 -- 172.21.15.47:0/1848112880 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c090040 con 0x558e70286a90 2024-03-21T11:43:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.033+0000 7fac94bf4740 1 -- 172.21.15.47:0/1848112880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e70286a90 msgr2=0x558e7028b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.033+0000 7fac94bf4740 1 --2- 172.21.15.47:0/1848112880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e70286a90 0x558e7028b590 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0ef430 tx=0x7fac7c0b3570 comp rx=0 tx=0).stop 2024-03-21T11:43:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.033+0000 7fac8b400640 1 --2- 172.21.15.47:0/247010985 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac700a3070 0x7fac700a5530 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x558e702872f0 tx=0x7fac8400b7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.034+0000 7fac94bf4740 1 -- 172.21.15.47:0/1848112880 shutdown_connections 2024-03-21T11:43:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.034+0000 7fac94bf4740 1 --2- 172.21.15.47:0/1848112880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e7028e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.034+0000 7fac94bf4740 1 --2- 172.21.15.47:0/1848112880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e70286a90 0x558e7028b590 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.034+0000 7fac94bf4740 1 --2- 172.21.15.47:0/1848112880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e702860a0 0x558e702864c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.034+0000 7fac94bf4740 1 -- 172.21.15.47:0/1848112880 >> 172.21.15.47:0/1848112880 conn(0x558e70281610 msgr2=0x558e70281a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:16.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.035+0000 7fac94bf4740 1 -- 172.21.15.47:0/1848112880 shutdown_connections 2024-03-21T11:43:16.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.035+0000 7fac94bf4740 1 -- 172.21.15.47:0/1848112880 wait complete. 2024-03-21T11:43:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.036+0000 7fac94bf4740 1 -- start start 2024-03-21T11:43:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.036+0000 7fac94bf4740 1 Processor -- start 2024-03-21T11:43:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.037+0000 7fac94bf4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e702860a0 0x558e7028fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.037+0000 7fac94bf4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e70286a90 0x558e70290430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.037+0000 7fac94bf4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e70297470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.037+0000 7fac94bf4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7028e970 con 0x558e7028bc00 2024-03-21T11:43:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.037+0000 7fac94bf4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e701a7eb0 con 0x558e70286a90 2024-03-21T11:43:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.037+0000 7fac94bf4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7028eaf0 con 0x558e702860a0 2024-03-21T11:43:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.037+0000 7fac94bf4740 10 client.0 _ll_register_callbacks cb 0x558e7019c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.037+0000 7fac82000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e70286a90 0x558e70290430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.037+0000 7fac82000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e70286a90 0x558e70290430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58264/0 (socket says 172.21.15.47:58264) 2024-03-21T11:43:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.037+0000 7fac82000640 1 -- 172.21.15.47:0/940411730 learned_addr learned my addr 172.21.15.47:0/940411730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.037+0000 7fac8aa00640 1 --2- 172.21.15.47:0/940411730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e702860a0 0x558e7028fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.037+0000 7fac8b400640 1 --2- 172.21.15.47:0/940411730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e70297470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.038+0000 7fac82000640 1 -- 172.21.15.47:0/940411730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e702860a0 msgr2=0x558e7028fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.038+0000 7fac82000640 1 --2- 172.21.15.47:0/940411730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e702860a0 0x558e7028fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.038+0000 7fac82000640 1 -- 172.21.15.47:0/940411730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 msgr2=0x558e70297470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.038+0000 7fac82000640 1 --2- 172.21.15.47:0/940411730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e70297470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.038+0000 7fac82000640 1 -- 172.21.15.47:0/940411730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e70028f70 con 0x558e70286a90 2024-03-21T11:43:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.038+0000 7fac82000640 1 --2- 172.21.15.47:0/940411730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e70286a90 0x558e70290430 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fac74004590 tx=0x7fac740039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76473677217664f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.038+0000 7fac78c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:43:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.039+0000 7fac78c00640 1 -- 172.21.15.47:0/940411730 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7401f020 con 0x558e70286a90 2024-03-21T11:43:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.039+0000 7fac78c00640 1 -- 172.21.15.47:0/940411730 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fac740099e0 con 0x558e70286a90 2024-03-21T11:43:16.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.039+0000 7fac94bf4740 1 -- 172.21.15.47:0/940411730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac7c0be790 con 0x558e70286a90 2024-03-21T11:43:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.039+0000 7fac78c00640 1 -- 172.21.15.47:0/940411730 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac74009be0 con 0x558e70286a90 2024-03-21T11:43:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.039+0000 7fac94bf4740 10 client.32003 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.039+0000 7fac94bf4740 1 -- 172.21.15.47:0/940411730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558e6fec0170 con 0x558e70286a90 2024-03-21T11:43:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.040+0000 7fac78c00640 1 -- 172.21.15.47:0/940411730 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac7c0be790 con 0x558e70286a90 2024-03-21T11:43:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.040+0000 7fac78c00640 1 -- 172.21.15.47:0/940411730 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fac7400ec00 con 0x558e70286a90 2024-03-21T11:43:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.040+0000 7fac94bf4740 10 client.32003 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.040+0000 7fac94bf4740 10 client.32003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.040+0000 7fac94bf4740 20 client.32003 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.040+0000 7fac94bf4740 1 --2- 172.21.15.47:0/940411730 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558e7029bd80 0x558e7029e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.040+0000 7fac94bf4740 4 client.32003 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.040+0000 7fac94bf4740 1 -- 172.21.15.47:0/940411730 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e701f93f0 con 0x558e7029bd80 2024-03-21T11:43:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.041+0000 7fac8aa00640 1 --2- 172.21.15.47:0/940411730 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558e7029bd80 0x558e7029e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.044+0000 7fac8aa00640 1 --2- 172.21.15.47:0/940411730 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558e7029bd80 0x558e7029e1c0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=352b2c2a76153d9f server_cookie=39cf0925a787d595 in_seq=0 out_seq=0 2024-03-21T11:43:16.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.044+0000 7fac78c00640 10 client.32003 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:16.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.047+0000 7fac78c00640 1 -- 172.21.15.47:0/940411730 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558e701f93f0 con 0x558e7029bd80 2024-03-21T11:43:16.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.047+0000 7fac78c00640 10 client.32003 handle_command_reply: tid=0 2024-03-21T11:43:16.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.047+0000 7fac94bf4740 1 client.32003 shutdown 2024-03-21T11:43:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.047+0000 7fac94bf4740 1 -- 172.21.15.47:0/940411730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e70286a90 msgr2=0x558e70290430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.047+0000 7fac94bf4740 1 --2- 172.21.15.47:0/940411730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e70286a90 0x558e70290430 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fac74004590 tx=0x7fac740039c0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.047+0000 7fac94bf4740 1 -- 172.21.15.47:0/940411730 shutdown_connections 2024-03-21T11:43:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.047+0000 7fac94bf4740 1 --2- 172.21.15.47:0/940411730 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558e7029bd80 0x558e7029e1c0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.047+0000 7fac94bf4740 1 --2- 172.21.15.47:0/940411730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e70297470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.047+0000 7fac94bf4740 1 --2- 172.21.15.47:0/940411730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e70286a90 0x558e70290430 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.047+0000 7fac94bf4740 1 --2- 172.21.15.47:0/940411730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e702860a0 0x558e7028fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.047+0000 7fac94bf4740 1 -- 172.21.15.47:0/940411730 >> 172.21.15.47:0/940411730 conn(0x558e70281610 msgr2=0x558e70280830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.047+0000 7fac78c00640 0 client.32003 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.048+0000 7fac94bf4740 1 -- 172.21.15.47:0/940411730 shutdown_connections 2024-03-21T11:43:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.048+0000 7fac94bf4740 1 -- 172.21.15.47:0/940411730 wait complete. 2024-03-21T11:43:16.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.048+0000 7fac94bf4740 20 client.32003 trim_cache size 0 max 16384 2024-03-21T11:43:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.084+0000 7fac94bf4740 1 Processor -- start 2024-03-21T11:43:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.084+0000 7fac94bf4740 1 -- start start 2024-03-21T11:43:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.085+0000 7fac94bf4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e702860a0 0x558e70284f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.085+0000 7fac94bf4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e70286a90 0x558e70287500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.085+0000 7fac94bf4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e70287a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.085+0000 7fac94bf4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c1a7e20 con 0x558e7028bc00 2024-03-21T11:43:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.085+0000 7fac94bf4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c1a97a0 con 0x558e702860a0 2024-03-21T11:43:16.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.085+0000 7fac94bf4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c1a8480 con 0x558e70286a90 2024-03-21T11:43:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.085+0000 7fac8aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e702860a0 0x558e70284f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.085+0000 7fac8b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e70287a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.085+0000 7fac8b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e70287a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34134/0 (socket says 172.21.15.47:34134) 2024-03-21T11:43:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.085+0000 7fac8b400640 1 -- 172.21.15.47:0/2068794386 learned_addr learned my addr 172.21.15.47:0/2068794386 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.085+0000 7fac82000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e70286a90 0x558e70287500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.086+0000 7fac82000640 1 -- 172.21.15.47:0/2068794386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e702860a0 msgr2=0x558e70284f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.086+0000 7fac82000640 1 --2- 172.21.15.47:0/2068794386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e702860a0 0x558e70284f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.086+0000 7fac82000640 1 -- 172.21.15.47:0/2068794386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 msgr2=0x558e70287a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.086+0000 7fac82000640 1 --2- 172.21.15.47:0/2068794386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e70287a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.086+0000 7fac82000640 1 -- 172.21.15.47:0/2068794386 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e70286e70 con 0x558e70286a90 2024-03-21T11:43:16.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.086+0000 7fac82000640 1 --2- 172.21.15.47:0/2068794386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e70286a90 0x558e70287500 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fac74011010 tx=0x7fac7400f9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6c4ef501b66ec62 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:16.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.087+0000 7fac7b400640 1 -- 172.21.15.47:0/2068794386 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac74033070 con 0x558e70286a90 2024-03-21T11:43:16.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.087+0000 7fac7b400640 1 -- 172.21.15.47:0/2068794386 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac7401b6c0 con 0x558e70286a90 2024-03-21T11:43:16.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.087+0000 7fac7b400640 1 -- 172.21.15.47:0/2068794386 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7401f030 con 0x558e70286a90 2024-03-21T11:43:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.088+0000 7fac94bf4740 1 -- 172.21.15.47:0/2068794386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e70286a90 msgr2=0x558e70287500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.088+0000 7fac94bf4740 1 --2- 172.21.15.47:0/2068794386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e70286a90 0x558e70287500 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fac74011010 tx=0x7fac7400f9c0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.088+0000 7fac94bf4740 1 -- 172.21.15.47:0/2068794386 shutdown_connections 2024-03-21T11:43:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.088+0000 7fac94bf4740 1 --2- 172.21.15.47:0/2068794386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e70287a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.088+0000 7fac94bf4740 1 --2- 172.21.15.47:0/2068794386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e70286a90 0x558e70287500 secure :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fac74011010 tx=0x7fac7400f9c0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.088+0000 7fac94bf4740 1 --2- 172.21.15.47:0/2068794386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e702860a0 0x558e70284f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.088+0000 7fac94bf4740 1 -- 172.21.15.47:0/2068794386 >> 172.21.15.47:0/2068794386 conn(0x558e70281610 msgr2=0x558e70284870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.088+0000 7fac94bf4740 1 -- 172.21.15.47:0/2068794386 shutdown_connections 2024-03-21T11:43:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.088+0000 7fac94bf4740 1 -- 172.21.15.47:0/2068794386 wait complete. 2024-03-21T11:43:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.089+0000 7fac94bf4740 1 -- start start 2024-03-21T11:43:16.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.089+0000 7fac94bf4740 1 Processor -- start 2024-03-21T11:43:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.090+0000 7fac94bf4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e702860a0 0x558e70295300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.090+0000 7fac94bf4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e70295840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.090+0000 7fac94bf4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e70296d40 0x558e70295dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.090+0000 7fac94bf4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c1a9140 con 0x558e7028bc00 2024-03-21T11:43:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.090+0000 7fac94bf4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c1a8ae0 con 0x558e702860a0 2024-03-21T11:43:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.090+0000 7fac94bf4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c1a5fe0 con 0x558e70296d40 2024-03-21T11:43:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.090+0000 7fac94bf4740 10 client.0 _ll_register_callbacks cb 0x7fac7400fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.090+0000 7fac8aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e702860a0 0x558e70295300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.090+0000 7fac82000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e70295840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.090+0000 7fac8b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e70296d40 0x558e70295dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.090+0000 7fac8aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e702860a0 0x558e70295300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58296/0 (socket says 172.21.15.47:58296) 2024-03-21T11:43:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.090+0000 7fac8aa00640 1 -- 172.21.15.47:0/691468203 learned_addr learned my addr 172.21.15.47:0/691468203 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.091+0000 7fac8b400640 1 -- 172.21.15.47:0/691468203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e702860a0 msgr2=0x558e70295300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.091+0000 7fac8b400640 1 --2- 172.21.15.47:0/691468203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e702860a0 0x558e70295300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.091+0000 7fac8b400640 1 -- 172.21.15.47:0/691468203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 msgr2=0x558e70295840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.091+0000 7fac8b400640 1 --2- 172.21.15.47:0/691468203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e70295840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.091+0000 7fac8b400640 1 -- 172.21.15.47:0/691468203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e70300190 con 0x558e70296d40 2024-03-21T11:43:16.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.091+0000 7fac8aa00640 1 --2- 172.21.15.47:0/691468203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e702860a0 0x558e70295300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:16.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.091+0000 7fac8b400640 1 --2- 172.21.15.47:0/691468203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e70296d40 0x558e70295dc0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x558e70298930 tx=0x7fac84005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac40cbcef30b254 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:16.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.091+0000 7fac78c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.093+0000 7fac78c00640 1 -- 172.21.15.47:0/691468203 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8402d060 con 0x558e70296d40 2024-03-21T11:43:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.093+0000 7fac78c00640 1 -- 172.21.15.47:0/691468203 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fac8401f840 con 0x558e70296d40 2024-03-21T11:43:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.094+0000 7fac94bf4740 1 -- 172.21.15.47:0/691468203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac7401b6c0 con 0x558e70296d40 2024-03-21T11:43:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.094+0000 7fac78c00640 1 -- 172.21.15.47:0/691468203 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac8401fa40 con 0x558e70296d40 2024-03-21T11:43:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.094+0000 7fac94bf4740 10 client.31951 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.094+0000 7fac94bf4740 1 -- 172.21.15.47:0/691468203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558e703015a0 con 0x558e70296d40 2024-03-21T11:43:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.095+0000 7fac78c00640 1 -- 172.21.15.47:0/691468203 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac7401b6c0 con 0x558e70296d40 2024-03-21T11:43:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.095+0000 7fac78c00640 1 -- 172.21.15.47:0/691468203 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fac8401d3b0 con 0x558e70296d40 2024-03-21T11:43:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.095+0000 7fac94bf4740 10 client.31951 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.095+0000 7fac94bf4740 10 client.31951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:16.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.095+0000 7fac94bf4740 20 client.31951 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.095+0000 7fac94bf4740 1 --2- 172.21.15.47:0/691468203 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558e70301ec0 0x558e703042e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.095+0000 7fac94bf4740 4 client.31951 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:43:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.095+0000 7fac94bf4740 1 -- 172.21.15.47:0/691468203 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558e701f3740 con 0x558e70301ec0 2024-03-21T11:43:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.096+0000 7fac8aa00640 1 --2- 172.21.15.47:0/691468203 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558e70301ec0 0x558e703042e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.097+0000 7fac8aa00640 1 --2- 172.21.15.47:0/691468203 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558e70301ec0 0x558e703042e0 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce2415540d5b8a38 server_cookie=f13430f46e270dba in_seq=0 out_seq=0 2024-03-21T11:43:16.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.097+0000 7fac78c00640 10 client.31951 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:16.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.103+0000 7fac78c00640 1 -- 172.21.15.47:0/691468203 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558e701f3740 con 0x558e70301ec0 2024-03-21T11:43:16.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.103+0000 7fac78c00640 10 client.31951 handle_command_reply: tid=0 2024-03-21T11:43:16.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.103+0000 7fac94bf4740 1 client.31951 shutdown 2024-03-21T11:43:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.104+0000 7fac94bf4740 1 -- 172.21.15.47:0/691468203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e70296d40 msgr2=0x558e70295dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.104+0000 7fac94bf4740 1 --2- 172.21.15.47:0/691468203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e70296d40 0x558e70295dc0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x558e70298930 tx=0x7fac84005cc0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.105+0000 7fac94bf4740 1 -- 172.21.15.47:0/691468203 shutdown_connections 2024-03-21T11:43:16.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.105+0000 7fac94bf4740 1 --2- 172.21.15.47:0/691468203 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558e70301ec0 0x558e703042e0 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.105+0000 7fac94bf4740 1 --2- 172.21.15.47:0/691468203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e70296d40 0x558e70295dc0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.105+0000 7fac94bf4740 1 --2- 172.21.15.47:0/691468203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e7028bc00 0x558e70295840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.105+0000 7fac94bf4740 1 --2- 172.21.15.47:0/691468203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e702860a0 0x558e70295300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.105+0000 7fac94bf4740 1 -- 172.21.15.47:0/691468203 >> 172.21.15.47:0/691468203 conn(0x558e70281610 msgr2=0x558e70285100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:16.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.105+0000 7fac78c00640 0 client.31951 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:16.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.105+0000 7fac94bf4740 1 -- 172.21.15.47:0/691468203 shutdown_connections 2024-03-21T11:43:16.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.106+0000 7fac94bf4740 1 -- 172.21.15.47:0/691468203 wait complete. 2024-03-21T11:43:16.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.107+0000 7fac94bf4740 20 client.31951 trim_cache size 0 max 16384 2024-03-21T11:43:16.108 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:43:16.108 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:43:16.108 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "f783bdff-a122-410a-a030-75330a72a3d8", 2024-03-21T11:43:16.108 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:43:16.108 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:43:16.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.108+0000 7fac91e00640 1 -- 172.21.15.47:0/247010985 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac700a3070 msgr2=0x7fac700a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.108+0000 7fac91e00640 1 --2- 172.21.15.47:0/247010985 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac700a3070 0x7fac700a5530 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x558e702872f0 tx=0x7fac8400b7e0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.108+0000 7fac91e00640 1 -- 172.21.15.47:0/247010985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac8c1016d0 msgr2=0x7fac8c195810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.108+0000 7fac91e00640 1 --2- 172.21.15.47:0/247010985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac8c1016d0 0x7fac8c195810 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0324c0 tx=0x7fac7c035d90 comp rx=0 tx=0).stop 2024-03-21T11:43:16.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.109+0000 7fac91e00640 1 -- 172.21.15.47:0/247010985 shutdown_connections 2024-03-21T11:43:16.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.109+0000 7fac91e00640 1 --2- 172.21.15.47:0/247010985 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac700a3070 0x7fac700a5530 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.109+0000 7fac91e00640 1 --2- 172.21.15.47:0/247010985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac8c103280 0x7fac8c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.109+0000 7fac91e00640 1 --2- 172.21.15.47:0/247010985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac8c1028c0 0x7fac8c195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.109+0000 7fac91e00640 1 --2- 172.21.15.47:0/247010985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac8c1016d0 0x7fac8c195810 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.109+0000 7fac91e00640 1 -- 172.21.15.47:0/247010985 >> 172.21.15.47:0/247010985 conn(0x7fac8c0fcc40 msgr2=0x7fac8c105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:16.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.109+0000 7fac91e00640 1 -- 172.21.15.47:0/247010985 shutdown_connections 2024-03-21T11:43:16.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.109+0000 7fac91e00640 1 -- 172.21.15.47:0/247010985 wait complete. 2024-03-21T11:43:16.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:43:16.135 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:43:16.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.424+0000 7fcf0ea00640 1 -- 172.21.15.47:0/420394326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf081016b0 msgr2=0x7fcf08101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.424+0000 7fcf0ea00640 1 --2- 172.21.15.47:0/420394326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf081016b0 0x7fcf08101ad0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009860 tx=0x7fcef802bb70 comp rx=0 tx=0).stop 2024-03-21T11:43:16.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.424+0000 7fcf0ea00640 1 -- 172.21.15.47:0/420394326 shutdown_connections 2024-03-21T11:43:16.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.424+0000 7fcf0ea00640 1 --2- 172.21.15.47:0/420394326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf08103260 0x7fcf08109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.424+0000 7fcf0ea00640 1 --2- 172.21.15.47:0/420394326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf081028a0 0x7fcf08102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.424+0000 7fcf0ea00640 1 --2- 172.21.15.47:0/420394326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf081016b0 0x7fcf08101ad0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.424+0000 7fcf0ea00640 1 -- 172.21.15.47:0/420394326 >> 172.21.15.47:0/420394326 conn(0x7fcf080fcc40 msgr2=0x7fcf080ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.424+0000 7fcf0ea00640 1 -- 172.21.15.47:0/420394326 shutdown_connections 2024-03-21T11:43:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.424+0000 7fcf0ea00640 1 -- 172.21.15.47:0/420394326 wait complete. 2024-03-21T11:43:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.425+0000 7fcf0ea00640 1 Processor -- start 2024-03-21T11:43:16.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.426+0000 7fcf0ea00640 1 -- start start 2024-03-21T11:43:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.426+0000 7fcf0ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf081016b0 0x7fcf08197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.426+0000 7fcf0ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf081028a0 0x7fcf08197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.426+0000 7fcf0ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf08103260 0x7fcf0819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.426+0000 7fcf0ea00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0810d380 con 0x7fcf08103260 2024-03-21T11:43:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.426+0000 7fcf0ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0810d200 con 0x7fcf081028a0 2024-03-21T11:43:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.426+0000 7fcf0ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0810d500 con 0x7fcf081016b0 2024-03-21T11:43:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.427+0000 7fcf07e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf081016b0 0x7fcf08197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.427+0000 7fcf07e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf081016b0 0x7fcf08197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:50982/0 (socket says 172.21.15.47:50982) 2024-03-21T11:43:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.427+0000 7fcf07400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf081028a0 0x7fcf08197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.427+0000 7fcf07e00640 1 -- 172.21.15.47:0/1052626562 learned_addr learned my addr 172.21.15.47:0/1052626562 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.427+0000 7fcf0ca00640 1 --2- 172.21.15.47:0/1052626562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf08103260 0x7fcf0819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.427+0000 7fcf07e00640 1 -- 172.21.15.47:0/1052626562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf081028a0 msgr2=0x7fcf08197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.427+0000 7fcf07e00640 1 --2- 172.21.15.47:0/1052626562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf081028a0 0x7fcf08197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.427+0000 7fcf07e00640 1 -- 172.21.15.47:0/1052626562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf08103260 msgr2=0x7fcf0819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.427+0000 7fcf07e00640 1 --2- 172.21.15.47:0/1052626562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf08103260 0x7fcf0819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.427+0000 7fcf07e00640 1 -- 172.21.15.47:0/1052626562 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0819f6a0 con 0x7fcf081016b0 2024-03-21T11:43:16.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.427+0000 7fcf07e00640 1 --2- 172.21.15.47:0/1052626562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf081016b0 0x7fcf08197a80 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fcef80349e0 tx=0x7fcef80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:16.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.428+0000 7fcf04c00640 1 -- 172.21.15.47:0/1052626562 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8041020 con 0x7fcf081016b0 2024-03-21T11:43:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.428+0000 7fcf04c00640 1 -- 172.21.15.47:0/1052626562 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcef80323b0 con 0x7fcf081016b0 2024-03-21T11:43:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.428+0000 7fcf0ea00640 1 -- 172.21.15.47:0/1052626562 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0819f930 con 0x7fcf081016b0 2024-03-21T11:43:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.428+0000 7fcf04c00640 1 -- 172.21.15.47:0/1052626562 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8039040 con 0x7fcf081016b0 2024-03-21T11:43:16.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.428+0000 7fcf0ea00640 1 -- 172.21.15.47:0/1052626562 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0819fef0 con 0x7fcf081016b0 2024-03-21T11:43:16.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.431+0000 7fcf11636740 1 Processor -- start 2024-03-21T11:43:16.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.432+0000 7fcf11636740 1 -- start start 2024-03-21T11:43:16.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.432+0000 7fcf04c00640 1 -- 172.21.15.47:0/1052626562 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcef8044020 con 0x7fcf081016b0 2024-03-21T11:43:16.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.432+0000 7fcf11636740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cee80 0x559f670cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.432+0000 7fcf11636740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cf870 0x559f670d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.432+0000 7fcf11636740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 0x559f670d6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.432+0000 7fcf11636740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f67034250 con 0x559f670d49e0 2024-03-21T11:43:16.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.432+0000 7fcf11636740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f66fdca30 con 0x559f670cee80 2024-03-21T11:43:16.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.432+0000 7fcf11636740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f670302f0 con 0x559f670cf870 2024-03-21T11:43:16.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.432+0000 7fcf0ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cee80 0x559f670cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.432+0000 7fcf0ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cee80 0x559f670cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58332/0 (socket says 172.21.15.47:58332) 2024-03-21T11:43:16.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.432+0000 7fcf0ca00640 1 -- 172.21.15.47:0/4030211432 learned_addr learned my addr 172.21.15.47:0/4030211432 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:16.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.432+0000 7fcf07e00640 1 --2- 172.21.15.47:0/4030211432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cf870 0x559f670d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.432+0000 7fcf07400640 1 --2- 172.21.15.47:0/4030211432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 0x559f670d6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.433+0000 7fcf0ca00640 1 -- 172.21.15.47:0/4030211432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cf870 msgr2=0x559f670d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.433+0000 7fcf0ca00640 1 --2- 172.21.15.47:0/4030211432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cf870 0x559f670d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.433+0000 7fcf0ca00640 1 -- 172.21.15.47:0/4030211432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 msgr2=0x559f670d6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.433+0000 7fcf0ca00640 1 --2- 172.21.15.47:0/4030211432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 0x559f670d6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.433+0000 7fcf0ca00640 1 -- 172.21.15.47:0/4030211432 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f66fc7690 con 0x559f670cee80 2024-03-21T11:43:16.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.433+0000 7fcf07400640 1 --2- 172.21.15.47:0/4030211432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 0x559f670d6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:16.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.433+0000 7fcf0ca00640 1 --2- 172.21.15.47:0/4030211432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cee80 0x559f670cf2a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000a430 tx=0x7fcf00009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d57acc3c0c0528cf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:16.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.434+0000 7fcefd200640 1 -- 172.21.15.47:0/4030211432 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00013790 con 0x559f670cee80 2024-03-21T11:43:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.434+0000 7fcefd200640 1 -- 172.21.15.47:0/4030211432 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf00019070 con 0x559f670cee80 2024-03-21T11:43:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.434+0000 7fcefd200640 1 -- 172.21.15.47:0/4030211432 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00014de0 con 0x559f670cee80 2024-03-21T11:43:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.434+0000 7fcf04c00640 1 --2- 172.21.15.47:0/1052626562 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fceec0a3020 0x7fceec0a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.434+0000 7fcf04c00640 1 -- 172.21.15.47:0/1052626562 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcef80f0fe0 con 0x7fcf081016b0 2024-03-21T11:43:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.434+0000 7fcf07e00640 1 --2- 172.21.15.47:0/1052626562 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fceec0a3020 0x7fceec0a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.435+0000 7fcf11636740 1 -- 172.21.15.47:0/4030211432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cee80 msgr2=0x559f670cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.435+0000 7fcf11636740 1 --2- 172.21.15.47:0/4030211432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cee80 0x559f670cf2a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000a430 tx=0x7fcf00009650 comp rx=0 tx=0).stop 2024-03-21T11:43:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.435+0000 7fcf11636740 1 -- 172.21.15.47:0/4030211432 shutdown_connections 2024-03-21T11:43:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.435+0000 7fcf11636740 1 --2- 172.21.15.47:0/4030211432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 0x559f670d6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.435+0000 7fcf11636740 1 --2- 172.21.15.47:0/4030211432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cf870 0x559f670d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.435+0000 7fcf11636740 1 --2- 172.21.15.47:0/4030211432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cee80 0x559f670cf2a0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.435+0000 7fcf11636740 1 -- 172.21.15.47:0/4030211432 >> 172.21.15.47:0/4030211432 conn(0x559f67097200 msgr2=0x559f670ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:16.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.435+0000 7fcf11636740 1 -- 172.21.15.47:0/4030211432 shutdown_connections 2024-03-21T11:43:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.435+0000 7fcf11636740 1 -- 172.21.15.47:0/4030211432 wait complete. 2024-03-21T11:43:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.435+0000 7fcf07e00640 1 --2- 172.21.15.47:0/1052626562 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fceec0a3020 0x7fceec0a54e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009830 tx=0x7fcef80f49a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.435+0000 7fcf11636740 1 -- start start 2024-03-21T11:43:16.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.436+0000 7fcf11636740 1 Processor -- start 2024-03-21T11:43:16.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.436+0000 7fcf11636740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 0x559f670d8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.436+0000 7fcf11636740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670cf870 0x559f670d93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.436+0000 7fcf11636740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670d49e0 0x559f670e0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.436+0000 7fcf11636740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f670d7990 con 0x559f670cf870 2024-03-21T11:43:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.436+0000 7fcf11636740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f66fefe80 con 0x559f670d49e0 2024-03-21T11:43:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.436+0000 7fcf11636740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f670d7b10 con 0x559f670cee80 2024-03-21T11:43:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.436+0000 7fcf0ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 0x559f670d8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.436+0000 7fcf11636740 10 client.0 _ll_register_callbacks cb 0x559f67002130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.436+0000 7fcf0ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670d49e0 0x559f670e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.437+0000 7fcf0ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670d49e0 0x559f670e0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58334/0 (socket says 172.21.15.47:58334) 2024-03-21T11:43:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.437+0000 7fcf0ca00640 1 -- 172.21.15.47:0/1219867267 learned_addr learned my addr 172.21.15.47:0/1219867267 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.437+0000 7fcf07400640 1 --2- 172.21.15.47:0/1219867267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670cf870 0x559f670d93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.437+0000 7fcf0ca00640 1 -- 172.21.15.47:0/1219867267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 msgr2=0x559f670d8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.437+0000 7fcf0ca00640 1 --2- 172.21.15.47:0/1219867267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 0x559f670d8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.437+0000 7fcf0ca00640 1 -- 172.21.15.47:0/1219867267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670cf870 msgr2=0x559f670d93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.437+0000 7fcf0ca00640 1 --2- 172.21.15.47:0/1219867267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670cf870 0x559f670d93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.437+0000 7fcf0ca00640 1 -- 172.21.15.47:0/1219867267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f66e127d0 con 0x559f670d49e0 2024-03-21T11:43:16.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.437+0000 7fcf07400640 1 --2- 172.21.15.47:0/1219867267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670cf870 0x559f670d93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:16.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.437+0000 7fcf0ca00640 1 --2- 172.21.15.47:0/1219867267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670d49e0 0x559f670e0410 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000a400 tx=0x7fcf0000d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=308630d931c9decc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:16.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.437+0000 7fcef1600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:43:16.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.438+0000 7fcef1600640 1 -- 172.21.15.47:0/1219867267 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf000139a0 con 0x559f670d49e0 2024-03-21T11:43:16.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.438+0000 7fcef1600640 1 -- 172.21.15.47:0/1219867267 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcf00013b40 con 0x559f670d49e0 2024-03-21T11:43:16.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.438+0000 7fcf11636740 1 -- 172.21.15.47:0/1219867267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf00019070 con 0x559f670d49e0 2024-03-21T11:43:16.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.439+0000 7fcef1600640 1 -- 172.21.15.47:0/1219867267 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf00013d40 con 0x559f670d49e0 2024-03-21T11:43:16.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.439+0000 7fcf11636740 10 client.32018 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:16.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.439+0000 7fcf11636740 1 -- 172.21.15.47:0/1219867267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559f66d483f0 con 0x559f670d49e0 2024-03-21T11:43:16.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.439+0000 7fcef1600640 1 -- 172.21.15.47:0/1219867267 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf00019070 con 0x559f670d49e0 2024-03-21T11:43:16.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.439+0000 7fcef1600640 1 -- 172.21.15.47:0/1219867267 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcf0001e3f0 con 0x559f670d49e0 2024-03-21T11:43:16.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.439+0000 7fcf11636740 10 client.32018 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:16.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.439+0000 7fcf11636740 10 client.32018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:16.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.439+0000 7fcf11636740 20 client.32018 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:16.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.439+0000 7fcf11636740 1 --2- 172.21.15.47:0/1219867267 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f670e4960 0x559f670e6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.440+0000 7fcf11636740 4 client.32018 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:16.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.440+0000 7fcf07400640 1 --2- 172.21.15.47:0/1219867267 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f670e4960 0x559f670e6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.440+0000 7fcf11636740 1 -- 172.21.15.47:0/1219867267 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f67041790 con 0x559f670e4960 2024-03-21T11:43:16.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.440+0000 7fcf07400640 1 --2- 172.21.15.47:0/1219867267 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f670e4960 0x559f670e6d50 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd5f55ddc68c9e6 server_cookie=7f1cbef8740a27f6 in_seq=0 out_seq=0 2024-03-21T11:43:16.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.440+0000 7fcef1600640 10 client.32018 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:16.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.444+0000 7fcef1600640 1 -- 172.21.15.47:0/1219867267 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559f67041790 con 0x559f670e4960 2024-03-21T11:43:16.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.444+0000 7fcef1600640 10 client.32018 handle_command_reply: tid=0 2024-03-21T11:43:16.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.444+0000 7fcf11636740 1 client.32018 shutdown 2024-03-21T11:43:16.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.445+0000 7fcf11636740 1 -- 172.21.15.47:0/1219867267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670d49e0 msgr2=0x559f670e0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.445+0000 7fcf11636740 1 --2- 172.21.15.47:0/1219867267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670d49e0 0x559f670e0410 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000a400 tx=0x7fcf0000d040 comp rx=0 tx=0).stop 2024-03-21T11:43:16.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.445+0000 7fcf11636740 1 -- 172.21.15.47:0/1219867267 shutdown_connections 2024-03-21T11:43:16.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.445+0000 7fcf11636740 1 --2- 172.21.15.47:0/1219867267 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f670e4960 0x559f670e6d50 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.445+0000 7fcf11636740 1 --2- 172.21.15.47:0/1219867267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670d49e0 0x559f670e0410 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.445+0000 7fcf11636740 1 --2- 172.21.15.47:0/1219867267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670cf870 0x559f670d93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.445+0000 7fcf11636740 1 --2- 172.21.15.47:0/1219867267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 0x559f670d8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.445+0000 7fcf11636740 1 -- 172.21.15.47:0/1219867267 >> 172.21.15.47:0/1219867267 conn(0x559f67097200 msgr2=0x559f670c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:16.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.445+0000 7fcef1600640 0 client.32018 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:16.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.445+0000 7fcf11636740 1 -- 172.21.15.47:0/1219867267 shutdown_connections 2024-03-21T11:43:16.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.445+0000 7fcf11636740 1 -- 172.21.15.47:0/1219867267 wait complete. 2024-03-21T11:43:16.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.445+0000 7fcf11636740 20 client.32018 trim_cache size 0 max 16384 2024-03-21T11:43:16.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.474+0000 7fcf11636740 1 Processor -- start 2024-03-21T11:43:16.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.476+0000 7fcf11636740 1 -- start start 2024-03-21T11:43:16.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.477+0000 7fcf11636740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 0x559f670d3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.477+0000 7fcf11636740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cf870 0x559f670e03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.477+0000 7fcf11636740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 0x559f670cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.478+0000 7fcf0ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 0x559f670d3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.478+0000 7fcf0ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 0x559f670d3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51000/0 (socket says 172.21.15.47:51000) 2024-03-21T11:43:16.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.478+0000 7fcf07400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cf870 0x559f670e03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.478+0000 7fcf07400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cf870 0x559f670e03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58336/0 (socket says 172.21.15.47:58336) 2024-03-21T11:43:16.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.478+0000 7fcf11636740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf081ac430 con 0x559f670d49e0 2024-03-21T11:43:16.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.478+0000 7fcf0ca00640 1 -- 172.21.15.47:0/10847113 learned_addr learned my addr 172.21.15.47:0/10847113 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:16.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.478+0000 7fcf11636740 1 -- 172.21.15.47:0/10847113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf081addb0 con 0x559f670cf870 2024-03-21T11:43:16.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.478+0000 7fcf11636740 1 -- 172.21.15.47:0/10847113 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf081aca90 con 0x559f670cee80 2024-03-21T11:43:16.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.479+0000 7fcf0ca00640 1 --2- 172.21.15.47:0/10847113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 0x559f670cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.479+0000 7fcf07400640 1 -- 172.21.15.47:0/10847113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 msgr2=0x559f670d3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.480+0000 7fcf07400640 1 --2- 172.21.15.47:0/10847113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 0x559f670d3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.480+0000 7fcf07400640 1 -- 172.21.15.47:0/10847113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 msgr2=0x559f670cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.480+0000 7fcf07400640 1 --2- 172.21.15.47:0/10847113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 0x559f670cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.480+0000 7fcf07400640 1 -- 172.21.15.47:0/10847113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f66d080f0 con 0x559f670cf870 2024-03-21T11:43:16.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.480+0000 7fcf0ca00640 1 --2- 172.21.15.47:0/10847113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 0x559f670cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:43:16.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.480+0000 7fcf07400640 1 --2- 172.21.15.47:0/10847113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cf870 0x559f670e03a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x559f670e74a0 tx=0x7fcef4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3deb595f5500ba5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:16.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.481+0000 7fcef3e00640 1 -- 172.21.15.47:0/10847113 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef401f2f0 con 0x559f670cf870 2024-03-21T11:43:16.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.481+0000 7fcef3e00640 1 -- 172.21.15.47:0/10847113 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcef4005500 con 0x559f670cf870 2024-03-21T11:43:16.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.481+0000 7fcef3e00640 1 -- 172.21.15.47:0/10847113 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef40035f0 con 0x559f670cf870 2024-03-21T11:43:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.482+0000 7fcf11636740 1 -- 172.21.15.47:0/10847113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cf870 msgr2=0x559f670e03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.482+0000 7fcf11636740 1 --2- 172.21.15.47:0/10847113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cf870 0x559f670e03a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x559f670e74a0 tx=0x7fcef4009260 comp rx=0 tx=0).stop 2024-03-21T11:43:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.482+0000 7fcf11636740 1 -- 172.21.15.47:0/10847113 shutdown_connections 2024-03-21T11:43:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.482+0000 7fcf11636740 1 --2- 172.21.15.47:0/10847113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 0x559f670cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.482+0000 7fcf11636740 1 --2- 172.21.15.47:0/10847113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cf870 0x559f670e03a0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.482+0000 7fcf11636740 1 --2- 172.21.15.47:0/10847113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 0x559f670d3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.482+0000 7fcf11636740 1 -- 172.21.15.47:0/10847113 >> 172.21.15.47:0/10847113 conn(0x559f67097200 msgr2=0x559f670d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.483+0000 7fcf11636740 1 -- 172.21.15.47:0/10847113 shutdown_connections 2024-03-21T11:43:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.483+0000 7fcf11636740 1 -- 172.21.15.47:0/10847113 wait complete. 2024-03-21T11:43:16.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.483+0000 7fcf11636740 1 -- start start 2024-03-21T11:43:16.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.484+0000 7fcf11636740 1 Processor -- start 2024-03-21T11:43:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.485+0000 7fcf11636740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 0x559f670dfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.485+0000 7fcf11636740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cf870 0x559f670e00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.485+0000 7fcf11636740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 0x559f670de1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.485+0000 7fcf11636740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf081ad750 con 0x559f670d49e0 2024-03-21T11:43:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.485+0000 7fcf11636740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf081ad0f0 con 0x559f670cf870 2024-03-21T11:43:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.485+0000 7fcf11636740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf081aa5f0 con 0x559f670cee80 2024-03-21T11:43:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.485+0000 7fcf11636740 10 client.0 _ll_register_callbacks cb 0x7fcef80f4870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.485+0000 7fcf07400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cf870 0x559f670e00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.485+0000 7fcf0ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 0x559f670dfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.485+0000 7fcf07400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cf870 0x559f670e00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58348/0 (socket says 172.21.15.47:58348) 2024-03-21T11:43:16.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.485+0000 7fcf07400640 1 -- 172.21.15.47:0/2923526592 learned_addr learned my addr 172.21.15.47:0/2923526592 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:16.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.486+0000 7fcf0ca00640 1 -- 172.21.15.47:0/2923526592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cf870 msgr2=0x559f670e00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.486+0000 7fcf0ca00640 1 --2- 172.21.15.47:0/2923526592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cf870 0x559f670e00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.486+0000 7fcf0ca00640 1 -- 172.21.15.47:0/2923526592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 msgr2=0x559f670de1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.486+0000 7fcf0ca00640 1 --2- 172.21.15.47:0/2923526592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 0x559f670de1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.486+0000 7fcf0ca00640 1 -- 172.21.15.47:0/2923526592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f670de800 con 0x559f670cee80 2024-03-21T11:43:16.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.489+0000 7fcf0ca00640 1 --2- 172.21.15.47:0/2923526592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 0x559f670dfb60 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcf0001e890 tx=0x7fcf00009990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1fb288538596ef8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:16.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.489+0000 7fcef1600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.489+0000 7fcef1600640 1 -- 172.21.15.47:0/2923526592 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00039070 con 0x559f670cee80 2024-03-21T11:43:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.490+0000 7fcef1600640 1 -- 172.21.15.47:0/2923526592 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcf0001ea00 con 0x559f670cee80 2024-03-21T11:43:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.490+0000 7fcf11636740 1 -- 172.21.15.47:0/2923526592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcef4005500 con 0x559f670cee80 2024-03-21T11:43:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.490+0000 7fcef1600640 1 -- 172.21.15.47:0/2923526592 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf00043050 con 0x559f670cee80 2024-03-21T11:43:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.490+0000 7fcf11636740 10 client.31969 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.490+0000 7fcf11636740 1 -- 172.21.15.47:0/2923526592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559f6715a190 con 0x559f670cee80 2024-03-21T11:43:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.491+0000 7fcef1600640 1 -- 172.21.15.47:0/2923526592 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcef4005500 con 0x559f670cee80 2024-03-21T11:43:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.491+0000 7fcef1600640 1 -- 172.21.15.47:0/2923526592 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcf0003e5e0 con 0x559f670cee80 2024-03-21T11:43:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.491+0000 7fcf11636740 10 client.31969 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.491+0000 7fcf11636740 10 client.31969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.491+0000 7fcf11636740 20 client.31969 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.491+0000 7fcf11636740 1 --2- 172.21.15.47:0/2923526592 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f6715ab20 0x559f6715cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.491+0000 7fcf11636740 4 client.31969 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:43:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.491+0000 7fcf11636740 1 -- 172.21.15.47:0/2923526592 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559f6703bae0 con 0x559f6715ab20 2024-03-21T11:43:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.492+0000 7fcf07400640 1 --2- 172.21.15.47:0/2923526592 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f6715ab20 0x559f6715cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:16.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.493+0000 7fcf07400640 1 --2- 172.21.15.47:0/2923526592 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f6715ab20 0x559f6715cf40 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5af1375fb93b940 server_cookie=55277cbc523fe1ef in_seq=0 out_seq=0 2024-03-21T11:43:16.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.493+0000 7fcef1600640 10 client.31969 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:16.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.498+0000 7fcef1600640 1 -- 172.21.15.47:0/2923526592 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x559f6703bae0 con 0x559f6715ab20 2024-03-21T11:43:16.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.498+0000 7fcef1600640 10 client.31969 handle_command_reply: tid=0 2024-03-21T11:43:16.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.498+0000 7fcf11636740 1 client.31969 shutdown 2024-03-21T11:43:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.498+0000 7fcf11636740 1 -- 172.21.15.47:0/2923526592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 msgr2=0x559f670dfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.498+0000 7fcf11636740 1 --2- 172.21.15.47:0/2923526592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 0x559f670dfb60 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcf0001e890 tx=0x7fcf00009990 comp rx=0 tx=0).stop 2024-03-21T11:43:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.498+0000 7fcf11636740 1 -- 172.21.15.47:0/2923526592 shutdown_connections 2024-03-21T11:43:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.498+0000 7fcf11636740 1 --2- 172.21.15.47:0/2923526592 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559f6715ab20 0x559f6715cf40 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.498+0000 7fcf11636740 1 --2- 172.21.15.47:0/2923526592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f670d49e0 0x559f670de1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.498+0000 7fcf11636740 1 --2- 172.21.15.47:0/2923526592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f670cf870 0x559f670e00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.498+0000 7fcf11636740 1 --2- 172.21.15.47:0/2923526592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f670cee80 0x559f670dfb60 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.498+0000 7fcf11636740 1 -- 172.21.15.47:0/2923526592 >> 172.21.15.47:0/2923526592 conn(0x559f67097200 msgr2=0x559f670d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.498+0000 7fcef1600640 0 client.31969 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.499+0000 7fcf11636740 1 -- 172.21.15.47:0/2923526592 shutdown_connections 2024-03-21T11:43:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.499+0000 7fcf11636740 1 -- 172.21.15.47:0/2923526592 wait complete. 2024-03-21T11:43:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.499+0000 7fcf11636740 20 client.31969 trim_cache size 0 max 16384 2024-03-21T11:43:16.501 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:43:16.501 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (138 inodes in the stack)", 2024-03-21T11:43:16.501 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:43:16.501 INFO:teuthology.orchestra.run.smithi047.stdout: "f783bdff-a122-410a-a030-75330a72a3d8": { 2024-03-21T11:43:16.501 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:43:16.501 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "f783bdff-a122-410a-a030-75330a72a3d8", 2024-03-21T11:43:16.501 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:43:16.501 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:43:16.501 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:43:16.502 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:43:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.500+0000 7fcf0ea00640 1 -- 172.21.15.47:0/1052626562 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fceec0a3020 msgr2=0x7fceec0a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.500+0000 7fcf0ea00640 1 --2- 172.21.15.47:0/1052626562 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fceec0a3020 0x7fceec0a54e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009830 tx=0x7fcef80f49a0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.500+0000 7fcf0ea00640 1 -- 172.21.15.47:0/1052626562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf081016b0 msgr2=0x7fcf08197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.500+0000 7fcf0ea00640 1 --2- 172.21.15.47:0/1052626562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf081016b0 0x7fcf08197a80 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fcef80349e0 tx=0x7fcef80042f0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.500+0000 7fcf0ea00640 1 -- 172.21.15.47:0/1052626562 shutdown_connections 2024-03-21T11:43:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.500+0000 7fcf0ea00640 1 --2- 172.21.15.47:0/1052626562 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fceec0a3020 0x7fceec0a54e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.500+0000 7fcf0ea00640 1 --2- 172.21.15.47:0/1052626562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf08103260 0x7fcf0819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.500+0000 7fcf0ea00640 1 --2- 172.21.15.47:0/1052626562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf081028a0 0x7fcf08197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.500+0000 7fcf0ea00640 1 --2- 172.21.15.47:0/1052626562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf081016b0 0x7fcf08197a80 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.500+0000 7fcf0ea00640 1 -- 172.21.15.47:0/1052626562 >> 172.21.15.47:0/1052626562 conn(0x7fcf080fcc40 msgr2=0x7fcf08105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.500+0000 7fcf0ea00640 1 -- 172.21.15.47:0/1052626562 shutdown_connections 2024-03-21T11:43:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:16.500+0000 7fcf0ea00640 1 -- 172.21.15.47:0/1052626562 wait complete. 2024-03-21T11:43:16.523 INFO:tasks.cephfs.filesystem:scrub status for tag:f783bdff-a122-410a-a030-75330a72a3d8 - {'path': '/', 'tag': 'f783bdff-a122-410a-a030-75330a72a3d8', 'options': 'recursive,force'} 2024-03-21T11:43:16.963 INFO:tasks.workunit.client.0.smithi047.stdout: 10 161113 23.04 MB/sec execute 240 sec latency 415.501 ms 2024-03-21T11:43:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: pgmap v998: 129 pgs: 129 active+clean; 554 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.0 MiB/s wr, 567 op/s 2024-03-21T11:43:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:43:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:17.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:16 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: pgmap v998: 129 pgs: 129 active+clean; 554 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.0 MiB/s wr, 567 op/s 2024-03-21T11:43:17.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:43:17.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:17.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:17.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:17.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:17.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:17.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:17.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:17.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:17.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:17.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:16 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: pgmap v998: 129 pgs: 129 active+clean; 554 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.0 MiB/s wr, 567 op/s 2024-03-21T11:43:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:43:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:17.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:17.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:17.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:17.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:17.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:16 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:17.963 INFO:tasks.workunit.client.0.smithi047.stdout: 10 161394 23.02 MB/sec execute 241 sec latency 511.697 ms 2024-03-21T11:43:18.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:17 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:18.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:17 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:43:18.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:17 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:18.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:17 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:43:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:17 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:18.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:17 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:43:18.963 INFO:tasks.workunit.client.0.smithi047.stdout: 10 161958 23.05 MB/sec execute 242 sec latency 93.597 ms 2024-03-21T11:43:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:18 smithi100 ceph-mon[36245]: pgmap v999: 129 pgs: 129 active+clean; 558 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.8 MiB/s wr, 676 op/s 2024-03-21T11:43:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:18 smithi102 ceph-mon[36748]: pgmap v999: 129 pgs: 129 active+clean; 558 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.8 MiB/s wr, 676 op/s 2024-03-21T11:43:19.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:18 smithi047 ceph-mon[31351]: pgmap v999: 129 pgs: 129 active+clean; 558 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.8 MiB/s wr, 676 op/s 2024-03-21T11:43:19.963 INFO:tasks.workunit.client.0.smithi047.stdout: 10 162509 23.11 MB/sec execute 243 sec latency 161.234 ms 2024-03-21T11:43:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.446+0000 7f2857000640 1 -- 172.21.15.47:0/686424720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2850101700 msgr2=0x7f2850101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.446+0000 7f2857000640 1 --2- 172.21.15.47:0/686424720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2850101700 0x7f2850101b20 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f2840009900 tx=0x7f284002bc30 comp rx=0 tx=0).stop 2024-03-21T11:43:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.446+0000 7f2857000640 1 -- 172.21.15.47:0/686424720 shutdown_connections 2024-03-21T11:43:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.446+0000 7f2857000640 1 --2- 172.21.15.47:0/686424720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28501032b0 0x7f2850109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.446+0000 7f2857000640 1 --2- 172.21.15.47:0/686424720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28501028f0 0x7f2850102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.446+0000 7f2857000640 1 --2- 172.21.15.47:0/686424720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2850101700 0x7f2850101b20 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.446+0000 7f2857000640 1 -- 172.21.15.47:0/686424720 >> 172.21.15.47:0/686424720 conn(0x7f28500fcc70 msgr2=0x7f28500ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.446+0000 7f2857000640 1 -- 172.21.15.47:0/686424720 shutdown_connections 2024-03-21T11:43:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.446+0000 7f2857000640 1 -- 172.21.15.47:0/686424720 wait complete. 2024-03-21T11:43:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.447+0000 7f2857000640 1 Processor -- start 2024-03-21T11:43:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.447+0000 7f2857000640 1 -- start start 2024-03-21T11:43:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.448+0000 7f2857000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2850101700 0x7f285019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.448+0000 7f2857000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28501028f0 0x7f285019e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.448+0000 7f2857000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28501032b0 0x7f28501a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.448+0000 7f2857000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285010d350 con 0x7f28501028f0 2024-03-21T11:43:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.448+0000 7f2857000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285010d1d0 con 0x7f2850101700 2024-03-21T11:43:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.448+0000 7f2857000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285010d4d0 con 0x7f28501032b0 2024-03-21T11:43:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.448+0000 7f284f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28501028f0 0x7f285019e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.448+0000 7f284f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28501028f0 0x7f285019e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45138/0 (socket says 172.21.15.47:45138) 2024-03-21T11:43:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.448+0000 7f284f400640 1 -- 172.21.15.47:0/595525172 learned_addr learned my addr 172.21.15.47:0/595525172 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.448+0000 7f284fe00640 1 --2- 172.21.15.47:0/595525172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2850101700 0x7f285019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.448+0000 7f2855000640 1 --2- 172.21.15.47:0/595525172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28501032b0 0x7f28501a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.449+0000 7f284f400640 1 -- 172.21.15.47:0/595525172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28501032b0 msgr2=0x7f28501a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.449+0000 7f284f400640 1 --2- 172.21.15.47:0/595525172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28501032b0 0x7f28501a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.449+0000 7f284f400640 1 -- 172.21.15.47:0/595525172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2850101700 msgr2=0x7f285019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.449+0000 7f284f400640 1 --2- 172.21.15.47:0/595525172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2850101700 0x7f285019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.449+0000 7f284f400640 1 -- 172.21.15.47:0/595525172 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28501a5bc0 con 0x7f28501028f0 2024-03-21T11:43:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.449+0000 7f284f400640 1 --2- 172.21.15.47:0/595525172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28501028f0 0x7f285019e4e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f283c00d950 tx=0x7f283c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.450+0000 7f284cc00640 1 -- 172.21.15.47:0/595525172 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c017030 con 0x7f28501028f0 2024-03-21T11:43:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.450+0000 7f284cc00640 1 -- 172.21.15.47:0/595525172 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f283c004020 con 0x7f28501028f0 2024-03-21T11:43:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.450+0000 7f284cc00640 1 -- 172.21.15.47:0/595525172 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c005650 con 0x7f28501028f0 2024-03-21T11:43:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.450+0000 7f2857000640 1 -- 172.21.15.47:0/595525172 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28501a5eb0 con 0x7f28501028f0 2024-03-21T11:43:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.450+0000 7f2857000640 1 -- 172.21.15.47:0/595525172 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28501a63c0 con 0x7f28501028f0 2024-03-21T11:43:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.453+0000 7f2859d8c740 1 Processor -- start 2024-03-21T11:43:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.454+0000 7f2859d8c740 1 -- start start 2024-03-21T11:43:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.454+0000 7f2859d8c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c21e0 0x5616ef5c2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.454+0000 7f2859d8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.454+0000 7f2859d8c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 0x5616ef5ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.454+0000 7f2859d8c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616ef5853b0 con 0x5616ef5c21e0 2024-03-21T11:43:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.454+0000 7f2859d8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616ef527830 con 0x5616ef5c2bd0 2024-03-21T11:43:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.454+0000 7f2859d8c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616ef4d05d0 con 0x5616ef5c7d40 2024-03-21T11:43:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.454+0000 7f284fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.454+0000 7f284fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48608/0 (socket says 172.21.15.47:48608) 2024-03-21T11:43:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.454+0000 7f284fe00640 1 -- 172.21.15.47:0/2065828976 learned_addr learned my addr 172.21.15.47:0/2065828976 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.454+0000 7f2855000640 1 --2- 172.21.15.47:0/2065828976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c21e0 0x5616ef5c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.455+0000 7f284fe00640 1 -- 172.21.15.47:0/2065828976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 msgr2=0x5616ef5ca1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:43:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.455+0000 7f284cc00640 1 -- 172.21.15.47:0/595525172 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f283c0041d0 con 0x7f28501028f0 2024-03-21T11:43:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.455+0000 7f284fe00640 1 --2- 172.21.15.47:0/2065828976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 0x5616ef5ca1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.455+0000 7f284fe00640 1 -- 172.21.15.47:0/2065828976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c21e0 msgr2=0x5616ef5c2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.455+0000 7f284fe00640 1 --2- 172.21.15.47:0/2065828976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c21e0 0x5616ef5c2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.455+0000 7f284fe00640 1 -- 172.21.15.47:0/2065828976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616ef4e82c0 con 0x5616ef5c2bd0 2024-03-21T11:43:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.455+0000 7f284fe00640 1 --2- 172.21.15.47:0/2065828976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5c76d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f2840009900 tx=0x7f28400095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7983f713a7cd8f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.455+0000 7f2845600640 1 -- 172.21.15.47:0/2065828976 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284002cb80 con 0x5616ef5c2bd0 2024-03-21T11:43:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.455+0000 7f2845600640 1 -- 172.21.15.47:0/2065828976 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2840005040 con 0x5616ef5c2bd0 2024-03-21T11:43:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.455+0000 7f2845600640 1 -- 172.21.15.47:0/2065828976 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840004900 con 0x5616ef5c2bd0 2024-03-21T11:43:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.456+0000 7f2859d8c740 1 -- 172.21.15.47:0/2065828976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 msgr2=0x5616ef5c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.456+0000 7f2859d8c740 1 --2- 172.21.15.47:0/2065828976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5c76d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f2840009900 tx=0x7f28400095e0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.456+0000 7f284cc00640 1 --2- 172.21.15.47:0/595525172 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28340a3040 0x7f28340a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.456+0000 7f284cc00640 1 -- 172.21.15.47:0/595525172 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f283c0cf170 con 0x7f28501028f0 2024-03-21T11:43:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.457+0000 7f2859d8c740 1 -- 172.21.15.47:0/2065828976 shutdown_connections 2024-03-21T11:43:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.457+0000 7f2859d8c740 1 --2- 172.21.15.47:0/2065828976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 0x5616ef5ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.457+0000 7f2859d8c740 1 --2- 172.21.15.47:0/2065828976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5c76d0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.457+0000 7f2859d8c740 1 --2- 172.21.15.47:0/2065828976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c21e0 0x5616ef5c2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.457+0000 7f2859d8c740 1 -- 172.21.15.47:0/2065828976 >> 172.21.15.47:0/2065828976 conn(0x5616ef5bd750 msgr2=0x5616ef5bdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.457+0000 7f2855000640 1 --2- 172.21.15.47:0/595525172 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28340a3040 0x7f28340a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.457+0000 7f2859d8c740 1 -- 172.21.15.47:0/2065828976 shutdown_connections 2024-03-21T11:43:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.457+0000 7f2859d8c740 1 -- 172.21.15.47:0/2065828976 wait complete. 2024-03-21T11:43:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.457+0000 7f2855000640 1 --2- 172.21.15.47:0/595525172 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28340a3040 0x7f28340a5500 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x5616ef5c3430 tx=0x7f2848009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.457+0000 7f2859d8c740 1 -- start start 2024-03-21T11:43:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.458+0000 7f2859d8c740 1 Processor -- start 2024-03-21T11:43:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.458+0000 7f2859d8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c21e0 0x5616ef5cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.458+0000 7f2859d8c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c2bd0 0x5616ef5cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.458+0000 7f2859d8c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 0x5616ef5d3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.458+0000 7f2859d8c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616ef4e3a20 con 0x5616ef5c2bd0 2024-03-21T11:43:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.458+0000 7f2859d8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616ef5238d0 con 0x5616ef5c21e0 2024-03-21T11:43:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.458+0000 7f2859d8c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616ef5cac90 con 0x5616ef5c7d40 2024-03-21T11:43:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.458+0000 7f2859d8c740 10 client.0 _ll_register_callbacks cb 0x5616ef5201c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.459+0000 7f284f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c2bd0 0x5616ef5cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.459+0000 7f284fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c21e0 0x5616ef5cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.459+0000 7f2855000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 0x5616ef5d3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.459+0000 7f284f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c2bd0 0x5616ef5cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45168/0 (socket says 172.21.15.47:45168) 2024-03-21T11:43:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.459+0000 7f284fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c21e0 0x5616ef5cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48618/0 (socket says 172.21.15.47:48618) 2024-03-21T11:43:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.459+0000 7f284f400640 1 -- 172.21.15.47:0/1071819659 learned_addr learned my addr 172.21.15.47:0/1071819659 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.459+0000 7f2855000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 0x5616ef5d3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33122/0 (socket says 172.21.15.47:33122) 2024-03-21T11:43:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.459+0000 7f284fe00640 1 -- 172.21.15.47:0/1071819659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 msgr2=0x5616ef5d3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.459+0000 7f284fe00640 1 --2- 172.21.15.47:0/1071819659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 0x5616ef5d3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.459+0000 7f284fe00640 1 -- 172.21.15.47:0/1071819659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c2bd0 msgr2=0x5616ef5cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.459+0000 7f284fe00640 1 --2- 172.21.15.47:0/1071819659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c2bd0 0x5616ef5cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.459+0000 7f284fe00640 1 -- 172.21.15.47:0/1071819659 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616ef23c210 con 0x5616ef5c21e0 2024-03-21T11:43:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.459+0000 7f284fe00640 1 --2- 172.21.15.47:0/1071819659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c21e0 0x5616ef5cc400 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x5616ef5be6c0 tx=0x7f284002ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f85713f85b30a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.459+0000 7f283a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:43:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.460+0000 7f283a000640 1 -- 172.21.15.47:0/1071819659 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28400046c0 con 0x5616ef5c21e0 2024-03-21T11:43:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.460+0000 7f2859d8c740 1 -- 172.21.15.47:0/1071819659 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2840005040 con 0x5616ef5c21e0 2024-03-21T11:43:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.460+0000 7f283a000640 1 -- 172.21.15.47:0/1071819659 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2840004860 con 0x5616ef5c21e0 2024-03-21T11:43:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.460+0000 7f283a000640 1 -- 172.21.15.47:0/1071819659 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2840004a60 con 0x5616ef5c21e0 2024-03-21T11:43:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.460+0000 7f2859d8c740 10 client.32036 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.460+0000 7f2859d8c740 1 -- 172.21.15.47:0/1071819659 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5616ef308620 con 0x5616ef5c21e0 2024-03-21T11:43:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.460+0000 7f283a000640 1 -- 172.21.15.47:0/1071819659 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2840005040 con 0x5616ef5c21e0 2024-03-21T11:43:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.460+0000 7f283a000640 1 -- 172.21.15.47:0/1071819659 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2840034440 con 0x5616ef5c21e0 2024-03-21T11:43:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.461+0000 7f2859d8c740 10 client.32036 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.461+0000 7f2859d8c740 10 client.32036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.461+0000 7f2859d8c740 20 client.32036 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.461+0000 7f2859d8c740 1 --2- 172.21.15.47:0/1071819659 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5616ef5d7e70 0x5616ef5da260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.461+0000 7f2859d8c740 4 client.32036 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.461+0000 7f2859d8c740 1 -- 172.21.15.47:0/1071819659 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616ef534d70 con 0x5616ef5d7e70 2024-03-21T11:43:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.461+0000 7f284f400640 1 --2- 172.21.15.47:0/1071819659 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5616ef5d7e70 0x5616ef5da260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.461+0000 7f284f400640 1 --2- 172.21.15.47:0/1071819659 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5616ef5d7e70 0x5616ef5da260 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66346302d9d0974d server_cookie=52ce0a78deec4d83 in_seq=0 out_seq=0 2024-03-21T11:43:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.461+0000 7f283a000640 10 client.32036 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.463+0000 7f283a000640 1 -- 172.21.15.47:0/1071819659 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5616ef534d70 con 0x5616ef5d7e70 2024-03-21T11:43:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.463+0000 7f283a000640 10 client.32036 handle_command_reply: tid=0 2024-03-21T11:43:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.463+0000 7f2859d8c740 1 client.32036 shutdown 2024-03-21T11:43:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.464+0000 7f2859d8c740 1 -- 172.21.15.47:0/1071819659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c21e0 msgr2=0x5616ef5cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.464+0000 7f2859d8c740 1 --2- 172.21.15.47:0/1071819659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c21e0 0x5616ef5cc400 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x5616ef5be6c0 tx=0x7f284002ccb0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.464+0000 7f2859d8c740 1 -- 172.21.15.47:0/1071819659 shutdown_connections 2024-03-21T11:43:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.464+0000 7f2859d8c740 1 --2- 172.21.15.47:0/1071819659 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5616ef5d7e70 0x5616ef5da260 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.464+0000 7f2859d8c740 1 --2- 172.21.15.47:0/1071819659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 0x5616ef5d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.464+0000 7f2859d8c740 1 --2- 172.21.15.47:0/1071819659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c2bd0 0x5616ef5cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.464+0000 7f2859d8c740 1 --2- 172.21.15.47:0/1071819659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c21e0 0x5616ef5cc400 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.464+0000 7f283a000640 0 client.32036 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.464+0000 7f2859d8c740 1 -- 172.21.15.47:0/1071819659 >> 172.21.15.47:0/1071819659 conn(0x5616ef5bd750 msgr2=0x5616ef5bc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.464+0000 7f2859d8c740 1 -- 172.21.15.47:0/1071819659 shutdown_connections 2024-03-21T11:43:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.464+0000 7f2859d8c740 1 -- 172.21.15.47:0/1071819659 wait complete. 2024-03-21T11:43:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.464+0000 7f2859d8c740 20 client.32036 trim_cache size 0 max 16384 2024-03-21T11:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.484+0000 7f2859d8c740 1 Processor -- start 2024-03-21T11:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.484+0000 7f2859d8c740 1 -- start start 2024-03-21T11:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.484+0000 7f2859d8c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c21e0 0x5616ef5de880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.484+0000 7f2859d8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5d3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.484+0000 7f2859d8c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c7d40 0x5616ef5dedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.484+0000 7f2859d8c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28501b0700 con 0x5616ef5c7d40 2024-03-21T11:43:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.484+0000 7f2859d8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28501b2080 con 0x5616ef5c2bd0 2024-03-21T11:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.484+0000 7f2859d8c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28501b0d60 con 0x5616ef5c21e0 2024-03-21T11:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.484+0000 7f2855000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c7d40 0x5616ef5dedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.484+0000 7f284f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5d3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.484+0000 7f284f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5d3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48626/0 (socket says 172.21.15.47:48626) 2024-03-21T11:43:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.484+0000 7f284f400640 1 -- 172.21.15.47:0/515468485 learned_addr learned my addr 172.21.15.47:0/515468485 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.485+0000 7f284f400640 1 -- 172.21.15.47:0/515468485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c21e0 msgr2=0x5616ef5de880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.485+0000 7f284f400640 1 --2- 172.21.15.47:0/515468485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c21e0 0x5616ef5de880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.485+0000 7f284f400640 1 -- 172.21.15.47:0/515468485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c7d40 msgr2=0x5616ef5dedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.485+0000 7f284f400640 1 --2- 172.21.15.47:0/515468485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c7d40 0x5616ef5dedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.485+0000 7f284f400640 1 -- 172.21.15.47:0/515468485 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616ef5cd1c0 con 0x5616ef5c2bd0 2024-03-21T11:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.485+0000 7f284f400640 1 --2- 172.21.15.47:0/515468485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5d3940 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f283c00df30 tx=0x7f283c00df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8650802d04de6953 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.485+0000 7f2844c00640 1 -- 172.21.15.47:0/515468485 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0b4070 con 0x5616ef5c2bd0 2024-03-21T11:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.486+0000 7f2844c00640 1 -- 172.21.15.47:0/515468485 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f283c014040 con 0x5616ef5c2bd0 2024-03-21T11:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.486+0000 7f2844c00640 1 -- 172.21.15.47:0/515468485 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c09f120 con 0x5616ef5c2bd0 2024-03-21T11:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.486+0000 7f2859d8c740 1 -- 172.21.15.47:0/515468485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 msgr2=0x5616ef5d3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.486+0000 7f2859d8c740 1 --2- 172.21.15.47:0/515468485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5d3940 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f283c00df30 tx=0x7f283c00df90 comp rx=0 tx=0).stop 2024-03-21T11:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.487+0000 7f2859d8c740 1 -- 172.21.15.47:0/515468485 shutdown_connections 2024-03-21T11:43:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.487+0000 7f2859d8c740 1 --2- 172.21.15.47:0/515468485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c7d40 0x5616ef5dedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.487+0000 7f2859d8c740 1 --2- 172.21.15.47:0/515468485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5d3940 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.487+0000 7f2859d8c740 1 --2- 172.21.15.47:0/515468485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c21e0 0x5616ef5de880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.487+0000 7f2859d8c740 1 -- 172.21.15.47:0/515468485 >> 172.21.15.47:0/515468485 conn(0x5616ef5bd750 msgr2=0x5616ef5c1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.487+0000 7f2859d8c740 1 -- 172.21.15.47:0/515468485 shutdown_connections 2024-03-21T11:43:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.487+0000 7f2859d8c740 1 -- 172.21.15.47:0/515468485 wait complete. 2024-03-21T11:43:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.487+0000 7f2859d8c740 1 -- start start 2024-03-21T11:43:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.487+0000 7f2859d8c740 1 Processor -- start 2024-03-21T11:43:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.488+0000 7f2859d8c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c21e0 0x5616ef5d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.488+0000 7f2859d8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.488+0000 7f2859d8c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 0x5616ef5d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.488+0000 7f2859d8c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28501b1a20 con 0x5616ef5c21e0 2024-03-21T11:43:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.488+0000 7f2859d8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28501b13c0 con 0x5616ef5c2bd0 2024-03-21T11:43:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.488+0000 7f2859d8c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28501ae8c0 con 0x5616ef5c7d40 2024-03-21T11:43:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.488+0000 7f2859d8c740 10 client.0 _ll_register_callbacks cb 0x7f2848009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.488+0000 7f284fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c21e0 0x5616ef5d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.488+0000 7f2855000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 0x5616ef5d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.488+0000 7f284fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c21e0 0x5616ef5d1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45194/0 (socket says 172.21.15.47:45194) 2024-03-21T11:43:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.488+0000 7f2855000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 0x5616ef5d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33142/0 (socket says 172.21.15.47:33142) 2024-03-21T11:43:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.488+0000 7f284fe00640 1 -- 172.21.15.47:0/239092556 learned_addr learned my addr 172.21.15.47:0/239092556 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.488+0000 7f284f400640 1 --2- 172.21.15.47:0/239092556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.489+0000 7f2855000640 1 -- 172.21.15.47:0/239092556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 msgr2=0x5616ef5d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.489+0000 7f2855000640 1 --2- 172.21.15.47:0/239092556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5d1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.489+0000 7f2855000640 1 -- 172.21.15.47:0/239092556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c21e0 msgr2=0x5616ef5d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.489+0000 7f2855000640 1 --2- 172.21.15.47:0/239092556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c21e0 0x5616ef5d1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.489+0000 7f2855000640 1 -- 172.21.15.47:0/239092556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616ef5d3040 con 0x5616ef5c7d40 2024-03-21T11:43:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.489+0000 7f284fe00640 1 --2- 172.21.15.47:0/239092556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c21e0 0x5616ef5d1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.489+0000 7f2855000640 1 --2- 172.21.15.47:0/239092556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 0x5616ef5d2a50 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f2848009e80 tx=0x7f2848004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7faede7c6467d0b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.489+0000 7f283a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.489+0000 7f283a000640 1 -- 172.21.15.47:0/239092556 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848030020 con 0x5616ef5c7d40 2024-03-21T11:43:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.489+0000 7f2859d8c740 1 -- 172.21.15.47:0/239092556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f283c014040 con 0x5616ef5c7d40 2024-03-21T11:43:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.489+0000 7f283a000640 1 -- 172.21.15.47:0/239092556 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f284800fa20 con 0x5616ef5c7d40 2024-03-21T11:43:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.490+0000 7f283a000640 1 -- 172.21.15.47:0/239092556 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f284803b050 con 0x5616ef5c7d40 2024-03-21T11:43:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.490+0000 7f2859d8c740 10 client.31981 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.490+0000 7f2859d8c740 1 -- 172.21.15.47:0/239092556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5616ef633a40 con 0x5616ef5c7d40 2024-03-21T11:43:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.490+0000 7f283a000640 1 -- 172.21.15.47:0/239092556 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f283c014040 con 0x5616ef5c7d40 2024-03-21T11:43:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.490+0000 7f283a000640 1 -- 172.21.15.47:0/239092556 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f284802a410 con 0x5616ef5c7d40 2024-03-21T11:43:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.490+0000 7f2859d8c740 10 client.31981 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.490+0000 7f2859d8c740 10 client.31981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.490+0000 7f2859d8c740 20 client.31981 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.490+0000 7f2859d8c740 1 --2- 172.21.15.47:0/239092556 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5616ef6343d0 0x5616ef6367f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.490+0000 7f2859d8c740 4 client.31981 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:43:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.490+0000 7f2859d8c740 1 -- 172.21.15.47:0/239092556 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5616ef52f0c0 con 0x5616ef6343d0 2024-03-21T11:43:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.491+0000 7f284fe00640 1 --2- 172.21.15.47:0/239092556 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5616ef6343d0 0x5616ef6367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.492+0000 7f284fe00640 1 --2- 172.21.15.47:0/239092556 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5616ef6343d0 0x5616ef6367f0 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bece9a28e09e5bc3 server_cookie=bc2fe07e65ba6e4a in_seq=0 out_seq=0 2024-03-21T11:43:20.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:20.492+0000 7f283a000640 10 client.31981 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:20.963 INFO:tasks.workunit.client.0.smithi047.stdout: 10 162790 23.09 MB/sec execute 244 sec latency 357.261 ms 2024-03-21T11:43:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f283a000640 1 -- 172.21.15.47:0/239092556 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5616ef52f0c0 con 0x5616ef6343d0 2024-03-21T11:43:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f283a000640 10 client.31981 handle_command_reply: tid=0 2024-03-21T11:43:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f2859d8c740 1 client.31981 shutdown 2024-03-21T11:43:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f2859d8c740 1 -- 172.21.15.47:0/239092556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 msgr2=0x5616ef5d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f2859d8c740 1 --2- 172.21.15.47:0/239092556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 0x5616ef5d2a50 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f2848009e80 tx=0x7f2848004a40 comp rx=0 tx=0).stop 2024-03-21T11:43:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f2859d8c740 1 -- 172.21.15.47:0/239092556 shutdown_connections 2024-03-21T11:43:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f2859d8c740 1 --2- 172.21.15.47:0/239092556 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5616ef6343d0 0x5616ef6367f0 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f2859d8c740 1 --2- 172.21.15.47:0/239092556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5616ef5c7d40 0x5616ef5d2a50 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f2859d8c740 1 --2- 172.21.15.47:0/239092556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616ef5c2bd0 0x5616ef5d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f2859d8c740 1 --2- 172.21.15.47:0/239092556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5616ef5c21e0 0x5616ef5d1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f2859d8c740 1 -- 172.21.15.47:0/239092556 >> 172.21.15.47:0/239092556 conn(0x5616ef5bd750 msgr2=0x5616ef5bc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f283a000640 0 client.31981 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f2859d8c740 1 -- 172.21.15.47:0/239092556 shutdown_connections 2024-03-21T11:43:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f2859d8c740 1 -- 172.21.15.47:0/239092556 wait complete. 2024-03-21T11:43:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.266+0000 7f2859d8c740 20 client.31981 trim_cache size 0 max 16384 2024-03-21T11:43:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:43:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:43:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:43:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:43:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.267+0000 7f2857000640 1 -- 172.21.15.47:0/595525172 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28340a3040 msgr2=0x7f28340a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.267+0000 7f2857000640 1 --2- 172.21.15.47:0/595525172 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28340a3040 0x7f28340a5500 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x5616ef5c3430 tx=0x7f2848009500 comp rx=0 tx=0).stop 2024-03-21T11:43:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.267+0000 7f2857000640 1 -- 172.21.15.47:0/595525172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28501028f0 msgr2=0x7f285019e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.267+0000 7f2857000640 1 --2- 172.21.15.47:0/595525172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28501028f0 0x7f285019e4e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f283c00d950 tx=0x7f283c00dd50 comp rx=0 tx=0).stop 2024-03-21T11:43:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.267+0000 7f2857000640 1 -- 172.21.15.47:0/595525172 shutdown_connections 2024-03-21T11:43:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.267+0000 7f2857000640 1 --2- 172.21.15.47:0/595525172 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28340a3040 0x7f28340a5500 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.267+0000 7f2857000640 1 --2- 172.21.15.47:0/595525172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28501032b0 0x7f28501a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.267+0000 7f2857000640 1 --2- 172.21.15.47:0/595525172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28501028f0 0x7f285019e4e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.267+0000 7f2857000640 1 --2- 172.21.15.47:0/595525172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2850101700 0x7f285019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.267+0000 7f2857000640 1 -- 172.21.15.47:0/595525172 >> 172.21.15.47:0/595525172 conn(0x7f28500fcc70 msgr2=0x7f28500738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.268+0000 7f2857000640 1 -- 172.21.15.47:0/595525172 shutdown_connections 2024-03-21T11:43:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:21.268+0000 7f2857000640 1 -- 172.21.15.47:0/595525172 wait complete. 2024-03-21T11:43:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:21 smithi100 ceph-mon[36245]: pgmap v1000: 129 pgs: 129 active+clean; 561 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 546 op/s 2024-03-21T11:43:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:21 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:21 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:21 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:21 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:21 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:43:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:21 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:43:21.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:21 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:21 smithi102 ceph-mon[36748]: pgmap v1000: 129 pgs: 129 active+clean; 561 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 546 op/s 2024-03-21T11:43:21.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:21 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:21 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:21 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:21 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:21 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:43:21.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:21 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:43:21.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:21 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:21 smithi047 ceph-mon[31351]: pgmap v1000: 129 pgs: 129 active+clean; 561 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 546 op/s 2024-03-21T11:43:21.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:21 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:21 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:21 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:21 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:21 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:43:21.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:21 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:43:21.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:21 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:43:21.964 INFO:tasks.workunit.client.0.smithi047.stdout: 10 163049 23.06 MB/sec execute 245 sec latency 525.101 ms 2024-03-21T11:43:22.964 INFO:tasks.workunit.client.0.smithi047.stdout: 10 163195 22.98 MB/sec execute 246 sec latency 403.229 ms 2024-03-21T11:43:23.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:23 smithi100 ceph-mon[36245]: pgmap v1001: 129 pgs: 129 active+clean; 561 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.9 MiB/s wr, 533 op/s 2024-03-21T11:43:23.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:23 smithi102 ceph-mon[36748]: pgmap v1001: 129 pgs: 129 active+clean; 561 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.9 MiB/s wr, 533 op/s 2024-03-21T11:43:23.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:23 smithi047 ceph-mon[31351]: pgmap v1001: 129 pgs: 129 active+clean; 561 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.9 MiB/s wr, 533 op/s 2024-03-21T11:43:23.964 INFO:tasks.workunit.client.0.smithi047.stdout: 10 163558 22.95 MB/sec execute 247 sec latency 153.938 ms 2024-03-21T11:43:24.965 INFO:tasks.workunit.client.0.smithi047.stdout: 10 163957 22.94 MB/sec execute 248 sec latency 217.911 ms 2024-03-21T11:43:25.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:25 smithi100 ceph-mon[36245]: pgmap v1002: 129 pgs: 129 active+clean; 559 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-03-21T11:43:25.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:25 smithi102 ceph-mon[36748]: pgmap v1002: 129 pgs: 129 active+clean; 559 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-03-21T11:43:25.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:25 smithi047 ceph-mon[31351]: pgmap v1002: 129 pgs: 129 active+clean; 559 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-03-21T11:43:25.965 INFO:tasks.workunit.client.0.smithi047.stdout: 10 164586 22.98 MB/sec execute 249 sec latency 102.541 ms 2024-03-21T11:43:26.965 INFO:tasks.workunit.client.0.smithi047.stdout: 10 165053 22.99 MB/sec execute 250 sec latency 182.742 ms 2024-03-21T11:43:27.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:27 smithi100 ceph-mon[36245]: pgmap v1003: 129 pgs: 129 active+clean; 559 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.0 MiB/s wr, 568 op/s 2024-03-21T11:43:27.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:27 smithi102 ceph-mon[36748]: pgmap v1003: 129 pgs: 129 active+clean; 559 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.0 MiB/s wr, 568 op/s 2024-03-21T11:43:27.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:27 smithi047 ceph-mon[31351]: pgmap v1003: 129 pgs: 129 active+clean; 559 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.0 MiB/s wr, 568 op/s 2024-03-21T11:43:27.965 INFO:tasks.workunit.client.0.smithi047.stdout: 10 165387 22.97 MB/sec execute 251 sec latency 327.062 ms 2024-03-21T11:43:28.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:28 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:43:28.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:28 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:43:28.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:28 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:43:28.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:28 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:43:28.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:28 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:43:28.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:28 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:43:28.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:28 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:43:28.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:28 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:43:28.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:28 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:43:28.965 INFO:tasks.workunit.client.0.smithi047.stdout: 10 165685 22.95 MB/sec execute 252 sec latency 119.871 ms 2024-03-21T11:43:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:29 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:43:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:29 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:43:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:29 smithi100 ceph-mon[36245]: pgmap v1004: 129 pgs: 129 active+clean; 558 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.1 MiB/s wr, 631 op/s 2024-03-21T11:43:29.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:29 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:43:29.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:29 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:43:29.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:29 smithi102 ceph-mon[36748]: pgmap v1004: 129 pgs: 129 active+clean; 558 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.1 MiB/s wr, 631 op/s 2024-03-21T11:43:29.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:29 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:43:29.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:29 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:43:29.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:29 smithi047 ceph-mon[31351]: pgmap v1004: 129 pgs: 129 active+clean; 558 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.1 MiB/s wr, 631 op/s 2024-03-21T11:43:29.965 INFO:tasks.workunit.client.0.smithi047.stdout: 10 166266 22.98 MB/sec execute 253 sec latency 74.749 ms 2024-03-21T11:43:30.965 INFO:tasks.workunit.client.0.smithi047.stdout: 10 166867 23.01 MB/sec execute 254 sec latency 79.648 ms 2024-03-21T11:43:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:31 smithi100 ceph-mon[36245]: pgmap v1005: 129 pgs: 129 active+clean; 564 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.5 MiB/s wr, 594 op/s 2024-03-21T11:43:31.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:31 smithi102 ceph-mon[36748]: pgmap v1005: 129 pgs: 129 active+clean; 564 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.5 MiB/s wr, 594 op/s 2024-03-21T11:43:31.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:31 smithi047 ceph-mon[31351]: pgmap v1005: 129 pgs: 129 active+clean; 564 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.5 MiB/s wr, 594 op/s 2024-03-21T11:43:31.965 INFO:tasks.workunit.client.0.smithi047.stdout: 10 167365 23.05 MB/sec execute 255 sec latency 109.898 ms 2024-03-21T11:43:32.965 INFO:tasks.workunit.client.0.smithi047.stdout: 10 167657 23.03 MB/sec execute 256 sec latency 378.125 ms 2024-03-21T11:43:33.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:33 smithi100 ceph-mon[36245]: pgmap v1006: 129 pgs: 129 active+clean; 568 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.4 MiB/s wr, 590 op/s 2024-03-21T11:43:33.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:33 smithi102 ceph-mon[36748]: pgmap v1006: 129 pgs: 129 active+clean; 568 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.4 MiB/s wr, 590 op/s 2024-03-21T11:43:33.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:33 smithi047 ceph-mon[31351]: pgmap v1006: 129 pgs: 129 active+clean; 568 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.4 MiB/s wr, 590 op/s 2024-03-21T11:43:33.965 INFO:tasks.workunit.client.0.smithi047.stdout: 10 167985 23.02 MB/sec execute 257 sec latency 113.864 ms 2024-03-21T11:43:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.461+0000 7fd88a400640 1 -- 172.21.15.47:0/4106124689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd884108ef0 msgr2=0x7fd884109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.461+0000 7fd88a400640 1 --2- 172.21.15.47:0/4106124689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd884108ef0 0x7fd884109370 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fd870009900 tx=0x7fd87002bc40 comp rx=0 tx=0).stop 2024-03-21T11:43:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.461+0000 7fd88a400640 1 -- 172.21.15.47:0/4106124689 shutdown_connections 2024-03-21T11:43:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.461+0000 7fd88a400640 1 --2- 172.21.15.47:0/4106124689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8841098b0 0x7fd884110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.461+0000 7fd88a400640 1 --2- 172.21.15.47:0/4106124689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd884108ef0 0x7fd884109370 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.461+0000 7fd88a400640 1 --2- 172.21.15.47:0/4106124689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd884107d00 0x7fd884108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.461+0000 7fd88a400640 1 -- 172.21.15.47:0/4106124689 >> 172.21.15.47:0/4106124689 conn(0x7fd884076cb0 msgr2=0x7fd8840790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.461+0000 7fd88a400640 1 -- 172.21.15.47:0/4106124689 shutdown_connections 2024-03-21T11:43:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.461+0000 7fd88a400640 1 -- 172.21.15.47:0/4106124689 wait complete. 2024-03-21T11:43:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.463+0000 7fd88a400640 1 Processor -- start 2024-03-21T11:43:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.464+0000 7fd88a400640 1 -- start start 2024-03-21T11:43:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.464+0000 7fd88a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd884107d00 0x7fd88419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.464+0000 7fd88a400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd884108ef0 0x7fd88419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.464+0000 7fd88a400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8841098b0 0x7fd8841a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.464+0000 7fd88a400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8841139b0 con 0x7fd8841098b0 2024-03-21T11:43:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.464+0000 7fd88a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd884113830 con 0x7fd884107d00 2024-03-21T11:43:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.464+0000 7fd88a400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd884113b30 con 0x7fd884108ef0 2024-03-21T11:43:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.464+0000 7fd883400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd884107d00 0x7fd88419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.465+0000 7fd883400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd884107d00 0x7fd88419df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36654/0 (socket says 172.21.15.47:36654) 2024-03-21T11:43:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.465+0000 7fd883400640 1 -- 172.21.15.47:0/1202554409 learned_addr learned my addr 172.21.15.47:0/1202554409 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.465+0000 7fd883400640 1 -- 172.21.15.47:0/1202554409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd884108ef0 msgr2=0x7fd88419e470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T11:43:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.465+0000 7fd883400640 1 --2- 172.21.15.47:0/1202554409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd884108ef0 0x7fd88419e470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.465+0000 7fd883400640 1 -- 172.21.15.47:0/1202554409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8841098b0 msgr2=0x7fd8841a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.465+0000 7fd883400640 1 --2- 172.21.15.47:0/1202554409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8841098b0 0x7fd8841a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.465+0000 7fd883400640 1 -- 172.21.15.47:0/1202554409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8841a5b50 con 0x7fd884107d00 2024-03-21T11:43:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.466+0000 7fd883400640 1 --2- 172.21.15.47:0/1202554409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd884107d00 0x7fd88419df30 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fd87400d770 tx=0x7fd87400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.466+0000 7fd87be00640 1 -- 172.21.15.47:0/1202554409 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874017070 con 0x7fd884107d00 2024-03-21T11:43:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.466+0000 7fd88a400640 1 -- 172.21.15.47:0/1202554409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8841a5e40 con 0x7fd884107d00 2024-03-21T11:43:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.466+0000 7fd87be00640 1 -- 172.21.15.47:0/1202554409 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd874009c30 con 0x7fd884107d00 2024-03-21T11:43:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.466+0000 7fd87be00640 1 -- 172.21.15.47:0/1202554409 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874003dd0 con 0x7fd884107d00 2024-03-21T11:43:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.466+0000 7fd88a400640 1 -- 172.21.15.47:0/1202554409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8841a6320 con 0x7fd884107d00 2024-03-21T11:43:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.475+0000 7fd87be00640 1 -- 172.21.15.47:0/1202554409 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd874004000 con 0x7fd884107d00 2024-03-21T11:43:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.475+0000 7fd88d1bf740 1 Processor -- start 2024-03-21T11:43:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.476+0000 7fd88d1bf740 1 -- start start 2024-03-21T11:43:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.476+0000 7fd88d1bf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da5181e0 0x5652da518600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.476+0000 7fd88d1bf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da518bd0 0x5652da51d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.476+0000 7fd88d1bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da51dd40 0x5652da5201d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.476+0000 7fd88d1bf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652da4db3b0 con 0x5652da5181e0 2024-03-21T11:43:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.476+0000 7fd88d1bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652da47d830 con 0x5652da51dd40 2024-03-21T11:43:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.477+0000 7fd88d1bf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652da4265d0 con 0x5652da518bd0 2024-03-21T11:43:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.477+0000 7fd882a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da51dd40 0x5652da5201d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.477+0000 7fd882a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da51dd40 0x5652da5201d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36666/0 (socket says 172.21.15.47:36666) 2024-03-21T11:43:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.477+0000 7fd882a00640 1 -- 172.21.15.47:0/1322858396 learned_addr learned my addr 172.21.15.47:0/1322858396 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.478+0000 7fd882a00640 1 -- 172.21.15.47:0/1322858396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da518bd0 msgr2=0x5652da51d6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:43:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.478+0000 7fd882a00640 1 --2- 172.21.15.47:0/1322858396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da518bd0 0x5652da51d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.478+0000 7fd882a00640 1 -- 172.21.15.47:0/1322858396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da5181e0 msgr2=0x5652da518600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.478+0000 7fd883e00640 1 --2- 172.21.15.47:0/1322858396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da5181e0 0x5652da518600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.478+0000 7fd882a00640 1 --2- 172.21.15.47:0/1322858396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da5181e0 0x5652da518600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.478+0000 7fd882a00640 1 -- 172.21.15.47:0/1322858396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652da43e2c0 con 0x5652da51dd40 2024-03-21T11:43:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.478+0000 7fd883e00640 1 --2- 172.21.15.47:0/1322858396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da5181e0 0x5652da518600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:43:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.478+0000 7fd882a00640 1 --2- 172.21.15.47:0/1322858396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da51dd40 0x5652da5201d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd87002c7c0 tx=0x7fd870005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=466159a029052ae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.479+0000 7fd87be00640 1 --2- 172.21.15.47:0/1202554409 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8680a3110 0x7fd8680a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.479+0000 7fd87be00640 1 -- 172.21.15.47:0/1202554409 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd8740cf180 con 0x7fd884107d00 2024-03-21T11:43:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.479+0000 7fd883e00640 1 --2- 172.21.15.47:0/1202554409 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8680a3110 0x7fd8680a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.480+0000 7fd878c00640 1 -- 172.21.15.47:0/1322858396 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870042020 con 0x5652da51dd40 2024-03-21T11:43:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.480+0000 7fd878c00640 1 -- 172.21.15.47:0/1322858396 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd870009520 con 0x5652da51dd40 2024-03-21T11:43:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.480+0000 7fd878c00640 1 -- 172.21.15.47:0/1322858396 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870003bc0 con 0x5652da51dd40 2024-03-21T11:43:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.481+0000 7fd883e00640 1 --2- 172.21.15.47:0/1202554409 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8680a3110 0x7fd8680a55d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x5652da519430 tx=0x7fd87c009b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.482+0000 7fd88d1bf740 1 -- 172.21.15.47:0/1322858396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da51dd40 msgr2=0x5652da5201d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.482+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/1322858396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da51dd40 0x5652da5201d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd87002c7c0 tx=0x7fd870005b50 comp rx=0 tx=0).stop 2024-03-21T11:43:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.483+0000 7fd88d1bf740 1 -- 172.21.15.47:0/1322858396 shutdown_connections 2024-03-21T11:43:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.483+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/1322858396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da51dd40 0x5652da5201d0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.483+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/1322858396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da518bd0 0x5652da51d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.483+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/1322858396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da5181e0 0x5652da518600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.483+0000 7fd88d1bf740 1 -- 172.21.15.47:0/1322858396 >> 172.21.15.47:0/1322858396 conn(0x5652da513750 msgr2=0x5652da513b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.483+0000 7fd88d1bf740 1 -- 172.21.15.47:0/1322858396 shutdown_connections 2024-03-21T11:43:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.483+0000 7fd88d1bf740 1 -- 172.21.15.47:0/1322858396 wait complete. 2024-03-21T11:43:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.486+0000 7fd88d1bf740 1 -- start start 2024-03-21T11:43:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.486+0000 7fd88d1bf740 1 Processor -- start 2024-03-21T11:43:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.488+0000 7fd88d1bf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da5181e0 0x5652da5333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.488+0000 7fd88d1bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da518bd0 0x5652da5338e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.488+0000 7fd88d1bf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da51dd40 0x5652da53a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.488+0000 7fd88d1bf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652da439a20 con 0x5652da5181e0 2024-03-21T11:43:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.488+0000 7fd88d1bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652da4798d0 con 0x5652da518bd0 2024-03-21T11:43:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.488+0000 7fd88d1bf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652da531ca0 con 0x5652da51dd40 2024-03-21T11:43:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.488+0000 7fd88d1bf740 10 client.0 _ll_register_callbacks cb 0x5652da4761c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.488+0000 7fd883e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da51dd40 0x5652da53a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.488+0000 7fd883e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da51dd40 0x5652da53a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:53748/0 (socket says 172.21.15.47:53748) 2024-03-21T11:43:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.488+0000 7fd883e00640 1 -- 172.21.15.47:0/2660190221 learned_addr learned my addr 172.21.15.47:0/2660190221 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.489+0000 7fd882a00640 1 --2- 172.21.15.47:0/2660190221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da518bd0 0x5652da5338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.489+0000 7fd883e00640 1 -- 172.21.15.47:0/2660190221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da518bd0 msgr2=0x5652da5338e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.489+0000 7fd883e00640 1 --2- 172.21.15.47:0/2660190221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da518bd0 0x5652da5338e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.489+0000 7fd883e00640 1 -- 172.21.15.47:0/2660190221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da5181e0 msgr2=0x5652da5333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.489+0000 7fd883e00640 1 --2- 172.21.15.47:0/2660190221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da5181e0 0x5652da5333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.489+0000 7fd883e00640 1 -- 172.21.15.47:0/2660190221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652da192210 con 0x5652da51dd40 2024-03-21T11:43:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.490+0000 7fd883e00640 1 --2- 172.21.15.47:0/2660190221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da51dd40 0x5652da53a920 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0116d0 tx=0x7fd87c009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58d40519abe7293b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.491+0000 7fd86d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.491+0000 7fd86d600640 1 -- 172.21.15.47:0/2660190221 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c022070 con 0x5652da51dd40 2024-03-21T11:43:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.492+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2660190221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd870009520 con 0x5652da51dd40 2024-03-21T11:43:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.493+0000 7fd86d600640 1 -- 172.21.15.47:0/2660190221 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd87c010a20 con 0x5652da51dd40 2024-03-21T11:43:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.493+0000 7fd86d600640 1 -- 172.21.15.47:0/2660190221 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd87c02e050 con 0x5652da51dd40 2024-03-21T11:43:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.493+0000 7fd88d1bf740 10 client.31987 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.493+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2660190221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5652da25e620 con 0x5652da51dd40 2024-03-21T11:43:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.494+0000 7fd86d600640 1 -- 172.21.15.47:0/2660190221 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd870009520 con 0x5652da51dd40 2024-03-21T11:43:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.494+0000 7fd86d600640 1 -- 172.21.15.47:0/2660190221 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd87c0263b0 con 0x5652da51dd40 2024-03-21T11:43:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.494+0000 7fd88d1bf740 10 client.31987 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.494+0000 7fd88d1bf740 10 client.31987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.494+0000 7fd88d1bf740 20 client.31987 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.494+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/2660190221 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5652da53f260 0x5652da5416b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.494+0000 7fd88d1bf740 4 client.31987 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.494+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2660190221 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652da48ad70 con 0x5652da53f260 2024-03-21T11:43:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.495+0000 7fd883400640 1 --2- 172.21.15.47:0/2660190221 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5652da53f260 0x5652da5416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.496+0000 7fd883400640 1 --2- 172.21.15.47:0/2660190221 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5652da53f260 0x5652da5416b0 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49fea812339f1db3 server_cookie=eae86f8c5848e735 in_seq=0 out_seq=0 2024-03-21T11:43:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.496+0000 7fd86d600640 10 client.31987 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.498+0000 7fd86d600640 1 -- 172.21.15.47:0/2660190221 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5652da48ad70 con 0x5652da53f260 2024-03-21T11:43:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.498+0000 7fd86d600640 10 client.31987 handle_command_reply: tid=0 2024-03-21T11:43:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.498+0000 7fd88d1bf740 1 client.31987 shutdown 2024-03-21T11:43:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.499+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2660190221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da51dd40 msgr2=0x5652da53a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.499+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/2660190221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da51dd40 0x5652da53a920 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0116d0 tx=0x7fd87c009500 comp rx=0 tx=0).stop 2024-03-21T11:43:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.499+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2660190221 shutdown_connections 2024-03-21T11:43:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.499+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/2660190221 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5652da53f260 0x5652da5416b0 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.499+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/2660190221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da51dd40 0x5652da53a920 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.499+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/2660190221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da518bd0 0x5652da5338e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.499+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/2660190221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da5181e0 0x5652da5333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.499+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2660190221 >> 172.21.15.47:0/2660190221 conn(0x5652da513750 msgr2=0x5652da512950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.500+0000 7fd86d600640 0 client.31987 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.501+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2660190221 shutdown_connections 2024-03-21T11:43:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.502+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2660190221 wait complete. 2024-03-21T11:43:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.502+0000 7fd88d1bf740 20 client.31987 trim_cache size 0 max 16384 2024-03-21T11:43:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.540+0000 7fd88d1bf740 1 Processor -- start 2024-03-21T11:43:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.540+0000 7fd88d1bf740 1 -- start start 2024-03-21T11:43:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.540+0000 7fd88d1bf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da5181e0 0x5652da50e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.540+0000 7fd88d1bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da518bd0 0x5652da53a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.540+0000 7fd88d1bf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da51dd40 0x5652da532670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.541+0000 7fd882a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da518bd0 0x5652da53a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.541+0000 7fd882a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da518bd0 0x5652da53a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36690/0 (socket says 172.21.15.47:36690) 2024-03-21T11:43:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.541+0000 7fd883400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da5181e0 0x5652da50e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.542+0000 7fd882a00640 1 -- 172.21.15.47:0/1036651406 learned_addr learned my addr 172.21.15.47:0/1036651406 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.542+0000 7fd88d1bf740 1 -- 172.21.15.47:0/1036651406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8841a7ee0 con 0x5652da51dd40 2024-03-21T11:43:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.542+0000 7fd88d1bf740 1 -- 172.21.15.47:0/1036651406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8841a9860 con 0x5652da518bd0 2024-03-21T11:43:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.542+0000 7fd88d1bf740 1 -- 172.21.15.47:0/1036651406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8841a8540 con 0x5652da5181e0 2024-03-21T11:43:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.542+0000 7fd883e00640 1 --2- 172.21.15.47:0/1036651406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da51dd40 0x5652da532670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.543+0000 7fd883400640 1 -- 172.21.15.47:0/1036651406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da518bd0 msgr2=0x5652da53a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.543+0000 7fd883400640 1 --2- 172.21.15.47:0/1036651406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da518bd0 0x5652da53a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.543+0000 7fd883400640 1 -- 172.21.15.47:0/1036651406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da51dd40 msgr2=0x5652da532670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.543+0000 7fd883400640 1 --2- 172.21.15.47:0/1036651406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da51dd40 0x5652da532670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.543+0000 7fd883400640 1 -- 172.21.15.47:0/1036651406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652da5199e0 con 0x5652da5181e0 2024-03-21T11:43:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.543+0000 7fd883e00640 1 --2- 172.21.15.47:0/1036651406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da51dd40 0x5652da532670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.543+0000 7fd883400640 1 --2- 172.21.15.47:0/1036651406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da5181e0 0x5652da50e5d0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x5652da519640 tx=0x7fd874004590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ac002ccfcf01359 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.544+0000 7fd86fe00640 1 -- 172.21.15.47:0/1036651406 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874020040 con 0x5652da5181e0 2024-03-21T11:43:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.544+0000 7fd86fe00640 1 -- 172.21.15.47:0/1036651406 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd87409bbc0 con 0x5652da5181e0 2024-03-21T11:43:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.544+0000 7fd86fe00640 1 -- 172.21.15.47:0/1036651406 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8740042f0 con 0x5652da5181e0 2024-03-21T11:43:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.546+0000 7fd88d1bf740 1 -- 172.21.15.47:0/1036651406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da5181e0 msgr2=0x5652da50e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.546+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/1036651406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da5181e0 0x5652da50e5d0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x5652da519640 tx=0x7fd874004590 comp rx=0 tx=0).stop 2024-03-21T11:43:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.546+0000 7fd88d1bf740 1 -- 172.21.15.47:0/1036651406 shutdown_connections 2024-03-21T11:43:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.546+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/1036651406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da51dd40 0x5652da532670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.546+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/1036651406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da518bd0 0x5652da53a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.546+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/1036651406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da5181e0 0x5652da50e5d0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.546+0000 7fd88d1bf740 1 -- 172.21.15.47:0/1036651406 >> 172.21.15.47:0/1036651406 conn(0x5652da513750 msgr2=0x5652da5169b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.547+0000 7fd88d1bf740 1 -- 172.21.15.47:0/1036651406 shutdown_connections 2024-03-21T11:43:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.548+0000 7fd88d1bf740 1 -- 172.21.15.47:0/1036651406 wait complete. 2024-03-21T11:43:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.548+0000 7fd88d1bf740 1 -- start start 2024-03-21T11:43:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.548+0000 7fd88d1bf740 1 Processor -- start 2024-03-21T11:43:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.549+0000 7fd88d1bf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da5181e0 0x5652da5380b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.549+0000 7fd88d1bf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da518bd0 0x5652da5385f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.549+0000 7fd88d1bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da51dd40 0x5652da539590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.549+0000 7fd88d1bf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8841a9200 con 0x5652da518bd0 2024-03-21T11:43:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.550+0000 7fd88d1bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8841a8ba0 con 0x5652da51dd40 2024-03-21T11:43:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.550+0000 7fd88d1bf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd884082220 con 0x5652da5181e0 2024-03-21T11:43:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.550+0000 7fd883e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da51dd40 0x5652da539590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.550+0000 7fd88d1bf740 10 client.0 _ll_register_callbacks cb 0x7fd87c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.550+0000 7fd883e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da51dd40 0x5652da539590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36696/0 (socket says 172.21.15.47:36696) 2024-03-21T11:43:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.550+0000 7fd883e00640 1 -- 172.21.15.47:0/2316084694 learned_addr learned my addr 172.21.15.47:0/2316084694 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.550+0000 7fd883400640 1 --2- 172.21.15.47:0/2316084694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da5181e0 0x5652da5380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.550+0000 7fd882a00640 1 --2- 172.21.15.47:0/2316084694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da518bd0 0x5652da5385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.550+0000 7fd883e00640 1 -- 172.21.15.47:0/2316084694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da5181e0 msgr2=0x5652da5380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.550+0000 7fd883e00640 1 --2- 172.21.15.47:0/2316084694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da5181e0 0x5652da5380b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.550+0000 7fd883e00640 1 -- 172.21.15.47:0/2316084694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da518bd0 msgr2=0x5652da5385f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.550+0000 7fd883e00640 1 --2- 172.21.15.47:0/2316084694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da518bd0 0x5652da5385f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.550+0000 7fd883e00640 1 -- 172.21.15.47:0/2316084694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652da539b80 con 0x5652da51dd40 2024-03-21T11:43:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.555+0000 7fd883e00640 1 --2- 172.21.15.47:0/2316084694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da51dd40 0x5652da539590 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009950 tx=0x7fd87c0104a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ce47375211f944a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.555+0000 7fd86d600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:43:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.557+0000 7fd86d600640 1 -- 172.21.15.47:0/2316084694 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c036070 con 0x5652da51dd40 2024-03-21T11:43:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.557+0000 7fd86d600640 1 -- 172.21.15.47:0/2316084694 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd87c010820 con 0x5652da51dd40 2024-03-21T11:43:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.557+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2316084694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd87409bbc0 con 0x5652da51dd40 2024-03-21T11:43:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.563+0000 7fd86d600640 1 -- 172.21.15.47:0/2316084694 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd87c040050 con 0x5652da51dd40 2024-03-21T11:43:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.563+0000 7fd88d1bf740 10 client.32060 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.563+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2316084694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5652da59ac40 con 0x5652da51dd40 2024-03-21T11:43:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.563+0000 7fd86d600640 1 -- 172.21.15.47:0/2316084694 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd87409bbc0 con 0x5652da51dd40 2024-03-21T11:43:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.565+0000 7fd86d600640 1 -- 172.21.15.47:0/2316084694 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd87c01ea50 con 0x5652da51dd40 2024-03-21T11:43:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.565+0000 7fd88d1bf740 10 client.32060 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.565+0000 7fd88d1bf740 10 client.32060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.565+0000 7fd88d1bf740 20 client.32060 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.565+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/2316084694 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5652da59b4b0 0x5652da59d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.565+0000 7fd88d1bf740 4 client.32060 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:43:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.565+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2316084694 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5652da4850c0 con 0x5652da59b4b0 2024-03-21T11:43:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.565+0000 7fd883400640 1 --2- 172.21.15.47:0/2316084694 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5652da59b4b0 0x5652da59d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.566+0000 7fd883400640 1 --2- 172.21.15.47:0/2316084694 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5652da59b4b0 0x5652da59d8d0 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39119ee083658c3b server_cookie=7cda13f32f412733 in_seq=0 out_seq=0 2024-03-21T11:43:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:34.566+0000 7fd86d600640 10 client.32060 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:34.966 INFO:tasks.workunit.client.0.smithi047.stdout: 10 168213 22.95 MB/sec execute 258 sec latency 313.615 ms 2024-03-21T11:43:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.382+0000 7fd86d600640 1 -- 172.21.15.47:0/2316084694 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5652da4850c0 con 0x5652da59b4b0 2024-03-21T11:43:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.382+0000 7fd86d600640 10 client.32060 handle_command_reply: tid=0 2024-03-21T11:43:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.382+0000 7fd88d1bf740 1 client.32060 shutdown 2024-03-21T11:43:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.383+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2316084694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da51dd40 msgr2=0x5652da539590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.383+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/2316084694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da51dd40 0x5652da539590 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009950 tx=0x7fd87c0104a0 comp rx=0 tx=0).stop 2024-03-21T11:43:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.383+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2316084694 shutdown_connections 2024-03-21T11:43:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.383+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/2316084694 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5652da59b4b0 0x5652da59d8d0 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.383+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/2316084694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652da51dd40 0x5652da539590 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.383+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/2316084694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5652da518bd0 0x5652da5385f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.383+0000 7fd88d1bf740 1 --2- 172.21.15.47:0/2316084694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5652da5181e0 0x5652da5380b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.383+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2316084694 >> 172.21.15.47:0/2316084694 conn(0x5652da513750 msgr2=0x5652da532670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.383+0000 7fd86d600640 0 client.32060 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.383+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2316084694 shutdown_connections 2024-03-21T11:43:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.384+0000 7fd88d1bf740 1 -- 172.21.15.47:0/2316084694 wait complete. 2024-03-21T11:43:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.384+0000 7fd88d1bf740 20 client.32060 trim_cache size 0 max 16384 2024-03-21T11:43:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:43:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:43:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:43:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:43:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.385+0000 7fd88a400640 1 -- 172.21.15.47:0/1202554409 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8680a3110 msgr2=0x7fd8680a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.385+0000 7fd88a400640 1 --2- 172.21.15.47:0/1202554409 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8680a3110 0x7fd8680a55d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x5652da519430 tx=0x7fd87c009b50 comp rx=0 tx=0).stop 2024-03-21T11:43:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.385+0000 7fd88a400640 1 -- 172.21.15.47:0/1202554409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd884107d00 msgr2=0x7fd88419df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.385+0000 7fd88a400640 1 --2- 172.21.15.47:0/1202554409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd884107d00 0x7fd88419df30 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fd87400d770 tx=0x7fd87400dc30 comp rx=0 tx=0).stop 2024-03-21T11:43:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.386+0000 7fd88a400640 1 -- 172.21.15.47:0/1202554409 shutdown_connections 2024-03-21T11:43:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.386+0000 7fd88a400640 1 --2- 172.21.15.47:0/1202554409 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8680a3110 0x7fd8680a55d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.386+0000 7fd88a400640 1 --2- 172.21.15.47:0/1202554409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8841098b0 0x7fd8841a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.386+0000 7fd88a400640 1 --2- 172.21.15.47:0/1202554409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd884108ef0 0x7fd88419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.386+0000 7fd88a400640 1 --2- 172.21.15.47:0/1202554409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd884107d00 0x7fd88419df30 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.386+0000 7fd88a400640 1 -- 172.21.15.47:0/1202554409 >> 172.21.15.47:0/1202554409 conn(0x7fd884076cb0 msgr2=0x7fd88410c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.386+0000 7fd88a400640 1 -- 172.21.15.47:0/1202554409 shutdown_connections 2024-03-21T11:43:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:35.386+0000 7fd88a400640 1 -- 172.21.15.47:0/1202554409 wait complete. 2024-03-21T11:43:35.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:35 smithi100 ceph-mon[36245]: pgmap v1007: 129 pgs: 129 active+clean; 571 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.0 MiB/s wr, 607 op/s 2024-03-21T11:43:35.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:35 smithi102 ceph-mon[36748]: pgmap v1007: 129 pgs: 129 active+clean; 571 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.0 MiB/s wr, 607 op/s 2024-03-21T11:43:35.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:35 smithi047 ceph-mon[31351]: pgmap v1007: 129 pgs: 129 active+clean; 571 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.0 MiB/s wr, 607 op/s 2024-03-21T11:43:35.966 INFO:tasks.workunit.client.0.smithi047.stdout: 10 168524 22.92 MB/sec execute 259 sec latency 408.617 ms 2024-03-21T11:43:36.966 INFO:tasks.workunit.client.0.smithi047.stdout: 10 169055 22.92 MB/sec execute 260 sec latency 61.837 ms 2024-03-21T11:43:37.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:37 smithi102 ceph-mon[36748]: pgmap v1008: 129 pgs: 129 active+clean; 572 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.8 MiB/s wr, 482 op/s 2024-03-21T11:43:37.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:37 smithi047 ceph-mon[31351]: pgmap v1008: 129 pgs: 129 active+clean; 572 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.8 MiB/s wr, 482 op/s 2024-03-21T11:43:37.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:37 smithi100 ceph-mon[36245]: pgmap v1008: 129 pgs: 129 active+clean; 572 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.8 MiB/s wr, 482 op/s 2024-03-21T11:43:37.966 INFO:tasks.workunit.client.0.smithi047.stdout: 10 169475 22.90 MB/sec execute 261 sec latency 346.905 ms 2024-03-21T11:43:38.966 INFO:tasks.workunit.client.0.smithi047.stdout: 10 169922 22.92 MB/sec execute 262 sec latency 136.744 ms 2024-03-21T11:43:39.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:39 smithi102 ceph-mon[36748]: pgmap v1009: 129 pgs: 129 active+clean; 569 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.3 MiB/s wr, 598 op/s 2024-03-21T11:43:39.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:39 smithi047 ceph-mon[31351]: pgmap v1009: 129 pgs: 129 active+clean; 569 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.3 MiB/s wr, 598 op/s 2024-03-21T11:43:39.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:39 smithi100 ceph-mon[36245]: pgmap v1009: 129 pgs: 129 active+clean; 569 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.3 MiB/s wr, 598 op/s 2024-03-21T11:43:39.966 INFO:tasks.workunit.client.0.smithi047.stdout: 10 170633 23.01 MB/sec execute 263 sec latency 48.223 ms 2024-03-21T11:43:40.967 INFO:tasks.workunit.client.0.smithi047.stdout: 10 171012 23.02 MB/sec execute 264 sec latency 130.810 ms 2024-03-21T11:43:41.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:41 smithi102 ceph-mon[36748]: pgmap v1010: 129 pgs: 129 active+clean; 565 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.8 MiB/s wr, 622 op/s 2024-03-21T11:43:41.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:41 smithi047 ceph-mon[31351]: pgmap v1010: 129 pgs: 129 active+clean; 565 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.8 MiB/s wr, 622 op/s 2024-03-21T11:43:41.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:41 smithi100 ceph-mon[36245]: pgmap v1010: 129 pgs: 129 active+clean; 565 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.8 MiB/s wr, 622 op/s 2024-03-21T11:43:41.967 INFO:tasks.workunit.client.0.smithi047.stdout: 10 171371 23.00 MB/sec execute 265 sec latency 63.576 ms 2024-03-21T11:43:42.967 INFO:tasks.workunit.client.0.smithi047.stdout: 10 171526 22.94 MB/sec execute 266 sec latency 353.912 ms 2024-03-21T11:43:43.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:43 smithi102 ceph-mon[36748]: pgmap v1011: 129 pgs: 129 active+clean; 567 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 578 op/s 2024-03-21T11:43:43.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:43 smithi047 ceph-mon[31351]: pgmap v1011: 129 pgs: 129 active+clean; 567 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 578 op/s 2024-03-21T11:43:43.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:43 smithi100 ceph-mon[36245]: pgmap v1011: 129 pgs: 129 active+clean; 567 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 578 op/s 2024-03-21T11:43:43.967 INFO:tasks.workunit.client.0.smithi047.stdout: 10 171778 22.90 MB/sec execute 267 sec latency 240.894 ms 2024-03-21T11:43:44.967 INFO:tasks.workunit.client.0.smithi047.stdout: 10 172327 22.89 MB/sec execute 268 sec latency 88.431 ms 2024-03-21T11:43:45.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:45 smithi102 ceph-mon[36748]: pgmap v1012: 129 pgs: 129 active+clean; 576 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.4 MiB/s wr, 604 op/s 2024-03-21T11:43:45.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:45 smithi047 ceph-mon[31351]: pgmap v1012: 129 pgs: 129 active+clean; 576 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.4 MiB/s wr, 604 op/s 2024-03-21T11:43:45.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:45 smithi100 ceph-mon[36245]: pgmap v1012: 129 pgs: 129 active+clean; 576 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.4 MiB/s wr, 604 op/s 2024-03-21T11:43:45.967 INFO:tasks.workunit.client.0.smithi047.stdout: 10 172979 22.95 MB/sec execute 269 sec latency 60.145 ms 2024-03-21T11:43:46.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:43:46.524 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:43:46.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.763+0000 7fc5e9e00640 1 -- 172.21.15.47:0/4147034669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5e4104ab0 msgr2=0x7fc5e4104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:46.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.763+0000 7fc5e9e00640 1 --2- 172.21.15.47:0/4147034669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5e4104ab0 0x7fc5e4104f30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009900 tx=0x7fc5cc02bc50 comp rx=0 tx=0).stop 2024-03-21T11:43:46.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.763+0000 7fc5e9e00640 1 -- 172.21.15.47:0/4147034669 shutdown_connections 2024-03-21T11:43:46.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.763+0000 7fc5e9e00640 1 --2- 172.21.15.47:0/4147034669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e4105470 0x7fc5e410bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.763+0000 7fc5e9e00640 1 --2- 172.21.15.47:0/4147034669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5e4104ab0 0x7fc5e4104f30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.763+0000 7fc5e9e00640 1 --2- 172.21.15.47:0/4147034669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5e41038c0 0x7fc5e4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.763+0000 7fc5e9e00640 1 -- 172.21.15.47:0/4147034669 >> 172.21.15.47:0/4147034669 conn(0x7fc5e40fee50 msgr2=0x7fc5e4101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:46.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.763+0000 7fc5e9e00640 1 -- 172.21.15.47:0/4147034669 shutdown_connections 2024-03-21T11:43:46.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.763+0000 7fc5e9e00640 1 -- 172.21.15.47:0/4147034669 wait complete. 2024-03-21T11:43:46.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.765+0000 7fc5e9e00640 1 Processor -- start 2024-03-21T11:43:46.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.765+0000 7fc5e9e00640 1 -- start start 2024-03-21T11:43:46.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.765+0000 7fc5e9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5e41038c0 0x7fc5e4199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.765+0000 7fc5e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e4104ab0 0x7fc5e419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.765+0000 7fc5e9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5e4105470 0x7fc5e41a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.765+0000 7fc5e9e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e410f570 con 0x7fc5e4105470 2024-03-21T11:43:46.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.765+0000 7fc5e9e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e410f3f0 con 0x7fc5e4104ab0 2024-03-21T11:43:46.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.765+0000 7fc5e9e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e410f6f0 con 0x7fc5e41038c0 2024-03-21T11:43:46.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.766+0000 7fc5e2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e4104ab0 0x7fc5e419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.766+0000 7fc5e2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e4104ab0 0x7fc5e419a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60620/0 (socket says 172.21.15.47:60620) 2024-03-21T11:43:46.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.766+0000 7fc5e2000640 1 -- 172.21.15.47:0/982082688 learned_addr learned my addr 172.21.15.47:0/982082688 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:46.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.766+0000 7fc5e2a00640 1 --2- 172.21.15.47:0/982082688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5e41038c0 0x7fc5e4199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.766+0000 7fc5e3400640 1 --2- 172.21.15.47:0/982082688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5e4105470 0x7fc5e41a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.766+0000 7fc5e2000640 1 -- 172.21.15.47:0/982082688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5e41038c0 msgr2=0x7fc5e4199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:46.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.766+0000 7fc5e2000640 1 --2- 172.21.15.47:0/982082688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5e41038c0 0x7fc5e4199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.766+0000 7fc5e2000640 1 -- 172.21.15.47:0/982082688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5e4105470 msgr2=0x7fc5e41a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:46.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.766+0000 7fc5e2000640 1 --2- 172.21.15.47:0/982082688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5e4105470 0x7fc5e41a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.766+0000 7fc5e2000640 1 -- 172.21.15.47:0/982082688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e41a17b0 con 0x7fc5e4104ab0 2024-03-21T11:43:46.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.767+0000 7fc5e2000640 1 --2- 172.21.15.47:0/982082688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e4104ab0 0x7fc5e419a0d0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009850 tx=0x7fc5cc004e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:46.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.767+0000 7fc5e8a00640 1 -- 172.21.15.47:0/982082688 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc041070 con 0x7fc5e4104ab0 2024-03-21T11:43:46.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.767+0000 7fc5e9e00640 1 -- 172.21.15.47:0/982082688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e41a1a40 con 0x7fc5e4104ab0 2024-03-21T11:43:46.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.767+0000 7fc5e9e00640 1 -- 172.21.15.47:0/982082688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e41a1fe0 con 0x7fc5e4104ab0 2024-03-21T11:43:46.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.767+0000 7fc5e8a00640 1 -- 172.21.15.47:0/982082688 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5cc0054e0 con 0x7fc5e4104ab0 2024-03-21T11:43:46.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.767+0000 7fc5e8a00640 1 -- 172.21.15.47:0/982082688 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc003e20 con 0x7fc5e4104ab0 2024-03-21T11:43:46.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.771+0000 7fc5e8a00640 1 -- 172.21.15.47:0/982082688 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc5cc044070 con 0x7fc5e4104ab0 2024-03-21T11:43:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.771+0000 7fc5eca0d740 1 Processor -- start 2024-03-21T11:43:46.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.771+0000 7fc5eca0d740 1 -- start start 2024-03-21T11:43:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.772+0000 7fc5eca0d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.772+0000 7fc5eca0d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe70870 0x55d7abe75370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.772+0000 7fc5eca0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe759e0 0x55d7abe77e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.772+0000 7fc5eca0d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7abdd5250 con 0x55d7abe6fe80 2024-03-21T11:43:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.772+0000 7fc5eca0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7abd7da30 con 0x55d7abe759e0 2024-03-21T11:43:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.772+0000 7fc5eca0d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7abdd12f0 con 0x55d7abe70870 2024-03-21T11:43:46.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.772+0000 7fc5e3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.772+0000 7fc5e3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe702a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52482/0 (socket says 172.21.15.47:52482) 2024-03-21T11:43:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.772+0000 7fc5e3400640 1 -- 172.21.15.47:0/2503645469 learned_addr learned my addr 172.21.15.47:0/2503645469 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.772+0000 7fc5e2a00640 1 --2- 172.21.15.47:0/2503645469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe70870 0x55d7abe75370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.772+0000 7fc5e2000640 1 --2- 172.21.15.47:0/2503645469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe759e0 0x55d7abe77e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.773+0000 7fc5e8a00640 1 --2- 172.21.15.47:0/982082688 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5c80a2f70 0x7fc5c80a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.773+0000 7fc5e2a00640 1 -- 172.21.15.47:0/2503645469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe759e0 msgr2=0x55d7abe77e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.773+0000 7fc5e2a00640 1 --2- 172.21.15.47:0/2503645469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe759e0 0x55d7abe77e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.773+0000 7fc5e2a00640 1 -- 172.21.15.47:0/2503645469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 msgr2=0x55d7abe702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.773+0000 7fc5e2a00640 1 --2- 172.21.15.47:0/2503645469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.773+0000 7fc5e2a00640 1 -- 172.21.15.47:0/2503645469 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7abd68690 con 0x55d7abe70870 2024-03-21T11:43:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.773+0000 7fc5e3400640 1 --2- 172.21.15.47:0/2503645469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.773+0000 7fc5e8a00640 1 -- 172.21.15.47:0/982082688 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc5cc0f10d0 con 0x7fc5e4104ab0 2024-03-21T11:43:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.774+0000 7fc5e2a00640 1 --2- 172.21.15.47:0/2503645469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe70870 0x55d7abe75370 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009bf0 tx=0x7fc5d4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5309e8ac132c2d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:46.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.774+0000 7fc5e3400640 1 --2- 172.21.15.47:0/982082688 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5c80a2f70 0x7fc5c80a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.775+0000 7fc5d3e00640 1 -- 172.21.15.47:0/2503645469 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4004050 con 0x55d7abe70870 2024-03-21T11:43:46.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.775+0000 7fc5e3400640 1 --2- 172.21.15.47:0/982082688 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5c80a2f70 0x7fc5c80a5430 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x55d7abe710d0 tx=0x7fc5dc004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:46.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.775+0000 7fc5d3e00640 1 -- 172.21.15.47:0/2503645469 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5d4005500 con 0x55d7abe70870 2024-03-21T11:43:46.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.775+0000 7fc5d3e00640 1 -- 172.21.15.47:0/2503645469 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4004de0 con 0x55d7abe70870 2024-03-21T11:43:46.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.775+0000 7fc5eca0d740 1 -- 172.21.15.47:0/2503645469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe70870 msgr2=0x55d7abe75370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:46.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.775+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/2503645469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe70870 0x55d7abe75370 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009bf0 tx=0x7fc5d4011810 comp rx=0 tx=0).stop 2024-03-21T11:43:46.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.776+0000 7fc5eca0d740 1 -- 172.21.15.47:0/2503645469 shutdown_connections 2024-03-21T11:43:46.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.776+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/2503645469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe759e0 0x55d7abe77e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.776+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/2503645469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe70870 0x55d7abe75370 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.776+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/2503645469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.776+0000 7fc5eca0d740 1 -- 172.21.15.47:0/2503645469 >> 172.21.15.47:0/2503645469 conn(0x55d7abe38200 msgr2=0x55d7abe6b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:46.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.776+0000 7fc5eca0d740 1 -- 172.21.15.47:0/2503645469 shutdown_connections 2024-03-21T11:43:46.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.776+0000 7fc5eca0d740 1 -- 172.21.15.47:0/2503645469 wait complete. 2024-03-21T11:43:46.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.777+0000 7fc5eca0d740 1 -- start start 2024-03-21T11:43:46.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.777+0000 7fc5eca0d740 1 Processor -- start 2024-03-21T11:43:46.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5eca0d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe79e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5eca0d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe70870 0x55d7abe7a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5eca0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe759e0 0x55d7abe813e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5eca0d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7abe78990 con 0x55d7abe6fe80 2024-03-21T11:43:46.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5eca0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7abd90e80 con 0x55d7abe759e0 2024-03-21T11:43:46.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5eca0d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7abe78b10 con 0x55d7abe70870 2024-03-21T11:43:46.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5eca0d740 10 client.0 _ll_register_callbacks cb 0x55d7abda3130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:46.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5e2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe79e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5e2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe70870 0x55d7abe7a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5e2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe70870 0x55d7abe7a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59040/0 (socket says 172.21.15.47:59040) 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5e2000640 1 -- 172.21.15.47:0/4196658792 learned_addr learned my addr 172.21.15.47:0/4196658792 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5e3400640 1 --2- 172.21.15.47:0/4196658792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe759e0 0x55d7abe813e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5e2000640 1 -- 172.21.15.47:0/4196658792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe759e0 msgr2=0x55d7abe813e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5e2000640 1 --2- 172.21.15.47:0/4196658792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe759e0 0x55d7abe813e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5e2000640 1 -- 172.21.15.47:0/4196658792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 msgr2=0x55d7abe79e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5e2000640 1 --2- 172.21.15.47:0/4196658792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe79e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.778+0000 7fc5e2000640 1 -- 172.21.15.47:0/4196658792 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7abbb37d0 con 0x55d7abe70870 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.779+0000 7fc5e2000640 1 --2- 172.21.15.47:0/4196658792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe70870 0x55d7abe7a3a0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc02c260 tx=0x7fc5cc005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a812fc62cca61590 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.779+0000 7fc5d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.779+0000 7fc5d0c00640 1 -- 172.21.15.47:0/4196658792 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc04b040 con 0x55d7abe70870 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.779+0000 7fc5eca0d740 1 -- 172.21.15.47:0/4196658792 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5d4005500 con 0x55d7abe70870 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.779+0000 7fc5d0c00640 1 -- 172.21.15.47:0/4196658792 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc5cc0f4d60 con 0x55d7abe70870 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.779+0000 7fc5d0c00640 1 -- 172.21.15.47:0/4196658792 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5cc0b97c0 con 0x55d7abe70870 2024-03-21T11:43:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.779+0000 7fc5eca0d740 10 client.32005 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:46.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.779+0000 7fc5eca0d740 1 -- 172.21.15.47:0/4196658792 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d7abae93f0 con 0x55d7abe70870 2024-03-21T11:43:46.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.780+0000 7fc5d0c00640 1 -- 172.21.15.47:0/4196658792 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5d4005500 con 0x55d7abe70870 2024-03-21T11:43:46.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.780+0000 7fc5d0c00640 1 -- 172.21.15.47:0/4196658792 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc5cc0b9b90 con 0x55d7abe70870 2024-03-21T11:43:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.780+0000 7fc5eca0d740 10 client.32005 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.780+0000 7fc5eca0d740 10 client.32005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.780+0000 7fc5eca0d740 20 client.32005 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.780+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/4196658792 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7abe85c60 0x55d7abe880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.780+0000 7fc5eca0d740 4 client.32005 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.780+0000 7fc5eca0d740 1 -- 172.21.15.47:0/4196658792 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7abde2790 con 0x55d7abe85c60 2024-03-21T11:43:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.780+0000 7fc5e2a00640 1 --2- 172.21.15.47:0/4196658792 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7abe85c60 0x55d7abe880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.781+0000 7fc5e2a00640 1 --2- 172.21.15.47:0/4196658792 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7abe85c60 0x55d7abe880e0 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3015b1f00053f3ff server_cookie=7d74eadc97fcc3d1 in_seq=0 out_seq=0 2024-03-21T11:43:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.781+0000 7fc5d0c00640 10 client.32005 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.782+0000 7fc5d0c00640 1 -- 172.21.15.47:0/4196658792 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d7abde2790 con 0x55d7abe85c60 2024-03-21T11:43:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.782+0000 7fc5d0c00640 10 client.32005 handle_command_reply: tid=0 2024-03-21T11:43:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.782+0000 7fc5eca0d740 1 client.32005 shutdown 2024-03-21T11:43:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.782+0000 7fc5eca0d740 1 -- 172.21.15.47:0/4196658792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe70870 msgr2=0x55d7abe7a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.782+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/4196658792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe70870 0x55d7abe7a3a0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc02c260 tx=0x7fc5cc005d50 comp rx=0 tx=0).stop 2024-03-21T11:43:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.782+0000 7fc5eca0d740 1 -- 172.21.15.47:0/4196658792 shutdown_connections 2024-03-21T11:43:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.782+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/4196658792 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7abe85c60 0x55d7abe880e0 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.783+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/4196658792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe759e0 0x55d7abe813e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.783+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/4196658792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe70870 0x55d7abe7a3a0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.783+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/4196658792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe79e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.783+0000 7fc5d0c00640 0 client.32005 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.783+0000 7fc5eca0d740 1 -- 172.21.15.47:0/4196658792 >> 172.21.15.47:0/4196658792 conn(0x55d7abe38200 msgr2=0x55d7abe6aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.783+0000 7fc5eca0d740 1 -- 172.21.15.47:0/4196658792 shutdown_connections 2024-03-21T11:43:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.783+0000 7fc5eca0d740 1 -- 172.21.15.47:0/4196658792 wait complete. 2024-03-21T11:43:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.783+0000 7fc5eca0d740 20 client.32005 trim_cache size 0 max 16384 2024-03-21T11:43:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.808+0000 7fc5eca0d740 1 Processor -- start 2024-03-21T11:43:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.808+0000 7fc5eca0d740 1 -- start start 2024-03-21T11:43:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.808+0000 7fc5eca0d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe6ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.809+0000 7fc5eca0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe70870 0x55d7abe813a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.809+0000 7fc5eca0d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe759e0 0x55d7abe6ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.809+0000 7fc5eca0d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c000f900 con 0x55d7abe6fe80 2024-03-21T11:43:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.809+0000 7fc5eca0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c0011280 con 0x55d7abe70870 2024-03-21T11:43:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.809+0000 7fc5eca0d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c000ff60 con 0x55d7abe759e0 2024-03-21T11:43:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.809+0000 7fc5e3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe759e0 0x55d7abe6ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.809+0000 7fc5e3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe759e0 0x55d7abe6ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59056/0 (socket says 172.21.15.47:59056) 2024-03-21T11:43:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.809+0000 7fc5e3400640 1 -- 172.21.15.47:0/725203298 learned_addr learned my addr 172.21.15.47:0/725203298 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.809+0000 7fc5e2a00640 1 --2- 172.21.15.47:0/725203298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe6ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.809+0000 7fc5e2000640 1 --2- 172.21.15.47:0/725203298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe70870 0x55d7abe813a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.810+0000 7fc5e3400640 1 -- 172.21.15.47:0/725203298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe70870 msgr2=0x55d7abe813a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.810+0000 7fc5e3400640 1 --2- 172.21.15.47:0/725203298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe70870 0x55d7abe813a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.810+0000 7fc5e3400640 1 -- 172.21.15.47:0/725203298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 msgr2=0x55d7abe6ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.810+0000 7fc5e3400640 1 --2- 172.21.15.47:0/725203298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe6ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.810+0000 7fc5e3400640 1 -- 172.21.15.47:0/725203298 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7abaa90f0 con 0x55d7abe759e0 2024-03-21T11:43:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.810+0000 7fc5e2a00640 1 --2- 172.21.15.47:0/725203298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe6ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.810+0000 7fc5e3400640 1 --2- 172.21.15.47:0/725203298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe759e0 0x55d7abe6ef70 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0117c0 tx=0x7fc5dc019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf890a799dfc98fd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.811+0000 7fc5d3400640 1 -- 172.21.15.47:0/725203298 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc01c9a0 con 0x55d7abe759e0 2024-03-21T11:43:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.811+0000 7fc5d3400640 1 -- 172.21.15.47:0/725203298 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5dc01cb40 con 0x55d7abe759e0 2024-03-21T11:43:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.811+0000 7fc5d3400640 1 -- 172.21.15.47:0/725203298 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc028640 con 0x55d7abe759e0 2024-03-21T11:43:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.813+0000 7fc5eca0d740 1 -- 172.21.15.47:0/725203298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe759e0 msgr2=0x55d7abe6ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.813+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/725203298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe759e0 0x55d7abe6ef70 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0117c0 tx=0x7fc5dc019000 comp rx=0 tx=0).stop 2024-03-21T11:43:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.813+0000 7fc5eca0d740 1 -- 172.21.15.47:0/725203298 shutdown_connections 2024-03-21T11:43:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.813+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/725203298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe759e0 0x55d7abe6ef70 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.813+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/725203298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe70870 0x55d7abe813a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.813+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/725203298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe6fe80 0x55d7abe6ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.813+0000 7fc5eca0d740 1 -- 172.21.15.47:0/725203298 >> 172.21.15.47:0/725203298 conn(0x55d7abe38200 msgr2=0x55d7abe71310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.813+0000 7fc5eca0d740 1 -- 172.21.15.47:0/725203298 shutdown_connections 2024-03-21T11:43:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.813+0000 7fc5eca0d740 1 -- 172.21.15.47:0/725203298 wait complete. 2024-03-21T11:43:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.814+0000 7fc5eca0d740 1 -- start start 2024-03-21T11:43:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.814+0000 7fc5eca0d740 1 Processor -- start 2024-03-21T11:43:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.815+0000 7fc5eca0d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe6fe80 0x55d7abe80d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.815+0000 7fc5eca0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe70870 0x55d7abe81280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.815+0000 7fc5eca0d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe759e0 0x55d7abe7f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.815+0000 7fc5eca0d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c0010c20 con 0x55d7abe759e0 2024-03-21T11:43:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.815+0000 7fc5eca0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c00105c0 con 0x55d7abe70870 2024-03-21T11:43:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.815+0000 7fc5eca0d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c000dac0 con 0x55d7abe6fe80 2024-03-21T11:43:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.815+0000 7fc5eca0d740 10 client.0 _ll_register_callbacks cb 0x7fc5cc009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.816+0000 7fc5e2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe6fe80 0x55d7abe80d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.816+0000 7fc5e2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe6fe80 0x55d7abe80d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59072/0 (socket says 172.21.15.47:59072) 2024-03-21T11:43:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.816+0000 7fc5e2a00640 1 -- 172.21.15.47:0/3094632785 learned_addr learned my addr 172.21.15.47:0/3094632785 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.816+0000 7fc5e2000640 1 --2- 172.21.15.47:0/3094632785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe70870 0x55d7abe81280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.816+0000 7fc5e3400640 1 --2- 172.21.15.47:0/3094632785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe759e0 0x55d7abe7f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.817+0000 7fc5e2a00640 1 -- 172.21.15.47:0/3094632785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe70870 msgr2=0x55d7abe81280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.817+0000 7fc5e2a00640 1 --2- 172.21.15.47:0/3094632785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe70870 0x55d7abe81280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.817+0000 7fc5e2a00640 1 -- 172.21.15.47:0/3094632785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe759e0 msgr2=0x55d7abe7f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.817+0000 7fc5e2a00640 1 --2- 172.21.15.47:0/3094632785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe759e0 0x55d7abe7f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.817+0000 7fc5e2a00640 1 -- 172.21.15.47:0/3094632785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7abe7fbf0 con 0x55d7abe6fe80 2024-03-21T11:43:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.817+0000 7fc5e2a00640 1 --2- 172.21.15.47:0/3094632785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe6fe80 0x55d7abe80d40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009230 tx=0x7fc5d4011dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ca79064875596c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.817+0000 7fc5d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.819+0000 7fc5d0c00640 1 -- 172.21.15.47:0/3094632785 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4032020 con 0x55d7abe6fe80 2024-03-21T11:43:46.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.819+0000 7fc5d0c00640 1 -- 172.21.15.47:0/3094632785 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc5d4005bf0 con 0x55d7abe6fe80 2024-03-21T11:43:46.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.819+0000 7fc5eca0d740 1 -- 172.21.15.47:0/3094632785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5dc01cb40 con 0x55d7abe6fe80 2024-03-21T11:43:46.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.820+0000 7fc5d0c00640 1 -- 172.21.15.47:0/3094632785 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5d402e5a0 con 0x55d7abe6fe80 2024-03-21T11:43:46.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.820+0000 7fc5eca0d740 10 client.32011 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:46.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.820+0000 7fc5eca0d740 1 -- 172.21.15.47:0/3094632785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d7abee2120 con 0x55d7abe6fe80 2024-03-21T11:43:46.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.820+0000 7fc5d0c00640 1 -- 172.21.15.47:0/3094632785 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5dc01cb40 con 0x55d7abe6fe80 2024-03-21T11:43:46.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.821+0000 7fc5d0c00640 1 -- 172.21.15.47:0/3094632785 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc5d402d690 con 0x55d7abe6fe80 2024-03-21T11:43:46.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.821+0000 7fc5eca0d740 10 client.32011 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:46.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.821+0000 7fc5eca0d740 10 client.32011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:46.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.821+0000 7fc5eca0d740 20 client.32011 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:46.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.821+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/3094632785 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7abee28f0 0x55d7abee4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:46.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.821+0000 7fc5eca0d740 4 client.32011 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:43:46.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.821+0000 7fc5eca0d740 1 -- 172.21.15.47:0/3094632785 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d7abddcae0 con 0x55d7abee28f0 2024-03-21T11:43:46.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.821+0000 7fc5e2000640 1 --2- 172.21.15.47:0/3094632785 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7abee28f0 0x55d7abee4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:46.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.822+0000 7fc5e2000640 1 --2- 172.21.15.47:0/3094632785 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7abee28f0 0x55d7abee4d10 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef8a071edfa8170 server_cookie=8e3293247bec9374 in_seq=0 out_seq=0 2024-03-21T11:43:46.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.822+0000 7fc5d0c00640 10 client.32011 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.830+0000 7fc5d0c00640 1 -- 172.21.15.47:0/3094632785 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d7abddcae0 con 0x55d7abee28f0 2024-03-21T11:43:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.830+0000 7fc5d0c00640 10 client.32011 handle_command_reply: tid=0 2024-03-21T11:43:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.830+0000 7fc5eca0d740 1 client.32011 shutdown 2024-03-21T11:43:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.831+0000 7fc5eca0d740 1 -- 172.21.15.47:0/3094632785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe6fe80 msgr2=0x55d7abe80d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:46.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.831+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/3094632785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe6fe80 0x55d7abe80d40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009230 tx=0x7fc5d4011dc0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.831+0000 7fc5eca0d740 1 -- 172.21.15.47:0/3094632785 shutdown_connections 2024-03-21T11:43:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.831+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/3094632785 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d7abee28f0 0x55d7abee4d10 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.831+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/3094632785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d7abe759e0 0x55d7abe7f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.831+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/3094632785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d7abe70870 0x55d7abe81280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.831+0000 7fc5d0c00640 0 client.32011 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.831+0000 7fc5eca0d740 1 --2- 172.21.15.47:0/3094632785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7abe6fe80 0x55d7abe80d40 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.831+0000 7fc5eca0d740 1 -- 172.21.15.47:0/3094632785 >> 172.21.15.47:0/3094632785 conn(0x55d7abe38200 msgr2=0x55d7abe6a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.831+0000 7fc5eca0d740 1 -- 172.21.15.47:0/3094632785 shutdown_connections 2024-03-21T11:43:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.831+0000 7fc5eca0d740 1 -- 172.21.15.47:0/3094632785 wait complete. 2024-03-21T11:43:46.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.831+0000 7fc5eca0d740 20 client.32011 trim_cache size 0 max 16384 2024-03-21T11:43:46.833 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:43:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:43:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:43:46.834 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:43:46.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.832+0000 7fc5e9e00640 1 -- 172.21.15.47:0/982082688 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5c80a2f70 msgr2=0x7fc5c80a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:46.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.832+0000 7fc5e9e00640 1 --2- 172.21.15.47:0/982082688 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5c80a2f70 0x7fc5c80a5430 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x55d7abe710d0 tx=0x7fc5dc004a40 comp rx=0 tx=0).stop 2024-03-21T11:43:46.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.832+0000 7fc5e9e00640 1 -- 172.21.15.47:0/982082688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e4104ab0 msgr2=0x7fc5e419a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:46.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.832+0000 7fc5e9e00640 1 --2- 172.21.15.47:0/982082688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e4104ab0 0x7fc5e419a0d0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009850 tx=0x7fc5cc004e30 comp rx=0 tx=0).stop 2024-03-21T11:43:46.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.832+0000 7fc5e9e00640 1 -- 172.21.15.47:0/982082688 shutdown_connections 2024-03-21T11:43:46.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.832+0000 7fc5e9e00640 1 --2- 172.21.15.47:0/982082688 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5c80a2f70 0x7fc5c80a5430 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.832+0000 7fc5e9e00640 1 --2- 172.21.15.47:0/982082688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5e4105470 0x7fc5e41a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.832+0000 7fc5e9e00640 1 --2- 172.21.15.47:0/982082688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5e4104ab0 0x7fc5e419a0d0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.832+0000 7fc5e9e00640 1 --2- 172.21.15.47:0/982082688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5e41038c0 0x7fc5e4199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:46.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.832+0000 7fc5e9e00640 1 -- 172.21.15.47:0/982082688 >> 172.21.15.47:0/982082688 conn(0x7fc5e40fee50 msgr2=0x7fc5e4107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:46.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.832+0000 7fc5e9e00640 1 -- 172.21.15.47:0/982082688 shutdown_connections 2024-03-21T11:43:46.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:46.832+0000 7fc5e9e00640 1 -- 172.21.15.47:0/982082688 wait complete. 2024-03-21T11:43:46.853 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:43:46.853 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:43:46.967 INFO:tasks.workunit.client.0.smithi047.stdout: 10 173530 22.98 MB/sec execute 270 sec latency 65.667 ms 2024-03-21T11:43:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.082+0000 7f158a000640 1 -- 172.21.15.47:0/863221994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15840756e0 msgr2=0x7f1584073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.082+0000 7f158a000640 1 --2- 172.21.15.47:0/863221994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15840756e0 0x7f1584073b30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1574009930 tx=0x7f157402bc30 comp rx=0 tx=0).stop 2024-03-21T11:43:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.082+0000 7f158a000640 1 -- 172.21.15.47:0/863221994 shutdown_connections 2024-03-21T11:43:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.082+0000 7f158a000640 1 --2- 172.21.15.47:0/863221994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f158410dc80 0x7f1584110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.082+0000 7f158a000640 1 --2- 172.21.15.47:0/863221994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1584074100 0x7f1584074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.082+0000 7f158a000640 1 --2- 172.21.15.47:0/863221994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15840756e0 0x7f1584073b30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.082+0000 7f158a000640 1 -- 172.21.15.47:0/863221994 >> 172.21.15.47:0/863221994 conn(0x7f15840fcc80 msgr2=0x7f15840ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.082+0000 7f158a000640 1 -- 172.21.15.47:0/863221994 shutdown_connections 2024-03-21T11:43:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.082+0000 7f158a000640 1 -- 172.21.15.47:0/863221994 wait complete. 2024-03-21T11:43:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.083+0000 7f158a000640 1 Processor -- start 2024-03-21T11:43:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.083+0000 7f158a000640 1 -- start start 2024-03-21T11:43:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.084+0000 7f158a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1584074100 0x7f158419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.084+0000 7f158a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f158410dc80 0x7f158419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.084+0000 7f158a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15841a3780 0x7f15841a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.084+0000 7f158a000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1584113950 con 0x7f158410dc80 2024-03-21T11:43:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.084+0000 7f158a000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15841137d0 con 0x7f15841a3780 2024-03-21T11:43:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.084+0000 7f158a000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1584113ad0 con 0x7f1584074100 2024-03-21T11:43:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.084+0000 7f1583400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1584074100 0x7f158419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.084+0000 7f1583e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15841a3780 0x7f15841a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.084+0000 7f1582a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f158410dc80 0x7f158419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.084+0000 7f1583400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1584074100 0x7f158419e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59098/0 (socket says 172.21.15.47:59098) 2024-03-21T11:43:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.084+0000 7f1583400640 1 -- 172.21.15.47:0/2987146322 learned_addr learned my addr 172.21.15.47:0/2987146322 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.085+0000 7f1583400640 1 -- 172.21.15.47:0/2987146322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15841a3780 msgr2=0x7f15841a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.085+0000 7f1583400640 1 --2- 172.21.15.47:0/2987146322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15841a3780 0x7f15841a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.085+0000 7f1583400640 1 -- 172.21.15.47:0/2987146322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f158410dc80 msgr2=0x7f158419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.085+0000 7f1583400640 1 --2- 172.21.15.47:0/2987146322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f158410dc80 0x7f158419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.085+0000 7f1583400640 1 -- 172.21.15.47:0/2987146322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15841a61e0 con 0x7f1584074100 2024-03-21T11:43:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.085+0000 7f1583e00640 1 --2- 172.21.15.47:0/2987146322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15841a3780 0x7f15841a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.085+0000 7f1583400640 1 --2- 172.21.15.47:0/2987146322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1584074100 0x7f158419e1e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f1574036040 tx=0x7f157402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.085+0000 7f157be00640 1 -- 172.21.15.47:0/2987146322 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1574043020 con 0x7f1584074100 2024-03-21T11:43:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.085+0000 7f158a000640 1 -- 172.21.15.47:0/2987146322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15841a6470 con 0x7f1584074100 2024-03-21T11:43:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.086+0000 7f157be00640 1 -- 172.21.15.47:0/2987146322 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15740054e0 con 0x7f1584074100 2024-03-21T11:43:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.086+0000 7f157be00640 1 -- 172.21.15.47:0/2987146322 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1574003af0 con 0x7f1584074100 2024-03-21T11:43:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.086+0000 7f158a000640 1 -- 172.21.15.47:0/2987146322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15841a6990 con 0x7f1584074100 2024-03-21T11:43:47.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.087+0000 7f158a000640 1 -- 172.21.15.47:0/2987146322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15840688e0 con 0x7f1584074100 2024-03-21T11:43:47.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.090+0000 7f157be00640 1 -- 172.21.15.47:0/2987146322 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1574046020 con 0x7f1584074100 2024-03-21T11:43:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.092+0000 7f157be00640 1 --2- 172.21.15.47:0/2987146322 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15680a2fa0 0x7f15680a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.093+0000 7f157be00640 1 -- 172.21.15.47:0/2987146322 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f15740f15e0 con 0x7f1584074100 2024-03-21T11:43:47.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.093+0000 7f1582a00640 1 --2- 172.21.15.47:0/2987146322 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15680a2fa0 0x7f15680a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.094+0000 7f1582a00640 1 --2- 172.21.15.47:0/2987146322 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15680a2fa0 0x7f15680a5460 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f158419f6f0 tx=0x7f1570005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:47.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.099+0000 7f157be00640 1 -- 172.21.15.47:0/2987146322 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f157404d020 con 0x7f1584074100 2024-03-21T11:43:47.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:47 smithi047 ceph-mon[31351]: pgmap v1013: 129 pgs: 129 active+clean; 576 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 564 op/s 2024-03-21T11:43:47.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:47 smithi102 ceph-mon[36748]: pgmap v1013: 129 pgs: 129 active+clean; 576 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 564 op/s 2024-03-21T11:43:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.588+0000 7f158a000640 1 -- 172.21.15.47:0/2987146322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f158402d440 con 0x7f1584074100 2024-03-21T11:43:47.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.589+0000 7f157be00640 1 -- 172.21.15.47:0/2987146322 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f15740be080 con 0x7f1584074100 2024-03-21T11:43:47.592 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:43:47.592 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:39:47.303461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:43:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-21T11:43:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.594+0000 7f158a000640 1 -- 172.21.15.47:0/2987146322 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15680a2fa0 msgr2=0x7f15680a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.594+0000 7f158a000640 1 --2- 172.21.15.47:0/2987146322 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15680a2fa0 0x7f15680a5460 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f158419f6f0 tx=0x7f1570005f90 comp rx=0 tx=0).stop 2024-03-21T11:43:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.594+0000 7f158a000640 1 -- 172.21.15.47:0/2987146322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1584074100 msgr2=0x7f158419e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.594+0000 7f158a000640 1 --2- 172.21.15.47:0/2987146322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1584074100 0x7f158419e1e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f1574036040 tx=0x7f157402c650 comp rx=0 tx=0).stop 2024-03-21T11:43:47.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.595+0000 7f158a000640 1 -- 172.21.15.47:0/2987146322 shutdown_connections 2024-03-21T11:43:47.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.595+0000 7f158a000640 1 --2- 172.21.15.47:0/2987146322 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f15680a2fa0 0x7f15680a5460 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.595+0000 7f158a000640 1 --2- 172.21.15.47:0/2987146322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15841a3780 0x7f15841a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.595+0000 7f158a000640 1 --2- 172.21.15.47:0/2987146322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f158410dc80 0x7f158419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.595+0000 7f158a000640 1 --2- 172.21.15.47:0/2987146322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1584074100 0x7f158419e1e0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.595+0000 7f158a000640 1 -- 172.21.15.47:0/2987146322 >> 172.21.15.47:0/2987146322 conn(0x7f15840fcc80 msgr2=0x7f1584101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:47.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.595+0000 7f158a000640 1 -- 172.21.15.47:0/2987146322 shutdown_connections 2024-03-21T11:43:47.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.595+0000 7f158a000640 1 -- 172.21.15.47:0/2987146322 wait complete. 2024-03-21T11:43:47.635 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:43:47.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:43:47.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:47 smithi100 ceph-mon[36245]: pgmap v1013: 129 pgs: 129 active+clean; 576 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 564 op/s 2024-03-21T11:43:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.933+0000 7fe4c9800640 1 -- 172.21.15.47:0/2987891785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4c4104990 msgr2=0x7fe4c4104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.933+0000 7fe4c9800640 1 --2- 172.21.15.47:0/2987891785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4c4104990 0x7fe4c4104e10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009900 tx=0x7fe4ac02bc30 comp rx=0 tx=0).stop 2024-03-21T11:43:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.933+0000 7fe4c9800640 1 -- 172.21.15.47:0/2987891785 shutdown_connections 2024-03-21T11:43:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.933+0000 7fe4c9800640 1 --2- 172.21.15.47:0/2987891785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4c4105350 0x7fe4c410bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.933+0000 7fe4c9800640 1 --2- 172.21.15.47:0/2987891785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4c4104990 0x7fe4c4104e10 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.933+0000 7fe4c9800640 1 --2- 172.21.15.47:0/2987891785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4c41037a0 0x7fe4c4103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.933+0000 7fe4c9800640 1 -- 172.21.15.47:0/2987891785 >> 172.21.15.47:0/2987891785 conn(0x7fe4c40fed10 msgr2=0x7fe4c4101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.933+0000 7fe4c9800640 1 -- 172.21.15.47:0/2987891785 shutdown_connections 2024-03-21T11:43:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.933+0000 7fe4c9800640 1 -- 172.21.15.47:0/2987891785 wait complete. 2024-03-21T11:43:47.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.934+0000 7fe4c9800640 1 Processor -- start 2024-03-21T11:43:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.934+0000 7fe4c9800640 1 -- start start 2024-03-21T11:43:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.934+0000 7fe4c9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4c41037a0 0x7fe4c4199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.934+0000 7fe4c9800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4c4104990 0x7fe4c4199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.934+0000 7fe4c9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4c4105350 0x7fe4c41a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.935+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4c41037a0 0x7fe4c4199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.935+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4c41037a0 0x7fe4c4199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60662/0 (socket says 172.21.15.47:60662) 2024-03-21T11:43:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.935+0000 7fe4c3e00640 1 -- 172.21.15.47:0/2862090464 learned_addr learned my addr 172.21.15.47:0/2862090464 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.935+0000 7fe4c9800640 1 -- 172.21.15.47:0/2862090464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c410f420 con 0x7fe4c4104990 2024-03-21T11:43:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.935+0000 7fe4c9800640 1 -- 172.21.15.47:0/2862090464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c410f2a0 con 0x7fe4c41037a0 2024-03-21T11:43:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.935+0000 7fe4c9800640 1 -- 172.21.15.47:0/2862090464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c410f5a0 con 0x7fe4c4105350 2024-03-21T11:43:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.935+0000 7fe4bbe00640 1 --2- 172.21.15.47:0/2862090464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4c4104990 0x7fe4c4199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.935+0000 7fe4c3e00640 1 -- 172.21.15.47:0/2862090464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4c4105350 msgr2=0x7fe4c41a1040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T11:43:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.935+0000 7fe4c3e00640 1 --2- 172.21.15.47:0/2862090464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4c4105350 0x7fe4c41a1040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.935+0000 7fe4c3e00640 1 -- 172.21.15.47:0/2862090464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4c4104990 msgr2=0x7fe4c4199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.935+0000 7fe4c3e00640 1 --2- 172.21.15.47:0/2862090464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4c4104990 0x7fe4c4199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.936+0000 7fe4c3e00640 1 -- 172.21.15.47:0/2862090464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c41a16b0 con 0x7fe4c41037a0 2024-03-21T11:43:47.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.936+0000 7fe4c3e00640 1 --2- 172.21.15.47:0/2862090464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4c41037a0 0x7fe4c4199a90 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e9c0 tx=0x7fe4b400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.936+0000 7fe4c1600640 1 -- 172.21.15.47:0/2862090464 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400cd80 con 0x7fe4c41037a0 2024-03-21T11:43:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.937+0000 7fe4c9800640 1 -- 172.21.15.47:0/2862090464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c41a19a0 con 0x7fe4c41037a0 2024-03-21T11:43:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.937+0000 7fe4c9800640 1 -- 172.21.15.47:0/2862090464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c41a1ec0 con 0x7fe4c41037a0 2024-03-21T11:43:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.937+0000 7fe4c1600640 1 -- 172.21.15.47:0/2862090464 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4b40035f0 con 0x7fe4c41037a0 2024-03-21T11:43:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.937+0000 7fe4c1600640 1 -- 172.21.15.47:0/2862090464 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4010690 con 0x7fe4c41037a0 2024-03-21T11:43:47.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.940+0000 7fe4cda23740 1 Processor -- start 2024-03-21T11:43:47.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.941+0000 7fe4cda23740 1 -- start start 2024-03-21T11:43:47.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.941+0000 7fe4cda23740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e62200 0x5563c9e62620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.942+0000 7fe4cda23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e633d0 0x5563c9e63850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.942+0000 7fe4cda23740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e63d90 0x5563c9e6a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.942+0000 7fe4cda23740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c9dbe2c0 con 0x5563c9e63d90 2024-03-21T11:43:47.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.942+0000 7fe4cda23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c9d66c90 con 0x5563c9e633d0 2024-03-21T11:43:47.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.942+0000 7fe4cda23740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c9dba360 con 0x5563c9e62200 2024-03-21T11:43:47.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.942+0000 7fe4c8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e62200 0x5563c9e62620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.942+0000 7fe4c8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e62200 0x5563c9e62620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59118/0 (socket says 172.21.15.47:59118) 2024-03-21T11:43:47.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.942+0000 7fe4c8e00640 1 -- 172.21.15.47:0/63728867 learned_addr learned my addr 172.21.15.47:0/63728867 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:47.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.942+0000 7fe4c1600640 1 -- 172.21.15.47:0/2862090464 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe4b4004030 con 0x7fe4c41037a0 2024-03-21T11:43:47.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.942+0000 7fe4bbe00640 1 --2- 172.21.15.47:0/63728867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e63d90 0x5563c9e6a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.943+0000 7fe4c8e00640 1 -- 172.21.15.47:0/63728867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e633d0 msgr2=0x5563c9e63850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:43:47.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.943+0000 7fe4c8e00640 1 --2- 172.21.15.47:0/63728867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e633d0 0x5563c9e63850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.943+0000 7fe4c8e00640 1 -- 172.21.15.47:0/63728867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e63d90 msgr2=0x5563c9e6a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:47.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.943+0000 7fe4c8e00640 1 --2- 172.21.15.47:0/63728867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e63d90 0x5563c9e6a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.943+0000 7fe4c8e00640 1 -- 172.21.15.47:0/63728867 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563c9d8b080 con 0x5563c9e62200 2024-03-21T11:43:47.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.943+0000 7fe4c8e00640 1 --2- 172.21.15.47:0/63728867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e62200 0x5563c9e62620 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009e30 tx=0x7fe4bc00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5482a2a046fc812e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:47.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.943+0000 7fe4b9600640 1 -- 172.21.15.47:0/63728867 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc004d20 con 0x5563c9e62200 2024-03-21T11:43:47.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.943+0000 7fe4b9600640 1 -- 172.21.15.47:0/63728867 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4bc00ee70 con 0x5563c9e62200 2024-03-21T11:43:47.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.943+0000 7fe4b9600640 1 -- 172.21.15.47:0/63728867 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc010820 con 0x5563c9e62200 2024-03-21T11:43:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.945+0000 7fe4c1600640 1 --2- 172.21.15.47:0/2862090464 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4a80a3020 0x7fe4a80a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.945+0000 7fe4cda23740 1 -- 172.21.15.47:0/63728867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e62200 msgr2=0x5563c9e62620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.945+0000 7fe4cda23740 1 --2- 172.21.15.47:0/63728867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e62200 0x5563c9e62620 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009e30 tx=0x7fe4bc00e720 comp rx=0 tx=0).stop 2024-03-21T11:43:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.946+0000 7fe4c1600640 1 -- 172.21.15.47:0/2862090464 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe4b40cfbe0 con 0x7fe4c41037a0 2024-03-21T11:43:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.946+0000 7fe4cda23740 1 -- 172.21.15.47:0/63728867 shutdown_connections 2024-03-21T11:43:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.946+0000 7fe4cda23740 1 --2- 172.21.15.47:0/63728867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e63d90 0x5563c9e6a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.946+0000 7fe4cda23740 1 --2- 172.21.15.47:0/63728867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e633d0 0x5563c9e63850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.946+0000 7fe4cda23740 1 --2- 172.21.15.47:0/63728867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e62200 0x5563c9e62620 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.946+0000 7fe4cda23740 1 -- 172.21.15.47:0/63728867 >> 172.21.15.47:0/63728867 conn(0x5563c9e5d5f0 msgr2=0x5563c9e5d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.946+0000 7fe4c3e00640 1 --2- 172.21.15.47:0/2862090464 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4a80a3020 0x7fe4a80a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.947+0000 7fe4cda23740 1 -- 172.21.15.47:0/63728867 shutdown_connections 2024-03-21T11:43:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.947+0000 7fe4cda23740 1 -- 172.21.15.47:0/63728867 wait complete. 2024-03-21T11:43:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.947+0000 7fe4cda23740 1 -- start start 2024-03-21T11:43:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.948+0000 7fe4c3e00640 1 --2- 172.21.15.47:0/2862090464 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4a80a3020 0x7fe4a80a54e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400c6a0 tx=0x7fe4b409bab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.948+0000 7fe4cda23740 1 Processor -- start 2024-03-21T11:43:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.949+0000 7fe4cda23740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 0x5563c9e6c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.949+0000 7fe4cda23740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e633d0 0x5563c9e6c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.949+0000 7fe4cda23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 0x5563c9e739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.949+0000 7fe4cda23740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c9e6ada0 con 0x5563c9e62200 2024-03-21T11:43:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.949+0000 7fe4cda23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c9d7a0d0 con 0x5563c9e63d90 2024-03-21T11:43:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.949+0000 7fe4cda23740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c9e6af20 con 0x5563c9e633d0 2024-03-21T11:43:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.949+0000 7fe4cda23740 10 client.0 _ll_register_callbacks cb 0x5563c9d7ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.950+0000 7fe4c8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 0x5563c9e6c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.950+0000 7fe4bbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e633d0 0x5563c9e6c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.950+0000 7fe4bbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e633d0 0x5563c9e6c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59130/0 (socket says 172.21.15.47:59130) 2024-03-21T11:43:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.950+0000 7fe4bbe00640 1 -- 172.21.15.47:0/3374660563 learned_addr learned my addr 172.21.15.47:0/3374660563 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:47.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.951+0000 7fe4bbe00640 1 -- 172.21.15.47:0/3374660563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 msgr2=0x5563c9e739a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:43:47.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.951+0000 7fe4bbe00640 1 --2- 172.21.15.47:0/3374660563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 0x5563c9e739a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.951+0000 7fe4bbe00640 1 -- 172.21.15.47:0/3374660563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 msgr2=0x5563c9e6c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:47.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.951+0000 7fe4bbe00640 1 --2- 172.21.15.47:0/3374660563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 0x5563c9e6c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.951+0000 7fe4bbe00640 1 -- 172.21.15.47:0/3374660563 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563c9ad23f0 con 0x5563c9e633d0 2024-03-21T11:43:47.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.951+0000 7fe4bbe00640 1 --2- 172.21.15.47:0/3374660563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e633d0 0x5563c9e6c960 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac036040 tx=0x7fe4ac004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5877239e89ab871 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:47.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.951+0000 7fe4b2000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:47.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.952+0000 7fe4b2000640 1 -- 172.21.15.47:0/3374660563 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac005d50 con 0x5563c9e633d0 2024-03-21T11:43:47.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.952+0000 7fe4b2000640 1 -- 172.21.15.47:0/3374660563 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe4ac005230 con 0x5563c9e633d0 2024-03-21T11:43:47.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.952+0000 7fe4cda23740 1 -- 172.21.15.47:0/3374660563 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4bc00ee70 con 0x5563c9e633d0 2024-03-21T11:43:47.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.953+0000 7fe4b2000640 1 -- 172.21.15.47:0/3374660563 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4ac005430 con 0x5563c9e633d0 2024-03-21T11:43:47.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.953+0000 7fe4cda23740 10 client.32026 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:47.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.953+0000 7fe4cda23740 1 -- 172.21.15.47:0/3374660563 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5563c9b9c7d0 con 0x5563c9e633d0 2024-03-21T11:43:47.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.954+0000 7fe4b2000640 1 -- 172.21.15.47:0/3374660563 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4bc00ee70 con 0x5563c9e633d0 2024-03-21T11:43:47.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.954+0000 7fe4b2000640 1 -- 172.21.15.47:0/3374660563 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe4ac003a00 con 0x5563c9e633d0 2024-03-21T11:43:47.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.954+0000 7fe4cda23740 10 client.32026 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:47.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.954+0000 7fe4cda23740 10 client.32026 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:43:47.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.954+0000 7fe4cda23740 20 client.32026 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:47.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.954+0000 7fe4cda23740 1 --2- 172.21.15.47:0/3374660563 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5563c9e77e70 0x5563c9e7a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.954+0000 7fe4cda23740 4 client.32026 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:47.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.954+0000 7fe4cda23740 1 -- 172.21.15.47:0/3374660563 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563c9dcb800 con 0x5563c9e77e70 2024-03-21T11:43:47.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.955+0000 7fe4c8e00640 1 --2- 172.21.15.47:0/3374660563 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5563c9e77e70 0x5563c9e7a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.955+0000 7fe4c8e00640 1 --2- 172.21.15.47:0/3374660563 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5563c9e77e70 0x5563c9e7a2d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2cdd8a1855b525e7 server_cookie=acf48954a0f8ae86 in_seq=0 out_seq=0 2024-03-21T11:43:47.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.955+0000 7fe4b2000640 10 client.32026 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:43:47.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.957+0000 7fe4b2000640 1 -- 172.21.15.47:0/3374660563 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5563c9dcb800 con 0x5563c9e77e70 2024-03-21T11:43:47.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.957+0000 7fe4b2000640 10 client.32026 handle_command_reply: tid=0 2024-03-21T11:43:47.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.957+0000 7fe4cda23740 1 client.32026 shutdown 2024-03-21T11:43:47.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.958+0000 7fe4cda23740 1 -- 172.21.15.47:0/3374660563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e633d0 msgr2=0x5563c9e6c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:47.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.958+0000 7fe4cda23740 1 --2- 172.21.15.47:0/3374660563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e633d0 0x5563c9e6c960 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac036040 tx=0x7fe4ac004270 comp rx=0 tx=0).stop 2024-03-21T11:43:47.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.958+0000 7fe4cda23740 1 -- 172.21.15.47:0/3374660563 shutdown_connections 2024-03-21T11:43:47.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.958+0000 7fe4cda23740 1 --2- 172.21.15.47:0/3374660563 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5563c9e77e70 0x5563c9e7a2d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.958+0000 7fe4cda23740 1 --2- 172.21.15.47:0/3374660563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 0x5563c9e739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.958+0000 7fe4cda23740 1 --2- 172.21.15.47:0/3374660563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e633d0 0x5563c9e6c960 secure :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac036040 tx=0x7fe4ac004270 comp rx=0 tx=0).stop 2024-03-21T11:43:47.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.958+0000 7fe4cda23740 1 --2- 172.21.15.47:0/3374660563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 0x5563c9e6c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.958+0000 7fe4cda23740 1 -- 172.21.15.47:0/3374660563 >> 172.21.15.47:0/3374660563 conn(0x5563c9e5d5f0 msgr2=0x5563c9e54160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:47.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.958+0000 7fe4b2000640 0 client.32026 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:43:47.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.958+0000 7fe4cda23740 1 -- 172.21.15.47:0/3374660563 shutdown_connections 2024-03-21T11:43:47.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.959+0000 7fe4cda23740 1 -- 172.21.15.47:0/3374660563 wait complete. 2024-03-21T11:43:47.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.960+0000 7fe4cda23740 20 client.32026 trim_cache size 0 max 16384 2024-03-21T11:43:47.969 INFO:tasks.workunit.client.0.smithi047.stdout: 10 173752 22.93 MB/sec execute 271 sec latency 427.471 ms 2024-03-21T11:43:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.983+0000 7fe4cda23740 1 Processor -- start 2024-03-21T11:43:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.983+0000 7fe4cda23740 1 -- start start 2024-03-21T11:43:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.983+0000 7fe4cda23740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 0x5563c9e66600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.983+0000 7fe4cda23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 0x5563c9e73960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.983+0000 7fe4cda23740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e6ca90 0x5563c9e66b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.983+0000 7fe4cda23740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a000f950 con 0x5563c9e62200 2024-03-21T11:43:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.983+0000 7fe4cda23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a00112d0 con 0x5563c9e63d90 2024-03-21T11:43:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.983+0000 7fe4cda23740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a000ffb0 con 0x5563c9e6ca90 2024-03-21T11:43:47.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.984+0000 7fe4c8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e6ca90 0x5563c9e66b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.984+0000 7fe4bbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 0x5563c9e73960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.984+0000 7fe4c8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e6ca90 0x5563c9e66b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59134/0 (socket says 172.21.15.47:59134) 2024-03-21T11:43:47.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.984+0000 7fe4c8e00640 1 -- 172.21.15.47:0/914855347 learned_addr learned my addr 172.21.15.47:0/914855347 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:47.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.984+0000 7fe4c8e00640 1 --2- 172.21.15.47:0/914855347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 0x5563c9e66600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.984+0000 7fe4bbe00640 1 -- 172.21.15.47:0/914855347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e6ca90 msgr2=0x5563c9e66b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:47.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.984+0000 7fe4bbe00640 1 --2- 172.21.15.47:0/914855347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e6ca90 0x5563c9e66b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.984+0000 7fe4bbe00640 1 -- 172.21.15.47:0/914855347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 msgr2=0x5563c9e66600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:47.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.984+0000 7fe4bbe00640 1 --2- 172.21.15.47:0/914855347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 0x5563c9e66600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.984+0000 7fe4bbe00640 1 -- 172.21.15.47:0/914855347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563c9e69eb0 con 0x5563c9e63d90 2024-03-21T11:43:47.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.984+0000 7fe4c8e00640 1 --2- 172.21.15.47:0/914855347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e6ca90 0x5563c9e66b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:47.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.984+0000 7fe4bbe00640 1 --2- 172.21.15.47:0/914855347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 0x5563c9e73960 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x5563c9e6d240 tx=0x7fe4ac02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa88e1207700f2b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:47.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.985+0000 7fe4b8c00640 1 -- 172.21.15.47:0/914855347 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac005d50 con 0x5563c9e63d90 2024-03-21T11:43:47.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.985+0000 7fe4b8c00640 1 -- 172.21.15.47:0/914855347 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4ac009780 con 0x5563c9e63d90 2024-03-21T11:43:47.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.985+0000 7fe4b8c00640 1 -- 172.21.15.47:0/914855347 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac034af0 con 0x5563c9e63d90 2024-03-21T11:43:47.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.986+0000 7fe4cda23740 1 -- 172.21.15.47:0/914855347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 msgr2=0x5563c9e73960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:47.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.986+0000 7fe4cda23740 1 --2- 172.21.15.47:0/914855347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 0x5563c9e73960 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x5563c9e6d240 tx=0x7fe4ac02bc30 comp rx=0 tx=0).stop 2024-03-21T11:43:47.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.987+0000 7fe4cda23740 1 -- 172.21.15.47:0/914855347 shutdown_connections 2024-03-21T11:43:47.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.987+0000 7fe4cda23740 1 --2- 172.21.15.47:0/914855347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e6ca90 0x5563c9e66b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.987+0000 7fe4cda23740 1 --2- 172.21.15.47:0/914855347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 0x5563c9e73960 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.987+0000 7fe4cda23740 1 --2- 172.21.15.47:0/914855347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 0x5563c9e66600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.987+0000 7fe4cda23740 1 -- 172.21.15.47:0/914855347 >> 172.21.15.47:0/914855347 conn(0x5563c9e5d5f0 msgr2=0x5563c9e54160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:47.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.987+0000 7fe4cda23740 1 -- 172.21.15.47:0/914855347 shutdown_connections 2024-03-21T11:43:47.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.987+0000 7fe4cda23740 1 -- 172.21.15.47:0/914855347 wait complete. 2024-03-21T11:43:47.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.987+0000 7fe4cda23740 1 -- start start 2024-03-21T11:43:47.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.988+0000 7fe4cda23740 1 Processor -- start 2024-03-21T11:43:47.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.988+0000 7fe4cda23740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 0x5563c9e739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.988+0000 7fe4cda23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 0x5563c9e71fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.988+0000 7fe4cda23740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e6ca90 0x5563c9e72550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.988+0000 7fe4cda23740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a0010c70 con 0x5563c9e62200 2024-03-21T11:43:47.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.988+0000 7fe4cda23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a0010610 con 0x5563c9e63d90 2024-03-21T11:43:47.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.988+0000 7fe4cda23740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a000db10 con 0x5563c9e6ca90 2024-03-21T11:43:47.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.988+0000 7fe4cda23740 10 client.0 _ll_register_callbacks cb 0x7fe4bc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:47.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.989+0000 7fe4c8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e6ca90 0x5563c9e72550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.989+0000 7fe4c8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e6ca90 0x5563c9e72550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59150/0 (socket says 172.21.15.47:59150) 2024-03-21T11:43:47.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.989+0000 7fe4c8e00640 1 -- 172.21.15.47:0/2024103565 learned_addr learned my addr 172.21.15.47:0/2024103565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:47.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.989+0000 7fe4c8e00640 1 --2- 172.21.15.47:0/2024103565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 0x5563c9e739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.990+0000 7fe4bbe00640 1 --2- 172.21.15.47:0/2024103565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 0x5563c9e71fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.990+0000 7fe4c8e00640 1 -- 172.21.15.47:0/2024103565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e6ca90 msgr2=0x5563c9e72550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:47.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.990+0000 7fe4c8e00640 1 --2- 172.21.15.47:0/2024103565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e6ca90 0x5563c9e72550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.990+0000 7fe4c8e00640 1 -- 172.21.15.47:0/2024103565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 msgr2=0x5563c9e71fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:47.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.990+0000 7fe4c8e00640 1 --2- 172.21.15.47:0/2024103565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 0x5563c9e71fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.990+0000 7fe4c8e00640 1 -- 172.21.15.47:0/2024103565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563c9e72d80 con 0x5563c9e62200 2024-03-21T11:43:47.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.991+0000 7fe4c8e00640 1 --2- 172.21.15.47:0/2024103565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 0x5563c9e739a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc006010 tx=0x7fe4bc009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cc2c041d8e81fb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:47.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.991+0000 7fe4b2000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:43:47.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.991+0000 7fe4b2000640 1 -- 172.21.15.47:0/2024103565 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc01d040 con 0x5563c9e62200 2024-03-21T11:43:47.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.991+0000 7fe4b2000640 1 -- 172.21.15.47:0/2024103565 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe4bc003b70 con 0x5563c9e62200 2024-03-21T11:43:47.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.991+0000 7fe4cda23740 1 -- 172.21.15.47:0/2024103565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4ac009780 con 0x5563c9e62200 2024-03-21T11:43:47.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.992+0000 7fe4b2000640 1 -- 172.21.15.47:0/2024103565 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4bc028310 con 0x5563c9e62200 2024-03-21T11:43:47.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.992+0000 7fe4cda23740 10 client.21444 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:47.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.992+0000 7fe4cda23740 1 -- 172.21.15.47:0/2024103565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5563c9ed4830 con 0x5563c9e62200 2024-03-21T11:43:47.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.993+0000 7fe4b2000640 1 -- 172.21.15.47:0/2024103565 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4ac009780 con 0x5563c9e62200 2024-03-21T11:43:47.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.993+0000 7fe4b2000640 1 -- 172.21.15.47:0/2024103565 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe4bc0053e0 con 0x5563c9e62200 2024-03-21T11:43:47.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.993+0000 7fe4cda23740 10 client.21444 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:47.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.993+0000 7fe4cda23740 10 client.21444 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:43:47.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.993+0000 7fe4cda23740 20 client.21444 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:47.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.993+0000 7fe4cda23740 1 --2- 172.21.15.47:0/2024103565 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5563c9ed5050 0x5563c9ed7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:47.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.993+0000 7fe4cda23740 4 client.21444 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:43:47.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.993+0000 7fe4cda23740 1 -- 172.21.15.47:0/2024103565 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5563c9dc5b50 con 0x5563c9ed5050 2024-03-21T11:43:47.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.993+0000 7fe4bbe00640 1 --2- 172.21.15.47:0/2024103565 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5563c9ed5050 0x5563c9ed7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:47.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.994+0000 7fe4bbe00640 1 --2- 172.21.15.47:0/2024103565 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5563c9ed5050 0x5563c9ed7470 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd3b0d12eac65661 server_cookie=c882f82b7add5d3d in_seq=0 out_seq=0 2024-03-21T11:43:47.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.994+0000 7fe4b2000640 10 client.21444 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:43:47.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.994+0000 7fe4b2000640 1 -- 172.21.15.47:0/2024103565 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5563c9dc5b50 con 0x5563c9ed5050 2024-03-21T11:43:47.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.994+0000 7fe4b2000640 10 client.21444 handle_command_reply: tid=0 2024-03-21T11:43:47.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.994+0000 7fe4cda23740 1 client.21444 shutdown 2024-03-21T11:43:47.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.994+0000 7fe4cda23740 1 -- 172.21.15.47:0/2024103565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 msgr2=0x5563c9e739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:47.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.994+0000 7fe4cda23740 1 --2- 172.21.15.47:0/2024103565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 0x5563c9e739a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc006010 tx=0x7fe4bc009740 comp rx=0 tx=0).stop 2024-03-21T11:43:47.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.995+0000 7fe4cda23740 1 -- 172.21.15.47:0/2024103565 shutdown_connections 2024-03-21T11:43:47.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.995+0000 7fe4cda23740 1 --2- 172.21.15.47:0/2024103565 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5563c9ed5050 0x5563c9ed7470 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.995+0000 7fe4cda23740 1 --2- 172.21.15.47:0/2024103565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563c9e6ca90 0x5563c9e72550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.995+0000 7fe4cda23740 1 --2- 172.21.15.47:0/2024103565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563c9e63d90 0x5563c9e71fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.995+0000 7fe4cda23740 1 --2- 172.21.15.47:0/2024103565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563c9e62200 0x5563c9e739a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.995+0000 7fe4cda23740 1 -- 172.21.15.47:0/2024103565 >> 172.21.15.47:0/2024103565 conn(0x5563c9e5d5f0 msgr2=0x5563c9e60850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:47.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.995+0000 7fe4b2000640 0 client.21444 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:43:47.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.995+0000 7fe4cda23740 1 -- 172.21.15.47:0/2024103565 shutdown_connections 2024-03-21T11:43:47.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.995+0000 7fe4cda23740 1 -- 172.21.15.47:0/2024103565 wait complete. 2024-03-21T11:43:47.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.995+0000 7fe4cda23740 20 client.21444 trim_cache size 0 max 16384 2024-03-21T11:43:47.999 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:43:47.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.996+0000 7fe4c9800640 1 -- 172.21.15.47:0/2862090464 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4a80a3020 msgr2=0x7fe4a80a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:47.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.996+0000 7fe4c9800640 1 --2- 172.21.15.47:0/2862090464 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4a80a3020 0x7fe4a80a54e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400c6a0 tx=0x7fe4b409bab0 comp rx=0 tx=0).stop 2024-03-21T11:43:47.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.996+0000 7fe4c9800640 1 -- 172.21.15.47:0/2862090464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4c41037a0 msgr2=0x7fe4c4199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.996+0000 7fe4c9800640 1 --2- 172.21.15.47:0/2862090464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4c41037a0 0x7fe4c4199a90 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e9c0 tx=0x7fe4b400ee80 comp rx=0 tx=0).stop 2024-03-21T11:43:48.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.996+0000 7fe4c9800640 1 -- 172.21.15.47:0/2862090464 shutdown_connections 2024-03-21T11:43:48.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.996+0000 7fe4c9800640 1 --2- 172.21.15.47:0/2862090464 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4a80a3020 0x7fe4a80a54e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.996+0000 7fe4c9800640 1 --2- 172.21.15.47:0/2862090464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4c4105350 0x7fe4c41a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.996+0000 7fe4c9800640 1 --2- 172.21.15.47:0/2862090464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4c4104990 0x7fe4c4199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.996+0000 7fe4c9800640 1 --2- 172.21.15.47:0/2862090464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4c41037a0 0x7fe4c4199a90 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.996+0000 7fe4c9800640 1 -- 172.21.15.47:0/2862090464 >> 172.21.15.47:0/2862090464 conn(0x7fe4c40fed10 msgr2=0x7fe4c4107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.996+0000 7fe4c9800640 1 -- 172.21.15.47:0/2862090464 shutdown_connections 2024-03-21T11:43:48.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:47.996+0000 7fe4c9800640 1 -- 172.21.15.47:0/2862090464 wait complete. 2024-03-21T11:43:48.015 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:43:48.016 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:43:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.269+0000 7f4d3b200640 1 -- 172.21.15.47:0/1008202959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d3410d9c0 msgr2=0x7f4d3410de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.269+0000 7f4d3b200640 1 --2- 172.21.15.47:0/1008202959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d3410d9c0 0x7f4d3410de40 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009be0 tx=0x7f4d2c02bef0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.271+0000 7f4d3b200640 1 -- 172.21.15.47:0/1008202959 shutdown_connections 2024-03-21T11:43:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.271+0000 7f4d3b200640 1 --2- 172.21.15.47:0/1008202959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d3410d9c0 0x7f4d3410de40 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.271+0000 7f4d3b200640 1 --2- 172.21.15.47:0/1008202959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d3410b010 0x7f4d3410d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.271+0000 7f4d3b200640 1 --2- 172.21.15.47:0/1008202959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d341086a0 0x7f4d3410aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.271+0000 7f4d3b200640 1 -- 172.21.15.47:0/1008202959 >> 172.21.15.47:0/1008202959 conn(0x7f4d3406e500 msgr2=0x7f4d34070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.273+0000 7f4d3b200640 1 -- 172.21.15.47:0/1008202959 shutdown_connections 2024-03-21T11:43:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.273+0000 7f4d3b200640 1 -- 172.21.15.47:0/1008202959 wait complete. 2024-03-21T11:43:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.274+0000 7f4d3b200640 1 Processor -- start 2024-03-21T11:43:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.274+0000 7f4d3b200640 1 -- start start 2024-03-21T11:43:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.275+0000 7f4d3b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d341086a0 0x7f4d341a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.275+0000 7f4d3b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d3410b010 0x7f4d341a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.275+0000 7f4d3b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d3410d9c0 0x7f4d341add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.275+0000 7f4d3b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3411c1a0 con 0x7f4d3410b010 2024-03-21T11:43:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.275+0000 7f4d3b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3411c020 con 0x7f4d341086a0 2024-03-21T11:43:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.275+0000 7f4d3b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3411c320 con 0x7f4d3410d9c0 2024-03-21T11:43:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.275+0000 7f4d33e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d341086a0 0x7f4d341a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.275+0000 7f4d39200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d3410d9c0 0x7f4d341add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.275+0000 7f4d33e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d341086a0 0x7f4d341a6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60704/0 (socket says 172.21.15.47:60704) 2024-03-21T11:43:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.275+0000 7f4d33e00640 1 -- 172.21.15.47:0/872662009 learned_addr learned my addr 172.21.15.47:0/872662009 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.276+0000 7f4d2be00640 1 --2- 172.21.15.47:0/872662009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d3410b010 0x7f4d341a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.276+0000 7f4d39200640 1 -- 172.21.15.47:0/872662009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d341086a0 msgr2=0x7f4d341a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.276+0000 7f4d39200640 1 --2- 172.21.15.47:0/872662009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d341086a0 0x7f4d341a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.276+0000 7f4d39200640 1 -- 172.21.15.47:0/872662009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d3410b010 msgr2=0x7f4d341a6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.276+0000 7f4d39200640 1 --2- 172.21.15.47:0/872662009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d3410b010 0x7f4d341a6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.276+0000 7f4d39200640 1 -- 172.21.15.47:0/872662009 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3410ff00 con 0x7f4d3410d9c0 2024-03-21T11:43:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.276+0000 7f4d33e00640 1 --2- 172.21.15.47:0/872662009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d341086a0 0x7f4d341a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.276+0000 7f4d39200640 1 --2- 172.21.15.47:0/872662009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d3410d9c0 0x7f4d341add10 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c036040 tx=0x7f4d2c0337b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.277+0000 7f4d31600640 1 -- 172.21.15.47:0/872662009 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c043020 con 0x7f4d3410d9c0 2024-03-21T11:43:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.277+0000 7f4d31600640 1 -- 172.21.15.47:0/872662009 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d2c003b80 con 0x7f4d3410d9c0 2024-03-21T11:43:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.277+0000 7f4d31600640 1 -- 172.21.15.47:0/872662009 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c034690 con 0x7f4d3410d9c0 2024-03-21T11:43:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.277+0000 7f4d3b200640 1 -- 172.21.15.47:0/872662009 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d34110190 con 0x7f4d3410d9c0 2024-03-21T11:43:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.277+0000 7f4d3b200640 1 -- 172.21.15.47:0/872662009 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d34110700 con 0x7f4d3410d9c0 2024-03-21T11:43:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.284+0000 7f4d31600640 1 -- 172.21.15.47:0/872662009 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4d2c046020 con 0x7f4d3410d9c0 2024-03-21T11:43:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.285+0000 7f4d3dfb6740 1 Processor -- start 2024-03-21T11:43:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.285+0000 7f4d3dfb6740 1 -- start start 2024-03-21T11:43:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.287+0000 7f4d3dfb6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aed810 0x5561a8aedc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.287+0000 7f4d3dfb6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aee9e0 0x5561a8aeee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.287+0000 7f4d3dfb6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aef3a0 0x5561a8af5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.287+0000 7f4d3dfb6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a8a522c0 con 0x5561a8aee9e0 2024-03-21T11:43:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.287+0000 7f4d3dfb6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a89fac90 con 0x5561a8aed810 2024-03-21T11:43:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.287+0000 7f4d3dfb6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a8a4e360 con 0x5561a8aef3a0 2024-03-21T11:43:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.287+0000 7f4d2be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aed810 0x5561a8aedc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.287+0000 7f4d2be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aed810 0x5561a8aedc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60720/0 (socket says 172.21.15.47:60720) 2024-03-21T11:43:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.287+0000 7f4d2be00640 1 -- 172.21.15.47:0/506020491 learned_addr learned my addr 172.21.15.47:0/506020491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.288+0000 7f4d31600640 1 --2- 172.21.15.47:0/872662009 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4d140a2fa0 0x7f4d140a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.288+0000 7f4d2be00640 1 --2- 172.21.15.47:0/506020491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aef3a0 0x5561a8af5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.288+0000 7f4d31600640 1 -- 172.21.15.47:0/872662009 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4d2c0f1f10 con 0x7f4d3410d9c0 2024-03-21T11:43:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.288+0000 7f4d33e00640 1 --2- 172.21.15.47:0/506020491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aee9e0 0x5561a8aeee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.288+0000 7f4d2be00640 1 -- 172.21.15.47:0/506020491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aef3a0 msgr2=0x5561a8af5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.289+0000 7f4d2be00640 1 --2- 172.21.15.47:0/506020491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aef3a0 0x5561a8af5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.289+0000 7f4d2be00640 1 -- 172.21.15.47:0/506020491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aee9e0 msgr2=0x5561a8aeee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.289+0000 7f4d2be00640 1 --2- 172.21.15.47:0/506020491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aee9e0 0x5561a8aeee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.289+0000 7f4d2be00640 1 -- 172.21.15.47:0/506020491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a8a1f080 con 0x5561a8aed810 2024-03-21T11:43:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.289+0000 7f4d33e00640 1 --2- 172.21.15.47:0/506020491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aee9e0 0x5561a8aeee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:48.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.289+0000 7f4d2be00640 1 --2- 172.21.15.47:0/506020491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aed810 0x5561a8aedc30 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0158b0 tx=0x7f4d1c009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=708d49dad5a42600 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.289+0000 7f4d39200640 1 --2- 172.21.15.47:0/872662009 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4d140a2fa0 0x7f4d140a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.290+0000 7f4d39200640 1 --2- 172.21.15.47:0/872662009 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4d140a2fa0 0x7f4d140a5460 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c0ba710 tx=0x7f4d2c004690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.291+0000 7f4d29600640 1 -- 172.21.15.47:0/506020491 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c015e60 con 0x5561a8aed810 2024-03-21T11:43:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.291+0000 7f4d29600640 1 -- 172.21.15.47:0/506020491 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d1c0047a0 con 0x5561a8aed810 2024-03-21T11:43:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.291+0000 7f4d29600640 1 -- 172.21.15.47:0/506020491 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c0106e0 con 0x5561a8aed810 2024-03-21T11:43:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.292+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/506020491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aed810 msgr2=0x5561a8aedc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.292+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/506020491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aed810 0x5561a8aedc30 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0158b0 tx=0x7f4d1c009330 comp rx=0 tx=0).stop 2024-03-21T11:43:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.293+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/506020491 shutdown_connections 2024-03-21T11:43:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.293+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/506020491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aef3a0 0x5561a8af5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.293+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/506020491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aee9e0 0x5561a8aeee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.293+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/506020491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aed810 0x5561a8aedc30 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.293+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/506020491 >> 172.21.15.47:0/506020491 conn(0x5561a8ae8c00 msgr2=0x5561a8ae9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.293+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/506020491 shutdown_connections 2024-03-21T11:43:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.293+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/506020491 wait complete. 2024-03-21T11:43:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.294+0000 7f4d3dfb6740 1 -- start start 2024-03-21T11:43:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.294+0000 7f4d3dfb6740 1 Processor -- start 2024-03-21T11:43:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.295+0000 7f4d3dfb6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aed810 0x5561a8af7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.295+0000 7f4d3dfb6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aee9e0 0x5561a8af8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.295+0000 7f4d3dfb6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aef3a0 0x5561a8aff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.295+0000 7f4d3dfb6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a8af63b0 con 0x5561a8aed810 2024-03-21T11:43:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.295+0000 7f4d3dfb6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a8a0e0d0 con 0x5561a8aee9e0 2024-03-21T11:43:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.295+0000 7f4d3dfb6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a8af6530 con 0x5561a8aef3a0 2024-03-21T11:43:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.295+0000 7f4d3dfb6740 10 client.0 _ll_register_callbacks cb 0x5561a8a12e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.296+0000 7f4d2be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aed810 0x5561a8af7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.296+0000 7f4d33e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aee9e0 0x5561a8af8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.296+0000 7f4d33e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aee9e0 0x5561a8af8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60734/0 (socket says 172.21.15.47:60734) 2024-03-21T11:43:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.296+0000 7f4d33e00640 1 -- 172.21.15.47:0/3400090879 learned_addr learned my addr 172.21.15.47:0/3400090879 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.296+0000 7f4d2be00640 1 --2- 172.21.15.47:0/3400090879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aef3a0 0x5561a8aff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.296+0000 7f4d33e00640 1 -- 172.21.15.47:0/3400090879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aef3a0 msgr2=0x5561a8aff0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.296+0000 7f4d33e00640 1 --2- 172.21.15.47:0/3400090879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aef3a0 0x5561a8aff0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.297+0000 7f4d33e00640 1 -- 172.21.15.47:0/3400090879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aed810 msgr2=0x5561a8af7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.297+0000 7f4d33e00640 1 --2- 172.21.15.47:0/3400090879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aed810 0x5561a8af7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.297+0000 7f4d33e00640 1 -- 172.21.15.47:0/3400090879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561a87663f0 con 0x5561a8aee9e0 2024-03-21T11:43:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.297+0000 7f4d2be00640 1 --2- 172.21.15.47:0/3400090879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aef3a0 0x5561a8aff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.299+0000 7f4d33e00640 1 --2- 172.21.15.47:0/3400090879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aee9e0 0x5561a8af8060 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f4d24012d20 tx=0x7f4d2400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40fbaf4893740db6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.299+0000 7f4d22000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:43:48.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.300+0000 7f4d22000640 1 -- 172.21.15.47:0/3400090879 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24009c10 con 0x5561a8aee9e0 2024-03-21T11:43:48.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.300+0000 7f4d22000640 1 -- 172.21.15.47:0/3400090879 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4d24009db0 con 0x5561a8aee9e0 2024-03-21T11:43:48.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.301+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/3400090879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d1c0047a0 con 0x5561a8aee9e0 2024-03-21T11:43:48.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.302+0000 7f4d22000640 1 -- 172.21.15.47:0/3400090879 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d24010430 con 0x5561a8aee9e0 2024-03-21T11:43:48.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.303+0000 7f4d3dfb6740 10 client.32105 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.303+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/3400090879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5561a88307d0 con 0x5561a8aee9e0 2024-03-21T11:43:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.304+0000 7f4d22000640 1 -- 172.21.15.47:0/3400090879 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d1c0047a0 con 0x5561a8aee9e0 2024-03-21T11:43:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.305+0000 7f4d22000640 1 -- 172.21.15.47:0/3400090879 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4d24026850 con 0x5561a8aee9e0 2024-03-21T11:43:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.305+0000 7f4d3dfb6740 10 client.32105 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.305+0000 7f4d3dfb6740 10 client.32105 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:43:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.305+0000 7f4d3dfb6740 20 client.32105 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:48.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.305+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/3400090879 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5561a8b034e0 0x5561a8b05920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.305+0000 7f4d3dfb6740 4 client.32105 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:48.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.307+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/3400090879 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561a8a5f800 con 0x5561a8b034e0 2024-03-21T11:43:48.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.306+0000 7f4d2be00640 1 --2- 172.21.15.47:0/3400090879 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5561a8b034e0 0x5561a8b05920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.308+0000 7f4d2be00640 1 --2- 172.21.15.47:0/3400090879 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5561a8b034e0 0x5561a8b05920 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8d1e43304723582b server_cookie=6ea59f43bc614dc0 in_seq=0 out_seq=0 2024-03-21T11:43:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.308+0000 7f4d22000640 10 client.32105 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:43:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.309+0000 7f4d22000640 1 -- 172.21.15.47:0/3400090879 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5561a8a5f800 con 0x5561a8b034e0 2024-03-21T11:43:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.309+0000 7f4d22000640 10 client.32105 handle_command_reply: tid=0 2024-03-21T11:43:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.309+0000 7f4d3dfb6740 1 client.32105 shutdown 2024-03-21T11:43:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.310+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/3400090879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aee9e0 msgr2=0x5561a8af8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.310+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/3400090879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aee9e0 0x5561a8af8060 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f4d24012d20 tx=0x7f4d2400f790 comp rx=0 tx=0).stop 2024-03-21T11:43:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.310+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/3400090879 shutdown_connections 2024-03-21T11:43:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.310+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/3400090879 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5561a8b034e0 0x5561a8b05920 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.310+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/3400090879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aef3a0 0x5561a8aff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.310+0000 7f4d22000640 0 client.32105 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:43:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.310+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/3400090879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aee9e0 0x5561a8af8060 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.310+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/3400090879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aed810 0x5561a8af7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.311+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/3400090879 >> 172.21.15.47:0/3400090879 conn(0x5561a8ae8c00 msgr2=0x5561a8ae7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.311+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/3400090879 shutdown_connections 2024-03-21T11:43:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.312+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/3400090879 wait complete. 2024-03-21T11:43:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.312+0000 7f4d3dfb6740 20 client.32105 trim_cache size 0 max 16384 2024-03-21T11:43:48.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:48 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2987146322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:43:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.335+0000 7f4d3dfb6740 1 Processor -- start 2024-03-21T11:43:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.336+0000 7f4d3dfb6740 1 -- start start 2024-03-21T11:43:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.338+0000 7f4d3dfb6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aed810 0x5561a8ae7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.338+0000 7f4d3dfb6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aee9e0 0x5561a8aff060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.338+0000 7f4d3dfb6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aef3a0 0x5561a8b0a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.338+0000 7f4d3dfb6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d341c1690 con 0x5561a8aef3a0 2024-03-21T11:43:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.338+0000 7f4d3dfb6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d341c3010 con 0x5561a8aed810 2024-03-21T11:43:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.338+0000 7f4d3dfb6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d341c1cf0 con 0x5561a8aee9e0 2024-03-21T11:43:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.339+0000 7f4d33e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aee9e0 0x5561a8aff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.339+0000 7f4d33e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aee9e0 0x5561a8aff060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59206/0 (socket says 172.21.15.47:59206) 2024-03-21T11:43:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.339+0000 7f4d33e00640 1 -- 172.21.15.47:0/1686008531 learned_addr learned my addr 172.21.15.47:0/1686008531 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.340+0000 7f4d33e00640 1 -- 172.21.15.47:0/1686008531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aed810 msgr2=0x5561a8ae7d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:43:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.340+0000 7f4d33e00640 1 --2- 172.21.15.47:0/1686008531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aed810 0x5561a8ae7d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.340+0000 7f4d33e00640 1 -- 172.21.15.47:0/1686008531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aef3a0 msgr2=0x5561a8b0a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.340+0000 7f4d33e00640 1 --2- 172.21.15.47:0/1686008531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aef3a0 0x5561a8b0a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.340+0000 7f4d33e00640 1 -- 172.21.15.47:0/1686008531 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a8ae9970 con 0x5561a8aee9e0 2024-03-21T11:43:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.341+0000 7f4d33e00640 1 --2- 172.21.15.47:0/1686008531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aee9e0 0x5561a8aff060 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4d240058c0 tx=0x7f4d24005650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7dfec659cb4ff3a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.341+0000 7f4d28c00640 1 -- 172.21.15.47:0/1686008531 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24009c10 con 0x5561a8aee9e0 2024-03-21T11:43:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.341+0000 7f4d28c00640 1 -- 172.21.15.47:0/1686008531 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d2402b020 con 0x5561a8aee9e0 2024-03-21T11:43:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.341+0000 7f4d28c00640 1 -- 172.21.15.47:0/1686008531 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24026950 con 0x5561a8aee9e0 2024-03-21T11:43:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.343+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/1686008531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aee9e0 msgr2=0x5561a8aff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.343+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/1686008531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aee9e0 0x5561a8aff060 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4d240058c0 tx=0x7f4d24005650 comp rx=0 tx=0).stop 2024-03-21T11:43:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.344+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/1686008531 shutdown_connections 2024-03-21T11:43:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.344+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/1686008531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aef3a0 0x5561a8b0a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.344+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/1686008531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aee9e0 0x5561a8aff060 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.344+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/1686008531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aed810 0x5561a8ae7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.344+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/1686008531 >> 172.21.15.47:0/1686008531 conn(0x5561a8ae8c00 msgr2=0x5561a8ae7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.345+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/1686008531 shutdown_connections 2024-03-21T11:43:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.345+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/1686008531 wait complete. 2024-03-21T11:43:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.345+0000 7f4d3dfb6740 1 -- start start 2024-03-21T11:43:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.346+0000 7f4d3dfb6740 1 Processor -- start 2024-03-21T11:43:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.347+0000 7f4d3dfb6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aed810 0x5561a8afb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.347+0000 7f4d3dfb6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aee9e0 0x5561a8afb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.347+0000 7f4d3dfb6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aef3a0 0x5561a8afe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.347+0000 7f4d3dfb6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d341c29b0 con 0x5561a8aee9e0 2024-03-21T11:43:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.347+0000 7f4d3dfb6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d341c2350 con 0x5561a8aef3a0 2024-03-21T11:43:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.347+0000 7f4d3dfb6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d341182c0 con 0x5561a8aed810 2024-03-21T11:43:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.347+0000 7f4d3dfb6740 10 client.0 _ll_register_callbacks cb 0x7f4d24009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.348+0000 7f4d2be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aed810 0x5561a8afb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.348+0000 7f4d2be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aed810 0x5561a8afb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59218/0 (socket says 172.21.15.47:59218) 2024-03-21T11:43:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.348+0000 7f4d2be00640 1 -- 172.21.15.47:0/2196905203 learned_addr learned my addr 172.21.15.47:0/2196905203 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.348+0000 7f4d2be00640 1 --2- 172.21.15.47:0/2196905203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aef3a0 0x5561a8afe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.348+0000 7f4d33e00640 1 --2- 172.21.15.47:0/2196905203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aee9e0 0x5561a8afb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.348+0000 7f4d2be00640 1 -- 172.21.15.47:0/2196905203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aef3a0 msgr2=0x5561a8afe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.348+0000 7f4d2be00640 1 --2- 172.21.15.47:0/2196905203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aef3a0 0x5561a8afe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.348+0000 7f4d2be00640 1 -- 172.21.15.47:0/2196905203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aee9e0 msgr2=0x5561a8afb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.348+0000 7f4d2be00640 1 --2- 172.21.15.47:0/2196905203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aee9e0 0x5561a8afb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.348+0000 7f4d2be00640 1 -- 172.21.15.47:0/2196905203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561a8afed50 con 0x5561a8aed810 2024-03-21T11:43:48.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.348+0000 7f4d33e00640 1 --2- 172.21.15.47:0/2196905203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aee9e0 0x5561a8afb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:43:48.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.349+0000 7f4d2be00640 1 --2- 172.21.15.47:0/2196905203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aed810 0x5561a8afb200 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c015e30 tx=0x7f4d1c009fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b400fbab09140642 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.349+0000 7f4d22000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:48.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.349+0000 7f4d22000640 1 -- 172.21.15.47:0/2196905203 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c015e60 con 0x5561a8aed810 2024-03-21T11:43:48.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.349+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/2196905203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d2402b020 con 0x5561a8aed810 2024-03-21T11:43:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.349+0000 7f4d22000640 1 -- 172.21.15.47:0/2196905203 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4d1c0099e0 con 0x5561a8aed810 2024-03-21T11:43:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.350+0000 7f4d22000640 1 -- 172.21.15.47:0/2196905203 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d1c024450 con 0x5561a8aed810 2024-03-21T11:43:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.350+0000 7f4d3dfb6740 10 client.32050 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.350+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/2196905203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5561a8b5f7d0 con 0x5561a8aed810 2024-03-21T11:43:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.350+0000 7f4d22000640 1 -- 172.21.15.47:0/2196905203 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d2402b020 con 0x5561a8aed810 2024-03-21T11:43:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.350+0000 7f4d22000640 1 -- 172.21.15.47:0/2196905203 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4d1c0106e0 con 0x5561a8aed810 2024-03-21T11:43:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.351+0000 7f4d3dfb6740 10 client.32050 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.351+0000 7f4d3dfb6740 10 client.32050 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:43:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.351+0000 7f4d3dfb6740 20 client.32050 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.351+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/2196905203 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5561a8b034e0 0x5561a8b61f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.351+0000 7f4d3dfb6740 4 client.32050 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:43:48.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.351+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/2196905203 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5561a8a59b50 con 0x5561a8b034e0 2024-03-21T11:43:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.357+0000 7f4d33e00640 1 --2- 172.21.15.47:0/2196905203 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5561a8b034e0 0x5561a8b61f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.358+0000 7f4d33e00640 1 --2- 172.21.15.47:0/2196905203 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5561a8b034e0 0x5561a8b61f80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4de99d909129d3c9 server_cookie=c07feab91ccc4 in_seq=0 out_seq=0 2024-03-21T11:43:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.358+0000 7f4d22000640 10 client.32050 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:43:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.358+0000 7f4d22000640 1 -- 172.21.15.47:0/2196905203 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5561a8a59b50 con 0x5561a8b034e0 2024-03-21T11:43:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.359+0000 7f4d22000640 10 client.32050 handle_command_reply: tid=0 2024-03-21T11:43:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.359+0000 7f4d3dfb6740 1 client.32050 shutdown 2024-03-21T11:43:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.359+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/2196905203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aed810 msgr2=0x5561a8afb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.359+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/2196905203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aed810 0x5561a8afb200 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c015e30 tx=0x7f4d1c009fb0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.359+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/2196905203 shutdown_connections 2024-03-21T11:43:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.359+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/2196905203 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5561a8b034e0 0x5561a8b61f80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.359+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/2196905203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561a8aef3a0 0x5561a8afe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.359+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/2196905203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561a8aee9e0 0x5561a8afb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.359+0000 7f4d3dfb6740 1 --2- 172.21.15.47:0/2196905203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561a8aed810 0x5561a8afb200 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.359+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/2196905203 >> 172.21.15.47:0/2196905203 conn(0x5561a8ae8c00 msgr2=0x5561a8ae7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.360+0000 7f4d22000640 0 client.32050 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:43:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.360+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/2196905203 shutdown_connections 2024-03-21T11:43:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.360+0000 7f4d3dfb6740 1 -- 172.21.15.47:0/2196905203 wait complete. 2024-03-21T11:43:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.361+0000 7f4d3dfb6740 20 client.32050 trim_cache size 0 max 16384 2024-03-21T11:43:48.363 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:43:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.363+0000 7f4d3b200640 1 -- 172.21.15.47:0/872662009 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4d140a2fa0 msgr2=0x7f4d140a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.363+0000 7f4d3b200640 1 --2- 172.21.15.47:0/872662009 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4d140a2fa0 0x7f4d140a5460 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c0ba710 tx=0x7f4d2c004690 comp rx=0 tx=0).stop 2024-03-21T11:43:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.363+0000 7f4d3b200640 1 -- 172.21.15.47:0/872662009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d3410d9c0 msgr2=0x7f4d341add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.363+0000 7f4d3b200640 1 --2- 172.21.15.47:0/872662009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d3410d9c0 0x7f4d341add10 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c036040 tx=0x7f4d2c0337b0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.363+0000 7f4d3b200640 1 -- 172.21.15.47:0/872662009 shutdown_connections 2024-03-21T11:43:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.363+0000 7f4d3b200640 1 --2- 172.21.15.47:0/872662009 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4d140a2fa0 0x7f4d140a5460 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.363+0000 7f4d3b200640 1 --2- 172.21.15.47:0/872662009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d3410d9c0 0x7f4d341add10 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.363+0000 7f4d3b200640 1 --2- 172.21.15.47:0/872662009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d3410b010 0x7f4d341a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.363+0000 7f4d3b200640 1 --2- 172.21.15.47:0/872662009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d341086a0 0x7f4d341a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.363+0000 7f4d3b200640 1 -- 172.21.15.47:0/872662009 >> 172.21.15.47:0/872662009 conn(0x7f4d3406e500 msgr2=0x7f4d3410b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.363+0000 7f4d3b200640 1 -- 172.21.15.47:0/872662009 shutdown_connections 2024-03-21T11:43:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.363+0000 7f4d3b200640 1 -- 172.21.15.47:0/872662009 wait complete. 2024-03-21T11:43:48.392 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:43:48.393 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:43:48.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:48 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2987146322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:43:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.629+0000 7fe781800640 1 -- 172.21.15.47:0/4292082533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe77c076060 msgr2=0x7fe77c076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.629+0000 7fe781800640 1 --2- 172.21.15.47:0/4292082533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe77c076060 0x7fe77c076480 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009860 tx=0x7fe76c02bb90 comp rx=0 tx=0).stop 2024-03-21T11:43:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.630+0000 7fe781800640 1 -- 172.21.15.47:0/4292082533 shutdown_connections 2024-03-21T11:43:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.630+0000 7fe781800640 1 --2- 172.21.15.47:0/4292082533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c077c40 0x7fe77c0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.630+0000 7fe781800640 1 --2- 172.21.15.47:0/4292082533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe77c0772a0 0x7fe77c077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.630+0000 7fe781800640 1 --2- 172.21.15.47:0/4292082533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe77c076060 0x7fe77c076480 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.630+0000 7fe781800640 1 -- 172.21.15.47:0/4292082533 >> 172.21.15.47:0/4292082533 conn(0x7fe77c0fee90 msgr2=0x7fe77c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.631+0000 7fe781800640 1 -- 172.21.15.47:0/4292082533 shutdown_connections 2024-03-21T11:43:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.631+0000 7fe781800640 1 -- 172.21.15.47:0/4292082533 wait complete. 2024-03-21T11:43:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.632+0000 7fe781800640 1 Processor -- start 2024-03-21T11:43:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.633+0000 7fe781800640 1 -- start start 2024-03-21T11:43:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.633+0000 7fe781800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe77c076060 0x7fe77c110db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.633+0000 7fe781800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c0772a0 0x7fe77c1112f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.633+0000 7fe781800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe77c077c40 0x7fe77c10f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.633+0000 7fe781800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c103d80 con 0x7fe77c077c40 2024-03-21T11:43:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.633+0000 7fe781800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c103c00 con 0x7fe77c0772a0 2024-03-21T11:43:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.633+0000 7fe781800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c103f00 con 0x7fe77c076060 2024-03-21T11:43:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.634+0000 7fe77a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c0772a0 0x7fe77c1112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.634+0000 7fe77a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c0772a0 0x7fe77c1112f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60784/0 (socket says 172.21.15.47:60784) 2024-03-21T11:43:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.634+0000 7fe77a000640 1 -- 172.21.15.47:0/4226942254 learned_addr learned my addr 172.21.15.47:0/4226942254 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.634+0000 7fe77aa00640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe77c076060 0x7fe77c110db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.634+0000 7fe77b400640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe77c077c40 0x7fe77c10f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.635+0000 7fe77a000640 1 -- 172.21.15.47:0/4226942254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe77c076060 msgr2=0x7fe77c110db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.635+0000 7fe77a000640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe77c076060 0x7fe77c110db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.635+0000 7fe77a000640 1 -- 172.21.15.47:0/4226942254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe77c077c40 msgr2=0x7fe77c10f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.635+0000 7fe77a000640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe77c077c40 0x7fe77c10f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.635+0000 7fe77a000640 1 -- 172.21.15.47:0/4226942254 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c10fbb0 con 0x7fe77c0772a0 2024-03-21T11:43:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.635+0000 7fe77b400640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe77c077c40 0x7fe77c10f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.636+0000 7fe77a000640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c0772a0 0x7fe77c1112f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe76800e9c0 tx=0x7fe76800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.638+0000 7fe773400640 1 -- 172.21.15.47:0/4226942254 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76800ccc0 con 0x7fe77c0772a0 2024-03-21T11:43:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.638+0000 7fe773400640 1 -- 172.21.15.47:0/4226942254 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe768005500 con 0x7fe77c0772a0 2024-03-21T11:43:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.638+0000 7fe773400640 1 -- 172.21.15.47:0/4226942254 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768010430 con 0x7fe77c0772a0 2024-03-21T11:43:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.638+0000 7fe781800640 1 -- 172.21.15.47:0/4226942254 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77c10fea0 con 0x7fe77c0772a0 2024-03-21T11:43:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.638+0000 7fe781800640 1 -- 172.21.15.47:0/4226942254 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77c118ff0 con 0x7fe77c0772a0 2024-03-21T11:43:48.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:48 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2987146322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:43:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.644+0000 7fe7845d7740 1 Processor -- start 2024-03-21T11:43:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.648+0000 7fe7845d7740 1 -- start start 2024-03-21T11:43:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.648+0000 7fe7845d7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a739600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.648+0000 7fe7845d7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a739bd0 0x562e5a73e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.649+0000 7fe7845d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a73ed40 0x562e5a7411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.649+0000 7fe77b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a739600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.649+0000 7fe77b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a739600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59242/0 (socket says 172.21.15.47:59242) 2024-03-21T11:43:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.649+0000 7fe77b400640 1 -- 172.21.15.47:0/2040042626 learned_addr learned my addr 172.21.15.47:0/2040042626 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.649+0000 7fe7845d7740 1 -- 172.21.15.47:0/2040042626 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5a6fc3b0 con 0x562e5a739bd0 2024-03-21T11:43:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.649+0000 7fe7845d7740 1 -- 172.21.15.47:0/2040042626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5a69e830 con 0x562e5a73ed40 2024-03-21T11:43:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.649+0000 7fe7845d7740 1 -- 172.21.15.47:0/2040042626 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5a6475d0 con 0x562e5a7391e0 2024-03-21T11:43:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.649+0000 7fe773400640 1 -- 172.21.15.47:0/4226942254 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe7680046c0 con 0x7fe77c0772a0 2024-03-21T11:43:48.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.649+0000 7fe77a000640 1 --2- 172.21.15.47:0/2040042626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a73ed40 0x562e5a7411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.650+0000 7fe77aa00640 1 --2- 172.21.15.47:0/2040042626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a739bd0 0x562e5a73e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.650+0000 7fe77b400640 1 -- 172.21.15.47:0/2040042626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a73ed40 msgr2=0x562e5a7411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.650+0000 7fe77b400640 1 --2- 172.21.15.47:0/2040042626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a73ed40 0x562e5a7411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.650+0000 7fe77b400640 1 -- 172.21.15.47:0/2040042626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a739bd0 msgr2=0x562e5a73e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.650+0000 7fe77b400640 1 --2- 172.21.15.47:0/2040042626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a739bd0 0x562e5a73e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.650+0000 7fe77b400640 1 -- 172.21.15.47:0/2040042626 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e5a65f2c0 con 0x562e5a7391e0 2024-03-21T11:43:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.650+0000 7fe77aa00640 1 --2- 172.21.15.47:0/2040042626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a739bd0 0x562e5a73e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:43:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.650+0000 7fe77b400640 1 --2- 172.21.15.47:0/2040042626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a739600 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fe77400a870 tx=0x7fe774009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ee474e6f5dcb32c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.651+0000 7fe767e00640 1 -- 172.21.15.47:0/2040042626 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe774018020 con 0x562e5a7391e0 2024-03-21T11:43:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.651+0000 7fe767e00640 1 -- 172.21.15.47:0/2040042626 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe774014400 con 0x562e5a7391e0 2024-03-21T11:43:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.652+0000 7fe767e00640 1 -- 172.21.15.47:0/2040042626 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7740135c0 con 0x562e5a7391e0 2024-03-21T11:43:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.653+0000 7fe7845d7740 1 -- 172.21.15.47:0/2040042626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 msgr2=0x562e5a739600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.653+0000 7fe7845d7740 1 --2- 172.21.15.47:0/2040042626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a739600 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fe77400a870 tx=0x7fe774009650 comp rx=0 tx=0).stop 2024-03-21T11:43:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.654+0000 7fe7845d7740 1 -- 172.21.15.47:0/2040042626 shutdown_connections 2024-03-21T11:43:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.658+0000 7fe7845d7740 1 --2- 172.21.15.47:0/2040042626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a73ed40 0x562e5a7411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.658+0000 7fe7845d7740 1 --2- 172.21.15.47:0/2040042626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a739bd0 0x562e5a73e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.658+0000 7fe7845d7740 1 --2- 172.21.15.47:0/2040042626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a739600 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.658+0000 7fe7845d7740 1 -- 172.21.15.47:0/2040042626 >> 172.21.15.47:0/2040042626 conn(0x562e5a734750 msgr2=0x562e5a734b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.658+0000 7fe7845d7740 1 -- 172.21.15.47:0/2040042626 shutdown_connections 2024-03-21T11:43:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.658+0000 7fe7845d7740 1 -- 172.21.15.47:0/2040042626 wait complete. 2024-03-21T11:43:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.658+0000 7fe7845d7740 1 -- start start 2024-03-21T11:43:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.659+0000 7fe7845d7740 1 Processor -- start 2024-03-21T11:43:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.660+0000 7fe7845d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a7391e0 0x562e5a743340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.660+0000 7fe7845d7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a739bd0 0x562e5a743880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.660+0000 7fe7845d7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a74a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.660+0000 7fe773400640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe7600a3130 0x7fe7600a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.660+0000 7fe773400640 1 -- 172.21.15.47:0/4226942254 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe7680cf9f0 con 0x7fe77c0772a0 2024-03-21T11:43:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.660+0000 7fe77aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a739bd0 0x562e5a743880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.660+0000 7fe77b400640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe7600a3130 0x7fe7600a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.660+0000 7fe77aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a739bd0 0x562e5a743880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59258/0 (socket says 172.21.15.47:59258) 2024-03-21T11:43:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.661+0000 7fe77aa00640 1 -- 172.21.15.47:0/427609153 learned_addr learned my addr 172.21.15.47:0/427609153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.661+0000 7fe7845d7740 1 -- 172.21.15.47:0/427609153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5a65aa20 con 0x562e5a73ed40 2024-03-21T11:43:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.661+0000 7fe7845d7740 1 -- 172.21.15.47:0/427609153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5a69a8d0 con 0x562e5a7391e0 2024-03-21T11:43:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.661+0000 7fe7845d7740 1 -- 172.21.15.47:0/427609153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5a741d90 con 0x562e5a739bd0 2024-03-21T11:43:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.661+0000 7fe7845d7740 10 client.0 _ll_register_callbacks cb 0x562e5a6971c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.661+0000 7fe77a000640 1 --2- 172.21.15.47:0/427609153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a74a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.661+0000 7fe77aa00640 1 -- 172.21.15.47:0/427609153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a7391e0 msgr2=0x562e5a743340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.661+0000 7fe77aa00640 1 --2- 172.21.15.47:0/427609153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a7391e0 0x562e5a743340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.661+0000 7fe77aa00640 1 -- 172.21.15.47:0/427609153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 msgr2=0x562e5a74a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.661+0000 7fe77aa00640 1 --2- 172.21.15.47:0/427609153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a74a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.662+0000 7fe77aa00640 1 -- 172.21.15.47:0/427609153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e5a3b3210 con 0x562e5a739bd0 2024-03-21T11:43:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.662+0000 7fe77b400640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe7600a3130 0x7fe7600a55f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe774009fd0 tx=0x7fe774004b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.662+0000 7fe77aa00640 1 --2- 172.21.15.47:0/427609153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a739bd0 0x562e5a743880 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe76c02c1a0 tx=0x7fe76c034530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cc57e16809ca73a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.662+0000 7fe764c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.663+0000 7fe764c00640 1 -- 172.21.15.47:0/427609153 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c039040 con 0x562e5a739bd0 2024-03-21T11:43:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.663+0000 7fe7845d7740 1 -- 172.21.15.47:0/427609153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe774014400 con 0x562e5a739bd0 2024-03-21T11:43:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.663+0000 7fe764c00640 1 -- 172.21.15.47:0/427609153 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe76c004240 con 0x562e5a739bd0 2024-03-21T11:43:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.664+0000 7fe764c00640 1 -- 172.21.15.47:0/427609153 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe76c04e050 con 0x562e5a739bd0 2024-03-21T11:43:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.664+0000 7fe7845d7740 10 client.32062 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.664+0000 7fe7845d7740 1 -- 172.21.15.47:0/427609153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562e5a47f620 con 0x562e5a739bd0 2024-03-21T11:43:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.664+0000 7fe764c00640 1 -- 172.21.15.47:0/427609153 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe774014400 con 0x562e5a739bd0 2024-03-21T11:43:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.665+0000 7fe764c00640 1 -- 172.21.15.47:0/427609153 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe76c02c950 con 0x562e5a739bd0 2024-03-21T11:43:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.665+0000 7fe7845d7740 10 client.32062 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.665+0000 7fe7845d7740 10 client.32062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.665+0000 7fe7845d7740 20 client.32062 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.665+0000 7fe7845d7740 1 --2- 172.21.15.47:0/427609153 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562e5a74f160 0x562e5a751550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.666+0000 7fe7845d7740 4 client.32062 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.666+0000 7fe77a000640 1 --2- 172.21.15.47:0/427609153 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562e5a74f160 0x562e5a751550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.666+0000 7fe7845d7740 1 -- 172.21.15.47:0/427609153 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e5a6abd70 con 0x562e5a74f160 2024-03-21T11:43:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.668+0000 7fe764c00640 10 client.32062 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.668+0000 7fe77a000640 1 --2- 172.21.15.47:0/427609153 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562e5a74f160 0x562e5a751550 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8867b222fe23be94 server_cookie=e0c797931c45c232 in_seq=0 out_seq=0 2024-03-21T11:43:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.669+0000 7fe764c00640 1 -- 172.21.15.47:0/427609153 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562e5a6abd70 con 0x562e5a74f160 2024-03-21T11:43:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.669+0000 7fe764c00640 10 client.32062 handle_command_reply: tid=0 2024-03-21T11:43:48.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.670+0000 7fe7845d7740 1 client.32062 shutdown 2024-03-21T11:43:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.673+0000 7fe7845d7740 1 -- 172.21.15.47:0/427609153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a739bd0 msgr2=0x562e5a743880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.673+0000 7fe7845d7740 1 --2- 172.21.15.47:0/427609153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a739bd0 0x562e5a743880 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe76c02c1a0 tx=0x7fe76c034530 comp rx=0 tx=0).stop 2024-03-21T11:43:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.674+0000 7fe7845d7740 1 -- 172.21.15.47:0/427609153 shutdown_connections 2024-03-21T11:43:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.674+0000 7fe7845d7740 1 --2- 172.21.15.47:0/427609153 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562e5a74f160 0x562e5a751550 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.674+0000 7fe764c00640 0 client.32062 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.674+0000 7fe7845d7740 1 --2- 172.21.15.47:0/427609153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a74a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.674+0000 7fe7845d7740 1 --2- 172.21.15.47:0/427609153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a739bd0 0x562e5a743880 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.674+0000 7fe7845d7740 1 --2- 172.21.15.47:0/427609153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a7391e0 0x562e5a743340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.674+0000 7fe7845d7740 1 -- 172.21.15.47:0/427609153 >> 172.21.15.47:0/427609153 conn(0x562e5a734750 msgr2=0x562e5a733950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.675+0000 7fe7845d7740 1 -- 172.21.15.47:0/427609153 shutdown_connections 2024-03-21T11:43:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.675+0000 7fe7845d7740 1 -- 172.21.15.47:0/427609153 wait complete. 2024-03-21T11:43:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.675+0000 7fe7845d7740 20 client.32062 trim_cache size 0 max 16384 2024-03-21T11:43:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.707+0000 7fe7845d7740 1 Processor -- start 2024-03-21T11:43:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.708+0000 7fe7845d7740 1 -- start start 2024-03-21T11:43:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.708+0000 7fe7845d7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a743340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.708+0000 7fe7845d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a739bd0 0x562e5a743880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.708+0000 7fe7845d7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a74a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.709+0000 7fe77aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a743340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.709+0000 7fe7845d7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c1231f0 con 0x562e5a73ed40 2024-03-21T11:43:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.709+0000 7fe7845d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c1cb890 con 0x562e5a739bd0 2024-03-21T11:43:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.709+0000 7fe7845d7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c123850 con 0x562e5a7391e0 2024-03-21T11:43:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.709+0000 7fe77aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a743340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59270/0 (socket says 172.21.15.47:59270) 2024-03-21T11:43:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.709+0000 7fe77aa00640 1 -- 172.21.15.47:0/2202593155 learned_addr learned my addr 172.21.15.47:0/2202593155 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.709+0000 7fe77b400640 1 --2- 172.21.15.47:0/2202593155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a74a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.709+0000 7fe77a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a739bd0 0x562e5a743880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.709+0000 7fe77b400640 1 -- 172.21.15.47:0/2202593155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 msgr2=0x562e5a743340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.709+0000 7fe77b400640 1 --2- 172.21.15.47:0/2202593155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a743340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.709+0000 7fe77b400640 1 -- 172.21.15.47:0/2202593155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a739bd0 msgr2=0x562e5a743880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.709+0000 7fe77b400640 1 --2- 172.21.15.47:0/2202593155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a739bd0 0x562e5a743880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.709+0000 7fe77b400640 1 -- 172.21.15.47:0/2202593155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e5a73e0c0 con 0x562e5a73ed40 2024-03-21T11:43:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.710+0000 7fe77b400640 1 --2- 172.21.15.47:0/2202593155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a74a880 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe77400a840 tx=0x7fe774009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32258d1d3611581e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.712+0000 7fe767400640 1 -- 172.21.15.47:0/2202593155 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77402e070 con 0x562e5a73ed40 2024-03-21T11:43:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.712+0000 7fe767400640 1 -- 172.21.15.47:0/2202593155 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe774032b30 con 0x562e5a73ed40 2024-03-21T11:43:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.712+0000 7fe767400640 1 -- 172.21.15.47:0/2202593155 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7740287e0 con 0x562e5a73ed40 2024-03-21T11:43:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.713+0000 7fe7845d7740 1 -- 172.21.15.47:0/2202593155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 msgr2=0x562e5a74a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.713+0000 7fe7845d7740 1 --2- 172.21.15.47:0/2202593155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a74a880 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe77400a840 tx=0x7fe774009390 comp rx=0 tx=0).stop 2024-03-21T11:43:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.713+0000 7fe7845d7740 1 -- 172.21.15.47:0/2202593155 shutdown_connections 2024-03-21T11:43:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.713+0000 7fe7845d7740 1 --2- 172.21.15.47:0/2202593155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a74a880 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.713+0000 7fe7845d7740 1 --2- 172.21.15.47:0/2202593155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a739bd0 0x562e5a743880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.713+0000 7fe7845d7740 1 --2- 172.21.15.47:0/2202593155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a743340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.713+0000 7fe7845d7740 1 -- 172.21.15.47:0/2202593155 >> 172.21.15.47:0/2202593155 conn(0x562e5a734750 msgr2=0x562e5a734b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.713+0000 7fe7845d7740 1 -- 172.21.15.47:0/2202593155 shutdown_connections 2024-03-21T11:43:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.714+0000 7fe7845d7740 1 -- 172.21.15.47:0/2202593155 wait complete. 2024-03-21T11:43:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.714+0000 7fe7845d7740 1 -- start start 2024-03-21T11:43:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.714+0000 7fe7845d7740 1 Processor -- start 2024-03-21T11:43:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.715+0000 7fe7845d7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a748660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.715+0000 7fe7845d7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a739bd0 0x562e5a748ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.715+0000 7fe7845d7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a749b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.715+0000 7fe7845d7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c1cb230 con 0x562e5a73ed40 2024-03-21T11:43:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.715+0000 7fe7845d7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c1cacf0 con 0x562e5a739bd0 2024-03-21T11:43:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.715+0000 7fe7845d7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c1213b0 con 0x562e5a7391e0 2024-03-21T11:43:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.715+0000 7fe7845d7740 10 client.0 _ll_register_callbacks cb 0x7fe768005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.715+0000 7fe77b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a749b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.715+0000 7fe77b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a749b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52710/0 (socket says 172.21.15.47:52710) 2024-03-21T11:43:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.715+0000 7fe77b400640 1 -- 172.21.15.47:0/2655774176 learned_addr learned my addr 172.21.15.47:0/2655774176 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.715+0000 7fe77aa00640 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a748660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.716+0000 7fe77a000640 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a739bd0 0x562e5a748ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.716+0000 7fe77b400640 1 -- 172.21.15.47:0/2655774176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 msgr2=0x562e5a748660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.717+0000 7fe77b400640 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a748660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.717+0000 7fe77b400640 1 -- 172.21.15.47:0/2655774176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a739bd0 msgr2=0x562e5a748ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.717+0000 7fe77b400640 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a739bd0 0x562e5a748ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.717+0000 7fe77b400640 1 -- 172.21.15.47:0/2655774176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e5a74a130 con 0x562e5a73ed40 2024-03-21T11:43:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.717+0000 7fe77a000640 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a739bd0 0x562e5a748ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.718+0000 7fe77aa00640 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a748660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:43:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.718+0000 7fe77b400640 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a749b40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe77400f010 tx=0x7fe774032a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8930b1c3130c83a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.718+0000 7fe764c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:43:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.720+0000 7fe764c00640 1 -- 172.21.15.47:0/2655774176 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77402e040 con 0x562e5a73ed40 2024-03-21T11:43:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.720+0000 7fe764c00640 1 -- 172.21.15.47:0/2655774176 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe774028520 con 0x562e5a73ed40 2024-03-21T11:43:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.720+0000 7fe7845d7740 1 -- 172.21.15.47:0/2655774176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe774032b30 con 0x562e5a73ed40 2024-03-21T11:43:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.721+0000 7fe764c00640 1 -- 172.21.15.47:0/2655774176 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe774028740 con 0x562e5a73ed40 2024-03-21T11:43:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.721+0000 7fe7845d7740 10 client.21465 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.721+0000 7fe7845d7740 1 -- 172.21.15.47:0/2655774176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562e5a7ab100 con 0x562e5a73ed40 2024-03-21T11:43:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.723+0000 7fe764c00640 1 -- 172.21.15.47:0/2655774176 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe774032b30 con 0x562e5a73ed40 2024-03-21T11:43:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.723+0000 7fe764c00640 1 -- 172.21.15.47:0/2655774176 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe774026040 con 0x562e5a73ed40 2024-03-21T11:43:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.723+0000 7fe7845d7740 10 client.21465 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.723+0000 7fe7845d7740 10 client.21465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.723+0000 7fe7845d7740 20 client.21465 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.723+0000 7fe7845d7740 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562e5a74f160 0x562e5a7ad800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.723+0000 7fe7845d7740 4 client.21465 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:43:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.723+0000 7fe7845d7740 1 -- 172.21.15.47:0/2655774176 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562e5a6a60c0 con 0x562e5a74f160 2024-03-21T11:43:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.724+0000 7fe77aa00640 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562e5a74f160 0x562e5a7ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.727+0000 7fe77aa00640 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562e5a74f160 0x562e5a7ad800 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8051eb3d033c5fb server_cookie=90aad6203cf6a52f in_seq=0 out_seq=0 2024-03-21T11:43:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:48.727+0000 7fe764c00640 10 client.21465 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.735+0000 7fe705200640 1 -- 172.21.15.47:0/1755663394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7000ffb60 msgr2=0x7fe700100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.736+0000 7fe705200640 1 --2- 172.21.15.47:0/1755663394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7000ffb60 0x7fe700100010 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009ca0 tx=0x7fe6f802c1a0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.737+0000 7fe705200640 1 -- 172.21.15.47:0/1755663394 shutdown_connections 2024-03-21T11:43:48.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.737+0000 7fe705200640 1 --2- 172.21.15.47:0/1755663394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7000ffb60 0x7fe700100010 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.737+0000 7fe705200640 1 --2- 172.21.15.47:0/1755663394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe700103730 0x7fe700105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.738+0000 7fe705200640 1 --2- 172.21.15.47:0/1755663394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe700100e00 0x7fe7001031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.738+0000 7fe705200640 1 -- 172.21.15.47:0/1755663394 >> 172.21.15.47:0/1755663394 conn(0x7fe7000fae60 msgr2=0x7fe7000fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.738+0000 7fe705200640 1 -- 172.21.15.47:0/1755663394 shutdown_connections 2024-03-21T11:43:48.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.739+0000 7fe705200640 1 -- 172.21.15.47:0/1755663394 wait complete. 2024-03-21T11:43:48.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.741+0000 7fe705200640 1 Processor -- start 2024-03-21T11:43:48.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.744+0000 7fe705200640 1 -- start start 2024-03-21T11:43:48.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.745+0000 7fe705200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7000ffb60 0x7fe700199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.745+0000 7fe705200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe700100e00 0x7fe700199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.745+0000 7fe705200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe700103730 0x7fe7001a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.745+0000 7fe705200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70010f5e0 con 0x7fe700103730 2024-03-21T11:43:48.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.745+0000 7fe705200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70010f460 con 0x7fe7000ffb60 2024-03-21T11:43:48.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.745+0000 7fe705200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70010f760 con 0x7fe700100e00 2024-03-21T11:43:48.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.746+0000 7fe6fe800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe700103730 0x7fe7001a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.746+0000 7fe6fe800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe700103730 0x7fe7001a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52730/0 (socket says 172.21.15.47:52730) 2024-03-21T11:43:48.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.746+0000 7fe6fe800640 1 -- 172.21.15.47:0/4121838061 learned_addr learned my addr 172.21.15.47:0/4121838061 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.746+0000 7fe6fde00640 1 --2- 172.21.15.47:0/4121838061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7000ffb60 0x7fe700199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.746+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/4121838061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe700100e00 0x7fe700199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.747+0000 7fe6fe800640 1 -- 172.21.15.47:0/4121838061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe700100e00 msgr2=0x7fe700199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.747+0000 7fe6fe800640 1 --2- 172.21.15.47:0/4121838061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe700100e00 0x7fe700199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.747+0000 7fe6fe800640 1 -- 172.21.15.47:0/4121838061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7000ffb60 msgr2=0x7fe700199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.747+0000 7fe6fe800640 1 --2- 172.21.15.47:0/4121838061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7000ffb60 0x7fe700199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.747+0000 7fe6fe800640 1 -- 172.21.15.47:0/4121838061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70006b0c0 con 0x7fe700103730 2024-03-21T11:43:48.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.747+0000 7fe6fe800640 1 --2- 172.21.15.47:0/4121838061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe700103730 0x7fe7001a0fe0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8005e30 tx=0x7fe6f8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.748+0000 7fe6f6a00640 1 -- 172.21.15.47:0/4121838061 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8039050 con 0x7fe700103730 2024-03-21T11:43:48.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.748+0000 7fe6f6a00640 1 -- 172.21.15.47:0/4121838061 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6f8045070 con 0x7fe700103730 2024-03-21T11:43:48.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.748+0000 7fe6f6a00640 1 -- 172.21.15.47:0/4121838061 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8040450 con 0x7fe700103730 2024-03-21T11:43:48.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.748+0000 7fe705200640 1 -- 172.21.15.47:0/4121838061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70006b350 con 0x7fe700103730 2024-03-21T11:43:48.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.748+0000 7fe705200640 1 -- 172.21.15.47:0/4121838061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70006b7f0 con 0x7fe700103730 2024-03-21T11:43:48.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.754+0000 7fe707f9a740 1 Processor -- start 2024-03-21T11:43:48.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.755+0000 7fe6f6a00640 1 -- 172.21.15.47:0/4121838061 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe6f8034790 con 0x7fe700103730 2024-03-21T11:43:48.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.757+0000 7fe707f9a740 1 -- start start 2024-03-21T11:43:48.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.757+0000 7fe707f9a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622c1810 0x5563622c1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.757+0000 7fe707f9a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c29e0 0x5563622c2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.757+0000 7fe707f9a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c33a0 0x5563622c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.757+0000 7fe707f9a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563622262c0 con 0x5563622c33a0 2024-03-21T11:43:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.757+0000 7fe707f9a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563621cec90 con 0x5563622c29e0 2024-03-21T11:43:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.757+0000 7fe707f9a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556362222360 con 0x5563622c1810 2024-03-21T11:43:48.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.759+0000 7fe6fde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c29e0 0x5563622c2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.759+0000 7fe6fde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c29e0 0x5563622c2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60824/0 (socket says 172.21.15.47:60824) 2024-03-21T11:43:48.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.759+0000 7fe6fde00640 1 -- 172.21.15.47:0/3292845362 learned_addr learned my addr 172.21.15.47:0/3292845362 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.759+0000 7fe6f6a00640 1 --2- 172.21.15.47:0/4121838061 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6e40a3020 0x7fe6e40a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.759+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/3292845362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c33a0 0x5563622c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.760+0000 7fe6f6a00640 1 -- 172.21.15.47:0/4121838061 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe6f80f2400 con 0x7fe700103730 2024-03-21T11:43:48.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.760+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/3292845362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622c1810 0x5563622c1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.760+0000 7fe6f7e00640 1 -- 172.21.15.47:0/3292845362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622c1810 msgr2=0x5563622c1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.760+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/3292845362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622c1810 0x5563622c1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.760+0000 7fe6f7e00640 1 -- 172.21.15.47:0/3292845362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c29e0 msgr2=0x5563622c2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.760+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/3292845362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c29e0 0x5563622c2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.760+0000 7fe6f7e00640 1 -- 172.21.15.47:0/3292845362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563621f3080 con 0x5563622c33a0 2024-03-21T11:43:48.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.762+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/3292845362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c33a0 0x5563622c9c30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x5563622c28e0 tx=0x7fe6f0009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49b9f61b78eb350 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.762+0000 7fe6fe800640 1 --2- 172.21.15.47:0/4121838061 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6e40a3020 0x7fe6e40a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.762+0000 7fe6ef400640 1 -- 172.21.15.47:0/3292845362 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00150f0 con 0x5563622c33a0 2024-03-21T11:43:48.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.762+0000 7fe6ef400640 1 -- 172.21.15.47:0/3292845362 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6f0009b40 con 0x5563622c33a0 2024-03-21T11:43:48.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.763+0000 7fe6ef400640 1 -- 172.21.15.47:0/3292845362 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00047a0 con 0x5563622c33a0 2024-03-21T11:43:48.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.763+0000 7fe707f9a740 1 -- 172.21.15.47:0/3292845362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c33a0 msgr2=0x5563622c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.763+0000 7fe707f9a740 1 --2- 172.21.15.47:0/3292845362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c33a0 0x5563622c9c30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x5563622c28e0 tx=0x7fe6f0009610 comp rx=0 tx=0).stop 2024-03-21T11:43:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.766+0000 7fe6fe800640 1 --2- 172.21.15.47:0/4121838061 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6e40a3020 0x7fe6e40a54e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8004af0 tx=0x7fe6f802c1a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.766+0000 7fe707f9a740 1 -- 172.21.15.47:0/3292845362 shutdown_connections 2024-03-21T11:43:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.766+0000 7fe707f9a740 1 --2- 172.21.15.47:0/3292845362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c33a0 0x5563622c9c30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.766+0000 7fe707f9a740 1 --2- 172.21.15.47:0/3292845362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c29e0 0x5563622c2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.766+0000 7fe707f9a740 1 --2- 172.21.15.47:0/3292845362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622c1810 0x5563622c1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.766+0000 7fe707f9a740 1 -- 172.21.15.47:0/3292845362 >> 172.21.15.47:0/3292845362 conn(0x5563622bcc00 msgr2=0x5563622bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.766+0000 7fe707f9a740 1 -- 172.21.15.47:0/3292845362 shutdown_connections 2024-03-21T11:43:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.766+0000 7fe707f9a740 1 -- 172.21.15.47:0/3292845362 wait complete. 2024-03-21T11:43:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.767+0000 7fe707f9a740 1 -- start start 2024-03-21T11:43:48.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.767+0000 7fe707f9a740 1 Processor -- start 2024-03-21T11:43:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.768+0000 7fe707f9a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c1810 0x5563622cba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.768+0000 7fe707f9a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c29e0 0x5563622cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.768+0000 7fe707f9a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622c33a0 0x5563622d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.768+0000 7fe707f9a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563622ca350 con 0x5563622c29e0 2024-03-21T11:43:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.768+0000 7fe707f9a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563621e20d0 con 0x5563622c1810 2024-03-21T11:43:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.768+0000 7fe707f9a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563622ca4d0 con 0x5563622c33a0 2024-03-21T11:43:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.768+0000 7fe707f9a740 10 client.0 _ll_register_callbacks cb 0x5563621e6e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.769+0000 7fe6f7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c1810 0x5563622cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.769+0000 7fe6f7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c1810 0x5563622cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60826/0 (socket says 172.21.15.47:60826) 2024-03-21T11:43:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.769+0000 7fe6f7e00640 1 -- 172.21.15.47:0/1342535815 learned_addr learned my addr 172.21.15.47:0/1342535815 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.769+0000 7fe6fde00640 1 --2- 172.21.15.47:0/1342535815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c29e0 0x5563622cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.770+0000 7fe6f7e00640 1 -- 172.21.15.47:0/1342535815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622c33a0 msgr2=0x5563622d3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.770+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/1342535815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622c33a0 0x5563622d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.770+0000 7fe6f7e00640 1 -- 172.21.15.47:0/1342535815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c29e0 msgr2=0x5563622cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.770+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/1342535815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c29e0 0x5563622cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.770+0000 7fe6f7e00640 1 -- 172.21.15.47:0/1342535815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556361f3a3f0 con 0x5563622c1810 2024-03-21T11:43:48.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.770+0000 7fe6fde00640 1 --2- 172.21.15.47:0/1342535815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c29e0 0x5563622cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:48.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.771+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/1342535815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c1810 0x5563622cba90 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00093d0 tx=0x7fe6f0009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1ca8759c7d195b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.771+0000 7fe6e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:43:48.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.771+0000 7fe6e3e00640 1 -- 172.21.15.47:0/1342535815 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0026030 con 0x5563622c1810 2024-03-21T11:43:48.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.771+0000 7fe707f9a740 1 -- 172.21.15.47:0/1342535815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6f0009b40 con 0x5563622c1810 2024-03-21T11:43:48.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.771+0000 7fe6e3e00640 1 -- 172.21.15.47:0/1342535815 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe6f0015d90 con 0x5563622c1810 2024-03-21T11:43:48.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.771+0000 7fe6e3e00640 1 -- 172.21.15.47:0/1342535815 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6f0015420 con 0x5563622c1810 2024-03-21T11:43:48.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.772+0000 7fe707f9a740 10 client.32126 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:48.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.772+0000 7fe707f9a740 1 -- 172.21.15.47:0/1342535815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5563620047d0 con 0x5563622c1810 2024-03-21T11:43:48.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.774+0000 7fe6e3e00640 1 -- 172.21.15.47:0/1342535815 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6f0009b40 con 0x5563622c1810 2024-03-21T11:43:48.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.775+0000 7fe6e3e00640 1 -- 172.21.15.47:0/1342535815 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe6f001f040 con 0x5563622c1810 2024-03-21T11:43:48.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.776+0000 7fe707f9a740 10 client.32126 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:48.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.776+0000 7fe707f9a740 10 client.32126 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:43:48.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.776+0000 7fe707f9a740 20 client.32126 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:48.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.776+0000 7fe707f9a740 1 --2- 172.21.15.47:0/1342535815 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5563622d71d0 0x5563622d95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.776+0000 7fe707f9a740 4 client.32126 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:48.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.776+0000 7fe707f9a740 1 -- 172.21.15.47:0/1342535815 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556362233800 con 0x5563622d71d0 2024-03-21T11:43:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.777+0000 7fe6fde00640 1 --2- 172.21.15.47:0/1342535815 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5563622d71d0 0x5563622d95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.779+0000 7fe6fde00640 1 --2- 172.21.15.47:0/1342535815 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5563622d71d0 0x5563622d95c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e32ff77b37955da server_cookie=7ee811a5701e94d in_seq=0 out_seq=0 2024-03-21T11:43:48.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.779+0000 7fe6e3e00640 10 client.32126 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:43:48.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.782+0000 7fe6e3e00640 1 -- 172.21.15.47:0/1342535815 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556362233800 con 0x5563622d71d0 2024-03-21T11:43:48.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.782+0000 7fe6e3e00640 10 client.32126 handle_command_reply: tid=0 2024-03-21T11:43:48.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.783+0000 7fe707f9a740 1 client.32126 shutdown 2024-03-21T11:43:48.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.784+0000 7fe707f9a740 1 -- 172.21.15.47:0/1342535815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c1810 msgr2=0x5563622cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.784+0000 7fe707f9a740 1 --2- 172.21.15.47:0/1342535815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c1810 0x5563622cba90 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00093d0 tx=0x7fe6f0009400 comp rx=0 tx=0).stop 2024-03-21T11:43:48.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.784+0000 7fe707f9a740 1 -- 172.21.15.47:0/1342535815 shutdown_connections 2024-03-21T11:43:48.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.784+0000 7fe707f9a740 1 --2- 172.21.15.47:0/1342535815 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5563622d71d0 0x5563622d95c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.784+0000 7fe707f9a740 1 --2- 172.21.15.47:0/1342535815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622c33a0 0x5563622d3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.784+0000 7fe6e3e00640 0 client.32126 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:43:48.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.784+0000 7fe707f9a740 1 --2- 172.21.15.47:0/1342535815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c29e0 0x5563622cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.784+0000 7fe707f9a740 1 --2- 172.21.15.47:0/1342535815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c1810 0x5563622cba90 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.785+0000 7fe707f9a740 1 -- 172.21.15.47:0/1342535815 >> 172.21.15.47:0/1342535815 conn(0x5563622bcc00 msgr2=0x5563622bbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.785+0000 7fe707f9a740 1 -- 172.21.15.47:0/1342535815 shutdown_connections 2024-03-21T11:43:48.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.785+0000 7fe707f9a740 1 -- 172.21.15.47:0/1342535815 wait complete. 2024-03-21T11:43:48.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.785+0000 7fe707f9a740 20 client.32126 trim_cache size 0 max 16384 2024-03-21T11:43:48.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.812+0000 7fe707f9a740 1 Processor -- start 2024-03-21T11:43:48.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.812+0000 7fe707f9a740 1 -- start start 2024-03-21T11:43:48.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.813+0000 7fe707f9a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622bcc00 0x5563622c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.813+0000 7fe707f9a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622c1810 0x5563622d2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.813+0000 7fe707f9a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c29e0 0x5563622c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.813+0000 7fe707f9a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7001b4900 con 0x5563622c29e0 2024-03-21T11:43:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.813+0000 7fe707f9a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7001b6280 con 0x5563622bcc00 2024-03-21T11:43:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.813+0000 7fe707f9a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7001b4f60 con 0x5563622c1810 2024-03-21T11:43:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.814+0000 7fe6f7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622bcc00 0x5563622c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.814+0000 7fe6f7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622bcc00 0x5563622c5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60834/0 (socket says 172.21.15.47:60834) 2024-03-21T11:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.814+0000 7fe6f7e00640 1 -- 172.21.15.47:0/3307631369 learned_addr learned my addr 172.21.15.47:0/3307631369 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.814+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/3307631369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c29e0 0x5563622c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.815+0000 7fe6fde00640 1 --2- 172.21.15.47:0/3307631369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622c1810 0x5563622d2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.815+0000 7fe6f7e00640 1 -- 172.21.15.47:0/3307631369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622c1810 msgr2=0x5563622d2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.815+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/3307631369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622c1810 0x5563622d2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.815+0000 7fe6f7e00640 1 -- 172.21.15.47:0/3307631369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c29e0 msgr2=0x5563622c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.815+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/3307631369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c29e0 0x5563622c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.815+0000 7fe6f7e00640 1 -- 172.21.15.47:0/3307631369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563622cb6a0 con 0x5563622bcc00 2024-03-21T11:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.815+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/3307631369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622bcc00 0x5563622c5b20 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0000c00 tx=0x7fe6f0015680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9baa31e316bc417b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.815+0000 7fe6eea00640 1 -- 172.21.15.47:0/3307631369 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0026040 con 0x5563622bcc00 2024-03-21T11:43:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.816+0000 7fe6eea00640 1 -- 172.21.15.47:0/3307631369 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6f00150d0 con 0x5563622bcc00 2024-03-21T11:43:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.816+0000 7fe6eea00640 1 -- 172.21.15.47:0/3307631369 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0022da0 con 0x5563622bcc00 2024-03-21T11:43:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.817+0000 7fe707f9a740 1 -- 172.21.15.47:0/3307631369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622bcc00 msgr2=0x5563622c5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.817+0000 7fe707f9a740 1 --2- 172.21.15.47:0/3307631369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622bcc00 0x5563622c5b20 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0000c00 tx=0x7fe6f0015680 comp rx=0 tx=0).stop 2024-03-21T11:43:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.817+0000 7fe707f9a740 1 -- 172.21.15.47:0/3307631369 shutdown_connections 2024-03-21T11:43:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.817+0000 7fe707f9a740 1 --2- 172.21.15.47:0/3307631369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c29e0 0x5563622c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.817+0000 7fe707f9a740 1 --2- 172.21.15.47:0/3307631369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622c1810 0x5563622d2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.817+0000 7fe707f9a740 1 --2- 172.21.15.47:0/3307631369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622bcc00 0x5563622c5b20 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.817+0000 7fe707f9a740 1 -- 172.21.15.47:0/3307631369 >> 172.21.15.47:0/3307631369 conn(0x5563622d9930 msgr2=0x5563622c06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.818+0000 7fe707f9a740 1 -- 172.21.15.47:0/3307631369 shutdown_connections 2024-03-21T11:43:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.819+0000 7fe707f9a740 1 -- 172.21.15.47:0/3307631369 wait complete. 2024-03-21T11:43:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.819+0000 7fe707f9a740 1 -- start start 2024-03-21T11:43:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.819+0000 7fe707f9a740 1 Processor -- start 2024-03-21T11:43:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.820+0000 7fe707f9a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622bcc00 0x5563622cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.820+0000 7fe707f9a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c1810 0x5563622cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.820+0000 7fe707f9a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c29e0 0x5563622d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.820+0000 7fe707f9a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7001b5c20 con 0x5563622c1810 2024-03-21T11:43:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.820+0000 7fe707f9a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7001b55c0 con 0x5563622c29e0 2024-03-21T11:43:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.820+0000 7fe707f9a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7001b2ac0 con 0x5563622bcc00 2024-03-21T11:43:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.820+0000 7fe707f9a740 10 client.0 _ll_register_callbacks cb 0x7fe6e800a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.821+0000 7fe6f7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622bcc00 0x5563622cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.821+0000 7fe6f7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622bcc00 0x5563622cf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:59338/0 (socket says 172.21.15.47:59338) 2024-03-21T11:43:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.821+0000 7fe6f7e00640 1 -- 172.21.15.47:0/495641939 learned_addr learned my addr 172.21.15.47:0/495641939 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.821+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/495641939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c29e0 0x5563622d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.821+0000 7fe6fde00640 1 --2- 172.21.15.47:0/495641939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c1810 0x5563622cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.822+0000 7fe6f7e00640 1 -- 172.21.15.47:0/495641939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c29e0 msgr2=0x5563622d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.822+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/495641939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c29e0 0x5563622d2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.822+0000 7fe6f7e00640 1 -- 172.21.15.47:0/495641939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c1810 msgr2=0x5563622cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.822+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/495641939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c1810 0x5563622cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.822+0000 7fe6f7e00640 1 -- 172.21.15.47:0/495641939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563622d2c10 con 0x5563622bcc00 2024-03-21T11:43:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.823+0000 7fe6f7e00640 1 --2- 172.21.15.47:0/495641939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622bcc00 0x5563622cf0c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0005ff0 tx=0x7fe6f00098e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eb7908ed00a5e17 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.823+0000 7fe6e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.823+0000 7fe6e3e00640 1 -- 172.21.15.47:0/495641939 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0026040 con 0x5563622bcc00 2024-03-21T11:43:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.823+0000 7fe6e3e00640 1 -- 172.21.15.47:0/495641939 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe6f0022d20 con 0x5563622bcc00 2024-03-21T11:43:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.823+0000 7fe707f9a740 1 -- 172.21.15.47:0/495641939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6f00150d0 con 0x5563622bcc00 2024-03-21T11:43:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.824+0000 7fe6e3e00640 1 -- 172.21.15.47:0/495641939 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6f000e6c0 con 0x5563622bcc00 2024-03-21T11:43:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.824+0000 7fe707f9a740 10 client.32077 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.824+0000 7fe707f9a740 1 -- 172.21.15.47:0/495641939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55636233bbe0 con 0x5563622bcc00 2024-03-21T11:43:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.824+0000 7fe6e3e00640 1 -- 172.21.15.47:0/495641939 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6f00150d0 con 0x5563622bcc00 2024-03-21T11:43:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.824+0000 7fe6e3e00640 1 -- 172.21.15.47:0/495641939 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe6f000f040 con 0x5563622bcc00 2024-03-21T11:43:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.824+0000 7fe707f9a740 10 client.32077 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.824+0000 7fe707f9a740 10 client.32077 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:43:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.824+0000 7fe707f9a740 20 client.32077 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.824+0000 7fe707f9a740 1 --2- 172.21.15.47:0/495641939 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5563622c33a0 0x55636233e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.824+0000 7fe707f9a740 4 client.32077 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:43:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.824+0000 7fe707f9a740 1 -- 172.21.15.47:0/495641939 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55636222db50 con 0x5563622c33a0 2024-03-21T11:43:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.826+0000 7fe6fde00640 1 --2- 172.21.15.47:0/495641939 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5563622c33a0 0x55636233e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.827+0000 7fe6fde00640 1 --2- 172.21.15.47:0/495641939 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5563622c33a0 0x55636233e430 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7d12615a8f50e1a server_cookie=77d8fff92fb86dcd in_seq=0 out_seq=0 2024-03-21T11:43:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.827+0000 7fe6e3e00640 10 client.32077 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:43:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.827+0000 7fe6e3e00640 1 -- 172.21.15.47:0/495641939 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55636222db50 con 0x5563622c33a0 2024-03-21T11:43:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.827+0000 7fe6e3e00640 10 client.32077 handle_command_reply: tid=0 2024-03-21T11:43:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.828+0000 7fe707f9a740 1 client.32077 shutdown 2024-03-21T11:43:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.828+0000 7fe707f9a740 1 -- 172.21.15.47:0/495641939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622bcc00 msgr2=0x5563622cf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.828+0000 7fe707f9a740 1 --2- 172.21.15.47:0/495641939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622bcc00 0x5563622cf0c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0005ff0 tx=0x7fe6f00098e0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.828+0000 7fe707f9a740 1 -- 172.21.15.47:0/495641939 shutdown_connections 2024-03-21T11:43:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.828+0000 7fe707f9a740 1 --2- 172.21.15.47:0/495641939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563622c29e0 0x5563622d2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.828+0000 7fe707f9a740 1 --2- 172.21.15.47:0/495641939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5563622c1810 0x5563622cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.828+0000 7fe707f9a740 1 --2- 172.21.15.47:0/495641939 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5563622c33a0 0x55636233e430 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.828+0000 7fe6e3e00640 0 client.32077 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:43:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.828+0000 7fe707f9a740 1 --2- 172.21.15.47:0/495641939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5563622bcc00 0x5563622cf0c0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.828+0000 7fe707f9a740 1 -- 172.21.15.47:0/495641939 >> 172.21.15.47:0/495641939 conn(0x5563622d9930 msgr2=0x5563622bb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.829+0000 7fe707f9a740 1 -- 172.21.15.47:0/495641939 shutdown_connections 2024-03-21T11:43:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.829+0000 7fe707f9a740 1 -- 172.21.15.47:0/495641939 wait complete. 2024-03-21T11:43:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.829+0000 7fe707f9a740 20 client.32077 trim_cache size 0 max 16384 2024-03-21T11:43:48.831 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:43:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.829+0000 7fe705200640 1 -- 172.21.15.47:0/4121838061 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6e40a3020 msgr2=0x7fe6e40a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.829+0000 7fe705200640 1 --2- 172.21.15.47:0/4121838061 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6e40a3020 0x7fe6e40a54e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8004af0 tx=0x7fe6f802c1a0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.829+0000 7fe705200640 1 -- 172.21.15.47:0/4121838061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe700103730 msgr2=0x7fe7001a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.830+0000 7fe705200640 1 --2- 172.21.15.47:0/4121838061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe700103730 0x7fe7001a0fe0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8005e30 tx=0x7fe6f8003ba0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.830+0000 7fe705200640 1 -- 172.21.15.47:0/4121838061 shutdown_connections 2024-03-21T11:43:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.830+0000 7fe705200640 1 --2- 172.21.15.47:0/4121838061 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe6e40a3020 0x7fe6e40a54e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.830+0000 7fe705200640 1 --2- 172.21.15.47:0/4121838061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe700103730 0x7fe7001a0fe0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.830+0000 7fe705200640 1 --2- 172.21.15.47:0/4121838061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe700100e00 0x7fe700199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.830+0000 7fe705200640 1 --2- 172.21.15.47:0/4121838061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7000ffb60 0x7fe700199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.830+0000 7fe705200640 1 -- 172.21.15.47:0/4121838061 >> 172.21.15.47:0/4121838061 conn(0x7fe7000fae60 msgr2=0x7fe700101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.830+0000 7fe705200640 1 -- 172.21.15.47:0/4121838061 shutdown_connections 2024-03-21T11:43:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:48.830+0000 7fe705200640 1 -- 172.21.15.47:0/4121838061 wait complete. 2024-03-21T11:43:48.855 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:43:48.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:43:48.968 INFO:tasks.workunit.client.0.smithi047.stdout: 10 174050 22.92 MB/sec execute 272 sec latency 192.618 ms 2024-03-21T11:43:49.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.187+0000 7f645ce00640 1 -- 172.21.15.47:0/2822609092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6458103260 msgr2=0x7f6458109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.187+0000 7f645ce00640 1 --2- 172.21.15.47:0/2822609092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6458103260 0x7f6458109af0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f6450009ca0 tx=0x7f645002c0f0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.187+0000 7f645ce00640 1 -- 172.21.15.47:0/2822609092 shutdown_connections 2024-03-21T11:43:49.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.187+0000 7f645ce00640 1 --2- 172.21.15.47:0/2822609092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6458103260 0x7f6458109af0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.187+0000 7f645ce00640 1 --2- 172.21.15.47:0/2822609092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64581028a0 0x7f6458102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.187+0000 7f645ce00640 1 --2- 172.21.15.47:0/2822609092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64581016b0 0x7f6458101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.187+0000 7f645ce00640 1 -- 172.21.15.47:0/2822609092 >> 172.21.15.47:0/2822609092 conn(0x7f64580fcc40 msgr2=0x7f64580ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:49.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:49 smithi047 ceph-mon[31351]: pgmap v1014: 129 pgs: 129 active+clean; 581 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-03-21T11:43:49.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.187+0000 7f645ce00640 1 -- 172.21.15.47:0/2822609092 shutdown_connections 2024-03-21T11:43:49.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.187+0000 7f645ce00640 1 -- 172.21.15.47:0/2822609092 wait complete. 2024-03-21T11:43:49.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.191+0000 7f645ce00640 1 Processor -- start 2024-03-21T11:43:49.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.192+0000 7f645ce00640 1 -- start start 2024-03-21T11:43:49.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.192+0000 7f645ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64581016b0 0x7f645819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.192+0000 7f645ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64581028a0 0x7f645819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.192+0000 7f645ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6458103260 0x7f64581a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.192+0000 7f645ce00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645810aa80 con 0x7f64581028a0 2024-03-21T11:43:49.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.192+0000 7f645ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645810a900 con 0x7f6458103260 2024-03-21T11:43:49.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.192+0000 7f645ce00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645810ac00 con 0x7f64581016b0 2024-03-21T11:43:49.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.193+0000 7f6456800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6458103260 0x7f64581a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.193+0000 7f6455e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64581016b0 0x7f645819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.193+0000 7f6456800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6458103260 0x7f64581a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40506/0 (socket says 172.21.15.47:40506) 2024-03-21T11:43:49.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.193+0000 7f6455e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64581016b0 0x7f645819e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37744/0 (socket says 172.21.15.47:37744) 2024-03-21T11:43:49.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.193+0000 7f6456800640 1 -- 172.21.15.47:0/4037440075 learned_addr learned my addr 172.21.15.47:0/4037440075 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:49.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.193+0000 7f6456800640 1 -- 172.21.15.47:0/4037440075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64581016b0 msgr2=0x7f645819e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.193+0000 7f6456800640 1 --2- 172.21.15.47:0/4037440075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64581016b0 0x7f645819e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.193+0000 7f6456800640 1 -- 172.21.15.47:0/4037440075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64581028a0 msgr2=0x7f645819e5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:43:49.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.193+0000 7f6456800640 1 --2- 172.21.15.47:0/4037440075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64581028a0 0x7f645819e5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.193+0000 7f6456800640 1 -- 172.21.15.47:0/4037440075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64581a5c80 con 0x7f6458103260 2024-03-21T11:43:49.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.194+0000 7f6456800640 1 --2- 172.21.15.47:0/4037440075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6458103260 0x7f64581a5610 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6450009bf0 tx=0x7f6450035ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.235+0000 7f644ea00640 1 -- 172.21.15.47:0/4037440075 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450039040 con 0x7f6458103260 2024-03-21T11:43:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.235+0000 7f645ce00640 1 -- 172.21.15.47:0/4037440075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645810ebf0 con 0x7f6458103260 2024-03-21T11:43:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.235+0000 7f645ce00640 1 -- 172.21.15.47:0/4037440075 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645810f1b0 con 0x7f6458103260 2024-03-21T11:43:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.236+0000 7f644ea00640 1 -- 172.21.15.47:0/4037440075 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6450044070 con 0x7f6458103260 2024-03-21T11:43:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.236+0000 7f644ea00640 1 -- 172.21.15.47:0/4037440075 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450004360 con 0x7f6458103260 2024-03-21T11:43:49.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.244+0000 7f644ea00640 1 -- 172.21.15.47:0/4037440075 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6450032690 con 0x7f6458103260 2024-03-21T11:43:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.246+0000 7f645f9f3740 1 Processor -- start 2024-03-21T11:43:49.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.247+0000 7f645f9f3740 1 -- start start 2024-03-21T11:43:49.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.248+0000 7f645f9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef7f810 0x56023ef7fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.248+0000 7f645f9f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef809e0 0x56023ef80e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.248+0000 7f645f9f3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef813a0 0x56023ef87c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.248+0000 7f645f9f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023eee42c0 con 0x56023ef809e0 2024-03-21T11:43:49.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.248+0000 7f645f9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023ee8cc90 con 0x56023ef7f810 2024-03-21T11:43:49.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.248+0000 7f645f9f3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023eee0360 con 0x56023ef813a0 2024-03-21T11:43:49.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.248+0000 7f6455400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef7f810 0x56023ef7fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.248+0000 7f6455400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef7f810 0x56023ef7fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40522/0 (socket says 172.21.15.47:40522) 2024-03-21T11:43:49.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.248+0000 7f6455400640 1 -- 172.21.15.47:0/3845279372 learned_addr learned my addr 172.21.15.47:0/3845279372 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:49.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.248+0000 7f6455400640 1 --2- 172.21.15.47:0/3845279372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef813a0 0x56023ef87c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.249+0000 7f6455400640 1 -- 172.21.15.47:0/3845279372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef813a0 msgr2=0x56023ef87c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:49.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.249+0000 7f6455400640 1 --2- 172.21.15.47:0/3845279372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef813a0 0x56023ef87c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.249+0000 7f6455400640 1 -- 172.21.15.47:0/3845279372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef809e0 msgr2=0x56023ef80e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:43:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.249+0000 7f644ea00640 1 --2- 172.21.15.47:0/4037440075 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f643c0a2fa0 0x7f643c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.250+0000 7f6455e00640 1 --2- 172.21.15.47:0/3845279372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef809e0 0x56023ef80e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.250+0000 7f6455400640 1 --2- 172.21.15.47:0/3845279372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef809e0 0x56023ef80e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.250+0000 7f6455400640 1 -- 172.21.15.47:0/3845279372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56023eeb1080 con 0x56023ef7f810 2024-03-21T11:43:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.250+0000 7f6455400640 1 --2- 172.21.15.47:0/4037440075 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f643c0a2fa0 0x7f643c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.251+0000 7f644ea00640 1 -- 172.21.15.47:0/4037440075 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f64500f18b0 con 0x7f6458103260 2024-03-21T11:43:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.252+0000 7f6455400640 1 --2- 172.21.15.47:0/4037440075 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f643c0a2fa0 0x7f643c0a5460 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6444015cc0 tx=0x7f644400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:49.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.253+0000 7f6455400640 1 --2- 172.21.15.47:0/3845279372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef7f810 0x56023ef7fc30 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6444015370 tx=0x7f6444009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8a1d9805118bd49 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.254+0000 7f6443400640 1 -- 172.21.15.47:0/3845279372 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444026070 con 0x56023ef7f810 2024-03-21T11:43:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.254+0000 7f6443400640 1 -- 172.21.15.47:0/3845279372 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6444004c90 con 0x56023ef7f810 2024-03-21T11:43:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.254+0000 7f6443400640 1 -- 172.21.15.47:0/3845279372 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644401d030 con 0x56023ef7f810 2024-03-21T11:43:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.255+0000 7f645f9f3740 1 -- 172.21.15.47:0/3845279372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef7f810 msgr2=0x56023ef7fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.255+0000 7f645f9f3740 1 --2- 172.21.15.47:0/3845279372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef7f810 0x56023ef7fc30 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6444015370 tx=0x7f6444009e30 comp rx=0 tx=0).stop 2024-03-21T11:43:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.257+0000 7f645f9f3740 1 -- 172.21.15.47:0/3845279372 shutdown_connections 2024-03-21T11:43:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.257+0000 7f645f9f3740 1 --2- 172.21.15.47:0/3845279372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef813a0 0x56023ef87c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.257+0000 7f645f9f3740 1 --2- 172.21.15.47:0/3845279372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef809e0 0x56023ef80e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.257+0000 7f645f9f3740 1 --2- 172.21.15.47:0/3845279372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef7f810 0x56023ef7fc30 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.257+0000 7f645f9f3740 1 -- 172.21.15.47:0/3845279372 >> 172.21.15.47:0/3845279372 conn(0x56023ef7ac00 msgr2=0x56023ef7b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.258+0000 7f645f9f3740 1 -- 172.21.15.47:0/3845279372 shutdown_connections 2024-03-21T11:43:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.258+0000 7f645f9f3740 1 -- 172.21.15.47:0/3845279372 wait complete. 2024-03-21T11:43:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.259+0000 7f645f9f3740 1 -- start start 2024-03-21T11:43:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.260+0000 7f645f9f3740 1 Processor -- start 2024-03-21T11:43:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.261+0000 7f645f9f3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef7f810 0x56023ef89a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.261+0000 7f645f9f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef809e0 0x56023ef89fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.261+0000 7f645f9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef813a0 0x56023ef90fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.261+0000 7f645f9f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023ef88350 con 0x56023ef809e0 2024-03-21T11:43:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.261+0000 7f645f9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023eea00d0 con 0x56023ef813a0 2024-03-21T11:43:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.261+0000 7f645f9f3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023ef884d0 con 0x56023ef7f810 2024-03-21T11:43:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.261+0000 7f645f9f3740 10 client.0 _ll_register_callbacks cb 0x56023eea4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.261+0000 7f6455400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef809e0 0x56023ef89fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.262+0000 7f6455400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef809e0 0x56023ef89fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35702/0 (socket says 172.21.15.47:35702) 2024-03-21T11:43:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.262+0000 7f6455400640 1 -- 172.21.15.47:0/1691258790 learned_addr learned my addr 172.21.15.47:0/1691258790 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.262+0000 7f6455400640 1 -- 172.21.15.47:0/1691258790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef7f810 msgr2=0x56023ef89a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.262+0000 7f6455400640 1 --2- 172.21.15.47:0/1691258790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef7f810 0x56023ef89a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.262+0000 7f6455400640 1 -- 172.21.15.47:0/1691258790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef813a0 msgr2=0x56023ef90fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:43:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.262+0000 7f6455400640 1 --2- 172.21.15.47:0/1691258790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef813a0 0x56023ef90fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.262+0000 7f6455400640 1 -- 172.21.15.47:0/1691258790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56023ebf83f0 con 0x56023ef809e0 2024-03-21T11:43:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.263+0000 7f6455400640 1 --2- 172.21.15.47:0/1691258790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef809e0 0x56023ef89fa0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f644400b010 tx=0x7f6444000f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d00f5239fec126 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.263+0000 7f6437e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:43:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.264+0000 7f6437e00640 1 -- 172.21.15.47:0/1691258790 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444031060 con 0x56023ef809e0 2024-03-21T11:43:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.264+0000 7f6437e00640 1 -- 172.21.15.47:0/1691258790 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f64440095c0 con 0x56023ef809e0 2024-03-21T11:43:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.264+0000 7f645f9f3740 1 -- 172.21.15.47:0/1691258790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6444004c90 con 0x56023ef809e0 2024-03-21T11:43:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.265+0000 7f6437e00640 1 -- 172.21.15.47:0/1691258790 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f644401d050 con 0x56023ef809e0 2024-03-21T11:43:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.265+0000 7f645f9f3740 10 client.21486 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.265+0000 7f645f9f3740 1 -- 172.21.15.47:0/1691258790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56023ecc27d0 con 0x56023ef809e0 2024-03-21T11:43:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.266+0000 7f6437e00640 1 -- 172.21.15.47:0/1691258790 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6444004c90 con 0x56023ef809e0 2024-03-21T11:43:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.266+0000 7f6437e00640 1 -- 172.21.15.47:0/1691258790 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f644402c820 con 0x56023ef809e0 2024-03-21T11:43:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.266+0000 7f645f9f3740 10 client.21486 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.266+0000 7f645f9f3740 10 client.21486 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:43:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.266+0000 7f645f9f3740 20 client.21486 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.266+0000 7f645f9f3740 1 --2- 172.21.15.47:0/1691258790 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56023efa6240 0x56023efa8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.266+0000 7f645f9f3740 4 client.21486 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.266+0000 7f645f9f3740 1 -- 172.21.15.47:0/1691258790 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56023eef1800 con 0x56023efa6240 2024-03-21T11:43:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.267+0000 7f6455e00640 1 --2- 172.21.15.47:0/1691258790 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56023efa6240 0x56023efa8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.268+0000 7f6437e00640 10 client.21486 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:43:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.268+0000 7f6455e00640 1 --2- 172.21.15.47:0/1691258790 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56023efa6240 0x56023efa8630 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=620ff51303073d53 server_cookie=ddda4949d49d8172 in_seq=0 out_seq=0 2024-03-21T11:43:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.269+0000 7f6437e00640 1 -- 172.21.15.47:0/1691258790 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56023eef1800 con 0x56023efa6240 2024-03-21T11:43:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.269+0000 7f6437e00640 10 client.21486 handle_command_reply: tid=0 2024-03-21T11:43:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.269+0000 7f645f9f3740 1 client.21486 shutdown 2024-03-21T11:43:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.270+0000 7f645f9f3740 1 -- 172.21.15.47:0/1691258790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef809e0 msgr2=0x56023ef89fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.270+0000 7f645f9f3740 1 --2- 172.21.15.47:0/1691258790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef809e0 0x56023ef89fa0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f644400b010 tx=0x7f6444000f60 comp rx=0 tx=0).stop 2024-03-21T11:43:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.271+0000 7f645f9f3740 1 -- 172.21.15.47:0/1691258790 shutdown_connections 2024-03-21T11:43:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.271+0000 7f645f9f3740 1 --2- 172.21.15.47:0/1691258790 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56023efa6240 0x56023efa8630 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.271+0000 7f645f9f3740 1 --2- 172.21.15.47:0/1691258790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef813a0 0x56023ef90fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.271+0000 7f6437e00640 0 client.21486 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:43:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.271+0000 7f645f9f3740 1 --2- 172.21.15.47:0/1691258790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef809e0 0x56023ef89fa0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.271+0000 7f645f9f3740 1 --2- 172.21.15.47:0/1691258790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef7f810 0x56023ef89a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.272+0000 7f645f9f3740 1 -- 172.21.15.47:0/1691258790 >> 172.21.15.47:0/1691258790 conn(0x56023ef7ac00 msgr2=0x56023ef79eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.273+0000 7f645f9f3740 1 -- 172.21.15.47:0/1691258790 shutdown_connections 2024-03-21T11:43:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.273+0000 7f645f9f3740 1 -- 172.21.15.47:0/1691258790 wait complete. 2024-03-21T11:43:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.273+0000 7f645f9f3740 20 client.21486 trim_cache size 0 max 16384 2024-03-21T11:43:49.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.312+0000 7f645f9f3740 1 Processor -- start 2024-03-21T11:43:49.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.314+0000 7f645f9f3740 1 -- start start 2024-03-21T11:43:49.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.314+0000 7f645f9f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef7f810 0x56023ef7e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.314+0000 7f645f9f3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef809e0 0x56023ef90fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.315+0000 7f645f9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef813a0 0x56023ef83a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.315+0000 7f645f9f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643800fa10 con 0x56023ef7f810 2024-03-21T11:43:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.315+0000 7f645f9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6438011390 con 0x56023ef813a0 2024-03-21T11:43:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.315+0000 7f645f9f3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6438010070 con 0x56023ef809e0 2024-03-21T11:43:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.315+0000 7f6455400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef809e0 0x56023ef90fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.315+0000 7f6455400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef809e0 0x56023ef90fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37780/0 (socket says 172.21.15.47:37780) 2024-03-21T11:43:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.315+0000 7f6455400640 1 -- 172.21.15.47:0/3482087831 learned_addr learned my addr 172.21.15.47:0/3482087831 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.315+0000 7f6456800640 1 --2- 172.21.15.47:0/3482087831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef813a0 0x56023ef83a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.315+0000 7f6455e00640 1 --2- 172.21.15.47:0/3482087831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef7f810 0x56023ef7e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.316+0000 7f6455400640 1 -- 172.21.15.47:0/3482087831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef813a0 msgr2=0x56023ef83a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:49.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.316+0000 7f6455400640 1 --2- 172.21.15.47:0/3482087831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef813a0 0x56023ef83a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.316+0000 7f6455400640 1 -- 172.21.15.47:0/3482087831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef7f810 msgr2=0x56023ef7e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:49.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.316+0000 7f6455400640 1 --2- 172.21.15.47:0/3482087831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef7f810 0x56023ef7e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.316+0000 7f6455400640 1 -- 172.21.15.47:0/3482087831 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56023efa8c90 con 0x56023ef809e0 2024-03-21T11:43:49.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.316+0000 7f6456800640 1 --2- 172.21.15.47:0/3482087831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef813a0 0x56023ef83a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:49.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.317+0000 7f6455400640 1 --2- 172.21.15.47:0/3482087831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef809e0 0x56023ef90fa0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f644402cbf0 tx=0x7f6444001c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3664893f2b8201d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:49.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.317+0000 7f6442a00640 1 -- 172.21.15.47:0/3482087831 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444031030 con 0x56023ef809e0 2024-03-21T11:43:49.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.317+0000 7f6442a00640 1 -- 172.21.15.47:0/3482087831 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f644401e040 con 0x56023ef809e0 2024-03-21T11:43:49.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.317+0000 7f6442a00640 1 -- 172.21.15.47:0/3482087831 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644403b080 con 0x56023ef809e0 2024-03-21T11:43:49.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.318+0000 7f645f9f3740 1 -- 172.21.15.47:0/3482087831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef809e0 msgr2=0x56023ef90fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.318+0000 7f645f9f3740 1 --2- 172.21.15.47:0/3482087831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef809e0 0x56023ef90fa0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f644402cbf0 tx=0x7f6444001c10 comp rx=0 tx=0).stop 2024-03-21T11:43:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.319+0000 7f645f9f3740 1 -- 172.21.15.47:0/3482087831 shutdown_connections 2024-03-21T11:43:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.319+0000 7f645f9f3740 1 --2- 172.21.15.47:0/3482087831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef813a0 0x56023ef83a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.319+0000 7f645f9f3740 1 --2- 172.21.15.47:0/3482087831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef809e0 0x56023ef90fa0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.319+0000 7f645f9f3740 1 --2- 172.21.15.47:0/3482087831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef7f810 0x56023ef7e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.319+0000 7f645f9f3740 1 -- 172.21.15.47:0/3482087831 >> 172.21.15.47:0/3482087831 conn(0x56023ef7ac00 msgr2=0x56023ef7de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.319+0000 7f645f9f3740 1 -- 172.21.15.47:0/3482087831 shutdown_connections 2024-03-21T11:43:49.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.319+0000 7f645f9f3740 1 -- 172.21.15.47:0/3482087831 wait complete. 2024-03-21T11:43:49.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.319+0000 7f645f9f3740 1 -- start start 2024-03-21T11:43:49.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.320+0000 7f645f9f3740 1 Processor -- start 2024-03-21T11:43:49.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.323+0000 7f645f9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef7f810 0x56023ef8c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.323+0000 7f645f9f3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef809e0 0x56023ef8cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.323+0000 7f645f9f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef813a0 0x56023ef8fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.323+0000 7f645f9f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6438010d30 con 0x56023ef813a0 2024-03-21T11:43:49.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.323+0000 7f645f9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64380106d0 con 0x56023ef7f810 2024-03-21T11:43:49.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.323+0000 7f645f9f3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643800dbd0 con 0x56023ef809e0 2024-03-21T11:43:49.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.323+0000 7f645f9f3740 10 client.0 _ll_register_callbacks cb 0x7f6444024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:49.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.324+0000 7f6455e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef7f810 0x56023ef8c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.324+0000 7f6455400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef809e0 0x56023ef8cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.324+0000 7f6456800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef813a0 0x56023ef8fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.324+0000 7f6455e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef7f810 0x56023ef8c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40552/0 (socket says 172.21.15.47:40552) 2024-03-21T11:43:49.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.324+0000 7f6455e00640 1 -- 172.21.15.47:0/2405352580 learned_addr learned my addr 172.21.15.47:0/2405352580 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:49.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.325+0000 7f6456800640 1 -- 172.21.15.47:0/2405352580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef809e0 msgr2=0x56023ef8cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:49.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.325+0000 7f6456800640 1 --2- 172.21.15.47:0/2405352580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef809e0 0x56023ef8cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.325+0000 7f6456800640 1 -- 172.21.15.47:0/2405352580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef7f810 msgr2=0x56023ef8c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:49.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.325+0000 7f6456800640 1 --2- 172.21.15.47:0/2405352580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef7f810 0x56023ef8c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.325+0000 7f6456800640 1 -- 172.21.15.47:0/2405352580 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56023ef90180 con 0x56023ef813a0 2024-03-21T11:43:49.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.326+0000 7f6456800640 1 --2- 172.21.15.47:0/2405352580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef813a0 0x56023ef8fb90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f64500093f0 tx=0x7f64500bd670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9362e21bb44c12d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:49.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.326+0000 7f6437e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:43:49.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.327+0000 7f6437e00640 1 -- 172.21.15.47:0/2405352580 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500c1da0 con 0x56023ef813a0 2024-03-21T11:43:49.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.327+0000 7f6437e00640 1 -- 172.21.15.47:0/2405352580 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f64500bdeb0 con 0x56023ef813a0 2024-03-21T11:43:49.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.327+0000 7f645f9f3740 1 -- 172.21.15.47:0/2405352580 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f644401e040 con 0x56023ef813a0 2024-03-21T11:43:49.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.328+0000 7f6437e00640 1 -- 172.21.15.47:0/2405352580 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64500c6660 con 0x56023ef813a0 2024-03-21T11:43:49.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.328+0000 7f645f9f3740 10 client.21492 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:49.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.328+0000 7f645f9f3740 1 -- 172.21.15.47:0/2405352580 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56023f001ae0 con 0x56023ef813a0 2024-03-21T11:43:49.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.330+0000 7f6437e00640 1 -- 172.21.15.47:0/2405352580 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f644401e040 con 0x56023ef813a0 2024-03-21T11:43:49.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.330+0000 7f6437e00640 1 -- 172.21.15.47:0/2405352580 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f645003c030 con 0x56023ef813a0 2024-03-21T11:43:49.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.330+0000 7f645f9f3740 10 client.21492 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:49.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.330+0000 7f645f9f3740 10 client.21492 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:43:49.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.330+0000 7f645f9f3740 20 client.21492 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:49.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.330+0000 7f645f9f3740 1 --2- 172.21.15.47:0/2405352580 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56023f002460 0x56023f004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.330+0000 7f645f9f3740 4 client.21492 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:43:49.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.330+0000 7f645f9f3740 1 -- 172.21.15.47:0/2405352580 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56023eeebb50 con 0x56023f002460 2024-03-21T11:43:49.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.331+0000 7f6455e00640 1 --2- 172.21.15.47:0/2405352580 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56023f002460 0x56023f004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.335+0000 7f6455e00640 1 --2- 172.21.15.47:0/2405352580 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56023f002460 0x56023f004880 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7a8e44aa3a77a53 server_cookie=a66448986a25bd73 in_seq=0 out_seq=0 2024-03-21T11:43:49.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.335+0000 7f6437e00640 10 client.21492 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:43:49.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.340+0000 7f6437e00640 1 -- 172.21.15.47:0/2405352580 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56023eeebb50 con 0x56023f002460 2024-03-21T11:43:49.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.340+0000 7f6437e00640 10 client.21492 handle_command_reply: tid=0 2024-03-21T11:43:49.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.341+0000 7f645f9f3740 1 client.21492 shutdown 2024-03-21T11:43:49.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.341+0000 7f645f9f3740 1 -- 172.21.15.47:0/2405352580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef813a0 msgr2=0x56023ef8fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.341+0000 7f645f9f3740 1 --2- 172.21.15.47:0/2405352580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef813a0 0x56023ef8fb90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f64500093f0 tx=0x7f64500bd670 comp rx=0 tx=0).stop 2024-03-21T11:43:49.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.341+0000 7f645f9f3740 1 -- 172.21.15.47:0/2405352580 shutdown_connections 2024-03-21T11:43:49.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.341+0000 7f645f9f3740 1 --2- 172.21.15.47:0/2405352580 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x56023f002460 0x56023f004880 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.341+0000 7f645f9f3740 1 --2- 172.21.15.47:0/2405352580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56023ef813a0 0x56023ef8fb90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.342+0000 7f6437e00640 0 client.21492 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:43:49.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.342+0000 7f645f9f3740 1 --2- 172.21.15.47:0/2405352580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56023ef809e0 0x56023ef8cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.342+0000 7f645f9f3740 1 --2- 172.21.15.47:0/2405352580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56023ef7f810 0x56023ef8c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.342+0000 7f645f9f3740 1 -- 172.21.15.47:0/2405352580 >> 172.21.15.47:0/2405352580 conn(0x56023ef7ac00 msgr2=0x56023ef7e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:49.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.342+0000 7f645f9f3740 1 -- 172.21.15.47:0/2405352580 shutdown_connections 2024-03-21T11:43:49.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.342+0000 7f645f9f3740 1 -- 172.21.15.47:0/2405352580 wait complete. 2024-03-21T11:43:49.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.342+0000 7f645f9f3740 20 client.21492 trim_cache size 0 max 16384 2024-03-21T11:43:49.344 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:43:49.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.343+0000 7f645ce00640 1 -- 172.21.15.47:0/4037440075 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f643c0a2fa0 msgr2=0x7f643c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.343+0000 7f645ce00640 1 --2- 172.21.15.47:0/4037440075 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f643c0a2fa0 0x7f643c0a5460 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6444015cc0 tx=0x7f644400b040 comp rx=0 tx=0).stop 2024-03-21T11:43:49.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.343+0000 7f645ce00640 1 -- 172.21.15.47:0/4037440075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6458103260 msgr2=0x7f64581a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.343+0000 7f645ce00640 1 --2- 172.21.15.47:0/4037440075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6458103260 0x7f64581a5610 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6450009bf0 tx=0x7f6450035ae0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.344+0000 7f645ce00640 1 -- 172.21.15.47:0/4037440075 shutdown_connections 2024-03-21T11:43:49.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.344+0000 7f645ce00640 1 --2- 172.21.15.47:0/4037440075 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f643c0a2fa0 0x7f643c0a5460 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.344+0000 7f645ce00640 1 --2- 172.21.15.47:0/4037440075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6458103260 0x7f64581a5610 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.344+0000 7f645ce00640 1 --2- 172.21.15.47:0/4037440075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64581028a0 0x7f645819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.344+0000 7f645ce00640 1 --2- 172.21.15.47:0/4037440075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64581016b0 0x7f645819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.344+0000 7f645ce00640 1 -- 172.21.15.47:0/4037440075 >> 172.21.15.47:0/4037440075 conn(0x7f64580fcc40 msgr2=0x7f645810d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.344+0000 7f645ce00640 1 -- 172.21.15.47:0/4037440075 shutdown_connections 2024-03-21T11:43:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.344+0000 7f645ce00640 1 -- 172.21.15.47:0/4037440075 wait complete. 2024-03-21T11:43:49.373 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:43:49.373 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:43:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:49 smithi102 ceph-mon[36748]: pgmap v1014: 129 pgs: 129 active+clean; 581 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-03-21T11:43:49.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:49 smithi100 ceph-mon[36245]: pgmap v1014: 129 pgs: 129 active+clean; 581 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-03-21T11:43:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.668+0000 7fe764c00640 1 -- 172.21.15.47:0/2655774176 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562e5a6a60c0 con 0x562e5a74f160 2024-03-21T11:43:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.668+0000 7fe764c00640 10 client.21465 handle_command_reply: tid=0 2024-03-21T11:43:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.668+0000 7fe7845d7740 1 client.21465 shutdown 2024-03-21T11:43:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.669+0000 7fe7845d7740 1 -- 172.21.15.47:0/2655774176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 msgr2=0x562e5a749b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.669+0000 7fe7845d7740 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a749b40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe77400f010 tx=0x7fe774032a50 comp rx=0 tx=0).stop 2024-03-21T11:43:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.669+0000 7fe7845d7740 1 -- 172.21.15.47:0/2655774176 shutdown_connections 2024-03-21T11:43:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.669+0000 7fe7845d7740 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562e5a74f160 0x562e5a7ad800 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.670+0000 7fe7845d7740 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e5a73ed40 0x562e5a749b40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.670+0000 7fe7845d7740 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e5a739bd0 0x562e5a748ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.670+0000 7fe7845d7740 1 --2- 172.21.15.47:0/2655774176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5a7391e0 0x562e5a748660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.670+0000 7fe7845d7740 1 -- 172.21.15.47:0/2655774176 >> 172.21.15.47:0/2655774176 conn(0x562e5a734750 msgr2=0x562e5a743340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.670+0000 7fe764c00640 0 client.21465 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.670+0000 7fe7845d7740 1 -- 172.21.15.47:0/2655774176 shutdown_connections 2024-03-21T11:43:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.671+0000 7fe7845d7740 1 -- 172.21.15.47:0/2655774176 wait complete. 2024-03-21T11:43:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.671+0000 7fe7845d7740 20 client.21465 trim_cache size 0 max 16384 2024-03-21T11:43:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:43:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:43:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:43:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:43:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.672+0000 7fe781800640 1 -- 172.21.15.47:0/4226942254 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe7600a3130 msgr2=0x7fe7600a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.672+0000 7fe781800640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe7600a3130 0x7fe7600a55f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe774009fd0 tx=0x7fe774004b50 comp rx=0 tx=0).stop 2024-03-21T11:43:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.672+0000 7fe781800640 1 -- 172.21.15.47:0/4226942254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c0772a0 msgr2=0x7fe77c1112f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.672+0000 7fe781800640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c0772a0 0x7fe77c1112f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe76800e9c0 tx=0x7fe76800ee80 comp rx=0 tx=0).stop 2024-03-21T11:43:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.673+0000 7fe781800640 1 -- 172.21.15.47:0/4226942254 shutdown_connections 2024-03-21T11:43:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.673+0000 7fe781800640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe7600a3130 0x7fe7600a55f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.673+0000 7fe781800640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe77c077c40 0x7fe77c10f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.673+0000 7fe781800640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77c0772a0 0x7fe77c1112f0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.673+0000 7fe781800640 1 --2- 172.21.15.47:0/4226942254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe77c076060 0x7fe77c110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.673+0000 7fe781800640 1 -- 172.21.15.47:0/4226942254 >> 172.21.15.47:0/4226942254 conn(0x7fe77c0fee90 msgr2=0x7fe77c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.673+0000 7fe781800640 1 -- 172.21.15.47:0/4226942254 shutdown_connections 2024-03-21T11:43:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:43:49.673+0000 7fe781800640 1 -- 172.21.15.47:0/4226942254 wait complete. 2024-03-21T11:43:49.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.690+0000 7f3279a00640 1 -- 172.21.15.47:0/12929711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f327410b860 msgr2=0x7f327410dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.690+0000 7f3279a00640 1 --2- 172.21.15.47:0/12929711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f327410b860 0x7f327410dc50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f326000da00 tx=0x7f326002ff10 comp rx=0 tx=0).stop 2024-03-21T11:43:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.693+0000 7f3279a00640 1 -- 172.21.15.47:0/12929711 shutdown_connections 2024-03-21T11:43:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.693+0000 7f3279a00640 1 --2- 172.21.15.47:0/12929711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327410e1e0 0x7f3274110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.693+0000 7f3279a00640 1 --2- 172.21.15.47:0/12929711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f327410b860 0x7f327410dc50 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.693+0000 7f3279a00640 1 --2- 172.21.15.47:0/12929711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3274108f30 0x7f327410b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.693+0000 7f3279a00640 1 -- 172.21.15.47:0/12929711 >> 172.21.15.47:0/12929711 conn(0x7f327406e500 msgr2=0x7f3274070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.694+0000 7f3279a00640 1 -- 172.21.15.47:0/12929711 shutdown_connections 2024-03-21T11:43:49.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.694+0000 7f3279a00640 1 -- 172.21.15.47:0/12929711 wait complete. 2024-03-21T11:43:49.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.696+0000 7f3279a00640 1 Processor -- start 2024-03-21T11:43:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.698+0000 7f3279a00640 1 -- start start 2024-03-21T11:43:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.699+0000 7f3279a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3274108f30 0x7f327419dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.699+0000 7f3279a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f327410b860 0x7f327419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.699+0000 7f3279a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327410e1e0 0x7f32741a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.699+0000 7f3279a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3274113950 con 0x7f327410e1e0 2024-03-21T11:43:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.699+0000 7f3279a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32741137d0 con 0x7f3274108f30 2024-03-21T11:43:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.699+0000 7f3279a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3274113ad0 con 0x7f327410b860 2024-03-21T11:43:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.699+0000 7f3272000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f327410b860 0x7f327419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.699+0000 7f3272a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3274108f30 0x7f327419dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.699+0000 7f3272000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f327410b860 0x7f327419e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37818/0 (socket says 172.21.15.47:37818) 2024-03-21T11:43:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.699+0000 7f3272000640 1 -- 172.21.15.47:0/1338257007 learned_addr learned my addr 172.21.15.47:0/1338257007 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.699+0000 7f3273400640 1 --2- 172.21.15.47:0/1338257007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327410e1e0 0x7f32741a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.700+0000 7f3272000640 1 -- 172.21.15.47:0/1338257007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3274108f30 msgr2=0x7f327419dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.700+0000 7f3272000640 1 --2- 172.21.15.47:0/1338257007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3274108f30 0x7f327419dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.700+0000 7f3272000640 1 -- 172.21.15.47:0/1338257007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327410e1e0 msgr2=0x7f32741a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.700+0000 7f3272000640 1 --2- 172.21.15.47:0/1338257007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327410e1e0 0x7f32741a5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.700+0000 7f3272000640 1 -- 172.21.15.47:0/1338257007 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32741a5bf0 con 0x7f327410b860 2024-03-21T11:43:49.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.702+0000 7f3272000640 1 --2- 172.21.15.47:0/1338257007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f327410b860 0x7f327419e510 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3260030450 tx=0x7f3260032d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:49.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.703+0000 7f326b400640 1 -- 172.21.15.47:0/1338257007 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260046020 con 0x7f327410b860 2024-03-21T11:43:49.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.703+0000 7f326b400640 1 -- 172.21.15.47:0/1338257007 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3260004040 con 0x7f327410b860 2024-03-21T11:43:49.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.703+0000 7f326b400640 1 -- 172.21.15.47:0/1338257007 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260038c70 con 0x7f327410b860 2024-03-21T11:43:49.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.703+0000 7f3279a00640 1 -- 172.21.15.47:0/1338257007 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32741a5e80 con 0x7f327410b860 2024-03-21T11:43:49.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.703+0000 7f3279a00640 1 -- 172.21.15.47:0/1338257007 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32741a62f0 con 0x7f327410b860 2024-03-21T11:43:49.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.708+0000 7f327c7f6740 1 Processor -- start 2024-03-21T11:43:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.709+0000 7f327c7f6740 1 -- start start 2024-03-21T11:43:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.710+0000 7f327c7f6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3de810 0x55ddde3dec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.710+0000 7f327c7f6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3df9e0 0x55ddde3dfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.710+0000 7f327c7f6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3e03a0 0x55ddde3e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.710+0000 7f327c7f6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddde3432c0 con 0x55ddde3de810 2024-03-21T11:43:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.710+0000 7f327c7f6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddde2ebc90 con 0x55ddde3e03a0 2024-03-21T11:43:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.710+0000 7f327c7f6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddde33f360 con 0x55ddde3df9e0 2024-03-21T11:43:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.710+0000 7f3273400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3de810 0x55ddde3dec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.710+0000 7f3272a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3df9e0 0x55ddde3dfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.711+0000 7f3273400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3de810 0x55ddde3dec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35742/0 (socket says 172.21.15.47:35742) 2024-03-21T11:43:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.711+0000 7f3273400640 1 -- 172.21.15.47:0/3220794853 learned_addr learned my addr 172.21.15.47:0/3220794853 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.711+0000 7f3273400640 1 -- 172.21.15.47:0/3220794853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3df9e0 msgr2=0x55ddde3dfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.711+0000 7f3273400640 1 --2- 172.21.15.47:0/3220794853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3df9e0 0x55ddde3dfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.711+0000 7f3273400640 1 -- 172.21.15.47:0/3220794853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3e03a0 msgr2=0x55ddde3e6c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:43:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.711+0000 7f3273400640 1 --2- 172.21.15.47:0/3220794853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3e03a0 0x55ddde3e6c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.711+0000 7f3273400640 1 -- 172.21.15.47:0/3220794853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddde310080 con 0x55ddde3de810 2024-03-21T11:43:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.712+0000 7f3273400640 1 --2- 172.21.15.47:0/3220794853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3de810 0x55ddde3dec30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f326c008d70 tx=0x7f326c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad3f78df8f605e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:49.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.713+0000 7f325fe00640 1 -- 172.21.15.47:0/3220794853 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c00f830 con 0x55ddde3de810 2024-03-21T11:43:49.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.713+0000 7f325fe00640 1 -- 172.21.15.47:0/3220794853 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f326c00de20 con 0x55ddde3de810 2024-03-21T11:43:49.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.713+0000 7f325fe00640 1 -- 172.21.15.47:0/3220794853 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c00e3b0 con 0x55ddde3de810 2024-03-21T11:43:49.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.715+0000 7f327c7f6740 1 -- 172.21.15.47:0/3220794853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3de810 msgr2=0x55ddde3dec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.715+0000 7f327c7f6740 1 --2- 172.21.15.47:0/3220794853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3de810 0x55ddde3dec30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f326c008d70 tx=0x7f326c00d710 comp rx=0 tx=0).stop 2024-03-21T11:43:49.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.715+0000 7f327c7f6740 1 -- 172.21.15.47:0/3220794853 shutdown_connections 2024-03-21T11:43:49.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.715+0000 7f327c7f6740 1 --2- 172.21.15.47:0/3220794853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3e03a0 0x55ddde3e6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.715+0000 7f327c7f6740 1 --2- 172.21.15.47:0/3220794853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3df9e0 0x55ddde3dfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.715+0000 7f327c7f6740 1 --2- 172.21.15.47:0/3220794853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3de810 0x55ddde3dec30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.715+0000 7f327c7f6740 1 -- 172.21.15.47:0/3220794853 >> 172.21.15.47:0/3220794853 conn(0x55ddde3d9c00 msgr2=0x55ddde3da000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:49.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.715+0000 7f326b400640 1 -- 172.21.15.47:0/1338257007 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3260049020 con 0x7f327410b860 2024-03-21T11:43:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.716+0000 7f327c7f6740 1 -- 172.21.15.47:0/3220794853 shutdown_connections 2024-03-21T11:43:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.717+0000 7f327c7f6740 1 -- 172.21.15.47:0/3220794853 wait complete. 2024-03-21T11:43:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.717+0000 7f327c7f6740 1 -- start start 2024-03-21T11:43:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.717+0000 7f327c7f6740 1 Processor -- start 2024-03-21T11:43:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.718+0000 7f327c7f6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3de810 0x55ddde3e8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.718+0000 7f327c7f6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3df9e0 0x55ddde3e91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.718+0000 7f327c7f6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3ee210 0x55ddde3f0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.718+0000 7f327c7f6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddde3e73b0 con 0x55ddde3ee210 2024-03-21T11:43:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.718+0000 7f327c7f6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddde2ff0d0 con 0x55ddde3de810 2024-03-21T11:43:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.718+0000 7f327c7f6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddde3e7530 con 0x55ddde3df9e0 2024-03-21T11:43:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.718+0000 7f327c7f6740 10 client.0 _ll_register_callbacks cb 0x55ddde303e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.720+0000 7f3272a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3df9e0 0x55ddde3e91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.720+0000 7f3272a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3df9e0 0x55ddde3e91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37844/0 (socket says 172.21.15.47:37844) 2024-03-21T11:43:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.720+0000 7f3272a00640 1 -- 172.21.15.47:0/2247760762 learned_addr learned my addr 172.21.15.47:0/2247760762 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.721+0000 7f3272000640 1 --2- 172.21.15.47:0/2247760762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3ee210 0x55ddde3f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.721+0000 7f3272a00640 1 -- 172.21.15.47:0/2247760762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3de810 msgr2=0x55ddde3e8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.721+0000 7f3272a00640 1 --2- 172.21.15.47:0/2247760762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3de810 0x55ddde3e8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.721+0000 7f326b400640 1 --2- 172.21.15.47:0/1338257007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32580a2eb0 0x7f32580a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.721+0000 7f3272a00640 1 -- 172.21.15.47:0/2247760762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3ee210 msgr2=0x55ddde3f0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.721+0000 7f3272a00640 1 --2- 172.21.15.47:0/2247760762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3ee210 0x55ddde3f0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.721+0000 7f3272a00640 1 -- 172.21.15.47:0/2247760762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddde0573f0 con 0x55ddde3df9e0 2024-03-21T11:43:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.721+0000 7f3272000640 1 --2- 172.21.15.47:0/2247760762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3ee210 0x55ddde3f0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:43:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.722+0000 7f326b400640 1 -- 172.21.15.47:0/1338257007 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f32600f5c90 con 0x7f327410b860 2024-03-21T11:43:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.722+0000 7f3273400640 1 --2- 172.21.15.47:0/1338257007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32580a2eb0 0x7f32580a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.722+0000 7f3272a00640 1 --2- 172.21.15.47:0/2247760762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3df9e0 0x55ddde3e91e0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3264012860 tx=0x7f3264012d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=843806456369721c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.722+0000 7f325cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.723+0000 7f325cc00640 1 -- 172.21.15.47:0/2247760762 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326401f060 con 0x55ddde3df9e0 2024-03-21T11:43:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.723+0000 7f325cc00640 1 -- 172.21.15.47:0/2247760762 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3264010d60 con 0x55ddde3df9e0 2024-03-21T11:43:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.723+0000 7f327c7f6740 1 -- 172.21.15.47:0/2247760762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f326c00de20 con 0x55ddde3df9e0 2024-03-21T11:43:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.723+0000 7f3273400640 1 --2- 172.21.15.47:0/1338257007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32580a2eb0 0x7f32580a5370 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f326c008340 tx=0x7f326c008270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.724+0000 7f325cc00640 1 -- 172.21.15.47:0/2247760762 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f326401a430 con 0x55ddde3df9e0 2024-03-21T11:43:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.724+0000 7f327c7f6740 10 client.32101 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.724+0000 7f327c7f6740 1 -- 172.21.15.47:0/2247760762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ddde1217d0 con 0x55ddde3df9e0 2024-03-21T11:43:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.725+0000 7f325cc00640 1 -- 172.21.15.47:0/2247760762 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f326c00de20 con 0x55ddde3df9e0 2024-03-21T11:43:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.725+0000 7f325cc00640 1 -- 172.21.15.47:0/2247760762 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3264023600 con 0x55ddde3df9e0 2024-03-21T11:43:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.725+0000 7f327c7f6740 10 client.32101 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.725+0000 7f327c7f6740 10 client.32101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.725+0000 7f327c7f6740 20 client.32101 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.725+0000 7f327c7f6740 1 --2- 172.21.15.47:0/2247760762 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ddde416bc0 0x55ddde419000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.725+0000 7f327c7f6740 4 client.32101 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.725+0000 7f327c7f6740 1 -- 172.21.15.47:0/2247760762 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddde350800 con 0x55ddde416bc0 2024-03-21T11:43:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.725+0000 7f3272000640 1 --2- 172.21.15.47:0/2247760762 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ddde416bc0 0x55ddde419000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.727+0000 7f3272000640 1 --2- 172.21.15.47:0/2247760762 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ddde416bc0 0x55ddde419000 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=270d2e21886b258a server_cookie=f95f96447f8a4f3b in_seq=0 out_seq=0 2024-03-21T11:43:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.728+0000 7f325cc00640 10 client.32101 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:49.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.729+0000 7f325cc00640 1 -- 172.21.15.47:0/2247760762 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ddde350800 con 0x55ddde416bc0 2024-03-21T11:43:49.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.729+0000 7f325cc00640 10 client.32101 handle_command_reply: tid=0 2024-03-21T11:43:49.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.730+0000 7f327c7f6740 1 client.32101 shutdown 2024-03-21T11:43:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.731+0000 7f327c7f6740 1 -- 172.21.15.47:0/2247760762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3df9e0 msgr2=0x55ddde3e91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.731+0000 7f327c7f6740 1 --2- 172.21.15.47:0/2247760762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3df9e0 0x55ddde3e91e0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3264012860 tx=0x7f3264012d20 comp rx=0 tx=0).stop 2024-03-21T11:43:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.731+0000 7f327c7f6740 1 -- 172.21.15.47:0/2247760762 shutdown_connections 2024-03-21T11:43:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.731+0000 7f327c7f6740 1 --2- 172.21.15.47:0/2247760762 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ddde416bc0 0x55ddde419000 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.731+0000 7f327c7f6740 1 --2- 172.21.15.47:0/2247760762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3ee210 0x55ddde3f0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.731+0000 7f327c7f6740 1 --2- 172.21.15.47:0/2247760762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3df9e0 0x55ddde3e91e0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.731+0000 7f327c7f6740 1 --2- 172.21.15.47:0/2247760762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3de810 0x55ddde3e8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.731+0000 7f325cc00640 0 client.32101 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.731+0000 7f327c7f6740 1 -- 172.21.15.47:0/2247760762 >> 172.21.15.47:0/2247760762 conn(0x55ddde3d9c00 msgr2=0x55ddde3d8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.732+0000 7f327c7f6740 1 -- 172.21.15.47:0/2247760762 shutdown_connections 2024-03-21T11:43:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.732+0000 7f327c7f6740 1 -- 172.21.15.47:0/2247760762 wait complete. 2024-03-21T11:43:49.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.732+0000 7f327c7f6740 20 client.32101 trim_cache size 0 max 16384 2024-03-21T11:43:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.769+0000 7f327c7f6740 1 Processor -- start 2024-03-21T11:43:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.770+0000 7f327c7f6740 1 -- start start 2024-03-21T11:43:49.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.770+0000 7f327c7f6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3de810 0x55ddde3d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.770+0000 7f327c7f6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3df9e0 0x55ddde3e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.770+0000 7f327c7f6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3ee210 0x55ddde3e3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.770+0000 7f327c7f6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325400f990 con 0x55ddde3ee210 2024-03-21T11:43:49.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.770+0000 7f327c7f6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3254011310 con 0x55ddde3df9e0 2024-03-21T11:43:49.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.770+0000 7f327c7f6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325400fff0 con 0x55ddde3de810 2024-03-21T11:43:49.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.770+0000 7f3272000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3df9e0 0x55ddde3e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.770+0000 7f3272a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3de810 0x55ddde3d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.770+0000 7f3272000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3df9e0 0x55ddde3e2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40590/0 (socket says 172.21.15.47:40590) 2024-03-21T11:43:49.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.770+0000 7f3272000640 1 -- 172.21.15.47:0/595181011 learned_addr learned my addr 172.21.15.47:0/595181011 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:49.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.771+0000 7f3272a00640 1 -- 172.21.15.47:0/595181011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3df9e0 msgr2=0x55ddde3e2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:49.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.771+0000 7f3272a00640 1 --2- 172.21.15.47:0/595181011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3df9e0 0x55ddde3e2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.771+0000 7f3272a00640 1 -- 172.21.15.47:0/595181011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3ee210 msgr2=0x55ddde3e3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.771+0000 7f3272a00640 1 --2- 172.21.15.47:0/595181011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3ee210 0x55ddde3e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.771+0000 7f3272a00640 1 -- 172.21.15.47:0/595181011 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddde3e6710 con 0x55ddde3de810 2024-03-21T11:43:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.771+0000 7f3272000640 1 --2- 172.21.15.47:0/595181011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3df9e0 0x55ddde3e2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.771+0000 7f3272a00640 1 --2- 172.21.15.47:0/595181011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3de810 0x55ddde3d8d30 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3264023a20 tx=0x7f3264012450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1c4fb9b2fb4d5ef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.772+0000 7f325f400640 1 -- 172.21.15.47:0/595181011 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264029080 con 0x55ddde3de810 2024-03-21T11:43:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.772+0000 7f325f400640 1 -- 172.21.15.47:0/595181011 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3264023c10 con 0x55ddde3de810 2024-03-21T11:43:49.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.772+0000 7f325f400640 1 -- 172.21.15.47:0/595181011 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264010b60 con 0x55ddde3de810 2024-03-21T11:43:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.773+0000 7f327c7f6740 1 -- 172.21.15.47:0/595181011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3de810 msgr2=0x55ddde3d8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.773+0000 7f327c7f6740 1 --2- 172.21.15.47:0/595181011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3de810 0x55ddde3d8d30 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3264023a20 tx=0x7f3264012450 comp rx=0 tx=0).stop 2024-03-21T11:43:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.773+0000 7f327c7f6740 1 -- 172.21.15.47:0/595181011 shutdown_connections 2024-03-21T11:43:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.773+0000 7f327c7f6740 1 --2- 172.21.15.47:0/595181011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3ee210 0x55ddde3e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.773+0000 7f327c7f6740 1 --2- 172.21.15.47:0/595181011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3df9e0 0x55ddde3e2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.773+0000 7f327c7f6740 1 --2- 172.21.15.47:0/595181011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3de810 0x55ddde3d8d30 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.773+0000 7f327c7f6740 1 -- 172.21.15.47:0/595181011 >> 172.21.15.47:0/595181011 conn(0x55ddde3d9c00 msgr2=0x55ddde3d8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:49.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.774+0000 7f327c7f6740 1 -- 172.21.15.47:0/595181011 shutdown_connections 2024-03-21T11:43:49.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.774+0000 7f327c7f6740 1 -- 172.21.15.47:0/595181011 wait complete. 2024-03-21T11:43:49.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.774+0000 7f327c7f6740 1 -- start start 2024-03-21T11:43:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.775+0000 7f327c7f6740 1 Processor -- start 2024-03-21T11:43:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.776+0000 7f327c7f6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3de810 0x55ddde425d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.776+0000 7f327c7f6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3df9e0 0x55ddde426250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.776+0000 7f327c7f6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3ee210 0x55ddde47a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.776+0000 7f327c7f6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3254010cb0 con 0x55ddde3df9e0 2024-03-21T11:43:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.776+0000 7f327c7f6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3254010650 con 0x55ddde3de810 2024-03-21T11:43:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.776+0000 7f327c7f6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325400db50 con 0x55ddde3ee210 2024-03-21T11:43:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.776+0000 7f327c7f6740 10 client.0 _ll_register_callbacks cb 0x7f326400a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.777+0000 7f3272a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3de810 0x55ddde425d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.777+0000 7f3273400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3ee210 0x55ddde47a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.777+0000 7f3273400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3ee210 0x55ddde47a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37858/0 (socket says 172.21.15.47:37858) 2024-03-21T11:43:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.777+0000 7f3273400640 1 -- 172.21.15.47:0/2654092924 learned_addr learned my addr 172.21.15.47:0/2654092924 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.777+0000 7f3272000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3df9e0 0x55ddde426250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.778+0000 7f3272a00640 1 -- 172.21.15.47:0/2654092924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3ee210 msgr2=0x55ddde47a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.778+0000 7f3272a00640 1 --2- 172.21.15.47:0/2654092924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3ee210 0x55ddde47a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.778+0000 7f3272a00640 1 -- 172.21.15.47:0/2654092924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3df9e0 msgr2=0x55ddde426250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.778+0000 7f3272a00640 1 --2- 172.21.15.47:0/2654092924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3df9e0 0x55ddde426250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.778+0000 7f3272a00640 1 -- 172.21.15.47:0/2654092924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddde429320 con 0x55ddde3de810 2024-03-21T11:43:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.779+0000 7f3272a00640 1 --2- 172.21.15.47:0/2654092924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3de810 0x55ddde425d10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f32640239a0 tx=0x7f326400a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c7b1fb5fcd3148a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.779+0000 7f325cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:43:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.779+0000 7f325cc00640 1 -- 172.21.15.47:0/2654092924 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264029070 con 0x55ddde3de810 2024-03-21T11:43:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.779+0000 7f325cc00640 1 -- 172.21.15.47:0/2654092924 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3264005630 con 0x55ddde3de810 2024-03-21T11:43:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.779+0000 7f327c7f6740 1 -- 172.21.15.47:0/2654092924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3264023c10 con 0x55ddde3de810 2024-03-21T11:43:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.780+0000 7f325cc00640 1 -- 172.21.15.47:0/2654092924 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f326403d050 con 0x55ddde3de810 2024-03-21T11:43:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.780+0000 7f327c7f6740 10 client.32156 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.780+0000 7f327c7f6740 1 -- 172.21.15.47:0/2654092924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ddde47b5b0 con 0x55ddde3de810 2024-03-21T11:43:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.780+0000 7f325cc00640 1 -- 172.21.15.47:0/2654092924 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3264023c10 con 0x55ddde3de810 2024-03-21T11:43:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.780+0000 7f325cc00640 1 -- 172.21.15.47:0/2654092924 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f32640245e0 con 0x55ddde3de810 2024-03-21T11:43:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.781+0000 7f327c7f6740 10 client.32156 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.781+0000 7f327c7f6740 10 client.32156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.781+0000 7f327c7f6740 20 client.32156 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.781+0000 7f327c7f6740 1 --2- 172.21.15.47:0/2654092924 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ddde47bd40 0x55ddde47e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.781+0000 7f327c7f6740 4 client.32156 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:43:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.781+0000 7f327c7f6740 1 -- 172.21.15.47:0/2654092924 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ddde34ab50 con 0x55ddde47bd40 2024-03-21T11:43:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.781+0000 7f3272000640 1 --2- 172.21.15.47:0/2654092924 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ddde47bd40 0x55ddde47e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:49.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.783+0000 7f3272000640 1 --2- 172.21.15.47:0/2654092924 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ddde47bd40 0x55ddde47e160 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b46f77493245de05 server_cookie=4653a4d8a189bed1 in_seq=0 out_seq=0 2024-03-21T11:43:49.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.783+0000 7f325cc00640 10 client.32156 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.783+0000 7f325cc00640 1 -- 172.21.15.47:0/2654092924 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ddde34ab50 con 0x55ddde47bd40 2024-03-21T11:43:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.783+0000 7f325cc00640 10 client.32156 handle_command_reply: tid=0 2024-03-21T11:43:49.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.783+0000 7f327c7f6740 1 client.32156 shutdown 2024-03-21T11:43:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.785+0000 7f327c7f6740 1 -- 172.21.15.47:0/2654092924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3de810 msgr2=0x55ddde425d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.785+0000 7f327c7f6740 1 --2- 172.21.15.47:0/2654092924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3de810 0x55ddde425d10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f32640239a0 tx=0x7f326400a430 comp rx=0 tx=0).stop 2024-03-21T11:43:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.785+0000 7f327c7f6740 1 -- 172.21.15.47:0/2654092924 shutdown_connections 2024-03-21T11:43:49.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.785+0000 7f327c7f6740 1 --2- 172.21.15.47:0/2654092924 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ddde47bd40 0x55ddde47e160 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.786+0000 7f325cc00640 0 client.32156 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.786+0000 7f327c7f6740 1 --2- 172.21.15.47:0/2654092924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ddde3ee210 0x55ddde47a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.786+0000 7f327c7f6740 1 --2- 172.21.15.47:0/2654092924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ddde3df9e0 0x55ddde426250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.786+0000 7f327c7f6740 1 --2- 172.21.15.47:0/2654092924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddde3de810 0x55ddde425d10 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.786+0000 7f327c7f6740 1 -- 172.21.15.47:0/2654092924 >> 172.21.15.47:0/2654092924 conn(0x55ddde3d9c00 msgr2=0x55ddde3d8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.787+0000 7f327c7f6740 1 -- 172.21.15.47:0/2654092924 shutdown_connections 2024-03-21T11:43:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.788+0000 7f327c7f6740 1 -- 172.21.15.47:0/2654092924 wait complete. 2024-03-21T11:43:49.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.788+0000 7f327c7f6740 20 client.32156 trim_cache size 0 max 16384 2024-03-21T11:43:49.789 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:43:49.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.789+0000 7f3279a00640 1 -- 172.21.15.47:0/1338257007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32580a2eb0 msgr2=0x7f32580a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.789+0000 7f3279a00640 1 --2- 172.21.15.47:0/1338257007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32580a2eb0 0x7f32580a5370 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f326c008340 tx=0x7f326c008270 comp rx=0 tx=0).stop 2024-03-21T11:43:49.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.789+0000 7f3279a00640 1 -- 172.21.15.47:0/1338257007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f327410b860 msgr2=0x7f327419e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:49.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.789+0000 7f3279a00640 1 --2- 172.21.15.47:0/1338257007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f327410b860 0x7f327419e510 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3260030450 tx=0x7f3260032d10 comp rx=0 tx=0).stop 2024-03-21T11:43:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.791+0000 7f3279a00640 1 -- 172.21.15.47:0/1338257007 shutdown_connections 2024-03-21T11:43:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.791+0000 7f3279a00640 1 --2- 172.21.15.47:0/1338257007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f32580a2eb0 0x7f32580a5370 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.791+0000 7f3279a00640 1 --2- 172.21.15.47:0/1338257007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f327410e1e0 0x7f32741a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.791+0000 7f3279a00640 1 --2- 172.21.15.47:0/1338257007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f327410b860 0x7f327419e510 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.791+0000 7f3279a00640 1 --2- 172.21.15.47:0/1338257007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3274108f30 0x7f327419dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.791+0000 7f3279a00640 1 -- 172.21.15.47:0/1338257007 >> 172.21.15.47:0/1338257007 conn(0x7f327406e500 msgr2=0x7f327410b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.791+0000 7f3279a00640 1 -- 172.21.15.47:0/1338257007 shutdown_connections 2024-03-21T11:43:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:49.791+0000 7f3279a00640 1 -- 172.21.15.47:0/1338257007 wait complete. 2024-03-21T11:43:49.968 INFO:tasks.workunit.client.0.smithi047.stdout: 10 174239 22.87 MB/sec execute 273 sec latency 571.785 ms 2024-03-21T11:43:50.820 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:43:50.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:43:50.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:43:50.968 INFO:tasks.workunit.client.0.smithi047.stdout: 10 174845 22.90 MB/sec execute 274 sec latency 95.101 ms 2024-03-21T11:43:51.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.064+0000 7f5005c00640 1 -- 172.21.15.47:0/950155167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50001038e0 msgr2=0x7f5000103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.064+0000 7f5005c00640 1 --2- 172.21.15.47:0/950155167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50001038e0 0x7f5000103d00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009900 tx=0x7f4ff002bc40 comp rx=0 tx=0).stop 2024-03-21T11:43:51.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.064+0000 7f5005c00640 1 -- 172.21.15.47:0/950155167 shutdown_connections 2024-03-21T11:43:51.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.064+0000 7f5005c00640 1 --2- 172.21.15.47:0/950155167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5000105490 0x7f500010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.064+0000 7f5005c00640 1 --2- 172.21.15.47:0/950155167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5000104ad0 0x7f5000104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.064+0000 7f5005c00640 1 --2- 172.21.15.47:0/950155167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50001038e0 0x7f5000103d00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.064+0000 7f5005c00640 1 -- 172.21.15.47:0/950155167 >> 172.21.15.47:0/950155167 conn(0x7f50000fee50 msgr2=0x7f5000101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:51.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.065+0000 7f5005c00640 1 -- 172.21.15.47:0/950155167 shutdown_connections 2024-03-21T11:43:51.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.065+0000 7f5005c00640 1 -- 172.21.15.47:0/950155167 wait complete. 2024-03-21T11:43:51.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.066+0000 7f5005c00640 1 Processor -- start 2024-03-21T11:43:51.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.066+0000 7f5005c00640 1 -- start start 2024-03-21T11:43:51.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.066+0000 7f5005c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50001038e0 0x7f50000700b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.066+0000 7f5005c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5000104ad0 0x7f50000705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.066+0000 7f5005c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5000105490 0x7f50001ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.067+0000 7f5005c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500010f530 con 0x7f50001038e0 2024-03-21T11:43:51.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.067+0000 7f5005c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500010f3b0 con 0x7f5000105490 2024-03-21T11:43:51.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.067+0000 7f5005c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500010f6b0 con 0x7f5000104ad0 2024-03-21T11:43:51.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.067+0000 7f4ffea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50001038e0 0x7f50000700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.067+0000 7f4ffe000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5000104ad0 0x7f50000705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.067+0000 7f4ffe000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5000104ad0 0x7f50000705f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37884/0 (socket says 172.21.15.47:37884) 2024-03-21T11:43:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.067+0000 7f4ffe000640 1 -- 172.21.15.47:0/1208343251 learned_addr learned my addr 172.21.15.47:0/1208343251 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.067+0000 7f4fff400640 1 --2- 172.21.15.47:0/1208343251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5000105490 0x7f50001ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.068+0000 7f4ffe000640 1 -- 172.21.15.47:0/1208343251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5000105490 msgr2=0x7f50001ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.068+0000 7f4ffe000640 1 --2- 172.21.15.47:0/1208343251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5000105490 0x7f50001ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.068+0000 7f4ffe000640 1 -- 172.21.15.47:0/1208343251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50001038e0 msgr2=0x7f50000700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.068+0000 7f4ffe000640 1 --2- 172.21.15.47:0/1208343251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50001038e0 0x7f50000700b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.068+0000 7f4ffe000640 1 -- 172.21.15.47:0/1208343251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5000079b00 con 0x7f5000104ad0 2024-03-21T11:43:51.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.068+0000 7f4ffea00640 1 --2- 172.21.15.47:0/1208343251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50001038e0 0x7f50000700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:51.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.068+0000 7f4ffe000640 1 --2- 172.21.15.47:0/1208343251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5000104ad0 0x7f50000705f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00e9f0 tx=0x7f4fec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:51.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.069+0000 7f4ff7400640 1 -- 172.21.15.47:0/1208343251 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec00cd80 con 0x7f5000104ad0 2024-03-21T11:43:51.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.069+0000 7f5005c00640 1 -- 172.21.15.47:0/1208343251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50001ae970 con 0x7f5000104ad0 2024-03-21T11:43:51.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.069+0000 7f4ff7400640 1 -- 172.21.15.47:0/1208343251 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fec010910 con 0x7f5000104ad0 2024-03-21T11:43:51.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.069+0000 7f4ff7400640 1 -- 172.21.15.47:0/1208343251 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0188f0 con 0x7f5000104ad0 2024-03-21T11:43:51.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.069+0000 7f5005c00640 1 -- 172.21.15.47:0/1208343251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50001aed20 con 0x7f5000104ad0 2024-03-21T11:43:51.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.074+0000 7f50089d1740 1 Processor -- start 2024-03-21T11:43:51.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.074+0000 7f50089d1740 1 -- start start 2024-03-21T11:43:51.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.074+0000 7f50089d1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da69810b0 0x555da69814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.074+0000 7f50089d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6981aa0 0x555da69865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.075+0000 7f50089d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da6986c10 0x555da69890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.075+0000 7f50089d1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555da68d5eb0 con 0x555da69810b0 2024-03-21T11:43:51.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.075+0000 7f50089d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555da687ea60 con 0x555da6986c10 2024-03-21T11:43:51.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.075+0000 7f50089d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555da68d1f50 con 0x555da6981aa0 2024-03-21T11:43:51.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.075+0000 7f4ffea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6981aa0 0x555da69865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.075+0000 7f4ffea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6981aa0 0x555da69865a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37896/0 (socket says 172.21.15.47:37896) 2024-03-21T11:43:51.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.075+0000 7f4ffea00640 1 -- 172.21.15.47:0/4076937397 learned_addr learned my addr 172.21.15.47:0/4076937397 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:51.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.075+0000 7f4fff400640 1 --2- 172.21.15.47:0/4076937397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da69810b0 0x555da69814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.075+0000 7f4ff7400640 1 -- 172.21.15.47:0/1208343251 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4fec010ac0 con 0x7f5000104ad0 2024-03-21T11:43:51.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.076+0000 7f4ffea00640 1 -- 172.21.15.47:0/4076937397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da6986c10 msgr2=0x555da69890a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.076+0000 7f4ffea00640 1 --2- 172.21.15.47:0/4076937397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da6986c10 0x555da69890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.076+0000 7f4ffea00640 1 -- 172.21.15.47:0/4076937397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da69810b0 msgr2=0x555da69814d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.076+0000 7f4ffea00640 1 --2- 172.21.15.47:0/4076937397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da69810b0 0x555da69814d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.076+0000 7f4ffea00640 1 -- 172.21.15.47:0/4076937397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555da66b27e0 con 0x555da6981aa0 2024-03-21T11:43:51.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.076+0000 7f4fff400640 1 --2- 172.21.15.47:0/4076937397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da69810b0 0x555da69814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:51.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.076+0000 7f4ffea00640 1 --2- 172.21.15.47:0/4076937397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6981aa0 0x555da69865a0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009900 tx=0x7f4ff002cd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4816223c29ebe9e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:51.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.076+0000 7f4febe00640 1 -- 172.21.15.47:0/4076937397 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0005d50 con 0x555da6981aa0 2024-03-21T11:43:51.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.076+0000 7f4febe00640 1 -- 172.21.15.47:0/4076937397 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ff0005520 con 0x555da6981aa0 2024-03-21T11:43:51.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.077+0000 7f4febe00640 1 -- 172.21.15.47:0/4076937397 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0004c40 con 0x555da6981aa0 2024-03-21T11:43:51.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.077+0000 7f50089d1740 1 -- 172.21.15.47:0/4076937397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6981aa0 msgr2=0x555da69865a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.077+0000 7f50089d1740 1 --2- 172.21.15.47:0/4076937397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6981aa0 0x555da69865a0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009900 tx=0x7f4ff002cd40 comp rx=0 tx=0).stop 2024-03-21T11:43:51.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.077+0000 7f50089d1740 1 -- 172.21.15.47:0/4076937397 shutdown_connections 2024-03-21T11:43:51.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.077+0000 7f50089d1740 1 --2- 172.21.15.47:0/4076937397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da6986c10 0x555da69890a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.077+0000 7f50089d1740 1 --2- 172.21.15.47:0/4076937397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6981aa0 0x555da69865a0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.077+0000 7f50089d1740 1 --2- 172.21.15.47:0/4076937397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da69810b0 0x555da69814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.077+0000 7f50089d1740 1 -- 172.21.15.47:0/4076937397 >> 172.21.15.47:0/4076937397 conn(0x555da696b610 msgr2=0x555da696ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:51.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.078+0000 7f50089d1740 1 -- 172.21.15.47:0/4076937397 shutdown_connections 2024-03-21T11:43:51.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.078+0000 7f50089d1740 1 -- 172.21.15.47:0/4076937397 wait complete. 2024-03-21T11:43:51.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.078+0000 7f50089d1740 1 -- start start 2024-03-21T11:43:51.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.078+0000 7f4ff7400640 1 --2- 172.21.15.47:0/1208343251 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4fe40a2eb0 0x7f4fe40a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.078+0000 7f50089d1740 1 Processor -- start 2024-03-21T11:43:51.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.079+0000 7f4ff7400640 1 -- 172.21.15.47:0/1208343251 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4fec0cfbc0 con 0x7f5000104ad0 2024-03-21T11:43:51.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.079+0000 7f4fff400640 1 --2- 172.21.15.47:0/1208343251 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4fe40a2eb0 0x7f4fe40a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.079+0000 7f50089d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da69810b0 0x555da698b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.079+0000 7f50089d1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da6981aa0 0x555da698b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.079+0000 7f50089d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6986c10 0x555da6992800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.079+0000 7f50089d1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555da6989bb0 con 0x555da6981aa0 2024-03-21T11:43:51.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.079+0000 7f50089d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555da6891eb0 con 0x555da69810b0 2024-03-21T11:43:51.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.079+0000 7f50089d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555da6989d30 con 0x555da6986c10 2024-03-21T11:43:51.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.080+0000 7f50089d1740 10 client.0 _ll_register_callbacks cb 0x555da6886150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:51.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.080+0000 7f4fff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6986c10 0x555da6992800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.080+0000 7f4fff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6986c10 0x555da6992800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37904/0 (socket says 172.21.15.47:37904) 2024-03-21T11:43:51.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.080+0000 7f4ffea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da69810b0 0x555da698b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.080+0000 7f4fff400640 1 -- 172.21.15.47:0/2496125741 learned_addr learned my addr 172.21.15.47:0/2496125741 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:51.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.080+0000 7f4ffe000640 1 --2- 172.21.15.47:0/2496125741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da6981aa0 0x555da698b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.081+0000 7f4ffe000640 1 -- 172.21.15.47:0/2496125741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6986c10 msgr2=0x555da6992800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.081+0000 7f4ffe000640 1 --2- 172.21.15.47:0/2496125741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6986c10 0x555da6992800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.081+0000 7f4ffe000640 1 -- 172.21.15.47:0/2496125741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da69810b0 msgr2=0x555da698b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.081+0000 7f4ffe000640 1 --2- 172.21.15.47:0/2496125741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da69810b0 0x555da698b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.081+0000 7f4ffe000640 1 -- 172.21.15.47:0/2496125741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555da6712f70 con 0x555da6981aa0 2024-03-21T11:43:51.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.081+0000 7f4fff400640 1 --2- 172.21.15.47:0/1208343251 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4fe40a2eb0 0x7f4fe40a5370 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x555da6982300 tx=0x7f4ff8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:51.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.081+0000 7f4ffe000640 1 --2- 172.21.15.47:0/2496125741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da6981aa0 0x555da698b7c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4fec098980 tx=0x7f4fec09b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9e845bf790cb6f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:51.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.081+0000 7f4fe8c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:43:51.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.081+0000 7f4fe8c00640 1 -- 172.21.15.47:0/2496125741 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0a4910 con 0x555da6981aa0 2024-03-21T11:43:51.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.081+0000 7f4fe8c00640 1 -- 172.21.15.47:0/2496125741 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4fec09b400 con 0x555da6981aa0 2024-03-21T11:43:51.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.081+0000 7f50089d1740 1 -- 172.21.15.47:0/2496125741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ff0005520 con 0x555da6981aa0 2024-03-21T11:43:51.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.082+0000 7f4fe8c00640 1 -- 172.21.15.47:0/2496125741 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fec09b640 con 0x555da6981aa0 2024-03-21T11:43:51.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.082+0000 7f50089d1740 10 client.21513 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:51.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.082+0000 7f50089d1740 1 -- 172.21.15.47:0/2496125741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555da65aa170 con 0x555da6981aa0 2024-03-21T11:43:51.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.083+0000 7f4fe8c00640 1 -- 172.21.15.47:0/2496125741 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ff0005520 con 0x555da6981aa0 2024-03-21T11:43:51.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.083+0000 7f4fe8c00640 1 -- 172.21.15.47:0/2496125741 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4fec09ae70 con 0x555da6981aa0 2024-03-21T11:43:51.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.083+0000 7f50089d1740 10 client.21513 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:51.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.083+0000 7f50089d1740 10 client.21513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:51.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.083+0000 7f50089d1740 20 client.21513 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:51.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.083+0000 7f50089d1740 1 --2- 172.21.15.47:0/2496125741 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555da6997020 0x555da6999440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.083+0000 7f50089d1740 4 client.21513 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:51.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.083+0000 7f50089d1740 1 -- 172.21.15.47:0/2496125741 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555da68e33f0 con 0x555da6997020 2024-03-21T11:43:51.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.084+0000 7f4ffea00640 1 --2- 172.21.15.47:0/2496125741 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555da6997020 0x555da6999440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.084+0000 7f4ffea00640 1 --2- 172.21.15.47:0/2496125741 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555da6997020 0x555da6999440 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb0402939c9e5c8 server_cookie=ccfeb16196678d72 in_seq=0 out_seq=0 2024-03-21T11:43:51.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.084+0000 7f4fe8c00640 10 client.21513 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:51.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.089+0000 7f4fe8c00640 1 -- 172.21.15.47:0/2496125741 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555da68e33f0 con 0x555da6997020 2024-03-21T11:43:51.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.089+0000 7f4fe8c00640 10 client.21513 handle_command_reply: tid=0 2024-03-21T11:43:51.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.089+0000 7f50089d1740 1 client.21513 shutdown 2024-03-21T11:43:51.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.089+0000 7f50089d1740 1 -- 172.21.15.47:0/2496125741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da6981aa0 msgr2=0x555da698b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.089+0000 7f50089d1740 1 --2- 172.21.15.47:0/2496125741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da6981aa0 0x555da698b7c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4fec098980 tx=0x7f4fec09b9d0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.090+0000 7f50089d1740 1 -- 172.21.15.47:0/2496125741 shutdown_connections 2024-03-21T11:43:51.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.090+0000 7f50089d1740 1 --2- 172.21.15.47:0/2496125741 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555da6997020 0x555da6999440 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.090+0000 7f50089d1740 1 --2- 172.21.15.47:0/2496125741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6986c10 0x555da6992800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.090+0000 7f4fe8c00640 0 client.21513 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:51.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.090+0000 7f50089d1740 1 --2- 172.21.15.47:0/2496125741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da6981aa0 0x555da698b7c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.090+0000 7f50089d1740 1 --2- 172.21.15.47:0/2496125741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da69810b0 0x555da698b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.090+0000 7f50089d1740 1 -- 172.21.15.47:0/2496125741 >> 172.21.15.47:0/2496125741 conn(0x555da696b610 msgr2=0x555da696a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:51.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.091+0000 7f50089d1740 1 -- 172.21.15.47:0/2496125741 shutdown_connections 2024-03-21T11:43:51.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.093+0000 7f50089d1740 1 -- 172.21.15.47:0/2496125741 wait complete. 2024-03-21T11:43:51.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.094+0000 7f50089d1740 20 client.21513 trim_cache size 0 max 16384 2024-03-21T11:43:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.123+0000 7f50089d1740 1 Processor -- start 2024-03-21T11:43:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.123+0000 7f50089d1740 1 -- start start 2024-03-21T11:43:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.123+0000 7f50089d1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da69810b0 0x555da6982c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.123+0000 7f50089d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da6981aa0 0x555da6983190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.123+0000 7f50089d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6986c10 0x555da69927c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.123+0000 7f50089d1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001b06c0 con 0x555da69810b0 2024-03-21T11:43:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.123+0000 7f50089d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001b2040 con 0x555da6981aa0 2024-03-21T11:43:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.123+0000 7f50089d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001b0d20 con 0x555da6986c10 2024-03-21T11:43:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.124+0000 7f4fff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6986c10 0x555da69927c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.124+0000 7f4ffe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da6981aa0 0x555da6983190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.124+0000 7f4fff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6986c10 0x555da69927c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37906/0 (socket says 172.21.15.47:37906) 2024-03-21T11:43:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.124+0000 7f4fff400640 1 -- 172.21.15.47:0/1075470752 learned_addr learned my addr 172.21.15.47:0/1075470752 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.124+0000 7f4ffea00640 1 --2- 172.21.15.47:0/1075470752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da69810b0 0x555da6982c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.124+0000 7f4fff400640 1 -- 172.21.15.47:0/1075470752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da6981aa0 msgr2=0x555da6983190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.124+0000 7f4fff400640 1 --2- 172.21.15.47:0/1075470752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da6981aa0 0x555da6983190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.124+0000 7f4fff400640 1 -- 172.21.15.47:0/1075470752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da69810b0 msgr2=0x555da6982c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.124+0000 7f4fff400640 1 --2- 172.21.15.47:0/1075470752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da69810b0 0x555da6982c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.124+0000 7f4fff400640 1 -- 172.21.15.47:0/1075470752 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555da6966c20 con 0x555da6986c10 2024-03-21T11:43:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.125+0000 7f4fff400640 1 --2- 172.21.15.47:0/1075470752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6986c10 0x555da69927c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8003d90 tx=0x7f4ff801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33cd425282461bc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.125+0000 7f4feb400640 1 -- 172.21.15.47:0/1075470752 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8004b20 con 0x555da6986c10 2024-03-21T11:43:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.125+0000 7f4feb400640 1 -- 172.21.15.47:0/1075470752 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ff8004cc0 con 0x555da6986c10 2024-03-21T11:43:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.125+0000 7f4feb400640 1 -- 172.21.15.47:0/1075470752 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8005460 con 0x555da6986c10 2024-03-21T11:43:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.127+0000 7f50089d1740 1 -- 172.21.15.47:0/1075470752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6986c10 msgr2=0x555da69927c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.127+0000 7f50089d1740 1 --2- 172.21.15.47:0/1075470752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6986c10 0x555da69927c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8003d90 tx=0x7f4ff801f000 comp rx=0 tx=0).stop 2024-03-21T11:43:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.127+0000 7f50089d1740 1 -- 172.21.15.47:0/1075470752 shutdown_connections 2024-03-21T11:43:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.127+0000 7f50089d1740 1 --2- 172.21.15.47:0/1075470752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da6986c10 0x555da69927c0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.127+0000 7f50089d1740 1 --2- 172.21.15.47:0/1075470752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da6981aa0 0x555da6983190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.127+0000 7f50089d1740 1 --2- 172.21.15.47:0/1075470752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da69810b0 0x555da6982c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.127+0000 7f50089d1740 1 -- 172.21.15.47:0/1075470752 >> 172.21.15.47:0/1075470752 conn(0x555da696b610 msgr2=0x555da6982540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.127+0000 7f50089d1740 1 -- 172.21.15.47:0/1075470752 shutdown_connections 2024-03-21T11:43:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.127+0000 7f50089d1740 1 -- 172.21.15.47:0/1075470752 wait complete. 2024-03-21T11:43:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.128+0000 7f50089d1740 1 -- start start 2024-03-21T11:43:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.128+0000 7f50089d1740 1 Processor -- start 2024-03-21T11:43:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.129+0000 7f50089d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da69810b0 0x555da69923d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.129+0000 7f50089d1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da6981aa0 0x555da6990a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.129+0000 7f50089d1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da6986c10 0x555da6990f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.129+0000 7f50089d1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001b19e0 con 0x555da6986c10 2024-03-21T11:43:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.129+0000 7f50089d1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001b1380 con 0x555da6981aa0 2024-03-21T11:43:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.129+0000 7f50089d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001add50 con 0x555da69810b0 2024-03-21T11:43:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.129+0000 7f50089d1740 10 client.0 _ll_register_callbacks cb 0x7f4fec09baf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.129+0000 7f4ffe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da6981aa0 0x555da6990a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.129+0000 7f4ffea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da69810b0 0x555da69923d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.129+0000 7f4ffea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da69810b0 0x555da69923d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37912/0 (socket says 172.21.15.47:37912) 2024-03-21T11:43:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.129+0000 7f4ffea00640 1 -- 172.21.15.47:0/1213508719 learned_addr learned my addr 172.21.15.47:0/1213508719 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.130+0000 7f4fff400640 1 --2- 172.21.15.47:0/1213508719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da6986c10 0x555da6990f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.130+0000 7f4ffea00640 1 -- 172.21.15.47:0/1213508719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da6981aa0 msgr2=0x555da6990a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.130+0000 7f4ffea00640 1 --2- 172.21.15.47:0/1213508719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da6981aa0 0x555da6990a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.130+0000 7f4ffea00640 1 -- 172.21.15.47:0/1213508719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da6986c10 msgr2=0x555da6990f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.130+0000 7f4ffea00640 1 --2- 172.21.15.47:0/1213508719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da6986c10 0x555da6990f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.130+0000 7f4ffea00640 1 -- 172.21.15.47:0/1213508719 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555da69914b0 con 0x555da69810b0 2024-03-21T11:43:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.130+0000 7f4fff400640 1 --2- 172.21.15.47:0/1213508719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da6986c10 0x555da6990f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:43:51.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.132+0000 7f4ffea00640 1 --2- 172.21.15.47:0/1213508719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da69810b0 0x555da69923d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f4ff002c7c0 tx=0x7f4ff0005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f7b444c6d96ae9f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:51.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.132+0000 7f4fe8c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:51.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.133+0000 7f4fe8c00640 1 -- 172.21.15.47:0/1213508719 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0059060 con 0x555da69810b0 2024-03-21T11:43:51.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.133+0000 7f4fe8c00640 1 -- 172.21.15.47:0/1213508719 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4ff0054170 con 0x555da69810b0 2024-03-21T11:43:51.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.133+0000 7f50089d1740 1 -- 172.21.15.47:0/1213508719 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ff8004cc0 con 0x555da69810b0 2024-03-21T11:43:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.133+0000 7f4fe8c00640 1 -- 172.21.15.47:0/1213508719 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ff0004c40 con 0x555da69810b0 2024-03-21T11:43:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.133+0000 7f50089d1740 10 client.32125 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.133+0000 7f50089d1740 1 -- 172.21.15.47:0/1213508719 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x555da69f3620 con 0x555da69810b0 2024-03-21T11:43:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.134+0000 7f4fe8c00640 1 -- 172.21.15.47:0/1213508719 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ff8004cc0 con 0x555da69810b0 2024-03-21T11:43:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.134+0000 7f4fe8c00640 1 -- 172.21.15.47:0/1213508719 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4ff004d040 con 0x555da69810b0 2024-03-21T11:43:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.134+0000 7f50089d1740 10 client.32125 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.134+0000 7f50089d1740 10 client.32125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.134+0000 7f50089d1740 20 client.32125 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.134+0000 7f50089d1740 1 --2- 172.21.15.47:0/1213508719 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555da6997020 0x555da69f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.134+0000 7f50089d1740 4 client.32125 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:43:51.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.134+0000 7f50089d1740 1 -- 172.21.15.47:0/1213508719 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555da68dd740 con 0x555da6997020 2024-03-21T11:43:51.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.135+0000 7f4ffe000640 1 --2- 172.21.15.47:0/1213508719 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555da6997020 0x555da69f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.135+0000 7f4ffe000640 1 --2- 172.21.15.47:0/1213508719 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555da6997020 0x555da69f5bb0 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3316e875fd944bd0 server_cookie=45fbc92e12bbffae in_seq=0 out_seq=0 2024-03-21T11:43:51.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.135+0000 7f4fe8c00640 10 client.32125 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:51.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.141+0000 7f4fe8c00640 1 -- 172.21.15.47:0/1213508719 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555da68dd740 con 0x555da6997020 2024-03-21T11:43:51.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.141+0000 7f4fe8c00640 10 client.32125 handle_command_reply: tid=0 2024-03-21T11:43:51.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.141+0000 7f50089d1740 1 client.32125 shutdown 2024-03-21T11:43:51.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.141+0000 7f50089d1740 1 -- 172.21.15.47:0/1213508719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da69810b0 msgr2=0x555da69923d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.141+0000 7f50089d1740 1 --2- 172.21.15.47:0/1213508719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da69810b0 0x555da69923d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f4ff002c7c0 tx=0x7f4ff0005e80 comp rx=0 tx=0).stop 2024-03-21T11:43:51.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.141+0000 7f50089d1740 1 -- 172.21.15.47:0/1213508719 shutdown_connections 2024-03-21T11:43:51.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.141+0000 7f50089d1740 1 --2- 172.21.15.47:0/1213508719 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555da6997020 0x555da69f5bb0 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.141+0000 7f50089d1740 1 --2- 172.21.15.47:0/1213508719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555da6986c10 0x555da6990f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.141+0000 7f50089d1740 1 --2- 172.21.15.47:0/1213508719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555da6981aa0 0x555da6990a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.141+0000 7f50089d1740 1 --2- 172.21.15.47:0/1213508719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555da69810b0 0x555da69923d0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.141+0000 7f50089d1740 1 -- 172.21.15.47:0/1213508719 >> 172.21.15.47:0/1213508719 conn(0x555da696b610 msgr2=0x555da6982dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:51.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.141+0000 7f4fe8c00640 0 client.32125 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:51.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.142+0000 7f50089d1740 1 -- 172.21.15.47:0/1213508719 shutdown_connections 2024-03-21T11:43:51.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.142+0000 7f50089d1740 1 -- 172.21.15.47:0/1213508719 wait complete. 2024-03-21T11:43:51.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.142+0000 7f50089d1740 20 client.32125 trim_cache size 0 max 16384 2024-03-21T11:43:51.144 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:43:51.144 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:43:51.144 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "3de65fb8-df01-4464-a428-6f9f71eac60f", 2024-03-21T11:43:51.144 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:43:51.144 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:43:51.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.143+0000 7f5005c00640 1 -- 172.21.15.47:0/1208343251 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4fe40a2eb0 msgr2=0x7f4fe40a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.143+0000 7f5005c00640 1 --2- 172.21.15.47:0/1208343251 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4fe40a2eb0 0x7f4fe40a5370 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x555da6982300 tx=0x7f4ff8009500 comp rx=0 tx=0).stop 2024-03-21T11:43:51.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.143+0000 7f5005c00640 1 -- 172.21.15.47:0/1208343251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5000104ad0 msgr2=0x7f50000705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.143+0000 7f5005c00640 1 --2- 172.21.15.47:0/1208343251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5000104ad0 0x7f50000705f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00e9f0 tx=0x7f4fec00eeb0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.143+0000 7f5005c00640 1 -- 172.21.15.47:0/1208343251 shutdown_connections 2024-03-21T11:43:51.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.144+0000 7f5005c00640 1 --2- 172.21.15.47:0/1208343251 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4fe40a2eb0 0x7f4fe40a5370 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.144+0000 7f5005c00640 1 --2- 172.21.15.47:0/1208343251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5000105490 0x7f50001ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.144+0000 7f5005c00640 1 --2- 172.21.15.47:0/1208343251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5000104ad0 0x7f50000705f0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.144+0000 7f5005c00640 1 --2- 172.21.15.47:0/1208343251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50001038e0 0x7f50000700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.144+0000 7f5005c00640 1 -- 172.21.15.47:0/1208343251 >> 172.21.15.47:0/1208343251 conn(0x7f50000fee50 msgr2=0x7f5000107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:51.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.144+0000 7f5005c00640 1 -- 172.21.15.47:0/1208343251 shutdown_connections 2024-03-21T11:43:51.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.144+0000 7f5005c00640 1 -- 172.21.15.47:0/1208343251 wait complete. 2024-03-21T11:43:51.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:43:51.170 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:43:51.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.413+0000 7f4448600640 1 -- 172.21.15.47:0/380143578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44401016b0 msgr2=0x7f4440101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.413+0000 7f4448600640 1 --2- 172.21.15.47:0/380143578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44401016b0 0x7f4440101ad0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4434009860 tx=0x7f443402bb70 comp rx=0 tx=0).stop 2024-03-21T11:43:51.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.415+0000 7f4448600640 1 -- 172.21.15.47:0/380143578 shutdown_connections 2024-03-21T11:43:51.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.415+0000 7f4448600640 1 --2- 172.21.15.47:0/380143578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4440103260 0x7f4440109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.415+0000 7f4448600640 1 --2- 172.21.15.47:0/380143578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44401028a0 0x7f4440102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.415+0000 7f4448600640 1 --2- 172.21.15.47:0/380143578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44401016b0 0x7f4440101ad0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.415+0000 7f4448600640 1 -- 172.21.15.47:0/380143578 >> 172.21.15.47:0/380143578 conn(0x7f44400fcc40 msgr2=0x7f44400ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:51.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.417+0000 7f4448600640 1 -- 172.21.15.47:0/380143578 shutdown_connections 2024-03-21T11:43:51.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.417+0000 7f4448600640 1 -- 172.21.15.47:0/380143578 wait complete. 2024-03-21T11:43:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.419+0000 7f4448600640 1 Processor -- start 2024-03-21T11:43:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.419+0000 7f4448600640 1 -- start start 2024-03-21T11:43:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.420+0000 7f4448600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44401016b0 0x7f4440197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.420+0000 7f4448600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44401028a0 0x7f4440197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.420+0000 7f4448600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4440103260 0x7f444019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.420+0000 7f4448600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444010d300 con 0x7f44401016b0 2024-03-21T11:43:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.420+0000 7f4448600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444010d180 con 0x7f4440103260 2024-03-21T11:43:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.420+0000 7f4448600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444010d480 con 0x7f44401028a0 2024-03-21T11:43:51.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.420+0000 7f4445c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44401016b0 0x7f4440197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.420+0000 7f4445200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44401028a0 0x7f4440197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.420+0000 7f4445200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44401028a0 0x7f4440197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37934/0 (socket says 172.21.15.47:37934) 2024-03-21T11:43:51.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.420+0000 7f4445200640 1 -- 172.21.15.47:0/726429802 learned_addr learned my addr 172.21.15.47:0/726429802 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:51.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.421+0000 7f4445200640 1 -- 172.21.15.47:0/726429802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4440103260 msgr2=0x7f444019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.421+0000 7f4445200640 1 --2- 172.21.15.47:0/726429802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4440103260 0x7f444019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.421+0000 7f4445200640 1 -- 172.21.15.47:0/726429802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44401016b0 msgr2=0x7f4440197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.421+0000 7f4445200640 1 --2- 172.21.15.47:0/726429802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44401016b0 0x7f4440197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.421+0000 7f4445200640 1 -- 172.21.15.47:0/726429802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44400739d0 con 0x7f44401028a0 2024-03-21T11:43:51.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.422+0000 7f4445200640 1 --2- 172.21.15.47:0/726429802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44401028a0 0x7f4440197d70 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f443000b2f0 tx=0x7f443000b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.422+0000 7f443e000640 1 -- 172.21.15.47:0/726429802 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430022070 con 0x7f44401028a0 2024-03-21T11:43:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.422+0000 7f443e000640 1 -- 172.21.15.47:0/726429802 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f443001ba00 con 0x7f44401028a0 2024-03-21T11:43:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.422+0000 7f443e000640 1 -- 172.21.15.47:0/726429802 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443001dba0 con 0x7f44401028a0 2024-03-21T11:43:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.422+0000 7f4448600640 1 -- 172.21.15.47:0/726429802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4440073cc0 con 0x7f44401028a0 2024-03-21T11:43:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.422+0000 7f4448600640 1 -- 172.21.15.47:0/726429802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44400741c0 con 0x7f44401028a0 2024-03-21T11:43:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.426+0000 7f444b3a4740 1 Processor -- start 2024-03-21T11:43:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.427+0000 7f444b3a4740 1 -- start start 2024-03-21T11:43:51.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.427+0000 7f443e000640 1 -- 172.21.15.47:0/726429802 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4430025070 con 0x7f44401028a0 2024-03-21T11:43:51.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.427+0000 7f444b3a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d2e80 0x55a6488d32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.427+0000 7f444b3a4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d3870 0x55a6488d8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.427+0000 7f444b3a4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d89e0 0x55a6488dae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.427+0000 7f444b3a4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a648838250 con 0x55a6488d3870 2024-03-21T11:43:51.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.427+0000 7f444b3a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6487e0a30 con 0x55a6488d2e80 2024-03-21T11:43:51.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.427+0000 7f444b3a4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6488342f0 con 0x55a6488d89e0 2024-03-21T11:43:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.428+0000 7f4446600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d2e80 0x55a6488d32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.428+0000 7f4446600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d2e80 0x55a6488d32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40676/0 (socket says 172.21.15.47:40676) 2024-03-21T11:43:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.428+0000 7f4446600640 1 -- 172.21.15.47:0/1164754925 learned_addr learned my addr 172.21.15.47:0/1164754925 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.428+0000 7f4445c00640 1 --2- 172.21.15.47:0/1164754925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d3870 0x55a6488d8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.428+0000 7f4445200640 1 --2- 172.21.15.47:0/1164754925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d89e0 0x55a6488dae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.429+0000 7f4446600640 1 -- 172.21.15.47:0/1164754925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d89e0 msgr2=0x55a6488dae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.429+0000 7f4446600640 1 --2- 172.21.15.47:0/1164754925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d89e0 0x55a6488dae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.429+0000 7f4446600640 1 -- 172.21.15.47:0/1164754925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d3870 msgr2=0x55a6488d8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.429+0000 7f4446600640 1 --2- 172.21.15.47:0/1164754925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d3870 0x55a6488d8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.429+0000 7f4446600640 1 -- 172.21.15.47:0/1164754925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6487cb690 con 0x55a6488d2e80 2024-03-21T11:43:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.429+0000 7f443e000640 1 --2- 172.21.15.47:0/726429802 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44280a2fa0 0x7f44280a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.429+0000 7f4446600640 1 --2- 172.21.15.47:0/1164754925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d2e80 0x55a6488d32a0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4438009ed0 tx=0x7f443800e7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1156e7607bc0920 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.430+0000 7f443e000640 1 -- 172.21.15.47:0/726429802 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f44300d7370 con 0x7f44401028a0 2024-03-21T11:43:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.430+0000 7f4445c00640 1 --2- 172.21.15.47:0/726429802 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44280a2fa0 0x7f44280a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.430+0000 7f442ea00640 1 -- 172.21.15.47:0/1164754925 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443801b070 con 0x55a6488d2e80 2024-03-21T11:43:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.430+0000 7f442ea00640 1 -- 172.21.15.47:0/1164754925 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f443800cd20 con 0x55a6488d2e80 2024-03-21T11:43:51.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.430+0000 7f442ea00640 1 -- 172.21.15.47:0/1164754925 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443800b360 con 0x55a6488d2e80 2024-03-21T11:43:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.431+0000 7f444b3a4740 1 -- 172.21.15.47:0/1164754925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d2e80 msgr2=0x55a6488d32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.431+0000 7f444b3a4740 1 --2- 172.21.15.47:0/1164754925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d2e80 0x55a6488d32a0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4438009ed0 tx=0x7f443800e7c0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.431+0000 7f444b3a4740 1 -- 172.21.15.47:0/1164754925 shutdown_connections 2024-03-21T11:43:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.431+0000 7f444b3a4740 1 --2- 172.21.15.47:0/1164754925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d89e0 0x55a6488dae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.431+0000 7f444b3a4740 1 --2- 172.21.15.47:0/1164754925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d3870 0x55a6488d8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.431+0000 7f444b3a4740 1 --2- 172.21.15.47:0/1164754925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d2e80 0x55a6488d32a0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.431+0000 7f444b3a4740 1 -- 172.21.15.47:0/1164754925 >> 172.21.15.47:0/1164754925 conn(0x55a64889b200 msgr2=0x55a6488ce7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.432+0000 7f444b3a4740 1 -- 172.21.15.47:0/1164754925 shutdown_connections 2024-03-21T11:43:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.432+0000 7f444b3a4740 1 -- 172.21.15.47:0/1164754925 wait complete. 2024-03-21T11:43:51.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.432+0000 7f4445c00640 1 --2- 172.21.15.47:0/726429802 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44280a2fa0 0x7f44280a5460 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f443402c180 tx=0x7f4434005710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:51.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.432+0000 7f444b3a4740 1 -- start start 2024-03-21T11:43:51.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.433+0000 7f444b3a4740 1 Processor -- start 2024-03-21T11:43:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.433+0000 7f444b3a4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d2e80 0x55a6488dce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.433+0000 7f444b3a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d3870 0x55a6488dd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.434+0000 7f444b3a4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d89e0 0x55a6488e4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.434+0000 7f444b3a4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6488db990 con 0x55a6488d89e0 2024-03-21T11:43:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.434+0000 7f444b3a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6487f3e80 con 0x55a6488d3870 2024-03-21T11:43:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.434+0000 7f444b3a4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6488dbb10 con 0x55a6488d2e80 2024-03-21T11:43:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.434+0000 7f444b3a4740 10 client.0 _ll_register_callbacks cb 0x55a648806130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.434+0000 7f4445c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d2e80 0x55a6488dce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.434+0000 7f4445c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d2e80 0x55a6488dce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37956/0 (socket says 172.21.15.47:37956) 2024-03-21T11:43:51.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.434+0000 7f4445c00640 1 -- 172.21.15.47:0/2252274269 learned_addr learned my addr 172.21.15.47:0/2252274269 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:51.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.434+0000 7f4445200640 1 --2- 172.21.15.47:0/2252274269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d3870 0x55a6488dd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.434+0000 7f4445c00640 1 -- 172.21.15.47:0/2252274269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d3870 msgr2=0x55a6488dd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.434+0000 7f4445c00640 1 --2- 172.21.15.47:0/2252274269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d3870 0x55a6488dd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.434+0000 7f4445c00640 1 -- 172.21.15.47:0/2252274269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d89e0 msgr2=0x55a6488e4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.434+0000 7f4445c00640 1 --2- 172.21.15.47:0/2252274269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d89e0 0x55a6488e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.434+0000 7f4445c00640 1 -- 172.21.15.47:0/2252274269 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6486167d0 con 0x55a6488d2e80 2024-03-21T11:43:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.435+0000 7f4445c00640 1 --2- 172.21.15.47:0/2252274269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d2e80 0x55a6488dce90 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f44340095f0 tx=0x7f443402bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0f91a19a060612f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.435+0000 7f4423400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:43:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.435+0000 7f4423400640 1 -- 172.21.15.47:0/2252274269 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443404e020 con 0x55a6488d2e80 2024-03-21T11:43:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.435+0000 7f4423400640 1 -- 172.21.15.47:0/2252274269 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f443402c920 con 0x55a6488d2e80 2024-03-21T11:43:51.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.436+0000 7f444b3a4740 1 -- 172.21.15.47:0/2252274269 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f443800cd20 con 0x55a6488d2e80 2024-03-21T11:43:51.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.437+0000 7f4423400640 1 -- 172.21.15.47:0/2252274269 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4434039040 con 0x55a6488d2e80 2024-03-21T11:43:51.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.437+0000 7f444b3a4740 10 client.32137 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:51.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.437+0000 7f444b3a4740 1 -- 172.21.15.47:0/2252274269 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a64854c3f0 con 0x55a6488d2e80 2024-03-21T11:43:51.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.439+0000 7f4423400640 1 -- 172.21.15.47:0/2252274269 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f443800cd20 con 0x55a6488d2e80 2024-03-21T11:43:51.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.439+0000 7f4423400640 1 -- 172.21.15.47:0/2252274269 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4434003680 con 0x55a6488d2e80 2024-03-21T11:43:51.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.440+0000 7f444b3a4740 10 client.32137 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:51.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.440+0000 7f444b3a4740 10 client.32137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:51.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.440+0000 7f444b3a4740 20 client.32137 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:51.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.440+0000 7f444b3a4740 1 --2- 172.21.15.47:0/2252274269 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a6488e8c90 0x55a6488eb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.440+0000 7f444b3a4740 4 client.32137 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:43:51.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.440+0000 7f444b3a4740 1 -- 172.21.15.47:0/2252274269 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a648845790 con 0x55a6488e8c90 2024-03-21T11:43:51.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.441+0000 7f4445200640 1 --2- 172.21.15.47:0/2252274269 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a6488e8c90 0x55a6488eb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.441+0000 7f4445200640 1 --2- 172.21.15.47:0/2252274269 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a6488e8c90 0x55a6488eb110 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=592c5fe5fbe7cc8d server_cookie=83cb0d0a80607744 in_seq=0 out_seq=0 2024-03-21T11:43:51.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.441+0000 7f4423400640 10 client.32137 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:51.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.445+0000 7f4423400640 1 -- 172.21.15.47:0/2252274269 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a648845790 con 0x55a6488e8c90 2024-03-21T11:43:51.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.445+0000 7f4423400640 10 client.32137 handle_command_reply: tid=0 2024-03-21T11:43:51.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.446+0000 7f444b3a4740 1 client.32137 shutdown 2024-03-21T11:43:51.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.446+0000 7f444b3a4740 1 -- 172.21.15.47:0/2252274269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d2e80 msgr2=0x55a6488dce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.446+0000 7f444b3a4740 1 --2- 172.21.15.47:0/2252274269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d2e80 0x55a6488dce90 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f44340095f0 tx=0x7f443402bb70 comp rx=0 tx=0).stop 2024-03-21T11:43:51.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.447+0000 7f444b3a4740 1 -- 172.21.15.47:0/2252274269 shutdown_connections 2024-03-21T11:43:51.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.447+0000 7f444b3a4740 1 --2- 172.21.15.47:0/2252274269 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a6488e8c90 0x55a6488eb110 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.447+0000 7f444b3a4740 1 --2- 172.21.15.47:0/2252274269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d89e0 0x55a6488e4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.447+0000 7f4423400640 0 client.32137 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:51.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.447+0000 7f444b3a4740 1 --2- 172.21.15.47:0/2252274269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d3870 0x55a6488dd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.447+0000 7f444b3a4740 1 --2- 172.21.15.47:0/2252274269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d2e80 0x55a6488dce90 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.447+0000 7f444b3a4740 1 -- 172.21.15.47:0/2252274269 >> 172.21.15.47:0/2252274269 conn(0x55a64889b200 msgr2=0x55a6488cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.448+0000 7f444b3a4740 1 -- 172.21.15.47:0/2252274269 shutdown_connections 2024-03-21T11:43:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.448+0000 7f444b3a4740 1 -- 172.21.15.47:0/2252274269 wait complete. 2024-03-21T11:43:51.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.448+0000 7f444b3a4740 20 client.32137 trim_cache size 0 max 16384 2024-03-21T11:43:51.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.477+0000 7f444b3a4740 1 Processor -- start 2024-03-21T11:43:51.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.477+0000 7f444b3a4740 1 -- start start 2024-03-21T11:43:51.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.477+0000 7f444b3a4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d2e80 0x55a6488d2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.477+0000 7f444b3a4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d3870 0x55a6488e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.477+0000 7f444b3a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d89e0 0x55a6488d4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.477+0000 7f444b3a4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44401a7d10 con 0x55a6488d2e80 2024-03-21T11:43:51.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.477+0000 7f444b3a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44401a9690 con 0x55a6488d89e0 2024-03-21T11:43:51.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.477+0000 7f444b3a4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44401a8370 con 0x55a6488d3870 2024-03-21T11:43:51.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.478+0000 7f4445200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d3870 0x55a6488e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.478+0000 7f4445200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d3870 0x55a6488e43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37970/0 (socket says 172.21.15.47:37970) 2024-03-21T11:43:51.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.478+0000 7f4445200640 1 -- 172.21.15.47:0/425490229 learned_addr learned my addr 172.21.15.47:0/425490229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:51.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.478+0000 7f4445c00640 1 --2- 172.21.15.47:0/425490229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d2e80 0x55a6488d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.479+0000 7f4445200640 1 -- 172.21.15.47:0/425490229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d89e0 msgr2=0x55a6488d4100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:43:51.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.479+0000 7f4445200640 1 --2- 172.21.15.47:0/425490229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d89e0 0x55a6488d4100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.479+0000 7f4445200640 1 -- 172.21.15.47:0/425490229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d2e80 msgr2=0x55a6488d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.479+0000 7f4445200640 1 --2- 172.21.15.47:0/425490229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d2e80 0x55a6488d2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.479+0000 7f4445200640 1 -- 172.21.15.47:0/425490229 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a64850c0f0 con 0x55a6488d3870 2024-03-21T11:43:51.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.479+0000 7f4445200640 1 --2- 172.21.15.47:0/425490229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d3870 0x55a6488e43d0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f44300a36f0 tx=0x7f44300d4950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfd1fdc52792e1aa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:51.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.480+0000 7f442e000640 1 -- 172.21.15.47:0/425490229 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44300a3760 con 0x55a6488d3870 2024-03-21T11:43:51.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.480+0000 7f442e000640 1 -- 172.21.15.47:0/425490229 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44300a7800 con 0x55a6488d3870 2024-03-21T11:43:51.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.480+0000 7f442e000640 1 -- 172.21.15.47:0/425490229 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443001bbb0 con 0x55a6488d3870 2024-03-21T11:43:51.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.481+0000 7f444b3a4740 1 -- 172.21.15.47:0/425490229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d3870 msgr2=0x55a6488e43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.481+0000 7f444b3a4740 1 --2- 172.21.15.47:0/425490229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d3870 0x55a6488e43d0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f44300a36f0 tx=0x7f44300d4950 comp rx=0 tx=0).stop 2024-03-21T11:43:51.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.482+0000 7f444b3a4740 1 -- 172.21.15.47:0/425490229 shutdown_connections 2024-03-21T11:43:51.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.482+0000 7f444b3a4740 1 --2- 172.21.15.47:0/425490229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d89e0 0x55a6488d4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.482+0000 7f444b3a4740 1 --2- 172.21.15.47:0/425490229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d3870 0x55a6488e43d0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.482+0000 7f444b3a4740 1 --2- 172.21.15.47:0/425490229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d2e80 0x55a6488d2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.482+0000 7f444b3a4740 1 -- 172.21.15.47:0/425490229 >> 172.21.15.47:0/425490229 conn(0x55a64889b200 msgr2=0x55a6488d1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:51.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.482+0000 7f444b3a4740 1 -- 172.21.15.47:0/425490229 shutdown_connections 2024-03-21T11:43:51.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.482+0000 7f444b3a4740 1 -- 172.21.15.47:0/425490229 wait complete. 2024-03-21T11:43:51.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.483+0000 7f444b3a4740 1 -- start start 2024-03-21T11:43:51.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.483+0000 7f444b3a4740 1 Processor -- start 2024-03-21T11:43:51.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.484+0000 7f444b3a4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d2e80 0x55a6488e1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.484+0000 7f444b3a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d3870 0x55a6488e22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.485+0000 7f444b3a4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d89e0 0x55a6488e3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.486+0000 7f4445200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d3870 0x55a6488e22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.486+0000 7f4445200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d3870 0x55a6488e22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40708/0 (socket says 172.21.15.47:40708) 2024-03-21T11:43:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.486+0000 7f4445c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d2e80 0x55a6488e1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.486+0000 7f4445c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d2e80 0x55a6488e1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:37984/0 (socket says 172.21.15.47:37984) 2024-03-21T11:43:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.486+0000 7f444b3a4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44401a9030 con 0x55a6488d89e0 2024-03-21T11:43:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.486+0000 7f444b3a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44401a89d0 con 0x55a6488d3870 2024-03-21T11:43:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.486+0000 7f4446600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d89e0 0x55a6488e3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.486+0000 7f444b3a4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44401a5ed0 con 0x55a6488d2e80 2024-03-21T11:43:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.487+0000 7f4446600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d89e0 0x55a6488e3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35878/0 (socket says 172.21.15.47:35878) 2024-03-21T11:43:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.487+0000 7f4446600640 1 -- 172.21.15.47:0/3761348290 learned_addr learned my addr 172.21.15.47:0/3761348290 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:43:51.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.487+0000 7f444b3a4740 10 client.0 _ll_register_callbacks cb 0x7f4430005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:43:51.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.487+0000 7f4446600640 1 -- 172.21.15.47:0/3761348290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d2e80 msgr2=0x55a6488e1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.487+0000 7f4446600640 1 --2- 172.21.15.47:0/3761348290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d2e80 0x55a6488e1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.487+0000 7f4446600640 1 -- 172.21.15.47:0/3761348290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d3870 msgr2=0x55a6488e22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:43:51.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.487+0000 7f4446600640 1 --2- 172.21.15.47:0/3761348290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d3870 0x55a6488e22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.487+0000 7f4446600640 1 -- 172.21.15.47:0/3761348290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6488e3830 con 0x55a6488d89e0 2024-03-21T11:43:51.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.490+0000 7f4446600640 1 --2- 172.21.15.47:0/3761348290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d89e0 0x55a6488e3240 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4438009ea0 tx=0x7f443800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2301395fbeb71b4f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:43:51.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.490+0000 7f4423400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:43:51.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:51 smithi102 ceph-mon[36748]: pgmap v1015: 129 pgs: 129 active+clean; 583 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 536 op/s 2024-03-21T11:43:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.490+0000 7f4423400640 1 -- 172.21.15.47:0/3761348290 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443801b050 con 0x55a6488d89e0 2024-03-21T11:43:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.490+0000 7f4423400640 1 -- 172.21.15.47:0/3761348290 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4438022650 con 0x55a6488d89e0 2024-03-21T11:43:51.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.491+0000 7f444b3a4740 1 -- 172.21.15.47:0/3761348290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44300a7800 con 0x55a6488d89e0 2024-03-21T11:43:51.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.491+0000 7f4423400640 1 -- 172.21.15.47:0/3761348290 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4438022850 con 0x55a6488d89e0 2024-03-21T11:43:51.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.491+0000 7f444b3a4740 10 client.21534 fetch_fsmap learned FSMap version 40 2024-03-21T11:43:51.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.491+0000 7f444b3a4740 1 -- 172.21.15.47:0/3761348290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a648944820 con 0x55a6488d89e0 2024-03-21T11:43:51.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.493+0000 7f4423400640 1 -- 172.21.15.47:0/3761348290 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44300a7800 con 0x55a6488d89e0 2024-03-21T11:43:51.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.493+0000 7f4423400640 1 -- 172.21.15.47:0/3761348290 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f443800b040 con 0x55a6488d89e0 2024-03-21T11:43:51.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.493+0000 7f444b3a4740 10 client.21534 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:43:51.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.493+0000 7f444b3a4740 10 client.21534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:43:51.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.493+0000 7f444b3a4740 20 client.21534 populate_metadata read hostname 'smithi047' 2024-03-21T11:43:51.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.493+0000 7f444b3a4740 1 --2- 172.21.15.47:0/3761348290 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a6488e8c90 0x55a648946f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:43:51.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.493+0000 7f444b3a4740 4 client.21534 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:43:51.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.493+0000 7f444b3a4740 1 -- 172.21.15.47:0/3761348290 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a64883fae0 con 0x55a6488e8c90 2024-03-21T11:43:51.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.494+0000 7f4445c00640 1 --2- 172.21.15.47:0/3761348290 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a6488e8c90 0x55a648946f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:43:51.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.495+0000 7f4445c00640 1 --2- 172.21.15.47:0/3761348290 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a6488e8c90 0x55a648946f70 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=261bb68b326c8f91 server_cookie=dc7ae0e71c9eec11 in_seq=0 out_seq=0 2024-03-21T11:43:51.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.495+0000 7f4423400640 10 client.21534 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:51.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.503+0000 7f4423400640 1 -- 172.21.15.47:0/3761348290 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55a64883fae0 con 0x55a6488e8c90 2024-03-21T11:43:51.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.503+0000 7f4423400640 10 client.21534 handle_command_reply: tid=0 2024-03-21T11:43:51.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.503+0000 7f444b3a4740 1 client.21534 shutdown 2024-03-21T11:43:51.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.505+0000 7f444b3a4740 1 -- 172.21.15.47:0/3761348290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d89e0 msgr2=0x55a6488e3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.505+0000 7f444b3a4740 1 --2- 172.21.15.47:0/3761348290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d89e0 0x55a6488e3240 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4438009ea0 tx=0x7f443800ccc0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.507+0000 7f444b3a4740 1 -- 172.21.15.47:0/3761348290 shutdown_connections 2024-03-21T11:43:51.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.507+0000 7f444b3a4740 1 --2- 172.21.15.47:0/3761348290 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a6488e8c90 0x55a648946f70 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.508+0000 7f444b3a4740 1 --2- 172.21.15.47:0/3761348290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6488d89e0 0x55a6488e3240 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.508+0000 7f444b3a4740 1 --2- 172.21.15.47:0/3761348290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6488d3870 0x55a6488e22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.508+0000 7f444b3a4740 1 --2- 172.21.15.47:0/3761348290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a6488d2e80 0x55a6488e1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.508+0000 7f4423400640 0 client.21534 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:43:51.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.508+0000 7f444b3a4740 1 -- 172.21.15.47:0/3761348290 >> 172.21.15.47:0/3761348290 conn(0x55a64889b200 msgr2=0x55a6488d1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:51.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.508+0000 7f444b3a4740 1 -- 172.21.15.47:0/3761348290 shutdown_connections 2024-03-21T11:43:51.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.508+0000 7f444b3a4740 1 -- 172.21.15.47:0/3761348290 wait complete. 2024-03-21T11:43:51.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.508+0000 7f444b3a4740 20 client.21534 trim_cache size 0 max 16384 2024-03-21T11:43:51.510 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:43:51.510 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (80 inodes in the stack)", 2024-03-21T11:43:51.510 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:43:51.510 INFO:teuthology.orchestra.run.smithi047.stdout: "3de65fb8-df01-4464-a428-6f9f71eac60f": { 2024-03-21T11:43:51.510 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:43:51.510 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "3de65fb8-df01-4464-a428-6f9f71eac60f", 2024-03-21T11:43:51.510 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:43:51.510 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:43:51.510 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:43:51.510 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:43:51.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.509+0000 7f4448600640 1 -- 172.21.15.47:0/726429802 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44280a2fa0 msgr2=0x7f44280a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.509+0000 7f4448600640 1 --2- 172.21.15.47:0/726429802 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44280a2fa0 0x7f44280a5460 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f443402c180 tx=0x7f4434005710 comp rx=0 tx=0).stop 2024-03-21T11:43:51.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.509+0000 7f4448600640 1 -- 172.21.15.47:0/726429802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44401028a0 msgr2=0x7f4440197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:43:51.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.509+0000 7f4448600640 1 --2- 172.21.15.47:0/726429802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44401028a0 0x7f4440197d70 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f443000b2f0 tx=0x7f443000b7b0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.510+0000 7f4448600640 1 -- 172.21.15.47:0/726429802 shutdown_connections 2024-03-21T11:43:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.510+0000 7f4448600640 1 --2- 172.21.15.47:0/726429802 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f44280a2fa0 0x7f44280a5460 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.510+0000 7f4448600640 1 --2- 172.21.15.47:0/726429802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4440103260 0x7f444019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.510+0000 7f4448600640 1 --2- 172.21.15.47:0/726429802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44401028a0 0x7f4440197d70 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.510+0000 7f4448600640 1 --2- 172.21.15.47:0/726429802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44401016b0 0x7f4440197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:43:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.510+0000 7f4448600640 1 -- 172.21.15.47:0/726429802 >> 172.21.15.47:0/726429802 conn(0x7f44400fcc40 msgr2=0x7f4440105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:43:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.510+0000 7f4448600640 1 -- 172.21.15.47:0/726429802 shutdown_connections 2024-03-21T11:43:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:43:51.510+0000 7f4448600640 1 -- 172.21.15.47:0/726429802 wait complete. 2024-03-21T11:43:51.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:51 smithi047 ceph-mon[31351]: pgmap v1015: 129 pgs: 129 active+clean; 583 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 536 op/s 2024-03-21T11:43:51.531 INFO:tasks.cephfs.filesystem:scrub status for tag:3de65fb8-df01-4464-a428-6f9f71eac60f - {'path': '/', 'tag': '3de65fb8-df01-4464-a428-6f9f71eac60f', 'options': 'recursive,force'} 2024-03-21T11:43:51.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:51 smithi100 ceph-mon[36245]: pgmap v1015: 129 pgs: 129 active+clean; 583 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 536 op/s 2024-03-21T11:43:51.968 INFO:tasks.workunit.client.0.smithi047.stdout: 10 175347 22.92 MB/sec execute 275 sec latency 84.691 ms 2024-03-21T11:43:52.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:43:52.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:52.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:52.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:52.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:52.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:52.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:52.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:52.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:43:52.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:52 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:43:52.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:52.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:52.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:52.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:52.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:52.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:52.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:52.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:43:52.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:52 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:43:52.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:52.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:52.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:52.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:52.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:52.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:52.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:52.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:43:52.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:52 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:52.968 INFO:tasks.workunit.client.0.smithi047.stdout: 10 175764 22.94 MB/sec execute 276 sec latency 311.621 ms 2024-03-21T11:43:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:53 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:53 smithi102 ceph-mon[36748]: pgmap v1016: 129 pgs: 129 active+clean; 583 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 494 op/s 2024-03-21T11:43:53.544 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:53 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:53.544 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:53 smithi100 ceph-mon[36245]: pgmap v1016: 129 pgs: 129 active+clean; 583 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 494 op/s 2024-03-21T11:43:53.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:53 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:43:53.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:53 smithi047 ceph-mon[31351]: pgmap v1016: 129 pgs: 129 active+clean; 583 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 494 op/s 2024-03-21T11:43:53.968 INFO:tasks.workunit.client.0.smithi047.stdout: 10 176093 22.90 MB/sec execute 277 sec latency 94.600 ms 2024-03-21T11:43:54.968 INFO:tasks.workunit.client.0.smithi047.stdout: 10 176585 22.92 MB/sec execute 278 sec latency 54.042 ms 2024-03-21T11:43:55.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:55 smithi102 ceph-mon[36748]: pgmap v1017: 129 pgs: 129 active+clean; 581 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.8 MiB/s wr, 667 op/s 2024-03-21T11:43:55.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:55 smithi047 ceph-mon[31351]: pgmap v1017: 129 pgs: 129 active+clean; 581 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.8 MiB/s wr, 667 op/s 2024-03-21T11:43:55.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:55 smithi100 ceph-mon[36245]: pgmap v1017: 129 pgs: 129 active+clean; 581 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.8 MiB/s wr, 667 op/s 2024-03-21T11:43:55.968 INFO:tasks.workunit.client.0.smithi047.stdout: 10 177042 22.95 MB/sec execute 279 sec latency 90.428 ms 2024-03-21T11:43:56.969 INFO:tasks.workunit.client.0.smithi047.stdout: 10 177649 22.95 MB/sec execute 280 sec latency 58.721 ms 2024-03-21T11:43:57.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:57 smithi102 ceph-mon[36748]: pgmap v1018: 129 pgs: 129 active+clean; 582 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.0 MiB/s wr, 614 op/s 2024-03-21T11:43:57.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:57 smithi047 ceph-mon[31351]: pgmap v1018: 129 pgs: 129 active+clean; 582 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.0 MiB/s wr, 614 op/s 2024-03-21T11:43:57.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:57 smithi100 ceph-mon[36245]: pgmap v1018: 129 pgs: 129 active+clean; 582 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.0 MiB/s wr, 614 op/s 2024-03-21T11:43:57.969 INFO:tasks.workunit.client.0.smithi047.stdout: 10 177839 22.90 MB/sec execute 281 sec latency 413.694 ms 2024-03-21T11:43:58.970 INFO:tasks.workunit.client.0.smithi047.stdout: 10 178152 22.89 MB/sec execute 282 sec latency 420.504 ms 2024-03-21T11:43:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:43:59 smithi102 ceph-mon[36748]: pgmap v1019: 129 pgs: 129 active+clean; 583 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 605 op/s 2024-03-21T11:43:59.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:43:59 smithi047 ceph-mon[31351]: pgmap v1019: 129 pgs: 129 active+clean; 583 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 605 op/s 2024-03-21T11:43:59.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:43:59 smithi100 ceph-mon[36245]: pgmap v1019: 129 pgs: 129 active+clean; 583 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 605 op/s 2024-03-21T11:43:59.970 INFO:tasks.workunit.client.0.smithi047.stdout: 10 178630 22.91 MB/sec execute 283 sec latency 81.690 ms 2024-03-21T11:44:00.970 INFO:tasks.workunit.client.0.smithi047.stdout: 10 179216 22.92 MB/sec execute 284 sec latency 111.915 ms 2024-03-21T11:44:01.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:01 smithi102 ceph-mon[36748]: pgmap v1020: 129 pgs: 129 active+clean; 588 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.6 MiB/s wr, 625 op/s 2024-03-21T11:44:01.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:01 smithi047 ceph-mon[31351]: pgmap v1020: 129 pgs: 129 active+clean; 588 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.6 MiB/s wr, 625 op/s 2024-03-21T11:44:01.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:01 smithi100 ceph-mon[36245]: pgmap v1020: 129 pgs: 129 active+clean; 588 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.6 MiB/s wr, 625 op/s 2024-03-21T11:44:01.970 INFO:tasks.workunit.client.0.smithi047.stdout: 10 179751 22.94 MB/sec execute 285 sec latency 61.789 ms 2024-03-21T11:44:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.912+0000 7f9834a00640 1 -- 172.21.15.47:0/530392215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9830104a70 msgr2=0x7f9830106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.912+0000 7f9834a00640 1 --2- 172.21.15.47:0/530392215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9830104a70 0x7f9830106e60 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f981c009900 tx=0x7f981c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:44:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.913+0000 7f9834a00640 1 -- 172.21.15.47:0/530392215 shutdown_connections 2024-03-21T11:44:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.913+0000 7f9834a00640 1 --2- 172.21.15.47:0/530392215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98301073a0 0x7f98301097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.913+0000 7f9834a00640 1 --2- 172.21.15.47:0/530392215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9830104a70 0x7f9830106e60 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.913+0000 7f9834a00640 1 --2- 172.21.15.47:0/530392215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9830102140 0x7f9830104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.913+0000 7f9834a00640 1 -- 172.21.15.47:0/530392215 >> 172.21.15.47:0/530392215 conn(0x7f98300fbb30 msgr2=0x7f98300fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.915+0000 7f9834a00640 1 -- 172.21.15.47:0/530392215 shutdown_connections 2024-03-21T11:44:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.915+0000 7f9834a00640 1 -- 172.21.15.47:0/530392215 wait complete. 2024-03-21T11:44:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.917+0000 7f9834a00640 1 Processor -- start 2024-03-21T11:44:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.917+0000 7f9834a00640 1 -- start start 2024-03-21T11:44:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.918+0000 7f9834a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9830102140 0x7f98301963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.918+0000 7f9834a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9830104a70 0x7f9830196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.918+0000 7f9834a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98301073a0 0x7f9830192f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.918+0000 7f9834a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983010d3a0 con 0x7f98301073a0 2024-03-21T11:44:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.918+0000 7f9834a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983010d220 con 0x7f9830104a70 2024-03-21T11:44:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.918+0000 7f9834a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983010d520 con 0x7f9830102140 2024-03-21T11:44:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.918+0000 7f982d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9830104a70 0x7f9830196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.918+0000 7f982de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9830102140 0x7f98301963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.918+0000 7f982d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9830104a70 0x7f9830196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49650/0 (socket says 172.21.15.47:49650) 2024-03-21T11:44:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.918+0000 7f982d400640 1 -- 172.21.15.47:0/1560760473 learned_addr learned my addr 172.21.15.47:0/1560760473 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.919+0000 7f982d400640 1 -- 172.21.15.47:0/1560760473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9830102140 msgr2=0x7f98301963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.919+0000 7f982d400640 1 --2- 172.21.15.47:0/1560760473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9830102140 0x7f98301963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.919+0000 7f982d400640 1 -- 172.21.15.47:0/1560760473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98301073a0 msgr2=0x7f9830192f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.919+0000 7f982d400640 1 --2- 172.21.15.47:0/1560760473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98301073a0 0x7f9830192f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.919+0000 7f982d400640 1 -- 172.21.15.47:0/1560760473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9830193820 con 0x7f9830104a70 2024-03-21T11:44:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.919+0000 7f982de00640 1 --2- 172.21.15.47:0/1560760473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9830102140 0x7f98301963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.922+0000 7f982d400640 1 --2- 172.21.15.47:0/1560760473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9830104a70 0x7f9830196930 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f981c004f50 tx=0x7f981c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.923+0000 7f9826a00640 1 -- 172.21.15.47:0/1560760473 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c041070 con 0x7f9830104a70 2024-03-21T11:44:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.923+0000 7f9834a00640 1 -- 172.21.15.47:0/1560760473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98301a5e50 con 0x7f9830104a70 2024-03-21T11:44:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.923+0000 7f9826a00640 1 -- 172.21.15.47:0/1560760473 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f981c005410 con 0x7f9830104a70 2024-03-21T11:44:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.923+0000 7f9826a00640 1 -- 172.21.15.47:0/1560760473 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c0346a0 con 0x7f9830104a70 2024-03-21T11:44:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.923+0000 7f9834a00640 1 -- 172.21.15.47:0/1560760473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98301a6200 con 0x7f9830104a70 2024-03-21T11:44:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.932+0000 7f9826a00640 1 -- 172.21.15.47:0/1560760473 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f981c044070 con 0x7f9830104a70 2024-03-21T11:44:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.932+0000 7f9837824740 1 Processor -- start 2024-03-21T11:44:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.933+0000 7f9837824740 1 -- start start 2024-03-21T11:44:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.935+0000 7f9826a00640 1 --2- 172.21.15.47:0/1560760473 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98140a30c0 0x7f98140a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.935+0000 7f9826a00640 1 -- 172.21.15.47:0/1560760473 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f981c0f1af0 con 0x7f9830104a70 2024-03-21T11:44:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.935+0000 7f9837824740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e91e0 0x55f9161e9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.935+0000 7f9837824740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e9bd0 0x55f9161ee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.935+0000 7f9837824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 0x55f9161f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.935+0000 7f9837824740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9161ac3b0 con 0x55f9161e91e0 2024-03-21T11:44:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.935+0000 7f9837824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91614e830 con 0x55f9161eed40 2024-03-21T11:44:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.935+0000 7f9837824740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9160f75d0 con 0x55f9161e9bd0 2024-03-21T11:44:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.936+0000 7f982d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e9bd0 0x55f9161ee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.936+0000 7f982e800640 1 --2- 172.21.15.47:0/1560760473 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98140a30c0 0x7f98140a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.936+0000 7f982d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e9bd0 0x55f9161ee6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56290/0 (socket says 172.21.15.47:56290) 2024-03-21T11:44:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.936+0000 7f982d400640 1 -- 172.21.15.47:0/2339520252 learned_addr learned my addr 172.21.15.47:0/2339520252 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.936+0000 7f982e800640 1 --2- 172.21.15.47:0/2339520252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 0x55f9161f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.936+0000 7f982d400640 1 -- 172.21.15.47:0/2339520252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 msgr2=0x55f9161f11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.936+0000 7f982d400640 1 --2- 172.21.15.47:0/2339520252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 0x55f9161f11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.936+0000 7f982d400640 1 -- 172.21.15.47:0/2339520252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e91e0 msgr2=0x55f9161e9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.936+0000 7f982d400640 1 --2- 172.21.15.47:0/2339520252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e91e0 0x55f9161e9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.937+0000 7f982d400640 1 -- 172.21.15.47:0/2339520252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91610f2c0 con 0x55f9161e9bd0 2024-03-21T11:44:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.937+0000 7f982d400640 1 --2- 172.21.15.47:0/2339520252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e9bd0 0x55f9161ee6d0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f981c0ba290 tx=0x7f981c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba09604192fa2373 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.937+0000 7f981b000640 1 -- 172.21.15.47:0/2339520252 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c04b040 con 0x55f9161e9bd0 2024-03-21T11:44:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.937+0000 7f981b000640 1 -- 172.21.15.47:0/2339520252 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f981c004c80 con 0x55f9161e9bd0 2024-03-21T11:44:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.937+0000 7f981b000640 1 -- 172.21.15.47:0/2339520252 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c0ef310 con 0x55f9161e9bd0 2024-03-21T11:44:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.938+0000 7f982e800640 1 --2- 172.21.15.47:0/1560760473 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98140a30c0 0x7f98140a5580 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9828009fd0 tx=0x7f9828009330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.938+0000 7f9837824740 1 -- 172.21.15.47:0/2339520252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e9bd0 msgr2=0x55f9161ee6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.938+0000 7f9837824740 1 --2- 172.21.15.47:0/2339520252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e9bd0 0x55f9161ee6d0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f981c0ba290 tx=0x7f981c005fd0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.939+0000 7f9837824740 1 -- 172.21.15.47:0/2339520252 shutdown_connections 2024-03-21T11:44:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.939+0000 7f9837824740 1 --2- 172.21.15.47:0/2339520252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 0x55f9161f11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.939+0000 7f9837824740 1 --2- 172.21.15.47:0/2339520252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e9bd0 0x55f9161ee6d0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.939+0000 7f9837824740 1 --2- 172.21.15.47:0/2339520252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e91e0 0x55f9161e9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.939+0000 7f9837824740 1 -- 172.21.15.47:0/2339520252 >> 172.21.15.47:0/2339520252 conn(0x55f9161e4750 msgr2=0x55f9161e4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.940+0000 7f9837824740 1 -- 172.21.15.47:0/2339520252 shutdown_connections 2024-03-21T11:44:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.940+0000 7f9837824740 1 -- 172.21.15.47:0/2339520252 wait complete. 2024-03-21T11:44:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.941+0000 7f9837824740 1 -- start start 2024-03-21T11:44:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.941+0000 7f9837824740 1 Processor -- start 2024-03-21T11:44:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.944+0000 7f9837824740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e91e0 0x55f9161f3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.944+0000 7f9837824740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e9bd0 0x55f9161f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.944+0000 7f9837824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 0x55f9161fa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.944+0000 7f982de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e91e0 0x55f9161f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.945+0000 7f982e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 0x55f9161fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.946+0000 7f982de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e91e0 0x55f9161f3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56296/0 (socket says 172.21.15.47:56296) 2024-03-21T11:44:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.946+0000 7f982de00640 1 -- 172.21.15.47:0/1714747025 learned_addr learned my addr 172.21.15.47:0/1714747025 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.945+0000 7f982d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e9bd0 0x55f9161f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.947+0000 7f9837824740 1 -- 172.21.15.47:0/1714747025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91610aa20 con 0x55f9161e9bd0 2024-03-21T11:44:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.947+0000 7f9837824740 1 -- 172.21.15.47:0/1714747025 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91614a8d0 con 0x55f9161eed40 2024-03-21T11:44:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.947+0000 7f9837824740 1 -- 172.21.15.47:0/1714747025 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9161f1ab0 con 0x55f9161e91e0 2024-03-21T11:44:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.948+0000 7f9837824740 10 client.0 _ll_register_callbacks cb 0x55f9161471c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.948+0000 7f982e800640 1 -- 172.21.15.47:0/1714747025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e91e0 msgr2=0x55f9161f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.948+0000 7f982e800640 1 --2- 172.21.15.47:0/1714747025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e91e0 0x55f9161f3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.948+0000 7f982e800640 1 -- 172.21.15.47:0/1714747025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e9bd0 msgr2=0x55f9161f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.948+0000 7f982e800640 1 --2- 172.21.15.47:0/1714747025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e9bd0 0x55f9161f35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.948+0000 7f982e800640 1 -- 172.21.15.47:0/1714747025 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f915e63210 con 0x55f9161eed40 2024-03-21T11:44:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.948+0000 7f982d400640 1 --2- 172.21.15.47:0/1714747025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e9bd0 0x55f9161f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:44:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.951+0000 7f982e800640 1 --2- 172.21.15.47:0/1714747025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 0x55f9161fa5e0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9828013840 tx=0x7f9828005e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6a6b3d1ba378b50 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.951+0000 7f980f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:44:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.952+0000 7f980f400640 1 -- 172.21.15.47:0/1714747025 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828024070 con 0x55f9161eed40 2024-03-21T11:44:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.952+0000 7f980f400640 1 -- 172.21.15.47:0/1714747025 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9828010d60 con 0x55f9161eed40 2024-03-21T11:44:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.952+0000 7f9837824740 1 -- 172.21.15.47:0/1714747025 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f981c004c80 con 0x55f9161eed40 2024-03-21T11:44:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.953+0000 7f980f400640 1 -- 172.21.15.47:0/1714747025 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9828004b20 con 0x55f9161eed40 2024-03-21T11:44:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.953+0000 7f9837824740 10 client.32192 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.953+0000 7f9837824740 1 -- 172.21.15.47:0/1714747025 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f915f2f620 con 0x55f9161eed40 2024-03-21T11:44:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.955+0000 7f980f400640 1 -- 172.21.15.47:0/1714747025 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f981c004c80 con 0x55f9161eed40 2024-03-21T11:44:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.955+0000 7f980f400640 1 -- 172.21.15.47:0/1714747025 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f982800f040 con 0x55f9161eed40 2024-03-21T11:44:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.955+0000 7f9837824740 10 client.32192 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.955+0000 7f9837824740 10 client.32192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.955+0000 7f9837824740 20 client.32192 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.955+0000 7f9837824740 1 --2- 172.21.15.47:0/1714747025 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f9161fee40 0x55f916201230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.955+0000 7f9837824740 4 client.32192 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.955+0000 7f9837824740 1 -- 172.21.15.47:0/1714747025 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f91615bd70 con 0x55f9161fee40 2024-03-21T11:44:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.957+0000 7f982de00640 1 --2- 172.21.15.47:0/1714747025 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f9161fee40 0x55f916201230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.963+0000 7f982de00640 1 --2- 172.21.15.47:0/1714747025 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f9161fee40 0x55f916201230 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f2cc1da0a2d4b0 server_cookie=52ef98911455a895 in_seq=0 out_seq=0 2024-03-21T11:44:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.964+0000 7f980f400640 10 client.32192 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:02.970 INFO:tasks.workunit.client.0.smithi047.stdout: 10 180151 22.95 MB/sec execute 286 sec latency 342.948 ms 2024-03-21T11:44:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.973+0000 7f980f400640 1 -- 172.21.15.47:0/1714747025 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f91615bd70 con 0x55f9161fee40 2024-03-21T11:44:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.973+0000 7f980f400640 10 client.32192 handle_command_reply: tid=0 2024-03-21T11:44:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.973+0000 7f9837824740 1 client.32192 shutdown 2024-03-21T11:44:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.974+0000 7f9837824740 1 -- 172.21.15.47:0/1714747025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 msgr2=0x55f9161fa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.974+0000 7f9837824740 1 --2- 172.21.15.47:0/1714747025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 0x55f9161fa5e0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9828013840 tx=0x7f9828005e90 comp rx=0 tx=0).stop 2024-03-21T11:44:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.974+0000 7f9837824740 1 -- 172.21.15.47:0/1714747025 shutdown_connections 2024-03-21T11:44:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.974+0000 7f9837824740 1 --2- 172.21.15.47:0/1714747025 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f9161fee40 0x55f916201230 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.974+0000 7f9837824740 1 --2- 172.21.15.47:0/1714747025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 0x55f9161fa5e0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.974+0000 7f9837824740 1 --2- 172.21.15.47:0/1714747025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e9bd0 0x55f9161f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.974+0000 7f9837824740 1 --2- 172.21.15.47:0/1714747025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e91e0 0x55f9161f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.974+0000 7f9837824740 1 -- 172.21.15.47:0/1714747025 >> 172.21.15.47:0/1714747025 conn(0x55f9161e4750 msgr2=0x55f9161e3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.974+0000 7f980f400640 0 client.32192 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.974+0000 7f9837824740 1 -- 172.21.15.47:0/1714747025 shutdown_connections 2024-03-21T11:44:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.975+0000 7f9837824740 1 -- 172.21.15.47:0/1714747025 wait complete. 2024-03-21T11:44:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:02.975+0000 7f9837824740 20 client.32192 trim_cache size 0 max 16384 2024-03-21T11:44:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.003+0000 7f9837824740 1 Processor -- start 2024-03-21T11:44:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.003+0000 7f9837824740 1 -- start start 2024-03-21T11:44:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.003+0000 7f9837824740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e91e0 0x55f9161f3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.003+0000 7f9837824740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e9bd0 0x55f9161f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.004+0000 7f9837824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 0x55f9161fa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.004+0000 7f9837824740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98301b05f0 con 0x55f9161e91e0 2024-03-21T11:44:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.004+0000 7f9837824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98301b1f70 con 0x55f9161eed40 2024-03-21T11:44:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.004+0000 7f9837824740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98301b0c50 con 0x55f9161e9bd0 2024-03-21T11:44:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.004+0000 7f982d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e9bd0 0x55f9161f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.004+0000 7f982d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e9bd0 0x55f9161f35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56312/0 (socket says 172.21.15.47:56312) 2024-03-21T11:44:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.004+0000 7f982d400640 1 -- 172.21.15.47:0/1566858169 learned_addr learned my addr 172.21.15.47:0/1566858169 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.004+0000 7f982de00640 1 --2- 172.21.15.47:0/1566858169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e91e0 0x55f9161f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.005+0000 7f982de00640 1 -- 172.21.15.47:0/1566858169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e9bd0 msgr2=0x55f9161f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.005+0000 7f982de00640 1 --2- 172.21.15.47:0/1566858169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e9bd0 0x55f9161f35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.005+0000 7f982de00640 1 -- 172.21.15.47:0/1566858169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 msgr2=0x55f9161fa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.005+0000 7f982de00640 1 --2- 172.21.15.47:0/1566858169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 0x55f9161fa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.005+0000 7f982de00640 1 -- 172.21.15.47:0/1566858169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9161ea9e0 con 0x55f9161e91e0 2024-03-21T11:44:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.006+0000 7f982de00640 1 --2- 172.21.15.47:0/1566858169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e91e0 0x55f9161f3060 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9820000c00 tx=0x7f9820009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a407ec69784d71b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.006+0000 7f981a600640 1 -- 172.21.15.47:0/1566858169 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820016c50 con 0x55f9161e91e0 2024-03-21T11:44:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.007+0000 7f981a600640 1 -- 172.21.15.47:0/1566858169 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9820016df0 con 0x55f9161e91e0 2024-03-21T11:44:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.007+0000 7f981a600640 1 -- 172.21.15.47:0/1566858169 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820013720 con 0x55f9161e91e0 2024-03-21T11:44:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.007+0000 7f9837824740 1 -- 172.21.15.47:0/1566858169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e91e0 msgr2=0x55f9161f3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.007+0000 7f9837824740 1 --2- 172.21.15.47:0/1566858169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e91e0 0x55f9161f3060 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9820000c00 tx=0x7f9820009800 comp rx=0 tx=0).stop 2024-03-21T11:44:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.008+0000 7f9837824740 1 -- 172.21.15.47:0/1566858169 shutdown_connections 2024-03-21T11:44:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.008+0000 7f9837824740 1 --2- 172.21.15.47:0/1566858169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161eed40 0x55f9161fa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.008+0000 7f9837824740 1 --2- 172.21.15.47:0/1566858169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e9bd0 0x55f9161f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.008+0000 7f9837824740 1 --2- 172.21.15.47:0/1566858169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161e91e0 0x55f9161f3060 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.008+0000 7f9837824740 1 -- 172.21.15.47:0/1566858169 >> 172.21.15.47:0/1566858169 conn(0x55f9161e4750 msgr2=0x55f9161f1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.008+0000 7f9837824740 1 -- 172.21.15.47:0/1566858169 shutdown_connections 2024-03-21T11:44:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.009+0000 7f9837824740 1 -- 172.21.15.47:0/1566858169 wait complete. 2024-03-21T11:44:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.009+0000 7f9837824740 1 -- start start 2024-03-21T11:44:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.009+0000 7f9837824740 1 Processor -- start 2024-03-21T11:44:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.010+0000 7f9837824740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e91e0 0x55f9161f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.011+0000 7f9837824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161e9bd0 0x55f9161f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.011+0000 7f9837824740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161eed40 0x55f9161f8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.011+0000 7f982de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e91e0 0x55f9161f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.011+0000 7f982de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e91e0 0x55f9161f79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56322/0 (socket says 172.21.15.47:56322) 2024-03-21T11:44:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.011+0000 7f982d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161e9bd0 0x55f9161f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.011+0000 7f9837824740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98301b1910 con 0x55f9161eed40 2024-03-21T11:44:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.011+0000 7f9837824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98301b12b0 con 0x55f9161e9bd0 2024-03-21T11:44:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.011+0000 7f9837824740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98301ae7b0 con 0x55f9161e91e0 2024-03-21T11:44:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.011+0000 7f982d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161e9bd0 0x55f9161f7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49700/0 (socket says 172.21.15.47:49700) 2024-03-21T11:44:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.011+0000 7f982d400640 1 -- 172.21.15.47:0/2412292444 learned_addr learned my addr 172.21.15.47:0/2412292444 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.011+0000 7f9837824740 10 client.0 _ll_register_callbacks cb 0x7f9828013630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.011+0000 7f982e800640 1 --2- 172.21.15.47:0/2412292444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161eed40 0x55f9161f8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.012+0000 7f982d400640 1 -- 172.21.15.47:0/2412292444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e91e0 msgr2=0x55f9161f79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.012+0000 7f982d400640 1 --2- 172.21.15.47:0/2412292444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e91e0 0x55f9161f79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.012+0000 7f982d400640 1 -- 172.21.15.47:0/2412292444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161eed40 msgr2=0x55f9161f8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.012+0000 7f982d400640 1 --2- 172.21.15.47:0/2412292444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161eed40 0x55f9161f8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.012+0000 7f982d400640 1 -- 172.21.15.47:0/2412292444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9161f94e0 con 0x55f9161e9bd0 2024-03-21T11:44:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.012+0000 7f982e800640 1 --2- 172.21.15.47:0/2412292444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161eed40 0x55f9161f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:44:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.012+0000 7f982de00640 1 --2- 172.21.15.47:0/2412292444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e91e0 0x55f9161f79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.013+0000 7f982d400640 1 --2- 172.21.15.47:0/2412292444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161e9bd0 0x55f9161f7f30 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f981c0bc0e0 tx=0x7f981c005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7ab68c00e29f536 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.013+0000 7f980f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:44:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.013+0000 7f980f400640 1 -- 172.21.15.47:0/2412292444 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c0a1060 con 0x55f9161e9bd0 2024-03-21T11:44:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.013+0000 7f980f400640 1 -- 172.21.15.47:0/2412292444 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f981c0be090 con 0x55f9161e9bd0 2024-03-21T11:44:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.014+0000 7f9837824740 1 -- 172.21.15.47:0/2412292444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9820016df0 con 0x55f9161e9bd0 2024-03-21T11:44:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.014+0000 7f980f400640 1 -- 172.21.15.47:0/2412292444 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f981c0da070 con 0x55f9161e9bd0 2024-03-21T11:44:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.014+0000 7f9837824740 10 client.32195 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.014+0000 7f9837824740 1 -- 172.21.15.47:0/2412292444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f9161fa7f0 con 0x55f9161e9bd0 2024-03-21T11:44:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.014+0000 7f980f400640 1 -- 172.21.15.47:0/2412292444 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9820016df0 con 0x55f9161e9bd0 2024-03-21T11:44:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.015+0000 7f980f400640 1 -- 172.21.15.47:0/2412292444 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f981c0a4050 con 0x55f9161e9bd0 2024-03-21T11:44:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.018+0000 7f9837824740 10 client.32195 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.018+0000 7f9837824740 10 client.32195 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.018+0000 7f9837824740 20 client.32195 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.018+0000 7f9837824740 1 --2- 172.21.15.47:0/2412292444 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f91625abe0 0x55f91625d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.018+0000 7f9837824740 4 client.32195 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:44:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.018+0000 7f9837824740 1 -- 172.21.15.47:0/2412292444 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f9161560c0 con 0x55f91625abe0 2024-03-21T11:44:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.018+0000 7f982de00640 1 --2- 172.21.15.47:0/2412292444 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f91625abe0 0x55f91625d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.021+0000 7f980f400640 10 client.32195 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:03.021+0000 7f982de00640 1 --2- 172.21.15.47:0/2412292444 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f91625abe0 0x55f91625d000 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fba9c0c09cc97d9 server_cookie=b75acd8ef772619a in_seq=0 out_seq=0 2024-03-21T11:44:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:03 smithi102 ceph-mon[36748]: pgmap v1021: 129 pgs: 129 active+clean; 589 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.1 MiB/s wr, 582 op/s 2024-03-21T11:44:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:03 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:44:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:03 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:44:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:03 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:44:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:03 smithi047 ceph-mon[31351]: pgmap v1021: 129 pgs: 129 active+clean; 589 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.1 MiB/s wr, 582 op/s 2024-03-21T11:44:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:03 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:44:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:03 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:44:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:03 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:44:03.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:03 smithi100 ceph-mon[36245]: pgmap v1021: 129 pgs: 129 active+clean; 589 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.1 MiB/s wr, 582 op/s 2024-03-21T11:44:03.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:03 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:44:03.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:03 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:44:03.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:03 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:44:03.970 INFO:tasks.workunit.client.0.smithi047.stdout: 10 180265 22.89 MB/sec execute 287 sec latency 660.772 ms 2024-03-21T11:44:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.182+0000 7f980f400640 1 -- 172.21.15.47:0/2412292444 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f9161560c0 con 0x55f91625abe0 2024-03-21T11:44:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.182+0000 7f980f400640 10 client.32195 handle_command_reply: tid=0 2024-03-21T11:44:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.182+0000 7f9837824740 1 client.32195 shutdown 2024-03-21T11:44:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.182+0000 7f9837824740 1 -- 172.21.15.47:0/2412292444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161e9bd0 msgr2=0x55f9161f7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.182+0000 7f9837824740 1 --2- 172.21.15.47:0/2412292444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161e9bd0 0x55f9161f7f30 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f981c0bc0e0 tx=0x7f981c005b50 comp rx=0 tx=0).stop 2024-03-21T11:44:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.183+0000 7f9837824740 1 -- 172.21.15.47:0/2412292444 shutdown_connections 2024-03-21T11:44:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.183+0000 7f9837824740 1 --2- 172.21.15.47:0/2412292444 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f91625abe0 0x55f91625d000 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.183+0000 7f9837824740 1 --2- 172.21.15.47:0/2412292444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9161eed40 0x55f9161f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.183+0000 7f9837824740 1 --2- 172.21.15.47:0/2412292444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9161e9bd0 0x55f9161f7f30 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.183+0000 7f9837824740 1 --2- 172.21.15.47:0/2412292444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9161e91e0 0x55f9161f79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.183+0000 7f9837824740 1 -- 172.21.15.47:0/2412292444 >> 172.21.15.47:0/2412292444 conn(0x55f9161e4750 msgr2=0x55f9161eaef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.183+0000 7f980f400640 0 client.32195 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.183+0000 7f9837824740 1 -- 172.21.15.47:0/2412292444 shutdown_connections 2024-03-21T11:44:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.183+0000 7f9837824740 1 -- 172.21.15.47:0/2412292444 wait complete. 2024-03-21T11:44:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.184+0000 7f9837824740 20 client.32195 trim_cache size 0 max 16384 2024-03-21T11:44:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:44:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:44:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:44:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:44:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.185+0000 7f9834a00640 1 -- 172.21.15.47:0/1560760473 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98140a30c0 msgr2=0x7f98140a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.185+0000 7f9834a00640 1 --2- 172.21.15.47:0/1560760473 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98140a30c0 0x7f98140a5580 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9828009fd0 tx=0x7f9828009330 comp rx=0 tx=0).stop 2024-03-21T11:44:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.185+0000 7f9834a00640 1 -- 172.21.15.47:0/1560760473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9830104a70 msgr2=0x7f9830196930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.185+0000 7f9834a00640 1 --2- 172.21.15.47:0/1560760473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9830104a70 0x7f9830196930 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f981c004f50 tx=0x7f981c0042f0 comp rx=0 tx=0).stop 2024-03-21T11:44:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.188+0000 7f9834a00640 1 -- 172.21.15.47:0/1560760473 shutdown_connections 2024-03-21T11:44:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.188+0000 7f9834a00640 1 --2- 172.21.15.47:0/1560760473 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f98140a30c0 0x7f98140a5580 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.188+0000 7f9834a00640 1 --2- 172.21.15.47:0/1560760473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98301073a0 0x7f9830192f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.188+0000 7f9834a00640 1 --2- 172.21.15.47:0/1560760473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9830104a70 0x7f9830196930 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.188+0000 7f9834a00640 1 --2- 172.21.15.47:0/1560760473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9830102140 0x7f98301963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.188+0000 7f9834a00640 1 -- 172.21.15.47:0/1560760473 >> 172.21.15.47:0/1560760473 conn(0x7f98300fbb30 msgr2=0x7f98301004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.188+0000 7f9834a00640 1 -- 172.21.15.47:0/1560760473 shutdown_connections 2024-03-21T11:44:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:04.189+0000 7f9834a00640 1 -- 172.21.15.47:0/1560760473 wait complete. 2024-03-21T11:44:04.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:04 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:44:04.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:04 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:44:04.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:04 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:44:04.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:04 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:44:04.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:04 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:44:04.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:04 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:44:04.970 INFO:tasks.workunit.client.0.smithi047.stdout: 10 180576 22.88 MB/sec execute 288 sec latency 673.220 ms 2024-03-21T11:44:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:05 smithi102 ceph-mon[36748]: pgmap v1022: 129 pgs: 129 active+clean; 598 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.7 MiB/s wr, 655 op/s 2024-03-21T11:44:05.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:05 smithi047 ceph-mon[31351]: pgmap v1022: 129 pgs: 129 active+clean; 598 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.7 MiB/s wr, 655 op/s 2024-03-21T11:44:05.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:05 smithi100 ceph-mon[36245]: pgmap v1022: 129 pgs: 129 active+clean; 598 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.7 MiB/s wr, 655 op/s 2024-03-21T11:44:05.970 INFO:tasks.workunit.client.0.smithi047.stdout: 10 180955 22.89 MB/sec execute 289 sec latency 163.511 ms 2024-03-21T11:44:06.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:06 smithi047 ceph-mon[31351]: pgmap v1023: 129 pgs: 129 active+clean; 603 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.8 MiB/s wr, 499 op/s 2024-03-21T11:44:06.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:06 smithi100 ceph-mon[36245]: pgmap v1023: 129 pgs: 129 active+clean; 603 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.8 MiB/s wr, 499 op/s 2024-03-21T11:44:06.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:06 smithi102 ceph-mon[36748]: pgmap v1023: 129 pgs: 129 active+clean; 603 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.8 MiB/s wr, 499 op/s 2024-03-21T11:44:06.970 INFO:tasks.workunit.client.0.smithi047.stdout: 10 181363 22.88 MB/sec execute 290 sec latency 91.550 ms 2024-03-21T11:44:07.971 INFO:tasks.workunit.client.0.smithi047.stdout: 10 181648 22.82 MB/sec execute 291 sec latency 220.139 ms 2024-03-21T11:44:08.971 INFO:tasks.workunit.client.0.smithi047.stdout: 10 181853 22.77 MB/sec execute 292 sec latency 364.207 ms 2024-03-21T11:44:09.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:08 smithi047 ceph-mon[31351]: pgmap v1024: 129 pgs: 129 active+clean; 603 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.7 MiB/s wr, 504 op/s 2024-03-21T11:44:09.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:08 smithi100 ceph-mon[36245]: pgmap v1024: 129 pgs: 129 active+clean; 603 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.7 MiB/s wr, 504 op/s 2024-03-21T11:44:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:08 smithi102 ceph-mon[36748]: pgmap v1024: 129 pgs: 129 active+clean; 603 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.7 MiB/s wr, 504 op/s 2024-03-21T11:44:09.971 INFO:tasks.workunit.client.0.smithi047.stdout: 10 182495 22.80 MB/sec execute 293 sec latency 86.201 ms 2024-03-21T11:44:10.971 INFO:tasks.workunit.client.0.smithi047.stdout: 10 183327 22.90 MB/sec execute 294 sec latency 48.034 ms 2024-03-21T11:44:11.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:10 smithi047 ceph-mon[31351]: pgmap v1025: 129 pgs: 129 active+clean; 590 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.7 MiB/s wr, 664 op/s 2024-03-21T11:44:11.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:10 smithi100 ceph-mon[36245]: pgmap v1025: 129 pgs: 129 active+clean; 590 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.7 MiB/s wr, 664 op/s 2024-03-21T11:44:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:10 smithi102 ceph-mon[36748]: pgmap v1025: 129 pgs: 129 active+clean; 590 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.7 MiB/s wr, 664 op/s 2024-03-21T11:44:11.971 INFO:tasks.workunit.client.0.smithi047.stdout: 10 183968 22.97 MB/sec execute 295 sec latency 70.169 ms 2024-03-21T11:44:12.971 INFO:tasks.workunit.client.0.smithi047.stdout: 10 184269 22.95 MB/sec execute 296 sec latency 108.925 ms 2024-03-21T11:44:13.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:12 smithi047 ceph-mon[31351]: pgmap v1026: 129 pgs: 129 active+clean; 592 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.4 MiB/s wr, 566 op/s 2024-03-21T11:44:13.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:12 smithi100 ceph-mon[36245]: pgmap v1026: 129 pgs: 129 active+clean; 592 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.4 MiB/s wr, 566 op/s 2024-03-21T11:44:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:12 smithi102 ceph-mon[36748]: pgmap v1026: 129 pgs: 129 active+clean; 592 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.4 MiB/s wr, 566 op/s 2024-03-21T11:44:13.971 INFO:tasks.workunit.client.0.smithi047.stdout: 10 184439 22.90 MB/sec execute 297 sec latency 521.779 ms 2024-03-21T11:44:14.971 INFO:tasks.workunit.client.0.smithi047.stdout: 10 184740 22.86 MB/sec execute 298 sec latency 91.377 ms 2024-03-21T11:44:15.041 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:14 smithi102 ceph-mon[36748]: pgmap v1027: 129 pgs: 129 active+clean; 602 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 647 op/s 2024-03-21T11:44:15.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:14 smithi047 ceph-mon[31351]: pgmap v1027: 129 pgs: 129 active+clean; 602 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 647 op/s 2024-03-21T11:44:15.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:14 smithi100 ceph-mon[36245]: pgmap v1027: 129 pgs: 129 active+clean; 602 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 647 op/s 2024-03-21T11:44:15.974 INFO:tasks.workunit.client.0.smithi047.stdout: 10 185226 22.87 MB/sec execute 299 sec latency 84.472 ms 2024-03-21T11:44:16.971 INFO:tasks.workunit.client.0.smithi047.stdout: 10 185757 22.88 MB/sec execute 300 sec latency 154.761 ms 2024-03-21T11:44:17.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:16 smithi047 ceph-mon[31351]: pgmap v1028: 129 pgs: 129 active+clean; 607 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 570 op/s 2024-03-21T11:44:17.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:16 smithi100 ceph-mon[36245]: pgmap v1028: 129 pgs: 129 active+clean; 607 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 570 op/s 2024-03-21T11:44:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:16 smithi102 ceph-mon[36748]: pgmap v1028: 129 pgs: 129 active+clean; 607 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 570 op/s 2024-03-21T11:44:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.372+0000 7fe722e00640 1 -- 172.21.15.47:0/3391969401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe71c1028c0 msgr2=0x7fe71c102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.372+0000 7fe722e00640 1 --2- 172.21.15.47:0/3391969401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe71c1028c0 0x7fe71c102d40 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe708009900 tx=0x7fe70802bc30 comp rx=0 tx=0).stop 2024-03-21T11:44:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.372+0000 7fe722e00640 1 -- 172.21.15.47:0/3391969401 shutdown_connections 2024-03-21T11:44:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.372+0000 7fe722e00640 1 --2- 172.21.15.47:0/3391969401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe71c103280 0x7fe71c109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.372+0000 7fe722e00640 1 --2- 172.21.15.47:0/3391969401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe71c1028c0 0x7fe71c102d40 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.372+0000 7fe722e00640 1 --2- 172.21.15.47:0/3391969401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe71c1016d0 0x7fe71c101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.372+0000 7fe722e00640 1 -- 172.21.15.47:0/3391969401 >> 172.21.15.47:0/3391969401 conn(0x7fe71c0fcc60 msgr2=0x7fe71c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.372+0000 7fe722e00640 1 -- 172.21.15.47:0/3391969401 shutdown_connections 2024-03-21T11:44:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.372+0000 7fe722e00640 1 -- 172.21.15.47:0/3391969401 wait complete. 2024-03-21T11:44:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.373+0000 7fe722e00640 1 Processor -- start 2024-03-21T11:44:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.374+0000 7fe722e00640 1 -- start start 2024-03-21T11:44:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.374+0000 7fe722e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe71c1016d0 0x7fe71c197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.374+0000 7fe722e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe71c1028c0 0x7fe71c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.374+0000 7fe722e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe71c103280 0x7fe71c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.374+0000 7fe722e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c10d350 con 0x7fe71c1016d0 2024-03-21T11:44:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.374+0000 7fe722e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c10d1d0 con 0x7fe71c1028c0 2024-03-21T11:44:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.374+0000 7fe722e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c10d4d0 con 0x7fe71c103280 2024-03-21T11:44:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.374+0000 7fe71be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe71c1016d0 0x7fe71c197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.374+0000 7fe71be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe71c1016d0 0x7fe71c197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32936/0 (socket says 172.21.15.47:32936) 2024-03-21T11:44:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.374+0000 7fe71be00640 1 -- 172.21.15.47:0/205895519 learned_addr learned my addr 172.21.15.47:0/205895519 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.374+0000 7fe71b400640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe71c1028c0 0x7fe71c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.375+0000 7fe720e00640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe71c103280 0x7fe71c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.375+0000 7fe71be00640 1 -- 172.21.15.47:0/205895519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe71c103280 msgr2=0x7fe71c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.375+0000 7fe71be00640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe71c103280 0x7fe71c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.375+0000 7fe71be00640 1 -- 172.21.15.47:0/205895519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe71c1028c0 msgr2=0x7fe71c197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.375+0000 7fe71be00640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe71c1028c0 0x7fe71c197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.375+0000 7fe71be00640 1 -- 172.21.15.47:0/205895519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c19f520 con 0x7fe71c1016d0 2024-03-21T11:44:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.375+0000 7fe720e00640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe71c103280 0x7fe71c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:44:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.375+0000 7fe71b400640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe71c1028c0 0x7fe71c197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.375+0000 7fe71be00640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe71c1016d0 0x7fe71c197900 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00d950 tx=0x7fe70c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.376+0000 7fe718c00640 1 -- 172.21.15.47:0/205895519 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c017020 con 0x7fe71c1016d0 2024-03-21T11:44:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.376+0000 7fe718c00640 1 -- 172.21.15.47:0/205895519 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe70c009c30 con 0x7fe71c1016d0 2024-03-21T11:44:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.376+0000 7fe722e00640 1 -- 172.21.15.47:0/205895519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71c19f810 con 0x7fe71c1016d0 2024-03-21T11:44:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.376+0000 7fe718c00640 1 -- 172.21.15.47:0/205895519 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00f720 con 0x7fe71c1016d0 2024-03-21T11:44:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.376+0000 7fe722e00640 1 -- 172.21.15.47:0/205895519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c19fd80 con 0x7fe71c1016d0 2024-03-21T11:44:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.379+0000 7fe718c00640 1 -- 172.21.15.47:0/205895519 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe70c01a020 con 0x7fe71c1016d0 2024-03-21T11:44:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.379+0000 7fe725bce740 1 Processor -- start 2024-03-21T11:44:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.379+0000 7fe725bce740 1 -- start start 2024-03-21T11:44:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.380+0000 7fe725bce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80c1e0 0x55d30e80c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.380+0000 7fe725bce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e80cbd0 0x55d30e8116d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.380+0000 7fe725bce740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e811d40 0x55d30e8141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.380+0000 7fe725bce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30e7cf3b0 con 0x55d30e80cbd0 2024-03-21T11:44:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.380+0000 7fe725bce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30e771830 con 0x55d30e80c1e0 2024-03-21T11:44:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.380+0000 7fe725bce740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30e71a5d0 con 0x55d30e811d40 2024-03-21T11:44:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.381+0000 7fe720e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80c1e0 0x55d30e80c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.381+0000 7fe71b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e811d40 0x55d30e8141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.381+0000 7fe720e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80c1e0 0x55d30e80c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32830/0 (socket says 172.21.15.47:32830) 2024-03-21T11:44:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.381+0000 7fe720e00640 1 -- 172.21.15.47:0/2808680766 learned_addr learned my addr 172.21.15.47:0/2808680766 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.381+0000 7fe71be00640 1 --2- 172.21.15.47:0/2808680766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e80cbd0 0x55d30e8116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.381+0000 7fe720e00640 1 -- 172.21.15.47:0/2808680766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e811d40 msgr2=0x55d30e8141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.381+0000 7fe720e00640 1 --2- 172.21.15.47:0/2808680766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e811d40 0x55d30e8141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.381+0000 7fe720e00640 1 -- 172.21.15.47:0/2808680766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e80cbd0 msgr2=0x55d30e8116d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.381+0000 7fe720e00640 1 --2- 172.21.15.47:0/2808680766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e80cbd0 0x55d30e8116d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.381+0000 7fe720e00640 1 -- 172.21.15.47:0/2808680766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d30e7322c0 con 0x55d30e80c1e0 2024-03-21T11:44:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.381+0000 7fe71be00640 1 --2- 172.21.15.47:0/2808680766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e80cbd0 0x55d30e8116d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:44:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.382+0000 7fe718c00640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe7000a3040 0x7fe7000a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.382+0000 7fe718c00640 1 -- 172.21.15.47:0/205895519 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe70c0cf6e0 con 0x7fe71c1016d0 2024-03-21T11:44:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.382+0000 7fe720e00640 1 --2- 172.21.15.47:0/2808680766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80c1e0 0x55d30e80c600 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe714009fd0 tx=0x7fe714010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b63b8feaa7dfa30c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.382+0000 7fe711600640 1 -- 172.21.15.47:0/2808680766 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714011700 con 0x55d30e80c1e0 2024-03-21T11:44:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.382+0000 7fe711600640 1 -- 172.21.15.47:0/2808680766 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe714011d80 con 0x55d30e80c1e0 2024-03-21T11:44:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.382+0000 7fe711600640 1 -- 172.21.15.47:0/2808680766 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71400e3c0 con 0x55d30e80c1e0 2024-03-21T11:44:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.383+0000 7fe71be00640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe7000a3040 0x7fe7000a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.383+0000 7fe725bce740 1 -- 172.21.15.47:0/2808680766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80c1e0 msgr2=0x55d30e80c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.383+0000 7fe725bce740 1 --2- 172.21.15.47:0/2808680766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80c1e0 0x55d30e80c600 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe714009fd0 tx=0x7fe714010860 comp rx=0 tx=0).stop 2024-03-21T11:44:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.384+0000 7fe725bce740 1 -- 172.21.15.47:0/2808680766 shutdown_connections 2024-03-21T11:44:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.384+0000 7fe725bce740 1 --2- 172.21.15.47:0/2808680766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e811d40 0x55d30e8141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.384+0000 7fe725bce740 1 --2- 172.21.15.47:0/2808680766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e80cbd0 0x55d30e8116d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.384+0000 7fe725bce740 1 --2- 172.21.15.47:0/2808680766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80c1e0 0x55d30e80c600 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.384+0000 7fe725bce740 1 -- 172.21.15.47:0/2808680766 >> 172.21.15.47:0/2808680766 conn(0x55d30e807750 msgr2=0x55d30e807b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.384+0000 7fe725bce740 1 -- 172.21.15.47:0/2808680766 shutdown_connections 2024-03-21T11:44:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.384+0000 7fe71be00640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe7000a3040 0x7fe7000a5500 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe70c09b720 tx=0x7fe70c010e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.384+0000 7fe725bce740 1 -- 172.21.15.47:0/2808680766 wait complete. 2024-03-21T11:44:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.385+0000 7fe725bce740 1 -- start start 2024-03-21T11:44:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.385+0000 7fe725bce740 1 Processor -- start 2024-03-21T11:44:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.386+0000 7fe725bce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80c1e0 0x55d30e81a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.386+0000 7fe725bce740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80cbd0 0x55d30e81ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.386+0000 7fe725bce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e811d40 0x55d30e821c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.386+0000 7fe725bce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30e72da20 con 0x55d30e811d40 2024-03-21T11:44:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.386+0000 7fe725bce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30e76d8d0 con 0x55d30e80c1e0 2024-03-21T11:44:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.386+0000 7fe725bce740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30e8190a0 con 0x55d30e80cbd0 2024-03-21T11:44:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.386+0000 7fe725bce740 10 client.0 _ll_register_callbacks cb 0x55d30e76a1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.386+0000 7fe720e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80c1e0 0x55d30e81a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.386+0000 7fe720e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80c1e0 0x55d30e81a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32834/0 (socket says 172.21.15.47:32834) 2024-03-21T11:44:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.386+0000 7fe720e00640 1 -- 172.21.15.47:0/4193531034 learned_addr learned my addr 172.21.15.47:0/4193531034 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.387+0000 7fe71b400640 1 --2- 172.21.15.47:0/4193531034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80cbd0 0x55d30e81ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.387+0000 7fe720e00640 1 --2- 172.21.15.47:0/4193531034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e811d40 0x55d30e821c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.387+0000 7fe720e00640 1 -- 172.21.15.47:0/4193531034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80cbd0 msgr2=0x55d30e81ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.387+0000 7fe720e00640 1 --2- 172.21.15.47:0/4193531034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80cbd0 0x55d30e81ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.387+0000 7fe720e00640 1 -- 172.21.15.47:0/4193531034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e811d40 msgr2=0x55d30e821c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.387+0000 7fe720e00640 1 --2- 172.21.15.47:0/4193531034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e811d40 0x55d30e821c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.387+0000 7fe720e00640 1 -- 172.21.15.47:0/4193531034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d30e486210 con 0x55d30e80c1e0 2024-03-21T11:44:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.387+0000 7fe71b400640 1 --2- 172.21.15.47:0/4193531034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80cbd0 0x55d30e81ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:44:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.387+0000 7fe720e00640 1 --2- 172.21.15.47:0/4193531034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80c1e0 0x55d30e81a700 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fe714011010 tx=0x7fe714005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29ac2797240664c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.387+0000 7fe706000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:44:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.388+0000 7fe706000640 1 -- 172.21.15.47:0/4193531034 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714023040 con 0x55d30e80c1e0 2024-03-21T11:44:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.388+0000 7fe706000640 1 -- 172.21.15.47:0/4193531034 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe714011600 con 0x55d30e80c1e0 2024-03-21T11:44:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.388+0000 7fe725bce740 1 -- 172.21.15.47:0/4193531034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe714011d80 con 0x55d30e80c1e0 2024-03-21T11:44:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.388+0000 7fe706000640 1 -- 172.21.15.47:0/4193531034 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe714011800 con 0x55d30e80c1e0 2024-03-21T11:44:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.388+0000 7fe725bce740 10 client.32207 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.388+0000 7fe725bce740 1 -- 172.21.15.47:0/4193531034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d30e552620 con 0x55d30e80c1e0 2024-03-21T11:44:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.388+0000 7fe706000640 1 -- 172.21.15.47:0/4193531034 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7140119a0 con 0x55d30e80c1e0 2024-03-21T11:44:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.389+0000 7fe706000640 1 -- 172.21.15.47:0/4193531034 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe714004f10 con 0x55d30e80c1e0 2024-03-21T11:44:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.389+0000 7fe725bce740 10 client.32207 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.389+0000 7fe725bce740 10 client.32207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.389+0000 7fe725bce740 20 client.32207 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.389+0000 7fe725bce740 1 --2- 172.21.15.47:0/4193531034 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d30e826290 0x55d30e828680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.389+0000 7fe725bce740 4 client.32207 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.389+0000 7fe725bce740 1 -- 172.21.15.47:0/4193531034 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe714011d80 con 0x55d30e826290 2024-03-21T11:44:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.390+0000 7fe71b400640 1 --2- 172.21.15.47:0/4193531034 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d30e826290 0x55d30e828680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.390+0000 7fe71b400640 1 --2- 172.21.15.47:0/4193531034 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d30e826290 0x55d30e828680 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e99292058870ba server_cookie=4cb73b3270434963 in_seq=0 out_seq=0 2024-03-21T11:44:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.390+0000 7fe706000640 10 client.32207 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.394+0000 7fe706000640 1 -- 172.21.15.47:0/4193531034 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fe714011d80 con 0x55d30e826290 2024-03-21T11:44:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.394+0000 7fe706000640 10 client.32207 handle_command_reply: tid=0 2024-03-21T11:44:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.394+0000 7fe725bce740 1 client.32207 shutdown 2024-03-21T11:44:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.394+0000 7fe725bce740 1 -- 172.21.15.47:0/4193531034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80c1e0 msgr2=0x55d30e81a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.394+0000 7fe725bce740 1 --2- 172.21.15.47:0/4193531034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80c1e0 0x55d30e81a700 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fe714011010 tx=0x7fe714005c40 comp rx=0 tx=0).stop 2024-03-21T11:44:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.394+0000 7fe725bce740 1 -- 172.21.15.47:0/4193531034 shutdown_connections 2024-03-21T11:44:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.394+0000 7fe725bce740 1 --2- 172.21.15.47:0/4193531034 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d30e826290 0x55d30e828680 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.394+0000 7fe725bce740 1 --2- 172.21.15.47:0/4193531034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e811d40 0x55d30e821c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.394+0000 7fe725bce740 1 --2- 172.21.15.47:0/4193531034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80cbd0 0x55d30e81ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.394+0000 7fe725bce740 1 --2- 172.21.15.47:0/4193531034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80c1e0 0x55d30e81a700 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.394+0000 7fe725bce740 1 -- 172.21.15.47:0/4193531034 >> 172.21.15.47:0/4193531034 conn(0x55d30e807750 msgr2=0x55d30e806950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.394+0000 7fe706000640 0 client.32207 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.395+0000 7fe725bce740 1 -- 172.21.15.47:0/4193531034 shutdown_connections 2024-03-21T11:44:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.395+0000 7fe725bce740 1 -- 172.21.15.47:0/4193531034 wait complete. 2024-03-21T11:44:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.395+0000 7fe725bce740 20 client.32207 trim_cache size 0 max 16384 2024-03-21T11:44:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.417+0000 7fe725bce740 1 Processor -- start 2024-03-21T11:44:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.418+0000 7fe725bce740 1 -- start start 2024-03-21T11:44:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.418+0000 7fe725bce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e80c1e0 0x55d30e80b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.418+0000 7fe725bce740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80cbd0 0x55d30e821c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.418+0000 7fe725bce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e811d40 0x55d30e8023a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.418+0000 7fe725bce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1a7e50 con 0x55d30e80c1e0 2024-03-21T11:44:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.418+0000 7fe725bce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1a97d0 con 0x55d30e811d40 2024-03-21T11:44:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.418+0000 7fe725bce740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1a84b0 con 0x55d30e80cbd0 2024-03-21T11:44:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.418+0000 7fe71b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80cbd0 0x55d30e821c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.418+0000 7fe720e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e80c1e0 0x55d30e80b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.418+0000 7fe71b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80cbd0 0x55d30e821c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46900/0 (socket says 172.21.15.47:46900) 2024-03-21T11:44:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.418+0000 7fe71b400640 1 -- 172.21.15.47:0/992355796 learned_addr learned my addr 172.21.15.47:0/992355796 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.419+0000 7fe720e00640 1 --2- 172.21.15.47:0/992355796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e811d40 0x55d30e8023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.419+0000 7fe71b400640 1 -- 172.21.15.47:0/992355796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e811d40 msgr2=0x55d30e8023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.419+0000 7fe71b400640 1 --2- 172.21.15.47:0/992355796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e811d40 0x55d30e8023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.419+0000 7fe71b400640 1 -- 172.21.15.47:0/992355796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e80c1e0 msgr2=0x55d30e80b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.419+0000 7fe71b400640 1 --2- 172.21.15.47:0/992355796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e80c1e0 0x55d30e80b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.419+0000 7fe71b400640 1 -- 172.21.15.47:0/992355796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d30e805d40 con 0x55d30e80cbd0 2024-03-21T11:44:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.419+0000 7fe720e00640 1 --2- 172.21.15.47:0/992355796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e80c1e0 0x55d30e80b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.420+0000 7fe71b400640 1 --2- 172.21.15.47:0/992355796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80cbd0 0x55d30e821c10 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe70802c7b0 tx=0x7fe70802c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c66daf577c8a2e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.420+0000 7fe710c00640 1 -- 172.21.15.47:0/992355796 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70804a070 con 0x55d30e80cbd0 2024-03-21T11:44:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.420+0000 7fe710c00640 1 -- 172.21.15.47:0/992355796 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe708005520 con 0x55d30e80cbd0 2024-03-21T11:44:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.420+0000 7fe710c00640 1 -- 172.21.15.47:0/992355796 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708004d20 con 0x55d30e80cbd0 2024-03-21T11:44:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.421+0000 7fe725bce740 1 -- 172.21.15.47:0/992355796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80cbd0 msgr2=0x55d30e821c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.421+0000 7fe725bce740 1 --2- 172.21.15.47:0/992355796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80cbd0 0x55d30e821c10 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe70802c7b0 tx=0x7fe70802c5d0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.422+0000 7fe725bce740 1 -- 172.21.15.47:0/992355796 shutdown_connections 2024-03-21T11:44:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.422+0000 7fe725bce740 1 --2- 172.21.15.47:0/992355796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e811d40 0x55d30e8023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.422+0000 7fe725bce740 1 --2- 172.21.15.47:0/992355796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80cbd0 0x55d30e821c10 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.422+0000 7fe725bce740 1 --2- 172.21.15.47:0/992355796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e80c1e0 0x55d30e80b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.422+0000 7fe725bce740 1 -- 172.21.15.47:0/992355796 >> 172.21.15.47:0/992355796 conn(0x55d30e807750 msgr2=0x55d30e80a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.422+0000 7fe725bce740 1 -- 172.21.15.47:0/992355796 shutdown_connections 2024-03-21T11:44:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.423+0000 7fe725bce740 1 -- 172.21.15.47:0/992355796 wait complete. 2024-03-21T11:44:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.423+0000 7fe725bce740 1 -- start start 2024-03-21T11:44:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.424+0000 7fe725bce740 1 Processor -- start 2024-03-21T11:44:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.425+0000 7fe725bce740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80c1e0 0x55d30e816710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.425+0000 7fe725bce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80cbd0 0x55d30e816c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.425+0000 7fe725bce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e811d40 0x55d30e821bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.425+0000 7fe725bce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1a9170 con 0x55d30e811d40 2024-03-21T11:44:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.425+0000 7fe725bce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1a8b10 con 0x55d30e80cbd0 2024-03-21T11:44:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.425+0000 7fe725bce740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1a6010 con 0x55d30e80c1e0 2024-03-21T11:44:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.425+0000 7fe725bce740 10 client.0 _ll_register_callbacks cb 0x7fe708009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.425+0000 7fe720e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80c1e0 0x55d30e816710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.426+0000 7fe720e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80c1e0 0x55d30e816710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:46902/0 (socket says 172.21.15.47:46902) 2024-03-21T11:44:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.426+0000 7fe720e00640 1 -- 172.21.15.47:0/2605076652 learned_addr learned my addr 172.21.15.47:0/2605076652 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.426+0000 7fe71b400640 1 --2- 172.21.15.47:0/2605076652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80cbd0 0x55d30e816c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.426+0000 7fe71b400640 1 -- 172.21.15.47:0/2605076652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80c1e0 msgr2=0x55d30e816710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.426+0000 7fe71b400640 1 --2- 172.21.15.47:0/2605076652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80c1e0 0x55d30e816710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.426+0000 7fe71b400640 1 -- 172.21.15.47:0/2605076652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e811d40 msgr2=0x55d30e821bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.426+0000 7fe71b400640 1 --2- 172.21.15.47:0/2605076652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e811d40 0x55d30e821bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.426+0000 7fe71b400640 1 -- 172.21.15.47:0/2605076652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d30e817c90 con 0x55d30e80cbd0 2024-03-21T11:44:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.427+0000 7fe71b400640 1 --2- 172.21.15.47:0/2605076652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80cbd0 0x55d30e816c50 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fe708036040 tx=0x7fe70802c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a52f0da453979408 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.427+0000 7fe706000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:44:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.427+0000 7fe706000640 1 -- 172.21.15.47:0/2605076652 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708053070 con 0x55d30e80cbd0 2024-03-21T11:44:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.427+0000 7fe706000640 1 -- 172.21.15.47:0/2605076652 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe708005040 con 0x55d30e80cbd0 2024-03-21T11:44:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.427+0000 7fe725bce740 1 -- 172.21.15.47:0/2605076652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe708005520 con 0x55d30e80cbd0 2024-03-21T11:44:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.427+0000 7fe706000640 1 -- 172.21.15.47:0/2605076652 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe70805f050 con 0x55d30e80cbd0 2024-03-21T11:44:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.428+0000 7fe725bce740 10 client.32213 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.428+0000 7fe725bce740 1 -- 172.21.15.47:0/2605076652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d30e886440 con 0x55d30e80cbd0 2024-03-21T11:44:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.428+0000 7fe706000640 1 -- 172.21.15.47:0/2605076652 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe708005520 con 0x55d30e80cbd0 2024-03-21T11:44:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.429+0000 7fe706000640 1 -- 172.21.15.47:0/2605076652 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe708004aa0 con 0x55d30e80cbd0 2024-03-21T11:44:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.429+0000 7fe725bce740 10 client.32213 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.429+0000 7fe725bce740 10 client.32213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.429+0000 7fe725bce740 20 client.32213 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.429+0000 7fe725bce740 1 --2- 172.21.15.47:0/2605076652 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d30e886d00 0x55d30e889120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.429+0000 7fe725bce740 4 client.32213 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:44:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.429+0000 7fe725bce740 1 -- 172.21.15.47:0/2605076652 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d30e802a80 con 0x55d30e886d00 2024-03-21T11:44:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.429+0000 7fe720e00640 1 --2- 172.21.15.47:0/2605076652 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d30e886d00 0x55d30e889120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.430+0000 7fe720e00640 1 --2- 172.21.15.47:0/2605076652 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d30e886d00 0x55d30e889120 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=284c5049b3257e61 server_cookie=1f97777623ea8dd0 in_seq=0 out_seq=0 2024-03-21T11:44:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:17.430+0000 7fe706000640 10 client.32213 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:17.971 INFO:tasks.workunit.client.0.smithi047.stdout: 10 186150 22.86 MB/sec execute 301 sec latency 428.476 ms 2024-03-21T11:44:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.724+0000 7fe706000640 1 -- 172.21.15.47:0/2605076652 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d30e802a80 con 0x55d30e886d00 2024-03-21T11:44:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.724+0000 7fe706000640 10 client.32213 handle_command_reply: tid=0 2024-03-21T11:44:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.724+0000 7fe725bce740 1 client.32213 shutdown 2024-03-21T11:44:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.724+0000 7fe725bce740 1 -- 172.21.15.47:0/2605076652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80cbd0 msgr2=0x55d30e816c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.724+0000 7fe725bce740 1 --2- 172.21.15.47:0/2605076652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80cbd0 0x55d30e816c50 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fe708036040 tx=0x7fe70802c330 comp rx=0 tx=0).stop 2024-03-21T11:44:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.724+0000 7fe725bce740 1 -- 172.21.15.47:0/2605076652 shutdown_connections 2024-03-21T11:44:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.724+0000 7fe725bce740 1 --2- 172.21.15.47:0/2605076652 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d30e886d00 0x55d30e889120 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.724+0000 7fe725bce740 1 --2- 172.21.15.47:0/2605076652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d30e811d40 0x55d30e821bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.724+0000 7fe725bce740 1 --2- 172.21.15.47:0/2605076652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d30e80cbd0 0x55d30e816c50 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.724+0000 7fe725bce740 1 --2- 172.21.15.47:0/2605076652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d30e80c1e0 0x55d30e816710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.724+0000 7fe725bce740 1 -- 172.21.15.47:0/2605076652 >> 172.21.15.47:0/2605076652 conn(0x55d30e807750 msgr2=0x55d30e81a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.724+0000 7fe706000640 0 client.32213 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.725+0000 7fe725bce740 1 -- 172.21.15.47:0/2605076652 shutdown_connections 2024-03-21T11:44:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.725+0000 7fe725bce740 1 -- 172.21.15.47:0/2605076652 wait complete. 2024-03-21T11:44:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.726+0000 7fe725bce740 20 client.32213 trim_cache size 0 max 16384 2024-03-21T11:44:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:44:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:44:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:44:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:44:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.727+0000 7fe722e00640 1 -- 172.21.15.47:0/205895519 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe7000a3040 msgr2=0x7fe7000a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.727+0000 7fe722e00640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe7000a3040 0x7fe7000a5500 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe70c09b720 tx=0x7fe70c010e30 comp rx=0 tx=0).stop 2024-03-21T11:44:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.727+0000 7fe722e00640 1 -- 172.21.15.47:0/205895519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe71c1016d0 msgr2=0x7fe71c197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.727+0000 7fe722e00640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe71c1016d0 0x7fe71c197900 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00d950 tx=0x7fe70c00de10 comp rx=0 tx=0).stop 2024-03-21T11:44:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.727+0000 7fe722e00640 1 -- 172.21.15.47:0/205895519 shutdown_connections 2024-03-21T11:44:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.727+0000 7fe722e00640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe7000a3040 0x7fe7000a5500 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.727+0000 7fe722e00640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe71c103280 0x7fe71c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.727+0000 7fe722e00640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe71c1028c0 0x7fe71c197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.727+0000 7fe722e00640 1 --2- 172.21.15.47:0/205895519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe71c1016d0 0x7fe71c197900 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.727+0000 7fe722e00640 1 -- 172.21.15.47:0/205895519 >> 172.21.15.47:0/205895519 conn(0x7fe71c0fcc60 msgr2=0x7fe71c105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.727+0000 7fe722e00640 1 -- 172.21.15.47:0/205895519 shutdown_connections 2024-03-21T11:44:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:18.728+0000 7fe722e00640 1 -- 172.21.15.47:0/205895519 wait complete. 2024-03-21T11:44:18.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 186311 22.82 MB/sec execute 302 sec latency 1186.908 ms 2024-03-21T11:44:19.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:18 smithi047 ceph-mon[31351]: pgmap v1029: 129 pgs: 129 active+clean; 608 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.1 MiB/s wr, 539 op/s 2024-03-21T11:44:19.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:18 smithi100 ceph-mon[36245]: pgmap v1029: 129 pgs: 129 active+clean; 608 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.1 MiB/s wr, 539 op/s 2024-03-21T11:44:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:18 smithi102 ceph-mon[36748]: pgmap v1029: 129 pgs: 129 active+clean; 608 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.1 MiB/s wr, 539 op/s 2024-03-21T11:44:19.971 INFO:tasks.workunit.client.0.smithi047.stdout: 10 186727 22.81 MB/sec execute 303 sec latency 128.536 ms 2024-03-21T11:44:20.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 187196 22.84 MB/sec execute 304 sec latency 85.966 ms 2024-03-21T11:44:21.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:20 smithi047 ceph-mon[31351]: pgmap v1030: 129 pgs: 129 active+clean; 608 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.0 MiB/s wr, 657 op/s 2024-03-21T11:44:21.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:20 smithi100 ceph-mon[36245]: pgmap v1030: 129 pgs: 129 active+clean; 608 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.0 MiB/s wr, 657 op/s 2024-03-21T11:44:21.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:20 smithi102 ceph-mon[36748]: pgmap v1030: 129 pgs: 129 active+clean; 608 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.0 MiB/s wr, 657 op/s 2024-03-21T11:44:21.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:44:21.532 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:44:21.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.796+0000 7fd7c1a00640 1 -- 172.21.15.47:0/3507643984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7bc103280 msgr2=0x7fd7bc109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:21.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.796+0000 7fd7c1a00640 1 --2- 172.21.15.47:0/3507643984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7bc103280 0x7fd7bc109b10 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009ca0 tx=0x7fd7b402c100 comp rx=0 tx=0).stop 2024-03-21T11:44:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.796+0000 7fd7c1a00640 1 -- 172.21.15.47:0/3507643984 shutdown_connections 2024-03-21T11:44:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.796+0000 7fd7c1a00640 1 --2- 172.21.15.47:0/3507643984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7bc103280 0x7fd7bc109b10 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.796+0000 7fd7c1a00640 1 --2- 172.21.15.47:0/3507643984 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7bc1028c0 0x7fd7bc102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.796+0000 7fd7c1a00640 1 --2- 172.21.15.47:0/3507643984 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7bc1016d0 0x7fd7bc101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.796+0000 7fd7c1a00640 1 -- 172.21.15.47:0/3507643984 >> 172.21.15.47:0/3507643984 conn(0x7fd7bc0fcc40 msgr2=0x7fd7bc0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.796+0000 7fd7c1a00640 1 -- 172.21.15.47:0/3507643984 shutdown_connections 2024-03-21T11:44:21.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.796+0000 7fd7c1a00640 1 -- 172.21.15.47:0/3507643984 wait complete. 2024-03-21T11:44:21.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.797+0000 7fd7c1a00640 1 Processor -- start 2024-03-21T11:44:21.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.797+0000 7fd7c1a00640 1 -- start start 2024-03-21T11:44:21.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7c1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7bc1016d0 0x7fd7bc199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7c1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7bc1028c0 0x7fd7bc19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7c1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7bc103280 0x7fd7bc1a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7c1a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc10d350 con 0x7fd7bc1016d0 2024-03-21T11:44:21.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7c1a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc10d1d0 con 0x7fd7bc103280 2024-03-21T11:44:21.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7c1a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc10d4d0 con 0x7fd7bc1028c0 2024-03-21T11:44:21.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7baa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7bc1016d0 0x7fd7bc199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7bb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7bc103280 0x7fd7bc1a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7ba000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7bc1028c0 0x7fd7bc19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7bb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7bc103280 0x7fd7bc1a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34618/0 (socket says 172.21.15.47:34618) 2024-03-21T11:44:21.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7ba000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7bc1028c0 0x7fd7bc19a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44592/0 (socket says 172.21.15.47:44592) 2024-03-21T11:44:21.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7bb400640 1 -- 172.21.15.47:0/733816945 learned_addr learned my addr 172.21.15.47:0/733816945 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:21.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7bb400640 1 -- 172.21.15.47:0/733816945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7bc1028c0 msgr2=0x7fd7bc19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:21.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7bb400640 1 --2- 172.21.15.47:0/733816945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7bc1028c0 0x7fd7bc19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7bb400640 1 -- 172.21.15.47:0/733816945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7bc1016d0 msgr2=0x7fd7bc199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:21.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7bb400640 1 --2- 172.21.15.47:0/733816945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7bc1016d0 0x7fd7bc199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.798+0000 7fd7bb400640 1 -- 172.21.15.47:0/733816945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc1a17d0 con 0x7fd7bc103280 2024-03-21T11:44:21.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.799+0000 7fd7bb400640 1 --2- 172.21.15.47:0/733816945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7bc103280 0x7fd7bc1a1160 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd7b402c640 tx=0x7fd7b4035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:21.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.799+0000 7fd7b3400640 1 -- 172.21.15.47:0/733816945 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4039040 con 0x7fd7bc103280 2024-03-21T11:44:21.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.799+0000 7fd7c1a00640 1 -- 172.21.15.47:0/733816945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc1a1a60 con 0x7fd7bc103280 2024-03-21T11:44:21.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.800+0000 7fd7b3400640 1 -- 172.21.15.47:0/733816945 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7b4044070 con 0x7fd7bc103280 2024-03-21T11:44:21.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.800+0000 7fd7b3400640 1 -- 172.21.15.47:0/733816945 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40043c0 con 0x7fd7bc103280 2024-03-21T11:44:21.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.800+0000 7fd7c1a00640 1 -- 172.21.15.47:0/733816945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc1a1fa0 con 0x7fd7bc103280 2024-03-21T11:44:21.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.804+0000 7fd7c45f1740 1 Processor -- start 2024-03-21T11:44:21.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.805+0000 7fd7b3400640 1 -- 172.21.15.47:0/733816945 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd7b4004560 con 0x7fd7bc103280 2024-03-21T11:44:21.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.806+0000 7fd7c45f1740 1 -- start start 2024-03-21T11:44:21.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.806+0000 7fd7c45f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9dee80 0x55835d9df2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.806+0000 7fd7c45f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 0x55835d9e4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.806+0000 7fd7c45f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9e49e0 0x55835d9e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.806+0000 7fd7c45f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55835d944250 con 0x55835d9df870 2024-03-21T11:44:21.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.806+0000 7fd7c45f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55835d8eca30 con 0x55835d9e49e0 2024-03-21T11:44:21.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.806+0000 7fd7c45f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55835d9402f0 con 0x55835d9dee80 2024-03-21T11:44:21.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.807+0000 7fd7ba000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9dee80 0x55835d9df2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.807+0000 7fd7ba000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9dee80 0x55835d9df2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44606/0 (socket says 172.21.15.47:44606) 2024-03-21T11:44:21.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.807+0000 7fd7ba000640 1 -- 172.21.15.47:0/304107149 learned_addr learned my addr 172.21.15.47:0/304107149 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:21.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.807+0000 7fd7ba000640 1 --2- 172.21.15.47:0/304107149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9e49e0 0x55835d9e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.807+0000 7fd7baa00640 1 --2- 172.21.15.47:0/304107149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 0x55835d9e4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.807+0000 7fd7b3400640 1 --2- 172.21.15.47:0/733816945 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7a00a2fa0 0x7fd7a00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.807+0000 7fd7ba000640 1 -- 172.21.15.47:0/304107149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9e49e0 msgr2=0x55835d9e6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:21.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.807+0000 7fd7ba000640 1 --2- 172.21.15.47:0/304107149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9e49e0 0x55835d9e6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.807+0000 7fd7ba000640 1 -- 172.21.15.47:0/304107149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 msgr2=0x55835d9e4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:21.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.808+0000 7fd7b3400640 1 -- 172.21.15.47:0/733816945 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd7b40f1c90 con 0x7fd7bc103280 2024-03-21T11:44:21.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.808+0000 7fd7ba000640 1 --2- 172.21.15.47:0/304107149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 0x55835d9e4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.808+0000 7fd7ba000640 1 -- 172.21.15.47:0/304107149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55835d8d7690 con 0x55835d9dee80 2024-03-21T11:44:21.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.808+0000 7fd7baa00640 1 --2- 172.21.15.47:0/304107149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 0x55835d9e4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:21.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.808+0000 7fd7bb400640 1 --2- 172.21.15.47:0/733816945 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7a00a2fa0 0x7fd7a00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.808+0000 7fd7ba000640 1 --2- 172.21.15.47:0/304107149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9dee80 0x55835d9df2a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80058c0 tx=0x7fd7a8005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe7bc43d65360973 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:21.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.809+0000 7fd7bb400640 1 --2- 172.21.15.47:0/733816945 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7a00a2fa0 0x7fd7a00a5460 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40bdd30 tx=0x7fd7b40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:21.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.813+0000 7fd7a7e00640 1 -- 172.21.15.47:0/304107149 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8004030 con 0x55835d9dee80 2024-03-21T11:44:21.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.813+0000 7fd7a7e00640 1 -- 172.21.15.47:0/304107149 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7a8005500 con 0x55835d9dee80 2024-03-21T11:44:21.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.813+0000 7fd7a7e00640 1 -- 172.21.15.47:0/304107149 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8004ce0 con 0x55835d9dee80 2024-03-21T11:44:21.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.814+0000 7fd7c45f1740 1 -- 172.21.15.47:0/304107149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9dee80 msgr2=0x55835d9df2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:21.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.814+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/304107149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9dee80 0x55835d9df2a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80058c0 tx=0x7fd7a8005850 comp rx=0 tx=0).stop 2024-03-21T11:44:21.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.817+0000 7fd7c45f1740 1 -- 172.21.15.47:0/304107149 shutdown_connections 2024-03-21T11:44:21.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.817+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/304107149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9e49e0 0x55835d9e6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.817+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/304107149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 0x55835d9e4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.817+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/304107149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9dee80 0x55835d9df2a0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.817+0000 7fd7c45f1740 1 -- 172.21.15.47:0/304107149 >> 172.21.15.47:0/304107149 conn(0x55835d9a7200 msgr2=0x55835d9da7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:21.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.818+0000 7fd7c45f1740 1 -- 172.21.15.47:0/304107149 shutdown_connections 2024-03-21T11:44:21.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.818+0000 7fd7c45f1740 1 -- 172.21.15.47:0/304107149 wait complete. 2024-03-21T11:44:21.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.819+0000 7fd7c45f1740 1 -- start start 2024-03-21T11:44:21.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.819+0000 7fd7c45f1740 1 Processor -- start 2024-03-21T11:44:21.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.821+0000 7fd7c45f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9dee80 0x55835d9f1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.821+0000 7fd7c45f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9df870 0x55835d9f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.821+0000 7fd7c45f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9e49e0 0x55835d9f8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.821+0000 7fd7c45f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55835d9e79e0 con 0x55835d9e49e0 2024-03-21T11:44:21.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.821+0000 7fd7c45f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55835d8ffe80 con 0x55835d9dee80 2024-03-21T11:44:21.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.821+0000 7fd7c45f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55835d9e7b60 con 0x55835d9df870 2024-03-21T11:44:21.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.821+0000 7fd7c45f1740 10 client.0 _ll_register_callbacks cb 0x55835d912130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:21.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.821+0000 7fd7ba000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9dee80 0x55835d9f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.821+0000 7fd7ba000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9dee80 0x55835d9f1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34634/0 (socket says 172.21.15.47:34634) 2024-03-21T11:44:21.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.821+0000 7fd7ba000640 1 -- 172.21.15.47:0/1076984704 learned_addr learned my addr 172.21.15.47:0/1076984704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:21.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.822+0000 7fd7ba000640 1 -- 172.21.15.47:0/1076984704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9df870 msgr2=0x55835d9f1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:21.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.822+0000 7fd7ba000640 1 --2- 172.21.15.47:0/1076984704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9df870 0x55835d9f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.822+0000 7fd7ba000640 1 -- 172.21.15.47:0/1076984704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9e49e0 msgr2=0x55835d9f8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:21.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.822+0000 7fd7ba000640 1 --2- 172.21.15.47:0/1076984704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9e49e0 0x55835d9f8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.822+0000 7fd7ba000640 1 -- 172.21.15.47:0/1076984704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55835d7227d0 con 0x55835d9dee80 2024-03-21T11:44:21.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.822+0000 7fd7ba000640 1 --2- 172.21.15.47:0/1076984704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9dee80 0x55835d9f1710 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009560 tx=0x7fd7a8009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48143bbd306c4c2a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:21.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.822+0000 7fd7a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:44:21.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.823+0000 7fd7a4c00640 1 -- 172.21.15.47:0/1076984704 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8004a60 con 0x55835d9dee80 2024-03-21T11:44:21.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.823+0000 7fd7a4c00640 1 -- 172.21.15.47:0/1076984704 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd7a8027030 con 0x55835d9dee80 2024-03-21T11:44:21.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.823+0000 7fd7c45f1740 1 -- 172.21.15.47:0/1076984704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7a8005500 con 0x55835d9dee80 2024-03-21T11:44:21.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.824+0000 7fd7a4c00640 1 -- 172.21.15.47:0/1076984704 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a8004a60 con 0x55835d9dee80 2024-03-21T11:44:21.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.824+0000 7fd7c45f1740 10 client.32225 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:21.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.824+0000 7fd7c45f1740 1 -- 172.21.15.47:0/1076984704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55835d6583f0 con 0x55835d9dee80 2024-03-21T11:44:21.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.824+0000 7fd7a4c00640 1 -- 172.21.15.47:0/1076984704 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7a8005500 con 0x55835d9dee80 2024-03-21T11:44:21.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.825+0000 7fd7a4c00640 1 -- 172.21.15.47:0/1076984704 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd7a801f440 con 0x55835d9dee80 2024-03-21T11:44:21.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.825+0000 7fd7c45f1740 10 client.32225 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:21.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.825+0000 7fd7c45f1740 10 client.32225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:21.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.825+0000 7fd7c45f1740 20 client.32225 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:21.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.825+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/1076984704 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835d9fd220 0x55835d9ff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.825+0000 7fd7c45f1740 4 client.32225 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:21.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.825+0000 7fd7c45f1740 1 -- 172.21.15.47:0/1076984704 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55835d951790 con 0x55835d9fd220 2024-03-21T11:44:21.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.825+0000 7fd7baa00640 1 --2- 172.21.15.47:0/1076984704 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835d9fd220 0x55835d9ff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.826+0000 7fd7baa00640 1 --2- 172.21.15.47:0/1076984704 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835d9fd220 0x55835d9ff650 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fa4355dacc7efe0 server_cookie=a99826a8e1c66301 in_seq=0 out_seq=0 2024-03-21T11:44:21.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.826+0000 7fd7a4c00640 10 client.32225 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.829+0000 7fd7a4c00640 1 -- 172.21.15.47:0/1076984704 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55835d951790 con 0x55835d9fd220 2024-03-21T11:44:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.829+0000 7fd7a4c00640 10 client.32225 handle_command_reply: tid=0 2024-03-21T11:44:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.829+0000 7fd7c45f1740 1 client.32225 shutdown 2024-03-21T11:44:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.830+0000 7fd7c45f1740 1 -- 172.21.15.47:0/1076984704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9dee80 msgr2=0x55835d9f1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.830+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/1076984704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9dee80 0x55835d9f1710 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009560 tx=0x7fd7a8009e20 comp rx=0 tx=0).stop 2024-03-21T11:44:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.830+0000 7fd7c45f1740 1 -- 172.21.15.47:0/1076984704 shutdown_connections 2024-03-21T11:44:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.830+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/1076984704 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835d9fd220 0x55835d9ff650 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.830+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/1076984704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9e49e0 0x55835d9f8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.830+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/1076984704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9df870 0x55835d9f1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.830+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/1076984704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9dee80 0x55835d9f1710 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.830+0000 7fd7c45f1740 1 -- 172.21.15.47:0/1076984704 >> 172.21.15.47:0/1076984704 conn(0x55835d9a7200 msgr2=0x55835d9d9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:21.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.830+0000 7fd7a4c00640 0 client.32225 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:21.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.830+0000 7fd7c45f1740 1 -- 172.21.15.47:0/1076984704 shutdown_connections 2024-03-21T11:44:21.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.830+0000 7fd7c45f1740 1 -- 172.21.15.47:0/1076984704 wait complete. 2024-03-21T11:44:21.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.830+0000 7fd7c45f1740 20 client.32225 trim_cache size 0 max 16384 2024-03-21T11:44:21.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.852+0000 7fd7c45f1740 1 Processor -- start 2024-03-21T11:44:21.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.852+0000 7fd7c45f1740 1 -- start start 2024-03-21T11:44:21.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.853+0000 7fd7c45f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9dee80 0x55835d9f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.853+0000 7fd7c45f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 0x55835d9f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.853+0000 7fd7c45f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9e49e0 0x55835d9e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.853+0000 7fd7c45f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1a7f30 con 0x55835d9df870 2024-03-21T11:44:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.853+0000 7fd7c45f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1a98b0 con 0x55835d9e49e0 2024-03-21T11:44:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.853+0000 7fd7c45f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1a8590 con 0x55835d9dee80 2024-03-21T11:44:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.853+0000 7fd7baa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 0x55835d9f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.853+0000 7fd7baa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 0x55835d9f8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39924/0 (socket says 172.21.15.47:39924) 2024-03-21T11:44:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.853+0000 7fd7baa00640 1 -- 172.21.15.47:0/1999227084 learned_addr learned my addr 172.21.15.47:0/1999227084 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.853+0000 7fd7ba000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9e49e0 0x55835d9e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.854+0000 7fd7ba000640 1 --2- 172.21.15.47:0/1999227084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9dee80 0x55835d9f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.854+0000 7fd7ba000640 1 -- 172.21.15.47:0/1999227084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9dee80 msgr2=0x55835d9f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:21.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.854+0000 7fd7ba000640 1 --2- 172.21.15.47:0/1999227084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9dee80 0x55835d9f1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.854+0000 7fd7ba000640 1 -- 172.21.15.47:0/1999227084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 msgr2=0x55835d9f8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:21.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.854+0000 7fd7ba000640 1 --2- 172.21.15.47:0/1999227084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 0x55835d9f8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.854+0000 7fd7ba000640 1 -- 172.21.15.47:0/1999227084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55835d6180f0 con 0x55835d9e49e0 2024-03-21T11:44:21.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.855+0000 7fd7ba000640 1 --2- 172.21.15.47:0/1999227084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9e49e0 0x55835d9e0a20 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009f20 tx=0x7fd7a80115f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25873d270d634c11 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:21.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.857+0000 7fd7a7400640 1 -- 172.21.15.47:0/1999227084 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a801fa20 con 0x55835d9e49e0 2024-03-21T11:44:21.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.857+0000 7fd7a7400640 1 -- 172.21.15.47:0/1999227084 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7a803b070 con 0x55835d9e49e0 2024-03-21T11:44:21.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.857+0000 7fd7a7400640 1 -- 172.21.15.47:0/1999227084 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8009a90 con 0x55835d9e49e0 2024-03-21T11:44:21.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.858+0000 7fd7c45f1740 1 -- 172.21.15.47:0/1999227084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9e49e0 msgr2=0x55835d9e0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:21.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.858+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/1999227084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9e49e0 0x55835d9e0a20 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009f20 tx=0x7fd7a80115f0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.858+0000 7fd7c45f1740 1 -- 172.21.15.47:0/1999227084 shutdown_connections 2024-03-21T11:44:21.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.858+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/1999227084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9e49e0 0x55835d9e0a20 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.858+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/1999227084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 0x55835d9f8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.858+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/1999227084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9dee80 0x55835d9f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.858+0000 7fd7c45f1740 1 -- 172.21.15.47:0/1999227084 >> 172.21.15.47:0/1999227084 conn(0x55835d9a7200 msgr2=0x55835d9e0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:21.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.858+0000 7fd7c45f1740 1 -- 172.21.15.47:0/1999227084 shutdown_connections 2024-03-21T11:44:21.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.860+0000 7fd7c45f1740 1 -- 172.21.15.47:0/1999227084 wait complete. 2024-03-21T11:44:21.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.861+0000 7fd7c45f1740 1 -- start start 2024-03-21T11:44:21.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.861+0000 7fd7c45f1740 1 Processor -- start 2024-03-21T11:44:21.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.862+0000 7fd7c45f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9dee80 0x55835d9f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.862+0000 7fd7c45f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 0x55835d9f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.863+0000 7fd7c45f1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9e49e0 0x55835d9f7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.863+0000 7fd7c45f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1a9250 con 0x55835d9df870 2024-03-21T11:44:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.863+0000 7fd7c45f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1a8bf0 con 0x55835d9dee80 2024-03-21T11:44:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.863+0000 7fd7c45f1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1a6110 con 0x55835d9e49e0 2024-03-21T11:44:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.863+0000 7fd7c45f1740 10 client.0 _ll_register_callbacks cb 0x7fd7ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.863+0000 7fd7ba000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9dee80 0x55835d9f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.863+0000 7fd7ba000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9dee80 0x55835d9f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34650/0 (socket says 172.21.15.47:34650) 2024-03-21T11:44:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.863+0000 7fd7ba000640 1 -- 172.21.15.47:0/410709341 learned_addr learned my addr 172.21.15.47:0/410709341 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.864+0000 7fd7baa00640 1 --2- 172.21.15.47:0/410709341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 0x55835d9f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.864+0000 7fd7ba000640 1 --2- 172.21.15.47:0/410709341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9e49e0 0x55835d9f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.864+0000 7fd7ba000640 1 -- 172.21.15.47:0/410709341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9e49e0 msgr2=0x55835d9f7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.864+0000 7fd7ba000640 1 --2- 172.21.15.47:0/410709341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9e49e0 0x55835d9f7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.864+0000 7fd7ba000640 1 -- 172.21.15.47:0/410709341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 msgr2=0x55835d9f68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.864+0000 7fd7ba000640 1 --2- 172.21.15.47:0/410709341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 0x55835d9f68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.864+0000 7fd7ba000640 1 -- 172.21.15.47:0/410709341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55835d9f7ef0 con 0x55835d9dee80 2024-03-21T11:44:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.865+0000 7fd7ba000640 1 --2- 172.21.15.47:0/410709341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9dee80 0x55835d9f63b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80057c0 tx=0x7fd7a8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92e36f17d799c727 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.865+0000 7fd7a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:44:21.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.868+0000 7fd7a4c00640 1 -- 172.21.15.47:0/410709341 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a804b060 con 0x55835d9dee80 2024-03-21T11:44:21.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.868+0000 7fd7a4c00640 1 -- 172.21.15.47:0/410709341 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd7a8009ca0 con 0x55835d9dee80 2024-03-21T11:44:21.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.868+0000 7fd7c45f1740 1 -- 172.21.15.47:0/410709341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7a803b070 con 0x55835d9dee80 2024-03-21T11:44:21.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.869+0000 7fd7a4c00640 1 -- 172.21.15.47:0/410709341 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a8004c00 con 0x55835d9dee80 2024-03-21T11:44:21.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.869+0000 7fd7c45f1740 10 client.32231 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:21.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.869+0000 7fd7c45f1740 1 -- 172.21.15.47:0/410709341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55835da58ae0 con 0x55835d9dee80 2024-03-21T11:44:21.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.869+0000 7fd7a4c00640 1 -- 172.21.15.47:0/410709341 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7a803b070 con 0x55835d9dee80 2024-03-21T11:44:21.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.869+0000 7fd7a4c00640 1 -- 172.21.15.47:0/410709341 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd7a8046450 con 0x55835d9dee80 2024-03-21T11:44:21.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.869+0000 7fd7c45f1740 10 client.32231 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:21.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.869+0000 7fd7c45f1740 10 client.32231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:21.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.870+0000 7fd7c45f1740 20 client.32231 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:21.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.870+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/410709341 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835da592d0 0x55835da5b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:21.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.870+0000 7fd7c45f1740 4 client.32231 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:44:21.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.870+0000 7fd7c45f1740 1 -- 172.21.15.47:0/410709341 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55835d94bae0 con 0x55835da592d0 2024-03-21T11:44:21.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.870+0000 7fd7baa00640 1 --2- 172.21.15.47:0/410709341 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835da592d0 0x55835da5b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:21.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.871+0000 7fd7a4c00640 10 client.32231 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:21.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.871+0000 7fd7baa00640 1 --2- 172.21.15.47:0/410709341 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835da592d0 0x55835da5b6f0 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8dea1039cc3061a server_cookie=81c57e2e92aa9599 in_seq=0 out_seq=0 2024-03-21T11:44:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.873+0000 7fd7a4c00640 1 -- 172.21.15.47:0/410709341 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55835d94bae0 con 0x55835da592d0 2024-03-21T11:44:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.873+0000 7fd7a4c00640 10 client.32231 handle_command_reply: tid=0 2024-03-21T11:44:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.873+0000 7fd7c45f1740 1 client.32231 shutdown 2024-03-21T11:44:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.875+0000 7fd7c45f1740 1 -- 172.21.15.47:0/410709341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9dee80 msgr2=0x55835d9f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.875+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/410709341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9dee80 0x55835d9f63b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80057c0 tx=0x7fd7a8005fd0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.875+0000 7fd7c45f1740 1 -- 172.21.15.47:0/410709341 shutdown_connections 2024-03-21T11:44:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.875+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/410709341 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55835da592d0 0x55835da5b6f0 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.875+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/410709341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55835d9e49e0 0x55835d9f7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.875+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/410709341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55835d9df870 0x55835d9f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.875+0000 7fd7c45f1740 1 --2- 172.21.15.47:0/410709341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55835d9dee80 0x55835d9f63b0 secure :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80057c0 tx=0x7fd7a8005fd0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.875+0000 7fd7c45f1740 1 -- 172.21.15.47:0/410709341 >> 172.21.15.47:0/410709341 conn(0x55835d9a7200 msgr2=0x55835d9e0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.875+0000 7fd7a4c00640 0 client.32231 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.876+0000 7fd7c45f1740 1 -- 172.21.15.47:0/410709341 shutdown_connections 2024-03-21T11:44:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.876+0000 7fd7c45f1740 1 -- 172.21.15.47:0/410709341 wait complete. 2024-03-21T11:44:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.876+0000 7fd7c45f1740 20 client.32231 trim_cache size 0 max 16384 2024-03-21T11:44:21.877 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:44:21.877 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:44:21.877 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:44:21.878 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:44:21.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.877+0000 7fd7c1a00640 1 -- 172.21.15.47:0/733816945 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7a00a2fa0 msgr2=0x7fd7a00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:21.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.877+0000 7fd7c1a00640 1 --2- 172.21.15.47:0/733816945 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7a00a2fa0 0x7fd7a00a5460 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40bdd30 tx=0x7fd7b40095c0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.877+0000 7fd7c1a00640 1 -- 172.21.15.47:0/733816945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7bc103280 msgr2=0x7fd7bc1a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:21.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.877+0000 7fd7c1a00640 1 --2- 172.21.15.47:0/733816945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7bc103280 0x7fd7bc1a1160 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd7b402c640 tx=0x7fd7b4035d10 comp rx=0 tx=0).stop 2024-03-21T11:44:21.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.878+0000 7fd7c1a00640 1 -- 172.21.15.47:0/733816945 shutdown_connections 2024-03-21T11:44:21.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.878+0000 7fd7c1a00640 1 --2- 172.21.15.47:0/733816945 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd7a00a2fa0 0x7fd7a00a5460 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.878+0000 7fd7c1a00640 1 --2- 172.21.15.47:0/733816945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7bc103280 0x7fd7bc1a1160 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.878+0000 7fd7c1a00640 1 --2- 172.21.15.47:0/733816945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7bc1028c0 0x7fd7bc19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.878+0000 7fd7c1a00640 1 --2- 172.21.15.47:0/733816945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7bc1016d0 0x7fd7bc199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:21.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.878+0000 7fd7c1a00640 1 -- 172.21.15.47:0/733816945 >> 172.21.15.47:0/733816945 conn(0x7fd7bc0fcc40 msgr2=0x7fd7bc0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:21.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.879+0000 7fd7c1a00640 1 -- 172.21.15.47:0/733816945 shutdown_connections 2024-03-21T11:44:21.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:21.879+0000 7fd7c1a00640 1 -- 172.21.15.47:0/733816945 wait complete. 2024-03-21T11:44:21.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:21 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:44:21.899 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:44:21.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:44:21.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 187744 22.88 MB/sec execute 305 sec latency 79.965 ms 2024-03-21T11:44:22.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:21 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:44:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.145+0000 7f22a9600640 1 -- 172.21.15.47:0/3719972543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22a41016c0 msgr2=0x7f22a4101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.145+0000 7f22a9600640 1 --2- 172.21.15.47:0/3719972543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22a41016c0 0x7f22a4101ae0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2294009860 tx=0x7f229402bb70 comp rx=0 tx=0).stop 2024-03-21T11:44:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.145+0000 7f22a9600640 1 -- 172.21.15.47:0/3719972543 shutdown_connections 2024-03-21T11:44:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.145+0000 7f22a9600640 1 --2- 172.21.15.47:0/3719972543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a4103270 0x7f22a4109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.145+0000 7f22a9600640 1 --2- 172.21.15.47:0/3719972543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22a41028b0 0x7f22a4102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.145+0000 7f22a9600640 1 --2- 172.21.15.47:0/3719972543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22a41016c0 0x7f22a4101ae0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.145+0000 7f22a9600640 1 -- 172.21.15.47:0/3719972543 >> 172.21.15.47:0/3719972543 conn(0x7f22a40fcc50 msgr2=0x7f22a40ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:22.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.145+0000 7f22a9600640 1 -- 172.21.15.47:0/3719972543 shutdown_connections 2024-03-21T11:44:22.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.146+0000 7f22a9600640 1 -- 172.21.15.47:0/3719972543 wait complete. 2024-03-21T11:44:22.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.147+0000 7f22a9600640 1 Processor -- start 2024-03-21T11:44:22.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.147+0000 7f22a9600640 1 -- start start 2024-03-21T11:44:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.147+0000 7f22a9600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22a41016c0 0x7f22a4199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.148+0000 7f22a9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a41028b0 0x7f22a419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.148+0000 7f22a9600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22a4103270 0x7f22a41a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.148+0000 7f22a9600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a410d340 con 0x7f22a4103270 2024-03-21T11:44:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.148+0000 7f22a9600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a410d1c0 con 0x7f22a41028b0 2024-03-21T11:44:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.148+0000 7f22a9600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a410d4c0 con 0x7f22a41016c0 2024-03-21T11:44:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.148+0000 7f22a2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22a41016c0 0x7f22a4199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.148+0000 7f22a2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22a41016c0 0x7f22a4199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44642/0 (socket says 172.21.15.47:44642) 2024-03-21T11:44:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.148+0000 7f22a2a00640 1 -- 172.21.15.47:0/1858323942 learned_addr learned my addr 172.21.15.47:0/1858323942 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.148+0000 7f22a3400640 1 --2- 172.21.15.47:0/1858323942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22a4103270 0x7f22a41a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.148+0000 7f22a2000640 1 --2- 172.21.15.47:0/1858323942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a41028b0 0x7f22a419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.149+0000 7f22a3400640 1 -- 172.21.15.47:0/1858323942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22a41016c0 msgr2=0x7f22a4199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.149+0000 7f22a3400640 1 --2- 172.21.15.47:0/1858323942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22a41016c0 0x7f22a4199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.149+0000 7f22a3400640 1 -- 172.21.15.47:0/1858323942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a41028b0 msgr2=0x7f22a419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.149+0000 7f22a3400640 1 --2- 172.21.15.47:0/1858323942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a41028b0 0x7f22a419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.149+0000 7f22a3400640 1 -- 172.21.15.47:0/1858323942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a41a17f0 con 0x7f22a4103270 2024-03-21T11:44:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.149+0000 7f22a3400640 1 --2- 172.21.15.47:0/1858323942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22a4103270 0x7f22a41a1180 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f229c00ef00 tx=0x7f229c00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.149+0000 7f229b400640 1 -- 172.21.15.47:0/1858323942 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c004b20 con 0x7f22a4103270 2024-03-21T11:44:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.150+0000 7f229b400640 1 -- 172.21.15.47:0/1858323942 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f229c003e50 con 0x7f22a4103270 2024-03-21T11:44:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.150+0000 7f22a9600640 1 -- 172.21.15.47:0/1858323942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a41a1ae0 con 0x7f22a4103270 2024-03-21T11:44:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.150+0000 7f229b400640 1 -- 172.21.15.47:0/1858323942 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c00b570 con 0x7f22a4103270 2024-03-21T11:44:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.150+0000 7f22a9600640 1 -- 172.21.15.47:0/1858323942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a41a1fe0 con 0x7f22a4103270 2024-03-21T11:44:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.152+0000 7f22a9600640 1 -- 172.21.15.47:0/1858323942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a41a2240 con 0x7f22a4103270 2024-03-21T11:44:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.154+0000 7f229b400640 1 -- 172.21.15.47:0/1858323942 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f229c01d020 con 0x7f22a4103270 2024-03-21T11:44:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.156+0000 7f229b400640 1 --2- 172.21.15.47:0/1858323942 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22880a2f20 0x7f22880a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.156+0000 7f229b400640 1 -- 172.21.15.47:0/1858323942 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f229c0cf660 con 0x7f22a4103270 2024-03-21T11:44:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.157+0000 7f22a2a00640 1 --2- 172.21.15.47:0/1858323942 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22880a2f20 0x7f22880a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.158+0000 7f22a2a00640 1 --2- 172.21.15.47:0/1858323942 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22880a2f20 0x7f22880a53e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f22940095f0 tx=0x7f2294034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.162+0000 7f229b400640 1 -- 172.21.15.47:0/1858323942 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f229c09c0a0 con 0x7f22a4103270 2024-03-21T11:44:22.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:21 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:44:22.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.548+0000 7f22a9600640 1 -- 172.21.15.47:0/1858323942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22a41a25d0 con 0x7f22a4103270 2024-03-21T11:44:22.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.552+0000 7f229b400640 1 -- 172.21.15.47:0/1858323942 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f229c09b9a0 con 0x7f22a4103270 2024-03-21T11:44:22.553 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:44:22.553 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:39:47.303461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:44:22.553 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-21T11:44:22.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.554+0000 7f22a9600640 1 -- 172.21.15.47:0/1858323942 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22880a2f20 msgr2=0x7f22880a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:22.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.555+0000 7f22a9600640 1 --2- 172.21.15.47:0/1858323942 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22880a2f20 0x7f22880a53e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f22940095f0 tx=0x7f2294034ad0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.555+0000 7f22a9600640 1 -- 172.21.15.47:0/1858323942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22a4103270 msgr2=0x7f22a41a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:22.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.555+0000 7f22a9600640 1 --2- 172.21.15.47:0/1858323942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22a4103270 0x7f22a41a1180 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f229c00ef00 tx=0x7f229c00c740 comp rx=0 tx=0).stop 2024-03-21T11:44:22.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.555+0000 7f22a9600640 1 -- 172.21.15.47:0/1858323942 shutdown_connections 2024-03-21T11:44:22.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.555+0000 7f22a9600640 1 --2- 172.21.15.47:0/1858323942 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22880a2f20 0x7f22880a53e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.555+0000 7f22a9600640 1 --2- 172.21.15.47:0/1858323942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22a4103270 0x7f22a41a1180 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.555+0000 7f22a9600640 1 --2- 172.21.15.47:0/1858323942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22a41028b0 0x7f22a419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.555+0000 7f22a9600640 1 --2- 172.21.15.47:0/1858323942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22a41016c0 0x7f22a4199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.555+0000 7f22a9600640 1 -- 172.21.15.47:0/1858323942 >> 172.21.15.47:0/1858323942 conn(0x7f22a40fcc50 msgr2=0x7f22a40738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:22.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.555+0000 7f22a9600640 1 -- 172.21.15.47:0/1858323942 shutdown_connections 2024-03-21T11:44:22.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.555+0000 7f22a9600640 1 -- 172.21.15.47:0/1858323942 wait complete. 2024-03-21T11:44:22.578 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:44:22.578 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:44:22.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.867+0000 7f476fa00640 1 -- 172.21.15.47:0/1435271600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4768104ad0 msgr2=0x7f4768104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:22.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.867+0000 7f476fa00640 1 --2- 172.21.15.47:0/1435271600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4768104ad0 0x7f4768104f50 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f4754009900 tx=0x7f475402bc40 comp rx=0 tx=0).stop 2024-03-21T11:44:22.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.867+0000 7f476fa00640 1 -- 172.21.15.47:0/1435271600 shutdown_connections 2024-03-21T11:44:22.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.867+0000 7f476fa00640 1 --2- 172.21.15.47:0/1435271600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4768105490 0x7f476810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.867+0000 7f476fa00640 1 --2- 172.21.15.47:0/1435271600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4768104ad0 0x7f4768104f50 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.867+0000 7f476fa00640 1 --2- 172.21.15.47:0/1435271600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47681038e0 0x7f4768103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.867+0000 7f476fa00640 1 -- 172.21.15.47:0/1435271600 >> 172.21.15.47:0/1435271600 conn(0x7f47680fee50 msgr2=0x7f4768101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:22.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.867+0000 7f476fa00640 1 -- 172.21.15.47:0/1435271600 shutdown_connections 2024-03-21T11:44:22.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.867+0000 7f476fa00640 1 -- 172.21.15.47:0/1435271600 wait complete. 2024-03-21T11:44:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.868+0000 7f476fa00640 1 Processor -- start 2024-03-21T11:44:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.869+0000 7f476fa00640 1 -- start start 2024-03-21T11:44:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.869+0000 7f476fa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47681038e0 0x7f4768079450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.869+0000 7f476fa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4768104ad0 0x7f4768079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.869+0000 7f476fa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4768105490 0x7f4768075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.869+0000 7f476fa00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476810f590 con 0x7f47681038e0 2024-03-21T11:44:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.869+0000 7f476fa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476810f410 con 0x7f4768104ad0 2024-03-21T11:44:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.869+0000 7f476fa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476810f710 con 0x7f4768105490 2024-03-21T11:44:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.869+0000 7f476da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4768105490 0x7f4768075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.869+0000 7f476da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4768105490 0x7f4768075fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44652/0 (socket says 172.21.15.47:44652) 2024-03-21T11:44:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.869+0000 7f476da00640 1 -- 172.21.15.47:0/4198186558 learned_addr learned my addr 172.21.15.47:0/4198186558 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.870+0000 7f4767e00640 1 --2- 172.21.15.47:0/4198186558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4768104ad0 0x7f4768079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.870+0000 7f476da00640 1 -- 172.21.15.47:0/4198186558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4768104ad0 msgr2=0x7f4768079990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.870+0000 7f476da00640 1 --2- 172.21.15.47:0/4198186558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4768104ad0 0x7f4768079990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.870+0000 7f476da00640 1 -- 172.21.15.47:0/4198186558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47681038e0 msgr2=0x7f4768079450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:44:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.870+0000 7f476da00640 1 --2- 172.21.15.47:0/4198186558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47681038e0 0x7f4768079450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.870+0000 7f476da00640 1 -- 172.21.15.47:0/4198186558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4768076880 con 0x7f4768105490 2024-03-21T11:44:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.870+0000 7f476da00640 1 --2- 172.21.15.47:0/4198186558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4768105490 0x7f4768075fc0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f476000eb10 tx=0x7f476000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.871+0000 7f4765600640 1 -- 172.21.15.47:0/4198186558 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760004b20 con 0x7f4768105490 2024-03-21T11:44:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.871+0000 7f4765600640 1 -- 172.21.15.47:0/4198186558 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4760004cc0 con 0x7f4768105490 2024-03-21T11:44:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.872+0000 7f4765600640 1 -- 172.21.15.47:0/4198186558 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760010760 con 0x7f4768105490 2024-03-21T11:44:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.872+0000 7f476fa00640 1 -- 172.21.15.47:0/4198186558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47681a6020 con 0x7f4768105490 2024-03-21T11:44:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.872+0000 7f476fa00640 1 -- 172.21.15.47:0/4198186558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47681a63d0 con 0x7f4768105490 2024-03-21T11:44:22.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.875+0000 7f4765600640 1 -- 172.21.15.47:0/4198186558 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4760005020 con 0x7f4768105490 2024-03-21T11:44:22.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.876+0000 7f4772766740 1 Processor -- start 2024-03-21T11:44:22.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.876+0000 7f4772766740 1 -- start start 2024-03-21T11:44:22.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.876+0000 7f4772766740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561a810 0x55798561ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.876+0000 7f4772766740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561b9e0 0x55798561be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.876+0000 7f4772766740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561c3a0 0x557985622c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.876+0000 7f4772766740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798557f2c0 con 0x55798561b9e0 2024-03-21T11:44:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.876+0000 7f4772766740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557985527c90 con 0x55798561c3a0 2024-03-21T11:44:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.876+0000 7f4772766740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798557b360 con 0x55798561a810 2024-03-21T11:44:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.877+0000 7f4767e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561a810 0x55798561ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.877+0000 7f4767e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561a810 0x55798561ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44660/0 (socket says 172.21.15.47:44660) 2024-03-21T11:44:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.877+0000 7f4767e00640 1 -- 172.21.15.47:0/3069957685 learned_addr learned my addr 172.21.15.47:0/3069957685 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:22.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.877+0000 7f4765600640 1 --2- 172.21.15.47:0/4198186558 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47500a2f20 0x7f47500a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.877+0000 7f4767e00640 1 -- 172.21.15.47:0/3069957685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561c3a0 msgr2=0x557985622c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:22.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.877+0000 7f4767e00640 1 --2- 172.21.15.47:0/3069957685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561c3a0 0x557985622c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.877+0000 7f4767e00640 1 -- 172.21.15.47:0/3069957685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561b9e0 msgr2=0x55798561be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:22.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.877+0000 7f4767e00640 1 --2- 172.21.15.47:0/3069957685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561b9e0 0x55798561be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.877+0000 7f4767e00640 1 -- 172.21.15.47:0/3069957685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55798554c080 con 0x55798561a810 2024-03-21T11:44:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.877+0000 7f4765600640 1 -- 172.21.15.47:0/4198186558 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f47600cfea0 con 0x7f4768105490 2024-03-21T11:44:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.878+0000 7f4767e00640 1 --2- 172.21.15.47:0/3069957685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561a810 0x55798561ac30 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f475402c8f0 tx=0x7f4754009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e25e5243b3600c58 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.878+0000 7f476da00640 1 --2- 172.21.15.47:0/4198186558 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47500a2f20 0x7f47500a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.878+0000 7f475a000640 1 -- 172.21.15.47:0/3069957685 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475402cb90 con 0x55798561a810 2024-03-21T11:44:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.878+0000 7f475a000640 1 -- 172.21.15.47:0/3069957685 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f475402cd30 con 0x55798561a810 2024-03-21T11:44:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.879+0000 7f475a000640 1 -- 172.21.15.47:0/3069957685 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754004920 con 0x55798561a810 2024-03-21T11:44:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.879+0000 7f4772766740 1 -- 172.21.15.47:0/3069957685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561a810 msgr2=0x55798561ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.879+0000 7f4772766740 1 --2- 172.21.15.47:0/3069957685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561a810 0x55798561ac30 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f475402c8f0 tx=0x7f4754009610 comp rx=0 tx=0).stop 2024-03-21T11:44:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.879+0000 7f476da00640 1 --2- 172.21.15.47:0/4198186558 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47500a2f20 0x7f47500a53e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f47600a46c0 tx=0x7f476000efd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.879+0000 7f4772766740 1 -- 172.21.15.47:0/3069957685 shutdown_connections 2024-03-21T11:44:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.879+0000 7f4772766740 1 --2- 172.21.15.47:0/3069957685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561c3a0 0x557985622c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.879+0000 7f4772766740 1 --2- 172.21.15.47:0/3069957685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561b9e0 0x55798561be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.879+0000 7f4772766740 1 --2- 172.21.15.47:0/3069957685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561a810 0x55798561ac30 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.879+0000 7f4772766740 1 -- 172.21.15.47:0/3069957685 >> 172.21.15.47:0/3069957685 conn(0x557985615c00 msgr2=0x557985616000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.879+0000 7f4772766740 1 -- 172.21.15.47:0/3069957685 shutdown_connections 2024-03-21T11:44:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.880+0000 7f4772766740 1 -- 172.21.15.47:0/3069957685 wait complete. 2024-03-21T11:44:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.880+0000 7f4772766740 1 -- start start 2024-03-21T11:44:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.881+0000 7f4772766740 1 Processor -- start 2024-03-21T11:44:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.881+0000 7f4772766740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561a810 0x5579856247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.881+0000 7f4772766740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561b9e0 0x557985624d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.881+0000 7f4772766740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561c3a0 0x55798562bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.881+0000 7f4772766740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557985623390 con 0x55798561c3a0 2024-03-21T11:44:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.881+0000 7f4772766740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798553b0d0 con 0x55798561a810 2024-03-21T11:44:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.881+0000 7f4772766740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557985623510 con 0x55798561b9e0 2024-03-21T11:44:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.881+0000 7f4772766740 10 client.0 _ll_register_callbacks cb 0x55798553fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.881+0000 7f476d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561b9e0 0x557985624d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.881+0000 7f4767e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561a810 0x5579856247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.881+0000 7f476d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561b9e0 0x557985624d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44668/0 (socket says 172.21.15.47:44668) 2024-03-21T11:44:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.881+0000 7f476d000640 1 -- 172.21.15.47:0/929274932 learned_addr learned my addr 172.21.15.47:0/929274932 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.882+0000 7f4767e00640 1 --2- 172.21.15.47:0/929274932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561c3a0 0x55798562bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.882+0000 7f476d000640 1 -- 172.21.15.47:0/929274932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561a810 msgr2=0x5579856247f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.882+0000 7f476d000640 1 --2- 172.21.15.47:0/929274932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561a810 0x5579856247f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.882+0000 7f476d000640 1 -- 172.21.15.47:0/929274932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561c3a0 msgr2=0x55798562bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.882+0000 7f476d000640 1 --2- 172.21.15.47:0/929274932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561c3a0 0x55798562bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.882+0000 7f476d000640 1 -- 172.21.15.47:0/929274932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579852933f0 con 0x55798561b9e0 2024-03-21T11:44:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.882+0000 7f4767e00640 1 --2- 172.21.15.47:0/929274932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561c3a0 0x55798562bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:44:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.882+0000 7f476d000640 1 --2- 172.21.15.47:0/929274932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561b9e0 0x557985624d30 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f475c0058c0 tx=0x7f475c00ea60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dbef90532a2e0ca server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.882+0000 7f474ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.882+0000 7f474ea00640 1 -- 172.21.15.47:0/929274932 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c004230 con 0x55798561b9e0 2024-03-21T11:44:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.882+0000 7f4772766740 1 -- 172.21.15.47:0/929274932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55798558c800 con 0x55798561b9e0 2024-03-21T11:44:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.882+0000 7f474ea00640 1 -- 172.21.15.47:0/929274932 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f475c0043d0 con 0x55798561b9e0 2024-03-21T11:44:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.883+0000 7f474ea00640 1 -- 172.21.15.47:0/929274932 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f475c010640 con 0x55798561b9e0 2024-03-21T11:44:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.883+0000 7f4772766740 10 client.32212 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.883+0000 7f4772766740 1 -- 172.21.15.47:0/929274932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55798535d7d0 con 0x55798561b9e0 2024-03-21T11:44:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.883+0000 7f474ea00640 1 -- 172.21.15.47:0/929274932 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55798558c800 con 0x55798561b9e0 2024-03-21T11:44:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.883+0000 7f474ea00640 1 -- 172.21.15.47:0/929274932 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f475c003bc0 con 0x55798561b9e0 2024-03-21T11:44:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.884+0000 7f4772766740 10 client.32212 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.884+0000 7f4772766740 10 client.32212 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:44:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.884+0000 7f4772766740 20 client.32212 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.884+0000 7f4772766740 1 --2- 172.21.15.47:0/929274932 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x557985630360 0x557985632780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.884+0000 7f4772766740 4 client.32212 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.884+0000 7f4772766740 1 -- 172.21.15.47:0/929274932 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557985586b50 con 0x557985630360 2024-03-21T11:44:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.884+0000 7f4767e00640 1 --2- 172.21.15.47:0/929274932 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x557985630360 0x557985632780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.885+0000 7f4767e00640 1 --2- 172.21.15.47:0/929274932 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x557985630360 0x557985632780 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de775e1a63ac37d8 server_cookie=27f26b4398f73849 in_seq=0 out_seq=0 2024-03-21T11:44:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.885+0000 7f474ea00640 10 client.32212 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:44:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.886+0000 7f474ea00640 1 -- 172.21.15.47:0/929274932 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557985586b50 con 0x557985630360 2024-03-21T11:44:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.886+0000 7f474ea00640 10 client.32212 handle_command_reply: tid=0 2024-03-21T11:44:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.886+0000 7f4772766740 1 client.32212 shutdown 2024-03-21T11:44:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.886+0000 7f4772766740 1 -- 172.21.15.47:0/929274932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561b9e0 msgr2=0x557985624d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.886+0000 7f4772766740 1 --2- 172.21.15.47:0/929274932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561b9e0 0x557985624d30 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f475c0058c0 tx=0x7f475c00ea60 comp rx=0 tx=0).stop 2024-03-21T11:44:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.886+0000 7f4772766740 1 -- 172.21.15.47:0/929274932 shutdown_connections 2024-03-21T11:44:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.886+0000 7f4772766740 1 --2- 172.21.15.47:0/929274932 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x557985630360 0x557985632780 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.886+0000 7f4772766740 1 --2- 172.21.15.47:0/929274932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561c3a0 0x55798562bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.886+0000 7f4772766740 1 --2- 172.21.15.47:0/929274932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561b9e0 0x557985624d30 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.886+0000 7f4772766740 1 --2- 172.21.15.47:0/929274932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561a810 0x5579856247f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.886+0000 7f4772766740 1 -- 172.21.15.47:0/929274932 >> 172.21.15.47:0/929274932 conn(0x557985615c00 msgr2=0x557985614eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.887+0000 7f474ea00640 0 client.32212 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:44:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.887+0000 7f4772766740 1 -- 172.21.15.47:0/929274932 shutdown_connections 2024-03-21T11:44:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.887+0000 7f4772766740 1 -- 172.21.15.47:0/929274932 wait complete. 2024-03-21T11:44:22.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.887+0000 7f4772766740 20 client.32212 trim_cache size 0 max 16384 2024-03-21T11:44:22.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.904+0000 7f4772766740 1 Processor -- start 2024-03-21T11:44:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.904+0000 7f4772766740 1 -- start start 2024-03-21T11:44:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.904+0000 7f4772766740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561a810 0x557985623e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.904+0000 7f4772766740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561b9e0 0x55798562bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.904+0000 7f4772766740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561c3a0 0x55798561f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.904+0000 7f4772766740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47681b06d0 con 0x55798561a810 2024-03-21T11:44:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.904+0000 7f4772766740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47681b2050 con 0x55798561b9e0 2024-03-21T11:44:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.904+0000 7f4772766740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47681b0d30 con 0x55798561c3a0 2024-03-21T11:44:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.905+0000 7f476d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561b9e0 0x55798562bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.905+0000 7f476d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561b9e0 0x55798562bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34708/0 (socket says 172.21.15.47:34708) 2024-03-21T11:44:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.905+0000 7f476d000640 1 -- 172.21.15.47:0/2388840069 learned_addr learned my addr 172.21.15.47:0/2388840069 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.905+0000 7f4767e00640 1 --2- 172.21.15.47:0/2388840069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561a810 0x557985623e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.905+0000 7f4767e00640 1 --2- 172.21.15.47:0/2388840069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561c3a0 0x55798561f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.905+0000 7f476d000640 1 -- 172.21.15.47:0/2388840069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561c3a0 msgr2=0x55798561f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.905+0000 7f476d000640 1 --2- 172.21.15.47:0/2388840069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561c3a0 0x55798561f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.905+0000 7f476d000640 1 -- 172.21.15.47:0/2388840069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561a810 msgr2=0x557985623e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.905+0000 7f476d000640 1 --2- 172.21.15.47:0/2388840069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561a810 0x557985623e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.905+0000 7f476d000640 1 -- 172.21.15.47:0/2388840069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55798561a410 con 0x55798561b9e0 2024-03-21T11:44:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.905+0000 7f4767e00640 1 --2- 172.21.15.47:0/2388840069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561a810 0x557985623e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:44:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.905+0000 7f476d000640 1 --2- 172.21.15.47:0/2388840069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561b9e0 0x55798562bd30 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f475c001eb0 tx=0x7f475c018890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c87273c18254d888 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.906+0000 7f4759600640 1 -- 172.21.15.47:0/2388840069 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c018bc0 con 0x55798561b9e0 2024-03-21T11:44:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.906+0000 7f4759600640 1 -- 172.21.15.47:0/2388840069 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f475c0044e0 con 0x55798561b9e0 2024-03-21T11:44:22.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.906+0000 7f4759600640 1 -- 172.21.15.47:0/2388840069 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0107e0 con 0x55798561b9e0 2024-03-21T11:44:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.906+0000 7f4772766740 1 -- 172.21.15.47:0/2388840069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561b9e0 msgr2=0x55798562bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.906+0000 7f4772766740 1 --2- 172.21.15.47:0/2388840069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561b9e0 0x55798562bd30 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f475c001eb0 tx=0x7f475c018890 comp rx=0 tx=0).stop 2024-03-21T11:44:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.907+0000 7f4772766740 1 -- 172.21.15.47:0/2388840069 shutdown_connections 2024-03-21T11:44:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.907+0000 7f4772766740 1 --2- 172.21.15.47:0/2388840069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561c3a0 0x55798561f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.907+0000 7f4772766740 1 --2- 172.21.15.47:0/2388840069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561b9e0 0x55798562bd30 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.907+0000 7f4772766740 1 --2- 172.21.15.47:0/2388840069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561a810 0x557985623e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.907+0000 7f4772766740 1 -- 172.21.15.47:0/2388840069 >> 172.21.15.47:0/2388840069 conn(0x557985615c00 msgr2=0x55798561ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.907+0000 7f4772766740 1 -- 172.21.15.47:0/2388840069 shutdown_connections 2024-03-21T11:44:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.907+0000 7f4772766740 1 -- 172.21.15.47:0/2388840069 wait complete. 2024-03-21T11:44:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.907+0000 7f4772766740 1 -- start start 2024-03-21T11:44:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.907+0000 7f4772766740 1 Processor -- start 2024-03-21T11:44:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.908+0000 7f4772766740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561a810 0x55798562b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.908+0000 7f4772766740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561b9e0 0x55798562bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.908+0000 7f4772766740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561c3a0 0x557985629ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.908+0000 7f4772766740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47681b19f0 con 0x55798561b9e0 2024-03-21T11:44:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.908+0000 7f4772766740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47681b1390 con 0x55798561c3a0 2024-03-21T11:44:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.908+0000 7f4772766740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47681ae890 con 0x55798561a810 2024-03-21T11:44:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.908+0000 7f4772766740 10 client.0 _ll_register_callbacks cb 0x7f475c009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.908+0000 7f476d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561b9e0 0x55798562bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.908+0000 7f4767e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561a810 0x55798562b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.908+0000 7f476d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561b9e0 0x55798562bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40024/0 (socket says 172.21.15.47:40024) 2024-03-21T11:44:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.908+0000 7f476d000640 1 -- 172.21.15.47:0/973018798 learned_addr learned my addr 172.21.15.47:0/973018798 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.908+0000 7f4767e00640 1 --2- 172.21.15.47:0/973018798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561c3a0 0x557985629ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.909+0000 7f476d000640 1 -- 172.21.15.47:0/973018798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561a810 msgr2=0x55798562b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.909+0000 7f476d000640 1 --2- 172.21.15.47:0/973018798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561a810 0x55798562b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.909+0000 7f476d000640 1 -- 172.21.15.47:0/973018798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561c3a0 msgr2=0x557985629ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.909+0000 7f476d000640 1 --2- 172.21.15.47:0/973018798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561c3a0 0x557985629ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.909+0000 7f476d000640 1 -- 172.21.15.47:0/973018798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55798562a1b0 con 0x55798561b9e0 2024-03-21T11:44:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.909+0000 7f476d000640 1 --2- 172.21.15.47:0/973018798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561b9e0 0x55798562bad0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f475c009250 tx=0x7f475c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88c4159fdffb2b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.909+0000 7f474ea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:44:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.910+0000 7f474ea00640 1 -- 172.21.15.47:0/973018798 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c018db0 con 0x55798561b9e0 2024-03-21T11:44:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.910+0000 7f4772766740 1 -- 172.21.15.47:0/973018798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f475c0044e0 con 0x55798561b9e0 2024-03-21T11:44:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.910+0000 7f474ea00640 1 -- 172.21.15.47:0/973018798 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f475c0107e0 con 0x55798561b9e0 2024-03-21T11:44:22.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.910+0000 7f474ea00640 1 -- 172.21.15.47:0/973018798 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f475c0109e0 con 0x55798561b9e0 2024-03-21T11:44:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.910+0000 7f4772766740 10 client.21585 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.910+0000 7f4772766740 1 -- 172.21.15.47:0/973018798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55798568c4f0 con 0x55798561b9e0 2024-03-21T11:44:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.910+0000 7f474ea00640 1 -- 172.21.15.47:0/973018798 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f475c0044e0 con 0x55798561b9e0 2024-03-21T11:44:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.911+0000 7f474ea00640 1 -- 172.21.15.47:0/973018798 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f475c039470 con 0x55798561b9e0 2024-03-21T11:44:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.911+0000 7f4772766740 10 client.21585 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.911+0000 7f4772766740 10 client.21585 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:44:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.911+0000 7f4772766740 20 client.21585 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.911+0000 7f4772766740 1 --2- 172.21.15.47:0/973018798 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55798568ccb0 0x55798568f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.911+0000 7f4772766740 4 client.21585 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:44:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.911+0000 7f4772766740 1 -- 172.21.15.47:0/973018798 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5579856245e0 con 0x55798568ccb0 2024-03-21T11:44:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.911+0000 7f4767e00640 1 --2- 172.21.15.47:0/973018798 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55798568ccb0 0x55798568f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.912+0000 7f4767e00640 1 --2- 172.21.15.47:0/973018798 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55798568ccb0 0x55798568f0d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2a488e7559dffd23 server_cookie=38b596095044f8b7 in_seq=0 out_seq=0 2024-03-21T11:44:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.912+0000 7f474ea00640 10 client.21585 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:44:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.912+0000 7f474ea00640 1 -- 172.21.15.47:0/973018798 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5579856245e0 con 0x55798568ccb0 2024-03-21T11:44:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.912+0000 7f474ea00640 10 client.21585 handle_command_reply: tid=0 2024-03-21T11:44:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.912+0000 7f4772766740 1 client.21585 shutdown 2024-03-21T11:44:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.912+0000 7f4772766740 1 -- 172.21.15.47:0/973018798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561b9e0 msgr2=0x55798562bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.912+0000 7f4772766740 1 --2- 172.21.15.47:0/973018798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561b9e0 0x55798562bad0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f475c009250 tx=0x7f475c005780 comp rx=0 tx=0).stop 2024-03-21T11:44:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.913+0000 7f4772766740 1 -- 172.21.15.47:0/973018798 shutdown_connections 2024-03-21T11:44:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.913+0000 7f4772766740 1 --2- 172.21.15.47:0/973018798 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55798568ccb0 0x55798568f0d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.913+0000 7f4772766740 1 --2- 172.21.15.47:0/973018798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55798561c3a0 0x557985629ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.913+0000 7f4772766740 1 --2- 172.21.15.47:0/973018798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55798561b9e0 0x55798562bad0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.913+0000 7f4772766740 1 --2- 172.21.15.47:0/973018798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55798561a810 0x55798562b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.913+0000 7f474ea00640 0 client.21585 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:44:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.913+0000 7f4772766740 1 -- 172.21.15.47:0/973018798 >> 172.21.15.47:0/973018798 conn(0x557985615c00 msgr2=0x55798561ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.913+0000 7f4772766740 1 -- 172.21.15.47:0/973018798 shutdown_connections 2024-03-21T11:44:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.913+0000 7f4772766740 1 -- 172.21.15.47:0/973018798 wait complete. 2024-03-21T11:44:22.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.913+0000 7f4772766740 20 client.21585 trim_cache size 0 max 16384 2024-03-21T11:44:22.917 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:44:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.914+0000 7f476fa00640 1 -- 172.21.15.47:0/4198186558 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47500a2f20 msgr2=0x7f47500a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.914+0000 7f476fa00640 1 --2- 172.21.15.47:0/4198186558 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47500a2f20 0x7f47500a53e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f47600a46c0 tx=0x7f476000efd0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.914+0000 7f476fa00640 1 -- 172.21.15.47:0/4198186558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4768105490 msgr2=0x7f4768075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.914+0000 7f476fa00640 1 --2- 172.21.15.47:0/4198186558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4768105490 0x7f4768075fc0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f476000eb10 tx=0x7f476000c2d0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.914+0000 7f476fa00640 1 -- 172.21.15.47:0/4198186558 shutdown_connections 2024-03-21T11:44:22.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.914+0000 7f476fa00640 1 --2- 172.21.15.47:0/4198186558 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f47500a2f20 0x7f47500a53e0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.914+0000 7f476fa00640 1 --2- 172.21.15.47:0/4198186558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4768105490 0x7f4768075fc0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.914+0000 7f476fa00640 1 --2- 172.21.15.47:0/4198186558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4768104ad0 0x7f4768079990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.914+0000 7f476fa00640 1 --2- 172.21.15.47:0/4198186558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47681038e0 0x7f4768079450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.914+0000 7f476fa00640 1 -- 172.21.15.47:0/4198186558 >> 172.21.15.47:0/4198186558 conn(0x7f47680fee50 msgr2=0x7f4768107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.915+0000 7f476fa00640 1 -- 172.21.15.47:0/4198186558 shutdown_connections 2024-03-21T11:44:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:22.915+0000 7f476fa00640 1 -- 172.21.15.47:0/4198186558 wait complete. 2024-03-21T11:44:22.933 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:44:22.933 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:44:22.971 INFO:tasks.workunit.client.0.smithi047.stdout: 10 188164 22.87 MB/sec execute 306 sec latency 133.206 ms 2024-03-21T11:44:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:22 smithi100 ceph-mon[36245]: pgmap v1031: 129 pgs: 129 active+clean; 609 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.2 MiB/s wr, 583 op/s 2024-03-21T11:44:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:22 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:22 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1858323942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:44:23.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.245+0000 7fb865e00640 1 -- 172.21.15.47:0/3675071711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb860104930 msgr2=0x7fb860106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.245+0000 7fb865e00640 1 --2- 172.21.15.47:0/3675071711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb860104930 0x7fb860106d20 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009900 tx=0x7fb84c02bc50 comp rx=0 tx=0).stop 2024-03-21T11:44:23.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.245+0000 7fb865e00640 1 -- 172.21.15.47:0/3675071711 shutdown_connections 2024-03-21T11:44:23.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.245+0000 7fb865e00640 1 --2- 172.21.15.47:0/3675071711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860107260 0x7fb8601096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.245+0000 7fb865e00640 1 --2- 172.21.15.47:0/3675071711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb860104930 0x7fb860106d20 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.245+0000 7fb865e00640 1 --2- 172.21.15.47:0/3675071711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb860102000 0x7fb8601043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.245+0000 7fb865e00640 1 -- 172.21.15.47:0/3675071711 >> 172.21.15.47:0/3675071711 conn(0x7fb8600fba10 msgr2=0x7fb8600fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:23.248 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:22 smithi102 ceph-mon[36748]: pgmap v1031: 129 pgs: 129 active+clean; 609 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.2 MiB/s wr, 583 op/s 2024-03-21T11:44:23.248 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.248 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.248 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.248 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.248 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.248 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:22 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.248 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:22 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1858323942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:44:23.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.246+0000 7fb865e00640 1 -- 172.21.15.47:0/3675071711 shutdown_connections 2024-03-21T11:44:23.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.246+0000 7fb865e00640 1 -- 172.21.15.47:0/3675071711 wait complete. 2024-03-21T11:44:23.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.247+0000 7fb865e00640 1 Processor -- start 2024-03-21T11:44:23.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.250+0000 7fb865e00640 1 -- start start 2024-03-21T11:44:23.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.250+0000 7fb865e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb860102000 0x7fb860101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.250+0000 7fb865e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860107260 0x7fb8600fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.250+0000 7fb865e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8601004f0 0x7fb8601009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.250+0000 7fb865e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86010d2c0 con 0x7fb860107260 2024-03-21T11:44:23.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.250+0000 7fb865e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86010d140 con 0x7fb860102000 2024-03-21T11:44:23.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.250+0000 7fb865e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86010d440 con 0x7fb8601004f0 2024-03-21T11:44:23.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.251+0000 7fb864a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb860102000 0x7fb860101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.251+0000 7fb864a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb860102000 0x7fb860101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34738/0 (socket says 172.21.15.47:34738) 2024-03-21T11:44:23.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.251+0000 7fb864a00640 1 -- 172.21.15.47:0/349572206 learned_addr learned my addr 172.21.15.47:0/349572206 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:23.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.251+0000 7fb864a00640 1 -- 172.21.15.47:0/349572206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8601004f0 msgr2=0x7fb8601009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.251+0000 7fb865400640 1 --2- 172.21.15.47:0/349572206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8601004f0 0x7fb8601009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.251+0000 7fb864a00640 1 --2- 172.21.15.47:0/349572206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8601004f0 0x7fb8601009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.251+0000 7fb864a00640 1 -- 172.21.15.47:0/349572206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860107260 msgr2=0x7fb8600fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.251+0000 7fb864a00640 1 --2- 172.21.15.47:0/349572206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860107260 0x7fb8600fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.251+0000 7fb864a00640 1 -- 172.21.15.47:0/349572206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8601a7b90 con 0x7fb860102000 2024-03-21T11:44:23.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.252+0000 7fb864a00640 1 --2- 172.21.15.47:0/349572206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb860102000 0x7fb860101970 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fb85400d950 tx=0x7fb85400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:23.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.252+0000 7fb85d600640 1 -- 172.21.15.47:0/349572206 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854017070 con 0x7fb860102000 2024-03-21T11:44:23.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.252+0000 7fb865e00640 1 -- 172.21.15.47:0/349572206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8601a7e80 con 0x7fb860102000 2024-03-21T11:44:23.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.252+0000 7fb85d600640 1 -- 172.21.15.47:0/349572206 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb854009c30 con 0x7fb860102000 2024-03-21T11:44:23.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.253+0000 7fb85d600640 1 -- 172.21.15.47:0/349572206 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85400f690 con 0x7fb860102000 2024-03-21T11:44:23.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.253+0000 7fb865e00640 1 -- 172.21.15.47:0/349572206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8601a83d0 con 0x7fb860102000 2024-03-21T11:44:23.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.262+0000 7fb86a02e740 1 Processor -- start 2024-03-21T11:44:23.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.262+0000 7fb86a02e740 1 -- start start 2024-03-21T11:44:23.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.262+0000 7fb86a02e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa19f0 0x55641caa1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.263+0000 7fb86a02e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa2bc0 0x55641caa3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.263+0000 7fb865400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa19f0 0x55641caa1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.263+0000 7fb86a02e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa3580 0x55641caa9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.263+0000 7fb86a02e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641ca062c0 con 0x55641caa19f0 2024-03-21T11:44:23.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.263+0000 7fb86a02e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641c9aec90 con 0x55641caa2bc0 2024-03-21T11:44:23.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.263+0000 7fb86a02e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641ca02360 con 0x55641caa3580 2024-03-21T11:44:23.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.264+0000 7fb85fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa3580 0x55641caa9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.264+0000 7fb85fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa3580 0x55641caa9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44710/0 (socket says 172.21.15.47:44710) 2024-03-21T11:44:23.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.264+0000 7fb85fe00640 1 -- 172.21.15.47:0/2507838760 learned_addr learned my addr 172.21.15.47:0/2507838760 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:23.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.267+0000 7fb85fe00640 1 -- 172.21.15.47:0/2507838760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa2bc0 msgr2=0x55641caa3040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:44:23.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.267+0000 7fb85fe00640 1 --2- 172.21.15.47:0/2507838760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa2bc0 0x55641caa3040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.267+0000 7fb85fe00640 1 -- 172.21.15.47:0/2507838760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa19f0 msgr2=0x55641caa1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:23.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.267+0000 7fb85fe00640 1 --2- 172.21.15.47:0/2507838760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa19f0 0x55641caa1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.267+0000 7fb85fe00640 1 -- 172.21.15.47:0/2507838760 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55641c9d3080 con 0x55641caa3580 2024-03-21T11:44:23.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.267+0000 7fb85fe00640 1 --2- 172.21.15.47:0/2507838760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa3580 0x55641caa9e10 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009850 tx=0x7fb84c005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0b1951592a26edc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:23.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.268+0000 7fb851c00640 1 -- 172.21.15.47:0/2507838760 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c041070 con 0x55641caa3580 2024-03-21T11:44:23.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.268+0000 7fb85d600640 1 -- 172.21.15.47:0/349572206 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb8540109d0 con 0x7fb860102000 2024-03-21T11:44:23.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.269+0000 7fb851c00640 1 -- 172.21.15.47:0/2507838760 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb84c02cba0 con 0x55641caa3580 2024-03-21T11:44:23.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.270+0000 7fb86a02e740 1 -- 172.21.15.47:0/2507838760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa3580 msgr2=0x55641caa9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.270+0000 7fb86a02e740 1 --2- 172.21.15.47:0/2507838760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa3580 0x55641caa9e10 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009850 tx=0x7fb84c005be0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.271+0000 7fb85d600640 1 --2- 172.21.15.47:0/349572206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb8480a2f70 0x7fb8480a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.271+0000 7fb85d600640 1 -- 172.21.15.47:0/349572206 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb8540ced70 con 0x7fb860102000 2024-03-21T11:44:23.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.272+0000 7fb865400640 1 --2- 172.21.15.47:0/349572206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb8480a2f70 0x7fb8480a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.272+0000 7fb86a02e740 1 -- 172.21.15.47:0/2507838760 shutdown_connections 2024-03-21T11:44:23.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.272+0000 7fb86a02e740 1 --2- 172.21.15.47:0/2507838760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa3580 0x55641caa9e10 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.272+0000 7fb86a02e740 1 --2- 172.21.15.47:0/2507838760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa2bc0 0x55641caa3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.272+0000 7fb86a02e740 1 --2- 172.21.15.47:0/2507838760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa19f0 0x55641caa1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.272+0000 7fb86a02e740 1 -- 172.21.15.47:0/2507838760 >> 172.21.15.47:0/2507838760 conn(0x55641ca9cde0 msgr2=0x55641ca9d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:23.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.273+0000 7fb865400640 1 --2- 172.21.15.47:0/349572206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb8480a2f70 0x7fb8480a5430 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x55641caa2ac0 tx=0x7fb8580075c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:23.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.276+0000 7fb86a02e740 1 -- 172.21.15.47:0/2507838760 shutdown_connections 2024-03-21T11:44:23.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.276+0000 7fb86a02e740 1 -- 172.21.15.47:0/2507838760 wait complete. 2024-03-21T11:44:23.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.276+0000 7fb86a02e740 1 -- start start 2024-03-21T11:44:23.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.277+0000 7fb86a02e740 1 Processor -- start 2024-03-21T11:44:23.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.279+0000 7fb86a02e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa19f0 0x55641caabc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.279+0000 7fb86a02e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa2bc0 0x55641caac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.279+0000 7fb86a02e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa3580 0x55641cab31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.279+0000 7fb86a02e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641caaa5d0 con 0x55641caa2bc0 2024-03-21T11:44:23.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.279+0000 7fb86a02e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641c9c20d0 con 0x55641caa3580 2024-03-21T11:44:23.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.279+0000 7fb86a02e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55641caaa750 con 0x55641caa19f0 2024-03-21T11:44:23.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.279+0000 7fb86a02e740 10 client.0 _ll_register_callbacks cb 0x55641c9c6e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:23.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.280+0000 7fb85fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa2bc0 0x55641caac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.280+0000 7fb85fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa2bc0 0x55641caac1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40058/0 (socket says 172.21.15.47:40058) 2024-03-21T11:44:23.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.280+0000 7fb85fe00640 1 -- 172.21.15.47:0/2093422032 learned_addr learned my addr 172.21.15.47:0/2093422032 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:23.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.281+0000 7fb864a00640 1 --2- 172.21.15.47:0/2093422032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa19f0 0x55641caabc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.282+0000 7fb864a00640 1 -- 172.21.15.47:0/2093422032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa3580 msgr2=0x55641cab31f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:44:23.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.282+0000 7fb864a00640 1 --2- 172.21.15.47:0/2093422032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa3580 0x55641cab31f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.282+0000 7fb864a00640 1 -- 172.21.15.47:0/2093422032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa2bc0 msgr2=0x55641caac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:23.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.282+0000 7fb864a00640 1 --2- 172.21.15.47:0/2093422032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa2bc0 0x55641caac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.282+0000 7fb864a00640 1 -- 172.21.15.47:0/2093422032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55641c71a3f0 con 0x55641caa19f0 2024-03-21T11:44:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.283+0000 7fb846000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.283+0000 7fb864a00640 1 --2- 172.21.15.47:0/2093422032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa19f0 0x55641caabc70 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fb8540cc530 tx=0x7fb85409ab70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbe9ef4c0715dde9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:23.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.284+0000 7fb846000640 1 -- 172.21.15.47:0/2093422032 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854020040 con 0x55641caa19f0 2024-03-21T11:44:23.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.284+0000 7fb846000640 1 -- 172.21.15.47:0/2093422032 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb8540bb070 con 0x55641caa19f0 2024-03-21T11:44:23.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.284+0000 7fb86a02e740 1 -- 172.21.15.47:0/2093422032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb84c02cba0 con 0x55641caa19f0 2024-03-21T11:44:23.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.285+0000 7fb846000640 1 -- 172.21.15.47:0/2093422032 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8540b4230 con 0x55641caa19f0 2024-03-21T11:44:23.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.287+0000 7fb86a02e740 10 client.32227 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:23.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.287+0000 7fb86a02e740 1 -- 172.21.15.47:0/2093422032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55641c7e47d0 con 0x55641caa19f0 2024-03-21T11:44:23.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.291+0000 7fb846000640 1 -- 172.21.15.47:0/2093422032 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb84c02cba0 con 0x55641caa19f0 2024-03-21T11:44:23.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.291+0000 7fb846000640 1 -- 172.21.15.47:0/2093422032 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb85409a660 con 0x55641caa19f0 2024-03-21T11:44:23.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.292+0000 7fb86a02e740 10 client.32227 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:23.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.292+0000 7fb86a02e740 10 client.32227 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:44:23.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.292+0000 7fb86a02e740 20 client.32227 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:23.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.292+0000 7fb86a02e740 1 --2- 172.21.15.47:0/2093422032 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55641cab7630 0x55641cab9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.292+0000 7fb86a02e740 4 client.32227 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:23.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.292+0000 7fb86a02e740 1 -- 172.21.15.47:0/2093422032 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55641ca13800 con 0x55641cab7630 2024-03-21T11:44:23.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.292+0000 7fb85fe00640 1 --2- 172.21.15.47:0/2093422032 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55641cab7630 0x55641cab9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.295+0000 7fb85fe00640 1 --2- 172.21.15.47:0/2093422032 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55641cab7630 0x55641cab9a60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=12853800062a84a6 server_cookie=b6e43264a3c2c757 in_seq=0 out_seq=0 2024-03-21T11:44:23.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.295+0000 7fb846000640 10 client.32227 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:44:23.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.299+0000 7fb846000640 1 -- 172.21.15.47:0/2093422032 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55641ca13800 con 0x55641cab7630 2024-03-21T11:44:23.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.299+0000 7fb846000640 10 client.32227 handle_command_reply: tid=0 2024-03-21T11:44:23.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.299+0000 7fb86a02e740 1 client.32227 shutdown 2024-03-21T11:44:23.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.299+0000 7fb86a02e740 1 -- 172.21.15.47:0/2093422032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa19f0 msgr2=0x55641caabc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.299+0000 7fb86a02e740 1 --2- 172.21.15.47:0/2093422032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa19f0 0x55641caabc70 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fb8540cc530 tx=0x7fb85409ab70 comp rx=0 tx=0).stop 2024-03-21T11:44:23.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.300+0000 7fb86a02e740 1 -- 172.21.15.47:0/2093422032 shutdown_connections 2024-03-21T11:44:23.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.300+0000 7fb86a02e740 1 --2- 172.21.15.47:0/2093422032 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55641cab7630 0x55641cab9a60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.300+0000 7fb86a02e740 1 --2- 172.21.15.47:0/2093422032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa3580 0x55641cab31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.300+0000 7fb846000640 0 client.32227 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:44:23.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.300+0000 7fb86a02e740 1 --2- 172.21.15.47:0/2093422032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa2bc0 0x55641caac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.300+0000 7fb86a02e740 1 --2- 172.21.15.47:0/2093422032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa19f0 0x55641caabc70 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.300+0000 7fb86a02e740 1 -- 172.21.15.47:0/2093422032 >> 172.21.15.47:0/2093422032 conn(0x55641ca9cde0 msgr2=0x55641ca9c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:23.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.300+0000 7fb86a02e740 1 -- 172.21.15.47:0/2093422032 shutdown_connections 2024-03-21T11:44:23.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.300+0000 7fb86a02e740 1 -- 172.21.15.47:0/2093422032 wait complete. 2024-03-21T11:44:23.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.301+0000 7fb86a02e740 20 client.32227 trim_cache size 0 max 16384 2024-03-21T11:44:23.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:22 smithi047 ceph-mon[31351]: pgmap v1031: 129 pgs: 129 active+clean; 609 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.2 MiB/s wr, 583 op/s 2024-03-21T11:44:23.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:22 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:23.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:22 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1858323942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:44:23.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.339+0000 7fb86a02e740 1 Processor -- start 2024-03-21T11:44:23.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.343+0000 7fb86a02e740 1 -- start start 2024-03-21T11:44:23.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.344+0000 7fb86a02e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa19f0 0x55641caac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.344+0000 7fb86a02e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa2bc0 0x55641cab31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.344+0000 7fb86a02e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa3580 0x55641caa0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.346+0000 7fb864a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa19f0 0x55641caac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.346+0000 7fb86a02e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8601c1af0 con 0x55641caa2bc0 2024-03-21T11:44:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.346+0000 7fb86a02e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8601c3470 con 0x55641caa19f0 2024-03-21T11:44:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.346+0000 7fb86a02e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8601c2150 con 0x55641caa3580 2024-03-21T11:44:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.346+0000 7fb864a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa19f0 0x55641caac110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34754/0 (socket says 172.21.15.47:34754) 2024-03-21T11:44:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.346+0000 7fb865400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa3580 0x55641caa0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.346+0000 7fb85fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa2bc0 0x55641cab31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.346+0000 7fb865400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa3580 0x55641caa0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44722/0 (socket says 172.21.15.47:44722) 2024-03-21T11:44:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.346+0000 7fb865400640 1 -- 172.21.15.47:0/3008767727 learned_addr learned my addr 172.21.15.47:0/3008767727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.346+0000 7fb865400640 1 -- 172.21.15.47:0/3008767727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa19f0 msgr2=0x55641caac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.347+0000 7fb865400640 1 --2- 172.21.15.47:0/3008767727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa19f0 0x55641caac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.347+0000 7fb865400640 1 -- 172.21.15.47:0/3008767727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa2bc0 msgr2=0x55641cab31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.347+0000 7fb865400640 1 --2- 172.21.15.47:0/3008767727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa2bc0 0x55641cab31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.347+0000 7fb865400640 1 -- 172.21.15.47:0/3008767727 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55641caa96e0 con 0x55641caa3580 2024-03-21T11:44:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.347+0000 7fb85fe00640 1 --2- 172.21.15.47:0/3008767727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa2bc0 0x55641cab31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.347+0000 7fb864a00640 1 --2- 172.21.15.47:0/3008767727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa19f0 0x55641caac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:44:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.347+0000 7fb865400640 1 --2- 172.21.15.47:0/3008767727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa3580 0x55641caa0750 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fb8580070f0 tx=0x7fb858012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d343fe50dcf0394 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.348+0000 7fb851200640 1 -- 172.21.15.47:0/3008767727 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85800d040 con 0x55641caa3580 2024-03-21T11:44:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.348+0000 7fb851200640 1 -- 172.21.15.47:0/3008767727 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb858024070 con 0x55641caa3580 2024-03-21T11:44:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.348+0000 7fb851200640 1 -- 172.21.15.47:0/3008767727 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858012870 con 0x55641caa3580 2024-03-21T11:44:23.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.349+0000 7fb86a02e740 1 -- 172.21.15.47:0/3008767727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa3580 msgr2=0x55641caa0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.349+0000 7fb86a02e740 1 --2- 172.21.15.47:0/3008767727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa3580 0x55641caa0750 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fb8580070f0 tx=0x7fb858012000 comp rx=0 tx=0).stop 2024-03-21T11:44:23.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.349+0000 7fb86a02e740 1 -- 172.21.15.47:0/3008767727 shutdown_connections 2024-03-21T11:44:23.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.349+0000 7fb86a02e740 1 --2- 172.21.15.47:0/3008767727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa3580 0x55641caa0750 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.349+0000 7fb86a02e740 1 --2- 172.21.15.47:0/3008767727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa2bc0 0x55641cab31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.349+0000 7fb86a02e740 1 --2- 172.21.15.47:0/3008767727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa19f0 0x55641caac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.349+0000 7fb86a02e740 1 -- 172.21.15.47:0/3008767727 >> 172.21.15.47:0/3008767727 conn(0x55641ca9cde0 msgr2=0x55641caa0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.350+0000 7fb86a02e740 1 -- 172.21.15.47:0/3008767727 shutdown_connections 2024-03-21T11:44:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.350+0000 7fb86a02e740 1 -- 172.21.15.47:0/3008767727 wait complete. 2024-03-21T11:44:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.351+0000 7fb86a02e740 1 -- start start 2024-03-21T11:44:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.351+0000 7fb86a02e740 1 Processor -- start 2024-03-21T11:44:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.352+0000 7fb86a02e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa19f0 0x55641cab2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.352+0000 7fb86a02e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa2bc0 0x55641cab1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.352+0000 7fb86a02e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa3580 0x55641cab18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.352+0000 7fb86a02e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8601c2e10 con 0x55641caa19f0 2024-03-21T11:44:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.352+0000 7fb86a02e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8601c27b0 con 0x55641caa3580 2024-03-21T11:44:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.352+0000 7fb86a02e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8601aeca0 con 0x55641caa2bc0 2024-03-21T11:44:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.352+0000 7fb86a02e740 10 client.0 _ll_register_callbacks cb 0x7fb85400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:23.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.353+0000 7fb865400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa3580 0x55641cab18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.353+0000 7fb865400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa3580 0x55641cab18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34766/0 (socket says 172.21.15.47:34766) 2024-03-21T11:44:23.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.353+0000 7fb865400640 1 -- 172.21.15.47:0/3656281839 learned_addr learned my addr 172.21.15.47:0/3656281839 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:23.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.353+0000 7fb864a00640 1 --2- 172.21.15.47:0/3656281839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa19f0 0x55641cab2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.353+0000 7fb85fe00640 1 --2- 172.21.15.47:0/3656281839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa2bc0 0x55641cab1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.354+0000 7fb865400640 1 -- 172.21.15.47:0/3656281839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa2bc0 msgr2=0x55641cab1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:23.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.354+0000 7fb865400640 1 --2- 172.21.15.47:0/3656281839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa2bc0 0x55641cab1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.354+0000 7fb865400640 1 -- 172.21.15.47:0/3656281839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa19f0 msgr2=0x55641cab2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:23.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.354+0000 7fb865400640 1 --2- 172.21.15.47:0/3656281839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa19f0 0x55641cab2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.354+0000 7fb865400640 1 -- 172.21.15.47:0/3656281839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55641cab1df0 con 0x55641caa3580 2024-03-21T11:44:23.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.355+0000 7fb865400640 1 --2- 172.21.15.47:0/3656281839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa3580 0x55641cab18b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fb85801fbc0 tx=0x7fb85801fbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b9be868c3004deb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:23.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.355+0000 7fb846000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:44:23.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.356+0000 7fb846000640 1 -- 172.21.15.47:0/3656281839 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858003c40 con 0x55641caa3580 2024-03-21T11:44:23.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.356+0000 7fb846000640 1 -- 172.21.15.47:0/3656281839 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb858003de0 con 0x55641caa3580 2024-03-21T11:44:23.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.357+0000 7fb86a02e740 1 -- 172.21.15.47:0/3656281839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb858024070 con 0x55641caa3580 2024-03-21T11:44:23.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.357+0000 7fb846000640 1 -- 172.21.15.47:0/3656281839 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8580125f0 con 0x55641caa3580 2024-03-21T11:44:23.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.357+0000 7fb86a02e740 10 client.32264 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:23.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.358+0000 7fb86a02e740 1 -- 172.21.15.47:0/3656281839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55641cb13b40 con 0x55641caa3580 2024-03-21T11:44:23.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.358+0000 7fb846000640 1 -- 172.21.15.47:0/3656281839 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb858024070 con 0x55641caa3580 2024-03-21T11:44:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.361+0000 7fb846000640 1 -- 172.21.15.47:0/3656281839 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb858012790 con 0x55641caa3580 2024-03-21T11:44:23.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.364+0000 7fb86a02e740 10 client.32264 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.364+0000 7fb86a02e740 10 client.32264 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:44:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.364+0000 7fb86a02e740 20 client.32264 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.364+0000 7fb86a02e740 1 --2- 172.21.15.47:0/3656281839 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55641cab7630 0x55641cb163a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.364+0000 7fb86a02e740 4 client.32264 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:44:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.364+0000 7fb86a02e740 1 -- 172.21.15.47:0/3656281839 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55641ca0db50 con 0x55641cab7630 2024-03-21T11:44:23.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.364+0000 7fb864a00640 1 --2- 172.21.15.47:0/3656281839 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55641cab7630 0x55641cb163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.365+0000 7fb864a00640 1 --2- 172.21.15.47:0/3656281839 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55641cab7630 0x55641cb163a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5642b47f91a6ca6f server_cookie=9c320797c05b65b3 in_seq=0 out_seq=0 2024-03-21T11:44:23.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.365+0000 7fb846000640 10 client.32264 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:44:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.367+0000 7fb846000640 1 -- 172.21.15.47:0/3656281839 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55641ca0db50 con 0x55641cab7630 2024-03-21T11:44:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.367+0000 7fb846000640 10 client.32264 handle_command_reply: tid=0 2024-03-21T11:44:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.367+0000 7fb86a02e740 1 client.32264 shutdown 2024-03-21T11:44:23.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.368+0000 7fb86a02e740 1 -- 172.21.15.47:0/3656281839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa3580 msgr2=0x55641cab18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.368+0000 7fb86a02e740 1 --2- 172.21.15.47:0/3656281839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa3580 0x55641cab18b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fb85801fbc0 tx=0x7fb85801fbf0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.368+0000 7fb86a02e740 1 -- 172.21.15.47:0/3656281839 shutdown_connections 2024-03-21T11:44:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.368+0000 7fb86a02e740 1 --2- 172.21.15.47:0/3656281839 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55641cab7630 0x55641cb163a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.368+0000 7fb86a02e740 1 --2- 172.21.15.47:0/3656281839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55641caa3580 0x55641cab18b0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.368+0000 7fb86a02e740 1 --2- 172.21.15.47:0/3656281839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55641caa2bc0 0x55641cab1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.368+0000 7fb86a02e740 1 --2- 172.21.15.47:0/3656281839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55641caa19f0 0x55641cab2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.368+0000 7fb846000640 0 client.32264 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:44:23.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.368+0000 7fb86a02e740 1 -- 172.21.15.47:0/3656281839 >> 172.21.15.47:0/3656281839 conn(0x55641ca9cde0 msgr2=0x55641caabf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.368+0000 7fb86a02e740 1 -- 172.21.15.47:0/3656281839 shutdown_connections 2024-03-21T11:44:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.368+0000 7fb86a02e740 1 -- 172.21.15.47:0/3656281839 wait complete. 2024-03-21T11:44:23.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.369+0000 7fb86a02e740 20 client.32264 trim_cache size 0 max 16384 2024-03-21T11:44:23.370 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:44:23.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.375+0000 7fb865e00640 1 -- 172.21.15.47:0/349572206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb8480a2f70 msgr2=0x7fb8480a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.375+0000 7fb865e00640 1 --2- 172.21.15.47:0/349572206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb8480a2f70 0x7fb8480a5430 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x55641caa2ac0 tx=0x7fb8580075c0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.375+0000 7fb865e00640 1 -- 172.21.15.47:0/349572206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb860102000 msgr2=0x7fb860101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.375+0000 7fb865e00640 1 --2- 172.21.15.47:0/349572206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb860102000 0x7fb860101970 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fb85400d950 tx=0x7fb85400de10 comp rx=0 tx=0).stop 2024-03-21T11:44:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.375+0000 7fb865e00640 1 -- 172.21.15.47:0/349572206 shutdown_connections 2024-03-21T11:44:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.375+0000 7fb865e00640 1 --2- 172.21.15.47:0/349572206 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb8480a2f70 0x7fb8480a5430 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.375+0000 7fb865e00640 1 --2- 172.21.15.47:0/349572206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8601004f0 0x7fb8601009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.376+0000 7fb865e00640 1 --2- 172.21.15.47:0/349572206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb860107260 0x7fb8600fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.376+0000 7fb865e00640 1 --2- 172.21.15.47:0/349572206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb860102000 0x7fb860101970 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.376+0000 7fb865e00640 1 -- 172.21.15.47:0/349572206 >> 172.21.15.47:0/349572206 conn(0x7fb8600fba10 msgr2=0x7fb860105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.376+0000 7fb865e00640 1 -- 172.21.15.47:0/349572206 shutdown_connections 2024-03-21T11:44:23.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.376+0000 7fb865e00640 1 -- 172.21.15.47:0/349572206 wait complete. 2024-03-21T11:44:23.414 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:44:23.414 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:44:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.785+0000 7f8da3c00640 1 -- 172.21.15.47:0/2136062301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c109870 msgr2=0x7f8d9c110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.785+0000 7f8da3c00640 1 --2- 172.21.15.47:0/2136062301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c109870 0x7f8d9c110100 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009860 tx=0x7f8d8c02bb70 comp rx=0 tx=0).stop 2024-03-21T11:44:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.786+0000 7f8da3c00640 1 -- 172.21.15.47:0/2136062301 shutdown_connections 2024-03-21T11:44:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.786+0000 7f8da3c00640 1 --2- 172.21.15.47:0/2136062301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c109870 0x7f8d9c110100 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.786+0000 7f8da3c00640 1 --2- 172.21.15.47:0/2136062301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9c108eb0 0x7f8d9c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.786+0000 7f8da3c00640 1 --2- 172.21.15.47:0/2136062301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d9c107cc0 0x7f8d9c1080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.786+0000 7f8da3c00640 1 -- 172.21.15.47:0/2136062301 >> 172.21.15.47:0/2136062301 conn(0x7f8d9c076c60 msgr2=0x7f8d9c0790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.786+0000 7f8da3c00640 1 -- 172.21.15.47:0/2136062301 shutdown_connections 2024-03-21T11:44:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.786+0000 7f8da3c00640 1 -- 172.21.15.47:0/2136062301 wait complete. 2024-03-21T11:44:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.788+0000 7f8da3c00640 1 Processor -- start 2024-03-21T11:44:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.788+0000 7f8da3c00640 1 -- start start 2024-03-21T11:44:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.789+0000 7f8da3c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c107cc0 0x7f8d9c07edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.789+0000 7f8da3c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d9c108eb0 0x7f8d9c07f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.790+0000 7f8da3c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9c109870 0x7f8d9c1ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.790+0000 7f8da3c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c113920 con 0x7f8d9c108eb0 2024-03-21T11:44:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.790+0000 7f8da3c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1137a0 con 0x7f8d9c109870 2024-03-21T11:44:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.790+0000 7f8da3c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c113aa0 con 0x7f8d9c107cc0 2024-03-21T11:44:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.790+0000 7f8da1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c107cc0 0x7f8d9c07edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.790+0000 7f8da1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c107cc0 0x7f8d9c07edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44738/0 (socket says 172.21.15.47:44738) 2024-03-21T11:44:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.790+0000 7f8da1200640 1 -- 172.21.15.47:0/1330829607 learned_addr learned my addr 172.21.15.47:0/1330829607 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.791+0000 7f8d9be00640 1 --2- 172.21.15.47:0/1330829607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d9c108eb0 0x7f8d9c07f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.791+0000 7f8da1c00640 1 --2- 172.21.15.47:0/1330829607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9c109870 0x7f8d9c1ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.791+0000 7f8da1200640 1 -- 172.21.15.47:0/1330829607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9c109870 msgr2=0x7f8d9c1ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.791+0000 7f8da1200640 1 --2- 172.21.15.47:0/1330829607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9c109870 0x7f8d9c1ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.791+0000 7f8da1200640 1 -- 172.21.15.47:0/1330829607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d9c108eb0 msgr2=0x7f8d9c07f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.791+0000 7f8da1200640 1 --2- 172.21.15.47:0/1330829607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d9c108eb0 0x7f8d9c07f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.791+0000 7f8da1200640 1 -- 172.21.15.47:0/1330829607 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c082470 con 0x7f8d9c107cc0 2024-03-21T11:44:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.792+0000 7f8da1200640 1 --2- 172.21.15.47:0/1330829607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c107cc0 0x7f8d9c07edb0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400ccb0 tx=0x7f8d9400a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.792+0000 7f8d99600640 1 -- 172.21.15.47:0/1330829607 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94003b70 con 0x7f8d9c107cc0 2024-03-21T11:44:23.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.792+0000 7f8d99600640 1 -- 172.21.15.47:0/1330829607 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d9400ae50 con 0x7f8d9c107cc0 2024-03-21T11:44:23.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.792+0000 7f8d99600640 1 -- 172.21.15.47:0/1330829607 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400f870 con 0x7f8d9c107cc0 2024-03-21T11:44:23.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.792+0000 7f8da3c00640 1 -- 172.21.15.47:0/1330829607 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c1ae5d0 con 0x7f8d9c107cc0 2024-03-21T11:44:23.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.792+0000 7f8da3c00640 1 -- 172.21.15.47:0/1330829607 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c1aead0 con 0x7f8d9c107cc0 2024-03-21T11:44:23.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.800+0000 7f8da69d0740 1 Processor -- start 2024-03-21T11:44:23.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.801+0000 7f8da69d0740 1 -- start start 2024-03-21T11:44:23.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.801+0000 7f8da69d0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d52810 0x5590f1d52c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.802+0000 7f8da69d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d539e0 0x5590f1d53e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.803+0000 7f8da69d0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d543a0 0x5590f1d5ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.804+0000 7f8da69d0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f1cb72c0 con 0x5590f1d52810 2024-03-21T11:44:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.804+0000 7f8da69d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f1c5fc90 con 0x5590f1d539e0 2024-03-21T11:44:23.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.804+0000 7f8da69d0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f1cb3360 con 0x5590f1d543a0 2024-03-21T11:44:23.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.804+0000 7f8d99600640 1 -- 172.21.15.47:0/1330829607 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8d9400fa10 con 0x7f8d9c107cc0 2024-03-21T11:44:23.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.804+0000 7f8d9be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d543a0 0x5590f1d5ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.804+0000 7f8d9be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d543a0 0x5590f1d5ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44754/0 (socket says 172.21.15.47:44754) 2024-03-21T11:44:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.804+0000 7f8d9be00640 1 -- 172.21.15.47:0/2440369822 learned_addr learned my addr 172.21.15.47:0/2440369822 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:23.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.804+0000 7f8da1c00640 1 --2- 172.21.15.47:0/2440369822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d52810 0x5590f1d52c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.805+0000 7f8d9be00640 1 -- 172.21.15.47:0/2440369822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d539e0 msgr2=0x5590f1d53e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:44:23.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.805+0000 7f8d9be00640 1 --2- 172.21.15.47:0/2440369822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d539e0 0x5590f1d53e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.805+0000 7f8d9be00640 1 -- 172.21.15.47:0/2440369822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d52810 msgr2=0x5590f1d52c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:23.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.805+0000 7f8d9be00640 1 --2- 172.21.15.47:0/2440369822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d52810 0x5590f1d52c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.805+0000 7f8d9be00640 1 -- 172.21.15.47:0/2440369822 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590f1c84080 con 0x5590f1d543a0 2024-03-21T11:44:23.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.806+0000 7f8d9be00640 1 --2- 172.21.15.47:0/2440369822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d543a0 0x5590f1d5ac30 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009b60 tx=0x7f8d8800e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf47bbf8ffef925f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:23.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.806+0000 7f8d92000640 1 -- 172.21.15.47:0/2440369822 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88016070 con 0x5590f1d543a0 2024-03-21T11:44:23.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.807+0000 7f8d92000640 1 -- 172.21.15.47:0/2440369822 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d88009e50 con 0x5590f1d543a0 2024-03-21T11:44:23.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.807+0000 7f8d99600640 1 --2- 172.21.15.47:0/1330829607 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d800a3020 0x7f8d800a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.807+0000 7f8d99600640 1 -- 172.21.15.47:0/1330829607 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8d940cfc90 con 0x7f8d9c107cc0 2024-03-21T11:44:23.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.808+0000 7f8d92000640 1 -- 172.21.15.47:0/2440369822 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d880107f0 con 0x5590f1d543a0 2024-03-21T11:44:23.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.808+0000 7f8da1c00640 1 --2- 172.21.15.47:0/1330829607 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d800a3020 0x7f8d800a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.809+0000 7f8da69d0740 1 -- 172.21.15.47:0/2440369822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d543a0 msgr2=0x5590f1d5ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.809+0000 7f8da69d0740 1 --2- 172.21.15.47:0/2440369822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d543a0 0x5590f1d5ac30 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009b60 tx=0x7f8d8800e6c0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.810+0000 7f8da1c00640 1 --2- 172.21.15.47:0/1330829607 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d800a3020 0x7f8d800a54e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009830 tx=0x7f8d8c0058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:23.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.810+0000 7f8da69d0740 1 -- 172.21.15.47:0/2440369822 shutdown_connections 2024-03-21T11:44:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.810+0000 7f8da69d0740 1 --2- 172.21.15.47:0/2440369822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d543a0 0x5590f1d5ac30 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.810+0000 7f8da69d0740 1 --2- 172.21.15.47:0/2440369822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d539e0 0x5590f1d53e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.810+0000 7f8da69d0740 1 --2- 172.21.15.47:0/2440369822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d52810 0x5590f1d52c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.810+0000 7f8da69d0740 1 -- 172.21.15.47:0/2440369822 >> 172.21.15.47:0/2440369822 conn(0x5590f1d4dc00 msgr2=0x5590f1d4e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.810+0000 7f8da69d0740 1 -- 172.21.15.47:0/2440369822 shutdown_connections 2024-03-21T11:44:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.810+0000 7f8da69d0740 1 -- 172.21.15.47:0/2440369822 wait complete. 2024-03-21T11:44:23.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.811+0000 7f8da69d0740 1 -- start start 2024-03-21T11:44:23.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.811+0000 7f8da69d0740 1 Processor -- start 2024-03-21T11:44:23.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.812+0000 7f8da69d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d52810 0x5590f1d5cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.812+0000 7f8da69d0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d539e0 0x5590f1d5d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.812+0000 7f8da69d0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d543a0 0x5590f1d64060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.813+0000 7f8d9be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d539e0 0x5590f1d5d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.813+0000 7f8d9be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d539e0 0x5590f1d5d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44756/0 (socket says 172.21.15.47:44756) 2024-03-21T11:44:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.813+0000 7f8da1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d52810 0x5590f1d5cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.813+0000 7f8da1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d52810 0x5590f1d5cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34790/0 (socket says 172.21.15.47:34790) 2024-03-21T11:44:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.813+0000 7f8da69d0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f1d5b3b0 con 0x5590f1d543a0 2024-03-21T11:44:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.813+0000 7f8da69d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f1c730d0 con 0x5590f1d52810 2024-03-21T11:44:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.813+0000 7f8da69d0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f1d5b530 con 0x5590f1d539e0 2024-03-21T11:44:23.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.813+0000 7f8d9be00640 1 -- 172.21.15.47:0/1154617632 learned_addr learned my addr 172.21.15.47:0/1154617632 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.813+0000 7f8da69d0740 10 client.0 _ll_register_callbacks cb 0x5590f1c77e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.814+0000 7f8da1c00640 1 --2- 172.21.15.47:0/1154617632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d543a0 0x5590f1d64060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.814+0000 7f8d9be00640 1 -- 172.21.15.47:0/1154617632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d52810 msgr2=0x5590f1d5cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.814+0000 7f8d9be00640 1 --2- 172.21.15.47:0/1154617632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d52810 0x5590f1d5cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.814+0000 7f8d9be00640 1 -- 172.21.15.47:0/1154617632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d543a0 msgr2=0x5590f1d64060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.814+0000 7f8d9be00640 1 --2- 172.21.15.47:0/1154617632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d543a0 0x5590f1d64060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.814+0000 7f8d9be00640 1 -- 172.21.15.47:0/1154617632 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590f19cb3f0 con 0x5590f1d539e0 2024-03-21T11:44:23.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.814+0000 7f8da1200640 1 --2- 172.21.15.47:0/1154617632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d52810 0x5590f1d5cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:23.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.814+0000 7f8da1c00640 1 --2- 172.21.15.47:0/1154617632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d543a0 0x5590f1d64060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:44:23.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.814+0000 7f8d9be00640 1 --2- 172.21.15.47:0/1154617632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d539e0 0x5590f1d5d020 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f8d880057b0 tx=0x7f8d8800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77f3e19678f0fc4a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:23.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.815+0000 7f8d86a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.819+0000 7f8d86a00640 1 -- 172.21.15.47:0/1154617632 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88016050 con 0x5590f1d539e0 2024-03-21T11:44:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.819+0000 7f8d86a00640 1 -- 172.21.15.47:0/1154617632 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8d88011470 con 0x5590f1d539e0 2024-03-21T11:44:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.819+0000 7f8da69d0740 1 -- 172.21.15.47:0/1154617632 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d88009e50 con 0x5590f1d539e0 2024-03-21T11:44:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.819+0000 7f8d86a00640 1 -- 172.21.15.47:0/1154617632 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d88030050 con 0x5590f1d539e0 2024-03-21T11:44:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.819+0000 7f8da69d0740 10 client.32245 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.820+0000 7f8da69d0740 1 -- 172.21.15.47:0/1154617632 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5590f1a957d0 con 0x5590f1d539e0 2024-03-21T11:44:23.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.821+0000 7f8d86a00640 1 -- 172.21.15.47:0/1154617632 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d88009e50 con 0x5590f1d539e0 2024-03-21T11:44:23.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.822+0000 7f8d86a00640 1 -- 172.21.15.47:0/1154617632 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8d8802b030 con 0x5590f1d539e0 2024-03-21T11:44:23.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.822+0000 7f8da69d0740 10 client.32245 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:23.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.822+0000 7f8da69d0740 10 client.32245 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:44:23.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.822+0000 7f8da69d0740 20 client.32245 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:23.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.823+0000 7f8da69d0740 1 --2- 172.21.15.47:0/1154617632 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5590f1d68250 0x5590f1d6a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.823+0000 7f8da69d0740 4 client.32245 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:23.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.823+0000 7f8da69d0740 1 -- 172.21.15.47:0/1154617632 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590f1cc4800 con 0x5590f1d68250 2024-03-21T11:44:23.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.824+0000 7f8da1200640 1 --2- 172.21.15.47:0/1154617632 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5590f1d68250 0x5590f1d6a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.828+0000 7f8d86a00640 10 client.32245 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:44:23.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.828+0000 7f8da1200640 1 --2- 172.21.15.47:0/1154617632 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5590f1d68250 0x5590f1d6a640 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92dfd234fd464485 server_cookie=3aa106acd8f73472 in_seq=0 out_seq=0 2024-03-21T11:44:23.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.831+0000 7f8d86a00640 1 -- 172.21.15.47:0/1154617632 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5590f1cc4800 con 0x5590f1d68250 2024-03-21T11:44:23.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.831+0000 7f8d86a00640 10 client.32245 handle_command_reply: tid=0 2024-03-21T11:44:23.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.832+0000 7f8da69d0740 1 client.32245 shutdown 2024-03-21T11:44:23.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.832+0000 7f8da69d0740 1 -- 172.21.15.47:0/1154617632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d539e0 msgr2=0x5590f1d5d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.832+0000 7f8da69d0740 1 --2- 172.21.15.47:0/1154617632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d539e0 0x5590f1d5d020 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f8d880057b0 tx=0x7f8d8800ef10 comp rx=0 tx=0).stop 2024-03-21T11:44:23.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.832+0000 7f8da69d0740 1 -- 172.21.15.47:0/1154617632 shutdown_connections 2024-03-21T11:44:23.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.832+0000 7f8da69d0740 1 --2- 172.21.15.47:0/1154617632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d543a0 0x5590f1d64060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.832+0000 7f8da69d0740 1 --2- 172.21.15.47:0/1154617632 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5590f1d68250 0x5590f1d6a640 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.832+0000 7f8da69d0740 1 --2- 172.21.15.47:0/1154617632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d539e0 0x5590f1d5d020 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.832+0000 7f8d86a00640 0 client.32245 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:44:23.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.832+0000 7f8da69d0740 1 --2- 172.21.15.47:0/1154617632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d52810 0x5590f1d5cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.832+0000 7f8da69d0740 1 -- 172.21.15.47:0/1154617632 >> 172.21.15.47:0/1154617632 conn(0x5590f1d4dc00 msgr2=0x5590f1d4ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:23.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.833+0000 7f8da69d0740 1 -- 172.21.15.47:0/1154617632 shutdown_connections 2024-03-21T11:44:23.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.833+0000 7f8da69d0740 1 -- 172.21.15.47:0/1154617632 wait complete. 2024-03-21T11:44:23.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.833+0000 7f8da69d0740 20 client.32245 trim_cache size 0 max 16384 2024-03-21T11:44:23.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.864+0000 7f8da69d0740 1 Processor -- start 2024-03-21T11:44:23.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.864+0000 7f8da69d0740 1 -- start start 2024-03-21T11:44:23.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.865+0000 7f8da69d0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d52810 0x5590f1d4c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.865+0000 7f8da69d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d539e0 0x5590f1d64020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.865+0000 7f8da69d0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d543a0 0x5590f1d4cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.865+0000 7f8da69d0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1b8f60 con 0x5590f1d543a0 2024-03-21T11:44:23.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.865+0000 7f8da69d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1ba8e0 con 0x5590f1d539e0 2024-03-21T11:44:23.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.865+0000 7f8da69d0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1b95c0 con 0x5590f1d52810 2024-03-21T11:44:23.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.865+0000 7f8da1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d52810 0x5590f1d4c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.865+0000 7f8da1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d52810 0x5590f1d4c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44762/0 (socket says 172.21.15.47:44762) 2024-03-21T11:44:23.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.865+0000 7f8da1200640 1 -- 172.21.15.47:0/131854846 learned_addr learned my addr 172.21.15.47:0/131854846 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:23.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.865+0000 7f8da1c00640 1 --2- 172.21.15.47:0/131854846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d543a0 0x5590f1d4cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.866+0000 7f8d9be00640 1 --2- 172.21.15.47:0/131854846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d539e0 0x5590f1d64020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.866+0000 7f8d9be00640 1 -- 172.21.15.47:0/131854846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d52810 msgr2=0x5590f1d4c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.866+0000 7f8d9be00640 1 --2- 172.21.15.47:0/131854846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d52810 0x5590f1d4c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.866+0000 7f8d9be00640 1 -- 172.21.15.47:0/131854846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d543a0 msgr2=0x5590f1d4cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.866+0000 7f8d9be00640 1 --2- 172.21.15.47:0/131854846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d543a0 0x5590f1d4cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.866+0000 7f8d9be00640 1 -- 172.21.15.47:0/131854846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590f1d6abe0 con 0x5590f1d539e0 2024-03-21T11:44:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.866+0000 7f8da1c00640 1 --2- 172.21.15.47:0/131854846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d543a0 0x5590f1d4cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.867+0000 7f8d9be00640 1 --2- 172.21.15.47:0/131854846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d539e0 0x5590f1d64020 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800f010 tx=0x7f8d8800e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=447e4f546f1c78c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.867+0000 7f8d91600640 1 -- 172.21.15.47:0/131854846 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88039070 con 0x5590f1d539e0 2024-03-21T11:44:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.867+0000 7f8d91600640 1 -- 172.21.15.47:0/131854846 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d88010e80 con 0x5590f1d539e0 2024-03-21T11:44:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.867+0000 7f8d91600640 1 -- 172.21.15.47:0/131854846 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88039070 con 0x5590f1d539e0 2024-03-21T11:44:23.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.868+0000 7f8da69d0740 1 -- 172.21.15.47:0/131854846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d539e0 msgr2=0x5590f1d64020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.868+0000 7f8da69d0740 1 --2- 172.21.15.47:0/131854846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d539e0 0x5590f1d64020 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800f010 tx=0x7f8d8800e6c0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.868+0000 7f8da69d0740 1 -- 172.21.15.47:0/131854846 shutdown_connections 2024-03-21T11:44:23.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.868+0000 7f8da69d0740 1 --2- 172.21.15.47:0/131854846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d543a0 0x5590f1d4cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.868+0000 7f8da69d0740 1 --2- 172.21.15.47:0/131854846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d539e0 0x5590f1d64020 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.868+0000 7f8da69d0740 1 --2- 172.21.15.47:0/131854846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d52810 0x5590f1d4c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.868+0000 7f8da69d0740 1 -- 172.21.15.47:0/131854846 >> 172.21.15.47:0/131854846 conn(0x5590f1d4dc00 msgr2=0x5590f1d4dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:23.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.869+0000 7f8da69d0740 1 -- 172.21.15.47:0/131854846 shutdown_connections 2024-03-21T11:44:23.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.869+0000 7f8da69d0740 1 -- 172.21.15.47:0/131854846 wait complete. 2024-03-21T11:44:23.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.869+0000 7f8da69d0740 1 -- start start 2024-03-21T11:44:23.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.869+0000 7f8da69d0740 1 Processor -- start 2024-03-21T11:44:23.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.870+0000 7f8da69d0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d52810 0x5590f1d5f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.870+0000 7f8da69d0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d539e0 0x5590f1d5fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.870+0000 7f8da69d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d543a0 0x5590f1d62bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.870+0000 7f8da69d0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1ba280 con 0x5590f1d539e0 2024-03-21T11:44:23.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.870+0000 7f8da69d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1b9c20 con 0x5590f1d543a0 2024-03-21T11:44:23.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.870+0000 7f8da69d0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1b7120 con 0x5590f1d52810 2024-03-21T11:44:23.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.870+0000 7f8da69d0740 10 client.0 _ll_register_callbacks cb 0x7f8d880099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:23.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.870+0000 7f8da1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d52810 0x5590f1d5f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.870+0000 7f8da1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d52810 0x5590f1d5f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44768/0 (socket says 172.21.15.47:44768) 2024-03-21T11:44:23.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.870+0000 7f8da1200640 1 -- 172.21.15.47:0/2422341202 learned_addr learned my addr 172.21.15.47:0/2422341202 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:23.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.870+0000 7f8d9be00640 1 --2- 172.21.15.47:0/2422341202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d539e0 0x5590f1d5fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.870+0000 7f8da1200640 1 -- 172.21.15.47:0/2422341202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d543a0 msgr2=0x5590f1d62bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:23.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.870+0000 7f8da1c00640 1 --2- 172.21.15.47:0/2422341202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d543a0 0x5590f1d62bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.871+0000 7f8da1200640 1 --2- 172.21.15.47:0/2422341202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d543a0 0x5590f1d62bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.871+0000 7f8da1200640 1 -- 172.21.15.47:0/2422341202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d539e0 msgr2=0x5590f1d5fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:23.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.871+0000 7f8da1200640 1 --2- 172.21.15.47:0/2422341202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d539e0 0x5590f1d5fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.871+0000 7f8da1200640 1 -- 172.21.15.47:0/2422341202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590f1d631a0 con 0x5590f1d52810 2024-03-21T11:44:23.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.871+0000 7f8da1c00640 1 --2- 172.21.15.47:0/2422341202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d543a0 0x5590f1d62bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:44:23.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.872+0000 7f8da1200640 1 --2- 172.21.15.47:0/2422341202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d52810 0x5590f1d5f6c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f8d940989a0 tx=0x7f8d94004870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=329350ab00dd899b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:23.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.872+0000 7f8d86a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:23.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.873+0000 7f8d86a00640 1 -- 172.21.15.47:0/2422341202 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9401d040 con 0x5590f1d52810 2024-03-21T11:44:23.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.873+0000 7f8d86a00640 1 -- 172.21.15.47:0/2422341202 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8d9409a2d0 con 0x5590f1d52810 2024-03-21T11:44:23.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.873+0000 7f8da69d0740 1 -- 172.21.15.47:0/2422341202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d88010e80 con 0x5590f1d52810 2024-03-21T11:44:23.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.878+0000 7f8d86a00640 1 -- 172.21.15.47:0/2422341202 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d940a49e0 con 0x5590f1d52810 2024-03-21T11:44:23.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.879+0000 7f8da69d0740 10 client.32251 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:23.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.879+0000 7f8da69d0740 1 -- 172.21.15.47:0/2422341202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5590f1dc39d0 con 0x5590f1d52810 2024-03-21T11:44:23.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.881+0000 7f8d86a00640 1 -- 172.21.15.47:0/2422341202 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d88010e80 con 0x5590f1d52810 2024-03-21T11:44:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.881+0000 7f8d86a00640 1 -- 172.21.15.47:0/2422341202 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8d940a4550 con 0x5590f1d52810 2024-03-21T11:44:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.883+0000 7f8da69d0740 10 client.32251 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.883+0000 7f8da69d0740 10 client.32251 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:44:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.883+0000 7f8da69d0740 20 client.32251 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.883+0000 7f8da69d0740 1 --2- 172.21.15.47:0/2422341202 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5590f1dc41f0 0x5590f1dc65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:23.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.884+0000 7f8da69d0740 4 client.32251 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:44:23.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.884+0000 7f8da69d0740 1 -- 172.21.15.47:0/2422341202 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5590f1cbeb50 con 0x5590f1dc41f0 2024-03-21T11:44:23.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.884+0000 7f8d9be00640 1 --2- 172.21.15.47:0/2422341202 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5590f1dc41f0 0x5590f1dc65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.885+0000 7f8d9be00640 1 --2- 172.21.15.47:0/2422341202 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5590f1dc41f0 0x5590f1dc65e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=86f5e2e2d62fd2f3 server_cookie=d5aa77dd7ea06a2f in_seq=0 out_seq=0 2024-03-21T11:44:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.885+0000 7f8d86a00640 10 client.32251 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:44:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.885+0000 7f8d86a00640 1 -- 172.21.15.47:0/2422341202 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5590f1cbeb50 con 0x5590f1dc41f0 2024-03-21T11:44:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.886+0000 7f8d86a00640 10 client.32251 handle_command_reply: tid=0 2024-03-21T11:44:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.886+0000 7f8da69d0740 1 client.32251 shutdown 2024-03-21T11:44:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.886+0000 7f8da69d0740 1 -- 172.21.15.47:0/2422341202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d52810 msgr2=0x5590f1d5f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.886+0000 7f8da69d0740 1 --2- 172.21.15.47:0/2422341202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d52810 0x5590f1d5f6c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f8d940989a0 tx=0x7f8d94004870 comp rx=0 tx=0).stop 2024-03-21T11:44:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.886+0000 7f8da69d0740 1 -- 172.21.15.47:0/2422341202 shutdown_connections 2024-03-21T11:44:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.886+0000 7f8da69d0740 1 --2- 172.21.15.47:0/2422341202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f1d543a0 0x5590f1d62bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.886+0000 7f8da69d0740 1 --2- 172.21.15.47:0/2422341202 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f1d539e0 0x5590f1d5fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.886+0000 7f8da69d0740 1 --2- 172.21.15.47:0/2422341202 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5590f1dc41f0 0x5590f1dc65e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.886+0000 7f8da69d0740 1 --2- 172.21.15.47:0/2422341202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f1d52810 0x5590f1d5f6c0 secure :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f8d940989a0 tx=0x7f8d94004870 comp rx=0 tx=0).stop 2024-03-21T11:44:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.886+0000 7f8d86a00640 0 client.32251 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:44:23.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.886+0000 7f8da69d0740 1 -- 172.21.15.47:0/2422341202 >> 172.21.15.47:0/2422341202 conn(0x5590f1d4dc00 msgr2=0x5590f1d50e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:23.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.887+0000 7f8da69d0740 1 -- 172.21.15.47:0/2422341202 shutdown_connections 2024-03-21T11:44:23.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.889+0000 7f8da69d0740 1 -- 172.21.15.47:0/2422341202 wait complete. 2024-03-21T11:44:23.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.889+0000 7f8da69d0740 20 client.32251 trim_cache size 0 max 16384 2024-03-21T11:44:23.890 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:44:23.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.891+0000 7f8da3c00640 1 -- 172.21.15.47:0/1330829607 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d800a3020 msgr2=0x7f8d800a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.891+0000 7f8da3c00640 1 --2- 172.21.15.47:0/1330829607 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d800a3020 0x7f8d800a54e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009830 tx=0x7f8d8c0058b0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.891+0000 7f8da3c00640 1 -- 172.21.15.47:0/1330829607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c107cc0 msgr2=0x7f8d9c07edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:23.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.891+0000 7f8da3c00640 1 --2- 172.21.15.47:0/1330829607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c107cc0 0x7f8d9c07edb0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400ccb0 tx=0x7f8d9400a620 comp rx=0 tx=0).stop 2024-03-21T11:44:23.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.891+0000 7f8da3c00640 1 -- 172.21.15.47:0/1330829607 shutdown_connections 2024-03-21T11:44:23.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.891+0000 7f8da3c00640 1 --2- 172.21.15.47:0/1330829607 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8d800a3020 0x7f8d800a54e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.891+0000 7f8da3c00640 1 --2- 172.21.15.47:0/1330829607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9c109870 0x7f8d9c1ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.891+0000 7f8da3c00640 1 --2- 172.21.15.47:0/1330829607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d9c108eb0 0x7f8d9c07f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.891+0000 7f8da3c00640 1 --2- 172.21.15.47:0/1330829607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d9c107cc0 0x7f8d9c07edb0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:23.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.891+0000 7f8da3c00640 1 -- 172.21.15.47:0/1330829607 >> 172.21.15.47:0/1330829607 conn(0x7f8d9c076c60 msgr2=0x7f8d9c10c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:23.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.892+0000 7f8da3c00640 1 -- 172.21.15.47:0/1330829607 shutdown_connections 2024-03-21T11:44:23.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:23.892+0000 7f8da3c00640 1 -- 172.21.15.47:0/1330829607 wait complete. 2024-03-21T11:44:23.919 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:44:23.919 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:44:23.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 188324 22.82 MB/sec execute 307 sec latency 457.393 ms 2024-03-21T11:44:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.184+0000 7f9c8b400640 1 -- 172.21.15.47:0/4275410761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c841038c0 msgr2=0x7f9c84103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.184+0000 7f9c8b400640 1 --2- 172.21.15.47:0/4275410761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c841038c0 0x7f9c84103ce0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009860 tx=0x7f9c7802bb70 comp rx=0 tx=0).stop 2024-03-21T11:44:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.184+0000 7f9c8b400640 1 -- 172.21.15.47:0/4275410761 shutdown_connections 2024-03-21T11:44:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.184+0000 7f9c8b400640 1 --2- 172.21.15.47:0/4275410761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c84105470 0x7f9c8410bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.184+0000 7f9c8b400640 1 --2- 172.21.15.47:0/4275410761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c84104ab0 0x7f9c84104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.184+0000 7f9c8b400640 1 --2- 172.21.15.47:0/4275410761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c841038c0 0x7f9c84103ce0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.184+0000 7f9c8b400640 1 -- 172.21.15.47:0/4275410761 >> 172.21.15.47:0/4275410761 conn(0x7f9c840fee50 msgr2=0x7f9c84101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.184+0000 7f9c8b400640 1 -- 172.21.15.47:0/4275410761 shutdown_connections 2024-03-21T11:44:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.184+0000 7f9c8b400640 1 -- 172.21.15.47:0/4275410761 wait complete. 2024-03-21T11:44:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.185+0000 7f9c8b400640 1 Processor -- start 2024-03-21T11:44:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.185+0000 7f9c8b400640 1 -- start start 2024-03-21T11:44:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.185+0000 7f9c8b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c841038c0 0x7f9c84199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.185+0000 7f9c8b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c84104ab0 0x7f9c8419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.185+0000 7f9c8b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c84105470 0x7f9c841a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.185+0000 7f9c8b400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8410f540 con 0x7f9c841038c0 2024-03-21T11:44:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.185+0000 7f9c8b400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8410f3c0 con 0x7f9c84104ab0 2024-03-21T11:44:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.185+0000 7f9c8b400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8410f6c0 con 0x7f9c84105470 2024-03-21T11:44:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.186+0000 7f9c83e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c84104ab0 0x7f9c8419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.186+0000 7f9c89400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c84105470 0x7f9c841a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.186+0000 7f9c83e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c84104ab0 0x7f9c8419a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34822/0 (socket says 172.21.15.47:34822) 2024-03-21T11:44:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.186+0000 7f9c83e00640 1 -- 172.21.15.47:0/627666408 learned_addr learned my addr 172.21.15.47:0/627666408 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.186+0000 7f9c88a00640 1 --2- 172.21.15.47:0/627666408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c841038c0 0x7f9c84199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.186+0000 7f9c83e00640 1 -- 172.21.15.47:0/627666408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c84105470 msgr2=0x7f9c841a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.186+0000 7f9c83e00640 1 --2- 172.21.15.47:0/627666408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c84105470 0x7f9c841a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.186+0000 7f9c83e00640 1 -- 172.21.15.47:0/627666408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c841038c0 msgr2=0x7f9c84199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.186+0000 7f9c83e00640 1 --2- 172.21.15.47:0/627666408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c841038c0 0x7f9c84199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.186+0000 7f9c83e00640 1 -- 172.21.15.47:0/627666408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c841a1810 con 0x7f9c84104ab0 2024-03-21T11:44:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.186+0000 7f9c83e00640 1 --2- 172.21.15.47:0/627666408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c84104ab0 0x7f9c8419a130 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000e9f0 tx=0x7f9c7000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.186+0000 7f9c81600640 1 -- 172.21.15.47:0/627666408 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7000cd80 con 0x7f9c84104ab0 2024-03-21T11:44:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.186+0000 7f9c81600640 1 -- 172.21.15.47:0/627666408 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c70010910 con 0x7f9c84104ab0 2024-03-21T11:44:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.186+0000 7f9c81600640 1 -- 172.21.15.47:0/627666408 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c70018990 con 0x7f9c84104ab0 2024-03-21T11:44:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.187+0000 7f9c8b400640 1 -- 172.21.15.47:0/627666408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c841a1b00 con 0x7f9c84104ab0 2024-03-21T11:44:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.187+0000 7f9c8b400640 1 -- 172.21.15.47:0/627666408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c841a2000 con 0x7f9c84104ab0 2024-03-21T11:44:24.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.190+0000 7f9c8e1b2740 1 Processor -- start 2024-03-21T11:44:24.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.190+0000 7f9c81600640 1 -- 172.21.15.47:0/627666408 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9c70010ac0 con 0x7f9c84104ab0 2024-03-21T11:44:24.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.190+0000 7f9c8e1b2740 1 -- start start 2024-03-21T11:44:24.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.190+0000 7f9c8e1b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f156810 0x55b89f156c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.190+0000 7f9c8e1b2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1579e0 0x55b89f157e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.190+0000 7f9c8e1b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f1583a0 0x55b89f15ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.190+0000 7f9c8e1b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89f0bb2c0 con 0x55b89f156810 2024-03-21T11:44:24.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.190+0000 7f9c8e1b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89f063c90 con 0x55b89f1583a0 2024-03-21T11:44:24.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.190+0000 7f9c8e1b2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89f0b7360 con 0x55b89f1579e0 2024-03-21T11:44:24.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.191+0000 7f9c88a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1579e0 0x55b89f157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.191+0000 7f9c88a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1579e0 0x55b89f157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44794/0 (socket says 172.21.15.47:44794) 2024-03-21T11:44:24.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.191+0000 7f9c88a00640 1 -- 172.21.15.47:0/985810446 learned_addr learned my addr 172.21.15.47:0/985810446 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:24.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.191+0000 7f9c89400640 1 --2- 172.21.15.47:0/985810446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f156810 0x55b89f156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.191+0000 7f9c83e00640 1 --2- 172.21.15.47:0/985810446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f1583a0 0x55b89f15ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.191+0000 7f9c88a00640 1 -- 172.21.15.47:0/985810446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f1583a0 msgr2=0x55b89f15ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.191+0000 7f9c88a00640 1 --2- 172.21.15.47:0/985810446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f1583a0 0x55b89f15ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.191+0000 7f9c88a00640 1 -- 172.21.15.47:0/985810446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f156810 msgr2=0x55b89f156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.191+0000 7f9c88a00640 1 --2- 172.21.15.47:0/985810446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f156810 0x55b89f156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.191+0000 7f9c88a00640 1 -- 172.21.15.47:0/985810446 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b89f088080 con 0x55b89f1579e0 2024-03-21T11:44:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.191+0000 7f9c88a00640 1 --2- 172.21.15.47:0/985810446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1579e0 0x55b89f157e60 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f9c780057c0 tx=0x7f9c78005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43c5402b06dbd98b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.192+0000 7f9c75c00640 1 -- 172.21.15.47:0/985810446 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78039040 con 0x55b89f1579e0 2024-03-21T11:44:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.192+0000 7f9c75c00640 1 -- 172.21.15.47:0/985810446 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c78034db0 con 0x55b89f1579e0 2024-03-21T11:44:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.192+0000 7f9c75c00640 1 -- 172.21.15.47:0/985810446 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7802c870 con 0x55b89f1579e0 2024-03-21T11:44:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.192+0000 7f9c81600640 1 --2- 172.21.15.47:0/627666408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9c6c0a2f20 0x7f9c6c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.193+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/985810446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1579e0 msgr2=0x55b89f157e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.193+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/985810446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1579e0 0x55b89f157e60 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f9c780057c0 tx=0x7f9c78005b20 comp rx=0 tx=0).stop 2024-03-21T11:44:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.193+0000 7f9c81600640 1 -- 172.21.15.47:0/627666408 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9c700cfad0 con 0x7f9c84104ab0 2024-03-21T11:44:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.193+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/985810446 shutdown_connections 2024-03-21T11:44:24.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.193+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/985810446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f1583a0 0x55b89f15ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.193+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/985810446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1579e0 0x55b89f157e60 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.193+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/985810446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f156810 0x55b89f156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.193+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/985810446 >> 172.21.15.47:0/985810446 conn(0x55b89f151c00 msgr2=0x55b89f152000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:24.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.193+0000 7f9c89400640 1 --2- 172.21.15.47:0/627666408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9c6c0a2f20 0x7f9c6c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.193+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/985810446 shutdown_connections 2024-03-21T11:44:24.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.193+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/985810446 wait complete. 2024-03-21T11:44:24.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.193+0000 7f9c8e1b2740 1 -- start start 2024-03-21T11:44:24.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.194+0000 7f9c8e1b2740 1 Processor -- start 2024-03-21T11:44:24.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.194+0000 7f9c89400640 1 --2- 172.21.15.47:0/627666408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9c6c0a2f20 0x7f9c6c0a53e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x55b89f1578e0 tx=0x7f9c7c010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.194+0000 7f9c8e1b2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f156810 0x55b89f1691d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.194+0000 7f9c8e1b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f1579e0 0x55b89f169710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.195+0000 7f9c8e1b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1583a0 0x55b89f170750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.195+0000 7f9c88a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f156810 0x55b89f1691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.195+0000 7f9c88a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f156810 0x55b89f1691d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44804/0 (socket says 172.21.15.47:44804) 2024-03-21T11:44:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.195+0000 7f9c8e1b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89f15f350 con 0x55b89f1583a0 2024-03-21T11:44:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.195+0000 7f9c88a00640 1 -- 172.21.15.47:0/299418216 learned_addr learned my addr 172.21.15.47:0/299418216 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.195+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/299418216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89f0770d0 con 0x55b89f1579e0 2024-03-21T11:44:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.195+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/299418216 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89f15f4d0 con 0x55b89f156810 2024-03-21T11:44:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.195+0000 7f9c89400640 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1583a0 0x55b89f170750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.195+0000 7f9c83e00640 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f1579e0 0x55b89f169710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.195+0000 7f9c8e1b2740 10 client.0 _ll_register_callbacks cb 0x55b89f07be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.196+0000 7f9c88a00640 1 -- 172.21.15.47:0/299418216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f1579e0 msgr2=0x55b89f169710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.196+0000 7f9c88a00640 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f1579e0 0x55b89f169710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.196+0000 7f9c88a00640 1 -- 172.21.15.47:0/299418216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1583a0 msgr2=0x55b89f170750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.196+0000 7f9c88a00640 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1583a0 0x55b89f170750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.196+0000 7f9c88a00640 1 -- 172.21.15.47:0/299418216 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b89edcf3f0 con 0x55b89f156810 2024-03-21T11:44:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.196+0000 7f9c83e00640 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f1579e0 0x55b89f169710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.196+0000 7f9c89400640 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1583a0 0x55b89f170750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.196+0000 7f9c88a00640 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f156810 0x55b89f1691d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f9c78037040 tx=0x7f9c78005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e13a0ceccf687b06 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.196+0000 7f9c6a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:24.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.199+0000 7f9c6a000640 1 -- 172.21.15.47:0/299418216 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78039040 con 0x55b89f156810 2024-03-21T11:44:24.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.199+0000 7f9c6a000640 1 -- 172.21.15.47:0/299418216 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9c78005050 con 0x55b89f156810 2024-03-21T11:44:24.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.201+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/299418216 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c78034db0 con 0x55b89f156810 2024-03-21T11:44:24.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.201+0000 7f9c6a000640 1 -- 172.21.15.47:0/299418216 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c78005250 con 0x55b89f156810 2024-03-21T11:44:24.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.201+0000 7f9c8e1b2740 10 client.32263 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:24.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.201+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/299418216 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b89ee997d0 con 0x55b89f156810 2024-03-21T11:44:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.202+0000 7f9c6a000640 1 -- 172.21.15.47:0/299418216 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c78034db0 con 0x55b89f156810 2024-03-21T11:44:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.203+0000 7f9c6a000640 1 -- 172.21.15.47:0/299418216 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9c78004960 con 0x55b89f156810 2024-03-21T11:44:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.203+0000 7f9c8e1b2740 10 client.32263 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.203+0000 7f9c8e1b2740 10 client.32263 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:44:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.204+0000 7f9c8e1b2740 20 client.32263 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.204+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b89f174890 0x55b89f176c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.204+0000 7f9c8e1b2740 4 client.32263 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:24.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.204+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/299418216 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b89f0c8800 con 0x55b89f174890 2024-03-21T11:44:24.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.205+0000 7f9c83e00640 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b89f174890 0x55b89f176c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.206+0000 7f9c83e00640 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b89f174890 0x55b89f176c80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ab2501db68c48c6 server_cookie=c1642d292a145fb6 in_seq=0 out_seq=0 2024-03-21T11:44:24.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.206+0000 7f9c6a000640 10 client.32263 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:44:24.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.207+0000 7f9c6a000640 1 -- 172.21.15.47:0/299418216 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b89f0c8800 con 0x55b89f174890 2024-03-21T11:44:24.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.207+0000 7f9c6a000640 10 client.32263 handle_command_reply: tid=0 2024-03-21T11:44:24.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.207+0000 7f9c8e1b2740 1 client.32263 shutdown 2024-03-21T11:44:24.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.208+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/299418216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f156810 msgr2=0x55b89f1691d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.208+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f156810 0x55b89f1691d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f9c78037040 tx=0x7f9c78005fd0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.208+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/299418216 shutdown_connections 2024-03-21T11:44:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.208+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b89f174890 0x55b89f176c80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.208+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1583a0 0x55b89f170750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.208+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f1579e0 0x55b89f169710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.208+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/299418216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f156810 0x55b89f1691d0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.208+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/299418216 >> 172.21.15.47:0/299418216 conn(0x55b89f151c00 msgr2=0x55b89f150eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.208+0000 7f9c6a000640 0 client.32263 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:44:24.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.208+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/299418216 shutdown_connections 2024-03-21T11:44:24.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.208+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/299418216 wait complete. 2024-03-21T11:44:24.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.209+0000 7f9c8e1b2740 20 client.32263 trim_cache size 0 max 16384 2024-03-21T11:44:24.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.223+0000 7f9c8e1b2740 1 Processor -- start 2024-03-21T11:44:24.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.223+0000 7f9c8e1b2740 1 -- start start 2024-03-21T11:44:24.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.224+0000 7f9c8e1b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f156810 0x55b89f15a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.224+0000 7f9c8e1b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1579e0 0x55b89f170670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.224+0000 7f9c8e1b2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1583a0 0x55b89f15af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.224+0000 7f9c8e1b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c841a8010 con 0x55b89f1579e0 2024-03-21T11:44:24.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.224+0000 7f9c88a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f156810 0x55b89f15a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.224+0000 7f9c8e1b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c841a9990 con 0x55b89f156810 2024-03-21T11:44:24.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.224+0000 7f9c8e1b2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c841a8670 con 0x55b89f1583a0 2024-03-21T11:44:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.224+0000 7f9c88a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f156810 0x55b89f15a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34850/0 (socket says 172.21.15.47:34850) 2024-03-21T11:44:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.224+0000 7f9c88a00640 1 -- 172.21.15.47:0/1752640294 learned_addr learned my addr 172.21.15.47:0/1752640294 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.224+0000 7f9c89400640 1 --2- 172.21.15.47:0/1752640294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1583a0 0x55b89f15af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.224+0000 7f9c83e00640 1 --2- 172.21.15.47:0/1752640294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1579e0 0x55b89f170670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.225+0000 7f9c88a00640 1 -- 172.21.15.47:0/1752640294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1583a0 msgr2=0x55b89f15af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.225+0000 7f9c88a00640 1 --2- 172.21.15.47:0/1752640294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1583a0 0x55b89f15af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.225+0000 7f9c88a00640 1 -- 172.21.15.47:0/1752640294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1579e0 msgr2=0x55b89f170670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.225+0000 7f9c88a00640 1 --2- 172.21.15.47:0/1752640294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1579e0 0x55b89f170670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.225+0000 7f9c88a00640 1 -- 172.21.15.47:0/1752640294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b89f14c2c0 con 0x55b89f156810 2024-03-21T11:44:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.225+0000 7f9c89400640 1 --2- 172.21.15.47:0/1752640294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1583a0 0x55b89f15af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:44:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.225+0000 7f9c83e00640 1 --2- 172.21.15.47:0/1752640294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1579e0 0x55b89f170670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:44:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.225+0000 7f9c88a00640 1 --2- 172.21.15.47:0/1752640294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f156810 0x55b89f15a9e0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f9c78034fd0 tx=0x7f9c78005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2363ec665cdbb27 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.225+0000 7f9c75200640 1 -- 172.21.15.47:0/1752640294 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78060070 con 0x55b89f156810 2024-03-21T11:44:24.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.225+0000 7f9c75200640 1 -- 172.21.15.47:0/1752640294 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c78009450 con 0x55b89f156810 2024-03-21T11:44:24.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.225+0000 7f9c75200640 1 -- 172.21.15.47:0/1752640294 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78039070 con 0x55b89f156810 2024-03-21T11:44:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.226+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/1752640294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f156810 msgr2=0x55b89f15a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.226+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/1752640294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f156810 0x55b89f15a9e0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f9c78034fd0 tx=0x7f9c78005850 comp rx=0 tx=0).stop 2024-03-21T11:44:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.226+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/1752640294 shutdown_connections 2024-03-21T11:44:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.226+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/1752640294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1583a0 0x55b89f15af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.226+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/1752640294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1579e0 0x55b89f170670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.226+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/1752640294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f156810 0x55b89f15a9e0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.226+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/1752640294 >> 172.21.15.47:0/1752640294 conn(0x55b89f151c00 msgr2=0x55b89f1556f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.226+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/1752640294 shutdown_connections 2024-03-21T11:44:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.226+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/1752640294 wait complete. 2024-03-21T11:44:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.226+0000 7f9c8e1b2740 1 -- start start 2024-03-21T11:44:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.227+0000 7f9c8e1b2740 1 Processor -- start 2024-03-21T11:44:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.227+0000 7f9c8e1b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1579e0 0x55b89f1700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.227+0000 7f9c8e1b2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1583a0 0x55b89f1705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.227+0000 7f9c8e1b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f16cbe0 0x55b89f16d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.227+0000 7f9c8e1b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c841a9330 con 0x55b89f1579e0 2024-03-21T11:44:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.227+0000 7f9c8e1b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c841a8cd0 con 0x55b89f16cbe0 2024-03-21T11:44:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.227+0000 7f9c8e1b2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c841a61d0 con 0x55b89f1583a0 2024-03-21T11:44:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.227+0000 7f9c8e1b2740 10 client.0 _ll_register_callbacks cb 0x7f9c7c009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.227+0000 7f9c88a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1579e0 0x55b89f1700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.227+0000 7f9c83e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1583a0 0x55b89f1705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.227+0000 7f9c88a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1579e0 0x55b89f1700a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40180/0 (socket says 172.21.15.47:40180) 2024-03-21T11:44:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.227+0000 7f9c88a00640 1 -- 172.21.15.47:0/1613173504 learned_addr learned my addr 172.21.15.47:0/1613173504 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.227+0000 7f9c89400640 1 --2- 172.21.15.47:0/1613173504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f16cbe0 0x55b89f16d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.228+0000 7f9c88a00640 1 -- 172.21.15.47:0/1613173504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1583a0 msgr2=0x55b89f1705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.228+0000 7f9c88a00640 1 --2- 172.21.15.47:0/1613173504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1583a0 0x55b89f1705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.228+0000 7f9c88a00640 1 -- 172.21.15.47:0/1613173504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f16cbe0 msgr2=0x55b89f16d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.228+0000 7f9c88a00640 1 --2- 172.21.15.47:0/1613173504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f16cbe0 0x55b89f16d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.228+0000 7f9c88a00640 1 -- 172.21.15.47:0/1613173504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b89f16d6d0 con 0x55b89f1579e0 2024-03-21T11:44:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.228+0000 7f9c83e00640 1 --2- 172.21.15.47:0/1613173504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1583a0 0x55b89f1705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.228+0000 7f9c88a00640 1 --2- 172.21.15.47:0/1613173504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1579e0 0x55b89f1700a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9c78042040 tx=0x7f9c780058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af9101d3f402b89 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.228+0000 7f9c6a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:44:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.228+0000 7f9c6a000640 1 -- 172.21.15.47:0/1613173504 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78060020 con 0x55b89f1579e0 2024-03-21T11:44:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.228+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/1613173504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c78009450 con 0x55b89f1579e0 2024-03-21T11:44:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.228+0000 7f9c6a000640 1 -- 172.21.15.47:0/1613173504 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9c780046c0 con 0x55b89f1579e0 2024-03-21T11:44:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.229+0000 7f9c6a000640 1 -- 172.21.15.47:0/1613173504 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c78039070 con 0x55b89f1579e0 2024-03-21T11:44:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.229+0000 7f9c8e1b2740 10 client.21627 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.229+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/1613173504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b89f1d1070 con 0x55b89f1579e0 2024-03-21T11:44:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.229+0000 7f9c6a000640 1 -- 172.21.15.47:0/1613173504 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c78009450 con 0x55b89f1579e0 2024-03-21T11:44:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.229+0000 7f9c6a000640 1 -- 172.21.15.47:0/1613173504 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9c7802c630 con 0x55b89f1579e0 2024-03-21T11:44:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.229+0000 7f9c8e1b2740 10 client.21627 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:24.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.229+0000 7f9c8e1b2740 10 client.21627 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:44:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.229+0000 7f9c8e1b2740 20 client.21627 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.229+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/1613173504 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b89f1d1780 0x55b89f1d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.229+0000 7f9c8e1b2740 4 client.21627 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:44:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.229+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/1613173504 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b89f0c2b50 con 0x55b89f1d1780 2024-03-21T11:44:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.230+0000 7f9c83e00640 1 --2- 172.21.15.47:0/1613173504 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b89f1d1780 0x55b89f1d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.230+0000 7f9c83e00640 1 --2- 172.21.15.47:0/1613173504 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b89f1d1780 0x55b89f1d3b70 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=387540acd1fdf638 server_cookie=960128f3744cd64c in_seq=0 out_seq=0 2024-03-21T11:44:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.230+0000 7f9c6a000640 10 client.21627 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:44:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.230+0000 7f9c6a000640 1 -- 172.21.15.47:0/1613173504 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b89f0c2b50 con 0x55b89f1d1780 2024-03-21T11:44:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.230+0000 7f9c6a000640 10 client.21627 handle_command_reply: tid=0 2024-03-21T11:44:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.230+0000 7f9c8e1b2740 1 client.21627 shutdown 2024-03-21T11:44:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.230+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/1613173504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1579e0 msgr2=0x55b89f1700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.230+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/1613173504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1579e0 0x55b89f1700a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9c78042040 tx=0x7f9c780058f0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.230+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/1613173504 shutdown_connections 2024-03-21T11:44:24.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.230+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/1613173504 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55b89f1d1780 0x55b89f1d3b70 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/1613173504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89f16cbe0 0x55b89f16d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/1613173504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b89f1583a0 0x55b89f1705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8e1b2740 1 --2- 172.21.15.47:0/1613173504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89f1579e0 0x55b89f1700a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c6a000640 0 client.21627 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:44:24.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/1613173504 >> 172.21.15.47:0/1613173504 conn(0x55b89f151c00 msgr2=0x55b89f150620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:24.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/1613173504 shutdown_connections 2024-03-21T11:44:24.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8e1b2740 1 -- 172.21.15.47:0/1613173504 wait complete. 2024-03-21T11:44:24.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8e1b2740 20 client.21627 trim_cache size 0 max 16384 2024-03-21T11:44:24.236 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:44:24.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8b400640 1 -- 172.21.15.47:0/627666408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9c6c0a2f20 msgr2=0x7f9c6c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8b400640 1 --2- 172.21.15.47:0/627666408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9c6c0a2f20 0x7f9c6c0a53e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x55b89f1578e0 tx=0x7f9c7c010040 comp rx=0 tx=0).stop 2024-03-21T11:44:24.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8b400640 1 -- 172.21.15.47:0/627666408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c84104ab0 msgr2=0x7f9c8419a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8b400640 1 --2- 172.21.15.47:0/627666408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c84104ab0 0x7f9c8419a130 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000e9f0 tx=0x7f9c7000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8b400640 1 -- 172.21.15.47:0/627666408 shutdown_connections 2024-03-21T11:44:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8b400640 1 --2- 172.21.15.47:0/627666408 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9c6c0a2f20 0x7f9c6c0a53e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8b400640 1 --2- 172.21.15.47:0/627666408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c84105470 0x7f9c841a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8b400640 1 --2- 172.21.15.47:0/627666408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c84104ab0 0x7f9c8419a130 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8b400640 1 --2- 172.21.15.47:0/627666408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c841038c0 0x7f9c84199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.231+0000 7f9c8b400640 1 -- 172.21.15.47:0/627666408 >> 172.21.15.47:0/627666408 conn(0x7f9c840fee50 msgr2=0x7f9c84107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.232+0000 7f9c8b400640 1 -- 172.21.15.47:0/627666408 shutdown_connections 2024-03-21T11:44:24.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.232+0000 7f9c8b400640 1 -- 172.21.15.47:0/627666408 wait complete. 2024-03-21T11:44:24.249 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:44:24.249 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:44:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.532+0000 7ff7be200640 1 -- 172.21.15.47:0/1262634303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b8109450 msgr2=0x7ff7b810b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.532+0000 7ff7be200640 1 --2- 172.21.15.47:0/1262634303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b8109450 0x7ff7b810b8e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009900 tx=0x7ff7a802bcc0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.532+0000 7ff7be200640 1 -- 172.21.15.47:0/1262634303 shutdown_connections 2024-03-21T11:44:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.533+0000 7ff7be200640 1 --2- 172.21.15.47:0/1262634303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b8109450 0x7ff7b810b8e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.533+0000 7ff7be200640 1 --2- 172.21.15.47:0/1262634303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7b8074080 0x7ff7b80744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.533+0000 7ff7be200640 1 --2- 172.21.15.47:0/1262634303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b8075660 0x7ff7b8073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.533+0000 7ff7be200640 1 -- 172.21.15.47:0/1262634303 >> 172.21.15.47:0/1262634303 conn(0x7ff7b80fcb10 msgr2=0x7ff7b80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.533+0000 7ff7be200640 1 -- 172.21.15.47:0/1262634303 shutdown_connections 2024-03-21T11:44:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.533+0000 7ff7be200640 1 -- 172.21.15.47:0/1262634303 wait complete. 2024-03-21T11:44:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.535+0000 7ff7be200640 1 Processor -- start 2024-03-21T11:44:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.536+0000 7ff7be200640 1 -- start start 2024-03-21T11:44:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.537+0000 7ff7be200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7b8074080 0x7ff7b8195600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.537+0000 7ff7be200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b8075660 0x7ff7b8195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.537+0000 7ff7be200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b8109450 0x7ff7b819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.538+0000 7ff7b7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b8075660 0x7ff7b8195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.538+0000 7ff7b7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b8075660 0x7ff7b8195b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40206/0 (socket says 172.21.15.47:40206) 2024-03-21T11:44:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.538+0000 7ff7b7e00640 1 -- 172.21.15.47:0/1592958600 learned_addr learned my addr 172.21.15.47:0/1592958600 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.538+0000 7ff7bd800640 1 --2- 172.21.15.47:0/1592958600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b8109450 0x7ff7b819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.538+0000 7ff7bce00640 1 --2- 172.21.15.47:0/1592958600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7b8074080 0x7ff7b8195600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.539+0000 7ff7be200640 1 -- 172.21.15.47:0/1592958600 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b810f460 con 0x7ff7b8075660 2024-03-21T11:44:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.539+0000 7ff7be200640 1 -- 172.21.15.47:0/1592958600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b810f2e0 con 0x7ff7b8109450 2024-03-21T11:44:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.539+0000 7ff7be200640 1 -- 172.21.15.47:0/1592958600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b810f5e0 con 0x7ff7b8074080 2024-03-21T11:44:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.540+0000 7ff7bd800640 1 -- 172.21.15.47:0/1592958600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7b8074080 msgr2=0x7ff7b8195600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.540+0000 7ff7bd800640 1 --2- 172.21.15.47:0/1592958600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7b8074080 0x7ff7b8195600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.540+0000 7ff7bd800640 1 -- 172.21.15.47:0/1592958600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b8075660 msgr2=0x7ff7b8195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.540+0000 7ff7bd800640 1 --2- 172.21.15.47:0/1592958600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b8075660 0x7ff7b8195b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.540+0000 7ff7bd800640 1 -- 172.21.15.47:0/1592958600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b81ae360 con 0x7ff7b8109450 2024-03-21T11:44:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.540+0000 7ff7bd800640 1 --2- 172.21.15.47:0/1592958600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b8109450 0x7ff7b819cbb0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7ff7a802c200 tx=0x7ff7a8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.541+0000 7ff7b5600640 1 -- 172.21.15.47:0/1592958600 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8043020 con 0x7ff7b8109450 2024-03-21T11:44:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.541+0000 7ff7b5600640 1 -- 172.21.15.47:0/1592958600 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7a80054e0 con 0x7ff7b8109450 2024-03-21T11:44:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.541+0000 7ff7b5600640 1 -- 172.21.15.47:0/1592958600 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8003ca0 con 0x7ff7b8109450 2024-03-21T11:44:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.541+0000 7ff7be200640 1 -- 172.21.15.47:0/1592958600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b81ae530 con 0x7ff7b8109450 2024-03-21T11:44:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.541+0000 7ff7be200640 1 -- 172.21.15.47:0/1592958600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b81ae9a0 con 0x7ff7b8109450 2024-03-21T11:44:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.548+0000 7ff7c2578740 1 Processor -- start 2024-03-21T11:44:24.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.554+0000 7ff7c2578740 1 -- start start 2024-03-21T11:44:24.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.554+0000 7ff7b5600640 1 -- 172.21.15.47:0/1592958600 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff7a8046020 con 0x7ff7b8109450 2024-03-21T11:44:24.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.554+0000 7ff7c2578740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866c9f0 0x55eed866ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.554+0000 7ff7c2578740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed866dbc0 0x55eed866e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.554+0000 7ff7c2578740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed8674e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.554+0000 7ff7c2578740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed85d12c0 con 0x55eed866dbc0 2024-03-21T11:44:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.554+0000 7ff7c2578740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8579c90 con 0x55eed866c9f0 2024-03-21T11:44:24.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.554+0000 7ff7c2578740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed85cd360 con 0x55eed866e580 2024-03-21T11:44:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.555+0000 7ff7b7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866c9f0 0x55eed866ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.555+0000 7ff7b7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866c9f0 0x55eed866ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34872/0 (socket says 172.21.15.47:34872) 2024-03-21T11:44:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.555+0000 7ff7b7e00640 1 -- 172.21.15.47:0/2414026501 learned_addr learned my addr 172.21.15.47:0/2414026501 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.555+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/2414026501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed8674e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.556+0000 7ff7b5600640 1 --2- 172.21.15.47:0/1592958600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff79c0a30f0 0x7ff79c0a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.556+0000 7ff7b5600640 1 -- 172.21.15.47:0/1592958600 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff7a80f17b0 con 0x7ff7b8109450 2024-03-21T11:44:24.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.557+0000 7ff7b7e00640 1 -- 172.21.15.47:0/2414026501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866c9f0 msgr2=0x55eed866ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.557+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/2414026501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866c9f0 0x55eed866ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.557+0000 7ff7b7e00640 1 -- 172.21.15.47:0/2414026501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed866dbc0 msgr2=0x55eed866e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.557+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/2414026501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed866dbc0 0x55eed866e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.557+0000 7ff7b7e00640 1 -- 172.21.15.47:0/2414026501 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eed859e080 con 0x55eed866e580 2024-03-21T11:44:24.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.557+0000 7ff7bd800640 1 --2- 172.21.15.47:0/1592958600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff79c0a30f0 0x7ff79c0a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.558+0000 7ff7bd800640 1 --2- 172.21.15.47:0/1592958600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff79c0a30f0 0x7ff79c0a55b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80bd910 tx=0x7ff7a8004ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:24.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.559+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/2414026501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed8674e10 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4005fa0 tx=0x7ff7a400b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ff95563b4c73a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:24.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.559+0000 7ff7ae000640 1 -- 172.21.15.47:0/2414026501 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4004030 con 0x55eed866e580 2024-03-21T11:44:24.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.559+0000 7ff7ae000640 1 -- 172.21.15.47:0/2414026501 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7a4012520 con 0x55eed866e580 2024-03-21T11:44:24.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.559+0000 7ff7ae000640 1 -- 172.21.15.47:0/2414026501 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40049c0 con 0x55eed866e580 2024-03-21T11:44:24.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.560+0000 7ff7c2578740 1 -- 172.21.15.47:0/2414026501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 msgr2=0x55eed8674e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.560+0000 7ff7c2578740 1 --2- 172.21.15.47:0/2414026501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed8674e10 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4005fa0 tx=0x7ff7a400b480 comp rx=0 tx=0).stop 2024-03-21T11:44:24.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.561+0000 7ff7c2578740 1 -- 172.21.15.47:0/2414026501 shutdown_connections 2024-03-21T11:44:24.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.561+0000 7ff7c2578740 1 --2- 172.21.15.47:0/2414026501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed8674e10 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.561+0000 7ff7c2578740 1 --2- 172.21.15.47:0/2414026501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed866dbc0 0x55eed866e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.561+0000 7ff7c2578740 1 --2- 172.21.15.47:0/2414026501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866c9f0 0x55eed866ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.561+0000 7ff7c2578740 1 -- 172.21.15.47:0/2414026501 >> 172.21.15.47:0/2414026501 conn(0x55eed8667de0 msgr2=0x55eed86681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:24.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.561+0000 7ff7c2578740 1 -- 172.21.15.47:0/2414026501 shutdown_connections 2024-03-21T11:44:24.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.561+0000 7ff7c2578740 1 -- 172.21.15.47:0/2414026501 wait complete. 2024-03-21T11:44:24.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.562+0000 7ff7c2578740 1 -- start start 2024-03-21T11:44:24.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.563+0000 7ff7c2578740 1 Processor -- start 2024-03-21T11:44:24.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.564+0000 7ff7c2578740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866c9f0 0x55eed8676d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.565+0000 7ff7c2578740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed866dbc0 0x55eed8677280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.565+0000 7ff7c2578740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866e580 0x55eed867e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.565+0000 7ff7b7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866c9f0 0x55eed8676d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.565+0000 7ff7b7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866c9f0 0x55eed8676d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44858/0 (socket says 172.21.15.47:44858) 2024-03-21T11:44:24.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.565+0000 7ff7c2578740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8675590 con 0x55eed866dbc0 2024-03-21T11:44:24.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.565+0000 7ff7c2578740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed858d0d0 con 0x55eed866e580 2024-03-21T11:44:24.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.565+0000 7ff7c2578740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed8675710 con 0x55eed866c9f0 2024-03-21T11:44:24.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.565+0000 7ff7c2578740 10 client.0 _ll_register_callbacks cb 0x55eed8591e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:24.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.565+0000 7ff7bce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed866dbc0 0x55eed8677280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.565+0000 7ff7b7e00640 1 -- 172.21.15.47:0/3884019969 learned_addr learned my addr 172.21.15.47:0/3884019969 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.566+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/3884019969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866e580 0x55eed867e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.566+0000 7ff7b7e00640 1 -- 172.21.15.47:0/3884019969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866e580 msgr2=0x55eed867e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.566+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/3884019969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866e580 0x55eed867e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.566+0000 7ff7b7e00640 1 -- 172.21.15.47:0/3884019969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed866dbc0 msgr2=0x55eed8677280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.566+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/3884019969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed866dbc0 0x55eed8677280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.566+0000 7ff7b7e00640 1 -- 172.21.15.47:0/3884019969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eed82e53f0 con 0x55eed866c9f0 2024-03-21T11:44:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.566+0000 7ff7bce00640 1 --2- 172.21.15.47:0/3884019969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed866dbc0 0x55eed8677280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.567+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/3884019969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866c9f0 0x55eed8676d40 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4006010 tx=0x7ff7a400be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43f781c2ccf35609 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.567+0000 7ff7a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.570+0000 7ff7a2a00640 1 -- 172.21.15.47:0/3884019969 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4009720 con 0x55eed866c9f0 2024-03-21T11:44:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.570+0000 7ff7a2a00640 1 -- 172.21.15.47:0/3884019969 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff7a401a040 con 0x55eed866c9f0 2024-03-21T11:44:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.573+0000 7ff7c2578740 1 -- 172.21.15.47:0/3884019969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7a4012520 con 0x55eed866c9f0 2024-03-21T11:44:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.574+0000 7ff7a2a00640 1 -- 172.21.15.47:0/3884019969 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7a40098c0 con 0x55eed866c9f0 2024-03-21T11:44:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.575+0000 7ff7c2578740 10 client.32281 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.575+0000 7ff7c2578740 1 -- 172.21.15.47:0/3884019969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55eed83af7d0 con 0x55eed866c9f0 2024-03-21T11:44:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.576+0000 7ff7a2a00640 1 -- 172.21.15.47:0/3884019969 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7a4012520 con 0x55eed866c9f0 2024-03-21T11:44:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.576+0000 7ff7a2a00640 1 -- 172.21.15.47:0/3884019969 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff7a4020440 con 0x55eed866c9f0 2024-03-21T11:44:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.576+0000 7ff7c2578740 10 client.32281 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.576+0000 7ff7c2578740 10 client.32281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.576+0000 7ff7c2578740 20 client.32281 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.576+0000 7ff7c2578740 1 --2- 172.21.15.47:0/3884019969 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55eed86823b0 0x55eed86847a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.576+0000 7ff7c2578740 4 client.32281 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.576+0000 7ff7c2578740 1 -- 172.21.15.47:0/3884019969 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eed85de800 con 0x55eed86823b0 2024-03-21T11:44:24.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.580+0000 7ff7bce00640 1 --2- 172.21.15.47:0/3884019969 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55eed86823b0 0x55eed86847a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.583+0000 7ff7a2a00640 10 client.32281 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:24.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.583+0000 7ff7bce00640 1 --2- 172.21.15.47:0/3884019969 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55eed86823b0 0x55eed86847a0 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e575f397afc53e5 server_cookie=48fde69d8668937f in_seq=0 out_seq=0 2024-03-21T11:44:24.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.589+0000 7ff7a2a00640 1 -- 172.21.15.47:0/3884019969 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55eed85de800 con 0x55eed86823b0 2024-03-21T11:44:24.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.589+0000 7ff7a2a00640 10 client.32281 handle_command_reply: tid=0 2024-03-21T11:44:24.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.589+0000 7ff7c2578740 1 client.32281 shutdown 2024-03-21T11:44:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.590+0000 7ff7c2578740 1 -- 172.21.15.47:0/3884019969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866c9f0 msgr2=0x55eed8676d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.590+0000 7ff7c2578740 1 --2- 172.21.15.47:0/3884019969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866c9f0 0x55eed8676d40 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4006010 tx=0x7ff7a400be70 comp rx=0 tx=0).stop 2024-03-21T11:44:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.590+0000 7ff7c2578740 1 -- 172.21.15.47:0/3884019969 shutdown_connections 2024-03-21T11:44:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.590+0000 7ff7c2578740 1 --2- 172.21.15.47:0/3884019969 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55eed86823b0 0x55eed86847a0 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.590+0000 7ff7c2578740 1 --2- 172.21.15.47:0/3884019969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866e580 0x55eed867e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.590+0000 7ff7c2578740 1 --2- 172.21.15.47:0/3884019969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed866dbc0 0x55eed8677280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.590+0000 7ff7c2578740 1 --2- 172.21.15.47:0/3884019969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866c9f0 0x55eed8676d40 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.590+0000 7ff7c2578740 1 -- 172.21.15.47:0/3884019969 >> 172.21.15.47:0/3884019969 conn(0x55eed8667de0 msgr2=0x55eed8667160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:24.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.590+0000 7ff7a2a00640 0 client.32281 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:24.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.590+0000 7ff7c2578740 1 -- 172.21.15.47:0/3884019969 shutdown_connections 2024-03-21T11:44:24.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.590+0000 7ff7c2578740 1 -- 172.21.15.47:0/3884019969 wait complete. 2024-03-21T11:44:24.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.591+0000 7ff7c2578740 20 client.32281 trim_cache size 0 max 16384 2024-03-21T11:44:24.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.618+0000 7ff7c2578740 1 Processor -- start 2024-03-21T11:44:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.618+0000 7ff7c2578740 1 -- start start 2024-03-21T11:44:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.619+0000 7ff7c2578740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866c9f0 0x55eed8688f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.619+0000 7ff7c2578740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed866dbc0 0x55eed867e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.619+0000 7ff7b7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866c9f0 0x55eed8688f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.619+0000 7ff7b7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866c9f0 0x55eed8688f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34880/0 (socket says 172.21.15.47:34880) 2024-03-21T11:44:24.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.620+0000 7ff7b7e00640 1 -- 172.21.15.47:0/1667584252 learned_addr learned my addr 172.21.15.47:0/1667584252 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:24.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.620+0000 7ff7c2578740 1 --2- 172.21.15.47:0/1667584252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed86894d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.620+0000 7ff7c2578740 1 -- 172.21.15.47:0/1667584252 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81b8d20 con 0x55eed866dbc0 2024-03-21T11:44:24.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.620+0000 7ff7c2578740 1 -- 172.21.15.47:0/1667584252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81ba6a0 con 0x55eed866c9f0 2024-03-21T11:44:24.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.620+0000 7ff7c2578740 1 -- 172.21.15.47:0/1667584252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81b9380 con 0x55eed866e580 2024-03-21T11:44:24.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.620+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/1667584252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed86894d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.620+0000 7ff7b7e00640 1 -- 172.21.15.47:0/1667584252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 msgr2=0x55eed86894d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.620+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/1667584252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed86894d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.621+0000 7ff7b7e00640 1 -- 172.21.15.47:0/1667584252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed866dbc0 msgr2=0x55eed867e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.621+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/1667584252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed866dbc0 0x55eed867e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.621+0000 7ff7b7e00640 1 -- 172.21.15.47:0/1667584252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eed8684b10 con 0x55eed866c9f0 2024-03-21T11:44:24.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.621+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/1667584252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866c9f0 0x55eed8688f90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4004620 tx=0x7ff7a4014fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b55b7ea73e451a23 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:24.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.622+0000 7ff7ad600640 1 -- 172.21.15.47:0/1667584252 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4016070 con 0x55eed866c9f0 2024-03-21T11:44:24.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.623+0000 7ff7ad600640 1 -- 172.21.15.47:0/1667584252 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7a400bcc0 con 0x55eed866c9f0 2024-03-21T11:44:24.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.623+0000 7ff7ad600640 1 -- 172.21.15.47:0/1667584252 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4020440 con 0x55eed866c9f0 2024-03-21T11:44:24.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.624+0000 7ff7c2578740 1 -- 172.21.15.47:0/1667584252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866c9f0 msgr2=0x55eed8688f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.624+0000 7ff7c2578740 1 --2- 172.21.15.47:0/1667584252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866c9f0 0x55eed8688f90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4004620 tx=0x7ff7a4014fa0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.624+0000 7ff7c2578740 1 -- 172.21.15.47:0/1667584252 shutdown_connections 2024-03-21T11:44:24.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.625+0000 7ff7c2578740 1 --2- 172.21.15.47:0/1667584252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed86894d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.625+0000 7ff7c2578740 1 --2- 172.21.15.47:0/1667584252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed866dbc0 0x55eed867e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.625+0000 7ff7c2578740 1 --2- 172.21.15.47:0/1667584252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866c9f0 0x55eed8688f90 secure :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4004620 tx=0x7ff7a4014fa0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.625+0000 7ff7c2578740 1 -- 172.21.15.47:0/1667584252 >> 172.21.15.47:0/1667584252 conn(0x55eed8667de0 msgr2=0x55eed866b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:24.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.625+0000 7ff7c2578740 1 -- 172.21.15.47:0/1667584252 shutdown_connections 2024-03-21T11:44:24.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.626+0000 7ff7c2578740 1 -- 172.21.15.47:0/1667584252 wait complete. 2024-03-21T11:44:24.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.627+0000 7ff7c2578740 1 -- start start 2024-03-21T11:44:24.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.627+0000 7ff7c2578740 1 Processor -- start 2024-03-21T11:44:24.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.630+0000 7ff7c2578740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866dbc0 0x55eed867a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.630+0000 7ff7c2578740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed867a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.630+0000 7ff7c2578740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed867d7a0 0x55eed867dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.630+0000 7ff7c2578740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81ba040 con 0x55eed867d7a0 2024-03-21T11:44:24.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.630+0000 7ff7c2578740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81b99e0 con 0x55eed866dbc0 2024-03-21T11:44:24.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.630+0000 7ff7c2578740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81b6ee0 con 0x55eed866e580 2024-03-21T11:44:24.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.630+0000 7ff7c2578740 10 client.0 _ll_register_callbacks cb 0x7ff7a400bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:24.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.631+0000 7ff7bce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed867a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.631+0000 7ff7bce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed867a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44872/0 (socket says 172.21.15.47:44872) 2024-03-21T11:44:24.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.631+0000 7ff7bce00640 1 -- 172.21.15.47:0/1472623284 learned_addr learned my addr 172.21.15.47:0/1472623284 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:24.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.631+0000 7ff7bce00640 1 -- 172.21.15.47:0/1472623284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866dbc0 msgr2=0x55eed867a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:24.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.631+0000 7ff7bce00640 1 --2- 172.21.15.47:0/1472623284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866dbc0 0x55eed867a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.631+0000 7ff7bce00640 1 -- 172.21.15.47:0/1472623284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed867d7a0 msgr2=0x55eed867dc20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:44:24.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.631+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/1472623284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed867d7a0 0x55eed867dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.631+0000 7ff7bce00640 1 --2- 172.21.15.47:0/1472623284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed867d7a0 0x55eed867dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.632+0000 7ff7bce00640 1 -- 172.21.15.47:0/1472623284 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eed867e160 con 0x55eed866e580 2024-03-21T11:44:24.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.632+0000 7ff7bce00640 1 --2- 172.21.15.47:0/1472623284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed867a780 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x55eed8668d50 tx=0x7ff7b0004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=755a8615bec9c4fb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:24.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.633+0000 7ff7a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:24.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.634+0000 7ff7a2a00640 1 -- 172.21.15.47:0/1472623284 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b001f020 con 0x55eed866e580 2024-03-21T11:44:24.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.634+0000 7ff7c2578740 1 -- 172.21.15.47:0/1472623284 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7a400bcc0 con 0x55eed866e580 2024-03-21T11:44:24.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.635+0000 7ff7a2a00640 1 -- 172.21.15.47:0/1472623284 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff7b0012aa0 con 0x55eed866e580 2024-03-21T11:44:24.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.639+0000 7ff7a2a00640 1 -- 172.21.15.47:0/1472623284 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7b002a050 con 0x55eed866e580 2024-03-21T11:44:24.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.639+0000 7ff7c2578740 10 client.32284 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:24.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.639+0000 7ff7c2578740 1 -- 172.21.15.47:0/1472623284 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55eed86de910 con 0x55eed866e580 2024-03-21T11:44:24.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.640+0000 7ff7a2a00640 1 -- 172.21.15.47:0/1472623284 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7a400bcc0 con 0x55eed866e580 2024-03-21T11:44:24.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.640+0000 7ff7a2a00640 1 -- 172.21.15.47:0/1472623284 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff7b000f040 con 0x55eed866e580 2024-03-21T11:44:24.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.640+0000 7ff7c2578740 10 client.32284 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:24.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.640+0000 7ff7c2578740 10 client.32284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:24.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.641+0000 7ff7c2578740 20 client.32284 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:24.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.641+0000 7ff7c2578740 1 --2- 172.21.15.47:0/1472623284 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55eed86df2a0 0x55eed86e16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:24.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.641+0000 7ff7c2578740 4 client.32284 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:44:24.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.641+0000 7ff7c2578740 1 -- 172.21.15.47:0/1472623284 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eed85d8b50 con 0x55eed86df2a0 2024-03-21T11:44:24.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.645+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/1472623284 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55eed86df2a0 0x55eed86e16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:24.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.647+0000 7ff7b7e00640 1 --2- 172.21.15.47:0/1472623284 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55eed86df2a0 0x55eed86e16c0 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94af60fab98c001 server_cookie=195751bb8210ee58 in_seq=0 out_seq=0 2024-03-21T11:44:24.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.647+0000 7ff7a2a00640 10 client.32284 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:24.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.659+0000 7ff7a2a00640 1 -- 172.21.15.47:0/1472623284 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eed85d8b50 con 0x55eed86df2a0 2024-03-21T11:44:24.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.659+0000 7ff7a2a00640 10 client.32284 handle_command_reply: tid=0 2024-03-21T11:44:24.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.660+0000 7ff7c2578740 1 client.32284 shutdown 2024-03-21T11:44:24.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.660+0000 7ff7c2578740 1 -- 172.21.15.47:0/1472623284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 msgr2=0x55eed867a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.660+0000 7ff7c2578740 1 --2- 172.21.15.47:0/1472623284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed867a780 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x55eed8668d50 tx=0x7ff7b0004580 comp rx=0 tx=0).stop 2024-03-21T11:44:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.661+0000 7ff7c2578740 1 -- 172.21.15.47:0/1472623284 shutdown_connections 2024-03-21T11:44:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.661+0000 7ff7c2578740 1 --2- 172.21.15.47:0/1472623284 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55eed86df2a0 0x55eed86e16c0 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.661+0000 7ff7c2578740 1 --2- 172.21.15.47:0/1472623284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eed867d7a0 0x55eed867dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.661+0000 7ff7c2578740 1 --2- 172.21.15.47:0/1472623284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eed866e580 0x55eed867a780 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.661+0000 7ff7c2578740 1 --2- 172.21.15.47:0/1472623284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eed866dbc0 0x55eed867a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.661+0000 7ff7c2578740 1 -- 172.21.15.47:0/1472623284 >> 172.21.15.47:0/1472623284 conn(0x55eed8667de0 msgr2=0x55eed8670df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:24.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.661+0000 7ff7a2a00640 0 client.32284 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:24.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.662+0000 7ff7c2578740 1 -- 172.21.15.47:0/1472623284 shutdown_connections 2024-03-21T11:44:24.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.662+0000 7ff7c2578740 1 -- 172.21.15.47:0/1472623284 wait complete. 2024-03-21T11:44:24.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.662+0000 7ff7c2578740 20 client.32284 trim_cache size 0 max 16384 2024-03-21T11:44:24.665 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:44:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.665+0000 7ff7be200640 1 -- 172.21.15.47:0/1592958600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff79c0a30f0 msgr2=0x7ff79c0a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.665+0000 7ff7be200640 1 --2- 172.21.15.47:0/1592958600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff79c0a30f0 0x7ff79c0a55b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80bd910 tx=0x7ff7a8004ec0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.665+0000 7ff7be200640 1 -- 172.21.15.47:0/1592958600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b8109450 msgr2=0x7ff7b819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.665+0000 7ff7be200640 1 --2- 172.21.15.47:0/1592958600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b8109450 0x7ff7b819cbb0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7ff7a802c200 tx=0x7ff7a8005d50 comp rx=0 tx=0).stop 2024-03-21T11:44:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.665+0000 7ff7be200640 1 -- 172.21.15.47:0/1592958600 shutdown_connections 2024-03-21T11:44:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.665+0000 7ff7be200640 1 --2- 172.21.15.47:0/1592958600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff79c0a30f0 0x7ff79c0a55b0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.665+0000 7ff7be200640 1 --2- 172.21.15.47:0/1592958600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b8109450 0x7ff7b819cbb0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.665+0000 7ff7be200640 1 --2- 172.21.15.47:0/1592958600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b8075660 0x7ff7b8195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.665+0000 7ff7be200640 1 --2- 172.21.15.47:0/1592958600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7b8074080 0x7ff7b8195600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:24.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.665+0000 7ff7be200640 1 -- 172.21.15.47:0/1592958600 >> 172.21.15.47:0/1592958600 conn(0x7ff7b80fcb10 msgr2=0x7ff7b8100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:24.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.666+0000 7ff7be200640 1 -- 172.21.15.47:0/1592958600 shutdown_connections 2024-03-21T11:44:24.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:24.666+0000 7ff7be200640 1 -- 172.21.15.47:0/1592958600 wait complete. 2024-03-21T11:44:24.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 188518 22.79 MB/sec execute 308 sec latency 146.193 ms 2024-03-21T11:44:25.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:24 smithi047 ceph-mon[31351]: pgmap v1032: 129 pgs: 129 active+clean; 608 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.1 MiB/s wr, 653 op/s 2024-03-21T11:44:25.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:24 smithi100 ceph-mon[36245]: pgmap v1032: 129 pgs: 129 active+clean; 608 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.1 MiB/s wr, 653 op/s 2024-03-21T11:44:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:24 smithi102 ceph-mon[36748]: pgmap v1032: 129 pgs: 129 active+clean; 608 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.1 MiB/s wr, 653 op/s 2024-03-21T11:44:25.701 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:44:25.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:44:25.702 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:44:25.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 188783 22.76 MB/sec execute 309 sec latency 151.179 ms 2024-03-21T11:44:25.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.978+0000 7fea47000640 1 -- 172.21.15.47:0/4131120468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea400756a0 msgr2=0x7fea40073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:25.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.978+0000 7fea47000640 1 --2- 172.21.15.47:0/4131120468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea400756a0 0x7fea40073af0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fea30009900 tx=0x7fea3002bc30 comp rx=0 tx=0).stop 2024-03-21T11:44:25.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.978+0000 7fea47000640 1 -- 172.21.15.47:0/4131120468 shutdown_connections 2024-03-21T11:44:25.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.978+0000 7fea47000640 1 --2- 172.21.15.47:0/4131120468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea40109560 0x7fea4010b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:25.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.978+0000 7fea47000640 1 --2- 172.21.15.47:0/4131120468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea400740c0 0x7fea40074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:25.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.978+0000 7fea47000640 1 --2- 172.21.15.47:0/4131120468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea400756a0 0x7fea40073af0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:25.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.978+0000 7fea47000640 1 -- 172.21.15.47:0/4131120468 >> 172.21.15.47:0/4131120468 conn(0x7fea400fcc40 msgr2=0x7fea400ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:25.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.978+0000 7fea47000640 1 -- 172.21.15.47:0/4131120468 shutdown_connections 2024-03-21T11:44:25.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.978+0000 7fea47000640 1 -- 172.21.15.47:0/4131120468 wait complete. 2024-03-21T11:44:25.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.982+0000 7fea47000640 1 Processor -- start 2024-03-21T11:44:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.982+0000 7fea47000640 1 -- start start 2024-03-21T11:44:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.982+0000 7fea47000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea400740c0 0x7fea4019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.982+0000 7fea47000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea400756a0 0x7fea4019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.982+0000 7fea47000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea40109560 0x7fea401a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.982+0000 7fea47000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea40101ea0 con 0x7fea400740c0 2024-03-21T11:44:25.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.982+0000 7fea47000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea40101d20 con 0x7fea40109560 2024-03-21T11:44:25.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.982+0000 7fea47000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea40102020 con 0x7fea400756a0 2024-03-21T11:44:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.983+0000 7fea3f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea400756a0 0x7fea4019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.983+0000 7fea45000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea40109560 0x7fea401a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.983+0000 7fea3f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea400756a0 0x7fea4019e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44896/0 (socket says 172.21.15.47:44896) 2024-03-21T11:44:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.983+0000 7fea3f400640 1 -- 172.21.15.47:0/391642161 learned_addr learned my addr 172.21.15.47:0/391642161 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.983+0000 7fea3fe00640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea400740c0 0x7fea4019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.984+0000 7fea3fe00640 1 -- 172.21.15.47:0/391642161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea400756a0 msgr2=0x7fea4019e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.984+0000 7fea3fe00640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea400756a0 0x7fea4019e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.984+0000 7fea3fe00640 1 -- 172.21.15.47:0/391642161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea40109560 msgr2=0x7fea401a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.984+0000 7fea3fe00640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea40109560 0x7fea401a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.984+0000 7fea3fe00640 1 -- 172.21.15.47:0/391642161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea401a5bf0 con 0x7fea400740c0 2024-03-21T11:44:25.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.984+0000 7fea45000640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea40109560 0x7fea401a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:25.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.984+0000 7fea3f400640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea400756a0 0x7fea4019e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:25.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.984+0000 7fea3fe00640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea400740c0 0x7fea4019dfd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fea30036040 tx=0x7fea30004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:25.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.985+0000 7fea3cc00640 1 -- 172.21.15.47:0/391642161 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea30043020 con 0x7fea400740c0 2024-03-21T11:44:25.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.985+0000 7fea3cc00640 1 -- 172.21.15.47:0/391642161 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea300054e0 con 0x7fea400740c0 2024-03-21T11:44:25.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.985+0000 7fea3cc00640 1 -- 172.21.15.47:0/391642161 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea30003bf0 con 0x7fea400740c0 2024-03-21T11:44:25.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.985+0000 7fea47000640 1 -- 172.21.15.47:0/391642161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea401a5e80 con 0x7fea400740c0 2024-03-21T11:44:25.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.985+0000 7fea47000640 1 -- 172.21.15.47:0/391642161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea401a6230 con 0x7fea400740c0 2024-03-21T11:44:25.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.989+0000 7fea49db2740 1 Processor -- start 2024-03-21T11:44:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.989+0000 7fea49db2740 1 -- start start 2024-03-21T11:44:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.989+0000 7fea3cc00640 1 -- 172.21.15.47:0/391642161 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fea30046020 con 0x7fea400740c0 2024-03-21T11:44:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.990+0000 7fea49db2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce10a0 0x55e748ce14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.990+0000 7fea49db2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce1a90 0x55e748ce6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:25.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.990+0000 7fea49db2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 0x55e748ce9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.990+0000 7fea49db2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748c46eb0 con 0x55e748ce10a0 2024-03-21T11:44:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.990+0000 7fea49db2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748befa60 con 0x55e748ce1a90 2024-03-21T11:44:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.990+0000 7fea49db2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748c42f50 con 0x55e748ce6c00 2024-03-21T11:44:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.990+0000 7fea45000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce10a0 0x55e748ce14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.990+0000 7fea3fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce1a90 0x55e748ce6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.990+0000 7fea3fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce1a90 0x55e748ce6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34922/0 (socket says 172.21.15.47:34922) 2024-03-21T11:44:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.990+0000 7fea3fe00640 1 -- 172.21.15.47:0/2290000321 learned_addr learned my addr 172.21.15.47:0/2290000321 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:25.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.991+0000 7fea3f400640 1 --2- 172.21.15.47:0/2290000321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 0x55e748ce9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:25.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.991+0000 7fea3fe00640 1 -- 172.21.15.47:0/2290000321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 msgr2=0x55e748ce9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:25.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.991+0000 7fea3fe00640 1 --2- 172.21.15.47:0/2290000321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 0x55e748ce9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:25.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.991+0000 7fea3fe00640 1 -- 172.21.15.47:0/2290000321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce10a0 msgr2=0x55e748ce14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:25.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.991+0000 7fea3fe00640 1 --2- 172.21.15.47:0/2290000321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce10a0 0x55e748ce14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:25.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.991+0000 7fea3fe00640 1 -- 172.21.15.47:0/2290000321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e748a237e0 con 0x55e748ce1a90 2024-03-21T11:44:25.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.991+0000 7fea45000640 1 --2- 172.21.15.47:0/2290000321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce10a0 0x55e748ce14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:25.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.991+0000 7fea3cc00640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fea240a2f70 0x7fea240a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:25.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.992+0000 7fea3fe00640 1 --2- 172.21.15.47:0/2290000321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce1a90 0x55e748ce6590 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fea300f7a90 tx=0x7fea300f7b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2c5a1b0aee0c2ac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:25.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.992+0000 7fea35600640 1 -- 172.21.15.47:0/2290000321 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea300b9f10 con 0x55e748ce1a90 2024-03-21T11:44:25.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.992+0000 7fea45000640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fea240a2f70 0x7fea240a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:25.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.992+0000 7fea3cc00640 1 -- 172.21.15.47:0/391642161 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fea300f1930 con 0x7fea400740c0 2024-03-21T11:44:25.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.992+0000 7fea35600640 1 -- 172.21.15.47:0/2290000321 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea300be370 con 0x55e748ce1a90 2024-03-21T11:44:25.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.992+0000 7fea35600640 1 -- 172.21.15.47:0/2290000321 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea300bd5d0 con 0x55e748ce1a90 2024-03-21T11:44:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.993+0000 7fea49db2740 1 -- 172.21.15.47:0/2290000321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce1a90 msgr2=0x55e748ce6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.993+0000 7fea49db2740 1 --2- 172.21.15.47:0/2290000321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce1a90 0x55e748ce6590 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fea300f7a90 tx=0x7fea300f7b80 comp rx=0 tx=0).stop 2024-03-21T11:44:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.993+0000 7fea45000640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fea240a2f70 0x7fea240a5430 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x55e748ce22f0 tx=0x7fea38009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.993+0000 7fea49db2740 1 -- 172.21.15.47:0/2290000321 shutdown_connections 2024-03-21T11:44:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.993+0000 7fea49db2740 1 --2- 172.21.15.47:0/2290000321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 0x55e748ce9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.993+0000 7fea49db2740 1 --2- 172.21.15.47:0/2290000321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce1a90 0x55e748ce6590 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.993+0000 7fea49db2740 1 --2- 172.21.15.47:0/2290000321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce10a0 0x55e748ce14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:25.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.993+0000 7fea49db2740 1 -- 172.21.15.47:0/2290000321 >> 172.21.15.47:0/2290000321 conn(0x55e748cdc610 msgr2=0x55e748cdca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:25.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.993+0000 7fea49db2740 1 -- 172.21.15.47:0/2290000321 shutdown_connections 2024-03-21T11:44:25.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.993+0000 7fea49db2740 1 -- 172.21.15.47:0/2290000321 wait complete. 2024-03-21T11:44:25.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.994+0000 7fea49db2740 1 -- start start 2024-03-21T11:44:25.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.994+0000 7fea49db2740 1 Processor -- start 2024-03-21T11:44:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.995+0000 7fea49db2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce10a0 0x55e748cead40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.995+0000 7fea49db2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce1a90 0x55e748ceb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.995+0000 7fea49db2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce6c00 0x55e748cf2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.995+0000 7fea49db2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748befa60 con 0x55e748ce6c00 2024-03-21T11:44:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.995+0000 7fea49db2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748c02eb0 con 0x55e748ce1a90 2024-03-21T11:44:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.995+0000 7fea49db2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748ce9970 con 0x55e748ce10a0 2024-03-21T11:44:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.995+0000 7fea49db2740 10 client.0 _ll_register_callbacks cb 0x55e748bf7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.995+0000 7fea3fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce10a0 0x55e748cead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.995+0000 7fea3fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce10a0 0x55e748cead40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44912/0 (socket says 172.21.15.47:44912) 2024-03-21T11:44:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.995+0000 7fea3fe00640 1 -- 172.21.15.47:0/234330299 learned_addr learned my addr 172.21.15.47:0/234330299 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:25.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.996+0000 7fea45000640 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce6c00 0x55e748cf2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.996+0000 7fea3f400640 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce1a90 0x55e748ceb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.996+0000 7fea3fe00640 1 -- 172.21.15.47:0/234330299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce1a90 msgr2=0x55e748ceb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.996+0000 7fea3fe00640 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce1a90 0x55e748ceb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.996+0000 7fea3fe00640 1 -- 172.21.15.47:0/234330299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce6c00 msgr2=0x55e748cf2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.996+0000 7fea3fe00640 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce6c00 0x55e748cf2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:25.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.996+0000 7fea3fe00640 1 -- 172.21.15.47:0/234330299 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e748a83f70 con 0x55e748ce10a0 2024-03-21T11:44:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.996+0000 7fea3f400640 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce1a90 0x55e748ceb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:44:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.996+0000 7fea45000640 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce6c00 0x55e748cf2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:44:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.996+0000 7fea3fe00640 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce10a0 0x55e748cead40 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fea300be520 tx=0x7fea30004310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1f8fcd73b5ed513 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.996+0000 7fea2a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.997+0000 7fea2a000640 1 -- 172.21.15.47:0/234330299 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea300bd7b0 con 0x55e748ce10a0 2024-03-21T11:44:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.997+0000 7fea2a000640 1 -- 172.21.15.47:0/234330299 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fea300043a0 con 0x55e748ce10a0 2024-03-21T11:44:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.997+0000 7fea49db2740 1 -- 172.21.15.47:0/234330299 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea300be370 con 0x55e748ce10a0 2024-03-21T11:44:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.997+0000 7fea2a000640 1 -- 172.21.15.47:0/234330299 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea300bda40 con 0x55e748ce10a0 2024-03-21T11:44:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.997+0000 7fea49db2740 10 client.32296 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:26.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.997+0000 7fea49db2740 1 -- 172.21.15.47:0/234330299 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e74891b170 con 0x55e748ce10a0 2024-03-21T11:44:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.997+0000 7fea2a000640 1 -- 172.21.15.47:0/234330299 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea300be370 con 0x55e748ce10a0 2024-03-21T11:44:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.998+0000 7fea2a000640 1 -- 172.21.15.47:0/234330299 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fea300bded0 con 0x55e748ce10a0 2024-03-21T11:44:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.998+0000 7fea49db2740 10 client.32296 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.998+0000 7fea49db2740 10 client.32296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.998+0000 7fea49db2740 20 client.32296 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.998+0000 7fea49db2740 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e748cf6850 0x55e748cf8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.998+0000 7fea49db2740 4 client.32296 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.998+0000 7fea49db2740 1 -- 172.21.15.47:0/234330299 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e748c543f0 con 0x55e748cf6850 2024-03-21T11:44:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.998+0000 7fea3f400640 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e748cf6850 0x55e748cf8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.999+0000 7fea2a000640 10 client.32296 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:25.999+0000 7fea3f400640 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e748cf6850 0x55e748cf8c40 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1144a0011be26d4 server_cookie=ec41209e9e884dca in_seq=0 out_seq=0 2024-03-21T11:44:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.001+0000 7fea2a000640 1 -- 172.21.15.47:0/234330299 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e748c543f0 con 0x55e748cf6850 2024-03-21T11:44:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.001+0000 7fea2a000640 10 client.32296 handle_command_reply: tid=0 2024-03-21T11:44:26.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.001+0000 7fea49db2740 1 client.32296 shutdown 2024-03-21T11:44:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.002+0000 7fea49db2740 1 -- 172.21.15.47:0/234330299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce10a0 msgr2=0x55e748cead40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.002+0000 7fea49db2740 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce10a0 0x55e748cead40 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fea300be520 tx=0x7fea30004310 comp rx=0 tx=0).stop 2024-03-21T11:44:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.002+0000 7fea49db2740 1 -- 172.21.15.47:0/234330299 shutdown_connections 2024-03-21T11:44:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.002+0000 7fea49db2740 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e748cf6850 0x55e748cf8c40 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.002+0000 7fea49db2740 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce6c00 0x55e748cf2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.002+0000 7fea49db2740 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce1a90 0x55e748ceb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.002+0000 7fea49db2740 1 --2- 172.21.15.47:0/234330299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce10a0 0x55e748cead40 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.002+0000 7fea2a000640 0 client.32296 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.002+0000 7fea49db2740 1 -- 172.21.15.47:0/234330299 >> 172.21.15.47:0/234330299 conn(0x55e748cdc610 msgr2=0x55e748cdb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.002+0000 7fea49db2740 1 -- 172.21.15.47:0/234330299 shutdown_connections 2024-03-21T11:44:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.002+0000 7fea49db2740 1 -- 172.21.15.47:0/234330299 wait complete. 2024-03-21T11:44:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.002+0000 7fea49db2740 20 client.32296 trim_cache size 0 max 16384 2024-03-21T11:44:26.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.036+0000 7fea49db2740 1 Processor -- start 2024-03-21T11:44:26.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.036+0000 7fea49db2740 1 -- start start 2024-03-21T11:44:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.036+0000 7fea49db2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce10a0 0x55e748ceafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.037+0000 7fea49db2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce1a90 0x55e748cf21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.037+0000 7fea49db2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 0x55e748cfd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.037+0000 7fea49db2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea401b06d0 con 0x55e748ce1a90 2024-03-21T11:44:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.037+0000 7fea49db2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea401b2050 con 0x55e748ce10a0 2024-03-21T11:44:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.037+0000 7fea49db2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea401b0d30 con 0x55e748ce6c00 2024-03-21T11:44:26.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.037+0000 7fea3fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce10a0 0x55e748ceafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.037+0000 7fea3fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce10a0 0x55e748ceafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34932/0 (socket says 172.21.15.47:34932) 2024-03-21T11:44:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.037+0000 7fea3fe00640 1 -- 172.21.15.47:0/2835426068 learned_addr learned my addr 172.21.15.47:0/2835426068 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.037+0000 7fea45000640 1 --2- 172.21.15.47:0/2835426068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 0x55e748cfd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.038+0000 7fea3fe00640 1 -- 172.21.15.47:0/2835426068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 msgr2=0x55e748cfd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.038+0000 7fea3fe00640 1 --2- 172.21.15.47:0/2835426068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 0x55e748cfd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.038+0000 7fea3fe00640 1 -- 172.21.15.47:0/2835426068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce1a90 msgr2=0x55e748cf21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:26.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.038+0000 7fea3fe00640 1 --2- 172.21.15.47:0/2835426068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce1a90 0x55e748cf21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.038+0000 7fea3fe00640 1 -- 172.21.15.47:0/2835426068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e748cd7c20 con 0x55e748ce10a0 2024-03-21T11:44:26.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.039+0000 7fea3fe00640 1 --2- 172.21.15.47:0/2835426068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce10a0 0x55e748ceafc0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fea300bd970 tx=0x7fea300036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce7ee2771e93bfb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:26.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.039+0000 7fea34c00640 1 -- 172.21.15.47:0/2835426068 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea300db070 con 0x55e748ce10a0 2024-03-21T11:44:26.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.039+0000 7fea34c00640 1 -- 172.21.15.47:0/2835426068 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea300be370 con 0x55e748ce10a0 2024-03-21T11:44:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.040+0000 7fea49db2740 1 -- 172.21.15.47:0/2835426068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce10a0 msgr2=0x55e748ceafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.040+0000 7fea49db2740 1 --2- 172.21.15.47:0/2835426068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce10a0 0x55e748ceafc0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fea300bd970 tx=0x7fea300036f0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.040+0000 7fea49db2740 1 -- 172.21.15.47:0/2835426068 shutdown_connections 2024-03-21T11:44:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.040+0000 7fea49db2740 1 --2- 172.21.15.47:0/2835426068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 0x55e748cfd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.040+0000 7fea49db2740 1 --2- 172.21.15.47:0/2835426068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce1a90 0x55e748cf21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.040+0000 7fea49db2740 1 --2- 172.21.15.47:0/2835426068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce10a0 0x55e748ceafc0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.040+0000 7fea49db2740 1 -- 172.21.15.47:0/2835426068 >> 172.21.15.47:0/2835426068 conn(0x55e748cdc610 msgr2=0x55e748ce2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:26.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.040+0000 7fea49db2740 1 -- 172.21.15.47:0/2835426068 shutdown_connections 2024-03-21T11:44:26.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.040+0000 7fea49db2740 1 -- 172.21.15.47:0/2835426068 wait complete. 2024-03-21T11:44:26.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.041+0000 7fea49db2740 1 -- start start 2024-03-21T11:44:26.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.041+0000 7fea49db2740 1 Processor -- start 2024-03-21T11:44:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.042+0000 7fea49db2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce10a0 0x55e748cf1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.042+0000 7fea49db2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce1a90 0x55e748cf0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.042+0000 7fea49db2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 0x55e748cf0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.042+0000 7fea49db2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea401b19f0 con 0x55e748ce1a90 2024-03-21T11:44:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.042+0000 7fea49db2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea401b1390 con 0x55e748ce10a0 2024-03-21T11:44:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.042+0000 7fea49db2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea401ae890 con 0x55e748ce6c00 2024-03-21T11:44:26.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.043+0000 7fea49db2740 10 client.0 _ll_register_callbacks cb 0x7fea38009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.043+0000 7fea3fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce10a0 0x55e748cf1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.043+0000 7fea3fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce10a0 0x55e748cf1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34940/0 (socket says 172.21.15.47:34940) 2024-03-21T11:44:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.043+0000 7fea3fe00640 1 -- 172.21.15.47:0/820551232 learned_addr learned my addr 172.21.15.47:0/820551232 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.043+0000 7fea45000640 1 --2- 172.21.15.47:0/820551232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 0x55e748cf0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.043+0000 7fea3f400640 1 --2- 172.21.15.47:0/820551232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce1a90 0x55e748cf0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.043+0000 7fea3fe00640 1 -- 172.21.15.47:0/820551232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 msgr2=0x55e748cf0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.043+0000 7fea3fe00640 1 --2- 172.21.15.47:0/820551232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 0x55e748cf0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.043+0000 7fea3fe00640 1 -- 172.21.15.47:0/820551232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce1a90 msgr2=0x55e748cf0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.043+0000 7fea3fe00640 1 --2- 172.21.15.47:0/820551232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce1a90 0x55e748cf0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.043+0000 7fea3fe00640 1 -- 172.21.15.47:0/820551232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e748cf0e90 con 0x55e748ce10a0 2024-03-21T11:44:26.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.043+0000 7fea3f400640 1 --2- 172.21.15.47:0/820551232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce1a90 0x55e748cf0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:44:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.044+0000 7fea3fe00640 1 --2- 172.21.15.47:0/820551232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce10a0 0x55e748cf1db0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fea300ba640 tx=0x7fea30000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c1d594fc5303007 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.044+0000 7fea2a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:44:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.044+0000 7fea2a000640 1 -- 172.21.15.47:0/820551232 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea300be0d0 con 0x55e748ce10a0 2024-03-21T11:44:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.044+0000 7fea49db2740 1 -- 172.21.15.47:0/820551232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea300be370 con 0x55e748ce10a0 2024-03-21T11:44:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.044+0000 7fea2a000640 1 -- 172.21.15.47:0/820551232 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fea300f7220 con 0x55e748ce10a0 2024-03-21T11:44:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.045+0000 7fea2a000640 1 -- 172.21.15.47:0/820551232 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea300f9a90 con 0x55e748ce10a0 2024-03-21T11:44:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.045+0000 7fea49db2740 10 client.32327 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.045+0000 7fea49db2740 1 -- 172.21.15.47:0/820551232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e748cf0e90 con 0x55e748ce10a0 2024-03-21T11:44:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.045+0000 7fea2a000640 1 -- 172.21.15.47:0/820551232 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea300a1070 con 0x55e748ce10a0 2024-03-21T11:44:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.045+0000 7fea2a000640 1 -- 172.21.15.47:0/820551232 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fea300c6280 con 0x55e748ce10a0 2024-03-21T11:44:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.045+0000 7fea49db2740 10 client.32327 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.045+0000 7fea49db2740 10 client.32327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.045+0000 7fea49db2740 20 client.32327 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.045+0000 7fea49db2740 1 --2- 172.21.15.47:0/820551232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e748d532c0 0x55e748d556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.046+0000 7fea49db2740 4 client.32327 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:44:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.046+0000 7fea49db2740 1 -- 172.21.15.47:0/820551232 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e748c4e740 con 0x55e748d532c0 2024-03-21T11:44:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.046+0000 7fea3f400640 1 --2- 172.21.15.47:0/820551232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e748d532c0 0x55e748d556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.047+0000 7fea3f400640 1 --2- 172.21.15.47:0/820551232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e748d532c0 0x55e748d556e0 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a96bf07f354ff4c server_cookie=4ce2b7a2bd8291ef in_seq=0 out_seq=0 2024-03-21T11:44:26.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.047+0000 7fea2a000640 10 client.32327 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:26.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.051+0000 7fea2a000640 1 -- 172.21.15.47:0/820551232 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e748c4e740 con 0x55e748d532c0 2024-03-21T11:44:26.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.051+0000 7fea2a000640 10 client.32327 handle_command_reply: tid=0 2024-03-21T11:44:26.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.051+0000 7fea49db2740 1 client.32327 shutdown 2024-03-21T11:44:26.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.051+0000 7fea49db2740 1 -- 172.21.15.47:0/820551232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce10a0 msgr2=0x55e748cf1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:26.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.051+0000 7fea49db2740 1 --2- 172.21.15.47:0/820551232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce10a0 0x55e748cf1db0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fea300ba640 tx=0x7fea30000bc0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.052+0000 7fea49db2740 1 -- 172.21.15.47:0/820551232 shutdown_connections 2024-03-21T11:44:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.052+0000 7fea49db2740 1 --2- 172.21.15.47:0/820551232 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e748d532c0 0x55e748d556e0 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.052+0000 7fea49db2740 1 --2- 172.21.15.47:0/820551232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e748ce6c00 0x55e748cf0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.052+0000 7fea49db2740 1 --2- 172.21.15.47:0/820551232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e748ce1a90 0x55e748cf0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.052+0000 7fea49db2740 1 --2- 172.21.15.47:0/820551232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e748ce10a0 0x55e748cf1db0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.052+0000 7fea49db2740 1 -- 172.21.15.47:0/820551232 >> 172.21.15.47:0/820551232 conn(0x55e748cdc610 msgr2=0x55e748ce2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.052+0000 7fea2a000640 0 client.32327 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.052+0000 7fea49db2740 1 -- 172.21.15.47:0/820551232 shutdown_connections 2024-03-21T11:44:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.052+0000 7fea49db2740 1 -- 172.21.15.47:0/820551232 wait complete. 2024-03-21T11:44:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.052+0000 7fea49db2740 20 client.32327 trim_cache size 0 max 16384 2024-03-21T11:44:26.055 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:44:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:44:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "da94a7ed-c009-4cbe-a58b-8049ef2b9467", 2024-03-21T11:44:26.055 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:44:26.055 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:44:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.053+0000 7fea47000640 1 -- 172.21.15.47:0/391642161 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fea240a2f70 msgr2=0x7fea240a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.053+0000 7fea47000640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fea240a2f70 0x7fea240a5430 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x55e748ce22f0 tx=0x7fea38009500 comp rx=0 tx=0).stop 2024-03-21T11:44:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.053+0000 7fea47000640 1 -- 172.21.15.47:0/391642161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea400740c0 msgr2=0x7fea4019dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.053+0000 7fea47000640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea400740c0 0x7fea4019dfd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fea30036040 tx=0x7fea30004e30 comp rx=0 tx=0).stop 2024-03-21T11:44:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.054+0000 7fea47000640 1 -- 172.21.15.47:0/391642161 shutdown_connections 2024-03-21T11:44:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.054+0000 7fea47000640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fea240a2f70 0x7fea240a5430 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.054+0000 7fea47000640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea40109560 0x7fea401a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.054+0000 7fea47000640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea400756a0 0x7fea4019e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.054+0000 7fea47000640 1 --2- 172.21.15.47:0/391642161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea400740c0 0x7fea4019dfd0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.054+0000 7fea47000640 1 -- 172.21.15.47:0/391642161 >> 172.21.15.47:0/391642161 conn(0x7fea400fcc40 msgr2=0x7fea40102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.055+0000 7fea47000640 1 -- 172.21.15.47:0/391642161 shutdown_connections 2024-03-21T11:44:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.055+0000 7fea47000640 1 -- 172.21.15.47:0/391642161 wait complete. 2024-03-21T11:44:26.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:44:26.082 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:44:26.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.324+0000 7fef61800640 1 -- 172.21.15.47:0/1984458849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5c077c40 msgr2=0x7fef5c0784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:26.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.324+0000 7fef61800640 1 --2- 172.21.15.47:0/1984458849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5c077c40 0x7fef5c0784e0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fef54009be0 tx=0x7fef5402bf10 comp rx=0 tx=0).stop 2024-03-21T11:44:26.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.325+0000 7fef61800640 1 -- 172.21.15.47:0/1984458849 shutdown_connections 2024-03-21T11:44:26.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.325+0000 7fef61800640 1 --2- 172.21.15.47:0/1984458849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5c077c40 0x7fef5c0784e0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.325+0000 7fef61800640 1 --2- 172.21.15.47:0/1984458849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef5c0772a0 0x7fef5c077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.325+0000 7fef61800640 1 --2- 172.21.15.47:0/1984458849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef5c076060 0x7fef5c076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.325+0000 7fef61800640 1 -- 172.21.15.47:0/1984458849 >> 172.21.15.47:0/1984458849 conn(0x7fef5c0fee50 msgr2=0x7fef5c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:26.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.326+0000 7fef61800640 1 -- 172.21.15.47:0/1984458849 shutdown_connections 2024-03-21T11:44:26.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.326+0000 7fef61800640 1 -- 172.21.15.47:0/1984458849 wait complete. 2024-03-21T11:44:26.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.328+0000 7fef61800640 1 Processor -- start 2024-03-21T11:44:26.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.329+0000 7fef61800640 1 -- start start 2024-03-21T11:44:26.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.330+0000 7fef61800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5c076060 0x7fef5c19deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.330+0000 7fef61800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef5c0772a0 0x7fef5c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.332+0000 7fef61800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef5c077c40 0x7fef5c1a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.332+0000 7fef61800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c113970 con 0x7fef5c077c40 2024-03-21T11:44:26.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.332+0000 7fef61800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c1137f0 con 0x7fef5c0772a0 2024-03-21T11:44:26.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.332+0000 7fef61800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c113af0 con 0x7fef5c076060 2024-03-21T11:44:26.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.332+0000 7fef5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5c076060 0x7fef5c19deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.332+0000 7fef5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5c076060 0x7fef5c19deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44950/0 (socket says 172.21.15.47:44950) 2024-03-21T11:44:26.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.332+0000 7fef5aa00640 1 -- 172.21.15.47:0/1009787931 learned_addr learned my addr 172.21.15.47:0/1009787931 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:26.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.333+0000 7fef5a000640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef5c0772a0 0x7fef5c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.333+0000 7fef5b400640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef5c077c40 0x7fef5c1a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.334+0000 7fef5aa00640 1 -- 172.21.15.47:0/1009787931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef5c0772a0 msgr2=0x7fef5c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:26.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.334+0000 7fef5aa00640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef5c0772a0 0x7fef5c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.334+0000 7fef5aa00640 1 -- 172.21.15.47:0/1009787931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef5c077c40 msgr2=0x7fef5c1a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:26.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.334+0000 7fef5aa00640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef5c077c40 0x7fef5c1a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.334+0000 7fef5aa00640 1 -- 172.21.15.47:0/1009787931 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c1a5ad0 con 0x7fef5c076060 2024-03-21T11:44:26.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.334+0000 7fef5b400640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef5c077c40 0x7fef5c1a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:26.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.338+0000 7fef5aa00640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5c076060 0x7fef5c19deb0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00c920 tx=0x7fef4c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:26.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.339+0000 7fef53400640 1 -- 172.21.15.47:0/1009787931 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c007bf0 con 0x7fef5c076060 2024-03-21T11:44:26.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.339+0000 7fef53400640 1 -- 172.21.15.47:0/1009787931 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef4c007d90 con 0x7fef5c076060 2024-03-21T11:44:26.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.339+0000 7fef53400640 1 -- 172.21.15.47:0/1009787931 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c018860 con 0x7fef5c076060 2024-03-21T11:44:26.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.339+0000 7fef61800640 1 -- 172.21.15.47:0/1009787931 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5c1a5dc0 con 0x7fef5c076060 2024-03-21T11:44:26.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.339+0000 7fef61800640 1 -- 172.21.15.47:0/1009787931 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5c1a6230 con 0x7fef5c076060 2024-03-21T11:44:26.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.346+0000 7fef53400640 1 -- 172.21.15.47:0/1009787931 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fef4c018a00 con 0x7fef5c076060 2024-03-21T11:44:26.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.347+0000 7fef64482740 1 Processor -- start 2024-03-21T11:44:26.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.347+0000 7fef64482740 1 -- start start 2024-03-21T11:44:26.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.348+0000 7fef64482740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd7e80 0x556677bd82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.349+0000 7fef64482740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bd8870 0x556677bdd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.349+0000 7fef64482740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bdd9e0 0x556677bdfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.349+0000 7fef64482740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556677b3d250 con 0x556677bd7e80 2024-03-21T11:44:26.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.349+0000 7fef64482740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556677ae5a30 con 0x556677bd8870 2024-03-21T11:44:26.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.349+0000 7fef64482740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556677b392f0 con 0x556677bdd9e0 2024-03-21T11:44:26.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.349+0000 7fef53400640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef400a30a0 0x7fef400a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.349+0000 7fef53400640 1 -- 172.21.15.47:0/1009787931 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fef4c0cf680 con 0x7fef5c076060 2024-03-21T11:44:26.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.349+0000 7fef5a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bdd9e0 0x556677bdfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.349+0000 7fef5b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd7e80 0x556677bd82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.349+0000 7fef5a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bdd9e0 0x556677bdfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44954/0 (socket says 172.21.15.47:44954) 2024-03-21T11:44:26.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.349+0000 7fef5a000640 1 -- 172.21.15.47:0/3909964586 learned_addr learned my addr 172.21.15.47:0/3909964586 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:26.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.350+0000 7fef5a000640 1 -- 172.21.15.47:0/3909964586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bd8870 msgr2=0x556677bdd370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:44:26.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.350+0000 7fef5a000640 1 --2- 172.21.15.47:0/3909964586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bd8870 0x556677bdd370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.350+0000 7fef5a000640 1 -- 172.21.15.47:0/3909964586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd7e80 msgr2=0x556677bd82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.350+0000 7fef5a000640 1 --2- 172.21.15.47:0/3909964586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd7e80 0x556677bd82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.350+0000 7fef5a000640 1 -- 172.21.15.47:0/3909964586 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556677ad0690 con 0x556677bdd9e0 2024-03-21T11:44:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.350+0000 7fef5b400640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef400a30a0 0x7fef400a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.350+0000 7fef5a000640 1 --2- 172.21.15.47:0/3909964586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bdd9e0 0x556677bdfe70 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fef4800b740 tx=0x7fef480045f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e395eb1300a46813 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:26.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.351+0000 7fef47e00640 1 -- 172.21.15.47:0/3909964586 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48003bd0 con 0x556677bdd9e0 2024-03-21T11:44:26.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.351+0000 7fef47e00640 1 -- 172.21.15.47:0/3909964586 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef48015070 con 0x556677bdd9e0 2024-03-21T11:44:26.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.351+0000 7fef47e00640 1 -- 172.21.15.47:0/3909964586 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48011730 con 0x556677bdd9e0 2024-03-21T11:44:26.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.352+0000 7fef64482740 1 -- 172.21.15.47:0/3909964586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bdd9e0 msgr2=0x556677bdfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:26.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.352+0000 7fef64482740 1 --2- 172.21.15.47:0/3909964586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bdd9e0 0x556677bdfe70 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fef4800b740 tx=0x7fef480045f0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.356+0000 7fef5b400640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef400a30a0 0x7fef400a5560 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fef54036040 tx=0x7fef5402bf10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:26.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.356+0000 7fef64482740 1 -- 172.21.15.47:0/3909964586 shutdown_connections 2024-03-21T11:44:26.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.356+0000 7fef64482740 1 --2- 172.21.15.47:0/3909964586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bdd9e0 0x556677bdfe70 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.356+0000 7fef64482740 1 --2- 172.21.15.47:0/3909964586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bd8870 0x556677bdd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.356+0000 7fef64482740 1 --2- 172.21.15.47:0/3909964586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd7e80 0x556677bd82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.356+0000 7fef64482740 1 -- 172.21.15.47:0/3909964586 >> 172.21.15.47:0/3909964586 conn(0x556677ba0200 msgr2=0x556677bd37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:26.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.357+0000 7fef64482740 1 -- 172.21.15.47:0/3909964586 shutdown_connections 2024-03-21T11:44:26.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.357+0000 7fef64482740 1 -- 172.21.15.47:0/3909964586 wait complete. 2024-03-21T11:44:26.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.357+0000 7fef64482740 1 -- start start 2024-03-21T11:44:26.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.357+0000 7fef64482740 1 Processor -- start 2024-03-21T11:44:26.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.358+0000 7fef64482740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.359+0000 7fef64482740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 0x556677be2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.359+0000 7fef64482740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 0x556677be9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.359+0000 7fef5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.359+0000 7fef5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44962/0 (socket says 172.21.15.47:44962) 2024-03-21T11:44:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.359+0000 7fef5aa00640 1 -- 172.21.15.47:0/1521354378 learned_addr learned my addr 172.21.15.47:0/1521354378 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.359+0000 7fef64482740 1 -- 172.21.15.47:0/1521354378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556677be0990 con 0x556677bd8870 2024-03-21T11:44:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.359+0000 7fef64482740 1 -- 172.21.15.47:0/1521354378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556677af8e80 con 0x556677bdd9e0 2024-03-21T11:44:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.359+0000 7fef64482740 1 -- 172.21.15.47:0/1521354378 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556677be0b10 con 0x556677bd7e80 2024-03-21T11:44:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.359+0000 7fef64482740 10 client.0 _ll_register_callbacks cb 0x556677b0b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.359+0000 7fef5b400640 1 --2- 172.21.15.47:0/1521354378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 0x556677be9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.360+0000 7fef5aa00640 1 -- 172.21.15.47:0/1521354378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 msgr2=0x556677be9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.360+0000 7fef5aa00640 1 --2- 172.21.15.47:0/1521354378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 0x556677be9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.360+0000 7fef5aa00640 1 -- 172.21.15.47:0/1521354378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 msgr2=0x556677be2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.360+0000 7fef5aa00640 1 --2- 172.21.15.47:0/1521354378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 0x556677be2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.360+0000 7fef5aa00640 1 -- 172.21.15.47:0/1521354378 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55667791b7d0 con 0x556677bd7e80 2024-03-21T11:44:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.360+0000 7fef5aa00640 1 --2- 172.21.15.47:0/1521354378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be1e00 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fef4c097e80 tx=0x7fef4c09c1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86f25a258631954e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.360+0000 7fef44c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.361+0000 7fef44c00640 1 -- 172.21.15.47:0/1521354378 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c0a4430 con 0x556677bd7e80 2024-03-21T11:44:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.361+0000 7fef44c00640 1 -- 172.21.15.47:0/1521354378 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fef4c00fac0 con 0x556677bd7e80 2024-03-21T11:44:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.361+0000 7fef64482740 1 -- 172.21.15.47:0/1521354378 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef48015070 con 0x556677bd7e80 2024-03-21T11:44:26.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.363+0000 7fef44c00640 1 -- 172.21.15.47:0/1521354378 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef4c08b9e0 con 0x556677bd7e80 2024-03-21T11:44:26.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.363+0000 7fef64482740 10 client.32311 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:26.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.363+0000 7fef64482740 1 -- 172.21.15.47:0/1521354378 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5566778513f0 con 0x556677bd7e80 2024-03-21T11:44:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.363+0000 7fef44c00640 1 -- 172.21.15.47:0/1521354378 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef48015070 con 0x556677bd7e80 2024-03-21T11:44:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.363+0000 7fef44c00640 1 -- 172.21.15.47:0/1521354378 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef4c0a3fa0 con 0x556677bd7e80 2024-03-21T11:44:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.363+0000 7fef64482740 10 client.32311 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:26.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.363+0000 7fef64482740 10 client.32311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:26.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.363+0000 7fef64482740 20 client.32311 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:26.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.363+0000 7fef64482740 1 --2- 172.21.15.47:0/1521354378 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556677bedb10 0x556677beff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.363+0000 7fef64482740 4 client.32311 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:26.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.363+0000 7fef64482740 1 -- 172.21.15.47:0/1521354378 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556677b4a790 con 0x556677bedb10 2024-03-21T11:44:26.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.365+0000 7fef5a000640 1 --2- 172.21.15.47:0/1521354378 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556677bedb10 0x556677beff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.366+0000 7fef5a000640 1 --2- 172.21.15.47:0/1521354378 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556677bedb10 0x556677beff90 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e8341bcdc97155c server_cookie=70400e1eb9bf6bb3 in_seq=0 out_seq=0 2024-03-21T11:44:26.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.366+0000 7fef44c00640 10 client.32311 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:26.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.367+0000 7fef44c00640 1 -- 172.21.15.47:0/1521354378 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556677b4a790 con 0x556677bedb10 2024-03-21T11:44:26.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.367+0000 7fef44c00640 10 client.32311 handle_command_reply: tid=0 2024-03-21T11:44:26.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.367+0000 7fef64482740 1 client.32311 shutdown 2024-03-21T11:44:26.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.368+0000 7fef64482740 1 -- 172.21.15.47:0/1521354378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 msgr2=0x556677be1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:26.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.368+0000 7fef64482740 1 --2- 172.21.15.47:0/1521354378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be1e00 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fef4c097e80 tx=0x7fef4c09c1e0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.368+0000 7fef64482740 1 -- 172.21.15.47:0/1521354378 shutdown_connections 2024-03-21T11:44:26.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.368+0000 7fef64482740 1 --2- 172.21.15.47:0/1521354378 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556677bedb10 0x556677beff90 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.368+0000 7fef64482740 1 --2- 172.21.15.47:0/1521354378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 0x556677be9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.368+0000 7fef64482740 1 --2- 172.21.15.47:0/1521354378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 0x556677be2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.368+0000 7fef64482740 1 --2- 172.21.15.47:0/1521354378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be1e00 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.368+0000 7fef64482740 1 -- 172.21.15.47:0/1521354378 >> 172.21.15.47:0/1521354378 conn(0x556677ba0200 msgr2=0x556677bd2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.368+0000 7fef44c00640 0 client.32311 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.368+0000 7fef64482740 1 -- 172.21.15.47:0/1521354378 shutdown_connections 2024-03-21T11:44:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.368+0000 7fef64482740 1 -- 172.21.15.47:0/1521354378 wait complete. 2024-03-21T11:44:26.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.368+0000 7fef64482740 20 client.32311 trim_cache size 0 max 16384 2024-03-21T11:44:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.388+0000 7fef64482740 1 Processor -- start 2024-03-21T11:44:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.388+0000 7fef64482740 1 -- start start 2024-03-21T11:44:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.388+0000 7fef64482740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.388+0000 7fef64482740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 0x556677be9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.389+0000 7fef64482740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 0x556677bd99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.389+0000 7fef64482740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c1b0590 con 0x556677bd8870 2024-03-21T11:44:26.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.389+0000 7fef64482740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c1b1f10 con 0x556677bdd9e0 2024-03-21T11:44:26.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.389+0000 7fef64482740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c1b0bf0 con 0x556677bd7e80 2024-03-21T11:44:26.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.389+0000 7fef5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.389+0000 7fef5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44978/0 (socket says 172.21.15.47:44978) 2024-03-21T11:44:26.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.389+0000 7fef5aa00640 1 -- 172.21.15.47:0/1491853466 learned_addr learned my addr 172.21.15.47:0/1491853466 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:26.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.389+0000 7fef5b400640 1 --2- 172.21.15.47:0/1491853466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 0x556677bd99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.389+0000 7fef5aa00640 1 -- 172.21.15.47:0/1491853466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 msgr2=0x556677bd99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:26.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.389+0000 7fef5aa00640 1 --2- 172.21.15.47:0/1491853466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 0x556677bd99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.389+0000 7fef5aa00640 1 -- 172.21.15.47:0/1491853466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 msgr2=0x556677be9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:26.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.389+0000 7fef5aa00640 1 --2- 172.21.15.47:0/1491853466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 0x556677be9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.389+0000 7fef5aa00640 1 -- 172.21.15.47:0/1491853466 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566778110f0 con 0x556677bd7e80 2024-03-21T11:44:26.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.390+0000 7fef5aa00640 1 --2- 172.21.15.47:0/1491853466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be22a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00fe70 tx=0x7fef4c09fcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31ea325ec579e2f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:26.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.390+0000 7fef47400640 1 -- 172.21.15.47:0/1491853466 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c0b3070 con 0x556677bd7e80 2024-03-21T11:44:26.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.390+0000 7fef47400640 1 -- 172.21.15.47:0/1491853466 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef4c08b040 con 0x556677bd7e80 2024-03-21T11:44:26.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.390+0000 7fef47400640 1 -- 172.21.15.47:0/1491853466 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c086220 con 0x556677bd7e80 2024-03-21T11:44:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.391+0000 7fef64482740 1 -- 172.21.15.47:0/1491853466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 msgr2=0x556677be22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.391+0000 7fef64482740 1 --2- 172.21.15.47:0/1491853466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be22a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00fe70 tx=0x7fef4c09fcf0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.391+0000 7fef64482740 1 -- 172.21.15.47:0/1491853466 shutdown_connections 2024-03-21T11:44:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.391+0000 7fef64482740 1 --2- 172.21.15.47:0/1491853466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 0x556677bd99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.391+0000 7fef64482740 1 --2- 172.21.15.47:0/1491853466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 0x556677be9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.391+0000 7fef64482740 1 --2- 172.21.15.47:0/1491853466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be22a0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.391+0000 7fef64482740 1 -- 172.21.15.47:0/1491853466 >> 172.21.15.47:0/1491853466 conn(0x556677ba0200 msgr2=0x556677bd92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.391+0000 7fef64482740 1 -- 172.21.15.47:0/1491853466 shutdown_connections 2024-03-21T11:44:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.391+0000 7fef64482740 1 -- 172.21.15.47:0/1491853466 wait complete. 2024-03-21T11:44:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.392+0000 7fef64482740 1 -- start start 2024-03-21T11:44:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.392+0000 7fef64482740 1 Processor -- start 2024-03-21T11:44:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.392+0000 7fef64482740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.392+0000 7fef64482740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 0x556677be7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.392+0000 7fef64482740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 0x556677be8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.392+0000 7fef64482740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c1b18b0 con 0x556677bd8870 2024-03-21T11:44:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.392+0000 7fef64482740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c1b1250 con 0x556677bdd9e0 2024-03-21T11:44:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.392+0000 7fef64482740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c1ae750 con 0x556677bd7e80 2024-03-21T11:44:26.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.392+0000 7fef64482740 10 client.0 _ll_register_callbacks cb 0x7fef4c00fdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.393+0000 7fef5a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 0x556677be7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.393+0000 7fef5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.393+0000 7fef5b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 0x556677be8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.393+0000 7fef5aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:44990/0 (socket says 172.21.15.47:44990) 2024-03-21T11:44:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.393+0000 7fef5aa00640 1 -- 172.21.15.47:0/143195162 learned_addr learned my addr 172.21.15.47:0/143195162 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:26.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.393+0000 7fef5a000640 1 --2- 172.21.15.47:0/143195162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 0x556677be7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40362/0 (socket says 172.21.15.47:40362) 2024-03-21T11:44:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.393+0000 7fef5aa00640 1 -- 172.21.15.47:0/143195162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 msgr2=0x556677be8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.393+0000 7fef5aa00640 1 --2- 172.21.15.47:0/143195162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 0x556677be8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.393+0000 7fef5aa00640 1 -- 172.21.15.47:0/143195162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 msgr2=0x556677be7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.393+0000 7fef5aa00640 1 --2- 172.21.15.47:0/143195162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 0x556677be7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.393+0000 7fef5aa00640 1 -- 172.21.15.47:0/143195162 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556677be8a80 con 0x556677bd7e80 2024-03-21T11:44:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.393+0000 7fef5a000640 1 --2- 172.21.15.47:0/143195162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 0x556677be7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.394+0000 7fef5aa00640 1 --2- 172.21.15.47:0/143195162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be6f40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00ff00 tx=0x7fef4c09c260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=270b024eeb5dc32f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.394+0000 7fef44c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.394+0000 7fef44c00640 1 -- 172.21.15.47:0/143195162 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c07e030 con 0x556677bd7e80 2024-03-21T11:44:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.394+0000 7fef64482740 1 -- 172.21.15.47:0/143195162 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef4c08b040 con 0x556677bd7e80 2024-03-21T11:44:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.394+0000 7fef44c00640 1 -- 172.21.15.47:0/143195162 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fef4c086220 con 0x556677bd7e80 2024-03-21T11:44:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.394+0000 7fef44c00640 1 -- 172.21.15.47:0/143195162 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef4c093050 con 0x556677bd7e80 2024-03-21T11:44:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.394+0000 7fef64482740 10 client.32317 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:26.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.394+0000 7fef64482740 1 -- 172.21.15.47:0/143195162 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556677be8a80 con 0x556677bd7e80 2024-03-21T11:44:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.394+0000 7fef44c00640 1 -- 172.21.15.47:0/143195162 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef4c08b040 con 0x556677bd7e80 2024-03-21T11:44:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.395+0000 7fef44c00640 1 -- 172.21.15.47:0/143195162 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef4c09ae70 con 0x556677bd7e80 2024-03-21T11:44:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.395+0000 7fef64482740 10 client.32317 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.395+0000 7fef64482740 10 client.32317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.395+0000 7fef64482740 20 client.32317 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.395+0000 7fef64482740 1 --2- 172.21.15.47:0/143195162 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556677c49e30 0x556677c4c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.395+0000 7fef64482740 4 client.32317 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:44:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.395+0000 7fef64482740 1 -- 172.21.15.47:0/143195162 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556677b44ae0 con 0x556677c49e30 2024-03-21T11:44:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.395+0000 7fef5a000640 1 --2- 172.21.15.47:0/143195162 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556677c49e30 0x556677c4c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef5a000640 1 --2- 172.21.15.47:0/143195162 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556677c49e30 0x556677c4c220 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a07cb39bb3f72f42 server_cookie=be3243b533304340 in_seq=0 out_seq=0 2024-03-21T11:44:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef44c00640 10 client.32317 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef44c00640 1 -- 172.21.15.47:0/143195162 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x7fef48015120 con 0x556677c49e30 2024-03-21T11:44:26.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef44c00640 10 client.32317 handle_command_reply: tid=0 2024-03-21T11:44:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef64482740 1 client.32317 shutdown 2024-03-21T11:44:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef64482740 1 -- 172.21.15.47:0/143195162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 msgr2=0x556677be6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef64482740 1 --2- 172.21.15.47:0/143195162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be6f40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00ff00 tx=0x7fef4c09c260 comp rx=0 tx=0).stop 2024-03-21T11:44:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef64482740 1 -- 172.21.15.47:0/143195162 shutdown_connections 2024-03-21T11:44:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef64482740 1 --2- 172.21.15.47:0/143195162 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556677c49e30 0x556677c4c220 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef64482740 1 --2- 172.21.15.47:0/143195162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556677bdd9e0 0x556677be8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef64482740 1 --2- 172.21.15.47:0/143195162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556677bd8870 0x556677be7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef64482740 1 --2- 172.21.15.47:0/143195162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556677bd7e80 0x556677be6f40 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef44c00640 0 client.32317 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef64482740 1 -- 172.21.15.47:0/143195162 >> 172.21.15.47:0/143195162 conn(0x556677ba0200 msgr2=0x556677bd9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef64482740 1 -- 172.21.15.47:0/143195162 shutdown_connections 2024-03-21T11:44:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef64482740 1 -- 172.21.15.47:0/143195162 wait complete. 2024-03-21T11:44:26.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.396+0000 7fef64482740 20 client.32317 trim_cache size 0 max 16384 2024-03-21T11:44:26.401 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:44:26.401 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (47 inodes in the stack)", 2024-03-21T11:44:26.401 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:44:26.401 INFO:teuthology.orchestra.run.smithi047.stdout: "da94a7ed-c009-4cbe-a58b-8049ef2b9467": { 2024-03-21T11:44:26.401 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:44:26.401 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "da94a7ed-c009-4cbe-a58b-8049ef2b9467", 2024-03-21T11:44:26.401 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:44:26.401 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:44:26.401 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:44:26.401 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:44:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.397+0000 7fef61800640 1 -- 172.21.15.47:0/1009787931 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef400a30a0 msgr2=0x7fef400a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.397+0000 7fef61800640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef400a30a0 0x7fef400a5560 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fef54036040 tx=0x7fef5402bf10 comp rx=0 tx=0).stop 2024-03-21T11:44:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.397+0000 7fef61800640 1 -- 172.21.15.47:0/1009787931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5c076060 msgr2=0x7fef5c19deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.397+0000 7fef61800640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5c076060 0x7fef5c19deb0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00c920 tx=0x7fef4c00cde0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.397+0000 7fef61800640 1 -- 172.21.15.47:0/1009787931 shutdown_connections 2024-03-21T11:44:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.397+0000 7fef61800640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fef400a30a0 0x7fef400a5560 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.397+0000 7fef61800640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef5c077c40 0x7fef5c1a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.397+0000 7fef61800640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef5c0772a0 0x7fef5c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.397+0000 7fef61800640 1 --2- 172.21.15.47:0/1009787931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef5c076060 0x7fef5c19deb0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.397+0000 7fef61800640 1 -- 172.21.15.47:0/1009787931 >> 172.21.15.47:0/1009787931 conn(0x7fef5c0fee50 msgr2=0x7fef5c100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.397+0000 7fef61800640 1 -- 172.21.15.47:0/1009787931 shutdown_connections 2024-03-21T11:44:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:26.397+0000 7fef61800640 1 -- 172.21.15.47:0/1009787931 wait complete. 2024-03-21T11:44:26.415 INFO:tasks.cephfs.filesystem:scrub status for tag:da94a7ed-c009-4cbe-a58b-8049ef2b9467 - {'path': '/', 'tag': 'da94a7ed-c009-4cbe-a58b-8049ef2b9467', 'options': 'recursive,force'} 2024-03-21T11:44:26.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 189160 22.74 MB/sec execute 310 sec latency 141.603 ms 2024-03-21T11:44:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: pgmap v1033: 129 pgs: 129 active+clean; 610 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.3 MiB/s wr, 578 op/s 2024-03-21T11:44:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:44:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.144 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:44:27.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:27.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:27.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: pgmap v1033: 129 pgs: 129 active+clean; 610 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.3 MiB/s wr, 578 op/s 2024-03-21T11:44:27.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:27.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:27.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:44:27.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:44:27.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:44:27.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:44:27.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:44:27.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:44:27.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:44:27.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:44:27.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:44:27.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:44:27.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.243 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.244 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:27.244 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:27.244 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: pgmap v1033: 129 pgs: 129 active+clean; 610 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.3 MiB/s wr, 578 op/s 2024-03-21T11:44:27.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:27.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:27.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:44:27.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:44:27.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:44:27.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:44:27.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:44:27.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:44:27.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:44:27.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:44:27.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:44:27.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:44:27.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.318 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.318 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:27.318 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:27.318 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:44:27.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 189605 22.75 MB/sec execute 311 sec latency 140.281 ms 2024-03-21T11:44:28.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 189839 22.72 MB/sec execute 312 sec latency 534.199 ms 2024-03-21T11:44:29.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:28 smithi100 ceph-mon[36245]: pgmap v1034: 129 pgs: 129 active+clean; 612 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 569 op/s 2024-03-21T11:44:29.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:28 smithi102 ceph-mon[36748]: pgmap v1034: 129 pgs: 129 active+clean; 612 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 569 op/s 2024-03-21T11:44:29.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:28 smithi047 ceph-mon[31351]: pgmap v1034: 129 pgs: 129 active+clean; 612 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 569 op/s 2024-03-21T11:44:29.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 190267 22.70 MB/sec execute 313 sec latency 154.912 ms 2024-03-21T11:44:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:30.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:29 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:30.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:30.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:29 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:30.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:30.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:29 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:30.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 190696 22.69 MB/sec execute 314 sec latency 100.676 ms 2024-03-21T11:44:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:30 smithi102 ceph-mon[36748]: pgmap v1035: 129 pgs: 129 active+clean; 612 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.1 MiB/s wr, 652 op/s 2024-03-21T11:44:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:30 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:30 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:30 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:44:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:30 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:44:31.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:30 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:30 smithi047 ceph-mon[31351]: pgmap v1035: 129 pgs: 129 active+clean; 612 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.1 MiB/s wr, 652 op/s 2024-03-21T11:44:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:30 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:30 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:30 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:44:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:30 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:44:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:30 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:30 smithi100 ceph-mon[36245]: pgmap v1035: 129 pgs: 129 active+clean; 612 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.1 MiB/s wr, 652 op/s 2024-03-21T11:44:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:30 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:30 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:30 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:44:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:30 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:44:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:30 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:44:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.906+0000 7f2047000640 1 -- 172.21.15.47:0/473159577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2040104ae0 msgr2=0x7f2040104f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.906+0000 7f2047000640 1 --2- 172.21.15.47:0/473159577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2040104ae0 0x7f2040104f60 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f202c009900 tx=0x7f202c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:44:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.906+0000 7f2047000640 1 -- 172.21.15.47:0/473159577 shutdown_connections 2024-03-21T11:44:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.906+0000 7f2047000640 1 --2- 172.21.15.47:0/473159577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20401054a0 0x7f204010bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.906+0000 7f2047000640 1 --2- 172.21.15.47:0/473159577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2040104ae0 0x7f2040104f60 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.906+0000 7f2047000640 1 --2- 172.21.15.47:0/473159577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20401038f0 0x7f2040103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.906+0000 7f2047000640 1 -- 172.21.15.47:0/473159577 >> 172.21.15.47:0/473159577 conn(0x7f20400fee60 msgr2=0x7f20401012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.906+0000 7f2047000640 1 -- 172.21.15.47:0/473159577 shutdown_connections 2024-03-21T11:44:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.906+0000 7f2047000640 1 -- 172.21.15.47:0/473159577 wait complete. 2024-03-21T11:44:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.908+0000 7f2047000640 1 Processor -- start 2024-03-21T11:44:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.908+0000 7f2047000640 1 -- start start 2024-03-21T11:44:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.909+0000 7f2047000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20401038f0 0x7f204019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.909+0000 7f2047000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2040104ae0 0x7f204019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.909+0000 7f2047000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20401054a0 0x7f20401a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.909+0000 7f2047000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204010f570 con 0x7f20401054a0 2024-03-21T11:44:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.909+0000 7f2047000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204010f3f0 con 0x7f20401038f0 2024-03-21T11:44:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.909+0000 7f2047000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204010f6f0 con 0x7f2040104ae0 2024-03-21T11:44:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.909+0000 7f203fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20401038f0 0x7f204019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.909+0000 7f203fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20401038f0 0x7f204019df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60572/0 (socket says 172.21.15.47:60572) 2024-03-21T11:44:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.909+0000 7f203fe00640 1 -- 172.21.15.47:0/632800734 learned_addr learned my addr 172.21.15.47:0/632800734 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.909+0000 7f203f400640 1 --2- 172.21.15.47:0/632800734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2040104ae0 0x7f204019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.910+0000 7f2045000640 1 --2- 172.21.15.47:0/632800734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20401054a0 0x7f20401a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.910+0000 7f203f400640 1 -- 172.21.15.47:0/632800734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20401038f0 msgr2=0x7f204019df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.910+0000 7f203f400640 1 --2- 172.21.15.47:0/632800734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20401038f0 0x7f204019df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.910+0000 7f203f400640 1 -- 172.21.15.47:0/632800734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20401054a0 msgr2=0x7f20401a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.910+0000 7f203f400640 1 --2- 172.21.15.47:0/632800734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20401054a0 0x7f20401a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.910+0000 7f203f400640 1 -- 172.21.15.47:0/632800734 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20401a5b70 con 0x7f2040104ae0 2024-03-21T11:44:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.911+0000 7f203f400640 1 --2- 172.21.15.47:0/632800734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2040104ae0 0x7f204019e490 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f202c009230 tx=0x7f202c02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.911+0000 7f203cc00640 1 -- 172.21.15.47:0/632800734 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c041070 con 0x7f2040104ae0 2024-03-21T11:44:31.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.911+0000 7f2047000640 1 -- 172.21.15.47:0/632800734 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20401a5e00 con 0x7f2040104ae0 2024-03-21T11:44:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.911+0000 7f203cc00640 1 -- 172.21.15.47:0/632800734 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f202c004450 con 0x7f2040104ae0 2024-03-21T11:44:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.911+0000 7f203cc00640 1 -- 172.21.15.47:0/632800734 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c003df0 con 0x7f2040104ae0 2024-03-21T11:44:31.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.912+0000 7f2047000640 1 -- 172.21.15.47:0/632800734 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20401a6270 con 0x7f2040104ae0 2024-03-21T11:44:31.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.920+0000 7f203cc00640 1 -- 172.21.15.47:0/632800734 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f202c044070 con 0x7f2040104ae0 2024-03-21T11:44:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.922+0000 7f203cc00640 1 --2- 172.21.15.47:0/632800734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f20240a2fc0 0x7f20240a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.923+0000 7f2049dc4740 1 Processor -- start 2024-03-21T11:44:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.923+0000 7f2049dc4740 1 -- start start 2024-03-21T11:44:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.923+0000 7f203cc00640 1 -- 172.21.15.47:0/632800734 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f202c0f0900 con 0x7f2040104ae0 2024-03-21T11:44:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.924+0000 7f2049dc4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479ce1e0 0x55c8479ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.924+0000 7f2049dc4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479cebd0 0x55c8479d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.924+0000 7f2049dc4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479d3d40 0x55c8479d61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.924+0000 7f2049dc4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8479913b0 con 0x55c8479cebd0 2024-03-21T11:44:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.924+0000 7f2049dc4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c847933830 con 0x55c8479d3d40 2024-03-21T11:44:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.924+0000 7f2049dc4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8478dc5d0 con 0x55c8479ce1e0 2024-03-21T11:44:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.924+0000 7f2045000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479d3d40 0x55c8479d61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.924+0000 7f2045000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479d3d40 0x55c8479d61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60582/0 (socket says 172.21.15.47:60582) 2024-03-21T11:44:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.924+0000 7f2045000640 1 -- 172.21.15.47:0/748114423 learned_addr learned my addr 172.21.15.47:0/748114423 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.924+0000 7f203f400640 1 --2- 172.21.15.47:0/748114423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479cebd0 0x55c8479d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.924+0000 7f203fe00640 1 --2- 172.21.15.47:0/632800734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f20240a2fc0 0x7f20240a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.925+0000 7f2045000640 1 -- 172.21.15.47:0/748114423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479ce1e0 msgr2=0x55c8479ce600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:44:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.925+0000 7f2045000640 1 --2- 172.21.15.47:0/748114423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479ce1e0 0x55c8479ce600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.925+0000 7f2045000640 1 -- 172.21.15.47:0/748114423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479cebd0 msgr2=0x55c8479d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.925+0000 7f2045000640 1 --2- 172.21.15.47:0/748114423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479cebd0 0x55c8479d36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.925+0000 7f2045000640 1 -- 172.21.15.47:0/748114423 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8478f42c0 con 0x55c8479d3d40 2024-03-21T11:44:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.925+0000 7f2045000640 1 --2- 172.21.15.47:0/748114423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479d3d40 0x55c8479d61d0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f2038009fd0 tx=0x7f20380118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=619e811fd3e61dfb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.925+0000 7f2035600640 1 -- 172.21.15.47:0/748114423 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038004d20 con 0x55c8479d3d40 2024-03-21T11:44:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.926+0000 7f2035600640 1 -- 172.21.15.47:0/748114423 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2038004050 con 0x55c8479d3d40 2024-03-21T11:44:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.926+0000 7f2035600640 1 -- 172.21.15.47:0/748114423 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038005260 con 0x55c8479d3d40 2024-03-21T11:44:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.926+0000 7f203fe00640 1 --2- 172.21.15.47:0/632800734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f20240a2fc0 0x7f20240a5480 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x55c8479cf430 tx=0x7f2030005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.927+0000 7f2049dc4740 1 -- 172.21.15.47:0/748114423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479d3d40 msgr2=0x55c8479d61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.927+0000 7f2049dc4740 1 --2- 172.21.15.47:0/748114423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479d3d40 0x55c8479d61d0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f2038009fd0 tx=0x7f20380118d0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.927+0000 7f2049dc4740 1 -- 172.21.15.47:0/748114423 shutdown_connections 2024-03-21T11:44:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.927+0000 7f2049dc4740 1 --2- 172.21.15.47:0/748114423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479d3d40 0x55c8479d61d0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.927+0000 7f2049dc4740 1 --2- 172.21.15.47:0/748114423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479cebd0 0x55c8479d36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.927+0000 7f2049dc4740 1 --2- 172.21.15.47:0/748114423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479ce1e0 0x55c8479ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.927+0000 7f2049dc4740 1 -- 172.21.15.47:0/748114423 >> 172.21.15.47:0/748114423 conn(0x55c8479c9750 msgr2=0x55c8479c9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.928+0000 7f2049dc4740 1 -- 172.21.15.47:0/748114423 shutdown_connections 2024-03-21T11:44:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.928+0000 7f2049dc4740 1 -- 172.21.15.47:0/748114423 wait complete. 2024-03-21T11:44:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.929+0000 7f2049dc4740 1 -- start start 2024-03-21T11:44:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.929+0000 7f2049dc4740 1 Processor -- start 2024-03-21T11:44:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.930+0000 7f2049dc4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479e9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.930+0000 7f2049dc4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479cebd0 0x55c8479e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.930+0000 7f2049dc4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479d3d40 0x55c8479f08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.930+0000 7f2049dc4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8478efa20 con 0x55c8479ce1e0 2024-03-21T11:44:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.930+0000 7f2049dc4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84792f8d0 con 0x55c8479cebd0 2024-03-21T11:44:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.930+0000 7f2049dc4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8479e7d00 con 0x55c8479d3d40 2024-03-21T11:44:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.931+0000 7f2049dc4740 10 client.0 _ll_register_callbacks cb 0x55c84792c1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.931+0000 7f2045000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479d3d40 0x55c8479f08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.931+0000 7f2045000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479d3d40 0x55c8479f08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:40940/0 (socket says 172.21.15.47:40940) 2024-03-21T11:44:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.931+0000 7f2045000640 1 -- 172.21.15.47:0/158083054 learned_addr learned my addr 172.21.15.47:0/158083054 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.931+0000 7f203f400640 1 --2- 172.21.15.47:0/158083054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479cebd0 0x55c8479e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.931+0000 7f203fe00640 1 --2- 172.21.15.47:0/158083054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479e9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.932+0000 7f2045000640 1 -- 172.21.15.47:0/158083054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479cebd0 msgr2=0x55c8479e98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.932+0000 7f2045000640 1 --2- 172.21.15.47:0/158083054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479cebd0 0x55c8479e98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.932+0000 7f2045000640 1 -- 172.21.15.47:0/158083054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 msgr2=0x55c8479e9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.932+0000 7f2045000640 1 --2- 172.21.15.47:0/158083054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479e9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.932+0000 7f2045000640 1 -- 172.21.15.47:0/158083054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c847648210 con 0x55c8479d3d40 2024-03-21T11:44:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.932+0000 7f203fe00640 1 --2- 172.21.15.47:0/158083054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479e9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:44:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.933+0000 7f2045000640 1 --2- 172.21.15.47:0/158083054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479d3d40 0x55c8479f08f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f203801b040 tx=0x7f2038004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51f7bd1577600364 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.933+0000 7f202a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.933+0000 7f202a000640 1 -- 172.21.15.47:0/158083054 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038028020 con 0x55c8479d3d40 2024-03-21T11:44:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.934+0000 7f202a000640 1 -- 172.21.15.47:0/158083054 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2038004410 con 0x55c8479d3d40 2024-03-21T11:44:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.934+0000 7f2049dc4740 1 -- 172.21.15.47:0/158083054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2038004050 con 0x55c8479d3d40 2024-03-21T11:44:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.934+0000 7f202a000640 1 -- 172.21.15.47:0/158083054 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2038033050 con 0x55c8479d3d40 2024-03-21T11:44:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.934+0000 7f2049dc4740 10 client.32326 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.934+0000 7f2049dc4740 1 -- 172.21.15.47:0/158083054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c847714620 con 0x55c8479d3d40 2024-03-21T11:44:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.934+0000 7f202a000640 1 -- 172.21.15.47:0/158083054 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2038004050 con 0x55c8479d3d40 2024-03-21T11:44:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.935+0000 7f202a000640 1 -- 172.21.15.47:0/158083054 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f203800f4e0 con 0x55c8479d3d40 2024-03-21T11:44:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.935+0000 7f2049dc4740 10 client.32326 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.935+0000 7f2049dc4740 10 client.32326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.935+0000 7f2049dc4740 20 client.32326 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.935+0000 7f2049dc4740 1 --2- 172.21.15.47:0/158083054 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8479f4fb0 0x55c8479f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.935+0000 7f2049dc4740 4 client.32326 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.935+0000 7f2049dc4740 1 -- 172.21.15.47:0/158083054 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c847940d70 con 0x55c8479f4fb0 2024-03-21T11:44:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.936+0000 7f203fe00640 1 --2- 172.21.15.47:0/158083054 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8479f4fb0 0x55c8479f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.937+0000 7f203fe00640 1 --2- 172.21.15.47:0/158083054 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8479f4fb0 0x55c8479f73a0 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b479747ad1b962de server_cookie=65c6129646c5d1ff in_seq=0 out_seq=0 2024-03-21T11:44:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.937+0000 7f202a000640 10 client.32326 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.939+0000 7f202a000640 1 -- 172.21.15.47:0/158083054 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c847940d70 con 0x55c8479f4fb0 2024-03-21T11:44:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.939+0000 7f202a000640 10 client.32326 handle_command_reply: tid=0 2024-03-21T11:44:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.939+0000 7f2049dc4740 1 client.32326 shutdown 2024-03-21T11:44:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.939+0000 7f2049dc4740 1 -- 172.21.15.47:0/158083054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479d3d40 msgr2=0x55c8479f08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.939+0000 7f2049dc4740 1 --2- 172.21.15.47:0/158083054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479d3d40 0x55c8479f08f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f203801b040 tx=0x7f2038004db0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.940+0000 7f2049dc4740 1 -- 172.21.15.47:0/158083054 shutdown_connections 2024-03-21T11:44:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.940+0000 7f2049dc4740 1 --2- 172.21.15.47:0/158083054 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8479f4fb0 0x55c8479f73a0 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.940+0000 7f2049dc4740 1 --2- 172.21.15.47:0/158083054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479d3d40 0x55c8479f08f0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.940+0000 7f2049dc4740 1 --2- 172.21.15.47:0/158083054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479cebd0 0x55c8479e98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.940+0000 7f2049dc4740 1 --2- 172.21.15.47:0/158083054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479e9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.940+0000 7f2049dc4740 1 -- 172.21.15.47:0/158083054 >> 172.21.15.47:0/158083054 conn(0x55c8479c9750 msgr2=0x55c8479c8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.940+0000 7f202a000640 0 client.32326 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.941+0000 7f2049dc4740 1 -- 172.21.15.47:0/158083054 shutdown_connections 2024-03-21T11:44:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.941+0000 7f2049dc4740 1 -- 172.21.15.47:0/158083054 wait complete. 2024-03-21T11:44:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.942+0000 7f2049dc4740 20 client.32326 trim_cache size 0 max 16384 2024-03-21T11:44:31.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 191287 22.74 MB/sec execute 315 sec latency 49.506 ms 2024-03-21T11:44:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.977+0000 7f2049dc4740 1 Processor -- start 2024-03-21T11:44:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.977+0000 7f2049dc4740 1 -- start start 2024-03-21T11:44:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.978+0000 7f2049dc4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479c9750 0x55c8479d3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.978+0000 7f2049dc4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479f08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.979+0000 7f2049dc4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479cebd0 0x55c8479e9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.979+0000 7f2049dc4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20401a7e10 con 0x55c8479ce1e0 2024-03-21T11:44:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.979+0000 7f2049dc4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20401a9790 con 0x55c8479c9750 2024-03-21T11:44:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.979+0000 7f2049dc4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20401a8470 con 0x55c8479cebd0 2024-03-21T11:44:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.979+0000 7f203fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479c9750 0x55c8479d3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.979+0000 7f203fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479c9750 0x55c8479d3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60608/0 (socket says 172.21.15.47:60608) 2024-03-21T11:44:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.979+0000 7f203fe00640 1 -- 172.21.15.47:0/2471317616 learned_addr learned my addr 172.21.15.47:0/2471317616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.979+0000 7f203f400640 1 --2- 172.21.15.47:0/2471317616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479f08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.980+0000 7f2045000640 1 --2- 172.21.15.47:0/2471317616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479cebd0 0x55c8479e9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.981+0000 7f203fe00640 1 -- 172.21.15.47:0/2471317616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479cebd0 msgr2=0x55c8479e9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.981+0000 7f203fe00640 1 --2- 172.21.15.47:0/2471317616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479cebd0 0x55c8479e9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.981+0000 7f203fe00640 1 -- 172.21.15.47:0/2471317616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 msgr2=0x55c8479f08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.981+0000 7f203fe00640 1 --2- 172.21.15.47:0/2471317616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479f08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.981+0000 7f203fe00640 1 -- 172.21.15.47:0/2471317616 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8479cefb0 con 0x55c8479c9750 2024-03-21T11:44:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.981+0000 7f2045000640 1 --2- 172.21.15.47:0/2471317616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479cebd0 0x55c8479e9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.981+0000 7f203f400640 1 --2- 172.21.15.47:0/2471317616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479f08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:44:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.983+0000 7f203fe00640 1 --2- 172.21.15.47:0/2471317616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479c9750 0x55c8479d3510 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f2030012fd0 tx=0x7f2030014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b29564d911ccf619 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.986+0000 7f2034c00640 1 -- 172.21.15.47:0/2471317616 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030028be0 con 0x55c8479c9750 2024-03-21T11:44:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.986+0000 7f2034c00640 1 -- 172.21.15.47:0/2471317616 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2030028d80 con 0x55c8479c9750 2024-03-21T11:44:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.986+0000 7f2034c00640 1 -- 172.21.15.47:0/2471317616 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030028be0 con 0x55c8479c9750 2024-03-21T11:44:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.987+0000 7f2049dc4740 1 -- 172.21.15.47:0/2471317616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479c9750 msgr2=0x55c8479d3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.987+0000 7f2049dc4740 1 --2- 172.21.15.47:0/2471317616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479c9750 0x55c8479d3510 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f2030012fd0 tx=0x7f2030014000 comp rx=0 tx=0).stop 2024-03-21T11:44:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.988+0000 7f2049dc4740 1 -- 172.21.15.47:0/2471317616 shutdown_connections 2024-03-21T11:44:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.988+0000 7f2049dc4740 1 --2- 172.21.15.47:0/2471317616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479cebd0 0x55c8479e9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.988+0000 7f2049dc4740 1 --2- 172.21.15.47:0/2471317616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479f08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.988+0000 7f2049dc4740 1 --2- 172.21.15.47:0/2471317616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479c9750 0x55c8479d3510 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.988+0000 7f2049dc4740 1 -- 172.21.15.47:0/2471317616 >> 172.21.15.47:0/2471317616 conn(0x55c8479cf7d0 msgr2=0x55c8479cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.988+0000 7f2049dc4740 1 -- 172.21.15.47:0/2471317616 shutdown_connections 2024-03-21T11:44:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.988+0000 7f2049dc4740 1 -- 172.21.15.47:0/2471317616 wait complete. 2024-03-21T11:44:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.989+0000 7f2049dc4740 1 -- start start 2024-03-21T11:44:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.989+0000 7f2049dc4740 1 Processor -- start 2024-03-21T11:44:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.990+0000 7f2049dc4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479c9750 0x55c8479edec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.990+0000 7f2049dc4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479ee400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.990+0000 7f2049dc4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479cebd0 0x55c8479ef410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.991+0000 7f2049dc4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20401a9130 con 0x55c8479ce1e0 2024-03-21T11:44:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.991+0000 7f2049dc4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20401a8ad0 con 0x55c8479cebd0 2024-03-21T11:44:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.991+0000 7f2049dc4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2040073410 con 0x55c8479c9750 2024-03-21T11:44:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.991+0000 7f203f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479ee400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.991+0000 7f203f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479ee400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42010/0 (socket says 172.21.15.47:42010) 2024-03-21T11:44:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.991+0000 7f203f400640 1 -- 172.21.15.47:0/4091958992 learned_addr learned my addr 172.21.15.47:0/4091958992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.991+0000 7f2045000640 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479cebd0 0x55c8479ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.992+0000 7f203fe00640 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479c9750 0x55c8479edec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.992+0000 7f2049dc4740 10 client.0 _ll_register_callbacks cb 0x7f202c005670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.993+0000 7f203f400640 1 -- 172.21.15.47:0/4091958992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479c9750 msgr2=0x55c8479edec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.993+0000 7f203f400640 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479c9750 0x55c8479edec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.993+0000 7f203f400640 1 -- 172.21.15.47:0/4091958992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479cebd0 msgr2=0x55c8479ef410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.993+0000 7f203f400640 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479cebd0 0x55c8479ef410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.993+0000 7f203f400640 1 -- 172.21.15.47:0/4091958992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8479efa00 con 0x55c8479ce1e0 2024-03-21T11:44:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.994+0000 7f203fe00640 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479c9750 0x55c8479edec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:44:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.994+0000 7f2045000640 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479cebd0 0x55c8479ef410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.994+0000 7f202a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:44:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.994+0000 7f203f400640 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479ee400 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f202c02c170 tx=0x7f202c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27f5475410ce5e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.995+0000 7f202a000640 1 -- 172.21.15.47:0/4091958992 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c04b040 con 0x55c8479ce1e0 2024-03-21T11:44:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.995+0000 7f202a000640 1 -- 172.21.15.47:0/4091958992 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f202c0b93c0 con 0x55c8479ce1e0 2024-03-21T11:44:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.996+0000 7f2049dc4740 1 -- 172.21.15.47:0/4091958992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2030028d80 con 0x55c8479ce1e0 2024-03-21T11:44:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.997+0000 7f202a000640 1 -- 172.21.15.47:0/4091958992 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f202c0bcd50 con 0x55c8479ce1e0 2024-03-21T11:44:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.997+0000 7f2049dc4740 10 client.21672 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.997+0000 7f2049dc4740 1 -- 172.21.15.47:0/4091958992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c847a50800 con 0x55c8479ce1e0 2024-03-21T11:44:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.998+0000 7f202a000640 1 -- 172.21.15.47:0/4091958992 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2030028d80 con 0x55c8479ce1e0 2024-03-21T11:44:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.998+0000 7f202a000640 1 -- 172.21.15.47:0/4091958992 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f202c0d2070 con 0x55c8479ce1e0 2024-03-21T11:44:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.999+0000 7f2049dc4740 10 client.21672 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.999+0000 7f2049dc4740 10 client.21672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.999+0000 7f2049dc4740 20 client.21672 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.999+0000 7f2049dc4740 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8479d3d40 0x55c847a531d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.999+0000 7f2049dc4740 4 client.21672 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:44:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:31.999+0000 7f2049dc4740 1 -- 172.21.15.47:0/4091958992 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c8479c4a80 con 0x55c8479d3d40 2024-03-21T11:44:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.000+0000 7f203fe00640 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8479d3d40 0x55c847a531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.004+0000 7f202a000640 10 client.21672 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.004+0000 7f203fe00640 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8479d3d40 0x55c847a531d0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15249b2fd833d88 server_cookie=614e51d1b483871d in_seq=0 out_seq=0 2024-03-21T11:44:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.955+0000 7f202a000640 1 -- 172.21.15.47:0/4091958992 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c8479c4a80 con 0x55c8479d3d40 2024-03-21T11:44:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.956+0000 7f202a000640 10 client.21672 handle_command_reply: tid=0 2024-03-21T11:44:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.956+0000 7f2049dc4740 1 client.21672 shutdown 2024-03-21T11:44:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.956+0000 7f2049dc4740 1 -- 172.21.15.47:0/4091958992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 msgr2=0x55c8479ee400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.956+0000 7f2049dc4740 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479ee400 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f202c02c170 tx=0x7f202c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:44:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.956+0000 7f2049dc4740 1 -- 172.21.15.47:0/4091958992 shutdown_connections 2024-03-21T11:44:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.956+0000 7f2049dc4740 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c8479d3d40 0x55c847a531d0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.956+0000 7f2049dc4740 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c8479cebd0 0x55c8479ef410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.956+0000 7f2049dc4740 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c8479ce1e0 0x55c8479ee400 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.956+0000 7f202a000640 0 client.21672 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.956+0000 7f2049dc4740 1 --2- 172.21.15.47:0/4091958992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c8479c9750 0x55c8479edec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.956+0000 7f2049dc4740 1 -- 172.21.15.47:0/4091958992 >> 172.21.15.47:0/4091958992 conn(0x55c8479cf7d0 msgr2=0x55c8479d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.957+0000 7f2049dc4740 1 -- 172.21.15.47:0/4091958992 shutdown_connections 2024-03-21T11:44:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.957+0000 7f2049dc4740 1 -- 172.21.15.47:0/4091958992 wait complete. 2024-03-21T11:44:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.957+0000 7f2049dc4740 20 client.21672 trim_cache size 0 max 16384 2024-03-21T11:44:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:44:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:44:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:44:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:44:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.961+0000 7f2047000640 1 -- 172.21.15.47:0/632800734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f20240a2fc0 msgr2=0x7f20240a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.961+0000 7f2047000640 1 --2- 172.21.15.47:0/632800734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f20240a2fc0 0x7f20240a5480 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x55c8479cf430 tx=0x7f2030005f90 comp rx=0 tx=0).stop 2024-03-21T11:44:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.961+0000 7f2047000640 1 -- 172.21.15.47:0/632800734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2040104ae0 msgr2=0x7f204019e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.961+0000 7f2047000640 1 --2- 172.21.15.47:0/632800734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2040104ae0 0x7f204019e490 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f202c009230 tx=0x7f202c02c650 comp rx=0 tx=0).stop 2024-03-21T11:44:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.962+0000 7f2047000640 1 -- 172.21.15.47:0/632800734 shutdown_connections 2024-03-21T11:44:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.962+0000 7f2047000640 1 --2- 172.21.15.47:0/632800734 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f20240a2fc0 0x7f20240a5480 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.962+0000 7f2047000640 1 --2- 172.21.15.47:0/632800734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20401054a0 0x7f20401a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.962+0000 7f2047000640 1 --2- 172.21.15.47:0/632800734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2040104ae0 0x7f204019e490 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.962+0000 7f2047000640 1 --2- 172.21.15.47:0/632800734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20401038f0 0x7f204019df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.962+0000 7f2047000640 1 -- 172.21.15.47:0/632800734 >> 172.21.15.47:0/632800734 conn(0x7f20400fee60 msgr2=0x7f2040077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.963+0000 7f2047000640 1 -- 172.21.15.47:0/632800734 shutdown_connections 2024-03-21T11:44:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:32.963+0000 7f2047000640 1 -- 172.21.15.47:0/632800734 wait complete. 2024-03-21T11:44:32.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 191481 22.71 MB/sec execute 316 sec latency 600.047 ms 2024-03-21T11:44:33.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:32 smithi102 ceph-mon[36748]: pgmap v1036: 129 pgs: 129 active+clean; 612 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-03-21T11:44:33.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:32 smithi047 ceph-mon[31351]: pgmap v1036: 129 pgs: 129 active+clean; 612 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-03-21T11:44:33.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:32 smithi100 ceph-mon[36245]: pgmap v1036: 129 pgs: 129 active+clean; 612 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-03-21T11:44:33.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 191744 22.69 MB/sec execute 317 sec latency 309.908 ms 2024-03-21T11:44:34.972 INFO:tasks.workunit.client.0.smithi047.stdout: 10 192046 22.67 MB/sec execute 318 sec latency 71.409 ms 2024-03-21T11:44:35.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:35 smithi047 ceph-mon[31351]: pgmap v1037: 129 pgs: 129 active+clean; 619 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.5 MiB/s wr, 526 op/s 2024-03-21T11:44:35.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:35 smithi100 ceph-mon[36245]: pgmap v1037: 129 pgs: 129 active+clean; 619 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.5 MiB/s wr, 526 op/s 2024-03-21T11:44:35.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:35 smithi102 ceph-mon[36748]: pgmap v1037: 129 pgs: 129 active+clean; 619 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.5 MiB/s wr, 526 op/s 2024-03-21T11:44:35.973 INFO:tasks.workunit.client.0.smithi047.stdout: 10 192493 22.67 MB/sec execute 319 sec latency 87.323 ms 2024-03-21T11:44:36.973 INFO:tasks.workunit.client.0.smithi047.stdout: 10 192972 22.66 MB/sec execute 320 sec latency 98.707 ms 2024-03-21T11:44:37.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:37 smithi047 ceph-mon[31351]: pgmap v1038: 129 pgs: 129 active+clean; 622 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 527 op/s 2024-03-21T11:44:37.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:37 smithi100 ceph-mon[36245]: pgmap v1038: 129 pgs: 129 active+clean; 622 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 527 op/s 2024-03-21T11:44:37.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:37 smithi102 ceph-mon[36748]: pgmap v1038: 129 pgs: 129 active+clean; 622 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 527 op/s 2024-03-21T11:44:37.973 INFO:tasks.workunit.client.0.smithi047.stdout: 10 193539 22.70 MB/sec execute 321 sec latency 70.387 ms 2024-03-21T11:44:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:38 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:44:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:38 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:44:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:38 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:44:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:38 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:44:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:38 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:44:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:38 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:44:38.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:38 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:44:38.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:38 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:44:38.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:38 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:44:38.973 INFO:tasks.workunit.client.0.smithi047.stdout: 10 193930 22.70 MB/sec execute 322 sec latency 294.828 ms 2024-03-21T11:44:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:39 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:44:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:39 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:44:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:39 smithi047 ceph-mon[31351]: pgmap v1039: 129 pgs: 129 active+clean; 615 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 548 op/s 2024-03-21T11:44:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:39 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:44:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:39 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:44:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:39 smithi100 ceph-mon[36245]: pgmap v1039: 129 pgs: 129 active+clean; 615 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 548 op/s 2024-03-21T11:44:39.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:39 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:44:39.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:39 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:44:39.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:39 smithi102 ceph-mon[36748]: pgmap v1039: 129 pgs: 129 active+clean; 615 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 548 op/s 2024-03-21T11:44:39.973 INFO:tasks.workunit.client.0.smithi047.stdout: 10 194290 22.69 MB/sec execute 323 sec latency 110.064 ms 2024-03-21T11:44:40.973 INFO:tasks.workunit.client.0.smithi047.stdout: 10 194730 22.68 MB/sec execute 324 sec latency 66.172 ms 2024-03-21T11:44:41.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:41 smithi047 ceph-mon[31351]: pgmap v1040: 129 pgs: 129 active+clean; 614 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-03-21T11:44:41.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:41 smithi100 ceph-mon[36245]: pgmap v1040: 129 pgs: 129 active+clean; 614 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-03-21T11:44:41.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:41 smithi102 ceph-mon[36748]: pgmap v1040: 129 pgs: 129 active+clean; 614 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-03-21T11:44:41.973 INFO:tasks.workunit.client.0.smithi047.stdout: 10 195168 22.68 MB/sec execute 325 sec latency 71.544 ms 2024-03-21T11:44:42.973 INFO:tasks.workunit.client.0.smithi047.stdout: 10 195816 22.71 MB/sec execute 326 sec latency 51.651 ms 2024-03-21T11:44:43.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:43 smithi047 ceph-mon[31351]: pgmap v1041: 129 pgs: 129 active+clean; 616 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.3 MiB/s wr, 561 op/s 2024-03-21T11:44:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:43 smithi100 ceph-mon[36245]: pgmap v1041: 129 pgs: 129 active+clean; 616 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.3 MiB/s wr, 561 op/s 2024-03-21T11:44:43.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:43 smithi102 ceph-mon[36748]: pgmap v1041: 129 pgs: 129 active+clean; 616 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.3 MiB/s wr, 561 op/s 2024-03-21T11:44:43.973 INFO:tasks.workunit.client.0.smithi047.stdout: 10 196167 22.71 MB/sec execute 327 sec latency 355.549 ms 2024-03-21T11:44:44.974 INFO:tasks.workunit.client.0.smithi047.stdout: 10 196426 22.69 MB/sec execute 328 sec latency 470.172 ms 2024-03-21T11:44:45.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:45 smithi100 ceph-mon[36245]: pgmap v1042: 129 pgs: 129 active+clean; 620 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-03-21T11:44:45.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:45 smithi102 ceph-mon[36748]: pgmap v1042: 129 pgs: 129 active+clean; 620 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-03-21T11:44:45.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:45 smithi047 ceph-mon[31351]: pgmap v1042: 129 pgs: 129 active+clean; 620 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-03-21T11:44:45.973 INFO:tasks.workunit.client.0.smithi047.stdout: 10 196705 22.67 MB/sec execute 329 sec latency 108.035 ms 2024-03-21T11:44:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.130+0000 7f227ae00640 1 -- 172.21.15.47:0/2288819787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2274103900 msgr2=0x7f2274103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.130+0000 7f227ae00640 1 --2- 172.21.15.47:0/2288819787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2274103900 0x7f2274103d20 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f2260009900 tx=0x7f226002bc60 comp rx=0 tx=0).stop 2024-03-21T11:44:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.130+0000 7f227ae00640 1 -- 172.21.15.47:0/2288819787 shutdown_connections 2024-03-21T11:44:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.130+0000 7f227ae00640 1 --2- 172.21.15.47:0/2288819787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22741054b0 0x7f227410bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.130+0000 7f227ae00640 1 --2- 172.21.15.47:0/2288819787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2274104af0 0x7f2274104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.130+0000 7f227ae00640 1 --2- 172.21.15.47:0/2288819787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2274103900 0x7f2274103d20 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.130+0000 7f227ae00640 1 -- 172.21.15.47:0/2288819787 >> 172.21.15.47:0/2288819787 conn(0x7f22740fee90 msgr2=0x7f22741012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.130+0000 7f227ae00640 1 -- 172.21.15.47:0/2288819787 shutdown_connections 2024-03-21T11:44:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.130+0000 7f227ae00640 1 -- 172.21.15.47:0/2288819787 wait complete. 2024-03-21T11:44:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.132+0000 7f227ae00640 1 Processor -- start 2024-03-21T11:44:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.132+0000 7f227ae00640 1 -- start start 2024-03-21T11:44:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.132+0000 7f227ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2274103900 0x7f227419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.132+0000 7f227ae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2274104af0 0x7f227419e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.132+0000 7f227ae00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22741054b0 0x7f22741a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.132+0000 7f227ae00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2274076530 con 0x7f22741054b0 2024-03-21T11:44:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.132+0000 7f227ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22740763b0 con 0x7f2274103900 2024-03-21T11:44:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.132+0000 7f227ae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22740766b0 con 0x7f2274104af0 2024-03-21T11:44:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.132+0000 7f2278e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22741054b0 0x7f22741a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.132+0000 7f2273400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2274104af0 0x7f227419e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.133+0000 7f2273400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2274104af0 0x7f227419e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41580/0 (socket says 172.21.15.47:41580) 2024-03-21T11:44:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.133+0000 7f2273400640 1 -- 172.21.15.47:0/2723916571 learned_addr learned my addr 172.21.15.47:0/2723916571 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.133+0000 7f2278e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22741054b0 0x7f22741a54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33014/0 (socket says 172.21.15.47:33014) 2024-03-21T11:44:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.133+0000 7f2273e00640 1 --2- 172.21.15.47:0/2723916571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2274103900 0x7f227419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.133+0000 7f2273400640 1 -- 172.21.15.47:0/2723916571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2274103900 msgr2=0x7f227419df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.133+0000 7f2273400640 1 --2- 172.21.15.47:0/2723916571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2274103900 0x7f227419df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.133+0000 7f2273400640 1 -- 172.21.15.47:0/2723916571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22741054b0 msgr2=0x7f22741a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.133+0000 7f2273400640 1 --2- 172.21.15.47:0/2723916571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22741054b0 0x7f22741a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.133+0000 7f2273400640 1 -- 172.21.15.47:0/2723916571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22741a5b60 con 0x7f2274104af0 2024-03-21T11:44:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.133+0000 7f2273400640 1 --2- 172.21.15.47:0/2723916571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2274104af0 0x7f227419e480 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f226400c890 tx=0x7f226400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.134+0000 7f2270c00640 1 -- 172.21.15.47:0/2723916571 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264004220 con 0x7f2274104af0 2024-03-21T11:44:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.134+0000 7f2270c00640 1 -- 172.21.15.47:0/2723916571 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22640043c0 con 0x7f2274104af0 2024-03-21T11:44:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.134+0000 7f2270c00640 1 -- 172.21.15.47:0/2723916571 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264004a00 con 0x7f2274104af0 2024-03-21T11:44:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.136+0000 7f227ae00640 1 -- 172.21.15.47:0/2723916571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22741a5e50 con 0x7f2274104af0 2024-03-21T11:44:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.136+0000 7f227ae00640 1 -- 172.21.15.47:0/2723916571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22741a62c0 con 0x7f2274104af0 2024-03-21T11:44:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.140+0000 7f2270c00640 1 -- 172.21.15.47:0/2723916571 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f22640079b0 con 0x7f2274104af0 2024-03-21T11:44:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.140+0000 7f227dbee740 1 Processor -- start 2024-03-21T11:44:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.140+0000 7f227dbee740 1 -- start start 2024-03-21T11:44:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.141+0000 7f227dbee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c21e0 0x5621976c2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.141+0000 7f227dbee740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c2bd0 0x5621976c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.141+0000 7f227dbee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c7d40 0x5621976ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.141+0000 7f227dbee740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621976853b0 con 0x5621976c2bd0 2024-03-21T11:44:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.141+0000 7f227dbee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562197627830 con 0x5621976c21e0 2024-03-21T11:44:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.141+0000 7f227dbee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621975d05d0 con 0x5621976c7d40 2024-03-21T11:44:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.141+0000 7f2278e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c21e0 0x5621976c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.141+0000 7f2273e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c2bd0 0x5621976c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.141+0000 7f2273e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c2bd0 0x5621976c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33018/0 (socket says 172.21.15.47:33018) 2024-03-21T11:44:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.141+0000 7f2273e00640 1 -- 172.21.15.47:0/1563114267 learned_addr learned my addr 172.21.15.47:0/1563114267 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.141+0000 7f2273e00640 1 -- 172.21.15.47:0/1563114267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c7d40 msgr2=0x5621976ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.141+0000 7f2273e00640 1 --2- 172.21.15.47:0/1563114267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c7d40 0x5621976ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.141+0000 7f2273e00640 1 -- 172.21.15.47:0/1563114267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c21e0 msgr2=0x5621976c2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.141+0000 7f2273e00640 1 --2- 172.21.15.47:0/1563114267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c21e0 0x5621976c2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.142+0000 7f2273e00640 1 -- 172.21.15.47:0/1563114267 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621975e82c0 con 0x5621976c2bd0 2024-03-21T11:44:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.142+0000 7f2270c00640 1 --2- 172.21.15.47:0/2723916571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22580a3040 0x7f22580a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.142+0000 7f2273e00640 1 --2- 172.21.15.47:0/1563114267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c2bd0 0x5621976c76d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2260000c00 tx=0x7f22600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4aa1fb5c7a4cca server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.142+0000 7f2270c00640 1 -- 172.21.15.47:0/2723916571 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f22640cf690 con 0x7f2274104af0 2024-03-21T11:44:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.142+0000 7f2278e00640 1 --2- 172.21.15.47:0/2723916571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22580a3040 0x7f22580a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.142+0000 7f2269600640 1 -- 172.21.15.47:0/1563114267 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2260043070 con 0x5621976c2bd0 2024-03-21T11:44:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.142+0000 7f2269600640 1 -- 172.21.15.47:0/1563114267 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2260004970 con 0x5621976c2bd0 2024-03-21T11:44:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.142+0000 7f2269600640 1 -- 172.21.15.47:0/1563114267 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22600053d0 con 0x5621976c2bd0 2024-03-21T11:44:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.143+0000 7f2278e00640 1 --2- 172.21.15.47:0/2723916571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22580a3040 0x7f22580a5500 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x5621976c3430 tx=0x7f226c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.143+0000 7f227dbee740 1 -- 172.21.15.47:0/1563114267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c2bd0 msgr2=0x5621976c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.143+0000 7f227dbee740 1 --2- 172.21.15.47:0/1563114267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c2bd0 0x5621976c76d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2260000c00 tx=0x7f22600046c0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.143+0000 7f227dbee740 1 -- 172.21.15.47:0/1563114267 shutdown_connections 2024-03-21T11:44:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.143+0000 7f227dbee740 1 --2- 172.21.15.47:0/1563114267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c7d40 0x5621976ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.143+0000 7f227dbee740 1 --2- 172.21.15.47:0/1563114267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c2bd0 0x5621976c76d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.143+0000 7f227dbee740 1 --2- 172.21.15.47:0/1563114267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c21e0 0x5621976c2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.143+0000 7f227dbee740 1 -- 172.21.15.47:0/1563114267 >> 172.21.15.47:0/1563114267 conn(0x5621976bd750 msgr2=0x5621976bdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.144+0000 7f227dbee740 1 -- 172.21.15.47:0/1563114267 shutdown_connections 2024-03-21T11:44:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.144+0000 7f227dbee740 1 -- 172.21.15.47:0/1563114267 wait complete. 2024-03-21T11:44:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.144+0000 7f227dbee740 1 -- start start 2024-03-21T11:44:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.145+0000 7f227dbee740 1 Processor -- start 2024-03-21T11:44:46.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.145+0000 7f227dbee740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.145+0000 7f227dbee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c2bd0 0x5621976cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.145+0000 7f227dbee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c7d40 0x5621976d3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.145+0000 7f227dbee740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621975e3a20 con 0x5621976c21e0 2024-03-21T11:44:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.145+0000 7f227dbee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621976238d0 con 0x5621976c7d40 2024-03-21T11:44:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.145+0000 7f227dbee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621976cac90 con 0x5621976c2bd0 2024-03-21T11:44:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.145+0000 7f227dbee740 10 client.0 _ll_register_callbacks cb 0x5621976201c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.146+0000 7f2273e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.146+0000 7f2273400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c2bd0 0x5621976cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.146+0000 7f2273400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c2bd0 0x5621976cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41594/0 (socket says 172.21.15.47:41594) 2024-03-21T11:44:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.146+0000 7f2273400640 1 -- 172.21.15.47:0/1192239369 learned_addr learned my addr 172.21.15.47:0/1192239369 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.146+0000 7f2273e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33022/0 (socket says 172.21.15.47:33022) 2024-03-21T11:44:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.146+0000 7f2278e00640 1 --2- 172.21.15.47:0/1192239369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c7d40 0x5621976d3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.146+0000 7f2273e00640 1 -- 172.21.15.47:0/1192239369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c2bd0 msgr2=0x5621976cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.146+0000 7f2273e00640 1 --2- 172.21.15.47:0/1192239369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c2bd0 0x5621976cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.146+0000 7f2273e00640 1 -- 172.21.15.47:0/1192239369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c7d40 msgr2=0x5621976d3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.146+0000 7f2273e00640 1 --2- 172.21.15.47:0/1192239369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c7d40 0x5621976d3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.146+0000 7f2273e00640 1 -- 172.21.15.47:0/1192239369 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56219733c210 con 0x5621976c21e0 2024-03-21T11:44:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.147+0000 7f2273e00640 1 --2- 172.21.15.47:0/1192239369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976cc400 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x5621976be6c0 tx=0x7f2260005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cae8eb41d7e09f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.147+0000 7f225e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:44:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.147+0000 7f225e000640 1 -- 172.21.15.47:0/1192239369 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2260043070 con 0x5621976c21e0 2024-03-21T11:44:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.147+0000 7f225e000640 1 -- 172.21.15.47:0/1192239369 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f226002c950 con 0x5621976c21e0 2024-03-21T11:44:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.147+0000 7f227dbee740 1 -- 172.21.15.47:0/1192239369 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2260004970 con 0x5621976c21e0 2024-03-21T11:44:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.148+0000 7f225e000640 1 -- 172.21.15.47:0/1192239369 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f226002cb50 con 0x5621976c21e0 2024-03-21T11:44:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.148+0000 7f227dbee740 10 client.21684 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.148+0000 7f227dbee740 1 -- 172.21.15.47:0/1192239369 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562197408620 con 0x5621976c21e0 2024-03-21T11:44:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.148+0000 7f225e000640 1 -- 172.21.15.47:0/1192239369 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2260004970 con 0x5621976c21e0 2024-03-21T11:44:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.149+0000 7f225e000640 1 -- 172.21.15.47:0/1192239369 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2260048450 con 0x5621976c21e0 2024-03-21T11:44:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.149+0000 7f227dbee740 10 client.21684 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.149+0000 7f227dbee740 10 client.21684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.149+0000 7f227dbee740 20 client.21684 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.149+0000 7f227dbee740 1 --2- 172.21.15.47:0/1192239369 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5621976d7e70 0x5621976da260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.149+0000 7f227dbee740 4 client.21684 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.149+0000 7f227dbee740 1 -- 172.21.15.47:0/1192239369 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562197634d70 con 0x5621976d7e70 2024-03-21T11:44:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.149+0000 7f2273400640 1 --2- 172.21.15.47:0/1192239369 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5621976d7e70 0x5621976da260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.151+0000 7f2273400640 1 --2- 172.21.15.47:0/1192239369 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5621976d7e70 0x5621976da260 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e9b7e728aa7d7b4 server_cookie=62702b6fdaa849f2 in_seq=0 out_seq=0 2024-03-21T11:44:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.151+0000 7f225e000640 10 client.21684 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.154+0000 7f225e000640 1 -- 172.21.15.47:0/1192239369 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562197634d70 con 0x5621976d7e70 2024-03-21T11:44:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.154+0000 7f225e000640 10 client.21684 handle_command_reply: tid=0 2024-03-21T11:44:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.154+0000 7f227dbee740 1 client.21684 shutdown 2024-03-21T11:44:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.154+0000 7f227dbee740 1 -- 172.21.15.47:0/1192239369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 msgr2=0x5621976cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.154+0000 7f227dbee740 1 --2- 172.21.15.47:0/1192239369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976cc400 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x5621976be6c0 tx=0x7f2260005710 comp rx=0 tx=0).stop 2024-03-21T11:44:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.154+0000 7f227dbee740 1 -- 172.21.15.47:0/1192239369 shutdown_connections 2024-03-21T11:44:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.154+0000 7f227dbee740 1 --2- 172.21.15.47:0/1192239369 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5621976d7e70 0x5621976da260 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.154+0000 7f227dbee740 1 --2- 172.21.15.47:0/1192239369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c7d40 0x5621976d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.154+0000 7f227dbee740 1 --2- 172.21.15.47:0/1192239369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c2bd0 0x5621976cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.154+0000 7f227dbee740 1 --2- 172.21.15.47:0/1192239369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976cc400 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.154+0000 7f227dbee740 1 -- 172.21.15.47:0/1192239369 >> 172.21.15.47:0/1192239369 conn(0x5621976bd750 msgr2=0x5621976bc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.154+0000 7f225e000640 0 client.21684 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.155+0000 7f227dbee740 1 -- 172.21.15.47:0/1192239369 shutdown_connections 2024-03-21T11:44:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.155+0000 7f227dbee740 1 -- 172.21.15.47:0/1192239369 wait complete. 2024-03-21T11:44:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.155+0000 7f227dbee740 20 client.21684 trim_cache size 0 max 16384 2024-03-21T11:44:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.178+0000 7f227dbee740 1 Processor -- start 2024-03-21T11:44:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.178+0000 7f227dbee740 1 -- start start 2024-03-21T11:44:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.179+0000 7f227dbee740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.179+0000 7f227dbee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c2bd0 0x5621976de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.179+0000 7f227dbee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c7d40 0x5621976def20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.179+0000 7f227dbee740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741b0710 con 0x5621976c21e0 2024-03-21T11:44:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.179+0000 7f227dbee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741b2090 con 0x5621976c7d40 2024-03-21T11:44:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.179+0000 7f227dbee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741b0d70 con 0x5621976c2bd0 2024-03-21T11:44:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.179+0000 7f2273e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.179+0000 7f2273e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976c10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33034/0 (socket says 172.21.15.47:33034) 2024-03-21T11:44:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.179+0000 7f2273e00640 1 -- 172.21.15.47:0/505065752 learned_addr learned my addr 172.21.15.47:0/505065752 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.179+0000 7f2273400640 1 --2- 172.21.15.47:0/505065752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c2bd0 0x5621976de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.180+0000 7f2278e00640 1 --2- 172.21.15.47:0/505065752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c7d40 0x5621976def20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.180+0000 7f2273e00640 1 -- 172.21.15.47:0/505065752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c2bd0 msgr2=0x5621976de9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.180+0000 7f2273e00640 1 --2- 172.21.15.47:0/505065752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c2bd0 0x5621976de9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.180+0000 7f2273e00640 1 -- 172.21.15.47:0/505065752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c7d40 msgr2=0x5621976def20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.180+0000 7f2273e00640 1 --2- 172.21.15.47:0/505065752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c7d40 0x5621976def20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.180+0000 7f2273e00640 1 -- 172.21.15.47:0/505065752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621976b83a0 con 0x5621976c21e0 2024-03-21T11:44:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.180+0000 7f2278e00640 1 --2- 172.21.15.47:0/505065752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c7d40 0x5621976def20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:44:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.180+0000 7f2273e00640 1 --2- 172.21.15.47:0/505065752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976c10c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f226002c1a0 tx=0x7f226002bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a886c2f02cb7e36e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.181+0000 7f2268c00640 1 -- 172.21.15.47:0/505065752 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226005f070 con 0x5621976c21e0 2024-03-21T11:44:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.181+0000 7f2268c00640 1 -- 172.21.15.47:0/505065752 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f226002c950 con 0x5621976c21e0 2024-03-21T11:44:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.181+0000 7f2268c00640 1 -- 172.21.15.47:0/505065752 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2260043040 con 0x5621976c21e0 2024-03-21T11:44:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.182+0000 7f227dbee740 1 -- 172.21.15.47:0/505065752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 msgr2=0x5621976c10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.183+0000 7f227dbee740 1 --2- 172.21.15.47:0/505065752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976c10c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f226002c1a0 tx=0x7f226002bc60 comp rx=0 tx=0).stop 2024-03-21T11:44:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.183+0000 7f227dbee740 1 -- 172.21.15.47:0/505065752 shutdown_connections 2024-03-21T11:44:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.183+0000 7f227dbee740 1 --2- 172.21.15.47:0/505065752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c7d40 0x5621976def20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.183+0000 7f227dbee740 1 --2- 172.21.15.47:0/505065752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c2bd0 0x5621976de9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.184+0000 7f227dbee740 1 --2- 172.21.15.47:0/505065752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976c10c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.184+0000 7f227dbee740 1 -- 172.21.15.47:0/505065752 >> 172.21.15.47:0/505065752 conn(0x5621976bd750 msgr2=0x5621976c09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.185+0000 7f227dbee740 1 -- 172.21.15.47:0/505065752 shutdown_connections 2024-03-21T11:44:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.185+0000 7f227dbee740 1 -- 172.21.15.47:0/505065752 wait complete. 2024-03-21T11:44:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.185+0000 7f227dbee740 1 -- start start 2024-03-21T11:44:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.186+0000 7f227dbee740 1 Processor -- start 2024-03-21T11:44:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.187+0000 7f227dbee740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.187+0000 7f227dbee740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c2bd0 0x5621976d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.187+0000 7f227dbee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c7d40 0x5621976d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.187+0000 7f227dbee740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741b1a30 con 0x5621976c21e0 2024-03-21T11:44:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.187+0000 7f227dbee740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741b13d0 con 0x5621976c2bd0 2024-03-21T11:44:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.187+0000 7f227dbee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741ae8d0 con 0x5621976c7d40 2024-03-21T11:44:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.187+0000 7f227dbee740 10 client.0 _ll_register_callbacks cb 0x7f22600095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.187+0000 7f2273400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c2bd0 0x5621976d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.187+0000 7f2273400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c2bd0 0x5621976d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58452/0 (socket says 172.21.15.47:58452) 2024-03-21T11:44:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.187+0000 7f2273400640 1 -- 172.21.15.47:0/3728041960 learned_addr learned my addr 172.21.15.47:0/3728041960 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.187+0000 7f2273e00640 1 --2- 172.21.15.47:0/3728041960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.188+0000 7f2278e00640 1 --2- 172.21.15.47:0/3728041960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c7d40 0x5621976d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.188+0000 7f2273400640 1 -- 172.21.15.47:0/3728041960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c7d40 msgr2=0x5621976d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.188+0000 7f2273400640 1 --2- 172.21.15.47:0/3728041960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c7d40 0x5621976d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.188+0000 7f2273400640 1 -- 172.21.15.47:0/3728041960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 msgr2=0x5621976d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.188+0000 7f2273400640 1 --2- 172.21.15.47:0/3728041960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.188+0000 7f2273400640 1 -- 172.21.15.47:0/3728041960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621976d25b0 con 0x5621976c2bd0 2024-03-21T11:44:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.188+0000 7f2278e00640 1 --2- 172.21.15.47:0/3728041960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c7d40 0x5621976d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:44:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.188+0000 7f2273400640 1 --2- 172.21.15.47:0/3728041960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c2bd0 0x5621976d1000 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f226409b960 tx=0x7f2264005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7612a7d52d488593 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.189+0000 7f225e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:44:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.189+0000 7f225e000640 1 -- 172.21.15.47:0/3728041960 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22640b9060 con 0x5621976c2bd0 2024-03-21T11:44:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.189+0000 7f227dbee740 1 -- 172.21.15.47:0/3728041960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f226002c950 con 0x5621976c2bd0 2024-03-21T11:44:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.189+0000 7f225e000640 1 -- 172.21.15.47:0/3728041960 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f22640cf340 con 0x5621976c2bd0 2024-03-21T11:44:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.189+0000 7f225e000640 1 -- 172.21.15.47:0/3728041960 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f226409fb20 con 0x5621976c2bd0 2024-03-21T11:44:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.189+0000 7f227dbee740 10 client.32372 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.189+0000 7f227dbee740 1 -- 172.21.15.47:0/3728041960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5621977337d0 con 0x5621976c2bd0 2024-03-21T11:44:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.190+0000 7f225e000640 1 -- 172.21.15.47:0/3728041960 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f226002c950 con 0x5621976c2bd0 2024-03-21T11:44:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.191+0000 7f225e000640 1 -- 172.21.15.47:0/3728041960 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f22640ba250 con 0x5621976c2bd0 2024-03-21T11:44:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.191+0000 7f227dbee740 10 client.32372 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.191+0000 7f227dbee740 10 client.32372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.191+0000 7f227dbee740 20 client.32372 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.192+0000 7f227dbee740 1 --2- 172.21.15.47:0/3728041960 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562197734110 0x562197736500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.192+0000 7f227dbee740 4 client.32372 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:44:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.192+0000 7f227dbee740 1 -- 172.21.15.47:0/3728041960 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56219762f0c0 con 0x562197734110 2024-03-21T11:44:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.192+0000 7f2273e00640 1 --2- 172.21.15.47:0/3728041960 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562197734110 0x562197736500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.193+0000 7f2273e00640 1 --2- 172.21.15.47:0/3728041960 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562197734110 0x562197736500 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13c1b551d5ca87d server_cookie=6724cd80a5cddab0 in_seq=0 out_seq=0 2024-03-21T11:44:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:46.193+0000 7f225e000640 10 client.32372 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:46.974 INFO:tasks.workunit.client.0.smithi047.stdout: 10 196916 22.65 MB/sec execute 330 sec latency 645.858 ms 2024-03-21T11:44:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.110+0000 7f225e000640 1 -- 172.21.15.47:0/3728041960 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56219762f0c0 con 0x562197734110 2024-03-21T11:44:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.110+0000 7f225e000640 10 client.32372 handle_command_reply: tid=0 2024-03-21T11:44:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.110+0000 7f227dbee740 1 client.32372 shutdown 2024-03-21T11:44:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.110+0000 7f227dbee740 1 -- 172.21.15.47:0/3728041960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c2bd0 msgr2=0x5621976d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.110+0000 7f227dbee740 1 --2- 172.21.15.47:0/3728041960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c2bd0 0x5621976d1000 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f226409b960 tx=0x7f2264005d20 comp rx=0 tx=0).stop 2024-03-21T11:44:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.111+0000 7f227dbee740 1 -- 172.21.15.47:0/3728041960 shutdown_connections 2024-03-21T11:44:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.111+0000 7f227dbee740 1 --2- 172.21.15.47:0/3728041960 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x562197734110 0x562197736500 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.111+0000 7f227dbee740 1 --2- 172.21.15.47:0/3728041960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5621976c7d40 0x5621976d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.111+0000 7f227dbee740 1 --2- 172.21.15.47:0/3728041960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621976c2bd0 0x5621976d1000 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.111+0000 7f227dbee740 1 --2- 172.21.15.47:0/3728041960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5621976c21e0 0x5621976d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.111+0000 7f227dbee740 1 -- 172.21.15.47:0/3728041960 >> 172.21.15.47:0/3728041960 conn(0x5621976bd750 msgr2=0x5621976c1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.111+0000 7f225e000640 0 client.32372 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.111+0000 7f227dbee740 1 -- 172.21.15.47:0/3728041960 shutdown_connections 2024-03-21T11:44:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.111+0000 7f227dbee740 1 -- 172.21.15.47:0/3728041960 wait complete. 2024-03-21T11:44:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.111+0000 7f227dbee740 20 client.32372 trim_cache size 0 max 16384 2024-03-21T11:44:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:44:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:44:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:44:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:44:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.113+0000 7f227ae00640 1 -- 172.21.15.47:0/2723916571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22580a3040 msgr2=0x7f22580a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.113+0000 7f227ae00640 1 --2- 172.21.15.47:0/2723916571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22580a3040 0x7f22580a5500 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x5621976c3430 tx=0x7f226c009500 comp rx=0 tx=0).stop 2024-03-21T11:44:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.113+0000 7f227ae00640 1 -- 172.21.15.47:0/2723916571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2274104af0 msgr2=0x7f227419e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.113+0000 7f227ae00640 1 --2- 172.21.15.47:0/2723916571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2274104af0 0x7f227419e480 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f226400c890 tx=0x7f226400cd50 comp rx=0 tx=0).stop 2024-03-21T11:44:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.113+0000 7f227ae00640 1 -- 172.21.15.47:0/2723916571 shutdown_connections 2024-03-21T11:44:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.113+0000 7f227ae00640 1 --2- 172.21.15.47:0/2723916571 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f22580a3040 0x7f22580a5500 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.113+0000 7f227ae00640 1 --2- 172.21.15.47:0/2723916571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22741054b0 0x7f22741a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.113+0000 7f227ae00640 1 --2- 172.21.15.47:0/2723916571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2274104af0 0x7f227419e480 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.113+0000 7f227ae00640 1 --2- 172.21.15.47:0/2723916571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2274103900 0x7f227419df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.113+0000 7f227ae00640 1 -- 172.21.15.47:0/2723916571 >> 172.21.15.47:0/2723916571 conn(0x7f22740fee90 msgr2=0x7f2274078f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.114+0000 7f227ae00640 1 -- 172.21.15.47:0/2723916571 shutdown_connections 2024-03-21T11:44:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:44:47.114+0000 7f227ae00640 1 -- 172.21.15.47:0/2723916571 wait complete. 2024-03-21T11:44:47.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:47 smithi100 ceph-mon[36245]: pgmap v1043: 129 pgs: 129 active+clean; 628 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.9 MiB/s wr, 536 op/s 2024-03-21T11:44:47.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:47 smithi102 ceph-mon[36748]: pgmap v1043: 129 pgs: 129 active+clean; 628 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.9 MiB/s wr, 536 op/s 2024-03-21T11:44:47.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:47 smithi047 ceph-mon[31351]: pgmap v1043: 129 pgs: 129 active+clean; 628 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.9 MiB/s wr, 536 op/s 2024-03-21T11:44:47.974 INFO:tasks.workunit.client.0.smithi047.stdout: 10 197254 22.62 MB/sec execute 331 sec latency 768.602 ms 2024-03-21T11:44:48.974 INFO:tasks.workunit.client.0.smithi047.stdout: 10 197608 22.62 MB/sec execute 332 sec latency 247.357 ms 2024-03-21T11:44:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:49 smithi100 ceph-mon[36245]: pgmap v1044: 129 pgs: 129 active+clean; 633 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.6 MiB/s wr, 519 op/s 2024-03-21T11:44:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:49 smithi102 ceph-mon[36748]: pgmap v1044: 129 pgs: 129 active+clean; 633 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.6 MiB/s wr, 519 op/s 2024-03-21T11:44:49.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:49 smithi047 ceph-mon[31351]: pgmap v1044: 129 pgs: 129 active+clean; 633 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.6 MiB/s wr, 519 op/s 2024-03-21T11:44:49.974 INFO:tasks.workunit.client.0.smithi047.stdout: 10 197840 22.57 MB/sec execute 333 sec latency 328.062 ms 2024-03-21T11:44:50.974 INFO:tasks.workunit.client.0.smithi047.stdout: 10 198172 22.54 MB/sec execute 334 sec latency 202.948 ms 2024-03-21T11:44:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:51 smithi100 ceph-mon[36245]: pgmap v1045: 129 pgs: 129 active+clean; 624 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.6 MiB/s wr, 638 op/s 2024-03-21T11:44:51.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:51 smithi102 ceph-mon[36748]: pgmap v1045: 129 pgs: 129 active+clean; 624 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.6 MiB/s wr, 638 op/s 2024-03-21T11:44:51.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:51 smithi047 ceph-mon[31351]: pgmap v1045: 129 pgs: 129 active+clean; 624 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.6 MiB/s wr, 638 op/s 2024-03-21T11:44:51.974 INFO:tasks.workunit.client.0.smithi047.stdout: 10 198785 22.59 MB/sec execute 335 sec latency 52.604 ms 2024-03-21T11:44:52.974 INFO:tasks.workunit.client.0.smithi047.stdout: 10 199333 22.60 MB/sec execute 336 sec latency 93.255 ms 2024-03-21T11:44:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:53 smithi100 ceph-mon[36245]: pgmap v1046: 129 pgs: 129 active+clean; 620 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 528 op/s 2024-03-21T11:44:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:53 smithi102 ceph-mon[36748]: pgmap v1046: 129 pgs: 129 active+clean; 620 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 528 op/s 2024-03-21T11:44:53.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:53 smithi047 ceph-mon[31351]: pgmap v1046: 129 pgs: 129 active+clean; 620 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 528 op/s 2024-03-21T11:44:53.974 INFO:tasks.workunit.client.0.smithi047.stdout: 10 199775 22.62 MB/sec execute 337 sec latency 145.187 ms 2024-03-21T11:44:54.974 INFO:tasks.workunit.client.0.smithi047.stdout: 10 199999 22.59 MB/sec execute 338 sec latency 418.543 ms 2024-03-21T11:44:55.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:55 smithi100 ceph-mon[36245]: pgmap v1047: 129 pgs: 129 active+clean; 623 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.9 MiB/s wr, 570 op/s 2024-03-21T11:44:55.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:55 smithi102 ceph-mon[36748]: pgmap v1047: 129 pgs: 129 active+clean; 623 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.9 MiB/s wr, 570 op/s 2024-03-21T11:44:55.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:55 smithi047 ceph-mon[31351]: pgmap v1047: 129 pgs: 129 active+clean; 623 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.9 MiB/s wr, 570 op/s 2024-03-21T11:44:55.974 INFO:tasks.workunit.client.0.smithi047.stdout: 10 200413 22.59 MB/sec execute 339 sec latency 76.570 ms 2024-03-21T11:44:56.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:44:56.416 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:44:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.910+0000 7f1a95c00640 1 -- 172.21.15.47:0/958992708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a90103260 msgr2=0x7f1a90109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.910+0000 7f1a95c00640 1 --2- 172.21.15.47:0/958992708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a90103260 0x7f1a90109af0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009ca0 tx=0x7f1a8802c100 comp rx=0 tx=0).stop 2024-03-21T11:44:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.911+0000 7f1a95c00640 1 -- 172.21.15.47:0/958992708 shutdown_connections 2024-03-21T11:44:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.911+0000 7f1a95c00640 1 --2- 172.21.15.47:0/958992708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a90103260 0x7f1a90109af0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.911+0000 7f1a95c00640 1 --2- 172.21.15.47:0/958992708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a901028a0 0x7f1a90102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.911+0000 7f1a95c00640 1 --2- 172.21.15.47:0/958992708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a901016b0 0x7f1a90101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.911+0000 7f1a95c00640 1 -- 172.21.15.47:0/958992708 >> 172.21.15.47:0/958992708 conn(0x7f1a900fcc40 msgr2=0x7f1a900ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.911+0000 7f1a95c00640 1 -- 172.21.15.47:0/958992708 shutdown_connections 2024-03-21T11:44:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.911+0000 7f1a95c00640 1 -- 172.21.15.47:0/958992708 wait complete. 2024-03-21T11:44:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.913+0000 7f1a95c00640 1 Processor -- start 2024-03-21T11:44:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.915+0000 7f1a95c00640 1 -- start start 2024-03-21T11:44:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.915+0000 7f1a95c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a901016b0 0x7f1a9010f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.915+0000 7f1a95c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a901028a0 0x7f1a9010fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.915+0000 7f1a95c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a90103260 0x7f1a9010c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.916+0000 7f1a95c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a90074d80 con 0x7f1a90103260 2024-03-21T11:44:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.916+0000 7f1a95c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a90074c00 con 0x7f1a901016b0 2024-03-21T11:44:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.916+0000 7f1a95c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a90074f00 con 0x7f1a901028a0 2024-03-21T11:44:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.916+0000 7f1a8ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a901016b0 0x7f1a9010f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.916+0000 7f1a8ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a901016b0 0x7f1a9010f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43586/0 (socket says 172.21.15.47:43586) 2024-03-21T11:44:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.916+0000 7f1a8ea00640 1 -- 172.21.15.47:0/3753944264 learned_addr learned my addr 172.21.15.47:0/3753944264 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.917+0000 7f1a8ea00640 1 -- 172.21.15.47:0/3753944264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a901028a0 msgr2=0x7f1a9010fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.917+0000 7f1a8ea00640 1 --2- 172.21.15.47:0/3753944264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a901028a0 0x7f1a9010fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.917+0000 7f1a8ea00640 1 -- 172.21.15.47:0/3753944264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a90103260 msgr2=0x7f1a9010c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.917+0000 7f1a8ea00640 1 --2- 172.21.15.47:0/3753944264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a90103260 0x7f1a9010c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.917+0000 7f1a8ea00640 1 -- 172.21.15.47:0/3753944264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9010cc80 con 0x7f1a901016b0 2024-03-21T11:44:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.918+0000 7f1a8ea00640 1 --2- 172.21.15.47:0/3753944264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a901016b0 0x7f1a9010f850 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000e9c0 tx=0x7f1a8000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.919+0000 7f1a87400640 1 -- 172.21.15.47:0/3753944264 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8000cd00 con 0x7f1a901016b0 2024-03-21T11:44:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.919+0000 7f1a87400640 1 -- 172.21.15.47:0/3753944264 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a800035f0 con 0x7f1a901016b0 2024-03-21T11:44:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.919+0000 7f1a87400640 1 -- 172.21.15.47:0/3753944264 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80010690 con 0x7f1a901016b0 2024-03-21T11:44:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.919+0000 7f1a95c00640 1 -- 172.21.15.47:0/3753944264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a90199850 con 0x7f1a901016b0 2024-03-21T11:44:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.919+0000 7f1a95c00640 1 -- 172.21.15.47:0/3753944264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a90199dd0 con 0x7f1a901016b0 2024-03-21T11:44:56.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.923+0000 7f1a87400640 1 -- 172.21.15.47:0/3753944264 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1a800037a0 con 0x7f1a901016b0 2024-03-21T11:44:56.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.926+0000 7f1a98998740 1 Processor -- start 2024-03-21T11:44:56.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.926+0000 7f1a98998740 1 -- start start 2024-03-21T11:44:56.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.927+0000 7f1a98998740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153cae80 0x55af153cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.927+0000 7f1a98998740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cb870 0x55af153d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.927+0000 7f1a98998740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153d09e0 0x55af153d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.927+0000 7f1a98998740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af15330250 con 0x55af153d09e0 2024-03-21T11:44:56.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.927+0000 7f1a98998740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af152d8a30 con 0x55af153cae80 2024-03-21T11:44:56.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.927+0000 7f1a98998740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af1532c2f0 con 0x55af153cb870 2024-03-21T11:44:56.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.927+0000 7f1a8ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cb870 0x55af153d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:56.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.927+0000 7f1a8ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cb870 0x55af153d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55646/0 (socket says 172.21.15.47:55646) 2024-03-21T11:44:56.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.927+0000 7f1a8ea00640 1 -- 172.21.15.47:0/3406076196 learned_addr learned my addr 172.21.15.47:0/3406076196 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:56.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.928+0000 7f1a8ea00640 1 -- 172.21.15.47:0/3406076196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153cae80 msgr2=0x55af153cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:56.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.928+0000 7f1a8ea00640 1 --2- 172.21.15.47:0/3406076196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153cae80 0x55af153cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.928+0000 7f1a8ea00640 1 -- 172.21.15.47:0/3406076196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153d09e0 msgr2=0x55af153d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:56.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.928+0000 7f1a8ea00640 1 --2- 172.21.15.47:0/3406076196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153d09e0 0x55af153d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.928+0000 7f1a87400640 1 --2- 172.21.15.47:0/3753944264 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a740a30a0 0x7f1a740a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.928+0000 7f1a8ea00640 1 -- 172.21.15.47:0/3406076196 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af152c3690 con 0x55af153cb870 2024-03-21T11:44:56.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.929+0000 7f1a87400640 1 -- 172.21.15.47:0/3753944264 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1a800cfad0 con 0x7f1a901016b0 2024-03-21T11:44:56.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.929+0000 7f1a8f400640 1 --2- 172.21.15.47:0/3753944264 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a740a30a0 0x7f1a740a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:56.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.930+0000 7f1a8ea00640 1 --2- 172.21.15.47:0/3406076196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cb870 0x55af153d0370 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f1a8009f470 tx=0x7f1a800981d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6ddb35eaee1ed2b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:56.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.930+0000 7f1a7be00640 1 -- 172.21.15.47:0/3406076196 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8009bc40 con 0x55af153cb870 2024-03-21T11:44:56.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.930+0000 7f1a7be00640 1 -- 172.21.15.47:0/3406076196 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a8009c570 con 0x55af153cb870 2024-03-21T11:44:56.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.930+0000 7f1a7be00640 1 -- 172.21.15.47:0/3406076196 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8009f4a0 con 0x55af153cb870 2024-03-21T11:44:56.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.932+0000 7f1a98998740 1 -- 172.21.15.47:0/3406076196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cb870 msgr2=0x55af153d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.932+0000 7f1a98998740 1 --2- 172.21.15.47:0/3406076196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cb870 0x55af153d0370 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f1a8009f470 tx=0x7f1a800981d0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.932+0000 7f1a8f400640 1 --2- 172.21.15.47:0/3753944264 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a740a30a0 0x7f1a740a5560 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1a88006010 tx=0x7f1a88032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.932+0000 7f1a98998740 1 -- 172.21.15.47:0/3406076196 shutdown_connections 2024-03-21T11:44:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.932+0000 7f1a98998740 1 --2- 172.21.15.47:0/3406076196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153d09e0 0x55af153d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.932+0000 7f1a98998740 1 --2- 172.21.15.47:0/3406076196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cb870 0x55af153d0370 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.932+0000 7f1a98998740 1 --2- 172.21.15.47:0/3406076196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153cae80 0x55af153cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.932+0000 7f1a98998740 1 -- 172.21.15.47:0/3406076196 >> 172.21.15.47:0/3406076196 conn(0x55af15393200 msgr2=0x55af153c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:56.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.934+0000 7f1a98998740 1 -- 172.21.15.47:0/3406076196 shutdown_connections 2024-03-21T11:44:56.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.934+0000 7f1a98998740 1 -- 172.21.15.47:0/3406076196 wait complete. 2024-03-21T11:44:56.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.935+0000 7f1a98998740 1 -- start start 2024-03-21T11:44:56.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.935+0000 7f1a98998740 1 Processor -- start 2024-03-21T11:44:56.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.937+0000 7f1a98998740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153cae80 0x55af153d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.937+0000 7f1a98998740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cb870 0x55af153d5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.937+0000 7f1a98998740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153d09e0 0x55af153dc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.937+0000 7f1a98998740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af153d38d0 con 0x55af153d09e0 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.937+0000 7f1a98998740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af152ebe80 con 0x55af153cae80 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.937+0000 7f1a98998740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af153d3a50 con 0x55af153cb870 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.937+0000 7f1a98998740 10 client.0 _ll_register_callbacks cb 0x55af152fe130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.937+0000 7f1a8e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cb870 0x55af153d5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.937+0000 7f1a8ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153cae80 0x55af153d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.937+0000 7f1a8ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153cae80 0x55af153d4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43608/0 (socket says 172.21.15.47:43608) 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.937+0000 7f1a8ea00640 1 -- 172.21.15.47:0/2122148720 learned_addr learned my addr 172.21.15.47:0/2122148720 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.937+0000 7f1a8e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cb870 0x55af153d5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55648/0 (socket says 172.21.15.47:55648) 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.938+0000 7f1a8f400640 1 --2- 172.21.15.47:0/2122148720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153d09e0 0x55af153dc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.938+0000 7f1a8e000640 1 -- 172.21.15.47:0/2122148720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153cae80 msgr2=0x55af153d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.938+0000 7f1a8e000640 1 --2- 172.21.15.47:0/2122148720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153cae80 0x55af153d4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.938+0000 7f1a8e000640 1 -- 172.21.15.47:0/2122148720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153d09e0 msgr2=0x55af153dc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.938+0000 7f1a8e000640 1 --2- 172.21.15.47:0/2122148720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153d09e0 0x55af153dc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.938+0000 7f1a8e000640 1 -- 172.21.15.47:0/2122148720 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af1510e7d0 con 0x55af153cb870 2024-03-21T11:44:56.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.940+0000 7f1a8e000640 1 --2- 172.21.15.47:0/2122148720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cb870 0x55af153d5360 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1a9010d7d0 tx=0x7f1a7c00b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=180c68774d0bb35d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:56.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.940+0000 7f1a78c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.941+0000 7f1a78c00640 1 -- 172.21.15.47:0/2122148720 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c015020 con 0x55af153cb870 2024-03-21T11:44:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.942+0000 7f1a78c00640 1 -- 172.21.15.47:0/2122148720 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1a7c011420 con 0x55af153cb870 2024-03-21T11:44:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.942+0000 7f1a98998740 1 -- 172.21.15.47:0/2122148720 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a8009c570 con 0x55af153cb870 2024-03-21T11:44:56.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.943+0000 7f1a78c00640 1 -- 172.21.15.47:0/2122148720 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a7c011660 con 0x55af153cb870 2024-03-21T11:44:56.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.943+0000 7f1a98998740 10 client.32347 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:56.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.943+0000 7f1a98998740 1 -- 172.21.15.47:0/2122148720 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55af150443f0 con 0x55af153cb870 2024-03-21T11:44:56.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.944+0000 7f1a78c00640 1 -- 172.21.15.47:0/2122148720 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a8009c570 con 0x55af153cb870 2024-03-21T11:44:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.945+0000 7f1a78c00640 1 -- 172.21.15.47:0/2122148720 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1a7c022020 con 0x55af153cb870 2024-03-21T11:44:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.945+0000 7f1a98998740 10 client.32347 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.945+0000 7f1a98998740 10 client.32347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.945+0000 7f1a98998740 20 client.32347 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.945+0000 7f1a98998740 1 --2- 172.21.15.47:0/2122148720 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55af153e0c40 0x55af153e30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.945+0000 7f1a98998740 4 client.32347 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.945+0000 7f1a98998740 1 -- 172.21.15.47:0/2122148720 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af1533d790 con 0x55af153e0c40 2024-03-21T11:44:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.945+0000 7f1a8ea00640 1 --2- 172.21.15.47:0/2122148720 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55af153e0c40 0x55af153e30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:56.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.948+0000 7f1a8ea00640 1 --2- 172.21.15.47:0/2122148720 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55af153e0c40 0x55af153e30c0 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff275542e20384a server_cookie=489f9ea7ac54e402 in_seq=0 out_seq=0 2024-03-21T11:44:56.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.948+0000 7f1a78c00640 10 client.32347 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:56.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.949+0000 7f1a78c00640 1 -- 172.21.15.47:0/2122148720 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55af1533d790 con 0x55af153e0c40 2024-03-21T11:44:56.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.949+0000 7f1a78c00640 10 client.32347 handle_command_reply: tid=0 2024-03-21T11:44:56.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.950+0000 7f1a98998740 1 client.32347 shutdown 2024-03-21T11:44:56.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.950+0000 7f1a98998740 1 -- 172.21.15.47:0/2122148720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cb870 msgr2=0x55af153d5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:56.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.950+0000 7f1a98998740 1 --2- 172.21.15.47:0/2122148720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cb870 0x55af153d5360 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1a9010d7d0 tx=0x7f1a7c00b4b0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.951+0000 7f1a98998740 1 -- 172.21.15.47:0/2122148720 shutdown_connections 2024-03-21T11:44:56.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.951+0000 7f1a98998740 1 --2- 172.21.15.47:0/2122148720 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55af153e0c40 0x55af153e30c0 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.951+0000 7f1a98998740 1 --2- 172.21.15.47:0/2122148720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153d09e0 0x55af153dc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.951+0000 7f1a98998740 1 --2- 172.21.15.47:0/2122148720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cb870 0x55af153d5360 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.951+0000 7f1a98998740 1 --2- 172.21.15.47:0/2122148720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153cae80 0x55af153d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.951+0000 7f1a78c00640 0 client.32347 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:56.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.951+0000 7f1a98998740 1 -- 172.21.15.47:0/2122148720 >> 172.21.15.47:0/2122148720 conn(0x55af15393200 msgr2=0x55af153c5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:56.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.951+0000 7f1a98998740 1 -- 172.21.15.47:0/2122148720 shutdown_connections 2024-03-21T11:44:56.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.951+0000 7f1a98998740 1 -- 172.21.15.47:0/2122148720 wait complete. 2024-03-21T11:44:56.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.952+0000 7f1a98998740 20 client.32347 trim_cache size 0 max 16384 2024-03-21T11:44:56.974 INFO:tasks.workunit.client.0.smithi047.stdout: 10 200810 22.59 MB/sec execute 340 sec latency 98.627 ms 2024-03-21T11:44:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.985+0000 7f1a98998740 1 Processor -- start 2024-03-21T11:44:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.985+0000 7f1a98998740 1 -- start start 2024-03-21T11:44:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.986+0000 7f1a98998740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153cae80 0x55af153cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.986+0000 7f1a98998740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153cb870 0x55af153dc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.986+0000 7f1a98998740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153d09e0 0x55af153e7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.986+0000 7f1a98998740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a901b9160 con 0x55af153cb870 2024-03-21T11:44:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.986+0000 7f1a98998740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a901baae0 con 0x55af153cae80 2024-03-21T11:44:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.986+0000 7f1a98998740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a901b97c0 con 0x55af153d09e0 2024-03-21T11:44:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.986+0000 7f1a8e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153cb870 0x55af153dc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.986+0000 7f1a8f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153d09e0 0x55af153e7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.986+0000 7f1a8f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153d09e0 0x55af153e7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55664/0 (socket says 172.21.15.47:55664) 2024-03-21T11:44:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.986+0000 7f1a8e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153cb870 0x55af153dc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54614/0 (socket says 172.21.15.47:54614) 2024-03-21T11:44:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.986+0000 7f1a8f400640 1 -- 172.21.15.47:0/20870986 learned_addr learned my addr 172.21.15.47:0/20870986 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.987+0000 7f1a8f400640 1 -- 172.21.15.47:0/20870986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153cae80 msgr2=0x55af153cc9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:44:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.987+0000 7f1a8f400640 1 --2- 172.21.15.47:0/20870986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153cae80 0x55af153cc9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.987+0000 7f1a8f400640 1 -- 172.21.15.47:0/20870986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153cb870 msgr2=0x55af153dc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.987+0000 7f1a8f400640 1 --2- 172.21.15.47:0/20870986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153cb870 0x55af153dc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.987+0000 7f1a8f400640 1 -- 172.21.15.47:0/20870986 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af150040f0 con 0x55af153d09e0 2024-03-21T11:44:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.987+0000 7f1a8f400640 1 --2- 172.21.15.47:0/20870986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153d09e0 0x55af153e7980 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1a88036000 tx=0x7f1a8803b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9d5ea8b8eddea39 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.988+0000 7f1a7b400640 1 -- 172.21.15.47:0/20870986 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8804f020 con 0x55af153d09e0 2024-03-21T11:44:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.988+0000 7f1a7b400640 1 -- 172.21.15.47:0/20870986 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a88003e50 con 0x55af153d09e0 2024-03-21T11:44:56.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.988+0000 7f1a7b400640 1 -- 172.21.15.47:0/20870986 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88039040 con 0x55af153d09e0 2024-03-21T11:44:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.989+0000 7f1a98998740 1 -- 172.21.15.47:0/20870986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153d09e0 msgr2=0x55af153e7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.989+0000 7f1a98998740 1 --2- 172.21.15.47:0/20870986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153d09e0 0x55af153e7980 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1a88036000 tx=0x7f1a8803b000 comp rx=0 tx=0).stop 2024-03-21T11:44:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.991+0000 7f1a98998740 1 -- 172.21.15.47:0/20870986 shutdown_connections 2024-03-21T11:44:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.991+0000 7f1a98998740 1 --2- 172.21.15.47:0/20870986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153d09e0 0x55af153e7980 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.991+0000 7f1a98998740 1 --2- 172.21.15.47:0/20870986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153cb870 0x55af153dc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.991+0000 7f1a98998740 1 --2- 172.21.15.47:0/20870986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153cae80 0x55af153cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.991+0000 7f1a98998740 1 -- 172.21.15.47:0/20870986 >> 172.21.15.47:0/20870986 conn(0x55af15393200 msgr2=0x55af153cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.991+0000 7f1a98998740 1 -- 172.21.15.47:0/20870986 shutdown_connections 2024-03-21T11:44:56.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.991+0000 7f1a98998740 1 -- 172.21.15.47:0/20870986 wait complete. 2024-03-21T11:44:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.992+0000 7f1a98998740 1 -- start start 2024-03-21T11:44:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.992+0000 7f1a98998740 1 Processor -- start 2024-03-21T11:44:56.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.993+0000 7f1a98998740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cae80 0x55af153dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.993+0000 7f1a98998740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153cb870 0x55af153dc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.993+0000 7f1a98998740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153d09e0 0x55af153da1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:56.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.993+0000 7f1a98998740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a901ba480 con 0x55af153cb870 2024-03-21T11:44:56.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.993+0000 7f1a98998740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a901b9e20 con 0x55af153d09e0 2024-03-21T11:44:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.993+0000 7f1a98998740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a901b7320 con 0x55af153cae80 2024-03-21T11:44:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.993+0000 7f1a8ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cae80 0x55af153dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.993+0000 7f1a8ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cae80 0x55af153dbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55678/0 (socket says 172.21.15.47:55678) 2024-03-21T11:44:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.993+0000 7f1a8ea00640 1 -- 172.21.15.47:0/1047442858 learned_addr learned my addr 172.21.15.47:0/1047442858 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.994+0000 7f1a8f400640 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153d09e0 0x55af153da1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.994+0000 7f1a98998740 10 client.0 _ll_register_callbacks cb 0x7f1a7c006e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.994+0000 7f1a8ea00640 1 -- 172.21.15.47:0/1047442858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153d09e0 msgr2=0x55af153da1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.995+0000 7f1a8e000640 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153cb870 0x55af153dc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.995+0000 7f1a8ea00640 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153d09e0 0x55af153da1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.996+0000 7f1a8ea00640 1 -- 172.21.15.47:0/1047442858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153cb870 msgr2=0x55af153dc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:56.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.996+0000 7f1a8ea00640 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153cb870 0x55af153dc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.996+0000 7f1a8ea00640 1 -- 172.21.15.47:0/1047442858 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af153da920 con 0x55af153cae80 2024-03-21T11:44:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.996+0000 7f1a98998740 1 -- 172.21.15.47:0/1047442858 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a88003e50 con 0x55af153cae80 2024-03-21T11:44:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.996+0000 7f1a8f400640 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153d09e0 0x55af153da1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:44:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.996+0000 7f1a8e000640 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153cb870 0x55af153dc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:44:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.997+0000 7f1a8ea00640 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cae80 0x55af153dbbd0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f1a8009bb70 tx=0x7f1a800a0050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aece8348b4e2ccf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:56.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.997+0000 7f1a78c00640 10 client.32353 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.997+0000 7f1a78c00640 1 -- 172.21.15.47:0/1047442858 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8009b7d0 con 0x55af153cae80 2024-03-21T11:44:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.998+0000 7f1a78c00640 1 -- 172.21.15.47:0/1047442858 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1a800d30c0 con 0x55af153cae80 2024-03-21T11:44:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.998+0000 7f1a78c00640 1 -- 172.21.15.47:0/1047442858 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a8009b9b0 con 0x55af153cae80 2024-03-21T11:44:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.998+0000 7f1a98998740 10 client.32353 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:56.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.998+0000 7f1a98998740 1 -- 172.21.15.47:0/1047442858 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=40}) v3 -- 0x55af1543cf90 con 0x55af153cae80 2024-03-21T11:44:57.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.999+0000 7f1a78c00640 1 -- 172.21.15.47:0/1047442858 <== mon.1 v2:172.21.15.102:3300/0 4 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1a800984e0 con 0x55af153cae80 2024-03-21T11:44:57.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.999+0000 7f1a98998740 10 client.32353 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:57.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.999+0000 7f1a98998740 10 client.32353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:44:57.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.999+0000 7f1a98998740 20 client.32353 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:57.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.999+0000 7f1a98998740 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55af1543d210 0x55af1543f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.999+0000 7f1a98998740 4 client.32353 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:44:57.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:56.999+0000 7f1a98998740 1 -- 172.21.15.47:0/1047442858 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55af15337ae0 con 0x55af1543d210 2024-03-21T11:44:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.001+0000 7f1a8e000640 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55af1543d210 0x55af1543f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.002+0000 7f1a8e000640 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55af1543d210 0x55af1543f600 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32bbce4844d971cc server_cookie=3ff1ccf1b7c5d8d8 in_seq=0 out_seq=0 2024-03-21T11:44:57.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.002+0000 7f1a78c00640 10 client.32353 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.003+0000 7f1a78c00640 1 -- 172.21.15.47:0/1047442858 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55af15337ae0 con 0x55af1543d210 2024-03-21T11:44:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.003+0000 7f1a78c00640 10 client.32353 handle_command_reply: tid=0 2024-03-21T11:44:57.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.003+0000 7f1a98998740 1 client.32353 shutdown 2024-03-21T11:44:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.004+0000 7f1a98998740 1 -- 172.21.15.47:0/1047442858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cae80 msgr2=0x55af153dbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.004+0000 7f1a98998740 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cae80 0x55af153dbbd0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f1a8009bb70 tx=0x7f1a800a0050 comp rx=0 tx=0).stop 2024-03-21T11:44:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.004+0000 7f1a98998740 1 -- 172.21.15.47:0/1047442858 shutdown_connections 2024-03-21T11:44:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.004+0000 7f1a98998740 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55af1543d210 0x55af1543f600 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.004+0000 7f1a98998740 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55af153d09e0 0x55af153da1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.004+0000 7f1a98998740 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55af153cb870 0x55af153dc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.004+0000 7f1a98998740 1 --2- 172.21.15.47:0/1047442858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af153cae80 0x55af153dbbd0 secure :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f1a8009bb70 tx=0x7f1a800a0050 comp rx=0 tx=0).stop 2024-03-21T11:44:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.004+0000 7f1a98998740 1 -- 172.21.15.47:0/1047442858 >> 172.21.15.47:0/1047442858 conn(0x55af15393200 msgr2=0x55af153ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:57.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.004+0000 7f1a78c00640 0 client.32353 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:44:57.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.005+0000 7f1a98998740 1 -- 172.21.15.47:0/1047442858 shutdown_connections 2024-03-21T11:44:57.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.005+0000 7f1a98998740 1 -- 172.21.15.47:0/1047442858 wait complete. 2024-03-21T11:44:57.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.005+0000 7f1a98998740 20 client.32353 trim_cache size 0 max 16384 2024-03-21T11:44:57.008 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:44:57.008 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:44:57.008 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:44:57.008 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:44:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.010+0000 7f1a95c00640 1 -- 172.21.15.47:0/3753944264 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a740a30a0 msgr2=0x7f1a740a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.010+0000 7f1a95c00640 1 --2- 172.21.15.47:0/3753944264 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a740a30a0 0x7f1a740a5560 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1a88006010 tx=0x7f1a88032590 comp rx=0 tx=0).stop 2024-03-21T11:44:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.010+0000 7f1a95c00640 1 -- 172.21.15.47:0/3753944264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a901016b0 msgr2=0x7f1a9010f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:57.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.010+0000 7f1a95c00640 1 --2- 172.21.15.47:0/3753944264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a901016b0 0x7f1a9010f850 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000e9c0 tx=0x7f1a8000ee80 comp rx=0 tx=0).stop 2024-03-21T11:44:57.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.012+0000 7f1a95c00640 1 -- 172.21.15.47:0/3753944264 shutdown_connections 2024-03-21T11:44:57.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.012+0000 7f1a95c00640 1 --2- 172.21.15.47:0/3753944264 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1a740a30a0 0x7f1a740a5560 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.012+0000 7f1a95c00640 1 --2- 172.21.15.47:0/3753944264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a90103260 0x7f1a9010c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.012+0000 7f1a95c00640 1 --2- 172.21.15.47:0/3753944264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a901028a0 0x7f1a9010fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.012+0000 7f1a95c00640 1 --2- 172.21.15.47:0/3753944264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a901016b0 0x7f1a9010f850 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.012+0000 7f1a95c00640 1 -- 172.21.15.47:0/3753944264 >> 172.21.15.47:0/3753944264 conn(0x7f1a900fcc40 msgr2=0x7f1a900fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:57.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.013+0000 7f1a95c00640 1 -- 172.21.15.47:0/3753944264 shutdown_connections 2024-03-21T11:44:57.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.013+0000 7f1a95c00640 1 -- 172.21.15.47:0/3753944264 wait complete. 2024-03-21T11:44:57.044 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:44:57.044 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:44:57.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:57 smithi047 ceph-mon[31351]: pgmap v1048: 129 pgs: 129 active+clean; 634 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.1 MiB/s wr, 597 op/s 2024-03-21T11:44:57.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.329+0000 7f044f000640 1 -- 172.21.15.47:0/2651284365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0448077c40 msgr2=0x7f04480784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:57.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.329+0000 7f044f000640 1 --2- 172.21.15.47:0/2651284365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0448077c40 0x7f04480784e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0434009860 tx=0x7f043402bb70 comp rx=0 tx=0).stop 2024-03-21T11:44:57.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.330+0000 7f044f000640 1 -- 172.21.15.47:0/2651284365 shutdown_connections 2024-03-21T11:44:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.330+0000 7f044f000640 1 --2- 172.21.15.47:0/2651284365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0448077c40 0x7f04480784e0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.330+0000 7f044f000640 1 --2- 172.21.15.47:0/2651284365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04480772a0 0x7f0448077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.330+0000 7f044f000640 1 --2- 172.21.15.47:0/2651284365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0448076060 0x7f0448076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.330+0000 7f044f000640 1 -- 172.21.15.47:0/2651284365 >> 172.21.15.47:0/2651284365 conn(0x7f04480fee50 msgr2=0x7f0448101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.331+0000 7f044f000640 1 -- 172.21.15.47:0/2651284365 shutdown_connections 2024-03-21T11:44:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.331+0000 7f044f000640 1 -- 172.21.15.47:0/2651284365 wait complete. 2024-03-21T11:44:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.332+0000 7f044f000640 1 Processor -- start 2024-03-21T11:44:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.333+0000 7f044f000640 1 -- start start 2024-03-21T11:44:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.333+0000 7f044f000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0448076060 0x7f0448110de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.333+0000 7f044f000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04480772a0 0x7f0448111320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.333+0000 7f044f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0448077c40 0x7f044810f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.333+0000 7f044f000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0448103d20 con 0x7f0448076060 2024-03-21T11:44:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.333+0000 7f044f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0448103ba0 con 0x7f0448077c40 2024-03-21T11:44:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.333+0000 7f044f000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0448103ea0 con 0x7f04480772a0 2024-03-21T11:44:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.333+0000 7f0447400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04480772a0 0x7f0448111320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.334+0000 7f0447400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04480772a0 0x7f0448111320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55694/0 (socket says 172.21.15.47:55694) 2024-03-21T11:44:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.334+0000 7f0447400640 1 -- 172.21.15.47:0/1244817049 learned_addr learned my addr 172.21.15.47:0/1244817049 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.334+0000 7f044d000640 1 --2- 172.21.15.47:0/1244817049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0448077c40 0x7f044810f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.334+0000 7f0447400640 1 -- 172.21.15.47:0/1244817049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0448077c40 msgr2=0x7f044810f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:57.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.334+0000 7f0447400640 1 --2- 172.21.15.47:0/1244817049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0448077c40 0x7f044810f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.334+0000 7f0447400640 1 -- 172.21.15.47:0/1244817049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0448076060 msgr2=0x7f0448110de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:57.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.334+0000 7f0447400640 1 --2- 172.21.15.47:0/1244817049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0448076060 0x7f0448110de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.334+0000 7f0447400640 1 -- 172.21.15.47:0/1244817049 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044810fb50 con 0x7f04480772a0 2024-03-21T11:44:57.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.334+0000 7f044d000640 1 --2- 172.21.15.47:0/1244817049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0448077c40 0x7f044810f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:57.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.335+0000 7f0447400640 1 --2- 172.21.15.47:0/1244817049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04480772a0 0x7f0448111320 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f043800e9c0 tx=0x7f043800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:57.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.335+0000 7f0444c00640 1 -- 172.21.15.47:0/1244817049 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043800ccc0 con 0x7f04480772a0 2024-03-21T11:44:57.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.336+0000 7f0444c00640 1 -- 172.21.15.47:0/1244817049 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0438005500 con 0x7f04480772a0 2024-03-21T11:44:57.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.336+0000 7f0444c00640 1 -- 172.21.15.47:0/1244817049 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438010430 con 0x7f04480772a0 2024-03-21T11:44:57.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.336+0000 7f044f000640 1 -- 172.21.15.47:0/1244817049 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044810fe40 con 0x7f04480772a0 2024-03-21T11:44:57.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.336+0000 7f044f000640 1 -- 172.21.15.47:0/1244817049 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04481aecb0 con 0x7f04480772a0 2024-03-21T11:44:57.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.338+0000 7f044f000640 1 -- 172.21.15.47:0/1244817049 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0448108f70 con 0x7f04480772a0 2024-03-21T11:44:57.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.342+0000 7f0444c00640 1 -- 172.21.15.47:0/1244817049 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f04380046c0 con 0x7f04480772a0 2024-03-21T11:44:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.344+0000 7f0444c00640 1 --2- 172.21.15.47:0/1244817049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f042c0a2fa0 0x7f042c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:57.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.345+0000 7f0444c00640 1 -- 172.21.15.47:0/1244817049 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f04380cfa80 con 0x7f04480772a0 2024-03-21T11:44:57.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.345+0000 7f0447e00640 1 --2- 172.21.15.47:0/1244817049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f042c0a2fa0 0x7f042c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:57.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.347+0000 7f0447e00640 1 --2- 172.21.15.47:0/1244817049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f042c0a2fa0 0x7f042c0a5460 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0440005c40 tx=0x7f0440005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:57.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.352+0000 7f0444c00640 1 -- 172.21.15.47:0/1244817049 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f043809c520 con 0x7f04480772a0 2024-03-21T11:44:57.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:57 smithi100 ceph-mon[36245]: pgmap v1048: 129 pgs: 129 active+clean; 634 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.1 MiB/s wr, 597 op/s 2024-03-21T11:44:57.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:57 smithi102 ceph-mon[36748]: pgmap v1048: 129 pgs: 129 active+clean; 634 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.1 MiB/s wr, 597 op/s 2024-03-21T11:44:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.737+0000 7f044f000640 1 -- 172.21.15.47:0/1244817049 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0448062340 con 0x7f04480772a0 2024-03-21T11:44:57.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.738+0000 7f0444c00640 1 -- 172.21.15.47:0/1244817049 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f043809be20 con 0x7f04480772a0 2024-03-21T11:44:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:44:57.742 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:39:47.303461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:44:57.743 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-21T11:44:57.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.741+0000 7f044f000640 1 -- 172.21.15.47:0/1244817049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f042c0a2fa0 msgr2=0x7f042c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:57.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.741+0000 7f044f000640 1 --2- 172.21.15.47:0/1244817049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f042c0a2fa0 0x7f042c0a5460 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0440005c40 tx=0x7f0440005bd0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.741+0000 7f044f000640 1 -- 172.21.15.47:0/1244817049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04480772a0 msgr2=0x7f0448111320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:57.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.741+0000 7f044f000640 1 --2- 172.21.15.47:0/1244817049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04480772a0 0x7f0448111320 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f043800e9c0 tx=0x7f043800ee80 comp rx=0 tx=0).stop 2024-03-21T11:44:57.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.741+0000 7f044f000640 1 -- 172.21.15.47:0/1244817049 shutdown_connections 2024-03-21T11:44:57.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.741+0000 7f044f000640 1 --2- 172.21.15.47:0/1244817049 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f042c0a2fa0 0x7f042c0a5460 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.741+0000 7f044f000640 1 --2- 172.21.15.47:0/1244817049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0448077c40 0x7f044810f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.741+0000 7f044f000640 1 --2- 172.21.15.47:0/1244817049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04480772a0 0x7f0448111320 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.741+0000 7f044f000640 1 --2- 172.21.15.47:0/1244817049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0448076060 0x7f0448110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:57.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.741+0000 7f044f000640 1 -- 172.21.15.47:0/1244817049 >> 172.21.15.47:0/1244817049 conn(0x7f04480fee50 msgr2=0x7f0448100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:57.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.742+0000 7f044f000640 1 -- 172.21.15.47:0/1244817049 shutdown_connections 2024-03-21T11:44:57.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:57.742+0000 7f044f000640 1 -- 172.21.15.47:0/1244817049 wait complete. 2024-03-21T11:44:57.782 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:44:57.782 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:44:57.975 INFO:tasks.workunit.client.0.smithi047.stdout: 10 201184 22.58 MB/sec execute 341 sec latency 117.273 ms 2024-03-21T11:44:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.359+0000 7f1daf800640 1 -- 172.21.15.47:0/3852379874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da81038e0 msgr2=0x7f1da8103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.359+0000 7f1daf800640 1 --2- 172.21.15.47:0/3852379874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da81038e0 0x7f1da8103d00 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009900 tx=0x7f1d9c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:44:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.360+0000 7f1daf800640 1 -- 172.21.15.47:0/3852379874 shutdown_connections 2024-03-21T11:44:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.360+0000 7f1daf800640 1 --2- 172.21.15.47:0/3852379874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da8105490 0x7f1da810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.360+0000 7f1daf800640 1 --2- 172.21.15.47:0/3852379874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1da8104ad0 0x7f1da8104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.360+0000 7f1daf800640 1 --2- 172.21.15.47:0/3852379874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da81038e0 0x7f1da8103d00 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.360+0000 7f1daf800640 1 -- 172.21.15.47:0/3852379874 >> 172.21.15.47:0/3852379874 conn(0x7f1da80fee50 msgr2=0x7f1da8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.360+0000 7f1daf800640 1 -- 172.21.15.47:0/3852379874 shutdown_connections 2024-03-21T11:44:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.360+0000 7f1daf800640 1 -- 172.21.15.47:0/3852379874 wait complete. 2024-03-21T11:44:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.362+0000 7f1daf800640 1 Processor -- start 2024-03-21T11:44:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.362+0000 7f1daf800640 1 -- start start 2024-03-21T11:44:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.362+0000 7f1daf800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da81038e0 0x7f1da819de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.362+0000 7f1daf800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1da8104ad0 0x7f1da819e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.362+0000 7f1daf800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da8105490 0x7f1da81a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.362+0000 7f1daf800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da810f5b0 con 0x7f1da8104ad0 2024-03-21T11:44:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.362+0000 7f1daf800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da810f430 con 0x7f1da81038e0 2024-03-21T11:44:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.362+0000 7f1daf800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da810f730 con 0x7f1da8105490 2024-03-21T11:44:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.363+0000 7f1dace00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da81038e0 0x7f1da819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.363+0000 7f1dace00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da81038e0 0x7f1da819de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43666/0 (socket says 172.21.15.47:43666) 2024-03-21T11:44:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.363+0000 7f1dace00640 1 -- 172.21.15.47:0/1722607443 learned_addr learned my addr 172.21.15.47:0/1722607443 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.363+0000 7f1dad800640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da8105490 0x7f1da81a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.363+0000 7f1da7e00640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1da8104ad0 0x7f1da819e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.363+0000 7f1dace00640 1 -- 172.21.15.47:0/1722607443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da8105490 msgr2=0x7f1da81a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.363+0000 7f1dace00640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da8105490 0x7f1da81a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.363+0000 7f1dace00640 1 -- 172.21.15.47:0/1722607443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1da8104ad0 msgr2=0x7f1da819e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.363+0000 7f1dace00640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1da8104ad0 0x7f1da819e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.363+0000 7f1dace00640 1 -- 172.21.15.47:0/1722607443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da81a5a30 con 0x7f1da81038e0 2024-03-21T11:44:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.363+0000 7f1da7e00640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1da8104ad0 0x7f1da819e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:44:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.364+0000 7f1dace00640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da81038e0 0x7f1da819de10 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009850 tx=0x7f1d9c02ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.364+0000 7f1da5600640 1 -- 172.21.15.47:0/1722607443 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c043020 con 0x7f1da81038e0 2024-03-21T11:44:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.364+0000 7f1daf800640 1 -- 172.21.15.47:0/1722607443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da81a5cc0 con 0x7f1da81038e0 2024-03-21T11:44:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.364+0000 7f1daf800640 1 -- 172.21.15.47:0/1722607443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da81a61b0 con 0x7f1da81038e0 2024-03-21T11:44:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.364+0000 7f1da5600640 1 -- 172.21.15.47:0/1722607443 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d9c0054e0 con 0x7f1da81038e0 2024-03-21T11:44:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.364+0000 7f1da5600640 1 -- 172.21.15.47:0/1722607443 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c003e40 con 0x7f1da81038e0 2024-03-21T11:44:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.369+0000 7f1db256c740 1 Processor -- start 2024-03-21T11:44:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.370+0000 7f1db256c740 1 -- start start 2024-03-21T11:44:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.370+0000 7f1db256c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 0x55a497d7ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.370+0000 7f1da5600640 1 -- 172.21.15.47:0/1722607443 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1d9c046020 con 0x7f1da81038e0 2024-03-21T11:44:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.370+0000 7f1db256c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 0x55a497d7be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.370+0000 7f1db256c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7c3a0 0x55a497d82c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.370+0000 7f1db256c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a497cdf2c0 con 0x55a497d7c3a0 2024-03-21T11:44:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.370+0000 7f1db256c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a497c87c90 con 0x55a497d7b9e0 2024-03-21T11:44:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.370+0000 7f1db256c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a497cdb360 con 0x55a497d7a810 2024-03-21T11:44:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.371+0000 7f1da7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7c3a0 0x55a497d82c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.371+0000 7f1da7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7c3a0 0x55a497d82c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54684/0 (socket says 172.21.15.47:54684) 2024-03-21T11:44:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.371+0000 7f1da7e00640 1 -- 172.21.15.47:0/2383952123 learned_addr learned my addr 172.21.15.47:0/2383952123 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.372+0000 7f1dace00640 1 --2- 172.21.15.47:0/2383952123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 0x55a497d7be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.372+0000 7f1dad800640 1 --2- 172.21.15.47:0/2383952123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 0x55a497d7ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.372+0000 7f1dace00640 1 -- 172.21.15.47:0/2383952123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 msgr2=0x55a497d7ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.372+0000 7f1dace00640 1 --2- 172.21.15.47:0/2383952123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 0x55a497d7ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.372+0000 7f1dace00640 1 -- 172.21.15.47:0/2383952123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7c3a0 msgr2=0x55a497d82c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.372+0000 7f1dace00640 1 --2- 172.21.15.47:0/2383952123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7c3a0 0x55a497d82c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.372+0000 7f1dace00640 1 -- 172.21.15.47:0/2383952123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a497cac080 con 0x55a497d7b9e0 2024-03-21T11:44:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.373+0000 7f1dace00640 1 --2- 172.21.15.47:0/2383952123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 0x55a497d7be60 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0ef060 tx=0x7f1d9c0f7b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e05850ea0af2d9e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:58.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.373+0000 7f1da5600640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1d900a2fa0 0x7f1d900a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.373+0000 7f1d9a000640 1 -- 172.21.15.47:0/2383952123 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c100070 con 0x55a497d7b9e0 2024-03-21T11:44:58.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.373+0000 7f1da5600640 1 -- 172.21.15.47:0/1722607443 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1d9c0f1920 con 0x7f1da81038e0 2024-03-21T11:44:58.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.373+0000 7f1d9a000640 1 -- 172.21.15.47:0/2383952123 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d9c005690 con 0x55a497d7b9e0 2024-03-21T11:44:58.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.373+0000 7f1d9a000640 1 -- 172.21.15.47:0/2383952123 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0bd800 con 0x55a497d7b9e0 2024-03-21T11:44:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.374+0000 7f1dad800640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1d900a2fa0 0x7f1d900a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.374+0000 7f1db256c740 1 -- 172.21.15.47:0/2383952123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 msgr2=0x55a497d7be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.374+0000 7f1db256c740 1 --2- 172.21.15.47:0/2383952123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 0x55a497d7be60 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0ef060 tx=0x7f1d9c0f7b40 comp rx=0 tx=0).stop 2024-03-21T11:44:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.374+0000 7f1db256c740 1 -- 172.21.15.47:0/2383952123 shutdown_connections 2024-03-21T11:44:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.374+0000 7f1db256c740 1 --2- 172.21.15.47:0/2383952123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7c3a0 0x55a497d82c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.374+0000 7f1db256c740 1 --2- 172.21.15.47:0/2383952123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 0x55a497d7be60 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.374+0000 7f1db256c740 1 --2- 172.21.15.47:0/2383952123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 0x55a497d7ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.374+0000 7f1db256c740 1 -- 172.21.15.47:0/2383952123 >> 172.21.15.47:0/2383952123 conn(0x55a497d75c00 msgr2=0x55a497d76000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:58.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.374+0000 7f1db256c740 1 -- 172.21.15.47:0/2383952123 shutdown_connections 2024-03-21T11:44:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.375+0000 7f1db256c740 1 -- 172.21.15.47:0/2383952123 wait complete. 2024-03-21T11:44:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.375+0000 7f1dad800640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1d900a2fa0 0x7f1d900a5460 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x55a497d7b8e0 tx=0x7f1da0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:58.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.375+0000 7f1db256c740 1 -- start start 2024-03-21T11:44:58.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.376+0000 7f1db256c740 1 Processor -- start 2024-03-21T11:44:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.376+0000 7f1db256c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 0x55a497d84550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.376+0000 7f1db256c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7b9e0 0x55a497d84a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.376+0000 7f1db256c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7c3a0 0x55a497d8bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.376+0000 7f1db256c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a497d83170 con 0x55a497d7b9e0 2024-03-21T11:44:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.376+0000 7f1db256c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a497c9b0d0 con 0x55a497d7c3a0 2024-03-21T11:44:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.376+0000 7f1db256c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a497d832f0 con 0x55a497d7a810 2024-03-21T11:44:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.376+0000 7f1db256c740 10 client.0 _ll_register_callbacks cb 0x55a497c9fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.377+0000 7f1da7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7b9e0 0x55a497d84a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.377+0000 7f1dad800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7c3a0 0x55a497d8bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.377+0000 7f1da7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7b9e0 0x55a497d84a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54694/0 (socket says 172.21.15.47:54694) 2024-03-21T11:44:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.377+0000 7f1da7e00640 1 -- 172.21.15.47:0/898666012 learned_addr learned my addr 172.21.15.47:0/898666012 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.377+0000 7f1dace00640 1 --2- 172.21.15.47:0/898666012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 0x55a497d84550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.377+0000 7f1dad800640 1 -- 172.21.15.47:0/898666012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 msgr2=0x55a497d84550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.377+0000 7f1dad800640 1 --2- 172.21.15.47:0/898666012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 0x55a497d84550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.377+0000 7f1dad800640 1 -- 172.21.15.47:0/898666012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7b9e0 msgr2=0x55a497d84a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.377+0000 7f1dad800640 1 --2- 172.21.15.47:0/898666012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7b9e0 0x55a497d84a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.377+0000 7f1dad800640 1 -- 172.21.15.47:0/898666012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4979f33f0 con 0x55a497d7c3a0 2024-03-21T11:44:58.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.378+0000 7f1dad800640 1 --2- 172.21.15.47:0/898666012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7c3a0 0x55a497d8bad0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1da0005fd0 tx=0x7f1da0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bec9e38275f1151 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:58.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.378+0000 7f1d8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:44:58.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.378+0000 7f1d8ea00640 1 -- 172.21.15.47:0/898666012 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0003970 con 0x55a497d7c3a0 2024-03-21T11:44:58.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.378+0000 7f1db256c740 1 -- 172.21.15.47:0/898666012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a497cec800 con 0x55a497d7c3a0 2024-03-21T11:44:58.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.378+0000 7f1d8ea00640 1 -- 172.21.15.47:0/898666012 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1da0018030 con 0x55a497d7c3a0 2024-03-21T11:44:58.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.379+0000 7f1d8ea00640 1 -- 172.21.15.47:0/898666012 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1da0034050 con 0x55a497d7c3a0 2024-03-21T11:44:58.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.379+0000 7f1db256c740 10 client.32402 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:58.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.379+0000 7f1db256c740 1 -- 172.21.15.47:0/898666012 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a497abd7d0 con 0x55a497d7c3a0 2024-03-21T11:44:58.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.379+0000 7f1d8ea00640 1 -- 172.21.15.47:0/898666012 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a497cec800 con 0x55a497d7c3a0 2024-03-21T11:44:58.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.380+0000 7f1d8ea00640 1 -- 172.21.15.47:0/898666012 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1da0029420 con 0x55a497d7c3a0 2024-03-21T11:44:58.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.380+0000 7f1db256c740 10 client.32402 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:58.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.380+0000 7f1db256c740 10 client.32402 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:44:58.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.380+0000 7f1db256c740 20 client.32402 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:58.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.380+0000 7f1db256c740 1 --2- 172.21.15.47:0/898666012 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a497d900e0 0x55a497d92500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.380+0000 7f1db256c740 4 client.32402 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:58.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.380+0000 7f1db256c740 1 -- 172.21.15.47:0/898666012 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a497ce6b50 con 0x55a497d900e0 2024-03-21T11:44:58.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.384+0000 7f1dace00640 1 --2- 172.21.15.47:0/898666012 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a497d900e0 0x55a497d92500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.385+0000 7f1dace00640 1 --2- 172.21.15.47:0/898666012 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a497d900e0 0x55a497d92500 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=82a78364fcaafbc2 server_cookie=af164d9a062afd4d in_seq=0 out_seq=0 2024-03-21T11:44:58.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.386+0000 7f1d8ea00640 10 client.32402 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:44:58.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.389+0000 7f1d8ea00640 1 -- 172.21.15.47:0/898666012 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a497ce6b50 con 0x55a497d900e0 2024-03-21T11:44:58.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.389+0000 7f1d8ea00640 10 client.32402 handle_command_reply: tid=0 2024-03-21T11:44:58.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.389+0000 7f1db256c740 1 client.32402 shutdown 2024-03-21T11:44:58.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:58 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1244817049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:44:58.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.392+0000 7f1db256c740 1 -- 172.21.15.47:0/898666012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7c3a0 msgr2=0x55a497d8bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.392+0000 7f1db256c740 1 --2- 172.21.15.47:0/898666012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7c3a0 0x55a497d8bad0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1da0005fd0 tx=0x7f1da0004ab0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.393+0000 7f1db256c740 1 -- 172.21.15.47:0/898666012 shutdown_connections 2024-03-21T11:44:58.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.393+0000 7f1db256c740 1 --2- 172.21.15.47:0/898666012 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a497d900e0 0x55a497d92500 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.393+0000 7f1db256c740 1 --2- 172.21.15.47:0/898666012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7c3a0 0x55a497d8bad0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.393+0000 7f1db256c740 1 --2- 172.21.15.47:0/898666012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7b9e0 0x55a497d84a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.393+0000 7f1db256c740 1 --2- 172.21.15.47:0/898666012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 0x55a497d84550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.393+0000 7f1db256c740 1 -- 172.21.15.47:0/898666012 >> 172.21.15.47:0/898666012 conn(0x55a497d75c00 msgr2=0x55a497d74eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:58.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.393+0000 7f1d8ea00640 0 client.32402 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:44:58.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.393+0000 7f1db256c740 1 -- 172.21.15.47:0/898666012 shutdown_connections 2024-03-21T11:44:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.394+0000 7f1db256c740 1 -- 172.21.15.47:0/898666012 wait complete. 2024-03-21T11:44:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.394+0000 7f1db256c740 20 client.32402 trim_cache size 0 max 16384 2024-03-21T11:44:58.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.421+0000 7f1db256c740 1 Processor -- start 2024-03-21T11:44:58.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.421+0000 7f1db256c740 1 -- start start 2024-03-21T11:44:58.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.421+0000 7f1db256c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7a810 0x55a497d7f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.421+0000 7f1db256c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 0x55a497d8ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.421+0000 7f1db256c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7c3a0 0x55a497d74620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.422+0000 7f1db256c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8800f9a0 con 0x55a497d7a810 2024-03-21T11:44:58.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.422+0000 7f1db256c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d88011320 con 0x55a497d7b9e0 2024-03-21T11:44:58.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.422+0000 7f1db256c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d88010000 con 0x55a497d7c3a0 2024-03-21T11:44:58.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.422+0000 7f1dace00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7a810 0x55a497d7f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.422+0000 7f1dad800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7c3a0 0x55a497d74620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.422+0000 7f1dad800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7c3a0 0x55a497d74620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55734/0 (socket says 172.21.15.47:55734) 2024-03-21T11:44:58.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.422+0000 7f1dad800640 1 -- 172.21.15.47:0/2215719843 learned_addr learned my addr 172.21.15.47:0/2215719843 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:58.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.423+0000 7f1dad800640 1 -- 172.21.15.47:0/2215719843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 msgr2=0x55a497d8ba90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:44:58.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.423+0000 7f1dad800640 1 --2- 172.21.15.47:0/2215719843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 0x55a497d8ba90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.423+0000 7f1dad800640 1 -- 172.21.15.47:0/2215719843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7a810 msgr2=0x55a497d7f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.423+0000 7f1dad800640 1 --2- 172.21.15.47:0/2215719843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7a810 0x55a497d7f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.423+0000 7f1dad800640 1 -- 172.21.15.47:0/2215719843 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a497d7b460 con 0x55a497d7c3a0 2024-03-21T11:44:58.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.423+0000 7f1dad800640 1 --2- 172.21.15.47:0/2215719843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7c3a0 0x55a497d74620 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f1da0000f80 tx=0x7f1da001ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b726d311cf3e0243 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:58.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.423+0000 7f1d99600640 1 -- 172.21.15.47:0/2215719843 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da003c030 con 0x55a497d7c3a0 2024-03-21T11:44:58.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.424+0000 7f1d99600640 1 -- 172.21.15.47:0/2215719843 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1da0003e50 con 0x55a497d7c3a0 2024-03-21T11:44:58.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.424+0000 7f1d99600640 1 -- 172.21.15.47:0/2215719843 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0004530 con 0x55a497d7c3a0 2024-03-21T11:44:58.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.425+0000 7f1db256c740 1 -- 172.21.15.47:0/2215719843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7c3a0 msgr2=0x55a497d74620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.425+0000 7f1db256c740 1 --2- 172.21.15.47:0/2215719843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7c3a0 0x55a497d74620 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f1da0000f80 tx=0x7f1da001ef40 comp rx=0 tx=0).stop 2024-03-21T11:44:58.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.425+0000 7f1db256c740 1 -- 172.21.15.47:0/2215719843 shutdown_connections 2024-03-21T11:44:58.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.425+0000 7f1db256c740 1 --2- 172.21.15.47:0/2215719843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7c3a0 0x55a497d74620 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.425+0000 7f1db256c740 1 --2- 172.21.15.47:0/2215719843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 0x55a497d8ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.425+0000 7f1db256c740 1 --2- 172.21.15.47:0/2215719843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7a810 0x55a497d7f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.425+0000 7f1db256c740 1 -- 172.21.15.47:0/2215719843 >> 172.21.15.47:0/2215719843 conn(0x55a497d75c00 msgr2=0x55a497d7ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:58.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.434+0000 7f1db256c740 1 -- 172.21.15.47:0/2215719843 shutdown_connections 2024-03-21T11:44:58.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.437+0000 7f1db256c740 1 -- 172.21.15.47:0/2215719843 wait complete. 2024-03-21T11:44:58.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.438+0000 7f1db256c740 1 -- start start 2024-03-21T11:44:58.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.438+0000 7f1db256c740 1 Processor -- start 2024-03-21T11:44:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.440+0000 7f1db256c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 0x55a497d8b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.440+0000 7f1db256c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 0x55a497d881c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.440+0000 7f1db256c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7c3a0 0x55a497d88700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.440+0000 7f1db256c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d88010cc0 con 0x55a497d7c3a0 2024-03-21T11:44:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.440+0000 7f1db256c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d88010660 con 0x55a497d7b9e0 2024-03-21T11:44:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.440+0000 7f1db256c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8800db60 con 0x55a497d7a810 2024-03-21T11:44:58.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.440+0000 7f1db256c740 10 client.0 _ll_register_callbacks cb 0x7f1da0009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:58.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.441+0000 7f1dace00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 0x55a497d8b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.441+0000 7f1dace00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 0x55a497d8b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55746/0 (socket says 172.21.15.47:55746) 2024-03-21T11:44:58.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.441+0000 7f1dace00640 1 -- 172.21.15.47:0/2856822204 learned_addr learned my addr 172.21.15.47:0/2856822204 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:58.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.441+0000 7f1da7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 0x55a497d881c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.441+0000 7f1da7e00640 1 -- 172.21.15.47:0/2856822204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 msgr2=0x55a497d8b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.441+0000 7f1da7e00640 1 --2- 172.21.15.47:0/2856822204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 0x55a497d8b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.441+0000 7f1da7e00640 1 -- 172.21.15.47:0/2856822204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7c3a0 msgr2=0x55a497d88700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.441+0000 7f1da7e00640 1 --2- 172.21.15.47:0/2856822204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7c3a0 0x55a497d88700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.441+0000 7f1da7e00640 1 -- 172.21.15.47:0/2856822204 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a497d8bd50 con 0x55a497d7b9e0 2024-03-21T11:44:58.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.441+0000 7f1dace00640 1 --2- 172.21.15.47:0/2856822204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 0x55a497d8b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:58.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.442+0000 7f1da7e00640 1 --2- 172.21.15.47:0/2856822204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 0x55a497d881c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f1d94004620 tx=0x7f1d94013b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=659afedd2f3a7359 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:58.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.442+0000 7f1d8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:44:58.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.442+0000 7f1d8ea00640 1 -- 172.21.15.47:0/2856822204 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94021060 con 0x55a497d7b9e0 2024-03-21T11:44:58.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.442+0000 7f1d8ea00640 1 -- 172.21.15.47:0/2856822204 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1d94011a20 con 0x55a497d7b9e0 2024-03-21T11:44:58.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.442+0000 7f1db256c740 1 -- 172.21.15.47:0/2856822204 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1da0003e50 con 0x55a497d7b9e0 2024-03-21T11:44:58.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.443+0000 7f1d8ea00640 1 -- 172.21.15.47:0/2856822204 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d94011c20 con 0x55a497d7b9e0 2024-03-21T11:44:58.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.443+0000 7f1db256c740 10 client.32405 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:58.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.443+0000 7f1db256c740 1 -- 172.21.15.47:0/2856822204 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a497dfda20 con 0x55a497d7b9e0 2024-03-21T11:44:58.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.444+0000 7f1d8ea00640 1 -- 172.21.15.47:0/2856822204 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1da0003e50 con 0x55a497d7b9e0 2024-03-21T11:44:58.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.444+0000 7f1d8ea00640 1 -- 172.21.15.47:0/2856822204 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1d9400f040 con 0x55a497d7b9e0 2024-03-21T11:44:58.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.448+0000 7f1db256c740 10 client.32405 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.448+0000 7f1db256c740 10 client.32405 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:44:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.448+0000 7f1db256c740 20 client.32405 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.448+0000 7f1db256c740 1 --2- 172.21.15.47:0/2856822204 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a497dfe2d0 0x55a497e006f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.448+0000 7f1db256c740 4 client.32405 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:44:58.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.448+0000 7f1db256c740 1 -- 172.21.15.47:0/2856822204 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a497c64500 con 0x55a497dfe2d0 2024-03-21T11:44:58.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.453+0000 7f1dace00640 1 --2- 172.21.15.47:0/2856822204 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a497dfe2d0 0x55a497e006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.455+0000 7f1dace00640 1 --2- 172.21.15.47:0/2856822204 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a497dfe2d0 0x55a497e006f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=17d44a8e05849124 server_cookie=b55370561ca055e in_seq=0 out_seq=0 2024-03-21T11:44:58.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.455+0000 7f1d8ea00640 10 client.32405 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:44:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.457+0000 7f1d8ea00640 1 -- 172.21.15.47:0/2856822204 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a497c64500 con 0x55a497dfe2d0 2024-03-21T11:44:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.457+0000 7f1d8ea00640 10 client.32405 handle_command_reply: tid=0 2024-03-21T11:44:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.457+0000 7f1db256c740 1 client.32405 shutdown 2024-03-21T11:44:58.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.459+0000 7f1db256c740 1 -- 172.21.15.47:0/2856822204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 msgr2=0x55a497d881c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.459+0000 7f1db256c740 1 --2- 172.21.15.47:0/2856822204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 0x55a497d881c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f1d94004620 tx=0x7f1d94013b70 comp rx=0 tx=0).stop 2024-03-21T11:44:58.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.459+0000 7f1db256c740 1 -- 172.21.15.47:0/2856822204 shutdown_connections 2024-03-21T11:44:58.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.459+0000 7f1db256c740 1 --2- 172.21.15.47:0/2856822204 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55a497dfe2d0 0x55a497e006f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.459+0000 7f1db256c740 1 --2- 172.21.15.47:0/2856822204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a497d7c3a0 0x55a497d88700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.459+0000 7f1db256c740 1 --2- 172.21.15.47:0/2856822204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a497d7b9e0 0x55a497d881c0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.459+0000 7f1db256c740 1 --2- 172.21.15.47:0/2856822204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a497d7a810 0x55a497d8b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.459+0000 7f1d8ea00640 0 client.32405 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:44:58.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.459+0000 7f1db256c740 1 -- 172.21.15.47:0/2856822204 >> 172.21.15.47:0/2856822204 conn(0x55a497d75c00 msgr2=0x55a497d78e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:58.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.460+0000 7f1db256c740 1 -- 172.21.15.47:0/2856822204 shutdown_connections 2024-03-21T11:44:58.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.460+0000 7f1db256c740 1 -- 172.21.15.47:0/2856822204 wait complete. 2024-03-21T11:44:58.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.460+0000 7f1db256c740 20 client.32405 trim_cache size 0 max 16384 2024-03-21T11:44:58.463 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:44:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.461+0000 7f1daf800640 1 -- 172.21.15.47:0/1722607443 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1d900a2fa0 msgr2=0x7f1d900a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.461+0000 7f1daf800640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1d900a2fa0 0x7f1d900a5460 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x55a497d7b8e0 tx=0x7f1da0009500 comp rx=0 tx=0).stop 2024-03-21T11:44:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.461+0000 7f1daf800640 1 -- 172.21.15.47:0/1722607443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da81038e0 msgr2=0x7f1da819de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.461+0000 7f1daf800640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da81038e0 0x7f1da819de10 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009850 tx=0x7f1d9c02ce60 comp rx=0 tx=0).stop 2024-03-21T11:44:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.461+0000 7f1daf800640 1 -- 172.21.15.47:0/1722607443 shutdown_connections 2024-03-21T11:44:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.461+0000 7f1daf800640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1d900a2fa0 0x7f1d900a5460 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.461+0000 7f1daf800640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1da8105490 0x7f1da81a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.461+0000 7f1daf800640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1da8104ad0 0x7f1da819e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.461+0000 7f1daf800640 1 --2- 172.21.15.47:0/1722607443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1da81038e0 0x7f1da819de10 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.461+0000 7f1daf800640 1 -- 172.21.15.47:0/1722607443 >> 172.21.15.47:0/1722607443 conn(0x7f1da80fee50 msgr2=0x7f1da8077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.462+0000 7f1daf800640 1 -- 172.21.15.47:0/1722607443 shutdown_connections 2024-03-21T11:44:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.462+0000 7f1daf800640 1 -- 172.21.15.47:0/1722607443 wait complete. 2024-03-21T11:44:58.481 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:58 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1244817049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:44:58.484 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:44:58.484 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:44:58.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:58 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1244817049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:44:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.739+0000 7fdc55e00640 1 -- 172.21.15.47:0/2999072652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc50103150 msgr2=0x7fdc501099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.739+0000 7fdc55e00640 1 --2- 172.21.15.47:0/2999072652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc50103150 0x7fdc501099e0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009ca0 tx=0x7fdc4802c0f0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.739+0000 7fdc55e00640 1 -- 172.21.15.47:0/2999072652 shutdown_connections 2024-03-21T11:44:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.739+0000 7fdc55e00640 1 --2- 172.21.15.47:0/2999072652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc50103150 0x7fdc501099e0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.739+0000 7fdc55e00640 1 --2- 172.21.15.47:0/2999072652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc50102790 0x7fdc50102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.739+0000 7fdc55e00640 1 --2- 172.21.15.47:0/2999072652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc501015a0 0x7fdc501019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.739+0000 7fdc55e00640 1 -- 172.21.15.47:0/2999072652 >> 172.21.15.47:0/2999072652 conn(0x7fdc500fcb10 msgr2=0x7fdc500fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.740+0000 7fdc55e00640 1 -- 172.21.15.47:0/2999072652 shutdown_connections 2024-03-21T11:44:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.740+0000 7fdc55e00640 1 -- 172.21.15.47:0/2999072652 wait complete. 2024-03-21T11:44:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.742+0000 7fdc55e00640 1 Processor -- start 2024-03-21T11:44:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.742+0000 7fdc55e00640 1 -- start start 2024-03-21T11:44:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.742+0000 7fdc55e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc501015a0 0x7fdc501a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.743+0000 7fdc55e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc50102790 0x7fdc501a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.743+0000 7fdc55e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc50103150 0x7fdc501a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.743+0000 7fdc55e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5010d290 con 0x7fdc50102790 2024-03-21T11:44:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.743+0000 7fdc55e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5010d110 con 0x7fdc50103150 2024-03-21T11:44:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.743+0000 7fdc55e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5010d410 con 0x7fdc501015a0 2024-03-21T11:44:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.743+0000 7fdc54a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc501015a0 0x7fdc501a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.743+0000 7fdc54a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc501015a0 0x7fdc501a72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55762/0 (socket says 172.21.15.47:55762) 2024-03-21T11:44:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.743+0000 7fdc54a00640 1 -- 172.21.15.47:0/2601752695 learned_addr learned my addr 172.21.15.47:0/2601752695 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.743+0000 7fdc55400640 1 --2- 172.21.15.47:0/2601752695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc50103150 0x7fdc501a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.744+0000 7fdc4fe00640 1 --2- 172.21.15.47:0/2601752695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc50102790 0x7fdc501a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.744+0000 7fdc54a00640 1 -- 172.21.15.47:0/2601752695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc50103150 msgr2=0x7fdc501a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.744+0000 7fdc54a00640 1 --2- 172.21.15.47:0/2601752695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc50103150 0x7fdc501a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.744+0000 7fdc54a00640 1 -- 172.21.15.47:0/2601752695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc50102790 msgr2=0x7fdc501a7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.744+0000 7fdc54a00640 1 --2- 172.21.15.47:0/2601752695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc50102790 0x7fdc501a7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.744+0000 7fdc54a00640 1 -- 172.21.15.47:0/2601752695 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc501a4720 con 0x7fdc501015a0 2024-03-21T11:44:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.745+0000 7fdc54a00640 1 --2- 172.21.15.47:0/2601752695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc501015a0 0x7fdc501a72f0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400b450 tx=0x7fdc4400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.747+0000 7fdc4d600640 1 -- 172.21.15.47:0/2601752695 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44015020 con 0x7fdc501015a0 2024-03-21T11:44:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.747+0000 7fdc55e00640 1 -- 172.21.15.47:0/2601752695 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc50073c20 con 0x7fdc501015a0 2024-03-21T11:44:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.747+0000 7fdc55e00640 1 -- 172.21.15.47:0/2601752695 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc500741c0 con 0x7fdc501015a0 2024-03-21T11:44:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.747+0000 7fdc4d600640 1 -- 172.21.15.47:0/2601752695 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc44011d70 con 0x7fdc501015a0 2024-03-21T11:44:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.747+0000 7fdc4d600640 1 -- 172.21.15.47:0/2601752695 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44015020 con 0x7fdc501015a0 2024-03-21T11:44:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.753+0000 7fdc5a02d740 1 Processor -- start 2024-03-21T11:44:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.753+0000 7fdc5a02d740 1 -- start start 2024-03-21T11:44:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.753+0000 7fdc4d600640 1 -- 172.21.15.47:0/2601752695 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdc44018020 con 0x7fdc501015a0 2024-03-21T11:44:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.759+0000 7fdc5a02d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442e9f0 0x5651e442ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.759+0000 7fdc5a02d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e442fbc0 0x5651e4430040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.759+0000 7fdc5a02d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e4430580 0x5651e4436e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.759+0000 7fdc5a02d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e43932c0 con 0x5651e442e9f0 2024-03-21T11:44:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.759+0000 7fdc5a02d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e433bc90 con 0x5651e442fbc0 2024-03-21T11:44:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.759+0000 7fdc5a02d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e438f360 con 0x5651e4430580 2024-03-21T11:44:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.759+0000 7fdc4fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e4430580 0x5651e4436e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.759+0000 7fdc4fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e4430580 0x5651e4436e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55768/0 (socket says 172.21.15.47:55768) 2024-03-21T11:44:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.759+0000 7fdc4fe00640 1 -- 172.21.15.47:0/3075266803 learned_addr learned my addr 172.21.15.47:0/3075266803 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.759+0000 7fdc54a00640 1 --2- 172.21.15.47:0/3075266803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e442fbc0 0x5651e4430040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.760+0000 7fdc54a00640 1 -- 172.21.15.47:0/3075266803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e4430580 msgr2=0x5651e4436e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.760+0000 7fdc54a00640 1 --2- 172.21.15.47:0/3075266803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e4430580 0x5651e4436e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.760+0000 7fdc54a00640 1 -- 172.21.15.47:0/3075266803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442e9f0 msgr2=0x5651e442ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.760+0000 7fdc54a00640 1 --2- 172.21.15.47:0/3075266803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442e9f0 0x5651e442ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.760+0000 7fdc54a00640 1 -- 172.21.15.47:0/3075266803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651e4360080 con 0x5651e442fbc0 2024-03-21T11:44:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.761+0000 7fdc54a00640 1 --2- 172.21.15.47:0/3075266803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e442fbc0 0x5651e4430040 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdc440cc260 tx=0x7fdc440d6690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f940a6fea2289485 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.761+0000 7fdc41c00640 1 -- 172.21.15.47:0/3075266803 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc440dd070 con 0x5651e442fbc0 2024-03-21T11:44:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.761+0000 7fdc41c00640 1 -- 172.21.15.47:0/3075266803 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc440d9b20 con 0x5651e442fbc0 2024-03-21T11:44:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.761+0000 7fdc41c00640 1 -- 172.21.15.47:0/3075266803 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc440d8ad0 con 0x5651e442fbc0 2024-03-21T11:44:58.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.768+0000 7fdc4d600640 1 --2- 172.21.15.47:0/2601752695 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc380a3310 0x7fdc380a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.769+0000 7fdc4d600640 1 -- 172.21.15.47:0/2601752695 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdc44011690 con 0x7fdc501015a0 2024-03-21T11:44:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.770+0000 7fdc55400640 1 --2- 172.21.15.47:0/2601752695 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc380a3310 0x7fdc380a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.775+0000 7fdc5a02d740 1 -- 172.21.15.47:0/3075266803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e442fbc0 msgr2=0x5651e4430040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.775+0000 7fdc55400640 1 --2- 172.21.15.47:0/2601752695 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc380a3310 0x7fdc380a57d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdc4802c630 tx=0x7fdc48004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:58.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.775+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/3075266803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e442fbc0 0x5651e4430040 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdc440cc260 tx=0x7fdc440d6690 comp rx=0 tx=0).stop 2024-03-21T11:44:58.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.775+0000 7fdc5a02d740 1 -- 172.21.15.47:0/3075266803 shutdown_connections 2024-03-21T11:44:58.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.775+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/3075266803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e4430580 0x5651e4436e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.775+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/3075266803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e442fbc0 0x5651e4430040 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.775+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/3075266803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442e9f0 0x5651e442ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.775+0000 7fdc5a02d740 1 -- 172.21.15.47:0/3075266803 >> 172.21.15.47:0/3075266803 conn(0x5651e4429de0 msgr2=0x5651e442a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:58.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.775+0000 7fdc5a02d740 1 -- 172.21.15.47:0/3075266803 shutdown_connections 2024-03-21T11:44:58.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.775+0000 7fdc5a02d740 1 -- 172.21.15.47:0/3075266803 wait complete. 2024-03-21T11:44:58.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.776+0000 7fdc5a02d740 1 -- start start 2024-03-21T11:44:58.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.776+0000 7fdc5a02d740 1 Processor -- start 2024-03-21T11:44:58.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.778+0000 7fdc5a02d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442e9f0 0x5651e4438ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.778+0000 7fdc5a02d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442fbc0 0x5651e4438ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.779+0000 7fdc5a02d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 0x5651e4440030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.779+0000 7fdc5a02d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e4437590 con 0x5651e442e9f0 2024-03-21T11:44:58.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.779+0000 7fdc5a02d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e434f0d0 con 0x5651e4430580 2024-03-21T11:44:58.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.779+0000 7fdc5a02d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e4437710 con 0x5651e442fbc0 2024-03-21T11:44:58.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.779+0000 7fdc5a02d740 10 client.0 _ll_register_callbacks cb 0x5651e4353e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:58.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.780+0000 7fdc55400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 0x5651e4440030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.780+0000 7fdc4fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442fbc0 0x5651e4438ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.780+0000 7fdc4fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442fbc0 0x5651e4438ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55784/0 (socket says 172.21.15.47:55784) 2024-03-21T11:44:58.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.780+0000 7fdc55400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 0x5651e4440030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43770/0 (socket says 172.21.15.47:43770) 2024-03-21T11:44:58.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.780+0000 7fdc4fe00640 1 -- 172.21.15.47:0/1736443872 learned_addr learned my addr 172.21.15.47:0/1736443872 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:58.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.780+0000 7fdc54a00640 1 --2- 172.21.15.47:0/1736443872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442e9f0 0x5651e4438ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.780+0000 7fdc4fe00640 1 -- 172.21.15.47:0/1736443872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 msgr2=0x5651e4440030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.780+0000 7fdc4fe00640 1 --2- 172.21.15.47:0/1736443872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 0x5651e4440030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.780+0000 7fdc4fe00640 1 -- 172.21.15.47:0/1736443872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442e9f0 msgr2=0x5651e4438ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.780+0000 7fdc4fe00640 1 --2- 172.21.15.47:0/1736443872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442e9f0 0x5651e4438ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.780+0000 7fdc4fe00640 1 -- 172.21.15.47:0/1736443872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651e40a73f0 con 0x5651e442fbc0 2024-03-21T11:44:58.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.781+0000 7fdc4fe00640 1 --2- 172.21.15.47:0/1736443872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442fbc0 0x5651e4438ff0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0136e0 tx=0x7fdc3c013ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaf8c793ba0dee4b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:58.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.781+0000 7fdc36000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.782+0000 7fdc36000640 1 -- 172.21.15.47:0/1736443872 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c011d70 con 0x5651e442fbc0 2024-03-21T11:44:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.782+0000 7fdc36000640 1 -- 172.21.15.47:0/1736443872 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdc3c01b3e0 con 0x5651e442fbc0 2024-03-21T11:44:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.782+0000 7fdc5a02d740 1 -- 172.21.15.47:0/1736443872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc440d9b20 con 0x5651e442fbc0 2024-03-21T11:44:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.784+0000 7fdc36000640 1 -- 172.21.15.47:0/1736443872 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc3c01b5e0 con 0x5651e442fbc0 2024-03-21T11:44:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.784+0000 7fdc5a02d740 10 client.32389 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.784+0000 7fdc5a02d740 1 -- 172.21.15.47:0/1736443872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5651e41717d0 con 0x5651e442fbc0 2024-03-21T11:44:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.785+0000 7fdc36000640 1 -- 172.21.15.47:0/1736443872 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc440d9b20 con 0x5651e442fbc0 2024-03-21T11:44:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.785+0000 7fdc36000640 1 -- 172.21.15.47:0/1736443872 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdc3c01a070 con 0x5651e442fbc0 2024-03-21T11:44:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.785+0000 7fdc5a02d740 10 client.32389 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.785+0000 7fdc5a02d740 10 client.32389 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:44:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.785+0000 7fdc5a02d740 20 client.32389 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.785+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/1736443872 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5651e4444600 0x5651e4446a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.785+0000 7fdc5a02d740 4 client.32389 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.785+0000 7fdc5a02d740 1 -- 172.21.15.47:0/1736443872 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651e43a0800 con 0x5651e4444600 2024-03-21T11:44:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.786+0000 7fdc54a00640 1 --2- 172.21.15.47:0/1736443872 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5651e4444600 0x5651e4446a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.788+0000 7fdc54a00640 1 --2- 172.21.15.47:0/1736443872 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5651e4444600 0x5651e4446a20 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=489241175196e6af server_cookie=39b314f6517ef13d in_seq=0 out_seq=0 2024-03-21T11:44:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.788+0000 7fdc36000640 10 client.32389 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:44:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.789+0000 7fdc36000640 1 -- 172.21.15.47:0/1736443872 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5651e43a0800 con 0x5651e4444600 2024-03-21T11:44:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.789+0000 7fdc36000640 10 client.32389 handle_command_reply: tid=0 2024-03-21T11:44:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.789+0000 7fdc5a02d740 1 client.32389 shutdown 2024-03-21T11:44:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.790+0000 7fdc5a02d740 1 -- 172.21.15.47:0/1736443872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442fbc0 msgr2=0x5651e4438ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.790+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/1736443872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442fbc0 0x5651e4438ff0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0136e0 tx=0x7fdc3c013ba0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.790+0000 7fdc5a02d740 1 -- 172.21.15.47:0/1736443872 shutdown_connections 2024-03-21T11:44:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.790+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/1736443872 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5651e4444600 0x5651e4446a20 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.790+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/1736443872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 0x5651e4440030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.790+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/1736443872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442fbc0 0x5651e4438ff0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.790+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/1736443872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442e9f0 0x5651e4438ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.790+0000 7fdc36000640 0 client.32389 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:44:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.790+0000 7fdc5a02d740 1 -- 172.21.15.47:0/1736443872 >> 172.21.15.47:0/1736443872 conn(0x5651e4429de0 msgr2=0x5651e4429160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.791+0000 7fdc5a02d740 1 -- 172.21.15.47:0/1736443872 shutdown_connections 2024-03-21T11:44:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.791+0000 7fdc5a02d740 1 -- 172.21.15.47:0/1736443872 wait complete. 2024-03-21T11:44:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.793+0000 7fdc5a02d740 20 client.32389 trim_cache size 0 max 16384 2024-03-21T11:44:58.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.822+0000 7fdc5a02d740 1 Processor -- start 2024-03-21T11:44:58.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.825+0000 7fdc5a02d740 1 -- start start 2024-03-21T11:44:58.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.826+0000 7fdc5a02d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442e9f0 0x5651e4424ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.826+0000 7fdc5a02d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442fbc0 0x5651e443fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.826+0000 7fdc5a02d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 0x5651e4433500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.826+0000 7fdc5a02d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3000f9f0 con 0x5651e442fbc0 2024-03-21T11:44:58.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.826+0000 7fdc5a02d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc30011370 con 0x5651e4430580 2024-03-21T11:44:58.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.826+0000 7fdc5a02d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc30010050 con 0x5651e442e9f0 2024-03-21T11:44:58.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.826+0000 7fdc54a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442e9f0 0x5651e4424ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.826+0000 7fdc54a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442e9f0 0x5651e4424ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55796/0 (socket says 172.21.15.47:55796) 2024-03-21T11:44:58.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.826+0000 7fdc54a00640 1 -- 172.21.15.47:0/2465287742 learned_addr learned my addr 172.21.15.47:0/2465287742 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:58.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.826+0000 7fdc55400640 1 --2- 172.21.15.47:0/2465287742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 0x5651e4433500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.827+0000 7fdc4fe00640 1 --2- 172.21.15.47:0/2465287742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442fbc0 0x5651e443fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.827+0000 7fdc54a00640 1 -- 172.21.15.47:0/2465287742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 msgr2=0x5651e4433500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.827+0000 7fdc54a00640 1 --2- 172.21.15.47:0/2465287742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 0x5651e4433500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.827+0000 7fdc54a00640 1 -- 172.21.15.47:0/2465287742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442fbc0 msgr2=0x5651e443fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.827+0000 7fdc54a00640 1 --2- 172.21.15.47:0/2465287742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442fbc0 0x5651e443fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.827+0000 7fdc54a00640 1 -- 172.21.15.47:0/2465287742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651e442a850 con 0x5651e442e9f0 2024-03-21T11:44:58.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.827+0000 7fdc54a00640 1 --2- 172.21.15.47:0/2465287742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442e9f0 0x5651e4424ff0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fdc4409ae50 tx=0x7fdc440047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff9f52ba6b6731c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:58.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.828+0000 7fdc41200640 1 -- 172.21.15.47:0/2465287742 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44015040 con 0x5651e442e9f0 2024-03-21T11:44:58.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.828+0000 7fdc41200640 1 -- 172.21.15.47:0/2465287742 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc440d6e90 con 0x5651e442e9f0 2024-03-21T11:44:58.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.828+0000 7fdc41200640 1 -- 172.21.15.47:0/2465287742 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc440dd040 con 0x5651e442e9f0 2024-03-21T11:44:58.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.831+0000 7fdc5a02d740 1 -- 172.21.15.47:0/2465287742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442e9f0 msgr2=0x5651e4424ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.831+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/2465287742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442e9f0 0x5651e4424ff0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fdc4409ae50 tx=0x7fdc440047d0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.832+0000 7fdc5a02d740 1 -- 172.21.15.47:0/2465287742 shutdown_connections 2024-03-21T11:44:58.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.832+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/2465287742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 0x5651e4433500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.832+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/2465287742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442fbc0 0x5651e443fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.832+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/2465287742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442e9f0 0x5651e4424ff0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.832+0000 7fdc5a02d740 1 -- 172.21.15.47:0/2465287742 >> 172.21.15.47:0/2465287742 conn(0x5651e4429de0 msgr2=0x5651e4432df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:58.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.832+0000 7fdc5a02d740 1 -- 172.21.15.47:0/2465287742 shutdown_connections 2024-03-21T11:44:58.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.832+0000 7fdc5a02d740 1 -- 172.21.15.47:0/2465287742 wait complete. 2024-03-21T11:44:58.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.833+0000 7fdc5a02d740 1 -- start start 2024-03-21T11:44:58.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.833+0000 7fdc5a02d740 1 Processor -- start 2024-03-21T11:44:58.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.834+0000 7fdc5a02d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442e9f0 0x5651e443fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.834+0000 7fdc5a02d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442fbc0 0x5651e443e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.834+0000 7fdc5a02d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 0x5651e443e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.835+0000 7fdc4fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442fbc0 0x5651e443e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.835+0000 7fdc4fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442fbc0 0x5651e443e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:55808/0 (socket says 172.21.15.47:55808) 2024-03-21T11:44:58.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.835+0000 7fdc5a02d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc30010d10 con 0x5651e442e9f0 2024-03-21T11:44:58.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.835+0000 7fdc5a02d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc300106b0 con 0x5651e4430580 2024-03-21T11:44:58.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.835+0000 7fdc5a02d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3000dbb0 con 0x5651e442fbc0 2024-03-21T11:44:58.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.835+0000 7fdc5a02d740 10 client.0 _ll_register_callbacks cb 0x7fdc3c005ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:58.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.836+0000 7fdc4fe00640 1 -- 172.21.15.47:0/40590530 learned_addr learned my addr 172.21.15.47:0/40590530 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:58.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.835+0000 7fdc55400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 0x5651e443e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.836+0000 7fdc4fe00640 1 -- 172.21.15.47:0/40590530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 msgr2=0x5651e443e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.836+0000 7fdc4fe00640 1 --2- 172.21.15.47:0/40590530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 0x5651e443e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.836+0000 7fdc4fe00640 1 -- 172.21.15.47:0/40590530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442e9f0 msgr2=0x5651e443fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:58.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.836+0000 7fdc4fe00640 1 --2- 172.21.15.47:0/40590530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442e9f0 0x5651e443fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.836+0000 7fdc4fe00640 1 -- 172.21.15.47:0/40590530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651e443f0d0 con 0x5651e442fbc0 2024-03-21T11:44:58.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.841+0000 7fdc4fe00640 1 --2- 172.21.15.47:0/40590530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442fbc0 0x5651e443e450 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c026af0 tx=0x7fdc3c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=333b04558247b910 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:58.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.841+0000 7fdc36000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:58.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.844+0000 7fdc36000640 1 -- 172.21.15.47:0/40590530 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c011d80 con 0x5651e442fbc0 2024-03-21T11:44:58.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.844+0000 7fdc36000640 1 -- 172.21.15.47:0/40590530 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdc3c026470 con 0x5651e442fbc0 2024-03-21T11:44:58.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.844+0000 7fdc5a02d740 1 -- 172.21.15.47:0/40590530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc440d6e90 con 0x5651e442fbc0 2024-03-21T11:44:58.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.844+0000 7fdc36000640 1 -- 172.21.15.47:0/40590530 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc3c036050 con 0x5651e442fbc0 2024-03-21T11:44:58.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.844+0000 7fdc5a02d740 10 client.32395 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:58.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.844+0000 7fdc5a02d740 1 -- 172.21.15.47:0/40590530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5651e44a0e90 con 0x5651e442fbc0 2024-03-21T11:44:58.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.846+0000 7fdc36000640 1 -- 172.21.15.47:0/40590530 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc440d6e90 con 0x5651e442fbc0 2024-03-21T11:44:58.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.846+0000 7fdc36000640 1 -- 172.21.15.47:0/40590530 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdc3c01a040 con 0x5651e442fbc0 2024-03-21T11:44:58.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.846+0000 7fdc5a02d740 10 client.32395 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:58.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.846+0000 7fdc5a02d740 10 client.32395 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:44:58.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.846+0000 7fdc5a02d740 20 client.32395 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:58.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.846+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/40590530 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5651e44a1790 0x5651e44a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:58.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.846+0000 7fdc5a02d740 4 client.32395 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:44:58.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.846+0000 7fdc5a02d740 1 -- 172.21.15.47:0/40590530 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5651e439ab50 con 0x5651e44a1790 2024-03-21T11:44:58.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.846+0000 7fdc54a00640 1 --2- 172.21.15.47:0/40590530 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5651e44a1790 0x5651e44a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:58.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.847+0000 7fdc54a00640 1 --2- 172.21.15.47:0/40590530 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5651e44a1790 0x5651e44a3bb0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=220c71c835d2e209 server_cookie=f368ab706bffebd5 in_seq=0 out_seq=0 2024-03-21T11:44:58.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.847+0000 7fdc36000640 10 client.32395 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:44:58.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.847+0000 7fdc36000640 1 -- 172.21.15.47:0/40590530 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5651e439ab50 con 0x5651e44a1790 2024-03-21T11:44:58.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.847+0000 7fdc36000640 10 client.32395 handle_command_reply: tid=0 2024-03-21T11:44:58.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.848+0000 7fdc5a02d740 1 client.32395 shutdown 2024-03-21T11:44:58.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.848+0000 7fdc5a02d740 1 -- 172.21.15.47:0/40590530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442fbc0 msgr2=0x5651e443e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.848+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/40590530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442fbc0 0x5651e443e450 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c026af0 tx=0x7fdc3c009b60 comp rx=0 tx=0).stop 2024-03-21T11:44:58.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.848+0000 7fdc5a02d740 1 -- 172.21.15.47:0/40590530 shutdown_connections 2024-03-21T11:44:58.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.848+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/40590530 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5651e44a1790 0x5651e44a3bb0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.849+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/40590530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651e4430580 0x5651e443e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.849+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/40590530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651e442fbc0 0x5651e443e450 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.849+0000 7fdc5a02d740 1 --2- 172.21.15.47:0/40590530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651e442e9f0 0x5651e443fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.849+0000 7fdc5a02d740 1 -- 172.21.15.47:0/40590530 >> 172.21.15.47:0/40590530 conn(0x5651e4429de0 msgr2=0x5651e4433680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:58.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.849+0000 7fdc36000640 0 client.32395 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:44:58.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.849+0000 7fdc5a02d740 1 -- 172.21.15.47:0/40590530 shutdown_connections 2024-03-21T11:44:58.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.849+0000 7fdc5a02d740 1 -- 172.21.15.47:0/40590530 wait complete. 2024-03-21T11:44:58.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.851+0000 7fdc5a02d740 20 client.32395 trim_cache size 0 max 16384 2024-03-21T11:44:58.853 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:44:58.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.853+0000 7fdc55e00640 1 -- 172.21.15.47:0/2601752695 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc380a3310 msgr2=0x7fdc380a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.853+0000 7fdc55e00640 1 --2- 172.21.15.47:0/2601752695 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc380a3310 0x7fdc380a57d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdc4802c630 tx=0x7fdc48004a40 comp rx=0 tx=0).stop 2024-03-21T11:44:58.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.853+0000 7fdc55e00640 1 -- 172.21.15.47:0/2601752695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc501015a0 msgr2=0x7fdc501a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:58.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.853+0000 7fdc55e00640 1 --2- 172.21.15.47:0/2601752695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc501015a0 0x7fdc501a72f0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400b450 tx=0x7fdc4400b910 comp rx=0 tx=0).stop 2024-03-21T11:44:58.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.854+0000 7fdc55e00640 1 -- 172.21.15.47:0/2601752695 shutdown_connections 2024-03-21T11:44:58.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.854+0000 7fdc55e00640 1 --2- 172.21.15.47:0/2601752695 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc380a3310 0x7fdc380a57d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.854+0000 7fdc55e00640 1 --2- 172.21.15.47:0/2601752695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc50103150 0x7fdc501a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.854+0000 7fdc55e00640 1 --2- 172.21.15.47:0/2601752695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc50102790 0x7fdc501a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.854+0000 7fdc55e00640 1 --2- 172.21.15.47:0/2601752695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc501015a0 0x7fdc501a72f0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:58.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.854+0000 7fdc55e00640 1 -- 172.21.15.47:0/2601752695 >> 172.21.15.47:0/2601752695 conn(0x7fdc500fcb10 msgr2=0x7fdc50105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.854+0000 7fdc55e00640 1 -- 172.21.15.47:0/2601752695 shutdown_connections 2024-03-21T11:44:58.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:58.854+0000 7fdc55e00640 1 -- 172.21.15.47:0/2601752695 wait complete. 2024-03-21T11:44:58.880 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:44:58.880 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:44:58.975 INFO:tasks.workunit.client.0.smithi047.stdout: 10 201565 22.56 MB/sec execute 342 sec latency 95.603 ms 2024-03-21T11:44:59.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:44:59 smithi047 ceph-mon[31351]: pgmap v1049: 129 pgs: 129 active+clean; 637 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 600 op/s 2024-03-21T11:44:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:44:59 smithi102 ceph-mon[36748]: pgmap v1049: 129 pgs: 129 active+clean; 637 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 600 op/s 2024-03-21T11:44:59.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.635+0000 7fac2ec00640 1 -- 172.21.15.47:0/3639715361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac28107390 msgr2=0x7fac281097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:59.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.635+0000 7fac2ec00640 1 --2- 172.21.15.47:0/3639715361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac28107390 0x7fac281097e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fac20009ca0 tx=0x7fac2002c100 comp rx=0 tx=0).stop 2024-03-21T11:44:59.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.637+0000 7fac2ec00640 1 -- 172.21.15.47:0/3639715361 shutdown_connections 2024-03-21T11:44:59.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.637+0000 7fac2ec00640 1 --2- 172.21.15.47:0/3639715361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac28107390 0x7fac281097e0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.637+0000 7fac2ec00640 1 --2- 172.21.15.47:0/3639715361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac280ffd00 0x7fac28106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.637+0000 7fac2ec00640 1 --2- 172.21.15.47:0/3639715361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac280ff310 0x7fac280ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.637+0000 7fac2ec00640 1 -- 172.21.15.47:0/3639715361 >> 172.21.15.47:0/3639715361 conn(0x7fac280fae60 msgr2=0x7fac280fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:59.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.638+0000 7fac2ec00640 1 -- 172.21.15.47:0/3639715361 shutdown_connections 2024-03-21T11:44:59.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.638+0000 7fac2ec00640 1 -- 172.21.15.47:0/3639715361 wait complete. 2024-03-21T11:44:59.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.640+0000 7fac2ec00640 1 Processor -- start 2024-03-21T11:44:59.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.642+0000 7fac2ec00640 1 -- start start 2024-03-21T11:44:59.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:44:59 smithi100 ceph-mon[36245]: pgmap v1049: 129 pgs: 129 active+clean; 637 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 600 op/s 2024-03-21T11:44:59.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.642+0000 7fac2ec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac280ff310 0x7fac28199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.642+0000 7fac2ec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac280ffd00 0x7fac2819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.642+0000 7fac2ec00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac28107390 0x7fac281a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.642+0000 7fac2ec00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2810d380 con 0x7fac28107390 2024-03-21T11:44:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.642+0000 7fac2ec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2810d200 con 0x7fac280ffd00 2024-03-21T11:44:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.642+0000 7fac2ec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2810d500 con 0x7fac280ff310 2024-03-21T11:44:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.643+0000 7fac27e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac280ff310 0x7fac28199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.643+0000 7fac27e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac280ff310 0x7fac28199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49028/0 (socket says 172.21.15.47:49028) 2024-03-21T11:44:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.643+0000 7fac27e00640 1 -- 172.21.15.47:0/1578310572 learned_addr learned my addr 172.21.15.47:0/1578310572 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:59.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.643+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac280ffd00 0x7fac2819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.643+0000 7fac2cc00640 1 --2- 172.21.15.47:0/1578310572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac28107390 0x7fac281a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.644+0000 7fac1fe00640 1 -- 172.21.15.47:0/1578310572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac280ff310 msgr2=0x7fac28199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:59.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.644+0000 7fac1fe00640 1 --2- 172.21.15.47:0/1578310572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac280ff310 0x7fac28199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.644+0000 7fac1fe00640 1 -- 172.21.15.47:0/1578310572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac28107390 msgr2=0x7fac281a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:59.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.644+0000 7fac1fe00640 1 --2- 172.21.15.47:0/1578310572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac28107390 0x7fac281a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.644+0000 7fac1fe00640 1 -- 172.21.15.47:0/1578310572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac281a1860 con 0x7fac280ffd00 2024-03-21T11:44:59.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.645+0000 7fac1fe00640 1 --2- 172.21.15.47:0/1578310572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac280ffd00 0x7fac2819a1e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fac1000b3f0 tx=0x7fac1000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:59.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.646+0000 7fac25600640 1 -- 172.21.15.47:0/1578310572 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10015020 con 0x7fac280ffd00 2024-03-21T11:44:59.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.646+0000 7fac25600640 1 -- 172.21.15.47:0/1578310572 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac10011d10 con 0x7fac280ffd00 2024-03-21T11:44:59.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.646+0000 7fac25600640 1 -- 172.21.15.47:0/1578310572 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1000f2e0 con 0x7fac280ffd00 2024-03-21T11:44:59.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.646+0000 7fac2ec00640 1 -- 172.21.15.47:0/1578310572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac281a1b50 con 0x7fac280ffd00 2024-03-21T11:44:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.646+0000 7fac2ec00640 1 -- 172.21.15.47:0/1578310572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac281a2050 con 0x7fac280ffd00 2024-03-21T11:44:59.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.651+0000 7fac31856740 1 Processor -- start 2024-03-21T11:44:59.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.652+0000 7fac31856740 1 -- start start 2024-03-21T11:44:59.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.653+0000 7fac31856740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb68c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.653+0000 7fac31856740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb69e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.653+0000 7fac31856740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb70c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.653+0000 7fac31856740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ddacd2c0 con 0x55c5ddb68810 2024-03-21T11:44:59.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.653+0000 7fac31856740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5dda75c90 con 0x55c5ddb6a3a0 2024-03-21T11:44:59.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.653+0000 7fac31856740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ddac9360 con 0x55c5ddb699e0 2024-03-21T11:44:59.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.654+0000 7fac27e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb69e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.654+0000 7fac27e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb69e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49044/0 (socket says 172.21.15.47:49044) 2024-03-21T11:44:59.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.654+0000 7fac27e00640 1 -- 172.21.15.47:0/3235247106 learned_addr learned my addr 172.21.15.47:0/3235247106 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:59.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.654+0000 7fac2cc00640 1 --2- 172.21.15.47:0/3235247106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb68c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.654+0000 7fac27e00640 1 -- 172.21.15.47:0/3235247106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb6a3a0 msgr2=0x55c5ddb70c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:44:59.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.654+0000 7fac27e00640 1 --2- 172.21.15.47:0/3235247106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb70c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.654+0000 7fac27e00640 1 -- 172.21.15.47:0/3235247106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 msgr2=0x55c5ddb68c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:59.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.654+0000 7fac27e00640 1 --2- 172.21.15.47:0/3235247106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb68c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.654+0000 7fac27e00640 1 -- 172.21.15.47:0/3235247106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5dda9a080 con 0x55c5ddb699e0 2024-03-21T11:44:59.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.655+0000 7fac27e00640 1 --2- 172.21.15.47:0/3235247106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb69e60 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fac18009d70 tx=0x7fac18011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7db746221fdfb6ac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:59.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.656+0000 7fac25600640 1 -- 172.21.15.47:0/1578310572 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fac10018020 con 0x7fac280ffd00 2024-03-21T11:44:59.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.656+0000 7fac1d200640 1 -- 172.21.15.47:0/3235247106 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18005220 con 0x55c5ddb699e0 2024-03-21T11:44:59.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.656+0000 7fac1d200640 1 -- 172.21.15.47:0/3235247106 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac18004c80 con 0x55c5ddb699e0 2024-03-21T11:44:59.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.656+0000 7fac1d200640 1 -- 172.21.15.47:0/3235247106 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18019a60 con 0x55c5ddb699e0 2024-03-21T11:44:59.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.657+0000 7fac31856740 1 -- 172.21.15.47:0/3235247106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 msgr2=0x55c5ddb69e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:59.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.657+0000 7fac31856740 1 --2- 172.21.15.47:0/3235247106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb69e60 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fac18009d70 tx=0x7fac18011810 comp rx=0 tx=0).stop 2024-03-21T11:44:59.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.657+0000 7fac31856740 1 -- 172.21.15.47:0/3235247106 shutdown_connections 2024-03-21T11:44:59.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.658+0000 7fac31856740 1 --2- 172.21.15.47:0/3235247106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb70c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.658+0000 7fac31856740 1 --2- 172.21.15.47:0/3235247106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb69e60 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.658+0000 7fac31856740 1 --2- 172.21.15.47:0/3235247106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb68c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.658+0000 7fac31856740 1 -- 172.21.15.47:0/3235247106 >> 172.21.15.47:0/3235247106 conn(0x55c5ddb63c00 msgr2=0x55c5ddb64000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:59.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.658+0000 7fac31856740 1 -- 172.21.15.47:0/3235247106 shutdown_connections 2024-03-21T11:44:59.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.658+0000 7fac31856740 1 -- 172.21.15.47:0/3235247106 wait complete. 2024-03-21T11:44:59.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.659+0000 7fac31856740 1 -- start start 2024-03-21T11:44:59.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.659+0000 7fac25600640 1 --2- 172.21.15.47:0/1578310572 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac0c0a3030 0x7fac0c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.660+0000 7fac31856740 1 Processor -- start 2024-03-21T11:44:59.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.660+0000 7fac25600640 1 -- 172.21.15.47:0/1578310572 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fac100cefb0 con 0x7fac280ffd00 2024-03-21T11:44:59.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.660+0000 7fac2cc00640 1 --2- 172.21.15.47:0/1578310572 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac0c0a3030 0x7fac0c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.661+0000 7fac31856740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb7b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.662+0000 7fac2cc00640 1 --2- 172.21.15.47:0/1578310572 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac0c0a3030 0x7fac0c0a54f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fac20009bf0 tx=0x7fac2002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.664+0000 7fac31856740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb7b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.664+0000 7fac31856740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb827b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.664+0000 7fac31856740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ddb713b0 con 0x55c5ddb68810 2024-03-21T11:44:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.664+0000 7fac31856740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5dda890d0 con 0x55c5ddb6a3a0 2024-03-21T11:44:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.664+0000 7fac31856740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ddb71530 con 0x55c5ddb699e0 2024-03-21T11:44:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.664+0000 7fac31856740 10 client.0 _ll_register_callbacks cb 0x55c5dda8de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.664+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb7b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.664+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb7b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49050/0 (socket says 172.21.15.47:49050) 2024-03-21T11:44:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.664+0000 7fac1fe00640 1 -- 172.21.15.47:0/759405194 learned_addr learned my addr 172.21.15.47:0/759405194 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.666+0000 7fac27e00640 1 --2- 172.21.15.47:0/759405194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb7b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.666+0000 7fac1fe00640 1 -- 172.21.15.47:0/759405194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb6a3a0 msgr2=0x55c5ddb827b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:44:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.666+0000 7fac1fe00640 1 --2- 172.21.15.47:0/759405194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb827b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.666+0000 7fac1fe00640 1 -- 172.21.15.47:0/759405194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 msgr2=0x55c5ddb7b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.666+0000 7fac1fe00640 1 --2- 172.21.15.47:0/759405194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb7b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.666+0000 7fac1fe00640 1 -- 172.21.15.47:0/759405194 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5dd7e13f0 con 0x55c5ddb699e0 2024-03-21T11:44:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.666+0000 7fac1fe00640 1 --2- 172.21.15.47:0/759405194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb7b770 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fac100976c0 tx=0x7fac1009af20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88e25ee0e7fee7fe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.666+0000 7fac15600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:44:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.667+0000 7fac15600640 1 -- 172.21.15.47:0/759405194 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac100b3020 con 0x55c5ddb699e0 2024-03-21T11:44:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.667+0000 7fac15600640 1 -- 172.21.15.47:0/759405194 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fac100978d0 con 0x55c5ddb699e0 2024-03-21T11:44:59.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.667+0000 7fac31856740 1 -- 172.21.15.47:0/759405194 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac18004c80 con 0x55c5ddb699e0 2024-03-21T11:44:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.668+0000 7fac15600640 1 -- 172.21.15.47:0/759405194 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac10010eb0 con 0x55c5ddb699e0 2024-03-21T11:44:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.668+0000 7fac31856740 10 client.32407 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.668+0000 7fac31856740 1 -- 172.21.15.47:0/759405194 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c5dd8ab7d0 con 0x55c5ddb699e0 2024-03-21T11:44:59.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.668+0000 7fac15600640 1 -- 172.21.15.47:0/759405194 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac18004c80 con 0x55c5ddb699e0 2024-03-21T11:44:59.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.670+0000 7fac15600640 1 -- 172.21.15.47:0/759405194 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fac100b4250 con 0x55c5ddb699e0 2024-03-21T11:44:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.670+0000 7fac31856740 10 client.32407 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.670+0000 7fac31856740 10 client.32407 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:44:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.670+0000 7fac31856740 20 client.32407 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.670+0000 7fac31856740 1 --2- 172.21.15.47:0/759405194 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c5ddb86c50 0x55c5ddb89080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.670+0000 7fac31856740 4 client.32407 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:44:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.670+0000 7fac31856740 1 -- 172.21.15.47:0/759405194 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5ddada800 con 0x55c5ddb86c50 2024-03-21T11:44:59.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.671+0000 7fac27e00640 1 --2- 172.21.15.47:0/759405194 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c5ddb86c50 0x55c5ddb89080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.672+0000 7fac27e00640 1 --2- 172.21.15.47:0/759405194 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c5ddb86c50 0x55c5ddb89080 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96ec2bbe7aa986eb server_cookie=2a67c126d9d57912 in_seq=0 out_seq=0 2024-03-21T11:44:59.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.672+0000 7fac15600640 10 client.32407 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:44:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.674+0000 7fac15600640 1 -- 172.21.15.47:0/759405194 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c5ddada800 con 0x55c5ddb86c50 2024-03-21T11:44:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.674+0000 7fac15600640 10 client.32407 handle_command_reply: tid=0 2024-03-21T11:44:59.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.674+0000 7fac31856740 1 client.32407 shutdown 2024-03-21T11:44:59.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.676+0000 7fac31856740 1 -- 172.21.15.47:0/759405194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 msgr2=0x55c5ddb7b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:59.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.676+0000 7fac31856740 1 --2- 172.21.15.47:0/759405194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb7b770 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fac100976c0 tx=0x7fac1009af20 comp rx=0 tx=0).stop 2024-03-21T11:44:59.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.676+0000 7fac31856740 1 -- 172.21.15.47:0/759405194 shutdown_connections 2024-03-21T11:44:59.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.676+0000 7fac31856740 1 --2- 172.21.15.47:0/759405194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb827b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.676+0000 7fac31856740 1 --2- 172.21.15.47:0/759405194 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c5ddb86c50 0x55c5ddb89080 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.676+0000 7fac31856740 1 --2- 172.21.15.47:0/759405194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb7b770 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.676+0000 7fac31856740 1 --2- 172.21.15.47:0/759405194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb7b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.676+0000 7fac31856740 1 -- 172.21.15.47:0/759405194 >> 172.21.15.47:0/759405194 conn(0x55c5ddb63c00 msgr2=0x55c5ddb62eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:59.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.676+0000 7fac15600640 0 client.32407 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:44:59.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.676+0000 7fac31856740 1 -- 172.21.15.47:0/759405194 shutdown_connections 2024-03-21T11:44:59.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.676+0000 7fac31856740 1 -- 172.21.15.47:0/759405194 wait complete. 2024-03-21T11:44:59.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.677+0000 7fac31856740 20 client.32407 trim_cache size 0 max 16384 2024-03-21T11:44:59.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.701+0000 7fac31856740 1 Processor -- start 2024-03-21T11:44:59.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.702+0000 7fac31856740 1 -- start start 2024-03-21T11:44:59.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.702+0000 7fac31856740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb88c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.703+0000 7fac31856740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb82770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.703+0000 7fac31856740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb6c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.703+0000 7fac31856740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac281a7fa0 con 0x55c5ddb68810 2024-03-21T11:44:59.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.703+0000 7fac31856740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac281a9920 con 0x55c5ddb6a3a0 2024-03-21T11:44:59.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.703+0000 7fac31856740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac281a8600 con 0x55c5ddb699e0 2024-03-21T11:44:59.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.703+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb82770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.703+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb82770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49062/0 (socket says 172.21.15.47:49062) 2024-03-21T11:44:59.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.703+0000 7fac1fe00640 1 -- 172.21.15.47:0/1525738178 learned_addr learned my addr 172.21.15.47:0/1525738178 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:59.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.704+0000 7fac2cc00640 1 --2- 172.21.15.47:0/1525738178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb6c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.704+0000 7fac27e00640 1 --2- 172.21.15.47:0/1525738178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb88c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.704+0000 7fac2cc00640 1 -- 172.21.15.47:0/1525738178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 msgr2=0x55c5ddb82770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:59.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.704+0000 7fac2cc00640 1 --2- 172.21.15.47:0/1525738178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb82770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.704+0000 7fac2cc00640 1 -- 172.21.15.47:0/1525738178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 msgr2=0x55c5ddb88c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:59.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.704+0000 7fac2cc00640 1 --2- 172.21.15.47:0/1525738178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb88c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.704+0000 7fac2cc00640 1 -- 172.21.15.47:0/1525738178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5ddb64670 con 0x55c5ddb6a3a0 2024-03-21T11:44:59.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.705+0000 7fac1fe00640 1 --2- 172.21.15.47:0/1525738178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb82770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:44:59.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.705+0000 7fac2cc00640 1 --2- 172.21.15.47:0/1525738178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb6c9e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fac200324c0 tx=0x7fac2003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dca944fb673e1608 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:59.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.705+0000 7fac17e00640 1 -- 172.21.15.47:0/1525738178 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20045070 con 0x55c5ddb6a3a0 2024-03-21T11:44:59.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.705+0000 7fac17e00640 1 -- 172.21.15.47:0/1525738178 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac20051070 con 0x55c5ddb6a3a0 2024-03-21T11:44:59.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.705+0000 7fac17e00640 1 -- 172.21.15.47:0/1525738178 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20039040 con 0x55c5ddb6a3a0 2024-03-21T11:44:59.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.706+0000 7fac31856740 1 -- 172.21.15.47:0/1525738178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb6a3a0 msgr2=0x55c5ddb6c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:59.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.706+0000 7fac31856740 1 --2- 172.21.15.47:0/1525738178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb6c9e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fac200324c0 tx=0x7fac2003f000 comp rx=0 tx=0).stop 2024-03-21T11:44:59.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.707+0000 7fac31856740 1 -- 172.21.15.47:0/1525738178 shutdown_connections 2024-03-21T11:44:59.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.707+0000 7fac31856740 1 --2- 172.21.15.47:0/1525738178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb6c9e0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.707+0000 7fac31856740 1 --2- 172.21.15.47:0/1525738178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb699e0 0x55c5ddb82770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.707+0000 7fac31856740 1 --2- 172.21.15.47:0/1525738178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb88c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.707+0000 7fac31856740 1 -- 172.21.15.47:0/1525738178 >> 172.21.15.47:0/1525738178 conn(0x55c5ddb63c00 msgr2=0x55c5ddb676f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:59.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.707+0000 7fac31856740 1 -- 172.21.15.47:0/1525738178 shutdown_connections 2024-03-21T11:44:59.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.707+0000 7fac31856740 1 -- 172.21.15.47:0/1525738178 wait complete. 2024-03-21T11:44:59.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.708+0000 7fac31856740 1 -- start start 2024-03-21T11:44:59.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.708+0000 7fac31856740 1 Processor -- start 2024-03-21T11:44:59.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.709+0000 7fac31856740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb81f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.709+0000 7fac31856740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb699e0 0x55c5ddb82460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.709+0000 7fac31856740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb7ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.710+0000 7fac31856740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac281a92c0 con 0x55c5ddb68810 2024-03-21T11:44:59.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.710+0000 7fac31856740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac281a8c60 con 0x55c5ddb699e0 2024-03-21T11:44:59.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.710+0000 7fac31856740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac281a6180 con 0x55c5ddb6a3a0 2024-03-21T11:44:59.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.710+0000 7fac31856740 10 client.0 _ll_register_callbacks cb 0x7fac1009b190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:44:59.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.710+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb699e0 0x55c5ddb82460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.710+0000 7fac1fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb699e0 0x55c5ddb82460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44136/0 (socket says 172.21.15.47:44136) 2024-03-21T11:44:59.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.710+0000 7fac1fe00640 1 -- 172.21.15.47:0/2804907839 learned_addr learned my addr 172.21.15.47:0/2804907839 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:44:59.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.710+0000 7fac2cc00640 1 --2- 172.21.15.47:0/2804907839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb7ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.710+0000 7fac27e00640 1 --2- 172.21.15.47:0/2804907839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb81f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.711+0000 7fac1fe00640 1 -- 172.21.15.47:0/2804907839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb6a3a0 msgr2=0x55c5ddb7ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:59.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.711+0000 7fac1fe00640 1 --2- 172.21.15.47:0/2804907839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb7ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.711+0000 7fac1fe00640 1 -- 172.21.15.47:0/2804907839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 msgr2=0x55c5ddb81f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:44:59.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.711+0000 7fac1fe00640 1 --2- 172.21.15.47:0/2804907839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb81f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.711+0000 7fac1fe00640 1 -- 172.21.15.47:0/2804907839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5ddb7f020 con 0x55c5ddb699e0 2024-03-21T11:44:59.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.712+0000 7fac1fe00640 1 --2- 172.21.15.47:0/2804907839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb699e0 0x55c5ddb82460 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fac1009b280 tx=0x7fac10097b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce3974d40367a54e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:44:59.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.712+0000 7fac15600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:44:59.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.712+0000 7fac15600640 1 -- 172.21.15.47:0/2804907839 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac100a8030 con 0x55c5ddb699e0 2024-03-21T11:44:59.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.712+0000 7fac15600640 1 -- 172.21.15.47:0/2804907839 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fac100ccbd0 con 0x55c5ddb699e0 2024-03-21T11:44:59.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.712+0000 7fac31856740 1 -- 172.21.15.47:0/2804907839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac20051070 con 0x55c5ddb699e0 2024-03-21T11:44:59.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.713+0000 7fac15600640 1 -- 172.21.15.47:0/2804907839 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac100b3040 con 0x55c5ddb699e0 2024-03-21T11:44:59.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.714+0000 7fac31856740 10 client.32432 fetch_fsmap learned FSMap version 40 2024-03-21T11:44:59.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.714+0000 7fac31856740 1 -- 172.21.15.47:0/2804907839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c5ddbe30e0 con 0x55c5ddb699e0 2024-03-21T11:44:59.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.715+0000 7fac15600640 1 -- 172.21.15.47:0/2804907839 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac20051070 con 0x55c5ddb699e0 2024-03-21T11:44:59.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.715+0000 7fac15600640 1 -- 172.21.15.47:0/2804907839 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fac1009a8a0 con 0x55c5ddb699e0 2024-03-21T11:44:59.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.715+0000 7fac31856740 10 client.32432 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:44:59.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.715+0000 7fac31856740 10 client.32432 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:44:59.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.715+0000 7fac31856740 20 client.32432 populate_metadata read hostname 'smithi047' 2024-03-21T11:44:59.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.715+0000 7fac31856740 1 --2- 172.21.15.47:0/2804907839 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c5ddbe3a60 0x55c5ddbe5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:44:59.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.715+0000 7fac31856740 4 client.32432 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:44:59.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.715+0000 7fac31856740 1 -- 172.21.15.47:0/2804907839 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c5ddad4b50 con 0x55c5ddbe3a60 2024-03-21T11:44:59.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.715+0000 7fac27e00640 1 --2- 172.21.15.47:0/2804907839 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c5ddbe3a60 0x55c5ddbe5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:44:59.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.720+0000 7fac27e00640 1 --2- 172.21.15.47:0/2804907839 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c5ddbe3a60 0x55c5ddbe5e80 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37837b4054145e63 server_cookie=db5c4558a02c083a in_seq=0 out_seq=0 2024-03-21T11:44:59.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.721+0000 7fac15600640 10 client.32432 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:44:59.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.721+0000 7fac15600640 1 -- 172.21.15.47:0/2804907839 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c5ddad4b50 con 0x55c5ddbe3a60 2024-03-21T11:44:59.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.721+0000 7fac15600640 10 client.32432 handle_command_reply: tid=0 2024-03-21T11:44:59.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.721+0000 7fac31856740 1 client.32432 shutdown 2024-03-21T11:44:59.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.722+0000 7fac31856740 1 -- 172.21.15.47:0/2804907839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb699e0 msgr2=0x55c5ddb82460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:59.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.722+0000 7fac31856740 1 --2- 172.21.15.47:0/2804907839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb699e0 0x55c5ddb82460 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fac1009b280 tx=0x7fac10097b80 comp rx=0 tx=0).stop 2024-03-21T11:44:59.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.722+0000 7fac31856740 1 -- 172.21.15.47:0/2804907839 shutdown_connections 2024-03-21T11:44:59.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.722+0000 7fac31856740 1 --2- 172.21.15.47:0/2804907839 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55c5ddbe3a60 0x55c5ddbe5e80 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.723+0000 7fac31856740 1 --2- 172.21.15.47:0/2804907839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5ddb6a3a0 0x55c5ddb7ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.723+0000 7fac15600640 0 client.32432 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:44:59.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.724+0000 7fac31856740 1 --2- 172.21.15.47:0/2804907839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5ddb699e0 0x55c5ddb82460 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.724+0000 7fac31856740 1 --2- 172.21.15.47:0/2804907839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5ddb68810 0x55c5ddb81f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.724+0000 7fac31856740 1 -- 172.21.15.47:0/2804907839 >> 172.21.15.47:0/2804907839 conn(0x55c5ddb63c00 msgr2=0x55c5ddb62620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:59.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.725+0000 7fac31856740 1 -- 172.21.15.47:0/2804907839 shutdown_connections 2024-03-21T11:44:59.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.725+0000 7fac31856740 1 -- 172.21.15.47:0/2804907839 wait complete. 2024-03-21T11:44:59.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.727+0000 7fac31856740 20 client.32432 trim_cache size 0 max 16384 2024-03-21T11:44:59.728 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:44:59.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.728+0000 7fac2ec00640 1 -- 172.21.15.47:0/1578310572 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac0c0a3030 msgr2=0x7fac0c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:59.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.728+0000 7fac2ec00640 1 --2- 172.21.15.47:0/1578310572 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac0c0a3030 0x7fac0c0a54f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fac20009bf0 tx=0x7fac2002e040 comp rx=0 tx=0).stop 2024-03-21T11:44:59.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.728+0000 7fac2ec00640 1 -- 172.21.15.47:0/1578310572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac280ffd00 msgr2=0x7fac2819a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:44:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.728+0000 7fac2ec00640 1 --2- 172.21.15.47:0/1578310572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac280ffd00 0x7fac2819a1e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fac1000b3f0 tx=0x7fac1000b8b0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.728+0000 7fac2ec00640 1 -- 172.21.15.47:0/1578310572 shutdown_connections 2024-03-21T11:44:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.728+0000 7fac2ec00640 1 --2- 172.21.15.47:0/1578310572 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fac0c0a3030 0x7fac0c0a54f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.729+0000 7fac2ec00640 1 --2- 172.21.15.47:0/1578310572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac28107390 0x7fac281a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.729+0000 7fac2ec00640 1 --2- 172.21.15.47:0/1578310572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac280ffd00 0x7fac2819a1e0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.729+0000 7fac2ec00640 1 --2- 172.21.15.47:0/1578310572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac280ff310 0x7fac28199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:44:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.729+0000 7fac2ec00640 1 -- 172.21.15.47:0/1578310572 >> 172.21.15.47:0/1578310572 conn(0x7fac280fae60 msgr2=0x7fac280fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:44:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.729+0000 7fac2ec00640 1 -- 172.21.15.47:0/1578310572 shutdown_connections 2024-03-21T11:44:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:44:59.729+0000 7fac2ec00640 1 -- 172.21.15.47:0/1578310572 wait complete. 2024-03-21T11:44:59.760 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:44:59.760 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:44:59.975 INFO:tasks.workunit.client.0.smithi047.stdout: 10 201819 22.53 MB/sec execute 343 sec latency 320.671 ms 2024-03-21T11:45:00.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.065+0000 7f6d54000640 1 -- 172.21.15.47:0/2383022558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d4c105470 msgr2=0x7f6d4c10bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.065+0000 7f6d54000640 1 --2- 172.21.15.47:0/2383022558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d4c105470 0x7f6d4c10bd00 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009be0 tx=0x7f6d4402bf10 comp rx=0 tx=0).stop 2024-03-21T11:45:00.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.067+0000 7f6d54000640 1 -- 172.21.15.47:0/2383022558 shutdown_connections 2024-03-21T11:45:00.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.067+0000 7f6d54000640 1 --2- 172.21.15.47:0/2383022558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d4c105470 0x7f6d4c10bd00 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.067+0000 7f6d54000640 1 --2- 172.21.15.47:0/2383022558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d4c104ab0 0x7f6d4c104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.067+0000 7f6d54000640 1 --2- 172.21.15.47:0/2383022558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c1038c0 0x7f6d4c103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.067+0000 7f6d54000640 1 -- 172.21.15.47:0/2383022558 >> 172.21.15.47:0/2383022558 conn(0x7f6d4c0fee50 msgr2=0x7f6d4c101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:00.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.067+0000 7f6d54000640 1 -- 172.21.15.47:0/2383022558 shutdown_connections 2024-03-21T11:45:00.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.067+0000 7f6d54000640 1 -- 172.21.15.47:0/2383022558 wait complete. 2024-03-21T11:45:00.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.068+0000 7f6d54000640 1 Processor -- start 2024-03-21T11:45:00.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.069+0000 7f6d54000640 1 -- start start 2024-03-21T11:45:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.069+0000 7f6d54000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d4c1038c0 0x7f6d4c195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.069+0000 7f6d54000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c104ab0 0x7f6d4c195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.069+0000 7f6d54000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d4c105470 0x7f6d4c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.069+0000 7f6d54000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c10f590 con 0x7f6d4c105470 2024-03-21T11:45:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.069+0000 7f6d54000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c10f410 con 0x7f6d4c104ab0 2024-03-21T11:45:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.069+0000 7f6d54000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c10f710 con 0x7f6d4c1038c0 2024-03-21T11:45:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.070+0000 7f6d51600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d4c1038c0 0x7f6d4c195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.070+0000 7f6d50c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c104ab0 0x7f6d4c195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.070+0000 7f6d51600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d4c1038c0 0x7f6d4c195730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49086/0 (socket says 172.21.15.47:49086) 2024-03-21T11:45:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.070+0000 7f6d51600640 1 -- 172.21.15.47:0/3106088147 learned_addr learned my addr 172.21.15.47:0/3106088147 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.070+0000 7f6d52000640 1 --2- 172.21.15.47:0/3106088147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d4c105470 0x7f6d4c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.070+0000 7f6d50c00640 1 -- 172.21.15.47:0/3106088147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d4c1038c0 msgr2=0x7f6d4c195730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.070+0000 7f6d50c00640 1 --2- 172.21.15.47:0/3106088147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d4c1038c0 0x7f6d4c195730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.070+0000 7f6d50c00640 1 -- 172.21.15.47:0/3106088147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d4c105470 msgr2=0x7f6d4c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.070+0000 7f6d50c00640 1 --2- 172.21.15.47:0/3106088147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d4c105470 0x7f6d4c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.070+0000 7f6d50c00640 1 -- 172.21.15.47:0/3106088147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c1ae490 con 0x7f6d4c104ab0 2024-03-21T11:45:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.071+0000 7f6d50c00640 1 --2- 172.21.15.47:0/3106088147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c104ab0 0x7f6d4c195c70 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800d6d0 tx=0x7f6d3800dad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.071+0000 7f6d4a000640 1 -- 172.21.15.47:0/3106088147 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38017070 con 0x7f6d4c104ab0 2024-03-21T11:45:00.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.071+0000 7f6d4a000640 1 -- 172.21.15.47:0/3106088147 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d380047a0 con 0x7f6d4c104ab0 2024-03-21T11:45:00.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.071+0000 7f6d54000640 1 -- 172.21.15.47:0/3106088147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4c1ae660 con 0x7f6d4c104ab0 2024-03-21T11:45:00.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.071+0000 7f6d54000640 1 -- 172.21.15.47:0/3106088147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4c1aebb0 con 0x7f6d4c104ab0 2024-03-21T11:45:00.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.072+0000 7f6d4a000640 1 -- 172.21.15.47:0/3106088147 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38009c10 con 0x7f6d4c104ab0 2024-03-21T11:45:00.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.081+0000 7f6d4a000640 1 -- 172.21.15.47:0/3106088147 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6d38010460 con 0x7f6d4c104ab0 2024-03-21T11:45:00.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.082+0000 7f6d56c02740 1 Processor -- start 2024-03-21T11:45:00.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.083+0000 7f6d56c02740 1 -- start start 2024-03-21T11:45:00.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.083+0000 7f6d56c02740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af6810 0x55e1e7af6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.083+0000 7f6d56c02740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af79e0 0x55e1e7af7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.083+0000 7f6d56c02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af83a0 0x55e1e7afec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.083+0000 7f6d56c02740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e7a5b2c0 con 0x55e1e7af79e0 2024-03-21T11:45:00.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.083+0000 7f6d56c02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e7a03c90 con 0x55e1e7af83a0 2024-03-21T11:45:00.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.083+0000 7f6d56c02740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e7a57360 con 0x55e1e7af6810 2024-03-21T11:45:00.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.084+0000 7f6d52000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af6810 0x55e1e7af6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.084+0000 7f6d4a000640 1 --2- 172.21.15.47:0/3106088147 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d340a2fa0 0x7f6d340a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.084+0000 7f6d52000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af6810 0x55e1e7af6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49100/0 (socket says 172.21.15.47:49100) 2024-03-21T11:45:00.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.084+0000 7f6d52000640 1 -- 172.21.15.47:0/2487278298 learned_addr learned my addr 172.21.15.47:0/2487278298 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:00.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.084+0000 7f6d4a000640 1 -- 172.21.15.47:0/3106088147 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6d380cf0c0 con 0x7f6d4c104ab0 2024-03-21T11:45:00.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.084+0000 7f6d50c00640 1 --2- 172.21.15.47:0/2487278298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af83a0 0x55e1e7afec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.085+0000 7f6d51600640 1 --2- 172.21.15.47:0/2487278298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af79e0 0x55e1e7af7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.085+0000 7f6d50c00640 1 -- 172.21.15.47:0/2487278298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af6810 msgr2=0x55e1e7af6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.085+0000 7f6d50c00640 1 --2- 172.21.15.47:0/2487278298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af6810 0x55e1e7af6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.085+0000 7f6d50c00640 1 -- 172.21.15.47:0/2487278298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af79e0 msgr2=0x55e1e7af7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.085+0000 7f6d50c00640 1 --2- 172.21.15.47:0/2487278298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af79e0 0x55e1e7af7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.085+0000 7f6d50c00640 1 -- 172.21.15.47:0/2487278298 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1e7a28080 con 0x55e1e7af83a0 2024-03-21T11:45:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.085+0000 7f6d52000640 1 --2- 172.21.15.47:0/3106088147 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d340a2fa0 0x7f6d340a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.085+0000 7f6d50c00640 1 --2- 172.21.15.47:0/2487278298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af83a0 0x55e1e7afec30 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f6d3809b880 tx=0x7f6d3809b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ad79248f22cf778 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.086+0000 7f6d3e600640 1 -- 172.21.15.47:0/2487278298 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38020040 con 0x55e1e7af83a0 2024-03-21T11:45:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.086+0000 7f6d3e600640 1 -- 172.21.15.47:0/2487278298 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d3809b8f0 con 0x55e1e7af83a0 2024-03-21T11:45:00.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.086+0000 7f6d3e600640 1 -- 172.21.15.47:0/2487278298 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d380bb040 con 0x55e1e7af83a0 2024-03-21T11:45:00.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.087+0000 7f6d56c02740 1 -- 172.21.15.47:0/2487278298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af83a0 msgr2=0x55e1e7afec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.087+0000 7f6d56c02740 1 --2- 172.21.15.47:0/2487278298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af83a0 0x55e1e7afec30 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f6d3809b880 tx=0x7f6d3809b560 comp rx=0 tx=0).stop 2024-03-21T11:45:00.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.090+0000 7f6d52000640 1 --2- 172.21.15.47:0/3106088147 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d340a2fa0 0x7f6d340a5460 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x55e1e7af78e0 tx=0x7f6d4402bf10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.090+0000 7f6d56c02740 1 -- 172.21.15.47:0/2487278298 shutdown_connections 2024-03-21T11:45:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.090+0000 7f6d56c02740 1 --2- 172.21.15.47:0/2487278298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af83a0 0x55e1e7afec30 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.090+0000 7f6d56c02740 1 --2- 172.21.15.47:0/2487278298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af79e0 0x55e1e7af7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.090+0000 7f6d56c02740 1 --2- 172.21.15.47:0/2487278298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af6810 0x55e1e7af6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.090+0000 7f6d56c02740 1 -- 172.21.15.47:0/2487278298 >> 172.21.15.47:0/2487278298 conn(0x55e1e7af1c00 msgr2=0x55e1e7af2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.091+0000 7f6d56c02740 1 -- 172.21.15.47:0/2487278298 shutdown_connections 2024-03-21T11:45:00.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.092+0000 7f6d56c02740 1 -- 172.21.15.47:0/2487278298 wait complete. 2024-03-21T11:45:00.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.093+0000 7f6d56c02740 1 -- start start 2024-03-21T11:45:00.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.093+0000 7f6d56c02740 1 Processor -- start 2024-03-21T11:45:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.094+0000 7f6d56c02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af6810 0x55e1e7b00a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.094+0000 7f6d56c02740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af79e0 0x55e1e7b00fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.094+0000 7f6d56c02740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af83a0 0x55e1e7b08000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.094+0000 7f6d56c02740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e7aff3b0 con 0x55e1e7af79e0 2024-03-21T11:45:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.094+0000 7f6d56c02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e7a170d0 con 0x55e1e7af6810 2024-03-21T11:45:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.094+0000 7f6d56c02740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e7aff530 con 0x55e1e7af83a0 2024-03-21T11:45:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.094+0000 7f6d56c02740 10 client.0 _ll_register_callbacks cb 0x55e1e7a1be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.095+0000 7f6d52000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af83a0 0x55e1e7b08000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.095+0000 7f6d52000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af83a0 0x55e1e7b08000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49106/0 (socket says 172.21.15.47:49106) 2024-03-21T11:45:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.095+0000 7f6d52000640 1 -- 172.21.15.47:0/3240968631 learned_addr learned my addr 172.21.15.47:0/3240968631 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.095+0000 7f6d51600640 1 --2- 172.21.15.47:0/3240968631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af6810 0x55e1e7b00a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.095+0000 7f6d52000640 1 -- 172.21.15.47:0/3240968631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af6810 msgr2=0x55e1e7b00a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.095+0000 7f6d52000640 1 --2- 172.21.15.47:0/3240968631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af6810 0x55e1e7b00a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.095+0000 7f6d52000640 1 -- 172.21.15.47:0/3240968631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af79e0 msgr2=0x55e1e7b00fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.095+0000 7f6d52000640 1 --2- 172.21.15.47:0/3240968631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af79e0 0x55e1e7b00fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.095+0000 7f6d52000640 1 -- 172.21.15.47:0/3240968631 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1e776f3f0 con 0x55e1e7af83a0 2024-03-21T11:45:00.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.096+0000 7f6d32a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:00.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.096+0000 7f6d52000640 1 --2- 172.21.15.47:0/3240968631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af83a0 0x55e1e7b08000 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009be0 tx=0x7f6d44009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e042a4216d440cb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.097+0000 7f6d32a00640 1 -- 172.21.15.47:0/3240968631 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d44052070 con 0x55e1e7af83a0 2024-03-21T11:45:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.097+0000 7f6d32a00640 1 -- 172.21.15.47:0/3240968631 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6d4402ce00 con 0x55e1e7af83a0 2024-03-21T11:45:00.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.099+0000 7f6d56c02740 1 -- 172.21.15.47:0/3240968631 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d3809b8f0 con 0x55e1e7af83a0 2024-03-21T11:45:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.099+0000 7f6d32a00640 1 -- 172.21.15.47:0/3240968631 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d44003970 con 0x55e1e7af83a0 2024-03-21T11:45:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.099+0000 7f6d56c02740 10 client.32425 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.099+0000 7f6d56c02740 1 -- 172.21.15.47:0/3240968631 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e1e78397d0 con 0x55e1e7af83a0 2024-03-21T11:45:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.100+0000 7f6d32a00640 1 -- 172.21.15.47:0/3240968631 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d3809b8f0 con 0x55e1e7af83a0 2024-03-21T11:45:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.100+0000 7f6d32a00640 1 -- 172.21.15.47:0/3240968631 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6d44043450 con 0x55e1e7af83a0 2024-03-21T11:45:00.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.101+0000 7f6d56c02740 10 client.32425 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:00.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.101+0000 7f6d56c02740 10 client.32425 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:45:00.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.101+0000 7f6d56c02740 20 client.32425 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:00.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.101+0000 7f6d56c02740 1 --2- 172.21.15.47:0/3240968631 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e1e7b0c510 0x55e1e7b0e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.101+0000 7f6d56c02740 4 client.32425 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:00.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.101+0000 7f6d56c02740 1 -- 172.21.15.47:0/3240968631 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1e7a68800 con 0x55e1e7b0c510 2024-03-21T11:45:00.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.101+0000 7f6d51600640 1 --2- 172.21.15.47:0/3240968631 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e1e7b0c510 0x55e1e7b0e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.102+0000 7f6d32a00640 10 client.32425 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:45:00.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.102+0000 7f6d51600640 1 --2- 172.21.15.47:0/3240968631 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e1e7b0c510 0x55e1e7b0e950 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7bf585f0f884d20f server_cookie=84113fad00944205 in_seq=0 out_seq=0 2024-03-21T11:45:00.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.103+0000 7f6d32a00640 1 -- 172.21.15.47:0/3240968631 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e1e7a68800 con 0x55e1e7b0c510 2024-03-21T11:45:00.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.104+0000 7f6d32a00640 10 client.32425 handle_command_reply: tid=0 2024-03-21T11:45:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.104+0000 7f6d56c02740 1 client.32425 shutdown 2024-03-21T11:45:00.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.105+0000 7f6d56c02740 1 -- 172.21.15.47:0/3240968631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af83a0 msgr2=0x55e1e7b08000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.105+0000 7f6d56c02740 1 --2- 172.21.15.47:0/3240968631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af83a0 0x55e1e7b08000 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009be0 tx=0x7f6d44009500 comp rx=0 tx=0).stop 2024-03-21T11:45:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.106+0000 7f6d56c02740 1 -- 172.21.15.47:0/3240968631 shutdown_connections 2024-03-21T11:45:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.106+0000 7f6d56c02740 1 --2- 172.21.15.47:0/3240968631 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e1e7b0c510 0x55e1e7b0e950 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.106+0000 7f6d56c02740 1 --2- 172.21.15.47:0/3240968631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af83a0 0x55e1e7b08000 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.106+0000 7f6d56c02740 1 --2- 172.21.15.47:0/3240968631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af79e0 0x55e1e7b00fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.106+0000 7f6d56c02740 1 --2- 172.21.15.47:0/3240968631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af6810 0x55e1e7b00a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.106+0000 7f6d56c02740 1 -- 172.21.15.47:0/3240968631 >> 172.21.15.47:0/3240968631 conn(0x55e1e7af1c00 msgr2=0x55e1e7af0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:00.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.106+0000 7f6d32a00640 0 client.32425 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:45:00.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.107+0000 7f6d56c02740 1 -- 172.21.15.47:0/3240968631 shutdown_connections 2024-03-21T11:45:00.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.107+0000 7f6d56c02740 1 -- 172.21.15.47:0/3240968631 wait complete. 2024-03-21T11:45:00.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.108+0000 7f6d56c02740 20 client.32425 trim_cache size 0 max 16384 2024-03-21T11:45:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.151+0000 7f6d56c02740 1 Processor -- start 2024-03-21T11:45:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.154+0000 7f6d56c02740 1 -- start start 2024-03-21T11:45:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.154+0000 7f6d56c02740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af6810 0x55e1e7af5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.154+0000 7f6d56c02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af79e0 0x55e1e7b07fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.154+0000 7f6d56c02740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af83a0 0x55e1e7afac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.154+0000 7f6d56c02740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c1b8fc0 con 0x55e1e7af83a0 2024-03-21T11:45:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.154+0000 7f6d56c02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c1ba940 con 0x55e1e7af79e0 2024-03-21T11:45:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.154+0000 7f6d56c02740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c1b9620 con 0x55e1e7af6810 2024-03-21T11:45:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.155+0000 7f6d52000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af83a0 0x55e1e7afac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.155+0000 7f6d52000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af83a0 0x55e1e7afac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39328/0 (socket says 172.21.15.47:39328) 2024-03-21T11:45:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.155+0000 7f6d52000640 1 -- 172.21.15.47:0/776344330 learned_addr learned my addr 172.21.15.47:0/776344330 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.155+0000 7f6d51600640 1 --2- 172.21.15.47:0/776344330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af6810 0x55e1e7af5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.155+0000 7f6d50c00640 1 --2- 172.21.15.47:0/776344330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af79e0 0x55e1e7b07fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.155+0000 7f6d51600640 1 -- 172.21.15.47:0/776344330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af79e0 msgr2=0x55e1e7b07fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.155+0000 7f6d51600640 1 --2- 172.21.15.47:0/776344330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af79e0 0x55e1e7b07fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.155+0000 7f6d51600640 1 -- 172.21.15.47:0/776344330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af83a0 msgr2=0x55e1e7afac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.155+0000 7f6d51600640 1 --2- 172.21.15.47:0/776344330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af83a0 0x55e1e7afac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.155+0000 7f6d51600640 1 -- 172.21.15.47:0/776344330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1e7af7460 con 0x55e1e7af6810 2024-03-21T11:45:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.156+0000 7f6d52000640 1 --2- 172.21.15.47:0/776344330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af83a0 0x55e1e7afac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.156+0000 7f6d51600640 1 --2- 172.21.15.47:0/776344330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af6810 0x55e1e7af5570 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x55e1e7b0ef50 tx=0x7f6d40005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbb59eb0d2b8da4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.158+0000 7f6d3dc00640 1 -- 172.21.15.47:0/776344330 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d400145f0 con 0x55e1e7af6810 2024-03-21T11:45:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.158+0000 7f6d3dc00640 1 -- 172.21.15.47:0/776344330 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d40014c70 con 0x55e1e7af6810 2024-03-21T11:45:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.158+0000 7f6d3dc00640 1 -- 172.21.15.47:0/776344330 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40012c40 con 0x55e1e7af6810 2024-03-21T11:45:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.161+0000 7f6d56c02740 1 -- 172.21.15.47:0/776344330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af6810 msgr2=0x55e1e7af5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.161+0000 7f6d56c02740 1 --2- 172.21.15.47:0/776344330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af6810 0x55e1e7af5570 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x55e1e7b0ef50 tx=0x7f6d40005e80 comp rx=0 tx=0).stop 2024-03-21T11:45:00.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.161+0000 7f6d56c02740 1 -- 172.21.15.47:0/776344330 shutdown_connections 2024-03-21T11:45:00.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.161+0000 7f6d56c02740 1 --2- 172.21.15.47:0/776344330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af83a0 0x55e1e7afac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.161+0000 7f6d56c02740 1 --2- 172.21.15.47:0/776344330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af79e0 0x55e1e7b07fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.161+0000 7f6d56c02740 1 --2- 172.21.15.47:0/776344330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af6810 0x55e1e7af5570 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.161+0000 7f6d56c02740 1 -- 172.21.15.47:0/776344330 >> 172.21.15.47:0/776344330 conn(0x55e1e7af1c00 msgr2=0x55e1e7af4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.165+0000 7f6d56c02740 1 -- 172.21.15.47:0/776344330 shutdown_connections 2024-03-21T11:45:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.165+0000 7f6d56c02740 1 -- 172.21.15.47:0/776344330 wait complete. 2024-03-21T11:45:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.165+0000 7f6d56c02740 1 -- start start 2024-03-21T11:45:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.166+0000 7f6d56c02740 1 Processor -- start 2024-03-21T11:45:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.170+0000 7f6d56c02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af6810 0x55e1e7b05a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.171+0000 7f6d56c02740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af79e0 0x55e1e7b05f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.171+0000 7f6d56c02740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af83a0 0x55e1e7b06f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.171+0000 7f6d56c02740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c1ba2e0 con 0x55e1e7af79e0 2024-03-21T11:45:00.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.171+0000 7f6d56c02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c1b9c80 con 0x55e1e7af6810 2024-03-21T11:45:00.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.171+0000 7f6d56c02740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c1b7180 con 0x55e1e7af83a0 2024-03-21T11:45:00.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.171+0000 7f6d56c02740 10 client.0 _ll_register_callbacks cb 0x7f6d44009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:00.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.171+0000 7f6d52000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af83a0 0x55e1e7b06f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.171+0000 7f6d51600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af6810 0x55e1e7b05a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.172+0000 7f6d52000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af83a0 0x55e1e7b06f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49124/0 (socket says 172.21.15.47:49124) 2024-03-21T11:45:00.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.172+0000 7f6d52000640 1 -- 172.21.15.47:0/1773629499 learned_addr learned my addr 172.21.15.47:0/1773629499 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:00.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.172+0000 7f6d51600640 1 -- 172.21.15.47:0/1773629499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af83a0 msgr2=0x55e1e7b06f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.172+0000 7f6d51600640 1 --2- 172.21.15.47:0/1773629499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af83a0 0x55e1e7b06f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.172+0000 7f6d51600640 1 -- 172.21.15.47:0/1773629499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af79e0 msgr2=0x55e1e7b05f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.172+0000 7f6d51600640 1 --2- 172.21.15.47:0/1773629499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af79e0 0x55e1e7b05f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.172+0000 7f6d51600640 1 -- 172.21.15.47:0/1773629499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1e7b07570 con 0x55e1e7af6810 2024-03-21T11:45:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.173+0000 7f6d32a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:45:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.173+0000 7f6d51600640 1 --2- 172.21.15.47:0/1773629499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af6810 0x55e1e7b05a30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009c20 tx=0x7f6d40014f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3f752bed50d627 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.173+0000 7f6d32a00640 1 -- 172.21.15.47:0/1773629499 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40012650 con 0x55e1e7af6810 2024-03-21T11:45:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.174+0000 7f6d32a00640 1 -- 172.21.15.47:0/1773629499 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6d400129c0 con 0x55e1e7af6810 2024-03-21T11:45:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.174+0000 7f6d56c02740 1 -- 172.21.15.47:0/1773629499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d40014c70 con 0x55e1e7af6810 2024-03-21T11:45:00.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.174+0000 7f6d32a00640 1 -- 172.21.15.47:0/1773629499 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d40036050 con 0x55e1e7af6810 2024-03-21T11:45:00.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.174+0000 7f6d56c02740 10 client.32447 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:00.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.175+0000 7f6d56c02740 1 -- 172.21.15.47:0/1773629499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e1e7b680f0 con 0x55e1e7af6810 2024-03-21T11:45:00.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.176+0000 7f6d32a00640 1 -- 172.21.15.47:0/1773629499 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d40014c70 con 0x55e1e7af6810 2024-03-21T11:45:00.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.176+0000 7f6d32a00640 1 -- 172.21.15.47:0/1773629499 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6d40010880 con 0x55e1e7af6810 2024-03-21T11:45:00.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.176+0000 7f6d56c02740 10 client.32447 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:00.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.176+0000 7f6d56c02740 10 client.32447 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:45:00.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.176+0000 7f6d56c02740 20 client.32447 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:00.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.176+0000 7f6d56c02740 1 --2- 172.21.15.47:0/1773629499 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e1e7b68a40 0x55e1e7b6ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.176+0000 7f6d56c02740 4 client.32447 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:45:00.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.176+0000 7f6d56c02740 1 -- 172.21.15.47:0/1773629499 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e1e7a62b50 con 0x55e1e7b68a40 2024-03-21T11:45:00.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.179+0000 7f6d50c00640 1 --2- 172.21.15.47:0/1773629499 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e1e7b68a40 0x55e1e7b6ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.180+0000 7f6d50c00640 1 --2- 172.21.15.47:0/1773629499 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e1e7b68a40 0x55e1e7b6ae60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=732f788c166a3add server_cookie=645f855ee4dd9804 in_seq=0 out_seq=0 2024-03-21T11:45:00.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.181+0000 7f6d32a00640 10 client.32447 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:45:00.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.181+0000 7f6d32a00640 1 -- 172.21.15.47:0/1773629499 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e1e7a62b50 con 0x55e1e7b68a40 2024-03-21T11:45:00.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.181+0000 7f6d32a00640 10 client.32447 handle_command_reply: tid=0 2024-03-21T11:45:00.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.181+0000 7f6d56c02740 1 client.32447 shutdown 2024-03-21T11:45:00.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.182+0000 7f6d56c02740 1 -- 172.21.15.47:0/1773629499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af6810 msgr2=0x55e1e7b05a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.182+0000 7f6d56c02740 1 --2- 172.21.15.47:0/1773629499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af6810 0x55e1e7b05a30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009c20 tx=0x7f6d40014f10 comp rx=0 tx=0).stop 2024-03-21T11:45:00.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.182+0000 7f6d56c02740 1 -- 172.21.15.47:0/1773629499 shutdown_connections 2024-03-21T11:45:00.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.182+0000 7f6d56c02740 1 --2- 172.21.15.47:0/1773629499 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e1e7b68a40 0x55e1e7b6ae60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.182+0000 7f6d56c02740 1 --2- 172.21.15.47:0/1773629499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e7af83a0 0x55e1e7b06f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.182+0000 7f6d56c02740 1 --2- 172.21.15.47:0/1773629499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e7af79e0 0x55e1e7b05f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.182+0000 7f6d56c02740 1 --2- 172.21.15.47:0/1773629499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e7af6810 0x55e1e7b05a30 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.182+0000 7f6d32a00640 0 client.32447 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:45:00.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.182+0000 7f6d56c02740 1 -- 172.21.15.47:0/1773629499 >> 172.21.15.47:0/1773629499 conn(0x55e1e7af1c00 msgr2=0x55e1e7af56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:00.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.185+0000 7f6d56c02740 1 -- 172.21.15.47:0/1773629499 shutdown_connections 2024-03-21T11:45:00.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.185+0000 7f6d56c02740 1 -- 172.21.15.47:0/1773629499 wait complete. 2024-03-21T11:45:00.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.186+0000 7f6d56c02740 20 client.32447 trim_cache size 0 max 16384 2024-03-21T11:45:00.187 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:45:00.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.188+0000 7f6d54000640 1 -- 172.21.15.47:0/3106088147 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d340a2fa0 msgr2=0x7f6d340a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.188+0000 7f6d54000640 1 --2- 172.21.15.47:0/3106088147 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d340a2fa0 0x7f6d340a5460 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x55e1e7af78e0 tx=0x7f6d4402bf10 comp rx=0 tx=0).stop 2024-03-21T11:45:00.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.188+0000 7f6d54000640 1 -- 172.21.15.47:0/3106088147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c104ab0 msgr2=0x7f6d4c195c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.188+0000 7f6d54000640 1 --2- 172.21.15.47:0/3106088147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c104ab0 0x7f6d4c195c70 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800d6d0 tx=0x7f6d3800dad0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.189+0000 7f6d54000640 1 -- 172.21.15.47:0/3106088147 shutdown_connections 2024-03-21T11:45:00.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.189+0000 7f6d54000640 1 --2- 172.21.15.47:0/3106088147 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6d340a2fa0 0x7f6d340a5460 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.189+0000 7f6d54000640 1 --2- 172.21.15.47:0/3106088147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d4c105470 0x7f6d4c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.189+0000 7f6d54000640 1 --2- 172.21.15.47:0/3106088147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c104ab0 0x7f6d4c195c70 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.189+0000 7f6d54000640 1 --2- 172.21.15.47:0/3106088147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d4c1038c0 0x7f6d4c195730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.189+0000 7f6d54000640 1 -- 172.21.15.47:0/3106088147 >> 172.21.15.47:0/3106088147 conn(0x7f6d4c0fee50 msgr2=0x7f6d4c077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.189+0000 7f6d54000640 1 -- 172.21.15.47:0/3106088147 shutdown_connections 2024-03-21T11:45:00.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.189+0000 7f6d54000640 1 -- 172.21.15.47:0/3106088147 wait complete. 2024-03-21T11:45:00.237 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:45:00.237 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:45:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.359+0000 7f272ba00640 1 -- 172.21.15.47:0/2641576558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2724103900 msgr2=0x7f2724103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.359+0000 7f272ba00640 1 --2- 172.21.15.47:0/2641576558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2724103900 0x7f2724103d20 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f2718009900 tx=0x7f271802bc30 comp rx=0 tx=0).stop 2024-03-21T11:45:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.359+0000 7f272ba00640 1 -- 172.21.15.47:0/2641576558 shutdown_connections 2024-03-21T11:45:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.359+0000 7f272ba00640 1 --2- 172.21.15.47:0/2641576558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27241054b0 0x7f272410bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.359+0000 7f272ba00640 1 --2- 172.21.15.47:0/2641576558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2724104af0 0x7f2724104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.359+0000 7f272ba00640 1 --2- 172.21.15.47:0/2641576558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2724103900 0x7f2724103d20 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.359+0000 7f272ba00640 1 -- 172.21.15.47:0/2641576558 >> 172.21.15.47:0/2641576558 conn(0x7f27240fee90 msgr2=0x7f27241012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.360+0000 7f272ba00640 1 -- 172.21.15.47:0/2641576558 shutdown_connections 2024-03-21T11:45:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.360+0000 7f272ba00640 1 -- 172.21.15.47:0/2641576558 wait complete. 2024-03-21T11:45:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.360+0000 7f272ba00640 1 Processor -- start 2024-03-21T11:45:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.361+0000 7f272ba00640 1 -- start start 2024-03-21T11:45:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.361+0000 7f272ba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2724103900 0x7f2724072b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.361+0000 7f272ba00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2724104af0 0x7f27240730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.361+0000 7f272ba00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27241054b0 0x7f27240711b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.361+0000 7f272ba00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272406bb10 con 0x7f27241054b0 2024-03-21T11:45:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.361+0000 7f272ba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272406b990 con 0x7f2724103900 2024-03-21T11:45:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.361+0000 7f272ba00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272406bc90 con 0x7f2724104af0 2024-03-21T11:45:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.362+0000 7f2729000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2724103900 0x7f2724072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.362+0000 7f2729000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2724103900 0x7f2724072b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44186/0 (socket says 172.21.15.47:44186) 2024-03-21T11:45:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.362+0000 7f2729000640 1 -- 172.21.15.47:0/1501576910 learned_addr learned my addr 172.21.15.47:0/1501576910 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.362+0000 7f2729a00640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27241054b0 0x7f27240711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.363+0000 7f2723e00640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2724104af0 0x7f27240730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.363+0000 7f2729a00640 1 -- 172.21.15.47:0/1501576910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2724104af0 msgr2=0x7f27240730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.363+0000 7f2729a00640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2724104af0 0x7f27240730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.363+0000 7f2729a00640 1 -- 172.21.15.47:0/1501576910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2724103900 msgr2=0x7f2724072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.363+0000 7f2729a00640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2724103900 0x7f2724072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.363+0000 7f2729a00640 1 -- 172.21.15.47:0/1501576910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2724079cc0 con 0x7f27241054b0 2024-03-21T11:45:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.364+0000 7f2729000640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2724103900 0x7f2724072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.364+0000 7f2729a00640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27241054b0 0x7f27240711b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f271c009fd0 tx=0x7f271c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.365+0000 7f2721600640 1 -- 172.21.15.47:0/1501576910 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c00cd60 con 0x7f27241054b0 2024-03-21T11:45:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.365+0000 7f272ba00640 1 -- 172.21.15.47:0/1501576910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27240717b0 con 0x7f27241054b0 2024-03-21T11:45:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.365+0000 7f2721600640 1 -- 172.21.15.47:0/1501576910 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f271c005020 con 0x7f27241054b0 2024-03-21T11:45:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.365+0000 7f2721600640 1 -- 172.21.15.47:0/1501576910 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c010690 con 0x7f27241054b0 2024-03-21T11:45:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.365+0000 7f272ba00640 1 -- 172.21.15.47:0/1501576910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2724071ca0 con 0x7f27241054b0 2024-03-21T11:45:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.372+0000 7f2721600640 1 -- 172.21.15.47:0/1501576910 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f271c010830 con 0x7f27241054b0 2024-03-21T11:45:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.376+0000 7f272e7d2740 1 Processor -- start 2024-03-21T11:45:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.379+0000 7f2721600640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f270c0a3040 0x7f270c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.379+0000 7f272e7d2740 1 -- start start 2024-03-21T11:45:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.379+0000 7f2721600640 1 -- 172.21.15.47:0/1501576910 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f271c0cfd70 con 0x7f27241054b0 2024-03-21T11:45:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.379+0000 7f272e7d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f601e0 0x559894f60600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.379+0000 7f272e7d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.380+0000 7f272e7d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f65d40 0x559894f681d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.380+0000 7f272e7d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559894f233b0 con 0x559894f601e0 2024-03-21T11:45:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.380+0000 7f272e7d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559894ec5830 con 0x559894f65d40 2024-03-21T11:45:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.380+0000 7f272e7d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559894e6e5d0 con 0x559894f60bd0 2024-03-21T11:45:00.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.380+0000 7f2723e00640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f270c0a3040 0x7f270c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.380+0000 7f2729a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f65d40 0x559894f681d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.381+0000 7f2729a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f65d40 0x559894f681d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44188/0 (socket says 172.21.15.47:44188) 2024-03-21T11:45:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.381+0000 7f2729a00640 1 -- 172.21.15.47:0/2914953033 learned_addr learned my addr 172.21.15.47:0/2914953033 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.381+0000 7f2723e00640 1 --2- 172.21.15.47:0/2914953033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.381+0000 7f2729a00640 1 -- 172.21.15.47:0/2914953033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 msgr2=0x559894f656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.381+0000 7f2729a00640 1 --2- 172.21.15.47:0/2914953033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.381+0000 7f2729a00640 1 -- 172.21.15.47:0/2914953033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f601e0 msgr2=0x559894f60600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.381+0000 7f2729a00640 1 --2- 172.21.15.47:0/2914953033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f601e0 0x559894f60600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.381+0000 7f2729a00640 1 -- 172.21.15.47:0/2914953033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559894e862c0 con 0x559894f65d40 2024-03-21T11:45:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.382+0000 7f2723e00640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f270c0a3040 0x7f270c0a5500 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f27240728e0 tx=0x7f27140156d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.382+0000 7f2729a00640 1 --2- 172.21.15.47:0/2914953033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f65d40 0x559894f681d0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f271c099b20 tx=0x7f271c0cd5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c02de8ad0b591ea0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.382+0000 7f2712000640 1 -- 172.21.15.47:0/2914953033 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c0b4070 con 0x559894f65d40 2024-03-21T11:45:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.383+0000 7f2712000640 1 -- 172.21.15.47:0/2914953033 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f271c0997c0 con 0x559894f65d40 2024-03-21T11:45:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.383+0000 7f2712000640 1 -- 172.21.15.47:0/2914953033 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c09c010 con 0x559894f65d40 2024-03-21T11:45:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.384+0000 7f272e7d2740 1 -- 172.21.15.47:0/2914953033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f65d40 msgr2=0x559894f681d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.384+0000 7f272e7d2740 1 --2- 172.21.15.47:0/2914953033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f65d40 0x559894f681d0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f271c099b20 tx=0x7f271c0cd5f0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.385+0000 7f272e7d2740 1 -- 172.21.15.47:0/2914953033 shutdown_connections 2024-03-21T11:45:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.385+0000 7f272e7d2740 1 --2- 172.21.15.47:0/2914953033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f65d40 0x559894f681d0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.385+0000 7f272e7d2740 1 --2- 172.21.15.47:0/2914953033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.385+0000 7f272e7d2740 1 --2- 172.21.15.47:0/2914953033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f601e0 0x559894f60600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.385+0000 7f272e7d2740 1 -- 172.21.15.47:0/2914953033 >> 172.21.15.47:0/2914953033 conn(0x559894f5b750 msgr2=0x559894f5bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.385+0000 7f272e7d2740 1 -- 172.21.15.47:0/2914953033 shutdown_connections 2024-03-21T11:45:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.386+0000 7f272e7d2740 1 -- 172.21.15.47:0/2914953033 wait complete. 2024-03-21T11:45:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.386+0000 7f272e7d2740 1 -- start start 2024-03-21T11:45:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.387+0000 7f272e7d2740 1 Processor -- start 2024-03-21T11:45:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.388+0000 7f272e7d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f601e0 0x559894f6a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.389+0000 7f272e7d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f60bd0 0x559894f6a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.389+0000 7f272e7d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f65d40 0x559894f71940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.389+0000 7f272e7d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559894e81a20 con 0x559894f60bd0 2024-03-21T11:45:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.389+0000 7f272e7d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559894ec18d0 con 0x559894f601e0 2024-03-21T11:45:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.389+0000 7f272e7d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559894f68cf0 con 0x559894f65d40 2024-03-21T11:45:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.389+0000 7f272e7d2740 10 client.0 _ll_register_callbacks cb 0x559894ebe1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.389+0000 7f2729a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f65d40 0x559894f71940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.390+0000 7f2729a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f65d40 0x559894f71940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49162/0 (socket says 172.21.15.47:49162) 2024-03-21T11:45:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.390+0000 7f2729a00640 1 -- 172.21.15.47:0/846359968 learned_addr learned my addr 172.21.15.47:0/846359968 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.390+0000 7f2723e00640 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f60bd0 0x559894f6a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.390+0000 7f2729a00640 1 -- 172.21.15.47:0/846359968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f601e0 msgr2=0x559894f6a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.391+0000 7f2729000640 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f601e0 0x559894f6a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.391+0000 7f2729a00640 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f601e0 0x559894f6a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.391+0000 7f2729a00640 1 -- 172.21.15.47:0/846359968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f60bd0 msgr2=0x559894f6a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.391+0000 7f2729a00640 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f60bd0 0x559894f6a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.391+0000 7f2729a00640 1 -- 172.21.15.47:0/846359968 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559894bda210 con 0x559894f65d40 2024-03-21T11:45:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.391+0000 7f2729000640 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f601e0 0x559894f6a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:45:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.392+0000 7f2723e00640 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f60bd0 0x559894f6a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.393+0000 7f2729a00640 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f65d40 0x559894f71940 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f271c00cfd0 tx=0x7f271c099320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9245b493f390505d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.393+0000 7f2706a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.393+0000 7f2706a00640 1 -- 172.21.15.47:0/846359968 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c080060 con 0x559894f65d40 2024-03-21T11:45:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.394+0000 7f2706a00640 1 -- 172.21.15.47:0/846359968 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f271c09c010 con 0x559894f65d40 2024-03-21T11:45:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.394+0000 7f272e7d2740 1 -- 172.21.15.47:0/846359968 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f271c0997c0 con 0x559894f65d40 2024-03-21T11:45:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.395+0000 7f2706a00640 1 -- 172.21.15.47:0/846359968 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f271c0bb070 con 0x559894f65d40 2024-03-21T11:45:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.395+0000 7f272e7d2740 10 client.32443 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.395+0000 7f272e7d2740 1 -- 172.21.15.47:0/846359968 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559894ca6620 con 0x559894f65d40 2024-03-21T11:45:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.396+0000 7f2706a00640 1 -- 172.21.15.47:0/846359968 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f271c0997c0 con 0x559894f65d40 2024-03-21T11:45:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.396+0000 7f2706a00640 1 -- 172.21.15.47:0/846359968 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f271c0a7070 con 0x559894f65d40 2024-03-21T11:45:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.397+0000 7f272e7d2740 10 client.32443 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.397+0000 7f272e7d2740 10 client.32443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.397+0000 7f272e7d2740 20 client.32443 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.397+0000 7f272e7d2740 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559894f75e30 0x559894f78220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.397+0000 7f272e7d2740 4 client.32443 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.397+0000 7f272e7d2740 1 -- 172.21.15.47:0/846359968 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559894ed2d70 con 0x559894f75e30 2024-03-21T11:45:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.397+0000 7f2729000640 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559894f75e30 0x559894f78220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.398+0000 7f2706a00640 10 client.32443 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.400+0000 7f2729000640 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559894f75e30 0x559894f78220 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3ffa5c039bae581 server_cookie=3bb019bc5e13a683 in_seq=0 out_seq=0 2024-03-21T11:45:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.402+0000 7f2706a00640 1 -- 172.21.15.47:0/846359968 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559894ed2d70 con 0x559894f75e30 2024-03-21T11:45:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.402+0000 7f2706a00640 10 client.32443 handle_command_reply: tid=0 2024-03-21T11:45:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.402+0000 7f272e7d2740 1 client.32443 shutdown 2024-03-21T11:45:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.403+0000 7f272e7d2740 1 -- 172.21.15.47:0/846359968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f65d40 msgr2=0x559894f71940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.403+0000 7f272e7d2740 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f65d40 0x559894f71940 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f271c00cfd0 tx=0x7f271c099320 comp rx=0 tx=0).stop 2024-03-21T11:45:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.405+0000 7f272e7d2740 1 -- 172.21.15.47:0/846359968 shutdown_connections 2024-03-21T11:45:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.405+0000 7f272e7d2740 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559894f75e30 0x559894f78220 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.405+0000 7f272e7d2740 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f65d40 0x559894f71940 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.405+0000 7f272e7d2740 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f60bd0 0x559894f6a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.405+0000 7f272e7d2740 1 --2- 172.21.15.47:0/846359968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f601e0 0x559894f6a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.405+0000 7f272e7d2740 1 -- 172.21.15.47:0/846359968 >> 172.21.15.47:0/846359968 conn(0x559894f5b750 msgr2=0x559894f5a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.405+0000 7f2706a00640 0 client.32443 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.405+0000 7f272e7d2740 1 -- 172.21.15.47:0/846359968 shutdown_connections 2024-03-21T11:45:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.405+0000 7f272e7d2740 1 -- 172.21.15.47:0/846359968 wait complete. 2024-03-21T11:45:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.406+0000 7f272e7d2740 20 client.32443 trim_cache size 0 max 16384 2024-03-21T11:45:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.440+0000 7f272e7d2740 1 Processor -- start 2024-03-21T11:45:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.441+0000 7f272e7d2740 1 -- start start 2024-03-21T11:45:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.441+0000 7f272e7d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f601e0 0x559894f699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.442+0000 7f272e7d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f71900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.442+0000 7f272e7d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f65d40 0x559894f6a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.442+0000 7f272e7d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27241b8ea0 con 0x559894f65d40 2024-03-21T11:45:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.442+0000 7f272e7d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27241ba820 con 0x559894f601e0 2024-03-21T11:45:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.442+0000 7f272e7d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27241b9500 con 0x559894f60bd0 2024-03-21T11:45:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.442+0000 7f2729000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f601e0 0x559894f699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.442+0000 7f2729000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f601e0 0x559894f699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44212/0 (socket says 172.21.15.47:44212) 2024-03-21T11:45:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.442+0000 7f2729000640 1 -- 172.21.15.47:0/3774341659 learned_addr learned my addr 172.21.15.47:0/3774341659 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.443+0000 7f2723e00640 1 --2- 172.21.15.47:0/3774341659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f71900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.444+0000 7f2729000640 1 -- 172.21.15.47:0/3774341659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 msgr2=0x559894f71900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.444+0000 7f2729000640 1 --2- 172.21.15.47:0/3774341659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f71900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.444+0000 7f2729000640 1 -- 172.21.15.47:0/3774341659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f65d40 msgr2=0x559894f6a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.444+0000 7f2729000640 1 --2- 172.21.15.47:0/3774341659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f65d40 0x559894f6a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.445+0000 7f2729000640 1 -- 172.21.15.47:0/3774341659 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559894f619e0 con 0x559894f601e0 2024-03-21T11:45:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.445+0000 7f2723e00640 1 --2- 172.21.15.47:0/3774341659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f71900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:45:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.446+0000 7f2729000640 1 --2- 172.21.15.47:0/3774341659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f601e0 0x559894f699e0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f271802c8e0 tx=0x7f2718005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c170d338a6b64ebe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.447+0000 7f2711600640 1 -- 172.21.15.47:0/3774341659 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718004100 con 0x559894f601e0 2024-03-21T11:45:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.447+0000 7f2711600640 1 -- 172.21.15.47:0/3774341659 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2718049030 con 0x559894f601e0 2024-03-21T11:45:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.447+0000 7f2711600640 1 -- 172.21.15.47:0/3774341659 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718004c20 con 0x559894f601e0 2024-03-21T11:45:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.449+0000 7f272e7d2740 1 -- 172.21.15.47:0/3774341659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f601e0 msgr2=0x559894f699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.449+0000 7f272e7d2740 1 --2- 172.21.15.47:0/3774341659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f601e0 0x559894f699e0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f271802c8e0 tx=0x7f2718005bb0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.449+0000 7f272e7d2740 1 -- 172.21.15.47:0/3774341659 shutdown_connections 2024-03-21T11:45:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.449+0000 7f272e7d2740 1 --2- 172.21.15.47:0/3774341659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f65d40 0x559894f6a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.449+0000 7f272e7d2740 1 --2- 172.21.15.47:0/3774341659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f71900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.449+0000 7f272e7d2740 1 --2- 172.21.15.47:0/3774341659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f601e0 0x559894f699e0 secure :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f271802c8e0 tx=0x7f2718005bb0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.449+0000 7f272e7d2740 1 -- 172.21.15.47:0/3774341659 >> 172.21.15.47:0/3774341659 conn(0x559894f5b750 msgr2=0x559894f5bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.450+0000 7f272e7d2740 1 -- 172.21.15.47:0/3774341659 shutdown_connections 2024-03-21T11:45:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.450+0000 7f272e7d2740 1 -- 172.21.15.47:0/3774341659 wait complete. 2024-03-21T11:45:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.450+0000 7f272e7d2740 1 -- start start 2024-03-21T11:45:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.450+0000 7f272e7d2740 1 Processor -- start 2024-03-21T11:45:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.451+0000 7f272e7d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f6f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.451+0000 7f272e7d2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f65d40 0x559894f6f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.451+0000 7f272e7d2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f707b0 0x559894f70c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.451+0000 7f272e7d2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27241ba1c0 con 0x559894f65d40 2024-03-21T11:45:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.451+0000 7f272e7d2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27241b9b60 con 0x559894f707b0 2024-03-21T11:45:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.451+0000 7f272e7d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27241b7060 con 0x559894f60bd0 2024-03-21T11:45:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.452+0000 7f2729000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f6f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.452+0000 7f272e7d2740 10 client.0 _ll_register_callbacks cb 0x7f271c0a5020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.452+0000 7f2729000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f6f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49188/0 (socket says 172.21.15.47:49188) 2024-03-21T11:45:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.452+0000 7f2729000640 1 -- 172.21.15.47:0/3745977538 learned_addr learned my addr 172.21.15.47:0/3745977538 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.452+0000 7f2729a00640 1 --2- 172.21.15.47:0/3745977538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f707b0 0x559894f70c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.452+0000 7f2729000640 1 -- 172.21.15.47:0/3745977538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f707b0 msgr2=0x559894f70c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.452+0000 7f2729000640 1 --2- 172.21.15.47:0/3745977538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f707b0 0x559894f70c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.452+0000 7f2729000640 1 -- 172.21.15.47:0/3745977538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f65d40 msgr2=0x559894f6f7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:45:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.453+0000 7f2729000640 1 --2- 172.21.15.47:0/3745977538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f65d40 0x559894f6f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.453+0000 7f2729000640 1 -- 172.21.15.47:0/3745977538 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559894f71170 con 0x559894f60bd0 2024-03-21T11:45:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.453+0000 7f2729a00640 1 --2- 172.21.15.47:0/3745977538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f707b0 0x559894f70c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:45:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.453+0000 7f2729000640 1 --2- 172.21.15.47:0/3745977538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f6f260 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f27180092a0 tx=0x7f2718004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bd3712b9e094d73 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.453+0000 7f2706a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.455+0000 7f2706a00640 1 -- 172.21.15.47:0/3745977538 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718052070 con 0x559894f60bd0 2024-03-21T11:45:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.455+0000 7f2706a00640 1 -- 172.21.15.47:0/3745977538 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2718004ac0 con 0x559894f60bd0 2024-03-21T11:45:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.456+0000 7f272e7d2740 1 -- 172.21.15.47:0/3745977538 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2718049030 con 0x559894f60bd0 2024-03-21T11:45:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.457+0000 7f2706a00640 1 -- 172.21.15.47:0/3745977538 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2718004cc0 con 0x559894f60bd0 2024-03-21T11:45:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.457+0000 7f272e7d2740 10 client.32449 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.457+0000 7f272e7d2740 1 -- 172.21.15.47:0/3745977538 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559894fd1be0 con 0x559894f60bd0 2024-03-21T11:45:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.457+0000 7f2706a00640 1 -- 172.21.15.47:0/3745977538 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2718049030 con 0x559894f60bd0 2024-03-21T11:45:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.457+0000 7f2706a00640 1 -- 172.21.15.47:0/3745977538 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2718003710 con 0x559894f60bd0 2024-03-21T11:45:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.457+0000 7f272e7d2740 10 client.32449 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.457+0000 7f272e7d2740 10 client.32449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.458+0000 7f272e7d2740 20 client.32449 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.458+0000 7f272e7d2740 1 --2- 172.21.15.47:0/3745977538 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559894fd23a0 0x559894fd47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.458+0000 7f272e7d2740 4 client.32449 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:45:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.458+0000 7f272e7d2740 1 -- 172.21.15.47:0/3745977538 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559894ecd0c0 con 0x559894fd23a0 2024-03-21T11:45:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.460+0000 7f2723e00640 1 --2- 172.21.15.47:0/3745977538 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559894fd23a0 0x559894fd47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.461+0000 7f2723e00640 1 --2- 172.21.15.47:0/3745977538 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559894fd23a0 0x559894fd47c0 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17aaf5ce5334131f server_cookie=14e5c5639e6d5059 in_seq=0 out_seq=0 2024-03-21T11:45:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:00.461+0000 7f2706a00640 10 client.32449 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.585+0000 7fd90ea00640 1 -- 172.21.15.47:0/1984956476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd908102790 msgr2=0x7fd908102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.585+0000 7fd90ea00640 1 --2- 172.21.15.47:0/1984956476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd908102790 0x7fd908102c10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009900 tx=0x7fd8fc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:45:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.589+0000 7fd90ea00640 1 -- 172.21.15.47:0/1984956476 shutdown_connections 2024-03-21T11:45:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.589+0000 7fd90ea00640 1 --2- 172.21.15.47:0/1984956476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd908103150 0x7fd9081099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.589+0000 7fd90ea00640 1 --2- 172.21.15.47:0/1984956476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd908102790 0x7fd908102c10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.589+0000 7fd90ea00640 1 --2- 172.21.15.47:0/1984956476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9081015a0 0x7fd9081019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.589+0000 7fd90ea00640 1 -- 172.21.15.47:0/1984956476 >> 172.21.15.47:0/1984956476 conn(0x7fd9080fcb10 msgr2=0x7fd9080fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.590+0000 7fd90ea00640 1 -- 172.21.15.47:0/1984956476 shutdown_connections 2024-03-21T11:45:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.590+0000 7fd90ea00640 1 -- 172.21.15.47:0/1984956476 wait complete. 2024-03-21T11:45:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.594+0000 7fd90ea00640 1 Processor -- start 2024-03-21T11:45:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.594+0000 7fd90ea00640 1 -- start start 2024-03-21T11:45:00.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.595+0000 7fd90ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9081015a0 0x7fd90819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.595+0000 7fd90ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd908102790 0x7fd90819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.595+0000 7fd90cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd908102790 0x7fd90819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.595+0000 7fd90cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd908102790 0x7fd90819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39412/0 (socket says 172.21.15.47:39412) 2024-03-21T11:45:00.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.596+0000 7fd90d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9081015a0 0x7fd90819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.596+0000 7fd90d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9081015a0 0x7fd90819dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44244/0 (socket says 172.21.15.47:44244) 2024-03-21T11:45:00.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.598+0000 7fd90ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd908103150 0x7fd9081a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.598+0000 7fd90cc00640 1 -- 172.21.15.47:0/2699425401 learned_addr learned my addr 172.21.15.47:0/2699425401 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:00.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.598+0000 7fd90ea00640 1 -- 172.21.15.47:0/2699425401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90810d270 con 0x7fd908102790 2024-03-21T11:45:00.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.598+0000 7fd90ea00640 1 -- 172.21.15.47:0/2699425401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90810d0f0 con 0x7fd9081015a0 2024-03-21T11:45:00.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.598+0000 7fd90ea00640 1 -- 172.21.15.47:0/2699425401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90810d3f0 con 0x7fd908103150 2024-03-21T11:45:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.598+0000 7fd90e000640 1 --2- 172.21.15.47:0/2699425401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd908103150 0x7fd9081a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.598+0000 7fd90d600640 1 -- 172.21.15.47:0/2699425401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd908103150 msgr2=0x7fd9081a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.599+0000 7fd90d600640 1 --2- 172.21.15.47:0/2699425401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd908103150 0x7fd9081a5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.599+0000 7fd90d600640 1 -- 172.21.15.47:0/2699425401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd908102790 msgr2=0x7fd90819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.599+0000 7fd90d600640 1 --2- 172.21.15.47:0/2699425401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd908102790 0x7fd90819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.599+0000 7fd90d600640 1 -- 172.21.15.47:0/2699425401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9081a5ae0 con 0x7fd9081015a0 2024-03-21T11:45:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.599+0000 7fd90cc00640 1 --2- 172.21.15.47:0/2699425401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd908102790 0x7fd90819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:00.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.600+0000 7fd90d600640 1 --2- 172.21.15.47:0/2699425401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9081015a0 0x7fd90819dec0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fd900005fd0 tx=0x7fd90000ef30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.601+0000 7fd906000640 1 -- 172.21.15.47:0/2699425401 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90000cde0 con 0x7fd9081015a0 2024-03-21T11:45:00.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.601+0000 7fd906000640 1 -- 172.21.15.47:0/2699425401 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd900011950 con 0x7fd9081015a0 2024-03-21T11:45:00.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.601+0000 7fd906000640 1 -- 172.21.15.47:0/2699425401 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90001aa80 con 0x7fd9081015a0 2024-03-21T11:45:00.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.601+0000 7fd90ea00640 1 -- 172.21.15.47:0/2699425401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9081a5dd0 con 0x7fd9081015a0 2024-03-21T11:45:00.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.602+0000 7fd90ea00640 1 -- 172.21.15.47:0/2699425401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9081a6240 con 0x7fd9081015a0 2024-03-21T11:45:00.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.606+0000 7fd906000640 1 -- 172.21.15.47:0/2699425401 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd900011b00 con 0x7fd9081015a0 2024-03-21T11:45:00.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.609+0000 7fd906000640 1 --2- 172.21.15.47:0/2699425401 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8f00a2fa0 0x7fd8f00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.609+0000 7fd906000640 1 -- 172.21.15.47:0/2699425401 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd9000cfd20 con 0x7fd9081015a0 2024-03-21T11:45:00.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.610+0000 7fd90cc00640 1 --2- 172.21.15.47:0/2699425401 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8f00a2fa0 0x7fd8f00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.611+0000 7fd90cc00640 1 --2- 172.21.15.47:0/2699425401 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8f00a2fa0 0x7fd8f00a5460 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc02c8e0 tx=0x7fd8fc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.613+0000 7fd912d87740 1 Processor -- start 2024-03-21T11:45:00.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.614+0000 7fd912d87740 1 -- start start 2024-03-21T11:45:00.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.614+0000 7fd912d87740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bb9f0 0x55a90e7bbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.614+0000 7fd912d87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bcbc0 0x55a90e7bd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.614+0000 7fd912d87740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bd580 0x55a90e7c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.614+0000 7fd912d87740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90e7202c0 con 0x55a90e7bb9f0 2024-03-21T11:45:00.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.614+0000 7fd912d87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90e6c8c90 con 0x55a90e7bcbc0 2024-03-21T11:45:00.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.614+0000 7fd912d87740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90e71c360 con 0x55a90e7bd580 2024-03-21T11:45:00.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.615+0000 7fd90d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bb9f0 0x55a90e7bbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.615+0000 7fd90cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bcbc0 0x55a90e7bd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.615+0000 7fd90d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bb9f0 0x55a90e7bbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39416/0 (socket says 172.21.15.47:39416) 2024-03-21T11:45:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.615+0000 7fd90d600640 1 -- 172.21.15.47:0/1884256818 learned_addr learned my addr 172.21.15.47:0/1884256818 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.615+0000 7fd90e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bd580 0x55a90e7c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.616+0000 7fd90d600640 1 -- 172.21.15.47:0/1884256818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bd580 msgr2=0x55a90e7c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.616+0000 7fd90d600640 1 --2- 172.21.15.47:0/1884256818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bd580 0x55a90e7c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.616+0000 7fd90d600640 1 -- 172.21.15.47:0/1884256818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bcbc0 msgr2=0x55a90e7bd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.616+0000 7fd90d600640 1 --2- 172.21.15.47:0/1884256818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bcbc0 0x55a90e7bd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.616+0000 7fd90d600640 1 -- 172.21.15.47:0/1884256818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a90e6ed080 con 0x55a90e7bb9f0 2024-03-21T11:45:00.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.616+0000 7fd90d600640 1 --2- 172.21.15.47:0/1884256818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bb9f0 0x55a90e7bbe10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd900098a90 tx=0x7fd90009c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23506bb9dc5fb7a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.617+0000 7fd8fa600640 1 -- 172.21.15.47:0/1884256818 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90009a3c0 con 0x55a90e7bb9f0 2024-03-21T11:45:00.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.617+0000 7fd8fa600640 1 -- 172.21.15.47:0/1884256818 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd900011470 con 0x55a90e7bb9f0 2024-03-21T11:45:00.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.617+0000 7fd8fa600640 1 -- 172.21.15.47:0/1884256818 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90009c320 con 0x55a90e7bb9f0 2024-03-21T11:45:00.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.618+0000 7fd912d87740 1 -- 172.21.15.47:0/1884256818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bb9f0 msgr2=0x55a90e7bbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.618+0000 7fd912d87740 1 --2- 172.21.15.47:0/1884256818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bb9f0 0x55a90e7bbe10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd900098a90 tx=0x7fd90009c0c0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.618+0000 7fd912d87740 1 -- 172.21.15.47:0/1884256818 shutdown_connections 2024-03-21T11:45:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.618+0000 7fd912d87740 1 --2- 172.21.15.47:0/1884256818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bd580 0x55a90e7c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.618+0000 7fd912d87740 1 --2- 172.21.15.47:0/1884256818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bcbc0 0x55a90e7bd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.618+0000 7fd912d87740 1 --2- 172.21.15.47:0/1884256818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bb9f0 0x55a90e7bbe10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.618+0000 7fd912d87740 1 -- 172.21.15.47:0/1884256818 >> 172.21.15.47:0/1884256818 conn(0x55a90e7b6de0 msgr2=0x55a90e7b71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:00.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.618+0000 7fd912d87740 1 -- 172.21.15.47:0/1884256818 shutdown_connections 2024-03-21T11:45:00.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.620+0000 7fd912d87740 1 -- 172.21.15.47:0/1884256818 wait complete. 2024-03-21T11:45:00.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.620+0000 7fd912d87740 1 -- start start 2024-03-21T11:45:00.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.621+0000 7fd912d87740 1 Processor -- start 2024-03-21T11:45:00.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.621+0000 7fd912d87740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bb9f0 0x55a90e7c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.621+0000 7fd912d87740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bcbc0 0x55a90e7c6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.621+0000 7fd912d87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bd580 0x55a90e7cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.621+0000 7fd912d87740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90e7c4590 con 0x55a90e7bcbc0 2024-03-21T11:45:00.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.621+0000 7fd912d87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90e6dc0d0 con 0x55a90e7bd580 2024-03-21T11:45:00.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.621+0000 7fd912d87740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90e7c4710 con 0x55a90e7bb9f0 2024-03-21T11:45:00.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.621+0000 7fd912d87740 10 client.0 _ll_register_callbacks cb 0x55a90e6e0e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:00.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.622+0000 7fd90e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bd580 0x55a90e7cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.622+0000 7fd90e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bd580 0x55a90e7cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44252/0 (socket says 172.21.15.47:44252) 2024-03-21T11:45:00.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.622+0000 7fd90e000640 1 -- 172.21.15.47:0/1709959887 learned_addr learned my addr 172.21.15.47:0/1709959887 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:00.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.622+0000 7fd90e000640 1 -- 172.21.15.47:0/1709959887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bb9f0 msgr2=0x55a90e7c5cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:45:00.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.622+0000 7fd90e000640 1 --2- 172.21.15.47:0/1709959887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bb9f0 0x55a90e7c5cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.622+0000 7fd90e000640 1 -- 172.21.15.47:0/1709959887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bcbc0 msgr2=0x55a90e7c6210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:45:00.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.623+0000 7fd90cc00640 1 --2- 172.21.15.47:0/1709959887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bcbc0 0x55a90e7c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.623+0000 7fd90e000640 1 --2- 172.21.15.47:0/1709959887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bcbc0 0x55a90e7c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:00.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.623+0000 7fd90e000640 1 -- 172.21.15.47:0/1709959887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a90e4343f0 con 0x55a90e7bd580 2024-03-21T11:45:00.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.624+0000 7fd90cc00640 1 --2- 172.21.15.47:0/1709959887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bcbc0 0x55a90e7c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:45:00.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.625+0000 7fd90e000640 1 --2- 172.21.15.47:0/1709959887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bd580 0x55a90e7cd250 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400be70 tx=0x7fd8f4012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f38e74ce4d00768 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:00.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.625+0000 7fd8eea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:45:00.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.626+0000 7fd8eea00640 1 -- 172.21.15.47:0/1709959887 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f401f060 con 0x55a90e7bd580 2024-03-21T11:45:00.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.627+0000 7fd912d87740 1 -- 172.21.15.47:0/1709959887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd900011470 con 0x55a90e7bd580 2024-03-21T11:45:00.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.627+0000 7fd8eea00640 1 -- 172.21.15.47:0/1709959887 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd8f40107e0 con 0x55a90e7bd580 2024-03-21T11:45:00.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.628+0000 7fd8eea00640 1 -- 172.21.15.47:0/1709959887 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8f4026050 con 0x55a90e7bd580 2024-03-21T11:45:00.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.629+0000 7fd912d87740 10 client.32474 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:00.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.629+0000 7fd912d87740 1 -- 172.21.15.47:0/1709959887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a90e4fe7d0 con 0x55a90e7bd580 2024-03-21T11:45:00.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.630+0000 7fd8eea00640 1 -- 172.21.15.47:0/1709959887 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd900011470 con 0x55a90e7bd580 2024-03-21T11:45:00.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.630+0000 7fd8eea00640 1 -- 172.21.15.47:0/1709959887 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd8f4019030 con 0x55a90e7bd580 2024-03-21T11:45:00.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.630+0000 7fd912d87740 10 client.32474 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:00.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.630+0000 7fd912d87740 10 client.32474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:00.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.630+0000 7fd912d87740 20 client.32474 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:00.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.630+0000 7fd912d87740 1 --2- 172.21.15.47:0/1709959887 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a90e7e2770 0x55a90e7e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:00.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.630+0000 7fd912d87740 4 client.32474 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:00.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.630+0000 7fd912d87740 1 -- 172.21.15.47:0/1709959887 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a90e72d800 con 0x55a90e7e2770 2024-03-21T11:45:00.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.633+0000 7fd90d600640 1 --2- 172.21.15.47:0/1709959887 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a90e7e2770 0x55a90e7e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:00.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.638+0000 7fd8eea00640 10 client.32474 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:00.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:00.638+0000 7fd90d600640 1 --2- 172.21.15.47:0/1709959887 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a90e7e2770 0x55a90e7e4bf0 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48d1c8606662ad6 server_cookie=22ce1e718929884f in_seq=0 out_seq=0 2024-03-21T11:45:00.975 INFO:tasks.workunit.client.0.smithi047.stdout: 10 201961 22.48 MB/sec execute 344 sec latency 464.142 ms 2024-03-21T11:45:01.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:01 smithi102 ceph-mon[36748]: pgmap v1050: 129 pgs: 129 active+clean; 631 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.2 MiB/s wr, 576 op/s 2024-03-21T11:45:01.592 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:01 smithi047 ceph-mon[31351]: pgmap v1050: 129 pgs: 129 active+clean; 631 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.2 MiB/s wr, 576 op/s 2024-03-21T11:45:01.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:01 smithi100 ceph-mon[36245]: pgmap v1050: 129 pgs: 129 active+clean; 631 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.2 MiB/s wr, 576 op/s 2024-03-21T11:45:01.976 INFO:tasks.workunit.client.0.smithi047.stdout: 10 201966 22.42 MB/sec execute 345 sec latency 1422.590 ms 2024-03-21T11:45:02.975 INFO:tasks.workunit.client.0.smithi047.stdout: 10 201983 22.35 MB/sec execute 346 sec latency 2422.676 ms 2024-03-21T11:45:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.428+0000 7f2706a00640 1 -- 172.21.15.47:0/3745977538 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559894ecd0c0 con 0x559894fd23a0 2024-03-21T11:45:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.428+0000 7f2706a00640 10 client.32449 handle_command_reply: tid=0 2024-03-21T11:45:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.428+0000 7f272e7d2740 1 client.32449 shutdown 2024-03-21T11:45:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.428+0000 7f272e7d2740 1 -- 172.21.15.47:0/3745977538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 msgr2=0x559894f6f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.428+0000 7f272e7d2740 1 --2- 172.21.15.47:0/3745977538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f6f260 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f27180092a0 tx=0x7f2718004760 comp rx=0 tx=0).stop 2024-03-21T11:45:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.428+0000 7f272e7d2740 1 -- 172.21.15.47:0/3745977538 shutdown_connections 2024-03-21T11:45:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.428+0000 7f272e7d2740 1 --2- 172.21.15.47:0/3745977538 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559894fd23a0 0x559894fd47c0 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.428+0000 7f272e7d2740 1 --2- 172.21.15.47:0/3745977538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559894f707b0 0x559894f70c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.428+0000 7f272e7d2740 1 --2- 172.21.15.47:0/3745977538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559894f65d40 0x559894f6f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.428+0000 7f272e7d2740 1 --2- 172.21.15.47:0/3745977538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559894f60bd0 0x559894f6f260 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.428+0000 7f272e7d2740 1 -- 172.21.15.47:0/3745977538 >> 172.21.15.47:0/3745977538 conn(0x559894f5b750 msgr2=0x559894f61ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.429+0000 7f2706a00640 0 client.32449 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.429+0000 7f272e7d2740 1 -- 172.21.15.47:0/3745977538 shutdown_connections 2024-03-21T11:45:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.429+0000 7f272e7d2740 1 -- 172.21.15.47:0/3745977538 wait complete. 2024-03-21T11:45:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.429+0000 7f272e7d2740 20 client.32449 trim_cache size 0 max 16384 2024-03-21T11:45:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:45:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:45:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:45:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:45:03.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.430+0000 7fd8eea00640 1 -- 172.21.15.47:0/1709959887 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a90e72d800 con 0x55a90e7e2770 2024-03-21T11:45:03.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.430+0000 7fd8eea00640 10 client.32474 handle_command_reply: tid=0 2024-03-21T11:45:03.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.430+0000 7fd912d87740 1 client.32474 shutdown 2024-03-21T11:45:03.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.433+0000 7fd912d87740 1 -- 172.21.15.47:0/1709959887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bd580 msgr2=0x55a90e7cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:03.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.433+0000 7fd912d87740 1 --2- 172.21.15.47:0/1709959887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bd580 0x55a90e7cd250 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400be70 tx=0x7fd8f4012820 comp rx=0 tx=0).stop 2024-03-21T11:45:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.433+0000 7f272ba00640 1 -- 172.21.15.47:0/1501576910 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f270c0a3040 msgr2=0x7f270c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.433+0000 7f272ba00640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f270c0a3040 0x7f270c0a5500 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f27240728e0 tx=0x7f27140156d0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.433+0000 7f272ba00640 1 -- 172.21.15.47:0/1501576910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27241054b0 msgr2=0x7f27240711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.433+0000 7f272ba00640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27241054b0 0x7f27240711b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f271c009fd0 tx=0x7f271c00ee50 comp rx=0 tx=0).stop 2024-03-21T11:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.433+0000 7fd912d87740 1 -- 172.21.15.47:0/1709959887 shutdown_connections 2024-03-21T11:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.433+0000 7fd912d87740 1 --2- 172.21.15.47:0/1709959887 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a90e7e2770 0x55a90e7e4bf0 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.433+0000 7fd912d87740 1 --2- 172.21.15.47:0/1709959887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bd580 0x55a90e7cd250 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.433+0000 7fd912d87740 1 --2- 172.21.15.47:0/1709959887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bcbc0 0x55a90e7c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.433+0000 7fd912d87740 1 --2- 172.21.15.47:0/1709959887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bb9f0 0x55a90e7c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.433+0000 7fd912d87740 1 -- 172.21.15.47:0/1709959887 >> 172.21.15.47:0/1709959887 conn(0x55a90e7b6de0 msgr2=0x55a90e7b6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:03.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.433+0000 7fd8eea00640 0 client.32474 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.434+0000 7f272ba00640 1 -- 172.21.15.47:0/1501576910 shutdown_connections 2024-03-21T11:45:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.434+0000 7f272ba00640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f270c0a3040 0x7f270c0a5500 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.434+0000 7f272ba00640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27241054b0 0x7f27240711b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.434+0000 7f272ba00640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2724104af0 0x7f27240730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.434+0000 7f272ba00640 1 --2- 172.21.15.47:0/1501576910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2724103900 0x7f2724072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.434+0000 7f272ba00640 1 -- 172.21.15.47:0/1501576910 >> 172.21.15.47:0/1501576910 conn(0x7f27240fee90 msgr2=0x7f2724107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.434+0000 7fd912d87740 1 -- 172.21.15.47:0/1709959887 shutdown_connections 2024-03-21T11:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.434+0000 7fd912d87740 1 -- 172.21.15.47:0/1709959887 wait complete. 2024-03-21T11:45:03.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.434+0000 7fd912d87740 20 client.32474 trim_cache size 0 max 16384 2024-03-21T11:45:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.434+0000 7f272ba00640 1 -- 172.21.15.47:0/1501576910 shutdown_connections 2024-03-21T11:45:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:03.434+0000 7f272ba00640 1 -- 172.21.15.47:0/1501576910 wait complete. 2024-03-21T11:45:03.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.460+0000 7fd912d87740 1 Processor -- start 2024-03-21T11:45:03.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.460+0000 7fd912d87740 1 -- start start 2024-03-21T11:45:03.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.461+0000 7fd912d87740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bb9f0 0x55a90e7b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:03.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.461+0000 7fd912d87740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bcbc0 0x55a90e7cd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:03.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.461+0000 7fd912d87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bd580 0x55a90e7b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:03.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.461+0000 7fd912d87740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9081a7d30 con 0x55a90e7bcbc0 2024-03-21T11:45:03.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.461+0000 7fd90d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bb9f0 0x55a90e7b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.461+0000 7fd90d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bb9f0 0x55a90e7b58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49220/0 (socket says 172.21.15.47:49220) 2024-03-21T11:45:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.462+0000 7fd912d87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9081a96b0 con 0x55a90e7bd580 2024-03-21T11:45:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.462+0000 7fd912d87740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9081a8390 con 0x55a90e7bb9f0 2024-03-21T11:45:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.462+0000 7fd90d600640 1 -- 172.21.15.47:0/97837360 learned_addr learned my addr 172.21.15.47:0/97837360 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.462+0000 7fd90e000640 1 --2- 172.21.15.47:0/97837360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bd580 0x55a90e7b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.462+0000 7fd90cc00640 1 --2- 172.21.15.47:0/97837360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bcbc0 0x55a90e7cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.462+0000 7fd90d600640 1 -- 172.21.15.47:0/97837360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bd580 msgr2=0x55a90e7b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.462+0000 7fd90d600640 1 --2- 172.21.15.47:0/97837360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bd580 0x55a90e7b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.462+0000 7fd90d600640 1 -- 172.21.15.47:0/97837360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bcbc0 msgr2=0x55a90e7cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.463+0000 7fd90d600640 1 --2- 172.21.15.47:0/97837360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bcbc0 0x55a90e7cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.463+0000 7fd90d600640 1 -- 172.21.15.47:0/97837360 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a90e7b7d20 con 0x55a90e7bb9f0 2024-03-21T11:45:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.464+0000 7fd90d600640 1 --2- 172.21.15.47:0/97837360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bb9f0 0x55a90e7b58d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fd90000c700 tx=0x7fd900009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0a6cdb96b812d1b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.464+0000 7fd8f9c00640 1 -- 172.21.15.47:0/97837360 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9000b9070 con 0x55a90e7bb9f0 2024-03-21T11:45:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.464+0000 7fd8f9c00640 1 -- 172.21.15.47:0/97837360 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd900011470 con 0x55a90e7bb9f0 2024-03-21T11:45:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.464+0000 7fd8f9c00640 1 -- 172.21.15.47:0/97837360 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9000a4640 con 0x55a90e7bb9f0 2024-03-21T11:45:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.465+0000 7fd912d87740 1 -- 172.21.15.47:0/97837360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bb9f0 msgr2=0x55a90e7b58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.465+0000 7fd912d87740 1 --2- 172.21.15.47:0/97837360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bb9f0 0x55a90e7b58d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fd90000c700 tx=0x7fd900009200 comp rx=0 tx=0).stop 2024-03-21T11:45:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.466+0000 7fd912d87740 1 -- 172.21.15.47:0/97837360 shutdown_connections 2024-03-21T11:45:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.466+0000 7fd912d87740 1 --2- 172.21.15.47:0/97837360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bd580 0x55a90e7b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.466+0000 7fd912d87740 1 --2- 172.21.15.47:0/97837360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bcbc0 0x55a90e7cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.466+0000 7fd912d87740 1 --2- 172.21.15.47:0/97837360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bb9f0 0x55a90e7b58d0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.466+0000 7fd912d87740 1 -- 172.21.15.47:0/97837360 >> 172.21.15.47:0/97837360 conn(0x55a90e7b6de0 msgr2=0x55a90e7e4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.466+0000 7fd912d87740 1 -- 172.21.15.47:0/97837360 shutdown_connections 2024-03-21T11:45:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.467+0000 7fd912d87740 1 -- 172.21.15.47:0/97837360 wait complete. 2024-03-21T11:45:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.467+0000 7fd912d87740 1 -- start start 2024-03-21T11:45:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.467+0000 7fd912d87740 1 Processor -- start 2024-03-21T11:45:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.468+0000 7fd912d87740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bb9f0 0x55a90e7c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.468+0000 7fd912d87740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bcbc0 0x55a90e7c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.468+0000 7fd912d87740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bd580 0x55a90e7cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.468+0000 7fd912d87740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9081a9050 con 0x55a90e7bd580 2024-03-21T11:45:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.468+0000 7fd912d87740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9081a89f0 con 0x55a90e7bb9f0 2024-03-21T11:45:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.468+0000 7fd912d87740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90819ab70 con 0x55a90e7bcbc0 2024-03-21T11:45:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.468+0000 7fd912d87740 10 client.0 _ll_register_callbacks cb 0x7fd8f400b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.468+0000 7fd90e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bd580 0x55a90e7cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.468+0000 7fd90d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bb9f0 0x55a90e7c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.468+0000 7fd90cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bcbc0 0x55a90e7c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.468+0000 7fd90e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bd580 0x55a90e7cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39462/0 (socket says 172.21.15.47:39462) 2024-03-21T11:45:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.468+0000 7fd90e000640 1 -- 172.21.15.47:0/2367906248 learned_addr learned my addr 172.21.15.47:0/2367906248 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.469+0000 7fd90e000640 1 -- 172.21.15.47:0/2367906248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bcbc0 msgr2=0x55a90e7c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.469+0000 7fd90e000640 1 --2- 172.21.15.47:0/2367906248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bcbc0 0x55a90e7c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.469+0000 7fd90e000640 1 -- 172.21.15.47:0/2367906248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bb9f0 msgr2=0x55a90e7c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.469+0000 7fd90e000640 1 --2- 172.21.15.47:0/2367906248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bb9f0 0x55a90e7c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.469+0000 7fd90e000640 1 -- 172.21.15.47:0/2367906248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a90e7cc4e0 con 0x55a90e7bd580 2024-03-21T11:45:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.469+0000 7fd90d600640 1 --2- 172.21.15.47:0/2367906248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bb9f0 0x55a90e7c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:45:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.470+0000 7fd90e000640 1 --2- 172.21.15.47:0/2367906248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bd580 0x55a90e7cbef0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd8f401ae40 tx=0x7fd8f4005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5834307434984e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.470+0000 7fd8eea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:45:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.470+0000 7fd8eea00640 1 -- 172.21.15.47:0/2367906248 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4005670 con 0x55a90e7bd580 2024-03-21T11:45:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.470+0000 7fd8eea00640 1 -- 172.21.15.47:0/2367906248 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd8f401a3f0 con 0x55a90e7bd580 2024-03-21T11:45:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.470+0000 7fd912d87740 1 -- 172.21.15.47:0/2367906248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd900011470 con 0x55a90e7bd580 2024-03-21T11:45:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.471+0000 7fd8eea00640 1 -- 172.21.15.47:0/2367906248 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8f401a630 con 0x55a90e7bd580 2024-03-21T11:45:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.471+0000 7fd912d87740 10 client.21756 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.471+0000 7fd912d87740 1 -- 172.21.15.47:0/2367906248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a90e83e1b0 con 0x55a90e7bd580 2024-03-21T11:45:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.471+0000 7fd8eea00640 1 -- 172.21.15.47:0/2367906248 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd900011470 con 0x55a90e7bd580 2024-03-21T11:45:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.471+0000 7fd8eea00640 1 -- 172.21.15.47:0/2367906248 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd8f4019070 con 0x55a90e7bd580 2024-03-21T11:45:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.472+0000 7fd912d87740 10 client.21756 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.472+0000 7fd912d87740 10 client.21756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.472+0000 7fd912d87740 20 client.21756 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.472+0000 7fd912d87740 1 --2- 172.21.15.47:0/2367906248 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a90e83eaa0 0x55a90e840ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.472+0000 7fd912d87740 4 client.21756 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:45:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.472+0000 7fd912d87740 1 -- 172.21.15.47:0/2367906248 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a90e727b50 con 0x55a90e83eaa0 2024-03-21T11:45:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.474+0000 7fd90d600640 1 --2- 172.21.15.47:0/2367906248 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a90e83eaa0 0x55a90e840ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:03.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.478+0000 7fd90d600640 1 --2- 172.21.15.47:0/2367906248 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a90e83eaa0 0x55a90e840ec0 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b90028a48edab801 server_cookie=494bb1e9ebed7d6c in_seq=0 out_seq=0 2024-03-21T11:45:03.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.478+0000 7fd8eea00640 10 client.21756 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:03.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.484+0000 7fd8eea00640 1 -- 172.21.15.47:0/2367906248 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a90e727b50 con 0x55a90e83eaa0 2024-03-21T11:45:03.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.484+0000 7fd8eea00640 10 client.21756 handle_command_reply: tid=0 2024-03-21T11:45:03.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.484+0000 7fd912d87740 1 client.21756 shutdown 2024-03-21T11:45:03.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.485+0000 7fd912d87740 1 -- 172.21.15.47:0/2367906248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bd580 msgr2=0x55a90e7cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:03.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.485+0000 7fd912d87740 1 --2- 172.21.15.47:0/2367906248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bd580 0x55a90e7cbef0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd8f401ae40 tx=0x7fd8f4005270 comp rx=0 tx=0).stop 2024-03-21T11:45:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.485+0000 7fd912d87740 1 -- 172.21.15.47:0/2367906248 shutdown_connections 2024-03-21T11:45:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.485+0000 7fd912d87740 1 --2- 172.21.15.47:0/2367906248 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a90e83eaa0 0x55a90e840ec0 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.485+0000 7fd912d87740 1 --2- 172.21.15.47:0/2367906248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a90e7bd580 0x55a90e7cbef0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.485+0000 7fd8eea00640 0 client.21756 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.485+0000 7fd912d87740 1 --2- 172.21.15.47:0/2367906248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a90e7bcbc0 0x55a90e7c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.485+0000 7fd912d87740 1 --2- 172.21.15.47:0/2367906248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a90e7bb9f0 0x55a90e7c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.485+0000 7fd912d87740 1 -- 172.21.15.47:0/2367906248 >> 172.21.15.47:0/2367906248 conn(0x55a90e7b6de0 msgr2=0x55a90e7ba040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:03.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.486+0000 7fd912d87740 1 -- 172.21.15.47:0/2367906248 shutdown_connections 2024-03-21T11:45:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.486+0000 7fd912d87740 1 -- 172.21.15.47:0/2367906248 wait complete. 2024-03-21T11:45:03.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.486+0000 7fd912d87740 20 client.21756 trim_cache size 0 max 16384 2024-03-21T11:45:03.488 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:45:03.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.488+0000 7fd90ea00640 1 -- 172.21.15.47:0/2699425401 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8f00a2fa0 msgr2=0x7fd8f00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:03.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.488+0000 7fd90ea00640 1 --2- 172.21.15.47:0/2699425401 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8f00a2fa0 0x7fd8f00a5460 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc02c8e0 tx=0x7fd8fc02df90 comp rx=0 tx=0).stop 2024-03-21T11:45:03.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.488+0000 7fd90ea00640 1 -- 172.21.15.47:0/2699425401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9081015a0 msgr2=0x7fd90819dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:03.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.488+0000 7fd90ea00640 1 --2- 172.21.15.47:0/2699425401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9081015a0 0x7fd90819dec0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fd900005fd0 tx=0x7fd90000ef30 comp rx=0 tx=0).stop 2024-03-21T11:45:03.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.488+0000 7fd90ea00640 1 -- 172.21.15.47:0/2699425401 shutdown_connections 2024-03-21T11:45:03.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.488+0000 7fd90ea00640 1 --2- 172.21.15.47:0/2699425401 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd8f00a2fa0 0x7fd8f00a5460 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.488+0000 7fd90ea00640 1 --2- 172.21.15.47:0/2699425401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd908103150 0x7fd9081a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.488+0000 7fd90ea00640 1 --2- 172.21.15.47:0/2699425401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd908102790 0x7fd90819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.488+0000 7fd90ea00640 1 --2- 172.21.15.47:0/2699425401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9081015a0 0x7fd90819dec0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:03.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.488+0000 7fd90ea00640 1 -- 172.21.15.47:0/2699425401 >> 172.21.15.47:0/2699425401 conn(0x7fd9080fcb10 msgr2=0x7fd9081039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:03.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.489+0000 7fd90ea00640 1 -- 172.21.15.47:0/2699425401 shutdown_connections 2024-03-21T11:45:03.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:03.489+0000 7fd90ea00640 1 -- 172.21.15.47:0/2699425401 wait complete. 2024-03-21T11:45:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:03 smithi102 ceph-mon[36748]: pgmap v1051: 129 pgs: 129 active+clean; 634 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.1 MiB/s wr, 446 op/s 2024-03-21T11:45:03.510 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:03 smithi047 ceph-mon[31351]: pgmap v1051: 129 pgs: 129 active+clean; 634 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.1 MiB/s wr, 446 op/s 2024-03-21T11:45:03.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:03 smithi100 ceph-mon[36245]: pgmap v1051: 129 pgs: 129 active+clean; 634 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.1 MiB/s wr, 446 op/s 2024-03-21T11:45:03.975 INFO:tasks.workunit.client.0.smithi047.stdout: 10 202079 22.31 MB/sec execute 347 sec latency 2598.846 ms 2024-03-21T11:45:04.522 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:45:04.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:45:04.523 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:45:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.846+0000 7fcdcda00640 1 -- 172.21.15.47:0/3722925196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdc80fff20 msgr2=0x7fcdc8102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.846+0000 7fcdcda00640 1 --2- 172.21.15.47:0/3722925196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdc80fff20 0x7fcdc8102350 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009900 tx=0x7fcdb802bc30 comp rx=0 tx=0).stop 2024-03-21T11:45:04.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.847+0000 7fcdcda00640 1 -- 172.21.15.47:0/3722925196 shutdown_connections 2024-03-21T11:45:04.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.847+0000 7fcdcda00640 1 --2- 172.21.15.47:0/3722925196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc8105240 0x7fcdc81056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.847+0000 7fcdcda00640 1 --2- 172.21.15.47:0/3722925196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8102890 0x7fcdc8104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.847+0000 7fcdcda00640 1 --2- 172.21.15.47:0/3722925196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdc80fff20 0x7fcdc8102350 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.847+0000 7fcdcda00640 1 -- 172.21.15.47:0/3722925196 >> 172.21.15.47:0/3722925196 conn(0x7fcdc80fbb40 msgr2=0x7fcdc80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:04.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.848+0000 7fcdcda00640 1 -- 172.21.15.47:0/3722925196 shutdown_connections 2024-03-21T11:45:04.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.848+0000 7fcdcda00640 1 -- 172.21.15.47:0/3722925196 wait complete. 2024-03-21T11:45:04.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.850+0000 7fcdcda00640 1 Processor -- start 2024-03-21T11:45:04.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.850+0000 7fcdcda00640 1 -- start start 2024-03-21T11:45:04.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.851+0000 7fcdcda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc80fff20 0x7fcdc819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.851+0000 7fcdcda00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdc8102890 0x7fcdc819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.851+0000 7fcdcda00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc8105240 0x7fcdc81a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.851+0000 7fcdcda00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc81139d0 con 0x7fcdc8105240 2024-03-21T11:45:04.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.851+0000 7fcdcda00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc8113850 con 0x7fcdc80fff20 2024-03-21T11:45:04.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.851+0000 7fcdcda00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc8113b50 con 0x7fcdc8102890 2024-03-21T11:45:04.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.851+0000 7fcdc6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc80fff20 0x7fcdc819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.851+0000 7fcdc6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc80fff20 0x7fcdc819e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44282/0 (socket says 172.21.15.47:44282) 2024-03-21T11:45:04.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.851+0000 7fcdc6a00640 1 -- 172.21.15.47:0/401031087 learned_addr learned my addr 172.21.15.47:0/401031087 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:04.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.852+0000 7fcdc6000640 1 --2- 172.21.15.47:0/401031087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdc8102890 0x7fcdc819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.852+0000 7fcdc6a00640 1 -- 172.21.15.47:0/401031087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdc8102890 msgr2=0x7fcdc819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:04.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.852+0000 7fcdc6a00640 1 --2- 172.21.15.47:0/401031087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdc8102890 0x7fcdc819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.852+0000 7fcdc6a00640 1 -- 172.21.15.47:0/401031087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc8105240 msgr2=0x7fcdc81a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:04.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.852+0000 7fcdc6a00640 1 --2- 172.21.15.47:0/401031087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc8105240 0x7fcdc81a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.852+0000 7fcdc6a00640 1 -- 172.21.15.47:0/401031087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc81a5c70 con 0x7fcdc80fff20 2024-03-21T11:45:04.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.852+0000 7fcdc6a00640 1 --2- 172.21.15.47:0/401031087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc80fff20 0x7fcdc819e050 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009230 tx=0x7fcdb802c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:04.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.854+0000 7fcdbf400640 1 -- 172.21.15.47:0/401031087 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8043020 con 0x7fcdc80fff20 2024-03-21T11:45:04.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.854+0000 7fcdbf400640 1 -- 172.21.15.47:0/401031087 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdb802c980 con 0x7fcdc80fff20 2024-03-21T11:45:04.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.854+0000 7fcdbf400640 1 -- 172.21.15.47:0/401031087 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb803e6b0 con 0x7fcdc80fff20 2024-03-21T11:45:04.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.854+0000 7fcdcda00640 1 -- 172.21.15.47:0/401031087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc81a5f00 con 0x7fcdc80fff20 2024-03-21T11:45:04.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.855+0000 7fcdcda00640 1 -- 172.21.15.47:0/401031087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc81a6370 con 0x7fcdc80fff20 2024-03-21T11:45:04.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.860+0000 7fcdd07dc740 1 Processor -- start 2024-03-21T11:45:04.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.861+0000 7fcdd07dc740 1 -- start start 2024-03-21T11:45:04.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.861+0000 7fcdd07dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f7e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.861+0000 7fcdd07dc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f83590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.861+0000 7fcdd07dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f83c00 0x55e1e4f86090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.862+0000 7fcdd07dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e4ee3eb0 con 0x55e1e4f83c00 2024-03-21T11:45:04.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.862+0000 7fcdd07dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e4e8ca60 con 0x55e1e4f7e0a0 2024-03-21T11:45:04.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.862+0000 7fcdd07dc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e4edff50 con 0x55e1e4f7ea90 2024-03-21T11:45:04.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.862+0000 7fcdc7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f7e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.862+0000 7fcdc6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f83c00 0x55e1e4f86090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.862+0000 7fcdc6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f83c00 0x55e1e4f86090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39492/0 (socket says 172.21.15.47:39492) 2024-03-21T11:45:04.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.862+0000 7fcdc6000640 1 -- 172.21.15.47:0/89170354 learned_addr learned my addr 172.21.15.47:0/89170354 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:04.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.863+0000 7fcdc6000640 1 -- 172.21.15.47:0/89170354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 msgr2=0x55e1e4f83590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:45:04.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.863+0000 7fcdc6000640 1 --2- 172.21.15.47:0/89170354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f83590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.863+0000 7fcdc6000640 1 -- 172.21.15.47:0/89170354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f7e0a0 msgr2=0x55e1e4f7e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:04.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.863+0000 7fcdbf400640 1 -- 172.21.15.47:0/401031087 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcdb8046020 con 0x7fcdc80fff20 2024-03-21T11:45:04.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.863+0000 7fcdc6000640 1 --2- 172.21.15.47:0/89170354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f7e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.863+0000 7fcdc6000640 1 -- 172.21.15.47:0/89170354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1e4cc07e0 con 0x55e1e4f83c00 2024-03-21T11:45:04.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.863+0000 7fcdc6000640 1 --2- 172.21.15.47:0/89170354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f83c00 0x55e1e4f86090 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400bce0 tx=0x7fcdb400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2172eb097cec3e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:04.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.864+0000 7fcdb3e00640 1 -- 172.21.15.47:0/89170354 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4015070 con 0x55e1e4f83c00 2024-03-21T11:45:04.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.864+0000 7fcdb3e00640 1 -- 172.21.15.47:0/89170354 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdb401a940 con 0x55e1e4f83c00 2024-03-21T11:45:04.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.864+0000 7fcdb3e00640 1 -- 172.21.15.47:0/89170354 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4011aa0 con 0x55e1e4f83c00 2024-03-21T11:45:04.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.866+0000 7fcdbf400640 1 --2- 172.21.15.47:0/401031087 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcdac0a30f0 0x7fcdac0a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.866+0000 7fcdbf400640 1 -- 172.21.15.47:0/401031087 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcdb80f1d80 con 0x7fcdc80fff20 2024-03-21T11:45:04.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.866+0000 7fcdd07dc740 1 -- 172.21.15.47:0/89170354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f83c00 msgr2=0x55e1e4f86090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:04.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.866+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/89170354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f83c00 0x55e1e4f86090 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400bce0 tx=0x7fcdb400f730 comp rx=0 tx=0).stop 2024-03-21T11:45:04.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.867+0000 7fcdc7400640 1 --2- 172.21.15.47:0/401031087 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcdac0a30f0 0x7fcdac0a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.869+0000 7fcdd07dc740 1 -- 172.21.15.47:0/89170354 shutdown_connections 2024-03-21T11:45:04.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.869+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/89170354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f83c00 0x55e1e4f86090 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.869+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/89170354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f83590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.869+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/89170354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f7e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.869+0000 7fcdd07dc740 1 -- 172.21.15.47:0/89170354 >> 172.21.15.47:0/89170354 conn(0x55e1e4f79610 msgr2=0x55e1e4f79a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:04.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.870+0000 7fcdc7400640 1 --2- 172.21.15.47:0/401031087 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcdac0a30f0 0x7fcdac0a55b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x55e1e4f7f2f0 tx=0x7fcdc0009430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:04.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.870+0000 7fcdd07dc740 1 -- 172.21.15.47:0/89170354 shutdown_connections 2024-03-21T11:45:04.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.871+0000 7fcdd07dc740 1 -- 172.21.15.47:0/89170354 wait complete. 2024-03-21T11:45:04.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.872+0000 7fcdd07dc740 1 -- start start 2024-03-21T11:45:04.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.872+0000 7fcdd07dc740 1 Processor -- start 2024-03-21T11:45:04.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.873+0000 7fcdd07dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f88320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.873+0000 7fcdd07dc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f88860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.873+0000 7fcdd07dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 0x55e1e4f8f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.873+0000 7fcdd07dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e4f86bb0 con 0x55e1e4f7e0a0 2024-03-21T11:45:04.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.873+0000 7fcdd07dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e4e9feb0 con 0x55e1e4f83c00 2024-03-21T11:45:04.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.873+0000 7fcdd07dc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1e4f86d30 con 0x55e1e4f7ea90 2024-03-21T11:45:04.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.873+0000 7fcdc6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f88860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.873+0000 7fcdd07dc740 10 client.0 _ll_register_callbacks cb 0x55e1e4e94150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:04.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.873+0000 7fcdc6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f88860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49254/0 (socket says 172.21.15.47:49254) 2024-03-21T11:45:04.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.873+0000 7fcdc6000640 1 -- 172.21.15.47:0/566865397 learned_addr learned my addr 172.21.15.47:0/566865397 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:04.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.873+0000 7fcdc7400640 1 --2- 172.21.15.47:0/566865397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 0x55e1e4f8f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.874+0000 7fcdc6a00640 1 --2- 172.21.15.47:0/566865397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f88320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.874+0000 7fcdc6000640 1 -- 172.21.15.47:0/566865397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 msgr2=0x55e1e4f8f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:04.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.874+0000 7fcdc6000640 1 --2- 172.21.15.47:0/566865397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 0x55e1e4f8f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.874+0000 7fcdc6000640 1 -- 172.21.15.47:0/566865397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7e0a0 msgr2=0x55e1e4f88320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:04.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.874+0000 7fcdc6000640 1 --2- 172.21.15.47:0/566865397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f88320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.874+0000 7fcdc6000640 1 -- 172.21.15.47:0/566865397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1e4d20f70 con 0x55e1e4f7ea90 2024-03-21T11:45:04.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.875+0000 7fcdc6000640 1 --2- 172.21.15.47:0/566865397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f88860 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400b440 tx=0x7fcdb400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e00d542595ed3b4c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:04.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.875+0000 7fcdb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:04.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.876+0000 7fcdb0c00640 1 -- 172.21.15.47:0/566865397 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4028060 con 0x55e1e4f7ea90 2024-03-21T11:45:04.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.876+0000 7fcdb0c00640 1 -- 172.21.15.47:0/566865397 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcdb401a460 con 0x55e1e4f7ea90 2024-03-21T11:45:04.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.876+0000 7fcdd07dc740 1 -- 172.21.15.47:0/566865397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdb401a940 con 0x55e1e4f7ea90 2024-03-21T11:45:04.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.876+0000 7fcdb0c00640 1 -- 172.21.15.47:0/566865397 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdb402f050 con 0x55e1e4f7ea90 2024-03-21T11:45:04.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.877+0000 7fcdd07dc740 10 client.32473 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:04.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.877+0000 7fcdd07dc740 1 -- 172.21.15.47:0/566865397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e1e4bb8170 con 0x55e1e4f7ea90 2024-03-21T11:45:04.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.878+0000 7fcdb0c00640 1 -- 172.21.15.47:0/566865397 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdb401a940 con 0x55e1e4f7ea90 2024-03-21T11:45:04.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.878+0000 7fcdb0c00640 1 -- 172.21.15.47:0/566865397 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcdb4011b40 con 0x55e1e4f7ea90 2024-03-21T11:45:04.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.878+0000 7fcdd07dc740 10 client.32473 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:04.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.878+0000 7fcdd07dc740 10 client.32473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:04.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.878+0000 7fcdd07dc740 20 client.32473 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:04.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.878+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/566865397 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e1e4f93d90 0x55e1e4f96180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.878+0000 7fcdd07dc740 4 client.32473 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:04.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.878+0000 7fcdd07dc740 1 -- 172.21.15.47:0/566865397 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1e4ef13f0 con 0x55e1e4f93d90 2024-03-21T11:45:04.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.879+0000 7fcdc6a00640 1 --2- 172.21.15.47:0/566865397 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e1e4f93d90 0x55e1e4f96180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.880+0000 7fcdc6a00640 1 --2- 172.21.15.47:0/566865397 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e1e4f93d90 0x55e1e4f96180 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=678087a0b090084e server_cookie=e3ca53a8e14f23ff in_seq=0 out_seq=0 2024-03-21T11:45:04.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.880+0000 7fcdb0c00640 10 client.32473 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:04.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.884+0000 7fcdb0c00640 1 -- 172.21.15.47:0/566865397 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e1e4ef13f0 con 0x55e1e4f93d90 2024-03-21T11:45:04.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.884+0000 7fcdb0c00640 10 client.32473 handle_command_reply: tid=0 2024-03-21T11:45:04.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.884+0000 7fcdd07dc740 1 client.32473 shutdown 2024-03-21T11:45:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.885+0000 7fcdd07dc740 1 -- 172.21.15.47:0/566865397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 msgr2=0x55e1e4f88860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.885+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/566865397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f88860 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400b440 tx=0x7fcdb400bf10 comp rx=0 tx=0).stop 2024-03-21T11:45:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.885+0000 7fcdd07dc740 1 -- 172.21.15.47:0/566865397 shutdown_connections 2024-03-21T11:45:04.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.885+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/566865397 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e1e4f93d90 0x55e1e4f96180 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.885+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/566865397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 0x55e1e4f8f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.885+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/566865397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f88860 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.885+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/566865397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f88320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.885+0000 7fcdd07dc740 1 -- 172.21.15.47:0/566865397 >> 172.21.15.47:0/566865397 conn(0x55e1e4f79610 msgr2=0x55e1e4f78830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:04.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.885+0000 7fcdb0c00640 0 client.32473 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:04.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.886+0000 7fcdd07dc740 1 -- 172.21.15.47:0/566865397 shutdown_connections 2024-03-21T11:45:04.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.886+0000 7fcdd07dc740 1 -- 172.21.15.47:0/566865397 wait complete. 2024-03-21T11:45:04.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.886+0000 7fcdd07dc740 20 client.32473 trim_cache size 0 max 16384 2024-03-21T11:45:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.918+0000 7fcdd07dc740 1 Processor -- start 2024-03-21T11:45:04.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.918+0000 7fcdd07dc740 1 -- start start 2024-03-21T11:45:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.918+0000 7fcdd07dc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.918+0000 7fcdd07dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f78bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.918+0000 7fcdd07dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 0x55e1e4f8f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.918+0000 7fcdd07dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc81b0740 con 0x55e1e4f7ea90 2024-03-21T11:45:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.918+0000 7fcdd07dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc81b20c0 con 0x55e1e4f83c00 2024-03-21T11:45:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.918+0000 7fcdd07dc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc81b0da0 con 0x55e1e4f7e0a0 2024-03-21T11:45:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.919+0000 7fcdc6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.919+0000 7fcdc6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f786b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49256/0 (socket says 172.21.15.47:49256) 2024-03-21T11:45:04.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.919+0000 7fcdc6a00640 1 -- 172.21.15.47:0/398991943 learned_addr learned my addr 172.21.15.47:0/398991943 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.919+0000 7fcdc7400640 1 --2- 172.21.15.47:0/398991943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 0x55e1e4f8f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.919+0000 7fcdc7400640 1 -- 172.21.15.47:0/398991943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7e0a0 msgr2=0x55e1e4f786b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.919+0000 7fcdc7400640 1 --2- 172.21.15.47:0/398991943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f786b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.919+0000 7fcdc7400640 1 -- 172.21.15.47:0/398991943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7ea90 msgr2=0x55e1e4f78bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.919+0000 7fcdc7400640 1 --2- 172.21.15.47:0/398991943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f78bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.920+0000 7fcdc7400640 1 -- 172.21.15.47:0/398991943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1e4f96710 con 0x55e1e4f83c00 2024-03-21T11:45:04.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.920+0000 7fcdc7400640 1 --2- 172.21.15.47:0/398991943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 0x55e1e4f8f860 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000cc80 tx=0x7fcdc0015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6a2a2a3643f243b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.921+0000 7fcdb3400640 1 -- 172.21.15.47:0/398991943 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0023410 con 0x55e1e4f83c00 2024-03-21T11:45:04.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.921+0000 7fcdb3400640 1 -- 172.21.15.47:0/398991943 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdc0028070 con 0x55e1e4f83c00 2024-03-21T11:45:04.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.923+0000 7fcdd07dc740 1 -- 172.21.15.47:0/398991943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 msgr2=0x55e1e4f8f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:04.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.923+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/398991943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 0x55e1e4f8f860 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000cc80 tx=0x7fcdc0015000 comp rx=0 tx=0).stop 2024-03-21T11:45:04.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.922+0000 7fcdb3400640 1 -- 172.21.15.47:0/398991943 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0021980 con 0x55e1e4f83c00 2024-03-21T11:45:04.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.924+0000 7fcdd07dc740 1 -- 172.21.15.47:0/398991943 shutdown_connections 2024-03-21T11:45:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.924+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/398991943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 0x55e1e4f8f860 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.924+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/398991943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f78bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.924+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/398991943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.924+0000 7fcdd07dc740 1 -- 172.21.15.47:0/398991943 >> 172.21.15.47:0/398991943 conn(0x55e1e4f79610 msgr2=0x55e1e4f77fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.924+0000 7fcdd07dc740 1 -- 172.21.15.47:0/398991943 shutdown_connections 2024-03-21T11:45:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.925+0000 7fcdd07dc740 1 -- 172.21.15.47:0/398991943 wait complete. 2024-03-21T11:45:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.925+0000 7fcdd07dc740 1 -- start start 2024-03-21T11:45:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.926+0000 7fcdd07dc740 1 Processor -- start 2024-03-21T11:45:04.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.926+0000 7fcdd07dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f8f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.926+0000 7fcdd07dc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f8df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.926+0000 7fcdd07dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 0x55e1e4f8e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.926+0000 7fcdd07dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc81b1a60 con 0x55e1e4f7e0a0 2024-03-21T11:45:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.926+0000 7fcdd07dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc81b1400 con 0x55e1e4f83c00 2024-03-21T11:45:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.926+0000 7fcdd07dc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc81ae900 con 0x55e1e4f7ea90 2024-03-21T11:45:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.926+0000 7fcdd07dc740 10 client.0 _ll_register_callbacks cb 0x7fcdb400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.927+0000 7fcdc6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f8df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.927+0000 7fcdc6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f8df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49266/0 (socket says 172.21.15.47:49266) 2024-03-21T11:45:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.927+0000 7fcdc6000640 1 -- 172.21.15.47:0/4006583267 learned_addr learned my addr 172.21.15.47:0/4006583267 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.927+0000 7fcdc7400640 1 --2- 172.21.15.47:0/4006583267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 0x55e1e4f8e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.928+0000 7fcdc6a00640 1 --2- 172.21.15.47:0/4006583267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f8f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.928+0000 7fcdc6000640 1 -- 172.21.15.47:0/4006583267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 msgr2=0x55e1e4f8e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.928+0000 7fcdc6000640 1 --2- 172.21.15.47:0/4006583267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 0x55e1e4f8e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.928+0000 7fcdc6000640 1 -- 172.21.15.47:0/4006583267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7e0a0 msgr2=0x55e1e4f8f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.928+0000 7fcdc6000640 1 --2- 172.21.15.47:0/4006583267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f8f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.928+0000 7fcdc6000640 1 -- 172.21.15.47:0/4006583267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1e4fef500 con 0x55e1e4f7ea90 2024-03-21T11:45:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.928+0000 7fcdc6a00640 1 --2- 172.21.15.47:0/4006583267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f8f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:45:04.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.935+0000 7fcdc6000640 1 --2- 172.21.15.47:0/4006583267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f8df10 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4011f40 tx=0x7fcdb40114f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=422749943a034b3c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:04.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.935+0000 7fcdb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:04.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.936+0000 7fcdb0c00640 1 -- 172.21.15.47:0/4006583267 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4028070 con 0x55e1e4f7ea90 2024-03-21T11:45:04.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.936+0000 7fcdb0c00640 1 -- 172.21.15.47:0/4006583267 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcdb401ae00 con 0x55e1e4f7ea90 2024-03-21T11:45:04.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.936+0000 7fcdd07dc740 1 -- 172.21.15.47:0/4006583267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdc0028070 con 0x55e1e4f7ea90 2024-03-21T11:45:04.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.937+0000 7fcdb0c00640 1 -- 172.21.15.47:0/4006583267 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdb4015070 con 0x55e1e4f7ea90 2024-03-21T11:45:04.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.937+0000 7fcdd07dc740 10 client.32479 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:04.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.937+0000 7fcdd07dc740 1 -- 172.21.15.47:0/4006583267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e1e4ff0660 con 0x55e1e4f7ea90 2024-03-21T11:45:04.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.937+0000 7fcdb0c00640 1 -- 172.21.15.47:0/4006583267 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdc0028070 con 0x55e1e4f7ea90 2024-03-21T11:45:04.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.938+0000 7fcdb0c00640 1 -- 172.21.15.47:0/4006583267 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcdb4011ad0 con 0x55e1e4f7ea90 2024-03-21T11:45:04.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.938+0000 7fcdd07dc740 10 client.32479 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.938+0000 7fcdd07dc740 10 client.32479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.938+0000 7fcdd07dc740 20 client.32479 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.938+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/4006583267 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e1e4ff0f30 0x55e1e4ff3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.938+0000 7fcdd07dc740 4 client.32479 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:45:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.938+0000 7fcdd07dc740 1 -- 172.21.15.47:0/4006583267 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e1e4eeb740 con 0x55e1e4ff0f30 2024-03-21T11:45:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.939+0000 7fcdc6a00640 1 --2- 172.21.15.47:0/4006583267 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e1e4ff0f30 0x55e1e4ff3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.941+0000 7fcdc6a00640 1 --2- 172.21.15.47:0/4006583267 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e1e4ff0f30 0x55e1e4ff3350 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=face68641f196df4 server_cookie=8439d6d3d1a8a6cf in_seq=0 out_seq=0 2024-03-21T11:45:04.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.941+0000 7fcdb0c00640 10 client.32479 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:04.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.944+0000 7fcdb0c00640 1 -- 172.21.15.47:0/4006583267 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e1e4eeb740 con 0x55e1e4ff0f30 2024-03-21T11:45:04.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.944+0000 7fcdb0c00640 10 client.32479 handle_command_reply: tid=0 2024-03-21T11:45:04.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.944+0000 7fcdd07dc740 1 client.32479 shutdown 2024-03-21T11:45:04.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.946+0000 7fcdd07dc740 1 -- 172.21.15.47:0/4006583267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 msgr2=0x55e1e4f8df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:04.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.946+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/4006583267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f8df10 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4011f40 tx=0x7fcdb40114f0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.946+0000 7fcdd07dc740 1 -- 172.21.15.47:0/4006583267 shutdown_connections 2024-03-21T11:45:04.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.946+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/4006583267 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e1e4ff0f30 0x55e1e4ff3350 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.946+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/4006583267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e1e4f83c00 0x55e1e4f8e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.946+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/4006583267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e1e4f7ea90 0x55e1e4f8df10 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.946+0000 7fcdb0c00640 0 client.32479 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:04.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.946+0000 7fcdd07dc740 1 --2- 172.21.15.47:0/4006583267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e1e4f7e0a0 0x55e1e4f8f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.946+0000 7fcdd07dc740 1 -- 172.21.15.47:0/4006583267 >> 172.21.15.47:0/4006583267 conn(0x55e1e4f79610 msgr2=0x55e1e4f78830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:04.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.949+0000 7fcdd07dc740 1 -- 172.21.15.47:0/4006583267 shutdown_connections 2024-03-21T11:45:04.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.950+0000 7fcdd07dc740 1 -- 172.21.15.47:0/4006583267 wait complete. 2024-03-21T11:45:04.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.950+0000 7fcdd07dc740 20 client.32479 trim_cache size 0 max 16384 2024-03-21T11:45:04.951 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:45:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:45:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "6e2a0303-f2f1-4487-86a6-0fa8887fbaf6", 2024-03-21T11:45:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:45:04.951 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:45:04.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.951+0000 7fcdcda00640 1 -- 172.21.15.47:0/401031087 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcdac0a30f0 msgr2=0x7fcdac0a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:04.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.951+0000 7fcdcda00640 1 --2- 172.21.15.47:0/401031087 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcdac0a30f0 0x7fcdac0a55b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x55e1e4f7f2f0 tx=0x7fcdc0009430 comp rx=0 tx=0).stop 2024-03-21T11:45:04.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.951+0000 7fcdcda00640 1 -- 172.21.15.47:0/401031087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc80fff20 msgr2=0x7fcdc819e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:04.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.951+0000 7fcdcda00640 1 --2- 172.21.15.47:0/401031087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc80fff20 0x7fcdc819e050 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009230 tx=0x7fcdb802c660 comp rx=0 tx=0).stop 2024-03-21T11:45:04.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.951+0000 7fcdcda00640 1 -- 172.21.15.47:0/401031087 shutdown_connections 2024-03-21T11:45:04.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.951+0000 7fcdcda00640 1 --2- 172.21.15.47:0/401031087 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcdac0a30f0 0x7fcdac0a55b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.951+0000 7fcdcda00640 1 --2- 172.21.15.47:0/401031087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc8105240 0x7fcdc81a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.951+0000 7fcdcda00640 1 --2- 172.21.15.47:0/401031087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdc8102890 0x7fcdc819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.951+0000 7fcdcda00640 1 --2- 172.21.15.47:0/401031087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc80fff20 0x7fcdc819e050 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:04.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.951+0000 7fcdcda00640 1 -- 172.21.15.47:0/401031087 >> 172.21.15.47:0/401031087 conn(0x7fcdc80fbb40 msgr2=0x7fcdc8103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:04.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.951+0000 7fcdcda00640 1 -- 172.21.15.47:0/401031087 shutdown_connections 2024-03-21T11:45:04.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:04.951+0000 7fcdcda00640 1 -- 172.21.15.47:0/401031087 wait complete. 2024-03-21T11:45:04.976 INFO:tasks.workunit.client.0.smithi047.stdout: 10 202329 22.28 MB/sec execute 348 sec latency 340.268 ms 2024-03-21T11:45:04.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:45:04.978 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:45:05.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.286+0000 7fc1d1c00640 1 -- 172.21.15.47:0/3007715121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c4007590 msgr2=0x7fc1c40079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:05.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.286+0000 7fc1d1c00640 1 --2- 172.21.15.47:0/3007715121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c4007590 0x7fc1c40079f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009900 tx=0x7fc1c002bd10 comp rx=0 tx=0).stop 2024-03-21T11:45:05.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.287+0000 7fc1d1c00640 1 -- 172.21.15.47:0/3007715121 shutdown_connections 2024-03-21T11:45:05.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.287+0000 7fc1d1c00640 1 --2- 172.21.15.47:0/3007715121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c409dc10 0x7fc1c40a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.287+0000 7fc1d1c00640 1 --2- 172.21.15.47:0/3007715121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c4007590 0x7fc1c40079f0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.287+0000 7fc1d1c00640 1 --2- 172.21.15.47:0/3007715121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1c4008b70 0x7fc1c4006fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.287+0000 7fc1d1c00640 1 -- 172.21.15.47:0/3007715121 >> 172.21.15.47:0/3007715121 conn(0x7fc1c40912d0 msgr2=0x7fc1c4093730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:05.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.287+0000 7fc1d1c00640 1 -- 172.21.15.47:0/3007715121 shutdown_connections 2024-03-21T11:45:05.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.288+0000 7fc1d1c00640 1 -- 172.21.15.47:0/3007715121 wait complete. 2024-03-21T11:45:05.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.289+0000 7fc1d1c00640 1 Processor -- start 2024-03-21T11:45:05.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.290+0000 7fc1d1c00640 1 -- start start 2024-03-21T11:45:05.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.290+0000 7fc1d1c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c4007590 0x7fc1c40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.290+0000 7fc1d1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c4008b70 0x7fc1c40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.290+0000 7fc1d1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1c409dc10 0x7fc1c4142490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.290+0000 7fc1d1c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40974b0 con 0x7fc1c409dc10 2024-03-21T11:45:05.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.290+0000 7fc1d1c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c4097330 con 0x7fc1c4008b70 2024-03-21T11:45:05.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.290+0000 7fc1d1c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c4097630 con 0x7fc1c4007590 2024-03-21T11:45:05.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.291+0000 7fc1cb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c4008b70 0x7fc1c40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.291+0000 7fc1cb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c4008b70 0x7fc1c40a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44358/0 (socket says 172.21.15.47:44358) 2024-03-21T11:45:05.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.291+0000 7fc1cb400640 1 -- 172.21.15.47:0/14633999 learned_addr learned my addr 172.21.15.47:0/14633999 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:05.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.291+0000 7fc1cbe00640 1 --2- 172.21.15.47:0/14633999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c4007590 0x7fc1c40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.292+0000 7fc1d1200640 1 --2- 172.21.15.47:0/14633999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1c409dc10 0x7fc1c4142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.293+0000 7fc1cb400640 1 -- 172.21.15.47:0/14633999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c4007590 msgr2=0x7fc1c40a51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.293+0000 7fc1cb400640 1 --2- 172.21.15.47:0/14633999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c4007590 0x7fc1c40a51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.293+0000 7fc1cb400640 1 -- 172.21.15.47:0/14633999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1c409dc10 msgr2=0x7fc1c4142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.293+0000 7fc1cb400640 1 --2- 172.21.15.47:0/14633999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1c409dc10 0x7fc1c4142490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.293+0000 7fc1cb400640 1 -- 172.21.15.47:0/14633999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c4142b00 con 0x7fc1c4008b70 2024-03-21T11:45:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.294+0000 7fc1cb400640 1 --2- 172.21.15.47:0/14633999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c4008b70 0x7fc1c40a5730 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009a20 tx=0x7fc1c00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.294+0000 7fc1c8c00640 1 -- 172.21.15.47:0/14633999 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0043020 con 0x7fc1c4008b70 2024-03-21T11:45:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.294+0000 7fc1d1c00640 1 -- 172.21.15.47:0/14633999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c4142d90 con 0x7fc1c4008b70 2024-03-21T11:45:05.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.294+0000 7fc1d1c00640 1 -- 172.21.15.47:0/14633999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c41432b0 con 0x7fc1c4008b70 2024-03-21T11:45:05.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.295+0000 7fc1c8c00640 1 -- 172.21.15.47:0/14633999 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1c00044c0 con 0x7fc1c4008b70 2024-03-21T11:45:05.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.295+0000 7fc1c8c00640 1 -- 172.21.15.47:0/14633999 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0003970 con 0x7fc1c4008b70 2024-03-21T11:45:05.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.300+0000 7fc1d601e740 1 Processor -- start 2024-03-21T11:45:05.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.300+0000 7fc1c8c00640 1 -- 172.21.15.47:0/14633999 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc1c0046020 con 0x7fc1c4008b70 2024-03-21T11:45:05.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.301+0000 7fc1d601e740 1 -- start start 2024-03-21T11:45:05.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.302+0000 7fc1d601e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67060 0x560ba3a67480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.302+0000 7fc1d601e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a67a50 0x560ba3a6c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.302+0000 7fc1d601e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a6cbc0 0x560ba3a6f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.302+0000 7fc1d601e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba39bb250 con 0x560ba3a67a50 2024-03-21T11:45:05.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.302+0000 7fc1d601e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba3963a30 con 0x560ba3a6cbc0 2024-03-21T11:45:05.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.302+0000 7fc1d601e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba39b72f0 con 0x560ba3a67060 2024-03-21T11:45:05.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.303+0000 7fc1d1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67060 0x560ba3a67480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.303+0000 7fc1d1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67060 0x560ba3a67480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49300/0 (socket says 172.21.15.47:49300) 2024-03-21T11:45:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.303+0000 7fc1d1200640 1 -- 172.21.15.47:0/2158434805 learned_addr learned my addr 172.21.15.47:0/2158434805 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.303+0000 7fc1cb400640 1 --2- 172.21.15.47:0/2158434805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a6cbc0 0x560ba3a6f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.303+0000 7fc1d1200640 1 -- 172.21.15.47:0/2158434805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a6cbc0 msgr2=0x560ba3a6f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.303+0000 7fc1d1200640 1 --2- 172.21.15.47:0/2158434805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a6cbc0 0x560ba3a6f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.303+0000 7fc1d1200640 1 -- 172.21.15.47:0/2158434805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a67a50 msgr2=0x560ba3a6c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.303+0000 7fc1d1200640 1 --2- 172.21.15.47:0/2158434805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a67a50 0x560ba3a6c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.303+0000 7fc1d1200640 1 -- 172.21.15.47:0/2158434805 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ba394e690 con 0x560ba3a67060 2024-03-21T11:45:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.304+0000 7fc1d1200640 1 --2- 172.21.15.47:0/2158434805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67060 0x560ba3a67480 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc067370 tx=0x7fc1cc076670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=133a85a2014cdc34 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.305+0000 7fc1b9600640 1 -- 172.21.15.47:0/2158434805 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc06b0d0 con 0x560ba3a67060 2024-03-21T11:45:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.305+0000 7fc1b9600640 1 -- 172.21.15.47:0/2158434805 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1cc07a070 con 0x560ba3a67060 2024-03-21T11:45:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.305+0000 7fc1b9600640 1 -- 172.21.15.47:0/2158434805 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc073a40 con 0x560ba3a67060 2024-03-21T11:45:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.305+0000 7fc1c8c00640 1 --2- 172.21.15.47:0/14633999 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc1b40a2fa0 0x7fc1b40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.305+0000 7fc1c8c00640 1 -- 172.21.15.47:0/14633999 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc1c00f1b40 con 0x7fc1c4008b70 2024-03-21T11:45:05.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.307+0000 7fc1d601e740 1 -- 172.21.15.47:0/2158434805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67060 msgr2=0x560ba3a67480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:05.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.307+0000 7fc1d601e740 1 --2- 172.21.15.47:0/2158434805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67060 0x560ba3a67480 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc067370 tx=0x7fc1cc076670 comp rx=0 tx=0).stop 2024-03-21T11:45:05.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.307+0000 7fc1cbe00640 1 --2- 172.21.15.47:0/14633999 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc1b40a2fa0 0x7fc1b40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.308+0000 7fc1d601e740 1 -- 172.21.15.47:0/2158434805 shutdown_connections 2024-03-21T11:45:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.308+0000 7fc1d601e740 1 --2- 172.21.15.47:0/2158434805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a6cbc0 0x560ba3a6f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.308+0000 7fc1d601e740 1 --2- 172.21.15.47:0/2158434805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a67a50 0x560ba3a6c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.308+0000 7fc1d601e740 1 --2- 172.21.15.47:0/2158434805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67060 0x560ba3a67480 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.308+0000 7fc1d601e740 1 -- 172.21.15.47:0/2158434805 >> 172.21.15.47:0/2158434805 conn(0x560ba3a1e200 msgr2=0x560ba3a519c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.309+0000 7fc1cbe00640 1 --2- 172.21.15.47:0/14633999 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc1b40a2fa0 0x7fc1b40a5460 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x560ba3a67e50 tx=0x7fc1bc00b850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.309+0000 7fc1d601e740 1 -- 172.21.15.47:0/2158434805 shutdown_connections 2024-03-21T11:45:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.309+0000 7fc1d601e740 1 -- 172.21.15.47:0/2158434805 wait complete. 2024-03-21T11:45:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.309+0000 7fc1d601e740 1 -- start start 2024-03-21T11:45:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.310+0000 7fc1d601e740 1 Processor -- start 2024-03-21T11:45:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.311+0000 7fc1d601e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67060 0x560ba3a710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.311+0000 7fc1d601e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a67a50 0x560ba3a715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.312+0000 7fc1cbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67060 0x560ba3a710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.312+0000 7fc1cbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67060 0x560ba3a710a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49310/0 (socket says 172.21.15.47:49310) 2024-03-21T11:45:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.312+0000 7fc1cbe00640 1 -- 172.21.15.47:0/4687428 learned_addr learned my addr 172.21.15.47:0/4687428 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.312+0000 7fc1d601e740 1 --2- 172.21.15.47:0/4687428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a6cbc0 0x560ba3a78620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.312+0000 7fc1d601e740 1 -- 172.21.15.47:0/4687428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba3a6fb70 con 0x560ba3a6cbc0 2024-03-21T11:45:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.313+0000 7fc1cb400640 1 --2- 172.21.15.47:0/4687428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a67a50 0x560ba3a715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.313+0000 7fc1d601e740 1 -- 172.21.15.47:0/4687428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba3976e80 con 0x560ba3a67a50 2024-03-21T11:45:05.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:05 smithi047 ceph-mon[31351]: pgmap v1052: 129 pgs: 129 active+clean; 654 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.3 MiB/s wr, 547 op/s 2024-03-21T11:45:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.313+0000 7fc1d601e740 1 -- 172.21.15.47:0/4687428 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba3a6fcf0 con 0x560ba3a67060 2024-03-21T11:45:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.313+0000 7fc1d601e740 10 client.0 _ll_register_callbacks cb 0x560ba3989130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.313+0000 7fc1d1200640 1 --2- 172.21.15.47:0/4687428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a6cbc0 0x560ba3a78620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.313+0000 7fc1cb400640 1 -- 172.21.15.47:0/4687428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67060 msgr2=0x560ba3a710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.313+0000 7fc1cb400640 1 --2- 172.21.15.47:0/4687428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67060 0x560ba3a710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.313+0000 7fc1cb400640 1 -- 172.21.15.47:0/4687428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a6cbc0 msgr2=0x560ba3a78620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:05.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.314+0000 7fc1cb400640 1 --2- 172.21.15.47:0/4687428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a6cbc0 0x560ba3a78620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.314+0000 7fc1cb400640 1 -- 172.21.15.47:0/4687428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ba37997d0 con 0x560ba3a67a50 2024-03-21T11:45:05.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.316+0000 7fc1cb400640 1 --2- 172.21.15.47:0/4687428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a67a50 0x560ba3a715e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00ef300 tx=0x7fc1c00be9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44557d7a7fed419a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:05.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.316+0000 7fc1ae000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:45:05.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.317+0000 7fc1ae000640 1 -- 172.21.15.47:0/4687428 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c00c68f0 con 0x560ba3a67a50 2024-03-21T11:45:05.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.318+0000 7fc1ae000640 1 -- 172.21.15.47:0/4687428 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc1c00bd1b0 con 0x560ba3a67a50 2024-03-21T11:45:05.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.318+0000 7fc1d601e740 1 -- 172.21.15.47:0/4687428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1cc07a070 con 0x560ba3a67a50 2024-03-21T11:45:05.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.318+0000 7fc1ae000640 1 -- 172.21.15.47:0/4687428 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1c0003b10 con 0x560ba3a67a50 2024-03-21T11:45:05.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.319+0000 7fc1d601e740 10 client.32507 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:05.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.319+0000 7fc1d601e740 1 -- 172.21.15.47:0/4687428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560ba36cf3f0 con 0x560ba3a67a50 2024-03-21T11:45:05.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.320+0000 7fc1ae000640 1 -- 172.21.15.47:0/4687428 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1cc07a070 con 0x560ba3a67a50 2024-03-21T11:45:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.321+0000 7fc1ae000640 1 -- 172.21.15.47:0/4687428 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc1c00bdf40 con 0x560ba3a67a50 2024-03-21T11:45:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.321+0000 7fc1d601e740 10 client.32507 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.321+0000 7fc1d601e740 10 client.32507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.321+0000 7fc1d601e740 20 client.32507 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.321+0000 7fc1d601e740 1 --2- 172.21.15.47:0/4687428 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560ba3a7ceb0 0x560ba3a7f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.321+0000 7fc1d601e740 4 client.32507 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.321+0000 7fc1d601e740 1 -- 172.21.15.47:0/4687428 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ba39c8790 con 0x560ba3a7ceb0 2024-03-21T11:45:05.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.322+0000 7fc1cbe00640 1 --2- 172.21.15.47:0/4687428 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560ba3a7ceb0 0x560ba3a7f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.323+0000 7fc1cbe00640 1 --2- 172.21.15.47:0/4687428 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560ba3a7ceb0 0x560ba3a7f310 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0fb48313e96f58a server_cookie=c8f9ae995c886383 in_seq=0 out_seq=0 2024-03-21T11:45:05.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.323+0000 7fc1ae000640 10 client.32507 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:05.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.326+0000 7fc1ae000640 1 -- 172.21.15.47:0/4687428 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560ba39c8790 con 0x560ba3a7ceb0 2024-03-21T11:45:05.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.326+0000 7fc1ae000640 10 client.32507 handle_command_reply: tid=0 2024-03-21T11:45:05.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.326+0000 7fc1d601e740 1 client.32507 shutdown 2024-03-21T11:45:05.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.327+0000 7fc1d601e740 1 -- 172.21.15.47:0/4687428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a67a50 msgr2=0x560ba3a715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:05.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.327+0000 7fc1d601e740 1 --2- 172.21.15.47:0/4687428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a67a50 0x560ba3a715e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00ef300 tx=0x7fc1c00be9f0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.327+0000 7fc1d601e740 1 -- 172.21.15.47:0/4687428 shutdown_connections 2024-03-21T11:45:05.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.327+0000 7fc1d601e740 1 --2- 172.21.15.47:0/4687428 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560ba3a7ceb0 0x560ba3a7f310 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.327+0000 7fc1d601e740 1 --2- 172.21.15.47:0/4687428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a6cbc0 0x560ba3a78620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.327+0000 7fc1d601e740 1 --2- 172.21.15.47:0/4687428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a67a50 0x560ba3a715e0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.327+0000 7fc1d601e740 1 --2- 172.21.15.47:0/4687428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67060 0x560ba3a710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.327+0000 7fc1ae000640 0 client.32507 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:05.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.327+0000 7fc1d601e740 1 -- 172.21.15.47:0/4687428 >> 172.21.15.47:0/4687428 conn(0x560ba3a1e200 msgr2=0x560ba3a50bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:05.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.327+0000 7fc1d601e740 1 -- 172.21.15.47:0/4687428 shutdown_connections 2024-03-21T11:45:05.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.327+0000 7fc1d601e740 1 -- 172.21.15.47:0/4687428 wait complete. 2024-03-21T11:45:05.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.328+0000 7fc1d601e740 20 client.32507 trim_cache size 0 max 16384 2024-03-21T11:45:05.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.364+0000 7fc1d601e740 1 Processor -- start 2024-03-21T11:45:05.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.364+0000 7fc1d601e740 1 -- start start 2024-03-21T11:45:05.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.364+0000 7fc1d601e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a67060 0x560ba3a55310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.364+0000 7fc1d601e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67a50 0x560ba3a785e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.364+0000 7fc1d601e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a6cbc0 0x560ba3a682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.364+0000 7fc1d601e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b000f990 con 0x560ba3a6cbc0 2024-03-21T11:45:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.364+0000 7fc1d601e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0011310 con 0x560ba3a67060 2024-03-21T11:45:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.364+0000 7fc1d601e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b000fff0 con 0x560ba3a67a50 2024-03-21T11:45:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.365+0000 7fc1cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a67060 0x560ba3a55310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.365+0000 7fc1cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a67060 0x560ba3a55310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44390/0 (socket says 172.21.15.47:44390) 2024-03-21T11:45:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.365+0000 7fc1cbe00640 1 -- 172.21.15.47:0/2122041990 learned_addr learned my addr 172.21.15.47:0/2122041990 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.365+0000 7fc1cb400640 1 --2- 172.21.15.47:0/2122041990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67a50 0x560ba3a785e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.366+0000 7fc1cbe00640 1 -- 172.21.15.47:0/2122041990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67a50 msgr2=0x560ba3a785e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.366+0000 7fc1cbe00640 1 --2- 172.21.15.47:0/2122041990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67a50 0x560ba3a785e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.366+0000 7fc1cbe00640 1 -- 172.21.15.47:0/2122041990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a6cbc0 msgr2=0x560ba3a682e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.366+0000 7fc1cbe00640 1 --2- 172.21.15.47:0/2122041990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a6cbc0 0x560ba3a682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.366+0000 7fc1cbe00640 1 -- 172.21.15.47:0/2122041990 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ba368f0f0 con 0x560ba3a67060 2024-03-21T11:45:05.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.366+0000 7fc1cbe00640 1 --2- 172.21.15.47:0/2122041990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a67060 0x560ba3a55310 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x560ba3a7fbc0 tx=0x7fc1bc013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1cf6ad889a57641 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:05.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.368+0000 7fc1b8c00640 1 -- 172.21.15.47:0/2122041990 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc028800 con 0x560ba3a67060 2024-03-21T11:45:05.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.368+0000 7fc1b8c00640 1 -- 172.21.15.47:0/2122041990 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1bc02d070 con 0x560ba3a67060 2024-03-21T11:45:05.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.368+0000 7fc1b8c00640 1 -- 172.21.15.47:0/2122041990 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc01fb90 con 0x560ba3a67060 2024-03-21T11:45:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.369+0000 7fc1d601e740 1 -- 172.21.15.47:0/2122041990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a67060 msgr2=0x560ba3a55310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.369+0000 7fc1d601e740 1 --2- 172.21.15.47:0/2122041990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a67060 0x560ba3a55310 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x560ba3a7fbc0 tx=0x7fc1bc013000 comp rx=0 tx=0).stop 2024-03-21T11:45:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.369+0000 7fc1d601e740 1 -- 172.21.15.47:0/2122041990 shutdown_connections 2024-03-21T11:45:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.369+0000 7fc1d601e740 1 --2- 172.21.15.47:0/2122041990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a6cbc0 0x560ba3a682e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.369+0000 7fc1d601e740 1 --2- 172.21.15.47:0/2122041990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67a50 0x560ba3a785e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.369+0000 7fc1d601e740 1 --2- 172.21.15.47:0/2122041990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a67060 0x560ba3a55310 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.369+0000 7fc1d601e740 1 -- 172.21.15.47:0/2122041990 >> 172.21.15.47:0/2122041990 conn(0x560ba3a1e200 msgr2=0x560ba3a54c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.370+0000 7fc1d601e740 1 -- 172.21.15.47:0/2122041990 shutdown_connections 2024-03-21T11:45:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.370+0000 7fc1d601e740 1 -- 172.21.15.47:0/2122041990 wait complete. 2024-03-21T11:45:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.370+0000 7fc1d601e740 1 -- start start 2024-03-21T11:45:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.371+0000 7fc1d601e740 1 Processor -- start 2024-03-21T11:45:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.373+0000 7fc1d601e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67a50 0x560ba3a76760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.373+0000 7fc1d601e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a6cbc0 0x560ba3a76ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.374+0000 7fc1d601e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a77ad0 0x560ba3a77f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.374+0000 7fc1cbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67a50 0x560ba3a76760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.374+0000 7fc1cbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67a50 0x560ba3a76760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:49332/0 (socket says 172.21.15.47:49332) 2024-03-21T11:45:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.374+0000 7fc1cb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a6cbc0 0x560ba3a76ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.374+0000 7fc1cb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a6cbc0 0x560ba3a76ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44406/0 (socket says 172.21.15.47:44406) 2024-03-21T11:45:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.374+0000 7fc1d601e740 1 -- 172.21.15.47:0/496626320 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0010cb0 con 0x560ba3a77ad0 2024-03-21T11:45:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.374+0000 7fc1cbe00640 1 -- 172.21.15.47:0/496626320 learned_addr learned my addr 172.21.15.47:0/496626320 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.374+0000 7fc1d601e740 1 -- 172.21.15.47:0/496626320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0010650 con 0x560ba3a6cbc0 2024-03-21T11:45:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.374+0000 7fc1d601e740 1 -- 172.21.15.47:0/496626320 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b000db50 con 0x560ba3a67a50 2024-03-21T11:45:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.374+0000 7fc1d601e740 10 client.0 _ll_register_callbacks cb 0x7fc1c00bb150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.374+0000 7fc1d1200640 1 --2- 172.21.15.47:0/496626320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a77ad0 0x560ba3a77f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.375+0000 7fc1cbe00640 1 -- 172.21.15.47:0/496626320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a6cbc0 msgr2=0x560ba3a76ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.375+0000 7fc1cbe00640 1 --2- 172.21.15.47:0/496626320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a6cbc0 0x560ba3a76ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.375+0000 7fc1cbe00640 1 -- 172.21.15.47:0/496626320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a77ad0 msgr2=0x560ba3a77f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.375+0000 7fc1cbe00640 1 --2- 172.21.15.47:0/496626320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a77ad0 0x560ba3a77f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.376+0000 7fc1cbe00640 1 -- 172.21.15.47:0/496626320 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ba3a78550 con 0x560ba3a67a50 2024-03-21T11:45:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.376+0000 7fc1cb400640 1 --2- 172.21.15.47:0/496626320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a6cbc0 0x560ba3a76ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.377+0000 7fc1cbe00640 1 --2- 172.21.15.47:0/496626320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67a50 0x560ba3a76760 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc000c00 tx=0x7fc1bc0114c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c74be201709975da server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.377+0000 7fc1ae000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.378+0000 7fc1ae000640 1 -- 172.21.15.47:0/496626320 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc01fa90 con 0x560ba3a67a50 2024-03-21T11:45:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.378+0000 7fc1ae000640 1 -- 172.21.15.47:0/496626320 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc1bc01fc30 con 0x560ba3a67a50 2024-03-21T11:45:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.378+0000 7fc1d601e740 1 -- 172.21.15.47:0/496626320 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1bc02d070 con 0x560ba3a67a50 2024-03-21T11:45:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.379+0000 7fc1ae000640 1 -- 172.21.15.47:0/496626320 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1bc043050 con 0x560ba3a67a50 2024-03-21T11:45:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.379+0000 7fc1d601e740 10 client.32494 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.379+0000 7fc1d601e740 1 -- 172.21.15.47:0/496626320 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560ba3ad9520 con 0x560ba3a67a50 2024-03-21T11:45:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.379+0000 7fc1ae000640 1 -- 172.21.15.47:0/496626320 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1bc02d070 con 0x560ba3a67a50 2024-03-21T11:45:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.380+0000 7fc1ae000640 1 -- 172.21.15.47:0/496626320 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc1bc03a610 con 0x560ba3a67a50 2024-03-21T11:45:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.380+0000 7fc1d601e740 10 client.32494 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.380+0000 7fc1d601e740 10 client.32494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.380+0000 7fc1d601e740 20 client.32494 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.380+0000 7fc1d601e740 1 --2- 172.21.15.47:0/496626320 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560ba3ad9eb0 0x560ba3adc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.380+0000 7fc1d601e740 4 client.32494 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:45:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.380+0000 7fc1d601e740 1 -- 172.21.15.47:0/496626320 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560ba39c2ae0 con 0x560ba3ad9eb0 2024-03-21T11:45:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.380+0000 7fc1cb400640 1 --2- 172.21.15.47:0/496626320 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560ba3ad9eb0 0x560ba3adc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.381+0000 7fc1cb400640 1 --2- 172.21.15.47:0/496626320 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560ba3ad9eb0 0x560ba3adc2d0 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=730276e981919a36 server_cookie=fc072e7086dad2ac in_seq=0 out_seq=0 2024-03-21T11:45:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.381+0000 7fc1ae000640 10 client.32494 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.389+0000 7fc1ae000640 1 -- 172.21.15.47:0/496626320 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x560ba39c2ae0 con 0x560ba3ad9eb0 2024-03-21T11:45:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.389+0000 7fc1ae000640 10 client.32494 handle_command_reply: tid=0 2024-03-21T11:45:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.389+0000 7fc1d601e740 1 client.32494 shutdown 2024-03-21T11:45:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.390+0000 7fc1d601e740 1 -- 172.21.15.47:0/496626320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67a50 msgr2=0x560ba3a76760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.390+0000 7fc1d601e740 1 --2- 172.21.15.47:0/496626320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67a50 0x560ba3a76760 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc000c00 tx=0x7fc1bc0114c0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.391+0000 7fc1d601e740 1 -- 172.21.15.47:0/496626320 shutdown_connections 2024-03-21T11:45:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.391+0000 7fc1d601e740 1 --2- 172.21.15.47:0/496626320 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560ba3ad9eb0 0x560ba3adc2d0 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.391+0000 7fc1d601e740 1 --2- 172.21.15.47:0/496626320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ba3a77ad0 0x560ba3a77f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.391+0000 7fc1d601e740 1 --2- 172.21.15.47:0/496626320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560ba3a6cbc0 0x560ba3a76ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.391+0000 7fc1d601e740 1 --2- 172.21.15.47:0/496626320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560ba3a67a50 0x560ba3a76760 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.391+0000 7fc1ae000640 0 client.32494 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.391+0000 7fc1d601e740 1 -- 172.21.15.47:0/496626320 >> 172.21.15.47:0/496626320 conn(0x560ba3a1e200 msgr2=0x560ba3a50bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.391+0000 7fc1d601e740 1 -- 172.21.15.47:0/496626320 shutdown_connections 2024-03-21T11:45:05.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.392+0000 7fc1d601e740 1 -- 172.21.15.47:0/496626320 wait complete. 2024-03-21T11:45:05.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.392+0000 7fc1d601e740 20 client.32494 trim_cache size 0 max 16384 2024-03-21T11:45:05.394 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:45:05.394 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (261 inodes in the stack)", 2024-03-21T11:45:05.394 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:45:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "6e2a0303-f2f1-4487-86a6-0fa8887fbaf6": { 2024-03-21T11:45:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:45:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "6e2a0303-f2f1-4487-86a6-0fa8887fbaf6", 2024-03-21T11:45:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:45:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:45:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:45:05.395 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:45:05.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.396+0000 7fc1d1c00640 1 -- 172.21.15.47:0/14633999 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc1b40a2fa0 msgr2=0x7fc1b40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:05.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.396+0000 7fc1d1c00640 1 --2- 172.21.15.47:0/14633999 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc1b40a2fa0 0x7fc1b40a5460 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x560ba3a67e50 tx=0x7fc1bc00b850 comp rx=0 tx=0).stop 2024-03-21T11:45:05.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.396+0000 7fc1d1c00640 1 -- 172.21.15.47:0/14633999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c4008b70 msgr2=0x7fc1c40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:05.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.396+0000 7fc1d1c00640 1 --2- 172.21.15.47:0/14633999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c4008b70 0x7fc1c40a5730 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009a20 tx=0x7fc1c00040c0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.396+0000 7fc1d1c00640 1 -- 172.21.15.47:0/14633999 shutdown_connections 2024-03-21T11:45:05.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.396+0000 7fc1d1c00640 1 --2- 172.21.15.47:0/14633999 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc1b40a2fa0 0x7fc1b40a5460 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.396+0000 7fc1d1c00640 1 --2- 172.21.15.47:0/14633999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1c409dc10 0x7fc1c4142490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.396+0000 7fc1d1c00640 1 --2- 172.21.15.47:0/14633999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c4008b70 0x7fc1c40a5730 secure :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009a20 tx=0x7fc1c00040c0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.396+0000 7fc1d1c00640 1 --2- 172.21.15.47:0/14633999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1c4007590 0x7fc1c40a51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:05.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.396+0000 7fc1d1c00640 1 -- 172.21.15.47:0/14633999 >> 172.21.15.47:0/14633999 conn(0x7fc1c40912d0 msgr2=0x7fc1c4097e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:05.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.397+0000 7fc1d1c00640 1 -- 172.21.15.47:0/14633999 shutdown_connections 2024-03-21T11:45:05.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:05.397+0000 7fc1d1c00640 1 -- 172.21.15.47:0/14633999 wait complete. 2024-03-21T11:45:05.432 INFO:tasks.cephfs.filesystem:scrub status for tag:6e2a0303-f2f1-4487-86a6-0fa8887fbaf6 - {'path': '/', 'tag': '6e2a0303-f2f1-4487-86a6-0fa8887fbaf6', 'options': 'recursive,force'} 2024-03-21T11:45:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:05 smithi102 ceph-mon[36748]: pgmap v1052: 129 pgs: 129 active+clean; 654 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.3 MiB/s wr, 547 op/s 2024-03-21T11:45:05.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:05 smithi100 ceph-mon[36245]: pgmap v1052: 129 pgs: 129 active+clean; 654 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.3 MiB/s wr, 547 op/s 2024-03-21T11:45:05.976 INFO:tasks.workunit.client.0.smithi047.stdout: 10 202575 22.26 MB/sec execute 349 sec latency 122.083 ms 2024-03-21T11:45:06.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:06 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:45:06.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:06 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:06 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:06.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:06 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:06.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:06 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:06.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:06 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:06.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:06 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:06 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:06.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:06 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:06 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:06.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:06 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:06 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:06 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:06.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:06 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:06 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:45:06.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:06 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:06 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:06.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:06 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:06.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:06 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:06.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:06 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:06.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:06 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:06 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:06.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:06 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:06 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:06.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:06 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:06 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:06 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:06.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:06 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.579 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:06 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:45:06.579 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:06 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.579 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:06 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:06.579 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:06 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:06.579 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:06 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:06.579 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:06 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:06.579 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:06 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.580 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:06 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:06.580 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:06 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.580 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:06 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:06.580 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:06 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.580 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:06 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.580 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:06 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:06.580 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:06 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:06.976 INFO:tasks.workunit.client.0.smithi047.stdout: 10 202959 22.25 MB/sec execute 350 sec latency 75.491 ms 2024-03-21T11:45:07.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:07 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:07.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:07 smithi102 ceph-mon[36748]: pgmap v1053: 129 pgs: 129 active+clean; 689 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 674 op/s 2024-03-21T11:45:07.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:07 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:07.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:07 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:07.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:07 smithi047 ceph-mon[31351]: pgmap v1053: 129 pgs: 129 active+clean; 689 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 674 op/s 2024-03-21T11:45:07.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:07 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:07.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:07 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:07.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:07 smithi100 ceph-mon[36245]: pgmap v1053: 129 pgs: 129 active+clean; 689 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 674 op/s 2024-03-21T11:45:07.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:07 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:07.976 INFO:tasks.workunit.client.0.smithi047.stdout: 10 203329 22.25 MB/sec execute 351 sec latency 123.785 ms 2024-03-21T11:45:08.976 INFO:tasks.workunit.client.0.smithi047.stdout: 10 203656 22.25 MB/sec execute 352 sec latency 137.703 ms 2024-03-21T11:45:09.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:09 smithi102 ceph-mon[36748]: pgmap v1054: 129 pgs: 129 active+clean; 687 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 652 op/s 2024-03-21T11:45:09.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:09 smithi047 ceph-mon[31351]: pgmap v1054: 129 pgs: 129 active+clean; 687 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 652 op/s 2024-03-21T11:45:09.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:09 smithi100 ceph-mon[36245]: pgmap v1054: 129 pgs: 129 active+clean; 687 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 652 op/s 2024-03-21T11:45:09.976 INFO:tasks.workunit.client.0.smithi047.stdout: 10 204170 22.27 MB/sec execute 353 sec latency 81.350 ms 2024-03-21T11:45:10.977 INFO:tasks.workunit.client.0.smithi047.stdout: 10 204653 22.28 MB/sec execute 354 sec latency 48.016 ms 2024-03-21T11:45:11.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:11 smithi102 ceph-mon[36748]: pgmap v1055: 129 pgs: 129 active+clean; 631 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 767 op/s 2024-03-21T11:45:11.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:11 smithi047 ceph-mon[31351]: pgmap v1055: 129 pgs: 129 active+clean; 631 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 767 op/s 2024-03-21T11:45:11.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:11 smithi100 ceph-mon[36245]: pgmap v1055: 129 pgs: 129 active+clean; 631 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 767 op/s 2024-03-21T11:45:11.977 INFO:tasks.workunit.client.0.smithi047.stdout: 10 205082 22.28 MB/sec execute 355 sec latency 89.439 ms 2024-03-21T11:45:12.977 INFO:tasks.workunit.client.0.smithi047.stdout: 10 205543 22.28 MB/sec execute 356 sec latency 72.874 ms 2024-03-21T11:45:13.198 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:13 smithi102 ceph-mon[36748]: pgmap v1056: 129 pgs: 129 active+clean; 622 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 759 op/s 2024-03-21T11:45:13.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:13 smithi047 ceph-mon[31351]: pgmap v1056: 129 pgs: 129 active+clean; 622 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 759 op/s 2024-03-21T11:45:13.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:13 smithi100 ceph-mon[36245]: pgmap v1056: 129 pgs: 129 active+clean; 622 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 759 op/s 2024-03-21T11:45:13.977 INFO:tasks.workunit.client.0.smithi047.stdout: 10 206057 22.29 MB/sec execute 357 sec latency 70.557 ms 2024-03-21T11:45:14.977 INFO:tasks.workunit.client.0.smithi047.stdout: 10 206629 22.30 MB/sec execute 358 sec latency 47.657 ms 2024-03-21T11:45:15.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:15 smithi102 ceph-mon[36748]: pgmap v1057: 129 pgs: 129 active+clean; 622 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 774 op/s 2024-03-21T11:45:15.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:15 smithi047 ceph-mon[31351]: pgmap v1057: 129 pgs: 129 active+clean; 622 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 774 op/s 2024-03-21T11:45:15.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:15 smithi100 ceph-mon[36245]: pgmap v1057: 129 pgs: 129 active+clean; 622 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 774 op/s 2024-03-21T11:45:15.977 INFO:tasks.workunit.client.0.smithi047.stdout: 10 207284 22.35 MB/sec execute 359 sec latency 50.792 ms 2024-03-21T11:45:16.977 INFO:tasks.workunit.client.0.smithi047.stdout: 10 207683 22.36 MB/sec execute 360 sec latency 338.776 ms 2024-03-21T11:45:17.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:17 smithi102 ceph-mon[36748]: pgmap v1058: 129 pgs: 129 active+clean; 627 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.3 MiB/s wr, 725 op/s 2024-03-21T11:45:17.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:17 smithi047 ceph-mon[31351]: pgmap v1058: 129 pgs: 129 active+clean; 627 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.3 MiB/s wr, 725 op/s 2024-03-21T11:45:17.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:17 smithi100 ceph-mon[36245]: pgmap v1058: 129 pgs: 129 active+clean; 627 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.3 MiB/s wr, 725 op/s 2024-03-21T11:45:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.970+0000 7f6a33200640 1 -- 172.21.15.47:0/2991759598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a2c0ff750 msgr2=0x7f6a2c0ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.970+0000 7f6a33200640 1 --2- 172.21.15.47:0/2991759598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a2c0ff750 0x7f6a2c0ffbd0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009860 tx=0x7f6a1802bb70 comp rx=0 tx=0).stop 2024-03-21T11:45:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.971+0000 7f6a33200640 1 -- 172.21.15.47:0/2991759598 shutdown_connections 2024-03-21T11:45:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.971+0000 7f6a33200640 1 --2- 172.21.15.47:0/2991759598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a2c100110 0x7f6a2c1005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.971+0000 7f6a33200640 1 --2- 172.21.15.47:0/2991759598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a2c0ff750 0x7f6a2c0ffbd0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.971+0000 7f6a33200640 1 --2- 172.21.15.47:0/2991759598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c101bd0 0x7f6a2c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.971+0000 7f6a33200640 1 -- 172.21.15.47:0/2991759598 >> 172.21.15.47:0/2991759598 conn(0x7f6a2c0fb2f0 msgr2=0x7f6a2c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.971+0000 7f6a33200640 1 -- 172.21.15.47:0/2991759598 shutdown_connections 2024-03-21T11:45:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.971+0000 7f6a33200640 1 -- 172.21.15.47:0/2991759598 wait complete. 2024-03-21T11:45:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.972+0000 7f6a33200640 1 Processor -- start 2024-03-21T11:45:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.972+0000 7f6a33200640 1 -- start start 2024-03-21T11:45:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.972+0000 7f6a33200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a2c0ff750 0x7f6a2c19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.972+0000 7f6a33200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a2c100110 0x7f6a2c19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.972+0000 7f6a33200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c101bd0 0x7f6a2c1a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.972+0000 7f6a33200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c1139a0 con 0x7f6a2c101bd0 2024-03-21T11:45:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.972+0000 7f6a33200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c113820 con 0x7f6a2c0ff750 2024-03-21T11:45:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.972+0000 7f6a33200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c113b20 con 0x7f6a2c100110 2024-03-21T11:45:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.973+0000 7f6a2be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a2c0ff750 0x7f6a2c19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.973+0000 7f6a2b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a2c100110 0x7f6a2c19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.973+0000 7f6a2be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a2c0ff750 0x7f6a2c19df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58034/0 (socket says 172.21.15.47:58034) 2024-03-21T11:45:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.973+0000 7f6a2be00640 1 -- 172.21.15.47:0/951853133 learned_addr learned my addr 172.21.15.47:0/951853133 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.973+0000 7f6a2b400640 1 --2- 172.21.15.47:0/951853133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a2c100110 0x7f6a2c19e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56664/0 (socket says 172.21.15.47:56664) 2024-03-21T11:45:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.973+0000 7f6a31200640 1 --2- 172.21.15.47:0/951853133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c101bd0 0x7f6a2c1a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.973+0000 7f6a2be00640 1 -- 172.21.15.47:0/951853133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a2c100110 msgr2=0x7f6a2c19e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.973+0000 7f6a2be00640 1 --2- 172.21.15.47:0/951853133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a2c100110 0x7f6a2c19e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.973+0000 7f6a2be00640 1 -- 172.21.15.47:0/951853133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c101bd0 msgr2=0x7f6a2c1a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.973+0000 7f6a2be00640 1 --2- 172.21.15.47:0/951853133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c101bd0 0x7f6a2c1a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.973+0000 7f6a2be00640 1 -- 172.21.15.47:0/951853133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c1a5b30 con 0x7f6a2c0ff750 2024-03-21T11:45:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.974+0000 7f6a2be00640 1 --2- 172.21.15.47:0/951853133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a2c0ff750 0x7f6a2c19df10 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00e9f0 tx=0x7f6a1c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.974+0000 7f6a28c00640 1 -- 172.21.15.47:0/951853133 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c00ccc0 con 0x7f6a2c0ff750 2024-03-21T11:45:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.974+0000 7f6a28c00640 1 -- 172.21.15.47:0/951853133 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c0035f0 con 0x7f6a2c0ff750 2024-03-21T11:45:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.974+0000 7f6a33200640 1 -- 172.21.15.47:0/951853133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2c1a5e20 con 0x7f6a2c0ff750 2024-03-21T11:45:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.974+0000 7f6a28c00640 1 -- 172.21.15.47:0/951853133 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c00ce60 con 0x7f6a2c0ff750 2024-03-21T11:45:17.979 INFO:tasks.workunit.client.0.smithi047.stdout: 10 207723 22.30 MB/sec execute 361 sec latency 1339.228 ms 2024-03-21T11:45:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.975+0000 7f6a33200640 1 -- 172.21.15.47:0/951853133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2c1a6280 con 0x7f6a2c0ff750 2024-03-21T11:45:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.980+0000 7f6a28c00640 1 -- 172.21.15.47:0/951853133 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6a1c0037a0 con 0x7f6a2c0ff750 2024-03-21T11:45:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.982+0000 7f6a28c00640 1 --2- 172.21.15.47:0/951853133 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a100a31b0 0x7f6a100a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.982+0000 7f6a28c00640 1 -- 172.21.15.47:0/951853133 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6a1c0cf8e0 con 0x7f6a2c0ff750 2024-03-21T11:45:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.983+0000 7f6a2b400640 1 --2- 172.21.15.47:0/951853133 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a100a31b0 0x7f6a100a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:17.984+0000 7f6a2b400640 1 --2- 172.21.15.47:0/951853133 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a100a31b0 0x7f6a100a5670 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6a1802c180 tx=0x7f6a18034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.017+0000 7f6a35ff8740 1 Processor -- start 2024-03-21T11:45:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.017+0000 7f6a35ff8740 1 -- start start 2024-03-21T11:45:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.017+0000 7f6a35ff8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d28600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.017+0000 7f6a35ff8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d28bd0 0x561dc5d2d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.017+0000 7f6a35ff8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d2dd40 0x561dc5d301d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.017+0000 7f6a35ff8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dc5ceb3b0 con 0x561dc5d2dd40 2024-03-21T11:45:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.017+0000 7f6a35ff8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dc5c8d830 con 0x561dc5d281e0 2024-03-21T11:45:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.017+0000 7f6a35ff8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dc5c365d0 con 0x561dc5d28bd0 2024-03-21T11:45:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.018+0000 7f6a2be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d28600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.018+0000 7f6a2be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d28600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58036/0 (socket says 172.21.15.47:58036) 2024-03-21T11:45:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.018+0000 7f6a2be00640 1 -- 172.21.15.47:0/1637496609 learned_addr learned my addr 172.21.15.47:0/1637496609 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.018+0000 7f6a31200640 1 --2- 172.21.15.47:0/1637496609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d2dd40 0x561dc5d301d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.018+0000 7f6a2b400640 1 --2- 172.21.15.47:0/1637496609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d28bd0 0x561dc5d2d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.018+0000 7f6a2be00640 1 -- 172.21.15.47:0/1637496609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d28bd0 msgr2=0x561dc5d2d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.018+0000 7f6a2be00640 1 --2- 172.21.15.47:0/1637496609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d28bd0 0x561dc5d2d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.018+0000 7f6a2be00640 1 -- 172.21.15.47:0/1637496609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d2dd40 msgr2=0x561dc5d301d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.018+0000 7f6a2be00640 1 --2- 172.21.15.47:0/1637496609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d2dd40 0x561dc5d301d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.018+0000 7f6a2be00640 1 -- 172.21.15.47:0/1637496609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dc5c4e2c0 con 0x561dc5d281e0 2024-03-21T11:45:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.018+0000 7f6a31200640 1 --2- 172.21.15.47:0/1637496609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d2dd40 0x561dc5d301d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.019+0000 7f6a2be00640 1 --2- 172.21.15.47:0/1637496609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d28600 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0a4100 tx=0x7f6a1c09b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87ade19e32b96863 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.019+0000 7f6a21600640 1 -- 172.21.15.47:0/1637496609 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c09fd70 con 0x561dc5d281e0 2024-03-21T11:45:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.019+0000 7f6a21600640 1 -- 172.21.15.47:0/1637496609 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c0a4630 con 0x561dc5d281e0 2024-03-21T11:45:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.019+0000 7f6a21600640 1 -- 172.21.15.47:0/1637496609 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0a4400 con 0x561dc5d281e0 2024-03-21T11:45:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.019+0000 7f6a35ff8740 1 -- 172.21.15.47:0/1637496609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 msgr2=0x561dc5d28600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.019+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/1637496609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d28600 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0a4100 tx=0x7f6a1c09b640 comp rx=0 tx=0).stop 2024-03-21T11:45:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.020+0000 7f6a35ff8740 1 -- 172.21.15.47:0/1637496609 shutdown_connections 2024-03-21T11:45:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.020+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/1637496609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d2dd40 0x561dc5d301d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.020+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/1637496609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d28bd0 0x561dc5d2d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.020+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/1637496609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d28600 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.020+0000 7f6a35ff8740 1 -- 172.21.15.47:0/1637496609 >> 172.21.15.47:0/1637496609 conn(0x561dc5d23750 msgr2=0x561dc5d23b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.020+0000 7f6a35ff8740 1 -- 172.21.15.47:0/1637496609 shutdown_connections 2024-03-21T11:45:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.020+0000 7f6a35ff8740 1 -- 172.21.15.47:0/1637496609 wait complete. 2024-03-21T11:45:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.020+0000 7f6a35ff8740 1 -- start start 2024-03-21T11:45:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.021+0000 7f6a35ff8740 1 Processor -- start 2024-03-21T11:45:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.021+0000 7f6a35ff8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d32310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.021+0000 7f6a35ff8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d28bd0 0x561dc5d32850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.021+0000 7f6a35ff8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d2dd40 0x561dc5d39890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.021+0000 7f6a35ff8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dc5c49a20 con 0x561dc5d28bd0 2024-03-21T11:45:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.021+0000 7f6a35ff8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dc5c898d0 con 0x561dc5d281e0 2024-03-21T11:45:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.021+0000 7f6a35ff8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dc5d30c30 con 0x561dc5d2dd40 2024-03-21T11:45:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.021+0000 7f6a35ff8740 10 client.0 _ll_register_callbacks cb 0x561dc5c861c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.022+0000 7f6a2be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d32310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.022+0000 7f6a2b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d28bd0 0x561dc5d32850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.022+0000 7f6a2be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d32310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58040/0 (socket says 172.21.15.47:58040) 2024-03-21T11:45:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.022+0000 7f6a2b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d28bd0 0x561dc5d32850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37654/0 (socket says 172.21.15.47:37654) 2024-03-21T11:45:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.022+0000 7f6a2be00640 1 -- 172.21.15.47:0/2798358912 learned_addr learned my addr 172.21.15.47:0/2798358912 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.022+0000 7f6a31200640 1 --2- 172.21.15.47:0/2798358912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d2dd40 0x561dc5d39890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.022+0000 7f6a2be00640 1 -- 172.21.15.47:0/2798358912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d2dd40 msgr2=0x561dc5d39890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.022+0000 7f6a2be00640 1 --2- 172.21.15.47:0/2798358912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d2dd40 0x561dc5d39890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.022+0000 7f6a2be00640 1 -- 172.21.15.47:0/2798358912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d28bd0 msgr2=0x561dc5d32850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.022+0000 7f6a2be00640 1 --2- 172.21.15.47:0/2798358912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d28bd0 0x561dc5d32850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.022+0000 7f6a2be00640 1 -- 172.21.15.47:0/2798358912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561dc59a2210 con 0x561dc5d281e0 2024-03-21T11:45:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.022+0000 7f6a31200640 1 --2- 172.21.15.47:0/2798358912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d2dd40 0x561dc5d39890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:45:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.022+0000 7f6a2be00640 1 --2- 172.21.15.47:0/2798358912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d32310 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00c6a0 tx=0x7f6a1c003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=637a1a933ce695c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.022+0000 7f6a16000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:45:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.023+0000 7f6a16000640 1 -- 172.21.15.47:0/2798358912 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c09b070 con 0x561dc5d281e0 2024-03-21T11:45:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.023+0000 7f6a16000640 1 -- 172.21.15.47:0/2798358912 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6a1c09b210 con 0x561dc5d281e0 2024-03-21T11:45:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.023+0000 7f6a35ff8740 1 -- 172.21.15.47:0/2798358912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a1c0a4630 con 0x561dc5d281e0 2024-03-21T11:45:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.023+0000 7f6a16000640 1 -- 172.21.15.47:0/2798358912 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a1c07ab10 con 0x561dc5d281e0 2024-03-21T11:45:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.023+0000 7f6a35ff8740 10 client.32525 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.023+0000 7f6a35ff8740 1 -- 172.21.15.47:0/2798358912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561dc5a6e620 con 0x561dc5d281e0 2024-03-21T11:45:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.023+0000 7f6a16000640 1 -- 172.21.15.47:0/2798358912 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c0a4630 con 0x561dc5d281e0 2024-03-21T11:45:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.023+0000 7f6a16000640 1 -- 172.21.15.47:0/2798358912 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6a1c0ac030 con 0x561dc5d281e0 2024-03-21T11:45:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.023+0000 7f6a35ff8740 10 client.32525 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.023+0000 7f6a35ff8740 10 client.32525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.023+0000 7f6a35ff8740 20 client.32525 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.024+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/2798358912 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561dc5d3dd80 0x561dc5d40170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.024+0000 7f6a35ff8740 4 client.32525 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.024+0000 7f6a35ff8740 1 -- 172.21.15.47:0/2798358912 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561dc5c9ad70 con 0x561dc5d3dd80 2024-03-21T11:45:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.024+0000 7f6a2b400640 1 --2- 172.21.15.47:0/2798358912 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561dc5d3dd80 0x561dc5d40170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.024+0000 7f6a16000640 10 client.32525 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.024+0000 7f6a2b400640 1 --2- 172.21.15.47:0/2798358912 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561dc5d3dd80 0x561dc5d40170 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fee49e26ab20e71 server_cookie=f09e78c4259efd9b in_seq=0 out_seq=0 2024-03-21T11:45:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.025+0000 7f6a16000640 1 -- 172.21.15.47:0/2798358912 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561dc5c9ad70 con 0x561dc5d3dd80 2024-03-21T11:45:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.026+0000 7f6a16000640 10 client.32525 handle_command_reply: tid=0 2024-03-21T11:45:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.026+0000 7f6a35ff8740 1 client.32525 shutdown 2024-03-21T11:45:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.026+0000 7f6a35ff8740 1 -- 172.21.15.47:0/2798358912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 msgr2=0x561dc5d32310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.026+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/2798358912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d32310 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00c6a0 tx=0x7f6a1c003e10 comp rx=0 tx=0).stop 2024-03-21T11:45:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.026+0000 7f6a35ff8740 1 -- 172.21.15.47:0/2798358912 shutdown_connections 2024-03-21T11:45:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.026+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/2798358912 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561dc5d3dd80 0x561dc5d40170 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.026+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/2798358912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d2dd40 0x561dc5d39890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.026+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/2798358912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d28bd0 0x561dc5d32850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.026+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/2798358912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d32310 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.026+0000 7f6a35ff8740 1 -- 172.21.15.47:0/2798358912 >> 172.21.15.47:0/2798358912 conn(0x561dc5d23750 msgr2=0x561dc5d22950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.026+0000 7f6a16000640 0 client.32525 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.026+0000 7f6a35ff8740 1 -- 172.21.15.47:0/2798358912 shutdown_connections 2024-03-21T11:45:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.026+0000 7f6a35ff8740 1 -- 172.21.15.47:0/2798358912 wait complete. 2024-03-21T11:45:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.026+0000 7f6a35ff8740 20 client.32525 trim_cache size 0 max 16384 2024-03-21T11:45:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.049+0000 7f6a35ff8740 1 Processor -- start 2024-03-21T11:45:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.049+0000 7f6a35ff8740 1 -- start start 2024-03-21T11:45:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.049+0000 7f6a35ff8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d327b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.049+0000 7f6a35ff8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d28bd0 0x561dc5d39850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.049+0000 7f6a35ff8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d2dd40 0x561dc5d270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.049+0000 7f6a35ff8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c1b0640 con 0x561dc5d28bd0 2024-03-21T11:45:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.049+0000 7f6a35ff8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c1b1fc0 con 0x561dc5d281e0 2024-03-21T11:45:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.049+0000 7f6a35ff8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c1b0ca0 con 0x561dc5d2dd40 2024-03-21T11:45:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.049+0000 7f6a2b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d28bd0 0x561dc5d39850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.049+0000 7f6a31200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d2dd40 0x561dc5d270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.049+0000 7f6a2be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.049+0000 7f6a31200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d2dd40 0x561dc5d270c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56690/0 (socket says 172.21.15.47:56690) 2024-03-21T11:45:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.049+0000 7f6a2be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d327b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58042/0 (socket says 172.21.15.47:58042) 2024-03-21T11:45:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.049+0000 7f6a31200640 1 -- 172.21.15.47:0/3882418498 learned_addr learned my addr 172.21.15.47:0/3882418498 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.050+0000 7f6a2be00640 1 -- 172.21.15.47:0/3882418498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d2dd40 msgr2=0x561dc5d270c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.050+0000 7f6a2be00640 1 --2- 172.21.15.47:0/3882418498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d2dd40 0x561dc5d270c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.050+0000 7f6a2be00640 1 -- 172.21.15.47:0/3882418498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d28bd0 msgr2=0x561dc5d39850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.050+0000 7f6a2be00640 1 --2- 172.21.15.47:0/3882418498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d28bd0 0x561dc5d39850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.050+0000 7f6a2be00640 1 -- 172.21.15.47:0/3882418498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dc5d404e0 con 0x561dc5d281e0 2024-03-21T11:45:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.050+0000 7f6a2b400640 1 --2- 172.21.15.47:0/3882418498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d28bd0 0x561dc5d39850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:45:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.050+0000 7f6a2be00640 1 --2- 172.21.15.47:0/3882418498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d327b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0985f0 tx=0x7f6a1c003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41511d5b637956e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.051+0000 7f6a20c00640 1 -- 172.21.15.47:0/3882418498 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c09ad20 con 0x561dc5d281e0 2024-03-21T11:45:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.051+0000 7f6a20c00640 1 -- 172.21.15.47:0/3882418498 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c0a4630 con 0x561dc5d281e0 2024-03-21T11:45:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.051+0000 7f6a20c00640 1 -- 172.21.15.47:0/3882418498 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c09ab30 con 0x561dc5d281e0 2024-03-21T11:45:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.052+0000 7f6a35ff8740 1 -- 172.21.15.47:0/3882418498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 msgr2=0x561dc5d327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.052+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/3882418498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d327b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0985f0 tx=0x7f6a1c003e80 comp rx=0 tx=0).stop 2024-03-21T11:45:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.052+0000 7f6a35ff8740 1 -- 172.21.15.47:0/3882418498 shutdown_connections 2024-03-21T11:45:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.052+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/3882418498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d2dd40 0x561dc5d270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.052+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/3882418498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d28bd0 0x561dc5d39850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.052+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/3882418498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d281e0 0x561dc5d327b0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.052+0000 7f6a35ff8740 1 -- 172.21.15.47:0/3882418498 >> 172.21.15.47:0/3882418498 conn(0x561dc5d23750 msgr2=0x561dc5d269b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.052+0000 7f6a35ff8740 1 -- 172.21.15.47:0/3882418498 shutdown_connections 2024-03-21T11:45:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.052+0000 7f6a35ff8740 1 -- 172.21.15.47:0/3882418498 wait complete. 2024-03-21T11:45:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.053+0000 7f6a35ff8740 1 -- start start 2024-03-21T11:45:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.053+0000 7f6a35ff8740 1 Processor -- start 2024-03-21T11:45:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.054+0000 7f6a35ff8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d281e0 0x561dc5d37180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.054+0000 7f6a35ff8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d28bd0 0x561dc5d376c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.054+0000 7f6a35ff8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d2dd40 0x561dc5d38610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.055+0000 7f6a2be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d281e0 0x561dc5d37180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.055+0000 7f6a2be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d281e0 0x561dc5d37180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:56706/0 (socket says 172.21.15.47:56706) 2024-03-21T11:45:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.055+0000 7f6a2be00640 1 -- 172.21.15.47:0/4268256499 learned_addr learned my addr 172.21.15.47:0/4268256499 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.055+0000 7f6a35ff8740 1 -- 172.21.15.47:0/4268256499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c1b1960 con 0x561dc5d2dd40 2024-03-21T11:45:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.055+0000 7f6a35ff8740 1 -- 172.21.15.47:0/4268256499 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c1b1300 con 0x561dc5d28bd0 2024-03-21T11:45:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.055+0000 7f6a35ff8740 1 -- 172.21.15.47:0/4268256499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c1ae800 con 0x561dc5d281e0 2024-03-21T11:45:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.055+0000 7f6a2b400640 1 --2- 172.21.15.47:0/4268256499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d28bd0 0x561dc5d376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.055+0000 7f6a35ff8740 10 client.0 _ll_register_callbacks cb 0x7f6a1c0b4f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.055+0000 7f6a31200640 1 --2- 172.21.15.47:0/4268256499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d2dd40 0x561dc5d38610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.055+0000 7f6a2be00640 1 -- 172.21.15.47:0/4268256499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d28bd0 msgr2=0x561dc5d376c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.055+0000 7f6a2be00640 1 --2- 172.21.15.47:0/4268256499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d28bd0 0x561dc5d376c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.055+0000 7f6a2be00640 1 -- 172.21.15.47:0/4268256499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d2dd40 msgr2=0x561dc5d38610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.056+0000 7f6a2be00640 1 --2- 172.21.15.47:0/4268256499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d2dd40 0x561dc5d38610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.056+0000 7f6a2be00640 1 -- 172.21.15.47:0/4268256499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561dc5d38c00 con 0x561dc5d281e0 2024-03-21T11:45:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.056+0000 7f6a31200640 1 --2- 172.21.15.47:0/4268256499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d2dd40 0x561dc5d38610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:45:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.056+0000 7f6a2be00640 1 --2- 172.21.15.47:0/4268256499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d281e0 0x561dc5d37180 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009530 tx=0x7f6a1c09b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84db6081a9ce7dc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.056+0000 7f6a16000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.056+0000 7f6a16000640 1 -- 172.21.15.47:0/4268256499 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c09ab30 con 0x561dc5d281e0 2024-03-21T11:45:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.056+0000 7f6a16000640 1 -- 172.21.15.47:0/4268256499 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6a1c004000 con 0x561dc5d281e0 2024-03-21T11:45:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.056+0000 7f6a35ff8740 1 -- 172.21.15.47:0/4268256499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a1c0a4630 con 0x561dc5d281e0 2024-03-21T11:45:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.057+0000 7f6a16000640 1 -- 172.21.15.47:0/4268256499 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a1c09ad20 con 0x561dc5d281e0 2024-03-21T11:45:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.057+0000 7f6a35ff8740 10 client.32512 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.058+0000 7f6a35ff8740 1 -- 172.21.15.47:0/4268256499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561dc5d38c00 con 0x561dc5d281e0 2024-03-21T11:45:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.058+0000 7f6a16000640 1 -- 172.21.15.47:0/4268256499 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c0a4630 con 0x561dc5d281e0 2024-03-21T11:45:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.058+0000 7f6a16000640 1 -- 172.21.15.47:0/4268256499 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6a1c0ac030 con 0x561dc5d281e0 2024-03-21T11:45:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.058+0000 7f6a35ff8740 10 client.32512 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.058+0000 7f6a35ff8740 10 client.32512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.058+0000 7f6a35ff8740 20 client.32512 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.058+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/4268256499 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561dc5d999f0 0x561dc5d9be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.058+0000 7f6a35ff8740 4 client.32512 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:45:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.058+0000 7f6a35ff8740 1 -- 172.21.15.47:0/4268256499 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561dc5d30900 con 0x561dc5d999f0 2024-03-21T11:45:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.060+0000 7f6a2b400640 1 --2- 172.21.15.47:0/4268256499 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561dc5d999f0 0x561dc5d9be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.061+0000 7f6a2b400640 1 --2- 172.21.15.47:0/4268256499 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561dc5d999f0 0x561dc5d9be10 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b6e7653355e7 server_cookie=8d5666df11a035fb in_seq=0 out_seq=0 2024-03-21T11:45:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:18.061+0000 7f6a16000640 10 client.32512 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:18 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:45:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:18 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:45:18.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:18 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:45:18.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:18 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:45:18.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:18 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:45:18.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:18 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:45:18.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:18 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:45:18.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:18 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:45:18.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:18 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:45:18.978 INFO:tasks.workunit.client.0.smithi047.stdout: 10 207846 22.26 MB/sec execute 362 sec latency 1713.426 ms 2024-03-21T11:45:19.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:19 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:45:19.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:19 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:45:19.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:19 smithi102 ceph-mon[36748]: pgmap v1059: 129 pgs: 129 active+clean; 634 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.3 MiB/s wr, 543 op/s 2024-03-21T11:45:19.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:19 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:45:19.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:19 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:45:19.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:19 smithi047 ceph-mon[31351]: pgmap v1059: 129 pgs: 129 active+clean; 634 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.3 MiB/s wr, 543 op/s 2024-03-21T11:45:19.646 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:19 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:45:19.646 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:19 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:45:19.646 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:19 smithi100 ceph-mon[36245]: pgmap v1059: 129 pgs: 129 active+clean; 634 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.3 MiB/s wr, 543 op/s 2024-03-21T11:45:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.655+0000 7f6a16000640 1 -- 172.21.15.47:0/4268256499 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561dc5d30900 con 0x561dc5d999f0 2024-03-21T11:45:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.655+0000 7f6a16000640 10 client.32512 handle_command_reply: tid=0 2024-03-21T11:45:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.655+0000 7f6a35ff8740 1 client.32512 shutdown 2024-03-21T11:45:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.655+0000 7f6a35ff8740 1 -- 172.21.15.47:0/4268256499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d281e0 msgr2=0x561dc5d37180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.655+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/4268256499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d281e0 0x561dc5d37180 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009530 tx=0x7f6a1c09b9f0 comp rx=0 tx=0).stop 2024-03-21T11:45:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.655+0000 7f6a35ff8740 1 -- 172.21.15.47:0/4268256499 shutdown_connections 2024-03-21T11:45:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.655+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/4268256499 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561dc5d999f0 0x561dc5d9be10 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.655+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/4268256499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561dc5d2dd40 0x561dc5d38610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.655+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/4268256499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dc5d28bd0 0x561dc5d376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.655+0000 7f6a35ff8740 1 --2- 172.21.15.47:0/4268256499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561dc5d281e0 0x561dc5d37180 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.655+0000 7f6a16000640 0 client.32512 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.655+0000 7f6a35ff8740 1 -- 172.21.15.47:0/4268256499 >> 172.21.15.47:0/4268256499 conn(0x561dc5d23750 msgr2=0x561dc5d27240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.656+0000 7f6a35ff8740 1 -- 172.21.15.47:0/4268256499 shutdown_connections 2024-03-21T11:45:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.656+0000 7f6a35ff8740 1 -- 172.21.15.47:0/4268256499 wait complete. 2024-03-21T11:45:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.656+0000 7f6a35ff8740 20 client.32512 trim_cache size 0 max 16384 2024-03-21T11:45:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:45:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:45:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:45:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:45:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.659+0000 7f6a33200640 1 -- 172.21.15.47:0/951853133 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a100a31b0 msgr2=0x7f6a100a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.659+0000 7f6a33200640 1 --2- 172.21.15.47:0/951853133 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a100a31b0 0x7f6a100a5670 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6a1802c180 tx=0x7f6a18034ad0 comp rx=0 tx=0).stop 2024-03-21T11:45:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.659+0000 7f6a33200640 1 -- 172.21.15.47:0/951853133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a2c0ff750 msgr2=0x7f6a2c19df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.659+0000 7f6a33200640 1 --2- 172.21.15.47:0/951853133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a2c0ff750 0x7f6a2c19df10 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00e9f0 tx=0x7f6a1c00eeb0 comp rx=0 tx=0).stop 2024-03-21T11:45:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.659+0000 7f6a33200640 1 -- 172.21.15.47:0/951853133 shutdown_connections 2024-03-21T11:45:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.659+0000 7f6a33200640 1 --2- 172.21.15.47:0/951853133 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6a100a31b0 0x7f6a100a5670 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.659+0000 7f6a33200640 1 --2- 172.21.15.47:0/951853133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a2c101bd0 0x7f6a2c1a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.659+0000 7f6a33200640 1 --2- 172.21.15.47:0/951853133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a2c100110 0x7f6a2c19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.659+0000 7f6a33200640 1 --2- 172.21.15.47:0/951853133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a2c0ff750 0x7f6a2c19df10 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.659+0000 7f6a33200640 1 -- 172.21.15.47:0/951853133 >> 172.21.15.47:0/951853133 conn(0x7f6a2c0fb2f0 msgr2=0x7f6a2c106e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.660+0000 7f6a33200640 1 -- 172.21.15.47:0/951853133 shutdown_connections 2024-03-21T11:45:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:19.660+0000 7f6a33200640 1 -- 172.21.15.47:0/951853133 wait complete. 2024-03-21T11:45:19.978 INFO:tasks.workunit.client.0.smithi047.stdout: 10 207983 22.23 MB/sec execute 363 sec latency 809.962 ms 2024-03-21T11:45:20.978 INFO:tasks.workunit.client.0.smithi047.stdout: 10 208195 22.19 MB/sec execute 364 sec latency 116.967 ms 2024-03-21T11:45:21.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:21 smithi102 ceph-mon[36748]: pgmap v1060: 129 pgs: 129 active+clean; 667 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.4 MiB/s wr, 633 op/s 2024-03-21T11:45:21.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:21 smithi047 ceph-mon[31351]: pgmap v1060: 129 pgs: 129 active+clean; 667 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.4 MiB/s wr, 633 op/s 2024-03-21T11:45:21.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:21 smithi100 ceph-mon[36245]: pgmap v1060: 129 pgs: 129 active+clean; 667 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.4 MiB/s wr, 633 op/s 2024-03-21T11:45:21.978 INFO:tasks.workunit.client.0.smithi047.stdout: 10 208581 22.19 MB/sec execute 365 sec latency 72.204 ms 2024-03-21T11:45:22.143 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:21 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e[65693]: *** Caught signal (Segmentation fault) ** 2024-03-21T11:45:22.144 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:21 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e[65693]: in thread 7fa2b396b700 thread_name:MR_Finisher 2024-03-21T11:45:22.144 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:21 smithi100 podman[73380]: 2024-03-21 11:45:21.993344578 +0000 UTC m=+0.039838647 container died d771cc7b5d6f1300df4a23364332a8b84cd50f211714daa2fb267c0ec8317e0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T11:45:22.978 INFO:tasks.workunit.client.0.smithi047.stdout: 10 208833 22.17 MB/sec execute 366 sec latency 155.401 ms 2024-03-21T11:45:23.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:23 smithi102 ceph-mon[36748]: pgmap v1061: 129 pgs: 129 active+clean; 659 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.5 MiB/s wr, 493 op/s 2024-03-21T11:45:23.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:23 smithi047 ceph-mon[31351]: pgmap v1061: 129 pgs: 129 active+clean; 659 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.5 MiB/s wr, 493 op/s 2024-03-21T11:45:23.593 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:23 smithi100 ceph-mon[36245]: pgmap v1061: 129 pgs: 129 active+clean; 659 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.5 MiB/s wr, 493 op/s 2024-03-21T11:45:23.978 INFO:tasks.workunit.client.0.smithi047.stdout: 10 209144 22.16 MB/sec execute 367 sec latency 180.879 ms 2024-03-21T11:45:24.978 INFO:tasks.workunit.client.0.smithi047.stdout: 10 209622 22.14 MB/sec execute 368 sec latency 105.798 ms 2024-03-21T11:45:25.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:25 smithi102 ceph-mon[36748]: pgmap v1062: 129 pgs: 129 active+clean; 649 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.0 MiB/s wr, 547 op/s 2024-03-21T11:45:25.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:25 smithi047 ceph-mon[31351]: pgmap v1062: 129 pgs: 129 active+clean; 649 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.0 MiB/s wr, 547 op/s 2024-03-21T11:45:25.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:25 smithi100 ceph-mon[36245]: pgmap v1062: 129 pgs: 129 active+clean; 649 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.0 MiB/s wr, 547 op/s 2024-03-21T11:45:25.978 INFO:tasks.workunit.client.0.smithi047.stdout: 10 209732 22.09 MB/sec execute 369 sec latency 505.894 ms 2024-03-21T11:45:26.978 INFO:tasks.workunit.client.0.smithi047.stdout: 10 210114 22.08 MB/sec execute 370 sec latency 607.228 ms 2024-03-21T11:45:27.106 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:26 smithi100 podman[73380]: 2024-03-21 11:45:26.655109983 +0000 UTC m=+4.701604033 container remove d771cc7b5d6f1300df4a23364332a8b84cd50f211714daa2fb267c0ec8317e0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-03-21T11:45:27.106 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:26 smithi100 systemd[1]: ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.e.service: Main process exited, code=exited, status=139/n/a 2024-03-21T11:45:27.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:27 smithi100 ceph-mon[36245]: pgmap v1063: 129 pgs: 129 active+clean; 629 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.0 MiB/s wr, 564 op/s 2024-03-21T11:45:27.393 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:27 smithi100 systemd[1]: ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.e.service: Failed with result 'exit-code'. 2024-03-21T11:45:27.393 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:27 smithi100 systemd[1]: ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.e.service: Consumed 1min 11.367s CPU time. 2024-03-21T11:45:27.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:27 smithi102 ceph-mon[36748]: pgmap v1063: 129 pgs: 129 active+clean; 629 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.0 MiB/s wr, 564 op/s 2024-03-21T11:45:27.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:27 smithi047 ceph-mon[31351]: pgmap v1063: 129 pgs: 129 active+clean; 629 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.0 MiB/s wr, 564 op/s 2024-03-21T11:45:27.978 INFO:tasks.workunit.client.0.smithi047.stdout: 10 210823 22.13 MB/sec execute 371 sec latency 88.760 ms 2024-03-21T11:45:28.978 INFO:tasks.workunit.client.0.smithi047.stdout: 10 211294 22.16 MB/sec execute 372 sec latency 70.116 ms 2024-03-21T11:45:29.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:29 smithi047 ceph-mon[31351]: pgmap v1064: 129 pgs: 129 active+clean; 632 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.1 MiB/s wr, 528 op/s 2024-03-21T11:45:29.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:29 smithi100 ceph-mon[36245]: pgmap v1064: 129 pgs: 129 active+clean; 632 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.1 MiB/s wr, 528 op/s 2024-03-21T11:45:29.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:29 smithi102 ceph-mon[36748]: pgmap v1064: 129 pgs: 129 active+clean; 632 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.1 MiB/s wr, 528 op/s 2024-03-21T11:45:29.978 INFO:tasks.workunit.client.0.smithi047.stdout: 10 211717 22.17 MB/sec execute 373 sec latency 69.976 ms 2024-03-21T11:45:30.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:30 smithi047 ceph-mon[31351]: pgmap v1065: 129 pgs: 129 active+clean; 635 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.0 MiB/s wr, 597 op/s 2024-03-21T11:45:30.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:30 smithi100 ceph-mon[36245]: pgmap v1065: 129 pgs: 129 active+clean; 635 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.0 MiB/s wr, 597 op/s 2024-03-21T11:45:30.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:30 smithi102 ceph-mon[36748]: pgmap v1065: 129 pgs: 129 active+clean; 635 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.0 MiB/s wr, 597 op/s 2024-03-21T11:45:30.980 INFO:tasks.workunit.client.0.smithi047.stdout: 10 212167 22.17 MB/sec execute 374 sec latency 52.624 ms 2024-03-21T11:45:31.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:31 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:45:31.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:31 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:45:31.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:31 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:45:31.980 INFO:tasks.workunit.client.0.smithi047.stdout: 10 212516 22.16 MB/sec execute 375 sec latency 74.225 ms 2024-03-21T11:45:32.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:32 smithi047 ceph-mon[31351]: pgmap v1066: 129 pgs: 129 active+clean; 637 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 463 op/s 2024-03-21T11:45:32.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:32 smithi100 ceph-mon[36245]: pgmap v1066: 129 pgs: 129 active+clean; 637 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 463 op/s 2024-03-21T11:45:32.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:32 smithi102 ceph-mon[36748]: pgmap v1066: 129 pgs: 129 active+clean; 637 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 463 op/s 2024-03-21T11:45:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.871+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2125375426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cc01054a0 msgr2=0x7f6cc010bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.871+0000 7f6cc5c00640 1 --2- 172.21.15.47:0/2125375426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cc01054a0 0x7f6cc010bd30 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009900 tx=0x7f6ca8034440 comp rx=0 tx=0).stop 2024-03-21T11:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.871+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2125375426 shutdown_connections 2024-03-21T11:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.871+0000 7f6cc5c00640 1 --2- 172.21.15.47:0/2125375426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cc01054a0 0x7f6cc010bd30 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.871+0000 7f6cc5c00640 1 --2- 172.21.15.47:0/2125375426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc0104ae0 0x7f6cc0104f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.871+0000 7f6cc5c00640 1 --2- 172.21.15.47:0/2125375426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc01038f0 0x7f6cc0103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.871+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2125375426 >> 172.21.15.47:0/2125375426 conn(0x7f6cc00fee60 msgr2=0x7f6cc01012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.877+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2125375426 shutdown_connections 2024-03-21T11:45:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.878+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2125375426 wait complete. 2024-03-21T11:45:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.879+0000 7f6cc5c00640 1 Processor -- start 2024-03-21T11:45:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.880+0000 7f6cc5c00640 1 -- start start 2024-03-21T11:45:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.881+0000 7f6cc5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc01038f0 0x7f6cc0195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.881+0000 7f6cc5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cc0104ae0 0x7f6cc0195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.882+0000 7f6cbea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc01038f0 0x7f6cc0195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.882+0000 7f6cbea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc01038f0 0x7f6cc0195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41646/0 (socket says 172.21.15.47:41646) 2024-03-21T11:45:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.882+0000 7f6cbea00640 1 -- 172.21.15.47:0/2655315119 learned_addr learned my addr 172.21.15.47:0/2655315119 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.882+0000 7f6cc5c00640 1 --2- 172.21.15.47:0/2655315119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc01054a0 0x7f6cc019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.882+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2655315119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc00786e0 con 0x7f6cc01054a0 2024-03-21T11:45:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.882+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2655315119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc0078560 con 0x7f6cc01038f0 2024-03-21T11:45:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.882+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2655315119 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc0078860 con 0x7f6cc0104ae0 2024-03-21T11:45:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.883+0000 7f6cbea00640 1 -- 172.21.15.47:0/2655315119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cc0104ae0 msgr2=0x7f6cc0195bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:45:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.883+0000 7f6cbea00640 1 --2- 172.21.15.47:0/2655315119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cc0104ae0 0x7f6cc0195bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.883+0000 7f6cbea00640 1 -- 172.21.15.47:0/2655315119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc01054a0 msgr2=0x7f6cc019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.884+0000 7f6cbea00640 1 --2- 172.21.15.47:0/2655315119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc01054a0 0x7f6cc019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.884+0000 7f6cbea00640 1 -- 172.21.15.47:0/2655315119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc019d2b0 con 0x7f6cc01038f0 2024-03-21T11:45:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.884+0000 7f6cbea00640 1 --2- 172.21.15.47:0/2655315119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc01038f0 0x7f6cc0195690 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000d770 tx=0x7f6cb000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.886+0000 7f6cb7400640 1 -- 172.21.15.47:0/2655315119 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0017070 con 0x7f6cc01038f0 2024-03-21T11:45:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.887+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2655315119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc019d5a0 con 0x7f6cc01038f0 2024-03-21T11:45:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.887+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2655315119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc019db40 con 0x7f6cc01038f0 2024-03-21T11:45:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.891+0000 7f6cb7400640 1 -- 172.21.15.47:0/2655315119 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cb0009c30 con 0x7f6cc01038f0 2024-03-21T11:45:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.891+0000 7f6cb7400640 1 -- 172.21.15.47:0/2655315119 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0003dd0 con 0x7f6cc01038f0 2024-03-21T11:45:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.894+0000 7f6cb7400640 1 -- 172.21.15.47:0/2655315119 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6cb0010460 con 0x7f6cc01038f0 2024-03-21T11:45:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.896+0000 7f6cb7400640 1 --2- 172.21.15.47:0/2655315119 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ca40a3040 0x7f6ca40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.896+0000 7f6cb7400640 1 -- 172.21.15.47:0/2655315119 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6cb00d0230 con 0x7f6cc01038f0 2024-03-21T11:45:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.897+0000 7f6cbe000640 1 --2- 172.21.15.47:0/2655315119 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ca40a3040 0x7f6ca40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.904+0000 7f6cbe000640 1 --2- 172.21.15.47:0/2655315119 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ca40a3040 0x7f6ca40a5500 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0196ba0 tx=0x7f6cb80096f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.907+0000 7f6cc89c5740 1 Processor -- start 2024-03-21T11:45:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.907+0000 7f6cc89c5740 1 -- start start 2024-03-21T11:45:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.908+0000 7f6cc89c5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1b1e0 0x560aa8e1b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.908+0000 7f6cc89c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e1bbd0 0x560aa8e206d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.908+0000 7f6cc89c5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e231d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.908+0000 7f6cc89c5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa8dde3b0 con 0x560aa8e1b1e0 2024-03-21T11:45:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.908+0000 7f6cc89c5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa8d80830 con 0x560aa8e20d40 2024-03-21T11:45:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.908+0000 7f6cc89c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa8d295d0 con 0x560aa8e1bbd0 2024-03-21T11:45:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.908+0000 7f6cbf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e231d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.908+0000 7f6cbf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e231d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41656/0 (socket says 172.21.15.47:41656) 2024-03-21T11:45:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.908+0000 7f6cbf400640 1 -- 172.21.15.47:0/4104819229 learned_addr learned my addr 172.21.15.47:0/4104819229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.908+0000 7f6cbea00640 1 --2- 172.21.15.47:0/4104819229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1b1e0 0x560aa8e1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.909+0000 7f6cbea00640 1 -- 172.21.15.47:0/4104819229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e1bbd0 msgr2=0x560aa8e206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.909+0000 7f6cbea00640 1 --2- 172.21.15.47:0/4104819229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e1bbd0 0x560aa8e206d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.909+0000 7f6cbea00640 1 -- 172.21.15.47:0/4104819229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 msgr2=0x560aa8e231d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.909+0000 7f6cbea00640 1 --2- 172.21.15.47:0/4104819229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e231d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.909+0000 7f6cbea00640 1 -- 172.21.15.47:0/4104819229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560aa8d412c0 con 0x560aa8e1b1e0 2024-03-21T11:45:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.909+0000 7f6cbf400640 1 --2- 172.21.15.47:0/4104819229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e231d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.910+0000 7f6cbea00640 1 --2- 172.21.15.47:0/4104819229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1b1e0 0x560aa8e1b600 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00cd940 tx=0x7f6cb009cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab5b3b5537f5435 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.911+0000 7f6cafe00640 1 -- 172.21.15.47:0/4104819229 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0020070 con 0x560aa8e1b1e0 2024-03-21T11:45:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.912+0000 7f6cafe00640 1 -- 172.21.15.47:0/4104819229 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cb009c570 con 0x560aa8e1b1e0 2024-03-21T11:45:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.912+0000 7f6cafe00640 1 -- 172.21.15.47:0/4104819229 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0098be0 con 0x560aa8e1b1e0 2024-03-21T11:45:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.916+0000 7f6cc89c5740 1 -- 172.21.15.47:0/4104819229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1b1e0 msgr2=0x560aa8e1b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.916+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/4104819229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1b1e0 0x560aa8e1b600 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00cd940 tx=0x7f6cb009cc70 comp rx=0 tx=0).stop 2024-03-21T11:45:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.916+0000 7f6cc89c5740 1 -- 172.21.15.47:0/4104819229 shutdown_connections 2024-03-21T11:45:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.916+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/4104819229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e231d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.916+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/4104819229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e1bbd0 0x560aa8e206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.916+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/4104819229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1b1e0 0x560aa8e1b600 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.916+0000 7f6cc89c5740 1 -- 172.21.15.47:0/4104819229 >> 172.21.15.47:0/4104819229 conn(0x560aa8e16750 msgr2=0x560aa8e16b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.916+0000 7f6cc89c5740 1 -- 172.21.15.47:0/4104819229 shutdown_connections 2024-03-21T11:45:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.916+0000 7f6cc89c5740 1 -- 172.21.15.47:0/4104819229 wait complete. 2024-03-21T11:45:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.917+0000 7f6cc89c5740 1 -- start start 2024-03-21T11:45:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.917+0000 7f6cc89c5740 1 Processor -- start 2024-03-21T11:45:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.921+0000 7f6cc89c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e1bbd0 0x560aa8e366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.921+0000 7f6cc89c5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e36be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.921+0000 7f6cc89c5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e3bc10 0x560aa8e3e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.922+0000 7f6cc89c5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa8d3ca20 con 0x560aa8e3bc10 2024-03-21T11:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.922+0000 7f6cbea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e1bbd0 0x560aa8e366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.922+0000 7f6cc89c5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa8d7c8d0 con 0x560aa8e20d40 2024-03-21T11:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.922+0000 7f6cc89c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa8e34d00 con 0x560aa8e1bbd0 2024-03-21T11:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.922+0000 7f6cbea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e1bbd0 0x560aa8e366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47050/0 (socket says 172.21.15.47:47050) 2024-03-21T11:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.922+0000 7f6cbea00640 1 -- 172.21.15.47:0/1748259813 learned_addr learned my addr 172.21.15.47:0/1748259813 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.922+0000 7f6cbe000640 1 --2- 172.21.15.47:0/1748259813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e36be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.922+0000 7f6cc89c5740 10 client.0 _ll_register_callbacks cb 0x560aa8d791c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.922+0000 7f6cbf400640 1 --2- 172.21.15.47:0/1748259813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e3bc10 0x560aa8e3e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.923+0000 7f6cbf400640 1 -- 172.21.15.47:0/1748259813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e1bbd0 msgr2=0x560aa8e366a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.923+0000 7f6cbf400640 1 --2- 172.21.15.47:0/1748259813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e1bbd0 0x560aa8e366a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.923+0000 7f6cbf400640 1 -- 172.21.15.47:0/1748259813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 msgr2=0x560aa8e36be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.923+0000 7f6cbf400640 1 --2- 172.21.15.47:0/1748259813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e36be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.923+0000 7f6cbf400640 1 -- 172.21.15.47:0/1748259813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560aa8a95210 con 0x560aa8e3bc10 2024-03-21T11:45:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.923+0000 7f6cbe000640 1 --2- 172.21.15.47:0/1748259813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e36be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:45:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.924+0000 7f6cbf400640 1 --2- 172.21.15.47:0/1748259813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e3bc10 0x560aa8e3e000 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x560aa8e176c0 tx=0x7f6ca8005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22c615836a8fca server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.924+0000 7f6cacc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:45:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.925+0000 7f6cacc00640 1 -- 172.21.15.47:0/1748259813 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8048060 con 0x560aa8e3bc10 2024-03-21T11:45:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.925+0000 7f6cacc00640 1 -- 172.21.15.47:0/1748259813 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6ca8004310 con 0x560aa8e3bc10 2024-03-21T11:45:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.925+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1748259813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cb009c570 con 0x560aa8e3bc10 2024-03-21T11:45:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.926+0000 7f6cacc00640 1 -- 172.21.15.47:0/1748259813 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ca804f050 con 0x560aa8e3bc10 2024-03-21T11:45:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.927+0000 7f6cc89c5740 10 client.21795 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.927+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1748259813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560aa8b61620 con 0x560aa8e3bc10 2024-03-21T11:45:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.928+0000 7f6cacc00640 1 -- 172.21.15.47:0/1748259813 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cb009c570 con 0x560aa8e3bc10 2024-03-21T11:45:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.928+0000 7f6cacc00640 1 -- 172.21.15.47:0/1748259813 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6ca8004b30 con 0x560aa8e3bc10 2024-03-21T11:45:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.928+0000 7f6cc89c5740 10 client.21795 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.928+0000 7f6cc89c5740 10 client.21795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.928+0000 7f6cc89c5740 20 client.21795 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.928+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1748259813 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560aa8e428e0 0x560aa8e44d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.928+0000 7f6cc89c5740 4 client.21795 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.928+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1748259813 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560aa8d8dd70 con 0x560aa8e428e0 2024-03-21T11:45:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.929+0000 7f6cbea00640 1 --2- 172.21.15.47:0/1748259813 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560aa8e428e0 0x560aa8e44d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.930+0000 7f6cacc00640 10 client.21795 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.931+0000 7f6cbea00640 1 --2- 172.21.15.47:0/1748259813 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560aa8e428e0 0x560aa8e44d00 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c48bb4dc4e6cca server_cookie=8212818791a418a2 in_seq=0 out_seq=0 2024-03-21T11:45:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.937+0000 7f6cacc00640 1 -- 172.21.15.47:0/1748259813 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560aa8d8dd70 con 0x560aa8e428e0 2024-03-21T11:45:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.937+0000 7f6cacc00640 10 client.21795 handle_command_reply: tid=0 2024-03-21T11:45:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.938+0000 7f6cc89c5740 1 client.21795 shutdown 2024-03-21T11:45:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.939+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1748259813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e3bc10 msgr2=0x560aa8e3e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.941+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1748259813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e3bc10 0x560aa8e3e000 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x560aa8e176c0 tx=0x7f6ca8005590 comp rx=0 tx=0).stop 2024-03-21T11:45:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.942+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1748259813 shutdown_connections 2024-03-21T11:45:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.942+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1748259813 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560aa8e428e0 0x560aa8e44d00 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.942+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1748259813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e3bc10 0x560aa8e3e000 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.942+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1748259813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e36be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.942+0000 7f6cacc00640 0 client.21795 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.942+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1748259813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e1bbd0 0x560aa8e366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.942+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1748259813 >> 172.21.15.47:0/1748259813 conn(0x560aa8e16750 msgr2=0x560aa8e15950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.943+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1748259813 shutdown_connections 2024-03-21T11:45:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.943+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1748259813 wait complete. 2024-03-21T11:45:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.943+0000 7f6cc89c5740 20 client.21795 trim_cache size 0 max 16384 2024-03-21T11:45:32.980 INFO:tasks.workunit.client.0.smithi047.stdout: 10 212962 22.17 MB/sec execute 376 sec latency 131.085 ms 2024-03-21T11:45:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.980+0000 7f6cc89c5740 1 Processor -- start 2024-03-21T11:45:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.980+0000 7f6cc89c5740 1 -- start start 2024-03-21T11:45:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.981+0000 7f6cc89c5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1bbd0 0x560aa8e366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.981+0000 7f6cc89c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e20d40 0x560aa8e36be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.981+0000 7f6cc89c5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e3bc10 0x560aa8e35100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.981+0000 7f6cc89c5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c00f940 con 0x560aa8e1bbd0 2024-03-21T11:45:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.981+0000 7f6cc89c5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c0112c0 con 0x560aa8e3bc10 2024-03-21T11:45:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.981+0000 7f6cc89c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c00ffa0 con 0x560aa8e20d40 2024-03-21T11:45:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.982+0000 7f6cbe000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e20d40 0x560aa8e36be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.982+0000 7f6cbe000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e20d40 0x560aa8e36be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47064/0 (socket says 172.21.15.47:47064) 2024-03-21T11:45:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.982+0000 7f6cbe000640 1 -- 172.21.15.47:0/1608168060 learned_addr learned my addr 172.21.15.47:0/1608168060 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.982+0000 7f6cbea00640 1 --2- 172.21.15.47:0/1608168060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1bbd0 0x560aa8e366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.983+0000 7f6cbe000640 1 -- 172.21.15.47:0/1608168060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e3bc10 msgr2=0x560aa8e35100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.983+0000 7f6cbe000640 1 --2- 172.21.15.47:0/1608168060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e3bc10 0x560aa8e35100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.983+0000 7f6cbe000640 1 -- 172.21.15.47:0/1608168060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1bbd0 msgr2=0x560aa8e366a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.983+0000 7f6cbe000640 1 --2- 172.21.15.47:0/1608168060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1bbd0 0x560aa8e366a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.983+0000 7f6cbe000640 1 -- 172.21.15.47:0/1608168060 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560aa8e20450 con 0x560aa8e20d40 2024-03-21T11:45:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.983+0000 7f6cbea00640 1 --2- 172.21.15.47:0/1608168060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1bbd0 0x560aa8e366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:45:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.983+0000 7f6cbe000640 1 --2- 172.21.15.47:0/1608168060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e20d40 0x560aa8e36be0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009fd0 tx=0x7f6cb8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c69b0974beaf117 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.984+0000 7f6caf400640 1 -- 172.21.15.47:0/1608168060 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8021070 con 0x560aa8e20d40 2024-03-21T11:45:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.984+0000 7f6caf400640 1 -- 172.21.15.47:0/1608168060 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cb8016030 con 0x560aa8e20d40 2024-03-21T11:45:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.984+0000 7f6caf400640 1 -- 172.21.15.47:0/1608168060 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb800f7a0 con 0x560aa8e20d40 2024-03-21T11:45:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.985+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1608168060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e20d40 msgr2=0x560aa8e36be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.985+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1608168060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e20d40 0x560aa8e36be0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009fd0 tx=0x7f6cb8004a40 comp rx=0 tx=0).stop 2024-03-21T11:45:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.986+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1608168060 shutdown_connections 2024-03-21T11:45:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.986+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1608168060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e3bc10 0x560aa8e35100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.986+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1608168060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e20d40 0x560aa8e36be0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.986+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1608168060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1bbd0 0x560aa8e366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.986+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1608168060 >> 172.21.15.47:0/1608168060 conn(0x560aa8e16750 msgr2=0x560aa8e16b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.987+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1608168060 shutdown_connections 2024-03-21T11:45:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.987+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1608168060 wait complete. 2024-03-21T11:45:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.988+0000 7f6cc89c5740 1 -- start start 2024-03-21T11:45:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.988+0000 7f6cc89c5740 1 Processor -- start 2024-03-21T11:45:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.989+0000 7f6cc89c5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1bbd0 0x560aa8e9a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.990+0000 7f6cc89c5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e9acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.990+0000 7f6cc89c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e3bc10 0x560aa8e54510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.990+0000 7f6cc89c5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c010c60 con 0x560aa8e1bbd0 2024-03-21T11:45:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.990+0000 7f6cc89c5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c010600 con 0x560aa8e20d40 2024-03-21T11:45:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.990+0000 7f6cc89c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c00db00 con 0x560aa8e3bc10 2024-03-21T11:45:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.990+0000 7f6cc89c5740 10 client.0 _ll_register_callbacks cb 0x7f6ca8005220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.990+0000 7f6cbf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e3bc10 0x560aa8e54510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.990+0000 7f6cbf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e3bc10 0x560aa8e54510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47078/0 (socket says 172.21.15.47:47078) 2024-03-21T11:45:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.990+0000 7f6cbf400640 1 -- 172.21.15.47:0/1784181549 learned_addr learned my addr 172.21.15.47:0/1784181549 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.991+0000 7f6cbe000640 1 --2- 172.21.15.47:0/1784181549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e9acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.991+0000 7f6cbea00640 1 --2- 172.21.15.47:0/1784181549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1bbd0 0x560aa8e9a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.992+0000 7f6cbf400640 1 -- 172.21.15.47:0/1784181549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 msgr2=0x560aa8e9acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.992+0000 7f6cbf400640 1 --2- 172.21.15.47:0/1784181549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e9acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.992+0000 7f6cbf400640 1 -- 172.21.15.47:0/1784181549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1bbd0 msgr2=0x560aa8e9a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.992+0000 7f6cbf400640 1 --2- 172.21.15.47:0/1784181549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1bbd0 0x560aa8e9a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.992+0000 7f6cbf400640 1 -- 172.21.15.47:0/1784181549 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560aa8e54bb0 con 0x560aa8e3bc10 2024-03-21T11:45:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.992+0000 7f6cbf400640 1 --2- 172.21.15.47:0/1784181549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e3bc10 0x560aa8e54510 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009850 tx=0x7f6ca80052d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ecdb12da920228c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.993+0000 7f6cacc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.993+0000 7f6cacc00640 1 -- 172.21.15.47:0/1784181549 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8048050 con 0x560aa8e3bc10 2024-03-21T11:45:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.993+0000 7f6cacc00640 1 -- 172.21.15.47:0/1784181549 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6ca8003a50 con 0x560aa8e3bc10 2024-03-21T11:45:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.994+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1784181549 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cb8016030 con 0x560aa8e3bc10 2024-03-21T11:45:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.995+0000 7f6cacc00640 1 -- 172.21.15.47:0/1784181549 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ca8061050 con 0x560aa8e3bc10 2024-03-21T11:45:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.996+0000 7f6cc89c5740 10 client.32524 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.996+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1784181549 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560aa8eaf100 con 0x560aa8e3bc10 2024-03-21T11:45:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.997+0000 7f6cacc00640 1 -- 172.21.15.47:0/1784181549 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cb8016030 con 0x560aa8e3bc10 2024-03-21T11:45:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.997+0000 7f6cacc00640 1 -- 172.21.15.47:0/1784181549 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6ca8003c50 con 0x560aa8e3bc10 2024-03-21T11:45:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.998+0000 7f6cc89c5740 10 client.32524 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.998+0000 7f6cc89c5740 10 client.32524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.998+0000 7f6cc89c5740 20 client.32524 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.998+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1784181549 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560aa8e428e0 0x560aa8eb1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.998+0000 7f6cc89c5740 4 client.32524 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:45:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.998+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1784181549 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560aa8d880c0 con 0x560aa8e428e0 2024-03-21T11:45:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:32.999+0000 7f6cbea00640 1 --2- 172.21.15.47:0/1784181549 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560aa8e428e0 0x560aa8eb1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.000+0000 7f6cbea00640 1 --2- 172.21.15.47:0/1784181549 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560aa8e428e0 0x560aa8eb1a90 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f4803cbe4fc91a server_cookie=cd5eba0fd69b5bc6 in_seq=0 out_seq=0 2024-03-21T11:45:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.000+0000 7f6cacc00640 10 client.32524 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.663+0000 7f6cacc00640 1 -- 172.21.15.47:0/1784181549 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560aa8d880c0 con 0x560aa8e428e0 2024-03-21T11:45:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.663+0000 7f6cacc00640 10 client.32524 handle_command_reply: tid=0 2024-03-21T11:45:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.663+0000 7f6cc89c5740 1 client.32524 shutdown 2024-03-21T11:45:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.664+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1784181549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e3bc10 msgr2=0x560aa8e54510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.664+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1784181549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e3bc10 0x560aa8e54510 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009850 tx=0x7f6ca80052d0 comp rx=0 tx=0).stop 2024-03-21T11:45:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.664+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1784181549 shutdown_connections 2024-03-21T11:45:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.664+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1784181549 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x560aa8e428e0 0x560aa8eb1a90 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.664+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1784181549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560aa8e3bc10 0x560aa8e54510 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.664+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1784181549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560aa8e20d40 0x560aa8e9acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.664+0000 7f6cc89c5740 1 --2- 172.21.15.47:0/1784181549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560aa8e1bbd0 0x560aa8e9a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.664+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1784181549 >> 172.21.15.47:0/1784181549 conn(0x560aa8e16750 msgr2=0x560aa8e36660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.664+0000 7f6cacc00640 0 client.32524 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.664+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1784181549 shutdown_connections 2024-03-21T11:45:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.664+0000 7f6cc89c5740 1 -- 172.21.15.47:0/1784181549 wait complete. 2024-03-21T11:45:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.665+0000 7f6cc89c5740 20 client.32524 trim_cache size 0 max 16384 2024-03-21T11:45:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:45:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:45:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:45:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:45:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.667+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2655315119 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ca40a3040 msgr2=0x7f6ca40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.667+0000 7f6cc5c00640 1 --2- 172.21.15.47:0/2655315119 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ca40a3040 0x7f6ca40a5500 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0196ba0 tx=0x7f6cb80096f0 comp rx=0 tx=0).stop 2024-03-21T11:45:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.667+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2655315119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc01038f0 msgr2=0x7f6cc0195690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.667+0000 7f6cc5c00640 1 --2- 172.21.15.47:0/2655315119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc01038f0 0x7f6cc0195690 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000d770 tx=0x7f6cb000dc30 comp rx=0 tx=0).stop 2024-03-21T11:45:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.668+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2655315119 shutdown_connections 2024-03-21T11:45:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.668+0000 7f6cc5c00640 1 --2- 172.21.15.47:0/2655315119 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ca40a3040 0x7f6ca40a5500 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.668+0000 7f6cc5c00640 1 --2- 172.21.15.47:0/2655315119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cc01054a0 0x7f6cc019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.668+0000 7f6cc5c00640 1 --2- 172.21.15.47:0/2655315119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cc0104ae0 0x7f6cc0195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.668+0000 7f6cc5c00640 1 --2- 172.21.15.47:0/2655315119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc01038f0 0x7f6cc0195690 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.668+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2655315119 >> 172.21.15.47:0/2655315119 conn(0x7f6cc00fee60 msgr2=0x7f6cc00ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.668+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2655315119 shutdown_connections 2024-03-21T11:45:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:33.668+0000 7f6cc5c00640 1 -- 172.21.15.47:0/2655315119 wait complete. 2024-03-21T11:45:33.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:45:33.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:45:33.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:45:33.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:45:33.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:45:33.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:45:33.980 INFO:tasks.workunit.client.0.smithi047.stdout: 10 213214 22.14 MB/sec execute 377 sec latency 475.321 ms 2024-03-21T11:45:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:45:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:45:33.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:45:34.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:34 smithi047 ceph-mon[31351]: pgmap v1067: 129 pgs: 129 active+clean; 637 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 484 op/s 2024-03-21T11:45:34.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:34 smithi100 ceph-mon[36245]: pgmap v1067: 129 pgs: 129 active+clean; 637 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 484 op/s 2024-03-21T11:45:34.980 INFO:tasks.workunit.client.0.smithi047.stdout: 10 213786 22.16 MB/sec execute 378 sec latency 56.537 ms 2024-03-21T11:45:34.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:34 smithi102 ceph-mon[36748]: pgmap v1067: 129 pgs: 129 active+clean; 637 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 484 op/s 2024-03-21T11:45:35.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:45:35.433 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:45:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.707+0000 7f5179a00640 1 -- 172.21.15.47:0/2847839556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5174102130 msgr2=0x7f5174104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.707+0000 7f5179a00640 1 --2- 172.21.15.47:0/2847839556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5174102130 0x7f5174104520 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5164009900 tx=0x7f516402bc30 comp rx=0 tx=0).stop 2024-03-21T11:45:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.708+0000 7f5179a00640 1 -- 172.21.15.47:0/2847839556 shutdown_connections 2024-03-21T11:45:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.708+0000 7f5179a00640 1 --2- 172.21.15.47:0/2847839556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5174107390 0x7f51741097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.708+0000 7f5179a00640 1 --2- 172.21.15.47:0/2847839556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5174104a60 0x7f5174106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.708+0000 7f5179a00640 1 --2- 172.21.15.47:0/2847839556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5174102130 0x7f5174104520 secure :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5164009900 tx=0x7f516402bc30 comp rx=0 tx=0).stop 2024-03-21T11:45:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.708+0000 7f5179a00640 1 -- 172.21.15.47:0/2847839556 >> 172.21.15.47:0/2847839556 conn(0x7f51740fbb40 msgr2=0x7f51740fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.708+0000 7f5179a00640 1 -- 172.21.15.47:0/2847839556 shutdown_connections 2024-03-21T11:45:35.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.708+0000 7f5179a00640 1 -- 172.21.15.47:0/2847839556 wait complete. 2024-03-21T11:45:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.709+0000 7f5179a00640 1 Processor -- start 2024-03-21T11:45:35.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.709+0000 7f5179a00640 1 -- start start 2024-03-21T11:45:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.710+0000 7f5179a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5174104a60 0x7f51741959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.710+0000 7f5179a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5174107390 0x7f5174195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.710+0000 7f5179a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51741a3730 0x7f51741a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.710+0000 7f5179a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517410d380 con 0x7f5174104a60 2024-03-21T11:45:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.710+0000 7f5179a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517410d200 con 0x7f51741a3730 2024-03-21T11:45:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.710+0000 7f5179a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517410d500 con 0x7f5174107390 2024-03-21T11:45:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.710+0000 7f5172a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5174104a60 0x7f51741959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.710+0000 7f5172a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5174104a60 0x7f51741959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35340/0 (socket says 172.21.15.47:35340) 2024-03-21T11:45:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.710+0000 7f5172a00640 1 -- 172.21.15.47:0/4256104542 learned_addr learned my addr 172.21.15.47:0/4256104542 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.710+0000 7f5172000640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5174107390 0x7f5174195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.711+0000 7f5173400640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51741a3730 0x7f51741a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.711+0000 7f5172000640 1 -- 172.21.15.47:0/4256104542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51741a3730 msgr2=0x7f51741a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.711+0000 7f5172000640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51741a3730 0x7f51741a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.711+0000 7f5172000640 1 -- 172.21.15.47:0/4256104542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5174104a60 msgr2=0x7f51741959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.711+0000 7f5172000640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5174104a60 0x7f51741959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.711+0000 7f5172000640 1 -- 172.21.15.47:0/4256104542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51741a61c0 con 0x7f5174107390 2024-03-21T11:45:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.711+0000 7f5172a00640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5174104a60 0x7f51741959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.712+0000 7f5172000640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5174107390 0x7f5174195f00 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f516000c940 tx=0x7f516000ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.712+0000 7f516b400640 1 -- 172.21.15.47:0/4256104542 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160007bb0 con 0x7f5174107390 2024-03-21T11:45:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.712+0000 7f516b400640 1 -- 172.21.15.47:0/4256104542 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5160007d50 con 0x7f5174107390 2024-03-21T11:45:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.712+0000 7f516b400640 1 -- 172.21.15.47:0/4256104542 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160018860 con 0x7f5174107390 2024-03-21T11:45:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.713+0000 7f5179a00640 1 -- 172.21.15.47:0/4256104542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51741a64b0 con 0x7f5174107390 2024-03-21T11:45:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.713+0000 7f5179a00640 1 -- 172.21.15.47:0/4256104542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51741a6980 con 0x7f5174107390 2024-03-21T11:45:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.717+0000 7f517c61f740 1 Processor -- start 2024-03-21T11:45:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.718+0000 7f517c61f740 1 -- start start 2024-03-21T11:45:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.718+0000 7f517c61f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f21773ee80 0x55f21773f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.718+0000 7f517c61f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f217744370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.718+0000 7f517c61f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2177449e0 0x55f217746e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.718+0000 7f517c61f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2176a4250 con 0x55f2177449e0 2024-03-21T11:45:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.718+0000 7f517c61f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f21764ca30 con 0x55f21773f870 2024-03-21T11:45:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.718+0000 7f517c61f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2176a02f0 con 0x55f21773ee80 2024-03-21T11:45:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.719+0000 7f5173400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f21773ee80 0x55f21773f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.719+0000 7f5172a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f217744370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.719+0000 7f5173400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f21773ee80 0x55f21773f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47112/0 (socket says 172.21.15.47:47112) 2024-03-21T11:45:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.719+0000 7f5173400640 1 -- 172.21.15.47:0/179059673 learned_addr learned my addr 172.21.15.47:0/179059673 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.719+0000 7f5172a00640 1 -- 172.21.15.47:0/179059673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f21773ee80 msgr2=0x55f21773f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.719+0000 7f5172a00640 1 --2- 172.21.15.47:0/179059673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f21773ee80 0x55f21773f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.719+0000 7f5172a00640 1 -- 172.21.15.47:0/179059673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2177449e0 msgr2=0x55f217746e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:45:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.719+0000 7f5172a00640 1 --2- 172.21.15.47:0/179059673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2177449e0 0x55f217746e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.719+0000 7f5172a00640 1 -- 172.21.15.47:0/179059673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f217637690 con 0x55f21773f870 2024-03-21T11:45:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.719+0000 7f516b400640 1 -- 172.21.15.47:0/4256104542 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5160018a00 con 0x7f5174107390 2024-03-21T11:45:35.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.720+0000 7f5172a00640 1 --2- 172.21.15.47:0/179059673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f217744370 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f516402c240 tx=0x7f5164005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a8d5a87c2318944 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:35.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.721+0000 7f515fe00640 1 -- 172.21.15.47:0/179059673 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5164045070 con 0x55f21773f870 2024-03-21T11:45:35.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.721+0000 7f515fe00640 1 -- 172.21.15.47:0/179059673 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51640046c0 con 0x55f21773f870 2024-03-21T11:45:35.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.721+0000 7f515fe00640 1 -- 172.21.15.47:0/179059673 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5164005170 con 0x55f21773f870 2024-03-21T11:45:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.722+0000 7f516b400640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f51580a2fa0 0x7f51580a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.722+0000 7f517c61f740 1 -- 172.21.15.47:0/179059673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 msgr2=0x55f217744370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.722+0000 7f517c61f740 1 --2- 172.21.15.47:0/179059673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f217744370 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f516402c240 tx=0x7f5164005e00 comp rx=0 tx=0).stop 2024-03-21T11:45:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.722+0000 7f516b400640 1 -- 172.21.15.47:0/4256104542 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f51600cf7f0 con 0x7f5174107390 2024-03-21T11:45:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.722+0000 7f5173400640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f51580a2fa0 0x7f51580a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.722+0000 7f517c61f740 1 -- 172.21.15.47:0/179059673 shutdown_connections 2024-03-21T11:45:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.722+0000 7f517c61f740 1 --2- 172.21.15.47:0/179059673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2177449e0 0x55f217746e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.722+0000 7f517c61f740 1 --2- 172.21.15.47:0/179059673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f217744370 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.722+0000 7f517c61f740 1 --2- 172.21.15.47:0/179059673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f21773ee80 0x55f21773f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.722+0000 7f517c61f740 1 -- 172.21.15.47:0/179059673 >> 172.21.15.47:0/179059673 conn(0x55f217707200 msgr2=0x55f21773a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.723+0000 7f517c61f740 1 -- 172.21.15.47:0/179059673 shutdown_connections 2024-03-21T11:45:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.723+0000 7f517c61f740 1 -- 172.21.15.47:0/179059673 wait complete. 2024-03-21T11:45:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.723+0000 7f517c61f740 1 -- start start 2024-03-21T11:45:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.723+0000 7f517c61f740 1 Processor -- start 2024-03-21T11:45:35.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.724+0000 7f5173400640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f51580a2fa0 0x7f51580a5460 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f516c005bd0 tx=0x7f516c0097e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:35.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.725+0000 7f517c61f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773ee80 0x55f217748e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.725+0000 7f517c61f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f2177493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.725+0000 7f517c61f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f217750390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.725+0000 7f517c61f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f217747930 con 0x55f21773ee80 2024-03-21T11:45:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.725+0000 7f517c61f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f21765fe80 con 0x55f21773f870 2024-03-21T11:45:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.725+0000 7f517c61f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f217747ab0 con 0x55f2177449e0 2024-03-21T11:45:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.725+0000 7f517c61f740 10 client.0 _ll_register_callbacks cb 0x55f217672130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.726+0000 7f5172000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f2177493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.726+0000 7f5172a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773ee80 0x55f217748e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.726+0000 7f5172000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f2177493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41736/0 (socket says 172.21.15.47:41736) 2024-03-21T11:45:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.726+0000 7f5172000640 1 -- 172.21.15.47:0/2381246676 learned_addr learned my addr 172.21.15.47:0/2381246676 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.726+0000 7f5173400640 1 --2- 172.21.15.47:0/2381246676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f217750390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.726+0000 7f5172000640 1 -- 172.21.15.47:0/2381246676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 msgr2=0x55f217750390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:35.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.726+0000 7f5172000640 1 --2- 172.21.15.47:0/2381246676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f217750390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.726+0000 7f5172000640 1 -- 172.21.15.47:0/2381246676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773ee80 msgr2=0x55f217748e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.726+0000 7f5172000640 1 --2- 172.21.15.47:0/2381246676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773ee80 0x55f217748e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.726+0000 7f5172000640 1 -- 172.21.15.47:0/2381246676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2174827d0 con 0x55f21773f870 2024-03-21T11:45:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.726+0000 7f5172a00640 1 --2- 172.21.15.47:0/2381246676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773ee80 0x55f217748e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:45:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.726+0000 7f5172000640 1 --2- 172.21.15.47:0/2381246676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f2177493c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f5160097f00 tx=0x7f516009b490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d77f26c45baad47 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.726+0000 7f515cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:45:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.727+0000 7f515cc00640 1 -- 172.21.15.47:0/2381246676 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51600b4e60 con 0x55f21773f870 2024-03-21T11:45:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.727+0000 7f515cc00640 1 -- 172.21.15.47:0/2381246676 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f516009af80 con 0x55f21773f870 2024-03-21T11:45:35.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.727+0000 7f517c61f740 1 -- 172.21.15.47:0/2381246676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51640046c0 con 0x55f21773f870 2024-03-21T11:45:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.728+0000 7f515cc00640 1 -- 172.21.15.47:0/2381246676 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f516009ce60 con 0x55f21773f870 2024-03-21T11:45:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.728+0000 7f517c61f740 10 client.32558 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:35.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.728+0000 7f517c61f740 1 -- 172.21.15.47:0/2381246676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f2173b83f0 con 0x55f21773f870 2024-03-21T11:45:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.728+0000 7f515cc00640 1 -- 172.21.15.47:0/2381246676 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51640046c0 con 0x55f21773f870 2024-03-21T11:45:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.728+0000 7f515cc00640 1 -- 172.21.15.47:0/2381246676 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5160020020 con 0x55f21773f870 2024-03-21T11:45:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.728+0000 7f517c61f740 10 client.32558 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.728+0000 7f517c61f740 10 client.32558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.728+0000 7f517c61f740 20 client.32558 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.728+0000 7f517c61f740 1 --2- 172.21.15.47:0/2381246676 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f217754a50 0x55f217756e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.728+0000 7f517c61f740 4 client.32558 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.728+0000 7f517c61f740 1 -- 172.21.15.47:0/2381246676 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2176b1790 con 0x55f217754a50 2024-03-21T11:45:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.729+0000 7f5172a00640 1 --2- 172.21.15.47:0/2381246676 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f217754a50 0x55f217756e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.729+0000 7f5172a00640 1 --2- 172.21.15.47:0/2381246676 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f217754a50 0x55f217756e80 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60422ce13019c705 server_cookie=3a82961dbc917dfd in_seq=0 out_seq=0 2024-03-21T11:45:35.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.729+0000 7f515cc00640 10 client.32558 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:35.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.735+0000 7f515cc00640 1 -- 172.21.15.47:0/2381246676 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f2176b1790 con 0x55f217754a50 2024-03-21T11:45:35.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.735+0000 7f515cc00640 10 client.32558 handle_command_reply: tid=0 2024-03-21T11:45:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.735+0000 7f517c61f740 1 client.32558 shutdown 2024-03-21T11:45:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.735+0000 7f517c61f740 1 -- 172.21.15.47:0/2381246676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 msgr2=0x55f2177493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.735+0000 7f517c61f740 1 --2- 172.21.15.47:0/2381246676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f2177493c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f5160097f00 tx=0x7f516009b490 comp rx=0 tx=0).stop 2024-03-21T11:45:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.735+0000 7f517c61f740 1 -- 172.21.15.47:0/2381246676 shutdown_connections 2024-03-21T11:45:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.735+0000 7f517c61f740 1 --2- 172.21.15.47:0/2381246676 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f217754a50 0x55f217756e80 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.735+0000 7f517c61f740 1 --2- 172.21.15.47:0/2381246676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f217750390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.735+0000 7f517c61f740 1 --2- 172.21.15.47:0/2381246676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f2177493c0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.735+0000 7f517c61f740 1 --2- 172.21.15.47:0/2381246676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773ee80 0x55f217748e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.735+0000 7f515cc00640 0 client.32558 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.735+0000 7f517c61f740 1 -- 172.21.15.47:0/2381246676 >> 172.21.15.47:0/2381246676 conn(0x55f217707200 msgr2=0x55f217739a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.735+0000 7f517c61f740 1 -- 172.21.15.47:0/2381246676 shutdown_connections 2024-03-21T11:45:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.735+0000 7f517c61f740 1 -- 172.21.15.47:0/2381246676 wait complete. 2024-03-21T11:45:35.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.736+0000 7f517c61f740 20 client.32558 trim_cache size 0 max 16384 2024-03-21T11:45:35.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.765+0000 7f517c61f740 1 Processor -- start 2024-03-21T11:45:35.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.765+0000 7f517c61f740 1 -- start start 2024-03-21T11:45:35.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.766+0000 7f517c61f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773ee80 0x55f217740a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.766+0000 7f517c61f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f217750350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.766+0000 7f517c61f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f21775b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.766+0000 7f517c61f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51741b0bc0 con 0x55f21773ee80 2024-03-21T11:45:35.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.766+0000 7f517c61f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51741b2540 con 0x55f21773f870 2024-03-21T11:45:35.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.766+0000 7f517c61f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51741b1220 con 0x55f2177449e0 2024-03-21T11:45:35.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.767+0000 7f5173400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f21775b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.767+0000 7f5172000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f217750350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.767+0000 7f5172000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f217750350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41746/0 (socket says 172.21.15.47:41746) 2024-03-21T11:45:35.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.767+0000 7f5172000640 1 -- 172.21.15.47:0/716446187 learned_addr learned my addr 172.21.15.47:0/716446187 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:35.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.767+0000 7f5173400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f21775b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47124/0 (socket says 172.21.15.47:47124) 2024-03-21T11:45:35.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.768+0000 7f5172000640 1 -- 172.21.15.47:0/716446187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 msgr2=0x55f21775b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:35.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.768+0000 7f5172000640 1 --2- 172.21.15.47:0/716446187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f21775b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.768+0000 7f5172000640 1 -- 172.21.15.47:0/716446187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773ee80 msgr2=0x55f217740a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:45:35.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.768+0000 7f5172000640 1 --2- 172.21.15.47:0/716446187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773ee80 0x55f217740a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.768+0000 7f5172000640 1 -- 172.21.15.47:0/716446187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2173780f0 con 0x55f21773f870 2024-03-21T11:45:35.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.768+0000 7f5173400640 1 --2- 172.21.15.47:0/716446187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f21775b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:35.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.769+0000 7f5172000640 1 --2- 172.21.15.47:0/716446187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f217750350 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f516009bac0 tx=0x7f516009b5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1b174fbac2b46a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:35.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.770+0000 7f515f400640 1 -- 172.21.15.47:0/716446187 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51600b4e60 con 0x55f21773f870 2024-03-21T11:45:35.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.770+0000 7f515f400640 1 -- 172.21.15.47:0/716446187 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51600b1070 con 0x55f21773f870 2024-03-21T11:45:35.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.770+0000 7f515f400640 1 -- 172.21.15.47:0/716446187 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51600a0e10 con 0x55f21773f870 2024-03-21T11:45:35.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.771+0000 7f517c61f740 1 -- 172.21.15.47:0/716446187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 msgr2=0x55f217750350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:35.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.771+0000 7f517c61f740 1 --2- 172.21.15.47:0/716446187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f217750350 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f516009bac0 tx=0x7f516009b5c0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.771+0000 7f517c61f740 1 -- 172.21.15.47:0/716446187 shutdown_connections 2024-03-21T11:45:35.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.771+0000 7f517c61f740 1 --2- 172.21.15.47:0/716446187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f21775b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.771+0000 7f517c61f740 1 --2- 172.21.15.47:0/716446187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773f870 0x55f217750350 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.771+0000 7f517c61f740 1 --2- 172.21.15.47:0/716446187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773ee80 0x55f217740a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.771+0000 7f517c61f740 1 -- 172.21.15.47:0/716446187 >> 172.21.15.47:0/716446187 conn(0x55f217707200 msgr2=0x55f217740310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:35.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.772+0000 7f517c61f740 1 -- 172.21.15.47:0/716446187 shutdown_connections 2024-03-21T11:45:35.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.772+0000 7f517c61f740 1 -- 172.21.15.47:0/716446187 wait complete. 2024-03-21T11:45:35.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.773+0000 7f517c61f740 1 -- start start 2024-03-21T11:45:35.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.773+0000 7f517c61f740 1 Processor -- start 2024-03-21T11:45:35.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.773+0000 7f517c61f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773ee80 0x55f21774e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.773+0000 7f517c61f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773f870 0x55f21774e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.773+0000 7f517c61f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f21774f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.773+0000 7f517c61f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51741b1ee0 con 0x55f21773f870 2024-03-21T11:45:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.773+0000 7f517c61f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51741b1880 con 0x55f21773ee80 2024-03-21T11:45:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.773+0000 7f517c61f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51741aed80 con 0x55f2177449e0 2024-03-21T11:45:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.774+0000 7f517c61f740 10 client.0 _ll_register_callbacks cb 0x7f516000cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.774+0000 7f5173400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f21774f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.774+0000 7f5173400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f21774f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47132/0 (socket says 172.21.15.47:47132) 2024-03-21T11:45:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.774+0000 7f5173400640 1 -- 172.21.15.47:0/398382739 learned_addr learned my addr 172.21.15.47:0/398382739 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.774+0000 7f5172000640 1 --2- 172.21.15.47:0/398382739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773f870 0x55f21774e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.774+0000 7f5172a00640 1 --2- 172.21.15.47:0/398382739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773ee80 0x55f21774e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.774+0000 7f5173400640 1 -- 172.21.15.47:0/398382739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773ee80 msgr2=0x55f21774e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.774+0000 7f5173400640 1 --2- 172.21.15.47:0/398382739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773ee80 0x55f21774e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.774+0000 7f5173400640 1 -- 172.21.15.47:0/398382739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773f870 msgr2=0x55f21774e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.775+0000 7f5173400640 1 --2- 172.21.15.47:0/398382739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773f870 0x55f21774e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.775+0000 7f5173400640 1 -- 172.21.15.47:0/398382739 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f21774fcf0 con 0x55f2177449e0 2024-03-21T11:45:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.775+0000 7f5172a00640 1 --2- 172.21.15.47:0/398382739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773ee80 0x55f21774e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:45:35.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.775+0000 7f5173400640 1 --2- 172.21.15.47:0/398382739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f21774f700 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x55f2177400d0 tx=0x7f516c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c466e0d96acf63ac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:35.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.775+0000 7f515cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:35.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.775+0000 7f515cc00640 1 -- 172.21.15.47:0/398382739 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c02e060 con 0x55f2177449e0 2024-03-21T11:45:35.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.776+0000 7f515cc00640 1 -- 172.21.15.47:0/398382739 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f516c020ae0 con 0x55f2177449e0 2024-03-21T11:45:35.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.776+0000 7f517c61f740 1 -- 172.21.15.47:0/398382739 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51600b1070 con 0x55f2177449e0 2024-03-21T11:45:35.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.776+0000 7f515cc00640 1 -- 172.21.15.47:0/398382739 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f516c020d20 con 0x55f2177449e0 2024-03-21T11:45:35.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.776+0000 7f517c61f740 10 client.32539 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:35.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.776+0000 7f517c61f740 1 -- 172.21.15.47:0/398382739 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f2177b0ae0 con 0x55f2177449e0 2024-03-21T11:45:35.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.777+0000 7f515cc00640 1 -- 172.21.15.47:0/398382739 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f516c0298f0 con 0x55f2177449e0 2024-03-21T11:45:35.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.777+0000 7f515cc00640 1 -- 172.21.15.47:0/398382739 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f516c00f040 con 0x55f2177449e0 2024-03-21T11:45:35.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.777+0000 7f517c61f740 10 client.32539 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:35.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.777+0000 7f517c61f740 10 client.32539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:35.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.777+0000 7f517c61f740 20 client.32539 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:35.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.777+0000 7f517c61f740 1 --2- 172.21.15.47:0/398382739 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f2177b1470 0x55f2177b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:35.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.777+0000 7f517c61f740 4 client.32539 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:45:35.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.777+0000 7f517c61f740 1 -- 172.21.15.47:0/398382739 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f51600b1070 con 0x55f2177b1470 2024-03-21T11:45:35.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.778+0000 7f5172a00640 1 --2- 172.21.15.47:0/398382739 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f2177b1470 0x55f2177b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:35.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.779+0000 7f515cc00640 10 client.32539 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:35.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.780+0000 7f5172a00640 1 --2- 172.21.15.47:0/398382739 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f2177b1470 0x55f2177b3890 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9d29133b76b06b server_cookie=b1ecfa8eb2ba1c63 in_seq=0 out_seq=0 2024-03-21T11:45:35.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.781+0000 7f515cc00640 1 -- 172.21.15.47:0/398382739 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f51600b1070 con 0x55f2177b1470 2024-03-21T11:45:35.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.781+0000 7f515cc00640 10 client.32539 handle_command_reply: tid=0 2024-03-21T11:45:35.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.781+0000 7f517c61f740 1 client.32539 shutdown 2024-03-21T11:45:35.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.781+0000 7f517c61f740 1 -- 172.21.15.47:0/398382739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 msgr2=0x55f21774f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:35.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.781+0000 7f517c61f740 1 --2- 172.21.15.47:0/398382739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f21774f700 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x55f2177400d0 tx=0x7f516c004a40 comp rx=0 tx=0).stop 2024-03-21T11:45:35.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.782+0000 7f517c61f740 1 -- 172.21.15.47:0/398382739 shutdown_connections 2024-03-21T11:45:35.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.782+0000 7f517c61f740 1 --2- 172.21.15.47:0/398382739 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f2177b1470 0x55f2177b3890 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.782+0000 7f517c61f740 1 --2- 172.21.15.47:0/398382739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f2177449e0 0x55f21774f700 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.782+0000 7f517c61f740 1 --2- 172.21.15.47:0/398382739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f21773f870 0x55f21774e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.782+0000 7f517c61f740 1 --2- 172.21.15.47:0/398382739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f21773ee80 0x55f21774e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.782+0000 7f517c61f740 1 -- 172.21.15.47:0/398382739 >> 172.21.15.47:0/398382739 conn(0x55f217707200 msgr2=0x55f217740ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:35.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.782+0000 7f515cc00640 0 client.32539 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:35.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.782+0000 7f517c61f740 1 -- 172.21.15.47:0/398382739 shutdown_connections 2024-03-21T11:45:35.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.782+0000 7f517c61f740 1 -- 172.21.15.47:0/398382739 wait complete. 2024-03-21T11:45:35.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.782+0000 7f517c61f740 20 client.32539 trim_cache size 0 max 16384 2024-03-21T11:45:35.784 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:45:35.785 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:45:35.785 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:45:35.785 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:45:35.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.783+0000 7f5179a00640 1 -- 172.21.15.47:0/4256104542 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f51580a2fa0 msgr2=0x7f51580a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:35.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.783+0000 7f5179a00640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f51580a2fa0 0x7f51580a5460 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f516c005bd0 tx=0x7f516c0097e0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.783+0000 7f5179a00640 1 -- 172.21.15.47:0/4256104542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5174107390 msgr2=0x7f5174195f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:35.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.783+0000 7f5179a00640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5174107390 0x7f5174195f00 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f516000c940 tx=0x7f516000ce00 comp rx=0 tx=0).stop 2024-03-21T11:45:35.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.784+0000 7f5179a00640 1 -- 172.21.15.47:0/4256104542 shutdown_connections 2024-03-21T11:45:35.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.784+0000 7f5179a00640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f51580a2fa0 0x7f51580a5460 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.784+0000 7f5179a00640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51741a3730 0x7f51741a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.784+0000 7f5179a00640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5174107390 0x7f5174195f00 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.784+0000 7f5179a00640 1 --2- 172.21.15.47:0/4256104542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5174104a60 0x7f51741959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:35.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.784+0000 7f5179a00640 1 -- 172.21.15.47:0/4256104542 >> 172.21.15.47:0/4256104542 conn(0x7f51740fbb40 msgr2=0x7f5174101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:35.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.785+0000 7f5179a00640 1 -- 172.21.15.47:0/4256104542 shutdown_connections 2024-03-21T11:45:35.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:35.785+0000 7f5179a00640 1 -- 172.21.15.47:0/4256104542 wait complete. 2024-03-21T11:45:35.817 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:45:35.818 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:45:35.980 INFO:tasks.workunit.client.0.smithi047.stdout: 10 214347 22.17 MB/sec execute 379 sec latency 77.782 ms 2024-03-21T11:45:36.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.075+0000 7f390a200640 1 -- 172.21.15.47:0/4038919582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3904103700 msgr2=0x7f3904103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.075+0000 7f390a200640 1 --2- 172.21.15.47:0/4038919582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3904103700 0x7f3904103b20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009860 tx=0x7f38f402bb70 comp rx=0 tx=0).stop 2024-03-21T11:45:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.075+0000 7f390a200640 1 -- 172.21.15.47:0/4038919582 shutdown_connections 2024-03-21T11:45:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.075+0000 7f390a200640 1 --2- 172.21.15.47:0/4038919582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39041052b0 0x7f390410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.075+0000 7f390a200640 1 --2- 172.21.15.47:0/4038919582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39041048f0 0x7f3904104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.075+0000 7f390a200640 1 --2- 172.21.15.47:0/4038919582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3904103700 0x7f3904103b20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.075+0000 7f390a200640 1 -- 172.21.15.47:0/4038919582 >> 172.21.15.47:0/4038919582 conn(0x7f39040fec70 msgr2=0x7f39041010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.075+0000 7f390a200640 1 -- 172.21.15.47:0/4038919582 shutdown_connections 2024-03-21T11:45:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.075+0000 7f390a200640 1 -- 172.21.15.47:0/4038919582 wait complete. 2024-03-21T11:45:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.076+0000 7f390a200640 1 Processor -- start 2024-03-21T11:45:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.076+0000 7f390a200640 1 -- start start 2024-03-21T11:45:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.077+0000 7f390a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3904103700 0x7f3904199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.077+0000 7f390a200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39041048f0 0x7f3904199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.077+0000 7f390a200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39041052b0 0x7f39041a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.077+0000 7f390a200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390410f350 con 0x7f3904103700 2024-03-21T11:45:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.077+0000 7f390a200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390410f1d0 con 0x7f39041048f0 2024-03-21T11:45:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.077+0000 7f390a200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390410f4d0 con 0x7f39041052b0 2024-03-21T11:45:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.077+0000 7f3908e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3904103700 0x7f3904199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.077+0000 7f38fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39041048f0 0x7f3904199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.077+0000 7f3908e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3904103700 0x7f3904199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35368/0 (socket says 172.21.15.47:35368) 2024-03-21T11:45:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.077+0000 7f3908e00640 1 -- 172.21.15.47:0/20777034 learned_addr learned my addr 172.21.15.47:0/20777034 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.077+0000 7f3909800640 1 --2- 172.21.15.47:0/20777034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39041052b0 0x7f39041a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.078+0000 7f3908e00640 1 -- 172.21.15.47:0/20777034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39041052b0 msgr2=0x7f39041a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.078+0000 7f3908e00640 1 --2- 172.21.15.47:0/20777034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39041052b0 0x7f39041a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.078+0000 7f3908e00640 1 -- 172.21.15.47:0/20777034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39041048f0 msgr2=0x7f3904199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.078+0000 7f3908e00640 1 --2- 172.21.15.47:0/20777034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39041048f0 0x7f3904199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.078+0000 7f3908e00640 1 -- 172.21.15.47:0/20777034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39041a1630 con 0x7f3904103700 2024-03-21T11:45:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.078+0000 7f38fbe00640 1 --2- 172.21.15.47:0/20777034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39041048f0 0x7f3904199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:36.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.078+0000 7f3908e00640 1 --2- 172.21.15.47:0/20777034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3904103700 0x7f3904199a10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f38f40349e0 tx=0x7f38f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:36.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.078+0000 7f3902000640 1 -- 172.21.15.47:0/20777034 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4041020 con 0x7f3904103700 2024-03-21T11:45:36.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.078+0000 7f3902000640 1 -- 172.21.15.47:0/20777034 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38f40323b0 con 0x7f3904103700 2024-03-21T11:45:36.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.078+0000 7f3902000640 1 -- 172.21.15.47:0/20777034 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4039040 con 0x7f3904103700 2024-03-21T11:45:36.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.078+0000 7f390a200640 1 -- 172.21.15.47:0/20777034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39041a18c0 con 0x7f3904103700 2024-03-21T11:45:36.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.078+0000 7f390a200640 1 -- 172.21.15.47:0/20777034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39041a1db0 con 0x7f3904103700 2024-03-21T11:45:36.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.080+0000 7f390a200640 1 -- 172.21.15.47:0/20777034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390410aaf0 con 0x7f3904103700 2024-03-21T11:45:36.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.083+0000 7f3902000640 1 -- 172.21.15.47:0/20777034 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f38f4044020 con 0x7f3904103700 2024-03-21T11:45:36.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.085+0000 7f3902000640 1 --2- 172.21.15.47:0/20777034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f38e80a2fa0 0x7f38e80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.085+0000 7f3902000640 1 -- 172.21.15.47:0/20777034 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f38f40f0fd0 con 0x7f3904103700 2024-03-21T11:45:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.086+0000 7f38fbe00640 1 --2- 172.21.15.47:0/20777034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f38e80a2fa0 0x7f38e80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.089+0000 7f38fbe00640 1 --2- 172.21.15.47:0/20777034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f38e80a2fa0 0x7f38e80a5460 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f390419af20 tx=0x7f38ec00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:36.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.094+0000 7f3902000640 1 -- 172.21.15.47:0/20777034 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f38f404b020 con 0x7f3904103700 2024-03-21T11:45:36.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.526+0000 7f390a200640 1 -- 172.21.15.47:0/20777034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f390402d070 con 0x7f3904103700 2024-03-21T11:45:36.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.528+0000 7f3902000640 1 -- 172.21.15.47:0/20777034 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f38f40bda70 con 0x7f3904103700 2024-03-21T11:45:36.531 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:45:36.531 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:39:47.303461+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/2785789126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2785789126},{"type":"v1","addr":"172.21.15.100:6835","nonce":2785789126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24553,24581,24535,24524,24523]},"id":1}]} 2024-03-21T11:45:36.531 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-03-21T11:45:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.533+0000 7f390a200640 1 -- 172.21.15.47:0/20777034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f38e80a2fa0 msgr2=0x7f38e80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.533+0000 7f390a200640 1 --2- 172.21.15.47:0/20777034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f38e80a2fa0 0x7f38e80a5460 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f390419af20 tx=0x7f38ec00b390 comp rx=0 tx=0).stop 2024-03-21T11:45:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.533+0000 7f390a200640 1 -- 172.21.15.47:0/20777034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3904103700 msgr2=0x7f3904199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.533+0000 7f390a200640 1 --2- 172.21.15.47:0/20777034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3904103700 0x7f3904199a10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f38f40349e0 tx=0x7f38f40042f0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.534+0000 7f390a200640 1 -- 172.21.15.47:0/20777034 shutdown_connections 2024-03-21T11:45:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.534+0000 7f390a200640 1 --2- 172.21.15.47:0/20777034 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f38e80a2fa0 0x7f38e80a5460 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.534+0000 7f390a200640 1 --2- 172.21.15.47:0/20777034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39041052b0 0x7f39041a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.534+0000 7f390a200640 1 --2- 172.21.15.47:0/20777034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39041048f0 0x7f3904199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.534+0000 7f390a200640 1 --2- 172.21.15.47:0/20777034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3904103700 0x7f3904199a10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.534+0000 7f390a200640 1 -- 172.21.15.47:0/20777034 >> 172.21.15.47:0/20777034 conn(0x7f39040fec70 msgr2=0x7f3904107cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.534+0000 7f390a200640 1 -- 172.21.15.47:0/20777034 shutdown_connections 2024-03-21T11:45:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.534+0000 7f390a200640 1 -- 172.21.15.47:0/20777034 wait complete. 2024-03-21T11:45:36.572 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:45:36.572 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:45:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.869+0000 7f24bcc00640 1 -- 172.21.15.47:0/4293159975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24b8104930 msgr2=0x7f24b8106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.869+0000 7f24bcc00640 1 --2- 172.21.15.47:0/4293159975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24b8104930 0x7f24b8106d20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009860 tx=0x7f24a402bb90 comp rx=0 tx=0).stop 2024-03-21T11:45:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.870+0000 7f24bcc00640 1 -- 172.21.15.47:0/4293159975 shutdown_connections 2024-03-21T11:45:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.870+0000 7f24bcc00640 1 --2- 172.21.15.47:0/4293159975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24b8107260 0x7f24b81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.870+0000 7f24bcc00640 1 --2- 172.21.15.47:0/4293159975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24b8104930 0x7f24b8106d20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.870+0000 7f24bcc00640 1 --2- 172.21.15.47:0/4293159975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b8102000 0x7f24b81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.870+0000 7f24bcc00640 1 -- 172.21.15.47:0/4293159975 >> 172.21.15.47:0/4293159975 conn(0x7f24b80fba10 msgr2=0x7f24b80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.870+0000 7f24bcc00640 1 -- 172.21.15.47:0/4293159975 shutdown_connections 2024-03-21T11:45:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.870+0000 7f24bcc00640 1 -- 172.21.15.47:0/4293159975 wait complete. 2024-03-21T11:45:36.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.872+0000 7f24bcc00640 1 Processor -- start 2024-03-21T11:45:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.872+0000 7f24bcc00640 1 -- start start 2024-03-21T11:45:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.873+0000 7f24bcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24b8102000 0x7f24b8101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.873+0000 7f24bcc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24b8104930 0x7f24b80fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.873+0000 7f24bcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b8107260 0x7f24b81004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.873+0000 7f24bcc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b810d2a0 con 0x7f24b8104930 2024-03-21T11:45:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.873+0000 7f24bcc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b810d120 con 0x7f24b8107260 2024-03-21T11:45:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.873+0000 7f24bcc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b810d420 con 0x7f24b8102000 2024-03-21T11:45:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.874+0000 7f24b7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24b8102000 0x7f24b8101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.875+0000 7f24b7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24b8102000 0x7f24b8101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47176/0 (socket says 172.21.15.47:47176) 2024-03-21T11:45:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.875+0000 7f24b7400640 1 -- 172.21.15.47:0/2410066155 learned_addr learned my addr 172.21.15.47:0/2410066155 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.875+0000 7f24b7e00640 1 --2- 172.21.15.47:0/2410066155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b8107260 0x7f24b81004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.875+0000 7f24b6a00640 1 --2- 172.21.15.47:0/2410066155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24b8104930 0x7f24b80fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.875+0000 7f24b7400640 1 -- 172.21.15.47:0/2410066155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b8107260 msgr2=0x7f24b81004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.875+0000 7f24b7400640 1 --2- 172.21.15.47:0/2410066155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b8107260 0x7f24b81004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.875+0000 7f24b7400640 1 -- 172.21.15.47:0/2410066155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24b8104930 msgr2=0x7f24b80fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.875+0000 7f24b7400640 1 --2- 172.21.15.47:0/2410066155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24b8104930 0x7f24b80fffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.875+0000 7f24b7400640 1 -- 172.21.15.47:0/2410066155 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b81a7d00 con 0x7f24b8102000 2024-03-21T11:45:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.876+0000 7f24b7400640 1 --2- 172.21.15.47:0/2410066155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24b8102000 0x7f24b8101970 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f24a800b4a0 tx=0x7f24a800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.876+0000 7f24afe00640 1 -- 172.21.15.47:0/2410066155 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8014070 con 0x7f24b8102000 2024-03-21T11:45:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.877+0000 7f24afe00640 1 -- 172.21.15.47:0/2410066155 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24a8010af0 con 0x7f24b8102000 2024-03-21T11:45:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.877+0000 7f24bcc00640 1 -- 172.21.15.47:0/2410066155 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b81a7ff0 con 0x7f24b8102000 2024-03-21T11:45:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.877+0000 7f24afe00640 1 -- 172.21.15.47:0/2410066155 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8017d10 con 0x7f24b8102000 2024-03-21T11:45:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.877+0000 7f24bcc00640 1 -- 172.21.15.47:0/2410066155 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b81a8610 con 0x7f24b8102000 2024-03-21T11:45:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.881+0000 7f24c0fbc740 1 Processor -- start 2024-03-21T11:45:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.882+0000 7f24c0fbc740 1 -- start start 2024-03-21T11:45:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.882+0000 7f24c0fbc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056d99f0 0x5562056d9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.882+0000 7f24c0fbc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056dabc0 0x5562056db040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.882+0000 7f24c0fbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056db580 0x5562056e1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.882+0000 7f24c0fbc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55620563e2c0 con 0x5562056dabc0 2024-03-21T11:45:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.882+0000 7f24c0fbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562055e6c90 con 0x5562056db580 2024-03-21T11:45:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.882+0000 7f24c0fbc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55620563a360 con 0x5562056d99f0 2024-03-21T11:45:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.883+0000 7f24b6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056db580 0x5562056e1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.883+0000 7f24b6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056db580 0x5562056e1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41804/0 (socket says 172.21.15.47:41804) 2024-03-21T11:45:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.883+0000 7f24b6a00640 1 -- 172.21.15.47:0/2451445395 learned_addr learned my addr 172.21.15.47:0/2451445395 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.883+0000 7f24afe00640 1 -- 172.21.15.47:0/2410066155 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f24a8017070 con 0x7f24b8102000 2024-03-21T11:45:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.883+0000 7f24b7e00640 1 --2- 172.21.15.47:0/2451445395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056d99f0 0x5562056d9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.883+0000 7f24b6a00640 1 -- 172.21.15.47:0/2451445395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056d99f0 msgr2=0x5562056d9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.883+0000 7f24b6a00640 1 --2- 172.21.15.47:0/2451445395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056d99f0 0x5562056d9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.883+0000 7f24b6a00640 1 -- 172.21.15.47:0/2451445395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056dabc0 msgr2=0x5562056db040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:45:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.884+0000 7f24b7400640 1 --2- 172.21.15.47:0/2451445395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056dabc0 0x5562056db040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.884+0000 7f24b6a00640 1 --2- 172.21.15.47:0/2451445395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056dabc0 0x5562056db040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.884+0000 7f24b6a00640 1 -- 172.21.15.47:0/2451445395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55620560b080 con 0x5562056db580 2024-03-21T11:45:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.884+0000 7f24b7e00640 1 --2- 172.21.15.47:0/2451445395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056d99f0 0x5562056d9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:45:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.884+0000 7f24b7400640 1 --2- 172.21.15.47:0/2451445395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056dabc0 0x5562056db040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:45:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.886+0000 7f24afe00640 1 --2- 172.21.15.47:0/2410066155 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f249c0a3070 0x7f249c0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.886+0000 7f24afe00640 1 -- 172.21.15.47:0/2410066155 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f24a80ceaa0 con 0x7f24b8102000 2024-03-21T11:45:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.886+0000 7f24b6a00640 1 --2- 172.21.15.47:0/2451445395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056db580 0x5562056e1e10 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f24a4000c00 tx=0x7f24a4009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4001f599813fd320 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.886+0000 7f24b7e00640 1 --2- 172.21.15.47:0/2410066155 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f249c0a3070 0x7f249c0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.887+0000 7f24a3e00640 1 -- 172.21.15.47:0/2451445395 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4039070 con 0x5562056db580 2024-03-21T11:45:36.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.887+0000 7f24a3e00640 1 -- 172.21.15.47:0/2451445395 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24a403a040 con 0x5562056db580 2024-03-21T11:45:36.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.887+0000 7f24a3e00640 1 -- 172.21.15.47:0/2451445395 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a40052a0 con 0x5562056db580 2024-03-21T11:45:36.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.888+0000 7f24b7e00640 1 --2- 172.21.15.47:0/2410066155 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f249c0a3070 0x7f249c0a5530 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009fd0 tx=0x7f24b00095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:36.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.889+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2451445395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056db580 msgr2=0x5562056e1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.889+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2451445395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056db580 0x5562056e1e10 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f24a4000c00 tx=0x7f24a4009520 comp rx=0 tx=0).stop 2024-03-21T11:45:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.889+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2451445395 shutdown_connections 2024-03-21T11:45:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.889+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2451445395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056db580 0x5562056e1e10 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.889+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2451445395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056dabc0 0x5562056db040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.890+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2451445395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056d99f0 0x5562056d9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.890+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2451445395 >> 172.21.15.47:0/2451445395 conn(0x5562056d4de0 msgr2=0x5562056d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:36.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.890+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2451445395 shutdown_connections 2024-03-21T11:45:36.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.890+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2451445395 wait complete. 2024-03-21T11:45:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.891+0000 7f24c0fbc740 1 -- start start 2024-03-21T11:45:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.891+0000 7f24c0fbc740 1 Processor -- start 2024-03-21T11:45:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.892+0000 7f24c0fbc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056d99f0 0x5562056e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.892+0000 7f24c0fbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056dabc0 0x5562056e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.892+0000 7f24c0fbc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 0x5562056eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.892+0000 7f24c0fbc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562056e2530 con 0x5562056db580 2024-03-21T11:45:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.892+0000 7f24c0fbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562055fa0d0 con 0x5562056dabc0 2024-03-21T11:45:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.892+0000 7f24c0fbc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562056e26b0 con 0x5562056d99f0 2024-03-21T11:45:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.892+0000 7f24c0fbc740 10 client.0 _ll_register_callbacks cb 0x5562055fee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:36.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.893+0000 7f24b7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056d99f0 0x5562056e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.893+0000 7f24b7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056d99f0 0x5562056e3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47208/0 (socket says 172.21.15.47:47208) 2024-03-21T11:45:36.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.893+0000 7f24b7400640 1 -- 172.21.15.47:0/2669898219 learned_addr learned my addr 172.21.15.47:0/2669898219 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:36.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.893+0000 7f24b7e00640 1 --2- 172.21.15.47:0/2669898219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 0x5562056eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.893+0000 7f24b6a00640 1 --2- 172.21.15.47:0/2669898219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056dabc0 0x5562056e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.893+0000 7f24b7400640 1 -- 172.21.15.47:0/2669898219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056dabc0 msgr2=0x5562056e40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:36.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.893+0000 7f24b7400640 1 --2- 172.21.15.47:0/2669898219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056dabc0 0x5562056e40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.893+0000 7f24b7400640 1 -- 172.21.15.47:0/2669898219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 msgr2=0x5562056eb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:36.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.893+0000 7f24b7400640 1 --2- 172.21.15.47:0/2669898219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 0x5562056eb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.893+0000 7f24b7400640 1 -- 172.21.15.47:0/2669898219 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562053523f0 con 0x5562056d99f0 2024-03-21T11:45:36.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.893+0000 7f24b7e00640 1 --2- 172.21.15.47:0/2669898219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 0x5562056eb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:45:36.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.894+0000 7f24b7400640 1 --2- 172.21.15.47:0/2669898219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056d99f0 0x5562056e3b70 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f24a80ced80 tx=0x7f24a8009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0e14e63ec419e20 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:36.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.894+0000 7f24a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:36.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.895+0000 7f24a0c00640 1 -- 172.21.15.47:0/2669898219 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a801e070 con 0x5562056d99f0 2024-03-21T11:45:36.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.895+0000 7f24a0c00640 1 -- 172.21.15.47:0/2669898219 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f24a8010ca0 con 0x5562056d99f0 2024-03-21T11:45:36.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.895+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2669898219 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24a403a040 con 0x5562056d99f0 2024-03-21T11:45:36.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.895+0000 7f24a0c00640 1 -- 172.21.15.47:0/2669898219 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24a809ade0 con 0x5562056d99f0 2024-03-21T11:45:36.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.895+0000 7f24c0fbc740 10 client.32554 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:36.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.895+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2669898219 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55620541c7d0 con 0x5562056d99f0 2024-03-21T11:45:36.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.895+0000 7f24a0c00640 1 -- 172.21.15.47:0/2669898219 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24a403a040 con 0x5562056d99f0 2024-03-21T11:45:36.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.896+0000 7f24a0c00640 1 -- 172.21.15.47:0/2669898219 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f24a80b9c70 con 0x5562056d99f0 2024-03-21T11:45:36.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.897+0000 7f24c0fbc740 10 client.32554 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:36.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.897+0000 7f24c0fbc740 10 client.32554 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:45:36.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.897+0000 7f24c0fbc740 20 client.32554 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:36.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.897+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2669898219 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5562056ef5e0 0x5562056f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.897+0000 7f24c0fbc740 4 client.32554 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:36.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.897+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2669898219 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55620564b800 con 0x5562056ef5e0 2024-03-21T11:45:36.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.898+0000 7f24b6a00640 1 --2- 172.21.15.47:0/2669898219 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5562056ef5e0 0x5562056f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.899+0000 7f24b6a00640 1 --2- 172.21.15.47:0/2669898219 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5562056ef5e0 0x5562056f1a80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=25337c3f99771250 server_cookie=ba0dd8775dc88514 in_seq=0 out_seq=0 2024-03-21T11:45:36.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.899+0000 7f24a0c00640 10 client.32554 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:45:36.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.901+0000 7f24a0c00640 1 -- 172.21.15.47:0/2669898219 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55620564b800 con 0x5562056ef5e0 2024-03-21T11:45:36.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.901+0000 7f24a0c00640 10 client.32554 handle_command_reply: tid=0 2024-03-21T11:45:36.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.901+0000 7f24c0fbc740 1 client.32554 shutdown 2024-03-21T11:45:36.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.901+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2669898219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056d99f0 msgr2=0x5562056e3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:36.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.901+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2669898219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056d99f0 0x5562056e3b70 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f24a80ced80 tx=0x7f24a8009230 comp rx=0 tx=0).stop 2024-03-21T11:45:36.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.902+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2669898219 shutdown_connections 2024-03-21T11:45:36.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.902+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2669898219 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5562056ef5e0 0x5562056f1a80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.902+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2669898219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 0x5562056eb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.902+0000 7f24a0c00640 0 client.32554 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:45:36.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.902+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2669898219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056dabc0 0x5562056e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.902+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2669898219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056d99f0 0x5562056e3b70 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.902+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2669898219 >> 172.21.15.47:0/2669898219 conn(0x5562056d4de0 msgr2=0x5562056d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:36.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.902+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2669898219 shutdown_connections 2024-03-21T11:45:36.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.902+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2669898219 wait complete. 2024-03-21T11:45:36.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.903+0000 7f24c0fbc740 20 client.32554 trim_cache size 0 max 16384 2024-03-21T11:45:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.924+0000 7f24c0fbc740 1 Processor -- start 2024-03-21T11:45:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.924+0000 7f24c0fbc740 1 -- start start 2024-03-21T11:45:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.924+0000 7f24c0fbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056d99f0 0x5562056dddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.924+0000 7f24c0fbc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056dabc0 0x5562056eb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.924+0000 7f24c0fbc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 0x5562056de330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.925+0000 7f24b6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056dabc0 0x5562056eb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.925+0000 7f24b6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056dabc0 0x5562056eb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47222/0 (socket says 172.21.15.47:47222) 2024-03-21T11:45:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.925+0000 7f24b7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056d99f0 0x5562056dddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.925+0000 7f24b7e00640 1 --2- 172.21.15.47:0/521336661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 0x5562056de330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.925+0000 7f24c0fbc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81c1610 con 0x5562056db580 2024-03-21T11:45:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.926+0000 7f24c0fbc740 1 -- 172.21.15.47:0/521336661 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81c2f70 con 0x5562056d99f0 2024-03-21T11:45:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.926+0000 7f24c0fbc740 1 -- 172.21.15.47:0/521336661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81c1c50 con 0x5562056dabc0 2024-03-21T11:45:36.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.925+0000 7f24b6a00640 1 -- 172.21.15.47:0/521336661 learned_addr learned my addr 172.21.15.47:0/521336661 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.927+0000 7f24b7400640 1 -- 172.21.15.47:0/521336661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056dabc0 msgr2=0x5562056eb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.927+0000 7f24b7400640 1 --2- 172.21.15.47:0/521336661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056dabc0 0x5562056eb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.927+0000 7f24b7400640 1 -- 172.21.15.47:0/521336661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 msgr2=0x5562056de330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.927+0000 7f24b7400640 1 --2- 172.21.15.47:0/521336661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 0x5562056de330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.927+0000 7f24b7400640 1 -- 172.21.15.47:0/521336661 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562056e4cf0 con 0x5562056d99f0 2024-03-21T11:45:36.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.929+0000 7f24b7400640 1 --2- 172.21.15.47:0/521336661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056d99f0 0x5562056dddf0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f24a800be60 tx=0x7f24a80992f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e20248d3fff1433f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.929+0000 7f24a3400640 1 -- 172.21.15.47:0/521336661 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a801e070 con 0x5562056d99f0 2024-03-21T11:45:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.929+0000 7f24a3400640 1 -- 172.21.15.47:0/521336661 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24a80a37f0 con 0x5562056d99f0 2024-03-21T11:45:36.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.929+0000 7f24a3400640 1 -- 172.21.15.47:0/521336661 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a80b9b50 con 0x5562056d99f0 2024-03-21T11:45:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.930+0000 7f24c0fbc740 1 -- 172.21.15.47:0/521336661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056d99f0 msgr2=0x5562056dddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:36.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.930+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/521336661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056d99f0 0x5562056dddf0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f24a800be60 tx=0x7f24a80992f0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.933+0000 7f24c0fbc740 1 -- 172.21.15.47:0/521336661 shutdown_connections 2024-03-21T11:45:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.933+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/521336661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 0x5562056de330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.933+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/521336661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056dabc0 0x5562056eb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.933+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/521336661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056d99f0 0x5562056dddf0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.933+0000 7f24c0fbc740 1 -- 172.21.15.47:0/521336661 >> 172.21.15.47:0/521336661 conn(0x5562056d4de0 msgr2=0x5562056d88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.934+0000 7f24c0fbc740 1 -- 172.21.15.47:0/521336661 shutdown_connections 2024-03-21T11:45:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.934+0000 7f24c0fbc740 1 -- 172.21.15.47:0/521336661 wait complete. 2024-03-21T11:45:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.935+0000 7f24c0fbc740 1 -- start start 2024-03-21T11:45:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.935+0000 7f24c0fbc740 1 Processor -- start 2024-03-21T11:45:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.937+0000 7f24c0fbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056d99f0 0x5562056eaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.937+0000 7f24c0fbc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056dabc0 0x5562056e9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.937+0000 7f24c0fbc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 0x5562056e9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.938+0000 7f24c0fbc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81c2910 con 0x5562056db580 2024-03-21T11:45:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.938+0000 7f24c0fbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81c22b0 con 0x5562056d99f0 2024-03-21T11:45:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.938+0000 7f24c0fbc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81ae7a0 con 0x5562056dabc0 2024-03-21T11:45:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.938+0000 7f24b7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056d99f0 0x5562056eaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.938+0000 7f24c0fbc740 10 client.0 _ll_register_callbacks cb 0x7f24a809a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.938+0000 7f24b7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056d99f0 0x5562056eaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41842/0 (socket says 172.21.15.47:41842) 2024-03-21T11:45:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.938+0000 7f24b7400640 1 -- 172.21.15.47:0/2471613224 learned_addr learned my addr 172.21.15.47:0/2471613224 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.938+0000 7f24b6a00640 1 --2- 172.21.15.47:0/2471613224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056dabc0 0x5562056e9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.939+0000 7f24b7e00640 1 --2- 172.21.15.47:0/2471613224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 0x5562056e9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.939+0000 7f24b7400640 1 -- 172.21.15.47:0/2471613224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056dabc0 msgr2=0x5562056e9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.939+0000 7f24b7400640 1 --2- 172.21.15.47:0/2471613224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056dabc0 0x5562056e9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.939+0000 7f24b7400640 1 -- 172.21.15.47:0/2471613224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 msgr2=0x5562056e9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.939+0000 7f24b7400640 1 --2- 172.21.15.47:0/2471613224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 0x5562056e9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.939+0000 7f24b7400640 1 -- 172.21.15.47:0/2471613224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562056ea2b0 con 0x5562056d99f0 2024-03-21T11:45:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.940+0000 7f24b7400640 1 --2- 172.21.15.47:0/2471613224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056d99f0 0x5562056eaf30 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f24a809ad70 tx=0x7f24a8009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7467bb40bad02370 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.940+0000 7f24a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:45:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.940+0000 7f24a0c00640 1 -- 172.21.15.47:0/2471613224 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a801e070 con 0x5562056d99f0 2024-03-21T11:45:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.940+0000 7f24a0c00640 1 -- 172.21.15.47:0/2471613224 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f24a80b9b50 con 0x5562056d99f0 2024-03-21T11:45:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.941+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2471613224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24a80a37f0 con 0x5562056d99f0 2024-03-21T11:45:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.942+0000 7f24a0c00640 1 -- 172.21.15.47:0/2471613224 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24a8010610 con 0x5562056d99f0 2024-03-21T11:45:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.942+0000 7f24c0fbc740 10 client.32579 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.942+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2471613224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55620574bee0 con 0x5562056d99f0 2024-03-21T11:45:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.943+0000 7f24a0c00640 1 -- 172.21.15.47:0/2471613224 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24a80a37f0 con 0x5562056d99f0 2024-03-21T11:45:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.944+0000 7f24a0c00640 1 -- 172.21.15.47:0/2471613224 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f24a80853e0 con 0x5562056d99f0 2024-03-21T11:45:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.944+0000 7f24c0fbc740 10 client.32579 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.944+0000 7f24c0fbc740 10 client.32579 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:45:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.944+0000 7f24c0fbc740 20 client.32579 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.944+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2471613224 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5562056ef5e0 0x55620574e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.944+0000 7f24c0fbc740 4 client.32579 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:45:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.944+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2471613224 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556205645b50 con 0x5562056ef5e0 2024-03-21T11:45:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.948+0000 7f24b6a00640 1 --2- 172.21.15.47:0/2471613224 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5562056ef5e0 0x55620574e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.948+0000 7f24b6a00640 1 --2- 172.21.15.47:0/2471613224 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5562056ef5e0 0x55620574e6f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5c8215254e94ef1 server_cookie=600d581397ef1ae8 in_seq=0 out_seq=0 2024-03-21T11:45:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.948+0000 7f24a0c00640 10 client.32579 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:45:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.948+0000 7f24a0c00640 1 -- 172.21.15.47:0/2471613224 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556205645b50 con 0x5562056ef5e0 2024-03-21T11:45:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.948+0000 7f24a0c00640 10 client.32579 handle_command_reply: tid=0 2024-03-21T11:45:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.948+0000 7f24c0fbc740 1 client.32579 shutdown 2024-03-21T11:45:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.949+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2471613224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056d99f0 msgr2=0x5562056eaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.949+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2471613224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056d99f0 0x5562056eaf30 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f24a809ad70 tx=0x7f24a8009230 comp rx=0 tx=0).stop 2024-03-21T11:45:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.949+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2471613224 shutdown_connections 2024-03-21T11:45:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.949+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2471613224 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5562056ef5e0 0x55620574e6f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.949+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2471613224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562056db580 0x5562056e9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.949+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2471613224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562056dabc0 0x5562056e9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.949+0000 7f24c0fbc740 1 --2- 172.21.15.47:0/2471613224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562056d99f0 0x5562056eaf30 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.949+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2471613224 >> 172.21.15.47:0/2471613224 conn(0x5562056d4de0 msgr2=0x5562056d38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.949+0000 7f24a0c00640 0 client.32579 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:45:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.949+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2471613224 shutdown_connections 2024-03-21T11:45:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.952+0000 7f24c0fbc740 1 -- 172.21.15.47:0/2471613224 wait complete. 2024-03-21T11:45:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.952+0000 7f24c0fbc740 20 client.32579 trim_cache size 0 max 16384 2024-03-21T11:45:36.953 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:45:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.953+0000 7f24bcc00640 1 -- 172.21.15.47:0/2410066155 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f249c0a3070 msgr2=0x7f249c0a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.953+0000 7f24bcc00640 1 --2- 172.21.15.47:0/2410066155 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f249c0a3070 0x7f249c0a5530 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009fd0 tx=0x7f24b00095c0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.953+0000 7f24bcc00640 1 -- 172.21.15.47:0/2410066155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24b8102000 msgr2=0x7f24b8101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.953+0000 7f24bcc00640 1 --2- 172.21.15.47:0/2410066155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24b8102000 0x7f24b8101970 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f24a800b4a0 tx=0x7f24a800b960 comp rx=0 tx=0).stop 2024-03-21T11:45:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.953+0000 7f24bcc00640 1 -- 172.21.15.47:0/2410066155 shutdown_connections 2024-03-21T11:45:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.953+0000 7f24bcc00640 1 --2- 172.21.15.47:0/2410066155 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f249c0a3070 0x7f249c0a5530 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.953+0000 7f24bcc00640 1 --2- 172.21.15.47:0/2410066155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b8107260 0x7f24b81004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.953+0000 7f24bcc00640 1 --2- 172.21.15.47:0/2410066155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24b8104930 0x7f24b80fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.953+0000 7f24bcc00640 1 --2- 172.21.15.47:0/2410066155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24b8102000 0x7f24b8101970 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.953+0000 7f24bcc00640 1 -- 172.21.15.47:0/2410066155 >> 172.21.15.47:0/2410066155 conn(0x7f24b80fba10 msgr2=0x7f24b8102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.953+0000 7f24bcc00640 1 -- 172.21.15.47:0/2410066155 shutdown_connections 2024-03-21T11:45:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:36.953+0000 7f24bcc00640 1 -- 172.21.15.47:0/2410066155 wait complete. 2024-03-21T11:45:36.971 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:36 smithi047 ceph-mon[31351]: pgmap v1068: 129 pgs: 129 active+clean; 634 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-03-21T11:45:36.972 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:36 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/20777034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:45:36.974 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:45:36.974 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:45:36.981 INFO:tasks.workunit.client.0.smithi047.stdout: 10 214863 22.20 MB/sec execute 380 sec latency 69.847 ms 2024-03-21T11:45:37.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:36 smithi100 ceph-mon[36245]: pgmap v1068: 129 pgs: 129 active+clean; 634 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-03-21T11:45:37.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:36 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/20777034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:45:37.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:36 smithi102 ceph-mon[36748]: pgmap v1068: 129 pgs: 129 active+clean; 634 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-03-21T11:45:37.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:36 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/20777034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:45:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.250+0000 7f6762c00640 1 -- 172.21.15.47:0/113463361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f675c100f10 msgr2=0x7f675c101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.250+0000 7f6762c00640 1 --2- 172.21.15.47:0/113463361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f675c100f10 0x7f675c101390 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f674800a910 tx=0x7f674802be70 comp rx=0 tx=0).stop 2024-03-21T11:45:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.251+0000 7f6762c00640 1 -- 172.21.15.47:0/113463361 shutdown_connections 2024-03-21T11:45:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.251+0000 7f6762c00640 1 --2- 172.21.15.47:0/113463361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1018d0 0x7f675c108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.251+0000 7f6762c00640 1 --2- 172.21.15.47:0/113463361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f675c100f10 0x7f675c101390 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.251+0000 7f6762c00640 1 --2- 172.21.15.47:0/113463361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f675c0ffd20 0x7f675c100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.251+0000 7f6762c00640 1 -- 172.21.15.47:0/113463361 >> 172.21.15.47:0/113463361 conn(0x7f675c0fb290 msgr2=0x7f675c0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.251+0000 7f6762c00640 1 -- 172.21.15.47:0/113463361 shutdown_connections 2024-03-21T11:45:37.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.251+0000 7f6762c00640 1 -- 172.21.15.47:0/113463361 wait complete. 2024-03-21T11:45:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.252+0000 7f6762c00640 1 Processor -- start 2024-03-21T11:45:37.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.252+0000 7f6762c00640 1 -- start start 2024-03-21T11:45:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.253+0000 7f6762c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f675c0ffd20 0x7f675c1957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.253+0000 7f6762c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f675c100f10 0x7f675c195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.253+0000 7f6762c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1018d0 0x7f675c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.253+0000 7f6762c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c10b180 con 0x7f675c0ffd20 2024-03-21T11:45:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.253+0000 7f6762c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c10b000 con 0x7f675c1018d0 2024-03-21T11:45:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.253+0000 7f6762c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c10b300 con 0x7f675c100f10 2024-03-21T11:45:37.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.253+0000 7f675b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f675c100f10 0x7f675c195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.253+0000 7f675b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f675c100f10 0x7f675c195cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47250/0 (socket says 172.21.15.47:47250) 2024-03-21T11:45:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.253+0000 7f675b400640 1 -- 172.21.15.47:0/1862430666 learned_addr learned my addr 172.21.15.47:0/1862430666 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.253+0000 7f6760c00640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1018d0 0x7f675c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.254+0000 7f675be00640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f675c0ffd20 0x7f675c1957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.254+0000 7f6760c00640 1 -- 172.21.15.47:0/1862430666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f675c100f10 msgr2=0x7f675c195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.254+0000 7f6760c00640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f675c100f10 0x7f675c195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.254+0000 7f6760c00640 1 -- 172.21.15.47:0/1862430666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f675c0ffd20 msgr2=0x7f675c1957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.254+0000 7f6760c00640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f675c0ffd20 0x7f675c1957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.254+0000 7f6760c00640 1 -- 172.21.15.47:0/1862430666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c1a17e0 con 0x7f675c1018d0 2024-03-21T11:45:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.254+0000 7f675be00640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f675c0ffd20 0x7f675c1957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.255+0000 7f6760c00640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1018d0 0x7f675c19cd60 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f674c00a870 tx=0x7f674c00ad30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.255+0000 7f6758c00640 1 -- 172.21.15.47:0/1862430666 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c0079b0 con 0x7f675c1018d0 2024-03-21T11:45:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.255+0000 7f6762c00640 1 -- 172.21.15.47:0/1862430666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675c1a1ad0 con 0x7f675c1018d0 2024-03-21T11:45:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.256+0000 7f6758c00640 1 -- 172.21.15.47:0/1862430666 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f674c005500 con 0x7f675c1018d0 2024-03-21T11:45:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.256+0000 7f6758c00640 1 -- 172.21.15.47:0/1862430666 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c004d00 con 0x7f675c1018d0 2024-03-21T11:45:37.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.256+0000 7f6762c00640 1 -- 172.21.15.47:0/1862430666 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c1a2020 con 0x7f675c1018d0 2024-03-21T11:45:37.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.260+0000 7f6758c00640 1 -- 172.21.15.47:0/1862430666 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f674c007b50 con 0x7f675c1018d0 2024-03-21T11:45:37.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.261+0000 7f676581d740 1 Processor -- start 2024-03-21T11:45:37.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.261+0000 7f676581d740 1 -- start start 2024-03-21T11:45:37.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.262+0000 7f676581d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 0x557ec2aefc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.262+0000 7f676581d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2af09e0 0x557ec2af0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.262+0000 7f676581d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2af7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.262+0000 7f676581d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ec2a542c0 con 0x557ec2aef810 2024-03-21T11:45:37.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.262+0000 7f676581d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ec29fcc90 con 0x557ec2af09e0 2024-03-21T11:45:37.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.262+0000 7f676581d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ec2a50360 con 0x557ec2af13a0 2024-03-21T11:45:37.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.262+0000 7f675be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2af09e0 0x557ec2af0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.262+0000 7f675be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2af09e0 0x557ec2af0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41868/0 (socket says 172.21.15.47:41868) 2024-03-21T11:45:37.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.262+0000 7f675be00640 1 -- 172.21.15.47:0/2323310334 learned_addr learned my addr 172.21.15.47:0/2323310334 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:37.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.262+0000 7f675b400640 1 --2- 172.21.15.47:0/2323310334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 0x557ec2aefc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.263+0000 7f675b400640 1 --2- 172.21.15.47:0/2323310334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2af7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.263+0000 7f675be00640 1 -- 172.21.15.47:0/2323310334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 msgr2=0x557ec2af7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.263+0000 7f675be00640 1 --2- 172.21.15.47:0/2323310334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2af7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.263+0000 7f675be00640 1 -- 172.21.15.47:0/2323310334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 msgr2=0x557ec2aefc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.263+0000 7f675be00640 1 --2- 172.21.15.47:0/2323310334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 0x557ec2aefc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.263+0000 7f675be00640 1 -- 172.21.15.47:0/2323310334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ec2a21080 con 0x557ec2af09e0 2024-03-21T11:45:37.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.263+0000 7f6758c00640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f67400a2ff0 0x7f67400a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.263+0000 7f6758c00640 1 -- 172.21.15.47:0/1862430666 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f674c0cea50 con 0x7f675c1018d0 2024-03-21T11:45:37.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.264+0000 7f675b400640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f67400a2ff0 0x7f67400a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.264+0000 7f675be00640 1 --2- 172.21.15.47:0/2323310334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2af09e0 0x557ec2af0e60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f675400f6b0 tx=0x7f675400fb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acc426dfd09a9b28 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.264+0000 7f675b400640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f67400a2ff0 0x7f67400a54b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x557ec2af08e0 tx=0x7f6748005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.264+0000 7f6751200640 1 -- 172.21.15.47:0/2323310334 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754004d20 con 0x557ec2af09e0 2024-03-21T11:45:37.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.264+0000 7f6751200640 1 -- 172.21.15.47:0/2323310334 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f675400de00 con 0x557ec2af09e0 2024-03-21T11:45:37.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.264+0000 7f6751200640 1 -- 172.21.15.47:0/2323310334 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754018d70 con 0x557ec2af09e0 2024-03-21T11:45:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.265+0000 7f676581d740 1 -- 172.21.15.47:0/2323310334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2af09e0 msgr2=0x557ec2af0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.265+0000 7f676581d740 1 --2- 172.21.15.47:0/2323310334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2af09e0 0x557ec2af0e60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f675400f6b0 tx=0x7f675400fb70 comp rx=0 tx=0).stop 2024-03-21T11:45:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.266+0000 7f676581d740 1 -- 172.21.15.47:0/2323310334 shutdown_connections 2024-03-21T11:45:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.266+0000 7f676581d740 1 --2- 172.21.15.47:0/2323310334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2af7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.266+0000 7f676581d740 1 --2- 172.21.15.47:0/2323310334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2af09e0 0x557ec2af0e60 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.266+0000 7f676581d740 1 --2- 172.21.15.47:0/2323310334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 0x557ec2aefc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.266+0000 7f676581d740 1 -- 172.21.15.47:0/2323310334 >> 172.21.15.47:0/2323310334 conn(0x557ec2aeac00 msgr2=0x557ec2aeb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.266+0000 7f676581d740 1 -- 172.21.15.47:0/2323310334 shutdown_connections 2024-03-21T11:45:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.266+0000 7f676581d740 1 -- 172.21.15.47:0/2323310334 wait complete. 2024-03-21T11:45:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.266+0000 7f676581d740 1 -- start start 2024-03-21T11:45:37.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.266+0000 7f676581d740 1 Processor -- start 2024-03-21T11:45:37.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.267+0000 7f676581d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2aef810 0x557ec2af9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.267+0000 7f676581d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2af09e0 0x557ec2af9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.267+0000 7f676581d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2af13a0 0x557ec2b00fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.267+0000 7f676581d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ec2af83b0 con 0x557ec2af13a0 2024-03-21T11:45:37.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.267+0000 7f676581d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ec2a100d0 con 0x557ec2af09e0 2024-03-21T11:45:37.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.267+0000 7f676581d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ec2af8530 con 0x557ec2aef810 2024-03-21T11:45:37.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.267+0000 7f676581d740 10 client.0 _ll_register_callbacks cb 0x557ec2a14e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:37.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.267+0000 7f675b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2af09e0 0x557ec2af9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.267+0000 7f675b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2af09e0 0x557ec2af9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41880/0 (socket says 172.21.15.47:41880) 2024-03-21T11:45:37.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.267+0000 7f6760c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2af13a0 0x557ec2b00fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.267+0000 7f675b400640 1 -- 172.21.15.47:0/412065620 learned_addr learned my addr 172.21.15.47:0/412065620 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:37.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.268+0000 7f675be00640 1 --2- 172.21.15.47:0/412065620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2aef810 0x557ec2af9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.268+0000 7f675b400640 1 -- 172.21.15.47:0/412065620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2aef810 msgr2=0x557ec2af9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.268+0000 7f675b400640 1 --2- 172.21.15.47:0/412065620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2aef810 0x557ec2af9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.268+0000 7f675b400640 1 -- 172.21.15.47:0/412065620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2af13a0 msgr2=0x557ec2b00fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.268+0000 7f675b400640 1 --2- 172.21.15.47:0/412065620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2af13a0 0x557ec2b00fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.268+0000 7f675b400640 1 -- 172.21.15.47:0/412065620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ec27683f0 con 0x557ec2af09e0 2024-03-21T11:45:37.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.268+0000 7f675be00640 1 --2- 172.21.15.47:0/412065620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2aef810 0x557ec2af9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:37.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.268+0000 7f6760c00640 1 --2- 172.21.15.47:0/412065620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2af13a0 0x557ec2b00fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:37.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.268+0000 7f675b400640 1 --2- 172.21.15.47:0/412065620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2af09e0 0x557ec2af9f70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f6748004c20 tx=0x7f674802be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec9cfd78bb19b32c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.268+0000 7f6745600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:45:37.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.269+0000 7f6745600640 1 -- 172.21.15.47:0/412065620 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6748054020 con 0x557ec2af09e0 2024-03-21T11:45:37.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.269+0000 7f676581d740 1 -- 172.21.15.47:0/412065620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f675400de00 con 0x557ec2af09e0 2024-03-21T11:45:37.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.269+0000 7f6745600640 1 -- 172.21.15.47:0/412065620 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6748004450 con 0x557ec2af09e0 2024-03-21T11:45:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.269+0000 7f6745600640 1 -- 172.21.15.47:0/412065620 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f674805f050 con 0x557ec2af09e0 2024-03-21T11:45:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.269+0000 7f676581d740 10 client.32591 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.269+0000 7f676581d740 1 -- 172.21.15.47:0/412065620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557ec28327d0 con 0x557ec2af09e0 2024-03-21T11:45:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.270+0000 7f6745600640 1 -- 172.21.15.47:0/412065620 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f675400de00 con 0x557ec2af09e0 2024-03-21T11:45:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.270+0000 7f6745600640 1 -- 172.21.15.47:0/412065620 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f674800b040 con 0x557ec2af09e0 2024-03-21T11:45:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.270+0000 7f676581d740 10 client.32591 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.270+0000 7f676581d740 10 client.32591 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:45:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.270+0000 7f676581d740 20 client.32591 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.270+0000 7f676581d740 1 --2- 172.21.15.47:0/412065620 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x557ec2b05560 0x557ec2b079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.270+0000 7f676581d740 4 client.32591 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.270+0000 7f676581d740 1 -- 172.21.15.47:0/412065620 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ec2a61800 con 0x557ec2b05560 2024-03-21T11:45:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.270+0000 7f675be00640 1 --2- 172.21.15.47:0/412065620 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x557ec2b05560 0x557ec2b079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.271+0000 7f6745600640 10 client.32591 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:45:37.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.271+0000 7f675be00640 1 --2- 172.21.15.47:0/412065620 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x557ec2b05560 0x557ec2b079c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4188b31de52d28d8 server_cookie=f03267ef39c5209a in_seq=0 out_seq=0 2024-03-21T11:45:37.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.273+0000 7f6745600640 1 -- 172.21.15.47:0/412065620 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557ec2a61800 con 0x557ec2b05560 2024-03-21T11:45:37.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.273+0000 7f6745600640 10 client.32591 handle_command_reply: tid=0 2024-03-21T11:45:37.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.273+0000 7f676581d740 1 client.32591 shutdown 2024-03-21T11:45:37.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.274+0000 7f676581d740 1 -- 172.21.15.47:0/412065620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2af09e0 msgr2=0x557ec2af9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.274+0000 7f676581d740 1 --2- 172.21.15.47:0/412065620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2af09e0 0x557ec2af9f70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f6748004c20 tx=0x7f674802be70 comp rx=0 tx=0).stop 2024-03-21T11:45:37.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.274+0000 7f676581d740 1 -- 172.21.15.47:0/412065620 shutdown_connections 2024-03-21T11:45:37.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.274+0000 7f676581d740 1 --2- 172.21.15.47:0/412065620 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x557ec2b05560 0x557ec2b079c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.274+0000 7f676581d740 1 --2- 172.21.15.47:0/412065620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2af13a0 0x557ec2b00fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.274+0000 7f676581d740 1 --2- 172.21.15.47:0/412065620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2af09e0 0x557ec2af9f70 secure :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f6748004c20 tx=0x7f674802be70 comp rx=0 tx=0).stop 2024-03-21T11:45:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.274+0000 7f676581d740 1 --2- 172.21.15.47:0/412065620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2aef810 0x557ec2af9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.274+0000 7f676581d740 1 -- 172.21.15.47:0/412065620 >> 172.21.15.47:0/412065620 conn(0x557ec2aeac00 msgr2=0x557ec2ae9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.274+0000 7f6745600640 0 client.32591 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:45:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.276+0000 7f676581d740 1 -- 172.21.15.47:0/412065620 shutdown_connections 2024-03-21T11:45:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.276+0000 7f676581d740 1 -- 172.21.15.47:0/412065620 wait complete. 2024-03-21T11:45:37.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.277+0000 7f676581d740 20 client.32591 trim_cache size 0 max 16384 2024-03-21T11:45:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.293+0000 7f676581d740 1 Processor -- start 2024-03-21T11:45:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.293+0000 7f676581d740 1 -- start start 2024-03-21T11:45:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.293+0000 7f676581d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2aeac00 0x557ec2ae9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.293+0000 7f676581d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 0x557ec2b00f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.293+0000 7f676581d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2b0c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.293+0000 7f676581d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c1ac230 con 0x557ec2aef810 2024-03-21T11:45:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.293+0000 7f676581d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c1adbb0 con 0x557ec2aeac00 2024-03-21T11:45:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.293+0000 7f676581d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c1ac890 con 0x557ec2af13a0 2024-03-21T11:45:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.293+0000 7f675be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2aeac00 0x557ec2ae9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.294+0000 7f675be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2aeac00 0x557ec2ae9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41894/0 (socket says 172.21.15.47:41894) 2024-03-21T11:45:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.294+0000 7f675be00640 1 -- 172.21.15.47:0/1691292660 learned_addr learned my addr 172.21.15.47:0/1691292660 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.294+0000 7f6760c00640 1 --2- 172.21.15.47:0/1691292660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2b0c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.294+0000 7f675b400640 1 --2- 172.21.15.47:0/1691292660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 0x557ec2b00f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.294+0000 7f675be00640 1 -- 172.21.15.47:0/1691292660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 msgr2=0x557ec2b0c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.294+0000 7f675be00640 1 --2- 172.21.15.47:0/1691292660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2b0c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.294+0000 7f675be00640 1 -- 172.21.15.47:0/1691292660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 msgr2=0x557ec2b00f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.294+0000 7f675be00640 1 --2- 172.21.15.47:0/1691292660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 0x557ec2b00f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.294+0000 7f675be00640 1 -- 172.21.15.47:0/1691292660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ec2aeb970 con 0x557ec2aeac00 2024-03-21T11:45:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.294+0000 7f675b400640 1 --2- 172.21.15.47:0/1691292660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 0x557ec2b00f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:45:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.294+0000 7f6760c00640 1 --2- 172.21.15.47:0/1691292660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2b0c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.294+0000 7f675be00640 1 --2- 172.21.15.47:0/1691292660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2aeac00 0x557ec2ae9d30 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x557ec2aebb70 tx=0x7f675400a590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6b2d36031e040d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.295+0000 7f6747e00640 1 -- 172.21.15.47:0/1691292660 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675400dc30 con 0x557ec2aeac00 2024-03-21T11:45:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.295+0000 7f6747e00640 1 -- 172.21.15.47:0/1691292660 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f675400ddd0 con 0x557ec2aeac00 2024-03-21T11:45:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.295+0000 7f6747e00640 1 -- 172.21.15.47:0/1691292660 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754018ad0 con 0x557ec2aeac00 2024-03-21T11:45:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.296+0000 7f676581d740 1 -- 172.21.15.47:0/1691292660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2aeac00 msgr2=0x557ec2ae9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.296+0000 7f676581d740 1 --2- 172.21.15.47:0/1691292660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2aeac00 0x557ec2ae9d30 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x557ec2aebb70 tx=0x7f675400a590 comp rx=0 tx=0).stop 2024-03-21T11:45:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.296+0000 7f676581d740 1 -- 172.21.15.47:0/1691292660 shutdown_connections 2024-03-21T11:45:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.296+0000 7f676581d740 1 --2- 172.21.15.47:0/1691292660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2b0c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.296+0000 7f676581d740 1 --2- 172.21.15.47:0/1691292660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 0x557ec2b00f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.296+0000 7f676581d740 1 --2- 172.21.15.47:0/1691292660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2aeac00 0x557ec2ae9d30 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.296+0000 7f676581d740 1 -- 172.21.15.47:0/1691292660 >> 172.21.15.47:0/1691292660 conn(0x557ec2af95f0 msgr2=0x557ec2ae9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.296+0000 7f676581d740 1 -- 172.21.15.47:0/1691292660 shutdown_connections 2024-03-21T11:45:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.297+0000 7f676581d740 1 -- 172.21.15.47:0/1691292660 wait complete. 2024-03-21T11:45:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.297+0000 7f676581d740 1 -- start start 2024-03-21T11:45:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.297+0000 7f676581d740 1 Processor -- start 2024-03-21T11:45:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.298+0000 7f676581d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2aeac00 0x557ec2b009b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.298+0000 7f676581d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 0x557ec2b00ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.298+0000 7f676581d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2afd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.298+0000 7f676581d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c1ad550 con 0x557ec2aef810 2024-03-21T11:45:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.298+0000 7f676581d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c1acef0 con 0x557ec2aeac00 2024-03-21T11:45:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.298+0000 7f676581d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c1aa3f0 con 0x557ec2af13a0 2024-03-21T11:45:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.298+0000 7f676581d740 10 client.0 _ll_register_callbacks cb 0x7f6748047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.298+0000 7f6760c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2afd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.298+0000 7f6760c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2afd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47272/0 (socket says 172.21.15.47:47272) 2024-03-21T11:45:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.298+0000 7f6760c00640 1 -- 172.21.15.47:0/1662547797 learned_addr learned my addr 172.21.15.47:0/1662547797 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.298+0000 7f675be00640 1 --2- 172.21.15.47:0/1662547797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2aeac00 0x557ec2b009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.299+0000 7f6760c00640 1 -- 172.21.15.47:0/1662547797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2aeac00 msgr2=0x557ec2b009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.299+0000 7f6760c00640 1 --2- 172.21.15.47:0/1662547797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2aeac00 0x557ec2b009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.299+0000 7f6760c00640 1 -- 172.21.15.47:0/1662547797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 msgr2=0x557ec2b00ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:45:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.299+0000 7f675b400640 1 --2- 172.21.15.47:0/1662547797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 0x557ec2b00ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.299+0000 7f6760c00640 1 --2- 172.21.15.47:0/1662547797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 0x557ec2b00ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.299+0000 7f6760c00640 1 -- 172.21.15.47:0/1662547797 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ec2afdd00 con 0x557ec2af13a0 2024-03-21T11:45:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.299+0000 7f675b400640 1 --2- 172.21.15.47:0/1662547797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 0x557ec2b00ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:45:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.299+0000 7f6760c00640 1 --2- 172.21.15.47:0/1662547797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2afd4d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f674c097820 tx=0x7f674c0bb120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=529b1e3ac1504963 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.299+0000 7f6745600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.299+0000 7f6745600640 1 -- 172.21.15.47:0/1662547797 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c09a4f0 con 0x557ec2af13a0 2024-03-21T11:45:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.300+0000 7f6745600640 1 -- 172.21.15.47:0/1662547797 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f674c096bf0 con 0x557ec2af13a0 2024-03-21T11:45:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.300+0000 7f676581d740 1 -- 172.21.15.47:0/1662547797 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f675400ddd0 con 0x557ec2af13a0 2024-03-21T11:45:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.300+0000 7f6745600640 1 -- 172.21.15.47:0/1662547797 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f674c0a3800 con 0x557ec2af13a0 2024-03-21T11:45:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.300+0000 7f676581d740 10 client.32578 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.300+0000 7f676581d740 1 -- 172.21.15.47:0/1662547797 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557ec2b6a520 con 0x557ec2af13a0 2024-03-21T11:45:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.300+0000 7f6745600640 1 -- 172.21.15.47:0/1662547797 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f675400ddd0 con 0x557ec2af13a0 2024-03-21T11:45:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.300+0000 7f6745600640 1 -- 172.21.15.47:0/1662547797 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f674c0a3a50 con 0x557ec2af13a0 2024-03-21T11:45:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.301+0000 7f676581d740 10 client.32578 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.301+0000 7f676581d740 10 client.32578 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:45:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.301+0000 7f676581d740 20 client.32578 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.301+0000 7f676581d740 1 --2- 172.21.15.47:0/1662547797 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x557ec2b6ae50 0x557ec2b6d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.301+0000 7f676581d740 4 client.32578 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:45:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.301+0000 7f676581d740 1 -- 172.21.15.47:0/1662547797 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557ec2a5bb50 con 0x557ec2b6ae50 2024-03-21T11:45:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.301+0000 7f675be00640 1 --2- 172.21.15.47:0/1662547797 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x557ec2b6ae50 0x557ec2b6d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.302+0000 7f675be00640 1 --2- 172.21.15.47:0/1662547797 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x557ec2b6ae50 0x557ec2b6d270 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=481483cc5560cd80 server_cookie=6dd85ea178f9b81a in_seq=0 out_seq=0 2024-03-21T11:45:37.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.302+0000 7f6745600640 10 client.32578 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:45:37.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.305+0000 7f6745600640 1 -- 172.21.15.47:0/1662547797 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557ec2a5bb50 con 0x557ec2b6ae50 2024-03-21T11:45:37.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.305+0000 7f6745600640 10 client.32578 handle_command_reply: tid=0 2024-03-21T11:45:37.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.305+0000 7f676581d740 1 client.32578 shutdown 2024-03-21T11:45:37.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.305+0000 7f676581d740 1 -- 172.21.15.47:0/1662547797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 msgr2=0x557ec2afd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.305+0000 7f676581d740 1 --2- 172.21.15.47:0/1662547797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2afd4d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f674c097820 tx=0x7f674c0bb120 comp rx=0 tx=0).stop 2024-03-21T11:45:37.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.306+0000 7f676581d740 1 -- 172.21.15.47:0/1662547797 shutdown_connections 2024-03-21T11:45:37.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.306+0000 7f676581d740 1 --2- 172.21.15.47:0/1662547797 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x557ec2b6ae50 0x557ec2b6d270 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.306+0000 7f676581d740 1 --2- 172.21.15.47:0/1662547797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ec2af13a0 0x557ec2afd4d0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.306+0000 7f676581d740 1 --2- 172.21.15.47:0/1662547797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ec2aef810 0x557ec2b00ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.306+0000 7f676581d740 1 --2- 172.21.15.47:0/1662547797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ec2aeac00 0x557ec2b009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.306+0000 7f6745600640 0 client.32578 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:45:37.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.306+0000 7f676581d740 1 -- 172.21.15.47:0/1662547797 >> 172.21.15.47:0/1662547797 conn(0x557ec2af95f0 msgr2=0x557ec2ae9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.306+0000 7f676581d740 1 -- 172.21.15.47:0/1662547797 shutdown_connections 2024-03-21T11:45:37.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.306+0000 7f676581d740 1 -- 172.21.15.47:0/1662547797 wait complete. 2024-03-21T11:45:37.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.306+0000 7f676581d740 20 client.32578 trim_cache size 0 max 16384 2024-03-21T11:45:37.308 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:45:37.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.308+0000 7f6762c00640 1 -- 172.21.15.47:0/1862430666 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f67400a2ff0 msgr2=0x7f67400a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.308+0000 7f6762c00640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f67400a2ff0 0x7f67400a54b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x557ec2af08e0 tx=0x7f6748005a20 comp rx=0 tx=0).stop 2024-03-21T11:45:37.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.308+0000 7f6762c00640 1 -- 172.21.15.47:0/1862430666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1018d0 msgr2=0x7f675c19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.308+0000 7f6762c00640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1018d0 0x7f675c19cd60 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f674c00a870 tx=0x7f674c00ad30 comp rx=0 tx=0).stop 2024-03-21T11:45:37.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.308+0000 7f6762c00640 1 -- 172.21.15.47:0/1862430666 shutdown_connections 2024-03-21T11:45:37.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.308+0000 7f6762c00640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f67400a2ff0 0x7f67400a54b0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.308+0000 7f6762c00640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f675c1018d0 0x7f675c19cd60 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.308+0000 7f6762c00640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f675c100f10 0x7f675c195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.308+0000 7f6762c00640 1 --2- 172.21.15.47:0/1862430666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f675c0ffd20 0x7f675c1957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.308+0000 7f6762c00640 1 -- 172.21.15.47:0/1862430666 >> 172.21.15.47:0/1862430666 conn(0x7f675c0fb290 msgr2=0x7f675c104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.308+0000 7f6762c00640 1 -- 172.21.15.47:0/1862430666 shutdown_connections 2024-03-21T11:45:37.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.308+0000 7f6762c00640 1 -- 172.21.15.47:0/1862430666 wait complete. 2024-03-21T11:45:37.333 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:45:37.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:45:37.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.560+0000 7f0b64000640 1 -- 172.21.15.47:0/2914668957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b5c0740a0 msgr2=0x7f0b5c074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.560+0000 7f0b64000640 1 --2- 172.21.15.47:0/2914668957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b5c0740a0 0x7f0b5c074500 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009900 tx=0x7f0b4802bcc0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.561+0000 7f0b64000640 1 -- 172.21.15.47:0/2914668957 shutdown_connections 2024-03-21T11:45:37.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.561+0000 7f0b64000640 1 --2- 172.21.15.47:0/2914668957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b5c109540 0x7f0b5c10b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.561+0000 7f0b64000640 1 --2- 172.21.15.47:0/2914668957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b5c0740a0 0x7f0b5c074500 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.561+0000 7f0b64000640 1 --2- 172.21.15.47:0/2914668957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b5c075680 0x7f0b5c073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.561+0000 7f0b64000640 1 -- 172.21.15.47:0/2914668957 >> 172.21.15.47:0/2914668957 conn(0x7f0b5c0fcc20 msgr2=0x7f0b5c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.561+0000 7f0b64000640 1 -- 172.21.15.47:0/2914668957 shutdown_connections 2024-03-21T11:45:37.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.561+0000 7f0b64000640 1 -- 172.21.15.47:0/2914668957 wait complete. 2024-03-21T11:45:37.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.562+0000 7f0b64000640 1 Processor -- start 2024-03-21T11:45:37.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.562+0000 7f0b64000640 1 -- start start 2024-03-21T11:45:37.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.562+0000 7f0b64000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b5c0740a0 0x7f0b5c072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.562+0000 7f0b64000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b5c075680 0x7f0b5c073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.562+0000 7f0b64000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b5c109540 0x7f0b5c071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.562+0000 7f0b64000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c06bb10 con 0x7f0b5c109540 2024-03-21T11:45:37.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.563+0000 7f0b64000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c06b990 con 0x7f0b5c0740a0 2024-03-21T11:45:37.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.563+0000 7f0b64000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c06bc90 con 0x7f0b5c075680 2024-03-21T11:45:37.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.563+0000 7f0b61600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b5c0740a0 0x7f0b5c072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.563+0000 7f0b61600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b5c0740a0 0x7f0b5c072cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41930/0 (socket says 172.21.15.47:41930) 2024-03-21T11:45:37.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.563+0000 7f0b61600640 1 -- 172.21.15.47:0/646400893 learned_addr learned my addr 172.21.15.47:0/646400893 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:37.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.563+0000 7f0b60c00640 1 --2- 172.21.15.47:0/646400893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b5c075680 0x7f0b5c073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.563+0000 7f0b62000640 1 --2- 172.21.15.47:0/646400893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b5c109540 0x7f0b5c071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.563+0000 7f0b60c00640 1 -- 172.21.15.47:0/646400893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b5c0740a0 msgr2=0x7f0b5c072cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.563+0000 7f0b60c00640 1 --2- 172.21.15.47:0/646400893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b5c0740a0 0x7f0b5c072cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.563+0000 7f0b60c00640 1 -- 172.21.15.47:0/646400893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b5c109540 msgr2=0x7f0b5c071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.563+0000 7f0b60c00640 1 --2- 172.21.15.47:0/646400893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b5c109540 0x7f0b5c071310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.563+0000 7f0b60c00640 1 -- 172.21.15.47:0/646400893 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c071ae0 con 0x7f0b5c075680 2024-03-21T11:45:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.564+0000 7f0b60c00640 1 --2- 172.21.15.47:0/646400893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b5c075680 0x7f0b5c073210 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f0b48036040 tx=0x7f0b4802c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.564+0000 7f0b5a000640 1 -- 172.21.15.47:0/646400893 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48043020 con 0x7f0b5c075680 2024-03-21T11:45:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.564+0000 7f0b64000640 1 -- 172.21.15.47:0/646400893 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5c071d70 con 0x7f0b5c075680 2024-03-21T11:45:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.564+0000 7f0b5a000640 1 -- 172.21.15.47:0/646400893 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b480054e0 con 0x7f0b5c075680 2024-03-21T11:45:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.564+0000 7f0b5a000640 1 -- 172.21.15.47:0/646400893 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48003b20 con 0x7f0b5c075680 2024-03-21T11:45:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.565+0000 7f0b64000640 1 -- 172.21.15.47:0/646400893 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5c1aa880 con 0x7f0b5c075680 2024-03-21T11:45:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.568+0000 7f0b66df7740 1 Processor -- start 2024-03-21T11:45:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.568+0000 7f0b66df7740 1 -- start start 2024-03-21T11:45:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.568+0000 7f0b66df7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d7810 0x55f9636d7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.568+0000 7f0b66df7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d89e0 0x55f9636d8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.568+0000 7f0b66df7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d93a0 0x55f9636dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.568+0000 7f0b66df7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96363c2c0 con 0x55f9636d89e0 2024-03-21T11:45:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.568+0000 7f0b66df7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9635e4c90 con 0x55f9636d7810 2024-03-21T11:45:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.568+0000 7f0b66df7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f963638360 con 0x55f9636d93a0 2024-03-21T11:45:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.569+0000 7f0b62000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d7810 0x55f9636d7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.569+0000 7f0b62000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d7810 0x55f9636d7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41942/0 (socket says 172.21.15.47:41942) 2024-03-21T11:45:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.569+0000 7f0b62000640 1 -- 172.21.15.47:0/2165594184 learned_addr learned my addr 172.21.15.47:0/2165594184 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.569+0000 7f0b61600640 1 --2- 172.21.15.47:0/2165594184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d89e0 0x55f9636d8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.569+0000 7f0b5a000640 1 -- 172.21.15.47:0/646400893 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0b48046020 con 0x7f0b5c075680 2024-03-21T11:45:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.569+0000 7f0b62000640 1 -- 172.21.15.47:0/2165594184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d93a0 msgr2=0x55f9636dfc30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:45:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.569+0000 7f0b62000640 1 --2- 172.21.15.47:0/2165594184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d93a0 0x55f9636dfc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.570+0000 7f0b62000640 1 -- 172.21.15.47:0/2165594184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d89e0 msgr2=0x55f9636d8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.570+0000 7f0b62000640 1 --2- 172.21.15.47:0/2165594184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d89e0 0x55f9636d8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.570+0000 7f0b62000640 1 -- 172.21.15.47:0/2165594184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f963609080 con 0x55f9636d7810 2024-03-21T11:45:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.570+0000 7f0b62000640 1 --2- 172.21.15.47:0/2165594184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d7810 0x55f9636d7c30 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009fd0 tx=0x7f0b540118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bb80c5cf187ea70 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.571+0000 7f0b4e600640 1 -- 172.21.15.47:0/2165594184 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b54004d20 con 0x55f9636d7810 2024-03-21T11:45:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.571+0000 7f0b4e600640 1 -- 172.21.15.47:0/2165594184 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b54004050 con 0x55f9636d7810 2024-03-21T11:45:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.571+0000 7f0b4e600640 1 -- 172.21.15.47:0/2165594184 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b54005260 con 0x55f9636d7810 2024-03-21T11:45:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.571+0000 7f0b5a000640 1 --2- 172.21.15.47:0/646400893 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0b440a2f20 0x7f0b440a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.571+0000 7f0b5a000640 1 -- 172.21.15.47:0/646400893 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0b480f1790 con 0x7f0b5c075680 2024-03-21T11:45:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.572+0000 7f0b66df7740 1 -- 172.21.15.47:0/2165594184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d7810 msgr2=0x55f9636d7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.572+0000 7f0b66df7740 1 --2- 172.21.15.47:0/2165594184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d7810 0x55f9636d7c30 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009fd0 tx=0x7f0b540118d0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.572+0000 7f0b66df7740 1 -- 172.21.15.47:0/2165594184 shutdown_connections 2024-03-21T11:45:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.572+0000 7f0b66df7740 1 --2- 172.21.15.47:0/2165594184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d93a0 0x55f9636dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.572+0000 7f0b66df7740 1 --2- 172.21.15.47:0/2165594184 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d89e0 0x55f9636d8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.572+0000 7f0b66df7740 1 --2- 172.21.15.47:0/2165594184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d7810 0x55f9636d7c30 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.572+0000 7f0b66df7740 1 -- 172.21.15.47:0/2165594184 >> 172.21.15.47:0/2165594184 conn(0x55f9636d2c00 msgr2=0x55f9636d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.572+0000 7f0b66df7740 1 -- 172.21.15.47:0/2165594184 shutdown_connections 2024-03-21T11:45:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.572+0000 7f0b66df7740 1 -- 172.21.15.47:0/2165594184 wait complete. 2024-03-21T11:45:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.573+0000 7f0b66df7740 1 -- start start 2024-03-21T11:45:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.573+0000 7f0b61600640 1 --2- 172.21.15.47:0/646400893 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0b440a2f20 0x7f0b440a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.573+0000 7f0b66df7740 1 Processor -- start 2024-03-21T11:45:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.574+0000 7f0b66df7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d7810 0x55f9636e1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.574+0000 7f0b66df7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d89e0 0x55f9636e20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.574+0000 7f0b66df7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d93a0 0x55f9636e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.574+0000 7f0b66df7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9636e03b0 con 0x55f9636d7810 2024-03-21T11:45:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.574+0000 7f0b66df7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9635f80d0 con 0x55f9636d89e0 2024-03-21T11:45:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.574+0000 7f0b66df7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9636e0530 con 0x55f9636d93a0 2024-03-21T11:45:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.574+0000 7f0b66df7740 10 client.0 _ll_register_callbacks cb 0x55f9635fce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.574+0000 7f0b61600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d7810 0x55f9636e1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.574+0000 7f0b60c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d89e0 0x55f9636e20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.574+0000 7f0b61600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d7810 0x55f9636e1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35510/0 (socket says 172.21.15.47:35510) 2024-03-21T11:45:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.574+0000 7f0b61600640 1 -- 172.21.15.47:0/3508557557 learned_addr learned my addr 172.21.15.47:0/3508557557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.575+0000 7f0b61600640 1 --2- 172.21.15.47:0/646400893 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0b440a2f20 0x7f0b440a53e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55f9636d3b70 tx=0x7f0b50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.575+0000 7f0b62000640 1 --2- 172.21.15.47:0/3508557557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d93a0 0x55f9636e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.575+0000 7f0b60c00640 1 -- 172.21.15.47:0/3508557557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d93a0 msgr2=0x55f9636e90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.575+0000 7f0b60c00640 1 --2- 172.21.15.47:0/3508557557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d93a0 0x55f9636e90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.575+0000 7f0b60c00640 1 -- 172.21.15.47:0/3508557557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d7810 msgr2=0x55f9636e1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.575+0000 7f0b60c00640 1 --2- 172.21.15.47:0/3508557557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d7810 0x55f9636e1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.575+0000 7f0b60c00640 1 -- 172.21.15.47:0/3508557557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9633503f0 con 0x55f9636d89e0 2024-03-21T11:45:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.575+0000 7f0b60c00640 1 --2- 172.21.15.47:0/3508557557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d89e0 0x55f9636e20a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009900 tx=0x7f0b4802bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b769ed8cb9309b16 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.576+0000 7f0b42a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:45:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.576+0000 7f0b42a00640 1 -- 172.21.15.47:0/3508557557 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b480bdad0 con 0x55f9636d89e0 2024-03-21T11:45:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.576+0000 7f0b42a00640 1 -- 172.21.15.47:0/3508557557 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0b48003cc0 con 0x55f9636d89e0 2024-03-21T11:45:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.576+0000 7f0b66df7740 1 -- 172.21.15.47:0/3508557557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b54004050 con 0x55f9636d89e0 2024-03-21T11:45:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.576+0000 7f0b42a00640 1 -- 172.21.15.47:0/3508557557 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b480c1c20 con 0x55f9636d89e0 2024-03-21T11:45:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.576+0000 7f0b66df7740 10 client.32606 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.576+0000 7f0b66df7740 1 -- 172.21.15.47:0/3508557557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f96341a7d0 con 0x55f9636d89e0 2024-03-21T11:45:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.577+0000 7f0b42a00640 1 -- 172.21.15.47:0/3508557557 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b54004050 con 0x55f9636d89e0 2024-03-21T11:45:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.577+0000 7f0b42a00640 1 -- 172.21.15.47:0/3508557557 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0b480c60e0 con 0x55f9636d89e0 2024-03-21T11:45:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.578+0000 7f0b66df7740 10 client.32606 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.578+0000 7f0b66df7740 10 client.32606 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:45:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.578+0000 7f0b66df7740 20 client.32606 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.578+0000 7f0b66df7740 1 --2- 172.21.15.47:0/3508557557 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f9636f5e40 0x55f9636f8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.578+0000 7f0b66df7740 4 client.32606 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.578+0000 7f0b66df7740 1 -- 172.21.15.47:0/3508557557 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f963649800 con 0x55f9636f5e40 2024-03-21T11:45:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.578+0000 7f0b61600640 1 --2- 172.21.15.47:0/3508557557 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f9636f5e40 0x55f9636f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.579+0000 7f0b61600640 1 --2- 172.21.15.47:0/3508557557 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f9636f5e40 0x55f9636f8260 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75616392441c807f server_cookie=abc3eed369dc0104 in_seq=0 out_seq=0 2024-03-21T11:45:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.579+0000 7f0b42a00640 10 client.32606 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:45:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.582+0000 7f0b42a00640 1 -- 172.21.15.47:0/3508557557 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f963649800 con 0x55f9636f5e40 2024-03-21T11:45:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.582+0000 7f0b42a00640 10 client.32606 handle_command_reply: tid=0 2024-03-21T11:45:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.582+0000 7f0b66df7740 1 client.32606 shutdown 2024-03-21T11:45:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.582+0000 7f0b66df7740 1 -- 172.21.15.47:0/3508557557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d89e0 msgr2=0x55f9636e20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.582+0000 7f0b66df7740 1 --2- 172.21.15.47:0/3508557557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d89e0 0x55f9636e20a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009900 tx=0x7f0b4802bcc0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.583+0000 7f0b66df7740 1 -- 172.21.15.47:0/3508557557 shutdown_connections 2024-03-21T11:45:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.583+0000 7f0b66df7740 1 --2- 172.21.15.47:0/3508557557 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f9636f5e40 0x55f9636f8260 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.583+0000 7f0b66df7740 1 --2- 172.21.15.47:0/3508557557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d93a0 0x55f9636e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.583+0000 7f0b66df7740 1 --2- 172.21.15.47:0/3508557557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d89e0 0x55f9636e20a0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.583+0000 7f0b66df7740 1 --2- 172.21.15.47:0/3508557557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d7810 0x55f9636e1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.583+0000 7f0b66df7740 1 -- 172.21.15.47:0/3508557557 >> 172.21.15.47:0/3508557557 conn(0x55f9636d2c00 msgr2=0x55f9636d5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.583+0000 7f0b42a00640 0 client.32606 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:45:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.583+0000 7f0b66df7740 1 -- 172.21.15.47:0/3508557557 shutdown_connections 2024-03-21T11:45:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.583+0000 7f0b66df7740 1 -- 172.21.15.47:0/3508557557 wait complete. 2024-03-21T11:45:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.583+0000 7f0b66df7740 20 client.32606 trim_cache size 0 max 16384 2024-03-21T11:45:37.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.610+0000 7f0b66df7740 1 Processor -- start 2024-03-21T11:45:37.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.611+0000 7f0b66df7740 1 -- start start 2024-03-21T11:45:37.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.611+0000 7f0b66df7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d7810 0x55f9636d6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.611+0000 7f0b66df7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d89e0 0x55f9636e9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.611+0000 7f0b66df7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d93a0 0x55f9636e1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.611+0000 7f0b66df7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c1b0700 con 0x55f9636d89e0 2024-03-21T11:45:37.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.611+0000 7f0b66df7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c1b2080 con 0x55f9636d93a0 2024-03-21T11:45:37.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.611+0000 7f0b66df7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c1b0d60 con 0x55f9636d7810 2024-03-21T11:45:37.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.612+0000 7f0b62000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d93a0 0x55f9636e1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.612+0000 7f0b62000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d93a0 0x55f9636e1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41958/0 (socket says 172.21.15.47:41958) 2024-03-21T11:45:37.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.612+0000 7f0b62000640 1 -- 172.21.15.47:0/16248737 learned_addr learned my addr 172.21.15.47:0/16248737 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:37.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.612+0000 7f0b62000640 1 -- 172.21.15.47:0/16248737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d7810 msgr2=0x55f9636d6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.612+0000 7f0b62000640 1 --2- 172.21.15.47:0/16248737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d7810 0x55f9636d6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.612+0000 7f0b62000640 1 -- 172.21.15.47:0/16248737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d89e0 msgr2=0x55f9636e9090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:45:37.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.613+0000 7f0b60c00640 1 --2- 172.21.15.47:0/16248737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d89e0 0x55f9636e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.613+0000 7f0b62000640 1 --2- 172.21.15.47:0/16248737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d89e0 0x55f9636e9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.613+0000 7f0b62000640 1 -- 172.21.15.47:0/16248737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9636d3650 con 0x55f9636d93a0 2024-03-21T11:45:37.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.613+0000 7f0b62000640 1 --2- 172.21.15.47:0/16248737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d93a0 0x55f9636e1220 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009f70 tx=0x7f0b54009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81992ad3d732ec1d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.614+0000 7f0b4dc00640 1 -- 172.21.15.47:0/16248737 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b54003b70 con 0x55f9636d93a0 2024-03-21T11:45:37.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.614+0000 7f0b4dc00640 1 -- 172.21.15.47:0/16248737 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b54003d10 con 0x55f9636d93a0 2024-03-21T11:45:37.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.614+0000 7f0b4dc00640 1 -- 172.21.15.47:0/16248737 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5400f9e0 con 0x55f9636d93a0 2024-03-21T11:45:37.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.614+0000 7f0b66df7740 1 -- 172.21.15.47:0/16248737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d93a0 msgr2=0x55f9636e1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.614+0000 7f0b66df7740 1 --2- 172.21.15.47:0/16248737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d93a0 0x55f9636e1220 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009f70 tx=0x7f0b54009be0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.615+0000 7f0b66df7740 1 -- 172.21.15.47:0/16248737 shutdown_connections 2024-03-21T11:45:37.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.615+0000 7f0b66df7740 1 --2- 172.21.15.47:0/16248737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d93a0 0x55f9636e1220 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.615+0000 7f0b66df7740 1 --2- 172.21.15.47:0/16248737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d89e0 0x55f9636e9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.615+0000 7f0b66df7740 1 --2- 172.21.15.47:0/16248737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d7810 0x55f9636d6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.615+0000 7f0b66df7740 1 -- 172.21.15.47:0/16248737 >> 172.21.15.47:0/16248737 conn(0x55f9636d2c00 msgr2=0x55f9636d5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.615+0000 7f0b66df7740 1 -- 172.21.15.47:0/16248737 shutdown_connections 2024-03-21T11:45:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.615+0000 7f0b66df7740 1 -- 172.21.15.47:0/16248737 wait complete. 2024-03-21T11:45:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.616+0000 7f0b66df7740 1 -- start start 2024-03-21T11:45:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.616+0000 7f0b66df7740 1 Processor -- start 2024-03-21T11:45:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.617+0000 7f0b66df7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d7810 0x55f9636e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.617+0000 7f0b66df7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d89e0 0x55f9636e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.617+0000 7f0b66df7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d93a0 0x55f9636e7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.617+0000 7f0b66df7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c1b1a20 con 0x55f9636d7810 2024-03-21T11:45:37.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.617+0000 7f0b66df7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c1b13c0 con 0x55f9636d93a0 2024-03-21T11:45:37.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.617+0000 7f0b66df7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c1ae8c0 con 0x55f9636d89e0 2024-03-21T11:45:37.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.617+0000 7f0b66df7740 10 client.0 _ll_register_callbacks cb 0x7f0b48005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:37.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.617+0000 7f0b60c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d89e0 0x55f9636e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.617+0000 7f0b61600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d7810 0x55f9636e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.617+0000 7f0b60c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d89e0 0x55f9636e6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47340/0 (socket says 172.21.15.47:47340) 2024-03-21T11:45:37.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.617+0000 7f0b60c00640 1 -- 172.21.15.47:0/1711443141 learned_addr learned my addr 172.21.15.47:0/1711443141 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:37.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.618+0000 7f0b62000640 1 --2- 172.21.15.47:0/1711443141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d93a0 0x55f9636e7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.618+0000 7f0b61600640 1 -- 172.21.15.47:0/1711443141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d89e0 msgr2=0x55f9636e6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.618+0000 7f0b61600640 1 --2- 172.21.15.47:0/1711443141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d89e0 0x55f9636e6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.618+0000 7f0b61600640 1 -- 172.21.15.47:0/1711443141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d93a0 msgr2=0x55f9636e7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.618+0000 7f0b61600640 1 --2- 172.21.15.47:0/1711443141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d93a0 0x55f9636e7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.618+0000 7f0b61600640 1 -- 172.21.15.47:0/1711443141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9636e85a0 con 0x55f9636d7810 2024-03-21T11:45:37.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.618+0000 7f0b62000640 1 --2- 172.21.15.47:0/1711443141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d93a0 0x55f9636e7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:45:37.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.618+0000 7f0b60c00640 1 --2- 172.21.15.47:0/1711443141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d89e0 0x55f9636e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:37.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.619+0000 7f0b61600640 1 --2- 172.21.15.47:0/1711443141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d7810 0x55f9636e6a60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55f9636f8840 tx=0x7f0b50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e14455d915d94de server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.619+0000 7f0b42a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:45:37.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.620+0000 7f0b42a00640 1 -- 172.21.15.47:0/1711443141 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5002bb90 con 0x55f9636d7810 2024-03-21T11:45:37.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.620+0000 7f0b42a00640 1 -- 172.21.15.47:0/1711443141 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0b5002bd30 con 0x55f9636d7810 2024-03-21T11:45:37.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.620+0000 7f0b66df7740 1 -- 172.21.15.47:0/1711443141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b54003d10 con 0x55f9636d7810 2024-03-21T11:45:37.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.621+0000 7f0b42a00640 1 -- 172.21.15.47:0/1711443141 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b50039050 con 0x55f9636d7810 2024-03-21T11:45:37.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.621+0000 7f0b66df7740 10 client.21852 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:37.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.621+0000 7f0b66df7740 1 -- 172.21.15.47:0/1711443141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f963751a90 con 0x55f9636d7810 2024-03-21T11:45:37.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.622+0000 7f0b42a00640 1 -- 172.21.15.47:0/1711443141 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b54003d10 con 0x55f9636d7810 2024-03-21T11:45:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.623+0000 7f0b42a00640 1 -- 172.21.15.47:0/1711443141 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0b500035f0 con 0x55f9636d7810 2024-03-21T11:45:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.623+0000 7f0b66df7740 10 client.21852 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.623+0000 7f0b66df7740 10 client.21852 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:45:37.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.623+0000 7f0b66df7740 20 client.21852 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:37.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.623+0000 7f0b66df7740 1 --2- 172.21.15.47:0/1711443141 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f9637523e0 0x55f9637547d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.623+0000 7f0b66df7740 4 client.21852 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:45:37.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.623+0000 7f0b66df7740 1 -- 172.21.15.47:0/1711443141 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f9636dffc0 con 0x55f9637523e0 2024-03-21T11:45:37.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.624+0000 7f0b60c00640 1 --2- 172.21.15.47:0/1711443141 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f9637523e0 0x55f9637547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.624+0000 7f0b60c00640 1 --2- 172.21.15.47:0/1711443141 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f9637523e0 0x55f9637547d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69ac2c4da6722b8a server_cookie=4b642bdd8d685a75 in_seq=0 out_seq=0 2024-03-21T11:45:37.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.624+0000 7f0b42a00640 10 client.21852 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:45:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.626+0000 7f0b42a00640 1 -- 172.21.15.47:0/1711443141 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f9636dffc0 con 0x55f9637523e0 2024-03-21T11:45:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.626+0000 7f0b42a00640 10 client.21852 handle_command_reply: tid=0 2024-03-21T11:45:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.626+0000 7f0b66df7740 1 client.21852 shutdown 2024-03-21T11:45:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.627+0000 7f0b66df7740 1 -- 172.21.15.47:0/1711443141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d7810 msgr2=0x55f9636e6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.627+0000 7f0b66df7740 1 --2- 172.21.15.47:0/1711443141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d7810 0x55f9636e6a60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55f9636f8840 tx=0x7f0b50004730 comp rx=0 tx=0).stop 2024-03-21T11:45:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.627+0000 7f0b66df7740 1 -- 172.21.15.47:0/1711443141 shutdown_connections 2024-03-21T11:45:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.627+0000 7f0b66df7740 1 --2- 172.21.15.47:0/1711443141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f9636d93a0 0x55f9636e7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.627+0000 7f0b66df7740 1 --2- 172.21.15.47:0/1711443141 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55f9637523e0 0x55f9637547d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.627+0000 7f0b66df7740 1 --2- 172.21.15.47:0/1711443141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9636d89e0 0x55f9636e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.627+0000 7f0b66df7740 1 --2- 172.21.15.47:0/1711443141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f9636d7810 0x55f9636e6a60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.627+0000 7f0b66df7740 1 -- 172.21.15.47:0/1711443141 >> 172.21.15.47:0/1711443141 conn(0x55f9636d2c00 msgr2=0x55f9636d66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.627+0000 7f0b42a00640 0 client.21852 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:45:37.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.627+0000 7f0b66df7740 1 -- 172.21.15.47:0/1711443141 shutdown_connections 2024-03-21T11:45:37.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.627+0000 7f0b66df7740 1 -- 172.21.15.47:0/1711443141 wait complete. 2024-03-21T11:45:37.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.627+0000 7f0b66df7740 20 client.21852 trim_cache size 0 max 16384 2024-03-21T11:45:37.629 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:45:37.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.628+0000 7f0b64000640 1 -- 172.21.15.47:0/646400893 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0b440a2f20 msgr2=0x7f0b440a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.628+0000 7f0b64000640 1 --2- 172.21.15.47:0/646400893 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0b440a2f20 0x7f0b440a53e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55f9636d3b70 tx=0x7f0b50005f90 comp rx=0 tx=0).stop 2024-03-21T11:45:37.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.628+0000 7f0b64000640 1 -- 172.21.15.47:0/646400893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b5c075680 msgr2=0x7f0b5c073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.628+0000 7f0b64000640 1 --2- 172.21.15.47:0/646400893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b5c075680 0x7f0b5c073210 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f0b48036040 tx=0x7f0b4802c6e0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.629+0000 7f0b64000640 1 -- 172.21.15.47:0/646400893 shutdown_connections 2024-03-21T11:45:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.629+0000 7f0b64000640 1 --2- 172.21.15.47:0/646400893 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0b440a2f20 0x7f0b440a53e0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.629+0000 7f0b64000640 1 --2- 172.21.15.47:0/646400893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b5c109540 0x7f0b5c071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.629+0000 7f0b64000640 1 --2- 172.21.15.47:0/646400893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b5c075680 0x7f0b5c073210 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.629+0000 7f0b64000640 1 --2- 172.21.15.47:0/646400893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b5c0740a0 0x7f0b5c072cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.629+0000 7f0b64000640 1 -- 172.21.15.47:0/646400893 >> 172.21.15.47:0/646400893 conn(0x7f0b5c0fcc20 msgr2=0x7f0b5c1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.629+0000 7f0b64000640 1 -- 172.21.15.47:0/646400893 shutdown_connections 2024-03-21T11:45:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.629+0000 7f0b64000640 1 -- 172.21.15.47:0/646400893 wait complete. 2024-03-21T11:45:37.642 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:37 smithi100 systemd[1]: ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.e.service: Scheduled restart job, restart counter is at 1. 2024-03-21T11:45:37.643 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:37 smithi100 systemd[1]: Stopped Ceph mds.e for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:45:37.643 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:37 smithi100 systemd[1]: ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.e.service: Consumed 1min 11.367s CPU time. 2024-03-21T11:45:37.643 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:37 smithi100 systemd[1]: Starting Ceph mds.e for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:45:37.657 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:45:37.657 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:45:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.934+0000 7f5fee200640 1 -- 172.21.15.47:0/987915430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fe81028c0 msgr2=0x7f5fe8102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.934+0000 7f5fee200640 1 --2- 172.21.15.47:0/987915430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fe81028c0 0x7f5fe8102d40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5fd401a910 tx=0x7f5fd403cc90 comp rx=0 tx=0).stop 2024-03-21T11:45:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.934+0000 7f5fee200640 1 -- 172.21.15.47:0/987915430 shutdown_connections 2024-03-21T11:45:37.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.934+0000 7f5fee200640 1 --2- 172.21.15.47:0/987915430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fe8103280 0x7f5fe8109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.934+0000 7f5fee200640 1 --2- 172.21.15.47:0/987915430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fe81028c0 0x7f5fe8102d40 secure :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5fd401a910 tx=0x7f5fd403cc90 comp rx=0 tx=0).stop 2024-03-21T11:45:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.934+0000 7f5fee200640 1 --2- 172.21.15.47:0/987915430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fe81016d0 0x7f5fe8101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.934+0000 7f5fee200640 1 -- 172.21.15.47:0/987915430 >> 172.21.15.47:0/987915430 conn(0x7f5fe80fcc40 msgr2=0x7f5fe80ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.934+0000 7f5fee200640 1 -- 172.21.15.47:0/987915430 shutdown_connections 2024-03-21T11:45:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.935+0000 7f5fee200640 1 -- 172.21.15.47:0/987915430 wait complete. 2024-03-21T11:45:37.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.936+0000 7f5fee200640 1 Processor -- start 2024-03-21T11:45:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.936+0000 7f5fee200640 1 -- start start 2024-03-21T11:45:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.936+0000 7f5fee200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fe81016d0 0x7f5fe818f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.936+0000 7f5fee200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fe8103280 0x7f5fe818fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.936+0000 7f5fee200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fe8194ab0 0x7f5fe8196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.936+0000 7f5fee200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe806de10 con 0x7f5fe81016d0 2024-03-21T11:45:37.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.937+0000 7f5fee200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe806dc90 con 0x7f5fe8194ab0 2024-03-21T11:45:37.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.937+0000 7f5fee200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe806df90 con 0x7f5fe8103280 2024-03-21T11:45:37.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.937+0000 7f5fe6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fe8103280 0x7f5fe818fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.937+0000 7f5fe6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fe8103280 0x7f5fe818fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47360/0 (socket says 172.21.15.47:47360) 2024-03-21T11:45:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.937+0000 7f5fe6a00640 1 -- 172.21.15.47:0/3167401490 learned_addr learned my addr 172.21.15.47:0/3167401490 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.937+0000 7f5fe7e00640 1 --2- 172.21.15.47:0/3167401490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fe8194ab0 0x7f5fe8196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.937+0000 7f5fe6a00640 1 -- 172.21.15.47:0/3167401490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fe8194ab0 msgr2=0x7f5fe8196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.937+0000 7f5fe6a00640 1 --2- 172.21.15.47:0/3167401490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fe8194ab0 0x7f5fe8196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.937+0000 7f5fe6a00640 1 -- 172.21.15.47:0/3167401490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fe81016d0 msgr2=0x7f5fe818f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.937+0000 7f5fe6a00640 1 --2- 172.21.15.47:0/3167401490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fe81016d0 0x7f5fe818f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.937+0000 7f5fe6a00640 1 -- 172.21.15.47:0/3167401490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe81975a0 con 0x7f5fe8103280 2024-03-21T11:45:37.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.938+0000 7f5fe6a00640 1 --2- 172.21.15.47:0/3167401490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fe8103280 0x7f5fe818fbd0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f5fd403d2a0 tx=0x7f5fd401a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.938+0000 7f5fdfe00640 1 -- 172.21.15.47:0/3167401490 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4052020 con 0x7f5fe8103280 2024-03-21T11:45:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.938+0000 7f5fdfe00640 1 -- 172.21.15.47:0/3167401490 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fd403dd20 con 0x7f5fe8103280 2024-03-21T11:45:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.938+0000 7f5fdfe00640 1 -- 172.21.15.47:0/3167401490 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4003cd0 con 0x7f5fe8103280 2024-03-21T11:45:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.938+0000 7f5fee200640 1 -- 172.21.15.47:0/3167401490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe80738d0 con 0x7f5fe8103280 2024-03-21T11:45:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.939+0000 7f5fee200640 1 -- 172.21.15.47:0/3167401490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe8073e50 con 0x7f5fe8103280 2024-03-21T11:45:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.944+0000 7f5fdfe00640 1 -- 172.21.15.47:0/3167401490 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5fd4055020 con 0x7f5fe8103280 2024-03-21T11:45:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.945+0000 7f5ff0feb740 1 Processor -- start 2024-03-21T11:45:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.946+0000 7f5ff0feb740 1 -- start start 2024-03-21T11:45:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.947+0000 7f5ff0feb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fa020 0x55e0473fa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.947+0000 7f5ff0feb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fb1f0 0x55e0473fb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.948+0000 7f5ff0feb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fbbb0 0x55e047402440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.948+0000 7f5ff0feb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0473562c0 con 0x55e0473fbbb0 2024-03-21T11:45:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.948+0000 7f5ff0feb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0472fec90 con 0x55e0473fb1f0 2024-03-21T11:45:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.947+0000 7f5fe7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fa020 0x55e0473fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.948+0000 7f5ff0feb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e047352360 con 0x55e0473fa020 2024-03-21T11:45:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.948+0000 7f5fe7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fa020 0x55e0473fa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47374/0 (socket says 172.21.15.47:47374) 2024-03-21T11:45:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.948+0000 7f5fe7e00640 1 -- 172.21.15.47:0/1739108275 learned_addr learned my addr 172.21.15.47:0/1739108275 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.948+0000 7f5fdfe00640 1 --2- 172.21.15.47:0/3167401490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5fcc0a2fd0 0x7f5fcc0a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.948+0000 7f5fe7400640 1 --2- 172.21.15.47:0/1739108275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fb1f0 0x55e0473fb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.948+0000 7f5fe6a00640 1 --2- 172.21.15.47:0/1739108275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fbbb0 0x55e047402440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.948+0000 7f5fdfe00640 1 -- 172.21.15.47:0/3167401490 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5fd4101d10 con 0x7f5fe8103280 2024-03-21T11:45:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.948+0000 7f5fe7e00640 1 --2- 172.21.15.47:0/3167401490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5fcc0a2fd0 0x7f5fcc0a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.949+0000 7f5fe6a00640 1 -- 172.21.15.47:0/1739108275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fa020 msgr2=0x55e0473fa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.949+0000 7f5fe6a00640 1 --2- 172.21.15.47:0/1739108275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fa020 0x55e0473fa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.949+0000 7f5fe6a00640 1 -- 172.21.15.47:0/1739108275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fb1f0 msgr2=0x55e0473fb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.949+0000 7f5fe6a00640 1 --2- 172.21.15.47:0/1739108275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fb1f0 0x55e0473fb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.949+0000 7f5fe6a00640 1 -- 172.21.15.47:0/1739108275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e047323080 con 0x55e0473fbbb0 2024-03-21T11:45:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.949+0000 7f5fe7400640 1 --2- 172.21.15.47:0/1739108275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fb1f0 0x55e0473fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:45:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.949+0000 7f5fe6a00640 1 --2- 172.21.15.47:0/1739108275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fbbb0 0x55e047402440 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40ce040 tx=0x7f5fd40a0de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a839ecce45bed2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.950+0000 7f5fd3e00640 1 -- 172.21.15.47:0/1739108275 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40cdbf0 con 0x55e0473fbbb0 2024-03-21T11:45:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.950+0000 7f5fd3e00640 1 -- 172.21.15.47:0/1739108275 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fd40c5e50 con 0x55e0473fbbb0 2024-03-21T11:45:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.950+0000 7f5fd3e00640 1 -- 172.21.15.47:0/1739108275 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40cda10 con 0x55e0473fbbb0 2024-03-21T11:45:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.951+0000 7f5ff0feb740 1 -- 172.21.15.47:0/1739108275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fbbb0 msgr2=0x55e047402440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.951+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/1739108275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fbbb0 0x55e047402440 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40ce040 tx=0x7f5fd40a0de0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.951+0000 7f5fe7e00640 1 --2- 172.21.15.47:0/3167401490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5fcc0a2fd0 0x7f5fcc0a5490 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009fd0 tx=0x7f5fe0009ba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.951+0000 7f5ff0feb740 1 -- 172.21.15.47:0/1739108275 shutdown_connections 2024-03-21T11:45:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.951+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/1739108275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fbbb0 0x55e047402440 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.951+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/1739108275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fb1f0 0x55e0473fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.951+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/1739108275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fa020 0x55e0473fa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.951+0000 7f5ff0feb740 1 -- 172.21.15.47:0/1739108275 >> 172.21.15.47:0/1739108275 conn(0x55e0473f5410 msgr2=0x55e0473f5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.952+0000 7f5ff0feb740 1 -- 172.21.15.47:0/1739108275 shutdown_connections 2024-03-21T11:45:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.952+0000 7f5ff0feb740 1 -- 172.21.15.47:0/1739108275 wait complete. 2024-03-21T11:45:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.952+0000 7f5ff0feb740 1 -- start start 2024-03-21T11:45:37.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.953+0000 7f5ff0feb740 1 Processor -- start 2024-03-21T11:45:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.953+0000 7f5ff0feb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fa020 0x55e047403d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.953+0000 7f5ff0feb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fb1f0 0x55e047404250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.953+0000 7f5ff0feb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fbbb0 0x55e04740b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.953+0000 7f5ff0feb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0472fec90 con 0x55e0473fbbb0 2024-03-21T11:45:37.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.953+0000 7f5ff0feb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0473120d0 con 0x55e0473fa020 2024-03-21T11:45:37.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.953+0000 7f5ff0feb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0473562c0 con 0x55e0473fb1f0 2024-03-21T11:45:37.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.953+0000 7f5ff0feb740 10 client.0 _ll_register_callbacks cb 0x55e047316e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:37.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.954+0000 7f5fe7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fa020 0x55e047403d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.954+0000 7f5fe7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fa020 0x55e047403d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41994/0 (socket says 172.21.15.47:41994) 2024-03-21T11:45:37.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.954+0000 7f5fe7400640 1 -- 172.21.15.47:0/1900413381 learned_addr learned my addr 172.21.15.47:0/1900413381 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.954+0000 7f5fe6a00640 1 --2- 172.21.15.47:0/1900413381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fb1f0 0x55e047404250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.954+0000 7f5fe7400640 1 -- 172.21.15.47:0/1900413381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fb1f0 msgr2=0x55e047404250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.954+0000 7f5fe7400640 1 --2- 172.21.15.47:0/1900413381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fb1f0 0x55e047404250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.954+0000 7f5fe7400640 1 -- 172.21.15.47:0/1900413381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fbbb0 msgr2=0x55e04740b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.954+0000 7f5fe7400640 1 --2- 172.21.15.47:0/1900413381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fbbb0 0x55e04740b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.954+0000 7f5fe7400640 1 -- 172.21.15.47:0/1900413381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e04706a3f0 con 0x55e0473fa020 2024-03-21T11:45:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.954+0000 7f5fe7400640 1 --2- 172.21.15.47:0/1900413381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fa020 0x55e047403d10 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8007fd0 tx=0x7f5fd800d5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71e3b218eff6034b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.955+0000 7f5fd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:45:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.955+0000 7f5fd0c00640 1 -- 172.21.15.47:0/1900413381 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd800de20 con 0x55e0473fa020 2024-03-21T11:45:37.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.955+0000 7f5fd0c00640 1 -- 172.21.15.47:0/1900413381 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5fd80125f0 con 0x55e0473fa020 2024-03-21T11:45:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.955+0000 7f5ff0feb740 1 -- 172.21.15.47:0/1900413381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fd40c5e50 con 0x55e0473fa020 2024-03-21T11:45:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.957+0000 7f5fd0c00640 1 -- 172.21.15.47:0/1900413381 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fd80127f0 con 0x55e0473fa020 2024-03-21T11:45:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.957+0000 7f5ff0feb740 10 client.32621 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.957+0000 7f5ff0feb740 1 -- 172.21.15.47:0/1900413381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e0471347d0 con 0x55e0473fa020 2024-03-21T11:45:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.958+0000 7f5fd0c00640 1 -- 172.21.15.47:0/1900413381 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fd40c5e50 con 0x55e0473fa020 2024-03-21T11:45:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.958+0000 7f5fd0c00640 1 -- 172.21.15.47:0/1900413381 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5fd8013c50 con 0x55e0473fa020 2024-03-21T11:45:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.959+0000 7f5ff0feb740 10 client.32621 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.959+0000 7f5ff0feb740 10 client.32621 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:45:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.959+0000 7f5ff0feb740 20 client.32621 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:37.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.959+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/1900413381 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e04740f820 0x55e047411c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.959+0000 7f5ff0feb740 4 client.32621 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.959+0000 7f5ff0feb740 1 -- 172.21.15.47:0/1900413381 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e047363800 con 0x55e04740f820 2024-03-21T11:45:37.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.960+0000 7f5fe6a00640 1 --2- 172.21.15.47:0/1900413381 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e04740f820 0x55e047411c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.962+0000 7f5fe6a00640 1 --2- 172.21.15.47:0/1900413381 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e04740f820 0x55e047411c50 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a70bdcd95b7c525 server_cookie=c1e4cd8f2156efb6 in_seq=0 out_seq=0 2024-03-21T11:45:37.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.962+0000 7f5fd0c00640 10 client.32621 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:45:37.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.963+0000 7f5fd0c00640 1 -- 172.21.15.47:0/1900413381 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e047363800 con 0x55e04740f820 2024-03-21T11:45:37.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.964+0000 7f5fd0c00640 10 client.32621 handle_command_reply: tid=0 2024-03-21T11:45:37.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.964+0000 7f5ff0feb740 1 client.32621 shutdown 2024-03-21T11:45:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.965+0000 7f5ff0feb740 1 -- 172.21.15.47:0/1900413381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fa020 msgr2=0x55e047403d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.968+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/1900413381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fa020 0x55e047403d10 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8007fd0 tx=0x7f5fd800d5f0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.969+0000 7f5ff0feb740 1 -- 172.21.15.47:0/1900413381 shutdown_connections 2024-03-21T11:45:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.969+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/1900413381 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e04740f820 0x55e047411c50 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.969+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/1900413381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fbbb0 0x55e04740b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.969+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/1900413381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fb1f0 0x55e047404250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.969+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/1900413381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fa020 0x55e047403d10 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.969+0000 7f5fd0c00640 0 client.32621 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:45:37.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.969+0000 7f5ff0feb740 1 -- 172.21.15.47:0/1900413381 >> 172.21.15.47:0/1900413381 conn(0x55e0473f5410 msgr2=0x55e0473ebeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.970+0000 7f5ff0feb740 1 -- 172.21.15.47:0/1900413381 shutdown_connections 2024-03-21T11:45:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.970+0000 7f5ff0feb740 1 -- 172.21.15.47:0/1900413381 wait complete. 2024-03-21T11:45:37.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.971+0000 7f5ff0feb740 20 client.32621 trim_cache size 0 max 16384 2024-03-21T11:45:37.981 INFO:tasks.workunit.client.0.smithi047.stdout: 10 215317 22.22 MB/sec execute 381 sec latency 90.212 ms 2024-03-21T11:45:37.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.997+0000 7f5ff0feb740 1 Processor -- start 2024-03-21T11:45:37.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.997+0000 7f5ff0feb740 1 -- start start 2024-03-21T11:45:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.997+0000 7f5ff0feb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fa020 0x55e0473f8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.997+0000 7f5ff0feb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fb1f0 0x55e04740b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.997+0000 7f5ff0feb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fbbb0 0x55e047416550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.998+0000 7f5ff0feb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe819feb0 con 0x55e0473fb1f0 2024-03-21T11:45:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.998+0000 7f5ff0feb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81a1830 con 0x55e0473fa020 2024-03-21T11:45:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.998+0000 7f5ff0feb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81a0510 con 0x55e0473fbbb0 2024-03-21T11:45:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.998+0000 7f5fe7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fa020 0x55e0473f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.998+0000 7f5fe7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fa020 0x55e0473f8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41996/0 (socket says 172.21.15.47:41996) 2024-03-21T11:45:37.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.998+0000 7f5fe7400640 1 -- 172.21.15.47:0/3790351991 learned_addr learned my addr 172.21.15.47:0/3790351991 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:38.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.998+0000 7f5fe7400640 1 -- 172.21.15.47:0/3790351991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fbbb0 msgr2=0x55e047416550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:38.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.998+0000 7f5fe7400640 1 --2- 172.21.15.47:0/3790351991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fbbb0 0x55e047416550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.998+0000 7f5fe7400640 1 -- 172.21.15.47:0/3790351991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fb1f0 msgr2=0x55e04740b250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:45:38.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.998+0000 7f5fe7400640 1 --2- 172.21.15.47:0/3790351991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fb1f0 0x55e04740b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.998+0000 7f5fe7400640 1 -- 172.21.15.47:0/3790351991 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0473ff2a0 con 0x55e0473fa020 2024-03-21T11:45:38.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.998+0000 7f5fe7400640 1 --2- 172.21.15.47:0/3790351991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fa020 0x55e0473f8670 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8001eb0 tx=0x7f5fd8006fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d287386a0e4e0d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:38.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.999+0000 7f5fd3400640 1 -- 172.21.15.47:0/3790351991 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd800de20 con 0x55e0473fa020 2024-03-21T11:45:38.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.999+0000 7f5fd3400640 1 -- 172.21.15.47:0/3790351991 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fd8018070 con 0x55e0473fa020 2024-03-21T11:45:38.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:37.999+0000 7f5fd3400640 1 -- 172.21.15.47:0/3790351991 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd801bcd0 con 0x55e0473fa020 2024-03-21T11:45:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.000+0000 7f5ff0feb740 1 -- 172.21.15.47:0/3790351991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fa020 msgr2=0x55e0473f8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.000+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/3790351991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fa020 0x55e0473f8670 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8001eb0 tx=0x7f5fd8006fc0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.000+0000 7f5ff0feb740 1 -- 172.21.15.47:0/3790351991 shutdown_connections 2024-03-21T11:45:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.000+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/3790351991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fbbb0 0x55e047416550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.000+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/3790351991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fb1f0 0x55e04740b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.000+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/3790351991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fa020 0x55e0473f8670 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.000+0000 7f5ff0feb740 1 -- 172.21.15.47:0/3790351991 >> 172.21.15.47:0/3790351991 conn(0x55e0473f5410 msgr2=0x55e0473ebeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.002+0000 7f5ff0feb740 1 -- 172.21.15.47:0/3790351991 shutdown_connections 2024-03-21T11:45:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.002+0000 7f5ff0feb740 1 -- 172.21.15.47:0/3790351991 wait complete. 2024-03-21T11:45:38.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.003+0000 7f5ff0feb740 1 -- start start 2024-03-21T11:45:38.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.004+0000 7f5ff0feb740 1 Processor -- start 2024-03-21T11:45:38.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.004+0000 7f5ff0feb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fa020 0x55e047407410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.004+0000 7f5ff0feb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fb1f0 0x55e047407950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.004+0000 7f5ff0feb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fbbb0 0x55e04740a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.004+0000 7f5ff0feb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81a11d0 con 0x55e0473fb1f0 2024-03-21T11:45:38.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.004+0000 7f5ff0feb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81a0b70 con 0x55e0473fbbb0 2024-03-21T11:45:38.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.004+0000 7f5ff0feb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe819e070 con 0x55e0473fa020 2024-03-21T11:45:38.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.005+0000 7f5ff0feb740 10 client.0 _ll_register_callbacks cb 0x7f5fd8007b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:38.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.005+0000 7f5fe7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fa020 0x55e047407410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.005+0000 7f5fe7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fa020 0x55e047407410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47400/0 (socket says 172.21.15.47:47400) 2024-03-21T11:45:38.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.005+0000 7f5fe7400640 1 -- 172.21.15.47:0/124684740 learned_addr learned my addr 172.21.15.47:0/124684740 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:38.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.005+0000 7f5fe7e00640 1 --2- 172.21.15.47:0/124684740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fbbb0 0x55e04740a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.005+0000 7f5fe6a00640 1 --2- 172.21.15.47:0/124684740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fb1f0 0x55e047407950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.005+0000 7f5fe7400640 1 -- 172.21.15.47:0/124684740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fbbb0 msgr2=0x55e04740a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:38.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.005+0000 7f5fe7400640 1 --2- 172.21.15.47:0/124684740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fbbb0 0x55e04740a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.005+0000 7f5fe7400640 1 -- 172.21.15.47:0/124684740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fb1f0 msgr2=0x55e047407950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.005+0000 7f5fe7400640 1 --2- 172.21.15.47:0/124684740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fb1f0 0x55e047407950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.006+0000 7f5fe7400640 1 -- 172.21.15.47:0/124684740 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e04740aef0 con 0x55e0473fa020 2024-03-21T11:45:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.006+0000 7f5fe6a00640 1 --2- 172.21.15.47:0/124684740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fb1f0 0x55e047407950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:45:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.006+0000 7f5fe7400640 1 --2- 172.21.15.47:0/124684740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fa020 0x55e047407410 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8010040 tx=0x7f5fd801be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=344a417d9cfadfb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.006+0000 7f5fd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:38.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.007+0000 7f5fd0c00640 1 -- 172.21.15.47:0/124684740 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8033060 con 0x55e0473fa020 2024-03-21T11:45:38.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.007+0000 7f5fd0c00640 1 -- 172.21.15.47:0/124684740 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5fd80125f0 con 0x55e0473fa020 2024-03-21T11:45:38.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.007+0000 7f5ff0feb740 1 -- 172.21.15.47:0/124684740 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fd8018070 con 0x55e0473fa020 2024-03-21T11:45:38.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.007+0000 7f5fd0c00640 1 -- 172.21.15.47:0/124684740 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fd800de20 con 0x55e0473fa020 2024-03-21T11:45:38.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.007+0000 7f5ff0feb740 10 client.32602 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:38.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.007+0000 7f5ff0feb740 1 -- 172.21.15.47:0/124684740 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e0473eca80 con 0x55e0473fa020 2024-03-21T11:45:38.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.008+0000 7f5fd0c00640 1 -- 172.21.15.47:0/124684740 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fd8018070 con 0x55e0473fa020 2024-03-21T11:45:38.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.009+0000 7f5fd0c00640 1 -- 172.21.15.47:0/124684740 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5fd8005c40 con 0x55e0473fa020 2024-03-21T11:45:38.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.009+0000 7f5ff0feb740 10 client.32602 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:38.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.009+0000 7f5ff0feb740 10 client.32602 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:45:38.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.009+0000 7f5ff0feb740 20 client.32602 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:38.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.009+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/124684740 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e0473ed280 0x55e0473ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.009+0000 7f5ff0feb740 4 client.32602 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:45:38.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.009+0000 7f5ff0feb740 1 -- 172.21.15.47:0/124684740 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e04735db50 con 0x55e0473ed280 2024-03-21T11:45:38.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.011+0000 7f5fe6a00640 1 --2- 172.21.15.47:0/124684740 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e0473ed280 0x55e0473ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.012+0000 7f5fe6a00640 1 --2- 172.21.15.47:0/124684740 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e0473ed280 0x55e0473ef6a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3dd8c791cefefc9 server_cookie=5c5f879315ff107d in_seq=0 out_seq=0 2024-03-21T11:45:38.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.012+0000 7f5fd0c00640 10 client.32602 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:45:38.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.012+0000 7f5fd0c00640 1 -- 172.21.15.47:0/124684740 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e04735db50 con 0x55e0473ed280 2024-03-21T11:45:38.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.012+0000 7f5fd0c00640 10 client.32602 handle_command_reply: tid=0 2024-03-21T11:45:38.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.012+0000 7f5ff0feb740 1 client.32602 shutdown 2024-03-21T11:45:38.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.012+0000 7f5ff0feb740 1 -- 172.21.15.47:0/124684740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fa020 msgr2=0x55e047407410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:38.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.012+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/124684740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fa020 0x55e047407410 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8010040 tx=0x7f5fd801be80 comp rx=0 tx=0).stop 2024-03-21T11:45:38.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.013+0000 7f5ff0feb740 1 -- 172.21.15.47:0/124684740 shutdown_connections 2024-03-21T11:45:38.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.013+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/124684740 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55e0473ed280 0x55e0473ef6a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.013+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/124684740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0473fbbb0 0x55e04740a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.013+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/124684740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0473fb1f0 0x55e047407950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.013+0000 7f5ff0feb740 1 --2- 172.21.15.47:0/124684740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e0473fa020 0x55e047407410 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.013+0000 7f5ff0feb740 1 -- 172.21.15.47:0/124684740 >> 172.21.15.47:0/124684740 conn(0x55e0473f5410 msgr2=0x55e0473fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:38.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.013+0000 7f5fd0c00640 0 client.32602 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:45:38.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.014+0000 7f5ff0feb740 1 -- 172.21.15.47:0/124684740 shutdown_connections 2024-03-21T11:45:38.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.014+0000 7f5ff0feb740 1 -- 172.21.15.47:0/124684740 wait complete. 2024-03-21T11:45:38.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.014+0000 7f5ff0feb740 20 client.32602 trim_cache size 0 max 16384 2024-03-21T11:45:38.016 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:45:38.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.015+0000 7f5fee200640 1 -- 172.21.15.47:0/3167401490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5fcc0a2fd0 msgr2=0x7f5fcc0a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:38.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.015+0000 7f5fee200640 1 --2- 172.21.15.47:0/3167401490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5fcc0a2fd0 0x7f5fcc0a5490 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009fd0 tx=0x7f5fe0009ba0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.015+0000 7f5fee200640 1 -- 172.21.15.47:0/3167401490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fe8103280 msgr2=0x7f5fe818fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:38.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.015+0000 7f5fee200640 1 --2- 172.21.15.47:0/3167401490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fe8103280 0x7f5fe818fbd0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f5fd403d2a0 tx=0x7f5fd401a180 comp rx=0 tx=0).stop 2024-03-21T11:45:38.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.016+0000 7f5fee200640 1 -- 172.21.15.47:0/3167401490 shutdown_connections 2024-03-21T11:45:38.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.016+0000 7f5fee200640 1 --2- 172.21.15.47:0/3167401490 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5fcc0a2fd0 0x7f5fcc0a5490 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.016+0000 7f5fee200640 1 --2- 172.21.15.47:0/3167401490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fe8194ab0 0x7f5fe8196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.016+0000 7f5fee200640 1 --2- 172.21.15.47:0/3167401490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fe8103280 0x7f5fe818fbd0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.016+0000 7f5fee200640 1 --2- 172.21.15.47:0/3167401490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fe81016d0 0x7f5fe818f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.016+0000 7f5fee200640 1 -- 172.21.15.47:0/3167401490 >> 172.21.15.47:0/3167401490 conn(0x7f5fe80fcc40 msgr2=0x7f5fe8070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:38.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.016+0000 7f5fee200640 1 -- 172.21.15.47:0/3167401490 shutdown_connections 2024-03-21T11:45:38.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.016+0000 7f5fee200640 1 -- 172.21.15.47:0/3167401490 wait complete. 2024-03-21T11:45:38.046 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:45:38.046 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:45:38.214 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:37 smithi100 podman[73613]: 2024-03-21 11:45:37.848351568 +0000 UTC m=+0.031573228 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:45:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.323+0000 7f74f2400640 1 -- 172.21.15.47:0/3579385742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74ec102000 msgr2=0x7f74ec1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.323+0000 7f74f2400640 1 --2- 172.21.15.47:0/3579385742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74ec102000 0x7f74ec1043f0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009be0 tx=0x7f74e402bef0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.323+0000 7f74f2400640 1 -- 172.21.15.47:0/3579385742 shutdown_connections 2024-03-21T11:45:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.323+0000 7f74f2400640 1 --2- 172.21.15.47:0/3579385742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74ec107260 0x7f74ec1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.323+0000 7f74f2400640 1 --2- 172.21.15.47:0/3579385742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74ec104930 0x7f74ec106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.323+0000 7f74f2400640 1 --2- 172.21.15.47:0/3579385742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74ec102000 0x7f74ec1043f0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.323+0000 7f74f2400640 1 -- 172.21.15.47:0/3579385742 >> 172.21.15.47:0/3579385742 conn(0x7f74ec0fba10 msgr2=0x7f74ec0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:38.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.324+0000 7f74f2400640 1 -- 172.21.15.47:0/3579385742 shutdown_connections 2024-03-21T11:45:38.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.324+0000 7f74f2400640 1 -- 172.21.15.47:0/3579385742 wait complete. 2024-03-21T11:45:38.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.325+0000 7f74f2400640 1 Processor -- start 2024-03-21T11:45:38.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.326+0000 7f74f2400640 1 -- start start 2024-03-21T11:45:38.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.326+0000 7f74f2400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74ec102000 0x7f74ec1956a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.326+0000 7f74f2400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74ec104930 0x7f74ec195be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.326+0000 7f74f2400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74ec107260 0x7f74ec1a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.326+0000 7f74f2400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec10d260 con 0x7f74ec104930 2024-03-21T11:45:38.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.326+0000 7f74f2400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec10d0e0 con 0x7f74ec102000 2024-03-21T11:45:38.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.326+0000 7f74f2400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec10d3e0 con 0x7f74ec107260 2024-03-21T11:45:38.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.326+0000 7f74f1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74ec102000 0x7f74ec1956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.326+0000 7f74f1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74ec107260 0x7f74ec1a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.326+0000 7f74f1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74ec107260 0x7f74ec1a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47420/0 (socket says 172.21.15.47:47420) 2024-03-21T11:45:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.326+0000 7f74f1a00640 1 -- 172.21.15.47:0/3339295901 learned_addr learned my addr 172.21.15.47:0/3339295901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.327+0000 7f74ebe00640 1 --2- 172.21.15.47:0/3339295901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74ec104930 0x7f74ec195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.327+0000 7f74f1000640 1 -- 172.21.15.47:0/3339295901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74ec107260 msgr2=0x7f74ec1a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.327+0000 7f74f1000640 1 --2- 172.21.15.47:0/3339295901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74ec107260 0x7f74ec1a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.327+0000 7f74f1000640 1 -- 172.21.15.47:0/3339295901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74ec104930 msgr2=0x7f74ec195be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.327+0000 7f74f1000640 1 --2- 172.21.15.47:0/3339295901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74ec104930 0x7f74ec195be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.327+0000 7f74f1000640 1 -- 172.21.15.47:0/3339295901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec1a5ae0 con 0x7f74ec102000 2024-03-21T11:45:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.327+0000 7f74f1a00640 1 --2- 172.21.15.47:0/3339295901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74ec107260 0x7f74ec1a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.327+0000 7f74f1000640 1 --2- 172.21.15.47:0/3339295901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74ec102000 0x7f74ec1956a0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f74e402c500 tx=0x7f74e4033580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:38.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.328+0000 7f74e9600640 1 -- 172.21.15.47:0/3339295901 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4043020 con 0x7f74ec102000 2024-03-21T11:45:38.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.328+0000 7f74e9600640 1 -- 172.21.15.47:0/3339295901 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74e4003e50 con 0x7f74ec102000 2024-03-21T11:45:38.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.328+0000 7f74f2400640 1 -- 172.21.15.47:0/3339295901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ec1a5d70 con 0x7f74ec102000 2024-03-21T11:45:38.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.328+0000 7f74e9600640 1 -- 172.21.15.47:0/3339295901 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4034690 con 0x7f74ec102000 2024-03-21T11:45:38.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.328+0000 7f74f2400640 1 -- 172.21.15.47:0/3339295901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ec1a6220 con 0x7f74ec102000 2024-03-21T11:45:38.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.333+0000 7f74e9600640 1 -- 172.21.15.47:0/3339295901 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f74e4046020 con 0x7f74ec102000 2024-03-21T11:45:38.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.335+0000 7f74e9600640 1 --2- 172.21.15.47:0/3339295901 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f74d00a3070 0x7f74d00a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.336+0000 7f74e9600640 1 -- 172.21.15.47:0/3339295901 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f74e40f1ed0 con 0x7f74ec102000 2024-03-21T11:45:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.336+0000 7f74ebe00640 1 --2- 172.21.15.47:0/3339295901 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f74d00a3070 0x7f74d00a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.337+0000 7f74ebe00640 1 --2- 172.21.15.47:0/3339295901 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f74d00a3070 0x7f74d00a5530 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f74d8005fd0 tx=0x7f74d8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:38.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.339+0000 7f74f6797740 1 Processor -- start 2024-03-21T11:45:38.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.340+0000 7f74f6797740 1 -- start start 2024-03-21T11:45:38.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.340+0000 7f74f6797740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f379f0 0x55d0b3f37e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.340+0000 7f74f6797740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f39040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.340+0000 7f74f6797740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 0x55d0b3f3fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.340+0000 7f74f6797740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b3e9c2c0 con 0x55d0b3f379f0 2024-03-21T11:45:38.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.340+0000 7f74f6797740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b3e44c90 con 0x55d0b3f38bc0 2024-03-21T11:45:38.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.340+0000 7f74f6797740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b3e98360 con 0x55d0b3f39580 2024-03-21T11:45:38.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.341+0000 7f74ebe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f39040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.341+0000 7f74ebe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f39040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42030/0 (socket says 172.21.15.47:42030) 2024-03-21T11:45:38.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.341+0000 7f74ebe00640 1 -- 172.21.15.47:0/578861757 learned_addr learned my addr 172.21.15.47:0/578861757 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:38.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.341+0000 7f74ebe00640 1 -- 172.21.15.47:0/578861757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 msgr2=0x55d0b3f3fe10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:45:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.341+0000 7f74ebe00640 1 --2- 172.21.15.47:0/578861757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 0x55d0b3f3fe10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.341+0000 7f74ebe00640 1 -- 172.21.15.47:0/578861757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f379f0 msgr2=0x55d0b3f37e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.341+0000 7f74ebe00640 1 --2- 172.21.15.47:0/578861757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f379f0 0x55d0b3f37e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.341+0000 7f74ebe00640 1 -- 172.21.15.47:0/578861757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0b3e69080 con 0x55d0b3f38bc0 2024-03-21T11:45:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.342+0000 7f74ebe00640 1 --2- 172.21.15.47:0/578861757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f39040 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f74d800b9e0 tx=0x7f74d8012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b2ff07363e80005 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:38.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.342+0000 7f74e2000640 1 -- 172.21.15.47:0/578861757 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8022070 con 0x55d0b3f38bc0 2024-03-21T11:45:38.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.343+0000 7f74e2000640 1 -- 172.21.15.47:0/578861757 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74d800fe50 con 0x55d0b3f38bc0 2024-03-21T11:45:38.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.343+0000 7f74e2000640 1 -- 172.21.15.47:0/578861757 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8022070 con 0x55d0b3f38bc0 2024-03-21T11:45:38.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.344+0000 7f74f6797740 1 -- 172.21.15.47:0/578861757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f38bc0 msgr2=0x55d0b3f39040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:38.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.345+0000 7f74f6797740 1 --2- 172.21.15.47:0/578861757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f39040 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f74d800b9e0 tx=0x7f74d8012000 comp rx=0 tx=0).stop 2024-03-21T11:45:38.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.345+0000 7f74f6797740 1 -- 172.21.15.47:0/578861757 shutdown_connections 2024-03-21T11:45:38.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.345+0000 7f74f6797740 1 --2- 172.21.15.47:0/578861757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 0x55d0b3f3fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.345+0000 7f74f6797740 1 --2- 172.21.15.47:0/578861757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f39040 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.345+0000 7f74f6797740 1 --2- 172.21.15.47:0/578861757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f379f0 0x55d0b3f37e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.345+0000 7f74f6797740 1 -- 172.21.15.47:0/578861757 >> 172.21.15.47:0/578861757 conn(0x55d0b3f32de0 msgr2=0x55d0b3f331e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:38.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.345+0000 7f74f6797740 1 -- 172.21.15.47:0/578861757 shutdown_connections 2024-03-21T11:45:38.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.345+0000 7f74f6797740 1 -- 172.21.15.47:0/578861757 wait complete. 2024-03-21T11:45:38.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.346+0000 7f74f6797740 1 -- start start 2024-03-21T11:45:38.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.346+0000 7f74f6797740 1 Processor -- start 2024-03-21T11:45:38.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.347+0000 7f74f6797740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f379f0 0x55d0b3f52ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.348+0000 7f74f6797740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f53010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.348+0000 7f74f1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f379f0 0x55d0b3f52ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.348+0000 7f74f6797740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 0x55d0b3f5a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.348+0000 7f74f1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f379f0 0x55d0b3f52ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42034/0 (socket says 172.21.15.47:42034) 2024-03-21T11:45:38.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.348+0000 7f74ebe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f53010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.348+0000 7f74ebe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f53010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35584/0 (socket says 172.21.15.47:35584) 2024-03-21T11:45:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.348+0000 7f74f1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 0x55d0b3f5a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.349+0000 7f74f1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 0x55d0b3f5a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47438/0 (socket says 172.21.15.47:47438) 2024-03-21T11:45:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.349+0000 7f74f6797740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b3f515a0 con 0x55d0b3f38bc0 2024-03-21T11:45:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.349+0000 7f74f1000640 1 -- 172.21.15.47:0/3492541614 learned_addr learned my addr 172.21.15.47:0/3492541614 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.349+0000 7f74f6797740 1 -- 172.21.15.47:0/3492541614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b3e580d0 con 0x55d0b3f379f0 2024-03-21T11:45:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.349+0000 7f74f6797740 1 -- 172.21.15.47:0/3492541614 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0b3f51720 con 0x55d0b3f39580 2024-03-21T11:45:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.349+0000 7f74f6797740 10 client.0 _ll_register_callbacks cb 0x55d0b3e5ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.349+0000 7f74f1000640 1 -- 172.21.15.47:0/3492541614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 msgr2=0x55d0b3f5a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.349+0000 7f74f1000640 1 --2- 172.21.15.47:0/3492541614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 0x55d0b3f5a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.349+0000 7f74f1000640 1 -- 172.21.15.47:0/3492541614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 msgr2=0x55d0b3f53010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.349+0000 7f74f1000640 1 --2- 172.21.15.47:0/3492541614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f53010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.349+0000 7f74f1000640 1 -- 172.21.15.47:0/3492541614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0b3bb03f0 con 0x55d0b3f379f0 2024-03-21T11:45:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.349+0000 7f74ebe00640 1 --2- 172.21.15.47:0/3492541614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f53010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:45:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.349+0000 7f74f1000640 1 --2- 172.21.15.47:0/3492541614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f379f0 0x55d0b3f52ad0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f74e40c66f0 tx=0x7f74e40bde80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3f8b22e8914b6dc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.350+0000 7f74d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:45:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.350+0000 7f74d6a00640 1 -- 172.21.15.47:0/3492541614 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e40c2120 con 0x55d0b3f379f0 2024-03-21T11:45:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.350+0000 7f74f6797740 1 -- 172.21.15.47:0/3492541614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74d800fe50 con 0x55d0b3f379f0 2024-03-21T11:45:38.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.350+0000 7f74d6a00640 1 -- 172.21.15.47:0/3492541614 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f74e40042f0 con 0x55d0b3f379f0 2024-03-21T11:45:38.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.350+0000 7f74d6a00640 1 -- 172.21.15.47:0/3492541614 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74e40be210 con 0x55d0b3f379f0 2024-03-21T11:45:38.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.350+0000 7f74f6797740 10 client.32639 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:38.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.350+0000 7f74f6797740 1 -- 172.21.15.47:0/3492541614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d0b3c7a7d0 con 0x55d0b3f379f0 2024-03-21T11:45:38.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.351+0000 7f74d6a00640 1 -- 172.21.15.47:0/3492541614 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74d800fe50 con 0x55d0b3f379f0 2024-03-21T11:45:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.351+0000 7f74d6a00640 1 -- 172.21.15.47:0/3492541614 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f74e4004b40 con 0x55d0b3f379f0 2024-03-21T11:45:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.351+0000 7f74f6797740 10 client.32639 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.351+0000 7f74f6797740 10 client.32639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.351+0000 7f74f6797740 20 client.32639 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.351+0000 7f74f6797740 1 --2- 172.21.15.47:0/3492541614 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d0b3f5e480 0x55d0b3f608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.351+0000 7f74f6797740 4 client.32639 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.351+0000 7f74f6797740 1 -- 172.21.15.47:0/3492541614 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0b3ea9800 con 0x55d0b3f5e480 2024-03-21T11:45:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.352+0000 7f74ebe00640 1 --2- 172.21.15.47:0/3492541614 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d0b3f5e480 0x55d0b3f608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.352+0000 7f74ebe00640 1 --2- 172.21.15.47:0/3492541614 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d0b3f5e480 0x55d0b3f608e0 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba17139919b602f2 server_cookie=df656128f409867d in_seq=0 out_seq=0 2024-03-21T11:45:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.352+0000 7f74d6a00640 10 client.32639 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:38.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.355+0000 7f74d6a00640 1 -- 172.21.15.47:0/3492541614 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d0b3ea9800 con 0x55d0b3f5e480 2024-03-21T11:45:38.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.355+0000 7f74d6a00640 10 client.32639 handle_command_reply: tid=0 2024-03-21T11:45:38.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.356+0000 7f74f6797740 1 client.32639 shutdown 2024-03-21T11:45:38.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.356+0000 7f74f6797740 1 -- 172.21.15.47:0/3492541614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f379f0 msgr2=0x55d0b3f52ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:38.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.356+0000 7f74f6797740 1 --2- 172.21.15.47:0/3492541614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f379f0 0x55d0b3f52ad0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f74e40c66f0 tx=0x7f74e40bde80 comp rx=0 tx=0).stop 2024-03-21T11:45:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.357+0000 7f74f6797740 1 -- 172.21.15.47:0/3492541614 shutdown_connections 2024-03-21T11:45:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.357+0000 7f74f6797740 1 --2- 172.21.15.47:0/3492541614 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d0b3f5e480 0x55d0b3f608e0 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.357+0000 7f74f6797740 1 --2- 172.21.15.47:0/3492541614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 0x55d0b3f5a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.357+0000 7f74f6797740 1 --2- 172.21.15.47:0/3492541614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f53010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.357+0000 7f74f6797740 1 --2- 172.21.15.47:0/3492541614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f379f0 0x55d0b3f52ad0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.357+0000 7f74f6797740 1 -- 172.21.15.47:0/3492541614 >> 172.21.15.47:0/3492541614 conn(0x55d0b3f32de0 msgr2=0x55d0b3f32160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:38.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.357+0000 7f74d6a00640 0 client.32639 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.357+0000 7f74f6797740 1 -- 172.21.15.47:0/3492541614 shutdown_connections 2024-03-21T11:45:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.358+0000 7f74f6797740 1 -- 172.21.15.47:0/3492541614 wait complete. 2024-03-21T11:45:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.358+0000 7f74f6797740 20 client.32639 trim_cache size 0 max 16384 2024-03-21T11:45:38.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.382+0000 7f74f6797740 1 Processor -- start 2024-03-21T11:45:38.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.383+0000 7f74f6797740 1 -- start start 2024-03-21T11:45:38.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.384+0000 7f74f6797740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f379f0 0x55d0b3f651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.384+0000 7f74f6797740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f5a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.384+0000 7f74f6797740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f39580 0x55d0b3f656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.384+0000 7f74f6797740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec1a7bb0 con 0x55d0b3f38bc0 2024-03-21T11:45:38.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.384+0000 7f74f6797740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec1a9530 con 0x55d0b3f39580 2024-03-21T11:45:38.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.384+0000 7f74f6797740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec1a8210 con 0x55d0b3f379f0 2024-03-21T11:45:38.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.384+0000 7f74f1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f379f0 0x55d0b3f651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.384+0000 7f74f1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f379f0 0x55d0b3f651a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47444/0 (socket says 172.21.15.47:47444) 2024-03-21T11:45:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.384+0000 7f74f1000640 1 -- 172.21.15.47:0/2607845800 learned_addr learned my addr 172.21.15.47:0/2607845800 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.384+0000 7f74f1a00640 1 --2- 172.21.15.47:0/2607845800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f39580 0x55d0b3f656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.385+0000 7f74f1000640 1 -- 172.21.15.47:0/2607845800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f39580 msgr2=0x55d0b3f656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.385+0000 7f74f1000640 1 --2- 172.21.15.47:0/2607845800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f39580 0x55d0b3f656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.385+0000 7f74f1000640 1 -- 172.21.15.47:0/2607845800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 msgr2=0x55d0b3f5a010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:45:38.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.385+0000 7f74f1000640 1 --2- 172.21.15.47:0/2607845800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f5a010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.385+0000 7f74f1000640 1 -- 172.21.15.47:0/2607845800 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0b3f38180 con 0x55d0b3f379f0 2024-03-21T11:45:38.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.385+0000 7f74f1000640 1 --2- 172.21.15.47:0/2607845800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f379f0 0x55d0b3f651a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f74e40babe0 tx=0x7f74e4034f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfa9d4278eab417a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:38.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.387+0000 7f74e1600640 1 -- 172.21.15.47:0/2607845800 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e40bd800 con 0x55d0b3f379f0 2024-03-21T11:45:38.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.388+0000 7f74e1600640 1 -- 172.21.15.47:0/2607845800 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74e40c2120 con 0x55d0b3f379f0 2024-03-21T11:45:38.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.388+0000 7f74e1600640 1 -- 172.21.15.47:0/2607845800 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e40042f0 con 0x55d0b3f379f0 2024-03-21T11:45:38.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.389+0000 7f74f6797740 1 -- 172.21.15.47:0/2607845800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f379f0 msgr2=0x55d0b3f651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:38.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.389+0000 7f74f6797740 1 --2- 172.21.15.47:0/2607845800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f379f0 0x55d0b3f651a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f74e40babe0 tx=0x7f74e4034f90 comp rx=0 tx=0).stop 2024-03-21T11:45:38.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.389+0000 7f74f6797740 1 -- 172.21.15.47:0/2607845800 shutdown_connections 2024-03-21T11:45:38.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.389+0000 7f74f6797740 1 --2- 172.21.15.47:0/2607845800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f39580 0x55d0b3f656e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.389+0000 7f74f6797740 1 --2- 172.21.15.47:0/2607845800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f5a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.389+0000 7f74f6797740 1 --2- 172.21.15.47:0/2607845800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f379f0 0x55d0b3f651a0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.389+0000 7f74f6797740 1 -- 172.21.15.47:0/2607845800 >> 172.21.15.47:0/2607845800 conn(0x55d0b3f32de0 msgr2=0x55d0b3f368d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:38.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.391+0000 7f74f6797740 1 -- 172.21.15.47:0/2607845800 shutdown_connections 2024-03-21T11:45:38.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.392+0000 7f74f6797740 1 -- 172.21.15.47:0/2607845800 wait complete. 2024-03-21T11:45:38.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.393+0000 7f74f6797740 1 -- start start 2024-03-21T11:45:38.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.393+0000 7f74f6797740 1 Processor -- start 2024-03-21T11:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.394+0000 7f74f6797740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f379f0 0x55d0b3f57c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.394+0000 7f74f6797740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f58180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.394+0000 7f74f6797740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 0x55d0b3f590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.394+0000 7f74f6797740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec1a8ed0 con 0x55d0b3f38bc0 2024-03-21T11:45:38.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.394+0000 7f74f6797740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec1a8870 con 0x55d0b3f379f0 2024-03-21T11:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.394+0000 7f74f6797740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec073240 con 0x55d0b3f39580 2024-03-21T11:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.394+0000 7f74f6797740 10 client.0 _ll_register_callbacks cb 0x7f74e40be0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.394+0000 7f74f1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f379f0 0x55d0b3f57c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.395+0000 7f74f1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f379f0 0x55d0b3f57c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42056/0 (socket says 172.21.15.47:42056) 2024-03-21T11:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.395+0000 7f74f1000640 1 -- 172.21.15.47:0/3457200330 learned_addr learned my addr 172.21.15.47:0/3457200330 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:38.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.395+0000 7f74f1a00640 1 --2- 172.21.15.47:0/3457200330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 0x55d0b3f590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.395+0000 7f74f1000640 1 -- 172.21.15.47:0/3457200330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 msgr2=0x55d0b3f590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.395+0000 7f74f1000640 1 --2- 172.21.15.47:0/3457200330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 0x55d0b3f590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.395+0000 7f74f1000640 1 -- 172.21.15.47:0/3457200330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 msgr2=0x55d0b3f58180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.396+0000 7f74f1000640 1 --2- 172.21.15.47:0/3457200330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f58180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.396+0000 7f74f1000640 1 -- 172.21.15.47:0/3457200330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0b3f596e0 con 0x55d0b3f379f0 2024-03-21T11:45:38.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.396+0000 7f74f1000640 1 --2- 172.21.15.47:0/3457200330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f379f0 0x55d0b3f57c40 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f74e40ba670 tx=0x7f74e40bde80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=316335d18b387904 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:38.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.396+0000 7f74d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:45:38.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.397+0000 7f74d6a00640 1 -- 172.21.15.47:0/3457200330 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e40a0070 con 0x55d0b3f379f0 2024-03-21T11:45:38.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.397+0000 7f74d6a00640 1 -- 172.21.15.47:0/3457200330 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f74e4003b70 con 0x55d0b3f379f0 2024-03-21T11:45:38.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.397+0000 7f74f6797740 1 -- 172.21.15.47:0/3457200330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74e40c2120 con 0x55d0b3f379f0 2024-03-21T11:45:38.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.397+0000 7f74d6a00640 1 -- 172.21.15.47:0/3457200330 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74e40c6c20 con 0x55d0b3f379f0 2024-03-21T11:45:38.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.397+0000 7f74f6797740 10 client.32645 fetch_fsmap learned FSMap version 40 2024-03-21T11:45:38.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.397+0000 7f74f6797740 1 -- 172.21.15.47:0/3457200330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d0b3f596e0 con 0x55d0b3f379f0 2024-03-21T11:45:38.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.398+0000 7f74d6a00640 1 -- 172.21.15.47:0/3457200330 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74e40c2120 con 0x55d0b3f379f0 2024-03-21T11:45:38.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.398+0000 7f74d6a00640 1 -- 172.21.15.47:0/3457200330 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 40) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f74e40bd0d0 con 0x55d0b3f379f0 2024-03-21T11:45:38.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.398+0000 7f74f6797740 10 client.32645 fetch_fsmap finished waiting for FSMap version 40 2024-03-21T11:45:38.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.398+0000 7f74f6797740 10 client.32645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:38.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.398+0000 7f74f6797740 20 client.32645 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:38.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.399+0000 7f74f6797740 1 --2- 172.21.15.47:0/3457200330 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d0b3fba680 0x55d0b3fbcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:38.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.399+0000 7f74f6797740 4 client.32645 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:45:38.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.399+0000 7f74f6797740 1 -- 172.21.15.47:0/3457200330 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d0b3ea3b50 con 0x55d0b3fba680 2024-03-21T11:45:38.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.399+0000 7f74ebe00640 1 --2- 172.21.15.47:0/3457200330 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d0b3fba680 0x55d0b3fbcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.400+0000 7f74ebe00640 1 --2- 172.21.15.47:0/3457200330 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d0b3fba680 0x55d0b3fbcaa0 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20f049807b126415 server_cookie=1ae8e17cf2b48263 in_seq=0 out_seq=0 2024-03-21T11:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.400+0000 7f74d6a00640 10 client.32645 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.400+0000 7f74d6a00640 1 -- 172.21.15.47:0/3457200330 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d0b3ea3b50 con 0x55d0b3fba680 2024-03-21T11:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.400+0000 7f74d6a00640 10 client.32645 handle_command_reply: tid=0 2024-03-21T11:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.400+0000 7f74f6797740 1 client.32645 shutdown 2024-03-21T11:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.401+0000 7f74f6797740 1 -- 172.21.15.47:0/3457200330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f379f0 msgr2=0x55d0b3f57c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.401+0000 7f74f6797740 1 --2- 172.21.15.47:0/3457200330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f379f0 0x55d0b3f57c40 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f74e40ba670 tx=0x7f74e40bde80 comp rx=0 tx=0).stop 2024-03-21T11:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.401+0000 7f74f6797740 1 -- 172.21.15.47:0/3457200330 shutdown_connections 2024-03-21T11:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.401+0000 7f74f6797740 1 --2- 172.21.15.47:0/3457200330 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55d0b3fba680 0x55d0b3fbcaa0 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.401+0000 7f74f6797740 1 --2- 172.21.15.47:0/3457200330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0b3f39580 0x55d0b3f590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.401+0000 7f74f6797740 1 --2- 172.21.15.47:0/3457200330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0b3f38bc0 0x55d0b3f58180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.401+0000 7f74f6797740 1 --2- 172.21.15.47:0/3457200330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0b3f379f0 0x55d0b3f57c40 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.401+0000 7f74d6a00640 0 client.32645 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.401+0000 7f74f6797740 1 -- 172.21.15.47:0/3457200330 >> 172.21.15.47:0/3457200330 conn(0x55d0b3f32de0 msgr2=0x55d0b3f3bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.401+0000 7f74f6797740 1 -- 172.21.15.47:0/3457200330 shutdown_connections 2024-03-21T11:45:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.401+0000 7f74f6797740 1 -- 172.21.15.47:0/3457200330 wait complete. 2024-03-21T11:45:38.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.401+0000 7f74f6797740 20 client.32645 trim_cache size 0 max 16384 2024-03-21T11:45:38.403 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:45:38.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.403+0000 7f74f2400640 1 -- 172.21.15.47:0/3339295901 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f74d00a3070 msgr2=0x7f74d00a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:38.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.403+0000 7f74f2400640 1 --2- 172.21.15.47:0/3339295901 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f74d00a3070 0x7f74d00a5530 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f74d8005fd0 tx=0x7f74d8005850 comp rx=0 tx=0).stop 2024-03-21T11:45:38.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.403+0000 7f74f2400640 1 -- 172.21.15.47:0/3339295901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74ec102000 msgr2=0x7f74ec1956a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:38.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.403+0000 7f74f2400640 1 --2- 172.21.15.47:0/3339295901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74ec102000 0x7f74ec1956a0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f74e402c500 tx=0x7f74e4033580 comp rx=0 tx=0).stop 2024-03-21T11:45:38.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.404+0000 7f74f2400640 1 -- 172.21.15.47:0/3339295901 shutdown_connections 2024-03-21T11:45:38.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.404+0000 7f74f2400640 1 --2- 172.21.15.47:0/3339295901 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f74d00a3070 0x7f74d00a5530 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.404+0000 7f74f2400640 1 --2- 172.21.15.47:0/3339295901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74ec107260 0x7f74ec1a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.404+0000 7f74f2400640 1 --2- 172.21.15.47:0/3339295901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74ec104930 0x7f74ec195be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.404+0000 7f74f2400640 1 --2- 172.21.15.47:0/3339295901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74ec102000 0x7f74ec1956a0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:38.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.404+0000 7f74f2400640 1 -- 172.21.15.47:0/3339295901 >> 172.21.15.47:0/3339295901 conn(0x7f74ec0fba10 msgr2=0x7f74ec1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:38.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.404+0000 7f74f2400640 1 -- 172.21.15.47:0/3339295901 shutdown_connections 2024-03-21T11:45:38.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:38.404+0000 7f74f2400640 1 -- 172.21.15.47:0/3339295901 wait complete. 2024-03-21T11:45:38.525 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:38 smithi100 podman[73613]: 2024-03-21 11:45:38.214675447 +0000 UTC m=+0.397897082 container create 9160af86b80f6309e802251f749a79657b9a77bb7d2886319068f1f08793c040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:45:38.797 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:38 smithi100 podman[73613]: 2024-03-21 11:45:38.678128172 +0000 UTC m=+0.861349821 container init 9160af86b80f6309e802251f749a79657b9a77bb7d2886319068f1f08793c040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:45:38.797 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:38 smithi100 podman[73613]: 2024-03-21 11:45:38.681845925 +0000 UTC m=+0.865067559 container start 9160af86b80f6309e802251f749a79657b9a77bb7d2886319068f1f08793c040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-21T11:45:38.798 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:38 smithi047 ceph-mon[31351]: pgmap v1069: 129 pgs: 129 active+clean; 636 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.2 MiB/s wr, 457 op/s 2024-03-21T11:45:38.981 INFO:tasks.workunit.client.0.smithi047.stdout: 10 215807 22.22 MB/sec execute 382 sec latency 78.384 ms 2024-03-21T11:45:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:38 smithi100 ceph-mon[36245]: pgmap v1069: 129 pgs: 129 active+clean; 636 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.2 MiB/s wr, 457 op/s 2024-03-21T11:45:39.143 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:38 smithi100 bash[73613]: 9160af86b80f6309e802251f749a79657b9a77bb7d2886319068f1f08793c040 2024-03-21T11:45:39.143 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:38 smithi100 systemd[1]: Started Ceph mds.e for a6f35682-e773-11ee-95cd-87774f69a715. 2024-03-21T11:45:39.143 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:45:38 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e[73627]: starting mds.e at 2024-03-21T11:45:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:38 smithi102 ceph-mon[36748]: pgmap v1069: 129 pgs: 129 active+clean; 636 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.2 MiB/s wr, 457 op/s 2024-03-21T11:45:39.429 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:45:39.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:45:39.429 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:45:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.690+0000 7f93fbe00640 1 -- 172.21.15.47:0/2121953071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93f410b010 msgr2=0x7f93f410d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.690+0000 7f93fbe00640 1 --2- 172.21.15.47:0/2121953071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93f410b010 0x7f93f410d480 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009900 tx=0x7f93e002bc30 comp rx=0 tx=0).stop 2024-03-21T11:45:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.690+0000 7f93fbe00640 1 -- 172.21.15.47:0/2121953071 shutdown_connections 2024-03-21T11:45:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.690+0000 7f93fbe00640 1 --2- 172.21.15.47:0/2121953071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93f410d9c0 0x7f93f410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.690+0000 7f93fbe00640 1 --2- 172.21.15.47:0/2121953071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93f410b010 0x7f93f410d480 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.690+0000 7f93fbe00640 1 --2- 172.21.15.47:0/2121953071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f41086a0 0x7f93f410aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.690+0000 7f93fbe00640 1 -- 172.21.15.47:0/2121953071 >> 172.21.15.47:0/2121953071 conn(0x7f93f406e500 msgr2=0x7f93f4070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.690+0000 7f93fbe00640 1 -- 172.21.15.47:0/2121953071 shutdown_connections 2024-03-21T11:45:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.691+0000 7f93fbe00640 1 -- 172.21.15.47:0/2121953071 wait complete. 2024-03-21T11:45:39.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.692+0000 7f93fbe00640 1 Processor -- start 2024-03-21T11:45:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.692+0000 7f93fbe00640 1 -- start start 2024-03-21T11:45:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.693+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f41086a0 0x7f93f41a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.693+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93f410b010 0x7f93f41a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.693+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93f410d9c0 0x7f93f41add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.693+0000 7f93fbe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f411c1a0 con 0x7f93f41086a0 2024-03-21T11:45:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.693+0000 7f93fbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f411c020 con 0x7f93f410b010 2024-03-21T11:45:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.693+0000 7f93fbe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f411c320 con 0x7f93f410d9c0 2024-03-21T11:45:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.693+0000 7f93f8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93f410b010 0x7f93f41a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.693+0000 7f93f9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93f410d9c0 0x7f93f41add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.693+0000 7f93f9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93f410d9c0 0x7f93f41add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52842/0 (socket says 172.21.15.47:52842) 2024-03-21T11:45:39.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.693+0000 7f93f9e00640 1 -- 172.21.15.47:0/3401692864 learned_addr learned my addr 172.21.15.47:0/3401692864 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:39.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.693+0000 7f93f9400640 1 --2- 172.21.15.47:0/3401692864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f41086a0 0x7f93f41a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.694+0000 7f93f9e00640 1 -- 172.21.15.47:0/3401692864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93f410b010 msgr2=0x7f93f41a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:39.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.694+0000 7f93f9e00640 1 --2- 172.21.15.47:0/3401692864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93f410b010 0x7f93f41a6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.694+0000 7f93f9e00640 1 -- 172.21.15.47:0/3401692864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f41086a0 msgr2=0x7f93f41a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:39.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.694+0000 7f93f9e00640 1 --2- 172.21.15.47:0/3401692864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f41086a0 0x7f93f41a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.694+0000 7f93f9e00640 1 -- 172.21.15.47:0/3401692864 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f41ae400 con 0x7f93f410d9c0 2024-03-21T11:45:39.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.694+0000 7f93f9400640 1 --2- 172.21.15.47:0/3401692864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f41086a0 0x7f93f41a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:45:39.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.694+0000 7f93f9e00640 1 --2- 172.21.15.47:0/3401692864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93f410d9c0 0x7f93f41add90 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00eb40 tx=0x7f93ec00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:39.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.694+0000 7f93f2000640 1 -- 172.21.15.47:0/3401692864 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec004b20 con 0x7f93f410d9c0 2024-03-21T11:45:39.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.694+0000 7f93f2000640 1 -- 172.21.15.47:0/3401692864 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93ec004cc0 con 0x7f93f410d9c0 2024-03-21T11:45:39.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.694+0000 7f93f2000640 1 -- 172.21.15.47:0/3401692864 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec010850 con 0x7f93f410d9c0 2024-03-21T11:45:39.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.694+0000 7f93fbe00640 1 -- 172.21.15.47:0/3401692864 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f41ae6f0 con 0x7f93f410d9c0 2024-03-21T11:45:39.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.695+0000 7f93fbe00640 1 -- 172.21.15.47:0/3401692864 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f41aebc0 con 0x7f93f410d9c0 2024-03-21T11:45:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.698+0000 7f93feb7a740 1 Processor -- start 2024-03-21T11:45:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.698+0000 7f93f2000640 1 -- 172.21.15.47:0/3401692864 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f93ec005020 con 0x7f93f410d9c0 2024-03-21T11:45:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.698+0000 7f93feb7a740 1 -- start start 2024-03-21T11:45:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93feb7a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c5b0a0 0x5640e8c5b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93feb7a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5ba90 0x5640e8c60590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93feb7a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c60c00 0x5640e8c63090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93feb7a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e8bc0eb0 con 0x5640e8c5b0a0 2024-03-21T11:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93feb7a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e8b69a60 con 0x5640e8c5ba90 2024-03-21T11:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93feb7a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e8bbcf50 con 0x5640e8c60c00 2024-03-21T11:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93f9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5ba90 0x5640e8c60590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93f9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5ba90 0x5640e8c60590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38922/0 (socket says 172.21.15.47:38922) 2024-03-21T11:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93f9400640 1 -- 172.21.15.47:0/3599336581 learned_addr learned my addr 172.21.15.47:0/3599336581 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93f8a00640 1 --2- 172.21.15.47:0/3599336581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c5b0a0 0x5640e8c5b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93f8a00640 1 --2- 172.21.15.47:0/3599336581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c60c00 0x5640e8c63090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93f9400640 1 -- 172.21.15.47:0/3599336581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c60c00 msgr2=0x5640e8c63090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93f9400640 1 --2- 172.21.15.47:0/3599336581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c60c00 0x5640e8c63090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93f9400640 1 -- 172.21.15.47:0/3599336581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c5b0a0 msgr2=0x5640e8c5b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:39.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.699+0000 7f93f9400640 1 --2- 172.21.15.47:0/3599336581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c5b0a0 0x5640e8c5b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.700+0000 7f93f9400640 1 -- 172.21.15.47:0/3599336581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640e899d7e0 con 0x5640e8c5ba90 2024-03-21T11:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.700+0000 7f93f9400640 1 --2- 172.21.15.47:0/3599336581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5ba90 0x5640e8c60590 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f93e800cca0 tx=0x7f93e800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=472b190620449d55 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.700+0000 7f93f2000640 1 --2- 172.21.15.47:0/3401692864 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93dc0a2fa0 0x7f93dc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.700+0000 7f93e6600640 1 -- 172.21.15.47:0/3599336581 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8005220 con 0x5640e8c5ba90 2024-03-21T11:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.700+0000 7f93e6600640 1 -- 172.21.15.47:0/3599336581 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93e8004c80 con 0x5640e8c5ba90 2024-03-21T11:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.700+0000 7f93e6600640 1 -- 172.21.15.47:0/3599336581 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8010890 con 0x5640e8c5ba90 2024-03-21T11:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.700+0000 7f93f2000640 1 -- 172.21.15.47:0/3401692864 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f93ec0cff90 con 0x7f93f410d9c0 2024-03-21T11:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.700+0000 7f93f8a00640 1 --2- 172.21.15.47:0/3401692864 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93dc0a2fa0 0x7f93dc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.702+0000 7f93feb7a740 1 -- 172.21.15.47:0/3599336581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5ba90 msgr2=0x5640e8c60590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.702+0000 7f93f8a00640 1 --2- 172.21.15.47:0/3401692864 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93dc0a2fa0 0x7f93dc0a5460 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f93e002c240 tx=0x7f93e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.702+0000 7f93feb7a740 1 --2- 172.21.15.47:0/3599336581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5ba90 0x5640e8c60590 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f93e800cca0 tx=0x7f93e800e720 comp rx=0 tx=0).stop 2024-03-21T11:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.702+0000 7f93feb7a740 1 -- 172.21.15.47:0/3599336581 shutdown_connections 2024-03-21T11:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.702+0000 7f93feb7a740 1 --2- 172.21.15.47:0/3599336581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c60c00 0x5640e8c63090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.702+0000 7f93feb7a740 1 --2- 172.21.15.47:0/3599336581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5ba90 0x5640e8c60590 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.702+0000 7f93feb7a740 1 --2- 172.21.15.47:0/3599336581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c5b0a0 0x5640e8c5b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.702+0000 7f93feb7a740 1 -- 172.21.15.47:0/3599336581 >> 172.21.15.47:0/3599336581 conn(0x5640e8c56610 msgr2=0x5640e8c56a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:39.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.702+0000 7f93feb7a740 1 -- 172.21.15.47:0/3599336581 shutdown_connections 2024-03-21T11:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.702+0000 7f93feb7a740 1 -- 172.21.15.47:0/3599336581 wait complete. 2024-03-21T11:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.703+0000 7f93feb7a740 1 -- start start 2024-03-21T11:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.703+0000 7f93feb7a740 1 Processor -- start 2024-03-21T11:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.704+0000 7f93feb7a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5b0a0 0x5640e8c651f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.704+0000 7f93feb7a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c5ba90 0x5640e8c65730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.704+0000 7f93feb7a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c60c00 0x5640e8c6c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.704+0000 7f93feb7a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e8c63bb0 con 0x5640e8c5ba90 2024-03-21T11:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.704+0000 7f93feb7a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e8b7ceb0 con 0x5640e8c60c00 2024-03-21T11:45:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.704+0000 7f93feb7a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e8c63d30 con 0x5640e8c5b0a0 2024-03-21T11:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.704+0000 7f93feb7a740 10 client.0 _ll_register_callbacks cb 0x5640e8b71150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.704+0000 7f93f9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5b0a0 0x5640e8c651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.704+0000 7f93f9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5b0a0 0x5640e8c651f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52858/0 (socket says 172.21.15.47:52858) 2024-03-21T11:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.704+0000 7f93f9400640 1 -- 172.21.15.47:0/1424245024 learned_addr learned my addr 172.21.15.47:0/1424245024 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.704+0000 7f93f9e00640 1 --2- 172.21.15.47:0/1424245024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c60c00 0x5640e8c6c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.705+0000 7f93f9400640 1 -- 172.21.15.47:0/1424245024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c60c00 msgr2=0x5640e8c6c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.705+0000 7f93f9400640 1 --2- 172.21.15.47:0/1424245024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c60c00 0x5640e8c6c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.705+0000 7f93f9400640 1 -- 172.21.15.47:0/1424245024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c5ba90 msgr2=0x5640e8c65730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.705+0000 7f93f9400640 1 --2- 172.21.15.47:0/1424245024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c5ba90 0x5640e8c65730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.705+0000 7f93f9400640 1 -- 172.21.15.47:0/1424245024 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640e89fdf70 con 0x5640e8c5b0a0 2024-03-21T11:45:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.705+0000 7f93f9e00640 1 --2- 172.21.15.47:0/1424245024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c60c00 0x5640e8c6c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.706+0000 7f93f9400640 1 --2- 172.21.15.47:0/1424245024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5b0a0 0x5640e8c651f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f93e800ec20 tx=0x7f93e80055b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a51c6b65c52eb264 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.706+0000 7f93daa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.706+0000 7f93daa00640 1 -- 172.21.15.47:0/1424245024 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e802a060 con 0x5640e8c5b0a0 2024-03-21T11:45:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.706+0000 7f93daa00640 1 -- 172.21.15.47:0/1424245024 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f93e80048d0 con 0x5640e8c5b0a0 2024-03-21T11:45:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.706+0000 7f93feb7a740 1 -- 172.21.15.47:0/1424245024 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93e8004c80 con 0x5640e8c5b0a0 2024-03-21T11:45:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.706+0000 7f93daa00640 1 -- 172.21.15.47:0/1424245024 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93e8004ad0 con 0x5640e8c5b0a0 2024-03-21T11:45:39.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.706+0000 7f93feb7a740 10 client.32635 fetch_fsmap learned FSMap version 41 2024-03-21T11:45:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.706+0000 7f93feb7a740 1 -- 172.21.15.47:0/1424245024 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5640e8895170 con 0x5640e8c5b0a0 2024-03-21T11:45:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.707+0000 7f93daa00640 1 -- 172.21.15.47:0/1424245024 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93e8004c80 con 0x5640e8c5b0a0 2024-03-21T11:45:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.707+0000 7f93daa00640 1 -- 172.21.15.47:0/1424245024 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f93e8025770 con 0x5640e8c5b0a0 2024-03-21T11:45:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.707+0000 7f93feb7a740 10 client.32635 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:45:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.707+0000 7f93feb7a740 10 client.32635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.707+0000 7f93feb7a740 20 client.32635 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.707+0000 7f93feb7a740 1 --2- 172.21.15.47:0/1424245024 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5640e8c70d10 0x5640e8c73120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.707+0000 7f93feb7a740 4 client.32635 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.707+0000 7f93feb7a740 1 -- 172.21.15.47:0/1424245024 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640e8bce3f0 con 0x5640e8c70d10 2024-03-21T11:45:39.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.707+0000 7f93f8a00640 1 --2- 172.21.15.47:0/1424245024 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5640e8c70d10 0x5640e8c73120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.708+0000 7f93f8a00640 1 --2- 172.21.15.47:0/1424245024 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5640e8c70d10 0x5640e8c73120 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca5e258cac71202e server_cookie=dbf33bc20be6fe2e in_seq=0 out_seq=0 2024-03-21T11:45:39.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.708+0000 7f93daa00640 10 client.32635 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:39.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93daa00640 1 -- 172.21.15.47:0/1424245024 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5640e8bce3f0 con 0x5640e8c70d10 2024-03-21T11:45:39.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93daa00640 10 client.32635 handle_command_reply: tid=0 2024-03-21T11:45:39.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93feb7a740 1 client.32635 shutdown 2024-03-21T11:45:39.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93feb7a740 1 -- 172.21.15.47:0/1424245024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5b0a0 msgr2=0x5640e8c651f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:39.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93feb7a740 1 --2- 172.21.15.47:0/1424245024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5b0a0 0x5640e8c651f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f93e800ec20 tx=0x7f93e80055b0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93feb7a740 1 -- 172.21.15.47:0/1424245024 shutdown_connections 2024-03-21T11:45:39.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93feb7a740 1 --2- 172.21.15.47:0/1424245024 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5640e8c70d10 0x5640e8c73120 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93feb7a740 1 --2- 172.21.15.47:0/1424245024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c60c00 0x5640e8c6c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93feb7a740 1 --2- 172.21.15.47:0/1424245024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c5ba90 0x5640e8c65730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93feb7a740 1 --2- 172.21.15.47:0/1424245024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5b0a0 0x5640e8c651f0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93feb7a740 1 -- 172.21.15.47:0/1424245024 >> 172.21.15.47:0/1424245024 conn(0x5640e8c56610 msgr2=0x5640e8c55830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:39.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93daa00640 0 client.32635 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:39.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93feb7a740 1 -- 172.21.15.47:0/1424245024 shutdown_connections 2024-03-21T11:45:39.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93feb7a740 1 -- 172.21.15.47:0/1424245024 wait complete. 2024-03-21T11:45:39.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.711+0000 7f93feb7a740 20 client.32635 trim_cache size 0 max 16384 2024-03-21T11:45:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.733+0000 7f93feb7a740 1 Processor -- start 2024-03-21T11:45:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.734+0000 7f93feb7a740 1 -- start start 2024-03-21T11:45:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.734+0000 7f93feb7a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5b0a0 0x5640e8c59f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.734+0000 7f93feb7a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5ba90 0x5640e8c6c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.734+0000 7f93feb7a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c60c00 0x5640e8c779e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.734+0000 7f93feb7a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f41b8f30 con 0x5640e8c60c00 2024-03-21T11:45:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.734+0000 7f93feb7a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f41ba8b0 con 0x5640e8c5ba90 2024-03-21T11:45:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.734+0000 7f93feb7a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f41b9590 con 0x5640e8c5b0a0 2024-03-21T11:45:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.734+0000 7f93f8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5ba90 0x5640e8c6c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.734+0000 7f93f9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5b0a0 0x5640e8c59f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.734+0000 7f93f8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5ba90 0x5640e8c6c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38942/0 (socket says 172.21.15.47:38942) 2024-03-21T11:45:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.734+0000 7f93f8a00640 1 -- 172.21.15.47:0/3625252212 learned_addr learned my addr 172.21.15.47:0/3625252212 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.734+0000 7f93f9e00640 1 --2- 172.21.15.47:0/3625252212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c60c00 0x5640e8c779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.735+0000 7f93f8a00640 1 -- 172.21.15.47:0/3625252212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5b0a0 msgr2=0x5640e8c59f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.735+0000 7f93f8a00640 1 --2- 172.21.15.47:0/3625252212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5b0a0 0x5640e8c59f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.735+0000 7f93f8a00640 1 -- 172.21.15.47:0/3625252212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c60c00 msgr2=0x5640e8c779e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.735+0000 7f93f8a00640 1 --2- 172.21.15.47:0/3625252212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c60c00 0x5640e8c779e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.735+0000 7f93f8a00640 1 -- 172.21.15.47:0/3625252212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640e8c55680 con 0x5640e8c5ba90 2024-03-21T11:45:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.735+0000 7f93f8a00640 1 --2- 172.21.15.47:0/3625252212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5ba90 0x5640e8c6c690 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f93e00096c0 tx=0x7f93e0043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d43af2ed8377952 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.735+0000 7f93e5c00640 1 -- 172.21.15.47:0/3625252212 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0055170 con 0x5640e8c5ba90 2024-03-21T11:45:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.735+0000 7f93e5c00640 1 -- 172.21.15.47:0/3625252212 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93e005b070 con 0x5640e8c5ba90 2024-03-21T11:45:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.735+0000 7f93e5c00640 1 -- 172.21.15.47:0/3625252212 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0004b60 con 0x5640e8c5ba90 2024-03-21T11:45:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.736+0000 7f93feb7a740 1 -- 172.21.15.47:0/3625252212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5ba90 msgr2=0x5640e8c6c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.736+0000 7f93feb7a740 1 --2- 172.21.15.47:0/3625252212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5ba90 0x5640e8c6c690 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f93e00096c0 tx=0x7f93e0043000 comp rx=0 tx=0).stop 2024-03-21T11:45:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.736+0000 7f93feb7a740 1 -- 172.21.15.47:0/3625252212 shutdown_connections 2024-03-21T11:45:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.736+0000 7f93feb7a740 1 --2- 172.21.15.47:0/3625252212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c60c00 0x5640e8c779e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.736+0000 7f93feb7a740 1 --2- 172.21.15.47:0/3625252212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5ba90 0x5640e8c6c690 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.736+0000 7f93feb7a740 1 --2- 172.21.15.47:0/3625252212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5b0a0 0x5640e8c59f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.736+0000 7f93feb7a740 1 -- 172.21.15.47:0/3625252212 >> 172.21.15.47:0/3625252212 conn(0x5640e8c56610 msgr2=0x5640e8c59870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.736+0000 7f93feb7a740 1 -- 172.21.15.47:0/3625252212 shutdown_connections 2024-03-21T11:45:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.736+0000 7f93feb7a740 1 -- 172.21.15.47:0/3625252212 wait complete. 2024-03-21T11:45:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.737+0000 7f93feb7a740 1 -- start start 2024-03-21T11:45:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.737+0000 7f93feb7a740 1 Processor -- start 2024-03-21T11:45:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93feb7a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5b0a0 0x5640e8c6be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93feb7a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5ba90 0x5640e8c6c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93feb7a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c60c00 0x5640e8c6a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93feb7a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f41ba250 con 0x5640e8c60c00 2024-03-21T11:45:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93feb7a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f41b9bf0 con 0x5640e8c5b0a0 2024-03-21T11:45:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93feb7a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f41b70f0 con 0x5640e8c5ba90 2024-03-21T11:45:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93feb7a740 10 client.0 _ll_register_callbacks cb 0x7f93e8014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93f9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5b0a0 0x5640e8c6be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93f9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5b0a0 0x5640e8c6be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38950/0 (socket says 172.21.15.47:38950) 2024-03-21T11:45:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93f8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5ba90 0x5640e8c6c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93f9400640 1 -- 172.21.15.47:0/3000635198 learned_addr learned my addr 172.21.15.47:0/3000635198 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93f9e00640 1 --2- 172.21.15.47:0/3000635198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c60c00 0x5640e8c6a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93f9400640 1 -- 172.21.15.47:0/3000635198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5ba90 msgr2=0x5640e8c6c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93f9400640 1 --2- 172.21.15.47:0/3000635198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5ba90 0x5640e8c6c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93f9400640 1 -- 172.21.15.47:0/3000635198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c60c00 msgr2=0x5640e8c6a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93f9400640 1 --2- 172.21.15.47:0/3000635198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c60c00 0x5640e8c6a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.738+0000 7f93f9400640 1 -- 172.21.15.47:0/3000635198 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640e8c6ab10 con 0x5640e8c5b0a0 2024-03-21T11:45:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.739+0000 7f93f9400640 1 --2- 172.21.15.47:0/3000635198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5b0a0 0x5640e8c6be90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f93e8025c60 tx=0x7f93e800ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=475c4328d70d453 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.739+0000 7f93daa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:45:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.739+0000 7f93daa00640 1 -- 172.21.15.47:0/3000635198 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e800c330 con 0x5640e8c5b0a0 2024-03-21T11:45:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.739+0000 7f93daa00640 1 -- 172.21.15.47:0/3000635198 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f93e8025680 con 0x5640e8c5b0a0 2024-03-21T11:45:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.739+0000 7f93feb7a740 1 -- 172.21.15.47:0/3000635198 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93e005b070 con 0x5640e8c5b0a0 2024-03-21T11:45:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.739+0000 7f93daa00640 1 -- 172.21.15.47:0/3000635198 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93e803e050 con 0x5640e8c5b0a0 2024-03-21T11:45:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.739+0000 7f93feb7a740 10 client.32669 fetch_fsmap learned FSMap version 41 2024-03-21T11:45:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.739+0000 7f93feb7a740 1 -- 172.21.15.47:0/3000635198 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5640e8cccf50 con 0x5640e8c5b0a0 2024-03-21T11:45:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.740+0000 7f93daa00640 1 -- 172.21.15.47:0/3000635198 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93e005b070 con 0x5640e8c5b0a0 2024-03-21T11:45:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.740+0000 7f93daa00640 1 -- 172.21.15.47:0/3000635198 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f93e8003680 con 0x5640e8c5b0a0 2024-03-21T11:45:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.740+0000 7f93feb7a740 10 client.32669 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:45:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.740+0000 7f93feb7a740 10 client.32669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.740+0000 7f93feb7a740 20 client.32669 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.740+0000 7f93feb7a740 1 --2- 172.21.15.47:0/3000635198 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5640e8ccd760 0x5640e8ccfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.740+0000 7f93feb7a740 4 client.32669 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:45:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.740+0000 7f93feb7a740 1 -- 172.21.15.47:0/3000635198 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5640e8bc8740 con 0x5640e8ccd760 2024-03-21T11:45:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.740+0000 7f93f8a00640 1 --2- 172.21.15.47:0/3000635198 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5640e8ccd760 0x5640e8ccfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:39.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.741+0000 7f93f8a00640 1 --2- 172.21.15.47:0/3000635198 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5640e8ccd760 0x5640e8ccfc20 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0a7893f2baca1c9 server_cookie=deca6b41fb693c18 in_seq=0 out_seq=0 2024-03-21T11:45:39.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.741+0000 7f93daa00640 10 client.32669 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:39.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.748+0000 7f93daa00640 1 -- 172.21.15.47:0/3000635198 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5640e8bc8740 con 0x5640e8ccd760 2024-03-21T11:45:39.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.748+0000 7f93daa00640 10 client.32669 handle_command_reply: tid=0 2024-03-21T11:45:39.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.748+0000 7f93feb7a740 1 client.32669 shutdown 2024-03-21T11:45:39.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.748+0000 7f93feb7a740 1 -- 172.21.15.47:0/3000635198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5b0a0 msgr2=0x5640e8c6be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:39.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.748+0000 7f93feb7a740 1 --2- 172.21.15.47:0/3000635198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5b0a0 0x5640e8c6be90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f93e8025c60 tx=0x7f93e800ef50 comp rx=0 tx=0).stop 2024-03-21T11:45:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.749+0000 7f93feb7a740 1 -- 172.21.15.47:0/3000635198 shutdown_connections 2024-03-21T11:45:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.749+0000 7f93feb7a740 1 --2- 172.21.15.47:0/3000635198 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5640e8ccd760 0x5640e8ccfc20 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.749+0000 7f93feb7a740 1 --2- 172.21.15.47:0/3000635198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5640e8c60c00 0x5640e8c6a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.749+0000 7f93feb7a740 1 --2- 172.21.15.47:0/3000635198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640e8c5ba90 0x5640e8c6c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.749+0000 7f93feb7a740 1 --2- 172.21.15.47:0/3000635198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e8c5b0a0 0x5640e8c6be90 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.749+0000 7f93feb7a740 1 -- 172.21.15.47:0/3000635198 >> 172.21.15.47:0/3000635198 conn(0x5640e8c56610 msgr2=0x5640e8c5a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.749+0000 7f93daa00640 0 client.32669 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.749+0000 7f93feb7a740 1 -- 172.21.15.47:0/3000635198 shutdown_connections 2024-03-21T11:45:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.749+0000 7f93feb7a740 1 -- 172.21.15.47:0/3000635198 wait complete. 2024-03-21T11:45:39.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.749+0000 7f93feb7a740 20 client.32669 trim_cache size 0 max 16384 2024-03-21T11:45:39.752 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:45:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:45:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "69878e0e-55c8-4428-a6a5-803777528aad", 2024-03-21T11:45:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:45:39.752 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:45:39.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.751+0000 7f93fbe00640 1 -- 172.21.15.47:0/3401692864 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93dc0a2fa0 msgr2=0x7f93dc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:39.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.751+0000 7f93fbe00640 1 --2- 172.21.15.47:0/3401692864 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93dc0a2fa0 0x7f93dc0a5460 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f93e002c240 tx=0x7f93e0005fd0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.751+0000 7f93fbe00640 1 -- 172.21.15.47:0/3401692864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93f410d9c0 msgr2=0x7f93f41add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:39.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.751+0000 7f93fbe00640 1 --2- 172.21.15.47:0/3401692864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93f410d9c0 0x7f93f41add90 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00eb40 tx=0x7f93ec00c2d0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.751+0000 7f93fbe00640 1 -- 172.21.15.47:0/3401692864 shutdown_connections 2024-03-21T11:45:39.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.751+0000 7f93fbe00640 1 --2- 172.21.15.47:0/3401692864 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93dc0a2fa0 0x7f93dc0a5460 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.751+0000 7f93fbe00640 1 --2- 172.21.15.47:0/3401692864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93f410d9c0 0x7f93f41add90 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.751+0000 7f93fbe00640 1 --2- 172.21.15.47:0/3401692864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93f410b010 0x7f93f41a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.751+0000 7f93fbe00640 1 --2- 172.21.15.47:0/3401692864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93f41086a0 0x7f93f41a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:39.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.751+0000 7f93fbe00640 1 -- 172.21.15.47:0/3401692864 >> 172.21.15.47:0/3401692864 conn(0x7f93f406e500 msgr2=0x7f93f4109c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:39.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.751+0000 7f93fbe00640 1 -- 172.21.15.47:0/3401692864 shutdown_connections 2024-03-21T11:45:39.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:39.752+0000 7f93fbe00640 1 -- 172.21.15.47:0/3401692864 wait complete. 2024-03-21T11:45:39.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:45:39.775 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:45:39.981 INFO:tasks.workunit.client.0.smithi047.stdout: 10 216234 22.21 MB/sec execute 383 sec latency 116.943 ms 2024-03-21T11:45:40.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.035+0000 7fc99ae00640 1 -- 172.21.15.47:0/1684538394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9941016d0 msgr2=0x7fc994101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:40.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.035+0000 7fc99ae00640 1 --2- 172.21.15.47:0/1684538394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9941016d0 0x7fc994101af0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fc984009900 tx=0x7fc98402bc40 comp rx=0 tx=0).stop 2024-03-21T11:45:40.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.035+0000 7fc99ae00640 1 -- 172.21.15.47:0/1684538394 shutdown_connections 2024-03-21T11:45:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.035+0000 7fc99ae00640 1 --2- 172.21.15.47:0/1684538394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc994103280 0x7fc994109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.035+0000 7fc99ae00640 1 --2- 172.21.15.47:0/1684538394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9941028c0 0x7fc994102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.035+0000 7fc99ae00640 1 --2- 172.21.15.47:0/1684538394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9941016d0 0x7fc994101af0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.035+0000 7fc99ae00640 1 -- 172.21.15.47:0/1684538394 >> 172.21.15.47:0/1684538394 conn(0x7fc9940fcc40 msgr2=0x7fc9940ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.035+0000 7fc99ae00640 1 -- 172.21.15.47:0/1684538394 shutdown_connections 2024-03-21T11:45:40.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.036+0000 7fc99ae00640 1 -- 172.21.15.47:0/1684538394 wait complete. 2024-03-21T11:45:40.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.038+0000 7fc99ae00640 1 Processor -- start 2024-03-21T11:45:40.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.038+0000 7fc99ae00640 1 -- start start 2024-03-21T11:45:40.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.039+0000 7fc99ae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9941016d0 0x7fc994110b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.039+0000 7fc99ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9941028c0 0x7fc994111080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.039+0000 7fc99ae00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc994103280 0x7fc994118130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.040+0000 7fc99ae00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994075150 con 0x7fc994103280 2024-03-21T11:45:40.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.040+0000 7fc99ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994074fd0 con 0x7fc9941028c0 2024-03-21T11:45:40.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.040+0000 7fc99ae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940752d0 con 0x7fc9941016d0 2024-03-21T11:45:40.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.040+0000 7fc993400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9941028c0 0x7fc994111080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.040+0000 7fc993400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9941028c0 0x7fc994111080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38962/0 (socket says 172.21.15.47:38962) 2024-03-21T11:45:40.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.040+0000 7fc993400640 1 -- 172.21.15.47:0/1415270266 learned_addr learned my addr 172.21.15.47:0/1415270266 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:40.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.040+0000 7fc998e00640 1 --2- 172.21.15.47:0/1415270266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc994103280 0x7fc994118130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.041+0000 7fc993e00640 1 --2- 172.21.15.47:0/1415270266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9941016d0 0x7fc994110b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.041+0000 7fc993e00640 1 -- 172.21.15.47:0/1415270266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9941028c0 msgr2=0x7fc994111080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:40.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.041+0000 7fc993e00640 1 --2- 172.21.15.47:0/1415270266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9941028c0 0x7fc994111080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.041+0000 7fc993e00640 1 -- 172.21.15.47:0/1415270266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc994103280 msgr2=0x7fc994118130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:40.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.041+0000 7fc993e00640 1 --2- 172.21.15.47:0/1415270266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc994103280 0x7fc994118130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.041+0000 7fc993e00640 1 -- 172.21.15.47:0/1415270266 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99419d4d0 con 0x7fc9941016d0 2024-03-21T11:45:40.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.042+0000 7fc993e00640 1 --2- 172.21.15.47:0/1415270266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9941016d0 0x7fc994110b40 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fc984009230 tx=0x7fc984005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:40.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.043+0000 7fc990c00640 1 -- 172.21.15.47:0/1415270266 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984041070 con 0x7fc9941016d0 2024-03-21T11:45:40.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.044+0000 7fc990c00640 1 -- 172.21.15.47:0/1415270266 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9840054e0 con 0x7fc9941016d0 2024-03-21T11:45:40.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.044+0000 7fc990c00640 1 -- 172.21.15.47:0/1415270266 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984034440 con 0x7fc9941016d0 2024-03-21T11:45:40.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.044+0000 7fc99ae00640 1 -- 172.21.15.47:0/1415270266 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99419d760 con 0x7fc9941016d0 2024-03-21T11:45:40.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.044+0000 7fc99ae00640 1 -- 172.21.15.47:0/1415270266 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99419db70 con 0x7fc9941016d0 2024-03-21T11:45:40.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.052+0000 7fc99dbaa740 1 Processor -- start 2024-03-21T11:45:40.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.053+0000 7fc99dbaa740 1 -- start start 2024-03-21T11:45:40.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.054+0000 7fc990c00640 1 -- 172.21.15.47:0/1415270266 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc984044070 con 0x7fc9941016d0 2024-03-21T11:45:40.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.055+0000 7fc99dbaa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a41302a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.055+0000 7fc99dbaa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a4130870 0x5590a4135370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.055+0000 7fc99dbaa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a41359e0 0x5590a4137e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.055+0000 7fc99dbaa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a4095250 con 0x5590a41359e0 2024-03-21T11:45:40.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.055+0000 7fc99dbaa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a403da30 con 0x5590a4130870 2024-03-21T11:45:40.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.055+0000 7fc99dbaa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a40912f0 con 0x5590a412fe80 2024-03-21T11:45:40.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.055+0000 7fc998e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a41302a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.055+0000 7fc998e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a41302a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52908/0 (socket says 172.21.15.47:52908) 2024-03-21T11:45:40.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.055+0000 7fc998e00640 1 -- 172.21.15.47:0/3562314002 learned_addr learned my addr 172.21.15.47:0/3562314002 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:40.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.056+0000 7fc993400640 1 --2- 172.21.15.47:0/3562314002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a41359e0 0x5590a4137e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.056+0000 7fc993e00640 1 --2- 172.21.15.47:0/3562314002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a4130870 0x5590a4135370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.056+0000 7fc998e00640 1 -- 172.21.15.47:0/3562314002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a4130870 msgr2=0x5590a4135370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:40.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.056+0000 7fc998e00640 1 --2- 172.21.15.47:0/3562314002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a4130870 0x5590a4135370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.056+0000 7fc998e00640 1 -- 172.21.15.47:0/3562314002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a41359e0 msgr2=0x5590a4137e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:40.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.056+0000 7fc998e00640 1 --2- 172.21.15.47:0/3562314002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a41359e0 0x5590a4137e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.056+0000 7fc998e00640 1 -- 172.21.15.47:0/3562314002 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590a4028690 con 0x5590a412fe80 2024-03-21T11:45:40.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.057+0000 7fc998e00640 1 --2- 172.21.15.47:0/3562314002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a41302a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00a430 tx=0x7fc98c009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f1db0d8b01591e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:40.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.057+0000 7fc990c00640 1 --2- 172.21.15.47:0/1415270266 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9780a30f0 0x7fc9780a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.057+0000 7fc990c00640 1 -- 172.21.15.47:0/1415270266 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc9840f0db0 con 0x7fc9941016d0 2024-03-21T11:45:40.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.058+0000 7fc989600640 1 -- 172.21.15.47:0/3562314002 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c013790 con 0x5590a412fe80 2024-03-21T11:45:40.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.058+0000 7fc989600640 1 -- 172.21.15.47:0/3562314002 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc98c019070 con 0x5590a412fe80 2024-03-21T11:45:40.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.058+0000 7fc989600640 1 -- 172.21.15.47:0/3562314002 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0122e0 con 0x5590a412fe80 2024-03-21T11:45:40.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.059+0000 7fc993e00640 1 --2- 172.21.15.47:0/1415270266 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9780a30f0 0x7fc9780a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.059+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3562314002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 msgr2=0x5590a41302a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:40.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.059+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/3562314002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a41302a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00a430 tx=0x7fc98c009650 comp rx=0 tx=0).stop 2024-03-21T11:45:40.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.059+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3562314002 shutdown_connections 2024-03-21T11:45:40.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.059+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/3562314002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a41359e0 0x5590a4137e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.059+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/3562314002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a4130870 0x5590a4135370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.059+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/3562314002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a41302a0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.059+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3562314002 >> 172.21.15.47:0/3562314002 conn(0x5590a40f8200 msgr2=0x5590a412b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:40.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.060+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3562314002 shutdown_connections 2024-03-21T11:45:40.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.060+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3562314002 wait complete. 2024-03-21T11:45:40.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.060+0000 7fc993e00640 1 --2- 172.21.15.47:0/1415270266 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9780a30f0 0x7fc9780a55b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc9840bd080 tx=0x7fc98402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:40.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.060+0000 7fc99dbaa740 1 -- start start 2024-03-21T11:45:40.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.061+0000 7fc99dbaa740 1 Processor -- start 2024-03-21T11:45:40.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.062+0000 7fc99dbaa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a4139ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.062+0000 7fc99dbaa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a4130870 0x5590a413a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.062+0000 7fc99dbaa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 0x5590a4141420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.062+0000 7fc99dbaa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a41389e0 con 0x5590a4130870 2024-03-21T11:45:40.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.063+0000 7fc998e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a4139ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.063+0000 7fc993400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a4130870 0x5590a413a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.063+0000 7fc998e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a4139ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52910/0 (socket says 172.21.15.47:52910) 2024-03-21T11:45:40.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.063+0000 7fc998e00640 1 -- 172.21.15.47:0/3822067891 learned_addr learned my addr 172.21.15.47:0/3822067891 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:40.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.063+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3822067891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a4050e80 con 0x5590a41359e0 2024-03-21T11:45:40.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.063+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3822067891 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a4138b60 con 0x5590a412fe80 2024-03-21T11:45:40.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.063+0000 7fc99dbaa740 10 client.0 _ll_register_callbacks cb 0x5590a4063130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:40.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.063+0000 7fc998e00640 1 --2- 172.21.15.47:0/3822067891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 0x5590a4141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:39 smithi047 ceph-mon[31351]: daemon mds.e restarted 2024-03-21T11:45:40.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:39 smithi047 ceph-mon[31351]: osdmap e96: 12 total, 12 up, 12 in 2024-03-21T11:45:40.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:39 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.100:6834/4278877119,v1:172.21.15.100:6835/4278877119] up:boot 2024-03-21T11:45:40.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:39 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:45:40.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:39 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T11:45:40.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.063+0000 7fc993400640 1 -- 172.21.15.47:0/3822067891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 msgr2=0x5590a4139ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:40.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.064+0000 7fc993400640 1 --2- 172.21.15.47:0/3822067891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a4139ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.064+0000 7fc993400640 1 -- 172.21.15.47:0/3822067891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 msgr2=0x5590a4141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:40.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.064+0000 7fc993400640 1 --2- 172.21.15.47:0/3822067891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 0x5590a4141420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.064+0000 7fc993400640 1 -- 172.21.15.47:0/3822067891 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590a3e737d0 con 0x5590a4130870 2024-03-21T11:45:40.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.064+0000 7fc998e00640 1 --2- 172.21.15.47:0/3822067891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 0x5590a4141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:45:40.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.064+0000 7fc993400640 1 --2- 172.21.15.47:0/3822067891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a4130870 0x5590a413a3e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc980005fd0 tx=0x7fc980012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7560dc327ed5b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:40.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.064+0000 7fc97e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:45:40.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.064+0000 7fc97e000640 1 -- 172.21.15.47:0/3822067891 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980004230 con 0x5590a4130870 2024-03-21T11:45:40.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.064+0000 7fc97e000640 1 -- 172.21.15.47:0/3822067891 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc9800043d0 con 0x5590a4130870 2024-03-21T11:45:40.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.064+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3822067891 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc98c019070 con 0x5590a4130870 2024-03-21T11:45:40.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.065+0000 7fc97e000640 1 -- 172.21.15.47:0/3822067891 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc980010730 con 0x5590a4130870 2024-03-21T11:45:40.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.065+0000 7fc99dbaa740 10 client.21891 fetch_fsmap learned FSMap version 41 2024-03-21T11:45:40.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.065+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3822067891 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5590a3da93f0 con 0x5590a4130870 2024-03-21T11:45:40.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.065+0000 7fc97e000640 1 -- 172.21.15.47:0/3822067891 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc98c019070 con 0x5590a4130870 2024-03-21T11:45:40.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.065+0000 7fc97e000640 1 -- 172.21.15.47:0/3822067891 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc980019070 con 0x5590a4130870 2024-03-21T11:45:40.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.065+0000 7fc99dbaa740 10 client.21891 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:45:40.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.065+0000 7fc99dbaa740 10 client.21891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:40.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.065+0000 7fc99dbaa740 20 client.21891 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:40.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.065+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/3822067891 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5590a4145dc0 0x5590a4148200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.065+0000 7fc99dbaa740 4 client.21891 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:40.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.065+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3822067891 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590a40a2790 con 0x5590a4145dc0 2024-03-21T11:45:40.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.065+0000 7fc998e00640 1 --2- 172.21.15.47:0/3822067891 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5590a4145dc0 0x5590a4148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.066+0000 7fc998e00640 1 --2- 172.21.15.47:0/3822067891 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5590a4145dc0 0x5590a4148200 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=712651a550dd06a6 server_cookie=51c125e1f838f3c9 in_seq=0 out_seq=0 2024-03-21T11:45:40.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.066+0000 7fc97e000640 10 client.21891 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:40.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.070+0000 7fc97e000640 1 -- 172.21.15.47:0/3822067891 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5590a40a2790 con 0x5590a4145dc0 2024-03-21T11:45:40.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.070+0000 7fc97e000640 10 client.21891 handle_command_reply: tid=0 2024-03-21T11:45:40.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.070+0000 7fc99dbaa740 1 client.21891 shutdown 2024-03-21T11:45:40.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.070+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3822067891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a4130870 msgr2=0x5590a413a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:40.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.070+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/3822067891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a4130870 0x5590a413a3e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc980005fd0 tx=0x7fc980012820 comp rx=0 tx=0).stop 2024-03-21T11:45:40.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.070+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3822067891 shutdown_connections 2024-03-21T11:45:40.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.070+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/3822067891 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5590a4145dc0 0x5590a4148200 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.070+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/3822067891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 0x5590a4141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.070+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/3822067891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a4130870 0x5590a413a3e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.070+0000 7fc97e000640 0 client.21891 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:40.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.070+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/3822067891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a4139ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.070+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3822067891 >> 172.21.15.47:0/3822067891 conn(0x5590a40f8200 msgr2=0x5590a412aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:40.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.071+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3822067891 shutdown_connections 2024-03-21T11:45:40.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.071+0000 7fc99dbaa740 1 -- 172.21.15.47:0/3822067891 wait complete. 2024-03-21T11:45:40.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.071+0000 7fc99dbaa740 20 client.21891 trim_cache size 0 max 16384 2024-03-21T11:45:40.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.095+0000 7fc99dbaa740 1 Processor -- start 2024-03-21T11:45:40.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.095+0000 7fc99dbaa740 1 -- start start 2024-03-21T11:45:40.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.096+0000 7fc99dbaa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a412fe80 0x5590a412ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.096+0000 7fc99dbaa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 0x5590a41413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.096+0000 7fc99dbaa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a413a510 0x5590a412ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.096+0000 7fc99dbaa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941a8030 con 0x5590a412fe80 2024-03-21T11:45:40.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.096+0000 7fc99dbaa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941a99b0 con 0x5590a41359e0 2024-03-21T11:45:40.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.096+0000 7fc99dbaa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941a8690 con 0x5590a413a510 2024-03-21T11:45:40.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.097+0000 7fc998e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a413a510 0x5590a412ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.097+0000 7fc998e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a413a510 0x5590a412ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52922/0 (socket says 172.21.15.47:52922) 2024-03-21T11:45:40.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.097+0000 7fc998e00640 1 -- 172.21.15.47:0/1924717237 learned_addr learned my addr 172.21.15.47:0/1924717237 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:40.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.097+0000 7fc993400640 1 --2- 172.21.15.47:0/1924717237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 0x5590a41413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.097+0000 7fc998e00640 1 --2- 172.21.15.47:0/1924717237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a412fe80 0x5590a412ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.097+0000 7fc998e00640 1 -- 172.21.15.47:0/1924717237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 msgr2=0x5590a41413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:40.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.097+0000 7fc998e00640 1 --2- 172.21.15.47:0/1924717237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 0x5590a41413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.097+0000 7fc998e00640 1 -- 172.21.15.47:0/1924717237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a412fe80 msgr2=0x5590a412ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:40.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.097+0000 7fc998e00640 1 --2- 172.21.15.47:0/1924717237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a412fe80 0x5590a412ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.097+0000 7fc998e00640 1 -- 172.21.15.47:0/1924717237 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590a3d690f0 con 0x5590a413a510 2024-03-21T11:45:40.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.098+0000 7fc998e00640 1 --2- 172.21.15.47:0/1924717237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a413a510 0x5590a412ef70 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x5590a4148a70 tx=0x7fc98c00d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a2f4c4b65b8657d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:40.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.098+0000 7fc988c00640 1 -- 172.21.15.47:0/1924717237 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c032070 con 0x5590a413a510 2024-03-21T11:45:40.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.098+0000 7fc988c00640 1 -- 172.21.15.47:0/1924717237 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc98c019070 con 0x5590a413a510 2024-03-21T11:45:40.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.098+0000 7fc988c00640 1 -- 172.21.15.47:0/1924717237 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c013890 con 0x5590a413a510 2024-03-21T11:45:40.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.099+0000 7fc99dbaa740 1 -- 172.21.15.47:0/1924717237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a413a510 msgr2=0x5590a412ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:40.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.099+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/1924717237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a413a510 0x5590a412ef70 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x5590a4148a70 tx=0x7fc98c00d040 comp rx=0 tx=0).stop 2024-03-21T11:45:40.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.099+0000 7fc99dbaa740 1 -- 172.21.15.47:0/1924717237 shutdown_connections 2024-03-21T11:45:40.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.099+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/1924717237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a413a510 0x5590a412ef70 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.099+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/1924717237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 0x5590a41413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.099+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/1924717237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a412fe80 0x5590a412ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.099+0000 7fc99dbaa740 1 -- 172.21.15.47:0/1924717237 >> 172.21.15.47:0/1924717237 conn(0x5590a40f8200 msgr2=0x5590a4131ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:40.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.100+0000 7fc99dbaa740 1 -- 172.21.15.47:0/1924717237 shutdown_connections 2024-03-21T11:45:40.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.101+0000 7fc99dbaa740 1 -- 172.21.15.47:0/1924717237 wait complete. 2024-03-21T11:45:40.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.101+0000 7fc99dbaa740 1 -- start start 2024-03-21T11:45:40.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.102+0000 7fc99dbaa740 1 Processor -- start 2024-03-21T11:45:40.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.102+0000 7fc99dbaa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a4141190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.103+0000 7fc99dbaa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 0x5590a413f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.103+0000 7fc99dbaa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a413a510 0x5590a413fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.103+0000 7fc998e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a4141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.103+0000 7fc99dbaa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941a9350 con 0x5590a413a510 2024-03-21T11:45:40.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.103+0000 7fc99dbaa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941a8cf0 con 0x5590a41359e0 2024-03-21T11:45:40.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.103+0000 7fc99dbaa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941a61f0 con 0x5590a412fe80 2024-03-21T11:45:40.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.103+0000 7fc99dbaa740 10 client.0 _ll_register_callbacks cb 0x7fc98000b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.103+0000 7fc993400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 0x5590a413f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.103+0000 7fc998e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a4141190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52924/0 (socket says 172.21.15.47:52924) 2024-03-21T11:45:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.103+0000 7fc998e00640 1 -- 172.21.15.47:0/471786098 learned_addr learned my addr 172.21.15.47:0/471786098 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.104+0000 7fc998e00640 1 --2- 172.21.15.47:0/471786098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a413a510 0x5590a413fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.104+0000 7fc998e00640 1 -- 172.21.15.47:0/471786098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 msgr2=0x5590a413f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.104+0000 7fc998e00640 1 --2- 172.21.15.47:0/471786098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 0x5590a413f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.104+0000 7fc998e00640 1 -- 172.21.15.47:0/471786098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a413a510 msgr2=0x5590a413fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.104+0000 7fc998e00640 1 --2- 172.21.15.47:0/471786098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a413a510 0x5590a413fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.104+0000 7fc998e00640 1 -- 172.21.15.47:0/471786098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590a41403a0 con 0x5590a412fe80 2024-03-21T11:45:40.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.105+0000 7fc998e00640 1 --2- 172.21.15.47:0/471786098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a4141190 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fc98c004a80 tx=0x7fc98c009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=238eceaddb0b2e6c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:40.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.105+0000 7fc97e000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:40.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.105+0000 7fc97e000640 1 -- 172.21.15.47:0/471786098 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c032070 con 0x5590a412fe80 2024-03-21T11:45:40.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.105+0000 7fc97e000640 1 -- 172.21.15.47:0/471786098 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc98c02c830 con 0x5590a412fe80 2024-03-21T11:45:40.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.105+0000 7fc99dbaa740 1 -- 172.21.15.47:0/471786098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc98c019070 con 0x5590a412fe80 2024-03-21T11:45:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.106+0000 7fc97e000640 1 -- 172.21.15.47:0/471786098 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc98c02ca30 con 0x5590a412fe80 2024-03-21T11:45:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.106+0000 7fc99dbaa740 10 client.32659 fetch_fsmap learned FSMap version 41 2024-03-21T11:45:40.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.106+0000 7fc99dbaa740 1 -- 172.21.15.47:0/471786098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5590a41a2550 con 0x5590a412fe80 2024-03-21T11:45:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.107+0000 7fc97e000640 1 -- 172.21.15.47:0/471786098 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc98c019070 con 0x5590a412fe80 2024-03-21T11:45:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.107+0000 7fc97e000640 1 -- 172.21.15.47:0/471786098 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc98c02c070 con 0x5590a412fe80 2024-03-21T11:45:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.107+0000 7fc99dbaa740 10 client.32659 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:45:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.107+0000 7fc99dbaa740 10 client.32659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.107+0000 7fc99dbaa740 20 client.32659 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.107+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/471786098 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5590a41a2cc0 0x5590a41a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:40.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.107+0000 7fc99dbaa740 4 client.32659 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:45:40.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.107+0000 7fc99dbaa740 1 -- 172.21.15.47:0/471786098 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5590a4139c90 con 0x5590a41a2cc0 2024-03-21T11:45:40.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.107+0000 7fc993400640 1 --2- 172.21.15.47:0/471786098 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5590a41a2cc0 0x5590a41a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:40.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.108+0000 7fc97e000640 10 client.32659 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:40.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.109+0000 7fc993400640 1 --2- 172.21.15.47:0/471786098 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5590a41a2cc0 0x5590a41a50e0 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=701e11474b8a506b server_cookie=e5f07b3286923218 in_seq=0 out_seq=0 2024-03-21T11:45:40.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.109+0000 7fc97e000640 1 -- 172.21.15.47:0/471786098 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x5590a4139c90 con 0x5590a41a2cc0 2024-03-21T11:45:40.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.109+0000 7fc97e000640 10 client.32659 handle_command_reply: tid=0 2024-03-21T11:45:40.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.109+0000 7fc99dbaa740 1 client.32659 shutdown 2024-03-21T11:45:40.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.110+0000 7fc99dbaa740 1 -- 172.21.15.47:0/471786098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 msgr2=0x5590a4141190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:40.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.110+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/471786098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a4141190 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fc98c004a80 tx=0x7fc98c009f30 comp rx=0 tx=0).stop 2024-03-21T11:45:40.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.110+0000 7fc99dbaa740 1 -- 172.21.15.47:0/471786098 shutdown_connections 2024-03-21T11:45:40.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.110+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/471786098 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5590a41a2cc0 0x5590a41a50e0 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.110+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/471786098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590a413a510 0x5590a413fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.110+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/471786098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590a41359e0 0x5590a413f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.110+0000 7fc99dbaa740 1 --2- 172.21.15.47:0/471786098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590a412fe80 0x5590a4141190 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.110+0000 7fc97e000640 0 client.32659 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:40.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.110+0000 7fc99dbaa740 1 -- 172.21.15.47:0/471786098 >> 172.21.15.47:0/471786098 conn(0x5590a40f8200 msgr2=0x5590a412a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:40.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.110+0000 7fc99dbaa740 1 -- 172.21.15.47:0/471786098 shutdown_connections 2024-03-21T11:45:40.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.110+0000 7fc99dbaa740 1 -- 172.21.15.47:0/471786098 wait complete. 2024-03-21T11:45:40.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.111+0000 7fc99dbaa740 20 client.32659 trim_cache size 0 max 16384 2024-03-21T11:45:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:45:40.112 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (46 inodes in the stack)", 2024-03-21T11:45:40.113 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:45:40.113 INFO:teuthology.orchestra.run.smithi047.stdout: "69878e0e-55c8-4428-a6a5-803777528aad": { 2024-03-21T11:45:40.113 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:45:40.113 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "69878e0e-55c8-4428-a6a5-803777528aad", 2024-03-21T11:45:40.113 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:45:40.113 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:45:40.113 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:45:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:45:40.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.111+0000 7fc99ae00640 1 -- 172.21.15.47:0/1415270266 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9780a30f0 msgr2=0x7fc9780a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:40.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.111+0000 7fc99ae00640 1 --2- 172.21.15.47:0/1415270266 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9780a30f0 0x7fc9780a55b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc9840bd080 tx=0x7fc98402cf90 comp rx=0 tx=0).stop 2024-03-21T11:45:40.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.111+0000 7fc99ae00640 1 -- 172.21.15.47:0/1415270266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9941016d0 msgr2=0x7fc994110b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:40.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.111+0000 7fc99ae00640 1 --2- 172.21.15.47:0/1415270266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9941016d0 0x7fc994110b40 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fc984009230 tx=0x7fc984005dd0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.112+0000 7fc99ae00640 1 -- 172.21.15.47:0/1415270266 shutdown_connections 2024-03-21T11:45:40.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.112+0000 7fc99ae00640 1 --2- 172.21.15.47:0/1415270266 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9780a30f0 0x7fc9780a55b0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.112+0000 7fc99ae00640 1 --2- 172.21.15.47:0/1415270266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc994103280 0x7fc994118130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.112+0000 7fc99ae00640 1 --2- 172.21.15.47:0/1415270266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9941028c0 0x7fc994111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.112+0000 7fc99ae00640 1 --2- 172.21.15.47:0/1415270266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9941016d0 0x7fc994110b40 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:40.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.112+0000 7fc99ae00640 1 -- 172.21.15.47:0/1415270266 >> 172.21.15.47:0/1415270266 conn(0x7fc9940fcc40 msgr2=0x7fc994105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:40.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.112+0000 7fc99ae00640 1 -- 172.21.15.47:0/1415270266 shutdown_connections 2024-03-21T11:45:40.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:45:40.112+0000 7fc99ae00640 1 -- 172.21.15.47:0/1415270266 wait complete. 2024-03-21T11:45:40.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:39 smithi100 ceph-mon[36245]: daemon mds.e restarted 2024-03-21T11:45:40.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:39 smithi100 ceph-mon[36245]: osdmap e96: 12 total, 12 up, 12 in 2024-03-21T11:45:40.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:39 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.100:6834/4278877119,v1:172.21.15.100:6835/4278877119] up:boot 2024-03-21T11:45:40.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:39 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:45:40.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:39 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T11:45:40.144 INFO:tasks.cephfs.filesystem:scrub status for tag:69878e0e-55c8-4428-a6a5-803777528aad - {'path': '/', 'tag': '69878e0e-55c8-4428-a6a5-803777528aad', 'options': 'recursive,force'} 2024-03-21T11:45:40.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:39 smithi102 ceph-mon[36748]: daemon mds.e restarted 2024-03-21T11:45:40.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:39 smithi102 ceph-mon[36748]: osdmap e96: 12 total, 12 up, 12 in 2024-03-21T11:45:40.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:39 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.100:6834/4278877119,v1:172.21.15.100:6835/4278877119] up:boot 2024-03-21T11:45:40.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:39 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:45:40.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:39 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T11:45:40.981 INFO:tasks.workunit.client.0.smithi047.stdout: 10 216679 22.23 MB/sec execute 384 sec latency 101.342 ms 2024-03-21T11:45:41.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: pgmap v1071: 129 pgs: 129 active+clean; 642 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.8 MiB/s wr, 451 op/s 2024-03-21T11:45:41.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:45:41.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:41.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:41.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:41.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:41.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:41.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:41.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:41.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:45:41.396 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.396 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.396 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:41 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: pgmap v1071: 129 pgs: 129 active+clean; 642 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.8 MiB/s wr, 451 op/s 2024-03-21T11:45:41.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:45:41.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:41.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:41.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:41.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:41.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:41.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:41.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:41.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:45:41.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:41 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: pgmap v1071: 129 pgs: 129 active+clean; 642 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.8 MiB/s wr, 451 op/s 2024-03-21T11:45:41.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:45:41.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:41.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:41.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:41.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:41.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:41.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:41.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:41.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:45:41.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:41 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:45:41.981 INFO:tasks.workunit.client.0.smithi047.stdout: 10 217198 22.24 MB/sec execute 385 sec latency 86.406 ms 2024-03-21T11:45:42.981 INFO:tasks.workunit.client.0.smithi047.stdout: 10 217680 22.25 MB/sec execute 386 sec latency 61.052 ms 2024-03-21T11:45:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:43 smithi100 ceph-mon[36245]: pgmap v1072: 129 pgs: 129 active+clean; 648 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.9 MiB/s wr, 494 op/s 2024-03-21T11:45:43.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:43 smithi102 ceph-mon[36748]: pgmap v1072: 129 pgs: 129 active+clean; 648 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.9 MiB/s wr, 494 op/s 2024-03-21T11:45:43.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:43 smithi047 ceph-mon[31351]: pgmap v1072: 129 pgs: 129 active+clean; 648 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.9 MiB/s wr, 494 op/s 2024-03-21T11:45:43.981 INFO:tasks.workunit.client.0.smithi047.stdout: 10 218252 22.27 MB/sec execute 387 sec latency 49.555 ms 2024-03-21T11:45:44.981 INFO:tasks.workunit.client.0.smithi047.stdout: 10 218728 22.27 MB/sec execute 388 sec latency 65.038 ms 2024-03-21T11:45:45.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:45 smithi102 ceph-mon[36748]: pgmap v1073: 129 pgs: 129 active+clean; 649 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 457 op/s 2024-03-21T11:45:45.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:45 smithi047 ceph-mon[31351]: pgmap v1073: 129 pgs: 129 active+clean; 649 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 457 op/s 2024-03-21T11:45:45.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:45 smithi100 ceph-mon[36245]: pgmap v1073: 129 pgs: 129 active+clean; 649 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 457 op/s 2024-03-21T11:45:45.981 INFO:tasks.workunit.client.0.smithi047.stdout: 10 219206 22.28 MB/sec execute 389 sec latency 60.436 ms 2024-03-21T11:45:46.981 INFO:tasks.workunit.client.0.smithi047.stdout: 10 219655 22.29 MB/sec execute 390 sec latency 387.886 ms 2024-03-21T11:45:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.984+0000 7fc2afa00640 1 -- 172.21.15.47:0/1733299651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a81054b0 msgr2=0x7fc2a810bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.984+0000 7fc2afa00640 1 --2- 172.21.15.47:0/1733299651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a81054b0 0x7fc2a810bd40 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009ca0 tx=0x7fc2a002c100 comp rx=0 tx=0).stop 2024-03-21T11:45:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.984+0000 7fc2afa00640 1 -- 172.21.15.47:0/1733299651 shutdown_connections 2024-03-21T11:45:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.984+0000 7fc2afa00640 1 --2- 172.21.15.47:0/1733299651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a81054b0 0x7fc2a810bd40 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.984+0000 7fc2afa00640 1 --2- 172.21.15.47:0/1733299651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2a8104af0 0x7fc2a8104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.984+0000 7fc2afa00640 1 --2- 172.21.15.47:0/1733299651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2a8103900 0x7fc2a8103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.984+0000 7fc2afa00640 1 -- 172.21.15.47:0/1733299651 >> 172.21.15.47:0/1733299651 conn(0x7fc2a80fee90 msgr2=0x7fc2a81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.984+0000 7fc2afa00640 1 -- 172.21.15.47:0/1733299651 shutdown_connections 2024-03-21T11:45:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.984+0000 7fc2afa00640 1 -- 172.21.15.47:0/1733299651 wait complete. 2024-03-21T11:45:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.985+0000 7fc2afa00640 1 Processor -- start 2024-03-21T11:45:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.986+0000 7fc2afa00640 1 -- start start 2024-03-21T11:45:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.986+0000 7fc2afa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2a8103900 0x7fc2a8199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.986+0000 7fc2afa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2a8104af0 0x7fc2a819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.986+0000 7fc2afa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a81054b0 0x7fc2a81a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.986+0000 7fc2afa00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a810f580 con 0x7fc2a8104af0 2024-03-21T11:45:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.986+0000 7fc2afa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a810f400 con 0x7fc2a81054b0 2024-03-21T11:45:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.986+0000 7fc2afa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a810f700 con 0x7fc2a8103900 2024-03-21T11:45:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.986+0000 7fc2ad000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2a8103900 0x7fc2a8199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.986+0000 7fc2ad000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2a8103900 0x7fc2a8199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52946/0 (socket says 172.21.15.47:52946) 2024-03-21T11:45:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.986+0000 7fc2ad000640 1 -- 172.21.15.47:0/1595973345 learned_addr learned my addr 172.21.15.47:0/1595973345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.986+0000 7fc2a7e00640 1 --2- 172.21.15.47:0/1595973345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2a8104af0 0x7fc2a819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.986+0000 7fc2ada00640 1 --2- 172.21.15.47:0/1595973345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a81054b0 0x7fc2a81a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.987+0000 7fc2ad000640 1 -- 172.21.15.47:0/1595973345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a81054b0 msgr2=0x7fc2a81a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.987+0000 7fc2ad000640 1 --2- 172.21.15.47:0/1595973345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a81054b0 0x7fc2a81a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.987+0000 7fc2ad000640 1 -- 172.21.15.47:0/1595973345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2a8104af0 msgr2=0x7fc2a819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.987+0000 7fc2ad000640 1 --2- 172.21.15.47:0/1595973345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2a8104af0 0x7fc2a819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.987+0000 7fc2ad000640 1 -- 172.21.15.47:0/1595973345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a81a1730 con 0x7fc2a8103900 2024-03-21T11:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.987+0000 7fc2ad000640 1 --2- 172.21.15.47:0/1595973345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2a8103900 0x7fc2a8199b10 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00e9c0 tx=0x7fc29c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.987+0000 7fc2a5600640 1 -- 172.21.15.47:0/1595973345 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c00cd80 con 0x7fc2a8103900 2024-03-21T11:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.988+0000 7fc2a5600640 1 -- 172.21.15.47:0/1595973345 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc29c0035f0 con 0x7fc2a8103900 2024-03-21T11:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.988+0000 7fc2afa00640 1 -- 172.21.15.47:0/1595973345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a81a1a20 con 0x7fc2a8103900 2024-03-21T11:45:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.988+0000 7fc2a5600640 1 -- 172.21.15.47:0/1595973345 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c010690 con 0x7fc2a8103900 2024-03-21T11:45:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.988+0000 7fc2afa00640 1 -- 172.21.15.47:0/1595973345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a81a1f20 con 0x7fc2a8103900 2024-03-21T11:45:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.991+0000 7fc2b27eb740 1 Processor -- start 2024-03-21T11:45:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.991+0000 7fc2b27eb740 1 -- start start 2024-03-21T11:45:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2a5600640 1 -- 172.21.15.47:0/1595973345 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc29c0037a0 con 0x7fc2a8103900 2024-03-21T11:45:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2b27eb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 0x55e9d3931600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2b27eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3931bd0 0x55e9d39366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2b27eb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3936d40 0x55e9d39391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2b27eb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d38f43b0 con 0x55e9d39311e0 2024-03-21T11:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2b27eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d3896830 con 0x55e9d3931bd0 2024-03-21T11:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2b27eb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d383f5d0 con 0x55e9d3936d40 2024-03-21T11:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2ad000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3931bd0 0x55e9d39366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3936d40 0x55e9d39391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2ad000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3931bd0 0x55e9d39366d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39046/0 (socket says 172.21.15.47:39046) 2024-03-21T11:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2ad000640 1 -- 172.21.15.47:0/863857070 learned_addr learned my addr 172.21.15.47:0/863857070 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2ada00640 1 --2- 172.21.15.47:0/863857070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 0x55e9d3931600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2ad000640 1 -- 172.21.15.47:0/863857070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3936d40 msgr2=0x55e9d39391d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2ad000640 1 --2- 172.21.15.47:0/863857070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3936d40 0x55e9d39391d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2ad000640 1 -- 172.21.15.47:0/863857070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 msgr2=0x55e9d3931600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2ad000640 1 --2- 172.21.15.47:0/863857070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 0x55e9d3931600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.992+0000 7fc2ad000640 1 -- 172.21.15.47:0/863857070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9d38572c0 con 0x55e9d3931bd0 2024-03-21T11:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.993+0000 7fc2ad000640 1 --2- 172.21.15.47:0/863857070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3931bd0 0x55e9d39366d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0d3ae0 tx=0x7fc29c0d3bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd1566c01bc7044a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.993+0000 7fc29a000640 1 -- 172.21.15.47:0/863857070 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0d8bb0 con 0x55e9d3931bd0 2024-03-21T11:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.993+0000 7fc29a000640 1 -- 172.21.15.47:0/863857070 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc29c0d3e20 con 0x55e9d3931bd0 2024-03-21T11:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.993+0000 7fc29a000640 1 -- 172.21.15.47:0/863857070 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0dc640 con 0x55e9d3931bd0 2024-03-21T11:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.993+0000 7fc2a5600640 1 --2- 172.21.15.47:0/1595973345 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc2900a2fc0 0x7fc2900a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.994+0000 7fc2a5600640 1 -- 172.21.15.47:0/1595973345 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc29c0cfb40 con 0x7fc2a8103900 2024-03-21T11:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.994+0000 7fc2ada00640 1 --2- 172.21.15.47:0/1595973345 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc2900a2fc0 0x7fc2900a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.994+0000 7fc2b27eb740 1 -- 172.21.15.47:0/863857070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3931bd0 msgr2=0x55e9d39366d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.994+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/863857070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3931bd0 0x55e9d39366d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0d3ae0 tx=0x7fc29c0d3bd0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.994+0000 7fc2b27eb740 1 -- 172.21.15.47:0/863857070 shutdown_connections 2024-03-21T11:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.994+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/863857070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3936d40 0x55e9d39391d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.994+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/863857070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3931bd0 0x55e9d39366d0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.994+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/863857070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 0x55e9d3931600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.994+0000 7fc2b27eb740 1 -- 172.21.15.47:0/863857070 >> 172.21.15.47:0/863857070 conn(0x55e9d392c750 msgr2=0x55e9d392cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.994+0000 7fc2b27eb740 1 -- 172.21.15.47:0/863857070 shutdown_connections 2024-03-21T11:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.994+0000 7fc2b27eb740 1 -- 172.21.15.47:0/863857070 wait complete. 2024-03-21T11:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.995+0000 7fc2b27eb740 1 -- start start 2024-03-21T11:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.995+0000 7fc2ada00640 1 --2- 172.21.15.47:0/1595973345 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc2900a2fc0 0x7fc2900a5480 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc2a002c640 tx=0x7fc2a0032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.995+0000 7fc2b27eb740 1 Processor -- start 2024-03-21T11:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.995+0000 7fc2b27eb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 0x55e9d393b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.995+0000 7fc2b27eb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3931bd0 0x55e9d393b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.995+0000 7fc2b27eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3936d40 0x55e9d3942940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.995+0000 7fc2b27eb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d3852a20 con 0x55e9d39311e0 2024-03-21T11:45:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.996+0000 7fc2b27eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d38928d0 con 0x55e9d3936d40 2024-03-21T11:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.996+0000 7fc2b27eb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9d3939cf0 con 0x55e9d3931bd0 2024-03-21T11:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.996+0000 7fc2a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3931bd0 0x55e9d393b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.996+0000 7fc2ad000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 0x55e9d393b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.996+0000 7fc2ada00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3936d40 0x55e9d3942940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.996+0000 7fc2b27eb740 10 client.0 _ll_register_callbacks cb 0x55e9d388f1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.996+0000 7fc2a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3931bd0 0x55e9d393b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52964/0 (socket says 172.21.15.47:52964) 2024-03-21T11:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.996+0000 7fc2a7e00640 1 -- 172.21.15.47:0/524264242 learned_addr learned my addr 172.21.15.47:0/524264242 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.996+0000 7fc2a7e00640 1 -- 172.21.15.47:0/524264242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3936d40 msgr2=0x55e9d3942940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.996+0000 7fc2a7e00640 1 --2- 172.21.15.47:0/524264242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3936d40 0x55e9d3942940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.996+0000 7fc2a7e00640 1 -- 172.21.15.47:0/524264242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 msgr2=0x55e9d393b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.996+0000 7fc2a7e00640 1 --2- 172.21.15.47:0/524264242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 0x55e9d393b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.996+0000 7fc2a7e00640 1 -- 172.21.15.47:0/524264242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9d35ab210 con 0x55e9d3931bd0 2024-03-21T11:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.996+0000 7fc2ad000640 1 --2- 172.21.15.47:0/524264242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 0x55e9d393b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.997+0000 7fc2a7e00640 1 --2- 172.21.15.47:0/524264242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3931bd0 0x55e9d393b900 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc294012720 tx=0x7fc294012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ef4eabfe6eccef7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.997+0000 7fc28ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.997+0000 7fc28ea00640 1 -- 172.21.15.47:0/524264242 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29400f8c0 con 0x55e9d3931bd0 2024-03-21T11:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.997+0000 7fc2b27eb740 1 -- 172.21.15.47:0/524264242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc29c0d3e20 con 0x55e9d3931bd0 2024-03-21T11:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.997+0000 7fc28ea00640 1 -- 172.21.15.47:0/524264242 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc29400fa60 con 0x55e9d3931bd0 2024-03-21T11:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.997+0000 7fc28ea00640 1 -- 172.21.15.47:0/524264242 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc29400fca0 con 0x55e9d3931bd0 2024-03-21T11:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.997+0000 7fc2b27eb740 10 client.32671 fetch_fsmap learned FSMap version 41 2024-03-21T11:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.997+0000 7fc2b27eb740 1 -- 172.21.15.47:0/524264242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e9d3677620 con 0x55e9d3931bd0 2024-03-21T11:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.997+0000 7fc28ea00640 1 -- 172.21.15.47:0/524264242 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc29c0d3e20 con 0x55e9d3931bd0 2024-03-21T11:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.998+0000 7fc28ea00640 1 -- 172.21.15.47:0/524264242 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc294005400 con 0x55e9d3931bd0 2024-03-21T11:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.998+0000 7fc2b27eb740 10 client.32671 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:45:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.998+0000 7fc2b27eb740 10 client.32671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.998+0000 7fc2b27eb740 20 client.32671 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.998+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/524264242 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9d3947230 0x55e9d39496a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.998+0000 7fc2b27eb740 4 client.32671 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:45:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.998+0000 7fc2b27eb740 1 -- 172.21.15.47:0/524264242 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9d38a3d70 con 0x55e9d3947230 2024-03-21T11:45:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.998+0000 7fc2ad000640 1 --2- 172.21.15.47:0/524264242 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9d3947230 0x55e9d39496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.999+0000 7fc2ad000640 1 --2- 172.21.15.47:0/524264242 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9d3947230 0x55e9d39496a0 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d87038e1fdb5fd2 server_cookie=7e686330f4f08912 in_seq=0 out_seq=0 2024-03-21T11:45:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:46.999+0000 7fc28ea00640 10 client.32671 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.000+0000 7fc28ea00640 1 -- 172.21.15.47:0/524264242 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e9d38a3d70 con 0x55e9d3947230 2024-03-21T11:45:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.000+0000 7fc28ea00640 10 client.32671 handle_command_reply: tid=0 2024-03-21T11:45:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.000+0000 7fc2b27eb740 1 client.32671 shutdown 2024-03-21T11:45:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.000+0000 7fc2b27eb740 1 -- 172.21.15.47:0/524264242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3931bd0 msgr2=0x55e9d393b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.000+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/524264242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3931bd0 0x55e9d393b900 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc294012720 tx=0x7fc294012be0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.000+0000 7fc2b27eb740 1 -- 172.21.15.47:0/524264242 shutdown_connections 2024-03-21T11:45:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.000+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/524264242 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9d3947230 0x55e9d39496a0 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.000+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/524264242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3936d40 0x55e9d3942940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.000+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/524264242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3931bd0 0x55e9d393b900 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.000+0000 7fc28ea00640 0 client.32671 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.000+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/524264242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 0x55e9d393b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.001+0000 7fc2b27eb740 1 -- 172.21.15.47:0/524264242 >> 172.21.15.47:0/524264242 conn(0x55e9d392c750 msgr2=0x55e9d392b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.001+0000 7fc2b27eb740 1 -- 172.21.15.47:0/524264242 shutdown_connections 2024-03-21T11:45:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.001+0000 7fc2b27eb740 1 -- 172.21.15.47:0/524264242 wait complete. 2024-03-21T11:45:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.001+0000 7fc2b27eb740 20 client.32671 trim_cache size 0 max 16384 2024-03-21T11:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.017+0000 7fc2b27eb740 1 Processor -- start 2024-03-21T11:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.017+0000 7fc2b27eb740 1 -- start start 2024-03-21T11:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.017+0000 7fc2b27eb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d39311e0 0x55e9d392f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.017+0000 7fc2b27eb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d3931bd0 0x55e9d3942900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.017+0000 7fc2b27eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3936d40 0x55e9d392fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.017+0000 7fc2b27eb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81a7ea0 con 0x55e9d3931bd0 2024-03-21T11:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.017+0000 7fc2b27eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81a9820 con 0x55e9d3936d40 2024-03-21T11:45:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.017+0000 7fc2b27eb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81a8500 con 0x55e9d39311e0 2024-03-21T11:45:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.017+0000 7fc2ad000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d39311e0 0x55e9d392f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.017+0000 7fc2a7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d3931bd0 0x55e9d3942900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.017+0000 7fc2ada00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3936d40 0x55e9d392fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.017+0000 7fc2ad000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d39311e0 0x55e9d392f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:52970/0 (socket says 172.21.15.47:52970) 2024-03-21T11:45:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.017+0000 7fc2ad000640 1 -- 172.21.15.47:0/2902448066 learned_addr learned my addr 172.21.15.47:0/2902448066 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.018+0000 7fc2ad000640 1 -- 172.21.15.47:0/2902448066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3936d40 msgr2=0x55e9d392fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.018+0000 7fc2ad000640 1 --2- 172.21.15.47:0/2902448066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3936d40 0x55e9d392fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.018+0000 7fc2ad000640 1 -- 172.21.15.47:0/2902448066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d3931bd0 msgr2=0x55e9d3942900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.018+0000 7fc2ad000640 1 --2- 172.21.15.47:0/2902448066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d3931bd0 0x55e9d3942900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.018+0000 7fc2ad000640 1 -- 172.21.15.47:0/2902448066 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9d39329e0 con 0x55e9d39311e0 2024-03-21T11:45:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.018+0000 7fc2a7e00640 1 --2- 172.21.15.47:0/2902448066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d3931bd0 0x55e9d3942900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:45:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.018+0000 7fc2ad000640 1 --2- 172.21.15.47:0/2902448066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d39311e0 0x55e9d392f9b0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c6a0 tx=0x7fc29c003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a6f1668bc95fb2f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.018+0000 7fc299600640 1 -- 172.21.15.47:0/2902448066 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c09bc50 con 0x55e9d39311e0 2024-03-21T11:45:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.019+0000 7fc299600640 1 -- 172.21.15.47:0/2902448066 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc29c0d3e20 con 0x55e9d39311e0 2024-03-21T11:45:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.019+0000 7fc299600640 1 -- 172.21.15.47:0/2902448066 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0bc0b0 con 0x55e9d39311e0 2024-03-21T11:45:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.019+0000 7fc2b27eb740 1 -- 172.21.15.47:0/2902448066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d39311e0 msgr2=0x55e9d392f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.019+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/2902448066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d39311e0 0x55e9d392f9b0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c6a0 tx=0x7fc29c003f10 comp rx=0 tx=0).stop 2024-03-21T11:45:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.019+0000 7fc2b27eb740 1 -- 172.21.15.47:0/2902448066 shutdown_connections 2024-03-21T11:45:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.019+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/2902448066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3936d40 0x55e9d392fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.019+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/2902448066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d3931bd0 0x55e9d3942900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.019+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/2902448066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d39311e0 0x55e9d392f9b0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.019+0000 7fc2b27eb740 1 -- 172.21.15.47:0/2902448066 >> 172.21.15.47:0/2902448066 conn(0x55e9d392c750 msgr2=0x55e9d3932ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.020+0000 7fc2b27eb740 1 -- 172.21.15.47:0/2902448066 shutdown_connections 2024-03-21T11:45:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.020+0000 7fc2b27eb740 1 -- 172.21.15.47:0/2902448066 wait complete. 2024-03-21T11:45:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.020+0000 7fc2b27eb740 1 -- start start 2024-03-21T11:45:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.020+0000 7fc2b27eb740 1 Processor -- start 2024-03-21T11:45:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2b27eb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 0x55e9d3940130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2b27eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3931bd0 0x55e9d3940670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2b27eb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3936d40 0x55e9d39415e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2b27eb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81a91c0 con 0x55e9d39311e0 2024-03-21T11:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2b27eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81a8b60 con 0x55e9d3931bd0 2024-03-21T11:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2b27eb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81a6080 con 0x55e9d3936d40 2024-03-21T11:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2b27eb740 10 client.0 _ll_register_callbacks cb 0x7fc294009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2ad000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 0x55e9d3940130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3931bd0 0x55e9d3940670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2ad000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 0x55e9d3940130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58136/0 (socket says 172.21.15.47:58136) 2024-03-21T11:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3931bd0 0x55e9d3940670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39080/0 (socket says 172.21.15.47:39080) 2024-03-21T11:45:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2a7e00640 1 -- 172.21.15.47:0/3287043995 learned_addr learned my addr 172.21.15.47:0/3287043995 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:45:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2ada00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3936d40 0x55e9d39415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2a7e00640 1 -- 172.21.15.47:0/3287043995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3936d40 msgr2=0x55e9d39415e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2a7e00640 1 --2- 172.21.15.47:0/3287043995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3936d40 0x55e9d39415e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2a7e00640 1 -- 172.21.15.47:0/3287043995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 msgr2=0x55e9d3940130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:45:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2a7e00640 1 --2- 172.21.15.47:0/3287043995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 0x55e9d3940130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.021+0000 7fc2a7e00640 1 -- 172.21.15.47:0/3287043995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9d3941bd0 con 0x55e9d3931bd0 2024-03-21T11:45:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.022+0000 7fc2a7e00640 1 --2- 172.21.15.47:0/3287043995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3931bd0 0x55e9d3940670 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fc2940096f0 tx=0x7fc29400fa40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f93eab091dc3546 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:45:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.022+0000 7fc28ea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:45:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.022+0000 7fc28ea00640 1 -- 172.21.15.47:0/3287043995 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29400fe40 con 0x55e9d3931bd0 2024-03-21T11:45:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.022+0000 7fc28ea00640 1 -- 172.21.15.47:0/3287043995 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc29401cad0 con 0x55e9d3931bd0 2024-03-21T11:45:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.022+0000 7fc2b27eb740 1 -- 172.21.15.47:0/3287043995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc29c0d3e20 con 0x55e9d3931bd0 2024-03-21T11:45:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.022+0000 7fc28ea00640 1 -- 172.21.15.47:0/3287043995 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc29401ccd0 con 0x55e9d3931bd0 2024-03-21T11:45:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.022+0000 7fc2b27eb740 10 client.32702 fetch_fsmap learned FSMap version 41 2024-03-21T11:45:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.022+0000 7fc2b27eb740 1 -- 172.21.15.47:0/3287043995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e9d39a2c40 con 0x55e9d3931bd0 2024-03-21T11:45:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.022+0000 7fc28ea00640 1 -- 172.21.15.47:0/3287043995 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc29c0d3e20 con 0x55e9d3931bd0 2024-03-21T11:45:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.023+0000 7fc28ea00640 1 -- 172.21.15.47:0/3287043995 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc294003c40 con 0x55e9d3931bd0 2024-03-21T11:45:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.023+0000 7fc2b27eb740 10 client.32702 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:45:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.023+0000 7fc2b27eb740 10 client.32702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:45:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.023+0000 7fc2b27eb740 20 client.32702 populate_metadata read hostname 'smithi047' 2024-03-21T11:45:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.023+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/3287043995 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9d3947230 0x55e9d39a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:45:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.023+0000 7fc2b27eb740 4 client.32702 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:45:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.023+0000 7fc2b27eb740 1 -- 172.21.15.47:0/3287043995 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e9d389e0c0 con 0x55e9d3947230 2024-03-21T11:45:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.023+0000 7fc2ad000640 1 --2- 172.21.15.47:0/3287043995 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9d3947230 0x55e9d39a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:45:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.024+0000 7fc2ad000640 1 --2- 172.21.15.47:0/3287043995 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9d3947230 0x55e9d39a53b0 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d79195994fa6387b server_cookie=a276defa4a79161a in_seq=0 out_seq=0 2024-03-21T11:45:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:47.024+0000 7fc28ea00640 10 client.32702 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:47.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:47 smithi102 ceph-mon[36748]: pgmap v1074: 129 pgs: 129 active+clean; 657 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.9 MiB/s wr, 441 op/s 2024-03-21T11:45:47.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:47 smithi047 ceph-mon[31351]: pgmap v1074: 129 pgs: 129 active+clean; 657 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.9 MiB/s wr, 441 op/s 2024-03-21T11:45:47.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:47 smithi100 ceph-mon[36245]: pgmap v1074: 129 pgs: 129 active+clean; 657 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.9 MiB/s wr, 441 op/s 2024-03-21T11:45:47.981 INFO:tasks.workunit.client.0.smithi047.stdout: 10 219734 22.24 MB/sec execute 391 sec latency 1262.886 ms 2024-03-21T11:45:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.257+0000 7fc28ea00640 1 -- 172.21.15.47:0/3287043995 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e9d389e0c0 con 0x55e9d3947230 2024-03-21T11:45:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.257+0000 7fc28ea00640 10 client.32702 handle_command_reply: tid=0 2024-03-21T11:45:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.257+0000 7fc2b27eb740 1 client.32702 shutdown 2024-03-21T11:45:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.258+0000 7fc2b27eb740 1 -- 172.21.15.47:0/3287043995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3931bd0 msgr2=0x55e9d3940670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.258+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/3287043995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3931bd0 0x55e9d3940670 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fc2940096f0 tx=0x7fc29400fa40 comp rx=0 tx=0).stop 2024-03-21T11:45:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.258+0000 7fc2b27eb740 1 -- 172.21.15.47:0/3287043995 shutdown_connections 2024-03-21T11:45:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.258+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/3287043995 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e9d3947230 0x55e9d39a53b0 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.258+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/3287043995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9d3936d40 0x55e9d39415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.258+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/3287043995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9d3931bd0 0x55e9d3940670 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.258+0000 7fc2b27eb740 1 --2- 172.21.15.47:0/3287043995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9d39311e0 0x55e9d3940130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.258+0000 7fc2b27eb740 1 -- 172.21.15.47:0/3287043995 >> 172.21.15.47:0/3287043995 conn(0x55e9d392c750 msgr2=0x55e9d392f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.258+0000 7fc28ea00640 0 client.32702 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:45:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.258+0000 7fc2b27eb740 1 -- 172.21.15.47:0/3287043995 shutdown_connections 2024-03-21T11:45:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.258+0000 7fc2b27eb740 1 -- 172.21.15.47:0/3287043995 wait complete. 2024-03-21T11:45:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.259+0000 7fc2b27eb740 20 client.32702 trim_cache size 0 max 16384 2024-03-21T11:45:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:45:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:45:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:45:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:45:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.259+0000 7fc2afa00640 1 -- 172.21.15.47:0/1595973345 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc2900a2fc0 msgr2=0x7fc2900a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.259+0000 7fc2afa00640 1 --2- 172.21.15.47:0/1595973345 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc2900a2fc0 0x7fc2900a5480 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc2a002c640 tx=0x7fc2a0032590 comp rx=0 tx=0).stop 2024-03-21T11:45:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.260+0000 7fc2afa00640 1 -- 172.21.15.47:0/1595973345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2a8103900 msgr2=0x7fc2a8199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:45:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.260+0000 7fc2afa00640 1 --2- 172.21.15.47:0/1595973345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2a8103900 0x7fc2a8199b10 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00e9c0 tx=0x7fc29c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:45:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.260+0000 7fc2afa00640 1 -- 172.21.15.47:0/1595973345 shutdown_connections 2024-03-21T11:45:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.260+0000 7fc2afa00640 1 --2- 172.21.15.47:0/1595973345 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc2900a2fc0 0x7fc2900a5480 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.260+0000 7fc2afa00640 1 --2- 172.21.15.47:0/1595973345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a81054b0 0x7fc2a81a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.260+0000 7fc2afa00640 1 --2- 172.21.15.47:0/1595973345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2a8104af0 0x7fc2a819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.260+0000 7fc2afa00640 1 --2- 172.21.15.47:0/1595973345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2a8103900 0x7fc2a8199b10 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:45:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.260+0000 7fc2afa00640 1 -- 172.21.15.47:0/1595973345 >> 172.21.15.47:0/1595973345 conn(0x7fc2a80fee90 msgr2=0x7fc2a8107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:45:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.260+0000 7fc2afa00640 1 -- 172.21.15.47:0/1595973345 shutdown_connections 2024-03-21T11:45:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:45:48.260+0000 7fc2afa00640 1 -- 172.21.15.47:0/1595973345 wait complete. 2024-03-21T11:45:48.982 INFO:tasks.workunit.client.0.smithi047.stdout: 10 220156 22.25 MB/sec execute 392 sec latency 852.189 ms 2024-03-21T11:45:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:49 smithi102 ceph-mon[36748]: pgmap v1075: 129 pgs: 129 active+clean; 658 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 2.7 MiB/s wr, 395 op/s 2024-03-21T11:45:49.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:49 smithi047 ceph-mon[31351]: pgmap v1075: 129 pgs: 129 active+clean; 658 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 2.7 MiB/s wr, 395 op/s 2024-03-21T11:45:49.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:49 smithi100 ceph-mon[36245]: pgmap v1075: 129 pgs: 129 active+clean; 658 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 2.7 MiB/s wr, 395 op/s 2024-03-21T11:45:49.981 INFO:tasks.workunit.client.0.smithi047.stdout: 10 220320 22.22 MB/sec execute 393 sec latency 603.880 ms 2024-03-21T11:45:50.981 INFO:tasks.workunit.client.0.smithi047.stdout: 10 220336 22.16 MB/sec execute 394 sec latency 1603.946 ms 2024-03-21T11:45:51.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:51 smithi102 ceph-mon[36748]: pgmap v1076: 129 pgs: 129 active+clean; 663 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.0 MiB/s wr, 481 op/s 2024-03-21T11:45:51.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:51 smithi047 ceph-mon[31351]: pgmap v1076: 129 pgs: 129 active+clean; 663 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.0 MiB/s wr, 481 op/s 2024-03-21T11:45:51.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:51 smithi100 ceph-mon[36245]: pgmap v1076: 129 pgs: 129 active+clean; 663 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.0 MiB/s wr, 481 op/s 2024-03-21T11:45:51.982 INFO:tasks.workunit.client.0.smithi047.stdout: 10 220424 22.12 MB/sec execute 395 sec latency 2052.221 ms 2024-03-21T11:45:52.982 INFO:tasks.workunit.client.0.smithi047.stdout: 10 220646 22.09 MB/sec execute 396 sec latency 217.733 ms 2024-03-21T11:45:53.445 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:53 smithi100 ceph-mon[36245]: pgmap v1077: 129 pgs: 129 active+clean; 678 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.7 MiB/s wr, 535 op/s 2024-03-21T11:45:53.445 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:53 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:45:53.445 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:53 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:45:53.445 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:53 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:45:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:53 smithi102 ceph-mon[36748]: pgmap v1077: 129 pgs: 129 active+clean; 678 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.7 MiB/s wr, 535 op/s 2024-03-21T11:45:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:53 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:45:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:53 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:45:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:53 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:45:53.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:53 smithi047 ceph-mon[31351]: pgmap v1077: 129 pgs: 129 active+clean; 678 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.7 MiB/s wr, 535 op/s 2024-03-21T11:45:53.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:53 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:45:53.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:53 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:45:53.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:53 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:45:53.982 INFO:tasks.workunit.client.0.smithi047.stdout: 10 220857 22.06 MB/sec execute 397 sec latency 239.370 ms 2024-03-21T11:45:54.457 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:54 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:45:54.457 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:54 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:45:54.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:54 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:45:54.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:54 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:45:54.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:54 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:45:54.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:54 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:45:54.982 INFO:tasks.workunit.client.0.smithi047.stdout: 10 221038 22.02 MB/sec execute 398 sec latency 152.005 ms 2024-03-21T11:45:55.352 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:55 smithi100 ceph-mon[36245]: pgmap v1078: 129 pgs: 129 active+clean; 690 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.9 MiB/s wr, 540 op/s 2024-03-21T11:45:55.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:55 smithi102 ceph-mon[36748]: pgmap v1078: 129 pgs: 129 active+clean; 690 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.9 MiB/s wr, 540 op/s 2024-03-21T11:45:55.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:55 smithi047 ceph-mon[31351]: pgmap v1078: 129 pgs: 129 active+clean; 690 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.9 MiB/s wr, 540 op/s 2024-03-21T11:45:55.982 INFO:tasks.workunit.client.0.smithi047.stdout: 10 221269 22.00 MB/sec execute 399 sec latency 155.683 ms 2024-03-21T11:45:56.982 INFO:tasks.workunit.client.0.smithi047.stdout: 10 221556 22.00 MB/sec execute 400 sec latency 132.331 ms 2024-03-21T11:45:57.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:57 smithi102 ceph-mon[36748]: pgmap v1079: 129 pgs: 129 active+clean; 691 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 10 MiB/s wr, 702 op/s 2024-03-21T11:45:57.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:57 smithi047 ceph-mon[31351]: pgmap v1079: 129 pgs: 129 active+clean; 691 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 10 MiB/s wr, 702 op/s 2024-03-21T11:45:57.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:57 smithi100 ceph-mon[36245]: pgmap v1079: 129 pgs: 129 active+clean; 691 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 10 MiB/s wr, 702 op/s 2024-03-21T11:45:57.982 INFO:tasks.workunit.client.0.smithi047.stdout: 10 222051 22.01 MB/sec execute 401 sec latency 74.526 ms 2024-03-21T11:45:58.982 INFO:tasks.workunit.client.0.smithi047.stdout: 10 222467 21.99 MB/sec execute 402 sec latency 114.828 ms 2024-03-21T11:45:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:45:59 smithi102 ceph-mon[36748]: pgmap v1080: 129 pgs: 129 active+clean; 686 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 601 op/s 2024-03-21T11:45:59.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:45:59 smithi047 ceph-mon[31351]: pgmap v1080: 129 pgs: 129 active+clean; 686 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 601 op/s 2024-03-21T11:45:59.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:45:59 smithi100 ceph-mon[36245]: pgmap v1080: 129 pgs: 129 active+clean; 686 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.5 MiB/s wr, 601 op/s 2024-03-21T11:45:59.982 INFO:tasks.workunit.client.0.smithi047.stdout: 10 222975 22.01 MB/sec execute 403 sec latency 43.959 ms 2024-03-21T11:46:00.982 INFO:tasks.workunit.client.0.smithi047.stdout: 10 223607 22.04 MB/sec execute 404 sec latency 74.990 ms 2024-03-21T11:46:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.471+0000 7fb597200640 1 -- 172.21.15.47:0/746416377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590108ef0 msgr2=0x7fb590109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.471+0000 7fb597200640 1 --2- 172.21.15.47:0/746416377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590108ef0 0x7fb590109370 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009900 tx=0x7fb57c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:46:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.471+0000 7fb597200640 1 -- 172.21.15.47:0/746416377 shutdown_connections 2024-03-21T11:46:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.471+0000 7fb597200640 1 --2- 172.21.15.47:0/746416377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5901098b0 0x7fb590110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.471+0000 7fb597200640 1 --2- 172.21.15.47:0/746416377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590108ef0 0x7fb590109370 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.471+0000 7fb597200640 1 --2- 172.21.15.47:0/746416377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb590107d00 0x7fb590108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.471+0000 7fb597200640 1 -- 172.21.15.47:0/746416377 >> 172.21.15.47:0/746416377 conn(0x7fb590076cb0 msgr2=0x7fb5900790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.472+0000 7fb597200640 1 -- 172.21.15.47:0/746416377 shutdown_connections 2024-03-21T11:46:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.472+0000 7fb597200640 1 -- 172.21.15.47:0/746416377 wait complete. 2024-03-21T11:46:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.473+0000 7fb597200640 1 Processor -- start 2024-03-21T11:46:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.473+0000 7fb597200640 1 -- start start 2024-03-21T11:46:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.473+0000 7fb597200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb590107d00 0x7fb59019dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.473+0000 7fb597200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590108ef0 0x7fb59019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.473+0000 7fb597200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5901098b0 0x7fb5901a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.473+0000 7fb597200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5901139b0 con 0x7fb590107d00 2024-03-21T11:46:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.473+0000 7fb597200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb590113830 con 0x7fb5901098b0 2024-03-21T11:46:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.473+0000 7fb597200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb590113b30 con 0x7fb590108ef0 2024-03-21T11:46:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.474+0000 7fb58f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590108ef0 0x7fb59019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.474+0000 7fb595200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5901098b0 0x7fb5901a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.474+0000 7fb58f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590108ef0 0x7fb59019e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42238/0 (socket says 172.21.15.47:42238) 2024-03-21T11:46:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.474+0000 7fb58f400640 1 -- 172.21.15.47:0/3796575904 learned_addr learned my addr 172.21.15.47:0/3796575904 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.475+0000 7fb595200640 1 -- 172.21.15.47:0/3796575904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590108ef0 msgr2=0x7fb59019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.475+0000 7fb595200640 1 --2- 172.21.15.47:0/3796575904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590108ef0 0x7fb59019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.475+0000 7fb595200640 1 -- 172.21.15.47:0/3796575904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb590107d00 msgr2=0x7fb59019dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.475+0000 7fb595200640 1 --2- 172.21.15.47:0/3796575904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb590107d00 0x7fb59019dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.475+0000 7fb595200640 1 -- 172.21.15.47:0/3796575904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59006b1f0 con 0x7fb5901098b0 2024-03-21T11:46:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.475+0000 7fb58f400640 1 --2- 172.21.15.47:0/3796575904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590108ef0 0x7fb59019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.475+0000 7fb595200640 1 --2- 172.21.15.47:0/3796575904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5901098b0 0x7fb5901a5490 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fb58800cab0 tx=0x7fb58800cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.476+0000 7fb58cc00640 1 -- 172.21.15.47:0/3796575904 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58800ae20 con 0x7fb5901098b0 2024-03-21T11:46:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.476+0000 7fb58cc00640 1 -- 172.21.15.47:0/3796575904 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb58800f930 con 0x7fb5901098b0 2024-03-21T11:46:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.476+0000 7fb58cc00640 1 -- 172.21.15.47:0/3796575904 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588018b30 con 0x7fb5901098b0 2024-03-21T11:46:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.476+0000 7fb597200640 1 -- 172.21.15.47:0/3796575904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59006b4e0 con 0x7fb5901098b0 2024-03-21T11:46:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.476+0000 7fb597200640 1 -- 172.21.15.47:0/3796575904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59006ba00 con 0x7fb5901098b0 2024-03-21T11:46:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.480+0000 7fb59a016740 1 Processor -- start 2024-03-21T11:46:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.483+0000 7fb58cc00640 1 -- 172.21.15.47:0/3796575904 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb58800fae0 con 0x7fb5901098b0 2024-03-21T11:46:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.483+0000 7fb59a016740 1 -- start start 2024-03-21T11:46:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.484+0000 7fb59a016740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f31e0 0x563e219f3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.484+0000 7fb59a016740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f3bd0 0x563e219f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.484+0000 7fb59a016740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f8d40 0x563e219fb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.484+0000 7fb59a016740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e219b63b0 con 0x563e219f3bd0 2024-03-21T11:46:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.484+0000 7fb59a016740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e21958830 con 0x563e219f8d40 2024-03-21T11:46:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.484+0000 7fb59a016740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e219015d0 con 0x563e219f31e0 2024-03-21T11:46:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.484+0000 7fb58f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f31e0 0x563e219f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.485+0000 7fb58f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f31e0 0x563e219f3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42240/0 (socket says 172.21.15.47:42240) 2024-03-21T11:46:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.485+0000 7fb58f400640 1 -- 172.21.15.47:0/1670664181 learned_addr learned my addr 172.21.15.47:0/1670664181 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.485+0000 7fb58f400640 1 --2- 172.21.15.47:0/1670664181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f8d40 0x563e219fb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.485+0000 7fb58fe00640 1 --2- 172.21.15.47:0/1670664181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f3bd0 0x563e219f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.485+0000 7fb58cc00640 1 --2- 172.21.15.47:0/3796575904 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5740a30c0 0x7fb5740a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.486+0000 7fb58cc00640 1 -- 172.21.15.47:0/3796575904 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb5880cfec0 con 0x7fb5901098b0 2024-03-21T11:46:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.486+0000 7fb595200640 1 --2- 172.21.15.47:0/3796575904 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5740a30c0 0x7fb5740a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.486+0000 7fb58f400640 1 -- 172.21.15.47:0/1670664181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f8d40 msgr2=0x563e219fb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.486+0000 7fb58f400640 1 --2- 172.21.15.47:0/1670664181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f8d40 0x563e219fb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.486+0000 7fb58f400640 1 -- 172.21.15.47:0/1670664181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f3bd0 msgr2=0x563e219f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.486+0000 7fb58f400640 1 --2- 172.21.15.47:0/1670664181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f3bd0 0x563e219f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.486+0000 7fb58f400640 1 -- 172.21.15.47:0/1670664181 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e219192c0 con 0x563e219f31e0 2024-03-21T11:46:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.487+0000 7fb58f400640 1 --2- 172.21.15.47:0/1670664181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f31e0 0x563e219f3600 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fb57c005ec0 tx=0x7fb57c004f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc8aa46b723a5ec1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.487+0000 7fb585600640 1 -- 172.21.15.47:0/1670664181 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c0046c0 con 0x563e219f31e0 2024-03-21T11:46:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.487+0000 7fb585600640 1 -- 172.21.15.47:0/1670664181 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb57c004f80 con 0x563e219f31e0 2024-03-21T11:46:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.487+0000 7fb585600640 1 -- 172.21.15.47:0/1670664181 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c003850 con 0x563e219f31e0 2024-03-21T11:46:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.488+0000 7fb59a016740 1 -- 172.21.15.47:0/1670664181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f31e0 msgr2=0x563e219f3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.488+0000 7fb59a016740 1 --2- 172.21.15.47:0/1670664181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f31e0 0x563e219f3600 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fb57c005ec0 tx=0x7fb57c004f10 comp rx=0 tx=0).stop 2024-03-21T11:46:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.489+0000 7fb59a016740 1 -- 172.21.15.47:0/1670664181 shutdown_connections 2024-03-21T11:46:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.489+0000 7fb59a016740 1 --2- 172.21.15.47:0/1670664181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f8d40 0x563e219fb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.489+0000 7fb59a016740 1 --2- 172.21.15.47:0/1670664181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f3bd0 0x563e219f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.489+0000 7fb59a016740 1 --2- 172.21.15.47:0/1670664181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f31e0 0x563e219f3600 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.489+0000 7fb59a016740 1 -- 172.21.15.47:0/1670664181 >> 172.21.15.47:0/1670664181 conn(0x563e219ee750 msgr2=0x563e219eeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.490+0000 7fb595200640 1 --2- 172.21.15.47:0/3796575904 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5740a30c0 0x7fb5740a5580 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb58809c190 tx=0x7fb5880d0110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:01.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:01 smithi102 ceph-mon[36748]: pgmap v1081: 129 pgs: 129 active+clean; 674 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.8 MiB/s wr, 674 op/s 2024-03-21T11:46:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.491+0000 7fb59a016740 1 -- 172.21.15.47:0/1670664181 shutdown_connections 2024-03-21T11:46:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.491+0000 7fb59a016740 1 -- 172.21.15.47:0/1670664181 wait complete. 2024-03-21T11:46:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.491+0000 7fb59a016740 1 -- start start 2024-03-21T11:46:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.492+0000 7fb59a016740 1 Processor -- start 2024-03-21T11:46:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.493+0000 7fb59a016740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f31e0 0x563e21a0e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.493+0000 7fb59a016740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f3bd0 0x563e21a0e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.493+0000 7fb59a016740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f8d40 0x563e21a15980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.493+0000 7fb59a016740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e21914a20 con 0x563e219f31e0 2024-03-21T11:46:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.493+0000 7fb59a016740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e219548d0 con 0x563e219f3bd0 2024-03-21T11:46:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.493+0000 7fb59a016740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e21a0cc60 con 0x563e219f8d40 2024-03-21T11:46:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.493+0000 7fb59a016740 10 client.0 _ll_register_callbacks cb 0x563e219511c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.493+0000 7fb58fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f3bd0 0x563e21a0e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.493+0000 7fb58fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f3bd0 0x563e21a0e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51248/0 (socket says 172.21.15.47:51248) 2024-03-21T11:46:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.493+0000 7fb58fe00640 1 -- 172.21.15.47:0/2768750196 learned_addr learned my addr 172.21.15.47:0/2768750196 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.493+0000 7fb58f400640 1 --2- 172.21.15.47:0/2768750196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f31e0 0x563e21a0e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.494+0000 7fb58f400640 1 --2- 172.21.15.47:0/2768750196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f8d40 0x563e21a15980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.494+0000 7fb58fe00640 1 -- 172.21.15.47:0/2768750196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f8d40 msgr2=0x563e21a15980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.494+0000 7fb58fe00640 1 --2- 172.21.15.47:0/2768750196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f8d40 0x563e21a15980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.494+0000 7fb58fe00640 1 -- 172.21.15.47:0/2768750196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f31e0 msgr2=0x563e21a0e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.494+0000 7fb58fe00640 1 --2- 172.21.15.47:0/2768750196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f31e0 0x563e21a0e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.494+0000 7fb58fe00640 1 -- 172.21.15.47:0/2768750196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e2166d210 con 0x563e219f3bd0 2024-03-21T11:46:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.494+0000 7fb58fe00640 1 --2- 172.21.15.47:0/2768750196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f3bd0 0x563e21a0e940 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fb58000f670 tx=0x7fb58000fb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7e9fe8a86e7d116 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.494+0000 7fb57a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.495+0000 7fb57a000640 1 -- 172.21.15.47:0/2768750196 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58000daa0 con 0x563e219f3bd0 2024-03-21T11:46:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.495+0000 7fb58f400640 1 --2- 172.21.15.47:0/2768750196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f8d40 0x563e21a15980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.495+0000 7fb57a000640 1 -- 172.21.15.47:0/2768750196 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb58000dc40 con 0x563e219f3bd0 2024-03-21T11:46:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.495+0000 7fb59a016740 1 -- 172.21.15.47:0/2768750196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb57c004f80 con 0x563e219f3bd0 2024-03-21T11:46:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.496+0000 7fb57a000640 1 -- 172.21.15.47:0/2768750196 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb58000de40 con 0x563e219f3bd0 2024-03-21T11:46:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.496+0000 7fb59a016740 10 client.32711 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.496+0000 7fb59a016740 1 -- 172.21.15.47:0/2768750196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563e21739620 con 0x563e219f3bd0 2024-03-21T11:46:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.496+0000 7fb57a000640 1 -- 172.21.15.47:0/2768750196 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb57c004f80 con 0x563e219f3bd0 2024-03-21T11:46:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.497+0000 7fb57a000640 1 -- 172.21.15.47:0/2768750196 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb5800106f0 con 0x563e219f3bd0 2024-03-21T11:46:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.497+0000 7fb59a016740 10 client.32711 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.497+0000 7fb59a016740 10 client.32711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.497+0000 7fb59a016740 20 client.32711 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.497+0000 7fb59a016740 1 --2- 172.21.15.47:0/2768750196 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e21a1a270 0x563e21a1c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.497+0000 7fb59a016740 4 client.32711 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.497+0000 7fb59a016740 1 -- 172.21.15.47:0/2768750196 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e21965d70 con 0x563e21a1a270 2024-03-21T11:46:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.499+0000 7fb58f400640 1 --2- 172.21.15.47:0/2768750196 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e21a1a270 0x563e21a1c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.500+0000 7fb58f400640 1 --2- 172.21.15.47:0/2768750196 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e21a1a270 0x563e21a1c660 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=639aae5971efe0c9 server_cookie=1a97aab5cc32d78a in_seq=0 out_seq=0 2024-03-21T11:46:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.500+0000 7fb57a000640 10 client.32711 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.504+0000 7fb57a000640 1 -- 172.21.15.47:0/2768750196 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563e21965d70 con 0x563e21a1a270 2024-03-21T11:46:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.505+0000 7fb57a000640 10 client.32711 handle_command_reply: tid=0 2024-03-21T11:46:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.505+0000 7fb59a016740 1 client.32711 shutdown 2024-03-21T11:46:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.505+0000 7fb59a016740 1 -- 172.21.15.47:0/2768750196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f3bd0 msgr2=0x563e21a0e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.505+0000 7fb59a016740 1 --2- 172.21.15.47:0/2768750196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f3bd0 0x563e21a0e940 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fb58000f670 tx=0x7fb58000fb30 comp rx=0 tx=0).stop 2024-03-21T11:46:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.505+0000 7fb59a016740 1 -- 172.21.15.47:0/2768750196 shutdown_connections 2024-03-21T11:46:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.505+0000 7fb59a016740 1 --2- 172.21.15.47:0/2768750196 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e21a1a270 0x563e21a1c660 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.505+0000 7fb59a016740 1 --2- 172.21.15.47:0/2768750196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f8d40 0x563e21a15980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.505+0000 7fb59a016740 1 --2- 172.21.15.47:0/2768750196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f3bd0 0x563e21a0e940 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.505+0000 7fb59a016740 1 --2- 172.21.15.47:0/2768750196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f31e0 0x563e21a0e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.505+0000 7fb57a000640 0 client.32711 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.505+0000 7fb59a016740 1 -- 172.21.15.47:0/2768750196 >> 172.21.15.47:0/2768750196 conn(0x563e219ee750 msgr2=0x563e219ed950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.506+0000 7fb59a016740 1 -- 172.21.15.47:0/2768750196 shutdown_connections 2024-03-21T11:46:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.506+0000 7fb59a016740 1 -- 172.21.15.47:0/2768750196 wait complete. 2024-03-21T11:46:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.506+0000 7fb59a016740 20 client.32711 trim_cache size 0 max 16384 2024-03-21T11:46:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.528+0000 7fb59a016740 1 Processor -- start 2024-03-21T11:46:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.528+0000 7fb59a016740 1 -- start start 2024-03-21T11:46:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.529+0000 7fb59a016740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f31e0 0x563e219ed0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.529+0000 7fb59a016740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f3bd0 0x563e21a15940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.529+0000 7fb59a016740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f8d40 0x563e219ed600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.529+0000 7fb59a016740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59007c170 con 0x563e219f8d40 2024-03-21T11:46:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.529+0000 7fb59a016740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59007daf0 con 0x563e219f31e0 2024-03-21T11:46:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.529+0000 7fb59a016740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59007c7d0 con 0x563e219f3bd0 2024-03-21T11:46:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.529+0000 7fb58fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f3bd0 0x563e21a15940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.530+0000 7fb58fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f3bd0 0x563e21a15940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42264/0 (socket says 172.21.15.47:42264) 2024-03-21T11:46:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.530+0000 7fb58fe00640 1 -- 172.21.15.47:0/1844425464 learned_addr learned my addr 172.21.15.47:0/1844425464 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.530+0000 7fb58f400640 1 --2- 172.21.15.47:0/1844425464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f8d40 0x563e219ed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.530+0000 7fb58fe00640 1 -- 172.21.15.47:0/1844425464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f31e0 msgr2=0x563e219ed0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:46:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.530+0000 7fb58fe00640 1 --2- 172.21.15.47:0/1844425464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f31e0 0x563e219ed0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.530+0000 7fb58fe00640 1 -- 172.21.15.47:0/1844425464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f8d40 msgr2=0x563e219ed600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.530+0000 7fb58fe00640 1 --2- 172.21.15.47:0/1844425464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f8d40 0x563e219ed600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.530+0000 7fb58fe00640 1 -- 172.21.15.47:0/1844425464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e21a1c9d0 con 0x563e219f3bd0 2024-03-21T11:46:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.530+0000 7fb58f400640 1 --2- 172.21.15.47:0/1844425464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f8d40 0x563e219ed600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.531+0000 7fb58fe00640 1 --2- 172.21.15.47:0/1844425464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f3bd0 0x563e21a15940 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fb580010bb0 tx=0x7fb58000a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b7f77614cf32a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.531+0000 7fb584c00640 1 -- 172.21.15.47:0/1844425464 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580030020 con 0x563e219f3bd0 2024-03-21T11:46:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.531+0000 7fb584c00640 1 -- 172.21.15.47:0/1844425464 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb580010da0 con 0x563e219f3bd0 2024-03-21T11:46:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.531+0000 7fb584c00640 1 -- 172.21.15.47:0/1844425464 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580010690 con 0x563e219f3bd0 2024-03-21T11:46:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.532+0000 7fb59a016740 1 -- 172.21.15.47:0/1844425464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f3bd0 msgr2=0x563e21a15940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.532+0000 7fb59a016740 1 --2- 172.21.15.47:0/1844425464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f3bd0 0x563e21a15940 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fb580010bb0 tx=0x7fb58000a300 comp rx=0 tx=0).stop 2024-03-21T11:46:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.532+0000 7fb59a016740 1 -- 172.21.15.47:0/1844425464 shutdown_connections 2024-03-21T11:46:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.532+0000 7fb59a016740 1 --2- 172.21.15.47:0/1844425464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f8d40 0x563e219ed600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.532+0000 7fb59a016740 1 --2- 172.21.15.47:0/1844425464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e219f3bd0 0x563e21a15940 secure :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fb580010bb0 tx=0x7fb58000a300 comp rx=0 tx=0).stop 2024-03-21T11:46:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.532+0000 7fb59a016740 1 --2- 172.21.15.47:0/1844425464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f31e0 0x563e219ed0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.532+0000 7fb59a016740 1 -- 172.21.15.47:0/1844425464 >> 172.21.15.47:0/1844425464 conn(0x563e219ee750 msgr2=0x563e219f4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.532+0000 7fb59a016740 1 -- 172.21.15.47:0/1844425464 shutdown_connections 2024-03-21T11:46:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.532+0000 7fb59a016740 1 -- 172.21.15.47:0/1844425464 wait complete. 2024-03-21T11:46:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.533+0000 7fb59a016740 1 -- start start 2024-03-21T11:46:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.533+0000 7fb59a016740 1 Processor -- start 2024-03-21T11:46:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.533+0000 7fb59a016740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f31e0 0x563e21a13880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.533+0000 7fb59a016740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f8d40 0x563e21a13dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.534+0000 7fb59a016740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e21a14dd0 0x563e21a15250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.534+0000 7fb59a016740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59007d490 con 0x563e219f31e0 2024-03-21T11:46:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.534+0000 7fb59a016740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59007ce30 con 0x563e219f8d40 2024-03-21T11:46:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.534+0000 7fb59a016740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900734b0 con 0x563e21a14dd0 2024-03-21T11:46:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.534+0000 7fb59a016740 10 client.0 _ll_register_callbacks cb 0x7fb58000a4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.534+0000 7fb58fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f8d40 0x563e21a13dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.534+0000 7fb58fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f8d40 0x563e21a13dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51268/0 (socket says 172.21.15.47:51268) 2024-03-21T11:46:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.534+0000 7fb58fe00640 1 -- 172.21.15.47:0/932846374 learned_addr learned my addr 172.21.15.47:0/932846374 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.534+0000 7fb58f400640 1 --2- 172.21.15.47:0/932846374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f31e0 0x563e21a13880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.535+0000 7fb58fe00640 1 -- 172.21.15.47:0/932846374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e21a14dd0 msgr2=0x563e21a15250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:46:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.535+0000 7fb58fe00640 1 --2- 172.21.15.47:0/932846374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e21a14dd0 0x563e21a15250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.535+0000 7fb58fe00640 1 -- 172.21.15.47:0/932846374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f31e0 msgr2=0x563e21a13880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.535+0000 7fb58fe00640 1 --2- 172.21.15.47:0/932846374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f31e0 0x563e21a13880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.535+0000 7fb58fe00640 1 -- 172.21.15.47:0/932846374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e21a15790 con 0x563e219f8d40 2024-03-21T11:46:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.535+0000 7fb58fe00640 1 --2- 172.21.15.47:0/932846374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f8d40 0x563e21a13dc0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb580001eb0 tx=0x7fb58000ab70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cb3d7cff36c8347 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.535+0000 7fb57a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.535+0000 7fb57a000640 1 -- 172.21.15.47:0/932846374 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580030070 con 0x563e219f8d40 2024-03-21T11:46:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.535+0000 7fb59a016740 1 -- 172.21.15.47:0/932846374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb580010da0 con 0x563e219f8d40 2024-03-21T11:46:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.536+0000 7fb57a000640 1 -- 172.21.15.47:0/932846374 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb58000dab0 con 0x563e219f8d40 2024-03-21T11:46:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.536+0000 7fb57a000640 1 -- 172.21.15.47:0/932846374 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb580004140 con 0x563e219f8d40 2024-03-21T11:46:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.536+0000 7fb59a016740 10 client.32714 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.536+0000 7fb59a016740 1 -- 172.21.15.47:0/932846374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563e21a76760 con 0x563e219f8d40 2024-03-21T11:46:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.536+0000 7fb57a000640 1 -- 172.21.15.47:0/932846374 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb580010da0 con 0x563e219f8d40 2024-03-21T11:46:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.537+0000 7fb57a000640 1 -- 172.21.15.47:0/932846374 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb58002b070 con 0x563e219f8d40 2024-03-21T11:46:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.537+0000 7fb59a016740 10 client.32714 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.537+0000 7fb59a016740 10 client.32714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.537+0000 7fb59a016740 20 client.32714 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.537+0000 7fb59a016740 1 --2- 172.21.15.47:0/932846374 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e21a77080 0x563e21a794a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.537+0000 7fb59a016740 4 client.32714 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:46:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.537+0000 7fb59a016740 1 -- 172.21.15.47:0/932846374 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563e219600c0 con 0x563e21a77080 2024-03-21T11:46:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.540+0000 7fb58f400640 1 --2- 172.21.15.47:0/932846374 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e21a77080 0x563e21a794a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.541+0000 7fb58f400640 1 --2- 172.21.15.47:0/932846374 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e21a77080 0x563e21a794a0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c549bb7d424483fd server_cookie=3dcfbb78aafbdeae in_seq=0 out_seq=0 2024-03-21T11:46:01.541 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:01.541+0000 7fb57a000640 10 client.32714 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:01.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:01 smithi047 ceph-mon[31351]: pgmap v1081: 129 pgs: 129 active+clean; 674 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.8 MiB/s wr, 674 op/s 2024-03-21T11:46:01.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:01 smithi100 ceph-mon[36245]: pgmap v1081: 129 pgs: 129 active+clean; 674 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.8 MiB/s wr, 674 op/s 2024-03-21T11:46:01.982 INFO:tasks.workunit.client.0.smithi047.stdout: 10 223809 22.02 MB/sec execute 405 sec latency 436.685 ms 2024-03-21T11:46:02.982 INFO:tasks.workunit.client.0.smithi047.stdout: 10 223917 21.98 MB/sec execute 406 sec latency 1405.606 ms 2024-03-21T11:46:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.026+0000 7fb57a000640 1 -- 172.21.15.47:0/932846374 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563e219600c0 con 0x563e21a77080 2024-03-21T11:46:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.026+0000 7fb57a000640 10 client.32714 handle_command_reply: tid=0 2024-03-21T11:46:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.026+0000 7fb59a016740 1 client.32714 shutdown 2024-03-21T11:46:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.026+0000 7fb59a016740 1 -- 172.21.15.47:0/932846374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f8d40 msgr2=0x563e21a13dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.027+0000 7fb59a016740 1 --2- 172.21.15.47:0/932846374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f8d40 0x563e21a13dc0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb580001eb0 tx=0x7fb58000ab70 comp rx=0 tx=0).stop 2024-03-21T11:46:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.027+0000 7fb59a016740 1 -- 172.21.15.47:0/932846374 shutdown_connections 2024-03-21T11:46:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.027+0000 7fb59a016740 1 --2- 172.21.15.47:0/932846374 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563e21a77080 0x563e21a794a0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.027+0000 7fb59a016740 1 --2- 172.21.15.47:0/932846374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e21a14dd0 0x563e21a15250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.027+0000 7fb59a016740 1 --2- 172.21.15.47:0/932846374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e219f8d40 0x563e21a13dc0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.027+0000 7fb59a016740 1 --2- 172.21.15.47:0/932846374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e219f31e0 0x563e21a13880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.027+0000 7fb59a016740 1 -- 172.21.15.47:0/932846374 >> 172.21.15.47:0/932846374 conn(0x563e219ee750 msgr2=0x563e219f19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.027+0000 7fb57a000640 0 client.32714 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.027+0000 7fb59a016740 1 -- 172.21.15.47:0/932846374 shutdown_connections 2024-03-21T11:46:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.028+0000 7fb59a016740 1 -- 172.21.15.47:0/932846374 wait complete. 2024-03-21T11:46:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.028+0000 7fb59a016740 20 client.32714 trim_cache size 0 max 16384 2024-03-21T11:46:03.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:46:03.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:46:03.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:46:03.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:46:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.030+0000 7fb597200640 1 -- 172.21.15.47:0/3796575904 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5740a30c0 msgr2=0x7fb5740a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.030+0000 7fb597200640 1 --2- 172.21.15.47:0/3796575904 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5740a30c0 0x7fb5740a5580 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb58809c190 tx=0x7fb5880d0110 comp rx=0 tx=0).stop 2024-03-21T11:46:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.030+0000 7fb597200640 1 -- 172.21.15.47:0/3796575904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5901098b0 msgr2=0x7fb5901a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.030+0000 7fb597200640 1 --2- 172.21.15.47:0/3796575904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5901098b0 0x7fb5901a5490 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fb58800cab0 tx=0x7fb58800cf70 comp rx=0 tx=0).stop 2024-03-21T11:46:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.031+0000 7fb597200640 1 -- 172.21.15.47:0/3796575904 shutdown_connections 2024-03-21T11:46:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.031+0000 7fb597200640 1 --2- 172.21.15.47:0/3796575904 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb5740a30c0 0x7fb5740a5580 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.031+0000 7fb597200640 1 --2- 172.21.15.47:0/3796575904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5901098b0 0x7fb5901a5490 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.031+0000 7fb597200640 1 --2- 172.21.15.47:0/3796575904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb590108ef0 0x7fb59019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.031+0000 7fb597200640 1 --2- 172.21.15.47:0/3796575904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb590107d00 0x7fb59019dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.031+0000 7fb597200640 1 -- 172.21.15.47:0/3796575904 >> 172.21.15.47:0/3796575904 conn(0x7fb590076cb0 msgr2=0x7fb59010c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.031+0000 7fb597200640 1 -- 172.21.15.47:0/3796575904 shutdown_connections 2024-03-21T11:46:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:03.031+0000 7fb597200640 1 -- 172.21.15.47:0/3796575904 wait complete. 2024-03-21T11:46:03.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:03 smithi102 ceph-mon[36748]: pgmap v1082: 129 pgs: 129 active+clean; 660 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 10 MiB/s wr, 698 op/s 2024-03-21T11:46:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:03 smithi047 ceph-mon[31351]: pgmap v1082: 129 pgs: 129 active+clean; 660 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 10 MiB/s wr, 698 op/s 2024-03-21T11:46:03.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:03 smithi100 ceph-mon[36245]: pgmap v1082: 129 pgs: 129 active+clean; 660 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 10 MiB/s wr, 698 op/s 2024-03-21T11:46:03.984 INFO:tasks.workunit.client.0.smithi047.stdout: 10 224183 21.96 MB/sec execute 407 sec latency 112.121 ms 2024-03-21T11:46:04.984 INFO:tasks.workunit.client.0.smithi047.stdout: 10 224652 21.97 MB/sec execute 408 sec latency 128.685 ms 2024-03-21T11:46:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:05 smithi102 ceph-mon[36748]: pgmap v1083: 129 pgs: 129 active+clean; 659 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.5 MiB/s wr, 590 op/s 2024-03-21T11:46:05.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:05 smithi047 ceph-mon[31351]: pgmap v1083: 129 pgs: 129 active+clean; 659 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.5 MiB/s wr, 590 op/s 2024-03-21T11:46:05.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:05 smithi100 ceph-mon[36245]: pgmap v1083: 129 pgs: 129 active+clean; 659 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.5 MiB/s wr, 590 op/s 2024-03-21T11:46:05.984 INFO:tasks.workunit.client.0.smithi047.stdout: 10 225009 21.96 MB/sec execute 409 sec latency 105.227 ms 2024-03-21T11:46:06.984 INFO:tasks.workunit.client.0.smithi047.stdout: 10 225458 21.97 MB/sec execute 410 sec latency 78.358 ms 2024-03-21T11:46:07.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:07 smithi102 ceph-mon[36748]: pgmap v1084: 129 pgs: 129 active+clean; 668 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.4 MiB/s wr, 697 op/s 2024-03-21T11:46:07.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:07 smithi047 ceph-mon[31351]: pgmap v1084: 129 pgs: 129 active+clean; 668 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.4 MiB/s wr, 697 op/s 2024-03-21T11:46:07.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:07 smithi100 ceph-mon[36245]: pgmap v1084: 129 pgs: 129 active+clean; 668 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.4 MiB/s wr, 697 op/s 2024-03-21T11:46:07.985 INFO:tasks.workunit.client.0.smithi047.stdout: 10 225787 21.97 MB/sec execute 411 sec latency 274.984 ms 2024-03-21T11:46:08.984 INFO:tasks.workunit.client.0.smithi047.stdout: 10 226245 21.95 MB/sec execute 412 sec latency 73.994 ms 2024-03-21T11:46:09.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:09 smithi102 ceph-mon[36748]: pgmap v1085: 129 pgs: 129 active+clean; 664 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.1 MiB/s wr, 557 op/s 2024-03-21T11:46:09.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:09 smithi047 ceph-mon[31351]: pgmap v1085: 129 pgs: 129 active+clean; 664 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.1 MiB/s wr, 557 op/s 2024-03-21T11:46:09.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:09 smithi100 ceph-mon[36245]: pgmap v1085: 129 pgs: 129 active+clean; 664 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.1 MiB/s wr, 557 op/s 2024-03-21T11:46:09.985 INFO:tasks.workunit.client.0.smithi047.stdout: 10 226688 21.96 MB/sec execute 413 sec latency 98.827 ms 2024-03-21T11:46:10.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:46:10.146 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:46:10.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.478+0000 7f0107200640 1 -- 172.21.15.47:0/4186507516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01000740c0 msgr2=0x7f0100074540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:10.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.478+0000 7f0107200640 1 --2- 172.21.15.47:0/4186507516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01000740c0 0x7f0100074540 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009900 tx=0x7f00ec02bc40 comp rx=0 tx=0).stop 2024-03-21T11:46:10.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.478+0000 7f0107200640 1 -- 172.21.15.47:0/4186507516 shutdown_connections 2024-03-21T11:46:10.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.478+0000 7f0107200640 1 --2- 172.21.15.47:0/4186507516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0100109560 0x7f010010b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.478+0000 7f0107200640 1 --2- 172.21.15.47:0/4186507516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01000740c0 0x7f0100074540 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.478+0000 7f0107200640 1 --2- 172.21.15.47:0/4186507516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01000756a0 0x7f0100073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.478+0000 7f0107200640 1 -- 172.21.15.47:0/4186507516 >> 172.21.15.47:0/4186507516 conn(0x7f01000fcc20 msgr2=0x7f01000ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:10.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.479+0000 7f0107200640 1 -- 172.21.15.47:0/4186507516 shutdown_connections 2024-03-21T11:46:10.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.479+0000 7f0107200640 1 -- 172.21.15.47:0/4186507516 wait complete. 2024-03-21T11:46:10.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.480+0000 7f0107200640 1 Processor -- start 2024-03-21T11:46:10.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.480+0000 7f0107200640 1 -- start start 2024-03-21T11:46:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f0107200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01000740c0 0x7f0100199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f0107200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01000756a0 0x7f010019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f0107200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0100109560 0x7f01001a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f0107200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010010f4f0 con 0x7f01000740c0 2024-03-21T11:46:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f0107200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010010f370 con 0x7f01000756a0 2024-03-21T11:46:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f0107200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010010f670 con 0x7f0100109560 2024-03-21T11:46:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f0105200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0100109560 0x7f01001a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f0105200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0100109560 0x7f01001a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60348/0 (socket says 172.21.15.47:60348) 2024-03-21T11:46:10.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f0105200640 1 -- 172.21.15.47:0/950186221 learned_addr learned my addr 172.21.15.47:0/950186221 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:10.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f00ff400640 1 --2- 172.21.15.47:0/950186221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01000756a0 0x7f010019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f0105200640 1 -- 172.21.15.47:0/950186221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01000756a0 msgr2=0x7f010019a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:10.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f0105200640 1 --2- 172.21.15.47:0/950186221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01000756a0 0x7f010019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f0105200640 1 -- 172.21.15.47:0/950186221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01000740c0 msgr2=0x7f0100199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:10.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f0105200640 1 --2- 172.21.15.47:0/950186221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01000740c0 0x7f0100199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.481+0000 7f0105200640 1 -- 172.21.15.47:0/950186221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0100105220 con 0x7f0100109560 2024-03-21T11:46:10.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.482+0000 7f0105200640 1 --2- 172.21.15.47:0/950186221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0100109560 0x7f01001a5910 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f00f800eaa0 tx=0x7f00f800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:10.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.482+0000 7f00fcc00640 1 -- 172.21.15.47:0/950186221 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800cdb0 con 0x7f0100109560 2024-03-21T11:46:10.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.482+0000 7f0107200640 1 -- 172.21.15.47:0/950186221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01001a5eb0 con 0x7f0100109560 2024-03-21T11:46:10.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.482+0000 7f00fcc00640 1 -- 172.21.15.47:0/950186221 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00f8005020 con 0x7f0100109560 2024-03-21T11:46:10.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.482+0000 7f00fcc00640 1 -- 172.21.15.47:0/950186221 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f80116d0 con 0x7f0100109560 2024-03-21T11:46:10.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.482+0000 7f0107200640 1 -- 172.21.15.47:0/950186221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01001a6370 con 0x7f0100109560 2024-03-21T11:46:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.487+0000 7f0109e70740 1 Processor -- start 2024-03-21T11:46:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.488+0000 7f00fcc00640 1 -- 172.21.15.47:0/950186221 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f00f80054d0 con 0x7f0100109560 2024-03-21T11:46:10.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.488+0000 7f0109e70740 1 -- start start 2024-03-21T11:46:10.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.488+0000 7f0109e70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e7e80 0x56228c8e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.488+0000 7f0109e70740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e8870 0x56228c8ed370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.488+0000 7f0109e70740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 0x56228c8efe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.488+0000 7f0109e70740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228c84d250 con 0x56228c8ed9e0 2024-03-21T11:46:10.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.488+0000 7f0109e70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228c7f5a30 con 0x56228c8e7e80 2024-03-21T11:46:10.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.488+0000 7f0109e70740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228c8492f0 con 0x56228c8e8870 2024-03-21T11:46:10.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.489+0000 7f00ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e7e80 0x56228c8e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.489+0000 7f00ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e7e80 0x56228c8e82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41442/0 (socket says 172.21.15.47:41442) 2024-03-21T11:46:10.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.489+0000 7f00ff400640 1 -- 172.21.15.47:0/2699422988 learned_addr learned my addr 172.21.15.47:0/2699422988 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:10.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.489+0000 7f00ff400640 1 --2- 172.21.15.47:0/2699422988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 0x56228c8efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.489+0000 7f00ffe00640 1 --2- 172.21.15.47:0/2699422988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e8870 0x56228c8ed370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.491+0000 7f00ffe00640 1 -- 172.21.15.47:0/2699422988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e7e80 msgr2=0x56228c8e82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:10.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.491+0000 7f00ffe00640 1 --2- 172.21.15.47:0/2699422988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e7e80 0x56228c8e82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.491+0000 7f00ffe00640 1 -- 172.21.15.47:0/2699422988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 msgr2=0x56228c8efe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:10.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.491+0000 7f00ffe00640 1 --2- 172.21.15.47:0/2699422988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 0x56228c8efe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.491+0000 7f00ffe00640 1 -- 172.21.15.47:0/2699422988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56228c7e0690 con 0x56228c8e8870 2024-03-21T11:46:10.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.492+0000 7f00ffe00640 1 --2- 172.21.15.47:0/2699422988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e8870 0x56228c8ed370 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009b10 tx=0x7f00f000e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5dea07a1e7cb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:10.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.492+0000 7f00fcc00640 1 --2- 172.21.15.47:0/950186221 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f00e40a2fa0 0x7f00e40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.492+0000 7f00fcc00640 1 -- 172.21.15.47:0/950186221 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f00f80cf6d0 con 0x7f0100109560 2024-03-21T11:46:10.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.493+0000 7f00f5600640 1 -- 172.21.15.47:0/2699422988 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f000ed50 con 0x56228c8e8870 2024-03-21T11:46:10.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.493+0000 7f00f5600640 1 -- 172.21.15.47:0/2699422988 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00f0005020 con 0x56228c8e8870 2024-03-21T11:46:10.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.493+0000 7f00f5600640 1 -- 172.21.15.47:0/2699422988 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0004b30 con 0x56228c8e8870 2024-03-21T11:46:10.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.494+0000 7f0109e70740 1 -- 172.21.15.47:0/2699422988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e8870 msgr2=0x56228c8ed370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:10.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.494+0000 7f0109e70740 1 --2- 172.21.15.47:0/2699422988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e8870 0x56228c8ed370 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009b10 tx=0x7f00f000e350 comp rx=0 tx=0).stop 2024-03-21T11:46:10.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.495+0000 7f0109e70740 1 -- 172.21.15.47:0/2699422988 shutdown_connections 2024-03-21T11:46:10.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.495+0000 7f0109e70740 1 --2- 172.21.15.47:0/2699422988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 0x56228c8efe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.495+0000 7f0109e70740 1 --2- 172.21.15.47:0/2699422988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e8870 0x56228c8ed370 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.495+0000 7f0109e70740 1 --2- 172.21.15.47:0/2699422988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e7e80 0x56228c8e82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.495+0000 7f0109e70740 1 -- 172.21.15.47:0/2699422988 >> 172.21.15.47:0/2699422988 conn(0x56228c8b0200 msgr2=0x56228c8e37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:10.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.495+0000 7f00ff400640 1 --2- 172.21.15.47:0/950186221 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f00e40a2fa0 0x7f00e40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.495+0000 7f0109e70740 1 -- 172.21.15.47:0/2699422988 shutdown_connections 2024-03-21T11:46:10.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.495+0000 7f0109e70740 1 -- 172.21.15.47:0/2699422988 wait complete. 2024-03-21T11:46:10.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.496+0000 7f0109e70740 1 -- start start 2024-03-21T11:46:10.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.497+0000 7f0109e70740 1 Processor -- start 2024-03-21T11:46:10.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.497+0000 7f00ff400640 1 --2- 172.21.15.47:0/950186221 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f00e40a2fa0 0x7f00e40a5460 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f00ec005ec0 tx=0x7f00ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:10.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.498+0000 7f0109e70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e7e80 0x56228c8f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.498+0000 7f0109e70740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e8870 0x56228c8f2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.498+0000 7f0109e70740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 0x56228c8f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.498+0000 7f0109e70740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228c8f0990 con 0x56228c8ed9e0 2024-03-21T11:46:10.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.498+0000 7f0109e70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228c808e80 con 0x56228c8e7e80 2024-03-21T11:46:10.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.498+0000 7f0109e70740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228c8f0b10 con 0x56228c8e8870 2024-03-21T11:46:10.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.498+0000 7f0109e70740 10 client.0 _ll_register_callbacks cb 0x56228c81b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:10.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.498+0000 7f00ff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e8870 0x56228c8f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.498+0000 7f00ff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e8870 0x56228c8f2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60358/0 (socket says 172.21.15.47:60358) 2024-03-21T11:46:10.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.498+0000 7f00ff400640 1 -- 172.21.15.47:0/2216989301 learned_addr learned my addr 172.21.15.47:0/2216989301 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:10.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.499+0000 7f0105200640 1 --2- 172.21.15.47:0/2216989301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 0x56228c8f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.499+0000 7f00ffe00640 1 --2- 172.21.15.47:0/2216989301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e7e80 0x56228c8f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.500+0000 7f00ff400640 1 -- 172.21.15.47:0/2216989301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e7e80 msgr2=0x56228c8f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:10.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.500+0000 7f00ff400640 1 --2- 172.21.15.47:0/2216989301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e7e80 0x56228c8f1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.500+0000 7f00ff400640 1 -- 172.21.15.47:0/2216989301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 msgr2=0x56228c8f93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:10.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.500+0000 7f00ff400640 1 --2- 172.21.15.47:0/2216989301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 0x56228c8f93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.500+0000 7f00ff400640 1 -- 172.21.15.47:0/2216989301 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56228c62b7d0 con 0x56228c8e8870 2024-03-21T11:46:10.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.501+0000 7f00ff400640 1 --2- 172.21.15.47:0/2216989301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e8870 0x56228c8f2360 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009900 tx=0x7f00ec02bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=858e2fa3dd7cc44c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:10.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.501+0000 7f00ea000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:10.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.501+0000 7f00ea000640 1 -- 172.21.15.47:0/2216989301 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec052020 con 0x56228c8e8870 2024-03-21T11:46:10.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.501+0000 7f00ea000640 1 -- 172.21.15.47:0/2216989301 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f00ec003cd0 con 0x56228c8e8870 2024-03-21T11:46:10.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.501+0000 7f0109e70740 1 -- 172.21.15.47:0/2216989301 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00f0005020 con 0x56228c8e8870 2024-03-21T11:46:10.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.503+0000 7f00ea000640 1 -- 172.21.15.47:0/2216989301 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00ec043450 con 0x56228c8e8870 2024-03-21T11:46:10.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.503+0000 7f0109e70740 10 client.32701 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:10.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.503+0000 7f0109e70740 1 -- 172.21.15.47:0/2216989301 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56228c5613f0 con 0x56228c8e8870 2024-03-21T11:46:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.503+0000 7f00ea000640 1 -- 172.21.15.47:0/2216989301 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00f0005020 con 0x56228c8e8870 2024-03-21T11:46:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.504+0000 7f00ea000640 1 -- 172.21.15.47:0/2216989301 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f00ec0345d0 con 0x56228c8e8870 2024-03-21T11:46:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.504+0000 7f0109e70740 10 client.32701 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.504+0000 7f0109e70740 10 client.32701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.504+0000 7f0109e70740 20 client.32701 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:10.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.504+0000 7f0109e70740 1 --2- 172.21.15.47:0/2216989301 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228c90ec70 0x56228c911110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.505+0000 7f0109e70740 4 client.32701 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:10.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.505+0000 7f0109e70740 1 -- 172.21.15.47:0/2216989301 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56228c85a790 con 0x56228c90ec70 2024-03-21T11:46:10.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.505+0000 7f00ffe00640 1 --2- 172.21.15.47:0/2216989301 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228c90ec70 0x56228c911110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.505+0000 7f00ffe00640 1 --2- 172.21.15.47:0/2216989301 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228c90ec70 0x56228c911110 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6018edac88d5a42e server_cookie=a9f66176b51a8b48 in_seq=0 out_seq=0 2024-03-21T11:46:10.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.505+0000 7f00ea000640 10 client.32701 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.507+0000 7f00ea000640 1 -- 172.21.15.47:0/2216989301 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56228c85a790 con 0x56228c90ec70 2024-03-21T11:46:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.508+0000 7f00ea000640 10 client.32701 handle_command_reply: tid=0 2024-03-21T11:46:10.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.508+0000 7f0109e70740 1 client.32701 shutdown 2024-03-21T11:46:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.509+0000 7f0109e70740 1 -- 172.21.15.47:0/2216989301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e8870 msgr2=0x56228c8f2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.509+0000 7f0109e70740 1 --2- 172.21.15.47:0/2216989301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e8870 0x56228c8f2360 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009900 tx=0x7f00ec02bc40 comp rx=0 tx=0).stop 2024-03-21T11:46:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.509+0000 7f0109e70740 1 -- 172.21.15.47:0/2216989301 shutdown_connections 2024-03-21T11:46:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.509+0000 7f0109e70740 1 --2- 172.21.15.47:0/2216989301 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228c90ec70 0x56228c911110 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.509+0000 7f0109e70740 1 --2- 172.21.15.47:0/2216989301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 0x56228c8f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.509+0000 7f0109e70740 1 --2- 172.21.15.47:0/2216989301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e8870 0x56228c8f2360 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.509+0000 7f0109e70740 1 --2- 172.21.15.47:0/2216989301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e7e80 0x56228c8f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.509+0000 7f0109e70740 1 -- 172.21.15.47:0/2216989301 >> 172.21.15.47:0/2216989301 conn(0x56228c8b0200 msgr2=0x56228c8e2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.509+0000 7f00ea000640 0 client.32701 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.509+0000 7f0109e70740 1 -- 172.21.15.47:0/2216989301 shutdown_connections 2024-03-21T11:46:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.509+0000 7f0109e70740 1 -- 172.21.15.47:0/2216989301 wait complete. 2024-03-21T11:46:10.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.509+0000 7f0109e70740 20 client.32701 trim_cache size 0 max 16384 2024-03-21T11:46:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.530+0000 7f0109e70740 1 Processor -- start 2024-03-21T11:46:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.530+0000 7f0109e70740 1 -- start start 2024-03-21T11:46:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.531+0000 7f0109e70740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e7e80 0x56228c8e3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.531+0000 7f0109e70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8ed9e0 0x56228c8f9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.531+0000 7f0109e70740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8e2200 0x56228c8e2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.531+0000 7f0109e70740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001a8040 con 0x56228c8e2200 2024-03-21T11:46:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.531+0000 7f0109e70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001a99c0 con 0x56228c8ed9e0 2024-03-21T11:46:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.531+0000 7f0109e70740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001a86a0 con 0x56228c8e7e80 2024-03-21T11:46:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.531+0000 7f00ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8ed9e0 0x56228c8f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.531+0000 7f00ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8ed9e0 0x56228c8f9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41458/0 (socket says 172.21.15.47:41458) 2024-03-21T11:46:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.531+0000 7f00ff400640 1 -- 172.21.15.47:0/3904620351 learned_addr learned my addr 172.21.15.47:0/3904620351 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.531+0000 7f0105200640 1 --2- 172.21.15.47:0/3904620351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8e2200 0x56228c8e2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.532+0000 7f00ff400640 1 -- 172.21.15.47:0/3904620351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e7e80 msgr2=0x56228c8e3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.532+0000 7f00ff400640 1 --2- 172.21.15.47:0/3904620351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e7e80 0x56228c8e3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.532+0000 7f00ff400640 1 -- 172.21.15.47:0/3904620351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8e2200 msgr2=0x56228c8e2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.532+0000 7f00ff400640 1 --2- 172.21.15.47:0/3904620351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8e2200 0x56228c8e2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.532+0000 7f00ff400640 1 -- 172.21.15.47:0/3904620351 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56228c5210f0 con 0x56228c8ed9e0 2024-03-21T11:46:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.532+0000 7f00ff400640 1 --2- 172.21.15.47:0/3904620351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8ed9e0 0x56228c8f9360 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f00ec034a60 tx=0x7f00ec0039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7cda97e9f3227ea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.532+0000 7f00f4c00640 1 -- 172.21.15.47:0/3904620351 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec052040 con 0x56228c8ed9e0 2024-03-21T11:46:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.532+0000 7f00f4c00640 1 -- 172.21.15.47:0/3904620351 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00ec02cd00 con 0x56228c8ed9e0 2024-03-21T11:46:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.532+0000 7f00f4c00640 1 -- 172.21.15.47:0/3904620351 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0435f0 con 0x56228c8ed9e0 2024-03-21T11:46:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.533+0000 7f0109e70740 1 -- 172.21.15.47:0/3904620351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8ed9e0 msgr2=0x56228c8f9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.533+0000 7f0109e70740 1 --2- 172.21.15.47:0/3904620351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8ed9e0 0x56228c8f9360 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f00ec034a60 tx=0x7f00ec0039e0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.533+0000 7f0109e70740 1 -- 172.21.15.47:0/3904620351 shutdown_connections 2024-03-21T11:46:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.533+0000 7f0109e70740 1 --2- 172.21.15.47:0/3904620351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8e2200 0x56228c8e2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.533+0000 7f0109e70740 1 --2- 172.21.15.47:0/3904620351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8ed9e0 0x56228c8f9360 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.533+0000 7f0109e70740 1 --2- 172.21.15.47:0/3904620351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e7e80 0x56228c8e3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.533+0000 7f0109e70740 1 -- 172.21.15.47:0/3904620351 >> 172.21.15.47:0/3904620351 conn(0x56228c8b0200 msgr2=0x56228c8e3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.533+0000 7f0109e70740 1 -- 172.21.15.47:0/3904620351 shutdown_connections 2024-03-21T11:46:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.534+0000 7f0109e70740 1 -- 172.21.15.47:0/3904620351 wait complete. 2024-03-21T11:46:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.534+0000 7f0109e70740 1 -- start start 2024-03-21T11:46:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.534+0000 7f0109e70740 1 Processor -- start 2024-03-21T11:46:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f0109e70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e2200 0x56228c8f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f0109e70740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e7e80 0x56228c8f9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f0109e70740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 0x56228c8f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f0109e70740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001a9360 con 0x56228c8ed9e0 2024-03-21T11:46:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f0109e70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001a8d00 con 0x56228c8e2200 2024-03-21T11:46:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f0109e70740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000734a0 con 0x56228c8e7e80 2024-03-21T11:46:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f0109e70740 10 client.0 _ll_register_callbacks cb 0x7f00ec02cc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f00ff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e7e80 0x56228c8f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f0105200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 0x56228c8f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f00ff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e7e80 0x56228c8f9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60378/0 (socket says 172.21.15.47:60378) 2024-03-21T11:46:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f00ff400640 1 -- 172.21.15.47:0/2614437822 learned_addr learned my addr 172.21.15.47:0/2614437822 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f00ffe00640 1 --2- 172.21.15.47:0/2614437822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e2200 0x56228c8f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f00ff400640 1 -- 172.21.15.47:0/2614437822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e2200 msgr2=0x56228c8f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f00ff400640 1 --2- 172.21.15.47:0/2614437822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e2200 0x56228c8f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f00ff400640 1 -- 172.21.15.47:0/2614437822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 msgr2=0x56228c8f74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.535+0000 7f00ff400640 1 --2- 172.21.15.47:0/2614437822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 0x56228c8f74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.536+0000 7f00ff400640 1 -- 172.21.15.47:0/2614437822 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56228c8f7ce0 con 0x56228c8e7e80 2024-03-21T11:46:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.536+0000 7f0105200640 1 --2- 172.21.15.47:0/2614437822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 0x56228c8f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.536+0000 7f00ff400640 1 --2- 172.21.15.47:0/2614437822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e7e80 0x56228c8f9330 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f00ec034b00 tx=0x7f00ec003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c55bf54a1b1d2052 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.536+0000 7f00ea000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.539+0000 7f00ea000640 1 -- 172.21.15.47:0/2614437822 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec064070 con 0x56228c8e7e80 2024-03-21T11:46:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.539+0000 7f00ea000640 1 -- 172.21.15.47:0/2614437822 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f00ec0435f0 con 0x56228c8e7e80 2024-03-21T11:46:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.539+0000 7f0109e70740 1 -- 172.21.15.47:0/2614437822 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00ec02cd00 con 0x56228c8e7e80 2024-03-21T11:46:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.540+0000 7f00ea000640 1 -- 172.21.15.47:0/2614437822 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00ec070050 con 0x56228c8e7e80 2024-03-21T11:46:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.540+0000 7f0109e70740 10 client.32704 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.540+0000 7f0109e70740 1 -- 172.21.15.47:0/2614437822 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56228c96b130 con 0x56228c8e7e80 2024-03-21T11:46:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.542+0000 7f00ea000640 1 -- 172.21.15.47:0/2614437822 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00ec02cd00 con 0x56228c8e7e80 2024-03-21T11:46:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.542+0000 7f00ea000640 1 -- 172.21.15.47:0/2614437822 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f00ec0608a0 con 0x56228c8e7e80 2024-03-21T11:46:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.542+0000 7f0109e70740 10 client.32704 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.542+0000 7f0109e70740 10 client.32704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.542+0000 7f0109e70740 20 client.32704 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.542+0000 7f0109e70740 1 --2- 172.21.15.47:0/2614437822 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228c96b7f0 0x56228c96dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.542+0000 7f0109e70740 4 client.32704 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:46:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.542+0000 7f0109e70740 1 -- 172.21.15.47:0/2614437822 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56228c854ae0 con 0x56228c96b7f0 2024-03-21T11:46:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.542+0000 7f00ffe00640 1 --2- 172.21.15.47:0/2614437822 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228c96b7f0 0x56228c96dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.543+0000 7f00ffe00640 1 --2- 172.21.15.47:0/2614437822 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228c96b7f0 0x56228c96dc10 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19055021738aa980 server_cookie=7a551bb269c81ed3 in_seq=0 out_seq=0 2024-03-21T11:46:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.543+0000 7f00ea000640 10 client.32704 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.545+0000 7f00ea000640 1 -- 172.21.15.47:0/2614437822 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56228c854ae0 con 0x56228c96b7f0 2024-03-21T11:46:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.545+0000 7f00ea000640 10 client.32704 handle_command_reply: tid=0 2024-03-21T11:46:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.545+0000 7f0109e70740 1 client.32704 shutdown 2024-03-21T11:46:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.545+0000 7f0109e70740 1 -- 172.21.15.47:0/2614437822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e7e80 msgr2=0x56228c8f9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.545+0000 7f0109e70740 1 --2- 172.21.15.47:0/2614437822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e7e80 0x56228c8f9330 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f00ec034b00 tx=0x7f00ec003f80 comp rx=0 tx=0).stop 2024-03-21T11:46:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.545+0000 7f0109e70740 1 -- 172.21.15.47:0/2614437822 shutdown_connections 2024-03-21T11:46:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.545+0000 7f0109e70740 1 --2- 172.21.15.47:0/2614437822 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56228c96b7f0 0x56228c96dc10 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.545+0000 7f0109e70740 1 --2- 172.21.15.47:0/2614437822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56228c8ed9e0 0x56228c8f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.545+0000 7f0109e70740 1 --2- 172.21.15.47:0/2614437822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56228c8e7e80 0x56228c8f9330 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.545+0000 7f0109e70740 1 --2- 172.21.15.47:0/2614437822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56228c8e2200 0x56228c8f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.545+0000 7f00ea000640 0 client.32704 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.545+0000 7f0109e70740 1 -- 172.21.15.47:0/2614437822 >> 172.21.15.47:0/2614437822 conn(0x56228c8b0200 msgr2=0x56228c8e2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.546+0000 7f0109e70740 1 -- 172.21.15.47:0/2614437822 shutdown_connections 2024-03-21T11:46:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.546+0000 7f0109e70740 1 -- 172.21.15.47:0/2614437822 wait complete. 2024-03-21T11:46:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.546+0000 7f0109e70740 20 client.32704 trim_cache size 0 max 16384 2024-03-21T11:46:10.547 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:46:10.547 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:46:10.547 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:46:10.547 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:46:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.546+0000 7f0107200640 1 -- 172.21.15.47:0/950186221 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f00e40a2fa0 msgr2=0x7f00e40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.546+0000 7f0107200640 1 --2- 172.21.15.47:0/950186221 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f00e40a2fa0 0x7f00e40a5460 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f00ec005ec0 tx=0x7f00ec005fd0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.546+0000 7f0107200640 1 -- 172.21.15.47:0/950186221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0100109560 msgr2=0x7f01001a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.546+0000 7f0107200640 1 --2- 172.21.15.47:0/950186221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0100109560 0x7f01001a5910 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f00f800eaa0 tx=0x7f00f800ef60 comp rx=0 tx=0).stop 2024-03-21T11:46:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.547+0000 7f0107200640 1 -- 172.21.15.47:0/950186221 shutdown_connections 2024-03-21T11:46:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.547+0000 7f0107200640 1 --2- 172.21.15.47:0/950186221 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f00e40a2fa0 0x7f00e40a5460 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.547+0000 7f0107200640 1 --2- 172.21.15.47:0/950186221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0100109560 0x7f01001a5910 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.547+0000 7f0107200640 1 --2- 172.21.15.47:0/950186221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01000756a0 0x7f010019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.547+0000 7f0107200640 1 --2- 172.21.15.47:0/950186221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01000740c0 0x7f0100199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.547+0000 7f0107200640 1 -- 172.21.15.47:0/950186221 >> 172.21.15.47:0/950186221 conn(0x7f01000fcc20 msgr2=0x7f01001054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.547+0000 7f0107200640 1 -- 172.21.15.47:0/950186221 shutdown_connections 2024-03-21T11:46:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.547+0000 7f0107200640 1 -- 172.21.15.47:0/950186221 wait complete. 2024-03-21T11:46:10.565 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:46:10.565 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:46:10.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.784+0000 7f340b000640 1 -- 172.21.15.47:0/3635518346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34041026d0 msgr2=0x7f3404104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:10.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.784+0000 7f340b000640 1 --2- 172.21.15.47:0/3635518346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34041026d0 0x7f3404104b40 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009860 tx=0x7f33f002bb90 comp rx=0 tx=0).stop 2024-03-21T11:46:10.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.784+0000 7f340b000640 1 -- 172.21.15.47:0/3635518346 shutdown_connections 2024-03-21T11:46:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.784+0000 7f340b000640 1 --2- 172.21.15.47:0/3635518346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3404105080 0x7f3404105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.784+0000 7f340b000640 1 --2- 172.21.15.47:0/3635518346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34041026d0 0x7f3404104b40 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.784+0000 7f340b000640 1 --2- 172.21.15.47:0/3635518346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34040ffd60 0x7f3404102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.784+0000 7f340b000640 1 -- 172.21.15.47:0/3635518346 >> 172.21.15.47:0/3635518346 conn(0x7f34040fb980 msgr2=0x7f34040fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.784+0000 7f340b000640 1 -- 172.21.15.47:0/3635518346 shutdown_connections 2024-03-21T11:46:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.784+0000 7f340b000640 1 -- 172.21.15.47:0/3635518346 wait complete. 2024-03-21T11:46:10.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.785+0000 7f340b000640 1 Processor -- start 2024-03-21T11:46:10.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.785+0000 7f340b000640 1 -- start start 2024-03-21T11:46:10.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f340b000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34040ffd60 0x7f340419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f340b000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34041026d0 0x7f340419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f340b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3404105080 0x7f34041a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f340b000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3404113740 con 0x7f34041026d0 2024-03-21T11:46:10.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f340b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34041135c0 con 0x7f3404105080 2024-03-21T11:46:10.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f340b000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34041138c0 con 0x7f34040ffd60 2024-03-21T11:46:10.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f3409200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34041026d0 0x7f340419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f3409c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34040ffd60 0x7f340419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f3409200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34041026d0 0x7f340419e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35908/0 (socket says 172.21.15.47:35908) 2024-03-21T11:46:10.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f3409200640 1 -- 172.21.15.47:0/328270803 learned_addr learned my addr 172.21.15.47:0/328270803 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:10.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f3409200640 1 -- 172.21.15.47:0/328270803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34040ffd60 msgr2=0x7f340419de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:10.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f3409200640 1 --2- 172.21.15.47:0/328270803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34040ffd60 0x7f340419de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f3409200640 1 -- 172.21.15.47:0/328270803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3404105080 msgr2=0x7f34041a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:10.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f3409200640 1 --2- 172.21.15.47:0/328270803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3404105080 0x7f34041a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:10.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f3409200640 1 -- 172.21.15.47:0/328270803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34041a59e0 con 0x7f34041026d0 2024-03-21T11:46:10.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.786+0000 7f3409c00640 1 --2- 172.21.15.47:0/328270803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34040ffd60 0x7f340419de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:10.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.787+0000 7f3409200640 1 --2- 172.21.15.47:0/328270803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34041026d0 0x7f340419e350 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f33f00349e0 tx=0x7f33f0034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:10.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.787+0000 7f3402000640 1 -- 172.21.15.47:0/328270803 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0041020 con 0x7f34041026d0 2024-03-21T11:46:10.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.787+0000 7f340b000640 1 -- 172.21.15.47:0/328270803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34041a5c70 con 0x7f34041026d0 2024-03-21T11:46:10.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.787+0000 7f3402000640 1 -- 172.21.15.47:0/328270803 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33f002ca10 con 0x7f34041026d0 2024-03-21T11:46:10.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.787+0000 7f3402000640 1 -- 172.21.15.47:0/328270803 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0039040 con 0x7f34041026d0 2024-03-21T11:46:10.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.787+0000 7f340b000640 1 -- 172.21.15.47:0/328270803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34041a61b0 con 0x7f34041026d0 2024-03-21T11:46:10.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.789+0000 7f340b000640 1 -- 172.21.15.47:0/328270803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34040fd470 con 0x7f34041026d0 2024-03-21T11:46:10.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.790+0000 7f3402000640 1 -- 172.21.15.47:0/328270803 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f33f0044020 con 0x7f34041026d0 2024-03-21T11:46:10.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.792+0000 7f3402000640 1 --2- 172.21.15.47:0/328270803 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33ec0a2fa0 0x7f33ec0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:10.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.792+0000 7f3402000640 1 -- 172.21.15.47:0/328270803 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f33f00f1260 con 0x7f34041026d0 2024-03-21T11:46:10.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.792+0000 7f3409c00640 1 --2- 172.21.15.47:0/328270803 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33ec0a2fa0 0x7f33ec0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:10.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.793+0000 7f3409c00640 1 --2- 172.21.15.47:0/328270803 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33ec0a2fa0 0x7f33ec0a5460 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009c20 tx=0x7f33f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:10.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:10.797+0000 7f3402000640 1 -- 172.21.15.47:0/328270803 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f33f004b020 con 0x7f34041026d0 2024-03-21T11:46:10.985 INFO:tasks.workunit.client.0.smithi047.stdout: 10 227225 21.98 MB/sec execute 414 sec latency 75.734 ms 2024-03-21T11:46:11.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.086+0000 7f340b000640 1 -- 172.21.15.47:0/328270803 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3404062340 con 0x7f34041026d0 2024-03-21T11:46:11.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.088+0000 7f3402000640 1 -- 172.21.15.47:0/328270803 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f33f00bdca0 con 0x7f34041026d0 2024-03-21T11:46:11.088 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:46:11.088 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:45:39.125075+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_32623":{"gid":32623,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4278877119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4278877119},{"type":"v1","addr":"172.21.15.100:6835","nonce":4278877119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24523,24524,24535,24553,24581]},"id":1}]} 2024-03-21T11:46:11.088 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 41 2024-03-21T11:46:11.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.090+0000 7f340b000640 1 -- 172.21.15.47:0/328270803 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33ec0a2fa0 msgr2=0x7f33ec0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.090+0000 7f340b000640 1 --2- 172.21.15.47:0/328270803 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33ec0a2fa0 0x7f33ec0a5460 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009c20 tx=0x7f33f8005f90 comp rx=0 tx=0).stop 2024-03-21T11:46:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.090+0000 7f340b000640 1 -- 172.21.15.47:0/328270803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34041026d0 msgr2=0x7f340419e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.090+0000 7f340b000640 1 --2- 172.21.15.47:0/328270803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34041026d0 0x7f340419e350 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f33f00349e0 tx=0x7f33f0034760 comp rx=0 tx=0).stop 2024-03-21T11:46:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.090+0000 7f340b000640 1 -- 172.21.15.47:0/328270803 shutdown_connections 2024-03-21T11:46:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.090+0000 7f340b000640 1 --2- 172.21.15.47:0/328270803 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f33ec0a2fa0 0x7f33ec0a5460 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.091+0000 7f340b000640 1 --2- 172.21.15.47:0/328270803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3404105080 0x7f34041a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.091+0000 7f340b000640 1 --2- 172.21.15.47:0/328270803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34041026d0 0x7f340419e350 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.091+0000 7f340b000640 1 --2- 172.21.15.47:0/328270803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34040ffd60 0x7f340419de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.091+0000 7f340b000640 1 -- 172.21.15.47:0/328270803 >> 172.21.15.47:0/328270803 conn(0x7f34040fb980 msgr2=0x7f3404100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.091+0000 7f340b000640 1 -- 172.21.15.47:0/328270803 shutdown_connections 2024-03-21T11:46:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.091+0000 7f340b000640 1 -- 172.21.15.47:0/328270803 wait complete. 2024-03-21T11:46:11.111 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:46:11.111 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:46:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.346+0000 7ff6c6400640 1 -- 172.21.15.47:0/1572719343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c0108eb0 msgr2=0x7ff6c0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.346+0000 7ff6c6400640 1 --2- 172.21.15.47:0/1572719343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c0108eb0 0x7ff6c0109330 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009900 tx=0x7ff6b002bc40 comp rx=0 tx=0).stop 2024-03-21T11:46:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.348+0000 7ff6c6400640 1 -- 172.21.15.47:0/1572719343 shutdown_connections 2024-03-21T11:46:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.348+0000 7ff6c6400640 1 --2- 172.21.15.47:0/1572719343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c0109870 0x7ff6c0110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.348+0000 7ff6c6400640 1 --2- 172.21.15.47:0/1572719343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c0108eb0 0x7ff6c0109330 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.348+0000 7ff6c6400640 1 --2- 172.21.15.47:0/1572719343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6c0107cc0 0x7ff6c01080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.348+0000 7ff6c6400640 1 -- 172.21.15.47:0/1572719343 >> 172.21.15.47:0/1572719343 conn(0x7ff6c0076c60 msgr2=0x7ff6c00790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.348+0000 7ff6c6400640 1 -- 172.21.15.47:0/1572719343 shutdown_connections 2024-03-21T11:46:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.348+0000 7ff6c6400640 1 -- 172.21.15.47:0/1572719343 wait complete. 2024-03-21T11:46:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.349+0000 7ff6c6400640 1 Processor -- start 2024-03-21T11:46:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.350+0000 7ff6c6400640 1 -- start start 2024-03-21T11:46:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.350+0000 7ff6c6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c0107cc0 0x7ff6c007ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.351+0000 7ff6bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c0107cc0 0x7ff6c007ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.351+0000 7ff6bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c0107cc0 0x7ff6c007ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41508/0 (socket says 172.21.15.47:41508) 2024-03-21T11:46:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.351+0000 7ff6c6400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6c0108eb0 0x7ff6c00804d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.351+0000 7ff6c6400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c0109870 0x7ff6c0081510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.351+0000 7ff6c6400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c007a650 con 0x7ff6c0108eb0 2024-03-21T11:46:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.351+0000 7ff6c6400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c007a4d0 con 0x7ff6c0107cc0 2024-03-21T11:46:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.351+0000 7ff6c6400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c007a7d0 con 0x7ff6c0109870 2024-03-21T11:46:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.351+0000 7ff6bf400640 1 -- 172.21.15.47:0/765028260 learned_addr learned my addr 172.21.15.47:0/765028260 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.352+0000 7ff6bfe00640 1 --2- 172.21.15.47:0/765028260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c0109870 0x7ff6c0081510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.352+0000 7ff6bea00640 1 --2- 172.21.15.47:0/765028260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6c0108eb0 0x7ff6c00804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.352+0000 7ff6bf400640 1 -- 172.21.15.47:0/765028260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c0109870 msgr2=0x7ff6c0081510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.352+0000 7ff6bf400640 1 --2- 172.21.15.47:0/765028260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c0109870 0x7ff6c0081510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.352+0000 7ff6bf400640 1 -- 172.21.15.47:0/765028260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6c0108eb0 msgr2=0x7ff6c00804d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.352+0000 7ff6bf400640 1 --2- 172.21.15.47:0/765028260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6c0108eb0 0x7ff6c00804d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.352+0000 7ff6bf400640 1 -- 172.21.15.47:0/765028260 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c0081b80 con 0x7ff6c0107cc0 2024-03-21T11:46:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.353+0000 7ff6bf400640 1 --2- 172.21.15.47:0/765028260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c0107cc0 0x7ff6c007ff90 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800e9c0 tx=0x7ff6a800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.353+0000 7ff6b7e00640 1 -- 172.21.15.47:0/765028260 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800ccc0 con 0x7ff6c0107cc0 2024-03-21T11:46:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.353+0000 7ff6b7e00640 1 -- 172.21.15.47:0/765028260 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6a80035f0 con 0x7ff6c0107cc0 2024-03-21T11:46:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.353+0000 7ff6b7e00640 1 -- 172.21.15.47:0/765028260 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800ce60 con 0x7ff6c0107cc0 2024-03-21T11:46:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.353+0000 7ff6c6400640 1 -- 172.21.15.47:0/765028260 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c0081e70 con 0x7ff6c0107cc0 2024-03-21T11:46:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.353+0000 7ff6c6400640 1 -- 172.21.15.47:0/765028260 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c0082370 con 0x7ff6c0107cc0 2024-03-21T11:46:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.358+0000 7ff6b7e00640 1 -- 172.21.15.47:0/765028260 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff6a80037a0 con 0x7ff6c0107cc0 2024-03-21T11:46:11.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.360+0000 7ff6b7e00640 1 --2- 172.21.15.47:0/765028260 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff6a40a2fa0 0x7ff6a40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.360+0000 7ff6b7e00640 1 -- 172.21.15.47:0/765028260 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff6a80cfae0 con 0x7ff6c0107cc0 2024-03-21T11:46:11.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.361+0000 7ff6c9059740 1 Processor -- start 2024-03-21T11:46:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.361+0000 7ff6c9059740 1 -- start start 2024-03-21T11:46:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.361+0000 7ff6bea00640 1 --2- 172.21.15.47:0/765028260 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff6a40a2fa0 0x7ff6a40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.361+0000 7ff6c9059740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa1810 0x555a48aa1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.361+0000 7ff6c9059740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa29e0 0x555a48aa2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.361+0000 7ff6c9059740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa33a0 0x555a48aa9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.361+0000 7ff6c9059740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a48a062c0 con 0x555a48aa29e0 2024-03-21T11:46:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.361+0000 7ff6c9059740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a489aec90 con 0x555a48aa1810 2024-03-21T11:46:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.361+0000 7ff6c9059740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a48a02360 con 0x555a48aa33a0 2024-03-21T11:46:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.362+0000 7ff6bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa1810 0x555a48aa1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.362+0000 7ff6bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa1810 0x555a48aa1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41512/0 (socket says 172.21.15.47:41512) 2024-03-21T11:46:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.362+0000 7ff6bf400640 1 -- 172.21.15.47:0/1068908389 learned_addr learned my addr 172.21.15.47:0/1068908389 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.363+0000 7ff6bfe00640 1 --2- 172.21.15.47:0/1068908389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa33a0 0x555a48aa9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.363+0000 7ff6bea00640 1 --2- 172.21.15.47:0/1068908389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa29e0 0x555a48aa2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.364+0000 7ff6bf400640 1 -- 172.21.15.47:0/1068908389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa33a0 msgr2=0x555a48aa9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.364+0000 7ff6bf400640 1 --2- 172.21.15.47:0/1068908389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa33a0 0x555a48aa9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.364+0000 7ff6bf400640 1 -- 172.21.15.47:0/1068908389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa29e0 msgr2=0x555a48aa2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.364+0000 7ff6bf400640 1 --2- 172.21.15.47:0/1068908389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa29e0 0x555a48aa2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.365+0000 7ff6bf400640 1 -- 172.21.15.47:0/1068908389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a489d3080 con 0x555a48aa1810 2024-03-21T11:46:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.365+0000 7ff6bfe00640 1 --2- 172.21.15.47:0/1068908389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa33a0 0x555a48aa9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.365+0000 7ff6bea00640 1 --2- 172.21.15.47:0/1068908389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa29e0 0x555a48aa2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.365+0000 7ff6bf400640 1 --2- 172.21.15.47:0/1068908389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa1810 0x555a48aa1c30 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8098850 tx=0x7ff6a809b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f24b92da3601cbaf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.366+0000 7ff6b4c00640 1 -- 172.21.15.47:0/1068908389 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a809ffd0 con 0x555a48aa1810 2024-03-21T11:46:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.366+0000 7ff6b4c00640 1 -- 172.21.15.47:0/1068908389 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6a80a4890 con 0x555a48aa1810 2024-03-21T11:46:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.366+0000 7ff6b4c00640 1 -- 172.21.15.47:0/1068908389 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a80a4660 con 0x555a48aa1810 2024-03-21T11:46:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.366+0000 7ff6bea00640 1 --2- 172.21.15.47:0/765028260 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff6a40a2fa0 0x7ff6a40a5460 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0036040 tx=0x7ff6b0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.367+0000 7ff6c9059740 1 -- 172.21.15.47:0/1068908389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa1810 msgr2=0x555a48aa1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.367+0000 7ff6c9059740 1 --2- 172.21.15.47:0/1068908389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa1810 0x555a48aa1c30 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8098850 tx=0x7ff6a809b8a0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.367+0000 7ff6c9059740 1 -- 172.21.15.47:0/1068908389 shutdown_connections 2024-03-21T11:46:11.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.367+0000 7ff6c9059740 1 --2- 172.21.15.47:0/1068908389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa33a0 0x555a48aa9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.367+0000 7ff6c9059740 1 --2- 172.21.15.47:0/1068908389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa29e0 0x555a48aa2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.367+0000 7ff6c9059740 1 --2- 172.21.15.47:0/1068908389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa1810 0x555a48aa1c30 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.367+0000 7ff6c9059740 1 -- 172.21.15.47:0/1068908389 >> 172.21.15.47:0/1068908389 conn(0x555a48a9cc00 msgr2=0x555a48a9d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.367+0000 7ff6c9059740 1 -- 172.21.15.47:0/1068908389 shutdown_connections 2024-03-21T11:46:11.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.367+0000 7ff6c9059740 1 -- 172.21.15.47:0/1068908389 wait complete. 2024-03-21T11:46:11.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.368+0000 7ff6c9059740 1 -- start start 2024-03-21T11:46:11.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.368+0000 7ff6c9059740 1 Processor -- start 2024-03-21T11:46:11.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.369+0000 7ff6c9059740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa1810 0x555a48aabaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.369+0000 7ff6c9059740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa29e0 0x555a48aac030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.369+0000 7ff6c9059740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa33a0 0x555a48ab3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.369+0000 7ff6c9059740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a48aaa3b0 con 0x555a48aa33a0 2024-03-21T11:46:11.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.369+0000 7ff6c9059740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a489c20d0 con 0x555a48aa29e0 2024-03-21T11:46:11.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.369+0000 7ff6c9059740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a48aaa530 con 0x555a48aa1810 2024-03-21T11:46:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.369+0000 7ff6bf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa1810 0x555a48aabaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.369+0000 7ff6c9059740 10 client.0 _ll_register_callbacks cb 0x555a489c6e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.369+0000 7ff6bf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa1810 0x555a48aabaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60440/0 (socket says 172.21.15.47:60440) 2024-03-21T11:46:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.369+0000 7ff6bf400640 1 -- 172.21.15.47:0/238941252 learned_addr learned my addr 172.21.15.47:0/238941252 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.369+0000 7ff6bea00640 1 --2- 172.21.15.47:0/238941252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa29e0 0x555a48aac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.369+0000 7ff6bfe00640 1 --2- 172.21.15.47:0/238941252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa33a0 0x555a48ab3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.370+0000 7ff6bf400640 1 -- 172.21.15.47:0/238941252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa29e0 msgr2=0x555a48aac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.370+0000 7ff6bf400640 1 --2- 172.21.15.47:0/238941252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa29e0 0x555a48aac030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.370+0000 7ff6bf400640 1 -- 172.21.15.47:0/238941252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa33a0 msgr2=0x555a48ab3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.370+0000 7ff6bf400640 1 --2- 172.21.15.47:0/238941252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa33a0 0x555a48ab3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.370+0000 7ff6bf400640 1 -- 172.21.15.47:0/238941252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a4871a3f0 con 0x555a48aa1810 2024-03-21T11:46:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.370+0000 7ff6bfe00640 1 --2- 172.21.15.47:0/238941252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa33a0 0x555a48ab3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.370+0000 7ff6bf400640 1 --2- 172.21.15.47:0/238941252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa1810 0x555a48aabaf0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8020000 tx=0x7ff6a80bc0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e6793698a609df4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.370+0000 7ff6ad600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.370+0000 7ff6ad600640 1 -- 172.21.15.47:0/238941252 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a809b2d0 con 0x555a48aa1810 2024-03-21T11:46:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.370+0000 7ff6ad600640 1 -- 172.21.15.47:0/238941252 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff6a809b470 con 0x555a48aa1810 2024-03-21T11:46:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.370+0000 7ff6c9059740 1 -- 172.21.15.47:0/238941252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6a80a4890 con 0x555a48aa1810 2024-03-21T11:46:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.371+0000 7ff6ad600640 1 -- 172.21.15.47:0/238941252 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6a8085120 con 0x555a48aa1810 2024-03-21T11:46:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.371+0000 7ff6c9059740 10 client.32722 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.371+0000 7ff6c9059740 1 -- 172.21.15.47:0/238941252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x555a487e47d0 con 0x555a48aa1810 2024-03-21T11:46:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.371+0000 7ff6ad600640 1 -- 172.21.15.47:0/238941252 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6a80a4890 con 0x555a48aa1810 2024-03-21T11:46:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.371+0000 7ff6ad600640 1 -- 172.21.15.47:0/238941252 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff6a8082030 con 0x555a48aa1810 2024-03-21T11:46:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.371+0000 7ff6c9059740 10 client.32722 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:11.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.371+0000 7ff6c9059740 10 client.32722 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:46:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.371+0000 7ff6c9059740 20 client.32722 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.371+0000 7ff6c9059740 1 --2- 172.21.15.47:0/238941252 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555a48ab7160 0x555a48ab9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.371+0000 7ff6c9059740 4 client.32722 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.371+0000 7ff6c9059740 1 -- 172.21.15.47:0/238941252 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a48a13800 con 0x555a48ab7160 2024-03-21T11:46:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.372+0000 7ff6bea00640 1 --2- 172.21.15.47:0/238941252 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555a48ab7160 0x555a48ab9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.373+0000 7ff6bea00640 1 --2- 172.21.15.47:0/238941252 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555a48ab7160 0x555a48ab9550 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5bb916da3a33cf58 server_cookie=22cc497e98a9a10f in_seq=0 out_seq=0 2024-03-21T11:46:11.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.373+0000 7ff6ad600640 10 client.32722 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:46:11.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6ad600640 1 -- 172.21.15.47:0/238941252 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555a48a13800 con 0x555a48ab7160 2024-03-21T11:46:11.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6ad600640 10 client.32722 handle_command_reply: tid=0 2024-03-21T11:46:11.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6c9059740 1 client.32722 shutdown 2024-03-21T11:46:11.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6c9059740 1 -- 172.21.15.47:0/238941252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa1810 msgr2=0x555a48aabaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6c9059740 1 --2- 172.21.15.47:0/238941252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa1810 0x555a48aabaf0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8020000 tx=0x7ff6a80bc0d0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6c9059740 1 -- 172.21.15.47:0/238941252 shutdown_connections 2024-03-21T11:46:11.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6c9059740 1 --2- 172.21.15.47:0/238941252 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555a48ab7160 0x555a48ab9550 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6c9059740 1 --2- 172.21.15.47:0/238941252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa33a0 0x555a48ab3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6c9059740 1 --2- 172.21.15.47:0/238941252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa29e0 0x555a48aac030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6ad600640 0 client.32722 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:46:11.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6c9059740 1 --2- 172.21.15.47:0/238941252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa1810 0x555a48aabaf0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6c9059740 1 -- 172.21.15.47:0/238941252 >> 172.21.15.47:0/238941252 conn(0x555a48a9cc00 msgr2=0x555a48a9beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6c9059740 1 -- 172.21.15.47:0/238941252 shutdown_connections 2024-03-21T11:46:11.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6c9059740 1 -- 172.21.15.47:0/238941252 wait complete. 2024-03-21T11:46:11.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.374+0000 7ff6c9059740 20 client.32722 trim_cache size 0 max 16384 2024-03-21T11:46:11.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.389+0000 7ff6c9059740 1 Processor -- start 2024-03-21T11:46:11.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.389+0000 7ff6c9059740 1 -- start start 2024-03-21T11:46:11.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.389+0000 7ff6c9059740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa1810 0x555a48a9bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.389+0000 7ff6c9059740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa29e0 0x555a48ab2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.389+0000 7ff6c9059740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa33a0 0x555a48abdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.389+0000 7ff6c9059740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c01b8f00 con 0x555a48aa33a0 2024-03-21T11:46:11.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.389+0000 7ff6c9059740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c01ba880 con 0x555a48aa29e0 2024-03-21T11:46:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.389+0000 7ff6c9059740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c01b9560 con 0x555a48aa1810 2024-03-21T11:46:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.389+0000 7ff6bf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa1810 0x555a48a9bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.389+0000 7ff6bea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa29e0 0x555a48ab2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.389+0000 7ff6bf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa1810 0x555a48a9bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60450/0 (socket says 172.21.15.47:60450) 2024-03-21T11:46:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.389+0000 7ff6bf400640 1 -- 172.21.15.47:0/2293531376 learned_addr learned my addr 172.21.15.47:0/2293531376 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.389+0000 7ff6bea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa29e0 0x555a48ab2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41532/0 (socket says 172.21.15.47:41532) 2024-03-21T11:46:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.389+0000 7ff6bfe00640 1 --2- 172.21.15.47:0/2293531376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa33a0 0x555a48abdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.390+0000 7ff6bf400640 1 -- 172.21.15.47:0/2293531376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa29e0 msgr2=0x555a48ab2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.390+0000 7ff6bf400640 1 --2- 172.21.15.47:0/2293531376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa29e0 0x555a48ab2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.390+0000 7ff6bf400640 1 -- 172.21.15.47:0/2293531376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa33a0 msgr2=0x555a48abdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.390+0000 7ff6bf400640 1 --2- 172.21.15.47:0/2293531376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa33a0 0x555a48abdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.390+0000 7ff6bf400640 1 -- 172.21.15.47:0/2293531376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a48a9d970 con 0x555a48aa1810 2024-03-21T11:46:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.390+0000 7ff6bea00640 1 --2- 172.21.15.47:0/2293531376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa29e0 0x555a48ab2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:11.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.390+0000 7ff6bf400640 1 --2- 172.21.15.47:0/2293531376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa1810 0x555a48a9bd30 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7ff6a809c730 tx=0x7ff6a8005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a074862a0259fe9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.390+0000 7ff6afe00640 1 -- 172.21.15.47:0/2293531376 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a809b9c0 con 0x555a48aa1810 2024-03-21T11:46:11.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.391+0000 7ff6afe00640 1 -- 172.21.15.47:0/2293531376 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6a80a4890 con 0x555a48aa1810 2024-03-21T11:46:11.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.391+0000 7ff6afe00640 1 -- 172.21.15.47:0/2293531376 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a80a4d00 con 0x555a48aa1810 2024-03-21T11:46:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.391+0000 7ff6c9059740 1 -- 172.21.15.47:0/2293531376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa1810 msgr2=0x555a48a9bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.391+0000 7ff6c9059740 1 --2- 172.21.15.47:0/2293531376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa1810 0x555a48a9bd30 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7ff6a809c730 tx=0x7ff6a8005ad0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.392+0000 7ff6c9059740 1 -- 172.21.15.47:0/2293531376 shutdown_connections 2024-03-21T11:46:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.392+0000 7ff6c9059740 1 --2- 172.21.15.47:0/2293531376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa33a0 0x555a48abdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.392+0000 7ff6c9059740 1 --2- 172.21.15.47:0/2293531376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa29e0 0x555a48ab2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.392+0000 7ff6c9059740 1 --2- 172.21.15.47:0/2293531376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa1810 0x555a48a9bd30 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.392+0000 7ff6c9059740 1 -- 172.21.15.47:0/2293531376 >> 172.21.15.47:0/2293531376 conn(0x555a48a9cc00 msgr2=0x555a48a9b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.392+0000 7ff6c9059740 1 -- 172.21.15.47:0/2293531376 shutdown_connections 2024-03-21T11:46:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.392+0000 7ff6c9059740 1 -- 172.21.15.47:0/2293531376 wait complete. 2024-03-21T11:46:11.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.392+0000 7ff6c9059740 1 -- start start 2024-03-21T11:46:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.392+0000 7ff6c9059740 1 Processor -- start 2024-03-21T11:46:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6c9059740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa1810 0x555a48ab2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6c9059740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa29e0 0x555a48aaf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6c9059740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa33a0 0x555a48aafd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6c9059740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c01ba220 con 0x555a48aa1810 2024-03-21T11:46:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6c9059740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c01b9bc0 con 0x555a48aa33a0 2024-03-21T11:46:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6c9059740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c01b70c0 con 0x555a48aa29e0 2024-03-21T11:46:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6c9059740 10 client.0 _ll_register_callbacks cb 0x7ff6b8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6bf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa1810 0x555a48ab2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6bfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa33a0 0x555a48aafd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6bfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa33a0 0x555a48aafd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41544/0 (socket says 172.21.15.47:41544) 2024-03-21T11:46:11.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6bf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa1810 0x555a48ab2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35962/0 (socket says 172.21.15.47:35962) 2024-03-21T11:46:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6bfe00640 1 -- 172.21.15.47:0/253709295 learned_addr learned my addr 172.21.15.47:0/253709295 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6bea00640 1 --2- 172.21.15.47:0/253709295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa29e0 0x555a48aaf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6bea00640 1 -- 172.21.15.47:0/253709295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa33a0 msgr2=0x555a48aafd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6bea00640 1 --2- 172.21.15.47:0/253709295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa33a0 0x555a48aafd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6bea00640 1 -- 172.21.15.47:0/253709295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa1810 msgr2=0x555a48ab2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6bea00640 1 --2- 172.21.15.47:0/253709295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa1810 0x555a48ab2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.393+0000 7ff6bea00640 1 -- 172.21.15.47:0/253709295 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a48ab0320 con 0x555a48aa29e0 2024-03-21T11:46:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.394+0000 7ff6bfe00640 1 --2- 172.21.15.47:0/253709295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa33a0 0x555a48aafd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.394+0000 7ff6bea00640 1 --2- 172.21.15.47:0/253709295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa29e0 0x555a48aaf7f0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff6b002efd0 tx=0x7ff6b002bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4267a4383b34c1d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.394+0000 7ff6ad600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.395+0000 7ff6ad600640 1 -- 172.21.15.47:0/253709295 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0047040 con 0x555a48aa29e0 2024-03-21T11:46:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.395+0000 7ff6ad600640 1 -- 172.21.15.47:0/253709295 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff6b0005300 con 0x555a48aa29e0 2024-03-21T11:46:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.395+0000 7ff6c9059740 1 -- 172.21.15.47:0/253709295 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6a80a4890 con 0x555a48aa29e0 2024-03-21T11:46:11.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.395+0000 7ff6ad600640 1 -- 172.21.15.47:0/253709295 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6b0065050 con 0x555a48aa29e0 2024-03-21T11:46:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.395+0000 7ff6c9059740 10 client.32728 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.395+0000 7ff6c9059740 1 -- 172.21.15.47:0/253709295 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x555a48b13940 con 0x555a48aa29e0 2024-03-21T11:46:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.395+0000 7ff6ad600640 1 -- 172.21.15.47:0/253709295 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6a80a4890 con 0x555a48aa29e0 2024-03-21T11:46:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.396+0000 7ff6ad600640 1 -- 172.21.15.47:0/253709295 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff6b0003790 con 0x555a48aa29e0 2024-03-21T11:46:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.396+0000 7ff6c9059740 10 client.32728 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.396+0000 7ff6c9059740 10 client.32728 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:46:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.396+0000 7ff6c9059740 20 client.32728 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.396+0000 7ff6c9059740 1 --2- 172.21.15.47:0/253709295 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555a48b14230 0x555a48b16650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.396+0000 7ff6c9059740 4 client.32728 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:46:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.396+0000 7ff6c9059740 1 -- 172.21.15.47:0/253709295 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555a48a0db50 con 0x555a48b14230 2024-03-21T11:46:11.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.396+0000 7ff6bf400640 1 --2- 172.21.15.47:0/253709295 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555a48b14230 0x555a48b16650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.396+0000 7ff6ad600640 10 client.32728 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:46:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.396+0000 7ff6bf400640 1 --2- 172.21.15.47:0/253709295 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555a48b14230 0x555a48b16650 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6b775a0523924aec server_cookie=f639025ad8b30e01 in_seq=0 out_seq=0 2024-03-21T11:46:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6ad600640 1 -- 172.21.15.47:0/253709295 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555a48a0db50 con 0x555a48b14230 2024-03-21T11:46:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6ad600640 10 client.32728 handle_command_reply: tid=0 2024-03-21T11:46:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6c9059740 1 client.32728 shutdown 2024-03-21T11:46:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6c9059740 1 -- 172.21.15.47:0/253709295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa29e0 msgr2=0x555a48aaf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6c9059740 1 --2- 172.21.15.47:0/253709295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa29e0 0x555a48aaf7f0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff6b002efd0 tx=0x7ff6b002bc40 comp rx=0 tx=0).stop 2024-03-21T11:46:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6c9059740 1 -- 172.21.15.47:0/253709295 shutdown_connections 2024-03-21T11:46:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6c9059740 1 --2- 172.21.15.47:0/253709295 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x555a48b14230 0x555a48b16650 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6c9059740 1 --2- 172.21.15.47:0/253709295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a48aa33a0 0x555a48aafd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6c9059740 1 --2- 172.21.15.47:0/253709295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555a48aa29e0 0x555a48aaf7f0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6c9059740 1 --2- 172.21.15.47:0/253709295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555a48aa1810 0x555a48ab2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6ad600640 0 client.32728 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:46:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6c9059740 1 -- 172.21.15.47:0/253709295 >> 172.21.15.47:0/253709295 conn(0x555a48a9cc00 msgr2=0x555a48a9beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6c9059740 1 -- 172.21.15.47:0/253709295 shutdown_connections 2024-03-21T11:46:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6c9059740 1 -- 172.21.15.47:0/253709295 wait complete. 2024-03-21T11:46:11.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.397+0000 7ff6c9059740 20 client.32728 trim_cache size 0 max 16384 2024-03-21T11:46:11.402 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:46:11.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.398+0000 7ff6c6400640 1 -- 172.21.15.47:0/765028260 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff6a40a2fa0 msgr2=0x7ff6a40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.398+0000 7ff6c6400640 1 --2- 172.21.15.47:0/765028260 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff6a40a2fa0 0x7ff6a40a5460 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0036040 tx=0x7ff6b0000bc0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.398+0000 7ff6c6400640 1 -- 172.21.15.47:0/765028260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c0107cc0 msgr2=0x7ff6c007ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.398+0000 7ff6c6400640 1 --2- 172.21.15.47:0/765028260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c0107cc0 0x7ff6c007ff90 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800e9c0 tx=0x7ff6a800edc0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.398+0000 7ff6c6400640 1 -- 172.21.15.47:0/765028260 shutdown_connections 2024-03-21T11:46:11.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.398+0000 7ff6c6400640 1 --2- 172.21.15.47:0/765028260 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff6a40a2fa0 0x7ff6a40a5460 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.398+0000 7ff6c6400640 1 --2- 172.21.15.47:0/765028260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c0109870 0x7ff6c0081510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.398+0000 7ff6c6400640 1 --2- 172.21.15.47:0/765028260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6c0108eb0 0x7ff6c00804d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.398+0000 7ff6c6400640 1 --2- 172.21.15.47:0/765028260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c0107cc0 0x7ff6c007ff90 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.398+0000 7ff6c6400640 1 -- 172.21.15.47:0/765028260 >> 172.21.15.47:0/765028260 conn(0x7ff6c0076c60 msgr2=0x7ff6c010c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.398+0000 7ff6c6400640 1 -- 172.21.15.47:0/765028260 shutdown_connections 2024-03-21T11:46:11.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.398+0000 7ff6c6400640 1 -- 172.21.15.47:0/765028260 wait complete. 2024-03-21T11:46:11.416 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:11 smithi047 ceph-mon[31351]: pgmap v1086: 129 pgs: 129 active+clean; 664 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-03-21T11:46:11.416 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:11 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/328270803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:46:11.419 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:46:11.419 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:46:11.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:11 smithi102 ceph-mon[36748]: pgmap v1086: 129 pgs: 129 active+clean; 664 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-03-21T11:46:11.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:11 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/328270803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:46:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.640+0000 7f50a9200640 1 -- 172.21.15.47:0/3117971277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50a4102190 msgr2=0x7f50a4104580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.640+0000 7f50a9200640 1 --2- 172.21.15.47:0/3117971277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50a4102190 0x7f50a4104580 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5094009900 tx=0x7f509402bc30 comp rx=0 tx=0).stop 2024-03-21T11:46:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.641+0000 7f50a9200640 1 -- 172.21.15.47:0/3117971277 shutdown_connections 2024-03-21T11:46:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.641+0000 7f50a9200640 1 --2- 172.21.15.47:0/3117971277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50a41073f0 0x7f50a41097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.641+0000 7f50a9200640 1 --2- 172.21.15.47:0/3117971277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50a4104ac0 0x7f50a4106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.641+0000 7f50a9200640 1 --2- 172.21.15.47:0/3117971277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50a4102190 0x7f50a4104580 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.641+0000 7f50a9200640 1 -- 172.21.15.47:0/3117971277 >> 172.21.15.47:0/3117971277 conn(0x7f50a40fbb40 msgr2=0x7f50a40fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.641+0000 7f50a9200640 1 -- 172.21.15.47:0/3117971277 shutdown_connections 2024-03-21T11:46:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.641+0000 7f50a9200640 1 -- 172.21.15.47:0/3117971277 wait complete. 2024-03-21T11:46:11.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.642+0000 7f50a9200640 1 Processor -- start 2024-03-21T11:46:11.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.642+0000 7f50a9200640 1 -- start start 2024-03-21T11:46:11.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50a4102190 0x7f50a4199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50a4104ac0 0x7f50a419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a9200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50a41073f0 0x7f50a41a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a9200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a410d370 con 0x7f50a41073f0 2024-03-21T11:46:11.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a9200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a410d1f0 con 0x7f50a4104ac0 2024-03-21T11:46:11.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a9200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a410d4f0 con 0x7f50a4102190 2024-03-21T11:46:11.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50a4102190 0x7f50a4199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50a41073f0 0x7f50a41a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50a4104ac0 0x7f50a419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50a4102190 0x7f50a4199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60472/0 (socket says 172.21.15.47:60472) 2024-03-21T11:46:11.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a1e00640 1 -- 172.21.15.47:0/1710236604 learned_addr learned my addr 172.21.15.47:0/1710236604 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:11.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:11 smithi100 ceph-mon[36245]: pgmap v1086: 129 pgs: 129 active+clean; 664 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-03-21T11:46:11.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:11 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/328270803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:46:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a1e00640 1 -- 172.21.15.47:0/1710236604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50a4104ac0 msgr2=0x7f50a419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a1e00640 1 --2- 172.21.15.47:0/1710236604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50a4104ac0 0x7f50a419a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a1e00640 1 -- 172.21.15.47:0/1710236604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50a41073f0 msgr2=0x7f50a41a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.643+0000 7f50a1e00640 1 --2- 172.21.15.47:0/1710236604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50a41073f0 0x7f50a41a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.644+0000 7f50a1e00640 1 -- 172.21.15.47:0/1710236604 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a41a1850 con 0x7f50a4102190 2024-03-21T11:46:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.644+0000 7f50a2800640 1 --2- 172.21.15.47:0/1710236604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50a41073f0 0x7f50a41a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.644+0000 7f50a1400640 1 --2- 172.21.15.47:0/1710236604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50a4104ac0 0x7f50a419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.644+0000 7f50a1e00640 1 --2- 172.21.15.47:0/1710236604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50a4102190 0x7f50a4199c40 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f5094005ab0 tx=0x7f509402c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.644+0000 7f509aa00640 1 -- 172.21.15.47:0/1710236604 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094043020 con 0x7f50a4102190 2024-03-21T11:46:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.644+0000 7f50a9200640 1 -- 172.21.15.47:0/1710236604 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a41a1ae0 con 0x7f50a4102190 2024-03-21T11:46:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.644+0000 7f509aa00640 1 -- 172.21.15.47:0/1710236604 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50940054e0 con 0x7f50a4102190 2024-03-21T11:46:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.644+0000 7f509aa00640 1 -- 172.21.15.47:0/1710236604 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094003ce0 con 0x7f50a4102190 2024-03-21T11:46:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.645+0000 7f50a9200640 1 -- 172.21.15.47:0/1710236604 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a41a2030 con 0x7f50a4102190 2024-03-21T11:46:11.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.648+0000 7f50abb22740 1 Processor -- start 2024-03-21T11:46:11.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.649+0000 7f50abb22740 1 -- start start 2024-03-21T11:46:11.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.649+0000 7f50abb22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b10810 0x563ce9b10c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.649+0000 7f50abb22740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b119e0 0x563ce9b11e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.649+0000 7f50abb22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b123a0 0x563ce9b18c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.649+0000 7f50abb22740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce9a752c0 con 0x563ce9b119e0 2024-03-21T11:46:11.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.649+0000 7f50abb22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce9a1dc90 con 0x563ce9b123a0 2024-03-21T11:46:11.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.649+0000 7f50abb22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce9a71360 con 0x563ce9b10810 2024-03-21T11:46:11.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.649+0000 7f509aa00640 1 -- 172.21.15.47:0/1710236604 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5094046020 con 0x7f50a4102190 2024-03-21T11:46:11.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.649+0000 7f50a1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b123a0 0x563ce9b18c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.649+0000 7f50a2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b10810 0x563ce9b10c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.649+0000 7f50a1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b123a0 0x563ce9b18c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41572/0 (socket says 172.21.15.47:41572) 2024-03-21T11:46:11.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.649+0000 7f50a1400640 1 -- 172.21.15.47:0/1479793238 learned_addr learned my addr 172.21.15.47:0/1479793238 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:11.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.650+0000 7f50a1e00640 1 --2- 172.21.15.47:0/1479793238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b119e0 0x563ce9b11e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.650+0000 7f50a1400640 1 -- 172.21.15.47:0/1479793238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b10810 msgr2=0x563ce9b10c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.650+0000 7f50a1400640 1 --2- 172.21.15.47:0/1479793238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b10810 0x563ce9b10c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.650+0000 7f50a1400640 1 -- 172.21.15.47:0/1479793238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b119e0 msgr2=0x563ce9b11e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.650+0000 7f50a1400640 1 --2- 172.21.15.47:0/1479793238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b119e0 0x563ce9b11e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.650+0000 7f50a1400640 1 -- 172.21.15.47:0/1479793238 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ce9a42080 con 0x563ce9b123a0 2024-03-21T11:46:11.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.650+0000 7f50a2800640 1 --2- 172.21.15.47:0/1479793238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b10810 0x563ce9b10c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:11.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.650+0000 7f50a1e00640 1 --2- 172.21.15.47:0/1479793238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b119e0 0x563ce9b11e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:11.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.650+0000 7f50a1400640 1 --2- 172.21.15.47:0/1479793238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b123a0 0x563ce9b18c30 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f5090010480 tx=0x7f5090010940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c21f9e9c4b6c337 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.651+0000 7f508f400640 1 -- 172.21.15.47:0/1479793238 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5090011760 con 0x563ce9b123a0 2024-03-21T11:46:11.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.651+0000 7f508f400640 1 -- 172.21.15.47:0/1479793238 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5090011de0 con 0x563ce9b123a0 2024-03-21T11:46:11.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.651+0000 7f508f400640 1 -- 172.21.15.47:0/1479793238 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509000e430 con 0x563ce9b123a0 2024-03-21T11:46:11.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.651+0000 7f509aa00640 1 --2- 172.21.15.47:0/1710236604 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50880a2f20 0x7f50880a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.651+0000 7f509aa00640 1 -- 172.21.15.47:0/1710236604 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f50940f1980 con 0x7f50a4102190 2024-03-21T11:46:11.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.651+0000 7f50a2800640 1 --2- 172.21.15.47:0/1710236604 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50880a2f20 0x7f50880a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.651+0000 7f50abb22740 1 -- 172.21.15.47:0/1479793238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b123a0 msgr2=0x563ce9b18c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.651+0000 7f50abb22740 1 --2- 172.21.15.47:0/1479793238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b123a0 0x563ce9b18c30 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f5090010480 tx=0x7f5090010940 comp rx=0 tx=0).stop 2024-03-21T11:46:11.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.652+0000 7f50abb22740 1 -- 172.21.15.47:0/1479793238 shutdown_connections 2024-03-21T11:46:11.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.652+0000 7f50abb22740 1 --2- 172.21.15.47:0/1479793238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b123a0 0x563ce9b18c30 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.652+0000 7f50abb22740 1 --2- 172.21.15.47:0/1479793238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b119e0 0x563ce9b11e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.652+0000 7f50abb22740 1 --2- 172.21.15.47:0/1479793238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b10810 0x563ce9b10c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.652+0000 7f50abb22740 1 -- 172.21.15.47:0/1479793238 >> 172.21.15.47:0/1479793238 conn(0x563ce9b0bc00 msgr2=0x563ce9b0c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.652+0000 7f50a2800640 1 --2- 172.21.15.47:0/1710236604 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50880a2f20 0x7f50880a53e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x563ce9b118e0 tx=0x7f509c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.652+0000 7f50abb22740 1 -- 172.21.15.47:0/1479793238 shutdown_connections 2024-03-21T11:46:11.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.652+0000 7f50abb22740 1 -- 172.21.15.47:0/1479793238 wait complete. 2024-03-21T11:46:11.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.653+0000 7f50abb22740 1 -- start start 2024-03-21T11:46:11.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.653+0000 7f50abb22740 1 Processor -- start 2024-03-21T11:46:11.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.654+0000 7f50abb22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b10810 0x563ce9b1aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.654+0000 7f50abb22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 0x563ce9b1af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.654+0000 7f50abb22740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b123a0 0x563ce9b21fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.654+0000 7f50abb22740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce9b193b0 con 0x563ce9b123a0 2024-03-21T11:46:11.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.655+0000 7f50abb22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce9a310d0 con 0x563ce9b10810 2024-03-21T11:46:11.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.655+0000 7f50abb22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ce9b19530 con 0x563ce9b119e0 2024-03-21T11:46:11.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.655+0000 7f50abb22740 10 client.0 _ll_register_callbacks cb 0x563ce9a35e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:11.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.655+0000 7f50a1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b10810 0x563ce9b1aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.655+0000 7f50a1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b10810 0x563ce9b1aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41588/0 (socket says 172.21.15.47:41588) 2024-03-21T11:46:11.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.655+0000 7f50a1e00640 1 -- 172.21.15.47:0/1723456510 learned_addr learned my addr 172.21.15.47:0/1723456510 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:11.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.655+0000 7f50a1400640 1 --2- 172.21.15.47:0/1723456510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 0x563ce9b1af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.655+0000 7f50a1e00640 1 -- 172.21.15.47:0/1723456510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 msgr2=0x563ce9b1af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.655+0000 7f50a1e00640 1 --2- 172.21.15.47:0/1723456510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 0x563ce9b1af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.655+0000 7f50a1e00640 1 -- 172.21.15.47:0/1723456510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b123a0 msgr2=0x563ce9b21fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.655+0000 7f50a1e00640 1 --2- 172.21.15.47:0/1723456510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b123a0 0x563ce9b21fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.655+0000 7f50a1e00640 1 -- 172.21.15.47:0/1723456510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ce97893f0 con 0x563ce9b10810 2024-03-21T11:46:11.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.656+0000 7f50a1e00640 1 --2- 172.21.15.47:0/1723456510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b10810 0x563ce9b1aa30 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f50940bdc50 tx=0x7f50940b9ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f79a19c76dc80bc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.656+0000 7f5083e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:11.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.656+0000 7f5083e00640 1 -- 172.21.15.47:0/1723456510 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50940bd620 con 0x563ce9b10810 2024-03-21T11:46:11.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.656+0000 7f5083e00640 1 -- 172.21.15.47:0/1723456510 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f50940c6240 con 0x563ce9b10810 2024-03-21T11:46:11.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.656+0000 7f50abb22740 1 -- 172.21.15.47:0/1723456510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5090011de0 con 0x563ce9b10810 2024-03-21T11:46:11.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.657+0000 7f5083e00640 1 -- 172.21.15.47:0/1723456510 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50940bbe00 con 0x563ce9b10810 2024-03-21T11:46:11.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.657+0000 7f50abb22740 10 client.32762 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:11.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.658+0000 7f50abb22740 1 -- 172.21.15.47:0/1723456510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563ce98537d0 con 0x563ce9b10810 2024-03-21T11:46:11.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.658+0000 7f5083e00640 1 -- 172.21.15.47:0/1723456510 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50940044a0 con 0x563ce9b10810 2024-03-21T11:46:11.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.658+0000 7f5083e00640 1 -- 172.21.15.47:0/1723456510 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f50940bd110 con 0x563ce9b10810 2024-03-21T11:46:11.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.658+0000 7f50abb22740 10 client.32762 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:11.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.658+0000 7f50abb22740 10 client.32762 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:46:11.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.658+0000 7f50abb22740 20 client.32762 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:11.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.658+0000 7f50abb22740 1 --2- 172.21.15.47:0/1723456510 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563ce9b264f0 0x563ce9b28900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.658+0000 7f50abb22740 4 client.32762 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:11.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.658+0000 7f50abb22740 1 -- 172.21.15.47:0/1723456510 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5090011de0 con 0x563ce9b264f0 2024-03-21T11:46:11.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.661+0000 7f50a1400640 1 --2- 172.21.15.47:0/1723456510 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563ce9b264f0 0x563ce9b28900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.662+0000 7f50a1400640 1 --2- 172.21.15.47:0/1723456510 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563ce9b264f0 0x563ce9b28900 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3352bb8b2be30f1f server_cookie=b63a4fa683a001bd in_seq=0 out_seq=0 2024-03-21T11:46:11.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.662+0000 7f5083e00640 10 client.32762 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:46:11.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.663+0000 7f5083e00640 1 -- 172.21.15.47:0/1723456510 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f5090011de0 con 0x563ce9b264f0 2024-03-21T11:46:11.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.663+0000 7f5083e00640 10 client.32762 handle_command_reply: tid=0 2024-03-21T11:46:11.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.663+0000 7f50abb22740 1 client.32762 shutdown 2024-03-21T11:46:11.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.663+0000 7f50abb22740 1 -- 172.21.15.47:0/1723456510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b10810 msgr2=0x563ce9b1aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.663+0000 7f50abb22740 1 --2- 172.21.15.47:0/1723456510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b10810 0x563ce9b1aa30 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f50940bdc50 tx=0x7f50940b9ed0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.664+0000 7f50abb22740 1 -- 172.21.15.47:0/1723456510 shutdown_connections 2024-03-21T11:46:11.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.664+0000 7f50abb22740 1 --2- 172.21.15.47:0/1723456510 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563ce9b264f0 0x563ce9b28900 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.664+0000 7f50abb22740 1 --2- 172.21.15.47:0/1723456510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b123a0 0x563ce9b21fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.664+0000 7f5083e00640 0 client.32762 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:46:11.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.664+0000 7f50abb22740 1 --2- 172.21.15.47:0/1723456510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 0x563ce9b1af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.664+0000 7f50abb22740 1 --2- 172.21.15.47:0/1723456510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b10810 0x563ce9b1aa30 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.664+0000 7f50abb22740 1 -- 172.21.15.47:0/1723456510 >> 172.21.15.47:0/1723456510 conn(0x563ce9b0bc00 msgr2=0x563ce9b0aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.664+0000 7f50abb22740 1 -- 172.21.15.47:0/1723456510 shutdown_connections 2024-03-21T11:46:11.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.665+0000 7f50abb22740 1 -- 172.21.15.47:0/1723456510 wait complete. 2024-03-21T11:46:11.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.665+0000 7f50abb22740 20 client.32762 trim_cache size 0 max 16384 2024-03-21T11:46:11.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.686+0000 7f50abb22740 1 Processor -- start 2024-03-21T11:46:11.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.686+0000 7f50abb22740 1 -- start start 2024-03-21T11:46:11.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.687+0000 7f50abb22740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b10810 0x563ce9b14c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.687+0000 7f50abb22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 0x563ce9b21f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.687+0000 7f50abb22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b123a0 0x563ce9b15150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.687+0000 7f50abb22740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a41ac2c0 con 0x563ce9b10810 2024-03-21T11:46:11.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.687+0000 7f50abb22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a41adc40 con 0x563ce9b123a0 2024-03-21T11:46:11.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.687+0000 7f50abb22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a41ac920 con 0x563ce9b119e0 2024-03-21T11:46:11.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.687+0000 7f50a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 0x563ce9b21f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.687+0000 7f50a2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b123a0 0x563ce9b15150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.687+0000 7f50a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 0x563ce9b21f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60498/0 (socket says 172.21.15.47:60498) 2024-03-21T11:46:11.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.687+0000 7f50a1400640 1 -- 172.21.15.47:0/2707679550 learned_addr learned my addr 172.21.15.47:0/2707679550 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:11.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.688+0000 7f50a1e00640 1 --2- 172.21.15.47:0/2707679550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b10810 0x563ce9b14c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.688+0000 7f50a1400640 1 -- 172.21.15.47:0/2707679550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b123a0 msgr2=0x563ce9b15150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.688+0000 7f50a1400640 1 --2- 172.21.15.47:0/2707679550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b123a0 0x563ce9b15150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.688+0000 7f50a1400640 1 -- 172.21.15.47:0/2707679550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b10810 msgr2=0x563ce9b14c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.688+0000 7f50a1400640 1 --2- 172.21.15.47:0/2707679550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b10810 0x563ce9b14c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.688+0000 7f50a1400640 1 -- 172.21.15.47:0/2707679550 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ce9b11460 con 0x563ce9b119e0 2024-03-21T11:46:11.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.688+0000 7f50a2800640 1 --2- 172.21.15.47:0/2707679550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b123a0 0x563ce9b15150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:11.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.688+0000 7f50a1400640 1 --2- 172.21.15.47:0/2707679550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 0x563ce9b21f70 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f509001b000 tx=0x7f509000bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cdc95fc1885aae0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.689+0000 7f508ea00640 1 -- 172.21.15.47:0/2707679550 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509002c170 con 0x563ce9b119e0 2024-03-21T11:46:11.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.689+0000 7f508ea00640 1 -- 172.21.15.47:0/2707679550 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f509001c480 con 0x563ce9b119e0 2024-03-21T11:46:11.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.689+0000 7f508ea00640 1 -- 172.21.15.47:0/2707679550 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5090005280 con 0x563ce9b119e0 2024-03-21T11:46:11.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.690+0000 7f50abb22740 1 -- 172.21.15.47:0/2707679550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 msgr2=0x563ce9b21f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.690+0000 7f50abb22740 1 --2- 172.21.15.47:0/2707679550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 0x563ce9b21f70 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f509001b000 tx=0x7f509000bfa0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.690+0000 7f50abb22740 1 -- 172.21.15.47:0/2707679550 shutdown_connections 2024-03-21T11:46:11.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.690+0000 7f50abb22740 1 --2- 172.21.15.47:0/2707679550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b123a0 0x563ce9b15150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.690+0000 7f50abb22740 1 --2- 172.21.15.47:0/2707679550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 0x563ce9b21f70 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.690+0000 7f50abb22740 1 --2- 172.21.15.47:0/2707679550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b10810 0x563ce9b14c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.690+0000 7f50abb22740 1 -- 172.21.15.47:0/2707679550 >> 172.21.15.47:0/2707679550 conn(0x563ce9b0bc00 msgr2=0x563ce9b0f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.691+0000 7f50abb22740 1 -- 172.21.15.47:0/2707679550 shutdown_connections 2024-03-21T11:46:11.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.691+0000 7f50abb22740 1 -- 172.21.15.47:0/2707679550 wait complete. 2024-03-21T11:46:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.692+0000 7f50abb22740 1 -- start start 2024-03-21T11:46:11.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.692+0000 7f50abb22740 1 Processor -- start 2024-03-21T11:46:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.693+0000 7f50abb22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b10810 0x563ce9b21f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.693+0000 7f50abb22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 0x563ce9b1ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.693+0000 7f50abb22740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b123a0 0x563ce9b1efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.693+0000 7f50abb22740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a41ad5e0 con 0x563ce9b123a0 2024-03-21T11:46:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.693+0000 7f50abb22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a41acf80 con 0x563ce9b10810 2024-03-21T11:46:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.693+0000 7f50abb22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a41aa480 con 0x563ce9b119e0 2024-03-21T11:46:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.693+0000 7f50abb22740 10 client.0 _ll_register_callbacks cb 0x7f5094003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:11.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.694+0000 7f50a1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b10810 0x563ce9b21f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.694+0000 7f50a1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b10810 0x563ce9b21f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41602/0 (socket says 172.21.15.47:41602) 2024-03-21T11:46:11.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.694+0000 7f50a1e00640 1 -- 172.21.15.47:0/954712719 learned_addr learned my addr 172.21.15.47:0/954712719 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:11.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.695+0000 7f50a2800640 1 --2- 172.21.15.47:0/954712719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b123a0 0x563ce9b1efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.695+0000 7f50a1400640 1 --2- 172.21.15.47:0/954712719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 0x563ce9b1ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.696+0000 7f50a1e00640 1 -- 172.21.15.47:0/954712719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 msgr2=0x563ce9b1ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.696+0000 7f50a1e00640 1 --2- 172.21.15.47:0/954712719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 0x563ce9b1ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.696+0000 7f50a1e00640 1 -- 172.21.15.47:0/954712719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b123a0 msgr2=0x563ce9b1efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.696+0000 7f50a1e00640 1 --2- 172.21.15.47:0/954712719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b123a0 0x563ce9b1efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.696+0000 7f50a1e00640 1 -- 172.21.15.47:0/954712719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ce9b9b590 con 0x563ce9b10810 2024-03-21T11:46:11.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.697+0000 7f50a2800640 1 --2- 172.21.15.47:0/954712719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b123a0 0x563ce9b1efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.700+0000 7f50a1e00640 1 --2- 172.21.15.47:0/954712719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b10810 0x563ce9b21f70 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f50940bd520 tx=0x7f50940f1c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc7b0fa5749704dd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.700+0000 7f5083e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.702+0000 7f5083e00640 1 -- 172.21.15.47:0/954712719 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50940044a0 con 0x563ce9b10810 2024-03-21T11:46:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.702+0000 7f5083e00640 1 -- 172.21.15.47:0/954712719 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f50940b9b90 con 0x563ce9b10810 2024-03-21T11:46:11.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.702+0000 7f50abb22740 1 -- 172.21.15.47:0/954712719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f509001c480 con 0x563ce9b10810 2024-03-21T11:46:11.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.712+0000 7f5083e00640 1 -- 172.21.15.47:0/954712719 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5094005690 con 0x563ce9b10810 2024-03-21T11:46:11.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.712+0000 7f50abb22740 10 client.32768 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:11.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.712+0000 7f50abb22740 1 -- 172.21.15.47:0/954712719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563ce9b9c840 con 0x563ce9b10810 2024-03-21T11:46:11.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.712+0000 7f5083e00640 1 -- 172.21.15.47:0/954712719 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f509001c480 con 0x563ce9b10810 2024-03-21T11:46:11.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.713+0000 7f5083e00640 1 -- 172.21.15.47:0/954712719 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5094004f80 con 0x563ce9b10810 2024-03-21T11:46:11.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.713+0000 7f50abb22740 10 client.32768 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:11.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.713+0000 7f50abb22740 10 client.32768 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:46:11.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.713+0000 7f50abb22740 20 client.32768 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:11.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.713+0000 7f50abb22740 1 --2- 172.21.15.47:0/954712719 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563ce9b9cf60 0x563ce9b9f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.713+0000 7f50abb22740 4 client.32768 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:46:11.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.713+0000 7f50abb22740 1 -- 172.21.15.47:0/954712719 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563ce9a82800 con 0x563ce9b9cf60 2024-03-21T11:46:11.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.713+0000 7f50a1400640 1 --2- 172.21.15.47:0/954712719 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563ce9b9cf60 0x563ce9b9f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.714+0000 7f50a1400640 1 --2- 172.21.15.47:0/954712719 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563ce9b9cf60 0x563ce9b9f380 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd6e1a9099707c0 server_cookie=9186ceb40cc03cd9 in_seq=0 out_seq=0 2024-03-21T11:46:11.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.714+0000 7f5083e00640 10 client.32768 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:46:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.715+0000 7f5083e00640 1 -- 172.21.15.47:0/954712719 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563ce9a82800 con 0x563ce9b9cf60 2024-03-21T11:46:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.715+0000 7f5083e00640 10 client.32768 handle_command_reply: tid=0 2024-03-21T11:46:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.715+0000 7f50abb22740 1 client.32768 shutdown 2024-03-21T11:46:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.716+0000 7f50abb22740 1 -- 172.21.15.47:0/954712719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b10810 msgr2=0x563ce9b21f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.716+0000 7f50abb22740 1 --2- 172.21.15.47:0/954712719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b10810 0x563ce9b21f70 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f50940bd520 tx=0x7f50940f1c80 comp rx=0 tx=0).stop 2024-03-21T11:46:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.716+0000 7f50abb22740 1 -- 172.21.15.47:0/954712719 shutdown_connections 2024-03-21T11:46:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.716+0000 7f50abb22740 1 --2- 172.21.15.47:0/954712719 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x563ce9b9cf60 0x563ce9b9f380 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.716+0000 7f50abb22740 1 --2- 172.21.15.47:0/954712719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ce9b123a0 0x563ce9b1efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.716+0000 7f50abb22740 1 --2- 172.21.15.47:0/954712719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ce9b119e0 0x563ce9b1ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.716+0000 7f50abb22740 1 --2- 172.21.15.47:0/954712719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ce9b10810 0x563ce9b21f70 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.716+0000 7f50abb22740 1 -- 172.21.15.47:0/954712719 >> 172.21.15.47:0/954712719 conn(0x563ce9b0bc00 msgr2=0x563ce9b14c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.716+0000 7f5083e00640 0 client.32768 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:46:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.716+0000 7f50abb22740 1 -- 172.21.15.47:0/954712719 shutdown_connections 2024-03-21T11:46:11.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.717+0000 7f50abb22740 1 -- 172.21.15.47:0/954712719 wait complete. 2024-03-21T11:46:11.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.717+0000 7f50abb22740 20 client.32768 trim_cache size 0 max 16384 2024-03-21T11:46:11.718 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:46:11.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.718+0000 7f50a9200640 1 -- 172.21.15.47:0/1710236604 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50880a2f20 msgr2=0x7f50880a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.718+0000 7f50a9200640 1 --2- 172.21.15.47:0/1710236604 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50880a2f20 0x7f50880a53e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x563ce9b118e0 tx=0x7f509c009500 comp rx=0 tx=0).stop 2024-03-21T11:46:11.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.718+0000 7f50a9200640 1 -- 172.21.15.47:0/1710236604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50a4102190 msgr2=0x7f50a4199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.718+0000 7f50a9200640 1 --2- 172.21.15.47:0/1710236604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50a4102190 0x7f50a4199c40 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f5094005ab0 tx=0x7f509402c6c0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.718+0000 7f50a9200640 1 -- 172.21.15.47:0/1710236604 shutdown_connections 2024-03-21T11:46:11.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.718+0000 7f50a9200640 1 --2- 172.21.15.47:0/1710236604 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f50880a2f20 0x7f50880a53e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.718+0000 7f50a9200640 1 --2- 172.21.15.47:0/1710236604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50a41073f0 0x7f50a41a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.718+0000 7f50a9200640 1 --2- 172.21.15.47:0/1710236604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50a4104ac0 0x7f50a419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.718+0000 7f50a9200640 1 --2- 172.21.15.47:0/1710236604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50a4102190 0x7f50a4199c40 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.718+0000 7f50a9200640 1 -- 172.21.15.47:0/1710236604 >> 172.21.15.47:0/1710236604 conn(0x7f50a40fbb40 msgr2=0x7f50a4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.718+0000 7f50a9200640 1 -- 172.21.15.47:0/1710236604 shutdown_connections 2024-03-21T11:46:11.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.718+0000 7f50a9200640 1 -- 172.21.15.47:0/1710236604 wait complete. 2024-03-21T11:46:11.740 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:46:11.740 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:46:11.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.954+0000 7f414f600640 1 -- 172.21.15.47:0/2525231396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4148104ab0 msgr2=0x7f4148104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.954+0000 7f414f600640 1 --2- 172.21.15.47:0/2525231396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4148104ab0 0x7f4148104f30 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f4130009900 tx=0x7f413002bc30 comp rx=0 tx=0).stop 2024-03-21T11:46:11.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.955+0000 7f414f600640 1 -- 172.21.15.47:0/2525231396 shutdown_connections 2024-03-21T11:46:11.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.955+0000 7f414f600640 1 --2- 172.21.15.47:0/2525231396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4148105470 0x7f414810bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.955+0000 7f414f600640 1 --2- 172.21.15.47:0/2525231396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4148104ab0 0x7f4148104f30 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.955+0000 7f414f600640 1 --2- 172.21.15.47:0/2525231396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41481038c0 0x7f4148103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.955+0000 7f414f600640 1 -- 172.21.15.47:0/2525231396 >> 172.21.15.47:0/2525231396 conn(0x7f41480fee50 msgr2=0x7f4148101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.955+0000 7f414f600640 1 -- 172.21.15.47:0/2525231396 shutdown_connections 2024-03-21T11:46:11.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.955+0000 7f414f600640 1 -- 172.21.15.47:0/2525231396 wait complete. 2024-03-21T11:46:11.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.956+0000 7f414f600640 1 Processor -- start 2024-03-21T11:46:11.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.956+0000 7f414f600640 1 -- start start 2024-03-21T11:46:11.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.956+0000 7f414f600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41481038c0 0x7f4148199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f414f600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4148104ab0 0x7f414819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f414f600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4148105470 0x7f41481a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f414f600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414810f5b0 con 0x7f4148104ab0 2024-03-21T11:46:11.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f414f600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414810f430 con 0x7f4148105470 2024-03-21T11:46:11.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f414f600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414810f730 con 0x7f41481038c0 2024-03-21T11:46:11.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f414cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41481038c0 0x7f4148199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f414cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41481038c0 0x7f4148199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60544/0 (socket says 172.21.15.47:60544) 2024-03-21T11:46:11.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f414cc00640 1 -- 172.21.15.47:0/2223805910 learned_addr learned my addr 172.21.15.47:0/2223805910 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:11.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f414d600640 1 --2- 172.21.15.47:0/2223805910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4148105470 0x7f41481a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f413fe00640 1 --2- 172.21.15.47:0/2223805910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4148104ab0 0x7f414819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f414cc00640 1 -- 172.21.15.47:0/2223805910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4148105470 msgr2=0x7f41481a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f414cc00640 1 --2- 172.21.15.47:0/2223805910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4148105470 0x7f41481a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f414cc00640 1 -- 172.21.15.47:0/2223805910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4148104ab0 msgr2=0x7f414819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f414cc00640 1 --2- 172.21.15.47:0/2223805910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4148104ab0 0x7f414819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.957+0000 7f414cc00640 1 -- 172.21.15.47:0/2223805910 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41481a1850 con 0x7f41481038c0 2024-03-21T11:46:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.958+0000 7f414cc00640 1 --2- 172.21.15.47:0/2223805910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41481038c0 0x7f4148199c40 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f413800b450 tx=0x7f413800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.958+0000 7f4146000640 1 -- 172.21.15.47:0/2223805910 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138015020 con 0x7f41481038c0 2024-03-21T11:46:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.958+0000 7f414f600640 1 -- 172.21.15.47:0/2223805910 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41481a1b40 con 0x7f41481038c0 2024-03-21T11:46:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.958+0000 7f4146000640 1 -- 172.21.15.47:0/2223805910 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4138011d70 con 0x7f41481038c0 2024-03-21T11:46:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.958+0000 7f4146000640 1 -- 172.21.15.47:0/2223805910 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413800f340 con 0x7f41481038c0 2024-03-21T11:46:11.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.958+0000 7f414f600640 1 -- 172.21.15.47:0/2223805910 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41481a2040 con 0x7f41481038c0 2024-03-21T11:46:11.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.962+0000 7f415224d740 1 Processor -- start 2024-03-21T11:46:11.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.962+0000 7f415224d740 1 -- start start 2024-03-21T11:46:11.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.963+0000 7f4146000640 1 -- 172.21.15.47:0/2223805910 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4138018020 con 0x7f41481038c0 2024-03-21T11:46:11.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.963+0000 7f415224d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c030810 0x55751c030c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.963+0000 7f415224d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c0319e0 0x55751c031e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.963+0000 7f415224d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0323a0 0x55751c038c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.963+0000 7f415224d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55751bf952c0 con 0x55751c0319e0 2024-03-21T11:46:11.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.963+0000 7f415224d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55751bf3dc90 con 0x55751c0323a0 2024-03-21T11:46:11.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.963+0000 7f415224d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55751bf91360 con 0x55751c030810 2024-03-21T11:46:11.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.963+0000 7f414d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c030810 0x55751c030c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.963+0000 7f414d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c030810 0x55751c030c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60550/0 (socket says 172.21.15.47:60550) 2024-03-21T11:46:11.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.963+0000 7f414d600640 1 -- 172.21.15.47:0/2164497106 learned_addr learned my addr 172.21.15.47:0/2164497106 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:11.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.964+0000 7f414cc00640 1 --2- 172.21.15.47:0/2164497106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c0319e0 0x55751c031e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.964+0000 7f414cc00640 1 -- 172.21.15.47:0/2164497106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c030810 msgr2=0x55751c030c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.964+0000 7f414cc00640 1 --2- 172.21.15.47:0/2164497106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c030810 0x55751c030c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.964+0000 7f414cc00640 1 -- 172.21.15.47:0/2164497106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0323a0 msgr2=0x55751c038c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.964+0000 7f414cc00640 1 --2- 172.21.15.47:0/2164497106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0323a0 0x55751c038c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.964+0000 7f414cc00640 1 -- 172.21.15.47:0/2164497106 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55751bf62080 con 0x55751c0319e0 2024-03-21T11:46:11.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.964+0000 7f4146000640 1 --2- 172.21.15.47:0/2223805910 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f412c0a2f20 0x7f412c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.965+0000 7f4146000640 1 -- 172.21.15.47:0/2223805910 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f41380cf2c0 con 0x7f41481038c0 2024-03-21T11:46:11.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.965+0000 7f414d600640 1 --2- 172.21.15.47:0/2223805910 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f412c0a2f20 0x7f412c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.965+0000 7f414cc00640 1 --2- 172.21.15.47:0/2164497106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c0319e0 0x55751c031e60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f41380cc970 tx=0x7f41380d2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75b033792f82467 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.966+0000 7f413dc00640 1 -- 172.21.15.47:0/2164497106 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413809b3d0 con 0x55751c0319e0 2024-03-21T11:46:11.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.966+0000 7f413dc00640 1 -- 172.21.15.47:0/2164497106 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f413809bd00 con 0x55751c0319e0 2024-03-21T11:46:11.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.966+0000 7f413dc00640 1 -- 172.21.15.47:0/2164497106 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138009280 con 0x55751c0319e0 2024-03-21T11:46:11.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.966+0000 7f415224d740 1 -- 172.21.15.47:0/2164497106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c0319e0 msgr2=0x55751c031e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.966+0000 7f414d600640 1 --2- 172.21.15.47:0/2223805910 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f412c0a2f20 0x7f412c0a53e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x55751c0318e0 tx=0x7f4140009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.966+0000 7f415224d740 1 --2- 172.21.15.47:0/2164497106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c0319e0 0x55751c031e60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f41380cc970 tx=0x7f41380d2f90 comp rx=0 tx=0).stop 2024-03-21T11:46:11.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.967+0000 7f415224d740 1 -- 172.21.15.47:0/2164497106 shutdown_connections 2024-03-21T11:46:11.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.967+0000 7f415224d740 1 --2- 172.21.15.47:0/2164497106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0323a0 0x55751c038c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.967+0000 7f415224d740 1 --2- 172.21.15.47:0/2164497106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c0319e0 0x55751c031e60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.967+0000 7f415224d740 1 --2- 172.21.15.47:0/2164497106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c030810 0x55751c030c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.967+0000 7f415224d740 1 -- 172.21.15.47:0/2164497106 >> 172.21.15.47:0/2164497106 conn(0x55751c02bc00 msgr2=0x55751c02c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.967+0000 7f415224d740 1 -- 172.21.15.47:0/2164497106 shutdown_connections 2024-03-21T11:46:11.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.967+0000 7f415224d740 1 -- 172.21.15.47:0/2164497106 wait complete. 2024-03-21T11:46:11.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.967+0000 7f415224d740 1 -- start start 2024-03-21T11:46:11.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.967+0000 7f415224d740 1 Processor -- start 2024-03-21T11:46:11.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.968+0000 7f415224d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c03aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.968+0000 7f415224d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0319e0 0x55751c03afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.968+0000 7f415224d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0323a0 0x55751c042000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.968+0000 7f415224d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55751c0393b0 con 0x55751c030810 2024-03-21T11:46:11.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.968+0000 7f415224d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55751bf510d0 con 0x55751c0319e0 2024-03-21T11:46:11.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.968+0000 7f415224d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55751c039530 con 0x55751c0323a0 2024-03-21T11:46:11.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.968+0000 7f415224d740 10 client.0 _ll_register_callbacks cb 0x55751bf55e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:11.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.968+0000 7f413fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0319e0 0x55751c03afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.968+0000 7f414cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c03aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.968+0000 7f414cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c03aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36056/0 (socket says 172.21.15.47:36056) 2024-03-21T11:46:11.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.968+0000 7f414cc00640 1 -- 172.21.15.47:0/1841871917 learned_addr learned my addr 172.21.15.47:0/1841871917 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:11.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.968+0000 7f414d600640 1 --2- 172.21.15.47:0/1841871917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0323a0 0x55751c042000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.969+0000 7f414cc00640 1 -- 172.21.15.47:0/1841871917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0323a0 msgr2=0x55751c042000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.969+0000 7f414cc00640 1 --2- 172.21.15.47:0/1841871917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0323a0 0x55751c042000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.969+0000 7f414cc00640 1 -- 172.21.15.47:0/1841871917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0319e0 msgr2=0x55751c03afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:11.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.969+0000 7f414cc00640 1 --2- 172.21.15.47:0/1841871917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0319e0 0x55751c03afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.969+0000 7f414cc00640 1 -- 172.21.15.47:0/1841871917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55751bca93f0 con 0x55751c030810 2024-03-21T11:46:11.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.969+0000 7f414cc00640 1 --2- 172.21.15.47:0/1841871917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c03aa80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f413809b300 tx=0x7f41380104d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3573a16a1f06a0c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.969+0000 7f4136000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.969+0000 7f4136000640 1 -- 172.21.15.47:0/1841871917 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138099740 con 0x55751c030810 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.969+0000 7f4136000640 1 -- 172.21.15.47:0/1841871917 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f413809ba60 con 0x55751c030810 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.969+0000 7f415224d740 1 -- 172.21.15.47:0/1841871917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f413809bd00 con 0x55751c030810 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.970+0000 7f4136000640 1 -- 172.21.15.47:0/1841871917 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f413809af60 con 0x55751c030810 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.970+0000 7f415224d740 10 client.21984 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.970+0000 7f415224d740 1 -- 172.21.15.47:0/1841871917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55751bd737d0 con 0x55751c030810 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.970+0000 7f4136000640 1 -- 172.21.15.47:0/1841871917 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f413809bd00 con 0x55751c030810 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.970+0000 7f4136000640 1 -- 172.21.15.47:0/1841871917 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f41380a3b80 con 0x55751c030810 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.970+0000 7f415224d740 10 client.21984 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.970+0000 7f415224d740 10 client.21984 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.970+0000 7f415224d740 20 client.21984 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.970+0000 7f415224d740 1 --2- 172.21.15.47:0/1841871917 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55751c046160 0x55751c048550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.970+0000 7f415224d740 4 client.21984 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.970+0000 7f415224d740 1 -- 172.21.15.47:0/1841871917 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55751bfa2800 con 0x55751c046160 2024-03-21T11:46:11.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.971+0000 7f413fe00640 1 --2- 172.21.15.47:0/1841871917 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55751c046160 0x55751c048550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:11.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.971+0000 7f413fe00640 1 --2- 172.21.15.47:0/1841871917 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55751c046160 0x55751c048550 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7006ccfd5cdeb4ef server_cookie=9b1f2a5bc9b1bd41 in_seq=0 out_seq=0 2024-03-21T11:46:11.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.971+0000 7f4136000640 10 client.21984 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:46:11.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.980+0000 7f4136000640 1 -- 172.21.15.47:0/1841871917 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55751bfa2800 con 0x55751c046160 2024-03-21T11:46:11.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.980+0000 7f4136000640 10 client.21984 handle_command_reply: tid=0 2024-03-21T11:46:11.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.980+0000 7f415224d740 1 client.21984 shutdown 2024-03-21T11:46:11.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.981+0000 7f415224d740 1 -- 172.21.15.47:0/1841871917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 msgr2=0x55751c03aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:11.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.981+0000 7f415224d740 1 --2- 172.21.15.47:0/1841871917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c03aa80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f413809b300 tx=0x7f41380104d0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.981+0000 7f415224d740 1 -- 172.21.15.47:0/1841871917 shutdown_connections 2024-03-21T11:46:11.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.981+0000 7f415224d740 1 --2- 172.21.15.47:0/1841871917 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55751c046160 0x55751c048550 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.981+0000 7f415224d740 1 --2- 172.21.15.47:0/1841871917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0323a0 0x55751c042000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.981+0000 7f415224d740 1 --2- 172.21.15.47:0/1841871917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0319e0 0x55751c03afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.981+0000 7f415224d740 1 --2- 172.21.15.47:0/1841871917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c03aa80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.981+0000 7f415224d740 1 -- 172.21.15.47:0/1841871917 >> 172.21.15.47:0/1841871917 conn(0x55751c02bc00 msgr2=0x55751c02aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.981+0000 7f4136000640 0 client.21984 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:46:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.981+0000 7f415224d740 1 -- 172.21.15.47:0/1841871917 shutdown_connections 2024-03-21T11:46:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.981+0000 7f415224d740 1 -- 172.21.15.47:0/1841871917 wait complete. 2024-03-21T11:46:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:11.982+0000 7f415224d740 20 client.21984 trim_cache size 0 max 16384 2024-03-21T11:46:11.984 INFO:tasks.workunit.client.0.smithi047.stdout: 10 227810 22.00 MB/sec execute 415 sec latency 74.264 ms 2024-03-21T11:46:12.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.001+0000 7f415224d740 1 Processor -- start 2024-03-21T11:46:12.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.002+0000 7f415224d740 1 -- start start 2024-03-21T11:46:12.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.002+0000 7f415224d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c02a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.002+0000 7f415224d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0319e0 0x55751c041fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.002+0000 7f415224d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0323a0 0x55751c02ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.002+0000 7f415224d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41481ac3c0 con 0x55751c030810 2024-03-21T11:46:12.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.002+0000 7f415224d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41481add40 con 0x55751c0319e0 2024-03-21T11:46:12.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.002+0000 7f415224d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41481aca20 con 0x55751c0323a0 2024-03-21T11:46:12.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.003+0000 7f414cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c02a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.003+0000 7f414cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c02a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36066/0 (socket says 172.21.15.47:36066) 2024-03-21T11:46:12.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.003+0000 7f414cc00640 1 -- 172.21.15.47:0/1201460991 learned_addr learned my addr 172.21.15.47:0/1201460991 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:12.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.004+0000 7f414d600640 1 --2- 172.21.15.47:0/1201460991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0323a0 0x55751c02ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.004+0000 7f414cc00640 1 -- 172.21.15.47:0/1201460991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0323a0 msgr2=0x55751c02ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.004+0000 7f414cc00640 1 --2- 172.21.15.47:0/1201460991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0323a0 0x55751c02ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.004+0000 7f414cc00640 1 -- 172.21.15.47:0/1201460991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0319e0 msgr2=0x55751c041fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.004+0000 7f414cc00640 1 --2- 172.21.15.47:0/1201460991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0319e0 0x55751c041fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.004+0000 7f414cc00640 1 -- 172.21.15.47:0/1201460991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55751c031460 con 0x55751c030810 2024-03-21T11:46:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.005+0000 7f414cc00640 1 --2- 172.21.15.47:0/1201460991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c02a620 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f41380109b0 tx=0x7f41380cf560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46740a23bef40d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.005+0000 7f413d200640 1 -- 172.21.15.47:0/1201460991 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413809b140 con 0x55751c030810 2024-03-21T11:46:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.005+0000 7f413d200640 1 -- 172.21.15.47:0/1201460991 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f413809bd00 con 0x55751c030810 2024-03-21T11:46:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.005+0000 7f413d200640 1 -- 172.21.15.47:0/1201460991 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138099740 con 0x55751c030810 2024-03-21T11:46:12.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.006+0000 7f415224d740 1 -- 172.21.15.47:0/1201460991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 msgr2=0x55751c02a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.006+0000 7f415224d740 1 --2- 172.21.15.47:0/1201460991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c02a620 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f41380109b0 tx=0x7f41380cf560 comp rx=0 tx=0).stop 2024-03-21T11:46:12.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.006+0000 7f415224d740 1 -- 172.21.15.47:0/1201460991 shutdown_connections 2024-03-21T11:46:12.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.006+0000 7f415224d740 1 --2- 172.21.15.47:0/1201460991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0323a0 0x55751c02ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.006+0000 7f415224d740 1 --2- 172.21.15.47:0/1201460991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0319e0 0x55751c041fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.006+0000 7f415224d740 1 --2- 172.21.15.47:0/1201460991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c02a620 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.006+0000 7f415224d740 1 -- 172.21.15.47:0/1201460991 >> 172.21.15.47:0/1201460991 conn(0x55751c02bc00 msgr2=0x55751c02f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.007+0000 7f415224d740 1 -- 172.21.15.47:0/1201460991 shutdown_connections 2024-03-21T11:46:12.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.007+0000 7f415224d740 1 -- 172.21.15.47:0/1201460991 wait complete. 2024-03-21T11:46:12.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.007+0000 7f415224d740 1 -- start start 2024-03-21T11:46:12.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.008+0000 7f415224d740 1 Processor -- start 2024-03-21T11:46:12.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.008+0000 7f415224d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c03dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.008+0000 7f415224d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0319e0 0x55751c03e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.008+0000 7f415224d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0323a0 0x55751c041490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.008+0000 7f415224d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41481ad6e0 con 0x55751c030810 2024-03-21T11:46:12.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.008+0000 7f415224d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41481ad080 con 0x55751c0323a0 2024-03-21T11:46:12.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.008+0000 7f415224d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41481aa580 con 0x55751c0319e0 2024-03-21T11:46:12.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.008+0000 7f415224d740 10 client.0 _ll_register_callbacks cb 0x7f41380d7300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:12.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.009+0000 7f414d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0323a0 0x55751c041490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.009+0000 7f414d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0323a0 0x55751c041490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41658/0 (socket says 172.21.15.47:41658) 2024-03-21T11:46:12.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.009+0000 7f414d600640 1 -- 172.21.15.47:0/1402877407 learned_addr learned my addr 172.21.15.47:0/1402877407 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:12.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.009+0000 7f414cc00640 1 --2- 172.21.15.47:0/1402877407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c03dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.009+0000 7f413fe00640 1 --2- 172.21.15.47:0/1402877407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0319e0 0x55751c03e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.009+0000 7f414cc00640 1 -- 172.21.15.47:0/1402877407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0319e0 msgr2=0x55751c03e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.009+0000 7f414cc00640 1 --2- 172.21.15.47:0/1402877407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0319e0 0x55751c03e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.009+0000 7f414cc00640 1 -- 172.21.15.47:0/1402877407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0323a0 msgr2=0x55751c041490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.009+0000 7f414cc00640 1 --2- 172.21.15.47:0/1402877407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0323a0 0x55751c041490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.009+0000 7f414cc00640 1 -- 172.21.15.47:0/1402877407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55751c041a80 con 0x55751c030810 2024-03-21T11:46:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.009+0000 7f414d600640 1 --2- 172.21.15.47:0/1402877407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0323a0 0x55751c041490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.010+0000 7f414cc00640 1 --2- 172.21.15.47:0/1402877407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c03dfd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f41380d7ef0 tx=0x7f413809bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6739ebfc7ad2a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.010+0000 7f4136000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:46:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.010+0000 7f4136000640 1 -- 172.21.15.47:0/1402877407 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41380d2bd0 con 0x55751c030810 2024-03-21T11:46:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.010+0000 7f4136000640 1 -- 172.21.15.47:0/1402877407 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f41380a3b80 con 0x55751c030810 2024-03-21T11:46:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.011+0000 7f415224d740 1 -- 172.21.15.47:0/1402877407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f413809bd00 con 0x55751c030810 2024-03-21T11:46:12.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.011+0000 7f4136000640 1 -- 172.21.15.47:0/1402877407 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41380d2bd0 con 0x55751c030810 2024-03-21T11:46:12.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.012+0000 7f415224d740 10 client.21990 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:12.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.012+0000 7f415224d740 1 -- 172.21.15.47:0/1402877407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55751c0aa710 con 0x55751c030810 2024-03-21T11:46:12.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.013+0000 7f4136000640 1 -- 172.21.15.47:0/1402877407 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f413809bd00 con 0x55751c030810 2024-03-21T11:46:12.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.013+0000 7f4136000640 1 -- 172.21.15.47:0/1402877407 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4138084b00 con 0x55751c030810 2024-03-21T11:46:12.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.013+0000 7f415224d740 10 client.21990 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:12.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.013+0000 7f415224d740 10 client.21990 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:46:12.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.013+0000 7f415224d740 20 client.21990 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:12.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.013+0000 7f415224d740 1 --2- 172.21.15.47:0/1402877407 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55751c0aac30 0x55751c0ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.013+0000 7f415224d740 4 client.21990 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:46:12.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.013+0000 7f415224d740 1 -- 172.21.15.47:0/1402877407 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55751bf9cb50 con 0x55751c0aac30 2024-03-21T11:46:12.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.015+0000 7f413fe00640 1 --2- 172.21.15.47:0/1402877407 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55751c0aac30 0x55751c0ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.017+0000 7f413fe00640 1 --2- 172.21.15.47:0/1402877407 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55751c0aac30 0x55751c0ad050 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7259fb9919fd75b server_cookie=7d5d8cbf9c12b857 in_seq=0 out_seq=0 2024-03-21T11:46:12.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.017+0000 7f4136000640 10 client.21990 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:46:12.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.017+0000 7f4136000640 1 -- 172.21.15.47:0/1402877407 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55751bf9cb50 con 0x55751c0aac30 2024-03-21T11:46:12.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.017+0000 7f4136000640 10 client.21990 handle_command_reply: tid=0 2024-03-21T11:46:12.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.017+0000 7f415224d740 1 client.21990 shutdown 2024-03-21T11:46:12.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.018+0000 7f415224d740 1 -- 172.21.15.47:0/1402877407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 msgr2=0x55751c03dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.018+0000 7f415224d740 1 --2- 172.21.15.47:0/1402877407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c03dfd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f41380d7ef0 tx=0x7f413809bbd0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.018+0000 7f415224d740 1 -- 172.21.15.47:0/1402877407 shutdown_connections 2024-03-21T11:46:12.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.018+0000 7f415224d740 1 --2- 172.21.15.47:0/1402877407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55751c0323a0 0x55751c041490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.018+0000 7f415224d740 1 --2- 172.21.15.47:0/1402877407 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55751c0aac30 0x55751c0ad050 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.018+0000 7f415224d740 1 --2- 172.21.15.47:0/1402877407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55751c0319e0 0x55751c03e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.018+0000 7f415224d740 1 --2- 172.21.15.47:0/1402877407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55751c030810 0x55751c03dfd0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.018+0000 7f415224d740 1 -- 172.21.15.47:0/1402877407 >> 172.21.15.47:0/1402877407 conn(0x55751c02bc00 msgr2=0x55751c034c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.018+0000 7f4136000640 0 client.21990 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:46:12.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.019+0000 7f415224d740 1 -- 172.21.15.47:0/1402877407 shutdown_connections 2024-03-21T11:46:12.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.020+0000 7f415224d740 1 -- 172.21.15.47:0/1402877407 wait complete. 2024-03-21T11:46:12.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.020+0000 7f415224d740 20 client.21990 trim_cache size 0 max 16384 2024-03-21T11:46:12.020 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:46:12.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.021+0000 7f414f600640 1 -- 172.21.15.47:0/2223805910 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f412c0a2f20 msgr2=0x7f412c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.021+0000 7f414f600640 1 --2- 172.21.15.47:0/2223805910 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f412c0a2f20 0x7f412c0a53e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x55751c0318e0 tx=0x7f4140009600 comp rx=0 tx=0).stop 2024-03-21T11:46:12.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.021+0000 7f414f600640 1 -- 172.21.15.47:0/2223805910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41481038c0 msgr2=0x7f4148199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.021+0000 7f414f600640 1 --2- 172.21.15.47:0/2223805910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41481038c0 0x7f4148199c40 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f413800b450 tx=0x7f413800b910 comp rx=0 tx=0).stop 2024-03-21T11:46:12.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.022+0000 7f414f600640 1 -- 172.21.15.47:0/2223805910 shutdown_connections 2024-03-21T11:46:12.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.022+0000 7f414f600640 1 --2- 172.21.15.47:0/2223805910 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f412c0a2f20 0x7f412c0a53e0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.022+0000 7f414f600640 1 --2- 172.21.15.47:0/2223805910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4148105470 0x7f41481a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.022+0000 7f414f600640 1 --2- 172.21.15.47:0/2223805910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4148104ab0 0x7f414819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.022+0000 7f414f600640 1 --2- 172.21.15.47:0/2223805910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41481038c0 0x7f4148199c40 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.022+0000 7f414f600640 1 -- 172.21.15.47:0/2223805910 >> 172.21.15.47:0/2223805910 conn(0x7f41480fee50 msgr2=0x7f4148107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.022+0000 7f414f600640 1 -- 172.21.15.47:0/2223805910 shutdown_connections 2024-03-21T11:46:12.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.023+0000 7f414f600640 1 -- 172.21.15.47:0/2223805910 wait complete. 2024-03-21T11:46:12.045 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:46:12.045 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:46:12.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.320+0000 7fdf01800640 1 -- 172.21.15.47:0/351748927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc0fff70 msgr2=0x7fdefc100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.321+0000 7fdf01800640 1 --2- 172.21.15.47:0/351748927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc0fff70 0x7fdefc100390 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009900 tx=0x7fdeec02bc30 comp rx=0 tx=0).stop 2024-03-21T11:46:12.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.321+0000 7fdf01800640 1 -- 172.21.15.47:0/351748927 shutdown_connections 2024-03-21T11:46:12.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.321+0000 7fdf01800640 1 --2- 172.21.15.47:0/351748927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdefc101270 0x7fdefc1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.321+0000 7fdf01800640 1 --2- 172.21.15.47:0/351748927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdefc1008d0 0x7fdefc100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.321+0000 7fdf01800640 1 --2- 172.21.15.47:0/351748927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc0fff70 0x7fdefc100390 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.321+0000 7fdf01800640 1 -- 172.21.15.47:0/351748927 >> 172.21.15.47:0/351748927 conn(0x7fdefc0fbb30 msgr2=0x7fdefc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.321+0000 7fdf01800640 1 -- 172.21.15.47:0/351748927 shutdown_connections 2024-03-21T11:46:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.321+0000 7fdf01800640 1 -- 172.21.15.47:0/351748927 wait complete. 2024-03-21T11:46:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.322+0000 7fdf01800640 1 Processor -- start 2024-03-21T11:46:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.323+0000 7fdf01800640 1 -- start start 2024-03-21T11:46:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.323+0000 7fdf01800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdefc0fff70 0x7fdefc105af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.323+0000 7fdf01800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdefc1008d0 0x7fdefc106030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.323+0000 7fdf01800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc101270 0x7fdefc102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.323+0000 7fdf01800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc10f560 con 0x7fdefc101270 2024-03-21T11:46:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.323+0000 7fdf01800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc10f3e0 con 0x7fdefc1008d0 2024-03-21T11:46:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.323+0000 7fdf01800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc10f6e0 con 0x7fdefc0fff70 2024-03-21T11:46:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.323+0000 7fdefaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdefc0fff70 0x7fdefc105af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.323+0000 7fdefaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdefc0fff70 0x7fdefc105af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60578/0 (socket says 172.21.15.47:60578) 2024-03-21T11:46:12.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.323+0000 7fdefa000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdefc1008d0 0x7fdefc106030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.323+0000 7fdefaa00640 1 -- 172.21.15.47:0/3209893287 learned_addr learned my addr 172.21.15.47:0/3209893287 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.324+0000 7fdefb400640 1 --2- 172.21.15.47:0/3209893287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc101270 0x7fdefc102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.324+0000 7fdefa000640 1 -- 172.21.15.47:0/3209893287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdefc0fff70 msgr2=0x7fdefc105af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.324+0000 7fdefa000640 1 --2- 172.21.15.47:0/3209893287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdefc0fff70 0x7fdefc105af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.324+0000 7fdefa000640 1 -- 172.21.15.47:0/3209893287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc101270 msgr2=0x7fdefc102660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.324+0000 7fdefa000640 1 --2- 172.21.15.47:0/3209893287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc101270 0x7fdefc102660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.324+0000 7fdefa000640 1 -- 172.21.15.47:0/3209893287 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc102f20 con 0x7fdefc1008d0 2024-03-21T11:46:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.325+0000 7fdefa000640 1 --2- 172.21.15.47:0/3209893287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdefc1008d0 0x7fdefc106030 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fdee8005fd0 tx=0x7fdee800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.325+0000 7fdef3400640 1 -- 172.21.15.47:0/3209893287 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8009c30 con 0x7fdefc1008d0 2024-03-21T11:46:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.325+0000 7fdf01800640 1 -- 172.21.15.47:0/3209893287 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc1a6090 con 0x7fdefc1008d0 2024-03-21T11:46:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.325+0000 7fdf01800640 1 -- 172.21.15.47:0/3209893287 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc1a64f0 con 0x7fdefc1008d0 2024-03-21T11:46:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.326+0000 7fdef3400640 1 -- 172.21.15.47:0/3209893287 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdee8009dd0 con 0x7fdefc1008d0 2024-03-21T11:46:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.326+0000 7fdef3400640 1 -- 172.21.15.47:0/3209893287 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8010460 con 0x7fdefc1008d0 2024-03-21T11:46:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.331+0000 7fdef3400640 1 -- 172.21.15.47:0/3209893287 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdee80035f0 con 0x7fdefc1008d0 2024-03-21T11:46:12.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.333+0000 7fdef3400640 1 --2- 172.21.15.47:0/3209893287 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdee00a3020 0x7fdee00a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.333+0000 7fdef3400640 1 -- 172.21.15.47:0/3209893287 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdee80cf200 con 0x7fdefc1008d0 2024-03-21T11:46:12.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.334+0000 7fdefaa00640 1 --2- 172.21.15.47:0/3209893287 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdee00a3020 0x7fdee00a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.335+0000 7fdefaa00640 1 --2- 172.21.15.47:0/3209893287 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdee00a3020 0x7fdee00a54e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdeec02c8e0 tx=0x7fdeec02c660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:12.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.335+0000 7fdf045bf740 1 Processor -- start 2024-03-21T11:46:12.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.336+0000 7fdf045bf740 1 -- start start 2024-03-21T11:46:12.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.336+0000 7fdf045bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcda810 0x55868bcdac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.336+0000 7fdf045bf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcdb9e0 0x55868bcdbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.336+0000 7fdf045bf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdc3a0 0x55868bce2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.336+0000 7fdf045bf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868bc3f2c0 con 0x55868bcdc3a0 2024-03-21T11:46:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.336+0000 7fdf045bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868bbe7c90 con 0x55868bcda810 2024-03-21T11:46:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.336+0000 7fdf045bf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868bc3b360 con 0x55868bcdb9e0 2024-03-21T11:46:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.336+0000 7fdefa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcdb9e0 0x55868bcdbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.336+0000 7fdefa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcdb9e0 0x55868bcdbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60584/0 (socket says 172.21.15.47:60584) 2024-03-21T11:46:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.337+0000 7fdefa000640 1 -- 172.21.15.47:0/4098385259 learned_addr learned my addr 172.21.15.47:0/4098385259 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.337+0000 7fdefaa00640 1 --2- 172.21.15.47:0/4098385259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcda810 0x55868bcdac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.337+0000 7fdefa000640 1 -- 172.21.15.47:0/4098385259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcda810 msgr2=0x55868bcdac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.337+0000 7fdefb400640 1 --2- 172.21.15.47:0/4098385259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdc3a0 0x55868bce2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.338+0000 7fdefa000640 1 --2- 172.21.15.47:0/4098385259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcda810 0x55868bcdac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.338+0000 7fdefa000640 1 -- 172.21.15.47:0/4098385259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdc3a0 msgr2=0x55868bce2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.338+0000 7fdefa000640 1 --2- 172.21.15.47:0/4098385259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdc3a0 0x55868bce2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.338+0000 7fdefa000640 1 -- 172.21.15.47:0/4098385259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55868bc0c080 con 0x55868bcdb9e0 2024-03-21T11:46:12.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.338+0000 7fdefb400640 1 --2- 172.21.15.47:0/4098385259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdc3a0 0x55868bce2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:12.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.338+0000 7fdefaa00640 1 --2- 172.21.15.47:0/4098385259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcda810 0x55868bcdac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:12.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.340+0000 7fdefa000640 1 --2- 172.21.15.47:0/4098385259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcdb9e0 0x55868bcdbe60 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fdee809b4d0 tx=0x7fdee8097910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e855ba3f071b9f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:12.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.340+0000 7fdee7e00640 1 -- 172.21.15.47:0/4098385259 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8020070 con 0x55868bcdb9e0 2024-03-21T11:46:12.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.340+0000 7fdee7e00640 1 -- 172.21.15.47:0/4098385259 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdee809f690 con 0x55868bcdb9e0 2024-03-21T11:46:12.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.340+0000 7fdee7e00640 1 -- 172.21.15.47:0/4098385259 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee809abf0 con 0x55868bcdb9e0 2024-03-21T11:46:12.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.341+0000 7fdf045bf740 1 -- 172.21.15.47:0/4098385259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcdb9e0 msgr2=0x55868bcdbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.341+0000 7fdf045bf740 1 --2- 172.21.15.47:0/4098385259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcdb9e0 0x55868bcdbe60 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fdee809b4d0 tx=0x7fdee8097910 comp rx=0 tx=0).stop 2024-03-21T11:46:12.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.342+0000 7fdf045bf740 1 -- 172.21.15.47:0/4098385259 shutdown_connections 2024-03-21T11:46:12.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.342+0000 7fdf045bf740 1 --2- 172.21.15.47:0/4098385259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdc3a0 0x55868bce2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.342+0000 7fdf045bf740 1 --2- 172.21.15.47:0/4098385259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcdb9e0 0x55868bcdbe60 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.342+0000 7fdf045bf740 1 --2- 172.21.15.47:0/4098385259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcda810 0x55868bcdac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.342+0000 7fdf045bf740 1 -- 172.21.15.47:0/4098385259 >> 172.21.15.47:0/4098385259 conn(0x55868bcd5c00 msgr2=0x55868bcd6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.342+0000 7fdf045bf740 1 -- 172.21.15.47:0/4098385259 shutdown_connections 2024-03-21T11:46:12.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.342+0000 7fdf045bf740 1 -- 172.21.15.47:0/4098385259 wait complete. 2024-03-21T11:46:12.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.343+0000 7fdf045bf740 1 -- start start 2024-03-21T11:46:12.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.343+0000 7fdf045bf740 1 Processor -- start 2024-03-21T11:46:12.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.344+0000 7fdf045bf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcda810 0x55868bce4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.344+0000 7fdf045bf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdb9e0 0x55868bce4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.344+0000 7fdf045bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 0x55868bcebfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.344+0000 7fdefaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcda810 0x55868bce4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.344+0000 7fdefaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcda810 0x55868bce4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60588/0 (socket says 172.21.15.47:60588) 2024-03-21T11:46:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.345+0000 7fdefa000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdb9e0 0x55868bce4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.345+0000 7fdefa000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdb9e0 0x55868bce4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36110/0 (socket says 172.21.15.47:36110) 2024-03-21T11:46:12.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.345+0000 7fdf045bf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868bce33b0 con 0x55868bcdb9e0 2024-03-21T11:46:12.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.345+0000 7fdf045bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868bbfb0d0 con 0x55868bcdc3a0 2024-03-21T11:46:12.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.345+0000 7fdf045bf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868bce3530 con 0x55868bcda810 2024-03-21T11:46:12.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.345+0000 7fdefaa00640 1 -- 172.21.15.47:0/2182669206 learned_addr learned my addr 172.21.15.47:0/2182669206 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:12.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.345+0000 7fdf045bf740 10 client.0 _ll_register_callbacks cb 0x55868bbffe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.346+0000 7fdefa000640 1 -- 172.21.15.47:0/2182669206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcda810 msgr2=0x55868bce4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.346+0000 7fdefa000640 1 --2- 172.21.15.47:0/2182669206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcda810 0x55868bce4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.346+0000 7fdefa000640 1 -- 172.21.15.47:0/2182669206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 msgr2=0x55868bcebfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.346+0000 7fdefa000640 1 --2- 172.21.15.47:0/2182669206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 0x55868bcebfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.346+0000 7fdefa000640 1 -- 172.21.15.47:0/2182669206 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55868b9533f0 con 0x55868bcdb9e0 2024-03-21T11:46:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.347+0000 7fdefa000640 1 --2- 172.21.15.47:0/2182669206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdb9e0 0x55868bce4f90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdee809b4d0 tx=0x7fdee800df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8bcafca74a9355 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.347+0000 7fdee4c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:46:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.347+0000 7fdee4c00640 1 -- 172.21.15.47:0/2182669206 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee80b3070 con 0x55868bcdb9e0 2024-03-21T11:46:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.347+0000 7fdee4c00640 1 -- 172.21.15.47:0/2182669206 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdee80afda0 con 0x55868bcdb9e0 2024-03-21T11:46:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.347+0000 7fdf045bf740 1 -- 172.21.15.47:0/2182669206 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdee809f690 con 0x55868bcdb9e0 2024-03-21T11:46:12.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.348+0000 7fdee4c00640 1 -- 172.21.15.47:0/2182669206 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdee8020070 con 0x55868bcdb9e0 2024-03-21T11:46:12.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.349+0000 7fdf045bf740 10 client.21999 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:12.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.349+0000 7fdf045bf740 1 -- 172.21.15.47:0/2182669206 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55868ba1d7d0 con 0x55868bcdb9e0 2024-03-21T11:46:12.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.350+0000 7fdee4c00640 1 -- 172.21.15.47:0/2182669206 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdee809f690 con 0x55868bcdb9e0 2024-03-21T11:46:12.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.350+0000 7fdee4c00640 1 -- 172.21.15.47:0/2182669206 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdee8084ad0 con 0x55868bcdb9e0 2024-03-21T11:46:12.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.350+0000 7fdf045bf740 10 client.21999 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:12.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.350+0000 7fdf045bf740 10 client.21999 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:46:12.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.350+0000 7fdf045bf740 20 client.21999 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:12.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.350+0000 7fdf045bf740 1 --2- 172.21.15.47:0/2182669206 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55868bcf00a0 0x55868bcf2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.350+0000 7fdf045bf740 4 client.21999 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:12.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.350+0000 7fdf045bf740 1 -- 172.21.15.47:0/2182669206 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55868bc4c800 con 0x55868bcf00a0 2024-03-21T11:46:12.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.351+0000 7fdefaa00640 1 --2- 172.21.15.47:0/2182669206 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55868bcf00a0 0x55868bcf2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.355+0000 7fdefaa00640 1 --2- 172.21.15.47:0/2182669206 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55868bcf00a0 0x55868bcf2490 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=193c239391971eac server_cookie=7dbd12b7193d18c4 in_seq=0 out_seq=0 2024-03-21T11:46:12.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.355+0000 7fdee4c00640 10 client.21999 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:46:12.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.356+0000 7fdee4c00640 1 -- 172.21.15.47:0/2182669206 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55868bc4c800 con 0x55868bcf00a0 2024-03-21T11:46:12.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.356+0000 7fdee4c00640 10 client.21999 handle_command_reply: tid=0 2024-03-21T11:46:12.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.356+0000 7fdf045bf740 1 client.21999 shutdown 2024-03-21T11:46:12.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.357+0000 7fdf045bf740 1 -- 172.21.15.47:0/2182669206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdb9e0 msgr2=0x55868bce4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.357+0000 7fdf045bf740 1 --2- 172.21.15.47:0/2182669206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdb9e0 0x55868bce4f90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdee809b4d0 tx=0x7fdee800df10 comp rx=0 tx=0).stop 2024-03-21T11:46:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.358+0000 7fdf045bf740 1 -- 172.21.15.47:0/2182669206 shutdown_connections 2024-03-21T11:46:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.358+0000 7fdf045bf740 1 --2- 172.21.15.47:0/2182669206 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55868bcf00a0 0x55868bcf2490 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.358+0000 7fdf045bf740 1 --2- 172.21.15.47:0/2182669206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 0x55868bcebfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.358+0000 7fdf045bf740 1 --2- 172.21.15.47:0/2182669206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdb9e0 0x55868bce4f90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.358+0000 7fdf045bf740 1 --2- 172.21.15.47:0/2182669206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcda810 0x55868bce4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.358+0000 7fdee4c00640 0 client.21999 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:46:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.358+0000 7fdf045bf740 1 -- 172.21.15.47:0/2182669206 >> 172.21.15.47:0/2182669206 conn(0x55868bcd5c00 msgr2=0x55868bcd4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.358+0000 7fdf045bf740 1 -- 172.21.15.47:0/2182669206 shutdown_connections 2024-03-21T11:46:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.358+0000 7fdf045bf740 1 -- 172.21.15.47:0/2182669206 wait complete. 2024-03-21T11:46:12.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.358+0000 7fdf045bf740 20 client.21999 trim_cache size 0 max 16384 2024-03-21T11:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.384+0000 7fdf045bf740 1 Processor -- start 2024-03-21T11:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.384+0000 7fdf045bf740 1 -- start start 2024-03-21T11:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.384+0000 7fdf045bf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcda810 0x55868bcf6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.384+0000 7fdf045bf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdb9e0 0x55868bcebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.384+0000 7fdf045bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 0x55868bcf7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.384+0000 7fdf045bf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc00f950 con 0x55868bcdb9e0 2024-03-21T11:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.384+0000 7fdf045bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc0112d0 con 0x55868bcdc3a0 2024-03-21T11:46:12.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.384+0000 7fdf045bf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc00ffb0 con 0x55868bcda810 2024-03-21T11:46:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.384+0000 7fdefa000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdb9e0 0x55868bcebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.384+0000 7fdefb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 0x55868bcf7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.384+0000 7fdefb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 0x55868bcf7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41688/0 (socket says 172.21.15.47:41688) 2024-03-21T11:46:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.384+0000 7fdefb400640 1 -- 172.21.15.47:0/142448700 learned_addr learned my addr 172.21.15.47:0/142448700 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.385+0000 7fdefaa00640 1 --2- 172.21.15.47:0/142448700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcda810 0x55868bcf6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.385+0000 7fdefa000640 1 -- 172.21.15.47:0/142448700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcda810 msgr2=0x55868bcf6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.385+0000 7fdefa000640 1 --2- 172.21.15.47:0/142448700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcda810 0x55868bcf6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.385+0000 7fdefa000640 1 -- 172.21.15.47:0/142448700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 msgr2=0x55868bcf7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.385+0000 7fdefa000640 1 --2- 172.21.15.47:0/142448700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 0x55868bcf7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.385+0000 7fdefa000640 1 -- 172.21.15.47:0/142448700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55868bce4470 con 0x55868bcdb9e0 2024-03-21T11:46:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.385+0000 7fdefa000640 1 --2- 172.21.15.47:0/142448700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdb9e0 0x55868bcebf90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdee8005d00 tx=0x7fdee809b080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2307250c480680b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.386+0000 7fdee7400640 1 -- 172.21.15.47:0/142448700 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee80b3070 con 0x55868bcdb9e0 2024-03-21T11:46:12.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.386+0000 7fdee7400640 1 -- 172.21.15.47:0/142448700 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdee809bc40 con 0x55868bcdb9e0 2024-03-21T11:46:12.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.386+0000 7fdee7400640 1 -- 172.21.15.47:0/142448700 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee809aca0 con 0x55868bcdb9e0 2024-03-21T11:46:12.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.387+0000 7fdf045bf740 1 -- 172.21.15.47:0/142448700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdb9e0 msgr2=0x55868bcebf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.387+0000 7fdf045bf740 1 --2- 172.21.15.47:0/142448700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdb9e0 0x55868bcebf90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdee8005d00 tx=0x7fdee809b080 comp rx=0 tx=0).stop 2024-03-21T11:46:12.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.387+0000 7fdf045bf740 1 -- 172.21.15.47:0/142448700 shutdown_connections 2024-03-21T11:46:12.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.387+0000 7fdf045bf740 1 --2- 172.21.15.47:0/142448700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 0x55868bcf7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.387+0000 7fdf045bf740 1 --2- 172.21.15.47:0/142448700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcdb9e0 0x55868bcebf90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.387+0000 7fdf045bf740 1 --2- 172.21.15.47:0/142448700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcda810 0x55868bcf6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.387+0000 7fdf045bf740 1 -- 172.21.15.47:0/142448700 >> 172.21.15.47:0/142448700 conn(0x55868bcd5c00 msgr2=0x55868bcd96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.387+0000 7fdf045bf740 1 -- 172.21.15.47:0/142448700 shutdown_connections 2024-03-21T11:46:12.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.387+0000 7fdf045bf740 1 -- 172.21.15.47:0/142448700 wait complete. 2024-03-21T11:46:12.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.388+0000 7fdf045bf740 1 -- start start 2024-03-21T11:46:12.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.388+0000 7fdf045bf740 1 Processor -- start 2024-03-21T11:46:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.389+0000 7fdf045bf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcda810 0x55868bce9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.389+0000 7fdf045bf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcdb9e0 0x55868bcea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.389+0000 7fdf045bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 0x55868bceb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.389+0000 7fdf045bf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc010c70 con 0x55868bcda810 2024-03-21T11:46:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.389+0000 7fdf045bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc010610 con 0x55868bcdc3a0 2024-03-21T11:46:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.389+0000 7fdf045bf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc00db10 con 0x55868bcdb9e0 2024-03-21T11:46:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.389+0000 7fdf045bf740 10 client.0 _ll_register_callbacks cb 0x7fdee8013070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.389+0000 7fdefaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcda810 0x55868bce9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.389+0000 7fdefb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 0x55868bceb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.389+0000 7fdefaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcda810 0x55868bce9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36132/0 (socket says 172.21.15.47:36132) 2024-03-21T11:46:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.389+0000 7fdefaa00640 1 -- 172.21.15.47:0/2799696784 learned_addr learned my addr 172.21.15.47:0/2799696784 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:12.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.389+0000 7fdefb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 0x55868bceb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41702/0 (socket says 172.21.15.47:41702) 2024-03-21T11:46:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.389+0000 7fdefa000640 1 --2- 172.21.15.47:0/2799696784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcdb9e0 0x55868bcea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.390+0000 7fdefb400640 1 -- 172.21.15.47:0/2799696784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcdb9e0 msgr2=0x55868bcea2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.390+0000 7fdefb400640 1 --2- 172.21.15.47:0/2799696784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcdb9e0 0x55868bcea2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.390+0000 7fdefb400640 1 -- 172.21.15.47:0/2799696784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcda810 msgr2=0x55868bce9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.390+0000 7fdefb400640 1 --2- 172.21.15.47:0/2799696784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcda810 0x55868bce9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.390+0000 7fdefb400640 1 -- 172.21.15.47:0/2799696784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55868bceb850 con 0x55868bcdc3a0 2024-03-21T11:46:12.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.390+0000 7fdefa000640 1 --2- 172.21.15.47:0/2799696784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcdb9e0 0x55868bcea2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:12.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.393+0000 7fdefb400640 1 --2- 172.21.15.47:0/2799696784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 0x55868bceb260 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fdef400aeb0 tx=0x7fdef4008610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e5641d1089dac3e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:12.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.393+0000 7fdee4c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:12.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.394+0000 7fdee4c00640 1 -- 172.21.15.47:0/2799696784 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4009c50 con 0x55868bcdc3a0 2024-03-21T11:46:12.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.394+0000 7fdee4c00640 1 -- 172.21.15.47:0/2799696784 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdef4009df0 con 0x55868bcdc3a0 2024-03-21T11:46:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.394+0000 7fdf045bf740 1 -- 172.21.15.47:0/2799696784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdee809bc40 con 0x55868bcdc3a0 2024-03-21T11:46:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.395+0000 7fdee4c00640 1 -- 172.21.15.47:0/2799696784 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdef4017480 con 0x55868bcdc3a0 2024-03-21T11:46:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.395+0000 7fdf045bf740 10 client.32789 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.395+0000 7fdf045bf740 1 -- 172.21.15.47:0/2799696784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55868bd4beb0 con 0x55868bcdc3a0 2024-03-21T11:46:12.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.396+0000 7fdee4c00640 1 -- 172.21.15.47:0/2799696784 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdee809bc40 con 0x55868bcdc3a0 2024-03-21T11:46:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.398+0000 7fdee4c00640 1 -- 172.21.15.47:0/2799696784 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdef4016900 con 0x55868bcdc3a0 2024-03-21T11:46:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.398+0000 7fdf045bf740 10 client.32789 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.398+0000 7fdf045bf740 10 client.32789 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:46:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.398+0000 7fdf045bf740 20 client.32789 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.398+0000 7fdf045bf740 1 --2- 172.21.15.47:0/2799696784 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55868bd4c760 0x55868bd4ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.398+0000 7fdf045bf740 4 client.32789 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:46:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.398+0000 7fdf045bf740 1 -- 172.21.15.47:0/2799696784 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55868bc46b50 con 0x55868bd4c760 2024-03-21T11:46:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.401+0000 7fdefaa00640 1 --2- 172.21.15.47:0/2799696784 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55868bd4c760 0x55868bd4ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.404+0000 7fdefaa00640 1 --2- 172.21.15.47:0/2799696784 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55868bd4c760 0x55868bd4ebe0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4371104d5a076908 server_cookie=5fd213cf32de515e in_seq=0 out_seq=0 2024-03-21T11:46:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.404+0000 7fdee4c00640 10 client.32789 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:46:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.405+0000 7fdee4c00640 1 -- 172.21.15.47:0/2799696784 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55868bc46b50 con 0x55868bd4c760 2024-03-21T11:46:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.405+0000 7fdee4c00640 10 client.32789 handle_command_reply: tid=0 2024-03-21T11:46:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.405+0000 7fdf045bf740 1 client.32789 shutdown 2024-03-21T11:46:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.406+0000 7fdf045bf740 1 -- 172.21.15.47:0/2799696784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 msgr2=0x55868bceb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.406+0000 7fdf045bf740 1 --2- 172.21.15.47:0/2799696784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 0x55868bceb260 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fdef400aeb0 tx=0x7fdef4008610 comp rx=0 tx=0).stop 2024-03-21T11:46:12.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.406+0000 7fdf045bf740 1 -- 172.21.15.47:0/2799696784 shutdown_connections 2024-03-21T11:46:12.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.406+0000 7fdf045bf740 1 --2- 172.21.15.47:0/2799696784 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55868bd4c760 0x55868bd4ebe0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.406+0000 7fdf045bf740 1 --2- 172.21.15.47:0/2799696784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55868bcdc3a0 0x55868bceb260 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.406+0000 7fdf045bf740 1 --2- 172.21.15.47:0/2799696784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868bcdb9e0 0x55868bcea2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.406+0000 7fdee4c00640 0 client.32789 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:46:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.406+0000 7fdf045bf740 1 --2- 172.21.15.47:0/2799696784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55868bcda810 0x55868bce9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.406+0000 7fdf045bf740 1 -- 172.21.15.47:0/2799696784 >> 172.21.15.47:0/2799696784 conn(0x55868bcd5c00 msgr2=0x55868bcdec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.407+0000 7fdf045bf740 1 -- 172.21.15.47:0/2799696784 shutdown_connections 2024-03-21T11:46:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.407+0000 7fdf045bf740 1 -- 172.21.15.47:0/2799696784 wait complete. 2024-03-21T11:46:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.407+0000 7fdf045bf740 20 client.32789 trim_cache size 0 max 16384 2024-03-21T11:46:12.409 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:46:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.408+0000 7fdf01800640 1 -- 172.21.15.47:0/3209893287 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdee00a3020 msgr2=0x7fdee00a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.408+0000 7fdf01800640 1 --2- 172.21.15.47:0/3209893287 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdee00a3020 0x7fdee00a54e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdeec02c8e0 tx=0x7fdeec02c660 comp rx=0 tx=0).stop 2024-03-21T11:46:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.408+0000 7fdf01800640 1 -- 172.21.15.47:0/3209893287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdefc1008d0 msgr2=0x7fdefc106030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.408+0000 7fdf01800640 1 --2- 172.21.15.47:0/3209893287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdefc1008d0 0x7fdefc106030 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fdee8005fd0 tx=0x7fdee800dde0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.408+0000 7fdf01800640 1 -- 172.21.15.47:0/3209893287 shutdown_connections 2024-03-21T11:46:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.408+0000 7fdf01800640 1 --2- 172.21.15.47:0/3209893287 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdee00a3020 0x7fdee00a54e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.408+0000 7fdf01800640 1 --2- 172.21.15.47:0/3209893287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdefc101270 0x7fdefc102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.408+0000 7fdf01800640 1 --2- 172.21.15.47:0/3209893287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdefc1008d0 0x7fdefc106030 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.408+0000 7fdf01800640 1 --2- 172.21.15.47:0/3209893287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdefc0fff70 0x7fdefc105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.408+0000 7fdf01800640 1 -- 172.21.15.47:0/3209893287 >> 172.21.15.47:0/3209893287 conn(0x7fdefc0fbb30 msgr2=0x7fdefc0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.409+0000 7fdf01800640 1 -- 172.21.15.47:0/3209893287 shutdown_connections 2024-03-21T11:46:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.409+0000 7fdf01800640 1 -- 172.21.15.47:0/3209893287 wait complete. 2024-03-21T11:46:12.441 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:46:12.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:46:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.679+0000 7f8aa7800640 1 -- 172.21.15.47:0/2242441830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa0105490 msgr2=0x7f8aa010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.679+0000 7f8aa7800640 1 --2- 172.21.15.47:0/2242441830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa0105490 0x7f8aa010bd20 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009ca0 tx=0x7f8a9802c100 comp rx=0 tx=0).stop 2024-03-21T11:46:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.679+0000 7f8aa7800640 1 -- 172.21.15.47:0/2242441830 shutdown_connections 2024-03-21T11:46:12.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.679+0000 7f8aa7800640 1 --2- 172.21.15.47:0/2242441830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa0105490 0x7f8aa010bd20 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.679+0000 7f8aa7800640 1 --2- 172.21.15.47:0/2242441830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0104ad0 0x7f8aa0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.679+0000 7f8aa7800640 1 --2- 172.21.15.47:0/2242441830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8aa01038e0 0x7f8aa0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.679+0000 7f8aa7800640 1 -- 172.21.15.47:0/2242441830 >> 172.21.15.47:0/2242441830 conn(0x7f8aa00fee50 msgr2=0x7f8aa0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.679+0000 7f8aa7800640 1 -- 172.21.15.47:0/2242441830 shutdown_connections 2024-03-21T11:46:12.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.679+0000 7f8aa7800640 1 -- 172.21.15.47:0/2242441830 wait complete. 2024-03-21T11:46:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.680+0000 7f8aa7800640 1 Processor -- start 2024-03-21T11:46:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.680+0000 7f8aa7800640 1 -- start start 2024-03-21T11:46:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa7800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa01038e0 0x7f8aa0199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa7800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0104ad0 0x7f8aa019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa7800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8aa0105490 0x7f8aa01a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa7800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa010f560 con 0x7f8aa0104ad0 2024-03-21T11:46:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa7800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa010f3e0 con 0x7f8aa01038e0 2024-03-21T11:46:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa7800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa010f6e0 con 0x7f8aa0105490 2024-03-21T11:46:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa01038e0 0x7f8aa0199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0104ad0 0x7f8aa019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa01038e0 0x7f8aa0199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41720/0 (socket says 172.21.15.47:41720) 2024-03-21T11:46:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa4e00640 1 -- 172.21.15.47:0/2074790468 learned_addr learned my addr 172.21.15.47:0/2074790468 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa5800640 1 --2- 172.21.15.47:0/2074790468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8aa0105490 0x7f8aa01a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa4e00640 1 -- 172.21.15.47:0/2074790468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8aa0105490 msgr2=0x7f8aa01a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa4e00640 1 --2- 172.21.15.47:0/2074790468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8aa0105490 0x7f8aa01a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa4e00640 1 -- 172.21.15.47:0/2074790468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0104ad0 msgr2=0x7f8aa019a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa4e00640 1 --2- 172.21.15.47:0/2074790468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0104ad0 0x7f8aa019a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.681+0000 7f8aa4e00640 1 -- 172.21.15.47:0/2074790468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa01a1840 con 0x7f8aa01038e0 2024-03-21T11:46:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.682+0000 7f8aa4e00640 1 --2- 172.21.15.47:0/2074790468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa01038e0 0x7f8aa0199c30 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400e9c0 tx=0x7f8a9400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.682+0000 7f8a9d600640 1 -- 172.21.15.47:0/2074790468 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9400cd80 con 0x7f8aa01038e0 2024-03-21T11:46:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.682+0000 7f8a9d600640 1 -- 172.21.15.47:0/2074790468 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a940035f0 con 0x7f8aa01038e0 2024-03-21T11:46:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.682+0000 7f8aa7800640 1 -- 172.21.15.47:0/2074790468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa01a1b30 con 0x7f8aa01038e0 2024-03-21T11:46:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.682+0000 7f8a9d600640 1 -- 172.21.15.47:0/2074790468 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94010690 con 0x7f8aa01038e0 2024-03-21T11:46:12.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.682+0000 7f8aa7800640 1 -- 172.21.15.47:0/2074790468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa01a2030 con 0x7f8aa01038e0 2024-03-21T11:46:12.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.685+0000 7f8a9d600640 1 -- 172.21.15.47:0/2074790468 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8a940037a0 con 0x7f8aa01038e0 2024-03-21T11:46:12.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.686+0000 7f8aaa40c740 1 Processor -- start 2024-03-21T11:46:12.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.686+0000 7f8aaa40c740 1 -- start start 2024-03-21T11:46:12.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.686+0000 7f8aaa40c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf2810 0x563af0cf2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.686+0000 7f8aaa40c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf39e0 0x563af0cf3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.686+0000 7f8aaa40c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf43a0 0x563af0cfac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.686+0000 7f8aaa40c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af0c572c0 con 0x563af0cf39e0 2024-03-21T11:46:12.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.686+0000 7f8aaa40c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af0bffc90 con 0x563af0cf43a0 2024-03-21T11:46:12.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.686+0000 7f8aaa40c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af0c53360 con 0x563af0cf2810 2024-03-21T11:46:12.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.686+0000 7f8aa5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf2810 0x563af0cf2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.686+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf43a0 0x563af0cfac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.686+0000 7f8aa5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf2810 0x563af0cf2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60632/0 (socket says 172.21.15.47:60632) 2024-03-21T11:46:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.686+0000 7f8aa5800640 1 -- 172.21.15.47:0/1404010678 learned_addr learned my addr 172.21.15.47:0/1404010678 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.687+0000 7f8aa4e00640 1 --2- 172.21.15.47:0/1404010678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf39e0 0x563af0cf3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.687+0000 7f8a9fe00640 1 -- 172.21.15.47:0/1404010678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf2810 msgr2=0x563af0cf2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.687+0000 7f8a9fe00640 1 --2- 172.21.15.47:0/1404010678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf2810 0x563af0cf2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.687+0000 7f8a9fe00640 1 -- 172.21.15.47:0/1404010678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf39e0 msgr2=0x563af0cf3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.687+0000 7f8a9fe00640 1 --2- 172.21.15.47:0/1404010678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf39e0 0x563af0cf3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.687+0000 7f8a9fe00640 1 -- 172.21.15.47:0/1404010678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563af0c24080 con 0x563af0cf43a0 2024-03-21T11:46:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.687+0000 7f8a9fe00640 1 --2- 172.21.15.47:0/1404010678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf43a0 0x563af0cfac30 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009c20 tx=0x7f8a8c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=342216908367300b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.687+0000 7f8a9d600640 1 --2- 172.21.15.47:0/2074790468 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8a880a2f20 0x7f8a880a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.688+0000 7f8a92000640 1 -- 172.21.15.47:0/1404010678 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c01e070 con 0x563af0cf43a0 2024-03-21T11:46:12.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.688+0000 7f8a92000640 1 -- 172.21.15.47:0/1404010678 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a8c009dd0 con 0x563af0cf43a0 2024-03-21T11:46:12.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.688+0000 7f8a92000640 1 -- 172.21.15.47:0/1404010678 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c018870 con 0x563af0cf43a0 2024-03-21T11:46:12.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.688+0000 7f8a9d600640 1 -- 172.21.15.47:0/2074790468 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8a940cfbd0 con 0x7f8aa01038e0 2024-03-21T11:46:12.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.688+0000 7f8aa5800640 1 --2- 172.21.15.47:0/2074790468 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8a880a2f20 0x7f8a880a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.688+0000 7f8aaa40c740 1 -- 172.21.15.47:0/1404010678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf43a0 msgr2=0x563af0cfac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.688+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/1404010678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf43a0 0x563af0cfac30 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009c20 tx=0x7f8a8c011810 comp rx=0 tx=0).stop 2024-03-21T11:46:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.689+0000 7f8aaa40c740 1 -- 172.21.15.47:0/1404010678 shutdown_connections 2024-03-21T11:46:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.689+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/1404010678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf43a0 0x563af0cfac30 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.689+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/1404010678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf39e0 0x563af0cf3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.689+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/1404010678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf2810 0x563af0cf2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.689+0000 7f8aaa40c740 1 -- 172.21.15.47:0/1404010678 >> 172.21.15.47:0/1404010678 conn(0x563af0cedc00 msgr2=0x563af0cee000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.689+0000 7f8aa5800640 1 --2- 172.21.15.47:0/2074790468 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8a880a2f20 0x7f8a880a53e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8a98006010 tx=0x7f8a98032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.689+0000 7f8aaa40c740 1 -- 172.21.15.47:0/1404010678 shutdown_connections 2024-03-21T11:46:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.689+0000 7f8aaa40c740 1 -- 172.21.15.47:0/1404010678 wait complete. 2024-03-21T11:46:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.689+0000 7f8aaa40c740 1 -- start start 2024-03-21T11:46:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.689+0000 7f8aaa40c740 1 Processor -- start 2024-03-21T11:46:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aaa40c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf2810 0x563af0cfca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aaa40c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf39e0 0x563af0cfcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aaa40c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf43a0 0x563af0d04000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aaa40c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af0cfb3b0 con 0x563af0cf43a0 2024-03-21T11:46:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aaa40c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af0c130d0 con 0x563af0cf2810 2024-03-21T11:46:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aaa40c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af0cfb530 con 0x563af0cf39e0 2024-03-21T11:46:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aaa40c740 10 client.0 _ll_register_callbacks cb 0x563af0c17e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aa4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf2810 0x563af0cfca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aa4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf2810 0x563af0cfca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41740/0 (socket says 172.21.15.47:41740) 2024-03-21T11:46:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aa4e00640 1 -- 172.21.15.47:0/1916817850 learned_addr learned my addr 172.21.15.47:0/1916817850 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8a9fe00640 1 --2- 172.21.15.47:0/1916817850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf39e0 0x563af0cfcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aa5800640 1 --2- 172.21.15.47:0/1916817850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf43a0 0x563af0d04000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aa4e00640 1 -- 172.21.15.47:0/1916817850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf39e0 msgr2=0x563af0cfcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aa4e00640 1 --2- 172.21.15.47:0/1916817850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf39e0 0x563af0cfcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aa4e00640 1 -- 172.21.15.47:0/1916817850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf43a0 msgr2=0x563af0d04000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.690+0000 7f8aa4e00640 1 --2- 172.21.15.47:0/1916817850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf43a0 0x563af0d04000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.691+0000 7f8aa4e00640 1 -- 172.21.15.47:0/1916817850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563af096b3f0 con 0x563af0cf2810 2024-03-21T11:46:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.691+0000 7f8a9fe00640 1 --2- 172.21.15.47:0/1916817850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf39e0 0x563af0cfcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.691+0000 7f8aa4e00640 1 --2- 172.21.15.47:0/1916817850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf2810 0x563af0cfca80 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f8a9409bc10 tx=0x7f8a940981e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b28119df29915bc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.691+0000 7f8a86a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.691+0000 7f8a86a00640 1 -- 172.21.15.47:0/1916817850 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9409a050 con 0x563af0cf2810 2024-03-21T11:46:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.691+0000 7f8aaa40c740 1 -- 172.21.15.47:0/1916817850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a8c009dd0 con 0x563af0cf2810 2024-03-21T11:46:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.691+0000 7f8a86a00640 1 -- 172.21.15.47:0/1916817850 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8a94003b10 con 0x563af0cf2810 2024-03-21T11:46:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.691+0000 7f8a86a00640 1 -- 172.21.15.47:0/1916817850 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a9409c610 con 0x563af0cf2810 2024-03-21T11:46:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.691+0000 7f8aaa40c740 10 client.32801 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.691+0000 7f8aaa40c740 1 -- 172.21.15.47:0/1916817850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563af0a357d0 con 0x563af0cf2810 2024-03-21T11:46:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.692+0000 7f8a86a00640 1 -- 172.21.15.47:0/1916817850 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a8c009dd0 con 0x563af0cf2810 2024-03-21T11:46:12.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.692+0000 7f8a86a00640 1 -- 172.21.15.47:0/1916817850 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8a9409b360 con 0x563af0cf2810 2024-03-21T11:46:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.692+0000 7f8aaa40c740 10 client.32801 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.692+0000 7f8aaa40c740 10 client.32801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.692+0000 7f8aaa40c740 20 client.32801 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.692+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/1916817850 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563af0d084e0 0x563af0d0a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.692+0000 7f8aaa40c740 4 client.32801 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.692+0000 7f8aaa40c740 1 -- 172.21.15.47:0/1916817850 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563af0c64800 con 0x563af0d084e0 2024-03-21T11:46:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.693+0000 7f8a9fe00640 1 --2- 172.21.15.47:0/1916817850 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563af0d084e0 0x563af0d0a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.693+0000 7f8a9fe00640 1 --2- 172.21.15.47:0/1916817850 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563af0d084e0 0x563af0d0a930 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feacc84888731779 server_cookie=98d82f944ed926c0 in_seq=0 out_seq=0 2024-03-21T11:46:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.693+0000 7f8a86a00640 10 client.32801 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8a86a00640 1 -- 172.21.15.47:0/1916817850 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563af0c64800 con 0x563af0d084e0 2024-03-21T11:46:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8a86a00640 10 client.32801 handle_command_reply: tid=0 2024-03-21T11:46:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8aaa40c740 1 client.32801 shutdown 2024-03-21T11:46:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8aaa40c740 1 -- 172.21.15.47:0/1916817850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf2810 msgr2=0x563af0cfca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/1916817850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf2810 0x563af0cfca80 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f8a9409bc10 tx=0x7f8a940981e0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8aaa40c740 1 -- 172.21.15.47:0/1916817850 shutdown_connections 2024-03-21T11:46:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/1916817850 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563af0d084e0 0x563af0d0a930 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8a86a00640 0 client.32801 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/1916817850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf43a0 0x563af0d04000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/1916817850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf39e0 0x563af0cfcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/1916817850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf2810 0x563af0cfca80 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8aaa40c740 1 -- 172.21.15.47:0/1916817850 >> 172.21.15.47:0/1916817850 conn(0x563af0cedc00 msgr2=0x563af0ceceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8aaa40c740 1 -- 172.21.15.47:0/1916817850 shutdown_connections 2024-03-21T11:46:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8aaa40c740 1 -- 172.21.15.47:0/1916817850 wait complete. 2024-03-21T11:46:12.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.695+0000 7f8aaa40c740 20 client.32801 trim_cache size 0 max 16384 2024-03-21T11:46:12.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.711+0000 7f8aaa40c740 1 Processor -- start 2024-03-21T11:46:12.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.711+0000 7f8aaa40c740 1 -- start start 2024-03-21T11:46:12.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.711+0000 7f8aaa40c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf2810 0x563af0cf6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.711+0000 7f8aaa40c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf39e0 0x563af0d03f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.712+0000 7f8aaa40c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf43a0 0x563af0cf7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.712+0000 7f8aaa40c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa01ac2c0 con 0x563af0cf39e0 2024-03-21T11:46:12.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.712+0000 7f8aaa40c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa01adc40 con 0x563af0cf2810 2024-03-21T11:46:12.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.712+0000 7f8aaa40c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa01ac920 con 0x563af0cf43a0 2024-03-21T11:46:12.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.712+0000 7f8aa4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf2810 0x563af0cf6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.712+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf39e0 0x563af0d03f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.712+0000 7f8aa4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf2810 0x563af0cf6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41752/0 (socket says 172.21.15.47:41752) 2024-03-21T11:46:12.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.712+0000 7f8aa4e00640 1 -- 172.21.15.47:0/3874835829 learned_addr learned my addr 172.21.15.47:0/3874835829 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:12.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.712+0000 7f8aa5800640 1 --2- 172.21.15.47:0/3874835829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf43a0 0x563af0cf7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.712+0000 7f8aa4e00640 1 -- 172.21.15.47:0/3874835829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf43a0 msgr2=0x563af0cf7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.712+0000 7f8aa4e00640 1 --2- 172.21.15.47:0/3874835829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf43a0 0x563af0cf7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.712+0000 7f8aa4e00640 1 -- 172.21.15.47:0/3874835829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf39e0 msgr2=0x563af0d03f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.712+0000 7f8aa4e00640 1 --2- 172.21.15.47:0/3874835829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf39e0 0x563af0d03f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.712+0000 7f8aa4e00640 1 -- 172.21.15.47:0/3874835829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563af0cee970 con 0x563af0cf2810 2024-03-21T11:46:12.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.713+0000 7f8aa4e00640 1 --2- 172.21.15.47:0/3874835829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf2810 0x563af0cf6c10 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f8a940058c0 tx=0x7f8a94098250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=493dd084cd936586 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:12.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.713+0000 7f8a91600640 1 -- 172.21.15.47:0/3874835829 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9409a050 con 0x563af0cf2810 2024-03-21T11:46:12.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.713+0000 7f8a91600640 1 -- 172.21.15.47:0/3874835829 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a9409bf10 con 0x563af0cf2810 2024-03-21T11:46:12.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.713+0000 7f8a91600640 1 -- 172.21.15.47:0/3874835829 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9409b360 con 0x563af0cf2810 2024-03-21T11:46:12.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.714+0000 7f8aaa40c740 1 -- 172.21.15.47:0/3874835829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf2810 msgr2=0x563af0cf6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.714+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/3874835829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf2810 0x563af0cf6c10 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f8a940058c0 tx=0x7f8a94098250 comp rx=0 tx=0).stop 2024-03-21T11:46:12.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.714+0000 7f8aaa40c740 1 -- 172.21.15.47:0/3874835829 shutdown_connections 2024-03-21T11:46:12.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.714+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/3874835829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf43a0 0x563af0cf7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.714+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/3874835829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf39e0 0x563af0d03f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.714+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/3874835829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf2810 0x563af0cf6c10 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.714+0000 7f8aaa40c740 1 -- 172.21.15.47:0/3874835829 >> 172.21.15.47:0/3874835829 conn(0x563af0cedc00 msgr2=0x563af0cf16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.715+0000 7f8aaa40c740 1 -- 172.21.15.47:0/3874835829 shutdown_connections 2024-03-21T11:46:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.715+0000 7f8aaa40c740 1 -- 172.21.15.47:0/3874835829 wait complete. 2024-03-21T11:46:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.715+0000 7f8aaa40c740 1 -- start start 2024-03-21T11:46:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.715+0000 7f8aaa40c740 1 Processor -- start 2024-03-21T11:46:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.716+0000 7f8aaa40c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf2810 0x563af0d01fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.716+0000 7f8aaa40c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf39e0 0x563af0d02510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.716+0000 7f8aaa40c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf43a0 0x563af0d03520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.716+0000 7f8aaa40c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa01ad5e0 con 0x563af0cf2810 2024-03-21T11:46:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.716+0000 7f8aaa40c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa01acf80 con 0x563af0cf43a0 2024-03-21T11:46:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.716+0000 7f8aaa40c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa01aa480 con 0x563af0cf39e0 2024-03-21T11:46:12.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.716+0000 7f8aaa40c740 10 client.0 _ll_register_callbacks cb 0x7f8a940d3300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:12.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.716+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf39e0 0x563af0d02510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.716+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf39e0 0x563af0d02510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60656/0 (socket says 172.21.15.47:60656) 2024-03-21T11:46:12.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.716+0000 7f8a9fe00640 1 -- 172.21.15.47:0/3201407947 learned_addr learned my addr 172.21.15.47:0/3201407947 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:12.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.716+0000 7f8aa5800640 1 --2- 172.21.15.47:0/3201407947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf43a0 0x563af0d03520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.717+0000 7f8a9fe00640 1 -- 172.21.15.47:0/3201407947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf43a0 msgr2=0x563af0d03520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.717+0000 7f8a9fe00640 1 --2- 172.21.15.47:0/3201407947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf43a0 0x563af0d03520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.717+0000 7f8a9fe00640 1 -- 172.21.15.47:0/3201407947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf2810 msgr2=0x563af0d01fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:12.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.717+0000 7f8a9fe00640 1 --2- 172.21.15.47:0/3201407947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf2810 0x563af0d01fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.717+0000 7f8a9fe00640 1 -- 172.21.15.47:0/3201407947 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563af0d03b10 con 0x563af0cf39e0 2024-03-21T11:46:12.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.717+0000 7f8a9fe00640 1 --2- 172.21.15.47:0/3201407947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf39e0 0x563af0d02510 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0099b0 tx=0x7f8a8c005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c6fa497797fcb2c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:12.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.717+0000 7f8a86a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:12.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.717+0000 7f8a86a00640 1 -- 172.21.15.47:0/3201407947 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c01e040 con 0x563af0cf39e0 2024-03-21T11:46:12.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.718+0000 7f8a86a00640 1 -- 172.21.15.47:0/3201407947 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8a8c016040 con 0x563af0cf39e0 2024-03-21T11:46:12.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.718+0000 7f8aaa40c740 1 -- 172.21.15.47:0/3201407947 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a9409bf10 con 0x563af0cf39e0 2024-03-21T11:46:12.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.718+0000 7f8a86a00640 1 -- 172.21.15.47:0/3201407947 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a8c037050 con 0x563af0cf39e0 2024-03-21T11:46:12.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.718+0000 7f8aaa40c740 10 client.32791 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:12.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.718+0000 7f8aaa40c740 1 -- 172.21.15.47:0/3201407947 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563af0d6ce90 con 0x563af0cf39e0 2024-03-21T11:46:12.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.718+0000 7f8a86a00640 1 -- 172.21.15.47:0/3201407947 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a9409bf10 con 0x563af0cf39e0 2024-03-21T11:46:12.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.719+0000 7f8a86a00640 1 -- 172.21.15.47:0/3201407947 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8a8c005040 con 0x563af0cf39e0 2024-03-21T11:46:12.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.719+0000 7f8aaa40c740 10 client.32791 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:12.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.719+0000 7f8aaa40c740 10 client.32791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:12.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.719+0000 7f8aaa40c740 20 client.32791 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:12.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.719+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/3201407947 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563af0d6d7c0 0x563af0d6fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:12.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.719+0000 7f8aaa40c740 4 client.32791 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:46:12.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.719+0000 7f8aaa40c740 1 -- 172.21.15.47:0/3201407947 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563af0c5eb50 con 0x563af0d6d7c0 2024-03-21T11:46:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.719+0000 7f8aa4e00640 1 --2- 172.21.15.47:0/3201407947 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563af0d6d7c0 0x563af0d6fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.719+0000 7f8aa4e00640 1 --2- 172.21.15.47:0/3201407947 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563af0d6d7c0 0x563af0d6fbe0 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e52067b5d0eb19 server_cookie=75540a586b9807ad in_seq=0 out_seq=0 2024-03-21T11:46:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.719+0000 7f8a86a00640 10 client.32791 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:12.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.723+0000 7f8a86a00640 1 -- 172.21.15.47:0/3201407947 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563af0c5eb50 con 0x563af0d6d7c0 2024-03-21T11:46:12.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.723+0000 7f8a86a00640 10 client.32791 handle_command_reply: tid=0 2024-03-21T11:46:12.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.723+0000 7f8aaa40c740 1 client.32791 shutdown 2024-03-21T11:46:12.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.723+0000 7f8aaa40c740 1 -- 172.21.15.47:0/3201407947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf39e0 msgr2=0x563af0d02510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.723+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/3201407947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf39e0 0x563af0d02510 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0099b0 tx=0x7f8a8c005b00 comp rx=0 tx=0).stop 2024-03-21T11:46:12.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.723+0000 7f8aaa40c740 1 -- 172.21.15.47:0/3201407947 shutdown_connections 2024-03-21T11:46:12.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.723+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/3201407947 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563af0d6d7c0 0x563af0d6fbe0 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.723+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/3201407947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563af0cf43a0 0x563af0d03520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.723+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/3201407947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563af0cf39e0 0x563af0d02510 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.723+0000 7f8aaa40c740 1 --2- 172.21.15.47:0/3201407947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563af0cf2810 0x563af0d01fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.723+0000 7f8aaa40c740 1 -- 172.21.15.47:0/3201407947 >> 172.21.15.47:0/3201407947 conn(0x563af0cedc00 msgr2=0x563af0cec620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.723+0000 7f8a86a00640 0 client.32791 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:12.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.724+0000 7f8aaa40c740 1 -- 172.21.15.47:0/3201407947 shutdown_connections 2024-03-21T11:46:12.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.724+0000 7f8aaa40c740 1 -- 172.21.15.47:0/3201407947 wait complete. 2024-03-21T11:46:12.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.724+0000 7f8aaa40c740 20 client.32791 trim_cache size 0 max 16384 2024-03-21T11:46:12.725 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:46:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.725+0000 7f8aa7800640 1 -- 172.21.15.47:0/2074790468 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8a880a2f20 msgr2=0x7f8a880a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.725+0000 7f8aa7800640 1 --2- 172.21.15.47:0/2074790468 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8a880a2f20 0x7f8a880a53e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8a98006010 tx=0x7f8a98032590 comp rx=0 tx=0).stop 2024-03-21T11:46:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.725+0000 7f8aa7800640 1 -- 172.21.15.47:0/2074790468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa01038e0 msgr2=0x7f8aa0199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.725+0000 7f8aa7800640 1 --2- 172.21.15.47:0/2074790468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa01038e0 0x7f8aa0199c30 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400e9c0 tx=0x7f8a9400ee80 comp rx=0 tx=0).stop 2024-03-21T11:46:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.725+0000 7f8aa7800640 1 -- 172.21.15.47:0/2074790468 shutdown_connections 2024-03-21T11:46:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.725+0000 7f8aa7800640 1 --2- 172.21.15.47:0/2074790468 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8a880a2f20 0x7f8a880a53e0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.725+0000 7f8aa7800640 1 --2- 172.21.15.47:0/2074790468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8aa0105490 0x7f8aa01a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.725+0000 7f8aa7800640 1 --2- 172.21.15.47:0/2074790468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8aa0104ad0 0x7f8aa019a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.725+0000 7f8aa7800640 1 --2- 172.21.15.47:0/2074790468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa01038e0 0x7f8aa0199c30 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.725+0000 7f8aa7800640 1 -- 172.21.15.47:0/2074790468 >> 172.21.15.47:0/2074790468 conn(0x7f8aa00fee50 msgr2=0x7f8aa0107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.725+0000 7f8aa7800640 1 -- 172.21.15.47:0/2074790468 shutdown_connections 2024-03-21T11:46:12.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:12.725+0000 7f8aa7800640 1 -- 172.21.15.47:0/2074790468 wait complete. 2024-03-21T11:46:12.985 INFO:tasks.workunit.client.0.smithi047.stdout: 10 228332 22.03 MB/sec execute 416 sec latency 58.656 ms 2024-03-21T11:46:13.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:13 smithi047 ceph-mon[31351]: pgmap v1087: 129 pgs: 129 active+clean; 664 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.6 MiB/s wr, 630 op/s 2024-03-21T11:46:13.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:13 smithi100 ceph-mon[36245]: pgmap v1087: 129 pgs: 129 active+clean; 664 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.6 MiB/s wr, 630 op/s 2024-03-21T11:46:13.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:13 smithi102 ceph-mon[36748]: pgmap v1087: 129 pgs: 129 active+clean; 664 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.6 MiB/s wr, 630 op/s 2024-03-21T11:46:13.745 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:46:13.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:46:13.746 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:46:13.985 INFO:tasks.workunit.client.0.smithi047.stdout: 10 228849 22.03 MB/sec execute 417 sec latency 69.098 ms 2024-03-21T11:46:14.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.038+0000 7fb32fc00640 1 -- 172.21.15.47:0/1164195034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb328077c40 msgr2=0x7fb3280784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.038+0000 7fb32fc00640 1 --2- 172.21.15.47:0/1164195034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb328077c40 0x7fb3280784e0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fb320009900 tx=0x7fb32002bc30 comp rx=0 tx=0).stop 2024-03-21T11:46:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.038+0000 7fb32fc00640 1 -- 172.21.15.47:0/1164195034 shutdown_connections 2024-03-21T11:46:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.038+0000 7fb32fc00640 1 --2- 172.21.15.47:0/1164195034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb328077c40 0x7fb3280784e0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.038+0000 7fb32fc00640 1 --2- 172.21.15.47:0/1164195034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3280772a0 0x7fb328077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.038+0000 7fb32fc00640 1 --2- 172.21.15.47:0/1164195034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb328076060 0x7fb328076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.038+0000 7fb32fc00640 1 -- 172.21.15.47:0/1164195034 >> 172.21.15.47:0/1164195034 conn(0x7fb3280fee50 msgr2=0x7fb328101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.039+0000 7fb32fc00640 1 -- 172.21.15.47:0/1164195034 shutdown_connections 2024-03-21T11:46:14.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.039+0000 7fb32fc00640 1 -- 172.21.15.47:0/1164195034 wait complete. 2024-03-21T11:46:14.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.039+0000 7fb32fc00640 1 Processor -- start 2024-03-21T11:46:14.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.040+0000 7fb32fc00640 1 -- start start 2024-03-21T11:46:14.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.040+0000 7fb32fc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb328076060 0x7fb32819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.040+0000 7fb32fc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3280772a0 0x7fb32819e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.040+0000 7fb32fc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb328077c40 0x7fb3281a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.040+0000 7fb32fc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb328113970 con 0x7fb3280772a0 2024-03-21T11:46:14.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.040+0000 7fb32fc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3281137f0 con 0x7fb328077c40 2024-03-21T11:46:14.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.040+0000 7fb32fc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb328113af0 con 0x7fb328076060 2024-03-21T11:46:14.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.040+0000 7fb32d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb328076060 0x7fb32819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.040+0000 7fb327e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3280772a0 0x7fb32819e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.040+0000 7fb32dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb328077c40 0x7fb3281a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.040+0000 7fb32d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb328076060 0x7fb32819df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60668/0 (socket says 172.21.15.47:60668) 2024-03-21T11:46:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.040+0000 7fb32d200640 1 -- 172.21.15.47:0/3277695730 learned_addr learned my addr 172.21.15.47:0/3277695730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.041+0000 7fb32d200640 1 -- 172.21.15.47:0/3277695730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb328077c40 msgr2=0x7fb3281a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.041+0000 7fb32d200640 1 --2- 172.21.15.47:0/3277695730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb328077c40 0x7fb3281a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.041+0000 7fb32d200640 1 -- 172.21.15.47:0/3277695730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3280772a0 msgr2=0x7fb32819e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.041+0000 7fb32d200640 1 --2- 172.21.15.47:0/3277695730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3280772a0 0x7fb32819e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.041+0000 7fb32d200640 1 -- 172.21.15.47:0/3277695730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3281a5b90 con 0x7fb328076060 2024-03-21T11:46:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.041+0000 7fb327e00640 1 --2- 172.21.15.47:0/3277695730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3280772a0 0x7fb32819e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.041+0000 7fb32dc00640 1 --2- 172.21.15.47:0/3277695730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb328077c40 0x7fb3281a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.041+0000 7fb32d200640 1 --2- 172.21.15.47:0/3277695730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb328076060 0x7fb32819df70 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fb31000ea60 tx=0x7fb31000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.041+0000 7fb325600640 1 -- 172.21.15.47:0/3277695730 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31000cde0 con 0x7fb328076060 2024-03-21T11:46:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.042+0000 7fb325600640 1 -- 172.21.15.47:0/3277695730 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3100035f0 con 0x7fb328076060 2024-03-21T11:46:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.042+0000 7fb32fc00640 1 -- 172.21.15.47:0/3277695730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3281a5e80 con 0x7fb328076060 2024-03-21T11:46:14.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.042+0000 7fb325600640 1 -- 172.21.15.47:0/3277695730 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310010670 con 0x7fb328076060 2024-03-21T11:46:14.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.042+0000 7fb32fc00640 1 -- 172.21.15.47:0/3277695730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3281a62e0 con 0x7fb328076060 2024-03-21T11:46:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.045+0000 7fb325600640 1 -- 172.21.15.47:0/3277695730 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb3100037a0 con 0x7fb328076060 2024-03-21T11:46:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.045+0000 7fb332976740 1 Processor -- start 2024-03-21T11:46:14.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.045+0000 7fb332976740 1 -- start start 2024-03-21T11:46:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.046+0000 7fb332976740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73ca84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.046+0000 7fb332976740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73ca8a90 0x561d73cad590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.046+0000 7fb332976740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73cadc00 0x561d73cb0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.046+0000 7fb332976740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d73c0deb0 con 0x561d73ca8a90 2024-03-21T11:46:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.046+0000 7fb332976740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d73bb6a60 con 0x561d73ca80a0 2024-03-21T11:46:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.046+0000 7fb332976740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d73c09f50 con 0x561d73cadc00 2024-03-21T11:46:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.046+0000 7fb32d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73ca8a90 0x561d73cad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.046+0000 7fb32dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73ca84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.046+0000 7fb32dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73ca84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41786/0 (socket says 172.21.15.47:41786) 2024-03-21T11:46:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.046+0000 7fb32dc00640 1 -- 172.21.15.47:0/2535887302 learned_addr learned my addr 172.21.15.47:0/2535887302 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:14.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.046+0000 7fb327e00640 1 --2- 172.21.15.47:0/2535887302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73cadc00 0x561d73cb0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.046+0000 7fb32dc00640 1 -- 172.21.15.47:0/2535887302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73cadc00 msgr2=0x561d73cb0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.046+0000 7fb32dc00640 1 --2- 172.21.15.47:0/2535887302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73cadc00 0x561d73cb0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.047+0000 7fb32dc00640 1 -- 172.21.15.47:0/2535887302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73ca8a90 msgr2=0x561d73cad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.047+0000 7fb32dc00640 1 --2- 172.21.15.47:0/2535887302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73ca8a90 0x561d73cad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.047+0000 7fb32dc00640 1 -- 172.21.15.47:0/2535887302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d739ea7e0 con 0x561d73ca80a0 2024-03-21T11:46:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.047+0000 7fb32dc00640 1 --2- 172.21.15.47:0/2535887302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73ca84c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fb32002cfd0 tx=0x7fb32002cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1faebcf99336f06b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:14.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.047+0000 7fb325600640 1 --2- 172.21.15.47:0/3277695730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb30c0a2f20 0x7fb30c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.047+0000 7fb325600640 1 -- 172.21.15.47:0/3277695730 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb3100cfa20 con 0x7fb328076060 2024-03-21T11:46:14.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.047+0000 7fb31e000640 1 -- 172.21.15.47:0/2535887302 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320042070 con 0x561d73ca80a0 2024-03-21T11:46:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.047+0000 7fb32d200640 1 --2- 172.21.15.47:0/3277695730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb30c0a2f20 0x7fb30c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.047+0000 7fb31e000640 1 -- 172.21.15.47:0/2535887302 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb320004ba0 con 0x561d73ca80a0 2024-03-21T11:46:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.047+0000 7fb31e000640 1 -- 172.21.15.47:0/2535887302 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320005300 con 0x561d73ca80a0 2024-03-21T11:46:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.048+0000 7fb332976740 1 -- 172.21.15.47:0/2535887302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 msgr2=0x561d73ca84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.048+0000 7fb332976740 1 --2- 172.21.15.47:0/2535887302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73ca84c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fb32002cfd0 tx=0x7fb32002cd30 comp rx=0 tx=0).stop 2024-03-21T11:46:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.048+0000 7fb332976740 1 -- 172.21.15.47:0/2535887302 shutdown_connections 2024-03-21T11:46:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.048+0000 7fb332976740 1 --2- 172.21.15.47:0/2535887302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73cadc00 0x561d73cb0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.048+0000 7fb332976740 1 --2- 172.21.15.47:0/2535887302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73ca8a90 0x561d73cad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.048+0000 7fb32d200640 1 --2- 172.21.15.47:0/3277695730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb30c0a2f20 0x7fb30c0a53e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb3100a4240 tx=0x7fb3100d2f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.048+0000 7fb332976740 1 --2- 172.21.15.47:0/2535887302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73ca84c0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.048+0000 7fb332976740 1 -- 172.21.15.47:0/2535887302 >> 172.21.15.47:0/2535887302 conn(0x561d73ca3610 msgr2=0x561d73ca3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.049+0000 7fb332976740 1 -- 172.21.15.47:0/2535887302 shutdown_connections 2024-03-21T11:46:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.049+0000 7fb332976740 1 -- 172.21.15.47:0/2535887302 wait complete. 2024-03-21T11:46:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.049+0000 7fb332976740 1 -- start start 2024-03-21T11:46:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.049+0000 7fb332976740 1 Processor -- start 2024-03-21T11:46:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.050+0000 7fb332976740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73ca80a0 0x561d73cb2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.050+0000 7fb332976740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 0x561d73cb27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.050+0000 7fb332976740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73cadc00 0x561d73cb9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.050+0000 7fb332976740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d73cb0b50 con 0x561d73ca80a0 2024-03-21T11:46:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.050+0000 7fb332976740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d73bc9eb0 con 0x561d73cadc00 2024-03-21T11:46:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.050+0000 7fb332976740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d73cb0cd0 con 0x561d73ca8a90 2024-03-21T11:46:14.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.050+0000 7fb332976740 10 client.0 _ll_register_callbacks cb 0x561d73bbe150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.050+0000 7fb32dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73ca80a0 0x561d73cb2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.050+0000 7fb32dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73ca80a0 0x561d73cb2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36224/0 (socket says 172.21.15.47:36224) 2024-03-21T11:46:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.050+0000 7fb32dc00640 1 -- 172.21.15.47:0/2887016028 learned_addr learned my addr 172.21.15.47:0/2887016028 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.050+0000 7fb327e00640 1 --2- 172.21.15.47:0/2887016028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 0x561d73cb27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.051+0000 7fb32dc00640 1 --2- 172.21.15.47:0/2887016028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73cadc00 0x561d73cb9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.051+0000 7fb32dc00640 1 -- 172.21.15.47:0/2887016028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 msgr2=0x561d73cb27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.051+0000 7fb32dc00640 1 --2- 172.21.15.47:0/2887016028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 0x561d73cb27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.051+0000 7fb32dc00640 1 -- 172.21.15.47:0/2887016028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73cadc00 msgr2=0x561d73cb9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.051+0000 7fb32dc00640 1 --2- 172.21.15.47:0/2887016028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73cadc00 0x561d73cb9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.051+0000 7fb32dc00640 1 -- 172.21.15.47:0/2887016028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d73a4af70 con 0x561d73ca80a0 2024-03-21T11:46:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.051+0000 7fb32dc00640 1 --2- 172.21.15.47:0/2887016028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73ca80a0 0x561d73cb2280 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb320036040 tx=0x7fb320004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283fe061d2496d44 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.051+0000 7fb316a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:46:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.051+0000 7fb316a00640 1 -- 172.21.15.47:0/2887016028 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320042040 con 0x561d73ca80a0 2024-03-21T11:46:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.051+0000 7fb316a00640 1 -- 172.21.15.47:0/2887016028 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb320034440 con 0x561d73ca80a0 2024-03-21T11:46:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.051+0000 7fb332976740 1 -- 172.21.15.47:0/2887016028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb320004ba0 con 0x561d73ca80a0 2024-03-21T11:46:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.052+0000 7fb316a00640 1 -- 172.21.15.47:0/2887016028 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb32005a050 con 0x561d73ca80a0 2024-03-21T11:46:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.052+0000 7fb332976740 10 client.22032 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.052+0000 7fb332976740 1 -- 172.21.15.47:0/2887016028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561d738e2170 con 0x561d73ca80a0 2024-03-21T11:46:14.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.052+0000 7fb316a00640 1 -- 172.21.15.47:0/2887016028 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb320004ba0 con 0x561d73ca80a0 2024-03-21T11:46:14.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.052+0000 7fb316a00640 1 -- 172.21.15.47:0/2887016028 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb32003e500 con 0x561d73ca80a0 2024-03-21T11:46:14.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.052+0000 7fb332976740 10 client.22032 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.052+0000 7fb332976740 10 client.22032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.052+0000 7fb332976740 20 client.22032 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.052+0000 7fb332976740 1 --2- 172.21.15.47:0/2887016028 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d73cbdcf0 0x561d73cc00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.052+0000 7fb332976740 4 client.22032 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.052+0000 7fb332976740 1 -- 172.21.15.47:0/2887016028 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d73c1b3f0 con 0x561d73cbdcf0 2024-03-21T11:46:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.053+0000 7fb327e00640 1 --2- 172.21.15.47:0/2887016028 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d73cbdcf0 0x561d73cc00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.053+0000 7fb327e00640 1 --2- 172.21.15.47:0/2887016028 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d73cbdcf0 0x561d73cc00e0 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f529a2312c2839c server_cookie=8ae64f7a0140c149 in_seq=0 out_seq=0 2024-03-21T11:46:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.053+0000 7fb316a00640 10 client.22032 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.057+0000 7fb316a00640 1 -- 172.21.15.47:0/2887016028 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561d73c1b3f0 con 0x561d73cbdcf0 2024-03-21T11:46:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.057+0000 7fb316a00640 10 client.22032 handle_command_reply: tid=0 2024-03-21T11:46:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.057+0000 7fb332976740 1 client.22032 shutdown 2024-03-21T11:46:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.057+0000 7fb332976740 1 -- 172.21.15.47:0/2887016028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73ca80a0 msgr2=0x561d73cb2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.057+0000 7fb332976740 1 --2- 172.21.15.47:0/2887016028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73ca80a0 0x561d73cb2280 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb320036040 tx=0x7fb320004df0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.057+0000 7fb332976740 1 -- 172.21.15.47:0/2887016028 shutdown_connections 2024-03-21T11:46:14.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.057+0000 7fb332976740 1 --2- 172.21.15.47:0/2887016028 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d73cbdcf0 0x561d73cc00e0 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.057+0000 7fb332976740 1 --2- 172.21.15.47:0/2887016028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73cadc00 0x561d73cb9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.057+0000 7fb332976740 1 --2- 172.21.15.47:0/2887016028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 0x561d73cb27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.057+0000 7fb332976740 1 --2- 172.21.15.47:0/2887016028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73ca80a0 0x561d73cb2280 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.057+0000 7fb316a00640 0 client.22032 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:14.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.057+0000 7fb332976740 1 -- 172.21.15.47:0/2887016028 >> 172.21.15.47:0/2887016028 conn(0x561d73ca3610 msgr2=0x561d73ca2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:14.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.058+0000 7fb332976740 1 -- 172.21.15.47:0/2887016028 shutdown_connections 2024-03-21T11:46:14.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.058+0000 7fb332976740 1 -- 172.21.15.47:0/2887016028 wait complete. 2024-03-21T11:46:14.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.058+0000 7fb332976740 20 client.22032 trim_cache size 0 max 16384 2024-03-21T11:46:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.083+0000 7fb332976740 1 Processor -- start 2024-03-21T11:46:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.084+0000 7fb332976740 1 -- start start 2024-03-21T11:46:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.084+0000 7fb332976740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73cb2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.084+0000 7fb332976740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 0x561d73cb9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.084+0000 7fb332976740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73cadc00 0x561d73ca6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.084+0000 7fb332976740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3281b0500 con 0x561d73cadc00 2024-03-21T11:46:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.084+0000 7fb332976740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3281b1e80 con 0x561d73ca80a0 2024-03-21T11:46:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.084+0000 7fb332976740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3281b0b60 con 0x561d73ca8a90 2024-03-21T11:46:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.084+0000 7fb32dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73cb2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.084+0000 7fb32dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73cb2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41810/0 (socket says 172.21.15.47:41810) 2024-03-21T11:46:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.084+0000 7fb32dc00640 1 -- 172.21.15.47:0/2044192273 learned_addr learned my addr 172.21.15.47:0/2044192273 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.085+0000 7fb32dc00640 1 --2- 172.21.15.47:0/2044192273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73cadc00 0x561d73ca6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.085+0000 7fb327e00640 1 --2- 172.21.15.47:0/2044192273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 0x561d73cb9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.085+0000 7fb32dc00640 1 -- 172.21.15.47:0/2044192273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 msgr2=0x561d73cb9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.085+0000 7fb32dc00640 1 --2- 172.21.15.47:0/2044192273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 0x561d73cb9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.085+0000 7fb32dc00640 1 -- 172.21.15.47:0/2044192273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73cadc00 msgr2=0x561d73ca6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.085+0000 7fb32dc00640 1 --2- 172.21.15.47:0/2044192273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73cadc00 0x561d73ca6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.085+0000 7fb32dc00640 1 -- 172.21.15.47:0/2044192273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d73ca2680 con 0x561d73ca80a0 2024-03-21T11:46:14.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.085+0000 7fb32dc00640 1 --2- 172.21.15.47:0/2044192273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73cb2720 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb320004ec0 tx=0x7fb32002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=281a4ed28df12cc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:14.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.086+0000 7fb31d600640 1 -- 172.21.15.47:0/2044192273 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320063070 con 0x561d73ca80a0 2024-03-21T11:46:14.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.086+0000 7fb31d600640 1 -- 172.21.15.47:0/2044192273 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb32002cd30 con 0x561d73ca80a0 2024-03-21T11:46:14.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.086+0000 7fb31d600640 1 -- 172.21.15.47:0/2044192273 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320042070 con 0x561d73ca80a0 2024-03-21T11:46:14.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.087+0000 7fb332976740 1 -- 172.21.15.47:0/2044192273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 msgr2=0x561d73cb2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.087+0000 7fb332976740 1 --2- 172.21.15.47:0/2044192273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73cb2720 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb320004ec0 tx=0x7fb32002bc30 comp rx=0 tx=0).stop 2024-03-21T11:46:14.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.087+0000 7fb332976740 1 -- 172.21.15.47:0/2044192273 shutdown_connections 2024-03-21T11:46:14.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.087+0000 7fb332976740 1 --2- 172.21.15.47:0/2044192273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73cadc00 0x561d73ca6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.087+0000 7fb332976740 1 --2- 172.21.15.47:0/2044192273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 0x561d73cb9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.087+0000 7fb332976740 1 --2- 172.21.15.47:0/2044192273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73cb2720 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.087+0000 7fb332976740 1 -- 172.21.15.47:0/2044192273 >> 172.21.15.47:0/2044192273 conn(0x561d73ca3610 msgr2=0x561d73ca6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:14.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.088+0000 7fb332976740 1 -- 172.21.15.47:0/2044192273 shutdown_connections 2024-03-21T11:46:14.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.088+0000 7fb332976740 1 -- 172.21.15.47:0/2044192273 wait complete. 2024-03-21T11:46:14.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.088+0000 7fb332976740 1 -- start start 2024-03-21T11:46:14.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.088+0000 7fb332976740 1 Processor -- start 2024-03-21T11:46:14.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.089+0000 7fb332976740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73cb9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.089+0000 7fb332976740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 0x561d73cb7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.089+0000 7fb332976740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73cadc00 0x561d73cb8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.089+0000 7fb332976740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3281b1820 con 0x561d73cadc00 2024-03-21T11:46:14.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.089+0000 7fb332976740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3281b11c0 con 0x561d73ca80a0 2024-03-21T11:46:14.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.089+0000 7fb332976740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3281ae6c0 con 0x561d73ca8a90 2024-03-21T11:46:14.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.089+0000 7fb332976740 10 client.0 _ll_register_callbacks cb 0x7fb3100d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:14.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.089+0000 7fb32dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73cb9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.089+0000 7fb327e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 0x561d73cb7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.090+0000 7fb327e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 0x561d73cb7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60700/0 (socket says 172.21.15.47:60700) 2024-03-21T11:46:14.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.090+0000 7fb327e00640 1 -- 172.21.15.47:0/635960480 learned_addr learned my addr 172.21.15.47:0/635960480 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:14.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.090+0000 7fb32dc00640 1 --2- 172.21.15.47:0/635960480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73cadc00 0x561d73cb8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.090+0000 7fb32dc00640 1 -- 172.21.15.47:0/635960480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 msgr2=0x561d73cb7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.090+0000 7fb32dc00640 1 --2- 172.21.15.47:0/635960480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 0x561d73cb7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.090+0000 7fb32dc00640 1 -- 172.21.15.47:0/635960480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 msgr2=0x561d73cb9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.090+0000 7fb32dc00640 1 --2- 172.21.15.47:0/635960480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73cb9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.090+0000 7fb32dc00640 1 -- 172.21.15.47:0/635960480 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d73cc0450 con 0x561d73cadc00 2024-03-21T11:46:14.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.091+0000 7fb32dc00640 1 --2- 172.21.15.47:0/635960480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73cadc00 0x561d73cb8390 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x561d73cb1440 tx=0x7fb320000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6a973fe0363cd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:14.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.091+0000 7fb316a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:46:14.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.091+0000 7fb316a00640 1 -- 172.21.15.47:0/635960480 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320042050 con 0x561d73cadc00 2024-03-21T11:46:14.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.091+0000 7fb316a00640 1 -- 172.21.15.47:0/635960480 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb320004040 con 0x561d73cadc00 2024-03-21T11:46:14.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.091+0000 7fb332976740 1 -- 172.21.15.47:0/635960480 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d73c15740 con 0x561d73cadc00 2024-03-21T11:46:14.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.092+0000 7fb316a00640 1 -- 172.21.15.47:0/635960480 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb320063070 con 0x561d73cadc00 2024-03-21T11:46:14.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.092+0000 7fb332976740 10 client.22038 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:14.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.092+0000 7fb332976740 1 -- 172.21.15.47:0/635960480 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561d73d1a470 con 0x561d73cadc00 2024-03-21T11:46:14.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.092+0000 7fb316a00640 1 -- 172.21.15.47:0/635960480 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561d73c15740 con 0x561d73cadc00 2024-03-21T11:46:14.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.092+0000 7fb316a00640 1 -- 172.21.15.47:0/635960480 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb320066030 con 0x561d73cadc00 2024-03-21T11:46:14.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.092+0000 7fb332976740 10 client.22038 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:14.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.092+0000 7fb332976740 10 client.22038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:14.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.092+0000 7fb332976740 20 client.22038 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:14.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.092+0000 7fb332976740 1 --2- 172.21.15.47:0/635960480 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d73d1ab00 0x561d73d1cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.092+0000 7fb332976740 4 client.22038 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:46:14.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.092+0000 7fb332976740 1 -- 172.21.15.47:0/635960480 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561d73b93a40 con 0x561d73d1ab00 2024-03-21T11:46:14.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.093+0000 7fb327e00640 1 --2- 172.21.15.47:0/635960480 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d73d1ab00 0x561d73d1cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.094+0000 7fb327e00640 1 --2- 172.21.15.47:0/635960480 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d73d1ab00 0x561d73d1cf20 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf4d9d4b441c69c server_cookie=464c3fd4d6b57fa1 in_seq=0 out_seq=0 2024-03-21T11:46:14.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.094+0000 7fb316a00640 10 client.22038 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.103+0000 7fb316a00640 1 -- 172.21.15.47:0/635960480 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561d73b93a40 con 0x561d73d1ab00 2024-03-21T11:46:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.103+0000 7fb316a00640 10 client.22038 handle_command_reply: tid=0 2024-03-21T11:46:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.103+0000 7fb332976740 1 client.22038 shutdown 2024-03-21T11:46:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.103+0000 7fb332976740 1 -- 172.21.15.47:0/635960480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73cadc00 msgr2=0x561d73cb8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.103+0000 7fb332976740 1 --2- 172.21.15.47:0/635960480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73cadc00 0x561d73cb8390 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x561d73cb1440 tx=0x7fb320000bc0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.103+0000 7fb332976740 1 -- 172.21.15.47:0/635960480 shutdown_connections 2024-03-21T11:46:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.103+0000 7fb332976740 1 --2- 172.21.15.47:0/635960480 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d73d1ab00 0x561d73d1cf20 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.103+0000 7fb332976740 1 --2- 172.21.15.47:0/635960480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d73cadc00 0x561d73cb8390 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.103+0000 7fb332976740 1 --2- 172.21.15.47:0/635960480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d73ca8a90 0x561d73cb7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.103+0000 7fb332976740 1 --2- 172.21.15.47:0/635960480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d73ca80a0 0x561d73cb9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.103+0000 7fb332976740 1 -- 172.21.15.47:0/635960480 >> 172.21.15.47:0/635960480 conn(0x561d73ca3610 msgr2=0x561d73ca7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.103+0000 7fb316a00640 0 client.22038 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.104+0000 7fb332976740 1 -- 172.21.15.47:0/635960480 shutdown_connections 2024-03-21T11:46:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.104+0000 7fb332976740 1 -- 172.21.15.47:0/635960480 wait complete. 2024-03-21T11:46:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.104+0000 7fb332976740 20 client.22038 trim_cache size 0 max 16384 2024-03-21T11:46:14.105 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:46:14.105 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:46:14.105 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "ed4db63c-c7f6-4fe4-97bf-be51fadb8c30", 2024-03-21T11:46:14.105 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:46:14.105 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:46:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.106+0000 7fb32fc00640 1 -- 172.21.15.47:0/3277695730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb30c0a2f20 msgr2=0x7fb30c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.106+0000 7fb32fc00640 1 --2- 172.21.15.47:0/3277695730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb30c0a2f20 0x7fb30c0a53e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb3100a4240 tx=0x7fb3100d2f90 comp rx=0 tx=0).stop 2024-03-21T11:46:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.106+0000 7fb32fc00640 1 -- 172.21.15.47:0/3277695730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb328076060 msgr2=0x7fb32819df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.106+0000 7fb32fc00640 1 --2- 172.21.15.47:0/3277695730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb328076060 0x7fb32819df70 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fb31000ea60 tx=0x7fb31000ef20 comp rx=0 tx=0).stop 2024-03-21T11:46:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.106+0000 7fb32fc00640 1 -- 172.21.15.47:0/3277695730 shutdown_connections 2024-03-21T11:46:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.106+0000 7fb32fc00640 1 --2- 172.21.15.47:0/3277695730 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb30c0a2f20 0x7fb30c0a53e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.106+0000 7fb32fc00640 1 --2- 172.21.15.47:0/3277695730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb328077c40 0x7fb3281a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.106+0000 7fb32fc00640 1 --2- 172.21.15.47:0/3277695730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3280772a0 0x7fb32819e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.106+0000 7fb32fc00640 1 --2- 172.21.15.47:0/3277695730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb328076060 0x7fb32819df70 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.106+0000 7fb32fc00640 1 -- 172.21.15.47:0/3277695730 >> 172.21.15.47:0/3277695730 conn(0x7fb3280fee50 msgr2=0x7fb328100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.107+0000 7fb32fc00640 1 -- 172.21.15.47:0/3277695730 shutdown_connections 2024-03-21T11:46:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.107+0000 7fb32fc00640 1 -- 172.21.15.47:0/3277695730 wait complete. 2024-03-21T11:46:14.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:46:14.136 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:46:14.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.368+0000 7f4e9d200640 1 -- 172.21.15.47:0/128358445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e98102060 msgr2=0x7f4e98104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.368+0000 7f4e9d200640 1 --2- 172.21.15.47:0/128358445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e98102060 0x7f4e98104450 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009900 tx=0x7f4e8802bc30 comp rx=0 tx=0).stop 2024-03-21T11:46:14.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.368+0000 7f4e9d200640 1 -- 172.21.15.47:0/128358445 shutdown_connections 2024-03-21T11:46:14.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.368+0000 7f4e9d200640 1 --2- 172.21.15.47:0/128358445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e981072c0 0x7f4e981096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.368+0000 7f4e9d200640 1 --2- 172.21.15.47:0/128358445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e98104990 0x7f4e98106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.368+0000 7f4e9d200640 1 --2- 172.21.15.47:0/128358445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e98102060 0x7f4e98104450 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.368+0000 7f4e9d200640 1 -- 172.21.15.47:0/128358445 >> 172.21.15.47:0/128358445 conn(0x7f4e980fba10 msgr2=0x7f4e980fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:14.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.369+0000 7f4e9d200640 1 -- 172.21.15.47:0/128358445 shutdown_connections 2024-03-21T11:46:14.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.369+0000 7f4e9d200640 1 -- 172.21.15.47:0/128358445 wait complete. 2024-03-21T11:46:14.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.369+0000 7f4e9d200640 1 Processor -- start 2024-03-21T11:46:14.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.370+0000 7f4e9d200640 1 -- start start 2024-03-21T11:46:14.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.370+0000 7f4e9d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e98102060 0x7f4e981019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.370+0000 7f4e9d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e98104990 0x7f4e98100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.370+0000 7f4e9d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e981072c0 0x7f4e98100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.370+0000 7f4e9d200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9810d210 con 0x7f4e98104990 2024-03-21T11:46:14.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.370+0000 7f4e9d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9810d090 con 0x7f4e98102060 2024-03-21T11:46:14.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.370+0000 7f4e9d200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9810d390 con 0x7f4e981072c0 2024-03-21T11:46:14.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.370+0000 7f4e97400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e98102060 0x7f4e981019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.370+0000 7f4e96a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e98104990 0x7f4e98100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.370+0000 7f4e97400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e98102060 0x7f4e981019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41838/0 (socket says 172.21.15.47:41838) 2024-03-21T11:46:14.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.370+0000 7f4e97400640 1 -- 172.21.15.47:0/1314697966 learned_addr learned my addr 172.21.15.47:0/1314697966 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.370+0000 7f4e97e00640 1 --2- 172.21.15.47:0/1314697966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e981072c0 0x7f4e98100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.371+0000 7f4e96a00640 1 -- 172.21.15.47:0/1314697966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e981072c0 msgr2=0x7f4e98100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.371+0000 7f4e96a00640 1 --2- 172.21.15.47:0/1314697966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e981072c0 0x7f4e98100550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.371+0000 7f4e96a00640 1 -- 172.21.15.47:0/1314697966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e98102060 msgr2=0x7f4e981019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.371+0000 7f4e96a00640 1 --2- 172.21.15.47:0/1314697966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e98102060 0x7f4e981019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.371+0000 7f4e96a00640 1 -- 172.21.15.47:0/1314697966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e98100a90 con 0x7f4e98104990 2024-03-21T11:46:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.371+0000 7f4e96a00640 1 --2- 172.21.15.47:0/1314697966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e98104990 0x7f4e98100010 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400e9c0 tx=0x7f4e8400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.371+0000 7f4e8fe00640 1 -- 172.21.15.47:0/1314697966 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8400ccc0 con 0x7f4e98104990 2024-03-21T11:46:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.371+0000 7f4e8fe00640 1 -- 172.21.15.47:0/1314697966 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e84010910 con 0x7f4e98104990 2024-03-21T11:46:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.371+0000 7f4e9d200640 1 -- 172.21.15.47:0/1314697966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e98100d80 con 0x7f4e98104990 2024-03-21T11:46:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.371+0000 7f4e8fe00640 1 -- 172.21.15.47:0/1314697966 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e840188c0 con 0x7f4e98104990 2024-03-21T11:46:14.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.372+0000 7f4e9d200640 1 -- 172.21.15.47:0/1314697966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e981a6370 con 0x7f4e98104990 2024-03-21T11:46:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.376+0000 7f4ea1607740 1 Processor -- start 2024-03-21T11:46:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.376+0000 7f4e8fe00640 1 -- 172.21.15.47:0/1314697966 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4e84010ac0 con 0x7f4e98104990 2024-03-21T11:46:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.376+0000 7f4ea1607740 1 -- start start 2024-03-21T11:46:14.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.377+0000 7f4ea1607740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9416d4050 0x55a9416d4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.377+0000 7f4ea1607740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416d9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.377+0000 7f4ea1607740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d9bb0 0x55a9416dc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.377+0000 7f4ea1607740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a941639250 con 0x55a9416d4a40 2024-03-21T11:46:14.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.377+0000 7f4ea1607740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9415e1a30 con 0x55a9416d4050 2024-03-21T11:46:14.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.377+0000 7f4ea1607740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9416352f0 con 0x55a9416d9bb0 2024-03-21T11:46:14.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.377+0000 7f4e97e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9416d4050 0x55a9416d4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.377+0000 7f4e97e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9416d4050 0x55a9416d4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41848/0 (socket says 172.21.15.47:41848) 2024-03-21T11:46:14.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.377+0000 7f4e97e00640 1 -- 172.21.15.47:0/3661438145 learned_addr learned my addr 172.21.15.47:0/3661438145 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:14.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.377+0000 7f4e96a00640 1 --2- 172.21.15.47:0/3661438145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d9bb0 0x55a9416dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.378+0000 7f4e97400640 1 --2- 172.21.15.47:0/3661438145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416d9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.378+0000 7f4e97e00640 1 -- 172.21.15.47:0/3661438145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d9bb0 msgr2=0x55a9416dc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.378+0000 7f4e97e00640 1 --2- 172.21.15.47:0/3661438145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d9bb0 0x55a9416dc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.378+0000 7f4e97e00640 1 -- 172.21.15.47:0/3661438145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 msgr2=0x55a9416d9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.378+0000 7f4e97e00640 1 --2- 172.21.15.47:0/3661438145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416d9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.378+0000 7f4e97e00640 1 -- 172.21.15.47:0/3661438145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9415cc690 con 0x55a9416d4050 2024-03-21T11:46:14.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.378+0000 7f4e97e00640 1 --2- 172.21.15.47:0/3661438145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9416d4050 0x55a9416d4470 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009fd0 tx=0x7f4e900118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7330117964d7f805 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:14.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.379+0000 7f4e8cc00640 1 -- 172.21.15.47:0/3661438145 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90004d20 con 0x55a9416d4050 2024-03-21T11:46:14.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.379+0000 7f4e8cc00640 1 -- 172.21.15.47:0/3661438145 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e90004050 con 0x55a9416d4050 2024-03-21T11:46:14.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.379+0000 7f4e8fe00640 1 --2- 172.21.15.47:0/1314697966 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e7c0a2fa0 0x7f4e7c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.379+0000 7f4e8cc00640 1 -- 172.21.15.47:0/3661438145 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90005240 con 0x55a9416d4050 2024-03-21T11:46:14.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.379+0000 7f4e8fe00640 1 -- 172.21.15.47:0/1314697966 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4e840cfb20 con 0x7f4e98104990 2024-03-21T11:46:14.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.379+0000 7f4e97400640 1 --2- 172.21.15.47:0/1314697966 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e7c0a2fa0 0x7f4e7c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.380+0000 7f4ea1607740 1 -- 172.21.15.47:0/3661438145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9416d4050 msgr2=0x55a9416d4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.380+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3661438145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9416d4050 0x55a9416d4470 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009fd0 tx=0x7f4e900118d0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.380+0000 7f4ea1607740 1 -- 172.21.15.47:0/3661438145 shutdown_connections 2024-03-21T11:46:14.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.380+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3661438145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d9bb0 0x55a9416dc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.380+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3661438145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416d9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.380+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3661438145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9416d4050 0x55a9416d4470 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.380+0000 7f4ea1607740 1 -- 172.21.15.47:0/3661438145 >> 172.21.15.47:0/3661438145 conn(0x55a94169c200 msgr2=0x55a9416cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.380+0000 7f4ea1607740 1 -- 172.21.15.47:0/3661438145 shutdown_connections 2024-03-21T11:46:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.380+0000 7f4e97400640 1 --2- 172.21.15.47:0/1314697966 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e7c0a2fa0 0x7f4e7c0a5460 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4e88000c00 tx=0x7f4e8802c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.380+0000 7f4ea1607740 1 -- 172.21.15.47:0/3661438145 wait complete. 2024-03-21T11:46:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.381+0000 7f4ea1607740 1 -- start start 2024-03-21T11:46:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.381+0000 7f4ea1607740 1 Processor -- start 2024-03-21T11:46:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.381+0000 7f4ea1607740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 0x55a9416de040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.381+0000 7f4ea1607740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.381+0000 7f4ea1607740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9416d9bb0 0x55a9416e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.381+0000 7f4ea1607740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9416dcbb0 con 0x55a9416d4a40 2024-03-21T11:46:14.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.381+0000 7f4ea1607740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9415f4e80 con 0x55a9416d9bb0 2024-03-21T11:46:14.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.381+0000 7f4ea1607740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9416dcd30 con 0x55a9416d4050 2024-03-21T11:46:14.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.381+0000 7f4ea1607740 10 client.0 _ll_register_callbacks cb 0x55a941607130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:14.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.381+0000 7f4e96a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.381+0000 7f4e97400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 0x55a9416de040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.382+0000 7f4e97e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9416d9bb0 0x55a9416e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.382+0000 7f4e96a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416de580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36282/0 (socket says 172.21.15.47:36282) 2024-03-21T11:46:14.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.382+0000 7f4e96a00640 1 -- 172.21.15.47:0/3733827989 learned_addr learned my addr 172.21.15.47:0/3733827989 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:14.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.382+0000 7f4e96a00640 1 -- 172.21.15.47:0/3733827989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 msgr2=0x55a9416de040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.382+0000 7f4e96a00640 1 --2- 172.21.15.47:0/3733827989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 0x55a9416de040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.382+0000 7f4e96a00640 1 -- 172.21.15.47:0/3733827989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9416d9bb0 msgr2=0x55a9416e55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.382+0000 7f4e96a00640 1 --2- 172.21.15.47:0/3733827989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9416d9bb0 0x55a9416e55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.382+0000 7f4e96a00640 1 -- 172.21.15.47:0/3733827989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9414177d0 con 0x55a9416d4a40 2024-03-21T11:46:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.383+0000 7f4e96a00640 1 --2- 172.21.15.47:0/3733827989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416de580 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4e8409bc10 tx=0x7f4e8409ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe44d8d8bae190ae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.383+0000 7f4e81600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:46:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.383+0000 7f4e81600640 1 -- 172.21.15.47:0/3733827989 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e840a45e0 con 0x55a9416d4a40 2024-03-21T11:46:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.383+0000 7f4e81600640 1 -- 172.21.15.47:0/3733827989 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4e8409b360 con 0x55a9416d4a40 2024-03-21T11:46:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.383+0000 7f4ea1607740 1 -- 172.21.15.47:0/3733827989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e90004050 con 0x55a9416d4a40 2024-03-21T11:46:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.383+0000 7f4e81600640 1 -- 172.21.15.47:0/3733827989 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e840d2fb0 con 0x55a9416d4a40 2024-03-21T11:46:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.383+0000 7f4ea1607740 10 client.22050 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.383+0000 7f4ea1607740 1 -- 172.21.15.47:0/3733827989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a94134d3f0 con 0x55a9416d4a40 2024-03-21T11:46:14.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.384+0000 7f4e81600640 1 -- 172.21.15.47:0/3733827989 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e90004050 con 0x55a9416d4a40 2024-03-21T11:46:14.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.384+0000 7f4e81600640 1 -- 172.21.15.47:0/3733827989 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4e84097d80 con 0x55a9416d4a40 2024-03-21T11:46:14.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.384+0000 7f4ea1607740 10 client.22050 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:14.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.384+0000 7f4ea1607740 10 client.22050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:14.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.384+0000 7f4ea1607740 20 client.22050 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:14.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.384+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3733827989 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a9416e9e40 0x55a9416ec230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.384+0000 7f4ea1607740 4 client.22050 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:14.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.384+0000 7f4ea1607740 1 -- 172.21.15.47:0/3733827989 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a941646790 con 0x55a9416e9e40 2024-03-21T11:46:14.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.385+0000 7f4e97400640 1 --2- 172.21.15.47:0/3733827989 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a9416e9e40 0x55a9416ec230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.385+0000 7f4e97400640 1 --2- 172.21.15.47:0/3733827989 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a9416e9e40 0x55a9416ec230 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428c43af032f6872 server_cookie=18f9caa40ae6daad in_seq=0 out_seq=0 2024-03-21T11:46:14.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.385+0000 7f4e81600640 10 client.22050 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:14.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.391+0000 7f4e81600640 1 -- 172.21.15.47:0/3733827989 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a941646790 con 0x55a9416e9e40 2024-03-21T11:46:14.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.391+0000 7f4e81600640 10 client.22050 handle_command_reply: tid=0 2024-03-21T11:46:14.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.392+0000 7f4ea1607740 1 client.22050 shutdown 2024-03-21T11:46:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.393+0000 7f4ea1607740 1 -- 172.21.15.47:0/3733827989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 msgr2=0x55a9416de580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.393+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3733827989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416de580 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4e8409bc10 tx=0x7f4e8409ba50 comp rx=0 tx=0).stop 2024-03-21T11:46:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.393+0000 7f4ea1607740 1 -- 172.21.15.47:0/3733827989 shutdown_connections 2024-03-21T11:46:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.393+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3733827989 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a9416e9e40 0x55a9416ec230 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.393+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3733827989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9416d9bb0 0x55a9416e55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.393+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3733827989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416de580 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.393+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3733827989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 0x55a9416de040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.393+0000 7f4e81600640 0 client.22050 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:14.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.393+0000 7f4ea1607740 1 -- 172.21.15.47:0/3733827989 >> 172.21.15.47:0/3733827989 conn(0x55a94169c200 msgr2=0x55a9416cebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:14.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.394+0000 7f4ea1607740 1 -- 172.21.15.47:0/3733827989 shutdown_connections 2024-03-21T11:46:14.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.395+0000 7f4ea1607740 1 -- 172.21.15.47:0/3733827989 wait complete. 2024-03-21T11:46:14.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.395+0000 7f4ea1607740 20 client.22050 trim_cache size 0 max 16384 2024-03-21T11:46:14.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.424+0000 7f4ea1607740 1 Processor -- start 2024-03-21T11:46:14.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.424+0000 7f4ea1607740 1 -- start start 2024-03-21T11:46:14.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.425+0000 7f4ea1607740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a94169c200 0x55a9416d9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.425+0000 7f4ea1607740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 0x55a9416e5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.425+0000 7f4ea1607740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.425+0000 7f4ea1607740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e981b0560 con 0x55a9416d4a40 2024-03-21T11:46:14.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.425+0000 7f4ea1607740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e981b1ee0 con 0x55a94169c200 2024-03-21T11:46:14.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.425+0000 7f4ea1607740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e981b0bc0 con 0x55a9416d4050 2024-03-21T11:46:14.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.425+0000 7f4e97400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a94169c200 0x55a9416d9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.425+0000 7f4e97400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a94169c200 0x55a9416d9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41878/0 (socket says 172.21.15.47:41878) 2024-03-21T11:46:14.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.425+0000 7f4e97400640 1 -- 172.21.15.47:0/3397134695 learned_addr learned my addr 172.21.15.47:0/3397134695 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:14.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.425+0000 7f4e96a00640 1 --2- 172.21.15.47:0/3397134695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 0x55a9416e5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.426+0000 7f4e97e00640 1 --2- 172.21.15.47:0/3397134695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416d2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.426+0000 7f4e97400640 1 -- 172.21.15.47:0/3397134695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 msgr2=0x55a9416e5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.426+0000 7f4e97400640 1 --2- 172.21.15.47:0/3397134695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 0x55a9416e5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.426+0000 7f4e97400640 1 -- 172.21.15.47:0/3397134695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 msgr2=0x55a9416d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.426+0000 7f4e97400640 1 --2- 172.21.15.47:0/3397134695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416d2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.426+0000 7f4e97400640 1 -- 172.21.15.47:0/3397134695 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a94130d0f0 con 0x55a94169c200 2024-03-21T11:46:14.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.426+0000 7f4e97e00640 1 --2- 172.21.15.47:0/3397134695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416d2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:14.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.430+0000 7f4e97400640 1 --2- 172.21.15.47:0/3397134695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a94169c200 0x55a9416d9010 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f4e8802cde0 tx=0x7f4e8802e050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eff1db619f3016f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:14.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.431+0000 7f4e83e00640 1 -- 172.21.15.47:0/3397134695 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e88004980 con 0x55a94169c200 2024-03-21T11:46:14.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.432+0000 7f4e83e00640 1 -- 172.21.15.47:0/3397134695 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e88004b20 con 0x55a94169c200 2024-03-21T11:46:14.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.432+0000 7f4e83e00640 1 -- 172.21.15.47:0/3397134695 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e88054a20 con 0x55a94169c200 2024-03-21T11:46:14.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.433+0000 7f4ea1607740 1 -- 172.21.15.47:0/3397134695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a94169c200 msgr2=0x55a9416d9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.433+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3397134695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a94169c200 0x55a9416d9010 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f4e8802cde0 tx=0x7f4e8802e050 comp rx=0 tx=0).stop 2024-03-21T11:46:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.433+0000 7f4ea1607740 1 -- 172.21.15.47:0/3397134695 shutdown_connections 2024-03-21T11:46:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.433+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3397134695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416d2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.433+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3397134695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 0x55a9416e5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.433+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3397134695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a94169c200 0x55a9416d9010 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.433+0000 7f4ea1607740 1 -- 172.21.15.47:0/3397134695 >> 172.21.15.47:0/3397134695 conn(0x55a9416d96f0 msgr2=0x55a9416cebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:14.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.434+0000 7f4ea1607740 1 -- 172.21.15.47:0/3397134695 shutdown_connections 2024-03-21T11:46:14.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.434+0000 7f4ea1607740 1 -- 172.21.15.47:0/3397134695 wait complete. 2024-03-21T11:46:14.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.435+0000 7f4ea1607740 1 -- start start 2024-03-21T11:46:14.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.436+0000 7f4ea1607740 1 Processor -- start 2024-03-21T11:46:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.436+0000 7f4ea1607740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a94169c200 0x55a9416e3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.436+0000 7f4ea1607740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 0x55a9416e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.436+0000 7f4ea1607740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416e49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.436+0000 7f4ea1607740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e981b1880 con 0x55a9416d4a40 2024-03-21T11:46:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.436+0000 7f4ea1607740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e981b1220 con 0x55a94169c200 2024-03-21T11:46:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.436+0000 7f4ea1607740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e981ae720 con 0x55a9416d4050 2024-03-21T11:46:14.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.436+0000 7f4ea1607740 10 client.0 _ll_register_callbacks cb 0x7f4e840982a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:14.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.437+0000 7f4e97400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a94169c200 0x55a9416e3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.437+0000 7f4e97400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a94169c200 0x55a9416e3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41882/0 (socket says 172.21.15.47:41882) 2024-03-21T11:46:14.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.437+0000 7f4e97400640 1 -- 172.21.15.47:0/3721545457 learned_addr learned my addr 172.21.15.47:0/3721545457 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:14.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.437+0000 7f4e96a00640 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 0x55a9416e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.438+0000 7f4e97e00640 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416e49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.438+0000 7f4e96a00640 1 -- 172.21.15.47:0/3721545457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a94169c200 msgr2=0x55a9416e3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.438+0000 7f4e96a00640 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a94169c200 0x55a9416e3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.438+0000 7f4e96a00640 1 -- 172.21.15.47:0/3721545457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 msgr2=0x55a9416e49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.438+0000 7f4e96a00640 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416e49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.438+0000 7f4e96a00640 1 -- 172.21.15.47:0/3721545457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9416e4fe0 con 0x55a9416d4050 2024-03-21T11:46:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.438+0000 7f4e97400640 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a94169c200 0x55a9416e3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.438+0000 7f4e97e00640 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416e49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.438+0000 7f4e96a00640 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 0x55a9416e3aa0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4e84010f50 tx=0x7f4e8409a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ef63190ee89f583 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.438+0000 7f4e81600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.439+0000 7f4e81600640 1 -- 172.21.15.47:0/3721545457 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8409b870 con 0x55a9416d4050 2024-03-21T11:46:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.439+0000 7f4e81600640 1 -- 172.21.15.47:0/3721545457 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4e84010430 con 0x55a9416d4050 2024-03-21T11:46:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.439+0000 7f4ea1607740 1 -- 172.21.15.47:0/3721545457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e88004b20 con 0x55a9416d4050 2024-03-21T11:46:14.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.440+0000 7f4e81600640 1 -- 172.21.15.47:0/3721545457 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e8409bf10 con 0x55a9416d4050 2024-03-21T11:46:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.440+0000 7f4ea1607740 10 client.32824 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.440+0000 7f4ea1607740 1 -- 172.21.15.47:0/3721545457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a941756e40 con 0x55a9416d4050 2024-03-21T11:46:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.440+0000 7f4e81600640 1 -- 172.21.15.47:0/3721545457 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e88004b20 con 0x55a9416d4050 2024-03-21T11:46:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.441+0000 7f4e81600640 1 -- 172.21.15.47:0/3721545457 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4e84096a80 con 0x55a9416d4050 2024-03-21T11:46:14.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.444+0000 7f4ea1607740 10 client.32824 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:14.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.444+0000 7f4ea1607740 10 client.32824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:14.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.444+0000 7f4ea1607740 20 client.32824 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:14.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.444+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a9416d9bb0 0x55a941759750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:14.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.444+0000 7f4ea1607740 4 client.32824 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:46:14.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.444+0000 7f4ea1607740 1 -- 172.21.15.47:0/3721545457 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a941640ae0 con 0x55a9416d9bb0 2024-03-21T11:46:14.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.452+0000 7f4e97400640 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a9416d9bb0 0x55a941759750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:14.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.455+0000 7f4e97400640 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a9416d9bb0 0x55a941759750 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46bcdfe9992e8527 server_cookie=1e59fa44d11407bb in_seq=0 out_seq=0 2024-03-21T11:46:14.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.455+0000 7f4e81600640 10 client.32824 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.461+0000 7f4e81600640 1 -- 172.21.15.47:0/3721545457 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55a941640ae0 con 0x55a9416d9bb0 2024-03-21T11:46:14.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.461+0000 7f4e81600640 10 client.32824 handle_command_reply: tid=0 2024-03-21T11:46:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.461+0000 7f4ea1607740 1 client.32824 shutdown 2024-03-21T11:46:14.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.462+0000 7f4ea1607740 1 -- 172.21.15.47:0/3721545457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 msgr2=0x55a9416e3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.462+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 0x55a9416e3aa0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4e84010f50 tx=0x7f4e8409a420 comp rx=0 tx=0).stop 2024-03-21T11:46:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.462+0000 7f4ea1607740 1 -- 172.21.15.47:0/3721545457 shutdown_connections 2024-03-21T11:46:14.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.462+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a9416d9bb0 0x55a941759750 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.462+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9416d4a40 0x55a9416e49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.462+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9416d4050 0x55a9416e3aa0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.462+0000 7f4e81600640 0 client.32824 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.462+0000 7f4ea1607740 1 --2- 172.21.15.47:0/3721545457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a94169c200 0x55a9416e3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.462+0000 7f4ea1607740 1 -- 172.21.15.47:0/3721545457 >> 172.21.15.47:0/3721545457 conn(0x55a9416d96f0 msgr2=0x55a9416d52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:14.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.464+0000 7f4ea1607740 1 -- 172.21.15.47:0/3721545457 shutdown_connections 2024-03-21T11:46:14.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.466+0000 7f4ea1607740 1 -- 172.21.15.47:0/3721545457 wait complete. 2024-03-21T11:46:14.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.466+0000 7f4ea1607740 20 client.32824 trim_cache size 0 max 16384 2024-03-21T11:46:14.467 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:46:14.467 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (154 inodes in the stack)", 2024-03-21T11:46:14.467 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:46:14.467 INFO:teuthology.orchestra.run.smithi047.stdout: "ed4db63c-c7f6-4fe4-97bf-be51fadb8c30": { 2024-03-21T11:46:14.467 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:46:14.467 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "ed4db63c-c7f6-4fe4-97bf-be51fadb8c30", 2024-03-21T11:46:14.467 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:46:14.467 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:46:14.467 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:46:14.467 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:46:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.470+0000 7f4e9d200640 1 -- 172.21.15.47:0/1314697966 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e7c0a2fa0 msgr2=0x7f4e7c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.470+0000 7f4e9d200640 1 --2- 172.21.15.47:0/1314697966 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e7c0a2fa0 0x7f4e7c0a5460 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4e88000c00 tx=0x7f4e8802c880 comp rx=0 tx=0).stop 2024-03-21T11:46:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.470+0000 7f4e9d200640 1 -- 172.21.15.47:0/1314697966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e98104990 msgr2=0x7f4e98100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.470+0000 7f4e9d200640 1 --2- 172.21.15.47:0/1314697966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e98104990 0x7f4e98100010 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400e9c0 tx=0x7f4e8400ee80 comp rx=0 tx=0).stop 2024-03-21T11:46:14.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.470+0000 7f4e9d200640 1 -- 172.21.15.47:0/1314697966 shutdown_connections 2024-03-21T11:46:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.470+0000 7f4e9d200640 1 --2- 172.21.15.47:0/1314697966 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e7c0a2fa0 0x7f4e7c0a5460 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.470+0000 7f4e9d200640 1 --2- 172.21.15.47:0/1314697966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e981072c0 0x7f4e98100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.470+0000 7f4e9d200640 1 --2- 172.21.15.47:0/1314697966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e98104990 0x7f4e98100010 secure :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400e9c0 tx=0x7f4e8400ee80 comp rx=0 tx=0).stop 2024-03-21T11:46:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.470+0000 7f4e9d200640 1 --2- 172.21.15.47:0/1314697966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e98102060 0x7f4e981019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.470+0000 7f4e9d200640 1 -- 172.21.15.47:0/1314697966 >> 172.21.15.47:0/1314697966 conn(0x7f4e980fba10 msgr2=0x7f4e98102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:14.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.471+0000 7f4e9d200640 1 -- 172.21.15.47:0/1314697966 shutdown_connections 2024-03-21T11:46:14.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:14.471+0000 7f4e9d200640 1 -- 172.21.15.47:0/1314697966 wait complete. 2024-03-21T11:46:14.498 INFO:tasks.cephfs.filesystem:scrub status for tag:ed4db63c-c7f6-4fe4-97bf-be51fadb8c30 - {'path': '/', 'tag': 'ed4db63c-c7f6-4fe4-97bf-be51fadb8c30', 'options': 'recursive,force'} 2024-03-21T11:46:14.985 INFO:tasks.workunit.client.0.smithi047.stdout: 10 229289 22.05 MB/sec execute 418 sec latency 70.857 ms 2024-03-21T11:46:15.542 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: pgmap v1088: 129 pgs: 129 active+clean; 667 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.3 MiB/s wr, 561 op/s 2024-03-21T11:46:15.542 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:46:15.542 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.542 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:15.542 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:15.542 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:15.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:15.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:15.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:15.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:15.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:15.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:15.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.543 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.544 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:15 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: pgmap v1088: 129 pgs: 129 active+clean; 667 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.3 MiB/s wr, 561 op/s 2024-03-21T11:46:15.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:46:15.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:15.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:15.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:15.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:15.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.568 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:15.569 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.569 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:15.569 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.569 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.569 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:15.569 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:15.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:15.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.571 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:15 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: pgmap v1088: 129 pgs: 129 active+clean; 667 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.3 MiB/s wr, 561 op/s 2024-03-21T11:46:15.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:46:15.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:15.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:15.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:15.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:15.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:15.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:15.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:15.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:15.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:15.645 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.645 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.645 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:15 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:15.985 INFO:tasks.workunit.client.0.smithi047.stdout: 10 229789 22.07 MB/sec execute 419 sec latency 47.293 ms 2024-03-21T11:46:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.211+0000 7f783a800640 1 -- 172.21.15.47:0/2749812128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7834075660 msgr2=0x7f7834073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.211+0000 7f783a800640 1 --2- 172.21.15.47:0/2749812128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7834075660 0x7f7834073ab0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f7828009900 tx=0x7f782802bcc0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.211+0000 7f783a800640 1 -- 172.21.15.47:0/2749812128 shutdown_connections 2024-03-21T11:46:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.211+0000 7f783a800640 1 --2- 172.21.15.47:0/2749812128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78341093d0 0x7f783410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.211+0000 7f783a800640 1 --2- 172.21.15.47:0/2749812128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7834074080 0x7f78340744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.211+0000 7f783a800640 1 --2- 172.21.15.47:0/2749812128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7834075660 0x7f7834073ab0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.211+0000 7f783a800640 1 -- 172.21.15.47:0/2749812128 >> 172.21.15.47:0/2749812128 conn(0x7f78340fcab0 msgr2=0x7f78340feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.211+0000 7f783a800640 1 -- 172.21.15.47:0/2749812128 shutdown_connections 2024-03-21T11:46:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.211+0000 7f783a800640 1 -- 172.21.15.47:0/2749812128 wait complete. 2024-03-21T11:46:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.212+0000 7f783a800640 1 Processor -- start 2024-03-21T11:46:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.212+0000 7f783a800640 1 -- start start 2024-03-21T11:46:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.212+0000 7f783a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7834074080 0x7f7834199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.212+0000 7f783a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7834075660 0x7f7834199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.213+0000 7f783a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78341093d0 0x7f78341a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.213+0000 7f783a800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783410f3f0 con 0x7f78341093d0 2024-03-21T11:46:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.213+0000 7f783a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783410f270 con 0x7f7834074080 2024-03-21T11:46:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.213+0000 7f783a800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783410f570 con 0x7f7834075660 2024-03-21T11:46:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.213+0000 7f7839e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78341093d0 0x7f78341a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.213+0000 7f7839e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78341093d0 0x7f78341a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36326/0 (socket says 172.21.15.47:36326) 2024-03-21T11:46:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.213+0000 7f7839e00640 1 -- 172.21.15.47:0/3573440052 learned_addr learned my addr 172.21.15.47:0/3573440052 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.213+0000 7f7839400640 1 --2- 172.21.15.47:0/3573440052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7834074080 0x7f7834199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.213+0000 7f7839e00640 1 -- 172.21.15.47:0/3573440052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7834075660 msgr2=0x7f7834199f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:46:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.213+0000 7f7839e00640 1 --2- 172.21.15.47:0/3573440052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7834075660 0x7f7834199f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.213+0000 7f7839e00640 1 -- 172.21.15.47:0/3573440052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7834074080 msgr2=0x7f7834199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.213+0000 7f7839e00640 1 --2- 172.21.15.47:0/3573440052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7834074080 0x7f7834199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.213+0000 7f7839e00640 1 -- 172.21.15.47:0/3573440052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78341a1640 con 0x7f78341093d0 2024-03-21T11:46:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.214+0000 7f7839e00640 1 --2- 172.21.15.47:0/3573440052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78341093d0 0x7f78341a0fd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f782c00eb10 tx=0x7f782c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.214+0000 7f7832000640 1 -- 172.21.15.47:0/3573440052 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c004b20 con 0x7f78341093d0 2024-03-21T11:46:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.214+0000 7f7832000640 1 -- 172.21.15.47:0/3573440052 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f782c004cc0 con 0x7f78341093d0 2024-03-21T11:46:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.214+0000 7f783a800640 1 -- 172.21.15.47:0/3573440052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78341a1930 con 0x7f78341093d0 2024-03-21T11:46:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.214+0000 7f7832000640 1 -- 172.21.15.47:0/3573440052 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c010760 con 0x7f78341093d0 2024-03-21T11:46:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.214+0000 7f783a800640 1 -- 172.21.15.47:0/3573440052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78341a1e80 con 0x7f78341093d0 2024-03-21T11:46:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.218+0000 7f783ebdf740 1 Processor -- start 2024-03-21T11:46:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.219+0000 7f7832000640 1 -- 172.21.15.47:0/3573440052 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f782c005020 con 0x7f78341093d0 2024-03-21T11:46:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.219+0000 7f783ebdf740 1 -- start start 2024-03-21T11:46:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.219+0000 7f783ebdf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b538327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.219+0000 7f783ebdf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53832da0 0x559b538378a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.219+0000 7f783ebdf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 0x559b5383a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.219+0000 7f783ebdf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b537f53b0 con 0x559b53837f10 2024-03-21T11:46:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.219+0000 7f783ebdf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b53797830 con 0x559b538323b0 2024-03-21T11:46:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.219+0000 7f783ebdf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b537405d0 con 0x559b53832da0 2024-03-21T11:46:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.220+0000 7f7839400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53832da0 0x559b538378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.220+0000 7f7838a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b538327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.220+0000 7f7839400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53832da0 0x559b538378a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60772/0 (socket says 172.21.15.47:60772) 2024-03-21T11:46:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.220+0000 7f7839400640 1 -- 172.21.15.47:0/2434959538 learned_addr learned my addr 172.21.15.47:0/2434959538 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.220+0000 7f7838a00640 1 --2- 172.21.15.47:0/2434959538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 0x559b5383a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.220+0000 7f7839400640 1 -- 172.21.15.47:0/2434959538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 msgr2=0x559b538327d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.220+0000 7f7839400640 1 --2- 172.21.15.47:0/2434959538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b538327d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.220+0000 7f7839400640 1 -- 172.21.15.47:0/2434959538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 msgr2=0x559b5383a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.220+0000 7f7839400640 1 --2- 172.21.15.47:0/2434959538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 0x559b5383a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.220+0000 7f7839400640 1 -- 172.21.15.47:0/2434959538 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b537582c0 con 0x559b53832da0 2024-03-21T11:46:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.220+0000 7f7838a00640 1 --2- 172.21.15.47:0/2434959538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b538327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.221+0000 7f7832000640 1 --2- 172.21.15.47:0/3573440052 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f781c0a2f10 0x7f781c0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.221+0000 7f7839400640 1 --2- 172.21.15.47:0/2434959538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53832da0 0x559b538378a0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f7828000c00 tx=0x7f782802c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42469c0f4a4613ea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.221+0000 7f7832000640 1 -- 172.21.15.47:0/3573440052 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f782c0cfea0 con 0x7f78341093d0 2024-03-21T11:46:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.221+0000 7f7826600640 1 -- 172.21.15.47:0/2434959538 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828043070 con 0x559b53832da0 2024-03-21T11:46:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.221+0000 7f7826600640 1 -- 172.21.15.47:0/2434959538 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7828004040 con 0x559b53832da0 2024-03-21T11:46:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.221+0000 7f7826600640 1 -- 172.21.15.47:0/2434959538 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828004c50 con 0x559b53832da0 2024-03-21T11:46:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.222+0000 7f7838a00640 1 --2- 172.21.15.47:0/3573440052 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f781c0a2f10 0x7f781c0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.222+0000 7f783ebdf740 1 -- 172.21.15.47:0/2434959538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53832da0 msgr2=0x559b538378a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.222+0000 7f783ebdf740 1 --2- 172.21.15.47:0/2434959538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53832da0 0x559b538378a0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f7828000c00 tx=0x7f782802c6f0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.223+0000 7f7838a00640 1 --2- 172.21.15.47:0/3573440052 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f781c0a2f10 0x7f781c0a53d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x559b53833600 tx=0x7f7820005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.223+0000 7f783ebdf740 1 -- 172.21.15.47:0/2434959538 shutdown_connections 2024-03-21T11:46:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.223+0000 7f783ebdf740 1 --2- 172.21.15.47:0/2434959538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 0x559b5383a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.223+0000 7f783ebdf740 1 --2- 172.21.15.47:0/2434959538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53832da0 0x559b538378a0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.223+0000 7f783ebdf740 1 --2- 172.21.15.47:0/2434959538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b538327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.223+0000 7f783ebdf740 1 -- 172.21.15.47:0/2434959538 >> 172.21.15.47:0/2434959538 conn(0x559b5382d920 msgr2=0x559b5382dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.223+0000 7f783ebdf740 1 -- 172.21.15.47:0/2434959538 shutdown_connections 2024-03-21T11:46:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.223+0000 7f783ebdf740 1 -- 172.21.15.47:0/2434959538 wait complete. 2024-03-21T11:46:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.224+0000 7f783ebdf740 1 -- start start 2024-03-21T11:46:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.224+0000 7f783ebdf740 1 Processor -- start 2024-03-21T11:46:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.224+0000 7f783ebdf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b5383c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.224+0000 7f783ebdf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53832da0 0x559b5383ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f783ebdf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 0x559b53843ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f783ebdf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b53753a20 con 0x559b53837f10 2024-03-21T11:46:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f783ebdf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b537938d0 con 0x559b538323b0 2024-03-21T11:46:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f783ebdf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5383ae60 con 0x559b53832da0 2024-03-21T11:46:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f7839400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b5383c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f783ebdf740 10 client.0 _ll_register_callbacks cb 0x559b537901c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f7839400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b5383c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41922/0 (socket says 172.21.15.47:41922) 2024-03-21T11:46:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f7839400640 1 -- 172.21.15.47:0/3669463983 learned_addr learned my addr 172.21.15.47:0/3669463983 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f7839e00640 1 --2- 172.21.15.47:0/3669463983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 0x559b53843ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f7839400640 1 -- 172.21.15.47:0/3669463983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53832da0 msgr2=0x559b5383ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f7839400640 1 --2- 172.21.15.47:0/3669463983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53832da0 0x559b5383ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f7839400640 1 -- 172.21.15.47:0/3669463983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 msgr2=0x559b53843ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f7839400640 1 --2- 172.21.15.47:0/3669463983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 0x559b53843ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f7839400640 1 -- 172.21.15.47:0/3669463983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b534ac210 con 0x559b538323b0 2024-03-21T11:46:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f7839400640 1 --2- 172.21.15.47:0/3669463983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b5383c530 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f782802c840 tx=0x7f7828004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faf965fb2b4b00c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.225+0000 7f781aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.226+0000 7f781aa00640 1 -- 172.21.15.47:0/3669463983 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828043070 con 0x559b538323b0 2024-03-21T11:46:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.226+0000 7f781aa00640 1 -- 172.21.15.47:0/3669463983 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7828004f10 con 0x559b538323b0 2024-03-21T11:46:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.226+0000 7f783ebdf740 1 -- 172.21.15.47:0/3669463983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7828004040 con 0x559b538323b0 2024-03-21T11:46:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.226+0000 7f781aa00640 1 -- 172.21.15.47:0/3669463983 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7828055050 con 0x559b538323b0 2024-03-21T11:46:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.226+0000 7f783ebdf740 10 client.32852 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.226+0000 7f783ebdf740 1 -- 172.21.15.47:0/3669463983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x559b53578620 con 0x559b538323b0 2024-03-21T11:46:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.227+0000 7f781aa00640 1 -- 172.21.15.47:0/3669463983 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7828004040 con 0x559b538323b0 2024-03-21T11:46:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.227+0000 7f781aa00640 1 -- 172.21.15.47:0/3669463983 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7828050030 con 0x559b538323b0 2024-03-21T11:46:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.227+0000 7f783ebdf740 10 client.32852 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.227+0000 7f783ebdf740 10 client.32852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.227+0000 7f783ebdf740 20 client.32852 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.227+0000 7f783ebdf740 1 --2- 172.21.15.47:0/3669463983 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b53847fa0 0x559b5384a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.227+0000 7f783ebdf740 4 client.32852 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.227+0000 7f783ebdf740 1 -- 172.21.15.47:0/3669463983 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b537a4d70 con 0x559b53847fa0 2024-03-21T11:46:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.227+0000 7f7838a00640 1 --2- 172.21.15.47:0/3669463983 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b53847fa0 0x559b5384a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.228+0000 7f7838a00640 1 --2- 172.21.15.47:0/3669463983 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b53847fa0 0x559b5384a390 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98323dd8b42e5363 server_cookie=3d09603c70f588e5 in_seq=0 out_seq=0 2024-03-21T11:46:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.228+0000 7f781aa00640 10 client.32852 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.236+0000 7f781aa00640 1 -- 172.21.15.47:0/3669463983 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559b537a4d70 con 0x559b53847fa0 2024-03-21T11:46:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.236+0000 7f781aa00640 10 client.32852 handle_command_reply: tid=0 2024-03-21T11:46:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.236+0000 7f783ebdf740 1 client.32852 shutdown 2024-03-21T11:46:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.237+0000 7f783ebdf740 1 -- 172.21.15.47:0/3669463983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 msgr2=0x559b5383c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.237+0000 7f783ebdf740 1 --2- 172.21.15.47:0/3669463983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b5383c530 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f782802c840 tx=0x7f7828004be0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.237+0000 7f783ebdf740 1 -- 172.21.15.47:0/3669463983 shutdown_connections 2024-03-21T11:46:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.237+0000 7f783ebdf740 1 --2- 172.21.15.47:0/3669463983 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b53847fa0 0x559b5384a390 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.237+0000 7f783ebdf740 1 --2- 172.21.15.47:0/3669463983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 0x559b53843ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.237+0000 7f783ebdf740 1 --2- 172.21.15.47:0/3669463983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53832da0 0x559b5383ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.237+0000 7f781aa00640 0 client.32852 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.237+0000 7f783ebdf740 1 --2- 172.21.15.47:0/3669463983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b5383c530 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.237+0000 7f783ebdf740 1 -- 172.21.15.47:0/3669463983 >> 172.21.15.47:0/3669463983 conn(0x559b5382d920 msgr2=0x559b5382cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.237+0000 7f783ebdf740 1 -- 172.21.15.47:0/3669463983 shutdown_connections 2024-03-21T11:46:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.237+0000 7f783ebdf740 1 -- 172.21.15.47:0/3669463983 wait complete. 2024-03-21T11:46:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.237+0000 7f783ebdf740 20 client.32852 trim_cache size 0 max 16384 2024-03-21T11:46:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.256+0000 7f783ebdf740 1 Processor -- start 2024-03-21T11:46:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.256+0000 7f783ebdf740 1 -- start start 2024-03-21T11:46:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.256+0000 7f783ebdf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b53831290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.256+0000 7f783ebdf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53832da0 0x559b53843a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.256+0000 7f783ebdf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53837f10 0x559b5384ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.256+0000 7f783ebdf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78341ac160 con 0x559b53832da0 2024-03-21T11:46:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.256+0000 7f783ebdf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78341adae0 con 0x559b538323b0 2024-03-21T11:46:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.256+0000 7f783ebdf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78341ac7c0 con 0x559b53837f10 2024-03-21T11:46:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.256+0000 7f7839400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b53831290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.257+0000 7f7839400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b53831290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41928/0 (socket says 172.21.15.47:41928) 2024-03-21T11:46:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.257+0000 7f7839400640 1 -- 172.21.15.47:0/2742324643 learned_addr learned my addr 172.21.15.47:0/2742324643 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.257+0000 7f7839400640 1 -- 172.21.15.47:0/2742324643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53837f10 msgr2=0x559b5384ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.257+0000 7f7839400640 1 --2- 172.21.15.47:0/2742324643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53837f10 0x559b5384ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.257+0000 7f7839400640 1 -- 172.21.15.47:0/2742324643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53832da0 msgr2=0x559b53843a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.257+0000 7f7839400640 1 --2- 172.21.15.47:0/2742324643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53832da0 0x559b53843a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.257+0000 7f7839400640 1 -- 172.21.15.47:0/2742324643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b538373b0 con 0x559b538323b0 2024-03-21T11:46:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.257+0000 7f7839400640 1 --2- 172.21.15.47:0/2742324643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b53831290 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f782802bd00 tx=0x7f782802c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c74ee4497ba2f6ad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.258+0000 7f7825c00640 1 -- 172.21.15.47:0/2742324643 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782805e070 con 0x559b538323b0 2024-03-21T11:46:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.258+0000 7f7825c00640 1 -- 172.21.15.47:0/2742324643 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7828004c50 con 0x559b538323b0 2024-03-21T11:46:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.258+0000 7f7825c00640 1 -- 172.21.15.47:0/2742324643 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828043040 con 0x559b538323b0 2024-03-21T11:46:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.259+0000 7f783ebdf740 1 -- 172.21.15.47:0/2742324643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 msgr2=0x559b53831290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.259+0000 7f783ebdf740 1 --2- 172.21.15.47:0/2742324643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b53831290 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f782802bd00 tx=0x7f782802c6f0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.259+0000 7f783ebdf740 1 -- 172.21.15.47:0/2742324643 shutdown_connections 2024-03-21T11:46:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.259+0000 7f783ebdf740 1 --2- 172.21.15.47:0/2742324643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b53837f10 0x559b5384ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.259+0000 7f783ebdf740 1 --2- 172.21.15.47:0/2742324643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53832da0 0x559b53843a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.259+0000 7f783ebdf740 1 --2- 172.21.15.47:0/2742324643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b538323b0 0x559b53831290 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.259+0000 7f783ebdf740 1 -- 172.21.15.47:0/2742324643 >> 172.21.15.47:0/2742324643 conn(0x559b5382d920 msgr2=0x559b53830b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.259+0000 7f783ebdf740 1 -- 172.21.15.47:0/2742324643 shutdown_connections 2024-03-21T11:46:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.259+0000 7f783ebdf740 1 -- 172.21.15.47:0/2742324643 wait complete. 2024-03-21T11:46:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.260+0000 7f783ebdf740 1 -- start start 2024-03-21T11:46:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.260+0000 7f783ebdf740 1 Processor -- start 2024-03-21T11:46:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.261+0000 7f783ebdf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b538323b0 0x559b53840c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.261+0000 7f783ebdf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b53832da0 0x559b53841180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.261+0000 7f783ebdf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 0x559b538420f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.261+0000 7f783ebdf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78341ad480 con 0x559b53837f10 2024-03-21T11:46:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.261+0000 7f783ebdf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78341ace20 con 0x559b53832da0 2024-03-21T11:46:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.261+0000 7f783ebdf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78341aa320 con 0x559b538323b0 2024-03-21T11:46:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.261+0000 7f783ebdf740 10 client.0 _ll_register_callbacks cb 0x7f78200094e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.261+0000 7f7839400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b538323b0 0x559b53840c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.261+0000 7f7838a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b53832da0 0x559b53841180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.261+0000 7f7838a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b53832da0 0x559b53841180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41940/0 (socket says 172.21.15.47:41940) 2024-03-21T11:46:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.261+0000 7f7838a00640 1 -- 172.21.15.47:0/270015856 learned_addr learned my addr 172.21.15.47:0/270015856 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.261+0000 7f7839e00640 1 --2- 172.21.15.47:0/270015856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 0x559b538420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.262+0000 7f7838a00640 1 -- 172.21.15.47:0/270015856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b538323b0 msgr2=0x559b53840c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.262+0000 7f7838a00640 1 --2- 172.21.15.47:0/270015856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b538323b0 0x559b53840c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.262+0000 7f7838a00640 1 -- 172.21.15.47:0/270015856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 msgr2=0x559b538420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.262+0000 7f7838a00640 1 --2- 172.21.15.47:0/270015856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 0x559b538420f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.262+0000 7f7838a00640 1 -- 172.21.15.47:0/270015856 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b538426e0 con 0x559b53832da0 2024-03-21T11:46:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.262+0000 7f7838a00640 1 --2- 172.21.15.47:0/270015856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b53832da0 0x559b53841180 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x559b5384aba0 tx=0x7f78200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d696b89528094852 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.262+0000 7f781aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.262+0000 7f781aa00640 1 -- 172.21.15.47:0/270015856 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78200272d0 con 0x559b53832da0 2024-03-21T11:46:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.263+0000 7f781aa00640 1 -- 172.21.15.47:0/270015856 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7820004020 con 0x559b53832da0 2024-03-21T11:46:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.263+0000 7f783ebdf740 1 -- 172.21.15.47:0/270015856 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7828004c50 con 0x559b53832da0 2024-03-21T11:46:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.263+0000 7f781aa00640 1 -- 172.21.15.47:0/270015856 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7820037050 con 0x559b53832da0 2024-03-21T11:46:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.263+0000 7f783ebdf740 10 client.32858 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.263+0000 7f783ebdf740 1 -- 172.21.15.47:0/270015856 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x559b53843a90 con 0x559b53832da0 2024-03-21T11:46:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.263+0000 7f781aa00640 1 -- 172.21.15.47:0/270015856 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7828004c50 con 0x559b53832da0 2024-03-21T11:46:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.263+0000 7f781aa00640 1 -- 172.21.15.47:0/270015856 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7820027520 con 0x559b53832da0 2024-03-21T11:46:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.263+0000 7f783ebdf740 10 client.32858 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.263+0000 7f783ebdf740 10 client.32858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.263+0000 7f783ebdf740 20 client.32858 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.263+0000 7f783ebdf740 1 --2- 172.21.15.47:0/270015856 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b538a3c30 0x559b538a6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.263+0000 7f783ebdf740 4 client.32858 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:46:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.263+0000 7f783ebdf740 1 -- 172.21.15.47:0/270015856 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559b5379f0c0 con 0x559b538a3c30 2024-03-21T11:46:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.264+0000 7f7839400640 1 --2- 172.21.15.47:0/270015856 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b538a3c30 0x559b538a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.264+0000 7f7839400640 1 --2- 172.21.15.47:0/270015856 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b538a3c30 0x559b538a6020 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f88249e08197c61 server_cookie=d3b1041cc6497104 in_seq=0 out_seq=0 2024-03-21T11:46:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:16.264+0000 7f781aa00640 10 client.32858 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:16.985 INFO:tasks.workunit.client.0.smithi047.stdout: 10 229952 22.03 MB/sec execute 420 sec latency 462.348 ms 2024-03-21T11:46:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.053+0000 7f781aa00640 1 -- 172.21.15.47:0/270015856 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559b5379f0c0 con 0x559b538a3c30 2024-03-21T11:46:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.053+0000 7f781aa00640 10 client.32858 handle_command_reply: tid=0 2024-03-21T11:46:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.053+0000 7f783ebdf740 1 client.32858 shutdown 2024-03-21T11:46:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.055+0000 7f783ebdf740 1 -- 172.21.15.47:0/270015856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b53832da0 msgr2=0x559b53841180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.055+0000 7f783ebdf740 1 --2- 172.21.15.47:0/270015856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b53832da0 0x559b53841180 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x559b5384aba0 tx=0x7f78200046c0 comp rx=0 tx=0).stop 2024-03-21T11:46:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.055+0000 7f783ebdf740 1 -- 172.21.15.47:0/270015856 shutdown_connections 2024-03-21T11:46:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.055+0000 7f783ebdf740 1 --2- 172.21.15.47:0/270015856 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x559b538a3c30 0x559b538a6020 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.055+0000 7f783ebdf740 1 --2- 172.21.15.47:0/270015856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559b53837f10 0x559b538420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.055+0000 7f783ebdf740 1 --2- 172.21.15.47:0/270015856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b53832da0 0x559b53841180 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.055+0000 7f783ebdf740 1 --2- 172.21.15.47:0/270015856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559b538323b0 0x559b53840c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.055+0000 7f783ebdf740 1 -- 172.21.15.47:0/270015856 >> 172.21.15.47:0/270015856 conn(0x559b5382d920 msgr2=0x559b53831410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.055+0000 7f781aa00640 0 client.32858 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.056+0000 7f783ebdf740 1 -- 172.21.15.47:0/270015856 shutdown_connections 2024-03-21T11:46:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.056+0000 7f783ebdf740 1 -- 172.21.15.47:0/270015856 wait complete. 2024-03-21T11:46:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.056+0000 7f783ebdf740 20 client.32858 trim_cache size 0 max 16384 2024-03-21T11:46:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:46:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:46:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:46:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:46:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.059+0000 7f783a800640 1 -- 172.21.15.47:0/3573440052 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f781c0a2f10 msgr2=0x7f781c0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.059+0000 7f783a800640 1 --2- 172.21.15.47:0/3573440052 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f781c0a2f10 0x7f781c0a53d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x559b53833600 tx=0x7f7820005f90 comp rx=0 tx=0).stop 2024-03-21T11:46:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.059+0000 7f783a800640 1 -- 172.21.15.47:0/3573440052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78341093d0 msgr2=0x7f78341a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.059+0000 7f783a800640 1 --2- 172.21.15.47:0/3573440052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78341093d0 0x7f78341a0fd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f782c00eb10 tx=0x7f782c00c2d0 comp rx=0 tx=0).stop 2024-03-21T11:46:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.059+0000 7f783a800640 1 -- 172.21.15.47:0/3573440052 shutdown_connections 2024-03-21T11:46:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.059+0000 7f783a800640 1 --2- 172.21.15.47:0/3573440052 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f781c0a2f10 0x7f781c0a53d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.059+0000 7f783a800640 1 --2- 172.21.15.47:0/3573440052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78341093d0 0x7f78341a0fd0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.059+0000 7f783a800640 1 --2- 172.21.15.47:0/3573440052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7834075660 0x7f7834199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.059+0000 7f783a800640 1 --2- 172.21.15.47:0/3573440052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7834074080 0x7f7834199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:17.060 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.059+0000 7f783a800640 1 -- 172.21.15.47:0/3573440052 >> 172.21.15.47:0/3573440052 conn(0x7f78340fcab0 msgr2=0x7f7834105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.060+0000 7f783a800640 1 -- 172.21.15.47:0/3573440052 shutdown_connections 2024-03-21T11:46:17.061 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:17.060+0000 7f783a800640 1 -- 172.21.15.47:0/3573440052 wait complete. 2024-03-21T11:46:17.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:17 smithi047 ceph-mon[31351]: pgmap v1089: 129 pgs: 129 active+clean; 673 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.0 MiB/s wr, 604 op/s 2024-03-21T11:46:17.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:17 smithi100 ceph-mon[36245]: pgmap v1089: 129 pgs: 129 active+clean; 673 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.0 MiB/s wr, 604 op/s 2024-03-21T11:46:17.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:17 smithi102 ceph-mon[36748]: pgmap v1089: 129 pgs: 129 active+clean; 673 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.0 MiB/s wr, 604 op/s 2024-03-21T11:46:17.985 INFO:tasks.workunit.client.0.smithi047.stdout: 10 230328 22.02 MB/sec execute 421 sec latency 430.662 ms 2024-03-21T11:46:18.985 INFO:tasks.workunit.client.0.smithi047.stdout: 10 230350 21.97 MB/sec execute 422 sec latency 1011.911 ms 2024-03-21T11:46:19.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:19 smithi047 ceph-mon[31351]: pgmap v1090: 129 pgs: 129 active+clean; 676 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.6 MiB/s wr, 483 op/s 2024-03-21T11:46:19.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:19 smithi100 ceph-mon[36245]: pgmap v1090: 129 pgs: 129 active+clean; 676 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.6 MiB/s wr, 483 op/s 2024-03-21T11:46:19.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:19 smithi102 ceph-mon[36748]: pgmap v1090: 129 pgs: 129 active+clean; 676 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.6 MiB/s wr, 483 op/s 2024-03-21T11:46:19.986 INFO:tasks.workunit.client.0.smithi047.stdout: 10 230478 21.93 MB/sec execute 423 sec latency 1608.338 ms 2024-03-21T11:46:20.986 INFO:tasks.workunit.client.0.smithi047.stdout: 10 230629 21.89 MB/sec execute 424 sec latency 219.601 ms 2024-03-21T11:46:21.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:21 smithi047 ceph-mon[31351]: pgmap v1091: 129 pgs: 129 active+clean; 682 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.3 MiB/s wr, 496 op/s 2024-03-21T11:46:21.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:21 smithi100 ceph-mon[36245]: pgmap v1091: 129 pgs: 129 active+clean; 682 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.3 MiB/s wr, 496 op/s 2024-03-21T11:46:21.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:21 smithi102 ceph-mon[36748]: pgmap v1091: 129 pgs: 129 active+clean; 682 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.3 MiB/s wr, 496 op/s 2024-03-21T11:46:21.986 INFO:tasks.workunit.client.0.smithi047.stdout: 10 230831 21.85 MB/sec execute 425 sec latency 192.082 ms 2024-03-21T11:46:22.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:22 smithi047 ceph-mon[31351]: pgmap v1092: 129 pgs: 129 active+clean; 712 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-03-21T11:46:22.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:22 smithi100 ceph-mon[36245]: pgmap v1092: 129 pgs: 129 active+clean; 712 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-03-21T11:46:22.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:22 smithi102 ceph-mon[36748]: pgmap v1092: 129 pgs: 129 active+clean; 712 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-03-21T11:46:22.986 INFO:tasks.workunit.client.0.smithi047.stdout: 10 231096 21.84 MB/sec execute 426 sec latency 196.215 ms 2024-03-21T11:46:23.986 INFO:tasks.workunit.client.0.smithi047.stdout: 10 231666 21.86 MB/sec execute 427 sec latency 80.126 ms 2024-03-21T11:46:24.986 INFO:tasks.workunit.client.0.smithi047.stdout: 10 232334 21.90 MB/sec execute 428 sec latency 67.924 ms 2024-03-21T11:46:25.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:24 smithi047 ceph-mon[31351]: pgmap v1093: 129 pgs: 129 active+clean; 713 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.1 MiB/s wr, 558 op/s 2024-03-21T11:46:25.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:24 smithi100 ceph-mon[36245]: pgmap v1093: 129 pgs: 129 active+clean; 713 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.1 MiB/s wr, 558 op/s 2024-03-21T11:46:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:24 smithi102 ceph-mon[36748]: pgmap v1093: 129 pgs: 129 active+clean; 713 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.1 MiB/s wr, 558 op/s 2024-03-21T11:46:25.986 INFO:tasks.workunit.client.0.smithi047.stdout: 10 232839 21.93 MB/sec execute 429 sec latency 90.076 ms 2024-03-21T11:46:26.987 INFO:tasks.workunit.client.0.smithi047.stdout: 10 233229 21.94 MB/sec execute 430 sec latency 73.891 ms 2024-03-21T11:46:27.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:26 smithi047 ceph-mon[31351]: pgmap v1094: 129 pgs: 129 active+clean; 675 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.6 MiB/s wr, 639 op/s 2024-03-21T11:46:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:26 smithi100 ceph-mon[36245]: pgmap v1094: 129 pgs: 129 active+clean; 675 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.6 MiB/s wr, 639 op/s 2024-03-21T11:46:27.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:26 smithi102 ceph-mon[36748]: pgmap v1094: 129 pgs: 129 active+clean; 675 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.6 MiB/s wr, 639 op/s 2024-03-21T11:46:27.987 INFO:tasks.workunit.client.0.smithi047.stdout: 10 233549 21.92 MB/sec execute 431 sec latency 94.272 ms 2024-03-21T11:46:28.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:27 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:46:28.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:27 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:46:28.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:27 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:46:28.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:27 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:46:28.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:27 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:46:28.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:27 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:46:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:27 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:46:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:27 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:46:28.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:27 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:46:28.987 INFO:tasks.workunit.client.0.smithi047.stdout: 10 233897 21.91 MB/sec execute 432 sec latency 86.325 ms 2024-03-21T11:46:29.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:28 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:46:29.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:28 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:46:29.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:28 smithi100 ceph-mon[36245]: pgmap v1095: 129 pgs: 129 active+clean; 673 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.0 MiB/s wr, 550 op/s 2024-03-21T11:46:29.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:28 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:46:29.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:28 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:46:29.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:28 smithi102 ceph-mon[36748]: pgmap v1095: 129 pgs: 129 active+clean; 673 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.0 MiB/s wr, 550 op/s 2024-03-21T11:46:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:28 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:46:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:28 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:46:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:28 smithi047 ceph-mon[31351]: pgmap v1095: 129 pgs: 129 active+clean; 673 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.0 MiB/s wr, 550 op/s 2024-03-21T11:46:29.987 INFO:tasks.workunit.client.0.smithi047.stdout: 10 234306 21.89 MB/sec execute 433 sec latency 90.445 ms 2024-03-21T11:46:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.280+0000 7f7ca9400640 1 -- 172.21.15.47:0/38789870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ca4101fa0 msgr2=0x7f7ca4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.280+0000 7f7ca9400640 1 --2- 172.21.15.47:0/38789870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ca4101fa0 0x7f7ca4108830 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009ca0 tx=0x7f7c9c02c100 comp rx=0 tx=0).stop 2024-03-21T11:46:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.280+0000 7f7ca9400640 1 -- 172.21.15.47:0/38789870 shutdown_connections 2024-03-21T11:46:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.280+0000 7f7ca9400640 1 --2- 172.21.15.47:0/38789870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ca4101fa0 0x7f7ca4108830 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.280+0000 7f7ca9400640 1 --2- 172.21.15.47:0/38789870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ca41015e0 0x7f7ca4101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.280+0000 7f7ca9400640 1 --2- 172.21.15.47:0/38789870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ca41003f0 0x7f7ca4100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.280+0000 7f7ca9400640 1 -- 172.21.15.47:0/38789870 >> 172.21.15.47:0/38789870 conn(0x7f7ca40fb980 msgr2=0x7f7ca40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.280+0000 7f7ca9400640 1 -- 172.21.15.47:0/38789870 shutdown_connections 2024-03-21T11:46:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.280+0000 7f7ca9400640 1 -- 172.21.15.47:0/38789870 wait complete. 2024-03-21T11:46:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.281+0000 7f7ca9400640 1 Processor -- start 2024-03-21T11:46:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.282+0000 7f7ca9400640 1 -- start start 2024-03-21T11:46:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.282+0000 7f7ca9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ca41003f0 0x7f7ca4197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.282+0000 7f7ca9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ca41015e0 0x7f7ca4197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.282+0000 7f7ca9400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ca4101fa0 0x7f7ca419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.282+0000 7f7ca9400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca410c080 con 0x7f7ca4101fa0 2024-03-21T11:46:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.282+0000 7f7ca9400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca410bf00 con 0x7f7ca41003f0 2024-03-21T11:46:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.282+0000 7f7ca9400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca410c200 con 0x7f7ca41015e0 2024-03-21T11:46:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.282+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ca41003f0 0x7f7ca4197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.282+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ca41003f0 0x7f7ca4197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56698/0 (socket says 172.21.15.47:56698) 2024-03-21T11:46:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.282+0000 7f7ca3e00640 1 -- 172.21.15.47:0/3523545960 learned_addr learned my addr 172.21.15.47:0/3523545960 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.282+0000 7f7ca8a00640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ca4101fa0 0x7f7ca419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.282+0000 7f7c9be00640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ca41015e0 0x7f7ca4197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.283+0000 7f7ca8a00640 1 -- 172.21.15.47:0/3523545960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ca41015e0 msgr2=0x7f7ca4197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.283+0000 7f7ca8a00640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ca41015e0 0x7f7ca4197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.283+0000 7f7ca8a00640 1 -- 172.21.15.47:0/3523545960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ca41003f0 msgr2=0x7f7ca4197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.283+0000 7f7ca8a00640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ca41003f0 0x7f7ca4197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.283+0000 7f7ca8a00640 1 -- 172.21.15.47:0/3523545960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca419f410 con 0x7f7ca4101fa0 2024-03-21T11:46:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.283+0000 7f7ca3e00640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ca41003f0 0x7f7ca4197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.283+0000 7f7ca8a00640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ca4101fa0 0x7f7ca419eda0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c037040 tx=0x7f7c9c035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.284+0000 7f7ca1600640 1 -- 172.21.15.47:0/3523545960 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c039040 con 0x7f7ca4101fa0 2024-03-21T11:46:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.284+0000 7f7ca9400640 1 -- 172.21.15.47:0/3523545960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca419f6a0 con 0x7f7ca4101fa0 2024-03-21T11:46:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.284+0000 7f7ca1600640 1 -- 172.21.15.47:0/3523545960 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c9c044070 con 0x7f7ca4101fa0 2024-03-21T11:46:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.284+0000 7f7ca1600640 1 -- 172.21.15.47:0/3523545960 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c004350 con 0x7f7ca4101fa0 2024-03-21T11:46:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.284+0000 7f7ca9400640 1 -- 172.21.15.47:0/3523545960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca419fbb0 con 0x7f7ca4101fa0 2024-03-21T11:46:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.288+0000 7f7cad7b0740 1 Processor -- start 2024-03-21T11:46:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.289+0000 7f7ca1600640 1 -- 172.21.15.47:0/3523545960 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7c9c0044f0 con 0x7f7ca4101fa0 2024-03-21T11:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.289+0000 7f7cad7b0740 1 -- start start 2024-03-21T11:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.289+0000 7f7cad7b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe83610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7cad7b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe83be0 0x55abfbe88170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7cad7b0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe887e0 0x55abfbe8ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7cad7b0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abfbe3d3b0 con 0x55abfbe887e0 2024-03-21T11:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7cad7b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abfbddf830 con 0x55abfbe831f0 2024-03-21T11:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7cad7b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abfbd885d0 con 0x55abfbe83be0 2024-03-21T11:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe83be0 0x55abfbe88170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe83be0 0x55abfbe88170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43310/0 (socket says 172.21.15.47:43310) 2024-03-21T11:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7ca3e00640 1 -- 172.21.15.47:0/3809968059 learned_addr learned my addr 172.21.15.47:0/3809968059 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7c9be00640 1 --2- 172.21.15.47:0/3809968059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe83610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7c9be00640 1 --2- 172.21.15.47:0/3809968059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe887e0 0x55abfbe8ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7ca3e00640 1 -- 172.21.15.47:0/3809968059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 msgr2=0x55abfbe83610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7ca3e00640 1 --2- 172.21.15.47:0/3809968059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe83610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7ca3e00640 1 -- 172.21.15.47:0/3809968059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe887e0 msgr2=0x55abfbe8ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7ca3e00640 1 --2- 172.21.15.47:0/3809968059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe887e0 0x55abfbe8ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7ca3e00640 1 -- 172.21.15.47:0/3809968059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abfbda02c0 con 0x55abfbe83be0 2024-03-21T11:46:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.290+0000 7f7c9be00640 1 --2- 172.21.15.47:0/3809968059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe887e0 0x55abfbe8ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.291+0000 7f7ca3e00640 1 --2- 172.21.15.47:0/3809968059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe83be0 0x55abfbe88170 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f7c940115f0 tx=0x7f7c94011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d265bdd6e1a3c43 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.291+0000 7f7ca1600640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7c880a2f10 0x7f7c880a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.291+0000 7f7c99200640 1 -- 172.21.15.47:0/3809968059 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c94004020 con 0x55abfbe83be0 2024-03-21T11:46:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.291+0000 7f7c99200640 1 -- 172.21.15.47:0/3809968059 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c940041c0 con 0x55abfbe83be0 2024-03-21T11:46:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.291+0000 7f7c99200640 1 -- 172.21.15.47:0/3809968059 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940039d0 con 0x55abfbe83be0 2024-03-21T11:46:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.291+0000 7f7ca1600640 1 -- 172.21.15.47:0/3523545960 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7c9c0f1cd0 con 0x7f7ca4101fa0 2024-03-21T11:46:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.291+0000 7f7c9be00640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7c880a2f10 0x7f7c880a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.292+0000 7f7cad7b0740 1 -- 172.21.15.47:0/3809968059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe83be0 msgr2=0x55abfbe88170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.292+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/3809968059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe83be0 0x55abfbe88170 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f7c940115f0 tx=0x7f7c94011ab0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.292+0000 7f7cad7b0740 1 -- 172.21.15.47:0/3809968059 shutdown_connections 2024-03-21T11:46:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.292+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/3809968059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe887e0 0x55abfbe8ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.292+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/3809968059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe83be0 0x55abfbe88170 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.292+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/3809968059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe83610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.292+0000 7f7c9be00640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7c880a2f10 0x7f7c880a53d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x55abfbe795a0 tx=0x7f7c8c013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.292+0000 7f7cad7b0740 1 -- 172.21.15.47:0/3809968059 >> 172.21.15.47:0/3809968059 conn(0x55abfbe758e0 msgr2=0x55abfbe75ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.292+0000 7f7cad7b0740 1 -- 172.21.15.47:0/3809968059 shutdown_connections 2024-03-21T11:46:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.292+0000 7f7cad7b0740 1 -- 172.21.15.47:0/3809968059 wait complete. 2024-03-21T11:46:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.292+0000 7f7cad7b0740 1 -- start start 2024-03-21T11:46:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.293+0000 7f7cad7b0740 1 Processor -- start 2024-03-21T11:46:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.293+0000 7f7cad7b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe8cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.293+0000 7f7cad7b0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe83be0 0x55abfbe8d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.293+0000 7f7cad7b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe887e0 0x55abfbe94320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.293+0000 7f7cad7b0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abfbd9ba20 con 0x55abfbe83be0 2024-03-21T11:46:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.293+0000 7f7cad7b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abfbddb8d0 con 0x55abfbe831f0 2024-03-21T11:46:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.293+0000 7f7cad7b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abfbe8b770 con 0x55abfbe887e0 2024-03-21T11:46:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.293+0000 7f7cad7b0740 10 client.0 _ll_register_callbacks cb 0x55abfbdd81c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.293+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe8cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.294+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe8cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56730/0 (socket says 172.21.15.47:56730) 2024-03-21T11:46:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.294+0000 7f7ca3e00640 1 -- 172.21.15.47:0/2614404882 learned_addr learned my addr 172.21.15.47:0/2614404882 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.294+0000 7f7ca8a00640 1 --2- 172.21.15.47:0/2614404882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe887e0 0x55abfbe94320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.294+0000 7f7c9be00640 1 --2- 172.21.15.47:0/2614404882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe83be0 0x55abfbe8d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.294+0000 7f7ca3e00640 1 -- 172.21.15.47:0/2614404882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe887e0 msgr2=0x55abfbe94320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.294+0000 7f7ca3e00640 1 --2- 172.21.15.47:0/2614404882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe887e0 0x55abfbe94320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.294+0000 7f7ca3e00640 1 -- 172.21.15.47:0/2614404882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe83be0 msgr2=0x55abfbe8d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.294+0000 7f7ca3e00640 1 --2- 172.21.15.47:0/2614404882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe83be0 0x55abfbe8d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.294+0000 7f7ca3e00640 1 -- 172.21.15.47:0/2614404882 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abfbaf4210 con 0x55abfbe831f0 2024-03-21T11:46:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.294+0000 7f7ca3e00640 1 --2- 172.21.15.47:0/2614404882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe8cda0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f7c9401b040 tx=0x7f7c940045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd97559afe908397 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.294+0000 7f7c91600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.295+0000 7f7c91600640 1 -- 172.21.15.47:0/2614404882 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c94027020 con 0x55abfbe831f0 2024-03-21T11:46:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.295+0000 7f7c91600640 1 -- 172.21.15.47:0/2614404882 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7c94016040 con 0x55abfbe831f0 2024-03-21T11:46:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.295+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2614404882 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c940041c0 con 0x55abfbe831f0 2024-03-21T11:46:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.295+0000 7f7c91600640 1 -- 172.21.15.47:0/2614404882 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c94032050 con 0x55abfbe831f0 2024-03-21T11:46:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.295+0000 7f7cad7b0740 10 client.32870 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.295+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2614404882 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55abfbbc0620 con 0x55abfbe831f0 2024-03-21T11:46:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.296+0000 7f7c91600640 1 -- 172.21.15.47:0/2614404882 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c940041c0 con 0x55abfbe831f0 2024-03-21T11:46:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.296+0000 7f7c91600640 1 -- 172.21.15.47:0/2614404882 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7c940036d0 con 0x55abfbe831f0 2024-03-21T11:46:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.296+0000 7f7cad7b0740 10 client.32870 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.296+0000 7f7cad7b0740 10 client.32870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.296+0000 7f7cad7b0740 20 client.32870 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.296+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2614404882 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55abfbe989e0 0x55abfbe9ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.296+0000 7f7cad7b0740 4 client.32870 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.296+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2614404882 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abfbdecd70 con 0x55abfbe989e0 2024-03-21T11:46:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.297+0000 7f7c9be00640 1 --2- 172.21.15.47:0/2614404882 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55abfbe989e0 0x55abfbe9ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.297+0000 7f7c9be00640 1 --2- 172.21.15.47:0/2614404882 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55abfbe989e0 0x55abfbe9ae10 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7d9ffa712d914a3 server_cookie=7415e0d9c0644c56 in_seq=0 out_seq=0 2024-03-21T11:46:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.297+0000 7f7c91600640 10 client.32870 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.299+0000 7f7c91600640 1 -- 172.21.15.47:0/2614404882 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55abfbdecd70 con 0x55abfbe989e0 2024-03-21T11:46:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.299+0000 7f7c91600640 10 client.32870 handle_command_reply: tid=0 2024-03-21T11:46:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.299+0000 7f7cad7b0740 1 client.32870 shutdown 2024-03-21T11:46:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.299+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2614404882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 msgr2=0x55abfbe8cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.299+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2614404882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe8cda0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f7c9401b040 tx=0x7f7c940045a0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.299+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2614404882 shutdown_connections 2024-03-21T11:46:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.299+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2614404882 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55abfbe989e0 0x55abfbe9ae10 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.299+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2614404882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe887e0 0x55abfbe94320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.299+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2614404882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe83be0 0x55abfbe8d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.299+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2614404882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe8cda0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.300+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2614404882 >> 172.21.15.47:0/2614404882 conn(0x55abfbe758e0 msgr2=0x55abfbe82880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.300+0000 7f7c91600640 0 client.32870 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.300+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2614404882 shutdown_connections 2024-03-21T11:46:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.300+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2614404882 wait complete. 2024-03-21T11:46:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.301+0000 7f7cad7b0740 20 client.32870 trim_cache size 0 max 16384 2024-03-21T11:46:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.320+0000 7f7cad7b0740 1 Processor -- start 2024-03-21T11:46:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.320+0000 7f7cad7b0740 1 -- start start 2024-03-21T11:46:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7cad7b0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe758e0 0x55abfbe82700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7cad7b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe94240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7cad7b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe83be0 0x55abfbe9f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7cad7b0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca41a7ca0 con 0x55abfbe758e0 2024-03-21T11:46:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7cad7b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca41a9620 con 0x55abfbe831f0 2024-03-21T11:46:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7cad7b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca41a8300 con 0x55abfbe83be0 2024-03-21T11:46:30.323 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7c9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe94240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.323 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe758e0 0x55abfbe82700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.323 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7c9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe94240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56738/0 (socket says 172.21.15.47:56738) 2024-03-21T11:46:30.323 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7c9be00640 1 -- 172.21.15.47:0/2029504896 learned_addr learned my addr 172.21.15.47:0/2029504896 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7ca8a00640 1 --2- 172.21.15.47:0/2029504896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe83be0 0x55abfbe9f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7c9be00640 1 -- 172.21.15.47:0/2029504896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe83be0 msgr2=0x55abfbe9f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7c9be00640 1 --2- 172.21.15.47:0/2029504896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe83be0 0x55abfbe9f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7c9be00640 1 -- 172.21.15.47:0/2029504896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe758e0 msgr2=0x55abfbe82700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.321+0000 7f7c9be00640 1 --2- 172.21.15.47:0/2029504896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe758e0 0x55abfbe82700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.322+0000 7f7c9be00640 1 -- 172.21.15.47:0/2029504896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abfbe70d10 con 0x55abfbe831f0 2024-03-21T11:46:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.322+0000 7f7ca8a00640 1 --2- 172.21.15.47:0/2029504896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe83be0 0x55abfbe9f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.322+0000 7f7c9be00640 1 --2- 172.21.15.47:0/2029504896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe94240 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009db0 tx=0x7f7c8c009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cca08a234d205a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.322+0000 7f7c93e00640 1 -- 172.21.15.47:0/2029504896 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c02d050 con 0x55abfbe831f0 2024-03-21T11:46:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.322+0000 7f7c93e00640 1 -- 172.21.15.47:0/2029504896 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c8c02d1f0 con 0x55abfbe831f0 2024-03-21T11:46:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.323+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2029504896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 msgr2=0x55abfbe94240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.323+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2029504896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe94240 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009db0 tx=0x7f7c8c009770 comp rx=0 tx=0).stop 2024-03-21T11:46:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.323+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2029504896 shutdown_connections 2024-03-21T11:46:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.323+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2029504896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe83be0 0x55abfbe9f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.323+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2029504896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe94240 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.323+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2029504896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe758e0 0x55abfbe82700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.323+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2029504896 >> 172.21.15.47:0/2029504896 conn(0x55abfbe882e0 msgr2=0x55abfbe81ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.323+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2029504896 shutdown_connections 2024-03-21T11:46:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.324+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2029504896 wait complete. 2024-03-21T11:46:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.324+0000 7f7cad7b0740 1 -- start start 2024-03-21T11:46:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.324+0000 7f7cad7b0740 1 Processor -- start 2024-03-21T11:46:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.324+0000 7f7cad7b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe758e0 0x55abfbe91950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.324+0000 7f7cad7b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe91e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7cad7b0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe83be0 0x55abfbe92ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7cad7b0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca41a8fc0 con 0x55abfbe83be0 2024-03-21T11:46:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7cad7b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca41a8960 con 0x55abfbe831f0 2024-03-21T11:46:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7cad7b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca41a5e60 con 0x55abfbe758e0 2024-03-21T11:46:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7cad7b0740 10 client.0 _ll_register_callbacks cb 0x7f7c8c009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7c9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe91e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe758e0 0x55abfbe91950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7c9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe91e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56744/0 (socket says 172.21.15.47:56744) 2024-03-21T11:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe758e0 0x55abfbe91950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:43346/0 (socket says 172.21.15.47:43346) 2024-03-21T11:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7c9be00640 1 -- 172.21.15.47:0/2229151394 learned_addr learned my addr 172.21.15.47:0/2229151394 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7ca8a00640 1 --2- 172.21.15.47:0/2229151394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe83be0 0x55abfbe92ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7c9be00640 1 -- 172.21.15.47:0/2229151394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe758e0 msgr2=0x55abfbe91950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7c9be00640 1 --2- 172.21.15.47:0/2229151394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe758e0 0x55abfbe91950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7c9be00640 1 -- 172.21.15.47:0/2229151394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe83be0 msgr2=0x55abfbe92ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7c9be00640 1 --2- 172.21.15.47:0/2229151394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe83be0 0x55abfbe92ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7c9be00640 1 -- 172.21.15.47:0/2229151394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abfbe93490 con 0x55abfbe831f0 2024-03-21T11:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7ca8a00640 1 --2- 172.21.15.47:0/2229151394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe83be0 0x55abfbe92ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.325+0000 7f7ca3e00640 1 --2- 172.21.15.47:0/2229151394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe758e0 0x55abfbe91950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.326+0000 7f7c9be00640 1 --2- 172.21.15.47:0/2229151394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe91e90 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c010010 tx=0x7f7c8c009410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=575149e65651b04b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.326+0000 7f7c91600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.326+0000 7f7c91600640 1 -- 172.21.15.47:0/2229151394 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c03b060 con 0x55abfbe831f0 2024-03-21T11:46:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.326+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2229151394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c8c02d1f0 con 0x55abfbe831f0 2024-03-21T11:46:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.326+0000 7f7c91600640 1 -- 172.21.15.47:0/2229151394 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7c8c009040 con 0x55abfbe831f0 2024-03-21T11:46:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.326+0000 7f7c91600640 1 -- 172.21.15.47:0/2229151394 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c8c042050 con 0x55abfbe831f0 2024-03-21T11:46:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.326+0000 7f7cad7b0740 10 client.32876 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.326+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2229151394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55abfbef42a0 con 0x55abfbe831f0 2024-03-21T11:46:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.327+0000 7f7c91600640 1 -- 172.21.15.47:0/2229151394 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c8c02d1f0 con 0x55abfbe831f0 2024-03-21T11:46:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.327+0000 7f7c91600640 1 -- 172.21.15.47:0/2229151394 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7c8c01f430 con 0x55abfbe831f0 2024-03-21T11:46:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.327+0000 7f7cad7b0740 10 client.32876 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.327+0000 7f7cad7b0740 10 client.32876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.327+0000 7f7cad7b0740 20 client.32876 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.327+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2229151394 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55abfbe887e0 0x55abfbef6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.327+0000 7f7cad7b0740 4 client.32876 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:46:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.327+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2229151394 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55abfbde70c0 con 0x55abfbe887e0 2024-03-21T11:46:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.327+0000 7f7ca3e00640 1 --2- 172.21.15.47:0/2229151394 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55abfbe887e0 0x55abfbef6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.328+0000 7f7ca3e00640 1 --2- 172.21.15.47:0/2229151394 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55abfbe887e0 0x55abfbef6b20 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ba75818809f218 server_cookie=5ef63d9574b2b5b8 in_seq=0 out_seq=0 2024-03-21T11:46:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:30.328+0000 7f7c91600640 10 client.32876 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:30.987 INFO:tasks.workunit.client.0.smithi047.stdout: 10 234575 21.88 MB/sec execute 434 sec latency 471.358 ms 2024-03-21T11:46:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.025+0000 7f7c91600640 1 -- 172.21.15.47:0/2229151394 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55abfbde70c0 con 0x55abfbe887e0 2024-03-21T11:46:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.025+0000 7f7c91600640 10 client.32876 handle_command_reply: tid=0 2024-03-21T11:46:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.025+0000 7f7cad7b0740 1 client.32876 shutdown 2024-03-21T11:46:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.026+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2229151394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 msgr2=0x55abfbe91e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.026+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2229151394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe91e90 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c010010 tx=0x7f7c8c009410 comp rx=0 tx=0).stop 2024-03-21T11:46:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.029+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2229151394 shutdown_connections 2024-03-21T11:46:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.029+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2229151394 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55abfbe887e0 0x55abfbef6b20 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.029+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2229151394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55abfbe83be0 0x55abfbe92ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.029+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2229151394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55abfbe831f0 0x55abfbe91e90 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.029+0000 7f7cad7b0740 1 --2- 172.21.15.47:0/2229151394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abfbe758e0 0x55abfbe91950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.029+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2229151394 >> 172.21.15.47:0/2229151394 conn(0x55abfbe882e0 msgr2=0x55abfbe742e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.029+0000 7f7c91600640 0 client.32876 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.030+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2229151394 shutdown_connections 2024-03-21T11:46:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.030+0000 7f7cad7b0740 1 -- 172.21.15.47:0/2229151394 wait complete. 2024-03-21T11:46:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.030+0000 7f7cad7b0740 20 client.32876 trim_cache size 0 max 16384 2024-03-21T11:46:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:46:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:46:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:46:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:46:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.032+0000 7f7ca9400640 1 -- 172.21.15.47:0/3523545960 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7c880a2f10 msgr2=0x7f7c880a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.032+0000 7f7ca9400640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7c880a2f10 0x7f7c880a53d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x55abfbe795a0 tx=0x7f7c8c013040 comp rx=0 tx=0).stop 2024-03-21T11:46:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.032+0000 7f7ca9400640 1 -- 172.21.15.47:0/3523545960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ca4101fa0 msgr2=0x7f7ca419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.032+0000 7f7ca9400640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ca4101fa0 0x7f7ca419eda0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c037040 tx=0x7f7c9c035d10 comp rx=0 tx=0).stop 2024-03-21T11:46:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.033+0000 7f7ca9400640 1 -- 172.21.15.47:0/3523545960 shutdown_connections 2024-03-21T11:46:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.034+0000 7f7ca9400640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7c880a2f10 0x7f7c880a53d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.034+0000 7f7ca9400640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ca4101fa0 0x7f7ca419eda0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.034+0000 7f7ca9400640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ca41015e0 0x7f7ca4197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.034+0000 7f7ca9400640 1 --2- 172.21.15.47:0/3523545960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ca41003f0 0x7f7ca4197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.034+0000 7f7ca9400640 1 -- 172.21.15.47:0/3523545960 >> 172.21.15.47:0/3523545960 conn(0x7f7ca40fb980 msgr2=0x7f7ca4068d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.034+0000 7f7ca9400640 1 -- 172.21.15.47:0/3523545960 shutdown_connections 2024-03-21T11:46:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:31.034+0000 7f7ca9400640 1 -- 172.21.15.47:0/3523545960 wait complete. 2024-03-21T11:46:31.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:30 smithi100 ceph-mon[36245]: pgmap v1096: 129 pgs: 129 active+clean; 677 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.0 MiB/s wr, 529 op/s 2024-03-21T11:46:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:30 smithi102 ceph-mon[36748]: pgmap v1096: 129 pgs: 129 active+clean; 677 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.0 MiB/s wr, 529 op/s 2024-03-21T11:46:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:30 smithi047 ceph-mon[31351]: pgmap v1096: 129 pgs: 129 active+clean; 677 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.0 MiB/s wr, 529 op/s 2024-03-21T11:46:31.987 INFO:tasks.workunit.client.0.smithi047.stdout: 10 235098 21.88 MB/sec execute 435 sec latency 211.664 ms 2024-03-21T11:46:32.987 INFO:tasks.workunit.client.0.smithi047.stdout: 10 235134 21.83 MB/sec execute 436 sec latency 1033.835 ms 2024-03-21T11:46:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:32 smithi100 ceph-mon[36245]: pgmap v1097: 129 pgs: 129 active+clean; 683 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.8 MiB/s wr, 569 op/s 2024-03-21T11:46:33.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:32 smithi102 ceph-mon[36748]: pgmap v1097: 129 pgs: 129 active+clean; 683 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.8 MiB/s wr, 569 op/s 2024-03-21T11:46:33.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:32 smithi047 ceph-mon[31351]: pgmap v1097: 129 pgs: 129 active+clean; 683 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.8 MiB/s wr, 569 op/s 2024-03-21T11:46:33.988 INFO:tasks.workunit.client.0.smithi047.stdout: 10 235294 21.80 MB/sec execute 437 sec latency 1390.640 ms 2024-03-21T11:46:34.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:33 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:46:34.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:33 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:46:34.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:33 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:46:34.987 INFO:tasks.workunit.client.0.smithi047.stdout: 10 235537 21.79 MB/sec execute 438 sec latency 256.252 ms 2024-03-21T11:46:35.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:34 smithi100 ceph-mon[36245]: pgmap v1098: 129 pgs: 129 active+clean; 689 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.3 MiB/s wr, 483 op/s 2024-03-21T11:46:35.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:34 smithi047 ceph-mon[31351]: pgmap v1098: 129 pgs: 129 active+clean; 689 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.3 MiB/s wr, 483 op/s 2024-03-21T11:46:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:34 smithi102 ceph-mon[36748]: pgmap v1098: 129 pgs: 129 active+clean; 689 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.3 MiB/s wr, 483 op/s 2024-03-21T11:46:35.988 INFO:tasks.workunit.client.0.smithi047.stdout: 10 235980 21.80 MB/sec execute 439 sec latency 154.840 ms 2024-03-21T11:46:36.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:35 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:46:36.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:35 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:46:36.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:35 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:46:36.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:35 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:46:36.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:35 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:46:36.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:35 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:46:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:46:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:46:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:46:36.988 INFO:tasks.workunit.client.0.smithi047.stdout: 10 236368 21.80 MB/sec execute 440 sec latency 118.810 ms 2024-03-21T11:46:37.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:36 smithi100 ceph-mon[36245]: pgmap v1099: 129 pgs: 129 active+clean; 700 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.3 MiB/s wr, 540 op/s 2024-03-21T11:46:37.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:36 smithi102 ceph-mon[36748]: pgmap v1099: 129 pgs: 129 active+clean; 700 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.3 MiB/s wr, 540 op/s 2024-03-21T11:46:37.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:36 smithi047 ceph-mon[31351]: pgmap v1099: 129 pgs: 129 active+clean; 700 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.3 MiB/s wr, 540 op/s 2024-03-21T11:46:37.988 INFO:tasks.workunit.client.0.smithi047.stdout: 10 236885 21.82 MB/sec execute 441 sec latency 172.033 ms 2024-03-21T11:46:38.988 INFO:tasks.workunit.client.0.smithi047.stdout: 10 237198 21.82 MB/sec execute 442 sec latency 142.700 ms 2024-03-21T11:46:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:38 smithi100 ceph-mon[36245]: pgmap v1100: 129 pgs: 129 active+clean; 696 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.2 MiB/s wr, 445 op/s 2024-03-21T11:46:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:38 smithi102 ceph-mon[36748]: pgmap v1100: 129 pgs: 129 active+clean; 696 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.2 MiB/s wr, 445 op/s 2024-03-21T11:46:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:38 smithi047 ceph-mon[31351]: pgmap v1100: 129 pgs: 129 active+clean; 696 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.2 MiB/s wr, 445 op/s 2024-03-21T11:46:39.989 INFO:tasks.workunit.client.0.smithi047.stdout: 10 237640 21.83 MB/sec execute 443 sec latency 71.718 ms 2024-03-21T11:46:40.989 INFO:tasks.workunit.client.0.smithi047.stdout: 10 238033 21.81 MB/sec execute 444 sec latency 75.433 ms 2024-03-21T11:46:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:40 smithi100 ceph-mon[36245]: pgmap v1101: 129 pgs: 129 active+clean; 686 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 512 op/s 2024-03-21T11:46:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:40 smithi102 ceph-mon[36748]: pgmap v1101: 129 pgs: 129 active+clean; 686 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 512 op/s 2024-03-21T11:46:41.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:40 smithi047 ceph-mon[31351]: pgmap v1101: 129 pgs: 129 active+clean; 686 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 512 op/s 2024-03-21T11:46:41.989 INFO:tasks.workunit.client.0.smithi047.stdout: 10 238240 21.78 MB/sec execute 445 sec latency 492.243 ms 2024-03-21T11:46:42.989 INFO:tasks.workunit.client.0.smithi047.stdout: 10 238407 21.75 MB/sec execute 446 sec latency 510.420 ms 2024-03-21T11:46:43.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:42 smithi100 ceph-mon[36245]: pgmap v1102: 129 pgs: 129 active+clean; 685 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.0 MiB/s wr, 555 op/s 2024-03-21T11:46:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:42 smithi102 ceph-mon[36748]: pgmap v1102: 129 pgs: 129 active+clean; 685 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.0 MiB/s wr, 555 op/s 2024-03-21T11:46:43.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:42 smithi047 ceph-mon[31351]: pgmap v1102: 129 pgs: 129 active+clean; 685 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.0 MiB/s wr, 555 op/s 2024-03-21T11:46:43.989 INFO:tasks.workunit.client.0.smithi047.stdout: 10 238650 21.73 MB/sec execute 447 sec latency 173.660 ms 2024-03-21T11:46:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.200+0000 7eff76000640 1 -- 172.21.15.47:0/1594859046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff701006c0 msgr2=0x7eff70100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.200+0000 7eff76000640 1 --2- 172.21.15.47:0/1594859046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff701006c0 0x7eff70100b20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7eff58009900 tx=0x7eff5802bc40 comp rx=0 tx=0).stop 2024-03-21T11:46:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.200+0000 7eff76000640 1 -- 172.21.15.47:0/1594859046 shutdown_connections 2024-03-21T11:46:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.200+0000 7eff76000640 1 --2- 172.21.15.47:0/1594859046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff70101060 0x7eff701014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.200+0000 7eff76000640 1 --2- 172.21.15.47:0/1594859046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff701006c0 0x7eff70100b20 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.200+0000 7eff76000640 1 --2- 172.21.15.47:0/1594859046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff700ffd60 0x7eff70100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.200+0000 7eff76000640 1 -- 172.21.15.47:0/1594859046 >> 172.21.15.47:0/1594859046 conn(0x7eff700fb980 msgr2=0x7eff700fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.200+0000 7eff76000640 1 -- 172.21.15.47:0/1594859046 shutdown_connections 2024-03-21T11:46:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.200+0000 7eff76000640 1 -- 172.21.15.47:0/1594859046 wait complete. 2024-03-21T11:46:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.201+0000 7eff76000640 1 Processor -- start 2024-03-21T11:46:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.202+0000 7eff76000640 1 -- start start 2024-03-21T11:46:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.202+0000 7eff76000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff700ffd60 0x7eff70104760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.202+0000 7eff76000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff701006c0 0x7eff70104ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.202+0000 7eff76000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff70101060 0x7eff7010bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.202+0000 7eff76000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7010f330 con 0x7eff701006c0 2024-03-21T11:46:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.202+0000 7eff76000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7010f1b0 con 0x7eff700ffd60 2024-03-21T11:46:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.202+0000 7eff76000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7010f4b0 con 0x7eff70101060 2024-03-21T11:46:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.202+0000 7eff74c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff700ffd60 0x7eff70104760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.202+0000 7eff74c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff700ffd60 0x7eff70104760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44052/0 (socket says 172.21.15.47:44052) 2024-03-21T11:46:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.202+0000 7eff74c00640 1 -- 172.21.15.47:0/1442154116 learned_addr learned my addr 172.21.15.47:0/1442154116 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.202+0000 7eff75600640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff70101060 0x7eff7010bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.203+0000 7eff74c00640 1 -- 172.21.15.47:0/1442154116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff70101060 msgr2=0x7eff7010bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.203+0000 7eff74c00640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff70101060 0x7eff7010bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.203+0000 7eff6fe00640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff701006c0 0x7eff70104ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.203+0000 7eff74c00640 1 -- 172.21.15.47:0/1442154116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff701006c0 msgr2=0x7eff70104ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.203+0000 7eff74c00640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff701006c0 0x7eff70104ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.203+0000 7eff74c00640 1 -- 172.21.15.47:0/1442154116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7019d220 con 0x7eff700ffd60 2024-03-21T11:46:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.203+0000 7eff75600640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff70101060 0x7eff7010bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.203+0000 7eff6fe00640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff701006c0 0x7eff70104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.204+0000 7eff74c00640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff700ffd60 0x7eff70104760 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7eff6000d980 tx=0x7eff6000de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.204+0000 7eff6d600640 1 -- 172.21.15.47:0/1442154116 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60017070 con 0x7eff700ffd60 2024-03-21T11:46:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.204+0000 7eff76000640 1 -- 172.21.15.47:0/1442154116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7019d510 con 0x7eff700ffd60 2024-03-21T11:46:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.204+0000 7eff76000640 1 -- 172.21.15.47:0/1442154116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7019dab0 con 0x7eff700ffd60 2024-03-21T11:46:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.204+0000 7eff6d600640 1 -- 172.21.15.47:0/1442154116 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff60009c30 con 0x7eff700ffd60 2024-03-21T11:46:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.204+0000 7eff6d600640 1 -- 172.21.15.47:0/1442154116 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60020060 con 0x7eff700ffd60 2024-03-21T11:46:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.208+0000 7eff6d600640 1 -- 172.21.15.47:0/1442154116 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7eff6000f620 con 0x7eff700ffd60 2024-03-21T11:46:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.209+0000 7eff7a26f740 1 Processor -- start 2024-03-21T11:46:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.209+0000 7eff7a26f740 1 -- start start 2024-03-21T11:46:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.210+0000 7eff7a26f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e72229820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.210+0000 7eff7a26f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e7222e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.210+0000 7eff7a26f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 0x555e722313f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.210+0000 7eff7a26f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e721db3b0 con 0x555e72229400 2024-03-21T11:46:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.210+0000 7eff7a26f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e7217d830 con 0x555e72229df0 2024-03-21T11:46:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.210+0000 7eff7a26f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e721265d0 con 0x555e7222ef60 2024-03-21T11:46:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.210+0000 7eff6d600640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff540a2f90 0x7eff540a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.210+0000 7eff6fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 0x555e722313f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.210+0000 7eff6fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 0x555e722313f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41106/0 (socket says 172.21.15.47:41106) 2024-03-21T11:46:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.210+0000 7eff6fe00640 1 -- 172.21.15.47:0/59790037 learned_addr learned my addr 172.21.15.47:0/59790037 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.210+0000 7eff6d600640 1 -- 172.21.15.47:0/1442154116 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7eff600cfff0 con 0x7eff700ffd60 2024-03-21T11:46:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.211+0000 7eff75600640 1 --2- 172.21.15.47:0/59790037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e72229820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.211+0000 7eff6fe00640 1 -- 172.21.15.47:0/59790037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 msgr2=0x555e7222e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.211+0000 7eff6fe00640 1 --2- 172.21.15.47:0/59790037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e7222e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.211+0000 7eff6fe00640 1 -- 172.21.15.47:0/59790037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 msgr2=0x555e72229820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.211+0000 7eff6fe00640 1 --2- 172.21.15.47:0/59790037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e72229820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.211+0000 7eff6fe00640 1 -- 172.21.15.47:0/59790037 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e7213e2c0 con 0x555e7222ef60 2024-03-21T11:46:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.211+0000 7eff75600640 1 --2- 172.21.15.47:0/59790037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e72229820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.211+0000 7eff75600640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff540a2f90 0x7eff540a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.211+0000 7eff6fe00640 1 --2- 172.21.15.47:0/59790037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 0x555e722313f0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7eff58006010 tx=0x7eff580095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14107742b9ca189a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.212+0000 7eff65c00640 1 -- 172.21.15.47:0/59790037 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5802cc50 con 0x555e7222ef60 2024-03-21T11:46:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.212+0000 7eff65c00640 1 -- 172.21.15.47:0/59790037 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff58005040 con 0x555e7222ef60 2024-03-21T11:46:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.212+0000 7eff65c00640 1 -- 172.21.15.47:0/59790037 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff580049b0 con 0x555e7222ef60 2024-03-21T11:46:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.213+0000 7eff7a26f740 1 -- 172.21.15.47:0/59790037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 msgr2=0x555e722313f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.213+0000 7eff7a26f740 1 --2- 172.21.15.47:0/59790037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 0x555e722313f0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7eff58006010 tx=0x7eff580095e0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.213+0000 7eff75600640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff540a2f90 0x7eff540a5450 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7eff680099e0 tx=0x7eff68004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.213+0000 7eff7a26f740 1 -- 172.21.15.47:0/59790037 shutdown_connections 2024-03-21T11:46:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.213+0000 7eff7a26f740 1 --2- 172.21.15.47:0/59790037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 0x555e722313f0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.213+0000 7eff7a26f740 1 --2- 172.21.15.47:0/59790037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e7222e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.213+0000 7eff7a26f740 1 --2- 172.21.15.47:0/59790037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e72229820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.213+0000 7eff7a26f740 1 -- 172.21.15.47:0/59790037 >> 172.21.15.47:0/59790037 conn(0x555e72213960 msgr2=0x555e72213d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.213+0000 7eff7a26f740 1 -- 172.21.15.47:0/59790037 shutdown_connections 2024-03-21T11:46:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.213+0000 7eff7a26f740 1 -- 172.21.15.47:0/59790037 wait complete. 2024-03-21T11:46:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.214+0000 7eff7a26f740 1 -- start start 2024-03-21T11:46:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.214+0000 7eff7a26f740 1 Processor -- start 2024-03-21T11:46:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff7a26f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e722335e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff7a26f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e72233b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff7a26f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 0x555e7223ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff7a26f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e72139a20 con 0x555e72229400 2024-03-21T11:46:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff7a26f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e721798d0 con 0x555e72229df0 2024-03-21T11:46:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff74c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e722335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff7a26f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e72231eb0 con 0x555e7222ef60 2024-03-21T11:46:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff74c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e722335e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60672/0 (socket says 172.21.15.47:60672) 2024-03-21T11:46:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff74c00640 1 -- 172.21.15.47:0/1321528668 learned_addr learned my addr 172.21.15.47:0/1321528668 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff7a26f740 10 client.0 _ll_register_callbacks cb 0x555e721761c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff6fe00640 1 --2- 172.21.15.47:0/1321528668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e72233b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff75600640 1 --2- 172.21.15.47:0/1321528668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 0x555e7223ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff74c00640 1 -- 172.21.15.47:0/1321528668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 msgr2=0x555e7223ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff74c00640 1 --2- 172.21.15.47:0/1321528668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 0x555e7223ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff74c00640 1 -- 172.21.15.47:0/1321528668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 msgr2=0x555e72233b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff74c00640 1 --2- 172.21.15.47:0/1321528668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e72233b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.215+0000 7eff74c00640 1 -- 172.21.15.47:0/1321528668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e71e92210 con 0x555e72229400 2024-03-21T11:46:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.216+0000 7eff74c00640 1 --2- 172.21.15.47:0/1321528668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e722335e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7eff60009610 tx=0x7eff6009ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f20302d97f5e94a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.216+0000 7eff5e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:46:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.216+0000 7eff5e000640 1 -- 172.21.15.47:0/1321528668 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff600a04e0 con 0x555e72229400 2024-03-21T11:46:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.216+0000 7eff5e000640 1 -- 172.21.15.47:0/1321528668 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7eff600a0120 con 0x555e72229400 2024-03-21T11:46:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.216+0000 7eff7a26f740 1 -- 172.21.15.47:0/1321528668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff58005040 con 0x555e72229400 2024-03-21T11:46:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.216+0000 7eff5e000640 1 -- 172.21.15.47:0/1321528668 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff600a4da0 con 0x555e72229400 2024-03-21T11:46:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.216+0000 7eff7a26f740 10 client.22092 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.216+0000 7eff7a26f740 1 -- 172.21.15.47:0/1321528668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x555e71f5e620 con 0x555e72229400 2024-03-21T11:46:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.217+0000 7eff5e000640 1 -- 172.21.15.47:0/1321528668 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff58005040 con 0x555e72229400 2024-03-21T11:46:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.217+0000 7eff5e000640 1 -- 172.21.15.47:0/1321528668 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7eff6009b970 con 0x555e72229400 2024-03-21T11:46:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.217+0000 7eff7a26f740 10 client.22092 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.217+0000 7eff7a26f740 10 client.22092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.217+0000 7eff7a26f740 20 client.22092 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.217+0000 7eff7a26f740 1 --2- 172.21.15.47:0/1321528668 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555e7223f420 0x555e72241810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.217+0000 7eff7a26f740 4 client.22092 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.217+0000 7eff7a26f740 1 -- 172.21.15.47:0/1321528668 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e7218ad70 con 0x555e7223f420 2024-03-21T11:46:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.260+0000 7eff6fe00640 1 --2- 172.21.15.47:0/1321528668 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555e7223f420 0x555e72241810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.261+0000 7eff6fe00640 1 --2- 172.21.15.47:0/1321528668 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555e7223f420 0x555e72241810 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=246654e2e305199b server_cookie=cf2971df17436c72 in_seq=0 out_seq=0 2024-03-21T11:46:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.261+0000 7eff5e000640 10 client.22092 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.263+0000 7eff5e000640 1 -- 172.21.15.47:0/1321528668 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555e7218ad70 con 0x555e7223f420 2024-03-21T11:46:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.263+0000 7eff5e000640 10 client.22092 handle_command_reply: tid=0 2024-03-21T11:46:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.264+0000 7eff7a26f740 1 client.22092 shutdown 2024-03-21T11:46:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.264+0000 7eff7a26f740 1 -- 172.21.15.47:0/1321528668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 msgr2=0x555e722335e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.264+0000 7eff7a26f740 1 --2- 172.21.15.47:0/1321528668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e722335e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7eff60009610 tx=0x7eff6009ca90 comp rx=0 tx=0).stop 2024-03-21T11:46:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.264+0000 7eff7a26f740 1 -- 172.21.15.47:0/1321528668 shutdown_connections 2024-03-21T11:46:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.264+0000 7eff7a26f740 1 --2- 172.21.15.47:0/1321528668 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555e7223f420 0x555e72241810 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.264+0000 7eff7a26f740 1 --2- 172.21.15.47:0/1321528668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 0x555e7223ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.264+0000 7eff7a26f740 1 --2- 172.21.15.47:0/1321528668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e72233b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.264+0000 7eff5e000640 0 client.22092 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.264+0000 7eff7a26f740 1 --2- 172.21.15.47:0/1321528668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e722335e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.264+0000 7eff7a26f740 1 -- 172.21.15.47:0/1321528668 >> 172.21.15.47:0/1321528668 conn(0x555e72213960 msgr2=0x555e72212b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.264+0000 7eff7a26f740 1 -- 172.21.15.47:0/1321528668 shutdown_connections 2024-03-21T11:46:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.264+0000 7eff7a26f740 1 -- 172.21.15.47:0/1321528668 wait complete. 2024-03-21T11:46:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.265+0000 7eff7a26f740 20 client.22092 trim_cache size 0 max 16384 2024-03-21T11:46:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.283+0000 7eff7a26f740 1 Processor -- start 2024-03-21T11:46:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.283+0000 7eff7a26f740 1 -- start start 2024-03-21T11:46:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.284+0000 7eff7a26f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e722331f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.284+0000 7eff7a26f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e7223ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.284+0000 7eff7a26f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 0x555e72233730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.284+0000 7eff7a26f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c00f970 con 0x555e72229400 2024-03-21T11:46:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.284+0000 7eff7a26f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c0112f0 con 0x555e72229df0 2024-03-21T11:46:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.284+0000 7eff7a26f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c00ffd0 con 0x555e7222ef60 2024-03-21T11:46:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.284+0000 7eff6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e7223ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.284+0000 7eff74c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e722331f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.284+0000 7eff6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e7223ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44080/0 (socket says 172.21.15.47:44080) 2024-03-21T11:46:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.284+0000 7eff6fe00640 1 -- 172.21.15.47:0/3760038027 learned_addr learned my addr 172.21.15.47:0/3760038027 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.284+0000 7eff75600640 1 --2- 172.21.15.47:0/3760038027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 0x555e72233730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.285+0000 7eff6fe00640 1 -- 172.21.15.47:0/3760038027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 msgr2=0x555e72233730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.285+0000 7eff6fe00640 1 --2- 172.21.15.47:0/3760038027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 0x555e72233730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.285+0000 7eff6fe00640 1 -- 172.21.15.47:0/3760038027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 msgr2=0x555e722331f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.285+0000 7eff6fe00640 1 --2- 172.21.15.47:0/3760038027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e722331f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.285+0000 7eff6fe00640 1 -- 172.21.15.47:0/3760038027 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e72211f80 con 0x555e72229df0 2024-03-21T11:46:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.285+0000 7eff6fe00640 1 --2- 172.21.15.47:0/3760038027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e7223ab20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7eff58005ec0 tx=0x7eff5802c5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb89624b8305d9e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.285+0000 7eff65200640 1 -- 172.21.15.47:0/3760038027 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58005470 con 0x555e72229df0 2024-03-21T11:46:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.285+0000 7eff65200640 1 -- 172.21.15.47:0/3760038027 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff58005610 con 0x555e72229df0 2024-03-21T11:46:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.285+0000 7eff65200640 1 -- 172.21.15.47:0/3760038027 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58003de0 con 0x555e72229df0 2024-03-21T11:46:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.286+0000 7eff7a26f740 1 -- 172.21.15.47:0/3760038027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 msgr2=0x555e7223ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.286+0000 7eff7a26f740 1 --2- 172.21.15.47:0/3760038027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e7223ab20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7eff58005ec0 tx=0x7eff5802c5e0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.286+0000 7eff7a26f740 1 -- 172.21.15.47:0/3760038027 shutdown_connections 2024-03-21T11:46:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.286+0000 7eff7a26f740 1 --2- 172.21.15.47:0/3760038027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e7222ef60 0x555e72233730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.286+0000 7eff7a26f740 1 --2- 172.21.15.47:0/3760038027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e7223ab20 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.286+0000 7eff7a26f740 1 --2- 172.21.15.47:0/3760038027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e72229400 0x555e722331f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.286+0000 7eff7a26f740 1 -- 172.21.15.47:0/3760038027 >> 172.21.15.47:0/3760038027 conn(0x555e72213960 msgr2=0x555e72213d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.286+0000 7eff7a26f740 1 -- 172.21.15.47:0/3760038027 shutdown_connections 2024-03-21T11:46:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.286+0000 7eff7a26f740 1 -- 172.21.15.47:0/3760038027 wait complete. 2024-03-21T11:46:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.287+0000 7eff7a26f740 1 -- start start 2024-03-21T11:46:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.287+0000 7eff7a26f740 1 Processor -- start 2024-03-21T11:46:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.287+0000 7eff7a26f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e72229400 0x555e7223a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.287+0000 7eff7a26f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e72238f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff7a26f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7222ef60 0x555e72239460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff7a26f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c010c90 con 0x555e7222ef60 2024-03-21T11:46:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff7a26f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c010630 con 0x555e72229df0 2024-03-21T11:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff7a26f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c00db30 con 0x555e72229400 2024-03-21T11:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff7a26f740 10 client.0 _ll_register_callbacks cb 0x7eff60010f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e72238f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff75600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7222ef60 0x555e72239460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff6fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e72238f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44090/0 (socket says 172.21.15.47:44090) 2024-03-21T11:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff6fe00640 1 -- 172.21.15.47:0/3366555985 learned_addr learned my addr 172.21.15.47:0/3366555985 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff74c00640 1 --2- 172.21.15.47:0/3366555985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e72229400 0x555e7223a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff6fe00640 1 -- 172.21.15.47:0/3366555985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e72229400 msgr2=0x555e7223a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff6fe00640 1 --2- 172.21.15.47:0/3366555985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e72229400 0x555e7223a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff6fe00640 1 -- 172.21.15.47:0/3366555985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7222ef60 msgr2=0x555e72239460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff6fe00640 1 --2- 172.21.15.47:0/3366555985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7222ef60 0x555e72239460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.288+0000 7eff6fe00640 1 -- 172.21.15.47:0/3366555985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e72239a50 con 0x555e72229df0 2024-03-21T11:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.289+0000 7eff6fe00640 1 --2- 172.21.15.47:0/3366555985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e72238f20 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7eff5802caa0 tx=0x7eff58000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29f99997dbbfdc4b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.289+0000 7eff5e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.289+0000 7eff5e000640 1 -- 172.21.15.47:0/3366555985 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58003d60 con 0x555e72229df0 2024-03-21T11:46:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.289+0000 7eff7a26f740 1 -- 172.21.15.47:0/3366555985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff58005610 con 0x555e72229df0 2024-03-21T11:46:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.289+0000 7eff5e000640 1 -- 172.21.15.47:0/3366555985 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7eff58003f00 con 0x555e72229df0 2024-03-21T11:46:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.289+0000 7eff5e000640 1 -- 172.21.15.47:0/3366555985 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff5805b050 con 0x555e72229df0 2024-03-21T11:46:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.289+0000 7eff7a26f740 10 client.32891 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.289+0000 7eff7a26f740 1 -- 172.21.15.47:0/3366555985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x555e7229baa0 con 0x555e72229df0 2024-03-21T11:46:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.290+0000 7eff5e000640 1 -- 172.21.15.47:0/3366555985 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff58005610 con 0x555e72229df0 2024-03-21T11:46:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.290+0000 7eff5e000640 1 -- 172.21.15.47:0/3366555985 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7eff58004dc0 con 0x555e72229df0 2024-03-21T11:46:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.290+0000 7eff7a26f740 10 client.32891 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.290+0000 7eff7a26f740 10 client.32891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.290+0000 7eff7a26f740 20 client.32891 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.290+0000 7eff7a26f740 1 --2- 172.21.15.47:0/3366555985 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555e7229bfe0 0x555e7229e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.290+0000 7eff7a26f740 4 client.32891 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:46:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.290+0000 7eff7a26f740 1 -- 172.21.15.47:0/3366555985 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555e721850c0 con 0x555e7229bfe0 2024-03-21T11:46:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.291+0000 7eff74c00640 1 --2- 172.21.15.47:0/3366555985 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555e7229bfe0 0x555e7229e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.291+0000 7eff74c00640 1 --2- 172.21.15.47:0/3366555985 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555e7229bfe0 0x555e7229e400 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7449dbc0e8cf0851 server_cookie=e357807d7e0f181c in_seq=0 out_seq=0 2024-03-21T11:46:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.291+0000 7eff5e000640 10 client.32891 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:44.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:46:44.500 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:46:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff5e000640 1 -- 172.21.15.47:0/3366555985 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555e721850c0 con 0x555e7229bfe0 2024-03-21T11:46:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff5e000640 10 client.32891 handle_command_reply: tid=0 2024-03-21T11:46:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff7a26f740 1 client.32891 shutdown 2024-03-21T11:46:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff7a26f740 1 -- 172.21.15.47:0/3366555985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 msgr2=0x555e72238f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff7a26f740 1 --2- 172.21.15.47:0/3366555985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e72238f20 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7eff5802caa0 tx=0x7eff58000bc0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff7a26f740 1 -- 172.21.15.47:0/3366555985 shutdown_connections 2024-03-21T11:46:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff7a26f740 1 --2- 172.21.15.47:0/3366555985 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x555e7229bfe0 0x555e7229e400 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff7a26f740 1 --2- 172.21.15.47:0/3366555985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e7222ef60 0x555e72239460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff7a26f740 1 --2- 172.21.15.47:0/3366555985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e72229df0 0x555e72238f20 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff7a26f740 1 --2- 172.21.15.47:0/3366555985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e72229400 0x555e7223a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff5e000640 0 client.32891 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff7a26f740 1 -- 172.21.15.47:0/3366555985 >> 172.21.15.47:0/3366555985 conn(0x555e72213960 msgr2=0x555e722331d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff7a26f740 1 -- 172.21.15.47:0/3366555985 shutdown_connections 2024-03-21T11:46:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff7a26f740 1 -- 172.21.15.47:0/3366555985 wait complete. 2024-03-21T11:46:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.662+0000 7eff7a26f740 20 client.32891 trim_cache size 0 max 16384 2024-03-21T11:46:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:46:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:46:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:46:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:46:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.663+0000 7eff76000640 1 -- 172.21.15.47:0/1442154116 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff540a2f90 msgr2=0x7eff540a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.663+0000 7eff76000640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff540a2f90 0x7eff540a5450 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7eff680099e0 tx=0x7eff68004a40 comp rx=0 tx=0).stop 2024-03-21T11:46:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.663+0000 7eff76000640 1 -- 172.21.15.47:0/1442154116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff700ffd60 msgr2=0x7eff70104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.663+0000 7eff76000640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff700ffd60 0x7eff70104760 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7eff6000d980 tx=0x7eff6000de40 comp rx=0 tx=0).stop 2024-03-21T11:46:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.664+0000 7eff76000640 1 -- 172.21.15.47:0/1442154116 shutdown_connections 2024-03-21T11:46:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.664+0000 7eff76000640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7eff540a2f90 0x7eff540a5450 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.664+0000 7eff76000640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff70101060 0x7eff7010bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.664+0000 7eff76000640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff701006c0 0x7eff70104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.664+0000 7eff76000640 1 --2- 172.21.15.47:0/1442154116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff700ffd60 0x7eff70104760 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.664+0000 7eff76000640 1 -- 172.21.15.47:0/1442154116 >> 172.21.15.47:0/1442154116 conn(0x7eff700fb980 msgr2=0x7eff700fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.664+0000 7eff76000640 1 -- 172.21.15.47:0/1442154116 shutdown_connections 2024-03-21T11:46:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:44.664+0000 7eff76000640 1 -- 172.21.15.47:0/1442154116 wait complete. 2024-03-21T11:46:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.770+0000 7f3559800640 1 -- 172.21.15.47:0/3452987406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35541073a0 msgr2=0x7f35541097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.770+0000 7f3559800640 1 --2- 172.21.15.47:0/3452987406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35541073a0 0x7f35541097f0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f354c009ca0 tx=0x7f354c02c100 comp rx=0 tx=0).stop 2024-03-21T11:46:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.770+0000 7f3559800640 1 -- 172.21.15.47:0/3452987406 shutdown_connections 2024-03-21T11:46:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.770+0000 7f3559800640 1 --2- 172.21.15.47:0/3452987406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35541073a0 0x7f35541097f0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.770+0000 7f3559800640 1 --2- 172.21.15.47:0/3452987406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3554104a70 0x7f3554106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.770+0000 7f3559800640 1 --2- 172.21.15.47:0/3452987406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3554102140 0x7f3554104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.770+0000 7f3559800640 1 -- 172.21.15.47:0/3452987406 >> 172.21.15.47:0/3452987406 conn(0x7f35540fbb30 msgr2=0x7f35540fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:44.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.773+0000 7f3559800640 1 -- 172.21.15.47:0/3452987406 shutdown_connections 2024-03-21T11:46:44.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.773+0000 7f3559800640 1 -- 172.21.15.47:0/3452987406 wait complete. 2024-03-21T11:46:44.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.774+0000 7f3559800640 1 Processor -- start 2024-03-21T11:46:44.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.774+0000 7f3559800640 1 -- start start 2024-03-21T11:46:44.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.775+0000 7f3559800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3554102140 0x7f3554101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.775+0000 7f3559800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3554104a70 0x7f35541000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.775+0000 7f3559800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35541073a0 0x7f3554100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.775+0000 7f3559800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355410d340 con 0x7f3554102140 2024-03-21T11:46:44.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.775+0000 7f3559800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355410d1c0 con 0x7f3554104a70 2024-03-21T11:46:44.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.775+0000 7f3559800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355410d4c0 con 0x7f35541073a0 2024-03-21T11:46:44.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.775+0000 7f3553400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35541073a0 0x7f3554100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.775+0000 7f3552a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3554102140 0x7f3554101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.775+0000 7f3553400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35541073a0 0x7f3554100630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41150/0 (socket says 172.21.15.47:41150) 2024-03-21T11:46:44.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.775+0000 7f3553400640 1 -- 172.21.15.47:0/384842051 learned_addr learned my addr 172.21.15.47:0/384842051 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:44.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.775+0000 7f3552000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3554104a70 0x7f35541000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.776+0000 7f3553400640 1 -- 172.21.15.47:0/384842051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3554104a70 msgr2=0x7f35541000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.776+0000 7f3553400640 1 --2- 172.21.15.47:0/384842051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3554104a70 0x7f35541000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.776+0000 7f3553400640 1 -- 172.21.15.47:0/384842051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3554102140 msgr2=0x7f3554101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.776+0000 7f3553400640 1 --2- 172.21.15.47:0/384842051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3554102140 0x7f3554101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.776+0000 7f3553400640 1 -- 172.21.15.47:0/384842051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3554100ec0 con 0x7f35541073a0 2024-03-21T11:46:44.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.776+0000 7f3552000640 1 --2- 172.21.15.47:0/384842051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3554104a70 0x7f35541000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:44.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.776+0000 7f3552a00640 1 --2- 172.21.15.47:0/384842051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3554102140 0x7f3554101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:44.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.776+0000 7f3553400640 1 --2- 172.21.15.47:0/384842051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35541073a0 0x7f3554100630 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f354c005cc0 tx=0x7f354c035d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:44.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.777+0000 7f354b400640 1 -- 172.21.15.47:0/384842051 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c039040 con 0x7f35541073a0 2024-03-21T11:46:44.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.777+0000 7f3559800640 1 -- 172.21.15.47:0/384842051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3554071cd0 con 0x7f35541073a0 2024-03-21T11:46:44.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.777+0000 7f354b400640 1 -- 172.21.15.47:0/384842051 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f354c044070 con 0x7f35541073a0 2024-03-21T11:46:44.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.777+0000 7f354b400640 1 -- 172.21.15.47:0/384842051 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c004480 con 0x7f35541073a0 2024-03-21T11:46:44.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.777+0000 7f3559800640 1 -- 172.21.15.47:0/384842051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3554072240 con 0x7f35541073a0 2024-03-21T11:46:44.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.783+0000 7f354b400640 1 -- 172.21.15.47:0/384842051 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f354c004620 con 0x7f35541073a0 2024-03-21T11:46:44.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.786+0000 7f354b400640 1 --2- 172.21.15.47:0/384842051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35380a3120 0x7f35380a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.787+0000 7f354b400640 1 -- 172.21.15.47:0/384842051 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f354c004220 con 0x7f35541073a0 2024-03-21T11:46:44.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.787+0000 7f355c611740 1 Processor -- start 2024-03-21T11:46:44.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.787+0000 7f3552a00640 1 --2- 172.21.15.47:0/384842051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35380a3120 0x7f35380a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.788+0000 7f355c611740 1 -- start start 2024-03-21T11:46:44.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.788+0000 7f355c611740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed05de80 0x5633ed05e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.788+0000 7f355c611740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05e870 0x5633ed063370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.788+0000 7f355c611740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed0639e0 0x5633ed065e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.788+0000 7f355c611740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ecfc3250 con 0x5633ed05de80 2024-03-21T11:46:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.788+0000 7f355c611740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ecf6ba30 con 0x5633ed05e870 2024-03-21T11:46:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.788+0000 7f355c611740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ecfbf2f0 con 0x5633ed0639e0 2024-03-21T11:46:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.788+0000 7f3553400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed0639e0 0x5633ed065e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.788+0000 7f3553400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed0639e0 0x5633ed065e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41162/0 (socket says 172.21.15.47:41162) 2024-03-21T11:46:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.788+0000 7f3553400640 1 -- 172.21.15.47:0/2723570990 learned_addr learned my addr 172.21.15.47:0/2723570990 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.788+0000 7f3552000640 1 --2- 172.21.15.47:0/2723570990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05e870 0x5633ed063370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.789+0000 7f3552a00640 1 --2- 172.21.15.47:0/2723570990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed05de80 0x5633ed05e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.789+0000 7f3553400640 1 -- 172.21.15.47:0/2723570990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05e870 msgr2=0x5633ed063370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.790+0000 7f3553400640 1 --2- 172.21.15.47:0/2723570990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05e870 0x5633ed063370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.790+0000 7f3553400640 1 -- 172.21.15.47:0/2723570990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed05de80 msgr2=0x5633ed05e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.790+0000 7f3553400640 1 --2- 172.21.15.47:0/2723570990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed05de80 0x5633ed05e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.790+0000 7f3553400640 1 -- 172.21.15.47:0/2723570990 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633ecf56690 con 0x5633ed0639e0 2024-03-21T11:46:44.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.790+0000 7f3552a00640 1 --2- 172.21.15.47:0/2723570990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed05de80 0x5633ed05e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:44.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.790+0000 7f3552a00640 1 --2- 172.21.15.47:0/384842051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35380a3120 0x7f35380a55e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3544009c20 tx=0x7f3544005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:44.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.790+0000 7f3552000640 1 --2- 172.21.15.47:0/2723570990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05e870 0x5633ed063370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:44.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.790+0000 7f3553400640 1 --2- 172.21.15.47:0/2723570990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed0639e0 0x5633ed065e70 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f354c037040 tx=0x7f354c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22f6a228d046405e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:44.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.790+0000 7f353fe00640 1 -- 172.21.15.47:0/2723570990 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0be570 con 0x5633ed0639e0 2024-03-21T11:46:44.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.791+0000 7f353fe00640 1 -- 172.21.15.47:0/2723570990 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f354c0bde70 con 0x5633ed0639e0 2024-03-21T11:46:44.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.791+0000 7f353fe00640 1 -- 172.21.15.47:0/2723570990 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0333a0 con 0x5633ed0639e0 2024-03-21T11:46:44.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.792+0000 7f355c611740 1 -- 172.21.15.47:0/2723570990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed0639e0 msgr2=0x5633ed065e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.792+0000 7f355c611740 1 --2- 172.21.15.47:0/2723570990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed0639e0 0x5633ed065e70 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f354c037040 tx=0x7f354c0095c0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.793+0000 7f355c611740 1 -- 172.21.15.47:0/2723570990 shutdown_connections 2024-03-21T11:46:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.793+0000 7f355c611740 1 --2- 172.21.15.47:0/2723570990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed0639e0 0x5633ed065e70 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.794+0000 7f355c611740 1 --2- 172.21.15.47:0/2723570990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05e870 0x5633ed063370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.794+0000 7f355c611740 1 --2- 172.21.15.47:0/2723570990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed05de80 0x5633ed05e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.794+0000 7f355c611740 1 -- 172.21.15.47:0/2723570990 >> 172.21.15.47:0/2723570990 conn(0x5633ed026200 msgr2=0x5633ed0597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:44.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.794+0000 7f355c611740 1 -- 172.21.15.47:0/2723570990 shutdown_connections 2024-03-21T11:46:44.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.794+0000 7f355c611740 1 -- 172.21.15.47:0/2723570990 wait complete. 2024-03-21T11:46:44.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.795+0000 7f355c611740 1 -- start start 2024-03-21T11:46:44.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.795+0000 7f355c611740 1 Processor -- start 2024-03-21T11:46:44.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.796+0000 7f355c611740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed05de80 0x5633ed070380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.796+0000 7f355c611740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05e870 0x5633ed0708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.796+0000 7f355c611740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed0639e0 0x5633ed077900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.796+0000 7f355c611740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ed06ef60 con 0x5633ed05de80 2024-03-21T11:46:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.796+0000 7f355c611740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ecf7ee80 con 0x5633ed05e870 2024-03-21T11:46:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.796+0000 7f355c611740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633ed06f0e0 con 0x5633ed0639e0 2024-03-21T11:46:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.796+0000 7f355c611740 10 client.0 _ll_register_callbacks cb 0x5633ecf91130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.796+0000 7f3552a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed05de80 0x5633ed070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.796+0000 7f3553400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed0639e0 0x5633ed077900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.796+0000 7f3552000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05e870 0x5633ed0708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.796+0000 7f3553400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed0639e0 0x5633ed077900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41178/0 (socket says 172.21.15.47:41178) 2024-03-21T11:46:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.796+0000 7f3553400640 1 -- 172.21.15.47:0/4050935172 learned_addr learned my addr 172.21.15.47:0/4050935172 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.797+0000 7f3553400640 1 -- 172.21.15.47:0/4050935172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05e870 msgr2=0x5633ed0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.797+0000 7f3553400640 1 --2- 172.21.15.47:0/4050935172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05e870 0x5633ed0708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.797+0000 7f3553400640 1 -- 172.21.15.47:0/4050935172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed05de80 msgr2=0x5633ed070380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.797+0000 7f3553400640 1 --2- 172.21.15.47:0/4050935172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed05de80 0x5633ed070380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.797+0000 7f3553400640 1 -- 172.21.15.47:0/4050935172 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633ecda17d0 con 0x5633ed0639e0 2024-03-21T11:46:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.797+0000 7f3552a00640 1 --2- 172.21.15.47:0/4050935172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed05de80 0x5633ed070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:44.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.797+0000 7f3553400640 1 --2- 172.21.15.47:0/4050935172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed0639e0 0x5633ed077900 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f354c0ef5b0 tx=0x7f354c0c1fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac7b84659076f3f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:44.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.797+0000 7f353cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:44.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.797+0000 7f353cc00640 1 -- 172.21.15.47:0/4050935172 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0335a0 con 0x5633ed0639e0 2024-03-21T11:46:44.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.798+0000 7f355c611740 1 -- 172.21.15.47:0/4050935172 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f354c0bde70 con 0x5633ed0639e0 2024-03-21T11:46:44.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.798+0000 7f353cc00640 1 -- 172.21.15.47:0/4050935172 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f354c0dcbe0 con 0x5633ed0639e0 2024-03-21T11:46:44.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.798+0000 7f353cc00640 1 -- 172.21.15.47:0/4050935172 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f354c0dce60 con 0x5633ed0639e0 2024-03-21T11:46:44.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.798+0000 7f355c611740 10 client.32881 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:44.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.798+0000 7f355c611740 1 -- 172.21.15.47:0/4050935172 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5633eccd73f0 con 0x5633ed0639e0 2024-03-21T11:46:44.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.798+0000 7f353cc00640 1 -- 172.21.15.47:0/4050935172 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f354c0bde70 con 0x5633ed0639e0 2024-03-21T11:46:44.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.798+0000 7f353cc00640 1 -- 172.21.15.47:0/4050935172 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f354c09cce0 con 0x5633ed0639e0 2024-03-21T11:46:44.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.798+0000 7f355c611740 10 client.32881 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:44.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.798+0000 7f355c611740 10 client.32881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.798+0000 7f355c611740 20 client.32881 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.799+0000 7f355c611740 1 --2- 172.21.15.47:0/4050935172 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633ed07bd60 0x5633ed07e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.799+0000 7f355c611740 4 client.32881 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.799+0000 7f355c611740 1 -- 172.21.15.47:0/4050935172 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633ecfd0790 con 0x5633ed07bd60 2024-03-21T11:46:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.799+0000 7f3552a00640 1 --2- 172.21.15.47:0/4050935172 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633ed07bd60 0x5633ed07e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.799+0000 7f353cc00640 10 client.32881 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.799+0000 7f3552a00640 1 --2- 172.21.15.47:0/4050935172 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633ed07bd60 0x5633ed07e150 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a816ab0ab0168a24 server_cookie=68f0cd840ec7a80b in_seq=0 out_seq=0 2024-03-21T11:46:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.800+0000 7f353cc00640 1 -- 172.21.15.47:0/4050935172 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5633ecfd0790 con 0x5633ed07bd60 2024-03-21T11:46:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.800+0000 7f353cc00640 10 client.32881 handle_command_reply: tid=0 2024-03-21T11:46:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.800+0000 7f355c611740 1 client.32881 shutdown 2024-03-21T11:46:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.800+0000 7f355c611740 1 -- 172.21.15.47:0/4050935172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed0639e0 msgr2=0x5633ed077900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.800+0000 7f355c611740 1 --2- 172.21.15.47:0/4050935172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed0639e0 0x5633ed077900 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f354c0ef5b0 tx=0x7f354c0c1fc0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.801+0000 7f355c611740 1 -- 172.21.15.47:0/4050935172 shutdown_connections 2024-03-21T11:46:44.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.801+0000 7f355c611740 1 --2- 172.21.15.47:0/4050935172 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633ed07bd60 0x5633ed07e150 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.801+0000 7f355c611740 1 --2- 172.21.15.47:0/4050935172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed0639e0 0x5633ed077900 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.801+0000 7f355c611740 1 --2- 172.21.15.47:0/4050935172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05e870 0x5633ed0708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.801+0000 7f355c611740 1 --2- 172.21.15.47:0/4050935172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed05de80 0x5633ed070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.801+0000 7f355c611740 1 -- 172.21.15.47:0/4050935172 >> 172.21.15.47:0/4050935172 conn(0x5633ed026200 msgr2=0x5633ed058a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.801+0000 7f353cc00640 0 client.32881 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.801+0000 7f355c611740 1 -- 172.21.15.47:0/4050935172 shutdown_connections 2024-03-21T11:46:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.801+0000 7f355c611740 1 -- 172.21.15.47:0/4050935172 wait complete. 2024-03-21T11:46:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.801+0000 7f355c611740 20 client.32881 trim_cache size 0 max 16384 2024-03-21T11:46:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.827+0000 7f355c611740 1 Processor -- start 2024-03-21T11:46:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.827+0000 7f355c611740 1 -- start start 2024-03-21T11:46:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.828+0000 7f355c611740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05de80 0x5633ed05fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.828+0000 7f355c611740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed05e870 0x5633ed0778c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.828+0000 7f355c611740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed0639e0 0x5633ed058200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.828+0000 7f355c611740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353400f950 con 0x5633ed0639e0 2024-03-21T11:46:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.828+0000 7f355c611740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35340112d0 con 0x5633ed05de80 2024-03-21T11:46:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.828+0000 7f355c611740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353400ffb0 con 0x5633ed05e870 2024-03-21T11:46:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.828+0000 7f3552a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05de80 0x5633ed05fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.828+0000 7f3552000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed05e870 0x5633ed0778c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.828+0000 7f3552000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed05e870 0x5633ed0778c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41180/0 (socket says 172.21.15.47:41180) 2024-03-21T11:46:44.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.828+0000 7f3552000640 1 -- 172.21.15.47:0/3357454056 learned_addr learned my addr 172.21.15.47:0/3357454056 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.828+0000 7f3552a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05de80 0x5633ed05fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44122/0 (socket says 172.21.15.47:44122) 2024-03-21T11:46:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.828+0000 7f3553400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed0639e0 0x5633ed058200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.829+0000 7f3552000640 1 -- 172.21.15.47:0/3357454056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05de80 msgr2=0x5633ed05fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.829+0000 7f3552000640 1 --2- 172.21.15.47:0/3357454056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05de80 0x5633ed05fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.829+0000 7f3552000640 1 -- 172.21.15.47:0/3357454056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed0639e0 msgr2=0x5633ed058200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.829+0000 7f3552000640 1 --2- 172.21.15.47:0/3357454056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed0639e0 0x5633ed058200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.829+0000 7f3552000640 1 -- 172.21.15.47:0/3357454056 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633ecc970f0 con 0x5633ed05e870 2024-03-21T11:46:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.829+0000 7f3552000640 1 --2- 172.21.15.47:0/3357454056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed05e870 0x5633ed0778c0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f3540005fd0 tx=0x7f354000bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cd68d66c0620b03 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:44.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.829+0000 7f353f400640 1 -- 172.21.15.47:0/3357454056 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354000be00 con 0x5633ed05e870 2024-03-21T11:46:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.829+0000 7f353f400640 1 -- 172.21.15.47:0/3357454056 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3540012450 con 0x5633ed05e870 2024-03-21T11:46:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.829+0000 7f353f400640 1 -- 172.21.15.47:0/3357454056 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540011670 con 0x5633ed05e870 2024-03-21T11:46:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.830+0000 7f355c611740 1 -- 172.21.15.47:0/3357454056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed05e870 msgr2=0x5633ed0778c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.830+0000 7f355c611740 1 --2- 172.21.15.47:0/3357454056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed05e870 0x5633ed0778c0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f3540005fd0 tx=0x7f354000bbb0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.830+0000 7f355c611740 1 -- 172.21.15.47:0/3357454056 shutdown_connections 2024-03-21T11:46:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.830+0000 7f355c611740 1 --2- 172.21.15.47:0/3357454056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed0639e0 0x5633ed058200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.830+0000 7f355c611740 1 --2- 172.21.15.47:0/3357454056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed05e870 0x5633ed0778c0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.830+0000 7f355c611740 1 --2- 172.21.15.47:0/3357454056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05de80 0x5633ed05fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.830+0000 7f355c611740 1 -- 172.21.15.47:0/3357454056 >> 172.21.15.47:0/3357454056 conn(0x5633ed026200 msgr2=0x5633ed05f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.831+0000 7f355c611740 1 -- 172.21.15.47:0/3357454056 shutdown_connections 2024-03-21T11:46:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.831+0000 7f355c611740 1 -- 172.21.15.47:0/3357454056 wait complete. 2024-03-21T11:46:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.831+0000 7f355c611740 1 -- start start 2024-03-21T11:46:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.831+0000 7f355c611740 1 Processor -- start 2024-03-21T11:46:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f355c611740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05de80 0x5633ed075370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f355c611740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed05e870 0x5633ed0758b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f355c611740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed0639e0 0x5633ed0768c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f355c611740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3534010c70 con 0x5633ed0639e0 2024-03-21T11:46:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f355c611740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3534010610 con 0x5633ed05de80 2024-03-21T11:46:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f355c611740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353400db10 con 0x5633ed05e870 2024-03-21T11:46:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f355c611740 10 client.0 _ll_register_callbacks cb 0x7f354c0bd9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f3552000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed05e870 0x5633ed0758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f3552000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed05e870 0x5633ed0758b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41188/0 (socket says 172.21.15.47:41188) 2024-03-21T11:46:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f3552000640 1 -- 172.21.15.47:0/3161278826 learned_addr learned my addr 172.21.15.47:0/3161278826 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f3553400640 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed0639e0 0x5633ed0768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f3552a00640 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05de80 0x5633ed075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f3552000640 1 -- 172.21.15.47:0/3161278826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05de80 msgr2=0x5633ed075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f3552000640 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05de80 0x5633ed075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f3552000640 1 -- 172.21.15.47:0/3161278826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed0639e0 msgr2=0x5633ed0768c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f3552000640 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed0639e0 0x5633ed0768c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f3552000640 1 -- 172.21.15.47:0/3161278826 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633ed076eb0 con 0x5633ed05e870 2024-03-21T11:46:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.832+0000 7f3553400640 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed0639e0 0x5633ed0768c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.833+0000 7f3552000640 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed05e870 0x5633ed0758b0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3540000c00 tx=0x7f3540004220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6054a70a29f411f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.833+0000 7f3552a00640 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05de80 0x5633ed075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.833+0000 7f353cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.833+0000 7f353cc00640 1 -- 172.21.15.47:0/3161278826 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540021040 con 0x5633ed05e870 2024-03-21T11:46:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.833+0000 7f355c611740 1 -- 172.21.15.47:0/3161278826 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3540012450 con 0x5633ed05e870 2024-03-21T11:46:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.833+0000 7f353cc00640 1 -- 172.21.15.47:0/3161278826 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3540012740 con 0x5633ed05e870 2024-03-21T11:46:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.833+0000 7f353cc00640 1 -- 172.21.15.47:0/3161278826 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3540012940 con 0x5633ed05e870 2024-03-21T11:46:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.834+0000 7f355c611740 10 client.32887 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.834+0000 7f355c611740 1 -- 172.21.15.47:0/3161278826 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5633ed0d7720 con 0x5633ed05e870 2024-03-21T11:46:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.834+0000 7f353cc00640 1 -- 172.21.15.47:0/3161278826 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3540012450 con 0x5633ed05e870 2024-03-21T11:46:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.834+0000 7f353cc00640 1 -- 172.21.15.47:0/3161278826 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f354002d030 con 0x5633ed05e870 2024-03-21T11:46:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.834+0000 7f355c611740 10 client.32887 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.834+0000 7f355c611740 10 client.32887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.834+0000 7f355c611740 20 client.32887 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.834+0000 7f355c611740 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633ed0d80f0 0x5633ed0da510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.834+0000 7f355c611740 4 client.32887 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:46:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.834+0000 7f355c611740 1 -- 172.21.15.47:0/3161278826 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5633ecfcaae0 con 0x5633ed0d80f0 2024-03-21T11:46:44.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.836+0000 7f3552a00640 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633ed0d80f0 0x5633ed0da510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:44.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.837+0000 7f3552a00640 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633ed0d80f0 0x5633ed0da510 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c84f21bc652af00 server_cookie=a8a5451e52270ed9 in_seq=0 out_seq=0 2024-03-21T11:46:44.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.838+0000 7f353cc00640 10 client.32887 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:44.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.838+0000 7f353cc00640 1 -- 172.21.15.47:0/3161278826 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5633ecfcaae0 con 0x5633ed0d80f0 2024-03-21T11:46:44.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.838+0000 7f353cc00640 10 client.32887 handle_command_reply: tid=0 2024-03-21T11:46:44.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.838+0000 7f355c611740 1 client.32887 shutdown 2024-03-21T11:46:44.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.839+0000 7f355c611740 1 -- 172.21.15.47:0/3161278826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed05e870 msgr2=0x5633ed0758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.839+0000 7f355c611740 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed05e870 0x5633ed0758b0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3540000c00 tx=0x7f3540004220 comp rx=0 tx=0).stop 2024-03-21T11:46:44.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.839+0000 7f355c611740 1 -- 172.21.15.47:0/3161278826 shutdown_connections 2024-03-21T11:46:44.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.839+0000 7f355c611740 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5633ed0d80f0 0x5633ed0da510 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.840+0000 7f355c611740 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5633ed0639e0 0x5633ed0768c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.840+0000 7f355c611740 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633ed05e870 0x5633ed0758b0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.840+0000 7f353cc00640 0 client.32887 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.840+0000 7f355c611740 1 --2- 172.21.15.47:0/3161278826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633ed05de80 0x5633ed075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.840+0000 7f355c611740 1 -- 172.21.15.47:0/3161278826 >> 172.21.15.47:0/3161278826 conn(0x5633ed026200 msgr2=0x5633ed05f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.840+0000 7f355c611740 1 -- 172.21.15.47:0/3161278826 shutdown_connections 2024-03-21T11:46:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.840+0000 7f355c611740 1 -- 172.21.15.47:0/3161278826 wait complete. 2024-03-21T11:46:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.841+0000 7f355c611740 20 client.32887 trim_cache size 0 max 16384 2024-03-21T11:46:44.842 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:46:44.843 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:46:44.843 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:46:44.843 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:46:44.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.842+0000 7f3559800640 1 -- 172.21.15.47:0/384842051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35380a3120 msgr2=0x7f35380a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.842+0000 7f3559800640 1 --2- 172.21.15.47:0/384842051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35380a3120 0x7f35380a55e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3544009c20 tx=0x7f3544005f90 comp rx=0 tx=0).stop 2024-03-21T11:46:44.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.842+0000 7f3559800640 1 -- 172.21.15.47:0/384842051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35541073a0 msgr2=0x7f3554100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:44.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.842+0000 7f3559800640 1 --2- 172.21.15.47:0/384842051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35541073a0 0x7f3554100630 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f354c005cc0 tx=0x7f354c035d90 comp rx=0 tx=0).stop 2024-03-21T11:46:44.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.842+0000 7f3559800640 1 -- 172.21.15.47:0/384842051 shutdown_connections 2024-03-21T11:46:44.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.842+0000 7f3559800640 1 --2- 172.21.15.47:0/384842051 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f35380a3120 0x7f35380a55e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.842+0000 7f3559800640 1 --2- 172.21.15.47:0/384842051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35541073a0 0x7f3554100630 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.842+0000 7f3559800640 1 --2- 172.21.15.47:0/384842051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3554104a70 0x7f35541000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.842+0000 7f3559800640 1 --2- 172.21.15.47:0/384842051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3554102140 0x7f3554101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:44.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.842+0000 7f3559800640 1 -- 172.21.15.47:0/384842051 >> 172.21.15.47:0/384842051 conn(0x7f35540fbb30 msgr2=0x7f35541029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:44.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.842+0000 7f3559800640 1 -- 172.21.15.47:0/384842051 shutdown_connections 2024-03-21T11:46:44.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:44.842+0000 7f3559800640 1 -- 172.21.15.47:0/384842051 wait complete. 2024-03-21T11:46:44.862 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:46:44.862 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:46:44.989 INFO:tasks.workunit.client.0.smithi047.stdout: 10 239019 21.74 MB/sec execute 448 sec latency 269.107 ms 2024-03-21T11:46:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.107+0000 7f8bffe00640 1 -- 172.21.15.47:0/694379412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c000734a0 msgr2=0x7f8c0010a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.107+0000 7f8bffe00640 1 --2- 172.21.15.47:0/694379412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c000734a0 0x7f8c0010a6b0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009ca0 tx=0x7f8bf002c100 comp rx=0 tx=0).stop 2024-03-21T11:46:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.107+0000 7f8bffe00640 1 -- 172.21.15.47:0/694379412 shutdown_connections 2024-03-21T11:46:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.107+0000 7f8bffe00640 1 --2- 172.21.15.47:0/694379412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c0010d760 0x7f8c001185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.107+0000 7f8bffe00640 1 --2- 172.21.15.47:0/694379412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c0010ac80 0x7f8c0010d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.107+0000 7f8bffe00640 1 --2- 172.21.15.47:0/694379412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c000734a0 0x7f8c0010a6b0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.107+0000 7f8bffe00640 1 -- 172.21.15.47:0/694379412 >> 172.21.15.47:0/694379412 conn(0x7f8c0006e3d0 msgr2=0x7f8c00070810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:45.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.107+0000 7f8bffe00640 1 -- 172.21.15.47:0/694379412 shutdown_connections 2024-03-21T11:46:45.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.107+0000 7f8bffe00640 1 -- 172.21.15.47:0/694379412 wait complete. 2024-03-21T11:46:45.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.108+0000 7f8bffe00640 1 Processor -- start 2024-03-21T11:46:45.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.109+0000 7f8bffe00640 1 -- start start 2024-03-21T11:46:45.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.109+0000 7f8bffe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c000734a0 0x7f8c001a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.109+0000 7f8bffe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c0010ac80 0x7f8c001a6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.109+0000 7f8bffe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0010d760 0x7f8c001adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.109+0000 7f8bffe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0011bf60 con 0x7f8c000734a0 2024-03-21T11:46:45.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.109+0000 7f8bffe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0011bde0 con 0x7f8c0010ac80 2024-03-21T11:46:45.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.109+0000 7f8bffe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0011c0e0 con 0x7f8c0010d760 2024-03-21T11:46:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.110+0000 7f8bfe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c0010ac80 0x7f8c001a6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.110+0000 7f8bff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0010d760 0x7f8c001adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.110+0000 7f8bfe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c0010ac80 0x7f8c001a6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44152/0 (socket says 172.21.15.47:44152) 2024-03-21T11:46:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.110+0000 7f8bff400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0010d760 0x7f8c001adbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41200/0 (socket says 172.21.15.47:41200) 2024-03-21T11:46:45.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.110+0000 7f8bfe000640 1 -- 172.21.15.47:0/1233197648 learned_addr learned my addr 172.21.15.47:0/1233197648 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:45.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.110+0000 7f8bfea00640 1 --2- 172.21.15.47:0/1233197648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c000734a0 0x7f8c001a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.110+0000 7f8bff400640 1 -- 172.21.15.47:0/1233197648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c0010ac80 msgr2=0x7f8c001a6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:45.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.110+0000 7f8bff400640 1 --2- 172.21.15.47:0/1233197648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c0010ac80 0x7f8c001a6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.110+0000 7f8bff400640 1 -- 172.21.15.47:0/1233197648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c000734a0 msgr2=0x7f8c001a6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:45.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.110+0000 7f8bff400640 1 --2- 172.21.15.47:0/1233197648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c000734a0 0x7f8c001a6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.110+0000 7f8bff400640 1 -- 172.21.15.47:0/1233197648 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0010ff00 con 0x7f8c0010d760 2024-03-21T11:46:45.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.111+0000 7f8bff400640 1 --2- 172.21.15.47:0/1233197648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0010d760 0x7f8c001adbd0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800d680 tx=0x7f8bf800db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:45.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.111+0000 7f8bf7400640 1 -- 172.21.15.47:0/1233197648 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8017070 con 0x7f8c0010d760 2024-03-21T11:46:45.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.111+0000 7f8bf7400640 1 -- 172.21.15.47:0/1233197648 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bf8009820 con 0x7f8c0010d760 2024-03-21T11:46:45.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.111+0000 7f8bf7400640 1 -- 172.21.15.47:0/1233197648 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8004e00 con 0x7f8c0010d760 2024-03-21T11:46:45.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.111+0000 7f8bffe00640 1 -- 172.21.15.47:0/1233197648 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c001101f0 con 0x7f8c0010d760 2024-03-21T11:46:45.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.111+0000 7f8bffe00640 1 -- 172.21.15.47:0/1233197648 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c001106f0 con 0x7f8c0010d760 2024-03-21T11:46:45.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.113+0000 7f8bffe00640 1 -- 172.21.15.47:0/1233197648 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c00070ed0 con 0x7f8c0010d760 2024-03-21T11:46:45.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.115+0000 7f8bf7400640 1 -- 172.21.15.47:0/1233197648 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8bf80099d0 con 0x7f8c0010d760 2024-03-21T11:46:45.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.117+0000 7f8bf7400640 1 --2- 172.21.15.47:0/1233197648 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8be40a2fa0 0x7f8be40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.118+0000 7f8bf7400640 1 -- 172.21.15.47:0/1233197648 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8bf80cf080 con 0x7f8c0010d760 2024-03-21T11:46:45.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.118+0000 7f8bfea00640 1 --2- 172.21.15.47:0/1233197648 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8be40a2fa0 0x7f8be40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.119+0000 7f8bfea00640 1 --2- 172.21.15.47:0/1233197648 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8be40a2fa0 0x7f8be40a5460 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0037040 tx=0x7f8bf00093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:45.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.126+0000 7f8bf7400640 1 -- 172.21.15.47:0/1233197648 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8bf809bb20 con 0x7f8c0010d760 2024-03-21T11:46:45.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:44 smithi100 ceph-mon[36245]: pgmap v1103: 129 pgs: 129 active+clean; 698 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.0 MiB/s wr, 509 op/s 2024-03-21T11:46:45.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:44 smithi102 ceph-mon[36748]: pgmap v1103: 129 pgs: 129 active+clean; 698 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.0 MiB/s wr, 509 op/s 2024-03-21T11:46:45.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:44 smithi047 ceph-mon[31351]: pgmap v1103: 129 pgs: 129 active+clean; 698 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.0 MiB/s wr, 509 op/s 2024-03-21T11:46:45.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.530+0000 7f8bffe00640 1 -- 172.21.15.47:0/1233197648 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c0002d070 con 0x7f8c0010d760 2024-03-21T11:46:45.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.531+0000 7f8bf7400640 1 -- 172.21.15.47:0/1233197648 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f8bf809b420 con 0x7f8c0010d760 2024-03-21T11:46:45.532 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:46:45.532 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:45:39.125075+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_32623":{"gid":32623,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4278877119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4278877119},{"type":"v1","addr":"172.21.15.100:6835","nonce":4278877119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24523,24524,24535,24553,24581]},"id":1}]} 2024-03-21T11:46:45.532 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 41 2024-03-21T11:46:45.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.534+0000 7f8bffe00640 1 -- 172.21.15.47:0/1233197648 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8be40a2fa0 msgr2=0x7f8be40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:45.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.534+0000 7f8bffe00640 1 --2- 172.21.15.47:0/1233197648 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8be40a2fa0 0x7f8be40a5460 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0037040 tx=0x7f8bf00093f0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.534+0000 7f8bffe00640 1 -- 172.21.15.47:0/1233197648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0010d760 msgr2=0x7f8c001adbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:45.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.534+0000 7f8bffe00640 1 --2- 172.21.15.47:0/1233197648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0010d760 0x7f8c001adbd0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800d680 tx=0x7f8bf800db40 comp rx=0 tx=0).stop 2024-03-21T11:46:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.535+0000 7f8bffe00640 1 -- 172.21.15.47:0/1233197648 shutdown_connections 2024-03-21T11:46:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.535+0000 7f8bffe00640 1 --2- 172.21.15.47:0/1233197648 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8be40a2fa0 0x7f8be40a5460 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.535+0000 7f8bffe00640 1 --2- 172.21.15.47:0/1233197648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c0010d760 0x7f8c001adbd0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.535+0000 7f8bffe00640 1 --2- 172.21.15.47:0/1233197648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c0010ac80 0x7f8c001a6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.535+0000 7f8bffe00640 1 --2- 172.21.15.47:0/1233197648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c000734a0 0x7f8c001a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.535+0000 7f8bffe00640 1 -- 172.21.15.47:0/1233197648 >> 172.21.15.47:0/1233197648 conn(0x7f8c0006e3d0 msgr2=0x7f8c001086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.535+0000 7f8bffe00640 1 -- 172.21.15.47:0/1233197648 shutdown_connections 2024-03-21T11:46:45.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.535+0000 7f8bffe00640 1 -- 172.21.15.47:0/1233197648 wait complete. 2024-03-21T11:46:45.565 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:46:45.565 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:46:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.823+0000 7fe449000640 1 -- 172.21.15.47:0/1047485488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe444104ab0 msgr2=0x7fe444104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.823+0000 7fe449000640 1 --2- 172.21.15.47:0/1047485488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe444104ab0 0x7fe444104f30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe430009860 tx=0x7fe43002bb70 comp rx=0 tx=0).stop 2024-03-21T11:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.823+0000 7fe449000640 1 -- 172.21.15.47:0/1047485488 shutdown_connections 2024-03-21T11:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.823+0000 7fe449000640 1 --2- 172.21.15.47:0/1047485488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe444105470 0x7fe44410bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.823+0000 7fe449000640 1 --2- 172.21.15.47:0/1047485488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe444104ab0 0x7fe444104f30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.823+0000 7fe449000640 1 --2- 172.21.15.47:0/1047485488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4441038c0 0x7fe444103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.823+0000 7fe449000640 1 -- 172.21.15.47:0/1047485488 >> 172.21.15.47:0/1047485488 conn(0x7fe4440fee50 msgr2=0x7fe444101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.824+0000 7fe449000640 1 -- 172.21.15.47:0/1047485488 shutdown_connections 2024-03-21T11:46:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.824+0000 7fe449000640 1 -- 172.21.15.47:0/1047485488 wait complete. 2024-03-21T11:46:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.825+0000 7fe449000640 1 Processor -- start 2024-03-21T11:46:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.825+0000 7fe449000640 1 -- start start 2024-03-21T11:46:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.825+0000 7fe449000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4441038c0 0x7fe444199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.825+0000 7fe449000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe444104ab0 0x7fe44419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.825+0000 7fe449000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe444105470 0x7fe4441a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.825+0000 7fe449000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44410f5e0 con 0x7fe444104ab0 2024-03-21T11:46:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.825+0000 7fe449000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44410f460 con 0x7fe4441038c0 2024-03-21T11:46:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.825+0000 7fe449000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44410f760 con 0x7fe444105470 2024-03-21T11:46:45.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.826+0000 7fe442800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe444105470 0x7fe4441a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.826+0000 7fe442800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe444105470 0x7fe4441a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41210/0 (socket says 172.21.15.47:41210) 2024-03-21T11:46:45.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.826+0000 7fe441e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4441038c0 0x7fe444199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.826+0000 7fe441400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe444104ab0 0x7fe44419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.826+0000 7fe442800640 1 -- 172.21.15.47:0/260338765 learned_addr learned my addr 172.21.15.47:0/260338765 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:45.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.826+0000 7fe442800640 1 -- 172.21.15.47:0/260338765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4441038c0 msgr2=0x7fe444199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:45.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.826+0000 7fe442800640 1 --2- 172.21.15.47:0/260338765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4441038c0 0x7fe444199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.826+0000 7fe442800640 1 -- 172.21.15.47:0/260338765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe444104ab0 msgr2=0x7fe44419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:45.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.826+0000 7fe442800640 1 --2- 172.21.15.47:0/260338765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe444104ab0 0x7fe44419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.826+0000 7fe442800640 1 -- 172.21.15.47:0/260338765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4441a1890 con 0x7fe444105470 2024-03-21T11:46:45.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.826+0000 7fe441e00640 1 --2- 172.21.15.47:0/260338765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4441038c0 0x7fe444199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:45.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.826+0000 7fe442800640 1 --2- 172.21.15.47:0/260338765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe444105470 0x7fe4441a11f0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009fd0 tx=0x7fe43c00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:45.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.827+0000 7fe43aa00640 1 -- 172.21.15.47:0/260338765 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c004b20 con 0x7fe444105470 2024-03-21T11:46:45.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.827+0000 7fe449000640 1 -- 172.21.15.47:0/260338765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4441a1b80 con 0x7fe444105470 2024-03-21T11:46:45.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.827+0000 7fe449000640 1 -- 172.21.15.47:0/260338765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4441a2120 con 0x7fe444105470 2024-03-21T11:46:45.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.827+0000 7fe43aa00640 1 -- 172.21.15.47:0/260338765 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe43c003e50 con 0x7fe444105470 2024-03-21T11:46:45.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.827+0000 7fe43aa00640 1 -- 172.21.15.47:0/260338765 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c00b570 con 0x7fe444105470 2024-03-21T11:46:45.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.830+0000 7fe44bc1e740 1 Processor -- start 2024-03-21T11:46:45.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.832+0000 7fe43aa00640 1 -- 172.21.15.47:0/260338765 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe43c01d020 con 0x7fe444105470 2024-03-21T11:46:45.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.833+0000 7fe44bc1e740 1 -- start start 2024-03-21T11:46:45.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.834+0000 7fe44bc1e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb426440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.834+0000 7fe44bc1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb4271f0 0x5581cb427670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.834+0000 7fe44bc1e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb427bb0 0x5581cb42e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.834+0000 7fe44bc1e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cb3822c0 con 0x5581cb427bb0 2024-03-21T11:46:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.834+0000 7fe44bc1e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cb32ac90 con 0x5581cb426020 2024-03-21T11:46:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.834+0000 7fe44bc1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cb37e360 con 0x5581cb4271f0 2024-03-21T11:46:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.834+0000 7fe441400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb426440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.834+0000 7fe441e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb4271f0 0x5581cb427670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.834+0000 7fe441400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb426440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44194/0 (socket says 172.21.15.47:44194) 2024-03-21T11:46:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.834+0000 7fe441400640 1 -- 172.21.15.47:0/2194664400 learned_addr learned my addr 172.21.15.47:0/2194664400 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.835+0000 7fe441e00640 1 -- 172.21.15.47:0/2194664400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 msgr2=0x5581cb426440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.835+0000 7fe441e00640 1 --2- 172.21.15.47:0/2194664400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb426440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.835+0000 7fe441e00640 1 -- 172.21.15.47:0/2194664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb427bb0 msgr2=0x5581cb42e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.835+0000 7fe441400640 1 --2- 172.21.15.47:0/2194664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb427bb0 0x5581cb42e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.835+0000 7fe441e00640 1 --2- 172.21.15.47:0/2194664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb427bb0 0x5581cb42e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.835+0000 7fe441e00640 1 -- 172.21.15.47:0/2194664400 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581cb34f080 con 0x5581cb4271f0 2024-03-21T11:46:45.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.835+0000 7fe441400640 1 --2- 172.21.15.47:0/2194664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb427bb0 0x5581cb42e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:45.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.836+0000 7fe441e00640 1 --2- 172.21.15.47:0/2194664400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb4271f0 0x5581cb427670 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe444104910 tx=0x7fe43400e5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd4b43d0bb79635e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:45.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.837+0000 7fe43aa00640 1 --2- 172.21.15.47:0/260338765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4280a2fa0 0x7fe4280a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.837+0000 7fe43aa00640 1 -- 172.21.15.47:0/260338765 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe43c0cf570 con 0x7fe444105470 2024-03-21T11:46:45.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.837+0000 7fe441400640 1 --2- 172.21.15.47:0/260338765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4280a2fa0 0x7fe4280a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.837+0000 7fe42f400640 1 -- 172.21.15.47:0/2194664400 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434028070 con 0x5581cb4271f0 2024-03-21T11:46:45.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.837+0000 7fe42f400640 1 -- 172.21.15.47:0/2194664400 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe434024960 con 0x5581cb4271f0 2024-03-21T11:46:45.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.837+0000 7fe42f400640 1 -- 172.21.15.47:0/2194664400 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4340238a0 con 0x5581cb4271f0 2024-03-21T11:46:45.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.838+0000 7fe44bc1e740 1 -- 172.21.15.47:0/2194664400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb4271f0 msgr2=0x5581cb427670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:45.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.838+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/2194664400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb4271f0 0x5581cb427670 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe444104910 tx=0x7fe43400e5a0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.838+0000 7fe441400640 1 --2- 172.21.15.47:0/260338765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4280a2fa0 0x7fe4280a5460 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x5581cb4270f0 tx=0x7fe43002bb70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:45.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.839+0000 7fe44bc1e740 1 -- 172.21.15.47:0/2194664400 shutdown_connections 2024-03-21T11:46:45.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.839+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/2194664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb427bb0 0x5581cb42e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.839+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/2194664400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb4271f0 0x5581cb427670 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.839+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/2194664400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb426440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.839+0000 7fe44bc1e740 1 -- 172.21.15.47:0/2194664400 >> 172.21.15.47:0/2194664400 conn(0x5581cb421410 msgr2=0x5581cb421810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:45.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.840+0000 7fe44bc1e740 1 -- 172.21.15.47:0/2194664400 shutdown_connections 2024-03-21T11:46:45.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.840+0000 7fe44bc1e740 1 -- 172.21.15.47:0/2194664400 wait complete. 2024-03-21T11:46:45.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.840+0000 7fe44bc1e740 1 -- start start 2024-03-21T11:46:45.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.841+0000 7fe44bc1e740 1 Processor -- start 2024-03-21T11:46:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.841+0000 7fe44bc1e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb4302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.842+0000 7fe44bc1e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb4271f0 0x5581cb430830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.842+0000 7fe44bc1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb437870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.842+0000 7fe44bc1e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cb42ebc0 con 0x5581cb4271f0 2024-03-21T11:46:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.842+0000 7fe44bc1e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cb33e0d0 con 0x5581cb426020 2024-03-21T11:46:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.842+0000 7fe44bc1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581cb42ed40 con 0x5581cb427bb0 2024-03-21T11:46:45.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.842+0000 7fe44bc1e740 10 client.0 _ll_register_callbacks cb 0x5581cb342e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.842+0000 7fe442800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb437870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.842+0000 7fe442800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb437870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41240/0 (socket says 172.21.15.47:41240) 2024-03-21T11:46:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.842+0000 7fe442800640 1 -- 172.21.15.47:0/3828387874 learned_addr learned my addr 172.21.15.47:0/3828387874 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.842+0000 7fe441e00640 1 --2- 172.21.15.47:0/3828387874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb4302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.843+0000 7fe442800640 1 -- 172.21.15.47:0/3828387874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 msgr2=0x5581cb4302f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.843+0000 7fe442800640 1 --2- 172.21.15.47:0/3828387874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb4302f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.843+0000 7fe442800640 1 -- 172.21.15.47:0/3828387874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb4271f0 msgr2=0x5581cb430830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:45.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.843+0000 7fe442800640 1 --2- 172.21.15.47:0/3828387874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb4271f0 0x5581cb430830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.843+0000 7fe442800640 1 -- 172.21.15.47:0/3828387874 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581cb0963f0 con 0x5581cb427bb0 2024-03-21T11:46:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.844+0000 7fe442800640 1 --2- 172.21.15.47:0/3828387874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb437870 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009fa0 tx=0x7fe43c099a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=735cb431c90536a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.844+0000 7fe423e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.844+0000 7fe423e00640 1 -- 172.21.15.47:0/3828387874 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c01a050 con 0x5581cb427bb0 2024-03-21T11:46:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.845+0000 7fe423e00640 1 -- 172.21.15.47:0/3828387874 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe43c004cc0 con 0x5581cb427bb0 2024-03-21T11:46:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.845+0000 7fe44bc1e740 1 -- 172.21.15.47:0/3828387874 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe434024960 con 0x5581cb427bb0 2024-03-21T11:46:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.845+0000 7fe423e00640 1 -- 172.21.15.47:0/3828387874 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe43c09fa60 con 0x5581cb427bb0 2024-03-21T11:46:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.845+0000 7fe44bc1e740 10 client.32905 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:45.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.845+0000 7fe44bc1e740 1 -- 172.21.15.47:0/3828387874 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5581cb1607d0 con 0x5581cb427bb0 2024-03-21T11:46:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.846+0000 7fe423e00640 1 -- 172.21.15.47:0/3828387874 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe434024960 con 0x5581cb427bb0 2024-03-21T11:46:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.846+0000 7fe423e00640 1 -- 172.21.15.47:0/3828387874 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe43c00b040 con 0x5581cb427bb0 2024-03-21T11:46:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.846+0000 7fe44bc1e740 10 client.32905 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.846+0000 7fe44bc1e740 10 client.32905 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:46:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.846+0000 7fe44bc1e740 20 client.32905 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.846+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/3828387874 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5581cb43bdd0 0x5581cb43e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.846+0000 7fe44bc1e740 4 client.32905 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.846+0000 7fe44bc1e740 1 -- 172.21.15.47:0/3828387874 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581cb38f800 con 0x5581cb43bdd0 2024-03-21T11:46:45.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.847+0000 7fe441e00640 1 --2- 172.21.15.47:0/3828387874 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5581cb43bdd0 0x5581cb43e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.849+0000 7fe441e00640 1 --2- 172.21.15.47:0/3828387874 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5581cb43bdd0 0x5581cb43e230 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9fe42750f8143ff7 server_cookie=a0f36d9097714f8e in_seq=0 out_seq=0 2024-03-21T11:46:45.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.850+0000 7fe423e00640 10 client.32905 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:46:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.851+0000 7fe423e00640 1 -- 172.21.15.47:0/3828387874 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5581cb38f800 con 0x5581cb43bdd0 2024-03-21T11:46:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.851+0000 7fe423e00640 10 client.32905 handle_command_reply: tid=0 2024-03-21T11:46:45.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.851+0000 7fe44bc1e740 1 client.32905 shutdown 2024-03-21T11:46:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.852+0000 7fe44bc1e740 1 -- 172.21.15.47:0/3828387874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 msgr2=0x5581cb437870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:45.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.852+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/3828387874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb437870 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009fa0 tx=0x7fe43c099a50 comp rx=0 tx=0).stop 2024-03-21T11:46:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.855+0000 7fe44bc1e740 1 -- 172.21.15.47:0/3828387874 shutdown_connections 2024-03-21T11:46:45.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.855+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/3828387874 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5581cb43bdd0 0x5581cb43e230 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.855+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/3828387874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb437870 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.855+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/3828387874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb4271f0 0x5581cb430830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.855+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/3828387874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb4302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.855+0000 7fe423e00640 0 client.32905 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:46:45.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.855+0000 7fe44bc1e740 1 -- 172.21.15.47:0/3828387874 >> 172.21.15.47:0/3828387874 conn(0x5581cb421410 msgr2=0x5581cb417eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:45.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.856+0000 7fe44bc1e740 1 -- 172.21.15.47:0/3828387874 shutdown_connections 2024-03-21T11:46:45.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.856+0000 7fe44bc1e740 1 -- 172.21.15.47:0/3828387874 wait complete. 2024-03-21T11:46:45.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.856+0000 7fe44bc1e740 20 client.32905 trim_cache size 0 max 16384 2024-03-21T11:46:45.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.880+0000 7fe44bc1e740 1 Processor -- start 2024-03-21T11:46:45.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.880+0000 7fe44bc1e740 1 -- start start 2024-03-21T11:46:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.880+0000 7fe44bc1e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb424d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.880+0000 7fe44bc1e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb4271f0 0x5581cb437830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.880+0000 7fe44bc1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb42a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.880+0000 7fe44bc1e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42400f990 con 0x5581cb4271f0 2024-03-21T11:46:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.880+0000 7fe44bc1e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe424011310 con 0x5581cb426020 2024-03-21T11:46:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.880+0000 7fe44bc1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42400fff0 con 0x5581cb427bb0 2024-03-21T11:46:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.881+0000 7fe442800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb42a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.881+0000 7fe442800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb42a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41254/0 (socket says 172.21.15.47:41254) 2024-03-21T11:46:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.881+0000 7fe442800640 1 -- 172.21.15.47:0/1620302839 learned_addr learned my addr 172.21.15.47:0/1620302839 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.881+0000 7fe441e00640 1 --2- 172.21.15.47:0/1620302839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb424d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.882+0000 7fe441e00640 1 -- 172.21.15.47:0/1620302839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 msgr2=0x5581cb42a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.882+0000 7fe441400640 1 --2- 172.21.15.47:0/1620302839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb4271f0 0x5581cb437830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.883+0000 7fe441e00640 1 --2- 172.21.15.47:0/1620302839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb42a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.883+0000 7fe441e00640 1 -- 172.21.15.47:0/1620302839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb4271f0 msgr2=0x5581cb437830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:45.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.883+0000 7fe441e00640 1 --2- 172.21.15.47:0/1620302839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb4271f0 0x5581cb437830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.883+0000 7fe441e00640 1 -- 172.21.15.47:0/1620302839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581cb4132c0 con 0x5581cb426020 2024-03-21T11:46:45.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.883+0000 7fe441400640 1 --2- 172.21.15.47:0/1620302839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb4271f0 0x5581cb437830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:45.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.884+0000 7fe441e00640 1 --2- 172.21.15.47:0/1620302839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb424d80 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe434004700 tx=0x7fe434009110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b75224a886839b70 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.885+0000 7fe42ea00640 1 -- 172.21.15.47:0/1620302839 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434028040 con 0x5581cb426020 2024-03-21T11:46:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.885+0000 7fe42ea00640 1 -- 172.21.15.47:0/1620302839 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe434037ae0 con 0x5581cb426020 2024-03-21T11:46:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.885+0000 7fe42ea00640 1 -- 172.21.15.47:0/1620302839 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434023bf0 con 0x5581cb426020 2024-03-21T11:46:45.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.888+0000 7fe44bc1e740 1 -- 172.21.15.47:0/1620302839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 msgr2=0x5581cb424d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.888+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/1620302839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb424d80 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe434004700 tx=0x7fe434009110 comp rx=0 tx=0).stop 2024-03-21T11:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.888+0000 7fe44bc1e740 1 -- 172.21.15.47:0/1620302839 shutdown_connections 2024-03-21T11:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.888+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/1620302839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb42a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.888+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/1620302839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb4271f0 0x5581cb437830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.888+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/1620302839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb424d80 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.888+0000 7fe44bc1e740 1 -- 172.21.15.47:0/1620302839 >> 172.21.15.47:0/1620302839 conn(0x5581cb421410 msgr2=0x5581cb424670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.889+0000 7fe44bc1e740 1 -- 172.21.15.47:0/1620302839 shutdown_connections 2024-03-21T11:46:45.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.889+0000 7fe44bc1e740 1 -- 172.21.15.47:0/1620302839 wait complete. 2024-03-21T11:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.889+0000 7fe44bc1e740 1 -- start start 2024-03-21T11:46:45.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.890+0000 7fe44bc1e740 1 Processor -- start 2024-03-21T11:46:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.891+0000 7fe44bc1e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb431a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.891+0000 7fe44bc1e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb4271f0 0x5581cb431f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.891+0000 7fe44bc1e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb436c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.891+0000 7fe44bc1e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe424010cb0 con 0x5581cb4271f0 2024-03-21T11:46:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.891+0000 7fe44bc1e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe424010650 con 0x5581cb426020 2024-03-21T11:46:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.891+0000 7fe44bc1e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42400db50 con 0x5581cb427bb0 2024-03-21T11:46:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.892+0000 7fe44bc1e740 10 client.0 _ll_register_callbacks cb 0x7fe43c0048f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.892+0000 7fe442800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb436c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.892+0000 7fe442800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb436c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41268/0 (socket says 172.21.15.47:41268) 2024-03-21T11:46:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.892+0000 7fe442800640 1 -- 172.21.15.47:0/391894551 learned_addr learned my addr 172.21.15.47:0/391894551 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.892+0000 7fe441e00640 1 --2- 172.21.15.47:0/391894551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb431a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.892+0000 7fe442800640 1 -- 172.21.15.47:0/391894551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 msgr2=0x5581cb431a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.892+0000 7fe442800640 1 --2- 172.21.15.47:0/391894551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb431a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.892+0000 7fe442800640 1 -- 172.21.15.47:0/391894551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb4271f0 msgr2=0x5581cb431f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.892+0000 7fe442800640 1 --2- 172.21.15.47:0/391894551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb4271f0 0x5581cb431f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.892+0000 7fe442800640 1 -- 172.21.15.47:0/391894551 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581cb4372d0 con 0x5581cb427bb0 2024-03-21T11:46:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.893+0000 7fe442800640 1 --2- 172.21.15.47:0/391894551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb436c90 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0044c0 tx=0x7fe43c099e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0d6716bba097dbe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:45.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.893+0000 7fe423e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.897+0000 7fe423e00640 1 -- 172.21.15.47:0/391894551 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c01a070 con 0x5581cb427bb0 2024-03-21T11:46:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.898+0000 7fe423e00640 1 -- 172.21.15.47:0/391894551 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe43c0ba2a0 con 0x5581cb427bb0 2024-03-21T11:46:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.898+0000 7fe44bc1e740 1 -- 172.21.15.47:0/391894551 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe434037ae0 con 0x5581cb427bb0 2024-03-21T11:46:45.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.904+0000 7fe423e00640 1 -- 172.21.15.47:0/391894551 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe43c09b450 con 0x5581cb427bb0 2024-03-21T11:46:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.905+0000 7fe44bc1e740 10 client.32911 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:45.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.905+0000 7fe44bc1e740 1 -- 172.21.15.47:0/391894551 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5581cb41ae60 con 0x5581cb427bb0 2024-03-21T11:46:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.905+0000 7fe423e00640 1 -- 172.21.15.47:0/391894551 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe434037ae0 con 0x5581cb427bb0 2024-03-21T11:46:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.906+0000 7fe423e00640 1 -- 172.21.15.47:0/391894551 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe43c09ae80 con 0x5581cb427bb0 2024-03-21T11:46:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.906+0000 7fe44bc1e740 10 client.32911 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.906+0000 7fe44bc1e740 10 client.32911 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:46:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.906+0000 7fe44bc1e740 20 client.32911 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.906+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/391894551 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5581cb41b5d0 0x5581cb41d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.906+0000 7fe44bc1e740 4 client.32911 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:46:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.906+0000 7fe44bc1e740 1 -- 172.21.15.47:0/391894551 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5581cb389b50 con 0x5581cb41b5d0 2024-03-21T11:46:45.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.906+0000 7fe441e00640 1 --2- 172.21.15.47:0/391894551 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5581cb41b5d0 0x5581cb41d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.907+0000 7fe441e00640 1 --2- 172.21.15.47:0/391894551 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5581cb41b5d0 0x5581cb41d9c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89d46d878b78e92 server_cookie=babcb06a08d40960 in_seq=0 out_seq=0 2024-03-21T11:46:45.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.907+0000 7fe423e00640 10 client.32911 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:46:45.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.913+0000 7fe423e00640 1 -- 172.21.15.47:0/391894551 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5581cb389b50 con 0x5581cb41b5d0 2024-03-21T11:46:45.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.913+0000 7fe423e00640 10 client.32911 handle_command_reply: tid=0 2024-03-21T11:46:45.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.913+0000 7fe44bc1e740 1 client.32911 shutdown 2024-03-21T11:46:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.913+0000 7fe44bc1e740 1 -- 172.21.15.47:0/391894551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 msgr2=0x5581cb436c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.913+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/391894551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb436c90 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0044c0 tx=0x7fe43c099e20 comp rx=0 tx=0).stop 2024-03-21T11:46:45.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.913+0000 7fe44bc1e740 1 -- 172.21.15.47:0/391894551 shutdown_connections 2024-03-21T11:46:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.913+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/391894551 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x5581cb41b5d0 0x5581cb41d9c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.913+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/391894551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5581cb427bb0 0x5581cb436c90 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.913+0000 7fe423e00640 0 client.32911 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:46:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.913+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/391894551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5581cb4271f0 0x5581cb431f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.913+0000 7fe44bc1e740 1 --2- 172.21.15.47:0/391894551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581cb426020 0x5581cb431a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.913+0000 7fe44bc1e740 1 -- 172.21.15.47:0/391894551 >> 172.21.15.47:0/391894551 conn(0x5581cb421410 msgr2=0x5581cb424f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.914+0000 7fe44bc1e740 1 -- 172.21.15.47:0/391894551 shutdown_connections 2024-03-21T11:46:45.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.914+0000 7fe44bc1e740 1 -- 172.21.15.47:0/391894551 wait complete. 2024-03-21T11:46:45.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.914+0000 7fe44bc1e740 20 client.32911 trim_cache size 0 max 16384 2024-03-21T11:46:45.916 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:46:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.915+0000 7fe449000640 1 -- 172.21.15.47:0/260338765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4280a2fa0 msgr2=0x7fe4280a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.915+0000 7fe449000640 1 --2- 172.21.15.47:0/260338765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4280a2fa0 0x7fe4280a5460 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x5581cb4270f0 tx=0x7fe43002bb70 comp rx=0 tx=0).stop 2024-03-21T11:46:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.915+0000 7fe449000640 1 -- 172.21.15.47:0/260338765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe444105470 msgr2=0x7fe4441a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.915+0000 7fe449000640 1 --2- 172.21.15.47:0/260338765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe444105470 0x7fe4441a11f0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009fd0 tx=0x7fe43c00c740 comp rx=0 tx=0).stop 2024-03-21T11:46:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.915+0000 7fe449000640 1 -- 172.21.15.47:0/260338765 shutdown_connections 2024-03-21T11:46:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.915+0000 7fe449000640 1 --2- 172.21.15.47:0/260338765 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe4280a2fa0 0x7fe4280a5460 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.915+0000 7fe449000640 1 --2- 172.21.15.47:0/260338765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe444105470 0x7fe4441a11f0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.915+0000 7fe449000640 1 --2- 172.21.15.47:0/260338765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe444104ab0 0x7fe44419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.915+0000 7fe449000640 1 --2- 172.21.15.47:0/260338765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4441038c0 0x7fe444199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.915+0000 7fe449000640 1 -- 172.21.15.47:0/260338765 >> 172.21.15.47:0/260338765 conn(0x7fe4440fee50 msgr2=0x7fe444107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.915+0000 7fe449000640 1 -- 172.21.15.47:0/260338765 shutdown_connections 2024-03-21T11:46:45.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:45.915+0000 7fe449000640 1 -- 172.21.15.47:0/260338765 wait complete. 2024-03-21T11:46:45.936 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:46:45.936 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:46:45.989 INFO:tasks.workunit.client.0.smithi047.stdout: 10 239641 21.74 MB/sec execute 449 sec latency 79.785 ms 2024-03-21T11:46:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.191+0000 7fb634c00640 1 -- 172.21.15.47:0/2443432333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb63010e1e0 msgr2=0x7fb630110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.191+0000 7fb634c00640 1 --2- 172.21.15.47:0/2443432333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb63010e1e0 0x7fb630110670 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb628009ca0 tx=0x7fb62802c0f0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.191+0000 7fb634c00640 1 -- 172.21.15.47:0/2443432333 shutdown_connections 2024-03-21T11:46:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.191+0000 7fb634c00640 1 --2- 172.21.15.47:0/2443432333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb63010e1e0 0x7fb630110670 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.191+0000 7fb634c00640 1 --2- 172.21.15.47:0/2443432333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63010b780 0x7fb63010db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.191+0000 7fb634c00640 1 --2- 172.21.15.47:0/2443432333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb63006a4b0 0x7fb63010b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.191+0000 7fb634c00640 1 -- 172.21.15.47:0/2443432333 >> 172.21.15.47:0/2443432333 conn(0x7fb63006e8a0 msgr2=0x7fb630070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.191+0000 7fb634c00640 1 -- 172.21.15.47:0/2443432333 shutdown_connections 2024-03-21T11:46:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.191+0000 7fb634c00640 1 -- 172.21.15.47:0/2443432333 wait complete. 2024-03-21T11:46:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.192+0000 7fb634c00640 1 Processor -- start 2024-03-21T11:46:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.192+0000 7fb634c00640 1 -- start start 2024-03-21T11:46:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.193+0000 7fb634c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb63006a4b0 0x7fb630119680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.193+0000 7fb634c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63010b780 0x7fb630119bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.193+0000 7fb634c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb63010e1e0 0x7fb630117cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.193+0000 7fb634c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6301125e0 con 0x7fb63010e1e0 2024-03-21T11:46:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.193+0000 7fb634c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb630112460 con 0x7fb63010b780 2024-03-21T11:46:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.193+0000 7fb634c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb630112760 con 0x7fb63006a4b0 2024-03-21T11:46:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.193+0000 7fb62e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb63010e1e0 0x7fb630117cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.193+0000 7fb62e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb63010e1e0 0x7fb630117cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60828/0 (socket says 172.21.15.47:60828) 2024-03-21T11:46:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.193+0000 7fb62e800640 1 -- 172.21.15.47:0/4089124304 learned_addr learned my addr 172.21.15.47:0/4089124304 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.194+0000 7fb62d400640 1 --2- 172.21.15.47:0/4089124304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63010b780 0x7fb630119bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.194+0000 7fb62d400640 1 -- 172.21.15.47:0/4089124304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb63006a4b0 msgr2=0x7fb630119680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T11:46:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.194+0000 7fb62d400640 1 --2- 172.21.15.47:0/4089124304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb63006a4b0 0x7fb630119680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.194+0000 7fb62d400640 1 -- 172.21.15.47:0/4089124304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb63010e1e0 msgr2=0x7fb630117cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.194+0000 7fb62d400640 1 --2- 172.21.15.47:0/4089124304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb63010e1e0 0x7fb630117cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.194+0000 7fb62d400640 1 -- 172.21.15.47:0/4089124304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6301183c0 con 0x7fb63010b780 2024-03-21T11:46:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.195+0000 7fb62d400640 1 --2- 172.21.15.47:0/4089124304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63010b780 0x7fb630119bc0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005fd0 tx=0x7fb61c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.195+0000 7fb626a00640 1 -- 172.21.15.47:0/4089124304 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c009c00 con 0x7fb63010b780 2024-03-21T11:46:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.196+0000 7fb626a00640 1 -- 172.21.15.47:0/4089124304 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb61c009da0 con 0x7fb63010b780 2024-03-21T11:46:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.196+0000 7fb626a00640 1 -- 172.21.15.47:0/4089124304 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c010460 con 0x7fb63010b780 2024-03-21T11:46:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.196+0000 7fb634c00640 1 -- 172.21.15.47:0/4089124304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6301186b0 con 0x7fb63010b780 2024-03-21T11:46:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.196+0000 7fb634c00640 1 -- 172.21.15.47:0/4089124304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6301b74b0 con 0x7fb63010b780 2024-03-21T11:46:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.202+0000 7fb637813740 1 Processor -- start 2024-03-21T11:46:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.203+0000 7fb637813740 1 -- start start 2024-03-21T11:46:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.203+0000 7fb626a00640 1 -- 172.21.15.47:0/4089124304 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb61c003f20 con 0x7fb63010b780 2024-03-21T11:46:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.203+0000 7fb637813740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e1810 0x5630bb4e1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.203+0000 7fb637813740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e29e0 0x5630bb4e2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.203+0000 7fb637813740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e33a0 0x5630bb4e9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.203+0000 7fb637813740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bb4462c0 con 0x5630bb4e29e0 2024-03-21T11:46:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.203+0000 7fb637813740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bb3eec90 con 0x5630bb4e33a0 2024-03-21T11:46:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.203+0000 7fb637813740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bb442360 con 0x5630bb4e1810 2024-03-21T11:46:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.204+0000 7fb62d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e33a0 0x5630bb4e9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.204+0000 7fb62d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e33a0 0x5630bb4e9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44250/0 (socket says 172.21.15.47:44250) 2024-03-21T11:46:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.204+0000 7fb62d400640 1 -- 172.21.15.47:0/1328070214 learned_addr learned my addr 172.21.15.47:0/1328070214 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.204+0000 7fb62e800640 1 --2- 172.21.15.47:0/1328070214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e1810 0x5630bb4e1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.204+0000 7fb62d400640 1 -- 172.21.15.47:0/1328070214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e1810 msgr2=0x5630bb4e1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.204+0000 7fb62d400640 1 --2- 172.21.15.47:0/1328070214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e1810 0x5630bb4e1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.204+0000 7fb62d400640 1 -- 172.21.15.47:0/1328070214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e29e0 msgr2=0x5630bb4e2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.204+0000 7fb62d400640 1 --2- 172.21.15.47:0/1328070214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e29e0 0x5630bb4e2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.204+0000 7fb62d400640 1 -- 172.21.15.47:0/1328070214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630bb413080 con 0x5630bb4e33a0 2024-03-21T11:46:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.205+0000 7fb626a00640 1 --2- 172.21.15.47:0/4089124304 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb6140a3050 0x7fb6140a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.205+0000 7fb62d400640 1 --2- 172.21.15.47:0/1328070214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e33a0 0x5630bb4e9c30 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0d7320 tx=0x7fb61c0d77e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68ec019b990cbd51 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.205+0000 7fb626a00640 1 -- 172.21.15.47:0/4089124304 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb61c0cf1a0 con 0x7fb63010b780 2024-03-21T11:46:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.205+0000 7fb61b000640 1 -- 172.21.15.47:0/1328070214 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c097780 con 0x5630bb4e33a0 2024-03-21T11:46:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.205+0000 7fb61b000640 1 -- 172.21.15.47:0/1328070214 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb61c09bbe0 con 0x5630bb4e33a0 2024-03-21T11:46:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.205+0000 7fb61b000640 1 -- 172.21.15.47:0/1328070214 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c09ae40 con 0x5630bb4e33a0 2024-03-21T11:46:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.206+0000 7fb62e800640 1 --2- 172.21.15.47:0/4089124304 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb6140a3050 0x7fb6140a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.208+0000 7fb62e800640 1 --2- 172.21.15.47:0/4089124304 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb6140a3050 0x7fb6140a5510 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb628032420 tx=0x7fb62802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.208+0000 7fb637813740 1 -- 172.21.15.47:0/1328070214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e33a0 msgr2=0x5630bb4e9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.208+0000 7fb637813740 1 --2- 172.21.15.47:0/1328070214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e33a0 0x5630bb4e9c30 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0d7320 tx=0x7fb61c0d77e0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.209+0000 7fb637813740 1 -- 172.21.15.47:0/1328070214 shutdown_connections 2024-03-21T11:46:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.209+0000 7fb637813740 1 --2- 172.21.15.47:0/1328070214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e33a0 0x5630bb4e9c30 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.209+0000 7fb637813740 1 --2- 172.21.15.47:0/1328070214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e29e0 0x5630bb4e2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.209+0000 7fb637813740 1 --2- 172.21.15.47:0/1328070214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e1810 0x5630bb4e1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.209+0000 7fb637813740 1 -- 172.21.15.47:0/1328070214 >> 172.21.15.47:0/1328070214 conn(0x5630bb4dcc00 msgr2=0x5630bb4dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.209+0000 7fb637813740 1 -- 172.21.15.47:0/1328070214 shutdown_connections 2024-03-21T11:46:46.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.211+0000 7fb637813740 1 -- 172.21.15.47:0/1328070214 wait complete. 2024-03-21T11:46:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.211+0000 7fb637813740 1 -- start start 2024-03-21T11:46:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.211+0000 7fb637813740 1 Processor -- start 2024-03-21T11:46:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.213+0000 7fb637813740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e1810 0x5630bb4eba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.213+0000 7fb637813740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e29e0 0x5630bb4ebf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.213+0000 7fb637813740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e33a0 0x5630bb4f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.213+0000 7fb637813740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bb4ea2f0 con 0x5630bb4e29e0 2024-03-21T11:46:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.213+0000 7fb637813740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bb4020d0 con 0x5630bb4e1810 2024-03-21T11:46:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.213+0000 7fb637813740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bb4ea470 con 0x5630bb4e33a0 2024-03-21T11:46:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.213+0000 7fb637813740 10 client.0 _ll_register_callbacks cb 0x5630bb406e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.213+0000 7fb62d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e29e0 0x5630bb4ebf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.213+0000 7fb62d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e29e0 0x5630bb4ebf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60836/0 (socket says 172.21.15.47:60836) 2024-03-21T11:46:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.213+0000 7fb62d400640 1 -- 172.21.15.47:0/2987243191 learned_addr learned my addr 172.21.15.47:0/2987243191 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.214+0000 7fb62e800640 1 --2- 172.21.15.47:0/2987243191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e33a0 0x5630bb4f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.213+0000 7fb62de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e1810 0x5630bb4eba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.214+0000 7fb62d400640 1 -- 172.21.15.47:0/2987243191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e33a0 msgr2=0x5630bb4f2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.214+0000 7fb62d400640 1 --2- 172.21.15.47:0/2987243191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e33a0 0x5630bb4f2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.214+0000 7fb62d400640 1 -- 172.21.15.47:0/2987243191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e1810 msgr2=0x5630bb4eba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.214+0000 7fb62d400640 1 --2- 172.21.15.47:0/2987243191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e1810 0x5630bb4eba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.214+0000 7fb62d400640 1 -- 172.21.15.47:0/2987243191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630bb15a3f0 con 0x5630bb4e29e0 2024-03-21T11:46:46.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.214+0000 7fb62e800640 1 --2- 172.21.15.47:0/2987243191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e33a0 0x5630bb4f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:46.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.214+0000 7fb62de00640 1 --2- 172.21.15.47:0/2987243191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e1810 0x5630bb4eba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:46.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.215+0000 7fb62d400640 1 --2- 172.21.15.47:0/2987243191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e29e0 0x5630bb4ebf80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb61c097e60 tx=0x7fb61c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c8783f0d7ecb68 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:46.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.215+0000 7fb60f400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:46:46.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.215+0000 7fb60f400640 1 -- 172.21.15.47:0/2987243191 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c020030 con 0x5630bb4e29e0 2024-03-21T11:46:46.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.215+0000 7fb60f400640 1 -- 172.21.15.47:0/2987243191 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb61c0035f0 con 0x5630bb4e29e0 2024-03-21T11:46:46.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.215+0000 7fb637813740 1 -- 172.21.15.47:0/2987243191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb61c09bbe0 con 0x5630bb4e29e0 2024-03-21T11:46:46.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.216+0000 7fb60f400640 1 -- 172.21.15.47:0/2987243191 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb61c09b020 con 0x5630bb4e29e0 2024-03-21T11:46:46.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.216+0000 7fb637813740 10 client.22128 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:46.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.216+0000 7fb637813740 1 -- 172.21.15.47:0/2987243191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5630bb2247d0 con 0x5630bb4e29e0 2024-03-21T11:46:46.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.216+0000 7fb60f400640 1 -- 172.21.15.47:0/2987243191 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb61c09bbe0 con 0x5630bb4e29e0 2024-03-21T11:46:46.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.217+0000 7fb60f400640 1 -- 172.21.15.47:0/2987243191 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb61c09b740 con 0x5630bb4e29e0 2024-03-21T11:46:46.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.217+0000 7fb637813740 10 client.22128 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.217+0000 7fb637813740 10 client.22128 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:46:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.218+0000 7fb637813740 20 client.22128 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:46.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.218+0000 7fb637813740 1 --2- 172.21.15.47:0/2987243191 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5630bb4f7120 0x5630bb4f9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.218+0000 7fb637813740 4 client.22128 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.218+0000 7fb637813740 1 -- 172.21.15.47:0/2987243191 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630bb453800 con 0x5630bb4f7120 2024-03-21T11:46:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.218+0000 7fb62de00640 1 --2- 172.21.15.47:0/2987243191 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5630bb4f7120 0x5630bb4f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.219+0000 7fb60f400640 10 client.22128 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:46:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.220+0000 7fb62de00640 1 --2- 172.21.15.47:0/2987243191 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5630bb4f7120 0x5630bb4f9510 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a06e5024d2e889cd server_cookie=b14af1d0196a192c in_seq=0 out_seq=0 2024-03-21T11:46:46.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.221+0000 7fb60f400640 1 -- 172.21.15.47:0/2987243191 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5630bb453800 con 0x5630bb4f7120 2024-03-21T11:46:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.221+0000 7fb60f400640 10 client.22128 handle_command_reply: tid=0 2024-03-21T11:46:46.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.221+0000 7fb637813740 1 client.22128 shutdown 2024-03-21T11:46:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.221+0000 7fb637813740 1 -- 172.21.15.47:0/2987243191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e29e0 msgr2=0x5630bb4ebf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.221+0000 7fb637813740 1 --2- 172.21.15.47:0/2987243191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e29e0 0x5630bb4ebf80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb61c097e60 tx=0x7fb61c0039c0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.222+0000 7fb637813740 1 -- 172.21.15.47:0/2987243191 shutdown_connections 2024-03-21T11:46:46.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.222+0000 7fb637813740 1 --2- 172.21.15.47:0/2987243191 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5630bb4f7120 0x5630bb4f9510 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.222+0000 7fb637813740 1 --2- 172.21.15.47:0/2987243191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e33a0 0x5630bb4f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.222+0000 7fb637813740 1 --2- 172.21.15.47:0/2987243191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e29e0 0x5630bb4ebf80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.222+0000 7fb637813740 1 --2- 172.21.15.47:0/2987243191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e1810 0x5630bb4eba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.222+0000 7fb637813740 1 -- 172.21.15.47:0/2987243191 >> 172.21.15.47:0/2987243191 conn(0x5630bb4dcc00 msgr2=0x5630bb4dbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.222+0000 7fb60f400640 0 client.22128 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:46:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.222+0000 7fb637813740 1 -- 172.21.15.47:0/2987243191 shutdown_connections 2024-03-21T11:46:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.222+0000 7fb637813740 1 -- 172.21.15.47:0/2987243191 wait complete. 2024-03-21T11:46:46.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.222+0000 7fb637813740 20 client.22128 trim_cache size 0 max 16384 2024-03-21T11:46:46.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:45 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1233197648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:46:46.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.243+0000 7fb637813740 1 Processor -- start 2024-03-21T11:46:46.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.243+0000 7fb637813740 1 -- start start 2024-03-21T11:46:46.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.244+0000 7fb637813740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e1810 0x5630bb4f9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.244+0000 7fb637813740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e33a0 0x5630bb4f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.244+0000 7fb637813740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e5b20 0x5630bb4e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.244+0000 7fb637813740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6301c1810 con 0x5630bb4e33a0 2024-03-21T11:46:46.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.244+0000 7fb637813740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6301c3190 con 0x5630bb4e5b20 2024-03-21T11:46:46.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.244+0000 7fb637813740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6301c1e70 con 0x5630bb4e1810 2024-03-21T11:46:46.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.244+0000 7fb62de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e1810 0x5630bb4f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.244+0000 7fb62de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e1810 0x5630bb4f9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41310/0 (socket says 172.21.15.47:41310) 2024-03-21T11:46:46.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.244+0000 7fb62de00640 1 -- 172.21.15.47:0/2189811828 learned_addr learned my addr 172.21.15.47:0/2189811828 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:46.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.244+0000 7fb62e800640 1 --2- 172.21.15.47:0/2189811828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e5b20 0x5630bb4e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.244+0000 7fb62d400640 1 --2- 172.21.15.47:0/2189811828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e33a0 0x5630bb4f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.244+0000 7fb62de00640 1 -- 172.21.15.47:0/2189811828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e5b20 msgr2=0x5630bb4e5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.244+0000 7fb62de00640 1 --2- 172.21.15.47:0/2189811828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e5b20 0x5630bb4e5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.245+0000 7fb62de00640 1 -- 172.21.15.47:0/2189811828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e33a0 msgr2=0x5630bb4f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.245+0000 7fb62de00640 1 --2- 172.21.15.47:0/2189811828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e33a0 0x5630bb4f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.245+0000 7fb62de00640 1 -- 172.21.15.47:0/2189811828 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630bb4d8810 con 0x5630bb4e1810 2024-03-21T11:46:46.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.245+0000 7fb62d400640 1 --2- 172.21.15.47:0/2189811828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e33a0 0x5630bb4f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.245+0000 7fb62de00640 1 --2- 172.21.15.47:0/2189811828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e1810 0x5630bb4f9130 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x5630bb4ddb70 tx=0x7fb620005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3884d58dcfae2252 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.245+0000 7fb61a600640 1 -- 172.21.15.47:0/2189811828 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620019a90 con 0x5630bb4e1810 2024-03-21T11:46:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.245+0000 7fb61a600640 1 -- 172.21.15.47:0/2189811828 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb62001f070 con 0x5630bb4e1810 2024-03-21T11:46:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.245+0000 7fb61a600640 1 -- 172.21.15.47:0/2189811828 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620004b00 con 0x5630bb4e1810 2024-03-21T11:46:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.246+0000 7fb637813740 1 -- 172.21.15.47:0/2189811828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e1810 msgr2=0x5630bb4f9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.246+0000 7fb637813740 1 --2- 172.21.15.47:0/2189811828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e1810 0x5630bb4f9130 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x5630bb4ddb70 tx=0x7fb620005c90 comp rx=0 tx=0).stop 2024-03-21T11:46:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.246+0000 7fb637813740 1 -- 172.21.15.47:0/2189811828 shutdown_connections 2024-03-21T11:46:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.246+0000 7fb637813740 1 --2- 172.21.15.47:0/2189811828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e5b20 0x5630bb4e5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.246+0000 7fb637813740 1 --2- 172.21.15.47:0/2189811828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e33a0 0x5630bb4f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.246+0000 7fb637813740 1 --2- 172.21.15.47:0/2189811828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e1810 0x5630bb4f9130 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.246+0000 7fb637813740 1 -- 172.21.15.47:0/2189811828 >> 172.21.15.47:0/2189811828 conn(0x5630bb4dcc00 msgr2=0x5630bb4e06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.247+0000 7fb637813740 1 -- 172.21.15.47:0/2189811828 shutdown_connections 2024-03-21T11:46:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.247+0000 7fb637813740 1 -- 172.21.15.47:0/2189811828 wait complete. 2024-03-21T11:46:46.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.247+0000 7fb637813740 1 -- start start 2024-03-21T11:46:46.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.247+0000 7fb637813740 1 Processor -- start 2024-03-21T11:46:46.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.248+0000 7fb637813740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e1810 0x5630bb4efb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.248+0000 7fb637813740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e33a0 0x5630bb4f0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.248+0000 7fb637813740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e5b20 0x5630bb552760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.248+0000 7fb637813740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6301c2b30 con 0x5630bb4e5b20 2024-03-21T11:46:46.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.248+0000 7fb637813740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6301c24d0 con 0x5630bb4e1810 2024-03-21T11:46:46.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.248+0000 7fb637813740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6301bf9d0 con 0x5630bb4e33a0 2024-03-21T11:46:46.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.248+0000 7fb637813740 10 client.0 _ll_register_callbacks cb 0x7fb61c0d7170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:46.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.248+0000 7fb62de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e1810 0x5630bb4efb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.248+0000 7fb62de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e1810 0x5630bb4efb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44284/0 (socket says 172.21.15.47:44284) 2024-03-21T11:46:46.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.248+0000 7fb62d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e33a0 0x5630bb4f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.248+0000 7fb62de00640 1 -- 172.21.15.47:0/2762483163 learned_addr learned my addr 172.21.15.47:0/2762483163 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:46.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.249+0000 7fb62de00640 1 -- 172.21.15.47:0/2762483163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e33a0 msgr2=0x5630bb4f0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.249+0000 7fb62de00640 1 --2- 172.21.15.47:0/2762483163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e33a0 0x5630bb4f0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.249+0000 7fb62de00640 1 -- 172.21.15.47:0/2762483163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e5b20 msgr2=0x5630bb552760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.249+0000 7fb62de00640 1 --2- 172.21.15.47:0/2762483163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e5b20 0x5630bb552760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.249+0000 7fb62de00640 1 -- 172.21.15.47:0/2762483163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630bb4dd670 con 0x5630bb4e1810 2024-03-21T11:46:46.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.249+0000 7fb62d400640 1 --2- 172.21.15.47:0/2762483163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e33a0 0x5630bb4f0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:46.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.249+0000 7fb62de00640 1 --2- 172.21.15.47:0/2762483163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e1810 0x5630bb4efb00 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fb620019b20 tx=0x7fb620019b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b75cb33e708794d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:46.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.249+0000 7fb60f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:46.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.250+0000 7fb60f400640 1 -- 172.21.15.47:0/2762483163 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620004c90 con 0x5630bb4e1810 2024-03-21T11:46:46.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.250+0000 7fb637813740 1 -- 172.21.15.47:0/2762483163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb62001f070 con 0x5630bb4e1810 2024-03-21T11:46:46.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.250+0000 7fb60f400640 1 -- 172.21.15.47:0/2762483163 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb620005050 con 0x5630bb4e1810 2024-03-21T11:46:46.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.250+0000 7fb60f400640 1 -- 172.21.15.47:0/2762483163 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb620031050 con 0x5630bb4e1810 2024-03-21T11:46:46.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.250+0000 7fb637813740 10 client.32942 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:46.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.250+0000 7fb637813740 1 -- 172.21.15.47:0/2762483163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5630bb553a20 con 0x5630bb4e1810 2024-03-21T11:46:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.250+0000 7fb60f400640 1 -- 172.21.15.47:0/2762483163 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb62001f070 con 0x5630bb4e1810 2024-03-21T11:46:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.251+0000 7fb60f400640 1 -- 172.21.15.47:0/2762483163 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb62000f040 con 0x5630bb4e1810 2024-03-21T11:46:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.251+0000 7fb637813740 10 client.32942 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.251+0000 7fb637813740 10 client.32942 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:46:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.251+0000 7fb637813740 20 client.32942 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.251+0000 7fb637813740 1 --2- 172.21.15.47:0/2762483163 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5630bb554340 0x5630bb556760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.251+0000 7fb637813740 4 client.32942 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:46:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.251+0000 7fb637813740 1 -- 172.21.15.47:0/2762483163 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5630bb44db50 con 0x5630bb554340 2024-03-21T11:46:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.251+0000 7fb62d400640 1 --2- 172.21.15.47:0/2762483163 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5630bb554340 0x5630bb556760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb62d400640 1 --2- 172.21.15.47:0/2762483163 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5630bb554340 0x5630bb556760 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=700432e407431d0d server_cookie=87840a9ebd9c83dd in_seq=0 out_seq=0 2024-03-21T11:46:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb60f400640 10 client.32942 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:46:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb60f400640 1 -- 172.21.15.47:0/2762483163 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5630bb44db50 con 0x5630bb554340 2024-03-21T11:46:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb60f400640 10 client.32942 handle_command_reply: tid=0 2024-03-21T11:46:46.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb637813740 1 client.32942 shutdown 2024-03-21T11:46:46.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb637813740 1 -- 172.21.15.47:0/2762483163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e1810 msgr2=0x5630bb4efb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb637813740 1 --2- 172.21.15.47:0/2762483163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e1810 0x5630bb4efb00 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fb620019b20 tx=0x7fb620019b50 comp rx=0 tx=0).stop 2024-03-21T11:46:46.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb637813740 1 -- 172.21.15.47:0/2762483163 shutdown_connections 2024-03-21T11:46:46.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb637813740 1 --2- 172.21.15.47:0/2762483163 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x5630bb554340 0x5630bb556760 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb637813740 1 --2- 172.21.15.47:0/2762483163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5630bb4e5b20 0x5630bb552760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb637813740 1 --2- 172.21.15.47:0/2762483163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5630bb4e33a0 0x5630bb4f0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb637813740 1 --2- 172.21.15.47:0/2762483163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5630bb4e1810 0x5630bb4efb00 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb60f400640 0 client.32942 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:46:46.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb637813740 1 -- 172.21.15.47:0/2762483163 >> 172.21.15.47:0/2762483163 conn(0x5630bb4dcc00 msgr2=0x5630bb4e06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:46.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.252+0000 7fb637813740 1 -- 172.21.15.47:0/2762483163 shutdown_connections 2024-03-21T11:46:46.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.253+0000 7fb637813740 1 -- 172.21.15.47:0/2762483163 wait complete. 2024-03-21T11:46:46.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.253+0000 7fb637813740 20 client.32942 trim_cache size 0 max 16384 2024-03-21T11:46:46.257 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:46:46.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.254+0000 7fb634c00640 1 -- 172.21.15.47:0/4089124304 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb6140a3050 msgr2=0x7fb6140a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.254+0000 7fb634c00640 1 --2- 172.21.15.47:0/4089124304 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb6140a3050 0x7fb6140a5510 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb628032420 tx=0x7fb62802e040 comp rx=0 tx=0).stop 2024-03-21T11:46:46.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.254+0000 7fb634c00640 1 -- 172.21.15.47:0/4089124304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63010b780 msgr2=0x7fb630119bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.254+0000 7fb634c00640 1 --2- 172.21.15.47:0/4089124304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63010b780 0x7fb630119bc0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005fd0 tx=0x7fb61c00dde0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.254+0000 7fb634c00640 1 -- 172.21.15.47:0/4089124304 shutdown_connections 2024-03-21T11:46:46.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.254+0000 7fb634c00640 1 --2- 172.21.15.47:0/4089124304 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb6140a3050 0x7fb6140a5510 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.254+0000 7fb634c00640 1 --2- 172.21.15.47:0/4089124304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb63010e1e0 0x7fb630117cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.254+0000 7fb634c00640 1 --2- 172.21.15.47:0/4089124304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb63010b780 0x7fb630119bc0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.254+0000 7fb634c00640 1 --2- 172.21.15.47:0/4089124304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb63006a4b0 0x7fb630119680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.254+0000 7fb634c00640 1 -- 172.21.15.47:0/4089124304 >> 172.21.15.47:0/4089124304 conn(0x7fb63006e8a0 msgr2=0x7fb630109700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:46.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.254+0000 7fb634c00640 1 -- 172.21.15.47:0/4089124304 shutdown_connections 2024-03-21T11:46:46.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.254+0000 7fb634c00640 1 -- 172.21.15.47:0/4089124304 wait complete. 2024-03-21T11:46:46.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:45 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1233197648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:46:46.277 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:46:46.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:46:46.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:45 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1233197648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:46:46.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.529+0000 7fc9b8000640 1 -- 172.21.15.47:0/2885976008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b01038e0 msgr2=0x7fc9b0103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.529+0000 7fc9b8000640 1 --2- 172.21.15.47:0/2885976008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b01038e0 0x7fc9b0103d00 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009900 tx=0x7fc9a402bc40 comp rx=0 tx=0).stop 2024-03-21T11:46:46.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.530+0000 7fc9b8000640 1 -- 172.21.15.47:0/2885976008 shutdown_connections 2024-03-21T11:46:46.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.530+0000 7fc9b8000640 1 --2- 172.21.15.47:0/2885976008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b0105490 0x7fc9b010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.530+0000 7fc9b8000640 1 --2- 172.21.15.47:0/2885976008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9b0104ad0 0x7fc9b0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.530+0000 7fc9b8000640 1 --2- 172.21.15.47:0/2885976008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b01038e0 0x7fc9b0103d00 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.530+0000 7fc9b8000640 1 -- 172.21.15.47:0/2885976008 >> 172.21.15.47:0/2885976008 conn(0x7fc9b00fee50 msgr2=0x7fc9b0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:46.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.530+0000 7fc9b8000640 1 -- 172.21.15.47:0/2885976008 shutdown_connections 2024-03-21T11:46:46.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.530+0000 7fc9b8000640 1 -- 172.21.15.47:0/2885976008 wait complete. 2024-03-21T11:46:46.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.531+0000 7fc9b8000640 1 Processor -- start 2024-03-21T11:46:46.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.532+0000 7fc9b8000640 1 -- start start 2024-03-21T11:46:46.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.532+0000 7fc9b8000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b01038e0 0x7fc9b019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.532+0000 7fc9b8000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b0104ad0 0x7fc9b019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.532+0000 7fc9b8000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9b0105490 0x7fc9b01a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.532+0000 7fc9b8000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b010f530 con 0x7fc9b0105490 2024-03-21T11:46:46.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.532+0000 7fc9b8000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b010f3b0 con 0x7fc9b01038e0 2024-03-21T11:46:46.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.532+0000 7fc9b8000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b010f6b0 con 0x7fc9b0104ad0 2024-03-21T11:46:46.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.532+0000 7fc9b5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b01038e0 0x7fc9b019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.533+0000 7fc9b4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b0104ad0 0x7fc9b019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.533+0000 7fc9b5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b01038e0 0x7fc9b019e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44294/0 (socket says 172.21.15.47:44294) 2024-03-21T11:46:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.533+0000 7fc9b5600640 1 -- 172.21.15.47:0/1704031036 learned_addr learned my addr 172.21.15.47:0/1704031036 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.533+0000 7fc9b6000640 1 --2- 172.21.15.47:0/1704031036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9b0105490 0x7fc9b01a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.533+0000 7fc9b5600640 1 -- 172.21.15.47:0/1704031036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b0104ad0 msgr2=0x7fc9b019e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.533+0000 7fc9b5600640 1 --2- 172.21.15.47:0/1704031036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b0104ad0 0x7fc9b019e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.533+0000 7fc9b5600640 1 -- 172.21.15.47:0/1704031036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9b0105490 msgr2=0x7fc9b01a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.533+0000 7fc9b5600640 1 --2- 172.21.15.47:0/1704031036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9b0105490 0x7fc9b01a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.533+0000 7fc9b5600640 1 -- 172.21.15.47:0/1704031036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b01a5c40 con 0x7fc9b01038e0 2024-03-21T11:46:46.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.533+0000 7fc9b6000640 1 --2- 172.21.15.47:0/1704031036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9b0105490 0x7fc9b01a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:46.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.534+0000 7fc9b5600640 1 --2- 172.21.15.47:0/1704031036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b01038e0 0x7fc9b019e020 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4000c00 tx=0x7fc9a4005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:46.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.534+0000 7fc9ae000640 1 -- 172.21.15.47:0/1704031036 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4041070 con 0x7fc9b01038e0 2024-03-21T11:46:46.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.534+0000 7fc9ae000640 1 -- 172.21.15.47:0/1704031036 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9a40044d0 con 0x7fc9b01038e0 2024-03-21T11:46:46.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.534+0000 7fc9ae000640 1 -- 172.21.15.47:0/1704031036 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4034440 con 0x7fc9b01038e0 2024-03-21T11:46:46.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.535+0000 7fc9b8000640 1 -- 172.21.15.47:0/1704031036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b01a5ed0 con 0x7fc9b01038e0 2024-03-21T11:46:46.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.535+0000 7fc9b8000640 1 -- 172.21.15.47:0/1704031036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b01a62e0 con 0x7fc9b01038e0 2024-03-21T11:46:46.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.544+0000 7fc9badb3740 1 Processor -- start 2024-03-21T11:46:46.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.544+0000 7fc9badb3740 1 -- start start 2024-03-21T11:46:46.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.545+0000 7fc9ae000640 1 -- 172.21.15.47:0/1704031036 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc9a4044070 con 0x7fc9b01038e0 2024-03-21T11:46:46.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.547+0000 7fc9badb3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff04c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.547+0000 7fc9b6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff04c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.547+0000 7fc9b6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff04c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41360/0 (socket says 172.21.15.47:41360) 2024-03-21T11:46:46.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.547+0000 7fc9badb3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 0x55fc6ff05e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.548+0000 7fc9badb3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 0x55fc6ff0cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.548+0000 7fc9badb3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6fe582c0 con 0x55fc6ff059f0 2024-03-21T11:46:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.548+0000 7fc9badb3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6fe00c90 con 0x55fc6ff063b0 2024-03-21T11:46:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.548+0000 7fc9badb3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6fe54360 con 0x55fc6ff04820 2024-03-21T11:46:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.548+0000 7fc9ae000640 1 --2- 172.21.15.47:0/1704031036 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9980a3020 0x7fc9980a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.548+0000 7fc9b6000640 1 -- 172.21.15.47:0/487860494 learned_addr learned my addr 172.21.15.47:0/487860494 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.548+0000 7fc9b4c00640 1 --2- 172.21.15.47:0/487860494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 0x55fc6ff0cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.548+0000 7fc9ae000640 1 -- 172.21.15.47:0/1704031036 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc9a40f1330 con 0x7fc9b01038e0 2024-03-21T11:46:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.548+0000 7fc9b4c00640 1 -- 172.21.15.47:0/487860494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 msgr2=0x55fc6ff04c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.548+0000 7fc9b4c00640 1 --2- 172.21.15.47:0/487860494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff04c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.549+0000 7fc9b4c00640 1 -- 172.21.15.47:0/487860494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 msgr2=0x55fc6ff05e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.549+0000 7fc9b4c00640 1 --2- 172.21.15.47:0/487860494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 0x55fc6ff05e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.549+0000 7fc9b4c00640 1 -- 172.21.15.47:0/487860494 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc6fe25080 con 0x55fc6ff063b0 2024-03-21T11:46:46.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.549+0000 7fc9b6000640 1 --2- 172.21.15.47:0/1704031036 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9980a3020 0x7fc9980a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.549+0000 7fc9b4c00640 1 --2- 172.21.15.47:0/487860494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 0x55fc6ff0cc40 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc99c005d00 tx=0x7fc99c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bace7f5dab049ed server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:46.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.550+0000 7fc9a2600640 1 -- 172.21.15.47:0/487860494 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c004220 con 0x55fc6ff063b0 2024-03-21T11:46:46.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.550+0000 7fc9a2600640 1 -- 172.21.15.47:0/487860494 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc99c00be20 con 0x55fc6ff063b0 2024-03-21T11:46:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.551+0000 7fc9a2600640 1 -- 172.21.15.47:0/487860494 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c0117e0 con 0x55fc6ff063b0 2024-03-21T11:46:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.551+0000 7fc9badb3740 1 -- 172.21.15.47:0/487860494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 msgr2=0x55fc6ff0cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.551+0000 7fc9badb3740 1 --2- 172.21.15.47:0/487860494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 0x55fc6ff0cc40 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc99c005d00 tx=0x7fc99c0107a0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.551+0000 7fc9b6000640 1 --2- 172.21.15.47:0/1704031036 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9980a3020 0x7fc9980a54e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x55fc6ff058f0 tx=0x7fc9a8004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.552+0000 7fc9badb3740 1 -- 172.21.15.47:0/487860494 shutdown_connections 2024-03-21T11:46:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.552+0000 7fc9badb3740 1 --2- 172.21.15.47:0/487860494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 0x55fc6ff0cc40 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.552+0000 7fc9badb3740 1 --2- 172.21.15.47:0/487860494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 0x55fc6ff05e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.552+0000 7fc9badb3740 1 --2- 172.21.15.47:0/487860494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff04c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.552+0000 7fc9badb3740 1 -- 172.21.15.47:0/487860494 >> 172.21.15.47:0/487860494 conn(0x55fc6feeec00 msgr2=0x55fc6feef000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.553+0000 7fc9badb3740 1 -- 172.21.15.47:0/487860494 shutdown_connections 2024-03-21T11:46:46.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.553+0000 7fc9badb3740 1 -- 172.21.15.47:0/487860494 wait complete. 2024-03-21T11:46:46.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.553+0000 7fc9badb3740 1 -- start start 2024-03-21T11:46:46.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.554+0000 7fc9badb3740 1 Processor -- start 2024-03-21T11:46:46.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.554+0000 7fc9badb3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff04820 0x55fc6ff0eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.554+0000 7fc9badb3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff059f0 0x55fc6ff0f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.554+0000 7fc9badb3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff063b0 0x55fc6ff16090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.554+0000 7fc9b5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff04820 0x55fc6ff0eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.554+0000 7fc9b5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff04820 0x55fc6ff0eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44302/0 (socket says 172.21.15.47:44302) 2024-03-21T11:46:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.555+0000 7fc9b5600640 1 -- 172.21.15.47:0/3196061496 learned_addr learned my addr 172.21.15.47:0/3196061496 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.555+0000 7fc9badb3740 1 -- 172.21.15.47:0/3196061496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6ff0d410 con 0x55fc6ff063b0 2024-03-21T11:46:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.555+0000 7fc9badb3740 1 -- 172.21.15.47:0/3196061496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6fe140d0 con 0x55fc6ff04820 2024-03-21T11:46:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.555+0000 7fc9b4c00640 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff059f0 0x55fc6ff0f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.555+0000 7fc9badb3740 1 -- 172.21.15.47:0/3196061496 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6ff0d590 con 0x55fc6ff059f0 2024-03-21T11:46:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.555+0000 7fc9badb3740 10 client.0 _ll_register_callbacks cb 0x55fc6fe18e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:46.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.555+0000 7fc9b6000640 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff063b0 0x55fc6ff16090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.555+0000 7fc9b5600640 1 -- 172.21.15.47:0/3196061496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff059f0 msgr2=0x55fc6ff0f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.556+0000 7fc9b5600640 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff059f0 0x55fc6ff0f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.556+0000 7fc9b5600640 1 -- 172.21.15.47:0/3196061496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff063b0 msgr2=0x55fc6ff16090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.556+0000 7fc9b5600640 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff063b0 0x55fc6ff16090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.556+0000 7fc9b5600640 1 -- 172.21.15.47:0/3196061496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc6fb6c3f0 con 0x55fc6ff04820 2024-03-21T11:46:46.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.556+0000 7fc9b4c00640 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff059f0 0x55fc6ff0f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:46.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.556+0000 7fc9b6000640 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff063b0 0x55fc6ff16090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:46.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.557+0000 7fc9b5600640 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff04820 0x55fc6ff0eb10 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc9a402c4f0 tx=0x7fc9a40bd1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74981bf08a45e4dc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:46.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.557+0000 7fc996a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:46.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.557+0000 7fc996a00640 1 -- 172.21.15.47:0/3196061496 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a404b040 con 0x55fc6ff04820 2024-03-21T11:46:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.557+0000 7fc9badb3740 1 -- 172.21.15.47:0/3196061496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc99c00be20 con 0x55fc6ff04820 2024-03-21T11:46:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.557+0000 7fc996a00640 1 -- 172.21.15.47:0/3196061496 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc9a40c5bf0 con 0x55fc6ff04820 2024-03-21T11:46:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.557+0000 7fc996a00640 1 -- 172.21.15.47:0/3196061496 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9a40bd670 con 0x55fc6ff04820 2024-03-21T11:46:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.557+0000 7fc9badb3740 10 client.32954 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.557+0000 7fc9badb3740 1 -- 172.21.15.47:0/3196061496 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55fc6fc367d0 con 0x55fc6ff04820 2024-03-21T11:46:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.557+0000 7fc996a00640 1 -- 172.21.15.47:0/3196061496 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc99c00be20 con 0x55fc6ff04820 2024-03-21T11:46:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.558+0000 7fc996a00640 1 -- 172.21.15.47:0/3196061496 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc9a40bcac0 con 0x55fc6ff04820 2024-03-21T11:46:46.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.558+0000 7fc9badb3740 10 client.32954 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:46.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.558+0000 7fc9badb3740 10 client.32954 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:46:46.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.558+0000 7fc9badb3740 20 client.32954 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:46.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.558+0000 7fc9badb3740 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55fc6ff1a470 0x55fc6ff1c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.558+0000 7fc9badb3740 4 client.32954 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:46.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.558+0000 7fc9badb3740 1 -- 172.21.15.47:0/3196061496 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc6fe65800 con 0x55fc6ff1a470 2024-03-21T11:46:46.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.558+0000 7fc9b4c00640 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55fc6ff1a470 0x55fc6ff1c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.559+0000 7fc9b4c00640 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55fc6ff1a470 0x55fc6ff1c880 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7eccc2dc6ee8f38c server_cookie=8ca28d75d562a1e9 in_seq=0 out_seq=0 2024-03-21T11:46:46.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.559+0000 7fc996a00640 10 client.32954 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:46:46.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.567+0000 7fc996a00640 1 -- 172.21.15.47:0/3196061496 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fc6fe65800 con 0x55fc6ff1a470 2024-03-21T11:46:46.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.567+0000 7fc996a00640 10 client.32954 handle_command_reply: tid=0 2024-03-21T11:46:46.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.567+0000 7fc9badb3740 1 client.32954 shutdown 2024-03-21T11:46:46.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.568+0000 7fc9badb3740 1 -- 172.21.15.47:0/3196061496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff04820 msgr2=0x55fc6ff0eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.568+0000 7fc9badb3740 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff04820 0x55fc6ff0eb10 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc9a402c4f0 tx=0x7fc9a40bd1b0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.568+0000 7fc9badb3740 1 -- 172.21.15.47:0/3196061496 shutdown_connections 2024-03-21T11:46:46.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.568+0000 7fc9badb3740 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff063b0 0x55fc6ff16090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.568+0000 7fc9badb3740 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55fc6ff1a470 0x55fc6ff1c880 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.568+0000 7fc9badb3740 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff059f0 0x55fc6ff0f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.568+0000 7fc9badb3740 1 --2- 172.21.15.47:0/3196061496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff04820 0x55fc6ff0eb10 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.568+0000 7fc9badb3740 1 -- 172.21.15.47:0/3196061496 >> 172.21.15.47:0/3196061496 conn(0x55fc6feeec00 msgr2=0x55fc6feedeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.568+0000 7fc996a00640 0 client.32954 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:46:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.568+0000 7fc9badb3740 1 -- 172.21.15.47:0/3196061496 shutdown_connections 2024-03-21T11:46:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.568+0000 7fc9badb3740 1 -- 172.21.15.47:0/3196061496 wait complete. 2024-03-21T11:46:46.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.568+0000 7fc9badb3740 20 client.32954 trim_cache size 0 max 16384 2024-03-21T11:46:46.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.593+0000 7fc9badb3740 1 Processor -- start 2024-03-21T11:46:46.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.593+0000 7fc9badb3740 1 -- start start 2024-03-21T11:46:46.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.593+0000 7fc9badb3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff0efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.593+0000 7fc9badb3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 0x55fc6ff16050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.593+0000 7fc9badb3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 0x55fc6fef2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.593+0000 7fc9badb3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b01a7e90 con 0x55fc6ff059f0 2024-03-21T11:46:46.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.593+0000 7fc9badb3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b01a9810 con 0x55fc6ff063b0 2024-03-21T11:46:46.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.593+0000 7fc9badb3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b01a84f0 con 0x55fc6ff04820 2024-03-21T11:46:46.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.593+0000 7fc9b5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff0efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.593+0000 7fc9b5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff0efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41364/0 (socket says 172.21.15.47:41364) 2024-03-21T11:46:46.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.593+0000 7fc9b5600640 1 -- 172.21.15.47:0/2471716422 learned_addr learned my addr 172.21.15.47:0/2471716422 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:46.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.593+0000 7fc9b6000640 1 --2- 172.21.15.47:0/2471716422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 0x55fc6fef2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.594+0000 7fc9b4c00640 1 --2- 172.21.15.47:0/2471716422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 0x55fc6ff16050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.594+0000 7fc9b5600640 1 -- 172.21.15.47:0/2471716422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 msgr2=0x55fc6fef2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.594+0000 7fc9b5600640 1 --2- 172.21.15.47:0/2471716422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 0x55fc6fef2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.594+0000 7fc9b5600640 1 -- 172.21.15.47:0/2471716422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 msgr2=0x55fc6ff16050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.594+0000 7fc9b5600640 1 --2- 172.21.15.47:0/2471716422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 0x55fc6ff16050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.594+0000 7fc9b5600640 1 -- 172.21.15.47:0/2471716422 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc6ff0fc90 con 0x55fc6ff04820 2024-03-21T11:46:46.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.594+0000 7fc9b4c00640 1 --2- 172.21.15.47:0/2471716422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 0x55fc6ff16050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:46.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.594+0000 7fc9b5600640 1 --2- 172.21.15.47:0/2471716422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff0efb0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40ba040 tx=0x7fc9a4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73dc485c36b5ff54 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:46.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.594+0000 7fc9a1c00640 1 -- 172.21.15.47:0/2471716422 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a404b070 con 0x55fc6ff04820 2024-03-21T11:46:46.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.595+0000 7fc9a1c00640 1 -- 172.21.15.47:0/2471716422 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9a40a80b0 con 0x55fc6ff04820 2024-03-21T11:46:46.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.595+0000 7fc9a1c00640 1 -- 172.21.15.47:0/2471716422 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40036f0 con 0x55fc6ff04820 2024-03-21T11:46:46.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.595+0000 7fc9badb3740 1 -- 172.21.15.47:0/2471716422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 msgr2=0x55fc6ff0efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.595+0000 7fc9badb3740 1 --2- 172.21.15.47:0/2471716422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff0efb0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40ba040 tx=0x7fc9a4005fd0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.595+0000 7fc9badb3740 1 -- 172.21.15.47:0/2471716422 shutdown_connections 2024-03-21T11:46:46.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.595+0000 7fc9badb3740 1 --2- 172.21.15.47:0/2471716422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 0x55fc6fef2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.595+0000 7fc9badb3740 1 --2- 172.21.15.47:0/2471716422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 0x55fc6ff16050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.595+0000 7fc9badb3740 1 --2- 172.21.15.47:0/2471716422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff0efb0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.595+0000 7fc9badb3740 1 -- 172.21.15.47:0/2471716422 >> 172.21.15.47:0/2471716422 conn(0x55fc6feeec00 msgr2=0x55fc6fef1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:46.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.596+0000 7fc9badb3740 1 -- 172.21.15.47:0/2471716422 shutdown_connections 2024-03-21T11:46:46.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.596+0000 7fc9badb3740 1 -- 172.21.15.47:0/2471716422 wait complete. 2024-03-21T11:46:46.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.596+0000 7fc9badb3740 1 -- start start 2024-03-21T11:46:46.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.597+0000 7fc9badb3740 1 Processor -- start 2024-03-21T11:46:46.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.598+0000 7fc9badb3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff119a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.598+0000 7fc9badb3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 0x55fc6ff11ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.598+0000 7fc9badb3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 0x55fc6ff14e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.598+0000 7fc9b5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff119a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.598+0000 7fc9b5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff119a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41380/0 (socket says 172.21.15.47:41380) 2024-03-21T11:46:46.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.599+0000 7fc9b5600640 1 -- 172.21.15.47:0/184651133 learned_addr learned my addr 172.21.15.47:0/184651133 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:46.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.599+0000 7fc9badb3740 1 -- 172.21.15.47:0/184651133 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b01a91b0 con 0x55fc6ff059f0 2024-03-21T11:46:46.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.599+0000 7fc9b4c00640 1 --2- 172.21.15.47:0/184651133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 0x55fc6ff11ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.599+0000 7fc9badb3740 1 -- 172.21.15.47:0/184651133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b01a8b50 con 0x55fc6ff063b0 2024-03-21T11:46:46.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.599+0000 7fc9badb3740 1 -- 172.21.15.47:0/184651133 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b0073390 con 0x55fc6ff04820 2024-03-21T11:46:46.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.599+0000 7fc9badb3740 10 client.0 _ll_register_callbacks cb 0x7fc9a40b92a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:46.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.599+0000 7fc9b5600640 1 -- 172.21.15.47:0/184651133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 msgr2=0x55fc6ff14e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.599+0000 7fc9b5600640 1 --2- 172.21.15.47:0/184651133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 0x55fc6ff14e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.599+0000 7fc9b5600640 1 -- 172.21.15.47:0/184651133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 msgr2=0x55fc6ff11ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:46.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.599+0000 7fc9b5600640 1 --2- 172.21.15.47:0/184651133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 0x55fc6ff11ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.599+0000 7fc9b5600640 1 -- 172.21.15.47:0/184651133 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc6ff15480 con 0x55fc6ff04820 2024-03-21T11:46:46.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.600+0000 7fc9b5600640 1 --2- 172.21.15.47:0/184651133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff119a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fc9a402c7c0 tx=0x7fc9a40bd2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc8a81f5e08e484a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:46.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.600+0000 7fc996a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.600+0000 7fc996a00640 1 -- 172.21.15.47:0/184651133 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a404b040 con 0x55fc6ff04820 2024-03-21T11:46:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.600+0000 7fc996a00640 1 -- 172.21.15.47:0/184651133 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc9a40b93c0 con 0x55fc6ff04820 2024-03-21T11:46:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.604+0000 7fc9badb3740 1 -- 172.21.15.47:0/184651133 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9a40a80b0 con 0x55fc6ff04820 2024-03-21T11:46:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.604+0000 7fc996a00640 1 -- 172.21.15.47:0/184651133 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9a40c17c0 con 0x55fc6ff04820 2024-03-21T11:46:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.605+0000 7fc9badb3740 10 client.32935 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.605+0000 7fc9badb3740 1 -- 172.21.15.47:0/184651133 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55fc6ff15480 con 0x55fc6ff04820 2024-03-21T11:46:46.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.605+0000 7fc996a00640 1 -- 172.21.15.47:0/184651133 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9a40a80b0 con 0x55fc6ff04820 2024-03-21T11:46:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.605+0000 7fc996a00640 1 -- 172.21.15.47:0/184651133 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc9a40dd4e0 con 0x55fc6ff04820 2024-03-21T11:46:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.605+0000 7fc9badb3740 10 client.32935 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.605+0000 7fc9badb3740 10 client.32935 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:46:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.605+0000 7fc9badb3740 20 client.32935 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.605+0000 7fc9badb3740 1 --2- 172.21.15.47:0/184651133 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55fc6ff76310 0x55fc6ff78730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.605+0000 7fc9badb3740 4 client.32935 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:46:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.605+0000 7fc9badb3740 1 -- 172.21.15.47:0/184651133 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fc6fe5fb50 con 0x55fc6ff76310 2024-03-21T11:46:46.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.705+0000 7fc9b4c00640 1 --2- 172.21.15.47:0/184651133 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55fc6ff76310 0x55fc6ff78730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:46.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.706+0000 7fc9b4c00640 1 --2- 172.21.15.47:0/184651133 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55fc6ff76310 0x55fc6ff78730 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ee7b92942bd8cea server_cookie=c8f5c6625b92dcf3 in_seq=0 out_seq=0 2024-03-21T11:46:46.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.706+0000 7fc996a00640 10 client.32935 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:46:46.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.717+0000 7fc996a00640 1 -- 172.21.15.47:0/184651133 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fc6fe5fb50 con 0x55fc6ff76310 2024-03-21T11:46:46.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.717+0000 7fc996a00640 10 client.32935 handle_command_reply: tid=0 2024-03-21T11:46:46.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.717+0000 7fc9badb3740 1 client.32935 shutdown 2024-03-21T11:46:46.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.719+0000 7fc9badb3740 1 -- 172.21.15.47:0/184651133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 msgr2=0x55fc6ff119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.719+0000 7fc9badb3740 1 --2- 172.21.15.47:0/184651133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff119a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fc9a402c7c0 tx=0x7fc9a40bd2e0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.720+0000 7fc9badb3740 1 -- 172.21.15.47:0/184651133 shutdown_connections 2024-03-21T11:46:46.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.720+0000 7fc9badb3740 1 --2- 172.21.15.47:0/184651133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc6ff063b0 0x55fc6ff14e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.720+0000 7fc9badb3740 1 --2- 172.21.15.47:0/184651133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc6ff059f0 0x55fc6ff11ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.720+0000 7fc9badb3740 1 --2- 172.21.15.47:0/184651133 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55fc6ff76310 0x55fc6ff78730 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.720+0000 7fc9badb3740 1 --2- 172.21.15.47:0/184651133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fc6ff04820 0x55fc6ff119a0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.720+0000 7fc9badb3740 1 -- 172.21.15.47:0/184651133 >> 172.21.15.47:0/184651133 conn(0x55fc6feeec00 msgr2=0x55fc6ff0ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:46.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.720+0000 7fc996a00640 0 client.32935 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:46:46.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.720+0000 7fc9badb3740 1 -- 172.21.15.47:0/184651133 shutdown_connections 2024-03-21T11:46:46.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.721+0000 7fc9badb3740 1 -- 172.21.15.47:0/184651133 wait complete. 2024-03-21T11:46:46.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.721+0000 7fc9badb3740 20 client.32935 trim_cache size 0 max 16384 2024-03-21T11:46:46.723 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:46:46.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.726+0000 7fc9b8000640 1 -- 172.21.15.47:0/1704031036 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9980a3020 msgr2=0x7fc9980a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.726+0000 7fc9b8000640 1 --2- 172.21.15.47:0/1704031036 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9980a3020 0x7fc9980a54e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x55fc6ff058f0 tx=0x7fc9a8004ab0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.726+0000 7fc9b8000640 1 -- 172.21.15.47:0/1704031036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b01038e0 msgr2=0x7fc9b019e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:46.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.726+0000 7fc9b8000640 1 --2- 172.21.15.47:0/1704031036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b01038e0 0x7fc9b019e020 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4000c00 tx=0x7fc9a4005d50 comp rx=0 tx=0).stop 2024-03-21T11:46:46.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.727+0000 7fc9b8000640 1 -- 172.21.15.47:0/1704031036 shutdown_connections 2024-03-21T11:46:46.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.727+0000 7fc9b8000640 1 --2- 172.21.15.47:0/1704031036 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc9980a3020 0x7fc9980a54e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.727+0000 7fc9b8000640 1 --2- 172.21.15.47:0/1704031036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc9b0105490 0x7fc9b01a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.727+0000 7fc9b8000640 1 --2- 172.21.15.47:0/1704031036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc9b0104ad0 0x7fc9b019e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.727+0000 7fc9b8000640 1 --2- 172.21.15.47:0/1704031036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b01038e0 0x7fc9b019e020 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:46.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.727+0000 7fc9b8000640 1 -- 172.21.15.47:0/1704031036 >> 172.21.15.47:0/1704031036 conn(0x7fc9b00fee50 msgr2=0x7fc9b0077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:46.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.729+0000 7fc9b8000640 1 -- 172.21.15.47:0/1704031036 shutdown_connections 2024-03-21T11:46:46.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:46.729+0000 7fc9b8000640 1 -- 172.21.15.47:0/1704031036 wait complete. 2024-03-21T11:46:46.763 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:46:46.763 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:46:46.989 INFO:tasks.workunit.client.0.smithi047.stdout: 10 240048 21.75 MB/sec execute 450 sec latency 192.847 ms 2024-03-21T11:46:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.055+0000 7efcfb200640 1 -- 172.21.15.47:0/3406916083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcfc102790 msgr2=0x7efcfc102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.055+0000 7efcfb200640 1 --2- 172.21.15.47:0/3406916083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcfc102790 0x7efcfc102c10 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7efce8009860 tx=0x7efce802bb70 comp rx=0 tx=0).stop 2024-03-21T11:46:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.055+0000 7efcfb200640 1 -- 172.21.15.47:0/3406916083 shutdown_connections 2024-03-21T11:46:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.055+0000 7efcfb200640 1 --2- 172.21.15.47:0/3406916083 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcfc103150 0x7efcfc1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.055+0000 7efcfb200640 1 --2- 172.21.15.47:0/3406916083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcfc102790 0x7efcfc102c10 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.055+0000 7efcfb200640 1 --2- 172.21.15.47:0/3406916083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcfc1015a0 0x7efcfc1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.055+0000 7efcfb200640 1 -- 172.21.15.47:0/3406916083 >> 172.21.15.47:0/3406916083 conn(0x7efcfc0fcb10 msgr2=0x7efcfc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:47.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.055+0000 7efcfb200640 1 -- 172.21.15.47:0/3406916083 shutdown_connections 2024-03-21T11:46:47.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.055+0000 7efcfb200640 1 -- 172.21.15.47:0/3406916083 wait complete. 2024-03-21T11:46:47.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.056+0000 7efcfb200640 1 Processor -- start 2024-03-21T11:46:47.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.056+0000 7efcfb200640 1 -- start start 2024-03-21T11:46:47.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.057+0000 7efcfb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcfc1015a0 0x7efcfc199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.057+0000 7efcfb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcfc102790 0x7efcfc199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.057+0000 7efcfb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcfc103150 0x7efcfc1a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.057+0000 7efcfb200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc10d230 con 0x7efcfc102790 2024-03-21T11:46:47.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.057+0000 7efcfb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc10d0b0 con 0x7efcfc103150 2024-03-21T11:46:47.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.057+0000 7efcfb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc10d3b0 con 0x7efcfc1015a0 2024-03-21T11:46:47.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.057+0000 7efcf9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcfc1015a0 0x7efcfc199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.057+0000 7efcf9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcfc1015a0 0x7efcfc199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41402/0 (socket says 172.21.15.47:41402) 2024-03-21T11:46:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.057+0000 7efcf9e00640 1 -- 172.21.15.47:0/3262868828 learned_addr learned my addr 172.21.15.47:0/3262868828 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:47.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.057+0000 7efcfa800640 1 --2- 172.21.15.47:0/3262868828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcfc103150 0x7efcfc1a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.058+0000 7efcf9400640 1 --2- 172.21.15.47:0/3262868828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcfc102790 0x7efcfc199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.058+0000 7efcf9e00640 1 -- 172.21.15.47:0/3262868828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcfc103150 msgr2=0x7efcfc1a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.058+0000 7efcf9e00640 1 --2- 172.21.15.47:0/3262868828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcfc103150 0x7efcfc1a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.058+0000 7efcf9e00640 1 -- 172.21.15.47:0/3262868828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcfc102790 msgr2=0x7efcfc199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.058+0000 7efcf9e00640 1 --2- 172.21.15.47:0/3262868828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcfc102790 0x7efcfc199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.058+0000 7efcf9e00640 1 -- 172.21.15.47:0/3262868828 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc1a1680 con 0x7efcfc1015a0 2024-03-21T11:46:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.058+0000 7efcf9e00640 1 --2- 172.21.15.47:0/3262868828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcfc1015a0 0x7efcfc199ab0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7efcec00e9f0 tx=0x7efcec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.059+0000 7efcf2a00640 1 -- 172.21.15.47:0/3262868828 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec00ccc0 con 0x7efcfc1015a0 2024-03-21T11:46:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.059+0000 7efcf2a00640 1 -- 172.21.15.47:0/3262868828 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcec0035f0 con 0x7efcfc1015a0 2024-03-21T11:46:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.059+0000 7efcf2a00640 1 -- 172.21.15.47:0/3262868828 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec00ce60 con 0x7efcfc1015a0 2024-03-21T11:46:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.059+0000 7efcfb200640 1 -- 172.21.15.47:0/3262868828 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc1a1970 con 0x7efcfc1015a0 2024-03-21T11:46:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.059+0000 7efcfb200640 1 -- 172.21.15.47:0/3262868828 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc1a1e40 con 0x7efcfc1015a0 2024-03-21T11:46:47.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:46 smithi047 ceph-mon[31351]: pgmap v1104: 129 pgs: 129 active+clean; 704 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.5 MiB/s wr, 512 op/s 2024-03-21T11:46:47.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.064+0000 7efd039fd740 1 Processor -- start 2024-03-21T11:46:47.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.064+0000 7efd039fd740 1 -- start start 2024-03-21T11:46:47.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.064+0000 7efd039fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3a9f0 0x558c87e3ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.064+0000 7efd039fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3bbc0 0x558c87e3c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.064+0000 7efd039fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3c580 0x558c87e42e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.064+0000 7efd039fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c87d9f2c0 con 0x558c87e3c580 2024-03-21T11:46:47.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.064+0000 7efd039fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c87d47c90 con 0x558c87e3bbc0 2024-03-21T11:46:47.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.064+0000 7efd039fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c87d9b360 con 0x558c87e3a9f0 2024-03-21T11:46:47.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.065+0000 7efcf2a00640 1 -- 172.21.15.47:0/3262868828 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7efcec0037a0 con 0x7efcfc1015a0 2024-03-21T11:46:47.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.065+0000 7efcf9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3bbc0 0x558c87e3c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.065+0000 7efcf9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3bbc0 0x558c87e3c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44344/0 (socket says 172.21.15.47:44344) 2024-03-21T11:46:47.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.065+0000 7efcf9e00640 1 -- 172.21.15.47:0/3609964750 learned_addr learned my addr 172.21.15.47:0/3609964750 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:47.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.066+0000 7efcfa800640 1 --2- 172.21.15.47:0/3609964750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3a9f0 0x558c87e3ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.066+0000 7efcf9e00640 1 -- 172.21.15.47:0/3609964750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3a9f0 msgr2=0x558c87e3ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.066+0000 7efcf9e00640 1 --2- 172.21.15.47:0/3609964750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3a9f0 0x558c87e3ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.066+0000 7efcf9e00640 1 -- 172.21.15.47:0/3609964750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3c580 msgr2=0x558c87e42e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.066+0000 7efcf9e00640 1 --2- 172.21.15.47:0/3609964750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3c580 0x558c87e42e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.066+0000 7efcf9e00640 1 -- 172.21.15.47:0/3609964750 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c87d6c080 con 0x558c87e3bbc0 2024-03-21T11:46:47.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.066+0000 7efcfa800640 1 --2- 172.21.15.47:0/3609964750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3a9f0 0x558c87e3ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:47.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.067+0000 7efcf9e00640 1 --2- 172.21.15.47:0/3609964750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3bbc0 0x558c87e3c040 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7efcec0d5a90 tx=0x7efcec0d5b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff2ed80d3c1d344b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:47.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.067+0000 7efcf2a00640 1 --2- 172.21.15.47:0/3262868828 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efce00a3070 0x7efce00a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.067+0000 7efcf2a00640 1 -- 172.21.15.47:0/3262868828 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efcec0cf9b0 con 0x7efcfc1015a0 2024-03-21T11:46:47.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.068+0000 7efcfa800640 1 --2- 172.21.15.47:0/3262868828 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efce00a3070 0x7efce00a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.068+0000 7efce7400640 1 -- 172.21.15.47:0/3609964750 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec0de070 con 0x558c87e3bbc0 2024-03-21T11:46:47.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.068+0000 7efce7400640 1 -- 172.21.15.47:0/3609964750 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcec09c3f0 con 0x558c87e3bbc0 2024-03-21T11:46:47.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.068+0000 7efce7400640 1 -- 172.21.15.47:0/3609964750 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec09b650 con 0x558c87e3bbc0 2024-03-21T11:46:47.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.069+0000 7efd039fd740 1 -- 172.21.15.47:0/3609964750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3bbc0 msgr2=0x558c87e3c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.069+0000 7efd039fd740 1 --2- 172.21.15.47:0/3609964750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3bbc0 0x558c87e3c040 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7efcec0d5a90 tx=0x7efcec0d5b80 comp rx=0 tx=0).stop 2024-03-21T11:46:47.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.069+0000 7efcfa800640 1 --2- 172.21.15.47:0/3262868828 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efce00a3070 0x7efce00a5530 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x558c87e3bac0 tx=0x7efcf4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:47.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.069+0000 7efd039fd740 1 -- 172.21.15.47:0/3609964750 shutdown_connections 2024-03-21T11:46:47.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.069+0000 7efd039fd740 1 --2- 172.21.15.47:0/3609964750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3c580 0x558c87e42e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.069+0000 7efd039fd740 1 --2- 172.21.15.47:0/3609964750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3bbc0 0x558c87e3c040 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.069+0000 7efd039fd740 1 --2- 172.21.15.47:0/3609964750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3a9f0 0x558c87e3ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.069+0000 7efd039fd740 1 -- 172.21.15.47:0/3609964750 >> 172.21.15.47:0/3609964750 conn(0x558c87e35de0 msgr2=0x558c87e361e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:47.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.070+0000 7efd039fd740 1 -- 172.21.15.47:0/3609964750 shutdown_connections 2024-03-21T11:46:47.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.070+0000 7efd039fd740 1 -- 172.21.15.47:0/3609964750 wait complete. 2024-03-21T11:46:47.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.071+0000 7efd039fd740 1 -- start start 2024-03-21T11:46:47.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.071+0000 7efd039fd740 1 Processor -- start 2024-03-21T11:46:47.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.072+0000 7efd039fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3a9f0 0x558c87e4d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.072+0000 7efd039fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3bbc0 0x558c87e4d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.072+0000 7efd039fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3c580 0x558c87e549b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.072+0000 7efd039fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c87e4bc50 con 0x558c87e3a9f0 2024-03-21T11:46:47.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.072+0000 7efd039fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c87d5b0d0 con 0x558c87e3bbc0 2024-03-21T11:46:47.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.072+0000 7efd039fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c87e4bdd0 con 0x558c87e3c580 2024-03-21T11:46:47.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.072+0000 7efd039fd740 10 client.0 _ll_register_callbacks cb 0x558c87d5fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:47.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.072+0000 7efcfa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3c580 0x558c87e549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.073+0000 7efcfa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3c580 0x558c87e549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41414/0 (socket says 172.21.15.47:41414) 2024-03-21T11:46:47.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.073+0000 7efcfa800640 1 -- 172.21.15.47:0/4228822032 learned_addr learned my addr 172.21.15.47:0/4228822032 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:47.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.073+0000 7efcf9400640 1 --2- 172.21.15.47:0/4228822032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3bbc0 0x558c87e4d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.073+0000 7efcf9e00640 1 --2- 172.21.15.47:0/4228822032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3a9f0 0x558c87e4d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.073+0000 7efcfa800640 1 -- 172.21.15.47:0/4228822032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3bbc0 msgr2=0x558c87e4d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.073+0000 7efcfa800640 1 --2- 172.21.15.47:0/4228822032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3bbc0 0x558c87e4d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.073+0000 7efcfa800640 1 -- 172.21.15.47:0/4228822032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3a9f0 msgr2=0x558c87e4d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.073+0000 7efcfa800640 1 --2- 172.21.15.47:0/4228822032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3a9f0 0x558c87e4d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.073+0000 7efcfa800640 1 -- 172.21.15.47:0/4228822032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c87ab33f0 con 0x558c87e3c580 2024-03-21T11:46:47.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.073+0000 7efcfa800640 1 --2- 172.21.15.47:0/4228822032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3c580 0x558c87e549b0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7efcf4003c40 tx=0x7efcf4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73bc66045cab89c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:47.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.073+0000 7efcdbe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:47.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.074+0000 7efcdbe00640 1 -- 172.21.15.47:0/4228822032 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf400fe60 con 0x558c87e3c580 2024-03-21T11:46:47.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.074+0000 7efcdbe00640 1 -- 172.21.15.47:0/4228822032 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efcf40045c0 con 0x558c87e3c580 2024-03-21T11:46:47.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.074+0000 7efd039fd740 1 -- 172.21.15.47:0/4228822032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcec09c3f0 con 0x558c87e3c580 2024-03-21T11:46:47.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.074+0000 7efcdbe00640 1 -- 172.21.15.47:0/4228822032 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcf40047c0 con 0x558c87e3c580 2024-03-21T11:46:47.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.075+0000 7efd039fd740 10 client.32947 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:47.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.075+0000 7efd039fd740 1 -- 172.21.15.47:0/4228822032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558c87b7d7d0 con 0x558c87e3c580 2024-03-21T11:46:47.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.075+0000 7efcdbe00640 1 -- 172.21.15.47:0/4228822032 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcec09c3f0 con 0x558c87e3c580 2024-03-21T11:46:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.076+0000 7efcdbe00640 1 -- 172.21.15.47:0/4228822032 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efcf4026070 con 0x558c87e3c580 2024-03-21T11:46:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.076+0000 7efd039fd740 10 client.32947 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.076+0000 7efd039fd740 10 client.32947 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:46:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.076+0000 7efd039fd740 20 client.32947 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.076+0000 7efd039fd740 1 --2- 172.21.15.47:0/4228822032 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x558c87e58f50 0x558c87e5b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.076+0000 7efd039fd740 4 client.32947 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:47.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.076+0000 7efd039fd740 1 -- 172.21.15.47:0/4228822032 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c87dac800 con 0x558c87e58f50 2024-03-21T11:46:47.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.077+0000 7efcf9e00640 1 --2- 172.21.15.47:0/4228822032 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x558c87e58f50 0x558c87e5b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.077+0000 7efcf9e00640 1 --2- 172.21.15.47:0/4228822032 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x558c87e58f50 0x558c87e5b3d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e483bba37721420 server_cookie=218d1d2f68ba224c in_seq=0 out_seq=0 2024-03-21T11:46:47.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.077+0000 7efcdbe00640 10 client.32947 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:46:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.078+0000 7efcdbe00640 1 -- 172.21.15.47:0/4228822032 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558c87dac800 con 0x558c87e58f50 2024-03-21T11:46:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.078+0000 7efcdbe00640 10 client.32947 handle_command_reply: tid=0 2024-03-21T11:46:47.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.078+0000 7efd039fd740 1 client.32947 shutdown 2024-03-21T11:46:47.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.079+0000 7efd039fd740 1 -- 172.21.15.47:0/4228822032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3c580 msgr2=0x558c87e549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.079+0000 7efd039fd740 1 --2- 172.21.15.47:0/4228822032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3c580 0x558c87e549b0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7efcf4003c40 tx=0x7efcf4004ab0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.079+0000 7efd039fd740 1 -- 172.21.15.47:0/4228822032 shutdown_connections 2024-03-21T11:46:47.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.079+0000 7efd039fd740 1 --2- 172.21.15.47:0/4228822032 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x558c87e58f50 0x558c87e5b3d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.079+0000 7efd039fd740 1 --2- 172.21.15.47:0/4228822032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3c580 0x558c87e549b0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.079+0000 7efd039fd740 1 --2- 172.21.15.47:0/4228822032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3bbc0 0x558c87e4d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.079+0000 7efd039fd740 1 --2- 172.21.15.47:0/4228822032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3a9f0 0x558c87e4d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.079+0000 7efcdbe00640 0 client.32947 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:46:47.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.079+0000 7efd039fd740 1 -- 172.21.15.47:0/4228822032 >> 172.21.15.47:0/4228822032 conn(0x558c87e35de0 msgr2=0x558c87e35160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:47.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.079+0000 7efd039fd740 1 -- 172.21.15.47:0/4228822032 shutdown_connections 2024-03-21T11:46:47.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.079+0000 7efd039fd740 1 -- 172.21.15.47:0/4228822032 wait complete. 2024-03-21T11:46:47.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.079+0000 7efd039fd740 20 client.32947 trim_cache size 0 max 16384 2024-03-21T11:46:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.107+0000 7efd039fd740 1 Processor -- start 2024-03-21T11:46:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.107+0000 7efd039fd740 1 -- start start 2024-03-21T11:46:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.107+0000 7efd039fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3a9f0 0x558c87e30ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.107+0000 7efd039fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3bbc0 0x558c87e54970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.107+0000 7efd039fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3c580 0x558c87e3ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.107+0000 7efd039fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc1a7d40 con 0x558c87e3c580 2024-03-21T11:46:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.107+0000 7efd039fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc1a96c0 con 0x558c87e3a9f0 2024-03-21T11:46:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.107+0000 7efd039fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc1a83a0 con 0x558c87e3bbc0 2024-03-21T11:46:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.108+0000 7efcf9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3a9f0 0x558c87e30ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.108+0000 7efcf9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3bbc0 0x558c87e54970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.108+0000 7efcf9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3bbc0 0x558c87e54970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41416/0 (socket says 172.21.15.47:41416) 2024-03-21T11:46:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.108+0000 7efcf9400640 1 -- 172.21.15.47:0/1647088187 learned_addr learned my addr 172.21.15.47:0/1647088187 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.108+0000 7efcf9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3a9f0 0x558c87e30ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44366/0 (socket says 172.21.15.47:44366) 2024-03-21T11:46:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.108+0000 7efcf9e00640 1 -- 172.21.15.47:0/1647088187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3bbc0 msgr2=0x558c87e54970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.108+0000 7efcf9e00640 1 --2- 172.21.15.47:0/1647088187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3bbc0 0x558c87e54970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.108+0000 7efcf9e00640 1 -- 172.21.15.47:0/1647088187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3c580 msgr2=0x558c87e3ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.108+0000 7efcf9e00640 1 --2- 172.21.15.47:0/1647088187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3c580 0x558c87e3ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.108+0000 7efcf9e00640 1 -- 172.21.15.47:0/1647088187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c87e36850 con 0x558c87e3a9f0 2024-03-21T11:46:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.109+0000 7efcf9e00640 1 --2- 172.21.15.47:0/1647088187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3a9f0 0x558c87e30ff0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7efcec0d5a60 tx=0x7efcec003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc4f5447a712553a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.109+0000 7efce6a00640 1 -- 172.21.15.47:0/1647088187 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec0de070 con 0x558c87e3a9f0 2024-03-21T11:46:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.109+0000 7efce6a00640 1 -- 172.21.15.47:0/1647088187 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcec09c3f0 con 0x558c87e3a9f0 2024-03-21T11:46:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.109+0000 7efce6a00640 1 -- 172.21.15.47:0/1647088187 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec0cd140 con 0x558c87e3a9f0 2024-03-21T11:46:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.110+0000 7efd039fd740 1 -- 172.21.15.47:0/1647088187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3a9f0 msgr2=0x558c87e30ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.110+0000 7efd039fd740 1 --2- 172.21.15.47:0/1647088187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3a9f0 0x558c87e30ff0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7efcec0d5a60 tx=0x7efcec003f10 comp rx=0 tx=0).stop 2024-03-21T11:46:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.110+0000 7efd039fd740 1 -- 172.21.15.47:0/1647088187 shutdown_connections 2024-03-21T11:46:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.110+0000 7efd039fd740 1 --2- 172.21.15.47:0/1647088187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3c580 0x558c87e3ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.110+0000 7efd039fd740 1 --2- 172.21.15.47:0/1647088187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3bbc0 0x558c87e54970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.110+0000 7efd039fd740 1 --2- 172.21.15.47:0/1647088187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3a9f0 0x558c87e30ff0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.110+0000 7efd039fd740 1 -- 172.21.15.47:0/1647088187 >> 172.21.15.47:0/1647088187 conn(0x558c87e35de0 msgr2=0x558c87e398d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.110+0000 7efd039fd740 1 -- 172.21.15.47:0/1647088187 shutdown_connections 2024-03-21T11:46:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.110+0000 7efd039fd740 1 -- 172.21.15.47:0/1647088187 wait complete. 2024-03-21T11:46:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.111+0000 7efd039fd740 1 -- start start 2024-03-21T11:46:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.111+0000 7efd039fd740 1 Processor -- start 2024-03-21T11:46:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.112+0000 7efd039fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3a9f0 0x558c87e54430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.112+0000 7efd039fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3bbc0 0x558c87e54970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.112+0000 7efd039fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3c580 0x558c87e51010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.112+0000 7efd039fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc1a9060 con 0x558c87e3c580 2024-03-21T11:46:47.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.112+0000 7efd039fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc1a8a00 con 0x558c87e3a9f0 2024-03-21T11:46:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.112+0000 7efd039fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc1a5fa0 con 0x558c87e3bbc0 2024-03-21T11:46:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.112+0000 7efd039fd740 10 client.0 _ll_register_callbacks cb 0x7efcf4009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.112+0000 7efcf9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3a9f0 0x558c87e54430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.112+0000 7efcf9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3bbc0 0x558c87e54970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.112+0000 7efcf9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3a9f0 0x558c87e54430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44380/0 (socket says 172.21.15.47:44380) 2024-03-21T11:46:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.112+0000 7efcf9e00640 1 -- 172.21.15.47:0/4221104500 learned_addr learned my addr 172.21.15.47:0/4221104500 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.112+0000 7efcfa800640 1 --2- 172.21.15.47:0/4221104500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3c580 0x558c87e51010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.113+0000 7efcf9e00640 1 -- 172.21.15.47:0/4221104500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3bbc0 msgr2=0x558c87e54970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.113+0000 7efcf9e00640 1 --2- 172.21.15.47:0/4221104500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3bbc0 0x558c87e54970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.113+0000 7efcf9e00640 1 -- 172.21.15.47:0/4221104500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3c580 msgr2=0x558c87e51010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.113+0000 7efcf9e00640 1 --2- 172.21.15.47:0/4221104500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3c580 0x558c87e51010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.113+0000 7efcf9e00640 1 -- 172.21.15.47:0/4221104500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c87e51840 con 0x558c87e3a9f0 2024-03-21T11:46:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.113+0000 7efcfa800640 1 --2- 172.21.15.47:0/4221104500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3c580 0x558c87e51010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.113+0000 7efcf9e00640 1 --2- 172.21.15.47:0/4221104500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3a9f0 0x558c87e54430 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7efcec0986c0 tx=0x7efcec09fe40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=704a4c1ffa701cad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.113+0000 7efcdbe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.113+0000 7efcdbe00640 1 -- 172.21.15.47:0/4221104500 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec0b9060 con 0x558c87e3a9f0 2024-03-21T11:46:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.114+0000 7efcdbe00640 1 -- 172.21.15.47:0/4221104500 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7efcec0cd140 con 0x558c87e3a9f0 2024-03-21T11:46:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.114+0000 7efd039fd740 1 -- 172.21.15.47:0/4221104500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcec09c3f0 con 0x558c87e3a9f0 2024-03-21T11:46:47.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.114+0000 7efcdbe00640 1 -- 172.21.15.47:0/4221104500 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcec0de070 con 0x558c87e3a9f0 2024-03-21T11:46:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.114+0000 7efd039fd740 10 client.32975 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.114+0000 7efd039fd740 1 -- 172.21.15.47:0/4221104500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558c87eb56c0 con 0x558c87e3a9f0 2024-03-21T11:46:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.114+0000 7efcdbe00640 1 -- 172.21.15.47:0/4221104500 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcec09c3f0 con 0x558c87e3a9f0 2024-03-21T11:46:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.114+0000 7efcdbe00640 1 -- 172.21.15.47:0/4221104500 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efcf4011d20 con 0x558c87e3a9f0 2024-03-21T11:46:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.115+0000 7efd039fd740 10 client.32975 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.115+0000 7efd039fd740 10 client.32975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:46:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.115+0000 7efd039fd740 20 client.32975 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.115+0000 7efd039fd740 1 --2- 172.21.15.47:0/4221104500 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x558c87eb5950 0x558c87eb7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.115+0000 7efd039fd740 4 client.32975 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:46:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.115+0000 7efd039fd740 1 -- 172.21.15.47:0/4221104500 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558c87da6b50 con 0x558c87eb5950 2024-03-21T11:46:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.116+0000 7efcf9400640 1 --2- 172.21.15.47:0/4221104500 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x558c87eb5950 0x558c87eb7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.117+0000 7efcdbe00640 10 client.32975 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:46:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.117+0000 7efcf9400640 1 --2- 172.21.15.47:0/4221104500 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x558c87eb5950 0x558c87eb7d40 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9cf4313153c4f959 server_cookie=d19621f9e8cc27bd in_seq=0 out_seq=0 2024-03-21T11:46:47.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.117+0000 7efcdbe00640 1 -- 172.21.15.47:0/4221104500 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558c87da6b50 con 0x558c87eb5950 2024-03-21T11:46:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.117+0000 7efcdbe00640 10 client.32975 handle_command_reply: tid=0 2024-03-21T11:46:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.117+0000 7efd039fd740 1 client.32975 shutdown 2024-03-21T11:46:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.117+0000 7efd039fd740 1 -- 172.21.15.47:0/4221104500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3a9f0 msgr2=0x558c87e54430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.117+0000 7efd039fd740 1 --2- 172.21.15.47:0/4221104500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3a9f0 0x558c87e54430 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7efcec0986c0 tx=0x7efcec09fe40 comp rx=0 tx=0).stop 2024-03-21T11:46:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.118+0000 7efd039fd740 1 -- 172.21.15.47:0/4221104500 shutdown_connections 2024-03-21T11:46:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.118+0000 7efd039fd740 1 --2- 172.21.15.47:0/4221104500 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x558c87eb5950 0x558c87eb7d40 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.118+0000 7efd039fd740 1 --2- 172.21.15.47:0/4221104500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558c87e3c580 0x558c87e51010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.118+0000 7efcdbe00640 0 client.32975 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:46:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.118+0000 7efd039fd740 1 --2- 172.21.15.47:0/4221104500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558c87e3bbc0 0x558c87e54970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.118+0000 7efd039fd740 1 --2- 172.21.15.47:0/4221104500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558c87e3a9f0 0x558c87e54430 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.118+0000 7efd039fd740 1 -- 172.21.15.47:0/4221104500 >> 172.21.15.47:0/4221104500 conn(0x558c87e35de0 msgr2=0x558c87e348d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.118+0000 7efd039fd740 1 -- 172.21.15.47:0/4221104500 shutdown_connections 2024-03-21T11:46:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.118+0000 7efd039fd740 1 -- 172.21.15.47:0/4221104500 wait complete. 2024-03-21T11:46:47.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.118+0000 7efd039fd740 20 client.32975 trim_cache size 0 max 16384 2024-03-21T11:46:47.122 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:46:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.119+0000 7efcfb200640 1 -- 172.21.15.47:0/3262868828 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efce00a3070 msgr2=0x7efce00a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.119+0000 7efcfb200640 1 --2- 172.21.15.47:0/3262868828 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efce00a3070 0x7efce00a5530 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x558c87e3bac0 tx=0x7efcf4009500 comp rx=0 tx=0).stop 2024-03-21T11:46:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.119+0000 7efcfb200640 1 -- 172.21.15.47:0/3262868828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcfc1015a0 msgr2=0x7efcfc199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.119+0000 7efcfb200640 1 --2- 172.21.15.47:0/3262868828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcfc1015a0 0x7efcfc199ab0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7efcec00e9f0 tx=0x7efcec00eeb0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.119+0000 7efcfb200640 1 -- 172.21.15.47:0/3262868828 shutdown_connections 2024-03-21T11:46:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.119+0000 7efcfb200640 1 --2- 172.21.15.47:0/3262868828 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7efce00a3070 0x7efce00a5530 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.119+0000 7efcfb200640 1 --2- 172.21.15.47:0/3262868828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcfc103150 0x7efcfc1a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.119+0000 7efcfb200640 1 --2- 172.21.15.47:0/3262868828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcfc102790 0x7efcfc199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.119+0000 7efcfb200640 1 --2- 172.21.15.47:0/3262868828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcfc1015a0 0x7efcfc199ab0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.119+0000 7efcfb200640 1 -- 172.21.15.47:0/3262868828 >> 172.21.15.47:0/3262868828 conn(0x7efcfc0fcb10 msgr2=0x7efcfc0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.119+0000 7efcfb200640 1 -- 172.21.15.47:0/3262868828 shutdown_connections 2024-03-21T11:46:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.119+0000 7efcfb200640 1 -- 172.21.15.47:0/3262868828 wait complete. 2024-03-21T11:46:47.140 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:46:47.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:46:47.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:46 smithi102 ceph-mon[36748]: pgmap v1104: 129 pgs: 129 active+clean; 704 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.5 MiB/s wr, 512 op/s 2024-03-21T11:46:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.364+0000 7fb082800640 1 -- 172.21.15.47:0/3505705435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c0ff310 msgr2=0x7fb07c0ff730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.364+0000 7fb082800640 1 --2- 172.21.15.47:0/3505705435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c0ff310 0x7fb07c0ff730 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009900 tx=0x7fb06c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:46:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.364+0000 7fb082800640 1 -- 172.21.15.47:0/3505705435 shutdown_connections 2024-03-21T11:46:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.364+0000 7fb082800640 1 --2- 172.21.15.47:0/3505705435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb07c107390 0x7fb07c1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.364+0000 7fb082800640 1 --2- 172.21.15.47:0/3505705435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb07c0ffd00 0x7fb07c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.364+0000 7fb082800640 1 --2- 172.21.15.47:0/3505705435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c0ff310 0x7fb07c0ff730 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.365+0000 7fb082800640 1 -- 172.21.15.47:0/3505705435 >> 172.21.15.47:0/3505705435 conn(0x7fb07c0fae60 msgr2=0x7fb07c0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:47.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.365+0000 7fb082800640 1 -- 172.21.15.47:0/3505705435 shutdown_connections 2024-03-21T11:46:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.365+0000 7fb082800640 1 -- 172.21.15.47:0/3505705435 wait complete. 2024-03-21T11:46:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.366+0000 7fb082800640 1 Processor -- start 2024-03-21T11:46:47.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.366+0000 7fb082800640 1 -- start start 2024-03-21T11:46:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.366+0000 7fb082800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb07c0ff310 0x7fb07c199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.366+0000 7fb082800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb07c0ffd00 0x7fb07c19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.366+0000 7fb082800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c107390 0x7fb07c1a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.367+0000 7fb082800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c10d400 con 0x7fb07c0ffd00 2024-03-21T11:46:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.367+0000 7fb082800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c10d280 con 0x7fb07c107390 2024-03-21T11:46:47.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.367+0000 7fb082800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c10d580 con 0x7fb07c0ff310 2024-03-21T11:46:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.367+0000 7fb07be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c107390 0x7fb07c1a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.367+0000 7fb07b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb07c0ff310 0x7fb07c199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.367+0000 7fb07be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c107390 0x7fb07c1a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44402/0 (socket says 172.21.15.47:44402) 2024-03-21T11:46:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.367+0000 7fb07be00640 1 -- 172.21.15.47:0/2577887018 learned_addr learned my addr 172.21.15.47:0/2577887018 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.367+0000 7fb07aa00640 1 --2- 172.21.15.47:0/2577887018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb07c0ffd00 0x7fb07c19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.367+0000 7fb07be00640 1 -- 172.21.15.47:0/2577887018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb07c0ff310 msgr2=0x7fb07c199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.367+0000 7fb07be00640 1 --2- 172.21.15.47:0/2577887018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb07c0ff310 0x7fb07c199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.367+0000 7fb07be00640 1 -- 172.21.15.47:0/2577887018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb07c0ffd00 msgr2=0x7fb07c19a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.367+0000 7fb07be00640 1 --2- 172.21.15.47:0/2577887018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb07c0ffd00 0x7fb07c19a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.367+0000 7fb07be00640 1 -- 172.21.15.47:0/2577887018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c1a18e0 con 0x7fb07c107390 2024-03-21T11:46:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.368+0000 7fb07b400640 1 --2- 172.21.15.47:0/2577887018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb07c0ff310 0x7fb07c199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.368+0000 7fb07be00640 1 --2- 172.21.15.47:0/2577887018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c107390 0x7fb07c1a1240 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fb07400e9d0 tx=0x7fb07400edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.368+0000 7fb073e00640 1 -- 172.21.15.47:0/2577887018 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074004d20 con 0x7fb07c107390 2024-03-21T11:46:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.368+0000 7fb073e00640 1 -- 172.21.15.47:0/2577887018 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb074005020 con 0x7fb07c107390 2024-03-21T11:46:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.368+0000 7fb073e00640 1 -- 172.21.15.47:0/2577887018 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074011690 con 0x7fb07c107390 2024-03-21T11:46:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.368+0000 7fb082800640 1 -- 172.21.15.47:0/2577887018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c1a1bd0 con 0x7fb07c107390 2024-03-21T11:46:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.368+0000 7fb082800640 1 -- 172.21.15.47:0/2577887018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c1a20f0 con 0x7fb07c107390 2024-03-21T11:46:47.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.372+0000 7fb085479740 1 Processor -- start 2024-03-21T11:46:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.373+0000 7fb073e00640 1 -- 172.21.15.47:0/2577887018 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb074005540 con 0x7fb07c107390 2024-03-21T11:46:47.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.373+0000 7fb085479740 1 -- start start 2024-03-21T11:46:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.373+0000 7fb085479740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3f2810 0x56136b3f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.373+0000 7fb085479740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f39e0 0x56136b3f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.373+0000 7fb085479740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f43a0 0x56136b3fac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.373+0000 7fb085479740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56136b3572c0 con 0x56136b3f39e0 2024-03-21T11:46:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.373+0000 7fb085479740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56136b2ffc90 con 0x56136b3f2810 2024-03-21T11:46:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.373+0000 7fb085479740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56136b353360 con 0x56136b3f43a0 2024-03-21T11:46:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.373+0000 7fb07aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3f2810 0x56136b3f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.373+0000 7fb07b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f39e0 0x56136b3f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.374+0000 7fb07aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3f2810 0x56136b3f2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44408/0 (socket says 172.21.15.47:44408) 2024-03-21T11:46:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.374+0000 7fb07aa00640 1 -- 172.21.15.47:0/1640559721 learned_addr learned my addr 172.21.15.47:0/1640559721 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.374+0000 7fb07aa00640 1 -- 172.21.15.47:0/1640559721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f43a0 msgr2=0x56136b3fac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.374+0000 7fb07aa00640 1 --2- 172.21.15.47:0/1640559721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f43a0 0x56136b3fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.374+0000 7fb07aa00640 1 -- 172.21.15.47:0/1640559721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f39e0 msgr2=0x56136b3f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.374+0000 7fb07aa00640 1 --2- 172.21.15.47:0/1640559721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f39e0 0x56136b3f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.374+0000 7fb07aa00640 1 -- 172.21.15.47:0/1640559721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56136b324080 con 0x56136b3f2810 2024-03-21T11:46:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.374+0000 7fb07aa00640 1 --2- 172.21.15.47:0/1640559721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3f2810 0x56136b3f2c30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fb068009ad0 tx=0x7fb068009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c29b6d68c9ed71d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.374+0000 7fb073e00640 1 --2- 172.21.15.47:0/2577887018 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0600a2fa0 0x7fb0600a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.374+0000 7fb070c00640 1 -- 172.21.15.47:0/1640559721 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb068017070 con 0x56136b3f2810 2024-03-21T11:46:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.375+0000 7fb070c00640 1 -- 172.21.15.47:0/1640559721 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb068010c20 con 0x56136b3f2810 2024-03-21T11:46:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.375+0000 7fb070c00640 1 -- 172.21.15.47:0/1640559721 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06801a5a0 con 0x56136b3f2810 2024-03-21T11:46:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.375+0000 7fb073e00640 1 -- 172.21.15.47:0/2577887018 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb0740cf820 con 0x7fb07c107390 2024-03-21T11:46:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.375+0000 7fb07b400640 1 --2- 172.21.15.47:0/2577887018 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0600a2fa0 0x7fb0600a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.376+0000 7fb085479740 1 -- 172.21.15.47:0/1640559721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3f2810 msgr2=0x56136b3f2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.376+0000 7fb085479740 1 --2- 172.21.15.47:0/1640559721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3f2810 0x56136b3f2c30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fb068009ad0 tx=0x7fb068009f90 comp rx=0 tx=0).stop 2024-03-21T11:46:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.376+0000 7fb085479740 1 -- 172.21.15.47:0/1640559721 shutdown_connections 2024-03-21T11:46:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.376+0000 7fb085479740 1 --2- 172.21.15.47:0/1640559721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f43a0 0x56136b3fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.376+0000 7fb085479740 1 --2- 172.21.15.47:0/1640559721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f39e0 0x56136b3f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.376+0000 7fb085479740 1 --2- 172.21.15.47:0/1640559721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3f2810 0x56136b3f2c30 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.376+0000 7fb085479740 1 -- 172.21.15.47:0/1640559721 >> 172.21.15.47:0/1640559721 conn(0x56136b3edc00 msgr2=0x56136b3ee000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:47.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.376+0000 7fb085479740 1 -- 172.21.15.47:0/1640559721 shutdown_connections 2024-03-21T11:46:47.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.376+0000 7fb07b400640 1 --2- 172.21.15.47:0/2577887018 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0600a2fa0 0x7fb0600a5460 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb06c02c240 tx=0x7fb06c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:47.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.376+0000 7fb085479740 1 -- 172.21.15.47:0/1640559721 wait complete. 2024-03-21T11:46:47.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.377+0000 7fb085479740 1 -- start start 2024-03-21T11:46:47.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.377+0000 7fb085479740 1 Processor -- start 2024-03-21T11:46:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.378+0000 7fb085479740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3f2810 0x56136b3fca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.378+0000 7fb085479740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f39e0 0x56136b3fcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.378+0000 7fb085479740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f43a0 0x56136b403f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.378+0000 7fb085479740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56136b3fb3b0 con 0x56136b3f39e0 2024-03-21T11:46:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.378+0000 7fb085479740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56136b3130d0 con 0x56136b3f2810 2024-03-21T11:46:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.378+0000 7fb085479740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56136b3fb530 con 0x56136b3f43a0 2024-03-21T11:46:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.378+0000 7fb085479740 10 client.0 _ll_register_callbacks cb 0x56136b317e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.378+0000 7fb07be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f43a0 0x56136b403f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.378+0000 7fb07be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f43a0 0x56136b403f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41466/0 (socket says 172.21.15.47:41466) 2024-03-21T11:46:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.378+0000 7fb07be00640 1 -- 172.21.15.47:0/1650442951 learned_addr learned my addr 172.21.15.47:0/1650442951 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.379+0000 7fb07b400640 1 --2- 172.21.15.47:0/1650442951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3f2810 0x56136b3fca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.379+0000 7fb07be00640 1 -- 172.21.15.47:0/1650442951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3f2810 msgr2=0x56136b3fca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.379+0000 7fb07be00640 1 --2- 172.21.15.47:0/1650442951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3f2810 0x56136b3fca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.379+0000 7fb07be00640 1 -- 172.21.15.47:0/1650442951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f39e0 msgr2=0x56136b3fcf40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:46:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.380+0000 7fb07be00640 1 --2- 172.21.15.47:0/1650442951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f39e0 0x56136b3fcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.380+0000 7fb07be00640 1 -- 172.21.15.47:0/1650442951 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56136b06b3f0 con 0x56136b3f43a0 2024-03-21T11:46:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.380+0000 7fb07b400640 1 --2- 172.21.15.47:0/1650442951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3f2810 0x56136b3fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.381+0000 7fb07be00640 1 --2- 172.21.15.47:0/1650442951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f43a0 0x56136b403f80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fb07409bb50 tx=0x7fb07409c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd9edca39ad6831a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:47.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.381+0000 7fb065600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.381+0000 7fb065600640 1 -- 172.21.15.47:0/1650442951 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07401f070 con 0x56136b3f43a0 2024-03-21T11:46:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.381+0000 7fb065600640 1 -- 172.21.15.47:0/1650442951 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb07409b010 con 0x56136b3f43a0 2024-03-21T11:46:47.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.381+0000 7fb085479740 1 -- 172.21.15.47:0/1650442951 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb068010c20 con 0x56136b3f43a0 2024-03-21T11:46:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.382+0000 7fb065600640 1 -- 172.21.15.47:0/1650442951 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0740a45d0 con 0x56136b3f43a0 2024-03-21T11:46:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.382+0000 7fb085479740 10 client.32959 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:47.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.382+0000 7fb085479740 1 -- 172.21.15.47:0/1650442951 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56136b1357d0 con 0x56136b3f43a0 2024-03-21T11:46:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.383+0000 7fb065600640 1 -- 172.21.15.47:0/1650442951 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb068010c20 con 0x56136b3f43a0 2024-03-21T11:46:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.383+0000 7fb065600640 1 -- 172.21.15.47:0/1650442951 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb07400f040 con 0x56136b3f43a0 2024-03-21T11:46:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.383+0000 7fb085479740 10 client.32959 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.383+0000 7fb085479740 10 client.32959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.383+0000 7fb085479740 20 client.32959 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.383+0000 7fb085479740 1 --2- 172.21.15.47:0/1650442951 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56136b408580 0x56136b40aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.383+0000 7fb085479740 4 client.32959 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:47.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.383+0000 7fb085479740 1 -- 172.21.15.47:0/1650442951 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56136b364800 con 0x56136b408580 2024-03-21T11:46:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.384+0000 7fb07b400640 1 --2- 172.21.15.47:0/1650442951 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56136b408580 0x56136b40aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.384+0000 7fb07b400640 1 --2- 172.21.15.47:0/1650442951 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56136b408580 0x56136b40aa40 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c3f3739a39355c server_cookie=25b3b709dc9d6f37 in_seq=0 out_seq=0 2024-03-21T11:46:47.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.385+0000 7fb065600640 10 client.32959 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.386+0000 7fb065600640 1 -- 172.21.15.47:0/1650442951 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56136b364800 con 0x56136b408580 2024-03-21T11:46:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.386+0000 7fb065600640 10 client.32959 handle_command_reply: tid=0 2024-03-21T11:46:47.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.386+0000 7fb085479740 1 client.32959 shutdown 2024-03-21T11:46:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.386+0000 7fb085479740 1 -- 172.21.15.47:0/1650442951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f43a0 msgr2=0x56136b403f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.386+0000 7fb085479740 1 --2- 172.21.15.47:0/1650442951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f43a0 0x56136b403f80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fb07409bb50 tx=0x7fb07409c380 comp rx=0 tx=0).stop 2024-03-21T11:46:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.387+0000 7fb085479740 1 -- 172.21.15.47:0/1650442951 shutdown_connections 2024-03-21T11:46:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.387+0000 7fb085479740 1 --2- 172.21.15.47:0/1650442951 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56136b408580 0x56136b40aa40 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.387+0000 7fb085479740 1 --2- 172.21.15.47:0/1650442951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f43a0 0x56136b403f80 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.387+0000 7fb085479740 1 --2- 172.21.15.47:0/1650442951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f39e0 0x56136b3fcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.387+0000 7fb085479740 1 --2- 172.21.15.47:0/1650442951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3f2810 0x56136b3fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.387+0000 7fb065600640 0 client.32959 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:47.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.387+0000 7fb085479740 1 -- 172.21.15.47:0/1650442951 >> 172.21.15.47:0/1650442951 conn(0x56136b3edc00 msgr2=0x56136b3eceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.387+0000 7fb085479740 1 -- 172.21.15.47:0/1650442951 shutdown_connections 2024-03-21T11:46:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.387+0000 7fb085479740 1 -- 172.21.15.47:0/1650442951 wait complete. 2024-03-21T11:46:47.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.388+0000 7fb085479740 20 client.32959 trim_cache size 0 max 16384 2024-03-21T11:46:47.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:46 smithi100 ceph-mon[36245]: pgmap v1104: 129 pgs: 129 active+clean; 704 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.5 MiB/s wr, 512 op/s 2024-03-21T11:46:47.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.403+0000 7fb085479740 1 Processor -- start 2024-03-21T11:46:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.403+0000 7fb085479740 1 -- start start 2024-03-21T11:46:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.403+0000 7fb085479740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3edc00 0x56136b3ec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.403+0000 7fb085479740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f2810 0x56136b403f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.403+0000 7fb085479740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f39e0 0x56136b3ecb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.403+0000 7fb085479740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c1ac2c0 con 0x56136b3f2810 2024-03-21T11:46:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.403+0000 7fb085479740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c1adc40 con 0x56136b3edc00 2024-03-21T11:46:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.403+0000 7fb085479740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c1ac920 con 0x56136b3f39e0 2024-03-21T11:46:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.403+0000 7fb07b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3edc00 0x56136b3ec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.404+0000 7fb07b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3edc00 0x56136b3ec620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44430/0 (socket says 172.21.15.47:44430) 2024-03-21T11:46:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.404+0000 7fb07b400640 1 -- 172.21.15.47:0/999952534 learned_addr learned my addr 172.21.15.47:0/999952534 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:47.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.404+0000 7fb07aa00640 1 --2- 172.21.15.47:0/999952534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f2810 0x56136b403f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.404+0000 7fb07be00640 1 --2- 172.21.15.47:0/999952534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f39e0 0x56136b3ecb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.404+0000 7fb07b400640 1 -- 172.21.15.47:0/999952534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f39e0 msgr2=0x56136b3ecb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.404+0000 7fb07b400640 1 --2- 172.21.15.47:0/999952534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f39e0 0x56136b3ecb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.404+0000 7fb07b400640 1 -- 172.21.15.47:0/999952534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f2810 msgr2=0x56136b403f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.404+0000 7fb07b400640 1 --2- 172.21.15.47:0/999952534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f2810 0x56136b403f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.404+0000 7fb07b400640 1 -- 172.21.15.47:0/999952534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56136b3fb6b0 con 0x56136b3edc00 2024-03-21T11:46:47.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.404+0000 7fb07be00640 1 --2- 172.21.15.47:0/999952534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f39e0 0x56136b3ecb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.404+0000 7fb07b400640 1 --2- 172.21.15.47:0/999952534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3edc00 0x56136b3ec620 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fb06c005ab0 tx=0x7fb06c042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2520340f35f362b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.405+0000 7fb067e00640 1 -- 172.21.15.47:0/999952534 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c05d070 con 0x56136b3edc00 2024-03-21T11:46:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.405+0000 7fb067e00640 1 -- 172.21.15.47:0/999952534 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb06c0055d0 con 0x56136b3edc00 2024-03-21T11:46:47.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.405+0000 7fb067e00640 1 -- 172.21.15.47:0/999952534 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c046070 con 0x56136b3edc00 2024-03-21T11:46:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.406+0000 7fb085479740 1 -- 172.21.15.47:0/999952534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3edc00 msgr2=0x56136b3ec620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.406+0000 7fb085479740 1 --2- 172.21.15.47:0/999952534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3edc00 0x56136b3ec620 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fb06c005ab0 tx=0x7fb06c042000 comp rx=0 tx=0).stop 2024-03-21T11:46:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.406+0000 7fb085479740 1 -- 172.21.15.47:0/999952534 shutdown_connections 2024-03-21T11:46:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.406+0000 7fb085479740 1 --2- 172.21.15.47:0/999952534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f39e0 0x56136b3ecb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.406+0000 7fb085479740 1 --2- 172.21.15.47:0/999952534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f2810 0x56136b403f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.406+0000 7fb085479740 1 --2- 172.21.15.47:0/999952534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3edc00 0x56136b3ec620 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.406+0000 7fb085479740 1 -- 172.21.15.47:0/999952534 >> 172.21.15.47:0/999952534 conn(0x56136b3ee350 msgr2=0x56136b3f16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.406+0000 7fb085479740 1 -- 172.21.15.47:0/999952534 shutdown_connections 2024-03-21T11:46:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.406+0000 7fb085479740 1 -- 172.21.15.47:0/999952534 wait complete. 2024-03-21T11:46:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.407+0000 7fb085479740 1 -- start start 2024-03-21T11:46:47.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.407+0000 7fb085479740 1 Processor -- start 2024-03-21T11:46:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.407+0000 7fb085479740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3edc00 0x56136b4039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.407+0000 7fb085479740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f2810 0x56136b403f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.407+0000 7fb085479740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f39e0 0x56136b402100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.407+0000 7fb085479740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c1ad5e0 con 0x56136b3f2810 2024-03-21T11:46:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.407+0000 7fb085479740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c1acf80 con 0x56136b3edc00 2024-03-21T11:46:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.407+0000 7fb085479740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c1aa480 con 0x56136b3f39e0 2024-03-21T11:46:47.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.408+0000 7fb085479740 10 client.0 _ll_register_callbacks cb 0x7fb074004f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.408+0000 7fb07aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f2810 0x56136b403f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.408+0000 7fb07b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3edc00 0x56136b4039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.408+0000 7fb07aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f2810 0x56136b403f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60982/0 (socket says 172.21.15.47:60982) 2024-03-21T11:46:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.408+0000 7fb07aa00640 1 -- 172.21.15.47:0/4049885276 learned_addr learned my addr 172.21.15.47:0/4049885276 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.408+0000 7fb07be00640 1 --2- 172.21.15.47:0/4049885276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f39e0 0x56136b402100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.408+0000 7fb07aa00640 1 -- 172.21.15.47:0/4049885276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f39e0 msgr2=0x56136b402100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.408+0000 7fb07aa00640 1 --2- 172.21.15.47:0/4049885276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f39e0 0x56136b402100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.408+0000 7fb07aa00640 1 -- 172.21.15.47:0/4049885276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3edc00 msgr2=0x56136b4039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.408+0000 7fb07aa00640 1 --2- 172.21.15.47:0/4049885276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3edc00 0x56136b4039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.409+0000 7fb07aa00640 1 -- 172.21.15.47:0/4049885276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56136b402930 con 0x56136b3f2810 2024-03-21T11:46:47.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.413+0000 7fb07aa00640 1 --2- 172.21.15.47:0/4049885276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f2810 0x56136b403f10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb068009ad0 tx=0x7fb068010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcccd797658cadbd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:47.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.413+0000 7fb065600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:46:47.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.414+0000 7fb065600640 1 -- 172.21.15.47:0/4049885276 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb068017040 con 0x56136b3f2810 2024-03-21T11:46:47.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.414+0000 7fb085479740 1 -- 172.21.15.47:0/4049885276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb06c0055d0 con 0x56136b3f2810 2024-03-21T11:46:47.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.415+0000 7fb065600640 1 -- 172.21.15.47:0/4049885276 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb06801c660 con 0x56136b3f2810 2024-03-21T11:46:47.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.416+0000 7fb065600640 1 -- 172.21.15.47:0/4049885276 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb068017040 con 0x56136b3f2810 2024-03-21T11:46:47.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.416+0000 7fb085479740 10 client.22170 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:47.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.416+0000 7fb085479740 1 -- 172.21.15.47:0/4049885276 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56136b46d2f0 con 0x56136b3f2810 2024-03-21T11:46:47.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.418+0000 7fb065600640 1 -- 172.21.15.47:0/4049885276 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb06c0055d0 con 0x56136b3f2810 2024-03-21T11:46:47.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.418+0000 7fb065600640 1 -- 172.21.15.47:0/4049885276 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb068013a10 con 0x56136b3f2810 2024-03-21T11:46:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.419+0000 7fb085479740 10 client.22170 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.419+0000 7fb085479740 10 client.22170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.419+0000 7fb085479740 20 client.22170 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.419+0000 7fb085479740 1 --2- 172.21.15.47:0/4049885276 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56136b3f43a0 0x56136b46f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.419+0000 7fb085479740 4 client.22170 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:46:47.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.419+0000 7fb085479740 1 -- 172.21.15.47:0/4049885276 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56136b35eb50 con 0x56136b3f43a0 2024-03-21T11:46:47.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.420+0000 7fb07b400640 1 --2- 172.21.15.47:0/4049885276 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56136b3f43a0 0x56136b46f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:47.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.422+0000 7fb07b400640 1 --2- 172.21.15.47:0/4049885276 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56136b3f43a0 0x56136b46f920 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edff54d3743a85e6 server_cookie=994f99030c512cab in_seq=0 out_seq=0 2024-03-21T11:46:47.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.422+0000 7fb065600640 10 client.22170 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:47.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.424+0000 7fb065600640 1 -- 172.21.15.47:0/4049885276 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56136b35eb50 con 0x56136b3f43a0 2024-03-21T11:46:47.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.424+0000 7fb065600640 10 client.22170 handle_command_reply: tid=0 2024-03-21T11:46:47.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.424+0000 7fb085479740 1 client.22170 shutdown 2024-03-21T11:46:47.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.425+0000 7fb085479740 1 -- 172.21.15.47:0/4049885276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f2810 msgr2=0x56136b403f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.425+0000 7fb085479740 1 --2- 172.21.15.47:0/4049885276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f2810 0x56136b403f10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb068009ad0 tx=0x7fb068010e50 comp rx=0 tx=0).stop 2024-03-21T11:46:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.426+0000 7fb085479740 1 -- 172.21.15.47:0/4049885276 shutdown_connections 2024-03-21T11:46:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.426+0000 7fb085479740 1 --2- 172.21.15.47:0/4049885276 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x56136b3f43a0 0x56136b46f920 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.426+0000 7fb085479740 1 --2- 172.21.15.47:0/4049885276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56136b3f39e0 0x56136b402100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.426+0000 7fb085479740 1 --2- 172.21.15.47:0/4049885276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56136b3f2810 0x56136b403f10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.426+0000 7fb085479740 1 --2- 172.21.15.47:0/4049885276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56136b3edc00 0x56136b4039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.426+0000 7fb085479740 1 -- 172.21.15.47:0/4049885276 >> 172.21.15.47:0/4049885276 conn(0x56136b3ee350 msgr2=0x56136b3ec620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.428+0000 7fb065600640 0 client.22170 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.429+0000 7fb085479740 1 -- 172.21.15.47:0/4049885276 shutdown_connections 2024-03-21T11:46:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.429+0000 7fb085479740 1 -- 172.21.15.47:0/4049885276 wait complete. 2024-03-21T11:46:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.429+0000 7fb085479740 20 client.22170 trim_cache size 0 max 16384 2024-03-21T11:46:47.431 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:46:47.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.432+0000 7fb082800640 1 -- 172.21.15.47:0/2577887018 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0600a2fa0 msgr2=0x7fb0600a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.432+0000 7fb082800640 1 --2- 172.21.15.47:0/2577887018 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0600a2fa0 0x7fb0600a5460 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb06c02c240 tx=0x7fb06c02df90 comp rx=0 tx=0).stop 2024-03-21T11:46:47.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.432+0000 7fb082800640 1 -- 172.21.15.47:0/2577887018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c107390 msgr2=0x7fb07c1a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:47.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.432+0000 7fb082800640 1 --2- 172.21.15.47:0/2577887018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c107390 0x7fb07c1a1240 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fb07400e9d0 tx=0x7fb07400edd0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.432+0000 7fb082800640 1 -- 172.21.15.47:0/2577887018 shutdown_connections 2024-03-21T11:46:47.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.432+0000 7fb082800640 1 --2- 172.21.15.47:0/2577887018 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb0600a2fa0 0x7fb0600a5460 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.432+0000 7fb082800640 1 --2- 172.21.15.47:0/2577887018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c107390 0x7fb07c1a1240 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.432+0000 7fb082800640 1 --2- 172.21.15.47:0/2577887018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb07c0ffd00 0x7fb07c19a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.432+0000 7fb082800640 1 --2- 172.21.15.47:0/2577887018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb07c0ff310 0x7fb07c199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:47.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.432+0000 7fb082800640 1 -- 172.21.15.47:0/2577887018 >> 172.21.15.47:0/2577887018 conn(0x7fb07c0fae60 msgr2=0x7fb07c0fb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:47.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.433+0000 7fb082800640 1 -- 172.21.15.47:0/2577887018 shutdown_connections 2024-03-21T11:46:47.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:47.433+0000 7fb082800640 1 -- 172.21.15.47:0/2577887018 wait complete. 2024-03-21T11:46:47.990 INFO:tasks.workunit.client.0.smithi047.stdout: 10 240638 21.79 MB/sec execute 451 sec latency 54.436 ms 2024-03-21T11:46:48.466 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:46:48.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:46:48.466 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:46:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.827+0000 7f4efb200640 1 -- 172.21.15.47:0/3727074122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef410dc80 msgr2=0x7f4ef4110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.827+0000 7f4efb200640 1 --2- 172.21.15.47:0/3727074122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef410dc80 0x7f4ef4110070 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009cd0 tx=0x7f4eec02c100 comp rx=0 tx=0).stop 2024-03-21T11:46:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.828+0000 7f4efb200640 1 -- 172.21.15.47:0/3727074122 shutdown_connections 2024-03-21T11:46:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.828+0000 7f4efb200640 1 --2- 172.21.15.47:0/3727074122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef410dc80 0x7f4ef4110070 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.828+0000 7f4efb200640 1 --2- 172.21.15.47:0/3727074122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ef4074100 0x7f4ef4074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.828+0000 7f4efb200640 1 --2- 172.21.15.47:0/3727074122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef40756e0 0x7f4ef4073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.828+0000 7f4efb200640 1 -- 172.21.15.47:0/3727074122 >> 172.21.15.47:0/3727074122 conn(0x7f4ef40fcc00 msgr2=0x7f4ef40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.828+0000 7f4efb200640 1 -- 172.21.15.47:0/3727074122 shutdown_connections 2024-03-21T11:46:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.828+0000 7f4efb200640 1 -- 172.21.15.47:0/3727074122 wait complete. 2024-03-21T11:46:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.830+0000 7f4efb200640 1 Processor -- start 2024-03-21T11:46:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.830+0000 7f4efb200640 1 -- start start 2024-03-21T11:46:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.830+0000 7f4efb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef4074100 0x7f4ef419e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.830+0000 7f4efb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef40756e0 0x7f4ef419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.831+0000 7f4efb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ef410dc80 0x7f4ef41a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.831+0000 7f4efb200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef41139a0 con 0x7f4ef40756e0 2024-03-21T11:46:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.831+0000 7f4efb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef4113820 con 0x7f4ef4074100 2024-03-21T11:46:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.831+0000 7f4efb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef4113b20 con 0x7f4ef410dc80 2024-03-21T11:46:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.831+0000 7f4ef3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef4074100 0x7f4ef419e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.831+0000 7f4ef3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef4074100 0x7f4ef419e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44454/0 (socket says 172.21.15.47:44454) 2024-03-21T11:46:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.831+0000 7f4ef3e00640 1 -- 172.21.15.47:0/1045657114 learned_addr learned my addr 172.21.15.47:0/1045657114 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.832+0000 7f4ef9200640 1 --2- 172.21.15.47:0/1045657114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ef410dc80 0x7f4ef41a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.833+0000 7f4ef3e00640 1 -- 172.21.15.47:0/1045657114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ef410dc80 msgr2=0x7f4ef41a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.833+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/1045657114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ef410dc80 0x7f4ef41a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.833+0000 7f4ef3e00640 1 -- 172.21.15.47:0/1045657114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef40756e0 msgr2=0x7f4ef419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.833+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/1045657114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef40756e0 0x7f4ef419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.833+0000 7f4ef3e00640 1 -- 172.21.15.47:0/1045657114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef41a5c90 con 0x7f4ef4074100 2024-03-21T11:46:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.833+0000 7f4ef9200640 1 --2- 172.21.15.47:0/1045657114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ef410dc80 0x7f4ef41a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.835+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/1045657114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef4074100 0x7f4ef419e070 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400d950 tx=0x7f4ee400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:48.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.836+0000 7f4ef0c00640 1 -- 172.21.15.47:0/1045657114 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4017070 con 0x7f4ef4074100 2024-03-21T11:46:48.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.836+0000 7f4ef0c00640 1 -- 172.21.15.47:0/1045657114 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ee4009c30 con 0x7f4ef4074100 2024-03-21T11:46:48.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.836+0000 7f4ef0c00640 1 -- 172.21.15.47:0/1045657114 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400f6b0 con 0x7f4ef4074100 2024-03-21T11:46:48.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.836+0000 7f4efb200640 1 -- 172.21.15.47:0/1045657114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef41a5f80 con 0x7f4ef4074100 2024-03-21T11:46:48.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.836+0000 7f4efb200640 1 -- 172.21.15.47:0/1045657114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef41a6510 con 0x7f4ef4074100 2024-03-21T11:46:48.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.841+0000 7f4ef0c00640 1 -- 172.21.15.47:0/1045657114 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4ee40109d0 con 0x7f4ef4074100 2024-03-21T11:46:48.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.844+0000 7f4ef0c00640 1 --2- 172.21.15.47:0/1045657114 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4ed80a2fa0 0x7f4ed80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.844+0000 7f4ef0c00640 1 -- 172.21.15.47:0/1045657114 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4ee40cec10 con 0x7f4ef4074100 2024-03-21T11:46:48.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.845+0000 7f4ef3400640 1 --2- 172.21.15.47:0/1045657114 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4ed80a2fa0 0x7f4ed80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.849+0000 7f4efde16740 1 Processor -- start 2024-03-21T11:46:48.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.850+0000 7f4ef3400640 1 --2- 172.21.15.47:0/1045657114 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4ed80a2fa0 0x7f4ed80a5460 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4ef419f580 tx=0x7f4ee0006c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.851+0000 7f4efde16740 1 -- start start 2024-03-21T11:46:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.851+0000 7f4efde16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce65970a0 0x55fce65974c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.851+0000 7f4efde16740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce6597a90 0x55fce659c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.851+0000 7f4efde16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce659cc00 0x55fce659f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.851+0000 7f4efde16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce64fceb0 con 0x55fce659cc00 2024-03-21T11:46:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.851+0000 7f4efde16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce64a5a60 con 0x55fce65970a0 2024-03-21T11:46:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.851+0000 7f4efde16740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce64f8f50 con 0x55fce6597a90 2024-03-21T11:46:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.852+0000 7f4ef9200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce659cc00 0x55fce659f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.852+0000 7f4ef9200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce659cc00 0x55fce659f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32794/0 (socket says 172.21.15.47:32794) 2024-03-21T11:46:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.852+0000 7f4ef9200640 1 -- 172.21.15.47:0/884040280 learned_addr learned my addr 172.21.15.47:0/884040280 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.852+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/884040280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce65970a0 0x55fce65974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.852+0000 7f4ef3400640 1 --2- 172.21.15.47:0/884040280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce6597a90 0x55fce659c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.852+0000 7f4ef3e00640 1 -- 172.21.15.47:0/884040280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce6597a90 msgr2=0x55fce659c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.852+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/884040280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce6597a90 0x55fce659c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.852+0000 7f4ef3e00640 1 -- 172.21.15.47:0/884040280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce659cc00 msgr2=0x55fce659f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.852+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/884040280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce659cc00 0x55fce659f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.853+0000 7f4ef3e00640 1 -- 172.21.15.47:0/884040280 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fce62d97e0 con 0x55fce65970a0 2024-03-21T11:46:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.853+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/884040280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce65970a0 0x55fce65974c0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40cc6f0 tx=0x7f4ee4010d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdd0e701e49c302f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.853+0000 7f4ee9600640 1 -- 172.21.15.47:0/884040280 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40b4070 con 0x55fce65970a0 2024-03-21T11:46:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.853+0000 7f4ee9600640 1 -- 172.21.15.47:0/884040280 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ee40a39c0 con 0x55fce65970a0 2024-03-21T11:46:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.853+0000 7f4ee9600640 1 -- 172.21.15.47:0/884040280 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4020070 con 0x55fce65970a0 2024-03-21T11:46:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.854+0000 7f4efde16740 1 -- 172.21.15.47:0/884040280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce65970a0 msgr2=0x55fce65974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.854+0000 7f4efde16740 1 --2- 172.21.15.47:0/884040280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce65970a0 0x55fce65974c0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40cc6f0 tx=0x7f4ee4010d70 comp rx=0 tx=0).stop 2024-03-21T11:46:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.857+0000 7f4efde16740 1 -- 172.21.15.47:0/884040280 shutdown_connections 2024-03-21T11:46:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.857+0000 7f4efde16740 1 --2- 172.21.15.47:0/884040280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce659cc00 0x55fce659f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.857+0000 7f4efde16740 1 --2- 172.21.15.47:0/884040280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce6597a90 0x55fce659c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.857+0000 7f4efde16740 1 --2- 172.21.15.47:0/884040280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce65970a0 0x55fce65974c0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.857+0000 7f4efde16740 1 -- 172.21.15.47:0/884040280 >> 172.21.15.47:0/884040280 conn(0x55fce6592610 msgr2=0x55fce6592a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.858+0000 7f4efde16740 1 -- 172.21.15.47:0/884040280 shutdown_connections 2024-03-21T11:46:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.858+0000 7f4efde16740 1 -- 172.21.15.47:0/884040280 wait complete. 2024-03-21T11:46:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.858+0000 7f4efde16740 1 -- start start 2024-03-21T11:46:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.859+0000 7f4efde16740 1 Processor -- start 2024-03-21T11:46:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.860+0000 7f4efde16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce65970a0 0x55fce65b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.860+0000 7f4efde16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce6597a90 0x55fce65b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.860+0000 7f4efde16740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce659cc00 0x55fce65b9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.860+0000 7f4efde16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce65b0b60 con 0x55fce65970a0 2024-03-21T11:46:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.860+0000 7f4efde16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce64b8eb0 con 0x55fce6597a90 2024-03-21T11:46:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.860+0000 7f4efde16740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce65b0ce0 con 0x55fce659cc00 2024-03-21T11:46:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.860+0000 7f4efde16740 10 client.0 _ll_register_callbacks cb 0x55fce64ad150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.861+0000 7f4ef3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce65970a0 0x55fce65b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.861+0000 7f4ef9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce659cc00 0x55fce65b9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.861+0000 7f4ef3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce6597a90 0x55fce65b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.861+0000 7f4ef9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce659cc00 0x55fce65b9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41538/0 (socket says 172.21.15.47:41538) 2024-03-21T11:46:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.861+0000 7f4ef9200640 1 -- 172.21.15.47:0/2199695165 learned_addr learned my addr 172.21.15.47:0/2199695165 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.862+0000 7f4ef3e00640 1 -- 172.21.15.47:0/2199695165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce659cc00 msgr2=0x55fce65b9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.862+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/2199695165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce659cc00 0x55fce65b9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.862+0000 7f4ef3e00640 1 -- 172.21.15.47:0/2199695165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce6597a90 msgr2=0x55fce65b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.862+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/2199695165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce6597a90 0x55fce65b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.862+0000 7f4ef3e00640 1 -- 172.21.15.47:0/2199695165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fce6339f70 con 0x55fce65970a0 2024-03-21T11:46:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.862+0000 7f4ef9200640 1 --2- 172.21.15.47:0/2199695165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce659cc00 0x55fce65b9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.863+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/2199695165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce65970a0 0x55fce65b22c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55fce65982f0 tx=0x7f4ee4097300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2b2b91d1c866da server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.863+0000 7f4ede000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:46:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.863+0000 7f4ede000640 1 -- 172.21.15.47:0/2199695165 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4020030 con 0x55fce65970a0 2024-03-21T11:46:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.863+0000 7f4efde16740 1 -- 172.21.15.47:0/2199695165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ee40a39c0 con 0x55fce65970a0 2024-03-21T11:46:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.864+0000 7f4ede000640 1 -- 172.21.15.47:0/2199695165 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4ee409a400 con 0x55fce65970a0 2024-03-21T11:46:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.864+0000 7f4ede000640 1 -- 172.21.15.47:0/2199695165 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ee40b4040 con 0x55fce65970a0 2024-03-21T11:46:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.864+0000 7f4efde16740 10 client.22179 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.864+0000 7f4efde16740 1 -- 172.21.15.47:0/2199695165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55fce61d1170 con 0x55fce65970a0 2024-03-21T11:46:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.865+0000 7f4ede000640 1 -- 172.21.15.47:0/2199695165 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ee40a39c0 con 0x55fce65970a0 2024-03-21T11:46:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.865+0000 7f4ede000640 1 -- 172.21.15.47:0/2199695165 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4ee4099f70 con 0x55fce65970a0 2024-03-21T11:46:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.866+0000 7f4efde16740 10 client.22179 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.866+0000 7f4efde16740 10 client.22179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.866+0000 7f4efde16740 20 client.22179 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.866+0000 7f4efde16740 1 --2- 172.21.15.47:0/2199695165 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fce65bdd30 0x55fce65c0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.866+0000 7f4efde16740 4 client.22179 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:48.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.866+0000 7f4efde16740 1 -- 172.21.15.47:0/2199695165 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fce650a3f0 con 0x55fce65bdd30 2024-03-21T11:46:48.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.867+0000 7f4ef3400640 1 --2- 172.21.15.47:0/2199695165 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fce65bdd30 0x55fce65c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.870+0000 7f4ede000640 10 client.22179 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:48.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.870+0000 7f4ef3400640 1 --2- 172.21.15.47:0/2199695165 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fce65bdd30 0x55fce65c0120 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=600479b08b922941 server_cookie=36c6461a4b308ea5 in_seq=0 out_seq=0 2024-03-21T11:46:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.897+0000 7f4ede000640 1 -- 172.21.15.47:0/2199695165 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fce650a3f0 con 0x55fce65bdd30 2024-03-21T11:46:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.897+0000 7f4ede000640 10 client.22179 handle_command_reply: tid=0 2024-03-21T11:46:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.897+0000 7f4efde16740 1 client.22179 shutdown 2024-03-21T11:46:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.899+0000 7f4efde16740 1 -- 172.21.15.47:0/2199695165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce65970a0 msgr2=0x55fce65b22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.899+0000 7f4efde16740 1 --2- 172.21.15.47:0/2199695165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce65970a0 0x55fce65b22c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55fce65982f0 tx=0x7f4ee4097300 comp rx=0 tx=0).stop 2024-03-21T11:46:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.900+0000 7f4efde16740 1 -- 172.21.15.47:0/2199695165 shutdown_connections 2024-03-21T11:46:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.900+0000 7f4efde16740 1 --2- 172.21.15.47:0/2199695165 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fce65bdd30 0x55fce65c0120 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.901+0000 7f4efde16740 1 --2- 172.21.15.47:0/2199695165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce659cc00 0x55fce65b9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.901+0000 7f4efde16740 1 --2- 172.21.15.47:0/2199695165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce6597a90 0x55fce65b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.901+0000 7f4efde16740 1 --2- 172.21.15.47:0/2199695165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce65970a0 0x55fce65b22c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.901+0000 7f4efde16740 1 -- 172.21.15.47:0/2199695165 >> 172.21.15.47:0/2199695165 conn(0x55fce6592610 msgr2=0x55fce6591830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.901+0000 7f4ede000640 0 client.22179 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.901+0000 7f4efde16740 1 -- 172.21.15.47:0/2199695165 shutdown_connections 2024-03-21T11:46:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.901+0000 7f4efde16740 1 -- 172.21.15.47:0/2199695165 wait complete. 2024-03-21T11:46:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.901+0000 7f4efde16740 20 client.22179 trim_cache size 0 max 16384 2024-03-21T11:46:48.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.937+0000 7f4efde16740 1 Processor -- start 2024-03-21T11:46:48.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.938+0000 7f4efde16740 1 -- start start 2024-03-21T11:46:48.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.938+0000 7f4efde16740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce65970a0 0x55fce65b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.939+0000 7f4efde16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce6597a90 0x55fce65b97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.939+0000 7f4efde16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce659cc00 0x55fce6598530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.939+0000 7f4efde16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef41a7fd0 con 0x55fce6597a90 2024-03-21T11:46:48.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.939+0000 7f4efde16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef41a9950 con 0x55fce659cc00 2024-03-21T11:46:48.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.939+0000 7f4efde16740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef41a8630 con 0x55fce65970a0 2024-03-21T11:46:48.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.939+0000 7f4ef3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce65970a0 0x55fce65b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.939+0000 7f4ef3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce65970a0 0x55fce65b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60816/0 (socket says 172.21.15.47:60816) 2024-03-21T11:46:48.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.939+0000 7f4ef3e00640 1 -- 172.21.15.47:0/1886099190 learned_addr learned my addr 172.21.15.47:0/1886099190 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:48.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.940+0000 7f4ef3e00640 1 -- 172.21.15.47:0/1886099190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce659cc00 msgr2=0x55fce6598530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:48.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.940+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/1886099190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce659cc00 0x55fce6598530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.940+0000 7f4ef3e00640 1 -- 172.21.15.47:0/1886099190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce6597a90 msgr2=0x55fce65b97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:48.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.940+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/1886099190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce6597a90 0x55fce65b97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.940+0000 7f4ef3e00640 1 -- 172.21.15.47:0/1886099190 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fce659bfc0 con 0x55fce65970a0 2024-03-21T11:46:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.942+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/1886099190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce65970a0 0x55fce65b2540 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009e00 tx=0x7f4ee4010d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b932e64e149347b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.942+0000 7f4ee8c00640 1 -- 172.21.15.47:0/1886099190 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4020070 con 0x55fce65970a0 2024-03-21T11:46:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.942+0000 7f4ee8c00640 1 -- 172.21.15.47:0/1886099190 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ee409f100 con 0x55fce65970a0 2024-03-21T11:46:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.942+0000 7f4ee8c00640 1 -- 172.21.15.47:0/1886099190 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee409be20 con 0x55fce65970a0 2024-03-21T11:46:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.944+0000 7f4efde16740 1 -- 172.21.15.47:0/1886099190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce65970a0 msgr2=0x55fce65b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.944+0000 7f4efde16740 1 --2- 172.21.15.47:0/1886099190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce65970a0 0x55fce65b2540 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009e00 tx=0x7f4ee4010d70 comp rx=0 tx=0).stop 2024-03-21T11:46:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.945+0000 7f4efde16740 1 -- 172.21.15.47:0/1886099190 shutdown_connections 2024-03-21T11:46:48.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.945+0000 7f4efde16740 1 --2- 172.21.15.47:0/1886099190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce659cc00 0x55fce6598530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.945+0000 7f4efde16740 1 --2- 172.21.15.47:0/1886099190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce6597a90 0x55fce65b97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.945+0000 7f4efde16740 1 --2- 172.21.15.47:0/1886099190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce65970a0 0x55fce65b2540 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.945+0000 7f4efde16740 1 -- 172.21.15.47:0/1886099190 >> 172.21.15.47:0/1886099190 conn(0x55fce6592610 msgr2=0x55fce6596100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:48.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.946+0000 7f4efde16740 1 -- 172.21.15.47:0/1886099190 shutdown_connections 2024-03-21T11:46:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.946+0000 7f4efde16740 1 -- 172.21.15.47:0/1886099190 wait complete. 2024-03-21T11:46:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.948+0000 7f4efde16740 1 -- start start 2024-03-21T11:46:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.949+0000 7f4efde16740 1 Processor -- start 2024-03-21T11:46:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.950+0000 7f4efde16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce65970a0 0x55fce65b7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.951+0000 7f4efde16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce6597a90 0x55fce65b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.951+0000 7f4efde16740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce659cc00 0x55fce65b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.951+0000 7f4ef3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce6597a90 0x55fce65b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.951+0000 7f4ef3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce6597a90 0x55fce65b7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36932/0 (socket says 172.21.15.47:36932) 2024-03-21T11:46:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.951+0000 7f4ef9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce659cc00 0x55fce65b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.952+0000 7f4ef9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce659cc00 0x55fce65b8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60818/0 (socket says 172.21.15.47:60818) 2024-03-21T11:46:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.952+0000 7f4ef9200640 1 -- 172.21.15.47:0/1279230189 learned_addr learned my addr 172.21.15.47:0/1279230189 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.952+0000 7f4efde16740 1 -- 172.21.15.47:0/1279230189 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef41a92f0 con 0x55fce65970a0 2024-03-21T11:46:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.952+0000 7f4efde16740 1 -- 172.21.15.47:0/1279230189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef41a8c90 con 0x55fce6597a90 2024-03-21T11:46:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.952+0000 7f4efde16740 1 -- 172.21.15.47:0/1279230189 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef40732c0 con 0x55fce659cc00 2024-03-21T11:46:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.952+0000 7f4efde16740 10 client.0 _ll_register_callbacks cb 0x7f4ee40b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.951+0000 7f4ef3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce65970a0 0x55fce65b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.953+0000 7f4ef3400640 1 -- 172.21.15.47:0/1279230189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce659cc00 msgr2=0x55fce65b8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.953+0000 7f4ef3400640 1 --2- 172.21.15.47:0/1279230189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce659cc00 0x55fce65b8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.953+0000 7f4ef3400640 1 -- 172.21.15.47:0/1279230189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce65970a0 msgr2=0x55fce65b7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.953+0000 7f4ef3400640 1 --2- 172.21.15.47:0/1279230189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce65970a0 0x55fce65b7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.953+0000 7f4ef3400640 1 -- 172.21.15.47:0/1279230189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fce65b9200 con 0x55fce6597a90 2024-03-21T11:46:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.953+0000 7f4ef9200640 1 --2- 172.21.15.47:0/1279230189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce659cc00 0x55fce65b8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.954+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/1279230189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce65970a0 0x55fce65b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.954+0000 7f4ef3400640 1 --2- 172.21.15.47:0/1279230189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce6597a90 0x55fce65b7c70 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x55fce6597e90 tx=0x7f4ee0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d05dbdc0334d9ab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.954+0000 7f4ede000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:48.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.955+0000 7f4ede000640 1 -- 172.21.15.47:0/1279230189 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee002c060 con 0x55fce6597a90 2024-03-21T11:46:48.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.955+0000 7f4ede000640 1 -- 172.21.15.47:0/1279230189 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4ee00059e0 con 0x55fce6597a90 2024-03-21T11:46:48.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.956+0000 7f4efde16740 1 -- 172.21.15.47:0/1279230189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ee409f100 con 0x55fce6597a90 2024-03-21T11:46:48.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.956+0000 7f4ede000640 1 -- 172.21.15.47:0/1279230189 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ee0015e50 con 0x55fce6597a90 2024-03-21T11:46:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.957+0000 7f4efde16740 10 client.33008 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.957+0000 7f4efde16740 1 -- 172.21.15.47:0/1279230189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55fce6619c50 con 0x55fce6597a90 2024-03-21T11:46:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.957+0000 7f4ede000640 1 -- 172.21.15.47:0/1279230189 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ee409f100 con 0x55fce6597a90 2024-03-21T11:46:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.958+0000 7f4ede000640 1 -- 172.21.15.47:0/1279230189 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4ee0012070 con 0x55fce6597a90 2024-03-21T11:46:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.958+0000 7f4efde16740 10 client.33008 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.958+0000 7f4efde16740 10 client.33008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.958+0000 7f4efde16740 20 client.33008 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.958+0000 7f4efde16740 1 --2- 172.21.15.47:0/1279230189 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fce661a500 0x55fce661c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.958+0000 7f4efde16740 4 client.33008 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:46:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.958+0000 7f4efde16740 1 -- 172.21.15.47:0/1279230189 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fce6504740 con 0x55fce661a500 2024-03-21T11:46:48.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.965+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/1279230189 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fce661a500 0x55fce661c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.967+0000 7f4ef3e00640 1 --2- 172.21.15.47:0/1279230189 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fce661a500 0x55fce661c920 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce5ed15aac8d37d5 server_cookie=7c37b55069e4626e in_seq=0 out_seq=0 2024-03-21T11:46:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.967+0000 7f4ede000640 10 client.33008 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:48.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.979+0000 7f4ede000640 1 -- 172.21.15.47:0/1279230189 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fce6504740 con 0x55fce661a500 2024-03-21T11:46:48.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.979+0000 7f4ede000640 10 client.33008 handle_command_reply: tid=0 2024-03-21T11:46:48.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.979+0000 7f4efde16740 1 client.33008 shutdown 2024-03-21T11:46:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.982+0000 7f4efde16740 1 -- 172.21.15.47:0/1279230189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce6597a90 msgr2=0x55fce65b7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.982+0000 7f4efde16740 1 --2- 172.21.15.47:0/1279230189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce6597a90 0x55fce65b7c70 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x55fce6597e90 tx=0x7f4ee0004750 comp rx=0 tx=0).stop 2024-03-21T11:46:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.982+0000 7f4efde16740 1 -- 172.21.15.47:0/1279230189 shutdown_connections 2024-03-21T11:46:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.982+0000 7f4efde16740 1 --2- 172.21.15.47:0/1279230189 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55fce661a500 0x55fce661c920 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.982+0000 7f4efde16740 1 --2- 172.21.15.47:0/1279230189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fce659cc00 0x55fce65b8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.982+0000 7f4efde16740 1 --2- 172.21.15.47:0/1279230189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fce6597a90 0x55fce65b7c70 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.982+0000 7f4efde16740 1 --2- 172.21.15.47:0/1279230189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fce65970a0 0x55fce65b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.982+0000 7f4efde16740 1 -- 172.21.15.47:0/1279230189 >> 172.21.15.47:0/1279230189 conn(0x55fce6592610 msgr2=0x55fce6596100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:48.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.982+0000 7f4ede000640 0 client.33008 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:48.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.982+0000 7f4efde16740 1 -- 172.21.15.47:0/1279230189 shutdown_connections 2024-03-21T11:46:48.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.982+0000 7f4efde16740 1 -- 172.21.15.47:0/1279230189 wait complete. 2024-03-21T11:46:48.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.982+0000 7f4efde16740 20 client.33008 trim_cache size 0 max 16384 2024-03-21T11:46:48.984 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:46:48.984 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:46:48.984 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "c3678c94-5e1e-4688-a26c-648edd393524", 2024-03-21T11:46:48.984 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:46:48.985 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:46:48.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.983+0000 7f4efb200640 1 -- 172.21.15.47:0/1045657114 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4ed80a2fa0 msgr2=0x7f4ed80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:48.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.983+0000 7f4efb200640 1 --2- 172.21.15.47:0/1045657114 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4ed80a2fa0 0x7f4ed80a5460 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4ef419f580 tx=0x7f4ee0006c40 comp rx=0 tx=0).stop 2024-03-21T11:46:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.983+0000 7f4efb200640 1 -- 172.21.15.47:0/1045657114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef4074100 msgr2=0x7f4ef419e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.983+0000 7f4efb200640 1 --2- 172.21.15.47:0/1045657114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef4074100 0x7f4ef419e070 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400d950 tx=0x7f4ee400de10 comp rx=0 tx=0).stop 2024-03-21T11:46:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.983+0000 7f4efb200640 1 -- 172.21.15.47:0/1045657114 shutdown_connections 2024-03-21T11:46:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.983+0000 7f4efb200640 1 --2- 172.21.15.47:0/1045657114 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4ed80a2fa0 0x7f4ed80a5460 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.983+0000 7f4efb200640 1 --2- 172.21.15.47:0/1045657114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4ef410dc80 0x7f4ef41a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.984+0000 7f4efb200640 1 --2- 172.21.15.47:0/1045657114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ef40756e0 0x7f4ef419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.984+0000 7f4efb200640 1 --2- 172.21.15.47:0/1045657114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef4074100 0x7f4ef419e070 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.984+0000 7f4efb200640 1 -- 172.21.15.47:0/1045657114 >> 172.21.15.47:0/1045657114 conn(0x7f4ef40fcc00 msgr2=0x7f4ef4101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.984+0000 7f4efb200640 1 -- 172.21.15.47:0/1045657114 shutdown_connections 2024-03-21T11:46:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:48.984+0000 7f4efb200640 1 -- 172.21.15.47:0/1045657114 wait complete. 2024-03-21T11:46:48.989 INFO:tasks.workunit.client.0.smithi047.stdout: 10 241019 21.80 MB/sec execute 452 sec latency 110.463 ms 2024-03-21T11:46:49.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:46:49.010 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:46:49.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:48 smithi102 ceph-mon[36748]: pgmap v1105: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 456 op/s 2024-03-21T11:46:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.255+0000 7fb2b8e00640 1 -- 172.21.15.47:0/570109769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b41038e0 msgr2=0x7fb2b4103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.255+0000 7fb2b8e00640 1 --2- 172.21.15.47:0/570109769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b41038e0 0x7fb2b4103d00 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009900 tx=0x7fb2a402bc30 comp rx=0 tx=0).stop 2024-03-21T11:46:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.255+0000 7fb2b8e00640 1 -- 172.21.15.47:0/570109769 shutdown_connections 2024-03-21T11:46:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.255+0000 7fb2b8e00640 1 --2- 172.21.15.47:0/570109769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b4105490 0x7fb2b410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.255+0000 7fb2b8e00640 1 --2- 172.21.15.47:0/570109769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b4104ad0 0x7fb2b4104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.255+0000 7fb2b8e00640 1 --2- 172.21.15.47:0/570109769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b41038e0 0x7fb2b4103d00 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.255+0000 7fb2b8e00640 1 -- 172.21.15.47:0/570109769 >> 172.21.15.47:0/570109769 conn(0x7fb2b40fee50 msgr2=0x7fb2b4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.256+0000 7fb2b8e00640 1 -- 172.21.15.47:0/570109769 shutdown_connections 2024-03-21T11:46:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.256+0000 7fb2b8e00640 1 -- 172.21.15.47:0/570109769 wait complete. 2024-03-21T11:46:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.257+0000 7fb2b8e00640 1 Processor -- start 2024-03-21T11:46:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.257+0000 7fb2b8e00640 1 -- start start 2024-03-21T11:46:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.258+0000 7fb2b8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b41038e0 0x7fb2b4079450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.258+0000 7fb2b8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b4104ad0 0x7fb2b4079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.258+0000 7fb2b1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b41038e0 0x7fb2b4079450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.258+0000 7fb2b8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b4105490 0x7fb2b4075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.258+0000 7fb2b1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b41038e0 0x7fb2b4079450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36940/0 (socket says 172.21.15.47:36940) 2024-03-21T11:46:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.259+0000 7fb2b1e00640 1 -- 172.21.15.47:0/387771815 learned_addr learned my addr 172.21.15.47:0/387771815 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.259+0000 7fb2b8e00640 1 -- 172.21.15.47:0/387771815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b410f580 con 0x7fb2b4105490 2024-03-21T11:46:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.259+0000 7fb2b8e00640 1 -- 172.21.15.47:0/387771815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b410f400 con 0x7fb2b41038e0 2024-03-21T11:46:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.259+0000 7fb2b8e00640 1 -- 172.21.15.47:0/387771815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b410f700 con 0x7fb2b4104ad0 2024-03-21T11:46:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.259+0000 7fb2b1400640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b4104ad0 0x7fb2b4079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.259+0000 7fb2b2800640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b4105490 0x7fb2b4075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.259+0000 7fb2b1400640 1 -- 172.21.15.47:0/387771815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b41038e0 msgr2=0x7fb2b4079450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.259+0000 7fb2b1400640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b41038e0 0x7fb2b4079450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.259+0000 7fb2b1400640 1 -- 172.21.15.47:0/387771815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b4105490 msgr2=0x7fb2b4075fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.259+0000 7fb2b1400640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b4105490 0x7fb2b4075fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.259+0000 7fb2b1400640 1 -- 172.21.15.47:0/387771815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b4076880 con 0x7fb2b4104ad0 2024-03-21T11:46:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.259+0000 7fb2b2800640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b4105490 0x7fb2b4075fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.260+0000 7fb2b1400640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b4104ad0 0x7fb2b4079990 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000d6d0 tx=0x7fb2a000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.260+0000 7fb2aaa00640 1 -- 172.21.15.47:0/387771815 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0017070 con 0x7fb2b4104ad0 2024-03-21T11:46:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.260+0000 7fb2aaa00640 1 -- 172.21.15.47:0/387771815 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2a0009820 con 0x7fb2b4104ad0 2024-03-21T11:46:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.260+0000 7fb2aaa00640 1 -- 172.21.15.47:0/387771815 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0004d00 con 0x7fb2b4104ad0 2024-03-21T11:46:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.260+0000 7fb2b8e00640 1 -- 172.21.15.47:0/387771815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b41a6070 con 0x7fb2b4104ad0 2024-03-21T11:46:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.262+0000 7fb2b8e00640 1 -- 172.21.15.47:0/387771815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b41a64d0 con 0x7fb2b4104ad0 2024-03-21T11:46:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.264+0000 7fb2aaa00640 1 -- 172.21.15.47:0/387771815 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb2a0005430 con 0x7fb2b4104ad0 2024-03-21T11:46:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.265+0000 7fb2bbbc9740 1 Processor -- start 2024-03-21T11:46:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.266+0000 7fb2bbbc9740 1 -- start start 2024-03-21T11:46:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.266+0000 7fb2bbbc9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.266+0000 7fb2bbbc9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717a9870 0x556f717ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.266+0000 7fb2bbbc9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717ae9e0 0x556f717b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.266+0000 7fb2bbbc9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7170e250 con 0x556f717a9870 2024-03-21T11:46:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.266+0000 7fb2bbbc9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f716b6a30 con 0x556f717ae9e0 2024-03-21T11:46:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.266+0000 7fb2bbbc9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7170a2f0 con 0x556f717a8e80 2024-03-21T11:46:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.267+0000 7fb2b2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.267+0000 7fb2b1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717ae9e0 0x556f717b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.267+0000 7fb2b2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717a92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60834/0 (socket says 172.21.15.47:60834) 2024-03-21T11:46:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.267+0000 7fb2b2800640 1 -- 172.21.15.47:0/46826393 learned_addr learned my addr 172.21.15.47:0/46826393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.267+0000 7fb2b1e00640 1 --2- 172.21.15.47:0/46826393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717a9870 0x556f717ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.267+0000 7fb2aaa00640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2980a2fa0 0x7fb2980a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.267+0000 7fb2b2800640 1 -- 172.21.15.47:0/46826393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717ae9e0 msgr2=0x556f717b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.267+0000 7fb2b2800640 1 --2- 172.21.15.47:0/46826393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717ae9e0 0x556f717b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.267+0000 7fb2b2800640 1 -- 172.21.15.47:0/46826393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717a9870 msgr2=0x556f717ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.267+0000 7fb2b2800640 1 --2- 172.21.15.47:0/46826393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717a9870 0x556f717ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.267+0000 7fb2b2800640 1 -- 172.21.15.47:0/46826393 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f716a1690 con 0x556f717a8e80 2024-03-21T11:46:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.267+0000 7fb2aaa00640 1 -- 172.21.15.47:0/387771815 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb2a00cefb0 con 0x7fb2b4104ad0 2024-03-21T11:46:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.267+0000 7fb2b1e00640 1 --2- 172.21.15.47:0/46826393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717a9870 0x556f717ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.267+0000 7fb2b2800640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2980a2fa0 0x7fb2980a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.268+0000 7fb2b2800640 1 --2- 172.21.15.47:0/46826393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717a92a0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00f4b0 tx=0x7fb2ac00f970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdbd63d562d87fbe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.268+0000 7fb29f400640 1 -- 172.21.15.47:0/46826393 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac018070 con 0x556f717a8e80 2024-03-21T11:46:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.268+0000 7fb29f400640 1 -- 172.21.15.47:0/46826393 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2ac021070 con 0x556f717a8e80 2024-03-21T11:46:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.268+0000 7fb29f400640 1 -- 172.21.15.47:0/46826393 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac010a00 con 0x556f717a8e80 2024-03-21T11:46:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.268+0000 7fb2b2800640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2980a2fa0 0x7fb2980a5460 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac020000 tx=0x7fb2ac01e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.269+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/46826393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 msgr2=0x556f717a92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.269+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/46826393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717a92a0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00f4b0 tx=0x7fb2ac00f970 comp rx=0 tx=0).stop 2024-03-21T11:46:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.269+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/46826393 shutdown_connections 2024-03-21T11:46:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.269+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/46826393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717ae9e0 0x556f717b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.269+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/46826393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717a9870 0x556f717ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.269+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/46826393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717a92a0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.269+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/46826393 >> 172.21.15.47:0/46826393 conn(0x556f71771200 msgr2=0x556f717a47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.269+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/46826393 shutdown_connections 2024-03-21T11:46:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.269+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/46826393 wait complete. 2024-03-21T11:46:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.269+0000 7fb2bbbc9740 1 -- start start 2024-03-21T11:46:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.270+0000 7fb2bbbc9740 1 Processor -- start 2024-03-21T11:46:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.270+0000 7fb2bbbc9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.270+0000 7fb2bbbc9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a9870 0x556f717b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.270+0000 7fb2bbbc9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 0x556f717ba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.270+0000 7fb2bbbc9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f717b1990 con 0x556f717ae9e0 2024-03-21T11:46:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.270+0000 7fb2bbbc9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f716c9e80 con 0x556f717a9870 2024-03-21T11:46:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.270+0000 7fb2bbbc9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f717b1b10 con 0x556f717a8e80 2024-03-21T11:46:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.270+0000 7fb2bbbc9740 10 client.0 _ll_register_callbacks cb 0x556f716dc130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.270+0000 7fb2b1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a9870 0x556f717b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.270+0000 7fb2b1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a9870 0x556f717b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36948/0 (socket says 172.21.15.47:36948) 2024-03-21T11:46:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.270+0000 7fb2b1400640 1 -- 172.21.15.47:0/411289814 learned_addr learned my addr 172.21.15.47:0/411289814 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.270+0000 7fb2b2800640 1 --2- 172.21.15.47:0/411289814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 0x556f717ba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.271+0000 7fb2b1e00640 1 --2- 172.21.15.47:0/411289814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.271+0000 7fb2b1400640 1 -- 172.21.15.47:0/411289814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 msgr2=0x556f717b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.271+0000 7fb2b1400640 1 --2- 172.21.15.47:0/411289814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.271+0000 7fb2b1400640 1 -- 172.21.15.47:0/411289814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 msgr2=0x556f717ba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.271+0000 7fb2b1400640 1 --2- 172.21.15.47:0/411289814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 0x556f717ba3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.271+0000 7fb2b1400640 1 -- 172.21.15.47:0/411289814 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f714ec7d0 con 0x556f717a9870 2024-03-21T11:46:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.271+0000 7fb2b1e00640 1 --2- 172.21.15.47:0/411289814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.271+0000 7fb2b1400640 1 --2- 172.21.15.47:0/411289814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a9870 0x556f717b3390 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000d6a0 tx=0x7fb2a00d25e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea1358d0100dde5d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.271+0000 7fb293e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:46:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.271+0000 7fb293e00640 1 -- 172.21.15.47:0/411289814 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0020040 con 0x556f717a9870 2024-03-21T11:46:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.271+0000 7fb293e00640 1 -- 172.21.15.47:0/411289814 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb2a009f0d0 con 0x556f717a9870 2024-03-21T11:46:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.271+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/411289814 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2ac021070 con 0x556f717a9870 2024-03-21T11:46:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.272+0000 7fb293e00640 1 -- 172.21.15.47:0/411289814 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2a009b9f0 con 0x556f717a9870 2024-03-21T11:46:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.272+0000 7fb2bbbc9740 10 client.33020 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.272+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/411289814 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556f714223f0 con 0x556f717a9870 2024-03-21T11:46:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.272+0000 7fb293e00640 1 -- 172.21.15.47:0/411289814 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2ac021070 con 0x556f717a9870 2024-03-21T11:46:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.272+0000 7fb293e00640 1 -- 172.21.15.47:0/411289814 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb2a0023060 con 0x556f717a9870 2024-03-21T11:46:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.272+0000 7fb2bbbc9740 10 client.33020 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.272+0000 7fb2bbbc9740 10 client.33020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.272+0000 7fb2bbbc9740 20 client.33020 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.272+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/411289814 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f717bec30 0x556f717c1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.272+0000 7fb2bbbc9740 4 client.33020 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.272+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/411289814 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f7171b790 con 0x556f717bec30 2024-03-21T11:46:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.272+0000 7fb2b1e00640 1 --2- 172.21.15.47:0/411289814 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f717bec30 0x556f717c1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.273+0000 7fb2b1e00640 1 --2- 172.21.15.47:0/411289814 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f717bec30 0x556f717c1060 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a68a8d8f0bd5e8 server_cookie=953d7eb81ba12dd0 in_seq=0 out_seq=0 2024-03-21T11:46:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.273+0000 7fb293e00640 10 client.33020 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.277+0000 7fb293e00640 1 -- 172.21.15.47:0/411289814 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556f7171b790 con 0x556f717bec30 2024-03-21T11:46:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.277+0000 7fb293e00640 10 client.33020 handle_command_reply: tid=0 2024-03-21T11:46:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.277+0000 7fb2bbbc9740 1 client.33020 shutdown 2024-03-21T11:46:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.277+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/411289814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a9870 msgr2=0x556f717b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.277+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/411289814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a9870 0x556f717b3390 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000d6a0 tx=0x7fb2a00d25e0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.277+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/411289814 shutdown_connections 2024-03-21T11:46:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.277+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/411289814 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f717bec30 0x556f717c1060 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.277+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/411289814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 0x556f717ba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.277+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/411289814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a9870 0x556f717b3390 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.277+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/411289814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.277+0000 7fb293e00640 0 client.33020 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.277+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/411289814 >> 172.21.15.47:0/411289814 conn(0x556f71771200 msgr2=0x556f717a3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.277+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/411289814 shutdown_connections 2024-03-21T11:46:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.278+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/411289814 wait complete. 2024-03-21T11:46:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.278+0000 7fb2bbbc9740 20 client.33020 trim_cache size 0 max 16384 2024-03-21T11:46:49.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.300+0000 7fb2bbbc9740 1 Processor -- start 2024-03-21T11:46:49.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.300+0000 7fb2bbbc9740 1 -- start start 2024-03-21T11:46:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.301+0000 7fb2bbbc9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717a8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.301+0000 7fb2bbbc9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a9870 0x556f717ba390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.301+0000 7fb2bbbc9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 0x556f717aa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.301+0000 7fb2bbbc9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b41b0790 con 0x556f717ae9e0 2024-03-21T11:46:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.301+0000 7fb2bbbc9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b41b2110 con 0x556f717a9870 2024-03-21T11:46:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.301+0000 7fb2bbbc9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b41b0df0 con 0x556f717a8e80 2024-03-21T11:46:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.301+0000 7fb2b1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a9870 0x556f717ba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.301+0000 7fb2b1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a9870 0x556f717ba390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36952/0 (socket says 172.21.15.47:36952) 2024-03-21T11:46:49.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.301+0000 7fb2b2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 0x556f717aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.301+0000 7fb2b1400640 1 -- 172.21.15.47:0/1813616047 learned_addr learned my addr 172.21.15.47:0/1813616047 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.301+0000 7fb2b1e00640 1 --2- 172.21.15.47:0/1813616047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717a8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.302+0000 7fb2b1e00640 1 -- 172.21.15.47:0/1813616047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a9870 msgr2=0x556f717ba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.302+0000 7fb2b1e00640 1 --2- 172.21.15.47:0/1813616047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a9870 0x556f717ba390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.302+0000 7fb2b1e00640 1 -- 172.21.15.47:0/1813616047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 msgr2=0x556f717aa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:49.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.302+0000 7fb2b1e00640 1 --2- 172.21.15.47:0/1813616047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 0x556f717aa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.302+0000 7fb2b1e00640 1 -- 172.21.15.47:0/1813616047 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f713e20f0 con 0x556f717a8e80 2024-03-21T11:46:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.302+0000 7fb2b1e00640 1 --2- 172.21.15.47:0/1813616047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717a8140 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4000c00 tx=0x7fb2a4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90a8930613f8f1a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.302+0000 7fb29ea00640 1 -- 172.21.15.47:0/1813616047 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a402cb60 con 0x556f717a8e80 2024-03-21T11:46:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.302+0000 7fb29ea00640 1 -- 172.21.15.47:0/1813616047 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2a402cd00 con 0x556f717a8e80 2024-03-21T11:46:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.303+0000 7fb29ea00640 1 -- 172.21.15.47:0/1813616047 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4005150 con 0x556f717a8e80 2024-03-21T11:46:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.303+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/1813616047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 msgr2=0x556f717a8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.303+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/1813616047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717a8140 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4000c00 tx=0x7fb2a4009260 comp rx=0 tx=0).stop 2024-03-21T11:46:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.304+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/1813616047 shutdown_connections 2024-03-21T11:46:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.304+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/1813616047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 0x556f717aa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.304+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/1813616047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a9870 0x556f717ba390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.304+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/1813616047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a8e80 0x556f717a8140 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.304+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/1813616047 >> 172.21.15.47:0/1813616047 conn(0x556f71771200 msgr2=0x556f717a7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.305+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/1813616047 shutdown_connections 2024-03-21T11:46:49.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.305+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/1813616047 wait complete. 2024-03-21T11:46:49.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.305+0000 7fb2bbbc9740 1 -- start start 2024-03-21T11:46:49.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.306+0000 7fb2bbbc9740 1 Processor -- start 2024-03-21T11:46:49.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.306+0000 7fb2bbbc9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a8e80 0x556f717b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.306+0000 7fb2bbbc9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a9870 0x556f717b8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.306+0000 7fb2bbbc9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 0x556f717b9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.306+0000 7fb2bbbc9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b41b1ab0 con 0x556f717ae9e0 2024-03-21T11:46:49.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.306+0000 7fb2bbbc9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b41b1450 con 0x556f717a8e80 2024-03-21T11:46:49.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.306+0000 7fb2bbbc9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b41ae950 con 0x556f717a9870 2024-03-21T11:46:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.306+0000 7fb2bbbc9740 10 client.0 _ll_register_callbacks cb 0x7fb2a00057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.306+0000 7fb2b1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a9870 0x556f717b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.306+0000 7fb2b1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a8e80 0x556f717b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.306+0000 7fb2b1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a9870 0x556f717b8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60866/0 (socket says 172.21.15.47:60866) 2024-03-21T11:46:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.306+0000 7fb2b1400640 1 -- 172.21.15.47:0/4194127945 learned_addr learned my addr 172.21.15.47:0/4194127945 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.307+0000 7fb2b2800640 1 --2- 172.21.15.47:0/4194127945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 0x556f717b9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.307+0000 7fb2b1400640 1 -- 172.21.15.47:0/4194127945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a8e80 msgr2=0x556f717b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.307+0000 7fb2b1400640 1 --2- 172.21.15.47:0/4194127945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a8e80 0x556f717b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.307+0000 7fb2b1400640 1 -- 172.21.15.47:0/4194127945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 msgr2=0x556f717b9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.307+0000 7fb2b1400640 1 --2- 172.21.15.47:0/4194127945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 0x556f717b9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.307+0000 7fb2b1400640 1 -- 172.21.15.47:0/4194127945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f717b9b60 con 0x556f717a9870 2024-03-21T11:46:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.307+0000 7fb2b2800640 1 --2- 172.21.15.47:0/4194127945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 0x556f717b9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:46:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.307+0000 7fb2b1400640 1 --2- 172.21.15.47:0/4194127945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a9870 0x556f717b8560 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000d6a0 tx=0x7fb2a009aea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4de5514adce4372e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.307+0000 7fb293e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.308+0000 7fb293e00640 1 -- 172.21.15.47:0/4194127945 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a00a3d00 con 0x556f717a9870 2024-03-21T11:46:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.308+0000 7fb293e00640 1 -- 172.21.15.47:0/4194127945 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb2a00d20b0 con 0x556f717a9870 2024-03-21T11:46:49.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.308+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/4194127945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2a402cd00 con 0x556f717a9870 2024-03-21T11:46:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.308+0000 7fb293e00640 1 -- 172.21.15.47:0/4194127945 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2a009b2f0 con 0x556f717a9870 2024-03-21T11:46:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.308+0000 7fb2bbbc9740 10 client.32995 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.308+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/4194127945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556f7181ab00 con 0x556f717a9870 2024-03-21T11:46:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.308+0000 7fb293e00640 1 -- 172.21.15.47:0/4194127945 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2a00d22b0 con 0x556f717a9870 2024-03-21T11:46:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.308+0000 7fb293e00640 1 -- 172.21.15.47:0/4194127945 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb2a009a740 con 0x556f717a9870 2024-03-21T11:46:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.309+0000 7fb2bbbc9740 10 client.32995 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.309+0000 7fb2bbbc9740 10 client.32995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.309+0000 7fb2bbbc9740 20 client.32995 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.309+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/4194127945 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f7181b230 0x556f7181d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.309+0000 7fb2bbbc9740 4 client.32995 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:46:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.309+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/4194127945 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fb2a402cd00 con 0x556f7181b230 2024-03-21T11:46:49.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.309+0000 7fb2b1e00640 1 --2- 172.21.15.47:0/4194127945 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f7181b230 0x556f7181d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:49.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.310+0000 7fb2b1e00640 1 --2- 172.21.15.47:0/4194127945 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f7181b230 0x556f7181d650 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf20d493ac485216 server_cookie=eb132d68c3cb5cf8 in_seq=0 out_seq=0 2024-03-21T11:46:49.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.310+0000 7fb293e00640 10 client.32995 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:49.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:48 smithi047 ceph-mon[31351]: pgmap v1105: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 456 op/s 2024-03-21T11:46:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.315+0000 7fb293e00640 1 -- 172.21.15.47:0/4194127945 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x7fb2a402cd00 con 0x556f7181b230 2024-03-21T11:46:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.315+0000 7fb293e00640 10 client.32995 handle_command_reply: tid=0 2024-03-21T11:46:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.315+0000 7fb2bbbc9740 1 client.32995 shutdown 2024-03-21T11:46:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.315+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/4194127945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a9870 msgr2=0x556f717b8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.315+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/4194127945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a9870 0x556f717b8560 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000d6a0 tx=0x7fb2a009aea0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.315+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/4194127945 shutdown_connections 2024-03-21T11:46:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.315+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/4194127945 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f7181b230 0x556f7181d650 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.315+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/4194127945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f717ae9e0 0x556f717b9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.315+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/4194127945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f717a9870 0x556f717b8560 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.315+0000 7fb2bbbc9740 1 --2- 172.21.15.47:0/4194127945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f717a8e80 0x556f717b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.315+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/4194127945 >> 172.21.15.47:0/4194127945 conn(0x556f71771200 msgr2=0x556f717a82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:49.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.315+0000 7fb293e00640 0 client.32995 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:49.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.316+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/4194127945 shutdown_connections 2024-03-21T11:46:49.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.316+0000 7fb2bbbc9740 1 -- 172.21.15.47:0/4194127945 wait complete. 2024-03-21T11:46:49.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.316+0000 7fb2bbbc9740 20 client.32995 trim_cache size 0 max 16384 2024-03-21T11:46:49.318 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:46:49.318 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (32 inodes in the stack)", 2024-03-21T11:46:49.318 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:46:49.318 INFO:teuthology.orchestra.run.smithi047.stdout: "c3678c94-5e1e-4688-a26c-648edd393524": { 2024-03-21T11:46:49.318 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:46:49.318 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "c3678c94-5e1e-4688-a26c-648edd393524", 2024-03-21T11:46:49.318 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:46:49.318 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:46:49.319 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:46:49.319 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:46:49.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.317+0000 7fb2b8e00640 1 -- 172.21.15.47:0/387771815 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2980a2fa0 msgr2=0x7fb2980a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:49.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.317+0000 7fb2b8e00640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2980a2fa0 0x7fb2980a5460 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac020000 tx=0x7fb2ac01e000 comp rx=0 tx=0).stop 2024-03-21T11:46:49.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.317+0000 7fb2b8e00640 1 -- 172.21.15.47:0/387771815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b4104ad0 msgr2=0x7fb2b4079990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.317+0000 7fb2b8e00640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b4104ad0 0x7fb2b4079990 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000d6d0 tx=0x7fb2a000db90 comp rx=0 tx=0).stop 2024-03-21T11:46:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.317+0000 7fb2b8e00640 1 -- 172.21.15.47:0/387771815 shutdown_connections 2024-03-21T11:46:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.317+0000 7fb2b8e00640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb2980a2fa0 0x7fb2980a5460 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.317+0000 7fb2b8e00640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2b4105490 0x7fb2b4075fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.317+0000 7fb2b8e00640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b4104ad0 0x7fb2b4079990 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.317+0000 7fb2b8e00640 1 --2- 172.21.15.47:0/387771815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2b41038e0 0x7fb2b4079450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.317+0000 7fb2b8e00640 1 -- 172.21.15.47:0/387771815 >> 172.21.15.47:0/387771815 conn(0x7fb2b40fee50 msgr2=0x7fb2b4107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.317+0000 7fb2b8e00640 1 -- 172.21.15.47:0/387771815 shutdown_connections 2024-03-21T11:46:49.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:46:49.317+0000 7fb2b8e00640 1 -- 172.21.15.47:0/387771815 wait complete. 2024-03-21T11:46:49.336 INFO:tasks.cephfs.filesystem:scrub status for tag:c3678c94-5e1e-4688-a26c-648edd393524 - {'path': '/', 'tag': 'c3678c94-5e1e-4688-a26c-648edd393524', 'options': 'recursive,force'} 2024-03-21T11:46:49.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:48 smithi100 ceph-mon[36245]: pgmap v1105: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.1 MiB/s wr, 456 op/s 2024-03-21T11:46:49.990 INFO:tasks.workunit.client.0.smithi047.stdout: 10 241427 21.78 MB/sec execute 453 sec latency 75.805 ms 2024-03-21T11:46:50.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:46:50.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:50.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:50.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:50.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:50.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:50.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:50.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:50.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:46:50.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:50 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:46:50.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:50.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:50.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:50.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:50.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:50.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:50.571 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.571 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:50.571 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.571 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:46:50.571 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.571 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.571 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:50 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:46:50.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:50.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:50.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:50.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:50.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:50.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:50.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:50.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:46:50.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:50 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:46:50.990 INFO:tasks.workunit.client.0.smithi047.stdout: 10 241859 21.79 MB/sec execute 454 sec latency 93.724 ms 2024-03-21T11:46:51.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:51 smithi102 ceph-mon[36748]: pgmap v1106: 129 pgs: 129 active+clean; 686 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.6 MiB/s wr, 492 op/s 2024-03-21T11:46:51.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:51 smithi047 ceph-mon[31351]: pgmap v1106: 129 pgs: 129 active+clean; 686 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.6 MiB/s wr, 492 op/s 2024-03-21T11:46:51.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:51 smithi100 ceph-mon[36245]: pgmap v1106: 129 pgs: 129 active+clean; 686 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.6 MiB/s wr, 492 op/s 2024-03-21T11:46:51.990 INFO:tasks.workunit.client.0.smithi047.stdout: 10 242285 21.79 MB/sec execute 455 sec latency 81.710 ms 2024-03-21T11:46:52.990 INFO:tasks.workunit.client.0.smithi047.stdout: 10 242770 21.80 MB/sec execute 456 sec latency 113.732 ms 2024-03-21T11:46:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:53 smithi102 ceph-mon[36748]: pgmap v1107: 129 pgs: 129 active+clean; 690 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.6 MiB/s wr, 528 op/s 2024-03-21T11:46:53.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:53 smithi047 ceph-mon[31351]: pgmap v1107: 129 pgs: 129 active+clean; 690 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.6 MiB/s wr, 528 op/s 2024-03-21T11:46:53.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:53 smithi100 ceph-mon[36245]: pgmap v1107: 129 pgs: 129 active+clean; 690 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.6 MiB/s wr, 528 op/s 2024-03-21T11:46:53.990 INFO:tasks.workunit.client.0.smithi047.stdout: 10 243175 21.79 MB/sec execute 457 sec latency 94.576 ms 2024-03-21T11:46:54.990 INFO:tasks.workunit.client.0.smithi047.stdout: 10 243760 21.80 MB/sec execute 458 sec latency 85.841 ms 2024-03-21T11:46:55.444 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:55 smithi100 ceph-mon[36245]: pgmap v1108: 129 pgs: 129 active+clean; 692 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.3 MiB/s wr, 480 op/s 2024-03-21T11:46:55.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:55 smithi102 ceph-mon[36748]: pgmap v1108: 129 pgs: 129 active+clean; 692 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.3 MiB/s wr, 480 op/s 2024-03-21T11:46:55.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:55 smithi047 ceph-mon[31351]: pgmap v1108: 129 pgs: 129 active+clean; 692 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.3 MiB/s wr, 480 op/s 2024-03-21T11:46:55.990 INFO:tasks.workunit.client.0.smithi047.stdout: 10 244354 21.84 MB/sec execute 459 sec latency 73.377 ms 2024-03-21T11:46:56.990 INFO:tasks.workunit.client.0.smithi047.stdout: 10 244943 21.86 MB/sec execute 460 sec latency 60.701 ms 2024-03-21T11:46:57.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:57 smithi102 ceph-mon[36748]: pgmap v1109: 129 pgs: 129 active+clean; 697 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.2 MiB/s wr, 498 op/s 2024-03-21T11:46:57.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:57 smithi047 ceph-mon[31351]: pgmap v1109: 129 pgs: 129 active+clean; 697 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.2 MiB/s wr, 498 op/s 2024-03-21T11:46:57.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:57 smithi100 ceph-mon[36245]: pgmap v1109: 129 pgs: 129 active+clean; 697 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.2 MiB/s wr, 498 op/s 2024-03-21T11:46:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.850+0000 7fdc50c00640 1 -- 172.21.15.47:0/672420297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc4c1016d0 msgr2=0x7fdc4c101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.850+0000 7fdc50c00640 1 --2- 172.21.15.47:0/672420297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc4c1016d0 0x7fdc4c101af0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009900 tx=0x7fdc3c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:46:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.851+0000 7fdc50c00640 1 -- 172.21.15.47:0/672420297 shutdown_connections 2024-03-21T11:46:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.851+0000 7fdc50c00640 1 --2- 172.21.15.47:0/672420297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc4c103280 0x7fdc4c109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.851+0000 7fdc50c00640 1 --2- 172.21.15.47:0/672420297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc4c1028c0 0x7fdc4c102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.851+0000 7fdc50c00640 1 --2- 172.21.15.47:0/672420297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc4c1016d0 0x7fdc4c101af0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.851+0000 7fdc50c00640 1 -- 172.21.15.47:0/672420297 >> 172.21.15.47:0/672420297 conn(0x7fdc4c0fcc60 msgr2=0x7fdc4c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.851+0000 7fdc50c00640 1 -- 172.21.15.47:0/672420297 shutdown_connections 2024-03-21T11:46:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.851+0000 7fdc50c00640 1 -- 172.21.15.47:0/672420297 wait complete. 2024-03-21T11:46:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.852+0000 7fdc50c00640 1 Processor -- start 2024-03-21T11:46:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.853+0000 7fdc50c00640 1 -- start start 2024-03-21T11:46:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.853+0000 7fdc50c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc4c1016d0 0x7fdc4c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.853+0000 7fdc50c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc4c1028c0 0x7fdc4c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.853+0000 7fdc50c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc4c103280 0x7fdc4c1a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.853+0000 7fdc50c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c10d3a0 con 0x7fdc4c1028c0 2024-03-21T11:46:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.853+0000 7fdc50c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c10d220 con 0x7fdc4c103280 2024-03-21T11:46:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.853+0000 7fdc50c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c10d520 con 0x7fdc4c1016d0 2024-03-21T11:46:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.854+0000 7fdc4a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc4c103280 0x7fdc4c1a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.854+0000 7fdc4a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc4c103280 0x7fdc4c1a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36966/0 (socket says 172.21.15.47:36966) 2024-03-21T11:46:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.854+0000 7fdc49e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc4c1016d0 0x7fdc4c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.854+0000 7fdc4a800640 1 -- 172.21.15.47:0/3407391750 learned_addr learned my addr 172.21.15.47:0/3407391750 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.854+0000 7fdc49400640 1 --2- 172.21.15.47:0/3407391750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc4c1028c0 0x7fdc4c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.855+0000 7fdc49e00640 1 -- 172.21.15.47:0/3407391750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc4c103280 msgr2=0x7fdc4c1a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.855+0000 7fdc49e00640 1 --2- 172.21.15.47:0/3407391750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc4c103280 0x7fdc4c1a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.855+0000 7fdc49e00640 1 -- 172.21.15.47:0/3407391750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc4c1028c0 msgr2=0x7fdc4c19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.855+0000 7fdc49e00640 1 --2- 172.21.15.47:0/3407391750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc4c1028c0 0x7fdc4c19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.855+0000 7fdc49e00640 1 -- 172.21.15.47:0/3407391750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4c1a1790 con 0x7fdc4c1016d0 2024-03-21T11:46:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.855+0000 7fdc49e00640 1 --2- 172.21.15.47:0/3407391750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc4c1016d0 0x7fdc4c199b70 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009230 tx=0x7fdc3c005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.856+0000 7fdc42a00640 1 -- 172.21.15.47:0/3407391750 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c043020 con 0x7fdc4c1016d0 2024-03-21T11:46:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.856+0000 7fdc42a00640 1 -- 172.21.15.47:0/3407391750 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc3c0054e0 con 0x7fdc4c1016d0 2024-03-21T11:46:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.856+0000 7fdc42a00640 1 -- 172.21.15.47:0/3407391750 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c003e60 con 0x7fdc4c1016d0 2024-03-21T11:46:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.859+0000 7fdc50c00640 1 -- 172.21.15.47:0/3407391750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4c1a1a20 con 0x7fdc4c1016d0 2024-03-21T11:46:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.859+0000 7fdc50c00640 1 -- 172.21.15.47:0/3407391750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4c1a1ef0 con 0x7fdc4c1016d0 2024-03-21T11:46:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.864+0000 7fdc42a00640 1 -- 172.21.15.47:0/3407391750 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdc3c046020 con 0x7fdc4c1016d0 2024-03-21T11:46:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.864+0000 7fdc539b2740 1 Processor -- start 2024-03-21T11:46:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.865+0000 7fdc539b2740 1 -- start start 2024-03-21T11:46:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.865+0000 7fdc539b2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089adee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.865+0000 7fdc539b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 0x55a089ae3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.865+0000 7fdc539b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089ae69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.865+0000 7fdc539b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a089a993b0 con 0x55a089adf3e0 2024-03-21T11:46:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.865+0000 7fdc539b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a089a3b830 con 0x55a089ae4550 2024-03-21T11:46:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.865+0000 7fdc539b2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0899e45d0 con 0x55a089ade9f0 2024-03-21T11:46:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.866+0000 7fdc49400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089ae69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.866+0000 7fdc49400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089ae69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36970/0 (socket says 172.21.15.47:36970) 2024-03-21T11:46:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.866+0000 7fdc49400640 1 -- 172.21.15.47:0/2999065565 learned_addr learned my addr 172.21.15.47:0/2999065565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.866+0000 7fdc49e00640 1 --2- 172.21.15.47:0/2999065565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 0x55a089ae3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.867+0000 7fdc42a00640 1 --2- 172.21.15.47:0/3407391750 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc300a30c0 0x7fdc300a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.867+0000 7fdc4a800640 1 --2- 172.21.15.47:0/2999065565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089adee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.868+0000 7fdc42a00640 1 -- 172.21.15.47:0/3407391750 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdc3c0f1580 con 0x7fdc4c1016d0 2024-03-21T11:46:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.868+0000 7fdc4a800640 1 --2- 172.21.15.47:0/3407391750 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc300a30c0 0x7fdc300a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.868+0000 7fdc49400640 1 -- 172.21.15.47:0/2999065565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 msgr2=0x55a089adee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.868+0000 7fdc49400640 1 --2- 172.21.15.47:0/2999065565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089adee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.868+0000 7fdc49400640 1 -- 172.21.15.47:0/2999065565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 msgr2=0x55a089ae3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.868+0000 7fdc49400640 1 --2- 172.21.15.47:0/2999065565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 0x55a089ae3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.868+0000 7fdc49400640 1 -- 172.21.15.47:0/2999065565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0899fc2c0 con 0x55a089ae4550 2024-03-21T11:46:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.869+0000 7fdc49e00640 1 --2- 172.21.15.47:0/2999065565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 0x55a089ae3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.869+0000 7fdc49400640 1 --2- 172.21.15.47:0/2999065565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089ae69e0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800bd10 tx=0x7fdc380107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b5d3823fce417 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.870+0000 7fdc37000640 1 -- 172.21.15.47:0/2999065565 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38005220 con 0x55a089ae4550 2024-03-21T11:46:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.870+0000 7fdc37000640 1 -- 172.21.15.47:0/2999065565 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc38004c80 con 0x55a089ae4550 2024-03-21T11:46:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.870+0000 7fdc37000640 1 -- 172.21.15.47:0/2999065565 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc380118a0 con 0x55a089ae4550 2024-03-21T11:46:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.871+0000 7fdc539b2740 1 -- 172.21.15.47:0/2999065565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 msgr2=0x55a089ae69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.871+0000 7fdc539b2740 1 --2- 172.21.15.47:0/2999065565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089ae69e0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800bd10 tx=0x7fdc380107a0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.871+0000 7fdc539b2740 1 -- 172.21.15.47:0/2999065565 shutdown_connections 2024-03-21T11:46:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.871+0000 7fdc539b2740 1 --2- 172.21.15.47:0/2999065565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089ae69e0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.871+0000 7fdc539b2740 1 --2- 172.21.15.47:0/2999065565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 0x55a089ae3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.871+0000 7fdc539b2740 1 --2- 172.21.15.47:0/2999065565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089adee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.871+0000 7fdc539b2740 1 -- 172.21.15.47:0/2999065565 >> 172.21.15.47:0/2999065565 conn(0x55a089ad9f60 msgr2=0x55a089ada360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.871+0000 7fdc4a800640 1 --2- 172.21.15.47:0/3407391750 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc300a30c0 0x7fdc300a5580 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x55a089adfc40 tx=0x7fdc44004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.871+0000 7fdc539b2740 1 -- 172.21.15.47:0/2999065565 shutdown_connections 2024-03-21T11:46:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.871+0000 7fdc539b2740 1 -- 172.21.15.47:0/2999065565 wait complete. 2024-03-21T11:46:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.872+0000 7fdc539b2740 1 -- start start 2024-03-21T11:46:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.872+0000 7fdc539b2740 1 Processor -- start 2024-03-21T11:46:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.872+0000 7fdc539b2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089ae8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.872+0000 7fdc539b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 0x55a089ae90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.872+0000 7fdc539b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089af0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.872+0000 7fdc539b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0899f7a20 con 0x55a089adf3e0 2024-03-21T11:46:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.872+0000 7fdc539b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a089a378d0 con 0x55a089ae4550 2024-03-21T11:46:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.872+0000 7fdc539b2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a089ae7500 con 0x55a089ade9f0 2024-03-21T11:46:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.873+0000 7fdc539b2740 10 client.0 _ll_register_callbacks cb 0x55a089a341c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.873+0000 7fdc49e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089ae8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.873+0000 7fdc49e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089ae8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60916/0 (socket says 172.21.15.47:60916) 2024-03-21T11:46:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.873+0000 7fdc49e00640 1 -- 172.21.15.47:0/411832674 learned_addr learned my addr 172.21.15.47:0/411832674 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.873+0000 7fdc4a800640 1 --2- 172.21.15.47:0/411832674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089af0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.873+0000 7fdc49400640 1 --2- 172.21.15.47:0/411832674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 0x55a089ae90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.873+0000 7fdc49e00640 1 -- 172.21.15.47:0/411832674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 msgr2=0x55a089af0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.873+0000 7fdc49e00640 1 --2- 172.21.15.47:0/411832674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089af0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.873+0000 7fdc49e00640 1 -- 172.21.15.47:0/411832674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 msgr2=0x55a089ae90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.873+0000 7fdc49e00640 1 --2- 172.21.15.47:0/411832674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 0x55a089ae90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.873+0000 7fdc49e00640 1 -- 172.21.15.47:0/411832674 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a089750210 con 0x55a089ade9f0 2024-03-21T11:46:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.873+0000 7fdc49400640 1 --2- 172.21.15.47:0/411832674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 0x55a089ae90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.874+0000 7fdc49e00640 1 --2- 172.21.15.47:0/411832674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089ae8b80 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0ba290 tx=0x7fdc3c0b9bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fbe240d7969b69d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.874+0000 7fdc2b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.874+0000 7fdc2b400640 1 -- 172.21.15.47:0/411832674 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c0bd690 con 0x55a089ade9f0 2024-03-21T11:46:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.874+0000 7fdc2b400640 1 -- 172.21.15.47:0/411832674 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdc3c0bbbc0 con 0x55a089ade9f0 2024-03-21T11:46:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.874+0000 7fdc539b2740 1 -- 172.21.15.47:0/411832674 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc38004c80 con 0x55a089ade9f0 2024-03-21T11:46:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.874+0000 7fdc2b400640 1 -- 172.21.15.47:0/411832674 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc3c0bd220 con 0x55a089ade9f0 2024-03-21T11:46:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.874+0000 7fdc539b2740 10 client.33007 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.874+0000 7fdc539b2740 1 -- 172.21.15.47:0/411832674 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a08981c620 con 0x55a089ade9f0 2024-03-21T11:46:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.875+0000 7fdc2b400640 1 -- 172.21.15.47:0/411832674 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc38004c80 con 0x55a089ade9f0 2024-03-21T11:46:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.875+0000 7fdc2b400640 1 -- 172.21.15.47:0/411832674 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdc3c050060 con 0x55a089ade9f0 2024-03-21T11:46:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.875+0000 7fdc539b2740 10 client.33007 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.875+0000 7fdc539b2740 10 client.33007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.875+0000 7fdc539b2740 20 client.33007 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.875+0000 7fdc539b2740 1 --2- 172.21.15.47:0/411832674 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a089af4a20 0x55a089af6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.875+0000 7fdc539b2740 4 client.33007 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:46:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.875+0000 7fdc539b2740 1 -- 172.21.15.47:0/411832674 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a089a48d70 con 0x55a089af4a20 2024-03-21T11:46:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.875+0000 7fdc49400640 1 --2- 172.21.15.47:0/411832674 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a089af4a20 0x55a089af6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.876+0000 7fdc49400640 1 --2- 172.21.15.47:0/411832674 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a089af4a20 0x55a089af6e90 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dbb8be670cfc10e server_cookie=91091f2897971b85 in_seq=0 out_seq=0 2024-03-21T11:46:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.876+0000 7fdc2b400640 10 client.33007 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.880+0000 7fdc2b400640 1 -- 172.21.15.47:0/411832674 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a089a48d70 con 0x55a089af4a20 2024-03-21T11:46:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.880+0000 7fdc2b400640 10 client.33007 handle_command_reply: tid=0 2024-03-21T11:46:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.880+0000 7fdc539b2740 1 client.33007 shutdown 2024-03-21T11:46:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.880+0000 7fdc539b2740 1 -- 172.21.15.47:0/411832674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 msgr2=0x55a089ae8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.880+0000 7fdc539b2740 1 --2- 172.21.15.47:0/411832674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089ae8b80 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0ba290 tx=0x7fdc3c0b9bc0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.881+0000 7fdc539b2740 1 -- 172.21.15.47:0/411832674 shutdown_connections 2024-03-21T11:46:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.881+0000 7fdc539b2740 1 --2- 172.21.15.47:0/411832674 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a089af4a20 0x55a089af6e90 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.881+0000 7fdc539b2740 1 --2- 172.21.15.47:0/411832674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089af0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.881+0000 7fdc539b2740 1 --2- 172.21.15.47:0/411832674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 0x55a089ae90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.881+0000 7fdc539b2740 1 --2- 172.21.15.47:0/411832674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089ae8b80 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.881+0000 7fdc539b2740 1 -- 172.21.15.47:0/411832674 >> 172.21.15.47:0/411832674 conn(0x55a089ad9f60 msgr2=0x55a089ad0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.881+0000 7fdc2b400640 0 client.33007 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.881+0000 7fdc539b2740 1 -- 172.21.15.47:0/411832674 shutdown_connections 2024-03-21T11:46:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.881+0000 7fdc539b2740 1 -- 172.21.15.47:0/411832674 wait complete. 2024-03-21T11:46:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.881+0000 7fdc539b2740 20 client.33007 trim_cache size 0 max 16384 2024-03-21T11:46:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.898+0000 7fdc539b2740 1 Processor -- start 2024-03-21T11:46:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.898+0000 7fdc539b2740 1 -- start start 2024-03-21T11:46:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.899+0000 7fdc539b2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089add1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.899+0000 7fdc539b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089adf3e0 0x55a089af00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.899+0000 7fdc539b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089ae4550 0x55a089add700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.899+0000 7fdc539b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1a7e70 con 0x55a089ae4550 2024-03-21T11:46:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.899+0000 7fdc539b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1a97f0 con 0x55a089adf3e0 2024-03-21T11:46:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.899+0000 7fdc539b2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1a84d0 con 0x55a089ade9f0 2024-03-21T11:46:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.899+0000 7fdc49400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089adf3e0 0x55a089af00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.899+0000 7fdc49400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089adf3e0 0x55a089af00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36996/0 (socket says 172.21.15.47:36996) 2024-03-21T11:46:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.899+0000 7fdc49400640 1 -- 172.21.15.47:0/2245990803 learned_addr learned my addr 172.21.15.47:0/2245990803 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.900+0000 7fdc4a800640 1 --2- 172.21.15.47:0/2245990803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089ae4550 0x55a089add700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.900+0000 7fdc49400640 1 -- 172.21.15.47:0/2245990803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 msgr2=0x55a089add1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:46:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.900+0000 7fdc49400640 1 --2- 172.21.15.47:0/2245990803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089add1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.900+0000 7fdc49400640 1 -- 172.21.15.47:0/2245990803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089ae4550 msgr2=0x55a089add700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.900+0000 7fdc49400640 1 --2- 172.21.15.47:0/2245990803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089ae4550 0x55a089add700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.900+0000 7fdc49400640 1 -- 172.21.15.47:0/2245990803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a089adf7c0 con 0x55a089adf3e0 2024-03-21T11:46:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.900+0000 7fdc4a800640 1 --2- 172.21.15.47:0/2245990803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089ae4550 0x55a089add700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:46:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.902+0000 7fdc49400640 1 --2- 172.21.15.47:0/2245990803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089adf3e0 0x55a089af00c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x55a089adaed0 tx=0x7fdc3800b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e2f30757f70d523 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.903+0000 7fdc36600640 1 -- 172.21.15.47:0/2245990803 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc380052e0 con 0x55a089adf3e0 2024-03-21T11:46:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.903+0000 7fdc36600640 1 -- 172.21.15.47:0/2245990803 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc38005480 con 0x55a089adf3e0 2024-03-21T11:46:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.903+0000 7fdc36600640 1 -- 172.21.15.47:0/2245990803 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3802ab30 con 0x55a089adf3e0 2024-03-21T11:46:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.904+0000 7fdc539b2740 1 -- 172.21.15.47:0/2245990803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089adf3e0 msgr2=0x55a089af00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.904+0000 7fdc539b2740 1 --2- 172.21.15.47:0/2245990803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089adf3e0 0x55a089af00c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x55a089adaed0 tx=0x7fdc3800b410 comp rx=0 tx=0).stop 2024-03-21T11:46:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.904+0000 7fdc539b2740 1 -- 172.21.15.47:0/2245990803 shutdown_connections 2024-03-21T11:46:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.904+0000 7fdc539b2740 1 --2- 172.21.15.47:0/2245990803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089ae4550 0x55a089add700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.904+0000 7fdc539b2740 1 --2- 172.21.15.47:0/2245990803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089adf3e0 0x55a089af00c0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.904+0000 7fdc539b2740 1 --2- 172.21.15.47:0/2245990803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089add1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.904+0000 7fdc539b2740 1 -- 172.21.15.47:0/2245990803 >> 172.21.15.47:0/2245990803 conn(0x55a089ad9f60 msgr2=0x55a089ae0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.904+0000 7fdc539b2740 1 -- 172.21.15.47:0/2245990803 shutdown_connections 2024-03-21T11:46:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.905+0000 7fdc539b2740 1 -- 172.21.15.47:0/2245990803 wait complete. 2024-03-21T11:46:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.905+0000 7fdc539b2740 1 -- start start 2024-03-21T11:46:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.905+0000 7fdc539b2740 1 Processor -- start 2024-03-21T11:46:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.906+0000 7fdc539b2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089aec110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.906+0000 7fdc539b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 0x55a089aec650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.906+0000 7fdc539b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089aef670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.906+0000 7fdc539b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1a9190 con 0x55a089adf3e0 2024-03-21T11:46:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.906+0000 7fdc539b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1a8b30 con 0x55a089ae4550 2024-03-21T11:46:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.906+0000 7fdc539b2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1a6070 con 0x55a089ade9f0 2024-03-21T11:46:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.906+0000 7fdc539b2740 10 client.0 _ll_register_callbacks cb 0x7fdc3c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:46:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.906+0000 7fdc49e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089aec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.906+0000 7fdc49e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089aec110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:60938/0 (socket says 172.21.15.47:60938) 2024-03-21T11:46:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.906+0000 7fdc49e00640 1 -- 172.21.15.47:0/1652320929 learned_addr learned my addr 172.21.15.47:0/1652320929 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:46:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.906+0000 7fdc4a800640 1 --2- 172.21.15.47:0/1652320929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089aef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.907+0000 7fdc49400640 1 --2- 172.21.15.47:0/1652320929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 0x55a089aec650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.907+0000 7fdc49e00640 1 -- 172.21.15.47:0/1652320929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 msgr2=0x55a089aef670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.907+0000 7fdc49e00640 1 --2- 172.21.15.47:0/1652320929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089aef670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.907+0000 7fdc49e00640 1 -- 172.21.15.47:0/1652320929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 msgr2=0x55a089aec650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:46:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.907+0000 7fdc49e00640 1 --2- 172.21.15.47:0/1652320929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 0x55a089aec650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.907+0000 7fdc49e00640 1 -- 172.21.15.47:0/1652320929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a089aefc60 con 0x55a089ade9f0 2024-03-21T11:46:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.907+0000 7fdc4a800640 1 --2- 172.21.15.47:0/1652320929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089aef670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:46:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.907+0000 7fdc49e00640 1 --2- 172.21.15.47:0/1652320929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089aec110 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0bd5c0 tx=0x7fdc3c0f4a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37fed0c3147cde1a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:46:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.907+0000 7fdc2b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:46:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.911+0000 7fdc2b400640 1 -- 172.21.15.47:0/1652320929 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c0bd400 con 0x55a089ade9f0 2024-03-21T11:46:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.911+0000 7fdc2b400640 1 -- 172.21.15.47:0/1652320929 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdc3c0c5e40 con 0x55a089ade9f0 2024-03-21T11:46:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.911+0000 7fdc539b2740 1 -- 172.21.15.47:0/1652320929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc38005480 con 0x55a089ade9f0 2024-03-21T11:46:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.913+0000 7fdc2b400640 1 -- 172.21.15.47:0/1652320929 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc3c0bdfc0 con 0x55a089ade9f0 2024-03-21T11:46:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.913+0000 7fdc539b2740 10 client.33010 fetch_fsmap learned FSMap version 41 2024-03-21T11:46:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.913+0000 7fdc539b2740 1 -- 172.21.15.47:0/1652320929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a089b61c40 con 0x55a089ade9f0 2024-03-21T11:46:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.914+0000 7fdc2b400640 1 -- 172.21.15.47:0/1652320929 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc38005480 con 0x55a089ade9f0 2024-03-21T11:46:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.915+0000 7fdc2b400640 1 -- 172.21.15.47:0/1652320929 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdc3c0f10c0 con 0x55a089ade9f0 2024-03-21T11:46:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.915+0000 7fdc539b2740 10 client.33010 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:46:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.915+0000 7fdc539b2740 10 client.33010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:46:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.915+0000 7fdc539b2740 20 client.33010 populate_metadata read hostname 'smithi047' 2024-03-21T11:46:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.915+0000 7fdc539b2740 1 --2- 172.21.15.47:0/1652320929 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a089b62390 0x55a089b647b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:46:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.915+0000 7fdc539b2740 4 client.33010 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:46:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.915+0000 7fdc539b2740 1 -- 172.21.15.47:0/1652320929 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a089a430c0 con 0x55a089b62390 2024-03-21T11:46:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.915+0000 7fdc49400640 1 --2- 172.21.15.47:0/1652320929 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a089b62390 0x55a089b647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:46:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.916+0000 7fdc49400640 1 --2- 172.21.15.47:0/1652320929 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a089b62390 0x55a089b647b0 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=922adc9c2f49fa91 server_cookie=2365ea53e25f1e3e in_seq=0 out_seq=0 2024-03-21T11:46:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:57.916+0000 7fdc2b400640 10 client.33010 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:57.990 INFO:tasks.workunit.client.0.smithi047.stdout: 10 245418 21.87 MB/sec execute 461 sec latency 70.293 ms 2024-03-21T11:46:58.990 INFO:tasks.workunit.client.0.smithi047.stdout: 10 245515 21.83 MB/sec execute 462 sec latency 831.478 ms 2024-03-21T11:46:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.116+0000 7fdc2b400640 1 -- 172.21.15.47:0/1652320929 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a089a430c0 con 0x55a089b62390 2024-03-21T11:46:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.117+0000 7fdc2b400640 10 client.33010 handle_command_reply: tid=0 2024-03-21T11:46:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.117+0000 7fdc539b2740 1 client.33010 shutdown 2024-03-21T11:46:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.119+0000 7fdc539b2740 1 -- 172.21.15.47:0/1652320929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 msgr2=0x55a089aec110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.119+0000 7fdc539b2740 1 --2- 172.21.15.47:0/1652320929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089aec110 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0bd5c0 tx=0x7fdc3c0f4a90 comp rx=0 tx=0).stop 2024-03-21T11:46:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.119+0000 7fdc539b2740 1 -- 172.21.15.47:0/1652320929 shutdown_connections 2024-03-21T11:46:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.120+0000 7fdc539b2740 1 --2- 172.21.15.47:0/1652320929 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a089b62390 0x55a089b647b0 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.120+0000 7fdc539b2740 1 --2- 172.21.15.47:0/1652320929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a089ae4550 0x55a089aef670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.120+0000 7fdc539b2740 1 --2- 172.21.15.47:0/1652320929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a089adf3e0 0x55a089aec650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.120+0000 7fdc539b2740 1 --2- 172.21.15.47:0/1652320929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a089ade9f0 0x55a089aec110 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.120+0000 7fdc539b2740 1 -- 172.21.15.47:0/1652320929 >> 172.21.15.47:0/1652320929 conn(0x55a089ad9f60 msgr2=0x55a089add1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.120+0000 7fdc2b400640 0 client.33010 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:46:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.120+0000 7fdc539b2740 1 -- 172.21.15.47:0/1652320929 shutdown_connections 2024-03-21T11:46:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.121+0000 7fdc539b2740 1 -- 172.21.15.47:0/1652320929 wait complete. 2024-03-21T11:46:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.121+0000 7fdc539b2740 20 client.33010 trim_cache size 0 max 16384 2024-03-21T11:46:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:46:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:46:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:46:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:46:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.123+0000 7fdc50c00640 1 -- 172.21.15.47:0/3407391750 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc300a30c0 msgr2=0x7fdc300a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.123+0000 7fdc50c00640 1 --2- 172.21.15.47:0/3407391750 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc300a30c0 0x7fdc300a5580 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x55a089adfc40 tx=0x7fdc44004ab0 comp rx=0 tx=0).stop 2024-03-21T11:46:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.123+0000 7fdc50c00640 1 -- 172.21.15.47:0/3407391750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc4c1016d0 msgr2=0x7fdc4c199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:46:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.123+0000 7fdc50c00640 1 --2- 172.21.15.47:0/3407391750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc4c1016d0 0x7fdc4c199b70 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009230 tx=0x7fdc3c005dd0 comp rx=0 tx=0).stop 2024-03-21T11:46:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.123+0000 7fdc50c00640 1 -- 172.21.15.47:0/3407391750 shutdown_connections 2024-03-21T11:46:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.123+0000 7fdc50c00640 1 --2- 172.21.15.47:0/3407391750 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc300a30c0 0x7fdc300a5580 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.123+0000 7fdc50c00640 1 --2- 172.21.15.47:0/3407391750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc4c103280 0x7fdc4c1a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.123+0000 7fdc50c00640 1 --2- 172.21.15.47:0/3407391750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc4c1028c0 0x7fdc4c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.123+0000 7fdc50c00640 1 --2- 172.21.15.47:0/3407391750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc4c1016d0 0x7fdc4c199b70 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:46:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.123+0000 7fdc50c00640 1 -- 172.21.15.47:0/3407391750 >> 172.21.15.47:0/3407391750 conn(0x7fdc4c0fcc60 msgr2=0x7fdc4c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:46:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.126+0000 7fdc50c00640 1 -- 172.21.15.47:0/3407391750 shutdown_connections 2024-03-21T11:46:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:46:59.126+0000 7fdc50c00640 1 -- 172.21.15.47:0/3407391750 wait complete. 2024-03-21T11:46:59.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:46:59 smithi102 ceph-mon[36748]: pgmap v1110: 129 pgs: 129 active+clean; 703 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.9 MiB/s wr, 478 op/s 2024-03-21T11:46:59.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:46:59 smithi047 ceph-mon[31351]: pgmap v1110: 129 pgs: 129 active+clean; 703 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.9 MiB/s wr, 478 op/s 2024-03-21T11:46:59.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:46:59 smithi100 ceph-mon[36245]: pgmap v1110: 129 pgs: 129 active+clean; 703 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.9 MiB/s wr, 478 op/s 2024-03-21T11:46:59.990 INFO:tasks.workunit.client.0.smithi047.stdout: 10 245841 21.84 MB/sec execute 463 sec latency 196.282 ms 2024-03-21T11:47:00.994 INFO:tasks.workunit.client.0.smithi047.stdout: 10 246267 21.85 MB/sec execute 464 sec latency 101.978 ms 2024-03-21T11:47:01.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:01 smithi102 ceph-mon[36748]: pgmap v1111: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.8 MiB/s wr, 498 op/s 2024-03-21T11:47:01.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:01 smithi047 ceph-mon[31351]: pgmap v1111: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.8 MiB/s wr, 498 op/s 2024-03-21T11:47:01.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:01 smithi100 ceph-mon[36245]: pgmap v1111: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.8 MiB/s wr, 498 op/s 2024-03-21T11:47:01.990 INFO:tasks.workunit.client.0.smithi047.stdout: 10 246618 21.83 MB/sec execute 465 sec latency 118.743 ms 2024-03-21T11:47:02.991 INFO:tasks.workunit.client.0.smithi047.stdout: 10 247013 21.81 MB/sec execute 466 sec latency 83.873 ms 2024-03-21T11:47:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:03 smithi102 ceph-mon[36748]: pgmap v1112: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.8 MiB/s wr, 509 op/s 2024-03-21T11:47:03.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:03 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:47:03.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:03 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:47:03.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:03 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:47:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:03 smithi047 ceph-mon[31351]: pgmap v1112: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.8 MiB/s wr, 509 op/s 2024-03-21T11:47:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:03 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:47:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:03 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:47:03.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:03 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:47:03.653 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:03 smithi100 ceph-mon[36245]: pgmap v1112: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.8 MiB/s wr, 509 op/s 2024-03-21T11:47:03.653 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:03 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:47:03.653 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:03 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:47:03.653 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:03 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:47:03.991 INFO:tasks.workunit.client.0.smithi047.stdout: 10 247593 21.83 MB/sec execute 467 sec latency 65.150 ms 2024-03-21T11:47:04.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:04 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:47:04.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:04 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:47:04.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:04 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:47:04.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:04 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:47:04.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:04 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:47:04.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:04 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:47:04.991 INFO:tasks.workunit.client.0.smithi047.stdout: 10 248206 21.85 MB/sec execute 468 sec latency 112.586 ms 2024-03-21T11:47:05.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:05 smithi047 ceph-mon[31351]: pgmap v1113: 129 pgs: 129 active+clean; 710 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.7 MiB/s wr, 509 op/s 2024-03-21T11:47:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:05 smithi102 ceph-mon[36748]: pgmap v1113: 129 pgs: 129 active+clean; 710 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.7 MiB/s wr, 509 op/s 2024-03-21T11:47:05.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:05 smithi100 ceph-mon[36245]: pgmap v1113: 129 pgs: 129 active+clean; 710 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.7 MiB/s wr, 509 op/s 2024-03-21T11:47:05.991 INFO:tasks.workunit.client.0.smithi047.stdout: 10 248712 21.87 MB/sec execute 469 sec latency 91.737 ms 2024-03-21T11:47:06.991 INFO:tasks.workunit.client.0.smithi047.stdout: 10 249264 21.91 MB/sec execute 470 sec latency 104.966 ms 2024-03-21T11:47:07.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:07 smithi102 ceph-mon[36748]: pgmap v1114: 129 pgs: 129 active+clean; 709 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.7 MiB/s wr, 500 op/s 2024-03-21T11:47:07.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:07 smithi047 ceph-mon[31351]: pgmap v1114: 129 pgs: 129 active+clean; 709 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.7 MiB/s wr, 500 op/s 2024-03-21T11:47:07.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:07 smithi100 ceph-mon[36245]: pgmap v1114: 129 pgs: 129 active+clean; 709 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.7 MiB/s wr, 500 op/s 2024-03-21T11:47:07.992 INFO:tasks.workunit.client.0.smithi047.stdout: 10 249621 21.90 MB/sec execute 471 sec latency 111.892 ms 2024-03-21T11:47:08.991 INFO:tasks.workunit.client.0.smithi047.stdout: 10 250057 21.90 MB/sec execute 472 sec latency 48.671 ms 2024-03-21T11:47:09.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:09 smithi102 ceph-mon[36748]: pgmap v1115: 129 pgs: 129 active+clean; 712 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.4 MiB/s wr, 460 op/s 2024-03-21T11:47:09.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:09 smithi047 ceph-mon[31351]: pgmap v1115: 129 pgs: 129 active+clean; 712 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.4 MiB/s wr, 460 op/s 2024-03-21T11:47:09.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:09 smithi100 ceph-mon[36245]: pgmap v1115: 129 pgs: 129 active+clean; 712 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.4 MiB/s wr, 460 op/s 2024-03-21T11:47:09.991 INFO:tasks.workunit.client.0.smithi047.stdout: 10 250457 21.88 MB/sec execute 473 sec latency 122.490 ms 2024-03-21T11:47:10.991 INFO:tasks.workunit.client.0.smithi047.stdout: 10 250885 21.89 MB/sec execute 474 sec latency 90.327 ms 2024-03-21T11:47:11.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:11 smithi102 ceph-mon[36748]: pgmap v1116: 129 pgs: 129 active+clean; 708 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 499 op/s 2024-03-21T11:47:11.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:11 smithi047 ceph-mon[31351]: pgmap v1116: 129 pgs: 129 active+clean; 708 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 499 op/s 2024-03-21T11:47:11.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:11 smithi100 ceph-mon[36245]: pgmap v1116: 129 pgs: 129 active+clean; 708 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 499 op/s 2024-03-21T11:47:11.991 INFO:tasks.workunit.client.0.smithi047.stdout: 10 251516 21.91 MB/sec execute 475 sec latency 61.566 ms 2024-03-21T11:47:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.356+0000 7f4ebd000640 1 -- 172.21.15.47:0/1057946761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4eb8104a70 msgr2=0x7f4eb8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.356+0000 7f4ebd000640 1 --2- 172.21.15.47:0/1057946761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4eb8104a70 0x7f4eb8106e60 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009900 tx=0x7f4ea402bc30 comp rx=0 tx=0).stop 2024-03-21T11:47:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.356+0000 7f4ebd000640 1 -- 172.21.15.47:0/1057946761 shutdown_connections 2024-03-21T11:47:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.356+0000 7f4ebd000640 1 --2- 172.21.15.47:0/1057946761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eb81073a0 0x7f4eb81097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.357+0000 7f4ebd000640 1 --2- 172.21.15.47:0/1057946761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4eb8104a70 0x7f4eb8106e60 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.357+0000 7f4ebd000640 1 --2- 172.21.15.47:0/1057946761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb8102140 0x7f4eb8104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.357+0000 7f4ebd000640 1 -- 172.21.15.47:0/1057946761 >> 172.21.15.47:0/1057946761 conn(0x7f4eb80fbb30 msgr2=0x7f4eb80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.357+0000 7f4ebd000640 1 -- 172.21.15.47:0/1057946761 shutdown_connections 2024-03-21T11:47:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.357+0000 7f4ebd000640 1 -- 172.21.15.47:0/1057946761 wait complete. 2024-03-21T11:47:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.358+0000 7f4ebd000640 1 Processor -- start 2024-03-21T11:47:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.358+0000 7f4ebd000640 1 -- start start 2024-03-21T11:47:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.358+0000 7f4ebd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eb8102140 0x7f4eb8101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.358+0000 7f4ebd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4eb8104a70 0x7f4eb81000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.358+0000 7f4ebd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb81073a0 0x7f4eb8100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.358+0000 7f4ebd000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb810d390 con 0x7f4eb81073a0 2024-03-21T11:47:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.358+0000 7f4ebd000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb810d210 con 0x7f4eb8102140 2024-03-21T11:47:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.358+0000 7f4ebd000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb810d510 con 0x7f4eb8104a70 2024-03-21T11:47:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.358+0000 7f4eb5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eb8102140 0x7f4eb8101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.358+0000 7f4eb5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eb8102140 0x7f4eb8101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60064/0 (socket says 172.21.15.47:60064) 2024-03-21T11:47:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.358+0000 7f4eb5e00640 1 -- 172.21.15.47:0/1501920373 learned_addr learned my addr 172.21.15.47:0/1501920373 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.359+0000 7f4eb6800640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb81073a0 0x7f4eb8100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.359+0000 7f4eb5400640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4eb8104a70 0x7f4eb81000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.359+0000 7f4eb5400640 1 -- 172.21.15.47:0/1501920373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eb8102140 msgr2=0x7f4eb8101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.359+0000 7f4eb5400640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eb8102140 0x7f4eb8101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.359+0000 7f4eb5400640 1 -- 172.21.15.47:0/1501920373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb81073a0 msgr2=0x7f4eb8100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.359+0000 7f4eb5400640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb81073a0 0x7f4eb8100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.359+0000 7f4eb5400640 1 -- 172.21.15.47:0/1501920373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb8100ec0 con 0x7f4eb8104a70 2024-03-21T11:47:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.359+0000 7f4eb6800640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb81073a0 0x7f4eb8100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.360+0000 7f4eb5400640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4eb8104a70 0x7f4eb81000f0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009230 tx=0x7f4ea402c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.360+0000 7f4eaea00640 1 -- 172.21.15.47:0/1501920373 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4043020 con 0x7f4eb8104a70 2024-03-21T11:47:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.360+0000 7f4eaea00640 1 -- 172.21.15.47:0/1501920373 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ea40054e0 con 0x7f4eb8104a70 2024-03-21T11:47:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.360+0000 7f4ebd000640 1 -- 172.21.15.47:0/1501920373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb81a1a70 con 0x7f4eb8104a70 2024-03-21T11:47:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.360+0000 7f4eaea00640 1 -- 172.21.15.47:0/1501920373 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4003b60 con 0x7f4eb8104a70 2024-03-21T11:47:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.360+0000 7f4ebd000640 1 -- 172.21.15.47:0/1501920373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb81a1fe0 con 0x7f4eb8104a70 2024-03-21T11:47:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.365+0000 7f4ebfda0740 1 Processor -- start 2024-03-21T11:47:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.365+0000 7f4ebfda0740 1 -- start start 2024-03-21T11:47:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.365+0000 7f4ebfda0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb3f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.365+0000 7f4ebfda0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb3fbd0 0x5585dcb446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.365+0000 7f4ebfda0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb44d40 0x5585dcb471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.365+0000 7f4ebfda0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585dcb023b0 con 0x5585dcb3fbd0 2024-03-21T11:47:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.365+0000 7f4ebfda0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585dcaa4830 con 0x5585dcb44d40 2024-03-21T11:47:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.365+0000 7f4ebfda0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585dca4d5d0 con 0x5585dcb3f1e0 2024-03-21T11:47:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.365+0000 7f4eb6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb3f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.365+0000 7f4eb6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb3f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35468/0 (socket says 172.21.15.47:35468) 2024-03-21T11:47:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.365+0000 7f4eb6800640 1 -- 172.21.15.47:0/2014048463 learned_addr learned my addr 172.21.15.47:0/2014048463 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.366+0000 7f4eaea00640 1 -- 172.21.15.47:0/1501920373 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4ea4046020 con 0x7f4eb8104a70 2024-03-21T11:47:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.366+0000 7f4eb5400640 1 --2- 172.21.15.47:0/2014048463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb44d40 0x5585dcb471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.366+0000 7f4eb5e00640 1 --2- 172.21.15.47:0/2014048463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb3fbd0 0x5585dcb446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.366+0000 7f4eb6800640 1 -- 172.21.15.47:0/2014048463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb44d40 msgr2=0x5585dcb471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.366+0000 7f4eb6800640 1 --2- 172.21.15.47:0/2014048463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb44d40 0x5585dcb471d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.366+0000 7f4eb6800640 1 -- 172.21.15.47:0/2014048463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb3fbd0 msgr2=0x5585dcb446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.366+0000 7f4eb6800640 1 --2- 172.21.15.47:0/2014048463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb3fbd0 0x5585dcb446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.366+0000 7f4eb6800640 1 -- 172.21.15.47:0/2014048463 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585dca652c0 con 0x5585dcb3f1e0 2024-03-21T11:47:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.366+0000 7f4eb5400640 1 --2- 172.21.15.47:0/2014048463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb44d40 0x5585dcb471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:47:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.367+0000 7f4eb6800640 1 --2- 172.21.15.47:0/2014048463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb3f600 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00117e0 tx=0x7f4eb0011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d57c8e8372f1e7fc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.368+0000 7f4ea3400640 1 -- 172.21.15.47:0/2014048463 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0011310 con 0x5585dcb3f1e0 2024-03-21T11:47:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.368+0000 7f4ea3400640 1 -- 172.21.15.47:0/2014048463 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4eb0003e50 con 0x5585dcb3f1e0 2024-03-21T11:47:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.368+0000 7f4ea3400640 1 -- 172.21.15.47:0/2014048463 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0004620 con 0x5585dcb3f1e0 2024-03-21T11:47:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.368+0000 7f4eaea00640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e9c0a30c0 0x7f4e9c0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.369+0000 7f4eaea00640 1 -- 172.21.15.47:0/1501920373 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4ea40f16e0 con 0x7f4eb8104a70 2024-03-21T11:47:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.369+0000 7f4eb5e00640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e9c0a30c0 0x7f4e9c0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.369+0000 7f4ebfda0740 1 -- 172.21.15.47:0/2014048463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 msgr2=0x5585dcb3f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.369+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/2014048463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb3f600 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00117e0 tx=0x7f4eb0011ca0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.370+0000 7f4ebfda0740 1 -- 172.21.15.47:0/2014048463 shutdown_connections 2024-03-21T11:47:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.370+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/2014048463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb44d40 0x5585dcb471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.370+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/2014048463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb3fbd0 0x5585dcb446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.370+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/2014048463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb3f600 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.370+0000 7f4ebfda0740 1 -- 172.21.15.47:0/2014048463 >> 172.21.15.47:0/2014048463 conn(0x5585dcb3a750 msgr2=0x5585dcb3ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.370+0000 7f4ebfda0740 1 -- 172.21.15.47:0/2014048463 shutdown_connections 2024-03-21T11:47:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.370+0000 7f4ebfda0740 1 -- 172.21.15.47:0/2014048463 wait complete. 2024-03-21T11:47:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.370+0000 7f4eb5e00640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e9c0a30c0 0x7f4e9c0a5580 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x5585dcb3ffd0 tx=0x7f4ea8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.371+0000 7f4ebfda0740 1 -- start start 2024-03-21T11:47:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.371+0000 7f4ebfda0740 1 Processor -- start 2024-03-21T11:47:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.372+0000 7f4ebfda0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb49420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.372+0000 7f4ebfda0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb3fbd0 0x5585dcb49960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.372+0000 7f4ebfda0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb44d40 0x5585dcb509a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.372+0000 7f4ebfda0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585dca60a20 con 0x5585dcb3fbd0 2024-03-21T11:47:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.372+0000 7f4ebfda0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585dcaa08d0 con 0x5585dcb44d40 2024-03-21T11:47:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.372+0000 7f4ebfda0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585dcb47cf0 con 0x5585dcb3f1e0 2024-03-21T11:47:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.372+0000 7f4ebfda0740 10 client.0 _ll_register_callbacks cb 0x5585dca9d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.372+0000 7f4eb5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb49420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.372+0000 7f4eb5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb49420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35478/0 (socket says 172.21.15.47:35478) 2024-03-21T11:47:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.372+0000 7f4eb5e00640 1 -- 172.21.15.47:0/3247616338 learned_addr learned my addr 172.21.15.47:0/3247616338 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.372+0000 7f4eb5400640 1 --2- 172.21.15.47:0/3247616338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb3fbd0 0x5585dcb49960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.372+0000 7f4eb6800640 1 --2- 172.21.15.47:0/3247616338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb44d40 0x5585dcb509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.373+0000 7f4eb5e00640 1 -- 172.21.15.47:0/3247616338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb44d40 msgr2=0x5585dcb509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.373+0000 7f4eb5e00640 1 --2- 172.21.15.47:0/3247616338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb44d40 0x5585dcb509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.373+0000 7f4eb5e00640 1 -- 172.21.15.47:0/3247616338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb3fbd0 msgr2=0x5585dcb49960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.373+0000 7f4eb5e00640 1 --2- 172.21.15.47:0/3247616338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb3fbd0 0x5585dcb49960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.373+0000 7f4eb5e00640 1 -- 172.21.15.47:0/3247616338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585dc7b9210 con 0x5585dcb3f1e0 2024-03-21T11:47:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.373+0000 7f4eb6800640 1 --2- 172.21.15.47:0/3247616338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb44d40 0x5585dcb509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.373+0000 7f4eb5e00640 1 --2- 172.21.15.47:0/3247616338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb49420 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8013010 tx=0x7f4ea8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d3cb82ee5cea069 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.373+0000 7f4e97e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:47:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.374+0000 7f4e97e00640 1 -- 172.21.15.47:0/3247616338 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8004020 con 0x5585dcb3f1e0 2024-03-21T11:47:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.374+0000 7f4e97e00640 1 -- 172.21.15.47:0/3247616338 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4ea8018030 con 0x5585dcb3f1e0 2024-03-21T11:47:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.374+0000 7f4ebfda0740 1 -- 172.21.15.47:0/3247616338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4eb0003e50 con 0x5585dcb3f1e0 2024-03-21T11:47:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.374+0000 7f4e97e00640 1 -- 172.21.15.47:0/3247616338 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ea8033050 con 0x5585dcb3f1e0 2024-03-21T11:47:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.374+0000 7f4ebfda0740 10 client.33022 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.374+0000 7f4ebfda0740 1 -- 172.21.15.47:0/3247616338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5585dc885620 con 0x5585dcb3f1e0 2024-03-21T11:47:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.375+0000 7f4e97e00640 1 -- 172.21.15.47:0/3247616338 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4eb0003e50 con 0x5585dcb3f1e0 2024-03-21T11:47:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.375+0000 7f4e97e00640 1 -- 172.21.15.47:0/3247616338 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4ea8029420 con 0x5585dcb3f1e0 2024-03-21T11:47:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.375+0000 7f4ebfda0740 10 client.33022 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.375+0000 7f4ebfda0740 10 client.33022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.375+0000 7f4ebfda0740 20 client.33022 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.375+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/3247616338 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585dcb552f0 0x5585dcb57700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.375+0000 7f4ebfda0740 4 client.33022 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.375+0000 7f4ebfda0740 1 -- 172.21.15.47:0/3247616338 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585dcab1d70 con 0x5585dcb552f0 2024-03-21T11:47:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.375+0000 7f4eb5400640 1 --2- 172.21.15.47:0/3247616338 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585dcb552f0 0x5585dcb57700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.376+0000 7f4eb5400640 1 --2- 172.21.15.47:0/3247616338 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585dcb552f0 0x5585dcb57700 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f555117b7655e2 server_cookie=47fb22dfd171c389 in_seq=0 out_seq=0 2024-03-21T11:47:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.376+0000 7f4e97e00640 10 client.33022 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.378+0000 7f4e97e00640 1 -- 172.21.15.47:0/3247616338 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5585dcab1d70 con 0x5585dcb552f0 2024-03-21T11:47:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.378+0000 7f4e97e00640 10 client.33022 handle_command_reply: tid=0 2024-03-21T11:47:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.378+0000 7f4ebfda0740 1 client.33022 shutdown 2024-03-21T11:47:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.378+0000 7f4ebfda0740 1 -- 172.21.15.47:0/3247616338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 msgr2=0x5585dcb49420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.378+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/3247616338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb49420 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8013010 tx=0x7f4ea8004730 comp rx=0 tx=0).stop 2024-03-21T11:47:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.378+0000 7f4ebfda0740 1 -- 172.21.15.47:0/3247616338 shutdown_connections 2024-03-21T11:47:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.378+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/3247616338 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585dcb552f0 0x5585dcb57700 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.378+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/3247616338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb44d40 0x5585dcb509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.378+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/3247616338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb3fbd0 0x5585dcb49960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.378+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/3247616338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb49420 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.378+0000 7f4ebfda0740 1 -- 172.21.15.47:0/3247616338 >> 172.21.15.47:0/3247616338 conn(0x5585dcb3a750 msgr2=0x5585dcb39950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.378+0000 7f4e97e00640 0 client.33022 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.379+0000 7f4ebfda0740 1 -- 172.21.15.47:0/3247616338 shutdown_connections 2024-03-21T11:47:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.379+0000 7f4ebfda0740 1 -- 172.21.15.47:0/3247616338 wait complete. 2024-03-21T11:47:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.379+0000 7f4ebfda0740 20 client.33022 trim_cache size 0 max 16384 2024-03-21T11:47:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.394+0000 7f4ebfda0740 1 Processor -- start 2024-03-21T11:47:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.394+0000 7f4ebfda0740 1 -- start start 2024-03-21T11:47:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.394+0000 7f4ebfda0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb498c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.394+0000 7f4ebfda0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb3fbd0 0x5585dcb50960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.394+0000 7f4ebfda0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb44d40 0x5585dcb3e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.394+0000 7f4ebfda0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81ac1d0 con 0x5585dcb44d40 2024-03-21T11:47:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.394+0000 7f4ebfda0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81adb50 con 0x5585dcb3fbd0 2024-03-21T11:47:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.394+0000 7f4ebfda0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81ac830 con 0x5585dcb3f1e0 2024-03-21T11:47:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.395+0000 7f4eb5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.395+0000 7f4eb5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb3fbd0 0x5585dcb50960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.395+0000 7f4eb5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35486/0 (socket says 172.21.15.47:35486) 2024-03-21T11:47:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.395+0000 7f4eb5e00640 1 -- 172.21.15.47:0/324499128 learned_addr learned my addr 172.21.15.47:0/324499128 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.395+0000 7f4eb6800640 1 --2- 172.21.15.47:0/324499128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb44d40 0x5585dcb3e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.395+0000 7f4eb5e00640 1 -- 172.21.15.47:0/324499128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb3fbd0 msgr2=0x5585dcb50960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.395+0000 7f4eb5e00640 1 --2- 172.21.15.47:0/324499128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb3fbd0 0x5585dcb50960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.395+0000 7f4eb5e00640 1 -- 172.21.15.47:0/324499128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb44d40 msgr2=0x5585dcb3e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.395+0000 7f4eb5e00640 1 --2- 172.21.15.47:0/324499128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb44d40 0x5585dcb3e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.395+0000 7f4eb5e00640 1 -- 172.21.15.47:0/324499128 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585dcb47e30 con 0x5585dcb3f1e0 2024-03-21T11:47:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.395+0000 7f4eb6800640 1 --2- 172.21.15.47:0/324499128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb44d40 0x5585dcb3e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:47:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.395+0000 7f4eb5e00640 1 --2- 172.21.15.47:0/324499128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb498c0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f4ea801efd0 tx=0x7f4ea8009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a590b6d0e97675e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.395+0000 7f4ea2a00640 1 -- 172.21.15.47:0/324499128 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea801e7c0 con 0x5585dcb3f1e0 2024-03-21T11:47:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.396+0000 7f4ea2a00640 1 -- 172.21.15.47:0/324499128 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ea8029a80 con 0x5585dcb3f1e0 2024-03-21T11:47:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.396+0000 7f4ea2a00640 1 -- 172.21.15.47:0/324499128 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80054c0 con 0x5585dcb3f1e0 2024-03-21T11:47:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.396+0000 7f4ebfda0740 1 -- 172.21.15.47:0/324499128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 msgr2=0x5585dcb498c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.396+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/324499128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb498c0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f4ea801efd0 tx=0x7f4ea8009c60 comp rx=0 tx=0).stop 2024-03-21T11:47:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.396+0000 7f4ebfda0740 1 -- 172.21.15.47:0/324499128 shutdown_connections 2024-03-21T11:47:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.396+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/324499128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb44d40 0x5585dcb3e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.396+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/324499128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb3fbd0 0x5585dcb50960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.396+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/324499128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3f1e0 0x5585dcb498c0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.396+0000 7f4ebfda0740 1 -- 172.21.15.47:0/324499128 >> 172.21.15.47:0/324499128 conn(0x5585dcb3a750 msgr2=0x5585dcb3d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.396+0000 7f4ebfda0740 1 -- 172.21.15.47:0/324499128 shutdown_connections 2024-03-21T11:47:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.397+0000 7f4ebfda0740 1 -- 172.21.15.47:0/324499128 wait complete. 2024-03-21T11:47:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.397+0000 7f4ebfda0740 1 -- start start 2024-03-21T11:47:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.397+0000 7f4ebfda0740 1 Processor -- start 2024-03-21T11:47:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.397+0000 7f4ebfda0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb3f1e0 0x5585dcb4e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4ebfda0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3fbd0 0x5585dcb4ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4ebfda0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb44d40 0x5585dcb4fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4ebfda0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81ad4f0 con 0x5585dcb44d40 2024-03-21T11:47:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4ebfda0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81ace90 con 0x5585dcb3f1e0 2024-03-21T11:47:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4ebfda0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81aa390 con 0x5585dcb3fbd0 2024-03-21T11:47:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4ebfda0740 10 client.0 _ll_register_callbacks cb 0x7f4ea8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4eb5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb3f1e0 0x5585dcb4e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4eb5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3fbd0 0x5585dcb4ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4eb5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb3f1e0 0x5585dcb4e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60106/0 (socket says 172.21.15.47:60106) 2024-03-21T11:47:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4eb5e00640 1 -- 172.21.15.47:0/2881017126 learned_addr learned my addr 172.21.15.47:0/2881017126 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4eb6800640 1 --2- 172.21.15.47:0/2881017126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb44d40 0x5585dcb4fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4eb5e00640 1 -- 172.21.15.47:0/2881017126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3fbd0 msgr2=0x5585dcb4ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4eb5e00640 1 --2- 172.21.15.47:0/2881017126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3fbd0 0x5585dcb4ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4eb5e00640 1 -- 172.21.15.47:0/2881017126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb44d40 msgr2=0x5585dcb4fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4eb5e00640 1 --2- 172.21.15.47:0/2881017126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb44d40 0x5585dcb4fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4eb5e00640 1 -- 172.21.15.47:0/2881017126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585dcb50420 con 0x5585dcb3f1e0 2024-03-21T11:47:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4eb5400640 1 --2- 172.21.15.47:0/2881017126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3fbd0 0x5585dcb4ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.398+0000 7f4eb5e00640 1 --2- 172.21.15.47:0/2881017126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb3f1e0 0x5585dcb4e8e0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8029a50 tx=0x7f4ea8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b29039956b2aea11 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.399+0000 7f4e97e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.399+0000 7f4e97e00640 1 -- 172.21.15.47:0/2881017126 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8044020 con 0x5585dcb3f1e0 2024-03-21T11:47:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.399+0000 7f4ebfda0740 1 -- 172.21.15.47:0/2881017126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ea8029a80 con 0x5585dcb3f1e0 2024-03-21T11:47:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.399+0000 7f4e97e00640 1 -- 172.21.15.47:0/2881017126 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4ea80051c0 con 0x5585dcb3f1e0 2024-03-21T11:47:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.399+0000 7f4e97e00640 1 -- 172.21.15.47:0/2881017126 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ea804f050 con 0x5585dcb3f1e0 2024-03-21T11:47:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.399+0000 7f4ebfda0740 10 client.33059 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.399+0000 7f4ebfda0740 1 -- 172.21.15.47:0/2881017126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5585dcbb1460 con 0x5585dcb3f1e0 2024-03-21T11:47:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.399+0000 7f4e97e00640 1 -- 172.21.15.47:0/2881017126 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ea8029a80 con 0x5585dcb3f1e0 2024-03-21T11:47:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.400+0000 7f4e97e00640 1 -- 172.21.15.47:0/2881017126 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4ea80035f0 con 0x5585dcb3f1e0 2024-03-21T11:47:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.400+0000 7f4ebfda0740 10 client.33059 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.400+0000 7f4ebfda0740 10 client.33059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.400+0000 7f4ebfda0740 20 client.33059 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.400+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/2881017126 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585dcbb1d20 0x5585dcbb4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.400+0000 7f4ebfda0740 4 client.33059 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:47:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.400+0000 7f4ebfda0740 1 -- 172.21.15.47:0/2881017126 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5585dcaac0c0 con 0x5585dcbb1d20 2024-03-21T11:47:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.400+0000 7f4eb5400640 1 --2- 172.21.15.47:0/2881017126 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585dcbb1d20 0x5585dcbb4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.401+0000 7f4eb5400640 1 --2- 172.21.15.47:0/2881017126 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585dcbb1d20 0x5585dcbb4140 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1060441dfee15ea server_cookie=8408e905b9d37f11 in_seq=0 out_seq=0 2024-03-21T11:47:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:12.401+0000 7f4e97e00640 10 client.33059 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:12.991 INFO:tasks.workunit.client.0.smithi047.stdout: 10 251730 21.89 MB/sec execute 476 sec latency 451.672 ms 2024-03-21T11:47:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.472+0000 7f4e97e00640 1 -- 172.21.15.47:0/2881017126 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5585dcaac0c0 con 0x5585dcbb1d20 2024-03-21T11:47:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.472+0000 7f4e97e00640 10 client.33059 handle_command_reply: tid=0 2024-03-21T11:47:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.472+0000 7f4ebfda0740 1 client.33059 shutdown 2024-03-21T11:47:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.472+0000 7f4ebfda0740 1 -- 172.21.15.47:0/2881017126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb3f1e0 msgr2=0x5585dcb4e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.472+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/2881017126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb3f1e0 0x5585dcb4e8e0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8029a50 tx=0x7f4ea8004270 comp rx=0 tx=0).stop 2024-03-21T11:47:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.472+0000 7f4ebfda0740 1 -- 172.21.15.47:0/2881017126 shutdown_connections 2024-03-21T11:47:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.472+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/2881017126 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5585dcbb1d20 0x5585dcbb4140 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.472+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/2881017126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585dcb44d40 0x5585dcb4fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.472+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/2881017126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585dcb3fbd0 0x5585dcb4ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.473+0000 7f4e97e00640 0 client.33059 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.473+0000 7f4ebfda0740 1 --2- 172.21.15.47:0/2881017126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5585dcb3f1e0 0x5585dcb4e8e0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.473+0000 7f4ebfda0740 1 -- 172.21.15.47:0/2881017126 >> 172.21.15.47:0/2881017126 conn(0x5585dcb3a750 msgr2=0x5585dcb3e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.474+0000 7f4ebfda0740 1 -- 172.21.15.47:0/2881017126 shutdown_connections 2024-03-21T11:47:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.474+0000 7f4ebfda0740 1 -- 172.21.15.47:0/2881017126 wait complete. 2024-03-21T11:47:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.474+0000 7f4ebfda0740 20 client.33059 trim_cache size 0 max 16384 2024-03-21T11:47:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:47:13.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:47:13.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:47:13.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:47:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.477+0000 7f4ebd000640 1 -- 172.21.15.47:0/1501920373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e9c0a30c0 msgr2=0x7f4e9c0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.477+0000 7f4ebd000640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e9c0a30c0 0x7f4e9c0a5580 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x5585dcb3ffd0 tx=0x7f4ea8005f90 comp rx=0 tx=0).stop 2024-03-21T11:47:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.477+0000 7f4ebd000640 1 -- 172.21.15.47:0/1501920373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4eb8104a70 msgr2=0x7f4eb81000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.477+0000 7f4ebd000640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4eb8104a70 0x7f4eb81000f0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009230 tx=0x7f4ea402c5d0 comp rx=0 tx=0).stop 2024-03-21T11:47:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.477+0000 7f4ebd000640 1 -- 172.21.15.47:0/1501920373 shutdown_connections 2024-03-21T11:47:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.477+0000 7f4ebd000640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f4e9c0a30c0 0x7f4e9c0a5580 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:13.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.477+0000 7f4ebd000640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4eb81073a0 0x7f4eb8100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:13.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.477+0000 7f4ebd000640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4eb8104a70 0x7f4eb81000f0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:13.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.477+0000 7f4ebd000640 1 --2- 172.21.15.47:0/1501920373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eb8102140 0x7f4eb8101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:13.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.477+0000 7f4ebd000640 1 -- 172.21.15.47:0/1501920373 >> 172.21.15.47:0/1501920373 conn(0x7f4eb80fbb30 msgr2=0x7f4eb8102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:13.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.478+0000 7f4ebd000640 1 -- 172.21.15.47:0/1501920373 shutdown_connections 2024-03-21T11:47:13.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:13.478+0000 7f4ebd000640 1 -- 172.21.15.47:0/1501920373 wait complete. 2024-03-21T11:47:13.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:13 smithi102 ceph-mon[36748]: pgmap v1117: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.6 MiB/s wr, 455 op/s 2024-03-21T11:47:13.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:13 smithi047 ceph-mon[31351]: pgmap v1117: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.6 MiB/s wr, 455 op/s 2024-03-21T11:47:13.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:13 smithi100 ceph-mon[36245]: pgmap v1117: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.6 MiB/s wr, 455 op/s 2024-03-21T11:47:13.991 INFO:tasks.workunit.client.0.smithi047.stdout: 10 252026 21.88 MB/sec execute 477 sec latency 738.969 ms 2024-03-21T11:47:14.991 INFO:tasks.workunit.client.0.smithi047.stdout: 10 252606 21.91 MB/sec execute 478 sec latency 77.480 ms 2024-03-21T11:47:15.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:15 smithi102 ceph-mon[36748]: pgmap v1118: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.6 MiB/s wr, 435 op/s 2024-03-21T11:47:15.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:15 smithi047 ceph-mon[31351]: pgmap v1118: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.6 MiB/s wr, 435 op/s 2024-03-21T11:47:15.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:15 smithi100 ceph-mon[36245]: pgmap v1118: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.6 MiB/s wr, 435 op/s 2024-03-21T11:47:15.991 INFO:tasks.workunit.client.0.smithi047.stdout: 10 253083 21.91 MB/sec execute 479 sec latency 88.330 ms 2024-03-21T11:47:16.992 INFO:tasks.workunit.client.0.smithi047.stdout: 10 253558 21.92 MB/sec execute 480 sec latency 92.328 ms 2024-03-21T11:47:17.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:17 smithi102 ceph-mon[36748]: pgmap v1119: 129 pgs: 129 active+clean; 703 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 428 op/s 2024-03-21T11:47:17.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:17 smithi047 ceph-mon[31351]: pgmap v1119: 129 pgs: 129 active+clean; 703 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 428 op/s 2024-03-21T11:47:17.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:17 smithi100 ceph-mon[36245]: pgmap v1119: 129 pgs: 129 active+clean; 703 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 428 op/s 2024-03-21T11:47:17.993 INFO:tasks.workunit.client.0.smithi047.stdout: 10 253879 21.91 MB/sec execute 481 sec latency 275.093 ms 2024-03-21T11:47:18.992 INFO:tasks.workunit.client.0.smithi047.stdout: 10 254076 21.89 MB/sec execute 482 sec latency 552.738 ms 2024-03-21T11:47:19.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:47:19.337 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:47:19.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:19 smithi102 ceph-mon[36748]: pgmap v1120: 129 pgs: 129 active+clean; 707 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.8 MiB/s wr, 506 op/s 2024-03-21T11:47:19.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:19 smithi047 ceph-mon[31351]: pgmap v1120: 129 pgs: 129 active+clean; 707 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.8 MiB/s wr, 506 op/s 2024-03-21T11:47:19.645 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:19 smithi100 ceph-mon[36245]: pgmap v1120: 129 pgs: 129 active+clean; 707 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.8 MiB/s wr, 506 op/s 2024-03-21T11:47:19.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.645+0000 7f28a9800640 1 -- 172.21.15.47:0/3939421236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28a4108910 msgr2=0x7f28a410ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:19.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.645+0000 7f28a9800640 1 --2- 172.21.15.47:0/3939421236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28a4108910 0x7f28a410ad00 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f28940074e0 tx=0x7f289402c8f0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.646+0000 7f28a9800640 1 -- 172.21.15.47:0/3939421236 shutdown_connections 2024-03-21T11:47:19.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.646+0000 7f28a9800640 1 --2- 172.21.15.47:0/3939421236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28a410dd70 0x7f28a4110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.646+0000 7f28a9800640 1 --2- 172.21.15.47:0/3939421236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a410b2d0 0x7f28a410d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.646+0000 7f28a9800640 1 --2- 172.21.15.47:0/3939421236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28a4108910 0x7f28a410ad00 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.646+0000 7f28a9800640 1 -- 172.21.15.47:0/3939421236 >> 172.21.15.47:0/3939421236 conn(0x7f28a406e500 msgr2=0x7f28a4070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:19.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.646+0000 7f28a9800640 1 -- 172.21.15.47:0/3939421236 shutdown_connections 2024-03-21T11:47:19.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.646+0000 7f28a9800640 1 -- 172.21.15.47:0/3939421236 wait complete. 2024-03-21T11:47:19.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.648+0000 7f28a9800640 1 Processor -- start 2024-03-21T11:47:19.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.648+0000 7f28a9800640 1 -- start start 2024-03-21T11:47:19.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.648+0000 7f28a9800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a4108910 0x7f28a41175b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.648+0000 7f28a9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28a410b2d0 0x7f28a4117af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.648+0000 7f28a9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28a410dd70 0x7f28a4114120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.649+0000 7f28a2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28a410b2d0 0x7f28a4117af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.649+0000 7f28a2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a4108910 0x7f28a41175b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.649+0000 7f28a2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28a410b2d0 0x7f28a4117af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59498/0 (socket says 172.21.15.47:59498) 2024-03-21T11:47:19.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.649+0000 7f28a2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a4108910 0x7f28a41175b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37090/0 (socket says 172.21.15.47:37090) 2024-03-21T11:47:19.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.651+0000 7f28a9800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a4113070 con 0x7f28a4108910 2024-03-21T11:47:19.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.651+0000 7f28a9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a4112ef0 con 0x7f28a410b2d0 2024-03-21T11:47:19.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.651+0000 7f28a9800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a4118050 con 0x7f28a410dd70 2024-03-21T11:47:19.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.651+0000 7f28a2000640 1 -- 172.21.15.47:0/4247889063 learned_addr learned my addr 172.21.15.47:0/4247889063 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:19.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.651+0000 7f28a2000640 1 -- 172.21.15.47:0/4247889063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28a410dd70 msgr2=0x7f28a4114120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:47:19.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.651+0000 7f28a2000640 1 --2- 172.21.15.47:0/4247889063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28a410dd70 0x7f28a4114120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.651+0000 7f28a2000640 1 -- 172.21.15.47:0/4247889063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a4108910 msgr2=0x7f28a41175b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:19.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.651+0000 7f28a2000640 1 --2- 172.21.15.47:0/4247889063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a4108910 0x7f28a41175b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.651+0000 7f28a2000640 1 -- 172.21.15.47:0/4247889063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a41149e0 con 0x7f28a410b2d0 2024-03-21T11:47:19.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.651+0000 7f28a2a00640 1 --2- 172.21.15.47:0/4247889063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a4108910 0x7f28a41175b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:19.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.653+0000 7f28a2000640 1 --2- 172.21.15.47:0/4247889063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28a410b2d0 0x7f28a4117af0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f2890006fd0 tx=0x7f289000b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:19.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.657+0000 7f289b400640 1 -- 172.21.15.47:0/4247889063 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890011830 con 0x7f28a410b2d0 2024-03-21T11:47:19.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.658+0000 7f289b400640 1 -- 172.21.15.47:0/4247889063 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2890016070 con 0x7f28a410b2d0 2024-03-21T11:47:19.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.658+0000 7f289b400640 1 -- 172.21.15.47:0/4247889063 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289000f2e0 con 0x7f28a410b2d0 2024-03-21T11:47:19.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.658+0000 7f28a9800640 1 -- 172.21.15.47:0/4247889063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a41ae8d0 con 0x7f28a410b2d0 2024-03-21T11:47:19.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.658+0000 7f28a9800640 1 -- 172.21.15.47:0/4247889063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a41aed60 con 0x7f28a410b2d0 2024-03-21T11:47:19.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.668+0000 7f28ac428740 1 Processor -- start 2024-03-21T11:47:19.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.670+0000 7f289b400640 1 -- 172.21.15.47:0/4247889063 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f289001b460 con 0x7f28a410b2d0 2024-03-21T11:47:19.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.671+0000 7f28ac428740 1 -- start start 2024-03-21T11:47:19.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.673+0000 7f28ac428740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77966e80 0x564d779672a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.674+0000 7f28a3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77966e80 0x564d779672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.674+0000 7f28a3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77966e80 0x564d779672a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47872/0 (socket says 172.21.15.47:47872) 2024-03-21T11:47:19.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.676+0000 7f28a3400640 1 -- 172.21.15.47:0/3029268658 learned_addr learned my addr 172.21.15.47:0/3029268658 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:19.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.676+0000 7f28ac428740 1 --2- 172.21.15.47:0/3029268658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 0x564d7796c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.676+0000 7f28ac428740 1 --2- 172.21.15.47:0/3029268658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 0x564d7796ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.676+0000 7f28ac428740 1 -- 172.21.15.47:0/3029268658 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d778cc250 con 0x564d7796c9e0 2024-03-21T11:47:19.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.676+0000 7f28ac428740 1 -- 172.21.15.47:0/3029268658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d77874a30 con 0x564d77967870 2024-03-21T11:47:19.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.676+0000 7f28ac428740 1 -- 172.21.15.47:0/3029268658 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d778c82f0 con 0x564d77966e80 2024-03-21T11:47:19.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.676+0000 7f28a2a00640 1 --2- 172.21.15.47:0/3029268658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 0x564d7796c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.679+0000 7f28a2000640 1 --2- 172.21.15.47:0/3029268658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 0x564d7796ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.680+0000 7f289b400640 1 --2- 172.21.15.47:0/4247889063 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28880a3120 0x7f28880a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.683+0000 7f28a2a00640 1 -- 172.21.15.47:0/3029268658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77966e80 msgr2=0x564d779672a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:19.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.683+0000 7f28a2a00640 1 --2- 172.21.15.47:0/3029268658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77966e80 0x564d779672a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.683+0000 7f28a2a00640 1 -- 172.21.15.47:0/3029268658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 msgr2=0x564d7796ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:19.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.683+0000 7f28a2a00640 1 --2- 172.21.15.47:0/3029268658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 0x564d7796ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.683+0000 7f28a2a00640 1 -- 172.21.15.47:0/3029268658 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d7785f690 con 0x564d77967870 2024-03-21T11:47:19.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.683+0000 7f28a2000640 1 --2- 172.21.15.47:0/3029268658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 0x564d7796ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:19.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.683+0000 7f28a3400640 1 --2- 172.21.15.47:0/3029268658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77966e80 0x564d779672a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:19.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.683+0000 7f289b400640 1 -- 172.21.15.47:0/4247889063 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f28900ced10 con 0x7f28a410b2d0 2024-03-21T11:47:19.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.684+0000 7f28a2a00640 1 --2- 172.21.15.47:0/3029268658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 0x564d7796c370 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f289402cf00 tx=0x7f2894000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc131352e06a8139 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:19.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.685+0000 7f288fe00640 1 -- 172.21.15.47:0/3029268658 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2894040070 con 0x564d77967870 2024-03-21T11:47:19.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.685+0000 7f288fe00640 1 -- 172.21.15.47:0/3029268658 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2894004bc0 con 0x564d77967870 2024-03-21T11:47:19.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.685+0000 7f288fe00640 1 -- 172.21.15.47:0/3029268658 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2894004490 con 0x564d77967870 2024-03-21T11:47:19.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.686+0000 7f28a3400640 1 --2- 172.21.15.47:0/4247889063 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28880a3120 0x7f28880a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.686+0000 7f28ac428740 1 -- 172.21.15.47:0/3029268658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 msgr2=0x564d7796c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:19.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.686+0000 7f28ac428740 1 --2- 172.21.15.47:0/3029268658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 0x564d7796c370 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f289402cf00 tx=0x7f2894000bc0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.686+0000 7f28ac428740 1 -- 172.21.15.47:0/3029268658 shutdown_connections 2024-03-21T11:47:19.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.686+0000 7f28ac428740 1 --2- 172.21.15.47:0/3029268658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 0x564d7796ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.686+0000 7f28ac428740 1 --2- 172.21.15.47:0/3029268658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 0x564d7796c370 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.686+0000 7f28ac428740 1 --2- 172.21.15.47:0/3029268658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77966e80 0x564d779672a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.686+0000 7f28ac428740 1 -- 172.21.15.47:0/3029268658 >> 172.21.15.47:0/3029268658 conn(0x564d7792f200 msgr2=0x564d779627f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:19.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.686+0000 7f28ac428740 1 -- 172.21.15.47:0/3029268658 shutdown_connections 2024-03-21T11:47:19.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.687+0000 7f28ac428740 1 -- 172.21.15.47:0/3029268658 wait complete. 2024-03-21T11:47:19.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.688+0000 7f28ac428740 1 -- start start 2024-03-21T11:47:19.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.688+0000 7f28ac428740 1 Processor -- start 2024-03-21T11:47:19.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.687+0000 7f28a3400640 1 --2- 172.21.15.47:0/4247889063 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28880a3120 0x7f28880a55e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x564d779680d0 tx=0x7f289c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:19.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.690+0000 7f28ac428740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77966e80 0x564d77970e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.691+0000 7f28ac428740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 0x564d779713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.691+0000 7f28ac428740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 0x564d77978410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.691+0000 7f28a2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77966e80 0x564d77970e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.691+0000 7f28a2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77966e80 0x564d77970e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47880/0 (socket says 172.21.15.47:47880) 2024-03-21T11:47:19.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.691+0000 7f28a2a00640 1 -- 172.21.15.47:0/2215660628 learned_addr learned my addr 172.21.15.47:0/2215660628 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:19.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.691+0000 7f28ac428740 1 -- 172.21.15.47:0/2215660628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d7796f980 con 0x564d7796c9e0 2024-03-21T11:47:19.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.691+0000 7f28ac428740 1 -- 172.21.15.47:0/2215660628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d77887e80 con 0x564d77967870 2024-03-21T11:47:19.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.691+0000 7f28ac428740 1 -- 172.21.15.47:0/2215660628 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d7796fb00 con 0x564d77966e80 2024-03-21T11:47:19.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.691+0000 7f28a2000640 1 --2- 172.21.15.47:0/2215660628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 0x564d779713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.691+0000 7f28ac428740 10 client.0 _ll_register_callbacks cb 0x564d7789a130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.692+0000 7f28a2000640 1 -- 172.21.15.47:0/2215660628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77966e80 msgr2=0x564d77970e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.692+0000 7f28a2000640 1 --2- 172.21.15.47:0/2215660628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77966e80 0x564d77970e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.692+0000 7f28a2000640 1 -- 172.21.15.47:0/2215660628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 msgr2=0x564d77978410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.692+0000 7f28a2000640 1 --2- 172.21.15.47:0/2215660628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 0x564d77978410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.692+0000 7f28a2000640 1 -- 172.21.15.47:0/2215660628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d776aa7d0 con 0x564d77967870 2024-03-21T11:47:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.692+0000 7f28a2000640 1 --2- 172.21.15.47:0/2215660628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 0x564d779713d0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f2890009fd0 tx=0x7f28900046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a03539318ba98e55 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:19.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.692+0000 7f288cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:19.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.697+0000 7f288cc00640 1 -- 172.21.15.47:0/2215660628 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289009f1a0 con 0x564d77967870 2024-03-21T11:47:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.697+0000 7f288cc00640 1 -- 172.21.15.47:0/2215660628 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f28900976c0 con 0x564d77967870 2024-03-21T11:47:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.697+0000 7f28ac428740 1 -- 172.21.15.47:0/2215660628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2894004bc0 con 0x564d77967870 2024-03-21T11:47:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.697+0000 7f288cc00640 1 -- 172.21.15.47:0/2215660628 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28900a3a60 con 0x564d77967870 2024-03-21T11:47:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.697+0000 7f28ac428740 10 client.33071 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.697+0000 7f28ac428740 1 -- 172.21.15.47:0/2215660628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564d775e03f0 con 0x564d77967870 2024-03-21T11:47:19.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.699+0000 7f288cc00640 1 -- 172.21.15.47:0/2215660628 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2894004bc0 con 0x564d77967870 2024-03-21T11:47:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.700+0000 7f288cc00640 1 -- 172.21.15.47:0/2215660628 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2890095ce0 con 0x564d77967870 2024-03-21T11:47:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.700+0000 7f28ac428740 10 client.33071 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.700+0000 7f28ac428740 10 client.33071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.700+0000 7f28ac428740 20 client.33071 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.700+0000 7f28ac428740 1 --2- 172.21.15.47:0/2215660628 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d7797cb70 0x564d7797f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.700+0000 7f28ac428740 4 client.33071 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.700+0000 7f28ac428740 1 -- 172.21.15.47:0/2215660628 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d778d9790 con 0x564d7797cb70 2024-03-21T11:47:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.701+0000 7f28a2a00640 1 --2- 172.21.15.47:0/2215660628 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d7797cb70 0x564d7797f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.704+0000 7f288cc00640 10 client.33071 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:19.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.704+0000 7f28a2a00640 1 --2- 172.21.15.47:0/2215660628 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d7797cb70 0x564d7797f010 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1a00201b94b22ea server_cookie=152ba713751faac in_seq=0 out_seq=0 2024-03-21T11:47:19.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.716+0000 7f288cc00640 1 -- 172.21.15.47:0/2215660628 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564d778d9790 con 0x564d7797cb70 2024-03-21T11:47:19.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.716+0000 7f288cc00640 10 client.33071 handle_command_reply: tid=0 2024-03-21T11:47:19.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.716+0000 7f28ac428740 1 client.33071 shutdown 2024-03-21T11:47:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.717+0000 7f28ac428740 1 -- 172.21.15.47:0/2215660628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 msgr2=0x564d779713d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.717+0000 7f28ac428740 1 --2- 172.21.15.47:0/2215660628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 0x564d779713d0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f2890009fd0 tx=0x7f28900046c0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.718+0000 7f28ac428740 1 -- 172.21.15.47:0/2215660628 shutdown_connections 2024-03-21T11:47:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.718+0000 7f28ac428740 1 --2- 172.21.15.47:0/2215660628 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d7797cb70 0x564d7797f010 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.718+0000 7f28ac428740 1 --2- 172.21.15.47:0/2215660628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 0x564d77978410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.718+0000 7f28ac428740 1 --2- 172.21.15.47:0/2215660628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 0x564d779713d0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.718+0000 7f288cc00640 0 client.33071 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.718+0000 7f28ac428740 1 --2- 172.21.15.47:0/2215660628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77966e80 0x564d77970e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.718+0000 7f28ac428740 1 -- 172.21.15.47:0/2215660628 >> 172.21.15.47:0/2215660628 conn(0x564d7792f200 msgr2=0x564d77961a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.718+0000 7f28ac428740 1 -- 172.21.15.47:0/2215660628 shutdown_connections 2024-03-21T11:47:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.719+0000 7f28ac428740 1 -- 172.21.15.47:0/2215660628 wait complete. 2024-03-21T11:47:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.719+0000 7f28ac428740 20 client.33071 trim_cache size 0 max 16384 2024-03-21T11:47:19.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.747+0000 7f28ac428740 1 Processor -- start 2024-03-21T11:47:19.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.747+0000 7f28ac428740 1 -- start start 2024-03-21T11:47:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.747+0000 7f28ac428740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d77966e80 0x564d77970e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.747+0000 7f28ac428740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 0x564d779783d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.747+0000 7f28ac428740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d7796c9e0 0x564d77965a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.747+0000 7f28ac428740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a41b8ef0 con 0x564d77966e80 2024-03-21T11:47:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.747+0000 7f28ac428740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a41ba870 con 0x564d77967870 2024-03-21T11:47:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.747+0000 7f28ac428740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a41b9550 con 0x564d7796c9e0 2024-03-21T11:47:19.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.748+0000 7f28a3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d7796c9e0 0x564d77965a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.748+0000 7f28a2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d77966e80 0x564d77970e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.748+0000 7f28a3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d7796c9e0 0x564d77965a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47884/0 (socket says 172.21.15.47:47884) 2024-03-21T11:47:19.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.748+0000 7f28a3400640 1 -- 172.21.15.47:0/2397268539 learned_addr learned my addr 172.21.15.47:0/2397268539 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:19.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.748+0000 7f28a2000640 1 --2- 172.21.15.47:0/2397268539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 0x564d779783d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.749+0000 7f28a3400640 1 -- 172.21.15.47:0/2397268539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 msgr2=0x564d779783d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:19.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.749+0000 7f28a3400640 1 --2- 172.21.15.47:0/2397268539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 0x564d779783d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.749+0000 7f28a3400640 1 -- 172.21.15.47:0/2397268539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d77966e80 msgr2=0x564d77970e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:19.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.749+0000 7f28a3400640 1 --2- 172.21.15.47:0/2397268539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d77966e80 0x564d77970e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.749+0000 7f28a3400640 1 -- 172.21.15.47:0/2397268539 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d775a00f0 con 0x564d7796c9e0 2024-03-21T11:47:19.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.749+0000 7f28a3400640 1 --2- 172.21.15.47:0/2397268539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d7796c9e0 0x564d77965a30 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f289c003b00 tx=0x7f289c013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cd2986a18dff419 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.755+0000 7f288f400640 1 -- 172.21.15.47:0/2397268539 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c004770 con 0x564d7796c9e0 2024-03-21T11:47:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.755+0000 7f288f400640 1 -- 172.21.15.47:0/2397268539 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f289c00f960 con 0x564d7796c9e0 2024-03-21T11:47:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.756+0000 7f288f400640 1 -- 172.21.15.47:0/2397268539 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c026a60 con 0x564d7796c9e0 2024-03-21T11:47:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.758+0000 7f28ac428740 1 -- 172.21.15.47:0/2397268539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d7796c9e0 msgr2=0x564d77965a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.758+0000 7f28ac428740 1 --2- 172.21.15.47:0/2397268539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d7796c9e0 0x564d77965a30 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f289c003b00 tx=0x7f289c013040 comp rx=0 tx=0).stop 2024-03-21T11:47:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.770+0000 7f28ac428740 1 -- 172.21.15.47:0/2397268539 shutdown_connections 2024-03-21T11:47:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.770+0000 7f28ac428740 1 --2- 172.21.15.47:0/2397268539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d7796c9e0 0x564d77965a30 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.770+0000 7f28ac428740 1 --2- 172.21.15.47:0/2397268539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77967870 0x564d779783d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.770+0000 7f28ac428740 1 --2- 172.21.15.47:0/2397268539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d77966e80 0x564d77970e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.770+0000 7f28ac428740 1 -- 172.21.15.47:0/2397268539 >> 172.21.15.47:0/2397268539 conn(0x564d7792f200 msgr2=0x564d77968b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.770+0000 7f28ac428740 1 -- 172.21.15.47:0/2397268539 shutdown_connections 2024-03-21T11:47:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.772+0000 7f28ac428740 1 -- 172.21.15.47:0/2397268539 wait complete. 2024-03-21T11:47:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.772+0000 7f28ac428740 1 -- start start 2024-03-21T11:47:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.773+0000 7f28ac428740 1 Processor -- start 2024-03-21T11:47:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.774+0000 7f28ac428740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77966e80 0x564d77977c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.774+0000 7f28ac428740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77967870 0x564d77978180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.774+0000 7f28ac428740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 0x564d77976250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.776+0000 7f28ac428740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a41ba210 con 0x564d7796c9e0 2024-03-21T11:47:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.776+0000 7f28ac428740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a41b9bb0 con 0x564d77966e80 2024-03-21T11:47:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.776+0000 7f28ac428740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a41b70b0 con 0x564d77967870 2024-03-21T11:47:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.776+0000 7f28a2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77966e80 0x564d77977c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.776+0000 7f28a2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77966e80 0x564d77977c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59534/0 (socket says 172.21.15.47:59534) 2024-03-21T11:47:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.776+0000 7f28a2a00640 1 -- 172.21.15.47:0/473814495 learned_addr learned my addr 172.21.15.47:0/473814495 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.777+0000 7f28ac428740 10 client.0 _ll_register_callbacks cb 0x7f2890010380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.777+0000 7f28a3400640 1 --2- 172.21.15.47:0/473814495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 0x564d77976250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.779+0000 7f28a2a00640 1 -- 172.21.15.47:0/473814495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77967870 msgr2=0x564d77978180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:47:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.779+0000 7f28a2a00640 1 --2- 172.21.15.47:0/473814495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77967870 0x564d77978180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.779+0000 7f28a2a00640 1 -- 172.21.15.47:0/473814495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 msgr2=0x564d77976250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.779+0000 7f28a2a00640 1 --2- 172.21.15.47:0/473814495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 0x564d77976250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.779+0000 7f28a2a00640 1 -- 172.21.15.47:0/473814495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d77976940 con 0x564d77966e80 2024-03-21T11:47:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.780+0000 7f28a2a00640 1 --2- 172.21.15.47:0/473814495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77966e80 0x564d77977c40 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f289400d090 tx=0x7f2894004cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98794477a1199aa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.780+0000 7f288cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.782+0000 7f288cc00640 1 -- 172.21.15.47:0/473814495 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2894040040 con 0x564d77966e80 2024-03-21T11:47:19.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.782+0000 7f288cc00640 1 -- 172.21.15.47:0/473814495 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f28940520d0 con 0x564d77966e80 2024-03-21T11:47:19.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.782+0000 7f28ac428740 1 -- 172.21.15.47:0/473814495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f289c00f960 con 0x564d77966e80 2024-03-21T11:47:19.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.785+0000 7f288cc00640 1 -- 172.21.15.47:0/473814495 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28940522d0 con 0x564d77966e80 2024-03-21T11:47:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.785+0000 7f28ac428740 10 client.33077 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.785+0000 7f28ac428740 1 -- 172.21.15.47:0/473814495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564d779d8f20 con 0x564d77966e80 2024-03-21T11:47:19.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.788+0000 7f288cc00640 1 -- 172.21.15.47:0/473814495 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f289c00f960 con 0x564d77966e80 2024-03-21T11:47:19.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.789+0000 7f288cc00640 1 -- 172.21.15.47:0/473814495 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2894052900 con 0x564d77966e80 2024-03-21T11:47:19.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.789+0000 7f28ac428740 10 client.33077 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:19.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.789+0000 7f28ac428740 10 client.33077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:19.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.789+0000 7f28ac428740 20 client.33077 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:19.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.789+0000 7f28ac428740 1 --2- 172.21.15.47:0/473814495 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d779d94c0 0x564d779db8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:19.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.789+0000 7f28ac428740 4 client.33077 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:47:19.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.789+0000 7f28ac428740 1 -- 172.21.15.47:0/473814495 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564d778d3ae0 con 0x564d779d94c0 2024-03-21T11:47:19.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.792+0000 7f28a2000640 1 --2- 172.21.15.47:0/473814495 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d779d94c0 0x564d779db8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:19.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.794+0000 7f28a2000640 1 --2- 172.21.15.47:0/473814495 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d779d94c0 0x564d779db8e0 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd9bfabce093ba6 server_cookie=cc86be14123a9010 in_seq=0 out_seq=0 2024-03-21T11:47:19.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.794+0000 7f288cc00640 10 client.33077 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:19.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.796+0000 7f288cc00640 1 -- 172.21.15.47:0/473814495 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564d778d3ae0 con 0x564d779d94c0 2024-03-21T11:47:19.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.796+0000 7f288cc00640 10 client.33077 handle_command_reply: tid=0 2024-03-21T11:47:19.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.796+0000 7f28ac428740 1 client.33077 shutdown 2024-03-21T11:47:19.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.799+0000 7f28ac428740 1 -- 172.21.15.47:0/473814495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77966e80 msgr2=0x564d77977c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:19.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.799+0000 7f28ac428740 1 --2- 172.21.15.47:0/473814495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77966e80 0x564d77977c40 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f289400d090 tx=0x7f2894004cb0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.799+0000 7f28ac428740 1 -- 172.21.15.47:0/473814495 shutdown_connections 2024-03-21T11:47:19.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.799+0000 7f28ac428740 1 --2- 172.21.15.47:0/473814495 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564d779d94c0 0x564d779db8e0 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.799+0000 7f288cc00640 0 client.33077 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:19.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.799+0000 7f28ac428740 1 --2- 172.21.15.47:0/473814495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d7796c9e0 0x564d77976250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.799+0000 7f28ac428740 1 --2- 172.21.15.47:0/473814495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d77967870 0x564d77978180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.799+0000 7f28ac428740 1 --2- 172.21.15.47:0/473814495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564d77966e80 0x564d77977c40 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.799+0000 7f28ac428740 1 -- 172.21.15.47:0/473814495 >> 172.21.15.47:0/473814495 conn(0x564d7792f200 msgr2=0x564d77961200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:19.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.800+0000 7f28ac428740 1 -- 172.21.15.47:0/473814495 shutdown_connections 2024-03-21T11:47:19.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.800+0000 7f28ac428740 1 -- 172.21.15.47:0/473814495 wait complete. 2024-03-21T11:47:19.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.801+0000 7f28ac428740 20 client.33077 trim_cache size 0 max 16384 2024-03-21T11:47:19.802 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:47:19.802 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:47:19.802 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:47:19.802 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:47:19.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.803+0000 7f28a9800640 1 -- 172.21.15.47:0/4247889063 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28880a3120 msgr2=0x7f28880a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:19.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.803+0000 7f28a9800640 1 --2- 172.21.15.47:0/4247889063 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28880a3120 0x7f28880a55e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x564d779680d0 tx=0x7f289c009500 comp rx=0 tx=0).stop 2024-03-21T11:47:19.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.803+0000 7f28a9800640 1 -- 172.21.15.47:0/4247889063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28a410b2d0 msgr2=0x7f28a4117af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:19.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.803+0000 7f28a9800640 1 --2- 172.21.15.47:0/4247889063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28a410b2d0 0x7f28a4117af0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f2890006fd0 tx=0x7f289000b8e0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.804+0000 7f28a9800640 1 -- 172.21.15.47:0/4247889063 shutdown_connections 2024-03-21T11:47:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.804+0000 7f28a9800640 1 --2- 172.21.15.47:0/4247889063 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28880a3120 0x7f28880a55e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.804+0000 7f28a9800640 1 --2- 172.21.15.47:0/4247889063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28a410dd70 0x7f28a4114120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.804+0000 7f28a9800640 1 --2- 172.21.15.47:0/4247889063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28a410b2d0 0x7f28a4117af0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.804+0000 7f28a9800640 1 --2- 172.21.15.47:0/4247889063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28a4108910 0x7f28a41175b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.804+0000 7f28a9800640 1 -- 172.21.15.47:0/4247889063 >> 172.21.15.47:0/4247889063 conn(0x7f28a406e500 msgr2=0x7f28a4109f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:19.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.805+0000 7f28a9800640 1 -- 172.21.15.47:0/4247889063 shutdown_connections 2024-03-21T11:47:19.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:19.805+0000 7f28a9800640 1 -- 172.21.15.47:0/4247889063 wait complete. 2024-03-21T11:47:19.844 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:47:19.845 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:47:19.993 INFO:tasks.workunit.client.0.smithi047.stdout: 10 254319 21.86 MB/sec execute 483 sec latency 244.583 ms 2024-03-21T11:47:20.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.208+0000 7f055ae00640 1 -- 172.21.15.47:0/3627207261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05541038e0 msgr2=0x7f0554103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:20.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.208+0000 7f055ae00640 1 --2- 172.21.15.47:0/3627207261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05541038e0 0x7f0554103d00 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f054c009860 tx=0x7f054c02bb70 comp rx=0 tx=0).stop 2024-03-21T11:47:20.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.208+0000 7f055ae00640 1 -- 172.21.15.47:0/3627207261 shutdown_connections 2024-03-21T11:47:20.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.208+0000 7f055ae00640 1 --2- 172.21.15.47:0/3627207261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0554105490 0x7f055410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:20.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.209+0000 7f055ae00640 1 --2- 172.21.15.47:0/3627207261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0554104ad0 0x7f0554104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:20.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.209+0000 7f055ae00640 1 --2- 172.21.15.47:0/3627207261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05541038e0 0x7f0554103d00 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:20.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.209+0000 7f055ae00640 1 -- 172.21.15.47:0/3627207261 >> 172.21.15.47:0/3627207261 conn(0x7f05540fee90 msgr2=0x7f05541012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:20.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.210+0000 7f055ae00640 1 -- 172.21.15.47:0/3627207261 shutdown_connections 2024-03-21T11:47:20.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.210+0000 7f055ae00640 1 -- 172.21.15.47:0/3627207261 wait complete. 2024-03-21T11:47:20.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.211+0000 7f055ae00640 1 Processor -- start 2024-03-21T11:47:20.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.213+0000 7f055ae00640 1 -- start start 2024-03-21T11:47:20.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.213+0000 7f055ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05541038e0 0x7f05540700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:20.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.213+0000 7f055ae00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0554104ad0 0x7f0554070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:20.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.213+0000 7f055ae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0554105490 0x7f05541a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:20.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.213+0000 7f055ae00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055410f580 con 0x7f0554104ad0 2024-03-21T11:47:20.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.213+0000 7f055ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055410f400 con 0x7f05541038e0 2024-03-21T11:47:20.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.213+0000 7f055ae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055410f700 con 0x7f0554105490 2024-03-21T11:47:20.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.214+0000 7f0553e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05541038e0 0x7f05540700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:20.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.214+0000 7f0553e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05541038e0 0x7f05540700e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59560/0 (socket says 172.21.15.47:59560) 2024-03-21T11:47:20.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.214+0000 7f0553e00640 1 -- 172.21.15.47:0/1679308907 learned_addr learned my addr 172.21.15.47:0/1679308907 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:20.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.214+0000 7f0558e00640 1 --2- 172.21.15.47:0/1679308907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0554105490 0x7f05541a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:20.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.214+0000 7f0553400640 1 --2- 172.21.15.47:0/1679308907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0554104ad0 0x7f0554070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:20.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.215+0000 7f0553e00640 1 -- 172.21.15.47:0/1679308907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0554105490 msgr2=0x7f05541a9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:20.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.215+0000 7f0553e00640 1 --2- 172.21.15.47:0/1679308907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0554105490 0x7f05541a9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:20.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.215+0000 7f0553e00640 1 -- 172.21.15.47:0/1679308907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0554104ad0 msgr2=0x7f0554070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:20.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.215+0000 7f0553e00640 1 --2- 172.21.15.47:0/1679308907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0554104ad0 0x7f0554070620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:20.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.215+0000 7f0553e00640 1 -- 172.21.15.47:0/1679308907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05541aa1d0 con 0x7f05541038e0 2024-03-21T11:47:20.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.216+0000 7f0553e00640 1 --2- 172.21.15.47:0/1679308907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05541038e0 0x7f05540700e0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f054c009780 tx=0x7f054c032440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:20.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.216+0000 7f0550c00640 1 -- 172.21.15.47:0/1679308907 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c039040 con 0x7f05541038e0 2024-03-21T11:47:20.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.217+0000 7f055ae00640 1 -- 172.21.15.47:0/1679308907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0554075ae0 con 0x7f05541038e0 2024-03-21T11:47:20.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.217+0000 7f055ae00640 1 -- 172.21.15.47:0/1679308907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0554076020 con 0x7f05541038e0 2024-03-21T11:47:20.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.217+0000 7f0550c00640 1 -- 172.21.15.47:0/1679308907 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f054c02c9f0 con 0x7f05541038e0 2024-03-21T11:47:20.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.217+0000 7f0550c00640 1 -- 172.21.15.47:0/1679308907 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c039040 con 0x7f05541038e0 2024-03-21T11:47:20.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.218+0000 7f055ae00640 1 -- 172.21.15.47:0/1679308907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05300052c0 con 0x7f05541038e0 2024-03-21T11:47:20.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.225+0000 7f0550c00640 1 -- 172.21.15.47:0/1679308907 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f054c005050 con 0x7f05541038e0 2024-03-21T11:47:20.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.228+0000 7f0550c00640 1 --2- 172.21.15.47:0/1679308907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05380a3260 0x7f05380a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:20.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.228+0000 7f0550c00640 1 -- 172.21.15.47:0/1679308907 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f054c101ee0 con 0x7f05541038e0 2024-03-21T11:47:20.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.228+0000 7f0553400640 1 --2- 172.21.15.47:0/1679308907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05380a3260 0x7f05380a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.230+0000 7f0553400640 1 --2- 172.21.15.47:0/1679308907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05380a3260 0x7f05380a5720 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0544005fd0 tx=0x7f0544005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:20.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.232+0000 7f0550c00640 1 -- 172.21.15.47:0/1679308907 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f054c05f070 con 0x7f05541038e0 2024-03-21T11:47:20.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.695+0000 7f055ae00640 1 -- 172.21.15.47:0/1679308907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05300055b0 con 0x7f05541038e0 2024-03-21T11:47:20.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.700+0000 7f0550c00640 1 -- 172.21.15.47:0/1679308907 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f054c0333a0 con 0x7f05541038e0 2024-03-21T11:47:20.701 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:47:20.702 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:45:39.125075+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_32623":{"gid":32623,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4278877119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4278877119},{"type":"v1","addr":"172.21.15.100:6835","nonce":4278877119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24523,24524,24535,24553,24581]},"id":1}]} 2024-03-21T11:47:20.702 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 41 2024-03-21T11:47:20.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.705+0000 7f055ae00640 1 -- 172.21.15.47:0/1679308907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05380a3260 msgr2=0x7f05380a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:20.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.705+0000 7f055ae00640 1 --2- 172.21.15.47:0/1679308907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05380a3260 0x7f05380a5720 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0544005fd0 tx=0x7f0544005c70 comp rx=0 tx=0).stop 2024-03-21T11:47:20.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.705+0000 7f055ae00640 1 -- 172.21.15.47:0/1679308907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05541038e0 msgr2=0x7f05540700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:20.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.705+0000 7f055ae00640 1 --2- 172.21.15.47:0/1679308907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05541038e0 0x7f05540700e0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f054c009780 tx=0x7f054c032440 comp rx=0 tx=0).stop 2024-03-21T11:47:20.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.705+0000 7f055ae00640 1 -- 172.21.15.47:0/1679308907 shutdown_connections 2024-03-21T11:47:20.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.705+0000 7f055ae00640 1 --2- 172.21.15.47:0/1679308907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f05380a3260 0x7f05380a5720 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:20.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.705+0000 7f055ae00640 1 --2- 172.21.15.47:0/1679308907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0554105490 0x7f05541a9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:20.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.705+0000 7f055ae00640 1 --2- 172.21.15.47:0/1679308907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0554104ad0 0x7f0554070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:20.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.705+0000 7f055ae00640 1 --2- 172.21.15.47:0/1679308907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05541038e0 0x7f05540700e0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:20.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.705+0000 7f055ae00640 1 -- 172.21.15.47:0/1679308907 >> 172.21.15.47:0/1679308907 conn(0x7f05540fee90 msgr2=0x7f0554107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:20.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.706+0000 7f055ae00640 1 -- 172.21.15.47:0/1679308907 shutdown_connections 2024-03-21T11:47:20.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:20.706+0000 7f055ae00640 1 -- 172.21.15.47:0/1679308907 wait complete. 2024-03-21T11:47:20.741 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:47:20.742 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:47:20.992 INFO:tasks.workunit.client.0.smithi047.stdout: 10 254342 21.82 MB/sec execute 484 sec latency 1050.179 ms 2024-03-21T11:47:21.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.094+0000 7fcdae600640 1 -- 172.21.15.47:0/4037188529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8102790 msgr2=0x7fcda8102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.094+0000 7fcdae600640 1 --2- 172.21.15.47:0/4037188529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8102790 0x7fcda8102c10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcd98009900 tx=0x7fcd9802bc40 comp rx=0 tx=0).stop 2024-03-21T11:47:21.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.095+0000 7fcdae600640 1 -- 172.21.15.47:0/4037188529 shutdown_connections 2024-03-21T11:47:21.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.095+0000 7fcdae600640 1 --2- 172.21.15.47:0/4037188529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda8103150 0x7fcda81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.095+0000 7fcdae600640 1 --2- 172.21.15.47:0/4037188529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8102790 0x7fcda8102c10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.095+0000 7fcdae600640 1 --2- 172.21.15.47:0/4037188529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcda81015a0 0x7fcda81019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.095+0000 7fcdae600640 1 -- 172.21.15.47:0/4037188529 >> 172.21.15.47:0/4037188529 conn(0x7fcda80fcb10 msgr2=0x7fcda80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:21.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.095+0000 7fcdae600640 1 -- 172.21.15.47:0/4037188529 shutdown_connections 2024-03-21T11:47:21.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.095+0000 7fcdae600640 1 -- 172.21.15.47:0/4037188529 wait complete. 2024-03-21T11:47:21.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.099+0000 7fcdae600640 1 Processor -- start 2024-03-21T11:47:21.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.099+0000 7fcdae600640 1 -- start start 2024-03-21T11:47:21.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.099+0000 7fcdae600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda81015a0 0x7fcda8199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.099+0000 7fcdae600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8102790 0x7fcda8199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.099+0000 7fcdae600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcda8103150 0x7fcda81a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.099+0000 7fcdae600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda810d230 con 0x7fcda8102790 2024-03-21T11:47:21.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.099+0000 7fcdae600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda810d0b0 con 0x7fcda81015a0 2024-03-21T11:47:21.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.099+0000 7fcdae600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda810d3b0 con 0x7fcda8103150 2024-03-21T11:47:21.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.100+0000 7fcdad200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda81015a0 0x7fcda8199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.100+0000 7fcdadc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcda8103150 0x7fcda81a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.100+0000 7fcdad200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda81015a0 0x7fcda8199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59572/0 (socket says 172.21.15.47:59572) 2024-03-21T11:47:21.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.100+0000 7fcdad200640 1 -- 172.21.15.47:0/4140133709 learned_addr learned my addr 172.21.15.47:0/4140133709 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:21.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.100+0000 7fcda7e00640 1 --2- 172.21.15.47:0/4140133709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8102790 0x7fcda8199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.101+0000 7fcdadc00640 1 -- 172.21.15.47:0/4140133709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda81015a0 msgr2=0x7fcda8199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.101+0000 7fcdadc00640 1 --2- 172.21.15.47:0/4140133709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda81015a0 0x7fcda8199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.101+0000 7fcdadc00640 1 -- 172.21.15.47:0/4140133709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8102790 msgr2=0x7fcda8199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.101+0000 7fcdadc00640 1 --2- 172.21.15.47:0/4140133709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8102790 0x7fcda8199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.101+0000 7fcdadc00640 1 -- 172.21.15.47:0/4140133709 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda81a16a0 con 0x7fcda8103150 2024-03-21T11:47:21.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.103+0000 7fcdadc00640 1 --2- 172.21.15.47:0/4140133709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcda8103150 0x7fcda81a1030 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400e970 tx=0x7fcd9400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:21.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.103+0000 7fcda5600640 1 -- 172.21.15.47:0/4140133709 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9400ccc0 con 0x7fcda8103150 2024-03-21T11:47:21.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.103+0000 7fcda5600640 1 -- 172.21.15.47:0/4140133709 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd94005020 con 0x7fcda8103150 2024-03-21T11:47:21.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.103+0000 7fcda5600640 1 -- 172.21.15.47:0/4140133709 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd94004a00 con 0x7fcda8103150 2024-03-21T11:47:21.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.103+0000 7fcdae600640 1 -- 172.21.15.47:0/4140133709 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda81a1990 con 0x7fcda8103150 2024-03-21T11:47:21.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.104+0000 7fcdae600640 1 -- 172.21.15.47:0/4140133709 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda81a1ee0 con 0x7fcda8103150 2024-03-21T11:47:21.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.111+0000 7fcda5600640 1 -- 172.21.15.47:0/4140133709 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcd940051d0 con 0x7fcda8103150 2024-03-21T11:47:21.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.113+0000 7fcdb2874740 1 Processor -- start 2024-03-21T11:47:21.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.113+0000 7fcdb2874740 1 -- start start 2024-03-21T11:47:21.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.113+0000 7fcdb2874740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9887b9f0 0x558e9887be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.113+0000 7fcdb2874740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887cbc0 0x558e9887d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.113+0000 7fcdb2874740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887d580 0x558e98883e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.114+0000 7fcdb2874740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e987e02c0 con 0x558e9887d580 2024-03-21T11:47:21.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.114+0000 7fcdb2874740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e98788c90 con 0x558e9887b9f0 2024-03-21T11:47:21.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.114+0000 7fcdb2874740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e987dc360 con 0x558e9887cbc0 2024-03-21T11:47:21.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.114+0000 7fcda7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9887b9f0 0x558e9887be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.115+0000 7fcdad200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887cbc0 0x558e9887d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.115+0000 7fcdad200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887cbc0 0x558e9887d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47922/0 (socket says 172.21.15.47:47922) 2024-03-21T11:47:21.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.115+0000 7fcdad200640 1 -- 172.21.15.47:0/3728754533 learned_addr learned my addr 172.21.15.47:0/3728754533 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:21.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.115+0000 7fcdad200640 1 -- 172.21.15.47:0/3728754533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9887b9f0 msgr2=0x558e9887be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.115+0000 7fcdad200640 1 --2- 172.21.15.47:0/3728754533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9887b9f0 0x558e9887be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.115+0000 7fcdad200640 1 -- 172.21.15.47:0/3728754533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887d580 msgr2=0x558e98883e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.115+0000 7fcdad200640 1 --2- 172.21.15.47:0/3728754533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887d580 0x558e98883e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.115+0000 7fcdad200640 1 -- 172.21.15.47:0/3728754533 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e987ad080 con 0x558e9887cbc0 2024-03-21T11:47:21.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.116+0000 7fcda5600640 1 --2- 172.21.15.47:0/4140133709 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcd8c0a2fa0 0x7fcd8c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.116+0000 7fcda5600640 1 -- 172.21.15.47:0/4140133709 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcd940cf780 con 0x7fcda8103150 2024-03-21T11:47:21.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.117+0000 7fcdad200640 1 --2- 172.21.15.47:0/3728754533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887cbc0 0x558e9887d040 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fcda00114b0 tx=0x7fcda0011970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d87745999b2726e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:21.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.117+0000 7fcd9e000640 1 -- 172.21.15.47:0/3728754533 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0012770 con 0x558e9887cbc0 2024-03-21T11:47:21.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.117+0000 7fcd9e000640 1 -- 172.21.15.47:0/3728754533 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcda0012df0 con 0x558e9887cbc0 2024-03-21T11:47:21.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.118+0000 7fcd9e000640 1 -- 172.21.15.47:0/3728754533 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda00193a0 con 0x558e9887cbc0 2024-03-21T11:47:21.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.118+0000 7fcdb2874740 1 -- 172.21.15.47:0/3728754533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887cbc0 msgr2=0x558e9887d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.118+0000 7fcdb2874740 1 --2- 172.21.15.47:0/3728754533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887cbc0 0x558e9887d040 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fcda00114b0 tx=0x7fcda0011970 comp rx=0 tx=0).stop 2024-03-21T11:47:21.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.120+0000 7fcdb2874740 1 -- 172.21.15.47:0/3728754533 shutdown_connections 2024-03-21T11:47:21.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.120+0000 7fcdb2874740 1 --2- 172.21.15.47:0/3728754533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887d580 0x558e98883e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.120+0000 7fcdb2874740 1 --2- 172.21.15.47:0/3728754533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887cbc0 0x558e9887d040 secure :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fcda00114b0 tx=0x7fcda0011970 comp rx=0 tx=0).stop 2024-03-21T11:47:21.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.120+0000 7fcdb2874740 1 --2- 172.21.15.47:0/3728754533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9887b9f0 0x558e9887be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.120+0000 7fcdb2874740 1 -- 172.21.15.47:0/3728754533 >> 172.21.15.47:0/3728754533 conn(0x558e98876de0 msgr2=0x558e988771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:21.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.122+0000 7fcdb2874740 1 -- 172.21.15.47:0/3728754533 shutdown_connections 2024-03-21T11:47:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.123+0000 7fcdb2874740 1 -- 172.21.15.47:0/3728754533 wait complete. 2024-03-21T11:47:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.123+0000 7fcda7e00640 1 --2- 172.21.15.47:0/4140133709 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcd8c0a2fa0 0x7fcd8c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.124+0000 7fcdb2874740 1 -- start start 2024-03-21T11:47:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.124+0000 7fcdb2874740 1 Processor -- start 2024-03-21T11:47:21.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.124+0000 7fcda7e00640 1 --2- 172.21.15.47:0/4140133709 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcd8c0a2fa0 0x7fcd8c0a5460 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fcd98036040 tx=0x7fcd98000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:21.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.127+0000 7fcdb2874740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887b9f0 0x558e9888e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.127+0000 7fcdb2874740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887d580 0x558e9888eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.127+0000 7fcdb2874740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e98893b60 0x558e98895f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.127+0000 7fcdb2874740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e98884590 con 0x558e9887b9f0 2024-03-21T11:47:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.127+0000 7fcdb2874740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e9879c0d0 con 0x558e98893b60 2024-03-21T11:47:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.127+0000 7fcdb2874740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e98884710 con 0x558e9887d580 2024-03-21T11:47:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.128+0000 7fcdb2874740 10 client.0 _ll_register_callbacks cb 0x558e987a0e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.128+0000 7fcda7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887d580 0x558e9888eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.128+0000 7fcdadc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e98893b60 0x558e98895f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.128+0000 7fcdadc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e98893b60 0x558e98895f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59586/0 (socket says 172.21.15.47:59586) 2024-03-21T11:47:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.128+0000 7fcdadc00640 1 -- 172.21.15.47:0/534322030 learned_addr learned my addr 172.21.15.47:0/534322030 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.129+0000 7fcdad200640 1 --2- 172.21.15.47:0/534322030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887b9f0 0x558e9888e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.129+0000 7fcdad200640 1 -- 172.21.15.47:0/534322030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887d580 msgr2=0x558e9888eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.129+0000 7fcdad200640 1 --2- 172.21.15.47:0/534322030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887d580 0x558e9888eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.131+0000 7fcdad200640 1 -- 172.21.15.47:0/534322030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e98893b60 msgr2=0x558e98895f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.131+0000 7fcdad200640 1 --2- 172.21.15.47:0/534322030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e98893b60 0x558e98895f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.131+0000 7fcdad200640 1 -- 172.21.15.47:0/534322030 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e984f43f0 con 0x558e9887b9f0 2024-03-21T11:47:21.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.131+0000 7fcdad200640 1 --2- 172.21.15.47:0/534322030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887b9f0 0x558e9888e5f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcda000b000 tx=0x7fcda0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a683520b54f4102b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:21.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.131+0000 7fcd92a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:47:21.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.132+0000 7fcd92a00640 1 -- 172.21.15.47:0/534322030 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda001a800 con 0x558e9887b9f0 2024-03-21T11:47:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.132+0000 7fcd92a00640 1 -- 172.21.15.47:0/534322030 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcda0012570 con 0x558e9887b9f0 2024-03-21T11:47:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.132+0000 7fcdb2874740 1 -- 172.21.15.47:0/534322030 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcda0012df0 con 0x558e9887b9f0 2024-03-21T11:47:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.133+0000 7fcd92a00640 1 -- 172.21.15.47:0/534322030 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcda0012770 con 0x558e9887b9f0 2024-03-21T11:47:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.133+0000 7fcdb2874740 10 client.22248 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:21.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.133+0000 7fcdb2874740 1 -- 172.21.15.47:0/534322030 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558e985be7d0 con 0x558e9887b9f0 2024-03-21T11:47:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.136+0000 7fcd92a00640 1 -- 172.21.15.47:0/534322030 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcda0012df0 con 0x558e9887b9f0 2024-03-21T11:47:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.136+0000 7fcd92a00640 1 -- 172.21.15.47:0/534322030 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcda000f040 con 0x558e9887b9f0 2024-03-21T11:47:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.136+0000 7fcdb2874740 10 client.22248 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.136+0000 7fcdb2874740 10 client.22248 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:47:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.136+0000 7fcdb2874740 20 client.22248 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.136+0000 7fcdb2874740 1 --2- 172.21.15.47:0/534322030 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558e9889a1b0 0x558e9889c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.136+0000 7fcdb2874740 4 client.22248 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:21.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.136+0000 7fcdb2874740 1 -- 172.21.15.47:0/534322030 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e987ed800 con 0x558e9889a1b0 2024-03-21T11:47:21.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.137+0000 7fcda7e00640 1 --2- 172.21.15.47:0/534322030 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558e9889a1b0 0x558e9889c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.143+0000 7fcd92a00640 10 client.22248 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:47:21.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.143+0000 7fcda7e00640 1 --2- 172.21.15.47:0/534322030 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558e9889a1b0 0x558e9889c5a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e61c0dbf5f287461 server_cookie=3c36ef7e846fb721 in_seq=0 out_seq=0 2024-03-21T11:47:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.146+0000 7fcd92a00640 1 -- 172.21.15.47:0/534322030 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558e987ed800 con 0x558e9889a1b0 2024-03-21T11:47:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.146+0000 7fcd92a00640 10 client.22248 handle_command_reply: tid=0 2024-03-21T11:47:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.147+0000 7fcdb2874740 1 client.22248 shutdown 2024-03-21T11:47:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.147+0000 7fcdb2874740 1 -- 172.21.15.47:0/534322030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887b9f0 msgr2=0x558e9888e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.147+0000 7fcdb2874740 1 --2- 172.21.15.47:0/534322030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887b9f0 0x558e9888e5f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcda000b000 tx=0x7fcda0004a40 comp rx=0 tx=0).stop 2024-03-21T11:47:21.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.150+0000 7fcdb2874740 1 -- 172.21.15.47:0/534322030 shutdown_connections 2024-03-21T11:47:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.150+0000 7fcdb2874740 1 --2- 172.21.15.47:0/534322030 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558e9889a1b0 0x558e9889c5a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.150+0000 7fcdb2874740 1 --2- 172.21.15.47:0/534322030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e98893b60 0x558e98895f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.150+0000 7fcdb2874740 1 --2- 172.21.15.47:0/534322030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887d580 0x558e9888eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.150+0000 7fcd92a00640 0 client.22248 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:47:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.150+0000 7fcdb2874740 1 --2- 172.21.15.47:0/534322030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887b9f0 0x558e9888e5f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.150+0000 7fcdb2874740 1 -- 172.21.15.47:0/534322030 >> 172.21.15.47:0/534322030 conn(0x558e98876de0 msgr2=0x558e98876160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.150+0000 7fcdb2874740 1 -- 172.21.15.47:0/534322030 shutdown_connections 2024-03-21T11:47:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.150+0000 7fcdb2874740 1 -- 172.21.15.47:0/534322030 wait complete. 2024-03-21T11:47:21.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.151+0000 7fcdb2874740 20 client.22248 trim_cache size 0 max 16384 2024-03-21T11:47:21.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.191+0000 7fcdb2874740 1 Processor -- start 2024-03-21T11:47:21.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.191+0000 7fcdb2874740 1 -- start start 2024-03-21T11:47:21.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.193+0000 7fcdb2874740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887b9f0 0x558e988758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.193+0000 7fcdb2874740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9887d580 0x558e98875e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.193+0000 7fcdb2874740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e98893b60 0x558e98876350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.193+0000 7fcdb2874740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda81a7cd0 con 0x558e98893b60 2024-03-21T11:47:21.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.193+0000 7fcdb2874740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda81a9650 con 0x558e9887d580 2024-03-21T11:47:21.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.193+0000 7fcdb2874740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda81a8330 con 0x558e9887b9f0 2024-03-21T11:47:21.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.193+0000 7fcdad200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887b9f0 0x558e988758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.194+0000 7fcdadc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e98893b60 0x558e98876350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.194+0000 7fcdad200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887b9f0 0x558e988758d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47948/0 (socket says 172.21.15.47:47948) 2024-03-21T11:47:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.194+0000 7fcdad200640 1 -- 172.21.15.47:0/1742006578 learned_addr learned my addr 172.21.15.47:0/1742006578 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.194+0000 7fcdadc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e98893b60 0x558e98876350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37174/0 (socket says 172.21.15.47:37174) 2024-03-21T11:47:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.194+0000 7fcda7e00640 1 --2- 172.21.15.47:0/1742006578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9887d580 0x558e98875e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.194+0000 7fcdad200640 1 -- 172.21.15.47:0/1742006578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9887d580 msgr2=0x558e98875e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.194+0000 7fcdad200640 1 --2- 172.21.15.47:0/1742006578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9887d580 0x558e98875e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.194+0000 7fcdad200640 1 -- 172.21.15.47:0/1742006578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e98893b60 msgr2=0x558e98876350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.194+0000 7fcdad200640 1 --2- 172.21.15.47:0/1742006578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e98893b60 0x558e98876350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.194+0000 7fcdad200640 1 -- 172.21.15.47:0/1742006578 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e9888d6b0 con 0x558e9887b9f0 2024-03-21T11:47:21.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.194+0000 7fcdadc00640 1 --2- 172.21.15.47:0/1742006578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e98893b60 0x558e98876350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:47:21.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.196+0000 7fcdad200640 1 --2- 172.21.15.47:0/1742006578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887b9f0 0x558e988758d0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fcda000a010 tx=0x7fcda00094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76a320f1c5051de4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:21.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.202+0000 7fcd9d600640 1 -- 172.21.15.47:0/1742006578 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0012570 con 0x558e9887b9f0 2024-03-21T11:47:21.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.202+0000 7fcd9d600640 1 -- 172.21.15.47:0/1742006578 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcda002b430 con 0x558e9887b9f0 2024-03-21T11:47:21.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.202+0000 7fcd9d600640 1 -- 172.21.15.47:0/1742006578 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda001a860 con 0x558e9887b9f0 2024-03-21T11:47:21.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.203+0000 7fcdb2874740 1 -- 172.21.15.47:0/1742006578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887b9f0 msgr2=0x558e988758d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.203+0000 7fcdb2874740 1 --2- 172.21.15.47:0/1742006578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887b9f0 0x558e988758d0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fcda000a010 tx=0x7fcda00094a0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.205+0000 7fcdb2874740 1 -- 172.21.15.47:0/1742006578 shutdown_connections 2024-03-21T11:47:21.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.205+0000 7fcdb2874740 1 --2- 172.21.15.47:0/1742006578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e98893b60 0x558e98876350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.205+0000 7fcdb2874740 1 --2- 172.21.15.47:0/1742006578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9887d580 0x558e98875e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.205+0000 7fcdb2874740 1 --2- 172.21.15.47:0/1742006578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887b9f0 0x558e988758d0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.205+0000 7fcdb2874740 1 -- 172.21.15.47:0/1742006578 >> 172.21.15.47:0/1742006578 conn(0x558e98876de0 msgr2=0x558e988771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.205+0000 7fcdb2874740 1 -- 172.21.15.47:0/1742006578 shutdown_connections 2024-03-21T11:47:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.205+0000 7fcdb2874740 1 -- 172.21.15.47:0/1742006578 wait complete. 2024-03-21T11:47:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.205+0000 7fcdb2874740 1 -- start start 2024-03-21T11:47:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.206+0000 7fcdb2874740 1 Processor -- start 2024-03-21T11:47:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.207+0000 7fcdb2874740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887b9f0 0x558e988ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.207+0000 7fcdb2874740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887d580 0x558e988a8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.207+0000 7fcdb2874740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e98893b60 0x558e988a9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.207+0000 7fcdb2874740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda81a8ff0 con 0x558e9887d580 2024-03-21T11:47:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.207+0000 7fcdb2874740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda81a8990 con 0x558e98893b60 2024-03-21T11:47:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.207+0000 7fcdb2874740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda81a5eb0 con 0x558e9887b9f0 2024-03-21T11:47:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.207+0000 7fcdb2874740 10 client.0 _ll_register_callbacks cb 0x7fcda0009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:21.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.208+0000 7fcdad200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887b9f0 0x558e988ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.208+0000 7fcdadc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e98893b60 0x558e988a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.208+0000 7fcdadc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e98893b60 0x558e988a9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59596/0 (socket says 172.21.15.47:59596) 2024-03-21T11:47:21.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.208+0000 7fcdadc00640 1 -- 172.21.15.47:0/2318217323 learned_addr learned my addr 172.21.15.47:0/2318217323 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:21.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.208+0000 7fcda7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887d580 0x558e988a8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.208+0000 7fcdad200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887b9f0 0x558e988ac0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47964/0 (socket says 172.21.15.47:47964) 2024-03-21T11:47:21.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.208+0000 7fcdadc00640 1 -- 172.21.15.47:0/2318217323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887b9f0 msgr2=0x558e988ac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.208+0000 7fcdadc00640 1 --2- 172.21.15.47:0/2318217323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887b9f0 0x558e988ac0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.208+0000 7fcdadc00640 1 -- 172.21.15.47:0/2318217323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887d580 msgr2=0x558e988a8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.209+0000 7fcdadc00640 1 --2- 172.21.15.47:0/2318217323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887d580 0x558e988a8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.209+0000 7fcdadc00640 1 -- 172.21.15.47:0/2318217323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e988ac6d0 con 0x558e98893b60 2024-03-21T11:47:21.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.209+0000 7fcdad200640 1 --2- 172.21.15.47:0/2318217323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887b9f0 0x558e988ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:21.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.209+0000 7fcda7e00640 1 --2- 172.21.15.47:0/2318217323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887d580 0x558e988a8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:21.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.211+0000 7fcdadc00640 1 --2- 172.21.15.47:0/2318217323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e98893b60 0x558e988a9080 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fcd940ccf40 tx=0x7fcd940056a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a30e6c13837bec95 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:21.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.211+0000 7fcd92a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:21.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.211+0000 7fcd92a00640 1 -- 172.21.15.47:0/2318217323 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd940b9060 con 0x558e98893b60 2024-03-21T11:47:21.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.211+0000 7fcd92a00640 1 -- 172.21.15.47:0/2318217323 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcd940cf430 con 0x558e98893b60 2024-03-21T11:47:21.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.212+0000 7fcdb2874740 1 -- 172.21.15.47:0/2318217323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcda002b430 con 0x558e98893b60 2024-03-21T11:47:21.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.213+0000 7fcd92a00640 1 -- 172.21.15.47:0/2318217323 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd940b1030 con 0x558e98893b60 2024-03-21T11:47:21.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.213+0000 7fcdb2874740 10 client.33098 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:21.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.213+0000 7fcdb2874740 1 -- 172.21.15.47:0/2318217323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558e98885b10 con 0x558e98893b60 2024-03-21T11:47:21.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.214+0000 7fcd92a00640 1 -- 172.21.15.47:0/2318217323 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcda002b430 con 0x558e98893b60 2024-03-21T11:47:21.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.215+0000 7fcd92a00640 1 -- 172.21.15.47:0/2318217323 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcd94004ba0 con 0x558e98893b60 2024-03-21T11:47:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.215+0000 7fcdb2874740 10 client.33098 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.215+0000 7fcdb2874740 10 client.33098 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:47:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.215+0000 7fcdb2874740 20 client.33098 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.215+0000 7fcdb2874740 1 --2- 172.21.15.47:0/2318217323 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558e9889a1b0 0x558e988884e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.215+0000 7fcdb2874740 4 client.33098 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:47:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.215+0000 7fcdb2874740 1 -- 172.21.15.47:0/2318217323 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558e987e7b50 con 0x558e9889a1b0 2024-03-21T11:47:21.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.216+0000 7fcdad200640 1 --2- 172.21.15.47:0/2318217323 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558e9889a1b0 0x558e988884e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.217+0000 7fcdad200640 1 --2- 172.21.15.47:0/2318217323 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558e9889a1b0 0x558e988884e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d74b657dd10068 server_cookie=651ec88a7b868781 in_seq=0 out_seq=0 2024-03-21T11:47:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.218+0000 7fcd92a00640 10 client.33098 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:47:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.218+0000 7fcd92a00640 1 -- 172.21.15.47:0/2318217323 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558e987e7b50 con 0x558e9889a1b0 2024-03-21T11:47:21.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.218+0000 7fcd92a00640 10 client.33098 handle_command_reply: tid=0 2024-03-21T11:47:21.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.220+0000 7fcdb2874740 1 client.33098 shutdown 2024-03-21T11:47:21.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.220+0000 7fcdb2874740 1 -- 172.21.15.47:0/2318217323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e98893b60 msgr2=0x558e988a9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.220+0000 7fcdb2874740 1 --2- 172.21.15.47:0/2318217323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e98893b60 0x558e988a9080 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fcd940ccf40 tx=0x7fcd940056a0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.220+0000 7fcdb2874740 1 -- 172.21.15.47:0/2318217323 shutdown_connections 2024-03-21T11:47:21.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.220+0000 7fcdb2874740 1 --2- 172.21.15.47:0/2318217323 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x558e9889a1b0 0x558e988884e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.220+0000 7fcdb2874740 1 --2- 172.21.15.47:0/2318217323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e98893b60 0x558e988a9080 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.220+0000 7fcdb2874740 1 --2- 172.21.15.47:0/2318217323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558e9887d580 0x558e988a8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.220+0000 7fcdb2874740 1 --2- 172.21.15.47:0/2318217323 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e9887b9f0 0x558e988ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.220+0000 7fcdb2874740 1 -- 172.21.15.47:0/2318217323 >> 172.21.15.47:0/2318217323 conn(0x558e98876de0 msgr2=0x558e98876160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:21.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.220+0000 7fcd92a00640 0 client.33098 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:47:21.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.220+0000 7fcdb2874740 1 -- 172.21.15.47:0/2318217323 shutdown_connections 2024-03-21T11:47:21.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.220+0000 7fcdb2874740 1 -- 172.21.15.47:0/2318217323 wait complete. 2024-03-21T11:47:21.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.221+0000 7fcdb2874740 20 client.33098 trim_cache size 0 max 16384 2024-03-21T11:47:21.222 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:47:21.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.225+0000 7fcdae600640 1 -- 172.21.15.47:0/4140133709 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcd8c0a2fa0 msgr2=0x7fcd8c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.225+0000 7fcdae600640 1 --2- 172.21.15.47:0/4140133709 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcd8c0a2fa0 0x7fcd8c0a5460 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fcd98036040 tx=0x7fcd98000bc0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.225+0000 7fcdae600640 1 -- 172.21.15.47:0/4140133709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcda8103150 msgr2=0x7fcda81a1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.225+0000 7fcdae600640 1 --2- 172.21.15.47:0/4140133709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcda8103150 0x7fcda81a1030 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400e970 tx=0x7fcd9400ee30 comp rx=0 tx=0).stop 2024-03-21T11:47:21.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.226+0000 7fcdae600640 1 -- 172.21.15.47:0/4140133709 shutdown_connections 2024-03-21T11:47:21.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.226+0000 7fcdae600640 1 --2- 172.21.15.47:0/4140133709 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcd8c0a2fa0 0x7fcd8c0a5460 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.226+0000 7fcdae600640 1 --2- 172.21.15.47:0/4140133709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcda8103150 0x7fcda81a1030 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.226+0000 7fcdae600640 1 --2- 172.21.15.47:0/4140133709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcda8102790 0x7fcda8199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.226+0000 7fcdae600640 1 --2- 172.21.15.47:0/4140133709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcda81015a0 0x7fcda8199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.226+0000 7fcdae600640 1 -- 172.21.15.47:0/4140133709 >> 172.21.15.47:0/4140133709 conn(0x7fcda80fcb10 msgr2=0x7fcda80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:21.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.226+0000 7fcdae600640 1 -- 172.21.15.47:0/4140133709 shutdown_connections 2024-03-21T11:47:21.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.226+0000 7fcdae600640 1 -- 172.21.15.47:0/4140133709 wait complete. 2024-03-21T11:47:21.258 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:47:21.258 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:47:21.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:21 smithi047 ceph-mon[31351]: pgmap v1121: 129 pgs: 129 active+clean; 712 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.9 MiB/s wr, 530 op/s 2024-03-21T11:47:21.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:21 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1679308907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:47:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.602+0000 7fb188800640 1 -- 172.21.15.47:0/2150189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1801038e0 msgr2=0x7fb180103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.602+0000 7fb188800640 1 --2- 172.21.15.47:0/2150189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1801038e0 0x7fb180103d00 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb174009900 tx=0x7fb17402bc30 comp rx=0 tx=0).stop 2024-03-21T11:47:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.603+0000 7fb188800640 1 -- 172.21.15.47:0/2150189 shutdown_connections 2024-03-21T11:47:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.603+0000 7fb188800640 1 --2- 172.21.15.47:0/2150189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb180105490 0x7fb18010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.603+0000 7fb188800640 1 --2- 172.21.15.47:0/2150189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb180104ad0 0x7fb180104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.603+0000 7fb188800640 1 --2- 172.21.15.47:0/2150189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1801038e0 0x7fb180103d00 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.603+0000 7fb188800640 1 -- 172.21.15.47:0/2150189 >> 172.21.15.47:0/2150189 conn(0x7fb1800fee50 msgr2=0x7fb180101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.603+0000 7fb188800640 1 -- 172.21.15.47:0/2150189 shutdown_connections 2024-03-21T11:47:21.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.603+0000 7fb188800640 1 -- 172.21.15.47:0/2150189 wait complete. 2024-03-21T11:47:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.604+0000 7fb188800640 1 Processor -- start 2024-03-21T11:47:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.605+0000 7fb188800640 1 -- start start 2024-03-21T11:47:21.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.605+0000 7fb188800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1801038e0 0x7fb180079450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.605+0000 7fb188800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb180104ad0 0x7fb180079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.606+0000 7fb188800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb180105490 0x7fb180075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.606+0000 7fb185e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1801038e0 0x7fb180079450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.606+0000 7fb185e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1801038e0 0x7fb180079450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37200/0 (socket says 172.21.15.47:37200) 2024-03-21T11:47:21.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.606+0000 7fb186800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb180105490 0x7fb180075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.606+0000 7fb186800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb180105490 0x7fb180075fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47988/0 (socket says 172.21.15.47:47988) 2024-03-21T11:47:21.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.609+0000 7fb185400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb180104ad0 0x7fb180079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.609+0000 7fb185400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb180104ad0 0x7fb180079990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59618/0 (socket says 172.21.15.47:59618) 2024-03-21T11:47:21.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.609+0000 7fb188800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18010f600 con 0x7fb1801038e0 2024-03-21T11:47:21.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.609+0000 7fb188800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18010f480 con 0x7fb180104ad0 2024-03-21T11:47:21.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.609+0000 7fb188800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18010f780 con 0x7fb180105490 2024-03-21T11:47:21.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.609+0000 7fb185e00640 1 -- 172.21.15.47:0/3069303950 learned_addr learned my addr 172.21.15.47:0/3069303950 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:21.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.610+0000 7fb186800640 1 -- 172.21.15.47:0/3069303950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb180104ad0 msgr2=0x7fb180079990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.610+0000 7fb186800640 1 --2- 172.21.15.47:0/3069303950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb180104ad0 0x7fb180079990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.610+0000 7fb186800640 1 -- 172.21.15.47:0/3069303950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1801038e0 msgr2=0x7fb180079450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.610+0000 7fb186800640 1 --2- 172.21.15.47:0/3069303950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1801038e0 0x7fb180079450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.610+0000 7fb186800640 1 -- 172.21.15.47:0/3069303950 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb180076880 con 0x7fb180105490 2024-03-21T11:47:21.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.610+0000 7fb186800640 1 --2- 172.21.15.47:0/3069303950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb180105490 0x7fb180075fc0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00be20 tx=0x7fb17c00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:21.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.612+0000 7fb17aa00640 1 -- 172.21.15.47:0/3069303950 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c019070 con 0x7fb180105490 2024-03-21T11:47:21.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.612+0000 7fb17aa00640 1 -- 172.21.15.47:0/3069303950 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb17c014920 con 0x7fb180105490 2024-03-21T11:47:21.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.612+0000 7fb17aa00640 1 -- 172.21.15.47:0/3069303950 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c012900 con 0x7fb180105490 2024-03-21T11:47:21.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.612+0000 7fb188800640 1 -- 172.21.15.47:0/3069303950 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1801a60a0 con 0x7fb180105490 2024-03-21T11:47:21.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.612+0000 7fb188800640 1 -- 172.21.15.47:0/3069303950 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1801a64b0 con 0x7fb180105490 2024-03-21T11:47:21.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.617+0000 7fb17aa00640 1 -- 172.21.15.47:0/3069303950 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb17c01c070 con 0x7fb180105490 2024-03-21T11:47:21.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.618+0000 7fb18b57a740 1 Processor -- start 2024-03-21T11:47:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.620+0000 7fb18b57a740 1 -- start start 2024-03-21T11:47:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.620+0000 7fb18b57a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a13111810 0x562a13111c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.620+0000 7fb18b57a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a131129e0 0x562a13112e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.620+0000 7fb18b57a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a13119c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.620+0000 7fb18b57a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a130762c0 con 0x562a131129e0 2024-03-21T11:47:21.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.620+0000 7fb18b57a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1301ec90 con 0x562a13111810 2024-03-21T11:47:21.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.620+0000 7fb18b57a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a13072360 con 0x562a131133a0 2024-03-21T11:47:21.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.621+0000 7fb185400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a13111810 0x562a13111c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.621+0000 7fb185400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a13111810 0x562a13111c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59622/0 (socket says 172.21.15.47:59622) 2024-03-21T11:47:21.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.621+0000 7fb185400640 1 -- 172.21.15.47:0/4012616272 learned_addr learned my addr 172.21.15.47:0/4012616272 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:21.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.621+0000 7fb185400640 1 --2- 172.21.15.47:0/4012616272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a13119c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.622+0000 7fb185400640 1 -- 172.21.15.47:0/4012616272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 msgr2=0x562a13119c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.622+0000 7fb185400640 1 --2- 172.21.15.47:0/4012616272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a13119c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.622+0000 7fb185400640 1 -- 172.21.15.47:0/4012616272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a131129e0 msgr2=0x562a13112e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.622+0000 7fb185400640 1 --2- 172.21.15.47:0/4012616272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a131129e0 0x562a13112e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.622+0000 7fb185400640 1 -- 172.21.15.47:0/4012616272 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a13043080 con 0x562a13111810 2024-03-21T11:47:21.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.623+0000 7fb185400640 1 --2- 172.21.15.47:0/4012616272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a13111810 0x562a13111c30 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb170009560 tx=0x7fb170005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1086ce55c2c6228 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:21.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.623+0000 7fb17aa00640 1 --2- 172.21.15.47:0/3069303950 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1680a3070 0x7fb1680a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.623+0000 7fb17aa00640 1 -- 172.21.15.47:0/3069303950 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb17c0cee90 con 0x7fb180105490 2024-03-21T11:47:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.624+0000 7fb16f000640 1 -- 172.21.15.47:0/4012616272 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170013e50 con 0x562a13111810 2024-03-21T11:47:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.624+0000 7fb16f000640 1 -- 172.21.15.47:0/4012616272 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1700131b0 con 0x562a13111810 2024-03-21T11:47:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.624+0000 7fb16f000640 1 -- 172.21.15.47:0/4012616272 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1700035f0 con 0x562a13111810 2024-03-21T11:47:21.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.625+0000 7fb185e00640 1 --2- 172.21.15.47:0/3069303950 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1680a3070 0x7fb1680a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.626+0000 7fb18b57a740 1 -- 172.21.15.47:0/4012616272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a13111810 msgr2=0x562a13111c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.626+0000 7fb18b57a740 1 --2- 172.21.15.47:0/4012616272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a13111810 0x562a13111c30 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb170009560 tx=0x7fb170005850 comp rx=0 tx=0).stop 2024-03-21T11:47:21.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.627+0000 7fb185e00640 1 --2- 172.21.15.47:0/3069303950 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1680a3070 0x7fb1680a5530 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb174006010 tx=0x7fb174004ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:21.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.627+0000 7fb18b57a740 1 -- 172.21.15.47:0/4012616272 shutdown_connections 2024-03-21T11:47:21.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.627+0000 7fb18b57a740 1 --2- 172.21.15.47:0/4012616272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a13119c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.627+0000 7fb18b57a740 1 --2- 172.21.15.47:0/4012616272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a131129e0 0x562a13112e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.627+0000 7fb18b57a740 1 --2- 172.21.15.47:0/4012616272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a13111810 0x562a13111c30 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.627+0000 7fb18b57a740 1 -- 172.21.15.47:0/4012616272 >> 172.21.15.47:0/4012616272 conn(0x562a1310cc00 msgr2=0x562a1310d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:21.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.628+0000 7fb18b57a740 1 -- 172.21.15.47:0/4012616272 shutdown_connections 2024-03-21T11:47:21.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.628+0000 7fb18b57a740 1 -- 172.21.15.47:0/4012616272 wait complete. 2024-03-21T11:47:21.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.628+0000 7fb18b57a740 1 -- start start 2024-03-21T11:47:21.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.629+0000 7fb18b57a740 1 Processor -- start 2024-03-21T11:47:21.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.630+0000 7fb18b57a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a13111810 0x562a1311bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.630+0000 7fb18b57a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a131129e0 0x562a1311c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.630+0000 7fb18b57a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a13123080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.630+0000 7fb18b57a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1311a3b0 con 0x562a13111810 2024-03-21T11:47:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.630+0000 7fb186800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a13123080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.631+0000 7fb186800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a13123080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:47998/0 (socket says 172.21.15.47:47998) 2024-03-21T11:47:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.631+0000 7fb186800640 1 -- 172.21.15.47:0/4257019592 learned_addr learned my addr 172.21.15.47:0/4257019592 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.630+0000 7fb185400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a131129e0 0x562a1311c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.631+0000 7fb18b57a740 1 -- 172.21.15.47:0/4257019592 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a130320d0 con 0x562a131129e0 2024-03-21T11:47:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.631+0000 7fb18b57a740 1 -- 172.21.15.47:0/4257019592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1311a530 con 0x562a131133a0 2024-03-21T11:47:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.631+0000 7fb18b57a740 10 client.0 _ll_register_callbacks cb 0x562a13036e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.632+0000 7fb186800640 1 -- 172.21.15.47:0/4257019592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a131129e0 msgr2=0x562a1311c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.632+0000 7fb186800640 1 --2- 172.21.15.47:0/4257019592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a131129e0 0x562a1311c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.632+0000 7fb186800640 1 -- 172.21.15.47:0/4257019592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a13111810 msgr2=0x562a1311bb00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:47:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.632+0000 7fb185e00640 1 --2- 172.21.15.47:0/4257019592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a13111810 0x562a1311bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.632+0000 7fb186800640 1 --2- 172.21.15.47:0/4257019592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a13111810 0x562a1311bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.632+0000 7fb186800640 1 -- 172.21.15.47:0/4257019592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a12d8a3f0 con 0x562a131133a0 2024-03-21T11:47:21.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.632+0000 7fb186800640 1 --2- 172.21.15.47:0/4257019592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a13123080 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fb17c097740 tx=0x7fb17c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a53f9a8f509ff49 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:21.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.632+0000 7fb163400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:47:21.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.633+0000 7fb18b57a740 1 -- 172.21.15.47:0/4257019592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1700131b0 con 0x562a131133a0 2024-03-21T11:47:21.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.633+0000 7fb163400640 1 -- 172.21.15.47:0/4257019592 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0a39b0 con 0x562a131133a0 2024-03-21T11:47:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.634+0000 7fb163400640 1 -- 172.21.15.47:0/4257019592 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb17c09a730 con 0x562a131133a0 2024-03-21T11:47:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.634+0000 7fb163400640 1 -- 172.21.15.47:0/4257019592 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb17c09a950 con 0x562a131133a0 2024-03-21T11:47:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.634+0000 7fb18b57a740 10 client.33073 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:21.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.634+0000 7fb18b57a740 1 -- 172.21.15.47:0/4257019592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=41}) v3 -- 0x562a12e547d0 con 0x562a131133a0 2024-03-21T11:47:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.638+0000 7fb163400640 1 -- 172.21.15.47:0/4257019592 <== mon.1 v2:172.21.15.102:3300/0 4 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb17c014440 con 0x562a131133a0 2024-03-21T11:47:21.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.638+0000 7fb18b57a740 10 client.33073 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.638+0000 7fb18b57a740 10 client.33073 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:47:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.638+0000 7fb18b57a740 20 client.33073 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.638+0000 7fb18b57a740 1 --2- 172.21.15.47:0/4257019592 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562a13127370 0x562a131297c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.638+0000 7fb18b57a740 4 client.33073 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.638+0000 7fb18b57a740 1 -- 172.21.15.47:0/4257019592 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a13083800 con 0x562a13127370 2024-03-21T11:47:21.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.638+0000 7fb185e00640 1 --2- 172.21.15.47:0/4257019592 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562a13127370 0x562a131297c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.639+0000 7fb185e00640 1 --2- 172.21.15.47:0/4257019592 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562a13127370 0x562a131297c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=24bfd67a4a351642 server_cookie=5645590feca04db7 in_seq=0 out_seq=0 2024-03-21T11:47:21.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.640+0000 7fb163400640 10 client.33073 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:47:21.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.641+0000 7fb163400640 1 -- 172.21.15.47:0/4257019592 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562a13083800 con 0x562a13127370 2024-03-21T11:47:21.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.641+0000 7fb163400640 10 client.33073 handle_command_reply: tid=0 2024-03-21T11:47:21.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.641+0000 7fb18b57a740 1 client.33073 shutdown 2024-03-21T11:47:21.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.641+0000 7fb18b57a740 1 -- 172.21.15.47:0/4257019592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 msgr2=0x562a13123080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:21 smithi100 ceph-mon[36245]: pgmap v1121: 129 pgs: 129 active+clean; 712 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.9 MiB/s wr, 530 op/s 2024-03-21T11:47:21.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:21 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1679308907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:47:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.642+0000 7fb18b57a740 1 --2- 172.21.15.47:0/4257019592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a13123080 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fb17c097740 tx=0x7fb17c009f90 comp rx=0 tx=0).stop 2024-03-21T11:47:21.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.643+0000 7fb18b57a740 1 -- 172.21.15.47:0/4257019592 shutdown_connections 2024-03-21T11:47:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.643+0000 7fb18b57a740 1 --2- 172.21.15.47:0/4257019592 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562a13127370 0x562a131297c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.643+0000 7fb18b57a740 1 --2- 172.21.15.47:0/4257019592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a13123080 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.643+0000 7fb163400640 0 client.33073 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:47:21.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.643+0000 7fb18b57a740 1 --2- 172.21.15.47:0/4257019592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a131129e0 0x562a1311c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.643+0000 7fb18b57a740 1 --2- 172.21.15.47:0/4257019592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a13111810 0x562a1311bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.643+0000 7fb18b57a740 1 -- 172.21.15.47:0/4257019592 >> 172.21.15.47:0/4257019592 conn(0x562a1310cc00 msgr2=0x562a1310beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:21.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.643+0000 7fb18b57a740 1 -- 172.21.15.47:0/4257019592 shutdown_connections 2024-03-21T11:47:21.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.644+0000 7fb18b57a740 1 -- 172.21.15.47:0/4257019592 wait complete. 2024-03-21T11:47:21.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.644+0000 7fb18b57a740 20 client.33073 trim_cache size 0 max 16384 2024-03-21T11:47:21.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.694+0000 7fb18b57a740 1 Processor -- start 2024-03-21T11:47:21.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.695+0000 7fb18b57a740 1 -- start start 2024-03-21T11:47:21.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.695+0000 7fb18b57a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a13111810 0x562a1310cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.695+0000 7fb18b57a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a131129e0 0x562a13123040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.695+0000 7fb18b57a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a131199c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.695+0000 7fb18b57a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1801b0720 con 0x562a131129e0 2024-03-21T11:47:21.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.695+0000 7fb18b57a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1801b20a0 con 0x562a13111810 2024-03-21T11:47:21.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.695+0000 7fb18b57a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1801b0d80 con 0x562a131133a0 2024-03-21T11:47:21.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.696+0000 7fb186800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a131199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.696+0000 7fb185e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a13111810 0x562a1310cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.696+0000 7fb186800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a131199c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48004/0 (socket says 172.21.15.47:48004) 2024-03-21T11:47:21.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.696+0000 7fb186800640 1 -- 172.21.15.47:0/722553153 learned_addr learned my addr 172.21.15.47:0/722553153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:21.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.696+0000 7fb185400640 1 --2- 172.21.15.47:0/722553153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a131129e0 0x562a13123040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.696+0000 7fb186800640 1 -- 172.21.15.47:0/722553153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a13111810 msgr2=0x562a1310cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.696+0000 7fb186800640 1 --2- 172.21.15.47:0/722553153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a13111810 0x562a1310cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.696+0000 7fb186800640 1 -- 172.21.15.47:0/722553153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a131129e0 msgr2=0x562a13123040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.699+0000 7fb186800640 1 --2- 172.21.15.47:0/722553153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a131129e0 0x562a13123040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.699+0000 7fb186800640 1 -- 172.21.15.47:0/722553153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a1310d520 con 0x562a131133a0 2024-03-21T11:47:21.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.699+0000 7fb185400640 1 --2- 172.21.15.47:0/722553153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a131129e0 0x562a13123040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:21.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.699+0000 7fb186800640 1 --2- 172.21.15.47:0/722553153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a131199c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb17c09b270 tx=0x7fb17c0ce070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea8cbab9d03ea8d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:21.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.701+0000 7fb16e600640 1 -- 172.21.15.47:0/722553153 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c09f160 con 0x562a131133a0 2024-03-21T11:47:21.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.702+0000 7fb16e600640 1 -- 172.21.15.47:0/722553153 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb17c095e50 con 0x562a131133a0 2024-03-21T11:47:21.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.702+0000 7fb16e600640 1 -- 172.21.15.47:0/722553153 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c014590 con 0x562a131133a0 2024-03-21T11:47:21.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.703+0000 7fb18b57a740 1 -- 172.21.15.47:0/722553153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 msgr2=0x562a131199c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.703+0000 7fb18b57a740 1 --2- 172.21.15.47:0/722553153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a131199c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb17c09b270 tx=0x7fb17c0ce070 comp rx=0 tx=0).stop 2024-03-21T11:47:21.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.703+0000 7fb18b57a740 1 -- 172.21.15.47:0/722553153 shutdown_connections 2024-03-21T11:47:21.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.703+0000 7fb18b57a740 1 --2- 172.21.15.47:0/722553153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131133a0 0x562a131199c0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.703+0000 7fb18b57a740 1 --2- 172.21.15.47:0/722553153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a131129e0 0x562a13123040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.703+0000 7fb18b57a740 1 --2- 172.21.15.47:0/722553153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a13111810 0x562a1310cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.703+0000 7fb18b57a740 1 -- 172.21.15.47:0/722553153 >> 172.21.15.47:0/722553153 conn(0x562a1310cc00 msgr2=0x562a131106f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:21.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.703+0000 7fb18b57a740 1 -- 172.21.15.47:0/722553153 shutdown_connections 2024-03-21T11:47:21.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.703+0000 7fb18b57a740 1 -- 172.21.15.47:0/722553153 wait complete. 2024-03-21T11:47:21.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.704+0000 7fb18b57a740 1 -- start start 2024-03-21T11:47:21.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.704+0000 7fb18b57a740 1 Processor -- start 2024-03-21T11:47:21.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.706+0000 7fb18b57a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a13111810 0x562a1311ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.706+0000 7fb18b57a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131129e0 0x562a1311f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.706+0000 7fb185e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a13111810 0x562a1311ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.707+0000 7fb185e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a13111810 0x562a1311ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37248/0 (socket says 172.21.15.47:37248) 2024-03-21T11:47:21.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.707+0000 7fb185400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131129e0 0x562a1311f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.707+0000 7fb185400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131129e0 0x562a1311f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48018/0 (socket says 172.21.15.47:48018) 2024-03-21T11:47:21.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.707+0000 7fb18b57a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a131133a0 0x562a13122190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.707+0000 7fb18b57a740 1 -- 172.21.15.47:0/3048458096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1801b1a40 con 0x562a13111810 2024-03-21T11:47:21.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.707+0000 7fb185400640 1 -- 172.21.15.47:0/3048458096 learned_addr learned my addr 172.21.15.47:0/3048458096 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:21.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.707+0000 7fb18b57a740 1 -- 172.21.15.47:0/3048458096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1801b13e0 con 0x562a131133a0 2024-03-21T11:47:21.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.707+0000 7fb18b57a740 1 -- 172.21.15.47:0/3048458096 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1801ae8e0 con 0x562a131129e0 2024-03-21T11:47:21.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.707+0000 7fb18b57a740 10 client.0 _ll_register_callbacks cb 0x7fb170009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:21.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.708+0000 7fb186800640 1 --2- 172.21.15.47:0/3048458096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a131133a0 0x562a13122190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.708+0000 7fb185e00640 1 -- 172.21.15.47:0/3048458096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131129e0 msgr2=0x562a1311f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.708+0000 7fb185e00640 1 --2- 172.21.15.47:0/3048458096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131129e0 0x562a1311f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.708+0000 7fb185e00640 1 -- 172.21.15.47:0/3048458096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a131133a0 msgr2=0x562a13122190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:21.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.708+0000 7fb185e00640 1 --2- 172.21.15.47:0/3048458096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a131133a0 0x562a13122190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.708+0000 7fb185e00640 1 -- 172.21.15.47:0/3048458096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a13122780 con 0x562a13111810 2024-03-21T11:47:21.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.709+0000 7fb186800640 1 --2- 172.21.15.47:0/3048458096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a131133a0 0x562a13122190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:21.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.709+0000 7fb185400640 1 --2- 172.21.15.47:0/3048458096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131129e0 0x562a1311f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:21.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.709+0000 7fb185e00640 1 --2- 172.21.15.47:0/3048458096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a13111810 0x562a1311ec30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb174000c00 tx=0x7fb17402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4deb299a0985 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:21.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.709+0000 7fb163400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:47:21.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.710+0000 7fb163400640 1 -- 172.21.15.47:0/3048458096 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174004310 con 0x562a13111810 2024-03-21T11:47:21.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.710+0000 7fb163400640 1 -- 172.21.15.47:0/3048458096 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb174051840 con 0x562a13111810 2024-03-21T11:47:21.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.711+0000 7fb18b57a740 1 -- 172.21.15.47:0/3048458096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb17c095e50 con 0x562a13111810 2024-03-21T11:47:21.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.711+0000 7fb163400640 1 -- 172.21.15.47:0/3048458096 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb17405e050 con 0x562a13111810 2024-03-21T11:47:21.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.712+0000 7fb18b57a740 10 client.22257 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:21.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.712+0000 7fb18b57a740 1 -- 172.21.15.47:0/3048458096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562a13183140 con 0x562a13111810 2024-03-21T11:47:21.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.713+0000 7fb163400640 1 -- 172.21.15.47:0/3048458096 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb17c095e50 con 0x562a13111810 2024-03-21T11:47:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.713+0000 7fb163400640 1 -- 172.21.15.47:0/3048458096 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb17404b040 con 0x562a13111810 2024-03-21T11:47:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.716+0000 7fb18b57a740 10 client.22257 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.716+0000 7fb18b57a740 10 client.22257 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:47:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.716+0000 7fb18b57a740 20 client.22257 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.716+0000 7fb18b57a740 1 --2- 172.21.15.47:0/3048458096 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562a13127370 0x562a13185a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.716+0000 7fb18b57a740 4 client.22257 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:47:21.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.716+0000 7fb18b57a740 1 -- 172.21.15.47:0/3048458096 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562a1307db50 con 0x562a13127370 2024-03-21T11:47:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.716+0000 7fb185400640 1 --2- 172.21.15.47:0/3048458096 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562a13127370 0x562a13185a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.717+0000 7fb185400640 1 --2- 172.21.15.47:0/3048458096 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562a13127370 0x562a13185a80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5efc9c61390c80b7 server_cookie=65ae66b1a2f8345a in_seq=0 out_seq=0 2024-03-21T11:47:21.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.717+0000 7fb163400640 10 client.22257 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:47:21.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.721+0000 7fb163400640 1 -- 172.21.15.47:0/3048458096 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562a1307db50 con 0x562a13127370 2024-03-21T11:47:21.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.721+0000 7fb163400640 10 client.22257 handle_command_reply: tid=0 2024-03-21T11:47:21.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.721+0000 7fb18b57a740 1 client.22257 shutdown 2024-03-21T11:47:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.722+0000 7fb18b57a740 1 -- 172.21.15.47:0/3048458096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a13111810 msgr2=0x562a1311ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.722+0000 7fb18b57a740 1 --2- 172.21.15.47:0/3048458096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a13111810 0x562a1311ec30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb174000c00 tx=0x7fb17402bc30 comp rx=0 tx=0).stop 2024-03-21T11:47:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.722+0000 7fb18b57a740 1 -- 172.21.15.47:0/3048458096 shutdown_connections 2024-03-21T11:47:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.722+0000 7fb18b57a740 1 --2- 172.21.15.47:0/3048458096 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x562a13127370 0x562a13185a80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.722+0000 7fb18b57a740 1 --2- 172.21.15.47:0/3048458096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a131133a0 0x562a13122190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.722+0000 7fb18b57a740 1 --2- 172.21.15.47:0/3048458096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a131129e0 0x562a1311f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.722+0000 7fb18b57a740 1 --2- 172.21.15.47:0/3048458096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562a13111810 0x562a1311ec30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.722+0000 7fb18b57a740 1 -- 172.21.15.47:0/3048458096 >> 172.21.15.47:0/3048458096 conn(0x562a1310cc00 msgr2=0x562a131106f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.722+0000 7fb163400640 0 client.22257 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:47:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.722+0000 7fb18b57a740 1 -- 172.21.15.47:0/3048458096 shutdown_connections 2024-03-21T11:47:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.724+0000 7fb18b57a740 1 -- 172.21.15.47:0/3048458096 wait complete. 2024-03-21T11:47:21.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.724+0000 7fb18b57a740 20 client.22257 trim_cache size 0 max 16384 2024-03-21T11:47:21.725 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:47:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.725+0000 7fb188800640 1 -- 172.21.15.47:0/3069303950 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1680a3070 msgr2=0x7fb1680a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.725+0000 7fb188800640 1 --2- 172.21.15.47:0/3069303950 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1680a3070 0x7fb1680a5530 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb174006010 tx=0x7fb174004ef0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.725+0000 7fb188800640 1 -- 172.21.15.47:0/3069303950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb180105490 msgr2=0x7fb180075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.725+0000 7fb188800640 1 --2- 172.21.15.47:0/3069303950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb180105490 0x7fb180075fc0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00be20 tx=0x7fb17c00bf10 comp rx=0 tx=0).stop 2024-03-21T11:47:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.725+0000 7fb188800640 1 -- 172.21.15.47:0/3069303950 shutdown_connections 2024-03-21T11:47:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.725+0000 7fb188800640 1 --2- 172.21.15.47:0/3069303950 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb1680a3070 0x7fb1680a5530 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.725+0000 7fb188800640 1 --2- 172.21.15.47:0/3069303950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb180105490 0x7fb180075fc0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.725+0000 7fb188800640 1 --2- 172.21.15.47:0/3069303950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb180104ad0 0x7fb180079990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.725+0000 7fb188800640 1 --2- 172.21.15.47:0/3069303950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1801038e0 0x7fb180079450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:21.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.725+0000 7fb188800640 1 -- 172.21.15.47:0/3069303950 >> 172.21.15.47:0/3069303950 conn(0x7fb1800fee50 msgr2=0x7fb180107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.729+0000 7fb188800640 1 -- 172.21.15.47:0/3069303950 shutdown_connections 2024-03-21T11:47:21.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:21.729+0000 7fb188800640 1 -- 172.21.15.47:0/3069303950 wait complete. 2024-03-21T11:47:21.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:21 smithi102 ceph-mon[36748]: pgmap v1121: 129 pgs: 129 active+clean; 712 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.9 MiB/s wr, 530 op/s 2024-03-21T11:47:21.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:21 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1679308907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:47:21.763 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:47:21.763 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:47:21.992 INFO:tasks.workunit.client.0.smithi047.stdout: 10 254360 21.78 MB/sec execute 485 sec latency 1879.654 ms 2024-03-21T11:47:22.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.113+0000 7f5c84200640 1 -- 172.21.15.47:0/2441418491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c7c105490 msgr2=0x7f5c7c10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.113+0000 7f5c84200640 1 --2- 172.21.15.47:0/2441418491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c7c105490 0x7f5c7c10bd20 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009900 tx=0x7f5c6c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:47:22.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.113+0000 7f5c84200640 1 -- 172.21.15.47:0/2441418491 shutdown_connections 2024-03-21T11:47:22.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.113+0000 7f5c84200640 1 --2- 172.21.15.47:0/2441418491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c7c105490 0x7f5c7c10bd20 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.113+0000 7f5c84200640 1 --2- 172.21.15.47:0/2441418491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c104ad0 0x7f5c7c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.113+0000 7f5c84200640 1 --2- 172.21.15.47:0/2441418491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c7c1038e0 0x7f5c7c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.113+0000 7f5c84200640 1 -- 172.21.15.47:0/2441418491 >> 172.21.15.47:0/2441418491 conn(0x7f5c7c0fee50 msgr2=0x7f5c7c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:22.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.115+0000 7f5c84200640 1 -- 172.21.15.47:0/2441418491 shutdown_connections 2024-03-21T11:47:22.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.115+0000 7f5c84200640 1 -- 172.21.15.47:0/2441418491 wait complete. 2024-03-21T11:47:22.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.116+0000 7f5c84200640 1 Processor -- start 2024-03-21T11:47:22.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.117+0000 7f5c84200640 1 -- start start 2024-03-21T11:47:22.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.117+0000 7f5c84200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c7c1038e0 0x7f5c7c19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.117+0000 7f5c84200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c104ad0 0x7f5c7c19e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.117+0000 7f5c84200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c7c105490 0x7f5c7c1a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.117+0000 7f5c84200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c10f5b0 con 0x7f5c7c1038e0 2024-03-21T11:47:22.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.117+0000 7f5c84200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c10f430 con 0x7f5c7c104ad0 2024-03-21T11:47:22.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.117+0000 7f5c84200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c10f730 con 0x7f5c7c105490 2024-03-21T11:47:22.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.118+0000 7f5c80e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c104ad0 0x7f5c7c19e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.118+0000 7f5c82200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c7c105490 0x7f5c7c1a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.118+0000 7f5c80e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c104ad0 0x7f5c7c19e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59660/0 (socket says 172.21.15.47:59660) 2024-03-21T11:47:22.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.118+0000 7f5c82200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c7c105490 0x7f5c7c1a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48034/0 (socket says 172.21.15.47:48034) 2024-03-21T11:47:22.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.118+0000 7f5c80e00640 1 -- 172.21.15.47:0/188304823 learned_addr learned my addr 172.21.15.47:0/188304823 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:22.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.118+0000 7f5c81800640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c7c1038e0 0x7f5c7c19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.118+0000 7f5c80e00640 1 -- 172.21.15.47:0/188304823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c7c105490 msgr2=0x7f5c7c1a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.118+0000 7f5c80e00640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c7c105490 0x7f5c7c1a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.118+0000 7f5c80e00640 1 -- 172.21.15.47:0/188304823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c7c1038e0 msgr2=0x7f5c7c19df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.118+0000 7f5c80e00640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c7c1038e0 0x7f5c7c19df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.118+0000 7f5c80e00640 1 -- 172.21.15.47:0/188304823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c1a5ba0 con 0x7f5c7c104ad0 2024-03-21T11:47:22.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.119+0000 7f5c82200640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c7c105490 0x7f5c7c1a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:22.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.119+0000 7f5c81800640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c7c1038e0 0x7f5c7c19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:22.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.120+0000 7f5c80e00640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c104ad0 0x7f5c7c19e4c0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800e9f0 tx=0x7f5c6800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:22.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.120+0000 7f5c7a000640 1 -- 172.21.15.47:0/188304823 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800cd80 con 0x7f5c7c104ad0 2024-03-21T11:47:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.120+0000 7f5c7a000640 1 -- 172.21.15.47:0/188304823 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c68005500 con 0x7f5c7c104ad0 2024-03-21T11:47:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.120+0000 7f5c84200640 1 -- 172.21.15.47:0/188304823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7c1a5e90 con 0x7f5c7c104ad0 2024-03-21T11:47:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.120+0000 7f5c7a000640 1 -- 172.21.15.47:0/188304823 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68010690 con 0x7f5c7c104ad0 2024-03-21T11:47:22.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.120+0000 7f5c84200640 1 -- 172.21.15.47:0/188304823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c1a6300 con 0x7f5c7c104ad0 2024-03-21T11:47:22.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.127+0000 7f5c86fc8740 1 Processor -- start 2024-03-21T11:47:22.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.127+0000 7f5c86fc8740 1 -- start start 2024-03-21T11:47:22.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.127+0000 7f5c86fc8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a32bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.127+0000 7f5c86fc8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32c9e0 0x55621a32ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.128+0000 7f5c86fc8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32d3a0 0x55621a333c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.128+0000 7f5c86fc8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55621a2902c0 con 0x55621a32b810 2024-03-21T11:47:22.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.128+0000 7f5c86fc8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55621a238c90 con 0x55621a32d3a0 2024-03-21T11:47:22.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.128+0000 7f5c86fc8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55621a28c360 con 0x55621a32c9e0 2024-03-21T11:47:22.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.130+0000 7f5c7a000640 1 -- 172.21.15.47:0/188304823 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5c680046c0 con 0x7f5c7c104ad0 2024-03-21T11:47:22.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.133+0000 7f5c7a000640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5c600a2f20 0x7f5c600a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.133+0000 7f5c82200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a32bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.133+0000 7f5c82200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a32bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37272/0 (socket says 172.21.15.47:37272) 2024-03-21T11:47:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.133+0000 7f5c82200640 1 -- 172.21.15.47:0/3797880159 learned_addr learned my addr 172.21.15.47:0/3797880159 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.133+0000 7f5c81800640 1 --2- 172.21.15.47:0/3797880159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32c9e0 0x55621a32ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.134+0000 7f5c82200640 1 -- 172.21.15.47:0/3797880159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32c9e0 msgr2=0x55621a32ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.134+0000 7f5c82200640 1 --2- 172.21.15.47:0/3797880159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32c9e0 0x55621a32ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.134+0000 7f5c82200640 1 -- 172.21.15.47:0/3797880159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32d3a0 msgr2=0x55621a333c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:47:22.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.134+0000 7f5c82200640 1 --2- 172.21.15.47:0/3797880159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32d3a0 0x55621a333c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.134+0000 7f5c82200640 1 -- 172.21.15.47:0/3797880159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55621a25d080 con 0x55621a32b810 2024-03-21T11:47:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.138+0000 7f5c82200640 1 --2- 172.21.15.47:0/3797880159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a32bc30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c02c240 tx=0x7f5c6c02c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2646629b14ce7c5d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.138+0000 7f5c7a000640 1 -- 172.21.15.47:0/188304823 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5c680cfc80 con 0x7f5c7c104ad0 2024-03-21T11:47:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.138+0000 7f5c82200640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5c600a2f20 0x7f5c600a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.138+0000 7f5c72a00640 1 -- 172.21.15.47:0/3797880159 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c045070 con 0x55621a32b810 2024-03-21T11:47:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.138+0000 7f5c72a00640 1 -- 172.21.15.47:0/3797880159 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c6c02cd30 con 0x55621a32b810 2024-03-21T11:47:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.138+0000 7f5c72a00640 1 -- 172.21.15.47:0/3797880159 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c005150 con 0x55621a32b810 2024-03-21T11:47:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.139+0000 7f5c86fc8740 1 -- 172.21.15.47:0/3797880159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 msgr2=0x55621a32bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.139+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/3797880159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a32bc30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c02c240 tx=0x7f5c6c02c6a0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.141+0000 7f5c86fc8740 1 -- 172.21.15.47:0/3797880159 shutdown_connections 2024-03-21T11:47:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.141+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/3797880159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32d3a0 0x55621a333c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.141+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/3797880159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32c9e0 0x55621a32ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.141+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/3797880159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a32bc30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.141+0000 7f5c86fc8740 1 -- 172.21.15.47:0/3797880159 >> 172.21.15.47:0/3797880159 conn(0x55621a326c00 msgr2=0x55621a327000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.142+0000 7f5c86fc8740 1 -- 172.21.15.47:0/3797880159 shutdown_connections 2024-03-21T11:47:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.142+0000 7f5c86fc8740 1 -- 172.21.15.47:0/3797880159 wait complete. 2024-03-21T11:47:22.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.142+0000 7f5c82200640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5c600a2f20 0x7f5c600a53e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c02c240 tx=0x7f5c6c02c6a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:22.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.142+0000 7f5c86fc8740 1 -- start start 2024-03-21T11:47:22.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.143+0000 7f5c86fc8740 1 Processor -- start 2024-03-21T11:47:22.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.143+0000 7f5c86fc8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a335a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.143+0000 7f5c86fc8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32c9e0 0x55621a335fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.144+0000 7f5c86fc8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32d3a0 0x55621a33d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.144+0000 7f5c80e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32c9e0 0x55621a335fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.144+0000 7f5c80e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32c9e0 0x55621a335fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59664/0 (socket says 172.21.15.47:59664) 2024-03-21T11:47:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.145+0000 7f5c81800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a335a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.145+0000 7f5c81800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a335a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37286/0 (socket says 172.21.15.47:37286) 2024-03-21T11:47:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.145+0000 7f5c86fc8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55621a334350 con 0x55621a32b810 2024-03-21T11:47:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.145+0000 7f5c81800640 1 -- 172.21.15.47:0/2002535068 learned_addr learned my addr 172.21.15.47:0/2002535068 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.145+0000 7f5c86fc8740 1 -- 172.21.15.47:0/2002535068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55621a24c0d0 con 0x55621a32c9e0 2024-03-21T11:47:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.145+0000 7f5c86fc8740 1 -- 172.21.15.47:0/2002535068 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55621a3344d0 con 0x55621a32d3a0 2024-03-21T11:47:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.145+0000 7f5c86fc8740 10 client.0 _ll_register_callbacks cb 0x55621a250e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:22.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.145+0000 7f5c80e00640 1 -- 172.21.15.47:0/2002535068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32d3a0 msgr2=0x55621a33d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.145+0000 7f5c80e00640 1 --2- 172.21.15.47:0/2002535068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32d3a0 0x55621a33d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.145+0000 7f5c80e00640 1 -- 172.21.15.47:0/2002535068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 msgr2=0x55621a335a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.145+0000 7f5c80e00640 1 --2- 172.21.15.47:0/2002535068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a335a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.145+0000 7f5c80e00640 1 -- 172.21.15.47:0/2002535068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556219fa43f0 con 0x55621a32c9e0 2024-03-21T11:47:22.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.145+0000 7f5c81800640 1 --2- 172.21.15.47:0/2002535068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a335a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:22.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.146+0000 7f5c67400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:22.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.146+0000 7f5c80e00640 1 --2- 172.21.15.47:0/2002535068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32c9e0 0x55621a335fc0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800c6a0 tx=0x7f5c6809ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adf84779565a9543 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:22.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.147+0000 7f5c67400640 1 -- 172.21.15.47:0/2002535068 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680a0170 con 0x55621a32c9e0 2024-03-21T11:47:22.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.147+0000 7f5c67400640 1 -- 172.21.15.47:0/2002535068 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5c68096b90 con 0x55621a32c9e0 2024-03-21T11:47:22.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.147+0000 7f5c86fc8740 1 -- 172.21.15.47:0/2002535068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c6c02cd30 con 0x55621a32c9e0 2024-03-21T11:47:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.151+0000 7f5c67400640 1 -- 172.21.15.47:0/2002535068 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c680a4a30 con 0x55621a32c9e0 2024-03-21T11:47:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.151+0000 7f5c86fc8740 10 client.33122 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.151+0000 7f5c86fc8740 1 -- 172.21.15.47:0/2002535068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55621a06e7d0 con 0x55621a32c9e0 2024-03-21T11:47:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.151+0000 7f5c67400640 1 -- 172.21.15.47:0/2002535068 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c6c02cd30 con 0x55621a32c9e0 2024-03-21T11:47:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.151+0000 7f5c67400640 1 -- 172.21.15.47:0/2002535068 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5c6809aee0 con 0x55621a32c9e0 2024-03-21T11:47:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.152+0000 7f5c86fc8740 10 client.33122 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.152+0000 7f5c86fc8740 10 client.33122 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:47:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.152+0000 7f5c86fc8740 20 client.33122 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.152+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/2002535068 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55621a341490 0x55621a3438b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.152+0000 7f5c86fc8740 4 client.33122 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.152+0000 7f5c86fc8740 1 -- 172.21.15.47:0/2002535068 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55621a29d800 con 0x55621a341490 2024-03-21T11:47:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.152+0000 7f5c81800640 1 --2- 172.21.15.47:0/2002535068 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55621a341490 0x55621a3438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.157+0000 7f5c67400640 10 client.33122 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:47:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.157+0000 7f5c81800640 1 --2- 172.21.15.47:0/2002535068 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55621a341490 0x55621a3438b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f34307221e0a417d server_cookie=51c14ddd8cba76b in_seq=0 out_seq=0 2024-03-21T11:47:22.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.160+0000 7f5c67400640 1 -- 172.21.15.47:0/2002535068 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55621a29d800 con 0x55621a341490 2024-03-21T11:47:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.160+0000 7f5c67400640 10 client.33122 handle_command_reply: tid=0 2024-03-21T11:47:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.160+0000 7f5c86fc8740 1 client.33122 shutdown 2024-03-21T11:47:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.160+0000 7f5c86fc8740 1 -- 172.21.15.47:0/2002535068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32c9e0 msgr2=0x55621a335fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.160+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/2002535068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32c9e0 0x55621a335fc0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800c6a0 tx=0x7f5c6809ba40 comp rx=0 tx=0).stop 2024-03-21T11:47:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.161+0000 7f5c86fc8740 1 -- 172.21.15.47:0/2002535068 shutdown_connections 2024-03-21T11:47:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.161+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/2002535068 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55621a341490 0x55621a3438b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.161+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/2002535068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32d3a0 0x55621a33d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.161+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/2002535068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32c9e0 0x55621a335fc0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.161+0000 7f5c67400640 0 client.33122 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:47:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.161+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/2002535068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a335a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.161+0000 7f5c86fc8740 1 -- 172.21.15.47:0/2002535068 >> 172.21.15.47:0/2002535068 conn(0x55621a326c00 msgr2=0x55621a325eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.162+0000 7f5c86fc8740 1 -- 172.21.15.47:0/2002535068 shutdown_connections 2024-03-21T11:47:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.162+0000 7f5c86fc8740 1 -- 172.21.15.47:0/2002535068 wait complete. 2024-03-21T11:47:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.162+0000 7f5c86fc8740 20 client.33122 trim_cache size 0 max 16384 2024-03-21T11:47:22.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.194+0000 7f5c86fc8740 1 Processor -- start 2024-03-21T11:47:22.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.194+0000 7f5c86fc8740 1 -- start start 2024-03-21T11:47:22.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.194+0000 7f5c86fc8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a348170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.194+0000 7f5c86fc8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32c9e0 0x55621a33cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.194+0000 7f5c86fc8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32d3a0 0x55621a3486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.194+0000 7f5c86fc8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1b0610 con 0x55621a32b810 2024-03-21T11:47:22.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.194+0000 7f5c86fc8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1b1f90 con 0x55621a32c9e0 2024-03-21T11:47:22.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.194+0000 7f5c86fc8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1b0c70 con 0x55621a32d3a0 2024-03-21T11:47:22.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.195+0000 7f5c82200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32d3a0 0x55621a3486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.195+0000 7f5c82200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32d3a0 0x55621a3486b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48062/0 (socket says 172.21.15.47:48062) 2024-03-21T11:47:22.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.195+0000 7f5c82200640 1 -- 172.21.15.47:0/3871369727 learned_addr learned my addr 172.21.15.47:0/3871369727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:22.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.195+0000 7f5c80e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32c9e0 0x55621a33cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.196+0000 7f5c80e00640 1 -- 172.21.15.47:0/3871369727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32d3a0 msgr2=0x55621a3486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.196+0000 7f5c80e00640 1 --2- 172.21.15.47:0/3871369727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32d3a0 0x55621a3486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.196+0000 7f5c80e00640 1 -- 172.21.15.47:0/3871369727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 msgr2=0x55621a348170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.196+0000 7f5c81800640 1 --2- 172.21.15.47:0/3871369727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a348170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.196+0000 7f5c80e00640 1 --2- 172.21.15.47:0/3871369727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a348170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.196+0000 7f5c80e00640 1 -- 172.21.15.47:0/3871369727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55621a3224e0 con 0x55621a32c9e0 2024-03-21T11:47:22.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.196+0000 7f5c80e00640 1 --2- 172.21.15.47:0/3871369727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32c9e0 0x55621a33cfc0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5c680cd760 tx=0x7f5c6809a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a4d3527dc869b42 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:22.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.197+0000 7f5c72000640 1 -- 172.21.15.47:0/3871369727 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680a0170 con 0x55621a32c9e0 2024-03-21T11:47:22.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.197+0000 7f5c72000640 1 -- 172.21.15.47:0/3871369727 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c680a1610 con 0x55621a32c9e0 2024-03-21T11:47:22.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.197+0000 7f5c72000640 1 -- 172.21.15.47:0/3871369727 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6809b470 con 0x55621a32c9e0 2024-03-21T11:47:22.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.198+0000 7f5c86fc8740 1 -- 172.21.15.47:0/3871369727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32c9e0 msgr2=0x55621a33cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.198+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/3871369727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32c9e0 0x55621a33cfc0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5c680cd760 tx=0x7f5c6809a530 comp rx=0 tx=0).stop 2024-03-21T11:47:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.200+0000 7f5c86fc8740 1 -- 172.21.15.47:0/3871369727 shutdown_connections 2024-03-21T11:47:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.200+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/3871369727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32d3a0 0x55621a3486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.200+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/3871369727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32c9e0 0x55621a33cfc0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.200+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/3871369727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32b810 0x55621a348170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.200+0000 7f5c86fc8740 1 -- 172.21.15.47:0/3871369727 >> 172.21.15.47:0/3871369727 conn(0x55621a326c00 msgr2=0x55621a32a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.201+0000 7f5c86fc8740 1 -- 172.21.15.47:0/3871369727 shutdown_connections 2024-03-21T11:47:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.201+0000 7f5c86fc8740 1 -- 172.21.15.47:0/3871369727 wait complete. 2024-03-21T11:47:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.201+0000 7f5c86fc8740 1 -- start start 2024-03-21T11:47:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.201+0000 7f5c86fc8740 1 Processor -- start 2024-03-21T11:47:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.204+0000 7f5c86fc8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32b810 0x55621a33a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.204+0000 7f5c86fc8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32c9e0 0x55621a33aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.204+0000 7f5c86fc8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32d3a0 0x55621a33bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.204+0000 7f5c81800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32b810 0x55621a33a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.204+0000 7f5c81800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32b810 0x55621a33a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59680/0 (socket says 172.21.15.47:59680) 2024-03-21T11:47:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.204+0000 7f5c81800640 1 -- 172.21.15.47:0/1897326580 learned_addr learned my addr 172.21.15.47:0/1897326580 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.204+0000 7f5c86fc8740 1 -- 172.21.15.47:0/1897326580 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1b1930 con 0x55621a32d3a0 2024-03-21T11:47:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.204+0000 7f5c86fc8740 1 -- 172.21.15.47:0/1897326580 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1b12d0 con 0x55621a32b810 2024-03-21T11:47:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.204+0000 7f5c86fc8740 1 -- 172.21.15.47:0/1897326580 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1ae7d0 con 0x55621a32c9e0 2024-03-21T11:47:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.205+0000 7f5c86fc8740 10 client.0 _ll_register_callbacks cb 0x7f5c68009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.205+0000 7f5c82200640 1 --2- 172.21.15.47:0/1897326580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32d3a0 0x55621a33bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.205+0000 7f5c81800640 1 -- 172.21.15.47:0/1897326580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32c9e0 msgr2=0x55621a33aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.205+0000 7f5c81800640 1 --2- 172.21.15.47:0/1897326580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32c9e0 0x55621a33aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.205+0000 7f5c81800640 1 -- 172.21.15.47:0/1897326580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32d3a0 msgr2=0x55621a33bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.205+0000 7f5c81800640 1 --2- 172.21.15.47:0/1897326580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32d3a0 0x55621a33bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.205+0000 7f5c81800640 1 -- 172.21.15.47:0/1897326580 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55621a33c4d0 con 0x55621a32b810 2024-03-21T11:47:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.206+0000 7f5c67400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.206+0000 7f5c81800640 1 --2- 172.21.15.47:0/1897326580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32b810 0x55621a33a990 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f5c74000f80 tx=0x7f5c74012410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52a5ea58cae2a41a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.206+0000 7f5c67400640 1 -- 172.21.15.47:0/1897326580 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74021020 con 0x55621a32b810 2024-03-21T11:47:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.206+0000 7f5c86fc8740 1 -- 172.21.15.47:0/1897326580 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c680a1610 con 0x55621a32b810 2024-03-21T11:47:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.206+0000 7f5c67400640 1 -- 172.21.15.47:0/1897326580 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5c7401d890 con 0x55621a32b810 2024-03-21T11:47:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.207+0000 7f5c67400640 1 -- 172.21.15.47:0/1897326580 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c7402c050 con 0x55621a32b810 2024-03-21T11:47:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.207+0000 7f5c86fc8740 10 client.33128 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.207+0000 7f5c86fc8740 1 -- 172.21.15.47:0/1897326580 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55621a39d080 con 0x55621a32b810 2024-03-21T11:47:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.207+0000 7f5c67400640 1 -- 172.21.15.47:0/1897326580 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c680a1610 con 0x55621a32b810 2024-03-21T11:47:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.208+0000 7f5c67400640 1 -- 172.21.15.47:0/1897326580 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5c740119c0 con 0x55621a32b810 2024-03-21T11:47:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.210+0000 7f5c86fc8740 10 client.33128 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.210+0000 7f5c86fc8740 10 client.33128 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:47:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.210+0000 7f5c86fc8740 20 client.33128 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.210+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/1897326580 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55621a39d9d0 0x55621a39fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.210+0000 7f5c86fc8740 4 client.33128 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:47:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.210+0000 7f5c86fc8740 1 -- 172.21.15.47:0/1897326580 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55621a297b50 con 0x55621a39d9d0 2024-03-21T11:47:22.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.211+0000 7f5c80e00640 1 --2- 172.21.15.47:0/1897326580 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55621a39d9d0 0x55621a39fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.212+0000 7f5c80e00640 1 --2- 172.21.15.47:0/1897326580 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55621a39d9d0 0x55621a39fdc0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18f5866f5ac445fc server_cookie=ffd8490f21ebe58d in_seq=0 out_seq=0 2024-03-21T11:47:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.212+0000 7f5c67400640 10 client.33128 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:47:22.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.213+0000 7f5c67400640 1 -- 172.21.15.47:0/1897326580 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55621a297b50 con 0x55621a39d9d0 2024-03-21T11:47:22.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.213+0000 7f5c67400640 10 client.33128 handle_command_reply: tid=0 2024-03-21T11:47:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.213+0000 7f5c86fc8740 1 client.33128 shutdown 2024-03-21T11:47:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.214+0000 7f5c86fc8740 1 -- 172.21.15.47:0/1897326580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32b810 msgr2=0x55621a33a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.214+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/1897326580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32b810 0x55621a33a990 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f5c74000f80 tx=0x7f5c74012410 comp rx=0 tx=0).stop 2024-03-21T11:47:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.214+0000 7f5c86fc8740 1 -- 172.21.15.47:0/1897326580 shutdown_connections 2024-03-21T11:47:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.214+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/1897326580 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55621a32d3a0 0x55621a33bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.214+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/1897326580 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55621a39d9d0 0x55621a39fdc0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.214+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/1897326580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55621a32c9e0 0x55621a33aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.214+0000 7f5c86fc8740 1 --2- 172.21.15.47:0/1897326580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55621a32b810 0x55621a33a990 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.214+0000 7f5c86fc8740 1 -- 172.21.15.47:0/1897326580 >> 172.21.15.47:0/1897326580 conn(0x55621a326c00 msgr2=0x55621a32fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.214+0000 7f5c67400640 0 client.33128 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:47:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.215+0000 7f5c86fc8740 1 -- 172.21.15.47:0/1897326580 shutdown_connections 2024-03-21T11:47:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.215+0000 7f5c86fc8740 1 -- 172.21.15.47:0/1897326580 wait complete. 2024-03-21T11:47:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.215+0000 7f5c86fc8740 20 client.33128 trim_cache size 0 max 16384 2024-03-21T11:47:22.217 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:47:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.216+0000 7f5c84200640 1 -- 172.21.15.47:0/188304823 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5c600a2f20 msgr2=0x7f5c600a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.216+0000 7f5c84200640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5c600a2f20 0x7f5c600a53e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c02c240 tx=0x7f5c6c02c6a0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.216+0000 7f5c84200640 1 -- 172.21.15.47:0/188304823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c104ad0 msgr2=0x7f5c7c19e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.216+0000 7f5c84200640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c104ad0 0x7f5c7c19e4c0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800e9f0 tx=0x7f5c6800edf0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.216+0000 7f5c84200640 1 -- 172.21.15.47:0/188304823 shutdown_connections 2024-03-21T11:47:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.216+0000 7f5c84200640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5c600a2f20 0x7f5c600a53e0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.216+0000 7f5c84200640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c7c105490 0x7f5c7c1a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.216+0000 7f5c84200640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c104ad0 0x7f5c7c19e4c0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.216+0000 7f5c84200640 1 --2- 172.21.15.47:0/188304823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c7c1038e0 0x7f5c7c19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.216+0000 7f5c84200640 1 -- 172.21.15.47:0/188304823 >> 172.21.15.47:0/188304823 conn(0x7f5c7c0fee50 msgr2=0x7f5c7c077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.216+0000 7f5c84200640 1 -- 172.21.15.47:0/188304823 shutdown_connections 2024-03-21T11:47:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.216+0000 7f5c84200640 1 -- 172.21.15.47:0/188304823 wait complete. 2024-03-21T11:47:22.248 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:47:22.248 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:47:22.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.590+0000 7f261fe00640 1 -- 172.21.15.47:0/2175707761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2620102000 msgr2=0x7f26201043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.590+0000 7f261fe00640 1 --2- 172.21.15.47:0/2175707761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2620102000 0x7f26201043f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2618009be0 tx=0x7f261802bef0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.591+0000 7f261fe00640 1 -- 172.21.15.47:0/2175707761 shutdown_connections 2024-03-21T11:47:22.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.591+0000 7f261fe00640 1 --2- 172.21.15.47:0/2175707761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2620107260 0x7f26201096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.591+0000 7f261fe00640 1 --2- 172.21.15.47:0/2175707761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2620104930 0x7f2620106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.591+0000 7f261fe00640 1 --2- 172.21.15.47:0/2175707761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2620102000 0x7f26201043f0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.591+0000 7f261fe00640 1 -- 172.21.15.47:0/2175707761 >> 172.21.15.47:0/2175707761 conn(0x7f26200fba10 msgr2=0x7f26200fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:22.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.591+0000 7f261fe00640 1 -- 172.21.15.47:0/2175707761 shutdown_connections 2024-03-21T11:47:22.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.591+0000 7f261fe00640 1 -- 172.21.15.47:0/2175707761 wait complete. 2024-03-21T11:47:22.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.593+0000 7f261fe00640 1 Processor -- start 2024-03-21T11:47:22.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.595+0000 7f261fe00640 1 -- start start 2024-03-21T11:47:22.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.595+0000 7f261fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2620104930 0x7f2620101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.595+0000 7f261fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2620107260 0x7f26200fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.595+0000 7f261fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26201004f0 0x7f26201009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.595+0000 7f261fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262010d210 con 0x7f2620104930 2024-03-21T11:47:22.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.595+0000 7f261fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262010d090 con 0x7f2620107260 2024-03-21T11:47:22.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.595+0000 7f261fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262010d390 con 0x7f26201004f0 2024-03-21T11:47:22.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.595+0000 7f261e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2620107260 0x7f26200fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.595+0000 7f261e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2620107260 0x7f26200fffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59700/0 (socket says 172.21.15.47:59700) 2024-03-21T11:47:22.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.595+0000 7f261e000640 1 -- 172.21.15.47:0/946999079 learned_addr learned my addr 172.21.15.47:0/946999079 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:22.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.596+0000 7f261e000640 1 -- 172.21.15.47:0/946999079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26201004f0 msgr2=0x7f26201009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.596+0000 7f261e000640 1 --2- 172.21.15.47:0/946999079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26201004f0 0x7f26201009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.596+0000 7f261e000640 1 -- 172.21.15.47:0/946999079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2620104930 msgr2=0x7f2620101970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:47:22.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.596+0000 7f261e000640 1 --2- 172.21.15.47:0/946999079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2620104930 0x7f2620101970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.596+0000 7f261e000640 1 -- 172.21.15.47:0/946999079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2620072090 con 0x7f2620107260 2024-03-21T11:47:22.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.598+0000 7f261e000640 1 --2- 172.21.15.47:0/946999079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2620107260 0x7f26200fffb0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f261000e9f0 tx=0x7f261000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:22.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.598+0000 7f2617400640 1 -- 172.21.15.47:0/946999079 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261000ccc0 con 0x7f2620107260 2024-03-21T11:47:22.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.598+0000 7f2617400640 1 -- 172.21.15.47:0/946999079 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2610010910 con 0x7f2620107260 2024-03-21T11:47:22.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.598+0000 7f261fe00640 1 -- 172.21.15.47:0/946999079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2620072380 con 0x7f2620107260 2024-03-21T11:47:22.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.599+0000 7f2617400640 1 -- 172.21.15.47:0/946999079 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610018960 con 0x7f2620107260 2024-03-21T11:47:22.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.599+0000 7f261fe00640 1 -- 172.21.15.47:0/946999079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2620072920 con 0x7f2620107260 2024-03-21T11:47:22.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.607+0000 7f2628265740 1 Processor -- start 2024-03-21T11:47:22.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.608+0000 7f2617400640 1 -- 172.21.15.47:0/946999079 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2610010ac0 con 0x7f2620107260 2024-03-21T11:47:22.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.608+0000 7f2628265740 1 -- start start 2024-03-21T11:47:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.609+0000 7f2628265740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bed9f0 0x55c356bede10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.609+0000 7f2628265740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356bef040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.609+0000 7f2628265740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bef580 0x55c356bf5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.609+0000 7f2628265740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c356b522c0 con 0x55c356beebc0 2024-03-21T11:47:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.609+0000 7f2628265740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c356afac90 con 0x55c356bef580 2024-03-21T11:47:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.609+0000 7f2628265740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c356b4e360 con 0x55c356bed9f0 2024-03-21T11:47:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.609+0000 7f261ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356bef040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.609+0000 7f261ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356bef040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37320/0 (socket says 172.21.15.47:37320) 2024-03-21T11:47:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.609+0000 7f261ea00640 1 -- 172.21.15.47:0/917154262 learned_addr learned my addr 172.21.15.47:0/917154262 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.609+0000 7f261f400640 1 --2- 172.21.15.47:0/917154262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bed9f0 0x55c356bede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.610+0000 7f261e000640 1 --2- 172.21.15.47:0/917154262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bef580 0x55c356bf5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.610+0000 7f261ea00640 1 -- 172.21.15.47:0/917154262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bed9f0 msgr2=0x55c356bede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.610+0000 7f261ea00640 1 --2- 172.21.15.47:0/917154262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bed9f0 0x55c356bede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.610+0000 7f261ea00640 1 -- 172.21.15.47:0/917154262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bef580 msgr2=0x55c356bf5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.610+0000 7f261ea00640 1 --2- 172.21.15.47:0/917154262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bef580 0x55c356bf5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.610+0000 7f261ea00640 1 -- 172.21.15.47:0/917154262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c356b1f080 con 0x55c356beebc0 2024-03-21T11:47:22.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.611+0000 7f261ea00640 1 --2- 172.21.15.47:0/917154262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356bef040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2618035000 tx=0x7f2618009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b386dfc369807e5a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.611+0000 7f2617400640 1 --2- 172.21.15.47:0/946999079 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26040a2fa0 0x7f26040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.612+0000 7f2617400640 1 -- 172.21.15.47:0/946999079 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f26100cfab0 con 0x7f2620107260 2024-03-21T11:47:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.612+0000 7f261f400640 1 --2- 172.21.15.47:0/946999079 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26040a2fa0 0x7f26040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.612+0000 7f260b400640 1 -- 172.21.15.47:0/917154262 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618041020 con 0x55c356beebc0 2024-03-21T11:47:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.612+0000 7f260b400640 1 -- 172.21.15.47:0/917154262 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2618033ab0 con 0x55c356beebc0 2024-03-21T11:47:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.612+0000 7f260b400640 1 -- 172.21.15.47:0/917154262 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261802c890 con 0x55c356beebc0 2024-03-21T11:47:22.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.614+0000 7f2628265740 1 -- 172.21.15.47:0/917154262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 msgr2=0x55c356bef040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.614+0000 7f2628265740 1 --2- 172.21.15.47:0/917154262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356bef040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2618035000 tx=0x7f2618009320 comp rx=0 tx=0).stop 2024-03-21T11:47:22.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.614+0000 7f261f400640 1 --2- 172.21.15.47:0/946999079 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26040a2fa0 0x7f26040a5460 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f260c00b590 tx=0x7f260c00b520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.615+0000 7f2628265740 1 -- 172.21.15.47:0/917154262 shutdown_connections 2024-03-21T11:47:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.615+0000 7f2628265740 1 --2- 172.21.15.47:0/917154262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bef580 0x55c356bf5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.615+0000 7f2628265740 1 --2- 172.21.15.47:0/917154262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356bef040 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.615+0000 7f2628265740 1 --2- 172.21.15.47:0/917154262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bed9f0 0x55c356bede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.615+0000 7f2628265740 1 -- 172.21.15.47:0/917154262 >> 172.21.15.47:0/917154262 conn(0x55c356be8de0 msgr2=0x55c356be91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.615+0000 7f2628265740 1 -- 172.21.15.47:0/917154262 shutdown_connections 2024-03-21T11:47:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.616+0000 7f2628265740 1 -- 172.21.15.47:0/917154262 wait complete. 2024-03-21T11:47:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.617+0000 7f2628265740 1 -- start start 2024-03-21T11:47:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.617+0000 7f2628265740 1 Processor -- start 2024-03-21T11:47:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.618+0000 7f2628265740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bed9f0 0x55c356bf7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.618+0000 7f2628265740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356bf80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.618+0000 7f2628265740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bef580 0x55c356bff130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.618+0000 7f2628265740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c356bf6530 con 0x55c356beebc0 2024-03-21T11:47:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.618+0000 7f2628265740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c356b0e0d0 con 0x55c356bed9f0 2024-03-21T11:47:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.618+0000 7f2628265740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c356bf66b0 con 0x55c356bef580 2024-03-21T11:47:22.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.618+0000 7f2628265740 10 client.0 _ll_register_callbacks cb 0x55c356b12e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.619+0000 7f261f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bef580 0x55c356bff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.619+0000 7f261f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bef580 0x55c356bff130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48102/0 (socket says 172.21.15.47:48102) 2024-03-21T11:47:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.619+0000 7f261f400640 1 -- 172.21.15.47:0/1757046922 learned_addr learned my addr 172.21.15.47:0/1757046922 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.619+0000 7f261ea00640 1 --2- 172.21.15.47:0/1757046922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bed9f0 0x55c356bf7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.619+0000 7f261e000640 1 --2- 172.21.15.47:0/1757046922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356bf80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.620+0000 7f261f400640 1 -- 172.21.15.47:0/1757046922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bed9f0 msgr2=0x55c356bf7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.620+0000 7f261f400640 1 --2- 172.21.15.47:0/1757046922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bed9f0 0x55c356bf7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.620+0000 7f261f400640 1 -- 172.21.15.47:0/1757046922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 msgr2=0x55c356bf80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.620+0000 7f261f400640 1 --2- 172.21.15.47:0/1757046922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356bf80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.620+0000 7f261f400640 1 -- 172.21.15.47:0/1757046922 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3568663f0 con 0x55c356bef580 2024-03-21T11:47:22.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.620+0000 7f261e000640 1 --2- 172.21.15.47:0/1757046922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356bf80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:22.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.621+0000 7f261f400640 1 --2- 172.21.15.47:0/1757046922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bef580 0x55c356bff130 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f260c00bfb0 tx=0x7f260c005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ecf6ca76c917cf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:22.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.621+0000 7f25ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:47:22.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.621+0000 7f25ffe00640 1 -- 172.21.15.47:0/1757046922 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c01e830 con 0x55c356bef580 2024-03-21T11:47:22.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.621+0000 7f2628265740 1 -- 172.21.15.47:0/1757046922 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2618033ab0 con 0x55c356bef580 2024-03-21T11:47:22.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.623+0000 7f25ffe00640 1 -- 172.21.15.47:0/1757046922 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f260c01e9d0 con 0x55c356bef580 2024-03-21T11:47:22.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.623+0000 7f25ffe00640 1 -- 172.21.15.47:0/1757046922 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f260c01ebd0 con 0x55c356bef580 2024-03-21T11:47:22.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.623+0000 7f2628265740 10 client.33097 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:22.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.623+0000 7f2628265740 1 -- 172.21.15.47:0/1757046922 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c3569307d0 con 0x55c356bef580 2024-03-21T11:47:22.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.623+0000 7f25ffe00640 1 -- 172.21.15.47:0/1757046922 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2618033ab0 con 0x55c356bef580 2024-03-21T11:47:22.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.624+0000 7f25ffe00640 1 -- 172.21.15.47:0/1757046922 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f260c01c3b0 con 0x55c356bef580 2024-03-21T11:47:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.624+0000 7f2628265740 10 client.33097 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:22.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.624+0000 7f2628265740 10 client.33097 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:47:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.624+0000 7f2628265740 20 client.33097 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.624+0000 7f2628265740 1 --2- 172.21.15.47:0/1757046922 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c356c03740 0x55c356c05ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.624+0000 7f2628265740 4 client.33097 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:22.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.624+0000 7f2628265740 1 -- 172.21.15.47:0/1757046922 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c356b5f800 con 0x55c356c03740 2024-03-21T11:47:22.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.630+0000 7f261ea00640 1 --2- 172.21.15.47:0/1757046922 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c356c03740 0x55c356c05ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.633+0000 7f261ea00640 1 --2- 172.21.15.47:0/1757046922 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c356c03740 0x55c356c05ba0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13ead4fbecdedaed server_cookie=12c07e506c4809a8 in_seq=0 out_seq=0 2024-03-21T11:47:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.633+0000 7f25ffe00640 10 client.33097 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:47:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.635+0000 7f25ffe00640 1 -- 172.21.15.47:0/1757046922 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c356b5f800 con 0x55c356c03740 2024-03-21T11:47:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.635+0000 7f25ffe00640 10 client.33097 handle_command_reply: tid=0 2024-03-21T11:47:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.637+0000 7f2628265740 1 client.33097 shutdown 2024-03-21T11:47:22.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.637+0000 7f2628265740 1 -- 172.21.15.47:0/1757046922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bef580 msgr2=0x55c356bff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.637+0000 7f2628265740 1 --2- 172.21.15.47:0/1757046922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bef580 0x55c356bff130 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f260c00bfb0 tx=0x7f260c005cc0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.638+0000 7f2628265740 1 -- 172.21.15.47:0/1757046922 shutdown_connections 2024-03-21T11:47:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.638+0000 7f2628265740 1 --2- 172.21.15.47:0/1757046922 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c356c03740 0x55c356c05ba0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.638+0000 7f2628265740 1 --2- 172.21.15.47:0/1757046922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bef580 0x55c356bff130 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.638+0000 7f2628265740 1 --2- 172.21.15.47:0/1757046922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356bf80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.638+0000 7f2628265740 1 --2- 172.21.15.47:0/1757046922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bed9f0 0x55c356bf7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.638+0000 7f2628265740 1 -- 172.21.15.47:0/1757046922 >> 172.21.15.47:0/1757046922 conn(0x55c356be8de0 msgr2=0x55c356be8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.638+0000 7f25ffe00640 0 client.33097 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:47:22.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.639+0000 7f2628265740 1 -- 172.21.15.47:0/1757046922 shutdown_connections 2024-03-21T11:47:22.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.639+0000 7f2628265740 1 -- 172.21.15.47:0/1757046922 wait complete. 2024-03-21T11:47:22.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.639+0000 7f2628265740 20 client.33097 trim_cache size 0 max 16384 2024-03-21T11:47:22.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.675+0000 7f2628265740 1 Processor -- start 2024-03-21T11:47:22.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.675+0000 7f2628265740 1 -- start start 2024-03-21T11:47:22.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.676+0000 7f2628265740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356be8de0 0x55c356be7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.676+0000 7f2628265740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bed9f0 0x55c356bff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.676+0000 7f2628265740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356c0a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.676+0000 7f2628265740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26201b4fc0 con 0x55c356beebc0 2024-03-21T11:47:22.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.676+0000 7f2628265740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26201b6940 con 0x55c356be8de0 2024-03-21T11:47:22.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.676+0000 7f2628265740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26201b5620 con 0x55c356bed9f0 2024-03-21T11:47:22.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.676+0000 7f261ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356be8de0 0x55c356be7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.676+0000 7f261ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356be8de0 0x55c356be7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59738/0 (socket says 172.21.15.47:59738) 2024-03-21T11:47:22.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.676+0000 7f261ea00640 1 -- 172.21.15.47:0/2358776225 learned_addr learned my addr 172.21.15.47:0/2358776225 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:22.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.676+0000 7f261f400640 1 --2- 172.21.15.47:0/2358776225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356c0a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.679+0000 7f261ea00640 1 -- 172.21.15.47:0/2358776225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bed9f0 msgr2=0x55c356bff0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.679+0000 7f261ea00640 1 --2- 172.21.15.47:0/2358776225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bed9f0 0x55c356bff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.679+0000 7f261ea00640 1 -- 172.21.15.47:0/2358776225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 msgr2=0x55c356c0a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.679+0000 7f261ea00640 1 --2- 172.21.15.47:0/2358776225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356c0a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.679+0000 7f261ea00640 1 -- 172.21.15.47:0/2358776225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c356c03520 con 0x55c356be8de0 2024-03-21T11:47:22.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.680+0000 7f261ea00640 1 --2- 172.21.15.47:0/2358776225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356be8de0 0x55c356be7fe0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f26180099a0 tx=0x7f2618000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a91cd9e325d8df46 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.680+0000 7f260aa00640 1 -- 172.21.15.47:0/2358776225 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618041040 con 0x55c356be8de0 2024-03-21T11:47:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.681+0000 7f260aa00640 1 -- 172.21.15.47:0/2358776225 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2618003df0 con 0x55c356be8de0 2024-03-21T11:47:22.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.681+0000 7f260aa00640 1 -- 172.21.15.47:0/2358776225 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261802c9d0 con 0x55c356be8de0 2024-03-21T11:47:22.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.682+0000 7f2628265740 1 -- 172.21.15.47:0/2358776225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356be8de0 msgr2=0x55c356be7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.682+0000 7f2628265740 1 --2- 172.21.15.47:0/2358776225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356be8de0 0x55c356be7fe0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f26180099a0 tx=0x7f2618000f40 comp rx=0 tx=0).stop 2024-03-21T11:47:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.682+0000 7f2628265740 1 -- 172.21.15.47:0/2358776225 shutdown_connections 2024-03-21T11:47:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.682+0000 7f2628265740 1 --2- 172.21.15.47:0/2358776225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356c0a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.682+0000 7f2628265740 1 --2- 172.21.15.47:0/2358776225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356bed9f0 0x55c356bff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.682+0000 7f2628265740 1 --2- 172.21.15.47:0/2358776225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356be8de0 0x55c356be7fe0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.682+0000 7f2628265740 1 -- 172.21.15.47:0/2358776225 >> 172.21.15.47:0/2358776225 conn(0x55c356bf6dc0 msgr2=0x55c356be78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:22.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.682+0000 7f2628265740 1 -- 172.21.15.47:0/2358776225 shutdown_connections 2024-03-21T11:47:22.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.682+0000 7f2628265740 1 -- 172.21.15.47:0/2358776225 wait complete. 2024-03-21T11:47:22.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.683+0000 7f2628265740 1 -- start start 2024-03-21T11:47:22.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.683+0000 7f2628265740 1 Processor -- start 2024-03-21T11:47:22.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.684+0000 7f2628265740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356be8de0 0x55c356bfbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.684+0000 7f2628265740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bed9f0 0x55c356bfc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.684+0000 7f2628265740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356c67840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.684+0000 7f261ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356be8de0 0x55c356bfbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.685+0000 7f261ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356be8de0 0x55c356bfbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48112/0 (socket says 172.21.15.47:48112) 2024-03-21T11:47:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.685+0000 7f261e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bed9f0 0x55c356bfc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.685+0000 7f261e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bed9f0 0x55c356bfc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59744/0 (socket says 172.21.15.47:59744) 2024-03-21T11:47:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.685+0000 7f2628265740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26201b62e0 con 0x55c356beebc0 2024-03-21T11:47:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.685+0000 7f2628265740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26201b5c80 con 0x55c356bed9f0 2024-03-21T11:47:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.685+0000 7f2628265740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26201b3180 con 0x55c356be8de0 2024-03-21T11:47:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.685+0000 7f261f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356c67840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.685+0000 7f2628265740 10 client.0 _ll_register_callbacks cb 0x7f260c00b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:22.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.685+0000 7f261ea00640 1 -- 172.21.15.47:0/2026101985 learned_addr learned my addr 172.21.15.47:0/2026101985 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:22.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.686+0000 7f261f400640 1 -- 172.21.15.47:0/2026101985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356be8de0 msgr2=0x55c356bfbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.686+0000 7f261f400640 1 --2- 172.21.15.47:0/2026101985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356be8de0 0x55c356bfbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.686+0000 7f261f400640 1 -- 172.21.15.47:0/2026101985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bed9f0 msgr2=0x55c356bfc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:22.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.686+0000 7f261f400640 1 --2- 172.21.15.47:0/2026101985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bed9f0 0x55c356bfc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.686+0000 7f261f400640 1 -- 172.21.15.47:0/2026101985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c356bff300 con 0x55c356beebc0 2024-03-21T11:47:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.686+0000 7f261e000640 1 --2- 172.21.15.47:0/2026101985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bed9f0 0x55c356bfc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.687+0000 7f261f400640 1 --2- 172.21.15.47:0/2026101985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356c67840 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f260c000c00 tx=0x7f260c01e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e90854669fa4cf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.687+0000 7f25ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:47:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.687+0000 7f25ffe00640 1 -- 172.21.15.47:0/2026101985 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c01cb40 con 0x55c356beebc0 2024-03-21T11:47:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.687+0000 7f25ffe00640 1 -- 172.21.15.47:0/2026101985 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f260c01cce0 con 0x55c356beebc0 2024-03-21T11:47:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.687+0000 7f2628265740 1 -- 172.21.15.47:0/2026101985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2618003df0 con 0x55c356beebc0 2024-03-21T11:47:22.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.688+0000 7f25ffe00640 1 -- 172.21.15.47:0/2026101985 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f260c03e430 con 0x55c356beebc0 2024-03-21T11:47:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.688+0000 7f2628265740 10 client.22281 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.688+0000 7f2628265740 1 -- 172.21.15.47:0/2026101985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c356c68d60 con 0x55c356beebc0 2024-03-21T11:47:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.690+0000 7f25ffe00640 1 -- 172.21.15.47:0/2026101985 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2618003df0 con 0x55c356beebc0 2024-03-21T11:47:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.690+0000 7f25ffe00640 1 -- 172.21.15.47:0/2026101985 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f260c047b60 con 0x55c356beebc0 2024-03-21T11:47:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.690+0000 7f2628265740 10 client.22281 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.690+0000 7f2628265740 10 client.22281 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:47:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.690+0000 7f2628265740 20 client.22281 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.690+0000 7f2628265740 1 --2- 172.21.15.47:0/2026101985 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c356bef580 0x55c356c6b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.690+0000 7f2628265740 4 client.22281 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:47:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.690+0000 7f2628265740 1 -- 172.21.15.47:0/2026101985 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c356b59b50 con 0x55c356bef580 2024-03-21T11:47:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.691+0000 7f261ea00640 1 --2- 172.21.15.47:0/2026101985 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c356bef580 0x55c356c6b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:22.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.692+0000 7f261ea00640 1 --2- 172.21.15.47:0/2026101985 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c356bef580 0x55c356c6b740 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f48a5082f7321bd7 server_cookie=f41bc5e5464f715f in_seq=0 out_seq=0 2024-03-21T11:47:22.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.692+0000 7f25ffe00640 10 client.22281 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:47:22.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.696+0000 7f25ffe00640 1 -- 172.21.15.47:0/2026101985 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c356b59b50 con 0x55c356bef580 2024-03-21T11:47:22.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.696+0000 7f25ffe00640 10 client.22281 handle_command_reply: tid=0 2024-03-21T11:47:22.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.696+0000 7f2628265740 1 client.22281 shutdown 2024-03-21T11:47:22.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.696+0000 7f2628265740 1 -- 172.21.15.47:0/2026101985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 msgr2=0x55c356c67840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.696+0000 7f2628265740 1 --2- 172.21.15.47:0/2026101985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356c67840 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f260c000c00 tx=0x7f260c01e860 comp rx=0 tx=0).stop 2024-03-21T11:47:22.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.696+0000 7f2628265740 1 -- 172.21.15.47:0/2026101985 shutdown_connections 2024-03-21T11:47:22.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.697+0000 7f2628265740 1 --2- 172.21.15.47:0/2026101985 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55c356bef580 0x55c356c6b740 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.697+0000 7f25ffe00640 0 client.22281 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:47:22.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.697+0000 7f2628265740 1 --2- 172.21.15.47:0/2026101985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c356beebc0 0x55c356c67840 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.697+0000 7f2628265740 1 --2- 172.21.15.47:0/2026101985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c356bed9f0 0x55c356bfc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.697+0000 7f2628265740 1 --2- 172.21.15.47:0/2026101985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c356be8de0 0x55c356bfbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.697+0000 7f2628265740 1 -- 172.21.15.47:0/2026101985 >> 172.21.15.47:0/2026101985 conn(0x55c356bf6dc0 msgr2=0x55c356be8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:22.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.697+0000 7f2628265740 1 -- 172.21.15.47:0/2026101985 shutdown_connections 2024-03-21T11:47:22.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.698+0000 7f2628265740 1 -- 172.21.15.47:0/2026101985 wait complete. 2024-03-21T11:47:22.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.700+0000 7f2628265740 20 client.22281 trim_cache size 0 max 16384 2024-03-21T11:47:22.702 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:47:22.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.703+0000 7f261fe00640 1 -- 172.21.15.47:0/946999079 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26040a2fa0 msgr2=0x7f26040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.703+0000 7f261fe00640 1 --2- 172.21.15.47:0/946999079 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26040a2fa0 0x7f26040a5460 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f260c00b590 tx=0x7f260c00b520 comp rx=0 tx=0).stop 2024-03-21T11:47:22.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.704+0000 7f261fe00640 1 -- 172.21.15.47:0/946999079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2620107260 msgr2=0x7f26200fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:22.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.704+0000 7f261fe00640 1 --2- 172.21.15.47:0/946999079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2620107260 0x7f26200fffb0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f261000e9f0 tx=0x7f261000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.704+0000 7f261fe00640 1 -- 172.21.15.47:0/946999079 shutdown_connections 2024-03-21T11:47:22.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.704+0000 7f261fe00640 1 --2- 172.21.15.47:0/946999079 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f26040a2fa0 0x7f26040a5460 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.704+0000 7f261fe00640 1 --2- 172.21.15.47:0/946999079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26201004f0 0x7f26201009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.704+0000 7f261fe00640 1 --2- 172.21.15.47:0/946999079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2620107260 0x7f26200fffb0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.704+0000 7f261fe00640 1 --2- 172.21.15.47:0/946999079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2620104930 0x7f2620101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:22.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.704+0000 7f261fe00640 1 -- 172.21.15.47:0/946999079 >> 172.21.15.47:0/946999079 conn(0x7f26200fba10 msgr2=0x7f2620102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:22.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.707+0000 7f261fe00640 1 -- 172.21.15.47:0/946999079 shutdown_connections 2024-03-21T11:47:22.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:22.707+0000 7f261fe00640 1 -- 172.21.15.47:0/946999079 wait complete. 2024-03-21T11:47:22.736 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:47:22.736 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:47:22.993 INFO:tasks.workunit.client.0.smithi047.stdout: 10 254431 21.74 MB/sec execute 486 sec latency 2507.934 ms 2024-03-21T11:47:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.059+0000 7fd139800640 1 -- 172.21.15.47:0/1373097388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd134108eb0 msgr2=0x7fd134109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.059+0000 7fd139800640 1 --2- 172.21.15.47:0/1373097388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd134108eb0 0x7fd134109330 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd120009860 tx=0x7fd12002bb70 comp rx=0 tx=0).stop 2024-03-21T11:47:23.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.059+0000 7fd139800640 1 -- 172.21.15.47:0/1373097388 shutdown_connections 2024-03-21T11:47:23.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.059+0000 7fd139800640 1 --2- 172.21.15.47:0/1373097388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd134109870 0x7fd134110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.059+0000 7fd139800640 1 --2- 172.21.15.47:0/1373097388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd134108eb0 0x7fd134109330 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.059+0000 7fd139800640 1 --2- 172.21.15.47:0/1373097388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd134107cc0 0x7fd1341080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.059+0000 7fd139800640 1 -- 172.21.15.47:0/1373097388 >> 172.21.15.47:0/1373097388 conn(0x7fd134076c60 msgr2=0x7fd1340790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:23.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.059+0000 7fd139800640 1 -- 172.21.15.47:0/1373097388 shutdown_connections 2024-03-21T11:47:23.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.059+0000 7fd139800640 1 -- 172.21.15.47:0/1373097388 wait complete. 2024-03-21T11:47:23.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.060+0000 7fd139800640 1 Processor -- start 2024-03-21T11:47:23.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.061+0000 7fd139800640 1 -- start start 2024-03-21T11:47:23.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.061+0000 7fd139800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd134107cc0 0x7fd13419e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.061+0000 7fd139800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd134108eb0 0x7fd13419e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.061+0000 7fd139800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd134109870 0x7fd1341a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.061+0000 7fd139800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd134113970 con 0x7fd134107cc0 2024-03-21T11:47:23.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.061+0000 7fd139800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1341137f0 con 0x7fd134108eb0 2024-03-21T11:47:23.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.061+0000 7fd139800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd134113af0 con 0x7fd134109870 2024-03-21T11:47:23.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.061+0000 7fd132000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd134108eb0 0x7fd13419e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.062+0000 7fd132000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd134108eb0 0x7fd13419e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59762/0 (socket says 172.21.15.47:59762) 2024-03-21T11:47:23.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.062+0000 7fd132000640 1 -- 172.21.15.47:0/3766016108 learned_addr learned my addr 172.21.15.47:0/3766016108 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:23.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.062+0000 7fd133400640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd134109870 0x7fd1341a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.062+0000 7fd132a00640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd134107cc0 0x7fd13419e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.062+0000 7fd133400640 1 -- 172.21.15.47:0/3766016108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd134108eb0 msgr2=0x7fd13419e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:23.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.062+0000 7fd133400640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd134108eb0 0x7fd13419e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.062+0000 7fd133400640 1 -- 172.21.15.47:0/3766016108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd134107cc0 msgr2=0x7fd13419e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:23.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.062+0000 7fd133400640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd134107cc0 0x7fd13419e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.062+0000 7fd133400640 1 -- 172.21.15.47:0/3766016108 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1341a5c40 con 0x7fd134109870 2024-03-21T11:47:23.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.062+0000 7fd132a00640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd134107cc0 0x7fd13419e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:47:23.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.062+0000 7fd132000640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd134108eb0 0x7fd13419e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:23.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.063+0000 7fd133400640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd134109870 0x7fd1341a55d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00ed70 tx=0x7fd12c00c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:23.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.063+0000 7fd12b400640 1 -- 172.21.15.47:0/3766016108 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c00ee60 con 0x7fd134109870 2024-03-21T11:47:23.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.063+0000 7fd139800640 1 -- 172.21.15.47:0/3766016108 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1341a5f30 con 0x7fd134109870 2024-03-21T11:47:23.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.063+0000 7fd139800640 1 -- 172.21.15.47:0/3766016108 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1341a63f0 con 0x7fd134109870 2024-03-21T11:47:23.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.063+0000 7fd12b400640 1 -- 172.21.15.47:0/3766016108 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd12c003b70 con 0x7fd134109870 2024-03-21T11:47:23.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.063+0000 7fd12b400640 1 -- 172.21.15.47:0/3766016108 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0178a0 con 0x7fd134109870 2024-03-21T11:47:23.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.067+0000 7fd13c5c7740 1 Processor -- start 2024-03-21T11:47:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.069+0000 7fd12b400640 1 -- 172.21.15.47:0/3766016108 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd12c004020 con 0x7fd134109870 2024-03-21T11:47:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.069+0000 7fd13c5c7740 1 -- start start 2024-03-21T11:47:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.069+0000 7fd13c5c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fb810 0x556e3e1fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.069+0000 7fd13c5c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fc9e0 0x556e3e1fce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.069+0000 7fd13c5c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fd3a0 0x556e3e203c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.069+0000 7fd13c5c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3e1602c0 con 0x556e3e1fd3a0 2024-03-21T11:47:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.070+0000 7fd13c5c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3e108c90 con 0x556e3e1fc9e0 2024-03-21T11:47:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.070+0000 7fd13c5c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3e15c360 con 0x556e3e1fb810 2024-03-21T11:47:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.070+0000 7fd132000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fd3a0 0x556e3e203c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.071+0000 7fd132000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fb810 0x556e3e1fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.071+0000 7fd132a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fc9e0 0x556e3e1fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.071+0000 7fd132a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fc9e0 0x556e3e1fce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59770/0 (socket says 172.21.15.47:59770) 2024-03-21T11:47:23.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.071+0000 7fd132a00640 1 -- 172.21.15.47:0/1614980800 learned_addr learned my addr 172.21.15.47:0/1614980800 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:23.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.071+0000 7fd12b400640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1180a3070 0x7fd1180a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.072+0000 7fd12b400640 1 -- 172.21.15.47:0/3766016108 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd12c0cf870 con 0x7fd134109870 2024-03-21T11:47:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.072+0000 7fd132a00640 1 -- 172.21.15.47:0/1614980800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fb810 msgr2=0x556e3e1fbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.072+0000 7fd132a00640 1 --2- 172.21.15.47:0/1614980800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fb810 0x556e3e1fbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.072+0000 7fd132a00640 1 -- 172.21.15.47:0/1614980800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fd3a0 msgr2=0x556e3e203c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.072+0000 7fd133400640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1180a3070 0x7fd1180a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.072+0000 7fd132a00640 1 --2- 172.21.15.47:0/1614980800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fd3a0 0x556e3e203c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.072+0000 7fd132a00640 1 -- 172.21.15.47:0/1614980800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e3e12d080 con 0x556e3e1fc9e0 2024-03-21T11:47:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.073+0000 7fd132000640 1 --2- 172.21.15.47:0/1614980800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fd3a0 0x556e3e203c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:47:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.073+0000 7fd133400640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1180a3070 0x7fd1180a5530 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd12c005c40 tx=0x7fd12c011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.073+0000 7fd132a00640 1 --2- 172.21.15.47:0/1614980800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fc9e0 0x556e3e1fce60 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fd12400cc50 tx=0x7fd12400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d377fceb872f2fe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.073+0000 7fd11fe00640 1 -- 172.21.15.47:0/1614980800 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124014070 con 0x556e3e1fc9e0 2024-03-21T11:47:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.074+0000 7fd11fe00640 1 -- 172.21.15.47:0/1614980800 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1240047a0 con 0x556e3e1fc9e0 2024-03-21T11:47:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.074+0000 7fd11fe00640 1 -- 172.21.15.47:0/1614980800 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124010890 con 0x556e3e1fc9e0 2024-03-21T11:47:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.075+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1614980800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fc9e0 msgr2=0x556e3e1fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.075+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1614980800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fc9e0 0x556e3e1fce60 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fd12400cc50 tx=0x7fd12400e720 comp rx=0 tx=0).stop 2024-03-21T11:47:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.075+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1614980800 shutdown_connections 2024-03-21T11:47:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.075+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1614980800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fd3a0 0x556e3e203c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.075+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1614980800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fc9e0 0x556e3e1fce60 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.075+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1614980800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fb810 0x556e3e1fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.075+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1614980800 >> 172.21.15.47:0/1614980800 conn(0x556e3e1f6c00 msgr2=0x556e3e1f7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:23.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.077+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1614980800 shutdown_connections 2024-03-21T11:47:23.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.077+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1614980800 wait complete. 2024-03-21T11:47:23.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.078+0000 7fd13c5c7740 1 -- start start 2024-03-21T11:47:23.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.078+0000 7fd13c5c7740 1 Processor -- start 2024-03-21T11:47:23.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.079+0000 7fd13c5c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fb810 0x556e3e216aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.079+0000 7fd13c5c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fc9e0 0x556e3e216fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.079+0000 7fd13c5c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fd3a0 0x556e3e21e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.079+0000 7fd13c5c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3e2153c0 con 0x556e3e1fd3a0 2024-03-21T11:47:23.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.079+0000 7fd13c5c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3e11c0d0 con 0x556e3e1fb810 2024-03-21T11:47:23.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.079+0000 7fd13c5c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3e215540 con 0x556e3e1fc9e0 2024-03-21T11:47:23.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.079+0000 7fd13c5c7740 10 client.0 _ll_register_callbacks cb 0x556e3e120e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:23.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.079+0000 7fd132000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fb810 0x556e3e216aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.080+0000 7fd132000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fb810 0x556e3e216aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59776/0 (socket says 172.21.15.47:59776) 2024-03-21T11:47:23.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.080+0000 7fd132000640 1 -- 172.21.15.47:0/1625489949 learned_addr learned my addr 172.21.15.47:0/1625489949 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:23.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.080+0000 7fd132000640 1 --2- 172.21.15.47:0/1625489949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fd3a0 0x556e3e21e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.080+0000 7fd132000640 1 -- 172.21.15.47:0/1625489949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fc9e0 msgr2=0x556e3e216fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:23.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.080+0000 7fd132000640 1 --2- 172.21.15.47:0/1625489949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fc9e0 0x556e3e216fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.080+0000 7fd132000640 1 -- 172.21.15.47:0/1625489949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fd3a0 msgr2=0x556e3e21e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:23.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.080+0000 7fd132000640 1 --2- 172.21.15.47:0/1625489949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fd3a0 0x556e3e21e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.080+0000 7fd132000640 1 -- 172.21.15.47:0/1625489949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e3de743f0 con 0x556e3e1fb810 2024-03-21T11:47:23.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.081+0000 7fd132000640 1 --2- 172.21.15.47:0/1625489949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fb810 0x556e3e216aa0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fd1200349e0 tx=0x7fd120032440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9901aac494e052ca server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:23.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.081+0000 7fd11cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:23.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.081+0000 7fd11cc00640 1 -- 172.21.15.47:0/1625489949 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd120039040 con 0x556e3e1fb810 2024-03-21T11:47:23.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.082+0000 7fd11cc00640 1 -- 172.21.15.47:0/1625489949 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd12002c920 con 0x556e3e1fb810 2024-03-21T11:47:23.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.082+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1625489949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1240047a0 con 0x556e3e1fb810 2024-03-21T11:47:23.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.082+0000 7fd11cc00640 1 -- 172.21.15.47:0/1625489949 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd120052050 con 0x556e3e1fb810 2024-03-21T11:47:23.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.083+0000 7fd13c5c7740 10 client.33155 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:23.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.083+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1625489949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556e3df3e7d0 con 0x556e3e1fb810 2024-03-21T11:47:23.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.083+0000 7fd11cc00640 1 -- 172.21.15.47:0/1625489949 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1240047a0 con 0x556e3e1fb810 2024-03-21T11:47:23.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.084+0000 7fd11cc00640 1 -- 172.21.15.47:0/1625489949 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd12003f040 con 0x556e3e1fb810 2024-03-21T11:47:23.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.084+0000 7fd13c5c7740 10 client.33155 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:23.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.084+0000 7fd13c5c7740 10 client.33155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:23.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.084+0000 7fd13c5c7740 20 client.33155 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:23.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.084+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1625489949 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3e2225c0 0x556e3e224a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.084+0000 7fd13c5c7740 4 client.33155 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:23.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.084+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1625489949 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e3e16d800 con 0x556e3e2225c0 2024-03-21T11:47:23.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.086+0000 7fd132a00640 1 --2- 172.21.15.47:0/1625489949 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3e2225c0 0x556e3e224a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.087+0000 7fd132a00640 1 --2- 172.21.15.47:0/1625489949 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3e2225c0 0x556e3e224a60 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c64f5b30ee9dcb12 server_cookie=84597a940b997e15 in_seq=0 out_seq=0 2024-03-21T11:47:23.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.087+0000 7fd11cc00640 10 client.33155 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:23.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.088+0000 7fd11cc00640 1 -- 172.21.15.47:0/1625489949 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556e3e16d800 con 0x556e3e2225c0 2024-03-21T11:47:23.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.088+0000 7fd11cc00640 10 client.33155 handle_command_reply: tid=0 2024-03-21T11:47:23.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.089+0000 7fd13c5c7740 1 client.33155 shutdown 2024-03-21T11:47:23.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.089+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1625489949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fb810 msgr2=0x556e3e216aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:23.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.089+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1625489949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fb810 0x556e3e216aa0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fd1200349e0 tx=0x7fd120032440 comp rx=0 tx=0).stop 2024-03-21T11:47:23.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.089+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1625489949 shutdown_connections 2024-03-21T11:47:23.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.089+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1625489949 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3e2225c0 0x556e3e224a60 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.089+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1625489949 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fd3a0 0x556e3e21e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.089+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1625489949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fc9e0 0x556e3e216fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.089+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1625489949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fb810 0x556e3e216aa0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.089+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1625489949 >> 172.21.15.47:0/1625489949 conn(0x556e3e1f6c00 msgr2=0x556e3e1f5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:23.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.089+0000 7fd11cc00640 0 client.33155 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:23.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.089+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1625489949 shutdown_connections 2024-03-21T11:47:23.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.090+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1625489949 wait complete. 2024-03-21T11:47:23.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.090+0000 7fd13c5c7740 20 client.33155 trim_cache size 0 max 16384 2024-03-21T11:47:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.133+0000 7fd13c5c7740 1 Processor -- start 2024-03-21T11:47:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.133+0000 7fd13c5c7740 1 -- start start 2024-03-21T11:47:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.134+0000 7fd13c5c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fb810 0x556e3e216f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.134+0000 7fd13c5c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fc9e0 0x556e3e21dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.134+0000 7fd13c5c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fd3a0 0x556e3e1fa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.134+0000 7fd13c5c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11400f950 con 0x556e3e1fb810 2024-03-21T11:47:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.134+0000 7fd13c5c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1140112d0 con 0x556e3e1fd3a0 2024-03-21T11:47:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.134+0000 7fd13c5c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11400ffb0 con 0x556e3e1fc9e0 2024-03-21T11:47:23.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.135+0000 7fd132a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fc9e0 0x556e3e21dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.135+0000 7fd132a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fc9e0 0x556e3e21dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48168/0 (socket says 172.21.15.47:48168) 2024-03-21T11:47:23.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.135+0000 7fd132a00640 1 -- 172.21.15.47:0/1848782674 learned_addr learned my addr 172.21.15.47:0/1848782674 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:23.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.135+0000 7fd132000640 1 --2- 172.21.15.47:0/1848782674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fb810 0x556e3e216f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.136+0000 7fd132a00640 1 -- 172.21.15.47:0/1848782674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fd3a0 msgr2=0x556e3e1fa570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:47:23.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.136+0000 7fd132a00640 1 --2- 172.21.15.47:0/1848782674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fd3a0 0x556e3e1fa570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.136+0000 7fd132a00640 1 -- 172.21.15.47:0/1848782674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fb810 msgr2=0x556e3e216f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:23.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.136+0000 7fd132a00640 1 --2- 172.21.15.47:0/1848782674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fb810 0x556e3e216f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.136+0000 7fd132a00640 1 -- 172.21.15.47:0/1848782674 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e3e1f7670 con 0x556e3e1fc9e0 2024-03-21T11:47:23.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.136+0000 7fd132a00640 1 --2- 172.21.15.47:0/1848782674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fc9e0 0x556e3e21dfe0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x556e3e1f7b70 tx=0x7fd12400ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b85b38cbf8719943 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:23.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.138+0000 7fd11f400640 1 -- 172.21.15.47:0/1848782674 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124014040 con 0x556e3e1fc9e0 2024-03-21T11:47:23.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.138+0000 7fd11f400640 1 -- 172.21.15.47:0/1848782674 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1240047a0 con 0x556e3e1fc9e0 2024-03-21T11:47:23.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.138+0000 7fd11f400640 1 -- 172.21.15.47:0/1848782674 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124025bc0 con 0x556e3e1fc9e0 2024-03-21T11:47:23.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.139+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1848782674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fc9e0 msgr2=0x556e3e21dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:23.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.139+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1848782674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fc9e0 0x556e3e21dfe0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x556e3e1f7b70 tx=0x7fd12400ef50 comp rx=0 tx=0).stop 2024-03-21T11:47:23.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.139+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1848782674 shutdown_connections 2024-03-21T11:47:23.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.139+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1848782674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fd3a0 0x556e3e1fa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.139+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1848782674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fc9e0 0x556e3e21dfe0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.139+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1848782674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fb810 0x556e3e216f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.139+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1848782674 >> 172.21.15.47:0/1848782674 conn(0x556e3e1f6c00 msgr2=0x556e3e1f9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:23.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.140+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1848782674 shutdown_connections 2024-03-21T11:47:23.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.140+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1848782674 wait complete. 2024-03-21T11:47:23.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.140+0000 7fd13c5c7740 1 -- start start 2024-03-21T11:47:23.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.141+0000 7fd13c5c7740 1 Processor -- start 2024-03-21T11:47:23.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.142+0000 7fd13c5c7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fb810 0x556e3e219b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.142+0000 7fd13c5c7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fc9e0 0x556e3e21a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.143+0000 7fd13c5c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fd3a0 0x556e3e21cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.143+0000 7fd132000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fb810 0x556e3e219b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.143+0000 7fd132000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fb810 0x556e3e219b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59798/0 (socket says 172.21.15.47:59798) 2024-03-21T11:47:23.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.143+0000 7fd13c5c7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd114010c70 con 0x556e3e1fc9e0 2024-03-21T11:47:23.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.143+0000 7fd13c5c7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd114010610 con 0x556e3e1fb810 2024-03-21T11:47:23.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.143+0000 7fd13c5c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11400db10 con 0x556e3e1fd3a0 2024-03-21T11:47:23.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.143+0000 7fd13c5c7740 10 client.0 _ll_register_callbacks cb 0x7fd1200096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:23.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.143+0000 7fd132a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fc9e0 0x556e3e21a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.143+0000 7fd132a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fc9e0 0x556e3e21a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37408/0 (socket says 172.21.15.47:37408) 2024-03-21T11:47:23.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.143+0000 7fd132a00640 1 -- 172.21.15.47:0/1122560894 learned_addr learned my addr 172.21.15.47:0/1122560894 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:23.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.144+0000 7fd132000640 1 -- 172.21.15.47:0/1122560894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fd3a0 msgr2=0x556e3e21cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:23.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.144+0000 7fd132000640 1 --2- 172.21.15.47:0/1122560894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fd3a0 0x556e3e21cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.144+0000 7fd132000640 1 -- 172.21.15.47:0/1122560894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fc9e0 msgr2=0x556e3e21a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:23.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.144+0000 7fd132000640 1 --2- 172.21.15.47:0/1122560894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fc9e0 0x556e3e21a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.144+0000 7fd132000640 1 -- 172.21.15.47:0/1122560894 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e3e21d5d0 con 0x556e3e1fb810 2024-03-21T11:47:23.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.145+0000 7fd132000640 1 --2- 172.21.15.47:0/1122560894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fb810 0x556e3e219b90 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x556e3e1fc890 tx=0x7fd120005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36ec4b344aca5650 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:23.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.145+0000 7fd11cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.146+0000 7fd11cc00640 1 -- 172.21.15.47:0/1122560894 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd120039040 con 0x556e3e1fb810 2024-03-21T11:47:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.146+0000 7fd11cc00640 1 -- 172.21.15.47:0/1122560894 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd1200042b0 con 0x556e3e1fb810 2024-03-21T11:47:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.146+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1122560894 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1240047a0 con 0x556e3e1fb810 2024-03-21T11:47:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.147+0000 7fd11cc00640 1 -- 172.21.15.47:0/1122560894 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd120067050 con 0x556e3e1fb810 2024-03-21T11:47:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.147+0000 7fd13c5c7740 10 client.33158 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.147+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1122560894 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556e3e27e240 con 0x556e3e1fb810 2024-03-21T11:47:23.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.148+0000 7fd11cc00640 1 -- 172.21.15.47:0/1122560894 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1240047a0 con 0x556e3e1fb810 2024-03-21T11:47:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.148+0000 7fd11cc00640 1 -- 172.21.15.47:0/1122560894 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd1200327a0 con 0x556e3e1fb810 2024-03-21T11:47:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.148+0000 7fd13c5c7740 10 client.33158 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.148+0000 7fd13c5c7740 10 client.33158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.148+0000 7fd13c5c7740 20 client.33158 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.148+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1122560894 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3e27eae0 0x556e3e280f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.148+0000 7fd13c5c7740 4 client.33158 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:47:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.148+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1122560894 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556e3e167b50 con 0x556e3e27eae0 2024-03-21T11:47:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.149+0000 7fd132a00640 1 --2- 172.21.15.47:0/1122560894 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3e27eae0 0x556e3e280f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.151+0000 7fd11cc00640 10 client.33158 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:23.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.151+0000 7fd132a00640 1 --2- 172.21.15.47:0/1122560894 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3e27eae0 0x556e3e280f00 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72e3564c8ae1d5b4 server_cookie=a4843afeb86bff02 in_seq=0 out_seq=0 2024-03-21T11:47:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.152+0000 7fd11cc00640 1 -- 172.21.15.47:0/1122560894 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556e3e167b50 con 0x556e3e27eae0 2024-03-21T11:47:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.152+0000 7fd11cc00640 10 client.33158 handle_command_reply: tid=0 2024-03-21T11:47:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.152+0000 7fd13c5c7740 1 client.33158 shutdown 2024-03-21T11:47:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.153+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1122560894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fb810 msgr2=0x556e3e219b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.153+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1122560894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fb810 0x556e3e219b90 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x556e3e1fc890 tx=0x7fd120005d30 comp rx=0 tx=0).stop 2024-03-21T11:47:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.153+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1122560894 shutdown_connections 2024-03-21T11:47:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.153+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1122560894 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556e3e27eae0 0x556e3e280f00 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.153+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1122560894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556e3e1fd3a0 0x556e3e21cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.153+0000 7fd11cc00640 0 client.33158 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.153+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1122560894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556e3e1fc9e0 0x556e3e21a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.153+0000 7fd13c5c7740 1 --2- 172.21.15.47:0/1122560894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556e3e1fb810 0x556e3e219b90 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.154+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1122560894 >> 172.21.15.47:0/1122560894 conn(0x556e3e1f6c00 msgr2=0x556e3e1fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.154+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1122560894 shutdown_connections 2024-03-21T11:47:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.155+0000 7fd13c5c7740 1 -- 172.21.15.47:0/1122560894 wait complete. 2024-03-21T11:47:23.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.155+0000 7fd13c5c7740 20 client.33158 trim_cache size 0 max 16384 2024-03-21T11:47:23.157 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:47:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.156+0000 7fd139800640 1 -- 172.21.15.47:0/3766016108 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1180a3070 msgr2=0x7fd1180a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.156+0000 7fd139800640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1180a3070 0x7fd1180a5530 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd12c005c40 tx=0x7fd12c011040 comp rx=0 tx=0).stop 2024-03-21T11:47:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.156+0000 7fd139800640 1 -- 172.21.15.47:0/3766016108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd134109870 msgr2=0x7fd1341a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.156+0000 7fd139800640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd134109870 0x7fd1341a55d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00ed70 tx=0x7fd12c00c6e0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.156+0000 7fd139800640 1 -- 172.21.15.47:0/3766016108 shutdown_connections 2024-03-21T11:47:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.156+0000 7fd139800640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd1180a3070 0x7fd1180a5530 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.156+0000 7fd139800640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd134109870 0x7fd1341a55d0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.156+0000 7fd139800640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd134108eb0 0x7fd13419e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.156+0000 7fd139800640 1 --2- 172.21.15.47:0/3766016108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd134107cc0 0x7fd13419e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.156+0000 7fd139800640 1 -- 172.21.15.47:0/3766016108 >> 172.21.15.47:0/3766016108 conn(0x7fd134076c60 msgr2=0x7fd13410c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.156+0000 7fd139800640 1 -- 172.21.15.47:0/3766016108 shutdown_connections 2024-03-21T11:47:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:23.156+0000 7fd139800640 1 -- 172.21.15.47:0/3766016108 wait complete. 2024-03-21T11:47:23.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:23 smithi047 ceph-mon[31351]: pgmap v1122: 129 pgs: 129 active+clean; 725 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.9 MiB/s wr, 514 op/s 2024-03-21T11:47:23.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:23 smithi100 ceph-mon[36245]: pgmap v1122: 129 pgs: 129 active+clean; 725 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.9 MiB/s wr, 514 op/s 2024-03-21T11:47:23.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:23 smithi102 ceph-mon[36748]: pgmap v1122: 129 pgs: 129 active+clean; 725 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.9 MiB/s wr, 514 op/s 2024-03-21T11:47:23.993 INFO:tasks.workunit.client.0.smithi047.stdout: 10 254677 21.72 MB/sec execute 487 sec latency 696.561 ms 2024-03-21T11:47:24.187 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:47:24.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:47:24.188 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:47:24.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.493+0000 7f43f2000640 1 -- 172.21.15.47:0/3552201649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43ec103260 msgr2=0x7f43ec109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.493+0000 7f43f2000640 1 --2- 172.21.15.47:0/3552201649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43ec103260 0x7f43ec109af0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009ca0 tx=0x7f43e402c120 comp rx=0 tx=0).stop 2024-03-21T11:47:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.493+0000 7f43f2000640 1 -- 172.21.15.47:0/3552201649 shutdown_connections 2024-03-21T11:47:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.493+0000 7f43f2000640 1 --2- 172.21.15.47:0/3552201649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43ec103260 0x7f43ec109af0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.493+0000 7f43f2000640 1 --2- 172.21.15.47:0/3552201649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ec1028a0 0x7f43ec102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.493+0000 7f43f2000640 1 --2- 172.21.15.47:0/3552201649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43ec1016b0 0x7f43ec101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.493+0000 7f43f2000640 1 -- 172.21.15.47:0/3552201649 >> 172.21.15.47:0/3552201649 conn(0x7f43ec0fcc40 msgr2=0x7f43ec0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.494+0000 7f43f2000640 1 -- 172.21.15.47:0/3552201649 shutdown_connections 2024-03-21T11:47:24.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.494+0000 7f43f2000640 1 -- 172.21.15.47:0/3552201649 wait complete. 2024-03-21T11:47:24.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.495+0000 7f43f2000640 1 Processor -- start 2024-03-21T11:47:24.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.495+0000 7f43f2000640 1 -- start start 2024-03-21T11:47:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.496+0000 7f43f2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43ec1016b0 0x7f43ec19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.496+0000 7f43f2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43ec1028a0 0x7f43ec19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.496+0000 7f43f2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ec103260 0x7f43ec1a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.496+0000 7f43f2000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec10d3a0 con 0x7f43ec103260 2024-03-21T11:47:24.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.496+0000 7f43f2000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec10d220 con 0x7f43ec1016b0 2024-03-21T11:47:24.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.496+0000 7f43f2000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec10d520 con 0x7f43ec1028a0 2024-03-21T11:47:24.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.497+0000 7f43ebe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ec103260 0x7f43ec1a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.497+0000 7f43ebe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ec103260 0x7f43ec1a5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37424/0 (socket says 172.21.15.47:37424) 2024-03-21T11:47:24.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.497+0000 7f43ebe00640 1 -- 172.21.15.47:0/876693862 learned_addr learned my addr 172.21.15.47:0/876693862 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:24.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.497+0000 7f43ebe00640 1 -- 172.21.15.47:0/876693862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43ec1028a0 msgr2=0x7f43ec19e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:24.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.497+0000 7f43ebe00640 1 --2- 172.21.15.47:0/876693862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43ec1028a0 0x7f43ec19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.497+0000 7f43ebe00640 1 -- 172.21.15.47:0/876693862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43ec1016b0 msgr2=0x7f43ec19dfd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T11:47:24.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.497+0000 7f43ebe00640 1 --2- 172.21.15.47:0/876693862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43ec1016b0 0x7f43ec19dfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.497+0000 7f43ebe00640 1 -- 172.21.15.47:0/876693862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec1a5bf0 con 0x7f43ec103260 2024-03-21T11:47:24.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.498+0000 7f43ebe00640 1 --2- 172.21.15.47:0/876693862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ec103260 0x7f43ec1a5580 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009bf0 tx=0x7f43e4035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:24.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.498+0000 7f43e3e00640 1 -- 172.21.15.47:0/876693862 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4039040 con 0x7f43ec103260 2024-03-21T11:47:24.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.498+0000 7f43e3e00640 1 -- 172.21.15.47:0/876693862 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43e4044070 con 0x7f43ec103260 2024-03-21T11:47:24.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.499+0000 7f43f2000640 1 -- 172.21.15.47:0/876693862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ec1a5e80 con 0x7f43ec103260 2024-03-21T11:47:24.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.499+0000 7f43e3e00640 1 -- 172.21.15.47:0/876693862 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e40046e0 con 0x7f43ec103260 2024-03-21T11:47:24.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.499+0000 7f43f2000640 1 -- 172.21.15.47:0/876693862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ec1a63c0 con 0x7f43ec103260 2024-03-21T11:47:24.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.506+0000 7f43f4dbf740 1 Processor -- start 2024-03-21T11:47:24.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.511+0000 7f43e3e00640 1 -- 172.21.15.47:0/876693862 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f43e4032900 con 0x7f43ec103260 2024-03-21T11:47:24.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.512+0000 7f43f4dbf740 1 -- start start 2024-03-21T11:47:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.512+0000 7f43f4dbf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e50900a0 0x55e5e50904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.512+0000 7f43f4dbf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5090a90 0x55e5e5095590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.512+0000 7f43f4dbf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5095c00 0x55e5e5098090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.512+0000 7f43f4dbf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e4ff5eb0 con 0x55e5e5095c00 2024-03-21T11:47:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.512+0000 7f43f4dbf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e4f9ea60 con 0x55e5e5090a90 2024-03-21T11:47:24.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.512+0000 7f43f4dbf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e4ff1f50 con 0x55e5e50900a0 2024-03-21T11:47:24.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.514+0000 7f43eb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5090a90 0x55e5e5095590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.514+0000 7f43eb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5090a90 0x55e5e5095590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59824/0 (socket says 172.21.15.47:59824) 2024-03-21T11:47:24.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.514+0000 7f43eb400640 1 -- 172.21.15.47:0/3248869185 learned_addr learned my addr 172.21.15.47:0/3248869185 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:24.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.514+0000 7f43eaa00640 1 --2- 172.21.15.47:0/3248869185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5095c00 0x55e5e5098090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.514+0000 7f43eaa00640 1 --2- 172.21.15.47:0/3248869185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e50900a0 0x55e5e50904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.515+0000 7f43eb400640 1 -- 172.21.15.47:0/3248869185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e50900a0 msgr2=0x55e5e50904c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:24.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.515+0000 7f43eb400640 1 --2- 172.21.15.47:0/3248869185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e50900a0 0x55e5e50904c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.515+0000 7f43eb400640 1 -- 172.21.15.47:0/3248869185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5095c00 msgr2=0x55e5e5098090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:24.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.515+0000 7f43eb400640 1 --2- 172.21.15.47:0/3248869185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5095c00 0x55e5e5098090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.515+0000 7f43eb400640 1 -- 172.21.15.47:0/3248869185 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5e4dd27e0 con 0x55e5e5090a90 2024-03-21T11:47:24.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.516+0000 7f43eb400640 1 --2- 172.21.15.47:0/3248869185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5090a90 0x55e5e5095590 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009740 tx=0x7f43dc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b558ab38cb64cda server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:24.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.516+0000 7f43d7e00640 1 -- 172.21.15.47:0/3248869185 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc015020 con 0x55e5e5090a90 2024-03-21T11:47:24.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.516+0000 7f43d7e00640 1 -- 172.21.15.47:0/3248869185 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43dc009e50 con 0x55e5e5090a90 2024-03-21T11:47:24.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.516+0000 7f43d7e00640 1 -- 172.21.15.47:0/3248869185 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0105e0 con 0x55e5e5090a90 2024-03-21T11:47:24.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.517+0000 7f43f4dbf740 1 -- 172.21.15.47:0/3248869185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5090a90 msgr2=0x55e5e5095590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:24.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.517+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/3248869185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5090a90 0x55e5e5095590 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009740 tx=0x7f43dc009c00 comp rx=0 tx=0).stop 2024-03-21T11:47:24.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.517+0000 7f43e3e00640 1 --2- 172.21.15.47:0/876693862 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f43d00a2fa0 0x7f43d00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.518+0000 7f43e3e00640 1 -- 172.21.15.47:0/876693862 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f43e40f2170 con 0x7f43ec103260 2024-03-21T11:47:24.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.518+0000 7f43eaa00640 1 --2- 172.21.15.47:0/876693862 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f43d00a2fa0 0x7f43d00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.518+0000 7f43f4dbf740 1 -- 172.21.15.47:0/3248869185 shutdown_connections 2024-03-21T11:47:24.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.518+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/3248869185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5095c00 0x55e5e5098090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.518+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/3248869185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5090a90 0x55e5e5095590 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.518+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/3248869185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e50900a0 0x55e5e50904c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.518+0000 7f43f4dbf740 1 -- 172.21.15.47:0/3248869185 >> 172.21.15.47:0/3248869185 conn(0x55e5e508b610 msgr2=0x55e5e508ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:24.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.520+0000 7f43eaa00640 1 --2- 172.21.15.47:0/876693862 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f43d00a2fa0 0x7f43d00a5460 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x55e5e50912f0 tx=0x7f43d800a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:24.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.521+0000 7f43f4dbf740 1 -- 172.21.15.47:0/3248869185 shutdown_connections 2024-03-21T11:47:24.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.522+0000 7f43f4dbf740 1 -- 172.21.15.47:0/3248869185 wait complete. 2024-03-21T11:47:24.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.523+0000 7f43f4dbf740 1 -- start start 2024-03-21T11:47:24.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.523+0000 7f43f4dbf740 1 Processor -- start 2024-03-21T11:47:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.524+0000 7f43f4dbf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e50900a0 0x55e5e509a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.524+0000 7f43f4dbf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5090a90 0x55e5e509a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.524+0000 7f43f4dbf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e5095c00 0x55e5e50a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.524+0000 7f43f4dbf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e5098bb0 con 0x55e5e5090a90 2024-03-21T11:47:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.524+0000 7f43f4dbf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e4fb1eb0 con 0x55e5e50900a0 2024-03-21T11:47:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.524+0000 7f43f4dbf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e5098d30 con 0x55e5e5095c00 2024-03-21T11:47:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.524+0000 7f43f4dbf740 10 client.0 _ll_register_callbacks cb 0x55e5e4fa6150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.524+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5090a90 0x55e5e509a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.525+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5090a90 0x55e5e509a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37446/0 (socket says 172.21.15.47:37446) 2024-03-21T11:47:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.525+0000 7f43eaa00640 1 -- 172.21.15.47:0/2650373102 learned_addr learned my addr 172.21.15.47:0/2650373102 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.525+0000 7f43ebe00640 1 --2- 172.21.15.47:0/2650373102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e5095c00 0x55e5e50a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.525+0000 7f43ebe00640 1 -- 172.21.15.47:0/2650373102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e50900a0 msgr2=0x55e5e509a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.525+0000 7f43ebe00640 1 --2- 172.21.15.47:0/2650373102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e50900a0 0x55e5e509a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.525+0000 7f43ebe00640 1 -- 172.21.15.47:0/2650373102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5090a90 msgr2=0x55e5e509a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.525+0000 7f43ebe00640 1 --2- 172.21.15.47:0/2650373102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5090a90 0x55e5e509a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.525+0000 7f43ebe00640 1 -- 172.21.15.47:0/2650373102 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5e4e32f70 con 0x55e5e5095c00 2024-03-21T11:47:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.526+0000 7f43d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:47:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.526+0000 7f43ebe00640 1 --2- 172.21.15.47:0/2650373102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e5095c00 0x55e5e50a18c0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f43e4048000 tx=0x7f43e40be340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea2742424644af65 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.526+0000 7f43d4c00640 1 -- 172.21.15.47:0/2650373102 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e40be910 con 0x55e5e5095c00 2024-03-21T11:47:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.526+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2650373102 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43dc009e50 con 0x55e5e5095c00 2024-03-21T11:47:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.526+0000 7f43d4c00640 1 -- 172.21.15.47:0/2650373102 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f43e40be710 con 0x55e5e5095c00 2024-03-21T11:47:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.529+0000 7f43d4c00640 1 -- 172.21.15.47:0/2650373102 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43e40be910 con 0x55e5e5095c00 2024-03-21T11:47:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.529+0000 7f43f4dbf740 10 client.33121 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.529+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2650373102 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e5e4cca170 con 0x55e5e5095c00 2024-03-21T11:47:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.531+0000 7f43d4c00640 1 -- 172.21.15.47:0/2650373102 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43dc009e50 con 0x55e5e5095c00 2024-03-21T11:47:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.531+0000 7f43d4c00640 1 -- 172.21.15.47:0/2650373102 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f43e403c030 con 0x55e5e5095c00 2024-03-21T11:47:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.531+0000 7f43f4dbf740 10 client.33121 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.531+0000 7f43f4dbf740 10 client.33121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.531+0000 7f43f4dbf740 20 client.33121 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.531+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2650373102 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e5e50a6210 0x55e5e50a86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.531+0000 7f43f4dbf740 4 client.33121 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.531+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2650373102 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5e50033f0 con 0x55e5e50a6210 2024-03-21T11:47:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.532+0000 7f43eb400640 1 --2- 172.21.15.47:0/2650373102 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e5e50a6210 0x55e5e50a86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.533+0000 7f43eb400640 1 --2- 172.21.15.47:0/2650373102 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e5e50a6210 0x55e5e50a86b0 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cea19c346dd3ba server_cookie=e6d4728ef00c093f in_seq=0 out_seq=0 2024-03-21T11:47:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.533+0000 7f43d4c00640 10 client.33121 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.539+0000 7f43d4c00640 1 -- 172.21.15.47:0/2650373102 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e5e50033f0 con 0x55e5e50a6210 2024-03-21T11:47:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.539+0000 7f43d4c00640 10 client.33121 handle_command_reply: tid=0 2024-03-21T11:47:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.539+0000 7f43f4dbf740 1 client.33121 shutdown 2024-03-21T11:47:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.540+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2650373102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e5095c00 msgr2=0x55e5e50a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.540+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2650373102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e5095c00 0x55e5e50a18c0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f43e4048000 tx=0x7f43e40be340 comp rx=0 tx=0).stop 2024-03-21T11:47:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.540+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2650373102 shutdown_connections 2024-03-21T11:47:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.540+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2650373102 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e5e50a6210 0x55e5e50a86b0 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.540+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2650373102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e5095c00 0x55e5e50a18c0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.540+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2650373102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5090a90 0x55e5e509a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.540+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2650373102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e50900a0 0x55e5e509a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.540+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2650373102 >> 172.21.15.47:0/2650373102 conn(0x55e5e508b610 msgr2=0x55e5e508a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.540+0000 7f43d4c00640 0 client.33121 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.540+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2650373102 shutdown_connections 2024-03-21T11:47:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.542+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2650373102 wait complete. 2024-03-21T11:47:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.542+0000 7f43f4dbf740 20 client.33121 trim_cache size 0 max 16384 2024-03-21T11:47:24.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.582+0000 7f43f4dbf740 1 Processor -- start 2024-03-21T11:47:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.584+0000 7f43f4dbf740 1 -- start start 2024-03-21T11:47:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.584+0000 7f43f4dbf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e50900a0 0x55e5e508eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.584+0000 7f43f4dbf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5090a90 0x55e5e50a1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.584+0000 7f43f4dbf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5095c00 0x55e5e50acf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.584+0000 7f43f4dbf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec1b0700 con 0x55e5e5095c00 2024-03-21T11:47:24.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.584+0000 7f43f4dbf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec1b2080 con 0x55e5e5090a90 2024-03-21T11:47:24.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.584+0000 7f43f4dbf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec1b0d60 con 0x55e5e50900a0 2024-03-21T11:47:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.584+0000 7f43eb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e50900a0 0x55e5e508eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.584+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5090a90 0x55e5e50a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.584+0000 7f43eb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e50900a0 0x55e5e508eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48212/0 (socket says 172.21.15.47:48212) 2024-03-21T11:47:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.584+0000 7f43eb400640 1 -- 172.21.15.47:0/2729139347 learned_addr learned my addr 172.21.15.47:0/2729139347 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:24.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.585+0000 7f43ebe00640 1 --2- 172.21.15.47:0/2729139347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5095c00 0x55e5e50acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.585+0000 7f43eb400640 1 -- 172.21.15.47:0/2729139347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5090a90 msgr2=0x55e5e50a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:24.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.585+0000 7f43eb400640 1 --2- 172.21.15.47:0/2729139347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5090a90 0x55e5e50a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.585+0000 7f43eb400640 1 -- 172.21.15.47:0/2729139347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5095c00 msgr2=0x55e5e50acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:24.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.585+0000 7f43eb400640 1 --2- 172.21.15.47:0/2729139347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5095c00 0x55e5e50acf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.585+0000 7f43eb400640 1 -- 172.21.15.47:0/2729139347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5e5086c20 con 0x55e5e50900a0 2024-03-21T11:47:24.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.585+0000 7f43eb400640 1 --2- 172.21.15.47:0/2729139347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e50900a0 0x55e5e508eee0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009660 tx=0x7f43dc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f03617e91de2ece2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:24.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.586+0000 7f43d7400640 1 -- 172.21.15.47:0/2729139347 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc015040 con 0x55e5e50900a0 2024-03-21T11:47:24.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.586+0000 7f43d7400640 1 -- 172.21.15.47:0/2729139347 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43dc011cc0 con 0x55e5e50900a0 2024-03-21T11:47:24.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.587+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2729139347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e50900a0 msgr2=0x55e5e508eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:24.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.587+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2729139347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e50900a0 0x55e5e508eee0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009660 tx=0x7f43dc000bc0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.588+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2729139347 shutdown_connections 2024-03-21T11:47:24.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.588+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2729139347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e5095c00 0x55e5e50acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.588+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2729139347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5090a90 0x55e5e50a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.588+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2729139347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e50900a0 0x55e5e508eee0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.588+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2729139347 >> 172.21.15.47:0/2729139347 conn(0x55e5e508b610 msgr2=0x55e5e508a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.588+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2729139347 shutdown_connections 2024-03-21T11:47:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.588+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2729139347 wait complete. 2024-03-21T11:47:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.588+0000 7f43f4dbf740 1 -- start start 2024-03-21T11:47:24.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.589+0000 7f43f4dbf740 1 Processor -- start 2024-03-21T11:47:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.589+0000 7f43f4dbf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e50900a0 0x55e5e50a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.589+0000 7f43f4dbf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e5090a90 0x55e5e509feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.589+0000 7f43f4dbf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5095c00 0x55e5e50a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.590+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e5090a90 0x55e5e509feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.590+0000 7f43ebe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5095c00 0x55e5e50a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.590+0000 7f43ebe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5095c00 0x55e5e50a04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59860/0 (socket says 172.21.15.47:59860) 2024-03-21T11:47:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.590+0000 7f43ebe00640 1 -- 172.21.15.47:0/2873894962 learned_addr learned my addr 172.21.15.47:0/2873894962 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.590+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2873894962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec1b1a20 con 0x55e5e50900a0 2024-03-21T11:47:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.590+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2873894962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec1b13c0 con 0x55e5e5095c00 2024-03-21T11:47:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.590+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2873894962 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec1ae860 con 0x55e5e5090a90 2024-03-21T11:47:24.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.591+0000 7f43f4dbf740 10 client.0 _ll_register_callbacks cb 0x7f43e40095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.591+0000 7f43eb400640 1 --2- 172.21.15.47:0/2873894962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e50900a0 0x55e5e50a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.591+0000 7f43ebe00640 1 -- 172.21.15.47:0/2873894962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e5090a90 msgr2=0x55e5e509feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.591+0000 7f43ebe00640 1 --2- 172.21.15.47:0/2873894962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e5090a90 0x55e5e509feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.591+0000 7f43ebe00640 1 -- 172.21.15.47:0/2873894962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e50900a0 msgr2=0x55e5e50a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.591+0000 7f43ebe00640 1 --2- 172.21.15.47:0/2873894962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e50900a0 0x55e5e50a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.592+0000 7f43ebe00640 1 -- 172.21.15.47:0/2873894962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5e50a0cc0 con 0x55e5e5095c00 2024-03-21T11:47:24.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.592+0000 7f43eb400640 1 --2- 172.21.15.47:0/2873894962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e50900a0 0x55e5e50a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:24.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.592+0000 7f43ebe00640 1 --2- 172.21.15.47:0/2873894962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5095c00 0x55e5e50a04c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f43e40bae80 tx=0x7f43e4035ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b00ecb66765f00e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:24.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.592+0000 7f43d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:24.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.594+0000 7f43d4c00640 1 -- 172.21.15.47:0/2873894962 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e40db060 con 0x55e5e5095c00 2024-03-21T11:47:24.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.594+0000 7f43d4c00640 1 -- 172.21.15.47:0/2873894962 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f43e4004d50 con 0x55e5e5095c00 2024-03-21T11:47:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.594+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2873894962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43dc011cc0 con 0x55e5e5095c00 2024-03-21T11:47:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.594+0000 7f43d4c00640 1 -- 172.21.15.47:0/2873894962 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43e40bde10 con 0x55e5e5095c00 2024-03-21T11:47:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.594+0000 7f43f4dbf740 10 client.33167 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.594+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2873894962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e5e5102c80 con 0x55e5e5095c00 2024-03-21T11:47:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.595+0000 7f43d4c00640 1 -- 172.21.15.47:0/2873894962 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43dc011cc0 con 0x55e5e5095c00 2024-03-21T11:47:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.597+0000 7f43d4c00640 1 -- 172.21.15.47:0/2873894962 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f43e4089320 con 0x55e5e5095c00 2024-03-21T11:47:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.597+0000 7f43f4dbf740 10 client.33167 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.597+0000 7f43f4dbf740 10 client.33167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.597+0000 7f43f4dbf740 20 client.33167 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.597+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2873894962 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e5e5103450 0x55e5e5105870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.597+0000 7f43f4dbf740 4 client.33167 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:47:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.597+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2873894962 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e5e4ffd740 con 0x55e5e5103450 2024-03-21T11:47:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.598+0000 7f43eb400640 1 --2- 172.21.15.47:0/2873894962 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e5e5103450 0x55e5e5105870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.600+0000 7f43eb400640 1 --2- 172.21.15.47:0/2873894962 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e5e5103450 0x55e5e5105870 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6ed4acd4de29753 server_cookie=ada2f73c95a22c49 in_seq=0 out_seq=0 2024-03-21T11:47:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.600+0000 7f43d4c00640 10 client.33167 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:24.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.602+0000 7f43d4c00640 1 -- 172.21.15.47:0/2873894962 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e5e4ffd740 con 0x55e5e5103450 2024-03-21T11:47:24.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.602+0000 7f43d4c00640 10 client.33167 handle_command_reply: tid=0 2024-03-21T11:47:24.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.602+0000 7f43f4dbf740 1 client.33167 shutdown 2024-03-21T11:47:24.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.603+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2873894962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5095c00 msgr2=0x55e5e50a04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:24.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.603+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2873894962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5095c00 0x55e5e50a04c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f43e40bae80 tx=0x7f43e4035ef0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.604+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2873894962 shutdown_connections 2024-03-21T11:47:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.604+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2873894962 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e5e5103450 0x55e5e5105870 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.604+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2873894962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5e5095c00 0x55e5e50a04c0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.604+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2873894962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5e5090a90 0x55e5e509feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.604+0000 7f43f4dbf740 1 --2- 172.21.15.47:0/2873894962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5e50900a0 0x55e5e50a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.604+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2873894962 >> 172.21.15.47:0/2873894962 conn(0x55e5e508b610 msgr2=0x55e5e5091530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.604+0000 7f43d4c00640 0 client.33167 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.604+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2873894962 shutdown_connections 2024-03-21T11:47:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.604+0000 7f43f4dbf740 1 -- 172.21.15.47:0/2873894962 wait complete. 2024-03-21T11:47:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.605+0000 7f43f4dbf740 20 client.33167 trim_cache size 0 max 16384 2024-03-21T11:47:24.608 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:47:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:47:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "4a62cd0f-6b23-482c-b58b-c4f861eb25fe", 2024-03-21T11:47:24.608 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:47:24.608 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:47:24.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.607+0000 7f43f2000640 1 -- 172.21.15.47:0/876693862 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f43d00a2fa0 msgr2=0x7f43d00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:24.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.607+0000 7f43f2000640 1 --2- 172.21.15.47:0/876693862 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f43d00a2fa0 0x7f43d00a5460 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x55e5e50912f0 tx=0x7f43d800a250 comp rx=0 tx=0).stop 2024-03-21T11:47:24.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.607+0000 7f43f2000640 1 -- 172.21.15.47:0/876693862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ec103260 msgr2=0x7f43ec1a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:24.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.607+0000 7f43f2000640 1 --2- 172.21.15.47:0/876693862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ec103260 0x7f43ec1a5580 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009bf0 tx=0x7f43e4035d80 comp rx=0 tx=0).stop 2024-03-21T11:47:24.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.608+0000 7f43f2000640 1 -- 172.21.15.47:0/876693862 shutdown_connections 2024-03-21T11:47:24.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.608+0000 7f43f2000640 1 --2- 172.21.15.47:0/876693862 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f43d00a2fa0 0x7f43d00a5460 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.608+0000 7f43f2000640 1 --2- 172.21.15.47:0/876693862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43ec103260 0x7f43ec1a5580 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.608+0000 7f43f2000640 1 --2- 172.21.15.47:0/876693862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43ec1028a0 0x7f43ec19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.608+0000 7f43f2000640 1 --2- 172.21.15.47:0/876693862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43ec1016b0 0x7f43ec19dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.608+0000 7f43f2000640 1 -- 172.21.15.47:0/876693862 >> 172.21.15.47:0/876693862 conn(0x7f43ec0fcc40 msgr2=0x7f43ec103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:24.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.609+0000 7f43f2000640 1 -- 172.21.15.47:0/876693862 shutdown_connections 2024-03-21T11:47:24.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.609+0000 7f43f2000640 1 -- 172.21.15.47:0/876693862 wait complete. 2024-03-21T11:47:24.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:47:24.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:47:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.954+0000 7f409c600640 1 -- 172.21.15.47:0/832990490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4094105490 msgr2=0x7f409410bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.954+0000 7f409c600640 1 --2- 172.21.15.47:0/832990490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4094105490 0x7f409410bd20 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f408c009900 tx=0x7f408c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:47:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.955+0000 7f409c600640 1 -- 172.21.15.47:0/832990490 shutdown_connections 2024-03-21T11:47:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.955+0000 7f409c600640 1 --2- 172.21.15.47:0/832990490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4094105490 0x7f409410bd20 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.955+0000 7f409c600640 1 --2- 172.21.15.47:0/832990490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4094104ad0 0x7f4094104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.955+0000 7f409c600640 1 --2- 172.21.15.47:0/832990490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40941038e0 0x7f4094103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.955+0000 7f409c600640 1 -- 172.21.15.47:0/832990490 >> 172.21.15.47:0/832990490 conn(0x7f40940fee50 msgr2=0x7f4094101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.955+0000 7f409c600640 1 -- 172.21.15.47:0/832990490 shutdown_connections 2024-03-21T11:47:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.956+0000 7f409c600640 1 -- 172.21.15.47:0/832990490 wait complete. 2024-03-21T11:47:24.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.957+0000 7f409c600640 1 Processor -- start 2024-03-21T11:47:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.957+0000 7f409c600640 1 -- start start 2024-03-21T11:47:24.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.957+0000 7f409c600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40941038e0 0x7f4094079450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.958+0000 7f409c600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4094104ad0 0x7f4094079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.958+0000 7f409c600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4094105490 0x7f4094075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.958+0000 7f409c600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409410f620 con 0x7f4094104ad0 2024-03-21T11:47:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.958+0000 7f409c600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409410f4a0 con 0x7f40941038e0 2024-03-21T11:47:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.958+0000 7f409c600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409410f7a0 con 0x7f4094105490 2024-03-21T11:47:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.958+0000 7f4099c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40941038e0 0x7f4094079450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.958+0000 7f4099c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40941038e0 0x7f4094079450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59872/0 (socket says 172.21.15.47:59872) 2024-03-21T11:47:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.958+0000 7f4099c00640 1 -- 172.21.15.47:0/2154729471 learned_addr learned my addr 172.21.15.47:0/2154729471 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.958+0000 7f4099200640 1 --2- 172.21.15.47:0/2154729471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4094104ad0 0x7f4094079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.958+0000 7f4099c00640 1 -- 172.21.15.47:0/2154729471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4094105490 msgr2=0x7f4094075fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:47:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.958+0000 7f4099c00640 1 --2- 172.21.15.47:0/2154729471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4094105490 0x7f4094075fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.958+0000 7f4099c00640 1 -- 172.21.15.47:0/2154729471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4094104ad0 msgr2=0x7f4094079990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.958+0000 7f4099c00640 1 --2- 172.21.15.47:0/2154729471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4094104ad0 0x7f4094079990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.958+0000 7f4099c00640 1 -- 172.21.15.47:0/2154729471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4094076880 con 0x7f40941038e0 2024-03-21T11:47:24.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.959+0000 7f4099c00640 1 --2- 172.21.15.47:0/2154729471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40941038e0 0x7f4094079450 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f408800b9c0 tx=0x7f408800be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.959+0000 7f4092000640 1 -- 172.21.15.47:0/2154729471 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4088019070 con 0x7f40941038e0 2024-03-21T11:47:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.959+0000 7f4092000640 1 -- 172.21.15.47:0/2154729471 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f408800cba0 con 0x7f40941038e0 2024-03-21T11:47:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.959+0000 7f4092000640 1 -- 172.21.15.47:0/2154729471 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4088012640 con 0x7f40941038e0 2024-03-21T11:47:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.959+0000 7f409c600640 1 -- 172.21.15.47:0/2154729471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40941a5ff0 con 0x7f40941038e0 2024-03-21T11:47:24.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.960+0000 7f409c600640 1 -- 172.21.15.47:0/2154729471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40941a63a0 con 0x7f40941038e0 2024-03-21T11:47:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.966+0000 7f4092000640 1 -- 172.21.15.47:0/2154729471 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f408801c070 con 0x7f40941038e0 2024-03-21T11:47:24.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.967+0000 7f409f3cc740 1 Processor -- start 2024-03-21T11:47:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.968+0000 7f409f3cc740 1 -- start start 2024-03-21T11:47:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.968+0000 7f409f3cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475afe80 0x55c5475b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.968+0000 7f409f3cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b0870 0x55c5475b5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.969+0000 7f409f3cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b59e0 0x55c5475b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.969+0000 7f409f3cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c547515250 con 0x55c5475b0870 2024-03-21T11:47:24.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.969+0000 7f409f3cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5474bda30 con 0x55c5475afe80 2024-03-21T11:47:24.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.969+0000 7f409f3cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5475112f0 con 0x55c5475b59e0 2024-03-21T11:47:24.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.969+0000 7f4092000640 1 --2- 172.21.15.47:0/2154729471 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f407c0a3020 0x7f407c0a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.969+0000 7f4092000640 1 -- 172.21.15.47:0/2154729471 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f40880ced60 con 0x7f40941038e0 2024-03-21T11:47:24.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.969+0000 7f4099c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b0870 0x55c5475b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.970+0000 7f409a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475afe80 0x55c5475b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.970+0000 7f409a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475afe80 0x55c5475b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59876/0 (socket says 172.21.15.47:59876) 2024-03-21T11:47:24.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.970+0000 7f409a600640 1 -- 172.21.15.47:0/1513421498 learned_addr learned my addr 172.21.15.47:0/1513421498 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:24.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.970+0000 7f409a600640 1 --2- 172.21.15.47:0/2154729471 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f407c0a3020 0x7f407c0a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.970+0000 7f4099200640 1 --2- 172.21.15.47:0/1513421498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b59e0 0x55c5475b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.971+0000 7f409a600640 1 -- 172.21.15.47:0/1513421498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b59e0 msgr2=0x55c5475b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.971+0000 7f409a600640 1 --2- 172.21.15.47:0/1513421498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b59e0 0x55c5475b7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.971+0000 7f409a600640 1 -- 172.21.15.47:0/1513421498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b0870 msgr2=0x55c5475b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.971+0000 7f409a600640 1 --2- 172.21.15.47:0/1513421498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b0870 0x55c5475b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.971+0000 7f409a600640 1 -- 172.21.15.47:0/1513421498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5474a8690 con 0x55c5475afe80 2024-03-21T11:47:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.971+0000 7f4099c00640 1 --2- 172.21.15.47:0/1513421498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b0870 0x55c5475b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:47:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.971+0000 7f409a600640 1 --2- 172.21.15.47:0/1513421498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475afe80 0x55c5475b02a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f408c005ab0 tx=0x7f408c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c944232a0be2b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.972+0000 7f4086a00640 1 -- 172.21.15.47:0/1513421498 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c045070 con 0x55c5475afe80 2024-03-21T11:47:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.972+0000 7f4086a00640 1 -- 172.21.15.47:0/1513421498 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f408c0046c0 con 0x55c5475afe80 2024-03-21T11:47:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.972+0000 7f4086a00640 1 -- 172.21.15.47:0/1513421498 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c041760 con 0x55c5475afe80 2024-03-21T11:47:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.974+0000 7f409a600640 1 --2- 172.21.15.47:0/2154729471 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f407c0a3020 0x7f407c0a54e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f408c02c8e0 tx=0x7f408c0419e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.974+0000 7f409f3cc740 1 -- 172.21.15.47:0/1513421498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475afe80 msgr2=0x55c5475b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.974+0000 7f409f3cc740 1 --2- 172.21.15.47:0/1513421498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475afe80 0x55c5475b02a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f408c005ab0 tx=0x7f408c005e80 comp rx=0 tx=0).stop 2024-03-21T11:47:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.975+0000 7f409f3cc740 1 -- 172.21.15.47:0/1513421498 shutdown_connections 2024-03-21T11:47:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.975+0000 7f409f3cc740 1 --2- 172.21.15.47:0/1513421498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b59e0 0x55c5475b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.975+0000 7f409f3cc740 1 --2- 172.21.15.47:0/1513421498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b0870 0x55c5475b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.975+0000 7f409f3cc740 1 --2- 172.21.15.47:0/1513421498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475afe80 0x55c5475b02a0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.975+0000 7f409f3cc740 1 -- 172.21.15.47:0/1513421498 >> 172.21.15.47:0/1513421498 conn(0x55c547578200 msgr2=0x55c5475ab7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.976+0000 7f409f3cc740 1 -- 172.21.15.47:0/1513421498 shutdown_connections 2024-03-21T11:47:24.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.976+0000 7f409f3cc740 1 -- 172.21.15.47:0/1513421498 wait complete. 2024-03-21T11:47:24.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.977+0000 7f409f3cc740 1 -- start start 2024-03-21T11:47:24.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.977+0000 7f409f3cc740 1 Processor -- start 2024-03-21T11:47:24.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.979+0000 7f409f3cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475afe80 0x55c5475b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.979+0000 7f409f3cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b0870 0x55c5475ba070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.979+0000 7f409f3cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b59e0 0x55c5475c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.980+0000 7f4099200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b0870 0x55c5475ba070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.980+0000 7f4099200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b0870 0x55c5475ba070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48252/0 (socket says 172.21.15.47:48252) 2024-03-21T11:47:24.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.983+0000 7f4099c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475afe80 0x55c5475b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.983+0000 7f409f3cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5475b8990 con 0x55c5475b59e0 2024-03-21T11:47:24.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.983+0000 7f409f3cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5474d0e80 con 0x55c5475afe80 2024-03-21T11:47:24.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.983+0000 7f409f3cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5475b8b10 con 0x55c5475b0870 2024-03-21T11:47:24.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.983+0000 7f4099c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475afe80 0x55c5475b9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59884/0 (socket says 172.21.15.47:59884) 2024-03-21T11:47:24.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.983+0000 7f4099c00640 1 -- 172.21.15.47:0/865497830 learned_addr learned my addr 172.21.15.47:0/865497830 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:24.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.983+0000 7f409f3cc740 10 client.0 _ll_register_callbacks cb 0x55c5474e3130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:24.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.984+0000 7f4099c00640 1 -- 172.21.15.47:0/865497830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b0870 msgr2=0x55c5475ba070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:24.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.984+0000 7f4099c00640 1 --2- 172.21.15.47:0/865497830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b0870 0x55c5475ba070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.984+0000 7f4099c00640 1 -- 172.21.15.47:0/865497830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b59e0 msgr2=0x55c5475c10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:24.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.984+0000 7f4099c00640 1 --2- 172.21.15.47:0/865497830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b59e0 0x55c5475c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.984+0000 7f4099c00640 1 -- 172.21.15.47:0/865497830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5472f37d0 con 0x55c5475afe80 2024-03-21T11:47:24.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.984+0000 7f4099c00640 1 --2- 172.21.15.47:0/865497830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475afe80 0x55c5475b9b30 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f40880a3580 tx=0x7f408809ad10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=348debe950bf6734 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:24.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.984+0000 7f407b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:24.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.984+0000 7f4099200640 1 --2- 172.21.15.47:0/865497830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b0870 0x55c5475ba070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-21T11:47:24.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.985+0000 7f407b400640 1 -- 172.21.15.47:0/865497830 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408809b0a0 con 0x55c5475afe80 2024-03-21T11:47:24.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.985+0000 7f407b400640 1 -- 172.21.15.47:0/865497830 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f40880bb070 con 0x55c5475afe80 2024-03-21T11:47:24.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.985+0000 7f409f3cc740 1 -- 172.21.15.47:0/865497830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f408c0046c0 con 0x55c5475afe80 2024-03-21T11:47:24.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.985+0000 7f407b400640 1 -- 172.21.15.47:0/865497830 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f408809f1f0 con 0x55c5475afe80 2024-03-21T11:47:24.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.985+0000 7f409f3cc740 10 client.33179 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:24.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.985+0000 7f409f3cc740 1 -- 172.21.15.47:0/865497830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c5472293f0 con 0x55c5475afe80 2024-03-21T11:47:24.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.986+0000 7f407b400640 1 -- 172.21.15.47:0/865497830 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f408c0046c0 con 0x55c5475afe80 2024-03-21T11:47:24.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.987+0000 7f407b400640 1 -- 172.21.15.47:0/865497830 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f40880a3620 con 0x55c5475afe80 2024-03-21T11:47:24.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.987+0000 7f409f3cc740 10 client.33179 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:24.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.987+0000 7f409f3cc740 10 client.33179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:24.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.988+0000 7f409f3cc740 20 client.33179 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:24.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.988+0000 7f409f3cc740 1 --2- 172.21.15.47:0/865497830 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5475c5880 0x55c5475c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:24.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.988+0000 7f409f3cc740 4 client.33179 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:24.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.988+0000 7f409f3cc740 1 -- 172.21.15.47:0/865497830 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c547522790 con 0x55c5475c5880 2024-03-21T11:47:24.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.992+0000 7f4099200640 1 --2- 172.21.15.47:0/865497830 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5475c5880 0x55c5475c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:24.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.993+0000 7f4099200640 1 --2- 172.21.15.47:0/865497830 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5475c5880 0x55c5475c7d00 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a500f99335bfa6 server_cookie=df91ce8dbcdbd4dd in_seq=0 out_seq=0 2024-03-21T11:47:24.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.993+0000 7f407b400640 10 client.33179 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:24.994 INFO:tasks.workunit.client.0.smithi047.stdout: 10 254903 21.70 MB/sec execute 488 sec latency 167.965 ms 2024-03-21T11:47:24.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.995+0000 7f407b400640 1 -- 172.21.15.47:0/865497830 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c547522790 con 0x55c5475c5880 2024-03-21T11:47:24.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.995+0000 7f407b400640 10 client.33179 handle_command_reply: tid=0 2024-03-21T11:47:24.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.995+0000 7f409f3cc740 1 client.33179 shutdown 2024-03-21T11:47:24.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.996+0000 7f409f3cc740 1 -- 172.21.15.47:0/865497830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475afe80 msgr2=0x55c5475b9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:24.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.996+0000 7f409f3cc740 1 --2- 172.21.15.47:0/865497830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475afe80 0x55c5475b9b30 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f40880a3580 tx=0x7f408809ad10 comp rx=0 tx=0).stop 2024-03-21T11:47:24.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.996+0000 7f409f3cc740 1 -- 172.21.15.47:0/865497830 shutdown_connections 2024-03-21T11:47:24.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.996+0000 7f409f3cc740 1 --2- 172.21.15.47:0/865497830 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5475c5880 0x55c5475c7d00 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.996+0000 7f409f3cc740 1 --2- 172.21.15.47:0/865497830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b59e0 0x55c5475c10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.996+0000 7f407b400640 0 client.33179 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:24.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.996+0000 7f409f3cc740 1 --2- 172.21.15.47:0/865497830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b0870 0x55c5475ba070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.996+0000 7f409f3cc740 1 --2- 172.21.15.47:0/865497830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475afe80 0x55c5475b9b30 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:24.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.997+0000 7f409f3cc740 1 -- 172.21.15.47:0/865497830 >> 172.21.15.47:0/865497830 conn(0x55c547578200 msgr2=0x55c5475aaa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:24.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.997+0000 7f409f3cc740 1 -- 172.21.15.47:0/865497830 shutdown_connections 2024-03-21T11:47:24.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.997+0000 7f409f3cc740 1 -- 172.21.15.47:0/865497830 wait complete. 2024-03-21T11:47:24.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:24.997+0000 7f409f3cc740 20 client.33179 trim_cache size 0 max 16384 2024-03-21T11:47:25.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.036+0000 7f409f3cc740 1 Processor -- start 2024-03-21T11:47:25.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.036+0000 7f409f3cc740 1 -- start start 2024-03-21T11:47:25.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.036+0000 7f409f3cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475afe80 0x55c5475b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:25.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.037+0000 7f409f3cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b0870 0x55c5475c1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:25.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.037+0000 7f409f3cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475b59e0 0x55c5475b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:25.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.037+0000 7f409f3cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941a7e80 con 0x55c5475afe80 2024-03-21T11:47:25.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.037+0000 7f409f3cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941a9800 con 0x55c5475b59e0 2024-03-21T11:47:25.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.037+0000 7f409f3cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941a84e0 con 0x55c5475b0870 2024-03-21T11:47:25.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.037+0000 7f4099c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475afe80 0x55c5475b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:25.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.037+0000 7f4099c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475afe80 0x55c5475b9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37510/0 (socket says 172.21.15.47:37510) 2024-03-21T11:47:25.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.037+0000 7f4099c00640 1 -- 172.21.15.47:0/2883125731 learned_addr learned my addr 172.21.15.47:0/2883125731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:25.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.037+0000 7f4099200640 1 --2- 172.21.15.47:0/2883125731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b0870 0x55c5475c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:25.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.037+0000 7f4099c00640 1 -- 172.21.15.47:0/2883125731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b0870 msgr2=0x55c5475c1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:25.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.038+0000 7f409a600640 1 --2- 172.21.15.47:0/2883125731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475b59e0 0x55c5475b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:25.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.038+0000 7f4099c00640 1 --2- 172.21.15.47:0/2883125731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b0870 0x55c5475c1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.038+0000 7f4099c00640 1 -- 172.21.15.47:0/2883125731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475b59e0 msgr2=0x55c5475b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:25.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.038+0000 7f4099c00640 1 --2- 172.21.15.47:0/2883125731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475b59e0 0x55c5475b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.038+0000 7f4099c00640 1 -- 172.21.15.47:0/2883125731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5471e90f0 con 0x55c5475afe80 2024-03-21T11:47:25.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.038+0000 7f4099200640 1 --2- 172.21.15.47:0/2883125731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b0870 0x55c5475c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:25.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.038+0000 7f409a600640 1 --2- 172.21.15.47:0/2883125731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475b59e0 0x55c5475b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:25.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.038+0000 7f4099c00640 1 --2- 172.21.15.47:0/2883125731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475afe80 0x55c5475b9e50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4088099350 tx=0x7f408809ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c6dc435035af1e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:25.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.038+0000 7f4086000640 1 -- 172.21.15.47:0/2883125731 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408809a6c0 con 0x55c5475afe80 2024-03-21T11:47:25.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.038+0000 7f4086000640 1 -- 172.21.15.47:0/2883125731 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f408809b0a0 con 0x55c5475afe80 2024-03-21T11:47:25.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.038+0000 7f4086000640 1 -- 172.21.15.47:0/2883125731 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40880a3620 con 0x55c5475afe80 2024-03-21T11:47:25.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.039+0000 7f409f3cc740 1 -- 172.21.15.47:0/2883125731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475afe80 msgr2=0x55c5475b9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:25.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.039+0000 7f409f3cc740 1 --2- 172.21.15.47:0/2883125731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475afe80 0x55c5475b9e50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4088099350 tx=0x7f408809ab30 comp rx=0 tx=0).stop 2024-03-21T11:47:25.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.039+0000 7f409f3cc740 1 -- 172.21.15.47:0/2883125731 shutdown_connections 2024-03-21T11:47:25.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.039+0000 7f409f3cc740 1 --2- 172.21.15.47:0/2883125731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475b59e0 0x55c5475b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.039+0000 7f409f3cc740 1 --2- 172.21.15.47:0/2883125731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475b0870 0x55c5475c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.039+0000 7f409f3cc740 1 --2- 172.21.15.47:0/2883125731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475afe80 0x55c5475b9e50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.039+0000 7f409f3cc740 1 -- 172.21.15.47:0/2883125731 >> 172.21.15.47:0/2883125731 conn(0x55c547578200 msgr2=0x55c5475b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:25.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.040+0000 7f409f3cc740 1 -- 172.21.15.47:0/2883125731 shutdown_connections 2024-03-21T11:47:25.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.040+0000 7f409f3cc740 1 -- 172.21.15.47:0/2883125731 wait complete. 2024-03-21T11:47:25.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.040+0000 7f409f3cc740 1 -- start start 2024-03-21T11:47:25.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.040+0000 7f409f3cc740 1 Processor -- start 2024-03-21T11:47:25.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.041+0000 7f409f3cc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475afe80 0x55c5475be990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.041+0000 7f409f3cc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b0870 0x55c5475beed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.041+0000 7f409f3cc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475b59e0 0x55c5475bfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.041+0000 7f409f3cc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941a91a0 con 0x55c5475b0870 2024-03-21T11:47:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.041+0000 7f409f3cc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941a8b40 con 0x55c5475b59e0 2024-03-21T11:47:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.041+0000 7f409f3cc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4094073270 con 0x55c5475afe80 2024-03-21T11:47:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.041+0000 7f4099c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475afe80 0x55c5475be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.041+0000 7f409f3cc740 10 client.0 _ll_register_callbacks cb 0x7f40880973c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.041+0000 7f4099c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475afe80 0x55c5475be990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48264/0 (socket says 172.21.15.47:48264) 2024-03-21T11:47:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.041+0000 7f4099c00640 1 -- 172.21.15.47:0/469244991 learned_addr learned my addr 172.21.15.47:0/469244991 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.041+0000 7f409a600640 1 --2- 172.21.15.47:0/469244991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475b59e0 0x55c5475bfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.042+0000 7f4099c00640 1 -- 172.21.15.47:0/469244991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475b59e0 msgr2=0x55c5475bfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.042+0000 7f4099c00640 1 --2- 172.21.15.47:0/469244991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475b59e0 0x55c5475bfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.042+0000 7f4099c00640 1 -- 172.21.15.47:0/469244991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b0870 msgr2=0x55c5475beed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:47:25.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.042+0000 7f4099200640 1 --2- 172.21.15.47:0/469244991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b0870 0x55c5475beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:25.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.042+0000 7f4099c00640 1 --2- 172.21.15.47:0/469244991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b0870 0x55c5475beed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.042+0000 7f4099c00640 1 -- 172.21.15.47:0/469244991 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5475c04d0 con 0x55c5475afe80 2024-03-21T11:47:25.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.042+0000 7f4099200640 1 --2- 172.21.15.47:0/469244991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b0870 0x55c5475beed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:25.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.043+0000 7f4099c00640 1 --2- 172.21.15.47:0/469244991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475afe80 0x55c5475be990 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f40880bb010 tx=0x7f408809ace0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0db01e4d149cfdc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:25.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.043+0000 7f407b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:47:25.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.043+0000 7f407b400640 1 -- 172.21.15.47:0/469244991 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40880a3ab0 con 0x55c5475afe80 2024-03-21T11:47:25.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.043+0000 7f407b400640 1 -- 172.21.15.47:0/469244991 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f40880a3620 con 0x55c5475afe80 2024-03-21T11:47:25.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.044+0000 7f409f3cc740 1 -- 172.21.15.47:0/469244991 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f408809b0a0 con 0x55c5475afe80 2024-03-21T11:47:25.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.045+0000 7f407b400640 1 -- 172.21.15.47:0/469244991 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40880a3820 con 0x55c5475afe80 2024-03-21T11:47:25.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.045+0000 7f409f3cc740 10 client.33133 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:25.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.045+0000 7f409f3cc740 1 -- 172.21.15.47:0/469244991 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c547621340 con 0x55c5475afe80 2024-03-21T11:47:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.045+0000 7f407b400640 1 -- 172.21.15.47:0/469244991 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f408809b0a0 con 0x55c5475afe80 2024-03-21T11:47:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.045+0000 7f407b400640 1 -- 172.21.15.47:0/469244991 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f40880a0380 con 0x55c5475afe80 2024-03-21T11:47:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.045+0000 7f409f3cc740 10 client.33133 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.045+0000 7f409f3cc740 10 client.33133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.045+0000 7f409f3cc740 20 client.33133 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.046+0000 7f409f3cc740 1 --2- 172.21.15.47:0/469244991 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5475c5880 0x55c547623ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.046+0000 7f409f3cc740 4 client.33133 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:47:25.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.046+0000 7f409f3cc740 1 -- 172.21.15.47:0/469244991 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c54751cae0 con 0x55c5475c5880 2024-03-21T11:47:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.046+0000 7f4099200640 1 --2- 172.21.15.47:0/469244991 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5475c5880 0x55c547623ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.047+0000 7f4099200640 1 --2- 172.21.15.47:0/469244991 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5475c5880 0x55c547623ac0 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ecd9b2f1dd99213 server_cookie=ff547bb209bbf13 in_seq=0 out_seq=0 2024-03-21T11:47:25.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.047+0000 7f407b400640 10 client.33133 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:25.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.048+0000 7f407b400640 1 -- 172.21.15.47:0/469244991 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55c54751cae0 con 0x55c5475c5880 2024-03-21T11:47:25.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.048+0000 7f407b400640 10 client.33133 handle_command_reply: tid=0 2024-03-21T11:47:25.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.048+0000 7f409f3cc740 1 client.33133 shutdown 2024-03-21T11:47:25.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.048+0000 7f409f3cc740 1 -- 172.21.15.47:0/469244991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475afe80 msgr2=0x55c5475be990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:25.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.048+0000 7f409f3cc740 1 --2- 172.21.15.47:0/469244991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475afe80 0x55c5475be990 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f40880bb010 tx=0x7f408809ace0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.048+0000 7f409f3cc740 1 -- 172.21.15.47:0/469244991 shutdown_connections 2024-03-21T11:47:25.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.048+0000 7f409f3cc740 1 --2- 172.21.15.47:0/469244991 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5475c5880 0x55c547623ac0 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.048+0000 7f409f3cc740 1 --2- 172.21.15.47:0/469244991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5475b59e0 0x55c5475bfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.048+0000 7f409f3cc740 1 --2- 172.21.15.47:0/469244991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5475b0870 0x55c5475beed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.048+0000 7f409f3cc740 1 --2- 172.21.15.47:0/469244991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5475afe80 0x55c5475be990 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.048+0000 7f409f3cc740 1 -- 172.21.15.47:0/469244991 >> 172.21.15.47:0/469244991 conn(0x55c547578200 msgr2=0x55c5475b96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:25.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.048+0000 7f407b400640 0 client.33133 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:25.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.049+0000 7f409f3cc740 1 -- 172.21.15.47:0/469244991 shutdown_connections 2024-03-21T11:47:25.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.049+0000 7f409f3cc740 1 -- 172.21.15.47:0/469244991 wait complete. 2024-03-21T11:47:25.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.049+0000 7f409f3cc740 20 client.33133 trim_cache size 0 max 16384 2024-03-21T11:47:25.052 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:47:25.052 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (286 inodes in the stack)", 2024-03-21T11:47:25.052 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:47:25.052 INFO:teuthology.orchestra.run.smithi047.stdout: "4a62cd0f-6b23-482c-b58b-c4f861eb25fe": { 2024-03-21T11:47:25.052 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:47:25.052 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "4a62cd0f-6b23-482c-b58b-c4f861eb25fe", 2024-03-21T11:47:25.052 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:47:25.052 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:47:25.052 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:47:25.052 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:47:25.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.051+0000 7f409c600640 1 -- 172.21.15.47:0/2154729471 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f407c0a3020 msgr2=0x7f407c0a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:25.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.051+0000 7f409c600640 1 --2- 172.21.15.47:0/2154729471 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f407c0a3020 0x7f407c0a54e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f408c02c8e0 tx=0x7f408c0419e0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.051+0000 7f409c600640 1 -- 172.21.15.47:0/2154729471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40941038e0 msgr2=0x7f4094079450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:25.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.051+0000 7f409c600640 1 --2- 172.21.15.47:0/2154729471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40941038e0 0x7f4094079450 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f408800b9c0 tx=0x7f408800be80 comp rx=0 tx=0).stop 2024-03-21T11:47:25.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.051+0000 7f409c600640 1 -- 172.21.15.47:0/2154729471 shutdown_connections 2024-03-21T11:47:25.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.051+0000 7f409c600640 1 --2- 172.21.15.47:0/2154729471 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f407c0a3020 0x7f407c0a54e0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.051+0000 7f409c600640 1 --2- 172.21.15.47:0/2154729471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4094105490 0x7f4094075fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.051+0000 7f409c600640 1 --2- 172.21.15.47:0/2154729471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4094104ad0 0x7f4094079990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.051+0000 7f409c600640 1 --2- 172.21.15.47:0/2154729471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40941038e0 0x7f4094079450 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:25.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.051+0000 7f409c600640 1 -- 172.21.15.47:0/2154729471 >> 172.21.15.47:0/2154729471 conn(0x7f40940fee50 msgr2=0x7f4094107d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:25.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.051+0000 7f409c600640 1 -- 172.21.15.47:0/2154729471 shutdown_connections 2024-03-21T11:47:25.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:25.051+0000 7f409c600640 1 -- 172.21.15.47:0/2154729471 wait complete. 2024-03-21T11:47:25.078 INFO:tasks.cephfs.filesystem:scrub status for tag:4a62cd0f-6b23-482c-b58b-c4f861eb25fe - {'path': '/', 'tag': '4a62cd0f-6b23-482c-b58b-c4f861eb25fe', 'options': 'recursive,force'} 2024-03-21T11:47:25.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:25 smithi047 ceph-mon[31351]: pgmap v1123: 129 pgs: 129 active+clean; 777 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 621 op/s 2024-03-21T11:47:25.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:25 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:47:25.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:25 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:25.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:25 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:47:25.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:25 smithi100 ceph-mon[36245]: pgmap v1123: 129 pgs: 129 active+clean; 777 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 621 op/s 2024-03-21T11:47:25.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:25 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:47:25.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:25 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:25.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:25 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:47:25.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:25 smithi102 ceph-mon[36748]: pgmap v1123: 129 pgs: 129 active+clean; 777 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 621 op/s 2024-03-21T11:47:25.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:25 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:47:25.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:25 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:25.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:25 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:47:25.994 INFO:tasks.workunit.client.0.smithi047.stdout: 10 255074 21.68 MB/sec execute 489 sec latency 342.338 ms 2024-03-21T11:47:26.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:47:26.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:47:26.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:47:26.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:47:26.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:47:26.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:47:26.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:47:26.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:26 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:47:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:47:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:47:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:47:26.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:47:26.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:47:26.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:47:26.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:26 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.678+0000 7fdc45c00640 1 -- 172.21.15.47:0/3199070906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc40101f70 msgr2=0x7fdc40104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.678+0000 7fdc45c00640 1 --2- 172.21.15.47:0/3199070906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc40101f70 0x7fdc40104360 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009900 tx=0x7fdc3002bc40 comp rx=0 tx=0).stop 2024-03-21T11:47:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.679+0000 7fdc45c00640 1 -- 172.21.15.47:0/3199070906 shutdown_connections 2024-03-21T11:47:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.679+0000 7fdc45c00640 1 --2- 172.21.15.47:0/3199070906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc401071d0 0x7fdc40109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.679+0000 7fdc45c00640 1 --2- 172.21.15.47:0/3199070906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc401048a0 0x7fdc40106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.679+0000 7fdc45c00640 1 --2- 172.21.15.47:0/3199070906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc40101f70 0x7fdc40104360 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.679+0000 7fdc45c00640 1 -- 172.21.15.47:0/3199070906 >> 172.21.15.47:0/3199070906 conn(0x7fdc400fb980 msgr2=0x7fdc400fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.681+0000 7fdc45c00640 1 -- 172.21.15.47:0/3199070906 shutdown_connections 2024-03-21T11:47:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.681+0000 7fdc45c00640 1 -- 172.21.15.47:0/3199070906 wait complete. 2024-03-21T11:47:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.682+0000 7fdc45c00640 1 Processor -- start 2024-03-21T11:47:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.682+0000 7fdc45c00640 1 -- start start 2024-03-21T11:47:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.683+0000 7fdc45c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc40101f70 0x7fdc4019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.684+0000 7fdc45c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc401048a0 0x7fdc4019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.684+0000 7fdc45c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc401071d0 0x7fdc401a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.684+0000 7fdc45c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4010afc0 con 0x7fdc401071d0 2024-03-21T11:47:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.684+0000 7fdc45c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4010ae40 con 0x7fdc40101f70 2024-03-21T11:47:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.684+0000 7fdc45c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4010b140 con 0x7fdc401048a0 2024-03-21T11:47:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.684+0000 7fdc3f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc401048a0 0x7fdc4019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.684+0000 7fdc3f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc401048a0 0x7fdc4019e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48284/0 (socket says 172.21.15.47:48284) 2024-03-21T11:47:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.684+0000 7fdc3f400640 1 -- 172.21.15.47:0/732782219 learned_addr learned my addr 172.21.15.47:0/732782219 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.684+0000 7fdc3fe00640 1 --2- 172.21.15.47:0/732782219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc40101f70 0x7fdc4019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.685+0000 7fdc3f400640 1 -- 172.21.15.47:0/732782219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc40101f70 msgr2=0x7fdc4019ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.685+0000 7fdc3f400640 1 --2- 172.21.15.47:0/732782219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc40101f70 0x7fdc4019ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.685+0000 7fdc3f400640 1 -- 172.21.15.47:0/732782219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc401071d0 msgr2=0x7fdc401a5330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:47:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.686+0000 7fdc3f400640 1 --2- 172.21.15.47:0/732782219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc401071d0 0x7fdc401a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.686+0000 7fdc3f400640 1 -- 172.21.15.47:0/732782219 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc401a59d0 con 0x7fdc401048a0 2024-03-21T11:47:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.686+0000 7fdc3f400640 1 --2- 172.21.15.47:0/732782219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc401048a0 0x7fdc4019e300 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00e9c0 tx=0x7fdc2c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.686+0000 7fdc3cc00640 1 -- 172.21.15.47:0/732782219 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c00cd80 con 0x7fdc401048a0 2024-03-21T11:47:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.687+0000 7fdc45c00640 1 -- 172.21.15.47:0/732782219 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc401a5cc0 con 0x7fdc401048a0 2024-03-21T11:47:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.687+0000 7fdc3cc00640 1 -- 172.21.15.47:0/732782219 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc2c005500 con 0x7fdc401048a0 2024-03-21T11:47:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.687+0000 7fdc3cc00640 1 -- 172.21.15.47:0/732782219 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c010690 con 0x7fdc401048a0 2024-03-21T11:47:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.688+0000 7fdc45c00640 1 -- 172.21.15.47:0/732782219 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc401a61d0 con 0x7fdc401048a0 2024-03-21T11:47:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.698+0000 7fdc3cc00640 1 -- 172.21.15.47:0/732782219 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdc2c0046c0 con 0x7fdc401048a0 2024-03-21T11:47:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.698+0000 7fdc4a019740 1 Processor -- start 2024-03-21T11:47:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.702+0000 7fdc4a019740 1 -- start start 2024-03-21T11:47:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.703+0000 7fdc4a019740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13bb3f0 0x55e3d13bb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.704+0000 7fdc4a019740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13bbde0 0x55e3d13c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.704+0000 7fdc4a019740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13c0f50 0x55e3d13c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.704+0000 7fdc4a019740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d137e3b0 con 0x55e3d13bb3f0 2024-03-21T11:47:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.704+0000 7fdc4a019740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d1320830 con 0x55e3d13bbde0 2024-03-21T11:47:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.704+0000 7fdc4a019740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d12c95d0 con 0x55e3d13c0f50 2024-03-21T11:47:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.704+0000 7fdc3cc00640 1 --2- 172.21.15.47:0/732782219 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc240a2f10 0x7fdc240a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.704+0000 7fdc3cc00640 1 -- 172.21.15.47:0/732782219 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdc2c0cfc50 con 0x7fdc401048a0 2024-03-21T11:47:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.705+0000 7fdc45200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13bb3f0 0x55e3d13bb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.705+0000 7fdc45200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13bb3f0 0x55e3d13bb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37562/0 (socket says 172.21.15.47:37562) 2024-03-21T11:47:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.705+0000 7fdc45200640 1 -- 172.21.15.47:0/2276970438 learned_addr learned my addr 172.21.15.47:0/2276970438 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.705+0000 7fdc3fe00640 1 --2- 172.21.15.47:0/2276970438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13bbde0 0x55e3d13c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.705+0000 7fdc3f400640 1 --2- 172.21.15.47:0/2276970438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13c0f50 0x55e3d13c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.706+0000 7fdc45200640 1 -- 172.21.15.47:0/2276970438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13c0f50 msgr2=0x55e3d13c33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.706+0000 7fdc45200640 1 --2- 172.21.15.47:0/2276970438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13c0f50 0x55e3d13c33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.706+0000 7fdc45200640 1 -- 172.21.15.47:0/2276970438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13bbde0 msgr2=0x55e3d13c08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.706+0000 7fdc45200640 1 --2- 172.21.15.47:0/2276970438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13bbde0 0x55e3d13c08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.706+0000 7fdc45200640 1 -- 172.21.15.47:0/2276970438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3d12e12c0 con 0x55e3d13bb3f0 2024-03-21T11:47:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.708+0000 7fdc45200640 1 --2- 172.21.15.47:0/2276970438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13bb3f0 0x55e3d13bb810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009eb0 tx=0x7fdc3800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c57a48d8ab1946 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:26.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.708+0000 7fdc45200640 1 --2- 172.21.15.47:0/732782219 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc240a2f10 0x7fdc240a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.709+0000 7fdc35600640 1 -- 172.21.15.47:0/2276970438 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3801f070 con 0x55e3d13bb3f0 2024-03-21T11:47:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.709+0000 7fdc35600640 1 -- 172.21.15.47:0/2276970438 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc38011c40 con 0x55e3d13bb3f0 2024-03-21T11:47:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.709+0000 7fdc35600640 1 -- 172.21.15.47:0/2276970438 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3800d3c0 con 0x55e3d13bb3f0 2024-03-21T11:47:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.711+0000 7fdc45200640 1 --2- 172.21.15.47:0/732782219 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc240a2f10 0x7fdc240a53d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdc38010fd0 tx=0x7fdc38010f20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.711+0000 7fdc4a019740 1 -- 172.21.15.47:0/2276970438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13bb3f0 msgr2=0x55e3d13bb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.711+0000 7fdc4a019740 1 --2- 172.21.15.47:0/2276970438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13bb3f0 0x55e3d13bb810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009eb0 tx=0x7fdc3800f730 comp rx=0 tx=0).stop 2024-03-21T11:47:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.713+0000 7fdc4a019740 1 -- 172.21.15.47:0/2276970438 shutdown_connections 2024-03-21T11:47:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.713+0000 7fdc4a019740 1 --2- 172.21.15.47:0/2276970438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13c0f50 0x55e3d13c33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.713+0000 7fdc4a019740 1 --2- 172.21.15.47:0/2276970438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13bbde0 0x55e3d13c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.713+0000 7fdc4a019740 1 --2- 172.21.15.47:0/2276970438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13bb3f0 0x55e3d13bb810 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.713+0000 7fdc4a019740 1 -- 172.21.15.47:0/2276970438 >> 172.21.15.47:0/2276970438 conn(0x55e3d13b6960 msgr2=0x55e3d13b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:26.715 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.714+0000 7fdc4a019740 1 -- 172.21.15.47:0/2276970438 shutdown_connections 2024-03-21T11:47:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.716+0000 7fdc4a019740 1 -- 172.21.15.47:0/2276970438 wait complete. 2024-03-21T11:47:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.716+0000 7fdc4a019740 1 -- start start 2024-03-21T11:47:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.716+0000 7fdc4a019740 1 Processor -- start 2024-03-21T11:47:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.717+0000 7fdc4a019740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13bb3f0 0x55e3d13c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.717+0000 7fdc4a019740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bbde0 0x55e3d13c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.717+0000 7fdc4a019740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13c0f50 0x55e3d13ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.717+0000 7fdc4a019740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d12dca20 con 0x55e3d13c0f50 2024-03-21T11:47:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.717+0000 7fdc4a019740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d131c8d0 con 0x55e3d13bb3f0 2024-03-21T11:47:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.717+0000 7fdc4a019740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d13c3f00 con 0x55e3d13bbde0 2024-03-21T11:47:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.717+0000 7fdc4a019740 10 client.0 _ll_register_callbacks cb 0x55e3d13191c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.718+0000 7fdc45200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13c0f50 0x55e3d13ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.718+0000 7fdc3f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bbde0 0x55e3d13c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.718+0000 7fdc3f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bbde0 0x55e3d13c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48308/0 (socket says 172.21.15.47:48308) 2024-03-21T11:47:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.718+0000 7fdc3f400640 1 -- 172.21.15.47:0/261772642 learned_addr learned my addr 172.21.15.47:0/261772642 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.718+0000 7fdc3fe00640 1 --2- 172.21.15.47:0/261772642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13bb3f0 0x55e3d13c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.719+0000 7fdc3f400640 1 -- 172.21.15.47:0/261772642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13bb3f0 msgr2=0x55e3d13c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.719+0000 7fdc3f400640 1 --2- 172.21.15.47:0/261772642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13bb3f0 0x55e3d13c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.719+0000 7fdc3f400640 1 -- 172.21.15.47:0/261772642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13c0f50 msgr2=0x55e3d13ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.719+0000 7fdc3f400640 1 --2- 172.21.15.47:0/261772642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13c0f50 0x55e3d13ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.719+0000 7fdc3f400640 1 -- 172.21.15.47:0/261772642 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3d1035210 con 0x55e3d13bbde0 2024-03-21T11:47:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.719+0000 7fdc3f400640 1 --2- 172.21.15.47:0/261772642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bbde0 0x55e3d13c5b10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00c6a0 tx=0x7fdc2c09bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0bef8d1308a9815 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.719+0000 7fdc2a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:47:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.719+0000 7fdc2a000640 1 -- 172.21.15.47:0/261772642 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c09bff0 con 0x55e3d13bbde0 2024-03-21T11:47:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.720+0000 7fdc2a000640 1 -- 172.21.15.47:0/261772642 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdc2c09b440 con 0x55e3d13bbde0 2024-03-21T11:47:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.720+0000 7fdc4a019740 1 -- 172.21.15.47:0/261772642 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc38011c40 con 0x55e3d13bbde0 2024-03-21T11:47:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.720+0000 7fdc2a000640 1 -- 172.21.15.47:0/261772642 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc2c0a0140 con 0x55e3d13bbde0 2024-03-21T11:47:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.720+0000 7fdc4a019740 10 client.33142 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.720+0000 7fdc4a019740 1 -- 172.21.15.47:0/261772642 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e3d1101620 con 0x55e3d13bbde0 2024-03-21T11:47:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.721+0000 7fdc2a000640 1 -- 172.21.15.47:0/261772642 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc38011c40 con 0x55e3d13bbde0 2024-03-21T11:47:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.721+0000 7fdc2a000640 1 -- 172.21.15.47:0/261772642 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdc2c09aeb0 con 0x55e3d13bbde0 2024-03-21T11:47:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.721+0000 7fdc4a019740 10 client.33142 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.721+0000 7fdc4a019740 10 client.33142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.721+0000 7fdc4a019740 20 client.33142 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.721+0000 7fdc4a019740 1 --2- 172.21.15.47:0/261772642 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e3d13d13c0 0x55e3d13d37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.721+0000 7fdc4a019740 4 client.33142 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:26.724 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.721+0000 7fdc4a019740 1 -- 172.21.15.47:0/261772642 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3d132dd70 con 0x55e3d13d13c0 2024-03-21T11:47:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:47:26.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:47:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:47:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:47:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:47:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:47:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:47:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:26 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:47:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.772+0000 7fdc3fe00640 1 --2- 172.21.15.47:0/261772642 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e3d13d13c0 0x55e3d13d37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.773+0000 7fdc3fe00640 1 --2- 172.21.15.47:0/261772642 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e3d13d13c0 0x55e3d13d37b0 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a08c246a1f32c0 server_cookie=363323f3d62efb6d in_seq=0 out_seq=0 2024-03-21T11:47:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.773+0000 7fdc2a000640 10 client.33142 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.779+0000 7fdc2a000640 1 -- 172.21.15.47:0/261772642 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e3d132dd70 con 0x55e3d13d13c0 2024-03-21T11:47:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.779+0000 7fdc2a000640 10 client.33142 handle_command_reply: tid=0 2024-03-21T11:47:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.779+0000 7fdc4a019740 1 client.33142 shutdown 2024-03-21T11:47:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.780+0000 7fdc4a019740 1 -- 172.21.15.47:0/261772642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bbde0 msgr2=0x55e3d13c5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.780+0000 7fdc4a019740 1 --2- 172.21.15.47:0/261772642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bbde0 0x55e3d13c5b10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00c6a0 tx=0x7fdc2c09bc20 comp rx=0 tx=0).stop 2024-03-21T11:47:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.780+0000 7fdc4a019740 1 -- 172.21.15.47:0/261772642 shutdown_connections 2024-03-21T11:47:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.780+0000 7fdc4a019740 1 --2- 172.21.15.47:0/261772642 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e3d13d13c0 0x55e3d13d37b0 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.780+0000 7fdc4a019740 1 --2- 172.21.15.47:0/261772642 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13c0f50 0x55e3d13ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.780+0000 7fdc4a019740 1 --2- 172.21.15.47:0/261772642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bbde0 0x55e3d13c5b10 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.780+0000 7fdc4a019740 1 --2- 172.21.15.47:0/261772642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13bb3f0 0x55e3d13c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.780+0000 7fdc4a019740 1 -- 172.21.15.47:0/261772642 >> 172.21.15.47:0/261772642 conn(0x55e3d13b6960 msgr2=0x55e3d13b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.780+0000 7fdc2a000640 0 client.33142 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.781+0000 7fdc4a019740 1 -- 172.21.15.47:0/261772642 shutdown_connections 2024-03-21T11:47:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.781+0000 7fdc4a019740 1 -- 172.21.15.47:0/261772642 wait complete. 2024-03-21T11:47:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.781+0000 7fdc4a019740 20 client.33142 trim_cache size 0 max 16384 2024-03-21T11:47:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.811+0000 7fdc4a019740 1 Processor -- start 2024-03-21T11:47:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.812+0000 7fdc4a019740 1 -- start start 2024-03-21T11:47:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.812+0000 7fdc4a019740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13bb3f0 0x55e3d13bd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.813+0000 7fdc4a019740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bbde0 0x55e3d13bd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.813+0000 7fdc4a019740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13c0f50 0x55e3d13ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.813+0000 7fdc4a019740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc401b0520 con 0x55e3d13bb3f0 2024-03-21T11:47:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.813+0000 7fdc4a019740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc401b1ea0 con 0x55e3d13c0f50 2024-03-21T11:47:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.813+0000 7fdc4a019740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc401b0b80 con 0x55e3d13bbde0 2024-03-21T11:47:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.813+0000 7fdc3fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13bb3f0 0x55e3d13bd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.813+0000 7fdc3f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bbde0 0x55e3d13bd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.814+0000 7fdc3f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bbde0 0x55e3d13bd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48316/0 (socket says 172.21.15.47:48316) 2024-03-21T11:47:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.814+0000 7fdc3f400640 1 -- 172.21.15.47:0/4271440429 learned_addr learned my addr 172.21.15.47:0/4271440429 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.814+0000 7fdc45200640 1 --2- 172.21.15.47:0/4271440429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13c0f50 0x55e3d13ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.814+0000 7fdc3f400640 1 -- 172.21.15.47:0/4271440429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13c0f50 msgr2=0x55e3d13ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.814+0000 7fdc3f400640 1 --2- 172.21.15.47:0/4271440429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13c0f50 0x55e3d13ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.814+0000 7fdc3f400640 1 -- 172.21.15.47:0/4271440429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13bb3f0 msgr2=0x55e3d13bd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.814+0000 7fdc3f400640 1 --2- 172.21.15.47:0/4271440429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13bb3f0 0x55e3d13bd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.814+0000 7fdc3f400640 1 -- 172.21.15.47:0/4271440429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3d13bb7d0 con 0x55e3d13bbde0 2024-03-21T11:47:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.814+0000 7fdc3f400640 1 --2- 172.21.15.47:0/4271440429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bbde0 0x55e3d13bd680 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c09bf80 tx=0x7fdc2c098200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab5d807261c47591 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.817+0000 7fdc34c00640 1 -- 172.21.15.47:0/4271440429 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c09bff0 con 0x55e3d13bbde0 2024-03-21T11:47:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.817+0000 7fdc34c00640 1 -- 172.21.15.47:0/4271440429 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc2c09b0b0 con 0x55e3d13bbde0 2024-03-21T11:47:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.817+0000 7fdc34c00640 1 -- 172.21.15.47:0/4271440429 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c09a130 con 0x55e3d13bbde0 2024-03-21T11:47:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.817+0000 7fdc4a019740 1 -- 172.21.15.47:0/4271440429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bbde0 msgr2=0x55e3d13bd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.817+0000 7fdc4a019740 1 --2- 172.21.15.47:0/4271440429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bbde0 0x55e3d13bd680 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c09bf80 tx=0x7fdc2c098200 comp rx=0 tx=0).stop 2024-03-21T11:47:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.818+0000 7fdc4a019740 1 -- 172.21.15.47:0/4271440429 shutdown_connections 2024-03-21T11:47:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.818+0000 7fdc4a019740 1 --2- 172.21.15.47:0/4271440429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13c0f50 0x55e3d13ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.818+0000 7fdc4a019740 1 --2- 172.21.15.47:0/4271440429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bbde0 0x55e3d13bd680 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.818+0000 7fdc4a019740 1 --2- 172.21.15.47:0/4271440429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13bb3f0 0x55e3d13bd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.818+0000 7fdc4a019740 1 -- 172.21.15.47:0/4271440429 >> 172.21.15.47:0/4271440429 conn(0x55e3d13b6960 msgr2=0x55e3d13b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.818+0000 7fdc4a019740 1 -- 172.21.15.47:0/4271440429 shutdown_connections 2024-03-21T11:47:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.818+0000 7fdc4a019740 1 -- 172.21.15.47:0/4271440429 wait complete. 2024-03-21T11:47:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.819+0000 7fdc4a019740 1 -- start start 2024-03-21T11:47:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.819+0000 7fdc4a019740 1 Processor -- start 2024-03-21T11:47:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.820+0000 7fdc4a019740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bb3f0 0x55e3d13ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.820+0000 7fdc4a019740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13bbde0 0x55e3d13caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.821+0000 7fdc4a019740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13c0f50 0x55e3d13cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.821+0000 7fdc4a019740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc401b1840 con 0x55e3d13c0f50 2024-03-21T11:47:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.821+0000 7fdc4a019740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc401b11e0 con 0x55e3d13bbde0 2024-03-21T11:47:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.821+0000 7fdc4a019740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc401ae6e0 con 0x55e3d13bb3f0 2024-03-21T11:47:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.821+0000 7fdc4a019740 10 client.0 _ll_register_callbacks cb 0x7fdc2c004a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.821+0000 7fdc3fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bb3f0 0x55e3d13ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.821+0000 7fdc3fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bb3f0 0x55e3d13ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48322/0 (socket says 172.21.15.47:48322) 2024-03-21T11:47:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.821+0000 7fdc3fe00640 1 -- 172.21.15.47:0/552668054 learned_addr learned my addr 172.21.15.47:0/552668054 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.822+0000 7fdc45200640 1 --2- 172.21.15.47:0/552668054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13c0f50 0x55e3d13cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.822+0000 7fdc3fe00640 1 -- 172.21.15.47:0/552668054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13bbde0 msgr2=0x55e3d13caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.822+0000 7fdc3fe00640 1 --2- 172.21.15.47:0/552668054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13bbde0 0x55e3d13caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.822+0000 7fdc3fe00640 1 -- 172.21.15.47:0/552668054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13c0f50 msgr2=0x55e3d13cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:26.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.822+0000 7fdc3fe00640 1 --2- 172.21.15.47:0/552668054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13c0f50 0x55e3d13cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:26.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.823+0000 7fdc3fe00640 1 -- 172.21.15.47:0/552668054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3d13cc4a0 con 0x55e3d13bb3f0 2024-03-21T11:47:26.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.823+0000 7fdc45200640 1 --2- 172.21.15.47:0/552668054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13c0f50 0x55e3d13cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:26.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.823+0000 7fdc3fe00640 1 --2- 172.21.15.47:0/552668054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bb3f0 0x55e3d13ca9b0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fdc30005ab0 tx=0x7fdc30000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c32d42b8945f26d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:26.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.823+0000 7fdc2a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:47:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.824+0000 7fdc2a000640 1 -- 172.21.15.47:0/552668054 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30004040 con 0x55e3d13bb3f0 2024-03-21T11:47:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.824+0000 7fdc4a019740 1 -- 172.21.15.47:0/552668054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc2c09b0b0 con 0x55e3d13bb3f0 2024-03-21T11:47:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.824+0000 7fdc2a000640 1 -- 172.21.15.47:0/552668054 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdc300041e0 con 0x55e3d13bb3f0 2024-03-21T11:47:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.824+0000 7fdc2a000640 1 -- 172.21.15.47:0/552668054 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc3005a050 con 0x55e3d13bb3f0 2024-03-21T11:47:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.824+0000 7fdc4a019740 10 client.33148 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.824+0000 7fdc4a019740 1 -- 172.21.15.47:0/552668054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e3d142d450 con 0x55e3d13bb3f0 2024-03-21T11:47:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.825+0000 7fdc2a000640 1 -- 172.21.15.47:0/552668054 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc2c09b0b0 con 0x55e3d13bb3f0 2024-03-21T11:47:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.825+0000 7fdc2a000640 1 -- 172.21.15.47:0/552668054 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdc3004a990 con 0x55e3d13bb3f0 2024-03-21T11:47:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.825+0000 7fdc4a019740 10 client.33148 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.825+0000 7fdc4a019740 10 client.33148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.825+0000 7fdc4a019740 20 client.33148 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.825+0000 7fdc4a019740 1 --2- 172.21.15.47:0/552668054 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e3d142dde0 0x55e3d1430200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.825+0000 7fdc4a019740 4 client.33148 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:47:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.825+0000 7fdc4a019740 1 -- 172.21.15.47:0/552668054 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e3d13280c0 con 0x55e3d142dde0 2024-03-21T11:47:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.826+0000 7fdc3f400640 1 --2- 172.21.15.47:0/552668054 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e3d142dde0 0x55e3d1430200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.827+0000 7fdc2a000640 10 client.33148 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:26.827+0000 7fdc3f400640 1 --2- 172.21.15.47:0/552668054 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e3d142dde0 0x55e3d1430200 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac309dd8a6f9cd1 server_cookie=a9937f277eb901f in_seq=0 out_seq=0 2024-03-21T11:47:26.994 INFO:tasks.workunit.client.0.smithi047.stdout: 10 255292 21.66 MB/sec execute 490 sec latency 429.962 ms 2024-03-21T11:47:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.406+0000 7fdc2a000640 1 -- 172.21.15.47:0/552668054 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e3d13280c0 con 0x55e3d142dde0 2024-03-21T11:47:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.406+0000 7fdc2a000640 10 client.33148 handle_command_reply: tid=0 2024-03-21T11:47:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.406+0000 7fdc4a019740 1 client.33148 shutdown 2024-03-21T11:47:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.406+0000 7fdc4a019740 1 -- 172.21.15.47:0/552668054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bb3f0 msgr2=0x55e3d13ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.406+0000 7fdc4a019740 1 --2- 172.21.15.47:0/552668054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bb3f0 0x55e3d13ca9b0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fdc30005ab0 tx=0x7fdc30000bc0 comp rx=0 tx=0).stop 2024-03-21T11:47:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.407+0000 7fdc4a019740 1 -- 172.21.15.47:0/552668054 shutdown_connections 2024-03-21T11:47:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.407+0000 7fdc4a019740 1 --2- 172.21.15.47:0/552668054 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e3d142dde0 0x55e3d1430200 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.407+0000 7fdc4a019740 1 --2- 172.21.15.47:0/552668054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e3d13c0f50 0x55e3d13cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.407+0000 7fdc4a019740 1 --2- 172.21.15.47:0/552668054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e3d13bbde0 0x55e3d13caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.407+0000 7fdc4a019740 1 --2- 172.21.15.47:0/552668054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3d13bb3f0 0x55e3d13ca9b0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.407+0000 7fdc4a019740 1 -- 172.21.15.47:0/552668054 >> 172.21.15.47:0/552668054 conn(0x55e3d13b6960 msgr2=0x55e3d13c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.407+0000 7fdc2a000640 0 client.33148 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.407+0000 7fdc4a019740 1 -- 172.21.15.47:0/552668054 shutdown_connections 2024-03-21T11:47:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.407+0000 7fdc4a019740 1 -- 172.21.15.47:0/552668054 wait complete. 2024-03-21T11:47:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.408+0000 7fdc4a019740 20 client.33148 trim_cache size 0 max 16384 2024-03-21T11:47:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:47:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:47:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:47:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:47:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.409+0000 7fdc45c00640 1 -- 172.21.15.47:0/732782219 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc240a2f10 msgr2=0x7fdc240a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.409+0000 7fdc45c00640 1 --2- 172.21.15.47:0/732782219 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc240a2f10 0x7fdc240a53d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdc38010fd0 tx=0x7fdc38010f20 comp rx=0 tx=0).stop 2024-03-21T11:47:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.409+0000 7fdc45c00640 1 -- 172.21.15.47:0/732782219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc401048a0 msgr2=0x7fdc4019e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.409+0000 7fdc45c00640 1 --2- 172.21.15.47:0/732782219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc401048a0 0x7fdc4019e300 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00e9c0 tx=0x7fdc2c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:47:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.409+0000 7fdc45c00640 1 -- 172.21.15.47:0/732782219 shutdown_connections 2024-03-21T11:47:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.409+0000 7fdc45c00640 1 --2- 172.21.15.47:0/732782219 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdc240a2f10 0x7fdc240a53d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.409+0000 7fdc45c00640 1 --2- 172.21.15.47:0/732782219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc401071d0 0x7fdc401a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.409+0000 7fdc45c00640 1 --2- 172.21.15.47:0/732782219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdc401048a0 0x7fdc4019e300 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.409+0000 7fdc45c00640 1 --2- 172.21.15.47:0/732782219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc40101f70 0x7fdc4019ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.409+0000 7fdc45c00640 1 -- 172.21.15.47:0/732782219 >> 172.21.15.47:0/732782219 conn(0x7fdc400fb980 msgr2=0x7fdc40102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.409+0000 7fdc45c00640 1 -- 172.21.15.47:0/732782219 shutdown_connections 2024-03-21T11:47:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:27.410+0000 7fdc45c00640 1 -- 172.21.15.47:0/732782219 wait complete. 2024-03-21T11:47:27.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:27 smithi047 ceph-mon[31351]: pgmap v1124: 129 pgs: 129 active+clean; 794 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 616 op/s 2024-03-21T11:47:27.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:27 smithi100 ceph-mon[36245]: pgmap v1124: 129 pgs: 129 active+clean; 794 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 616 op/s 2024-03-21T11:47:27.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:27 smithi102 ceph-mon[36748]: pgmap v1124: 129 pgs: 129 active+clean; 794 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 616 op/s 2024-03-21T11:47:27.994 INFO:tasks.workunit.client.0.smithi047.stdout: 10 255528 21.65 MB/sec execute 491 sec latency 313.116 ms 2024-03-21T11:47:28.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:28 smithi047 ceph-mon[31351]: pgmap v1125: 129 pgs: 129 active+clean; 782 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 14 MiB/s wr, 613 op/s 2024-03-21T11:47:28.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:28 smithi100 ceph-mon[36245]: pgmap v1125: 129 pgs: 129 active+clean; 782 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 14 MiB/s wr, 613 op/s 2024-03-21T11:47:28.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:28 smithi102 ceph-mon[36748]: pgmap v1125: 129 pgs: 129 active+clean; 782 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 14 MiB/s wr, 613 op/s 2024-03-21T11:47:28.994 INFO:tasks.workunit.client.0.smithi047.stdout: 10 255748 21.62 MB/sec execute 492 sec latency 160.908 ms 2024-03-21T11:47:29.994 INFO:tasks.workunit.client.0.smithi047.stdout: 10 256181 21.63 MB/sec execute 493 sec latency 109.838 ms 2024-03-21T11:47:30.994 INFO:tasks.workunit.client.0.smithi047.stdout: 10 256496 21.62 MB/sec execute 494 sec latency 130.689 ms 2024-03-21T11:47:31.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:30 smithi100 ceph-mon[36245]: pgmap v1126: 129 pgs: 129 active+clean; 730 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 660 op/s 2024-03-21T11:47:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:30 smithi102 ceph-mon[36748]: pgmap v1126: 129 pgs: 129 active+clean; 730 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 660 op/s 2024-03-21T11:47:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:30 smithi047 ceph-mon[31351]: pgmap v1126: 129 pgs: 129 active+clean; 730 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 660 op/s 2024-03-21T11:47:31.994 INFO:tasks.workunit.client.0.smithi047.stdout: 10 256954 21.63 MB/sec execute 495 sec latency 94.671 ms 2024-03-21T11:47:32.994 INFO:tasks.workunit.client.0.smithi047.stdout: 10 257381 21.62 MB/sec execute 496 sec latency 83.847 ms 2024-03-21T11:47:33.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:32 smithi100 ceph-mon[36245]: pgmap v1127: 129 pgs: 129 active+clean; 708 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 12 MiB/s wr, 693 op/s 2024-03-21T11:47:33.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:32 smithi102 ceph-mon[36748]: pgmap v1127: 129 pgs: 129 active+clean; 708 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 12 MiB/s wr, 693 op/s 2024-03-21T11:47:33.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:32 smithi047 ceph-mon[31351]: pgmap v1127: 129 pgs: 129 active+clean; 708 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 12 MiB/s wr, 693 op/s 2024-03-21T11:47:33.995 INFO:tasks.workunit.client.0.smithi047.stdout: 10 257931 21.63 MB/sec execute 497 sec latency 47.358 ms 2024-03-21T11:47:34.995 INFO:tasks.workunit.client.0.smithi047.stdout: 10 258472 21.66 MB/sec execute 498 sec latency 97.899 ms 2024-03-21T11:47:35.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:34 smithi047 ceph-mon[31351]: pgmap v1128: 129 pgs: 129 active+clean; 700 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 712 op/s 2024-03-21T11:47:35.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:34 smithi100 ceph-mon[36245]: pgmap v1128: 129 pgs: 129 active+clean; 700 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 712 op/s 2024-03-21T11:47:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:34 smithi102 ceph-mon[36748]: pgmap v1128: 129 pgs: 129 active+clean; 700 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 712 op/s 2024-03-21T11:47:35.995 INFO:tasks.workunit.client.0.smithi047.stdout: 10 258834 21.66 MB/sec execute 499 sec latency 77.404 ms 2024-03-21T11:47:36.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:35 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:47:36.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:35 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:47:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:35 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:47:36.995 INFO:tasks.workunit.client.0.smithi047.stdout: 10 259226 21.66 MB/sec execute 500 sec latency 71.861 ms 2024-03-21T11:47:37.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:36 smithi100 ceph-mon[36245]: pgmap v1129: 129 pgs: 129 active+clean; 698 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 602 op/s 2024-03-21T11:47:37.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:36 smithi102 ceph-mon[36748]: pgmap v1129: 129 pgs: 129 active+clean; 698 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 602 op/s 2024-03-21T11:47:37.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:36 smithi047 ceph-mon[31351]: pgmap v1129: 129 pgs: 129 active+clean; 698 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 602 op/s 2024-03-21T11:47:37.995 INFO:tasks.workunit.client.0.smithi047.stdout: 10 259628 21.66 MB/sec execute 501 sec latency 97.246 ms 2024-03-21T11:47:38.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:47:38.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:47:38.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:47:38.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:37 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:47:38.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:37 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:47:38.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:37 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:47:38.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:47:38.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:47:38.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:47:38.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:37 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:47:38.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:37 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:47:38.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:37 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:47:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:47:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:47:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:47:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:37 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:47:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:37 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:47:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:37 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:47:38.995 INFO:tasks.workunit.client.0.smithi047.stdout: 10 260181 21.65 MB/sec execute 502 sec latency 113.954 ms 2024-03-21T11:47:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:38 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:47:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:38 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:47:39.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:38 smithi100 ceph-mon[36245]: pgmap v1130: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.0 MiB/s wr, 580 op/s 2024-03-21T11:47:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:38 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:47:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:38 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:47:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:38 smithi102 ceph-mon[36748]: pgmap v1130: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.0 MiB/s wr, 580 op/s 2024-03-21T11:47:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:38 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:47:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:38 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:47:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:38 smithi047 ceph-mon[31351]: pgmap v1130: 129 pgs: 129 active+clean; 702 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.0 MiB/s wr, 580 op/s 2024-03-21T11:47:39.995 INFO:tasks.workunit.client.0.smithi047.stdout: 10 260584 21.66 MB/sec execute 503 sec latency 103.079 ms 2024-03-21T11:47:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.585+0000 7fb4a8e00640 1 -- 172.21.15.47:0/2144142659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a41054b0 msgr2=0x7fb4a410bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.585+0000 7fb4a8e00640 1 --2- 172.21.15.47:0/2144142659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a41054b0 0x7fb4a410bd40 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009be0 tx=0x7fb49c02bef0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.585+0000 7fb4a8e00640 1 -- 172.21.15.47:0/2144142659 shutdown_connections 2024-03-21T11:47:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.585+0000 7fb4a8e00640 1 --2- 172.21.15.47:0/2144142659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a41054b0 0x7fb4a410bd40 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.585+0000 7fb4a8e00640 1 --2- 172.21.15.47:0/2144142659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4a4104af0 0x7fb4a4104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.585+0000 7fb4a8e00640 1 --2- 172.21.15.47:0/2144142659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a4103900 0x7fb4a4103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.585+0000 7fb4a8e00640 1 -- 172.21.15.47:0/2144142659 >> 172.21.15.47:0/2144142659 conn(0x7fb4a40fee90 msgr2=0x7fb4a41012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.585+0000 7fb4a8e00640 1 -- 172.21.15.47:0/2144142659 shutdown_connections 2024-03-21T11:47:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.585+0000 7fb4a8e00640 1 -- 172.21.15.47:0/2144142659 wait complete. 2024-03-21T11:47:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.587+0000 7fb4a8e00640 1 Processor -- start 2024-03-21T11:47:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.587+0000 7fb4a8e00640 1 -- start start 2024-03-21T11:47:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.587+0000 7fb4a8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4a4103900 0x7fb4a4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.587+0000 7fb4a8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a4104af0 0x7fb4a4199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.587+0000 7fb4a8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a41054b0 0x7fb4a41a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.587+0000 7fb4a8e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a410f560 con 0x7fb4a4104af0 2024-03-21T11:47:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.587+0000 7fb4a8e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a410f3e0 con 0x7fb4a41054b0 2024-03-21T11:47:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.587+0000 7fb4a8e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a410f6e0 con 0x7fb4a4103900 2024-03-21T11:47:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.587+0000 7fb4a2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a41054b0 0x7fb4a41a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.587+0000 7fb4a2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a41054b0 0x7fb4a41a1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39754/0 (socket says 172.21.15.47:39754) 2024-03-21T11:47:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.588+0000 7fb4a2800640 1 -- 172.21.15.47:0/278651127 learned_addr learned my addr 172.21.15.47:0/278651127 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.588+0000 7fb4a1400640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a4104af0 0x7fb4a4199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.588+0000 7fb4a2800640 1 -- 172.21.15.47:0/278651127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4a4103900 msgr2=0x7fb4a4199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.588+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4a4103900 0x7fb4a4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.588+0000 7fb4a2800640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4a4103900 0x7fb4a4199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.588+0000 7fb4a2800640 1 -- 172.21.15.47:0/278651127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a4104af0 msgr2=0x7fb4a4199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.588+0000 7fb4a2800640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a4104af0 0x7fb4a4199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.588+0000 7fb4a2800640 1 -- 172.21.15.47:0/278651127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a41a16d0 con 0x7fb4a41054b0 2024-03-21T11:47:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.588+0000 7fb4a1400640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a4104af0 0x7fb4a4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:47:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.588+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4a4103900 0x7fb4a4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.589+0000 7fb4a2800640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a41054b0 0x7fb4a41a1060 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb49c036040 tx=0x7fb49c0337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.589+0000 7fb49aa00640 1 -- 172.21.15.47:0/278651127 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c043020 con 0x7fb4a41054b0 2024-03-21T11:47:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.589+0000 7fb49aa00640 1 -- 172.21.15.47:0/278651127 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb49c02ca60 con 0x7fb4a41054b0 2024-03-21T11:47:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.589+0000 7fb4a8e00640 1 -- 172.21.15.47:0/278651127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a41a1960 con 0x7fb4a41054b0 2024-03-21T11:47:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.589+0000 7fb49aa00640 1 -- 172.21.15.47:0/278651127 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c034690 con 0x7fb4a41054b0 2024-03-21T11:47:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.589+0000 7fb4a8e00640 1 -- 172.21.15.47:0/278651127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a41a1e40 con 0x7fb4a41054b0 2024-03-21T11:47:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.593+0000 7fb49aa00640 1 -- 172.21.15.47:0/278651127 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb49c046020 con 0x7fb4a41054b0 2024-03-21T11:47:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.593+0000 7fb4abc0d740 1 Processor -- start 2024-03-21T11:47:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.593+0000 7fb4abc0d740 1 -- start start 2024-03-21T11:47:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.593+0000 7fb4abc0d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe51e0 0x5645afbe5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.593+0000 7fb4abc0d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.593+0000 7fb4abc0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbead40 0x5645afbed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.593+0000 7fb4abc0d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645afba83b0 con 0x5645afbe5bd0 2024-03-21T11:47:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.594+0000 7fb4abc0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645afb4a830 con 0x5645afbead40 2024-03-21T11:47:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.594+0000 7fb4abc0d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645afaf35d0 con 0x5645afbe51e0 2024-03-21T11:47:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.594+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe51e0 0x5645afbe5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.594+0000 7fb4a1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.594+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe51e0 0x5645afbe5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51878/0 (socket says 172.21.15.47:51878) 2024-03-21T11:47:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.594+0000 7fb4a1400640 1 -- 172.21.15.47:0/2008041230 learned_addr learned my addr 172.21.15.47:0/2008041230 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.594+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/2008041230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbea6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47372/0 (socket says 172.21.15.47:47372) 2024-03-21T11:47:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.594+0000 7fb4a1400640 1 --2- 172.21.15.47:0/2008041230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbead40 0x5645afbed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.594+0000 7fb4a1e00640 1 -- 172.21.15.47:0/2008041230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe51e0 msgr2=0x5645afbe5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.594+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/2008041230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe51e0 0x5645afbe5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.594+0000 7fb4a1e00640 1 -- 172.21.15.47:0/2008041230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbead40 msgr2=0x5645afbed1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.594+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/2008041230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbead40 0x5645afbed1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.594+0000 7fb4a1e00640 1 -- 172.21.15.47:0/2008041230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645afb0b2c0 con 0x5645afbe5bd0 2024-03-21T11:47:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.594+0000 7fb4a1400640 1 --2- 172.21.15.47:0/2008041230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbead40 0x5645afbed1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:47:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.595+0000 7fb49aa00640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb4880a3040 0x7fb4880a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.595+0000 7fb49aa00640 1 -- 172.21.15.47:0/278651127 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb49c0f1fa0 con 0x7fb4a41054b0 2024-03-21T11:47:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.595+0000 7fb4a1400640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb4880a3040 0x7fb4880a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.595+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/2008041230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbea6d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb494009c30 tx=0x7fb494010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d684e15a01c96a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.596+0000 7fb48f400640 1 -- 172.21.15.47:0/2008041230 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb494005020 con 0x5645afbe5bd0 2024-03-21T11:47:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.596+0000 7fb48f400640 1 -- 172.21.15.47:0/2008041230 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4940047a0 con 0x5645afbe5bd0 2024-03-21T11:47:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.596+0000 7fb48f400640 1 -- 172.21.15.47:0/2008041230 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4940126b0 con 0x5645afbe5bd0 2024-03-21T11:47:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.596+0000 7fb4a1400640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb4880a3040 0x7fb4880a5500 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb490009fd0 tx=0x7fb490004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.597+0000 7fb4abc0d740 1 -- 172.21.15.47:0/2008041230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 msgr2=0x5645afbea6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.597+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/2008041230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbea6d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb494009c30 tx=0x7fb494010740 comp rx=0 tx=0).stop 2024-03-21T11:47:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.597+0000 7fb4abc0d740 1 -- 172.21.15.47:0/2008041230 shutdown_connections 2024-03-21T11:47:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.597+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/2008041230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbead40 0x5645afbed1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.597+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/2008041230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbea6d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.597+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/2008041230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe51e0 0x5645afbe5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.597+0000 7fb4abc0d740 1 -- 172.21.15.47:0/2008041230 >> 172.21.15.47:0/2008041230 conn(0x5645afbe0750 msgr2=0x5645afbe0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.597+0000 7fb4abc0d740 1 -- 172.21.15.47:0/2008041230 shutdown_connections 2024-03-21T11:47:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.597+0000 7fb4abc0d740 1 -- 172.21.15.47:0/2008041230 wait complete. 2024-03-21T11:47:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.598+0000 7fb4abc0d740 1 -- start start 2024-03-21T11:47:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.598+0000 7fb4abc0d740 1 Processor -- start 2024-03-21T11:47:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.598+0000 7fb4abc0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbe51e0 0x5645afbef420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.598+0000 7fb4abc0d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.598+0000 7fb4abc0d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbead40 0x5645afbf6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.598+0000 7fb4abc0d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645afb06a20 con 0x5645afbe5bd0 2024-03-21T11:47:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.598+0000 7fb4abc0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645afb468d0 con 0x5645afbe51e0 2024-03-21T11:47:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.598+0000 7fb4abc0d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645afbedcf0 con 0x5645afbead40 2024-03-21T11:47:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.598+0000 7fb4abc0d740 10 client.0 _ll_register_callbacks cb 0x5645afb431c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.599+0000 7fb4a1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbe51e0 0x5645afbef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.599+0000 7fb4a1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbe51e0 0x5645afbef420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39782/0 (socket says 172.21.15.47:39782) 2024-03-21T11:47:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.599+0000 7fb4a1e00640 1 -- 172.21.15.47:0/517502743 learned_addr learned my addr 172.21.15.47:0/517502743 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.599+0000 7fb4a1400640 1 --2- 172.21.15.47:0/517502743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.599+0000 7fb4a2800640 1 --2- 172.21.15.47:0/517502743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbead40 0x5645afbf6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.599+0000 7fb4a1e00640 1 -- 172.21.15.47:0/517502743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbead40 msgr2=0x5645afbf6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.599+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/517502743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbead40 0x5645afbf6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.599+0000 7fb4a1e00640 1 -- 172.21.15.47:0/517502743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 msgr2=0x5645afbef960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.599+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/517502743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbef960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.599+0000 7fb4a1e00640 1 -- 172.21.15.47:0/517502743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645af85f210 con 0x5645afbe51e0 2024-03-21T11:47:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.599+0000 7fb4a1400640 1 --2- 172.21.15.47:0/517502743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbef960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.600+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/517502743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbe51e0 0x5645afbef420 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb494019000 tx=0x7fb494004ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f8b2df356f9bcc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.600+0000 7fb483e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.600+0000 7fb483e00640 1 -- 172.21.15.47:0/517502743 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb494009d20 con 0x5645afbe51e0 2024-03-21T11:47:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.600+0000 7fb483e00640 1 -- 172.21.15.47:0/517502743 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb494012450 con 0x5645afbe51e0 2024-03-21T11:47:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.600+0000 7fb4abc0d740 1 -- 172.21.15.47:0/517502743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4940047a0 con 0x5645afbe51e0 2024-03-21T11:47:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.601+0000 7fb483e00640 1 -- 172.21.15.47:0/517502743 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb494012650 con 0x5645afbe51e0 2024-03-21T11:47:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.601+0000 7fb4abc0d740 10 client.33209 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.601+0000 7fb4abc0d740 1 -- 172.21.15.47:0/517502743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5645af92b620 con 0x5645afbe51e0 2024-03-21T11:47:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.601+0000 7fb483e00640 1 -- 172.21.15.47:0/517502743 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4940047a0 con 0x5645afbe51e0 2024-03-21T11:47:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.602+0000 7fb483e00640 1 -- 172.21.15.47:0/517502743 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb49400f040 con 0x5645afbe51e0 2024-03-21T11:47:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.602+0000 7fb4abc0d740 10 client.33209 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.602+0000 7fb4abc0d740 10 client.33209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.603+0000 7fb4abc0d740 20 client.33209 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.603+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/517502743 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5645afbfaed0 0x5645afbfd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.603+0000 7fb4abc0d740 4 client.33209 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.603+0000 7fb4abc0d740 1 -- 172.21.15.47:0/517502743 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645afb57d70 con 0x5645afbfaed0 2024-03-21T11:47:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.603+0000 7fb4a1400640 1 --2- 172.21.15.47:0/517502743 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5645afbfaed0 0x5645afbfd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.605+0000 7fb4a1400640 1 --2- 172.21.15.47:0/517502743 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5645afbfaed0 0x5645afbfd2c0 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f47c34b43bc0f7 server_cookie=d42ed0484a20b898 in_seq=0 out_seq=0 2024-03-21T11:47:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.605+0000 7fb483e00640 10 client.33209 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.606+0000 7fb483e00640 1 -- 172.21.15.47:0/517502743 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5645afb57d70 con 0x5645afbfaed0 2024-03-21T11:47:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.606+0000 7fb483e00640 10 client.33209 handle_command_reply: tid=0 2024-03-21T11:47:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.606+0000 7fb4abc0d740 1 client.33209 shutdown 2024-03-21T11:47:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.607+0000 7fb4abc0d740 1 -- 172.21.15.47:0/517502743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbe51e0 msgr2=0x5645afbef420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.607+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/517502743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbe51e0 0x5645afbef420 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb494019000 tx=0x7fb494004ec0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.608+0000 7fb4abc0d740 1 -- 172.21.15.47:0/517502743 shutdown_connections 2024-03-21T11:47:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.608+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/517502743 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5645afbfaed0 0x5645afbfd2c0 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.608+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/517502743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbead40 0x5645afbf6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.608+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/517502743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbef960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.608+0000 7fb483e00640 0 client.33209 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.608+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/517502743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbe51e0 0x5645afbef420 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.608+0000 7fb4abc0d740 1 -- 172.21.15.47:0/517502743 >> 172.21.15.47:0/517502743 conn(0x5645afbe0750 msgr2=0x5645afbdf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.608+0000 7fb4abc0d740 1 -- 172.21.15.47:0/517502743 shutdown_connections 2024-03-21T11:47:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.608+0000 7fb4abc0d740 1 -- 172.21.15.47:0/517502743 wait complete. 2024-03-21T11:47:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.608+0000 7fb4abc0d740 20 client.33209 trim_cache size 0 max 16384 2024-03-21T11:47:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.635+0000 7fb4abc0d740 1 Processor -- start 2024-03-21T11:47:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.636+0000 7fb4abc0d740 1 -- start start 2024-03-21T11:47:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.636+0000 7fb4abc0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbe51e0 0x5645afbe40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.636+0000 7fb4abc0d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe5bd0 0x5645afc01ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.636+0000 7fb4abc0d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbead40 0x5645afc020e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.636+0000 7fb4abc0d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a41ac1e0 con 0x5645afbead40 2024-03-21T11:47:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.636+0000 7fb4abc0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a41adb60 con 0x5645afbe51e0 2024-03-21T11:47:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.636+0000 7fb4abc0d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a41ac840 con 0x5645afbe5bd0 2024-03-21T11:47:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.636+0000 7fb4a1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbe51e0 0x5645afbe40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.636+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe5bd0 0x5645afc01ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.636+0000 7fb4a1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbe51e0 0x5645afbe40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39794/0 (socket says 172.21.15.47:39794) 2024-03-21T11:47:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.636+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe5bd0 0x5645afc01ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51906/0 (socket says 172.21.15.47:51906) 2024-03-21T11:47:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.636+0000 7fb4a1e00640 1 -- 172.21.15.47:0/3327890848 learned_addr learned my addr 172.21.15.47:0/3327890848 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.636+0000 7fb4a2800640 1 --2- 172.21.15.47:0/3327890848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbead40 0x5645afc020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.637+0000 7fb4a1e00640 1 -- 172.21.15.47:0/3327890848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe5bd0 msgr2=0x5645afc01ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.637+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/3327890848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe5bd0 0x5645afc01ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.637+0000 7fb4a1e00640 1 -- 172.21.15.47:0/3327890848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbead40 msgr2=0x5645afc020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.637+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/3327890848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbead40 0x5645afc020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.637+0000 7fb4a1e00640 1 -- 172.21.15.47:0/3327890848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645afbda9c0 con 0x5645afbe51e0 2024-03-21T11:47:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.637+0000 7fb4a1400640 1 --2- 172.21.15.47:0/3327890848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe5bd0 0x5645afc01ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.637+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/3327890848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbe51e0 0x5645afbe40c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb494010780 tx=0x7fb494005640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76f8b3e0a7f1e452 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.637+0000 7fb48ea00640 1 -- 172.21.15.47:0/3327890848 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb494004780 con 0x5645afbe51e0 2024-03-21T11:47:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.637+0000 7fb48ea00640 1 -- 172.21.15.47:0/3327890848 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb49401e020 con 0x5645afbe51e0 2024-03-21T11:47:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.637+0000 7fb48ea00640 1 -- 172.21.15.47:0/3327890848 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49401a480 con 0x5645afbe51e0 2024-03-21T11:47:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.638+0000 7fb4abc0d740 1 -- 172.21.15.47:0/3327890848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbe51e0 msgr2=0x5645afbe40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.638+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/3327890848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbe51e0 0x5645afbe40c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb494010780 tx=0x7fb494005640 comp rx=0 tx=0).stop 2024-03-21T11:47:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.638+0000 7fb4abc0d740 1 -- 172.21.15.47:0/3327890848 shutdown_connections 2024-03-21T11:47:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.638+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/3327890848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbead40 0x5645afc020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.639+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/3327890848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe5bd0 0x5645afc01ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.639+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/3327890848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbe51e0 0x5645afbe40c0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.639+0000 7fb4abc0d740 1 -- 172.21.15.47:0/3327890848 >> 172.21.15.47:0/3327890848 conn(0x5645afbe0750 msgr2=0x5645afbe39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.639+0000 7fb4abc0d740 1 -- 172.21.15.47:0/3327890848 shutdown_connections 2024-03-21T11:47:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.639+0000 7fb4abc0d740 1 -- 172.21.15.47:0/3327890848 wait complete. 2024-03-21T11:47:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.639+0000 7fb4abc0d740 1 -- start start 2024-03-21T11:47:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.639+0000 7fb4abc0d740 1 Processor -- start 2024-03-21T11:47:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.640+0000 7fb4abc0d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe51e0 0x5645afbf3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.640+0000 7fb4abc0d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbf37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.640+0000 7fb4abc0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbead40 0x5645afbf4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.640+0000 7fb4abc0d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a41ad500 con 0x5645afbe5bd0 2024-03-21T11:47:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.640+0000 7fb4abc0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a41acea0 con 0x5645afbead40 2024-03-21T11:47:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.640+0000 7fb4abc0d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a41aa3a0 con 0x5645afbe51e0 2024-03-21T11:47:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.640+0000 7fb4abc0d740 10 client.0 _ll_register_callbacks cb 0x7fb490009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.640+0000 7fb4a1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe51e0 0x5645afbf3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.640+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbf37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.640+0000 7fb4a1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe51e0 0x5645afbf3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51920/0 (socket says 172.21.15.47:51920) 2024-03-21T11:47:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.640+0000 7fb4a1e00640 1 -- 172.21.15.47:0/2407938643 learned_addr learned my addr 172.21.15.47:0/2407938643 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.641+0000 7fb4a1e00640 1 -- 172.21.15.47:0/2407938643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbead40 msgr2=0x5645afbf4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.641+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/2407938643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbead40 0x5645afbf4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.641+0000 7fb4a1e00640 1 -- 172.21.15.47:0/2407938643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 msgr2=0x5645afbf37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:40.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.641+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/2407938643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbf37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.641+0000 7fb4a1e00640 1 -- 172.21.15.47:0/2407938643 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645afbf4d50 con 0x5645afbe51e0 2024-03-21T11:47:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.641+0000 7fb4a1400640 1 --2- 172.21.15.47:0/2407938643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbf37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.641+0000 7fb4a1e00640 1 --2- 172.21.15.47:0/2407938643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe51e0 0x5645afbf3260 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fb494010cc0 tx=0x7fb494004640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ede0b14abdc6b64 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.641+0000 7fb483e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:47:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.642+0000 7fb483e00640 1 -- 172.21.15.47:0/2407938643 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb494004780 con 0x5645afbe51e0 2024-03-21T11:47:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.642+0000 7fb483e00640 1 -- 172.21.15.47:0/2407938643 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb494009530 con 0x5645afbe51e0 2024-03-21T11:47:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.642+0000 7fb4abc0d740 1 -- 172.21.15.47:0/2407938643 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb49401e020 con 0x5645afbe51e0 2024-03-21T11:47:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.642+0000 7fb483e00640 1 -- 172.21.15.47:0/2407938643 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb49401a480 con 0x5645afbe51e0 2024-03-21T11:47:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.643+0000 7fb4abc0d740 10 client.33163 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.643+0000 7fb4abc0d740 1 -- 172.21.15.47:0/2407938643 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5645afbf6060 con 0x5645afbe51e0 2024-03-21T11:47:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.643+0000 7fb483e00640 1 -- 172.21.15.47:0/2407938643 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb49401e020 con 0x5645afbe51e0 2024-03-21T11:47:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.643+0000 7fb483e00640 1 -- 172.21.15.47:0/2407938643 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb494034890 con 0x5645afbe51e0 2024-03-21T11:47:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.643+0000 7fb4abc0d740 10 client.33163 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.643+0000 7fb4abc0d740 10 client.33163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.643+0000 7fb4abc0d740 20 client.33163 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.643+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/2407938643 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5645afc569a0 0x5645afc58dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.643+0000 7fb4abc0d740 4 client.33163 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:47:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.643+0000 7fb4abc0d740 1 -- 172.21.15.47:0/2407938643 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5645afb520c0 con 0x5645afc569a0 2024-03-21T11:47:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.644+0000 7fb4a1400640 1 --2- 172.21.15.47:0/2407938643 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5645afc569a0 0x5645afc58dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.645+0000 7fb4a1400640 1 --2- 172.21.15.47:0/2407938643 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5645afc569a0 0x5645afc58dc0 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31b331a510347ef9 server_cookie=e75cb95a53aa6178 in_seq=0 out_seq=0 2024-03-21T11:47:40.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:40.645+0000 7fb483e00640 10 client.33163 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:40.995 INFO:tasks.workunit.client.0.smithi047.stdout: 10 260968 21.67 MB/sec execute 504 sec latency 254.515 ms 2024-03-21T11:47:41.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:40 smithi100 ceph-mon[36245]: pgmap v1131: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.2 MiB/s wr, 523 op/s 2024-03-21T11:47:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:40 smithi102 ceph-mon[36748]: pgmap v1131: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.2 MiB/s wr, 523 op/s 2024-03-21T11:47:41.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:40 smithi047 ceph-mon[31351]: pgmap v1131: 129 pgs: 129 active+clean; 706 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.2 MiB/s wr, 523 op/s 2024-03-21T11:47:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.608+0000 7fb483e00640 1 -- 172.21.15.47:0/2407938643 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5645afb520c0 con 0x5645afc569a0 2024-03-21T11:47:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.608+0000 7fb483e00640 10 client.33163 handle_command_reply: tid=0 2024-03-21T11:47:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.608+0000 7fb4abc0d740 1 client.33163 shutdown 2024-03-21T11:47:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.609+0000 7fb4abc0d740 1 -- 172.21.15.47:0/2407938643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe51e0 msgr2=0x5645afbf3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.609+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/2407938643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe51e0 0x5645afbf3260 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fb494010cc0 tx=0x7fb494004640 comp rx=0 tx=0).stop 2024-03-21T11:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.609+0000 7fb4abc0d740 1 -- 172.21.15.47:0/2407938643 shutdown_connections 2024-03-21T11:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.609+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/2407938643 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5645afc569a0 0x5645afc58dc0 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.609+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/2407938643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645afbead40 0x5645afbf4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.609+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/2407938643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5645afbe5bd0 0x5645afbf37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.609+0000 7fb4abc0d740 1 --2- 172.21.15.47:0/2407938643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645afbe51e0 0x5645afbf3260 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.609+0000 7fb483e00640 0 client.33163 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.609+0000 7fb4abc0d740 1 -- 172.21.15.47:0/2407938643 >> 172.21.15.47:0/2407938643 conn(0x5645afbe0750 msgr2=0x5645afbe4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.609+0000 7fb4abc0d740 1 -- 172.21.15.47:0/2407938643 shutdown_connections 2024-03-21T11:47:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.609+0000 7fb4abc0d740 1 -- 172.21.15.47:0/2407938643 wait complete. 2024-03-21T11:47:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.609+0000 7fb4abc0d740 20 client.33163 trim_cache size 0 max 16384 2024-03-21T11:47:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:47:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:47:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:47:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:47:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.610+0000 7fb4a8e00640 1 -- 172.21.15.47:0/278651127 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb4880a3040 msgr2=0x7fb4880a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.610+0000 7fb4a8e00640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb4880a3040 0x7fb4880a5500 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb490009fd0 tx=0x7fb490004730 comp rx=0 tx=0).stop 2024-03-21T11:47:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.610+0000 7fb4a8e00640 1 -- 172.21.15.47:0/278651127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a41054b0 msgr2=0x7fb4a41a1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.610+0000 7fb4a8e00640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a41054b0 0x7fb4a41a1060 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb49c036040 tx=0x7fb49c0337b0 comp rx=0 tx=0).stop 2024-03-21T11:47:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.610+0000 7fb4a8e00640 1 -- 172.21.15.47:0/278651127 shutdown_connections 2024-03-21T11:47:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.610+0000 7fb4a8e00640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb4880a3040 0x7fb4880a5500 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.610+0000 7fb4a8e00640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a41054b0 0x7fb4a41a1060 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.610+0000 7fb4a8e00640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4a4104af0 0x7fb4a4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.610+0000 7fb4a8e00640 1 --2- 172.21.15.47:0/278651127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4a4103900 0x7fb4a4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.610+0000 7fb4a8e00640 1 -- 172.21.15.47:0/278651127 >> 172.21.15.47:0/278651127 conn(0x7fb4a40fee90 msgr2=0x7fb4a4107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.610+0000 7fb4a8e00640 1 -- 172.21.15.47:0/278651127 shutdown_connections 2024-03-21T11:47:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:41.610+0000 7fb4a8e00640 1 -- 172.21.15.47:0/278651127 wait complete. 2024-03-21T11:47:41.995 INFO:tasks.workunit.client.0.smithi047.stdout: 10 261284 21.67 MB/sec execute 505 sec latency 585.943 ms 2024-03-21T11:47:42.995 INFO:tasks.workunit.client.0.smithi047.stdout: 10 261790 21.66 MB/sec execute 506 sec latency 88.546 ms 2024-03-21T11:47:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:42 smithi102 ceph-mon[36748]: pgmap v1132: 129 pgs: 129 active+clean; 709 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.3 MiB/s wr, 423 op/s 2024-03-21T11:47:43.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:42 smithi047 ceph-mon[31351]: pgmap v1132: 129 pgs: 129 active+clean; 709 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.3 MiB/s wr, 423 op/s 2024-03-21T11:47:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:42 smithi100 ceph-mon[36245]: pgmap v1132: 129 pgs: 129 active+clean; 709 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.3 MiB/s wr, 423 op/s 2024-03-21T11:47:43.995 INFO:tasks.workunit.client.0.smithi047.stdout: 10 262196 21.66 MB/sec execute 507 sec latency 101.489 ms 2024-03-21T11:47:44.995 INFO:tasks.workunit.client.0.smithi047.stdout: 10 262839 21.69 MB/sec execute 508 sec latency 77.758 ms 2024-03-21T11:47:45.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:45 smithi102 ceph-mon[36748]: pgmap v1133: 129 pgs: 129 active+clean; 708 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.7 MiB/s wr, 481 op/s 2024-03-21T11:47:45.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:45 smithi047 ceph-mon[31351]: pgmap v1133: 129 pgs: 129 active+clean; 708 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.7 MiB/s wr, 481 op/s 2024-03-21T11:47:45.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:45 smithi100 ceph-mon[36245]: pgmap v1133: 129 pgs: 129 active+clean; 708 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.7 MiB/s wr, 481 op/s 2024-03-21T11:47:45.995 INFO:tasks.workunit.client.0.smithi047.stdout: 10 263281 21.71 MB/sec execute 509 sec latency 59.386 ms 2024-03-21T11:47:46.996 INFO:tasks.workunit.client.0.smithi047.stdout: 10 263744 21.71 MB/sec execute 510 sec latency 83.889 ms 2024-03-21T11:47:47.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:47 smithi102 ceph-mon[36748]: pgmap v1134: 129 pgs: 129 active+clean; 711 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.5 MiB/s wr, 448 op/s 2024-03-21T11:47:47.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:47 smithi047 ceph-mon[31351]: pgmap v1134: 129 pgs: 129 active+clean; 711 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.5 MiB/s wr, 448 op/s 2024-03-21T11:47:47.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:47 smithi100 ceph-mon[36245]: pgmap v1134: 129 pgs: 129 active+clean; 711 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.5 MiB/s wr, 448 op/s 2024-03-21T11:47:47.996 INFO:tasks.workunit.client.0.smithi047.stdout: 10 264119 21.70 MB/sec execute 511 sec latency 79.300 ms 2024-03-21T11:47:48.996 INFO:tasks.workunit.client.0.smithi047.stdout: 10 264528 21.69 MB/sec execute 512 sec latency 151.435 ms 2024-03-21T11:47:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:49 smithi102 ceph-mon[36748]: pgmap v1135: 129 pgs: 129 active+clean; 714 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.5 MiB/s wr, 454 op/s 2024-03-21T11:47:49.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:49 smithi047 ceph-mon[31351]: pgmap v1135: 129 pgs: 129 active+clean; 714 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.5 MiB/s wr, 454 op/s 2024-03-21T11:47:49.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:49 smithi100 ceph-mon[36245]: pgmap v1135: 129 pgs: 129 active+clean; 714 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.5 MiB/s wr, 454 op/s 2024-03-21T11:47:49.996 INFO:tasks.workunit.client.0.smithi047.stdout: 10 264999 21.71 MB/sec execute 513 sec latency 95.986 ms 2024-03-21T11:47:50.997 INFO:tasks.workunit.client.0.smithi047.stdout: 10 265650 21.73 MB/sec execute 514 sec latency 71.843 ms 2024-03-21T11:47:51.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:51 smithi102 ceph-mon[36748]: pgmap v1136: 129 pgs: 129 active+clean; 719 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 465 op/s 2024-03-21T11:47:51.563 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:51 smithi047 ceph-mon[31351]: pgmap v1136: 129 pgs: 129 active+clean; 719 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 465 op/s 2024-03-21T11:47:51.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:51 smithi100 ceph-mon[36245]: pgmap v1136: 129 pgs: 129 active+clean; 719 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 465 op/s 2024-03-21T11:47:51.997 INFO:tasks.workunit.client.0.smithi047.stdout: 10 266222 21.76 MB/sec execute 515 sec latency 69.818 ms 2024-03-21T11:47:52.997 INFO:tasks.workunit.client.0.smithi047.stdout: 10 266625 21.76 MB/sec execute 516 sec latency 76.686 ms 2024-03-21T11:47:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:53 smithi102 ceph-mon[36748]: pgmap v1137: 129 pgs: 129 active+clean; 722 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.5 MiB/s wr, 446 op/s 2024-03-21T11:47:53.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:53 smithi047 ceph-mon[31351]: pgmap v1137: 129 pgs: 129 active+clean; 722 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.5 MiB/s wr, 446 op/s 2024-03-21T11:47:53.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:53 smithi100 ceph-mon[36245]: pgmap v1137: 129 pgs: 129 active+clean; 722 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.5 MiB/s wr, 446 op/s 2024-03-21T11:47:53.997 INFO:tasks.workunit.client.0.smithi047.stdout: 10 266990 21.76 MB/sec execute 517 sec latency 68.131 ms 2024-03-21T11:47:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.812+0000 7f2791c00640 1 -- 172.21.15.47:0/3817379322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f278c104910 msgr2=0x7f278c104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.812+0000 7f2791c00640 1 --2- 172.21.15.47:0/3817379322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f278c104910 0x7f278c104d90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f2778009900 tx=0x7f277802bc30 comp rx=0 tx=0).stop 2024-03-21T11:47:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.813+0000 7f2791c00640 1 -- 172.21.15.47:0/3817379322 shutdown_connections 2024-03-21T11:47:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.813+0000 7f2791c00640 1 --2- 172.21.15.47:0/3817379322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f278c1052d0 0x7f278c10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.813+0000 7f2791c00640 1 --2- 172.21.15.47:0/3817379322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f278c104910 0x7f278c104d90 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.813+0000 7f2791c00640 1 --2- 172.21.15.47:0/3817379322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f278c103720 0x7f278c103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.813+0000 7f2791c00640 1 -- 172.21.15.47:0/3817379322 >> 172.21.15.47:0/3817379322 conn(0x7f278c0fecb0 msgr2=0x7f278c1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.814+0000 7f2791c00640 1 -- 172.21.15.47:0/3817379322 shutdown_connections 2024-03-21T11:47:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.814+0000 7f2791c00640 1 -- 172.21.15.47:0/3817379322 wait complete. 2024-03-21T11:47:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.817+0000 7f2791c00640 1 Processor -- start 2024-03-21T11:47:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.820+0000 7f2791c00640 1 -- start start 2024-03-21T11:47:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.820+0000 7f2791c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f278c103720 0x7f278c1a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.820+0000 7f2791c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f278c104910 0x7f278c1a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.820+0000 7f2791c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f278c1052d0 0x7f278c1adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.820+0000 7f2791c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c10f460 con 0x7f278c1052d0 2024-03-21T11:47:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.820+0000 7f2791c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c10f2e0 con 0x7f278c104910 2024-03-21T11:47:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.820+0000 7f2791c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c10f5e0 con 0x7f278c103720 2024-03-21T11:47:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.821+0000 7f2791200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f278c1052d0 0x7f278c1adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.821+0000 7f2791200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f278c1052d0 0x7f278c1adca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38838/0 (socket says 172.21.15.47:38838) 2024-03-21T11:47:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.821+0000 7f2791200640 1 -- 172.21.15.47:0/1064346410 learned_addr learned my addr 172.21.15.47:0/1064346410 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.821+0000 7f278be00640 1 --2- 172.21.15.47:0/1064346410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f278c103720 0x7f278c1a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.821+0000 7f2791200640 1 -- 172.21.15.47:0/1064346410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f278c103720 msgr2=0x7f278c1a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.821+0000 7f2791200640 1 --2- 172.21.15.47:0/1064346410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f278c103720 0x7f278c1a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.821+0000 7f2791200640 1 -- 172.21.15.47:0/1064346410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f278c104910 msgr2=0x7f278c1a6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.821+0000 7f2791200640 1 --2- 172.21.15.47:0/1064346410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f278c104910 0x7f278c1a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.821+0000 7f2791200640 1 -- 172.21.15.47:0/1064346410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c1ae310 con 0x7f278c1052d0 2024-03-21T11:47:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.822+0000 7f2791200640 1 --2- 172.21.15.47:0/1064346410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f278c1052d0 0x7f278c1adca0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f278400eaa0 tx=0x7f278400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.822+0000 7f2788c00640 1 -- 172.21.15.47:0/1064346410 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278400cdb0 con 0x7f278c1052d0 2024-03-21T11:47:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.822+0000 7f2791c00640 1 -- 172.21.15.47:0/1064346410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c1ae600 con 0x7f278c1052d0 2024-03-21T11:47:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.822+0000 7f2788c00640 1 -- 172.21.15.47:0/1064346410 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2784005020 con 0x7f278c1052d0 2024-03-21T11:47:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.824+0000 7f2791c00640 1 -- 172.21.15.47:0/1064346410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c1aeaf0 con 0x7f278c1052d0 2024-03-21T11:47:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.824+0000 7f2788c00640 1 -- 172.21.15.47:0/1064346410 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27840116b0 con 0x7f278c1052d0 2024-03-21T11:47:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.829+0000 7f2795fa0740 1 Processor -- start 2024-03-21T11:47:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.830+0000 7f2788c00640 1 -- 172.21.15.47:0/1064346410 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2784011850 con 0x7f278c1052d0 2024-03-21T11:47:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.830+0000 7f2795fa0740 1 -- start start 2024-03-21T11:47:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.831+0000 7f2795fa0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ed7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.831+0000 7f2795fa0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 0x55ff92edc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.831+0000 7f2795fa0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92edcf50 0x55ff92edf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.831+0000 7f2795fa0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92e9a3b0 con 0x55ff92ed7de0 2024-03-21T11:47:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.831+0000 7f2795fa0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92e3c830 con 0x55ff92edcf50 2024-03-21T11:47:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.831+0000 7f2795fa0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92de55d0 con 0x55ff92ed73f0 2024-03-21T11:47:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.832+0000 7f278b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92edcf50 0x55ff92edf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.832+0000 7f278b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ed7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.832+0000 7f278b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ed7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54108/0 (socket says 172.21.15.47:54108) 2024-03-21T11:47:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.832+0000 7f278b400640 1 -- 172.21.15.47:0/3740570600 learned_addr learned my addr 172.21.15.47:0/3740570600 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.833+0000 7f278be00640 1 --2- 172.21.15.47:0/3740570600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 0x55ff92edc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.834+0000 7f2788c00640 1 --2- 172.21.15.47:0/1064346410 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f27700a3050 0x7f27700a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.834+0000 7f2788c00640 1 -- 172.21.15.47:0/1064346410 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f27840cfb40 con 0x7f278c1052d0 2024-03-21T11:47:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.834+0000 7f278b400640 1 -- 172.21.15.47:0/3740570600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92edcf50 msgr2=0x55ff92edf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.834+0000 7f278b400640 1 --2- 172.21.15.47:0/3740570600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92edcf50 0x55ff92edf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.834+0000 7f278b400640 1 -- 172.21.15.47:0/3740570600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 msgr2=0x55ff92edc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.834+0000 7f278b400640 1 --2- 172.21.15.47:0/3740570600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 0x55ff92edc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.834+0000 7f2791200640 1 --2- 172.21.15.47:0/1064346410 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f27700a3050 0x7f27700a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.835+0000 7f278b400640 1 -- 172.21.15.47:0/3740570600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff92dfd2c0 con 0x55ff92ed73f0 2024-03-21T11:47:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.835+0000 7f278be00640 1 --2- 172.21.15.47:0/3740570600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 0x55ff92edc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.835+0000 7f278b400640 1 --2- 172.21.15.47:0/3740570600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ed7810 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f2778009230 tx=0x7f2778000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bad4dafcedaf9ec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.837+0000 7f2791200640 1 --2- 172.21.15.47:0/1064346410 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f27700a3050 0x7f27700a5510 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2784005510 tx=0x7f2784015000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.837+0000 7f2781600640 1 -- 172.21.15.47:0/3740570600 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778048070 con 0x55ff92ed73f0 2024-03-21T11:47:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.837+0000 7f2781600640 1 -- 172.21.15.47:0/3740570600 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2778004ba0 con 0x55ff92ed73f0 2024-03-21T11:47:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.837+0000 7f2781600640 1 -- 172.21.15.47:0/3740570600 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778005240 con 0x55ff92ed73f0 2024-03-21T11:47:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.839+0000 7f2795fa0740 1 -- 172.21.15.47:0/3740570600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 msgr2=0x55ff92ed7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.839+0000 7f2795fa0740 1 --2- 172.21.15.47:0/3740570600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ed7810 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f2778009230 tx=0x7f2778000bc0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.839+0000 7f2795fa0740 1 -- 172.21.15.47:0/3740570600 shutdown_connections 2024-03-21T11:47:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.840+0000 7f2795fa0740 1 --2- 172.21.15.47:0/3740570600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92edcf50 0x55ff92edf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.840+0000 7f2795fa0740 1 --2- 172.21.15.47:0/3740570600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 0x55ff92edc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.840+0000 7f2795fa0740 1 --2- 172.21.15.47:0/3740570600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ed7810 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.840+0000 7f2795fa0740 1 -- 172.21.15.47:0/3740570600 >> 172.21.15.47:0/3740570600 conn(0x55ff92ed2960 msgr2=0x55ff92ed2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.841+0000 7f2795fa0740 1 -- 172.21.15.47:0/3740570600 shutdown_connections 2024-03-21T11:47:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.841+0000 7f2795fa0740 1 -- 172.21.15.47:0/3740570600 wait complete. 2024-03-21T11:47:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.842+0000 7f2795fa0740 1 -- start start 2024-03-21T11:47:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.842+0000 7f2795fa0740 1 Processor -- start 2024-03-21T11:47:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.843+0000 7f2795fa0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ee1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.843+0000 7f2795fa0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92ed7de0 0x55ff92ee1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.843+0000 7f2795fa0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92edcf50 0x55ff92ee8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.843+0000 7f278b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ee1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.844+0000 7f278b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ee1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54112/0 (socket says 172.21.15.47:54112) 2024-03-21T11:47:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.844+0000 7f278b400640 1 -- 172.21.15.47:0/679858106 learned_addr learned my addr 172.21.15.47:0/679858106 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.844+0000 7f2795fa0740 1 -- 172.21.15.47:0/679858106 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92df8a20 con 0x55ff92edcf50 2024-03-21T11:47:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.844+0000 7f2795fa0740 1 -- 172.21.15.47:0/679858106 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92e388d0 con 0x55ff92ed7de0 2024-03-21T11:47:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.844+0000 7f2795fa0740 1 -- 172.21.15.47:0/679858106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92edfea0 con 0x55ff92ed73f0 2024-03-21T11:47:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.844+0000 7f2795fa0740 10 client.0 _ll_register_callbacks cb 0x55ff92e351c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.844+0000 7f278b400640 1 --2- 172.21.15.47:0/679858106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92edcf50 0x55ff92ee8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.845+0000 7f278b400640 1 -- 172.21.15.47:0/679858106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92ed7de0 msgr2=0x55ff92ee1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.845+0000 7f278b400640 1 --2- 172.21.15.47:0/679858106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92ed7de0 0x55ff92ee1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.845+0000 7f278b400640 1 -- 172.21.15.47:0/679858106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92edcf50 msgr2=0x55ff92ee8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.845+0000 7f278b400640 1 --2- 172.21.15.47:0/679858106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92edcf50 0x55ff92ee8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.845+0000 7f278b400640 1 -- 172.21.15.47:0/679858106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff92b51210 con 0x55ff92ed73f0 2024-03-21T11:47:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.854+0000 7f278b400640 1 --2- 172.21.15.47:0/679858106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ee1580 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f277802c240 tx=0x7f277802c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d54df4338c825cac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.855+0000 7f2776000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.859+0000 7f2776000640 1 -- 172.21.15.47:0/679858106 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778048030 con 0x55ff92ed73f0 2024-03-21T11:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.859+0000 7f2776000640 1 -- 172.21.15.47:0/679858106 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f27780052c0 con 0x55ff92ed73f0 2024-03-21T11:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.859+0000 7f2795fa0740 1 -- 172.21.15.47:0/679858106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2778004ba0 con 0x55ff92ed73f0 2024-03-21T11:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.860+0000 7f2776000640 1 -- 172.21.15.47:0/679858106 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27780054c0 con 0x55ff92ed73f0 2024-03-21T11:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.860+0000 7f2795fa0740 10 client.33175 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.860+0000 7f2795fa0740 1 -- 172.21.15.47:0/679858106 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ff92c1d620 con 0x55ff92ed73f0 2024-03-21T11:47:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.861+0000 7f2776000640 1 -- 172.21.15.47:0/679858106 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2778004ba0 con 0x55ff92ed73f0 2024-03-21T11:47:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.861+0000 7f2776000640 1 -- 172.21.15.47:0/679858106 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2778034440 con 0x55ff92ed73f0 2024-03-21T11:47:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.861+0000 7f2795fa0740 10 client.33175 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.861+0000 7f2795fa0740 10 client.33175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.861+0000 7f2795fa0740 20 client.33175 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.861+0000 7f2795fa0740 1 --2- 172.21.15.47:0/679858106 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff92eecfd0 0x55ff92eef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.861+0000 7f2795fa0740 4 client.33175 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.861+0000 7f2795fa0740 1 -- 172.21.15.47:0/679858106 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff92e49d70 con 0x55ff92eecfd0 2024-03-21T11:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.862+0000 7f278be00640 1 --2- 172.21.15.47:0/679858106 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff92eecfd0 0x55ff92eef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.864+0000 7f278be00640 1 --2- 172.21.15.47:0/679858106 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff92eecfd0 0x55ff92eef3c0 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=395d772368fe248 server_cookie=ac3b55f52625f5ad in_seq=0 out_seq=0 2024-03-21T11:47:54.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.864+0000 7f2776000640 10 client.33175 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.869+0000 7f2776000640 1 -- 172.21.15.47:0/679858106 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ff92e49d70 con 0x55ff92eecfd0 2024-03-21T11:47:54.870 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.870+0000 7f2776000640 10 client.33175 handle_command_reply: tid=0 2024-03-21T11:47:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.870+0000 7f2795fa0740 1 client.33175 shutdown 2024-03-21T11:47:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.872+0000 7f2795fa0740 1 -- 172.21.15.47:0/679858106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 msgr2=0x55ff92ee1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.872+0000 7f2795fa0740 1 --2- 172.21.15.47:0/679858106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ee1580 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f277802c240 tx=0x7f277802c660 comp rx=0 tx=0).stop 2024-03-21T11:47:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.872+0000 7f2795fa0740 1 -- 172.21.15.47:0/679858106 shutdown_connections 2024-03-21T11:47:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.872+0000 7f2795fa0740 1 --2- 172.21.15.47:0/679858106 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff92eecfd0 0x55ff92eef3c0 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.872+0000 7f2795fa0740 1 --2- 172.21.15.47:0/679858106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92edcf50 0x55ff92ee8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.872+0000 7f2795fa0740 1 --2- 172.21.15.47:0/679858106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92ed7de0 0x55ff92ee1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.872+0000 7f2795fa0740 1 --2- 172.21.15.47:0/679858106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ee1580 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.872+0000 7f2795fa0740 1 -- 172.21.15.47:0/679858106 >> 172.21.15.47:0/679858106 conn(0x55ff92ed2960 msgr2=0x55ff92ed1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.873+0000 7f2776000640 0 client.33175 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.874+0000 7f2795fa0740 1 -- 172.21.15.47:0/679858106 shutdown_connections 2024-03-21T11:47:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.874+0000 7f2795fa0740 1 -- 172.21.15.47:0/679858106 wait complete. 2024-03-21T11:47:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.876+0000 7f2795fa0740 20 client.33175 trim_cache size 0 max 16384 2024-03-21T11:47:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.907+0000 7f2795fa0740 1 Processor -- start 2024-03-21T11:47:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.907+0000 7f2795fa0740 1 -- start start 2024-03-21T11:47:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.908+0000 7f2795fa0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92ed73f0 0x55ff92ed5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.908+0000 7f2795fa0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 0x55ff92ee8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.908+0000 7f2795fa0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92edcf50 0x55ff92ed6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.908+0000 7f2795fa0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1b8e30 con 0x55ff92ed7de0 2024-03-21T11:47:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.908+0000 7f2795fa0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1ba7b0 con 0x55ff92ed73f0 2024-03-21T11:47:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.908+0000 7f2795fa0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1b9490 con 0x55ff92edcf50 2024-03-21T11:47:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.908+0000 7f278be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 0x55ff92ee8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.908+0000 7f278be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 0x55ff92ee8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38866/0 (socket says 172.21.15.47:38866) 2024-03-21T11:47:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.908+0000 7f278be00640 1 -- 172.21.15.47:0/3168461226 learned_addr learned my addr 172.21.15.47:0/3168461226 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.909+0000 7f278b400640 1 --2- 172.21.15.47:0/3168461226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92ed73f0 0x55ff92ed5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.909+0000 7f278b400640 1 --2- 172.21.15.47:0/3168461226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92edcf50 0x55ff92ed6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.909+0000 7f278b400640 1 -- 172.21.15.47:0/3168461226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92edcf50 msgr2=0x55ff92ed6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.909+0000 7f278b400640 1 --2- 172.21.15.47:0/3168461226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92edcf50 0x55ff92ed6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.909+0000 7f278b400640 1 -- 172.21.15.47:0/3168461226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 msgr2=0x55ff92ee8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.909+0000 7f278b400640 1 --2- 172.21.15.47:0/3168461226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 0x55ff92ee8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.909+0000 7f278b400640 1 -- 172.21.15.47:0/3168461226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff92ed8bf0 con 0x55ff92ed73f0 2024-03-21T11:47:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.910+0000 7f278be00640 1 --2- 172.21.15.47:0/3168461226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 0x55ff92ee8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.910+0000 7f278b400640 1 --2- 172.21.15.47:0/3168461226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92ed73f0 0x55ff92ed5bc0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f27780092a0 tx=0x7f2778000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6346dac0746d6b39 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.911+0000 7f2780c00640 1 -- 172.21.15.47:0/3168461226 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778048030 con 0x55ff92ed73f0 2024-03-21T11:47:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.911+0000 7f2780c00640 1 -- 172.21.15.47:0/3168461226 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2778049040 con 0x55ff92ed73f0 2024-03-21T11:47:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.911+0000 7f2780c00640 1 -- 172.21.15.47:0/3168461226 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778034440 con 0x55ff92ed73f0 2024-03-21T11:47:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.912+0000 7f2795fa0740 1 -- 172.21.15.47:0/3168461226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92ed73f0 msgr2=0x55ff92ed5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.912+0000 7f2795fa0740 1 --2- 172.21.15.47:0/3168461226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92ed73f0 0x55ff92ed5bc0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f27780092a0 tx=0x7f2778000bc0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.912+0000 7f2795fa0740 1 -- 172.21.15.47:0/3168461226 shutdown_connections 2024-03-21T11:47:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.912+0000 7f2795fa0740 1 --2- 172.21.15.47:0/3168461226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92edcf50 0x55ff92ed6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.912+0000 7f2795fa0740 1 --2- 172.21.15.47:0/3168461226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 0x55ff92ee8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.912+0000 7f2795fa0740 1 --2- 172.21.15.47:0/3168461226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92ed73f0 0x55ff92ed5bc0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.912+0000 7f2795fa0740 1 -- 172.21.15.47:0/3168461226 >> 172.21.15.47:0/3168461226 conn(0x55ff92ed2960 msgr2=0x55ff92ed2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.912+0000 7f2795fa0740 1 -- 172.21.15.47:0/3168461226 shutdown_connections 2024-03-21T11:47:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.912+0000 7f2795fa0740 1 -- 172.21.15.47:0/3168461226 wait complete. 2024-03-21T11:47:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.913+0000 7f2795fa0740 1 -- start start 2024-03-21T11:47:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.913+0000 7f2795fa0740 1 Processor -- start 2024-03-21T11:47:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.914+0000 7f2795fa0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ee6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.914+0000 7f2795fa0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 0x55ff92ee6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.914+0000 7f2795fa0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92edcf50 0x55ff92ee7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.914+0000 7f2795fa0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1ba150 con 0x55ff92ed7de0 2024-03-21T11:47:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.914+0000 7f2795fa0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1b9af0 con 0x55ff92edcf50 2024-03-21T11:47:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.914+0000 7f2795fa0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1b6ff0 con 0x55ff92ed73f0 2024-03-21T11:47:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.914+0000 7f2795fa0740 10 client.0 _ll_register_callbacks cb 0x7f2778046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.914+0000 7f278b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ee6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.915+0000 7f278b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ee6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54128/0 (socket says 172.21.15.47:54128) 2024-03-21T11:47:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.915+0000 7f278b400640 1 -- 172.21.15.47:0/3198370692 learned_addr learned my addr 172.21.15.47:0/3198370692 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.915+0000 7f278b400640 1 --2- 172.21.15.47:0/3198370692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92edcf50 0x55ff92ee7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.915+0000 7f278b400640 1 -- 172.21.15.47:0/3198370692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92edcf50 msgr2=0x55ff92ee7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.915+0000 7f278b400640 1 --2- 172.21.15.47:0/3198370692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92edcf50 0x55ff92ee7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.916 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.915+0000 7f278b400640 1 -- 172.21.15.47:0/3198370692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 msgr2=0x55ff92ee6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.915+0000 7f278b400640 1 --2- 172.21.15.47:0/3198370692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 0x55ff92ee6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.915+0000 7f278b400640 1 -- 172.21.15.47:0/3198370692 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff92ee83a0 con 0x55ff92ed73f0 2024-03-21T11:47:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.916+0000 7f278b400640 1 --2- 172.21.15.47:0/3198370692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ee6860 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f2778034b70 tx=0x7f277802c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eff3c72d9b67a3eb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.916+0000 7f2776000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:47:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.917+0000 7f2776000640 1 -- 172.21.15.47:0/3198370692 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778048040 con 0x55ff92ed73f0 2024-03-21T11:47:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.917+0000 7f2776000640 1 -- 172.21.15.47:0/3198370692 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2778034640 con 0x55ff92ed73f0 2024-03-21T11:47:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.917+0000 7f2795fa0740 1 -- 172.21.15.47:0/3198370692 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2778049040 con 0x55ff92ed73f0 2024-03-21T11:47:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.918+0000 7f2776000640 1 -- 172.21.15.47:0/3198370692 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2778009780 con 0x55ff92ed73f0 2024-03-21T11:47:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.918+0000 7f2795fa0740 10 client.33181 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.918+0000 7f2795fa0740 1 -- 172.21.15.47:0/3198370692 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ff92f48d20 con 0x55ff92ed73f0 2024-03-21T11:47:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.919+0000 7f2776000640 1 -- 172.21.15.47:0/3198370692 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2778049040 con 0x55ff92ed73f0 2024-03-21T11:47:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.919+0000 7f2776000640 1 -- 172.21.15.47:0/3198370692 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2778003b20 con 0x55ff92ed73f0 2024-03-21T11:47:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.919+0000 7f2795fa0740 10 client.33181 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.919+0000 7f2795fa0740 10 client.33181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.919+0000 7f2795fa0740 20 client.33181 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.919+0000 7f2795fa0740 1 --2- 172.21.15.47:0/3198370692 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff92f49310 0x55ff92f4b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.919+0000 7f2795fa0740 4 client.33181 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:47:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.919+0000 7f2795fa0740 1 -- 172.21.15.47:0/3198370692 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ff92e440c0 con 0x55ff92f49310 2024-03-21T11:47:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.920+0000 7f278be00640 1 --2- 172.21.15.47:0/3198370692 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff92f49310 0x55ff92f4b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.920+0000 7f278be00640 1 --2- 172.21.15.47:0/3198370692 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff92f49310 0x55ff92f4b730 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ffcab828aec838 server_cookie=b331d1efcad74460 in_seq=0 out_seq=0 2024-03-21T11:47:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:54.920+0000 7f2776000640 10 client.33181 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:54.999 INFO:tasks.workunit.client.0.smithi047.stdout: 10 267414 21.75 MB/sec execute 518 sec latency 65.488 ms 2024-03-21T11:47:55.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:47:55.079 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:47:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.446+0000 7f754f200640 1 -- 172.21.15.47:0/3258400195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7548103260 msgr2=0x7f7548109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:55.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.446+0000 7f754f200640 1 --2- 172.21.15.47:0/3258400195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7548103260 0x7f7548109af0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7540009be0 tx=0x7f7540034490 comp rx=0 tx=0).stop 2024-03-21T11:47:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.446+0000 7f754f200640 1 -- 172.21.15.47:0/3258400195 shutdown_connections 2024-03-21T11:47:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.446+0000 7f754f200640 1 --2- 172.21.15.47:0/3258400195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7548103260 0x7f7548109af0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.446+0000 7f754f200640 1 --2- 172.21.15.47:0/3258400195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75481028a0 0x7f7548102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.446+0000 7f754f200640 1 --2- 172.21.15.47:0/3258400195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75481016b0 0x7f7548101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.446+0000 7f754f200640 1 -- 172.21.15.47:0/3258400195 >> 172.21.15.47:0/3258400195 conn(0x7f75480fcc20 msgr2=0x7f75480ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:55.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.447+0000 7f754f200640 1 -- 172.21.15.47:0/3258400195 shutdown_connections 2024-03-21T11:47:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.447+0000 7f754f200640 1 -- 172.21.15.47:0/3258400195 wait complete. 2024-03-21T11:47:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.448+0000 7f754f200640 1 Processor -- start 2024-03-21T11:47:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.449+0000 7f754f200640 1 -- start start 2024-03-21T11:47:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.449+0000 7f754f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75481016b0 0x7f75481957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.449+0000 7f754f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75481028a0 0x7f7548195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.449+0000 7f754f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7548103260 0x7f754819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.449+0000 7f754f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754810b1b0 con 0x7f75481028a0 2024-03-21T11:47:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.449+0000 7f754f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754810b030 con 0x7f7548103260 2024-03-21T11:47:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.449+0000 7f754f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754810b330 con 0x7f75481016b0 2024-03-21T11:47:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.449+0000 7f7547e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75481016b0 0x7f75481957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.449+0000 7f7547e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75481016b0 0x7f75481957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54154/0 (socket says 172.21.15.47:54154) 2024-03-21T11:47:55.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.449+0000 7f7547e00640 1 -- 172.21.15.47:0/1124488208 learned_addr learned my addr 172.21.15.47:0/1124488208 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:55.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.449+0000 7f754d200640 1 --2- 172.21.15.47:0/1124488208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7548103260 0x7f754819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.450+0000 7f7547400640 1 --2- 172.21.15.47:0/1124488208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75481028a0 0x7f7548195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.450+0000 7f754d200640 1 -- 172.21.15.47:0/1124488208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75481016b0 msgr2=0x7f75481957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:55.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.450+0000 7f754d200640 1 --2- 172.21.15.47:0/1124488208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75481016b0 0x7f75481957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.450+0000 7f754d200640 1 -- 172.21.15.47:0/1124488208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75481028a0 msgr2=0x7f7548195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:55.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.450+0000 7f754d200640 1 --2- 172.21.15.47:0/1124488208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75481028a0 0x7f7548195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.450+0000 7f754d200640 1 -- 172.21.15.47:0/1124488208 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754819d410 con 0x7f7548103260 2024-03-21T11:47:55.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.451+0000 7f754d200640 1 --2- 172.21.15.47:0/1124488208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7548103260 0x7f754819cda0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f754003e040 tx=0x7f754003a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:55.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.452+0000 7f7544c00640 1 -- 172.21.15.47:0/1124488208 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754004b020 con 0x7f7548103260 2024-03-21T11:47:55.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.452+0000 7f7544c00640 1 -- 172.21.15.47:0/1124488208 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f754003a720 con 0x7f7548103260 2024-03-21T11:47:55.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.452+0000 7f754f200640 1 -- 172.21.15.47:0/1124488208 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754819d6a0 con 0x7f7548103260 2024-03-21T11:47:55.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.452+0000 7f7544c00640 1 -- 172.21.15.47:0/1124488208 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540040040 con 0x7f7548103260 2024-03-21T11:47:55.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.452+0000 7f754f200640 1 -- 172.21.15.47:0/1124488208 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754819dc30 con 0x7f7548103260 2024-03-21T11:47:55.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.457+0000 7f7551fe5740 1 Processor -- start 2024-03-21T11:47:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.459+0000 7f7544c00640 1 -- 172.21.15.47:0/1124488208 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f754004e020 con 0x7f7548103260 2024-03-21T11:47:55.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.460+0000 7f7551fe5740 1 -- start start 2024-03-21T11:47:55.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.461+0000 7f7544c00640 1 --2- 172.21.15.47:0/1124488208 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f752c0a2fa0 0x7f752c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.462+0000 7f7544c00640 1 -- 172.21.15.47:0/1124488208 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f75400fa450 con 0x7f7548103260 2024-03-21T11:47:55.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.462+0000 7f7547400640 1 --2- 172.21.15.47:0/1124488208 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f752c0a2fa0 0x7f752c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.463+0000 7f7547400640 1 --2- 172.21.15.47:0/1124488208 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f752c0a2fa0 0x7f752c0a5460 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7548196d00 tx=0x7f75340090e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.463+0000 7f7551fe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37144e80 0x561d371452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.463+0000 7f7551fe5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37145870 0x561d3714a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.463+0000 7f7551fe5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d3714a9e0 0x561d3714ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.464+0000 7f7551fe5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d370aa250 con 0x561d3714a9e0 2024-03-21T11:47:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.464+0000 7f7551fe5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d37052a30 con 0x561d37145870 2024-03-21T11:47:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.464+0000 7f7551fe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d370a62f0 con 0x561d37144e80 2024-03-21T11:47:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.464+0000 7f7547e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37144e80 0x561d371452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.464+0000 7f7547e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37144e80 0x561d371452a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54162/0 (socket says 172.21.15.47:54162) 2024-03-21T11:47:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.464+0000 7f7547e00640 1 -- 172.21.15.47:0/1620608605 learned_addr learned my addr 172.21.15.47:0/1620608605 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.464+0000 7f7547400640 1 --2- 172.21.15.47:0/1620608605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37145870 0x561d3714a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.464+0000 7f754d200640 1 --2- 172.21.15.47:0/1620608605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d3714a9e0 0x561d3714ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.465+0000 7f7547400640 1 -- 172.21.15.47:0/1620608605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37144e80 msgr2=0x561d371452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.465+0000 7f7547400640 1 --2- 172.21.15.47:0/1620608605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37144e80 0x561d371452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.465+0000 7f7547400640 1 -- 172.21.15.47:0/1620608605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d3714a9e0 msgr2=0x561d3714ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.465+0000 7f7547400640 1 --2- 172.21.15.47:0/1620608605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d3714a9e0 0x561d3714ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.465+0000 7f7547400640 1 -- 172.21.15.47:0/1620608605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d3703d690 con 0x561d37145870 2024-03-21T11:47:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.466+0000 7f7547400640 1 --2- 172.21.15.47:0/1620608605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37145870 0x561d3714a370 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f753400eb40 tx=0x7f7534011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cd98f33ab2d6d77 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.466+0000 7f753d600640 1 -- 172.21.15.47:0/1620608605 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753401b040 con 0x561d37145870 2024-03-21T11:47:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.466+0000 7f753d600640 1 -- 172.21.15.47:0/1620608605 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7534024070 con 0x561d37145870 2024-03-21T11:47:55.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.466+0000 7f753d600640 1 -- 172.21.15.47:0/1620608605 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753401f3f0 con 0x561d37145870 2024-03-21T11:47:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.468+0000 7f7551fe5740 1 -- 172.21.15.47:0/1620608605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37145870 msgr2=0x561d3714a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.468+0000 7f7551fe5740 1 --2- 172.21.15.47:0/1620608605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37145870 0x561d3714a370 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f753400eb40 tx=0x7f7534011000 comp rx=0 tx=0).stop 2024-03-21T11:47:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.468+0000 7f7551fe5740 1 -- 172.21.15.47:0/1620608605 shutdown_connections 2024-03-21T11:47:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.468+0000 7f7551fe5740 1 --2- 172.21.15.47:0/1620608605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d3714a9e0 0x561d3714ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.468+0000 7f7551fe5740 1 --2- 172.21.15.47:0/1620608605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37145870 0x561d3714a370 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.468+0000 7f7551fe5740 1 --2- 172.21.15.47:0/1620608605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37144e80 0x561d371452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.468+0000 7f7551fe5740 1 -- 172.21.15.47:0/1620608605 >> 172.21.15.47:0/1620608605 conn(0x561d3710d200 msgr2=0x561d371407f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.468+0000 7f7551fe5740 1 -- 172.21.15.47:0/1620608605 shutdown_connections 2024-03-21T11:47:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.468+0000 7f7551fe5740 1 -- 172.21.15.47:0/1620608605 wait complete. 2024-03-21T11:47:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.469+0000 7f7551fe5740 1 -- start start 2024-03-21T11:47:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.469+0000 7f7551fe5740 1 Processor -- start 2024-03-21T11:47:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.470+0000 7f7551fe5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d37144e80 0x561d3714ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.470+0000 7f7551fe5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37145870 0x561d3714f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.470+0000 7f7551fe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d3714a9e0 0x561d37156410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.470+0000 7f7551fe5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d3714d990 con 0x561d37144e80 2024-03-21T11:47:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.470+0000 7f7551fe5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d37065e80 con 0x561d37145870 2024-03-21T11:47:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.470+0000 7f7551fe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d3714db10 con 0x561d3714a9e0 2024-03-21T11:47:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.470+0000 7f754d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d3714a9e0 0x561d37156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.470+0000 7f7547e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d37144e80 0x561d3714ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.470+0000 7f754d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d3714a9e0 0x561d37156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54168/0 (socket says 172.21.15.47:54168) 2024-03-21T11:47:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.470+0000 7f754d200640 1 -- 172.21.15.47:0/3482912886 learned_addr learned my addr 172.21.15.47:0/3482912886 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.470+0000 7f7547400640 1 --2- 172.21.15.47:0/3482912886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37145870 0x561d3714f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.470+0000 7f7551fe5740 10 client.0 _ll_register_callbacks cb 0x561d37078130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.471+0000 7f754d200640 1 -- 172.21.15.47:0/3482912886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37145870 msgr2=0x561d3714f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.471+0000 7f754d200640 1 --2- 172.21.15.47:0/3482912886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37145870 0x561d3714f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.471+0000 7f754d200640 1 -- 172.21.15.47:0/3482912886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d37144e80 msgr2=0x561d3714ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.471+0000 7f754d200640 1 --2- 172.21.15.47:0/3482912886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d37144e80 0x561d3714ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.471+0000 7f754d200640 1 -- 172.21.15.47:0/3482912886 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d36e887d0 con 0x561d3714a9e0 2024-03-21T11:47:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.471+0000 7f754d200640 1 --2- 172.21.15.47:0/3482912886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d3714a9e0 0x561d37156410 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f75400f7f30 tx=0x7f75400c6210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bb4298cd358e07b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.471+0000 7f7532000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:47:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.472+0000 7f7532000640 1 -- 172.21.15.47:0/3482912886 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75400df060 con 0x561d3714a9e0 2024-03-21T11:47:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.472+0000 7f7532000640 1 -- 172.21.15.47:0/3482912886 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f75400b10c0 con 0x561d3714a9e0 2024-03-21T11:47:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.472+0000 7f7551fe5740 1 -- 172.21.15.47:0/3482912886 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7534024070 con 0x561d3714a9e0 2024-03-21T11:47:55.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.472+0000 7f7532000640 1 -- 172.21.15.47:0/3482912886 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75400e4050 con 0x561d3714a9e0 2024-03-21T11:47:55.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.472+0000 7f7551fe5740 10 client.33193 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:55.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.472+0000 7f7551fe5740 1 -- 172.21.15.47:0/3482912886 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561d36dbe3f0 con 0x561d3714a9e0 2024-03-21T11:47:55.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.473+0000 7f7532000640 1 -- 172.21.15.47:0/3482912886 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7534024070 con 0x561d3714a9e0 2024-03-21T11:47:55.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.473+0000 7f7532000640 1 -- 172.21.15.47:0/3482912886 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f75400c6ae0 con 0x561d3714a9e0 2024-03-21T11:47:55.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.473+0000 7f7551fe5740 10 client.33193 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:55.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.473+0000 7f7551fe5740 10 client.33193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:55.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.473+0000 7f7551fe5740 20 client.33193 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:55.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.473+0000 7f7551fe5740 1 --2- 172.21.15.47:0/3482912886 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d3715ad10 0x561d3715d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.473+0000 7f7551fe5740 4 client.33193 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:55.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.473+0000 7f7551fe5740 1 -- 172.21.15.47:0/3482912886 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d370b7790 con 0x561d3715ad10 2024-03-21T11:47:55.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.475+0000 7f7547e00640 1 --2- 172.21.15.47:0/3482912886 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d3715ad10 0x561d3715d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.476+0000 7f7547e00640 1 --2- 172.21.15.47:0/3482912886 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d3715ad10 0x561d3715d100 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66d6befb9e0d249 server_cookie=4b52c591a107e46b in_seq=0 out_seq=0 2024-03-21T11:47:55.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.476+0000 7f7532000640 10 client.33193 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:55.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:55 smithi102 ceph-mon[36748]: pgmap v1138: 129 pgs: 129 active+clean; 731 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.1 MiB/s wr, 510 op/s 2024-03-21T11:47:55.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:55 smithi047 ceph-mon[31351]: pgmap v1138: 129 pgs: 129 active+clean; 731 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.1 MiB/s wr, 510 op/s 2024-03-21T11:47:55.645 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:55 smithi100 ceph-mon[36245]: pgmap v1138: 129 pgs: 129 active+clean; 731 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.1 MiB/s wr, 510 op/s 2024-03-21T11:47:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.953+0000 7f2776000640 1 -- 172.21.15.47:0/3198370692 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ff92e440c0 con 0x55ff92f49310 2024-03-21T11:47:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.953+0000 7f2776000640 10 client.33181 handle_command_reply: tid=0 2024-03-21T11:47:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.953+0000 7f2795fa0740 1 client.33181 shutdown 2024-03-21T11:47:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.953+0000 7f7532000640 1 -- 172.21.15.47:0/3482912886 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561d370b7790 con 0x561d3715ad10 2024-03-21T11:47:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.953+0000 7f7532000640 10 client.33193 handle_command_reply: tid=0 2024-03-21T11:47:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f2795fa0740 1 -- 172.21.15.47:0/3198370692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 msgr2=0x55ff92ee6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f2795fa0740 1 --2- 172.21.15.47:0/3198370692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ee6860 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f2778034b70 tx=0x7f277802c880 comp rx=0 tx=0).stop 2024-03-21T11:47:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f2795fa0740 1 -- 172.21.15.47:0/3198370692 shutdown_connections 2024-03-21T11:47:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f2795fa0740 1 --2- 172.21.15.47:0/3198370692 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ff92f49310 0x55ff92f4b730 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f2795fa0740 1 --2- 172.21.15.47:0/3198370692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff92edcf50 0x55ff92ee7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f2795fa0740 1 --2- 172.21.15.47:0/3198370692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff92ed7de0 0x55ff92ee6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f2795fa0740 1 --2- 172.21.15.47:0/3198370692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ff92ed73f0 0x55ff92ee6860 secure :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f2778034b70 tx=0x7f277802c880 comp rx=0 tx=0).stop 2024-03-21T11:47:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f2795fa0740 1 -- 172.21.15.47:0/3198370692 >> 172.21.15.47:0/3198370692 conn(0x55ff92ed2960 msgr2=0x55ff92ed5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f2776000640 0 client.33181 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f2795fa0740 1 -- 172.21.15.47:0/3198370692 shutdown_connections 2024-03-21T11:47:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f2795fa0740 1 -- 172.21.15.47:0/3198370692 wait complete. 2024-03-21T11:47:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f7551fe5740 1 client.33193 shutdown 2024-03-21T11:47:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f7551fe5740 1 -- 172.21.15.47:0/3482912886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d3714a9e0 msgr2=0x561d37156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f7551fe5740 1 --2- 172.21.15.47:0/3482912886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d3714a9e0 0x561d37156410 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f75400f7f30 tx=0x7f75400c6210 comp rx=0 tx=0).stop 2024-03-21T11:47:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f7551fe5740 1 -- 172.21.15.47:0/3482912886 shutdown_connections 2024-03-21T11:47:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f7551fe5740 1 --2- 172.21.15.47:0/3482912886 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d3715ad10 0x561d3715d100 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f7551fe5740 1 --2- 172.21.15.47:0/3482912886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d3714a9e0 0x561d37156410 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f7551fe5740 1 --2- 172.21.15.47:0/3482912886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37145870 0x561d3714f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f7551fe5740 1 --2- 172.21.15.47:0/3482912886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d37144e80 0x561d3714ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.954+0000 7f7551fe5740 1 -- 172.21.15.47:0/3482912886 >> 172.21.15.47:0/3482912886 conn(0x561d3710d200 msgr2=0x561d3713fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.955+0000 7f7532000640 0 client.33193 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.955+0000 7f2795fa0740 20 client.33181 trim_cache size 0 max 16384 2024-03-21T11:47:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:47:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:47:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:47:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:47:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.955+0000 7f7551fe5740 1 -- 172.21.15.47:0/3482912886 shutdown_connections 2024-03-21T11:47:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.955+0000 7f7551fe5740 1 -- 172.21.15.47:0/3482912886 wait complete. 2024-03-21T11:47:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.956+0000 7f7551fe5740 20 client.33193 trim_cache size 0 max 16384 2024-03-21T11:47:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.956+0000 7f2791c00640 1 -- 172.21.15.47:0/1064346410 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f27700a3050 msgr2=0x7f27700a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.956+0000 7f2791c00640 1 --2- 172.21.15.47:0/1064346410 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f27700a3050 0x7f27700a5510 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2784005510 tx=0x7f2784015000 comp rx=0 tx=0).stop 2024-03-21T11:47:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.956+0000 7f2791c00640 1 -- 172.21.15.47:0/1064346410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f278c1052d0 msgr2=0x7f278c1adca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.956+0000 7f2791c00640 1 --2- 172.21.15.47:0/1064346410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f278c1052d0 0x7f278c1adca0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f278400eaa0 tx=0x7f278400ef60 comp rx=0 tx=0).stop 2024-03-21T11:47:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.957+0000 7f2791c00640 1 -- 172.21.15.47:0/1064346410 shutdown_connections 2024-03-21T11:47:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.957+0000 7f2791c00640 1 --2- 172.21.15.47:0/1064346410 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f27700a3050 0x7f27700a5510 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.957+0000 7f2791c00640 1 --2- 172.21.15.47:0/1064346410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f278c1052d0 0x7f278c1adca0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.957+0000 7f2791c00640 1 --2- 172.21.15.47:0/1064346410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f278c104910 0x7f278c1a6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.957+0000 7f2791c00640 1 --2- 172.21.15.47:0/1064346410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f278c103720 0x7f278c1a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.957+0000 7f2791c00640 1 -- 172.21.15.47:0/1064346410 >> 172.21.15.47:0/1064346410 conn(0x7f278c0fecb0 msgr2=0x7f278c077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.957+0000 7f2791c00640 1 -- 172.21.15.47:0/1064346410 shutdown_connections 2024-03-21T11:47:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:47:55.957+0000 7f2791c00640 1 -- 172.21.15.47:0/1064346410 wait complete. 2024-03-21T11:47:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.985+0000 7f7551fe5740 1 Processor -- start 2024-03-21T11:47:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.985+0000 7f7551fe5740 1 -- start start 2024-03-21T11:47:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.985+0000 7f7551fe5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37144e80 0x561d37143a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.985+0000 7f7551fe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37145870 0x561d371563d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.985+0000 7f7551fe5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d3714a9e0 0x561d37143f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.985+0000 7f7551fe5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75481a7f30 con 0x561d3714a9e0 2024-03-21T11:47:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.985+0000 7f7551fe5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75481a98b0 con 0x561d37144e80 2024-03-21T11:47:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.985+0000 7f7551fe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75481a8590 con 0x561d37145870 2024-03-21T11:47:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.985+0000 7f7547e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37144e80 0x561d37143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.985+0000 7f7547400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37145870 0x561d371563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.985+0000 7f7547e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37144e80 0x561d37143a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39968/0 (socket says 172.21.15.47:39968) 2024-03-21T11:47:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.985+0000 7f7547e00640 1 -- 172.21.15.47:0/148569795 learned_addr learned my addr 172.21.15.47:0/148569795 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.985+0000 7f754d200640 1 --2- 172.21.15.47:0/148569795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d3714a9e0 0x561d37143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.986+0000 7f7547e00640 1 -- 172.21.15.47:0/148569795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37145870 msgr2=0x561d371563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.986+0000 7f7547e00640 1 --2- 172.21.15.47:0/148569795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37145870 0x561d371563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.986+0000 7f7547e00640 1 -- 172.21.15.47:0/148569795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d3714a9e0 msgr2=0x561d37143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.986+0000 7f7547e00640 1 --2- 172.21.15.47:0/148569795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d3714a9e0 0x561d37143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.986+0000 7f7547e00640 1 -- 172.21.15.47:0/148569795 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d36d7e0f0 con 0x561d37144e80 2024-03-21T11:47:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.986+0000 7f7547400640 1 --2- 172.21.15.47:0/148569795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37145870 0x561d371563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.986+0000 7f7547e00640 1 --2- 172.21.15.47:0/148569795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37144e80 0x561d37143a30 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7538005dc0 tx=0x7f7538009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2b4cdab4d854ca3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.986+0000 7f753cc00640 1 -- 172.21.15.47:0/148569795 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7538004230 con 0x561d37144e80 2024-03-21T11:47:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.987+0000 7f753cc00640 1 -- 172.21.15.47:0/148569795 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7538010e30 con 0x561d37144e80 2024-03-21T11:47:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.987+0000 7f753cc00640 1 -- 172.21.15.47:0/148569795 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7538003ae0 con 0x561d37144e80 2024-03-21T11:47:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.987+0000 7f7551fe5740 1 -- 172.21.15.47:0/148569795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37144e80 msgr2=0x561d37143a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.987+0000 7f7551fe5740 1 --2- 172.21.15.47:0/148569795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37144e80 0x561d37143a30 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7538005dc0 tx=0x7f7538009260 comp rx=0 tx=0).stop 2024-03-21T11:47:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.987+0000 7f7551fe5740 1 -- 172.21.15.47:0/148569795 shutdown_connections 2024-03-21T11:47:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.987+0000 7f7551fe5740 1 --2- 172.21.15.47:0/148569795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d3714a9e0 0x561d37143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.987+0000 7f7551fe5740 1 --2- 172.21.15.47:0/148569795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37145870 0x561d371563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.987+0000 7f7551fe5740 1 --2- 172.21.15.47:0/148569795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d37144e80 0x561d37143a30 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.987+0000 7f7551fe5740 1 -- 172.21.15.47:0/148569795 >> 172.21.15.47:0/148569795 conn(0x561d3710d200 msgr2=0x561d37146ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.987+0000 7f7551fe5740 1 -- 172.21.15.47:0/148569795 shutdown_connections 2024-03-21T11:47:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.987+0000 7f7551fe5740 1 -- 172.21.15.47:0/148569795 wait complete. 2024-03-21T11:47:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.988+0000 7f7551fe5740 1 -- start start 2024-03-21T11:47:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.988+0000 7f7551fe5740 1 Processor -- start 2024-03-21T11:47:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f7551fe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37144e80 0x561d37154370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f7551fe5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d37145870 0x561d371548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f7551fe5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d3714a9e0 0x561d37155850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f7551fe5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75481a9250 con 0x561d37145870 2024-03-21T11:47:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f7551fe5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75481a8bf0 con 0x561d3714a9e0 2024-03-21T11:47:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f7551fe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75481a60f0 con 0x561d37144e80 2024-03-21T11:47:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f7551fe5740 10 client.0 _ll_register_callbacks cb 0x7f7540005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f7547e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37144e80 0x561d37154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f7547e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37144e80 0x561d37154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54188/0 (socket says 172.21.15.47:54188) 2024-03-21T11:47:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f7547e00640 1 -- 172.21.15.47:0/2761219678 learned_addr learned my addr 172.21.15.47:0/2761219678 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f754d200640 1 --2- 172.21.15.47:0/2761219678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d3714a9e0 0x561d37155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f7547400640 1 --2- 172.21.15.47:0/2761219678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d37145870 0x561d371548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f754d200640 1 -- 172.21.15.47:0/2761219678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37144e80 msgr2=0x561d37154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f754d200640 1 --2- 172.21.15.47:0/2761219678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37144e80 0x561d37154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f754d200640 1 -- 172.21.15.47:0/2761219678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d37145870 msgr2=0x561d371548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f754d200640 1 --2- 172.21.15.47:0/2761219678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d37145870 0x561d371548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.989+0000 7f754d200640 1 -- 172.21.15.47:0/2761219678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d37155e40 con 0x561d3714a9e0 2024-03-21T11:47:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.990+0000 7f754d200640 1 --2- 172.21.15.47:0/2761219678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d3714a9e0 0x561d37155850 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f7540006010 tx=0x7f75400c6280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a245b98e161e00b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.990+0000 7f7532000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.990+0000 7f7532000640 1 -- 172.21.15.47:0/2761219678 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75400c6330 con 0x561d3714a9e0 2024-03-21T11:47:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.990+0000 7f7532000640 1 -- 172.21.15.47:0/2761219678 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f75400fa190 con 0x561d3714a9e0 2024-03-21T11:47:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.990+0000 7f7551fe5740 1 -- 172.21.15.47:0/2761219678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7538010e30 con 0x561d3714a9e0 2024-03-21T11:47:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.990+0000 7f7532000640 1 -- 172.21.15.47:0/2761219678 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75400d4050 con 0x561d3714a9e0 2024-03-21T11:47:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.990+0000 7f7551fe5740 10 client.33239 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.990+0000 7f7551fe5740 1 -- 172.21.15.47:0/2761219678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561d371b6e10 con 0x561d3714a9e0 2024-03-21T11:47:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.991+0000 7f7532000640 1 -- 172.21.15.47:0/2761219678 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7538010e30 con 0x561d3714a9e0 2024-03-21T11:47:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.991+0000 7f7532000640 1 -- 172.21.15.47:0/2761219678 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7540004b40 con 0x561d3714a9e0 2024-03-21T11:47:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.991+0000 7f7551fe5740 10 client.33239 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.991+0000 7f7551fe5740 10 client.33239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.991+0000 7f7551fe5740 20 client.33239 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.991+0000 7f7551fe5740 1 --2- 172.21.15.47:0/2761219678 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d371b7540 0x561d371b9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.991+0000 7f7551fe5740 4 client.33239 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:47:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.991+0000 7f7551fe5740 1 -- 172.21.15.47:0/2761219678 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561d3714ec80 con 0x561d371b7540 2024-03-21T11:47:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.992+0000 7f7547e00640 1 --2- 172.21.15.47:0/2761219678 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d371b7540 0x561d371b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.993+0000 7f7547e00640 1 --2- 172.21.15.47:0/2761219678 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d371b7540 0x561d371b9960 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=912a8bbfbbd205f3 server_cookie=7b1d687d22fa334f in_seq=0 out_seq=0 2024-03-21T11:47:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.993+0000 7f7532000640 10 client.33239 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.995+0000 7f7532000640 1 -- 172.21.15.47:0/2761219678 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561d3714ec80 con 0x561d371b7540 2024-03-21T11:47:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.995+0000 7f7532000640 10 client.33239 handle_command_reply: tid=0 2024-03-21T11:47:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.995+0000 7f7551fe5740 1 client.33239 shutdown 2024-03-21T11:47:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.995+0000 7f7551fe5740 1 -- 172.21.15.47:0/2761219678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d3714a9e0 msgr2=0x561d37155850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.995+0000 7f7551fe5740 1 --2- 172.21.15.47:0/2761219678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d3714a9e0 0x561d37155850 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f7540006010 tx=0x7f75400c6280 comp rx=0 tx=0).stop 2024-03-21T11:47:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.995+0000 7f7551fe5740 1 -- 172.21.15.47:0/2761219678 shutdown_connections 2024-03-21T11:47:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.995+0000 7f7551fe5740 1 --2- 172.21.15.47:0/2761219678 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561d371b7540 0x561d371b9960 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.995+0000 7f7551fe5740 1 --2- 172.21.15.47:0/2761219678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d3714a9e0 0x561d37155850 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.995+0000 7f7551fe5740 1 --2- 172.21.15.47:0/2761219678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d37145870 0x561d371548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.995+0000 7f7551fe5740 1 --2- 172.21.15.47:0/2761219678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561d37144e80 0x561d37154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.995+0000 7f7532000640 0 client.33239 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.995+0000 7f7551fe5740 1 -- 172.21.15.47:0/2761219678 >> 172.21.15.47:0/2761219678 conn(0x561d3710d200 msgr2=0x561d3713f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.995+0000 7f7551fe5740 1 -- 172.21.15.47:0/2761219678 shutdown_connections 2024-03-21T11:47:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.996+0000 7f7551fe5740 1 -- 172.21.15.47:0/2761219678 wait complete. 2024-03-21T11:47:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.996+0000 7f7551fe5740 20 client.33239 trim_cache size 0 max 16384 2024-03-21T11:47:55.998 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:47:55.998 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:47:55.998 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:47:55.998 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:47:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.996+0000 7f754f200640 1 -- 172.21.15.47:0/1124488208 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f752c0a2fa0 msgr2=0x7f752c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.996+0000 7f754f200640 1 --2- 172.21.15.47:0/1124488208 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f752c0a2fa0 0x7f752c0a5460 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7548196d00 tx=0x7f75340090e0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.996+0000 7f754f200640 1 -- 172.21.15.47:0/1124488208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7548103260 msgr2=0x7f754819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.996+0000 7f754f200640 1 --2- 172.21.15.47:0/1124488208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7548103260 0x7f754819cda0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f754003e040 tx=0x7f754003a5b0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.997+0000 7f754f200640 1 -- 172.21.15.47:0/1124488208 shutdown_connections 2024-03-21T11:47:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.997+0000 7f754f200640 1 --2- 172.21.15.47:0/1124488208 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f752c0a2fa0 0x7f752c0a5460 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.997+0000 7f754f200640 1 --2- 172.21.15.47:0/1124488208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7548103260 0x7f754819cda0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.997+0000 7f754f200640 1 --2- 172.21.15.47:0/1124488208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75481028a0 0x7f7548195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.997+0000 7f754f200640 1 --2- 172.21.15.47:0/1124488208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75481016b0 0x7f75481957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.997+0000 7f754f200640 1 -- 172.21.15.47:0/1124488208 >> 172.21.15.47:0/1124488208 conn(0x7f75480fcc20 msgr2=0x7f7548105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.997+0000 7f754f200640 1 -- 172.21.15.47:0/1124488208 shutdown_connections 2024-03-21T11:47:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:55.997+0000 7f754f200640 1 -- 172.21.15.47:0/1124488208 wait complete. 2024-03-21T11:47:56.000 INFO:tasks.workunit.client.0.smithi047.stdout: 10 267507 21.71 MB/sec execute 519 sec latency 875.379 ms 2024-03-21T11:47:56.016 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:47:56.017 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:47:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.332+0000 7fad77200640 1 -- 172.21.15.47:0/3620770991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad781052b0 msgr2=0x7fad7810bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:56.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.332+0000 7fad77200640 1 --2- 172.21.15.47:0/3620770991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad781052b0 0x7fad7810bb40 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fad68009860 tx=0x7fad6802bb70 comp rx=0 tx=0).stop 2024-03-21T11:47:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.333+0000 7fad77200640 1 -- 172.21.15.47:0/3620770991 shutdown_connections 2024-03-21T11:47:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.333+0000 7fad77200640 1 --2- 172.21.15.47:0/3620770991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad781052b0 0x7fad7810bb40 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.333+0000 7fad77200640 1 --2- 172.21.15.47:0/3620770991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad781048f0 0x7fad78104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.333+0000 7fad77200640 1 --2- 172.21.15.47:0/3620770991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad78103700 0x7fad78103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:56.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.333+0000 7fad77200640 1 -- 172.21.15.47:0/3620770991 >> 172.21.15.47:0/3620770991 conn(0x7fad780fec70 msgr2=0x7fad781010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:56.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.333+0000 7fad77200640 1 -- 172.21.15.47:0/3620770991 shutdown_connections 2024-03-21T11:47:56.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.334+0000 7fad77200640 1 -- 172.21.15.47:0/3620770991 wait complete. 2024-03-21T11:47:56.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.335+0000 7fad77200640 1 Processor -- start 2024-03-21T11:47:56.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.335+0000 7fad77200640 1 -- start start 2024-03-21T11:47:56.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.336+0000 7fad77200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad78103700 0x7fad7806d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:56.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.336+0000 7fad77200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad781048f0 0x7fad7806dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:56.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.336+0000 7fad77200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad781052b0 0x7fad78072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:56.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.336+0000 7fad77200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78076a40 con 0x7fad78103700 2024-03-21T11:47:56.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.336+0000 7fad77200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad780768c0 con 0x7fad781052b0 2024-03-21T11:47:56.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.336+0000 7fad77200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78076bc0 con 0x7fad781048f0 2024-03-21T11:47:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.337+0000 7fad76800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad781052b0 0x7fad78072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.337+0000 7fad76800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad781052b0 0x7fad78072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40004/0 (socket says 172.21.15.47:40004) 2024-03-21T11:47:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.337+0000 7fad76800640 1 -- 172.21.15.47:0/2489576940 learned_addr learned my addr 172.21.15.47:0/2489576940 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.337+0000 7fad75e00640 1 --2- 172.21.15.47:0/2489576940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad78103700 0x7fad7806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.337+0000 7fad76800640 1 -- 172.21.15.47:0/2489576940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad781048f0 msgr2=0x7fad7806dad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:47:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.337+0000 7fad76800640 1 --2- 172.21.15.47:0/2489576940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad781048f0 0x7fad7806dad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.337+0000 7fad76800640 1 -- 172.21.15.47:0/2489576940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad78103700 msgr2=0x7fad7806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.337+0000 7fad76800640 1 --2- 172.21.15.47:0/2489576940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad78103700 0x7fad7806d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.337+0000 7fad76800640 1 -- 172.21.15.47:0/2489576940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad78073150 con 0x7fad781052b0 2024-03-21T11:47:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.337+0000 7fad75e00640 1 --2- 172.21.15.47:0/2489576940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad78103700 0x7fad7806d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:47:56.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.338+0000 7fad76800640 1 --2- 172.21.15.47:0/2489576940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad781052b0 0x7fad78072ae0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fad68009780 tx=0x7fad68004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:56.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.340+0000 7fad6ea00640 1 -- 172.21.15.47:0/2489576940 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68041020 con 0x7fad781052b0 2024-03-21T11:47:56.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.340+0000 7fad6ea00640 1 -- 172.21.15.47:0/2489576940 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad680347b0 con 0x7fad781052b0 2024-03-21T11:47:56.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.340+0000 7fad6ea00640 1 -- 172.21.15.47:0/2489576940 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68039040 con 0x7fad781052b0 2024-03-21T11:47:56.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.341+0000 7fad77200640 1 -- 172.21.15.47:0/2489576940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad780733e0 con 0x7fad781052b0 2024-03-21T11:47:56.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.341+0000 7fad77200640 1 -- 172.21.15.47:0/2489576940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7810e420 con 0x7fad781052b0 2024-03-21T11:47:56.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.343+0000 7fad77200640 1 -- 172.21.15.47:0/2489576940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad78108230 con 0x7fad781052b0 2024-03-21T11:47:56.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.346+0000 7fad6ea00640 1 -- 172.21.15.47:0/2489576940 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fad68044020 con 0x7fad781052b0 2024-03-21T11:47:56.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.348+0000 7fad6ea00640 1 --2- 172.21.15.47:0/2489576940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad5c0a2ff0 0x7fad5c0a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:56.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.349+0000 7fad6ea00640 1 -- 172.21.15.47:0/2489576940 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fad680f11a0 con 0x7fad781052b0 2024-03-21T11:47:56.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.349+0000 7fad75e00640 1 --2- 172.21.15.47:0/2489576940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad5c0a2ff0 0x7fad5c0a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:56.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.351+0000 7fad75e00640 1 --2- 172.21.15.47:0/2489576940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad5c0a2ff0 0x7fad5c0a54b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fad78104750 tx=0x7fad64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:56.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.355+0000 7fad6ea00640 1 -- 172.21.15.47:0/2489576940 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fad6804b020 con 0x7fad781052b0 2024-03-21T11:47:56.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.772+0000 7fad77200640 1 -- 172.21.15.47:0/2489576940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad7802d290 con 0x7fad781052b0 2024-03-21T11:47:56.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.773+0000 7fad6ea00640 1 -- 172.21.15.47:0/2489576940 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fad680bdc40 con 0x7fad781052b0 2024-03-21T11:47:56.774 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:47:56.774 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:45:39.125075+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_32623":{"gid":32623,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4278877119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4278877119},{"type":"v1","addr":"172.21.15.100:6835","nonce":4278877119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24523,24524,24535,24553,24581]},"id":1}]} 2024-03-21T11:47:56.775 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 41 2024-03-21T11:47:56.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.775+0000 7fad77200640 1 -- 172.21.15.47:0/2489576940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad5c0a2ff0 msgr2=0x7fad5c0a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:56.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.775+0000 7fad77200640 1 --2- 172.21.15.47:0/2489576940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad5c0a2ff0 0x7fad5c0a54b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fad78104750 tx=0x7fad64009260 comp rx=0 tx=0).stop 2024-03-21T11:47:56.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.775+0000 7fad77200640 1 -- 172.21.15.47:0/2489576940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad781052b0 msgr2=0x7fad78072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:56.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.775+0000 7fad77200640 1 --2- 172.21.15.47:0/2489576940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad781052b0 0x7fad78072ae0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fad68009780 tx=0x7fad68004270 comp rx=0 tx=0).stop 2024-03-21T11:47:56.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.775+0000 7fad77200640 1 -- 172.21.15.47:0/2489576940 shutdown_connections 2024-03-21T11:47:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.775+0000 7fad77200640 1 --2- 172.21.15.47:0/2489576940 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fad5c0a2ff0 0x7fad5c0a54b0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.775+0000 7fad77200640 1 --2- 172.21.15.47:0/2489576940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad781052b0 0x7fad78072ae0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.775+0000 7fad77200640 1 --2- 172.21.15.47:0/2489576940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad781048f0 0x7fad7806dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.775+0000 7fad77200640 1 --2- 172.21.15.47:0/2489576940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad78103700 0x7fad7806d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.775+0000 7fad77200640 1 -- 172.21.15.47:0/2489576940 >> 172.21.15.47:0/2489576940 conn(0x7fad780fec70 msgr2=0x7fad78079490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.776+0000 7fad77200640 1 -- 172.21.15.47:0/2489576940 shutdown_connections 2024-03-21T11:47:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:56.776+0000 7fad77200640 1 -- 172.21.15.47:0/2489576940 wait complete. 2024-03-21T11:47:56.801 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:47:56.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:47:56.998 INFO:tasks.workunit.client.0.smithi047.stdout: 10 267832 21.70 MB/sec execute 520 sec latency 324.763 ms 2024-03-21T11:47:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.114+0000 7f84e0800640 1 -- 172.21.15.47:0/1741932616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d80756e0 msgr2=0x7f84d8073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.114+0000 7f84e0800640 1 --2- 172.21.15.47:0/1741932616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d80756e0 0x7f84d8073b30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009cd0 tx=0x7f84d402c0e0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.114+0000 7f84e0800640 1 -- 172.21.15.47:0/1741932616 shutdown_connections 2024-03-21T11:47:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.114+0000 7f84e0800640 1 --2- 172.21.15.47:0/1741932616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84d810dc80 0x7f84d8110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.114+0000 7f84e0800640 1 --2- 172.21.15.47:0/1741932616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84d8074100 0x7f84d8074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.114+0000 7f84e0800640 1 --2- 172.21.15.47:0/1741932616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d80756e0 0x7f84d8073b30 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.114+0000 7f84e0800640 1 -- 172.21.15.47:0/1741932616 >> 172.21.15.47:0/1741932616 conn(0x7f84d80fcc40 msgr2=0x7f84d80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.114+0000 7f84e0800640 1 -- 172.21.15.47:0/1741932616 shutdown_connections 2024-03-21T11:47:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.114+0000 7f84e0800640 1 -- 172.21.15.47:0/1741932616 wait complete. 2024-03-21T11:47:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.115+0000 7f84e0800640 1 Processor -- start 2024-03-21T11:47:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.116+0000 7f84e0800640 1 -- start start 2024-03-21T11:47:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.116+0000 7f84e0800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84d8074100 0x7f84d819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.116+0000 7f84e0800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84d80756e0 0x7f84d819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.116+0000 7f84e0800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d810dc80 0x7f84d81a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.116+0000 7f84e0800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81139d0 con 0x7f84d80756e0 2024-03-21T11:47:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.116+0000 7f84e0800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d8113850 con 0x7f84d810dc80 2024-03-21T11:47:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.116+0000 7f84e0800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d8113b50 con 0x7f84d8074100 2024-03-21T11:47:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.116+0000 7f84dde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84d8074100 0x7f84d819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.116+0000 7f84dde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84d8074100 0x7f84d819e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54204/0 (socket says 172.21.15.47:54204) 2024-03-21T11:47:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.116+0000 7f84dde00640 1 -- 172.21.15.47:0/4124821472 learned_addr learned my addr 172.21.15.47:0/4124821472 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.117+0000 7f84de800640 1 --2- 172.21.15.47:0/4124821472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d810dc80 0x7f84d81a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.117+0000 7f84dd400640 1 --2- 172.21.15.47:0/4124821472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84d80756e0 0x7f84d819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.117+0000 7f84dde00640 1 -- 172.21.15.47:0/4124821472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d810dc80 msgr2=0x7f84d81a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.117+0000 7f84dde00640 1 --2- 172.21.15.47:0/4124821472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d810dc80 0x7f84d81a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.117+0000 7f84dde00640 1 -- 172.21.15.47:0/4124821472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84d80756e0 msgr2=0x7f84d819e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.117+0000 7f84dde00640 1 --2- 172.21.15.47:0/4124821472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84d80756e0 0x7f84d819e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.117+0000 7f84dde00640 1 -- 172.21.15.47:0/4124821472 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d81a5cf0 con 0x7f84d8074100 2024-03-21T11:47:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.117+0000 7f84dde00640 1 --2- 172.21.15.47:0/4124821472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84d8074100 0x7f84d819e0d0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f84d40325d0 tx=0x7f84d4035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.118+0000 7f84d2a00640 1 -- 172.21.15.47:0/4124821472 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4039040 con 0x7f84d8074100 2024-03-21T11:47:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.118+0000 7f84d2a00640 1 -- 172.21.15.47:0/4124821472 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84d4044070 con 0x7f84d8074100 2024-03-21T11:47:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.118+0000 7f84d2a00640 1 -- 172.21.15.47:0/4124821472 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4004540 con 0x7f84d8074100 2024-03-21T11:47:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.118+0000 7f84e0800640 1 -- 172.21.15.47:0/4124821472 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d81a5f80 con 0x7f84d8074100 2024-03-21T11:47:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.118+0000 7f84e0800640 1 -- 172.21.15.47:0/4124821472 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d81a6410 con 0x7f84d8074100 2024-03-21T11:47:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.124+0000 7f84d2a00640 1 -- 172.21.15.47:0/4124821472 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f84d4032870 con 0x7f84d8074100 2024-03-21T11:47:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.124+0000 7f84e3445740 1 Processor -- start 2024-03-21T11:47:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.124+0000 7f84e3445740 1 -- start start 2024-03-21T11:47:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.124+0000 7f84e3445740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0b810 0x556c49d0bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.124+0000 7f84e3445740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d0ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.124+0000 7f84e3445740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0d3a0 0x556c49d13c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.124+0000 7f84e3445740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c49c702c0 con 0x556c49d0b810 2024-03-21T11:47:57.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.124+0000 7f84e3445740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c49c18c90 con 0x556c49d0c9e0 2024-03-21T11:47:57.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.124+0000 7f84e3445740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c49c6c360 con 0x556c49d0d3a0 2024-03-21T11:47:57.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.125+0000 7f84dde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d0ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.125+0000 7f84de800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0b810 0x556c49d0bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.125+0000 7f84de800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0b810 0x556c49d0bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38978/0 (socket says 172.21.15.47:38978) 2024-03-21T11:47:57.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.125+0000 7f84de800640 1 -- 172.21.15.47:0/2127544827 learned_addr learned my addr 172.21.15.47:0/2127544827 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.125+0000 7f84dd400640 1 --2- 172.21.15.47:0/2127544827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0d3a0 0x556c49d13c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.126+0000 7f84de800640 1 -- 172.21.15.47:0/2127544827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0d3a0 msgr2=0x556c49d13c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.126+0000 7f84de800640 1 --2- 172.21.15.47:0/2127544827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0d3a0 0x556c49d13c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.126+0000 7f84de800640 1 -- 172.21.15.47:0/2127544827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 msgr2=0x556c49d0ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.126+0000 7f84de800640 1 --2- 172.21.15.47:0/2127544827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d0ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.126+0000 7f84de800640 1 -- 172.21.15.47:0/2127544827 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c49c3d080 con 0x556c49d0b810 2024-03-21T11:47:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.126+0000 7f84dde00640 1 --2- 172.21.15.47:0/2127544827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d0ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.127+0000 7f84d2a00640 1 --2- 172.21.15.47:0/4124821472 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84c00a3070 0x7f84c00a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.127+0000 7f84de800640 1 --2- 172.21.15.47:0/2127544827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0b810 0x556c49d0bc30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f84c400b7a0 tx=0x7f84c400bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393f8b34909ad8b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.128+0000 7f84d2a00640 1 -- 172.21.15.47:0/4124821472 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f84d40f1930 con 0x7f84d8074100 2024-03-21T11:47:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.128+0000 7f84cb000640 1 -- 172.21.15.47:0/2127544827 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c400d950 con 0x556c49d0b810 2024-03-21T11:47:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.128+0000 7f84cb000640 1 -- 172.21.15.47:0/2127544827 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84c4012420 con 0x556c49d0b810 2024-03-21T11:47:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.128+0000 7f84cb000640 1 -- 172.21.15.47:0/2127544827 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c400d950 con 0x556c49d0b810 2024-03-21T11:47:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.129+0000 7f84e3445740 1 -- 172.21.15.47:0/2127544827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0b810 msgr2=0x556c49d0bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.129+0000 7f84e3445740 1 --2- 172.21.15.47:0/2127544827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0b810 0x556c49d0bc30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f84c400b7a0 tx=0x7f84c400bc60 comp rx=0 tx=0).stop 2024-03-21T11:47:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.130+0000 7f84e3445740 1 -- 172.21.15.47:0/2127544827 shutdown_connections 2024-03-21T11:47:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.130+0000 7f84e3445740 1 --2- 172.21.15.47:0/2127544827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0d3a0 0x556c49d13c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.130+0000 7f84e3445740 1 --2- 172.21.15.47:0/2127544827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d0ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.130+0000 7f84e3445740 1 --2- 172.21.15.47:0/2127544827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0b810 0x556c49d0bc30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.130+0000 7f84e3445740 1 -- 172.21.15.47:0/2127544827 >> 172.21.15.47:0/2127544827 conn(0x556c49d06c00 msgr2=0x556c49d07000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.130+0000 7f84dde00640 1 --2- 172.21.15.47:0/4124821472 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84c00a3070 0x7f84c00a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.131+0000 7f84e3445740 1 -- 172.21.15.47:0/2127544827 shutdown_connections 2024-03-21T11:47:57.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.131+0000 7f84e3445740 1 -- 172.21.15.47:0/2127544827 wait complete. 2024-03-21T11:47:57.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.132+0000 7f84e3445740 1 -- start start 2024-03-21T11:47:57.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.132+0000 7f84dde00640 1 --2- 172.21.15.47:0/4124821472 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84c00a3070 0x7f84c00a5530 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f84d40bdc00 tx=0x7f84d402c0e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.132+0000 7f84e3445740 1 Processor -- start 2024-03-21T11:47:57.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.133+0000 7f84e3445740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0b810 0x556c49d15a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.133+0000 7f84e3445740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d15fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.133+0000 7f84e3445740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 0x556c49d1d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.133+0000 7f84e3445740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c49d143b0 con 0x556c49d0d3a0 2024-03-21T11:47:57.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.133+0000 7f84e3445740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c49c2c0d0 con 0x556c49d0c9e0 2024-03-21T11:47:57.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.133+0000 7f84e3445740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c49d14530 con 0x556c49d0b810 2024-03-21T11:47:57.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.133+0000 7f84e3445740 10 client.0 _ll_register_callbacks cb 0x556c49c30e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:57.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.133+0000 7f84de800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0b810 0x556c49d15a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.133+0000 7f84dd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d15fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.133+0000 7f84dd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d15fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40040/0 (socket says 172.21.15.47:40040) 2024-03-21T11:47:57.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.133+0000 7f84dd400640 1 -- 172.21.15.47:0/2529848883 learned_addr learned my addr 172.21.15.47:0/2529848883 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.134+0000 7f84de800640 1 --2- 172.21.15.47:0/2529848883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 0x556c49d1d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.134+0000 7f84dd400640 1 -- 172.21.15.47:0/2529848883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0b810 msgr2=0x556c49d15a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.134+0000 7f84dd400640 1 --2- 172.21.15.47:0/2529848883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0b810 0x556c49d15a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.134+0000 7f84dd400640 1 -- 172.21.15.47:0/2529848883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 msgr2=0x556c49d1d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.134+0000 7f84dd400640 1 --2- 172.21.15.47:0/2529848883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 0x556c49d1d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.134+0000 7f84dd400640 1 -- 172.21.15.47:0/2529848883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c499843f0 con 0x556c49d0c9e0 2024-03-21T11:47:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.134+0000 7f84de800640 1 --2- 172.21.15.47:0/2529848883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 0x556c49d1d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.135+0000 7f84dd400640 1 --2- 172.21.15.47:0/2529848883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d15fd0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f84cc012420 tx=0x7f84cc0128e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9148df99497e8f71 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.135+0000 7f84bf400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.135+0000 7f84bf400640 1 -- 172.21.15.47:0/2529848883 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc004220 con 0x556c49d0c9e0 2024-03-21T11:47:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.135+0000 7f84bf400640 1 -- 172.21.15.47:0/2529848883 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f84cc0043c0 con 0x556c49d0c9e0 2024-03-21T11:47:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.136+0000 7f84e3445740 1 -- 172.21.15.47:0/2529848883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84c4012420 con 0x556c49d0c9e0 2024-03-21T11:47:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.136+0000 7f84bf400640 1 -- 172.21.15.47:0/2529848883 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84cc0106a0 con 0x556c49d0c9e0 2024-03-21T11:47:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.136+0000 7f84e3445740 10 client.33254 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.137+0000 7f84e3445740 1 -- 172.21.15.47:0/2529848883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556c49a4e7d0 con 0x556c49d0c9e0 2024-03-21T11:47:57.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.140+0000 7f84bf400640 1 -- 172.21.15.47:0/2529848883 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84c4012420 con 0x556c49d0c9e0 2024-03-21T11:47:57.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.140+0000 7f84bf400640 1 -- 172.21.15.47:0/2529848883 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f84cc00f040 con 0x556c49d0c9e0 2024-03-21T11:47:57.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.141+0000 7f84e3445740 10 client.33254 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:57.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.141+0000 7f84e3445740 10 client.33254 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:47:57.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.142+0000 7f84e3445740 20 client.33254 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:57.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.142+0000 7f84e3445740 1 --2- 172.21.15.47:0/2529848883 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x556c49d21480 0x556c49d238c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.142+0000 7f84e3445740 4 client.33254 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:57.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.142+0000 7f84e3445740 1 -- 172.21.15.47:0/2529848883 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c49c7d800 con 0x556c49d21480 2024-03-21T11:47:57.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.142+0000 7f84de800640 1 --2- 172.21.15.47:0/2529848883 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x556c49d21480 0x556c49d238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.145+0000 7f84de800640 1 --2- 172.21.15.47:0/2529848883 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x556c49d21480 0x556c49d238c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2fa9a2431511e48e server_cookie=ef09ec77ace3485f in_seq=0 out_seq=0 2024-03-21T11:47:57.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.145+0000 7f84bf400640 10 client.33254 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:47:57.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.147+0000 7f84bf400640 1 -- 172.21.15.47:0/2529848883 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556c49c7d800 con 0x556c49d21480 2024-03-21T11:47:57.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.147+0000 7f84bf400640 10 client.33254 handle_command_reply: tid=0 2024-03-21T11:47:57.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.147+0000 7f84e3445740 1 client.33254 shutdown 2024-03-21T11:47:57.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.149+0000 7f84e3445740 1 -- 172.21.15.47:0/2529848883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 msgr2=0x556c49d15fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.149+0000 7f84e3445740 1 --2- 172.21.15.47:0/2529848883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d15fd0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f84cc012420 tx=0x7f84cc0128e0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.150+0000 7f84e3445740 1 -- 172.21.15.47:0/2529848883 shutdown_connections 2024-03-21T11:47:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.150+0000 7f84e3445740 1 --2- 172.21.15.47:0/2529848883 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x556c49d21480 0x556c49d238c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.150+0000 7f84e3445740 1 --2- 172.21.15.47:0/2529848883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 0x556c49d1d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.150+0000 7f84e3445740 1 --2- 172.21.15.47:0/2529848883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d15fd0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.150+0000 7f84e3445740 1 --2- 172.21.15.47:0/2529848883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0b810 0x556c49d15a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.150+0000 7f84e3445740 1 -- 172.21.15.47:0/2529848883 >> 172.21.15.47:0/2529848883 conn(0x556c49d06c00 msgr2=0x556c49d05eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.150+0000 7f84bf400640 0 client.33254 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:47:57.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.150+0000 7f84e3445740 1 -- 172.21.15.47:0/2529848883 shutdown_connections 2024-03-21T11:47:57.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.150+0000 7f84e3445740 1 -- 172.21.15.47:0/2529848883 wait complete. 2024-03-21T11:47:57.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.151+0000 7f84e3445740 20 client.33254 trim_cache size 0 max 16384 2024-03-21T11:47:57.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.172+0000 7f84e3445740 1 Processor -- start 2024-03-21T11:47:57.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.173+0000 7f84e3445740 1 -- start start 2024-03-21T11:47:57.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.173+0000 7f84e3445740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0b810 0x556c49d0fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.173+0000 7f84e3445740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d1cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.173+0000 7f84e3445740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 0x556c49d0ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.173+0000 7f84e3445740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81b0980 con 0x556c49d0d3a0 2024-03-21T11:47:57.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.173+0000 7f84e3445740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81b2300 con 0x556c49d0c9e0 2024-03-21T11:47:57.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.174+0000 7f84de800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0b810 0x556c49d0fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.174+0000 7f84dd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d1cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.174+0000 7f84de800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0b810 0x556c49d0fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54234/0 (socket says 172.21.15.47:54234) 2024-03-21T11:47:57.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.174+0000 7f84de800640 1 -- 172.21.15.47:0/3839500407 learned_addr learned my addr 172.21.15.47:0/3839500407 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.174+0000 7f84e3445740 1 -- 172.21.15.47:0/3839500407 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81b0fe0 con 0x556c49d0b810 2024-03-21T11:47:57.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.175+0000 7f84de800640 1 --2- 172.21.15.47:0/3839500407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 0x556c49d0ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.176+0000 7f84de800640 1 -- 172.21.15.47:0/3839500407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 msgr2=0x556c49d1cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.176+0000 7f84de800640 1 --2- 172.21.15.47:0/3839500407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d1cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.176+0000 7f84de800640 1 -- 172.21.15.47:0/3839500407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 msgr2=0x556c49d0ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.176+0000 7f84de800640 1 --2- 172.21.15.47:0/3839500407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 0x556c49d0ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.176+0000 7f84de800640 1 -- 172.21.15.47:0/3839500407 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c49d07970 con 0x556c49d0b810 2024-03-21T11:47:57.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.177+0000 7f84de800640 1 --2- 172.21.15.47:0/3839500407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0b810 0x556c49d0fa80 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f84c400b6c0 tx=0x7f84c4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f0c8b08c4c72349 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.177+0000 7f84ca600640 1 -- 172.21.15.47:0/3839500407 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4012420 con 0x556c49d0b810 2024-03-21T11:47:57.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.177+0000 7f84ca600640 1 -- 172.21.15.47:0/3839500407 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84c400de30 con 0x556c49d0b810 2024-03-21T11:47:57.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.177+0000 7f84ca600640 1 -- 172.21.15.47:0/3839500407 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c402b5a0 con 0x556c49d0b810 2024-03-21T11:47:57.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.179+0000 7f84e3445740 1 -- 172.21.15.47:0/3839500407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0b810 msgr2=0x556c49d0fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.179+0000 7f84e3445740 1 --2- 172.21.15.47:0/3839500407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0b810 0x556c49d0fa80 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f84c400b6c0 tx=0x7f84c4013000 comp rx=0 tx=0).stop 2024-03-21T11:47:57.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.181+0000 7f84e3445740 1 -- 172.21.15.47:0/3839500407 shutdown_connections 2024-03-21T11:47:57.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.181+0000 7f84e3445740 1 --2- 172.21.15.47:0/3839500407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 0x556c49d0ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.182+0000 7f84e3445740 1 --2- 172.21.15.47:0/3839500407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0c9e0 0x556c49d1cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.182+0000 7f84e3445740 1 --2- 172.21.15.47:0/3839500407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0b810 0x556c49d0fa80 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.182+0000 7f84e3445740 1 -- 172.21.15.47:0/3839500407 >> 172.21.15.47:0/3839500407 conn(0x556c49d06c00 msgr2=0x556c49d0a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.182+0000 7f84e3445740 1 -- 172.21.15.47:0/3839500407 shutdown_connections 2024-03-21T11:47:57.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.182+0000 7f84e3445740 1 -- 172.21.15.47:0/3839500407 wait complete. 2024-03-21T11:47:57.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.183+0000 7f84e3445740 1 -- start start 2024-03-21T11:47:57.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.183+0000 7f84e3445740 1 Processor -- start 2024-03-21T11:47:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.187+0000 7f84e3445740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0b810 0x556c49d171f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.187+0000 7f84e3445740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0c9e0 0x556c49d17730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.187+0000 7f84e3445740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 0x556c49d1c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.187+0000 7f84e3445740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81b1ca0 con 0x556c49d0d3a0 2024-03-21T11:47:57.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.187+0000 7f84e3445740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81b1640 con 0x556c49d0b810 2024-03-21T11:47:57.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.187+0000 7f84e3445740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81aeb00 con 0x556c49d0c9e0 2024-03-21T11:47:57.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.187+0000 7f84e3445740 10 client.0 _ll_register_callbacks cb 0x7f84cc009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:57.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.187+0000 7f84de800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0b810 0x556c49d171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.187+0000 7f84dd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0c9e0 0x556c49d17730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.187+0000 7f84de800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0b810 0x556c49d171f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40054/0 (socket says 172.21.15.47:40054) 2024-03-21T11:47:57.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.187+0000 7f84de800640 1 -- 172.21.15.47:0/3790558560 learned_addr learned my addr 172.21.15.47:0/3790558560 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.188+0000 7f84de800640 1 -- 172.21.15.47:0/3790558560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0c9e0 msgr2=0x556c49d17730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.188+0000 7f84de800640 1 --2- 172.21.15.47:0/3790558560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0c9e0 0x556c49d17730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.188+0000 7f84de800640 1 -- 172.21.15.47:0/3790558560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 msgr2=0x556c49d1c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.188+0000 7f84de800640 1 --2- 172.21.15.47:0/3790558560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 0x556c49d1c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.188+0000 7f84de800640 1 -- 172.21.15.47:0/3790558560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c49d1ca50 con 0x556c49d0b810 2024-03-21T11:47:57.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.188+0000 7f84de800640 1 --2- 172.21.15.47:0/3790558560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0b810 0x556c49d171f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f84c400b370 tx=0x7f84c4012510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4823efbfbe139c73 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.188+0000 7f84bf400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:57.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.189+0000 7f84bf400640 1 -- 172.21.15.47:0/3790558560 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4038060 con 0x556c49d0b810 2024-03-21T11:47:57.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.189+0000 7f84bf400640 1 -- 172.21.15.47:0/3790558560 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f84c402b2b0 con 0x556c49d0b810 2024-03-21T11:47:57.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.189+0000 7f84e3445740 1 -- 172.21.15.47:0/3790558560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84c400de30 con 0x556c49d0b810 2024-03-21T11:47:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.190+0000 7f84bf400640 1 -- 172.21.15.47:0/3790558560 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84c4012650 con 0x556c49d0b810 2024-03-21T11:47:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.190+0000 7f84e3445740 10 client.33260 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.190+0000 7f84e3445740 1 -- 172.21.15.47:0/3790558560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556c49d8e560 con 0x556c49d0b810 2024-03-21T11:47:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.190+0000 7f84bf400640 1 -- 172.21.15.47:0/3790558560 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84c400de30 con 0x556c49d0b810 2024-03-21T11:47:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.191+0000 7f84bf400640 1 -- 172.21.15.47:0/3790558560 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f84c4013070 con 0x556c49d0b810 2024-03-21T11:47:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.191+0000 7f84e3445740 10 client.33260 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.191+0000 7f84e3445740 10 client.33260 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:47:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.191+0000 7f84e3445740 20 client.33260 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.191+0000 7f84e3445740 1 --2- 172.21.15.47:0/3790558560 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x556c49d8ecc0 0x556c49d910e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.191+0000 7f84e3445740 4 client.33260 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:47:57.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.191+0000 7f84e3445740 1 -- 172.21.15.47:0/3790558560 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556c49c77b50 con 0x556c49d8ecc0 2024-03-21T11:47:57.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.192+0000 7f84dd400640 1 --2- 172.21.15.47:0/3790558560 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x556c49d8ecc0 0x556c49d910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.195+0000 7f84dd400640 1 --2- 172.21.15.47:0/3790558560 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x556c49d8ecc0 0x556c49d910e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=44052a294e127738 server_cookie=ec46e6479458ebaa in_seq=0 out_seq=0 2024-03-21T11:47:57.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.195+0000 7f84bf400640 10 client.33260 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:47:57.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.196+0000 7f84bf400640 1 -- 172.21.15.47:0/3790558560 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556c49c77b50 con 0x556c49d8ecc0 2024-03-21T11:47:57.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.196+0000 7f84bf400640 10 client.33260 handle_command_reply: tid=0 2024-03-21T11:47:57.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.196+0000 7f84e3445740 1 client.33260 shutdown 2024-03-21T11:47:57.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.199+0000 7f84e3445740 1 -- 172.21.15.47:0/3790558560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0b810 msgr2=0x556c49d171f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.199+0000 7f84e3445740 1 --2- 172.21.15.47:0/3790558560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0b810 0x556c49d171f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f84c400b370 tx=0x7f84c4012510 comp rx=0 tx=0).stop 2024-03-21T11:47:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.199+0000 7f84e3445740 1 -- 172.21.15.47:0/3790558560 shutdown_connections 2024-03-21T11:47:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.199+0000 7f84e3445740 1 --2- 172.21.15.47:0/3790558560 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x556c49d8ecc0 0x556c49d910e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.199+0000 7f84e3445740 1 --2- 172.21.15.47:0/3790558560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c49d0d3a0 0x556c49d1c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.199+0000 7f84e3445740 1 --2- 172.21.15.47:0/3790558560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c49d0c9e0 0x556c49d17730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.199+0000 7f84e3445740 1 --2- 172.21.15.47:0/3790558560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c49d0b810 0x556c49d171f0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.199+0000 7f84e3445740 1 -- 172.21.15.47:0/3790558560 >> 172.21.15.47:0/3790558560 conn(0x556c49d06c00 msgr2=0x556c49d05620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.200+0000 7f84bf400640 0 client.33260 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:47:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.200+0000 7f84e3445740 1 -- 172.21.15.47:0/3790558560 shutdown_connections 2024-03-21T11:47:57.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.200+0000 7f84e3445740 1 -- 172.21.15.47:0/3790558560 wait complete. 2024-03-21T11:47:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.202+0000 7f84e3445740 20 client.33260 trim_cache size 0 max 16384 2024-03-21T11:47:57.204 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:47:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.204+0000 7f84e0800640 1 -- 172.21.15.47:0/4124821472 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84c00a3070 msgr2=0x7f84c00a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.204+0000 7f84e0800640 1 --2- 172.21.15.47:0/4124821472 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84c00a3070 0x7f84c00a5530 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f84d40bdc00 tx=0x7f84d402c0e0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.204+0000 7f84e0800640 1 -- 172.21.15.47:0/4124821472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84d8074100 msgr2=0x7f84d819e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.205+0000 7f84e0800640 1 --2- 172.21.15.47:0/4124821472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84d8074100 0x7f84d819e0d0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f84d40325d0 tx=0x7f84d4035d10 comp rx=0 tx=0).stop 2024-03-21T11:47:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.205+0000 7f84e0800640 1 -- 172.21.15.47:0/4124821472 shutdown_connections 2024-03-21T11:47:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.206+0000 7f84e0800640 1 --2- 172.21.15.47:0/4124821472 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84c00a3070 0x7f84c00a5530 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.206+0000 7f84e0800640 1 --2- 172.21.15.47:0/4124821472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d810dc80 0x7f84d81a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.206+0000 7f84e0800640 1 --2- 172.21.15.47:0/4124821472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84d80756e0 0x7f84d819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.206+0000 7f84e0800640 1 --2- 172.21.15.47:0/4124821472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84d8074100 0x7f84d819e0d0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.206+0000 7f84e0800640 1 -- 172.21.15.47:0/4124821472 >> 172.21.15.47:0/4124821472 conn(0x7f84d80fcc40 msgr2=0x7f84d8101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.207+0000 7f84e0800640 1 -- 172.21.15.47:0/4124821472 shutdown_connections 2024-03-21T11:47:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.207+0000 7f84e0800640 1 -- 172.21.15.47:0/4124821472 wait complete. 2024-03-21T11:47:57.236 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:47:57.236 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:47:57.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:57 smithi102 ceph-mon[36748]: pgmap v1139: 129 pgs: 129 active+clean; 733 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.7 MiB/s wr, 396 op/s 2024-03-21T11:47:57.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:57 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2489576940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:47:57.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.529+0000 7faf56000640 1 -- 172.21.15.47:0/4055321109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf50109870 msgr2=0x7faf50110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.529+0000 7faf56000640 1 --2- 172.21.15.47:0/4055321109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf50109870 0x7faf50110100 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7faf48009860 tx=0x7faf4802bb70 comp rx=0 tx=0).stop 2024-03-21T11:47:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.529+0000 7faf56000640 1 -- 172.21.15.47:0/4055321109 shutdown_connections 2024-03-21T11:47:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.529+0000 7faf56000640 1 --2- 172.21.15.47:0/4055321109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf50109870 0x7faf50110100 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.529+0000 7faf56000640 1 --2- 172.21.15.47:0/4055321109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf50108eb0 0x7faf50109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.529+0000 7faf56000640 1 --2- 172.21.15.47:0/4055321109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf50107cc0 0x7faf501080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.529+0000 7faf56000640 1 -- 172.21.15.47:0/4055321109 >> 172.21.15.47:0/4055321109 conn(0x7faf50076c60 msgr2=0x7faf500790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.536+0000 7faf56000640 1 -- 172.21.15.47:0/4055321109 shutdown_connections 2024-03-21T11:47:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.536+0000 7faf56000640 1 -- 172.21.15.47:0/4055321109 wait complete. 2024-03-21T11:47:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.537+0000 7faf56000640 1 Processor -- start 2024-03-21T11:47:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.538+0000 7faf56000640 1 -- start start 2024-03-21T11:47:57.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.539+0000 7faf56000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf50107cc0 0x7faf5019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.539+0000 7faf56000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf50108eb0 0x7faf5019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.539+0000 7faf56000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf50109870 0x7faf501a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.539+0000 7faf56000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf501139a0 con 0x7faf50109870 2024-03-21T11:47:57.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.539+0000 7faf56000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf50113820 con 0x7faf50108eb0 2024-03-21T11:47:57.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.539+0000 7faf56000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf50113b20 con 0x7faf50107cc0 2024-03-21T11:47:57.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.539+0000 7faf46a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf50108eb0 0x7faf5019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.540+0000 7faf46a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf50108eb0 0x7faf5019e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40072/0 (socket says 172.21.15.47:40072) 2024-03-21T11:47:57.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.540+0000 7faf46a00640 1 -- 172.21.15.47:0/173042748 learned_addr learned my addr 172.21.15.47:0/173042748 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.540+0000 7faf4fe00640 1 --2- 172.21.15.47:0/173042748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf50109870 0x7faf501a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.541+0000 7faf46a00640 1 -- 172.21.15.47:0/173042748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf50107cc0 msgr2=0x7faf5019e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.541+0000 7faf46a00640 1 --2- 172.21.15.47:0/173042748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf50107cc0 0x7faf5019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.541+0000 7faf46a00640 1 -- 172.21.15.47:0/173042748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf50109870 msgr2=0x7faf501a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.541+0000 7faf46a00640 1 --2- 172.21.15.47:0/173042748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf50109870 0x7faf501a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.541+0000 7faf46a00640 1 -- 172.21.15.47:0/173042748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf501a5c30 con 0x7faf50108eb0 2024-03-21T11:47:57.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.543+0000 7faf46a00640 1 --2- 172.21.15.47:0/173042748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf50108eb0 0x7faf5019e550 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7faf4000ba50 tx=0x7faf4000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.545+0000 7faf4cc00640 1 -- 172.21.15.47:0/173042748 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40019070 con 0x7faf50108eb0 2024-03-21T11:47:57.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.545+0000 7faf4cc00640 1 -- 172.21.15.47:0/173042748 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf4000cbe0 con 0x7faf50108eb0 2024-03-21T11:47:57.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.545+0000 7faf4cc00640 1 -- 172.21.15.47:0/173042748 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400076e0 con 0x7faf50108eb0 2024-03-21T11:47:57.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.545+0000 7faf56000640 1 -- 172.21.15.47:0/173042748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf501a5f20 con 0x7faf50108eb0 2024-03-21T11:47:57.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.545+0000 7faf56000640 1 -- 172.21.15.47:0/173042748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf501a6410 con 0x7faf50108eb0 2024-03-21T11:47:57.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.550+0000 7faf58d7d740 1 Processor -- start 2024-03-21T11:47:57.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.551+0000 7faf58d7d740 1 -- start start 2024-03-21T11:47:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.551+0000 7faf58d7d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b416810 0x55c95b416c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.551+0000 7faf58d7d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b4179e0 0x55c95b417e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.551+0000 7faf58d7d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b4183a0 0x55c95b41ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.551+0000 7faf58d7d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95b37b2c0 con 0x55c95b416810 2024-03-21T11:47:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.551+0000 7faf58d7d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95b323c90 con 0x55c95b4183a0 2024-03-21T11:47:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.551+0000 7faf58d7d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95b377360 con 0x55c95b4179e0 2024-03-21T11:47:57.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.552+0000 7faf4fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b416810 0x55c95b416c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.552+0000 7faf4cc00640 1 -- 172.21.15.47:0/173042748 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7faf4001c070 con 0x7faf50108eb0 2024-03-21T11:47:57.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.552+0000 7faf4f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b4179e0 0x55c95b417e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.552+0000 7faf4fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b416810 0x55c95b416c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39048/0 (socket says 172.21.15.47:39048) 2024-03-21T11:47:57.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.552+0000 7faf4fe00640 1 -- 172.21.15.47:0/767026492 learned_addr learned my addr 172.21.15.47:0/767026492 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.553+0000 7faf46a00640 1 --2- 172.21.15.47:0/767026492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b4183a0 0x55c95b41ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.553+0000 7faf4fe00640 1 -- 172.21.15.47:0/767026492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b4179e0 msgr2=0x55c95b417e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.553+0000 7faf4fe00640 1 --2- 172.21.15.47:0/767026492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b4179e0 0x55c95b417e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.553+0000 7faf4fe00640 1 -- 172.21.15.47:0/767026492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b4183a0 msgr2=0x55c95b41ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.553+0000 7faf4fe00640 1 --2- 172.21.15.47:0/767026492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b4183a0 0x55c95b41ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.553+0000 7faf4fe00640 1 -- 172.21.15.47:0/767026492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c95b348080 con 0x55c95b416810 2024-03-21T11:47:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.553+0000 7faf46a00640 1 --2- 172.21.15.47:0/767026492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b4183a0 0x55c95b41ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.553+0000 7faf4fe00640 1 --2- 172.21.15.47:0/767026492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b416810 0x55c95b416c30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faf480349e0 tx=0x7faf48032380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e55a20ad6ae4203 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.554+0000 7faf3fe00640 1 -- 172.21.15.47:0/767026492 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48043020 con 0x55c95b416810 2024-03-21T11:47:57.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.555+0000 7faf3fe00640 1 -- 172.21.15.47:0/767026492 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf48004040 con 0x55c95b416810 2024-03-21T11:47:57.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.555+0000 7faf3fe00640 1 -- 172.21.15.47:0/767026492 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48039040 con 0x55c95b416810 2024-03-21T11:47:57.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.555+0000 7faf4cc00640 1 --2- 172.21.15.47:0/173042748 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faf340a2ff0 0x7faf340a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.555+0000 7faf4cc00640 1 -- 172.21.15.47:0/173042748 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7faf400ced10 con 0x7faf50108eb0 2024-03-21T11:47:57.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.555+0000 7faf4f400640 1 --2- 172.21.15.47:0/173042748 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faf340a2ff0 0x7faf340a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.556+0000 7faf58d7d740 1 -- 172.21.15.47:0/767026492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b416810 msgr2=0x55c95b416c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.556+0000 7faf58d7d740 1 --2- 172.21.15.47:0/767026492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b416810 0x55c95b416c30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faf480349e0 tx=0x7faf48032380 comp rx=0 tx=0).stop 2024-03-21T11:47:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.557+0000 7faf58d7d740 1 -- 172.21.15.47:0/767026492 shutdown_connections 2024-03-21T11:47:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.557+0000 7faf58d7d740 1 --2- 172.21.15.47:0/767026492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b4183a0 0x55c95b41ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.557+0000 7faf58d7d740 1 --2- 172.21.15.47:0/767026492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b4179e0 0x55c95b417e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.557+0000 7faf58d7d740 1 --2- 172.21.15.47:0/767026492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b416810 0x55c95b416c30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.557+0000 7faf58d7d740 1 -- 172.21.15.47:0/767026492 >> 172.21.15.47:0/767026492 conn(0x55c95b411c00 msgr2=0x55c95b412000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.557+0000 7faf58d7d740 1 -- 172.21.15.47:0/767026492 shutdown_connections 2024-03-21T11:47:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.558+0000 7faf4f400640 1 --2- 172.21.15.47:0/173042748 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faf340a2ff0 0x7faf340a54b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7faf38005fd0 tx=0x7faf38005a00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.558+0000 7faf58d7d740 1 -- 172.21.15.47:0/767026492 wait complete. 2024-03-21T11:47:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.558+0000 7faf58d7d740 1 -- start start 2024-03-21T11:47:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.559+0000 7faf58d7d740 1 Processor -- start 2024-03-21T11:47:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.559+0000 7faf58d7d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b416810 0x55c95b420a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.559+0000 7faf58d7d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b4179e0 0x55c95b420f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.559+0000 7faf58d7d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4183a0 0x55c95b427fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.559+0000 7faf58d7d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95b41f350 con 0x55c95b4183a0 2024-03-21T11:47:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.559+0000 7faf46a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b4179e0 0x55c95b420f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.559+0000 7faf4fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4183a0 0x55c95b427fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.560+0000 7faf46a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b4179e0 0x55c95b420f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40094/0 (socket says 172.21.15.47:40094) 2024-03-21T11:47:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.560+0000 7faf46a00640 1 -- 172.21.15.47:0/661698849 learned_addr learned my addr 172.21.15.47:0/661698849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.560+0000 7faf4f400640 1 --2- 172.21.15.47:0/661698849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b416810 0x55c95b420a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.560+0000 7faf58d7d740 1 -- 172.21.15.47:0/661698849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95b3370d0 con 0x55c95b4179e0 2024-03-21T11:47:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.560+0000 7faf58d7d740 1 -- 172.21.15.47:0/661698849 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c95b41f4d0 con 0x55c95b416810 2024-03-21T11:47:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.560+0000 7faf58d7d740 10 client.0 _ll_register_callbacks cb 0x55c95b33be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.561+0000 7faf46a00640 1 -- 172.21.15.47:0/661698849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b416810 msgr2=0x55c95b420a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.561+0000 7faf46a00640 1 --2- 172.21.15.47:0/661698849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b416810 0x55c95b420a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.561+0000 7faf46a00640 1 -- 172.21.15.47:0/661698849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4183a0 msgr2=0x55c95b427fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.561+0000 7faf46a00640 1 --2- 172.21.15.47:0/661698849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4183a0 0x55c95b427fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.561+0000 7faf46a00640 1 -- 172.21.15.47:0/661698849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c95b08f3f0 con 0x55c95b4179e0 2024-03-21T11:47:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.561+0000 7faf4f400640 1 --2- 172.21.15.47:0/661698849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b416810 0x55c95b420a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:47:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.561+0000 7faf46a00640 1 --2- 172.21.15.47:0/661698849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b4179e0 0x55c95b420f70 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7faf400a3530 tx=0x7faf4009b100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bdd45848889a681 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.561+0000 7faf3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.561+0000 7faf3cc00640 1 -- 172.21.15.47:0/661698849 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400a3a60 con 0x55c95b4179e0 2024-03-21T11:47:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.561+0000 7faf3cc00640 1 -- 172.21.15.47:0/661698849 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7faf400bb0b0 con 0x55c95b4179e0 2024-03-21T11:47:57.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.562+0000 7faf58d7d740 1 -- 172.21.15.47:0/661698849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf48004040 con 0x55c95b4179e0 2024-03-21T11:47:57.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.562+0000 7faf3cc00640 1 -- 172.21.15.47:0/661698849 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf400b4150 con 0x55c95b4179e0 2024-03-21T11:47:57.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.562+0000 7faf58d7d740 10 client.33266 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:57.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.562+0000 7faf58d7d740 1 -- 172.21.15.47:0/661698849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c95b1597d0 con 0x55c95b4179e0 2024-03-21T11:47:57.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.562+0000 7faf3cc00640 1 -- 172.21.15.47:0/661698849 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf48004040 con 0x55c95b4179e0 2024-03-21T11:47:57.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.563+0000 7faf3cc00640 1 -- 172.21.15.47:0/661698849 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7faf40099fc0 con 0x55c95b4179e0 2024-03-21T11:47:57.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.563+0000 7faf58d7d740 10 client.33266 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:57.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.563+0000 7faf58d7d740 10 client.33266 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:47:57.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.563+0000 7faf58d7d740 20 client.33266 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:57.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.563+0000 7faf58d7d740 1 --2- 172.21.15.47:0/661698849 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c95b42c320 0x55c95b42e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.563+0000 7faf58d7d740 4 client.33266 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:57.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.563+0000 7faf58d7d740 1 -- 172.21.15.47:0/661698849 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c95b388800 con 0x55c95b42c320 2024-03-21T11:47:57.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.563+0000 7faf4f400640 1 --2- 172.21.15.47:0/661698849 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c95b42c320 0x55c95b42e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.564+0000 7faf4f400640 1 --2- 172.21.15.47:0/661698849 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c95b42c320 0x55c95b42e770 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8895e2812e3dd3ef server_cookie=9556b43c894d73b7 in_seq=0 out_seq=0 2024-03-21T11:47:57.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:57 smithi047 ceph-mon[31351]: pgmap v1139: 129 pgs: 129 active+clean; 733 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.7 MiB/s wr, 396 op/s 2024-03-21T11:47:57.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:57 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2489576940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:47:57.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.564+0000 7faf3cc00640 10 client.33266 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:47:57.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.565+0000 7faf3cc00640 1 -- 172.21.15.47:0/661698849 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c95b388800 con 0x55c95b42c320 2024-03-21T11:47:57.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.565+0000 7faf3cc00640 10 client.33266 handle_command_reply: tid=0 2024-03-21T11:47:57.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.565+0000 7faf58d7d740 1 client.33266 shutdown 2024-03-21T11:47:57.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.565+0000 7faf58d7d740 1 -- 172.21.15.47:0/661698849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b4179e0 msgr2=0x55c95b420f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.565+0000 7faf58d7d740 1 --2- 172.21.15.47:0/661698849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b4179e0 0x55c95b420f70 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7faf400a3530 tx=0x7faf4009b100 comp rx=0 tx=0).stop 2024-03-21T11:47:57.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.565+0000 7faf58d7d740 1 -- 172.21.15.47:0/661698849 shutdown_connections 2024-03-21T11:47:57.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.565+0000 7faf58d7d740 1 --2- 172.21.15.47:0/661698849 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c95b42c320 0x55c95b42e770 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.565+0000 7faf3cc00640 0 client.33266 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:47:57.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.565+0000 7faf58d7d740 1 --2- 172.21.15.47:0/661698849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4183a0 0x55c95b427fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.565+0000 7faf58d7d740 1 --2- 172.21.15.47:0/661698849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b4179e0 0x55c95b420f70 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.565+0000 7faf58d7d740 1 --2- 172.21.15.47:0/661698849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b416810 0x55c95b420a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.565+0000 7faf58d7d740 1 -- 172.21.15.47:0/661698849 >> 172.21.15.47:0/661698849 conn(0x55c95b411c00 msgr2=0x55c95b410eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.566+0000 7faf58d7d740 1 -- 172.21.15.47:0/661698849 shutdown_connections 2024-03-21T11:47:57.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.566+0000 7faf58d7d740 1 -- 172.21.15.47:0/661698849 wait complete. 2024-03-21T11:47:57.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.566+0000 7faf58d7d740 20 client.33266 trim_cache size 0 max 16384 2024-03-21T11:47:57.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.591+0000 7faf58d7d740 1 Processor -- start 2024-03-21T11:47:57.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.591+0000 7faf58d7d740 1 -- start start 2024-03-21T11:47:57.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.591+0000 7faf58d7d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b411c00 0x55c95b41ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.591+0000 7faf58d7d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b416810 0x55c95b427f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.592+0000 7faf58d7d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4179e0 0x55c95b41b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.592+0000 7faf58d7d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf501b0760 con 0x55c95b4179e0 2024-03-21T11:47:57.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.592+0000 7faf58d7d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf501b20e0 con 0x55c95b416810 2024-03-21T11:47:57.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.592+0000 7faf58d7d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf501b0dc0 con 0x55c95b411c00 2024-03-21T11:47:57.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.592+0000 7faf46a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b416810 0x55c95b427f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.592+0000 7faf46a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b416810 0x55c95b427f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40100/0 (socket says 172.21.15.47:40100) 2024-03-21T11:47:57.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.592+0000 7faf46a00640 1 -- 172.21.15.47:0/2516546871 learned_addr learned my addr 172.21.15.47:0/2516546871 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.592+0000 7faf4f400640 1 --2- 172.21.15.47:0/2516546871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b411c00 0x55c95b41ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.593+0000 7faf4fe00640 1 --2- 172.21.15.47:0/2516546871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4179e0 0x55c95b41b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.593+0000 7faf4fe00640 1 -- 172.21.15.47:0/2516546871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b411c00 msgr2=0x55c95b41ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.593+0000 7faf4fe00640 1 --2- 172.21.15.47:0/2516546871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b411c00 0x55c95b41ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.593+0000 7faf4fe00640 1 -- 172.21.15.47:0/2516546871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b416810 msgr2=0x55c95b427f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.593+0000 7faf4fe00640 1 --2- 172.21.15.47:0/2516546871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b416810 0x55c95b427f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.593+0000 7faf4fe00640 1 -- 172.21.15.47:0/2516546871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c95b417460 con 0x55c95b4179e0 2024-03-21T11:47:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.594+0000 7faf4fe00640 1 --2- 172.21.15.47:0/2516546871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4179e0 0x55c95b41b150 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7faf48009780 tx=0x7faf4802c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2c515cc674379d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.594+0000 7faf3f400640 1 -- 172.21.15.47:0/2516546871 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48039070 con 0x55c95b4179e0 2024-03-21T11:47:57.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.595+0000 7faf3f400640 1 -- 172.21.15.47:0/2516546871 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf4804c020 con 0x55c95b4179e0 2024-03-21T11:47:57.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.595+0000 7faf3f400640 1 -- 172.21.15.47:0/2516546871 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48043040 con 0x55c95b4179e0 2024-03-21T11:47:57.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.595+0000 7faf58d7d740 1 -- 172.21.15.47:0/2516546871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4179e0 msgr2=0x55c95b41b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.595+0000 7faf58d7d740 1 --2- 172.21.15.47:0/2516546871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4179e0 0x55c95b41b150 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7faf48009780 tx=0x7faf4802c510 comp rx=0 tx=0).stop 2024-03-21T11:47:57.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.597+0000 7faf58d7d740 1 -- 172.21.15.47:0/2516546871 shutdown_connections 2024-03-21T11:47:57.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.597+0000 7faf58d7d740 1 --2- 172.21.15.47:0/2516546871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4179e0 0x55c95b41b150 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.597+0000 7faf58d7d740 1 --2- 172.21.15.47:0/2516546871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b416810 0x55c95b427f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.597+0000 7faf58d7d740 1 --2- 172.21.15.47:0/2516546871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b411c00 0x55c95b41ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.597+0000 7faf58d7d740 1 -- 172.21.15.47:0/2516546871 >> 172.21.15.47:0/2516546871 conn(0x55c95b41fc40 msgr2=0x55c95b4156f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.597+0000 7faf58d7d740 1 -- 172.21.15.47:0/2516546871 shutdown_connections 2024-03-21T11:47:57.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.597+0000 7faf58d7d740 1 -- 172.21.15.47:0/2516546871 wait complete. 2024-03-21T11:47:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.599+0000 7faf58d7d740 1 -- start start 2024-03-21T11:47:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.599+0000 7faf58d7d740 1 Processor -- start 2024-03-21T11:47:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.600+0000 7faf58d7d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b411c00 0x55c95b427df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.600+0000 7faf58d7d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b416810 0x55c95b4264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.600+0000 7faf58d7d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4179e0 0x55c95b4269e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.600+0000 7faf58d7d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf501b1a80 con 0x55c95b4179e0 2024-03-21T11:47:57.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.600+0000 7faf58d7d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf501b1420 con 0x55c95b411c00 2024-03-21T11:47:57.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.600+0000 7faf58d7d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf501ae920 con 0x55c95b416810 2024-03-21T11:47:57.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.600+0000 7faf58d7d740 10 client.0 _ll_register_callbacks cb 0x7faf4000c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:57.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.600+0000 7faf46a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b416810 0x55c95b4264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.600+0000 7faf46a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b416810 0x55c95b4264a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54314/0 (socket says 172.21.15.47:54314) 2024-03-21T11:47:57.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.600+0000 7faf46a00640 1 -- 172.21.15.47:0/1556505807 learned_addr learned my addr 172.21.15.47:0/1556505807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.600+0000 7faf4f400640 1 --2- 172.21.15.47:0/1556505807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b411c00 0x55c95b427df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.601+0000 7faf4fe00640 1 --2- 172.21.15.47:0/1556505807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4179e0 0x55c95b4269e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.601+0000 7faf46a00640 1 -- 172.21.15.47:0/1556505807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b411c00 msgr2=0x55c95b427df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.601+0000 7faf46a00640 1 --2- 172.21.15.47:0/1556505807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b411c00 0x55c95b427df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.601+0000 7faf46a00640 1 -- 172.21.15.47:0/1556505807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4179e0 msgr2=0x55c95b4269e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.601+0000 7faf46a00640 1 --2- 172.21.15.47:0/1556505807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4179e0 0x55c95b4269e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.602+0000 7faf46a00640 1 -- 172.21.15.47:0/1556505807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c95b427210 con 0x55c95b416810 2024-03-21T11:47:57.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.602+0000 7faf4fe00640 1 --2- 172.21.15.47:0/1556505807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4179e0 0x55c95b4269e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:57.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.602+0000 7faf46a00640 1 --2- 172.21.15.47:0/1556505807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b416810 0x55c95b4264a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7faf40097a80 tx=0x7faf40014d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=609bfabcbe669b54 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.602+0000 7faf3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:47:57.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.603+0000 7faf3cc00640 1 -- 172.21.15.47:0/1556505807 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400a3a60 con 0x55c95b416810 2024-03-21T11:47:57.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.603+0000 7faf3cc00640 1 -- 172.21.15.47:0/1556505807 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7faf400a35d0 con 0x55c95b416810 2024-03-21T11:47:57.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.603+0000 7faf58d7d740 1 -- 172.21.15.47:0/1556505807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf4804c020 con 0x55c95b416810 2024-03-21T11:47:57.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.604+0000 7faf3cc00640 1 -- 172.21.15.47:0/1556505807 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf400a37d0 con 0x55c95b416810 2024-03-21T11:47:57.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.604+0000 7faf58d7d740 10 client.33232 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:57.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.604+0000 7faf58d7d740 1 -- 172.21.15.47:0/1556505807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c95b488bd0 con 0x55c95b416810 2024-03-21T11:47:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.604+0000 7faf3cc00640 1 -- 172.21.15.47:0/1556505807 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf4804c020 con 0x55c95b416810 2024-03-21T11:47:57.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.605+0000 7faf3cc00640 1 -- 172.21.15.47:0/1556505807 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7faf4009a550 con 0x55c95b416810 2024-03-21T11:47:57.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.605+0000 7faf58d7d740 10 client.33232 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:57.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.605+0000 7faf58d7d740 10 client.33232 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:47:57.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.605+0000 7faf58d7d740 20 client.33232 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:57.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.605+0000 7faf58d7d740 1 --2- 172.21.15.47:0/1556505807 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c95b4183a0 0x55c95b48b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.605+0000 7faf58d7d740 4 client.33232 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:47:57.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.605+0000 7faf58d7d740 1 -- 172.21.15.47:0/1556505807 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c95b412580 con 0x55c95b4183a0 2024-03-21T11:47:57.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.606+0000 7faf4f400640 1 --2- 172.21.15.47:0/1556505807 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c95b4183a0 0x55c95b48b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.607+0000 7faf4f400640 1 --2- 172.21.15.47:0/1556505807 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c95b4183a0 0x55c95b48b3e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9fc1a7f546ef9ce1 server_cookie=20b1598c34172b4d in_seq=0 out_seq=0 2024-03-21T11:47:57.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.607+0000 7faf3cc00640 10 client.33232 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:47:57.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.609+0000 7faf3cc00640 1 -- 172.21.15.47:0/1556505807 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c95b412580 con 0x55c95b4183a0 2024-03-21T11:47:57.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.609+0000 7faf3cc00640 10 client.33232 handle_command_reply: tid=0 2024-03-21T11:47:57.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.609+0000 7faf58d7d740 1 client.33232 shutdown 2024-03-21T11:47:57.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.610+0000 7faf58d7d740 1 -- 172.21.15.47:0/1556505807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b416810 msgr2=0x55c95b4264a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.610+0000 7faf58d7d740 1 --2- 172.21.15.47:0/1556505807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b416810 0x55c95b4264a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7faf40097a80 tx=0x7faf40014d60 comp rx=0 tx=0).stop 2024-03-21T11:47:57.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.610+0000 7faf58d7d740 1 -- 172.21.15.47:0/1556505807 shutdown_connections 2024-03-21T11:47:57.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.610+0000 7faf58d7d740 1 --2- 172.21.15.47:0/1556505807 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55c95b4183a0 0x55c95b48b3e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.610+0000 7faf58d7d740 1 --2- 172.21.15.47:0/1556505807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c95b4179e0 0x55c95b4269e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.610+0000 7faf58d7d740 1 --2- 172.21.15.47:0/1556505807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c95b416810 0x55c95b4264a0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.610+0000 7faf58d7d740 1 --2- 172.21.15.47:0/1556505807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c95b411c00 0x55c95b427df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.610+0000 7faf58d7d740 1 -- 172.21.15.47:0/1556505807 >> 172.21.15.47:0/1556505807 conn(0x55c95b41fc40 msgr2=0x55c95b410620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.610+0000 7faf3cc00640 0 client.33232 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:47:57.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.611+0000 7faf58d7d740 1 -- 172.21.15.47:0/1556505807 shutdown_connections 2024-03-21T11:47:57.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.612+0000 7faf58d7d740 1 -- 172.21.15.47:0/1556505807 wait complete. 2024-03-21T11:47:57.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.612+0000 7faf58d7d740 20 client.33232 trim_cache size 0 max 16384 2024-03-21T11:47:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:47:57.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.613+0000 7faf56000640 1 -- 172.21.15.47:0/173042748 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faf340a2ff0 msgr2=0x7faf340a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.613+0000 7faf56000640 1 --2- 172.21.15.47:0/173042748 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faf340a2ff0 0x7faf340a54b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7faf38005fd0 tx=0x7faf38005a00 comp rx=0 tx=0).stop 2024-03-21T11:47:57.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.613+0000 7faf56000640 1 -- 172.21.15.47:0/173042748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf50108eb0 msgr2=0x7faf5019e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.613+0000 7faf56000640 1 --2- 172.21.15.47:0/173042748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf50108eb0 0x7faf5019e550 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7faf4000ba50 tx=0x7faf4000bf10 comp rx=0 tx=0).stop 2024-03-21T11:47:57.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.614+0000 7faf56000640 1 -- 172.21.15.47:0/173042748 shutdown_connections 2024-03-21T11:47:57.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.614+0000 7faf56000640 1 --2- 172.21.15.47:0/173042748 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faf340a2ff0 0x7faf340a54b0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.614+0000 7faf56000640 1 --2- 172.21.15.47:0/173042748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf50109870 0x7faf501a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.614+0000 7faf56000640 1 --2- 172.21.15.47:0/173042748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf50108eb0 0x7faf5019e550 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.614+0000 7faf56000640 1 --2- 172.21.15.47:0/173042748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf50107cc0 0x7faf5019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.614+0000 7faf56000640 1 -- 172.21.15.47:0/173042748 >> 172.21.15.47:0/173042748 conn(0x7faf50076c60 msgr2=0x7faf5010c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.614+0000 7faf56000640 1 -- 172.21.15.47:0/173042748 shutdown_connections 2024-03-21T11:47:57.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.614+0000 7faf56000640 1 -- 172.21.15.47:0/173042748 wait complete. 2024-03-21T11:47:57.641 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:47:57.641 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:47:57.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:57 smithi100 ceph-mon[36245]: pgmap v1139: 129 pgs: 129 active+clean; 733 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.7 MiB/s wr, 396 op/s 2024-03-21T11:47:57.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:57 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2489576940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:47:57.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.890+0000 7ffacca00640 1 -- 172.21.15.47:0/106139450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac81086a0 msgr2=0x7ffac8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.890+0000 7ffacca00640 1 --2- 172.21.15.47:0/106139450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac81086a0 0x7ffac8108ac0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009ca0 tx=0x7ffac002c100 comp rx=0 tx=0).stop 2024-03-21T11:47:57.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.890+0000 7ffacca00640 1 -- 172.21.15.47:0/106139450 shutdown_connections 2024-03-21T11:47:57.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.890+0000 7ffacca00640 1 --2- 172.21.15.47:0/106139450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffac810ff20 0x7ffac8112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.890+0000 7ffacca00640 1 --2- 172.21.15.47:0/106139450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac8109000 0x7ffac810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.890+0000 7ffacca00640 1 --2- 172.21.15.47:0/106139450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac81086a0 0x7ffac8108ac0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.890+0000 7ffacca00640 1 -- 172.21.15.47:0/106139450 >> 172.21.15.47:0/106139450 conn(0x7ffac806e500 msgr2=0x7ffac8070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.890+0000 7ffacca00640 1 -- 172.21.15.47:0/106139450 shutdown_connections 2024-03-21T11:47:57.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.890+0000 7ffacca00640 1 -- 172.21.15.47:0/106139450 wait complete. 2024-03-21T11:47:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.891+0000 7ffacca00640 1 Processor -- start 2024-03-21T11:47:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.891+0000 7ffacca00640 1 -- start start 2024-03-21T11:47:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.892+0000 7ffacca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac81086a0 0x7ffac810b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.892+0000 7ffacca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffac8109000 0x7ffac8109970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.892+0000 7ffacca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac810ff20 0x7ffac8109eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.892+0000 7ffacca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8115b00 con 0x7ffac81086a0 2024-03-21T11:47:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.892+0000 7ffacca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8115980 con 0x7ffac810ff20 2024-03-21T11:47:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.892+0000 7ffacca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8115c80 con 0x7ffac8109000 2024-03-21T11:47:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.892+0000 7ffac6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac810ff20 0x7ffac8109eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.892+0000 7ffac5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffac8109000 0x7ffac8109970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.892+0000 7ffac6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac810ff20 0x7ffac8109eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40134/0 (socket says 172.21.15.47:40134) 2024-03-21T11:47:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.892+0000 7ffac6800640 1 -- 172.21.15.47:0/3502487806 learned_addr learned my addr 172.21.15.47:0/3502487806 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.893+0000 7ffac5400640 1 -- 172.21.15.47:0/3502487806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac810ff20 msgr2=0x7ffac8109eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.893+0000 7ffac5400640 1 --2- 172.21.15.47:0/3502487806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac810ff20 0x7ffac8109eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.893+0000 7ffac5400640 1 -- 172.21.15.47:0/3502487806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac81086a0 msgr2=0x7ffac810b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.893+0000 7ffac5400640 1 --2- 172.21.15.47:0/3502487806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac81086a0 0x7ffac810b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.893+0000 7ffac5400640 1 -- 172.21.15.47:0/3502487806 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac810a740 con 0x7ffac8109000 2024-03-21T11:47:57.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.893+0000 7ffac6800640 1 --2- 172.21.15.47:0/3502487806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac810ff20 0x7ffac8109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:57.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.894+0000 7ffac5400640 1 --2- 172.21.15.47:0/3502487806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffac8109000 0x7ffac8109970 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ffab000e9f0 tx=0x7ffab000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.895+0000 7ffabea00640 1 -- 172.21.15.47:0/3502487806 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab000cd80 con 0x7ffac8109000 2024-03-21T11:47:57.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.895+0000 7ffabea00640 1 -- 172.21.15.47:0/3502487806 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffab00035f0 con 0x7ffac8109000 2024-03-21T11:47:57.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.895+0000 7ffabea00640 1 -- 172.21.15.47:0/3502487806 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0010690 con 0x7ffac8109000 2024-03-21T11:47:57.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.895+0000 7ffacca00640 1 -- 172.21.15.47:0/3502487806 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac81aa2e0 con 0x7ffac8109000 2024-03-21T11:47:57.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.895+0000 7ffacca00640 1 -- 172.21.15.47:0/3502487806 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac81aa7e0 con 0x7ffac8109000 2024-03-21T11:47:57.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.899+0000 7ffacf785740 1 Processor -- start 2024-03-21T11:47:57.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.899+0000 7ffacf785740 1 -- start start 2024-03-21T11:47:57.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.899+0000 7ffacf785740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb5ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.899+0000 7ffacf785740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb5fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.899+0000 7ffacf785740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb603a0 0x55e5fdb66c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.899+0000 7ffacf785740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5fdac32c0 con 0x55e5fdb5f9e0 2024-03-21T11:47:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.899+0000 7ffacf785740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5fda6bc90 con 0x55e5fdb5e810 2024-03-21T11:47:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.899+0000 7ffacf785740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5fdabf360 con 0x55e5fdb603a0 2024-03-21T11:47:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.899+0000 7ffac5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb5fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.899+0000 7ffac5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb5fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39106/0 (socket says 172.21.15.47:39106) 2024-03-21T11:47:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.899+0000 7ffac5e00640 1 -- 172.21.15.47:0/1125935816 learned_addr learned my addr 172.21.15.47:0/1125935816 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.899+0000 7ffac6800640 1 --2- 172.21.15.47:0/1125935816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb5ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.900+0000 7ffac5e00640 1 -- 172.21.15.47:0/1125935816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb603a0 msgr2=0x55e5fdb66c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:47:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.900+0000 7ffac5e00640 1 --2- 172.21.15.47:0/1125935816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb603a0 0x55e5fdb66c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.900+0000 7ffac5e00640 1 -- 172.21.15.47:0/1125935816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 msgr2=0x55e5fdb5ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.900+0000 7ffac5e00640 1 --2- 172.21.15.47:0/1125935816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb5ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.900+0000 7ffac5e00640 1 -- 172.21.15.47:0/1125935816 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5fda90080 con 0x55e5fdb5f9e0 2024-03-21T11:47:57.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.900+0000 7ffac5e00640 1 --2- 172.21.15.47:0/1125935816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb5fe60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffac0005e30 tx=0x7ffac0005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc2ec4de546f5ec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.900+0000 7ffabea00640 1 -- 172.21.15.47:0/3502487806 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ffab0003b30 con 0x7ffac8109000 2024-03-21T11:47:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.901+0000 7ffab7000640 1 -- 172.21.15.47:0/1125935816 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0041020 con 0x55e5fdb5f9e0 2024-03-21T11:47:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.901+0000 7ffab7000640 1 -- 172.21.15.47:0/1125935816 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffac0003970 con 0x55e5fdb5f9e0 2024-03-21T11:47:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.901+0000 7ffab7000640 1 -- 172.21.15.47:0/1125935816 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0039040 con 0x55e5fdb5f9e0 2024-03-21T11:47:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.901+0000 7ffacf785740 1 -- 172.21.15.47:0/1125935816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 msgr2=0x55e5fdb5fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.901+0000 7ffacf785740 1 --2- 172.21.15.47:0/1125935816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb5fe60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffac0005e30 tx=0x7ffac0005f70 comp rx=0 tx=0).stop 2024-03-21T11:47:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.902+0000 7ffacf785740 1 -- 172.21.15.47:0/1125935816 shutdown_connections 2024-03-21T11:47:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.902+0000 7ffacf785740 1 --2- 172.21.15.47:0/1125935816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb603a0 0x55e5fdb66c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.902+0000 7ffacf785740 1 --2- 172.21.15.47:0/1125935816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb5fe60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.902+0000 7ffacf785740 1 --2- 172.21.15.47:0/1125935816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb5ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.902+0000 7ffacf785740 1 -- 172.21.15.47:0/1125935816 >> 172.21.15.47:0/1125935816 conn(0x55e5fdb59c00 msgr2=0x55e5fdb5a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.902+0000 7ffacf785740 1 -- 172.21.15.47:0/1125935816 shutdown_connections 2024-03-21T11:47:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.902+0000 7ffacf785740 1 -- 172.21.15.47:0/1125935816 wait complete. 2024-03-21T11:47:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.902+0000 7ffabea00640 1 --2- 172.21.15.47:0/3502487806 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffaac0a2eb0 0x7ffaac0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.902+0000 7ffabea00640 1 -- 172.21.15.47:0/3502487806 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffab00cfb00 con 0x7ffac8109000 2024-03-21T11:47:57.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.902+0000 7ffacf785740 1 -- start start 2024-03-21T11:47:57.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.902+0000 7ffac5e00640 1 --2- 172.21.15.47:0/3502487806 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffaac0a2eb0 0x7ffaac0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.902+0000 7ffacf785740 1 Processor -- start 2024-03-21T11:47:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.903+0000 7ffac5e00640 1 --2- 172.21.15.47:0/3502487806 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffaac0a2eb0 0x7ffaac0a5370 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ffac0005c70 tx=0x7ffac002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.903+0000 7ffacf785740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb68ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.903+0000 7ffacf785740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb69020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.903+0000 7ffacf785740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb603a0 0x55e5fdb6ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.903+0000 7ffacf785740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5fdb67350 con 0x55e5fdb5f9e0 2024-03-21T11:47:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.904+0000 7ffacf785740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5fda7f0d0 con 0x55e5fdb5e810 2024-03-21T11:47:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.904+0000 7ffacf785740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5fdb674d0 con 0x55e5fdb603a0 2024-03-21T11:47:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.904+0000 7ffac5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb68ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.904+0000 7ffacf785740 10 client.0 _ll_register_callbacks cb 0x55e5fda83e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.904+0000 7ffac6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb603a0 0x55e5fdb6ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.904+0000 7ffac5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb68ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40140/0 (socket says 172.21.15.47:40140) 2024-03-21T11:47:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.904+0000 7ffac5e00640 1 -- 172.21.15.47:0/1043624909 learned_addr learned my addr 172.21.15.47:0/1043624909 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.904+0000 7ffac5400640 1 --2- 172.21.15.47:0/1043624909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb69020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.904+0000 7ffac5e00640 1 -- 172.21.15.47:0/1043624909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb603a0 msgr2=0x55e5fdb6ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.904+0000 7ffac5e00640 1 --2- 172.21.15.47:0/1043624909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb603a0 0x55e5fdb6ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.904+0000 7ffac5e00640 1 -- 172.21.15.47:0/1043624909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 msgr2=0x55e5fdb69020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.904+0000 7ffac5e00640 1 --2- 172.21.15.47:0/1043624909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb69020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.904+0000 7ffac5e00640 1 -- 172.21.15.47:0/1043624909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5fd7d73f0 con 0x55e5fdb5e810 2024-03-21T11:47:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.905+0000 7ffac5400640 1 --2- 172.21.15.47:0/1043624909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb69020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.905+0000 7ffac5e00640 1 --2- 172.21.15.47:0/1043624909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb68ae0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7ffac002c640 tx=0x7ffac0005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d50e4a72e63dc946 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.905+0000 7ffaab400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.905+0000 7ffaab400640 1 -- 172.21.15.47:0/1043624909 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac004a070 con 0x55e5fdb5e810 2024-03-21T11:47:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.905+0000 7ffaab400640 1 -- 172.21.15.47:0/1043624909 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffac00046f0 con 0x55e5fdb5e810 2024-03-21T11:47:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.906+0000 7ffacf785740 1 -- 172.21.15.47:0/1043624909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffac0003970 con 0x55e5fdb5e810 2024-03-21T11:47:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.906+0000 7ffaab400640 1 -- 172.21.15.47:0/1043624909 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffac0039070 con 0x55e5fdb5e810 2024-03-21T11:47:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.906+0000 7ffacf785740 10 client.33284 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:57.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.906+0000 7ffacf785740 1 -- 172.21.15.47:0/1043624909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e5fd8a17d0 con 0x55e5fdb5e810 2024-03-21T11:47:57.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.906+0000 7ffaab400640 1 -- 172.21.15.47:0/1043624909 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffac0003970 con 0x55e5fdb5e810 2024-03-21T11:47:57.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.906+0000 7ffaab400640 1 -- 172.21.15.47:0/1043624909 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffac004e070 con 0x55e5fdb5e810 2024-03-21T11:47:57.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.907+0000 7ffacf785740 10 client.33284 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:57.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.907+0000 7ffacf785740 10 client.33284 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:47:57.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.907+0000 7ffacf785740 20 client.33284 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:57.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.907+0000 7ffacf785740 1 --2- 172.21.15.47:0/1043624909 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e5fdb74120 0x55e5fdb76510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.907+0000 7ffacf785740 4 client.33284 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.907+0000 7ffacf785740 1 -- 172.21.15.47:0/1043624909 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5fdad0800 con 0x55e5fdb74120 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.907+0000 7ffac5400640 1 --2- 172.21.15.47:0/1043624909 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e5fdb74120 0x55e5fdb76510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.908+0000 7ffac5400640 1 --2- 172.21.15.47:0/1043624909 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e5fdb74120 0x55e5fdb76510 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5aaf793501c08bf0 server_cookie=8ec4664e45221c2e in_seq=0 out_seq=0 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.908+0000 7ffaab400640 10 client.33284 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.909+0000 7ffaab400640 1 -- 172.21.15.47:0/1043624909 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e5fdad0800 con 0x55e5fdb74120 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.909+0000 7ffaab400640 10 client.33284 handle_command_reply: tid=0 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.909+0000 7ffacf785740 1 client.33284 shutdown 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.909+0000 7ffacf785740 1 -- 172.21.15.47:0/1043624909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 msgr2=0x55e5fdb68ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.909+0000 7ffacf785740 1 --2- 172.21.15.47:0/1043624909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb68ae0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7ffac002c640 tx=0x7ffac0005f70 comp rx=0 tx=0).stop 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.909+0000 7ffacf785740 1 -- 172.21.15.47:0/1043624909 shutdown_connections 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.909+0000 7ffacf785740 1 --2- 172.21.15.47:0/1043624909 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e5fdb74120 0x55e5fdb76510 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.909+0000 7ffacf785740 1 --2- 172.21.15.47:0/1043624909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb603a0 0x55e5fdb6ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.909+0000 7ffacf785740 1 --2- 172.21.15.47:0/1043624909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb69020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.909+0000 7ffacf785740 1 --2- 172.21.15.47:0/1043624909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb68ae0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.909+0000 7ffacf785740 1 -- 172.21.15.47:0/1043624909 >> 172.21.15.47:0/1043624909 conn(0x55e5fdb59c00 msgr2=0x55e5fdb58eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.909+0000 7ffaab400640 0 client.33284 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:47:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.909+0000 7ffacf785740 1 -- 172.21.15.47:0/1043624909 shutdown_connections 2024-03-21T11:47:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.909+0000 7ffacf785740 1 -- 172.21.15.47:0/1043624909 wait complete. 2024-03-21T11:47:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.910+0000 7ffacf785740 20 client.33284 trim_cache size 0 max 16384 2024-03-21T11:47:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.926+0000 7ffacf785740 1 Processor -- start 2024-03-21T11:47:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.926+0000 7ffacf785740 1 -- start start 2024-03-21T11:47:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.926+0000 7ffacf785740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb59c00 0x55e5fdb5d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.926+0000 7ffacf785740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb5e810 0x55e5fdb6ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.926+0000 7ffacf785740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb62b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.926+0000 7ffacf785740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81b4ac0 con 0x55e5fdb5f9e0 2024-03-21T11:47:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.926+0000 7ffacf785740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81b6440 con 0x55e5fdb59c00 2024-03-21T11:47:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.926+0000 7ffacf785740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81b5120 con 0x55e5fdb5e810 2024-03-21T11:47:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.926+0000 7ffac5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb59c00 0x55e5fdb5d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.926+0000 7ffac5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb5e810 0x55e5fdb6ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.926+0000 7ffac5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb5e810 0x55e5fdb6ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54342/0 (socket says 172.21.15.47:54342) 2024-03-21T11:47:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.927+0000 7ffac5400640 1 -- 172.21.15.47:0/1750756095 learned_addr learned my addr 172.21.15.47:0/1750756095 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.926+0000 7ffac5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb59c00 0x55e5fdb5d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40156/0 (socket says 172.21.15.47:40156) 2024-03-21T11:47:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.927+0000 7ffac6800640 1 --2- 172.21.15.47:0/1750756095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb62b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.927+0000 7ffac5400640 1 -- 172.21.15.47:0/1750756095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb59c00 msgr2=0x55e5fdb5d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.927+0000 7ffac5400640 1 --2- 172.21.15.47:0/1750756095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb59c00 0x55e5fdb5d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.927+0000 7ffac5400640 1 -- 172.21.15.47:0/1750756095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 msgr2=0x55e5fdb62b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.927+0000 7ffac5400640 1 --2- 172.21.15.47:0/1750756095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb62b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.927+0000 7ffac5400640 1 -- 172.21.15.47:0/1750756095 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5fdb66f60 con 0x55e5fdb5e810 2024-03-21T11:47:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.927+0000 7ffac5e00640 1 --2- 172.21.15.47:0/1750756095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb59c00 0x55e5fdb5d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.927+0000 7ffac5400640 1 --2- 172.21.15.47:0/1750756095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb5e810 0x55e5fdb6ff60 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ffab00982a0 tx=0x7ffab00bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b205e0647788ccc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.928+0000 7ffab6600640 1 -- 172.21.15.47:0/1750756095 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab00b6030 con 0x55e5fdb5e810 2024-03-21T11:47:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.928+0000 7ffab6600640 1 -- 172.21.15.47:0/1750756095 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffab00a4850 con 0x55e5fdb5e810 2024-03-21T11:47:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.928+0000 7ffab6600640 1 -- 172.21.15.47:0/1750756095 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab009e040 con 0x55e5fdb5e810 2024-03-21T11:47:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.928+0000 7ffacf785740 1 -- 172.21.15.47:0/1750756095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb5e810 msgr2=0x55e5fdb6ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.928+0000 7ffacf785740 1 --2- 172.21.15.47:0/1750756095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb5e810 0x55e5fdb6ff60 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ffab00982a0 tx=0x7ffab00bc040 comp rx=0 tx=0).stop 2024-03-21T11:47:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.928+0000 7ffacf785740 1 -- 172.21.15.47:0/1750756095 shutdown_connections 2024-03-21T11:47:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.928+0000 7ffacf785740 1 --2- 172.21.15.47:0/1750756095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb62b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.928+0000 7ffacf785740 1 --2- 172.21.15.47:0/1750756095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb5e810 0x55e5fdb6ff60 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.928+0000 7ffacf785740 1 --2- 172.21.15.47:0/1750756095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb59c00 0x55e5fdb5d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.928+0000 7ffacf785740 1 -- 172.21.15.47:0/1750756095 >> 172.21.15.47:0/1750756095 conn(0x55e5fdb76810 msgr2=0x55e5fdb5ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.929+0000 7ffacf785740 1 -- 172.21.15.47:0/1750756095 shutdown_connections 2024-03-21T11:47:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.929+0000 7ffacf785740 1 -- 172.21.15.47:0/1750756095 wait complete. 2024-03-21T11:47:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.929+0000 7ffacf785740 1 -- start start 2024-03-21T11:47:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.929+0000 7ffacf785740 1 Processor -- start 2024-03-21T11:47:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.930+0000 7ffacf785740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb59c00 0x55e5fdb6bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.930+0000 7ffacf785740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb6c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.930+0000 7ffacf785740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb6f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.930+0000 7ffacf785740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81b5de0 con 0x55e5fdb59c00 2024-03-21T11:47:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.930+0000 7ffacf785740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81b5780 con 0x55e5fdb5e810 2024-03-21T11:47:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.930+0000 7ffacf785740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81b2c80 con 0x55e5fdb5f9e0 2024-03-21T11:47:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.930+0000 7ffacf785740 10 client.0 _ll_register_callbacks cb 0x7ffab0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.930+0000 7ffac5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb59c00 0x55e5fdb6bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.930+0000 7ffac5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb6c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.930+0000 7ffac5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb59c00 0x55e5fdb6bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39132/0 (socket says 172.21.15.47:39132) 2024-03-21T11:47:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.930+0000 7ffac5e00640 1 -- 172.21.15.47:0/2695580923 learned_addr learned my addr 172.21.15.47:0/2695580923 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.930+0000 7ffac6800640 1 --2- 172.21.15.47:0/2695580923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb6f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.931+0000 7ffac5e00640 1 -- 172.21.15.47:0/2695580923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb5f9e0 msgr2=0x55e5fdb6f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.931+0000 7ffac5e00640 1 --2- 172.21.15.47:0/2695580923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb6f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.931+0000 7ffac5e00640 1 -- 172.21.15.47:0/2695580923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 msgr2=0x55e5fdb6c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.931+0000 7ffac5e00640 1 --2- 172.21.15.47:0/2695580923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb6c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.931+0000 7ffac5e00640 1 -- 172.21.15.47:0/2695580923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5fdb6fa60 con 0x55e5fdb59c00 2024-03-21T11:47:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.931+0000 7ffac5400640 1 --2- 172.21.15.47:0/2695580923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb6c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:57.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.935+0000 7ffac5e00640 1 --2- 172.21.15.47:0/2695580923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb59c00 0x55e5fdb6bf10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffac0046640 tx=0x7ffac0032a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33afea8c75e06a6e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:57.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.935+0000 7ffaab400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:47:57.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.936+0000 7ffaab400640 1 -- 172.21.15.47:0/2695580923 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0068070 con 0x55e5fdb59c00 2024-03-21T11:47:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.936+0000 7ffacf785740 1 -- 172.21.15.47:0/2695580923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffab00a4850 con 0x55e5fdb59c00 2024-03-21T11:47:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.936+0000 7ffaab400640 1 -- 172.21.15.47:0/2695580923 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffac0046d40 con 0x55e5fdb59c00 2024-03-21T11:47:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.936+0000 7ffaab400640 1 -- 172.21.15.47:0/2695580923 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffac004a040 con 0x55e5fdb59c00 2024-03-21T11:47:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.937+0000 7ffacf785740 10 client.22416 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.937+0000 7ffacf785740 1 -- 172.21.15.47:0/2695580923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e5fdbcffd0 con 0x55e5fdb59c00 2024-03-21T11:47:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.937+0000 7ffaab400640 1 -- 172.21.15.47:0/2695580923 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffab00a4850 con 0x55e5fdb59c00 2024-03-21T11:47:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.938+0000 7ffaab400640 1 -- 172.21.15.47:0/2695580923 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffac006b020 con 0x55e5fdb59c00 2024-03-21T11:47:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.938+0000 7ffacf785740 10 client.22416 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.938+0000 7ffacf785740 10 client.22416 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:47:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.938+0000 7ffacf785740 20 client.22416 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.938+0000 7ffacf785740 1 --2- 172.21.15.47:0/2695580923 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e5fdb603a0 0x55e5fdbd2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.938+0000 7ffacf785740 4 client.22416 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:47:57.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.938+0000 7ffacf785740 1 -- 172.21.15.47:0/2695580923 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e5fdb686c0 con 0x55e5fdb603a0 2024-03-21T11:47:57.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.940+0000 7ffac5400640 1 --2- 172.21.15.47:0/2695580923 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e5fdb603a0 0x55e5fdbd2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:57.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.943+0000 7ffaab400640 10 client.22416 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:47:57.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.943+0000 7ffac5400640 1 --2- 172.21.15.47:0/2695580923 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e5fdb603a0 0x55e5fdbd2710 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2df1ab2cf274346f server_cookie=89b7fdcff61326ef in_seq=0 out_seq=0 2024-03-21T11:47:57.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.944+0000 7ffaab400640 1 -- 172.21.15.47:0/2695580923 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e5fdb686c0 con 0x55e5fdb603a0 2024-03-21T11:47:57.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.944+0000 7ffaab400640 10 client.22416 handle_command_reply: tid=0 2024-03-21T11:47:57.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.944+0000 7ffacf785740 1 client.22416 shutdown 2024-03-21T11:47:57.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.944+0000 7ffacf785740 1 -- 172.21.15.47:0/2695580923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb59c00 msgr2=0x55e5fdb6bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.944+0000 7ffacf785740 1 --2- 172.21.15.47:0/2695580923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb59c00 0x55e5fdb6bf10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffac0046640 tx=0x7ffac0032a30 comp rx=0 tx=0).stop 2024-03-21T11:47:57.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.944+0000 7ffacf785740 1 -- 172.21.15.47:0/2695580923 shutdown_connections 2024-03-21T11:47:57.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.944+0000 7ffacf785740 1 --2- 172.21.15.47:0/2695580923 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55e5fdb603a0 0x55e5fdbd2710 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.944+0000 7ffacf785740 1 --2- 172.21.15.47:0/2695580923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5fdb5f9e0 0x55e5fdb6f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.944+0000 7ffacf785740 1 --2- 172.21.15.47:0/2695580923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e5fdb5e810 0x55e5fdb6c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.944+0000 7ffacf785740 1 --2- 172.21.15.47:0/2695580923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e5fdb59c00 0x55e5fdb6bf10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.944+0000 7ffaab400640 0 client.22416 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:47:57.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.944+0000 7ffacf785740 1 -- 172.21.15.47:0/2695580923 >> 172.21.15.47:0/2695580923 conn(0x55e5fdb76810 msgr2=0x55e5fdb5d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.945+0000 7ffacf785740 1 -- 172.21.15.47:0/2695580923 shutdown_connections 2024-03-21T11:47:57.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.945+0000 7ffacf785740 1 -- 172.21.15.47:0/2695580923 wait complete. 2024-03-21T11:47:57.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.945+0000 7ffacf785740 20 client.22416 trim_cache size 0 max 16384 2024-03-21T11:47:57.948 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:47:57.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.948+0000 7ffacca00640 1 -- 172.21.15.47:0/3502487806 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffaac0a2eb0 msgr2=0x7ffaac0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.948+0000 7ffacca00640 1 --2- 172.21.15.47:0/3502487806 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffaac0a2eb0 0x7ffaac0a5370 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ffac0005c70 tx=0x7ffac002e040 comp rx=0 tx=0).stop 2024-03-21T11:47:57.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.948+0000 7ffacca00640 1 -- 172.21.15.47:0/3502487806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffac8109000 msgr2=0x7ffac8109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:57.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.948+0000 7ffacca00640 1 --2- 172.21.15.47:0/3502487806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffac8109000 0x7ffac8109970 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ffab000e9f0 tx=0x7ffab000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.948+0000 7ffacca00640 1 -- 172.21.15.47:0/3502487806 shutdown_connections 2024-03-21T11:47:57.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.948+0000 7ffacca00640 1 --2- 172.21.15.47:0/3502487806 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffaac0a2eb0 0x7ffaac0a5370 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.948+0000 7ffacca00640 1 --2- 172.21.15.47:0/3502487806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac810ff20 0x7ffac8109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.948+0000 7ffacca00640 1 --2- 172.21.15.47:0/3502487806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffac8109000 0x7ffac8109970 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.948+0000 7ffacca00640 1 --2- 172.21.15.47:0/3502487806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac81086a0 0x7ffac810b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:57.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.948+0000 7ffacca00640 1 -- 172.21.15.47:0/3502487806 >> 172.21.15.47:0/3502487806 conn(0x7ffac806e500 msgr2=0x7ffac80701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:57.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.949+0000 7ffacca00640 1 -- 172.21.15.47:0/3502487806 shutdown_connections 2024-03-21T11:47:57.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:57.949+0000 7ffacca00640 1 -- 172.21.15.47:0/3502487806 wait complete. 2024-03-21T11:47:57.971 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:47:57.971 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:47:57.998 INFO:tasks.workunit.client.0.smithi047.stdout: 10 268403 21.71 MB/sec execute 521 sec latency 81.468 ms 2024-03-21T11:47:58.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.204+0000 7f70a6200640 1 -- 172.21.15.47:0/2153658878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a01016b0 msgr2=0x7f70a0101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.204+0000 7f70a6200640 1 --2- 172.21.15.47:0/2153658878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a01016b0 0x7f70a0101ad0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f7090009900 tx=0x7f709002bc30 comp rx=0 tx=0).stop 2024-03-21T11:47:58.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.204+0000 7f70a6200640 1 -- 172.21.15.47:0/2153658878 shutdown_connections 2024-03-21T11:47:58.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.204+0000 7f70a6200640 1 --2- 172.21.15.47:0/2153658878 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70a0103260 0x7f70a0109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.204+0000 7f70a6200640 1 --2- 172.21.15.47:0/2153658878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70a01028a0 0x7f70a0102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.204+0000 7f70a6200640 1 --2- 172.21.15.47:0/2153658878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a01016b0 0x7f70a0101ad0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.204+0000 7f70a6200640 1 -- 172.21.15.47:0/2153658878 >> 172.21.15.47:0/2153658878 conn(0x7f70a00fcc20 msgr2=0x7f70a00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:58.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.205+0000 7f70a6200640 1 -- 172.21.15.47:0/2153658878 shutdown_connections 2024-03-21T11:47:58.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.205+0000 7f70a6200640 1 -- 172.21.15.47:0/2153658878 wait complete. 2024-03-21T11:47:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.206+0000 7f70a6200640 1 Processor -- start 2024-03-21T11:47:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.206+0000 7f70a6200640 1 -- start start 2024-03-21T11:47:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.206+0000 7f70a6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70a01016b0 0x7f70a0197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.206+0000 7f70a6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a01028a0 0x7f70a0197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.206+0000 7f70a6200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70a0103260 0x7f70a019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.206+0000 7f70a6200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a010d300 con 0x7f70a0103260 2024-03-21T11:47:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.206+0000 7f70a6200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a010d180 con 0x7f70a01028a0 2024-03-21T11:47:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.206+0000 7f70a6200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a010d480 con 0x7f70a01016b0 2024-03-21T11:47:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.206+0000 7f709f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70a01016b0 0x7f70a0197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.206+0000 7f709ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a01028a0 0x7f70a0197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.206+0000 7f709f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70a01016b0 0x7f70a0197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54352/0 (socket says 172.21.15.47:54352) 2024-03-21T11:47:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.206+0000 7f709f400640 1 -- 172.21.15.47:0/3931847669 learned_addr learned my addr 172.21.15.47:0/3931847669 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.207+0000 7f709fe00640 1 --2- 172.21.15.47:0/3931847669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70a0103260 0x7f70a019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.207+0000 7f709f400640 1 -- 172.21.15.47:0/3931847669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a01028a0 msgr2=0x7f70a0197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.207+0000 7f709f400640 1 --2- 172.21.15.47:0/3931847669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a01028a0 0x7f70a0197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.207+0000 7f709f400640 1 -- 172.21.15.47:0/3931847669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70a0103260 msgr2=0x7f70a019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.207+0000 7f709f400640 1 --2- 172.21.15.47:0/3931847669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70a0103260 0x7f70a019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.207+0000 7f709f400640 1 -- 172.21.15.47:0/3931847669 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a019f620 con 0x7f70a01016b0 2024-03-21T11:47:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.207+0000 7f709f400640 1 --2- 172.21.15.47:0/3931847669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70a01016b0 0x7f70a0197a50 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f7090000c00 tx=0x7f709002c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:58.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.208+0000 7f7097e00640 1 -- 172.21.15.47:0/3931847669 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090041070 con 0x7f70a01016b0 2024-03-21T11:47:58.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.208+0000 7f7097e00640 1 -- 172.21.15.47:0/3931847669 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7090004450 con 0x7f70a01016b0 2024-03-21T11:47:58.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.208+0000 7f70a6200640 1 -- 172.21.15.47:0/3931847669 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a019f8b0 con 0x7f70a01016b0 2024-03-21T11:47:58.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.208+0000 7f7097e00640 1 -- 172.21.15.47:0/3931847669 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090034440 con 0x7f70a01016b0 2024-03-21T11:47:58.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.208+0000 7f70a6200640 1 -- 172.21.15.47:0/3931847669 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a019fda0 con 0x7f70a01016b0 2024-03-21T11:47:58.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.212+0000 7f7097e00640 1 -- 172.21.15.47:0/3931847669 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7090044070 con 0x7f70a01016b0 2024-03-21T11:47:58.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.214+0000 7f7097e00640 1 --2- 172.21.15.47:0/3931847669 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f70840a2f20 0x7f70840a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.214+0000 7f7097e00640 1 -- 172.21.15.47:0/3931847669 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f70900f1300 con 0x7f70a01016b0 2024-03-21T11:47:58.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.216+0000 7f709ea00640 1 --2- 172.21.15.47:0/3931847669 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f70840a2f20 0x7f70840a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.217+0000 7f709ea00640 1 --2- 172.21.15.47:0/3931847669 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f70840a2f20 0x7f70840a53e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f70a0198f10 tx=0x7f708c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:58.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.217+0000 7f70a8df5740 1 Processor -- start 2024-03-21T11:47:58.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.218+0000 7f70a8df5740 1 -- start start 2024-03-21T11:47:58.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.218+0000 7f70a8df5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0f5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.218+0000 7f70a8df5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0f6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.218+0000 7f70a8df5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f67b0 0x55ffbb0fd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.218+0000 7f70a8df5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbb0552c0 con 0x55ffbb0f5df0 2024-03-21T11:47:58.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.218+0000 7f70a8df5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbaffdc90 con 0x55ffbb0f67b0 2024-03-21T11:47:58.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.218+0000 7f70a8df5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbb051360 con 0x55ffbb0f4c20 2024-03-21T11:47:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.219+0000 7f709f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.219+0000 7f709f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0f5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54368/0 (socket says 172.21.15.47:54368) 2024-03-21T11:47:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.219+0000 7f709f400640 1 -- 172.21.15.47:0/3786475785 learned_addr learned my addr 172.21.15.47:0/3786475785 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.219+0000 7f709f400640 1 -- 172.21.15.47:0/3786475785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f67b0 msgr2=0x55ffbb0fd040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:47:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.219+0000 7f709fe00640 1 --2- 172.21.15.47:0/3786475785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f67b0 0x55ffbb0fd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.219+0000 7f709f400640 1 --2- 172.21.15.47:0/3786475785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f67b0 0x55ffbb0fd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.219+0000 7f709f400640 1 -- 172.21.15.47:0/3786475785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 msgr2=0x55ffbb0f6270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:47:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.219+0000 7f709ea00640 1 --2- 172.21.15.47:0/3786475785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.219+0000 7f709f400640 1 --2- 172.21.15.47:0/3786475785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0f6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.219+0000 7f709f400640 1 -- 172.21.15.47:0/3786475785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffbb022080 con 0x55ffbb0f4c20 2024-03-21T11:47:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.220+0000 7f709ea00640 1 --2- 172.21.15.47:0/3786475785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.220+0000 7f709f400640 1 --2- 172.21.15.47:0/3786475785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0f5040 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f70900bd430 tx=0x7f7090005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df6966b404098355 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.220+0000 7f708be00640 1 -- 172.21.15.47:0/3786475785 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709004b070 con 0x55ffbb0f4c20 2024-03-21T11:47:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.220+0000 7f708be00640 1 -- 172.21.15.47:0/3786475785 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7090004c80 con 0x55ffbb0f4c20 2024-03-21T11:47:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.220+0000 7f708be00640 1 -- 172.21.15.47:0/3786475785 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70900bba30 con 0x55ffbb0f4c20 2024-03-21T11:47:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.221+0000 7f70a8df5740 1 -- 172.21.15.47:0/3786475785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f4c20 msgr2=0x55ffbb0f5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.221+0000 7f70a8df5740 1 --2- 172.21.15.47:0/3786475785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0f5040 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f70900bd430 tx=0x7f7090005e50 comp rx=0 tx=0).stop 2024-03-21T11:47:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.222+0000 7f70a8df5740 1 -- 172.21.15.47:0/3786475785 shutdown_connections 2024-03-21T11:47:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.222+0000 7f70a8df5740 1 --2- 172.21.15.47:0/3786475785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f67b0 0x55ffbb0fd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.222+0000 7f70a8df5740 1 --2- 172.21.15.47:0/3786475785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.222+0000 7f70a8df5740 1 --2- 172.21.15.47:0/3786475785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0f5040 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.222+0000 7f70a8df5740 1 -- 172.21.15.47:0/3786475785 >> 172.21.15.47:0/3786475785 conn(0x55ffbb0f0010 msgr2=0x55ffbb0f0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.222+0000 7f70a8df5740 1 -- 172.21.15.47:0/3786475785 shutdown_connections 2024-03-21T11:47:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.222+0000 7f70a8df5740 1 -- 172.21.15.47:0/3786475785 wait complete. 2024-03-21T11:47:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.222+0000 7f70a8df5740 1 -- start start 2024-03-21T11:47:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.223+0000 7f70a8df5740 1 Processor -- start 2024-03-21T11:47:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.223+0000 7f70a8df5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.223+0000 7f70a8df5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0facf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.223+0000 7f70a8df5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f67b0 0x55ffbb0fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.223+0000 7f70a8df5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbb0eb690 con 0x55ffbb0f4c20 2024-03-21T11:47:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.223+0000 7f70a8df5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbb0110d0 con 0x55ffbb0f5df0 2024-03-21T11:47:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.223+0000 7f70a8df5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbb0eb810 con 0x55ffbb0f67b0 2024-03-21T11:47:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.223+0000 7f70a8df5740 10 client.0 _ll_register_callbacks cb 0x55ffbb015e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.224+0000 7f709ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0facf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.224+0000 7f709ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0facf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40184/0 (socket says 172.21.15.47:40184) 2024-03-21T11:47:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.224+0000 7f709ea00640 1 -- 172.21.15.47:0/422963699 learned_addr learned my addr 172.21.15.47:0/422963699 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.224+0000 7f709fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f67b0 0x55ffbb0fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.224+0000 7f709f400640 1 --2- 172.21.15.47:0/422963699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.224+0000 7f709ea00640 1 -- 172.21.15.47:0/422963699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f67b0 msgr2=0x55ffbb0fbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.224+0000 7f709ea00640 1 --2- 172.21.15.47:0/422963699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f67b0 0x55ffbb0fbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.224+0000 7f709ea00640 1 -- 172.21.15.47:0/422963699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f4c20 msgr2=0x55ffbb0fa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.224+0000 7f709ea00640 1 --2- 172.21.15.47:0/422963699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0fa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.224+0000 7f709ea00640 1 -- 172.21.15.47:0/422963699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffbad693f0 con 0x55ffbb0f5df0 2024-03-21T11:47:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.225+0000 7f709ea00640 1 --2- 172.21.15.47:0/422963699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0facf0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f708c00ed70 tx=0x7f708c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17a2970d084baaa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.225+0000 7f7088c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.225+0000 7f7088c00640 1 -- 172.21.15.47:0/422963699 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c004220 con 0x55ffbb0f5df0 2024-03-21T11:47:58.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.225+0000 7f70a8df5740 1 -- 172.21.15.47:0/422963699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7090004c80 con 0x55ffbb0f5df0 2024-03-21T11:47:58.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.225+0000 7f7088c00640 1 -- 172.21.15.47:0/422963699 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f708c0043c0 con 0x55ffbb0f5df0 2024-03-21T11:47:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.225+0000 7f7088c00640 1 -- 172.21.15.47:0/422963699 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f708c030050 con 0x55ffbb0f5df0 2024-03-21T11:47:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.226+0000 7f70a8df5740 10 client.33299 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.226+0000 7f70a8df5740 1 -- 172.21.15.47:0/422963699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ffbae337d0 con 0x55ffbb0f5df0 2024-03-21T11:47:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.226+0000 7f7088c00640 1 -- 172.21.15.47:0/422963699 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7090004c80 con 0x55ffbb0f5df0 2024-03-21T11:47:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.226+0000 7f7088c00640 1 -- 172.21.15.47:0/422963699 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f708c024030 con 0x55ffbb0f5df0 2024-03-21T11:47:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.227+0000 7f70a8df5740 10 client.33299 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.227+0000 7f70a8df5740 10 client.33299 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:47:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.227+0000 7f70a8df5740 20 client.33299 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.227+0000 7f70a8df5740 1 --2- 172.21.15.47:0/422963699 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ffbb10e9e0 0x55ffbb110e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.227+0000 7f70a8df5740 4 client.33299 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.227+0000 7f70a8df5740 1 -- 172.21.15.47:0/422963699 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffbb062800 con 0x55ffbb10e9e0 2024-03-21T11:47:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.227+0000 7f709f400640 1 --2- 172.21.15.47:0/422963699 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ffbb10e9e0 0x55ffbb110e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.228+0000 7f709f400640 1 --2- 172.21.15.47:0/422963699 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ffbb10e9e0 0x55ffbb110e20 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef163d2afdbe76cf server_cookie=a7f4c5f7ce119983 in_seq=0 out_seq=0 2024-03-21T11:47:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.228+0000 7f7088c00640 10 client.33299 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:47:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f7088c00640 1 -- 172.21.15.47:0/422963699 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ffbb062800 con 0x55ffbb10e9e0 2024-03-21T11:47:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f7088c00640 10 client.33299 handle_command_reply: tid=0 2024-03-21T11:47:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f70a8df5740 1 client.33299 shutdown 2024-03-21T11:47:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f70a8df5740 1 -- 172.21.15.47:0/422963699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f5df0 msgr2=0x55ffbb0facf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f70a8df5740 1 --2- 172.21.15.47:0/422963699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0facf0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f708c00ed70 tx=0x7f708c004730 comp rx=0 tx=0).stop 2024-03-21T11:47:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f70a8df5740 1 -- 172.21.15.47:0/422963699 shutdown_connections 2024-03-21T11:47:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f70a8df5740 1 --2- 172.21.15.47:0/422963699 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ffbb10e9e0 0x55ffbb110e20 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f70a8df5740 1 --2- 172.21.15.47:0/422963699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f67b0 0x55ffbb0fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f70a8df5740 1 --2- 172.21.15.47:0/422963699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0facf0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f70a8df5740 1 --2- 172.21.15.47:0/422963699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f70a8df5740 1 -- 172.21.15.47:0/422963699 >> 172.21.15.47:0/422963699 conn(0x55ffbb0f0010 msgr2=0x55ffbb0eaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f7088c00640 0 client.33299 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:47:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f70a8df5740 1 -- 172.21.15.47:0/422963699 shutdown_connections 2024-03-21T11:47:58.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f70a8df5740 1 -- 172.21.15.47:0/422963699 wait complete. 2024-03-21T11:47:58.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.229+0000 7f70a8df5740 20 client.33299 trim_cache size 0 max 16384 2024-03-21T11:47:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.246+0000 7f70a8df5740 1 Processor -- start 2024-03-21T11:47:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.246+0000 7f70a8df5740 1 -- start start 2024-03-21T11:47:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.247+0000 7f70a8df5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0edab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.247+0000 7f70a8df5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0edff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.247+0000 7f70a8df5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f67b0 0x55ffbb0ee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.247+0000 7f70a8df5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01a7f00 con 0x55ffbb0f5df0 2024-03-21T11:47:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.247+0000 7f70a8df5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01a9880 con 0x55ffbb0f4c20 2024-03-21T11:47:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.247+0000 7f70a8df5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01a8560 con 0x55ffbb0f67b0 2024-03-21T11:47:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.247+0000 7f709f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0edab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.247+0000 7f709f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0edab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40198/0 (socket says 172.21.15.47:40198) 2024-03-21T11:47:58.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.247+0000 7f709f400640 1 -- 172.21.15.47:0/2674173047 learned_addr learned my addr 172.21.15.47:0/2674173047 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:58.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.247+0000 7f709fe00640 1 --2- 172.21.15.47:0/2674173047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f67b0 0x55ffbb0ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.247+0000 7f709ea00640 1 --2- 172.21.15.47:0/2674173047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0edff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.248+0000 7f709f400640 1 -- 172.21.15.47:0/2674173047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f67b0 msgr2=0x55ffbb0ee530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.248+0000 7f709f400640 1 --2- 172.21.15.47:0/2674173047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f67b0 0x55ffbb0ee530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.248+0000 7f709f400640 1 -- 172.21.15.47:0/2674173047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 msgr2=0x55ffbb0edff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.248+0000 7f709f400640 1 --2- 172.21.15.47:0/2674173047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0edff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.248+0000 7f709f400640 1 -- 172.21.15.47:0/2674173047 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffbb0f0d80 con 0x55ffbb0f4c20 2024-03-21T11:47:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.248+0000 7f709f400640 1 --2- 172.21.15.47:0/2674173047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0edab0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f70900eeac0 tx=0x7f709002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaf7c5f235edd11b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.248+0000 7f708b400640 1 -- 172.21.15.47:0/2674173047 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70900db070 con 0x55ffbb0f4c20 2024-03-21T11:47:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.249+0000 7f708b400640 1 -- 172.21.15.47:0/2674173047 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7090004c80 con 0x55ffbb0f4c20 2024-03-21T11:47:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.249+0000 7f708b400640 1 -- 172.21.15.47:0/2674173047 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709004b040 con 0x55ffbb0f4c20 2024-03-21T11:47:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.250+0000 7f70a8df5740 1 -- 172.21.15.47:0/2674173047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f4c20 msgr2=0x55ffbb0edab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.250+0000 7f70a8df5740 1 --2- 172.21.15.47:0/2674173047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0edab0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f70900eeac0 tx=0x7f709002bc30 comp rx=0 tx=0).stop 2024-03-21T11:47:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.250+0000 7f70a8df5740 1 -- 172.21.15.47:0/2674173047 shutdown_connections 2024-03-21T11:47:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.250+0000 7f70a8df5740 1 --2- 172.21.15.47:0/2674173047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f67b0 0x55ffbb0ee530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.250+0000 7f70a8df5740 1 --2- 172.21.15.47:0/2674173047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 0x55ffbb0edff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.250+0000 7f70a8df5740 1 --2- 172.21.15.47:0/2674173047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f4c20 0x55ffbb0edab0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.250+0000 7f70a8df5740 1 -- 172.21.15.47:0/2674173047 >> 172.21.15.47:0/2674173047 conn(0x55ffbb0f0010 msgr2=0x55ffbb0f3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.250+0000 7f70a8df5740 1 -- 172.21.15.47:0/2674173047 shutdown_connections 2024-03-21T11:47:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.250+0000 7f70a8df5740 1 -- 172.21.15.47:0/2674173047 wait complete. 2024-03-21T11:47:58.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.250+0000 7f70a8df5740 1 -- start start 2024-03-21T11:47:58.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.251+0000 7f70a8df5740 1 Processor -- start 2024-03-21T11:47:58.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.251+0000 7f70a8df5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f4c20 0x55ffbb107560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.251+0000 7f70a8df5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 0x55ffbb107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.251+0000 7f70a8df5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f67b0 0x55ffbb10aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.252+0000 7f709f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f4c20 0x55ffbb107560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.252+0000 7f709f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f4c20 0x55ffbb107560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54386/0 (socket says 172.21.15.47:54386) 2024-03-21T11:47:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.252+0000 7f709f400640 1 -- 172.21.15.47:0/3637887943 learned_addr learned my addr 172.21.15.47:0/3637887943 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.252+0000 7f70a8df5740 1 -- 172.21.15.47:0/3637887943 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01a9220 con 0x55ffbb0f5df0 2024-03-21T11:47:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.252+0000 7f70a8df5740 1 -- 172.21.15.47:0/3637887943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01a8bc0 con 0x55ffbb0f67b0 2024-03-21T11:47:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.252+0000 7f70a8df5740 1 -- 172.21.15.47:0/3637887943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01a60c0 con 0x55ffbb0f4c20 2024-03-21T11:47:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.252+0000 7f709fe00640 1 --2- 172.21.15.47:0/3637887943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f67b0 0x55ffbb10aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.252+0000 7f70a8df5740 10 client.0 _ll_register_callbacks cb 0x7f708c00e9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.252+0000 7f709fe00640 1 -- 172.21.15.47:0/3637887943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f4c20 msgr2=0x55ffbb107560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.252+0000 7f709ea00640 1 --2- 172.21.15.47:0/3637887943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 0x55ffbb107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.252+0000 7f709fe00640 1 --2- 172.21.15.47:0/3637887943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f4c20 0x55ffbb107560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.252+0000 7f709fe00640 1 -- 172.21.15.47:0/3637887943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 msgr2=0x55ffbb107aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.252+0000 7f709fe00640 1 --2- 172.21.15.47:0/3637887943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 0x55ffbb107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.252+0000 7f709fe00640 1 -- 172.21.15.47:0/3637887943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffbb10aff0 con 0x55ffbb0f67b0 2024-03-21T11:47:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.253+0000 7f709f400640 1 --2- 172.21.15.47:0/3637887943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f4c20 0x55ffbb107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.253+0000 7f709fe00640 1 --2- 172.21.15.47:0/3637887943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f67b0 0x55ffbb10aa00 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x55ffbb0f0f80 tx=0x7f7098012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=642ad8b56148f24f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.253+0000 7f7088c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.253+0000 7f7088c00640 1 -- 172.21.15.47:0/3637887943 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098021060 con 0x55ffbb0f67b0 2024-03-21T11:47:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.253+0000 7f7088c00640 1 -- 172.21.15.47:0/3637887943 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7098011af0 con 0x55ffbb0f67b0 2024-03-21T11:47:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.253+0000 7f70a8df5740 1 -- 172.21.15.47:0/3637887943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7090004c80 con 0x55ffbb0f67b0 2024-03-21T11:47:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.254+0000 7f7088c00640 1 -- 172.21.15.47:0/3637887943 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7098011cf0 con 0x55ffbb0f67b0 2024-03-21T11:47:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.254+0000 7f70a8df5740 10 client.33305 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.254+0000 7f70a8df5740 1 -- 172.21.15.47:0/3637887943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ffbb16a810 con 0x55ffbb0f67b0 2024-03-21T11:47:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.255+0000 7f7088c00640 1 -- 172.21.15.47:0/3637887943 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7090004c80 con 0x55ffbb0f67b0 2024-03-21T11:47:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.255+0000 7f7088c00640 1 -- 172.21.15.47:0/3637887943 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f709801b030 con 0x55ffbb0f67b0 2024-03-21T11:47:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.255+0000 7f70a8df5740 10 client.33305 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.255+0000 7f70a8df5740 10 client.33305 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:47:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.255+0000 7f70a8df5740 20 client.33305 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.255+0000 7f70a8df5740 1 --2- 172.21.15.47:0/3637887943 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ffbb16b180 0x55ffbb16d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.255+0000 7f70a8df5740 4 client.33305 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:47:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.255+0000 7f70a8df5740 1 -- 172.21.15.47:0/3637887943 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ffbb05cb50 con 0x55ffbb16b180 2024-03-21T11:47:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.255+0000 7f709f400640 1 --2- 172.21.15.47:0/3637887943 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ffbb16b180 0x55ffbb16d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.256+0000 7f709f400640 1 --2- 172.21.15.47:0/3637887943 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ffbb16b180 0x55ffbb16d5a0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=796734e167e67275 server_cookie=a931e3686fccf073 in_seq=0 out_seq=0 2024-03-21T11:47:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.256+0000 7f7088c00640 10 client.33305 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:47:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.256+0000 7f7088c00640 1 -- 172.21.15.47:0/3637887943 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ffbb05cb50 con 0x55ffbb16b180 2024-03-21T11:47:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.256+0000 7f7088c00640 10 client.33305 handle_command_reply: tid=0 2024-03-21T11:47:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.256+0000 7f70a8df5740 1 client.33305 shutdown 2024-03-21T11:47:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.256+0000 7f70a8df5740 1 -- 172.21.15.47:0/3637887943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f67b0 msgr2=0x55ffbb10aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.256+0000 7f70a8df5740 1 --2- 172.21.15.47:0/3637887943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f67b0 0x55ffbb10aa00 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x55ffbb0f0f80 tx=0x7f7098012b60 comp rx=0 tx=0).stop 2024-03-21T11:47:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.257+0000 7f70a8df5740 1 -- 172.21.15.47:0/3637887943 shutdown_connections 2024-03-21T11:47:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.257+0000 7f70a8df5740 1 --2- 172.21.15.47:0/3637887943 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55ffbb16b180 0x55ffbb16d5a0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.257+0000 7f70a8df5740 1 --2- 172.21.15.47:0/3637887943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffbb0f67b0 0x55ffbb10aa00 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.257+0000 7f70a8df5740 1 --2- 172.21.15.47:0/3637887943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ffbb0f5df0 0x55ffbb107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.257+0000 7f70a8df5740 1 --2- 172.21.15.47:0/3637887943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ffbb0f4c20 0x55ffbb107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.257+0000 7f70a8df5740 1 -- 172.21.15.47:0/3637887943 >> 172.21.15.47:0/3637887943 conn(0x55ffbb0f0010 msgr2=0x55ffbb0ea620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.257+0000 7f7088c00640 0 client.33305 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:47:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.257+0000 7f70a8df5740 1 -- 172.21.15.47:0/3637887943 shutdown_connections 2024-03-21T11:47:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.257+0000 7f70a8df5740 1 -- 172.21.15.47:0/3637887943 wait complete. 2024-03-21T11:47:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.257+0000 7f70a8df5740 20 client.33305 trim_cache size 0 max 16384 2024-03-21T11:47:58.260 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:47:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.258+0000 7f70a6200640 1 -- 172.21.15.47:0/3931847669 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f70840a2f20 msgr2=0x7f70840a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.258+0000 7f70a6200640 1 --2- 172.21.15.47:0/3931847669 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f70840a2f20 0x7f70840a53e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f70a0198f10 tx=0x7f708c005f90 comp rx=0 tx=0).stop 2024-03-21T11:47:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.258+0000 7f70a6200640 1 -- 172.21.15.47:0/3931847669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70a01016b0 msgr2=0x7f70a0197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.258+0000 7f70a6200640 1 --2- 172.21.15.47:0/3931847669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70a01016b0 0x7f70a0197a50 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f7090000c00 tx=0x7f709002c640 comp rx=0 tx=0).stop 2024-03-21T11:47:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.258+0000 7f70a6200640 1 -- 172.21.15.47:0/3931847669 shutdown_connections 2024-03-21T11:47:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.258+0000 7f70a6200640 1 --2- 172.21.15.47:0/3931847669 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f70840a2f20 0x7f70840a53e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.258+0000 7f70a6200640 1 --2- 172.21.15.47:0/3931847669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70a0103260 0x7f70a019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.258+0000 7f70a6200640 1 --2- 172.21.15.47:0/3931847669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a01028a0 0x7f70a0197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.258+0000 7f70a6200640 1 --2- 172.21.15.47:0/3931847669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70a01016b0 0x7f70a0197a50 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.258+0000 7f70a6200640 1 -- 172.21.15.47:0/3931847669 >> 172.21.15.47:0/3931847669 conn(0x7f70a00fcc20 msgr2=0x7f70a0105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.258+0000 7f70a6200640 1 -- 172.21.15.47:0/3931847669 shutdown_connections 2024-03-21T11:47:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.258+0000 7f70a6200640 1 -- 172.21.15.47:0/3931847669 wait complete. 2024-03-21T11:47:58.278 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:47:58.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:47:58.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.538+0000 7f84d0800640 1 -- 172.21.15.47:0/2841700710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c810d2a0 msgr2=0x7f84c810d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.538+0000 7f84d0800640 1 --2- 172.21.15.47:0/2841700710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c810d2a0 0x7f84c810d720 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009900 tx=0x7f84b802bc30 comp rx=0 tx=0).stop 2024-03-21T11:47:58.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.538+0000 7f84d0800640 1 -- 172.21.15.47:0/2841700710 shutdown_connections 2024-03-21T11:47:58.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.538+0000 7f84d0800640 1 --2- 172.21.15.47:0/2841700710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84c810dc60 0x7f84c81144f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.538+0000 7f84d0800640 1 --2- 172.21.15.47:0/2841700710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c810d2a0 0x7f84c810d720 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.538+0000 7f84d0800640 1 --2- 172.21.15.47:0/2841700710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c810c0b0 0x7f84c810c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.538+0000 7f84d0800640 1 -- 172.21.15.47:0/2841700710 >> 172.21.15.47:0/2841700710 conn(0x7f84c806c060 msgr2=0x7f84c806c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:58.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.538+0000 7f84d0800640 1 -- 172.21.15.47:0/2841700710 shutdown_connections 2024-03-21T11:47:58.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.538+0000 7f84d0800640 1 -- 172.21.15.47:0/2841700710 wait complete. 2024-03-21T11:47:58.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.539+0000 7f84d0800640 1 Processor -- start 2024-03-21T11:47:58.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.539+0000 7f84d0800640 1 -- start start 2024-03-21T11:47:58.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.539+0000 7f84d0800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84c810c0b0 0x7f84c81a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84d0800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c810d2a0 0x7f84c81a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84d0800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c810dc60 0x7f84c81a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84d0800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c8117df0 con 0x7f84c810c0b0 2024-03-21T11:47:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84d0800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c8117c70 con 0x7f84c810d2a0 2024-03-21T11:47:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84d0800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c8117f70 con 0x7f84c810dc60 2024-03-21T11:47:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84cde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84c810c0b0 0x7f84c81a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84cd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c810d2a0 0x7f84c81a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84cd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c810d2a0 0x7f84c81a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40224/0 (socket says 172.21.15.47:40224) 2024-03-21T11:47:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84ce800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c810dc60 0x7f84c81a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84cd400640 1 -- 172.21.15.47:0/1061159332 learned_addr learned my addr 172.21.15.47:0/1061159332 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84cd400640 1 -- 172.21.15.47:0/1061159332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c810dc60 msgr2=0x7f84c81a9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84cd400640 1 --2- 172.21.15.47:0/1061159332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c810dc60 0x7f84c81a9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84cd400640 1 -- 172.21.15.47:0/1061159332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84c810c0b0 msgr2=0x7f84c81a24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84cd400640 1 --2- 172.21.15.47:0/1061159332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84c810c0b0 0x7f84c81a24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84cd400640 1 -- 172.21.15.47:0/1061159332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c81aa0e0 con 0x7f84c810d2a0 2024-03-21T11:47:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.540+0000 7f84cde00640 1 --2- 172.21.15.47:0/1061159332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84c810c0b0 0x7f84c81a24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:58.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.541+0000 7f84cd400640 1 --2- 172.21.15.47:0/1061159332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c810d2a0 0x7f84c81a2a00 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005ab0 tx=0x7f84b802c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:58.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.541+0000 7f84c2a00640 1 -- 172.21.15.47:0/1061159332 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8043020 con 0x7f84c810d2a0 2024-03-21T11:47:58.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.541+0000 7f84d0800640 1 -- 172.21.15.47:0/1061159332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c81aa370 con 0x7f84c810d2a0 2024-03-21T11:47:58.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.541+0000 7f84c2a00640 1 -- 172.21.15.47:0/1061159332 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84b80054e0 con 0x7f84c810d2a0 2024-03-21T11:47:58.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.541+0000 7f84c2a00640 1 -- 172.21.15.47:0/1061159332 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8003c70 con 0x7f84c810d2a0 2024-03-21T11:47:58.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.541+0000 7f84d0800640 1 -- 172.21.15.47:0/1061159332 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c81aa8c0 con 0x7f84c810d2a0 2024-03-21T11:47:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.545+0000 7f84c2a00640 1 -- 172.21.15.47:0/1061159332 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f84b8046020 con 0x7f84c810d2a0 2024-03-21T11:47:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.545+0000 7f84d3606740 1 Processor -- start 2024-03-21T11:47:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.545+0000 7f84d3606740 1 -- start start 2024-03-21T11:47:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.545+0000 7f84d3606740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de4810 0x558454de4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.545+0000 7f84d3606740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de59e0 0x558454de5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.545+0000 7f84d3606740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de63a0 0x558454decc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.545+0000 7f84d3606740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558454d492c0 con 0x558454de4810 2024-03-21T11:47:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.545+0000 7f84d3606740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558454cf1c90 con 0x558454de63a0 2024-03-21T11:47:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.545+0000 7f84d3606740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558454d45360 con 0x558454de59e0 2024-03-21T11:47:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.546+0000 7f84cd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de63a0 0x558454decc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.546+0000 7f84cde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de59e0 0x558454de5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.546+0000 7f84cde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de59e0 0x558454de5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54408/0 (socket says 172.21.15.47:54408) 2024-03-21T11:47:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.546+0000 7f84cde00640 1 -- 172.21.15.47:0/8070198 learned_addr learned my addr 172.21.15.47:0/8070198 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.546+0000 7f84ce800640 1 --2- 172.21.15.47:0/8070198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de4810 0x558454de4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.546+0000 7f84cd400640 1 -- 172.21.15.47:0/8070198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de59e0 msgr2=0x558454de5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.546+0000 7f84cd400640 1 --2- 172.21.15.47:0/8070198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de59e0 0x558454de5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.546+0000 7f84cd400640 1 -- 172.21.15.47:0/8070198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de4810 msgr2=0x558454de4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.546+0000 7f84cd400640 1 --2- 172.21.15.47:0/8070198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de4810 0x558454de4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.546+0000 7f84cd400640 1 -- 172.21.15.47:0/8070198 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558454d16080 con 0x558454de63a0 2024-03-21T11:47:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.547+0000 7f84cd400640 1 --2- 172.21.15.47:0/8070198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de63a0 0x558454decc30 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f84b80ef030 tx=0x7f84b80f7b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0180c6d155a04 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.547+0000 7f84c2a00640 1 --2- 172.21.15.47:0/1061159332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84b00a2f20 0x7f84b00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.547+0000 7f84c2a00640 1 -- 172.21.15.47:0/1061159332 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f84b80f18f0 con 0x7f84c810d2a0 2024-03-21T11:47:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.547+0000 7f84b7000640 1 -- 172.21.15.47:0/8070198 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8100070 con 0x558454de63a0 2024-03-21T11:47:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.547+0000 7f84ce800640 1 --2- 172.21.15.47:0/1061159332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84b00a2f20 0x7f84b00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.547+0000 7f84b7000640 1 -- 172.21.15.47:0/8070198 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84b8004250 con 0x558454de63a0 2024-03-21T11:47:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.547+0000 7f84b7000640 1 -- 172.21.15.47:0/8070198 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b80bd7d0 con 0x558454de63a0 2024-03-21T11:47:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.548+0000 7f84d3606740 1 -- 172.21.15.47:0/8070198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de63a0 msgr2=0x558454decc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.548+0000 7f84d3606740 1 --2- 172.21.15.47:0/8070198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de63a0 0x558454decc30 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f84b80ef030 tx=0x7f84b80f7b40 comp rx=0 tx=0).stop 2024-03-21T11:47:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.548+0000 7f84ce800640 1 --2- 172.21.15.47:0/1061159332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84b00a2f20 0x7f84b00a53e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x558454de58e0 tx=0x7f84c4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.548+0000 7f84d3606740 1 -- 172.21.15.47:0/8070198 shutdown_connections 2024-03-21T11:47:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.548+0000 7f84d3606740 1 --2- 172.21.15.47:0/8070198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de63a0 0x558454decc30 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.548+0000 7f84d3606740 1 --2- 172.21.15.47:0/8070198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de59e0 0x558454de5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.548+0000 7f84d3606740 1 --2- 172.21.15.47:0/8070198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de4810 0x558454de4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.548+0000 7f84d3606740 1 -- 172.21.15.47:0/8070198 >> 172.21.15.47:0/8070198 conn(0x558454ddfc00 msgr2=0x558454de0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.548+0000 7f84d3606740 1 -- 172.21.15.47:0/8070198 shutdown_connections 2024-03-21T11:47:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.548+0000 7f84d3606740 1 -- 172.21.15.47:0/8070198 wait complete. 2024-03-21T11:47:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.549+0000 7f84d3606740 1 -- start start 2024-03-21T11:47:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.549+0000 7f84d3606740 1 Processor -- start 2024-03-21T11:47:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.549+0000 7f84d3606740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de4810 0x558454dee810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.549+0000 7f84d3606740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de59e0 0x558454deed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.549+0000 7f84d3606740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de63a0 0x558454df5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.549+0000 7f84d3606740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558454ded170 con 0x558454de59e0 2024-03-21T11:47:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.549+0000 7f84d3606740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558454d050d0 con 0x558454de63a0 2024-03-21T11:47:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.549+0000 7f84d3606740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558454ded2f0 con 0x558454de4810 2024-03-21T11:47:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.549+0000 7f84d3606740 10 client.0 _ll_register_callbacks cb 0x558454d09e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.549+0000 7f84ce800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de63a0 0x558454df5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.549+0000 7f84cd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de59e0 0x558454deed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.550+0000 7f84ce800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de63a0 0x558454df5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40234/0 (socket says 172.21.15.47:40234) 2024-03-21T11:47:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.550+0000 7f84ce800640 1 -- 172.21.15.47:0/3730390677 learned_addr learned my addr 172.21.15.47:0/3730390677 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.550+0000 7f84cde00640 1 --2- 172.21.15.47:0/3730390677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de4810 0x558454dee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.550+0000 7f84cd400640 1 -- 172.21.15.47:0/3730390677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de4810 msgr2=0x558454dee810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.550+0000 7f84cd400640 1 --2- 172.21.15.47:0/3730390677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de4810 0x558454dee810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.550+0000 7f84cd400640 1 -- 172.21.15.47:0/3730390677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de63a0 msgr2=0x558454df5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.550+0000 7f84cd400640 1 --2- 172.21.15.47:0/3730390677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de63a0 0x558454df5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.550+0000 7f84cd400640 1 -- 172.21.15.47:0/3730390677 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558454a5d3f0 con 0x558454de59e0 2024-03-21T11:47:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.550+0000 7f84ce800640 1 --2- 172.21.15.47:0/3730390677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de63a0 0x558454df5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.550+0000 7f84cd400640 1 --2- 172.21.15.47:0/3730390677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de59e0 0x558454deed50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f84b8006010 tx=0x7f84b80f7790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1ae33ca211bb5e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.550+0000 7f84ab400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:47:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.550+0000 7f84ab400640 1 -- 172.21.15.47:0/3730390677 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8100050 con 0x558454de59e0 2024-03-21T11:47:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.550+0000 7f84ab400640 1 -- 172.21.15.47:0/3730390677 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f84b8004c80 con 0x558454de59e0 2024-03-21T11:47:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.551+0000 7f84d3606740 1 -- 172.21.15.47:0/3730390677 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558454d56800 con 0x558454de59e0 2024-03-21T11:47:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.551+0000 7f84ab400640 1 -- 172.21.15.47:0/3730390677 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84b80bd5f0 con 0x558454de59e0 2024-03-21T11:47:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.551+0000 7f84d3606740 10 client.22440 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.551+0000 7f84d3606740 1 -- 172.21.15.47:0/3730390677 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558454b277d0 con 0x558454de59e0 2024-03-21T11:47:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.551+0000 7f84ab400640 1 -- 172.21.15.47:0/3730390677 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558454d56800 con 0x558454de59e0 2024-03-21T11:47:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.551+0000 7f84ab400640 1 -- 172.21.15.47:0/3730390677 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f84b80ba300 con 0x558454de59e0 2024-03-21T11:47:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.551+0000 7f84d3606740 10 client.22440 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.551+0000 7f84d3606740 10 client.22440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.551+0000 7f84d3606740 20 client.22440 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.551+0000 7f84d3606740 1 --2- 172.21.15.47:0/3730390677 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558454df9fb0 0x558454dfc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.551+0000 7f84d3606740 4 client.22440 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.551+0000 7f84d3606740 1 -- 172.21.15.47:0/3730390677 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558454d50b50 con 0x558454df9fb0 2024-03-21T11:47:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.552+0000 7f84cde00640 1 --2- 172.21.15.47:0/3730390677 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558454df9fb0 0x558454dfc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.552+0000 7f84cde00640 1 --2- 172.21.15.47:0/3730390677 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558454df9fb0 0x558454dfc3a0 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d610d2927cc7779 server_cookie=ccba0240dcef4c47 in_seq=0 out_seq=0 2024-03-21T11:47:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.552+0000 7f84ab400640 10 client.22440 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.555+0000 7f84ab400640 1 -- 172.21.15.47:0/3730390677 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558454d50b50 con 0x558454df9fb0 2024-03-21T11:47:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.556+0000 7f84ab400640 10 client.22440 handle_command_reply: tid=0 2024-03-21T11:47:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.556+0000 7f84d3606740 1 client.22440 shutdown 2024-03-21T11:47:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.556+0000 7f84d3606740 1 -- 172.21.15.47:0/3730390677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de59e0 msgr2=0x558454deed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.556+0000 7f84d3606740 1 --2- 172.21.15.47:0/3730390677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de59e0 0x558454deed50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f84b8006010 tx=0x7f84b80f7790 comp rx=0 tx=0).stop 2024-03-21T11:47:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.556+0000 7f84d3606740 1 -- 172.21.15.47:0/3730390677 shutdown_connections 2024-03-21T11:47:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.556+0000 7f84d3606740 1 --2- 172.21.15.47:0/3730390677 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558454df9fb0 0x558454dfc3a0 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.556+0000 7f84d3606740 1 --2- 172.21.15.47:0/3730390677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de63a0 0x558454df5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.556+0000 7f84d3606740 1 --2- 172.21.15.47:0/3730390677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de59e0 0x558454deed50 secure :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f84b8006010 tx=0x7f84b80f7790 comp rx=0 tx=0).stop 2024-03-21T11:47:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.556+0000 7f84ab400640 0 client.22440 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.556+0000 7f84d3606740 1 --2- 172.21.15.47:0/3730390677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de4810 0x558454dee810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.556+0000 7f84d3606740 1 -- 172.21.15.47:0/3730390677 >> 172.21.15.47:0/3730390677 conn(0x558454ddfc00 msgr2=0x558454ddeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.557+0000 7f84d3606740 1 -- 172.21.15.47:0/3730390677 shutdown_connections 2024-03-21T11:47:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.557+0000 7f84d3606740 1 -- 172.21.15.47:0/3730390677 wait complete. 2024-03-21T11:47:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.557+0000 7f84d3606740 20 client.22440 trim_cache size 0 max 16384 2024-03-21T11:47:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.580+0000 7f84d3606740 1 Processor -- start 2024-03-21T11:47:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.580+0000 7f84d3606740 1 -- start start 2024-03-21T11:47:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.580+0000 7f84d3606740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de4810 0x558454de3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.580+0000 7f84d3606740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de63a0 0x558454df5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.581+0000 7f84d3606740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454e00c60 0x558454e010f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.581+0000 7f84d3606740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c81b4b00 con 0x558454de4810 2024-03-21T11:47:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.581+0000 7f84d3606740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c81b6480 con 0x558454e00c60 2024-03-21T11:47:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.581+0000 7f84d3606740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c81b5160 con 0x558454de63a0 2024-03-21T11:47:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.581+0000 7f84cd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de63a0 0x558454df5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.581+0000 7f84cd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de63a0 0x558454df5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:54422/0 (socket says 172.21.15.47:54422) 2024-03-21T11:47:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.581+0000 7f84cd400640 1 -- 172.21.15.47:0/303046799 learned_addr learned my addr 172.21.15.47:0/303046799 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.582+0000 7f84ce800640 1 --2- 172.21.15.47:0/303046799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454e00c60 0x558454e010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.582+0000 7f84cde00640 1 --2- 172.21.15.47:0/303046799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de4810 0x558454de3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.582+0000 7f84cd400640 1 -- 172.21.15.47:0/303046799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454e00c60 msgr2=0x558454e010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.582+0000 7f84cd400640 1 --2- 172.21.15.47:0/303046799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454e00c60 0x558454e010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.582+0000 7f84cd400640 1 -- 172.21.15.47:0/303046799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de4810 msgr2=0x558454de3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.582+0000 7f84cd400640 1 --2- 172.21.15.47:0/303046799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de4810 0x558454de3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.582+0000 7f84cd400640 1 -- 172.21.15.47:0/303046799 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558454de5460 con 0x558454de63a0 2024-03-21T11:47:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.582+0000 7f84cde00640 1 --2- 172.21.15.47:0/303046799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de4810 0x558454de3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:47:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.583+0000 7f84cd400640 1 --2- 172.21.15.47:0/303046799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de63a0 0x558454df5d50 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f84b8000c00 tx=0x7f84b80f7800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb4a8ece2aa47508 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.583+0000 7f84b6600640 1 -- 172.21.15.47:0/303046799 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8084070 con 0x558454de63a0 2024-03-21T11:47:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.583+0000 7f84b6600640 1 -- 172.21.15.47:0/303046799 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84b80db070 con 0x558454de63a0 2024-03-21T11:47:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.583+0000 7f84b6600640 1 -- 172.21.15.47:0/303046799 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8100050 con 0x558454de63a0 2024-03-21T11:47:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.584+0000 7f84d3606740 1 -- 172.21.15.47:0/303046799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de63a0 msgr2=0x558454df5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.584+0000 7f84d3606740 1 --2- 172.21.15.47:0/303046799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de63a0 0x558454df5d50 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f84b8000c00 tx=0x7f84b80f7800 comp rx=0 tx=0).stop 2024-03-21T11:47:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.584+0000 7f84d3606740 1 -- 172.21.15.47:0/303046799 shutdown_connections 2024-03-21T11:47:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.584+0000 7f84d3606740 1 --2- 172.21.15.47:0/303046799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454e00c60 0x558454e010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.584+0000 7f84d3606740 1 --2- 172.21.15.47:0/303046799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454de63a0 0x558454df5d50 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.584+0000 7f84d3606740 1 --2- 172.21.15.47:0/303046799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de4810 0x558454de3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.584+0000 7f84d3606740 1 -- 172.21.15.47:0/303046799 >> 172.21.15.47:0/303046799 conn(0x558454ddfc00 msgr2=0x558454de2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.584+0000 7f84d3606740 1 -- 172.21.15.47:0/303046799 shutdown_connections 2024-03-21T11:47:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.584+0000 7f84d3606740 1 -- 172.21.15.47:0/303046799 wait complete. 2024-03-21T11:47:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.585+0000 7f84d3606740 1 -- start start 2024-03-21T11:47:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.585+0000 7f84d3606740 1 Processor -- start 2024-03-21T11:47:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.586+0000 7f84d3606740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de4810 0x558454df1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.586+0000 7f84d3606740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de63a0 0x558454df1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.586+0000 7f84d3606740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454e00c60 0x558454df4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.586+0000 7f84d3606740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c81b5e20 con 0x558454de63a0 2024-03-21T11:47:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.586+0000 7f84d3606740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c81b57c0 con 0x558454de4810 2024-03-21T11:47:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.586+0000 7f84d3606740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c81b2cc0 con 0x558454e00c60 2024-03-21T11:47:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.586+0000 7f84d3606740 10 client.0 _ll_register_callbacks cb 0x7f84c4009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.586+0000 7f84cde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de4810 0x558454df1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.586+0000 7f84cde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de4810 0x558454df1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40258/0 (socket says 172.21.15.47:40258) 2024-03-21T11:47:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.586+0000 7f84cde00640 1 -- 172.21.15.47:0/1427940352 learned_addr learned my addr 172.21.15.47:0/1427940352 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.586+0000 7f84ce800640 1 --2- 172.21.15.47:0/1427940352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454e00c60 0x558454df4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.586+0000 7f84cd400640 1 --2- 172.21.15.47:0/1427940352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de63a0 0x558454df1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.587+0000 7f84ce800640 1 -- 172.21.15.47:0/1427940352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de4810 msgr2=0x558454df1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.587+0000 7f84ce800640 1 --2- 172.21.15.47:0/1427940352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de4810 0x558454df1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.587+0000 7f84ce800640 1 -- 172.21.15.47:0/1427940352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de63a0 msgr2=0x558454df1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.587+0000 7f84ce800640 1 --2- 172.21.15.47:0/1427940352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de63a0 0x558454df1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.587+0000 7f84ce800640 1 -- 172.21.15.47:0/1427940352 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558454df55d0 con 0x558454e00c60 2024-03-21T11:47:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.587+0000 7f84cd400640 1 --2- 172.21.15.47:0/1427940352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de63a0 0x558454df1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.587+0000 7f84ce800640 1 --2- 172.21.15.47:0/1427940352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454e00c60 0x558454df4fe0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009fd0 tx=0x7f84c4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcac38ae2acd8342 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.587+0000 7f84ab400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:47:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.588+0000 7f84ab400640 1 -- 172.21.15.47:0/1427940352 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c402e060 con 0x558454e00c60 2024-03-21T11:47:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.588+0000 7f84d3606740 1 -- 172.21.15.47:0/1427940352 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84b80db070 con 0x558454e00c60 2024-03-21T11:47:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.588+0000 7f84ab400640 1 -- 172.21.15.47:0/1427940352 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f84c400fb40 con 0x558454e00c60 2024-03-21T11:47:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.588+0000 7f84ab400640 1 -- 172.21.15.47:0/1427940352 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84c4035050 con 0x558454e00c60 2024-03-21T11:47:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.589+0000 7f84d3606740 10 client.33280 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.589+0000 7f84d3606740 1 -- 172.21.15.47:0/1427940352 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558454e5e6c0 con 0x558454e00c60 2024-03-21T11:47:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.593+0000 7f84ab400640 1 -- 172.21.15.47:0/1427940352 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84b80db070 con 0x558454e00c60 2024-03-21T11:47:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.593+0000 7f84ab400640 1 -- 172.21.15.47:0/1427940352 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f84c4028070 con 0x558454e00c60 2024-03-21T11:47:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.593+0000 7f84d3606740 10 client.33280 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.593+0000 7f84d3606740 10 client.33280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.593+0000 7f84d3606740 20 client.33280 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.593+0000 7f84d3606740 1 --2- 172.21.15.47:0/1427940352 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558454e5f080 0x558454e614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.593+0000 7f84d3606740 4 client.33280 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:47:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.593+0000 7f84d3606740 1 -- 172.21.15.47:0/1427940352 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558454cce500 con 0x558454e5f080 2024-03-21T11:47:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.593+0000 7f84cde00640 1 --2- 172.21.15.47:0/1427940352 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558454e5f080 0x558454e614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:58.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.594+0000 7f84cde00640 1 --2- 172.21.15.47:0/1427940352 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558454e5f080 0x558454e614a0 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768e459292bbd5b3 server_cookie=3dee3d6e504e1044 in_seq=0 out_seq=0 2024-03-21T11:47:58.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.594+0000 7f84ab400640 10 client.33280 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84ab400640 1 -- 172.21.15.47:0/1427940352 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558454cce500 con 0x558454e5f080 2024-03-21T11:47:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84ab400640 10 client.33280 handle_command_reply: tid=0 2024-03-21T11:47:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84d3606740 1 client.33280 shutdown 2024-03-21T11:47:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84d3606740 1 -- 172.21.15.47:0/1427940352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454e00c60 msgr2=0x558454df4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84d3606740 1 --2- 172.21.15.47:0/1427940352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454e00c60 0x558454df4fe0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009fd0 tx=0x7f84c4004ab0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84d3606740 1 -- 172.21.15.47:0/1427940352 shutdown_connections 2024-03-21T11:47:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84d3606740 1 --2- 172.21.15.47:0/1427940352 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x558454e5f080 0x558454e614a0 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84d3606740 1 --2- 172.21.15.47:0/1427940352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558454e00c60 0x558454df4fe0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84d3606740 1 --2- 172.21.15.47:0/1427940352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558454de63a0 0x558454df1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84ab400640 0 client.33280 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84d3606740 1 --2- 172.21.15.47:0/1427940352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558454de4810 0x558454df1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84d3606740 1 -- 172.21.15.47:0/1427940352 >> 172.21.15.47:0/1427940352 conn(0x558454ddfc00 msgr2=0x558454e01980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84d3606740 1 -- 172.21.15.47:0/1427940352 shutdown_connections 2024-03-21T11:47:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84d3606740 1 -- 172.21.15.47:0/1427940352 wait complete. 2024-03-21T11:47:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.600+0000 7f84d3606740 20 client.33280 trim_cache size 0 max 16384 2024-03-21T11:47:58.602 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:47:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.601+0000 7f84d0800640 1 -- 172.21.15.47:0/1061159332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84b00a2f20 msgr2=0x7f84b00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.601+0000 7f84d0800640 1 --2- 172.21.15.47:0/1061159332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84b00a2f20 0x7f84b00a53e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x558454de58e0 tx=0x7f84c4009500 comp rx=0 tx=0).stop 2024-03-21T11:47:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.601+0000 7f84d0800640 1 -- 172.21.15.47:0/1061159332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c810d2a0 msgr2=0x7f84c81a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.601+0000 7f84d0800640 1 --2- 172.21.15.47:0/1061159332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c810d2a0 0x7f84c81a2a00 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005ab0 tx=0x7f84b802c6c0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.601+0000 7f84d0800640 1 -- 172.21.15.47:0/1061159332 shutdown_connections 2024-03-21T11:47:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.601+0000 7f84d0800640 1 --2- 172.21.15.47:0/1061159332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f84b00a2f20 0x7f84b00a53e0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.601+0000 7f84d0800640 1 --2- 172.21.15.47:0/1061159332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c810dc60 0x7f84c81a9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.601+0000 7f84d0800640 1 --2- 172.21.15.47:0/1061159332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c810d2a0 0x7f84c81a2a00 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.601+0000 7f84d0800640 1 --2- 172.21.15.47:0/1061159332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84c810c0b0 0x7f84c81a24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.601+0000 7f84d0800640 1 -- 172.21.15.47:0/1061159332 >> 172.21.15.47:0/1061159332 conn(0x7f84c806c060 msgr2=0x7f84c81105f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.602+0000 7f84d0800640 1 -- 172.21.15.47:0/1061159332 shutdown_connections 2024-03-21T11:47:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:58.602+0000 7f84d0800640 1 -- 172.21.15.47:0/1061159332 wait complete. 2024-03-21T11:47:58.998 INFO:tasks.workunit.client.0.smithi047.stdout: 10 268985 21.75 MB/sec execute 522 sec latency 67.931 ms 2024-03-21T11:47:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:47:59 smithi102 ceph-mon[36748]: pgmap v1140: 129 pgs: 129 active+clean; 737 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.0 MiB/s wr, 433 op/s 2024-03-21T11:47:59.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:47:59 smithi047 ceph-mon[31351]: pgmap v1140: 129 pgs: 129 active+clean; 737 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.0 MiB/s wr, 433 op/s 2024-03-21T11:47:59.631 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:47:59.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:47:59.631 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:47:59.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:47:59 smithi100 ceph-mon[36245]: pgmap v1140: 129 pgs: 129 active+clean; 737 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.0 MiB/s wr, 433 op/s 2024-03-21T11:47:59.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.892+0000 7fce58e00640 1 -- 172.21.15.47:0/1296427089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce54111de0 msgr2=0x7fce541141f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:59.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.893+0000 7fce58e00640 1 --2- 172.21.15.47:0/1296427089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce54111de0 0x7fce541141f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fce4c013fd0 tx=0x7fce4c0345d0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.893+0000 7fce58e00640 1 -- 172.21.15.47:0/1296427089 shutdown_connections 2024-03-21T11:47:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.893+0000 7fce58e00640 1 --2- 172.21.15.47:0/1296427089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce54111de0 0x7fce541141f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.893+0000 7fce58e00640 1 --2- 172.21.15.47:0/1296427089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce5410b010 0x7fce5410b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.893+0000 7fce58e00640 1 --2- 172.21.15.47:0/1296427089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce541086a0 0x7fce5410aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.893+0000 7fce58e00640 1 -- 172.21.15.47:0/1296427089 >> 172.21.15.47:0/1296427089 conn(0x7fce5406e500 msgr2=0x7fce54070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:59.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.893+0000 7fce58e00640 1 -- 172.21.15.47:0/1296427089 shutdown_connections 2024-03-21T11:47:59.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.893+0000 7fce58e00640 1 -- 172.21.15.47:0/1296427089 wait complete. 2024-03-21T11:47:59.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.895+0000 7fce58e00640 1 Processor -- start 2024-03-21T11:47:59.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.895+0000 7fce58e00640 1 -- start start 2024-03-21T11:47:59.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.896+0000 7fce58e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce541086a0 0x7fce5419e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.896+0000 7fce58e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce5410b010 0x7fce5419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.896+0000 7fce58e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce54111de0 0x7fce541a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.896+0000 7fce58e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5410e680 con 0x7fce541086a0 2024-03-21T11:47:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.896+0000 7fce58e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5410e500 con 0x7fce54111de0 2024-03-21T11:47:59.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.896+0000 7fce58e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5410e800 con 0x7fce5410b010 2024-03-21T11:47:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.896+0000 7fce51400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce5410b010 0x7fce5419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.896+0000 7fce51400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce5410b010 0x7fce5419e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57366/0 (socket says 172.21.15.47:57366) 2024-03-21T11:47:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.896+0000 7fce51400640 1 -- 172.21.15.47:0/3977979007 learned_addr learned my addr 172.21.15.47:0/3977979007 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.896+0000 7fce52800640 1 --2- 172.21.15.47:0/3977979007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce54111de0 0x7fce541a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.897+0000 7fce51e00640 1 --2- 172.21.15.47:0/3977979007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce541086a0 0x7fce5419e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.897+0000 7fce52800640 1 -- 172.21.15.47:0/3977979007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce5410b010 msgr2=0x7fce5419e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.897+0000 7fce52800640 1 --2- 172.21.15.47:0/3977979007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce5410b010 0x7fce5419e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.897+0000 7fce52800640 1 -- 172.21.15.47:0/3977979007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce541086a0 msgr2=0x7fce5419e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:59.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.897+0000 7fce52800640 1 --2- 172.21.15.47:0/3977979007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce541086a0 0x7fce5419e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.897+0000 7fce52800640 1 -- 172.21.15.47:0/3977979007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce541a5c70 con 0x7fce54111de0 2024-03-21T11:47:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.899+0000 7fce52800640 1 --2- 172.21.15.47:0/3977979007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce54111de0 0x7fce541a5600 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fce4c034be0 tx=0x7fce4c03b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.899+0000 7fce4aa00640 1 -- 172.21.15.47:0/3977979007 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c04a070 con 0x7fce54111de0 2024-03-21T11:47:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.899+0000 7fce4aa00640 1 -- 172.21.15.47:0/3977979007 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce4c004720 con 0x7fce54111de0 2024-03-21T11:47:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.899+0000 7fce4aa00640 1 -- 172.21.15.47:0/3977979007 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c04f460 con 0x7fce54111de0 2024-03-21T11:47:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.899+0000 7fce58e00640 1 -- 172.21.15.47:0/3977979007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce541a5f00 con 0x7fce54111de0 2024-03-21T11:47:59.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.900+0000 7fce58e00640 1 -- 172.21.15.47:0/3977979007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce541a6390 con 0x7fce54111de0 2024-03-21T11:47:59.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.906+0000 7fce5bc06740 1 Processor -- start 2024-03-21T11:47:59.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.908+0000 7fce4aa00640 1 -- 172.21.15.47:0/3977979007 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fce4c043040 con 0x7fce54111de0 2024-03-21T11:47:59.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.908+0000 7fce5bc06740 1 -- start start 2024-03-21T11:47:59.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.908+0000 7fce5bc06740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf1671f0a0 0x55cf1671f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.908+0000 7fce5bc06740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671fa90 0x55cf16724590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.908+0000 7fce5bc06740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf16724c00 0x55cf16727090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.908+0000 7fce5bc06740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf16684eb0 con 0x55cf1671fa90 2024-03-21T11:47:59.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.908+0000 7fce5bc06740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf1662da60 con 0x55cf1671f0a0 2024-03-21T11:47:59.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.908+0000 7fce5bc06740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf16680f50 con 0x55cf16724c00 2024-03-21T11:47:59.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.909+0000 7fce51e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671fa90 0x55cf16724590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.909+0000 7fce51e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671fa90 0x55cf16724590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38996/0 (socket says 172.21.15.47:38996) 2024-03-21T11:47:59.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.909+0000 7fce51e00640 1 -- 172.21.15.47:0/3251116459 learned_addr learned my addr 172.21.15.47:0/3251116459 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:59.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.909+0000 7fce51e00640 1 -- 172.21.15.47:0/3251116459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf16724c00 msgr2=0x55cf16727090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:59.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.909+0000 7fce51e00640 1 --2- 172.21.15.47:0/3251116459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf16724c00 0x55cf16727090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.909+0000 7fce51e00640 1 -- 172.21.15.47:0/3251116459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf1671f0a0 msgr2=0x55cf1671f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:59.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.910+0000 7fce51e00640 1 --2- 172.21.15.47:0/3251116459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf1671f0a0 0x55cf1671f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.910+0000 7fce51e00640 1 -- 172.21.15.47:0/3251116459 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf164617e0 con 0x55cf1671fa90 2024-03-21T11:47:59.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.910+0000 7fce51e00640 1 --2- 172.21.15.47:0/3251116459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671fa90 0x55cf16724590 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fce4400b820 tx=0x7fce4400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dec51e31437769c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:59.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.910+0000 7fce3f400640 1 -- 172.21.15.47:0/3251116459 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4400d950 con 0x55cf1671fa90 2024-03-21T11:47:59.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.910+0000 7fce4aa00640 1 --2- 172.21.15.47:0/3977979007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fce380a3070 0x7fce380a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.911+0000 7fce3f400640 1 -- 172.21.15.47:0/3251116459 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce44019070 con 0x55cf1671fa90 2024-03-21T11:47:59.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.911+0000 7fce4aa00640 1 -- 172.21.15.47:0/3977979007 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fce4c0fa800 con 0x7fce54111de0 2024-03-21T11:47:59.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.911+0000 7fce51400640 1 --2- 172.21.15.47:0/3977979007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fce380a3070 0x7fce380a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.911+0000 7fce3f400640 1 -- 172.21.15.47:0/3251116459 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce440123e0 con 0x55cf1671fa90 2024-03-21T11:47:59.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.912+0000 7fce5bc06740 1 -- 172.21.15.47:0/3251116459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671fa90 msgr2=0x55cf16724590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:59.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.912+0000 7fce5bc06740 1 --2- 172.21.15.47:0/3251116459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671fa90 0x55cf16724590 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fce4400b820 tx=0x7fce4400bce0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.912+0000 7fce51400640 1 --2- 172.21.15.47:0/3977979007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fce380a3070 0x7fce380a5530 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fce40005fa0 tx=0x7fce40005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:59.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.912+0000 7fce5bc06740 1 -- 172.21.15.47:0/3251116459 shutdown_connections 2024-03-21T11:47:59.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.912+0000 7fce5bc06740 1 --2- 172.21.15.47:0/3251116459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf16724c00 0x55cf16727090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.912+0000 7fce5bc06740 1 --2- 172.21.15.47:0/3251116459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671fa90 0x55cf16724590 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.912+0000 7fce5bc06740 1 --2- 172.21.15.47:0/3251116459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf1671f0a0 0x55cf1671f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.912+0000 7fce5bc06740 1 -- 172.21.15.47:0/3251116459 >> 172.21.15.47:0/3251116459 conn(0x55cf1671a610 msgr2=0x55cf1671aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:59.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.912+0000 7fce5bc06740 1 -- 172.21.15.47:0/3251116459 shutdown_connections 2024-03-21T11:47:59.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.913+0000 7fce5bc06740 1 -- 172.21.15.47:0/3251116459 wait complete. 2024-03-21T11:47:59.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.913+0000 7fce5bc06740 1 -- start start 2024-03-21T11:47:59.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.914+0000 7fce5bc06740 1 Processor -- start 2024-03-21T11:47:59.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.914+0000 7fce5bc06740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf1671f0a0 0x55cf1673a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.914+0000 7fce5bc06740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671fa90 0x55cf1673a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.914+0000 7fce5bc06740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf16724c00 0x55cf16741790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.914+0000 7fce5bc06740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf16727c00 con 0x55cf1671fa90 2024-03-21T11:47:59.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.914+0000 7fce5bc06740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf16640eb0 con 0x55cf1671f0a0 2024-03-21T11:47:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.914+0000 7fce5bc06740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf16727d80 con 0x55cf16724c00 2024-03-21T11:47:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.914+0000 7fce5bc06740 10 client.0 _ll_register_callbacks cb 0x55cf16635150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.915+0000 7fce51e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf1671f0a0 0x55cf1673a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.915+0000 7fce51e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf1671f0a0 0x55cf1673a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44504/0 (socket says 172.21.15.47:44504) 2024-03-21T11:47:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.915+0000 7fce51e00640 1 -- 172.21.15.47:0/2592084235 learned_addr learned my addr 172.21.15.47:0/2592084235 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.915+0000 7fce52800640 1 --2- 172.21.15.47:0/2592084235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf16724c00 0x55cf16741790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.915+0000 7fce51e00640 1 -- 172.21.15.47:0/2592084235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf16724c00 msgr2=0x55cf16741790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.915+0000 7fce51e00640 1 --2- 172.21.15.47:0/2592084235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf16724c00 0x55cf16741790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.915+0000 7fce51e00640 1 -- 172.21.15.47:0/2592084235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671fa90 msgr2=0x55cf1673a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.915+0000 7fce51e00640 1 --2- 172.21.15.47:0/2592084235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671fa90 0x55cf1673a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.915+0000 7fce51e00640 1 -- 172.21.15.47:0/2592084235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf164c1f70 con 0x55cf1671f0a0 2024-03-21T11:47:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.916+0000 7fce51e00640 1 --2- 172.21.15.47:0/2592084235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf1671f0a0 0x55cf1673a210 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fce44000c00 tx=0x7fce440147a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0416e3c8e5628af server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:59.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.916+0000 7fce33e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:59.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.916+0000 7fce33e00640 1 -- 172.21.15.47:0/2592084235 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44023070 con 0x55cf1671f0a0 2024-03-21T11:47:59.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.916+0000 7fce33e00640 1 -- 172.21.15.47:0/2592084235 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fce4400dd60 con 0x55cf1671f0a0 2024-03-21T11:47:59.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.917+0000 7fce5bc06740 1 -- 172.21.15.47:0/2592084235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce44019070 con 0x55cf1671f0a0 2024-03-21T11:47:59.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.917+0000 7fce33e00640 1 -- 172.21.15.47:0/2592084235 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce4402d050 con 0x55cf1671f0a0 2024-03-21T11:47:59.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.917+0000 7fce5bc06740 10 client.33326 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:59.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.917+0000 7fce5bc06740 1 -- 172.21.15.47:0/2592084235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55cf16359170 con 0x55cf1671f0a0 2024-03-21T11:47:59.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.918+0000 7fce33e00640 1 -- 172.21.15.47:0/2592084235 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce44019070 con 0x55cf1671f0a0 2024-03-21T11:47:59.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.918+0000 7fce33e00640 1 -- 172.21.15.47:0/2592084235 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fce44013070 con 0x55cf1671f0a0 2024-03-21T11:47:59.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.918+0000 7fce5bc06740 10 client.33326 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:59.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.918+0000 7fce5bc06740 10 client.33326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:59.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.918+0000 7fce5bc06740 20 client.33326 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:59.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.918+0000 7fce5bc06740 1 --2- 172.21.15.47:0/2592084235 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cf16745d30 0x55cf16748180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.918+0000 7fce5bc06740 4 client.33326 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:47:59.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.918+0000 7fce5bc06740 1 -- 172.21.15.47:0/2592084235 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf166923f0 con 0x55cf16745d30 2024-03-21T11:47:59.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.919+0000 7fce51400640 1 --2- 172.21.15.47:0/2592084235 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cf16745d30 0x55cf16748180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.921+0000 7fce51400640 1 --2- 172.21.15.47:0/2592084235 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cf16745d30 0x55cf16748180 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76b6464996abbd0 server_cookie=b9b909cedf848e88 in_seq=0 out_seq=0 2024-03-21T11:47:59.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.921+0000 7fce33e00640 10 client.33326 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.928+0000 7fce33e00640 1 -- 172.21.15.47:0/2592084235 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cf166923f0 con 0x55cf16745d30 2024-03-21T11:47:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.928+0000 7fce33e00640 10 client.33326 handle_command_reply: tid=0 2024-03-21T11:47:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.928+0000 7fce5bc06740 1 client.33326 shutdown 2024-03-21T11:47:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.928+0000 7fce5bc06740 1 -- 172.21.15.47:0/2592084235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf1671f0a0 msgr2=0x55cf1673a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:59.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.928+0000 7fce5bc06740 1 --2- 172.21.15.47:0/2592084235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf1671f0a0 0x55cf1673a210 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fce44000c00 tx=0x7fce440147a0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.928+0000 7fce5bc06740 1 -- 172.21.15.47:0/2592084235 shutdown_connections 2024-03-21T11:47:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.928+0000 7fce5bc06740 1 --2- 172.21.15.47:0/2592084235 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cf16745d30 0x55cf16748180 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.929+0000 7fce33e00640 0 client.33326 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.929+0000 7fce5bc06740 1 --2- 172.21.15.47:0/2592084235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf16724c00 0x55cf16741790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.929+0000 7fce5bc06740 1 --2- 172.21.15.47:0/2592084235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671fa90 0x55cf1673a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.929+0000 7fce5bc06740 1 --2- 172.21.15.47:0/2592084235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf1671f0a0 0x55cf1673a210 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.929+0000 7fce5bc06740 1 -- 172.21.15.47:0/2592084235 >> 172.21.15.47:0/2592084235 conn(0x55cf1671a610 msgr2=0x55cf16719830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:59.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.929+0000 7fce5bc06740 1 -- 172.21.15.47:0/2592084235 shutdown_connections 2024-03-21T11:47:59.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.930+0000 7fce5bc06740 1 -- 172.21.15.47:0/2592084235 wait complete. 2024-03-21T11:47:59.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.930+0000 7fce5bc06740 20 client.33326 trim_cache size 0 max 16384 2024-03-21T11:47:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.959+0000 7fce5bc06740 1 Processor -- start 2024-03-21T11:47:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.960+0000 7fce5bc06740 1 -- start start 2024-03-21T11:47:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.960+0000 7fce5bc06740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671f0a0 0x55cf1673a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.960+0000 7fce5bc06740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf1671fa90 0x55cf16741750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.960+0000 7fce5bc06740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf16724c00 0x55cf16720320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.960+0000 7fce5bc06740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce541b0710 con 0x55cf1671f0a0 2024-03-21T11:47:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.960+0000 7fce5bc06740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce541b2090 con 0x55cf16724c00 2024-03-21T11:47:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.960+0000 7fce5bc06740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce541b0d70 con 0x55cf1671fa90 2024-03-21T11:47:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.960+0000 7fce51400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf1671fa90 0x55cf16741750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.960+0000 7fce52800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf16724c00 0x55cf16720320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.960+0000 7fce51400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf1671fa90 0x55cf16741750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57392/0 (socket says 172.21.15.47:57392) 2024-03-21T11:47:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.960+0000 7fce51400640 1 -- 172.21.15.47:0/2121258591 learned_addr learned my addr 172.21.15.47:0/2121258591 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.960+0000 7fce51e00640 1 --2- 172.21.15.47:0/2121258591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671f0a0 0x55cf1673a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.961+0000 7fce52800640 1 -- 172.21.15.47:0/2121258591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf1671fa90 msgr2=0x55cf16741750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.961+0000 7fce52800640 1 --2- 172.21.15.47:0/2121258591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf1671fa90 0x55cf16741750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.961+0000 7fce52800640 1 -- 172.21.15.47:0/2121258591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671f0a0 msgr2=0x55cf1673a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.961+0000 7fce52800640 1 --2- 172.21.15.47:0/2121258591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671f0a0 0x55cf1673a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.961+0000 7fce52800640 1 -- 172.21.15.47:0/2121258591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf16715c20 con 0x55cf16724c00 2024-03-21T11:47:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.961+0000 7fce52800640 1 --2- 172.21.15.47:0/2121258591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf16724c00 0x55cf16720320 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0f7fc0 tx=0x7fce4c0345d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da816e4f6502a0b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.961+0000 7fce3ea00640 1 -- 172.21.15.47:0/2121258591 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c013c10 con 0x55cf16724c00 2024-03-21T11:47:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.961+0000 7fce3ea00640 1 -- 172.21.15.47:0/2121258591 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce4c0c72a0 con 0x55cf16724c00 2024-03-21T11:47:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.961+0000 7fce3ea00640 1 -- 172.21.15.47:0/2121258591 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c0396a0 con 0x55cf16724c00 2024-03-21T11:47:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.962+0000 7fce5bc06740 1 -- 172.21.15.47:0/2121258591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf16724c00 msgr2=0x55cf16720320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.962+0000 7fce5bc06740 1 --2- 172.21.15.47:0/2121258591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf16724c00 0x55cf16720320 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0f7fc0 tx=0x7fce4c0345d0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.962+0000 7fce5bc06740 1 -- 172.21.15.47:0/2121258591 shutdown_connections 2024-03-21T11:47:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.962+0000 7fce5bc06740 1 --2- 172.21.15.47:0/2121258591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf16724c00 0x55cf16720320 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.962+0000 7fce5bc06740 1 --2- 172.21.15.47:0/2121258591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf1671fa90 0x55cf16741750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.962+0000 7fce5bc06740 1 --2- 172.21.15.47:0/2121258591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671f0a0 0x55cf1673a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.962+0000 7fce5bc06740 1 -- 172.21.15.47:0/2121258591 >> 172.21.15.47:0/2121258591 conn(0x55cf1671a610 msgr2=0x55cf1671e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.962+0000 7fce5bc06740 1 -- 172.21.15.47:0/2121258591 shutdown_connections 2024-03-21T11:47:59.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.962+0000 7fce5bc06740 1 -- 172.21.15.47:0/2121258591 wait complete. 2024-03-21T11:47:59.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.963+0000 7fce5bc06740 1 -- start start 2024-03-21T11:47:59.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.963+0000 7fce5bc06740 1 Processor -- start 2024-03-21T11:47:59.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.963+0000 7fce5bc06740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671f0a0 0x55cf1673f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce5bc06740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf1671fa90 0x55cf1673fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce5bc06740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf16724c00 0x55cf16740c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce5bc06740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce541b1a30 con 0x55cf1671f0a0 2024-03-21T11:47:59.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce5bc06740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce541b13d0 con 0x55cf16724c00 2024-03-21T11:47:59.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce5bc06740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce541ae8d0 con 0x55cf1671fa90 2024-03-21T11:47:59.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce5bc06740 10 client.0 _ll_register_callbacks cb 0x7fce4400b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:47:59.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce51e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671f0a0 0x55cf1673f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce52800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf16724c00 0x55cf16740c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce51e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671f0a0 0x55cf1673f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39020/0 (socket says 172.21.15.47:39020) 2024-03-21T11:47:59.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce51e00640 1 -- 172.21.15.47:0/3839240806 learned_addr learned my addr 172.21.15.47:0/3839240806 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:47:59.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce51400640 1 --2- 172.21.15.47:0/3839240806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf1671fa90 0x55cf1673fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce52800640 1 -- 172.21.15.47:0/3839240806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf1671fa90 msgr2=0x55cf1673fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:59.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce52800640 1 --2- 172.21.15.47:0/3839240806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf1671fa90 0x55cf1673fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce52800640 1 -- 172.21.15.47:0/3839240806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671f0a0 msgr2=0x55cf1673f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:47:59.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce52800640 1 --2- 172.21.15.47:0/3839240806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671f0a0 0x55cf1673f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce52800640 1 -- 172.21.15.47:0/3839240806 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf1671b080 con 0x55cf16724c00 2024-03-21T11:47:59.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce51e00640 1 --2- 172.21.15.47:0/3839240806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671f0a0 0x55cf1673f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:47:59.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce52800640 1 --2- 172.21.15.47:0/3839240806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf16724c00 0x55cf16740c00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fce4c034d40 tx=0x7fce4c039fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=254b3da6c84caf2e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:47:59.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.964+0000 7fce33e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:47:59.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.965+0000 7fce33e00640 1 -- 172.21.15.47:0/3839240806 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c011060 con 0x55cf16724c00 2024-03-21T11:47:59.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.965+0000 7fce33e00640 1 -- 172.21.15.47:0/3839240806 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fce4c0c6e00 con 0x55cf16724c00 2024-03-21T11:47:59.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.965+0000 7fce5bc06740 1 -- 172.21.15.47:0/3839240806 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce4c0c72a0 con 0x55cf16724c00 2024-03-21T11:47:59.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.965+0000 7fce33e00640 1 -- 172.21.15.47:0/3839240806 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce4c0cf5b0 con 0x55cf16724c00 2024-03-21T11:47:59.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.965+0000 7fce5bc06740 10 client.33332 fetch_fsmap learned FSMap version 41 2024-03-21T11:47:59.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.965+0000 7fce5bc06740 1 -- 172.21.15.47:0/3839240806 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55cf167a1d60 con 0x55cf16724c00 2024-03-21T11:47:59.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.965+0000 7fce33e00640 1 -- 172.21.15.47:0/3839240806 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce4c0c72a0 con 0x55cf16724c00 2024-03-21T11:47:59.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.965+0000 7fce33e00640 1 -- 172.21.15.47:0/3839240806 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fce4c00b030 con 0x55cf16724c00 2024-03-21T11:47:59.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.965+0000 7fce5bc06740 10 client.33332 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:47:59.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.966+0000 7fce5bc06740 10 client.33332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:47:59.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.966+0000 7fce5bc06740 20 client.33332 populate_metadata read hostname 'smithi047' 2024-03-21T11:47:59.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.966+0000 7fce5bc06740 1 --2- 172.21.15.47:0/3839240806 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cf167a24a0 0x55cf167a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:47:59.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.966+0000 7fce5bc06740 4 client.33332 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:47:59.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.966+0000 7fce5bc06740 1 -- 172.21.15.47:0/3839240806 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cf1668c740 con 0x55cf167a24a0 2024-03-21T11:47:59.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.966+0000 7fce51e00640 1 --2- 172.21.15.47:0/3839240806 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cf167a24a0 0x55cf167a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:47:59.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.966+0000 7fce51e00640 1 --2- 172.21.15.47:0/3839240806 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cf167a24a0 0x55cf167a48c0 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0239ae742925159 server_cookie=329d6090a9e8927c in_seq=0 out_seq=0 2024-03-21T11:47:59.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.966+0000 7fce33e00640 10 client.33332 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:59.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.968+0000 7fce33e00640 1 -- 172.21.15.47:0/3839240806 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cf1668c740 con 0x55cf167a24a0 2024-03-21T11:47:59.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.968+0000 7fce33e00640 10 client.33332 handle_command_reply: tid=0 2024-03-21T11:47:59.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.968+0000 7fce5bc06740 1 client.33332 shutdown 2024-03-21T11:47:59.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.969+0000 7fce5bc06740 1 -- 172.21.15.47:0/3839240806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf16724c00 msgr2=0x55cf16740c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:59.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.969+0000 7fce5bc06740 1 --2- 172.21.15.47:0/3839240806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf16724c00 0x55cf16740c00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fce4c034d40 tx=0x7fce4c039fd0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.969+0000 7fce5bc06740 1 -- 172.21.15.47:0/3839240806 shutdown_connections 2024-03-21T11:47:59.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.969+0000 7fce5bc06740 1 --2- 172.21.15.47:0/3839240806 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55cf167a24a0 0x55cf167a48c0 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.969+0000 7fce5bc06740 1 --2- 172.21.15.47:0/3839240806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cf16724c00 0x55cf16740c00 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.969+0000 7fce5bc06740 1 --2- 172.21.15.47:0/3839240806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf1671fa90 0x55cf1673fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.969+0000 7fce33e00640 0 client.33332 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:47:59.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.969+0000 7fce5bc06740 1 --2- 172.21.15.47:0/3839240806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cf1671f0a0 0x55cf1673f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.969+0000 7fce5bc06740 1 -- 172.21.15.47:0/3839240806 >> 172.21.15.47:0/3839240806 conn(0x55cf1671a610 msgr2=0x55cf1671e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:59.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.969+0000 7fce5bc06740 1 -- 172.21.15.47:0/3839240806 shutdown_connections 2024-03-21T11:47:59.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.969+0000 7fce5bc06740 1 -- 172.21.15.47:0/3839240806 wait complete. 2024-03-21T11:47:59.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.969+0000 7fce5bc06740 20 client.33332 trim_cache size 0 max 16384 2024-03-21T11:47:59.972 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:47:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:47:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "8bfbde3b-6d9a-48d2-93f2-dfbdd90d614d", 2024-03-21T11:47:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:47:59.972 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:47:59.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.970+0000 7fce58e00640 1 -- 172.21.15.47:0/3977979007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fce380a3070 msgr2=0x7fce380a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:59.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.970+0000 7fce58e00640 1 --2- 172.21.15.47:0/3977979007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fce380a3070 0x7fce380a5530 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fce40005fa0 tx=0x7fce40005f30 comp rx=0 tx=0).stop 2024-03-21T11:47:59.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.970+0000 7fce58e00640 1 -- 172.21.15.47:0/3977979007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce54111de0 msgr2=0x7fce541a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:47:59.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.970+0000 7fce58e00640 1 --2- 172.21.15.47:0/3977979007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce54111de0 0x7fce541a5600 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fce4c034be0 tx=0x7fce4c03b8b0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.970+0000 7fce58e00640 1 -- 172.21.15.47:0/3977979007 shutdown_connections 2024-03-21T11:47:59.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.970+0000 7fce58e00640 1 --2- 172.21.15.47:0/3977979007 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fce380a3070 0x7fce380a5530 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.970+0000 7fce58e00640 1 --2- 172.21.15.47:0/3977979007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce54111de0 0x7fce541a5600 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.970+0000 7fce58e00640 1 --2- 172.21.15.47:0/3977979007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce5410b010 0x7fce5419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.970+0000 7fce58e00640 1 --2- 172.21.15.47:0/3977979007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce541086a0 0x7fce5419e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:47:59.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.970+0000 7fce58e00640 1 -- 172.21.15.47:0/3977979007 >> 172.21.15.47:0/3977979007 conn(0x7fce5406e500 msgr2=0x7fce54111060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:47:59.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.971+0000 7fce58e00640 1 -- 172.21.15.47:0/3977979007 shutdown_connections 2024-03-21T11:47:59.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:47:59.971+0000 7fce58e00640 1 -- 172.21.15.47:0/3977979007 wait complete. 2024-03-21T11:47:59.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:47:59.993 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:47:59.998 INFO:tasks.workunit.client.0.smithi047.stdout: 10 269481 21.76 MB/sec execute 523 sec latency 152.021 ms 2024-03-21T11:48:00.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.214+0000 7f5b6f200640 1 -- 172.21.15.47:0/4090280386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b680fff30 msgr2=0x7f5b68102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:00.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.214+0000 7f5b6f200640 1 --2- 172.21.15.47:0/4090280386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b680fff30 0x7f5b68102360 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009900 tx=0x7f5b5802bc60 comp rx=0 tx=0).stop 2024-03-21T11:48:00.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.214+0000 7f5b6f200640 1 -- 172.21.15.47:0/4090280386 shutdown_connections 2024-03-21T11:48:00.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.214+0000 7f5b6f200640 1 --2- 172.21.15.47:0/4090280386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b68105250 0x7f5b681056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.214+0000 7f5b6f200640 1 --2- 172.21.15.47:0/4090280386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b681028a0 0x7f5b68104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.214+0000 7f5b6f200640 1 --2- 172.21.15.47:0/4090280386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b680fff30 0x7f5b68102360 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.214+0000 7f5b6f200640 1 -- 172.21.15.47:0/4090280386 >> 172.21.15.47:0/4090280386 conn(0x7f5b680fbb50 msgr2=0x7f5b680fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:00.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.214+0000 7f5b6f200640 1 -- 172.21.15.47:0/4090280386 shutdown_connections 2024-03-21T11:48:00.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.214+0000 7f5b6f200640 1 -- 172.21.15.47:0/4090280386 wait complete. 2024-03-21T11:48:00.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.215+0000 7f5b6f200640 1 Processor -- start 2024-03-21T11:48:00.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.215+0000 7f5b6f200640 1 -- start start 2024-03-21T11:48:00.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.216+0000 7f5b6f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b680fff30 0x7f5b6819e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.216+0000 7f5b6f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b681028a0 0x7f5b6819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.216+0000 7f5b6f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b68105250 0x7f5b681a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.216+0000 7f5b6f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b68113a20 con 0x7f5b681028a0 2024-03-21T11:48:00.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.216+0000 7f5b6f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b681138a0 con 0x7f5b680fff30 2024-03-21T11:48:00.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.216+0000 7f5b6f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b68113ba0 con 0x7f5b68105250 2024-03-21T11:48:00.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.216+0000 7f5b67e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b680fff30 0x7f5b6819e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.216+0000 7f5b67e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b680fff30 0x7f5b6819e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44530/0 (socket says 172.21.15.47:44530) 2024-03-21T11:48:00.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.216+0000 7f5b67e00640 1 -- 172.21.15.47:0/3430551543 learned_addr learned my addr 172.21.15.47:0/3430551543 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:00.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.216+0000 7f5b6d200640 1 --2- 172.21.15.47:0/3430551543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b68105250 0x7f5b681a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.216+0000 7f5b67400640 1 --2- 172.21.15.47:0/3430551543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b681028a0 0x7f5b6819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.216+0000 7f5b67e00640 1 -- 172.21.15.47:0/3430551543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b68105250 msgr2=0x7f5b681a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:00.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.216+0000 7f5b67e00640 1 --2- 172.21.15.47:0/3430551543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b68105250 0x7f5b681a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.216+0000 7f5b67e00640 1 -- 172.21.15.47:0/3430551543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b681028a0 msgr2=0x7f5b6819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:00.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.217+0000 7f5b67e00640 1 --2- 172.21.15.47:0/3430551543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b681028a0 0x7f5b6819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.217+0000 7f5b67e00640 1 -- 172.21.15.47:0/3430551543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b681a5c50 con 0x7f5b680fff30 2024-03-21T11:48:00.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.217+0000 7f5b67e00640 1 --2- 172.21.15.47:0/3430551543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b680fff30 0x7f5b6819e030 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f5b5802c270 tx=0x7f5b58005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:00.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.218+0000 7f5b64c00640 1 -- 172.21.15.47:0/3430551543 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b58041070 con 0x7f5b680fff30 2024-03-21T11:48:00.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.218+0000 7f5b6f200640 1 -- 172.21.15.47:0/3430551543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b681a5ee0 con 0x7f5b680fff30 2024-03-21T11:48:00.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.218+0000 7f5b64c00640 1 -- 172.21.15.47:0/3430551543 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b580044d0 con 0x7f5b680fff30 2024-03-21T11:48:00.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.218+0000 7f5b64c00640 1 -- 172.21.15.47:0/3430551543 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b58034440 con 0x7f5b680fff30 2024-03-21T11:48:00.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.219+0000 7f5b6f200640 1 -- 172.21.15.47:0/3430551543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b681a6340 con 0x7f5b680fff30 2024-03-21T11:48:00.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.223+0000 7f5b64c00640 1 -- 172.21.15.47:0/3430551543 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5b58044070 con 0x7f5b680fff30 2024-03-21T11:48:00.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.223+0000 7f5b71f7a740 1 Processor -- start 2024-03-21T11:48:00.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.223+0000 7f5b71f7a740 1 -- start start 2024-03-21T11:48:00.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.223+0000 7f5b71f7a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.223+0000 7f5b71f7a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d60870 0x563a89d65370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.223+0000 7f5b71f7a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d659e0 0x563a89d67e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.224+0000 7f5b71f7a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a89cc5250 con 0x563a89d60870 2024-03-21T11:48:00.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.224+0000 7f5b71f7a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a89c6da30 con 0x563a89d659e0 2024-03-21T11:48:00.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.224+0000 7f5b71f7a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a89cc12f0 con 0x563a89d5fe80 2024-03-21T11:48:00.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.224+0000 7f5b6d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.224+0000 7f5b67400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d659e0 0x563a89d67e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.224+0000 7f5b67e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d60870 0x563a89d65370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.224+0000 7f5b6d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d602a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57410/0 (socket says 172.21.15.47:57410) 2024-03-21T11:48:00.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.224+0000 7f5b6d200640 1 -- 172.21.15.47:0/2552214232 learned_addr learned my addr 172.21.15.47:0/2552214232 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:00.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.224+0000 7f5b6d200640 1 -- 172.21.15.47:0/2552214232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d659e0 msgr2=0x563a89d67e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:00.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.224+0000 7f5b6d200640 1 --2- 172.21.15.47:0/2552214232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d659e0 0x563a89d67e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.224+0000 7f5b6d200640 1 -- 172.21.15.47:0/2552214232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d60870 msgr2=0x563a89d65370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:00.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.224+0000 7f5b6d200640 1 --2- 172.21.15.47:0/2552214232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d60870 0x563a89d65370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.224+0000 7f5b6d200640 1 -- 172.21.15.47:0/2552214232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a89c58690 con 0x563a89d5fe80 2024-03-21T11:48:00.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.225+0000 7f5b64c00640 1 --2- 172.21.15.47:0/3430551543 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5b4c0a2f70 0x7f5b4c0a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.225+0000 7f5b6d200640 1 --2- 172.21.15.47:0/2552214232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d602a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f5b60010370 tx=0x7f5b60010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=610098e258698312 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:00.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.225+0000 7f5b64c00640 1 -- 172.21.15.47:0/3430551543 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5b580f13c0 con 0x7f5b680fff30 2024-03-21T11:48:00.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.225+0000 7f5b67e00640 1 --2- 172.21.15.47:0/3430551543 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5b4c0a2f70 0x7f5b4c0a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.225+0000 7f5b5d600640 1 -- 172.21.15.47:0/2552214232 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60016070 con 0x563a89d5fe80 2024-03-21T11:48:00.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.225+0000 7f5b5d600640 1 -- 172.21.15.47:0/2552214232 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b60004d20 con 0x563a89d5fe80 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.225+0000 7f5b5d600640 1 -- 172.21.15.47:0/2552214232 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60005400 con 0x563a89d5fe80 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.226+0000 7f5b71f7a740 1 -- 172.21.15.47:0/2552214232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 msgr2=0x563a89d602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.226+0000 7f5b67e00640 1 --2- 172.21.15.47:0/3430551543 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5b4c0a2f70 0x7f5b4c0a5430 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x563a89d612e0 tx=0x7f5b580f4ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.226+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/2552214232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d602a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f5b60010370 tx=0x7f5b60010830 comp rx=0 tx=0).stop 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.226+0000 7f5b71f7a740 1 -- 172.21.15.47:0/2552214232 shutdown_connections 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.226+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/2552214232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d659e0 0x563a89d67e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.226+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/2552214232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d60870 0x563a89d65370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.226+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/2552214232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d602a0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.226+0000 7f5b71f7a740 1 -- 172.21.15.47:0/2552214232 >> 172.21.15.47:0/2552214232 conn(0x563a89d28200 msgr2=0x563a89d5b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.226+0000 7f5b71f7a740 1 -- 172.21.15.47:0/2552214232 shutdown_connections 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.226+0000 7f5b71f7a740 1 -- 172.21.15.47:0/2552214232 wait complete. 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.227+0000 7f5b71f7a740 1 -- start start 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.227+0000 7f5b71f7a740 1 Processor -- start 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.227+0000 7f5b71f7a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d69e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.228+0000 7f5b71f7a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d60870 0x563a89d6a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.228+0000 7f5b71f7a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d659e0 0x563a89d713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.228+0000 7f5b71f7a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a89d68990 con 0x563a89d659e0 2024-03-21T11:48:00.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.228+0000 7f5b71f7a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a89c80e80 con 0x563a89d60870 2024-03-21T11:48:00.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.228+0000 7f5b71f7a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a89d68b10 con 0x563a89d5fe80 2024-03-21T11:48:00.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.228+0000 7f5b71f7a740 10 client.0 _ll_register_callbacks cb 0x563a89c93130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.228+0000 7f5b6d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d69e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.228+0000 7f5b6d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d69e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57426/0 (socket says 172.21.15.47:57426) 2024-03-21T11:48:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.228+0000 7f5b6d200640 1 -- 172.21.15.47:0/4160299617 learned_addr learned my addr 172.21.15.47:0/4160299617 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.228+0000 7f5b6d200640 1 --2- 172.21.15.47:0/4160299617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d659e0 0x563a89d713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.228+0000 7f5b67400640 1 --2- 172.21.15.47:0/4160299617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d60870 0x563a89d6a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.229+0000 7f5b6d200640 1 -- 172.21.15.47:0/4160299617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d60870 msgr2=0x563a89d6a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.229+0000 7f5b6d200640 1 --2- 172.21.15.47:0/4160299617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d60870 0x563a89d6a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.229+0000 7f5b6d200640 1 -- 172.21.15.47:0/4160299617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d659e0 msgr2=0x563a89d713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.229+0000 7f5b6d200640 1 --2- 172.21.15.47:0/4160299617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d659e0 0x563a89d713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.229+0000 7f5b6d200640 1 -- 172.21.15.47:0/4160299617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a89aa37d0 con 0x563a89d5fe80 2024-03-21T11:48:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.229+0000 7f5b6d200640 1 --2- 172.21.15.47:0/4160299617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d69e20 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005620 tx=0x7f5b60009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=650428f7d42c2de7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:00.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.229+0000 7f5b52000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:48:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.230+0000 7f5b52000640 1 -- 172.21.15.47:0/4160299617 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60016070 con 0x563a89d5fe80 2024-03-21T11:48:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.230+0000 7f5b52000640 1 -- 172.21.15.47:0/4160299617 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5b60005880 con 0x563a89d5fe80 2024-03-21T11:48:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.230+0000 7f5b71f7a740 1 -- 172.21.15.47:0/4160299617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b60004d20 con 0x563a89d5fe80 2024-03-21T11:48:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.230+0000 7f5b52000640 1 -- 172.21.15.47:0/4160299617 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b60023040 con 0x563a89d5fe80 2024-03-21T11:48:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.230+0000 7f5b71f7a740 10 client.33301 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.230+0000 7f5b71f7a740 1 -- 172.21.15.47:0/4160299617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563a899d93f0 con 0x563a89d5fe80 2024-03-21T11:48:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.231+0000 7f5b52000640 1 -- 172.21.15.47:0/4160299617 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b60032070 con 0x563a89d5fe80 2024-03-21T11:48:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.231+0000 7f5b52000640 1 -- 172.21.15.47:0/4160299617 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5b6000f040 con 0x563a89d5fe80 2024-03-21T11:48:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.231+0000 7f5b71f7a740 10 client.33301 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.231+0000 7f5b71f7a740 10 client.33301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.231+0000 7f5b71f7a740 20 client.33301 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.231+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/4160299617 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563a89d75a90 0x563a89d77e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.231+0000 7f5b71f7a740 4 client.33301 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:48:00.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.231+0000 7f5b71f7a740 1 -- 172.21.15.47:0/4160299617 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a89cd2790 con 0x563a89d75a90 2024-03-21T11:48:00.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.231+0000 7f5b67400640 1 --2- 172.21.15.47:0/4160299617 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563a89d75a90 0x563a89d77e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.232+0000 7f5b67400640 1 --2- 172.21.15.47:0/4160299617 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563a89d75a90 0x563a89d77e80 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b6d1757c0b1c46 server_cookie=a289c716fcb70590 in_seq=0 out_seq=0 2024-03-21T11:48:00.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.232+0000 7f5b52000640 10 client.33301 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:00.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.234+0000 7f5b52000640 1 -- 172.21.15.47:0/4160299617 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563a89cd2790 con 0x563a89d75a90 2024-03-21T11:48:00.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.234+0000 7f5b52000640 10 client.33301 handle_command_reply: tid=0 2024-03-21T11:48:00.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.234+0000 7f5b71f7a740 1 client.33301 shutdown 2024-03-21T11:48:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.234+0000 7f5b71f7a740 1 -- 172.21.15.47:0/4160299617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 msgr2=0x563a89d69e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.234+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/4160299617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d69e20 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005620 tx=0x7f5b60009bb0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.234+0000 7f5b71f7a740 1 -- 172.21.15.47:0/4160299617 shutdown_connections 2024-03-21T11:48:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.234+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/4160299617 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563a89d75a90 0x563a89d77e80 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.234+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/4160299617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d659e0 0x563a89d713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.234+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/4160299617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d60870 0x563a89d6a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.234+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/4160299617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d69e20 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.234+0000 7f5b71f7a740 1 -- 172.21.15.47:0/4160299617 >> 172.21.15.47:0/4160299617 conn(0x563a89d28200 msgr2=0x563a89d5aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.235+0000 7f5b52000640 0 client.33301 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.235+0000 7f5b71f7a740 1 -- 172.21.15.47:0/4160299617 shutdown_connections 2024-03-21T11:48:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.235+0000 7f5b71f7a740 1 -- 172.21.15.47:0/4160299617 wait complete. 2024-03-21T11:48:00.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.235+0000 7f5b71f7a740 20 client.33301 trim_cache size 0 max 16384 2024-03-21T11:48:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.256+0000 7f5b71f7a740 1 Processor -- start 2024-03-21T11:48:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.256+0000 7f5b71f7a740 1 -- start start 2024-03-21T11:48:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.256+0000 7f5b71f7a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d5fe80 0x563a89d5f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.256+0000 7f5b71f7a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d60870 0x563a89d71330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.256+0000 7f5b71f7a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d659e0 0x563a89d5a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.256+0000 7f5b71f7a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b681b05f0 con 0x563a89d659e0 2024-03-21T11:48:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.256+0000 7f5b71f7a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b681b1f70 con 0x563a89d5fe80 2024-03-21T11:48:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.256+0000 7f5b71f7a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b681b0c50 con 0x563a89d60870 2024-03-21T11:48:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.256+0000 7f5b67400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d60870 0x563a89d71330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.256+0000 7f5b67400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d60870 0x563a89d71330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:57438/0 (socket says 172.21.15.47:57438) 2024-03-21T11:48:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.256+0000 7f5b67400640 1 -- 172.21.15.47:0/1051679244 learned_addr learned my addr 172.21.15.47:0/1051679244 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.257+0000 7f5b6d200640 1 --2- 172.21.15.47:0/1051679244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d659e0 0x563a89d5a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.257+0000 7f5b6d200640 1 -- 172.21.15.47:0/1051679244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d60870 msgr2=0x563a89d71330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.257+0000 7f5b6d200640 1 --2- 172.21.15.47:0/1051679244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d60870 0x563a89d71330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.257+0000 7f5b6d200640 1 -- 172.21.15.47:0/1051679244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d5fe80 msgr2=0x563a89d5f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.257+0000 7f5b6d200640 1 --2- 172.21.15.47:0/1051679244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d5fe80 0x563a89d5f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.257+0000 7f5b6d200640 1 -- 172.21.15.47:0/1051679244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a899990f0 con 0x563a89d659e0 2024-03-21T11:48:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.257+0000 7f5b67400640 1 --2- 172.21.15.47:0/1051679244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d60870 0x563a89d71330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:48:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.257+0000 7f5b6d200640 1 --2- 172.21.15.47:0/1051679244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d659e0 0x563a89d5a200 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005620 tx=0x7f5b60010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47778a02542e3a7b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.258+0000 7f5b5cc00640 1 -- 172.21.15.47:0/1051679244 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6002f070 con 0x563a89d659e0 2024-03-21T11:48:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.258+0000 7f5b5cc00640 1 -- 172.21.15.47:0/1051679244 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b60032040 con 0x563a89d659e0 2024-03-21T11:48:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.258+0000 7f5b5cc00640 1 -- 172.21.15.47:0/1051679244 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6001e070 con 0x563a89d659e0 2024-03-21T11:48:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.259+0000 7f5b71f7a740 1 -- 172.21.15.47:0/1051679244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d659e0 msgr2=0x563a89d5a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.259+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/1051679244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d659e0 0x563a89d5a200 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005620 tx=0x7f5b60010830 comp rx=0 tx=0).stop 2024-03-21T11:48:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.259+0000 7f5b71f7a740 1 -- 172.21.15.47:0/1051679244 shutdown_connections 2024-03-21T11:48:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.259+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/1051679244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d659e0 0x563a89d5a200 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.259+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/1051679244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d60870 0x563a89d71330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.259+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/1051679244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d5fe80 0x563a89d5f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.259+0000 7f5b71f7a740 1 -- 172.21.15.47:0/1051679244 >> 172.21.15.47:0/1051679244 conn(0x563a89d28200 msgr2=0x563a89d5ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.259+0000 7f5b71f7a740 1 -- 172.21.15.47:0/1051679244 shutdown_connections 2024-03-21T11:48:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.259+0000 7f5b71f7a740 1 -- 172.21.15.47:0/1051679244 wait complete. 2024-03-21T11:48:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.260+0000 7f5b71f7a740 1 -- start start 2024-03-21T11:48:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.260+0000 7f5b71f7a740 1 Processor -- start 2024-03-21T11:48:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.261+0000 7f5b71f7a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d6ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.261+0000 7f5b71f7a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d60870 0x563a89d6f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.261+0000 7f5b71f7a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d659e0 0x563a89d703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.261+0000 7f5b71f7a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b681b1910 con 0x563a89d60870 2024-03-21T11:48:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.261+0000 7f5b71f7a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b681b12b0 con 0x563a89d659e0 2024-03-21T11:48:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.261+0000 7f5b71f7a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b681ae7b0 con 0x563a89d5fe80 2024-03-21T11:48:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.261+0000 7f5b71f7a740 10 client.0 _ll_register_callbacks cb 0x7f5b580f4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.261+0000 7f5b6d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d659e0 0x563a89d703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.261+0000 7f5b6d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d659e0 0x563a89d703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44568/0 (socket says 172.21.15.47:44568) 2024-03-21T11:48:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.261+0000 7f5b6d200640 1 -- 172.21.15.47:0/971512462 learned_addr learned my addr 172.21.15.47:0/971512462 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.261+0000 7f5b6d200640 1 --2- 172.21.15.47:0/971512462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d6ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.261+0000 7f5b67400640 1 --2- 172.21.15.47:0/971512462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d60870 0x563a89d6f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.262+0000 7f5b6d200640 1 -- 172.21.15.47:0/971512462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 msgr2=0x563a89d6ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.262+0000 7f5b6d200640 1 --2- 172.21.15.47:0/971512462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d6ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.262+0000 7f5b6d200640 1 -- 172.21.15.47:0/971512462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d60870 msgr2=0x563a89d6f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.262+0000 7f5b6d200640 1 --2- 172.21.15.47:0/971512462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d60870 0x563a89d6f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.262+0000 7f5b6d200640 1 -- 172.21.15.47:0/971512462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a89d70990 con 0x563a89d659e0 2024-03-21T11:48:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.262+0000 7f5b6d200640 1 --2- 172.21.15.47:0/971512462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d659e0 0x563a89d703a0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f5b6001a720 tx=0x7f5b60005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ca9350ddf11ec49 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.262+0000 7f5b52000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:48:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.262+0000 7f5b52000640 1 -- 172.21.15.47:0/971512462 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6002f030 con 0x563a89d659e0 2024-03-21T11:48:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.263+0000 7f5b52000640 1 -- 172.21.15.47:0/971512462 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5b60003b70 con 0x563a89d659e0 2024-03-21T11:48:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.263+0000 7f5b71f7a740 1 -- 172.21.15.47:0/971512462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b60032040 con 0x563a89d659e0 2024-03-21T11:48:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.263+0000 7f5b52000640 1 -- 172.21.15.47:0/971512462 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b60048050 con 0x563a89d659e0 2024-03-21T11:48:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.263+0000 7f5b71f7a740 10 client.33347 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.263+0000 7f5b71f7a740 1 -- 172.21.15.47:0/971512462 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563a89dd1780 con 0x563a89d659e0 2024-03-21T11:48:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.263+0000 7f5b52000640 1 -- 172.21.15.47:0/971512462 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b60032040 con 0x563a89d659e0 2024-03-21T11:48:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.263+0000 7f5b52000640 1 -- 172.21.15.47:0/971512462 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5b60012630 con 0x563a89d659e0 2024-03-21T11:48:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.264+0000 7f5b71f7a740 10 client.33347 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.264+0000 7f5b71f7a740 10 client.33347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.264+0000 7f5b71f7a740 20 client.33347 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.264+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/971512462 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563a89dd1ff0 0x563a89dd4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.264+0000 7f5b71f7a740 4 client.33347 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:48:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.264+0000 7f5b71f7a740 1 -- 172.21.15.47:0/971512462 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563a89cccae0 con 0x563a89dd1ff0 2024-03-21T11:48:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.264+0000 7f5b67400640 1 --2- 172.21.15.47:0/971512462 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563a89dd1ff0 0x563a89dd4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.264+0000 7f5b67400640 1 --2- 172.21.15.47:0/971512462 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563a89dd1ff0 0x563a89dd4410 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d7a46c0978e9d46 server_cookie=c7ad7ba8fe709729 in_seq=0 out_seq=0 2024-03-21T11:48:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.264+0000 7f5b52000640 10 client.33347 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.275+0000 7f5b52000640 1 -- 172.21.15.47:0/971512462 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563a89cccae0 con 0x563a89dd1ff0 2024-03-21T11:48:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.275+0000 7f5b52000640 10 client.33347 handle_command_reply: tid=0 2024-03-21T11:48:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.275+0000 7f5b71f7a740 1 client.33347 shutdown 2024-03-21T11:48:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.275+0000 7f5b71f7a740 1 -- 172.21.15.47:0/971512462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d659e0 msgr2=0x563a89d703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.275+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/971512462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d659e0 0x563a89d703a0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f5b6001a720 tx=0x7f5b60005fd0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.275+0000 7f5b71f7a740 1 -- 172.21.15.47:0/971512462 shutdown_connections 2024-03-21T11:48:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.275+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/971512462 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563a89dd1ff0 0x563a89dd4410 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.275+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/971512462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a89d659e0 0x563a89d703a0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.275+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/971512462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a89d60870 0x563a89d6f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.275+0000 7f5b71f7a740 1 --2- 172.21.15.47:0/971512462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a89d5fe80 0x563a89d6ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.275+0000 7f5b52000640 0 client.33347 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.275+0000 7f5b71f7a740 1 -- 172.21.15.47:0/971512462 >> 172.21.15.47:0/971512462 conn(0x563a89d28200 msgr2=0x563a89d5f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.276+0000 7f5b71f7a740 1 -- 172.21.15.47:0/971512462 shutdown_connections 2024-03-21T11:48:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.276+0000 7f5b71f7a740 1 -- 172.21.15.47:0/971512462 wait complete. 2024-03-21T11:48:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.276+0000 7f5b71f7a740 20 client.33347 trim_cache size 0 max 16384 2024-03-21T11:48:00.278 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:48:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (7 inodes in the stack)", 2024-03-21T11:48:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:48:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "8bfbde3b-6d9a-48d2-93f2-dfbdd90d614d": { 2024-03-21T11:48:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:48:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "8bfbde3b-6d9a-48d2-93f2-dfbdd90d614d", 2024-03-21T11:48:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:48:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:48:00.278 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:48:00.278 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:48:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.276+0000 7f5b6f200640 1 -- 172.21.15.47:0/3430551543 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5b4c0a2f70 msgr2=0x7f5b4c0a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.276+0000 7f5b6f200640 1 --2- 172.21.15.47:0/3430551543 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5b4c0a2f70 0x7f5b4c0a5430 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x563a89d612e0 tx=0x7f5b580f4ee0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.276+0000 7f5b6f200640 1 -- 172.21.15.47:0/3430551543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b680fff30 msgr2=0x7f5b6819e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.276+0000 7f5b6f200640 1 --2- 172.21.15.47:0/3430551543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b680fff30 0x7f5b6819e030 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f5b5802c270 tx=0x7f5b58005a40 comp rx=0 tx=0).stop 2024-03-21T11:48:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.276+0000 7f5b6f200640 1 -- 172.21.15.47:0/3430551543 shutdown_connections 2024-03-21T11:48:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.276+0000 7f5b6f200640 1 --2- 172.21.15.47:0/3430551543 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f5b4c0a2f70 0x7f5b4c0a5430 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.276+0000 7f5b6f200640 1 --2- 172.21.15.47:0/3430551543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b68105250 0x7f5b681a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.276+0000 7f5b6f200640 1 --2- 172.21.15.47:0/3430551543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b681028a0 0x7f5b6819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.276+0000 7f5b6f200640 1 --2- 172.21.15.47:0/3430551543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b680fff30 0x7f5b6819e030 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.276+0000 7f5b6f200640 1 -- 172.21.15.47:0/3430551543 >> 172.21.15.47:0/3430551543 conn(0x7f5b680fbb50 msgr2=0x7f5b68103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.277+0000 7f5b6f200640 1 -- 172.21.15.47:0/3430551543 shutdown_connections 2024-03-21T11:48:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:00.277+0000 7f5b6f200640 1 -- 172.21.15.47:0/3430551543 wait complete. 2024-03-21T11:48:00.298 INFO:tasks.cephfs.filesystem:scrub status for tag:8bfbde3b-6d9a-48d2-93f2-dfbdd90d614d - {'path': '/', 'tag': '8bfbde3b-6d9a-48d2-93f2-dfbdd90d614d', 'options': 'recursive,force'} 2024-03-21T11:48:00.999 INFO:tasks.workunit.client.0.smithi047.stdout: 10 269839 21.75 MB/sec execute 524 sec latency 86.966 ms 2024-03-21T11:48:01.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: pgmap v1141: 129 pgs: 129 active+clean; 732 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.7 MiB/s wr, 515 op/s 2024-03-21T11:48:01.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:48:01.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:01.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:01.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:01.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:01.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:01.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:01.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:01.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:01.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:01 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: pgmap v1141: 129 pgs: 129 active+clean; 732 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.7 MiB/s wr, 515 op/s 2024-03-21T11:48:01.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:48:01.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:01.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:01.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:01.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:01.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:01.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:01.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:01.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:01.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:01 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: pgmap v1141: 129 pgs: 129 active+clean; 732 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.7 MiB/s wr, 515 op/s 2024-03-21T11:48:01.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:48:01.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:01.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:01.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:01.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:01.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:01.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:01.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:01.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:01.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:01 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:01.999 INFO:tasks.workunit.client.0.smithi047.stdout: 10 269994 21.72 MB/sec execute 525 sec latency 406.091 ms 2024-03-21T11:48:02.999 INFO:tasks.workunit.client.0.smithi047.stdout: 10 270233 21.71 MB/sec execute 526 sec latency 241.629 ms 2024-03-21T11:48:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:03 smithi102 ceph-mon[36748]: pgmap v1142: 129 pgs: 129 active+clean; 734 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.6 MiB/s wr, 500 op/s 2024-03-21T11:48:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:03 smithi047 ceph-mon[31351]: pgmap v1142: 129 pgs: 129 active+clean; 734 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.6 MiB/s wr, 500 op/s 2024-03-21T11:48:03.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:03 smithi100 ceph-mon[36245]: pgmap v1142: 129 pgs: 129 active+clean; 734 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.6 MiB/s wr, 500 op/s 2024-03-21T11:48:03.999 INFO:tasks.workunit.client.0.smithi047.stdout: 10 270814 21.72 MB/sec execute 527 sec latency 103.958 ms 2024-03-21T11:48:04.999 INFO:tasks.workunit.client.0.smithi047.stdout: 10 271387 21.74 MB/sec execute 528 sec latency 72.745 ms 2024-03-21T11:48:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:05 smithi102 ceph-mon[36748]: pgmap v1143: 129 pgs: 129 active+clean; 743 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.0 MiB/s wr, 620 op/s 2024-03-21T11:48:05.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:05 smithi047 ceph-mon[31351]: pgmap v1143: 129 pgs: 129 active+clean; 743 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.0 MiB/s wr, 620 op/s 2024-03-21T11:48:05.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:05 smithi100 ceph-mon[36245]: pgmap v1143: 129 pgs: 129 active+clean; 743 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.0 MiB/s wr, 620 op/s 2024-03-21T11:48:05.999 INFO:tasks.workunit.client.0.smithi047.stdout: 10 271799 21.75 MB/sec execute 529 sec latency 87.930 ms 2024-03-21T11:48:06.999 INFO:tasks.workunit.client.0.smithi047.stdout: 10 271987 21.73 MB/sec execute 530 sec latency 394.491 ms 2024-03-21T11:48:07.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:07 smithi047 ceph-mon[31351]: pgmap v1144: 129 pgs: 129 active+clean; 745 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.5 MiB/s wr, 540 op/s 2024-03-21T11:48:07.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:07 smithi100 ceph-mon[36245]: pgmap v1144: 129 pgs: 129 active+clean; 745 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.5 MiB/s wr, 540 op/s 2024-03-21T11:48:07.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:07 smithi102 ceph-mon[36748]: pgmap v1144: 129 pgs: 129 active+clean; 745 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.5 MiB/s wr, 540 op/s 2024-03-21T11:48:07.999 INFO:tasks.workunit.client.0.smithi047.stdout: 10 272269 21.71 MB/sec execute 531 sec latency 217.256 ms 2024-03-21T11:48:09.000 INFO:tasks.workunit.client.0.smithi047.stdout: 10 272546 21.70 MB/sec execute 532 sec latency 284.231 ms 2024-03-21T11:48:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.145+0000 7f46a9600640 1 -- 172.21.15.47:0/396025243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a40756e0 msgr2=0x7f46a4073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.145+0000 7f46a9600640 1 --2- 172.21.15.47:0/396025243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a40756e0 0x7f46a4073b30 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f4694009900 tx=0x7f469402bc30 comp rx=0 tx=0).stop 2024-03-21T11:48:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.145+0000 7f46a9600640 1 -- 172.21.15.47:0/396025243 shutdown_connections 2024-03-21T11:48:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.145+0000 7f46a9600640 1 --2- 172.21.15.47:0/396025243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a410dcc0 0x7f46a41100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.145+0000 7f46a9600640 1 --2- 172.21.15.47:0/396025243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4074100 0x7f46a4074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.145+0000 7f46a9600640 1 --2- 172.21.15.47:0/396025243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a40756e0 0x7f46a4073b30 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.145+0000 7f46a9600640 1 -- 172.21.15.47:0/396025243 >> 172.21.15.47:0/396025243 conn(0x7f46a40fcc60 msgr2=0x7f46a40ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.145+0000 7f46a9600640 1 -- 172.21.15.47:0/396025243 shutdown_connections 2024-03-21T11:48:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.145+0000 7f46a9600640 1 -- 172.21.15.47:0/396025243 wait complete. 2024-03-21T11:48:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.146+0000 7f46a9600640 1 Processor -- start 2024-03-21T11:48:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.146+0000 7f46a9600640 1 -- start start 2024-03-21T11:48:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f46a9600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4074100 0x7f46a419df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f46a9600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a40756e0 0x7f46a419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f46a9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a410dcc0 0x7f46a41a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f46a9600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a4113960 con 0x7f46a4074100 2024-03-21T11:48:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f46a9600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a41137e0 con 0x7f46a410dcc0 2024-03-21T11:48:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f46a9600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a4113ae0 con 0x7f46a40756e0 2024-03-21T11:48:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f46a2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4074100 0x7f46a419df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f46a3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a410dcc0 0x7f46a41a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f469be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a40756e0 0x7f46a419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f469be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a40756e0 0x7f46a419e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51674/0 (socket says 172.21.15.47:51674) 2024-03-21T11:48:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f469be00640 1 -- 172.21.15.47:0/3867079880 learned_addr learned my addr 172.21.15.47:0/3867079880 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f46a3400640 1 -- 172.21.15.47:0/3867079880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a40756e0 msgr2=0x7f46a419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f46a3400640 1 --2- 172.21.15.47:0/3867079880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a40756e0 0x7f46a419e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f46a3400640 1 -- 172.21.15.47:0/3867079880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4074100 msgr2=0x7f46a419df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f46a3400640 1 --2- 172.21.15.47:0/3867079880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4074100 0x7f46a419df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.147+0000 7f46a3400640 1 -- 172.21.15.47:0/3867079880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a41a5b40 con 0x7f46a410dcc0 2024-03-21T11:48:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.148+0000 7f46a3400640 1 --2- 172.21.15.47:0/3867079880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a410dcc0 0x7f46a41a54d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f469c00e940 tx=0x7f469c00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.148+0000 7f469b400640 1 -- 172.21.15.47:0/3867079880 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c004d20 con 0x7f46a410dcc0 2024-03-21T11:48:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.148+0000 7f469b400640 1 -- 172.21.15.47:0/3867079880 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f469c005020 con 0x7f46a410dcc0 2024-03-21T11:48:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.148+0000 7f469b400640 1 -- 172.21.15.47:0/3867079880 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c0116b0 con 0x7f46a410dcc0 2024-03-21T11:48:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.149+0000 7f46a9600640 1 -- 172.21.15.47:0/3867079880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a41a5e30 con 0x7f46a410dcc0 2024-03-21T11:48:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.149+0000 7f46a9600640 1 -- 172.21.15.47:0/3867079880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a41a62a0 con 0x7f46a410dcc0 2024-03-21T11:48:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.153+0000 7f46ac232740 1 Processor -- start 2024-03-21T11:48:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.154+0000 7f469b400640 1 -- 172.21.15.47:0/3867079880 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f469c0054d0 con 0x7f46a410dcc0 2024-03-21T11:48:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.154+0000 7f46ac232740 1 -- start start 2024-03-21T11:48:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.155+0000 7f46ac232740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166de600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.155+0000 7f46ac232740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 0x557a166e36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.155+0000 7f46ac232740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166e3d40 0x557a166e61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.155+0000 7f46ac232740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a166a13b0 con 0x557a166debd0 2024-03-21T11:48:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.155+0000 7f46ac232740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a16643830 con 0x557a166e3d40 2024-03-21T11:48:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.155+0000 7f46ac232740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a165ec5d0 con 0x557a166de1e0 2024-03-21T11:48:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.156+0000 7f469be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166de600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.156+0000 7f469be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166de600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51686/0 (socket says 172.21.15.47:51686) 2024-03-21T11:48:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.156+0000 7f469be00640 1 -- 172.21.15.47:0/2671382130 learned_addr learned my addr 172.21.15.47:0/2671382130 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.156+0000 7f469be00640 1 --2- 172.21.15.47:0/2671382130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166e3d40 0x557a166e61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.156+0000 7f469b400640 1 --2- 172.21.15.47:0/3867079880 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46880a30c0 0x7f46880a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.157+0000 7f469b400640 1 -- 172.21.15.47:0/3867079880 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f469c0cfa40 con 0x7f46a410dcc0 2024-03-21T11:48:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.157+0000 7f469be00640 1 -- 172.21.15.47:0/2671382130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166e3d40 msgr2=0x557a166e61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.157+0000 7f469be00640 1 --2- 172.21.15.47:0/2671382130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166e3d40 0x557a166e61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.157+0000 7f46a2a00640 1 --2- 172.21.15.47:0/2671382130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 0x557a166e36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.157+0000 7f46a3400640 1 --2- 172.21.15.47:0/3867079880 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46880a30c0 0x7f46880a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.157+0000 7f469be00640 1 -- 172.21.15.47:0/2671382130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 msgr2=0x557a166e36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.157+0000 7f469be00640 1 --2- 172.21.15.47:0/2671382130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 0x557a166e36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.157+0000 7f469be00640 1 -- 172.21.15.47:0/2671382130 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a166042c0 con 0x557a166de1e0 2024-03-21T11:48:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.157+0000 7f46a2a00640 1 --2- 172.21.15.47:0/2671382130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 0x557a166e36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:48:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.158+0000 7f469be00640 1 --2- 172.21.15.47:0/2671382130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166de600 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f468c00b290 tx=0x7f468c0040e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99adb04fcd70504e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.158+0000 7f4693400640 1 -- 172.21.15.47:0/2671382130 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c012450 con 0x557a166de1e0 2024-03-21T11:48:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.159+0000 7f4693400640 1 -- 172.21.15.47:0/2671382130 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f468c012ad0 con 0x557a166de1e0 2024-03-21T11:48:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.159+0000 7f4693400640 1 -- 172.21.15.47:0/2671382130 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c010840 con 0x557a166de1e0 2024-03-21T11:48:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.160+0000 7f46ac232740 1 -- 172.21.15.47:0/2671382130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 msgr2=0x557a166de600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.160+0000 7f46ac232740 1 --2- 172.21.15.47:0/2671382130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166de600 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f468c00b290 tx=0x7f468c0040e0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.160+0000 7f46a3400640 1 --2- 172.21.15.47:0/3867079880 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46880a30c0 0x7f46880a5580 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f469c000f80 tx=0x7f469c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.160+0000 7f46ac232740 1 -- 172.21.15.47:0/2671382130 shutdown_connections 2024-03-21T11:48:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.160+0000 7f46ac232740 1 --2- 172.21.15.47:0/2671382130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166e3d40 0x557a166e61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.160+0000 7f46ac232740 1 --2- 172.21.15.47:0/2671382130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 0x557a166e36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.160+0000 7f46ac232740 1 --2- 172.21.15.47:0/2671382130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166de600 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.160+0000 7f46ac232740 1 -- 172.21.15.47:0/2671382130 >> 172.21.15.47:0/2671382130 conn(0x557a166d9750 msgr2=0x557a166d9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.161+0000 7f46ac232740 1 -- 172.21.15.47:0/2671382130 shutdown_connections 2024-03-21T11:48:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.161+0000 7f46ac232740 1 -- 172.21.15.47:0/2671382130 wait complete. 2024-03-21T11:48:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.161+0000 7f46ac232740 1 -- start start 2024-03-21T11:48:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.162+0000 7f46ac232740 1 Processor -- start 2024-03-21T11:48:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.163+0000 7f46ac232740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166e8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.163+0000 7f46ac232740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 0x557a166e88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.163+0000 7f46ac232740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166e3d40 0x557a166ef910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.163+0000 7f46ac232740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a165ffa20 con 0x557a166debd0 2024-03-21T11:48:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.163+0000 7f46ac232740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1663f8d0 con 0x557a166e3d40 2024-03-21T11:48:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.163+0000 7f46ac232740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a166e6cf0 con 0x557a166de1e0 2024-03-21T11:48:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.163+0000 7f46ac232740 10 client.0 _ll_register_callbacks cb 0x557a1663c1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.163+0000 7f46a2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 0x557a166e88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.163+0000 7f46a2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 0x557a166e88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33612/0 (socket says 172.21.15.47:33612) 2024-03-21T11:48:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.163+0000 7f46a2a00640 1 -- 172.21.15.47:0/1574203427 learned_addr learned my addr 172.21.15.47:0/1574203427 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.163+0000 7f469be00640 1 --2- 172.21.15.47:0/1574203427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.163+0000 7f469be00640 1 --2- 172.21.15.47:0/1574203427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166e3d40 0x557a166ef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.164+0000 7f46a2a00640 1 -- 172.21.15.47:0/1574203427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 msgr2=0x557a166e8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.164+0000 7f46a2a00640 1 --2- 172.21.15.47:0/1574203427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166e8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.164+0000 7f46a2a00640 1 -- 172.21.15.47:0/1574203427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166e3d40 msgr2=0x557a166ef910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.164+0000 7f46a2a00640 1 --2- 172.21.15.47:0/1574203427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166e3d40 0x557a166ef910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.164+0000 7f46a2a00640 1 -- 172.21.15.47:0/1574203427 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a16358210 con 0x557a166debd0 2024-03-21T11:48:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.164+0000 7f46a2a00640 1 --2- 172.21.15.47:0/1574203427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 0x557a166e88d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4694036040 tx=0x7f46940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d866b1ce9aaf7fa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.164+0000 7f4687e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:48:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.165+0000 7f4687e00640 1 -- 172.21.15.47:0/1574203427 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4694042070 con 0x557a166debd0 2024-03-21T11:48:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.165+0000 7f4687e00640 1 -- 172.21.15.47:0/1574203427 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4694005230 con 0x557a166debd0 2024-03-21T11:48:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.165+0000 7f46ac232740 1 -- 172.21.15.47:0/1574203427 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f468c012ad0 con 0x557a166debd0 2024-03-21T11:48:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.165+0000 7f4687e00640 1 -- 172.21.15.47:0/1574203427 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f469404f050 con 0x557a166debd0 2024-03-21T11:48:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.165+0000 7f46ac232740 10 client.22482 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.165+0000 7f46ac232740 1 -- 172.21.15.47:0/1574203427 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x557a16424620 con 0x557a166debd0 2024-03-21T11:48:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.166+0000 7f4687e00640 1 -- 172.21.15.47:0/1574203427 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f468c012ad0 con 0x557a166debd0 2024-03-21T11:48:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.166+0000 7f4687e00640 1 -- 172.21.15.47:0/1574203427 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4694003890 con 0x557a166debd0 2024-03-21T11:48:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.166+0000 7f46ac232740 10 client.22482 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.166+0000 7f46ac232740 10 client.22482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.166+0000 7f46ac232740 20 client.22482 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.166+0000 7f46ac232740 1 --2- 172.21.15.47:0/1574203427 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557a166f4190 0x557a166f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.166+0000 7f46ac232740 4 client.22482 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:48:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.166+0000 7f46ac232740 1 -- 172.21.15.47:0/1574203427 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a16650d70 con 0x557a166f4190 2024-03-21T11:48:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.166+0000 7f469be00640 1 --2- 172.21.15.47:0/1574203427 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557a166f4190 0x557a166f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.167+0000 7f469be00640 1 --2- 172.21.15.47:0/1574203427 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557a166f4190 0x557a166f6580 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=672d957026a73474 server_cookie=916962e776156e1e in_seq=0 out_seq=0 2024-03-21T11:48:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.167+0000 7f4687e00640 10 client.22482 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.168+0000 7f4687e00640 1 -- 172.21.15.47:0/1574203427 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557a16650d70 con 0x557a166f4190 2024-03-21T11:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.168+0000 7f4687e00640 10 client.22482 handle_command_reply: tid=0 2024-03-21T11:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.168+0000 7f46ac232740 1 client.22482 shutdown 2024-03-21T11:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.168+0000 7f46ac232740 1 -- 172.21.15.47:0/1574203427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 msgr2=0x557a166e88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.168+0000 7f46ac232740 1 --2- 172.21.15.47:0/1574203427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 0x557a166e88d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4694036040 tx=0x7f46940042f0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.169+0000 7f46ac232740 1 -- 172.21.15.47:0/1574203427 shutdown_connections 2024-03-21T11:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.169+0000 7f46ac232740 1 --2- 172.21.15.47:0/1574203427 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557a166f4190 0x557a166f6580 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.169+0000 7f46ac232740 1 --2- 172.21.15.47:0/1574203427 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166e3d40 0x557a166ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.169+0000 7f46ac232740 1 --2- 172.21.15.47:0/1574203427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 0x557a166e88d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.169+0000 7f46ac232740 1 --2- 172.21.15.47:0/1574203427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.169+0000 7f46ac232740 1 -- 172.21.15.47:0/1574203427 >> 172.21.15.47:0/1574203427 conn(0x557a166d9750 msgr2=0x557a166d8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.169+0000 7f4687e00640 0 client.22482 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.169+0000 7f46ac232740 1 -- 172.21.15.47:0/1574203427 shutdown_connections 2024-03-21T11:48:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.169+0000 7f46ac232740 1 -- 172.21.15.47:0/1574203427 wait complete. 2024-03-21T11:48:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.169+0000 7f46ac232740 20 client.22482 trim_cache size 0 max 16384 2024-03-21T11:48:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.191+0000 7f46ac232740 1 Processor -- start 2024-03-21T11:48:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.191+0000 7f46ac232740 1 -- start start 2024-03-21T11:48:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.191+0000 7f46ac232740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166d9750 0x557a166dc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.191+0000 7f46ac232740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166de1e0 0x557a166ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.191+0000 7f46ac232740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166debd0 0x557a166dcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.191+0000 7f46ac232740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a41b0730 con 0x557a166de1e0 2024-03-21T11:48:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.191+0000 7f46ac232740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a41b20b0 con 0x557a166debd0 2024-03-21T11:48:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.191+0000 7f46ac232740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a41b0d90 con 0x557a166d9750 2024-03-21T11:48:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.192+0000 7f469be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166d9750 0x557a166dc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.192+0000 7f46a2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166de1e0 0x557a166ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.192+0000 7f469be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166d9750 0x557a166dc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51702/0 (socket says 172.21.15.47:51702) 2024-03-21T11:48:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.192+0000 7f469be00640 1 -- 172.21.15.47:0/2464559246 learned_addr learned my addr 172.21.15.47:0/2464559246 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.192+0000 7f469be00640 1 --2- 172.21.15.47:0/2464559246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166debd0 0x557a166dcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.192+0000 7f469be00640 1 -- 172.21.15.47:0/2464559246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166debd0 msgr2=0x557a166dcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.192+0000 7f469be00640 1 --2- 172.21.15.47:0/2464559246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166debd0 0x557a166dcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.192+0000 7f469be00640 1 -- 172.21.15.47:0/2464559246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166de1e0 msgr2=0x557a166ef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.192+0000 7f469be00640 1 --2- 172.21.15.47:0/2464559246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166de1e0 0x557a166ef8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.192+0000 7f469be00640 1 -- 172.21.15.47:0/2464559246 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a166f68f0 con 0x557a166d9750 2024-03-21T11:48:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.192+0000 7f469be00640 1 --2- 172.21.15.47:0/2464559246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166d9750 0x557a166dc9b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f468c00bf30 tx=0x7f468c017040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e261aa816b23eb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.193+0000 7f4692a00640 1 -- 172.21.15.47:0/2464559246 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c035070 con 0x557a166d9750 2024-03-21T11:48:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.193+0000 7f4692a00640 1 -- 172.21.15.47:0/2464559246 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f468c0043d0 con 0x557a166d9750 2024-03-21T11:48:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.193+0000 7f4692a00640 1 -- 172.21.15.47:0/2464559246 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c02ecd0 con 0x557a166d9750 2024-03-21T11:48:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.195+0000 7f46ac232740 1 -- 172.21.15.47:0/2464559246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166d9750 msgr2=0x557a166dc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.195+0000 7f46ac232740 1 --2- 172.21.15.47:0/2464559246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166d9750 0x557a166dc9b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f468c00bf30 tx=0x7f468c017040 comp rx=0 tx=0).stop 2024-03-21T11:48:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.195+0000 7f46ac232740 1 -- 172.21.15.47:0/2464559246 shutdown_connections 2024-03-21T11:48:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.195+0000 7f46ac232740 1 --2- 172.21.15.47:0/2464559246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166debd0 0x557a166dcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.195+0000 7f46ac232740 1 --2- 172.21.15.47:0/2464559246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166de1e0 0x557a166ef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.195+0000 7f46ac232740 1 --2- 172.21.15.47:0/2464559246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166d9750 0x557a166dc9b0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.195+0000 7f46ac232740 1 -- 172.21.15.47:0/2464559246 >> 172.21.15.47:0/2464559246 conn(0x557a166df7d0 msgr2=0x557a166dfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.196+0000 7f46ac232740 1 -- 172.21.15.47:0/2464559246 shutdown_connections 2024-03-21T11:48:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.196+0000 7f46ac232740 1 -- 172.21.15.47:0/2464559246 wait complete. 2024-03-21T11:48:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.196+0000 7f46ac232740 1 -- start start 2024-03-21T11:48:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.197+0000 7f46ac232740 1 Processor -- start 2024-03-21T11:48:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.197+0000 7f46ac232740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166d9750 0x557a166ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.197+0000 7f46ac232740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166ef660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.197+0000 7f46ac232740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 0x557a166ed780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.197+0000 7f46ac232740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a41b1a50 con 0x557a166debd0 2024-03-21T11:48:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.197+0000 7f46ac232740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a41b13f0 con 0x557a166d9750 2024-03-21T11:48:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.198+0000 7f46ac232740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a41ae8f0 con 0x557a166de1e0 2024-03-21T11:48:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.198+0000 7f46ac232740 10 client.0 _ll_register_callbacks cb 0x7f4694005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.198+0000 7f469be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166d9750 0x557a166ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.198+0000 7f46a2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166ef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.198+0000 7f46a2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166ef660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:51710/0 (socket says 172.21.15.47:51710) 2024-03-21T11:48:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.198+0000 7f46a2a00640 1 -- 172.21.15.47:0/113586866 learned_addr learned my addr 172.21.15.47:0/113586866 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.199+0000 7f46a2a00640 1 -- 172.21.15.47:0/113586866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166d9750 msgr2=0x557a166ef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.199+0000 7f46a2a00640 1 --2- 172.21.15.47:0/113586866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166d9750 0x557a166ef120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.199+0000 7f46a2a00640 1 -- 172.21.15.47:0/113586866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 msgr2=0x557a166ed780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.199+0000 7f46a2a00640 1 --2- 172.21.15.47:0/113586866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 0x557a166ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.199+0000 7f46a2a00640 1 -- 172.21.15.47:0/113586866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a166eddc0 con 0x557a166de1e0 2024-03-21T11:48:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.199+0000 7f46a2a00640 1 --2- 172.21.15.47:0/113586866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166ef660 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f469402c170 tx=0x7f4694005810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=315203e5b9b446ba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.199+0000 7f4687e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:48:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.201+0000 7f4687e00640 1 -- 172.21.15.47:0/113586866 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4694042070 con 0x557a166de1e0 2024-03-21T11:48:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.201+0000 7f4687e00640 1 -- 172.21.15.47:0/113586866 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f4694003a00 con 0x557a166de1e0 2024-03-21T11:48:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.201+0000 7f46ac232740 1 -- 172.21.15.47:0/113586866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f468c0043d0 con 0x557a166de1e0 2024-03-21T11:48:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.202+0000 7f4687e00640 1 -- 172.21.15.47:0/113586866 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f469402cd50 con 0x557a166de1e0 2024-03-21T11:48:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.202+0000 7f46ac232740 10 client.33325 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.202+0000 7f46ac232740 1 -- 172.21.15.47:0/113586866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x557a16761450 con 0x557a166de1e0 2024-03-21T11:48:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.203+0000 7f4687e00640 1 -- 172.21.15.47:0/113586866 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f468c0043d0 con 0x557a166de1e0 2024-03-21T11:48:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.203+0000 7f4687e00640 1 -- 172.21.15.47:0/113586866 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4694034440 con 0x557a166de1e0 2024-03-21T11:48:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.203+0000 7f46ac232740 10 client.33325 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.203+0000 7f46ac232740 10 client.33325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.203+0000 7f46ac232740 20 client.33325 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.203+0000 7f46ac232740 1 --2- 172.21.15.47:0/113586866 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557a166e3d40 0x557a16763d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.203+0000 7f46ac232740 4 client.33325 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:48:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.203+0000 7f46ac232740 1 -- 172.21.15.47:0/113586866 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557a1664b0c0 con 0x557a166e3d40 2024-03-21T11:48:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.203+0000 7f469be00640 1 --2- 172.21.15.47:0/113586866 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557a166e3d40 0x557a16763d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.204+0000 7f469be00640 1 --2- 172.21.15.47:0/113586866 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557a166e3d40 0x557a16763d60 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9afb3b11dbdbb3b8 server_cookie=21961ab905110a3a in_seq=0 out_seq=0 2024-03-21T11:48:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:09.204+0000 7f4687e00640 10 client.33325 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:09.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:09 smithi047 ceph-mon[31351]: pgmap v1145: 129 pgs: 129 active+clean; 746 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.1 MiB/s wr, 583 op/s 2024-03-21T11:48:09.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:09 smithi100 ceph-mon[36245]: pgmap v1145: 129 pgs: 129 active+clean; 746 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.1 MiB/s wr, 583 op/s 2024-03-21T11:48:09.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:09 smithi102 ceph-mon[36748]: pgmap v1145: 129 pgs: 129 active+clean; 746 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.1 MiB/s wr, 583 op/s 2024-03-21T11:48:09.999 INFO:tasks.workunit.client.0.smithi047.stdout: 10 272740 21.67 MB/sec execute 533 sec latency 768.023 ms 2024-03-21T11:48:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.214+0000 7f4687e00640 1 -- 172.21.15.47:0/113586866 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557a1664b0c0 con 0x557a166e3d40 2024-03-21T11:48:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.214+0000 7f4687e00640 10 client.33325 handle_command_reply: tid=0 2024-03-21T11:48:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.214+0000 7f46ac232740 1 client.33325 shutdown 2024-03-21T11:48:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.215+0000 7f46ac232740 1 -- 172.21.15.47:0/113586866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 msgr2=0x557a166ef660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.215+0000 7f46ac232740 1 --2- 172.21.15.47:0/113586866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166ef660 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f469402c170 tx=0x7f4694005810 comp rx=0 tx=0).stop 2024-03-21T11:48:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.215+0000 7f46ac232740 1 -- 172.21.15.47:0/113586866 shutdown_connections 2024-03-21T11:48:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.215+0000 7f46ac232740 1 --2- 172.21.15.47:0/113586866 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x557a166e3d40 0x557a16763d60 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.215+0000 7f46ac232740 1 --2- 172.21.15.47:0/113586866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557a166debd0 0x557a166ed780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.215+0000 7f46ac232740 1 --2- 172.21.15.47:0/113586866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557a166de1e0 0x557a166ef660 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.215+0000 7f46ac232740 1 --2- 172.21.15.47:0/113586866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a166d9750 0x557a166ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.215+0000 7f4687e00640 0 client.33325 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.215+0000 7f46ac232740 1 -- 172.21.15.47:0/113586866 >> 172.21.15.47:0/113586866 conn(0x557a166df7d0 msgr2=0x557a166e8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.215+0000 7f46ac232740 1 -- 172.21.15.47:0/113586866 shutdown_connections 2024-03-21T11:48:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.215+0000 7f46ac232740 1 -- 172.21.15.47:0/113586866 wait complete. 2024-03-21T11:48:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.215+0000 7f46ac232740 20 client.33325 trim_cache size 0 max 16384 2024-03-21T11:48:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:48:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:48:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:48:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:48:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.216+0000 7f46a9600640 1 -- 172.21.15.47:0/3867079880 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46880a30c0 msgr2=0x7f46880a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.216+0000 7f46a9600640 1 --2- 172.21.15.47:0/3867079880 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46880a30c0 0x7f46880a5580 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f469c000f80 tx=0x7f469c0058d0 comp rx=0 tx=0).stop 2024-03-21T11:48:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.216+0000 7f46a9600640 1 -- 172.21.15.47:0/3867079880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a410dcc0 msgr2=0x7f46a41a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.216+0000 7f46a9600640 1 --2- 172.21.15.47:0/3867079880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a410dcc0 0x7f46a41a54d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f469c00e940 tx=0x7f469c00ee00 comp rx=0 tx=0).stop 2024-03-21T11:48:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.216+0000 7f46a9600640 1 -- 172.21.15.47:0/3867079880 shutdown_connections 2024-03-21T11:48:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.216+0000 7f46a9600640 1 --2- 172.21.15.47:0/3867079880 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f46880a30c0 0x7f46880a5580 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.216+0000 7f46a9600640 1 --2- 172.21.15.47:0/3867079880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a410dcc0 0x7f46a41a54d0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.216+0000 7f46a9600640 1 --2- 172.21.15.47:0/3867079880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46a40756e0 0x7f46a419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.216+0000 7f46a9600640 1 --2- 172.21.15.47:0/3867079880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46a4074100 0x7f46a419df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.216+0000 7f46a9600640 1 -- 172.21.15.47:0/3867079880 >> 172.21.15.47:0/3867079880 conn(0x7f46a40fcc60 msgr2=0x7f46a4101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.218+0000 7f46a9600640 1 -- 172.21.15.47:0/3867079880 shutdown_connections 2024-03-21T11:48:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:10.218+0000 7f46a9600640 1 -- 172.21.15.47:0/3867079880 wait complete. 2024-03-21T11:48:10.999 INFO:tasks.workunit.client.0.smithi047.stdout: 10 273236 21.69 MB/sec execute 534 sec latency 1004.146 ms 2024-03-21T11:48:11.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:11 smithi047 ceph-mon[31351]: pgmap v1146: 129 pgs: 129 active+clean; 751 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.5 MiB/s wr, 531 op/s 2024-03-21T11:48:11.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:11 smithi100 ceph-mon[36245]: pgmap v1146: 129 pgs: 129 active+clean; 751 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.5 MiB/s wr, 531 op/s 2024-03-21T11:48:11.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:11 smithi102 ceph-mon[36748]: pgmap v1146: 129 pgs: 129 active+clean; 751 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.5 MiB/s wr, 531 op/s 2024-03-21T11:48:11.999 INFO:tasks.workunit.client.0.smithi047.stdout: 10 273535 21.67 MB/sec execute 535 sec latency 547.614 ms 2024-03-21T11:48:13.000 INFO:tasks.workunit.client.0.smithi047.stdout: 10 273893 21.68 MB/sec execute 536 sec latency 557.925 ms 2024-03-21T11:48:13.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:13 smithi047 ceph-mon[31351]: pgmap v1147: 129 pgs: 129 active+clean; 753 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.4 MiB/s wr, 493 op/s 2024-03-21T11:48:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:13 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:48:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:13 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:48:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:13 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:48:13.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:13 smithi100 ceph-mon[36245]: pgmap v1147: 129 pgs: 129 active+clean; 753 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.4 MiB/s wr, 493 op/s 2024-03-21T11:48:13.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:13 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:48:13.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:13 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:48:13.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:13 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:48:13.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:13 smithi102 ceph-mon[36748]: pgmap v1147: 129 pgs: 129 active+clean; 753 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.4 MiB/s wr, 493 op/s 2024-03-21T11:48:13.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:13 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:48:13.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:13 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:48:13.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:13 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:48:14.000 INFO:tasks.workunit.client.0.smithi047.stdout: 10 274241 21.67 MB/sec execute 537 sec latency 145.868 ms 2024-03-21T11:48:14.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:14 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:48:14.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:14 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:48:14.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:14 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:48:14.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:14 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:48:14.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:14 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:48:14.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:14 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:48:15.000 INFO:tasks.workunit.client.0.smithi047.stdout: 10 274695 21.67 MB/sec execute 538 sec latency 53.733 ms 2024-03-21T11:48:15.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:15 smithi047 ceph-mon[31351]: pgmap v1148: 129 pgs: 129 active+clean; 748 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.2 MiB/s wr, 624 op/s 2024-03-21T11:48:15.641 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:15 smithi102 ceph-mon[36748]: pgmap v1148: 129 pgs: 129 active+clean; 748 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.2 MiB/s wr, 624 op/s 2024-03-21T11:48:15.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:15 smithi100 ceph-mon[36245]: pgmap v1148: 129 pgs: 129 active+clean; 748 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.2 MiB/s wr, 624 op/s 2024-03-21T11:48:16.000 INFO:tasks.workunit.client.0.smithi047.stdout: 10 275088 21.66 MB/sec execute 539 sec latency 65.468 ms 2024-03-21T11:48:17.000 INFO:tasks.workunit.client.0.smithi047.stdout: 10 275462 21.66 MB/sec execute 540 sec latency 420.219 ms 2024-03-21T11:48:17.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:17 smithi047 ceph-mon[31351]: pgmap v1149: 129 pgs: 129 active+clean; 748 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 508 op/s 2024-03-21T11:48:17.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:17 smithi100 ceph-mon[36245]: pgmap v1149: 129 pgs: 129 active+clean; 748 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 508 op/s 2024-03-21T11:48:17.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:17 smithi102 ceph-mon[36748]: pgmap v1149: 129 pgs: 129 active+clean; 748 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 508 op/s 2024-03-21T11:48:18.000 INFO:tasks.workunit.client.0.smithi047.stdout: 10 275729 21.64 MB/sec execute 541 sec latency 730.207 ms 2024-03-21T11:48:19.002 INFO:tasks.workunit.client.0.smithi047.stdout: 10 275978 21.63 MB/sec execute 542 sec latency 305.728 ms 2024-03-21T11:48:19.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:19 smithi047 ceph-mon[31351]: pgmap v1150: 129 pgs: 129 active+clean; 745 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.9 MiB/s wr, 582 op/s 2024-03-21T11:48:19.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:19 smithi100 ceph-mon[36245]: pgmap v1150: 129 pgs: 129 active+clean; 745 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.9 MiB/s wr, 582 op/s 2024-03-21T11:48:19.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:19 smithi102 ceph-mon[36748]: pgmap v1150: 129 pgs: 129 active+clean; 745 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.9 MiB/s wr, 582 op/s 2024-03-21T11:48:20.000 INFO:tasks.workunit.client.0.smithi047.stdout: 10 276532 21.67 MB/sec execute 543 sec latency 87.025 ms 2024-03-21T11:48:20.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:20 smithi100 ceph-mon[36245]: pgmap v1151: 129 pgs: 129 active+clean; 734 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 560 op/s 2024-03-21T11:48:20.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:20 smithi102 ceph-mon[36748]: pgmap v1151: 129 pgs: 129 active+clean; 734 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 560 op/s 2024-03-21T11:48:20.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:20 smithi047 ceph-mon[31351]: pgmap v1151: 129 pgs: 129 active+clean; 734 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.0 MiB/s wr, 560 op/s 2024-03-21T11:48:21.000 INFO:tasks.workunit.client.0.smithi047.stdout: 10 276910 21.66 MB/sec execute 544 sec latency 98.067 ms 2024-03-21T11:48:22.000 INFO:tasks.workunit.client.0.smithi047.stdout: 10 277213 21.64 MB/sec execute 545 sec latency 316.099 ms 2024-03-21T11:48:23.000 INFO:tasks.workunit.client.0.smithi047.stdout: 10 277301 21.61 MB/sec execute 546 sec latency 737.937 ms 2024-03-21T11:48:23.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:22 smithi100 ceph-mon[36245]: pgmap v1152: 129 pgs: 129 active+clean; 737 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.7 MiB/s wr, 555 op/s 2024-03-21T11:48:23.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:22 smithi102 ceph-mon[36748]: pgmap v1152: 129 pgs: 129 active+clean; 737 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.7 MiB/s wr, 555 op/s 2024-03-21T11:48:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:22 smithi047 ceph-mon[31351]: pgmap v1152: 129 pgs: 129 active+clean; 737 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.7 MiB/s wr, 555 op/s 2024-03-21T11:48:23.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.450+0000 7f3a2fe00640 1 -- 172.21.15.47:0/321806395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a30072be0 msgr2=0x7f3a30073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.452+0000 7f3a2fe00640 1 --2- 172.21.15.47:0/321806395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a30072be0 0x7f3a30073060 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009900 tx=0x7f3a1c02bc40 comp rx=0 tx=0).stop 2024-03-21T11:48:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.452+0000 7f3a2fe00640 1 -- 172.21.15.47:0/321806395 shutdown_connections 2024-03-21T11:48:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.452+0000 7f3a2fe00640 1 --2- 172.21.15.47:0/321806395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a3010f9e0 0x7f3a30111dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.452+0000 7f3a2fe00640 1 --2- 172.21.15.47:0/321806395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a30072be0 0x7f3a30073060 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.452+0000 7f3a2fe00640 1 --2- 172.21.15.47:0/321806395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a30109280 0x7f3a300726a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.452+0000 7f3a2fe00640 1 -- 172.21.15.47:0/321806395 >> 172.21.15.47:0/321806395 conn(0x7f3a3006e510 msgr2=0x7f3a30070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.452+0000 7f3a2fe00640 1 -- 172.21.15.47:0/321806395 shutdown_connections 2024-03-21T11:48:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.453+0000 7f3a2fe00640 1 -- 172.21.15.47:0/321806395 wait complete. 2024-03-21T11:48:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.454+0000 7f3a2fe00640 1 Processor -- start 2024-03-21T11:48:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.454+0000 7f3a2fe00640 1 -- start start 2024-03-21T11:48:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.455+0000 7f3a2fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a30072be0 0x7f3a3019de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.456+0000 7f3a2fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a30109280 0x7f3a3019e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.456+0000 7f3a2fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a3010f9e0 0x7f3a301a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.457+0000 7f3a2fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a30113810 con 0x7f3a3010f9e0 2024-03-21T11:48:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.457+0000 7f3a2fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a30113690 con 0x7f3a30072be0 2024-03-21T11:48:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.457+0000 7f3a2fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a30113990 con 0x7f3a30109280 2024-03-21T11:48:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.457+0000 7f3a2e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a30109280 0x7f3a3019e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.457+0000 7f3a2ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a30072be0 0x7f3a3019de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.457+0000 7f3a2ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a30072be0 0x7f3a3019de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49036/0 (socket says 172.21.15.47:49036) 2024-03-21T11:48:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.457+0000 7f3a2f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a3010f9e0 0x7f3a301a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.458+0000 7f3a2f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a3010f9e0 0x7f3a301a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55962/0 (socket says 172.21.15.47:55962) 2024-03-21T11:48:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.458+0000 7f3a2f400640 1 -- 172.21.15.47:0/2624963556 learned_addr learned my addr 172.21.15.47:0/2624963556 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.457+0000 7f3a2e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a30109280 0x7f3a3019e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33232/0 (socket says 172.21.15.47:33232) 2024-03-21T11:48:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.459+0000 7f3a2e000640 1 -- 172.21.15.47:0/2624963556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a30072be0 msgr2=0x7f3a3019de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.459+0000 7f3a2e000640 1 --2- 172.21.15.47:0/2624963556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a30072be0 0x7f3a3019de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.459+0000 7f3a2e000640 1 -- 172.21.15.47:0/2624963556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a3010f9e0 msgr2=0x7f3a301a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.459+0000 7f3a2e000640 1 --2- 172.21.15.47:0/2624963556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a3010f9e0 0x7f3a301a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.459+0000 7f3a2e000640 1 -- 172.21.15.47:0/2624963556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a301a5a50 con 0x7f3a30109280 2024-03-21T11:48:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.460+0000 7f3a2e000640 1 --2- 172.21.15.47:0/2624963556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a30109280 0x7f3a3019e370 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009230 tx=0x7f3a1c004e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.461+0000 7f3a27400640 1 -- 172.21.15.47:0/2624963556 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c041070 con 0x7f3a30109280 2024-03-21T11:48:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.461+0000 7f3a27400640 1 -- 172.21.15.47:0/2624963556 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a1c0054e0 con 0x7f3a30109280 2024-03-21T11:48:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.461+0000 7f3a27400640 1 -- 172.21.15.47:0/2624963556 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c003e10 con 0x7f3a30109280 2024-03-21T11:48:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.462+0000 7f3a2fe00640 1 -- 172.21.15.47:0/2624963556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a301a5ce0 con 0x7f3a30109280 2024-03-21T11:48:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.462+0000 7f3a2fe00640 1 -- 172.21.15.47:0/2624963556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a301a6150 con 0x7f3a30109280 2024-03-21T11:48:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.470+0000 7f3a38a04740 1 Processor -- start 2024-03-21T11:48:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.470+0000 7f3a38a04740 1 -- start start 2024-03-21T11:48:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.471+0000 7f3a38a04740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804ac00 0x5607c804b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.471+0000 7f3a38a04740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804b5f0 0x5607c80500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.471+0000 7f3a38a04740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c8050760 0x5607c8052bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.472+0000 7f3a38a04740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c80053b0 con 0x5607c8050760 2024-03-21T11:48:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.472+0000 7f3a38a04740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c7fa7830 con 0x5607c804ac00 2024-03-21T11:48:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.472+0000 7f3a38a04740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c7f505d0 con 0x5607c804b5f0 2024-03-21T11:48:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.472+0000 7f3a2ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804b5f0 0x5607c80500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.472+0000 7f3a2ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804b5f0 0x5607c80500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33248/0 (socket says 172.21.15.47:33248) 2024-03-21T11:48:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.472+0000 7f3a2ea00640 1 -- 172.21.15.47:0/2330091066 learned_addr learned my addr 172.21.15.47:0/2330091066 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.473+0000 7f3a2ea00640 1 -- 172.21.15.47:0/2330091066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804ac00 msgr2=0x5607c804b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.473+0000 7f3a2ea00640 1 --2- 172.21.15.47:0/2330091066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804ac00 0x5607c804b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.473+0000 7f3a2ea00640 1 -- 172.21.15.47:0/2330091066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c8050760 msgr2=0x5607c8052bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:48:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.473+0000 7f3a2ea00640 1 --2- 172.21.15.47:0/2330091066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c8050760 0x5607c8052bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.473+0000 7f3a2ea00640 1 -- 172.21.15.47:0/2330091066 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607c7f682c0 con 0x5607c804b5f0 2024-03-21T11:48:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.473+0000 7f3a2ea00640 1 --2- 172.21.15.47:0/2330091066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804b5f0 0x5607c80500f0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009c80 tx=0x7f3a200117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e087687a319e59b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.474+0000 7f3a1be00640 1 -- 172.21.15.47:0/2330091066 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20012450 con 0x5607c804b5f0 2024-03-21T11:48:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.474+0000 7f3a1be00640 1 -- 172.21.15.47:0/2330091066 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a20012ad0 con 0x5607c804b5f0 2024-03-21T11:48:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.474+0000 7f3a1be00640 1 -- 172.21.15.47:0/2330091066 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20012450 con 0x5607c804b5f0 2024-03-21T11:48:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.475+0000 7f3a38a04740 1 -- 172.21.15.47:0/2330091066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804b5f0 msgr2=0x5607c80500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.475+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2330091066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804b5f0 0x5607c80500f0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009c80 tx=0x7f3a200117b0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.475+0000 7f3a27400640 1 -- 172.21.15.47:0/2624963556 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3a1c044070 con 0x7f3a30109280 2024-03-21T11:48:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.477+0000 7f3a38a04740 1 -- 172.21.15.47:0/2330091066 shutdown_connections 2024-03-21T11:48:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.477+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2330091066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c8050760 0x5607c8052bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.477+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2330091066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804b5f0 0x5607c80500f0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.477+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2330091066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804ac00 0x5607c804b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.477+0000 7f3a38a04740 1 -- 172.21.15.47:0/2330091066 >> 172.21.15.47:0/2330091066 conn(0x5607c8046170 msgr2=0x5607c8046570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.479+0000 7f3a38a04740 1 -- 172.21.15.47:0/2330091066 shutdown_connections 2024-03-21T11:48:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.479+0000 7f3a38a04740 1 -- 172.21.15.47:0/2330091066 wait complete. 2024-03-21T11:48:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.480+0000 7f3a38a04740 1 -- start start 2024-03-21T11:48:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.480+0000 7f3a38a04740 1 Processor -- start 2024-03-21T11:48:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.481+0000 7f3a38a04740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804ac00 0x5607c8054ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.481+0000 7f3a27400640 1 --2- 172.21.15.47:0/2624963556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a140a3050 0x7f3a140a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.481+0000 7f3a2f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804ac00 0x5607c8054ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.481+0000 7f3a2f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804ac00 0x5607c8054ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49064/0 (socket says 172.21.15.47:49064) 2024-03-21T11:48:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.481+0000 7f3a2f400640 1 -- 172.21.15.47:0/2914220979 learned_addr learned my addr 172.21.15.47:0/2914220979 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.481+0000 7f3a27400640 1 -- 172.21.15.47:0/2624963556 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3a1c0f0e40 con 0x7f3a30109280 2024-03-21T11:48:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.481+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804b5f0 0x5607c80553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.482+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c8050760 0x5607c805c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.482+0000 7f3a38a04740 1 -- 172.21.15.47:0/2914220979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c7f63a20 con 0x5607c8050760 2024-03-21T11:48:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.482+0000 7f3a38a04740 1 -- 172.21.15.47:0/2914220979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c7fa38d0 con 0x5607c804ac00 2024-03-21T11:48:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.482+0000 7f3a38a04740 1 -- 172.21.15.47:0/2914220979 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607c8053710 con 0x5607c804b5f0 2024-03-21T11:48:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.482+0000 7f3a38a04740 10 client.0 _ll_register_callbacks cb 0x5607c7fa01c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.483+0000 7f3a2e000640 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804b5f0 0x5607c80553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.483+0000 7f3a2f400640 1 -- 172.21.15.47:0/2914220979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804b5f0 msgr2=0x5607c80553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.483+0000 7f3a2f400640 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804b5f0 0x5607c80553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.483+0000 7f3a2f400640 1 -- 172.21.15.47:0/2914220979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c8050760 msgr2=0x5607c805c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.483+0000 7f3a2f400640 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c8050760 0x5607c805c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.483+0000 7f3a2f400640 1 -- 172.21.15.47:0/2914220979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607c7cbc210 con 0x5607c804ac00 2024-03-21T11:48:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.483+0000 7f3a2e000640 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804b5f0 0x5607c80553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:48:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.484+0000 7f3a2f400640 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804ac00 0x5607c8054ea0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009db0 tx=0x7f3a28009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0adc1c77c65dc96 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.484+0000 7f3a18c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:48:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.484+0000 7f3a18c00640 1 -- 172.21.15.47:0/2914220979 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2800c8a0 con 0x5607c804ac00 2024-03-21T11:48:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.484+0000 7f3a18c00640 1 -- 172.21.15.47:0/2914220979 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3a28012070 con 0x5607c804ac00 2024-03-21T11:48:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.485+0000 7f3a38a04740 1 -- 172.21.15.47:0/2914220979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a20012ad0 con 0x5607c804ac00 2024-03-21T11:48:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.485+0000 7f3a2ea00640 1 --2- 172.21.15.47:0/2624963556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a140a3050 0x7f3a140a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.486+0000 7f3a18c00640 1 -- 172.21.15.47:0/2914220979 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a2800c8a0 con 0x5607c804ac00 2024-03-21T11:48:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.486+0000 7f3a38a04740 10 client.33371 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.486+0000 7f3a38a04740 1 -- 172.21.15.47:0/2914220979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5607c7d88620 con 0x5607c804ac00 2024-03-21T11:48:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.487+0000 7f3a18c00640 1 -- 172.21.15.47:0/2914220979 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a2801a070 con 0x5607c804ac00 2024-03-21T11:48:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.487+0000 7f3a18c00640 1 -- 172.21.15.47:0/2914220979 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3a28015450 con 0x5607c804ac00 2024-03-21T11:48:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.487+0000 7f3a38a04740 10 client.33371 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.487+0000 7f3a38a04740 10 client.33371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.487+0000 7f3a38a04740 20 client.33371 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.487+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5607c80610d0 0x5607c80634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.488+0000 7f3a38a04740 4 client.33371 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:48:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.488+0000 7f3a38a04740 1 -- 172.21.15.47:0/2914220979 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a20012ad0 con 0x5607c80610d0 2024-03-21T11:48:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.488+0000 7f3a2e000640 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5607c80610d0 0x5607c80634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.489+0000 7f3a2e000640 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5607c80610d0 0x5607c80634f0 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7fb8d72e81c098 server_cookie=868865a62f73a986 in_seq=0 out_seq=0 2024-03-21T11:48:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.489+0000 7f3a18c00640 10 client.33371 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.491+0000 7f3a18c00640 1 -- 172.21.15.47:0/2914220979 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f3a20012ad0 con 0x5607c80610d0 2024-03-21T11:48:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.491+0000 7f3a2ea00640 1 --2- 172.21.15.47:0/2624963556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a140a3050 0x7f3a140a5510 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009530 tx=0x7f3a200056d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.491+0000 7f3a18c00640 10 client.33371 handle_command_reply: tid=0 2024-03-21T11:48:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.491+0000 7f3a38a04740 1 client.33371 shutdown 2024-03-21T11:48:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.492+0000 7f3a38a04740 1 -- 172.21.15.47:0/2914220979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804ac00 msgr2=0x5607c8054ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.492+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804ac00 0x5607c8054ea0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009db0 tx=0x7f3a28009ea0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.492+0000 7f3a38a04740 1 -- 172.21.15.47:0/2914220979 shutdown_connections 2024-03-21T11:48:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.492+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5607c80610d0 0x5607c80634f0 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.492+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c8050760 0x5607c805c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.493+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804b5f0 0x5607c80553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.493+0000 7f3a18c00640 0 client.33371 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.493+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2914220979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804ac00 0x5607c8054ea0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.493+0000 7f3a38a04740 1 -- 172.21.15.47:0/2914220979 >> 172.21.15.47:0/2914220979 conn(0x5607c8046170 msgr2=0x5607c803cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.496+0000 7f3a38a04740 1 -- 172.21.15.47:0/2914220979 shutdown_connections 2024-03-21T11:48:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.496+0000 7f3a38a04740 1 -- 172.21.15.47:0/2914220979 wait complete. 2024-03-21T11:48:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.497+0000 7f3a38a04740 20 client.33371 trim_cache size 0 max 16384 2024-03-21T11:48:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.543+0000 7f3a38a04740 1 Processor -- start 2024-03-21T11:48:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.543+0000 7f3a38a04740 1 -- start start 2024-03-21T11:48:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.544+0000 7f3a38a04740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c804ac00 0x5607c8055340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.544+0000 7f3a38a04740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804b5f0 0x5607c8050020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.544+0000 7f3a38a04740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c8050760 0x5607c8046550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.544+0000 7f3a38a04740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a301b04c0 con 0x5607c804ac00 2024-03-21T11:48:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.544+0000 7f3a38a04740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a301b1e40 con 0x5607c804b5f0 2024-03-21T11:48:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.544+0000 7f3a38a04740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a301b0b20 con 0x5607c8050760 2024-03-21T11:48:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.544+0000 7f3a2e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804b5f0 0x5607c8050020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.545+0000 7f3a2e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804b5f0 0x5607c8050020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49068/0 (socket says 172.21.15.47:49068) 2024-03-21T11:48:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.545+0000 7f3a2e000640 1 -- 172.21.15.47:0/2155575912 learned_addr learned my addr 172.21.15.47:0/2155575912 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.545+0000 7f3a2e000640 1 -- 172.21.15.47:0/2155575912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c8050760 msgr2=0x5607c8046550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:48:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.545+0000 7f3a2e000640 1 --2- 172.21.15.47:0/2155575912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c8050760 0x5607c8046550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.545+0000 7f3a2e000640 1 -- 172.21.15.47:0/2155575912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c804ac00 msgr2=0x5607c8055340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.545+0000 7f3a2e000640 1 --2- 172.21.15.47:0/2155575912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c804ac00 0x5607c8055340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.545+0000 7f3a2e000640 1 -- 172.21.15.47:0/2155575912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607c803bf80 con 0x5607c804b5f0 2024-03-21T11:48:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.547+0000 7f3a2e000640 1 --2- 172.21.15.47:0/2155575912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804b5f0 0x5607c8050020 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c003fd0 tx=0x7f3a1c0f0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28c1796e491110e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.547+0000 7f3a1b400640 1 -- 172.21.15.47:0/2155575912 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c0d5070 con 0x5607c804b5f0 2024-03-21T11:48:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.547+0000 7f3a1b400640 1 -- 172.21.15.47:0/2155575912 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a1c005690 con 0x5607c804b5f0 2024-03-21T11:48:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.547+0000 7f3a1b400640 1 -- 172.21.15.47:0/2155575912 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c0ee660 con 0x5607c804b5f0 2024-03-21T11:48:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.552+0000 7f3a38a04740 1 -- 172.21.15.47:0/2155575912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804b5f0 msgr2=0x5607c8050020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.552+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2155575912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804b5f0 0x5607c8050020 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c003fd0 tx=0x7f3a1c0f0040 comp rx=0 tx=0).stop 2024-03-21T11:48:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.552+0000 7f3a38a04740 1 -- 172.21.15.47:0/2155575912 shutdown_connections 2024-03-21T11:48:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.552+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2155575912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c8050760 0x5607c8046550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.552+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2155575912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804b5f0 0x5607c8050020 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.552+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2155575912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c804ac00 0x5607c8055340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.552+0000 7f3a38a04740 1 -- 172.21.15.47:0/2155575912 >> 172.21.15.47:0/2155575912 conn(0x5607c8046170 msgr2=0x5607c80386a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.552+0000 7f3a38a04740 1 -- 172.21.15.47:0/2155575912 shutdown_connections 2024-03-21T11:48:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.552+0000 7f3a38a04740 1 -- 172.21.15.47:0/2155575912 wait complete. 2024-03-21T11:48:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.553+0000 7f3a38a04740 1 -- start start 2024-03-21T11:48:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.553+0000 7f3a38a04740 1 Processor -- start 2024-03-21T11:48:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.554+0000 7f3a38a04740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804ac00 0x5607c80b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.554+0000 7f3a38a04740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804b5f0 0x5607c80b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.554+0000 7f3a38a04740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c8050760 0x5607c80bb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.555+0000 7f3a38a04740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a301b17e0 con 0x5607c8050760 2024-03-21T11:48:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.555+0000 7f3a38a04740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a301b1180 con 0x5607c804b5f0 2024-03-21T11:48:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.555+0000 7f3a38a04740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a301ae680 con 0x5607c804ac00 2024-03-21T11:48:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.555+0000 7f3a38a04740 10 client.0 _ll_register_callbacks cb 0x7f3a28009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.556+0000 7f3a2ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804ac00 0x5607c80b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.556+0000 7f3a2e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804b5f0 0x5607c80b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.556+0000 7f3a2ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804ac00 0x5607c80b67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:33278/0 (socket says 172.21.15.47:33278) 2024-03-21T11:48:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.556+0000 7f3a2ea00640 1 -- 172.21.15.47:0/2129019470 learned_addr learned my addr 172.21.15.47:0/2129019470 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.556+0000 7f3a2f400640 1 --2- 172.21.15.47:0/2129019470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c8050760 0x5607c80bb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.557+0000 7f3a2f400640 1 -- 172.21.15.47:0/2129019470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804ac00 msgr2=0x5607c80b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.558+0000 7f3a2f400640 1 --2- 172.21.15.47:0/2129019470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804ac00 0x5607c80b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.559+0000 7f3a2f400640 1 -- 172.21.15.47:0/2129019470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804b5f0 msgr2=0x5607c80b8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.559+0000 7f3a2f400640 1 --2- 172.21.15.47:0/2129019470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804b5f0 0x5607c80b8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.559+0000 7f3a2f400640 1 -- 172.21.15.47:0/2129019470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607c80bb7a0 con 0x5607c8050760 2024-03-21T11:48:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.560+0000 7f3a2e000640 1 --2- 172.21.15.47:0/2129019470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804b5f0 0x5607c80b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:48:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.561+0000 7f3a2ea00640 1 --2- 172.21.15.47:0/2129019470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804ac00 0x5607c80b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:48:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.566+0000 7f3a2f400640 1 --2- 172.21.15.47:0/2129019470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c8050760 0x5607c80bb260 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3a280158f0 tx=0x7f3a28015660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23b2a7a4bdac184 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.566+0000 7f3a18c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:48:23.567 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.567+0000 7f3a18c00640 1 -- 172.21.15.47:0/2129019470 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2802d060 con 0x5607c8050760 2024-03-21T11:48:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.567+0000 7f3a18c00640 1 -- 172.21.15.47:0/2129019470 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3a2800cc50 con 0x5607c8050760 2024-03-21T11:48:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.567+0000 7f3a38a04740 1 -- 172.21.15.47:0/2129019470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607c7fb4d70 con 0x5607c8050760 2024-03-21T11:48:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.567+0000 7f3a18c00640 1 -- 172.21.15.47:0/2129019470 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a28034050 con 0x5607c8050760 2024-03-21T11:48:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.567+0000 7f3a38a04740 10 client.22491 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.567+0000 7f3a38a04740 1 -- 172.21.15.47:0/2129019470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5607c80bca60 con 0x5607c8050760 2024-03-21T11:48:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.569+0000 7f3a18c00640 1 -- 172.21.15.47:0/2129019470 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5607c7fb4d70 con 0x5607c8050760 2024-03-21T11:48:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.572+0000 7f3a18c00640 1 -- 172.21.15.47:0/2129019470 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3a28027070 con 0x5607c8050760 2024-03-21T11:48:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.576+0000 7f3a38a04740 10 client.22491 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.576+0000 7f3a38a04740 10 client.22491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.576+0000 7f3a38a04740 20 client.22491 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.576+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2129019470 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5607c80bd300 0x5607c80bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.576+0000 7f3a38a04740 4 client.22491 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:48:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.576+0000 7f3a38a04740 1 -- 172.21.15.47:0/2129019470 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5607c7faf0c0 con 0x5607c80bd300 2024-03-21T11:48:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.576+0000 7f3a2ea00640 1 --2- 172.21.15.47:0/2129019470 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5607c80bd300 0x5607c80bf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.577+0000 7f3a2ea00640 1 --2- 172.21.15.47:0/2129019470 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5607c80bd300 0x5607c80bf720 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67cda7a03e64d3b server_cookie=cfbde793ba68d0b7 in_seq=0 out_seq=0 2024-03-21T11:48:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:23.577+0000 7f3a18c00640 10 client.22491 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:24.000 INFO:tasks.workunit.client.0.smithi047.stdout: 10 277390 21.57 MB/sec execute 547 sec latency 617.838 ms 2024-03-21T11:48:25.001 INFO:tasks.workunit.client.0.smithi047.stdout: 10 277394 21.53 MB/sec execute 548 sec latency 1617.909 ms 2024-03-21T11:48:25.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:24 smithi100 ceph-mon[36245]: pgmap v1153: 129 pgs: 129 active+clean; 739 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 582 op/s 2024-03-21T11:48:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:24 smithi102 ceph-mon[36748]: pgmap v1153: 129 pgs: 129 active+clean; 739 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 582 op/s 2024-03-21T11:48:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:24 smithi047 ceph-mon[31351]: pgmap v1153: 129 pgs: 129 active+clean; 739 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 582 op/s 2024-03-21T11:48:26.000 INFO:tasks.workunit.client.0.smithi047.stdout: 10 277426 21.50 MB/sec execute 549 sec latency 2617.967 ms 2024-03-21T11:48:27.001 INFO:tasks.workunit.client.0.smithi047.stdout: 10 277568 21.49 MB/sec execute 550 sec latency 2997.184 ms 2024-03-21T11:48:27.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:26 smithi100 ceph-mon[36245]: pgmap v1154: 129 pgs: 129 active+clean; 751 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 439 op/s 2024-03-21T11:48:27.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:26 smithi102 ceph-mon[36748]: pgmap v1154: 129 pgs: 129 active+clean; 751 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 439 op/s 2024-03-21T11:48:27.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:26 smithi047 ceph-mon[31351]: pgmap v1154: 129 pgs: 129 active+clean; 751 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.3 MiB/s wr, 439 op/s 2024-03-21T11:48:28.001 INFO:tasks.workunit.client.0.smithi047.stdout: 10 277655 21.45 MB/sec execute 551 sec latency 812.135 ms 2024-03-21T11:48:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.010+0000 7f3a18c00640 1 -- 172.21.15.47:0/2129019470 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5607c7faf0c0 con 0x5607c80bd300 2024-03-21T11:48:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.010+0000 7f3a18c00640 10 client.22491 handle_command_reply: tid=0 2024-03-21T11:48:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.010+0000 7f3a38a04740 1 client.22491 shutdown 2024-03-21T11:48:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.010+0000 7f3a38a04740 1 -- 172.21.15.47:0/2129019470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c8050760 msgr2=0x5607c80bb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.010+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2129019470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c8050760 0x5607c80bb260 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3a280158f0 tx=0x7f3a28015660 comp rx=0 tx=0).stop 2024-03-21T11:48:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.010+0000 7f3a38a04740 1 -- 172.21.15.47:0/2129019470 shutdown_connections 2024-03-21T11:48:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.010+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2129019470 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5607c80bd300 0x5607c80bf720 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.010+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2129019470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607c8050760 0x5607c80bb260 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.010+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2129019470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607c804b5f0 0x5607c80b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.010+0000 7f3a38a04740 1 --2- 172.21.15.47:0/2129019470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607c804ac00 0x5607c80b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.010+0000 7f3a18c00640 0 client.22491 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.010+0000 7f3a38a04740 1 -- 172.21.15.47:0/2129019470 >> 172.21.15.47:0/2129019470 conn(0x5607c8046170 msgr2=0x5607c8046550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.011+0000 7f3a38a04740 1 -- 172.21.15.47:0/2129019470 shutdown_connections 2024-03-21T11:48:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.011+0000 7f3a38a04740 1 -- 172.21.15.47:0/2129019470 wait complete. 2024-03-21T11:48:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.011+0000 7f3a38a04740 20 client.22491 trim_cache size 0 max 16384 2024-03-21T11:48:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:48:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:48:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:48:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:48:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.011+0000 7f3a2fe00640 1 -- 172.21.15.47:0/2624963556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a140a3050 msgr2=0x7f3a140a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.011+0000 7f3a2fe00640 1 --2- 172.21.15.47:0/2624963556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a140a3050 0x7f3a140a5510 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009530 tx=0x7f3a200056d0 comp rx=0 tx=0).stop 2024-03-21T11:48:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.011+0000 7f3a2fe00640 1 -- 172.21.15.47:0/2624963556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a30109280 msgr2=0x7f3a3019e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.011+0000 7f3a2fe00640 1 --2- 172.21.15.47:0/2624963556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a30109280 0x7f3a3019e370 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009230 tx=0x7f3a1c004e30 comp rx=0 tx=0).stop 2024-03-21T11:48:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.012+0000 7f3a2fe00640 1 -- 172.21.15.47:0/2624963556 shutdown_connections 2024-03-21T11:48:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.012+0000 7f3a2fe00640 1 --2- 172.21.15.47:0/2624963556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a140a3050 0x7f3a140a5510 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.012+0000 7f3a2fe00640 1 --2- 172.21.15.47:0/2624963556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a3010f9e0 0x7f3a301a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.012+0000 7f3a2fe00640 1 --2- 172.21.15.47:0/2624963556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a30109280 0x7f3a3019e370 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.012+0000 7f3a2fe00640 1 --2- 172.21.15.47:0/2624963556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a30072be0 0x7f3a3019de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.012+0000 7f3a2fe00640 1 -- 172.21.15.47:0/2624963556 >> 172.21.15.47:0/2624963556 conn(0x7f3a3006e510 msgr2=0x7f3a3010b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.012+0000 7f3a2fe00640 1 -- 172.21.15.47:0/2624963556 shutdown_connections 2024-03-21T11:48:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:28.014+0000 7f3a2fe00640 1 -- 172.21.15.47:0/2624963556 wait complete. 2024-03-21T11:48:29.001 INFO:tasks.workunit.client.0.smithi047.stdout: 10 277959 21.43 MB/sec execute 552 sec latency 132.890 ms 2024-03-21T11:48:29.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:28 smithi102 ceph-mon[36748]: pgmap v1155: 129 pgs: 129 active+clean; 778 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.1 MiB/s wr, 517 op/s 2024-03-21T11:48:29.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:28 smithi047 ceph-mon[31351]: pgmap v1155: 129 pgs: 129 active+clean; 778 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.1 MiB/s wr, 517 op/s 2024-03-21T11:48:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:28 smithi100 ceph-mon[36245]: pgmap v1155: 129 pgs: 129 active+clean; 778 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.1 MiB/s wr, 517 op/s 2024-03-21T11:48:30.001 INFO:tasks.workunit.client.0.smithi047.stdout: 10 278172 21.42 MB/sec execute 553 sec latency 225.452 ms 2024-03-21T11:48:30.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:48:30.299 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:48:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.592+0000 7f48c6400640 1 -- 172.21.15.47:0/1017698364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c0105250 msgr2=0x7f48c01056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.592+0000 7f48c6400640 1 --2- 172.21.15.47:0/1017698364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c0105250 0x7f48c01056d0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009ca0 tx=0x7f48b802c0f0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.592+0000 7f48c6400640 1 -- 172.21.15.47:0/1017698364 shutdown_connections 2024-03-21T11:48:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.592+0000 7f48c6400640 1 --2- 172.21.15.47:0/1017698364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c0105250 0x7f48c01056d0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.592+0000 7f48c6400640 1 --2- 172.21.15.47:0/1017698364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c01028a0 0x7f48c0104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.592+0000 7f48c6400640 1 --2- 172.21.15.47:0/1017698364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48c00fff30 0x7f48c0102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.592+0000 7f48c6400640 1 -- 172.21.15.47:0/1017698364 >> 172.21.15.47:0/1017698364 conn(0x7f48c00fbb50 msgr2=0x7f48c00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.593+0000 7f48c6400640 1 -- 172.21.15.47:0/1017698364 shutdown_connections 2024-03-21T11:48:30.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.593+0000 7f48c6400640 1 -- 172.21.15.47:0/1017698364 wait complete. 2024-03-21T11:48:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.595+0000 7f48c6400640 1 Processor -- start 2024-03-21T11:48:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.595+0000 7f48c6400640 1 -- start start 2024-03-21T11:48:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.596+0000 7f48c6400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c00fff30 0x7f48c019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.596+0000 7f48c6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c01028a0 0x7f48c019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.596+0000 7f48c6400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48c0105250 0x7f48c01a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.596+0000 7f48c6400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0113910 con 0x7f48c00fff30 2024-03-21T11:48:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.596+0000 7f48c6400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0113790 con 0x7f48c01028a0 2024-03-21T11:48:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.596+0000 7f48c6400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0113a90 con 0x7f48c0105250 2024-03-21T11:48:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.596+0000 7f48bea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c01028a0 0x7f48c019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.596+0000 7f48bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48c0105250 0x7f48c01a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.596+0000 7f48bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48c0105250 0x7f48c01a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35704/0 (socket says 172.21.15.47:35704) 2024-03-21T11:48:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.596+0000 7f48bfe00640 1 -- 172.21.15.47:0/1490238283 learned_addr learned my addr 172.21.15.47:0/1490238283 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.596+0000 7f48bea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c01028a0 0x7f48c019e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60444/0 (socket says 172.21.15.47:60444) 2024-03-21T11:48:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.596+0000 7f48bf400640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c00fff30 0x7f48c019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.597+0000 7f48bfe00640 1 -- 172.21.15.47:0/1490238283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c01028a0 msgr2=0x7f48c019e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:30.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.597+0000 7f48bfe00640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c01028a0 0x7f48c019e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.597+0000 7f48bfe00640 1 -- 172.21.15.47:0/1490238283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c00fff30 msgr2=0x7f48c019df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:30.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.597+0000 7f48bfe00640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c00fff30 0x7f48c019df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.597+0000 7f48bfe00640 1 -- 172.21.15.47:0/1490238283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c01a5b70 con 0x7f48c0105250 2024-03-21T11:48:30.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.597+0000 7f48bea00640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c01028a0 0x7f48c019e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:48:30.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.598+0000 7f48bf400640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c00fff30 0x7f48c019df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:48:30.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.599+0000 7f48bfe00640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48c0105250 0x7f48c01a5500 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f48b8005e30 tx=0x7f48b8032ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.601+0000 7f48b7e00640 1 -- 172.21.15.47:0/1490238283 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8041020 con 0x7f48c0105250 2024-03-21T11:48:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.602+0000 7f48b7e00640 1 -- 172.21.15.47:0/1490238283 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48b8034cd0 con 0x7f48c0105250 2024-03-21T11:48:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.602+0000 7f48c6400640 1 -- 172.21.15.47:0/1490238283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c01a5e00 con 0x7f48c0105250 2024-03-21T11:48:30.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.602+0000 7f48b7e00640 1 -- 172.21.15.47:0/1490238283 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8039040 con 0x7f48c0105250 2024-03-21T11:48:30.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.602+0000 7f48c6400640 1 -- 172.21.15.47:0/1490238283 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c01a62c0 con 0x7f48c0105250 2024-03-21T11:48:30.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.610+0000 7f48b7e00640 1 -- 172.21.15.47:0/1490238283 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f48b8044020 con 0x7f48c0105250 2024-03-21T11:48:30.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.610+0000 7f48c9189740 1 Processor -- start 2024-03-21T11:48:30.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.612+0000 7f48c9189740 1 -- start start 2024-03-21T11:48:30.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.612+0000 7f48c9189740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735c9b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.612+0000 7f48c9189740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735ca0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.612+0000 7f48c9189740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735ca09e0 0x55c735ca2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.612+0000 7f48c9189740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c735c00250 con 0x55c735ca09e0 2024-03-21T11:48:30.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.612+0000 7f48c9189740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c735ba8a30 con 0x55c735c9b870 2024-03-21T11:48:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.612+0000 7f48c9189740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c735bfc2f0 con 0x55c735c9ae80 2024-03-21T11:48:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.612+0000 7f48bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735ca0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.613+0000 7f48bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735ca0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60452/0 (socket says 172.21.15.47:60452) 2024-03-21T11:48:30.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.613+0000 7f48bf400640 1 -- 172.21.15.47:0/1542074269 learned_addr learned my addr 172.21.15.47:0/1542074269 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.613+0000 7f48bea00640 1 --2- 172.21.15.47:0/1542074269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735c9b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.614+0000 7f48b7e00640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48a40a2fa0 0x7f48a40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.615+0000 7f48bf400640 1 -- 172.21.15.47:0/1542074269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 msgr2=0x55c735c9b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.615+0000 7f48bf400640 1 --2- 172.21.15.47:0/1542074269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735c9b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.615+0000 7f48bf400640 1 -- 172.21.15.47:0/1542074269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735ca09e0 msgr2=0x55c735ca2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.615+0000 7f48bf400640 1 --2- 172.21.15.47:0/1542074269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735ca09e0 0x55c735ca2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.615+0000 7f48bf400640 1 -- 172.21.15.47:0/1542074269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c735b93690 con 0x55c735c9b870 2024-03-21T11:48:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.615+0000 7f48bea00640 1 --2- 172.21.15.47:0/1542074269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735c9b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:48:30.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.615+0000 7f48b7e00640 1 -- 172.21.15.47:0/1490238283 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f48b80f17c0 con 0x7f48c0105250 2024-03-21T11:48:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.615+0000 7f48bfe00640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48a40a2fa0 0x7f48a40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.615+0000 7f48bf400640 1 --2- 172.21.15.47:0/1542074269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735ca0370 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f48b000bd40 tx=0x7f48b000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f30a2fa87770635 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.616+0000 7f48b4c00640 1 -- 172.21.15.47:0/1542074269 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0004220 con 0x55c735c9b870 2024-03-21T11:48:30.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.616+0000 7f48b4c00640 1 -- 172.21.15.47:0/1542074269 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48b000be30 con 0x55c735c9b870 2024-03-21T11:48:30.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.616+0000 7f48b4c00640 1 -- 172.21.15.47:0/1542074269 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0011730 con 0x55c735c9b870 2024-03-21T11:48:30.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.616+0000 7f48bfe00640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48a40a2fa0 0x7f48a40a5460 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f48b80be9d0 tx=0x7f48b802c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:30.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.618+0000 7f48c9189740 1 -- 172.21.15.47:0/1542074269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 msgr2=0x55c735ca0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:30.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.618+0000 7f48c9189740 1 --2- 172.21.15.47:0/1542074269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735ca0370 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f48b000bd40 tx=0x7f48b000f730 comp rx=0 tx=0).stop 2024-03-21T11:48:30.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.619+0000 7f48c9189740 1 -- 172.21.15.47:0/1542074269 shutdown_connections 2024-03-21T11:48:30.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.619+0000 7f48c9189740 1 --2- 172.21.15.47:0/1542074269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735ca09e0 0x55c735ca2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.619+0000 7f48c9189740 1 --2- 172.21.15.47:0/1542074269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735ca0370 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.619+0000 7f48c9189740 1 --2- 172.21.15.47:0/1542074269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735c9b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.619+0000 7f48c9189740 1 -- 172.21.15.47:0/1542074269 >> 172.21.15.47:0/1542074269 conn(0x55c735c63200 msgr2=0x55c735c967f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.619+0000 7f48c9189740 1 -- 172.21.15.47:0/1542074269 shutdown_connections 2024-03-21T11:48:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.619+0000 7f48c9189740 1 -- 172.21.15.47:0/1542074269 wait complete. 2024-03-21T11:48:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.620+0000 7f48c9189740 1 -- start start 2024-03-21T11:48:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.620+0000 7f48c9189740 1 Processor -- start 2024-03-21T11:48:30.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.621+0000 7f48c9189740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9ae80 0x55c735cb5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.621+0000 7f48c9189740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735c9b870 0x55c735cb6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.621+0000 7f48c9189740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735ca09e0 0x55c735cbd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.621+0000 7f48c9189740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c735cb49a0 con 0x55c735c9b870 2024-03-21T11:48:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.621+0000 7f48c9189740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c735bbbe80 con 0x55c735c9ae80 2024-03-21T11:48:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.621+0000 7f48c9189740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c735cb4b20 con 0x55c735ca09e0 2024-03-21T11:48:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.621+0000 7f48c9189740 10 client.0 _ll_register_callbacks cb 0x55c735bce130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.622+0000 7f48bea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9ae80 0x55c735cb5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.622+0000 7f48bea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9ae80 0x55c735cb5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60460/0 (socket says 172.21.15.47:60460) 2024-03-21T11:48:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.622+0000 7f48bea00640 1 -- 172.21.15.47:0/3312140286 learned_addr learned my addr 172.21.15.47:0/3312140286 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.622+0000 7f48bea00640 1 --2- 172.21.15.47:0/3312140286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735ca09e0 0x55c735cbd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.623+0000 7f48bea00640 1 -- 172.21.15.47:0/3312140286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9ae80 msgr2=0x55c735cb5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:30.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.623+0000 7f48bea00640 1 --2- 172.21.15.47:0/3312140286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9ae80 0x55c735cb5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.623+0000 7f48bea00640 1 -- 172.21.15.47:0/3312140286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735c9b870 msgr2=0x55c735cb6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:30.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.623+0000 7f48bea00640 1 --2- 172.21.15.47:0/3312140286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735c9b870 0x55c735cb6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.623+0000 7f48bea00640 1 -- 172.21.15.47:0/3312140286 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7359de7d0 con 0x55c735ca09e0 2024-03-21T11:48:30.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.624+0000 7f48bea00640 1 --2- 172.21.15.47:0/3312140286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735ca09e0 0x55c735cbd450 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f48ac012fc0 tx=0x7f48ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c999613043150727 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:30.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.624+0000 7f48a9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:48:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.626+0000 7f48a9600640 1 -- 172.21.15.47:0/3312140286 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac004030 con 0x55c735ca09e0 2024-03-21T11:48:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.626+0000 7f48a9600640 1 -- 172.21.15.47:0/3312140286 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f48ac0041d0 con 0x55c735ca09e0 2024-03-21T11:48:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.626+0000 7f48c9189740 1 -- 172.21.15.47:0/3312140286 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48b000be30 con 0x55c735ca09e0 2024-03-21T11:48:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.627+0000 7f48a9600640 1 -- 172.21.15.47:0/3312140286 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48ac0043d0 con 0x55c735ca09e0 2024-03-21T11:48:30.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.627+0000 7f48c9189740 10 client.33349 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:30.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.627+0000 7f48c9189740 1 -- 172.21.15.47:0/3312140286 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c7359143f0 con 0x55c735ca09e0 2024-03-21T11:48:30.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.628+0000 7f48a9600640 1 -- 172.21.15.47:0/3312140286 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48b000be30 con 0x55c735ca09e0 2024-03-21T11:48:30.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.628+0000 7f48a9600640 1 -- 172.21.15.47:0/3312140286 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f48ac027020 con 0x55c735ca09e0 2024-03-21T11:48:30.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.631+0000 7f48c9189740 10 client.33349 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:30.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.631+0000 7f48c9189740 10 client.33349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:30.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.631+0000 7f48c9189740 20 client.33349 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:30.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.631+0000 7f48c9189740 1 --2- 172.21.15.47:0/3312140286 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c735cc1c60 0x55c735cc40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.631+0000 7f48c9189740 4 client.33349 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:48:30.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.631+0000 7f48c9189740 1 -- 172.21.15.47:0/3312140286 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c735c0d790 con 0x55c735cc1c60 2024-03-21T11:48:30.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.632+0000 7f48bf400640 1 --2- 172.21.15.47:0/3312140286 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c735cc1c60 0x55c735cc40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.633+0000 7f48bf400640 1 --2- 172.21.15.47:0/3312140286 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c735cc1c60 0x55c735cc40e0 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=649b2c5efc911fe7 server_cookie=4a8fcdfe23fbd880 in_seq=0 out_seq=0 2024-03-21T11:48:30.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.633+0000 7f48a9600640 10 client.33349 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:30.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.634+0000 7f48a9600640 1 -- 172.21.15.47:0/3312140286 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c735c0d790 con 0x55c735cc1c60 2024-03-21T11:48:30.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.634+0000 7f48a9600640 10 client.33349 handle_command_reply: tid=0 2024-03-21T11:48:30.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.634+0000 7f48c9189740 1 client.33349 shutdown 2024-03-21T11:48:30.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.635+0000 7f48c9189740 1 -- 172.21.15.47:0/3312140286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735ca09e0 msgr2=0x55c735cbd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:30.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.635+0000 7f48c9189740 1 --2- 172.21.15.47:0/3312140286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735ca09e0 0x55c735cbd450 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f48ac012fc0 tx=0x7f48ac0046c0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.636+0000 7f48c9189740 1 -- 172.21.15.47:0/3312140286 shutdown_connections 2024-03-21T11:48:30.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.636+0000 7f48c9189740 1 --2- 172.21.15.47:0/3312140286 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c735cc1c60 0x55c735cc40e0 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.636+0000 7f48c9189740 1 --2- 172.21.15.47:0/3312140286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735ca09e0 0x55c735cbd450 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.636+0000 7f48c9189740 1 --2- 172.21.15.47:0/3312140286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735c9b870 0x55c735cb6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.636+0000 7f48c9189740 1 --2- 172.21.15.47:0/3312140286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9ae80 0x55c735cb5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.636+0000 7f48c9189740 1 -- 172.21.15.47:0/3312140286 >> 172.21.15.47:0/3312140286 conn(0x55c735c63200 msgr2=0x55c735c95a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:30.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.636+0000 7f48a9600640 0 client.33349 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:30.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.636+0000 7f48c9189740 1 -- 172.21.15.47:0/3312140286 shutdown_connections 2024-03-21T11:48:30.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.636+0000 7f48c9189740 1 -- 172.21.15.47:0/3312140286 wait complete. 2024-03-21T11:48:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.637+0000 7f48c9189740 20 client.33349 trim_cache size 0 max 16384 2024-03-21T11:48:30.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.669+0000 7f48c9189740 1 Processor -- start 2024-03-21T11:48:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.670+0000 7f48c9189740 1 -- start start 2024-03-21T11:48:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.670+0000 7f48c9189740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735c9fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.670+0000 7f48c9189740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735cbd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.670+0000 7f48c9189740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735ca09e0 0x55c735cc89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.670+0000 7f48c9189740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c01a7f90 con 0x55c735ca09e0 2024-03-21T11:48:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.670+0000 7f48c9189740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c01a9910 con 0x55c735c9b870 2024-03-21T11:48:30.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.670+0000 7f48c9189740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c01a85f0 con 0x55c735c9ae80 2024-03-21T11:48:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.670+0000 7f48bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735cbd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.670+0000 7f48bea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735c9fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.670+0000 7f48bea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735c9fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35718/0 (socket says 172.21.15.47:35718) 2024-03-21T11:48:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.670+0000 7f48bea00640 1 -- 172.21.15.47:0/1785920772 learned_addr learned my addr 172.21.15.47:0/1785920772 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.671+0000 7f48bf400640 1 -- 172.21.15.47:0/1785920772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 msgr2=0x55c735c9fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.671+0000 7f48bf400640 1 --2- 172.21.15.47:0/1785920772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735c9fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.671+0000 7f48bf400640 1 -- 172.21.15.47:0/1785920772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735ca09e0 msgr2=0x55c735cc89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:30.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.671+0000 7f48bf400640 1 --2- 172.21.15.47:0/1785920772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735ca09e0 0x55c735cc89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.671+0000 7f48bf400640 1 -- 172.21.15.47:0/1785920772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7358d40f0 con 0x55c735c9b870 2024-03-21T11:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.672+0000 7f48bf400640 1 --2- 172.21.15.47:0/1785920772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735cbd410 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x55c735c97740 tx=0x7f48b0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87560818a50536f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.672+0000 7f48abe00640 1 -- 172.21.15.47:0/1785920772 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b002e070 con 0x55c735c9b870 2024-03-21T11:48:30.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.673+0000 7f48abe00640 1 -- 172.21.15.47:0/1785920772 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48b00280d0 con 0x55c735c9b870 2024-03-21T11:48:30.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.673+0000 7f48abe00640 1 -- 172.21.15.47:0/1785920772 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b00055e0 con 0x55c735c9b870 2024-03-21T11:48:30.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.674+0000 7f48c9189740 1 -- 172.21.15.47:0/1785920772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 msgr2=0x55c735cbd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.674+0000 7f48c9189740 1 --2- 172.21.15.47:0/1785920772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735cbd410 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x55c735c97740 tx=0x7f48b0000bc0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.675+0000 7f48c9189740 1 -- 172.21.15.47:0/1785920772 shutdown_connections 2024-03-21T11:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.675+0000 7f48c9189740 1 --2- 172.21.15.47:0/1785920772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735ca09e0 0x55c735cc89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.675+0000 7f48c9189740 1 --2- 172.21.15.47:0/1785920772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735cbd410 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.675+0000 7f48c9189740 1 --2- 172.21.15.47:0/1785920772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735c9fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.675+0000 7f48c9189740 1 -- 172.21.15.47:0/1785920772 >> 172.21.15.47:0/1785920772 conn(0x55c735c63200 msgr2=0x55c735c9cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:30.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.675+0000 7f48c9189740 1 -- 172.21.15.47:0/1785920772 shutdown_connections 2024-03-21T11:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.675+0000 7f48c9189740 1 -- 172.21.15.47:0/1785920772 wait complete. 2024-03-21T11:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.676+0000 7f48c9189740 1 -- start start 2024-03-21T11:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.676+0000 7f48c9189740 1 Processor -- start 2024-03-21T11:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.677+0000 7f48c9189740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735cbad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.677+0000 7f48c9189740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735cbb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.677+0000 7f48c9189740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735ca09e0 0x55c735cbc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.677+0000 7f48c9189740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c01a92b0 con 0x55c735ca09e0 2024-03-21T11:48:30.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.677+0000 7f48c9189740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c01a8c50 con 0x55c735c9b870 2024-03-21T11:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.677+0000 7f48c9189740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0073530 con 0x55c735c9ae80 2024-03-21T11:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.677+0000 7f48c9189740 10 client.0 _ll_register_callbacks cb 0x7f48ac00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.678+0000 7f48bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735cbb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.678+0000 7f48bea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735cbad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.678+0000 7f48bea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735cbad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35722/0 (socket says 172.21.15.47:35722) 2024-03-21T11:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.678+0000 7f48bea00640 1 -- 172.21.15.47:0/113011809 learned_addr learned my addr 172.21.15.47:0/113011809 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:30.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.678+0000 7f48bea00640 1 --2- 172.21.15.47:0/113011809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735ca09e0 0x55c735cbc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.679+0000 7f48bea00640 1 -- 172.21.15.47:0/113011809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 msgr2=0x55c735cbb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:30.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.679+0000 7f48bea00640 1 --2- 172.21.15.47:0/113011809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735cbb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.679+0000 7f48bea00640 1 -- 172.21.15.47:0/113011809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735ca09e0 msgr2=0x55c735cbc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:30.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.679+0000 7f48bea00640 1 --2- 172.21.15.47:0/113011809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735ca09e0 0x55c735cbc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.679+0000 7f48bea00640 1 -- 172.21.15.47:0/113011809 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c735cbc840 con 0x55c735c9ae80 2024-03-21T11:48:30.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.679+0000 7f48bea00640 1 --2- 172.21.15.47:0/113011809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735cbad00 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f48ac001eb0 tx=0x7f48ac00c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5de1905b904157a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:30.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.679+0000 7f48a9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:48:30.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.680+0000 7f48a9600640 1 -- 172.21.15.47:0/113011809 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac00fbc0 con 0x55c735c9ae80 2024-03-21T11:48:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.680+0000 7f48a9600640 1 -- 172.21.15.47:0/113011809 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f48ac00fd60 con 0x55c735c9ae80 2024-03-21T11:48:30.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.682+0000 7f48c9189740 1 -- 172.21.15.47:0/113011809 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48b00280d0 con 0x55c735c9ae80 2024-03-21T11:48:30.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.685+0000 7f48a9600640 1 -- 172.21.15.47:0/113011809 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48ac005030 con 0x55c735c9ae80 2024-03-21T11:48:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.686+0000 7f48c9189740 10 client.33355 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:30.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.686+0000 7f48c9189740 1 -- 172.21.15.47:0/113011809 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c735d1d760 con 0x55c735c9ae80 2024-03-21T11:48:30.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.689+0000 7f48a9600640 1 -- 172.21.15.47:0/113011809 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48b00280d0 con 0x55c735c9ae80 2024-03-21T11:48:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.689+0000 7f48a9600640 1 -- 172.21.15.47:0/113011809 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f48ac027070 con 0x55c735c9ae80 2024-03-21T11:48:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.689+0000 7f48c9189740 10 client.33355 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.689+0000 7f48c9189740 10 client.33355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.689+0000 7f48c9189740 20 client.33355 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.689+0000 7f48c9189740 1 --2- 172.21.15.47:0/113011809 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c735d1e090 0x55c735d204b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.689+0000 7f48c9189740 4 client.33355 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:48:30.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.689+0000 7f48c9189740 1 -- 172.21.15.47:0/113011809 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c735c07ae0 con 0x55c735d1e090 2024-03-21T11:48:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.690+0000 7f48bf400640 1 --2- 172.21.15.47:0/113011809 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c735d1e090 0x55c735d204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.690+0000 7f48bf400640 1 --2- 172.21.15.47:0/113011809 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c735d1e090 0x55c735d204b0 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1290ffb610c4614e server_cookie=ca476c7d052fc6f in_seq=0 out_seq=0 2024-03-21T11:48:30.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.690+0000 7f48a9600640 10 client.33355 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.691+0000 7f48a9600640 1 -- 172.21.15.47:0/113011809 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c735c07ae0 con 0x55c735d1e090 2024-03-21T11:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.691+0000 7f48a9600640 10 client.33355 handle_command_reply: tid=0 2024-03-21T11:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.691+0000 7f48c9189740 1 client.33355 shutdown 2024-03-21T11:48:30.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.692+0000 7f48c9189740 1 -- 172.21.15.47:0/113011809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 msgr2=0x55c735cbad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:30.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.692+0000 7f48c9189740 1 --2- 172.21.15.47:0/113011809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735cbad00 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f48ac001eb0 tx=0x7f48ac00c460 comp rx=0 tx=0).stop 2024-03-21T11:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.693+0000 7f48c9189740 1 -- 172.21.15.47:0/113011809 shutdown_connections 2024-03-21T11:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.693+0000 7f48c9189740 1 --2- 172.21.15.47:0/113011809 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c735d1e090 0x55c735d204b0 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.693+0000 7f48c9189740 1 --2- 172.21.15.47:0/113011809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c735ca09e0 0x55c735cbc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.693+0000 7f48c9189740 1 --2- 172.21.15.47:0/113011809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c735c9b870 0x55c735cbb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.693+0000 7f48c9189740 1 --2- 172.21.15.47:0/113011809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c735c9ae80 0x55c735cbad00 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.693+0000 7f48c9189740 1 -- 172.21.15.47:0/113011809 >> 172.21.15.47:0/113011809 conn(0x55c735c63200 msgr2=0x55c735c99a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.693+0000 7f48a9600640 0 client.33355 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.693+0000 7f48c9189740 1 -- 172.21.15.47:0/113011809 shutdown_connections 2024-03-21T11:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.693+0000 7f48c9189740 1 -- 172.21.15.47:0/113011809 wait complete. 2024-03-21T11:48:30.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.693+0000 7f48c9189740 20 client.33355 trim_cache size 0 max 16384 2024-03-21T11:48:30.694 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:48:30.694 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:48:30.694 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:48:30.694 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.694+0000 7f48c6400640 1 -- 172.21.15.47:0/1490238283 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48a40a2fa0 msgr2=0x7f48a40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.694+0000 7f48c6400640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48a40a2fa0 0x7f48a40a5460 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f48b80be9d0 tx=0x7f48b802c0f0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.694+0000 7f48c6400640 1 -- 172.21.15.47:0/1490238283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48c0105250 msgr2=0x7f48c01a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.694+0000 7f48c6400640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48c0105250 0x7f48c01a5500 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f48b8005e30 tx=0x7f48b8032ec0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.694+0000 7f48c6400640 1 -- 172.21.15.47:0/1490238283 shutdown_connections 2024-03-21T11:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.694+0000 7f48c6400640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f48a40a2fa0 0x7f48a40a5460 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.694+0000 7f48c6400640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48c0105250 0x7f48c01a5500 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.694+0000 7f48c6400640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c01028a0 0x7f48c019e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.694+0000 7f48c6400640 1 --2- 172.21.15.47:0/1490238283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c00fff30 0x7f48c019df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.694+0000 7f48c6400640 1 -- 172.21.15.47:0/1490238283 >> 172.21.15.47:0/1490238283 conn(0x7f48c00fbb50 msgr2=0x7f48c0100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:30.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.695+0000 7f48c6400640 1 -- 172.21.15.47:0/1490238283 shutdown_connections 2024-03-21T11:48:30.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.695+0000 7f48c6400640 1 -- 172.21.15.47:0/1490238283 wait complete. 2024-03-21T11:48:30.718 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:48:30.718 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:48:30.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.979+0000 7f7ee3e00640 1 -- 172.21.15.47:0/3133396106 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8032340 con 0x7f7ee4104ad0 2024-03-21T11:48:30.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.979+0000 7f7eec000640 1 -- 172.21.15.47:0/3133396106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4104ad0 msgr2=0x7f7ee4104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:30.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.979+0000 7f7eec000640 1 --2- 172.21.15.47:0/3133396106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4104ad0 0x7f7ee4104f50 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009860 tx=0x7f7ed802bb70 comp rx=0 tx=0).stop 2024-03-21T11:48:30.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.980+0000 7f7eec000640 1 -- 172.21.15.47:0/3133396106 shutdown_connections 2024-03-21T11:48:30.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.980+0000 7f7eec000640 1 --2- 172.21.15.47:0/3133396106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ee4105490 0x7f7ee410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.980+0000 7f7eec000640 1 --2- 172.21.15.47:0/3133396106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4104ad0 0x7f7ee4104f50 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.980+0000 7f7eec000640 1 --2- 172.21.15.47:0/3133396106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ee41038e0 0x7f7ee4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.980+0000 7f7eec000640 1 -- 172.21.15.47:0/3133396106 >> 172.21.15.47:0/3133396106 conn(0x7f7ee40fee90 msgr2=0x7f7ee41012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:30.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.983+0000 7f7eec000640 1 -- 172.21.15.47:0/3133396106 shutdown_connections 2024-03-21T11:48:30.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.983+0000 7f7eec000640 1 -- 172.21.15.47:0/3133396106 wait complete. 2024-03-21T11:48:30.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.985+0000 7f7eec000640 1 Processor -- start 2024-03-21T11:48:30.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.985+0000 7f7eec000640 1 -- start start 2024-03-21T11:48:30.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.986+0000 7f7eec000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ee41038e0 0x7f7ee4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.986+0000 7f7eec000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4104ad0 0x7f7ee4071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.986+0000 7f7eec000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ee4105490 0x7f7ee40722c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.986+0000 7f7eec000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee406b450 con 0x7f7ee4105490 2024-03-21T11:48:30.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.986+0000 7f7eec000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee406b2d0 con 0x7f7ee4104ad0 2024-03-21T11:48:30.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.986+0000 7f7eec000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee406b5d0 con 0x7f7ee41038e0 2024-03-21T11:48:30.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.986+0000 7f7ee8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4104ad0 0x7f7ee4071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.986+0000 7f7ee8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4104ad0 0x7f7ee4071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60502/0 (socket says 172.21.15.47:60502) 2024-03-21T11:48:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.986+0000 7f7ee8c00640 1 -- 172.21.15.47:0/2039154338 learned_addr learned my addr 172.21.15.47:0/2039154338 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.986+0000 7f7eea000640 1 --2- 172.21.15.47:0/2039154338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ee4105490 0x7f7ee40722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.987+0000 7f7ee8c00640 1 -- 172.21.15.47:0/2039154338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ee41038e0 msgr2=0x7f7ee4070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.987+0000 7f7ee8c00640 1 --2- 172.21.15.47:0/2039154338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ee41038e0 0x7f7ee4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.987+0000 7f7ee8c00640 1 -- 172.21.15.47:0/2039154338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ee4105490 msgr2=0x7f7ee40722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.987+0000 7f7ee8c00640 1 --2- 172.21.15.47:0/2039154338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ee4105490 0x7f7ee40722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:30.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.987+0000 7f7ee8c00640 1 -- 172.21.15.47:0/2039154338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee4079c30 con 0x7f7ee4104ad0 2024-03-21T11:48:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.987+0000 7f7ee8c00640 1 --2- 172.21.15.47:0/2039154338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4104ad0 0x7f7ee4071280 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80349e0 tx=0x7f7ed80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.988+0000 7f7ee2000640 1 -- 172.21.15.47:0/2039154338 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8041030 con 0x7f7ee4104ad0 2024-03-21T11:48:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.988+0000 7f7ee2000640 1 -- 172.21.15.47:0/2039154338 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ed80347b0 con 0x7f7ee4104ad0 2024-03-21T11:48:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.988+0000 7f7ee2000640 1 -- 172.21.15.47:0/2039154338 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8039040 con 0x7f7ee4104ad0 2024-03-21T11:48:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.988+0000 7f7eec000640 1 -- 172.21.15.47:0/2039154338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee4072800 con 0x7f7ee4104ad0 2024-03-21T11:48:30.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.988+0000 7f7eec000640 1 -- 172.21.15.47:0/2039154338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee4072c70 con 0x7f7ee4104ad0 2024-03-21T11:48:30.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.990+0000 7f7eec000640 1 -- 172.21.15.47:0/2039154338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec40052c0 con 0x7f7ee4104ad0 2024-03-21T11:48:30.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.993+0000 7f7ee2000640 1 -- 172.21.15.47:0/2039154338 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7ed8044070 con 0x7f7ee4104ad0 2024-03-21T11:48:30.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.996+0000 7f7ee2000640 1 --2- 172.21.15.47:0/2039154338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ecc0a2fa0 0x7f7ecc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:30.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.997+0000 7f7ee2000640 1 -- 172.21.15.47:0/2039154338 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7ed80f1500 con 0x7f7ee4104ad0 2024-03-21T11:48:30.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.997+0000 7f7ee2000640 1 -- 172.21.15.47:0/2039154338 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7ed80be020 con 0x7f7ee4104ad0 2024-03-21T11:48:30.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.997+0000 7f7ee9600640 1 --2- 172.21.15.47:0/2039154338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ecc0a2fa0 0x7f7ecc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:30.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:30.998+0000 7f7ee9600640 1 --2- 172.21.15.47:0/2039154338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ecc0a2fa0 0x7f7ecc0a5460 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4104930 tx=0x7f7ed0009660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:31.001 INFO:tasks.workunit.client.0.smithi047.stdout: 10 278411 21.41 MB/sec execute 554 sec latency 139.638 ms 2024-03-21T11:48:31.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:30 smithi047 ceph-mon[31351]: pgmap v1156: 129 pgs: 129 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 567 op/s 2024-03-21T11:48:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:30 smithi102 ceph-mon[36748]: pgmap v1156: 129 pgs: 129 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 567 op/s 2024-03-21T11:48:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:30 smithi100 ceph-mon[36245]: pgmap v1156: 129 pgs: 129 active+clean; 807 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 567 op/s 2024-03-21T11:48:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.401+0000 7f7eec000640 1 -- 172.21.15.47:0/2039154338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ec40055b0 con 0x7f7ee4104ad0 2024-03-21T11:48:31.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.402+0000 7f7ee2000640 1 -- 172.21.15.47:0/2039154338 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f7ed80bd920 con 0x7f7ee4104ad0 2024-03-21T11:48:31.402 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:48:31.403 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:45:39.125075+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_32623":{"gid":32623,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4278877119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4278877119},{"type":"v1","addr":"172.21.15.100:6835","nonce":4278877119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24523,24524,24535,24553,24581]},"id":1}]} 2024-03-21T11:48:31.403 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 41 2024-03-21T11:48:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.408+0000 7f7eec000640 1 -- 172.21.15.47:0/2039154338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ecc0a2fa0 msgr2=0x7f7ecc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.408+0000 7f7eec000640 1 --2- 172.21.15.47:0/2039154338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ecc0a2fa0 0x7f7ecc0a5460 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4104930 tx=0x7f7ed0009660 comp rx=0 tx=0).stop 2024-03-21T11:48:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.408+0000 7f7eec000640 1 -- 172.21.15.47:0/2039154338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4104ad0 msgr2=0x7f7ee4071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.408+0000 7f7eec000640 1 --2- 172.21.15.47:0/2039154338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4104ad0 0x7f7ee4071280 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80349e0 tx=0x7f7ed80042f0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.408+0000 7f7eec000640 1 -- 172.21.15.47:0/2039154338 shutdown_connections 2024-03-21T11:48:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.408+0000 7f7eec000640 1 --2- 172.21.15.47:0/2039154338 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7ecc0a2fa0 0x7f7ecc0a5460 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.408+0000 7f7eec000640 1 --2- 172.21.15.47:0/2039154338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ee4105490 0x7f7ee40722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.408+0000 7f7eec000640 1 --2- 172.21.15.47:0/2039154338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4104ad0 0x7f7ee4071280 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.408+0000 7f7eec000640 1 --2- 172.21.15.47:0/2039154338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ee41038e0 0x7f7ee4070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.408+0000 7f7eec000640 1 -- 172.21.15.47:0/2039154338 >> 172.21.15.47:0/2039154338 conn(0x7f7ee40fee90 msgr2=0x7f7ee4107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.408+0000 7f7eec000640 1 -- 172.21.15.47:0/2039154338 shutdown_connections 2024-03-21T11:48:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.408+0000 7f7eec000640 1 -- 172.21.15.47:0/2039154338 wait complete. 2024-03-21T11:48:31.438 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:48:31.439 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:48:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.716+0000 7f93eda00640 1 -- 172.21.15.47:0/3122572963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e8102140 msgr2=0x7f93e8104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.716+0000 7f93eda00640 1 --2- 172.21.15.47:0/3122572963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e8102140 0x7f93e8104530 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009900 tx=0x7f93d802bc40 comp rx=0 tx=0).stop 2024-03-21T11:48:31.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.716+0000 7f93eda00640 1 -- 172.21.15.47:0/3122572963 shutdown_connections 2024-03-21T11:48:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.716+0000 7f93eda00640 1 --2- 172.21.15.47:0/3122572963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e81073a0 0x7f93e81097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.716+0000 7f93eda00640 1 --2- 172.21.15.47:0/3122572963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e8104a70 0x7f93e8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.716+0000 7f93eda00640 1 --2- 172.21.15.47:0/3122572963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e8102140 0x7f93e8104530 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.716+0000 7f93eda00640 1 -- 172.21.15.47:0/3122572963 >> 172.21.15.47:0/3122572963 conn(0x7f93e80fbb30 msgr2=0x7f93e80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.716+0000 7f93eda00640 1 -- 172.21.15.47:0/3122572963 shutdown_connections 2024-03-21T11:48:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.716+0000 7f93eda00640 1 -- 172.21.15.47:0/3122572963 wait complete. 2024-03-21T11:48:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.717+0000 7f93eda00640 1 Processor -- start 2024-03-21T11:48:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.718+0000 7f93eda00640 1 -- start start 2024-03-21T11:48:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.718+0000 7f93eda00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e8102140 0x7f93e8195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.719+0000 7f93eda00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e8104a70 0x7f93e8195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.719+0000 7f93eda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e81073a0 0x7f93e81a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.719+0000 7f93eda00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e810d390 con 0x7f93e8104a70 2024-03-21T11:48:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.719+0000 7f93eda00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e810d210 con 0x7f93e81073a0 2024-03-21T11:48:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.719+0000 7f93eda00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e810d510 con 0x7f93e8102140 2024-03-21T11:48:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.719+0000 7f93e6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e8102140 0x7f93e8195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.719+0000 7f93e6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e8102140 0x7f93e8195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35754/0 (socket says 172.21.15.47:35754) 2024-03-21T11:48:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.719+0000 7f93e6a00640 1 -- 172.21.15.47:0/1016739076 learned_addr learned my addr 172.21.15.47:0/1016739076 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:31.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.720+0000 7f93e6a00640 1 -- 172.21.15.47:0/1016739076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e81073a0 msgr2=0x7f93e81a56c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:48:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.720+0000 7f93e6a00640 1 --2- 172.21.15.47:0/1016739076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e81073a0 0x7f93e81a56c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.720+0000 7f93e6a00640 1 -- 172.21.15.47:0/1016739076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e8104a70 msgr2=0x7f93e8195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.720+0000 7f93e6a00640 1 --2- 172.21.15.47:0/1016739076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e8104a70 0x7f93e8195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.720+0000 7f93e6a00640 1 -- 172.21.15.47:0/1016739076 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e81a5cb0 con 0x7f93e8102140 2024-03-21T11:48:31.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.720+0000 7f93e6a00640 1 --2- 172.21.15.47:0/1016739076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e8102140 0x7f93e8195870 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f93d8005ab0 tx=0x7f93d80049e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.722+0000 7f93df400640 1 -- 172.21.15.47:0/1016739076 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d8041070 con 0x7f93e8102140 2024-03-21T11:48:31.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.722+0000 7f93df400640 1 -- 172.21.15.47:0/1016739076 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93d8004450 con 0x7f93e8102140 2024-03-21T11:48:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.722+0000 7f93eda00640 1 -- 172.21.15.47:0/1016739076 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e81a5f40 con 0x7f93e8102140 2024-03-21T11:48:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.722+0000 7f93eda00640 1 -- 172.21.15.47:0/1016739076 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e81a63b0 con 0x7f93e8102140 2024-03-21T11:48:31.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.722+0000 7f93df400640 1 -- 172.21.15.47:0/1016739076 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d8003e50 con 0x7f93e8102140 2024-03-21T11:48:31.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.727+0000 7f93f0646740 1 Processor -- start 2024-03-21T11:48:31.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.727+0000 7f93df400640 1 -- 172.21.15.47:0/1016739076 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f93d8044070 con 0x7f93e8102140 2024-03-21T11:48:31.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.727+0000 7f93f0646740 1 -- start start 2024-03-21T11:48:31.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.728+0000 7f93f0646740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa0810 0x559813fa0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.728+0000 7f93f0646740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa19e0 0x559813fa1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.728+0000 7f93f0646740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa23a0 0x559813fa8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.728+0000 7f93f0646740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559813f052c0 con 0x559813fa19e0 2024-03-21T11:48:31.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.728+0000 7f93f0646740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559813eadc90 con 0x559813fa0810 2024-03-21T11:48:31.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.728+0000 7f93f0646740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559813f01360 con 0x559813fa23a0 2024-03-21T11:48:31.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.728+0000 7f93e7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa0810 0x559813fa0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.728+0000 7f93e7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa0810 0x559813fa0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60512/0 (socket says 172.21.15.47:60512) 2024-03-21T11:48:31.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.728+0000 7f93e7400640 1 -- 172.21.15.47:0/421554214 learned_addr learned my addr 172.21.15.47:0/421554214 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:31.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.728+0000 7f93e6a00640 1 --2- 172.21.15.47:0/421554214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa19e0 0x559813fa1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.729+0000 7f93e6000640 1 --2- 172.21.15.47:0/421554214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa23a0 0x559813fa8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.730+0000 7f93df400640 1 --2- 172.21.15.47:0/1016739076 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93cc0a3210 0x7f93cc0a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.730+0000 7f93e7400640 1 -- 172.21.15.47:0/421554214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa23a0 msgr2=0x559813fa8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:31.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.730+0000 7f93e7400640 1 --2- 172.21.15.47:0/421554214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa23a0 0x559813fa8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.730+0000 7f93e7400640 1 -- 172.21.15.47:0/421554214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa19e0 msgr2=0x559813fa1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:31.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.730+0000 7f93e7400640 1 --2- 172.21.15.47:0/421554214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa19e0 0x559813fa1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.730+0000 7f93e7400640 1 -- 172.21.15.47:0/421554214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559813ed2080 con 0x559813fa0810 2024-03-21T11:48:31.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.730+0000 7f93df400640 1 -- 172.21.15.47:0/1016739076 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f93d80f11b0 con 0x7f93e8102140 2024-03-21T11:48:31.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.731+0000 7f93e6000640 1 --2- 172.21.15.47:0/421554214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa23a0 0x559813fa8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:48:31.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.731+0000 7f93e7400640 1 --2- 172.21.15.47:0/1016739076 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93cc0a3210 0x7f93cc0a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.731+0000 7f93e6a00640 1 --2- 172.21.15.47:0/421554214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa19e0 0x559813fa1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:48:31.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.732+0000 7f93e7400640 1 --2- 172.21.15.47:0/1016739076 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93cc0a3210 0x7f93cc0a56d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f93e0014000 tx=0x7f93e0016000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:31.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.733+0000 7f93e7400640 1 --2- 172.21.15.47:0/421554214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa0810 0x559813fa0c30 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009e30 tx=0x7f93e000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d52b033aa119498e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.733+0000 7f93d3e00640 1 -- 172.21.15.47:0/421554214 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e000ee70 con 0x559813fa0810 2024-03-21T11:48:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.733+0000 7f93d3e00640 1 -- 172.21.15.47:0/421554214 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93e0025b60 con 0x559813fa0810 2024-03-21T11:48:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.733+0000 7f93d3e00640 1 -- 172.21.15.47:0/421554214 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e001ada0 con 0x559813fa0810 2024-03-21T11:48:31.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.734+0000 7f93f0646740 1 -- 172.21.15.47:0/421554214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa0810 msgr2=0x559813fa0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:31.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.734+0000 7f93f0646740 1 --2- 172.21.15.47:0/421554214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa0810 0x559813fa0c30 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009e30 tx=0x7f93e000e720 comp rx=0 tx=0).stop 2024-03-21T11:48:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.735+0000 7f93f0646740 1 -- 172.21.15.47:0/421554214 shutdown_connections 2024-03-21T11:48:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.735+0000 7f93f0646740 1 --2- 172.21.15.47:0/421554214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa23a0 0x559813fa8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.735+0000 7f93f0646740 1 --2- 172.21.15.47:0/421554214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa19e0 0x559813fa1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.735+0000 7f93f0646740 1 --2- 172.21.15.47:0/421554214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa0810 0x559813fa0c30 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.735+0000 7f93f0646740 1 -- 172.21.15.47:0/421554214 >> 172.21.15.47:0/421554214 conn(0x559813f9bc00 msgr2=0x559813f9c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.735+0000 7f93f0646740 1 -- 172.21.15.47:0/421554214 shutdown_connections 2024-03-21T11:48:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.735+0000 7f93f0646740 1 -- 172.21.15.47:0/421554214 wait complete. 2024-03-21T11:48:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.735+0000 7f93f0646740 1 -- start start 2024-03-21T11:48:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.736+0000 7f93f0646740 1 Processor -- start 2024-03-21T11:48:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.736+0000 7f93f0646740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa0810 0x559813faaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.736+0000 7f93f0646740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa19e0 0x559813fab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.736+0000 7f93f0646740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa23a0 0x559813fb2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.736+0000 7f93f0646740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559813fa93b0 con 0x559813fa23a0 2024-03-21T11:48:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.736+0000 7f93f0646740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559813ec10d0 con 0x559813fa19e0 2024-03-21T11:48:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.736+0000 7f93f0646740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559813fa9530 con 0x559813fa0810 2024-03-21T11:48:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.736+0000 7f93f0646740 10 client.0 _ll_register_callbacks cb 0x559813ec5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.737+0000 7f93e6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa19e0 0x559813fab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.737+0000 7f93e6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa19e0 0x559813fab000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60514/0 (socket says 172.21.15.47:60514) 2024-03-21T11:48:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.737+0000 7f93e6000640 1 -- 172.21.15.47:0/2536035052 learned_addr learned my addr 172.21.15.47:0/2536035052 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.737+0000 7f93e7400640 1 --2- 172.21.15.47:0/2536035052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa23a0 0x559813fb2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.738+0000 7f93e7400640 1 -- 172.21.15.47:0/2536035052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa0810 msgr2=0x559813faaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.738+0000 7f93e7400640 1 --2- 172.21.15.47:0/2536035052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa0810 0x559813faaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.738+0000 7f93e7400640 1 -- 172.21.15.47:0/2536035052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa19e0 msgr2=0x559813fab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:31.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.738+0000 7f93e7400640 1 --2- 172.21.15.47:0/2536035052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa19e0 0x559813fab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.738+0000 7f93e7400640 1 -- 172.21.15.47:0/2536035052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559813c193f0 con 0x559813fa23a0 2024-03-21T11:48:31.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.739+0000 7f93e7400640 1 --2- 172.21.15.47:0/2536035052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa23a0 0x559813fb2040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f93e0006010 tx=0x7f93e001a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514e4a4610a4a43b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:31.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.739+0000 7f93d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:48:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.741+0000 7f93d0c00640 1 -- 172.21.15.47:0/2536035052 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0003d80 con 0x559813fa23a0 2024-03-21T11:48:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.742+0000 7f93d0c00640 1 -- 172.21.15.47:0/2536035052 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f93e0025480 con 0x559813fa23a0 2024-03-21T11:48:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.742+0000 7f93f0646740 1 -- 172.21.15.47:0/2536035052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93e0025b60 con 0x559813fa23a0 2024-03-21T11:48:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.742+0000 7f93d0c00640 1 -- 172.21.15.47:0/2536035052 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93e003b050 con 0x559813fa23a0 2024-03-21T11:48:31.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.742+0000 7f93f0646740 10 client.22515 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:31.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.743+0000 7f93f0646740 1 -- 172.21.15.47:0/2536035052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x559813ce37d0 con 0x559813fa23a0 2024-03-21T11:48:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.745+0000 7f93d0c00640 1 -- 172.21.15.47:0/2536035052 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93e0025b60 con 0x559813fa23a0 2024-03-21T11:48:31.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.745+0000 7f93d0c00640 1 -- 172.21.15.47:0/2536035052 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f93e0036030 con 0x559813fa23a0 2024-03-21T11:48:31.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.745+0000 7f93f0646740 10 client.22515 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:31.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.745+0000 7f93f0646740 10 client.22515 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:48:31.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.745+0000 7f93f0646740 20 client.22515 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:31.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.746+0000 7f93f0646740 1 --2- 172.21.15.47:0/2536035052 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559813fb61f0 0x559813fb85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.746+0000 7f93f0646740 4 client.22515 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:48:31.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.746+0000 7f93f0646740 1 -- 172.21.15.47:0/2536035052 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559813f12800 con 0x559813fb61f0 2024-03-21T11:48:31.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.748+0000 7f93e6a00640 1 --2- 172.21.15.47:0/2536035052 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559813fb61f0 0x559813fb85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.749+0000 7f93e6a00640 1 --2- 172.21.15.47:0/2536035052 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559813fb61f0 0x559813fb85e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=af3b89161e5ea34b server_cookie=de682c6618d8bdc1 in_seq=0 out_seq=0 2024-03-21T11:48:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.749+0000 7f93d0c00640 10 client.22515 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:48:31.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.750+0000 7f93d0c00640 1 -- 172.21.15.47:0/2536035052 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559813f12800 con 0x559813fb61f0 2024-03-21T11:48:31.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.750+0000 7f93d0c00640 10 client.22515 handle_command_reply: tid=0 2024-03-21T11:48:31.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.750+0000 7f93f0646740 1 client.22515 shutdown 2024-03-21T11:48:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.751+0000 7f93f0646740 1 -- 172.21.15.47:0/2536035052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa23a0 msgr2=0x559813fb2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.751+0000 7f93f0646740 1 --2- 172.21.15.47:0/2536035052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa23a0 0x559813fb2040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f93e0006010 tx=0x7f93e001a5b0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.752+0000 7f93f0646740 1 -- 172.21.15.47:0/2536035052 shutdown_connections 2024-03-21T11:48:31.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.752+0000 7f93f0646740 1 --2- 172.21.15.47:0/2536035052 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559813fb61f0 0x559813fb85e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.752+0000 7f93f0646740 1 --2- 172.21.15.47:0/2536035052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa23a0 0x559813fb2040 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.752+0000 7f93f0646740 1 --2- 172.21.15.47:0/2536035052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa19e0 0x559813fab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.752+0000 7f93d0c00640 0 client.22515 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:48:31.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.752+0000 7f93f0646740 1 --2- 172.21.15.47:0/2536035052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa0810 0x559813faaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.752+0000 7f93f0646740 1 -- 172.21.15.47:0/2536035052 >> 172.21.15.47:0/2536035052 conn(0x559813f9bc00 msgr2=0x559813f9aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:31.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.753+0000 7f93f0646740 1 -- 172.21.15.47:0/2536035052 shutdown_connections 2024-03-21T11:48:31.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.753+0000 7f93f0646740 1 -- 172.21.15.47:0/2536035052 wait complete. 2024-03-21T11:48:31.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.753+0000 7f93f0646740 20 client.22515 trim_cache size 0 max 16384 2024-03-21T11:48:31.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.776+0000 7f93f0646740 1 Processor -- start 2024-03-21T11:48:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.777+0000 7f93f0646740 1 -- start start 2024-03-21T11:48:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.777+0000 7f93f0646740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa0810 0x559813fa4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.777+0000 7f93f0646740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa19e0 0x559813fb2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.777+0000 7f93f0646740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa23a0 0x559813fa5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.777+0000 7f93f0646740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c800fa40 con 0x559813fa0810 2024-03-21T11:48:31.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.777+0000 7f93f0646740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c80113c0 con 0x559813fa19e0 2024-03-21T11:48:31.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.777+0000 7f93f0646740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c80100a0 con 0x559813fa23a0 2024-03-21T11:48:31.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.777+0000 7f93e6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa0810 0x559813fa4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.778+0000 7f93e7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa23a0 0x559813fa5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.778+0000 7f93e6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa19e0 0x559813fb2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.778+0000 7f93e7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa23a0 0x559813fa5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35786/0 (socket says 172.21.15.47:35786) 2024-03-21T11:48:31.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.778+0000 7f93e7400640 1 -- 172.21.15.47:0/542667616 learned_addr learned my addr 172.21.15.47:0/542667616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:31.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.778+0000 7f93e7400640 1 -- 172.21.15.47:0/542667616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa19e0 msgr2=0x559813fb2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:31.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.778+0000 7f93e7400640 1 --2- 172.21.15.47:0/542667616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa19e0 0x559813fb2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.778+0000 7f93e7400640 1 -- 172.21.15.47:0/542667616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa0810 msgr2=0x559813fa4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:31.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.778+0000 7f93e7400640 1 --2- 172.21.15.47:0/542667616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa0810 0x559813fa4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.778+0000 7f93e7400640 1 -- 172.21.15.47:0/542667616 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559813fb8bc0 con 0x559813fa23a0 2024-03-21T11:48:31.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.779+0000 7f93e7400640 1 --2- 172.21.15.47:0/542667616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa23a0 0x559813fa5150 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009e30 tx=0x7f93e000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fe8d322dd585380 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:31.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.779+0000 7f93d3400640 1 -- 172.21.15.47:0/542667616 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0004930 con 0x559813fa23a0 2024-03-21T11:48:31.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.780+0000 7f93d3400640 1 -- 172.21.15.47:0/542667616 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93e000ee70 con 0x559813fa23a0 2024-03-21T11:48:31.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.780+0000 7f93d3400640 1 -- 172.21.15.47:0/542667616 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0003de0 con 0x559813fa23a0 2024-03-21T11:48:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.781+0000 7f93f0646740 1 -- 172.21.15.47:0/542667616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa23a0 msgr2=0x559813fa5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.781+0000 7f93f0646740 1 --2- 172.21.15.47:0/542667616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa23a0 0x559813fa5150 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009e30 tx=0x7f93e000e720 comp rx=0 tx=0).stop 2024-03-21T11:48:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.781+0000 7f93f0646740 1 -- 172.21.15.47:0/542667616 shutdown_connections 2024-03-21T11:48:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.781+0000 7f93f0646740 1 --2- 172.21.15.47:0/542667616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa23a0 0x559813fa5150 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.781+0000 7f93f0646740 1 --2- 172.21.15.47:0/542667616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa19e0 0x559813fb2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.781+0000 7f93f0646740 1 --2- 172.21.15.47:0/542667616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa0810 0x559813fa4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.781+0000 7f93f0646740 1 -- 172.21.15.47:0/542667616 >> 172.21.15.47:0/542667616 conn(0x559813f9bc00 msgr2=0x559813f9f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.781+0000 7f93f0646740 1 -- 172.21.15.47:0/542667616 shutdown_connections 2024-03-21T11:48:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.781+0000 7f93f0646740 1 -- 172.21.15.47:0/542667616 wait complete. 2024-03-21T11:48:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.782+0000 7f93f0646740 1 -- start start 2024-03-21T11:48:31.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.782+0000 7f93f0646740 1 Processor -- start 2024-03-21T11:48:31.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.782+0000 7f93f0646740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa0810 0x559813fb1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.782+0000 7f93f0646740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa19e0 0x559813fb1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.783+0000 7f93f0646740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa23a0 0x559813faff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.783+0000 7f93f0646740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c8010d60 con 0x559813fa19e0 2024-03-21T11:48:31.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.783+0000 7f93f0646740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c8010700 con 0x559813fa23a0 2024-03-21T11:48:31.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.783+0000 7f93f0646740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c800dc00 con 0x559813fa0810 2024-03-21T11:48:31.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.783+0000 7f93f0646740 10 client.0 _ll_register_callbacks cb 0x7f93d80f4b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:31.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.783+0000 7f93e7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa23a0 0x559813faff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.783+0000 7f93e7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa23a0 0x559813faff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60530/0 (socket says 172.21.15.47:60530) 2024-03-21T11:48:31.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.783+0000 7f93e7400640 1 -- 172.21.15.47:0/2311982285 learned_addr learned my addr 172.21.15.47:0/2311982285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:31.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.783+0000 7f93e6a00640 1 --2- 172.21.15.47:0/2311982285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa0810 0x559813fb1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.784+0000 7f93e7400640 1 -- 172.21.15.47:0/2311982285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa0810 msgr2=0x559813fb1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:31.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.784+0000 7f93e7400640 1 --2- 172.21.15.47:0/2311982285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa0810 0x559813fb1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.784+0000 7f93e7400640 1 -- 172.21.15.47:0/2311982285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa19e0 msgr2=0x559813fb1e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:48:31.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.784+0000 7f93e6000640 1 --2- 172.21.15.47:0/2311982285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa19e0 0x559813fb1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.784+0000 7f93e7400640 1 --2- 172.21.15.47:0/2311982285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa19e0 0x559813fb1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.784+0000 7f93e7400640 1 -- 172.21.15.47:0/2311982285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559813fa1200 con 0x559813fa23a0 2024-03-21T11:48:31.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.784+0000 7f93e7400640 1 --2- 172.21.15.47:0/2311982285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa23a0 0x559813faff10 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f93e0025ce0 tx=0x7f93e0004960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d82d25f5977fd9e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:31.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.784+0000 7f93d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:48:31.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.786+0000 7f93d0c00640 1 -- 172.21.15.47:0/2311982285 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e001ae40 con 0x559813fa23a0 2024-03-21T11:48:31.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.786+0000 7f93d0c00640 1 -- 172.21.15.47:0/2311982285 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f93e001a590 con 0x559813fa23a0 2024-03-21T11:48:31.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.786+0000 7f93f0646740 1 -- 172.21.15.47:0/2311982285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93e000ee70 con 0x559813fa23a0 2024-03-21T11:48:31.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.787+0000 7f93d0c00640 1 -- 172.21.15.47:0/2311982285 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93e0025690 con 0x559813fa23a0 2024-03-21T11:48:31.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.788+0000 7f93f0646740 10 client.33413 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:31.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.788+0000 7f93f0646740 1 -- 172.21.15.47:0/2311982285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x559814012220 con 0x559813fa23a0 2024-03-21T11:48:31.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.788+0000 7f93d0c00640 1 -- 172.21.15.47:0/2311982285 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93e000ee70 con 0x559813fa23a0 2024-03-21T11:48:31.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.789+0000 7f93d0c00640 1 -- 172.21.15.47:0/2311982285 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f93e0036040 con 0x559813fa23a0 2024-03-21T11:48:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.789+0000 7f93f0646740 10 client.33413 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.789+0000 7f93f0646740 10 client.33413 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:48:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.789+0000 7f93f0646740 20 client.33413 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.789+0000 7f93f0646740 1 --2- 172.21.15.47:0/2311982285 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559814012b20 0x559814014f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.789+0000 7f93f0646740 4 client.33413 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:48:31.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.789+0000 7f93f0646740 1 -- 172.21.15.47:0/2311982285 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559813f0cb50 con 0x559814012b20 2024-03-21T11:48:31.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.789+0000 7f93e6a00640 1 --2- 172.21.15.47:0/2311982285 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559814012b20 0x559814014f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.790+0000 7f93e6a00640 1 --2- 172.21.15.47:0/2311982285 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559814012b20 0x559814014f40 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=217793a55d1ec252 server_cookie=fd0ba7ae1c3099a4 in_seq=0 out_seq=0 2024-03-21T11:48:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.790+0000 7f93d0c00640 10 client.33413 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:48:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.790+0000 7f93d0c00640 1 -- 172.21.15.47:0/2311982285 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559813f0cb50 con 0x559814012b20 2024-03-21T11:48:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.790+0000 7f93d0c00640 10 client.33413 handle_command_reply: tid=0 2024-03-21T11:48:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.790+0000 7f93f0646740 1 client.33413 shutdown 2024-03-21T11:48:31.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.791+0000 7f93f0646740 1 -- 172.21.15.47:0/2311982285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa23a0 msgr2=0x559813faff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:31.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.791+0000 7f93f0646740 1 --2- 172.21.15.47:0/2311982285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa23a0 0x559813faff10 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f93e0025ce0 tx=0x7f93e0004960 comp rx=0 tx=0).stop 2024-03-21T11:48:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.791+0000 7f93f0646740 1 -- 172.21.15.47:0/2311982285 shutdown_connections 2024-03-21T11:48:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.791+0000 7f93f0646740 1 --2- 172.21.15.47:0/2311982285 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x559814012b20 0x559814014f40 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.791+0000 7f93f0646740 1 --2- 172.21.15.47:0/2311982285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559813fa23a0 0x559813faff10 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.791+0000 7f93f0646740 1 --2- 172.21.15.47:0/2311982285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559813fa19e0 0x559813fb1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.791+0000 7f93f0646740 1 --2- 172.21.15.47:0/2311982285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559813fa0810 0x559813fb1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.791+0000 7f93f0646740 1 -- 172.21.15.47:0/2311982285 >> 172.21.15.47:0/2311982285 conn(0x559813f9bc00 msgr2=0x559813f9a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.791+0000 7f93d0c00640 0 client.33413 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:48:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.792+0000 7f93f0646740 1 -- 172.21.15.47:0/2311982285 shutdown_connections 2024-03-21T11:48:31.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.792+0000 7f93f0646740 1 -- 172.21.15.47:0/2311982285 wait complete. 2024-03-21T11:48:31.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.793+0000 7f93f0646740 20 client.33413 trim_cache size 0 max 16384 2024-03-21T11:48:31.794 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:48:31.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.798+0000 7f93eda00640 1 -- 172.21.15.47:0/1016739076 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93cc0a3210 msgr2=0x7f93cc0a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:31.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.798+0000 7f93eda00640 1 --2- 172.21.15.47:0/1016739076 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93cc0a3210 0x7f93cc0a56d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f93e0014000 tx=0x7f93e0016000 comp rx=0 tx=0).stop 2024-03-21T11:48:31.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.798+0000 7f93eda00640 1 -- 172.21.15.47:0/1016739076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e8102140 msgr2=0x7f93e8195870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:31.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.798+0000 7f93eda00640 1 --2- 172.21.15.47:0/1016739076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e8102140 0x7f93e8195870 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f93d8005ab0 tx=0x7f93d80049e0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.798+0000 7f93eda00640 1 -- 172.21.15.47:0/1016739076 shutdown_connections 2024-03-21T11:48:31.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.798+0000 7f93eda00640 1 --2- 172.21.15.47:0/1016739076 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f93cc0a3210 0x7f93cc0a56d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.798+0000 7f93eda00640 1 --2- 172.21.15.47:0/1016739076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e81073a0 0x7f93e81a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.798+0000 7f93eda00640 1 --2- 172.21.15.47:0/1016739076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e8104a70 0x7f93e8195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.798+0000 7f93eda00640 1 --2- 172.21.15.47:0/1016739076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e8102140 0x7f93e8195870 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:31.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.798+0000 7f93eda00640 1 -- 172.21.15.47:0/1016739076 >> 172.21.15.47:0/1016739076 conn(0x7f93e80fbb30 msgr2=0x7f93e8101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:31.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.798+0000 7f93eda00640 1 -- 172.21.15.47:0/1016739076 shutdown_connections 2024-03-21T11:48:31.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:31.799+0000 7f93eda00640 1 -- 172.21.15.47:0/1016739076 wait complete. 2024-03-21T11:48:31.830 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:48:31.831 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:48:32.001 INFO:tasks.workunit.client.0.smithi047.stdout: 10 278669 21.40 MB/sec execute 555 sec latency 159.947 ms 2024-03-21T11:48:32.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.115+0000 7f3767800640 1 -- 172.21.15.47:0/2572224539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376010e1e0 msgr2=0x7f3760110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.115+0000 7f3767800640 1 --2- 172.21.15.47:0/2572224539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376010e1e0 0x7f3760110670 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f3758009ca0 tx=0x7f375802c100 comp rx=0 tx=0).stop 2024-03-21T11:48:32.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.117+0000 7f3767800640 1 -- 172.21.15.47:0/2572224539 shutdown_connections 2024-03-21T11:48:32.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.117+0000 7f3767800640 1 --2- 172.21.15.47:0/2572224539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376010e1e0 0x7f3760110670 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.118+0000 7f3767800640 1 --2- 172.21.15.47:0/2572224539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f376010b780 0x7f376010db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.118+0000 7f3767800640 1 --2- 172.21.15.47:0/2572224539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376006a4b0 0x7f376010b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.118+0000 7f3767800640 1 -- 172.21.15.47:0/2572224539 >> 172.21.15.47:0/2572224539 conn(0x7f376006e8a0 msgr2=0x7f3760070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.119+0000 7f3767800640 1 -- 172.21.15.47:0/2572224539 shutdown_connections 2024-03-21T11:48:32.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.119+0000 7f3767800640 1 -- 172.21.15.47:0/2572224539 wait complete. 2024-03-21T11:48:32.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.122+0000 7f3767800640 1 Processor -- start 2024-03-21T11:48:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.123+0000 7f3767800640 1 -- start start 2024-03-21T11:48:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.124+0000 7f3767800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376006a4b0 0x7f37601a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.124+0000 7f3767800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376010b780 0x7f37601a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.124+0000 7f3767800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f376010e1e0 0x7f37601add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.124+0000 7f3767800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37601138e0 con 0x7f376010e1e0 2024-03-21T11:48:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.124+0000 7f3767800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3760113760 con 0x7f376010b780 2024-03-21T11:48:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.124+0000 7f3767800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3760113a60 con 0x7f376006a4b0 2024-03-21T11:48:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.124+0000 7f375fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376010b780 0x7f37601a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.124+0000 7f375fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376010b780 0x7f37601a6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60560/0 (socket says 172.21.15.47:60560) 2024-03-21T11:48:32.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.125+0000 7f3764e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376006a4b0 0x7f37601a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.125+0000 7f3765800640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f376010e1e0 0x7f37601add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.125+0000 7f375fe00640 1 -- 172.21.15.47:0/81293352 learned_addr learned my addr 172.21.15.47:0/81293352 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.126+0000 7f3765800640 1 -- 172.21.15.47:0/81293352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376006a4b0 msgr2=0x7f37601a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.126+0000 7f3765800640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376006a4b0 0x7f37601a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.126+0000 7f3765800640 1 -- 172.21.15.47:0/81293352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376010b780 msgr2=0x7f37601a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.126+0000 7f3765800640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376010b780 0x7f37601a6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.126+0000 7f3765800640 1 -- 172.21.15.47:0/81293352 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37601ae3c0 con 0x7f376010e1e0 2024-03-21T11:48:32.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.126+0000 7f3764e00640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376006a4b0 0x7f37601a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:48:32.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.126+0000 7f375fe00640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376010b780 0x7f37601a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:48:32.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.126+0000 7f3765800640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f376010e1e0 0x7f37601add50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3758037040 tx=0x7f3758035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.126+0000 7f375d600640 1 -- 172.21.15.47:0/81293352 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758039040 con 0x7f376010e1e0 2024-03-21T11:48:32.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.127+0000 7f375d600640 1 -- 172.21.15.47:0/81293352 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3758044070 con 0x7f376010e1e0 2024-03-21T11:48:32.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.127+0000 7f3767800640 1 -- 172.21.15.47:0/81293352 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37601ae650 con 0x7f376010e1e0 2024-03-21T11:48:32.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.127+0000 7f3767800640 1 -- 172.21.15.47:0/81293352 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37601aeb60 con 0x7f376010e1e0 2024-03-21T11:48:32.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.127+0000 7f375d600640 1 -- 172.21.15.47:0/81293352 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758004350 con 0x7f376010e1e0 2024-03-21T11:48:32.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.132+0000 7f375d600640 1 -- 172.21.15.47:0/81293352 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f37580044f0 con 0x7f376010e1e0 2024-03-21T11:48:32.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.133+0000 7f376a57c740 1 Processor -- start 2024-03-21T11:48:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.133+0000 7f376a57c740 1 -- start start 2024-03-21T11:48:32.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.134+0000 7f376a57c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388ba810 0x564f388bac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.134+0000 7f376a57c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bb9e0 0x564f388bbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.134+0000 7f376a57c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388bc3a0 0x564f388c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.134+0000 7f376a57c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3881f2c0 con 0x564f388bc3a0 2024-03-21T11:48:32.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.134+0000 7f376a57c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f387c7c90 con 0x564f388bb9e0 2024-03-21T11:48:32.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.134+0000 7f376a57c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3881b360 con 0x564f388ba810 2024-03-21T11:48:32.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.134+0000 7f3764e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bb9e0 0x564f388bbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.134+0000 7f3764e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bb9e0 0x564f388bbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60570/0 (socket says 172.21.15.47:60570) 2024-03-21T11:48:32.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.134+0000 7f3764e00640 1 -- 172.21.15.47:0/1483184432 learned_addr learned my addr 172.21.15.47:0/1483184432 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.134+0000 7f375fe00640 1 --2- 172.21.15.47:0/1483184432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388ba810 0x564f388bac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.135+0000 7f3764e00640 1 -- 172.21.15.47:0/1483184432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388ba810 msgr2=0x564f388bac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.135+0000 7f3764e00640 1 --2- 172.21.15.47:0/1483184432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388ba810 0x564f388bac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.135+0000 7f3764e00640 1 -- 172.21.15.47:0/1483184432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388bc3a0 msgr2=0x564f388c2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.135+0000 7f3764e00640 1 --2- 172.21.15.47:0/1483184432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388bc3a0 0x564f388c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.135+0000 7f3764e00640 1 -- 172.21.15.47:0/1483184432 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f387ec080 con 0x564f388bb9e0 2024-03-21T11:48:32.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.135+0000 7f375d600640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f37480a2fa0 0x7f37480a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.135+0000 7f375d600640 1 -- 172.21.15.47:0/81293352 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f37580f1af0 con 0x7f376010e1e0 2024-03-21T11:48:32.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.135+0000 7f375fe00640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f37480a2fa0 0x7f37480a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.136+0000 7f3764e00640 1 --2- 172.21.15.47:0/1483184432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bb9e0 0x564f388bbe60 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3754009cd0 tx=0x7f37540117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=303de76b9f519c5b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.136+0000 7f375fe00640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f37480a2fa0 0x7f37480a5460 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x564f388bb8e0 tx=0x7f374c014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.137+0000 7f3752000640 1 -- 172.21.15.47:0/1483184432 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3754012450 con 0x564f388bb9e0 2024-03-21T11:48:32.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.137+0000 7f3752000640 1 -- 172.21.15.47:0/1483184432 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3754012ad0 con 0x564f388bb9e0 2024-03-21T11:48:32.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.137+0000 7f3752000640 1 -- 172.21.15.47:0/1483184432 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375400fca0 con 0x564f388bb9e0 2024-03-21T11:48:32.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.138+0000 7f376a57c740 1 -- 172.21.15.47:0/1483184432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bb9e0 msgr2=0x564f388bbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.138+0000 7f376a57c740 1 --2- 172.21.15.47:0/1483184432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bb9e0 0x564f388bbe60 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3754009cd0 tx=0x7f37540117b0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.138+0000 7f376a57c740 1 -- 172.21.15.47:0/1483184432 shutdown_connections 2024-03-21T11:48:32.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.138+0000 7f376a57c740 1 --2- 172.21.15.47:0/1483184432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388bc3a0 0x564f388c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.138+0000 7f376a57c740 1 --2- 172.21.15.47:0/1483184432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bb9e0 0x564f388bbe60 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.138+0000 7f376a57c740 1 --2- 172.21.15.47:0/1483184432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388ba810 0x564f388bac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.138+0000 7f376a57c740 1 -- 172.21.15.47:0/1483184432 >> 172.21.15.47:0/1483184432 conn(0x564f388b5c00 msgr2=0x564f388b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.140+0000 7f376a57c740 1 -- 172.21.15.47:0/1483184432 shutdown_connections 2024-03-21T11:48:32.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.140+0000 7f376a57c740 1 -- 172.21.15.47:0/1483184432 wait complete. 2024-03-21T11:48:32.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.141+0000 7f376a57c740 1 -- start start 2024-03-21T11:48:32.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.141+0000 7f376a57c740 1 Processor -- start 2024-03-21T11:48:32.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.142+0000 7f376a57c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388ba810 0x564f388c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.142+0000 7f376a57c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388bb9e0 0x564f388c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.142+0000 7f376a57c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bc3a0 0x564f388cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.142+0000 7f376a57c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f388c33b0 con 0x564f388bb9e0 2024-03-21T11:48:32.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.142+0000 7f376a57c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f387db0d0 con 0x564f388bc3a0 2024-03-21T11:48:32.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.142+0000 7f376a57c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f388c3530 con 0x564f388ba810 2024-03-21T11:48:32.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.142+0000 7f376a57c740 10 client.0 _ll_register_callbacks cb 0x564f387dfe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.143+0000 7f3764e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388ba810 0x564f388c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.143+0000 7f3764e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388ba810 0x564f388c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35814/0 (socket says 172.21.15.47:35814) 2024-03-21T11:48:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.143+0000 7f3764e00640 1 -- 172.21.15.47:0/2324404997 learned_addr learned my addr 172.21.15.47:0/2324404997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.143+0000 7f375fe00640 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388bb9e0 0x564f388c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.143+0000 7f3765800640 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bc3a0 0x564f388cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.143+0000 7f3764e00640 1 -- 172.21.15.47:0/2324404997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bc3a0 msgr2=0x564f388cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.143+0000 7f3764e00640 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bc3a0 0x564f388cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.143+0000 7f3764e00640 1 -- 172.21.15.47:0/2324404997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388bb9e0 msgr2=0x564f388c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.143+0000 7f3764e00640 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388bb9e0 0x564f388c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.144+0000 7f3764e00640 1 -- 172.21.15.47:0/2324404997 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f385333f0 con 0x564f388ba810 2024-03-21T11:48:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.144+0000 7f3765800640 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bc3a0 0x564f388cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:48:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.144+0000 7f375fe00640 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388bb9e0 0x564f388c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:48:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.144+0000 7f3746a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:48:32.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.144+0000 7f3764e00640 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388ba810 0x564f388c48b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f3754009ca0 tx=0x7f375400f6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a25cd823717316d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.145+0000 7f3746a00640 1 -- 172.21.15.47:0/2324404997 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3754029020 con 0x564f388ba810 2024-03-21T11:48:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.145+0000 7f3746a00640 1 -- 172.21.15.47:0/2324404997 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3754012c80 con 0x564f388ba810 2024-03-21T11:48:32.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.145+0000 7f376a57c740 1 -- 172.21.15.47:0/2324404997 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3754012ad0 con 0x564f388ba810 2024-03-21T11:48:32.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.146+0000 7f3746a00640 1 -- 172.21.15.47:0/2324404997 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3754004040 con 0x564f388ba810 2024-03-21T11:48:32.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.146+0000 7f376a57c740 10 client.33382 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:32.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.146+0000 7f376a57c740 1 -- 172.21.15.47:0/2324404997 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564f385fd7d0 con 0x564f388ba810 2024-03-21T11:48:32.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.148+0000 7f3746a00640 1 -- 172.21.15.47:0/2324404997 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3754012ad0 con 0x564f388ba810 2024-03-21T11:48:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.148+0000 7f3746a00640 1 -- 172.21.15.47:0/2324404997 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3754003700 con 0x564f388ba810 2024-03-21T11:48:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.148+0000 7f376a57c740 10 client.33382 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.148+0000 7f376a57c740 10 client.33382 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:48:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.148+0000 7f376a57c740 20 client.33382 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.148+0000 7f376a57c740 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x564f388e1030 0x564f388e3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.148+0000 7f376a57c740 4 client.33382 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:48:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.148+0000 7f376a57c740 1 -- 172.21.15.47:0/2324404997 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f3882c800 con 0x564f388e1030 2024-03-21T11:48:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.149+0000 7f375fe00640 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x564f388e1030 0x564f388e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.152+0000 7f375fe00640 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x564f388e1030 0x564f388e3420 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=42994e7db37c33b0 server_cookie=9afe915e11fb89b3 in_seq=0 out_seq=0 2024-03-21T11:48:32.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.152+0000 7f3746a00640 10 client.33382 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:48:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.154+0000 7f3746a00640 1 -- 172.21.15.47:0/2324404997 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564f3882c800 con 0x564f388e1030 2024-03-21T11:48:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.155+0000 7f3746a00640 10 client.33382 handle_command_reply: tid=0 2024-03-21T11:48:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.155+0000 7f376a57c740 1 client.33382 shutdown 2024-03-21T11:48:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.155+0000 7f376a57c740 1 -- 172.21.15.47:0/2324404997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388ba810 msgr2=0x564f388c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.155+0000 7f376a57c740 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388ba810 0x564f388c48b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f3754009ca0 tx=0x7f375400f6d0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.155+0000 7f376a57c740 1 -- 172.21.15.47:0/2324404997 shutdown_connections 2024-03-21T11:48:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.155+0000 7f376a57c740 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x564f388e1030 0x564f388e3420 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.155+0000 7f376a57c740 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bc3a0 0x564f388cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.155+0000 7f376a57c740 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388bb9e0 0x564f388c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.156+0000 7f376a57c740 1 --2- 172.21.15.47:0/2324404997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388ba810 0x564f388c48b0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.156+0000 7f3746a00640 0 client.33382 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:48:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.156+0000 7f376a57c740 1 -- 172.21.15.47:0/2324404997 >> 172.21.15.47:0/2324404997 conn(0x564f388b5c00 msgr2=0x564f388b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.156+0000 7f376a57c740 1 -- 172.21.15.47:0/2324404997 shutdown_connections 2024-03-21T11:48:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.156+0000 7f376a57c740 1 -- 172.21.15.47:0/2324404997 wait complete. 2024-03-21T11:48:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.156+0000 7f376a57c740 20 client.33382 trim_cache size 0 max 16384 2024-03-21T11:48:32.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.183+0000 7f376a57c740 1 Processor -- start 2024-03-21T11:48:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.183+0000 7f376a57c740 1 -- start start 2024-03-21T11:48:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.183+0000 7f376a57c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388ba810 0x564f388b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.183+0000 7f376a57c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388bb9e0 0x564f388cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.183+0000 7f376a57c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bc3a0 0x564f388e7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.183+0000 7f376a57c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374000fa30 con 0x564f388ba810 2024-03-21T11:48:32.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.183+0000 7f376a57c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37400113b0 con 0x564f388bc3a0 2024-03-21T11:48:32.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.183+0000 7f376a57c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3740010090 con 0x564f388bb9e0 2024-03-21T11:48:32.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.184+0000 7f375fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388bb9e0 0x564f388cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.184+0000 7f3765800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bc3a0 0x564f388e7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.185+0000 7f3765800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bc3a0 0x564f388e7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60590/0 (socket says 172.21.15.47:60590) 2024-03-21T11:48:32.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.184+0000 7f375fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388bb9e0 0x564f388cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35824/0 (socket says 172.21.15.47:35824) 2024-03-21T11:48:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.185+0000 7f375fe00640 1 -- 172.21.15.47:0/2343883341 learned_addr learned my addr 172.21.15.47:0/2343883341 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.185+0000 7f3764e00640 1 --2- 172.21.15.47:0/2343883341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388ba810 0x564f388b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.186+0000 7f375fe00640 1 -- 172.21.15.47:0/2343883341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bc3a0 msgr2=0x564f388e7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.186+0000 7f375fe00640 1 --2- 172.21.15.47:0/2343883341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bc3a0 0x564f388e7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.186+0000 7f375fe00640 1 -- 172.21.15.47:0/2343883341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388ba810 msgr2=0x564f388b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.186+0000 7f375fe00640 1 --2- 172.21.15.47:0/2343883341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388ba810 0x564f388b8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.186+0000 7f375fe00640 1 -- 172.21.15.47:0/2343883341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f388e39e0 con 0x564f388bb9e0 2024-03-21T11:48:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.187+0000 7f375fe00640 1 --2- 172.21.15.47:0/2343883341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388bb9e0 0x564f388cbd50 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f37601a7cb0 tx=0x7f374c01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec3000c556490dff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.188+0000 7f3765800640 1 --2- 172.21.15.47:0/2343883341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bc3a0 0x564f388e7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:48:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.188+0000 7f3751600640 1 -- 172.21.15.47:0/2343883341 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c02d070 con 0x564f388bb9e0 2024-03-21T11:48:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.189+0000 7f3751600640 1 -- 172.21.15.47:0/2343883341 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f374c0272d0 con 0x564f388bb9e0 2024-03-21T11:48:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.189+0000 7f3751600640 1 -- 172.21.15.47:0/2343883341 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c01e3f0 con 0x564f388bb9e0 2024-03-21T11:48:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.190+0000 7f376a57c740 1 -- 172.21.15.47:0/2343883341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388bb9e0 msgr2=0x564f388cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.190+0000 7f376a57c740 1 --2- 172.21.15.47:0/2343883341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388bb9e0 0x564f388cbd50 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f37601a7cb0 tx=0x7f374c01a000 comp rx=0 tx=0).stop 2024-03-21T11:48:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.191+0000 7f376a57c740 1 -- 172.21.15.47:0/2343883341 shutdown_connections 2024-03-21T11:48:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.191+0000 7f376a57c740 1 --2- 172.21.15.47:0/2343883341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bc3a0 0x564f388e7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.191+0000 7f376a57c740 1 --2- 172.21.15.47:0/2343883341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388bb9e0 0x564f388cbd50 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.191+0000 7f376a57c740 1 --2- 172.21.15.47:0/2343883341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388ba810 0x564f388b8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.191+0000 7f376a57c740 1 -- 172.21.15.47:0/2343883341 >> 172.21.15.47:0/2343883341 conn(0x564f388b5c00 msgr2=0x564f388b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.191+0000 7f376a57c740 1 -- 172.21.15.47:0/2343883341 shutdown_connections 2024-03-21T11:48:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.191+0000 7f376a57c740 1 -- 172.21.15.47:0/2343883341 wait complete. 2024-03-21T11:48:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.193+0000 7f376a57c740 1 -- start start 2024-03-21T11:48:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.193+0000 7f376a57c740 1 Processor -- start 2024-03-21T11:48:32.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.194+0000 7f376a57c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388ba810 0x564f388c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.195+0000 7f376a57c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bb9e0 0x564f388c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.195+0000 7f376a57c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388bc3a0 0x564f388cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.195+0000 7f376a57c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3740010d50 con 0x564f388ba810 2024-03-21T11:48:32.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.195+0000 7f376a57c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37400106f0 con 0x564f388bb9e0 2024-03-21T11:48:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.195+0000 7f376a57c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374000dbb0 con 0x564f388bc3a0 2024-03-21T11:48:32.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.195+0000 7f376a57c740 10 client.0 _ll_register_callbacks cb 0x7f374c009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.196+0000 7f3764e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388ba810 0x564f388c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.196+0000 7f375fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bb9e0 0x564f388c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.196+0000 7f3765800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388bc3a0 0x564f388cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.196+0000 7f3765800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388bc3a0 0x564f388cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35826/0 (socket says 172.21.15.47:35826) 2024-03-21T11:48:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.196+0000 7f3765800640 1 -- 172.21.15.47:0/1663523905 learned_addr learned my addr 172.21.15.47:0/1663523905 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.196+0000 7f375fe00640 1 --2- 172.21.15.47:0/1663523905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bb9e0 0x564f388c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60606/0 (socket says 172.21.15.47:60606) 2024-03-21T11:48:32.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.197+0000 7f375fe00640 1 -- 172.21.15.47:0/1663523905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388bc3a0 msgr2=0x564f388cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.197+0000 7f375fe00640 1 --2- 172.21.15.47:0/1663523905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388bc3a0 0x564f388cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.197+0000 7f375fe00640 1 -- 172.21.15.47:0/1663523905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388ba810 msgr2=0x564f388c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.197+0000 7f375fe00640 1 --2- 172.21.15.47:0/1663523905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388ba810 0x564f388c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.197+0000 7f375fe00640 1 -- 172.21.15.47:0/1663523905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f388cb8c0 con 0x564f388bb9e0 2024-03-21T11:48:32.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.198+0000 7f375fe00640 1 --2- 172.21.15.47:0/1663523905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bb9e0 0x564f388c8320 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x564f388c2630 tx=0x7f374c003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52a6a2b5296f02b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.198+0000 7f3746a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:48:32.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.200+0000 7f3746a00640 1 -- 172.21.15.47:0/1663523905 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c02d070 con 0x564f388bb9e0 2024-03-21T11:48:32.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.200+0000 7f3746a00640 1 -- 172.21.15.47:0/1663523905 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f374c019050 con 0x564f388bb9e0 2024-03-21T11:48:32.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.200+0000 7f376a57c740 1 -- 172.21.15.47:0/1663523905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f374c0272d0 con 0x564f388bb9e0 2024-03-21T11:48:32.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.203+0000 7f3746a00640 1 -- 172.21.15.47:0/1663523905 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f374c005250 con 0x564f388bb9e0 2024-03-21T11:48:32.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.204+0000 7f376a57c740 10 client.33428 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:32.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.204+0000 7f376a57c740 1 -- 172.21.15.47:0/1663523905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564f3893d170 con 0x564f388bb9e0 2024-03-21T11:48:32.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.204+0000 7f3746a00640 1 -- 172.21.15.47:0/1663523905 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f374c0272d0 con 0x564f388bb9e0 2024-03-21T11:48:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.205+0000 7f3746a00640 1 -- 172.21.15.47:0/1663523905 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f374c01b040 con 0x564f388bb9e0 2024-03-21T11:48:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.205+0000 7f376a57c740 10 client.33428 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.205+0000 7f376a57c740 10 client.33428 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:48:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.205+0000 7f376a57c740 20 client.33428 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.205+0000 7f376a57c740 1 --2- 172.21.15.47:0/1663523905 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x564f3893d9d0 0x564f3893fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.205+0000 7f376a57c740 4 client.33428 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:48:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.205+0000 7f376a57c740 1 -- 172.21.15.47:0/1663523905 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564f38826b50 con 0x564f3893d9d0 2024-03-21T11:48:32.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.206+0000 7f3764e00640 1 --2- 172.21.15.47:0/1663523905 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x564f3893d9d0 0x564f3893fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.207+0000 7f3764e00640 1 --2- 172.21.15.47:0/1663523905 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x564f3893d9d0 0x564f3893fdf0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a1111599cd007d46 server_cookie=1d4693ef141ab16a in_seq=0 out_seq=0 2024-03-21T11:48:32.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.207+0000 7f3746a00640 10 client.33428 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:48:32.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.207+0000 7f3746a00640 1 -- 172.21.15.47:0/1663523905 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564f38826b50 con 0x564f3893d9d0 2024-03-21T11:48:32.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.207+0000 7f3746a00640 10 client.33428 handle_command_reply: tid=0 2024-03-21T11:48:32.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.207+0000 7f376a57c740 1 client.33428 shutdown 2024-03-21T11:48:32.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.208+0000 7f376a57c740 1 -- 172.21.15.47:0/1663523905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bb9e0 msgr2=0x564f388c8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.208+0000 7f376a57c740 1 --2- 172.21.15.47:0/1663523905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bb9e0 0x564f388c8320 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x564f388c2630 tx=0x7f374c003e10 comp rx=0 tx=0).stop 2024-03-21T11:48:32.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.209+0000 7f376a57c740 1 -- 172.21.15.47:0/1663523905 shutdown_connections 2024-03-21T11:48:32.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.209+0000 7f376a57c740 1 --2- 172.21.15.47:0/1663523905 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x564f3893d9d0 0x564f3893fdf0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.209+0000 7f376a57c740 1 --2- 172.21.15.47:0/1663523905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f388bc3a0 0x564f388cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.209+0000 7f376a57c740 1 --2- 172.21.15.47:0/1663523905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f388bb9e0 0x564f388c8320 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.209+0000 7f376a57c740 1 --2- 172.21.15.47:0/1663523905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f388ba810 0x564f388c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.209+0000 7f376a57c740 1 -- 172.21.15.47:0/1663523905 >> 172.21.15.47:0/1663523905 conn(0x564f388b5c00 msgr2=0x564f388e7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.209+0000 7f3746a00640 0 client.33428 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:48:32.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.210+0000 7f376a57c740 1 -- 172.21.15.47:0/1663523905 shutdown_connections 2024-03-21T11:48:32.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.210+0000 7f376a57c740 1 -- 172.21.15.47:0/1663523905 wait complete. 2024-03-21T11:48:32.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.210+0000 7f376a57c740 20 client.33428 trim_cache size 0 max 16384 2024-03-21T11:48:32.210 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:48:32.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.211+0000 7f3767800640 1 -- 172.21.15.47:0/81293352 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f37480a2fa0 msgr2=0x7f37480a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.211+0000 7f3767800640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f37480a2fa0 0x7f37480a5460 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x564f388bb8e0 tx=0x7f374c014040 comp rx=0 tx=0).stop 2024-03-21T11:48:32.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.211+0000 7f3767800640 1 -- 172.21.15.47:0/81293352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f376010e1e0 msgr2=0x7f37601add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.211+0000 7f3767800640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f376010e1e0 0x7f37601add50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3758037040 tx=0x7f3758035d10 comp rx=0 tx=0).stop 2024-03-21T11:48:32.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.212+0000 7f3767800640 1 -- 172.21.15.47:0/81293352 shutdown_connections 2024-03-21T11:48:32.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.212+0000 7f3767800640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f37480a2fa0 0x7f37480a5460 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.212+0000 7f3767800640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f376010e1e0 0x7f37601add50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.212+0000 7f3767800640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f376010b780 0x7f37601a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.212+0000 7f3767800640 1 --2- 172.21.15.47:0/81293352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f376006a4b0 0x7f37601a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.212+0000 7f3767800640 1 -- 172.21.15.47:0/81293352 >> 172.21.15.47:0/81293352 conn(0x7f376006e8a0 msgr2=0x7f3760109700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.212+0000 7f3767800640 1 -- 172.21.15.47:0/81293352 shutdown_connections 2024-03-21T11:48:32.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.212+0000 7f3767800640 1 -- 172.21.15.47:0/81293352 wait complete. 2024-03-21T11:48:32.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:31 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2039154338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:48:32.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:31 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2039154338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:48:32.243 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:48:32.244 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:48:32.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:31 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2039154338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:48:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.513+0000 7f9386000640 1 -- 172.21.15.47:0/1771642608 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937402cc80 con 0x7f9388108eb0 2024-03-21T11:48:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.514+0000 7f938e000640 1 -- 172.21.15.47:0/1771642608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388108eb0 msgr2=0x7f9388109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.514+0000 7f938e000640 1 --2- 172.21.15.47:0/1771642608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388108eb0 0x7f9388109330 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f9374009900 tx=0x7f937402bc30 comp rx=0 tx=0).stop 2024-03-21T11:48:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.515+0000 7f938e000640 1 -- 172.21.15.47:0/1771642608 shutdown_connections 2024-03-21T11:48:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.515+0000 7f938e000640 1 --2- 172.21.15.47:0/1771642608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388109870 0x7f9388110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.515+0000 7f938e000640 1 --2- 172.21.15.47:0/1771642608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388108eb0 0x7f9388109330 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.515+0000 7f938e000640 1 --2- 172.21.15.47:0/1771642608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9388107cc0 0x7f93881080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.515+0000 7f938e000640 1 -- 172.21.15.47:0/1771642608 >> 172.21.15.47:0/1771642608 conn(0x7f9388076c60 msgr2=0x7f93880790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.515+0000 7f938e000640 1 -- 172.21.15.47:0/1771642608 shutdown_connections 2024-03-21T11:48:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.515+0000 7f938e000640 1 -- 172.21.15.47:0/1771642608 wait complete. 2024-03-21T11:48:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.516+0000 7f938e000640 1 Processor -- start 2024-03-21T11:48:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.517+0000 7f938e000640 1 -- start start 2024-03-21T11:48:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.518+0000 7f938e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388107cc0 0x7f938819e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.518+0000 7f938e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388108eb0 0x7f938819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.518+0000 7f938e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9388109870 0x7f93881a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.518+0000 7f938e000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93881139c0 con 0x7f9388109870 2024-03-21T11:48:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.518+0000 7f938e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388113840 con 0x7f9388107cc0 2024-03-21T11:48:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.518+0000 7f938e000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388113b40 con 0x7f9388108eb0 2024-03-21T11:48:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.518+0000 7f9386a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388108eb0 0x7f938819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.518+0000 7f9386a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388108eb0 0x7f938819e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35838/0 (socket says 172.21.15.47:35838) 2024-03-21T11:48:32.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.518+0000 7f9386a00640 1 -- 172.21.15.47:0/2648446440 learned_addr learned my addr 172.21.15.47:0/2648446440 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.519+0000 7f9387e00640 1 --2- 172.21.15.47:0/2648446440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9388109870 0x7f93881a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.519+0000 7f9386a00640 1 -- 172.21.15.47:0/2648446440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388107cc0 msgr2=0x7f938819e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.519+0000 7f9386a00640 1 --2- 172.21.15.47:0/2648446440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388107cc0 0x7f938819e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.519+0000 7f9386a00640 1 -- 172.21.15.47:0/2648446440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9388109870 msgr2=0x7f93881a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.519+0000 7f9386a00640 1 --2- 172.21.15.47:0/2648446440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9388109870 0x7f93881a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.520+0000 7f9386a00640 1 -- 172.21.15.47:0/2648446440 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93881a5c50 con 0x7f9388108eb0 2024-03-21T11:48:32.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.520+0000 7f9387e00640 1 --2- 172.21.15.47:0/2648446440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9388109870 0x7f93881a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:48:32.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.520+0000 7f9386a00640 1 --2- 172.21.15.47:0/2648446440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388108eb0 0x7f938819e570 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f9374009230 tx=0x7f9374005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.521+0000 7f937fe00640 1 -- 172.21.15.47:0/2648446440 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374041070 con 0x7f9388108eb0 2024-03-21T11:48:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.521+0000 7f938e000640 1 -- 172.21.15.47:0/2648446440 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93881a5ee0 con 0x7f9388108eb0 2024-03-21T11:48:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.521+0000 7f937fe00640 1 -- 172.21.15.47:0/2648446440 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93740044d0 con 0x7f9388108eb0 2024-03-21T11:48:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.521+0000 7f937fe00640 1 -- 172.21.15.47:0/2648446440 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374034440 con 0x7f9388108eb0 2024-03-21T11:48:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.521+0000 7f938e000640 1 -- 172.21.15.47:0/2648446440 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93881a6350 con 0x7f9388108eb0 2024-03-21T11:48:32.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.527+0000 7f9390d8e740 1 Processor -- start 2024-03-21T11:48:32.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.528+0000 7f937fe00640 1 -- 172.21.15.47:0/2648446440 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9374044070 con 0x7f9388108eb0 2024-03-21T11:48:32.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.529+0000 7f9390d8e740 1 -- start start 2024-03-21T11:48:32.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.531+0000 7f9390d8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562347449810 0x562347449c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.531+0000 7f9390d8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744a9e0 0x56234744ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.531+0000 7f9390d8e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 0x562347451c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.531+0000 7f9390d8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623473ae2c0 con 0x56234744a9e0 2024-03-21T11:48:32.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.531+0000 7f9390d8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562347356c90 con 0x562347449810 2024-03-21T11:48:32.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.531+0000 7f9390d8e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623473aa360 con 0x56234744b3a0 2024-03-21T11:48:32.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.532+0000 7f937fe00640 1 --2- 172.21.15.47:0/2648446440 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f936c0a3020 0x7f936c0a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.532+0000 7f937fe00640 1 -- 172.21.15.47:0/2648446440 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f93740f14c0 con 0x7f9388108eb0 2024-03-21T11:48:32.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.532+0000 7f9387400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744a9e0 0x56234744ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.532+0000 7f9387400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744a9e0 0x56234744ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57434/0 (socket says 172.21.15.47:57434) 2024-03-21T11:48:32.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.532+0000 7f9387400640 1 -- 172.21.15.47:0/676453457 learned_addr learned my addr 172.21.15.47:0/676453457 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.533+0000 7f9387e00640 1 --2- 172.21.15.47:0/676453457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562347449810 0x562347449c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.533+0000 7f9387400640 1 -- 172.21.15.47:0/676453457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 msgr2=0x562347451c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:48:32.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.533+0000 7f9387400640 1 --2- 172.21.15.47:0/676453457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 0x562347451c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.533+0000 7f9387400640 1 -- 172.21.15.47:0/676453457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562347449810 msgr2=0x562347449c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.533+0000 7f9387400640 1 --2- 172.21.15.47:0/676453457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562347449810 0x562347449c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.533+0000 7f9387400640 1 -- 172.21.15.47:0/676453457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56234737b080 con 0x56234744a9e0 2024-03-21T11:48:32.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.533+0000 7f9387e00640 1 --2- 172.21.15.47:0/2648446440 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f936c0a3020 0x7f936c0a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.534+0000 7f9387400640 1 --2- 172.21.15.47:0/676453457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744a9e0 0x56234744ae60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9378009bf0 tx=0x7f937800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3575087e5755267 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.535+0000 7f9387e00640 1 --2- 172.21.15.47:0/2648446440 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f936c0a3020 0x7f936c0a54e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x56234744a8e0 tx=0x7f938000c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.535+0000 7f9373e00640 1 -- 172.21.15.47:0/676453457 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378005020 con 0x56234744a9e0 2024-03-21T11:48:32.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.535+0000 7f9373e00640 1 -- 172.21.15.47:0/676453457 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93780047a0 con 0x56234744a9e0 2024-03-21T11:48:32.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.536+0000 7f9373e00640 1 -- 172.21.15.47:0/676453457 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378010690 con 0x56234744a9e0 2024-03-21T11:48:32.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.536+0000 7f9390d8e740 1 -- 172.21.15.47:0/676453457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744a9e0 msgr2=0x56234744ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.536+0000 7f9390d8e740 1 --2- 172.21.15.47:0/676453457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744a9e0 0x56234744ae60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9378009bf0 tx=0x7f937800e720 comp rx=0 tx=0).stop 2024-03-21T11:48:32.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.536+0000 7f9390d8e740 1 -- 172.21.15.47:0/676453457 shutdown_connections 2024-03-21T11:48:32.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.536+0000 7f9390d8e740 1 --2- 172.21.15.47:0/676453457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 0x562347451c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.536+0000 7f9390d8e740 1 --2- 172.21.15.47:0/676453457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744a9e0 0x56234744ae60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.536+0000 7f9390d8e740 1 --2- 172.21.15.47:0/676453457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562347449810 0x562347449c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.536+0000 7f9390d8e740 1 -- 172.21.15.47:0/676453457 >> 172.21.15.47:0/676453457 conn(0x562347444c00 msgr2=0x562347445000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.537+0000 7f9390d8e740 1 -- 172.21.15.47:0/676453457 shutdown_connections 2024-03-21T11:48:32.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.537+0000 7f9390d8e740 1 -- 172.21.15.47:0/676453457 wait complete. 2024-03-21T11:48:32.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.537+0000 7f9390d8e740 1 -- start start 2024-03-21T11:48:32.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.538+0000 7f9390d8e740 1 Processor -- start 2024-03-21T11:48:32.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.538+0000 7f9390d8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562347449810 0x562347464ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.538+0000 7f9390d8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56234744a9e0 0x562347465000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.538+0000 7f9390d8e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 0x56234746c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.538+0000 7f9390d8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562347463460 con 0x562347449810 2024-03-21T11:48:32.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.538+0000 7f9390d8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56234736a0d0 con 0x56234744a9e0 2024-03-21T11:48:32.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.538+0000 7f9390d8e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623474635e0 con 0x56234744b3a0 2024-03-21T11:48:32.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.538+0000 7f9390d8e740 10 client.0 _ll_register_callbacks cb 0x56234736ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:32.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.539+0000 7f9386a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56234744a9e0 0x562347465000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.539+0000 7f9386a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56234744a9e0 0x562347465000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60634/0 (socket says 172.21.15.47:60634) 2024-03-21T11:48:32.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.539+0000 7f9386a00640 1 -- 172.21.15.47:0/3277019587 learned_addr learned my addr 172.21.15.47:0/3277019587 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.539+0000 7f9387400640 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562347449810 0x562347464ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.539+0000 7f9387e00640 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 0x56234746c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.539+0000 7f9386a00640 1 -- 172.21.15.47:0/3277019587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 msgr2=0x56234746c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.539+0000 7f9386a00640 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 0x56234746c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.539+0000 7f9386a00640 1 -- 172.21.15.47:0/3277019587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562347449810 msgr2=0x562347464ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.539+0000 7f9386a00640 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562347449810 0x562347464ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.539+0000 7f9386a00640 1 -- 172.21.15.47:0/3277019587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623470c23f0 con 0x56234744a9e0 2024-03-21T11:48:32.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.539+0000 7f9387e00640 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 0x56234746c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:48:32.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.539+0000 7f9387400640 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562347449810 0x562347464ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:48:32.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.540+0000 7f9386a00640 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56234744a9e0 0x562347465000 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f9374006010 tx=0x7f93740bd650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=461990a05866fcc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.540+0000 7f9370c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:48:32.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.540+0000 7f9370c00640 1 -- 172.21.15.47:0/3277019587 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937404b040 con 0x56234744a9e0 2024-03-21T11:48:32.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.540+0000 7f9370c00640 1 -- 172.21.15.47:0/3277019587 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f93740f1190 con 0x56234744a9e0 2024-03-21T11:48:32.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.540+0000 7f9390d8e740 1 -- 172.21.15.47:0/3277019587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93780047a0 con 0x56234744a9e0 2024-03-21T11:48:32.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.541+0000 7f9370c00640 1 -- 172.21.15.47:0/3277019587 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93740bdf00 con 0x56234744a9e0 2024-03-21T11:48:32.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.541+0000 7f9390d8e740 10 client.33437 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:32.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.541+0000 7f9390d8e740 1 -- 172.21.15.47:0/3277019587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56234718c7d0 con 0x56234744a9e0 2024-03-21T11:48:32.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.541+0000 7f9370c00640 1 -- 172.21.15.47:0/3277019587 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93780047a0 con 0x56234744a9e0 2024-03-21T11:48:32.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.541+0000 7f9370c00640 1 -- 172.21.15.47:0/3277019587 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f93740a8070 con 0x56234744a9e0 2024-03-21T11:48:32.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.541+0000 7f9390d8e740 10 client.33437 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:32.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.541+0000 7f9390d8e740 10 client.33437 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:48:32.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.542+0000 7f9390d8e740 20 client.33437 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:32.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.542+0000 7f9390d8e740 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x562347470470 0x5623474728d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.542+0000 7f9390d8e740 4 client.33437 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:48:32.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.542+0000 7f9390d8e740 1 -- 172.21.15.47:0/3277019587 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623473bb800 con 0x562347470470 2024-03-21T11:48:32.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.542+0000 7f9387400640 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x562347470470 0x5623474728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.544+0000 7f9387400640 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x562347470470 0x5623474728d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe4560090036348e server_cookie=45c05faa1a90f349 in_seq=0 out_seq=0 2024-03-21T11:48:32.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.544+0000 7f9370c00640 10 client.33437 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:48:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.546+0000 7f9370c00640 1 -- 172.21.15.47:0/3277019587 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5623473bb800 con 0x562347470470 2024-03-21T11:48:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.546+0000 7f9370c00640 10 client.33437 handle_command_reply: tid=0 2024-03-21T11:48:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.546+0000 7f9390d8e740 1 client.33437 shutdown 2024-03-21T11:48:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.547+0000 7f9390d8e740 1 -- 172.21.15.47:0/3277019587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56234744a9e0 msgr2=0x562347465000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.547+0000 7f9390d8e740 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56234744a9e0 0x562347465000 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f9374006010 tx=0x7f93740bd650 comp rx=0 tx=0).stop 2024-03-21T11:48:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.548+0000 7f9390d8e740 1 -- 172.21.15.47:0/3277019587 shutdown_connections 2024-03-21T11:48:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.548+0000 7f9390d8e740 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x562347470470 0x5623474728d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.548+0000 7f9390d8e740 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 0x56234746c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.548+0000 7f9370c00640 0 client.33437 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:48:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.548+0000 7f9390d8e740 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56234744a9e0 0x562347465000 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.548+0000 7f9390d8e740 1 --2- 172.21.15.47:0/3277019587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562347449810 0x562347464ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.548+0000 7f9390d8e740 1 -- 172.21.15.47:0/3277019587 >> 172.21.15.47:0/3277019587 conn(0x562347444c00 msgr2=0x562347443eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.550+0000 7f9390d8e740 1 -- 172.21.15.47:0/3277019587 shutdown_connections 2024-03-21T11:48:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.550+0000 7f9390d8e740 1 -- 172.21.15.47:0/3277019587 wait complete. 2024-03-21T11:48:32.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.551+0000 7f9390d8e740 20 client.33437 trim_cache size 0 max 16384 2024-03-21T11:48:32.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.576+0000 7f9390d8e740 1 Processor -- start 2024-03-21T11:48:32.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.576+0000 7f9390d8e740 1 -- start start 2024-03-21T11:48:32.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.578+0000 7f9390d8e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562347449810 0x562347443620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.578+0000 7f9390d8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56234744a9e0 0x56234746c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.578+0000 7f9390d8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744b3a0 0x562347443b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.578+0000 7f9390d8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93881a7f30 con 0x56234744b3a0 2024-03-21T11:48:32.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.578+0000 7f9390d8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93881a98b0 con 0x56234744a9e0 2024-03-21T11:48:32.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.578+0000 7f9390d8e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93881a8590 con 0x562347449810 2024-03-21T11:48:32.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.578+0000 7f9386a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56234744a9e0 0x56234746c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.578+0000 7f9386a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56234744a9e0 0x56234746c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60648/0 (socket says 172.21.15.47:60648) 2024-03-21T11:48:32.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.578+0000 7f9386a00640 1 -- 172.21.15.47:0/3356326198 learned_addr learned my addr 172.21.15.47:0/3356326198 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.579+0000 7f9387e00640 1 --2- 172.21.15.47:0/3356326198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744b3a0 0x562347443b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.579+0000 7f9387400640 1 --2- 172.21.15.47:0/3356326198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562347449810 0x562347443620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.579+0000 7f9386a00640 1 -- 172.21.15.47:0/3356326198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562347449810 msgr2=0x562347443620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.579+0000 7f9386a00640 1 --2- 172.21.15.47:0/3356326198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562347449810 0x562347443620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.579+0000 7f9386a00640 1 -- 172.21.15.47:0/3356326198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744b3a0 msgr2=0x562347443b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.579+0000 7f9386a00640 1 --2- 172.21.15.47:0/3356326198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744b3a0 0x562347443b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.579+0000 7f9386a00640 1 -- 172.21.15.47:0/3356326198 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562347444970 con 0x56234744a9e0 2024-03-21T11:48:32.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.579+0000 7f9386a00640 1 --2- 172.21.15.47:0/3356326198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56234744a9e0 0x56234746c000 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f9374000c00 tx=0x7f9374009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f340a045e2b5f1e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.580+0000 7f9373400640 1 -- 172.21.15.47:0/3356326198 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937404b070 con 0x56234744a9e0 2024-03-21T11:48:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.580+0000 7f9373400640 1 -- 172.21.15.47:0/3356326198 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93740d1e50 con 0x56234744a9e0 2024-03-21T11:48:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.580+0000 7f9373400640 1 -- 172.21.15.47:0/3356326198 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93740f1190 con 0x56234744a9e0 2024-03-21T11:48:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.581+0000 7f9390d8e740 1 -- 172.21.15.47:0/3356326198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56234744a9e0 msgr2=0x56234746c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.581+0000 7f9390d8e740 1 --2- 172.21.15.47:0/3356326198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56234744a9e0 0x56234746c000 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f9374000c00 tx=0x7f9374009260 comp rx=0 tx=0).stop 2024-03-21T11:48:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.581+0000 7f9390d8e740 1 -- 172.21.15.47:0/3356326198 shutdown_connections 2024-03-21T11:48:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.581+0000 7f9390d8e740 1 --2- 172.21.15.47:0/3356326198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744b3a0 0x562347443b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.581+0000 7f9390d8e740 1 --2- 172.21.15.47:0/3356326198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56234744a9e0 0x56234746c000 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.581+0000 7f9390d8e740 1 --2- 172.21.15.47:0/3356326198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562347449810 0x562347443620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.581+0000 7f9390d8e740 1 -- 172.21.15.47:0/3356326198 >> 172.21.15.47:0/3356326198 conn(0x562347444c00 msgr2=0x5623474486f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.581+0000 7f9390d8e740 1 -- 172.21.15.47:0/3356326198 shutdown_connections 2024-03-21T11:48:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.581+0000 7f9390d8e740 1 -- 172.21.15.47:0/3356326198 wait complete. 2024-03-21T11:48:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.581+0000 7f9390d8e740 1 -- start start 2024-03-21T11:48:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.582+0000 7f9390d8e740 1 Processor -- start 2024-03-21T11:48:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.582+0000 7f9390d8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562347449810 0x56234746b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.582+0000 7f9390d8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744a9e0 0x56234746bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.582+0000 7f9390d8e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 0x562347469e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.582+0000 7f9390d8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93881a9250 con 0x56234744a9e0 2024-03-21T11:48:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.582+0000 7f9390d8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93881a8bf0 con 0x562347449810 2024-03-21T11:48:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.582+0000 7f9390d8e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388082260 con 0x56234744b3a0 2024-03-21T11:48:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.582+0000 7f9390d8e740 10 client.0 _ll_register_callbacks cb 0x7f9374005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.583+0000 7f9387400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562347449810 0x56234746b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.583+0000 7f9387400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562347449810 0x56234746b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60662/0 (socket says 172.21.15.47:60662) 2024-03-21T11:48:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.583+0000 7f9387400640 1 -- 172.21.15.47:0/2023323954 learned_addr learned my addr 172.21.15.47:0/2023323954 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.583+0000 7f9387e00640 1 --2- 172.21.15.47:0/2023323954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 0x562347469e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.583+0000 7f9387400640 1 -- 172.21.15.47:0/2023323954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 msgr2=0x562347469e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.583+0000 7f9387400640 1 --2- 172.21.15.47:0/2023323954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 0x562347469e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.583+0000 7f9387400640 1 -- 172.21.15.47:0/2023323954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744a9e0 msgr2=0x56234746bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.583+0000 7f9387400640 1 --2- 172.21.15.47:0/2023323954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744a9e0 0x56234746bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.583+0000 7f9387400640 1 -- 172.21.15.47:0/2023323954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56234746a460 con 0x562347449810 2024-03-21T11:48:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.583+0000 7f9387400640 1 --2- 172.21.15.47:0/2023323954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562347449810 0x56234746b7c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f9378009230 tx=0x7f9378005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31cbe963bd605949 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.583+0000 7f9370c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:48:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.584+0000 7f9370c00640 1 -- 172.21.15.47:0/2023323954 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937801d040 con 0x562347449810 2024-03-21T11:48:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.584+0000 7f9390d8e740 1 -- 172.21.15.47:0/2023323954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93740d1e50 con 0x562347449810 2024-03-21T11:48:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.584+0000 7f9370c00640 1 -- 172.21.15.47:0/2023323954 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9378009530 con 0x562347449810 2024-03-21T11:48:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.584+0000 7f9370c00640 1 -- 172.21.15.47:0/2023323954 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9378038050 con 0x562347449810 2024-03-21T11:48:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.584+0000 7f9390d8e740 10 client.33443 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.584+0000 7f9390d8e740 1 -- 172.21.15.47:0/2023323954 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5623474cc7c0 con 0x562347449810 2024-03-21T11:48:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.585+0000 7f9370c00640 1 -- 172.21.15.47:0/2023323954 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93740d1e50 con 0x562347449810 2024-03-21T11:48:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.585+0000 7f9370c00640 1 -- 172.21.15.47:0/2023323954 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f93780053a0 con 0x562347449810 2024-03-21T11:48:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.585+0000 7f9390d8e740 10 client.33443 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.585+0000 7f9390d8e740 10 client.33443 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:48:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.585+0000 7f9390d8e740 20 client.33443 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.585+0000 7f9390d8e740 1 --2- 172.21.15.47:0/2023323954 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5623474ccdf0 0x5623474cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.585+0000 7f9390d8e740 4 client.33443 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:48:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.585+0000 7f9390d8e740 1 -- 172.21.15.47:0/2023323954 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5623474648b0 con 0x5623474ccdf0 2024-03-21T11:48:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.586+0000 7f9386a00640 1 --2- 172.21.15.47:0/2023323954 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5623474ccdf0 0x5623474cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.587+0000 7f9386a00640 1 --2- 172.21.15.47:0/2023323954 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5623474ccdf0 0x5623474cf210 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7773af773a3ca1d1 server_cookie=390107c581470d5c in_seq=0 out_seq=0 2024-03-21T11:48:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.587+0000 7f9370c00640 10 client.33443 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:48:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.600+0000 7f9370c00640 1 -- 172.21.15.47:0/2023323954 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5623474648b0 con 0x5623474ccdf0 2024-03-21T11:48:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.600+0000 7f9370c00640 10 client.33443 handle_command_reply: tid=0 2024-03-21T11:48:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.600+0000 7f9390d8e740 1 client.33443 shutdown 2024-03-21T11:48:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.600+0000 7f9390d8e740 1 -- 172.21.15.47:0/2023323954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562347449810 msgr2=0x56234746b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.600+0000 7f9390d8e740 1 --2- 172.21.15.47:0/2023323954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562347449810 0x56234746b7c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f9378009230 tx=0x7f9378005f10 comp rx=0 tx=0).stop 2024-03-21T11:48:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.600+0000 7f9390d8e740 1 -- 172.21.15.47:0/2023323954 shutdown_connections 2024-03-21T11:48:32.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.600+0000 7f9390d8e740 1 --2- 172.21.15.47:0/2023323954 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x5623474ccdf0 0x5623474cf210 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.600+0000 7f9390d8e740 1 --2- 172.21.15.47:0/2023323954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56234744b3a0 0x562347469e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.600+0000 7f9390d8e740 1 --2- 172.21.15.47:0/2023323954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56234744a9e0 0x56234746bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.600+0000 7f9390d8e740 1 --2- 172.21.15.47:0/2023323954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562347449810 0x56234746b7c0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.600+0000 7f9370c00640 0 client.33443 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:48:32.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.600+0000 7f9390d8e740 1 -- 172.21.15.47:0/2023323954 >> 172.21.15.47:0/2023323954 conn(0x562347444c00 msgr2=0x562347443620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.600+0000 7f9390d8e740 1 -- 172.21.15.47:0/2023323954 shutdown_connections 2024-03-21T11:48:32.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.600+0000 7f9390d8e740 1 -- 172.21.15.47:0/2023323954 wait complete. 2024-03-21T11:48:32.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.601+0000 7f9390d8e740 20 client.33443 trim_cache size 0 max 16384 2024-03-21T11:48:32.602 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:48:32.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.601+0000 7f938e000640 1 -- 172.21.15.47:0/2648446440 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f936c0a3020 msgr2=0x7f936c0a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.601+0000 7f938e000640 1 --2- 172.21.15.47:0/2648446440 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f936c0a3020 0x7f936c0a54e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x56234744a8e0 tx=0x7f938000c040 comp rx=0 tx=0).stop 2024-03-21T11:48:32.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.601+0000 7f938e000640 1 -- 172.21.15.47:0/2648446440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388108eb0 msgr2=0x7f938819e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.601+0000 7f938e000640 1 --2- 172.21.15.47:0/2648446440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388108eb0 0x7f938819e570 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f9374009230 tx=0x7f9374005d50 comp rx=0 tx=0).stop 2024-03-21T11:48:32.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.602+0000 7f938e000640 1 -- 172.21.15.47:0/2648446440 shutdown_connections 2024-03-21T11:48:32.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.602+0000 7f938e000640 1 --2- 172.21.15.47:0/2648446440 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f936c0a3020 0x7f936c0a54e0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.602+0000 7f938e000640 1 --2- 172.21.15.47:0/2648446440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9388109870 0x7f93881a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.602+0000 7f938e000640 1 --2- 172.21.15.47:0/2648446440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388108eb0 0x7f938819e570 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.602+0000 7f938e000640 1 --2- 172.21.15.47:0/2648446440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9388107cc0 0x7f938819e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.602+0000 7f938e000640 1 -- 172.21.15.47:0/2648446440 >> 172.21.15.47:0/2648446440 conn(0x7f9388076c60 msgr2=0x7f938810c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.602+0000 7f938e000640 1 -- 172.21.15.47:0/2648446440 shutdown_connections 2024-03-21T11:48:32.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.602+0000 7f938e000640 1 -- 172.21.15.47:0/2648446440 wait complete. 2024-03-21T11:48:32.622 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:48:32.622 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:48:32.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.901+0000 7f1ff2400640 1 -- 172.21.15.47:0/236380596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec103260 msgr2=0x7f1fec109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.901+0000 7f1ff2400640 1 --2- 172.21.15.47:0/236380596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec103260 0x7f1fec109af0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009ca0 tx=0x7f1fe402c100 comp rx=0 tx=0).stop 2024-03-21T11:48:32.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.901+0000 7f1ff2400640 1 -- 172.21.15.47:0/236380596 shutdown_connections 2024-03-21T11:48:32.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.901+0000 7f1ff2400640 1 --2- 172.21.15.47:0/236380596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec103260 0x7f1fec109af0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.901+0000 7f1ff2400640 1 --2- 172.21.15.47:0/236380596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fec1028a0 0x7f1fec102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.901+0000 7f1ff2400640 1 --2- 172.21.15.47:0/236380596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fec1016b0 0x7f1fec101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.901+0000 7f1ff2400640 1 -- 172.21.15.47:0/236380596 >> 172.21.15.47:0/236380596 conn(0x7f1fec0fcc40 msgr2=0x7f1fec0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.902+0000 7f1ff2400640 1 -- 172.21.15.47:0/236380596 shutdown_connections 2024-03-21T11:48:32.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.902+0000 7f1ff2400640 1 -- 172.21.15.47:0/236380596 wait complete. 2024-03-21T11:48:32.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.903+0000 7f1ff2400640 1 Processor -- start 2024-03-21T11:48:32.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.903+0000 7f1ff2400640 1 -- start start 2024-03-21T11:48:32.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.903+0000 7f1ff2400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fec1016b0 0x7f1fec19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.903+0000 7f1ff2400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec1028a0 0x7f1fec19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.903+0000 7f1ff2400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fec103260 0x7f1fec1a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.903+0000 7f1ff2400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec10d330 con 0x7f1fec1016b0 2024-03-21T11:48:32.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.903+0000 7f1ff2400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec10d1b0 con 0x7f1fec1028a0 2024-03-21T11:48:32.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.903+0000 7f1ff2400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec10d4b0 con 0x7f1fec103260 2024-03-21T11:48:32.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.904+0000 7f1feaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec1028a0 0x7f1fec19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.904+0000 7f1feb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fec1016b0 0x7f1fec19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.904+0000 7f1feaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec1028a0 0x7f1fec19e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60690/0 (socket says 172.21.15.47:60690) 2024-03-21T11:48:32.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.904+0000 7f1feaa00640 1 -- 172.21.15.47:0/1815011869 learned_addr learned my addr 172.21.15.47:0/1815011869 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.904+0000 7f1febe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fec103260 0x7f1fec1a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.904+0000 7f1feaa00640 1 -- 172.21.15.47:0/1815011869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fec103260 msgr2=0x7f1fec1a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.904+0000 7f1feaa00640 1 --2- 172.21.15.47:0/1815011869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fec103260 0x7f1fec1a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.904+0000 7f1feaa00640 1 -- 172.21.15.47:0/1815011869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fec1016b0 msgr2=0x7f1fec19dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.904+0000 7f1feaa00640 1 --2- 172.21.15.47:0/1815011869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fec1016b0 0x7f1fec19dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.904+0000 7f1feaa00640 1 -- 172.21.15.47:0/1815011869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec1a5bc0 con 0x7f1fec1028a0 2024-03-21T11:48:32.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.904+0000 7f1feaa00640 1 --2- 172.21.15.47:0/1815011869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec1028a0 0x7f1fec19e4e0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800d950 tx=0x7f1fd800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.905+0000 7f1fe3e00640 1 -- 172.21.15.47:0/1815011869 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8017070 con 0x7f1fec1028a0 2024-03-21T11:48:32.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.905+0000 7f1ff2400640 1 -- 172.21.15.47:0/1815011869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fec1a5eb0 con 0x7f1fec1028a0 2024-03-21T11:48:32.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.905+0000 7f1ff2400640 1 -- 172.21.15.47:0/1815011869 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fec1a63c0 con 0x7f1fec1028a0 2024-03-21T11:48:32.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.905+0000 7f1fe3e00640 1 -- 172.21.15.47:0/1815011869 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fd8009c30 con 0x7f1fec1028a0 2024-03-21T11:48:32.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.905+0000 7f1fe3e00640 1 -- 172.21.15.47:0/1815011869 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd800f6b0 con 0x7f1fec1028a0 2024-03-21T11:48:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.910+0000 7f1ff5173740 1 Processor -- start 2024-03-21T11:48:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.910+0000 7f1ff5173740 1 -- start start 2024-03-21T11:48:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.911+0000 7f1ff5173740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef7820 0x557119ef7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.911+0000 7f1ff5173740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.911+0000 7f1ff5173740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef93b0 0x557119effc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.911+0000 7f1ff5173740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557119e4b2c0 con 0x557119ef93b0 2024-03-21T11:48:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.911+0000 7f1ff5173740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557119df3c90 con 0x557119ef7820 2024-03-21T11:48:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.911+0000 7f1ff5173740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557119e47360 con 0x557119ef89f0 2024-03-21T11:48:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.911+0000 7f1feb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.911+0000 7f1feb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35906/0 (socket says 172.21.15.47:35906) 2024-03-21T11:48:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.911+0000 7f1feb400640 1 -- 172.21.15.47:0/952284138 learned_addr learned my addr 172.21.15.47:0/952284138 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.912+0000 7f1febe00640 1 --2- 172.21.15.47:0/952284138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef7820 0x557119ef7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.914+0000 7f1feb400640 1 -- 172.21.15.47:0/952284138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef7820 msgr2=0x557119ef7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.914+0000 7f1feb400640 1 --2- 172.21.15.47:0/952284138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef7820 0x557119ef7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.914+0000 7f1feb400640 1 -- 172.21.15.47:0/952284138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef93b0 msgr2=0x557119effc40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:48:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.914+0000 7f1feb400640 1 --2- 172.21.15.47:0/952284138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef93b0 0x557119effc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.914+0000 7f1feb400640 1 -- 172.21.15.47:0/952284138 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557119e18080 con 0x557119ef89f0 2024-03-21T11:48:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.915+0000 7f1fe3e00640 1 -- 172.21.15.47:0/1815011869 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1fd800f850 con 0x7f1fec1028a0 2024-03-21T11:48:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.915+0000 7f1feb400640 1 --2- 172.21.15.47:0/952284138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef8e70 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00bcb0 tx=0x7f1fdc0327b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50618728245be29c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.918+0000 7f1fe3e00640 1 --2- 172.21.15.47:0/1815011869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1fd00a3020 0x7f1fd00a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.918+0000 7f1fe3e00640 1 -- 172.21.15.47:0/1815011869 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1fd80cf230 con 0x7f1fec1028a0 2024-03-21T11:48:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.918+0000 7f1fe0c00640 1 -- 172.21.15.47:0/952284138 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc005020 con 0x557119ef89f0 2024-03-21T11:48:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.918+0000 7f1fe0c00640 1 -- 172.21.15.47:0/952284138 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc0047a0 con 0x557119ef89f0 2024-03-21T11:48:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.918+0000 7f1fe0c00640 1 -- 172.21.15.47:0/952284138 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc033810 con 0x557119ef89f0 2024-03-21T11:48:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.919+0000 7f1ff5173740 1 -- 172.21.15.47:0/952284138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 msgr2=0x557119ef8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.919+0000 7f1ff5173740 1 --2- 172.21.15.47:0/952284138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef8e70 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00bcb0 tx=0x7f1fdc0327b0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.920+0000 7f1febe00640 1 --2- 172.21.15.47:0/1815011869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1fd00a3020 0x7f1fd00a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.921+0000 7f1ff5173740 1 -- 172.21.15.47:0/952284138 shutdown_connections 2024-03-21T11:48:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.921+0000 7f1ff5173740 1 --2- 172.21.15.47:0/952284138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef93b0 0x557119effc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.921+0000 7f1ff5173740 1 --2- 172.21.15.47:0/952284138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef8e70 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.921+0000 7f1ff5173740 1 --2- 172.21.15.47:0/952284138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef7820 0x557119ef7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.921+0000 7f1ff5173740 1 -- 172.21.15.47:0/952284138 >> 172.21.15.47:0/952284138 conn(0x557119ee1c00 msgr2=0x557119ee2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.922+0000 7f1ff5173740 1 -- 172.21.15.47:0/952284138 shutdown_connections 2024-03-21T11:48:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.923+0000 7f1febe00640 1 --2- 172.21.15.47:0/1815011869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1fd00a3020 0x7f1fd00a54e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4005e30 tx=0x7f1fe4032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.923+0000 7f1ff5173740 1 -- 172.21.15.47:0/952284138 wait complete. 2024-03-21T11:48:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.923+0000 7f1ff5173740 1 -- start start 2024-03-21T11:48:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.924+0000 7f1ff5173740 1 Processor -- start 2024-03-21T11:48:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.925+0000 7f1ff5173740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef7820 0x557119eee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.925+0000 7f1ff5173740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef89f0 0x557119eee7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.926+0000 7f1ff5173740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef93b0 0x557119ef5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.926+0000 7f1ff5173740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557119ee6570 con 0x557119ef89f0 2024-03-21T11:48:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.926+0000 7f1ff5173740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557119e070d0 con 0x557119ef7820 2024-03-21T11:48:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.926+0000 7f1ff5173740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557119ee66f0 con 0x557119ef93b0 2024-03-21T11:48:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.926+0000 7f1ff5173740 10 client.0 _ll_register_callbacks cb 0x557119e0be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.926+0000 7f1feaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef89f0 0x557119eee7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.926+0000 7f1feaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef89f0 0x557119eee7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57492/0 (socket says 172.21.15.47:57492) 2024-03-21T11:48:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.926+0000 7f1feaa00640 1 -- 172.21.15.47:0/3442525687 learned_addr learned my addr 172.21.15.47:0/3442525687 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.927+0000 7f1feb400640 1 --2- 172.21.15.47:0/3442525687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef7820 0x557119eee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.927+0000 7f1feaa00640 1 -- 172.21.15.47:0/3442525687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef93b0 msgr2=0x557119ef5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.927+0000 7f1feaa00640 1 --2- 172.21.15.47:0/3442525687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef93b0 0x557119ef5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.927+0000 7f1feaa00640 1 -- 172.21.15.47:0/3442525687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef7820 msgr2=0x557119eee2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.927+0000 7f1feaa00640 1 --2- 172.21.15.47:0/3442525687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef7820 0x557119eee2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.927+0000 7f1feaa00640 1 -- 172.21.15.47:0/3442525687 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557119b5f3f0 con 0x557119ef89f0 2024-03-21T11:48:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.927+0000 7f1feaa00640 1 --2- 172.21.15.47:0/3442525687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef89f0 0x557119eee7e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80a3a50 tx=0x7f1fd809b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b3e1df709773c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.928+0000 7f1fd5600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:48:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.928+0000 7f1fd5600640 1 -- 172.21.15.47:0/3442525687 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8020070 con 0x557119ef89f0 2024-03-21T11:48:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.928+0000 7f1fd5600640 1 -- 172.21.15.47:0/3442525687 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1fd8097c90 con 0x557119ef89f0 2024-03-21T11:48:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.928+0000 7f1ff5173740 1 -- 172.21.15.47:0/3442525687 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fdc0047a0 con 0x557119ef89f0 2024-03-21T11:48:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.929+0000 7f1fd5600640 1 -- 172.21.15.47:0/3442525687 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fd809f6c0 con 0x557119ef89f0 2024-03-21T11:48:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.929+0000 7f1ff5173740 10 client.22554 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.929+0000 7f1ff5173740 1 -- 172.21.15.47:0/3442525687 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x557119c297d0 con 0x557119ef89f0 2024-03-21T11:48:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.929+0000 7f1fd5600640 1 -- 172.21.15.47:0/3442525687 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc0047a0 con 0x557119ef89f0 2024-03-21T11:48:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.929+0000 7f1fd5600640 1 -- 172.21.15.47:0/3442525687 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1fd8084c40 con 0x557119ef89f0 2024-03-21T11:48:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.929+0000 7f1ff5173740 10 client.22554 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.929+0000 7f1ff5173740 10 client.22554 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:48:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.929+0000 7f1ff5173740 20 client.22554 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.929+0000 7f1ff5173740 1 --2- 172.21.15.47:0/3442525687 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557119efc480 0x557119efe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.930+0000 7f1ff5173740 4 client.22554 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:48:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.930+0000 7f1ff5173740 1 -- 172.21.15.47:0/3442525687 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557119e58800 con 0x557119efc480 2024-03-21T11:48:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.930+0000 7f1feb400640 1 --2- 172.21.15.47:0/3442525687 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557119efc480 0x557119efe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.937+0000 7f1fd5600640 10 client.22554 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:48:32.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.937+0000 7f1feb400640 1 --2- 172.21.15.47:0/3442525687 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557119efc480 0x557119efe8b0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c53e47ee62d2ef9b server_cookie=727e615444581d9d in_seq=0 out_seq=0 2024-03-21T11:48:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.938+0000 7f1fd5600640 1 -- 172.21.15.47:0/3442525687 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557119e58800 con 0x557119efc480 2024-03-21T11:48:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.938+0000 7f1fd5600640 10 client.22554 handle_command_reply: tid=0 2024-03-21T11:48:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.939+0000 7f1ff5173740 1 client.22554 shutdown 2024-03-21T11:48:32.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.939+0000 7f1ff5173740 1 -- 172.21.15.47:0/3442525687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef89f0 msgr2=0x557119eee7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.939+0000 7f1ff5173740 1 --2- 172.21.15.47:0/3442525687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef89f0 0x557119eee7e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80a3a50 tx=0x7f1fd809b040 comp rx=0 tx=0).stop 2024-03-21T11:48:32.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.939+0000 7f1ff5173740 1 -- 172.21.15.47:0/3442525687 shutdown_connections 2024-03-21T11:48:32.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.939+0000 7f1ff5173740 1 --2- 172.21.15.47:0/3442525687 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557119efc480 0x557119efe8b0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.940+0000 7f1fd5600640 0 client.22554 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:48:32.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.940+0000 7f1ff5173740 1 --2- 172.21.15.47:0/3442525687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef93b0 0x557119ef5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.940+0000 7f1ff5173740 1 --2- 172.21.15.47:0/3442525687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef89f0 0x557119eee7e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.940+0000 7f1ff5173740 1 --2- 172.21.15.47:0/3442525687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef7820 0x557119eee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.940+0000 7f1ff5173740 1 -- 172.21.15.47:0/3442525687 >> 172.21.15.47:0/3442525687 conn(0x557119ee1c00 msgr2=0x557119ee0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.940+0000 7f1ff5173740 1 -- 172.21.15.47:0/3442525687 shutdown_connections 2024-03-21T11:48:32.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.940+0000 7f1ff5173740 1 -- 172.21.15.47:0/3442525687 wait complete. 2024-03-21T11:48:32.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.940+0000 7f1ff5173740 20 client.22554 trim_cache size 0 max 16384 2024-03-21T11:48:32.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.968+0000 7f1ff5173740 1 Processor -- start 2024-03-21T11:48:32.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.968+0000 7f1ff5173740 1 -- start start 2024-03-21T11:48:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.969+0000 7f1ff5173740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef7820 0x557119ee8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.969+0000 7f1ff5173740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.969+0000 7f1ff5173740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef93b0 0x557119ee8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.969+0000 7f1ff5173740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc00f970 con 0x557119ef7820 2024-03-21T11:48:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.969+0000 7f1ff5173740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0112f0 con 0x557119ef93b0 2024-03-21T11:48:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.969+0000 7f1ff5173740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc00ffd0 con 0x557119ef89f0 2024-03-21T11:48:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.970+0000 7f1feaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.970+0000 7f1feaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35928/0 (socket says 172.21.15.47:35928) 2024-03-21T11:48:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.970+0000 7f1feaa00640 1 -- 172.21.15.47:0/1290800575 learned_addr learned my addr 172.21.15.47:0/1290800575 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.970+0000 7f1feaa00640 1 -- 172.21.15.47:0/1290800575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef93b0 msgr2=0x557119ee8fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:48:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.970+0000 7f1feaa00640 1 --2- 172.21.15.47:0/1290800575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef93b0 0x557119ee8fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.970+0000 7f1feaa00640 1 -- 172.21.15.47:0/1290800575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef7820 msgr2=0x557119ee8a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:48:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.970+0000 7f1feaa00640 1 --2- 172.21.15.47:0/1290800575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef7820 0x557119ee8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.971+0000 7f1feaa00640 1 -- 172.21.15.47:0/1290800575 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557119ee2970 con 0x557119ef89f0 2024-03-21T11:48:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.971+0000 7f1feaa00640 1 --2- 172.21.15.47:0/1290800575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef57e0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8097a80 tx=0x7f1fd809b0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0aa724109bfb051 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.972+0000 7f1fd7e00640 1 -- 172.21.15.47:0/1290800575 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80b9070 con 0x557119ef89f0 2024-03-21T11:48:32.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.972+0000 7f1fd7e00640 1 -- 172.21.15.47:0/1290800575 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fd8097c90 con 0x557119ef89f0 2024-03-21T11:48:32.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.972+0000 7f1fd7e00640 1 -- 172.21.15.47:0/1290800575 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8020070 con 0x557119ef89f0 2024-03-21T11:48:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.975+0000 7f1ff5173740 1 -- 172.21.15.47:0/1290800575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 msgr2=0x557119ef57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.975+0000 7f1ff5173740 1 --2- 172.21.15.47:0/1290800575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef57e0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8097a80 tx=0x7f1fd809b0b0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.976+0000 7f1ff5173740 1 -- 172.21.15.47:0/1290800575 shutdown_connections 2024-03-21T11:48:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.976+0000 7f1ff5173740 1 --2- 172.21.15.47:0/1290800575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef93b0 0x557119ee8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.976+0000 7f1ff5173740 1 --2- 172.21.15.47:0/1290800575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef57e0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.976+0000 7f1ff5173740 1 --2- 172.21.15.47:0/1290800575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef7820 0x557119ee8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.976+0000 7f1ff5173740 1 -- 172.21.15.47:0/1290800575 >> 172.21.15.47:0/1290800575 conn(0x557119ee1c00 msgr2=0x557119ee56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.976+0000 7f1ff5173740 1 -- 172.21.15.47:0/1290800575 shutdown_connections 2024-03-21T11:48:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.976+0000 7f1ff5173740 1 -- 172.21.15.47:0/1290800575 wait complete. 2024-03-21T11:48:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.977+0000 7f1ff5173740 1 -- start start 2024-03-21T11:48:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.977+0000 7f1ff5173740 1 Processor -- start 2024-03-21T11:48:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.978+0000 7f1ff5173740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef7820 0x557119ef1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.978+0000 7f1ff5173740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.978+0000 7f1ff5173740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef93b0 0x557119ef49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.978+0000 7f1ff5173740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc010c90 con 0x557119ef7820 2024-03-21T11:48:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.978+0000 7f1ff5173740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc010630 con 0x557119ef93b0 2024-03-21T11:48:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.978+0000 7f1ff5173740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc00db30 con 0x557119ef89f0 2024-03-21T11:48:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.978+0000 7f1ff5173740 10 client.0 _ll_register_callbacks cb 0x7f1fd809b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.979+0000 7f1feb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef7820 0x557119ef1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.979+0000 7f1feb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef7820 0x557119ef1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57520/0 (socket says 172.21.15.47:57520) 2024-03-21T11:48:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.979+0000 7f1feb400640 1 -- 172.21.15.47:0/980043456 learned_addr learned my addr 172.21.15.47:0/980043456 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.979+0000 7f1febe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef93b0 0x557119ef49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.979+0000 7f1feaa00640 1 --2- 172.21.15.47:0/980043456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.979+0000 7f1feb400640 1 -- 172.21.15.47:0/980043456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 msgr2=0x557119ef1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.979+0000 7f1feb400640 1 --2- 172.21.15.47:0/980043456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.979+0000 7f1feb400640 1 -- 172.21.15.47:0/980043456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef93b0 msgr2=0x557119ef49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.980+0000 7f1feb400640 1 --2- 172.21.15.47:0/980043456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef93b0 0x557119ef49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.980+0000 7f1feb400640 1 -- 172.21.15.47:0/980043456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557119ef4fd0 con 0x557119ef7820 2024-03-21T11:48:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.980+0000 7f1febe00640 1 --2- 172.21.15.47:0/980043456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef93b0 0x557119ef49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:48:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.981+0000 7f1feb400640 1 --2- 172.21.15.47:0/980043456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef7820 0x557119ef1540 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x557119ee2b70 tx=0x7f1fdc0046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a898b73f61ed30fc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.981+0000 7f1fd5600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:48:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.982+0000 7f1fd5600640 1 -- 172.21.15.47:0/980043456 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc0049a0 con 0x557119ef7820 2024-03-21T11:48:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.982+0000 7f1fd5600640 1 -- 172.21.15.47:0/980043456 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1fdc004b40 con 0x557119ef7820 2024-03-21T11:48:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.982+0000 7f1ff5173740 1 -- 172.21.15.47:0/980043456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fd8097c90 con 0x557119ef7820 2024-03-21T11:48:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.983+0000 7f1fd5600640 1 -- 172.21.15.47:0/980043456 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fdc004fb0 con 0x557119ef7820 2024-03-21T11:48:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.983+0000 7f1ff5173740 10 client.22557 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.983+0000 7f1ff5173740 1 -- 172.21.15.47:0/980043456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x557119f583b0 con 0x557119ef7820 2024-03-21T11:48:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.984+0000 7f1fd5600640 1 -- 172.21.15.47:0/980043456 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fd8097c90 con 0x557119ef7820 2024-03-21T11:48:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.984+0000 7f1fd5600640 1 -- 172.21.15.47:0/980043456 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1fdc04aa10 con 0x557119ef7820 2024-03-21T11:48:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.984+0000 7f1ff5173740 10 client.22557 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.984+0000 7f1ff5173740 10 client.22557 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:48:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.984+0000 7f1ff5173740 20 client.22557 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.984+0000 7f1ff5173740 1 --2- 172.21.15.47:0/980043456 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557119efc480 0x557119f5ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.984+0000 7f1ff5173740 4 client.22557 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:48:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.984+0000 7f1ff5173740 1 -- 172.21.15.47:0/980043456 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557119e52b50 con 0x557119efc480 2024-03-21T11:48:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.985+0000 7f1feaa00640 1 --2- 172.21.15.47:0/980043456 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557119efc480 0x557119f5ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.985+0000 7f1feaa00640 1 --2- 172.21.15.47:0/980043456 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557119efc480 0x557119f5ab40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72f7287e352c2e3d server_cookie=23b0b0bdbb857c3 in_seq=0 out_seq=0 2024-03-21T11:48:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.986+0000 7f1fd5600640 10 client.22557 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:48:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.986+0000 7f1fd5600640 1 -- 172.21.15.47:0/980043456 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557119e52b50 con 0x557119efc480 2024-03-21T11:48:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.986+0000 7f1fd5600640 10 client.22557 handle_command_reply: tid=0 2024-03-21T11:48:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.986+0000 7f1ff5173740 1 client.22557 shutdown 2024-03-21T11:48:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.986+0000 7f1ff5173740 1 -- 172.21.15.47:0/980043456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef7820 msgr2=0x557119ef1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.986+0000 7f1ff5173740 1 --2- 172.21.15.47:0/980043456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef7820 0x557119ef1540 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x557119ee2b70 tx=0x7f1fdc0046f0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.987+0000 7f1ff5173740 1 -- 172.21.15.47:0/980043456 shutdown_connections 2024-03-21T11:48:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.987+0000 7f1ff5173740 1 --2- 172.21.15.47:0/980043456 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x557119efc480 0x557119f5ab40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.987+0000 7f1ff5173740 1 --2- 172.21.15.47:0/980043456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557119ef93b0 0x557119ef49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.987+0000 7f1ff5173740 1 --2- 172.21.15.47:0/980043456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557119ef89f0 0x557119ef1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.987+0000 7f1ff5173740 1 --2- 172.21.15.47:0/980043456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557119ef7820 0x557119ef1540 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.987+0000 7f1ff5173740 1 -- 172.21.15.47:0/980043456 >> 172.21.15.47:0/980043456 conn(0x557119ee1c00 msgr2=0x557119ee0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.987+0000 7f1fd5600640 0 client.22557 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:48:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.987+0000 7f1ff5173740 1 -- 172.21.15.47:0/980043456 shutdown_connections 2024-03-21T11:48:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.987+0000 7f1ff5173740 1 -- 172.21.15.47:0/980043456 wait complete. 2024-03-21T11:48:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.987+0000 7f1ff5173740 20 client.22557 trim_cache size 0 max 16384 2024-03-21T11:48:32.988 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:48:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.989+0000 7f1ff2400640 1 -- 172.21.15.47:0/1815011869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1fd00a3020 msgr2=0x7f1fd00a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.989+0000 7f1ff2400640 1 --2- 172.21.15.47:0/1815011869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1fd00a3020 0x7f1fd00a54e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4005e30 tx=0x7f1fe4032590 comp rx=0 tx=0).stop 2024-03-21T11:48:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.989+0000 7f1ff2400640 1 -- 172.21.15.47:0/1815011869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec1028a0 msgr2=0x7f1fec19e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.989+0000 7f1ff2400640 1 --2- 172.21.15.47:0/1815011869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec1028a0 0x7f1fec19e4e0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800d950 tx=0x7f1fd800de10 comp rx=0 tx=0).stop 2024-03-21T11:48:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.989+0000 7f1ff2400640 1 -- 172.21.15.47:0/1815011869 shutdown_connections 2024-03-21T11:48:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.989+0000 7f1ff2400640 1 --2- 172.21.15.47:0/1815011869 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1fd00a3020 0x7f1fd00a54e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.989+0000 7f1ff2400640 1 --2- 172.21.15.47:0/1815011869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fec103260 0x7f1fec1a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.989+0000 7f1ff2400640 1 --2- 172.21.15.47:0/1815011869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fec1028a0 0x7f1fec19e4e0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.989+0000 7f1ff2400640 1 --2- 172.21.15.47:0/1815011869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fec1016b0 0x7f1fec19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:32.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.989+0000 7f1ff2400640 1 -- 172.21.15.47:0/1815011869 >> 172.21.15.47:0/1815011869 conn(0x7f1fec0fcc40 msgr2=0x7f1fec0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:32.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.990+0000 7f1ff2400640 1 -- 172.21.15.47:0/1815011869 shutdown_connections 2024-03-21T11:48:32.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:32.990+0000 7f1ff2400640 1 -- 172.21.15.47:0/1815011869 wait complete. 2024-03-21T11:48:33.001 INFO:tasks.workunit.client.0.smithi047.stdout: 10 278993 21.39 MB/sec execute 556 sec latency 133.940 ms 2024-03-21T11:48:33.016 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:48:33.016 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:48:33.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:32 smithi102 ceph-mon[36748]: pgmap v1157: 129 pgs: 129 active+clean; 792 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 12 MiB/s wr, 567 op/s 2024-03-21T11:48:33.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.270+0000 7f6ef9800640 1 -- 172.21.15.47:0/4241836069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef4105230 msgr2=0x7f6ef41056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:33.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.270+0000 7f6ef9800640 1 --2- 172.21.15.47:0/4241836069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef4105230 0x7f6ef41056b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009be0 tx=0x7f6eec02bef0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.271+0000 7f6ef9800640 1 -- 172.21.15.47:0/4241836069 shutdown_connections 2024-03-21T11:48:33.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.271+0000 7f6ef9800640 1 --2- 172.21.15.47:0/4241836069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef4105230 0x7f6ef41056b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.271+0000 7f6ef9800640 1 --2- 172.21.15.47:0/4241836069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef4102880 0x7f6ef4104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.271+0000 7f6ef9800640 1 --2- 172.21.15.47:0/4241836069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef40fff10 0x7f6ef4102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.271+0000 7f6ef9800640 1 -- 172.21.15.47:0/4241836069 >> 172.21.15.47:0/4241836069 conn(0x7f6ef40fbb30 msgr2=0x7f6ef40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:33.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.271+0000 7f6ef9800640 1 -- 172.21.15.47:0/4241836069 shutdown_connections 2024-03-21T11:48:33.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.271+0000 7f6ef9800640 1 -- 172.21.15.47:0/4241836069 wait complete. 2024-03-21T11:48:33.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.272+0000 7f6ef9800640 1 Processor -- start 2024-03-21T11:48:33.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.272+0000 7f6ef9800640 1 -- start start 2024-03-21T11:48:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.272+0000 7f6ef9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef40fff10 0x7f6ef419e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.272+0000 7f6ef9800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef4102880 0x7f6ef419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.272+0000 7f6ef9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef4105230 0x7f6ef41a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.272+0000 7f6ef9800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef41139c0 con 0x7f6ef4102880 2024-03-21T11:48:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.272+0000 7f6ef9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4113840 con 0x7f6ef40fff10 2024-03-21T11:48:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.272+0000 7f6ef9800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4113b40 con 0x7f6ef4105230 2024-03-21T11:48:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.272+0000 7f6ef2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef40fff10 0x7f6ef419e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.273+0000 7f6ef2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef40fff10 0x7f6ef419e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60738/0 (socket says 172.21.15.47:60738) 2024-03-21T11:48:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.273+0000 7f6ef2a00640 1 -- 172.21.15.47:0/2751309598 learned_addr learned my addr 172.21.15.47:0/2751309598 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.273+0000 7f6ef3400640 1 --2- 172.21.15.47:0/2751309598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef4105230 0x7f6ef41a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.273+0000 7f6ef2000640 1 --2- 172.21.15.47:0/2751309598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef4102880 0x7f6ef419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.273+0000 7f6ef2000640 1 -- 172.21.15.47:0/2751309598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef4105230 msgr2=0x7f6ef41a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:33.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.273+0000 7f6ef2000640 1 --2- 172.21.15.47:0/2751309598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef4105230 0x7f6ef41a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.273+0000 7f6ef2000640 1 -- 172.21.15.47:0/2751309598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef40fff10 msgr2=0x7f6ef419e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:33.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.273+0000 7f6ef2000640 1 --2- 172.21.15.47:0/2751309598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef40fff10 0x7f6ef419e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.273+0000 7f6ef2000640 1 -- 172.21.15.47:0/2751309598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef41a5be0 con 0x7f6ef4102880 2024-03-21T11:48:33.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.274+0000 7f6ef2000640 1 --2- 172.21.15.47:0/2751309598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef4102880 0x7f6ef419e550 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000b3f0 tx=0x7f6ee000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:33.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.274+0000 7f6eeb400640 1 -- 172.21.15.47:0/2751309598 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0015020 con 0x7f6ef4102880 2024-03-21T11:48:33.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.274+0000 7f6ef9800640 1 -- 172.21.15.47:0/2751309598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef41a5ed0 con 0x7f6ef4102880 2024-03-21T11:48:33.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.274+0000 7f6eeb400640 1 -- 172.21.15.47:0/2751309598 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ee0011d10 con 0x7f6ef4102880 2024-03-21T11:48:33.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.274+0000 7f6eeb400640 1 -- 172.21.15.47:0/2751309598 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee000f2e0 con 0x7f6ef4102880 2024-03-21T11:48:33.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.274+0000 7f6ef9800640 1 -- 172.21.15.47:0/2751309598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef41a6460 con 0x7f6ef4102880 2024-03-21T11:48:33.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.278+0000 7f6efc5a5740 1 Processor -- start 2024-03-21T11:48:33.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.279+0000 7f6efc5a5740 1 -- start start 2024-03-21T11:48:33.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.279+0000 7f6efc5a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db0415810 0x564db0415c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.279+0000 7f6efc5a5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db04169e0 0x564db0416e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.279+0000 7f6efc5a5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04173a0 0x564db041dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.279+0000 7f6efc5a5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db037a2c0 con 0x564db04169e0 2024-03-21T11:48:33.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.279+0000 7f6efc5a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db0322c90 con 0x564db0415810 2024-03-21T11:48:33.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.279+0000 7f6efc5a5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db0376360 con 0x564db04173a0 2024-03-21T11:48:33.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.280+0000 7f6ef3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db0415810 0x564db0415c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.280+0000 7f6ef3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db0415810 0x564db0415c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60752/0 (socket says 172.21.15.47:60752) 2024-03-21T11:48:33.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.280+0000 7f6ef3400640 1 -- 172.21.15.47:0/2464176219 learned_addr learned my addr 172.21.15.47:0/2464176219 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:33.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.280+0000 7f6ef2a00640 1 --2- 172.21.15.47:0/2464176219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db04169e0 0x564db0416e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.280+0000 7f6ef3400640 1 -- 172.21.15.47:0/2464176219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04173a0 msgr2=0x564db041dc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:48:33.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.280+0000 7f6ef3400640 1 --2- 172.21.15.47:0/2464176219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04173a0 0x564db041dc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.280+0000 7f6ef3400640 1 -- 172.21.15.47:0/2464176219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db04169e0 msgr2=0x564db0416e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:33.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.280+0000 7f6ef3400640 1 --2- 172.21.15.47:0/2464176219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db04169e0 0x564db0416e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.280+0000 7f6ef3400640 1 -- 172.21.15.47:0/2464176219 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db0347080 con 0x564db0415810 2024-03-21T11:48:33.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.281+0000 7f6ef3400640 1 --2- 172.21.15.47:0/2464176219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db0415810 0x564db0415c30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f6eec02c500 tx=0x7f6eec033b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70b038cf40045651 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:33.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.282+0000 7f6edfe00640 1 -- 172.21.15.47:0/2464176219 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec043030 con 0x564db0415810 2024-03-21T11:48:33.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.282+0000 7f6edfe00640 1 -- 172.21.15.47:0/2464176219 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6eec033cb0 con 0x564db0415810 2024-03-21T11:48:33.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.282+0000 7f6edfe00640 1 -- 172.21.15.47:0/2464176219 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec02cab0 con 0x564db0415810 2024-03-21T11:48:33.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.283+0000 7f6eeb400640 1 -- 172.21.15.47:0/2751309598 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6ee0018020 con 0x7f6ef4102880 2024-03-21T11:48:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.284+0000 7f6efc5a5740 1 -- 172.21.15.47:0/2464176219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db0415810 msgr2=0x564db0415c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.284+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/2464176219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db0415810 0x564db0415c30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f6eec02c500 tx=0x7f6eec033b40 comp rx=0 tx=0).stop 2024-03-21T11:48:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.285+0000 7f6efc5a5740 1 -- 172.21.15.47:0/2464176219 shutdown_connections 2024-03-21T11:48:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.285+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/2464176219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04173a0 0x564db041dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.285+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/2464176219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db04169e0 0x564db0416e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.285+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/2464176219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db0415810 0x564db0415c30 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.285+0000 7f6efc5a5740 1 -- 172.21.15.47:0/2464176219 >> 172.21.15.47:0/2464176219 conn(0x564db0410c00 msgr2=0x564db0411000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:33.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.285+0000 7f6eeb400640 1 --2- 172.21.15.47:0/2751309598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ed80a2fa0 0x7f6ed80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.285+0000 7f6eeb400640 1 -- 172.21.15.47:0/2751309598 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6ee00cee90 con 0x7f6ef4102880 2024-03-21T11:48:33.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.286+0000 7f6ef2a00640 1 --2- 172.21.15.47:0/2751309598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ed80a2fa0 0x7f6ed80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.288+0000 7f6efc5a5740 1 -- 172.21.15.47:0/2464176219 shutdown_connections 2024-03-21T11:48:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.289+0000 7f6efc5a5740 1 -- 172.21.15.47:0/2464176219 wait complete. 2024-03-21T11:48:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.289+0000 7f6ef2a00640 1 --2- 172.21.15.47:0/2751309598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ed80a2fa0 0x7f6ed80a5460 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x564db0411b70 tx=0x7f6ee4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.289+0000 7f6efc5a5740 1 -- start start 2024-03-21T11:48:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.290+0000 7f6efc5a5740 1 Processor -- start 2024-03-21T11:48:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.290+0000 7f6efc5a5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db0415810 0x564db041fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.290+0000 7f6efc5a5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db041fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.290+0000 7f6efc5a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db04173a0 0x564db0427030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.290+0000 7f6efc5a5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db041e350 con 0x564db0415810 2024-03-21T11:48:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.290+0000 7f6efc5a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db03360d0 con 0x564db04173a0 2024-03-21T11:48:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.290+0000 7f6efc5a5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db041e4d0 con 0x564db04169e0 2024-03-21T11:48:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.290+0000 7f6efc5a5740 10 client.0 _ll_register_callbacks cb 0x564db033ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.291+0000 7f6ef2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db041fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.291+0000 7f6ef2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db041fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35982/0 (socket says 172.21.15.47:35982) 2024-03-21T11:48:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.291+0000 7f6ef2000640 1 -- 172.21.15.47:0/3465172114 learned_addr learned my addr 172.21.15.47:0/3465172114 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.291+0000 7f6ef3400640 1 --2- 172.21.15.47:0/3465172114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db04173a0 0x564db0427030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.292+0000 7f6ef2000640 1 -- 172.21.15.47:0/3465172114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db04173a0 msgr2=0x564db0427030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.292+0000 7f6ef2000640 1 --2- 172.21.15.47:0/3465172114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db04173a0 0x564db0427030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.292+0000 7f6ef2000640 1 -- 172.21.15.47:0/3465172114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db0415810 msgr2=0x564db041fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.292+0000 7f6ef2000640 1 --2- 172.21.15.47:0/3465172114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db0415810 0x564db041fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.292+0000 7f6ef2000640 1 -- 172.21.15.47:0/3465172114 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564db008e3f0 con 0x564db04169e0 2024-03-21T11:48:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.292+0000 7f6ef2000640 1 --2- 172.21.15.47:0/3465172114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db041fff0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000bdb0 tx=0x7f6ee0009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6324aba8ec728625 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.293+0000 7f6edcc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:48:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.293+0000 7f6edcc00640 1 -- 172.21.15.47:0/3465172114 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee009b230 con 0x564db04169e0 2024-03-21T11:48:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.293+0000 7f6edcc00640 1 -- 172.21.15.47:0/3465172114 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6ee0010510 con 0x564db04169e0 2024-03-21T11:48:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.293+0000 7f6efc5a5740 1 -- 172.21.15.47:0/3465172114 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6eec033cb0 con 0x564db04169e0 2024-03-21T11:48:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.294+0000 7f6edcc00640 1 -- 172.21.15.47:0/3465172114 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ee009f380 con 0x564db04169e0 2024-03-21T11:48:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.294+0000 7f6efc5a5740 10 client.33415 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.294+0000 7f6efc5a5740 1 -- 172.21.15.47:0/3465172114 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564db01587d0 con 0x564db04169e0 2024-03-21T11:48:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.294+0000 7f6edcc00640 1 -- 172.21.15.47:0/3465172114 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6eec033cb0 con 0x564db04169e0 2024-03-21T11:48:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.295+0000 7f6edcc00640 1 -- 172.21.15.47:0/3465172114 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6ee009a680 con 0x564db04169e0 2024-03-21T11:48:33.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.295+0000 7f6efc5a5740 10 client.33415 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:33.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.295+0000 7f6efc5a5740 10 client.33415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:33.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.295+0000 7f6efc5a5740 20 client.33415 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:33.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.295+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/3465172114 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564db043c4f0 0x564db043e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.295+0000 7f6efc5a5740 4 client.33415 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:48:33.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.295+0000 7f6efc5a5740 1 -- 172.21.15.47:0/3465172114 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564db0387800 con 0x564db043c4f0 2024-03-21T11:48:33.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.296+0000 7f6ef2a00640 1 --2- 172.21.15.47:0/3465172114 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564db043c4f0 0x564db043e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.298+0000 7f6ef2a00640 1 --2- 172.21.15.47:0/3465172114 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564db043c4f0 0x564db043e950 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b12300049d2311b0 server_cookie=ea2a323ec7ec43e5 in_seq=0 out_seq=0 2024-03-21T11:48:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.298+0000 7f6edcc00640 10 client.33415 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.299+0000 7f6edcc00640 1 -- 172.21.15.47:0/3465172114 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564db0387800 con 0x564db043c4f0 2024-03-21T11:48:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.299+0000 7f6edcc00640 10 client.33415 handle_command_reply: tid=0 2024-03-21T11:48:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.300+0000 7f6efc5a5740 1 client.33415 shutdown 2024-03-21T11:48:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.301+0000 7f6efc5a5740 1 -- 172.21.15.47:0/3465172114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 msgr2=0x564db041fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.301+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/3465172114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db041fff0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000bdb0 tx=0x7f6ee0009200 comp rx=0 tx=0).stop 2024-03-21T11:48:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.301+0000 7f6efc5a5740 1 -- 172.21.15.47:0/3465172114 shutdown_connections 2024-03-21T11:48:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.301+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/3465172114 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564db043c4f0 0x564db043e950 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.301+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/3465172114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db04173a0 0x564db0427030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.301+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/3465172114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db041fff0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.301+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/3465172114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db0415810 0x564db041fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.301+0000 7f6efc5a5740 1 -- 172.21.15.47:0/3465172114 >> 172.21.15.47:0/3465172114 conn(0x564db0410c00 msgr2=0x564db040feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:33.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.301+0000 7f6edcc00640 0 client.33415 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:33.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.301+0000 7f6efc5a5740 1 -- 172.21.15.47:0/3465172114 shutdown_connections 2024-03-21T11:48:33.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.301+0000 7f6efc5a5740 1 -- 172.21.15.47:0/3465172114 wait complete. 2024-03-21T11:48:33.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.302+0000 7f6efc5a5740 20 client.33415 trim_cache size 0 max 16384 2024-03-21T11:48:33.313 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:32 smithi047 ceph-mon[31351]: pgmap v1157: 129 pgs: 129 active+clean; 792 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 12 MiB/s wr, 567 op/s 2024-03-21T11:48:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.318+0000 7f6efc5a5740 1 Processor -- start 2024-03-21T11:48:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.319+0000 7f6efc5a5740 1 -- start start 2024-03-21T11:48:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.319+0000 7f6efc5a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db0415810 0x564db0414570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.319+0000 7f6efc5a5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db0426ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.319+0000 7f6efc5a5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db04173a0 0x564db0419a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.319+0000 7f6efc5a5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef41a7e90 con 0x564db04173a0 2024-03-21T11:48:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.319+0000 7f6efc5a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef41a9810 con 0x564db0415810 2024-03-21T11:48:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.319+0000 7f6efc5a5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef41a84f0 con 0x564db04169e0 2024-03-21T11:48:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.319+0000 7f6ef2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db0426ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.319+0000 7f6ef2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db0415810 0x564db0414570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.319+0000 7f6ef2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db0426ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:35986/0 (socket says 172.21.15.47:35986) 2024-03-21T11:48:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.319+0000 7f6ef2000640 1 -- 172.21.15.47:0/748518032 learned_addr learned my addr 172.21.15.47:0/748518032 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.319+0000 7f6ef3400640 1 --2- 172.21.15.47:0/748518032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db04173a0 0x564db0419a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.320+0000 7f6ef2000640 1 -- 172.21.15.47:0/748518032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db0415810 msgr2=0x564db0414570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.320+0000 7f6ef2000640 1 --2- 172.21.15.47:0/748518032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db0415810 0x564db0414570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.320+0000 7f6ef2000640 1 -- 172.21.15.47:0/748518032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db04173a0 msgr2=0x564db0419a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.320+0000 7f6ef2000640 1 --2- 172.21.15.47:0/748518032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db04173a0 0x564db0419a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.320+0000 7f6ef2000640 1 -- 172.21.15.47:0/748518032 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db0411670 con 0x564db04169e0 2024-03-21T11:48:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.320+0000 7f6ef3400640 1 --2- 172.21.15.47:0/748518032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db04173a0 0x564db0419a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:48:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.320+0000 7f6ef2000640 1 --2- 172.21.15.47:0/748518032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db0426ff0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0097690 tx=0x7f6ee0099740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e8d105144212f85 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.321+0000 7f6edf400640 1 -- 172.21.15.47:0/748518032 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee009b230 con 0x564db04169e0 2024-03-21T11:48:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.321+0000 7f6edf400640 1 -- 172.21.15.47:0/748518032 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ee00a3c40 con 0x564db04169e0 2024-03-21T11:48:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.321+0000 7f6edf400640 1 -- 172.21.15.47:0/748518032 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00b40c0 con 0x564db04169e0 2024-03-21T11:48:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.322+0000 7f6efc5a5740 1 -- 172.21.15.47:0/748518032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 msgr2=0x564db0426ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.322+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/748518032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db0426ff0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0097690 tx=0x7f6ee0099740 comp rx=0 tx=0).stop 2024-03-21T11:48:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.322+0000 7f6efc5a5740 1 -- 172.21.15.47:0/748518032 shutdown_connections 2024-03-21T11:48:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.322+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/748518032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db04173a0 0x564db0419a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.322+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/748518032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db0426ff0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.322+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/748518032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db0415810 0x564db0414570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.322+0000 7f6efc5a5740 1 -- 172.21.15.47:0/748518032 >> 172.21.15.47:0/748518032 conn(0x564db0410c00 msgr2=0x564db0413e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.322+0000 7f6efc5a5740 1 -- 172.21.15.47:0/748518032 shutdown_connections 2024-03-21T11:48:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.322+0000 7f6efc5a5740 1 -- 172.21.15.47:0/748518032 wait complete. 2024-03-21T11:48:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.323+0000 7f6efc5a5740 1 -- start start 2024-03-21T11:48:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.323+0000 7f6efc5a5740 1 Processor -- start 2024-03-21T11:48:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.323+0000 7f6efc5a5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db0415810 0x564db0422d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.323+0000 7f6efc5a5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db04232d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.323+0000 7f6efc5a5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db04173a0 0x564db04262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.323+0000 7f6efc5a5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef41a91b0 con 0x564db0415810 2024-03-21T11:48:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.323+0000 7f6efc5a5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef41a8b50 con 0x564db04173a0 2024-03-21T11:48:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.323+0000 7f6efc5a5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef410e630 con 0x564db04169e0 2024-03-21T11:48:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.323+0000 7f6efc5a5740 10 client.0 _ll_register_callbacks cb 0x7f6ee0011ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.324+0000 7f6ef2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db0415810 0x564db0422d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.324+0000 7f6ef3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db04173a0 0x564db04262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.324+0000 7f6ef2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db0415810 0x564db0422d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57580/0 (socket says 172.21.15.47:57580) 2024-03-21T11:48:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.324+0000 7f6ef3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db04173a0 0x564db04262f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60770/0 (socket says 172.21.15.47:60770) 2024-03-21T11:48:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.324+0000 7f6ef2a00640 1 -- 172.21.15.47:0/342905210 learned_addr learned my addr 172.21.15.47:0/342905210 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.324+0000 7f6ef3400640 1 -- 172.21.15.47:0/342905210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 msgr2=0x564db04232d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.324+0000 7f6ef3400640 1 --2- 172.21.15.47:0/342905210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db04232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.324+0000 7f6ef3400640 1 -- 172.21.15.47:0/342905210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db0415810 msgr2=0x564db0422d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.324+0000 7f6ef3400640 1 --2- 172.21.15.47:0/342905210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db0415810 0x564db0422d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.324+0000 7f6ef3400640 1 -- 172.21.15.47:0/342905210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564db04268e0 con 0x564db04173a0 2024-03-21T11:48:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.325+0000 7f6ef3400640 1 --2- 172.21.15.47:0/342905210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db04173a0 0x564db04262f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6eec034fd0 tx=0x7f6eec0336e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06feb4425c18602 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.325+0000 7f6edcc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:48:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.325+0000 7f6edcc00640 1 -- 172.21.15.47:0/342905210 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec043050 con 0x564db04173a0 2024-03-21T11:48:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.325+0000 7f6edcc00640 1 -- 172.21.15.47:0/342905210 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6eec003d80 con 0x564db04173a0 2024-03-21T11:48:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.325+0000 7f6efc5a5740 1 -- 172.21.15.47:0/342905210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ee00a3c40 con 0x564db04173a0 2024-03-21T11:48:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.326+0000 7f6edcc00640 1 -- 172.21.15.47:0/342905210 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eec05b050 con 0x564db04173a0 2024-03-21T11:48:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.326+0000 7f6efc5a5740 10 client.33476 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.326+0000 7f6efc5a5740 1 -- 172.21.15.47:0/342905210 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564db0498560 con 0x564db04173a0 2024-03-21T11:48:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.326+0000 7f6edcc00640 1 -- 172.21.15.47:0/342905210 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ee00a3c40 con 0x564db04173a0 2024-03-21T11:48:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.326+0000 7f6edcc00640 1 -- 172.21.15.47:0/342905210 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6eec004540 con 0x564db04173a0 2024-03-21T11:48:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.327+0000 7f6efc5a5740 10 client.33476 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.327+0000 7f6efc5a5740 10 client.33476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.327+0000 7f6efc5a5740 20 client.33476 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.327+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/342905210 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564db0498ab0 0x564db049aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.327+0000 7f6efc5a5740 4 client.33476 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:48:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.327+0000 7f6efc5a5740 1 -- 172.21.15.47:0/342905210 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564db0381b50 con 0x564db0498ab0 2024-03-21T11:48:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.327+0000 7f6ef2a00640 1 --2- 172.21.15.47:0/342905210 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564db0498ab0 0x564db049aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.328+0000 7f6ef2a00640 1 --2- 172.21.15.47:0/342905210 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564db0498ab0 0x564db049aea0 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64901f96d1d4d466 server_cookie=b715f5dc292bb03b in_seq=0 out_seq=0 2024-03-21T11:48:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.328+0000 7f6edcc00640 10 client.33476 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.330+0000 7f6edcc00640 1 -- 172.21.15.47:0/342905210 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564db0381b50 con 0x564db0498ab0 2024-03-21T11:48:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.330+0000 7f6edcc00640 10 client.33476 handle_command_reply: tid=0 2024-03-21T11:48:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.330+0000 7f6efc5a5740 1 client.33476 shutdown 2024-03-21T11:48:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.330+0000 7f6efc5a5740 1 -- 172.21.15.47:0/342905210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db04173a0 msgr2=0x564db04262f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.330+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/342905210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db04173a0 0x564db04262f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6eec034fd0 tx=0x7f6eec0336e0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.330+0000 7f6efc5a5740 1 -- 172.21.15.47:0/342905210 shutdown_connections 2024-03-21T11:48:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.330+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/342905210 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x564db0498ab0 0x564db049aea0 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.330+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/342905210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564db04173a0 0x564db04262f0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.330+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/342905210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564db04169e0 0x564db04232d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.330+0000 7f6efc5a5740 1 --2- 172.21.15.47:0/342905210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564db0415810 0x564db0422d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.330+0000 7f6efc5a5740 1 -- 172.21.15.47:0/342905210 >> 172.21.15.47:0/342905210 conn(0x564db0410c00 msgr2=0x564db04146f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.330+0000 7f6edcc00640 0 client.33476 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.330+0000 7f6efc5a5740 1 -- 172.21.15.47:0/342905210 shutdown_connections 2024-03-21T11:48:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.330+0000 7f6efc5a5740 1 -- 172.21.15.47:0/342905210 wait complete. 2024-03-21T11:48:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.331+0000 7f6efc5a5740 20 client.33476 trim_cache size 0 max 16384 2024-03-21T11:48:33.331 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:48:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.332+0000 7f6ef9800640 1 -- 172.21.15.47:0/2751309598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ed80a2fa0 msgr2=0x7f6ed80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.332+0000 7f6ef9800640 1 --2- 172.21.15.47:0/2751309598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ed80a2fa0 0x7f6ed80a5460 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x564db0411b70 tx=0x7f6ee4009260 comp rx=0 tx=0).stop 2024-03-21T11:48:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.332+0000 7f6ef9800640 1 -- 172.21.15.47:0/2751309598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef4102880 msgr2=0x7f6ef419e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.332+0000 7f6ef9800640 1 --2- 172.21.15.47:0/2751309598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef4102880 0x7f6ef419e550 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000b3f0 tx=0x7f6ee000b8b0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.332+0000 7f6ef9800640 1 -- 172.21.15.47:0/2751309598 shutdown_connections 2024-03-21T11:48:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.332+0000 7f6ef9800640 1 --2- 172.21.15.47:0/2751309598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f6ed80a2fa0 0x7f6ed80a5460 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.332+0000 7f6ef9800640 1 --2- 172.21.15.47:0/2751309598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ef4105230 0x7f6ef41a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.332+0000 7f6ef9800640 1 --2- 172.21.15.47:0/2751309598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ef4102880 0x7f6ef419e550 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.332+0000 7f6ef9800640 1 --2- 172.21.15.47:0/2751309598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef40fff10 0x7f6ef419e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.332+0000 7f6ef9800640 1 -- 172.21.15.47:0/2751309598 >> 172.21.15.47:0/2751309598 conn(0x7f6ef40fbb30 msgr2=0x7f6ef4100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.332+0000 7f6ef9800640 1 -- 172.21.15.47:0/2751309598 shutdown_connections 2024-03-21T11:48:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:33.332+0000 7f6ef9800640 1 -- 172.21.15.47:0/2751309598 wait complete. 2024-03-21T11:48:33.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:32 smithi100 ceph-mon[36245]: pgmap v1157: 129 pgs: 129 active+clean; 792 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 12 MiB/s wr, 567 op/s 2024-03-21T11:48:34.002 INFO:tasks.workunit.client.0.smithi047.stdout: 10 279376 21.38 MB/sec execute 557 sec latency 112.767 ms 2024-03-21T11:48:34.350 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:48:34.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:48:34.350 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:48:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.582+0000 7f8ebf200640 1 -- 172.21.15.47:0/3955607504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ec01010f0 msgr2=0x7f8ec0101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.582+0000 7f8ebf200640 1 --2- 172.21.15.47:0/3955607504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ec01010f0 0x7f8ec0101570 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8009ca0 tx=0x7f8eb802c100 comp rx=0 tx=0).stop 2024-03-21T11:48:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.583+0000 7f8ebf200640 1 -- 172.21.15.47:0/3955607504 shutdown_connections 2024-03-21T11:48:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.583+0000 7f8ebf200640 1 --2- 172.21.15.47:0/3955607504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ec01010f0 0x7f8ec0101570 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.583+0000 7f8ebf200640 1 --2- 172.21.15.47:0/3955607504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ec0100750 0x7f8ec0100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.583+0000 7f8ebf200640 1 --2- 172.21.15.47:0/3955607504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ec00ffdf0 0x7f8ec0100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.583+0000 7f8ebf200640 1 -- 172.21.15.47:0/3955607504 >> 172.21.15.47:0/3955607504 conn(0x7f8ec00fba10 msgr2=0x7f8ec00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:34.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.583+0000 7f8ebf200640 1 -- 172.21.15.47:0/3955607504 shutdown_connections 2024-03-21T11:48:34.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.584+0000 7f8ebf200640 1 -- 172.21.15.47:0/3955607504 wait complete. 2024-03-21T11:48:34.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.585+0000 7f8ebf200640 1 Processor -- start 2024-03-21T11:48:34.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.585+0000 7f8ebf200640 1 -- start start 2024-03-21T11:48:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.585+0000 7f8ebf200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ec00ffdf0 0x7f8ec0199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.585+0000 7f8ebf200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ec0100750 0x7f8ec0199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.585+0000 7f8ebf200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ec01010f0 0x7f8ec01a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.586+0000 7f8ebf200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec010f3d0 con 0x7f8ec0100750 2024-03-21T11:48:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.586+0000 7f8ebf200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec010f250 con 0x7f8ec01010f0 2024-03-21T11:48:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.586+0000 7f8ebf200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec010f550 con 0x7f8ec00ffdf0 2024-03-21T11:48:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.586+0000 7f8ebde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ec00ffdf0 0x7f8ec0199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.586+0000 7f8ebde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ec00ffdf0 0x7f8ec0199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36010/0 (socket says 172.21.15.47:36010) 2024-03-21T11:48:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.586+0000 7f8ebde00640 1 -- 172.21.15.47:0/3204407694 learned_addr learned my addr 172.21.15.47:0/3204407694 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.586+0000 7f8ebe800640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ec01010f0 0x7f8ec01a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.586+0000 7f8ebd400640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ec0100750 0x7f8ec0199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.586+0000 7f8ebde00640 1 -- 172.21.15.47:0/3204407694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ec01010f0 msgr2=0x7f8ec01a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.586+0000 7f8ebde00640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ec01010f0 0x7f8ec01a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.586+0000 7f8ebde00640 1 -- 172.21.15.47:0/3204407694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ec0100750 msgr2=0x7f8ec0199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.586+0000 7f8ebde00640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ec0100750 0x7f8ec0199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.586+0000 7f8ebde00640 1 -- 172.21.15.47:0/3204407694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec01a1630 con 0x7f8ec00ffdf0 2024-03-21T11:48:34.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.586+0000 7f8ebe800640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ec01010f0 0x7f8ec01a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:48:34.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.587+0000 7f8ebde00640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ec00ffdf0 0x7f8ec0199a10 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000e9f0 tx=0x7f8eb000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:34.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.587+0000 7f8eb6a00640 1 -- 172.21.15.47:0/3204407694 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb000cd80 con 0x7f8ec00ffdf0 2024-03-21T11:48:34.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.587+0000 7f8eb6a00640 1 -- 172.21.15.47:0/3204407694 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8eb0010910 con 0x7f8ec00ffdf0 2024-03-21T11:48:34.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.587+0000 7f8ebf200640 1 -- 172.21.15.47:0/3204407694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec01a1920 con 0x7f8ec00ffdf0 2024-03-21T11:48:34.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.587+0000 7f8eb6a00640 1 -- 172.21.15.47:0/3204407694 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0018990 con 0x7f8ec00ffdf0 2024-03-21T11:48:34.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.588+0000 7f8ebf200640 1 -- 172.21.15.47:0/3204407694 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec01a1e70 con 0x7f8ec00ffdf0 2024-03-21T11:48:34.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.591+0000 7f8eb6a00640 1 -- 172.21.15.47:0/3204407694 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8eb0010ac0 con 0x7f8ec00ffdf0 2024-03-21T11:48:34.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.591+0000 7f8ec7837740 1 Processor -- start 2024-03-21T11:48:34.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.591+0000 7f8ec7837740 1 -- start start 2024-03-21T11:48:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.592+0000 7f8ec7837740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91081e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.592+0000 7f8ec7837740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c91081eca0 0x55c9108237a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.592+0000 7f8ec7837740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c910823e10 0x55c9108262a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.592+0000 7f8ec7837740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c910783eb0 con 0x55c91081eca0 2024-03-21T11:48:34.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.592+0000 7f8ec7837740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91072ca60 con 0x55c910823e10 2024-03-21T11:48:34.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.592+0000 7f8ec7837740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91077ff50 con 0x55c91081e2b0 2024-03-21T11:48:34.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.592+0000 7f8ebe800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91081e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.592+0000 7f8ebe800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91081e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36014/0 (socket says 172.21.15.47:36014) 2024-03-21T11:48:34.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.592+0000 7f8ebd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c910823e10 0x55c9108262a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.592+0000 7f8ebe800640 1 -- 172.21.15.47:0/847879951 learned_addr learned my addr 172.21.15.47:0/847879951 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:34.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.592+0000 7f8ebde00640 1 --2- 172.21.15.47:0/847879951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c91081eca0 0x55c9108237a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.593+0000 7f8ebd400640 1 -- 172.21.15.47:0/847879951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 msgr2=0x55c91081e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.593+0000 7f8ebd400640 1 --2- 172.21.15.47:0/847879951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91081e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.593+0000 7f8ebd400640 1 -- 172.21.15.47:0/847879951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c91081eca0 msgr2=0x55c9108237a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.593+0000 7f8ebd400640 1 --2- 172.21.15.47:0/847879951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c91081eca0 0x55c9108237a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.593+0000 7f8ebd400640 1 -- 172.21.15.47:0/847879951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9105607e0 con 0x55c910823e10 2024-03-21T11:48:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.593+0000 7f8ebe800640 1 --2- 172.21.15.47:0/847879951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91081e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:48:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.593+0000 7f8eb6a00640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8ea40a2f20 0x7f8ea40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.593+0000 7f8ebd400640 1 --2- 172.21.15.47:0/847879951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c910823e10 0x55c9108262a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009bf0 tx=0x7f8eac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a09e1b2b55b4a1c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.593+0000 7f8eb6a00640 1 -- 172.21.15.47:0/3204407694 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8eb00cfad0 con 0x7f8ec00ffdf0 2024-03-21T11:48:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.593+0000 7f8eab000640 1 -- 172.21.15.47:0/847879951 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac004050 con 0x55c910823e10 2024-03-21T11:48:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.593+0000 7f8eab000640 1 -- 172.21.15.47:0/847879951 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8eac005500 con 0x55c910823e10 2024-03-21T11:48:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.593+0000 7f8eab000640 1 -- 172.21.15.47:0/847879951 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac004e40 con 0x55c910823e10 2024-03-21T11:48:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.593+0000 7f8ebe800640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8ea40a2f20 0x7f8ea40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.594+0000 7f8ec7837740 1 -- 172.21.15.47:0/847879951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c910823e10 msgr2=0x55c9108262a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.594+0000 7f8ec7837740 1 --2- 172.21.15.47:0/847879951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c910823e10 0x55c9108262a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009bf0 tx=0x7f8eac011810 comp rx=0 tx=0).stop 2024-03-21T11:48:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.594+0000 7f8ebe800640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8ea40a2f20 0x7f8ea40a53e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8006010 tx=0x7f8eb8032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.594+0000 7f8ec7837740 1 -- 172.21.15.47:0/847879951 shutdown_connections 2024-03-21T11:48:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.594+0000 7f8ec7837740 1 --2- 172.21.15.47:0/847879951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c910823e10 0x55c9108262a0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.594+0000 7f8ec7837740 1 --2- 172.21.15.47:0/847879951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c91081eca0 0x55c9108237a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.594+0000 7f8ec7837740 1 --2- 172.21.15.47:0/847879951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91081e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.594+0000 7f8ec7837740 1 -- 172.21.15.47:0/847879951 >> 172.21.15.47:0/847879951 conn(0x55c910819820 msgr2=0x55c910819c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.594+0000 7f8ec7837740 1 -- 172.21.15.47:0/847879951 shutdown_connections 2024-03-21T11:48:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.594+0000 7f8ec7837740 1 -- 172.21.15.47:0/847879951 wait complete. 2024-03-21T11:48:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.595+0000 7f8ec7837740 1 -- start start 2024-03-21T11:48:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.595+0000 7f8ec7837740 1 Processor -- start 2024-03-21T11:48:34.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.595+0000 7f8ec7837740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c910828520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.595+0000 7f8ec7837740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 0x55c910828a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.595+0000 7f8ec7837740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 0x55c91082faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.595+0000 7f8ec7837740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c910826e10 con 0x55c910823e10 2024-03-21T11:48:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.595+0000 7f8ec7837740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91073feb0 con 0x55c91081eca0 2024-03-21T11:48:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.595+0000 7f8ec7837740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c910826f90 con 0x55c91081e2b0 2024-03-21T11:48:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.595+0000 7f8ec7837740 10 client.0 _ll_register_callbacks cb 0x55c910734150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.596+0000 7f8ebd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 0x55c910828a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.596+0000 7f8ebe800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 0x55c91082faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.596+0000 7f8ebde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c910828520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.596+0000 7f8ebd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 0x55c910828a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60780/0 (socket says 172.21.15.47:60780) 2024-03-21T11:48:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.596+0000 7f8ebd400640 1 -- 172.21.15.47:0/577301008 learned_addr learned my addr 172.21.15.47:0/577301008 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:34.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.596+0000 7f8ebde00640 1 -- 172.21.15.47:0/577301008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 msgr2=0x55c910828a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.596+0000 7f8ebde00640 1 --2- 172.21.15.47:0/577301008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 0x55c910828a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.596+0000 7f8ebde00640 1 -- 172.21.15.47:0/577301008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 msgr2=0x55c91082faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.596+0000 7f8ebde00640 1 --2- 172.21.15.47:0/577301008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 0x55c91082faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.596+0000 7f8ebde00640 1 -- 172.21.15.47:0/577301008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9105c0f70 con 0x55c91081e2b0 2024-03-21T11:48:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.596+0000 7f8ebde00640 1 --2- 172.21.15.47:0/577301008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c910828520 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x55c91081f710 tx=0x7f8eb0098250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcc524dd72ddd5f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.596+0000 7f8e9f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:48:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.597+0000 7f8e9f400640 1 -- 172.21.15.47:0/577301008 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb00bad80 con 0x55c91081e2b0 2024-03-21T11:48:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.597+0000 7f8e9f400640 1 -- 172.21.15.47:0/577301008 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8eb009b310 con 0x55c91081e2b0 2024-03-21T11:48:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.597+0000 7f8ec7837740 1 -- 172.21.15.47:0/577301008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8eac005500 con 0x55c91081e2b0 2024-03-21T11:48:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.597+0000 7f8e9f400640 1 -- 172.21.15.47:0/577301008 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8eb009bec0 con 0x55c91081e2b0 2024-03-21T11:48:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.597+0000 7f8ec7837740 10 client.33430 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:34.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.597+0000 7f8ec7837740 1 -- 172.21.15.47:0/577301008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c910458170 con 0x55c91081e2b0 2024-03-21T11:48:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.597+0000 7f8e9f400640 1 -- 172.21.15.47:0/577301008 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8eac005500 con 0x55c91081e2b0 2024-03-21T11:48:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.597+0000 7f8e9f400640 1 -- 172.21.15.47:0/577301008 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8eb0018bc0 con 0x55c91081e2b0 2024-03-21T11:48:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.597+0000 7f8ec7837740 10 client.33430 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.597+0000 7f8ec7837740 10 client.33430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.597+0000 7f8ec7837740 20 client.33430 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.597+0000 7f8ec7837740 1 --2- 172.21.15.47:0/577301008 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c910834300 0x55c9108367a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.597+0000 7f8ec7837740 4 client.33430 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:48:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.597+0000 7f8ec7837740 1 -- 172.21.15.47:0/577301008 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9107913f0 con 0x55c910834300 2024-03-21T11:48:34.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.598+0000 7f8ebd400640 1 --2- 172.21.15.47:0/577301008 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c910834300 0x55c9108367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.598+0000 7f8ebd400640 1 --2- 172.21.15.47:0/577301008 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c910834300 0x55c9108367a0 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef624e7580949626 server_cookie=bf6d9efeee0f4442 in_seq=0 out_seq=0 2024-03-21T11:48:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.598+0000 7f8e9f400640 10 client.33430 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.602+0000 7f8e9f400640 1 -- 172.21.15.47:0/577301008 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c9107913f0 con 0x55c910834300 2024-03-21T11:48:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.602+0000 7f8e9f400640 10 client.33430 handle_command_reply: tid=0 2024-03-21T11:48:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.602+0000 7f8ec7837740 1 client.33430 shutdown 2024-03-21T11:48:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.602+0000 7f8ec7837740 1 -- 172.21.15.47:0/577301008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 msgr2=0x55c910828520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.602+0000 7f8ec7837740 1 --2- 172.21.15.47:0/577301008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c910828520 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x55c91081f710 tx=0x7f8eb0098250 comp rx=0 tx=0).stop 2024-03-21T11:48:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.602+0000 7f8ec7837740 1 -- 172.21.15.47:0/577301008 shutdown_connections 2024-03-21T11:48:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.602+0000 7f8ec7837740 1 --2- 172.21.15.47:0/577301008 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c910834300 0x55c9108367a0 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.602+0000 7f8ec7837740 1 --2- 172.21.15.47:0/577301008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 0x55c91082faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.602+0000 7f8ec7837740 1 --2- 172.21.15.47:0/577301008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 0x55c910828a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.602+0000 7f8ec7837740 1 --2- 172.21.15.47:0/577301008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c910828520 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.602+0000 7f8e9f400640 0 client.33430 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:34.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.602+0000 7f8ec7837740 1 -- 172.21.15.47:0/577301008 >> 172.21.15.47:0/577301008 conn(0x55c910819820 msgr2=0x55c910818a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:34.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.603+0000 7f8ec7837740 1 -- 172.21.15.47:0/577301008 shutdown_connections 2024-03-21T11:48:34.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.603+0000 7f8ec7837740 1 -- 172.21.15.47:0/577301008 wait complete. 2024-03-21T11:48:34.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.603+0000 7f8ec7837740 20 client.33430 trim_cache size 0 max 16384 2024-03-21T11:48:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.630+0000 7f8ec7837740 1 Processor -- start 2024-03-21T11:48:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.631+0000 7f8ec7837740 1 -- start start 2024-03-21T11:48:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.631+0000 7f8ec7837740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91081f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.631+0000 7f8ec7837740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 0x55c91082fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.631+0000 7f8ec7837740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 0x55c91081fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.631+0000 7f8ec7837740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec01ac100 con 0x55c910823e10 2024-03-21T11:48:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.631+0000 7f8ec7837740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec01ada80 con 0x55c91081eca0 2024-03-21T11:48:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.631+0000 7f8ec7837740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec01ac760 con 0x55c91081e2b0 2024-03-21T11:48:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.631+0000 7f8ebde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91081f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.631+0000 7f8ebe800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 0x55c91081fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.631+0000 7f8ebde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91081f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36038/0 (socket says 172.21.15.47:36038) 2024-03-21T11:48:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.631+0000 7f8ebde00640 1 -- 172.21.15.47:0/3526483424 learned_addr learned my addr 172.21.15.47:0/3526483424 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:34.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.631+0000 7f8ebd400640 1 --2- 172.21.15.47:0/3526483424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 0x55c91082fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.632+0000 7f8ebde00640 1 -- 172.21.15.47:0/3526483424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 msgr2=0x55c91082fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.632+0000 7f8ebde00640 1 --2- 172.21.15.47:0/3526483424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 0x55c91082fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.632+0000 7f8ebde00640 1 -- 172.21.15.47:0/3526483424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 msgr2=0x55c91081fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.632+0000 7f8ebde00640 1 --2- 172.21.15.47:0/3526483424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 0x55c91081fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.632+0000 7f8ebde00640 1 -- 172.21.15.47:0/3526483424 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c91081f530 con 0x55c91081e2b0 2024-03-21T11:48:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.632+0000 7f8ebe800640 1 --2- 172.21.15.47:0/3526483424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 0x55c91081fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:48:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.632+0000 7f8ebde00640 1 --2- 172.21.15.47:0/3526483424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91081f740 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f8eb009be50 tx=0x7f8eb000cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c418c8ac5ddd5f5a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.632+0000 7f8eaa600640 1 -- 172.21.15.47:0/3526483424 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb009ff60 con 0x55c91081e2b0 2024-03-21T11:48:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.632+0000 7f8eaa600640 1 -- 172.21.15.47:0/3526483424 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8eb0097c10 con 0x55c91081e2b0 2024-03-21T11:48:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.632+0000 7f8eaa600640 1 -- 172.21.15.47:0/3526483424 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb009b570 con 0x55c91081e2b0 2024-03-21T11:48:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.633+0000 7f8ec7837740 1 -- 172.21.15.47:0/3526483424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 msgr2=0x55c91081f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.633+0000 7f8ec7837740 1 --2- 172.21.15.47:0/3526483424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91081f740 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f8eb009be50 tx=0x7f8eb000cf20 comp rx=0 tx=0).stop 2024-03-21T11:48:34.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.633+0000 7f8ec7837740 1 -- 172.21.15.47:0/3526483424 shutdown_connections 2024-03-21T11:48:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.633+0000 7f8ec7837740 1 --2- 172.21.15.47:0/3526483424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 0x55c91081fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.633+0000 7f8ec7837740 1 --2- 172.21.15.47:0/3526483424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 0x55c91082fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.633+0000 7f8ec7837740 1 --2- 172.21.15.47:0/3526483424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91081f740 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.633+0000 7f8ec7837740 1 -- 172.21.15.47:0/3526483424 >> 172.21.15.47:0/3526483424 conn(0x55c910819820 msgr2=0x55c91081d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.633+0000 7f8ec7837740 1 -- 172.21.15.47:0/3526483424 shutdown_connections 2024-03-21T11:48:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.634+0000 7f8ec7837740 1 -- 172.21.15.47:0/3526483424 wait complete. 2024-03-21T11:48:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.634+0000 7f8ec7837740 1 -- start start 2024-03-21T11:48:34.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.634+0000 7f8ec7837740 1 Processor -- start 2024-03-21T11:48:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ec7837740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91082e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ec7837740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 0x55c91082e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ec7837740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 0x55c91082eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ec7837740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec01ad420 con 0x55c910823e10 2024-03-21T11:48:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ec7837740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec01acdc0 con 0x55c91081eca0 2024-03-21T11:48:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ec7837740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec01aa2c0 con 0x55c91081e2b0 2024-03-21T11:48:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ec7837740 10 client.0 _ll_register_callbacks cb 0x7f8eb00d3200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:34.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ebde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91082e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ebde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91082e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36044/0 (socket says 172.21.15.47:36044) 2024-03-21T11:48:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ebde00640 1 -- 172.21.15.47:0/1159959008 learned_addr learned my addr 172.21.15.47:0/1159959008 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ebd400640 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 0x55c91082e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ebe800640 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 0x55c91082eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ebd400640 1 -- 172.21.15.47:0/1159959008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 msgr2=0x55c91082e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ebd400640 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91082e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ebd400640 1 -- 172.21.15.47:0/1159959008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 msgr2=0x55c91082eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ebd400640 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 0x55c91082eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ebd400640 1 -- 172.21.15.47:0/1159959008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c91088fc30 con 0x55c91081eca0 2024-03-21T11:48:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ebe800640 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 0x55c91082eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:48:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.635+0000 7f8ebde00640 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91082e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:48:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.636+0000 7f8ebd400640 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 0x55c91082e650 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x55c910836fe0 tx=0x7f8eac005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a680f8ec20ab7033 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.636+0000 7f8e9f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:48:34.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.636+0000 7f8e9f400640 1 -- 172.21.15.47:0/1159959008 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac004250 con 0x55c91081eca0 2024-03-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.636+0000 7f8ec7837740 1 -- 172.21.15.47:0/1159959008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8eb0097c10 con 0x55c91081eca0 2024-03-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.636+0000 7f8e9f400640 1 -- 172.21.15.47:0/1159959008 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f8eac0043f0 con 0x55c91081eca0 2024-03-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.637+0000 7f8e9f400640 1 -- 172.21.15.47:0/1159959008 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8eac03c050 con 0x55c91081eca0 2024-03-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.637+0000 7f8ec7837740 10 client.33494 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.637+0000 7f8ec7837740 1 -- 172.21.15.47:0/1159959008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c910890eb0 con 0x55c91081eca0 2024-03-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.637+0000 7f8e9f400640 1 -- 172.21.15.47:0/1159959008 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8eb0097c10 con 0x55c91081eca0 2024-03-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.638+0000 7f8e9f400640 1 -- 172.21.15.47:0/1159959008 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8eac02e5a0 con 0x55c91081eca0 2024-03-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.638+0000 7f8ec7837740 10 client.33494 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.638+0000 7f8ec7837740 10 client.33494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.638+0000 7f8ec7837740 20 client.33494 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.638+0000 7f8ec7837740 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c910891670 0x55c910893a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.638+0000 7f8ec7837740 4 client.33494 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.638+0000 7f8ec7837740 1 -- 172.21.15.47:0/1159959008 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c91078b740 con 0x55c910891670 2024-03-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.638+0000 7f8ebde00640 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c910891670 0x55c910893a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.639+0000 7f8ebde00640 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c910891670 0x55c910893a90 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89597080786e9f13 server_cookie=68bb60762887b63 in_seq=0 out_seq=0 2024-03-21T11:48:34.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.639+0000 7f8e9f400640 10 client.33494 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.645+0000 7f8e9f400640 1 -- 172.21.15.47:0/1159959008 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c91078b740 con 0x55c910891670 2024-03-21T11:48:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.645+0000 7f8e9f400640 10 client.33494 handle_command_reply: tid=0 2024-03-21T11:48:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.645+0000 7f8ec7837740 1 client.33494 shutdown 2024-03-21T11:48:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.645+0000 7f8ec7837740 1 -- 172.21.15.47:0/1159959008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 msgr2=0x55c91082e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.645+0000 7f8ec7837740 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 0x55c91082e650 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x55c910836fe0 tx=0x7f8eac005db0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.646+0000 7f8ec7837740 1 -- 172.21.15.47:0/1159959008 shutdown_connections 2024-03-21T11:48:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.646+0000 7f8ec7837740 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c910891670 0x55c910893a90 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.646+0000 7f8ec7837740 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c910823e10 0x55c91082eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.646+0000 7f8ec7837740 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c91081eca0 0x55c91082e650 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.646+0000 7f8ec7837740 1 --2- 172.21.15.47:0/1159959008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c91081e2b0 0x55c91082e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.646+0000 7f8e9f400640 0 client.33494 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.646+0000 7f8ec7837740 1 -- 172.21.15.47:0/1159959008 >> 172.21.15.47:0/1159959008 conn(0x55c910819820 msgr2=0x55c9108181f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.646+0000 7f8ec7837740 1 -- 172.21.15.47:0/1159959008 shutdown_connections 2024-03-21T11:48:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.646+0000 7f8ec7837740 1 -- 172.21.15.47:0/1159959008 wait complete. 2024-03-21T11:48:34.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.646+0000 7f8ec7837740 20 client.33494 trim_cache size 0 max 16384 2024-03-21T11:48:34.647 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:48:34.647 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:48:34.647 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "778b6f3c-73e5-4d40-b923-7aa2169cb62f", 2024-03-21T11:48:34.647 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:48:34.648 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:48:34.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.648+0000 7f8ebf200640 1 -- 172.21.15.47:0/3204407694 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8ea40a2f20 msgr2=0x7f8ea40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.648+0000 7f8ebf200640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8ea40a2f20 0x7f8ea40a53e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8006010 tx=0x7f8eb8032590 comp rx=0 tx=0).stop 2024-03-21T11:48:34.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.648+0000 7f8ebf200640 1 -- 172.21.15.47:0/3204407694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ec00ffdf0 msgr2=0x7f8ec0199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.648+0000 7f8ebf200640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ec00ffdf0 0x7f8ec0199a10 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000e9f0 tx=0x7f8eb000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.649+0000 7f8ebf200640 1 -- 172.21.15.47:0/3204407694 shutdown_connections 2024-03-21T11:48:34.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.649+0000 7f8ebf200640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f8ea40a2f20 0x7f8ea40a53e0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.649+0000 7f8ebf200640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ec01010f0 0x7f8ec01a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.649+0000 7f8ebf200640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ec0100750 0x7f8ec0199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.649+0000 7f8ebf200640 1 --2- 172.21.15.47:0/3204407694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ec00ffdf0 0x7f8ec0199a10 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.649+0000 7f8ebf200640 1 -- 172.21.15.47:0/3204407694 >> 172.21.15.47:0/3204407694 conn(0x7f8ec00fba10 msgr2=0x7f8ec00fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:34.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.649+0000 7f8ebf200640 1 -- 172.21.15.47:0/3204407694 shutdown_connections 2024-03-21T11:48:34.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.649+0000 7f8ebf200640 1 -- 172.21.15.47:0/3204407694 wait complete. 2024-03-21T11:48:34.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:48:34.668 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:48:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.884+0000 7f57ab400640 1 -- 172.21.15.47:0/2104407798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a4109560 msgr2=0x7f57a410b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.884+0000 7f57ab400640 1 --2- 172.21.15.47:0/2104407798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a4109560 0x7f57a410b9d0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f579c009ca0 tx=0x7f579c02c100 comp rx=0 tx=0).stop 2024-03-21T11:48:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.885+0000 7f57ab400640 1 -- 172.21.15.47:0/2104407798 shutdown_connections 2024-03-21T11:48:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.885+0000 7f57ab400640 1 --2- 172.21.15.47:0/2104407798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a4109560 0x7f57a410b9d0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.885+0000 7f57ab400640 1 --2- 172.21.15.47:0/2104407798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a40740c0 0x7f57a4074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.885+0000 7f57ab400640 1 --2- 172.21.15.47:0/2104407798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57a40756a0 0x7f57a4073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.885+0000 7f57ab400640 1 -- 172.21.15.47:0/2104407798 >> 172.21.15.47:0/2104407798 conn(0x7f57a40fcc40 msgr2=0x7f57a40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.886+0000 7f57ab400640 1 -- 172.21.15.47:0/2104407798 shutdown_connections 2024-03-21T11:48:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.886+0000 7f57ab400640 1 -- 172.21.15.47:0/2104407798 wait complete. 2024-03-21T11:48:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.887+0000 7f57ab400640 1 Processor -- start 2024-03-21T11:48:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.887+0000 7f57ab400640 1 -- start start 2024-03-21T11:48:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.887+0000 7f57ab400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57a40740c0 0x7f57a4199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.887+0000 7f57ab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a40756a0 0x7f57a419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.887+0000 7f57ab400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a4109560 0x7f57a41a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.887+0000 7f57ab400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a410f4f0 con 0x7f57a40740c0 2024-03-21T11:48:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.887+0000 7f57ab400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a410f370 con 0x7f57a40756a0 2024-03-21T11:48:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.887+0000 7f57ab400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a410f670 con 0x7f57a4109560 2024-03-21T11:48:34.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.887+0000 7f57a3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a40756a0 0x7f57a419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.888+0000 7f57a3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a40756a0 0x7f57a419a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60824/0 (socket says 172.21.15.47:60824) 2024-03-21T11:48:34.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.888+0000 7f57a3e00640 1 -- 172.21.15.47:0/3659094102 learned_addr learned my addr 172.21.15.47:0/3659094102 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:34.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.888+0000 7f57a9400640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a4109560 0x7f57a41a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.888+0000 7f57a8a00640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57a40740c0 0x7f57a4199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.888+0000 7f57a9400640 1 -- 172.21.15.47:0/3659094102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a40756a0 msgr2=0x7f57a419a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.888+0000 7f57a9400640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a40756a0 0x7f57a419a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.888+0000 7f57a9400640 1 -- 172.21.15.47:0/3659094102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57a40740c0 msgr2=0x7f57a4199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.888+0000 7f57a9400640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57a40740c0 0x7f57a4199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.888+0000 7f57a9400640 1 -- 172.21.15.47:0/3659094102 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a41a1780 con 0x7f57a4109560 2024-03-21T11:48:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.888+0000 7f57a8a00640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57a40740c0 0x7f57a4199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:48:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.889+0000 7f57a9400640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a4109560 0x7f57a41a1110 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f579c037040 tx=0x7f579c035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.889+0000 7f57a1600640 1 -- 172.21.15.47:0/3659094102 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c039040 con 0x7f57a4109560 2024-03-21T11:48:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.889+0000 7f57a1600640 1 -- 172.21.15.47:0/3659094102 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f579c044070 con 0x7f57a4109560 2024-03-21T11:48:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.889+0000 7f57a1600640 1 -- 172.21.15.47:0/3659094102 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c004350 con 0x7f57a4109560 2024-03-21T11:48:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.889+0000 7f57ab400640 1 -- 172.21.15.47:0/3659094102 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a406b0c0 con 0x7f57a4109560 2024-03-21T11:48:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.890+0000 7f57ab400640 1 -- 172.21.15.47:0/3659094102 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a406b530 con 0x7f57a4109560 2024-03-21T11:48:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.894+0000 7f57ae18c740 1 Processor -- start 2024-03-21T11:48:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.895+0000 7f57a1600640 1 -- 172.21.15.47:0/3659094102 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f579c0044f0 con 0x7f57a4109560 2024-03-21T11:48:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.895+0000 7f57ae18c740 1 -- start start 2024-03-21T11:48:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.897+0000 7f57ae18c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564e690 0x55ba1564eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.897+0000 7f57ae18c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564f080 0x55ba15653b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.897+0000 7f57ae18c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba156541f0 0x55ba15656680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.897+0000 7f57ae18c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba155ab250 con 0x55ba156541f0 2024-03-21T11:48:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.897+0000 7f57ae18c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba15553a30 con 0x55ba1564e690 2024-03-21T11:48:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.897+0000 7f57ae18c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba155a72f0 con 0x55ba1564f080 2024-03-21T11:48:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.897+0000 7f57a1600640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f578c0a30f0 0x7f578c0a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.897+0000 7f57a1600640 1 -- 172.21.15.47:0/3659094102 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f579c0f1c90 con 0x7f57a4109560 2024-03-21T11:48:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.897+0000 7f57a3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564e690 0x55ba1564eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.897+0000 7f57a8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564f080 0x55ba15653b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.897+0000 7f57a8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564f080 0x55ba15653b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36074/0 (socket says 172.21.15.47:36074) 2024-03-21T11:48:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.898+0000 7f57a8a00640 1 -- 172.21.15.47:0/3881042944 learned_addr learned my addr 172.21.15.47:0/3881042944 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.897+0000 7f57a3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564e690 0x55ba1564eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60836/0 (socket says 172.21.15.47:60836) 2024-03-21T11:48:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.898+0000 7f57a3e00640 1 --2- 172.21.15.47:0/3881042944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba156541f0 0x55ba15656680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.898+0000 7f57a9400640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f578c0a30f0 0x7f578c0a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.898+0000 7f57a3e00640 1 -- 172.21.15.47:0/3881042944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564f080 msgr2=0x55ba15653b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.898+0000 7f57a3e00640 1 --2- 172.21.15.47:0/3881042944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564f080 0x55ba15653b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.898+0000 7f57a3e00640 1 -- 172.21.15.47:0/3881042944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba156541f0 msgr2=0x55ba15656680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.898+0000 7f57a3e00640 1 --2- 172.21.15.47:0/3881042944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba156541f0 0x55ba15656680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.898+0000 7f57a3e00640 1 -- 172.21.15.47:0/3881042944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba1553e690 con 0x55ba1564e690 2024-03-21T11:48:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.899+0000 7f57a3e00640 1 --2- 172.21.15.47:0/3881042944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564e690 0x55ba1564eab0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5790009620 tx=0x7f5790009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed4496122ea25cb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.899+0000 7f57a9400640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f578c0a30f0 0x7f578c0a55b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f579c0bdcd0 tx=0x7f579c02c100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.900+0000 7f5795c00640 1 -- 172.21.15.47:0/3881042944 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790004030 con 0x55ba1564e690 2024-03-21T11:48:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.900+0000 7f5795c00640 1 -- 172.21.15.47:0/3881042944 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5790005500 con 0x55ba1564e690 2024-03-21T11:48:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.900+0000 7f5795c00640 1 -- 172.21.15.47:0/3881042944 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790004cb0 con 0x55ba1564e690 2024-03-21T11:48:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.901+0000 7f57ae18c740 1 -- 172.21.15.47:0/3881042944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564e690 msgr2=0x55ba1564eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.901+0000 7f57ae18c740 1 --2- 172.21.15.47:0/3881042944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564e690 0x55ba1564eab0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5790009620 tx=0x7f5790009530 comp rx=0 tx=0).stop 2024-03-21T11:48:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.901+0000 7f57ae18c740 1 -- 172.21.15.47:0/3881042944 shutdown_connections 2024-03-21T11:48:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.901+0000 7f57ae18c740 1 --2- 172.21.15.47:0/3881042944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba156541f0 0x55ba15656680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.901+0000 7f57ae18c740 1 --2- 172.21.15.47:0/3881042944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564f080 0x55ba15653b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.901+0000 7f57ae18c740 1 --2- 172.21.15.47:0/3881042944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564e690 0x55ba1564eab0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.901+0000 7f57ae18c740 1 -- 172.21.15.47:0/3881042944 >> 172.21.15.47:0/3881042944 conn(0x55ba1560e200 msgr2=0x55ba156417f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:34.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.902+0000 7f57ae18c740 1 -- 172.21.15.47:0/3881042944 shutdown_connections 2024-03-21T11:48:34.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.902+0000 7f57ae18c740 1 -- 172.21.15.47:0/3881042944 wait complete. 2024-03-21T11:48:34.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.902+0000 7f57ae18c740 1 -- start start 2024-03-21T11:48:34.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.902+0000 7f57ae18c740 1 Processor -- start 2024-03-21T11:48:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.903+0000 7f57ae18c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564e690 0x55ba156586d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.903+0000 7f57ae18c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564f080 0x55ba15658c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.903+0000 7f57ae18c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba156541f0 0x55ba1565fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.903+0000 7f57ae18c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba156571a0 con 0x55ba156541f0 2024-03-21T11:48:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.903+0000 7f57ae18c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba15566e80 con 0x55ba1564f080 2024-03-21T11:48:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.903+0000 7f57ae18c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba15657320 con 0x55ba1564e690 2024-03-21T11:48:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.903+0000 7f57ae18c740 10 client.0 _ll_register_callbacks cb 0x55ba15579130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.903+0000 7f57a8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564f080 0x55ba15658c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.903+0000 7f57a8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564f080 0x55ba15658c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60848/0 (socket says 172.21.15.47:60848) 2024-03-21T11:48:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.903+0000 7f57a8a00640 1 -- 172.21.15.47:0/1808657446 learned_addr learned my addr 172.21.15.47:0/1808657446 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:34.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.903+0000 7f57a3e00640 1 --2- 172.21.15.47:0/1808657446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba156541f0 0x55ba1565fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.904+0000 7f57a8a00640 1 -- 172.21.15.47:0/1808657446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564e690 msgr2=0x55ba156586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.904+0000 7f57a3e00640 1 --2- 172.21.15.47:0/1808657446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564e690 0x55ba156586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.904+0000 7f57a8a00640 1 --2- 172.21.15.47:0/1808657446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564e690 0x55ba156586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.904+0000 7f57a8a00640 1 -- 172.21.15.47:0/1808657446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba156541f0 msgr2=0x55ba1565fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.904+0000 7f57a8a00640 1 --2- 172.21.15.47:0/1808657446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba156541f0 0x55ba1565fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.904+0000 7f57a8a00640 1 -- 172.21.15.47:0/1808657446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba153897d0 con 0x55ba1564f080 2024-03-21T11:48:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.904+0000 7f57a8a00640 1 --2- 172.21.15.47:0/1808657446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564f080 0x55ba15658c10 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5798009d00 tx=0x7f57980039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e7c5d31f3a3e760 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.904+0000 7f578a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:48:34.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.905+0000 7f578a000640 1 -- 172.21.15.47:0/1808657446 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798022860 con 0x55ba1564f080 2024-03-21T11:48:34.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.905+0000 7f578a000640 1 -- 172.21.15.47:0/1808657446 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f5798022a00 con 0x55ba1564f080 2024-03-21T11:48:34.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.905+0000 7f57ae18c740 1 -- 172.21.15.47:0/1808657446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5790005500 con 0x55ba1564f080 2024-03-21T11:48:34.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.905+0000 7f578a000640 1 -- 172.21.15.47:0/1808657446 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5798022c00 con 0x55ba1564f080 2024-03-21T11:48:34.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.905+0000 7f57ae18c740 10 client.33506 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.905+0000 7f57ae18c740 1 -- 172.21.15.47:0/1808657446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ba152bf3f0 con 0x55ba1564f080 2024-03-21T11:48:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.905+0000 7f578a000640 1 -- 172.21.15.47:0/1808657446 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5790005500 con 0x55ba1564f080 2024-03-21T11:48:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.905+0000 7f578a000640 1 -- 172.21.15.47:0/1808657446 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f579802a020 con 0x55ba1564f080 2024-03-21T11:48:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.906+0000 7f57ae18c740 10 client.33506 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.906+0000 7f57ae18c740 10 client.33506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.906+0000 7f57ae18c740 20 client.33506 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.906+0000 7f57ae18c740 1 --2- 172.21.15.47:0/1808657446 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ba15664500 0x55ba15666920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.906+0000 7f57ae18c740 4 client.33506 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:48:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.906+0000 7f57ae18c740 1 -- 172.21.15.47:0/1808657446 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba155b8790 con 0x55ba15664500 2024-03-21T11:48:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.906+0000 7f57a3e00640 1 --2- 172.21.15.47:0/1808657446 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ba15664500 0x55ba15666920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.907+0000 7f57a3e00640 1 --2- 172.21.15.47:0/1808657446 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ba15664500 0x55ba15666920 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b071c9533af8390 server_cookie=afe0fe0af90c552 in_seq=0 out_seq=0 2024-03-21T11:48:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.907+0000 7f578a000640 10 client.33506 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.912+0000 7f578a000640 1 -- 172.21.15.47:0/1808657446 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ba155b8790 con 0x55ba15664500 2024-03-21T11:48:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.912+0000 7f578a000640 10 client.33506 handle_command_reply: tid=0 2024-03-21T11:48:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.912+0000 7f57ae18c740 1 client.33506 shutdown 2024-03-21T11:48:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.912+0000 7f57ae18c740 1 -- 172.21.15.47:0/1808657446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564f080 msgr2=0x55ba15658c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.912+0000 7f57ae18c740 1 --2- 172.21.15.47:0/1808657446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564f080 0x55ba15658c10 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5798009d00 tx=0x7f57980039c0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.912+0000 7f57ae18c740 1 -- 172.21.15.47:0/1808657446 shutdown_connections 2024-03-21T11:48:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.912+0000 7f57ae18c740 1 --2- 172.21.15.47:0/1808657446 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ba15664500 0x55ba15666920 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.912+0000 7f57ae18c740 1 --2- 172.21.15.47:0/1808657446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba156541f0 0x55ba1565fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.912+0000 7f578a000640 0 client.33506 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.912+0000 7f57ae18c740 1 --2- 172.21.15.47:0/1808657446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564f080 0x55ba15658c10 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.912+0000 7f57ae18c740 1 --2- 172.21.15.47:0/1808657446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564e690 0x55ba156586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.913+0000 7f57ae18c740 1 -- 172.21.15.47:0/1808657446 >> 172.21.15.47:0/1808657446 conn(0x55ba1560e200 msgr2=0x55ba15640a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:34.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.913+0000 7f57ae18c740 1 -- 172.21.15.47:0/1808657446 shutdown_connections 2024-03-21T11:48:34.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.913+0000 7f57ae18c740 1 -- 172.21.15.47:0/1808657446 wait complete. 2024-03-21T11:48:34.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.913+0000 7f57ae18c740 20 client.33506 trim_cache size 0 max 16384 2024-03-21T11:48:34.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.934+0000 7f57ae18c740 1 Processor -- start 2024-03-21T11:48:34.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.934+0000 7f57ae18c740 1 -- start start 2024-03-21T11:48:34.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.935+0000 7f57ae18c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba1564e690 0x55ba15653700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.935+0000 7f57ae18c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564f080 0x55ba1565fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.935+0000 7f57ae18c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba156541f0 0x55ba15644a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.935+0000 7f57ae18c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a40714c0 con 0x55ba1564e690 2024-03-21T11:48:34.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.935+0000 7f57ae18c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a4072e40 con 0x55ba156541f0 2024-03-21T11:48:34.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.935+0000 7f57ae18c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a4071b20 con 0x55ba1564f080 2024-03-21T11:48:34.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.935+0000 7f57a3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba1564e690 0x55ba15653700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.935+0000 7f57a8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564f080 0x55ba1565fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.935+0000 7f57a3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba1564e690 0x55ba15653700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57674/0 (socket says 172.21.15.47:57674) 2024-03-21T11:48:34.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.935+0000 7f57a3e00640 1 -- 172.21.15.47:0/679578577 learned_addr learned my addr 172.21.15.47:0/679578577 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:34.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.935+0000 7f57a3e00640 1 --2- 172.21.15.47:0/679578577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba156541f0 0x55ba15644a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.936+0000 7f57a8a00640 1 -- 172.21.15.47:0/679578577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba156541f0 msgr2=0x55ba15644a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.936+0000 7f57a8a00640 1 --2- 172.21.15.47:0/679578577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba156541f0 0x55ba15644a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.936+0000 7f57a8a00640 1 -- 172.21.15.47:0/679578577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba1564e690 msgr2=0x55ba15653700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.936+0000 7f57a8a00640 1 --2- 172.21.15.47:0/679578577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba1564e690 0x55ba15653700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.936+0000 7f57a8a00640 1 -- 172.21.15.47:0/679578577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba1527f0f0 con 0x55ba1564f080 2024-03-21T11:48:34.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.936+0000 7f57a8a00640 1 --2- 172.21.15.47:0/679578577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564f080 0x55ba1565fc10 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f5798020680 tx=0x7f5798020470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe6e7ea396c25e87 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:34.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.936+0000 7f5795200640 1 -- 172.21.15.47:0/679578577 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579803e070 con 0x55ba1564f080 2024-03-21T11:48:34.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.936+0000 7f5795200640 1 -- 172.21.15.47:0/679578577 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57980208f0 con 0x55ba1564f080 2024-03-21T11:48:34.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.936+0000 7f5795200640 1 -- 172.21.15.47:0/679578577 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798004810 con 0x55ba1564f080 2024-03-21T11:48:34.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.937+0000 7f57ae18c740 1 -- 172.21.15.47:0/679578577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564f080 msgr2=0x55ba1565fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.937+0000 7f57ae18c740 1 --2- 172.21.15.47:0/679578577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564f080 0x55ba1565fc10 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f5798020680 tx=0x7f5798020470 comp rx=0 tx=0).stop 2024-03-21T11:48:34.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.937+0000 7f57ae18c740 1 -- 172.21.15.47:0/679578577 shutdown_connections 2024-03-21T11:48:34.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.937+0000 7f57ae18c740 1 --2- 172.21.15.47:0/679578577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba156541f0 0x55ba15644a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.937+0000 7f57ae18c740 1 --2- 172.21.15.47:0/679578577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba1564f080 0x55ba1565fc10 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.937+0000 7f57ae18c740 1 --2- 172.21.15.47:0/679578577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba1564e690 0x55ba15653700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.937+0000 7f57ae18c740 1 -- 172.21.15.47:0/679578577 >> 172.21.15.47:0/679578577 conn(0x55ba1560e200 msgr2=0x55ba15640a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:34.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.937+0000 7f57ae18c740 1 -- 172.21.15.47:0/679578577 shutdown_connections 2024-03-21T11:48:34.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.937+0000 7f57ae18c740 1 -- 172.21.15.47:0/679578577 wait complete. 2024-03-21T11:48:34.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.938+0000 7f57ae18c740 1 -- start start 2024-03-21T11:48:34.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.938+0000 7f57ae18c740 1 Processor -- start 2024-03-21T11:48:34.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.939+0000 7f57ae18c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba1564e690 0x55ba1565d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.939+0000 7f57ae18c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564f080 0x55ba1565ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.939+0000 7f57ae18c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba156541f0 0x55ba1565edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.939+0000 7f57ae18c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a40727e0 con 0x55ba1564e690 2024-03-21T11:48:34.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.939+0000 7f57ae18c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a4072180 con 0x55ba1564f080 2024-03-21T11:48:34.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.939+0000 7f57ae18c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a406f680 con 0x55ba156541f0 2024-03-21T11:48:34.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.939+0000 7f57ae18c740 10 client.0 _ll_register_callbacks cb 0x7f5798004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:34.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.939+0000 7f57a3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba1564e690 0x55ba1565d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.939+0000 7f57a3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba1564e690 0x55ba1565d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57686/0 (socket says 172.21.15.47:57686) 2024-03-21T11:48:34.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.939+0000 7f57a3e00640 1 -- 172.21.15.47:0/208939989 learned_addr learned my addr 172.21.15.47:0/208939989 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:34.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.939+0000 7f57a8a00640 1 --2- 172.21.15.47:0/208939989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564f080 0x55ba1565ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.940+0000 7f57a3e00640 1 --2- 172.21.15.47:0/208939989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba156541f0 0x55ba1565edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.940+0000 7f57a8a00640 1 -- 172.21.15.47:0/208939989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba156541f0 msgr2=0x55ba1565edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.940+0000 7f57a8a00640 1 --2- 172.21.15.47:0/208939989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba156541f0 0x55ba1565edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.940+0000 7f57a8a00640 1 -- 172.21.15.47:0/208939989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba1564e690 msgr2=0x55ba1565d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:34.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.940+0000 7f57a8a00640 1 --2- 172.21.15.47:0/208939989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba1564e690 0x55ba1565d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.940+0000 7f57a8a00640 1 -- 172.21.15.47:0/208939989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba1565f3a0 con 0x55ba1564f080 2024-03-21T11:48:34.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.940+0000 7f57a3e00640 1 --2- 172.21.15.47:0/208939989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba1564e690 0x55ba1565d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:48:34.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.940+0000 7f57a8a00640 1 --2- 172.21.15.47:0/208939989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564f080 0x55ba1565ddf0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5798020600 tx=0x7f579802d680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1a1b48a18239149 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:34.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.940+0000 7f578a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:48:34.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.941+0000 7f578a000640 1 -- 172.21.15.47:0/208939989 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579803e070 con 0x55ba1564f080 2024-03-21T11:48:34.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.941+0000 7f578a000640 1 -- 172.21.15.47:0/208939989 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f579802da00 con 0x55ba1564f080 2024-03-21T11:48:34.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.941+0000 7f57ae18c740 1 -- 172.21.15.47:0/208939989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57980208f0 con 0x55ba1564f080 2024-03-21T11:48:34.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.942+0000 7f578a000640 1 -- 172.21.15.47:0/208939989 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5798020aa0 con 0x55ba1564f080 2024-03-21T11:48:34.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.942+0000 7f57ae18c740 10 client.33512 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:34.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.942+0000 7f57ae18c740 1 -- 172.21.15.47:0/208939989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ba156c0260 con 0x55ba1564f080 2024-03-21T11:48:34.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.942+0000 7f578a000640 1 -- 172.21.15.47:0/208939989 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57980208f0 con 0x55ba1564f080 2024-03-21T11:48:34.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.942+0000 7f578a000640 1 -- 172.21.15.47:0/208939989 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f579802a040 con 0x55ba1564f080 2024-03-21T11:48:34.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.942+0000 7f57ae18c740 10 client.33512 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:34.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.942+0000 7f57ae18c740 10 client.33512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:34.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.942+0000 7f57ae18c740 20 client.33512 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:34.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.942+0000 7f57ae18c740 1 --2- 172.21.15.47:0/208939989 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ba156c0ab0 0x55ba156c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:34.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.942+0000 7f57ae18c740 4 client.33512 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:48:34.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.942+0000 7f57ae18c740 1 -- 172.21.15.47:0/208939989 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba155b2ae0 con 0x55ba156c0ab0 2024-03-21T11:48:34.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.943+0000 7f57a3e00640 1 --2- 172.21.15.47:0/208939989 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ba156c0ab0 0x55ba156c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:34.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.944+0000 7f578a000640 10 client.33512 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:34.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.944+0000 7f57a3e00640 1 --2- 172.21.15.47:0/208939989 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ba156c0ab0 0x55ba156c2ed0 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=889eb96836c427be server_cookie=62bf3b44669d603 in_seq=0 out_seq=0 2024-03-21T11:48:34.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.946+0000 7f578a000640 1 -- 172.21.15.47:0/208939989 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55ba155b2ae0 con 0x55ba156c0ab0 2024-03-21T11:48:34.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.946+0000 7f578a000640 10 client.33512 handle_command_reply: tid=0 2024-03-21T11:48:34.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.946+0000 7f57ae18c740 1 client.33512 shutdown 2024-03-21T11:48:34.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.946+0000 7f57ae18c740 1 -- 172.21.15.47:0/208939989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564f080 msgr2=0x55ba1565ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.946+0000 7f57ae18c740 1 --2- 172.21.15.47:0/208939989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564f080 0x55ba1565ddf0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5798020600 tx=0x7f579802d680 comp rx=0 tx=0).stop 2024-03-21T11:48:34.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.946+0000 7f57ae18c740 1 -- 172.21.15.47:0/208939989 shutdown_connections 2024-03-21T11:48:34.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.946+0000 7f57ae18c740 1 --2- 172.21.15.47:0/208939989 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55ba156c0ab0 0x55ba156c2ed0 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.946+0000 7f57ae18c740 1 --2- 172.21.15.47:0/208939989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ba156541f0 0x55ba1565edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.946+0000 7f57ae18c740 1 --2- 172.21.15.47:0/208939989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba1564f080 0x55ba1565ddf0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.946+0000 7f57ae18c740 1 --2- 172.21.15.47:0/208939989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba1564e690 0x55ba1565d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.946+0000 7f57ae18c740 1 -- 172.21.15.47:0/208939989 >> 172.21.15.47:0/208939989 conn(0x55ba1560e200 msgr2=0x55ba15644a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:34.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.946+0000 7f578a000640 0 client.33512 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:34.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.948+0000 7f57ae18c740 1 -- 172.21.15.47:0/208939989 shutdown_connections 2024-03-21T11:48:34.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.948+0000 7f57ae18c740 1 -- 172.21.15.47:0/208939989 wait complete. 2024-03-21T11:48:34.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.948+0000 7f57ae18c740 20 client.33512 trim_cache size 0 max 16384 2024-03-21T11:48:34.951 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:48:34.951 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (68 inodes in the stack)", 2024-03-21T11:48:34.951 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:48:34.952 INFO:teuthology.orchestra.run.smithi047.stdout: "778b6f3c-73e5-4d40-b923-7aa2169cb62f": { 2024-03-21T11:48:34.952 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:48:34.952 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "778b6f3c-73e5-4d40-b923-7aa2169cb62f", 2024-03-21T11:48:34.952 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:48:34.952 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:48:34.952 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:48:34.952 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:48:34.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.948+0000 7f57ab400640 1 -- 172.21.15.47:0/3659094102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f578c0a30f0 msgr2=0x7f578c0a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.948+0000 7f57ab400640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f578c0a30f0 0x7f578c0a55b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f579c0bdcd0 tx=0x7f579c02c100 comp rx=0 tx=0).stop 2024-03-21T11:48:34.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.948+0000 7f57ab400640 1 -- 172.21.15.47:0/3659094102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a4109560 msgr2=0x7f57a41a1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:34.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.948+0000 7f57ab400640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a4109560 0x7f57a41a1110 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f579c037040 tx=0x7f579c035d10 comp rx=0 tx=0).stop 2024-03-21T11:48:34.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.949+0000 7f57ab400640 1 -- 172.21.15.47:0/3659094102 shutdown_connections 2024-03-21T11:48:34.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.949+0000 7f57ab400640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f578c0a30f0 0x7f578c0a55b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.949+0000 7f57ab400640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57a4109560 0x7f57a41a1110 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.949+0000 7f57ab400640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a40756a0 0x7f57a419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.949+0000 7f57ab400640 1 --2- 172.21.15.47:0/3659094102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57a40740c0 0x7f57a4199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:34.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.949+0000 7f57ab400640 1 -- 172.21.15.47:0/3659094102 >> 172.21.15.47:0/3659094102 conn(0x7f57a40fcc40 msgr2=0x7f57a41054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:34.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.949+0000 7f57ab400640 1 -- 172.21.15.47:0/3659094102 shutdown_connections 2024-03-21T11:48:34.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:48:34.949+0000 7f57ab400640 1 -- 172.21.15.47:0/3659094102 wait complete. 2024-03-21T11:48:34.980 INFO:tasks.cephfs.filesystem:scrub status for tag:778b6f3c-73e5-4d40-b923-7aa2169cb62f - {'path': '/', 'tag': '778b6f3c-73e5-4d40-b923-7aa2169cb62f', 'options': 'recursive,force'} 2024-03-21T11:48:35.002 INFO:tasks.workunit.client.0.smithi047.stdout: 10 279821 21.39 MB/sec execute 558 sec latency 136.517 ms 2024-03-21T11:48:35.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:34 smithi047 ceph-mon[31351]: pgmap v1158: 129 pgs: 129 active+clean; 736 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 11 MiB/s wr, 681 op/s 2024-03-21T11:48:35.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:34 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:48:35.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:34 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:35.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:34 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:34 smithi102 ceph-mon[36748]: pgmap v1158: 129 pgs: 129 active+clean; 736 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 11 MiB/s wr, 681 op/s 2024-03-21T11:48:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:34 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:48:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:34 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:34 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:35.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:34 smithi100 ceph-mon[36245]: pgmap v1158: 129 pgs: 129 active+clean; 736 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 11 MiB/s wr, 681 op/s 2024-03-21T11:48:35.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:34 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:48:35.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:34 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:35.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:34 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:36.002 INFO:tasks.workunit.client.0.smithi047.stdout: 10 280310 21.39 MB/sec execute 559 sec latency 118.750 ms 2024-03-21T11:48:36.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:35 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:36.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:35 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:36.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:35 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:35 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:36.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:35 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:36.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:35 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:35 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:36.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:35 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:35 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:35 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:48:36.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:35 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:35 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:35 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:35 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:36.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:35 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:36.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:35 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:35 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:36.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:35 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:36.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:35 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:35 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:36.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:35 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:35 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:35 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:48:36.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:35 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:35 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:35 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:35 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:35 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:35 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:35 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:35 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:35 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:35 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:36.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:35 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:35 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:35 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:48:36.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:35 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:35 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:36.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:35 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:48:37.002 INFO:tasks.workunit.client.0.smithi047.stdout: 10 280921 21.42 MB/sec execute 560 sec latency 57.463 ms 2024-03-21T11:48:37.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:36 smithi102 ceph-mon[36748]: pgmap v1159: 129 pgs: 129 active+clean; 726 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 10 MiB/s wr, 599 op/s 2024-03-21T11:48:37.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:36 smithi047 ceph-mon[31351]: pgmap v1159: 129 pgs: 129 active+clean; 726 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 10 MiB/s wr, 599 op/s 2024-03-21T11:48:37.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:36 smithi100 ceph-mon[36245]: pgmap v1159: 129 pgs: 129 active+clean; 726 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 10 MiB/s wr, 599 op/s 2024-03-21T11:48:38.002 INFO:tasks.workunit.client.0.smithi047.stdout: 10 281438 21.43 MB/sec execute 561 sec latency 57.611 ms 2024-03-21T11:48:38.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:48:38.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:48:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:48:39.002 INFO:tasks.workunit.client.0.smithi047.stdout: 10 281792 21.43 MB/sec execute 562 sec latency 69.073 ms 2024-03-21T11:48:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:38 smithi102 ceph-mon[36748]: pgmap v1160: 129 pgs: 129 active+clean; 721 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.2 MiB/s wr, 668 op/s 2024-03-21T11:48:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:38 smithi047 ceph-mon[31351]: pgmap v1160: 129 pgs: 129 active+clean; 721 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.2 MiB/s wr, 668 op/s 2024-03-21T11:48:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:38 smithi100 ceph-mon[36245]: pgmap v1160: 129 pgs: 129 active+clean; 721 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.2 MiB/s wr, 668 op/s 2024-03-21T11:48:40.002 INFO:tasks.workunit.client.0.smithi047.stdout: 10 282198 21.44 MB/sec execute 563 sec latency 95.082 ms 2024-03-21T11:48:40.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:39 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:48:40.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:39 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:48:40.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:39 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:48:40.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:39 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:48:40.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:39 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:48:40.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:39 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:48:40.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:39 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:48:40.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:39 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:48:40.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:39 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:48:41.002 INFO:tasks.workunit.client.0.smithi047.stdout: 10 282651 21.44 MB/sec execute 564 sec latency 53.893 ms 2024-03-21T11:48:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.212+0000 7f144f000640 1 -- 172.21.15.47:0/2906709102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14481052d0 msgr2=0x7f144810bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.212+0000 7f144f000640 1 --2- 172.21.15.47:0/2906709102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14481052d0 0x7f144810bb60 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f1440009ca0 tx=0x7f144002c100 comp rx=0 tx=0).stop 2024-03-21T11:48:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.212+0000 7f144f000640 1 -- 172.21.15.47:0/2906709102 shutdown_connections 2024-03-21T11:48:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.212+0000 7f144f000640 1 --2- 172.21.15.47:0/2906709102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14481052d0 0x7f144810bb60 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.212+0000 7f144f000640 1 --2- 172.21.15.47:0/2906709102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1448104910 0x7f1448104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.212+0000 7f144f000640 1 --2- 172.21.15.47:0/2906709102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1448103720 0x7f1448103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.212+0000 7f144f000640 1 -- 172.21.15.47:0/2906709102 >> 172.21.15.47:0/2906709102 conn(0x7f14480fecb0 msgr2=0x7f14481010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.212+0000 7f144f000640 1 -- 172.21.15.47:0/2906709102 shutdown_connections 2024-03-21T11:48:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.212+0000 7f144f000640 1 -- 172.21.15.47:0/2906709102 wait complete. 2024-03-21T11:48:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.213+0000 7f144f000640 1 Processor -- start 2024-03-21T11:48:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.213+0000 7f144f000640 1 -- start start 2024-03-21T11:48:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.213+0000 7f144f000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1448103720 0x7f14481999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144f000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1448104910 0x7f1448199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14481052d0 0x7f14481a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144f000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144810f3a0 con 0x7f1448103720 2024-03-21T11:48:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144810f220 con 0x7f14481052d0 2024-03-21T11:48:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144f000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144810f520 con 0x7f1448104910 2024-03-21T11:48:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1448104910 0x7f1448199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1448103720 0x7f14481999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1448104910 0x7f1448199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36172/0 (socket says 172.21.15.47:36172) 2024-03-21T11:48:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144d200640 1 -- 172.21.15.47:0/2217781647 learned_addr learned my addr 172.21.15.47:0/2217781647 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144e600640 1 --2- 172.21.15.47:0/2217781647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14481052d0 0x7f14481a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144dc00640 1 -- 172.21.15.47:0/2217781647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1448104910 msgr2=0x7f1448199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144dc00640 1 --2- 172.21.15.47:0/2217781647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1448104910 0x7f1448199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144dc00640 1 -- 172.21.15.47:0/2217781647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14481052d0 msgr2=0x7f14481a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144dc00640 1 --2- 172.21.15.47:0/2217781647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14481052d0 0x7f14481a0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144dc00640 1 -- 172.21.15.47:0/2217781647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14481a15d0 con 0x7f1448103720 2024-03-21T11:48:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.214+0000 7f144d200640 1 --2- 172.21.15.47:0/2217781647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1448104910 0x7f1448199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:48:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.215+0000 7f144dc00640 1 --2- 172.21.15.47:0/2217781647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1448103720 0x7f14481999b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f143c00e9c0 tx=0x7f143c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.215+0000 7f1446000640 1 -- 172.21.15.47:0/2217781647 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c00cd80 con 0x7f1448103720 2024-03-21T11:48:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.215+0000 7f144f000640 1 -- 172.21.15.47:0/2217781647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14481a18c0 con 0x7f1448103720 2024-03-21T11:48:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.215+0000 7f1446000640 1 -- 172.21.15.47:0/2217781647 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f143c010910 con 0x7f1448103720 2024-03-21T11:48:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.215+0000 7f1446000640 1 -- 172.21.15.47:0/2217781647 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c018980 con 0x7f1448103720 2024-03-21T11:48:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.215+0000 7f144f000640 1 -- 172.21.15.47:0/2217781647 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14481a1e60 con 0x7f1448103720 2024-03-21T11:48:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.219+0000 7f14533e6740 1 Processor -- start 2024-03-21T11:48:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.219+0000 7f14533e6740 1 -- start start 2024-03-21T11:48:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f14533e6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f210703f0 0x556f21070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f14533e6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21070de0 0x556f210758e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f14533e6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21075f50 0x556f210783e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f14533e6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f210333b0 con 0x556f21070de0 2024-03-21T11:48:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f14533e6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f20fd5830 con 0x556f210703f0 2024-03-21T11:48:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f14533e6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f20f7e5d0 con 0x556f21075f50 2024-03-21T11:48:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f144e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f210703f0 0x556f21070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f144d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21075f50 0x556f210783e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f144e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f210703f0 0x556f21070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52936/0 (socket says 172.21.15.47:52936) 2024-03-21T11:48:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f144e600640 1 -- 172.21.15.47:0/4052068065 learned_addr learned my addr 172.21.15.47:0/4052068065 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f144e600640 1 -- 172.21.15.47:0/4052068065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21075f50 msgr2=0x556f210783e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f144e600640 1 --2- 172.21.15.47:0/4052068065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21075f50 0x556f210783e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f144e600640 1 -- 172.21.15.47:0/4052068065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21070de0 msgr2=0x556f210758e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:48:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f144e600640 1 --2- 172.21.15.47:0/4052068065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21070de0 0x556f210758e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f144e600640 1 -- 172.21.15.47:0/4052068065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f20f962c0 con 0x556f210703f0 2024-03-21T11:48:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.220+0000 7f144d200640 1 --2- 172.21.15.47:0/4052068065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21075f50 0x556f210783e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:48:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.221+0000 7f144e600640 1 --2- 172.21.15.47:0/4052068065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f210703f0 0x556f21070810 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f1440005e30 tx=0x7f14400323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe2682512beb44b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.221+0000 7f1446000640 1 -- 172.21.15.47:0/2217781647 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f143c01f020 con 0x7f1448103720 2024-03-21T11:48:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.221+0000 7f143aa00640 1 -- 172.21.15.47:0/4052068065 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440043020 con 0x556f210703f0 2024-03-21T11:48:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.221+0000 7f143aa00640 1 -- 172.21.15.47:0/4052068065 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1440032c90 con 0x556f210703f0 2024-03-21T11:48:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.221+0000 7f143aa00640 1 -- 172.21.15.47:0/4052068065 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440039040 con 0x556f210703f0 2024-03-21T11:48:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.222+0000 7f14533e6740 1 -- 172.21.15.47:0/4052068065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f210703f0 msgr2=0x556f21070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.222+0000 7f14533e6740 1 --2- 172.21.15.47:0/4052068065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f210703f0 0x556f21070810 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f1440005e30 tx=0x7f14400323d0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.222+0000 7f14533e6740 1 -- 172.21.15.47:0/4052068065 shutdown_connections 2024-03-21T11:48:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.222+0000 7f14533e6740 1 --2- 172.21.15.47:0/4052068065 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21075f50 0x556f210783e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.222+0000 7f14533e6740 1 --2- 172.21.15.47:0/4052068065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21070de0 0x556f210758e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.222+0000 7f14533e6740 1 --2- 172.21.15.47:0/4052068065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f210703f0 0x556f21070810 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.222+0000 7f14533e6740 1 -- 172.21.15.47:0/4052068065 >> 172.21.15.47:0/4052068065 conn(0x556f2106b960 msgr2=0x556f2106bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.223+0000 7f14533e6740 1 -- 172.21.15.47:0/4052068065 shutdown_connections 2024-03-21T11:48:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.223+0000 7f14533e6740 1 -- 172.21.15.47:0/4052068065 wait complete. 2024-03-21T11:48:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.223+0000 7f14533e6740 1 -- start start 2024-03-21T11:48:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.224+0000 7f14533e6740 1 Processor -- start 2024-03-21T11:48:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.224+0000 7f1446000640 1 --2- 172.21.15.47:0/2217781647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14300a2eb0 0x7f14300a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.224+0000 7f1446000640 1 -- 172.21.15.47:0/2217781647 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f143c0cf840 con 0x7f1448103720 2024-03-21T11:48:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.224+0000 7f14533e6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f210703f0 0x556f21082d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.225+0000 7f144e600640 1 --2- 172.21.15.47:0/2217781647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14300a2eb0 0x7f14300a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.225+0000 7f14533e6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21070de0 0x556f210832c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.225+0000 7f14533e6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f21075f50 0x556f2108a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.225+0000 7f144e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f21075f50 0x556f2108a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.225+0000 7f144d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21070de0 0x556f210832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.225+0000 7f144e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f21075f50 0x556f2108a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52944/0 (socket says 172.21.15.47:52944) 2024-03-21T11:48:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.225+0000 7f144e600640 1 -- 172.21.15.47:0/2290835659 learned_addr learned my addr 172.21.15.47:0/2290835659 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.225+0000 7f14533e6740 1 -- 172.21.15.47:0/2290835659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f20f91a20 con 0x556f210703f0 2024-03-21T11:48:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.225+0000 7f14533e6740 1 -- 172.21.15.47:0/2290835659 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f20fd18d0 con 0x556f21075f50 2024-03-21T11:48:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.225+0000 7f14533e6740 1 -- 172.21.15.47:0/2290835659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f21078ea0 con 0x556f21070de0 2024-03-21T11:48:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.225+0000 7f14533e6740 10 client.0 _ll_register_callbacks cb 0x556f20fce1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.226+0000 7f144d200640 1 -- 172.21.15.47:0/2290835659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f21075f50 msgr2=0x556f2108a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.226+0000 7f144d200640 1 --2- 172.21.15.47:0/2290835659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f21075f50 0x556f2108a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.226+0000 7f144d200640 1 -- 172.21.15.47:0/2290835659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f210703f0 msgr2=0x556f21082d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:48:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.226+0000 7f144d200640 1 --2- 172.21.15.47:0/2290835659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f210703f0 0x556f21082d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.226+0000 7f144d200640 1 -- 172.21.15.47:0/2290835659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f20cea210 con 0x556f21070de0 2024-03-21T11:48:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.227+0000 7f144e600640 1 --2- 172.21.15.47:0/2217781647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14300a2eb0 0x7f14300a5370 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f144002f020 tx=0x7f1440032920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.227+0000 7f144d200640 1 --2- 172.21.15.47:0/2290835659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21070de0 0x556f210832c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f1434009640 tx=0x7f143400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a3ae0f31623b893 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.227+0000 7f142f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:48:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.227+0000 7f142f400640 1 -- 172.21.15.47:0/2290835659 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14340120d0 con 0x556f21070de0 2024-03-21T11:48:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.227+0000 7f142f400640 1 -- 172.21.15.47:0/2290835659 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1434005020 con 0x556f21070de0 2024-03-21T11:48:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.228+0000 7f14533e6740 1 -- 172.21.15.47:0/2290835659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1440032c90 con 0x556f21070de0 2024-03-21T11:48:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.228+0000 7f142f400640 1 -- 172.21.15.47:0/2290835659 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1434012270 con 0x556f21070de0 2024-03-21T11:48:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.228+0000 7f14533e6740 10 client.33460 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.228+0000 7f14533e6740 1 -- 172.21.15.47:0/2290835659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556f20db6620 con 0x556f21070de0 2024-03-21T11:48:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.229+0000 7f142f400640 1 -- 172.21.15.47:0/2290835659 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1440032c90 con 0x556f21070de0 2024-03-21T11:48:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.229+0000 7f142f400640 1 -- 172.21.15.47:0/2290835659 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f143401c470 con 0x556f21070de0 2024-03-21T11:48:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.229+0000 7f14533e6740 10 client.33460 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.229+0000 7f14533e6740 10 client.33460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.229+0000 7f14533e6740 20 client.33460 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.229+0000 7f14533e6740 1 --2- 172.21.15.47:0/2290835659 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f2108ebd0 0x556f21091000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.229+0000 7f14533e6740 4 client.33460 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:48:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.229+0000 7f14533e6740 1 -- 172.21.15.47:0/2290835659 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f20fe2d70 con 0x556f2108ebd0 2024-03-21T11:48:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.230+0000 7f144dc00640 1 --2- 172.21.15.47:0/2290835659 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f2108ebd0 0x556f21091000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.231+0000 7f144dc00640 1 --2- 172.21.15.47:0/2290835659 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f2108ebd0 0x556f21091000 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f318ae6d66683fa5 server_cookie=2db02bf25b6a7dc3 in_seq=0 out_seq=0 2024-03-21T11:48:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.231+0000 7f142f400640 10 client.33460 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.235+0000 7f142f400640 1 -- 172.21.15.47:0/2290835659 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556f20fe2d70 con 0x556f2108ebd0 2024-03-21T11:48:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.236+0000 7f142f400640 10 client.33460 handle_command_reply: tid=0 2024-03-21T11:48:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.236+0000 7f14533e6740 1 client.33460 shutdown 2024-03-21T11:48:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.236+0000 7f14533e6740 1 -- 172.21.15.47:0/2290835659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21070de0 msgr2=0x556f210832c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.236+0000 7f14533e6740 1 --2- 172.21.15.47:0/2290835659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21070de0 0x556f210832c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f1434009640 tx=0x7f143400f790 comp rx=0 tx=0).stop 2024-03-21T11:48:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.236+0000 7f14533e6740 1 -- 172.21.15.47:0/2290835659 shutdown_connections 2024-03-21T11:48:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.236+0000 7f14533e6740 1 --2- 172.21.15.47:0/2290835659 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f2108ebd0 0x556f21091000 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.236+0000 7f14533e6740 1 --2- 172.21.15.47:0/2290835659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f21075f50 0x556f2108a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.236+0000 7f14533e6740 1 --2- 172.21.15.47:0/2290835659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21070de0 0x556f210832c0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.236+0000 7f142f400640 0 client.33460 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.236+0000 7f14533e6740 1 --2- 172.21.15.47:0/2290835659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f210703f0 0x556f21082d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.236+0000 7f14533e6740 1 -- 172.21.15.47:0/2290835659 >> 172.21.15.47:0/2290835659 conn(0x556f2106b960 msgr2=0x556f2106ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.236+0000 7f14533e6740 1 -- 172.21.15.47:0/2290835659 shutdown_connections 2024-03-21T11:48:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.236+0000 7f14533e6740 1 -- 172.21.15.47:0/2290835659 wait complete. 2024-03-21T11:48:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.237+0000 7f14533e6740 20 client.33460 trim_cache size 0 max 16384 2024-03-21T11:48:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.254+0000 7f14533e6740 1 Processor -- start 2024-03-21T11:48:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.254+0000 7f14533e6740 1 -- start start 2024-03-21T11:48:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.255+0000 7f14533e6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f210703f0 0x556f21072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.255+0000 7f14533e6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21070de0 0x556f21072640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.255+0000 7f14533e6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21075f50 0x556f2108a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.255+0000 7f14533e6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14481a7c60 con 0x556f21075f50 2024-03-21T11:48:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.255+0000 7f14533e6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14481a95e0 con 0x556f210703f0 2024-03-21T11:48:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.255+0000 7f14533e6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14481a82c0 con 0x556f21070de0 2024-03-21T11:48:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.255+0000 7f144d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21070de0 0x556f21072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.255+0000 7f144d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21070de0 0x556f21072640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36206/0 (socket says 172.21.15.47:36206) 2024-03-21T11:48:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.255+0000 7f144d200640 1 -- 172.21.15.47:0/126276546 learned_addr learned my addr 172.21.15.47:0/126276546 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.255+0000 7f144dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f210703f0 0x556f21072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.255+0000 7f144e600640 1 --2- 172.21.15.47:0/126276546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21075f50 0x556f2108a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.256+0000 7f144d200640 1 -- 172.21.15.47:0/126276546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f210703f0 msgr2=0x556f21072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.256+0000 7f144d200640 1 --2- 172.21.15.47:0/126276546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f210703f0 0x556f21072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.256+0000 7f144d200640 1 -- 172.21.15.47:0/126276546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21075f50 msgr2=0x556f2108a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.256+0000 7f144d200640 1 --2- 172.21.15.47:0/126276546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21075f50 0x556f2108a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.256+0000 7f144d200640 1 -- 172.21.15.47:0/126276546 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f21070030 con 0x556f21070de0 2024-03-21T11:48:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.256+0000 7f144dc00640 1 --2- 172.21.15.47:0/126276546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f210703f0 0x556f21072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:48:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.256+0000 7f144d200640 1 --2- 172.21.15.47:0/126276546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21070de0 0x556f21072640 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f143401c890 tx=0x7f14340059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb9cf1cb8b1bd4f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.256+0000 7f143a000640 1 -- 172.21.15.47:0/126276546 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143401ca80 con 0x556f21070de0 2024-03-21T11:48:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.256+0000 7f143a000640 1 -- 172.21.15.47:0/126276546 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1434005520 con 0x556f21070de0 2024-03-21T11:48:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.256+0000 7f143a000640 1 -- 172.21.15.47:0/126276546 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434004b70 con 0x556f21070de0 2024-03-21T11:48:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.257+0000 7f14533e6740 1 -- 172.21.15.47:0/126276546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21070de0 msgr2=0x556f21072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.257+0000 7f14533e6740 1 --2- 172.21.15.47:0/126276546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21070de0 0x556f21072640 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f143401c890 tx=0x7f14340059b0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.257+0000 7f14533e6740 1 -- 172.21.15.47:0/126276546 shutdown_connections 2024-03-21T11:48:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.257+0000 7f14533e6740 1 --2- 172.21.15.47:0/126276546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21075f50 0x556f2108a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.257+0000 7f14533e6740 1 --2- 172.21.15.47:0/126276546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f21070de0 0x556f21072640 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.257+0000 7f14533e6740 1 --2- 172.21.15.47:0/126276546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f210703f0 0x556f21072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.257+0000 7f14533e6740 1 -- 172.21.15.47:0/126276546 >> 172.21.15.47:0/126276546 conn(0x556f2106b960 msgr2=0x556f2106bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.258+0000 7f14533e6740 1 -- 172.21.15.47:0/126276546 shutdown_connections 2024-03-21T11:48:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.258+0000 7f14533e6740 1 -- 172.21.15.47:0/126276546 wait complete. 2024-03-21T11:48:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.258+0000 7f14533e6740 1 -- start start 2024-03-21T11:48:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.258+0000 7f14533e6740 1 Processor -- start 2024-03-21T11:48:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.259+0000 7f14533e6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f210703f0 0x556f21087ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.259+0000 7f14533e6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21070de0 0x556f210881e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.259+0000 7f14533e6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f21075f50 0x556f210891f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.259+0000 7f14533e6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14481a8f80 con 0x556f21070de0 2024-03-21T11:48:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.259+0000 7f14533e6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14481a8920 con 0x556f21075f50 2024-03-21T11:48:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.259+0000 7f14533e6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14481a5e20 con 0x556f210703f0 2024-03-21T11:48:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.259+0000 7f14533e6740 10 client.0 _ll_register_callbacks cb 0x7f1434009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.259+0000 7f144dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f210703f0 0x556f21087ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.259+0000 7f144dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f210703f0 0x556f21087ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:36222/0 (socket says 172.21.15.47:36222) 2024-03-21T11:48:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.259+0000 7f144dc00640 1 -- 172.21.15.47:0/1486861859 learned_addr learned my addr 172.21.15.47:0/1486861859 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.259+0000 7f144e600640 1 --2- 172.21.15.47:0/1486861859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f21075f50 0x556f210891f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.259+0000 7f144d200640 1 --2- 172.21.15.47:0/1486861859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21070de0 0x556f210881e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.260+0000 7f144dc00640 1 -- 172.21.15.47:0/1486861859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f21075f50 msgr2=0x556f210891f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.260+0000 7f144dc00640 1 --2- 172.21.15.47:0/1486861859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f21075f50 0x556f210891f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.260+0000 7f144dc00640 1 -- 172.21.15.47:0/1486861859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21070de0 msgr2=0x556f210881e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.260+0000 7f144dc00640 1 --2- 172.21.15.47:0/1486861859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21070de0 0x556f210881e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.260+0000 7f144dc00640 1 -- 172.21.15.47:0/1486861859 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f21079010 con 0x556f210703f0 2024-03-21T11:48:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.260+0000 7f144d200640 1 --2- 172.21.15.47:0/1486861859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21070de0 0x556f210881e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:48:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.260+0000 7f144dc00640 1 --2- 172.21.15.47:0/1486861859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f210703f0 0x556f21087ca0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f143c0ccf50 tx=0x7f143c00cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85c834e89ef4997a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.260+0000 7f142f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:48:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.261+0000 7f142f400640 1 -- 172.21.15.47:0/1486861859 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c0a4590 con 0x556f210703f0 2024-03-21T11:48:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.261+0000 7f142f400640 1 -- 172.21.15.47:0/1486861859 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f143c0cd060 con 0x556f210703f0 2024-03-21T11:48:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.261+0000 7f14533e6740 1 -- 172.21.15.47:0/1486861859 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1434005520 con 0x556f210703f0 2024-03-21T11:48:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.261+0000 7f142f400640 1 -- 172.21.15.47:0/1486861859 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f143c010430 con 0x556f210703f0 2024-03-21T11:48:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.261+0000 7f14533e6740 10 client.33466 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.261+0000 7f14533e6740 1 -- 172.21.15.47:0/1486861859 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556f210ea6c0 con 0x556f210703f0 2024-03-21T11:48:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.261+0000 7f142f400640 1 -- 172.21.15.47:0/1486861859 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1434005520 con 0x556f210703f0 2024-03-21T11:48:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.261+0000 7f142f400640 1 -- 172.21.15.47:0/1486861859 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f143c010ac0 con 0x556f210703f0 2024-03-21T11:48:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.262+0000 7f14533e6740 10 client.33466 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.262+0000 7f14533e6740 10 client.33466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.262+0000 7f14533e6740 20 client.33466 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.262+0000 7f14533e6740 1 --2- 172.21.15.47:0/1486861859 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f210eaff0 0x556f210ed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.262+0000 7f14533e6740 4 client.33466 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:48:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.262+0000 7f14533e6740 1 -- 172.21.15.47:0/1486861859 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556f20fdd0c0 con 0x556f210eaff0 2024-03-21T11:48:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.262+0000 7f144d200640 1 --2- 172.21.15.47:0/1486861859 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f210eaff0 0x556f210ed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.263+0000 7f142f400640 10 client.33466 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:41.263+0000 7f144d200640 1 --2- 172.21.15.47:0/1486861859 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f210eaff0 0x556f210ed410 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e78fe07f8f849b1 server_cookie=f5730be7d25cf6f8 in_seq=0 out_seq=0 2024-03-21T11:48:41.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:41 smithi100 ceph-mon[36245]: pgmap v1161: 129 pgs: 129 active+clean; 725 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.6 MiB/s wr, 657 op/s 2024-03-21T11:48:41.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:41 smithi102 ceph-mon[36748]: pgmap v1161: 129 pgs: 129 active+clean; 725 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.6 MiB/s wr, 657 op/s 2024-03-21T11:48:41.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:41 smithi047 ceph-mon[31351]: pgmap v1161: 129 pgs: 129 active+clean; 725 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.6 MiB/s wr, 657 op/s 2024-03-21T11:48:42.002 INFO:tasks.workunit.client.0.smithi047.stdout: 10 282860 21.42 MB/sec execute 565 sec latency 571.809 ms 2024-03-21T11:48:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.131+0000 7f142f400640 1 -- 172.21.15.47:0/1486861859 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556f20fdd0c0 con 0x556f210eaff0 2024-03-21T11:48:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.131+0000 7f142f400640 10 client.33466 handle_command_reply: tid=0 2024-03-21T11:48:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.131+0000 7f14533e6740 1 client.33466 shutdown 2024-03-21T11:48:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.131+0000 7f14533e6740 1 -- 172.21.15.47:0/1486861859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f210703f0 msgr2=0x556f21087ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.132+0000 7f14533e6740 1 --2- 172.21.15.47:0/1486861859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f210703f0 0x556f21087ca0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f143c0ccf50 tx=0x7f143c00cf20 comp rx=0 tx=0).stop 2024-03-21T11:48:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.132+0000 7f14533e6740 1 -- 172.21.15.47:0/1486861859 shutdown_connections 2024-03-21T11:48:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.132+0000 7f14533e6740 1 --2- 172.21.15.47:0/1486861859 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x556f210eaff0 0x556f210ed410 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.132+0000 7f14533e6740 1 --2- 172.21.15.47:0/1486861859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f21075f50 0x556f210891f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.132+0000 7f142f400640 0 client.33466 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.132+0000 7f14533e6740 1 --2- 172.21.15.47:0/1486861859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f21070de0 0x556f210881e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.132+0000 7f14533e6740 1 --2- 172.21.15.47:0/1486861859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556f210703f0 0x556f21087ca0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.132+0000 7f14533e6740 1 -- 172.21.15.47:0/1486861859 >> 172.21.15.47:0/1486861859 conn(0x556f2106b960 msgr2=0x556f21082c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.132+0000 7f14533e6740 1 -- 172.21.15.47:0/1486861859 shutdown_connections 2024-03-21T11:48:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.132+0000 7f14533e6740 1 -- 172.21.15.47:0/1486861859 wait complete. 2024-03-21T11:48:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.132+0000 7f14533e6740 20 client.33466 trim_cache size 0 max 16384 2024-03-21T11:48:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:48:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:48:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:48:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:48:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.133+0000 7f144f000640 1 -- 172.21.15.47:0/2217781647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14300a2eb0 msgr2=0x7f14300a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.133+0000 7f144f000640 1 --2- 172.21.15.47:0/2217781647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14300a2eb0 0x7f14300a5370 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f144002f020 tx=0x7f1440032920 comp rx=0 tx=0).stop 2024-03-21T11:48:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.133+0000 7f144f000640 1 -- 172.21.15.47:0/2217781647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1448103720 msgr2=0x7f14481999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.133+0000 7f144f000640 1 --2- 172.21.15.47:0/2217781647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1448103720 0x7f14481999b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f143c00e9c0 tx=0x7f143c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:48:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.133+0000 7f144f000640 1 -- 172.21.15.47:0/2217781647 shutdown_connections 2024-03-21T11:48:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.133+0000 7f144f000640 1 --2- 172.21.15.47:0/2217781647 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f14300a2eb0 0x7f14300a5370 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.133+0000 7f144f000640 1 --2- 172.21.15.47:0/2217781647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14481052d0 0x7f14481a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.133+0000 7f144f000640 1 --2- 172.21.15.47:0/2217781647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1448104910 0x7f1448199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.133+0000 7f144f000640 1 --2- 172.21.15.47:0/2217781647 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1448103720 0x7f14481999b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.133+0000 7f144f000640 1 -- 172.21.15.47:0/2217781647 >> 172.21.15.47:0/2217781647 conn(0x7f14480fecb0 msgr2=0x7f1448107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.134+0000 7f144f000640 1 -- 172.21.15.47:0/2217781647 shutdown_connections 2024-03-21T11:48:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:42.134+0000 7f144f000640 1 -- 172.21.15.47:0/2217781647 wait complete. 2024-03-21T11:48:43.002 INFO:tasks.workunit.client.0.smithi047.stdout: 10 283404 21.42 MB/sec execute 566 sec latency 620.270 ms 2024-03-21T11:48:43.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:43 smithi100 ceph-mon[36245]: pgmap v1162: 129 pgs: 129 active+clean; 725 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.9 MiB/s wr, 504 op/s 2024-03-21T11:48:43.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:43 smithi102 ceph-mon[36748]: pgmap v1162: 129 pgs: 129 active+clean; 725 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.9 MiB/s wr, 504 op/s 2024-03-21T11:48:43.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:43 smithi047 ceph-mon[31351]: pgmap v1162: 129 pgs: 129 active+clean; 725 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.9 MiB/s wr, 504 op/s 2024-03-21T11:48:44.002 INFO:tasks.workunit.client.0.smithi047.stdout: 10 283820 21.42 MB/sec execute 567 sec latency 187.658 ms 2024-03-21T11:48:45.003 INFO:tasks.workunit.client.0.smithi047.stdout: 10 284392 21.44 MB/sec execute 568 sec latency 101.244 ms 2024-03-21T11:48:45.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:45 smithi100 ceph-mon[36245]: pgmap v1163: 129 pgs: 129 active+clean; 724 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.6 MiB/s wr, 614 op/s 2024-03-21T11:48:45.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:45 smithi102 ceph-mon[36748]: pgmap v1163: 129 pgs: 129 active+clean; 724 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.6 MiB/s wr, 614 op/s 2024-03-21T11:48:45.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:45 smithi047 ceph-mon[31351]: pgmap v1163: 129 pgs: 129 active+clean; 724 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.6 MiB/s wr, 614 op/s 2024-03-21T11:48:46.003 INFO:tasks.workunit.client.0.smithi047.stdout: 10 284842 21.44 MB/sec execute 569 sec latency 93.272 ms 2024-03-21T11:48:47.003 INFO:tasks.workunit.client.0.smithi047.stdout: 10 285309 21.47 MB/sec execute 570 sec latency 88.050 ms 2024-03-21T11:48:47.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:47 smithi100 ceph-mon[36245]: pgmap v1164: 129 pgs: 129 active+clean; 726 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.3 MiB/s wr, 470 op/s 2024-03-21T11:48:47.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:47 smithi102 ceph-mon[36748]: pgmap v1164: 129 pgs: 129 active+clean; 726 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.3 MiB/s wr, 470 op/s 2024-03-21T11:48:47.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:47 smithi047 ceph-mon[31351]: pgmap v1164: 129 pgs: 129 active+clean; 726 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.3 MiB/s wr, 470 op/s 2024-03-21T11:48:48.003 INFO:tasks.workunit.client.0.smithi047.stdout: 10 285934 21.48 MB/sec execute 571 sec latency 57.902 ms 2024-03-21T11:48:48.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:48 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:48:48.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:48 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:48:48.495 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:48 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:48:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:48 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:48:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:48 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:48:48.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:48 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:48:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:48 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:48:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:48 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:48:48.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:48 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:48:49.003 INFO:tasks.workunit.client.0.smithi047.stdout: 10 286333 21.48 MB/sec execute 572 sec latency 59.419 ms 2024-03-21T11:48:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:49 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:48:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:49 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:48:49.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:49 smithi102 ceph-mon[36748]: pgmap v1165: 129 pgs: 129 active+clean; 733 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.9 MiB/s wr, 552 op/s 2024-03-21T11:48:49.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:49 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:48:49.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:49 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:48:49.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:49 smithi047 ceph-mon[31351]: pgmap v1165: 129 pgs: 129 active+clean; 733 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.9 MiB/s wr, 552 op/s 2024-03-21T11:48:49.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:49 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:48:49.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:49 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:48:49.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:49 smithi100 ceph-mon[36245]: pgmap v1165: 129 pgs: 129 active+clean; 733 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.9 MiB/s wr, 552 op/s 2024-03-21T11:48:50.003 INFO:tasks.workunit.client.0.smithi047.stdout: 10 286507 21.46 MB/sec execute 573 sec latency 581.856 ms 2024-03-21T11:48:51.003 INFO:tasks.workunit.client.0.smithi047.stdout: 10 286687 21.44 MB/sec execute 574 sec latency 683.585 ms 2024-03-21T11:48:51.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:51 smithi102 ceph-mon[36748]: pgmap v1166: 129 pgs: 129 active+clean; 736 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.8 MiB/s wr, 483 op/s 2024-03-21T11:48:51.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:51 smithi047 ceph-mon[31351]: pgmap v1166: 129 pgs: 129 active+clean; 736 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.8 MiB/s wr, 483 op/s 2024-03-21T11:48:51.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:51 smithi100 ceph-mon[36245]: pgmap v1166: 129 pgs: 129 active+clean; 736 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.8 MiB/s wr, 483 op/s 2024-03-21T11:48:52.003 INFO:tasks.workunit.client.0.smithi047.stdout: 10 286880 21.42 MB/sec execute 575 sec latency 129.677 ms 2024-03-21T11:48:53.004 INFO:tasks.workunit.client.0.smithi047.stdout: 10 287308 21.42 MB/sec execute 576 sec latency 125.783 ms 2024-03-21T11:48:53.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:53 smithi102 ceph-mon[36748]: pgmap v1167: 129 pgs: 129 active+clean; 745 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 426 op/s 2024-03-21T11:48:53.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:53 smithi047 ceph-mon[31351]: pgmap v1167: 129 pgs: 129 active+clean; 745 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 426 op/s 2024-03-21T11:48:53.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:53 smithi100 ceph-mon[36245]: pgmap v1167: 129 pgs: 129 active+clean; 745 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 426 op/s 2024-03-21T11:48:54.004 INFO:tasks.workunit.client.0.smithi047.stdout: 10 287865 21.43 MB/sec execute 577 sec latency 133.387 ms 2024-03-21T11:48:55.004 INFO:tasks.workunit.client.0.smithi047.stdout: 10 288121 21.41 MB/sec execute 578 sec latency 544.963 ms 2024-03-21T11:48:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.365+0000 7fd661000640 1 -- 172.21.15.47:0/4163182115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c10ff20 msgr2=0x7fd65c112310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.365+0000 7fd661000640 1 --2- 172.21.15.47:0/4163182115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c10ff20 0x7fd65c112310 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd654009be0 tx=0x7fd65402bef0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.365+0000 7fd661000640 1 -- 172.21.15.47:0/4163182115 shutdown_connections 2024-03-21T11:48:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.365+0000 7fd661000640 1 --2- 172.21.15.47:0/4163182115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c10ff20 0x7fd65c112310 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.365+0000 7fd661000640 1 --2- 172.21.15.47:0/4163182115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd65c109000 0x7fd65c10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.365+0000 7fd661000640 1 --2- 172.21.15.47:0/4163182115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c1086a0 0x7fd65c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.365+0000 7fd661000640 1 -- 172.21.15.47:0/4163182115 >> 172.21.15.47:0/4163182115 conn(0x7fd65c06e6e0 msgr2=0x7fd65c070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.365+0000 7fd661000640 1 -- 172.21.15.47:0/4163182115 shutdown_connections 2024-03-21T11:48:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.365+0000 7fd661000640 1 -- 172.21.15.47:0/4163182115 wait complete. 2024-03-21T11:48:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.367+0000 7fd661000640 1 Processor -- start 2024-03-21T11:48:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.369+0000 7fd661000640 1 -- start start 2024-03-21T11:48:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.369+0000 7fd661000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd65c1086a0 0x7fd65c1a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.369+0000 7fd661000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c109000 0x7fd65c1a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.369+0000 7fd661000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c10ff20 0x7fd65c1a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.369+0000 7fd661000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c115b60 con 0x7fd65c109000 2024-03-21T11:48:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.369+0000 7fd661000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1159e0 con 0x7fd65c10ff20 2024-03-21T11:48:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.369+0000 7fd661000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c115ce0 con 0x7fd65c1086a0 2024-03-21T11:48:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.369+0000 7fd659e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd65c1086a0 0x7fd65c1a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.370+0000 7fd659e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd65c1086a0 0x7fd65c1a22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48742/0 (socket says 172.21.15.47:48742) 2024-03-21T11:48:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.370+0000 7fd659e00640 1 -- 172.21.15.47:0/727467793 learned_addr learned my addr 172.21.15.47:0/727467793 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.370+0000 7fd65a800640 1 --2- 172.21.15.47:0/727467793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c10ff20 0x7fd65c1a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.370+0000 7fd659400640 1 --2- 172.21.15.47:0/727467793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c109000 0x7fd65c1a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.370+0000 7fd659e00640 1 -- 172.21.15.47:0/727467793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c10ff20 msgr2=0x7fd65c1a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.370+0000 7fd659e00640 1 --2- 172.21.15.47:0/727467793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c10ff20 0x7fd65c1a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.370+0000 7fd659e00640 1 -- 172.21.15.47:0/727467793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c109000 msgr2=0x7fd65c1a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.370+0000 7fd659e00640 1 --2- 172.21.15.47:0/727467793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c109000 0x7fd65c1a27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.370+0000 7fd659e00640 1 -- 172.21.15.47:0/727467793 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c1a9ec0 con 0x7fd65c1086a0 2024-03-21T11:48:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.372+0000 7fd659e00640 1 --2- 172.21.15.47:0/727467793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd65c1086a0 0x7fd65c1a22a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00e9c0 tx=0x7fd64c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.372+0000 7fd652a00640 1 -- 172.21.15.47:0/727467793 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c00cde0 con 0x7fd65c1086a0 2024-03-21T11:48:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.372+0000 7fd652a00640 1 -- 172.21.15.47:0/727467793 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd64c0035f0 con 0x7fd65c1086a0 2024-03-21T11:48:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.372+0000 7fd652a00640 1 -- 172.21.15.47:0/727467793 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c010690 con 0x7fd65c1086a0 2024-03-21T11:48:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.372+0000 7fd661000640 1 -- 172.21.15.47:0/727467793 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c1aa1b0 con 0x7fd65c1086a0 2024-03-21T11:48:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.372+0000 7fd661000640 1 -- 172.21.15.47:0/727467793 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c1aa6b0 con 0x7fd65c1086a0 2024-03-21T11:48:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.382+0000 7fd652a00640 1 -- 172.21.15.47:0/727467793 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd64c0037a0 con 0x7fd65c1086a0 2024-03-21T11:48:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.385+0000 7fd663da3740 1 Processor -- start 2024-03-21T11:48:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.387+0000 7fd652a00640 1 --2- 172.21.15.47:0/727467793 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd6400a3040 0x7fd6400a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.387+0000 7fd652a00640 1 -- 172.21.15.47:0/727467793 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd64c0cfac0 con 0x7fd65c1086a0 2024-03-21T11:48:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.389+0000 7fd663da3740 1 -- start start 2024-03-21T11:48:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.389+0000 7fd663da3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f31e0 0x55a1661f3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.389+0000 7fd663da3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f3bd0 0x55a1661f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.390+0000 7fd663da3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f8d40 0x55a1661fb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.390+0000 7fd663da3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1661b63b0 con 0x55a1661f8d40 2024-03-21T11:48:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.390+0000 7fd663da3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a166158830 con 0x55a1661f3bd0 2024-03-21T11:48:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.390+0000 7fd663da3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1661015d0 con 0x55a1661f31e0 2024-03-21T11:48:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.390+0000 7fd659e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f31e0 0x55a1661f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.390+0000 7fd659e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f31e0 0x55a1661f3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48756/0 (socket says 172.21.15.47:48756) 2024-03-21T11:48:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.390+0000 7fd659e00640 1 -- 172.21.15.47:0/1303753537 learned_addr learned my addr 172.21.15.47:0/1303753537 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.391+0000 7fd659400640 1 --2- 172.21.15.47:0/1303753537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f3bd0 0x55a1661f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.391+0000 7fd65a800640 1 --2- 172.21.15.47:0/727467793 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd6400a3040 0x7fd6400a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.392+0000 7fd659e00640 1 -- 172.21.15.47:0/1303753537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f3bd0 msgr2=0x55a1661f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.392+0000 7fd659e00640 1 --2- 172.21.15.47:0/1303753537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f3bd0 0x55a1661f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.392+0000 7fd659e00640 1 -- 172.21.15.47:0/1303753537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f8d40 msgr2=0x55a1661fb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.392+0000 7fd659e00640 1 --2- 172.21.15.47:0/1303753537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f8d40 0x55a1661fb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.392+0000 7fd659e00640 1 -- 172.21.15.47:0/1303753537 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1661192c0 con 0x55a1661f31e0 2024-03-21T11:48:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.393+0000 7fd659e00640 1 --2- 172.21.15.47:0/1303753537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f31e0 0x55a1661f3600 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0a4340 tx=0x7fd64c098230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e61968e58412391 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.393+0000 7fd647400640 1 -- 172.21.15.47:0/1303753537 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c0bc070 con 0x55a1661f31e0 2024-03-21T11:48:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.394+0000 7fd647400640 1 -- 172.21.15.47:0/1303753537 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd64c09be60 con 0x55a1661f31e0 2024-03-21T11:48:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.394+0000 7fd647400640 1 -- 172.21.15.47:0/1303753537 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c0babd0 con 0x55a1661f31e0 2024-03-21T11:48:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.395+0000 7fd663da3740 1 -- 172.21.15.47:0/1303753537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f31e0 msgr2=0x55a1661f3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.395+0000 7fd663da3740 1 --2- 172.21.15.47:0/1303753537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f31e0 0x55a1661f3600 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0a4340 tx=0x7fd64c098230 comp rx=0 tx=0).stop 2024-03-21T11:48:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.395+0000 7fd65a800640 1 --2- 172.21.15.47:0/727467793 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd6400a3040 0x7fd6400a5500 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd654006010 tx=0x7fd654005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.397+0000 7fd663da3740 1 -- 172.21.15.47:0/1303753537 shutdown_connections 2024-03-21T11:48:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.397+0000 7fd663da3740 1 --2- 172.21.15.47:0/1303753537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f8d40 0x55a1661fb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.397+0000 7fd663da3740 1 --2- 172.21.15.47:0/1303753537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f3bd0 0x55a1661f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.397+0000 7fd663da3740 1 --2- 172.21.15.47:0/1303753537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f31e0 0x55a1661f3600 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.398+0000 7fd663da3740 1 -- 172.21.15.47:0/1303753537 >> 172.21.15.47:0/1303753537 conn(0x55a1661ee750 msgr2=0x55a1661eeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.398+0000 7fd663da3740 1 -- 172.21.15.47:0/1303753537 shutdown_connections 2024-03-21T11:48:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.398+0000 7fd663da3740 1 -- 172.21.15.47:0/1303753537 wait complete. 2024-03-21T11:48:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.399+0000 7fd663da3740 1 -- start start 2024-03-21T11:48:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.399+0000 7fd663da3740 1 Processor -- start 2024-03-21T11:48:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.400+0000 7fd663da3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f31e0 0x55a1661fd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.400+0000 7fd663da3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f3bd0 0x55a1661fd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.400+0000 7fd663da3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f8d40 0x55a166204910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.401+0000 7fd663da3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a166114a20 con 0x55a1661f3bd0 2024-03-21T11:48:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.401+0000 7fd663da3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1661548d0 con 0x55a1661f8d40 2024-03-21T11:48:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.401+0000 7fd663da3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1661fbc90 con 0x55a1661f31e0 2024-03-21T11:48:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.401+0000 7fd663da3740 10 client.0 _ll_register_callbacks cb 0x55a1661511c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.401+0000 7fd659e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f31e0 0x55a1661fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.401+0000 7fd659e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f31e0 0x55a1661fd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48766/0 (socket says 172.21.15.47:48766) 2024-03-21T11:48:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.401+0000 7fd659e00640 1 -- 172.21.15.47:0/1376681144 learned_addr learned my addr 172.21.15.47:0/1376681144 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.402+0000 7fd659400640 1 --2- 172.21.15.47:0/1376681144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f3bd0 0x55a1661fd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.402+0000 7fd65a800640 1 --2- 172.21.15.47:0/1376681144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f8d40 0x55a166204910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.402+0000 7fd659e00640 1 -- 172.21.15.47:0/1376681144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f8d40 msgr2=0x55a166204910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.402+0000 7fd659e00640 1 --2- 172.21.15.47:0/1376681144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f8d40 0x55a166204910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.402+0000 7fd659e00640 1 -- 172.21.15.47:0/1376681144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f3bd0 msgr2=0x55a1661fd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.402+0000 7fd659e00640 1 --2- 172.21.15.47:0/1376681144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f3bd0 0x55a1661fd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.404 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.403+0000 7fd659e00640 1 -- 172.21.15.47:0/1376681144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a165e6d210 con 0x55a1661f31e0 2024-03-21T11:48:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.405+0000 7fd659e00640 1 --2- 172.21.15.47:0/1376681144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f31e0 0x55a1661fd400 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fd64c09bdf0 tx=0x7fd64c0a0030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59f8a7e23fcad1dc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.405+0000 7fd63be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:48:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.405+0000 7fd63be00640 1 -- 172.21.15.47:0/1376681144 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c0badd0 con 0x55a1661f31e0 2024-03-21T11:48:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.405+0000 7fd63be00640 1 -- 172.21.15.47:0/1376681144 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd64c09b2b0 con 0x55a1661f31e0 2024-03-21T11:48:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.406+0000 7fd663da3740 1 -- 172.21.15.47:0/1376681144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd64c09be60 con 0x55a1661f31e0 2024-03-21T11:48:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.407+0000 7fd63be00640 1 -- 172.21.15.47:0/1376681144 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd64c0bc050 con 0x55a1661f31e0 2024-03-21T11:48:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.407+0000 7fd663da3740 10 client.33478 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.407+0000 7fd663da3740 1 -- 172.21.15.47:0/1376681144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a165f39620 con 0x55a1661f31e0 2024-03-21T11:48:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.407+0000 7fd63be00640 1 -- 172.21.15.47:0/1376681144 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd64c09be60 con 0x55a1661f31e0 2024-03-21T11:48:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.408+0000 7fd63be00640 1 -- 172.21.15.47:0/1376681144 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd64c09ad20 con 0x55a1661f31e0 2024-03-21T11:48:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.408+0000 7fd663da3740 10 client.33478 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.408+0000 7fd663da3740 10 client.33478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.408+0000 7fd663da3740 20 client.33478 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.408+0000 7fd663da3740 1 --2- 172.21.15.47:0/1376681144 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a166208e00 0x55a16620b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.408+0000 7fd663da3740 4 client.33478 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:48:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.408+0000 7fd663da3740 1 -- 172.21.15.47:0/1376681144 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a166165d70 con 0x55a166208e00 2024-03-21T11:48:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.413+0000 7fd659400640 1 --2- 172.21.15.47:0/1376681144 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a166208e00 0x55a16620b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.417+0000 7fd659400640 1 --2- 172.21.15.47:0/1376681144 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a166208e00 0x55a16620b1f0 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55039127e650cb5d server_cookie=6ac5ad2f84c513ff in_seq=0 out_seq=0 2024-03-21T11:48:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.417+0000 7fd63be00640 10 client.33478 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.420+0000 7fd63be00640 1 -- 172.21.15.47:0/1376681144 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a166165d70 con 0x55a166208e00 2024-03-21T11:48:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.420+0000 7fd63be00640 10 client.33478 handle_command_reply: tid=0 2024-03-21T11:48:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.420+0000 7fd663da3740 1 client.33478 shutdown 2024-03-21T11:48:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.420+0000 7fd663da3740 1 -- 172.21.15.47:0/1376681144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f31e0 msgr2=0x55a1661fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.420+0000 7fd663da3740 1 --2- 172.21.15.47:0/1376681144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f31e0 0x55a1661fd400 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fd64c09bdf0 tx=0x7fd64c0a0030 comp rx=0 tx=0).stop 2024-03-21T11:48:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.421+0000 7fd663da3740 1 -- 172.21.15.47:0/1376681144 shutdown_connections 2024-03-21T11:48:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.421+0000 7fd663da3740 1 --2- 172.21.15.47:0/1376681144 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a166208e00 0x55a16620b1f0 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.421+0000 7fd663da3740 1 --2- 172.21.15.47:0/1376681144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f8d40 0x55a166204910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.421+0000 7fd663da3740 1 --2- 172.21.15.47:0/1376681144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f3bd0 0x55a1661fd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.421+0000 7fd663da3740 1 --2- 172.21.15.47:0/1376681144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f31e0 0x55a1661fd400 secure :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fd64c09bdf0 tx=0x7fd64c0a0030 comp rx=0 tx=0).stop 2024-03-21T11:48:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.421+0000 7fd663da3740 1 -- 172.21.15.47:0/1376681144 >> 172.21.15.47:0/1376681144 conn(0x55a1661ee750 msgr2=0x55a1661ed950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.421+0000 7fd63be00640 0 client.33478 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.423+0000 7fd663da3740 1 -- 172.21.15.47:0/1376681144 shutdown_connections 2024-03-21T11:48:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.425+0000 7fd663da3740 1 -- 172.21.15.47:0/1376681144 wait complete. 2024-03-21T11:48:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.425+0000 7fd663da3740 20 client.33478 trim_cache size 0 max 16384 2024-03-21T11:48:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.460+0000 7fd663da3740 1 Processor -- start 2024-03-21T11:48:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.460+0000 7fd663da3740 1 -- start start 2024-03-21T11:48:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.461+0000 7fd663da3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f3bd0 0x55a1661f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.461+0000 7fd663da3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f8d40 0x55a1662048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.461+0000 7fd663da3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a166210960 0x55a1661fc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.461+0000 7fd663da3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1b4b70 con 0x55a1661f8d40 2024-03-21T11:48:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.461+0000 7fd663da3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1b64f0 con 0x55a166210960 2024-03-21T11:48:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.461+0000 7fd663da3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1b51d0 con 0x55a1661f3bd0 2024-03-21T11:48:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.462+0000 7fd659400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f8d40 0x55a1662048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.462+0000 7fd659e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f3bd0 0x55a1661f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.462+0000 7fd659e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f3bd0 0x55a1661f20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48772/0 (socket says 172.21.15.47:48772) 2024-03-21T11:48:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.462+0000 7fd659e00640 1 -- 172.21.15.47:0/2325101341 learned_addr learned my addr 172.21.15.47:0/2325101341 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.462+0000 7fd65a800640 1 --2- 172.21.15.47:0/2325101341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a166210960 0x55a1661fc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.462+0000 7fd65a800640 1 -- 172.21.15.47:0/2325101341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f3bd0 msgr2=0x55a1661f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.462+0000 7fd65a800640 1 --2- 172.21.15.47:0/2325101341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f3bd0 0x55a1661f20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.462+0000 7fd65a800640 1 -- 172.21.15.47:0/2325101341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f8d40 msgr2=0x55a1662048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.462+0000 7fd65a800640 1 --2- 172.21.15.47:0/2325101341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f8d40 0x55a1662048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.462+0000 7fd65a800640 1 -- 172.21.15.47:0/2325101341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1661fbdd0 con 0x55a166210960 2024-03-21T11:48:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.463+0000 7fd65a800640 1 --2- 172.21.15.47:0/2325101341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a166210960 0x55a1661fc030 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fd65402c430 tx=0x7fd654041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a68ce594be381274 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.463+0000 7fd646a00640 1 -- 172.21.15.47:0/2325101341 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654051030 con 0x55a166210960 2024-03-21T11:48:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.464+0000 7fd646a00640 1 -- 172.21.15.47:0/2325101341 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd65402cd70 con 0x55a166210960 2024-03-21T11:48:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.464+0000 7fd663da3740 1 -- 172.21.15.47:0/2325101341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a166210960 msgr2=0x55a1661fc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.465+0000 7fd646a00640 1 -- 172.21.15.47:0/2325101341 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654051030 con 0x55a166210960 2024-03-21T11:48:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.465+0000 7fd663da3740 1 --2- 172.21.15.47:0/2325101341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a166210960 0x55a1661fc030 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fd65402c430 tx=0x7fd654041000 comp rx=0 tx=0).stop 2024-03-21T11:48:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.465+0000 7fd663da3740 1 -- 172.21.15.47:0/2325101341 shutdown_connections 2024-03-21T11:48:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.465+0000 7fd663da3740 1 --2- 172.21.15.47:0/2325101341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a166210960 0x55a1661fc030 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.465+0000 7fd663da3740 1 --2- 172.21.15.47:0/2325101341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f8d40 0x55a1662048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.465+0000 7fd663da3740 1 --2- 172.21.15.47:0/2325101341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1661f3bd0 0x55a1661f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.465+0000 7fd663da3740 1 -- 172.21.15.47:0/2325101341 >> 172.21.15.47:0/2325101341 conn(0x55a1661ee750 msgr2=0x55a1661f19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.465+0000 7fd663da3740 1 -- 172.21.15.47:0/2325101341 shutdown_connections 2024-03-21T11:48:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.465+0000 7fd663da3740 1 -- 172.21.15.47:0/2325101341 wait complete. 2024-03-21T11:48:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.466+0000 7fd663da3740 1 -- start start 2024-03-21T11:48:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.466+0000 7fd663da3740 1 Processor -- start 2024-03-21T11:48:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.468+0000 7fd663da3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f3bd0 0x55a1662040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.468+0000 7fd663da3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f8d40 0x55a166204620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.468+0000 7fd663da3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a166210960 0x55a1662026f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.469+0000 7fd663da3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1b5e90 con 0x55a1661f3bd0 2024-03-21T11:48:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.469+0000 7fd663da3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1b5830 con 0x55a1661f8d40 2024-03-21T11:48:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.469+0000 7fd663da3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1b2d30 con 0x55a166210960 2024-03-21T11:48:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.469+0000 7fd65a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a166210960 0x55a1662026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.469+0000 7fd65a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a166210960 0x55a1662026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48774/0 (socket says 172.21.15.47:48774) 2024-03-21T11:48:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.469+0000 7fd65a800640 1 -- 172.21.15.47:0/1268626559 learned_addr learned my addr 172.21.15.47:0/1268626559 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:48:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.469+0000 7fd659400640 1 --2- 172.21.15.47:0/1268626559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f8d40 0x55a166204620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.469+0000 7fd663da3740 10 client.0 _ll_register_callbacks cb 0x7fd64c09bc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:48:55.471 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.470+0000 7fd659e00640 1 --2- 172.21.15.47:0/1268626559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f3bd0 0x55a1662040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.470+0000 7fd65a800640 1 -- 172.21.15.47:0/1268626559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f8d40 msgr2=0x55a166204620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.470+0000 7fd65a800640 1 --2- 172.21.15.47:0/1268626559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f8d40 0x55a166204620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.470+0000 7fd65a800640 1 -- 172.21.15.47:0/1268626559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f3bd0 msgr2=0x55a1662040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:48:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.470+0000 7fd65a800640 1 --2- 172.21.15.47:0/1268626559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f3bd0 0x55a1662040e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.470+0000 7fd65a800640 1 -- 172.21.15.47:0/1268626559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a166202ce0 con 0x55a166210960 2024-03-21T11:48:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.474+0000 7fd65a800640 1 --2- 172.21.15.47:0/1268626559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a166210960 0x55a1662026f0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fd654036040 tx=0x7fd654034430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4da1f9a1d391650f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:48:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.474+0000 7fd63be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:48:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.477+0000 7fd63be00640 1 -- 172.21.15.47:0/1268626559 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654051040 con 0x55a166210960 2024-03-21T11:48:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.478+0000 7fd63be00640 1 -- 172.21.15.47:0/1268626559 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd654034a60 con 0x55a166210960 2024-03-21T11:48:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.478+0000 7fd663da3740 1 -- 172.21.15.47:0/1268626559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd65402cd70 con 0x55a166210960 2024-03-21T11:48:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.479+0000 7fd63be00640 1 -- 172.21.15.47:0/1268626559 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd654068050 con 0x55a166210960 2024-03-21T11:48:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.479+0000 7fd663da3740 10 client.33484 fetch_fsmap learned FSMap version 41 2024-03-21T11:48:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.479+0000 7fd663da3740 1 -- 172.21.15.47:0/1268626559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a166264e40 con 0x55a166210960 2024-03-21T11:48:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.479+0000 7fd63be00640 1 -- 172.21.15.47:0/1268626559 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd65402cd70 con 0x55a166210960 2024-03-21T11:48:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.480+0000 7fd63be00640 1 -- 172.21.15.47:0/1268626559 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd65404b040 con 0x55a166210960 2024-03-21T11:48:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.480+0000 7fd663da3740 10 client.33484 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:48:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.480+0000 7fd663da3740 10 client.33484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:48:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.480+0000 7fd663da3740 20 client.33484 populate_metadata read hostname 'smithi047' 2024-03-21T11:48:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.480+0000 7fd663da3740 1 --2- 172.21.15.47:0/1268626559 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a166265670 0x55a166267ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:48:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.480+0000 7fd663da3740 4 client.33484 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:48:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.480+0000 7fd663da3740 1 -- 172.21.15.47:0/1268626559 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a1661600c0 con 0x55a166265670 2024-03-21T11:48:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.480+0000 7fd659e00640 1 --2- 172.21.15.47:0/1268626559 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a166265670 0x55a166267ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:48:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.482+0000 7fd659e00640 1 --2- 172.21.15.47:0/1268626559 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a166265670 0x55a166267ad0 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c64876c065cbe2d server_cookie=3158e012be5b3469 in_seq=0 out_seq=0 2024-03-21T11:48:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:55.482+0000 7fd63be00640 10 client.33484 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:55.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:55 smithi102 ceph-mon[36748]: pgmap v1168: 129 pgs: 129 active+clean; 747 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-03-21T11:48:55.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:55 smithi047 ceph-mon[31351]: pgmap v1168: 129 pgs: 129 active+clean; 747 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-03-21T11:48:55.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:55 smithi100 ceph-mon[36245]: pgmap v1168: 129 pgs: 129 active+clean; 747 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-03-21T11:48:56.004 INFO:tasks.workunit.client.0.smithi047.stdout: 10 288193 21.38 MB/sec execute 579 sec latency 1094.360 ms 2024-03-21T11:48:57.005 INFO:tasks.workunit.client.0.smithi047.stdout: 10 288400 21.36 MB/sec execute 580 sec latency 514.932 ms 2024-03-21T11:48:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.127+0000 7fd63be00640 1 -- 172.21.15.47:0/1268626559 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a1661600c0 con 0x55a166265670 2024-03-21T11:48:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.127+0000 7fd63be00640 10 client.33484 handle_command_reply: tid=0 2024-03-21T11:48:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.127+0000 7fd663da3740 1 client.33484 shutdown 2024-03-21T11:48:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.127+0000 7fd663da3740 1 -- 172.21.15.47:0/1268626559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a166210960 msgr2=0x55a1662026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.127+0000 7fd663da3740 1 --2- 172.21.15.47:0/1268626559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a166210960 0x55a1662026f0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fd654036040 tx=0x7fd654034430 comp rx=0 tx=0).stop 2024-03-21T11:48:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.127+0000 7fd663da3740 1 -- 172.21.15.47:0/1268626559 shutdown_connections 2024-03-21T11:48:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.127+0000 7fd663da3740 1 --2- 172.21.15.47:0/1268626559 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55a166265670 0x55a166267ad0 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.127+0000 7fd663da3740 1 --2- 172.21.15.47:0/1268626559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a166210960 0x55a1662026f0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.127+0000 7fd663da3740 1 --2- 172.21.15.47:0/1268626559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1661f8d40 0x55a166204620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.127+0000 7fd663da3740 1 --2- 172.21.15.47:0/1268626559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1661f3bd0 0x55a1662040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.127+0000 7fd663da3740 1 -- 172.21.15.47:0/1268626559 >> 172.21.15.47:0/1268626559 conn(0x55a1661ee750 msgr2=0x55a1661fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.127+0000 7fd63be00640 0 client.33484 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:48:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.128+0000 7fd663da3740 1 -- 172.21.15.47:0/1268626559 shutdown_connections 2024-03-21T11:48:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.128+0000 7fd663da3740 1 -- 172.21.15.47:0/1268626559 wait complete. 2024-03-21T11:48:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.128+0000 7fd663da3740 20 client.33484 trim_cache size 0 max 16384 2024-03-21T11:48:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:48:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:48:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:48:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:48:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.129+0000 7fd661000640 1 -- 172.21.15.47:0/727467793 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd6400a3040 msgr2=0x7fd6400a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.129+0000 7fd661000640 1 --2- 172.21.15.47:0/727467793 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd6400a3040 0x7fd6400a5500 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd654006010 tx=0x7fd654005e30 comp rx=0 tx=0).stop 2024-03-21T11:48:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.129+0000 7fd661000640 1 -- 172.21.15.47:0/727467793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd65c1086a0 msgr2=0x7fd65c1a22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:48:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.129+0000 7fd661000640 1 --2- 172.21.15.47:0/727467793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd65c1086a0 0x7fd65c1a22a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00e9c0 tx=0x7fd64c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:48:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.130+0000 7fd661000640 1 -- 172.21.15.47:0/727467793 shutdown_connections 2024-03-21T11:48:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.130+0000 7fd661000640 1 --2- 172.21.15.47:0/727467793 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd6400a3040 0x7fd6400a5500 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.130+0000 7fd661000640 1 --2- 172.21.15.47:0/727467793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c10ff20 0x7fd65c1a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.130+0000 7fd661000640 1 --2- 172.21.15.47:0/727467793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c109000 0x7fd65c1a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.130+0000 7fd661000640 1 --2- 172.21.15.47:0/727467793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd65c1086a0 0x7fd65c1a22a0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:48:57.131 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.130+0000 7fd661000640 1 -- 172.21.15.47:0/727467793 >> 172.21.15.47:0/727467793 conn(0x7fd65c06e6e0 msgr2=0x7fd65c1097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:48:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.130+0000 7fd661000640 1 -- 172.21.15.47:0/727467793 shutdown_connections 2024-03-21T11:48:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:48:57.130+0000 7fd661000640 1 -- 172.21.15.47:0/727467793 wait complete. 2024-03-21T11:48:57.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:57 smithi102 ceph-mon[36748]: pgmap v1169: 129 pgs: 129 active+clean; 746 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 384 op/s 2024-03-21T11:48:57.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:57 smithi047 ceph-mon[31351]: pgmap v1169: 129 pgs: 129 active+clean; 746 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 384 op/s 2024-03-21T11:48:57.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:57 smithi100 ceph-mon[36245]: pgmap v1169: 129 pgs: 129 active+clean; 746 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 384 op/s 2024-03-21T11:48:58.005 INFO:tasks.workunit.client.0.smithi047.stdout: 10 288664 21.34 MB/sec execute 581 sec latency 232.068 ms 2024-03-21T11:48:59.005 INFO:tasks.workunit.client.0.smithi047.stdout: 10 288921 21.34 MB/sec execute 582 sec latency 293.481 ms 2024-03-21T11:48:59.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:48:59 smithi102 ceph-mon[36748]: pgmap v1170: 129 pgs: 129 active+clean; 766 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.3 MiB/s wr, 556 op/s 2024-03-21T11:48:59.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:48:59 smithi047 ceph-mon[31351]: pgmap v1170: 129 pgs: 129 active+clean; 766 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.3 MiB/s wr, 556 op/s 2024-03-21T11:48:59.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:48:59 smithi100 ceph-mon[36245]: pgmap v1170: 129 pgs: 129 active+clean; 766 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 7.3 MiB/s wr, 556 op/s 2024-03-21T11:49:00.005 INFO:tasks.workunit.client.0.smithi047.stdout: 10 289285 21.36 MB/sec execute 583 sec latency 107.481 ms 2024-03-21T11:49:01.005 INFO:tasks.workunit.client.0.smithi047.stdout: 10 289733 21.37 MB/sec execute 584 sec latency 71.441 ms 2024-03-21T11:49:01.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:01 smithi102 ceph-mon[36748]: pgmap v1171: 129 pgs: 129 active+clean; 773 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.6 MiB/s wr, 546 op/s 2024-03-21T11:49:01.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:01 smithi047 ceph-mon[31351]: pgmap v1171: 129 pgs: 129 active+clean; 773 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.6 MiB/s wr, 546 op/s 2024-03-21T11:49:01.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:01 smithi100 ceph-mon[36245]: pgmap v1171: 129 pgs: 129 active+clean; 773 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.6 MiB/s wr, 546 op/s 2024-03-21T11:49:02.006 INFO:tasks.workunit.client.0.smithi047.stdout: 10 290085 21.35 MB/sec execute 585 sec latency 54.981 ms 2024-03-21T11:49:03.006 INFO:tasks.workunit.client.0.smithi047.stdout: 10 290341 21.33 MB/sec execute 586 sec latency 117.929 ms 2024-03-21T11:49:03.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:03 smithi102 ceph-mon[36748]: pgmap v1172: 129 pgs: 129 active+clean; 762 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.7 MiB/s wr, 556 op/s 2024-03-21T11:49:03.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:03 smithi047 ceph-mon[31351]: pgmap v1172: 129 pgs: 129 active+clean; 762 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.7 MiB/s wr, 556 op/s 2024-03-21T11:49:03.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:03 smithi100 ceph-mon[36245]: pgmap v1172: 129 pgs: 129 active+clean; 762 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.7 MiB/s wr, 556 op/s 2024-03-21T11:49:04.006 INFO:tasks.workunit.client.0.smithi047.stdout: 10 290919 21.36 MB/sec execute 587 sec latency 71.266 ms 2024-03-21T11:49:04.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:49:04.981 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:49:05.006 INFO:tasks.workunit.client.0.smithi047.stdout: 10 291423 21.35 MB/sec execute 588 sec latency 113.869 ms 2024-03-21T11:49:05.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.253+0000 7f1e55600640 1 -- 172.21.15.47:0/1325964767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e50100e00 msgr2=0x7f1e501031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:05.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.253+0000 7f1e55600640 1 --2- 172.21.15.47:0/1325964767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e50100e00 0x7f1e501031f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009900 tx=0x7f1e4002bcc0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.253+0000 7f1e55600640 1 -- 172.21.15.47:0/1325964767 shutdown_connections 2024-03-21T11:49:05.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.253+0000 7f1e55600640 1 --2- 172.21.15.47:0/1325964767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e50106230 0x7f1e501086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.253+0000 7f1e55600640 1 --2- 172.21.15.47:0/1325964767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e50103730 0x7f1e50105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.253+0000 7f1e55600640 1 --2- 172.21.15.47:0/1325964767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e50100e00 0x7f1e501031f0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.253+0000 7f1e55600640 1 -- 172.21.15.47:0/1325964767 >> 172.21.15.47:0/1325964767 conn(0x7f1e500fae60 msgr2=0x7f1e500fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:05.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.253+0000 7f1e55600640 1 -- 172.21.15.47:0/1325964767 shutdown_connections 2024-03-21T11:49:05.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.253+0000 7f1e55600640 1 -- 172.21.15.47:0/1325964767 wait complete. 2024-03-21T11:49:05.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.255+0000 7f1e55600640 1 Processor -- start 2024-03-21T11:49:05.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.255+0000 7f1e55600640 1 -- start start 2024-03-21T11:49:05.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.256+0000 7f1e55600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e50100e00 0x7f1e50199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.256+0000 7f1e55600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e50103730 0x7f1e5019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.256+0000 7f1e55600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e50106230 0x7f1e501a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.256+0000 7f1e55600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5010c280 con 0x7f1e50103730 2024-03-21T11:49:05.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.256+0000 7f1e55600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5010c100 con 0x7f1e50106230 2024-03-21T11:49:05.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.256+0000 7f1e55600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5010c400 con 0x7f1e50100e00 2024-03-21T11:49:05.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.256+0000 7f1e4ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e50100e00 0x7f1e50199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.256+0000 7f1e4e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e50103730 0x7f1e5019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.256+0000 7f1e4ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e50100e00 0x7f1e50199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45460/0 (socket says 172.21.15.47:45460) 2024-03-21T11:49:05.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.256+0000 7f1e4f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e50106230 0x7f1e501a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.257+0000 7f1e4ea00640 1 -- 172.21.15.47:0/2276235485 learned_addr learned my addr 172.21.15.47:0/2276235485 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:05.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.257+0000 7f1e4ea00640 1 -- 172.21.15.47:0/2276235485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e50106230 msgr2=0x7f1e501a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:05.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.257+0000 7f1e4ea00640 1 --2- 172.21.15.47:0/2276235485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e50106230 0x7f1e501a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.257+0000 7f1e4ea00640 1 -- 172.21.15.47:0/2276235485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e50103730 msgr2=0x7f1e5019a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:05.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.257+0000 7f1e4ea00640 1 --2- 172.21.15.47:0/2276235485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e50103730 0x7f1e5019a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.257+0000 7f1e4ea00640 1 -- 172.21.15.47:0/2276235485 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e501a1850 con 0x7f1e50100e00 2024-03-21T11:49:05.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.257+0000 7f1e4e000640 1 --2- 172.21.15.47:0/2276235485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e50103730 0x7f1e5019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:05.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.257+0000 7f1e4ea00640 1 --2- 172.21.15.47:0/2276235485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e50100e00 0x7f1e50199c40 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f1e40000c00 tx=0x7f1e4002c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:05.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.258+0000 7f1e47400640 1 -- 172.21.15.47:0/2276235485 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40043020 con 0x7f1e50100e00 2024-03-21T11:49:05.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.258+0000 7f1e47400640 1 -- 172.21.15.47:0/2276235485 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e400054e0 con 0x7f1e50100e00 2024-03-21T11:49:05.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.258+0000 7f1e47400640 1 -- 172.21.15.47:0/2276235485 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40003c20 con 0x7f1e50100e00 2024-03-21T11:49:05.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.258+0000 7f1e55600640 1 -- 172.21.15.47:0/2276235485 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e501a1ae0 con 0x7f1e50100e00 2024-03-21T11:49:05.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.258+0000 7f1e55600640 1 -- 172.21.15.47:0/2276235485 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e501a1fe0 con 0x7f1e50100e00 2024-03-21T11:49:05.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.262+0000 7f1e47400640 1 -- 172.21.15.47:0/2276235485 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1e40046020 con 0x7f1e50100e00 2024-03-21T11:49:05.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.262+0000 7f1e58220740 1 Processor -- start 2024-03-21T11:49:05.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.263+0000 7f1e58220740 1 -- start start 2024-03-21T11:49:05.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.264+0000 7f1e58220740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 0x55c5c82552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.264+0000 7f1e58220740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8255870 0x55c5c825a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.264+0000 7f1e58220740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 0x55c5c825ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.264+0000 7f1e58220740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c81ba250 con 0x55c5c8254e80 2024-03-21T11:49:05.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.264+0000 7f1e58220740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c8162a30 con 0x55c5c8255870 2024-03-21T11:49:05.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.264+0000 7f1e58220740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c81b62f0 con 0x55c5c825a9e0 2024-03-21T11:49:05.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.264+0000 7f1e4f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 0x55c5c82552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.264+0000 7f1e4f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 0x55c5c82552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59768/0 (socket says 172.21.15.47:59768) 2024-03-21T11:49:05.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.264+0000 7f1e4f400640 1 -- 172.21.15.47:0/3474083573 learned_addr learned my addr 172.21.15.47:0/3474083573 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:05.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.265+0000 7f1e4ea00640 1 --2- 172.21.15.47:0/3474083573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8255870 0x55c5c825a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.265+0000 7f1e47400640 1 --2- 172.21.15.47:0/2276235485 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1e340a2fa0 0x7f1e340a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.265+0000 7f1e4f400640 1 -- 172.21.15.47:0/3474083573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 msgr2=0x55c5c825ce70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:49:05.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.265+0000 7f1e4f400640 1 --2- 172.21.15.47:0/3474083573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 0x55c5c825ce70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.265+0000 7f1e4f400640 1 -- 172.21.15.47:0/3474083573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8255870 msgr2=0x55c5c825a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:05.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.265+0000 7f1e4f400640 1 --2- 172.21.15.47:0/3474083573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8255870 0x55c5c825a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.265+0000 7f1e4f400640 1 -- 172.21.15.47:0/3474083573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5c814d690 con 0x55c5c8254e80 2024-03-21T11:49:05.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.265+0000 7f1e4ea00640 1 --2- 172.21.15.47:0/3474083573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8255870 0x55c5c825a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:05.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.265+0000 7f1e47400640 1 -- 172.21.15.47:0/2276235485 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1e400f1880 con 0x7f1e50100e00 2024-03-21T11:49:05.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.266+0000 7f1e4f400640 1 --2- 172.21.15.47:0/2276235485 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1e340a2fa0 0x7f1e340a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.267+0000 7f1e4f400640 1 --2- 172.21.15.47:0/3474083573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 0x55c5c82552a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009fd0 tx=0x7f1e480118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ace039aab65b8f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:05.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.267+0000 7f1e4f400640 1 --2- 172.21.15.47:0/2276235485 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1e340a2fa0 0x7f1e340a5460 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1e48019000 tx=0x7f1e4801b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:05.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.267+0000 7f1e3b400640 1 -- 172.21.15.47:0/3474083573 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e48003c10 con 0x55c5c8254e80 2024-03-21T11:49:05.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.268+0000 7f1e3b400640 1 -- 172.21.15.47:0/3474083573 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e4800fbc0 con 0x55c5c8254e80 2024-03-21T11:49:05.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.268+0000 7f1e3b400640 1 -- 172.21.15.47:0/3474083573 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4801be60 con 0x55c5c8254e80 2024-03-21T11:49:05.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.268+0000 7f1e58220740 1 -- 172.21.15.47:0/3474083573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 msgr2=0x55c5c82552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:05.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.268+0000 7f1e58220740 1 --2- 172.21.15.47:0/3474083573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 0x55c5c82552a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009fd0 tx=0x7f1e480118d0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.269+0000 7f1e58220740 1 -- 172.21.15.47:0/3474083573 shutdown_connections 2024-03-21T11:49:05.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.269+0000 7f1e58220740 1 --2- 172.21.15.47:0/3474083573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 0x55c5c825ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.269+0000 7f1e58220740 1 --2- 172.21.15.47:0/3474083573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8255870 0x55c5c825a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.269+0000 7f1e58220740 1 --2- 172.21.15.47:0/3474083573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 0x55c5c82552a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.269+0000 7f1e58220740 1 -- 172.21.15.47:0/3474083573 >> 172.21.15.47:0/3474083573 conn(0x55c5c821d200 msgr2=0x55c5c82507f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:05.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.269+0000 7f1e58220740 1 -- 172.21.15.47:0/3474083573 shutdown_connections 2024-03-21T11:49:05.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.269+0000 7f1e58220740 1 -- 172.21.15.47:0/3474083573 wait complete. 2024-03-21T11:49:05.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.269+0000 7f1e58220740 1 -- start start 2024-03-21T11:49:05.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.270+0000 7f1e58220740 1 Processor -- start 2024-03-21T11:49:05.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.270+0000 7f1e58220740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 0x55c5c825ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.270+0000 7f1e58220740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c8255870 0x55c5c825f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.270+0000 7f1e58220740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c825a9e0 0x55c5c8266490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.270+0000 7f1e58220740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c825d990 con 0x55c5c8254e80 2024-03-21T11:49:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.270+0000 7f1e58220740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c8175e80 con 0x55c5c825a9e0 2024-03-21T11:49:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.270+0000 7f1e58220740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c825db10 con 0x55c5c8255870 2024-03-21T11:49:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.270+0000 7f1e58220740 10 client.0 _ll_register_callbacks cb 0x55c5c8188130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.270+0000 7f1e4e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c8255870 0x55c5c825f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.270+0000 7f1e4f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c825a9e0 0x55c5c8266490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.271+0000 7f1e4e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c8255870 0x55c5c825f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45486/0 (socket says 172.21.15.47:45486) 2024-03-21T11:49:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.271+0000 7f1e4e000640 1 -- 172.21.15.47:0/4290514697 learned_addr learned my addr 172.21.15.47:0/4290514697 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.271+0000 7f1e4ea00640 1 --2- 172.21.15.47:0/4290514697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 0x55c5c825ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.271+0000 7f1e4f400640 1 -- 172.21.15.47:0/4290514697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c8255870 msgr2=0x55c5c825f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.271+0000 7f1e4f400640 1 --2- 172.21.15.47:0/4290514697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c8255870 0x55c5c825f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.271+0000 7f1e4f400640 1 -- 172.21.15.47:0/4290514697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 msgr2=0x55c5c825ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.271+0000 7f1e4f400640 1 --2- 172.21.15.47:0/4290514697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 0x55c5c825ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.271+0000 7f1e4f400640 1 -- 172.21.15.47:0/4290514697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5c7f987d0 con 0x55c5c825a9e0 2024-03-21T11:49:05.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.271+0000 7f1e4f400640 1 --2- 172.21.15.47:0/4290514697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c825a9e0 0x55c5c8266490 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f1e48000f80 tx=0x7f1e48003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd88bcb7421c49be server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:05.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.272+0000 7f1e2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:49:05.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.272+0000 7f1e2fe00640 1 -- 172.21.15.47:0/4290514697 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e48032020 con 0x55c5c825a9e0 2024-03-21T11:49:05.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.272+0000 7f1e58220740 1 -- 172.21.15.47:0/4290514697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e4800fbc0 con 0x55c5c825a9e0 2024-03-21T11:49:05.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.272+0000 7f1e2fe00640 1 -- 172.21.15.47:0/4290514697 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1e4800f4e0 con 0x55c5c825a9e0 2024-03-21T11:49:05.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.272+0000 7f1e2fe00640 1 -- 172.21.15.47:0/4290514697 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e48028040 con 0x55c5c825a9e0 2024-03-21T11:49:05.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.272+0000 7f1e58220740 10 client.33560 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.272+0000 7f1e58220740 1 -- 172.21.15.47:0/4290514697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c5c7ece3f0 con 0x55c5c825a9e0 2024-03-21T11:49:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.273+0000 7f1e2fe00640 1 -- 172.21.15.47:0/4290514697 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e4800fbc0 con 0x55c5c825a9e0 2024-03-21T11:49:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.273+0000 7f1e2fe00640 1 -- 172.21.15.47:0/4290514697 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1e4802eac0 con 0x55c5c825a9e0 2024-03-21T11:49:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.273+0000 7f1e58220740 10 client.33560 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.273+0000 7f1e58220740 10 client.33560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:49:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.273+0000 7f1e58220740 20 client.33560 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.273+0000 7f1e58220740 1 --2- 172.21.15.47:0/4290514697 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5c826aa90 0x55c5c826ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.273+0000 7f1e58220740 4 client.33560 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.273+0000 7f1e58220740 1 -- 172.21.15.47:0/4290514697 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5c81c7790 con 0x55c5c826aa90 2024-03-21T11:49:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.273+0000 7f1e4ea00640 1 --2- 172.21.15.47:0/4290514697 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5c826aa90 0x55c5c826ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.274+0000 7f1e4ea00640 1 --2- 172.21.15.47:0/4290514697 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5c826aa90 0x55c5c826ced0 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a432a389606d313 server_cookie=de78d8afa5d42791 in_seq=0 out_seq=0 2024-03-21T11:49:05.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.274+0000 7f1e2fe00640 10 client.33560 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:05.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.278+0000 7f1e2fe00640 1 -- 172.21.15.47:0/4290514697 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c5c81c7790 con 0x55c5c826aa90 2024-03-21T11:49:05.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.278+0000 7f1e2fe00640 10 client.33560 handle_command_reply: tid=0 2024-03-21T11:49:05.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.278+0000 7f1e58220740 1 client.33560 shutdown 2024-03-21T11:49:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.278+0000 7f1e58220740 1 -- 172.21.15.47:0/4290514697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c825a9e0 msgr2=0x55c5c8266490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.278+0000 7f1e58220740 1 --2- 172.21.15.47:0/4290514697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c825a9e0 0x55c5c8266490 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f1e48000f80 tx=0x7f1e48003f90 comp rx=0 tx=0).stop 2024-03-21T11:49:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.278+0000 7f1e58220740 1 -- 172.21.15.47:0/4290514697 shutdown_connections 2024-03-21T11:49:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.278+0000 7f1e58220740 1 --2- 172.21.15.47:0/4290514697 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5c826aa90 0x55c5c826ced0 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.278+0000 7f1e2fe00640 0 client.33560 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.278+0000 7f1e58220740 1 --2- 172.21.15.47:0/4290514697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c825a9e0 0x55c5c8266490 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.278+0000 7f1e58220740 1 --2- 172.21.15.47:0/4290514697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c8255870 0x55c5c825f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.278+0000 7f1e58220740 1 --2- 172.21.15.47:0/4290514697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 0x55c5c825ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.278+0000 7f1e58220740 1 -- 172.21.15.47:0/4290514697 >> 172.21.15.47:0/4290514697 conn(0x55c5c821d200 msgr2=0x55c5c824fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.279+0000 7f1e58220740 1 -- 172.21.15.47:0/4290514697 shutdown_connections 2024-03-21T11:49:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.279+0000 7f1e58220740 1 -- 172.21.15.47:0/4290514697 wait complete. 2024-03-21T11:49:05.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.279+0000 7f1e58220740 20 client.33560 trim_cache size 0 max 16384 2024-03-21T11:49:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.299+0000 7f1e58220740 1 Processor -- start 2024-03-21T11:49:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.299+0000 7f1e58220740 1 -- start start 2024-03-21T11:49:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.299+0000 7f1e58220740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8254e80 0x55c5c8256980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.299+0000 7f1e58220740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8255870 0x55c5c8266450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.299+0000 7f1e58220740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 0x55c5c824f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.299+0000 7f1e58220740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e501ac370 con 0x55c5c8255870 2024-03-21T11:49:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.299+0000 7f1e58220740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e501adcf0 con 0x55c5c8254e80 2024-03-21T11:49:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.299+0000 7f1e58220740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e501ac9d0 con 0x55c5c825a9e0 2024-03-21T11:49:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.300+0000 7f1e4f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 0x55c5c824f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.300+0000 7f1e4ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8254e80 0x55c5c8256980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.300+0000 7f1e4f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 0x55c5c824f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45498/0 (socket says 172.21.15.47:45498) 2024-03-21T11:49:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.300+0000 7f1e4f400640 1 -- 172.21.15.47:0/2022814709 learned_addr learned my addr 172.21.15.47:0/2022814709 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.300+0000 7f1e4e000640 1 --2- 172.21.15.47:0/2022814709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8255870 0x55c5c8266450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.300+0000 7f1e4f400640 1 -- 172.21.15.47:0/2022814709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8254e80 msgr2=0x55c5c8256980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.300+0000 7f1e4f400640 1 --2- 172.21.15.47:0/2022814709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8254e80 0x55c5c8256980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.300+0000 7f1e4f400640 1 -- 172.21.15.47:0/2022814709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8255870 msgr2=0x55c5c8266450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.300+0000 7f1e4f400640 1 --2- 172.21.15.47:0/2022814709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8255870 0x55c5c8266450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.300+0000 7f1e4f400640 1 -- 172.21.15.47:0/2022814709 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5c7e8e0f0 con 0x55c5c825a9e0 2024-03-21T11:49:05.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.300+0000 7f1e4ea00640 1 --2- 172.21.15.47:0/2022814709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8254e80 0x55c5c8256980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:05.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.300+0000 7f1e4f400640 1 --2- 172.21.15.47:0/2022814709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 0x55c5c824f200 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f1e48003f00 tx=0x7f1e48003e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=937de2af0d65766b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:05.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.301+0000 7f1e3aa00640 1 -- 172.21.15.47:0/2022814709 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e48032040 con 0x55c5c825a9e0 2024-03-21T11:49:05.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.301+0000 7f1e3aa00640 1 -- 172.21.15.47:0/2022814709 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e4801bbe0 con 0x55c5c825a9e0 2024-03-21T11:49:05.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.301+0000 7f1e3aa00640 1 -- 172.21.15.47:0/2022814709 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800f4e0 con 0x55c5c825a9e0 2024-03-21T11:49:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.302+0000 7f1e58220740 1 -- 172.21.15.47:0/2022814709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 msgr2=0x55c5c824f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.302+0000 7f1e58220740 1 --2- 172.21.15.47:0/2022814709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 0x55c5c824f200 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f1e48003f00 tx=0x7f1e48003e70 comp rx=0 tx=0).stop 2024-03-21T11:49:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.302+0000 7f1e58220740 1 -- 172.21.15.47:0/2022814709 shutdown_connections 2024-03-21T11:49:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.302+0000 7f1e58220740 1 --2- 172.21.15.47:0/2022814709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 0x55c5c824f200 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.302+0000 7f1e58220740 1 --2- 172.21.15.47:0/2022814709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8255870 0x55c5c8266450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.302+0000 7f1e58220740 1 --2- 172.21.15.47:0/2022814709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8254e80 0x55c5c8256980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.302+0000 7f1e58220740 1 -- 172.21.15.47:0/2022814709 >> 172.21.15.47:0/2022814709 conn(0x55c5c821d200 msgr2=0x55c5c8256270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.302+0000 7f1e58220740 1 -- 172.21.15.47:0/2022814709 shutdown_connections 2024-03-21T11:49:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.302+0000 7f1e58220740 1 -- 172.21.15.47:0/2022814709 wait complete. 2024-03-21T11:49:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.302+0000 7f1e58220740 1 -- start start 2024-03-21T11:49:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.302+0000 7f1e58220740 1 Processor -- start 2024-03-21T11:49:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.303+0000 7f1e58220740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 0x55c5c8264230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.303+0000 7f1e58220740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8255870 0x55c5c8264770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.303+0000 7f1e58220740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 0x55c5c82656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.303+0000 7f1e58220740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e501ad690 con 0x55c5c8254e80 2024-03-21T11:49:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.303+0000 7f1e58220740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e501ad030 con 0x55c5c8255870 2024-03-21T11:49:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.303+0000 7f1e58220740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e501aa530 con 0x55c5c825a9e0 2024-03-21T11:49:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.303+0000 7f1e58220740 10 client.0 _ll_register_callbacks cb 0x7f1e400b9f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.303+0000 7f1e4e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8255870 0x55c5c8264770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.303+0000 7f1e4e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8255870 0x55c5c8264770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54720/0 (socket says 172.21.15.47:54720) 2024-03-21T11:49:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.303+0000 7f1e4e000640 1 -- 172.21.15.47:0/3976902792 learned_addr learned my addr 172.21.15.47:0/3976902792 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.303+0000 7f1e4ea00640 1 --2- 172.21.15.47:0/3976902792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 0x55c5c8264230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.304+0000 7f1e4f400640 1 --2- 172.21.15.47:0/3976902792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 0x55c5c82656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.304+0000 7f1e4e000640 1 -- 172.21.15.47:0/3976902792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 msgr2=0x55c5c82656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.304+0000 7f1e4e000640 1 --2- 172.21.15.47:0/3976902792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 0x55c5c82656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.304+0000 7f1e4e000640 1 -- 172.21.15.47:0/3976902792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 msgr2=0x55c5c8264230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.304+0000 7f1e4e000640 1 --2- 172.21.15.47:0/3976902792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 0x55c5c8264230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.304+0000 7f1e4e000640 1 -- 172.21.15.47:0/3976902792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5c8265cd0 con 0x55c5c8255870 2024-03-21T11:49:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.304+0000 7f1e4e000640 1 --2- 172.21.15.47:0/3976902792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8255870 0x55c5c8264770 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c01a920 tx=0x7f1e3c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96b132913c1b78ad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:05.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.304+0000 7f1e2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:49:05.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.305+0000 7f1e2fe00640 1 -- 172.21.15.47:0/3976902792 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c004230 con 0x55c5c8255870 2024-03-21T11:49:05.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.305+0000 7f1e58220740 1 -- 172.21.15.47:0/3976902792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e4801bbe0 con 0x55c5c8255870 2024-03-21T11:49:05.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.305+0000 7f1e2fe00640 1 -- 172.21.15.47:0/3976902792 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f1e3c0043d0 con 0x55c5c8255870 2024-03-21T11:49:05.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.305+0000 7f1e2fe00640 1 -- 172.21.15.47:0/3976902792 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e3c02f050 con 0x55c5c8255870 2024-03-21T11:49:05.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.305+0000 7f1e58220740 10 client.33566 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.305+0000 7f1e58220740 1 -- 172.21.15.47:0/3976902792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c5c82c6aa0 con 0x55c5c8255870 2024-03-21T11:49:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.306+0000 7f1e2fe00640 1 -- 172.21.15.47:0/3976902792 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e4801bbe0 con 0x55c5c8255870 2024-03-21T11:49:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.306+0000 7f1e2fe00640 1 -- 172.21.15.47:0/3976902792 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1e3c023070 con 0x55c5c8255870 2024-03-21T11:49:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.306+0000 7f1e58220740 10 client.33566 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.306+0000 7f1e58220740 10 client.33566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:49:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.306+0000 7f1e58220740 20 client.33566 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.306+0000 7f1e58220740 1 --2- 172.21.15.47:0/3976902792 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5c826aa90 0x55c5c82c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.306+0000 7f1e58220740 4 client.33566 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:49:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.306+0000 7f1e58220740 1 -- 172.21.15.47:0/3976902792 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c5c81c1ae0 con 0x55c5c826aa90 2024-03-21T11:49:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.306+0000 7f1e4ea00640 1 --2- 172.21.15.47:0/3976902792 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5c826aa90 0x55c5c82c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.307+0000 7f1e4ea00640 1 --2- 172.21.15.47:0/3976902792 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5c826aa90 0x55c5c82c9490 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dc6936798d6ef63 server_cookie=b7862d21b5e43b45 in_seq=0 out_seq=0 2024-03-21T11:49:05.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.307+0000 7f1e2fe00640 10 client.33566 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.308+0000 7f1e2fe00640 1 -- 172.21.15.47:0/3976902792 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c5c81c1ae0 con 0x55c5c826aa90 2024-03-21T11:49:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.308+0000 7f1e2fe00640 10 client.33566 handle_command_reply: tid=0 2024-03-21T11:49:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.308+0000 7f1e58220740 1 client.33566 shutdown 2024-03-21T11:49:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.308+0000 7f1e58220740 1 -- 172.21.15.47:0/3976902792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8255870 msgr2=0x55c5c8264770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.308+0000 7f1e58220740 1 --2- 172.21.15.47:0/3976902792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8255870 0x55c5c8264770 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c01a920 tx=0x7f1e3c00f790 comp rx=0 tx=0).stop 2024-03-21T11:49:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.308+0000 7f1e58220740 1 -- 172.21.15.47:0/3976902792 shutdown_connections 2024-03-21T11:49:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.308+0000 7f1e58220740 1 --2- 172.21.15.47:0/3976902792 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55c5c826aa90 0x55c5c82c9490 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.308+0000 7f1e58220740 1 --2- 172.21.15.47:0/3976902792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c5c825a9e0 0x55c5c82656e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.308+0000 7f1e58220740 1 --2- 172.21.15.47:0/3976902792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5c8255870 0x55c5c8264770 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.308+0000 7f1e58220740 1 --2- 172.21.15.47:0/3976902792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c5c8254e80 0x55c5c8264230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.308+0000 7f1e58220740 1 -- 172.21.15.47:0/3976902792 >> 172.21.15.47:0/3976902792 conn(0x55c5c821d200 msgr2=0x55c5c8256b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.308+0000 7f1e2fe00640 0 client.33566 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.308+0000 7f1e58220740 1 -- 172.21.15.47:0/3976902792 shutdown_connections 2024-03-21T11:49:05.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.308+0000 7f1e58220740 1 -- 172.21.15.47:0/3976902792 wait complete. 2024-03-21T11:49:05.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.309+0000 7f1e58220740 20 client.33566 trim_cache size 0 max 16384 2024-03-21T11:49:05.312 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:49:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:49:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:49:05.312 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:49:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.310+0000 7f1e55600640 1 -- 172.21.15.47:0/2276235485 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1e340a2fa0 msgr2=0x7f1e340a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.310+0000 7f1e55600640 1 --2- 172.21.15.47:0/2276235485 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1e340a2fa0 0x7f1e340a5460 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1e48019000 tx=0x7f1e4801b000 comp rx=0 tx=0).stop 2024-03-21T11:49:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.310+0000 7f1e55600640 1 -- 172.21.15.47:0/2276235485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e50100e00 msgr2=0x7f1e50199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.310+0000 7f1e55600640 1 --2- 172.21.15.47:0/2276235485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e50100e00 0x7f1e50199c40 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f1e40000c00 tx=0x7f1e4002c6e0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.310+0000 7f1e55600640 1 -- 172.21.15.47:0/2276235485 shutdown_connections 2024-03-21T11:49:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.310+0000 7f1e55600640 1 --2- 172.21.15.47:0/2276235485 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f1e340a2fa0 0x7f1e340a5460 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.310+0000 7f1e55600640 1 --2- 172.21.15.47:0/2276235485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e50106230 0x7f1e501a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.310+0000 7f1e55600640 1 --2- 172.21.15.47:0/2276235485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e50103730 0x7f1e5019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.310+0000 7f1e55600640 1 --2- 172.21.15.47:0/2276235485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e50100e00 0x7f1e50199c40 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.310+0000 7f1e55600640 1 -- 172.21.15.47:0/2276235485 >> 172.21.15.47:0/2276235485 conn(0x7f1e500fae60 msgr2=0x7f1e50100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:05.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.310+0000 7f1e55600640 1 -- 172.21.15.47:0/2276235485 shutdown_connections 2024-03-21T11:49:05.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.310+0000 7f1e55600640 1 -- 172.21.15.47:0/2276235485 wait complete. 2024-03-21T11:49:05.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:05 smithi047 ceph-mon[31351]: pgmap v1173: 129 pgs: 129 active+clean; 740 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.4 MiB/s wr, 674 op/s 2024-03-21T11:49:05.334 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:49:05.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:49:05.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:05 smithi102 ceph-mon[36748]: pgmap v1173: 129 pgs: 129 active+clean; 740 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.4 MiB/s wr, 674 op/s 2024-03-21T11:49:05.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.620+0000 7fce8ee00640 1 -- 172.21.15.47:0/2513485372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce88101f70 msgr2=0x7fce88104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:05.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.620+0000 7fce8ee00640 1 --2- 172.21.15.47:0/2513485372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce88101f70 0x7fce88104360 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009900 tx=0x7fce7c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:49:05.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.620+0000 7fce8ee00640 1 -- 172.21.15.47:0/2513485372 shutdown_connections 2024-03-21T11:49:05.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.620+0000 7fce8ee00640 1 --2- 172.21.15.47:0/2513485372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce881071d0 0x7fce88109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.620+0000 7fce8ee00640 1 --2- 172.21.15.47:0/2513485372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce881048a0 0x7fce88106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.620+0000 7fce8ee00640 1 --2- 172.21.15.47:0/2513485372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce88101f70 0x7fce88104360 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.620+0000 7fce8ee00640 1 -- 172.21.15.47:0/2513485372 >> 172.21.15.47:0/2513485372 conn(0x7fce880fb980 msgr2=0x7fce880fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:05.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.620+0000 7fce8ee00640 1 -- 172.21.15.47:0/2513485372 shutdown_connections 2024-03-21T11:49:05.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.620+0000 7fce8ee00640 1 -- 172.21.15.47:0/2513485372 wait complete. 2024-03-21T11:49:05.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.621+0000 7fce8ee00640 1 Processor -- start 2024-03-21T11:49:05.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.621+0000 7fce8ee00640 1 -- start start 2024-03-21T11:49:05.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.621+0000 7fce8ee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce88101f70 0x7fce881999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.621+0000 7fce8ee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce881048a0 0x7fce88199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8ee00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce881071d0 0x7fce881a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8ee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8810d1d0 con 0x7fce88101f70 2024-03-21T11:49:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8ee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8810d050 con 0x7fce881048a0 2024-03-21T11:49:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8ee00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8810d350 con 0x7fce881071d0 2024-03-21T11:49:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce88101f70 0x7fce881999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce881071d0 0x7fce881a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce881048a0 0x7fce88199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce88101f70 0x7fce881999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59824/0 (socket says 172.21.15.47:59824) 2024-03-21T11:49:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8da00640 1 -- 172.21.15.47:0/1785315482 learned_addr learned my addr 172.21.15.47:0/1785315482 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:05.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce881048a0 0x7fce88199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54732/0 (socket says 172.21.15.47:54732) 2024-03-21T11:49:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8e400640 1 -- 172.21.15.47:0/1785315482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce881048a0 msgr2=0x7fce88199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8e400640 1 --2- 172.21.15.47:0/1785315482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce881048a0 0x7fce88199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8e400640 1 -- 172.21.15.47:0/1785315482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce88101f70 msgr2=0x7fce881999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8e400640 1 --2- 172.21.15.47:0/1785315482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce88101f70 0x7fce881999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8e400640 1 -- 172.21.15.47:0/1785315482 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce881a1600 con 0x7fce881071d0 2024-03-21T11:49:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.622+0000 7fce8da00640 1 --2- 172.21.15.47:0/1785315482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce88101f70 0x7fce881999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.623+0000 7fce8e400640 1 --2- 172.21.15.47:0/1785315482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce881071d0 0x7fce881a0f90 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fce8000e940 tx=0x7fce8000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.623+0000 7fce86000640 1 -- 172.21.15.47:0/1785315482 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80004d20 con 0x7fce881071d0 2024-03-21T11:49:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.623+0000 7fce86000640 1 -- 172.21.15.47:0/1785315482 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce80005020 con 0x7fce881071d0 2024-03-21T11:49:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.623+0000 7fce86000640 1 -- 172.21.15.47:0/1785315482 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce800116b0 con 0x7fce881071d0 2024-03-21T11:49:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.623+0000 7fce8ee00640 1 -- 172.21.15.47:0/1785315482 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce881a18f0 con 0x7fce881071d0 2024-03-21T11:49:05.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.623+0000 7fce8ee00640 1 -- 172.21.15.47:0/1785315482 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce881a1df0 con 0x7fce881071d0 2024-03-21T11:49:05.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.625+0000 7fce8ee00640 1 -- 172.21.15.47:0/1785315482 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce88108d20 con 0x7fce881071d0 2024-03-21T11:49:05.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.627+0000 7fce86000640 1 -- 172.21.15.47:0/1785315482 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fce800054d0 con 0x7fce881071d0 2024-03-21T11:49:05.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.629+0000 7fce86000640 1 --2- 172.21.15.47:0/1785315482 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fce700a2fa0 0x7fce700a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:05.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.629+0000 7fce86000640 1 -- 172.21.15.47:0/1785315482 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fce800cfb00 con 0x7fce881071d0 2024-03-21T11:49:05.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.629+0000 7fce8da00640 1 --2- 172.21.15.47:0/1785315482 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fce700a2fa0 0x7fce700a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:05.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.630+0000 7fce8da00640 1 --2- 172.21.15.47:0/1785315482 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fce700a2fa0 0x7fce700a5460 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fce7c02c8e0 tx=0x7fce7c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:05.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.637+0000 7fce86000640 1 -- 172.21.15.47:0/1785315482 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fce80016030 con 0x7fce881071d0 2024-03-21T11:49:05.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:05 smithi100 ceph-mon[36245]: pgmap v1173: 129 pgs: 129 active+clean; 740 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 8.4 MiB/s wr, 674 op/s 2024-03-21T11:49:05.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.991+0000 7fce8ee00640 1 -- 172.21.15.47:0/1785315482 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce8802d070 con 0x7fce881071d0 2024-03-21T11:49:05.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.992+0000 7fce86000640 1 -- 172.21.15.47:0/1785315482 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+11472 (secure 0 0 0) 0x7fce8009bef0 con 0x7fce881071d0 2024-03-21T11:49:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:49:05.993 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:45:39.125075+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_32623":{"gid":32623,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4278877119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4278877119},{"type":"v1","addr":"172.21.15.100:6835","nonce":4278877119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24523,24524,24535,24553,24581]},"id":1}]} 2024-03-21T11:49:05.993 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 41 2024-03-21T11:49:05.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.998+0000 7fce8ee00640 1 -- 172.21.15.47:0/1785315482 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fce700a2fa0 msgr2=0x7fce700a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:05.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.998+0000 7fce8ee00640 1 --2- 172.21.15.47:0/1785315482 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fce700a2fa0 0x7fce700a5460 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fce7c02c8e0 tx=0x7fce7c02df90 comp rx=0 tx=0).stop 2024-03-21T11:49:05.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.998+0000 7fce8ee00640 1 -- 172.21.15.47:0/1785315482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce881071d0 msgr2=0x7fce881a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:05.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.998+0000 7fce8ee00640 1 --2- 172.21.15.47:0/1785315482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce881071d0 0x7fce881a0f90 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fce8000e940 tx=0x7fce8000ee00 comp rx=0 tx=0).stop 2024-03-21T11:49:06.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.999+0000 7fce8ee00640 1 -- 172.21.15.47:0/1785315482 shutdown_connections 2024-03-21T11:49:06.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.999+0000 7fce8ee00640 1 --2- 172.21.15.47:0/1785315482 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fce700a2fa0 0x7fce700a5460 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.999+0000 7fce8ee00640 1 --2- 172.21.15.47:0/1785315482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce881071d0 0x7fce881a0f90 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.999+0000 7fce8ee00640 1 --2- 172.21.15.47:0/1785315482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce881048a0 0x7fce88199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.999+0000 7fce8ee00640 1 --2- 172.21.15.47:0/1785315482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce88101f70 0x7fce881999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:05.999+0000 7fce8ee00640 1 -- 172.21.15.47:0/1785315482 >> 172.21.15.47:0/1785315482 conn(0x7fce880fb980 msgr2=0x7fce881016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.000+0000 7fce8ee00640 1 -- 172.21.15.47:0/1785315482 shutdown_connections 2024-03-21T11:49:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.000+0000 7fce8ee00640 1 -- 172.21.15.47:0/1785315482 wait complete. 2024-03-21T11:49:06.006 INFO:tasks.workunit.client.0.smithi047.stdout: 10 291945 21.37 MB/sec execute 589 sec latency 93.227 ms 2024-03-21T11:49:06.030 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:49:06.030 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:49:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.298+0000 7fdd4f200640 1 -- 172.21.15.47:0/3632714035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481037a0 msgr2=0x7fdd48103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.298+0000 7fdd4f200640 1 --2- 172.21.15.47:0/3632714035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481037a0 0x7fdd48103bc0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009900 tx=0x7fdd3c02bc60 comp rx=0 tx=0).stop 2024-03-21T11:49:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.298+0000 7fdd4f200640 1 -- 172.21.15.47:0/3632714035 shutdown_connections 2024-03-21T11:49:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.298+0000 7fdd4f200640 1 --2- 172.21.15.47:0/3632714035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd48105350 0x7fdd4810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.298+0000 7fdd4f200640 1 --2- 172.21.15.47:0/3632714035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd48104990 0x7fdd48104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.298+0000 7fdd4f200640 1 --2- 172.21.15.47:0/3632714035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481037a0 0x7fdd48103bc0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.298+0000 7fdd4f200640 1 -- 172.21.15.47:0/3632714035 >> 172.21.15.47:0/3632714035 conn(0x7fdd480fed10 msgr2=0x7fdd48101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.299+0000 7fdd4f200640 1 -- 172.21.15.47:0/3632714035 shutdown_connections 2024-03-21T11:49:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.299+0000 7fdd4f200640 1 -- 172.21.15.47:0/3632714035 wait complete. 2024-03-21T11:49:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.301+0000 7fdd4f200640 1 Processor -- start 2024-03-21T11:49:06.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.301+0000 7fdd4f200640 1 -- start start 2024-03-21T11:49:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.303+0000 7fdd4f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481037a0 0x7fdd4819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.303+0000 7fdd4f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd48104990 0x7fdd4819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.303+0000 7fdd4f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd48105350 0x7fdd481a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.303+0000 7fdd4f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4810f4e0 con 0x7fdd48104990 2024-03-21T11:49:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.303+0000 7fdd4f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4810f360 con 0x7fdd481037a0 2024-03-21T11:49:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.303+0000 7fdd4f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4810f660 con 0x7fdd48105350 2024-03-21T11:49:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.303+0000 7fdd4de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481037a0 0x7fdd4819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.303+0000 7fdd4de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481037a0 0x7fdd4819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54754/0 (socket says 172.21.15.47:54754) 2024-03-21T11:49:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.303+0000 7fdd4de00640 1 -- 172.21.15.47:0/1174535405 learned_addr learned my addr 172.21.15.47:0/1174535405 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.303+0000 7fdd4d400640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd48104990 0x7fdd4819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.304+0000 7fdd4e800640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd48105350 0x7fdd481a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.304+0000 7fdd4de00640 1 -- 172.21.15.47:0/1174535405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd48105350 msgr2=0x7fdd481a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.304+0000 7fdd4de00640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd48105350 0x7fdd481a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.304+0000 7fdd4de00640 1 -- 172.21.15.47:0/1174535405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd48104990 msgr2=0x7fdd4819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.304+0000 7fdd4de00640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd48104990 0x7fdd4819e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.304+0000 7fdd4de00640 1 -- 172.21.15.47:0/1174535405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd481a5b70 con 0x7fdd481037a0 2024-03-21T11:49:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.304+0000 7fdd4d400640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd48104990 0x7fdd4819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.305+0000 7fdd4de00640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481037a0 0x7fdd4819df50 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c02efd0 tx=0x7fdd3c005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.305+0000 7fdd42a00640 1 -- 172.21.15.47:0/1174535405 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c041070 con 0x7fdd481037a0 2024-03-21T11:49:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.305+0000 7fdd42a00640 1 -- 172.21.15.47:0/1174535405 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd3c0054e0 con 0x7fdd481037a0 2024-03-21T11:49:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.305+0000 7fdd42a00640 1 -- 172.21.15.47:0/1174535405 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c034440 con 0x7fdd481037a0 2024-03-21T11:49:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.305+0000 7fdd4f200640 1 -- 172.21.15.47:0/1174535405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd481a5e00 con 0x7fdd481037a0 2024-03-21T11:49:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.305+0000 7fdd4f200640 1 -- 172.21.15.47:0/1174535405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd481a61b0 con 0x7fdd481037a0 2024-03-21T11:49:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.310+0000 7fdd535c2740 1 Processor -- start 2024-03-21T11:49:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.310+0000 7fdd535c2740 1 -- start start 2024-03-21T11:49:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.310+0000 7fdd535c2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f939f0 0x55b767f93e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.310+0000 7fdd535c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 0x55b767f95040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.310+0000 7fdd535c2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f95580 0x55b767f9be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.310+0000 7fdd535c2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b767ef82c0 con 0x55b767f939f0 2024-03-21T11:49:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.310+0000 7fdd535c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b767ea0c90 con 0x55b767f94bc0 2024-03-21T11:49:06.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.310+0000 7fdd535c2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b767ef4360 con 0x55b767f95580 2024-03-21T11:49:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.311+0000 7fdd4e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f939f0 0x55b767f93e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.311+0000 7fdd4d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f95580 0x55b767f9be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.311+0000 7fdd4d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f95580 0x55b767f9be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45558/0 (socket says 172.21.15.47:45558) 2024-03-21T11:49:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.311+0000 7fdd4d400640 1 -- 172.21.15.47:0/3919462019 learned_addr learned my addr 172.21.15.47:0/3919462019 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.311+0000 7fdd4de00640 1 --2- 172.21.15.47:0/3919462019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 0x55b767f95040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:06 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1785315482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:49:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.312+0000 7fdd4d400640 1 -- 172.21.15.47:0/3919462019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 msgr2=0x55b767f95040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.312+0000 7fdd4d400640 1 --2- 172.21.15.47:0/3919462019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 0x55b767f95040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.312+0000 7fdd4d400640 1 -- 172.21.15.47:0/3919462019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f939f0 msgr2=0x55b767f93e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.312+0000 7fdd4d400640 1 --2- 172.21.15.47:0/3919462019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f939f0 0x55b767f93e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.312+0000 7fdd4d400640 1 -- 172.21.15.47:0/3919462019 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b767ec5080 con 0x55b767f95580 2024-03-21T11:49:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.314+0000 7fdd4d400640 1 --2- 172.21.15.47:0/3919462019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f95580 0x55b767f9be10 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fdd380104a0 tx=0x7fdd38010960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a091044b1f205092 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.314+0000 7fdd37000640 1 -- 172.21.15.47:0/3919462019 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38005220 con 0x55b767f95580 2024-03-21T11:49:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.315+0000 7fdd37000640 1 -- 172.21.15.47:0/3919462019 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd38004c80 con 0x55b767f95580 2024-03-21T11:49:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.315+0000 7fdd37000640 1 -- 172.21.15.47:0/3919462019 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38011c00 con 0x55b767f95580 2024-03-21T11:49:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.315+0000 7fdd535c2740 1 -- 172.21.15.47:0/3919462019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f95580 msgr2=0x55b767f9be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.315+0000 7fdd535c2740 1 --2- 172.21.15.47:0/3919462019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f95580 0x55b767f9be10 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fdd380104a0 tx=0x7fdd38010960 comp rx=0 tx=0).stop 2024-03-21T11:49:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.316+0000 7fdd42a00640 1 -- 172.21.15.47:0/1174535405 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdd3c044070 con 0x7fdd481037a0 2024-03-21T11:49:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.316+0000 7fdd535c2740 1 -- 172.21.15.47:0/3919462019 shutdown_connections 2024-03-21T11:49:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.316+0000 7fdd535c2740 1 --2- 172.21.15.47:0/3919462019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f95580 0x55b767f9be10 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.316+0000 7fdd535c2740 1 --2- 172.21.15.47:0/3919462019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 0x55b767f95040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.316+0000 7fdd535c2740 1 --2- 172.21.15.47:0/3919462019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f939f0 0x55b767f93e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.316+0000 7fdd535c2740 1 -- 172.21.15.47:0/3919462019 >> 172.21.15.47:0/3919462019 conn(0x55b767f8ede0 msgr2=0x55b767f8f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.317+0000 7fdd535c2740 1 -- 172.21.15.47:0/3919462019 shutdown_connections 2024-03-21T11:49:06.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.317+0000 7fdd535c2740 1 -- 172.21.15.47:0/3919462019 wait complete. 2024-03-21T11:49:06.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.318+0000 7fdd535c2740 1 -- start start 2024-03-21T11:49:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.318+0000 7fdd535c2740 1 Processor -- start 2024-03-21T11:49:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.319+0000 7fdd535c2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767faec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.319+0000 7fdd535c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 0x55b767faf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.319+0000 7fdd535c2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f95580 0x55b767fb61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.319+0000 7fdd535c2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b767fad5a0 con 0x55b767f95580 2024-03-21T11:49:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.319+0000 7fdd535c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b767eb40d0 con 0x55b767f94bc0 2024-03-21T11:49:06.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.319+0000 7fdd535c2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b767fad720 con 0x55b767f939f0 2024-03-21T11:49:06.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.319+0000 7fdd535c2740 10 client.0 _ll_register_callbacks cb 0x55b767eb8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:06.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.320+0000 7fdd42a00640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd300a2ec0 0x7fdd300a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.320+0000 7fdd4e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767faec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.320+0000 7fdd4e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767faec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45568/0 (socket says 172.21.15.47:45568) 2024-03-21T11:49:06.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.320+0000 7fdd4e800640 1 -- 172.21.15.47:0/3024630203 learned_addr learned my addr 172.21.15.47:0/3024630203 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.321+0000 7fdd4d400640 1 --2- 172.21.15.47:0/3024630203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f95580 0x55b767fb61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.321+0000 7fdd4de00640 1 --2- 172.21.15.47:0/3024630203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 0x55b767faf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.321+0000 7fdd4e800640 1 -- 172.21.15.47:0/3024630203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 msgr2=0x55b767faf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.321+0000 7fdd4e800640 1 --2- 172.21.15.47:0/3024630203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 0x55b767faf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.321+0000 7fdd4e800640 1 -- 172.21.15.47:0/3024630203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f95580 msgr2=0x55b767fb61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.321+0000 7fdd4e800640 1 --2- 172.21.15.47:0/3024630203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f95580 0x55b767fb61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.321+0000 7fdd4e800640 1 -- 172.21.15.47:0/3024630203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b767c0c3f0 con 0x55b767f939f0 2024-03-21T11:49:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.321+0000 7fdd42a00640 1 -- 172.21.15.47:0/1174535405 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdd3c0f6990 con 0x7fdd481037a0 2024-03-21T11:49:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.321+0000 7fdd4e800640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd300a2ec0 0x7fdd300a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.321+0000 7fdd4d400640 1 --2- 172.21.15.47:0/3024630203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f95580 0x55b767fb61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:49:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.322+0000 7fdd4e800640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd300a2ec0 0x7fdd300a5380 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400acb0 tx=0x7fdd44016000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.322+0000 7fdd4e800640 1 --2- 172.21.15.47:0/3024630203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767faec40 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400bf80 tx=0x7fdd4400a7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e8631333df95d6a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.322+0000 7fdd2b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:49:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.323+0000 7fdd2b400640 1 -- 172.21.15.47:0/3024630203 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44026060 con 0x55b767f939f0 2024-03-21T11:49:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.323+0000 7fdd535c2740 1 -- 172.21.15.47:0/3024630203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd38004c80 con 0x55b767f939f0 2024-03-21T11:49:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.323+0000 7fdd2b400640 1 -- 172.21.15.47:0/3024630203 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdd44021420 con 0x55b767f939f0 2024-03-21T11:49:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.323+0000 7fdd2b400640 1 -- 172.21.15.47:0/3024630203 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd4402d050 con 0x55b767f939f0 2024-03-21T11:49:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.323+0000 7fdd535c2740 10 client.33514 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.323+0000 7fdd535c2740 1 -- 172.21.15.47:0/3024630203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55b767cd67d0 con 0x55b767f939f0 2024-03-21T11:49:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.324+0000 7fdd2b400640 1 -- 172.21.15.47:0/3024630203 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd38004c80 con 0x55b767f939f0 2024-03-21T11:49:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.324+0000 7fdd2b400640 1 -- 172.21.15.47:0/3024630203 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdd44017070 con 0x55b767f939f0 2024-03-21T11:49:06.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.324+0000 7fdd535c2740 10 client.33514 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:06.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.324+0000 7fdd535c2740 10 client.33514 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:49:06.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.324+0000 7fdd535c2740 20 client.33514 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:06.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.324+0000 7fdd535c2740 1 --2- 172.21.15.47:0/3024630203 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55b767fba7b0 0x55b767fbcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.324+0000 7fdd535c2740 4 client.33514 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:06.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.324+0000 7fdd535c2740 1 -- 172.21.15.47:0/3024630203 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b767f05800 con 0x55b767fba7b0 2024-03-21T11:49:06.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.325+0000 7fdd4de00640 1 --2- 172.21.15.47:0/3024630203 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55b767fba7b0 0x55b767fbcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.326+0000 7fdd4de00640 1 --2- 172.21.15.47:0/3024630203 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55b767fba7b0 0x55b767fbcc20 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f489103cd2f1b59a server_cookie=f46df1b36a7a853f in_seq=0 out_seq=0 2024-03-21T11:49:06.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.326+0000 7fdd2b400640 10 client.33514 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:49:06.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.327+0000 7fdd2b400640 1 -- 172.21.15.47:0/3024630203 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b767f05800 con 0x55b767fba7b0 2024-03-21T11:49:06.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.327+0000 7fdd2b400640 10 client.33514 handle_command_reply: tid=0 2024-03-21T11:49:06.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.327+0000 7fdd535c2740 1 client.33514 shutdown 2024-03-21T11:49:06.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.328+0000 7fdd535c2740 1 -- 172.21.15.47:0/3024630203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 msgr2=0x55b767faec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.328+0000 7fdd535c2740 1 --2- 172.21.15.47:0/3024630203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767faec40 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400bf80 tx=0x7fdd4400a7b0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.328+0000 7fdd535c2740 1 -- 172.21.15.47:0/3024630203 shutdown_connections 2024-03-21T11:49:06.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.328+0000 7fdd535c2740 1 --2- 172.21.15.47:0/3024630203 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55b767fba7b0 0x55b767fbcc20 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.328+0000 7fdd535c2740 1 --2- 172.21.15.47:0/3024630203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f95580 0x55b767fb61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.328+0000 7fdd535c2740 1 --2- 172.21.15.47:0/3024630203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 0x55b767faf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.328+0000 7fdd535c2740 1 --2- 172.21.15.47:0/3024630203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767faec40 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.328+0000 7fdd535c2740 1 -- 172.21.15.47:0/3024630203 >> 172.21.15.47:0/3024630203 conn(0x55b767f8ede0 msgr2=0x55b767f8e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.328+0000 7fdd2b400640 0 client.33514 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:49:06.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.328+0000 7fdd535c2740 1 -- 172.21.15.47:0/3024630203 shutdown_connections 2024-03-21T11:49:06.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.328+0000 7fdd535c2740 1 -- 172.21.15.47:0/3024630203 wait complete. 2024-03-21T11:49:06.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.328+0000 7fdd535c2740 20 client.33514 trim_cache size 0 max 16384 2024-03-21T11:49:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.347+0000 7fdd535c2740 1 Processor -- start 2024-03-21T11:49:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.347+0000 7fdd535c2740 1 -- start start 2024-03-21T11:49:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.347+0000 7fdd535c2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767faef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.347+0000 7fdd535c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 0x55b767fb6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.347+0000 7fdd535c2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f95580 0x55b767f8d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.347+0000 7fdd535c2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd481a7d90 con 0x55b767f95580 2024-03-21T11:49:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.347+0000 7fdd535c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd481a9710 con 0x55b767f94bc0 2024-03-21T11:49:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.347+0000 7fdd535c2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd481a83f0 con 0x55b767f939f0 2024-03-21T11:49:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.347+0000 7fdd4de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767faef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.347+0000 7fdd4de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767faef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45570/0 (socket says 172.21.15.47:45570) 2024-03-21T11:49:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.347+0000 7fdd4de00640 1 -- 172.21.15.47:0/4276455751 learned_addr learned my addr 172.21.15.47:0/4276455751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.347+0000 7fdd4d400640 1 --2- 172.21.15.47:0/4276455751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 0x55b767fb6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.348+0000 7fdd4e800640 1 --2- 172.21.15.47:0/4276455751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f95580 0x55b767f8d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.348+0000 7fdd4de00640 1 -- 172.21.15.47:0/4276455751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 msgr2=0x55b767fb6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.348+0000 7fdd4de00640 1 --2- 172.21.15.47:0/4276455751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 0x55b767fb6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.348+0000 7fdd4de00640 1 -- 172.21.15.47:0/4276455751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f95580 msgr2=0x55b767f8d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.348+0000 7fdd4de00640 1 --2- 172.21.15.47:0/4276455751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f95580 0x55b767f8d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.348+0000 7fdd4de00640 1 -- 172.21.15.47:0/4276455751 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b767f94640 con 0x55b767f939f0 2024-03-21T11:49:06.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.348+0000 7fdd4de00640 1 --2- 172.21.15.47:0/4276455751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767faef60 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c006010 tx=0x7fdd3c078040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e00768e00cd2a8af server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.349+0000 7fdd36600640 1 -- 172.21.15.47:0/4276455751 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c04b040 con 0x55b767f939f0 2024-03-21T11:49:06.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.349+0000 7fdd36600640 1 -- 172.21.15.47:0/4276455751 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd3c005690 con 0x55b767f939f0 2024-03-21T11:49:06.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.349+0000 7fdd36600640 1 -- 172.21.15.47:0/4276455751 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0780b0 con 0x55b767f939f0 2024-03-21T11:49:06.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.349+0000 7fdd535c2740 1 -- 172.21.15.47:0/4276455751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 msgr2=0x55b767faef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.349+0000 7fdd535c2740 1 --2- 172.21.15.47:0/4276455751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767faef60 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c006010 tx=0x7fdd3c078040 comp rx=0 tx=0).stop 2024-03-21T11:49:06.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.350+0000 7fdd535c2740 1 -- 172.21.15.47:0/4276455751 shutdown_connections 2024-03-21T11:49:06.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.350+0000 7fdd535c2740 1 --2- 172.21.15.47:0/4276455751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f95580 0x55b767f8d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.350+0000 7fdd535c2740 1 --2- 172.21.15.47:0/4276455751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f94bc0 0x55b767fb6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.350+0000 7fdd535c2740 1 --2- 172.21.15.47:0/4276455751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767faef60 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.350+0000 7fdd535c2740 1 -- 172.21.15.47:0/4276455751 >> 172.21.15.47:0/4276455751 conn(0x55b767f8ede0 msgr2=0x55b767fae850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.350+0000 7fdd535c2740 1 -- 172.21.15.47:0/4276455751 shutdown_connections 2024-03-21T11:49:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.350+0000 7fdd535c2740 1 -- 172.21.15.47:0/4276455751 wait complete. 2024-03-21T11:49:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.350+0000 7fdd535c2740 1 -- start start 2024-03-21T11:49:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.351+0000 7fdd535c2740 1 Processor -- start 2024-03-21T11:49:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.351+0000 7fdd535c2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767fb1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.351+0000 7fdd535c2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f94bc0 0x55b767fb22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.351+0000 7fdd535c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f95580 0x55b767fb52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.351+0000 7fdd535c2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd481a90b0 con 0x55b767f94bc0 2024-03-21T11:49:06.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.351+0000 7fdd535c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd481a8a50 con 0x55b767f95580 2024-03-21T11:49:06.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.351+0000 7fdd535c2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4819cec0 con 0x55b767f939f0 2024-03-21T11:49:06.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.351+0000 7fdd535c2740 10 client.0 _ll_register_callbacks cb 0x7fdd44009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:06.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.351+0000 7fdd4d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f94bc0 0x55b767fb22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.351+0000 7fdd4d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f94bc0 0x55b767fb22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59868/0 (socket says 172.21.15.47:59868) 2024-03-21T11:49:06.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.351+0000 7fdd4d400640 1 -- 172.21.15.47:0/1212762404 learned_addr learned my addr 172.21.15.47:0/1212762404 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.352+0000 7fdd4de00640 1 --2- 172.21.15.47:0/1212762404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767fb1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.352+0000 7fdd4e800640 1 --2- 172.21.15.47:0/1212762404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f95580 0x55b767fb52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.352+0000 7fdd4d400640 1 -- 172.21.15.47:0/1212762404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 msgr2=0x55b767fb1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.352+0000 7fdd4d400640 1 --2- 172.21.15.47:0/1212762404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767fb1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.352+0000 7fdd4d400640 1 -- 172.21.15.47:0/1212762404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f95580 msgr2=0x55b767fb52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.352+0000 7fdd4d400640 1 --2- 172.21.15.47:0/1212762404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f95580 0x55b767fb52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.352+0000 7fdd4d400640 1 -- 172.21.15.47:0/1212762404 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b767fb58b0 con 0x55b767f94bc0 2024-03-21T11:49:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.352+0000 7fdd4d400640 1 --2- 172.21.15.47:0/1212762404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f94bc0 0x55b767fb22f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdd380104a0 tx=0x7fdd38005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0752e0f989d5b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.352+0000 7fdd2b400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:49:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.353+0000 7fdd2b400640 1 -- 172.21.15.47:0/1212762404 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38005570 con 0x55b767f94bc0 2024-03-21T11:49:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.353+0000 7fdd2b400640 1 -- 172.21.15.47:0/1212762404 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdd38004660 con 0x55b767f94bc0 2024-03-21T11:49:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.353+0000 7fdd535c2740 1 -- 172.21.15.47:0/1212762404 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b767effb50 con 0x55b767f94bc0 2024-03-21T11:49:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.353+0000 7fdd2b400640 1 -- 172.21.15.47:0/1212762404 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd38004860 con 0x55b767f94bc0 2024-03-21T11:49:06.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.353+0000 7fdd535c2740 10 client.22665 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.353+0000 7fdd535c2740 1 -- 172.21.15.47:0/1212762404 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55b768016560 con 0x55b767f94bc0 2024-03-21T11:49:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.354+0000 7fdd2b400640 1 -- 172.21.15.47:0/1212762404 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b767effb50 con 0x55b767f94bc0 2024-03-21T11:49:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.354+0000 7fdd2b400640 1 -- 172.21.15.47:0/1212762404 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdd38011c20 con 0x55b767f94bc0 2024-03-21T11:49:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.354+0000 7fdd535c2740 10 client.22665 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.354+0000 7fdd535c2740 10 client.22665 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:49:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.354+0000 7fdd535c2740 20 client.22665 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.354+0000 7fdd535c2740 1 --2- 172.21.15.47:0/1212762404 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55b768016b60 0x55b768018f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.354+0000 7fdd535c2740 4 client.22665 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:49:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.354+0000 7fdd535c2740 1 -- 172.21.15.47:0/1212762404 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b767e7d500 con 0x55b768016b60 2024-03-21T11:49:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.354+0000 7fdd4de00640 1 --2- 172.21.15.47:0/1212762404 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55b768016b60 0x55b768018f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.355+0000 7fdd4de00640 1 --2- 172.21.15.47:0/1212762404 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55b768016b60 0x55b768018f50 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d107c6c366f8e5ea server_cookie=338a665d386427b2 in_seq=0 out_seq=0 2024-03-21T11:49:06.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.355+0000 7fdd2b400640 10 client.22665 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:49:06.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.358+0000 7fdd2b400640 1 -- 172.21.15.47:0/1212762404 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b767e7d500 con 0x55b768016b60 2024-03-21T11:49:06.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.358+0000 7fdd2b400640 10 client.22665 handle_command_reply: tid=0 2024-03-21T11:49:06.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.358+0000 7fdd535c2740 1 client.22665 shutdown 2024-03-21T11:49:06.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.359+0000 7fdd535c2740 1 -- 172.21.15.47:0/1212762404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f94bc0 msgr2=0x55b767fb22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.359+0000 7fdd535c2740 1 --2- 172.21.15.47:0/1212762404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f94bc0 0x55b767fb22f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdd380104a0 tx=0x7fdd38005270 comp rx=0 tx=0).stop 2024-03-21T11:49:06.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.359+0000 7fdd535c2740 1 -- 172.21.15.47:0/1212762404 shutdown_connections 2024-03-21T11:49:06.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.359+0000 7fdd535c2740 1 --2- 172.21.15.47:0/1212762404 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x55b768016b60 0x55b768018f50 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.359+0000 7fdd535c2740 1 --2- 172.21.15.47:0/1212762404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b767f95580 0x55b767fb52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.359+0000 7fdd535c2740 1 --2- 172.21.15.47:0/1212762404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b767f94bc0 0x55b767fb22f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.359+0000 7fdd535c2740 1 --2- 172.21.15.47:0/1212762404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b767f939f0 0x55b767fb1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.359+0000 7fdd2b400640 0 client.22665 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:49:06.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.359+0000 7fdd535c2740 1 -- 172.21.15.47:0/1212762404 >> 172.21.15.47:0/1212762404 conn(0x55b767f8ede0 msgr2=0x55b767f92040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.359+0000 7fdd535c2740 1 -- 172.21.15.47:0/1212762404 shutdown_connections 2024-03-21T11:49:06.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.359+0000 7fdd535c2740 1 -- 172.21.15.47:0/1212762404 wait complete. 2024-03-21T11:49:06.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.359+0000 7fdd535c2740 20 client.22665 trim_cache size 0 max 16384 2024-03-21T11:49:06.368 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:49:06.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.360+0000 7fdd4f200640 1 -- 172.21.15.47:0/1174535405 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd300a2ec0 msgr2=0x7fdd300a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.360+0000 7fdd4f200640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd300a2ec0 0x7fdd300a5380 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400acb0 tx=0x7fdd44016000 comp rx=0 tx=0).stop 2024-03-21T11:49:06.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.360+0000 7fdd4f200640 1 -- 172.21.15.47:0/1174535405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481037a0 msgr2=0x7fdd4819df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.360+0000 7fdd4f200640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481037a0 0x7fdd4819df50 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c02efd0 tx=0x7fdd3c005e40 comp rx=0 tx=0).stop 2024-03-21T11:49:06.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.360+0000 7fdd4f200640 1 -- 172.21.15.47:0/1174535405 shutdown_connections 2024-03-21T11:49:06.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.360+0000 7fdd4f200640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdd300a2ec0 0x7fdd300a5380 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.360+0000 7fdd4f200640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd48105350 0x7fdd481a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.360+0000 7fdd4f200640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd48104990 0x7fdd4819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.360+0000 7fdd4f200640 1 --2- 172.21.15.47:0/1174535405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd481037a0 0x7fdd4819df50 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.360+0000 7fdd4f200640 1 -- 172.21.15.47:0/1174535405 >> 172.21.15.47:0/1174535405 conn(0x7fdd480fed10 msgr2=0x7fdd48077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.360+0000 7fdd4f200640 1 -- 172.21.15.47:0/1174535405 shutdown_connections 2024-03-21T11:49:06.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.360+0000 7fdd4f200640 1 -- 172.21.15.47:0/1174535405 wait complete. 2024-03-21T11:49:06.380 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:49:06.380 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:49:06.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:06 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1785315482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:49:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.594+0000 7f54cbc00640 1 -- 172.21.15.47:0/2000460746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54c41038e0 msgr2=0x7f54c4103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.594+0000 7f54cbc00640 1 --2- 172.21.15.47:0/2000460746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54c41038e0 0x7f54c4103d00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009860 tx=0x7f54b802bb70 comp rx=0 tx=0).stop 2024-03-21T11:49:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.594+0000 7f54cbc00640 1 -- 172.21.15.47:0/2000460746 shutdown_connections 2024-03-21T11:49:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.594+0000 7f54cbc00640 1 --2- 172.21.15.47:0/2000460746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c4105490 0x7f54c410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.594+0000 7f54cbc00640 1 --2- 172.21.15.47:0/2000460746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54c4104ad0 0x7f54c4104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.594+0000 7f54cbc00640 1 --2- 172.21.15.47:0/2000460746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54c41038e0 0x7f54c4103d00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.594+0000 7f54cbc00640 1 -- 172.21.15.47:0/2000460746 >> 172.21.15.47:0/2000460746 conn(0x7f54c40fee50 msgr2=0x7f54c4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.594+0000 7f54cbc00640 1 -- 172.21.15.47:0/2000460746 shutdown_connections 2024-03-21T11:49:06.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.594+0000 7f54cbc00640 1 -- 172.21.15.47:0/2000460746 wait complete. 2024-03-21T11:49:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.595+0000 7f54cbc00640 1 Processor -- start 2024-03-21T11:49:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.595+0000 7f54cbc00640 1 -- start start 2024-03-21T11:49:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.595+0000 7f54cbc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54c41038e0 0x7f54c4199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.595+0000 7f54cbc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54c4104ad0 0x7f54c419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.595+0000 7f54cbc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c4105490 0x7f54c41a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.596+0000 7f54cbc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c410f5b0 con 0x7f54c4104ad0 2024-03-21T11:49:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.596+0000 7f54cbc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c410f430 con 0x7f54c4105490 2024-03-21T11:49:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.596+0000 7f54cbc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c410f730 con 0x7f54c41038e0 2024-03-21T11:49:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.596+0000 7f54c9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54c41038e0 0x7f54c4199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.596+0000 7f54c9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54c41038e0 0x7f54c4199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45612/0 (socket says 172.21.15.47:45612) 2024-03-21T11:49:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.596+0000 7f54c9200640 1 -- 172.21.15.47:0/2690862653 learned_addr learned my addr 172.21.15.47:0/2690862653 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.596+0000 7f54c3e00640 1 --2- 172.21.15.47:0/2690862653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54c4104ad0 0x7f54c419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.596+0000 7f54c9c00640 1 --2- 172.21.15.47:0/2690862653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c4105490 0x7f54c41a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.596+0000 7f54c9200640 1 -- 172.21.15.47:0/2690862653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c4105490 msgr2=0x7f54c41a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.596+0000 7f54c9200640 1 --2- 172.21.15.47:0/2690862653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c4105490 0x7f54c41a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.596+0000 7f54c9200640 1 -- 172.21.15.47:0/2690862653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54c4104ad0 msgr2=0x7f54c419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.596+0000 7f54c9200640 1 --2- 172.21.15.47:0/2690862653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54c4104ad0 0x7f54c419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.596+0000 7f54c9200640 1 -- 172.21.15.47:0/2690862653 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c41a18b0 con 0x7f54c41038e0 2024-03-21T11:49:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.597+0000 7f54c9200640 1 --2- 172.21.15.47:0/2690862653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54c41038e0 0x7f54c4199c70 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f54b80349e0 tx=0x7f54b80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.598+0000 7f54c1600640 1 -- 172.21.15.47:0/2690862653 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8041020 con 0x7f54c41038e0 2024-03-21T11:49:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.598+0000 7f54c1600640 1 -- 172.21.15.47:0/2690862653 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54b80323b0 con 0x7f54c41038e0 2024-03-21T11:49:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.598+0000 7f54cbc00640 1 -- 172.21.15.47:0/2690862653 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c41a1b40 con 0x7f54c41038e0 2024-03-21T11:49:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.598+0000 7f54cbc00640 1 -- 172.21.15.47:0/2690862653 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c41a2030 con 0x7f54c41038e0 2024-03-21T11:49:06.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.598+0000 7f54c1600640 1 -- 172.21.15.47:0/2690862653 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8039040 con 0x7f54c41038e0 2024-03-21T11:49:06.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.602+0000 7f54c1600640 1 -- 172.21.15.47:0/2690862653 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f54b8044020 con 0x7f54c41038e0 2024-03-21T11:49:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.602+0000 7f54ce9dc740 1 Processor -- start 2024-03-21T11:49:06.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.602+0000 7f54ce9dc740 1 -- start start 2024-03-21T11:49:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.603+0000 7f54ce9dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b664810 0x56454b664c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.603+0000 7f54ce9dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6659e0 0x56454b665e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.603+0000 7f54ce9dc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b6663a0 0x56454b66cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.603+0000 7f54ce9dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56454b5c92c0 con 0x56454b6659e0 2024-03-21T11:49:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.603+0000 7f54ce9dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56454b571c90 con 0x56454b664810 2024-03-21T11:49:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.603+0000 7f54ce9dc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56454b5c5360 con 0x56454b6663a0 2024-03-21T11:49:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.603+0000 7f54c9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b664810 0x56454b664c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.603+0000 7f54c9200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6659e0 0x56454b665e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.603+0000 7f54c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b6663a0 0x56454b66cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.603+0000 7f54c9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b664810 0x56454b664c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54810/0 (socket says 172.21.15.47:54810) 2024-03-21T11:49:06.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.603+0000 7f54c9c00640 1 -- 172.21.15.47:0/3425703542 learned_addr learned my addr 172.21.15.47:0/3425703542 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.603+0000 7f54c3e00640 1 -- 172.21.15.47:0/3425703542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b664810 msgr2=0x56454b664c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.603+0000 7f54c3e00640 1 --2- 172.21.15.47:0/3425703542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b664810 0x56454b664c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.604+0000 7f54c3e00640 1 -- 172.21.15.47:0/3425703542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6659e0 msgr2=0x56454b665e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.604+0000 7f54c3e00640 1 --2- 172.21.15.47:0/3425703542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6659e0 0x56454b665e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.604+0000 7f54c3e00640 1 -- 172.21.15.47:0/3425703542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56454b596080 con 0x56454b6663a0 2024-03-21T11:49:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.604+0000 7f54c1600640 1 --2- 172.21.15.47:0/2690862653 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54ac0a3070 0x7f54ac0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.604+0000 7f54c3e00640 1 --2- 172.21.15.47:0/3425703542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b6663a0 0x56454b66cc30 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f54b0009970 tx=0x7f54b0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf4771d63a1f81c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.604+0000 7f54c9c00640 1 --2- 172.21.15.47:0/2690862653 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54ac0a3070 0x7f54ac0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.604+0000 7f54c1600640 1 -- 172.21.15.47:0/2690862653 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f54b80f0e30 con 0x7f54c41038e0 2024-03-21T11:49:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.604+0000 7f54b6000640 1 -- 172.21.15.47:0/3425703542 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b0017070 con 0x56454b6663a0 2024-03-21T11:49:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.604+0000 7f54b6000640 1 -- 172.21.15.47:0/3425703542 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54b0013470 con 0x56454b6663a0 2024-03-21T11:49:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.604+0000 7f54b6000640 1 -- 172.21.15.47:0/3425703542 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b0012630 con 0x56454b6663a0 2024-03-21T11:49:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.605+0000 7f54ce9dc740 1 -- 172.21.15.47:0/3425703542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b6663a0 msgr2=0x56454b66cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.605+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/3425703542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b6663a0 0x56454b66cc30 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f54b0009970 tx=0x7f54b0009e30 comp rx=0 tx=0).stop 2024-03-21T11:49:06.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.605+0000 7f54c9c00640 1 --2- 172.21.15.47:0/2690862653 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54ac0a3070 0x7f54ac0a5530 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x56454b6658e0 tx=0x7f54bc010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.605+0000 7f54ce9dc740 1 -- 172.21.15.47:0/3425703542 shutdown_connections 2024-03-21T11:49:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.605+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/3425703542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b6663a0 0x56454b66cc30 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.605+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/3425703542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6659e0 0x56454b665e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.605+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/3425703542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b664810 0x56454b664c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.605+0000 7f54ce9dc740 1 -- 172.21.15.47:0/3425703542 >> 172.21.15.47:0/3425703542 conn(0x56454b65fc00 msgr2=0x56454b660000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.605+0000 7f54ce9dc740 1 -- 172.21.15.47:0/3425703542 shutdown_connections 2024-03-21T11:49:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.605+0000 7f54ce9dc740 1 -- 172.21.15.47:0/3425703542 wait complete. 2024-03-21T11:49:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.606+0000 7f54ce9dc740 1 -- start start 2024-03-21T11:49:06.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.606+0000 7f54ce9dc740 1 Processor -- start 2024-03-21T11:49:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.606+0000 7f54ce9dc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b66ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.606+0000 7f54ce9dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6659e0 0x56454b66ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.606+0000 7f54ce9dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6663a0 0x56454b675fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.606+0000 7f54ce9dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56454b66d3b0 con 0x56454b6663a0 2024-03-21T11:49:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.606+0000 7f54ce9dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56454b5850d0 con 0x56454b6659e0 2024-03-21T11:49:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.606+0000 7f54ce9dc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56454b66d530 con 0x56454b664810 2024-03-21T11:49:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.606+0000 7f54ce9dc740 10 client.0 _ll_register_callbacks cb 0x56454b589e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.607+0000 7f54c9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b66ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.607+0000 7f54c3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6659e0 0x56454b66ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.607+0000 7f54c9c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6663a0 0x56454b675fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.607+0000 7f54c9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b66ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45622/0 (socket says 172.21.15.47:45622) 2024-03-21T11:49:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.607+0000 7f54c9200640 1 -- 172.21.15.47:0/1138837811 learned_addr learned my addr 172.21.15.47:0/1138837811 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.607+0000 7f54c9200640 1 -- 172.21.15.47:0/1138837811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6659e0 msgr2=0x56454b66ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.607+0000 7f54c9200640 1 --2- 172.21.15.47:0/1138837811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6659e0 0x56454b66ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.607+0000 7f54c9200640 1 -- 172.21.15.47:0/1138837811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6663a0 msgr2=0x56454b675fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.607+0000 7f54c9200640 1 --2- 172.21.15.47:0/1138837811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6663a0 0x56454b675fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.607+0000 7f54c9200640 1 -- 172.21.15.47:0/1138837811 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56454b2dd3f0 con 0x56454b664810 2024-03-21T11:49:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.607+0000 7f54c9c00640 1 --2- 172.21.15.47:0/1138837811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6663a0 0x56454b675fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.607+0000 7f54c9200640 1 --2- 172.21.15.47:0/1138837811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b66ea20 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009830 tx=0x7f54b8005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93745724283aff33 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.607+0000 7f54aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:49:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.608+0000 7f54aaa00640 1 -- 172.21.15.47:0/1138837811 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80bccb0 con 0x56454b664810 2024-03-21T11:49:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.608+0000 7f54aaa00640 1 -- 172.21.15.47:0/1138837811 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f54b8032840 con 0x56454b664810 2024-03-21T11:49:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.608+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1138837811 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54b0013470 con 0x56454b664810 2024-03-21T11:49:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.608+0000 7f54aaa00640 1 -- 172.21.15.47:0/1138837811 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54b80347d0 con 0x56454b664810 2024-03-21T11:49:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.608+0000 7f54ce9dc740 10 client.33526 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.608+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1138837811 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56454b3a77d0 con 0x56454b664810 2024-03-21T11:49:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.608+0000 7f54aaa00640 1 -- 172.21.15.47:0/1138837811 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54b0013470 con 0x56454b664810 2024-03-21T11:49:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.608+0000 7f54aaa00640 1 -- 172.21.15.47:0/1138837811 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f54b80f4020 con 0x56454b664810 2024-03-21T11:49:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.608+0000 7f54ce9dc740 10 client.33526 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:06.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.608+0000 7f54ce9dc740 10 client.33526 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:49:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.609+0000 7f54ce9dc740 20 client.33526 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.609+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1138837811 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56454b67a5b0 0x56454b67ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.609+0000 7f54ce9dc740 4 client.33526 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.609+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1138837811 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56454b5d6800 con 0x56454b67a5b0 2024-03-21T11:49:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.609+0000 7f54c3e00640 1 --2- 172.21.15.47:0/1138837811 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56454b67a5b0 0x56454b67ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.610+0000 7f54c3e00640 1 --2- 172.21.15.47:0/1138837811 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56454b67a5b0 0x56454b67ca10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bd30a7b215e3a05a server_cookie=b2208d6fa5cbb4a2 in_seq=0 out_seq=0 2024-03-21T11:49:06.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.610+0000 7f54aaa00640 10 client.33526 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:49:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.611+0000 7f54aaa00640 1 -- 172.21.15.47:0/1138837811 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56454b5d6800 con 0x56454b67a5b0 2024-03-21T11:49:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.611+0000 7f54aaa00640 10 client.33526 handle_command_reply: tid=0 2024-03-21T11:49:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.611+0000 7f54ce9dc740 1 client.33526 shutdown 2024-03-21T11:49:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.611+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1138837811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 msgr2=0x56454b66ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.611+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1138837811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b66ea20 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009830 tx=0x7f54b8005c90 comp rx=0 tx=0).stop 2024-03-21T11:49:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.611+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1138837811 shutdown_connections 2024-03-21T11:49:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.611+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1138837811 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56454b67a5b0 0x56454b67ca10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.611+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1138837811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6663a0 0x56454b675fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.611+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1138837811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6659e0 0x56454b66ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.611+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1138837811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b66ea20 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.611+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1138837811 >> 172.21.15.47:0/1138837811 conn(0x56454b65fc00 msgr2=0x56454b65eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.611+0000 7f54aaa00640 0 client.33526 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:49:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.612+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1138837811 shutdown_connections 2024-03-21T11:49:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.612+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1138837811 wait complete. 2024-03-21T11:49:06.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.612+0000 7f54ce9dc740 20 client.33526 trim_cache size 0 max 16384 2024-03-21T11:49:06.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.630+0000 7f54ce9dc740 1 Processor -- start 2024-03-21T11:49:06.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.630+0000 7f54ce9dc740 1 -- start start 2024-03-21T11:49:06.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.630+0000 7f54ce9dc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b66eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.630+0000 7f54ce9dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6659e0 0x56454b675f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.630+0000 7f54ce9dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6663a0 0x56454b668c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.630+0000 7f54ce9dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a400f8b0 con 0x56454b6663a0 2024-03-21T11:49:06.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.630+0000 7f54ce9dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4011230 con 0x56454b6659e0 2024-03-21T11:49:06.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.630+0000 7f54ce9dc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a400ff10 con 0x56454b664810 2024-03-21T11:49:06.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.630+0000 7f54c9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b66eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.630+0000 7f54c9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b66eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45634/0 (socket says 172.21.15.47:45634) 2024-03-21T11:49:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.630+0000 7f54c9200640 1 -- 172.21.15.47:0/1138309891 learned_addr learned my addr 172.21.15.47:0/1138309891 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.630+0000 7f54c3e00640 1 --2- 172.21.15.47:0/1138309891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6659e0 0x56454b675f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.631+0000 7f54c9c00640 1 --2- 172.21.15.47:0/1138309891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6663a0 0x56454b668c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.631+0000 7f54c9200640 1 -- 172.21.15.47:0/1138309891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6659e0 msgr2=0x56454b675f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.631+0000 7f54c9200640 1 --2- 172.21.15.47:0/1138309891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6659e0 0x56454b675f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.631+0000 7f54c9200640 1 -- 172.21.15.47:0/1138309891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6663a0 msgr2=0x56454b668c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.631+0000 7f54c9200640 1 --2- 172.21.15.47:0/1138309891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6663a0 0x56454b668c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.631+0000 7f54c9200640 1 -- 172.21.15.47:0/1138309891 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56454b665460 con 0x56454b664810 2024-03-21T11:49:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.631+0000 7f54c9c00640 1 --2- 172.21.15.47:0/1138309891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6663a0 0x56454b668c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.631+0000 7f54c9200640 1 --2- 172.21.15.47:0/1138309891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b66eca0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f54b8000c00 tx=0x7f54b802bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25212692b255162 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.632+0000 7f54b5600640 1 -- 172.21.15.47:0/1138309891 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80d6070 con 0x56454b664810 2024-03-21T11:49:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.632+0000 7f54b5600640 1 -- 172.21.15.47:0/1138309891 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54b80bd170 con 0x56454b664810 2024-03-21T11:49:06.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.632+0000 7f54b5600640 1 -- 172.21.15.47:0/1138309891 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80044e0 con 0x56454b664810 2024-03-21T11:49:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.632+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1138309891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 msgr2=0x56454b66eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.632+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1138309891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b66eca0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f54b8000c00 tx=0x7f54b802bb70 comp rx=0 tx=0).stop 2024-03-21T11:49:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.632+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1138309891 shutdown_connections 2024-03-21T11:49:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.632+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1138309891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6663a0 0x56454b668c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.632+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1138309891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6659e0 0x56454b675f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.632+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1138309891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b66eca0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.632+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1138309891 >> 172.21.15.47:0/1138309891 conn(0x56454b65fc00 msgr2=0x56454b65eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.633+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1138309891 shutdown_connections 2024-03-21T11:49:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.633+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1138309891 wait complete. 2024-03-21T11:49:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.633+0000 7f54ce9dc740 1 -- start start 2024-03-21T11:49:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.633+0000 7f54ce9dc740 1 Processor -- start 2024-03-21T11:49:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.633+0000 7f54ce9dc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b673c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.633+0000 7f54ce9dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6659e0 0x56454b6741a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54ce9dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6663a0 0x56454b6751b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54c9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b673c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54ce9dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4010bd0 con 0x56454b6659e0 2024-03-21T11:49:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54ce9dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4010570 con 0x56454b6663a0 2024-03-21T11:49:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54ce9dc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a400da70 con 0x56454b664810 2024-03-21T11:49:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54c9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6663a0 0x56454b6751b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54c9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b673c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45646/0 (socket says 172.21.15.47:45646) 2024-03-21T11:49:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54c9200640 1 -- 172.21.15.47:0/1330033470 learned_addr learned my addr 172.21.15.47:0/1330033470 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54ce9dc740 10 client.0 _ll_register_callbacks cb 0x7f54b80f4560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54c3e00640 1 --2- 172.21.15.47:0/1330033470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6659e0 0x56454b6741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54c9200640 1 -- 172.21.15.47:0/1330033470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6663a0 msgr2=0x56454b6751b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54c9200640 1 --2- 172.21.15.47:0/1330033470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6663a0 0x56454b6751b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54c9200640 1 -- 172.21.15.47:0/1330033470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6659e0 msgr2=0x56454b6741a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54c9200640 1 --2- 172.21.15.47:0/1330033470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6659e0 0x56454b6741a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.634+0000 7f54c9200640 1 -- 172.21.15.47:0/1330033470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56454b6757a0 con 0x56454b664810 2024-03-21T11:49:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.635+0000 7f54c9200640 1 --2- 172.21.15.47:0/1330033470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b673c60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f54b8005e00 tx=0x7f54b8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acc74d3dee25cc57 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.635+0000 7f54aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:49:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.635+0000 7f54aaa00640 1 -- 172.21.15.47:0/1330033470 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80d6040 con 0x56454b664810 2024-03-21T11:49:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.635+0000 7f54aaa00640 1 -- 172.21.15.47:0/1330033470 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f54b8032840 con 0x56454b664810 2024-03-21T11:49:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.635+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1330033470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54b80bd170 con 0x56454b664810 2024-03-21T11:49:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.636+0000 7f54aaa00640 1 -- 172.21.15.47:0/1330033470 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54b80bcad0 con 0x56454b664810 2024-03-21T11:49:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.636+0000 7f54ce9dc740 10 client.33532 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.636+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1330033470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56454b6d63b0 con 0x56454b664810 2024-03-21T11:49:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.636+0000 7f54aaa00640 1 -- 172.21.15.47:0/1330033470 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54b80bd170 con 0x56454b664810 2024-03-21T11:49:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.636+0000 7f54aaa00640 1 -- 172.21.15.47:0/1330033470 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f54b80b7ce0 con 0x56454b664810 2024-03-21T11:49:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.637+0000 7f54ce9dc740 10 client.33532 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.637+0000 7f54ce9dc740 10 client.33532 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:49:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.637+0000 7f54ce9dc740 20 client.33532 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.637+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1330033470 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56454b6d6930 0x56454b6d8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.637+0000 7f54ce9dc740 4 client.33532 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:49:06.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.637+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1330033470 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56454b5d0b50 con 0x56454b6d6930 2024-03-21T11:49:06.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.638+0000 7f54c3e00640 1 --2- 172.21.15.47:0/1330033470 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56454b6d6930 0x56454b6d8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.639+0000 7f54c3e00640 1 --2- 172.21.15.47:0/1330033470 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56454b6d6930 0x56454b6d8d50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a8835e83ef6a01c server_cookie=90ca060ebb818bc0 in_seq=0 out_seq=0 2024-03-21T11:49:06.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.639+0000 7f54aaa00640 10 client.33532 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:49:06.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.639+0000 7f54aaa00640 1 -- 172.21.15.47:0/1330033470 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56454b5d0b50 con 0x56454b6d6930 2024-03-21T11:49:06.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.639+0000 7f54aaa00640 10 client.33532 handle_command_reply: tid=0 2024-03-21T11:49:06.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.639+0000 7f54ce9dc740 1 client.33532 shutdown 2024-03-21T11:49:06.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.640+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1330033470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 msgr2=0x56454b673c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.640+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1330033470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b673c60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f54b8005e00 tx=0x7f54b8005f90 comp rx=0 tx=0).stop 2024-03-21T11:49:06.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.640+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1330033470 shutdown_connections 2024-03-21T11:49:06.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.640+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1330033470 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x56454b6d6930 0x56454b6d8d50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.640+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1330033470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56454b6663a0 0x56454b6751b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.640+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1330033470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56454b6659e0 0x56454b6741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.640+0000 7f54ce9dc740 1 --2- 172.21.15.47:0/1330033470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56454b664810 0x56454b673c60 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.640+0000 7f54aaa00640 0 client.33532 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:49:06.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.640+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1330033470 >> 172.21.15.47:0/1330033470 conn(0x56454b65fc00 msgr2=0x56454b6636f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.640+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1330033470 shutdown_connections 2024-03-21T11:49:06.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.640+0000 7f54ce9dc740 1 -- 172.21.15.47:0/1330033470 wait complete. 2024-03-21T11:49:06.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.640+0000 7f54ce9dc740 20 client.33532 trim_cache size 0 max 16384 2024-03-21T11:49:06.642 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:49:06.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.641+0000 7f54cbc00640 1 -- 172.21.15.47:0/2690862653 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54ac0a3070 msgr2=0x7f54ac0a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.641+0000 7f54cbc00640 1 --2- 172.21.15.47:0/2690862653 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54ac0a3070 0x7f54ac0a5530 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x56454b6658e0 tx=0x7f54bc010040 comp rx=0 tx=0).stop 2024-03-21T11:49:06.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.641+0000 7f54cbc00640 1 -- 172.21.15.47:0/2690862653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54c41038e0 msgr2=0x7f54c4199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.641+0000 7f54cbc00640 1 --2- 172.21.15.47:0/2690862653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54c41038e0 0x7f54c4199c70 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f54b80349e0 tx=0x7f54b80042f0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.641+0000 7f54cbc00640 1 -- 172.21.15.47:0/2690862653 shutdown_connections 2024-03-21T11:49:06.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.641+0000 7f54cbc00640 1 --2- 172.21.15.47:0/2690862653 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f54ac0a3070 0x7f54ac0a5530 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.641+0000 7f54cbc00640 1 --2- 172.21.15.47:0/2690862653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54c4105490 0x7f54c41a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.641+0000 7f54cbc00640 1 --2- 172.21.15.47:0/2690862653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54c4104ad0 0x7f54c419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.641+0000 7f54cbc00640 1 --2- 172.21.15.47:0/2690862653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54c41038e0 0x7f54c4199c70 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.641+0000 7f54cbc00640 1 -- 172.21.15.47:0/2690862653 >> 172.21.15.47:0/2690862653 conn(0x7f54c40fee50 msgr2=0x7f54c4107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.641+0000 7f54cbc00640 1 -- 172.21.15.47:0/2690862653 shutdown_connections 2024-03-21T11:49:06.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.641+0000 7f54cbc00640 1 -- 172.21.15.47:0/2690862653 wait complete. 2024-03-21T11:49:06.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:06 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1785315482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:49:06.663 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:49:06.663 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:49:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.886+0000 7fe0bbe00640 1 -- 172.21.15.47:0/131130700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0b4109000 msgr2=0x7fe0b410f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.886+0000 7fe0bbe00640 1 --2- 172.21.15.47:0/131130700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0b4109000 0x7fe0b410f9e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009900 tx=0x7fe0a002bc60 comp rx=0 tx=0).stop 2024-03-21T11:49:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.887+0000 7fe0bbe00640 1 -- 172.21.15.47:0/131130700 shutdown_connections 2024-03-21T11:49:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.887+0000 7fe0bbe00640 1 --2- 172.21.15.47:0/131130700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0b410ff20 0x7fe0b4112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.887+0000 7fe0bbe00640 1 --2- 172.21.15.47:0/131130700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0b4109000 0x7fe0b410f9e0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.887+0000 7fe0bbe00640 1 --2- 172.21.15.47:0/131130700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0b41086a0 0x7fe0b4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.887+0000 7fe0bbe00640 1 -- 172.21.15.47:0/131130700 >> 172.21.15.47:0/131130700 conn(0x7fe0b406e500 msgr2=0x7fe0b4070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.887+0000 7fe0bbe00640 1 -- 172.21.15.47:0/131130700 shutdown_connections 2024-03-21T11:49:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.887+0000 7fe0bbe00640 1 -- 172.21.15.47:0/131130700 wait complete. 2024-03-21T11:49:06.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.888+0000 7fe0bbe00640 1 Processor -- start 2024-03-21T11:49:06.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.889+0000 7fe0bbe00640 1 -- start start 2024-03-21T11:49:06.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.889+0000 7fe0bbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0b41086a0 0x7fe0b41a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.889+0000 7fe0bbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0b4109000 0x7fe0b41a2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.889+0000 7fe0bbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0b410ff20 0x7fe0b41a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.889+0000 7fe0bbe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b4115c70 con 0x7fe0b410ff20 2024-03-21T11:49:06.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.889+0000 7fe0bbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b4115af0 con 0x7fe0b4109000 2024-03-21T11:49:06.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.889+0000 7fe0bbe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b4115df0 con 0x7fe0b41086a0 2024-03-21T11:49:06.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.889+0000 7fe0b8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0b4109000 0x7fe0b41a2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.889+0000 7fe0b8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0b4109000 0x7fe0b41a2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54836/0 (socket says 172.21.15.47:54836) 2024-03-21T11:49:06.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.889+0000 7fe0b8a00640 1 -- 172.21.15.47:0/2244407396 learned_addr learned my addr 172.21.15.47:0/2244407396 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.889+0000 7fe0b9400640 1 --2- 172.21.15.47:0/2244407396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0b41086a0 0x7fe0b41a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.890+0000 7fe0b9400640 1 -- 172.21.15.47:0/2244407396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0b4109000 msgr2=0x7fe0b41a2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.890+0000 7fe0b9400640 1 --2- 172.21.15.47:0/2244407396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0b4109000 0x7fe0b41a2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.890+0000 7fe0b9400640 1 -- 172.21.15.47:0/2244407396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0b410ff20 msgr2=0x7fe0b41a9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.890+0000 7fe0b9400640 1 --2- 172.21.15.47:0/2244407396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0b410ff20 0x7fe0b41a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.890+0000 7fe0b9400640 1 -- 172.21.15.47:0/2244407396 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b41aa180 con 0x7fe0b41086a0 2024-03-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.891+0000 7fe0b9400640 1 --2- 172.21.15.47:0/2244407396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0b41086a0 0x7fe0b41a2560 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800b3f0 tx=0x7fe0a800b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.891+0000 7fe0b2000640 1 -- 172.21.15.47:0/2244407396 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8015020 con 0x7fe0b41086a0 2024-03-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.891+0000 7fe0b2000640 1 -- 172.21.15.47:0/2244407396 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0a8011d10 con 0x7fe0b41086a0 2024-03-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.891+0000 7fe0bbe00640 1 -- 172.21.15.47:0/2244407396 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b41aa470 con 0x7fe0b41086a0 2024-03-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.891+0000 7fe0b2000640 1 -- 172.21.15.47:0/2244407396 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a800f2e0 con 0x7fe0b41086a0 2024-03-21T11:49:06.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.892+0000 7fe0bbe00640 1 -- 172.21.15.47:0/2244407396 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b41aaa10 con 0x7fe0b41086a0 2024-03-21T11:49:06.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.897+0000 7fe0b2000640 1 -- 172.21.15.47:0/2244407396 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe0a8018020 con 0x7fe0b41086a0 2024-03-21T11:49:06.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.899+0000 7fe0beb85740 1 Processor -- start 2024-03-21T11:49:06.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.900+0000 7fe0b2000640 1 --2- 172.21.15.47:0/2244407396 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe09c0a3070 0x7fe09c0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.900+0000 7fe0b2000640 1 -- 172.21.15.47:0/2244407396 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe0a80cf030 con 0x7fe0b41086a0 2024-03-21T11:49:06.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.900+0000 7fe0beb85740 1 -- start start 2024-03-21T11:49:06.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.901+0000 7fe0b9e00640 1 --2- 172.21.15.47:0/2244407396 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe09c0a3070 0x7fe09c0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.901+0000 7fe0beb85740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c7ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.901+0000 7fe0beb85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c7fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.901+0000 7fe0beb85740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 0x561084c86c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.901+0000 7fe0beb85740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084be32c0 con 0x561084c803a0 2024-03-21T11:49:06.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.901+0000 7fe0beb85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084b8bc90 con 0x561084c7f9e0 2024-03-21T11:49:06.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.901+0000 7fe0beb85740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084bdf360 con 0x561084c7e810 2024-03-21T11:49:06.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.901+0000 7fe0b9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c7ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.901+0000 7fe0b9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c7ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45650/0 (socket says 172.21.15.47:45650) 2024-03-21T11:49:06.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.901+0000 7fe0b9400640 1 -- 172.21.15.47:0/1852995643 learned_addr learned my addr 172.21.15.47:0/1852995643 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.902+0000 7fe0b8a00640 1 --2- 172.21.15.47:0/1852995643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c7fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.902+0000 7fe0b9e00640 1 --2- 172.21.15.47:0/1852995643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 0x561084c86c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.903+0000 7fe0b9400640 1 -- 172.21.15.47:0/1852995643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 msgr2=0x561084c7fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.903+0000 7fe0b9400640 1 --2- 172.21.15.47:0/1852995643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c7fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.903+0000 7fe0b9400640 1 -- 172.21.15.47:0/1852995643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 msgr2=0x561084c86c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.903+0000 7fe0b9400640 1 --2- 172.21.15.47:0/1852995643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 0x561084c86c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.903+0000 7fe0b9400640 1 -- 172.21.15.47:0/1852995643 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561084bb0080 con 0x561084c7e810 2024-03-21T11:49:06.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.903+0000 7fe0b9e00640 1 --2- 172.21.15.47:0/1852995643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 0x561084c86c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:49:06.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.903+0000 7fe0b9400640 1 --2- 172.21.15.47:0/1852995643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c7ec30 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80977d0 tx=0x7fe0a80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d71cfb5b0303a672 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.904+0000 7fe0a6600640 1 -- 172.21.15.47:0/1852995643 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a809f4c0 con 0x561084c7e810 2024-03-21T11:49:06.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.904+0000 7fe0a6600640 1 -- 172.21.15.47:0/1852995643 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0a80a3d80 con 0x561084c7e810 2024-03-21T11:49:06.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.904+0000 7fe0a6600640 1 -- 172.21.15.47:0/1852995643 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a80979e0 con 0x561084c7e810 2024-03-21T11:49:06.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.905+0000 7fe0b9e00640 1 --2- 172.21.15.47:0/2244407396 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe09c0a3070 0x7fe09c0a5530 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe0b406b390 tx=0x7fe0ac00a4e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.905+0000 7fe0beb85740 1 -- 172.21.15.47:0/1852995643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 msgr2=0x561084c7ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.905+0000 7fe0beb85740 1 --2- 172.21.15.47:0/1852995643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c7ec30 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80977d0 tx=0x7fe0a80046c0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.905+0000 7fe0beb85740 1 -- 172.21.15.47:0/1852995643 shutdown_connections 2024-03-21T11:49:06.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.905+0000 7fe0beb85740 1 --2- 172.21.15.47:0/1852995643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 0x561084c86c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.905+0000 7fe0beb85740 1 --2- 172.21.15.47:0/1852995643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c7fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.905+0000 7fe0beb85740 1 --2- 172.21.15.47:0/1852995643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c7ec30 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.905+0000 7fe0beb85740 1 -- 172.21.15.47:0/1852995643 >> 172.21.15.47:0/1852995643 conn(0x561084c79c00 msgr2=0x561084c7a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.905+0000 7fe0beb85740 1 -- 172.21.15.47:0/1852995643 shutdown_connections 2024-03-21T11:49:06.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.905+0000 7fe0beb85740 1 -- 172.21.15.47:0/1852995643 wait complete. 2024-03-21T11:49:06.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.906+0000 7fe0beb85740 1 -- start start 2024-03-21T11:49:06.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.906+0000 7fe0beb85740 1 Processor -- start 2024-03-21T11:49:06.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.907+0000 7fe0beb85740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c7e810 0x561084c91290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.907+0000 7fe0beb85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c917d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.907+0000 7fe0beb85740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c803a0 0x561084c98810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.907+0000 7fe0beb85740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084c873b0 con 0x561084c7e810 2024-03-21T11:49:06.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.907+0000 7fe0beb85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084b9f0d0 con 0x561084c7f9e0 2024-03-21T11:49:06.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.907+0000 7fe0beb85740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084c87530 con 0x561084c803a0 2024-03-21T11:49:06.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.908+0000 7fe0beb85740 10 client.0 _ll_register_callbacks cb 0x561084ba3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:06.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.908+0000 7fe0b9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c803a0 0x561084c98810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.908+0000 7fe0b9400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c7e810 0x561084c91290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.908+0000 7fe0b9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c803a0 0x561084c98810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45656/0 (socket says 172.21.15.47:45656) 2024-03-21T11:49:06.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.908+0000 7fe0b9e00640 1 -- 172.21.15.47:0/2275461755 learned_addr learned my addr 172.21.15.47:0/2275461755 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.908+0000 7fe0b8a00640 1 --2- 172.21.15.47:0/2275461755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c917d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.909+0000 7fe0b9400640 1 -- 172.21.15.47:0/2275461755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c803a0 msgr2=0x561084c98810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.909+0000 7fe0b9400640 1 --2- 172.21.15.47:0/2275461755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c803a0 0x561084c98810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.909+0000 7fe0b9400640 1 -- 172.21.15.47:0/2275461755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 msgr2=0x561084c917d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.909+0000 7fe0b9400640 1 --2- 172.21.15.47:0/2275461755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c917d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.909+0000 7fe0b9400640 1 -- 172.21.15.47:0/2275461755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610848f73f0 con 0x561084c7e810 2024-03-21T11:49:06.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.909+0000 7fe0b9400640 1 --2- 172.21.15.47:0/2275461755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c7e810 0x561084c91290 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8097d40 tx=0x7fe0a809f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100dff05cd8c1bdb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.909+0000 7fe09aa00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:49:06.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.910+0000 7fe09aa00640 1 -- 172.21.15.47:0/2275461755 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a80979e0 con 0x561084c7e810 2024-03-21T11:49:06.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.910+0000 7fe09aa00640 1 -- 172.21.15.47:0/2275461755 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe0a80b4210 con 0x561084c7e810 2024-03-21T11:49:06.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.910+0000 7fe0beb85740 1 -- 172.21.15.47:0/2275461755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0a80a3d80 con 0x561084c7e810 2024-03-21T11:49:06.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.910+0000 7fe09aa00640 1 -- 172.21.15.47:0/2275461755 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0a8097b80 con 0x561084c7e810 2024-03-21T11:49:06.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.911+0000 7fe0beb85740 10 client.22686 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:06.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.911+0000 7fe0beb85740 1 -- 172.21.15.47:0/2275461755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5610849c17d0 con 0x561084c7e810 2024-03-21T11:49:06.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.911+0000 7fe09aa00640 1 -- 172.21.15.47:0/2275461755 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0a80a3d80 con 0x561084c7e810 2024-03-21T11:49:06.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.911+0000 7fe09aa00640 1 -- 172.21.15.47:0/2275461755 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe0a80970c0 con 0x561084c7e810 2024-03-21T11:49:06.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.912+0000 7fe0beb85740 10 client.22686 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:06.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.912+0000 7fe0beb85740 10 client.22686 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:49:06.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.912+0000 7fe0beb85740 20 client.22686 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:06.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.912+0000 7fe0beb85740 1 --2- 172.21.15.47:0/2275461755 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561084c9c8e0 0x561084c9ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.912+0000 7fe0beb85740 4 client.22686 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:06.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.912+0000 7fe0beb85740 1 -- 172.21.15.47:0/2275461755 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561084bf0800 con 0x561084c9c8e0 2024-03-21T11:49:06.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.912+0000 7fe0b8a00640 1 --2- 172.21.15.47:0/2275461755 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561084c9c8e0 0x561084c9ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.913+0000 7fe0b8a00640 1 --2- 172.21.15.47:0/2275461755 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561084c9c8e0 0x561084c9ecd0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d406d8d0cb08a11 server_cookie=6d110787bdc83bc9 in_seq=0 out_seq=0 2024-03-21T11:49:06.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.913+0000 7fe09aa00640 10 client.22686 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:49:06.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.915+0000 7fe09aa00640 1 -- 172.21.15.47:0/2275461755 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561084bf0800 con 0x561084c9c8e0 2024-03-21T11:49:06.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.915+0000 7fe09aa00640 10 client.22686 handle_command_reply: tid=0 2024-03-21T11:49:06.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.915+0000 7fe0beb85740 1 client.22686 shutdown 2024-03-21T11:49:06.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.915+0000 7fe0beb85740 1 -- 172.21.15.47:0/2275461755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c7e810 msgr2=0x561084c91290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.915+0000 7fe0beb85740 1 --2- 172.21.15.47:0/2275461755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c7e810 0x561084c91290 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8097d40 tx=0x7fe0a809f540 comp rx=0 tx=0).stop 2024-03-21T11:49:06.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.916+0000 7fe0beb85740 1 -- 172.21.15.47:0/2275461755 shutdown_connections 2024-03-21T11:49:06.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.916+0000 7fe0beb85740 1 --2- 172.21.15.47:0/2275461755 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561084c9c8e0 0x561084c9ecd0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.916+0000 7fe0beb85740 1 --2- 172.21.15.47:0/2275461755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c803a0 0x561084c98810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.916+0000 7fe0beb85740 1 --2- 172.21.15.47:0/2275461755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c917d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.916+0000 7fe0beb85740 1 --2- 172.21.15.47:0/2275461755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c7e810 0x561084c91290 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.916+0000 7fe0beb85740 1 -- 172.21.15.47:0/2275461755 >> 172.21.15.47:0/2275461755 conn(0x561084c79c00 msgr2=0x561084c78eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.916+0000 7fe09aa00640 0 client.22686 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:49:06.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.916+0000 7fe0beb85740 1 -- 172.21.15.47:0/2275461755 shutdown_connections 2024-03-21T11:49:06.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.916+0000 7fe0beb85740 1 -- 172.21.15.47:0/2275461755 wait complete. 2024-03-21T11:49:06.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.916+0000 7fe0beb85740 20 client.22686 trim_cache size 0 max 16384 2024-03-21T11:49:06.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.937+0000 7fe0beb85740 1 Processor -- start 2024-03-21T11:49:06.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.937+0000 7fe0beb85740 1 -- start start 2024-03-21T11:49:06.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.938+0000 7fe0beb85740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c915b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.938+0000 7fe0beb85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c987d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.938+0000 7fe0beb85740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 0x561084c78620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.938+0000 7fe0beb85740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b41b0950 con 0x561084c803a0 2024-03-21T11:49:06.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.938+0000 7fe0beb85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b41b22d0 con 0x561084c7f9e0 2024-03-21T11:49:06.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.938+0000 7fe0beb85740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b41b0fb0 con 0x561084c7e810 2024-03-21T11:49:06.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.938+0000 7fe0b8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.938+0000 7fe0b8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54862/0 (socket says 172.21.15.47:54862) 2024-03-21T11:49:06.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.938+0000 7fe0b9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 0x561084c78620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.938+0000 7fe0b8a00640 1 -- 172.21.15.47:0/3235891661 learned_addr learned my addr 172.21.15.47:0/3235891661 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.938+0000 7fe0b9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c915b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.939+0000 7fe0b9400640 1 -- 172.21.15.47:0/3235891661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 msgr2=0x561084c987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.939+0000 7fe0b9400640 1 --2- 172.21.15.47:0/3235891661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.939+0000 7fe0b9400640 1 -- 172.21.15.47:0/3235891661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 msgr2=0x561084c78620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.939+0000 7fe0b9400640 1 --2- 172.21.15.47:0/3235891661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 0x561084c78620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.939+0000 7fe0b9400640 1 -- 172.21.15.47:0/3235891661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561084c86570 con 0x561084c7e810 2024-03-21T11:49:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.939+0000 7fe0b9e00640 1 --2- 172.21.15.47:0/3235891661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 0x561084c78620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.939+0000 7fe0b9400640 1 --2- 172.21.15.47:0/3235891661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c915b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800b3c0 tx=0x7fe0a8099880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c690e4900adb1dcb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.939+0000 7fe0a5c00640 1 -- 172.21.15.47:0/3235891661 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a80a3d80 con 0x561084c7e810 2024-03-21T11:49:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.940+0000 7fe0a5c00640 1 -- 172.21.15.47:0/3235891661 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0a809a440 con 0x561084c7e810 2024-03-21T11:49:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.940+0000 7fe0a5c00640 1 -- 172.21.15.47:0/3235891661 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a80a38f0 con 0x561084c7e810 2024-03-21T11:49:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.940+0000 7fe0beb85740 1 -- 172.21.15.47:0/3235891661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 msgr2=0x561084c915b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.940+0000 7fe0beb85740 1 --2- 172.21.15.47:0/3235891661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c915b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800b3c0 tx=0x7fe0a8099880 comp rx=0 tx=0).stop 2024-03-21T11:49:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.941+0000 7fe0beb85740 1 -- 172.21.15.47:0/3235891661 shutdown_connections 2024-03-21T11:49:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.941+0000 7fe0beb85740 1 --2- 172.21.15.47:0/3235891661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 0x561084c78620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.941+0000 7fe0beb85740 1 --2- 172.21.15.47:0/3235891661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.941+0000 7fe0beb85740 1 --2- 172.21.15.47:0/3235891661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c915b0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.941+0000 7fe0beb85740 1 -- 172.21.15.47:0/3235891661 >> 172.21.15.47:0/3235891661 conn(0x561084c79c00 msgr2=0x561084c90ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.941+0000 7fe0beb85740 1 -- 172.21.15.47:0/3235891661 shutdown_connections 2024-03-21T11:49:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.941+0000 7fe0beb85740 1 -- 172.21.15.47:0/3235891661 wait complete. 2024-03-21T11:49:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.941+0000 7fe0beb85740 1 -- start start 2024-03-21T11:49:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.941+0000 7fe0beb85740 1 Processor -- start 2024-03-21T11:49:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.942+0000 7fe0beb85740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c95fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.942+0000 7fe0beb85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c964f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.942+0000 7fe0beb85740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 0x561084c974b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.942+0000 7fe0beb85740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b41b1c70 con 0x561084c803a0 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.942+0000 7fe0beb85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b41b1610 con 0x561084c7f9e0 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.942+0000 7fe0beb85740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b41aeb50 con 0x561084c7e810 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.942+0000 7fe0beb85740 10 client.0 _ll_register_callbacks cb 0x7fe0a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.942+0000 7fe0b9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c95fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.942+0000 7fe0b8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.942+0000 7fe0b8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54870/0 (socket says 172.21.15.47:54870) 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.942+0000 7fe0b9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c95fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45680/0 (socket says 172.21.15.47:45680) 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.942+0000 7fe0b8a00640 1 -- 172.21.15.47:0/3263427235 learned_addr learned my addr 172.21.15.47:0/3263427235 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.942+0000 7fe0b9e00640 1 --2- 172.21.15.47:0/3263427235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 0x561084c974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.943+0000 7fe0b8a00640 1 -- 172.21.15.47:0/3263427235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 msgr2=0x561084c95fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.943+0000 7fe0b8a00640 1 --2- 172.21.15.47:0/3263427235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c95fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.943+0000 7fe0b8a00640 1 -- 172.21.15.47:0/3263427235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 msgr2=0x561084c974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.943+0000 7fe0b8a00640 1 --2- 172.21.15.47:0/3263427235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 0x561084c974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.943+0000 7fe0b8a00640 1 -- 172.21.15.47:0/3263427235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561084c97aa0 con 0x561084c7f9e0 2024-03-21T11:49:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.943+0000 7fe0b9400640 1 --2- 172.21.15.47:0/3263427235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c95fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.943+0000 7fe0b8a00640 1 --2- 172.21.15.47:0/3263427235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c964f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0036040 tx=0x7fe0a002cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92eabcd0967ba803 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.943+0000 7fe09aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:49:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.944+0000 7fe09aa00640 1 -- 172.21.15.47:0/3263427235 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a004b070 con 0x561084c7f9e0 2024-03-21T11:49:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.944+0000 7fe09aa00640 1 -- 172.21.15.47:0/3263427235 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fe0a0004450 con 0x561084c7f9e0 2024-03-21T11:49:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.944+0000 7fe0beb85740 1 -- 172.21.15.47:0/3263427235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0a809a440 con 0x561084c7f9e0 2024-03-21T11:49:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.944+0000 7fe09aa00640 1 -- 172.21.15.47:0/3263427235 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0a0055050 con 0x561084c7f9e0 2024-03-21T11:49:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.944+0000 7fe0beb85740 10 client.33626 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.944+0000 7fe0beb85740 1 -- 172.21.15.47:0/3263427235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561084cf7f90 con 0x561084c7f9e0 2024-03-21T11:49:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.945+0000 7fe09aa00640 1 -- 172.21.15.47:0/3263427235 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0a809a440 con 0x561084c7f9e0 2024-03-21T11:49:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.945+0000 7fe09aa00640 1 -- 172.21.15.47:0/3263427235 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe0a0004b00 con 0x561084c7f9e0 2024-03-21T11:49:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.945+0000 7fe0beb85740 10 client.33626 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.945+0000 7fe0beb85740 10 client.33626 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:49:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.945+0000 7fe0beb85740 20 client.33626 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.945+0000 7fe0beb85740 1 --2- 172.21.15.47:0/3263427235 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561084cf8930 0x561084cfad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.945+0000 7fe0beb85740 4 client.33626 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:49:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.945+0000 7fe0beb85740 1 -- 172.21.15.47:0/3263427235 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561084beab50 con 0x561084cf8930 2024-03-21T11:49:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.946+0000 7fe0b9400640 1 --2- 172.21.15.47:0/3263427235 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561084cf8930 0x561084cfad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.946+0000 7fe09aa00640 10 client.33626 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:49:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.946+0000 7fe0b9400640 1 --2- 172.21.15.47:0/3263427235 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561084cf8930 0x561084cfad50 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6d1ca91f350b10b server_cookie=1a8f63420d3dd3f9 in_seq=0 out_seq=0 2024-03-21T11:49:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.946+0000 7fe09aa00640 1 -- 172.21.15.47:0/3263427235 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561084beab50 con 0x561084cf8930 2024-03-21T11:49:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.947+0000 7fe09aa00640 10 client.33626 handle_command_reply: tid=0 2024-03-21T11:49:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.947+0000 7fe0beb85740 1 client.33626 shutdown 2024-03-21T11:49:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.947+0000 7fe0beb85740 1 -- 172.21.15.47:0/3263427235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 msgr2=0x561084c964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.947+0000 7fe0beb85740 1 --2- 172.21.15.47:0/3263427235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c964f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0036040 tx=0x7fe0a002cc50 comp rx=0 tx=0).stop 2024-03-21T11:49:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.947+0000 7fe0beb85740 1 -- 172.21.15.47:0/3263427235 shutdown_connections 2024-03-21T11:49:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.947+0000 7fe0beb85740 1 --2- 172.21.15.47:0/3263427235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561084c803a0 0x561084c974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.947+0000 7fe0beb85740 1 --2- 172.21.15.47:0/3263427235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561084c7f9e0 0x561084c964f0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.947+0000 7fe0beb85740 1 --2- 172.21.15.47:0/3263427235 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x561084cf8930 0x561084cfad50 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.947+0000 7fe0beb85740 1 --2- 172.21.15.47:0/3263427235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561084c7e810 0x561084c95fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.947+0000 7fe0beb85740 1 -- 172.21.15.47:0/3263427235 >> 172.21.15.47:0/3263427235 conn(0x561084c79c00 msgr2=0x561084c7ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.947+0000 7fe09aa00640 0 client.33626 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:49:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.948+0000 7fe0beb85740 1 -- 172.21.15.47:0/3263427235 shutdown_connections 2024-03-21T11:49:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.948+0000 7fe0beb85740 1 -- 172.21.15.47:0/3263427235 wait complete. 2024-03-21T11:49:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.948+0000 7fe0beb85740 20 client.33626 trim_cache size 0 max 16384 2024-03-21T11:49:06.953 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:49:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.949+0000 7fe0bbe00640 1 -- 172.21.15.47:0/2244407396 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe09c0a3070 msgr2=0x7fe09c0a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.949+0000 7fe0bbe00640 1 --2- 172.21.15.47:0/2244407396 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe09c0a3070 0x7fe09c0a5530 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe0b406b390 tx=0x7fe0ac00a4e0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.949+0000 7fe0bbe00640 1 -- 172.21.15.47:0/2244407396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0b41086a0 msgr2=0x7fe0b41a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.949+0000 7fe0bbe00640 1 --2- 172.21.15.47:0/2244407396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0b41086a0 0x7fe0b41a2560 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800b3f0 tx=0x7fe0a800b8b0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.949+0000 7fe0bbe00640 1 -- 172.21.15.47:0/2244407396 shutdown_connections 2024-03-21T11:49:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.949+0000 7fe0bbe00640 1 --2- 172.21.15.47:0/2244407396 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe09c0a3070 0x7fe09c0a5530 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.949+0000 7fe0bbe00640 1 --2- 172.21.15.47:0/2244407396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0b410ff20 0x7fe0b41a9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.949+0000 7fe0bbe00640 1 --2- 172.21.15.47:0/2244407396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0b4109000 0x7fe0b41a2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.949+0000 7fe0bbe00640 1 --2- 172.21.15.47:0/2244407396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0b41086a0 0x7fe0b41a2560 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.949+0000 7fe0bbe00640 1 -- 172.21.15.47:0/2244407396 >> 172.21.15.47:0/2244407396 conn(0x7fe0b406e500 msgr2=0x7fe0b4070170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.949+0000 7fe0bbe00640 1 -- 172.21.15.47:0/2244407396 shutdown_connections 2024-03-21T11:49:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:06.949+0000 7fe0bbe00640 1 -- 172.21.15.47:0/2244407396 wait complete. 2024-03-21T11:49:06.973 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:49:06.973 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:49:07.006 INFO:tasks.workunit.client.0.smithi047.stdout: 10 292450 21.38 MB/sec execute 590 sec latency 75.729 ms 2024-03-21T11:49:07.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.226+0000 7ffb38800640 1 -- 172.21.15.47:0/4022976691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb30104ad0 msgr2=0x7ffb30104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.226+0000 7ffb38800640 1 --2- 172.21.15.47:0/4022976691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb30104ad0 0x7ffb30104f50 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009860 tx=0x7ffb2002bb70 comp rx=0 tx=0).stop 2024-03-21T11:49:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.226+0000 7ffb38800640 1 -- 172.21.15.47:0/4022976691 shutdown_connections 2024-03-21T11:49:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.226+0000 7ffb38800640 1 --2- 172.21.15.47:0/4022976691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb30105490 0x7ffb3010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.226+0000 7ffb38800640 1 --2- 172.21.15.47:0/4022976691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb30104ad0 0x7ffb30104f50 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.226+0000 7ffb38800640 1 --2- 172.21.15.47:0/4022976691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb301038e0 0x7ffb30103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.226+0000 7ffb38800640 1 -- 172.21.15.47:0/4022976691 >> 172.21.15.47:0/4022976691 conn(0x7ffb300fee50 msgr2=0x7ffb30101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.226+0000 7ffb38800640 1 -- 172.21.15.47:0/4022976691 shutdown_connections 2024-03-21T11:49:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.226+0000 7ffb38800640 1 -- 172.21.15.47:0/4022976691 wait complete. 2024-03-21T11:49:07.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.228+0000 7ffb38800640 1 Processor -- start 2024-03-21T11:49:07.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.228+0000 7ffb38800640 1 -- start start 2024-03-21T11:49:07.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.228+0000 7ffb38800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb301038e0 0x7ffb30079450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.228+0000 7ffb38800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb30104ad0 0x7ffb30079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.228+0000 7ffb38800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb30105490 0x7ffb30075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.228+0000 7ffb38800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3010f560 con 0x7ffb30104ad0 2024-03-21T11:49:07.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.228+0000 7ffb38800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3010f3e0 con 0x7ffb30105490 2024-03-21T11:49:07.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.228+0000 7ffb38800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3010f6e0 con 0x7ffb301038e0 2024-03-21T11:49:07.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.228+0000 7ffb35e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb301038e0 0x7ffb30079450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.228+0000 7ffb35e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb301038e0 0x7ffb30079450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45706/0 (socket says 172.21.15.47:45706) 2024-03-21T11:49:07.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.229+0000 7ffb35e00640 1 -- 172.21.15.47:0/2820385068 learned_addr learned my addr 172.21.15.47:0/2820385068 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.228+0000 7ffb36800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb30105490 0x7ffb30075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.229+0000 7ffb35400640 1 --2- 172.21.15.47:0/2820385068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb30104ad0 0x7ffb30079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.229+0000 7ffb36800640 1 -- 172.21.15.47:0/2820385068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb301038e0 msgr2=0x7ffb30079450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.229+0000 7ffb36800640 1 --2- 172.21.15.47:0/2820385068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb301038e0 0x7ffb30079450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.229+0000 7ffb36800640 1 -- 172.21.15.47:0/2820385068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb30104ad0 msgr2=0x7ffb30079990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.229+0000 7ffb36800640 1 --2- 172.21.15.47:0/2820385068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb30104ad0 0x7ffb30079990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.229+0000 7ffb36800640 1 -- 172.21.15.47:0/2820385068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb30076880 con 0x7ffb30105490 2024-03-21T11:49:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.229+0000 7ffb36800640 1 --2- 172.21.15.47:0/2820385068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb30105490 0x7ffb30075fc0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00da10 tx=0x7ffb2c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.230+0000 7ffb2aa00640 1 -- 172.21.15.47:0/2820385068 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c00bd10 con 0x7ffb30105490 2024-03-21T11:49:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.230+0000 7ffb38800640 1 -- 172.21.15.47:0/2820385068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb301a6080 con 0x7ffb30105490 2024-03-21T11:49:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.230+0000 7ffb2aa00640 1 -- 172.21.15.47:0/2820385068 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb2c003e50 con 0x7ffb30105490 2024-03-21T11:49:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.230+0000 7ffb2aa00640 1 -- 172.21.15.47:0/2820385068 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0106c0 con 0x7ffb30105490 2024-03-21T11:49:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.230+0000 7ffb38800640 1 -- 172.21.15.47:0/2820385068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb301a6430 con 0x7ffb30105490 2024-03-21T11:49:07.233 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:07 smithi047 ceph-mon[31351]: pgmap v1174: 129 pgs: 129 active+clean; 740 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-03-21T11:49:07.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.235+0000 7ffb3b5a0740 1 Processor -- start 2024-03-21T11:49:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.236+0000 7ffb2aa00640 1 -- 172.21.15.47:0/2820385068 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ffb2c003970 con 0x7ffb30105490 2024-03-21T11:49:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.236+0000 7ffb3b5a0740 1 -- start start 2024-03-21T11:49:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.237+0000 7ffb3b5a0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d3810 0x5644c57d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.237+0000 7ffb3b5a0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d49e0 0x5644c57d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.237+0000 7ffb3b5a0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d53a0 0x5644c57dbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.237+0000 7ffb3b5a0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c57382c0 con 0x5644c57d53a0 2024-03-21T11:49:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.237+0000 7ffb3b5a0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c56e0c90 con 0x5644c57d49e0 2024-03-21T11:49:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.237+0000 7ffb3b5a0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c5734360 con 0x5644c57d3810 2024-03-21T11:49:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.237+0000 7ffb35e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d49e0 0x5644c57d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.237+0000 7ffb35e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d49e0 0x5644c57d4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54898/0 (socket says 172.21.15.47:54898) 2024-03-21T11:49:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.237+0000 7ffb35e00640 1 -- 172.21.15.47:0/2479079893 learned_addr learned my addr 172.21.15.47:0/2479079893 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.238+0000 7ffb35e00640 1 -- 172.21.15.47:0/2479079893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d3810 msgr2=0x5644c57d3c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:49:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.238+0000 7ffb35e00640 1 --2- 172.21.15.47:0/2479079893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d3810 0x5644c57d3c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.238+0000 7ffb35e00640 1 -- 172.21.15.47:0/2479079893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d53a0 msgr2=0x5644c57dbc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:49:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.238+0000 7ffb35e00640 1 --2- 172.21.15.47:0/2479079893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d53a0 0x5644c57dbc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.238+0000 7ffb35e00640 1 -- 172.21.15.47:0/2479079893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644c5705080 con 0x5644c57d49e0 2024-03-21T11:49:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.240+0000 7ffb2aa00640 1 --2- 172.21.15.47:0/2820385068 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb180a2f20 0x7ffb180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.240+0000 7ffb35e00640 1 --2- 172.21.15.47:0/2479079893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d49e0 0x5644c57d4e60 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009da0 tx=0x7ffb24011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=140118cdd802ebb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.241+0000 7ffb2aa00640 1 -- 172.21.15.47:0/2820385068 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffb2c0cfa80 con 0x7ffb30105490 2024-03-21T11:49:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.241+0000 7ffb35400640 1 --2- 172.21.15.47:0/2820385068 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb180a2f20 0x7ffb180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.242+0000 7ffb1f000640 1 -- 172.21.15.47:0/2479079893 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24004220 con 0x5644c57d49e0 2024-03-21T11:49:07.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.242+0000 7ffb1f000640 1 -- 172.21.15.47:0/2479079893 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb240043c0 con 0x5644c57d49e0 2024-03-21T11:49:07.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.242+0000 7ffb1f000640 1 -- 172.21.15.47:0/2479079893 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24019990 con 0x5644c57d49e0 2024-03-21T11:49:07.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.243+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/2479079893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d49e0 msgr2=0x5644c57d4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.243+0000 7ffb35400640 1 --2- 172.21.15.47:0/2820385068 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb180a2f20 0x7ffb180a53e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009830 tx=0x7ffb20034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.243+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/2479079893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d49e0 0x5644c57d4e60 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009da0 tx=0x7ffb24011810 comp rx=0 tx=0).stop 2024-03-21T11:49:07.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.244+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/2479079893 shutdown_connections 2024-03-21T11:49:07.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.244+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/2479079893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d53a0 0x5644c57dbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.244+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/2479079893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d49e0 0x5644c57d4e60 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.244+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/2479079893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d3810 0x5644c57d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.244+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/2479079893 >> 172.21.15.47:0/2479079893 conn(0x5644c57cec00 msgr2=0x5644c57cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.245+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/2479079893 shutdown_connections 2024-03-21T11:49:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.245+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/2479079893 wait complete. 2024-03-21T11:49:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.245+0000 7ffb3b5a0740 1 -- start start 2024-03-21T11:49:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.246+0000 7ffb3b5a0740 1 Processor -- start 2024-03-21T11:49:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.246+0000 7ffb3b5a0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d3810 0x5644c57ddab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.246+0000 7ffb3b5a0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d49e0 0x5644c57ddff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.246+0000 7ffb3b5a0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d53a0 0x5644c57e5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.246+0000 7ffb3b5a0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c57dc3b0 con 0x5644c57d3810 2024-03-21T11:49:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.246+0000 7ffb3b5a0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c56f40d0 con 0x5644c57d53a0 2024-03-21T11:49:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.246+0000 7ffb3b5a0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644c57dc530 con 0x5644c57d49e0 2024-03-21T11:49:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.247+0000 7ffb3b5a0740 10 client.0 _ll_register_callbacks cb 0x5644c56f8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.247+0000 7ffb36800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d53a0 0x5644c57e5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.247+0000 7ffb36800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d53a0 0x5644c57e5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54912/0 (socket says 172.21.15.47:54912) 2024-03-21T11:49:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.247+0000 7ffb36800640 1 -- 172.21.15.47:0/4216398750 learned_addr learned my addr 172.21.15.47:0/4216398750 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.247+0000 7ffb35e00640 1 --2- 172.21.15.47:0/4216398750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d3810 0x5644c57ddab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.248+0000 7ffb35400640 1 --2- 172.21.15.47:0/4216398750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d49e0 0x5644c57ddff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.248+0000 7ffb36800640 1 -- 172.21.15.47:0/4216398750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d49e0 msgr2=0x5644c57ddff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.248+0000 7ffb36800640 1 --2- 172.21.15.47:0/4216398750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d49e0 0x5644c57ddff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.248+0000 7ffb36800640 1 -- 172.21.15.47:0/4216398750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d3810 msgr2=0x5644c57ddab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.248+0000 7ffb36800640 1 --2- 172.21.15.47:0/4216398750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d3810 0x5644c57ddab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.248+0000 7ffb36800640 1 -- 172.21.15.47:0/4216398750 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644c544c3f0 con 0x5644c57d53a0 2024-03-21T11:49:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.248+0000 7ffb36800640 1 --2- 172.21.15.47:0/4216398750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d53a0 0x5644c57e5030 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c098280 tx=0x7ffb2c09b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29cb42ef980295e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.248+0000 7ffb13400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:49:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.248+0000 7ffb13400640 1 -- 172.21.15.47:0/4216398750 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c09be20 con 0x5644c57d53a0 2024-03-21T11:49:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.248+0000 7ffb13400640 1 -- 172.21.15.47:0/4216398750 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffb2c004250 con 0x5644c57d53a0 2024-03-21T11:49:07.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.248+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/4216398750 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb240043c0 con 0x5644c57d53a0 2024-03-21T11:49:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.249+0000 7ffb13400640 1 -- 172.21.15.47:0/4216398750 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb2c09ff70 con 0x5644c57d53a0 2024-03-21T11:49:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.249+0000 7ffb3b5a0740 10 client.33638 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.249+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/4216398750 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5644c55167d0 con 0x5644c57d53a0 2024-03-21T11:49:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.250+0000 7ffb13400640 1 -- 172.21.15.47:0/4216398750 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb240043c0 con 0x5644c57d53a0 2024-03-21T11:49:07.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.250+0000 7ffb13400640 1 -- 172.21.15.47:0/4216398750 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffb2c024060 con 0x5644c57d53a0 2024-03-21T11:49:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.251+0000 7ffb3b5a0740 10 client.33638 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.251+0000 7ffb3b5a0740 10 client.33638 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:49:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.251+0000 7ffb3b5a0740 20 client.33638 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.251+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/4216398750 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5644c57fa4c0 0x5644c57fc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.251+0000 7ffb3b5a0740 4 client.33638 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:07.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.251+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/4216398750 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644c5745800 con 0x5644c57fa4c0 2024-03-21T11:49:07.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.252+0000 7ffb35e00640 1 --2- 172.21.15.47:0/4216398750 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5644c57fa4c0 0x5644c57fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.254+0000 7ffb13400640 10 client.33638 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:49:07.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.254+0000 7ffb35e00640 1 --2- 172.21.15.47:0/4216398750 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5644c57fa4c0 0x5644c57fc910 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d579109da882d17c server_cookie=e55d6bbc1b2615c0 in_seq=0 out_seq=0 2024-03-21T11:49:07.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.255+0000 7ffb13400640 1 -- 172.21.15.47:0/4216398750 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5644c5745800 con 0x5644c57fa4c0 2024-03-21T11:49:07.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.255+0000 7ffb13400640 10 client.33638 handle_command_reply: tid=0 2024-03-21T11:49:07.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.255+0000 7ffb3b5a0740 1 client.33638 shutdown 2024-03-21T11:49:07.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.256+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/4216398750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d53a0 msgr2=0x5644c57e5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.256+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/4216398750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d53a0 0x5644c57e5030 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c098280 tx=0x7ffb2c09b840 comp rx=0 tx=0).stop 2024-03-21T11:49:07.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.256+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/4216398750 shutdown_connections 2024-03-21T11:49:07.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.256+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/4216398750 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5644c57fa4c0 0x5644c57fc910 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.256+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/4216398750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d53a0 0x5644c57e5030 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.256+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/4216398750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d49e0 0x5644c57ddff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.256+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/4216398750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d3810 0x5644c57ddab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.256+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/4216398750 >> 172.21.15.47:0/4216398750 conn(0x5644c57cec00 msgr2=0x5644c57cdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:07.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.256+0000 7ffb13400640 0 client.33638 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:49:07.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.257+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/4216398750 shutdown_connections 2024-03-21T11:49:07.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.257+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/4216398750 wait complete. 2024-03-21T11:49:07.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.257+0000 7ffb3b5a0740 20 client.33638 trim_cache size 0 max 16384 2024-03-21T11:49:07.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.280+0000 7ffb3b5a0740 1 Processor -- start 2024-03-21T11:49:07.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.280+0000 7ffb3b5a0740 1 -- start start 2024-03-21T11:49:07.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.280+0000 7ffb3b5a0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d3810 0x5644c57cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.280+0000 7ffb3b5a0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d49e0 0x5644c57e4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.280+0000 7ffb3b5a0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d53a0 0x5644c57cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.280+0000 7ffb3b5a0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb301a7e40 con 0x5644c57d49e0 2024-03-21T11:49:07.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.280+0000 7ffb3b5a0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb301a97c0 con 0x5644c57d3810 2024-03-21T11:49:07.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.280+0000 7ffb3b5a0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb301a84a0 con 0x5644c57d53a0 2024-03-21T11:49:07.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.281+0000 7ffb36800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d53a0 0x5644c57cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.281+0000 7ffb35e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d3810 0x5644c57cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.281+0000 7ffb36800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d53a0 0x5644c57cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45736/0 (socket says 172.21.15.47:45736) 2024-03-21T11:49:07.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.281+0000 7ffb35e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d3810 0x5644c57cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54918/0 (socket says 172.21.15.47:54918) 2024-03-21T11:49:07.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.281+0000 7ffb36800640 1 -- 172.21.15.47:0/4142971614 learned_addr learned my addr 172.21.15.47:0/4142971614 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:07.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.281+0000 7ffb35400640 1 --2- 172.21.15.47:0/4142971614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d49e0 0x5644c57e4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.281+0000 7ffb36800640 1 -- 172.21.15.47:0/4142971614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d3810 msgr2=0x5644c57cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:07.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.281+0000 7ffb36800640 1 --2- 172.21.15.47:0/4142971614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d3810 0x5644c57cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.281+0000 7ffb36800640 1 -- 172.21.15.47:0/4142971614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d49e0 msgr2=0x5644c57e4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:07.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.282+0000 7ffb36800640 1 --2- 172.21.15.47:0/4142971614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d49e0 0x5644c57e4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.282+0000 7ffb36800640 1 -- 172.21.15.47:0/4142971614 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644c57cf970 con 0x5644c57d53a0 2024-03-21T11:49:07.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.282+0000 7ffb35400640 1 --2- 172.21.15.47:0/4142971614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d49e0 0x5644c57e4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:49:07.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.283+0000 7ffb36800640 1 --2- 172.21.15.47:0/4142971614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d53a0 0x5644c57cdb60 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0cd240 tx=0x7ffb2c098150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6696cc7c523c4151 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:07.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.283+0000 7ffb1e600640 1 -- 172.21.15.47:0/4142971614 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c005e30 con 0x5644c57d53a0 2024-03-21T11:49:07.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.283+0000 7ffb1e600640 1 -- 172.21.15.47:0/4142971614 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb2c0a4830 con 0x5644c57d53a0 2024-03-21T11:49:07.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.283+0000 7ffb1e600640 1 -- 172.21.15.47:0/4142971614 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c098490 con 0x5644c57d53a0 2024-03-21T11:49:07.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.284+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/4142971614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d53a0 msgr2=0x5644c57cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.284+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/4142971614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d53a0 0x5644c57cdb60 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0cd240 tx=0x7ffb2c098150 comp rx=0 tx=0).stop 2024-03-21T11:49:07.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.285+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/4142971614 shutdown_connections 2024-03-21T11:49:07.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.285+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/4142971614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d53a0 0x5644c57cdb60 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.285+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/4142971614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d49e0 0x5644c57e4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.285+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/4142971614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d3810 0x5644c57cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.285+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/4142971614 >> 172.21.15.47:0/4142971614 conn(0x5644c57cec00 msgr2=0x5644c57ddab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:07.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.285+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/4142971614 shutdown_connections 2024-03-21T11:49:07.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.285+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/4142971614 wait complete. 2024-03-21T11:49:07.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.286+0000 7ffb3b5a0740 1 -- start start 2024-03-21T11:49:07.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.286+0000 7ffb3b5a0740 1 Processor -- start 2024-03-21T11:49:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.287+0000 7ffb3b5a0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d3810 0x5644c57e06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.287+0000 7ffb3b5a0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d49e0 0x5644c57e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.287+0000 7ffb3b5a0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d53a0 0x5644c57e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.287+0000 7ffb3b5a0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb301a9160 con 0x5644c57d3810 2024-03-21T11:49:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.287+0000 7ffb3b5a0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb301a8b00 con 0x5644c57d49e0 2024-03-21T11:49:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.287+0000 7ffb3b5a0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb301a1970 con 0x5644c57d53a0 2024-03-21T11:49:07.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.287+0000 7ffb3b5a0740 10 client.0 _ll_register_callbacks cb 0x7ffb2c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:07.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.287+0000 7ffb36800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d53a0 0x5644c57e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.288+0000 7ffb36800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d53a0 0x5644c57e3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45750/0 (socket says 172.21.15.47:45750) 2024-03-21T11:49:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.288+0000 7ffb36800640 1 -- 172.21.15.47:0/1022862221 learned_addr learned my addr 172.21.15.47:0/1022862221 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.288+0000 7ffb35e00640 1 --2- 172.21.15.47:0/1022862221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d3810 0x5644c57e06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.288+0000 7ffb35400640 1 --2- 172.21.15.47:0/1022862221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d49e0 0x5644c57e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.288+0000 7ffb35e00640 1 -- 172.21.15.47:0/1022862221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d53a0 msgr2=0x5644c57e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.288+0000 7ffb35e00640 1 --2- 172.21.15.47:0/1022862221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d53a0 0x5644c57e3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.288+0000 7ffb35e00640 1 -- 172.21.15.47:0/1022862221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d49e0 msgr2=0x5644c57e0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.288+0000 7ffb35e00640 1 --2- 172.21.15.47:0/1022862221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d49e0 0x5644c57e0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.288+0000 7ffb35e00640 1 -- 172.21.15.47:0/1022862221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644c57e41f0 con 0x5644c57d3810 2024-03-21T11:49:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.289+0000 7ffb35e00640 1 --2- 172.21.15.47:0/1022862221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d3810 0x5644c57e06f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ffb24019470 tx=0x7ffb24009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1764235702e44b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.289+0000 7ffb13400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:49:07.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.289+0000 7ffb13400640 1 -- 172.21.15.47:0/1022862221 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24005600 con 0x5644c57d3810 2024-03-21T11:49:07.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.289+0000 7ffb13400640 1 -- 172.21.15.47:0/1022862221 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ffb240035f0 con 0x5644c57d3810 2024-03-21T11:49:07.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.290+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/1022862221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb2c0a4830 con 0x5644c57d3810 2024-03-21T11:49:07.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.291+0000 7ffb13400640 1 -- 172.21.15.47:0/1022862221 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb24038050 con 0x5644c57d3810 2024-03-21T11:49:07.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.291+0000 7ffb3b5a0740 10 client.22704 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:07.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.291+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/1022862221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5644c5855db0 con 0x5644c57d3810 2024-03-21T11:49:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.293+0000 7ffb13400640 1 -- 172.21.15.47:0/1022862221 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb2c0a4830 con 0x5644c57d3810 2024-03-21T11:49:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.293+0000 7ffb13400640 1 -- 172.21.15.47:0/1022862221 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffb24019c00 con 0x5644c57d3810 2024-03-21T11:49:07.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.293+0000 7ffb3b5a0740 10 client.22704 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:07.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.293+0000 7ffb3b5a0740 10 client.22704 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:49:07.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.293+0000 7ffb3b5a0740 20 client.22704 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:07.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.293+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/1022862221 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5644c5856720 0x5644c5858b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.293+0000 7ffb3b5a0740 4 client.22704 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:49:07.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.293+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/1022862221 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5644c573fb50 con 0x5644c5856720 2024-03-21T11:49:07.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.294+0000 7ffb35400640 1 --2- 172.21.15.47:0/1022862221 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5644c5856720 0x5644c5858b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.295+0000 7ffb35400640 1 --2- 172.21.15.47:0/1022862221 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5644c5856720 0x5644c5858b40 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d098112a1cc0da98 server_cookie=b5380b1d0fe5d475 in_seq=0 out_seq=0 2024-03-21T11:49:07.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.295+0000 7ffb13400640 10 client.22704 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:49:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.297+0000 7ffb13400640 1 -- 172.21.15.47:0/1022862221 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5644c573fb50 con 0x5644c5856720 2024-03-21T11:49:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.297+0000 7ffb13400640 10 client.22704 handle_command_reply: tid=0 2024-03-21T11:49:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.297+0000 7ffb3b5a0740 1 client.22704 shutdown 2024-03-21T11:49:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.298+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/1022862221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d3810 msgr2=0x5644c57e06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.298+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/1022862221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d3810 0x5644c57e06f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ffb24019470 tx=0x7ffb24009900 comp rx=0 tx=0).stop 2024-03-21T11:49:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.298+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/1022862221 shutdown_connections 2024-03-21T11:49:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.298+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/1022862221 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x5644c5856720 0x5644c5858b40 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.298+0000 7ffb13400640 0 client.22704 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:49:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.298+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/1022862221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5644c57d53a0 0x5644c57e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.298+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/1022862221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5644c57d49e0 0x5644c57e0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.298+0000 7ffb3b5a0740 1 --2- 172.21.15.47:0/1022862221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5644c57d3810 0x5644c57e06f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.298+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/1022862221 >> 172.21.15.47:0/1022862221 conn(0x5644c57cec00 msgr2=0x5644c57d1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:07.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.298+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/1022862221 shutdown_connections 2024-03-21T11:49:07.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.298+0000 7ffb3b5a0740 1 -- 172.21.15.47:0/1022862221 wait complete. 2024-03-21T11:49:07.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.298+0000 7ffb3b5a0740 20 client.22704 trim_cache size 0 max 16384 2024-03-21T11:49:07.300 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:49:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.299+0000 7ffb38800640 1 -- 172.21.15.47:0/2820385068 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb180a2f20 msgr2=0x7ffb180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.299+0000 7ffb38800640 1 --2- 172.21.15.47:0/2820385068 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb180a2f20 0x7ffb180a53e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009830 tx=0x7ffb20034ab0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.299+0000 7ffb38800640 1 -- 172.21.15.47:0/2820385068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb30105490 msgr2=0x7ffb30075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.299+0000 7ffb38800640 1 --2- 172.21.15.47:0/2820385068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb30105490 0x7ffb30075fc0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00da10 tx=0x7ffb2c00de10 comp rx=0 tx=0).stop 2024-03-21T11:49:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.299+0000 7ffb38800640 1 -- 172.21.15.47:0/2820385068 shutdown_connections 2024-03-21T11:49:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.299+0000 7ffb38800640 1 --2- 172.21.15.47:0/2820385068 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ffb180a2f20 0x7ffb180a53e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.299+0000 7ffb38800640 1 --2- 172.21.15.47:0/2820385068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb30105490 0x7ffb30075fc0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.299+0000 7ffb38800640 1 --2- 172.21.15.47:0/2820385068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb30104ad0 0x7ffb30079990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.299+0000 7ffb38800640 1 --2- 172.21.15.47:0/2820385068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb301038e0 0x7ffb30079450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.299+0000 7ffb38800640 1 -- 172.21.15.47:0/2820385068 >> 172.21.15.47:0/2820385068 conn(0x7ffb300fee50 msgr2=0x7ffb30107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.300+0000 7ffb38800640 1 -- 172.21.15.47:0/2820385068 shutdown_connections 2024-03-21T11:49:07.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.300+0000 7ffb38800640 1 -- 172.21.15.47:0/2820385068 wait complete. 2024-03-21T11:49:07.321 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:49:07.321 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:49:07.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:07 smithi102 ceph-mon[36748]: pgmap v1174: 129 pgs: 129 active+clean; 740 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-03-21T11:49:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.572+0000 7ff8dac00640 1 -- 172.21.15.47:0/1838095856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d4104ad0 msgr2=0x7ff8d4104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.572+0000 7ff8dac00640 1 --2- 172.21.15.47:0/1838095856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d4104ad0 0x7ff8d4104f50 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009900 tx=0x7ff8c002bc40 comp rx=0 tx=0).stop 2024-03-21T11:49:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.572+0000 7ff8dac00640 1 -- 172.21.15.47:0/1838095856 shutdown_connections 2024-03-21T11:49:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.572+0000 7ff8dac00640 1 --2- 172.21.15.47:0/1838095856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8d4105490 0x7ff8d410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.572+0000 7ff8dac00640 1 --2- 172.21.15.47:0/1838095856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d4104ad0 0x7ff8d4104f50 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.572+0000 7ff8dac00640 1 --2- 172.21.15.47:0/1838095856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d41038e0 0x7ff8d4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.572+0000 7ff8dac00640 1 -- 172.21.15.47:0/1838095856 >> 172.21.15.47:0/1838095856 conn(0x7ff8d40fee50 msgr2=0x7ff8d4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.573+0000 7ff8dac00640 1 -- 172.21.15.47:0/1838095856 shutdown_connections 2024-03-21T11:49:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.573+0000 7ff8dac00640 1 -- 172.21.15.47:0/1838095856 wait complete. 2024-03-21T11:49:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.574+0000 7ff8dac00640 1 Processor -- start 2024-03-21T11:49:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.574+0000 7ff8dac00640 1 -- start start 2024-03-21T11:49:07.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.575+0000 7ff8dac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d41038e0 0x7ff8d4199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.575+0000 7ff8dac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d4104ad0 0x7ff8d419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.575+0000 7ff8dac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8d4105490 0x7ff8d41a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.575+0000 7ff8dac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d410f5b0 con 0x7ff8d4104ad0 2024-03-21T11:49:07.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.575+0000 7ff8dac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d410f430 con 0x7ff8d4105490 2024-03-21T11:49:07.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.575+0000 7ff8dac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d410f730 con 0x7ff8d41038e0 2024-03-21T11:49:07.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.575+0000 7ff8d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d41038e0 0x7ff8d4199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.575+0000 7ff8d8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8d4105490 0x7ff8d41a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.575+0000 7ff8d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d41038e0 0x7ff8d4199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45760/0 (socket says 172.21.15.47:45760) 2024-03-21T11:49:07.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.575+0000 7ff8d3e00640 1 -- 172.21.15.47:0/3742269600 learned_addr learned my addr 172.21.15.47:0/3742269600 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:07.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.576+0000 7ff8d3400640 1 --2- 172.21.15.47:0/3742269600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d4104ad0 0x7ff8d419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.576+0000 7ff8d3e00640 1 -- 172.21.15.47:0/3742269600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8d4105490 msgr2=0x7ff8d41a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.576+0000 7ff8d3e00640 1 --2- 172.21.15.47:0/3742269600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8d4105490 0x7ff8d41a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.576+0000 7ff8d3e00640 1 -- 172.21.15.47:0/3742269600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d4104ad0 msgr2=0x7ff8d419a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.576+0000 7ff8d3e00640 1 --2- 172.21.15.47:0/3742269600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d4104ad0 0x7ff8d419a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.576+0000 7ff8d3e00640 1 -- 172.21.15.47:0/3742269600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d41a17e0 con 0x7ff8d41038e0 2024-03-21T11:49:07.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.576+0000 7ff8d3400640 1 --2- 172.21.15.47:0/3742269600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d4104ad0 0x7ff8d419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:07.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.576+0000 7ff8d3e00640 1 --2- 172.21.15.47:0/3742269600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d41038e0 0x7ff8d4199bc0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7ff8c400e9c0 tx=0x7ff8c400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:07.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.577+0000 7ff8d0c00640 1 -- 172.21.15.47:0/3742269600 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c400cd80 con 0x7ff8d41038e0 2024-03-21T11:49:07.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.577+0000 7ff8dac00640 1 -- 172.21.15.47:0/3742269600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d41a1ad0 con 0x7ff8d41038e0 2024-03-21T11:49:07.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.577+0000 7ff8d0c00640 1 -- 172.21.15.47:0/3742269600 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8c40035f0 con 0x7ff8d41038e0 2024-03-21T11:49:07.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.577+0000 7ff8d0c00640 1 -- 172.21.15.47:0/3742269600 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4010690 con 0x7ff8d41038e0 2024-03-21T11:49:07.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.577+0000 7ff8dac00640 1 -- 172.21.15.47:0/3742269600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d41a2040 con 0x7ff8d41038e0 2024-03-21T11:49:07.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.580+0000 7ff8d0c00640 1 -- 172.21.15.47:0/3742269600 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff8c40037a0 con 0x7ff8d41038e0 2024-03-21T11:49:07.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.580+0000 7ff8dd9ac740 1 Processor -- start 2024-03-21T11:49:07.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.581+0000 7ff8dd9ac740 1 -- start start 2024-03-21T11:49:07.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.581+0000 7ff8dd9ac740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636ee9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.581+0000 7ff8dd9ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 0x55e636eeae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.581+0000 7ff8dd9ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636eeb3a0 0x55e636ef1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.581+0000 7ff8dd9ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e636e4e2c0 con 0x55e636eea9e0 2024-03-21T11:49:07.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.581+0000 7ff8dd9ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e636df6c90 con 0x55e636eeb3a0 2024-03-21T11:49:07.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.581+0000 7ff8dd9ac740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e636e4a360 con 0x55e636ee9810 2024-03-21T11:49:07.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.581+0000 7ff8d8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636ee9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.581+0000 7ff8d8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636ee9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:45770/0 (socket says 172.21.15.47:45770) 2024-03-21T11:49:07.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.581+0000 7ff8d8c00640 1 -- 172.21.15.47:0/3270710736 learned_addr learned my addr 172.21.15.47:0/3270710736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:07.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8d3400640 1 --2- 172.21.15.47:0/3270710736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636eeb3a0 0x55e636ef1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8d3e00640 1 --2- 172.21.15.47:0/3270710736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 0x55e636eeae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8d8c00640 1 -- 172.21.15.47:0/3270710736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636eeb3a0 msgr2=0x55e636ef1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:07.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8d8c00640 1 --2- 172.21.15.47:0/3270710736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636eeb3a0 0x55e636ef1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8d8c00640 1 -- 172.21.15.47:0/3270710736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 msgr2=0x55e636eeae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:07.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8d8c00640 1 --2- 172.21.15.47:0/3270710736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 0x55e636eeae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8d8c00640 1 -- 172.21.15.47:0/3270710736 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e636e1b080 con 0x55e636ee9810 2024-03-21T11:49:07.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8d3400640 1 --2- 172.21.15.47:0/3270710736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636eeb3a0 0x55e636ef1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:07.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8d8c00640 1 --2- 172.21.15.47:0/3270710736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636ee9c30 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009fd0 tx=0x7ff8cc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55a4e65f238e92e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:07.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8d0c00640 1 --2- 172.21.15.47:0/3742269600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8b80a2fa0 0x7ff8b80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8d0c00640 1 -- 172.21.15.47:0/3742269600 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff8c40cfbd0 con 0x7ff8d41038e0 2024-03-21T11:49:07.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8c9200640 1 -- 172.21.15.47:0/3270710736 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc004d20 con 0x55e636ee9810 2024-03-21T11:49:07.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8d3e00640 1 --2- 172.21.15.47:0/3742269600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8b80a2fa0 0x7ff8b80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8c9200640 1 -- 172.21.15.47:0/3270710736 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8cc004050 con 0x55e636ee9810 2024-03-21T11:49:07.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.582+0000 7ff8c9200640 1 -- 172.21.15.47:0/3270710736 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc005220 con 0x55e636ee9810 2024-03-21T11:49:07.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.583+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/3270710736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 msgr2=0x55e636ee9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.583+0000 7ff8d3e00640 1 --2- 172.21.15.47:0/3742269600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8b80a2fa0 0x7ff8b80a5460 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4098370 tx=0x7ff8c40097a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:07.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.583+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/3270710736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636ee9c30 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009fd0 tx=0x7ff8cc0118d0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.584+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/3270710736 shutdown_connections 2024-03-21T11:49:07.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.584+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/3270710736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636eeb3a0 0x55e636ef1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.584+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/3270710736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 0x55e636eeae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.584+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/3270710736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636ee9c30 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.584+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/3270710736 >> 172.21.15.47:0/3270710736 conn(0x55e636ee4c00 msgr2=0x55e636ee5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:07.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.584+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/3270710736 shutdown_connections 2024-03-21T11:49:07.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.584+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/3270710736 wait complete. 2024-03-21T11:49:07.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.584+0000 7ff8dd9ac740 1 -- start start 2024-03-21T11:49:07.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.584+0000 7ff8dd9ac740 1 Processor -- start 2024-03-21T11:49:07.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.585+0000 7ff8dd9ac740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636efc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.585+0000 7ff8dd9ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636eea9e0 0x55e636efc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.585+0000 7ff8dd9ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eeb3a0 0x55e636f037c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.585+0000 7ff8dd9ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e636ef23b0 con 0x55e636eeb3a0 2024-03-21T11:49:07.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.585+0000 7ff8dd9ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e636e0a0d0 con 0x55e636eea9e0 2024-03-21T11:49:07.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.585+0000 7ff8dd9ac740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e636ef2530 con 0x55e636ee9810 2024-03-21T11:49:07.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.585+0000 7ff8dd9ac740 10 client.0 _ll_register_callbacks cb 0x55e636e0ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:07.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.585+0000 7ff8d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636eea9e0 0x55e636efc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.585+0000 7ff8d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636eea9e0 0x55e636efc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54966/0 (socket says 172.21.15.47:54966) 2024-03-21T11:49:07.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.585+0000 7ff8d3400640 1 -- 172.21.15.47:0/2609661431 learned_addr learned my addr 172.21.15.47:0/2609661431 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:07.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.585+0000 7ff8d8c00640 1 --2- 172.21.15.47:0/2609661431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636efc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.585+0000 7ff8d8c00640 1 --2- 172.21.15.47:0/2609661431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eeb3a0 0x55e636f037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.586+0000 7ff8d8c00640 1 -- 172.21.15.47:0/2609661431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 msgr2=0x55e636efc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:07.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.586+0000 7ff8d8c00640 1 --2- 172.21.15.47:0/2609661431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636efc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.586+0000 7ff8d8c00640 1 -- 172.21.15.47:0/2609661431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636eea9e0 msgr2=0x55e636efc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:07.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.586+0000 7ff8d8c00640 1 --2- 172.21.15.47:0/2609661431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636eea9e0 0x55e636efc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.586+0000 7ff8d8c00640 1 -- 172.21.15.47:0/2609661431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e636b623f0 con 0x55e636eeb3a0 2024-03-21T11:49:07.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.586+0000 7ff8d8c00640 1 --2- 172.21.15.47:0/2609661431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eeb3a0 0x55e636f037c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009fa0 tx=0x7ff8cc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb49bd1ebf547b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:07.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.586+0000 7ff8bd600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:49:07.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.586+0000 7ff8bd600640 1 -- 172.21.15.47:0/2609661431 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc02d060 con 0x55e636eeb3a0 2024-03-21T11:49:07.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.587+0000 7ff8bd600640 1 -- 172.21.15.47:0/2609661431 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff8cc00f4e0 con 0x55e636eeb3a0 2024-03-21T11:49:07.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.587+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/2609661431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8cc004050 con 0x55e636eeb3a0 2024-03-21T11:49:07.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.588+0000 7ff8bd600640 1 -- 172.21.15.47:0/2609661431 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8cc00f6e0 con 0x55e636eeb3a0 2024-03-21T11:49:07.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.588+0000 7ff8dd9ac740 10 client.22716 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:07.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.588+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/2609661431 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e636c2c7d0 con 0x55e636eeb3a0 2024-03-21T11:49:07.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.588+0000 7ff8bd600640 1 -- 172.21.15.47:0/2609661431 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8cc004050 con 0x55e636eeb3a0 2024-03-21T11:49:07.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.588+0000 7ff8bd600640 1 -- 172.21.15.47:0/2609661431 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff8cc025040 con 0x55e636eeb3a0 2024-03-21T11:49:07.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.588+0000 7ff8dd9ac740 10 client.22716 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:07.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.588+0000 7ff8dd9ac740 10 client.22716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:49:07.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.588+0000 7ff8dd9ac740 20 client.22716 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:07.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.588+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/2609661431 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e636f07a80 0x55e636f09e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.588+0000 7ff8dd9ac740 4 client.22716 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:07.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.588+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/2609661431 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e636e5b800 con 0x55e636f07a80 2024-03-21T11:49:07.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.591+0000 7ff8d3400640 1 --2- 172.21.15.47:0/2609661431 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e636f07a80 0x55e636f09e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.592+0000 7ff8d3400640 1 --2- 172.21.15.47:0/2609661431 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e636f07a80 0x55e636f09e70 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53275a0cac75c901 server_cookie=87e7c8b6e4b800d9 in_seq=0 out_seq=0 2024-03-21T11:49:07.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.592+0000 7ff8bd600640 10 client.22716 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:07.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.597+0000 7ff8bd600640 1 -- 172.21.15.47:0/2609661431 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e636e5b800 con 0x55e636f07a80 2024-03-21T11:49:07.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.597+0000 7ff8bd600640 10 client.22716 handle_command_reply: tid=0 2024-03-21T11:49:07.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.597+0000 7ff8dd9ac740 1 client.22716 shutdown 2024-03-21T11:49:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.597+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/2609661431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eeb3a0 msgr2=0x55e636f037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.597+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/2609661431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eeb3a0 0x55e636f037c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009fa0 tx=0x7ff8cc005fd0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.598+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/2609661431 shutdown_connections 2024-03-21T11:49:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.598+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/2609661431 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e636f07a80 0x55e636f09e70 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.598+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/2609661431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eeb3a0 0x55e636f037c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.598+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/2609661431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636eea9e0 0x55e636efc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.598+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/2609661431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636efc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.598+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/2609661431 >> 172.21.15.47:0/2609661431 conn(0x55e636ee4c00 msgr2=0x55e636ee3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.598+0000 7ff8bd600640 0 client.22716 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:07.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.599+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/2609661431 shutdown_connections 2024-03-21T11:49:07.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.599+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/2609661431 wait complete. 2024-03-21T11:49:07.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.600+0000 7ff8dd9ac740 20 client.22716 trim_cache size 0 max 16384 2024-03-21T11:49:07.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.618+0000 7ff8dd9ac740 1 Processor -- start 2024-03-21T11:49:07.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.618+0000 7ff8dd9ac740 1 -- start start 2024-03-21T11:49:07.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.618+0000 7ff8dd9ac740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee4c00 0x55e636f0e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.618+0000 7ff8dd9ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636ee9810 0x55e636f03780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.618+0000 7ff8dd9ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 0x55e636f0ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.618+0000 7ff8dd9ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d41a7ef0 con 0x55e636eea9e0 2024-03-21T11:49:07.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.618+0000 7ff8dd9ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d41a9870 con 0x55e636ee9810 2024-03-21T11:49:07.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.618+0000 7ff8dd9ac740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d41a8550 con 0x55e636ee4c00 2024-03-21T11:49:07.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.619+0000 7ff8d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636ee9810 0x55e636f03780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.619+0000 7ff8d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636ee9810 0x55e636f03780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54982/0 (socket says 172.21.15.47:54982) 2024-03-21T11:49:07.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.619+0000 7ff8d3400640 1 -- 172.21.15.47:0/3891738519 learned_addr learned my addr 172.21.15.47:0/3891738519 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:07.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.619+0000 7ff8d8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee4c00 0x55e636f0e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.619+0000 7ff8d8c00640 1 --2- 172.21.15.47:0/3891738519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 0x55e636f0ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.619+0000 7ff8d3400640 1 -- 172.21.15.47:0/3891738519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee4c00 msgr2=0x55e636f0e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:07.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.619+0000 7ff8d3400640 1 --2- 172.21.15.47:0/3891738519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee4c00 0x55e636f0e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.619+0000 7ff8d3400640 1 -- 172.21.15.47:0/3891738519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 msgr2=0x55e636f0ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:07.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.619+0000 7ff8d3400640 1 --2- 172.21.15.47:0/3891738519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 0x55e636f0ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.619+0000 7ff8d3400640 1 -- 172.21.15.47:0/3891738519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e636eeea90 con 0x55e636ee9810 2024-03-21T11:49:07.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.620+0000 7ff8d3400640 1 --2- 172.21.15.47:0/3891738519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636ee9810 0x55e636f03780 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7ff8c002c180 tx=0x7ff8c002e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdfac3fabcc8e79e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:07.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.620+0000 7ff8bfe00640 1 -- 172.21.15.47:0/3891738519 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c004a070 con 0x55e636ee9810 2024-03-21T11:49:07.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.620+0000 7ff8bfe00640 1 -- 172.21.15.47:0/3891738519 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e636e5b800 con 0x55e636ee9810 2024-03-21T11:49:07.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.620+0000 7ff8bfe00640 1 -- 172.21.15.47:0/3891738519 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0005460 con 0x55e636ee9810 2024-03-21T11:49:07.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.621+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/3891738519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636ee9810 msgr2=0x55e636f03780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.621+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/3891738519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636ee9810 0x55e636f03780 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7ff8c002c180 tx=0x7ff8c002e040 comp rx=0 tx=0).stop 2024-03-21T11:49:07.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.621+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/3891738519 shutdown_connections 2024-03-21T11:49:07.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.621+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/3891738519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 0x55e636f0ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.621+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/3891738519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636ee9810 0x55e636f03780 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.621+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/3891738519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee4c00 0x55e636f0e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.621+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/3891738519 >> 172.21.15.47:0/3891738519 conn(0x55e636ef2c40 msgr2=0x55e636ee86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:07.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.621+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/3891738519 shutdown_connections 2024-03-21T11:49:07.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.621+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/3891738519 wait complete. 2024-03-21T11:49:07.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.622+0000 7ff8dd9ac740 1 -- start start 2024-03-21T11:49:07.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.622+0000 7ff8dd9ac740 1 Processor -- start 2024-03-21T11:49:07.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.622+0000 7ff8dd9ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636ee4c00 0x55e636eff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.622+0000 7ff8dd9ac740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636efff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.622+0000 7ff8dd9ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 0x55e636f02ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.622+0000 7ff8dd9ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d41a9210 con 0x55e636eea9e0 2024-03-21T11:49:07.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.622+0000 7ff8dd9ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d41a8bb0 con 0x55e636ee4c00 2024-03-21T11:49:07.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.622+0000 7ff8dd9ac740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d41a60b0 con 0x55e636ee9810 2024-03-21T11:49:07.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.623+0000 7ff8dd9ac740 10 client.0 _ll_register_callbacks cb 0x7ff8cc009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:07.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.623+0000 7ff8d8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636ee4c00 0x55e636eff9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.623+0000 7ff8d8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636ee4c00 0x55e636eff9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54998/0 (socket says 172.21.15.47:54998) 2024-03-21T11:49:07.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.623+0000 7ff8d8c00640 1 -- 172.21.15.47:0/983829457 learned_addr learned my addr 172.21.15.47:0/983829457 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:07.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.623+0000 7ff8d3400640 1 --2- 172.21.15.47:0/983829457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636efff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.623+0000 7ff8d8c00640 1 --2- 172.21.15.47:0/983829457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 0x55e636f02ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.623+0000 7ff8d8c00640 1 -- 172.21.15.47:0/983829457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 msgr2=0x55e636efff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:07.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.623+0000 7ff8d8c00640 1 --2- 172.21.15.47:0/983829457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636efff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.623+0000 7ff8d8c00640 1 -- 172.21.15.47:0/983829457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 msgr2=0x55e636f02ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:07.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.623+0000 7ff8d8c00640 1 --2- 172.21.15.47:0/983829457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 0x55e636f02ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.623+0000 7ff8d8c00640 1 -- 172.21.15.47:0/983829457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e636f034e0 con 0x55e636ee4c00 2024-03-21T11:49:07.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.623+0000 7ff8d3400640 1 --2- 172.21.15.47:0/983829457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636efff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:07.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.624+0000 7ff8d8c00640 1 --2- 172.21.15.47:0/983829457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636ee4c00 0x55e636eff9e0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009fd0 tx=0x7ff8cc004ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ed960ae27771d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:07.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.624+0000 7ff8bd600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:49:07.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.624+0000 7ff8bd600640 1 -- 172.21.15.47:0/983829457 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc02d040 con 0x55e636ee4c00 2024-03-21T11:49:07.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.624+0000 7ff8bd600640 1 -- 172.21.15.47:0/983829457 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7ff8cc028bd0 con 0x55e636ee4c00 2024-03-21T11:49:07.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.627+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/983829457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e636e5b800 con 0x55e636ee4c00 2024-03-21T11:49:07.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.627+0000 7ff8bd600640 1 -- 172.21.15.47:0/983829457 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8cc004190 con 0x55e636ee4c00 2024-03-21T11:49:07.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.627+0000 7ff8dd9ac740 10 client.33662 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:07.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.627+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/983829457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e636f63d70 con 0x55e636ee4c00 2024-03-21T11:49:07.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.628+0000 7ff8bd600640 1 -- 172.21.15.47:0/983829457 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e636e5b800 con 0x55e636ee4c00 2024-03-21T11:49:07.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.628+0000 7ff8bd600640 1 -- 172.21.15.47:0/983829457 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff8cc00f980 con 0x55e636ee4c00 2024-03-21T11:49:07.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.628+0000 7ff8dd9ac740 10 client.33662 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:07.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.628+0000 7ff8dd9ac740 10 client.33662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:49:07.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.628+0000 7ff8dd9ac740 20 client.33662 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:07.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.628+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/983829457 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e636eeb3a0 0x55e636f66600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:07.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.629+0000 7ff8dd9ac740 4 client.33662 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:49:07.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.629+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/983829457 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e636e55b50 con 0x55e636eeb3a0 2024-03-21T11:49:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.629+0000 7ff8d3400640 1 --2- 172.21.15.47:0/983829457 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e636eeb3a0 0x55e636f66600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.630+0000 7ff8d3400640 1 --2- 172.21.15.47:0/983829457 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e636eeb3a0 0x55e636f66600 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c634c8c84e27212 server_cookie=724bac67431a89e7 in_seq=0 out_seq=0 2024-03-21T11:49:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.630+0000 7ff8bd600640 10 client.33662 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:07.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.631+0000 7ff8bd600640 1 -- 172.21.15.47:0/983829457 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e636e55b50 con 0x55e636eeb3a0 2024-03-21T11:49:07.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.631+0000 7ff8bd600640 10 client.33662 handle_command_reply: tid=0 2024-03-21T11:49:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.631+0000 7ff8dd9ac740 1 client.33662 shutdown 2024-03-21T11:49:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.631+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/983829457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636ee4c00 msgr2=0x55e636eff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.631+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/983829457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636ee4c00 0x55e636eff9e0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009fd0 tx=0x7ff8cc004ac0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.631+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/983829457 shutdown_connections 2024-03-21T11:49:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.631+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/983829457 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55e636eeb3a0 0x55e636f66600 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.631+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/983829457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e636eea9e0 0x55e636f02ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.631+0000 7ff8bd600640 0 client.33662 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.631+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/983829457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e636ee9810 0x55e636efff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.631+0000 7ff8dd9ac740 1 --2- 172.21.15.47:0/983829457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e636ee4c00 0x55e636eff9e0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.631+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/983829457 >> 172.21.15.47:0/983829457 conn(0x55e636ef2c40 msgr2=0x55e636eedc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.631+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/983829457 shutdown_connections 2024-03-21T11:49:07.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.631+0000 7ff8dd9ac740 1 -- 172.21.15.47:0/983829457 wait complete. 2024-03-21T11:49:07.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.632+0000 7ff8dd9ac740 20 client.33662 trim_cache size 0 max 16384 2024-03-21T11:49:07.634 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:49:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.632+0000 7ff8dac00640 1 -- 172.21.15.47:0/3742269600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8b80a2fa0 msgr2=0x7ff8b80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.632+0000 7ff8dac00640 1 --2- 172.21.15.47:0/3742269600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8b80a2fa0 0x7ff8b80a5460 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4098370 tx=0x7ff8c40097a0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.632+0000 7ff8dac00640 1 -- 172.21.15.47:0/3742269600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d41038e0 msgr2=0x7ff8d4199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.632+0000 7ff8dac00640 1 --2- 172.21.15.47:0/3742269600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d41038e0 0x7ff8d4199bc0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7ff8c400e9c0 tx=0x7ff8c400ee80 comp rx=0 tx=0).stop 2024-03-21T11:49:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.633+0000 7ff8dac00640 1 -- 172.21.15.47:0/3742269600 shutdown_connections 2024-03-21T11:49:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.633+0000 7ff8dac00640 1 --2- 172.21.15.47:0/3742269600 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7ff8b80a2fa0 0x7ff8b80a5460 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.633+0000 7ff8dac00640 1 --2- 172.21.15.47:0/3742269600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8d4105490 0x7ff8d41a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.633+0000 7ff8dac00640 1 --2- 172.21.15.47:0/3742269600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8d4104ad0 0x7ff8d419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.633+0000 7ff8dac00640 1 --2- 172.21.15.47:0/3742269600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8d41038e0 0x7ff8d4199bc0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.633+0000 7ff8dac00640 1 -- 172.21.15.47:0/3742269600 >> 172.21.15.47:0/3742269600 conn(0x7ff8d40fee50 msgr2=0x7ff8d4107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.633+0000 7ff8dac00640 1 -- 172.21.15.47:0/3742269600 shutdown_connections 2024-03-21T11:49:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:07.633+0000 7ff8dac00640 1 -- 172.21.15.47:0/3742269600 wait complete. 2024-03-21T11:49:07.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:07 smithi100 ceph-mon[36245]: pgmap v1174: 129 pgs: 129 active+clean; 740 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-03-21T11:49:08.007 INFO:tasks.workunit.client.0.smithi047.stdout: 10 292899 21.39 MB/sec execute 591 sec latency 81.085 ms 2024-03-21T11:49:08.658 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T11:49:08.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:49:08.659 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T11:49:08.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.947+0000 7f0fd1c00640 1 -- 172.21.15.47:0/2896943941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fcc10b1b0 msgr2=0x7f0fcc10d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:08.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.947+0000 7f0fd1c00640 1 --2- 172.21.15.47:0/2896943941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fcc10b1b0 0x7f0fcc10d5a0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009900 tx=0x7f0fbc02bc40 comp rx=0 tx=0).stop 2024-03-21T11:49:08.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.947+0000 7f0fd1c00640 1 -- 172.21.15.47:0/2896943941 shutdown_connections 2024-03-21T11:49:08.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.947+0000 7f0fd1c00640 1 --2- 172.21.15.47:0/2896943941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fcc10dc30 0x7f0fcc1100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.947+0000 7f0fd1c00640 1 --2- 172.21.15.47:0/2896943941 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fcc10b1b0 0x7f0fcc10d5a0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.947+0000 7f0fd1c00640 1 --2- 172.21.15.47:0/2896943941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fcc108880 0x7f0fcc10ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.947+0000 7f0fd1c00640 1 -- 172.21.15.47:0/2896943941 >> 172.21.15.47:0/2896943941 conn(0x7f0fcc06e580 msgr2=0x7f0fcc0709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:08.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.948+0000 7f0fd1c00640 1 -- 172.21.15.47:0/2896943941 shutdown_connections 2024-03-21T11:49:08.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.948+0000 7f0fd1c00640 1 -- 172.21.15.47:0/2896943941 wait complete. 2024-03-21T11:49:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.949+0000 7f0fd1c00640 1 Processor -- start 2024-03-21T11:49:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.949+0000 7f0fd1c00640 1 -- start start 2024-03-21T11:49:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.950+0000 7f0fd1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fcc108880 0x7f0fcc1a22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:08.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.950+0000 7f0fd1c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fcc10b1b0 0x7f0fcc1a2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:08.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.950+0000 7f0fd1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fcc10dc30 0x7f0fcc1a9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:08.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.950+0000 7f0fd1c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc113820 con 0x7f0fcc10dc30 2024-03-21T11:49:08.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.950+0000 7f0fd1c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1136a0 con 0x7f0fcc108880 2024-03-21T11:49:08.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.950+0000 7f0fd1c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1139a0 con 0x7f0fcc10b1b0 2024-03-21T11:49:08.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.950+0000 7f0fcbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fcc108880 0x7f0fcc1a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:08.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.950+0000 7f0fcbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fcc108880 0x7f0fcc1a22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51596/0 (socket says 172.21.15.47:51596) 2024-03-21T11:49:08.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.950+0000 7f0fcbe00640 1 -- 172.21.15.47:0/830265285 learned_addr learned my addr 172.21.15.47:0/830265285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.950+0000 7f0fcb400640 1 --2- 172.21.15.47:0/830265285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fcc10b1b0 0x7f0fcc1a2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.951+0000 7f0fd1200640 1 --2- 172.21.15.47:0/830265285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fcc10dc30 0x7f0fcc1a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.951+0000 7f0fcbe00640 1 -- 172.21.15.47:0/830265285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fcc10b1b0 msgr2=0x7f0fcc1a2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.951+0000 7f0fcbe00640 1 --2- 172.21.15.47:0/830265285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fcc10b1b0 0x7f0fcc1a2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.951+0000 7f0fcbe00640 1 -- 172.21.15.47:0/830265285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fcc10dc30 msgr2=0x7f0fcc1a9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:08.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.951+0000 7f0fcbe00640 1 --2- 172.21.15.47:0/830265285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fcc10dc30 0x7f0fcc1a9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.951+0000 7f0fcbe00640 1 -- 172.21.15.47:0/830265285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc1a9ef0 con 0x7f0fcc108880 2024-03-21T11:49:08.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.952+0000 7f0fcbe00640 1 --2- 172.21.15.47:0/830265285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fcc108880 0x7f0fcc1a22d0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400cce0 tx=0x7f0fc400a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:08.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.952+0000 7f0fc8c00640 1 -- 172.21.15.47:0/830265285 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc400f650 con 0x7f0fcc108880 2024-03-21T11:49:08.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.952+0000 7f0fc8c00640 1 -- 172.21.15.47:0/830265285 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fc400fcd0 con 0x7f0fcc108880 2024-03-21T11:49:08.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.952+0000 7f0fc8c00640 1 -- 172.21.15.47:0/830265285 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4018c70 con 0x7f0fcc108880 2024-03-21T11:49:08.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.952+0000 7f0fd1c00640 1 -- 172.21.15.47:0/830265285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc1aa1e0 con 0x7f0fcc108880 2024-03-21T11:49:08.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.952+0000 7f0fd1c00640 1 -- 172.21.15.47:0/830265285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc1aa6b0 con 0x7f0fcc108880 2024-03-21T11:49:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.957+0000 7f0fc8c00640 1 -- 172.21.15.47:0/830265285 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0fc400f7f0 con 0x7f0fcc108880 2024-03-21T11:49:08.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.957+0000 7f0fd5e01740 1 Processor -- start 2024-03-21T11:49:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.957+0000 7f0fd5e01740 1 -- start start 2024-03-21T11:49:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.958+0000 7f0fd5e01740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e09ac0 0x561bd1e09ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:08.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.958+0000 7f0fd5e01740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e0efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.958+0000 7f0fd5e01740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e0f620 0x561bd1e11ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.958+0000 7f0fd5e01740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd1d66eb0 con 0x561bd1e0f620 2024-03-21T11:49:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.958+0000 7f0fd5e01740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd1d0fa60 con 0x561bd1e0a4b0 2024-03-21T11:49:08.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.958+0000 7f0fd5e01740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd1d62f50 con 0x561bd1e09ac0 2024-03-21T11:49:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.958+0000 7f0fd1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e09ac0 0x561bd1e09ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.958+0000 7f0fcb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e0f620 0x561bd1e11ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.958+0000 7f0fcb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e0f620 0x561bd1e11ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52032/0 (socket says 172.21.15.47:52032) 2024-03-21T11:49:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.958+0000 7f0fcb400640 1 -- 172.21.15.47:0/3934145229 learned_addr learned my addr 172.21.15.47:0/3934145229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.958+0000 7f0fd1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e09ac0 0x561bd1e09ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48732/0 (socket says 172.21.15.47:48732) 2024-03-21T11:49:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.958+0000 7f0fcbe00640 1 --2- 172.21.15.47:0/3934145229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e0efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.959+0000 7f0fcb400640 1 -- 172.21.15.47:0/3934145229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e09ac0 msgr2=0x561bd1e09ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.959+0000 7f0fcb400640 1 --2- 172.21.15.47:0/3934145229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e09ac0 0x561bd1e09ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.959+0000 7f0fcb400640 1 -- 172.21.15.47:0/3934145229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 msgr2=0x561bd1e0efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.959+0000 7f0fcb400640 1 --2- 172.21.15.47:0/3934145229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e0efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.959+0000 7f0fcb400640 1 -- 172.21.15.47:0/3934145229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bd1b437e0 con 0x561bd1e0f620 2024-03-21T11:49:08.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.959+0000 7f0fc8c00640 1 --2- 172.21.15.47:0/830265285 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0fb00a2ff0 0x7f0fb00a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:08.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.960+0000 7f0fcb400640 1 --2- 172.21.15.47:0/3934145229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e0f620 0x561bd1e11ab0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc02efd0 tx=0x7f0fbc02c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1bf2c141510739 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:08.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.960+0000 7f0fc8c00640 1 -- 172.21.15.47:0/830265285 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0fc40cfb10 con 0x7f0fcc108880 2024-03-21T11:49:08.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.960+0000 7f0fd1200640 1 --2- 172.21.15.47:0/830265285 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0fb00a2ff0 0x7f0fb00a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:08.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.960+0000 7f0fc1600640 1 -- 172.21.15.47:0/3934145229 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc02ccd0 con 0x561bd1e0f620 2024-03-21T11:49:08.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.960+0000 7f0fc1600640 1 -- 172.21.15.47:0/3934145229 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fbc005040 con 0x561bd1e0f620 2024-03-21T11:49:08.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.960+0000 7f0fc1600640 1 -- 172.21.15.47:0/3934145229 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc004a60 con 0x561bd1e0f620 2024-03-21T11:49:08.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.961+0000 7f0fd5e01740 1 -- 172.21.15.47:0/3934145229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e0f620 msgr2=0x561bd1e11ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:08.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.961+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/3934145229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e0f620 0x561bd1e11ab0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc02efd0 tx=0x7f0fbc02c690 comp rx=0 tx=0).stop 2024-03-21T11:49:08.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.961+0000 7f0fd1200640 1 --2- 172.21.15.47:0/830265285 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0fb00a2ff0 0x7f0fb00a54b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x561bd1e0ad10 tx=0x7f0fb800c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:08.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.962+0000 7f0fd5e01740 1 -- 172.21.15.47:0/3934145229 shutdown_connections 2024-03-21T11:49:08.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.962+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/3934145229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e0f620 0x561bd1e11ab0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.962+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/3934145229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e0efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.962+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/3934145229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e09ac0 0x561bd1e09ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.962+0000 7f0fd5e01740 1 -- 172.21.15.47:0/3934145229 >> 172.21.15.47:0/3934145229 conn(0x561bd1e05030 msgr2=0x561bd1e05430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:08.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.962+0000 7f0fd5e01740 1 -- 172.21.15.47:0/3934145229 shutdown_connections 2024-03-21T11:49:08.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.962+0000 7f0fd5e01740 1 -- 172.21.15.47:0/3934145229 wait complete. 2024-03-21T11:49:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.962+0000 7f0fd5e01740 1 -- start start 2024-03-21T11:49:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.962+0000 7f0fd5e01740 1 Processor -- start 2024-03-21T11:49:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.963+0000 7f0fd5e01740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e09ac0 0x561bd1e13c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.963+0000 7f0fd5e01740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e0a4b0 0x561bd1e141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.963+0000 7f0fd5e01740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0f620 0x561bd1e1b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:08.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.963+0000 7f0fd5e01740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd1e12570 con 0x561bd1e09ac0 2024-03-21T11:49:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.963+0000 7f0fd5e01740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd1d22eb0 con 0x561bd1e0f620 2024-03-21T11:49:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.963+0000 7f0fd5e01740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd1e126f0 con 0x561bd1e0a4b0 2024-03-21T11:49:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.963+0000 7f0fcb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e0a4b0 0x561bd1e141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.963+0000 7f0fd1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0f620 0x561bd1e1b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.963+0000 7f0fd5e01740 10 client.0 _ll_register_callbacks cb 0x561bd1d17150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.963+0000 7f0fcb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e0a4b0 0x561bd1e141b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48748/0 (socket says 172.21.15.47:48748) 2024-03-21T11:49:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.963+0000 7f0fcb400640 1 -- 172.21.15.47:0/19196865 learned_addr learned my addr 172.21.15.47:0/19196865 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.963+0000 7f0fcbe00640 1 --2- 172.21.15.47:0/19196865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e09ac0 0x561bd1e13c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.964+0000 7f0fcb400640 1 -- 172.21.15.47:0/19196865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0f620 msgr2=0x561bd1e1b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.964+0000 7f0fcb400640 1 --2- 172.21.15.47:0/19196865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0f620 0x561bd1e1b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.964+0000 7f0fcb400640 1 -- 172.21.15.47:0/19196865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e09ac0 msgr2=0x561bd1e13c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.964+0000 7f0fcb400640 1 --2- 172.21.15.47:0/19196865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e09ac0 0x561bd1e13c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.964+0000 7f0fcb400640 1 -- 172.21.15.47:0/19196865 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bd1ba3f70 con 0x561bd1e0a4b0 2024-03-21T11:49:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.964+0000 7f0fcbe00640 1 --2- 172.21.15.47:0/19196865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e09ac0 0x561bd1e13c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.964+0000 7f0fcb400640 1 --2- 172.21.15.47:0/19196865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e0a4b0 0x561bd1e141b0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc02c7c0 tx=0x7f0fbc0051f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bee0da323ca9bb15 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.964+0000 7f0fb6000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:49:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.965+0000 7f0fb6000640 1 -- 172.21.15.47:0/19196865 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc004be0 con 0x561bd1e0a4b0 2024-03-21T11:49:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.965+0000 7f0fb6000640 1 -- 172.21.15.47:0/19196865 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0fbc004d80 con 0x561bd1e0a4b0 2024-03-21T11:49:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.965+0000 7f0fd5e01740 1 -- 172.21.15.47:0/19196865 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fbc005040 con 0x561bd1e0a4b0 2024-03-21T11:49:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.965+0000 7f0fb6000640 1 -- 172.21.15.47:0/19196865 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fbc046480 con 0x561bd1e0a4b0 2024-03-21T11:49:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.965+0000 7f0fd5e01740 10 client.33589 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.965+0000 7f0fd5e01740 1 -- 172.21.15.47:0/19196865 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561bd1a3b170 con 0x561bd1e0a4b0 2024-03-21T11:49:08.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.965+0000 7f0fb6000640 1 -- 172.21.15.47:0/19196865 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fbc005040 con 0x561bd1e0a4b0 2024-03-21T11:49:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.966+0000 7f0fb6000640 1 -- 172.21.15.47:0/19196865 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0fbc0345c0 con 0x561bd1e0a4b0 2024-03-21T11:49:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.966+0000 7f0fd5e01740 10 client.33589 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.966+0000 7f0fd5e01740 10 client.33589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:49:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.966+0000 7f0fd5e01740 20 client.33589 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.966+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/19196865 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561bd1e1f6e0 0x561bd1e21ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.966+0000 7f0fd5e01740 4 client.33589 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.966+0000 7f0fd5e01740 1 -- 172.21.15.47:0/19196865 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bd1d743f0 con 0x561bd1e1f6e0 2024-03-21T11:49:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.966+0000 7f0fcbe00640 1 --2- 172.21.15.47:0/19196865 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561bd1e1f6e0 0x561bd1e21ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.967+0000 7f0fcbe00640 1 --2- 172.21.15.47:0/19196865 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561bd1e1f6e0 0x561bd1e21ad0 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1d86d90a8a360e2 server_cookie=501f21af973de403 in_seq=0 out_seq=0 2024-03-21T11:49:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.967+0000 7f0fb6000640 10 client.33589 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.970+0000 7f0fb6000640 1 -- 172.21.15.47:0/19196865 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561bd1d743f0 con 0x561bd1e1f6e0 2024-03-21T11:49:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.970+0000 7f0fb6000640 10 client.33589 handle_command_reply: tid=0 2024-03-21T11:49:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.970+0000 7f0fd5e01740 1 client.33589 shutdown 2024-03-21T11:49:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.970+0000 7f0fd5e01740 1 -- 172.21.15.47:0/19196865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e0a4b0 msgr2=0x561bd1e141b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.971+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/19196865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e0a4b0 0x561bd1e141b0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc02c7c0 tx=0x7f0fbc0051f0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.971+0000 7f0fd5e01740 1 -- 172.21.15.47:0/19196865 shutdown_connections 2024-03-21T11:49:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.971+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/19196865 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561bd1e1f6e0 0x561bd1e21ad0 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.971+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/19196865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0f620 0x561bd1e1b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.971+0000 7f0fb6000640 0 client.33589 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.971+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/19196865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e0a4b0 0x561bd1e141b0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.971+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/19196865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e09ac0 0x561bd1e13c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.971+0000 7f0fd5e01740 1 -- 172.21.15.47:0/19196865 >> 172.21.15.47:0/19196865 conn(0x561bd1e05030 msgr2=0x561bd1dfba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.971+0000 7f0fd5e01740 1 -- 172.21.15.47:0/19196865 shutdown_connections 2024-03-21T11:49:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.971+0000 7f0fd5e01740 1 -- 172.21.15.47:0/19196865 wait complete. 2024-03-21T11:49:08.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.971+0000 7f0fd5e01740 20 client.33589 trim_cache size 0 max 16384 2024-03-21T11:49:09.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.999+0000 7f0fd5e01740 1 Processor -- start 2024-03-21T11:49:09.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.999+0000 7f0fd5e01740 1 -- start start 2024-03-21T11:49:09.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.999+0000 7f0fd5e01740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e09ac0 0x561bd1e0af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.999+0000 7f0fd5e01740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e1b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.999+0000 7f0fd5e01740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e0f620 0x561bd1e0b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.999+0000 7f0fd5e01740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1b0420 con 0x561bd1e09ac0 2024-03-21T11:49:09.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.999+0000 7f0fd5e01740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1b1da0 con 0x561bd1e0a4b0 2024-03-21T11:49:09.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:08.999+0000 7f0fd5e01740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1b0a80 con 0x561bd1e0f620 2024-03-21T11:49:09.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.000+0000 7f0fcb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e1b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.000+0000 7f0fcb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e1b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51616/0 (socket says 172.21.15.47:51616) 2024-03-21T11:49:09.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.000+0000 7f0fcb400640 1 -- 172.21.15.47:0/1782041261 learned_addr learned my addr 172.21.15.47:0/1782041261 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:09.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.000+0000 7f0fcbe00640 1 --2- 172.21.15.47:0/1782041261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e09ac0 0x561bd1e0af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.000+0000 7f0fd1200640 1 --2- 172.21.15.47:0/1782041261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e0f620 0x561bd1e0b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.000+0000 7f0fcb400640 1 -- 172.21.15.47:0/1782041261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e0f620 msgr2=0x561bd1e0b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.000+0000 7f0fcb400640 1 --2- 172.21.15.47:0/1782041261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e0f620 0x561bd1e0b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.000+0000 7f0fcb400640 1 -- 172.21.15.47:0/1782041261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e09ac0 msgr2=0x561bd1e0af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.000+0000 7f0fcb400640 1 --2- 172.21.15.47:0/1782041261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e09ac0 0x561bd1e0af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.000+0000 7f0fcb400640 1 -- 172.21.15.47:0/1782041261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bd1e14520 con 0x561bd1e0a4b0 2024-03-21T11:49:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.000+0000 7f0fd1200640 1 --2- 172.21.15.47:0/1782041261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e0f620 0x561bd1e0b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:49:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.001+0000 7f0fcb400640 1 --2- 172.21.15.47:0/1782041261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e1b1b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc02c250 tx=0x7f0fbc02c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=568f4c18d95e299d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.001+0000 7f0fc0c00640 1 -- 172.21.15.47:0/1782041261 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc034640 con 0x561bd1e0a4b0 2024-03-21T11:49:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.001+0000 7f0fc0c00640 1 -- 172.21.15.47:0/1782041261 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fbc05d070 con 0x561bd1e0a4b0 2024-03-21T11:49:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.001+0000 7f0fc0c00640 1 -- 172.21.15.47:0/1782041261 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc046620 con 0x561bd1e0a4b0 2024-03-21T11:49:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.002+0000 7f0fd5e01740 1 -- 172.21.15.47:0/1782041261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 msgr2=0x561bd1e1b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.002+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/1782041261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e1b1b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc02c250 tx=0x7f0fbc02c8b0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.002+0000 7f0fd5e01740 1 -- 172.21.15.47:0/1782041261 shutdown_connections 2024-03-21T11:49:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.002+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/1782041261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e0f620 0x561bd1e0b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.002+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/1782041261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e1b1b0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.002+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/1782041261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e09ac0 0x561bd1e0af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.002+0000 7f0fd5e01740 1 -- 172.21.15.47:0/1782041261 >> 172.21.15.47:0/1782041261 conn(0x561bd1e05030 msgr2=0x561bd1e08b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.002+0000 7f0fd5e01740 1 -- 172.21.15.47:0/1782041261 shutdown_connections 2024-03-21T11:49:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.002+0000 7f0fd5e01740 1 -- 172.21.15.47:0/1782041261 wait complete. 2024-03-21T11:49:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.003+0000 7f0fd5e01740 1 -- start start 2024-03-21T11:49:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.003+0000 7f0fd5e01740 1 Processor -- start 2024-03-21T11:49:09.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.003+0000 7f0fd5e01740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e09ac0 0x561bd1e1aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.003+0000 7f0fd5e01740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e1aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.004+0000 7f0fd5e01740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e0f620 0x561bd1e19180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.004+0000 7f0fd5e01740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1b1740 con 0x561bd1e0f620 2024-03-21T11:49:09.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.004+0000 7f0fd5e01740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1b10e0 con 0x561bd1e0a4b0 2024-03-21T11:49:09.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.004+0000 7f0fd5e01740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1ae630 con 0x561bd1e09ac0 2024-03-21T11:49:09.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.004+0000 7f0fd5e01740 10 client.0 _ll_register_callbacks cb 0x7f0fc40d34b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:09.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.004+0000 7f0fcbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e09ac0 0x561bd1e1aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.004+0000 7f0fcbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e09ac0 0x561bd1e1aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48772/0 (socket says 172.21.15.47:48772) 2024-03-21T11:49:09.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.004+0000 7f0fcbe00640 1 -- 172.21.15.47:0/1345817438 learned_addr learned my addr 172.21.15.47:0/1345817438 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:09.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.004+0000 7f0fcb400640 1 --2- 172.21.15.47:0/1345817438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e1aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.004+0000 7f0fd1200640 1 --2- 172.21.15.47:0/1345817438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e0f620 0x561bd1e19180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.005+0000 7f0fcbe00640 1 -- 172.21.15.47:0/1345817438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 msgr2=0x561bd1e1aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:09.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.005+0000 7f0fcbe00640 1 --2- 172.21.15.47:0/1345817438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e1aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.005+0000 7f0fcbe00640 1 -- 172.21.15.47:0/1345817438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e0f620 msgr2=0x561bd1e19180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:09.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.005+0000 7f0fcbe00640 1 --2- 172.21.15.47:0/1345817438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e0f620 0x561bd1e19180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.005+0000 7f0fcbe00640 1 -- 172.21.15.47:0/1345817438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bd1e198f0 con 0x561bd1e09ac0 2024-03-21T11:49:09.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.005+0000 7f0fcb400640 1 --2- 172.21.15.47:0/1345817438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e1aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:09.009 INFO:tasks.workunit.client.0.smithi047.stdout: 10 293388 21.40 MB/sec execute 592 sec latency 75.845 ms 2024-03-21T11:49:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.008+0000 7f0fcbe00640 1 --2- 172.21.15.47:0/1345817438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e09ac0 0x561bd1e1aab0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f0fc409bbb0 tx=0x7f0fc40d35e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74d1ef18b20aa84d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.008+0000 7f0fb6000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:49:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.008+0000 7f0fb6000640 1 -- 172.21.15.47:0/1345817438 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40b4070 con 0x561bd1e09ac0 2024-03-21T11:49:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.008+0000 7f0fb6000640 1 -- 172.21.15.47:0/1345817438 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0fc409a1b0 con 0x561bd1e09ac0 2024-03-21T11:49:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.008+0000 7f0fd5e01740 1 -- 172.21.15.47:0/1345817438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fbc05d070 con 0x561bd1e09ac0 2024-03-21T11:49:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.008+0000 7f0fb6000640 1 -- 172.21.15.47:0/1345817438 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fc409bc80 con 0x561bd1e09ac0 2024-03-21T11:49:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.008+0000 7f0fd5e01740 10 client.33592 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.008+0000 7f0fd5e01740 1 -- 172.21.15.47:0/1345817438 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561bd1e7b830 con 0x561bd1e09ac0 2024-03-21T11:49:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.009+0000 7f0fb6000640 1 -- 172.21.15.47:0/1345817438 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fbc05d070 con 0x561bd1e09ac0 2024-03-21T11:49:09.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.009+0000 7f0fb6000640 1 -- 172.21.15.47:0/1345817438 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0fc4098520 con 0x561bd1e09ac0 2024-03-21T11:49:09.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.009+0000 7f0fd5e01740 10 client.33592 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:09.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.009+0000 7f0fd5e01740 10 client.33592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:49:09.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.009+0000 7f0fd5e01740 20 client.33592 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:09.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.009+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/1345817438 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561bd1e7c0f0 0x561bd1e7e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.009+0000 7f0fd5e01740 4 client.33592 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T11:49:09.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.009+0000 7f0fd5e01740 1 -- 172.21.15.47:0/1345817438 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561bd1d6e740 con 0x561bd1e7c0f0 2024-03-21T11:49:09.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.011+0000 7f0fcb400640 1 --2- 172.21.15.47:0/1345817438 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561bd1e7c0f0 0x561bd1e7e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.014+0000 7f0fcb400640 1 --2- 172.21.15.47:0/1345817438 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561bd1e7c0f0 0x561bd1e7e510 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76586e65760f9b7a server_cookie=3773a36d340f0255 in_seq=0 out_seq=0 2024-03-21T11:49:09.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.014+0000 7f0fb6000640 10 client.33592 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:09.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.022+0000 7f0fb6000640 1 -- 172.21.15.47:0/1345817438 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561bd1d6e740 con 0x561bd1e7c0f0 2024-03-21T11:49:09.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.022+0000 7f0fb6000640 10 client.33592 handle_command_reply: tid=0 2024-03-21T11:49:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.022+0000 7f0fd5e01740 1 client.33592 shutdown 2024-03-21T11:49:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.022+0000 7f0fd5e01740 1 -- 172.21.15.47:0/1345817438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e09ac0 msgr2=0x561bd1e1aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.022+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/1345817438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e09ac0 0x561bd1e1aab0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f0fc409bbb0 tx=0x7f0fc40d35e0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.022+0000 7f0fd5e01740 1 -- 172.21.15.47:0/1345817438 shutdown_connections 2024-03-21T11:49:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.022+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/1345817438 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x561bd1e7c0f0 0x561bd1e7e510 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.022+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/1345817438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561bd1e0f620 0x561bd1e19180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.022+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/1345817438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561bd1e0a4b0 0x561bd1e1aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.022+0000 7f0fd5e01740 1 --2- 172.21.15.47:0/1345817438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bd1e09ac0 0x561bd1e1aab0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.022+0000 7f0fd5e01740 1 -- 172.21.15.47:0/1345817438 >> 172.21.15.47:0/1345817438 conn(0x561bd1e05030 msgr2=0x561bd1dfb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:09.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.022+0000 7f0fb6000640 0 client.33592 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:09.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.023+0000 7f0fd5e01740 1 -- 172.21.15.47:0/1345817438 shutdown_connections 2024-03-21T11:49:09.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.023+0000 7f0fd5e01740 1 -- 172.21.15.47:0/1345817438 wait complete. 2024-03-21T11:49:09.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.023+0000 7f0fd5e01740 20 client.33592 trim_cache size 0 max 16384 2024-03-21T11:49:09.026 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:49:09.026 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T11:49:09.026 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "bba34861-fd0b-4a13-897f-d8ffaaef8f0c", 2024-03-21T11:49:09.026 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T11:49:09.026 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:49:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.024+0000 7f0fd1c00640 1 -- 172.21.15.47:0/830265285 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0fb00a2ff0 msgr2=0x7f0fb00a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.024+0000 7f0fd1c00640 1 --2- 172.21.15.47:0/830265285 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0fb00a2ff0 0x7f0fb00a54b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x561bd1e0ad10 tx=0x7f0fb800c040 comp rx=0 tx=0).stop 2024-03-21T11:49:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.024+0000 7f0fd1c00640 1 -- 172.21.15.47:0/830265285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fcc108880 msgr2=0x7f0fcc1a22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.024+0000 7f0fd1c00640 1 --2- 172.21.15.47:0/830265285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fcc108880 0x7f0fcc1a22d0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400cce0 tx=0x7f0fc400a620 comp rx=0 tx=0).stop 2024-03-21T11:49:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.025+0000 7f0fd1c00640 1 -- 172.21.15.47:0/830265285 shutdown_connections 2024-03-21T11:49:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.025+0000 7f0fd1c00640 1 --2- 172.21.15.47:0/830265285 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0fb00a2ff0 0x7f0fb00a54b0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.025+0000 7f0fd1c00640 1 --2- 172.21.15.47:0/830265285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fcc10dc30 0x7f0fcc1a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.025+0000 7f0fd1c00640 1 --2- 172.21.15.47:0/830265285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fcc10b1b0 0x7f0fcc1a2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.025+0000 7f0fd1c00640 1 --2- 172.21.15.47:0/830265285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fcc108880 0x7f0fcc1a22d0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.025+0000 7f0fd1c00640 1 -- 172.21.15.47:0/830265285 >> 172.21.15.47:0/830265285 conn(0x7f0fcc06e580 msgr2=0x7f0fcc108c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:09.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.025+0000 7f0fd1c00640 1 -- 172.21.15.47:0/830265285 shutdown_connections 2024-03-21T11:49:09.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.025+0000 7f0fd1c00640 1 -- 172.21.15.47:0/830265285 wait complete. 2024-03-21T11:49:09.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:49:09.054 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:49:09.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:09 smithi047 ceph-mon[31351]: pgmap v1175: 129 pgs: 129 active+clean; 747 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.8 MiB/s wr, 641 op/s 2024-03-21T11:49:09.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.366+0000 7fb015200640 1 -- 172.21.15.47:0/3187481291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb010108eb0 msgr2=0x7fb010109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:09.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.366+0000 7fb015200640 1 --2- 172.21.15.47:0/3187481291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb010108eb0 0x7fb010109330 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fb000009900 tx=0x7fb00002bc30 comp rx=0 tx=0).stop 2024-03-21T11:49:09.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.367+0000 7fb015200640 1 -- 172.21.15.47:0/3187481291 shutdown_connections 2024-03-21T11:49:09.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.368+0000 7fb015200640 1 --2- 172.21.15.47:0/3187481291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb010109870 0x7fb010110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.368+0000 7fb015200640 1 --2- 172.21.15.47:0/3187481291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb010108eb0 0x7fb010109330 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.368+0000 7fb015200640 1 --2- 172.21.15.47:0/3187481291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb010107cc0 0x7fb0101080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.368+0000 7fb015200640 1 -- 172.21.15.47:0/3187481291 >> 172.21.15.47:0/3187481291 conn(0x7fb010076c60 msgr2=0x7fb0100790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:09.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.368+0000 7fb015200640 1 -- 172.21.15.47:0/3187481291 shutdown_connections 2024-03-21T11:49:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.369+0000 7fb015200640 1 -- 172.21.15.47:0/3187481291 wait complete. 2024-03-21T11:49:09.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.371+0000 7fb015200640 1 Processor -- start 2024-03-21T11:49:09.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.371+0000 7fb015200640 1 -- start start 2024-03-21T11:49:09.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.372+0000 7fb015200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb010107cc0 0x7fb0100819a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.372+0000 7fb015200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb010108eb0 0x7fb010081ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.372+0000 7fb015200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb010109870 0x7fb01007ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.372+0000 7fb015200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01007a940 con 0x7fb010108eb0 2024-03-21T11:49:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.372+0000 7fb015200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01007a7c0 con 0x7fb010107cc0 2024-03-21T11:49:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.372+0000 7fb015200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01007aac0 con 0x7fb010109870 2024-03-21T11:49:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.373+0000 7fb00e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb010109870 0x7fb01007ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.373+0000 7fb00e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb010109870 0x7fb01007ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48782/0 (socket says 172.21.15.47:48782) 2024-03-21T11:49:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.373+0000 7fb00e800640 1 -- 172.21.15.47:0/3457189900 learned_addr learned my addr 172.21.15.47:0/3457189900 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.373+0000 7fb00e800640 1 -- 172.21.15.47:0/3457189900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb010107cc0 msgr2=0x7fb0100819a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.373+0000 7fb00e800640 1 --2- 172.21.15.47:0/3457189900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb010107cc0 0x7fb0100819a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.373+0000 7fb00e800640 1 -- 172.21.15.47:0/3457189900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb010108eb0 msgr2=0x7fb010081ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.373+0000 7fb00e800640 1 --2- 172.21.15.47:0/3457189900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb010108eb0 0x7fb010081ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.373+0000 7fb00e800640 1 -- 172.21.15.47:0/3457189900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0100806e0 con 0x7fb010109870 2024-03-21T11:49:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.374+0000 7fb00e800640 1 --2- 172.21.15.47:0/3457189900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb010109870 0x7fb01007ffe0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fb00800b970 tx=0x7fb00800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.374+0000 7fb006a00640 1 -- 172.21.15.47:0/3457189900 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008019070 con 0x7fb010109870 2024-03-21T11:49:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.374+0000 7fb015200640 1 -- 172.21.15.47:0/3457189900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0100809d0 con 0x7fb010109870 2024-03-21T11:49:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.375+0000 7fb015200640 1 -- 172.21.15.47:0/3457189900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0100710f0 con 0x7fb010109870 2024-03-21T11:49:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.375+0000 7fb006a00640 1 -- 172.21.15.47:0/3457189900 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb00800cb40 con 0x7fb010109870 2024-03-21T11:49:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.375+0000 7fb006a00640 1 -- 172.21.15.47:0/3457189900 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008023060 con 0x7fb010109870 2024-03-21T11:49:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.379+0000 7fb017e32740 1 Processor -- start 2024-03-21T11:49:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.382+0000 7fb017e32740 1 -- start start 2024-03-21T11:49:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.383+0000 7fb017e32740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f216de80 0x5561f216e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.383+0000 7fb017e32740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 0x5561f2173370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.383+0000 7fb017e32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f21739e0 0x5561f2175e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.383+0000 7fb017e32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f20d3250 con 0x5561f21739e0 2024-03-21T11:49:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.383+0000 7fb017e32740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f207ba30 con 0x5561f216de80 2024-03-21T11:49:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.383+0000 7fb017e32740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f20cf2f0 con 0x5561f216e870 2024-03-21T11:49:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.383+0000 7fb006a00640 1 -- 172.21.15.47:0/3457189900 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb00801c070 con 0x7fb010109870 2024-03-21T11:49:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.383+0000 7fb00de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 0x5561f2173370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.383+0000 7fb00de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 0x5561f2173370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48790/0 (socket says 172.21.15.47:48790) 2024-03-21T11:49:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.383+0000 7fb00de00640 1 -- 172.21.15.47:0/431596727 learned_addr learned my addr 172.21.15.47:0/431596727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.383+0000 7fb00d400640 1 --2- 172.21.15.47:0/431596727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f216de80 0x5561f216e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.384+0000 7fb00d400640 1 --2- 172.21.15.47:0/431596727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f21739e0 0x5561f2175e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.384+0000 7fb00d400640 1 -- 172.21.15.47:0/431596727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 msgr2=0x5561f2173370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.384+0000 7fb00d400640 1 --2- 172.21.15.47:0/431596727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 0x5561f2173370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.384+0000 7fb00d400640 1 -- 172.21.15.47:0/431596727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f21739e0 msgr2=0x5561f2175e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.384+0000 7fb00d400640 1 --2- 172.21.15.47:0/431596727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f21739e0 0x5561f2175e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.384+0000 7fb00d400640 1 -- 172.21.15.47:0/431596727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561f2066690 con 0x5561f216de80 2024-03-21T11:49:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.384+0000 7fb00d400640 1 --2- 172.21.15.47:0/431596727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f216de80 0x5561f216e2a0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb000009850 tx=0x7fb000005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=310b069e5193baec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.385+0000 7fafff400640 1 -- 172.21.15.47:0/431596727 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00002ca40 con 0x5561f216de80 2024-03-21T11:49:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.385+0000 7fafff400640 1 -- 172.21.15.47:0/431596727 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb00002cbe0 con 0x5561f216de80 2024-03-21T11:49:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.385+0000 7fafff400640 1 -- 172.21.15.47:0/431596727 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0000046c0 con 0x5561f216de80 2024-03-21T11:49:09.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.386+0000 7fb006a00640 1 --2- 172.21.15.47:0/3457189900 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faff40a3070 0x7faff40a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.387+0000 7fb00de00640 1 --2- 172.21.15.47:0/3457189900 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faff40a3070 0x7faff40a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.387+0000 7fb006a00640 1 -- 172.21.15.47:0/3457189900 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb0080cee10 con 0x7fb010109870 2024-03-21T11:49:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.389+0000 7fb00de00640 1 --2- 172.21.15.47:0/3457189900 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faff40a3070 0x7faff40a5530 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7faff8005fd0 tx=0x7faff8005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.389+0000 7fb017e32740 1 -- 172.21.15.47:0/431596727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f216de80 msgr2=0x5561f216e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.389+0000 7fb017e32740 1 --2- 172.21.15.47:0/431596727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f216de80 0x5561f216e2a0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb000009850 tx=0x7fb000005a20 comp rx=0 tx=0).stop 2024-03-21T11:49:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.389+0000 7fb017e32740 1 -- 172.21.15.47:0/431596727 shutdown_connections 2024-03-21T11:49:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.389+0000 7fb017e32740 1 --2- 172.21.15.47:0/431596727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f21739e0 0x5561f2175e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.389+0000 7fb017e32740 1 --2- 172.21.15.47:0/431596727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 0x5561f2173370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.389+0000 7fb017e32740 1 --2- 172.21.15.47:0/431596727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f216de80 0x5561f216e2a0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.389+0000 7fb017e32740 1 -- 172.21.15.47:0/431596727 >> 172.21.15.47:0/431596727 conn(0x5561f2136200 msgr2=0x5561f21697f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:09.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.390+0000 7fb017e32740 1 -- 172.21.15.47:0/431596727 shutdown_connections 2024-03-21T11:49:09.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.390+0000 7fb017e32740 1 -- 172.21.15.47:0/431596727 wait complete. 2024-03-21T11:49:09.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.391+0000 7fb017e32740 1 -- start start 2024-03-21T11:49:09.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.391+0000 7fb017e32740 1 Processor -- start 2024-03-21T11:49:09.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.392+0000 7fb017e32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216de80 0x5561f2177e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.392+0000 7fb017e32740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 0x5561f2178390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.392+0000 7fb017e32740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f21739e0 0x5561f217f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.392+0000 7fb017e32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f21768d0 con 0x5561f216de80 2024-03-21T11:49:09.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.392+0000 7fb017e32740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f208ee80 con 0x5561f21739e0 2024-03-21T11:49:09.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.392+0000 7fb017e32740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f2176a50 con 0x5561f216e870 2024-03-21T11:49:09.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.392+0000 7fb017e32740 10 client.0 _ll_register_callbacks cb 0x5561f20a1130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:09.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.393+0000 7fb00d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 0x5561f2178390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.393+0000 7fb00d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 0x5561f2178390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48806/0 (socket says 172.21.15.47:48806) 2024-03-21T11:49:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.393+0000 7fb00d400640 1 -- 172.21.15.47:0/397564055 learned_addr learned my addr 172.21.15.47:0/397564055 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.393+0000 7fb00de00640 1 --2- 172.21.15.47:0/397564055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216de80 0x5561f2177e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.393+0000 7fb00e800640 1 --2- 172.21.15.47:0/397564055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f21739e0 0x5561f217f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.393+0000 7fb00e800640 1 -- 172.21.15.47:0/397564055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 msgr2=0x5561f2178390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.393+0000 7fb00e800640 1 --2- 172.21.15.47:0/397564055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 0x5561f2178390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.393+0000 7fb00e800640 1 -- 172.21.15.47:0/397564055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216de80 msgr2=0x5561f2177e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.393+0000 7fb00e800640 1 --2- 172.21.15.47:0/397564055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216de80 0x5561f2177e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.394+0000 7fb00e800640 1 -- 172.21.15.47:0/397564055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561f1eb17d0 con 0x5561f21739e0 2024-03-21T11:49:09.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.394+0000 7fb00de00640 1 --2- 172.21.15.47:0/397564055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216de80 0x5561f2177e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:09.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.395+0000 7fb00e800640 1 --2- 172.21.15.47:0/397564055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f21739e0 0x5561f217f3d0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fb0080975b0 tx=0x7fb00809b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20f911d0fc20d26 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:09.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.395+0000 7faff3e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:49:09.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.395+0000 7faff3e00640 1 -- 172.21.15.47:0/397564055 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00809b150 con 0x5561f21739e0 2024-03-21T11:49:09.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.395+0000 7faff3e00640 1 -- 172.21.15.47:0/397564055 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb0080a36d0 con 0x5561f21739e0 2024-03-21T11:49:09.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.395+0000 7fb017e32740 1 -- 172.21.15.47:0/397564055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb00002cbe0 con 0x5561f21739e0 2024-03-21T11:49:09.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.396+0000 7faff3e00640 1 -- 172.21.15.47:0/397564055 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0080b4240 con 0x5561f21739e0 2024-03-21T11:49:09.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.396+0000 7fb017e32740 10 client.33686 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:09.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.396+0000 7fb017e32740 1 -- 172.21.15.47:0/397564055 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5561f1de73f0 con 0x5561f21739e0 2024-03-21T11:49:09.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.397+0000 7faff3e00640 1 -- 172.21.15.47:0/397564055 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb00002cbe0 con 0x5561f21739e0 2024-03-21T11:49:09.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.397+0000 7faff3e00640 1 -- 172.21.15.47:0/397564055 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb00809a5a0 con 0x5561f21739e0 2024-03-21T11:49:09.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.397+0000 7fb017e32740 10 client.33686 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:09.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.397+0000 7fb017e32740 10 client.33686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:49:09.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.397+0000 7fb017e32740 20 client.33686 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:09.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.397+0000 7fb017e32740 1 --2- 172.21.15.47:0/397564055 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5561f2183bd0 0x5561f2186050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.397+0000 7fb017e32740 4 client.33686 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:09.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.397+0000 7fb017e32740 1 -- 172.21.15.47:0/397564055 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561f20e0790 con 0x5561f2183bd0 2024-03-21T11:49:09.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.399+0000 7fb00de00640 1 --2- 172.21.15.47:0/397564055 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5561f2183bd0 0x5561f2186050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.410+0000 7fb00de00640 1 --2- 172.21.15.47:0/397564055 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5561f2183bd0 0x5561f2186050 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d17ce10d76eaac server_cookie=38fd8449b3fad926 in_seq=0 out_seq=0 2024-03-21T11:49:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.410+0000 7faff3e00640 10 client.33686 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.413+0000 7faff3e00640 1 -- 172.21.15.47:0/397564055 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5561f20e0790 con 0x5561f2183bd0 2024-03-21T11:49:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.413+0000 7faff3e00640 10 client.33686 handle_command_reply: tid=0 2024-03-21T11:49:09.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.413+0000 7fb017e32740 1 client.33686 shutdown 2024-03-21T11:49:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.414+0000 7fb017e32740 1 -- 172.21.15.47:0/397564055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f21739e0 msgr2=0x5561f217f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.414+0000 7fb017e32740 1 --2- 172.21.15.47:0/397564055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f21739e0 0x5561f217f3d0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fb0080975b0 tx=0x7fb00809b850 comp rx=0 tx=0).stop 2024-03-21T11:49:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.414+0000 7fb017e32740 1 -- 172.21.15.47:0/397564055 shutdown_connections 2024-03-21T11:49:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.415+0000 7fb017e32740 1 --2- 172.21.15.47:0/397564055 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5561f2183bd0 0x5561f2186050 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.415+0000 7fb017e32740 1 --2- 172.21.15.47:0/397564055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f21739e0 0x5561f217f3d0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.415+0000 7fb017e32740 1 --2- 172.21.15.47:0/397564055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 0x5561f2178390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.415+0000 7fb017e32740 1 --2- 172.21.15.47:0/397564055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216de80 0x5561f2177e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.415+0000 7faff3e00640 0 client.33686 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.415+0000 7fb017e32740 1 -- 172.21.15.47:0/397564055 >> 172.21.15.47:0/397564055 conn(0x5561f2136200 msgr2=0x5561f2168a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:09.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.415+0000 7fb017e32740 1 -- 172.21.15.47:0/397564055 shutdown_connections 2024-03-21T11:49:09.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.416+0000 7fb017e32740 1 -- 172.21.15.47:0/397564055 wait complete. 2024-03-21T11:49:09.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.416+0000 7fb017e32740 20 client.33686 trim_cache size 0 max 16384 2024-03-21T11:49:09.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.451+0000 7fb017e32740 1 Processor -- start 2024-03-21T11:49:09.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.451+0000 7fb017e32740 1 -- start start 2024-03-21T11:49:09.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.452+0000 7fb017e32740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f216de80 0x5561f2172ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.452+0000 7fb017e32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216e870 0x5561f217f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.452+0000 7fb017e32740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f21739e0 0x5561f216ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.452+0000 7fb017e32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec00f950 con 0x5561f216e870 2024-03-21T11:49:09.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.452+0000 7fb017e32740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec0112d0 con 0x5561f216de80 2024-03-21T11:49:09.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.452+0000 7fb017e32740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec00ffb0 con 0x5561f21739e0 2024-03-21T11:49:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.452+0000 7fb00e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f21739e0 0x5561f216ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.452+0000 7fb00e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f21739e0 0x5561f216ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48812/0 (socket says 172.21.15.47:48812) 2024-03-21T11:49:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.452+0000 7fb00e800640 1 -- 172.21.15.47:0/1034710383 learned_addr learned my addr 172.21.15.47:0/1034710383 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.452+0000 7fb00de00640 1 --2- 172.21.15.47:0/1034710383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f216de80 0x5561f2172ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.452+0000 7fb00d400640 1 --2- 172.21.15.47:0/1034710383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216e870 0x5561f217f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.453+0000 7fb00e800640 1 -- 172.21.15.47:0/1034710383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f216de80 msgr2=0x5561f2172ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.453+0000 7fb00e800640 1 --2- 172.21.15.47:0/1034710383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f216de80 0x5561f2172ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.453+0000 7fb00e800640 1 -- 172.21.15.47:0/1034710383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216e870 msgr2=0x5561f217f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:09.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.453+0000 7fb00e800640 1 --2- 172.21.15.47:0/1034710383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216e870 0x5561f217f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.453+0000 7fb00e800640 1 -- 172.21.15.47:0/1034710383 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561f1da70f0 con 0x5561f21739e0 2024-03-21T11:49:09.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.454+0000 7fb00d400640 1 --2- 172.21.15.47:0/1034710383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216e870 0x5561f217f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:09.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.454+0000 7fb00e800640 1 --2- 172.21.15.47:0/1034710383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f21739e0 0x5561f216ca30 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fb00809e040 tx=0x7fb008014be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ee102095116370b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:09.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.454+0000 7faffea00640 1 -- 172.21.15.47:0/1034710383 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00809b150 con 0x5561f21739e0 2024-03-21T11:49:09.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.454+0000 7faffea00640 1 -- 172.21.15.47:0/1034710383 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb00809f2a0 con 0x5561f21739e0 2024-03-21T11:49:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.455+0000 7fb017e32740 1 -- 172.21.15.47:0/1034710383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f21739e0 msgr2=0x5561f216ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:09.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.455+0000 7fb017e32740 1 --2- 172.21.15.47:0/1034710383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f21739e0 0x5561f216ca30 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fb00809e040 tx=0x7fb008014be0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.456+0000 7fb017e32740 1 -- 172.21.15.47:0/1034710383 shutdown_connections 2024-03-21T11:49:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.456+0000 7fb017e32740 1 --2- 172.21.15.47:0/1034710383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f21739e0 0x5561f216ca30 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.456+0000 7fb017e32740 1 --2- 172.21.15.47:0/1034710383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216e870 0x5561f217f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.456+0000 7fb017e32740 1 --2- 172.21.15.47:0/1034710383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f216de80 0x5561f2172ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.456+0000 7fb017e32740 1 -- 172.21.15.47:0/1034710383 >> 172.21.15.47:0/1034710383 conn(0x5561f2136200 msgr2=0x5561f216fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:09.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.457+0000 7fb017e32740 1 -- 172.21.15.47:0/1034710383 shutdown_connections 2024-03-21T11:49:09.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.457+0000 7fb017e32740 1 -- 172.21.15.47:0/1034710383 wait complete. 2024-03-21T11:49:09.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.458+0000 7fb017e32740 1 -- start start 2024-03-21T11:49:09.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.458+0000 7fb017e32740 1 Processor -- start 2024-03-21T11:49:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.459+0000 7fb017e32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216de80 0x5561f217d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.459+0000 7fb017e32740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 0x5561f217d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.459+0000 7fb017e32740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f21739e0 0x5561f217e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.459+0000 7fb017e32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec010c70 con 0x5561f216de80 2024-03-21T11:49:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.459+0000 7fb017e32740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec010610 con 0x5561f21739e0 2024-03-21T11:49:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.459+0000 7fb017e32740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec00db10 con 0x5561f216e870 2024-03-21T11:49:09.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.459+0000 7fb017e32740 10 client.0 _ll_register_callbacks cb 0x7fb008096cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.461+0000 7fb00d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 0x5561f217d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.461+0000 7fb00de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216de80 0x5561f217d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.461+0000 7fb00e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f21739e0 0x5561f217e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.461+0000 7fb00e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f21739e0 0x5561f217e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51680/0 (socket says 172.21.15.47:51680) 2024-03-21T11:49:09.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.461+0000 7fb00e800640 1 -- 172.21.15.47:0/3274511696 learned_addr learned my addr 172.21.15.47:0/3274511696 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:09.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.462+0000 7fb00e800640 1 -- 172.21.15.47:0/3274511696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 msgr2=0x5561f217d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:09.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.462+0000 7fb00e800640 1 --2- 172.21.15.47:0/3274511696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 0x5561f217d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.462+0000 7fb00e800640 1 -- 172.21.15.47:0/3274511696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216de80 msgr2=0x5561f217d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:09.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.462+0000 7fb00e800640 1 --2- 172.21.15.47:0/3274511696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216de80 0x5561f217d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.462+0000 7fb00e800640 1 -- 172.21.15.47:0/3274511696 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561f217ee30 con 0x5561f21739e0 2024-03-21T11:49:09.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.463+0000 7fb00e800640 1 --2- 172.21.15.47:0/3274511696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f21739e0 0x5561f217e840 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fb00800b940 tx=0x7fb008097150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b8a1fa86422272c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:09.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.463+0000 7faff3e00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:49:09.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.464+0000 7faff3e00640 1 -- 172.21.15.47:0/3274511696 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0080b8030 con 0x5561f21739e0 2024-03-21T11:49:09.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.464+0000 7faff3e00640 1 -- 172.21.15.47:0/3274511696 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb008014440 con 0x5561f21739e0 2024-03-21T11:49:09.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.464+0000 7fb017e32740 1 -- 172.21.15.47:0/3274511696 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb00809f2a0 con 0x5561f21739e0 2024-03-21T11:49:09.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.464+0000 7faff3e00640 1 -- 172.21.15.47:0/3274511696 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0080a3b60 con 0x5561f21739e0 2024-03-21T11:49:09.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.464+0000 7fb017e32740 10 client.33692 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:09.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.465+0000 7fb017e32740 1 -- 172.21.15.47:0/3274511696 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5561f21dfd50 con 0x5561f21739e0 2024-03-21T11:49:09.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.465+0000 7faff3e00640 1 -- 172.21.15.47:0/3274511696 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb00809f2a0 con 0x5561f21739e0 2024-03-21T11:49:09.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.466+0000 7faff3e00640 1 -- 172.21.15.47:0/3274511696 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb008014640 con 0x5561f21739e0 2024-03-21T11:49:09.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.466+0000 7fb017e32740 10 client.33692 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:09.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.466+0000 7fb017e32740 10 client.33692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:49:09.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.466+0000 7fb017e32740 20 client.33692 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:09.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.466+0000 7fb017e32740 1 --2- 172.21.15.47:0/3274511696 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5561f21e03f0 0x5561f21e2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:09.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.466+0000 7fb017e32740 4 client.33692 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:49:09.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.466+0000 7fb017e32740 1 -- 172.21.15.47:0/3274511696 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5561f20daae0 con 0x5561f21e03f0 2024-03-21T11:49:09.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.469+0000 7fb00de00640 1 --2- 172.21.15.47:0/3274511696 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5561f21e03f0 0x5561f21e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.471+0000 7faff3e00640 10 client.33692 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:09.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.471+0000 7fb00de00640 1 --2- 172.21.15.47:0/3274511696 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5561f21e03f0 0x5561f21e2810 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b80c848f9b8f69e server_cookie=8074722fb8199f89 in_seq=0 out_seq=0 2024-03-21T11:49:09.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.474+0000 7faff3e00640 1 -- 172.21.15.47:0/3274511696 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5561f20daae0 con 0x5561f21e03f0 2024-03-21T11:49:09.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.474+0000 7faff3e00640 10 client.33692 handle_command_reply: tid=0 2024-03-21T11:49:09.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.474+0000 7fb017e32740 1 client.33692 shutdown 2024-03-21T11:49:09.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.475+0000 7fb017e32740 1 -- 172.21.15.47:0/3274511696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f21739e0 msgr2=0x5561f217e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:09.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.475+0000 7fb017e32740 1 --2- 172.21.15.47:0/3274511696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f21739e0 0x5561f217e840 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fb00800b940 tx=0x7fb008097150 comp rx=0 tx=0).stop 2024-03-21T11:49:09.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.475+0000 7fb017e32740 1 -- 172.21.15.47:0/3274511696 shutdown_connections 2024-03-21T11:49:09.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.475+0000 7fb017e32740 1 --2- 172.21.15.47:0/3274511696 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5561f21e03f0 0x5561f21e2810 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.475+0000 7fb017e32740 1 --2- 172.21.15.47:0/3274511696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5561f21739e0 0x5561f217e840 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.475+0000 7fb017e32740 1 --2- 172.21.15.47:0/3274511696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5561f216e870 0x5561f217d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.475+0000 7fb017e32740 1 --2- 172.21.15.47:0/3274511696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f216de80 0x5561f217d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.475+0000 7faff3e00640 0 client.33692 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:09.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.475+0000 7fb017e32740 1 -- 172.21.15.47:0/3274511696 >> 172.21.15.47:0/3274511696 conn(0x5561f2136200 msgr2=0x5561f216fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:09.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.475+0000 7fb017e32740 1 -- 172.21.15.47:0/3274511696 shutdown_connections 2024-03-21T11:49:09.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.475+0000 7fb017e32740 1 -- 172.21.15.47:0/3274511696 wait complete. 2024-03-21T11:49:09.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.475+0000 7fb017e32740 20 client.33692 trim_cache size 0 max 16384 2024-03-21T11:49:09.477 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:49:09.477 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "scrub active (178 inodes in the stack)", 2024-03-21T11:49:09.477 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T11:49:09.477 INFO:teuthology.orchestra.run.smithi047.stdout: "bba34861-fd0b-4a13-897f-d8ffaaef8f0c": { 2024-03-21T11:49:09.477 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T11:49:09.478 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "bba34861-fd0b-4a13-897f-d8ffaaef8f0c", 2024-03-21T11:49:09.478 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T11:49:09.478 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:09.478 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:09.478 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:49:09.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.476+0000 7fb015200640 1 -- 172.21.15.47:0/3457189900 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faff40a3070 msgr2=0x7faff40a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:09.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.476+0000 7fb015200640 1 --2- 172.21.15.47:0/3457189900 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faff40a3070 0x7faff40a5530 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7faff8005fd0 tx=0x7faff8005f00 comp rx=0 tx=0).stop 2024-03-21T11:49:09.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.476+0000 7fb015200640 1 -- 172.21.15.47:0/3457189900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb010109870 msgr2=0x7fb01007ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:09.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.476+0000 7fb015200640 1 --2- 172.21.15.47:0/3457189900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb010109870 0x7fb01007ffe0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fb00800b970 tx=0x7fb00800be30 comp rx=0 tx=0).stop 2024-03-21T11:49:09.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.476+0000 7fb015200640 1 -- 172.21.15.47:0/3457189900 shutdown_connections 2024-03-21T11:49:09.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.476+0000 7fb015200640 1 --2- 172.21.15.47:0/3457189900 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7faff40a3070 0x7faff40a5530 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.476+0000 7fb015200640 1 --2- 172.21.15.47:0/3457189900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb010109870 0x7fb01007ffe0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.476+0000 7fb015200640 1 --2- 172.21.15.47:0/3457189900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb010108eb0 0x7fb010081ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.476+0000 7fb015200640 1 --2- 172.21.15.47:0/3457189900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb010107cc0 0x7fb0100819a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:09.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.476+0000 7fb015200640 1 -- 172.21.15.47:0/3457189900 >> 172.21.15.47:0/3457189900 conn(0x7fb010076c60 msgr2=0x7fb01010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:09.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.476+0000 7fb015200640 1 -- 172.21.15.47:0/3457189900 shutdown_connections 2024-03-21T11:49:09.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:09.476+0000 7fb015200640 1 -- 172.21.15.47:0/3457189900 wait complete. 2024-03-21T11:49:09.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:09 smithi102 ceph-mon[36748]: pgmap v1175: 129 pgs: 129 active+clean; 747 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.8 MiB/s wr, 641 op/s 2024-03-21T11:49:09.498 INFO:tasks.cephfs.filesystem:scrub status for tag:bba34861-fd0b-4a13-897f-d8ffaaef8f0c - {'path': '/', 'tag': 'bba34861-fd0b-4a13-897f-d8ffaaef8f0c', 'options': 'recursive,force'} 2024-03-21T11:49:09.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:09 smithi100 ceph-mon[36245]: pgmap v1175: 129 pgs: 129 active+clean; 747 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.8 MiB/s wr, 641 op/s 2024-03-21T11:49:10.007 INFO:tasks.workunit.client.0.smithi047.stdout: 10 293800 21.40 MB/sec execute 593 sec latency 116.442 ms 2024-03-21T11:49:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.330+0000 7fd5a0c00640 1 -- 172.21.15.47:0/524620762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd59c104ac0 msgr2=0x7fd59c104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.330+0000 7fd5a0c00640 1 --2- 172.21.15.47:0/524620762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd59c104ac0 0x7fd59c104f40 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fd588009900 tx=0x7fd58802bc40 comp rx=0 tx=0).stop 2024-03-21T11:49:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.331+0000 7fd5a0c00640 1 -- 172.21.15.47:0/524620762 shutdown_connections 2024-03-21T11:49:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.331+0000 7fd5a0c00640 1 --2- 172.21.15.47:0/524620762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd59c105480 0x7fd59c10bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.331+0000 7fd5a0c00640 1 --2- 172.21.15.47:0/524620762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd59c104ac0 0x7fd59c104f40 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.331+0000 7fd5a0c00640 1 --2- 172.21.15.47:0/524620762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd59c1038d0 0x7fd59c103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.331+0000 7fd5a0c00640 1 -- 172.21.15.47:0/524620762 >> 172.21.15.47:0/524620762 conn(0x7fd59c0fee60 msgr2=0x7fd59c101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.333+0000 7fd5a0c00640 1 -- 172.21.15.47:0/524620762 shutdown_connections 2024-03-21T11:49:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.334+0000 7fd5a0c00640 1 -- 172.21.15.47:0/524620762 wait complete. 2024-03-21T11:49:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.335+0000 7fd5a0c00640 1 Processor -- start 2024-03-21T11:49:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.336+0000 7fd5a0c00640 1 -- start start 2024-03-21T11:49:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.336+0000 7fd5a0c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd59c1038d0 0x7fd59c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.336+0000 7fd5a0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd59c104ac0 0x7fd59c199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.336+0000 7fd5a0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd59c105480 0x7fd59c1a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.337+0000 7fd599400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd59c104ac0 0x7fd59c199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.337+0000 7fd599400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd59c104ac0 0x7fd59c199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51696/0 (socket says 172.21.15.47:51696) 2024-03-21T11:49:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.337+0000 7fd599400640 1 -- 172.21.15.47:0/2315727520 learned_addr learned my addr 172.21.15.47:0/2315727520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.337+0000 7fd5a0c00640 1 -- 172.21.15.47:0/2315727520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c10f600 con 0x7fd59c105480 2024-03-21T11:49:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.337+0000 7fd5a0c00640 1 -- 172.21.15.47:0/2315727520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c10f480 con 0x7fd59c104ac0 2024-03-21T11:49:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.337+0000 7fd5a0c00640 1 -- 172.21.15.47:0/2315727520 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c10f780 con 0x7fd59c1038d0 2024-03-21T11:49:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.337+0000 7fd599e00640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd59c1038d0 0x7fd59c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.339+0000 7fd59a800640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd59c105480 0x7fd59c1a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.340+0000 7fd59a800640 1 -- 172.21.15.47:0/2315727520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd59c1038d0 msgr2=0x7fd59c199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.340+0000 7fd59a800640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd59c1038d0 0x7fd59c199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.340+0000 7fd59a800640 1 -- 172.21.15.47:0/2315727520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd59c104ac0 msgr2=0x7fd59c199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.340+0000 7fd59a800640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd59c104ac0 0x7fd59c199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.340+0000 7fd59a800640 1 -- 172.21.15.47:0/2315727520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c1a15b0 con 0x7fd59c105480 2024-03-21T11:49:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.340+0000 7fd599400640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd59c104ac0 0x7fd59c199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:49:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.341+0000 7fd599e00640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd59c1038d0 0x7fd59c199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:49:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.341+0000 7fd59a800640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd59c105480 0x7fd59c1a0f40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00be20 tx=0x7fd58c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.343+0000 7fd592a00640 1 -- 172.21.15.47:0/2315727520 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c019070 con 0x7fd59c105480 2024-03-21T11:49:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.343+0000 7fd592a00640 1 -- 172.21.15.47:0/2315727520 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd58c014920 con 0x7fd59c105480 2024-03-21T11:49:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.343+0000 7fd5a0c00640 1 -- 172.21.15.47:0/2315727520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59c1a18a0 con 0x7fd59c105480 2024-03-21T11:49:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.344+0000 7fd5a0c00640 1 -- 172.21.15.47:0/2315727520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59c1a1e40 con 0x7fd59c105480 2024-03-21T11:49:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.344+0000 7fd592a00640 1 -- 172.21.15.47:0/2315727520 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c012920 con 0x7fd59c105480 2024-03-21T11:49:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.349+0000 7fd5a39a9740 1 Processor -- start 2024-03-21T11:49:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.351+0000 7fd592a00640 1 -- 172.21.15.47:0/2315727520 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd58c01c070 con 0x7fd59c105480 2024-03-21T11:49:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.352+0000 7fd5a39a9740 1 -- start start 2024-03-21T11:49:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.352+0000 7fd5a39a9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c721e0 0x5562b8c72600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.352+0000 7fd5a39a9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 0x5562b8c776d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.352+0000 7fd5a39a9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c77d40 0x5562b8c7a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.352+0000 7fd5a39a9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b8c353b0 con 0x5562b8c77d40 2024-03-21T11:49:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.352+0000 7fd5a39a9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b8bd7830 con 0x5562b8c721e0 2024-03-21T11:49:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.352+0000 7fd5a39a9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b8b805d0 con 0x5562b8c72bd0 2024-03-21T11:49:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.353+0000 7fd599400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c721e0 0x5562b8c72600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.353+0000 7fd599400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c721e0 0x5562b8c72600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51708/0 (socket says 172.21.15.47:51708) 2024-03-21T11:49:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.353+0000 7fd599400640 1 -- 172.21.15.47:0/658218379 learned_addr learned my addr 172.21.15.47:0/658218379 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.353+0000 7fd599400640 1 --2- 172.21.15.47:0/658218379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c77d40 0x5562b8c7a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.354+0000 7fd599400640 1 -- 172.21.15.47:0/658218379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 msgr2=0x5562b8c776d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:49:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.354+0000 7fd599400640 1 --2- 172.21.15.47:0/658218379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 0x5562b8c776d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.354+0000 7fd599400640 1 -- 172.21.15.47:0/658218379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c77d40 msgr2=0x5562b8c7a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.354+0000 7fd599400640 1 --2- 172.21.15.47:0/658218379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c77d40 0x5562b8c7a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.354+0000 7fd599400640 1 -- 172.21.15.47:0/658218379 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562b8b982c0 con 0x5562b8c721e0 2024-03-21T11:49:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.354+0000 7fd592a00640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5800a3040 0x7fd5800a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.354+0000 7fd592a00640 1 -- 172.21.15.47:0/2315727520 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd58c0ceee0 con 0x7fd59c105480 2024-03-21T11:49:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.354+0000 7fd599400640 1 --2- 172.21.15.47:0/658218379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c721e0 0x5562b8c72600 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fd588005ab0 tx=0x7fd5880095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3796f87a52cd8e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.355+0000 7fd587000640 1 -- 172.21.15.47:0/658218379 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588049070 con 0x5562b8c721e0 2024-03-21T11:49:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.355+0000 7fd587000640 1 -- 172.21.15.47:0/658218379 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5880044a0 con 0x5562b8c721e0 2024-03-21T11:49:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.355+0000 7fd587000640 1 -- 172.21.15.47:0/658218379 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588004bf0 con 0x5562b8c721e0 2024-03-21T11:49:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.356+0000 7fd599e00640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5800a3040 0x7fd5800a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.356+0000 7fd5a39a9740 1 -- 172.21.15.47:0/658218379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c721e0 msgr2=0x5562b8c72600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.357+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/658218379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c721e0 0x5562b8c72600 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fd588005ab0 tx=0x7fd5880095a0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.357+0000 7fd5a39a9740 1 -- 172.21.15.47:0/658218379 shutdown_connections 2024-03-21T11:49:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.357+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/658218379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c77d40 0x5562b8c7a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.357+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/658218379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 0x5562b8c776d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.357+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/658218379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c721e0 0x5562b8c72600 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.357+0000 7fd5a39a9740 1 -- 172.21.15.47:0/658218379 >> 172.21.15.47:0/658218379 conn(0x5562b8c6d750 msgr2=0x5562b8c6db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.357+0000 7fd599e00640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5800a3040 0x7fd5800a5500 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x5562b8c72fd0 tx=0x7fd594005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.357+0000 7fd5a39a9740 1 -- 172.21.15.47:0/658218379 shutdown_connections 2024-03-21T11:49:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.358+0000 7fd5a39a9740 1 -- 172.21.15.47:0/658218379 wait complete. 2024-03-21T11:49:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.359+0000 7fd5a39a9740 1 -- start start 2024-03-21T11:49:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.359+0000 7fd5a39a9740 1 Processor -- start 2024-03-21T11:49:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.360+0000 7fd5a39a9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c721e0 0x5562b8c84bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.360+0000 7fd5a39a9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 0x5562b8c85110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.360+0000 7fd5a39a9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c77d40 0x5562b8c8c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.360+0000 7fd5a39a9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b8b93a20 con 0x5562b8c721e0 2024-03-21T11:49:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.360+0000 7fd5a39a9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b8bd38d0 con 0x5562b8c77d40 2024-03-21T11:49:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.360+0000 7fd5a39a9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b8c7ac90 con 0x5562b8c72bd0 2024-03-21T11:49:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.360+0000 7fd5a39a9740 10 client.0 _ll_register_callbacks cb 0x5562b8bd01c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.361+0000 7fd59a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c77d40 0x5562b8c8c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.361+0000 7fd59a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c77d40 0x5562b8c8c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51720/0 (socket says 172.21.15.47:51720) 2024-03-21T11:49:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.361+0000 7fd59a800640 1 -- 172.21.15.47:0/1204654334 learned_addr learned my addr 172.21.15.47:0/1204654334 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.361+0000 7fd599400640 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 0x5562b8c85110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.361+0000 7fd599e00640 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c721e0 0x5562b8c84bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.362+0000 7fd59a800640 1 -- 172.21.15.47:0/1204654334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 msgr2=0x5562b8c85110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.363+0000 7fd59a800640 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 0x5562b8c85110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.363+0000 7fd59a800640 1 -- 172.21.15.47:0/1204654334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c721e0 msgr2=0x5562b8c84bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.363+0000 7fd59a800640 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c721e0 0x5562b8c84bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.363+0000 7fd59a800640 1 -- 172.21.15.47:0/1204654334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562b88ec210 con 0x5562b8c77d40 2024-03-21T11:49:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.363+0000 7fd599400640 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 0x5562b8c85110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.363+0000 7fd599e00640 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c721e0 0x5562b8c84bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:49:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.364+0000 7fd59a800640 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c77d40 0x5562b8c8c150 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0976e0 tx=0x7fd58c09ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f84684bdda77923 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.364+0000 7fd57b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:49:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.365+0000 7fd57b400640 1 -- 172.21.15.47:0/1204654334 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c0a3c90 con 0x5562b8c77d40 2024-03-21T11:49:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.365+0000 7fd57b400640 1 -- 172.21.15.47:0/1204654334 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd58c0a3800 con 0x5562b8c77d40 2024-03-21T11:49:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.365+0000 7fd5a39a9740 1 -- 172.21.15.47:0/1204654334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5880044a0 con 0x5562b8c77d40 2024-03-21T11:49:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.368+0000 7fd57b400640 1 -- 172.21.15.47:0/1204654334 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd58c0a3a00 con 0x5562b8c77d40 2024-03-21T11:49:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.368+0000 7fd5a39a9740 10 client.33701 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.368+0000 7fd5a39a9740 1 -- 172.21.15.47:0/1204654334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5562b89b8620 con 0x5562b8c77d40 2024-03-21T11:49:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.372+0000 7fd57b400640 1 -- 172.21.15.47:0/1204654334 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5880044a0 con 0x5562b8c77d40 2024-03-21T11:49:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.372+0000 7fd57b400640 1 -- 172.21.15.47:0/1204654334 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd58c0ccbb0 con 0x5562b8c77d40 2024-03-21T11:49:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.372+0000 7fd5a39a9740 10 client.33701 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.372+0000 7fd5a39a9740 10 client.33701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:49:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.372+0000 7fd5a39a9740 20 client.33701 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.372+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5562b8c909f0 0x5562b8c92e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.372+0000 7fd5a39a9740 4 client.33701 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.372+0000 7fd5a39a9740 1 -- 172.21.15.47:0/1204654334 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562b8be4d70 con 0x5562b8c909f0 2024-03-21T11:49:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.373+0000 7fd599e00640 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5562b8c909f0 0x5562b8c92e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.374+0000 7fd599e00640 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5562b8c909f0 0x5562b8c92e00 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9158b7bd69ad75e9 server_cookie=2f3bad3130e3eac in_seq=0 out_seq=0 2024-03-21T11:49:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.374+0000 7fd57b400640 10 client.33701 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.378+0000 7fd57b400640 1 -- 172.21.15.47:0/1204654334 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5562b8be4d70 con 0x5562b8c909f0 2024-03-21T11:49:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.378+0000 7fd57b400640 10 client.33701 handle_command_reply: tid=0 2024-03-21T11:49:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.378+0000 7fd5a39a9740 1 client.33701 shutdown 2024-03-21T11:49:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.379+0000 7fd5a39a9740 1 -- 172.21.15.47:0/1204654334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c77d40 msgr2=0x5562b8c8c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.379+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c77d40 0x5562b8c8c150 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0976e0 tx=0x7fd58c09ba40 comp rx=0 tx=0).stop 2024-03-21T11:49:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.380+0000 7fd5a39a9740 1 -- 172.21.15.47:0/1204654334 shutdown_connections 2024-03-21T11:49:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.380+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5562b8c909f0 0x5562b8c92e00 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.380+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c77d40 0x5562b8c8c150 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.380+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 0x5562b8c85110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.380+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/1204654334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c721e0 0x5562b8c84bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.380+0000 7fd5a39a9740 1 -- 172.21.15.47:0/1204654334 >> 172.21.15.47:0/1204654334 conn(0x5562b8c6d750 msgr2=0x5562b8c6c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.380+0000 7fd57b400640 0 client.33701 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.381+0000 7fd5a39a9740 1 -- 172.21.15.47:0/1204654334 shutdown_connections 2024-03-21T11:49:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.382+0000 7fd5a39a9740 1 -- 172.21.15.47:0/1204654334 wait complete. 2024-03-21T11:49:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.382+0000 7fd5a39a9740 20 client.33701 trim_cache size 0 max 16384 2024-03-21T11:49:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.408+0000 7fd5a39a9740 1 Processor -- start 2024-03-21T11:49:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.408+0000 7fd5a39a9740 1 -- start start 2024-03-21T11:49:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.408+0000 7fd5a39a9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c721e0 0x5562b8c6c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.408+0000 7fd5a39a9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c72bd0 0x5562b8c8c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.408+0000 7fd5a39a9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c77d40 0x5562b8c6c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.408+0000 7fd5a39a9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c00f900 con 0x5562b8c77d40 2024-03-21T11:49:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.408+0000 7fd5a39a9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c011280 con 0x5562b8c72bd0 2024-03-21T11:49:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.408+0000 7fd5a39a9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c00ff60 con 0x5562b8c721e0 2024-03-21T11:49:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.409+0000 7fd599400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c72bd0 0x5562b8c8c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.409+0000 7fd599400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c72bd0 0x5562b8c8c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51730/0 (socket says 172.21.15.47:51730) 2024-03-21T11:49:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.409+0000 7fd599400640 1 -- 172.21.15.47:0/3430043909 learned_addr learned my addr 172.21.15.47:0/3430043909 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.409+0000 7fd599e00640 1 --2- 172.21.15.47:0/3430043909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c721e0 0x5562b8c6c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.409+0000 7fd59a800640 1 --2- 172.21.15.47:0/3430043909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c77d40 0x5562b8c6c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.409+0000 7fd599400640 1 -- 172.21.15.47:0/3430043909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c721e0 msgr2=0x5562b8c6c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.409+0000 7fd599400640 1 --2- 172.21.15.47:0/3430043909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c721e0 0x5562b8c6c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.409+0000 7fd599400640 1 -- 172.21.15.47:0/3430043909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c77d40 msgr2=0x5562b8c6c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.409+0000 7fd599400640 1 --2- 172.21.15.47:0/3430043909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c77d40 0x5562b8c6c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.409+0000 7fd599400640 1 -- 172.21.15.47:0/3430043909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562b8c739e0 con 0x5562b8c72bd0 2024-03-21T11:49:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.409+0000 7fd599e00640 1 --2- 172.21.15.47:0/3430043909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c721e0 0x5562b8c6c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.410+0000 7fd599400640 1 --2- 172.21.15.47:0/3430043909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c72bd0 0x5562b8c8c110 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fd588047040 tx=0x7fd58802c2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dcbb244591bfc1c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.410+0000 7fd586600640 1 -- 172.21.15.47:0/3430043909 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588049070 con 0x5562b8c72bd0 2024-03-21T11:49:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.411+0000 7fd586600640 1 -- 172.21.15.47:0/3430043909 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5880044a0 con 0x5562b8c72bd0 2024-03-21T11:49:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.411+0000 7fd586600640 1 -- 172.21.15.47:0/3430043909 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588004d60 con 0x5562b8c72bd0 2024-03-21T11:49:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.411+0000 7fd5a39a9740 1 -- 172.21.15.47:0/3430043909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c72bd0 msgr2=0x5562b8c8c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.411+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/3430043909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c72bd0 0x5562b8c8c110 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fd588047040 tx=0x7fd58802c2c0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.411+0000 7fd5a39a9740 1 -- 172.21.15.47:0/3430043909 shutdown_connections 2024-03-21T11:49:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.412+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/3430043909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c77d40 0x5562b8c6c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.412+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/3430043909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c72bd0 0x5562b8c8c110 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.412+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/3430043909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c721e0 0x5562b8c6c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.412+0000 7fd5a39a9740 1 -- 172.21.15.47:0/3430043909 >> 172.21.15.47:0/3430043909 conn(0x5562b8c6d750 msgr2=0x5562b8c73ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.412+0000 7fd5a39a9740 1 -- 172.21.15.47:0/3430043909 shutdown_connections 2024-03-21T11:49:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.412+0000 7fd5a39a9740 1 -- 172.21.15.47:0/3430043909 wait complete. 2024-03-21T11:49:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.413+0000 7fd5a39a9740 1 -- start start 2024-03-21T11:49:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.413+0000 7fd5a39a9740 1 Processor -- start 2024-03-21T11:49:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.413+0000 7fd5a39a9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c721e0 0x5562b8c87c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.414+0000 7fd5a39a9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 0x5562b8c881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.414+0000 7fd5a39a9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c77d40 0x5562b8c8b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.414+0000 7fd5a39a9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c010c20 con 0x5562b8c721e0 2024-03-21T11:49:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.414+0000 7fd5a39a9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c0105c0 con 0x5562b8c77d40 2024-03-21T11:49:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.414+0000 7fd5a39a9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c00dac0 con 0x5562b8c72bd0 2024-03-21T11:49:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.414+0000 7fd599400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 0x5562b8c881b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.414+0000 7fd599400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 0x5562b8c881b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:48854/0 (socket says 172.21.15.47:48854) 2024-03-21T11:49:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.414+0000 7fd5a39a9740 10 client.0 _ll_register_callbacks cb 0x7fd58c00cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.414+0000 7fd599400640 1 -- 172.21.15.47:0/3680398236 learned_addr learned my addr 172.21.15.47:0/3680398236 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.414+0000 7fd59a800640 1 --2- 172.21.15.47:0/3680398236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c77d40 0x5562b8c8b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.415+0000 7fd599e00640 1 --2- 172.21.15.47:0/3680398236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c721e0 0x5562b8c87c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.415+0000 7fd599400640 1 -- 172.21.15.47:0/3680398236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c77d40 msgr2=0x5562b8c8b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.415+0000 7fd599400640 1 --2- 172.21.15.47:0/3680398236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c77d40 0x5562b8c8b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.415+0000 7fd599400640 1 -- 172.21.15.47:0/3680398236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c721e0 msgr2=0x5562b8c87c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.415+0000 7fd599400640 1 --2- 172.21.15.47:0/3680398236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c721e0 0x5562b8c87c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.415+0000 7fd599400640 1 -- 172.21.15.47:0/3680398236 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562b8c8b770 con 0x5562b8c72bd0 2024-03-21T11:49:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.415+0000 7fd59a800640 1 --2- 172.21.15.47:0/3680398236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c77d40 0x5562b8c8b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.418+0000 7fd599400640 1 --2- 172.21.15.47:0/3680398236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 0x5562b8c881b0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fd588006010 tx=0x7fd58802cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3523e4efba87b72f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.418+0000 7fd57b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:49:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.418+0000 7fd57b400640 1 -- 172.21.15.47:0/3680398236 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588049040 con 0x5562b8c72bd0 2024-03-21T11:49:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.418+0000 7fd57b400640 1 -- 172.21.15.47:0/3680398236 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fd588005200 con 0x5562b8c72bd0 2024-03-21T11:49:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.418+0000 7fd5a39a9740 1 -- 172.21.15.47:0/3680398236 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5880044a0 con 0x5562b8c72bd0 2024-03-21T11:49:10.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.419+0000 7fd57b400640 1 -- 172.21.15.47:0/3680398236 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd588005400 con 0x5562b8c72bd0 2024-03-21T11:49:10.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.419+0000 7fd5a39a9740 10 client.33613 fetch_fsmap learned FSMap version 41 2024-03-21T11:49:10.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.419+0000 7fd5a39a9740 1 -- 172.21.15.47:0/3680398236 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5562b8cfd750 con 0x5562b8c72bd0 2024-03-21T11:49:10.420 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.419+0000 7fd57b400640 1 -- 172.21.15.47:0/3680398236 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5880044a0 con 0x5562b8c72bd0 2024-03-21T11:49:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.421+0000 7fd57b400640 1 -- 172.21.15.47:0/3680398236 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 41) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd5880035f0 con 0x5562b8c72bd0 2024-03-21T11:49:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.421+0000 7fd5a39a9740 10 client.33613 fetch_fsmap finished waiting for FSMap version 41 2024-03-21T11:49:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.421+0000 7fd5a39a9740 10 client.33613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T11:49:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.421+0000 7fd5a39a9740 20 client.33613 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.421+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/3680398236 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5562b8cfdb10 0x5562b8cfff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.421+0000 7fd5a39a9740 4 client.33613 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T11:49:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.421+0000 7fd5a39a9740 1 -- 172.21.15.47:0/3680398236 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5562b8bdf0c0 con 0x5562b8cfdb10 2024-03-21T11:49:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.422+0000 7fd599e00640 1 --2- 172.21.15.47:0/3680398236 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5562b8cfdb10 0x5562b8cfff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.428+0000 7fd599e00640 1 --2- 172.21.15.47:0/3680398236 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5562b8cfdb10 0x5562b8cfff00 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc81104248a42d65 server_cookie=a18affd2d3195d3d in_seq=0 out_seq=0 2024-03-21T11:49:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:10.428+0000 7fd57b400640 10 client.33613 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:10.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub queued for path: / 2024-03-21T11:49:10.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: active paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:10 smithi102 ceph-mon[36748]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub queued for path: / 2024-03-21T11:49:10.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:49:10.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:49:10.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:49:10.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:49:10.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:49:10.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:49:10.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:49:10.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:49:10.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: active paths [/] 2024-03-21T11:49:10.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:10 smithi047 ceph-mon[31351]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub queued for path: / 2024-03-21T11:49:10.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.644 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: active paths [/] 2024-03-21T11:49:10.645 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.645 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:10.645 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:10 smithi100 ceph-mon[36245]: scrub summary: idle+waiting paths [/] 2024-03-21T11:49:11.007 INFO:tasks.workunit.client.0.smithi047.stdout: 10 294033 21.38 MB/sec execute 594 sec latency 420.670 ms 2024-03-21T11:49:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.479+0000 7fd57b400640 1 -- 172.21.15.47:0/3680398236 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5562b8bdf0c0 con 0x5562b8cfdb10 2024-03-21T11:49:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.479+0000 7fd57b400640 10 client.33613 handle_command_reply: tid=0 2024-03-21T11:49:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.479+0000 7fd5a39a9740 1 client.33613 shutdown 2024-03-21T11:49:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.479+0000 7fd5a39a9740 1 -- 172.21.15.47:0/3680398236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 msgr2=0x5562b8c881b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.479+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/3680398236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 0x5562b8c881b0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fd588006010 tx=0x7fd58802cc50 comp rx=0 tx=0).stop 2024-03-21T11:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.483+0000 7fd5a39a9740 1 -- 172.21.15.47:0/3680398236 shutdown_connections 2024-03-21T11:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.483+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/3680398236 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x5562b8cfdb10 0x5562b8cfff00 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.483+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/3680398236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5562b8c77d40 0x5562b8c8b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.483+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/3680398236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5562b8c72bd0 0x5562b8c881b0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.483+0000 7fd5a39a9740 1 --2- 172.21.15.47:0/3680398236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5562b8c721e0 0x5562b8c87c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.483+0000 7fd5a39a9740 1 -- 172.21.15.47:0/3680398236 >> 172.21.15.47:0/3680398236 conn(0x5562b8c6d750 msgr2=0x5562b8c709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.483+0000 7fd57b400640 0 client.33613 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.483+0000 7fd5a39a9740 1 -- 172.21.15.47:0/3680398236 shutdown_connections 2024-03-21T11:49:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.483+0000 7fd5a39a9740 1 -- 172.21.15.47:0/3680398236 wait complete. 2024-03-21T11:49:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.484+0000 7fd5a39a9740 20 client.33613 trim_cache size 0 max 16384 2024-03-21T11:49:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T11:49:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T11:49:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T11:49:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T11:49:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.485+0000 7fd5a0c00640 1 -- 172.21.15.47:0/2315727520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5800a3040 msgr2=0x7fd5800a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.485+0000 7fd5a0c00640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5800a3040 0x7fd5800a5500 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x5562b8c72fd0 tx=0x7fd594005f90 comp rx=0 tx=0).stop 2024-03-21T11:49:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.485+0000 7fd5a0c00640 1 -- 172.21.15.47:0/2315727520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd59c105480 msgr2=0x7fd59c1a0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.485+0000 7fd5a0c00640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd59c105480 0x7fd59c1a0f40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00be20 tx=0x7fd58c00bf10 comp rx=0 tx=0).stop 2024-03-21T11:49:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.485+0000 7fd5a0c00640 1 -- 172.21.15.47:0/2315727520 shutdown_connections 2024-03-21T11:49:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.485+0000 7fd5a0c00640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fd5800a3040 0x7fd5800a5500 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.485+0000 7fd5a0c00640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd59c105480 0x7fd59c1a0f40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.485+0000 7fd5a0c00640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd59c104ac0 0x7fd59c199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.485+0000 7fd5a0c00640 1 --2- 172.21.15.47:0/2315727520 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd59c1038d0 0x7fd59c199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.485+0000 7fd5a0c00640 1 -- 172.21.15.47:0/2315727520 >> 172.21.15.47:0/2315727520 conn(0x7fd59c0fee60 msgr2=0x7fd59c107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.486+0000 7fd5a0c00640 1 -- 172.21.15.47:0/2315727520 shutdown_connections 2024-03-21T11:49:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T11:49:11.486+0000 7fd5a0c00640 1 -- 172.21.15.47:0/2315727520 wait complete. 2024-03-21T11:49:11.566 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:11 smithi047 ceph-mon[31351]: pgmap v1176: 129 pgs: 129 active+clean; 751 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.5 MiB/s wr, 488 op/s 2024-03-21T11:49:11.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:11 smithi100 ceph-mon[36245]: pgmap v1176: 129 pgs: 129 active+clean; 751 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.5 MiB/s wr, 488 op/s 2024-03-21T11:49:11.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:11 smithi102 ceph-mon[36748]: pgmap v1176: 129 pgs: 129 active+clean; 751 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.5 MiB/s wr, 488 op/s 2024-03-21T11:49:12.007 INFO:tasks.workunit.client.0.smithi047.stdout: 10 294371 21.38 MB/sec execute 595 sec latency 798.959 ms 2024-03-21T11:49:13.007 INFO:tasks.workunit.client.0.smithi047.stdout: 10 294747 21.38 MB/sec execute 596 sec latency 84.456 ms 2024-03-21T11:49:13.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:13 smithi047 ceph-mon[31351]: pgmap v1177: 129 pgs: 129 active+clean; 752 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 427 op/s 2024-03-21T11:49:13.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:13 smithi100 ceph-mon[36245]: pgmap v1177: 129 pgs: 129 active+clean; 752 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 427 op/s 2024-03-21T11:49:13.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:13 smithi102 ceph-mon[36748]: pgmap v1177: 129 pgs: 129 active+clean; 752 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 427 op/s 2024-03-21T11:49:14.007 INFO:tasks.workunit.client.0.smithi047.stdout: 10 295144 21.38 MB/sec execute 597 sec latency 82.588 ms 2024-03-21T11:49:15.007 INFO:tasks.workunit.client.0.smithi047.stdout: 10 295551 21.38 MB/sec execute 598 sec latency 106.381 ms 2024-03-21T11:49:15.542 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:15 smithi102 ceph-mon[36748]: pgmap v1178: 129 pgs: 129 active+clean; 753 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.1 MiB/s wr, 631 op/s 2024-03-21T11:49:15.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:15 smithi047 ceph-mon[31351]: pgmap v1178: 129 pgs: 129 active+clean; 753 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.1 MiB/s wr, 631 op/s 2024-03-21T11:49:15.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:15 smithi100 ceph-mon[36245]: pgmap v1178: 129 pgs: 129 active+clean; 753 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.1 MiB/s wr, 631 op/s 2024-03-21T11:49:16.007 INFO:tasks.workunit.client.0.smithi047.stdout: 10 296116 21.38 MB/sec execute 599 sec latency 175.967 ms 2024-03-21T11:49:17.007 INFO:tasks.workunit.client.0.smithi047.stdout: 10 cleanup 600 sec 2024-03-21T11:49:17.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:17 smithi047 ceph-mon[31351]: pgmap v1179: 129 pgs: 129 active+clean; 753 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.3 MiB/s wr, 480 op/s 2024-03-21T11:49:17.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:17 smithi100 ceph-mon[36245]: pgmap v1179: 129 pgs: 129 active+clean; 753 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.3 MiB/s wr, 480 op/s 2024-03-21T11:49:17.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:17 smithi102 ceph-mon[36748]: pgmap v1179: 129 pgs: 129 active+clean; 753 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.3 MiB/s wr, 480 op/s 2024-03-21T11:49:18.007 INFO:tasks.workunit.client.0.smithi047.stdout: 7 cleanup 601 sec 2024-03-21T11:49:18.488 INFO:tasks.workunit.client.0.smithi047.stdout: 0 cleanup 602 sec 2024-03-21T11:49:18.488 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-21T11:49:18.488 INFO:tasks.workunit.client.0.smithi047.stdout: Operation Count AvgLat MaxLat 2024-03-21T11:49:18.488 INFO:tasks.workunit.client.0.smithi047.stdout: ---------------------------------------- 2024-03-21T11:49:18.488 INFO:tasks.workunit.client.0.smithi047.stdout: NTCreateX 412199 5.885 2997.178 2024-03-21T11:49:18.488 INFO:tasks.workunit.client.0.smithi047.stdout: Close 302763 0.010 3.339 2024-03-21T11:49:18.488 INFO:tasks.workunit.client.0.smithi047.stdout: Rename 17469 26.608 1405.599 2024-03-21T11:49:18.488 INFO:tasks.workunit.client.0.smithi047.stdout: Unlink 83277 5.893 550.865 2024-03-21T11:49:18.489 INFO:tasks.workunit.client.0.smithi047.stdout: Qpathinfo 373946 4.264 2598.831 2024-03-21T11:49:18.489 INFO:tasks.workunit.client.0.smithi047.stdout: Qfileinfo 65194 0.003 1.254 2024-03-21T11:49:18.489 INFO:tasks.workunit.client.0.smithi047.stdout: Qfsinfo 68484 0.634 70.704 2024-03-21T11:49:18.489 INFO:tasks.workunit.client.0.smithi047.stdout: Sfileinfo 33629 0.439 637.891 2024-03-21T11:49:18.489 INFO:tasks.workunit.client.0.smithi047.stdout: Find 144472 0.639 1401.657 2024-03-21T11:49:18.489 INFO:tasks.workunit.client.0.smithi047.stdout: WriteX 203682 0.207 2470.345 2024-03-21T11:49:18.489 INFO:tasks.workunit.client.0.smithi047.stdout: ReadX 646429 0.161 2231.726 2024-03-21T11:49:18.489 INFO:tasks.workunit.client.0.smithi047.stdout: LockX 1342 4.549 54.800 2024-03-21T11:49:18.489 INFO:tasks.workunit.client.0.smithi047.stdout: UnlockX 1342 3.712 38.526 2024-03-21T11:49:18.489 INFO:tasks.workunit.client.0.smithi047.stdout: Flush 28908 24.215 2512.645 2024-03-21T11:49:18.489 INFO:tasks.workunit.client.0.smithi047.stdout: 2024-03-21T11:49:18.489 INFO:tasks.workunit.client.0.smithi047.stdout:Throughput 21.3836 MB/sec 10 clients 10 procs max_latency=2997.184 ms 2024-03-21T11:49:18.490 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T11:49:18.491 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T11:49:19.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:19 smithi047 ceph-mon[31351]: pgmap v1180: 129 pgs: 129 active+clean; 741 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 528 op/s 2024-03-21T11:49:19.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:19 smithi100 ceph-mon[36245]: pgmap v1180: 129 pgs: 129 active+clean; 741 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 528 op/s 2024-03-21T11:49:19.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:19 smithi102 ceph-mon[36748]: pgmap v1180: 129 pgs: 129 active+clean; 741 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.7 MiB/s wr, 528 op/s 2024-03-21T11:49:20.695 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-03-21T11:49:20.695 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-21T11:49:21.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:21 smithi047 ceph-mon[31351]: pgmap v1181: 129 pgs: 129 active+clean; 738 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.7 MiB/s wr, 507 op/s 2024-03-21T11:49:21.642 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:21 smithi100 ceph-mon[36245]: pgmap v1181: 129 pgs: 129 active+clean; 738 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.7 MiB/s wr, 507 op/s 2024-03-21T11:49:21.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:21 smithi102 ceph-mon[36748]: pgmap v1181: 129 pgs: 129 active+clean; 738 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.7 MiB/s wr, 507 op/s 2024-03-21T11:49:21.835 DEBUG:teuthology.parallel:result is None 2024-03-21T11:49:21.836 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T11:49:21.921 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T11:49:21.921 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-21T11:49:21.931 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T11:49:21.948 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-21T11:49:21.979 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-21T11:49:22.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:22 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:49:22.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:22 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:49:22.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:22 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:49:23.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:23 smithi047 ceph-mon[31351]: pgmap v1182: 129 pgs: 129 active+clean; 718 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 479 op/s 2024-03-21T11:49:23.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:23 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:49:23.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:23 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:49:23.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:23 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:49:23.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:23 smithi100 ceph-mon[36245]: pgmap v1182: 129 pgs: 129 active+clean; 718 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 479 op/s 2024-03-21T11:49:23.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:23 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:49:23.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:23 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:49:23.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:23 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:49:23.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:23 smithi102 ceph-mon[36748]: pgmap v1182: 129 pgs: 129 active+clean; 718 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 479 op/s 2024-03-21T11:49:23.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:23 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:49:23.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:23 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:49:23.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:23 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:49:24.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:24 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:49:24.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:24 smithi100 ceph-mon[36245]: scrub summary: idle 2024-03-21T11:49:24.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:24 smithi100 ceph-mon[36245]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:49:24.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:24 smithi100 ceph-mon[36245]: pgmap v1183: 129 pgs: 129 active+clean; 606 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.6 MiB/s wr, 518 op/s 2024-03-21T11:49:24.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:24 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:49:24.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:24 smithi102 ceph-mon[36748]: scrub summary: idle 2024-03-21T11:49:24.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:24 smithi102 ceph-mon[36748]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:49:24.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:24 smithi102 ceph-mon[36748]: pgmap v1183: 129 pgs: 129 active+clean; 606 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.6 MiB/s wr, 518 op/s 2024-03-21T11:49:24.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:24 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:49:24.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:24 smithi047 ceph-mon[31351]: scrub summary: idle 2024-03-21T11:49:24.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:24 smithi047 ceph-mon[31351]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T11:49:24.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:24 smithi047 ceph-mon[31351]: pgmap v1183: 129 pgs: 129 active+clean; 606 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.6 MiB/s wr, 518 op/s 2024-03-21T11:49:27.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:26 smithi102 ceph-mon[36748]: pgmap v1184: 129 pgs: 129 active+clean; 600 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-03-21T11:49:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:26 smithi047 ceph-mon[31351]: pgmap v1184: 129 pgs: 129 active+clean; 600 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-03-21T11:49:27.345 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:26 smithi100 ceph-mon[36245]: pgmap v1184: 129 pgs: 129 active+clean; 600 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-03-21T11:49:29.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:28 smithi102 ceph-mon[36748]: pgmap v1185: 129 pgs: 129 active+clean; 604 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-03-21T11:49:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:28 smithi047 ceph-mon[31351]: pgmap v1185: 129 pgs: 129 active+clean; 604 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-03-21T11:49:29.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:28 smithi100 ceph-mon[36245]: pgmap v1185: 129 pgs: 129 active+clean; 604 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-03-21T11:49:31.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:30 smithi102 ceph-mon[36748]: pgmap v1186: 129 pgs: 129 active+clean; 600 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-03-21T11:49:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:30 smithi047 ceph-mon[31351]: pgmap v1186: 129 pgs: 129 active+clean; 600 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-03-21T11:49:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:30 smithi100 ceph-mon[36245]: pgmap v1186: 129 pgs: 129 active+clean; 600 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-03-21T11:49:33.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:32 smithi102 ceph-mon[36748]: pgmap v1187: 129 pgs: 129 active+clean; 591 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2024-03-21T11:49:33.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:32 smithi047 ceph-mon[31351]: pgmap v1187: 129 pgs: 129 active+clean; 591 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2024-03-21T11:49:33.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:32 smithi100 ceph-mon[36245]: pgmap v1187: 129 pgs: 129 active+clean; 591 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2024-03-21T11:49:35.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:34 smithi102 ceph-mon[36748]: pgmap v1188: 129 pgs: 129 active+clean; 570 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 3.0 MiB/s wr, 197 op/s 2024-03-21T11:49:35.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:34 smithi047 ceph-mon[31351]: pgmap v1188: 129 pgs: 129 active+clean; 570 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 3.0 MiB/s wr, 197 op/s 2024-03-21T11:49:35.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:34 smithi100 ceph-mon[36245]: pgmap v1188: 129 pgs: 129 active+clean; 570 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 3.0 MiB/s wr, 197 op/s 2024-03-21T11:49:37.221 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:36 smithi047 ceph-mon[31351]: pgmap v1189: 129 pgs: 129 active+clean; 570 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 2.7 MiB/s wr, 119 op/s 2024-03-21T11:49:37.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:36 smithi102 ceph-mon[36748]: pgmap v1189: 129 pgs: 129 active+clean; 570 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 2.7 MiB/s wr, 119 op/s 2024-03-21T11:49:37.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:36 smithi100 ceph-mon[36245]: pgmap v1189: 129 pgs: 129 active+clean; 570 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 2.7 MiB/s wr, 119 op/s 2024-03-21T11:49:39.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:38 smithi102 ceph-mon[36748]: pgmap v1190: 129 pgs: 129 active+clean; 560 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 3.2 MiB/s wr, 151 op/s 2024-03-21T11:49:39.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:38 smithi047 ceph-mon[31351]: pgmap v1190: 129 pgs: 129 active+clean; 560 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 3.2 MiB/s wr, 151 op/s 2024-03-21T11:49:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:38 smithi100 ceph-mon[36245]: pgmap v1190: 129 pgs: 129 active+clean; 560 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 3.2 MiB/s wr, 151 op/s 2024-03-21T11:49:39.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T11:49:39.499 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T11:49:39.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.759+0000 7f2bd3e00640 1 -- 172.21.15.47:0/2305435494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4103150 msgr2=0x7f2bd41099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:39.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.759+0000 7f2bd3e00640 1 --2- 172.21.15.47:0/2305435494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4103150 0x7f2bd41099e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009ca0 tx=0x7f2bcc02c0f0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.760+0000 7f2bd3e00640 1 -- 172.21.15.47:0/2305435494 shutdown_connections 2024-03-21T11:49:39.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.760+0000 7f2bd3e00640 1 --2- 172.21.15.47:0/2305435494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4103150 0x7f2bd41099e0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.760+0000 7f2bd3e00640 1 --2- 172.21.15.47:0/2305435494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd4102790 0x7f2bd4102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.760+0000 7f2bd3e00640 1 --2- 172.21.15.47:0/2305435494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bd41015a0 0x7f2bd41019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.760+0000 7f2bd3e00640 1 -- 172.21.15.47:0/2305435494 >> 172.21.15.47:0/2305435494 conn(0x7f2bd40fcb10 msgr2=0x7f2bd40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:39.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.760+0000 7f2bd3e00640 1 -- 172.21.15.47:0/2305435494 shutdown_connections 2024-03-21T11:49:39.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.760+0000 7f2bd3e00640 1 -- 172.21.15.47:0/2305435494 wait complete. 2024-03-21T11:49:39.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.761+0000 7f2bd3e00640 1 Processor -- start 2024-03-21T11:49:39.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.761+0000 7f2bd3e00640 1 -- start start 2024-03-21T11:49:39.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.761+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd41015a0 0x7f2bd4199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.761+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bd4102790 0x7f2bd4199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.761+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4103150 0x7f2bd41a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.761+0000 7f2bd3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd410d230 con 0x7f2bd4102790 2024-03-21T11:49:39.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.761+0000 7f2bd3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd410d0b0 con 0x7f2bd41015a0 2024-03-21T11:49:39.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.761+0000 7f2bd3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd410d3b0 con 0x7f2bd4103150 2024-03-21T11:49:39.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.762+0000 7f2bca000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bd4102790 0x7f2bd4199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.762+0000 7f2bd3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4103150 0x7f2bd41a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.762+0000 7f2bca000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bd4102790 0x7f2bd4199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45508/0 (socket says 172.21.15.47:45508) 2024-03-21T11:49:39.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.762+0000 7f2bca000640 1 -- 172.21.15.47:0/4197253229 learned_addr learned my addr 172.21.15.47:0/4197253229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:39.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.762+0000 7f2bd2a00640 1 --2- 172.21.15.47:0/4197253229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd41015a0 0x7f2bd4199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.762+0000 7f2bd3400640 1 -- 172.21.15.47:0/4197253229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd41015a0 msgr2=0x7f2bd4199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:39.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.762+0000 7f2bd3400640 1 --2- 172.21.15.47:0/4197253229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd41015a0 0x7f2bd4199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.762+0000 7f2bd3400640 1 -- 172.21.15.47:0/4197253229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bd4102790 msgr2=0x7f2bd4199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:39.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.762+0000 7f2bd3400640 1 --2- 172.21.15.47:0/4197253229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bd4102790 0x7f2bd4199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.762+0000 7f2bd3400640 1 -- 172.21.15.47:0/4197253229 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd41a1640 con 0x7f2bd4103150 2024-03-21T11:49:39.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.763+0000 7f2bd3400640 1 --2- 172.21.15.47:0/4197253229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4103150 0x7f2bd41a0fd0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc037040 tx=0x7f2bcc032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:39.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.763+0000 7f2bcbe00640 1 -- 172.21.15.47:0/4197253229 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc041020 con 0x7f2bd4103150 2024-03-21T11:49:39.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.763+0000 7f2bcbe00640 1 -- 172.21.15.47:0/4197253229 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bcc034cd0 con 0x7f2bd4103150 2024-03-21T11:49:39.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.763+0000 7f2bd3e00640 1 -- 172.21.15.47:0/4197253229 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd41a18d0 con 0x7f2bd4103150 2024-03-21T11:49:39.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.763+0000 7f2bcbe00640 1 -- 172.21.15.47:0/4197253229 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc039040 con 0x7f2bd4103150 2024-03-21T11:49:39.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.763+0000 7f2bd3e00640 1 -- 172.21.15.47:0/4197253229 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd41a1de0 con 0x7f2bd4103150 2024-03-21T11:49:39.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.767+0000 7f2bcbe00640 1 -- 172.21.15.47:0/4197253229 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2bcc044020 con 0x7f2bd4103150 2024-03-21T11:49:39.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.768+0000 7f2bdc58a740 1 Processor -- start 2024-03-21T11:49:39.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.768+0000 7f2bdc58a740 1 -- start start 2024-03-21T11:49:39.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.768+0000 7f2bdc58a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316050 0x563f79316470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.768+0000 7f2bdc58a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f79316a40 0x563f7931b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.768+0000 7f2bdc58a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f7931bbb0 0x563f7931e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.768+0000 7f2bdc58a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7927b250 con 0x563f7931bbb0 2024-03-21T11:49:39.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.768+0000 7f2bdc58a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f79223a30 con 0x563f79316a40 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.768+0000 7f2bdc58a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f792772f0 con 0x563f79316050 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.769+0000 7f2bcbe00640 1 --2- 172.21.15.47:0/4197253229 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2bb80a2fa0 0x7f2bb80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.769+0000 7f2bca000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316050 0x563f79316470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.769+0000 7f2bd2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f79316a40 0x563f7931b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.769+0000 7f2bca000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316050 0x563f79316470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42560/0 (socket says 172.21.15.47:42560) 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.769+0000 7f2bd2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f79316a40 0x563f7931b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33784/0 (socket says 172.21.15.47:33784) 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.769+0000 7f2bca000640 1 -- 172.21.15.47:0/1827814177 learned_addr learned my addr 172.21.15.47:0/1827814177 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.769+0000 7f2bca000640 1 --2- 172.21.15.47:0/1827814177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f7931bbb0 0x563f7931e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.769+0000 7f2bcbe00640 1 -- 172.21.15.47:0/4197253229 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2bcc0f14f0 con 0x7f2bd4103150 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.769+0000 7f2bd3400640 1 --2- 172.21.15.47:0/4197253229 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2bb80a2fa0 0x7f2bb80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.769+0000 7f2bd2a00640 1 -- 172.21.15.47:0/1827814177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316050 msgr2=0x563f79316470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.769+0000 7f2bd2a00640 1 --2- 172.21.15.47:0/1827814177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316050 0x563f79316470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.769+0000 7f2bd2a00640 1 -- 172.21.15.47:0/1827814177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f7931bbb0 msgr2=0x563f7931e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.769+0000 7f2bd2a00640 1 --2- 172.21.15.47:0/1827814177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f7931bbb0 0x563f7931e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.769+0000 7f2bd2a00640 1 -- 172.21.15.47:0/1827814177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f7920e690 con 0x563f79316a40 2024-03-21T11:49:39.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.770+0000 7f2bd2a00640 1 --2- 172.21.15.47:0/1827814177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f79316a40 0x563f7931b540 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009bf0 tx=0x7f2bc4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea3209a74d6b01a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.770+0000 7f2bc3e00640 1 -- 172.21.15.47:0/1827814177 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4009da0 con 0x563f79316a40 2024-03-21T11:49:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.770+0000 7f2bc3e00640 1 -- 172.21.15.47:0/1827814177 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bc4005020 con 0x563f79316a40 2024-03-21T11:49:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.770+0000 7f2bd3400640 1 --2- 172.21.15.47:0/4197253229 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2bb80a2fa0 0x7f2bb80a5460 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0b9d20 tx=0x7f2bcc0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.770+0000 7f2bc3e00640 1 -- 172.21.15.47:0/1827814177 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4004b40 con 0x563f79316a40 2024-03-21T11:49:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.771+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1827814177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f79316a40 msgr2=0x563f7931b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.771+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/1827814177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f79316a40 0x563f7931b540 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009bf0 tx=0x7f2bc4011810 comp rx=0 tx=0).stop 2024-03-21T11:49:39.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.771+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1827814177 shutdown_connections 2024-03-21T11:49:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.771+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/1827814177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f7931bbb0 0x563f7931e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.771+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/1827814177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f79316a40 0x563f7931b540 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.771+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/1827814177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316050 0x563f79316470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.771+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1827814177 >> 172.21.15.47:0/1827814177 conn(0x563f792de200 msgr2=0x563f793119c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.771+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1827814177 shutdown_connections 2024-03-21T11:49:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.771+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1827814177 wait complete. 2024-03-21T11:49:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.772+0000 7f2bdc58a740 1 -- start start 2024-03-21T11:49:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.772+0000 7f2bdc58a740 1 Processor -- start 2024-03-21T11:49:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.772+0000 7f2bdc58a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316050 0x563f79320000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.772+0000 7f2bdc58a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316a40 0x563f79320540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.772+0000 7f2bdc58a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f7931bbb0 0x563f79327580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.772+0000 7f2bdc58a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7931eb60 con 0x563f79316050 2024-03-21T11:49:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.772+0000 7f2bdc58a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f79236e80 con 0x563f7931bbb0 2024-03-21T11:49:39.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.772+0000 7f2bdc58a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7931ece0 con 0x563f79316a40 2024-03-21T11:49:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.772+0000 7f2bdc58a740 10 client.0 _ll_register_callbacks cb 0x563f79249130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.773+0000 7f2bd2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316a40 0x563f79320540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.773+0000 7f2bd2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316a40 0x563f79320540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42564/0 (socket says 172.21.15.47:42564) 2024-03-21T11:49:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.773+0000 7f2bd2a00640 1 -- 172.21.15.47:0/3416866740 learned_addr learned my addr 172.21.15.47:0/3416866740 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.773+0000 7f2bca000640 1 --2- 172.21.15.47:0/3416866740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316050 0x563f79320000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.773+0000 7f2bca000640 1 --2- 172.21.15.47:0/3416866740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f7931bbb0 0x563f79327580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.773+0000 7f2bd2a00640 1 -- 172.21.15.47:0/3416866740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f7931bbb0 msgr2=0x563f79327580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.773+0000 7f2bd2a00640 1 --2- 172.21.15.47:0/3416866740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f7931bbb0 0x563f79327580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.773+0000 7f2bd2a00640 1 -- 172.21.15.47:0/3416866740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316050 msgr2=0x563f79320000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.773+0000 7f2bd2a00640 1 --2- 172.21.15.47:0/3416866740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316050 0x563f79320000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.773+0000 7f2bd2a00640 1 -- 172.21.15.47:0/3416866740 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f790597d0 con 0x563f79316a40 2024-03-21T11:49:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.773+0000 7f2bca000640 1 --2- 172.21.15.47:0/3416866740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f7931bbb0 0x563f79327580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:39.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.774+0000 7f2bd2a00640 1 --2- 172.21.15.47:0/3416866740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316a40 0x563f79320540 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f2bc401b040 tx=0x7f2bc4009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ea6f080c955f692 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.774+0000 7f2bc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:49:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.774+0000 7f2bc0c00640 1 -- 172.21.15.47:0/3416866740 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4029020 con 0x563f79316a40 2024-03-21T11:49:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.774+0000 7f2bc0c00640 1 -- 172.21.15.47:0/3416866740 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2bc4004ab0 con 0x563f79316a40 2024-03-21T11:49:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.774+0000 7f2bdc58a740 1 -- 172.21.15.47:0/3416866740 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bc4005020 con 0x563f79316a40 2024-03-21T11:49:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.774+0000 7f2bc0c00640 1 -- 172.21.15.47:0/3416866740 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bc4034050 con 0x563f79316a40 2024-03-21T11:49:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.774+0000 7f2bdc58a740 10 client.33625 fetch_fsmap learned FSMap version 43 2024-03-21T11:49:39.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.774+0000 7f2bdc58a740 1 -- 172.21.15.47:0/3416866740 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x563f78f8f3f0 con 0x563f79316a40 2024-03-21T11:49:39.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.775+0000 7f2bc0c00640 1 -- 172.21.15.47:0/3416866740 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bc4005020 con 0x563f79316a40 2024-03-21T11:49:39.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.775+0000 7f2bc0c00640 1 -- 172.21.15.47:0/3416866740 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 43) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f2bc4019450 con 0x563f79316a40 2024-03-21T11:49:39.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.775+0000 7f2bdc58a740 10 client.33625 fetch_fsmap finished waiting for FSMap version 43 2024-03-21T11:49:39.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.775+0000 7f2bdc58a740 10 client.33625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:49:39.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.775+0000 7f2bdc58a740 20 client.33625 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:39.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.775+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/3416866740 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563f7932bc20 0x563f7932e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.775+0000 7f2bdc58a740 4 client.33625 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:39.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.775+0000 7f2bdc58a740 1 -- 172.21.15.47:0/3416866740 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f79288790 con 0x563f7932bc20 2024-03-21T11:49:39.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.775+0000 7f2bca000640 1 --2- 172.21.15.47:0/3416866740 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563f7932bc20 0x563f7932e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.776+0000 7f2bca000640 1 --2- 172.21.15.47:0/3416866740 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563f7932bc20 0x563f7932e080 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a93c4401423ae612 server_cookie=df1c898b98a5dbd1 in_seq=0 out_seq=0 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.776+0000 7f2bc0c00640 10 client.33625 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bc0c00640 1 -- 172.21.15.47:0/3416866740 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563f79288790 con 0x563f7932bc20 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bc0c00640 10 client.33625 handle_command_reply: tid=0 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bdc58a740 1 client.33625 shutdown 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bdc58a740 1 -- 172.21.15.47:0/3416866740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316a40 msgr2=0x563f79320540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/3416866740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316a40 0x563f79320540 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f2bc401b040 tx=0x7f2bc4009fd0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bdc58a740 1 -- 172.21.15.47:0/3416866740 shutdown_connections 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/3416866740 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563f7932bc20 0x563f7932e080 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/3416866740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f7931bbb0 0x563f79327580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/3416866740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316a40 0x563f79320540 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/3416866740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316050 0x563f79320000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bc0c00640 0 client.33625 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bdc58a740 1 -- 172.21.15.47:0/3416866740 >> 172.21.15.47:0/3416866740 conn(0x563f792de200 msgr2=0x563f79310bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bdc58a740 1 -- 172.21.15.47:0/3416866740 shutdown_connections 2024-03-21T11:49:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bdc58a740 1 -- 172.21.15.47:0/3416866740 wait complete. 2024-03-21T11:49:39.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.777+0000 7f2bdc58a740 20 client.33625 trim_cache size 0 max 16384 2024-03-21T11:49:39.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.797+0000 7f2bdc58a740 1 Processor -- start 2024-03-21T11:49:39.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.797+0000 7f2bdc58a740 1 -- start start 2024-03-21T11:49:39.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.797+0000 7f2bdc58a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f79316050 0x563f79317bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.797+0000 7f2bdc58a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316a40 0x563f79327540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.797+0000 7f2bdc58a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f7931bbb0 0x563f79314c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.797+0000 7f2bdc58a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd41ac060 con 0x563f79316a40 2024-03-21T11:49:39.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.797+0000 7f2bdc58a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd41adad0 con 0x563f79316050 2024-03-21T11:49:39.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.797+0000 7f2bdc58a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd41ac720 con 0x563f7931bbb0 2024-03-21T11:49:39.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.798+0000 7f2bca000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f79316050 0x563f79317bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.798+0000 7f2bd2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316a40 0x563f79327540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.798+0000 7f2bca000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f79316050 0x563f79317bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33796/0 (socket says 172.21.15.47:33796) 2024-03-21T11:49:39.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.798+0000 7f2bca000640 1 -- 172.21.15.47:0/1376779686 learned_addr learned my addr 172.21.15.47:0/1376779686 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:39.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.798+0000 7f2bca000640 1 --2- 172.21.15.47:0/1376779686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f7931bbb0 0x563f79314c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.798+0000 7f2bca000640 1 -- 172.21.15.47:0/1376779686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f7931bbb0 msgr2=0x563f79314c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:39.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.798+0000 7f2bca000640 1 --2- 172.21.15.47:0/1376779686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f7931bbb0 0x563f79314c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.798+0000 7f2bca000640 1 -- 172.21.15.47:0/1376779686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316a40 msgr2=0x563f79327540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:39.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.798+0000 7f2bca000640 1 --2- 172.21.15.47:0/1376779686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316a40 0x563f79327540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.798+0000 7f2bca000640 1 -- 172.21.15.47:0/1376779686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f78f4f0f0 con 0x563f79316050 2024-03-21T11:49:39.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.798+0000 7f2bd2a00640 1 --2- 172.21.15.47:0/1376779686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316a40 0x563f79327540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:39.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.798+0000 7f2bca000640 1 --2- 172.21.15.47:0/1376779686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f79316050 0x563f79317bc0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009fd0 tx=0x7f2bbc009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92804d1a008cf955 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:39.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.799+0000 7f2bc3400640 1 -- 172.21.15.47:0/1376779686 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc00f790 con 0x563f79316050 2024-03-21T11:49:39.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.799+0000 7f2bc3400640 1 -- 172.21.15.47:0/1376779686 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bbc00fe10 con 0x563f79316050 2024-03-21T11:49:39.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.799+0000 7f2bc3400640 1 -- 172.21.15.47:0/1376779686 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc030b80 con 0x563f79316050 2024-03-21T11:49:39.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.800+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1376779686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f79316050 msgr2=0x563f79317bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:39.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.800+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/1376779686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f79316050 0x563f79317bc0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009fd0 tx=0x7f2bbc009c40 comp rx=0 tx=0).stop 2024-03-21T11:49:39.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.800+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1376779686 shutdown_connections 2024-03-21T11:49:39.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.800+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/1376779686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f7931bbb0 0x563f79314c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.800+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/1376779686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316a40 0x563f79327540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.800+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/1376779686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f79316050 0x563f79317bc0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.800+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1376779686 >> 172.21.15.47:0/1376779686 conn(0x563f792de200 msgr2=0x563f793174b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:39.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.800+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1376779686 shutdown_connections 2024-03-21T11:49:39.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.800+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1376779686 wait complete. 2024-03-21T11:49:39.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.801+0000 7f2bdc58a740 1 -- start start 2024-03-21T11:49:39.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.801+0000 7f2bdc58a740 1 Processor -- start 2024-03-21T11:49:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.801+0000 7f2bdc58a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316050 0x563f79325060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.801+0000 7f2bdc58a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316a40 0x563f793255a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.801+0000 7f2bdc58a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f7931bbb0 0x563f793265b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.801+0000 7f2bdc58a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd41ad470 con 0x563f79316050 2024-03-21T11:49:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.801+0000 7f2bdc58a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd41acde0 con 0x563f7931bbb0 2024-03-21T11:49:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.801+0000 7f2bdc58a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd41aa220 con 0x563f79316a40 2024-03-21T11:49:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.801+0000 7f2bdc58a740 10 client.0 _ll_register_callbacks cb 0x7f2bbc009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.802+0000 7f2bd2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316a40 0x563f793255a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.802+0000 7f2bd2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316a40 0x563f793255a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42584/0 (socket says 172.21.15.47:42584) 2024-03-21T11:49:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.802+0000 7f2bd2a00640 1 -- 172.21.15.47:0/1927424227 learned_addr learned my addr 172.21.15.47:0/1927424227 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.802+0000 7f2bca000640 1 --2- 172.21.15.47:0/1927424227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316050 0x563f79325060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.802+0000 7f2bca000640 1 --2- 172.21.15.47:0/1927424227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f7931bbb0 0x563f793265b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.802+0000 7f2bd2a00640 1 -- 172.21.15.47:0/1927424227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f7931bbb0 msgr2=0x563f793265b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.802+0000 7f2bd2a00640 1 --2- 172.21.15.47:0/1927424227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f7931bbb0 0x563f793265b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.802+0000 7f2bd2a00640 1 -- 172.21.15.47:0/1927424227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316050 msgr2=0x563f79325060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.802+0000 7f2bd2a00640 1 --2- 172.21.15.47:0/1927424227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316050 0x563f79325060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.802+0000 7f2bd2a00640 1 -- 172.21.15.47:0/1927424227 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f79326ba0 con 0x563f79316a40 2024-03-21T11:49:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.802+0000 7f2bca000640 1 --2- 172.21.15.47:0/1927424227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f7931bbb0 0x563f793265b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.802+0000 7f2bd2a00640 1 --2- 172.21.15.47:0/1927424227 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316a40 0x563f793255a0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4000c00 tx=0x7f2bc4009ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56a86490382462c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.802+0000 7f2bc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:49:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.802+0000 7f2bc0c00640 1 -- 172.21.15.47:0/1927424227 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4029040 con 0x563f79316a40 2024-03-21T11:49:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.803+0000 7f2bc0c00640 1 -- 172.21.15.47:0/1927424227 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f2bc4019ab0 con 0x563f79316a40 2024-03-21T11:49:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.803+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1927424227 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bbc00fe10 con 0x563f79316a40 2024-03-21T11:49:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.803+0000 7f2bc0c00640 1 -- 172.21.15.47:0/1927424227 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bc4019650 con 0x563f79316a40 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.803+0000 7f2bdc58a740 10 client.33631 fetch_fsmap learned FSMap version 43 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.803+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1927424227 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x563f79387920 con 0x563f79316a40 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.803+0000 7f2bc0c00640 1 -- 172.21.15.47:0/1927424227 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bbc00fe10 con 0x563f79316a40 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.803+0000 7f2bc0c00640 1 -- 172.21.15.47:0/1927424227 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 43) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f2bc40035f0 con 0x563f79316a40 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.803+0000 7f2bdc58a740 10 client.33631 fetch_fsmap finished waiting for FSMap version 43 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.803+0000 7f2bdc58a740 10 client.33631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.803+0000 7f2bdc58a740 20 client.33631 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.803+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/1927424227 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563f79388290 0x563f7938a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.803+0000 7f2bdc58a740 4 client.33631 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.803+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1927424227 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563f79282ae0 con 0x563f79388290 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.804+0000 7f2bca000640 1 --2- 172.21.15.47:0/1927424227 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563f79388290 0x563f7938a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.804+0000 7f2bca000640 1 --2- 172.21.15.47:0/1927424227 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563f79388290 0x563f7938a6b0 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eac4754864d7eb0 server_cookie=eada97253e8b62bb in_seq=0 out_seq=0 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.804+0000 7f2bc0c00640 10 client.33631 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.804+0000 7f2bc0c00640 1 -- 172.21.15.47:0/1927424227 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563f79282ae0 con 0x563f79388290 2024-03-21T11:49:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.804+0000 7f2bc0c00640 10 client.33631 handle_command_reply: tid=0 2024-03-21T11:49:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.804+0000 7f2bdc58a740 1 client.33631 shutdown 2024-03-21T11:49:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.804+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1927424227 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316a40 msgr2=0x563f793255a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.804+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/1927424227 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316a40 0x563f793255a0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4000c00 tx=0x7f2bc4009ae0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.805+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1927424227 shutdown_connections 2024-03-21T11:49:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.805+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/1927424227 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x563f79388290 0x563f7938a6b0 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.805+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/1927424227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f7931bbb0 0x563f793265b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.805+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/1927424227 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563f79316a40 0x563f793255a0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.805+0000 7f2bc0c00640 0 client.33631 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.805+0000 7f2bdc58a740 1 --2- 172.21.15.47:0/1927424227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f79316050 0x563f79325060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.805+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1927424227 >> 172.21.15.47:0/1927424227 conn(0x563f792de200 msgr2=0x563f79317c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.805+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1927424227 shutdown_connections 2024-03-21T11:49:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.805+0000 7f2bdc58a740 1 -- 172.21.15.47:0/1927424227 wait complete. 2024-03-21T11:49:39.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.805+0000 7f2bdc58a740 20 client.33631 trim_cache size 0 max 16384 2024-03-21T11:49:39.811 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:49:39.811 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T11:49:39.811 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T11:49:39.811 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:49:39.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.806+0000 7f2bd3e00640 1 -- 172.21.15.47:0/4197253229 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2bb80a2fa0 msgr2=0x7f2bb80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:39.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.806+0000 7f2bd3e00640 1 --2- 172.21.15.47:0/4197253229 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2bb80a2fa0 0x7f2bb80a5460 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0b9d20 tx=0x7f2bcc0095c0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.806+0000 7f2bd3e00640 1 -- 172.21.15.47:0/4197253229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4103150 msgr2=0x7f2bd41a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:39.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.806+0000 7f2bd3e00640 1 --2- 172.21.15.47:0/4197253229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4103150 0x7f2bd41a0fd0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc037040 tx=0x7f2bcc032ea0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.806+0000 7f2bd3e00640 1 -- 172.21.15.47:0/4197253229 shutdown_connections 2024-03-21T11:49:39.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.806+0000 7f2bd3e00640 1 --2- 172.21.15.47:0/4197253229 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f2bb80a2fa0 0x7f2bb80a5460 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.806+0000 7f2bd3e00640 1 --2- 172.21.15.47:0/4197253229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4103150 0x7f2bd41a0fd0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.806+0000 7f2bd3e00640 1 --2- 172.21.15.47:0/4197253229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bd4102790 0x7f2bd4199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.806+0000 7f2bd3e00640 1 --2- 172.21.15.47:0/4197253229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bd41015a0 0x7f2bd4199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:39.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.806+0000 7f2bd3e00640 1 -- 172.21.15.47:0/4197253229 >> 172.21.15.47:0/4197253229 conn(0x7f2bd40fcb10 msgr2=0x7f2bd40738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:39.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.806+0000 7f2bd3e00640 1 -- 172.21.15.47:0/4197253229 shutdown_connections 2024-03-21T11:49:39.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:39.806+0000 7f2bd3e00640 1 -- 172.21.15.47:0/4197253229 wait complete. 2024-03-21T11:49:39.822 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T11:49:39.823 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:49:40.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.006+0000 7f0c80000640 1 -- 172.21.15.47:0/1335783069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c781028b0 msgr2=0x7f0c78102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.006+0000 7f0c80000640 1 --2- 172.21.15.47:0/1335783069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c781028b0 0x7f0c78102d30 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009900 tx=0x7f0c6402bc30 comp rx=0 tx=0).stop 2024-03-21T11:49:40.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.006+0000 7f0c80000640 1 -- 172.21.15.47:0/1335783069 shutdown_connections 2024-03-21T11:49:40.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.006+0000 7f0c80000640 1 --2- 172.21.15.47:0/1335783069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c78103270 0x7f0c78109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.006+0000 7f0c80000640 1 --2- 172.21.15.47:0/1335783069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c781028b0 0x7f0c78102d30 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.006+0000 7f0c80000640 1 --2- 172.21.15.47:0/1335783069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c781016c0 0x7f0c78101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.006+0000 7f0c80000640 1 -- 172.21.15.47:0/1335783069 >> 172.21.15.47:0/1335783069 conn(0x7f0c780fcc50 msgr2=0x7f0c780ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.006+0000 7f0c80000640 1 -- 172.21.15.47:0/1335783069 shutdown_connections 2024-03-21T11:49:40.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.006+0000 7f0c80000640 1 -- 172.21.15.47:0/1335783069 wait complete. 2024-03-21T11:49:40.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.007+0000 7f0c80000640 1 Processor -- start 2024-03-21T11:49:40.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.007+0000 7f0c80000640 1 -- start start 2024-03-21T11:49:40.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c80000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c781016c0 0x7f0c78075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c80000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c781028b0 0x7f0c78073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c80000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c78103270 0x7f0c78073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c80000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7810d390 con 0x7f0c78103270 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c80000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7810d210 con 0x7f0c781028b0 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c80000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7810d510 con 0x7f0c781016c0 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c7cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c781028b0 0x7f0c78073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c7d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c781016c0 0x7f0c78075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c7cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c781028b0 0x7f0c78073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33826/0 (socket says 172.21.15.47:33826) 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c7cc00640 1 -- 172.21.15.47:0/980087251 learned_addr learned my addr 172.21.15.47:0/980087251 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c7e000640 1 --2- 172.21.15.47:0/980087251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c78103270 0x7f0c78073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c7cc00640 1 -- 172.21.15.47:0/980087251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c781016c0 msgr2=0x7f0c78075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c7cc00640 1 --2- 172.21.15.47:0/980087251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c781016c0 0x7f0c78075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c7cc00640 1 -- 172.21.15.47:0/980087251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c78103270 msgr2=0x7f0c78073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c7cc00640 1 --2- 172.21.15.47:0/980087251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c78103270 0x7f0c78073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.008+0000 7f0c7cc00640 1 -- 172.21.15.47:0/980087251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c78074860 con 0x7f0c781028b0 2024-03-21T11:49:40.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.009+0000 7f0c7cc00640 1 --2- 172.21.15.47:0/980087251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c781028b0 0x7f0c78073a90 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f0c64000c00 tx=0x7f0c6402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.009+0000 7f0c76000640 1 -- 172.21.15.47:0/980087251 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64043020 con 0x7f0c781028b0 2024-03-21T11:49:40.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.009+0000 7f0c76000640 1 -- 172.21.15.47:0/980087251 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c640054e0 con 0x7f0c781028b0 2024-03-21T11:49:40.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.009+0000 7f0c80000640 1 -- 172.21.15.47:0/980087251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c781a1aa0 con 0x7f0c781028b0 2024-03-21T11:49:40.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.009+0000 7f0c76000640 1 -- 172.21.15.47:0/980087251 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64003c40 con 0x7f0c781028b0 2024-03-21T11:49:40.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.009+0000 7f0c80000640 1 -- 172.21.15.47:0/980087251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c781a1ff0 con 0x7f0c781028b0 2024-03-21T11:49:40.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.010+0000 7f0c80000640 1 -- 172.21.15.47:0/980087251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c78108980 con 0x7f0c781028b0 2024-03-21T11:49:40.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.012+0000 7f0c76000640 1 -- 172.21.15.47:0/980087251 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0c64046020 con 0x7f0c781028b0 2024-03-21T11:49:40.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.013+0000 7f0c76000640 1 --2- 172.21.15.47:0/980087251 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c5c0a2f20 0x7f0c5c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.013+0000 7f0c76000640 1 -- 172.21.15.47:0/980087251 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0c640f18b0 con 0x7f0c781028b0 2024-03-21T11:49:40.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.014+0000 7f0c7d600640 1 --2- 172.21.15.47:0/980087251 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c5c0a2f20 0x7f0c5c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.014+0000 7f0c7d600640 1 --2- 172.21.15.47:0/980087251 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c5c0a2f20 0x7f0c5c0a53e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f0c78102710 tx=0x7f0c70009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.017+0000 7f0c76000640 1 -- 172.21.15.47:0/980087251 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0c6404d020 con 0x7f0c781028b0 2024-03-21T11:49:40.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:39 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:49:40.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.255+0000 7f0c80000640 1 -- 172.21.15.47:0/980087251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c7802d070 con 0x7f0c781028b0 2024-03-21T11:49:40.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.256+0000 7f0c76000640 1 -- 172.21.15.47:0/980087251 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+11481 (secure 0 0 0) 0x7f0c640be2f0 con 0x7f0c781028b0 2024-03-21T11:49:40.257 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:49:40.258 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:49:23.191513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_32623":{"gid":32623,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4278877119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4278877119},{"type":"v1","addr":"172.21.15.100:6835","nonce":4278877119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24524,24523,24535,24581,24553]},"id":1}]} 2024-03-21T11:49:40.258 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 43 2024-03-21T11:49:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.259+0000 7f0c80000640 1 -- 172.21.15.47:0/980087251 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c5c0a2f20 msgr2=0x7f0c5c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.259+0000 7f0c80000640 1 --2- 172.21.15.47:0/980087251 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c5c0a2f20 0x7f0c5c0a53e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f0c78102710 tx=0x7f0c70009500 comp rx=0 tx=0).stop 2024-03-21T11:49:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.259+0000 7f0c80000640 1 -- 172.21.15.47:0/980087251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c781028b0 msgr2=0x7f0c78073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.259+0000 7f0c80000640 1 --2- 172.21.15.47:0/980087251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c781028b0 0x7f0c78073a90 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f0c64000c00 tx=0x7f0c6402c650 comp rx=0 tx=0).stop 2024-03-21T11:49:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.259+0000 7f0c80000640 1 -- 172.21.15.47:0/980087251 shutdown_connections 2024-03-21T11:49:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.259+0000 7f0c80000640 1 --2- 172.21.15.47:0/980087251 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0c5c0a2f20 0x7f0c5c0a53e0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.259+0000 7f0c80000640 1 --2- 172.21.15.47:0/980087251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c78103270 0x7f0c78073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.259+0000 7f0c80000640 1 --2- 172.21.15.47:0/980087251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c781028b0 0x7f0c78073a90 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.259+0000 7f0c80000640 1 --2- 172.21.15.47:0/980087251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c781016c0 0x7f0c78075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.259+0000 7f0c80000640 1 -- 172.21.15.47:0/980087251 >> 172.21.15.47:0/980087251 conn(0x7f0c780fcc50 msgr2=0x7f0c78105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.260+0000 7f0c80000640 1 -- 172.21.15.47:0/980087251 shutdown_connections 2024-03-21T11:49:40.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.260+0000 7f0c80000640 1 -- 172.21.15.47:0/980087251 wait complete. 2024-03-21T11:49:40.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:39 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:49:40.277 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:49:40.277 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T11:49:40.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:39 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T11:49:40.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.477+0000 7f36eb200640 1 -- 172.21.15.47:0/3540443040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ec109460 msgr2=0x7f36ec10b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.477+0000 7f36eb200640 1 --2- 172.21.15.47:0/3540443040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ec109460 0x7f36ec10b8f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009ca0 tx=0x7f36e402c100 comp rx=0 tx=0).stop 2024-03-21T11:49:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.477+0000 7f36eb200640 1 -- 172.21.15.47:0/3540443040 shutdown_connections 2024-03-21T11:49:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.477+0000 7f36eb200640 1 --2- 172.21.15.47:0/3540443040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ec109460 0x7f36ec10b8f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.477+0000 7f36eb200640 1 --2- 172.21.15.47:0/3540443040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36ec074100 0x7f36ec074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.477+0000 7f36eb200640 1 --2- 172.21.15.47:0/3540443040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36ec0756e0 0x7f36ec073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.477+0000 7f36eb200640 1 -- 172.21.15.47:0/3540443040 >> 172.21.15.47:0/3540443040 conn(0x7f36ec0fcb10 msgr2=0x7f36ec0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.477+0000 7f36eb200640 1 -- 172.21.15.47:0/3540443040 shutdown_connections 2024-03-21T11:49:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.477+0000 7f36eb200640 1 -- 172.21.15.47:0/3540443040 wait complete. 2024-03-21T11:49:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.478+0000 7f36eb200640 1 Processor -- start 2024-03-21T11:49:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.479+0000 7f36eb200640 1 -- start start 2024-03-21T11:49:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.479+0000 7f36eb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ec074100 0x7f36ec1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.479+0000 7f36eb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36ec0756e0 0x7f36ec199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.479+0000 7f36eb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36ec109460 0x7f36ec1a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.479+0000 7f36eb200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec10f3e0 con 0x7f36ec074100 2024-03-21T11:49:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.479+0000 7f36eb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec10f260 con 0x7f36ec0756e0 2024-03-21T11:49:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.479+0000 7f36eb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec10f560 con 0x7f36ec109460 2024-03-21T11:49:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.479+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ec074100 0x7f36ec1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.479+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ec074100 0x7f36ec1999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45592/0 (socket says 172.21.15.47:45592) 2024-03-21T11:49:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.479+0000 7f36e9e00640 1 -- 172.21.15.47:0/3619684511 learned_addr learned my addr 172.21.15.47:0/3619684511 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.479+0000 7f36ea800640 1 --2- 172.21.15.47:0/3619684511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36ec109460 0x7f36ec1a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.479+0000 7f36e9400640 1 --2- 172.21.15.47:0/3619684511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36ec0756e0 0x7f36ec199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.480+0000 7f36e9e00640 1 -- 172.21.15.47:0/3619684511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36ec109460 msgr2=0x7f36ec1a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.480+0000 7f36e9e00640 1 --2- 172.21.15.47:0/3619684511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36ec109460 0x7f36ec1a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.480+0000 7f36e9e00640 1 -- 172.21.15.47:0/3619684511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36ec0756e0 msgr2=0x7f36ec199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.480+0000 7f36e9e00640 1 --2- 172.21.15.47:0/3619684511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36ec0756e0 0x7f36ec199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.480+0000 7f36e9e00640 1 -- 172.21.15.47:0/3619684511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec1a1610 con 0x7f36ec074100 2024-03-21T11:49:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.480+0000 7f36e9e00640 1 --2- 172.21.15.47:0/3619684511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ec074100 0x7f36ec1999f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00e9f0 tx=0x7f36dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.480+0000 7f36e2a00640 1 -- 172.21.15.47:0/3619684511 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc00cd80 con 0x7f36ec074100 2024-03-21T11:49:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.480+0000 7f36eb200640 1 -- 172.21.15.47:0/3619684511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ec1a1900 con 0x7f36ec074100 2024-03-21T11:49:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.481+0000 7f36eb200640 1 -- 172.21.15.47:0/3619684511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ec1a1ea0 con 0x7f36ec074100 2024-03-21T11:49:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.481+0000 7f36e2a00640 1 -- 172.21.15.47:0/3619684511 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36dc010910 con 0x7f36ec074100 2024-03-21T11:49:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.481+0000 7f36e2a00640 1 -- 172.21.15.47:0/3619684511 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc018980 con 0x7f36ec074100 2024-03-21T11:49:40.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.484+0000 7f36f3d7f740 1 Processor -- start 2024-03-21T11:49:40.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.485+0000 7f36f3d7f740 1 -- start start 2024-03-21T11:49:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.485+0000 7f36f3d7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708c9f0 0x564da708ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.485+0000 7f36f3d7f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708dbc0 0x564da708e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.485+0000 7f36f3d7f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da7094e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.485+0000 7f36f3d7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da6ff12c0 con 0x564da708c9f0 2024-03-21T11:49:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.485+0000 7f36f3d7f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da6f99c90 con 0x564da708dbc0 2024-03-21T11:49:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.485+0000 7f36f3d7f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da6fed360 con 0x564da708e580 2024-03-21T11:49:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.485+0000 7f36e9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da7094e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.485+0000 7f36ea800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708c9f0 0x564da708ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.485+0000 7f36e9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da7094e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42622/0 (socket says 172.21.15.47:42622) 2024-03-21T11:49:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.485+0000 7f36e9400640 1 -- 172.21.15.47:0/2333737439 learned_addr learned my addr 172.21.15.47:0/2333737439 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.485+0000 7f36e2a00640 1 -- 172.21.15.47:0/3619684511 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f36dc010ac0 con 0x7f36ec074100 2024-03-21T11:49:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.486+0000 7f36e9400640 1 -- 172.21.15.47:0/2333737439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708dbc0 msgr2=0x564da708e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.486+0000 7f36e9e00640 1 --2- 172.21.15.47:0/2333737439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708dbc0 0x564da708e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.486+0000 7f36e9400640 1 --2- 172.21.15.47:0/2333737439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708dbc0 0x564da708e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.486+0000 7f36e9400640 1 -- 172.21.15.47:0/2333737439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708c9f0 msgr2=0x564da708ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.486+0000 7f36e9400640 1 --2- 172.21.15.47:0/2333737439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708c9f0 0x564da708ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.486+0000 7f36e9400640 1 -- 172.21.15.47:0/2333737439 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564da6fbe080 con 0x564da708e580 2024-03-21T11:49:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.486+0000 7f36e9e00640 1 --2- 172.21.15.47:0/2333737439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708dbc0 0x564da708e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:49:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.486+0000 7f36ea800640 1 --2- 172.21.15.47:0/2333737439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708c9f0 0x564da708ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.486+0000 7f36e9400640 1 --2- 172.21.15.47:0/2333737439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da7094e10 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009c90 tx=0x7f36d8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eef17684a18a5af server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.487+0000 7f36d7400640 1 -- 172.21.15.47:0/2333737439 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d801e070 con 0x564da708e580 2024-03-21T11:49:40.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.487+0000 7f36d7400640 1 -- 172.21.15.47:0/2333737439 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36d8019b10 con 0x564da708e580 2024-03-21T11:49:40.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.487+0000 7f36d7400640 1 -- 172.21.15.47:0/2333737439 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d800eac0 con 0x564da708e580 2024-03-21T11:49:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.487+0000 7f36e2a00640 1 --2- 172.21.15.47:0/3619684511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36d00a2f20 0x7f36d00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.487+0000 7f36e2a00640 1 -- 172.21.15.47:0/3619684511 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f36dc0cfb60 con 0x7f36ec074100 2024-03-21T11:49:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.487+0000 7f36ea800640 1 --2- 172.21.15.47:0/3619684511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36d00a2f20 0x7f36d00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.487+0000 7f36f3d7f740 1 -- 172.21.15.47:0/2333737439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 msgr2=0x564da7094e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.487+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/2333737439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da7094e10 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009c90 tx=0x7f36d8011810 comp rx=0 tx=0).stop 2024-03-21T11:49:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.488+0000 7f36f3d7f740 1 -- 172.21.15.47:0/2333737439 shutdown_connections 2024-03-21T11:49:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.488+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/2333737439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da7094e10 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.488+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/2333737439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708dbc0 0x564da708e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.488+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/2333737439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708c9f0 0x564da708ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.488+0000 7f36f3d7f740 1 -- 172.21.15.47:0/2333737439 >> 172.21.15.47:0/2333737439 conn(0x564da7087de0 msgr2=0x564da70881e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.488+0000 7f36f3d7f740 1 -- 172.21.15.47:0/2333737439 shutdown_connections 2024-03-21T11:49:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.488+0000 7f36f3d7f740 1 -- 172.21.15.47:0/2333737439 wait complete. 2024-03-21T11:49:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.488+0000 7f36ea800640 1 --2- 172.21.15.47:0/3619684511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36d00a2f20 0x7f36d00a53e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f36e402c640 tx=0x7f36e4032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.489+0000 7f36f3d7f740 1 -- start start 2024-03-21T11:49:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.489+0000 7f36f3d7f740 1 Processor -- start 2024-03-21T11:49:40.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.489+0000 7f36f3d7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708c9f0 0x564da7096bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.489+0000 7f36f3d7f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708dbc0 0x564da7097110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.489+0000 7f36f3d7f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da709e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.489+0000 7f36f3d7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da7095590 con 0x564da708c9f0 2024-03-21T11:49:40.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.489+0000 7f36f3d7f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da6fad0d0 con 0x564da708dbc0 2024-03-21T11:49:40.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.489+0000 7f36f3d7f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564da7095710 con 0x564da708e580 2024-03-21T11:49:40.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.490+0000 7f36f3d7f740 10 client.0 _ll_register_callbacks cb 0x564da6fb1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:40.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.490+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708c9f0 0x564da7096bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.490+0000 7f36e9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708dbc0 0x564da7097110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.490+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708c9f0 0x564da7096bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45618/0 (socket says 172.21.15.47:45618) 2024-03-21T11:49:40.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.490+0000 7f36e9e00640 1 -- 172.21.15.47:0/911653093 learned_addr learned my addr 172.21.15.47:0/911653093 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:40.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.490+0000 7f36ea800640 1 --2- 172.21.15.47:0/911653093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da709e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.490+0000 7f36e9400640 1 -- 172.21.15.47:0/911653093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 msgr2=0x564da709e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.490+0000 7f36e9400640 1 --2- 172.21.15.47:0/911653093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da709e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.490+0000 7f36e9400640 1 -- 172.21.15.47:0/911653093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708c9f0 msgr2=0x564da7096bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.490+0000 7f36e9400640 1 --2- 172.21.15.47:0/911653093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708c9f0 0x564da7096bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.490+0000 7f36e9400640 1 -- 172.21.15.47:0/911653093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564da6d053f0 con 0x564da708dbc0 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.490+0000 7f36ea800640 1 --2- 172.21.15.47:0/911653093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da709e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.490+0000 7f36e9e00640 1 --2- 172.21.15.47:0/911653093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708c9f0 0x564da7096bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.491+0000 7f36e9400640 1 --2- 172.21.15.47:0/911653093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708dbc0 0x564da7097110 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x564da7088d50 tx=0x7f36d8019a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c035ec9b27bd750 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.491+0000 7f36cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.491+0000 7f36cbe00640 1 -- 172.21.15.47:0/911653093 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d801e070 con 0x564da708dbc0 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.491+0000 7f36cbe00640 1 -- 172.21.15.47:0/911653093 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f36d8004230 con 0x564da708dbc0 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.491+0000 7f36f3d7f740 1 -- 172.21.15.47:0/911653093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36d8019b10 con 0x564da708dbc0 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.491+0000 7f36cbe00640 1 -- 172.21.15.47:0/911653093 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36d8033050 con 0x564da708dbc0 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.491+0000 7f36f3d7f740 10 client.33740 fetch_fsmap learned FSMap version 43 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.491+0000 7f36f3d7f740 1 -- 172.21.15.47:0/911653093 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x564da6dcf7d0 con 0x564da708dbc0 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.492+0000 7f36cbe00640 1 -- 172.21.15.47:0/911653093 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36d8019b10 con 0x564da708dbc0 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.492+0000 7f36cbe00640 1 -- 172.21.15.47:0/911653093 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 43) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f36d800e850 con 0x564da708dbc0 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.492+0000 7f36f3d7f740 10 client.33740 fetch_fsmap finished waiting for FSMap version 43 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.492+0000 7f36f3d7f740 10 client.33740 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:49:40.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.492+0000 7f36f3d7f740 20 client.33740 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.492+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/911653093 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564da70a23a0 0x564da70a4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.492+0000 7f36f3d7f740 4 client.33740 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.492+0000 7f36f3d7f740 1 -- 172.21.15.47:0/911653093 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564da6ffe800 con 0x564da70a23a0 2024-03-21T11:49:40.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.492+0000 7f36e9e00640 1 --2- 172.21.15.47:0/911653093 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564da70a23a0 0x564da70a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.493+0000 7f36e9e00640 1 --2- 172.21.15.47:0/911653093 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564da70a23a0 0x564da70a4800 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3ffe0af77f0c96fb server_cookie=b1978653df8c93a3 in_seq=0 out_seq=0 2024-03-21T11:49:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.493+0000 7f36cbe00640 10 client.33740 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:49:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36cbe00640 1 -- 172.21.15.47:0/911653093 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564da6ffe800 con 0x564da70a23a0 2024-03-21T11:49:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36cbe00640 10 client.33740 handle_command_reply: tid=0 2024-03-21T11:49:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36f3d7f740 1 client.33740 shutdown 2024-03-21T11:49:40.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36f3d7f740 1 -- 172.21.15.47:0/911653093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708dbc0 msgr2=0x564da7097110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/911653093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708dbc0 0x564da7097110 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x564da7088d50 tx=0x7f36d8019a20 comp rx=0 tx=0).stop 2024-03-21T11:49:40.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36f3d7f740 1 -- 172.21.15.47:0/911653093 shutdown_connections 2024-03-21T11:49:40.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/911653093 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564da70a23a0 0x564da70a4800 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/911653093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da709e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/911653093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708dbc0 0x564da7097110 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/911653093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708c9f0 0x564da7096bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36cbe00640 0 client.33740 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:49:40.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36f3d7f740 1 -- 172.21.15.47:0/911653093 >> 172.21.15.47:0/911653093 conn(0x564da7087de0 msgr2=0x564da7087160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36f3d7f740 1 -- 172.21.15.47:0/911653093 shutdown_connections 2024-03-21T11:49:40.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36f3d7f740 1 -- 172.21.15.47:0/911653093 wait complete. 2024-03-21T11:49:40.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.494+0000 7f36f3d7f740 20 client.33740 trim_cache size 0 max 16384 2024-03-21T11:49:40.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.508+0000 7f36f3d7f740 1 Processor -- start 2024-03-21T11:49:40.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.508+0000 7f36f3d7f740 1 -- start start 2024-03-21T11:49:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.508+0000 7f36f3d7f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708c9f0 0x564da7086fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.508+0000 7f36f3d7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708dbc0 0x564da709e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.508+0000 7f36f3d7f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da70a9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.508+0000 7f36f3d7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc00f990 con 0x564da708dbc0 2024-03-21T11:49:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.508+0000 7f36f3d7f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc011400 con 0x564da708c9f0 2024-03-21T11:49:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.508+0000 7f36f3d7f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc010050 con 0x564da708e580 2024-03-21T11:49:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.508+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708c9f0 0x564da7086fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.508+0000 7f36e9400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708dbc0 0x564da709e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.508+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708c9f0 0x564da7086fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33870/0 (socket says 172.21.15.47:33870) 2024-03-21T11:49:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.509+0000 7f36e9e00640 1 -- 172.21.15.47:0/4101468929 learned_addr learned my addr 172.21.15.47:0/4101468929 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.509+0000 7f36ea800640 1 --2- 172.21.15.47:0/4101468929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da70a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.509+0000 7f36e9e00640 1 -- 172.21.15.47:0/4101468929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 msgr2=0x564da70a9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.509+0000 7f36e9e00640 1 --2- 172.21.15.47:0/4101468929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da70a9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.509+0000 7f36e9e00640 1 -- 172.21.15.47:0/4101468929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708dbc0 msgr2=0x564da709e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.509+0000 7f36e9e00640 1 --2- 172.21.15.47:0/4101468929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708dbc0 0x564da709e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.509+0000 7f36e9e00640 1 -- 172.21.15.47:0/4101468929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564da708d640 con 0x564da708c9f0 2024-03-21T11:49:40.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.509+0000 7f36e9e00640 1 --2- 172.21.15.47:0/4101468929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708c9f0 0x564da7086fe0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0a4380 tx=0x7f36dc0bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51a62ed123ffd5c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.509+0000 7f36d6a00640 1 -- 172.21.15.47:0/4101468929 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0a48b0 con 0x564da708c9f0 2024-03-21T11:49:40.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.509+0000 7f36d6a00640 1 -- 172.21.15.47:0/4101468929 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36dc099ff0 con 0x564da708c9f0 2024-03-21T11:49:40.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.509+0000 7f36d6a00640 1 -- 172.21.15.47:0/4101468929 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc010430 con 0x564da708c9f0 2024-03-21T11:49:40.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.510+0000 7f36f3d7f740 1 -- 172.21.15.47:0/4101468929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708c9f0 msgr2=0x564da7086fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.510+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/4101468929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708c9f0 0x564da7086fe0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0a4380 tx=0x7f36dc0bc040 comp rx=0 tx=0).stop 2024-03-21T11:49:40.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.511+0000 7f36f3d7f740 1 -- 172.21.15.47:0/4101468929 shutdown_connections 2024-03-21T11:49:40.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.511+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/4101468929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708e580 0x564da70a9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.511+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/4101468929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708dbc0 0x564da709e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.511+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/4101468929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708c9f0 0x564da7086fe0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.511+0000 7f36f3d7f740 1 -- 172.21.15.47:0/4101468929 >> 172.21.15.47:0/4101468929 conn(0x564da7087de0 msgr2=0x564da70868d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.511+0000 7f36f3d7f740 1 -- 172.21.15.47:0/4101468929 shutdown_connections 2024-03-21T11:49:40.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.511+0000 7f36f3d7f740 1 -- 172.21.15.47:0/4101468929 wait complete. 2024-03-21T11:49:40.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.511+0000 7f36f3d7f740 1 -- start start 2024-03-21T11:49:40.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.511+0000 7f36f3d7f740 1 Processor -- start 2024-03-21T11:49:40.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36f3d7f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708c9f0 0x564da709c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36f3d7f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708dbc0 0x564da709cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36f3d7f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708e580 0x564da709d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36f3d7f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc010da0 con 0x564da708dbc0 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36f3d7f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc010710 con 0x564da708e580 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36f3d7f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc00db50 con 0x564da708c9f0 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36f3d7f740 10 client.0 _ll_register_callbacks cb 0x7f36d801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708c9f0 0x564da709c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36ea800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708e580 0x564da709d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36e9400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708dbc0 0x564da709cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36e9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708c9f0 0x564da709c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42652/0 (socket says 172.21.15.47:42652) 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36e9e00640 1 -- 172.21.15.47:0/2225503727 learned_addr learned my addr 172.21.15.47:0/2225503727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36e9e00640 1 -- 172.21.15.47:0/2225503727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708e580 msgr2=0x564da709d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36e9e00640 1 --2- 172.21.15.47:0/2225503727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708e580 0x564da709d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36e9e00640 1 -- 172.21.15.47:0/2225503727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708dbc0 msgr2=0x564da709cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36e9e00640 1 --2- 172.21.15.47:0/2225503727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708dbc0 0x564da709cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.512+0000 7f36e9e00640 1 -- 172.21.15.47:0/2225503727 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564da70fdcd0 con 0x564da708c9f0 2024-03-21T11:49:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.513+0000 7f36e9e00640 1 --2- 172.21.15.47:0/2225503727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708c9f0 0x564da709c7e0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0983b0 tx=0x7f36dc00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc86fc68b231d1fd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.513+0000 7f36cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:49:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.513+0000 7f36cbe00640 1 -- 172.21.15.47:0/2225503727 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc09b540 con 0x564da708c9f0 2024-03-21T11:49:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.513+0000 7f36f3d7f740 1 -- 172.21.15.47:0/2225503727 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36dc099ff0 con 0x564da708c9f0 2024-03-21T11:49:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.513+0000 7f36cbe00640 1 -- 172.21.15.47:0/2225503727 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f36dc09e050 con 0x564da708c9f0 2024-03-21T11:49:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.513+0000 7f36cbe00640 1 -- 172.21.15.47:0/2225503727 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36dc018b20 con 0x564da708c9f0 2024-03-21T11:49:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.513+0000 7f36f3d7f740 10 client.33655 fetch_fsmap learned FSMap version 43 2024-03-21T11:49:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.513+0000 7f36f3d7f740 1 -- 172.21.15.47:0/2225503727 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x564da70feeb0 con 0x564da708c9f0 2024-03-21T11:49:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.513+0000 7f36cbe00640 1 -- 172.21.15.47:0/2225503727 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36dc099ff0 con 0x564da708c9f0 2024-03-21T11:49:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.513+0000 7f36cbe00640 1 -- 172.21.15.47:0/2225503727 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 43) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f36dc096ac0 con 0x564da708c9f0 2024-03-21T11:49:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.513+0000 7f36f3d7f740 10 client.33655 fetch_fsmap finished waiting for FSMap version 43 2024-03-21T11:49:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.513+0000 7f36f3d7f740 10 client.33655 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-03-21T11:49:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.513+0000 7f36f3d7f740 20 client.33655 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:40.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.514+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/2225503727 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564da70ff7f0 0x564da7101c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.514+0000 7f36f3d7f740 4 client.33655 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:49:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.514+0000 7f36f3d7f740 1 -- 172.21.15.47:0/2225503727 --> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564da6ff8b50 con 0x564da70ff7f0 2024-03-21T11:49:40.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.514+0000 7f36e9400640 1 --2- 172.21.15.47:0/2225503727 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564da70ff7f0 0x564da7101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.514+0000 7f36e9400640 1 --2- 172.21.15.47:0/2225503727 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564da70ff7f0 0x564da7101c10 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8f1659e5cc790ccd server_cookie=e6cc7bc6da7619c1 in_seq=0 out_seq=0 2024-03-21T11:49:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.514+0000 7f36cbe00640 10 client.33655 ms_handle_connect on v2:172.21.15.102:6834/1182573386 2024-03-21T11:49:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.514+0000 7f36cbe00640 1 -- 172.21.15.47:0/2225503727 <== mds.4 v2:172.21.15.102:6834/1182573386 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564da6ff8b50 con 0x564da70ff7f0 2024-03-21T11:49:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.514+0000 7f36cbe00640 10 client.33655 handle_command_reply: tid=0 2024-03-21T11:49:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.515+0000 7f36f3d7f740 1 client.33655 shutdown 2024-03-21T11:49:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.515+0000 7f36f3d7f740 1 -- 172.21.15.47:0/2225503727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708c9f0 msgr2=0x564da709c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.515+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/2225503727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708c9f0 0x564da709c7e0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0983b0 tx=0x7f36dc00cf90 comp rx=0 tx=0).stop 2024-03-21T11:49:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.515+0000 7f36f3d7f740 1 -- 172.21.15.47:0/2225503727 shutdown_connections 2024-03-21T11:49:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.515+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/2225503727 >> [v2:172.21.15.102:6834/1182573386,v1:172.21.15.102:6835/1182573386] conn(0x564da70ff7f0 0x564da7101c10 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.515+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/2225503727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564da708e580 0x564da709d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.515+0000 7f36cbe00640 0 client.33655 ms_handle_reset on v2:172.21.15.102:6834/1182573386 2024-03-21T11:49:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.515+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/2225503727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564da708dbc0 0x564da709cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.515+0000 7f36f3d7f740 1 --2- 172.21.15.47:0/2225503727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564da708c9f0 0x564da709c7e0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.515+0000 7f36f3d7f740 1 -- 172.21.15.47:0/2225503727 >> 172.21.15.47:0/2225503727 conn(0x564da7087de0 msgr2=0x564da7091680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.515+0000 7f36f3d7f740 1 -- 172.21.15.47:0/2225503727 shutdown_connections 2024-03-21T11:49:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.515+0000 7f36f3d7f740 1 -- 172.21.15.47:0/2225503727 wait complete. 2024-03-21T11:49:40.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.515+0000 7f36f3d7f740 20 client.33655 trim_cache size 0 max 16384 2024-03-21T11:49:40.520 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:49:40.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.516+0000 7f36eb200640 1 -- 172.21.15.47:0/3619684511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36d00a2f20 msgr2=0x7f36d00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.516+0000 7f36eb200640 1 --2- 172.21.15.47:0/3619684511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36d00a2f20 0x7f36d00a53e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f36e402c640 tx=0x7f36e4032590 comp rx=0 tx=0).stop 2024-03-21T11:49:40.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.516+0000 7f36eb200640 1 -- 172.21.15.47:0/3619684511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ec074100 msgr2=0x7f36ec1999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.516+0000 7f36eb200640 1 --2- 172.21.15.47:0/3619684511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ec074100 0x7f36ec1999f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00e9f0 tx=0x7f36dc00eeb0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.516+0000 7f36eb200640 1 -- 172.21.15.47:0/3619684511 shutdown_connections 2024-03-21T11:49:40.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.516+0000 7f36eb200640 1 --2- 172.21.15.47:0/3619684511 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f36d00a2f20 0x7f36d00a53e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.516+0000 7f36eb200640 1 --2- 172.21.15.47:0/3619684511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36ec109460 0x7f36ec1a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.516+0000 7f36eb200640 1 --2- 172.21.15.47:0/3619684511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36ec0756e0 0x7f36ec199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.516+0000 7f36eb200640 1 --2- 172.21.15.47:0/3619684511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ec074100 0x7f36ec1999f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.516+0000 7f36eb200640 1 -- 172.21.15.47:0/3619684511 >> 172.21.15.47:0/3619684511 conn(0x7f36ec0fcb10 msgr2=0x7f36ec105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.516+0000 7f36eb200640 1 -- 172.21.15.47:0/3619684511 shutdown_connections 2024-03-21T11:49:40.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.516+0000 7f36eb200640 1 -- 172.21.15.47:0/3619684511 wait complete. 2024-03-21T11:49:40.534 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:49:40.534 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T11:49:40.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.720+0000 7fa25ea00640 1 -- 172.21.15.47:0/1185202650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa258076060 msgr2=0x7fa258076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.720+0000 7fa25ea00640 1 --2- 172.21.15.47:0/1185202650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa258076060 0x7fa258076480 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fa248009900 tx=0x7fa24802bc40 comp rx=0 tx=0).stop 2024-03-21T11:49:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.720+0000 7fa25ea00640 1 -- 172.21.15.47:0/1185202650 shutdown_connections 2024-03-21T11:49:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.720+0000 7fa25ea00640 1 --2- 172.21.15.47:0/1185202650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa258077c40 0x7fa2580784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.720+0000 7fa25ea00640 1 --2- 172.21.15.47:0/1185202650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2580772a0 0x7fa258077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.721+0000 7fa25ea00640 1 --2- 172.21.15.47:0/1185202650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa258076060 0x7fa258076480 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.721+0000 7fa25ea00640 1 -- 172.21.15.47:0/1185202650 >> 172.21.15.47:0/1185202650 conn(0x7fa2580fee50 msgr2=0x7fa258101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.721+0000 7fa25ea00640 1 -- 172.21.15.47:0/1185202650 shutdown_connections 2024-03-21T11:49:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.721+0000 7fa25ea00640 1 -- 172.21.15.47:0/1185202650 wait complete. 2024-03-21T11:49:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.722+0000 7fa25ea00640 1 Processor -- start 2024-03-21T11:49:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.722+0000 7fa25ea00640 1 -- start start 2024-03-21T11:49:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.722+0000 7fa25ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa258076060 0x7fa25819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.722+0000 7fa25ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580772a0 0x7fa25819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.722+0000 7fa25ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa258077c40 0x7fa2581a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.722+0000 7fa25ea00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa258113970 con 0x7fa258077c40 2024-03-21T11:49:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.723+0000 7fa25ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581137f0 con 0x7fa2580772a0 2024-03-21T11:49:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.723+0000 7fa25ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa258113af0 con 0x7fa258076060 2024-03-21T11:49:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.723+0000 7fa257e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa258076060 0x7fa25819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.723+0000 7fa257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580772a0 0x7fa25819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.723+0000 7fa257e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa258076060 0x7fa25819dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42662/0 (socket says 172.21.15.47:42662) 2024-03-21T11:49:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.723+0000 7fa257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580772a0 0x7fa25819e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33896/0 (socket says 172.21.15.47:33896) 2024-03-21T11:49:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.723+0000 7fa257e00640 1 -- 172.21.15.47:0/3620711692 learned_addr learned my addr 172.21.15.47:0/3620711692 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.723+0000 7fa25ca00640 1 --2- 172.21.15.47:0/3620711692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa258077c40 0x7fa2581a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.723+0000 7fa257400640 1 -- 172.21.15.47:0/3620711692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa258076060 msgr2=0x7fa25819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.723+0000 7fa257400640 1 --2- 172.21.15.47:0/3620711692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa258076060 0x7fa25819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.723+0000 7fa257400640 1 -- 172.21.15.47:0/3620711692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa258077c40 msgr2=0x7fa2581a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.723+0000 7fa257400640 1 --2- 172.21.15.47:0/3620711692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa258077c40 0x7fa2581a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.723+0000 7fa257400640 1 -- 172.21.15.47:0/3620711692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2581a5bf0 con 0x7fa2580772a0 2024-03-21T11:49:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.723+0000 7fa25ca00640 1 --2- 172.21.15.47:0/3620711692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa258077c40 0x7fa2581a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.724+0000 7fa257400640 1 --2- 172.21.15.47:0/3620711692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580772a0 0x7fa25819e510 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fa24400e9f0 tx=0x7fa24400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.724+0000 7fa254c00640 1 -- 172.21.15.47:0/3620711692 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24400cd80 con 0x7fa2580772a0 2024-03-21T11:49:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.724+0000 7fa254c00640 1 -- 172.21.15.47:0/3620711692 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2440035f0 con 0x7fa2580772a0 2024-03-21T11:49:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.724+0000 7fa25ea00640 1 -- 172.21.15.47:0/3620711692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2581a5ee0 con 0x7fa2580772a0 2024-03-21T11:49:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.724+0000 7fa254c00640 1 -- 172.21.15.47:0/3620711692 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244010690 con 0x7fa2580772a0 2024-03-21T11:49:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.724+0000 7fa25ea00640 1 -- 172.21.15.47:0/3620711692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2581a63a0 con 0x7fa2580772a0 2024-03-21T11:49:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.728+0000 7fa2617e9740 1 Processor -- start 2024-03-21T11:49:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.728+0000 7fa2617e9740 1 -- start start 2024-03-21T11:49:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.728+0000 7fa254c00640 1 -- 172.21.15.47:0/3620711692 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa244003b30 con 0x7fa2580772a0 2024-03-21T11:49:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.728+0000 7fa2617e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b5810 0x55bb6a1b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.728+0000 7fa2617e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.728+0000 7fa2617e9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1bdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.728+0000 7fa2617e9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a11a2c0 con 0x55bb6a1b73a0 2024-03-21T11:49:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.728+0000 7fa2617e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a0c2c90 con 0x55bb6a1b69e0 2024-03-21T11:49:40.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.728+0000 7fa2617e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a116360 con 0x55bb6a1b5810 2024-03-21T11:49:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.729+0000 7fa25ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b5810 0x55bb6a1b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.729+0000 7fa25ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b5810 0x55bb6a1b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42674/0 (socket says 172.21.15.47:42674) 2024-03-21T11:49:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.729+0000 7fa25ca00640 1 -- 172.21.15.47:0/2914163371 learned_addr learned my addr 172.21.15.47:0/2914163371 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.729+0000 7fa257400640 1 --2- 172.21.15.47:0/2914163371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1bdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.729+0000 7fa257e00640 1 --2- 172.21.15.47:0/2914163371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.729+0000 7fa25ca00640 1 -- 172.21.15.47:0/2914163371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b69e0 msgr2=0x55bb6a1b6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.729+0000 7fa25ca00640 1 --2- 172.21.15.47:0/2914163371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1b6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.729+0000 7fa25ca00640 1 -- 172.21.15.47:0/2914163371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 msgr2=0x55bb6a1bdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.729+0000 7fa25ca00640 1 --2- 172.21.15.47:0/2914163371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1bdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.729+0000 7fa25ca00640 1 -- 172.21.15.47:0/2914163371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb6a0e7080 con 0x55bb6a1b5810 2024-03-21T11:49:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.729+0000 7fa25ca00640 1 --2- 172.21.15.47:0/2914163371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b5810 0x55bb6a1b5c30 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fa250009fb0 tx=0x7fa250010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0c7a0df810a45c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.730+0000 7fa24d200640 1 -- 172.21.15.47:0/2914163371 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250016070 con 0x55bb6a1b5810 2024-03-21T11:49:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.730+0000 7fa24d200640 1 -- 172.21.15.47:0/2914163371 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa250012bf0 con 0x55bb6a1b5810 2024-03-21T11:49:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.730+0000 7fa24d200640 1 -- 172.21.15.47:0/2914163371 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250011c10 con 0x55bb6a1b5810 2024-03-21T11:49:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.730+0000 7fa254c00640 1 --2- 172.21.15.47:0/3620711692 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa23c0a2f20 0x7fa23c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.730+0000 7fa254c00640 1 -- 172.21.15.47:0/3620711692 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa2440cfaf0 con 0x7fa2580772a0 2024-03-21T11:49:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.730+0000 7fa257e00640 1 --2- 172.21.15.47:0/3620711692 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa23c0a2f20 0x7fa23c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.730+0000 7fa2617e9740 1 -- 172.21.15.47:0/2914163371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b5810 msgr2=0x55bb6a1b5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.730+0000 7fa2617e9740 1 --2- 172.21.15.47:0/2914163371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b5810 0x55bb6a1b5c30 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fa250009fb0 tx=0x7fa250010800 comp rx=0 tx=0).stop 2024-03-21T11:49:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.731+0000 7fa2617e9740 1 -- 172.21.15.47:0/2914163371 shutdown_connections 2024-03-21T11:49:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.731+0000 7fa2617e9740 1 --2- 172.21.15.47:0/2914163371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1bdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.731+0000 7fa2617e9740 1 --2- 172.21.15.47:0/2914163371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.731+0000 7fa2617e9740 1 --2- 172.21.15.47:0/2914163371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b5810 0x55bb6a1b5c30 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.731+0000 7fa2617e9740 1 -- 172.21.15.47:0/2914163371 >> 172.21.15.47:0/2914163371 conn(0x55bb6a1b0c00 msgr2=0x55bb6a1b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.731+0000 7fa2617e9740 1 -- 172.21.15.47:0/2914163371 shutdown_connections 2024-03-21T11:49:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.731+0000 7fa2617e9740 1 -- 172.21.15.47:0/2914163371 wait complete. 2024-03-21T11:49:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.731+0000 7fa2617e9740 1 -- start start 2024-03-21T11:49:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.731+0000 7fa257e00640 1 --2- 172.21.15.47:0/3620711692 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa23c0a2f20 0x7fa23c0a53e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa248000c00 tx=0x7fa24802c890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.731+0000 7fa2617e9740 1 Processor -- start 2024-03-21T11:49:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa2617e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b5810 0x55bb6a1bfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa2617e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1bffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa2617e9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa2617e9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a1be3b0 con 0x55bb6a1b73a0 2024-03-21T11:49:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa2617e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a0d60d0 con 0x55bb6a1b5810 2024-03-21T11:49:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa2617e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6a1be530 con 0x55bb6a1b69e0 2024-03-21T11:49:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa257e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b5810 0x55bb6a1bfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa257400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1bffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa2617e9740 10 client.0 _ll_register_callbacks cb 0x55bb6a0dae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa257e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b5810 0x55bb6a1bfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33926/0 (socket says 172.21.15.47:33926) 2024-03-21T11:49:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa257400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1bffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42690/0 (socket says 172.21.15.47:42690) 2024-03-21T11:49:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa257e00640 1 -- 172.21.15.47:0/1158222370 learned_addr learned my addr 172.21.15.47:0/1158222370 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa25ca00640 1 --2- 172.21.15.47:0/1158222370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa257400640 1 -- 172.21.15.47:0/1158222370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b5810 msgr2=0x55bb6a1bfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa257400640 1 --2- 172.21.15.47:0/1158222370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b5810 0x55bb6a1bfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa257400640 1 -- 172.21.15.47:0/1158222370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 msgr2=0x55bb6a1c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa257400640 1 --2- 172.21.15.47:0/1158222370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.732+0000 7fa257400640 1 -- 172.21.15.47:0/1158222370 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb69e2e3f0 con 0x55bb6a1b69e0 2024-03-21T11:49:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.733+0000 7fa257400640 1 --2- 172.21.15.47:0/1158222370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1bffc0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa24400c6a0 tx=0x7fa244098110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c00e5041d7380cec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.733+0000 7fa241600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:49:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.733+0000 7fa241600640 1 -- 172.21.15.47:0/1158222370 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24409b7d0 con 0x55bb6a1b69e0 2024-03-21T11:49:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.733+0000 7fa241600640 1 -- 172.21.15.47:0/1158222370 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa2440984e0 con 0x55bb6a1b69e0 2024-03-21T11:49:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.733+0000 7fa2617e9740 1 -- 172.21.15.47:0/1158222370 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa250012bf0 con 0x55bb6a1b69e0 2024-03-21T11:49:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.733+0000 7fa241600640 1 -- 172.21.15.47:0/1158222370 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa244009450 con 0x55bb6a1b69e0 2024-03-21T11:49:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.733+0000 7fa2617e9740 10 client.33667 fetch_fsmap learned FSMap version 43 2024-03-21T11:49:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.733+0000 7fa2617e9740 1 -- 172.21.15.47:0/1158222370 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55bb69ef87d0 con 0x55bb6a1b69e0 2024-03-21T11:49:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.733+0000 7fa241600640 1 -- 172.21.15.47:0/1158222370 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa250012bf0 con 0x55bb6a1b69e0 2024-03-21T11:49:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.734+0000 7fa241600640 1 -- 172.21.15.47:0/1158222370 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 43) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fa2440a43f0 con 0x55bb6a1b69e0 2024-03-21T11:49:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.734+0000 7fa2617e9740 10 client.33667 fetch_fsmap finished waiting for FSMap version 43 2024-03-21T11:49:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.734+0000 7fa2617e9740 10 client.33667 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:49:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.734+0000 7fa2617e9740 20 client.33667 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.734+0000 7fa2617e9740 1 --2- 172.21.15.47:0/1158222370 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55bb6a1cb360 0x55bb6a1cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.734+0000 7fa2617e9740 4 client.33667 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.734+0000 7fa2617e9740 1 -- 172.21.15.47:0/1158222370 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb6a127800 con 0x55bb6a1cb360 2024-03-21T11:49:40.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.734+0000 7fa257e00640 1 --2- 172.21.15.47:0/1158222370 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55bb6a1cb360 0x55bb6a1cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.735+0000 7fa241600640 10 client.33667 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:49:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.735+0000 7fa257e00640 1 --2- 172.21.15.47:0/1158222370 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55bb6a1cb360 0x55bb6a1cd7a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a5fa9a37376000e server_cookie=bb98862446be41ab in_seq=0 out_seq=0 2024-03-21T11:49:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa241600640 1 -- 172.21.15.47:0/1158222370 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bb6a127800 con 0x55bb6a1cb360 2024-03-21T11:49:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa241600640 10 client.33667 handle_command_reply: tid=0 2024-03-21T11:49:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa2617e9740 1 client.33667 shutdown 2024-03-21T11:49:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa2617e9740 1 -- 172.21.15.47:0/1158222370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b69e0 msgr2=0x55bb6a1bffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa2617e9740 1 --2- 172.21.15.47:0/1158222370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1bffc0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa24400c6a0 tx=0x7fa244098110 comp rx=0 tx=0).stop 2024-03-21T11:49:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa2617e9740 1 -- 172.21.15.47:0/1158222370 shutdown_connections 2024-03-21T11:49:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa2617e9740 1 --2- 172.21.15.47:0/1158222370 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55bb6a1cb360 0x55bb6a1cd7a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa2617e9740 1 --2- 172.21.15.47:0/1158222370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa2617e9740 1 --2- 172.21.15.47:0/1158222370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1bffc0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa241600640 0 client.33667 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:49:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa2617e9740 1 --2- 172.21.15.47:0/1158222370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b5810 0x55bb6a1bfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa2617e9740 1 -- 172.21.15.47:0/1158222370 >> 172.21.15.47:0/1158222370 conn(0x55bb6a1b0c00 msgr2=0x55bb6a1afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa2617e9740 1 -- 172.21.15.47:0/1158222370 shutdown_connections 2024-03-21T11:49:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa2617e9740 1 -- 172.21.15.47:0/1158222370 wait complete. 2024-03-21T11:49:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.736+0000 7fa2617e9740 20 client.33667 trim_cache size 0 max 16384 2024-03-21T11:49:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.750+0000 7fa2617e9740 1 Processor -- start 2024-03-21T11:49:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.750+0000 7fa2617e9740 1 -- start start 2024-03-21T11:49:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.750+0000 7fa2617e9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b5810 0x55bb6a1b9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.750+0000 7fa2617e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.750+0000 7fa2617e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1ba150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.750+0000 7fa2617e9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581b0750 con 0x55bb6a1b5810 2024-03-21T11:49:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.750+0000 7fa2617e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581b21c0 con 0x55bb6a1b69e0 2024-03-21T11:49:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.750+0000 7fa2617e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581b0e10 con 0x55bb6a1b73a0 2024-03-21T11:49:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.750+0000 7fa257e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b5810 0x55bb6a1b9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.750+0000 7fa257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.750+0000 7fa25ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1ba150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.750+0000 7fa257400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1c6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33942/0 (socket says 172.21.15.47:33942) 2024-03-21T11:49:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.751+0000 7fa257e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b5810 0x55bb6a1b9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45668/0 (socket says 172.21.15.47:45668) 2024-03-21T11:49:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.751+0000 7fa25ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1ba150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42704/0 (socket says 172.21.15.47:42704) 2024-03-21T11:49:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.751+0000 7fa257400640 1 -- 172.21.15.47:0/551682298 learned_addr learned my addr 172.21.15.47:0/551682298 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.751+0000 7fa257400640 1 -- 172.21.15.47:0/551682298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b73a0 msgr2=0x55bb6a1ba150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.751+0000 7fa257400640 1 --2- 172.21.15.47:0/551682298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1ba150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.751+0000 7fa257400640 1 -- 172.21.15.47:0/551682298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b5810 msgr2=0x55bb6a1b9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.751+0000 7fa257400640 1 --2- 172.21.15.47:0/551682298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b5810 0x55bb6a1b9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.751+0000 7fa257400640 1 -- 172.21.15.47:0/551682298 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb6a1b0970 con 0x55bb6a1b69e0 2024-03-21T11:49:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.751+0000 7fa257e00640 1 --2- 172.21.15.47:0/551682298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b5810 0x55bb6a1b9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.751+0000 7fa25ca00640 1 --2- 172.21.15.47:0/551682298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1ba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.751+0000 7fa257400640 1 --2- 172.21.15.47:0/551682298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1c6fc0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa2440095f0 tx=0x7fa2440a47b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d3f2a92870c07f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.751+0000 7fa243e00640 1 -- 172.21.15.47:0/551682298 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24409b7d0 con 0x55bb6a1b69e0 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.751+0000 7fa243e00640 1 -- 172.21.15.47:0/551682298 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa24409c570 con 0x55bb6a1b69e0 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.751+0000 7fa243e00640 1 -- 172.21.15.47:0/551682298 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2440a43f0 con 0x55bb6a1b69e0 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.752+0000 7fa2617e9740 1 -- 172.21.15.47:0/551682298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b69e0 msgr2=0x55bb6a1c6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.752+0000 7fa2617e9740 1 --2- 172.21.15.47:0/551682298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1c6fc0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa2440095f0 tx=0x7fa2440a47b0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.752+0000 7fa2617e9740 1 -- 172.21.15.47:0/551682298 shutdown_connections 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.752+0000 7fa2617e9740 1 --2- 172.21.15.47:0/551682298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1ba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.752+0000 7fa2617e9740 1 --2- 172.21.15.47:0/551682298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1c6fc0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.752+0000 7fa2617e9740 1 --2- 172.21.15.47:0/551682298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b5810 0x55bb6a1b9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.752+0000 7fa2617e9740 1 -- 172.21.15.47:0/551682298 >> 172.21.15.47:0/551682298 conn(0x55bb6a1b0c00 msgr2=0x55bb6a1b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.752+0000 7fa2617e9740 1 -- 172.21.15.47:0/551682298 shutdown_connections 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.752+0000 7fa2617e9740 1 -- 172.21.15.47:0/551682298 wait complete. 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.753+0000 7fa2617e9740 1 -- start start 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.753+0000 7fa2617e9740 1 Processor -- start 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.753+0000 7fa2617e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b5810 0x55bb6a1c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.753+0000 7fa2617e9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.753+0000 7fa2617e9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.753+0000 7fa2617e9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581b1b60 con 0x55bb6a1b73a0 2024-03-21T11:49:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.753+0000 7fa2617e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581b14d0 con 0x55bb6a1b5810 2024-03-21T11:49:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.753+0000 7fa2617e9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581ae910 con 0x55bb6a1b69e0 2024-03-21T11:49:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.753+0000 7fa2617e9740 10 client.0 _ll_register_callbacks cb 0x7fa24400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa257400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa257e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b5810 0x55bb6a1c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa257400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1c5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42716/0 (socket says 172.21.15.47:42716) 2024-03-21T11:49:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa257400640 1 -- 172.21.15.47:0/3761445879 learned_addr learned my addr 172.21.15.47:0/3761445879 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa25ca00640 1 --2- 172.21.15.47:0/3761445879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa257400640 1 -- 172.21.15.47:0/3761445879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b5810 msgr2=0x55bb6a1c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa257400640 1 --2- 172.21.15.47:0/3761445879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b5810 0x55bb6a1c4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa257400640 1 -- 172.21.15.47:0/3761445879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 msgr2=0x55bb6a1c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa257400640 1 --2- 172.21.15.47:0/3761445879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa257400640 1 -- 172.21.15.47:0/3761445879 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb6a1c6740 con 0x55bb6a1b69e0 2024-03-21T11:49:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa257e00640 1 --2- 172.21.15.47:0/3761445879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b5810 0x55bb6a1c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa25ca00640 1 --2- 172.21.15.47:0/3761445879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa257400640 1 --2- 172.21.15.47:0/3761445879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1c5140 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa24401f000 tx=0x7fa244098110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b0f63ddb293384 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa241600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:49:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa241600640 1 -- 172.21.15.47:0/3761445879 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24409b7d0 con 0x55bb6a1b69e0 2024-03-21T11:49:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa2617e9740 1 -- 172.21.15.47:0/3761445879 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa24409c570 con 0x55bb6a1b69e0 2024-03-21T11:49:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.754+0000 7fa241600640 1 -- 172.21.15.47:0/3761445879 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa24409b540 con 0x55bb6a1b69e0 2024-03-21T11:49:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.755+0000 7fa241600640 1 -- 172.21.15.47:0/3761445879 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa244097bc0 con 0x55bb6a1b69e0 2024-03-21T11:49:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.755+0000 7fa2617e9740 10 client.33673 fetch_fsmap learned FSMap version 43 2024-03-21T11:49:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.755+0000 7fa2617e9740 1 -- 172.21.15.47:0/3761445879 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55bb6a1c6740 con 0x55bb6a1b69e0 2024-03-21T11:49:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.755+0000 7fa241600640 1 -- 172.21.15.47:0/3761445879 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa24409c570 con 0x55bb6a1b69e0 2024-03-21T11:49:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.755+0000 7fa241600640 1 -- 172.21.15.47:0/3761445879 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 43) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fa24407ace0 con 0x55bb6a1b69e0 2024-03-21T11:49:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.755+0000 7fa2617e9740 10 client.33673 fetch_fsmap finished waiting for FSMap version 43 2024-03-21T11:49:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.755+0000 7fa2617e9740 10 client.33673 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-21T11:49:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.755+0000 7fa2617e9740 20 client.33673 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.755+0000 7fa2617e9740 1 --2- 172.21.15.47:0/3761445879 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55bb6a227760 0x55bb6a229b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.755+0000 7fa2617e9740 4 client.33673 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:49:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.755+0000 7fa2617e9740 1 -- 172.21.15.47:0/3761445879 --> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bb6a121b50 con 0x55bb6a227760 2024-03-21T11:49:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.755+0000 7fa257e00640 1 --2- 172.21.15.47:0/3761445879 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55bb6a227760 0x55bb6a229b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa257e00640 1 --2- 172.21.15.47:0/3761445879 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55bb6a227760 0x55bb6a229b80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7771112a6279788c server_cookie=e320044f8979a897 in_seq=0 out_seq=0 2024-03-21T11:49:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa241600640 10 client.33673 ms_handle_connect on v2:172.21.15.100:6836/1118767699 2024-03-21T11:49:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa241600640 1 -- 172.21.15.47:0/3761445879 <== mds.3 v2:172.21.15.100:6836/1118767699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bb6a121b50 con 0x55bb6a227760 2024-03-21T11:49:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa241600640 10 client.33673 handle_command_reply: tid=0 2024-03-21T11:49:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa2617e9740 1 client.33673 shutdown 2024-03-21T11:49:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa2617e9740 1 -- 172.21.15.47:0/3761445879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b69e0 msgr2=0x55bb6a1c5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa2617e9740 1 --2- 172.21.15.47:0/3761445879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1c5140 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa24401f000 tx=0x7fa244098110 comp rx=0 tx=0).stop 2024-03-21T11:49:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa2617e9740 1 -- 172.21.15.47:0/3761445879 shutdown_connections 2024-03-21T11:49:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa2617e9740 1 --2- 172.21.15.47:0/3761445879 >> [v2:172.21.15.100:6836/1118767699,v1:172.21.15.100:6837/1118767699] conn(0x55bb6a227760 0x55bb6a229b80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa2617e9740 1 --2- 172.21.15.47:0/3761445879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6a1b73a0 0x55bb6a1c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa2617e9740 1 --2- 172.21.15.47:0/3761445879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb6a1b69e0 0x55bb6a1c5140 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa241600640 0 client.33673 ms_handle_reset on v2:172.21.15.100:6836/1118767699 2024-03-21T11:49:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa2617e9740 1 --2- 172.21.15.47:0/3761445879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6a1b5810 0x55bb6a1c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa2617e9740 1 -- 172.21.15.47:0/3761445879 >> 172.21.15.47:0/3761445879 conn(0x55bb6a1b0c00 msgr2=0x55bb6a1af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa2617e9740 1 -- 172.21.15.47:0/3761445879 shutdown_connections 2024-03-21T11:49:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.756+0000 7fa2617e9740 1 -- 172.21.15.47:0/3761445879 wait complete. 2024-03-21T11:49:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.757+0000 7fa2617e9740 20 client.33673 trim_cache size 0 max 16384 2024-03-21T11:49:40.763 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:49:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.757+0000 7fa25ea00640 1 -- 172.21.15.47:0/3620711692 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa23c0a2f20 msgr2=0x7fa23c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.757+0000 7fa25ea00640 1 --2- 172.21.15.47:0/3620711692 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa23c0a2f20 0x7fa23c0a53e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa248000c00 tx=0x7fa24802c890 comp rx=0 tx=0).stop 2024-03-21T11:49:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.757+0000 7fa25ea00640 1 -- 172.21.15.47:0/3620711692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580772a0 msgr2=0x7fa25819e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.757+0000 7fa25ea00640 1 --2- 172.21.15.47:0/3620711692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580772a0 0x7fa25819e510 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fa24400e9f0 tx=0x7fa24400eeb0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.757+0000 7fa25ea00640 1 -- 172.21.15.47:0/3620711692 shutdown_connections 2024-03-21T11:49:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.757+0000 7fa25ea00640 1 --2- 172.21.15.47:0/3620711692 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa23c0a2f20 0x7fa23c0a53e0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.757+0000 7fa25ea00640 1 --2- 172.21.15.47:0/3620711692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa258077c40 0x7fa2581a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.757+0000 7fa25ea00640 1 --2- 172.21.15.47:0/3620711692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580772a0 0x7fa25819e510 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.757+0000 7fa25ea00640 1 --2- 172.21.15.47:0/3620711692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa258076060 0x7fa25819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.757+0000 7fa25ea00640 1 -- 172.21.15.47:0/3620711692 >> 172.21.15.47:0/3620711692 conn(0x7fa2580fee50 msgr2=0x7fa2581005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.757+0000 7fa25ea00640 1 -- 172.21.15.47:0/3620711692 shutdown_connections 2024-03-21T11:49:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.757+0000 7fa25ea00640 1 -- 172.21.15.47:0/3620711692 wait complete. 2024-03-21T11:49:40.775 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:49:40.775 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T11:49:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.979+0000 7fc553e00640 1 -- 172.21.15.47:0/3530657275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5541015a0 msgr2=0x7fc5541019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.979+0000 7fc553e00640 1 --2- 172.21.15.47:0/3530657275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5541015a0 0x7fc5541019c0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fc544009860 tx=0x7fc54402bb70 comp rx=0 tx=0).stop 2024-03-21T11:49:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.979+0000 7fc553e00640 1 -- 172.21.15.47:0/3530657275 shutdown_connections 2024-03-21T11:49:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.979+0000 7fc553e00640 1 --2- 172.21.15.47:0/3530657275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc554103150 0x7fc5541099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.979+0000 7fc553e00640 1 --2- 172.21.15.47:0/3530657275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc554102790 0x7fc554102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.979+0000 7fc553e00640 1 --2- 172.21.15.47:0/3530657275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5541015a0 0x7fc5541019c0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.979+0000 7fc553e00640 1 -- 172.21.15.47:0/3530657275 >> 172.21.15.47:0/3530657275 conn(0x7fc5540fcb10 msgr2=0x7fc5540fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.979+0000 7fc553e00640 1 -- 172.21.15.47:0/3530657275 shutdown_connections 2024-03-21T11:49:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.979+0000 7fc553e00640 1 -- 172.21.15.47:0/3530657275 wait complete. 2024-03-21T11:49:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.980+0000 7fc553e00640 1 Processor -- start 2024-03-21T11:49:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.980+0000 7fc553e00640 1 -- start start 2024-03-21T11:49:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.980+0000 7fc553e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5541015a0 0x7fc554199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.980+0000 7fc553e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc554102790 0x7fc554199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.981+0000 7fc553e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc554103150 0x7fc5541a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.981+0000 7fc553e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55410d230 con 0x7fc554103150 2024-03-21T11:49:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.981+0000 7fc553e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55410d0b0 con 0x7fc5541015a0 2024-03-21T11:49:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.981+0000 7fc553e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55410d3b0 con 0x7fc554102790 2024-03-21T11:49:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.981+0000 7fc552a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5541015a0 0x7fc554199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.981+0000 7fc552a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5541015a0 0x7fc554199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33968/0 (socket says 172.21.15.47:33968) 2024-03-21T11:49:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.981+0000 7fc552a00640 1 -- 172.21.15.47:0/2798130617 learned_addr learned my addr 172.21.15.47:0/2798130617 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:40.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.981+0000 7fc54be00640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc554102790 0x7fc554199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.981+0000 7fc553400640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc554103150 0x7fc5541a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.982+0000 7fc552a00640 1 -- 172.21.15.47:0/2798130617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc554102790 msgr2=0x7fc554199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.982+0000 7fc552a00640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc554102790 0x7fc554199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.982+0000 7fc552a00640 1 -- 172.21.15.47:0/2798130617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc554103150 msgr2=0x7fc5541a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.982+0000 7fc552a00640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc554103150 0x7fc5541a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.982+0000 7fc552a00640 1 -- 172.21.15.47:0/2798130617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5541a16d0 con 0x7fc5541015a0 2024-03-21T11:49:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.982+0000 7fc553400640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc554103150 0x7fc5541a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:49:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.982+0000 7fc552a00640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5541015a0 0x7fc554199ab0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fc5440349e0 tx=0x7fc544032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.982+0000 7fc54b400640 1 -- 172.21.15.47:0/2798130617 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544041020 con 0x7fc5541015a0 2024-03-21T11:49:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.982+0000 7fc54b400640 1 -- 172.21.15.47:0/2798130617 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc54402c9f0 con 0x7fc5541015a0 2024-03-21T11:49:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.982+0000 7fc553e00640 1 -- 172.21.15.47:0/2798130617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5541a1960 con 0x7fc5541015a0 2024-03-21T11:49:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.982+0000 7fc54b400640 1 -- 172.21.15.47:0/2798130617 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544039040 con 0x7fc5541015a0 2024-03-21T11:49:40.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.982+0000 7fc553e00640 1 -- 172.21.15.47:0/2798130617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5541a1e50 con 0x7fc5541015a0 2024-03-21T11:49:40.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.986+0000 7fc54b400640 1 -- 172.21.15.47:0/2798130617 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc544044020 con 0x7fc5541015a0 2024-03-21T11:49:40.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.987+0000 7fc55c393740 1 Processor -- start 2024-03-21T11:49:40.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.987+0000 7fc55c393740 1 -- start start 2024-03-21T11:49:40.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.987+0000 7fc54b400640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5340a2f20 0x7fc5340a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.987+0000 7fc54b400640 1 -- 172.21.15.47:0/2798130617 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc5440f1060 con 0x7fc5541015a0 2024-03-21T11:49:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.987+0000 7fc55c393740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 0x55b40b2b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.987+0000 7fc55c393740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b3bc0 0x55b40b2b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.987+0000 7fc55c393740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b4580 0x55b40b2bae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.987+0000 7fc55c393740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40b2172c0 con 0x55b40b2b4580 2024-03-21T11:49:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.987+0000 7fc55c393740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40b1bfc90 con 0x55b40b2b29f0 2024-03-21T11:49:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.987+0000 7fc55c393740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40b213360 con 0x55b40b2b3bc0 2024-03-21T11:49:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.988+0000 7fc54be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b3bc0 0x55b40b2b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.988+0000 7fc552a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 0x55b40b2b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.988+0000 7fc553400640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5340a2f20 0x7fc5340a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.988+0000 7fc54be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b3bc0 0x55b40b2b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42746/0 (socket says 172.21.15.47:42746) 2024-03-21T11:49:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.988+0000 7fc54be00640 1 -- 172.21.15.47:0/305996871 learned_addr learned my addr 172.21.15.47:0/305996871 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.988+0000 7fc552a00640 1 -- 172.21.15.47:0/305996871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b3bc0 msgr2=0x55b40b2b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.988+0000 7fc553400640 1 --2- 172.21.15.47:0/305996871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b4580 0x55b40b2bae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.988+0000 7fc552a00640 1 --2- 172.21.15.47:0/305996871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b3bc0 0x55b40b2b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.988+0000 7fc552a00640 1 -- 172.21.15.47:0/305996871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b4580 msgr2=0x55b40b2bae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.988+0000 7fc552a00640 1 --2- 172.21.15.47:0/305996871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b4580 0x55b40b2bae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.988+0000 7fc552a00640 1 -- 172.21.15.47:0/305996871 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b40b1e4080 con 0x55b40b2b29f0 2024-03-21T11:49:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.988+0000 7fc552a00640 1 --2- 172.21.15.47:0/305996871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 0x55b40b2b2e10 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fc5440ee770 tx=0x7fc544005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1be30e0bbfd291ba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.988+0000 7fc553400640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5340a2f20 0x7fc5340a53e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc54c005fd0 tx=0x7fc54c009b00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.989+0000 7fc543400640 1 -- 172.21.15.47:0/305996871 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5440bd3a0 con 0x55b40b2b29f0 2024-03-21T11:49:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.989+0000 7fc543400640 1 -- 172.21.15.47:0/305996871 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5440c14f0 con 0x55b40b2b29f0 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.989+0000 7fc543400640 1 -- 172.21.15.47:0/305996871 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5440c1170 con 0x55b40b2b29f0 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.990+0000 7fc55c393740 1 -- 172.21.15.47:0/305996871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 msgr2=0x55b40b2b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.990+0000 7fc55c393740 1 --2- 172.21.15.47:0/305996871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 0x55b40b2b2e10 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fc5440ee770 tx=0x7fc544005c90 comp rx=0 tx=0).stop 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.990+0000 7fc55c393740 1 -- 172.21.15.47:0/305996871 shutdown_connections 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.990+0000 7fc55c393740 1 --2- 172.21.15.47:0/305996871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b4580 0x55b40b2bae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.990+0000 7fc55c393740 1 --2- 172.21.15.47:0/305996871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b3bc0 0x55b40b2b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.990+0000 7fc55c393740 1 --2- 172.21.15.47:0/305996871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 0x55b40b2b2e10 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.990+0000 7fc55c393740 1 -- 172.21.15.47:0/305996871 >> 172.21.15.47:0/305996871 conn(0x55b40b2adde0 msgr2=0x55b40b2ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.990+0000 7fc55c393740 1 -- 172.21.15.47:0/305996871 shutdown_connections 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.990+0000 7fc55c393740 1 -- 172.21.15.47:0/305996871 wait complete. 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.990+0000 7fc55c393740 1 -- start start 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.991+0000 7fc55c393740 1 Processor -- start 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.991+0000 7fc55c393740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b29f0 0x55b40b2bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.991+0000 7fc55c393740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b3bc0 0x55b40b2bcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.991+0000 7fc55c393740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b4580 0x55b40b2c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.991+0000 7fc55c393740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40b2bb350 con 0x55b40b2b29f0 2024-03-21T11:49:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.991+0000 7fc55c393740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40b1d30d0 con 0x55b40b2b3bc0 2024-03-21T11:49:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.991+0000 7fc55c393740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40b2bb4d0 con 0x55b40b2b4580 2024-03-21T11:49:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.991+0000 7fc55c393740 10 client.0 _ll_register_callbacks cb 0x55b40b1d7e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.991+0000 7fc54be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b3bc0 0x55b40b2bcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.991+0000 7fc553400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b4580 0x55b40b2c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.991+0000 7fc54be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b3bc0 0x55b40b2bcf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33992/0 (socket says 172.21.15.47:33992) 2024-03-21T11:49:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.991+0000 7fc54be00640 1 -- 172.21.15.47:0/1603255028 learned_addr learned my addr 172.21.15.47:0/1603255028 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.991+0000 7fc552a00640 1 --2- 172.21.15.47:0/1603255028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b29f0 0x55b40b2bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc54be00640 1 -- 172.21.15.47:0/1603255028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b4580 msgr2=0x55b40b2c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc54be00640 1 --2- 172.21.15.47:0/1603255028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b4580 0x55b40b2c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc54be00640 1 -- 172.21.15.47:0/1603255028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b29f0 msgr2=0x55b40b2bca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc54be00640 1 --2- 172.21.15.47:0/1603255028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b29f0 0x55b40b2bca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc54be00640 1 -- 172.21.15.47:0/1603255028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b40af2b3f0 con 0x55b40b2b3bc0 2024-03-21T11:49:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc553400640 1 --2- 172.21.15.47:0/1603255028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b4580 0x55b40b2c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc552a00640 1 --2- 172.21.15.47:0/1603255028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b29f0 0x55b40b2bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc54be00640 1 --2- 172.21.15.47:0/1603255028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b3bc0 0x55b40b2bcf50 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009fd0 tx=0x7fc53c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b6045312d3cb28b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc53be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:49:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc53be00640 1 -- 172.21.15.47:0/1603255028 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c00f920 con 0x55b40b2b3bc0 2024-03-21T11:49:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc55c393740 1 -- 172.21.15.47:0/1603255028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5440c14f0 con 0x55b40b2b3bc0 2024-03-21T11:49:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc53be00640 1 -- 172.21.15.47:0/1603255028 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc53c00fac0 con 0x55b40b2b3bc0 2024-03-21T11:49:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc53be00640 1 -- 172.21.15.47:0/1603255028 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc53c00fcc0 con 0x55b40b2b3bc0 2024-03-21T11:49:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc55c393740 10 client.33776 fetch_fsmap learned FSMap version 43 2024-03-21T11:49:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.992+0000 7fc55c393740 1 -- 172.21.15.47:0/1603255028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55b40aff57d0 con 0x55b40b2b3bc0 2024-03-21T11:49:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.993+0000 7fc53be00640 1 -- 172.21.15.47:0/1603255028 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5440c14f0 con 0x55b40b2b3bc0 2024-03-21T11:49:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.993+0000 7fc53be00640 1 -- 172.21.15.47:0/1603255028 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 43) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fc53c01c440 con 0x55b40b2b3bc0 2024-03-21T11:49:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.993+0000 7fc55c393740 10 client.33776 fetch_fsmap finished waiting for FSMap version 43 2024-03-21T11:49:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.993+0000 7fc55c393740 10 client.33776 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:49:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.993+0000 7fc55c393740 20 client.33776 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.993+0000 7fc55c393740 1 --2- 172.21.15.47:0/1603255028 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b40b2c8420 0x55b40b2ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.993+0000 7fc55c393740 4 client.33776 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.993+0000 7fc55c393740 1 -- 172.21.15.47:0/1603255028 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b40b224800 con 0x55b40b2c8420 2024-03-21T11:49:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.993+0000 7fc552a00640 1 --2- 172.21.15.47:0/1603255028 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b40b2c8420 0x55b40b2ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.994+0000 7fc552a00640 1 --2- 172.21.15.47:0/1603255028 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b40b2c8420 0x55b40b2ca8a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9980635911f7775 server_cookie=92d6f8a97364ffa9 in_seq=0 out_seq=0 2024-03-21T11:49:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.994+0000 7fc53be00640 10 client.33776 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:49:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.995+0000 7fc53be00640 1 -- 172.21.15.47:0/1603255028 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b40b224800 con 0x55b40b2c8420 2024-03-21T11:49:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.995+0000 7fc53be00640 10 client.33776 handle_command_reply: tid=0 2024-03-21T11:49:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.995+0000 7fc55c393740 1 client.33776 shutdown 2024-03-21T11:49:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.995+0000 7fc55c393740 1 -- 172.21.15.47:0/1603255028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b3bc0 msgr2=0x55b40b2bcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.995+0000 7fc55c393740 1 --2- 172.21.15.47:0/1603255028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b3bc0 0x55b40b2bcf50 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009fd0 tx=0x7fc53c012b60 comp rx=0 tx=0).stop 2024-03-21T11:49:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.995+0000 7fc55c393740 1 -- 172.21.15.47:0/1603255028 shutdown_connections 2024-03-21T11:49:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.995+0000 7fc55c393740 1 --2- 172.21.15.47:0/1603255028 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b40b2c8420 0x55b40b2ca8a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.995+0000 7fc55c393740 1 --2- 172.21.15.47:0/1603255028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b4580 0x55b40b2c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.995+0000 7fc55c393740 1 --2- 172.21.15.47:0/1603255028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b3bc0 0x55b40b2bcf50 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.995+0000 7fc55c393740 1 --2- 172.21.15.47:0/1603255028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b29f0 0x55b40b2bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.995+0000 7fc53be00640 0 client.33776 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:49:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.995+0000 7fc55c393740 1 -- 172.21.15.47:0/1603255028 >> 172.21.15.47:0/1603255028 conn(0x55b40b2adde0 msgr2=0x55b40b2ad160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.995+0000 7fc55c393740 1 -- 172.21.15.47:0/1603255028 shutdown_connections 2024-03-21T11:49:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.995+0000 7fc55c393740 1 -- 172.21.15.47:0/1603255028 wait complete. 2024-03-21T11:49:40.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:40.996+0000 7fc55c393740 20 client.33776 trim_cache size 0 max 16384 2024-03-21T11:49:41.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.009+0000 7fc55c393740 1 Processor -- start 2024-03-21T11:49:41.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.009+0000 7fc55c393740 1 -- start start 2024-03-21T11:49:41.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.009+0000 7fc55c393740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 0x55b40b2ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.009+0000 7fc55c393740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b3bc0 0x55b40b2c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.009+0000 7fc55c393740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b4580 0x55b40b2ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.009+0000 7fc55c393740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541ac0c0 con 0x55b40b2b4580 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.009+0000 7fc55c393740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541adb30 con 0x55b40b2b29f0 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.009+0000 7fc55c393740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541ac780 con 0x55b40b2b3bc0 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.009+0000 7fc54be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b3bc0 0x55b40b2c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.009+0000 7fc553400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b4580 0x55b40b2ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.009+0000 7fc54be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b3bc0 0x55b40b2c3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42750/0 (socket says 172.21.15.47:42750) 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.009+0000 7fc54be00640 1 -- 172.21.15.47:0/336924926 learned_addr learned my addr 172.21.15.47:0/336924926 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.010+0000 7fc552a00640 1 --2- 172.21.15.47:0/336924926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 0x55b40b2ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.010+0000 7fc54be00640 1 -- 172.21.15.47:0/336924926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 msgr2=0x55b40b2ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.010+0000 7fc54be00640 1 --2- 172.21.15.47:0/336924926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 0x55b40b2ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.010+0000 7fc54be00640 1 -- 172.21.15.47:0/336924926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b4580 msgr2=0x55b40b2ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.010+0000 7fc54be00640 1 --2- 172.21.15.47:0/336924926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b4580 0x55b40b2ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.010+0000 7fc54be00640 1 -- 172.21.15.47:0/336924926 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b40b2aed50 con 0x55b40b2b3bc0 2024-03-21T11:49:41.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.010+0000 7fc54be00640 1 --2- 172.21.15.47:0/336924926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b3bc0 0x55b40b2c3f50 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009530 tx=0x7fc53c009e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b4d297d208afd80 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:41.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.010+0000 7fc542a00640 1 -- 172.21.15.47:0/336924926 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c00f920 con 0x55b40b2b3bc0 2024-03-21T11:49:41.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.010+0000 7fc542a00640 1 -- 172.21.15.47:0/336924926 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc53c00fac0 con 0x55b40b2b3bc0 2024-03-21T11:49:41.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.010+0000 7fc542a00640 1 -- 172.21.15.47:0/336924926 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c0035f0 con 0x55b40b2b3bc0 2024-03-21T11:49:41.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.011+0000 7fc55c393740 1 -- 172.21.15.47:0/336924926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b3bc0 msgr2=0x55b40b2c3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.011+0000 7fc55c393740 1 --2- 172.21.15.47:0/336924926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b3bc0 0x55b40b2c3f50 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009530 tx=0x7fc53c009e70 comp rx=0 tx=0).stop 2024-03-21T11:49:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.011+0000 7fc55c393740 1 -- 172.21.15.47:0/336924926 shutdown_connections 2024-03-21T11:49:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.011+0000 7fc55c393740 1 --2- 172.21.15.47:0/336924926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b4580 0x55b40b2ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.011+0000 7fc55c393740 1 --2- 172.21.15.47:0/336924926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b3bc0 0x55b40b2c3f50 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.011+0000 7fc55c393740 1 --2- 172.21.15.47:0/336924926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 0x55b40b2ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.011+0000 7fc55c393740 1 -- 172.21.15.47:0/336924926 >> 172.21.15.47:0/336924926 conn(0x55b40b2adde0 msgr2=0x55b40b2ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.011+0000 7fc55c393740 1 -- 172.21.15.47:0/336924926 shutdown_connections 2024-03-21T11:49:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.011+0000 7fc55c393740 1 -- 172.21.15.47:0/336924926 wait complete. 2024-03-21T11:49:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.012+0000 7fc55c393740 1 -- start start 2024-03-21T11:49:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.012+0000 7fc55c393740 1 Processor -- start 2024-03-21T11:49:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.012+0000 7fc55c393740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 0x55b40b2bf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.012+0000 7fc55c393740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b3bc0 0x55b40b2bfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.012+0000 7fc55c393740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b4580 0x55b40b2c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.012+0000 7fc55c393740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541ad4d0 con 0x55b40b2b3bc0 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.012+0000 7fc55c393740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541ace40 con 0x55b40b2b29f0 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.012+0000 7fc55c393740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541aa280 con 0x55b40b2b4580 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.012+0000 7fc55c393740 10 client.0 _ll_register_callbacks cb 0x7fc53c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.013+0000 7fc553400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b4580 0x55b40b2c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.013+0000 7fc553400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b4580 0x55b40b2c2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42754/0 (socket says 172.21.15.47:42754) 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.013+0000 7fc553400640 1 -- 172.21.15.47:0/1095565425 learned_addr learned my addr 172.21.15.47:0/1095565425 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.013+0000 7fc552a00640 1 --2- 172.21.15.47:0/1095565425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 0x55b40b2bf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.013+0000 7fc54be00640 1 --2- 172.21.15.47:0/1095565425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b3bc0 0x55b40b2bfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.013+0000 7fc553400640 1 -- 172.21.15.47:0/1095565425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 msgr2=0x55b40b2bf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.013+0000 7fc553400640 1 --2- 172.21.15.47:0/1095565425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 0x55b40b2bf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.013+0000 7fc553400640 1 -- 172.21.15.47:0/1095565425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b3bc0 msgr2=0x55b40b2bfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.013+0000 7fc553400640 1 --2- 172.21.15.47:0/1095565425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b3bc0 0x55b40b2bfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.013+0000 7fc553400640 1 -- 172.21.15.47:0/1095565425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b40b2c3480 con 0x55b40b2b4580 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.013+0000 7fc553400640 1 --2- 172.21.15.47:0/1095565425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b4580 0x55b40b2c2e90 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fc54c012e20 tx=0x7fc54c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b14c418bc5985986 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.013+0000 7fc53be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:49:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.014+0000 7fc53be00640 1 -- 172.21.15.47:0/1095565425 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c003b70 con 0x55b40b2b4580 2024-03-21T11:49:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.014+0000 7fc55c393740 1 -- 172.21.15.47:0/1095565425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc53c00fac0 con 0x55b40b2b4580 2024-03-21T11:49:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.014+0000 7fc53be00640 1 -- 172.21.15.47:0/1095565425 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fc54c003d10 con 0x55b40b2b4580 2024-03-21T11:49:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.014+0000 7fc53be00640 1 -- 172.21.15.47:0/1095565425 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc54c033050 con 0x55b40b2b4580 2024-03-21T11:49:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.014+0000 7fc55c393740 10 client.33691 fetch_fsmap learned FSMap version 43 2024-03-21T11:49:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.014+0000 7fc55c393740 1 -- 172.21.15.47:0/1095565425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55b40b3240f0 con 0x55b40b2b4580 2024-03-21T11:49:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.014+0000 7fc53be00640 1 -- 172.21.15.47:0/1095565425 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc53c00fac0 con 0x55b40b2b4580 2024-03-21T11:49:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.014+0000 7fc53be00640 1 -- 172.21.15.47:0/1095565425 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 43) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fc54c02b670 con 0x55b40b2b4580 2024-03-21T11:49:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.014+0000 7fc55c393740 10 client.33691 fetch_fsmap finished waiting for FSMap version 43 2024-03-21T11:49:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.014+0000 7fc55c393740 10 client.33691 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-21T11:49:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.014+0000 7fc55c393740 20 client.33691 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.014+0000 7fc55c393740 1 --2- 172.21.15.47:0/1095565425 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b40b324b80 0x55b40b326fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.014+0000 7fc55c393740 4 client.33691 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:49:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.014+0000 7fc55c393740 1 -- 172.21.15.47:0/1095565425 --> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b40b21eb50 con 0x55b40b324b80 2024-03-21T11:49:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.015+0000 7fc552a00640 1 --2- 172.21.15.47:0/1095565425 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b40b324b80 0x55b40b326fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.015+0000 7fc552a00640 1 --2- 172.21.15.47:0/1095565425 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b40b324b80 0x55b40b326fa0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7bb3e87151f50c71 server_cookie=d8462fbf6ebc3a30 in_seq=0 out_seq=0 2024-03-21T11:49:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.015+0000 7fc53be00640 10 client.33691 ms_handle_connect on v2:172.21.15.102:6836/1198140504 2024-03-21T11:49:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.015+0000 7fc53be00640 1 -- 172.21.15.47:0/1095565425 <== mds.2 v2:172.21.15.102:6836/1198140504 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b40b21eb50 con 0x55b40b324b80 2024-03-21T11:49:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.015+0000 7fc53be00640 10 client.33691 handle_command_reply: tid=0 2024-03-21T11:49:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.015+0000 7fc55c393740 1 client.33691 shutdown 2024-03-21T11:49:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.016+0000 7fc55c393740 1 -- 172.21.15.47:0/1095565425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b4580 msgr2=0x55b40b2c2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.016+0000 7fc55c393740 1 --2- 172.21.15.47:0/1095565425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b4580 0x55b40b2c2e90 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fc54c012e20 tx=0x7fc54c004a40 comp rx=0 tx=0).stop 2024-03-21T11:49:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.016+0000 7fc55c393740 1 -- 172.21.15.47:0/1095565425 shutdown_connections 2024-03-21T11:49:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.016+0000 7fc55c393740 1 --2- 172.21.15.47:0/1095565425 >> [v2:172.21.15.102:6836/1198140504,v1:172.21.15.102:6837/1198140504] conn(0x55b40b324b80 0x55b40b326fa0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.016+0000 7fc55c393740 1 --2- 172.21.15.47:0/1095565425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40b2b4580 0x55b40b2c2e90 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.016+0000 7fc55c393740 1 --2- 172.21.15.47:0/1095565425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b40b2b3bc0 0x55b40b2bfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.016+0000 7fc53be00640 0 client.33691 ms_handle_reset on v2:172.21.15.102:6836/1198140504 2024-03-21T11:49:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.016+0000 7fc55c393740 1 --2- 172.21.15.47:0/1095565425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b40b2b29f0 0x55b40b2bf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.016+0000 7fc55c393740 1 -- 172.21.15.47:0/1095565425 >> 172.21.15.47:0/1095565425 conn(0x55b40b2adde0 msgr2=0x55b40b2b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.016+0000 7fc55c393740 1 -- 172.21.15.47:0/1095565425 shutdown_connections 2024-03-21T11:49:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.016+0000 7fc55c393740 1 -- 172.21.15.47:0/1095565425 wait complete. 2024-03-21T11:49:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.016+0000 7fc55c393740 20 client.33691 trim_cache size 0 max 16384 2024-03-21T11:49:41.021 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:49:41.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.017+0000 7fc553e00640 1 -- 172.21.15.47:0/2798130617 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5340a2f20 msgr2=0x7fc5340a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.017+0000 7fc553e00640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5340a2f20 0x7fc5340a53e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc54c005fd0 tx=0x7fc54c009b00 comp rx=0 tx=0).stop 2024-03-21T11:49:41.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.017+0000 7fc553e00640 1 -- 172.21.15.47:0/2798130617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5541015a0 msgr2=0x7fc554199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.017+0000 7fc553e00640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5541015a0 0x7fc554199ab0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fc5440349e0 tx=0x7fc544032430 comp rx=0 tx=0).stop 2024-03-21T11:49:41.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.017+0000 7fc553e00640 1 -- 172.21.15.47:0/2798130617 shutdown_connections 2024-03-21T11:49:41.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.017+0000 7fc553e00640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc5340a2f20 0x7fc5340a53e0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.017+0000 7fc553e00640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc554103150 0x7fc5541a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.017+0000 7fc553e00640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc554102790 0x7fc554199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.017+0000 7fc553e00640 1 --2- 172.21.15.47:0/2798130617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5541015a0 0x7fc554199ab0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.017+0000 7fc553e00640 1 -- 172.21.15.47:0/2798130617 >> 172.21.15.47:0/2798130617 conn(0x7fc5540fcb10 msgr2=0x7fc554107f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.017+0000 7fc553e00640 1 -- 172.21.15.47:0/2798130617 shutdown_connections 2024-03-21T11:49:41.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.017+0000 7fc553e00640 1 -- 172.21.15.47:0/2798130617 wait complete. 2024-03-21T11:49:41.035 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:49:41.035 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T11:49:41.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:40 smithi047 ceph-mon[31351]: pgmap v1191: 129 pgs: 129 active+clean; 556 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 3.1 MiB/s wr, 155 op/s 2024-03-21T11:49:41.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:40 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/980087251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:49:41.200 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:40 smithi100 ceph-mon[36245]: pgmap v1191: 129 pgs: 129 active+clean; 556 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 3.1 MiB/s wr, 155 op/s 2024-03-21T11:49:41.201 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:40 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/980087251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:49:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:40 smithi102 ceph-mon[36748]: pgmap v1191: 129 pgs: 129 active+clean; 556 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 3.1 MiB/s wr, 155 op/s 2024-03-21T11:49:41.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:40 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/980087251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:49:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.414+0000 7f9318c00640 1 -- 172.21.15.47:0/1431298161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f930c09df20 msgr2=0x7f930c0a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.414+0000 7f9318c00640 1 --2- 172.21.15.47:0/1431298161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f930c09df20 0x7f930c0a47b0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f931406b090 tx=0x7f93140973c0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.416+0000 7f9318c00640 1 -- 172.21.15.47:0/1431298161 shutdown_connections 2024-03-21T11:49:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.416+0000 7f9318c00640 1 --2- 172.21.15.47:0/1431298161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f930c09df20 0x7f930c0a47b0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.416+0000 7f9318c00640 1 --2- 172.21.15.47:0/1431298161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930c09d560 0x7f930c09d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.416+0000 7f9318c00640 1 --2- 172.21.15.47:0/1431298161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f930c09c370 0x7f930c09c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.416+0000 7f9318c00640 1 -- 172.21.15.47:0/1431298161 >> 172.21.15.47:0/1431298161 conn(0x7f930c00b390 msgr2=0x7f930c00b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.416+0000 7f9318c00640 1 -- 172.21.15.47:0/1431298161 shutdown_connections 2024-03-21T11:49:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.416+0000 7f9318c00640 1 -- 172.21.15.47:0/1431298161 wait complete. 2024-03-21T11:49:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.417+0000 7f9318c00640 1 Processor -- start 2024-03-21T11:49:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.418+0000 7f9318c00640 1 -- start start 2024-03-21T11:49:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.419+0000 7f9318c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f930c09c370 0x7f930c132740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.419+0000 7f9318c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f930c09d560 0x7f930c132c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.419+0000 7f9318c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930c09df20 0x7f930c139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.419+0000 7f9318c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c0a8050 con 0x7f930c09c370 2024-03-21T11:49:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.419+0000 7f9318c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c0a7ed0 con 0x7f930c09df20 2024-03-21T11:49:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.419+0000 7f9318c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c0a81d0 con 0x7f930c09d560 2024-03-21T11:49:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.419+0000 7f9313e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930c09df20 0x7f930c139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.419+0000 7f9313e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930c09df20 0x7f930c139ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34004/0 (socket says 172.21.15.47:34004) 2024-03-21T11:49:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.419+0000 7f9313e00640 1 -- 172.21.15.47:0/3887646098 learned_addr learned my addr 172.21.15.47:0/3887646098 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.420+0000 7f9312a00640 1 --2- 172.21.15.47:0/3887646098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f930c09d560 0x7f930c132c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.420+0000 7f9313e00640 1 -- 172.21.15.47:0/3887646098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f930c09d560 msgr2=0x7f930c132c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.420+0000 7f9313e00640 1 --2- 172.21.15.47:0/3887646098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f930c09d560 0x7f930c132c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.420+0000 7f9313e00640 1 -- 172.21.15.47:0/3887646098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f930c09c370 msgr2=0x7f930c132740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.420+0000 7f9313e00640 1 --2- 172.21.15.47:0/3887646098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f930c09c370 0x7f930c132740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.420+0000 7f9313e00640 1 -- 172.21.15.47:0/3887646098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c13a310 con 0x7f930c09df20 2024-03-21T11:49:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.420+0000 7f9313e00640 1 --2- 172.21.15.47:0/3887646098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930c09df20 0x7f930c139ca0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f9314050670 tx=0x7f9314069300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.421+0000 7f9307e00640 1 -- 172.21.15.47:0/3887646098 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93140ad030 con 0x7f930c09df20 2024-03-21T11:49:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.421+0000 7f9307e00640 1 -- 172.21.15.47:0/3887646098 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9314069680 con 0x7f930c09df20 2024-03-21T11:49:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.421+0000 7f9307e00640 1 -- 172.21.15.47:0/3887646098 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93140a96d0 con 0x7f930c09df20 2024-03-21T11:49:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.421+0000 7f9318c00640 1 -- 172.21.15.47:0/3887646098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930c13a5a0 con 0x7f930c09df20 2024-03-21T11:49:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.421+0000 7f9318c00640 1 -- 172.21.15.47:0/3887646098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930c13aaf0 con 0x7f930c09df20 2024-03-21T11:49:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.431+0000 7f931ce09740 1 Processor -- start 2024-03-21T11:49:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.431+0000 7f9307e00640 1 -- 172.21.15.47:0/3887646098 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f93140b0070 con 0x7f930c09df20 2024-03-21T11:49:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.432+0000 7f931ce09740 1 -- start start 2024-03-21T11:49:41.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.432+0000 7f931ce09740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 0x55f245cf3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.432+0000 7f931ce09740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf4bc0 0x55f245cf5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.432+0000 7f931ce09740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf5580 0x55f245cfbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.432+0000 7f931ce09740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f245c582c0 con 0x55f245cf5580 2024-03-21T11:49:41.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.432+0000 7f931ce09740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f245c00c90 con 0x55f245cf39f0 2024-03-21T11:49:41.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.432+0000 7f931ce09740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f245c54360 con 0x55f245cf4bc0 2024-03-21T11:49:41.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.434+0000 7f9313400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf4bc0 0x55f245cf5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.434+0000 7f9313400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf4bc0 0x55f245cf5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42770/0 (socket says 172.21.15.47:42770) 2024-03-21T11:49:41.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.434+0000 7f9313400640 1 -- 172.21.15.47:0/1382307973 learned_addr learned my addr 172.21.15.47:0/1382307973 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:41.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.434+0000 7f9307e00640 1 --2- 172.21.15.47:0/3887646098 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92fc0a2fa0 0x7f92fc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.434+0000 7f9312a00640 1 --2- 172.21.15.47:0/1382307973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf5580 0x55f245cfbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.434+0000 7f9312a00640 1 --2- 172.21.15.47:0/1382307973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 0x55f245cf3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.435+0000 7f9307e00640 1 -- 172.21.15.47:0/3887646098 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f931415d500 con 0x7f930c09df20 2024-03-21T11:49:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.436+0000 7f9313400640 1 -- 172.21.15.47:0/1382307973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 msgr2=0x55f245cf3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.436+0000 7f9313400640 1 --2- 172.21.15.47:0/1382307973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 0x55f245cf3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.436+0000 7f9313400640 1 -- 172.21.15.47:0/1382307973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf5580 msgr2=0x55f245cfbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.436+0000 7f9313e00640 1 --2- 172.21.15.47:0/3887646098 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92fc0a2fa0 0x7f92fc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.437+0000 7f9313400640 1 --2- 172.21.15.47:0/1382307973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf5580 0x55f245cfbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.438+0000 7f9313400640 1 -- 172.21.15.47:0/1382307973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f245c25080 con 0x55f245cf4bc0 2024-03-21T11:49:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.438+0000 7f9312a00640 1 --2- 172.21.15.47:0/1382307973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf5580 0x55f245cfbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:49:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.438+0000 7f9313e00640 1 --2- 172.21.15.47:0/3887646098 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92fc0a2fa0 0x7f92fc0a5460 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9314066e40 tx=0x7f9314069040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.439+0000 7f9313400640 1 --2- 172.21.15.47:0/1382307973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf4bc0 0x55f245cf5040 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f93080097b0 tx=0x7f9308009c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e221e3db3568a865 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.439+0000 7f92fbe00640 1 -- 172.21.15.47:0/1382307973 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308015020 con 0x55f245cf4bc0 2024-03-21T11:49:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.439+0000 7f92fbe00640 1 -- 172.21.15.47:0/1382307973 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9308011420 con 0x55f245cf4bc0 2024-03-21T11:49:41.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.439+0000 7f92fbe00640 1 -- 172.21.15.47:0/1382307973 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93080105e0 con 0x55f245cf4bc0 2024-03-21T11:49:41.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.441+0000 7f931ce09740 1 -- 172.21.15.47:0/1382307973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf4bc0 msgr2=0x55f245cf5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.441+0000 7f931ce09740 1 --2- 172.21.15.47:0/1382307973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf4bc0 0x55f245cf5040 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f93080097b0 tx=0x7f9308009c70 comp rx=0 tx=0).stop 2024-03-21T11:49:41.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.442+0000 7f931ce09740 1 -- 172.21.15.47:0/1382307973 shutdown_connections 2024-03-21T11:49:41.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.442+0000 7f931ce09740 1 --2- 172.21.15.47:0/1382307973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf5580 0x55f245cfbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.442+0000 7f931ce09740 1 --2- 172.21.15.47:0/1382307973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf4bc0 0x55f245cf5040 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.442+0000 7f931ce09740 1 --2- 172.21.15.47:0/1382307973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 0x55f245cf3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.442+0000 7f931ce09740 1 -- 172.21.15.47:0/1382307973 >> 172.21.15.47:0/1382307973 conn(0x55f245ceede0 msgr2=0x55f245cef1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.444+0000 7f931ce09740 1 -- 172.21.15.47:0/1382307973 shutdown_connections 2024-03-21T11:49:41.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.444+0000 7f931ce09740 1 -- 172.21.15.47:0/1382307973 wait complete. 2024-03-21T11:49:41.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.444+0000 7f931ce09740 1 -- start start 2024-03-21T11:49:41.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.445+0000 7f931ce09740 1 Processor -- start 2024-03-21T11:49:41.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.445+0000 7f931ce09740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 0x55f245cfdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.446+0000 7f931ce09740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf4bc0 0x55f245cfe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.446+0000 7f931ce09740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf5580 0x55f245d051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.446+0000 7f931ce09740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f245cfc590 con 0x55f245cf4bc0 2024-03-21T11:49:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.446+0000 7f931ce09740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f245c140d0 con 0x55f245cf39f0 2024-03-21T11:49:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.446+0000 7f931ce09740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f245cfc710 con 0x55f245cf5580 2024-03-21T11:49:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.446+0000 7f931ce09740 10 client.0 _ll_register_callbacks cb 0x55f245c18e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.446+0000 7f9312a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 0x55f245cfdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.446+0000 7f9312a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 0x55f245cfdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34022/0 (socket says 172.21.15.47:34022) 2024-03-21T11:49:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.446+0000 7f9312a00640 1 -- 172.21.15.47:0/2098839841 learned_addr learned my addr 172.21.15.47:0/2098839841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.446+0000 7f9312a00640 1 --2- 172.21.15.47:0/2098839841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf5580 0x55f245d051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.448+0000 7f9312a00640 1 -- 172.21.15.47:0/2098839841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf5580 msgr2=0x55f245d051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.448+0000 7f9312a00640 1 --2- 172.21.15.47:0/2098839841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf5580 0x55f245d051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.448+0000 7f9312a00640 1 -- 172.21.15.47:0/2098839841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf4bc0 msgr2=0x55f245cfe180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.448+0000 7f9312a00640 1 --2- 172.21.15.47:0/2098839841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf4bc0 0x55f245cfe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.448+0000 7f9312a00640 1 -- 172.21.15.47:0/2098839841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f24596c3f0 con 0x55f245cf39f0 2024-03-21T11:49:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.452+0000 7f9312a00640 1 --2- 172.21.15.47:0/2098839841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 0x55f245cfdc40 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f9300013870 tx=0x7f9300008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ff71d213157f938 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.452+0000 7f92f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:49:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.452+0000 7f92f8c00640 1 -- 172.21.15.47:0/2098839841 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930001a020 con 0x55f245cf39f0 2024-03-21T11:49:41.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.453+0000 7f931ce09740 1 -- 172.21.15.47:0/2098839841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9308011420 con 0x55f245cf39f0 2024-03-21T11:49:41.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.453+0000 7f92f8c00640 1 -- 172.21.15.47:0/2098839841 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f930000d8f0 con 0x55f245cf39f0 2024-03-21T11:49:41.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.455+0000 7f92f8c00640 1 -- 172.21.15.47:0/2098839841 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f930000daf0 con 0x55f245cf39f0 2024-03-21T11:49:41.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.455+0000 7f931ce09740 10 client.33791 fetch_fsmap learned FSMap version 43 2024-03-21T11:49:41.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.455+0000 7f931ce09740 1 -- 172.21.15.47:0/2098839841 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f245a367d0 con 0x55f245cf39f0 2024-03-21T11:49:41.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.456+0000 7f92f8c00640 1 -- 172.21.15.47:0/2098839841 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9308011420 con 0x55f245cf39f0 2024-03-21T11:49:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.456+0000 7f92f8c00640 1 -- 172.21.15.47:0/2098839841 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 43) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f9300027020 con 0x55f245cf39f0 2024-03-21T11:49:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.456+0000 7f931ce09740 10 client.33791 fetch_fsmap finished waiting for FSMap version 43 2024-03-21T11:49:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.456+0000 7f931ce09740 10 client.33791 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:49:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.456+0000 7f931ce09740 20 client.33791 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.456+0000 7f931ce09740 1 --2- 172.21.15.47:0/2098839841 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f245d09690 0x55f245d0bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.456+0000 7f931ce09740 4 client.33791 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.457+0000 7f9313400640 1 --2- 172.21.15.47:0/2098839841 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f245d09690 0x55f245d0bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.457+0000 7f931ce09740 1 -- 172.21.15.47:0/2098839841 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f245c65800 con 0x55f245d09690 2024-03-21T11:49:41.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.464+0000 7f9313400640 1 --2- 172.21.15.47:0/2098839841 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f245d09690 0x55f245d0bb10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f42e67ff54a59af6 server_cookie=2180ae280851753a in_seq=0 out_seq=0 2024-03-21T11:49:41.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.464+0000 7f92f8c00640 10 client.33791 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:49:41.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.465+0000 7f92f8c00640 1 -- 172.21.15.47:0/2098839841 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f245c65800 con 0x55f245d09690 2024-03-21T11:49:41.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.465+0000 7f92f8c00640 10 client.33791 handle_command_reply: tid=0 2024-03-21T11:49:41.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.465+0000 7f931ce09740 1 client.33791 shutdown 2024-03-21T11:49:41.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.468+0000 7f931ce09740 1 -- 172.21.15.47:0/2098839841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 msgr2=0x55f245cfdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.468+0000 7f931ce09740 1 --2- 172.21.15.47:0/2098839841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 0x55f245cfdc40 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f9300013870 tx=0x7f9300008040 comp rx=0 tx=0).stop 2024-03-21T11:49:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.468+0000 7f931ce09740 1 -- 172.21.15.47:0/2098839841 shutdown_connections 2024-03-21T11:49:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.468+0000 7f931ce09740 1 --2- 172.21.15.47:0/2098839841 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f245d09690 0x55f245d0bb10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.468+0000 7f931ce09740 1 --2- 172.21.15.47:0/2098839841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf5580 0x55f245d051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.468+0000 7f92f8c00640 0 client.33791 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:49:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.468+0000 7f931ce09740 1 --2- 172.21.15.47:0/2098839841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf4bc0 0x55f245cfe180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.468+0000 7f931ce09740 1 --2- 172.21.15.47:0/2098839841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 0x55f245cfdc40 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.469+0000 7f931ce09740 1 -- 172.21.15.47:0/2098839841 >> 172.21.15.47:0/2098839841 conn(0x55f245ceede0 msgr2=0x55f245cee160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.472+0000 7f931ce09740 1 -- 172.21.15.47:0/2098839841 shutdown_connections 2024-03-21T11:49:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.473+0000 7f931ce09740 1 -- 172.21.15.47:0/2098839841 wait complete. 2024-03-21T11:49:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.476+0000 7f931ce09740 20 client.33791 trim_cache size 0 max 16384 2024-03-21T11:49:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.525+0000 7f931ce09740 1 Processor -- start 2024-03-21T11:49:41.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.525+0000 7f931ce09740 1 -- start start 2024-03-21T11:49:41.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.526+0000 7f931ce09740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf39f0 0x55f245cfdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.526+0000 7f931ce09740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf4bc0 0x55f245d05180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.526+0000 7f931ce09740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf5580 0x55f245ced8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.526+0000 7f931ce09740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c144dc0 con 0x55f245cf4bc0 2024-03-21T11:49:41.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.526+0000 7f931ce09740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c146830 con 0x55f245cf5580 2024-03-21T11:49:41.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.526+0000 7f931ce09740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c145480 con 0x55f245cf39f0 2024-03-21T11:49:41.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.527+0000 7f9312a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf39f0 0x55f245cfdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.527+0000 7f9312a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf39f0 0x55f245cfdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42788/0 (socket says 172.21.15.47:42788) 2024-03-21T11:49:41.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.527+0000 7f9312a00640 1 -- 172.21.15.47:0/1585968403 learned_addr learned my addr 172.21.15.47:0/1585968403 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:41.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.527+0000 7f9312a00640 1 --2- 172.21.15.47:0/1585968403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf5580 0x55f245ced8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.528+0000 7f9313400640 1 --2- 172.21.15.47:0/1585968403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf4bc0 0x55f245d05180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.529+0000 7f9312a00640 1 -- 172.21.15.47:0/1585968403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf5580 msgr2=0x55f245ced8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.529+0000 7f9312a00640 1 --2- 172.21.15.47:0/1585968403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf5580 0x55f245ced8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.530+0000 7f9312a00640 1 -- 172.21.15.47:0/1585968403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf4bc0 msgr2=0x55f245d05180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.530+0000 7f9312a00640 1 --2- 172.21.15.47:0/1585968403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf4bc0 0x55f245d05180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.530+0000 7f9312a00640 1 -- 172.21.15.47:0/1585968403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f245cf4640 con 0x55f245cf39f0 2024-03-21T11:49:41.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.530+0000 7f9313400640 1 --2- 172.21.15.47:0/1585968403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf4bc0 0x55f245d05180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:41.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.531+0000 7f9312a00640 1 --2- 172.21.15.47:0/1585968403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf39f0 0x55f245cfdec0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f9300013fd0 tx=0x7f930000c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16c4b9bbbfe93d5a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.535+0000 7f92fb400640 1 -- 172.21.15.47:0/1585968403 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930001a070 con 0x55f245cf39f0 2024-03-21T11:49:41.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.535+0000 7f92fb400640 1 -- 172.21.15.47:0/1585968403 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9300010040 con 0x55f245cf39f0 2024-03-21T11:49:41.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.535+0000 7f92fb400640 1 -- 172.21.15.47:0/1585968403 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930001f770 con 0x55f245cf39f0 2024-03-21T11:49:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.540+0000 7f931ce09740 1 -- 172.21.15.47:0/1585968403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf39f0 msgr2=0x55f245cfdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.540+0000 7f931ce09740 1 --2- 172.21.15.47:0/1585968403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf39f0 0x55f245cfdec0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f9300013fd0 tx=0x7f930000c710 comp rx=0 tx=0).stop 2024-03-21T11:49:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.540+0000 7f931ce09740 1 -- 172.21.15.47:0/1585968403 shutdown_connections 2024-03-21T11:49:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.540+0000 7f931ce09740 1 --2- 172.21.15.47:0/1585968403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf5580 0x55f245ced8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.540+0000 7f931ce09740 1 --2- 172.21.15.47:0/1585968403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf4bc0 0x55f245d05180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.540+0000 7f931ce09740 1 --2- 172.21.15.47:0/1585968403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf39f0 0x55f245cfdec0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.540+0000 7f931ce09740 1 -- 172.21.15.47:0/1585968403 >> 172.21.15.47:0/1585968403 conn(0x55f245ceede0 msgr2=0x55f245cf28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.540+0000 7f931ce09740 1 -- 172.21.15.47:0/1585968403 shutdown_connections 2024-03-21T11:49:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.540+0000 7f931ce09740 1 -- 172.21.15.47:0/1585968403 wait complete. 2024-03-21T11:49:41.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.541+0000 7f931ce09740 1 -- start start 2024-03-21T11:49:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.542+0000 7f931ce09740 1 Processor -- start 2024-03-21T11:49:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.543+0000 7f931ce09740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 0x55f245d02bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.543+0000 7f931ce09740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf4bc0 0x55f245d03130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.543+0000 7f931ce09740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf5580 0x55f245d04140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.543+0000 7f931ce09740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c1461d0 con 0x55f245cf5580 2024-03-21T11:49:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.543+0000 7f931ce09740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c145b40 con 0x55f245cf39f0 2024-03-21T11:49:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.543+0000 7f931ce09740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c142f80 con 0x55f245cf4bc0 2024-03-21T11:49:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.543+0000 7f931ce09740 10 client.0 _ll_register_callbacks cb 0x7f930000c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.544+0000 7f9312a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf5580 0x55f245d04140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.544+0000 7f9312a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf5580 0x55f245d04140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45780/0 (socket says 172.21.15.47:45780) 2024-03-21T11:49:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.544+0000 7f9312a00640 1 -- 172.21.15.47:0/1350223180 learned_addr learned my addr 172.21.15.47:0/1350223180 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.544+0000 7f9312a00640 1 -- 172.21.15.47:0/1350223180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf4bc0 msgr2=0x55f245d03130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T11:49:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.544+0000 7f9312a00640 1 --2- 172.21.15.47:0/1350223180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf4bc0 0x55f245d03130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.544+0000 7f9312a00640 1 -- 172.21.15.47:0/1350223180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 msgr2=0x55f245d02bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T11:49:41.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.544+0000 7f9312a00640 1 --2- 172.21.15.47:0/1350223180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 0x55f245d02bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.545+0000 7f9312a00640 1 -- 172.21.15.47:0/1350223180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f245d04730 con 0x55f245cf5580 2024-03-21T11:49:41.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.545+0000 7f9312a00640 1 --2- 172.21.15.47:0/1350223180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf5580 0x55f245d04140 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f930000d530 tx=0x7f9300016660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d42e1040e7c7399 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:41.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.545+0000 7f92f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T11:49:41.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.548+0000 7f92f8c00640 1 -- 172.21.15.47:0/1350223180 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930001a040 con 0x55f245cf5580 2024-03-21T11:49:41.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.548+0000 7f92f8c00640 1 -- 172.21.15.47:0/1350223180 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f930001f7b0 con 0x55f245cf5580 2024-03-21T11:49:41.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.548+0000 7f931ce09740 1 -- 172.21.15.47:0/1350223180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9300010040 con 0x55f245cf5580 2024-03-21T11:49:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.549+0000 7f92f8c00640 1 -- 172.21.15.47:0/1350223180 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f930001f9b0 con 0x55f245cf5580 2024-03-21T11:49:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.549+0000 7f931ce09740 10 client.22836 fetch_fsmap learned FSMap version 43 2024-03-21T11:49:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.549+0000 7f931ce09740 1 -- 172.21.15.47:0/1350223180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f245d76300 con 0x55f245cf5580 2024-03-21T11:49:41.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.550+0000 7f92f8c00640 1 -- 172.21.15.47:0/1350223180 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9300010040 con 0x55f245cf5580 2024-03-21T11:49:41.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.551+0000 7f92f8c00640 1 -- 172.21.15.47:0/1350223180 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 43) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f9300027040 con 0x55f245cf5580 2024-03-21T11:49:41.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.551+0000 7f931ce09740 10 client.22836 fetch_fsmap finished waiting for FSMap version 43 2024-03-21T11:49:41.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.551+0000 7f931ce09740 10 client.22836 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-21T11:49:41.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.551+0000 7f931ce09740 20 client.22836 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:41.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.551+0000 7f931ce09740 1 --2- 172.21.15.47:0/1350223180 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f245d09690 0x55f245d78d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.551+0000 7f931ce09740 4 client.22836 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:49:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.552+0000 7f9313400640 1 --2- 172.21.15.47:0/1350223180 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f245d09690 0x55f245d78d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.553+0000 7f9313400640 1 --2- 172.21.15.47:0/1350223180 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f245d09690 0x55f245d78d20 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa6ac02b8335a213 server_cookie=93b6c291b8675744 in_seq=0 out_seq=0 2024-03-21T11:49:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.553+0000 7f931ce09740 1 -- 172.21.15.47:0/1350223180 --> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f245c5fb50 con 0x55f245d09690 2024-03-21T11:49:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.553+0000 7f92f8c00640 10 client.22836 ms_handle_connect on v2:172.21.15.102:6838/2885246980 2024-03-21T11:49:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.553+0000 7f92f8c00640 1 -- 172.21.15.47:0/1350223180 <== mds.1 v2:172.21.15.102:6838/2885246980 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f245c5fb50 con 0x55f245d09690 2024-03-21T11:49:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.553+0000 7f92f8c00640 10 client.22836 handle_command_reply: tid=0 2024-03-21T11:49:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.553+0000 7f931ce09740 1 client.22836 shutdown 2024-03-21T11:49:41.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.556+0000 7f931ce09740 1 -- 172.21.15.47:0/1350223180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf5580 msgr2=0x55f245d04140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.556+0000 7f931ce09740 1 --2- 172.21.15.47:0/1350223180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf5580 0x55f245d04140 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f930000d530 tx=0x7f9300016660 comp rx=0 tx=0).stop 2024-03-21T11:49:41.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.556+0000 7f931ce09740 1 -- 172.21.15.47:0/1350223180 shutdown_connections 2024-03-21T11:49:41.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.556+0000 7f931ce09740 1 --2- 172.21.15.47:0/1350223180 >> [v2:172.21.15.102:6838/2885246980,v1:172.21.15.102:6839/2885246980] conn(0x55f245d09690 0x55f245d78d20 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.556+0000 7f931ce09740 1 --2- 172.21.15.47:0/1350223180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f245cf5580 0x55f245d04140 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.556+0000 7f931ce09740 1 --2- 172.21.15.47:0/1350223180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f245cf4bc0 0x55f245d03130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.556+0000 7f931ce09740 1 --2- 172.21.15.47:0/1350223180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f245cf39f0 0x55f245d02bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.556+0000 7f931ce09740 1 -- 172.21.15.47:0/1350223180 >> 172.21.15.47:0/1350223180 conn(0x55f245ceede0 msgr2=0x55f245cf28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.556+0000 7f92f8c00640 0 client.22836 ms_handle_reset on v2:172.21.15.102:6838/2885246980 2024-03-21T11:49:41.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.561+0000 7f931ce09740 1 -- 172.21.15.47:0/1350223180 shutdown_connections 2024-03-21T11:49:41.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.561+0000 7f931ce09740 1 -- 172.21.15.47:0/1350223180 wait complete. 2024-03-21T11:49:41.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.563+0000 7f931ce09740 20 client.22836 trim_cache size 0 max 16384 2024-03-21T11:49:41.567 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:49:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.567+0000 7f9318c00640 1 -- 172.21.15.47:0/3887646098 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92fc0a2fa0 msgr2=0x7f92fc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.567+0000 7f9318c00640 1 --2- 172.21.15.47:0/3887646098 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92fc0a2fa0 0x7f92fc0a5460 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9314066e40 tx=0x7f9314069040 comp rx=0 tx=0).stop 2024-03-21T11:49:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.567+0000 7f9318c00640 1 -- 172.21.15.47:0/3887646098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930c09df20 msgr2=0x7f930c139ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.567+0000 7f9318c00640 1 --2- 172.21.15.47:0/3887646098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930c09df20 0x7f930c139ca0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f9314050670 tx=0x7f9314069300 comp rx=0 tx=0).stop 2024-03-21T11:49:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.567+0000 7f9318c00640 1 -- 172.21.15.47:0/3887646098 shutdown_connections 2024-03-21T11:49:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.567+0000 7f9318c00640 1 --2- 172.21.15.47:0/3887646098 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92fc0a2fa0 0x7f92fc0a5460 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.567+0000 7f9318c00640 1 --2- 172.21.15.47:0/3887646098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930c09df20 0x7f930c139ca0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.567+0000 7f9318c00640 1 --2- 172.21.15.47:0/3887646098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f930c09d560 0x7f930c132c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.567+0000 7f9318c00640 1 --2- 172.21.15.47:0/3887646098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f930c09c370 0x7f930c132740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.567+0000 7f9318c00640 1 -- 172.21.15.47:0/3887646098 >> 172.21.15.47:0/3887646098 conn(0x7f930c00b390 msgr2=0x7f930c0a0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.568+0000 7f9318c00640 1 -- 172.21.15.47:0/3887646098 shutdown_connections 2024-03-21T11:49:41.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.568+0000 7f9318c00640 1 -- 172.21.15.47:0/3887646098 wait complete. 2024-03-21T11:49:41.601 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T11:49:41.601 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T11:49:41.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.905+0000 7f6629600640 1 -- 172.21.15.47:0/1700106295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6624072ce0 msgr2=0x7f6624073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.905+0000 7f6629600640 1 --2- 172.21.15.47:0/1700106295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6624072ce0 0x7f6624073160 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6610009900 tx=0x7f661002bc30 comp rx=0 tx=0).stop 2024-03-21T11:49:41.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.905+0000 7f6629600640 1 -- 172.21.15.47:0/1700106295 shutdown_connections 2024-03-21T11:49:41.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.905+0000 7f6629600640 1 --2- 172.21.15.47:0/1700106295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f662410fa70 0x7f6624111ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.905+0000 7f6629600640 1 --2- 172.21.15.47:0/1700106295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6624072ce0 0x7f6624073160 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.905+0000 7f6629600640 1 --2- 172.21.15.47:0/1700106295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6624109340 0x7f6624072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.905+0000 7f6629600640 1 -- 172.21.15.47:0/1700106295 >> 172.21.15.47:0/1700106295 conn(0x7f662406e580 msgr2=0x7f66240709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.905+0000 7f6629600640 1 -- 172.21.15.47:0/1700106295 shutdown_connections 2024-03-21T11:49:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.905+0000 7f6629600640 1 -- 172.21.15.47:0/1700106295 wait complete. 2024-03-21T11:49:41.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.906+0000 7f6629600640 1 Processor -- start 2024-03-21T11:49:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.907+0000 7f6629600640 1 -- start start 2024-03-21T11:49:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.907+0000 7f6629600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6624109340 0x7f66241a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.907+0000 7f6629600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f662410fa70 0x7f66241a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.907+0000 7f6629600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66241a7a50 0x7f66241a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.907+0000 7f6629600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6624115a00 con 0x7f66241a7a50 2024-03-21T11:49:41.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.907+0000 7f6629600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6624115880 con 0x7f6624109340 2024-03-21T11:49:41.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.907+0000 7f6629600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6624115b80 con 0x7f662410fa70 2024-03-21T11:49:41.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.907+0000 7f6623e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6624109340 0x7f66241a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.907+0000 7f6623e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6624109340 0x7f66241a24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34060/0 (socket says 172.21.15.47:34060) 2024-03-21T11:49:41.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.907+0000 7f6623e00640 1 -- 172.21.15.47:0/570473183 learned_addr learned my addr 172.21.15.47:0/570473183 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:41.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.908+0000 7f6628c00640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66241a7a50 0x7f66241a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.908+0000 7f6623e00640 1 -- 172.21.15.47:0/570473183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f662410fa70 msgr2=0x7f66241a29f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T11:49:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.908+0000 7f6623400640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f662410fa70 0x7f66241a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.908+0000 7f6623e00640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f662410fa70 0x7f66241a29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.908+0000 7f6623e00640 1 -- 172.21.15.47:0/570473183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66241a7a50 msgr2=0x7f66241a9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.908+0000 7f6623e00640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66241a7a50 0x7f66241a9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.908+0000 7f6623e00640 1 -- 172.21.15.47:0/570473183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66241aa4b0 con 0x7f6624109340 2024-03-21T11:49:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.908+0000 7f6628c00640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66241a7a50 0x7f66241a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.908+0000 7f6623400640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f662410fa70 0x7f66241a29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:49:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.909+0000 7f6623e00640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6624109340 0x7f66241a24b0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f661400ebc0 tx=0x7f661400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.909+0000 7f6620c00640 1 -- 172.21.15.47:0/570473183 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614003b70 con 0x7f6624109340 2024-03-21T11:49:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.909+0000 7f6620c00640 1 -- 172.21.15.47:0/570473183 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6614003d10 con 0x7f6624109340 2024-03-21T11:49:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.909+0000 7f6620c00640 1 -- 172.21.15.47:0/570473183 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614019980 con 0x7f6624109340 2024-03-21T11:49:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.909+0000 7f6629600640 1 -- 172.21.15.47:0/570473183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66241aa7a0 con 0x7f6624109340 2024-03-21T11:49:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.909+0000 7f6629600640 1 -- 172.21.15.47:0/570473183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66241aaca0 con 0x7f6624109340 2024-03-21T11:49:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.915+0000 7f6620c00640 1 -- 172.21.15.47:0/570473183 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6614019b20 con 0x7f6624109340 2024-03-21T11:49:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.917+0000 7f6620c00640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f66080a3020 0x7f66080a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.917+0000 7f662d9c4740 1 Processor -- start 2024-03-21T11:49:41.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.917+0000 7f6620c00640 1 -- 172.21.15.47:0/570473183 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f66140d0040 con 0x7f6624109340 2024-03-21T11:49:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.917+0000 7f6623400640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f66080a3020 0x7f66080a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.918+0000 7f662d9c4740 1 -- start start 2024-03-21T11:49:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.918+0000 7f662d9c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfb3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.918+0000 7f662d9c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb4bc0 0x55f95bfb5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.918+0000 7f662d9c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb5580 0x55f95bfbbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.918+0000 7f662d9c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95bf182c0 con 0x55f95bfb4bc0 2024-03-21T11:49:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.918+0000 7f662d9c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95bec0c90 con 0x55f95bfb5580 2024-03-21T11:49:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.918+0000 7f662d9c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95bf14360 con 0x55f95bfb39f0 2024-03-21T11:49:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.918+0000 7f6623e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfb3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.918+0000 7f6623e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfb3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42812/0 (socket says 172.21.15.47:42812) 2024-03-21T11:49:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.918+0000 7f6623e00640 1 -- 172.21.15.47:0/3839280407 learned_addr learned my addr 172.21.15.47:0/3839280407 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.918+0000 7f6623400640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f66080a3020 0x7f66080a54e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f66241a39c0 tx=0x7f6610000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.919+0000 7f6628c00640 1 --2- 172.21.15.47:0/3839280407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb5580 0x55f95bfbbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.919+0000 7f6623400640 1 --2- 172.21.15.47:0/3839280407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb4bc0 0x55f95bfb5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.919+0000 7f6623e00640 1 -- 172.21.15.47:0/3839280407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb5580 msgr2=0x55f95bfbbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.919+0000 7f6623e00640 1 --2- 172.21.15.47:0/3839280407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb5580 0x55f95bfbbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.919+0000 7f6623e00640 1 -- 172.21.15.47:0/3839280407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb4bc0 msgr2=0x55f95bfb5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.919+0000 7f6623e00640 1 --2- 172.21.15.47:0/3839280407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb4bc0 0x55f95bfb5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.919+0000 7f6623e00640 1 -- 172.21.15.47:0/3839280407 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f95bee5080 con 0x55f95bfb39f0 2024-03-21T11:49:41.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.919+0000 7f6623400640 1 --2- 172.21.15.47:0/3839280407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb4bc0 0x55f95bfb5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T11:49:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.919+0000 7f6623e00640 1 --2- 172.21.15.47:0/3839280407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfb3e10 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f6614098d50 tx=0x7f661409bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa2fd8cb2dc74770 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.919+0000 7f6619200640 1 -- 172.21.15.47:0/3839280407 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66140b4020 con 0x55f95bfb39f0 2024-03-21T11:49:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.919+0000 7f6619200640 1 -- 172.21.15.47:0/3839280407 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66140a04d0 con 0x55f95bfb39f0 2024-03-21T11:49:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.919+0000 7f6619200640 1 -- 172.21.15.47:0/3839280407 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66140a00d0 con 0x55f95bfb39f0 2024-03-21T11:49:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.920+0000 7f662d9c4740 1 -- 172.21.15.47:0/3839280407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 msgr2=0x55f95bfb3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.920+0000 7f662d9c4740 1 --2- 172.21.15.47:0/3839280407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfb3e10 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f6614098d50 tx=0x7f661409bce0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.920+0000 7f662d9c4740 1 -- 172.21.15.47:0/3839280407 shutdown_connections 2024-03-21T11:49:41.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.920+0000 7f662d9c4740 1 --2- 172.21.15.47:0/3839280407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb5580 0x55f95bfbbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.920+0000 7f662d9c4740 1 --2- 172.21.15.47:0/3839280407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb4bc0 0x55f95bfb5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.920+0000 7f662d9c4740 1 --2- 172.21.15.47:0/3839280407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfb3e10 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.920+0000 7f662d9c4740 1 -- 172.21.15.47:0/3839280407 >> 172.21.15.47:0/3839280407 conn(0x55f95bfaede0 msgr2=0x55f95bfaf1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.921+0000 7f662d9c4740 1 -- 172.21.15.47:0/3839280407 shutdown_connections 2024-03-21T11:49:41.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.921+0000 7f662d9c4740 1 -- 172.21.15.47:0/3839280407 wait complete. 2024-03-21T11:49:41.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.921+0000 7f662d9c4740 1 -- start start 2024-03-21T11:49:41.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.921+0000 7f662d9c4740 1 Processor -- start 2024-03-21T11:49:41.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.922+0000 7f662d9c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfbdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.922+0000 7f662d9c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfbe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.922+0000 7f662d9c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb5580 0x55f95bfc5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.922+0000 7f662d9c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95bfbc530 con 0x55f95bfb5580 2024-03-21T11:49:41.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.922+0000 7f662d9c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95bed40d0 con 0x55f95bfb4bc0 2024-03-21T11:49:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.922+0000 7f662d9c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95bfbc6b0 con 0x55f95bfb39f0 2024-03-21T11:49:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.922+0000 7f662d9c4740 10 client.0 _ll_register_callbacks cb 0x55f95bed8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.922+0000 7f6623e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfbdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.922+0000 7f6623400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfbe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.922+0000 7f6623e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfbdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42826/0 (socket says 172.21.15.47:42826) 2024-03-21T11:49:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.922+0000 7f6623e00640 1 -- 172.21.15.47:0/1492182930 learned_addr learned my addr 172.21.15.47:0/1492182930 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.922+0000 7f6628c00640 1 --2- 172.21.15.47:0/1492182930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb5580 0x55f95bfc5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.923+0000 7f6623e00640 1 -- 172.21.15.47:0/1492182930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 msgr2=0x55f95bfbe150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.923+0000 7f6623e00640 1 --2- 172.21.15.47:0/1492182930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfbe150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.923+0000 7f6623e00640 1 -- 172.21.15.47:0/1492182930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb5580 msgr2=0x55f95bfc5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.923+0000 7f6623e00640 1 --2- 172.21.15.47:0/1492182930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb5580 0x55f95bfc5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.923+0000 7f6623e00640 1 -- 172.21.15.47:0/1492182930 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f95bc2c3f0 con 0x55f95bfb39f0 2024-03-21T11:49:41.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.923+0000 7f6628c00640 1 --2- 172.21.15.47:0/1492182930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb5580 0x55f95bfc5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.923+0000 7f6623e00640 1 --2- 172.21.15.47:0/1492182930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfbdc10 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f661409c080 tx=0x7f66140a4d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38cc10ef57a8314a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.923+0000 7f660d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.923+0000 7f660d600640 1 -- 172.21.15.47:0/1492182930 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66140b9030 con 0x55f95bfb39f0 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.923+0000 7f660d600640 1 -- 172.21.15.47:0/1492182930 <== mon.1 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f661409a680 con 0x55f95bfb39f0 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.923+0000 7f662d9c4740 1 -- 172.21.15.47:0/1492182930 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66140a04d0 con 0x55f95bfb39f0 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.924+0000 7f660d600640 1 -- 172.21.15.47:0/1492182930 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66140b4030 con 0x55f95bfb39f0 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.924+0000 7f662d9c4740 10 client.33712 fetch_fsmap learned FSMap version 43 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.924+0000 7f662d9c4740 1 -- 172.21.15.47:0/1492182930 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f95bcf67d0 con 0x55f95bfb39f0 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.924+0000 7f660d600640 1 -- 172.21.15.47:0/1492182930 <== mon.1 v2:172.21.15.102:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66140a04d0 con 0x55f95bfb39f0 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.924+0000 7f660d600640 1 -- 172.21.15.47:0/1492182930 <== mon.1 v2:172.21.15.102:3300/0 5 ==== fsmap(e 43) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f661409b7d0 con 0x55f95bfb39f0 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.924+0000 7f662d9c4740 10 client.33712 fetch_fsmap finished waiting for FSMap version 43 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.924+0000 7f662d9c4740 10 client.33712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.924+0000 7f662d9c4740 20 client.33712 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.924+0000 7f662d9c4740 1 --2- 172.21.15.47:0/1492182930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f95bfc9260 0x55f95bfcb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.924+0000 7f662d9c4740 4 client.33712 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T11:49:41.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.924+0000 7f662d9c4740 1 -- 172.21.15.47:0/1492182930 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f95bf25800 con 0x55f95bfc9260 2024-03-21T11:49:41.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.924+0000 7f6623400640 1 --2- 172.21.15.47:0/1492182930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f95bfc9260 0x55f95bfcb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.925+0000 7f6623400640 1 --2- 172.21.15.47:0/1492182930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f95bfc9260 0x55f95bfcb650 crc :-1 s=READY pgs=621 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=583e4811e0e36c51 server_cookie=21b1240ee2a87d8c in_seq=0 out_seq=0 2024-03-21T11:49:41.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.925+0000 7f660d600640 10 client.33712 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.926+0000 7f660d600640 1 -- 172.21.15.47:0/1492182930 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f95bf25800 con 0x55f95bfc9260 2024-03-21T11:49:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.926+0000 7f660d600640 10 client.33712 handle_command_reply: tid=0 2024-03-21T11:49:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.926+0000 7f662d9c4740 1 client.33712 shutdown 2024-03-21T11:49:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.926+0000 7f662d9c4740 1 -- 172.21.15.47:0/1492182930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 msgr2=0x55f95bfbdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.926+0000 7f662d9c4740 1 --2- 172.21.15.47:0/1492182930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfbdc10 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f661409c080 tx=0x7f66140a4d90 comp rx=0 tx=0).stop 2024-03-21T11:49:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.926+0000 7f662d9c4740 1 -- 172.21.15.47:0/1492182930 shutdown_connections 2024-03-21T11:49:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.926+0000 7f662d9c4740 1 --2- 172.21.15.47:0/1492182930 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f95bfc9260 0x55f95bfcb650 crc :-1 s=READY pgs=621 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.926+0000 7f662d9c4740 1 --2- 172.21.15.47:0/1492182930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb5580 0x55f95bfc5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.926+0000 7f662d9c4740 1 --2- 172.21.15.47:0/1492182930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfbe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.926+0000 7f660d600640 0 client.33712 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.926+0000 7f662d9c4740 1 --2- 172.21.15.47:0/1492182930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfbdc10 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.926+0000 7f662d9c4740 1 -- 172.21.15.47:0/1492182930 >> 172.21.15.47:0/1492182930 conn(0x55f95bfaede0 msgr2=0x55f95bfae160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.927+0000 7f662d9c4740 1 -- 172.21.15.47:0/1492182930 shutdown_connections 2024-03-21T11:49:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.927+0000 7f662d9c4740 1 -- 172.21.15.47:0/1492182930 wait complete. 2024-03-21T11:49:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.927+0000 7f662d9c4740 20 client.33712 trim_cache size 0 max 16384 2024-03-21T11:49:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.943+0000 7f662d9c4740 1 Processor -- start 2024-03-21T11:49:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.943+0000 7f662d9c4740 1 -- start start 2024-03-21T11:49:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f662d9c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfbbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f662d9c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfc5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f662d9c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb5580 0x55f95bfb7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f662d9c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66241b4fa0 con 0x55f95bfb5580 2024-03-21T11:49:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f662d9c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66241b6a10 con 0x55f95bfb4bc0 2024-03-21T11:49:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f662d9c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66241b5660 con 0x55f95bfb39f0 2024-03-21T11:49:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f6623e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfbbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f6623400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfc5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f6623e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfbbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42830/0 (socket says 172.21.15.47:42830) 2024-03-21T11:49:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f6623400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfc5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34096/0 (socket says 172.21.15.47:34096) 2024-03-21T11:49:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f6623e00640 1 -- 172.21.15.47:0/2855643762 learned_addr learned my addr 172.21.15.47:0/2855643762 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f6628c00640 1 --2- 172.21.15.47:0/2855643762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb5580 0x55f95bfb7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f6623e00640 1 -- 172.21.15.47:0/2855643762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 msgr2=0x55f95bfc5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f6623e00640 1 --2- 172.21.15.47:0/2855643762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfc5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f6623e00640 1 -- 172.21.15.47:0/2855643762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb5580 msgr2=0x55f95bfb7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f6623e00640 1 --2- 172.21.15.47:0/2855643762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb5580 0x55f95bfb7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.944+0000 7f6623e00640 1 -- 172.21.15.47:0/2855643762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f95bfbc7e0 con 0x55f95bfb39f0 2024-03-21T11:49:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.945+0000 7f6623e00640 1 --2- 172.21.15.47:0/2855643762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfbbaf0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f66140987e0 tx=0x7f661409bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fc2800335bc3456 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.945+0000 7f660fe00640 1 -- 172.21.15.47:0/2855643762 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66140b9070 con 0x55f95bfb39f0 2024-03-21T11:49:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.945+0000 7f660fe00640 1 -- 172.21.15.47:0/2855643762 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f661409ca80 con 0x55f95bfb39f0 2024-03-21T11:49:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.945+0000 7f660fe00640 1 -- 172.21.15.47:0/2855643762 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661409b7d0 con 0x55f95bfb39f0 2024-03-21T11:49:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.946+0000 7f662d9c4740 1 -- 172.21.15.47:0/2855643762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 msgr2=0x55f95bfbbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.946+0000 7f662d9c4740 1 --2- 172.21.15.47:0/2855643762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfbbaf0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f66140987e0 tx=0x7f661409bce0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.946+0000 7f662d9c4740 1 -- 172.21.15.47:0/2855643762 shutdown_connections 2024-03-21T11:49:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.946+0000 7f662d9c4740 1 --2- 172.21.15.47:0/2855643762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb5580 0x55f95bfb7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.946+0000 7f662d9c4740 1 --2- 172.21.15.47:0/2855643762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfc5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.946+0000 7f662d9c4740 1 --2- 172.21.15.47:0/2855643762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb39f0 0x55f95bfbbaf0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.946+0000 7f662d9c4740 1 -- 172.21.15.47:0/2855643762 >> 172.21.15.47:0/2855643762 conn(0x55f95bfaede0 msgr2=0x55f95bfae160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.946+0000 7f662d9c4740 1 -- 172.21.15.47:0/2855643762 shutdown_connections 2024-03-21T11:49:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.946+0000 7f662d9c4740 1 -- 172.21.15.47:0/2855643762 wait complete. 2024-03-21T11:49:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.946+0000 7f662d9c4740 1 -- start start 2024-03-21T11:49:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.946+0000 7f662d9c4740 1 Processor -- start 2024-03-21T11:49:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f662d9c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb39f0 0x55f95bfc12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f662d9c4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfc17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f662d9c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb5580 0x55f95bfc47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f662d9c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66241b63b0 con 0x55f95bfb39f0 2024-03-21T11:49:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f662d9c4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66241b5d20 con 0x55f95bfb4bc0 2024-03-21T11:49:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f662d9c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66241b3160 con 0x55f95bfb5580 2024-03-21T11:49:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f662d9c4740 10 client.0 _ll_register_callbacks cb 0x7f661409c1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T11:49:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f6623400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfc17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f6623400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfc17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34108/0 (socket says 172.21.15.47:34108) 2024-03-21T11:49:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f6623400640 1 -- 172.21.15.47:0/544589940 learned_addr learned my addr 172.21.15.47:0/544589940 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f6623e00640 1 --2- 172.21.15.47:0/544589940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb39f0 0x55f95bfc12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f6628c00640 1 --2- 172.21.15.47:0/544589940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb5580 0x55f95bfc47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f6623400640 1 -- 172.21.15.47:0/544589940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb5580 msgr2=0x55f95bfc47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f6623400640 1 --2- 172.21.15.47:0/544589940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb5580 0x55f95bfc47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f6623400640 1 -- 172.21.15.47:0/544589940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb39f0 msgr2=0x55f95bfc12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T11:49:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.947+0000 7f6623400640 1 --2- 172.21.15.47:0/544589940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb39f0 0x55f95bfc12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.948+0000 7f6623400640 1 -- 172.21.15.47:0/544589940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f95bfc4dd0 con 0x55f95bfb4bc0 2024-03-21T11:49:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.948+0000 7f6628c00640 1 --2- 172.21.15.47:0/544589940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb5580 0x55f95bfc47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.948+0000 7f6623400640 1 --2- 172.21.15.47:0/544589940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfc17f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f6610006010 tx=0x7f661002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4756175cacd86602 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.948+0000 7f660d600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T11:49:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.948+0000 7f660d600640 1 -- 172.21.15.47:0/544589940 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610050ac0 con 0x55f95bfb4bc0 2024-03-21T11:49:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.948+0000 7f662d9c4740 1 -- 172.21.15.47:0/544589940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f95bf1fb50 con 0x55f95bfb4bc0 2024-03-21T11:49:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.948+0000 7f660d600640 1 -- 172.21.15.47:0/544589940 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f6610050c60 con 0x55f95bfb4bc0 2024-03-21T11:49:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.948+0000 7f660d600640 1 -- 172.21.15.47:0/544589940 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6610050e60 con 0x55f95bfb4bc0 2024-03-21T11:49:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.949+0000 7f662d9c4740 10 client.33812 fetch_fsmap learned FSMap version 43 2024-03-21T11:49:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.949+0000 7f662d9c4740 1 -- 172.21.15.47:0/544589940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f95c021380 con 0x55f95bfb4bc0 2024-03-21T11:49:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.949+0000 7f660d600640 1 -- 172.21.15.47:0/544589940 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f95bf1fb50 con 0x55f95bfb4bc0 2024-03-21T11:49:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.950+0000 7f660d600640 1 -- 172.21.15.47:0/544589940 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 43) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f6610048040 con 0x55f95bfb4bc0 2024-03-21T11:49:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.950+0000 7f662d9c4740 10 client.33812 fetch_fsmap finished waiting for FSMap version 43 2024-03-21T11:49:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.950+0000 7f662d9c4740 10 client.33812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T11:49:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.950+0000 7f662d9c4740 20 client.33812 populate_metadata read hostname 'smithi047' 2024-03-21T11:49:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.950+0000 7f662d9c4740 1 --2- 172.21.15.47:0/544589940 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f95c021cb0 0x55f95c0240d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.950+0000 7f662d9c4740 4 client.33812 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T11:49:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.950+0000 7f662d9c4740 1 -- 172.21.15.47:0/544589940 --> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f95be9d500 con 0x55f95c021cb0 2024-03-21T11:49:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.950+0000 7f6623e00640 1 --2- 172.21.15.47:0/544589940 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f95c021cb0 0x55f95c0240d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.951+0000 7f660d600640 10 client.33812 ms_handle_connect on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.951+0000 7f6623e00640 1 --2- 172.21.15.47:0/544589940 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f95c021cb0 0x55f95c0240d0 crc :-1 s=READY pgs=622 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6011601529322fdb server_cookie=93a615a02b15833e in_seq=0 out_seq=0 2024-03-21T11:49:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.951+0000 7f660d600640 1 -- 172.21.15.47:0/544589940 <== mds.0 v2:172.21.15.100:6838/2708345792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f95be9d500 con 0x55f95c021cb0 2024-03-21T11:49:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.951+0000 7f660d600640 10 client.33812 handle_command_reply: tid=0 2024-03-21T11:49:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.951+0000 7f662d9c4740 1 client.33812 shutdown 2024-03-21T11:49:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.951+0000 7f662d9c4740 1 -- 172.21.15.47:0/544589940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 msgr2=0x55f95bfc17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.951+0000 7f662d9c4740 1 --2- 172.21.15.47:0/544589940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfc17f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f6610006010 tx=0x7f661002bc30 comp rx=0 tx=0).stop 2024-03-21T11:49:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.952+0000 7f662d9c4740 1 -- 172.21.15.47:0/544589940 shutdown_connections 2024-03-21T11:49:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.952+0000 7f662d9c4740 1 --2- 172.21.15.47:0/544589940 >> [v2:172.21.15.100:6838/2708345792,v1:172.21.15.100:6839/2708345792] conn(0x55f95c021cb0 0x55f95c0240d0 crc :-1 s=READY pgs=622 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.952+0000 7f662d9c4740 1 --2- 172.21.15.47:0/544589940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f95bfb5580 0x55f95bfc47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.952+0000 7f662d9c4740 1 --2- 172.21.15.47:0/544589940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f95bfb4bc0 0x55f95bfc17f0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.952+0000 7f662d9c4740 1 --2- 172.21.15.47:0/544589940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f95bfb39f0 0x55f95bfc12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.952+0000 7f662d9c4740 1 -- 172.21.15.47:0/544589940 >> 172.21.15.47:0/544589940 conn(0x55f95bfaede0 msgr2=0x55f95bfb28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.952+0000 7f660d600640 0 client.33812 ms_handle_reset on v2:172.21.15.100:6838/2708345792 2024-03-21T11:49:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.952+0000 7f662d9c4740 1 -- 172.21.15.47:0/544589940 shutdown_connections 2024-03-21T11:49:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.952+0000 7f662d9c4740 1 -- 172.21.15.47:0/544589940 wait complete. 2024-03-21T11:49:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.952+0000 7f662d9c4740 20 client.33812 trim_cache size 0 max 16384 2024-03-21T11:49:41.956 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T11:49:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.953+0000 7f6629600640 1 -- 172.21.15.47:0/570473183 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f66080a3020 msgr2=0x7f66080a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.953+0000 7f6629600640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f66080a3020 0x7f66080a54e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f66241a39c0 tx=0x7f6610000bc0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.953+0000 7f6629600640 1 -- 172.21.15.47:0/570473183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6624109340 msgr2=0x7f66241a24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.953+0000 7f6629600640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6624109340 0x7f66241a24b0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f661400ebc0 tx=0x7f661400c6a0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.953+0000 7f6629600640 1 -- 172.21.15.47:0/570473183 shutdown_connections 2024-03-21T11:49:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.953+0000 7f6629600640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f66080a3020 0x7f66080a54e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.953+0000 7f6629600640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66241a7a50 0x7f66241a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.953+0000 7f6629600640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f662410fa70 0x7f66241a29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.953+0000 7f6629600640 1 --2- 172.21.15.47:0/570473183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6624109340 0x7f66241a24b0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.953+0000 7f6629600640 1 -- 172.21.15.47:0/570473183 >> 172.21.15.47:0/570473183 conn(0x7f662406e580 msgr2=0x7f66241097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.953+0000 7f6629600640 1 -- 172.21.15.47:0/570473183 shutdown_connections 2024-03-21T11:49:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:41.953+0000 7f6629600640 1 -- 172.21.15.47:0/570473183 wait complete. 2024-03-21T11:49:42.973 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-21T11:49:42.973 INFO:tasks.fwd_scrub:done joining 2024-03-21T11:49:42.973 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-21T11:49:42.984 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:49:43.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:42 smithi047 ceph-mon[31351]: pgmap v1192: 129 pgs: 129 active+clean; 553 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 2.9 MiB/s wr, 155 op/s 2024-03-21T11:49:43.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:42 smithi100 ceph-mon[36245]: pgmap v1192: 129 pgs: 129 active+clean; 553 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 2.9 MiB/s wr, 155 op/s 2024-03-21T11:49:43.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:42 smithi102 ceph-mon[36748]: pgmap v1192: 129 pgs: 129 active+clean; 553 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 2.9 MiB/s wr, 155 op/s 2024-03-21T11:49:43.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.281+0000 7fb6ada00640 1 -- 172.21.15.47:0/2756971272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a81038e0 msgr2=0x7fb6a8103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.281+0000 7fb6ada00640 1 --2- 172.21.15.47:0/2756971272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a81038e0 0x7fb6a8103d00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fb698009900 tx=0x7fb69802bc40 comp rx=0 tx=0).stop 2024-03-21T11:49:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.282+0000 7fb6ada00640 1 -- 172.21.15.47:0/2756971272 shutdown_connections 2024-03-21T11:49:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.282+0000 7fb6ada00640 1 --2- 172.21.15.47:0/2756971272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a8105490 0x7fb6a810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.282+0000 7fb6ada00640 1 --2- 172.21.15.47:0/2756971272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6a8104ad0 0x7fb6a8104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.282+0000 7fb6ada00640 1 --2- 172.21.15.47:0/2756971272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a81038e0 0x7fb6a8103d00 secure :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fb698009900 tx=0x7fb69802bc40 comp rx=0 tx=0).stop 2024-03-21T11:49:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.282+0000 7fb6ada00640 1 -- 172.21.15.47:0/2756971272 >> 172.21.15.47:0/2756971272 conn(0x7fb6a80fee50 msgr2=0x7fb6a8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.282+0000 7fb6ada00640 1 -- 172.21.15.47:0/2756971272 shutdown_connections 2024-03-21T11:49:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.282+0000 7fb6ada00640 1 -- 172.21.15.47:0/2756971272 wait complete. 2024-03-21T11:49:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.283+0000 7fb6ada00640 1 Processor -- start 2024-03-21T11:49:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.283+0000 7fb6ada00640 1 -- start start 2024-03-21T11:49:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.284+0000 7fb6ada00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a8104ad0 0x7fb6a8199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:43.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.284+0000 7fb6ada00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6a8105490 0x7fb6a819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:43.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.284+0000 7fb6ada00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a819f370 0x7fb6a81a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:43.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.284+0000 7fb6ada00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a810f5b0 con 0x7fb6a8105490 2024-03-21T11:49:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.284+0000 7fb6ada00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a810f430 con 0x7fb6a8104ad0 2024-03-21T11:49:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.284+0000 7fb6ada00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a810f730 con 0x7fb6a819f370 2024-03-21T11:49:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.284+0000 7fb6a6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a8104ad0 0x7fb6a8199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.284+0000 7fb6a6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a8104ad0 0x7fb6a8199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34120/0 (socket says 172.21.15.47:34120) 2024-03-21T11:49:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.284+0000 7fb6a6a00640 1 -- 172.21.15.47:0/1612330613 learned_addr learned my addr 172.21.15.47:0/1612330613 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.284+0000 7fb6a6000640 1 --2- 172.21.15.47:0/1612330613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6a8105490 0x7fb6a819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.285+0000 7fb6a6a00640 1 -- 172.21.15.47:0/1612330613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a819f370 msgr2=0x7fb6a81a1760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T11:49:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.285+0000 7fb6a6a00640 1 --2- 172.21.15.47:0/1612330613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a819f370 0x7fb6a81a1760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.285+0000 7fb6a6a00640 1 -- 172.21.15.47:0/1612330613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6a8105490 msgr2=0x7fb6a819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.285+0000 7fb6a6a00640 1 --2- 172.21.15.47:0/1612330613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6a8105490 0x7fb6a819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.285+0000 7fb6a6a00640 1 -- 172.21.15.47:0/1612330613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a81a1dd0 con 0x7fb6a8104ad0 2024-03-21T11:49:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.285+0000 7fb6a6000640 1 --2- 172.21.15.47:0/1612330613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6a8105490 0x7fb6a819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.286+0000 7fb6a6a00640 1 --2- 172.21.15.47:0/1612330613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a8104ad0 0x7fb6a8199dd0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb698036040 tx=0x7fb698004ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.286+0000 7fb69f400640 1 -- 172.21.15.47:0/1612330613 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb698041070 con 0x7fb6a8104ad0 2024-03-21T11:49:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.286+0000 7fb69f400640 1 -- 172.21.15.47:0/1612330613 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6980044d0 con 0x7fb6a8104ad0 2024-03-21T11:49:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.286+0000 7fb69f400640 1 -- 172.21.15.47:0/1612330613 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb698003df0 con 0x7fb6a8104ad0 2024-03-21T11:49:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.286+0000 7fb6ada00640 1 -- 172.21.15.47:0/1612330613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a81a2060 con 0x7fb6a8104ad0 2024-03-21T11:49:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.286+0000 7fb6ada00640 1 -- 172.21.15.47:0/1612330613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a81a24d0 con 0x7fb6a8104ad0 2024-03-21T11:49:43.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.287+0000 7fb6ada00640 1 -- 172.21.15.47:0/1612330613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a810ac80 con 0x7fb6a8104ad0 2024-03-21T11:49:43.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.289+0000 7fb69f400640 1 -- 172.21.15.47:0/1612330613 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb698044070 con 0x7fb6a8104ad0 2024-03-21T11:49:43.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.291+0000 7fb69f400640 1 --2- 172.21.15.47:0/1612330613 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb68c0a2ff0 0x7fb68c0a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:43.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.291+0000 7fb69f400640 1 -- 172.21.15.47:0/1612330613 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb6980f1520 con 0x7fb6a8104ad0 2024-03-21T11:49:43.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.291+0000 7fb6a6000640 1 --2- 172.21.15.47:0/1612330613 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb68c0a2ff0 0x7fb68c0a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:43.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.292+0000 7fb6a6000640 1 --2- 172.21.15.47:0/1612330613 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb68c0a2ff0 0x7fb68c0a54b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb6940058c0 tx=0x7fb694005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:43.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.293+0000 7fb69f400640 1 -- 172.21.15.47:0/1612330613 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb6980bdf60 con 0x7fb6a8104ad0 2024-03-21T11:49:43.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.642+0000 7fb6ada00640 1 -- 172.21.15.47:0/1612330613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6a80622a0 con 0x7fb6a8104ad0 2024-03-21T11:49:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.645+0000 7fb69f400640 1 -- 172.21.15.47:0/1612330613 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24747 (secure 0 0 0) 0x7fb6980bd860 con 0x7fb6a8104ad0 2024-03-21T11:49:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:49:43.650 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":96,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:45:38.900262+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:49:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.652+0000 7fb6ada00640 1 -- 172.21.15.47:0/1612330613 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb68c0a2ff0 msgr2=0x7fb68c0a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.652+0000 7fb6ada00640 1 --2- 172.21.15.47:0/1612330613 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb68c0a2ff0 0x7fb68c0a54b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb6940058c0 tx=0x7fb694005850 comp rx=0 tx=0).stop 2024-03-21T11:49:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.652+0000 7fb6ada00640 1 -- 172.21.15.47:0/1612330613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a8104ad0 msgr2=0x7fb6a8199dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.652+0000 7fb6ada00640 1 --2- 172.21.15.47:0/1612330613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a8104ad0 0x7fb6a8199dd0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb698036040 tx=0x7fb698004ec0 comp rx=0 tx=0).stop 2024-03-21T11:49:43.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.653+0000 7fb6ada00640 1 -- 172.21.15.47:0/1612330613 shutdown_connections 2024-03-21T11:49:43.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.653+0000 7fb6ada00640 1 --2- 172.21.15.47:0/1612330613 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb68c0a2ff0 0x7fb68c0a54b0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:43.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.653+0000 7fb6ada00640 1 --2- 172.21.15.47:0/1612330613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a819f370 0x7fb6a81a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:43.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.653+0000 7fb6ada00640 1 --2- 172.21.15.47:0/1612330613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6a8105490 0x7fb6a819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:43.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.653+0000 7fb6ada00640 1 --2- 172.21.15.47:0/1612330613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a8104ad0 0x7fb6a8199dd0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:43.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.653+0000 7fb6ada00640 1 -- 172.21.15.47:0/1612330613 >> 172.21.15.47:0/1612330613 conn(0x7fb6a80fee50 msgr2=0x7fb6a8107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:43.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.653+0000 7fb6ada00640 1 -- 172.21.15.47:0/1612330613 shutdown_connections 2024-03-21T11:49:43.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.654+0000 7fb6ada00640 1 -- 172.21.15.47:0/1612330613 wait complete. 2024-03-21T11:49:43.683 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T11:15:35.545245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T11:20:47.016174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T11:20:48.108506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T11:20:52.254154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T11:49:43.684 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T11:49:43.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.917+0000 7f0e73e00640 1 -- 172.21.15.47:0/3455378940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e74104930 msgr2=0x7f0e74106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:43.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.917+0000 7f0e73e00640 1 --2- 172.21.15.47:0/3455378940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e74104930 0x7f0e74106d20 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009900 tx=0x7f0e5c02bc30 comp rx=0 tx=0).stop 2024-03-21T11:49:43.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.918+0000 7f0e73e00640 1 -- 172.21.15.47:0/3455378940 shutdown_connections 2024-03-21T11:49:43.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.918+0000 7f0e73e00640 1 --2- 172.21.15.47:0/3455378940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e74107260 0x7f0e741096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:43.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.918+0000 7f0e73e00640 1 --2- 172.21.15.47:0/3455378940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e74104930 0x7f0e74106d20 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:43.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.918+0000 7f0e73e00640 1 --2- 172.21.15.47:0/3455378940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e74102000 0x7f0e741043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:43.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.918+0000 7f0e73e00640 1 -- 172.21.15.47:0/3455378940 >> 172.21.15.47:0/3455378940 conn(0x7f0e740fba10 msgr2=0x7f0e740fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:43.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.918+0000 7f0e73e00640 1 -- 172.21.15.47:0/3455378940 shutdown_connections 2024-03-21T11:49:43.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.918+0000 7f0e73e00640 1 -- 172.21.15.47:0/3455378940 wait complete. 2024-03-21T11:49:43.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.919+0000 7f0e73e00640 1 Processor -- start 2024-03-21T11:49:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.919+0000 7f0e73e00640 1 -- start start 2024-03-21T11:49:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.919+0000 7f0e73e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e74102000 0x7f0e74197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.919+0000 7f0e73e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e74104930 0x7f0e74197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.919+0000 7f0e73e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e74107260 0x7f0e7419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.919+0000 7f0e73e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7410d210 con 0x7f0e74102000 2024-03-21T11:49:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.919+0000 7f0e73e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7410d090 con 0x7f0e74107260 2024-03-21T11:49:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.919+0000 7f0e73e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7410d390 con 0x7f0e74104930 2024-03-21T11:49:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.919+0000 7f0e6be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e74104930 0x7f0e74197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.919+0000 7f0e6be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e74104930 0x7f0e74197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42870/0 (socket says 172.21.15.47:42870) 2024-03-21T11:49:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.919+0000 7f0e6be00640 1 -- 172.21.15.47:0/744358567 learned_addr learned my addr 172.21.15.47:0/744358567 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.919+0000 7f0e73400640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e74107260 0x7f0e7419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.919+0000 7f0e72a00640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e74102000 0x7f0e74197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:43.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.920+0000 7f0e6be00640 1 -- 172.21.15.47:0/744358567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e74107260 msgr2=0x7f0e7419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.920+0000 7f0e6be00640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e74107260 0x7f0e7419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.920+0000 7f0e6be00640 1 -- 172.21.15.47:0/744358567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e74102000 msgr2=0x7f0e74197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.920+0000 7f0e6be00640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e74102000 0x7f0e74197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.920+0000 7f0e6be00640 1 -- 172.21.15.47:0/744358567 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7419f4b0 con 0x7f0e74104930 2024-03-21T11:49:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.920+0000 7f0e73400640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e74107260 0x7f0e7419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.920+0000 7f0e6be00640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e74104930 0x7f0e74197dd0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c000c00 tx=0x7f0e5c02c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.920+0000 7f0e6b400640 1 -- 172.21.15.47:0/744358567 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c041070 con 0x7f0e74104930 2024-03-21T11:49:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.920+0000 7f0e73e00640 1 -- 172.21.15.47:0/744358567 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7419f740 con 0x7f0e74104930 2024-03-21T11:49:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.920+0000 7f0e6b400640 1 -- 172.21.15.47:0/744358567 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e5c004450 con 0x7f0e74104930 2024-03-21T11:49:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.920+0000 7f0e6b400640 1 -- 172.21.15.47:0/744358567 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c034440 con 0x7f0e74104930 2024-03-21T11:49:43.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.920+0000 7f0e73e00640 1 -- 172.21.15.47:0/744358567 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7419fc30 con 0x7f0e74104930 2024-03-21T11:49:43.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.921+0000 7f0e73e00640 1 -- 172.21.15.47:0/744358567 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e740fd500 con 0x7f0e74104930 2024-03-21T11:49:43.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.923+0000 7f0e6b400640 1 -- 172.21.15.47:0/744358567 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0e5c044070 con 0x7f0e74104930 2024-03-21T11:49:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.924+0000 7f0e6b400640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0e580a2f20 0x7f0e580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.925+0000 7f0e6b400640 1 -- 172.21.15.47:0/744358567 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f0e5c0f1250 con 0x7f0e74104930 2024-03-21T11:49:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.925+0000 7f0e72a00640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0e580a2f20 0x7f0e580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.925+0000 7f0e72a00640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0e580a2f20 0x7f0e580a53e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009bd0 tx=0x7f0e64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:43.933+0000 7f0e6b400640 1 -- 172.21.15.47:0/744358567 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0e5c0bd730 con 0x7f0e74104930 2024-03-21T11:49:44.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.135+0000 7f0e73e00640 1 -- 172.21.15.47:0/744358567 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e740622a0 con 0x7f0e74104930 2024-03-21T11:49:44.175 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:43 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1612330613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:49:44.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.136+0000 7f0e6b400640 1 -- 172.21.15.47:0/744358567 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e5c0c17d0 con 0x7f0e74104930 2024-03-21T11:49:44.175 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T11:49:44.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.139+0000 7f0e73e00640 1 -- 172.21.15.47:0/744358567 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0e580a2f20 msgr2=0x7f0e580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:44.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.139+0000 7f0e73e00640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0e580a2f20 0x7f0e580a53e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009bd0 tx=0x7f0e64005f90 comp rx=0 tx=0).stop 2024-03-21T11:49:44.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.139+0000 7f0e73e00640 1 -- 172.21.15.47:0/744358567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e74104930 msgr2=0x7f0e74197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:44.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.139+0000 7f0e73e00640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e74104930 0x7f0e74197dd0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c000c00 tx=0x7f0e5c02c640 comp rx=0 tx=0).stop 2024-03-21T11:49:44.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.139+0000 7f0e73e00640 1 -- 172.21.15.47:0/744358567 shutdown_connections 2024-03-21T11:49:44.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.139+0000 7f0e73e00640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f0e580a2f20 0x7f0e580a53e0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.139+0000 7f0e73e00640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e74107260 0x7f0e7419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.139+0000 7f0e73e00640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0e74104930 0x7f0e74197dd0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.139+0000 7f0e73e00640 1 --2- 172.21.15.47:0/744358567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e74102000 0x7f0e74197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.139+0000 7f0e73e00640 1 -- 172.21.15.47:0/744358567 >> 172.21.15.47:0/744358567 conn(0x7f0e740fba10 msgr2=0x7f0e74102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:44.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.139+0000 7f0e73e00640 1 -- 172.21.15.47:0/744358567 shutdown_connections 2024-03-21T11:49:44.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.139+0000 7f0e73e00640 1 -- 172.21.15.47:0/744358567 wait complete. 2024-03-21T11:49:44.178 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T11:49:44.202 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:43 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1612330613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:49:44.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:43 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1612330613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:49:44.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.398+0000 7fcc7ce00640 1 -- 172.21.15.47:0/2649734595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc78104990 msgr2=0x7fcc78104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:44.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.399+0000 7fcc7ce00640 1 --2- 172.21.15.47:0/2649734595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc78104990 0x7fcc78104e10 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fcc60009900 tx=0x7fcc6002bc50 comp rx=0 tx=0).stop 2024-03-21T11:49:44.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.399+0000 7fcc7ce00640 1 -- 172.21.15.47:0/2649734595 shutdown_connections 2024-03-21T11:49:44.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.399+0000 7fcc7ce00640 1 --2- 172.21.15.47:0/2649734595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc78105350 0x7fcc7810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.399+0000 7fcc7ce00640 1 --2- 172.21.15.47:0/2649734595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc78104990 0x7fcc78104e10 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.399+0000 7fcc7ce00640 1 --2- 172.21.15.47:0/2649734595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc781037a0 0x7fcc78103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.399+0000 7fcc7ce00640 1 -- 172.21.15.47:0/2649734595 >> 172.21.15.47:0/2649734595 conn(0x7fcc780fed10 msgr2=0x7fcc78101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:44.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.399+0000 7fcc7ce00640 1 -- 172.21.15.47:0/2649734595 shutdown_connections 2024-03-21T11:49:44.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.399+0000 7fcc7ce00640 1 -- 172.21.15.47:0/2649734595 wait complete. 2024-03-21T11:49:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.400+0000 7fcc7ce00640 1 Processor -- start 2024-03-21T11:49:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.400+0000 7fcc7ce00640 1 -- start start 2024-03-21T11:49:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.400+0000 7fcc7ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc781037a0 0x7fcc78199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.400+0000 7fcc7ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc78104990 0x7fcc7819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.400+0000 7fcc7ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc78105350 0x7fcc781a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.400+0000 7fcc7ce00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7810f400 con 0x7fcc78104990 2024-03-21T11:49:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.400+0000 7fcc7ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7810f280 con 0x7fcc781037a0 2024-03-21T11:49:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.400+0000 7fcc7ce00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7810f580 con 0x7fcc78105350 2024-03-21T11:49:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.400+0000 7fcc77400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc781037a0 0x7fcc78199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.400+0000 7fcc77400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc781037a0 0x7fcc78199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34148/0 (socket says 172.21.15.47:34148) 2024-03-21T11:49:44.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.400+0000 7fcc77400640 1 -- 172.21.15.47:0/112143849 learned_addr learned my addr 172.21.15.47:0/112143849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.400+0000 7fcc77e00640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc78105350 0x7fcc781a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.400+0000 7fcc6ea00640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc78104990 0x7fcc7819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.401+0000 7fcc77400640 1 -- 172.21.15.47:0/112143849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc78105350 msgr2=0x7fcc781a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.401+0000 7fcc77400640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc78105350 0x7fcc781a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.401+0000 7fcc77400640 1 -- 172.21.15.47:0/112143849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc78104990 msgr2=0x7fcc7819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.401+0000 7fcc77400640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc78104990 0x7fcc7819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.401+0000 7fcc77400640 1 -- 172.21.15.47:0/112143849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc781a1760 con 0x7fcc781037a0 2024-03-21T11:49:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.401+0000 7fcc77e00640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc78105350 0x7fcc781a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.401+0000 7fcc77400640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc781037a0 0x7fcc78199b40 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009d20 tx=0x7fcc6800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.401+0000 7fcc74c00640 1 -- 172.21.15.47:0/112143849 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68010630 con 0x7fcc781037a0 2024-03-21T11:49:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.401+0000 7fcc7ce00640 1 -- 172.21.15.47:0/112143849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc781a1a50 con 0x7fcc781037a0 2024-03-21T11:49:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.401+0000 7fcc74c00640 1 -- 172.21.15.47:0/112143849 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc68010cb0 con 0x7fcc781037a0 2024-03-21T11:49:44.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.401+0000 7fcc74c00640 1 -- 172.21.15.47:0/112143849 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68018de0 con 0x7fcc781037a0 2024-03-21T11:49:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.401+0000 7fcc7ce00640 1 -- 172.21.15.47:0/112143849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc781a1fc0 con 0x7fcc781037a0 2024-03-21T11:49:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.403+0000 7fcc7ce00640 1 -- 172.21.15.47:0/112143849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7810ab40 con 0x7fcc781037a0 2024-03-21T11:49:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.404+0000 7fcc74c00640 1 -- 172.21.15.47:0/112143849 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcc6801f020 con 0x7fcc781037a0 2024-03-21T11:49:44.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.405+0000 7fcc74c00640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc5c0a2fa0 0x7fcc5c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.406+0000 7fcc74c00640 1 -- 172.21.15.47:0/112143849 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fcc680cfba0 con 0x7fcc781037a0 2024-03-21T11:49:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.406+0000 7fcc6ea00640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc5c0a2fa0 0x7fcc5c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:44.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.406+0000 7fcc6ea00640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc5c0a2fa0 0x7fcc5c0a5460 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fcc6002c7d0 tx=0x7fcc60005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:44.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.410+0000 7fcc74c00640 1 -- 172.21.15.47:0/112143849 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcc6809c5e0 con 0x7fcc781037a0 2024-03-21T11:49:44.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.606+0000 7fcc7ce00640 1 -- 172.21.15.47:0/112143849 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc780622a0 con 0x7fcc781037a0 2024-03-21T11:49:44.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.607+0000 7fcc74c00640 1 -- 172.21.15.47:0/112143849 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc6809bee0 con 0x7fcc781037a0 2024-03-21T11:49:44.608 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:49:44.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.609+0000 7fcc7ce00640 1 -- 172.21.15.47:0/112143849 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc5c0a2fa0 msgr2=0x7fcc5c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.609+0000 7fcc7ce00640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc5c0a2fa0 0x7fcc5c0a5460 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fcc6002c7d0 tx=0x7fcc60005fd0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.609+0000 7fcc7ce00640 1 -- 172.21.15.47:0/112143849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc781037a0 msgr2=0x7fcc78199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.609+0000 7fcc7ce00640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc781037a0 0x7fcc78199b40 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009d20 tx=0x7fcc6800c6a0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.609+0000 7fcc7ce00640 1 -- 172.21.15.47:0/112143849 shutdown_connections 2024-03-21T11:49:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.609+0000 7fcc7ce00640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fcc5c0a2fa0 0x7fcc5c0a5460 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.609+0000 7fcc7ce00640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc78105350 0x7fcc781a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.609+0000 7fcc7ce00640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc78104990 0x7fcc7819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.609+0000 7fcc7ce00640 1 --2- 172.21.15.47:0/112143849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc781037a0 0x7fcc78199b40 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.609+0000 7fcc7ce00640 1 -- 172.21.15.47:0/112143849 >> 172.21.15.47:0/112143849 conn(0x7fcc780fed10 msgr2=0x7fcc78107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.609+0000 7fcc7ce00640 1 -- 172.21.15.47:0/112143849 shutdown_connections 2024-03-21T11:49:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.610+0000 7fcc7ce00640 1 -- 172.21.15.47:0/112143849 wait complete. 2024-03-21T11:49:44.627 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T11:49:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.883+0000 7f09fae00640 1 -- 172.21.15.47:0/3394623860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09f4104ad0 msgr2=0x7f09f4104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.883+0000 7f09fae00640 1 --2- 172.21.15.47:0/3394623860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09f4104ad0 0x7f09f4104f50 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009900 tx=0x7f09dc02bc30 comp rx=0 tx=0).stop 2024-03-21T11:49:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.884+0000 7f09fae00640 1 -- 172.21.15.47:0/3394623860 shutdown_connections 2024-03-21T11:49:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.884+0000 7f09fae00640 1 --2- 172.21.15.47:0/3394623860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09f4105490 0x7f09f410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.884+0000 7f09fae00640 1 --2- 172.21.15.47:0/3394623860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09f4104ad0 0x7f09f4104f50 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.884+0000 7f09fae00640 1 --2- 172.21.15.47:0/3394623860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f41038e0 0x7f09f4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.884+0000 7f09fae00640 1 -- 172.21.15.47:0/3394623860 >> 172.21.15.47:0/3394623860 conn(0x7f09f40fee50 msgr2=0x7f09f4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.884+0000 7f09fae00640 1 -- 172.21.15.47:0/3394623860 shutdown_connections 2024-03-21T11:49:44.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.884+0000 7f09fae00640 1 -- 172.21.15.47:0/3394623860 wait complete. 2024-03-21T11:49:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.885+0000 7f09fae00640 1 Processor -- start 2024-03-21T11:49:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.885+0000 7f09fae00640 1 -- start start 2024-03-21T11:49:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.885+0000 7f09fae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09f41038e0 0x7f09f4199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.885+0000 7f09fae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09f4104ad0 0x7f09f419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.885+0000 7f09fae00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f4105490 0x7f09f41a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.885+0000 7f09fae00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f410f5b0 con 0x7f09f4105490 2024-03-21T11:49:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.885+0000 7f09fae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f410f430 con 0x7f09f41038e0 2024-03-21T11:49:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.885+0000 7f09fae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f410f730 con 0x7f09f4104ad0 2024-03-21T11:49:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.885+0000 7f09f3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09f41038e0 0x7f09f4199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.885+0000 7f09f3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09f41038e0 0x7f09f4199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34160/0 (socket says 172.21.15.47:34160) 2024-03-21T11:49:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.885+0000 7f09f3e00640 1 -- 172.21.15.47:0/3100592907 learned_addr learned my addr 172.21.15.47:0/3100592907 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.885+0000 7f09ebe00640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09f4104ad0 0x7f09f419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.886+0000 7f09f8e00640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f4105490 0x7f09f41a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.886+0000 7f09f3e00640 1 -- 172.21.15.47:0/3100592907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09f4104ad0 msgr2=0x7f09f419a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.886+0000 7f09f3e00640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09f4104ad0 0x7f09f419a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.886+0000 7f09f3e00640 1 -- 172.21.15.47:0/3100592907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f4105490 msgr2=0x7f09f41a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.886+0000 7f09f3e00640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f4105490 0x7f09f41a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.886+0000 7f09f3e00640 1 -- 172.21.15.47:0/3100592907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f41a17e0 con 0x7f09f41038e0 2024-03-21T11:49:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.886+0000 7f09ebe00640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09f4104ad0 0x7f09f419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:49:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.886+0000 7f09f3e00640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09f41038e0 0x7f09f4199bc0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f09e400e9c0 tx=0x7f09e400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.887+0000 7f09f1600640 1 -- 172.21.15.47:0/3100592907 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e400cd80 con 0x7f09f41038e0 2024-03-21T11:49:44.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.887+0000 7f09f1600640 1 -- 172.21.15.47:0/3100592907 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09e40035f0 con 0x7f09f41038e0 2024-03-21T11:49:44.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.887+0000 7f09fae00640 1 -- 172.21.15.47:0/3100592907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f41a1ad0 con 0x7f09f41038e0 2024-03-21T11:49:44.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.887+0000 7f09f1600640 1 -- 172.21.15.47:0/3100592907 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4010690 con 0x7f09f41038e0 2024-03-21T11:49:44.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.887+0000 7f09fae00640 1 -- 172.21.15.47:0/3100592907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f41a2040 con 0x7f09f41038e0 2024-03-21T11:49:44.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.888+0000 7f09fae00640 1 -- 172.21.15.47:0/3100592907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f410ac00 con 0x7f09f41038e0 2024-03-21T11:49:44.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.890+0000 7f09f1600640 1 -- 172.21.15.47:0/3100592907 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f09e40037a0 con 0x7f09f41038e0 2024-03-21T11:49:44.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.891+0000 7f09f1600640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f09d80a2fa0 0x7f09d80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:44.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.891+0000 7f09f1600640 1 -- 172.21.15.47:0/3100592907 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f09e40cfb00 con 0x7f09f41038e0 2024-03-21T11:49:44.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.892+0000 7f09ebe00640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f09d80a2fa0 0x7f09d80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:44.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.892+0000 7f09ebe00640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f09d80a2fa0 0x7f09d80a5460 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f09dc02c240 tx=0x7f09dc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:44.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:44.895+0000 7f09f1600640 1 -- 172.21.15.47:0/3100592907 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f09e409c580 con 0x7f09f41038e0 2024-03-21T11:49:45.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.097+0000 7f09fae00640 1 -- 172.21.15.47:0/3100592907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09f4062340 con 0x7f09f41038e0 2024-03-21T11:49:45.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.098+0000 7f09f1600640 1 -- 172.21.15.47:0/3100592907 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f09e409be80 con 0x7f09f41038e0 2024-03-21T11:49:45.099 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T11:49:45.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.101+0000 7f09fae00640 1 -- 172.21.15.47:0/3100592907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f09d80a2fa0 msgr2=0x7f09d80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:45.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.101+0000 7f09fae00640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f09d80a2fa0 0x7f09d80a5460 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f09dc02c240 tx=0x7f09dc02df90 comp rx=0 tx=0).stop 2024-03-21T11:49:45.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.101+0000 7f09fae00640 1 -- 172.21.15.47:0/3100592907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09f41038e0 msgr2=0x7f09f4199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:45.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.101+0000 7f09fae00640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09f41038e0 0x7f09f4199bc0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f09e400e9c0 tx=0x7f09e400ee80 comp rx=0 tx=0).stop 2024-03-21T11:49:45.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.101+0000 7f09fae00640 1 -- 172.21.15.47:0/3100592907 shutdown_connections 2024-03-21T11:49:45.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.101+0000 7f09fae00640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f09d80a2fa0 0x7f09d80a5460 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.101+0000 7f09fae00640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09f4105490 0x7f09f41a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.101+0000 7f09fae00640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09f4104ad0 0x7f09f419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.101+0000 7f09fae00640 1 --2- 172.21.15.47:0/3100592907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09f41038e0 0x7f09f4199bc0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.101+0000 7f09fae00640 1 -- 172.21.15.47:0/3100592907 >> 172.21.15.47:0/3100592907 conn(0x7f09f40fee50 msgr2=0x7f09f4107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:45.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.101+0000 7f09fae00640 1 -- 172.21.15.47:0/3100592907 shutdown_connections 2024-03-21T11:49:45.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.101+0000 7f09fae00640 1 -- 172.21.15.47:0/3100592907 wait complete. 2024-03-21T11:49:45.121 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T11:49:45.194 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:44 smithi047 ceph-mon[31351]: pgmap v1193: 129 pgs: 129 active+clean; 548 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 3.6 MiB/s wr, 192 op/s 2024-03-21T11:49:45.195 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:44 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/744358567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:49:45.195 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:44 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/112143849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:49:45.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:44 smithi102 ceph-mon[36748]: pgmap v1193: 129 pgs: 129 active+clean; 548 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 3.6 MiB/s wr, 192 op/s 2024-03-21T11:49:45.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:44 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/744358567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:49:45.242 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:44 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/112143849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:49:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.316+0000 7f18b1800640 1 -- 172.21.15.47:0/2004685899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ac0740c0 msgr2=0x7f18ac074540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.316+0000 7f18b1800640 1 --2- 172.21.15.47:0/2004685899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ac0740c0 0x7f18ac074540 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f1898009900 tx=0x7f189802bc30 comp rx=0 tx=0).stop 2024-03-21T11:49:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.316+0000 7f18b1800640 1 -- 172.21.15.47:0/2004685899 shutdown_connections 2024-03-21T11:49:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.316+0000 7f18b1800640 1 --2- 172.21.15.47:0/2004685899 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18ac109560 0x7f18ac10b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.316+0000 7f18b1800640 1 --2- 172.21.15.47:0/2004685899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ac0740c0 0x7f18ac074540 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.316+0000 7f18b1800640 1 --2- 172.21.15.47:0/2004685899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ac0756a0 0x7f18ac073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.316+0000 7f18b1800640 1 -- 172.21.15.47:0/2004685899 >> 172.21.15.47:0/2004685899 conn(0x7f18ac0fcc40 msgr2=0x7f18ac0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.316+0000 7f18b1800640 1 -- 172.21.15.47:0/2004685899 shutdown_connections 2024-03-21T11:49:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.316+0000 7f18b1800640 1 -- 172.21.15.47:0/2004685899 wait complete. 2024-03-21T11:49:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.317+0000 7f18b1800640 1 Processor -- start 2024-03-21T11:49:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.317+0000 7f18b1800640 1 -- start start 2024-03-21T11:49:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18b1800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ac0740c0 0x7f18ac199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18b1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ac0756a0 0x7f18ac19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18b1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18ac109560 0x7f18ac1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18b1800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac10f540 con 0x7f18ac109560 2024-03-21T11:49:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18b1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac10f3c0 con 0x7f18ac0756a0 2024-03-21T11:49:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18b1800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac10f6c0 con 0x7f18ac0740c0 2024-03-21T11:49:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ac0740c0 0x7f18ac199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18aa000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ac0756a0 0x7f18ac19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ac0740c0 0x7f18ac199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42926/0 (socket says 172.21.15.47:42926) 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18aaa00640 1 -- 172.21.15.47:0/600554565 learned_addr learned my addr 172.21.15.47:0/600554565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18aa000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ac0756a0 0x7f18ac19a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34188/0 (socket says 172.21.15.47:34188) 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18ab400640 1 --2- 172.21.15.47:0/600554565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18ac109560 0x7f18ac1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18aa000640 1 -- 172.21.15.47:0/600554565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ac0740c0 msgr2=0x7f18ac199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18aa000640 1 --2- 172.21.15.47:0/600554565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ac0740c0 0x7f18ac199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18aa000640 1 -- 172.21.15.47:0/600554565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18ac109560 msgr2=0x7f18ac1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18aa000640 1 --2- 172.21.15.47:0/600554565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18ac109560 0x7f18ac1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.318+0000 7f18aa000640 1 -- 172.21.15.47:0/600554565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac1a1810 con 0x7f18ac0756a0 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.319+0000 7f18aa000640 1 --2- 172.21.15.47:0/600554565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ac0756a0 0x7f18ac19a130 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f1898036040 tx=0x7f189802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.319+0000 7f18a3400640 1 -- 172.21.15.47:0/600554565 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898043020 con 0x7f18ac0756a0 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.319+0000 7f18a3400640 1 -- 172.21.15.47:0/600554565 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18980054e0 con 0x7f18ac0756a0 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.319+0000 7f18b1800640 1 -- 172.21.15.47:0/600554565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ac1a1aa0 con 0x7f18ac0756a0 2024-03-21T11:49:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.319+0000 7f18a3400640 1 -- 172.21.15.47:0/600554565 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898003c30 con 0x7f18ac0756a0 2024-03-21T11:49:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.319+0000 7f18b1800640 1 -- 172.21.15.47:0/600554565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac1a1fc0 con 0x7f18ac0756a0 2024-03-21T11:49:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.320+0000 7f18b1800640 1 -- 172.21.15.47:0/600554565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ac1082e0 con 0x7f18ac0756a0 2024-03-21T11:49:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.322+0000 7f18a3400640 1 -- 172.21.15.47:0/600554565 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1898046020 con 0x7f18ac0756a0 2024-03-21T11:49:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.324+0000 7f18a3400640 1 --2- 172.21.15.47:0/600554565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18900a2f20 0x7f18900a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.324+0000 7f18a3400640 1 -- 172.21.15.47:0/600554565 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f18980f1bb0 con 0x7f18ac0756a0 2024-03-21T11:49:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.324+0000 7f18aaa00640 1 --2- 172.21.15.47:0/600554565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18900a2f20 0x7f18900a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.325+0000 7f18aaa00640 1 --2- 172.21.15.47:0/600554565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18900a2f20 0x7f18900a53e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f189c009c90 tx=0x7f189c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.326+0000 7f18a3400640 1 -- 172.21.15.47:0/600554565 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f189804d020 con 0x7f18ac0756a0 2024-03-21T11:49:45.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:44 smithi100 ceph-mon[36245]: pgmap v1193: 129 pgs: 129 active+clean; 548 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 3.6 MiB/s wr, 192 op/s 2024-03-21T11:49:45.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:44 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/744358567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:49:45.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:44 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/112143849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:49:45.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.532+0000 7f18b1800640 1 -- 172.21.15.47:0/600554565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f18ac02d200 con 0x7f18ac0756a0 2024-03-21T11:49:45.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.533+0000 7f18a3400640 1 -- 172.21.15.47:0/600554565 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) v1 ==== 103+0+11 (secure 0 0 0) 0x7f18980be5f0 con 0x7f18ac0756a0 2024-03-21T11:49:45.534 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:49:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.537+0000 7f18b1800640 1 -- 172.21.15.47:0/600554565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18900a2f20 msgr2=0x7f18900a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.537+0000 7f18b1800640 1 --2- 172.21.15.47:0/600554565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18900a2f20 0x7f18900a53e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f189c009c90 tx=0x7f189c005f90 comp rx=0 tx=0).stop 2024-03-21T11:49:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.537+0000 7f18b1800640 1 -- 172.21.15.47:0/600554565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ac0756a0 msgr2=0x7f18ac19a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.537+0000 7f18b1800640 1 --2- 172.21.15.47:0/600554565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ac0756a0 0x7f18ac19a130 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f1898036040 tx=0x7f189802c650 comp rx=0 tx=0).stop 2024-03-21T11:49:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.537+0000 7f18b1800640 1 -- 172.21.15.47:0/600554565 shutdown_connections 2024-03-21T11:49:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.537+0000 7f18b1800640 1 --2- 172.21.15.47:0/600554565 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f18900a2f20 0x7f18900a53e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.537+0000 7f18b1800640 1 --2- 172.21.15.47:0/600554565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18ac109560 0x7f18ac1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.537+0000 7f18b1800640 1 --2- 172.21.15.47:0/600554565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18ac0756a0 0x7f18ac19a130 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.537+0000 7f18b1800640 1 --2- 172.21.15.47:0/600554565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18ac0740c0 0x7f18ac199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.537+0000 7f18b1800640 1 -- 172.21.15.47:0/600554565 >> 172.21.15.47:0/600554565 conn(0x7f18ac0fcc40 msgr2=0x7f18ac1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:45.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.537+0000 7f18b1800640 1 -- 172.21.15.47:0/600554565 shutdown_connections 2024-03-21T11:49:45.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.537+0000 7f18b1800640 1 -- 172.21.15.47:0/600554565 wait complete. 2024-03-21T11:49:45.554 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-21T11:49:45.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.754+0000 7f88b2e00640 1 -- 172.21.15.47:0/2537262239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac103270 msgr2=0x7f88ac109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:45.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.754+0000 7f88b2e00640 1 --2- 172.21.15.47:0/2537262239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac103270 0x7f88ac109b00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009ca0 tx=0x7f88a402c0f0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.754+0000 7f88b2e00640 1 -- 172.21.15.47:0/2537262239 shutdown_connections 2024-03-21T11:49:45.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.754+0000 7f88b2e00640 1 --2- 172.21.15.47:0/2537262239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac103270 0x7f88ac109b00 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.754+0000 7f88b2e00640 1 --2- 172.21.15.47:0/2537262239 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac1028b0 0x7f88ac102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.754+0000 7f88b2e00640 1 --2- 172.21.15.47:0/2537262239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88ac1016c0 0x7f88ac101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.754+0000 7f88b2e00640 1 -- 172.21.15.47:0/2537262239 >> 172.21.15.47:0/2537262239 conn(0x7f88ac0fcc50 msgr2=0x7f88ac0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:45.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.754+0000 7f88b2e00640 1 -- 172.21.15.47:0/2537262239 shutdown_connections 2024-03-21T11:49:45.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.754+0000 7f88b2e00640 1 -- 172.21.15.47:0/2537262239 wait complete. 2024-03-21T11:49:45.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.755+0000 7f88b2e00640 1 Processor -- start 2024-03-21T11:49:45.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.755+0000 7f88b2e00640 1 -- start start 2024-03-21T11:49:45.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88b2e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88ac1016c0 0x7f88ac197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:45.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88b2e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac1028b0 0x7f88ac197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:45.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88b2e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac103270 0x7f88ac19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:45.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88b2e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac10d340 con 0x7f88ac103270 2024-03-21T11:49:45.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88b2e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac10d1c0 con 0x7f88ac1028b0 2024-03-21T11:49:45.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88b2e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac10d4c0 con 0x7f88ac1016c0 2024-03-21T11:49:45.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88abe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88ac1016c0 0x7f88ac197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:45.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88ab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac1028b0 0x7f88ac197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88abe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88ac1016c0 0x7f88ac197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42940/0 (socket says 172.21.15.47:42940) 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88abe00640 1 -- 172.21.15.47:0/652081587 learned_addr learned my addr 172.21.15.47:0/652081587 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88ab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac1028b0 0x7f88ac197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:34198/0 (socket says 172.21.15.47:34198) 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88b0e00640 1 --2- 172.21.15.47:0/652081587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac103270 0x7f88ac19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88abe00640 1 -- 172.21.15.47:0/652081587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac1028b0 msgr2=0x7f88ac197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88abe00640 1 --2- 172.21.15.47:0/652081587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac1028b0 0x7f88ac197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88abe00640 1 -- 172.21.15.47:0/652081587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac103270 msgr2=0x7f88ac19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88abe00640 1 --2- 172.21.15.47:0/652081587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac103270 0x7f88ac19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88abe00640 1 -- 172.21.15.47:0/652081587 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac19f540 con 0x7f88ac1016c0 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.756+0000 7f88b0e00640 1 --2- 172.21.15.47:0/652081587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac103270 0x7f88ac19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.757+0000 7f88abe00640 1 --2- 172.21.15.47:0/652081587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88ac1016c0 0x7f88ac197920 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f889c00e9c0 tx=0x7f889c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.757+0000 7f88a8c00640 1 -- 172.21.15.47:0/652081587 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c00cd80 con 0x7f88ac1016c0 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.757+0000 7f88a8c00640 1 -- 172.21.15.47:0/652081587 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f889c0035f0 con 0x7f88ac1016c0 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.757+0000 7f88a8c00640 1 -- 172.21.15.47:0/652081587 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c010690 con 0x7f88ac1016c0 2024-03-21T11:49:45.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.757+0000 7f88b2e00640 1 -- 172.21.15.47:0/652081587 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ac19f830 con 0x7f88ac1016c0 2024-03-21T11:49:45.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.757+0000 7f88b2e00640 1 -- 172.21.15.47:0/652081587 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac19fd80 con 0x7f88ac1016c0 2024-03-21T11:49:45.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.758+0000 7f88b2e00640 1 -- 172.21.15.47:0/652081587 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ac1089e0 con 0x7f88ac1016c0 2024-03-21T11:49:45.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.760+0000 7f88a8c00640 1 -- 172.21.15.47:0/652081587 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f889c0037a0 con 0x7f88ac1016c0 2024-03-21T11:49:45.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.762+0000 7f88a8c00640 1 --2- 172.21.15.47:0/652081587 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f88900a2fa0 0x7f88900a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:49:45.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.762+0000 7f88a8c00640 1 -- 172.21.15.47:0/652081587 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f889c0cfb40 con 0x7f88ac1016c0 2024-03-21T11:49:45.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.763+0000 7f88ab400640 1 --2- 172.21.15.47:0/652081587 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f88900a2fa0 0x7f88900a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:49:45.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.763+0000 7f88ab400640 1 --2- 172.21.15.47:0/652081587 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f88900a2fa0 0x7f88900a5460 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f88ac198e30 tx=0x7f8898005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:49:45.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:45.765+0000 7f88a8c00640 1 -- 172.21.15.47:0/652081587 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f889c09c580 con 0x7f88ac1016c0 2024-03-21T11:49:46.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:46.031+0000 7f88b2e00640 1 -- 172.21.15.47:0/652081587 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f88ac0622d0 con 0x7f88ac1016c0 2024-03-21T11:49:46.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:46.044+0000 7f88a8c00640 1 -- 172.21.15.47:0/652081587 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) v1 ==== 81+0+741842 (secure 0 0 0) 0x7f889c09be80 con 0x7f88ac1016c0 2024-03-21T11:49:46.058 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:49:46.058 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:49:46.058 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 20, 2024-03-21T11:49:46.058 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.058 INFO:teuthology.orchestra.run.smithi047.stdout: "active_gid": 14152, 2024-03-21T11:49:46.059 INFO:teuthology.orchestra.run.smithi047.stdout: "active_name": "x", 2024-03-21T11:49:46.059 INFO:teuthology.orchestra.run.smithi047.stdout: "active_addrs": { 2024-03-21T11:49:46.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-03-21T11:49:46.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-03-21T11:49:46.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6800", 2024-03-21T11:49:46.059 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2057671763 2024-03-21T11:49:46.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-21T11:49:46.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6801", 2024-03-21T11:49:46.059 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2057671763 2024-03-21T11:49:46.059 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.059 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T11:49:46.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.060 INFO:teuthology.orchestra.run.smithi047.stdout: "active_addr": "172.21.15.47:6801/2057671763", 2024-03-21T11:49:46.060 INFO:teuthology.orchestra.run.smithi047.stdout: "active_change": "2024-03-21T11:12:48.837174+0000", 2024-03-21T11:49:46.060 INFO:teuthology.orchestra.run.smithi047.stdout: "active_mgr_features": 4540701547738038271, 2024-03-21T11:49:46.060 INFO:teuthology.orchestra.run.smithi047.stdout: "available": true, 2024-03-21T11:49:46.060 INFO:teuthology.orchestra.run.smithi047.stdout: "standbys": [ 2024-03-21T11:49:46.060 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.060 INFO:teuthology.orchestra.run.smithi047.stdout: "gid": 24109, 2024-03-21T11:49:46.060 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "z", 2024-03-21T11:49:46.060 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_features": 4540701547738038271, 2024-03-21T11:49:46.060 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-03-21T11:49:46.060 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.060 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-03-21T11:49:46.060 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.061 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.061 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.061 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:46.061 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:46.061 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.061 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.061 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.061 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:46.061 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.061 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.061 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.062 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T11:49:46.062 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.062 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.062 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.062 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.062 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.062 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.062 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.062 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.062 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.062 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.063 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.063 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.063 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.063 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.063 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.063 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.063 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.063 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.063 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.063 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.063 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.063 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.063 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.065 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.065 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.065 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.066 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.066 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.066 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.066 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.066 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.066 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.066 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.066 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.066 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.066 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.066 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.066 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.066 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.067 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.067 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.067 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.067 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.067 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.067 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.067 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.067 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.067 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.067 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.067 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.067 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.067 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.068 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.068 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.068 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.068 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.068 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.068 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.068 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.068 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.068 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.068 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.069 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.069 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.069 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.069 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.069 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.069 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.069 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.069 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-03-21T11:49:46.069 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-03-21T11:49:46.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.070 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.070 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.070 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.070 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.070 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.070 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.070 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-03-21T11:49:46.070 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.070 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.070 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.070 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-03-21T11:49:46.071 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-03-21T11:49:46.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.071 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.071 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.071 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-03-21T11:49:46.071 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.071 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.071 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.071 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-03-21T11:49:46.071 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.071 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.071 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.071 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-03-21T11:49:46.072 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-03-21T11:49:46.072 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.072 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.072 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.072 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.072 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.072 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.072 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.072 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-03-21T11:49:46.072 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.072 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.072 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-03-21T11:49:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-03-21T11:49:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-03-21T11:49:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.074 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-03-21T11:49:46.074 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-03-21T11:49:46.074 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.074 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.074 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.074 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-03-21T11:49:46.074 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.074 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.074 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.074 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-03-21T11:49:46.074 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.074 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.074 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.075 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-03-21T11:49:46.075 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-03-21T11:49:46.075 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.075 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.075 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.075 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.075 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.075 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.075 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.075 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-03-21T11:49:46.075 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.075 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.076 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.076 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-03-21T11:49:46.076 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-03-21T11:49:46.076 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.076 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.076 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.076 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.076 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.076 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.076 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.076 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T11:49:46.077 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.077 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.077 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.077 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-03-21T11:49:46.077 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-03-21T11:49:46.077 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.077 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.077 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.077 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.077 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.077 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.078 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.078 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-03-21T11:49:46.078 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.078 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.078 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.078 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.078 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.078 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-03-21T11:49:46.078 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.078 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.078 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.079 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-03-21T11:49:46.079 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-03-21T11:49:46.079 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.079 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.079 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.079 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.079 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.079 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.079 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.079 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T11:49:46.079 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.079 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.079 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.080 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-03-21T11:49:46.080 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-03-21T11:49:46.080 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.080 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.080 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.080 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-03-21T11:49:46.080 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.080 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.080 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.080 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T11:49:46.080 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T11:49:46.080 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.080 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.081 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-03-21T11:49:46.081 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-03-21T11:49:46.081 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:46.081 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.081 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.081 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:46.081 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-03-21T11:49:46.081 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-03-21T11:49:46.081 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.081 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T11:49:46.081 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T11:49:46.082 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.082 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.082 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-03-21T11:49:46.082 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-03-21T11:49:46.082 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:46.082 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.082 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.082 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-03-21T11:49:46.082 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T11:49:46.082 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-03-21T11:49:46.082 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.082 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T11:49:46.082 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-03-21T11:49:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-03-21T11:49:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T11:49:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T11:49:46.083 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T11:49:46.084 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.084 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.084 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-03-21T11:49:46.084 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-03-21T11:49:46.084 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.084 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.084 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.084 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-03-21T11:49:46.084 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-03-21T11:49:46.084 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-03-21T11:49:46.084 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.084 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-03-21T11:49:46.085 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T11:49:46.085 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.085 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.085 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-03-21T11:49:46.085 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-03-21T11:49:46.085 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.085 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.085 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.085 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-03-21T11:49:46.085 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.085 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.085 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.085 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T11:49:46.086 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T11:49:46.086 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.086 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.086 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-03-21T11:49:46.086 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-03-21T11:49:46.086 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:46.086 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.086 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.086 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:46.086 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-03-21T11:49:46.086 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-03-21T11:49:46.086 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.087 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T11:49:46.087 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T11:49:46.087 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.087 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.087 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.087 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.087 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.087 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.087 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.087 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.087 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.087 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.088 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.088 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.088 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.088 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.088 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.088 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.088 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.088 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.088 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.088 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.088 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.088 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.089 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.089 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.089 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.089 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.089 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.089 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.089 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.089 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.089 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.089 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.089 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.089 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.090 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.090 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.090 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.090 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.090 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.090 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.090 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.090 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.090 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.090 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.090 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.091 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.091 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.091 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.091 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.091 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.091 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.091 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.091 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.091 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.091 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.092 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.092 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.092 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.092 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.092 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.092 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.092 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.092 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.092 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.092 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.092 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.092 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.093 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-03-21T11:49:46.094 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-03-21T11:49:46.094 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.094 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.094 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.094 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-03-21T11:49:46.094 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.094 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.094 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.094 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-03-21T11:49:46.094 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-03-21T11:49:46.094 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-03-21T11:49:46.094 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-03-21T11:49:46.094 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-03-21T11:49:46.095 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.095 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-03-21T11:49:46.095 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.095 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.095 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.095 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-03-21T11:49:46.095 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-03-21T11:49:46.095 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.095 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.095 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.095 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.095 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.095 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.096 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.096 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T11:49:46.096 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.096 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.096 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.096 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:46.096 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:46.096 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.096 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.096 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.096 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:46.096 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.096 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.097 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.097 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T11:49:46.097 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.097 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.097 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.097 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-03-21T11:49:46.097 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-03-21T11:49:46.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.097 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.097 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.097 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T11:49:46.098 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T11:49:46.098 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.098 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.098 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T11:49:46.098 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T11:49:46.098 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.098 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.098 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-03-21T11:49:46.098 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-03-21T11:49:46.098 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:46.098 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.098 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.099 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.099 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.099 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.099 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.099 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T11:49:46.099 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.099 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.099 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.099 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.099 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.099 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-03-21T11:49:46.099 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.100 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.100 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.100 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-03-21T11:49:46.100 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-03-21T11:49:46.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.100 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.100 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.100 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-03-21T11:49:46.100 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.100 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.100 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.100 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T11:49:46.100 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.100 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.101 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.101 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-03-21T11:49:46.101 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-03-21T11:49:46.101 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.101 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.101 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.101 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-03-21T11:49:46.101 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.101 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.101 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.101 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T11:49:46.101 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.101 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.102 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.102 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-03-21T11:49:46.102 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-03-21T11:49:46.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.102 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.102 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.102 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-03-21T11:49:46.102 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.102 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.102 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.102 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T11:49:46.103 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.103 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.103 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.103 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-03-21T11:49:46.103 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-03-21T11:49:46.103 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.103 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.103 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.103 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.103 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.103 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.103 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.104 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T11:49:46.104 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T11:49:46.104 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.104 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.104 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-03-21T11:49:46.104 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-03-21T11:49:46.104 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.104 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.104 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.104 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:46.104 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.104 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.105 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.105 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T11:49:46.105 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.105 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.105 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.105 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-03-21T11:49:46.105 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T11:49:46.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.105 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.105 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.106 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-03-21T11:49:46.106 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.106 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.106 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.106 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T11:49:46.106 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.106 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.106 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.106 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-03-21T11:49:46.106 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-03-21T11:49:46.106 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.106 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.107 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.107 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.107 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.107 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.107 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.107 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-03-21T11:49:46.107 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-03-21T11:49:46.107 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-03-21T11:49:46.107 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.107 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T11:49:46.107 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.107 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.107 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.108 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-03-21T11:49:46.108 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-03-21T11:49:46.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.108 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.108 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.108 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.108 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.108 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.108 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.108 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T11:49:46.108 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.108 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.109 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.109 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-03-21T11:49:46.109 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-03-21T11:49:46.109 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.109 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.109 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.109 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.109 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.109 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.109 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.109 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T11:49:46.109 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.110 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.110 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.110 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-03-21T11:49:46.110 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-03-21T11:49:46.110 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.110 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.110 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.110 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.110 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.110 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.110 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.110 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T11:49:46.110 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-03-21T11:49:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-03-21T11:49:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T11:49:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T11:49:46.111 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.112 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.112 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.112 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-03-21T11:49:46.112 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-03-21T11:49:46.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.112 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.112 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.112 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T11:49:46.112 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.112 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.112 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.112 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-03-21T11:49:46.112 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.113 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.113 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.113 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-03-21T11:49:46.113 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-03-21T11:49:46.113 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.113 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.113 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.113 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T11:49:46.113 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.113 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.113 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.113 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-03-21T11:49:46.114 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.114 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.114 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.114 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-03-21T11:49:46.114 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-03-21T11:49:46.114 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.114 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.114 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.114 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T11:49:46.114 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.114 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.114 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.114 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T11:49:46.115 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.115 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.115 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.115 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-03-21T11:49:46.115 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-03-21T11:49:46.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.115 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.115 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.115 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T11:49:46.115 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.115 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.115 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.116 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-03-21T11:49:46.116 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.116 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.116 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.116 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-03-21T11:49:46.116 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-03-21T11:49:46.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.116 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.116 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.116 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T11:49:46.116 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.116 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.116 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.117 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-03-21T11:49:46.117 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.117 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.117 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.117 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-03-21T11:49:46.117 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-03-21T11:49:46.117 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.117 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.117 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.117 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T11:49:46.117 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.117 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.117 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.118 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-03-21T11:49:46.118 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.118 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.118 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.118 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-03-21T11:49:46.118 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-03-21T11:49:46.118 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.118 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.118 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.118 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T11:49:46.118 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.118 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.118 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.119 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-03-21T11:49:46.119 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.119 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.119 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.119 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-03-21T11:49:46.119 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-03-21T11:49:46.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.119 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.119 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.119 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T11:49:46.119 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.119 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.120 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.120 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-03-21T11:49:46.120 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.120 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.120 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.120 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-03-21T11:49:46.120 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-03-21T11:49:46.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.120 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.120 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.120 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T11:49:46.120 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.120 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.121 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.121 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-03-21T11:49:46.121 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.121 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.121 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.121 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-03-21T11:49:46.121 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-03-21T11:49:46.121 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.121 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.121 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.122 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T11:49:46.122 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.122 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.122 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.122 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T11:49:46.122 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.122 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.122 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.122 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-03-21T11:49:46.122 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-03-21T11:49:46.122 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.122 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.122 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.123 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T11:49:46.123 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.123 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.123 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.123 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-03-21T11:49:46.123 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.123 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.123 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.123 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-03-21T11:49:46.124 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-03-21T11:49:46.124 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.124 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.124 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.124 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T11:49:46.124 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.124 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.124 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.124 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T11:49:46.124 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.124 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.124 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.125 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-03-21T11:49:46.125 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-03-21T11:49:46.125 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.125 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.125 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.125 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T11:49:46.125 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.125 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.125 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.125 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-03-21T11:49:46.125 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.125 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.125 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.126 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-03-21T11:49:46.126 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-03-21T11:49:46.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.126 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.126 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.126 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T11:49:46.126 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.126 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.126 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.126 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-03-21T11:49:46.126 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.126 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.127 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.127 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-03-21T11:49:46.127 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-03-21T11:49:46.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.127 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.127 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.127 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.127 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.127 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.127 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.127 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T11:49:46.127 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-03-21T11:49:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-03-21T11:49:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T11:49:46.128 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.129 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.129 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.129 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-03-21T11:49:46.129 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T11:49:46.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.129 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.129 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.129 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-03-21T11:49:46.129 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.129 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.129 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.130 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T11:49:46.130 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.130 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.130 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.130 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-03-21T11:49:46.130 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-03-21T11:49:46.130 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.130 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.130 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.130 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-03-21T11:49:46.130 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.130 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.130 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.131 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T11:49:46.131 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.131 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.131 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.131 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-03-21T11:49:46.131 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-03-21T11:49:46.131 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.131 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.131 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.131 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-03-21T11:49:46.131 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.131 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.131 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.132 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-03-21T11:49:46.132 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.132 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.132 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.132 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-03-21T11:49:46.132 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-03-21T11:49:46.132 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.132 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.132 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.132 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.132 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.132 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.132 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.133 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T11:49:46.133 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.133 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.133 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.133 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-03-21T11:49:46.133 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-03-21T11:49:46.133 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.133 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.134 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.134 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:46.134 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.134 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.134 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.134 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-03-21T11:49:46.134 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.134 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.134 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.134 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-03-21T11:49:46.135 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-03-21T11:49:46.135 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.135 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.135 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.135 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T11:49:46.135 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.135 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.135 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.135 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T11:49:46.135 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.135 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.135 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.135 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-03-21T11:49:46.136 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-03-21T11:49:46.136 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.136 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.136 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.136 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:46.136 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.136 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.136 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.136 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-03-21T11:49:46.136 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.136 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.136 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.136 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-03-21T11:49:46.137 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-03-21T11:49:46.137 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.137 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.137 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.137 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.137 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.137 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.137 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.137 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T11:49:46.137 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.137 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.137 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.137 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-03-21T11:49:46.138 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-03-21T11:49:46.138 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.138 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.138 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.138 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.138 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.138 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.138 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.138 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T11:49:46.138 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.138 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.138 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.139 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.139 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.139 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.139 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.139 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.139 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.139 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.139 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.139 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.139 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.139 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.139 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.139 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.139 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.140 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.141 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.141 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T11:49:46.141 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.141 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.141 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.141 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.141 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.141 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.141 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.141 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.141 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.141 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.142 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.142 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.142 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T11:49:46.142 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.142 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.142 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.142 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.142 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.142 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.142 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.142 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.142 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.143 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.143 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.143 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.143 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.143 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.143 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.143 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.143 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.143 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.143 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.143 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.143 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.143 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.143 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.144 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.144 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.144 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.144 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.144 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.144 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.144 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.144 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.144 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.144 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.144 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.144 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.144 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.145 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T11:49:46.145 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T11:49:46.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.145 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.145 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.145 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.145 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.145 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.145 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.145 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T11:49:46.145 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.145 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.145 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.146 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T11:49:46.146 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T11:49:46.146 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.146 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.146 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.146 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-03-21T11:49:46.146 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.146 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.146 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.146 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T11:49:46.146 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.146 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.147 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.147 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-03-21T11:49:46.147 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-03-21T11:49:46.147 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.147 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.147 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.147 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.147 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.147 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.147 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.147 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-03-21T11:49:46.147 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.147 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.148 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.148 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-03-21T11:49:46.148 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-03-21T11:49:46.148 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.148 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.148 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.148 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.148 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.148 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.148 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.148 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T11:49:46.148 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.148 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.149 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.150 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-03-21T11:49:46.150 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-03-21T11:49:46.150 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.150 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.150 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.150 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-03-21T11:49:46.150 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.150 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.150 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.150 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-03-21T11:49:46.150 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-03-21T11:49:46.150 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.150 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-03-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-03-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-03-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.152 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T11:49:46.152 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.152 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.152 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.152 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-03-21T11:49:46.152 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-03-21T11:49:46.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.152 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.152 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.152 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T11:49:46.152 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.152 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.153 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.153 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T11:49:46.153 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.153 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.153 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.153 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-03-21T11:49:46.153 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-03-21T11:49:46.153 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.153 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.153 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.153 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.153 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.153 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.154 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.154 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T11:49:46.154 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.154 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.154 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.154 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-03-21T11:49:46.154 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-03-21T11:49:46.154 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.154 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.154 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.155 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.155 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.155 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.155 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.155 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T11:49:46.155 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.155 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.155 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.155 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-03-21T11:49:46.155 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-03-21T11:49:46.155 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.155 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.156 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.156 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.156 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.156 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.156 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.156 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T11:49:46.156 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.156 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.156 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.156 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-03-21T11:49:46.156 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-03-21T11:49:46.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.156 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.157 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.158 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.158 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.158 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.158 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.158 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T11:49:46.158 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.158 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.158 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.158 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-03-21T11:49:46.158 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-03-21T11:49:46.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.158 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.159 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.159 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-03-21T11:49:46.159 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.159 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.159 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.159 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-03-21T11:49:46.159 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.159 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.159 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.159 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-03-21T11:49:46.159 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-03-21T11:49:46.159 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.160 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.160 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.160 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.160 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.160 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.160 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.160 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T11:49:46.160 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.160 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.160 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.160 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-03-21T11:49:46.160 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-03-21T11:49:46.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.161 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.161 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.161 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.161 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.161 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.161 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.161 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T11:49:46.161 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.161 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.161 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.161 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-03-21T11:49:46.161 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-03-21T11:49:46.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.162 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.162 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.162 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.162 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.162 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.162 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.162 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T11:49:46.162 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.162 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.162 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.162 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-03-21T11:49:46.162 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-03-21T11:49:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T11:49:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-03-21T11:49:46.163 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-03-21T11:49:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T11:49:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-03-21T11:49:46.164 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-03-21T11:49:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T11:49:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-03-21T11:49:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.166 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.167 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.168 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.169 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.170 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.171 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.171 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.171 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.171 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.171 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.171 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-03-21T11:49:46.171 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-03-21T11:49:46.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.171 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.171 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.171 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-03-21T11:49:46.171 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.171 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.172 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.172 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T11:49:46.172 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.172 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.172 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.172 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-03-21T11:49:46.172 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-03-21T11:49:46.172 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.172 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.172 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.172 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-03-21T11:49:46.172 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.172 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.173 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.173 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T11:49:46.173 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.173 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.173 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.173 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.173 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.173 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-03-21T11:49:46.173 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.173 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.173 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.173 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T11:49:46.174 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T11:49:46.174 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.174 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.174 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.174 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.174 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.174 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.174 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.174 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.174 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.174 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.174 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.175 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T11:49:46.175 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T11:49:46.175 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.175 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.175 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.175 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.175 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.175 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.175 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.175 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.175 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.175 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.175 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.176 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-03-21T11:49:46.177 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T11:49:46.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.177 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.177 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.177 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.177 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.177 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.177 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.177 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.177 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.177 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.177 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.177 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T11:49:46.178 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T11:49:46.178 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.178 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.178 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.178 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.178 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.178 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.178 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.178 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.178 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.178 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.178 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.179 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T11:49:46.179 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T11:49:46.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.179 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.179 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.179 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.179 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.179 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.179 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.179 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.179 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.179 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.179 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.180 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T11:49:46.180 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T11:49:46.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.180 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.180 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.180 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.180 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.180 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.180 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.180 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.180 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.180 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.180 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.181 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T11:49:46.181 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T11:49:46.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.181 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.181 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.181 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.181 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.181 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.181 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.181 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.181 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.182 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.182 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.182 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T11:49:46.182 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T11:49:46.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.182 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.182 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.182 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.182 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.182 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.182 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.182 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.182 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.183 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.183 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.183 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T11:49:46.183 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T11:49:46.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.183 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.183 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.183 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.183 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.183 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.183 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.183 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.184 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.184 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.184 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.184 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T11:49:46.184 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T11:49:46.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.184 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.184 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.184 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.184 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.184 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.184 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.184 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.185 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.185 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.185 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.185 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T11:49:46.185 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T11:49:46.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.185 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.185 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.185 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.185 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.185 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.185 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.185 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.186 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.186 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.186 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.186 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T11:49:46.186 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T11:49:46.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.186 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.186 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.186 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.186 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.186 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.186 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.186 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.187 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.187 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.187 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.187 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T11:49:46.187 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T11:49:46.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.187 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.187 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.187 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.187 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.187 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.187 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.188 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.188 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.188 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.188 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.188 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T11:49:46.188 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T11:49:46.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.188 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.188 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.188 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T11:49:46.188 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.188 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.188 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.189 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.189 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.189 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.189 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.189 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T11:49:46.189 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T11:49:46.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.189 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.189 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.189 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.189 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.189 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.190 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.190 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.190 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.190 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.190 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.190 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-03-21T11:49:46.190 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-03-21T11:49:46.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.190 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.190 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.190 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.190 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.191 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.192 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.192 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.192 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.192 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.192 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.192 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T11:49:46.192 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T11:49:46.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.192 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.192 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.192 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.192 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.192 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.194 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.194 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.194 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.194 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.194 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.194 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T11:49:46.194 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T11:49:46.194 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.194 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.194 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.194 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.194 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.195 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.195 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.195 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.195 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.195 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.195 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.195 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T11:49:46.195 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T11:49:46.195 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.195 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.195 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.195 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-03-21T11:49:46.196 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.197 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.197 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.197 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.197 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.197 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.197 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.197 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T11:49:46.197 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T11:49:46.197 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.197 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.197 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.197 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.198 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.198 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.198 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.198 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.198 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.198 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.198 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.198 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T11:49:46.198 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T11:49:46.198 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.198 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.198 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.199 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.200 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.200 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.200 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.200 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.200 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.200 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.200 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.200 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T11:49:46.200 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T11:49:46.200 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.200 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.200 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.201 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.202 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.202 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.202 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.202 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.202 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.202 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.202 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.202 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T11:49:46.202 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T11:49:46.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.202 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.202 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.202 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.203 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.203 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.203 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.203 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.203 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.203 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.203 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.203 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T11:49:46.203 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T11:49:46.203 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.203 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.203 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.204 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.205 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.206 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.206 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.206 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.206 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.206 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.206 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.206 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T11:49:46.206 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T11:49:46.206 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.206 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.206 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.206 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.206 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.207 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.207 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.207 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.207 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.207 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.207 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.207 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T11:49:46.207 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T11:49:46.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.207 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.207 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.207 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T11:49:46.208 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.208 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.208 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.208 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.208 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.208 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.208 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.208 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T11:49:46.208 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T11:49:46.208 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.208 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.208 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.209 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.209 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.209 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.209 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.209 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.209 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.209 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.209 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.209 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T11:49:46.209 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T11:49:46.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.209 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.209 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.210 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-03-21T11:49:46.210 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.210 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.210 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.210 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.210 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.210 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.210 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.210 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T11:49:46.210 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T11:49:46.210 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.210 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.210 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-03-21T11:49:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T11:49:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T11:49:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.211 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.212 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.212 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.212 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.212 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.212 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.212 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.212 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.212 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.212 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T11:49:46.212 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T11:49:46.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.212 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.212 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.213 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T11:49:46.213 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.213 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.213 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.213 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.213 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.213 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.213 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.213 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T11:49:46.213 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T11:49:46.213 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.213 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.214 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.214 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.214 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.214 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.214 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.214 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.214 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.214 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.214 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.214 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T11:49:46.214 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T11:49:46.214 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T11:49:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T11:49:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T11:49:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T11:49:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T11:49:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T11:49:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-03-21T11:49:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-03-21T11:49:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-03-21T11:49:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-03-21T11:49:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-03-21T11:49:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-03-21T11:49:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-03-21T11:49:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-03-21T11:49:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-03-21T11:49:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-03-21T11:49:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-03-21T11:49:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-03-21T11:49:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-03-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-03-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T11:49:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T11:49:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-03-21T11:49:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T11:49:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T11:49:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-03-21T11:49:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-03-21T11:49:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-03-21T11:49:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-03-21T11:49:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-03-21T11:49:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-03-21T11:49:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.236 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.236 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.236 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.236 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.236 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.236 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-03-21T11:49:46.236 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-03-21T11:49:46.236 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.236 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.236 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.236 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-03-21T11:49:46.236 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.237 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.238 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-03-21T11:49:46.238 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-03-21T11:49:46.238 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-03-21T11:49:46.238 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.238 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.238 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.238 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.238 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.238 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-03-21T11:49:46.238 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-03-21T11:49:46.238 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.238 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.238 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.239 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.241 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.241 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.241 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.241 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.241 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.241 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.241 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.241 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.241 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.241 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.241 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.241 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.241 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.241 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.242 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.242 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.242 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.242 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.242 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.242 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.242 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.242 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.242 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.242 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.242 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.242 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.242 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.243 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.243 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.243 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.243 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.243 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.243 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.243 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.243 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.243 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.243 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.243 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.243 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.244 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.244 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.244 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.244 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.244 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.244 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.244 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.244 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.244 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.244 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.244 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.244 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.244 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.245 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.245 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.245 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.245 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.245 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.245 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.245 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.245 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.245 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.245 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.245 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.245 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.246 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.246 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.246 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-03-21T11:49:46.246 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-03-21T11:49:46.246 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.246 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.246 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.246 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-03-21T11:49:46.246 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.246 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.246 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.246 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T11:49:46.246 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.247 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.247 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.247 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-03-21T11:49:46.247 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-03-21T11:49:46.247 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.247 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.247 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.247 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-03-21T11:49:46.247 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.247 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.247 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.247 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T11:49:46.247 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-03-21T11:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-03-21T11:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-03-21T11:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-03-21T11:49:46.248 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.249 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.249 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.249 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-03-21T11:49:46.249 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-03-21T11:49:46.249 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.249 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.249 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.249 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-03-21T11:49:46.249 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.249 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.249 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.249 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T11:49:46.249 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.250 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.250 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.250 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-03-21T11:49:46.250 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-03-21T11:49:46.250 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.250 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.250 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.250 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.250 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.250 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.250 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.250 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T11:49:46.251 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.251 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.251 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.251 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:46.251 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:46.251 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.251 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.251 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.251 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:46.251 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.251 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.251 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.252 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T11:49:46.252 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.252 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.252 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.252 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-03-21T11:49:46.252 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-03-21T11:49:46.252 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.252 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.252 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.252 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-03-21T11:49:46.252 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.252 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.253 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.253 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T11:49:46.253 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.253 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.253 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.253 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.253 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.253 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-03-21T11:49:46.253 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.253 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.253 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.254 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.254 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.254 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.254 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.254 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.254 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.254 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.254 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.254 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.254 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.254 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.254 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.254 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.254 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.255 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.255 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.255 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.255 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.255 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.255 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.255 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.255 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.255 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.255 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.255 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.255 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.255 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.256 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.256 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.256 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.256 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.256 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.256 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.256 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.256 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.256 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.256 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.256 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.256 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.258 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.258 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.258 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.258 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.258 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.258 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.258 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.258 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.258 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.258 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.258 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.258 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.259 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-03-21T11:49:46.259 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-03-21T11:49:46.259 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.259 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.259 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.259 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-03-21T11:49:46.259 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.259 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.259 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.259 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.259 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.259 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.259 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.260 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-03-21T11:49:46.260 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-03-21T11:49:46.260 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.260 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.260 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.260 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-03-21T11:49:46.260 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.260 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.260 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.260 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.260 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.260 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.260 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.261 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:46.261 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:46.261 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.261 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.261 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.261 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:46.261 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.261 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.261 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.261 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.261 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.261 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.261 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.262 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.262 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.262 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-03-21T11:49:46.262 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-03-21T11:49:46.262 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-03-21T11:49:46.262 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.262 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-03-21T11:49:46.262 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-03-21T11:49:46.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.262 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.262 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.263 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-03-21T11:49:46.263 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.263 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.263 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.263 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T11:49:46.263 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.263 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.263 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.263 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-03-21T11:49:46.263 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-03-21T11:49:46.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.263 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.264 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.264 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-03-21T11:49:46.264 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.264 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.264 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.264 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T11:49:46.264 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.264 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.264 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.264 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-03-21T11:49:46.264 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-03-21T11:49:46.264 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.265 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.265 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.265 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.265 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.265 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.265 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.265 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-03-21T11:49:46.265 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.265 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.265 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.265 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:46.265 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:46.265 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.266 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.266 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.266 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-03-21T11:49:46.266 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-03-21T11:49:46.266 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.266 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.266 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T11:49:46.266 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.266 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.266 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.266 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.266 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.267 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.267 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.267 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.267 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.267 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.267 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.267 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.267 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.267 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.267 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.267 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.267 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.267 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.269 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.269 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.269 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.269 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.270 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.270 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.270 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.270 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.270 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.270 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.270 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.270 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.270 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.270 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.270 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.270 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.271 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.271 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.271 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.271 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.271 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.271 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.271 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.271 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.271 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.271 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.271 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.271 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.271 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.272 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.272 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.272 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.272 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.272 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.272 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.272 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.272 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.272 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.272 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.272 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.272 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.272 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-03-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-03-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.274 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.274 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.274 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.274 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.274 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.274 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-03-21T11:49:46.274 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.274 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.274 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.274 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-03-21T11:49:46.274 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-03-21T11:49:46.274 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.275 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.275 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.275 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-03-21T11:49:46.275 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.275 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.275 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.275 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-03-21T11:49:46.275 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.275 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.275 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.275 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-03-21T11:49:46.275 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-03-21T11:49:46.275 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.276 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.277 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.277 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T11:49:46.277 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T11:49:46.277 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-03-21T11:49:46.277 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.277 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T11:49:46.277 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.277 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.277 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.277 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-03-21T11:49:46.277 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-03-21T11:49:46.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.278 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.278 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.278 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.278 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.278 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.278 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.278 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-03-21T11:49:46.278 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.278 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.278 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.278 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-03-21T11:49:46.279 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-03-21T11:49:46.279 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.279 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.279 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.279 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-03-21T11:49:46.279 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.279 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.279 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.279 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T11:49:46.279 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.279 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.279 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.279 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.279 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.280 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-03-21T11:49:46.280 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.280 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.280 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.280 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.280 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.280 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.280 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.280 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.280 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.280 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.281 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.281 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.281 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.281 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.281 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.281 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.281 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.281 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.281 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.281 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.281 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.281 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.281 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.282 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.282 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.282 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.282 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.282 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.282 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.282 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.282 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.282 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.282 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.282 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.282 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.282 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.283 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.283 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.283 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.283 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.283 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.283 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.283 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.283 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.283 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.283 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.283 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.283 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.283 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.284 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.285 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.285 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.285 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.285 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.285 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.285 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.285 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.285 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.285 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.285 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-03-21T11:49:46.285 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.285 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.286 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.286 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.286 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.286 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.286 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.286 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.286 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.286 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.286 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.286 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.286 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.286 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.286 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.287 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.288 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.288 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.288 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.288 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.288 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.288 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.288 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.288 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.288 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.288 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.288 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.288 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.288 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.290 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.290 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.290 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.290 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.290 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.290 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.290 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.290 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.290 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.290 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.290 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.290 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.290 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-03-21T11:49:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-03-21T11:49:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-03-21T11:49:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.291 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-03-21T11:49:46.292 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.292 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.292 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.292 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T11:49:46.292 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.292 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.292 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.292 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-03-21T11:49:46.292 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-03-21T11:49:46.292 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.292 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.292 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.293 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.293 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-03-21T11:49:46.293 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.293 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.293 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T11:49:46.293 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.293 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.293 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.293 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.293 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.293 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.293 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.293 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.294 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.294 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.294 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.294 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.294 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.294 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.294 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.294 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.294 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.294 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.294 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.294 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.294 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.294 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.295 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.295 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.295 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.295 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.295 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.295 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.295 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.295 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.295 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.295 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.295 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.295 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.296 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.296 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.296 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.296 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.296 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.296 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.296 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.296 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.296 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.296 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.296 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.296 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.296 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.297 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.297 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.297 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.297 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.297 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.297 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.297 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.297 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.297 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.297 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.297 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.297 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.297 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.298 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.298 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.298 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.298 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.298 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.298 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.298 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.298 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.298 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.298 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.298 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-03-21T11:49:46.298 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.299 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.299 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.299 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-03-21T11:49:46.299 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-03-21T11:49:46.299 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.299 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.299 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.299 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-03-21T11:49:46.299 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.299 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.299 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.299 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-03-21T11:49:46.299 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T11:49:46.299 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.300 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.300 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.300 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.300 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.300 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.300 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.300 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.300 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.300 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.300 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.300 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.300 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.301 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.301 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.301 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.301 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.301 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.301 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.301 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.301 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.301 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.301 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.301 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.301 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.301 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.302 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.302 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.302 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.302 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.302 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.302 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.302 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.302 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.302 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.302 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.302 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.302 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.303 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.304 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.304 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.304 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.304 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.304 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.304 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.304 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.304 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.304 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.304 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.304 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.304 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.305 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.305 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.305 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-03-21T11:49:46.305 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-03-21T11:49:46.305 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.305 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.305 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.305 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.305 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.305 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.305 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.305 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-03-21T11:49:46.306 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T11:49:46.306 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.306 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.306 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-03-21T11:49:46.306 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-03-21T11:49:46.306 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.306 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.306 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.306 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-03-21T11:49:46.306 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.306 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.307 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.307 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-03-21T11:49:46.307 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.307 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.307 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.307 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-03-21T11:49:46.307 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-03-21T11:49:46.307 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.307 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.307 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.307 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-03-21T11:49:46.307 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.308 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.308 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.308 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T11:49:46.308 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.308 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.308 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.308 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-03-21T11:49:46.308 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-03-21T11:49:46.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.308 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.308 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.308 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.308 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.309 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.309 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.309 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-03-21T11:49:46.309 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.309 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.309 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.309 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-03-21T11:49:46.309 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-03-21T11:49:46.309 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.309 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.309 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.310 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-03-21T11:49:46.310 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.310 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.310 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.310 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T11:49:46.310 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T11:49:46.310 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.310 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.310 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.310 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.310 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-03-21T11:49:46.310 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.311 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.311 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.311 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.311 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.311 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.311 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.311 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.311 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.311 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.311 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.311 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.311 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.311 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.312 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.312 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.312 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.312 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.312 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.312 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.312 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.312 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.312 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.312 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.312 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.312 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.312 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.313 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.313 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.313 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.313 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.313 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.313 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.313 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.313 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.313 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.313 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.313 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.313 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.313 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.314 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.314 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.314 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.314 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.314 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.314 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.314 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.314 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.314 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.314 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.314 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.314 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.314 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.314 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.315 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.315 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.315 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.315 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.315 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.315 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.315 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.315 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.315 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.315 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.315 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.315 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.315 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.316 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.317 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.317 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.317 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.317 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.317 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.317 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.317 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.317 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.317 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.317 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.317 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.317 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.317 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.317 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.318 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.318 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.318 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.318 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.318 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.318 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.318 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.318 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.318 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.318 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.318 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.318 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.318 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.319 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.319 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.319 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.319 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.319 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.319 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.319 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.319 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.319 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.319 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.319 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.319 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.319 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-03-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.322 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.322 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.322 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.322 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.322 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.322 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.322 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.322 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.322 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.322 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.322 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.322 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.322 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.322 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.323 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.323 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.323 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.323 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.323 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.323 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.323 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.323 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.323 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.323 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.323 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.323 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.323 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.324 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.325 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.326 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.326 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.326 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.326 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.326 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.326 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.326 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.326 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.326 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.326 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.326 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.326 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.326 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.327 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-03-21T11:49:46.327 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.327 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.327 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.327 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-03-21T11:49:46.327 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-03-21T11:49:46.327 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.327 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.327 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.327 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.328 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.328 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.328 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.328 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T11:49:46.328 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.328 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.328 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.328 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.328 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.328 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.328 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.328 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.328 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.329 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.329 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.329 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.329 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.329 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.329 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.329 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.329 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.329 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.329 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.329 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.329 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.329 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.329 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.330 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.331 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.332 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.332 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.332 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.332 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.332 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.332 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.332 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.332 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.332 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.332 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.332 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.332 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.332 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.333 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.333 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.333 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.333 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.333 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.333 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.333 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.333 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.333 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-03-21T11:49:46.333 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-03-21T11:49:46.333 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.333 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.333 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.334 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.334 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.334 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.334 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.334 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-03-21T11:49:46.334 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-03-21T11:49:46.334 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-03-21T11:49:46.334 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.334 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-03-21T11:49:46.334 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.334 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.334 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.335 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.335 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.335 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-03-21T11:49:46.335 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.335 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.335 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.335 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.335 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.335 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.335 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.335 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.335 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.336 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.336 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.336 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.336 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.336 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.336 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.336 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.336 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.336 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.336 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.336 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.336 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.336 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.336 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.337 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.337 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.337 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.337 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.337 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.337 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.337 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.337 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.337 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.337 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.337 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.337 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.337 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.338 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.338 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.338 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.338 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.338 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.338 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.338 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.338 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.338 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.338 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.338 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.338 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.339 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.339 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.339 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.339 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.339 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.339 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.339 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.339 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.339 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.339 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.339 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.339 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.339 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.340 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.340 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.340 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.340 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.340 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.340 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.340 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.340 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.340 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.340 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.340 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-03-21T11:49:46.340 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.341 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.341 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.341 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.341 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.341 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.341 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.341 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.341 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.341 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.341 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.341 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.342 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.342 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.342 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.342 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.342 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.342 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.342 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.342 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.342 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.342 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.342 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.342 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.342 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.343 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.343 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.343 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.343 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.343 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.343 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.343 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.343 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.343 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.343 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.343 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.343 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.343 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.344 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.344 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.344 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.344 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.344 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.344 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.344 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.344 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.344 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.344 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.344 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.344 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.344 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.344 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.345 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.345 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.345 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.345 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.345 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.345 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.345 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.345 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.345 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.345 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.345 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.345 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.345 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.346 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.346 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.346 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.346 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.346 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.346 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.346 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.346 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-03-21T11:49:46.346 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.346 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.346 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.346 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.347 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.347 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.347 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.347 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.347 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.347 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.347 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.347 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.347 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.347 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.347 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.347 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.347 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.347 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.348 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.348 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.348 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.348 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.348 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.348 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.348 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.348 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.348 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.348 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.348 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.348 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.348 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.349 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.349 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.349 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.349 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.349 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.349 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.349 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.349 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.349 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.349 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.349 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.349 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.349 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.350 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.350 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.350 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.350 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.350 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.350 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.350 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.350 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.350 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.350 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.350 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.350 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.351 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.351 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.351 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.351 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.351 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.351 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.351 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.351 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.351 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.351 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.351 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.351 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.351 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.352 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-03-21T11:49:46.353 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-03-21T11:49:46.353 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.353 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.353 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.353 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-03-21T11:49:46.353 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-03-21T11:49:46.353 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.353 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.353 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-03-21T11:49:46.353 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T11:49:46.353 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.353 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.354 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.354 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-03-21T11:49:46.354 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.354 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.354 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.354 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-03-21T11:49:46.354 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-03-21T11:49:46.354 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.354 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.354 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.354 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.354 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.355 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.355 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.355 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T11:49:46.355 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.355 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.355 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.355 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-03-21T11:49:46.355 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-03-21T11:49:46.355 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.355 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.355 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.355 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.355 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.356 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.356 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.356 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.356 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.356 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.356 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.356 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.356 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.356 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.356 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.356 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.358 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.358 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.358 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.358 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.358 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.358 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.358 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.358 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.358 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.358 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.358 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.358 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.359 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.360 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.360 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.360 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.360 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.360 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.360 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.360 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.360 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.360 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.360 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.360 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.360 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.360 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.361 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.361 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.361 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.361 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.361 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.361 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-03-21T11:49:46.361 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-03-21T11:49:46.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.361 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.361 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.361 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-03-21T11:49:46.361 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.361 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.362 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.362 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-03-21T11:49:46.362 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.362 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.362 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.362 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:46.362 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:46.362 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.362 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.362 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.362 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T11:49:46.363 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.363 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.363 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.363 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-03-21T11:49:46.363 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.363 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.363 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.363 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.363 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.363 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-03-21T11:49:46.363 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.363 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.364 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.364 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-03-21T11:49:46.364 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-03-21T11:49:46.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.364 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.364 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.364 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.364 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.364 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.364 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.364 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.364 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.364 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.364 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.365 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-03-21T11:49:46.365 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-03-21T11:49:46.365 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.365 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.365 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.365 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.365 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.365 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.365 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.365 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T11:49:46.365 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T11:49:46.365 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.365 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.366 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.366 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.366 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.366 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.366 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.366 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.366 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.366 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.366 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.366 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.366 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.366 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.367 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.368 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.368 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.368 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.368 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.368 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.368 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.368 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.368 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.368 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.368 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.368 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.368 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.368 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.369 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.369 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.369 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.369 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.369 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.369 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.369 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.369 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.369 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.369 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.369 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.369 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.369 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.369 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.371 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.371 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-03-21T11:49:46.371 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-03-21T11:49:46.371 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.371 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.371 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.371 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.371 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.371 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.371 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.371 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.371 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.371 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.372 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.372 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T11:49:46.372 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T11:49:46.372 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.372 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.372 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.372 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-03-21T11:49:46.372 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.372 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.372 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.372 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.372 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.372 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.373 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.374 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T11:49:46.374 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T11:49:46.374 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.374 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.374 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.374 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-03-21T11:49:46.374 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.374 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.374 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.374 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T11:49:46.374 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.374 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.375 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.375 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T11:49:46.375 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T11:49:46.375 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.375 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.375 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.375 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-03-21T11:49:46.375 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.375 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.375 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.375 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T11:49:46.375 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-03-21T11:49:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-03-21T11:49:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-03-21T11:49:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-03-21T11:49:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-03-21T11:49:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-03-21T11:49:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-03-21T11:49:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-03-21T11:49:46.378 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.378 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.378 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.378 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.378 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.378 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-03-21T11:49:46.378 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-03-21T11:49:46.378 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.378 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.378 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.378 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-03-21T11:49:46.378 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.379 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.380 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.380 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.380 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.380 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.380 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.380 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.380 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.380 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.380 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.380 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.380 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.380 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.380 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.381 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.382 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.382 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.382 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.382 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.382 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.382 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.382 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.382 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.382 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.382 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.382 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.382 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.382 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.383 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.383 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.383 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.383 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.383 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.383 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.383 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.383 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.383 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.383 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.383 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.383 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.384 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.384 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.384 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.384 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.384 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.384 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.384 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.384 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.384 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.384 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.384 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.384 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.384 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-03-21T11:49:46.385 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-03-21T11:49:46.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.385 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.385 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.385 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.385 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.385 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.385 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.385 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.385 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.385 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.385 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.385 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-03-21T11:49:46.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T11:49:46.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.386 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.386 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.386 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.386 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.386 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.386 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.386 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.386 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.386 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.386 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.386 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-03-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-03-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.388 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-03-21T11:49:46.388 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.388 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.388 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.388 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-03-21T11:49:46.388 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-03-21T11:49:46.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.388 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.388 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.388 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.388 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.388 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.389 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.389 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.389 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.389 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.389 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.389 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-03-21T11:49:46.389 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-03-21T11:49:46.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.390 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.390 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.390 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.390 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.390 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.390 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.390 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.390 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.390 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.390 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.390 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.390 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.391 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.392 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.392 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.392 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.392 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.392 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.392 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.392 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.392 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.392 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.392 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.392 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.393 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.393 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.393 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.393 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.393 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.393 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.393 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.393 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.393 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.393 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.393 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.393 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.394 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.394 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.394 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.394 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.394 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.394 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.394 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.394 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.394 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.394 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.394 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.394 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.396 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T11:49:46.396 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T11:49:46.396 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.396 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.396 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.396 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.396 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.396 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.396 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.396 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.396 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.396 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.396 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.397 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T11:49:46.397 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T11:49:46.397 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.397 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.397 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.397 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.397 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.397 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.397 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.397 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.397 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.397 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.397 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.397 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.398 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.398 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.398 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-03-21T11:49:46.398 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.398 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.398 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.398 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.398 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.398 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.398 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.398 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.398 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.398 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.399 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.399 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.399 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.399 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.399 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.399 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.399 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.399 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.399 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.399 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.399 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.399 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.399 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.400 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.400 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.400 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.400 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.400 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.400 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.400 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.400 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.400 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.400 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.400 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.400 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.400 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.400 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.401 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.401 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.401 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.401 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.401 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.401 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.401 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.401 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.401 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.401 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.401 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.401 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.401 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.401 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.402 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.402 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.402 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.402 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.402 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.402 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.402 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.402 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.402 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.402 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.402 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.402 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.402 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.402 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.403 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.403 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.403 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.403 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.403 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.403 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.403 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.403 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.403 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.403 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.403 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-03-21T11:49:46.403 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.403 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.404 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.404 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.404 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.404 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.404 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.404 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.404 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.404 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.404 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.404 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.404 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.404 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.404 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.404 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.405 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.405 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.405 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.405 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.405 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.405 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.405 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.405 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.405 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.405 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.405 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.405 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.405 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.406 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.406 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.406 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.406 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.406 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.406 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.406 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.406 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.406 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.406 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.406 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.406 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.407 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.407 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.407 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.407 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.407 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.407 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.407 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.407 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.407 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.407 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.407 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.407 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.407 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.408 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.408 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.408 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.408 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.408 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.408 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.408 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.408 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.408 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.408 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.408 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.408 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.408 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.409 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.409 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.409 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-03-21T11:49:46.409 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-03-21T11:49:46.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.409 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.409 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.409 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-03-21T11:49:46.409 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.409 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.409 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.409 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T11:49:46.410 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.410 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.410 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.410 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.410 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.410 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-03-21T11:49:46.410 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.410 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.410 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.410 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.410 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.410 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.411 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.411 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.411 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.411 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.411 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.411 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.411 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.411 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.411 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.411 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.411 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.411 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.411 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.413 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.413 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.413 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.413 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.413 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.413 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.413 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.413 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.413 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.413 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.413 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.414 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.414 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.414 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.414 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.414 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.414 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.414 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.414 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.414 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.414 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.414 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.414 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.415 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.415 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.415 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.415 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.415 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.415 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.415 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.415 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.415 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.415 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.415 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.415 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-03-21T11:49:46.416 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-03-21T11:49:46.416 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.416 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.416 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.416 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.416 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.416 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.416 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.416 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.416 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.416 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.416 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.416 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-03-21T11:49:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-03-21T11:49:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-03-21T11:49:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.417 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-03-21T11:49:46.418 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-03-21T11:49:46.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.418 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.418 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.418 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.418 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.418 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.418 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.418 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.418 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.418 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.418 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.419 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-03-21T11:49:46.419 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-03-21T11:49:46.419 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.419 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.419 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.419 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.419 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.419 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.419 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.419 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.419 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.419 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.419 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.420 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-03-21T11:49:46.420 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-03-21T11:49:46.420 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.420 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.420 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.420 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.420 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.420 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.420 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.420 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.420 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.420 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.421 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.421 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-03-21T11:49:46.421 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-03-21T11:49:46.421 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.421 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.421 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.421 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.421 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.421 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.421 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.421 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.421 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.421 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-03-21T11:49:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-03-21T11:49:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-03-21T11:49:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-03-21T11:49:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T11:49:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-03-21T11:49:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-03-21T11:49:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-03-21T11:49:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-03-21T11:49:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-03-21T11:49:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-03-21T11:49:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-03-21T11:49:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-03-21T11:49:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-03-21T11:49:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-03-21T11:49:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-03-21T11:49:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-03-21T11:49:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-03-21T11:49:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-03-21T11:49:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-03-21T11:49:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T11:49:46.447 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.448 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-03-21T11:49:46.449 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.449 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.449 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.449 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.449 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.449 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.449 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.449 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.449 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.449 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.449 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.449 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.450 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.450 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.450 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.450 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.450 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.450 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.450 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.450 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.450 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.450 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.450 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.450 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.450 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.451 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.451 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.451 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.451 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.451 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.451 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.451 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.451 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.451 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.451 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.451 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.451 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.452 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.452 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.452 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.452 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.452 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.452 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.452 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.452 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.452 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.452 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.452 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.452 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.452 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.453 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.453 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.453 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.453 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.453 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.453 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.453 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.453 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.453 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.453 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.453 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.453 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.453 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.454 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.454 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.454 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.454 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.454 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.454 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.454 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.454 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.454 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.454 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.454 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.454 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.455 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.455 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-03-21T11:49:46.455 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.455 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.455 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.455 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-03-21T11:49:46.455 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-03-21T11:49:46.455 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.455 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.455 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.455 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.455 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.455 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.456 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.456 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T11:49:46.456 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.456 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.456 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.456 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.456 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-03-21T11:49:46.456 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-03-21T11:49:46.456 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.456 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.456 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.456 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.456 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.457 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.457 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.457 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T11:49:46.457 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.457 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.457 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.457 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-03-21T11:49:46.457 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-03-21T11:49:46.457 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.457 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.457 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.457 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.457 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.458 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.458 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.458 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T11:49:46.458 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.458 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.458 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.458 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-03-21T11:49:46.458 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-03-21T11:49:46.458 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.458 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.458 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.458 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.459 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.459 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.459 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.459 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T11:49:46.459 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.459 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.459 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.459 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-03-21T11:49:46.459 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-03-21T11:49:46.459 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.459 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.459 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.459 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.460 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.460 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.460 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.460 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T11:49:46.460 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.460 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.460 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.460 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-03-21T11:49:46.460 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-03-21T11:49:46.460 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.460 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.460 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.461 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.461 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.461 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.461 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.461 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.461 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.461 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.461 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.461 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-03-21T11:49:46.461 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-03-21T11:49:46.461 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.461 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.461 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.462 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.462 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.462 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.462 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.462 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.462 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.462 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.462 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.462 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-03-21T11:49:46.462 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-03-21T11:49:46.462 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.462 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.463 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.463 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T11:49:46.463 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.463 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.463 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.463 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.463 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.463 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.463 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.463 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-03-21T11:49:46.463 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-03-21T11:49:46.463 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.463 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.464 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.464 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.464 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.464 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.464 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.464 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.464 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.464 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.464 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.464 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:46.464 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:46.464 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.464 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.465 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.466 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-03-21T11:49:46.466 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.466 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.466 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.466 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.466 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.466 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.466 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.466 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-03-21T11:49:46.466 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-03-21T11:49:46.466 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.466 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.466 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.467 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.467 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.467 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.467 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.467 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.467 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.467 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.467 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.467 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-03-21T11:49:46.467 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-03-21T11:49:46.467 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.467 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.468 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.469 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.469 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.469 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.469 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.469 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.469 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.469 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.469 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.469 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.469 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.469 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.469 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.469 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.469 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.470 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.470 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.470 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.470 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.470 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.470 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.470 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.470 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.470 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.470 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.470 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.470 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.471 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.471 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.471 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.471 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.471 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.471 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.471 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.471 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.471 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.471 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.471 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.471 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.471 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.472 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.473 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.474 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.474 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.474 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.474 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.474 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.474 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.474 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.474 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-03-21T11:49:46.474 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-03-21T11:49:46.474 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.474 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.474 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.475 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.475 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.475 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.475 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.475 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.475 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.475 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.475 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.475 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-03-21T11:49:46.475 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-03-21T11:49:46.475 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.475 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.475 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.476 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T11:49:46.476 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.476 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.476 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.476 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.476 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.476 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.476 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.476 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.476 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.476 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-03-21T11:49:46.477 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.477 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.477 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.477 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.477 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.477 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.477 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.477 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.477 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.477 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.477 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.477 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.477 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.477 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.478 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.478 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.478 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.478 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.478 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.478 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.478 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.478 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.478 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.478 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.478 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.478 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.479 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.479 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.479 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.479 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.479 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.479 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.479 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.479 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.479 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.479 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.479 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.479 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.479 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.480 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.480 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.480 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.480 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.480 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.480 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.480 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.480 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.480 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.480 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.480 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.480 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.480 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.480 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.481 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.482 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.482 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.482 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.482 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.482 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-03-21T11:49:46.482 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.482 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.482 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.482 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.482 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.482 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.482 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.483 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.483 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.483 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.483 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.483 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.483 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.483 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.483 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.483 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.483 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.483 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.483 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.483 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.483 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.484 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.485 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.485 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.485 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.485 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.485 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.485 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.485 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.485 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.485 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.485 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.485 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.485 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.486 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.486 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.486 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.486 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.486 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.486 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.486 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.486 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.486 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.486 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.486 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.486 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.486 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.487 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.488 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-03-21T11:49:46.488 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.488 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.488 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.488 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T11:49:46.488 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.488 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.488 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.488 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-03-21T11:49:46.488 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-03-21T11:49:46.488 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.488 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.489 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.489 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.489 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.489 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.489 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.489 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-03-21T11:49:46.489 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.489 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.489 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.489 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-03-21T11:49:46.489 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-03-21T11:49:46.489 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.489 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.490 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.490 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:46.490 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.490 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.490 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.490 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T11:49:46.490 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.490 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.490 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.490 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-03-21T11:49:46.490 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T11:49:46.490 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.491 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.491 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.491 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.491 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.491 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.491 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.491 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T11:49:46.491 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.491 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.491 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.491 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.491 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.492 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-03-21T11:49:46.492 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.492 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.492 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.492 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-03-21T11:49:46.492 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-03-21T11:49:46.492 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:46.492 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.492 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.492 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-03-21T11:49:46.492 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.492 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.492 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.493 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.494 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.495 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.495 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.495 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.495 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.495 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.495 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.495 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.495 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.495 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.495 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.495 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.495 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.495 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.496 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.496 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.496 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.496 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.496 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.496 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.496 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.496 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.496 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.496 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.496 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.496 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.497 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.497 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.497 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.497 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.497 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.497 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.497 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.497 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.497 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.497 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.497 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.497 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.497 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.498 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.498 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.498 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.498 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.498 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.498 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.498 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.498 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.498 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.498 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.498 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.498 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.498 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.498 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.499 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.500 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.500 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.500 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-03-21T11:49:46.500 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-03-21T11:49:46.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.500 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.500 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.500 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.500 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.500 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.500 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.500 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.500 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.501 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.502 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.502 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-03-21T11:49:46.502 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-03-21T11:49:46.502 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.502 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.502 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.502 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T11:49:46.502 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.502 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.502 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.502 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.502 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.503 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.503 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.503 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.503 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.503 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.503 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T11:49:46.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.503 INFO:teuthology.orchestra.run.smithi047.stdout: "gid": 24113, 2024-03-21T11:49:46.503 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "y", 2024-03-21T11:49:46.503 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_features": 4540701547738038271, 2024-03-21T11:49:46.503 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-03-21T11:49:46.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.503 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.504 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.505 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.505 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.505 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.505 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.505 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.505 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.505 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.505 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.505 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.505 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.505 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.505 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.506 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.506 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.506 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.506 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.506 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.506 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.506 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.506 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.506 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.506 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.506 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.506 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.507 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.508 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.508 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.508 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.508 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.508 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.508 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.508 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.508 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.508 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.508 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.508 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.508 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.508 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.508 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.509 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.510 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.510 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-03-21T11:49:46.510 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-03-21T11:49:46.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.510 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.510 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.510 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.510 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.510 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.510 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.510 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-03-21T11:49:46.510 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.511 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.512 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.512 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-03-21T11:49:46.512 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-03-21T11:49:46.512 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.512 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.512 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.512 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.512 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.512 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.512 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.512 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-03-21T11:49:46.512 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.512 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.513 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.514 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-03-21T11:49:46.514 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-03-21T11:49:46.514 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.514 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.514 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.514 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-03-21T11:49:46.514 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.514 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.514 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.514 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-03-21T11:49:46.514 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.514 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.514 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.515 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-03-21T11:49:46.515 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-03-21T11:49:46.515 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.515 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.515 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.515 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.515 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.515 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.515 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.515 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-03-21T11:49:46.515 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.515 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.516 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.517 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-03-21T11:49:46.517 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-03-21T11:49:46.517 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.517 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.517 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.517 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.517 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.517 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.517 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.517 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-03-21T11:49:46.517 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.517 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.517 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.518 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.518 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.518 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-03-21T11:49:46.518 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.518 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.518 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.518 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-03-21T11:49:46.518 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-03-21T11:49:46.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.518 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.518 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.519 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.519 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.519 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.519 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.519 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T11:49:46.519 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.519 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.519 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.519 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-03-21T11:49:46.519 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-03-21T11:49:46.519 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.519 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.519 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.520 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-03-21T11:49:46.521 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T11:49:46.522 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-03-21T11:49:46.522 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.522 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T11:49:46.522 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.522 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.522 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.522 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-03-21T11:49:46.522 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-03-21T11:49:46.522 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.522 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.522 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.522 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-03-21T11:49:46.523 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-03-21T11:49:46.524 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-03-21T11:49:46.524 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.524 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-03-21T11:49:46.524 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T11:49:46.524 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.524 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.524 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-03-21T11:49:46.524 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-03-21T11:49:46.524 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.524 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.524 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.524 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-03-21T11:49:46.525 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.525 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.525 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.525 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T11:49:46.525 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T11:49:46.525 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.525 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.525 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-03-21T11:49:46.525 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-03-21T11:49:46.525 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:46.525 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.525 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.525 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:46.526 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-03-21T11:49:46.526 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-03-21T11:49:46.526 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.526 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T11:49:46.526 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T11:49:46.526 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.526 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.526 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.526 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.526 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.526 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.526 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.526 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.527 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.527 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.527 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.527 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.527 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.527 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.527 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.527 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.527 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.527 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.527 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.527 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.527 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.527 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.528 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.528 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.528 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.528 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.528 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.528 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.528 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.528 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.528 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.528 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.528 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.528 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.528 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.529 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.529 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.529 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.529 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.529 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.529 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.529 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.529 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.529 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.529 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.529 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.529 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.529 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.530 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.530 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.530 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.530 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.530 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.530 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.530 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.530 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.530 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.530 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.530 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.530 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.531 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.531 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.531 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.531 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.531 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.531 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.531 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.531 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.531 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-03-21T11:49:46.531 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-03-21T11:49:46.531 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.531 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.532 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.534 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-03-21T11:49:46.534 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-03-21T11:49:46.534 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.534 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.534 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.534 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.534 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.534 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.534 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.534 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T11:49:46.534 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.534 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.534 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.535 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:46.535 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:46.535 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.535 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.535 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.535 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:46.535 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.535 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.535 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.535 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T11:49:46.535 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.535 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.535 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.536 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.536 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-03-21T11:49:46.536 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-03-21T11:49:46.536 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.536 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.536 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.536 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T11:49:46.536 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T11:49:46.536 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.536 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.536 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T11:49:46.536 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T11:49:46.536 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.537 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.538 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.538 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.538 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.538 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-03-21T11:49:46.538 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.538 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.538 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.538 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-03-21T11:49:46.538 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-03-21T11:49:46.538 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.538 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.538 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.539 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-03-21T11:49:46.539 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.539 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.539 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.539 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T11:49:46.539 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.539 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.539 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.539 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-03-21T11:49:46.539 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-03-21T11:49:46.539 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.540 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.540 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.540 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-03-21T11:49:46.540 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.540 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.540 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.540 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T11:49:46.540 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.540 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.540 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.540 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-03-21T11:49:46.540 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-03-21T11:49:46.540 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.541 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.541 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.541 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-03-21T11:49:46.541 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.541 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.541 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.541 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T11:49:46.541 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.541 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.541 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.541 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-03-21T11:49:46.541 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-03-21T11:49:46.542 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.542 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.542 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.542 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.542 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.542 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.542 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.542 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T11:49:46.542 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T11:49:46.542 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.542 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.542 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-03-21T11:49:46.543 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-03-21T11:49:46.543 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.543 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.543 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.543 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:46.543 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.543 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.543 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.543 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T11:49:46.543 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.543 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.543 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.544 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-03-21T11:49:46.545 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-03-21T11:49:46.545 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.545 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.545 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.545 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.545 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.545 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.545 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.545 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-03-21T11:49:46.545 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-03-21T11:49:46.545 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-03-21T11:49:46.545 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.545 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T11:49:46.545 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.546 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.546 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.546 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-03-21T11:49:46.546 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-03-21T11:49:46.546 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.546 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.546 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.546 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.546 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.546 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.546 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.547 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T11:49:46.547 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.547 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.547 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.547 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-03-21T11:49:46.547 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-03-21T11:49:46.547 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.547 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.547 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.547 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.547 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.547 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.547 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.548 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T11:49:46.548 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.548 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.548 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.548 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-03-21T11:49:46.548 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-03-21T11:49:46.548 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.548 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.548 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.548 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.548 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.548 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.548 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.549 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T11:49:46.550 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.550 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.550 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.550 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-03-21T11:49:46.550 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-03-21T11:49:46.550 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.550 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.550 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.550 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T11:49:46.550 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.550 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.550 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.551 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-03-21T11:49:46.552 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.552 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.552 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.552 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-03-21T11:49:46.552 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-03-21T11:49:46.552 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.552 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.552 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.552 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T11:49:46.552 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.552 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.552 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.553 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T11:49:46.553 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.553 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.553 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.553 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-03-21T11:49:46.553 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-03-21T11:49:46.553 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.553 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.553 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.553 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T11:49:46.553 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.553 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.554 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.554 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-03-21T11:49:46.554 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.554 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.554 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.554 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-03-21T11:49:46.554 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-03-21T11:49:46.554 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.554 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.554 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.554 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T11:49:46.554 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.555 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-03-21T11:49:46.556 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.557 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.558 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.558 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-03-21T11:49:46.558 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-03-21T11:49:46.558 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.558 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.558 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.558 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T11:49:46.558 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.558 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.558 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.558 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-03-21T11:49:46.558 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.558 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-03-21T11:49:46.560 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-03-21T11:49:46.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.561 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.561 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.561 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T11:49:46.561 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.561 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.561 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.561 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T11:49:46.561 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.561 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.561 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.561 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-03-21T11:49:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-03-21T11:49:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T11:49:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-03-21T11:49:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-03-21T11:49:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-03-21T11:49:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T11:49:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-03-21T11:49:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-03-21T11:49:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-03-21T11:49:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T11:49:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-03-21T11:49:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-03-21T11:49:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T11:49:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.565 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-03-21T11:49:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T11:49:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-03-21T11:49:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T11:49:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.566 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-03-21T11:49:46.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-03-21T11:49:46.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.570 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.570 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.570 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:46.570 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.570 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.570 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.570 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-03-21T11:49:46.570 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.570 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.570 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.570 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-03-21T11:49:46.570 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-03-21T11:49:46.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.571 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.571 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.571 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T11:49:46.571 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.571 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.571 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.571 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T11:49:46.571 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.571 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.571 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.571 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-03-21T11:49:46.571 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.572 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.573 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.574 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.574 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.574 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.574 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.574 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T11:49:46.574 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.574 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.574 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.574 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.574 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.574 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.574 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.574 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.576 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.577 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.577 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.577 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.577 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.577 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.577 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.577 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.577 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T11:49:46.577 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.577 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.577 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.577 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.578 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.579 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.579 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.579 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.579 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.579 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.579 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.579 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.579 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.579 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.579 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.579 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.580 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.580 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.580 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.580 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.580 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.580 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T11:49:46.580 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T11:49:46.580 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.580 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.580 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.580 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.580 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.581 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.581 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.581 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T11:49:46.581 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.581 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.581 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.581 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T11:49:46.581 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T11:49:46.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.581 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.581 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.581 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-03-21T11:49:46.582 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.582 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.582 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.582 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T11:49:46.582 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.582 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.582 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.582 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-03-21T11:49:46.582 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-03-21T11:49:46.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.582 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.582 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.582 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.583 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.583 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.583 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.583 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-03-21T11:49:46.583 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.583 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.583 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.583 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-03-21T11:49:46.583 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-03-21T11:49:46.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.583 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.583 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.584 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.584 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.584 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.584 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.584 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T11:49:46.584 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.584 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.584 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.584 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-03-21T11:49:46.584 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-03-21T11:49:46.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.584 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.584 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.585 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.585 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.585 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.585 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.585 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-03-21T11:49:46.585 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.585 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.585 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.585 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-03-21T11:49:46.585 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-03-21T11:49:46.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.585 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.586 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.586 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-03-21T11:49:46.586 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.586 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.586 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.586 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-03-21T11:49:46.586 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-03-21T11:49:46.586 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.586 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T11:49:46.586 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.586 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.586 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.587 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-03-21T11:49:46.587 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-03-21T11:49:46.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.587 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.587 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.587 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-03-21T11:49:46.587 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.587 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.587 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.587 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T11:49:46.587 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.587 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.587 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.588 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-03-21T11:49:46.588 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-03-21T11:49:46.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.588 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.588 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.588 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T11:49:46.588 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.588 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.588 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.588 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T11:49:46.588 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.588 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.588 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-03-21T11:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-03-21T11:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T11:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.589 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.590 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-03-21T11:49:46.590 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-03-21T11:49:46.590 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.590 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.590 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.590 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.590 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.590 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.590 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.590 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T11:49:46.590 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.590 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.591 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.591 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-03-21T11:49:46.591 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-03-21T11:49:46.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.591 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.591 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.591 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.591 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.591 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.591 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.591 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T11:49:46.591 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.591 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.592 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.593 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-03-21T11:49:46.593 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-03-21T11:49:46.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.593 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.593 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.593 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.593 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.593 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.593 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.593 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T11:49:46.593 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.593 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.594 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.594 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-03-21T11:49:46.594 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-03-21T11:49:46.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.594 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.594 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.594 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-03-21T11:49:46.594 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.594 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.594 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.594 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-03-21T11:49:46.594 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.595 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.595 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.595 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-03-21T11:49:46.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-03-21T11:49:46.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.595 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.595 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.595 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.595 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.595 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.595 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.595 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T11:49:46.595 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.596 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.596 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.596 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-03-21T11:49:46.596 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-03-21T11:49:46.596 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.596 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.596 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.596 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.596 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.596 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.596 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.596 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T11:49:46.596 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.597 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.598 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.598 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-03-21T11:49:46.598 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-03-21T11:49:46.598 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.598 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.598 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.598 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.598 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.598 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.598 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.598 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T11:49:46.598 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.599 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-03-21T11:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-03-21T11:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T11:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.600 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.601 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.602 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.602 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.602 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.602 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.602 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.602 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.602 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.602 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.602 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.602 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.602 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.602 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.602 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.603 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.603 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.603 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.603 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.603 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.603 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.603 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.603 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.603 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.603 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.603 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.603 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.604 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.604 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.604 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.604 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.604 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.604 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.604 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.604 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.604 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.604 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.604 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.604 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.605 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.606 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.606 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.606 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.606 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.606 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.606 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.606 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.606 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-03-21T11:49:46.606 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-03-21T11:49:46.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.606 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.606 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.606 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-03-21T11:49:46.607 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.608 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.608 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.608 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T11:49:46.608 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.608 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.608 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.608 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-03-21T11:49:46.608 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.608 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.610 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T11:49:46.610 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T11:49:46.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.610 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.610 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.610 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.610 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.610 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.610 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.610 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.610 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.610 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.610 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.611 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T11:49:46.611 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T11:49:46.611 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.611 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.611 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.611 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.611 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.611 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.611 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.611 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.611 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.611 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.611 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.612 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-03-21T11:49:46.612 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T11:49:46.612 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.612 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.612 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.612 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.612 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.612 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.612 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.612 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.612 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.612 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.612 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.614 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T11:49:46.614 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T11:49:46.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.614 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.614 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.614 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.614 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.614 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.614 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.614 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.614 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.614 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.614 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.615 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T11:49:46.615 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T11:49:46.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.615 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.615 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.615 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.615 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.615 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.615 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.615 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.615 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.615 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.615 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.616 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T11:49:46.616 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T11:49:46.616 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.616 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.616 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.616 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.616 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.616 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.616 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.616 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.616 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.617 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.618 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.618 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T11:49:46.618 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T11:49:46.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.618 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.618 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.618 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.618 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.618 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.618 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.618 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.618 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.618 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.619 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.619 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T11:49:46.619 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T11:49:46.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.619 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.619 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.619 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.619 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.619 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.619 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.619 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.619 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.620 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.620 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.620 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T11:49:46.620 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T11:49:46.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.620 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.620 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.620 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.620 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.620 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.620 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.620 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.620 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.621 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.621 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.621 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T11:49:46.621 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T11:49:46.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.621 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.621 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.621 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.621 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.621 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.621 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.622 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.622 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.622 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.622 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.622 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T11:49:46.622 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T11:49:46.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.622 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.622 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.622 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.622 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.622 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.622 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.623 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.623 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.623 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.623 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.623 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T11:49:46.623 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T11:49:46.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.623 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.623 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.623 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T11:49:46.623 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.623 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.624 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.624 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.624 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.624 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.624 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.624 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T11:49:46.624 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T11:49:46.624 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.624 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.624 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.624 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.624 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.624 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.625 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.625 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.625 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.625 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.625 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.625 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-03-21T11:49:46.625 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-03-21T11:49:46.625 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.625 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.625 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.625 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.625 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.626 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.626 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.626 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.626 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.626 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.626 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.626 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T11:49:46.626 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T11:49:46.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.626 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.626 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.626 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T11:49:46.626 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.627 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.627 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.627 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.627 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.627 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.627 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.627 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T11:49:46.627 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T11:49:46.627 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.627 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.627 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.627 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.627 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.628 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.628 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.628 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.628 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.628 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.628 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.628 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T11:49:46.628 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T11:49:46.628 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.628 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.628 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.628 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.628 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T11:49:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T11:49:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T11:49:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T11:49:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T11:49:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T11:49:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-03-21T11:49:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T11:49:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T11:49:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T11:49:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T11:49:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T11:49:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T11:49:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T11:49:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T11:49:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T11:49:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T11:49:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T11:49:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T11:49:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T11:49:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T11:49:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T11:49:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T11:49:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-03-21T11:49:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T11:49:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T11:49:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-03-21T11:49:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T11:49:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T11:49:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T11:49:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T11:49:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T11:49:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T11:49:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T11:49:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T11:49:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T11:49:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T11:49:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T11:49:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.651 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.652 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:46.652 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.652 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.652 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.652 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.652 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.652 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.652 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.652 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T11:49:46.652 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T11:49:46.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.652 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.652 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.653 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T11:49:46.654 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.654 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.654 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.654 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.654 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.654 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.654 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.654 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-03-21T11:49:46.654 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-03-21T11:49:46.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.654 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.654 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-03-21T11:49:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-03-21T11:49:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.656 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-03-21T11:49:46.657 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.658 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.658 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.658 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.658 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.658 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.658 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.658 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-03-21T11:49:46.658 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-03-21T11:49:46.658 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.658 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.658 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.658 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.658 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.659 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.660 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.660 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.660 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.660 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.660 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.660 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.660 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.660 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.660 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.660 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.660 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.660 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.661 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.661 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.661 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.661 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.661 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.661 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.661 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.661 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.661 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.661 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.661 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.661 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.661 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.662 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.662 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.662 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.662 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.662 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.662 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.662 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.662 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.662 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.662 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.662 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.662 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.662 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.663 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.663 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.663 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.663 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.663 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.663 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.663 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.663 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.663 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.663 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.663 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.663 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.663 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.664 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.664 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.664 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.664 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.664 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.664 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.664 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.664 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-03-21T11:49:46.664 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-03-21T11:49:46.664 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.664 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.664 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.664 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.665 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.666 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.666 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.666 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.666 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.666 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.666 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.666 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T11:49:46.666 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T11:49:46.666 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.666 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.666 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.666 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-03-21T11:49:46.666 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.667 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.667 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.667 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.667 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.667 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.667 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.667 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T11:49:46.667 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T11:49:46.667 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.667 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.667 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.667 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-03-21T11:49:46.667 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.668 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.668 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.668 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.668 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.668 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.668 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.668 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-03-21T11:49:46.668 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-03-21T11:49:46.668 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.668 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.668 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.668 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.668 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.669 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.669 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.669 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.669 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.669 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.669 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.669 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-03-21T11:49:46.669 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-03-21T11:49:46.669 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.669 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.669 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.669 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-03-21T11:49:46.670 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.670 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.670 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.670 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.670 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.670 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.670 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.670 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.670 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-03-21T11:49:46.670 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-03-21T11:49:46.670 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.670 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.670 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.671 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.672 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.672 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-03-21T11:49:46.672 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-03-21T11:49:46.672 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-03-21T11:49:46.672 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.672 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.672 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.672 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.672 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.672 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-03-21T11:49:46.672 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-03-21T11:49:46.672 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.672 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.673 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.674 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.674 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-03-21T11:49:46.674 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-03-21T11:49:46.674 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.674 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.674 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.674 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.674 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.674 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.674 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.674 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-03-21T11:49:46.674 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.674 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.675 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.676 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.676 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.676 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.676 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.676 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.676 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.676 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.676 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.676 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.677 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.677 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.677 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.678 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.678 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.678 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.678 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.678 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.678 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.678 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.678 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.678 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.678 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.678 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.678 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.680 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.681 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-03-21T11:49:46.682 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-03-21T11:49:46.683 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.684 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.684 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.684 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-03-21T11:49:46.684 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.684 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.684 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.684 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T11:49:46.684 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.684 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.684 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.684 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-03-21T11:49:46.684 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-03-21T11:49:46.684 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.685 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.685 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.685 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.685 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.685 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.685 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.685 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T11:49:46.685 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.685 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.685 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.685 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:46.685 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:46.686 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.686 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.686 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.686 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:46.686 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.686 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.686 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.686 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T11:49:46.686 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.686 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.686 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.686 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-03-21T11:49:46.686 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.687 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.688 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.689 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.690 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.690 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.690 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.690 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.690 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.690 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.690 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.690 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.690 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.690 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.690 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.690 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.691 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.692 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.692 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.692 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.692 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.692 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.692 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.692 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.692 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.692 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.692 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.692 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.693 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.693 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.693 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.693 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-03-21T11:49:46.693 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-03-21T11:49:46.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.693 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.693 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.693 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-03-21T11:49:46.693 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.693 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.693 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.693 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.694 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.694 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.694 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.694 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-03-21T11:49:46.694 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-03-21T11:49:46.694 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.694 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.694 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.694 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-03-21T11:49:46.694 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.694 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.694 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.695 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.695 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.695 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.695 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.695 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:46.695 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:46.695 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.695 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.695 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.695 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:46.695 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.695 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-03-21T11:49:46.696 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-03-21T11:49:46.697 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.697 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.697 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.697 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-03-21T11:49:46.697 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.697 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.697 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.697 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T11:49:46.697 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.697 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.697 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.697 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-03-21T11:49:46.698 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-03-21T11:49:46.698 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.698 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.698 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.698 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-03-21T11:49:46.698 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.698 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.698 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.698 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T11:49:46.698 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.698 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.698 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.699 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-03-21T11:49:46.699 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-03-21T11:49:46.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.699 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.699 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.699 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.699 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.699 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.699 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.699 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-03-21T11:49:46.699 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.699 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.699 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.700 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:46.700 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:46.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.700 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.700 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.700 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-03-21T11:49:46.700 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-03-21T11:49:46.700 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.700 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.700 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T11:49:46.700 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.700 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.700 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.701 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.702 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.703 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.703 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.703 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.703 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.703 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.703 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.703 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.703 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.703 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.703 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.703 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.703 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.703 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.704 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.705 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.705 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.705 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.705 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.705 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.705 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.705 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.705 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.705 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.705 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-03-21T11:49:46.705 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-03-21T11:49:46.705 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.705 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-03-21T11:49:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-03-21T11:49:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-03-21T11:49:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.706 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.707 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.707 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-03-21T11:49:46.707 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.707 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.707 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.707 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-03-21T11:49:46.707 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.707 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.707 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.707 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-03-21T11:49:46.707 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-03-21T11:49:46.707 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.707 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.708 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.708 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-03-21T11:49:46.708 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.708 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.708 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.708 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T11:49:46.708 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.708 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.708 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.708 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-03-21T11:49:46.708 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-03-21T11:49:46.708 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.708 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.709 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.709 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T11:49:46.709 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T11:49:46.709 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-03-21T11:49:46.709 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.709 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T11:49:46.709 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.709 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.709 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.709 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-03-21T11:49:46.709 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-03-21T11:49:46.709 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.709 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.710 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.710 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.710 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.710 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.710 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.710 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-03-21T11:49:46.710 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.710 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.710 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.710 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-03-21T11:49:46.710 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-03-21T11:49:46.711 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.711 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.711 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.711 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-03-21T11:49:46.711 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.711 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.711 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.711 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T11:49:46.711 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.711 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.711 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.711 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.711 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-03-21T11:49:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.713 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.713 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.713 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.713 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.713 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.713 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.713 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.713 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.713 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.713 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.713 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.713 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.713 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.714 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.714 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.714 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.714 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.714 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.714 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.714 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.714 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.714 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.714 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.714 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.714 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.715 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.715 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.715 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.715 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.715 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.715 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.715 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.715 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.715 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.715 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.715 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.715 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.715 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.715 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.716 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.716 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.716 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.716 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.716 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.716 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.716 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.716 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.716 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.716 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.716 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.716 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.717 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.718 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.718 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.718 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.718 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.718 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.718 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.718 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.718 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.718 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.718 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.718 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.718 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.718 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.719 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.719 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.719 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.719 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.719 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.719 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.719 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.719 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.719 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.719 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.719 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.719 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.719 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.720 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.720 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.720 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.720 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.720 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.720 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.720 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.720 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.720 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.720 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.720 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.720 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.720 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.722 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.722 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.722 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.722 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.722 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.722 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.722 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.722 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.722 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.722 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.722 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.722 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.722 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.722 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.723 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.723 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-03-21T11:49:46.723 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.723 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.723 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.723 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-03-21T11:49:46.723 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-03-21T11:49:46.723 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.723 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.723 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.723 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-03-21T11:49:46.723 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.724 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.724 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.724 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T11:49:46.724 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.724 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.724 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.724 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-03-21T11:49:46.724 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-03-21T11:49:46.724 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.724 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.724 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.724 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.725 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.726 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.726 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.726 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.726 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.726 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.726 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.726 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.726 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.726 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.726 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.726 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.726 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.726 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.727 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.728 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.728 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.728 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.728 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.728 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.728 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.728 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.728 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.728 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.728 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.728 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.728 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.728 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.728 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.729 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.729 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.729 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.729 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.729 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.729 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.729 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.729 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.729 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.729 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.729 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.729 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.729 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.730 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.730 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.730 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.730 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.730 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.730 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.730 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.730 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-03-21T11:49:46.730 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.730 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.730 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.731 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-03-21T11:49:46.731 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-03-21T11:49:46.731 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.731 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.731 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.731 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-03-21T11:49:46.731 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.731 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.731 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.731 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-03-21T11:49:46.731 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T11:49:46.731 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.731 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.732 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.733 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.733 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.733 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.733 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.733 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.733 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.733 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.733 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.733 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.733 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.733 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.733 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.733 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.734 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.734 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.734 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.734 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.734 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.734 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.734 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.734 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.734 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.734 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.734 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.734 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.735 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.736 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.736 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.736 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.736 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.736 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.736 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.736 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.736 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.736 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.736 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.736 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.736 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.737 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-03-21T11:49:46.737 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-03-21T11:49:46.737 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.737 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.737 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.737 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.737 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.737 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.737 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.737 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-03-21T11:49:46.738 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T11:49:46.738 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.738 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.738 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.738 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-03-21T11:49:46.738 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-03-21T11:49:46.738 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.738 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.738 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.738 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-03-21T11:49:46.738 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.738 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.739 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.739 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-03-21T11:49:46.739 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.739 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.739 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.739 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-03-21T11:49:46.739 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-03-21T11:49:46.739 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.739 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.739 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.739 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-03-21T11:49:46.739 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.740 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.740 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.740 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T11:49:46.740 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.740 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.740 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.740 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.740 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-03-21T11:49:46.740 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-03-21T11:49:46.740 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.740 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.740 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.740 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-03-21T11:49:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-03-21T11:49:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-03-21T11:49:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.741 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-03-21T11:49:46.742 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.742 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.742 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.742 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T11:49:46.742 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T11:49:46.742 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.742 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.742 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.742 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.742 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.742 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-03-21T11:49:46.742 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.743 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.743 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.743 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.743 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.743 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.743 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.743 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.743 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.743 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.743 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.743 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.743 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.743 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.744 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.744 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.744 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.744 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.744 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.744 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.744 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.744 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.744 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.744 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.744 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.744 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.744 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.745 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.745 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.745 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.745 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.745 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.745 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.745 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.745 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.745 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.745 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.745 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.745 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.745 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.746 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.746 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.746 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.746 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.746 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.746 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.746 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.746 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.746 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.746 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.746 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.746 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.746 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.747 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.747 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.747 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.747 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.747 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.747 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.747 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.747 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.747 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.747 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.747 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.747 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.748 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.748 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.748 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.748 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.748 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.748 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.748 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-03-21T11:49:46.748 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.748 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.748 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.748 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.749 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.749 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.749 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.749 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.749 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.749 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.749 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.749 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.749 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.749 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.749 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.749 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.749 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.750 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.750 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.750 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.750 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.750 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.750 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.750 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.750 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.750 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.750 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.750 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.751 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.751 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.751 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.751 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.751 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.751 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.751 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.751 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.751 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.751 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.751 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.751 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.751 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.752 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.752 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.752 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.752 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.752 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.752 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.752 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.752 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.752 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.752 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.752 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.752 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.752 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.752 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.753 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.753 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.753 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.753 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.753 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.753 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.753 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.753 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.753 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.753 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.753 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.753 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.754 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.754 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.754 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.754 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-03-21T11:49:46.754 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.754 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.754 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.754 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.754 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.754 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.755 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.755 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.755 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.755 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.755 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.755 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.755 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.755 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.755 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.755 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.755 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.755 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.755 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.756 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.756 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.756 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.756 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.756 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.756 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.756 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.756 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.756 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.756 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.756 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.756 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.756 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.757 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.757 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.757 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.757 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.757 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.757 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.757 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.757 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.757 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.757 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.757 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.757 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.757 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.758 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.758 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.758 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.758 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.758 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.758 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.758 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.758 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.758 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.758 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.758 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.758 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.758 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.759 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.759 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.759 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.759 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.759 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.759 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.759 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.759 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.759 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.759 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.759 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.759 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.760 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-03-21T11:49:46.760 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.760 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.760 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.760 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-03-21T11:49:46.760 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-03-21T11:49:46.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.760 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.760 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.760 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.760 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.760 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.760 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.760 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T11:49:46.761 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.761 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.761 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.761 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.761 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.761 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.761 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.761 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.761 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.761 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.761 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.761 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.762 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.762 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.762 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.762 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.762 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.762 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.762 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.762 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.762 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.762 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.762 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.762 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.762 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.763 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.763 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.763 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.763 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.763 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.763 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.763 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.763 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.763 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.763 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.763 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.763 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.764 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.765 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.765 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.765 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.765 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.765 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.765 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.765 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.765 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.765 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.765 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.766 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.766 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.766 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.766 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-03-21T11:49:46.766 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-03-21T11:49:46.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.766 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.766 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.766 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.766 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.766 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.766 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-03-21T11:49:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-03-21T11:49:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-03-21T11:49:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-03-21T11:49:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-03-21T11:49:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.768 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.768 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.768 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.768 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.768 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.768 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.768 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.768 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.768 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.768 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.768 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.768 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.769 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.771 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.771 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.771 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.771 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.771 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.771 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.771 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.771 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.771 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.771 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.771 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.771 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.771 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.772 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.772 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.772 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.772 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.772 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.772 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.772 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.772 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.772 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.772 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.772 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.773 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.773 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.773 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.773 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.773 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.773 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-03-21T11:49:46.773 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.773 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.773 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.773 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.774 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.774 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.774 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.774 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.774 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.774 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.774 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.774 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.774 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.774 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.774 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.774 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.774 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.775 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.775 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.775 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.775 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.775 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.775 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.775 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.775 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.775 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.775 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.775 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.776 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.777 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.777 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.777 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.777 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.777 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.777 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.777 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.777 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.777 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.777 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.777 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.777 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.777 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.778 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.778 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.778 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.778 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.778 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.778 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.778 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.778 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.778 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.778 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.778 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.778 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.778 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.779 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-03-21T11:49:46.779 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.779 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.779 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.779 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.779 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.779 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.779 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.779 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.779 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.779 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.780 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.781 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.781 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.781 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.781 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.781 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.781 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.781 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.781 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.781 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.781 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.781 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.781 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.782 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.782 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.782 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.782 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.782 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.782 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.782 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.782 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.782 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.782 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.782 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.782 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.783 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.783 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.783 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.783 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.783 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.783 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.783 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.783 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.783 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.783 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.783 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.783 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.783 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.784 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.784 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.784 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.784 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.784 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.784 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.784 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:46.784 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:46.784 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.784 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.784 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.784 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:46.784 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.785 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.785 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.785 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.785 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.785 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.785 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.785 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-03-21T11:49:46.785 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-03-21T11:49:46.785 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.785 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.785 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.785 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-03-21T11:49:46.786 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-03-21T11:49:46.786 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.786 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.786 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-03-21T11:49:46.786 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T11:49:46.786 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.786 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.786 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.786 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.786 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-03-21T11:49:46.786 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-03-21T11:49:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-03-21T11:49:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T11:49:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.787 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.788 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.788 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-03-21T11:49:46.788 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-03-21T11:49:46.788 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.788 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.788 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.788 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.788 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.788 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.788 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.788 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.788 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.789 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.789 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.789 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.789 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.789 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.789 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.789 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.789 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.789 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.789 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.789 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.789 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.790 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.790 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.790 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.790 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.790 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.790 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.790 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.790 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.790 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.790 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.790 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.790 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.790 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.790 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.791 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.791 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.791 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.791 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.791 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.791 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.791 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.791 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.791 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.791 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.791 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.791 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.793 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.793 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.793 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.793 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.793 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.793 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.793 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.793 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.793 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.793 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.793 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.793 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.794 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.794 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.794 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.794 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.794 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.794 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-03-21T11:49:46.794 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-03-21T11:49:46.794 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.794 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.794 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.794 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-03-21T11:49:46.794 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.795 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.795 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.795 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-03-21T11:49:46.795 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.795 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.795 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.795 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:46.795 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:46.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.795 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.795 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T11:49:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-03-21T11:49:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-03-21T11:49:46.796 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.797 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.797 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.797 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-03-21T11:49:46.797 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-03-21T11:49:46.797 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.797 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.797 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.797 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.797 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.797 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.797 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.797 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.797 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.798 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.798 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.798 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-03-21T11:49:46.798 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-03-21T11:49:46.798 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.798 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.798 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.798 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.798 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.798 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.798 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.799 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T11:49:46.799 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T11:49:46.799 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.799 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.799 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.799 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.799 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.799 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.799 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.799 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.799 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.799 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.800 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.800 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.800 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.800 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.800 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.800 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.800 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.800 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.800 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.800 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.800 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.800 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.800 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.801 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.801 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.801 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.801 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.801 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.801 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.801 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.801 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.801 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.801 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.801 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.801 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.801 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.802 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.802 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.802 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.802 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.802 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.802 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.802 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.802 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.802 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.802 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.802 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.802 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.802 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.802 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.803 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.804 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.804 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.804 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.804 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-03-21T11:49:46.804 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-03-21T11:49:46.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.804 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.804 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.804 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.804 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.804 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.804 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.805 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-03-21T11:49:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-03-21T11:49:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-03-21T11:49:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.806 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.807 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.808 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.808 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T11:49:46.808 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T11:49:46.808 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.808 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.808 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.808 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-03-21T11:49:46.808 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.808 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.808 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.808 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T11:49:46.808 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.809 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.810 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.810 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.810 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-03-21T11:49:46.810 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-03-21T11:49:46.810 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.810 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.810 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.810 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-03-21T11:49:46.810 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.810 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.810 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.810 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-03-21T11:49:46.810 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-03-21T11:49:46.810 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.811 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.812 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.812 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.812 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.812 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.812 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.812 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-03-21T11:49:46.812 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.812 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.812 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.812 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.812 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.813 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.813 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.813 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.813 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.813 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.813 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.813 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.813 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.813 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.813 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.813 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.813 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.813 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.813 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.814 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.814 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.814 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.814 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.814 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.814 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.814 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.814 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.814 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.814 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.814 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.814 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.814 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.815 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.815 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.815 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.815 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.815 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.815 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.815 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.815 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.815 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.815 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.815 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.816 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-03-21T11:49:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-03-21T11:49:46.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-03-21T11:49:46.818 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-03-21T11:49:46.819 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.821 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.822 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.822 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.822 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-03-21T11:49:46.822 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-03-21T11:49:46.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.822 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.822 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.822 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.822 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.822 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.822 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.823 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.823 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.823 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.823 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.823 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.823 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.823 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.823 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.823 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.823 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.823 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.823 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.824 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.824 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.824 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.824 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.824 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.824 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.824 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.824 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.824 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.824 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.824 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.824 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.824 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.825 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.826 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.826 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.826 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.826 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.826 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.826 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.826 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.826 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.826 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.826 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.826 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.826 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.826 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.828 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.829 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.830 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.830 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.830 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.830 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-03-21T11:49:46.830 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.830 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.830 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.830 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.830 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.830 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.830 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.831 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.831 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.831 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.831 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.831 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.831 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.831 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.831 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.831 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.831 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.831 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.831 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.831 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.831 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.832 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.833 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.833 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.833 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.833 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.833 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.833 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.833 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.833 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.833 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.833 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.833 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.833 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.834 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.835 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.835 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.835 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.835 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.835 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.835 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.835 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.835 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.835 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.835 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.835 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-03-21T11:49:46.835 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.836 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.836 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.836 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.836 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.836 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.836 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.836 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.836 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.836 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.836 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.836 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.836 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.837 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.837 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.837 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.837 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.837 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.837 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.837 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.837 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.837 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.837 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.837 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.837 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.838 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.838 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.838 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.838 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.838 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.838 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.838 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.838 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.838 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.838 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.838 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.838 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.839 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.839 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.839 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.839 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.839 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.839 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.839 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.839 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.839 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.839 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.839 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.839 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.839 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.839 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.840 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-03-21T11:49:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-03-21T11:49:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-03-21T11:49:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.841 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T11:49:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-03-21T11:49:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.842 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.843 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.843 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.843 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.843 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.843 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.843 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.843 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.843 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.843 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.843 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.843 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.843 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.843 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.843 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.844 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.844 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.844 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.844 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.844 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.844 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.844 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.844 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.844 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.844 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.844 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.844 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.844 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.845 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.845 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.845 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.845 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.845 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.845 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.845 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.845 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.845 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.845 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.845 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.845 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.845 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.846 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.846 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.846 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.846 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.846 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.846 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.846 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.846 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.846 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.846 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.846 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.846 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.846 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.847 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-03-21T11:49:46.848 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.849 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.850 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.850 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.850 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.850 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.850 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.850 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.850 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.850 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.850 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.850 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-03-21T11:49:46.850 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-03-21T11:49:46.850 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.851 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.852 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.853 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.854 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.854 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.854 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.854 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.854 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.854 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.854 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-03-21T11:49:46.854 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-03-21T11:49:46.854 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.854 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.854 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.854 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.855 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.855 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.855 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.855 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.855 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.855 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.855 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.855 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-03-21T11:49:46.855 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-03-21T11:49:46.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.855 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.855 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.856 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.856 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T11:49:46.856 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-03-21T11:49:46.856 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.856 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.856 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.856 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.856 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.856 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-03-21T11:49:46.856 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-03-21T11:49:46.856 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.856 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.856 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.857 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.857 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.857 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.857 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.857 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.857 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.857 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.857 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.857 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-03-21T11:49:46.857 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-03-21T11:49:46.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.857 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.858 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.858 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.858 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.858 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.858 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.858 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.858 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.858 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.858 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.858 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-03-21T11:49:46.858 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-03-21T11:49:46.858 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.858 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.859 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.859 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.859 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.859 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.859 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.859 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.859 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.859 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.859 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.859 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.859 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.860 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-03-21T11:49:46.860 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.860 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.860 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.860 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-03-21T11:49:46.860 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-03-21T11:49:46.860 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.860 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.860 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.860 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.860 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.860 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.860 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.860 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T11:49:46.861 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.861 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.861 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.861 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-03-21T11:49:46.861 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-03-21T11:49:46.861 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.861 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.861 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.861 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.861 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.861 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.861 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.862 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-03-21T11:49:46.862 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.862 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.862 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.862 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.862 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.862 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.862 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.862 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.862 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.862 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.862 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.863 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.863 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.863 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.863 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.863 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.863 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.863 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.863 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.863 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.863 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.863 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.863 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.864 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.864 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.864 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.864 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.864 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.864 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.864 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.864 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.864 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.864 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.864 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.864 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.864 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.865 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.865 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.865 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.865 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.865 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.865 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.865 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.865 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.865 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.865 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.865 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.865 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.866 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.867 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.867 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.867 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.867 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.867 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.867 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.867 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.867 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.867 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.867 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.867 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.867 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-03-21T11:49:46.868 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.868 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.868 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.868 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.868 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.868 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.868 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.868 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.868 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.868 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.868 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.868 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.868 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.869 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.870 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.871 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.872 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-03-21T11:49:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.873 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.874 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.876 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.877 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.878 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-03-21T11:49:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-03-21T11:49:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-03-21T11:49:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T11:49:46.879 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.880 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.881 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.881 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.881 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-03-21T11:49:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.882 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.883 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.884 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.885 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.886 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.887 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-03-21T11:49:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-03-21T11:49:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-03-21T11:49:46.888 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T11:49:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-03-21T11:49:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-03-21T11:49:46.889 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T11:49:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-03-21T11:49:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-03-21T11:49:46.890 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T11:49:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-03-21T11:49:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-03-21T11:49:46.891 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T11:49:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-03-21T11:49:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-03-21T11:49:46.892 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.893 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.894 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.895 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T11:49:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-03-21T11:49:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-03-21T11:49:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.896 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-03-21T11:49:46.897 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-03-21T11:49:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-03-21T11:49:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-03-21T11:49:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-03-21T11:49:46.898 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.899 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.900 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.901 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.902 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.903 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.904 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.905 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-03-21T11:49:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-03-21T11:49:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.906 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-03-21T11:49:46.907 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.908 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.909 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.910 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.911 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.912 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.913 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-03-21T11:49:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.914 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.915 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.916 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.917 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.919 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.920 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-03-21T11:49:46.921 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-03-21T11:49:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T11:49:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.922 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.923 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-03-21T11:49:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-03-21T11:49:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-03-21T11:49:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-03-21T11:49:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.924 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.925 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.926 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.927 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.928 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.929 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.930 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.931 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-03-21T11:49:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-03-21T11:49:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.932 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-03-21T11:49:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-03-21T11:49:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-03-21T11:49:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.933 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-03-21T11:49:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-03-21T11:49:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T11:49:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.934 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T11:49:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: "modules": [ 2024-03-21T11:49:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-03-21T11:49:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: "dashboard", 2024-03-21T11:49:46.935 INFO:teuthology.orchestra.run.smithi047.stdout: "iostat", 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "nfs", 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "restful" 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.936 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T11:49:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.937 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.938 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.939 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.940 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.941 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-03-21T11:49:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-03-21T11:49:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.942 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.943 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.944 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-03-21T11:49:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-03-21T11:49:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-03-21T11:49:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.945 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-03-21T11:49:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-03-21T11:49:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-03-21T11:49:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-03-21T11:49:46.946 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.947 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-03-21T11:49:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-03-21T11:49:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-03-21T11:49:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.948 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T11:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-03-21T11:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-03-21T11:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.949 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-03-21T11:49:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-03-21T11:49:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.950 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-03-21T11:49:46.951 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:46.952 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.953 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-03-21T11:49:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T11:49:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-03-21T11:49:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T11:49:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-03-21T11:49:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-03-21T11:49:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.954 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T11:49:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T11:49:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T11:49:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-03-21T11:49:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-03-21T11:49:46.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-03-21T11:49:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-03-21T11:49:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-03-21T11:49:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-03-21T11:49:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T11:49:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.956 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-03-21T11:49:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-03-21T11:49:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-03-21T11:49:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T11:49:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T11:49:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.957 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-03-21T11:49:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-03-21T11:49:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-03-21T11:49:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-03-21T11:49:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T11:49:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T11:49:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.958 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.959 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.960 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:46.961 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-03-21T11:49:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-03-21T11:49:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T11:49:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-03-21T11:49:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-03-21T11:49:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-03-21T11:49:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-03-21T11:49:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-03-21T11:49:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-03-21T11:49:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-03-21T11:49:46.965 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-03-21T11:49:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-03-21T11:49:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-03-21T11:49:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-03-21T11:49:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.966 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T11:49:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.967 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T11:49:46.968 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T11:49:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T11:49:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T11:49:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-03-21T11:49:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-03-21T11:49:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.969 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T11:49:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-03-21T11:49:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:46.970 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:46.971 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:46.971 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-03-21T11:49:46.971 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-03-21T11:49:46.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.971 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.971 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.971 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-03-21T11:49:46.971 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.971 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.971 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.971 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T11:49:46.971 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.971 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.971 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.972 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-03-21T11:49:46.972 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-03-21T11:49:46.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.972 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.972 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.972 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-03-21T11:49:46.972 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.972 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.972 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.972 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T11:49:46.972 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.972 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.972 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.974 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-03-21T11:49:46.974 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-03-21T11:49:46.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.974 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.974 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.974 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.974 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.974 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.974 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.974 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T11:49:46.974 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T11:49:46.974 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.974 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.975 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-03-21T11:49:46.975 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-03-21T11:49:46.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.975 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.975 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.975 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:46.975 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.975 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.975 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.975 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T11:49:46.975 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.975 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.975 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-03-21T11:49:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T11:49:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-03-21T11:49:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T11:49:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.976 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-03-21T11:49:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-03-21T11:49:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-03-21T11:49:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-03-21T11:49:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-03-21T11:49:46.977 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T11:49:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-03-21T11:49:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-03-21T11:49:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.978 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.979 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.980 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T11:49:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-03-21T11:49:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-03-21T11:49:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T11:49:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.981 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T11:49:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-03-21T11:49:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-03-21T11:49:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T11:49:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.982 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.983 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-03-21T11:49:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-03-21T11:49:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-03-21T11:49:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T11:49:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.984 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T11:49:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-03-21T11:49:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-03-21T11:49:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T11:49:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.985 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-03-21T11:49:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-03-21T11:49:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-03-21T11:49:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T11:49:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.986 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-03-21T11:49:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-03-21T11:49:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-03-21T11:49:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T11:49:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.987 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-03-21T11:49:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-03-21T11:49:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-03-21T11:49:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T11:49:46.988 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-03-21T11:49:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-03-21T11:49:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-03-21T11:49:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T11:49:46.989 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-03-21T11:49:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-03-21T11:49:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-03-21T11:49:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T11:49:46.990 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-03-21T11:49:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-03-21T11:49:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-03-21T11:49:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T11:49:46.991 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.992 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-03-21T11:49:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-03-21T11:49:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-03-21T11:49:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T11:49:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.993 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T11:49:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-03-21T11:49:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-03-21T11:49:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T11:49:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.994 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-03-21T11:49:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-03-21T11:49:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-03-21T11:49:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T11:49:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.995 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-03-21T11:49:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-03-21T11:49:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-03-21T11:49:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.996 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T11:49:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-03-21T11:49:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-03-21T11:49:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.997 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T11:49:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-03-21T11:49:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T11:49:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-03-21T11:49:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.998 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T11:49:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-03-21T11:49:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-03-21T11:49:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-03-21T11:49:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:46.999 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.000 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-03-21T11:49:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-03-21T11:49:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-03-21T11:49:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.001 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T11:49:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-03-21T11:49:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-03-21T11:49:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.002 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-03-21T11:49:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-03-21T11:49:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-03-21T11:49:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T11:49:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.003 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T11:49:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-03-21T11:49:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-03-21T11:49:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.004 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-03-21T11:49:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-03-21T11:49:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-03-21T11:49:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.005 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T11:49:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-03-21T11:49:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-03-21T11:49:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.006 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.007 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-03-21T11:49:47.008 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-03-21T11:49:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T11:49:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.009 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T11:49:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.010 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.011 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.012 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.013 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T11:49:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T11:49:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T11:49:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-03-21T11:49:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.014 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-03-21T11:49:47.015 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-03-21T11:49:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-03-21T11:49:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.016 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-03-21T11:49:47.017 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-03-21T11:49:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-03-21T11:49:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-03-21T11:49:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.018 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-03-21T11:49:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-03-21T11:49:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T11:49:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-03-21T11:49:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-03-21T11:49:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.019 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-03-21T11:49:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T11:49:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-03-21T11:49:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-03-21T11:49:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.020 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.021 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T11:49:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-03-21T11:49:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-03-21T11:49:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.022 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.023 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.024 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T11:49:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-03-21T11:49:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-03-21T11:49:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.025 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T11:49:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-03-21T11:49:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-03-21T11:49:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.026 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-03-21T11:49:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-03-21T11:49:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-03-21T11:49:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-03-21T11:49:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.027 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T11:49:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-03-21T11:49:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-03-21T11:49:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.028 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.029 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T11:49:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-03-21T11:49:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-03-21T11:49:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.030 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T11:49:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-03-21T11:49:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-03-21T11:49:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.031 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.032 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T11:49:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.033 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-03-21T11:49:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.034 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.035 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.036 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.037 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.038 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-03-21T11:49:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-03-21T11:49:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-03-21T11:49:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.039 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T11:49:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-03-21T11:49:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-03-21T11:49:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-03-21T11:49:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.040 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T11:49:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-03-21T11:49:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.041 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T11:49:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T11:49:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T11:49:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T11:49:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T11:49:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T11:49:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.044 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-03-21T11:49:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T11:49:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.045 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T11:49:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T11:49:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.046 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T11:49:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T11:49:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.047 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.048 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T11:49:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T11:49:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.049 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T11:49:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T11:49:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.050 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T11:49:47.052 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T11:49:47.053 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T11:49:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T11:49:47.054 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T11:49:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T11:49:47.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.056 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-03-21T11:49:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-03-21T11:49:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.057 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.058 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T11:49:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T11:49:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T11:49:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.059 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.060 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T11:49:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T11:49:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.061 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T11:49:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T11:49:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.062 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-03-21T11:49:47.063 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T11:49:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T11:49:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.064 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T11:49:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T11:49:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.065 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T11:49:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T11:49:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.066 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.067 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.068 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.069 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T11:49:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T11:49:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.070 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T11:49:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T11:49:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.071 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T11:49:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T11:49:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.072 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T11:49:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T11:49:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.073 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T11:49:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T11:49:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T11:49:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.074 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T11:49:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T11:49:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.075 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T11:49:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T11:49:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-03-21T11:49:47.076 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T11:49:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T11:49:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-03-21T11:49:47.077 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T11:49:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T11:49:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.078 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T11:49:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T11:49:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.079 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T11:49:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T11:49:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T11:49:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.080 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T11:49:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T11:49:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.081 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T11:49:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T11:49:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.082 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T11:49:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T11:49:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.083 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.084 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T11:49:47.085 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.086 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.086 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.086 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.086 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.086 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.086 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.086 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-03-21T11:49:47.086 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-03-21T11:49:47.086 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.086 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.086 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.086 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.086 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.087 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.087 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.087 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.087 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.087 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.087 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.087 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-03-21T11:49:47.087 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-03-21T11:49:47.087 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.087 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.087 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.087 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.087 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.089 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.089 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.089 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.089 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.089 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.089 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.090 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-03-21T11:49:47.090 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-03-21T11:49:47.090 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.090 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.090 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.090 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.090 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.090 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.090 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.090 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-03-21T11:49:47.090 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.090 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.090 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.091 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.093 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.094 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.094 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.094 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.094 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.094 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.094 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.094 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.094 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.094 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.094 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.094 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.094 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.095 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.096 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.097 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.098 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.098 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-03-21T11:49:47.098 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.098 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.098 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.098 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-03-21T11:49:47.098 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T11:49:47.098 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.098 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.098 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.098 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.098 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.098 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.099 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.100 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.101 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.102 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.103 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.104 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.105 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.106 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.106 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-03-21T11:49:47.106 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.106 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.106 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.106 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-03-21T11:49:47.106 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-03-21T11:49:47.106 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.106 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.106 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.106 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.107 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.110 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.110 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.110 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.110 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.110 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.110 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.110 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.110 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.110 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.110 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.110 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.110 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.110 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.110 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.111 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.111 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.111 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.111 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.111 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.111 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.111 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.111 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.111 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.111 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.111 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.111 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.112 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.114 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-03-21T11:49:47.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.116 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.117 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.118 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.119 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.120 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.121 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.121 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.121 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.121 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.121 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.121 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.121 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.121 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.121 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.122 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.123 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.123 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.123 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.123 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.123 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.123 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.123 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.123 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.123 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.123 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.123 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.123 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-03-21T11:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-03-21T11:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-03-21T11:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.124 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T11:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.126 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-03-21T11:49:47.127 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T11:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-03-21T11:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-03-21T11:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-03-21T11:49:47.128 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T11:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-03-21T11:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-03-21T11:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.129 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-03-21T11:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-03-21T11:49:47.130 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.131 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.132 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.133 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.134 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.135 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-03-21T11:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-03-21T11:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.136 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-03-21T11:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-03-21T11:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-03-21T11:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-03-21T11:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.137 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-03-21T11:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-03-21T11:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-03-21T11:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-03-21T11:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.138 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-03-21T11:49:47.139 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T11:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-03-21T11:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-03-21T11:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.140 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-03-21T11:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-03-21T11:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-03-21T11:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-03-21T11:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.141 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T11:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-03-21T11:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.142 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.143 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.144 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.145 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.146 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.147 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-03-21T11:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.148 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.149 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.150 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.151 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.152 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-03-21T11:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.153 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-03-21T11:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-03-21T11:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-03-21T11:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T11:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-03-21T11:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-03-21T11:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-03-21T11:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T11:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.156 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.157 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.158 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.159 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.159 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.159 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.159 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.159 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.159 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.159 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.159 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.159 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.159 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.159 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.159 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.159 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.159 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.160 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.160 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.160 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.160 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.160 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.160 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.160 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.160 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.160 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.160 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.160 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.161 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.162 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-03-21T11:49:47.162 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.162 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.162 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.162 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-03-21T11:49:47.162 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T11:49:47.162 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.162 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.162 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.162 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.162 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.163 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.163 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.163 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.163 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.163 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.163 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.163 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.163 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.163 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.163 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.163 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.163 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.163 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.163 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.164 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.165 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.165 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.165 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.165 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.165 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.165 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.165 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.165 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.165 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.165 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.165 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.165 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.166 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.166 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.166 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.166 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.166 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.166 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.166 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.166 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.166 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.166 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.166 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.167 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.167 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.167 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.167 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.167 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.167 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.167 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.167 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.167 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.167 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-03-21T11:49:47.167 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-03-21T11:49:47.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.167 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.168 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.168 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.168 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.168 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.168 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.168 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-03-21T11:49:47.168 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T11:49:47.168 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.168 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.168 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-03-21T11:49:47.168 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-03-21T11:49:47.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.168 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.169 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.169 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-03-21T11:49:47.169 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.169 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.169 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.169 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-03-21T11:49:47.169 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.169 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.169 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.169 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-03-21T11:49:47.169 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-03-21T11:49:47.169 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.169 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.170 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.170 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-03-21T11:49:47.170 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.170 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.170 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.170 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T11:49:47.170 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.170 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.170 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.170 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-03-21T11:49:47.170 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-03-21T11:49:47.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.170 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.171 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.171 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.171 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.171 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.171 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.171 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-03-21T11:49:47.171 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.171 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.171 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.171 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-03-21T11:49:47.171 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-03-21T11:49:47.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.171 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.172 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.172 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-03-21T11:49:47.172 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.172 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.172 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.172 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T11:49:47.172 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T11:49:47.172 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.172 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.173 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-03-21T11:49:47.173 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.173 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.173 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.173 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.173 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.173 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.173 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.173 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.173 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.173 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.173 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.173 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.174 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.174 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.174 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.174 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.174 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.174 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.174 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.174 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.174 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.174 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.174 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.174 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.176 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.176 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.176 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.176 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.176 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.176 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.176 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.176 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.176 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.176 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.176 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.176 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.176 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.177 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.177 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.177 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.177 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.177 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.177 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.177 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.177 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.177 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.177 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.177 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.178 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.178 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.178 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.178 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.178 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.178 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.178 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.178 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-03-21T11:49:47.178 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.178 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.178 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.178 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.179 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.180 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.180 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.180 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.180 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.180 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.180 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.180 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.180 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.180 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.180 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.180 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.180 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.181 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.182 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.182 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.182 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.182 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.182 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.182 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.182 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.182 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.182 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.182 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.182 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.182 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.182 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.183 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.183 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.183 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.183 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.183 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.183 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.183 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.183 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.183 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.183 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.183 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.183 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.184 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.185 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.185 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.185 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.185 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.185 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.185 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.185 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.185 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.185 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.185 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.185 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.185 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.186 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.186 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.186 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.186 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.186 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.186 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.186 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.186 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.186 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.186 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.186 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.187 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.187 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.187 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.187 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.187 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.187 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.187 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.187 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.187 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.187 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.187 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.187 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.187 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.188 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.188 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.188 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.188 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.188 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.188 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.188 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.188 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.188 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.188 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.188 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.189 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.189 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.189 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.189 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.189 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.189 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.189 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.189 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-03-21T11:49:47.189 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.189 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.190 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.190 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-03-21T11:49:47.190 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-03-21T11:49:47.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.190 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.190 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.190 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.190 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.190 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.190 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.190 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T11:49:47.190 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.190 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.191 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.191 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.191 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.191 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.191 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.191 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.191 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.191 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.191 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.191 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.191 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.191 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.192 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.192 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.192 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.192 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.192 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.192 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.192 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.192 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.192 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.192 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.192 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.193 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.193 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.193 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.193 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.193 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.193 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.193 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.193 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.193 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.193 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.193 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.194 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.194 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.194 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.194 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.194 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.194 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.194 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.194 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.194 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.194 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.194 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.194 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.194 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.195 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.195 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.195 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.195 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.195 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.195 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.195 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.195 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.195 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.195 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.195 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.195 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.196 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.196 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.196 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.196 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.196 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.196 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.196 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.196 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-03-21T11:49:47.196 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-03-21T11:49:47.196 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.196 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.196 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.196 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.197 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.197 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.197 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.197 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-03-21T11:49:47.197 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-03-21T11:49:47.197 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-03-21T11:49:47.197 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.197 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-03-21T11:49:47.197 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.197 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.197 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.197 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.197 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.198 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-03-21T11:49:47.198 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.198 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.198 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.198 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.198 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.198 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.198 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.198 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.198 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.198 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.198 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.199 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.199 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.199 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.199 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.199 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.199 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.199 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.199 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.199 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.199 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.199 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.199 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.200 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.200 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.200 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.200 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.200 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.200 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.200 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.200 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.200 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.200 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.200 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.200 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.200 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.201 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.201 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.201 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.201 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.201 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.201 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.201 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.201 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.201 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.201 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.201 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.201 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.201 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.202 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.202 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.202 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.202 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.202 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.202 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.202 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.202 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.202 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.202 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.202 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.203 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.203 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.203 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.203 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.203 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.203 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.203 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.203 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.203 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.203 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.203 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-03-21T11:49:47.203 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.204 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.204 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.204 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.204 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.204 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.204 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.204 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.204 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.204 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.204 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.204 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.204 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.204 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.204 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.205 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.205 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.205 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.205 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.205 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.205 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.205 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.205 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.205 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.205 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.205 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.206 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.206 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.206 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.206 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.206 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.206 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.206 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.206 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.206 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.206 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.206 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.206 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.207 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.207 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.207 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.207 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.207 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.207 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.207 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.207 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.207 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.207 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.207 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.207 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.207 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.207 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.208 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.209 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.209 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.209 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.209 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.209 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-03-21T11:49:47.209 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.209 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.209 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.209 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.209 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.209 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.210 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.210 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.210 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.210 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.210 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.210 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.210 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.210 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.210 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.210 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.210 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.210 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.210 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.210 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.211 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.211 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.211 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.211 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.211 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.211 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.211 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.211 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.211 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.211 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.211 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.211 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.211 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.212 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.212 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.212 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.212 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.212 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.212 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.212 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.212 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.212 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.212 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.212 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.212 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.213 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.213 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.213 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.213 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.213 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.213 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.213 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.213 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.213 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.213 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.213 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.213 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.213 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.214 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.214 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.214 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.214 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.214 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.214 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.214 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.214 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.214 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.214 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:47.214 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:47.214 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.214 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.215 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.216 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-03-21T11:49:47.216 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-03-21T11:49:47.216 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.216 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.216 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-03-21T11:49:47.216 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T11:49:47.216 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.216 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.216 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.216 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.216 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-03-21T11:49:47.216 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.217 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.217 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.217 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-03-21T11:49:47.217 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-03-21T11:49:47.217 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.217 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.217 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.217 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.217 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.217 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.217 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.217 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T11:49:47.217 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.217 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.218 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.218 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-03-21T11:49:47.218 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-03-21T11:49:47.218 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.218 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.218 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.218 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.218 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.218 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.218 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.218 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.218 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.219 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.220 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.220 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.220 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.220 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.220 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.220 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.220 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.220 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.220 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.220 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.220 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.220 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.221 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.221 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.221 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.221 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.221 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.221 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.221 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.221 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.221 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.221 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.221 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.221 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.222 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.222 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.222 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.222 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.222 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.222 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.222 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.222 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.222 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.222 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.222 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.222 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.222 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.223 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.224 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.224 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-03-21T11:49:47.224 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-03-21T11:49:47.224 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.224 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.224 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.224 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-03-21T11:49:47.224 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.224 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.224 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.224 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-03-21T11:49:47.224 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.225 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.225 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.225 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T11:49:47.225 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T11:49:47.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.225 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.225 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.225 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T11:49:47.225 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.225 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.225 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.225 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-03-21T11:49:47.226 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.226 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.226 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.226 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-03-21T11:49:47.226 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.226 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.226 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.226 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-03-21T11:49:47.226 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-03-21T11:49:47.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.227 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.228 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.229 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.229 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.229 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.229 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.229 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.229 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.229 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.229 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.229 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.229 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.229 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.229 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.229 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.229 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.230 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.231 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.231 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.231 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.231 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.231 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.231 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.231 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.231 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.231 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.231 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.231 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.231 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.231 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.232 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.232 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.232 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.232 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.232 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.232 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.232 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.232 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.232 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.232 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.232 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.232 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.232 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.233 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.233 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.233 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.233 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.233 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.233 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.233 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.233 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-03-21T11:49:47.233 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-03-21T11:49:47.233 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.233 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.233 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.234 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.234 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.234 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.234 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.234 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.234 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.234 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.234 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.234 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T11:49:47.234 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T11:49:47.234 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.234 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.235 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.236 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-03-21T11:49:47.236 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.236 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.236 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.236 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.236 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.236 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.236 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.236 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T11:49:47.236 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T11:49:47.236 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.236 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.236 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.237 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-03-21T11:49:47.237 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.237 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.237 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.237 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T11:49:47.237 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.237 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.237 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.237 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T11:49:47.237 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T11:49:47.237 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.237 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.237 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.238 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-03-21T11:49:47.238 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.238 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.238 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.238 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T11:49:47.238 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.238 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.238 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.238 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-03-21T11:49:47.238 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-03-21T11:49:47.238 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.238 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.239 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-03-21T11:49:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-03-21T11:49:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-03-21T11:49:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-03-21T11:49:47.240 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-03-21T11:49:47.241 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.241 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.241 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.241 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-03-21T11:49:47.241 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-03-21T11:49:47.241 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-03-21T11:49:47.241 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.241 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.241 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.241 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.241 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.241 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.242 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.242 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-03-21T11:49:47.242 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.242 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.242 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.242 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.242 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.242 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.242 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.242 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.242 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.242 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.243 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.243 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.243 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.243 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.243 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.243 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.243 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.243 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.243 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.243 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.243 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.243 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.243 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.244 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.244 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.244 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.244 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.244 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.244 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.244 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.244 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.244 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.244 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.244 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.244 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.245 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.245 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.245 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.245 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.245 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.245 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.245 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.245 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.245 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.245 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.245 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.245 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.246 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.247 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.247 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.247 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.247 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.247 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.247 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.247 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.247 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.247 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.247 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.247 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-03-21T11:49:47.247 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-03-21T11:49:47.247 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.248 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.248 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.248 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T11:49:47.248 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.248 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.248 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.248 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.248 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.248 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.248 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.248 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-03-21T11:49:47.248 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T11:49:47.248 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.249 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.249 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.249 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.249 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.249 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.249 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.249 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.249 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.249 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.249 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.249 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-03-21T11:49:47.249 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-03-21T11:49:47.250 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.250 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.250 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.250 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.250 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.250 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.250 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.250 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.250 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.250 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.250 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.250 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.250 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.251 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.252 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.253 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.254 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.255 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.255 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.255 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.255 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.255 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.255 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.255 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.255 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.255 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.255 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.255 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.255 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.255 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.256 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.256 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.256 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.256 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.256 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.256 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.256 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.256 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.256 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.256 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.256 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.256 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.256 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.256 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T11:49:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T11:49:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.259 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.260 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.261 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.262 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.262 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.262 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.262 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.262 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.262 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.262 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.262 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.262 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.262 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.262 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.262 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.263 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.263 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.263 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.263 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.263 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.263 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.263 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.263 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.263 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.263 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.263 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.263 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.263 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.263 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.264 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.264 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.264 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.264 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.264 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.264 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.264 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.264 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.264 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.264 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.264 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.264 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.265 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.265 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.265 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.265 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.265 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.265 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.265 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-03-21T11:49:47.265 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.265 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.265 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.265 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.266 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.266 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.266 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.266 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.266 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.266 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.266 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.266 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.266 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.266 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.266 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.266 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.266 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.266 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.267 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.267 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.267 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.267 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.267 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.267 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.267 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.267 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.267 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.267 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.267 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.267 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.267 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.268 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.269 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.270 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.271 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.272 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.273 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.274 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.274 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.274 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.274 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.274 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.274 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.274 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.274 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.274 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.274 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.274 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.274 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.275 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.275 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.275 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.275 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.275 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.275 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.275 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.275 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.275 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.275 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.275 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.275 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.276 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-03-21T11:49:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-03-21T11:49:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-03-21T11:49:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-03-21T11:49:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-03-21T11:49:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-03-21T11:49:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-03-21T11:49:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.279 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-03-21T11:49:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-03-21T11:49:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.280 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.281 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-03-21T11:49:47.281 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-03-21T11:49:47.281 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T11:49:47.281 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.281 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.281 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.281 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.281 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.281 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.281 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.281 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.282 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.282 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.282 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-03-21T11:49:47.282 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-03-21T11:49:47.282 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.282 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.282 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.282 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.282 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.282 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.282 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.282 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.282 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.283 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.283 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.283 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-03-21T11:49:47.283 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-03-21T11:49:47.283 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.283 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.283 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.283 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.283 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.283 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.283 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.284 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.285 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.286 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.286 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.286 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-03-21T11:49:47.286 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-03-21T11:49:47.286 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.286 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.286 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.286 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.286 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.286 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.286 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.286 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.286 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.287 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.287 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.287 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-03-21T11:49:47.287 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-03-21T11:49:47.287 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.287 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.287 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.287 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.287 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.287 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.287 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.287 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.288 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.288 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.288 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.288 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-03-21T11:49:47.288 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-03-21T11:49:47.288 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.288 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.288 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.288 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.288 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.288 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.288 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.289 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.289 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.289 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.289 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.289 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.289 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.289 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-03-21T11:49:47.289 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.289 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.289 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.289 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-03-21T11:49:47.290 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-03-21T11:49:47.290 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.290 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.290 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.290 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.290 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.290 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.290 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.290 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T11:49:47.290 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.290 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.290 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.290 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-03-21T11:49:47.291 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-03-21T11:49:47.291 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.291 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.291 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.291 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.291 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.291 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.291 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.291 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-03-21T11:49:47.291 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.291 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.291 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.292 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.292 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.292 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.292 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.292 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.292 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.292 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.292 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.292 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.292 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.292 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.292 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.292 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.292 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.293 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.293 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.293 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.293 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.293 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.293 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.294 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.294 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.294 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.294 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.294 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.294 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.294 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.294 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.294 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.294 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.294 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.294 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.294 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.295 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.296 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.297 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.297 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.297 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.297 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.297 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.297 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-03-21T11:49:47.297 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.297 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.297 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.297 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.297 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.298 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.298 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.298 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.298 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.298 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.298 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.298 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.298 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.298 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.298 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.298 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.298 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.298 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.299 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.300 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.301 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-03-21T11:49:47.302 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.303 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.304 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.305 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.305 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.305 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.305 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.305 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.305 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.305 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.305 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.305 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.305 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.305 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.305 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.305 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.306 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.306 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.306 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.306 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.306 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.306 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.306 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.306 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.306 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.306 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.306 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.307 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.308 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.309 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.310 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.311 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.312 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.313 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.314 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.314 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.314 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.314 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.314 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.314 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.314 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.314 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.314 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.314 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.314 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.314 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.314 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.315 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T11:49:47.316 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.316 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.316 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.316 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-03-21T11:49:47.316 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-03-21T11:49:47.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.316 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.316 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.316 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.316 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.316 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.316 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.317 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T11:49:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-03-21T11:49:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-03-21T11:49:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.318 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T11:49:47.319 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.320 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.321 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-03-21T11:49:47.322 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.323 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.324 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.324 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-03-21T11:49:47.324 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-03-21T11:49:47.324 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.324 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.324 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.324 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.324 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.324 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.324 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-03-21T11:49:47.324 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-03-21T11:49:47.324 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.324 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.325 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.325 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-03-21T11:49:47.325 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.325 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.325 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.325 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.325 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.325 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.325 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.325 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-03-21T11:49:47.325 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-03-21T11:49:47.325 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.326 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.327 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.329 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.330 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.331 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.332 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.332 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.332 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.332 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.332 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-03-21T11:49:47.332 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-03-21T11:49:47.332 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.332 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.332 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.332 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.332 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.332 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.332 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.333 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.334 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.334 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.334 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-03-21T11:49:47.334 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-03-21T11:49:47.334 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.334 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.334 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.334 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T11:49:47.334 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.334 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.334 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.334 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.334 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.335 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.336 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.336 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.336 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.336 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.336 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.336 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.336 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.336 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.336 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.336 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.336 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.336 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.336 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.336 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.337 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.338 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.338 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.338 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.338 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.338 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.338 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.338 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.338 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.338 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.338 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.338 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.338 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.338 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.339 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.339 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.339 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.339 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.339 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.339 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.339 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.339 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.339 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.339 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.339 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.339 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.340 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.340 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.340 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.340 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.340 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.340 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.340 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.340 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.340 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.340 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.340 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-03-21T11:49:47.341 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.341 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.341 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.341 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.341 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.341 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.341 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.341 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.341 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.341 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.341 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.341 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.341 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.342 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.342 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.342 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.342 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.342 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.342 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.342 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.342 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.342 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.342 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.342 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.342 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.343 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.343 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.343 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.343 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.343 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.343 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.343 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.343 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.343 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.343 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.343 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.343 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.344 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.344 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.344 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.344 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.344 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.344 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.344 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.344 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.344 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.344 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.344 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.344 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.344 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.345 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.345 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.345 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.345 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.345 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.345 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.345 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.345 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.345 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.345 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.345 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.345 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.345 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.346 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.346 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.346 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.346 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.346 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.346 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.346 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-03-21T11:49:47.346 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-03-21T11:49:47.346 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.346 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.346 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.346 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-03-21T11:49:47.346 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.347 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.347 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.347 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T11:49:47.347 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.347 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.347 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.347 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.347 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-03-21T11:49:47.347 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-03-21T11:49:47.347 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.347 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.347 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.347 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.347 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.348 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.348 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.348 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-03-21T11:49:47.348 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.348 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.348 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.348 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-03-21T11:49:47.348 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-03-21T11:49:47.348 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.348 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.348 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.349 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T11:49:47.349 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.349 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.349 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.349 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T11:49:47.349 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.349 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.349 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.349 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-03-21T11:49:47.349 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T11:49:47.349 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.349 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.349 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.350 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T11:49:47.350 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.350 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.350 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.350 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T11:49:47.350 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.350 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.350 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.350 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.350 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.350 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.351 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-03-21T11:49:47.351 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T11:49:47.351 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T11:49:47.351 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T11:49:47.351 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-03-21T11:49:47.351 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-03-21T11:49:47.351 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T11:49:47.351 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.351 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.351 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-03-21T11:49:47.351 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.351 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.351 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.351 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-03-21T11:49:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-03-21T11:49:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.352 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.353 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.354 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.354 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.354 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T11:49:47.354 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T11:49:47.354 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.354 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.354 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.354 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.354 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.354 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.354 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.354 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.354 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.355 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.355 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.355 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.355 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.355 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.355 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.355 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.355 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.355 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.355 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T11:49:47.356 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T11:49:47.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.356 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.356 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.356 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.356 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.356 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.356 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.356 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.357 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.357 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.357 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.357 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T11:49:47.357 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T11:49:47.357 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.357 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.357 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.357 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T11:49:47.357 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.358 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.358 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T11:49:47.358 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T11:49:47.358 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T11:49:47.358 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T11:49:47.358 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T11:49:47.358 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T11:49:47.358 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T11:49:47.358 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.358 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.358 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.358 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.358 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.359 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T11:49:47.359 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T11:49:47.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T11:49:47.359 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.359 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T11:49:47.359 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T11:49:47.359 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.359 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.359 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.359 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.359 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.359 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.359 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.360 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-03-21T11:49:47.361 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-03-21T11:49:47.362 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T11:49:47.362 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T11:49:47.362 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T11:49:47.362 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T11:49:47.362 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T11:49:47.362 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T11:49:47.362 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T11:49:47.362 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T11:49:47.362 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T11:49:47.362 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T11:49:47.362 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T11:49:47.362 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.362 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.362 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: "services": { 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: "dashboard": "https://172.21.15.47:8443/" 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: "always_on_modules": { 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus": [ 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-03-21T11:49:47.363 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific": [ 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy": [ 2024-03-21T11:49:47.364 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-03-21T11:49:47.365 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-03-21T11:49:47.365 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-03-21T11:49:47.365 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-03-21T11:49:47.365 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-03-21T11:49:47.365 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-03-21T11:49:47.365 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-03-21T11:49:47.365 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-03-21T11:49:47.365 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-03-21T11:49:47.365 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-03-21T11:49:47.365 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.365 INFO:teuthology.orchestra.run.smithi047.stdout: "reef": [ 2024-03-21T11:49:47.365 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-03-21T11:49:47.365 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-03-21T11:49:47.365 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-03-21T11:49:47.366 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-03-21T11:49:47.366 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-03-21T11:49:47.366 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-03-21T11:49:47.366 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-03-21T11:49:47.366 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-03-21T11:49:47.366 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-03-21T11:49:47.366 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-03-21T11:49:47.366 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T11:49:47.366 INFO:teuthology.orchestra.run.smithi047.stdout: "squid": [ 2024-03-21T11:49:47.366 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-03-21T11:49:47.366 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-03-21T11:49:47.366 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-03-21T11:49:47.366 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-03-21T11:49:47.367 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-03-21T11:49:47.367 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-03-21T11:49:47.367 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-03-21T11:49:47.367 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-03-21T11:49:47.367 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-03-21T11:49:47.367 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-03-21T11:49:47.367 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T11:49:47.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.367 INFO:teuthology.orchestra.run.smithi047.stdout: "last_failure_osd_epoch": 3, 2024-03-21T11:49:47.367 INFO:teuthology.orchestra.run.smithi047.stdout: "active_clients": [ 2024-03-21T11:49:47.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.367 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-03-21T11:49:47.367 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 4154356547 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "libcephsqlite", 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1443923713 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.368 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T11:49:47.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.369 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.369 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-03-21T11:49:47.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-03-21T11:49:47.369 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.369 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-03-21T11:49:47.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-03-21T11:49:47.369 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2690553164 2024-03-21T11:49:47.369 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.369 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T11:49:47.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:47.369 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.369 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-03-21T11:49:47.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-03-21T11:49:47.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T11:49:47.370 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-03-21T11:49:47.370 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-03-21T11:49:47.370 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 595972321 2024-03-21T11:49:47.370 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.370 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T11:49:47.370 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:47.370 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T11:49:47.370 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:49:47.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:45 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3100592907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:49:47.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:45 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/600554565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:49:47.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:46 smithi100 ceph-mon[36245]: pgmap v1194: 129 pgs: 129 active+clean; 548 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 2.8 MiB/s wr, 147 op/s 2024-03-21T11:49:47.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:46 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/652081587' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T11:49:47.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:46 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:47.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:46 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:47.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:45 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3100592907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:49:47.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:45 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/600554565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:49:47.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:46 smithi102 ceph-mon[36748]: pgmap v1194: 129 pgs: 129 active+clean; 548 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 2.8 MiB/s wr, 147 op/s 2024-03-21T11:49:47.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:46 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/652081587' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T11:49:47.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:46 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:47.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:46 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:46.052+0000 7f88b2e00640 1 -- 172.21.15.47:0/652081587 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f88900a2fa0 msgr2=0x7f88900a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:46.052+0000 7f88b2e00640 1 --2- 172.21.15.47:0/652081587 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f88900a2fa0 0x7f88900a5460 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f88ac198e30 tx=0x7f8898005f90 comp rx=0 tx=0).stop 2024-03-21T11:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:46.052+0000 7f88b2e00640 1 -- 172.21.15.47:0/652081587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88ac1016c0 msgr2=0x7f88ac197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:46.052+0000 7f88b2e00640 1 --2- 172.21.15.47:0/652081587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88ac1016c0 0x7f88ac197920 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f889c00e9c0 tx=0x7f889c00ee80 comp rx=0 tx=0).stop 2024-03-21T11:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:46.052+0000 7f88b2e00640 1 -- 172.21.15.47:0/652081587 shutdown_connections 2024-03-21T11:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:46.052+0000 7f88b2e00640 1 --2- 172.21.15.47:0/652081587 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f88900a2fa0 0x7f88900a5460 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:46.052+0000 7f88b2e00640 1 --2- 172.21.15.47:0/652081587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac103270 0x7f88ac19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:46.052+0000 7f88b2e00640 1 --2- 172.21.15.47:0/652081587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac1028b0 0x7f88ac197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:46.052+0000 7f88b2e00640 1 --2- 172.21.15.47:0/652081587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88ac1016c0 0x7f88ac197920 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:49:47.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:46.052+0000 7f88b2e00640 1 -- 172.21.15.47:0/652081587 >> 172.21.15.47:0/652081587 conn(0x7f88ac0fcc50 msgr2=0x7f88ac105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:49:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:46.052+0000 7f88b2e00640 1 -- 172.21.15.47:0/652081587 shutdown_connections 2024-03-21T11:49:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:49:46.052+0000 7f88b2e00640 1 -- 172.21.15.47:0/652081587 wait complete. 2024-03-21T11:49:47.380 DEBUG:tasks.check_counter:Getting stats from a 2024-03-21T11:49:47.380 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-21T11:49:47.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:45 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3100592907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:49:47.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:45 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/600554565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:49:47.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:46 smithi047 ceph-mon[31351]: pgmap v1194: 129 pgs: 129 active+clean; 548 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 2.8 MiB/s wr, 147 op/s 2024-03-21T11:49:47.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:46 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/652081587' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T11:49:47.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:46 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:47.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:46 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:47.849 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:49:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:49:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:49:48.741 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:48 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:48.742 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:48 smithi102 ceph-mon[36748]: pgmap v1195: 129 pgs: 129 active+clean; 546 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 3.3 MiB/s wr, 176 op/s 2024-03-21T11:49:48.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:48.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:48.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:49:48.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:49:48.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:48 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:48.816 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:48 smithi047 ceph-mon[31351]: pgmap v1195: 129 pgs: 129 active+clean; 546 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 3.3 MiB/s wr, 176 op/s 2024-03-21T11:49:48.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:48.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:48.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T11:49:48.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T11:49:48.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:48 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' 2024-03-21T11:49:48.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:48 smithi100 ceph-mon[36245]: pgmap v1195: 129 pgs: 129 active+clean; 546 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 3.3 MiB/s wr, 176 op/s 2024-03-21T11:49:50.583 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:49:50.583 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T11:49:50.583 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2211, 2024-03-21T11:49:50.583 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2213, 2024-03-21T11:49:50.583 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 38445889, 2024-03-21T11:49:50.583 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 703099, 2024-03-21T11:49:50.583 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 9, 2024-03-21T11:49:50.583 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 4, 2024-03-21T11:49:50.583 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.551950384, 2024-03-21T11:49:50.583 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.201090414, 2024-03-21T11:49:50.583 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.181587907, 2024-03-21T11:49:50.583 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.124044510, 2024-03-21T11:49:50.584 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:49:50.584 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 2211, 2024-03-21T11:49:50.584 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.278248659, 2024-03-21T11:49:50.584 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000125847 2024-03-21T11:49:50.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.584 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:49:50.584 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.584 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.584 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.584 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T11:49:50.585 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T11:49:50.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.585 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T11:49:50.585 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2976, 2024-03-21T11:49:50.585 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2955, 2024-03-21T11:49:50.585 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 51910684, 2024-03-21T11:49:50.585 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 1071013, 2024-03-21T11:49:50.585 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 6, 2024-03-21T11:49:50.585 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 5, 2024-03-21T11:49:50.585 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.692734342, 2024-03-21T11:49:50.585 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.206541182, 2024-03-21T11:49:50.585 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.291124398, 2024-03-21T11:49:50.586 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.157731922, 2024-03-21T11:49:50.586 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:49:50.586 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 2954, 2024-03-21T11:49:50.586 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.314918902, 2024-03-21T11:49:50.586 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000106607 2024-03-21T11:49:50.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.586 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:49:50.586 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.586 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.586 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.586 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 946320, 2024-03-21T11:49:50.586 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 281680 2024-03-21T11:49:50.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.587 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T11:49:50.587 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 882, 2024-03-21T11:49:50.587 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 1233, 2024-03-21T11:49:50.587 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 56308674, 2024-03-21T11:49:50.587 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 1064003, 2024-03-21T11:49:50.587 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 8, 2024-03-21T11:49:50.587 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 5, 2024-03-21T11:49:50.587 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.345325547, 2024-03-21T11:49:50.587 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.096048292, 2024-03-21T11:49:50.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.182586545, 2024-03-21T11:49:50.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.046420154, 2024-03-21T11:49:50.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:49:50.588 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 1232, 2024-03-21T11:49:50.588 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.156162753, 2024-03-21T11:49:50.588 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000126755 2024-03-21T11:49:50.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:49:50.588 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.588 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.588 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-21T11:49:50.588 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 785424 2024-03-21T11:49:50.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.589 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-03-21T11:49:50.589 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 1, 2024-03-21T11:49:50.589 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-03-21T11:49:50.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.589 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-MDSRank": { 2024-03-21T11:49:50.589 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-03-21T11:49:50.589 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-03-21T11:49:50.589 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 9454, 2024-03-21T11:49:50.589 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.449019741, 2024-03-21T11:49:50.589 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000047495 2024-03-21T11:49:50.590 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:50.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.590 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-PurgeQueue": { 2024-03-21T11:49:50.590 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-03-21T11:49:50.590 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-03-21T11:49:50.590 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.590 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.590 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.590 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:50.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.591 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": { 2024-03-21T11:49:50.591 INFO:teuthology.orchestra.run.smithi047.stdout: "request": 0, 2024-03-21T11:49:50.591 INFO:teuthology.orchestra.run.smithi047.stdout: "reply": 0, 2024-03-21T11:49:50.591 INFO:teuthology.orchestra.run.smithi047.stdout: "reply_latency": { 2024-03-21T11:49:50.591 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.591 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.591 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.592 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_reply": 0, 2024-03-21T11:49:50.592 INFO:teuthology.orchestra.run.smithi047.stdout: "forward": 0, 2024-03-21T11:49:50.592 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_complete": 0, 2024-03-21T11:49:50.592 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_keys": 0, 2024-03-21T11:49:50.592 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_commit": 0, 2024-03-21T11:49:50.592 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_split": 0, 2024-03-21T11:49:50.592 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_merge": 0, 2024-03-21T11:49:50.592 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes": 14618, 2024-03-21T11:49:50.592 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_top": 0, 2024-03-21T11:49:50.592 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_bottom": 14618, 2024-03-21T11:49:50.592 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pin_tail": 0, 2024-03-21T11:49:50.592 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pinned": 14618, 2024-03-21T11:49:50.592 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_expired": 0, 2024-03-21T11:49:50.593 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_with_caps": 0, 2024-03-21T11:49:50.593 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": 0, 2024-03-21T11:49:50.593 INFO:teuthology.orchestra.run.smithi047.stdout: "subtrees": 1, 2024-03-21T11:49:50.593 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse": 0, 2024-03-21T11:49:50.593 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_hit": 0, 2024-03-21T11:49:50.593 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_forward": 0, 2024-03-21T11:49:50.593 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_discover": 0, 2024-03-21T11:49:50.593 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_dir_fetch": 0, 2024-03-21T11:49:50.593 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_remote_ino": 0, 2024-03-21T11:49:50.593 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_lock": 0, 2024-03-21T11:49:50.593 INFO:teuthology.orchestra.run.smithi047.stdout: "load_cent": 0, 2024-03-21T11:49:50.593 INFO:teuthology.orchestra.run.smithi047.stdout: "q": 0, 2024-03-21T11:49:50.593 INFO:teuthology.orchestra.run.smithi047.stdout: "exported": 0, 2024-03-21T11:49:50.594 INFO:teuthology.orchestra.run.smithi047.stdout: "exported_inodes": 0, 2024-03-21T11:49:50.594 INFO:teuthology.orchestra.run.smithi047.stdout: "imported": 0, 2024-03-21T11:49:50.594 INFO:teuthology.orchestra.run.smithi047.stdout: "imported_inodes": 0, 2024-03-21T11:49:50.594 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_dir_fetch": 0, 2024-03-21T11:49:50.594 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_backtrace_fetch": 0, 2024-03-21T11:49:50.594 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_peer_discover": 0, 2024-03-21T11:49:50.594 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rfiles": 0, 2024-03-21T11:49:50.594 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rbytes": 0, 2024-03-21T11:49:50.594 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rsnaps": 0, 2024-03-21T11:49:50.594 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T11:49:50.594 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_set_tag": 0, 2024-03-21T11:49:50.594 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T11:49:50.594 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_inotable_repaired": 0, 2024-03-21T11:49:50.594 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_inodes": 0, 2024-03-21T11:49:50.595 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T11:49:50.595 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T11:49:50.595 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_file_inodes": 0, 2024-03-21T11:49:50.595 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_inode_file_caps": 0, 2024-03-21T11:49:50.595 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T11:49:50.595 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_grant": 0, 2024-03-21T11:49:50.595 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T11:49:50.595 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T11:49:50.595 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T11:49:50.595 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps": 0, 2024-03-21T11:49:50.595 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps_dirty": 0, 2024-03-21T11:49:50.595 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_cap_release": 0, 2024-03-21T11:49:50.595 INFO:teuthology.orchestra.run.smithi047.stdout: "process_request_cap_release": 0 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache": { 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update": 0, 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update_receipt": 0, 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_try_discover": 0, 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_send_discover": 0, 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_handle_discover": 0, 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays": 0, 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_delayed": 0, 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_enqueuing": 0, 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_created": 0, 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_enqueued": 0, 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_reintegrated": 0, 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_migrated": 0, 2024-03-21T11:49:50.596 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_processing": 0, 2024-03-21T11:49:50.597 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_enqueued": 0, 2024-03-21T11:49:50.597 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_prioritized": 0, 2024-03-21T11:49:50.597 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_started": 0, 2024-03-21T11:49:50.597 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_completed": 0, 2024-03-21T11:49:50.597 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T11:49:50.597 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_exportdir": 0, 2024-03-21T11:49:50.597 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_flush": 0, 2024-03-21T11:49:50.597 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragmentdir": 0, 2024-03-21T11:49:50.597 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragstats": 0, 2024-03-21T11:49:50.597 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_inodestats": 0 2024-03-21T11:49:50.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.597 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log": { 2024-03-21T11:49:50.597 INFO:teuthology.orchestra.run.smithi047.stdout: "evlrg": 0, 2024-03-21T11:49:50.597 INFO:teuthology.orchestra.run.smithi047.stdout: "evadd": 0, 2024-03-21T11:49:50.598 INFO:teuthology.orchestra.run.smithi047.stdout: "evex": 0, 2024-03-21T11:49:50.598 INFO:teuthology.orchestra.run.smithi047.stdout: "evtrm": 0, 2024-03-21T11:49:50.598 INFO:teuthology.orchestra.run.smithi047.stdout: "ev": 69338, 2024-03-21T11:49:50.598 INFO:teuthology.orchestra.run.smithi047.stdout: "evexg": 0, 2024-03-21T11:49:50.598 INFO:teuthology.orchestra.run.smithi047.stdout: "evexd": 0, 2024-03-21T11:49:50.598 INFO:teuthology.orchestra.run.smithi047.stdout: "segadd": 0, 2024-03-21T11:49:50.598 INFO:teuthology.orchestra.run.smithi047.stdout: "segex": 0, 2024-03-21T11:49:50.598 INFO:teuthology.orchestra.run.smithi047.stdout: "segtrm": 0, 2024-03-21T11:49:50.598 INFO:teuthology.orchestra.run.smithi047.stdout: "seg": 81, 2024-03-21T11:49:50.598 INFO:teuthology.orchestra.run.smithi047.stdout: "segmjr": 7, 2024-03-21T11:49:50.598 INFO:teuthology.orchestra.run.smithi047.stdout: "segexg": 0, 2024-03-21T11:49:50.598 INFO:teuthology.orchestra.run.smithi047.stdout: "segexd": 0, 2024-03-21T11:49:50.598 INFO:teuthology.orchestra.run.smithi047.stdout: "expos": 4194304, 2024-03-21T11:49:50.599 INFO:teuthology.orchestra.run.smithi047.stdout: "wrpos": 148357176, 2024-03-21T11:49:50.599 INFO:teuthology.orchestra.run.smithi047.stdout: "rdpos": 148356038, 2024-03-21T11:49:50.599 INFO:teuthology.orchestra.run.smithi047.stdout: "jlat": { 2024-03-21T11:49:50.599 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.599 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.599 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.599 INFO:teuthology.orchestra.run.smithi047.stdout: "replayed": 69338 2024-03-21T11:49:50.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.599 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_mem": { 2024-03-21T11:49:50.600 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 61, 2024-03-21T11:49:50.600 INFO:teuthology.orchestra.run.smithi047.stdout: "ino+": 14625, 2024-03-21T11:49:50.600 INFO:teuthology.orchestra.run.smithi047.stdout: "ino-": 14564, 2024-03-21T11:49:50.600 INFO:teuthology.orchestra.run.smithi047.stdout: "dir": 12, 2024-03-21T11:49:50.600 INFO:teuthology.orchestra.run.smithi047.stdout: "dir+": 35, 2024-03-21T11:49:50.600 INFO:teuthology.orchestra.run.smithi047.stdout: "dir-": 23, 2024-03-21T11:49:50.601 INFO:teuthology.orchestra.run.smithi047.stdout: "dn": 14618, 2024-03-21T11:49:50.601 INFO:teuthology.orchestra.run.smithi047.stdout: "dn+": 14768, 2024-03-21T11:49:50.601 INFO:teuthology.orchestra.run.smithi047.stdout: "dn-": 150, 2024-03-21T11:49:50.601 INFO:teuthology.orchestra.run.smithi047.stdout: "cap": 0, 2024-03-21T11:49:50.601 INFO:teuthology.orchestra.run.smithi047.stdout: "cap+": 0, 2024-03-21T11:49:50.601 INFO:teuthology.orchestra.run.smithi047.stdout: "cap-": 0, 2024-03-21T11:49:50.601 INFO:teuthology.orchestra.run.smithi047.stdout: "rss": 65188, 2024-03-21T11:49:50.602 INFO:teuthology.orchestra.run.smithi047.stdout: "heap": 215420 2024-03-21T11:49:50.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.602 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_server": { 2024-03-21T11:49:50.602 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_client_request": 0, 2024-03-21T11:49:50.602 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_server_request": 0, 2024-03-21T11:49:50.602 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_request": 0, 2024-03-21T11:49:50.603 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_session": 0, 2024-03-21T11:49:50.603 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_peer_request": 0, 2024-03-21T11:49:50.603 INFO:teuthology.orchestra.run.smithi047.stdout: "req_create_latency": { 2024-03-21T11:49:50.603 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.603 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.603 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.603 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getattr_latency": { 2024-03-21T11:49:50.603 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.603 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.604 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.604 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getfilelock_latency": { 2024-03-21T11:49:50.604 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.604 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.604 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.604 INFO:teuthology.orchestra.run.smithi047.stdout: "req_link_latency": { 2024-03-21T11:49:50.604 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.604 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.604 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.604 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookup_latency": { 2024-03-21T11:49:50.604 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.605 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.605 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.605 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookuphash_latency": { 2024-03-21T11:49:50.605 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.605 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.605 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.605 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupino_latency": { 2024-03-21T11:49:50.605 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.605 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.606 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.606 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupname_latency": { 2024-03-21T11:49:50.606 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.606 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.606 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.606 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupparent_latency": { 2024-03-21T11:49:50.606 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.606 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.606 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.606 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupsnap_latency": { 2024-03-21T11:49:50.607 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.607 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.607 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.607 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lssnap_latency": { 2024-03-21T11:49:50.607 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.607 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.607 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.607 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mkdir_latency": { 2024-03-21T11:49:50.607 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.608 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.608 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.608 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mknod_latency": { 2024-03-21T11:49:50.608 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.608 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.608 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.608 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mksnap_latency": { 2024-03-21T11:49:50.608 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: "req_open_latency": { 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: "req_readdir_latency": { 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rename_latency": { 2024-03-21T11:49:50.609 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.610 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.610 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.610 INFO:teuthology.orchestra.run.smithi047.stdout: "req_renamesnap_latency": { 2024-03-21T11:49:50.610 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.610 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.610 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.610 INFO:teuthology.orchestra.run.smithi047.stdout: "req_snapdiff_latency": { 2024-03-21T11:49:50.610 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.610 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.610 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmdir_latency": { 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmsnap_latency": { 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmxattr_latency": { 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.611 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setattr_latency": { 2024-03-21T11:49:50.612 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.612 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.612 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.612 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setdirlayout_latency": { 2024-03-21T11:49:50.612 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.612 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.612 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.612 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setfilelock_latency": { 2024-03-21T11:49:50.612 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.612 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.612 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setlayout_latency": { 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setxattr_latency": { 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: "req_symlink_latency": { 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.614 INFO:teuthology.orchestra.run.smithi047.stdout: "req_unlink_latency": { 2024-03-21T11:49:50.614 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.614 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.614 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.614 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_revoke_eviction": 0, 2024-03-21T11:49:50.614 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_acquisition_throttle": 0, 2024-03-21T11:49:50.614 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getvxattr_latency": { 2024-03-21T11:49:50.614 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.614 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.614 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.614 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:50.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.615 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_sessions": { 2024-03-21T11:49:50.615 INFO:teuthology.orchestra.run.smithi047.stdout: "session_count": 1, 2024-03-21T11:49:50.615 INFO:teuthology.orchestra.run.smithi047.stdout: "session_add": 2, 2024-03-21T11:49:50.615 INFO:teuthology.orchestra.run.smithi047.stdout: "session_remove": 1, 2024-03-21T11:49:50.615 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_open": 1, 2024-03-21T11:49:50.615 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_stale": 0, 2024-03-21T11:49:50.615 INFO:teuthology.orchestra.run.smithi047.stdout: "total_load": 0, 2024-03-21T11:49:50.615 INFO:teuthology.orchestra.run.smithi047.stdout: "average_load": 0, 2024-03-21T11:49:50.615 INFO:teuthology.orchestra.run.smithi047.stdout: "avg_session_uptime": 2863, 2024-03-21T11:49:50.615 INFO:teuthology.orchestra.run.smithi047.stdout: "mdthresh_evicted": 0 2024-03-21T11:49:50.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.615 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-03-21T11:49:50.616 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-03-21T11:49:50.616 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-03-21T11:49:50.616 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T11:49:50.616 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-03-21T11:49:50.616 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T11:49:50.616 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-03-21T11:49:50.616 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T11:49:50.616 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T11:49:50.616 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T11:49:50.616 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T11:49:50.616 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T11:49:50.616 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-03-21T11:49:50.616 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T11:49:50.616 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T11:49:50.617 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-03-21T11:49:50.617 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-03-21T11:49:50.617 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-03-21T11:49:50.617 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-03-21T11:49:50.617 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T11:49:50.617 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T11:49:50.617 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T11:49:50.617 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T11:49:50.617 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T11:49:50.617 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-03-21T11:49:50.618 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-03-21T11:49:50.618 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-03-21T11:49:50.618 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T11:49:50.618 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T11:49:50.618 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-03-21T11:49:50.619 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-03-21T11:49:50.619 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-03-21T11:49:50.619 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-03-21T11:49:50.619 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T11:49:50.619 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-03-21T11:49:50.619 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T11:49:50.620 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-03-21T11:49:50.620 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 241991, 2024-03-21T11:49:50.620 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 128, 2024-03-21T11:49:50.620 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-03-21T11:49:50.620 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-03-21T11:49:50.620 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-03-21T11:49:50.620 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-03-21T11:49:50.620 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-03-21T11:49:50.620 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-03-21T11:49:50.620 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-03-21T11:49:50.620 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-03-21T11:49:50.621 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 8888, 2024-03-21T11:49:50.621 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 170, 2024-03-21T11:49:50.621 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T11:49:50.621 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-03-21T11:49:50.621 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-03-21T11:49:50.621 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-03-21T11:49:50.621 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 10449758, 2024-03-21T11:49:50.621 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 64955, 2024-03-21T11:49:50.622 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-03-21T11:49:50.622 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-03-21T11:49:50.622 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-03-21T11:49:50.622 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-03-21T11:49:50.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.622 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter": { 2024-03-21T11:49:50.622 INFO:teuthology.orchestra.run.smithi047.stdout: "op_active": 0, 2024-03-21T11:49:50.622 INFO:teuthology.orchestra.run.smithi047.stdout: "op_laggy": 0, 2024-03-21T11:49:50.622 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send": 5569, 2024-03-21T11:49:50.623 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send_bytes": 21, 2024-03-21T11:49:50.623 INFO:teuthology.orchestra.run.smithi047.stdout: "op_resend": 3, 2024-03-21T11:49:50.623 INFO:teuthology.orchestra.run.smithi047.stdout: "op_reply": 5569, 2024-03-21T11:49:50.623 INFO:teuthology.orchestra.run.smithi047.stdout: "op_latency": { 2024-03-21T11:49:50.623 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 5569, 2024-03-21T11:49:50.623 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 40.445120869, 2024-03-21T11:49:50.623 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.007262546 2024-03-21T11:49:50.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.623 INFO:teuthology.orchestra.run.smithi047.stdout: "op_inflight": 0, 2024-03-21T11:49:50.624 INFO:teuthology.orchestra.run.smithi047.stdout: "oplen_avg": { 2024-03-21T11:49:50.624 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 5569, 2024-03-21T11:49:50.624 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 5570 2024-03-21T11:49:50.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.624 INFO:teuthology.orchestra.run.smithi047.stdout: "op": 5569, 2024-03-21T11:49:50.624 INFO:teuthology.orchestra.run.smithi047.stdout: "op_r": 5569, 2024-03-21T11:49:50.624 INFO:teuthology.orchestra.run.smithi047.stdout: "op_w": 0, 2024-03-21T11:49:50.625 INFO:teuthology.orchestra.run.smithi047.stdout: "op_rmw": 0, 2024-03-21T11:49:50.625 INFO:teuthology.orchestra.run.smithi047.stdout: "op_pg": 0, 2024-03-21T11:49:50.625 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_stat": 3183, 2024-03-21T11:49:50.625 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_create": 0, 2024-03-21T11:49:50.625 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_read": 2384, 2024-03-21T11:49:50.625 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_write": 0, 2024-03-21T11:49:50.626 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writefull": 0, 2024-03-21T11:49:50.626 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writesame": 0, 2024-03-21T11:49:50.626 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_append": 0, 2024-03-21T11:49:50.626 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_zero": 0, 2024-03-21T11:49:50.626 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_truncate": 0, 2024-03-21T11:49:50.626 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_delete": 0, 2024-03-21T11:49:50.626 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_mapext": 0, 2024-03-21T11:49:50.627 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_sparse_read": 0, 2024-03-21T11:49:50.627 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_clonerange": 0, 2024-03-21T11:49:50.627 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_getxattr": 1, 2024-03-21T11:49:50.627 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_setxattr": 0, 2024-03-21T11:49:50.627 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_cmpxattr": 0, 2024-03-21T11:49:50.627 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_rmxattr": 0, 2024-03-21T11:49:50.627 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_resetxattrs": 0, 2024-03-21T11:49:50.628 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_call": 0, 2024-03-21T11:49:50.628 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_watch": 0, 2024-03-21T11:49:50.628 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_notify": 0, 2024-03-21T11:49:50.628 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T11:49:50.628 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls": 0, 2024-03-21T11:49:50.628 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls_filter": 0, 2024-03-21T11:49:50.628 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_other": 0, 2024-03-21T11:49:50.628 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_active": 0, 2024-03-21T11:49:50.629 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_send": 0, 2024-03-21T11:49:50.629 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_resend": 0, 2024-03-21T11:49:50.629 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_ping": 0, 2024-03-21T11:49:50.629 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_active": 0, 2024-03-21T11:49:50.629 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_send": 0, 2024-03-21T11:49:50.629 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_resend": 0, 2024-03-21T11:49:50.629 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_active": 0, 2024-03-21T11:49:50.630 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_send": 0, 2024-03-21T11:49:50.630 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_resend": 0, 2024-03-21T11:49:50.630 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_active": 0, 2024-03-21T11:49:50.630 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_send": 0, 2024-03-21T11:49:50.630 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_resend": 0, 2024-03-21T11:49:50.630 INFO:teuthology.orchestra.run.smithi047.stdout: "command_active": 0, 2024-03-21T11:49:50.630 INFO:teuthology.orchestra.run.smithi047.stdout: "command_send": 0, 2024-03-21T11:49:50.631 INFO:teuthology.orchestra.run.smithi047.stdout: "command_resend": 0, 2024-03-21T11:49:50.631 INFO:teuthology.orchestra.run.smithi047.stdout: "map_epoch": 96, 2024-03-21T11:49:50.631 INFO:teuthology.orchestra.run.smithi047.stdout: "map_full": 0, 2024-03-21T11:49:50.631 INFO:teuthology.orchestra.run.smithi047.stdout: "map_inc": 5, 2024-03-21T11:49:50.631 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_sessions": 11, 2024-03-21T11:49:50.631 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_open": 13, 2024-03-21T11:49:50.631 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_close": 2, 2024-03-21T11:49:50.632 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_laggy": 0, 2024-03-21T11:49:50.632 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_wr": 0, 2024-03-21T11:49:50.632 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_rd": 2, 2024-03-21T11:49:50.632 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_del": 0 2024-03-21T11:49:50.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.632 INFO:teuthology.orchestra.run.smithi047.stdout: "oft": { 2024-03-21T11:49:50.633 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_objs": 0, 2024-03-21T11:49:50.633 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_kv_pairs": 0, 2024-03-21T11:49:50.633 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_updates": 0, 2024-03-21T11:49:50.633 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_removes": 0 2024-03-21T11:49:50.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.633 INFO:teuthology.orchestra.run.smithi047.stdout: "osd-slow-ops": { 2024-03-21T11:49:50.633 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_ops_count": 0 2024-03-21T11:49:50.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.634 INFO:teuthology.orchestra.run.smithi047.stdout: "purge_queue": { 2024-03-21T11:49:50.634 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops": 0, 2024-03-21T11:49:50.634 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T11:49:50.634 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing": 0, 2024-03-21T11:49:50.634 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_high_water": 0, 2024-03-21T11:49:50.635 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed_ops": 0, 2024-03-21T11:49:50.635 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed": 0, 2024-03-21T11:49:50.635 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_item_in_journal": 0 2024-03-21T11:49:50.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.635 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T11:49:50.635 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T11:49:50.636 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-03-21T11:49:50.636 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T11:49:50.636 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 6065, 2024-03-21T11:49:50.636 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 146090956, 2024-03-21T11:49:50.636 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T11:49:50.637 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 6065, 2024-03-21T11:49:50.637 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T11:49:50.637 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T11:49:50.637 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 6065, 2024-03-21T11:49:50.637 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 146090956, 2024-03-21T11:49:50.637 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T11:49:50.638 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.638 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.638 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.638 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:50.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.638 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_bytes": { 2024-03-21T11:49:50.638 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T11:49:50.638 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-03-21T11:49:50.638 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T11:49:50.639 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T11:49:50.639 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T11:49:50.639 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T11:49:50.639 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T11:49:50.639 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 5569, 2024-03-21T11:49:50.639 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 144164749, 2024-03-21T11:49:50.639 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 5569, 2024-03-21T11:49:50.639 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 144164749, 2024-03-21T11:49:50.639 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T11:49:50.639 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.639 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_ops": { 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 1024, 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 5569, 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 5569, 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 5569, 2024-03-21T11:49:50.640 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 5569, 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle": { 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T11:49:50.641 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T11:49:50.642 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T11:49:50.642 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 0, 2024-03-21T11:49:50.642 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 0, 2024-03-21T11:49:50.642 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T11:49:50.642 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.642 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.642 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.642 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:50.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:50.642 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle-0x563ef8e3e780": { 2024-03-21T11:49:50.642 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T11:49:50.642 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-03-21T11:49:50.642 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 0, 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 0, 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:50.643 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:49:51.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:50 smithi047 ceph-mon[31351]: pgmap v1196: 129 pgs: 129 active+clean; 545 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-03-21T11:49:51.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:50 smithi102 ceph-mon[36748]: pgmap v1196: 129 pgs: 129 active+clean; 545 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-03-21T11:49:51.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:50 smithi100 ceph-mon[36245]: pgmap v1196: 129 pgs: 129 active+clean; 545 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 3.4 MiB/s wr, 167 op/s 2024-03-21T11:49:51.669 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-21T11:49:51.670 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-21T11:49:51.670 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-03-21T11:49:51.670 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-03-21T11:49:51.670 DEBUG:tasks.check_counter:Getting stats from d 2024-03-21T11:49:51.670 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-21T11:49:52.125 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:49:53.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:52 smithi102 ceph-mon[36748]: pgmap v1197: 129 pgs: 129 active+clean; 541 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 2.9 MiB/s wr, 155 op/s 2024-03-21T11:49:53.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:52 smithi047 ceph-mon[31351]: pgmap v1197: 129 pgs: 129 active+clean; 541 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 2.9 MiB/s wr, 155 op/s 2024-03-21T11:49:53.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:52 smithi100 ceph-mon[36245]: pgmap v1197: 129 pgs: 129 active+clean; 541 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 2.9 MiB/s wr, 155 op/s 2024-03-21T11:49:54.906 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:49:54.906 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T11:49:54.906 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 4845, 2024-03-21T11:49:54.906 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 4847, 2024-03-21T11:49:54.906 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 15313024, 2024-03-21T11:49:54.907 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 1540717, 2024-03-21T11:49:54.907 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 6, 2024-03-21T11:49:54.907 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 2, 2024-03-21T11:49:54.907 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.859108023, 2024-03-21T11:49:54.907 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.247216364, 2024-03-21T11:49:54.907 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.255421402, 2024-03-21T11:49:54.907 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.283499193, 2024-03-21T11:49:54.907 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:49:54.907 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 4845, 2024-03-21T11:49:54.907 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.466747595, 2024-03-21T11:49:54.907 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000096335 2024-03-21T11:49:54.907 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.907 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:49:54.908 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.908 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.908 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.908 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T11:49:54.908 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T11:49:54.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.908 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T11:49:54.908 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 930, 2024-03-21T11:49:54.908 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 907, 2024-03-21T11:49:54.908 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 5253158, 2024-03-21T11:49:54.908 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 421708, 2024-03-21T11:49:54.908 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-03-21T11:49:54.909 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-03-21T11:49:54.909 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.213458519, 2024-03-21T11:49:54.909 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.073187764, 2024-03-21T11:49:54.909 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.096210553, 2024-03-21T11:49:54.909 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.042918748, 2024-03-21T11:49:54.909 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:49:54.909 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 907, 2024-03-21T11:49:54.909 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.117754774, 2024-03-21T11:49:54.909 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000129828 2024-03-21T11:49:54.909 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.909 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:49:54.909 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.909 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.910 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.910 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 951296, 2024-03-21T11:49:54.910 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 286496 2024-03-21T11:49:54.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.910 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T11:49:54.910 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 86, 2024-03-21T11:49:54.910 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 444, 2024-03-21T11:49:54.910 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 21006, 2024-03-21T11:49:54.910 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 812568, 2024-03-21T11:49:54.910 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 7, 2024-03-21T11:49:54.910 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 4, 2024-03-21T11:49:54.910 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.089560151, 2024-03-21T11:49:54.911 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.062021594, 2024-03-21T11:49:54.911 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.010889707, 2024-03-21T11:49:54.911 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.008334845, 2024-03-21T11:49:54.911 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:49:54.911 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 442, 2024-03-21T11:49:54.911 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.061995949, 2024-03-21T11:49:54.911 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000140262 2024-03-21T11:49:54.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.911 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:49:54.911 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.911 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.911 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.911 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T11:49:54.912 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 785856 2024-03-21T11:49:54.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.912 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-03-21T11:49:54.912 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 1, 2024-03-21T11:49:54.912 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-03-21T11:49:54.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.912 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-MDSRank": { 2024-03-21T11:49:54.912 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-03-21T11:49:54.912 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-03-21T11:49:54.912 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 8928, 2024-03-21T11:49:54.912 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.419957216, 2024-03-21T11:49:54.912 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000047038 2024-03-21T11:49:54.912 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:54.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.913 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-PurgeQueue": { 2024-03-21T11:49:54.913 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-03-21T11:49:54.913 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-03-21T11:49:54.913 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.913 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.913 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.913 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:54.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.913 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": { 2024-03-21T11:49:54.913 INFO:teuthology.orchestra.run.smithi047.stdout: "request": 0, 2024-03-21T11:49:54.913 INFO:teuthology.orchestra.run.smithi047.stdout: "reply": 0, 2024-03-21T11:49:54.913 INFO:teuthology.orchestra.run.smithi047.stdout: "reply_latency": { 2024-03-21T11:49:54.913 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.914 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.914 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.914 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.914 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_reply": 0, 2024-03-21T11:49:54.914 INFO:teuthology.orchestra.run.smithi047.stdout: "forward": 0, 2024-03-21T11:49:54.914 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_complete": 0, 2024-03-21T11:49:54.914 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_keys": 0, 2024-03-21T11:49:54.914 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_commit": 0, 2024-03-21T11:49:54.914 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_split": 0, 2024-03-21T11:49:54.914 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_merge": 0, 2024-03-21T11:49:54.914 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes": 1674, 2024-03-21T11:49:54.914 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_top": 0, 2024-03-21T11:49:54.914 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_bottom": 1674, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pin_tail": 0, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pinned": 1674, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_expired": 0, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_with_caps": 0, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": 0, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "subtrees": 1, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse": 0, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_hit": 0, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_forward": 0, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_discover": 0, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_dir_fetch": 0, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_remote_ino": 0, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_lock": 0, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "load_cent": 0, 2024-03-21T11:49:54.915 INFO:teuthology.orchestra.run.smithi047.stdout: "q": 0, 2024-03-21T11:49:54.916 INFO:teuthology.orchestra.run.smithi047.stdout: "exported": 0, 2024-03-21T11:49:54.916 INFO:teuthology.orchestra.run.smithi047.stdout: "exported_inodes": 0, 2024-03-21T11:49:54.916 INFO:teuthology.orchestra.run.smithi047.stdout: "imported": 0, 2024-03-21T11:49:54.916 INFO:teuthology.orchestra.run.smithi047.stdout: "imported_inodes": 0, 2024-03-21T11:49:54.916 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_dir_fetch": 0, 2024-03-21T11:49:54.916 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_backtrace_fetch": 0, 2024-03-21T11:49:54.916 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_peer_discover": 0, 2024-03-21T11:49:54.916 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rfiles": 0, 2024-03-21T11:49:54.916 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rbytes": 0, 2024-03-21T11:49:54.916 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rsnaps": 0, 2024-03-21T11:49:54.916 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T11:49:54.916 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_set_tag": 0, 2024-03-21T11:49:54.916 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T11:49:54.916 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_inotable_repaired": 0, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_inodes": 0, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_file_inodes": 0, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_inode_file_caps": 0, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_grant": 0, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps": 0, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps_dirty": 0, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_cap_release": 0, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "process_request_cap_release": 0 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache": { 2024-03-21T11:49:54.917 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update": 0, 2024-03-21T11:49:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update_receipt": 0, 2024-03-21T11:49:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_try_discover": 0, 2024-03-21T11:49:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_send_discover": 0, 2024-03-21T11:49:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_handle_discover": 0, 2024-03-21T11:49:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays": 0, 2024-03-21T11:49:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_delayed": 0, 2024-03-21T11:49:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_enqueuing": 0, 2024-03-21T11:49:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_created": 0, 2024-03-21T11:49:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_enqueued": 0, 2024-03-21T11:49:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_reintegrated": 0, 2024-03-21T11:49:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_migrated": 0, 2024-03-21T11:49:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_processing": 0, 2024-03-21T11:49:54.918 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_enqueued": 0, 2024-03-21T11:49:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_prioritized": 0, 2024-03-21T11:49:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_started": 0, 2024-03-21T11:49:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_completed": 0, 2024-03-21T11:49:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T11:49:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_exportdir": 0, 2024-03-21T11:49:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_flush": 0, 2024-03-21T11:49:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragmentdir": 0, 2024-03-21T11:49:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragstats": 0, 2024-03-21T11:49:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_inodestats": 0 2024-03-21T11:49:54.919 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log": { 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "evlrg": 0, 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "evadd": 0, 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "evex": 0, 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "evtrm": 0, 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "ev": 13068, 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "evexg": 0, 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "evexd": 0, 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "segadd": 0, 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "segex": 0, 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "segtrm": 0, 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "seg": 15, 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "segmjr": 3, 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "segexg": 0, 2024-03-21T11:49:54.920 INFO:teuthology.orchestra.run.smithi047.stdout: "segexd": 0, 2024-03-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi047.stdout: "expos": 4194304, 2024-03-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi047.stdout: "wrpos": 22332499, 2024-03-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi047.stdout: "rdpos": 22331361, 2024-03-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi047.stdout: "jlat": { 2024-03-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi047.stdout: "replayed": 13068 2024-03-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_mem": { 2024-03-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 30, 2024-03-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi047.stdout: "ino+": 1697, 2024-03-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi047.stdout: "ino-": 1667, 2024-03-21T11:49:54.922 INFO:teuthology.orchestra.run.smithi047.stdout: "dir": 11, 2024-03-21T11:49:54.922 INFO:teuthology.orchestra.run.smithi047.stdout: "dir+": 32, 2024-03-21T11:49:54.922 INFO:teuthology.orchestra.run.smithi047.stdout: "dir-": 21, 2024-03-21T11:49:54.922 INFO:teuthology.orchestra.run.smithi047.stdout: "dn": 1674, 2024-03-21T11:49:54.922 INFO:teuthology.orchestra.run.smithi047.stdout: "dn+": 1741, 2024-03-21T11:49:54.922 INFO:teuthology.orchestra.run.smithi047.stdout: "dn-": 67, 2024-03-21T11:49:54.922 INFO:teuthology.orchestra.run.smithi047.stdout: "cap": 0, 2024-03-21T11:49:54.922 INFO:teuthology.orchestra.run.smithi047.stdout: "cap+": 0, 2024-03-21T11:49:54.922 INFO:teuthology.orchestra.run.smithi047.stdout: "cap-": 0, 2024-03-21T11:49:54.922 INFO:teuthology.orchestra.run.smithi047.stdout: "rss": 53348, 2024-03-21T11:49:54.922 INFO:teuthology.orchestra.run.smithi047.stdout: "heap": 215420 2024-03-21T11:49:54.922 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.922 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_server": { 2024-03-21T11:49:54.923 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_client_request": 0, 2024-03-21T11:49:54.923 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_server_request": 0, 2024-03-21T11:49:54.923 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_request": 0, 2024-03-21T11:49:54.923 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_session": 0, 2024-03-21T11:49:54.923 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_peer_request": 0, 2024-03-21T11:49:54.923 INFO:teuthology.orchestra.run.smithi047.stdout: "req_create_latency": { 2024-03-21T11:49:54.923 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.923 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.923 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.923 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.923 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getattr_latency": { 2024-03-21T11:49:54.923 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.923 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.923 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.924 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.924 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getfilelock_latency": { 2024-03-21T11:49:54.924 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.924 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.924 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.924 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.924 INFO:teuthology.orchestra.run.smithi047.stdout: "req_link_latency": { 2024-03-21T11:49:54.924 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.924 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.924 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.924 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.924 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookup_latency": { 2024-03-21T11:49:54.924 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.924 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.925 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.925 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookuphash_latency": { 2024-03-21T11:49:54.925 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.925 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.925 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.925 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupino_latency": { 2024-03-21T11:49:54.925 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.925 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.925 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.925 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupname_latency": { 2024-03-21T11:49:54.926 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.926 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.926 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.926 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupparent_latency": { 2024-03-21T11:49:54.926 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.926 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.926 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.926 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupsnap_latency": { 2024-03-21T11:49:54.926 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.926 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.926 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.927 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.927 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lssnap_latency": { 2024-03-21T11:49:54.927 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.927 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.927 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.927 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.927 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mkdir_latency": { 2024-03-21T11:49:54.927 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.927 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.928 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.928 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.928 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mknod_latency": { 2024-03-21T11:49:54.928 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.928 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.928 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.928 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.928 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mksnap_latency": { 2024-03-21T11:49:54.928 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.928 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.928 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.929 INFO:teuthology.orchestra.run.smithi047.stdout: "req_open_latency": { 2024-03-21T11:49:54.929 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.929 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.929 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.929 INFO:teuthology.orchestra.run.smithi047.stdout: "req_readdir_latency": { 2024-03-21T11:49:54.929 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.929 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.929 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.929 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rename_latency": { 2024-03-21T11:49:54.929 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.930 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.930 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.930 INFO:teuthology.orchestra.run.smithi047.stdout: "req_renamesnap_latency": { 2024-03-21T11:49:54.930 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.930 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.930 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.930 INFO:teuthology.orchestra.run.smithi047.stdout: "req_snapdiff_latency": { 2024-03-21T11:49:54.930 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.930 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.930 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.931 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmdir_latency": { 2024-03-21T11:49:54.931 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.931 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.931 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.931 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.931 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmsnap_latency": { 2024-03-21T11:49:54.931 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.931 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.931 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.931 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.931 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmxattr_latency": { 2024-03-21T11:49:54.931 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.931 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.932 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.932 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setattr_latency": { 2024-03-21T11:49:54.932 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.932 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.932 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.932 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setdirlayout_latency": { 2024-03-21T11:49:54.932 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.932 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.932 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.932 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setfilelock_latency": { 2024-03-21T11:49:54.933 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.933 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.933 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.933 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setlayout_latency": { 2024-03-21T11:49:54.933 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.933 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.933 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.933 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setxattr_latency": { 2024-03-21T11:49:54.933 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.933 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.933 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.934 INFO:teuthology.orchestra.run.smithi047.stdout: "req_symlink_latency": { 2024-03-21T11:49:54.934 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.934 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.934 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.934 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.934 INFO:teuthology.orchestra.run.smithi047.stdout: "req_unlink_latency": { 2024-03-21T11:49:54.934 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.934 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.934 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.934 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.934 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_revoke_eviction": 0, 2024-03-21T11:49:54.934 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_acquisition_throttle": 0, 2024-03-21T11:49:54.934 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getvxattr_latency": { 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_sessions": { 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: "session_count": 1, 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: "session_add": 2, 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: "session_remove": 1, 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_open": 1, 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_stale": 0, 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: "total_load": 0, 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: "average_load": 0, 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: "avg_session_uptime": 2865, 2024-03-21T11:49:54.935 INFO:teuthology.orchestra.run.smithi047.stdout: "mdthresh_evicted": 0 2024-03-21T11:49:54.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.936 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-03-21T11:49:54.936 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-03-21T11:49:54.936 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-03-21T11:49:54.936 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T11:49:54.936 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-03-21T11:49:54.936 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T11:49:54.936 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-03-21T11:49:54.936 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T11:49:54.936 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T11:49:54.936 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T11:49:54.936 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T11:49:54.936 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T11:49:54.936 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-03-21T11:49:54.937 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T11:49:54.937 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T11:49:54.937 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-03-21T11:49:54.937 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-03-21T11:49:54.937 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-03-21T11:49:54.937 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-03-21T11:49:54.937 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T11:49:54.937 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T11:49:54.937 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T11:49:54.937 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T11:49:54.937 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T11:49:54.937 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-03-21T11:49:54.937 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-03-21T11:49:54.937 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-03-21T11:49:54.938 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T11:49:54.938 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T11:49:54.938 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-03-21T11:49:54.938 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-03-21T11:49:54.938 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-03-21T11:49:54.938 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-03-21T11:49:54.938 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T11:49:54.938 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-03-21T11:49:54.938 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T11:49:54.938 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-03-21T11:49:54.938 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 201722, 2024-03-21T11:49:54.938 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 77, 2024-03-21T11:49:54.938 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-03-21T11:49:54.938 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 8888, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 170, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 1261473, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 9080, 2024-03-21T11:49:54.939 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-03-21T11:49:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-03-21T11:49:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-03-21T11:49:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-03-21T11:49:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter": { 2024-03-21T11:49:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: "op_active": 0, 2024-03-21T11:49:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: "op_laggy": 0, 2024-03-21T11:49:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send": 5354, 2024-03-21T11:49:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send_bytes": 21, 2024-03-21T11:49:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: "op_resend": 3, 2024-03-21T11:49:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: "op_reply": 5354, 2024-03-21T11:49:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: "op_latency": { 2024-03-21T11:49:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 5354, 2024-03-21T11:49:54.940 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 32.902281812, 2024-03-21T11:49:54.941 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.006145364 2024-03-21T11:49:54.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.941 INFO:teuthology.orchestra.run.smithi047.stdout: "op_inflight": 0, 2024-03-21T11:49:54.941 INFO:teuthology.orchestra.run.smithi047.stdout: "oplen_avg": { 2024-03-21T11:49:54.941 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 5354, 2024-03-21T11:49:54.941 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 5355 2024-03-21T11:49:54.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.941 INFO:teuthology.orchestra.run.smithi047.stdout: "op": 5354, 2024-03-21T11:49:54.941 INFO:teuthology.orchestra.run.smithi047.stdout: "op_r": 5354, 2024-03-21T11:49:54.941 INFO:teuthology.orchestra.run.smithi047.stdout: "op_w": 0, 2024-03-21T11:49:54.941 INFO:teuthology.orchestra.run.smithi047.stdout: "op_rmw": 0, 2024-03-21T11:49:54.941 INFO:teuthology.orchestra.run.smithi047.stdout: "op_pg": 0, 2024-03-21T11:49:54.941 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_stat": 3287, 2024-03-21T11:49:54.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_create": 0, 2024-03-21T11:49:54.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_read": 2065, 2024-03-21T11:49:54.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_write": 0, 2024-03-21T11:49:54.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writefull": 0, 2024-03-21T11:49:54.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writesame": 0, 2024-03-21T11:49:54.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_append": 0, 2024-03-21T11:49:54.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_zero": 0, 2024-03-21T11:49:54.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_truncate": 0, 2024-03-21T11:49:54.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_delete": 0, 2024-03-21T11:49:54.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_mapext": 0, 2024-03-21T11:49:54.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_sparse_read": 0, 2024-03-21T11:49:54.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_clonerange": 0, 2024-03-21T11:49:54.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_getxattr": 1, 2024-03-21T11:49:54.942 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_setxattr": 0, 2024-03-21T11:49:54.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_cmpxattr": 0, 2024-03-21T11:49:54.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_rmxattr": 0, 2024-03-21T11:49:54.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_resetxattrs": 0, 2024-03-21T11:49:54.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_call": 0, 2024-03-21T11:49:54.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_watch": 0, 2024-03-21T11:49:54.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_notify": 0, 2024-03-21T11:49:54.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T11:49:54.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls": 0, 2024-03-21T11:49:54.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls_filter": 0, 2024-03-21T11:49:54.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_other": 0, 2024-03-21T11:49:54.943 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_active": 0, 2024-03-21T11:49:54.943 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_send": 0, 2024-03-21T11:49:54.943 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_resend": 0, 2024-03-21T11:49:54.943 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_ping": 0, 2024-03-21T11:49:54.944 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_active": 0, 2024-03-21T11:49:54.944 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_send": 0, 2024-03-21T11:49:54.944 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_resend": 0, 2024-03-21T11:49:54.944 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_active": 0, 2024-03-21T11:49:54.944 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_send": 0, 2024-03-21T11:49:54.944 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_resend": 0, 2024-03-21T11:49:54.944 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_active": 0, 2024-03-21T11:49:54.944 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_send": 0, 2024-03-21T11:49:54.944 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_resend": 0, 2024-03-21T11:49:54.944 INFO:teuthology.orchestra.run.smithi047.stdout: "command_active": 0, 2024-03-21T11:49:54.944 INFO:teuthology.orchestra.run.smithi047.stdout: "command_send": 0, 2024-03-21T11:49:54.944 INFO:teuthology.orchestra.run.smithi047.stdout: "command_resend": 0, 2024-03-21T11:49:54.944 INFO:teuthology.orchestra.run.smithi047.stdout: "map_epoch": 96, 2024-03-21T11:49:54.944 INFO:teuthology.orchestra.run.smithi047.stdout: "map_full": 0, 2024-03-21T11:49:54.945 INFO:teuthology.orchestra.run.smithi047.stdout: "map_inc": 5, 2024-03-21T11:49:54.945 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_sessions": 6, 2024-03-21T11:49:54.945 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_open": 7, 2024-03-21T11:49:54.945 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_close": 1, 2024-03-21T11:49:54.945 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_laggy": 0, 2024-03-21T11:49:54.945 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_wr": 0, 2024-03-21T11:49:54.945 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_rd": 2, 2024-03-21T11:49:54.945 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_del": 0 2024-03-21T11:49:54.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.945 INFO:teuthology.orchestra.run.smithi047.stdout: "oft": { 2024-03-21T11:49:54.945 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_objs": 0, 2024-03-21T11:49:54.945 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_kv_pairs": 0, 2024-03-21T11:49:54.945 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_updates": 0, 2024-03-21T11:49:54.946 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_removes": 0 2024-03-21T11:49:54.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.946 INFO:teuthology.orchestra.run.smithi047.stdout: "osd-slow-ops": { 2024-03-21T11:49:54.946 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_ops_count": 0 2024-03-21T11:49:54.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.946 INFO:teuthology.orchestra.run.smithi047.stdout: "purge_queue": { 2024-03-21T11:49:54.946 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops": 0, 2024-03-21T11:49:54.946 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T11:49:54.946 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing": 0, 2024-03-21T11:49:54.946 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_high_water": 0, 2024-03-21T11:49:54.946 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed_ops": 0, 2024-03-21T11:49:54.946 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed": 0, 2024-03-21T11:49:54.946 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_item_in_journal": 0 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 5857, 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 20031253, 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 5857, 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 5857, 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 20031253, 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T11:49:54.947 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_bytes": { 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 5354, 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 18138334, 2024-03-21T11:49:54.948 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 5354, 2024-03-21T11:49:54.949 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 18138334, 2024-03-21T11:49:54.949 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T11:49:54.949 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.949 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.949 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.949 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:54.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.949 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_ops": { 2024-03-21T11:49:54.949 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T11:49:54.949 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 1024, 2024-03-21T11:49:54.949 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T11:49:54.949 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T11:49:54.949 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T11:49:54.950 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T11:49:54.950 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T11:49:54.950 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 5354, 2024-03-21T11:49:54.950 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 5354, 2024-03-21T11:49:54.950 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 5354, 2024-03-21T11:49:54.950 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 5354, 2024-03-21T11:49:54.950 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T11:49:54.950 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.950 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.950 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.950 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:54.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.950 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle": { 2024-03-21T11:49:54.950 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 0, 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 0, 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.951 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle-0x55703622e780": { 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 0, 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 0, 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T11:49:54.952 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:54.953 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:54.953 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:54.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:54.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:54.953 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:49:55.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:54 smithi102 ceph-mon[36748]: pgmap v1198: 129 pgs: 129 active+clean; 533 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 2.9 MiB/s wr, 164 op/s 2024-03-21T11:49:55.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:54 smithi047 ceph-mon[31351]: pgmap v1198: 129 pgs: 129 active+clean; 533 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 2.9 MiB/s wr, 164 op/s 2024-03-21T11:49:55.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:54 smithi100 ceph-mon[36245]: pgmap v1198: 129 pgs: 129 active+clean; 533 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 2.9 MiB/s wr, 164 op/s 2024-03-21T11:49:55.919 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-03-21T11:49:55.919 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-03-21T11:49:55.920 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2024-03-21T11:49:55.920 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2024-03-21T11:49:55.920 DEBUG:tasks.check_counter:Getting stats from g 2024-03-21T11:49:55.920 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-21T11:49:56.427 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:49:57.194 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:56 smithi100 ceph-mon[36245]: pgmap v1199: 129 pgs: 129 active+clean; 532 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 2.0 MiB/s wr, 115 op/s 2024-03-21T11:49:57.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:56 smithi102 ceph-mon[36748]: pgmap v1199: 129 pgs: 129 active+clean; 532 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 2.0 MiB/s wr, 115 op/s 2024-03-21T11:49:57.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:56 smithi047 ceph-mon[31351]: pgmap v1199: 129 pgs: 129 active+clean; 532 MiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 2.0 MiB/s wr, 115 op/s 2024-03-21T11:49:58.746 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:49:58.746 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T11:49:58.746 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 471, 2024-03-21T11:49:58.746 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 466, 2024-03-21T11:49:58.746 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 825888, 2024-03-21T11:49:58.747 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 281840, 2024-03-21T11:49:58.747 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-03-21T11:49:58.747 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-03-21T11:49:58.747 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.155460706, 2024-03-21T11:49:58.747 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.069629531, 2024-03-21T11:49:58.747 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.059521616, 2024-03-21T11:49:58.747 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.021610875, 2024-03-21T11:49:58.747 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:49:58.747 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 465, 2024-03-21T11:49:58.747 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.063137561, 2024-03-21T11:49:58.747 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000135779 2024-03-21T11:49:58.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:58.747 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:49:58.747 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:58.748 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:58.748 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:58.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:58.748 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 825888, 2024-03-21T11:49:58.748 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 281840 2024-03-21T11:49:58.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:58.748 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T11:49:58.748 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 6, 2024-03-21T11:49:58.748 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 359, 2024-03-21T11:49:58.748 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 3392, 2024-03-21T11:49:58.748 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 218768, 2024-03-21T11:49:58.748 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-03-21T11:49:58.748 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 2, 2024-03-21T11:49:58.749 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.064540974, 2024-03-21T11:49:58.749 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.053604775, 2024-03-21T11:49:58.749 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.004326138, 2024-03-21T11:49:58.749 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T11:49:58.749 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:49:58.749 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 358, 2024-03-21T11:49:58.749 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.052912855, 2024-03-21T11:49:58.749 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000147801 2024-03-21T11:49:58.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:58.749 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:49:58.749 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:58.749 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:58.749 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:58.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:58.750 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-21T11:49:58.750 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 218768 2024-03-21T11:49:58.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:58.750 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T11:49:58.750 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 0, 2024-03-21T11:49:58.750 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2, 2024-03-21T11:49:58.750 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 0, 2024-03-21T11:49:58.750 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 0, 2024-03-21T11:49:58.750 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-03-21T11:49:58.750 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-03-21T11:49:58.750 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.001164458, 2024-03-21T11:49:58.750 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T11:49:58.750 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.000920633, 2024-03-21T11:49:58.750 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-03-21T11:49:58.751 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T11:49:58.752 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-03-21T11:49:58.753 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T11:49:58.754 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T11:49:58.754 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-03-21T11:49:58.754 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-03-21T11:49:58.754 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-03-21T11:49:58.754 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-03-21T11:49:58.754 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T11:49:58.754 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-03-21T11:49:58.754 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T11:49:58.754 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-03-21T11:49:58.754 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 148704, 2024-03-21T11:49:58.754 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 29, 2024-03-21T11:49:58.754 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-03-21T11:49:58.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 473, 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 759280, 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 473, 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T11:49:58.756 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T11:49:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 473, 2024-03-21T11:49:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 759280, 2024-03-21T11:49:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T11:49:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:49:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:49:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:49:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:49:58.757 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:49:59.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:49:58 smithi102 ceph-mon[36748]: pgmap v1200: 129 pgs: 129 active+clean; 530 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-03-21T11:49:59.280 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:49:58 smithi047 ceph-mon[31351]: pgmap v1200: 129 pgs: 129 active+clean; 530 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-03-21T11:49:59.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:49:58 smithi100 ceph-mon[36245]: pgmap v1200: 129 pgs: 129 active+clean; 530 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-03-21T11:49:59.801 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-21T11:49:59.801 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-21T11:49:59.801 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-03-21T11:49:59.802 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-03-21T11:49:59.802 DEBUG:tasks.check_counter:Getting stats from j 2024-03-21T11:49:59.802 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-21T11:50:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T11:50:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:00 smithi102 ceph-mon[36748]: pgmap v1201: 129 pgs: 129 active+clean; 529 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2024-03-21T11:50:01.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:00 smithi102 ceph-mon[36748]: overall HEALTH_OK 2024-03-21T11:50:01.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:00 smithi047 ceph-mon[31351]: pgmap v1201: 129 pgs: 129 active+clean; 529 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2024-03-21T11:50:01.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:00 smithi047 ceph-mon[31351]: overall HEALTH_OK 2024-03-21T11:50:01.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:00 smithi100 ceph-mon[36245]: pgmap v1201: 129 pgs: 129 active+clean; 529 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2024-03-21T11:50:01.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:00 smithi100 ceph-mon[36245]: overall HEALTH_OK 2024-03-21T11:50:02.787 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T11:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T11:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 0, 2024-03-21T11:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2, 2024-03-21T11:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 0, 2024-03-21T11:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 0, 2024-03-21T11:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-03-21T11:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-03-21T11:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.004597715, 2024-03-21T11:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T11:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.001281411, 2024-03-21T11:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T11:50:02.788 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T11:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T11:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:50:02.789 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T11:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 482, 2024-03-21T11:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 475, 2024-03-21T11:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 830864, 2024-03-21T11:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 286656, 2024-03-21T11:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-03-21T11:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-03-21T11:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.161063583, 2024-03-21T11:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.079425441, 2024-03-21T11:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.053662050, 2024-03-21T11:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.022430017, 2024-03-21T11:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:02.790 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 475, 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.056551425, 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000119055 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 830864, 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 286656 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2, 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 362, 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 320, 2024-03-21T11:50:02.791 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 219200, 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 2, 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.064804217, 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.053894897, 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.003985898, 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 360, 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.052859297, 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000146831 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:50:02.792 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T11:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 219200 2024-03-21T11:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-03-21T11:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-03-21T11:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-03-21T11:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-03-21T11:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-03-21T11:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-03-21T11:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T11:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-03-21T11:50:02.793 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T11:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-03-21T11:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T11:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T11:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T11:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T11:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T11:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-03-21T11:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T11:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T11:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-03-21T11:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-03-21T11:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-03-21T11:50:02.794 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-03-21T11:50:02.795 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 150766, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 31, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-03-21T11:50:02.796 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-03-21T11:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T11:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-03-21T11:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-03-21T11:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-03-21T11:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-03-21T11:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-03-21T11:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-03-21T11:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-03-21T11:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-03-21T11:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-03-21T11:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T11:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T11:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T11:50:02.797 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-03-21T11:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T11:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 480, 2024-03-21T11:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 760190, 2024-03-21T11:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 480, 2024-03-21T11:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T11:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T11:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 480, 2024-03-21T11:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 760190, 2024-03-21T11:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T11:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T11:50:02.798 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T11:50:02.799 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T11:50:02.799 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:50:02.799 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T11:50:02.799 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T11:50:03.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:02 smithi102 ceph-mon[36748]: pgmap v1202: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 598 KiB/s wr, 61 op/s 2024-03-21T11:50:03.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:02 smithi047 ceph-mon[31351]: pgmap v1202: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 598 KiB/s wr, 61 op/s 2024-03-21T11:50:03.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:02 smithi100 ceph-mon[36245]: pgmap v1202: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 598 KiB/s wr, 61 op/s 2024-03-21T11:50:03.705 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-21T11:50:03.705 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-21T11:50:03.706 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-21T11:50:03.706 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-21T11:50:03.706 DEBUG:tasks.check_counter:Getting stats from b 2024-03-21T11:50:03.706 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-21T11:50:04.109 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T11:50:05.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:04 smithi100 ceph-mon[36245]: pgmap v1203: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 418 KiB/s wr, 50 op/s 2024-03-21T11:50:05.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:04 smithi102 ceph-mon[36748]: pgmap v1203: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 418 KiB/s wr, 50 op/s 2024-03-21T11:50:05.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:04 smithi047 ceph-mon[31351]: pgmap v1203: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 418 KiB/s wr, 50 op/s 2024-03-21T11:50:05.955 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T11:50:05.955 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T11:50:05.955 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 277337, 2024-03-21T11:50:05.955 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 408343, 2024-03-21T11:50:05.955 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 62300570, 2024-03-21T11:50:05.955 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 388404929, 2024-03-21T11:50:05.955 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 35, 2024-03-21T11:50:05.955 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 5, 2024-03-21T11:50:05.955 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 26.353555250, 2024-03-21T11:50:05.955 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 8.897354330, 2024-03-21T11:50:05.956 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 10.879782327, 2024-03-21T11:50:05.956 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 6.324565093, 2024-03-21T11:50:05.956 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:05.956 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 408342, 2024-03-21T11:50:05.956 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 137.407363666, 2024-03-21T11:50:05.956 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000336500 2024-03-21T11:50:05.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.956 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:05.956 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 402481, 2024-03-21T11:50:05.956 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.108931684, 2024-03-21T11:50:05.956 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000270 2024-03-21T11:50:05.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.956 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 998352, 2024-03-21T11:50:05.956 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 345104 2024-03-21T11:50:05.957 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.957 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T11:50:05.957 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 542213, 2024-03-21T11:50:05.957 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 738100, 2024-03-21T11:50:05.957 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 122593577, 2024-03-21T11:50:05.957 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 463845193, 2024-03-21T11:50:05.957 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 10, 2024-03-21T11:50:05.957 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 5, 2024-03-21T11:50:05.957 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 40.589780189, 2024-03-21T11:50:05.957 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 15.669626826, 2024-03-21T11:50:05.957 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 22.376155615, 2024-03-21T11:50:05.958 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 5.333525274, 2024-03-21T11:50:05.958 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:05.958 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 738104, 2024-03-21T11:50:05.958 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 169.390572922, 2024-03-21T11:50:05.958 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000229494 2024-03-21T11:50:05.958 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.958 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:05.958 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1005424, 2024-03-21T11:50:05.958 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.213804587, 2024-03-21T11:50:05.958 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000212 2024-03-21T11:50:05.959 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.959 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T11:50:05.959 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 807584 2024-03-21T11:50:05.960 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.960 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T11:50:05.960 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 1094350, 2024-03-21T11:50:05.960 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 1599439, 2024-03-21T11:50:05.960 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 324621734, 2024-03-21T11:50:05.960 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 1367146491, 2024-03-21T11:50:05.961 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 581, 2024-03-21T11:50:05.961 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 9, 2024-03-21T11:50:05.961 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 83.594885575, 2024-03-21T11:50:05.961 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 41.078228901, 2024-03-21T11:50:05.961 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 43.813487245, 2024-03-21T11:50:05.961 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 6.447148118, 2024-03-21T11:50:05.961 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:05.961 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1599432, 2024-03-21T11:50:05.961 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 212.419738285, 2024-03-21T11:50:05.961 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000132809 2024-03-21T11:50:05.961 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.961 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:05.961 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1407832, 2024-03-21T11:50:05.961 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.397261887, 2024-03-21T11:50:05.962 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000282 2024-03-21T11:50:05.962 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.962 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 94976, 2024-03-21T11:50:05.962 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 31760 2024-03-21T11:50:05.962 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.962 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-21T11:50:05.962 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-03-21T11:50:05.963 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-21T11:50:05.963 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.963 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-03-21T11:50:05.963 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T11:50:05.964 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T11:50:05.964 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 223363, 2024-03-21T11:50:05.964 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 330.878579920, 2024-03-21T11:50:05.964 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001481349 2024-03-21T11:50:05.964 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:05.964 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.964 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-03-21T11:50:05.965 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T11:50:05.965 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T11:50:05.965 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 151834, 2024-03-21T11:50:05.965 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 30.455721985, 2024-03-21T11:50:05.965 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000200585 2024-03-21T11:50:05.965 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:05.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.966 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-03-21T11:50:05.966 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 502592, 2024-03-21T11:50:05.966 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 473583, 2024-03-21T11:50:05.966 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-03-21T11:50:05.966 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 473583, 2024-03-21T11:50:05.967 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 3111.066017480, 2024-03-21T11:50:05.967 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.006569209 2024-03-21T11:50:05.967 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.967 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-03-21T11:50:05.967 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 29009, 2024-03-21T11:50:05.967 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 4, 2024-03-21T11:50:05.967 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 1, 2024-03-21T11:50:05.968 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 5726, 2024-03-21T11:50:05.968 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 21, 2024-03-21T11:50:05.968 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 32, 2024-03-21T11:50:05.968 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 853, 2024-03-21T11:50:05.968 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 301, 2024-03-21T11:50:05.968 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 541, 2024-03-21T11:50:05.968 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 11, 2024-03-21T11:50:05.968 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 423, 2024-03-21T11:50:05.969 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 294, 2024-03-21T11:50:05.969 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 1, 2024-03-21T11:50:05.969 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 1, 2024-03-21T11:50:05.969 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 5, 2024-03-21T11:50:05.969 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 744157, 2024-03-21T11:50:05.969 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 418025, 2024-03-21T11:50:05.969 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 29005, 2024-03-21T11:50:05.969 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 70, 2024-03-21T11:50:05.970 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-03-21T11:50:05.970 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-03-21T11:50:05.970 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 5894, 2024-03-21T11:50:05.970 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-03-21T11:50:05.970 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-03-21T11:50:05.970 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 9, 2024-03-21T11:50:05.970 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 166, 2024-03-21T11:50:05.971 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 9, 2024-03-21T11:50:05.971 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 40, 2024-03-21T11:50:05.971 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-03-21T11:50:05.971 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 11038, 2024-03-21T11:50:05.971 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-03-21T11:50:05.971 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 4, 2024-03-21T11:50:05.971 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 234, 2024-03-21T11:50:05.972 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 0, 2024-03-21T11:50:05.972 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 11038, 2024-03-21T11:50:05.972 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-03-21T11:50:05.972 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T11:50:05.972 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-03-21T11:50:05.972 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 2019, 2024-03-21T11:50:05.973 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 46, 2024-03-21T11:50:05.973 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 1973, 2024-03-21T11:50:05.973 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 9019, 2024-03-21T11:50:05.973 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 22, 2024-03-21T11:50:05.973 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 117215, 2024-03-21T11:50:05.973 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 305223, 2024-03-21T11:50:05.973 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T11:50:05.973 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T11:50:05.973 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T11:50:05.973 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 231080, 2024-03-21T11:50:05.974 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 64314, 2024-03-21T11:50:05.974 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 26417, 2024-03-21T11:50:05.974 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 295525 2024-03-21T11:50:05.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.974 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-03-21T11:50:05.974 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 645, 2024-03-21T11:50:05.974 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 388, 2024-03-21T11:50:05.974 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 215, 2024-03-21T11:50:05.974 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 385, 2024-03-21T11:50:05.974 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 2093, 2024-03-21T11:50:05.974 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 16, 2024-03-21T11:50:05.974 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-03-21T11:50:05.974 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-03-21T11:50:05.974 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 80082, 2024-03-21T11:50:05.975 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 80066, 2024-03-21T11:50:05.975 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-03-21T11:50:05.975 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-03-21T11:50:05.975 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-03-21T11:50:05.975 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-03-21T11:50:05.975 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-03-21T11:50:05.975 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-03-21T11:50:05.975 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-03-21T11:50:05.975 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 46, 2024-03-21T11:50:05.975 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 9, 2024-03-21T11:50:05.975 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-03-21T11:50:05.975 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 53, 2024-03-21T11:50:05.975 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-03-21T11:50:05.975 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 378958, 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 374939, 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 374939, 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 4019, 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 3955, 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 510, 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 504, 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 504, 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 6, 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 1, 2024-03-21T11:50:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 6, 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 800910891, 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 808731714, 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 0, 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 118837, 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1734.924029930, 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.014599190 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 0 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 47, 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 80375, 2024-03-21T11:50:05.977 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 80328, 2024-03-21T11:50:05.978 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 42, 2024-03-21T11:50:05.978 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 505, 2024-03-21T11:50:05.978 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 463, 2024-03-21T11:50:05.978 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 853, 2024-03-21T11:50:05.978 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 185418, 2024-03-21T11:50:05.978 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 184565, 2024-03-21T11:50:05.978 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 1, 2024-03-21T11:50:05.978 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 86378, 2024-03-21T11:50:05.978 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 86377, 2024-03-21T11:50:05.978 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 79280, 2024-03-21T11:50:05.978 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 231804 2024-03-21T11:50:05.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.978 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 663239, 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 3169, 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 502592, 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 10613, 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 7874, 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 80141, 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 436.252404378, 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.005443560 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 5894, 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 180.973750492, 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.030704742 2024-03-21T11:50:05.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.980 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-03-21T11:50:05.980 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.980 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.980 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.980 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-03-21T11:50:05.980 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.980 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.980 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.980 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-03-21T11:50:05.980 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 167686, 2024-03-21T11:50:05.980 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 556.141301000, 2024-03-21T11:50:05.980 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.003316563 2024-03-21T11:50:05.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.981 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-03-21T11:50:05.981 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.981 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.981 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.981 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-03-21T11:50:05.981 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.981 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.981 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.982 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-03-21T11:50:05.982 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.982 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.982 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.982 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-03-21T11:50:05.982 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.982 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.982 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.982 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-03-21T11:50:05.982 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.983 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.983 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.983 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-03-21T11:50:05.983 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.984 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.984 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.984 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-03-21T11:50:05.984 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 94, 2024-03-21T11:50:05.984 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.510519562, 2024-03-21T11:50:05.984 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.005431059 2024-03-21T11:50:05.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.985 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-03-21T11:50:05.985 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.985 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.985 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.986 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-03-21T11:50:05.986 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.986 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.986 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.986 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-03-21T11:50:05.986 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 9, 2024-03-21T11:50:05.986 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.010498694, 2024-03-21T11:50:05.986 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001166521 2024-03-21T11:50:05.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 119643, 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1467.344987346, 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.012264361 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 17204, 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 115.217663773, 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.006697143 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.988 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-03-21T11:50:05.988 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.988 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.988 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.988 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-03-21T11:50:05.988 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 77, 2024-03-21T11:50:05.988 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1.051957727, 2024-03-21T11:50:05.988 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.013661788 2024-03-21T11:50:05.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.988 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-03-21T11:50:05.988 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.988 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.988 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 210, 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 7.001792006, 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.033341866 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.989 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-03-21T11:50:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2600, 2024-03-21T11:50:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 8.055532066, 2024-03-21T11:50:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.003098281 2024-03-21T11:50:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-03-21T11:50:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-03-21T11:50:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 9, 2024-03-21T11:50:05.990 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.020643489, 2024-03-21T11:50:05.991 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.002293721 2024-03-21T11:50:05.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.991 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-03-21T11:50:05.991 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.991 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.991 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.991 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-03-21T11:50:05.991 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 80016, 2024-03-21T11:50:05.991 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 338.484966947, 2024-03-21T11:50:05.991 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.004230216 2024-03-21T11:50:05.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.991 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-03-21T11:50:05.991 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-03-21T11:50:05.992 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-03-21T11:50:05.992 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:05.992 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:05.992 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:05.992 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:05.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.992 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-03-21T11:50:05.992 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 1, 2024-03-21T11:50:05.992 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 4, 2024-03-21T11:50:05.992 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 3, 2024-03-21T11:50:05.992 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 1, 2024-03-21T11:50:05.992 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-03-21T11:50:05.992 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 28211, 2024-03-21T11:50:05.992 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 28211, 2024-03-21T11:50:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 1585, 2024-03-21T11:50:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-03-21T11:50:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-21T11:50:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-21T11:50:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-21T11:50:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T11:50:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-21T11:50:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T11:50:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-21T11:50:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T11:50:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T11:50:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T11:50:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T11:50:05.994 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T11:50:05.994 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-21T11:50:05.994 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T11:50:05.994 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T11:50:05.994 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-21T11:50:05.994 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-21T11:50:05.994 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-21T11:50:05.994 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-21T11:50:05.994 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T11:50:05.994 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T11:50:05.994 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T11:50:05.994 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T11:50:05.994 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 288137, 2024-03-21T11:50:05.995 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 114, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 8888, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 170, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-21T11:50:05.996 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 803468, 2024-03-21T11:50:05.997 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 10678, 2024-03-21T11:50:05.997 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-21T11:50:05.997 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-21T11:50:05.997 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-21T11:50:05.997 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-21T11:50:05.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.997 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-03-21T11:50:05.997 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-03-21T11:50:05.997 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-03-21T11:50:05.997 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 379406, 2024-03-21T11:50:05.997 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 876058529, 2024-03-21T11:50:05.997 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 32, 2024-03-21T11:50:05.997 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 379406, 2024-03-21T11:50:05.997 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-03-21T11:50:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 379406, 2024-03-21T11:50:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 22544.352473262, 2024-03-21T11:50:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.059420126 2024-03-21T11:50:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-03-21T11:50:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-03-21T11:50:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 379406, 2024-03-21T11:50:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 510293 2024-03-21T11:50:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 379406, 2024-03-21T11:50:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 13220, 2024-03-21T11:50:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 366186, 2024-03-21T11:50:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-03-21T11:50:05.998 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 5449, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 75229, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 2131, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 120921, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 1458, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 21, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 161157, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 11089, 2024-03-21T11:50:05.999 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 114142, 2024-03-21T11:50:06.000 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-03-21T11:50:06.000 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-03-21T11:50:06.000 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-03-21T11:50:06.000 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-03-21T11:50:06.000 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-03-21T11:50:06.000 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-03-21T11:50:06.000 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T11:50:06.000 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-03-21T11:50:06.000 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-03-21T11:50:06.000 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-03-21T11:50:06.000 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-03-21T11:50:06.000 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-03-21T11:50:06.000 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-03-21T11:50:06.001 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-03-21T11:50:06.001 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-03-21T11:50:06.001 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-03-21T11:50:06.001 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-03-21T11:50:06.001 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-03-21T11:50:06.001 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-03-21T11:50:06.001 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-03-21T11:50:06.001 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-03-21T11:50:06.001 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-03-21T11:50:06.002 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-03-21T11:50:06.002 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-03-21T11:50:06.002 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-03-21T11:50:06.002 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-03-21T11:50:06.002 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 96, 2024-03-21T11:50:06.002 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-03-21T11:50:06.002 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 11, 2024-03-21T11:50:06.002 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 12, 2024-03-21T11:50:06.002 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 12, 2024-03-21T11:50:06.003 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 0, 2024-03-21T11:50:06.003 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-03-21T11:50:06.003 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 13929, 2024-03-21T11:50:06.003 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 10, 2024-03-21T11:50:06.003 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 4757 2024-03-21T11:50:06.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:06.003 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-03-21T11:50:06.003 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 1, 2024-03-21T11:50:06.003 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-03-21T11:50:06.004 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-03-21T11:50:06.004 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-03-21T11:50:06.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:06.004 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-03-21T11:50:06.004 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-03-21T11:50:06.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:06.004 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-03-21T11:50:06.004 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-03-21T11:50:06.005 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 10, 2024-03-21T11:50:06.005 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-03-21T11:50:06.005 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 9, 2024-03-21T11:50:06.005 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 160082, 2024-03-21T11:50:06.005 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 80066, 2024-03-21T11:50:06.005 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-03-21T11:50:06.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:06.005 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T11:50:06.005 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:06.006 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T11:50:06.006 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:06.006 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 1913896, 2024-03-21T11:50:06.006 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 349610690, 2024-03-21T11:50:06.006 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:06.006 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 1913896, 2024-03-21T11:50:06.006 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T11:50:06.006 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T11:50:06.007 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1913896, 2024-03-21T11:50:06.007 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 349610690, 2024-03-21T11:50:06.007 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:06.007 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:06.007 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:06.007 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:06.007 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:06.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:06.007 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-03-21T11:50:06.008 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:06.008 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T11:50:06.008 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:06.008 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T11:50:06.008 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T11:50:06.008 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:06.008 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:06.008 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 379406, 2024-03-21T11:50:06.008 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 884785255, 2024-03-21T11:50:06.009 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 379406, 2024-03-21T11:50:06.009 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 884785255, 2024-03-21T11:50:06.009 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:06.009 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:06.009 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:06.009 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:06.009 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:06.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:06.010 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-03-21T11:50:06.010 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:06.010 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-03-21T11:50:06.010 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:06.010 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T11:50:06.010 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T11:50:06.010 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:06.010 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:06.010 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 379406, 2024-03-21T11:50:06.011 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 379406, 2024-03-21T11:50:06.011 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 379406, 2024-03-21T11:50:06.011 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 379406, 2024-03-21T11:50:06.011 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:06.011 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:06.011 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:06.011 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:06.011 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:06.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:06.012 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-03-21T11:50:06.012 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:06.012 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T11:50:06.012 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:06.012 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 80066, 2024-03-21T11:50:06.012 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 8726794, 2024-03-21T11:50:06.012 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:06.012 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 80066, 2024-03-21T11:50:06.013 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T11:50:06.013 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T11:50:06.013 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 2084, 2024-03-21T11:50:06.013 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 8726794, 2024-03-21T11:50:06.013 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:06.013 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:06.013 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:06.013 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:06.013 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:06.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:06.014 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x55655f282500": { 2024-03-21T11:50:06.014 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:06.014 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T11:50:06.014 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:06.014 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 378958, 2024-03-21T11:50:06.014 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 804537410, 2024-03-21T11:50:06.014 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:06.014 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 378958, 2024-03-21T11:50:06.015 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T11:50:06.015 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T11:50:06.015 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 118837, 2024-03-21T11:50:06.015 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 804537410, 2024-03-21T11:50:06.015 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:06.015 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:06.015 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:06.015 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:06.015 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:06.015 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:06.015 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T11:50:06.668 INFO:tasks.check_counter:Daemon mds.b mds.exported=9 2024-03-21T11:50:06.668 INFO:tasks.check_counter:Daemon mds.b mds.imported=9 2024-03-21T11:50:06.669 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=645 2024-03-21T11:50:06.669 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=388 2024-03-21T11:50:06.669 DEBUG:tasks.check_counter:Getting stats from e 2024-03-21T11:50:06.669 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-21T11:50:06.966 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T11:50:07.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:06 smithi102 ceph-mon[36748]: pgmap v1204: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 166 KiB/s wr, 23 op/s 2024-03-21T11:50:07.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:06 smithi047 ceph-mon[31351]: pgmap v1204: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 166 KiB/s wr, 23 op/s 2024-03-21T11:50:07.391 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:06 smithi100 ceph-mon[36245]: pgmap v1204: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 166 KiB/s wr, 23 op/s 2024-03-21T11:50:09.117 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:08 smithi100 ceph-mon[36245]: pgmap v1205: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-03-21T11:50:09.118 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T11:50:09.118 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T11:50:09.118 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 361, 2024-03-21T11:50:09.118 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 363, 2024-03-21T11:50:09.118 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 21084797, 2024-03-21T11:50:09.118 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 114817, 2024-03-21T11:50:09.118 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 7, 2024-03-21T11:50:09.118 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 3, 2024-03-21T11:50:09.118 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.044500453, 2024-03-21T11:50:09.118 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.009524707, 2024-03-21T11:50:09.118 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.021067982, 2024-03-21T11:50:09.119 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.010655743, 2024-03-21T11:50:09.119 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:09.119 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 361, 2024-03-21T11:50:09.119 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.048209556, 2024-03-21T11:50:09.119 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000133544 2024-03-21T11:50:09.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.119 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:09.119 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.119 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.119 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.119 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T11:50:09.119 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T11:50:09.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.120 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T11:50:09.120 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 468, 2024-03-21T11:50:09.120 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 464, 2024-03-21T11:50:09.120 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 64935650, 2024-03-21T11:50:09.120 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 166865, 2024-03-21T11:50:09.120 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 5, 2024-03-21T11:50:09.120 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-03-21T11:50:09.120 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.083652032, 2024-03-21T11:50:09.120 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.010612152, 2024-03-21T11:50:09.120 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.070582403, 2024-03-21T11:50:09.120 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.013075511, 2024-03-21T11:50:09.120 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:09.120 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 464, 2024-03-21T11:50:09.120 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.037145709, 2024-03-21T11:50:09.121 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000080055 2024-03-21T11:50:09.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.121 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:09.121 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.121 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.121 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.121 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 193408, 2024-03-21T11:50:09.121 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 45968 2024-03-21T11:50:09.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.121 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T11:50:09.121 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 288, 2024-03-21T11:50:09.121 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 344, 2024-03-21T11:50:09.121 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 45442551, 2024-03-21T11:50:09.122 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 274371, 2024-03-21T11:50:09.122 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 6, 2024-03-21T11:50:09.122 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 5, 2024-03-21T11:50:09.122 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.057960565, 2024-03-21T11:50:09.122 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.006964577, 2024-03-21T11:50:09.122 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.040642090, 2024-03-21T11:50:09.122 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.008109101, 2024-03-21T11:50:09.122 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:09.122 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 342, 2024-03-21T11:50:09.122 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.033013671, 2024-03-21T11:50:09.122 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000096531 2024-03-21T11:50:09.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.122 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:09.122 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.123 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.123 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.123 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.123 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-21T11:50:09.123 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 183104 2024-03-21T11:50:09.123 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.123 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-21T11:50:09.123 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-03-21T11:50:09.123 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-21T11:50:09.123 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.123 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-03-21T11:50:09.123 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T11:50:09.123 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T11:50:09.123 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1730, 2024-03-21T11:50:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.163932564, 2024-03-21T11:50:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000094758 2024-03-21T11:50:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-03-21T11:50:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T11:50:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T11:50:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-03-21T11:50:09.124 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 0, 2024-03-21T11:50:09.125 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 0, 2024-03-21T11:50:09.125 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-03-21T11:50:09.125 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.125 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.125 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.125 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.125 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-03-21T11:50:09.125 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 0, 2024-03-21T11:50:09.125 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-03-21T11:50:09.126 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-03-21T11:50:09.126 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 0, 2024-03-21T11:50:09.126 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-03-21T11:50:09.126 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-03-21T11:50:09.126 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 12854, 2024-03-21T11:50:09.126 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 8208, 2024-03-21T11:50:09.126 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 4646, 2024-03-21T11:50:09.126 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-03-21T11:50:09.126 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 1355, 2024-03-21T11:50:09.126 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 0, 2024-03-21T11:50:09.126 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 0, 2024-03-21T11:50:09.126 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 0, 2024-03-21T11:50:09.126 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 2, 2024-03-21T11:50:09.127 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 0, 2024-03-21T11:50:09.127 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 0, 2024-03-21T11:50:09.127 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 0, 2024-03-21T11:50:09.127 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 0, 2024-03-21T11:50:09.127 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-03-21T11:50:09.127 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-03-21T11:50:09.127 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-03-21T11:50:09.127 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-03-21T11:50:09.127 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-03-21T11:50:09.127 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-03-21T11:50:09.127 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-03-21T11:50:09.127 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-03-21T11:50:09.127 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-03-21T11:50:09.127 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 0, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 4, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 234, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 0, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 0, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 0, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-03-21T11:50:09.128 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T11:50:09.129 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 0, 2024-03-21T11:50:09.129 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T11:50:09.129 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T11:50:09.129 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T11:50:09.129 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 0, 2024-03-21T11:50:09.129 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 0, 2024-03-21T11:50:09.129 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 0, 2024-03-21T11:50:09.129 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 0 2024-03-21T11:50:09.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.129 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-03-21T11:50:09.130 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-03-21T11:50:09.130 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-03-21T11:50:09.130 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-03-21T11:50:09.130 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 0, 2024-03-21T11:50:09.130 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 0, 2024-03-21T11:50:09.130 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 0, 2024-03-21T11:50:09.130 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-03-21T11:50:09.130 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-03-21T11:50:09.130 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 0, 2024-03-21T11:50:09.130 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 0, 2024-03-21T11:50:09.130 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-03-21T11:50:09.130 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-03-21T11:50:09.130 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-03-21T11:50:09.130 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-03-21T11:50:09.131 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-03-21T11:50:09.131 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-03-21T11:50:09.131 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-03-21T11:50:09.131 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T11:50:09.131 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-03-21T11:50:09.131 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-03-21T11:50:09.131 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-03-21T11:50:09.131 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-03-21T11:50:09.131 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-03-21T11:50:09.131 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.131 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-03-21T11:50:09.131 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-03-21T11:50:09.131 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 0, 2024-03-21T11:50:09.131 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 62068, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 0, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 6, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 16, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 800910891, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 808731714, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 808729217, 2024-03-21T11:50:09.132 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-03-21T11:50:09.133 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.133 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.133 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.133 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.133 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 62068 2024-03-21T11:50:09.133 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.133 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-03-21T11:50:09.134 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 33, 2024-03-21T11:50:09.134 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 12656, 2024-03-21T11:50:09.134 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 12623, 2024-03-21T11:50:09.134 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 32, 2024-03-21T11:50:09.134 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 103, 2024-03-21T11:50:09.134 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 71, 2024-03-21T11:50:09.134 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 12854, 2024-03-21T11:50:09.134 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 13476, 2024-03-21T11:50:09.134 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 622, 2024-03-21T11:50:09.134 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 0, 2024-03-21T11:50:09.134 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 0, 2024-03-21T11:50:09.134 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 0, 2024-03-21T11:50:09.134 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 64372, 2024-03-21T11:50:09.134 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 215420 2024-03-21T11:50:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-03-21T11:50:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 0, 2024-03-21T11:50:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 0, 2024-03-21T11:50:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 0, 2024-03-21T11:50:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 0, 2024-03-21T11:50:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 0, 2024-03-21T11:50:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-03-21T11:50:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-03-21T11:50:09.135 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-03-21T11:50:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.136 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-03-21T11:50:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-03-21T11:50:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-03-21T11:50:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.137 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.138 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.138 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-03-21T11:50:09.138 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.138 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.138 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.138 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.138 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-03-21T11:50:09.138 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.138 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.138 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.138 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.138 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-03-21T11:50:09.138 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.138 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.139 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.139 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-03-21T11:50:09.139 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.139 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.139 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.139 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-03-21T11:50:09.139 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.139 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.139 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.140 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.140 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-03-21T11:50:09.140 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.140 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.141 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.141 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.141 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-03-21T11:50:09.141 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.141 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.141 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.141 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.141 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-03-21T11:50:09.141 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.141 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.141 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.141 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.141 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-03-21T11:50:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-03-21T11:50:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-03-21T11:50:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.142 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.143 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-03-21T11:50:09.143 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.143 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.143 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.143 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.143 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-03-21T11:50:09.143 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.143 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.143 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.143 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.143 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-03-21T11:50:09.144 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.144 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.144 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.144 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.144 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-03-21T11:50:09.144 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.144 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.144 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.144 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.144 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-03-21T11:50:09.144 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.144 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.144 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.144 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.146 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-03-21T11:50:09.146 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.146 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.146 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.146 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.146 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-03-21T11:50:09.146 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.146 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.147 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.147 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.147 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-03-21T11:50:09.147 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.147 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.147 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.147 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.147 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-03-21T11:50:09.147 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.147 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.147 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.147 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.147 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-03-21T11:50:09.147 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-03-21T11:50:09.148 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-03-21T11:50:09.148 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.148 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.148 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.148 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:09.148 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.148 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-03-21T11:50:09.148 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 1, 2024-03-21T11:50:09.148 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 1, 2024-03-21T11:50:09.148 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 0, 2024-03-21T11:50:09.148 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 0, 2024-03-21T11:50:09.148 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-03-21T11:50:09.148 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 0, 2024-03-21T11:50:09.148 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 0, 2024-03-21T11:50:09.149 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 3148, 2024-03-21T11:50:09.149 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-03-21T11:50:09.149 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.149 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-21T11:50:09.149 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-21T11:50:09.149 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-21T11:50:09.149 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T11:50:09.149 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-21T11:50:09.149 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T11:50:09.149 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-21T11:50:09.149 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T11:50:09.149 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T11:50:09.149 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T11:50:09.149 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T11:50:09.150 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T11:50:09.150 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-21T11:50:09.150 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T11:50:09.150 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T11:50:09.150 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-21T11:50:09.150 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-21T11:50:09.150 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-21T11:50:09.150 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-21T11:50:09.150 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T11:50:09.150 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T11:50:09.150 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T11:50:09.150 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T11:50:09.150 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T11:50:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-21T11:50:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-21T11:50:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-21T11:50:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T11:50:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T11:50:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-21T11:50:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-21T11:50:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-21T11:50:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-21T11:50:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T11:50:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-21T11:50:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T11:50:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-21T11:50:09.151 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 222151, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 80, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 8888, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 170, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-21T11:50:09.152 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-21T11:50:09.153 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 9165855, 2024-03-21T11:50:09.153 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 53358, 2024-03-21T11:50:09.153 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-21T11:50:09.153 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-21T11:50:09.153 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-21T11:50:09.153 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-21T11:50:09.153 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.153 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-03-21T11:50:09.153 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-03-21T11:50:09.153 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-03-21T11:50:09.153 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 1028, 2024-03-21T11:50:09.153 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 26, 2024-03-21T11:50:09.153 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 4, 2024-03-21T11:50:09.153 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 1028, 2024-03-21T11:50:09.154 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-03-21T11:50:09.154 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1028, 2024-03-21T11:50:09.154 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 14.007655265, 2024-03-21T11:50:09.154 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.013626123 2024-03-21T11:50:09.154 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.154 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-03-21T11:50:09.154 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-03-21T11:50:09.154 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1028, 2024-03-21T11:50:09.154 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1029 2024-03-21T11:50:09.154 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.154 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 1028, 2024-03-21T11:50:09.154 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 1028, 2024-03-21T11:50:09.154 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 0, 2024-03-21T11:50:09.154 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-03-21T11:50:09.155 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-03-21T11:50:09.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 530, 2024-03-21T11:50:09.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 0, 2024-03-21T11:50:09.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 495, 2024-03-21T11:50:09.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 0, 2024-03-21T11:50:09.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 0, 2024-03-21T11:50:09.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-03-21T11:50:09.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-03-21T11:50:09.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-03-21T11:50:09.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-03-21T11:50:09.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 0, 2024-03-21T11:50:09.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-03-21T11:50:09.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-03-21T11:50:09.155 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-03-21T11:50:09.156 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 2, 2024-03-21T11:50:09.156 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 0, 2024-03-21T11:50:09.156 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-03-21T11:50:09.156 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-03-21T11:50:09.156 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-03-21T11:50:09.156 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-03-21T11:50:09.156 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-03-21T11:50:09.156 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-03-21T11:50:09.156 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T11:50:09.156 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-03-21T11:50:09.156 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-03-21T11:50:09.156 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-03-21T11:50:09.157 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-03-21T11:50:09.157 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-03-21T11:50:09.157 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-03-21T11:50:09.157 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-03-21T11:50:09.157 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-03-21T11:50:09.157 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-03-21T11:50:09.157 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-03-21T11:50:09.157 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-03-21T11:50:09.157 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-03-21T11:50:09.157 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-03-21T11:50:09.157 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 0, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 0, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 9, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 9, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 0, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 0, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 2, 2024-03-21T11:50:09.158 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 0 2024-03-21T11:50:09.159 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.159 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-03-21T11:50:09.159 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 0, 2024-03-21T11:50:09.159 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-03-21T11:50:09.159 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-03-21T11:50:09.159 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-03-21T11:50:09.159 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.159 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-03-21T11:50:09.159 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-03-21T11:50:09.159 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.159 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-03-21T11:50:09.159 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-03-21T11:50:09.159 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T11:50:09.159 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-03-21T11:50:09.160 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 0, 2024-03-21T11:50:09.160 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 0, 2024-03-21T11:50:09.160 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 0, 2024-03-21T11:50:09.160 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-03-21T11:50:09.160 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.160 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T11:50:09.160 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:09.160 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T11:50:09.160 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:09.160 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 1113, 2024-03-21T11:50:09.160 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 131355479, 2024-03-21T11:50:09.160 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:09.160 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 1113, 2024-03-21T11:50:09.160 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1113, 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 131355479, 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T11:50:09.161 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T11:50:09.162 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:09.162 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:09.162 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 1028, 2024-03-21T11:50:09.162 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 130981416, 2024-03-21T11:50:09.162 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1028, 2024-03-21T11:50:09.162 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 130981416, 2024-03-21T11:50:09.162 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:09.162 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.162 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.162 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.162 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:09.162 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.162 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 1028, 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 1028, 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1028, 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1028, 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.163 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-03-21T11:50:09.164 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x55dfdaba6640": { 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T11:50:09.165 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T11:50:09.166 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-03-21T11:50:09.166 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-03-21T11:50:09.166 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:09.166 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:09.166 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:09.166 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:09.166 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:09.166 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:09.166 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T11:50:09.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:08 smithi102 ceph-mon[36748]: pgmap v1205: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-03-21T11:50:09.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:08 smithi047 ceph-mon[31351]: pgmap v1205: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-03-21T11:50:10.034 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-21T11:50:10.035 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-21T11:50:10.035 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-03-21T11:50:10.035 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-03-21T11:50:10.035 DEBUG:tasks.check_counter:Getting stats from h 2024-03-21T11:50:10.035 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-21T11:50:10.296 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T11:50:11.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:10 smithi102 ceph-mon[36748]: pgmap v1206: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-03-21T11:50:11.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:10 smithi047 ceph-mon[31351]: pgmap v1206: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-03-21T11:50:11.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:10 smithi100 ceph-mon[36245]: pgmap v1206: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-03-21T11:50:12.069 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T11:50:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T11:50:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 225276, 2024-03-21T11:50:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 284295, 2024-03-21T11:50:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 70355741, 2024-03-21T11:50:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 226053186, 2024-03-21T11:50:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 15, 2024-03-21T11:50:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-03-21T11:50:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 14.174265392, 2024-03-21T11:50:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 5.653693074, 2024-03-21T11:50:12.070 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 7.915971668, 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 1.565315776, 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 284294, 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 29.309736353, 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000103096 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 225410, 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.072488490, 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000321 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 972016, 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 315856 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.071 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 117964, 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 144598, 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 26860065, 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 106082853, 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 62, 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 7, 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 7.957459758, 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 2.945149169, 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 4.268317873, 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 1.130276610, 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 144595, 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 18.305383356, 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000126597 2024-03-21T11:50:12.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 211405, 2024-03-21T11:50:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.037479065, 2024-03-21T11:50:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000177 2024-03-21T11:50:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-21T11:50:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 872128 2024-03-21T11:50:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T11:50:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 450142, 2024-03-21T11:50:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 314670, 2024-03-21T11:50:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 116277498, 2024-03-21T11:50:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 104590951, 2024-03-21T11:50:12.073 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 52, 2024-03-21T11:50:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 6, 2024-03-21T11:50:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 25.181539903, 2024-03-21T11:50:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 10.172466566, 2024-03-21T11:50:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 17.519711823, 2024-03-21T11:50:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.851043471, 2024-03-21T11:50:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 314669, 2024-03-21T11:50:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 18.809785438, 2024-03-21T11:50:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000059776 2024-03-21T11:50:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 718163, 2024-03-21T11:50:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.126609711, 2024-03-21T11:50:12.074 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000176 2024-03-21T11:50:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-21T11:50:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 448 2024-03-21T11:50:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-21T11:50:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-03-21T11:50:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-21T11:50:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-03-21T11:50:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T11:50:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T11:50:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 51053, 2024-03-21T11:50:12.075 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 46.743784638, 2024-03-21T11:50:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000915593 2024-03-21T11:50:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-03-21T11:50:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T11:50:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T11:50:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 30085, 2024-03-21T11:50:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 3.063824315, 2024-03-21T11:50:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000101838 2024-03-21T11:50:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-03-21T11:50:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 155756, 2024-03-21T11:50:12.076 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 98383, 2024-03-21T11:50:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-03-21T11:50:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 98383, 2024-03-21T11:50:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 488.855386626, 2024-03-21T11:50:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.004968900 2024-03-21T11:50:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-03-21T11:50:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 57373, 2024-03-21T11:50:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-03-21T11:50:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-03-21T11:50:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 11, 2024-03-21T11:50:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-03-21T11:50:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-03-21T11:50:12.077 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 27, 2024-03-21T11:50:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 8, 2024-03-21T11:50:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 7, 2024-03-21T11:50:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 12, 2024-03-21T11:50:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 15, 2024-03-21T11:50:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 215, 2024-03-21T11:50:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 1, 2024-03-21T11:50:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 1, 2024-03-21T11:50:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 3, 2024-03-21T11:50:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 230030, 2024-03-21T11:50:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 101184, 2024-03-21T11:50:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 57371, 2024-03-21T11:50:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 107, 2024-03-21T11:50:12.078 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 2206, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 3, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 6, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 3, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 35, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 1726, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 674, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 212105939, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 0, 2024-03-21T11:50:12.079 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 1726, 2024-03-21T11:50:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-03-21T11:50:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T11:50:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-03-21T11:50:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 210, 2024-03-21T11:50:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T11:50:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 210, 2024-03-21T11:50:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 1516, 2024-03-21T11:50:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 10, 2024-03-21T11:50:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 10117, 2024-03-21T11:50:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 44776, 2024-03-21T11:50:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T11:50:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T11:50:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T11:50:12.080 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 33781, 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 11874, 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 5646, 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 72686 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 419, 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 1255, 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 676, 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 1181, 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 477, 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 2, 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 15292, 2024-03-21T11:50:12.081 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 15290, 2024-03-21T11:50:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-03-21T11:50:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-03-21T11:50:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-03-21T11:50:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-03-21T11:50:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-03-21T11:50:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-03-21T11:50:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-03-21T11:50:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T11:50:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 3, 2024-03-21T11:50:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-03-21T11:50:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-03-21T11:50:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-03-21T11:50:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-03-21T11:50:12.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 72696, 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 72696, 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 95, 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 95, 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 7, 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-03-21T11:50:12.083 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-03-21T11:50:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 4194304, 2024-03-21T11:50:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 151545599, 2024-03-21T11:50:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 0, 2024-03-21T11:50:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-03-21T11:50:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 25262, 2024-03-21T11:50:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 316.508800702, 2024-03-21T11:50:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.012529047 2024-03-21T11:50:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 0 2024-03-21T11:50:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-03-21T11:50:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 23, 2024-03-21T11:50:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 15422, 2024-03-21T11:50:12.084 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 15399, 2024-03-21T11:50:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 20, 2024-03-21T11:50:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 628, 2024-03-21T11:50:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 608, 2024-03-21T11:50:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 27, 2024-03-21T11:50:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 27373, 2024-03-21T11:50:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 27346, 2024-03-21T11:50:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 1, 2024-03-21T11:50:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 16742, 2024-03-21T11:50:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 16741, 2024-03-21T11:50:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 65244, 2024-03-21T11:50:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 223612 2024-03-21T11:50:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-03-21T11:50:12.085 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 203276, 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 14, 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 155756, 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 2290, 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 1987, 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 15257, 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 135.118924306, 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.008856192 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 7647, 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 100.441703221, 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.013134785 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.086 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 30076, 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 43.204567227, 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001436513 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.087 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-03-21T11:50:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-03-21T11:50:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1, 2024-03-21T11:50:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000129277, 2024-03-21T11:50:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000129277 2024-03-21T11:50:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-03-21T11:50:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-03-21T11:50:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-03-21T11:50:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.089 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-03-21T11:50:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-03-21T11:50:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 5, 2024-03-21T11:50:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.012630769, 2024-03-21T11:50:12.090 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.002526153 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.091 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-03-21T11:50:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 25942, 2024-03-21T11:50:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 129.182418744, 2024-03-21T11:50:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.004979663 2024-03-21T11:50:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-03-21T11:50:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 3357, 2024-03-21T11:50:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 30.533985121, 2024-03-21T11:50:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.009095616 2024-03-21T11:50:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-03-21T11:50:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.092 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-03-21T11:50:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-03-21T11:50:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 10, 2024-03-21T11:50:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.076246800, 2024-03-21T11:50:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.007624680 2024-03-21T11:50:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-03-21T11:50:12.093 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-03-21T11:50:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-03-21T11:50:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 14, 2024-03-21T11:50:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.211219706, 2024-03-21T11:50:12.094 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.015087121 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 792, 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.897784157, 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001133565 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.095 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-03-21T11:50:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-03-21T11:50:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-03-21T11:50:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 15282, 2024-03-21T11:50:12.096 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 49.175777298, 2024-03-21T11:50:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.003217888 2024-03-21T11:50:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-03-21T11:50:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-03-21T11:50:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-03-21T11:50:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-03-21T11:50:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 1, 2024-03-21T11:50:12.097 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 2, 2024-03-21T11:50:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 1, 2024-03-21T11:50:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 1, 2024-03-21T11:50:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-03-21T11:50:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 10872, 2024-03-21T11:50:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 10872, 2024-03-21T11:50:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 1578, 2024-03-21T11:50:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-03-21T11:50:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-21T11:50:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-21T11:50:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-21T11:50:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T11:50:12.098 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-21T11:50:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T11:50:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-21T11:50:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T11:50:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T11:50:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T11:50:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T11:50:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T11:50:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-21T11:50:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T11:50:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T11:50:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-21T11:50:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-21T11:50:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-21T11:50:12.099 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-21T11:50:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T11:50:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T11:50:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T11:50:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T11:50:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T11:50:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-21T11:50:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-21T11:50:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-21T11:50:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T11:50:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T11:50:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-21T11:50:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-21T11:50:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-21T11:50:12.100 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-21T11:50:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T11:50:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-21T11:50:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T11:50:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-21T11:50:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 300620, 2024-03-21T11:50:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 127, 2024-03-21T11:50:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 176, 2024-03-21T11:50:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 2, 2024-03-21T11:50:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-21T11:50:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-21T11:50:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-21T11:50:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-21T11:50:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-21T11:50:12.101 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-21T11:50:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 8888, 2024-03-21T11:50:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 170, 2024-03-21T11:50:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T11:50:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-21T11:50:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-21T11:50:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-21T11:50:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 98049, 2024-03-21T11:50:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 802, 2024-03-21T11:50:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-21T11:50:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-21T11:50:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-21T11:50:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-21T11:50:12.102 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-03-21T11:50:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-03-21T11:50:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-03-21T11:50:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 60176, 2024-03-21T11:50:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 150664593, 2024-03-21T11:50:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 33, 2024-03-21T11:50:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 60176, 2024-03-21T11:50:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-03-21T11:50:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 60176, 2024-03-21T11:50:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1196.217372736, 2024-03-21T11:50:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.019878645 2024-03-21T11:50:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-03-21T11:50:12.103 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 60176, 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 61372 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 60176, 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 2435, 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 57741, 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 0, 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 10, 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 709, 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 25971, 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 397, 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-03-21T11:50:12.104 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-03-21T11:50:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-03-21T11:50:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-03-21T11:50:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 30619, 2024-03-21T11:50:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-03-21T11:50:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-03-21T11:50:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-03-21T11:50:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 1726, 2024-03-21T11:50:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 20, 2024-03-21T11:50:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-03-21T11:50:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-03-21T11:50:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-03-21T11:50:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-03-21T11:50:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-03-21T11:50:12.105 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-03-21T11:50:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T11:50:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-03-21T11:50:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-03-21T11:50:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-03-21T11:50:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-03-21T11:50:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-03-21T11:50:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-03-21T11:50:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-03-21T11:50:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-03-21T11:50:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-03-21T11:50:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-03-21T11:50:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-03-21T11:50:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-03-21T11:50:12.106 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 96, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 5, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 12, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 15, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 3, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 1461, 2024-03-21T11:50:12.107 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 0, 2024-03-21T11:50:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 459 2024-03-21T11:50:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-03-21T11:50:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 1, 2024-03-21T11:50:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-03-21T11:50:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-03-21T11:50:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-03-21T11:50:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-03-21T11:50:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-03-21T11:50:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-03-21T11:50:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-03-21T11:50:12.108 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 10, 2024-03-21T11:50:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-03-21T11:50:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 6, 2024-03-21T11:50:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 30572, 2024-03-21T11:50:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 15290, 2024-03-21T11:50:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-03-21T11:50:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T11:50:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T11:50:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 793378, 2024-03-21T11:50:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 145304308, 2024-03-21T11:50:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:12.109 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 793378, 2024-03-21T11:50:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T11:50:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T11:50:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 793378, 2024-03-21T11:50:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 145304308, 2024-03-21T11:50:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-03-21T11:50:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T11:50:12.110 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T11:50:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T11:50:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 60176, 2024-03-21T11:50:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 152331139, 2024-03-21T11:50:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 60176, 2024-03-21T11:50:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 152331139, 2024-03-21T11:50:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:12.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-03-21T11:50:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-03-21T11:50:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T11:50:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T11:50:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 60176, 2024-03-21T11:50:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 60176, 2024-03-21T11:50:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 60176, 2024-03-21T11:50:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 60176, 2024-03-21T11:50:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:12.112 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 15290, 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1666546, 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 15290, 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T11:50:12.113 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 709, 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1666546, 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x562a78458500": { 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 72696, 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 147351295, 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:12.114 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 72696, 2024-03-21T11:50:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T11:50:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T11:50:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 25262, 2024-03-21T11:50:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 147351295, 2024-03-21T11:50:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:12.115 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:12.115 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T11:50:13.121 INFO:tasks.check_counter:Daemon mds.h mds.exported=3 2024-03-21T11:50:13.121 INFO:tasks.check_counter:Daemon mds.h mds.imported=3 2024-03-21T11:50:13.121 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=419 2024-03-21T11:50:13.122 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=1255 2024-03-21T11:50:13.122 DEBUG:tasks.check_counter:Getting stats from k 2024-03-21T11:50:13.122 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-21T11:50:13.226 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:12 smithi100 ceph-mon[36245]: pgmap v1207: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-03-21T11:50:13.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:12 smithi102 ceph-mon[36748]: pgmap v1207: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-03-21T11:50:13.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:12 smithi047 ceph-mon[31351]: pgmap v1207: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-03-21T11:50:13.425 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T11:50:15.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:14 smithi100 ceph-mon[36245]: pgmap v1208: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-03-21T11:50:15.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:14 smithi102 ceph-mon[36748]: pgmap v1208: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-03-21T11:50:15.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:14 smithi047 ceph-mon[31351]: pgmap v1208: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-03-21T11:50:15.510 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T11:50:15.510 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T11:50:15.510 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 2513, 2024-03-21T11:50:15.510 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 2492, 2024-03-21T11:50:15.510 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 22570286, 2024-03-21T11:50:15.510 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 928024, 2024-03-21T11:50:15.511 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 7, 2024-03-21T11:50:15.511 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 3, 2024-03-21T11:50:15.511 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.220982462, 2024-03-21T11:50:15.511 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.073410307, 2024-03-21T11:50:15.511 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.073359885, 2024-03-21T11:50:15.511 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.062600241, 2024-03-21T11:50:15.511 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:15.511 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2491, 2024-03-21T11:50:15.511 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.107795620, 2024-03-21T11:50:15.511 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000043274 2024-03-21T11:50:15.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.511 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:15.511 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.511 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 951024, 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 290752 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 1967, 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 2332, 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 75989594, 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 1497946, 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 10, 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 6, 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.204401479, 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.044279259, 2024-03-21T11:50:15.512 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.092223590, 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.054468027, 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2330, 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.078965235, 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000033890 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 873008 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T11:50:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 1765, 2024-03-21T11:50:15.514 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 1765, 2024-03-21T11:50:15.514 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 51342789, 2024-03-21T11:50:15.514 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 560447, 2024-03-21T11:50:15.514 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 7, 2024-03-21T11:50:15.514 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 5, 2024-03-21T11:50:15.514 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.146755564, 2024-03-21T11:50:15.514 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.030917827, 2024-03-21T11:50:15.514 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.067005540, 2024-03-21T11:50:15.514 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.039458822, 2024-03-21T11:50:15.514 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:15.514 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1764, 2024-03-21T11:50:15.514 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.056194637, 2024-03-21T11:50:15.514 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000031856 2024-03-21T11:50:15.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.515 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:15.515 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.515 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.515 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.515 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-21T11:50:15.515 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 448 2024-03-21T11:50:15.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.515 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-21T11:50:15.515 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-03-21T11:50:15.515 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-21T11:50:15.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.515 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 9701, 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.205817814, 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000021216 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:15.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 0, 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 0, 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 0, 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 0, 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-03-21T11:50:15.517 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-03-21T11:50:15.518 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 15325, 2024-03-21T11:50:15.518 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 0, 2024-03-21T11:50:15.518 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 15325, 2024-03-21T11:50:15.518 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-03-21T11:50:15.518 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 15325, 2024-03-21T11:50:15.518 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 0, 2024-03-21T11:50:15.518 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 0, 2024-03-21T11:50:15.518 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 0, 2024-03-21T11:50:15.518 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 1, 2024-03-21T11:50:15.518 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 0, 2024-03-21T11:50:15.518 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 0, 2024-03-21T11:50:15.518 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 0, 2024-03-21T11:50:15.518 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 0, 2024-03-21T11:50:15.518 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 0, 2024-03-21T11:50:15.519 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T11:50:15.520 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-03-21T11:50:15.520 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T11:50:15.520 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-03-21T11:50:15.520 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 0, 2024-03-21T11:50:15.520 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T11:50:15.520 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T11:50:15.520 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 0, 2024-03-21T11:50:15.520 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-03-21T11:50:15.520 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T11:50:15.520 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 0, 2024-03-21T11:50:15.520 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T11:50:15.520 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T11:50:15.520 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T11:50:15.520 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 0, 2024-03-21T11:50:15.521 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 0, 2024-03-21T11:50:15.521 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 0, 2024-03-21T11:50:15.521 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 0 2024-03-21T11:50:15.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.521 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-03-21T11:50:15.521 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-03-21T11:50:15.521 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-03-21T11:50:15.521 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-03-21T11:50:15.521 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 0, 2024-03-21T11:50:15.521 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 0, 2024-03-21T11:50:15.521 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 0, 2024-03-21T11:50:15.521 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-03-21T11:50:15.521 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-03-21T11:50:15.521 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 0, 2024-03-21T11:50:15.522 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 0, 2024-03-21T11:50:15.522 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-03-21T11:50:15.522 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-03-21T11:50:15.522 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-03-21T11:50:15.522 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-03-21T11:50:15.522 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-03-21T11:50:15.522 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-03-21T11:50:15.522 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-03-21T11:50:15.522 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T11:50:15.522 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-03-21T11:50:15.522 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-03-21T11:50:15.522 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-03-21T11:50:15.522 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-03-21T11:50:15.522 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 0, 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 72696, 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 0, 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 95, 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 7, 2024-03-21T11:50:15.523 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-03-21T11:50:15.525 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-03-21T11:50:15.525 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 4194304, 2024-03-21T11:50:15.525 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 151545599, 2024-03-21T11:50:15.525 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 151544461, 2024-03-21T11:50:15.525 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-03-21T11:50:15.525 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.525 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.525 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.525 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.526 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 72696 2024-03-21T11:50:15.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.526 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-03-21T11:50:15.526 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 15, 2024-03-21T11:50:15.526 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 15323, 2024-03-21T11:50:15.526 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 15308, 2024-03-21T11:50:15.526 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 13, 2024-03-21T11:50:15.526 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 37, 2024-03-21T11:50:15.526 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 24, 2024-03-21T11:50:15.526 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 15325, 2024-03-21T11:50:15.526 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 15474, 2024-03-21T11:50:15.527 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 149, 2024-03-21T11:50:15.527 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 0, 2024-03-21T11:50:15.527 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 0, 2024-03-21T11:50:15.527 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 0, 2024-03-21T11:50:15.527 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 65300, 2024-03-21T11:50:15.527 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 215420 2024-03-21T11:50:15.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.527 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-03-21T11:50:15.527 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 0, 2024-03-21T11:50:15.527 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 0, 2024-03-21T11:50:15.527 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 0, 2024-03-21T11:50:15.527 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 0, 2024-03-21T11:50:15.528 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 0, 2024-03-21T11:50:15.528 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-03-21T11:50:15.528 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.528 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.528 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.528 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.528 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-03-21T11:50:15.528 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.528 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.528 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.528 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.528 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-03-21T11:50:15.528 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.528 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.529 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.529 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-03-21T11:50:15.529 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.529 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.529 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.529 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-03-21T11:50:15.529 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.529 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.529 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.529 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-03-21T11:50:15.529 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.530 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.530 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.530 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-03-21T11:50:15.530 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.530 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.530 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.530 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-03-21T11:50:15.530 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.530 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.530 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.532 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-03-21T11:50:15.532 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.532 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.533 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.533 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-03-21T11:50:15.533 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.533 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.533 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.533 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-03-21T11:50:15.533 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.533 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.533 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.534 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-03-21T11:50:15.534 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.534 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.534 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.534 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.534 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-03-21T11:50:15.534 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.534 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.534 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.534 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.534 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-03-21T11:50:15.534 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.534 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.535 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.535 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.535 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-03-21T11:50:15.535 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.535 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.535 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.535 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.535 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-03-21T11:50:15.535 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.535 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.535 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.535 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.535 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-03-21T11:50:15.536 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.536 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.536 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.536 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.536 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-03-21T11:50:15.536 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.536 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.536 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.536 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.536 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-03-21T11:50:15.536 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.536 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.536 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.537 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-03-21T11:50:15.537 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.537 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.537 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.537 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-03-21T11:50:15.537 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.537 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.537 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.537 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-03-21T11:50:15.537 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.538 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.538 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.538 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-03-21T11:50:15.538 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.538 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.538 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.538 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-03-21T11:50:15.538 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.538 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.538 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.539 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-03-21T11:50:15.539 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.539 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.539 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.539 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.539 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-03-21T11:50:15.539 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.539 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.539 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.539 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.539 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-03-21T11:50:15.539 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-03-21T11:50:15.539 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-03-21T11:50:15.539 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.540 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.540 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.540 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:15.540 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.540 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-03-21T11:50:15.540 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 1, 2024-03-21T11:50:15.540 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 2, 2024-03-21T11:50:15.540 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 1, 2024-03-21T11:50:15.540 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 1, 2024-03-21T11:50:15.540 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-03-21T11:50:15.540 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 0, 2024-03-21T11:50:15.540 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 0, 2024-03-21T11:50:15.540 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 3152, 2024-03-21T11:50:15.540 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-21T11:50:15.541 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T11:50:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T11:50:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-21T11:50:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-21T11:50:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-21T11:50:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-21T11:50:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T11:50:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T11:50:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T11:50:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T11:50:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T11:50:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-21T11:50:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-21T11:50:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-21T11:50:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T11:50:15.543 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T11:50:15.543 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-21T11:50:15.543 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-21T11:50:15.543 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-21T11:50:15.543 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-21T11:50:15.543 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T11:50:15.543 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-21T11:50:15.543 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T11:50:15.543 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-21T11:50:15.543 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 241052, 2024-03-21T11:50:15.543 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 84, 2024-03-21T11:50:15.543 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-03-21T11:50:15.543 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-03-21T11:50:15.543 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 8888, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 170, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 10839036, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 61673, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-21T11:50:15.544 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-21T11:50:15.545 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-21T11:50:15.545 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-21T11:50:15.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.545 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-03-21T11:50:15.545 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-03-21T11:50:15.545 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-03-21T11:50:15.545 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 5730, 2024-03-21T11:50:15.545 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 21, 2024-03-21T11:50:15.545 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 3, 2024-03-21T11:50:15.545 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 5730, 2024-03-21T11:50:15.545 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-03-21T11:50:15.545 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 5730, 2024-03-21T11:50:15.545 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 35.853111805, 2024-03-21T11:50:15.545 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.006257087 2024-03-21T11:50:15.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.546 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-03-21T11:50:15.546 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-03-21T11:50:15.546 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 5730, 2024-03-21T11:50:15.546 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 5731 2024-03-21T11:50:15.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.546 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 5730, 2024-03-21T11:50:15.546 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 5730, 2024-03-21T11:50:15.546 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 0, 2024-03-21T11:50:15.546 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-03-21T11:50:15.546 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-03-21T11:50:15.546 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 3299, 2024-03-21T11:50:15.546 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 0, 2024-03-21T11:50:15.546 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 2429, 2024-03-21T11:50:15.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 0, 2024-03-21T11:50:15.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 0, 2024-03-21T11:50:15.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-03-21T11:50:15.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-03-21T11:50:15.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-03-21T11:50:15.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-03-21T11:50:15.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 0, 2024-03-21T11:50:15.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-03-21T11:50:15.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-03-21T11:50:15.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-03-21T11:50:15.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 1, 2024-03-21T11:50:15.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 0, 2024-03-21T11:50:15.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-03-21T11:50:15.547 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-03-21T11:50:15.548 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-03-21T11:50:15.548 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-03-21T11:50:15.548 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-03-21T11:50:15.548 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-03-21T11:50:15.548 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T11:50:15.548 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-03-21T11:50:15.548 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-03-21T11:50:15.548 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-03-21T11:50:15.548 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-03-21T11:50:15.548 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-03-21T11:50:15.548 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-03-21T11:50:15.548 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-03-21T11:50:15.548 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 96, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 5, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 11, 2024-03-21T11:50:15.549 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 13, 2024-03-21T11:50:15.550 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 2, 2024-03-21T11:50:15.550 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-03-21T11:50:15.550 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 0, 2024-03-21T11:50:15.550 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 2, 2024-03-21T11:50:15.550 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 0 2024-03-21T11:50:15.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.550 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-03-21T11:50:15.550 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 0, 2024-03-21T11:50:15.550 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-03-21T11:50:15.550 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-03-21T11:50:15.550 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-03-21T11:50:15.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.550 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 0, 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 0, 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 0, 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:15.551 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 6241, 2024-03-21T11:50:15.552 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 149311771, 2024-03-21T11:50:15.552 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:15.552 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 6241, 2024-03-21T11:50:15.552 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T11:50:15.552 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T11:50:15.552 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 6241, 2024-03-21T11:50:15.552 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 149311771, 2024-03-21T11:50:15.552 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:15.552 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.552 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.552 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.552 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:15.552 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.552 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 5730, 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 147353851, 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 5730, 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 147353851, 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.553 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-03-21T11:50:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-03-21T11:50:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T11:50:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T11:50:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 5730, 2024-03-21T11:50:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 5730, 2024-03-21T11:50:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 5730, 2024-03-21T11:50:15.554 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 5730, 2024-03-21T11:50:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-03-21T11:50:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T11:50:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T11:50:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T11:50:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:15.555 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T11:50:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T11:50:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-03-21T11:50:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-03-21T11:50:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T11:50:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x559b424fc6e0": { 2024-03-21T11:50:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T11:50:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T11:50:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T11:50:15.557 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T11:50:16.402 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-21T11:50:16.402 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-21T11:50:16.402 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-03-21T11:50:16.402 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-03-21T11:50:16.402 DEBUG:tasks.check_counter:Getting stats from c 2024-03-21T11:50:16.403 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-21T11:50:17.164 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:50:17.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:16 smithi102 ceph-mon[36748]: pgmap v1209: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:50:17.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:16 smithi047 ceph-mon[31351]: pgmap v1209: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:50:17.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:16 smithi100 ceph-mon[36245]: pgmap v1209: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:50:19.230 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:18 smithi047 ceph-mon[31351]: pgmap v1210: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T11:50:19.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:18 smithi102 ceph-mon[36748]: pgmap v1210: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T11:50:19.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:18 smithi100 ceph-mon[36245]: pgmap v1210: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T11:50:19.394 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-03-21T11:50:19.394 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T11:50:19.394 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 478997, 2024-03-21T11:50:19.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 633059, 2024-03-21T11:50:19.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 139573463, 2024-03-21T11:50:19.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 451428710, 2024-03-21T11:50:19.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 16, 2024-03-21T11:50:19.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 4, 2024-03-21T11:50:19.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 38.409477469, 2024-03-21T11:50:19.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 15.255374428, 2024-03-21T11:50:19.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 26.454360864, 2024-03-21T11:50:19.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 1.939191874, 2024-03-21T11:50:19.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:19.395 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 633059, 2024-03-21T11:50:19.395 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 142.191351279, 2024-03-21T11:50:19.395 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000224609 2024-03-21T11:50:19.395 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.396 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:19.396 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 609326, 2024-03-21T11:50:19.396 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.236189443, 2024-03-21T11:50:19.396 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000387 2024-03-21T11:50:19.396 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.396 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 979920, 2024-03-21T11:50:19.396 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 313184 2024-03-21T11:50:19.396 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.396 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T11:50:19.396 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 150870, 2024-03-21T11:50:19.396 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 183614, 2024-03-21T11:50:19.396 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 34467686, 2024-03-21T11:50:19.396 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 213945592, 2024-03-21T11:50:19.396 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 60, 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 7, 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 23.344829688, 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 11.982995540, 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 8.273589105, 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 3.555130126, 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 183612, 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 39.748339140, 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000216480 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 222903, 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.069998018, 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000314 2024-03-21T11:50:19.397 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.398 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T11:50:19.398 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 797712 2024-03-21T11:50:19.398 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.398 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T11:50:19.398 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 417481, 2024-03-21T11:50:19.398 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 388988, 2024-03-21T11:50:19.398 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 104141527, 2024-03-21T11:50:19.398 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 176009902, 2024-03-21T11:50:19.398 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 54, 2024-03-21T11:50:19.398 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 6, 2024-03-21T11:50:19.398 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 36.186099611, 2024-03-21T11:50:19.398 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 16.245287351, 2024-03-21T11:50:19.398 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 22.809592253, 2024-03-21T11:50:19.398 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 2.222937505, 2024-03-21T11:50:19.399 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:19.399 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 388986, 2024-03-21T11:50:19.399 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 70.890785250, 2024-03-21T11:50:19.399 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000182245 2024-03-21T11:50:19.399 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.399 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:19.399 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 716093, 2024-03-21T11:50:19.399 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.179595444, 2024-03-21T11:50:19.399 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000250 2024-03-21T11:50:19.399 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.399 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T11:50:19.399 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T11:50:19.399 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.399 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 73859, 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 152.749358320, 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.002068121 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-03-21T11:50:19.400 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 55512, 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 13.255113763, 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000238779 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 237308, 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 187722, 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 187722, 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1394.550955696, 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.007428809 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 49586, 2024-03-21T11:50:19.401 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-03-21T11:50:19.402 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-03-21T11:50:19.402 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 76, 2024-03-21T11:50:19.402 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 4, 2024-03-21T11:50:19.402 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 4, 2024-03-21T11:50:19.402 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 453, 2024-03-21T11:50:19.402 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 282, 2024-03-21T11:50:19.402 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 125, 2024-03-21T11:50:19.402 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 46, 2024-03-21T11:50:19.402 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 50, 2024-03-21T11:50:19.402 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 241, 2024-03-21T11:50:19.402 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 1, 2024-03-21T11:50:19.402 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 1, 2024-03-21T11:50:19.402 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 5, 2024-03-21T11:50:19.402 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 358349, 2024-03-21T11:50:19.403 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 172546, 2024-03-21T11:50:19.403 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 49586, 2024-03-21T11:50:19.403 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 101, 2024-03-21T11:50:19.403 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-03-21T11:50:19.403 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-03-21T11:50:19.403 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 2876, 2024-03-21T11:50:19.403 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 0, 2024-03-21T11:50:19.403 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2024-03-21T11:50:19.403 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 5, 2024-03-21T11:50:19.403 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 35, 2024-03-21T11:50:19.403 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 5, 2024-03-21T11:50:19.403 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 76, 2024-03-21T11:50:19.403 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 4566, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 674, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 212105939, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 4566, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 588, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 588, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 3978, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 22, 2024-03-21T11:50:19.404 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 42531, 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 97917, 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 87216, 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 22667, 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 9768, 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 118173 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 70, 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 1000, 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 545, 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 926, 2024-03-21T11:50:19.405 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 908, 2024-03-21T11:50:19.406 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 2, 2024-03-21T11:50:19.406 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-03-21T11:50:19.406 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-03-21T11:50:19.406 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 29813, 2024-03-21T11:50:19.406 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 29811, 2024-03-21T11:50:19.406 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2024-03-21T11:50:19.406 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-03-21T11:50:19.406 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-03-21T11:50:19.406 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-03-21T11:50:19.406 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-03-21T11:50:19.406 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-03-21T11:50:19.406 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-03-21T11:50:19.406 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T11:50:19.406 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 5, 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 8, 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 144300, 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 37944, 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 36871, 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 107429, 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 1073, 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 174, 2024-03-21T11:50:19.407 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 46, 2024-03-21T11:50:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 44, 2024-03-21T11:50:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 130, 2024-03-21T11:50:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 9, 2024-03-21T11:50:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-03-21T11:50:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 2, 2024-03-21T11:50:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 81578438, 2024-03-21T11:50:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 304024996, 2024-03-21T11:50:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 0, 2024-03-21T11:50:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-03-21T11:50:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 40602, 2024-03-21T11:50:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 600.006214998, 2024-03-21T11:50:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.014777750 2024-03-21T11:50:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 0 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 26, 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 29944, 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 29918, 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 25, 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 547, 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 522, 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 453, 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 63031, 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 62578, 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 1, 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 33307, 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 33306, 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 74920, 2024-03-21T11:50:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 223612 2024-03-21T11:50:19.410 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.410 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-03-21T11:50:19.410 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 315790, 2024-03-21T11:50:19.410 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 3419, 2024-03-21T11:50:19.410 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 237308, 2024-03-21T11:50:19.410 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 3082, 2024-03-21T11:50:19.410 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 6871, 2024-03-21T11:50:19.410 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-03-21T11:50:19.410 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 29736, 2024-03-21T11:50:19.410 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 220.021324989, 2024-03-21T11:50:19.410 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.007399156 2024-03-21T11:50:19.410 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.410 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 7647, 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 174.541463717, 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.022824828 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-03-21T11:50:19.411 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 68954, 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 198.789609659, 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.002882930 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1, 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000127876, 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000127876 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-03-21T11:50:19.412 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.413 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.413 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.413 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.413 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-03-21T11:50:19.413 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.413 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.413 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.413 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.413 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-03-21T11:50:19.413 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.413 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.414 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.414 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.414 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-03-21T11:50:19.414 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.414 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.414 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.414 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 15, 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.221740189, 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.014782679 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.416 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-03-21T11:50:19.416 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.416 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.416 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.416 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.416 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-03-21T11:50:19.416 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 44828, 2024-03-21T11:50:19.416 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 601.153564807, 2024-03-21T11:50:19.416 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.013410224 2024-03-21T11:50:19.416 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.416 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-03-21T11:50:19.416 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 5952, 2024-03-21T11:50:19.416 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 55.538735513, 2024-03-21T11:50:19.416 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.009331104 2024-03-21T11:50:19.417 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.417 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-03-21T11:50:19.417 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.417 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.417 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.417 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.417 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-03-21T11:50:19.417 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.417 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.417 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.417 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.417 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-03-21T11:50:19.417 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 28, 2024-03-21T11:50:19.417 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.483271040, 2024-03-21T11:50:19.418 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.017259680 2024-03-21T11:50:19.418 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.418 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-03-21T11:50:19.418 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.418 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.418 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.418 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.418 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-03-21T11:50:19.418 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.418 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.418 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.418 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.418 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-03-21T11:50:19.418 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 92, 2024-03-21T11:50:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 3.794388955, 2024-03-21T11:50:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.041243358 2024-03-21T11:50:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-03-21T11:50:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-03-21T11:50:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 684, 2024-03-21T11:50:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1.978959186, 2024-03-21T11:50:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.002893215 2024-03-21T11:50:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.419 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-03-21T11:50:19.421 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.421 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.421 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.421 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.422 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-03-21T11:50:19.422 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.422 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.422 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.422 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.422 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-03-21T11:50:19.422 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.422 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.422 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.422 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.423 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-03-21T11:50:19.423 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 29785, 2024-03-21T11:50:19.423 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 138.027769765, 2024-03-21T11:50:19.423 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.004634136 2024-03-21T11:50:19.423 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.423 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-03-21T11:50:19.423 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-03-21T11:50:19.423 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-03-21T11:50:19.423 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.423 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.423 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.423 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:19.423 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.423 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 1, 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 2, 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 1, 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 1, 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 19959, 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 19959, 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 1585, 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T11:50:19.424 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-03-21T11:50:19.425 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T11:50:19.426 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T11:50:19.426 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T11:50:19.426 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T11:50:19.426 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T11:50:19.426 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-03-21T11:50:19.426 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-03-21T11:50:19.426 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-03-21T11:50:19.426 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T11:50:19.426 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T11:50:19.426 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-03-21T11:50:19.426 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-03-21T11:50:19.426 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-03-21T11:50:19.426 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 308463, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 133, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 616, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 7, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-03-21T11:50:19.427 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 8888, 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 170, 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 434576, 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 5494, 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-03-21T11:50:19.428 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 108367, 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 306947696, 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 33, 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 108367, 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 108367, 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 2048.740719322, 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.018905577 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 108367, 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 110338 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.429 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 108367, 2024-03-21T11:50:19.430 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 5477, 2024-03-21T11:50:19.430 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 102890, 2024-03-21T11:50:19.430 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-03-21T11:50:19.430 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-03-21T11:50:19.430 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 0, 2024-03-21T11:50:19.430 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 349, 2024-03-21T11:50:19.430 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 911, 2024-03-21T11:50:19.430 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 41513, 2024-03-21T11:50:19.430 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 484, 2024-03-21T11:50:19.430 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-03-21T11:50:19.430 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-03-21T11:50:19.430 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-03-21T11:50:19.430 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 4, 2024-03-21T11:50:19.430 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 59729, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 4566, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 548, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2024-03-21T11:50:19.431 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-03-21T11:50:19.432 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-03-21T11:50:19.432 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-03-21T11:50:19.432 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-03-21T11:50:19.432 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-03-21T11:50:19.432 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-03-21T11:50:19.432 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-03-21T11:50:19.432 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-03-21T11:50:19.432 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-03-21T11:50:19.432 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-03-21T11:50:19.432 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-03-21T11:50:19.432 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-03-21T11:50:19.432 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-03-21T11:50:19.432 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-03-21T11:50:19.432 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 96, 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 5, 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 12, 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 17, 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 5, 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 1597, 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 0, 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 637 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 1, 2024-03-21T11:50:19.433 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 0 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 10, 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 8, 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 59596, 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 29811, 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-03-21T11:50:19.434 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 1047344, 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 189326935, 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 1047344, 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 1047344, 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 189326935, 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.435 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.436 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.436 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:19.436 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.436 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-03-21T11:50:19.436 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:19.436 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-03-21T11:50:19.436 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:19.436 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-03-21T11:50:19.436 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-03-21T11:50:19.436 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:19.436 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:19.436 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 108367, 2024-03-21T11:50:19.436 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 310196887, 2024-03-21T11:50:19.436 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 108367, 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 310196887, 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:19.437 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 108367, 2024-03-21T11:50:19.438 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 108367, 2024-03-21T11:50:19.438 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 108367, 2024-03-21T11:50:19.438 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 108367, 2024-03-21T11:50:19.438 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:19.438 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.438 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.438 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.438 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:19.438 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.438 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-03-21T11:50:19.438 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:19.438 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-03-21T11:50:19.438 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:19.438 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 29811, 2024-03-21T11:50:19.439 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 3249191, 2024-03-21T11:50:19.439 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:19.439 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 29811, 2024-03-21T11:50:19.439 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-03-21T11:50:19.439 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-03-21T11:50:19.439 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 911, 2024-03-21T11:50:19.439 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 3249191, 2024-03-21T11:50:19.439 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:19.439 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.439 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.439 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.439 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:19.439 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:19.439 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x561b3a108500": { 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 144300, 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 299830692, 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 144300, 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 40602, 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 299830692, 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:19.440 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:19.441 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:19.441 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:19.441 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-03-21T11:50:20.388 INFO:tasks.check_counter:Daemon mds.c mds.exported=5 2024-03-21T11:50:20.388 INFO:tasks.check_counter:Daemon mds.c mds.imported=5 2024-03-21T11:50:20.388 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=70 2024-03-21T11:50:20.388 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=1000 2024-03-21T11:50:20.388 DEBUG:tasks.check_counter:Getting stats from f 2024-03-21T11:50:20.388 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-21T11:50:20.678 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:50:21.075 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:20 smithi102 ceph-mon[36748]: pgmap v1211: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-21T11:50:21.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:20 smithi047 ceph-mon[31351]: pgmap v1211: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-21T11:50:21.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:20 smithi100 ceph-mon[36245]: pgmap v1211: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-21T11:50:22.682 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-03-21T11:50:22.682 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T11:50:22.682 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 444087, 2024-03-21T11:50:22.682 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 299618, 2024-03-21T11:50:22.683 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 116146647, 2024-03-21T11:50:22.683 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 109103334, 2024-03-21T11:50:22.683 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 13, 2024-03-21T11:50:22.683 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 5, 2024-03-21T11:50:22.683 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 24.936336135, 2024-03-21T11:50:22.683 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 9.366752542, 2024-03-21T11:50:22.683 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 18.036456155, 2024-03-21T11:50:22.683 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.766626736, 2024-03-21T11:50:22.683 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:22.683 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 299617, 2024-03-21T11:50:22.683 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 28.541615744, 2024-03-21T11:50:22.683 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000095260 2024-03-21T11:50:22.683 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.683 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:22.684 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 691088, 2024-03-21T11:50:22.684 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.132063386, 2024-03-21T11:50:22.684 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000191 2024-03-21T11:50:22.684 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.684 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-21T11:50:22.684 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 448 2024-03-21T11:50:22.684 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.684 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T11:50:22.684 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 92420, 2024-03-21T11:50:22.684 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 90919, 2024-03-21T11:50:22.684 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 22162302, 2024-03-21T11:50:22.684 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 55448944, 2024-03-21T11:50:22.684 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 62, 2024-03-21T11:50:22.685 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 6, 2024-03-21T11:50:22.685 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 6.400322699, 2024-03-21T11:50:22.685 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 2.259927028, 2024-03-21T11:50:22.685 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 3.734807743, 2024-03-21T11:50:22.685 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.815058792, 2024-03-21T11:50:22.685 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:22.685 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 90917, 2024-03-21T11:50:22.685 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 16.449964191, 2024-03-21T11:50:22.685 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000180933 2024-03-21T11:50:22.685 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.685 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:22.685 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 137109, 2024-03-21T11:50:22.685 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.029070168, 2024-03-21T11:50:22.685 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000212 2024-03-21T11:50:22.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.686 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 2496, 2024-03-21T11:50:22.686 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 801376 2024-03-21T11:50:22.686 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.686 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T11:50:22.686 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 184296, 2024-03-21T11:50:22.686 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 264558, 2024-03-21T11:50:22.686 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 54771279, 2024-03-21T11:50:22.686 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 257278883, 2024-03-21T11:50:22.686 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 55, 2024-03-21T11:50:22.686 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 7, 2024-03-21T11:50:22.686 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 17.202039735, 2024-03-21T11:50:22.686 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 7.437510763, 2024-03-21T11:50:22.686 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 7.805861425, 2024-03-21T11:50:22.687 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 1.007476633, 2024-03-21T11:50:22.687 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:22.687 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 264557, 2024-03-21T11:50:22.687 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 41.757080248, 2024-03-21T11:50:22.687 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000157837 2024-03-21T11:50:22.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.687 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:22.687 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 230814, 2024-03-21T11:50:22.687 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.077959253, 2024-03-21T11:50:22.687 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000337 2024-03-21T11:50:22.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.687 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 965920, 2024-03-21T11:50:22.687 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 317696 2024-03-21T11:50:22.687 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-03-21T11:50:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-03-21T11:50:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-03-21T11:50:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-03-21T11:50:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-03-21T11:50:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-03-21T11:50:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 49238, 2024-03-21T11:50:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 48.174321443, 2024-03-21T11:50:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000978397 2024-03-21T11:50:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-03-21T11:50:22.688 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-03-21T11:50:22.689 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-03-21T11:50:22.689 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 28590, 2024-03-21T11:50:22.689 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 3.572917741, 2024-03-21T11:50:22.689 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000124970 2024-03-21T11:50:22.689 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:22.689 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.689 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-03-21T11:50:22.689 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 92196, 2024-03-21T11:50:22.689 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 92036, 2024-03-21T11:50:22.689 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-03-21T11:50:22.689 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 92036, 2024-03-21T11:50:22.689 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 432.485586473, 2024-03-21T11:50:22.689 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.004699091 2024-03-21T11:50:22.689 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.690 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-03-21T11:50:22.690 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 160, 2024-03-21T11:50:22.690 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-03-21T11:50:22.690 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-03-21T11:50:22.690 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 11, 2024-03-21T11:50:22.690 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 0, 2024-03-21T11:50:22.690 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2024-03-21T11:50:22.690 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 13, 2024-03-21T11:50:22.690 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 1, 2024-03-21T11:50:22.690 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 1, 2024-03-21T11:50:22.690 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 11, 2024-03-21T11:50:22.690 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 11, 2024-03-21T11:50:22.690 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 223, 2024-03-21T11:50:22.691 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 0, 2024-03-21T11:50:22.691 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 0, 2024-03-21T11:50:22.691 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 2, 2024-03-21T11:50:22.691 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 154173, 2024-03-21T11:50:22.691 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 84533, 2024-03-21T11:50:22.691 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 159, 2024-03-21T11:50:22.691 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 106, 2024-03-21T11:50:22.691 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-03-21T11:50:22.691 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-03-21T11:50:22.691 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 1652, 2024-03-21T11:50:22.691 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 0, 2024-03-21T11:50:22.691 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2024-03-21T11:50:22.691 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 2, 2024-03-21T11:50:22.691 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 9, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 2, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 23, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 1572, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 674, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 212105939, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 1572, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 189, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T11:50:22.692 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 189, 2024-03-21T11:50:22.693 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 1383, 2024-03-21T11:50:22.693 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 2, 2024-03-21T11:50:22.693 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 16326, 2024-03-21T11:50:22.693 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 44641, 2024-03-21T11:50:22.693 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T11:50:22.693 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T11:50:22.693 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T11:50:22.693 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 39661, 2024-03-21T11:50:22.693 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 11843, 2024-03-21T11:50:22.693 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 5703, 2024-03-21T11:50:22.693 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 59986 2024-03-21T11:50:22.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.693 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-03-21T11:50:22.693 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 383, 2024-03-21T11:50:22.694 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 1275, 2024-03-21T11:50:22.694 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 685, 2024-03-21T11:50:22.694 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 1203, 2024-03-21T11:50:22.694 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 414, 2024-03-21T11:50:22.694 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 1, 2024-03-21T11:50:22.694 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-03-21T11:50:22.694 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-03-21T11:50:22.694 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 14598, 2024-03-21T11:50:22.694 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 14597, 2024-03-21T11:50:22.694 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2024-03-21T11:50:22.694 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-03-21T11:50:22.694 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-03-21T11:50:22.694 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-03-21T11:50:22.694 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-03-21T11:50:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-03-21T11:50:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-03-21T11:50:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T11:50:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 2, 2024-03-21T11:50:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-03-21T11:50:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 0, 2024-03-21T11:50:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-03-21T11:50:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-03-21T11:50:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-03-21T11:50:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-03-21T11:50:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 69402, 2024-03-21T11:50:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2024-03-21T11:50:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2024-03-21T11:50:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 69402, 2024-03-21T11:50:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-03-21T11:50:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2024-03-21T11:50:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 81, 2024-03-21T11:50:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2024-03-21T11:50:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2024-03-21T11:50:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 81, 2024-03-21T11:50:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 7, 2024-03-21T11:50:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-03-21T11:50:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2024-03-21T11:50:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 4194304, 2024-03-21T11:50:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 148420519, 2024-03-21T11:50:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 0, 2024-03-21T11:50:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 24278, 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 264.257725834, 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.010884657 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 0 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 18, 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 14725, 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 14707, 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 14, 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 641, 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 627, 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 13, 2024-03-21T11:50:22.697 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 30141, 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 30128, 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 0, 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 16278, 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 16278, 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 64724, 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 223612 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 132089, 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 13, 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 92196, 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 1902, 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 666, 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-03-21T11:50:22.698 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 14570, 2024-03-21T11:50:22.699 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 94.349033529, 2024-03-21T11:50:22.699 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.006475568 2024-03-21T11:50:22.699 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.699 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-03-21T11:50:22.699 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1875, 2024-03-21T11:50:22.699 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 19.070415136, 2024-03-21T11:50:22.699 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.010170888 2024-03-21T11:50:22.699 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.699 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-03-21T11:50:22.699 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.699 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.699 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.699 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.699 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-03-21T11:50:22.700 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.700 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.700 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.700 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.700 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-03-21T11:50:22.700 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 34280, 2024-03-21T11:50:22.700 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 40.725716492, 2024-03-21T11:50:22.700 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001188031 2024-03-21T11:50:22.700 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.700 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-03-21T11:50:22.700 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.700 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.700 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.700 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.701 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.702 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-03-21T11:50:22.702 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.702 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.702 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.702 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.702 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-03-21T11:50:22.705 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.706 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.706 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.706 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.706 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-03-21T11:50:22.706 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 5, 2024-03-21T11:50:22.706 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.064146872, 2024-03-21T11:50:22.706 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.012829374 2024-03-21T11:50:22.706 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.706 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-03-21T11:50:22.706 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.706 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.706 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.706 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.707 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-03-21T11:50:22.707 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.707 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.707 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.707 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.707 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-03-21T11:50:22.707 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.707 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.707 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.707 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.707 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-03-21T11:50:22.707 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 23292, 2024-03-21T11:50:22.707 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 209.627272179, 2024-03-21T11:50:22.708 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.008999968 2024-03-21T11:50:22.708 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.708 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-03-21T11:50:22.708 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2951, 2024-03-21T11:50:22.708 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 22.604810909, 2024-03-21T11:50:22.708 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.007660051 2024-03-21T11:50:22.708 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.708 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-03-21T11:50:22.708 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.708 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.708 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.708 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.708 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-03-21T11:50:22.708 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 9, 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.140368448, 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.015596494 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-03-21T11:50:22.709 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.710 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.710 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.710 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.710 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-03-21T11:50:22.710 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 13, 2024-03-21T11:50:22.710 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.324820083, 2024-03-21T11:50:22.710 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.024986160 2024-03-21T11:50:22.710 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.710 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-03-21T11:50:22.710 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.710 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.710 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.710 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.710 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-03-21T11:50:22.711 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 452, 2024-03-21T11:50:22.711 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.412794892, 2024-03-21T11:50:22.711 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000913263 2024-03-21T11:50:22.711 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.711 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-03-21T11:50:22.711 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.711 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.711 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.711 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.711 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-03-21T11:50:22.711 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.711 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.711 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.711 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.712 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-03-21T11:50:22.712 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.712 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.712 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.712 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.712 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-03-21T11:50:22.712 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 14589, 2024-03-21T11:50:22.712 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 45.166207933, 2024-03-21T11:50:22.712 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.003095908 2024-03-21T11:50:22.712 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.712 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-03-21T11:50:22.712 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-03-21T11:50:22.712 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-03-21T11:50:22.712 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 1, 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 2, 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 1, 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 1, 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 9744, 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 9744, 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 3152, 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-03-21T11:50:22.713 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.714 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-03-21T11:50:22.714 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-03-21T11:50:22.714 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-03-21T11:50:22.714 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T11:50:22.714 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-03-21T11:50:22.714 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T11:50:22.714 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-03-21T11:50:22.714 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T11:50:22.714 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T11:50:22.714 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T11:50:22.714 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T11:50:22.714 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T11:50:22.714 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-03-21T11:50:22.715 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T11:50:22.715 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T11:50:22.715 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-03-21T11:50:22.715 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-03-21T11:50:22.715 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-03-21T11:50:22.715 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-03-21T11:50:22.715 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T11:50:22.715 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T11:50:22.715 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T11:50:22.715 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T11:50:22.715 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T11:50:22.715 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-03-21T11:50:22.715 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-03-21T11:50:22.715 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 309382, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 136, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 704, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 8, 2024-03-21T11:50:22.716 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-03-21T11:50:22.717 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-03-21T11:50:22.717 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-03-21T11:50:22.717 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-03-21T11:50:22.717 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-03-21T11:50:22.717 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-03-21T11:50:22.717 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 8888, 2024-03-21T11:50:22.717 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 170, 2024-03-21T11:50:22.717 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T11:50:22.717 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-03-21T11:50:22.717 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-03-21T11:50:22.717 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-03-21T11:50:22.717 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 68027, 2024-03-21T11:50:22.717 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 458, 2024-03-21T11:50:22.717 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-03-21T11:50:22.718 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-03-21T11:50:22.718 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-03-21T11:50:22.718 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-03-21T11:50:22.718 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.718 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-03-21T11:50:22.718 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-03-21T11:50:22.718 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-03-21T11:50:22.718 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 57703, 2024-03-21T11:50:22.718 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 147414234, 2024-03-21T11:50:22.718 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 33, 2024-03-21T11:50:22.718 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 57703, 2024-03-21T11:50:22.718 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-03-21T11:50:22.718 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 57703, 2024-03-21T11:50:22.718 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1119.662076155, 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.019403879 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 57703, 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 58925 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 57703, 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 2303, 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 55400, 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 0, 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 10, 2024-03-21T11:50:22.719 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 731, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 25009, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 406, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 29232, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 1572, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 20, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-03-21T11:50:22.720 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-03-21T11:50:22.721 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-03-21T11:50:22.721 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-03-21T11:50:22.721 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-03-21T11:50:22.721 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T11:50:22.721 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-03-21T11:50:22.721 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-03-21T11:50:22.721 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2024-03-21T11:50:22.721 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-03-21T11:50:22.721 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-03-21T11:50:22.721 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-03-21T11:50:22.721 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-03-21T11:50:22.721 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-03-21T11:50:22.721 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-03-21T11:50:22.721 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 96, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 5, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 12, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 17, 2024-03-21T11:50:22.722 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 5, 2024-03-21T11:50:22.723 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-03-21T11:50:22.723 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 1460, 2024-03-21T11:50:22.723 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 0, 2024-03-21T11:50:22.723 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 485 2024-03-21T11:50:22.723 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.723 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-03-21T11:50:22.723 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 1, 2024-03-21T11:50:22.723 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-03-21T11:50:22.723 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-03-21T11:50:22.723 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-03-21T11:50:22.723 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.723 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-03-21T11:50:22.723 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 0 2024-03-21T11:50:22.723 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.724 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-03-21T11:50:22.724 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-03-21T11:50:22.724 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 10, 2024-03-21T11:50:22.724 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-03-21T11:50:22.724 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 7, 2024-03-21T11:50:22.724 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 29186, 2024-03-21T11:50:22.724 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 14597, 2024-03-21T11:50:22.724 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-03-21T11:50:22.724 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.724 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T11:50:22.724 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:22.724 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-03-21T11:50:22.724 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:22.724 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 720799, 2024-03-21T11:50:22.725 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 130420831, 2024-03-21T11:50:22.725 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:22.725 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 720799, 2024-03-21T11:50:22.725 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-03-21T11:50:22.725 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-03-21T11:50:22.725 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 720799, 2024-03-21T11:50:22.725 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 130420831, 2024-03-21T11:50:22.725 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:22.725 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.725 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.725 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.725 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:22.725 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.725 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-03-21T11:50:22.726 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:22.726 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-03-21T11:50:22.726 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:22.726 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-03-21T11:50:22.726 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-03-21T11:50:22.726 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:22.726 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:22.726 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 57703, 2024-03-21T11:50:22.726 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 149005243, 2024-03-21T11:50:22.726 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 57703, 2024-03-21T11:50:22.726 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 149005243, 2024-03-21T11:50:22.726 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:22.726 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.726 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.727 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.727 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:22.727 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.727 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-03-21T11:50:22.727 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:22.727 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-03-21T11:50:22.727 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:22.727 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-03-21T11:50:22.727 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-03-21T11:50:22.727 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:22.727 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:22.727 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 57703, 2024-03-21T11:50:22.727 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 57703, 2024-03-21T11:50:22.727 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 57703, 2024-03-21T11:50:22.728 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 57703, 2024-03-21T11:50:22.728 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:22.728 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.728 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.728 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.728 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:22.728 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.728 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-03-21T11:50:22.728 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:22.728 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-03-21T11:50:22.728 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:22.728 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 14597, 2024-03-21T11:50:22.728 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1591009, 2024-03-21T11:50:22.728 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 14597, 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 731, 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1591009, 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x558e13dae500": { 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-03-21T11:50:22.729 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:22.730 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 69402, 2024-03-21T11:50:22.730 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 144226215, 2024-03-21T11:50:22.730 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:22.730 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 69402, 2024-03-21T11:50:22.730 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-03-21T11:50:22.730 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-03-21T11:50:22.730 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 24278, 2024-03-21T11:50:22.730 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 144226215, 2024-03-21T11:50:22.730 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:22.730 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:22.730 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:22.730 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:22.730 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:22.730 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:22.731 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-03-21T11:50:23.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:22 smithi102 ceph-mon[36748]: pgmap v1212: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:50:23.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:22 smithi047 ceph-mon[31351]: pgmap v1212: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:50:23.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:22 smithi100 ceph-mon[36245]: pgmap v1212: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:50:23.507 INFO:tasks.check_counter:Daemon mds.f mds.exported=2 2024-03-21T11:50:23.507 INFO:tasks.check_counter:Daemon mds.f mds.imported=2 2024-03-21T11:50:23.507 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=383 2024-03-21T11:50:23.507 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=1275 2024-03-21T11:50:23.507 DEBUG:tasks.check_counter:Getting stats from i 2024-03-21T11:50:23.508 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-21T11:50:23.774 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:50:25.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:24 smithi102 ceph-mon[36748]: pgmap v1213: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:50:25.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:24 smithi047 ceph-mon[31351]: pgmap v1213: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:50:25.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:24 smithi100 ceph-mon[36245]: pgmap v1213: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-03-21T11:50:25.517 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-03-21T11:50:25.517 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T11:50:25.517 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 195247, 2024-03-21T11:50:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 148547, 2024-03-21T11:50:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 54242738, 2024-03-21T11:50:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 57962849, 2024-03-21T11:50:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 16, 2024-03-21T11:50:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 4, 2024-03-21T11:50:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 11.208156704, 2024-03-21T11:50:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 4.383373644, 2024-03-21T11:50:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 8.031629676, 2024-03-21T11:50:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.199452066, 2024-03-21T11:50:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 148545, 2024-03-21T11:50:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 8.525126507, 2024-03-21T11:50:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000057390 2024-03-21T11:50:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 248183, 2024-03-21T11:50:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.064679432, 2024-03-21T11:50:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000260 2024-03-21T11:50:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 928, 2024-03-21T11:50:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 736 2024-03-21T11:50:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T11:50:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 324042, 2024-03-21T11:50:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 168966, 2024-03-21T11:50:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 87497853, 2024-03-21T11:50:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 29892818, 2024-03-21T11:50:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 13, 2024-03-21T11:50:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 7, 2024-03-21T11:50:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 15.196384410, 2024-03-21T11:50:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 5.279414722, 2024-03-21T11:50:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 13.083665802, 2024-03-21T11:50:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.037938287, 2024-03-21T11:50:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 168964, 2024-03-21T11:50:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 7.804906185, 2024-03-21T11:50:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000046192 2024-03-21T11:50:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 486501, 2024-03-21T11:50:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.088007980, 2024-03-21T11:50:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000180 2024-03-21T11:50:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-21T11:50:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 790400 2024-03-21T11:50:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T11:50:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 50327, 2024-03-21T11:50:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 27735, 2024-03-21T11:50:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 14589814, 2024-03-21T11:50:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 11088862, 2024-03-21T11:50:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 99, 2024-03-21T11:50:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 6, 2024-03-21T11:50:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 3.008409334, 2024-03-21T11:50:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 1.179434797, 2024-03-21T11:50:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 2.219337867, 2024-03-21T11:50:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.129013842, 2024-03-21T11:50:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 27735, 2024-03-21T11:50:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1.826115521, 2024-03-21T11:50:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000065841 2024-03-21T11:50:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 69254, 2024-03-21T11:50:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.012813743, 2024-03-21T11:50:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000185 2024-03-21T11:50:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 982160, 2024-03-21T11:50:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 324944 2024-03-21T11:50:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-03-21T11:50:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-03-21T11:50:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-03-21T11:50:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-03-21T11:50:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-03-21T11:50:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-03-21T11:50:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 7591, 2024-03-21T11:50:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 5.548235943, 2024-03-21T11:50:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000730896 2024-03-21T11:50:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-03-21T11:50:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-03-21T11:50:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-03-21T11:50:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 4176, 2024-03-21T11:50:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.261853220, 2024-03-21T11:50:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000062704 2024-03-21T11:50:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-03-21T11:50:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 71656, 2024-03-21T11:50:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 8321, 2024-03-21T11:50:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-03-21T11:50:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 8321, 2024-03-21T11:50:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 45.506047925, 2024-03-21T11:50:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.005468819 2024-03-21T11:50:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-03-21T11:50:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 63335, 2024-03-21T11:50:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-03-21T11:50:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-03-21T11:50:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 11, 2024-03-21T11:50:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 0, 2024-03-21T11:50:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2024-03-21T11:50:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 19, 2024-03-21T11:50:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 4, 2024-03-21T11:50:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 15, 2024-03-21T11:50:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 0, 2024-03-21T11:50:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 13, 2024-03-21T11:50:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 226, 2024-03-21T11:50:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 0, 2024-03-21T11:50:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 0, 2024-03-21T11:50:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 6, 2024-03-21T11:50:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 82885, 2024-03-21T11:50:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 7058, 2024-03-21T11:50:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 63335, 2024-03-21T11:50:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 107, 2024-03-21T11:50:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-03-21T11:50:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-03-21T11:50:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 67, 2024-03-21T11:50:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 0, 2024-03-21T11:50:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2024-03-21T11:50:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 3, 2024-03-21T11:50:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 0, 2024-03-21T11:50:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 3, 2024-03-21T11:50:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 42, 2024-03-21T11:50:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-03-21T11:50:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 613, 2024-03-21T11:50:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2024-03-21T11:50:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 674, 2024-03-21T11:50:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 212105939, 2024-03-21T11:50:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-03-21T11:50:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 613, 2024-03-21T11:50:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-03-21T11:50:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T11:50:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-03-21T11:50:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 0, 2024-03-21T11:50:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T11:50:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T11:50:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 613, 2024-03-21T11:50:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 0, 2024-03-21T11:50:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 872, 2024-03-21T11:50:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 9149, 2024-03-21T11:50:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T11:50:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T11:50:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T11:50:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 6277, 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 2391, 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 1184, 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 7653 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 753, 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 1047, 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 574, 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 958, 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 761, 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 0, 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 1664, 2024-03-21T11:50:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 1664, 2024-03-21T11:50:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2024-03-21T11:50:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-03-21T11:50:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-03-21T11:50:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-03-21T11:50:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-03-21T11:50:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-03-21T11:50:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-03-21T11:50:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T11:50:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 3, 2024-03-21T11:50:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-03-21T11:50:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 0, 2024-03-21T11:50:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-03-21T11:50:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-03-21T11:50:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-03-21T11:50:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-03-21T11:50:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 13085, 2024-03-21T11:50:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2024-03-21T11:50:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2024-03-21T11:50:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 13085, 2024-03-21T11:50:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-03-21T11:50:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2024-03-21T11:50:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 15, 2024-03-21T11:50:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2024-03-21T11:50:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2024-03-21T11:50:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 15, 2024-03-21T11:50:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 3, 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 4194304, 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 22342682, 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 0, 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 3528, 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 53.449560065, 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.015150102 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 0 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 26, 2024-03-21T11:50:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 1803, 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 1777, 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 25, 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 515, 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 490, 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 19, 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 3445, 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 3426, 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 0, 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 2421, 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 2421, 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 59920, 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 223612 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-03-21T11:50:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 79745, 2024-03-21T11:50:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 13, 2024-03-21T11:50:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 71656, 2024-03-21T11:50:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 125, 2024-03-21T11:50:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 2524, 2024-03-21T11:50:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-03-21T11:50:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1636, 2024-03-21T11:50:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 24.773859629, 2024-03-21T11:50:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.015142945 2024-03-21T11:50:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-03-21T11:50:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 9, 2024-03-21T11:50:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.019895699, 2024-03-21T11:50:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.002210633 2024-03-21T11:50:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-03-21T11:50:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-03-21T11:50:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-03-21T11:50:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2457, 2024-03-21T11:50:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 2.241707351, 2024-03-21T11:50:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000912375 2024-03-21T11:50:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-03-21T11:50:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-03-21T11:50:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-03-21T11:50:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-03-21T11:50:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-03-21T11:50:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-03-21T11:50:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.538 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.538 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.538 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.538 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-03-21T11:50:25.538 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.538 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-03-21T11:50:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-03-21T11:50:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-03-21T11:50:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-03-21T11:50:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2488, 2024-03-21T11:50:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 4.284379755, 2024-03-21T11:50:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001722017 2024-03-21T11:50:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-03-21T11:50:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 67, 2024-03-21T11:50:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1.077068119, 2024-03-21T11:50:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.016075643 2024-03-21T11:50:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-03-21T11:50:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-03-21T11:50:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-03-21T11:50:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-03-21T11:50:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-03-21T11:50:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-03-21T11:50:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-03-21T11:50:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-03-21T11:50:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-03-21T11:50:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1664, 2024-03-21T11:50:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 13.109137372, 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.007878087 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 1, 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 2, 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 1, 2024-03-21T11:50:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 1, 2024-03-21T11:50:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-03-21T11:50:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 953, 2024-03-21T11:50:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 953, 2024-03-21T11:50:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 1603, 2024-03-21T11:50:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-03-21T11:50:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-03-21T11:50:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-03-21T11:50:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-03-21T11:50:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T11:50:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-03-21T11:50:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T11:50:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-03-21T11:50:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T11:50:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T11:50:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T11:50:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T11:50:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T11:50:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-03-21T11:50:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T11:50:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T11:50:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-03-21T11:50:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-03-21T11:50:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-03-21T11:50:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-03-21T11:50:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T11:50:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T11:50:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T11:50:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T11:50:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T11:50:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-03-21T11:50:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-03-21T11:50:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-03-21T11:50:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T11:50:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T11:50:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-03-21T11:50:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-03-21T11:50:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-03-21T11:50:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-03-21T11:50:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T11:50:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 292971, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 126, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 528, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 6, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 8888, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 170, 2024-03-21T11:50:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T11:50:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-03-21T11:50:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-03-21T11:50:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-03-21T11:50:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 102629, 2024-03-21T11:50:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 429, 2024-03-21T11:50:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-03-21T11:50:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-03-21T11:50:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-03-21T11:50:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-03-21T11:50:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-03-21T11:50:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-03-21T11:50:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-03-21T11:50:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 8873, 2024-03-21T11:50:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 18642874, 2024-03-21T11:50:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 28, 2024-03-21T11:50:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 8873, 2024-03-21T11:50:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-03-21T11:50:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 8873, 2024-03-21T11:50:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 205.731157890, 2024-03-21T11:50:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.023186200 2024-03-21T11:50:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-03-21T11:50:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-03-21T11:50:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 8873, 2024-03-21T11:50:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 9506 2024-03-21T11:50:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 8873, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 946, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 7927, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 0, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 10, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 333, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 3861, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 227, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2024-03-21T11:50:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 3344, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 613, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 20, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-03-21T11:50:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2024-03-21T11:50:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-03-21T11:50:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-03-21T11:50:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-03-21T11:50:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-03-21T11:50:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-03-21T11:50:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-03-21T11:50:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-03-21T11:50:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-03-21T11:50:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-03-21T11:50:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-03-21T11:50:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-03-21T11:50:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-03-21T11:50:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-03-21T11:50:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-03-21T11:50:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-03-21T11:50:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-03-21T11:50:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 96, 2024-03-21T11:50:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-03-21T11:50:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 5, 2024-03-21T11:50:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 12, 2024-03-21T11:50:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 15, 2024-03-21T11:50:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 3, 2024-03-21T11:50:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-03-21T11:50:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 856, 2024-03-21T11:50:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 0, 2024-03-21T11:50:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 242 2024-03-21T11:50:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 1, 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 0 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 10, 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 5, 2024-03-21T11:50:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 3328, 2024-03-21T11:50:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 1664, 2024-03-21T11:50:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-03-21T11:50:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T11:50:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-03-21T11:50:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 569612, 2024-03-21T11:50:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 106325401, 2024-03-21T11:50:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 569612, 2024-03-21T11:50:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-03-21T11:50:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-03-21T11:50:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 569612, 2024-03-21T11:50:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 106325401, 2024-03-21T11:50:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-03-21T11:50:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-03-21T11:50:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-03-21T11:50:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-03-21T11:50:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 8873, 2024-03-21T11:50:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 18824250, 2024-03-21T11:50:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 8873, 2024-03-21T11:50:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 18824250, 2024-03-21T11:50:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-03-21T11:50:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 8873, 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 8873, 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 8873, 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 8873, 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 1664, 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 181376, 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 1664, 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 333, 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 181376, 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x55835b46a500": { 2024-03-21T11:50:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-03-21T11:50:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 13085, 2024-03-21T11:50:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 18148378, 2024-03-21T11:50:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 13085, 2024-03-21T11:50:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-03-21T11:50:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-03-21T11:50:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 3528, 2024-03-21T11:50:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 18148378, 2024-03-21T11:50:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:25.563 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-03-21T11:50:26.329 INFO:tasks.check_counter:Daemon mds.i mds.exported=3 2024-03-21T11:50:26.330 INFO:tasks.check_counter:Daemon mds.i mds.imported=3 2024-03-21T11:50:26.330 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=753 2024-03-21T11:50:26.330 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=1047 2024-03-21T11:50:26.330 DEBUG:tasks.check_counter:Getting stats from l 2024-03-21T11:50:26.330 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid a6f35682-e773-11ee-95cd-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-21T11:50:26.664 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T11:50:27.039 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:26 smithi102 ceph-mon[36748]: pgmap v1214: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:50:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:26 smithi047 ceph-mon[31351]: pgmap v1214: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:50:27.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:26 smithi100 ceph-mon[36245]: pgmap v1214: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T11:50:28.487 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-03-21T11:50:28.487 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T11:50:28.487 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 1885, 2024-03-21T11:50:28.487 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 1887, 2024-03-21T11:50:28.487 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 111574513, 2024-03-21T11:50:28.487 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 599494, 2024-03-21T11:50:28.487 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 9, 2024-03-21T11:50:28.487 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 4, 2024-03-21T11:50:28.487 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.196112458, 2024-03-21T11:50:28.487 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.034109894, 2024-03-21T11:50:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.105485440, 2024-03-21T11:50:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.045073814, 2024-03-21T11:50:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1885, 2024-03-21T11:50:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.074022542, 2024-03-21T11:50:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000039269 2024-03-21T11:50:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T11:50:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T11:50:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T11:50:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 2029, 2024-03-21T11:50:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 2008, 2024-03-21T11:50:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 89411102, 2024-03-21T11:50:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 775341, 2024-03-21T11:50:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 5, 2024-03-21T11:50:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 4, 2024-03-21T11:50:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.213442810, 2024-03-21T11:50:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.050119769, 2024-03-21T11:50:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.102440360, 2024-03-21T11:50:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.053346315, 2024-03-21T11:50:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:28.489 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2007, 2024-03-21T11:50:28.490 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.094388931, 2024-03-21T11:50:28.490 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000047029 2024-03-21T11:50:28.490 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.490 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:28.490 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.490 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.490 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.490 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.490 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 952112, 2024-03-21T11:50:28.490 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 293248 2024-03-21T11:50:28.490 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.490 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T11:50:28.490 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 2393, 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 2759, 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 101411930, 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 1635932, 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 8, 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 5, 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.256578139, 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.063047792, 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.108221288, 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.068220528, 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2757, 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.164327089, 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000059603 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.491 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-03-21T11:50:28.492 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.492 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.492 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.492 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.492 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-03-21T11:50:28.492 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 877184 2024-03-21T11:50:28.492 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.492 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-03-21T11:50:28.492 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-03-21T11:50:28.492 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-03-21T11:50:28.492 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.492 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-03-21T11:50:28.492 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-03-21T11:50:28.492 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-03-21T11:50:28.493 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 9784, 2024-03-21T11:50:28.493 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.231642327, 2024-03-21T11:50:28.493 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000023675 2024-03-21T11:50:28.493 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:28.493 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.493 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-03-21T11:50:28.493 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-03-21T11:50:28.493 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-03-21T11:50:28.493 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.493 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.493 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.493 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:28.493 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.493 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-03-21T11:50:28.494 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 0, 2024-03-21T11:50:28.494 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 0, 2024-03-21T11:50:28.494 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-03-21T11:50:28.494 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.494 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.494 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.494 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.494 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-03-21T11:50:28.494 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 0, 2024-03-21T11:50:28.494 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-03-21T11:50:28.494 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-03-21T11:50:28.494 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 0, 2024-03-21T11:50:28.494 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 0, 2024-03-21T11:50:28.494 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 29846, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 5143, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 24703, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 0, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 22498, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 0, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 0, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 0, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 1, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 0, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 0, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 0, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 0, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-03-21T11:50:28.495 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-03-21T11:50:28.496 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 0, 2024-03-21T11:50:28.496 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 0, 2024-03-21T11:50:28.496 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2024-03-21T11:50:28.496 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 0, 2024-03-21T11:50:28.496 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 0, 2024-03-21T11:50:28.496 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 0, 2024-03-21T11:50:28.496 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 0, 2024-03-21T11:50:28.496 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-03-21T11:50:28.496 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 0, 2024-03-21T11:50:28.496 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2024-03-21T11:50:28.496 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 0, 2024-03-21T11:50:28.496 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 0, 2024-03-21T11:50:28.496 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-03-21T11:50:28.496 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 0, 2024-03-21T11:50:28.497 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 0, 2024-03-21T11:50:28.498 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 0, 2024-03-21T11:50:28.498 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 0 2024-03-21T11:50:28.498 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.498 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-03-21T11:50:28.498 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 0, 2024-03-21T11:50:28.498 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 0, 2024-03-21T11:50:28.498 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 0, 2024-03-21T11:50:28.498 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 0, 2024-03-21T11:50:28.498 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 0, 2024-03-21T11:50:28.498 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 0, 2024-03-21T11:50:28.498 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-03-21T11:50:28.498 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-03-21T11:50:28.498 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 0, 2024-03-21T11:50:28.498 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 0, 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 0, 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 0, 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.499 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 0, 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 144300, 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 0, 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 130, 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 13, 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2024-03-21T11:50:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 81578438, 2024-03-21T11:50:28.501 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 304024996, 2024-03-21T11:50:28.501 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 304024397, 2024-03-21T11:50:28.501 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-03-21T11:50:28.501 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.501 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.501 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.501 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.501 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 144300 2024-03-21T11:50:28.501 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.501 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-03-21T11:50:28.501 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 15, 2024-03-21T11:50:28.501 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 29850, 2024-03-21T11:50:28.501 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 29835, 2024-03-21T11:50:28.501 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 13, 2024-03-21T11:50:28.502 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 90, 2024-03-21T11:50:28.502 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 77, 2024-03-21T11:50:28.502 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 29846, 2024-03-21T11:50:28.502 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 30261, 2024-03-21T11:50:28.502 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 415, 2024-03-21T11:50:28.502 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 0, 2024-03-21T11:50:28.502 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 0, 2024-03-21T11:50:28.502 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 0, 2024-03-21T11:50:28.502 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 81600, 2024-03-21T11:50:28.502 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 215420 2024-03-21T11:50:28.502 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.502 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-03-21T11:50:28.502 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 0, 2024-03-21T11:50:28.502 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 0, 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 0, 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 0, 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 0, 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-03-21T11:50:28.503 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-03-21T11:50:28.504 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-03-21T11:50:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-03-21T11:50:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-03-21T11:50:28.506 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.506 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.506 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.506 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.506 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-03-21T11:50:28.506 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.506 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.506 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.506 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.506 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-03-21T11:50:28.506 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.506 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.506 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.506 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.507 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-03-21T11:50:28.507 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.507 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.508 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.508 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.508 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-03-21T11:50:28.508 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.508 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.508 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.508 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.508 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-03-21T11:50:28.508 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.508 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.508 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.508 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.508 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.509 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-03-21T11:50:28.510 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.510 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.510 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.510 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.510 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-03-21T11:50:28.510 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.510 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.510 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.510 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.510 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-03-21T11:50:28.510 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.510 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.511 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.511 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.511 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-03-21T11:50:28.511 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.511 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.511 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.511 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.511 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-03-21T11:50:28.511 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.511 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.511 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.512 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.513 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.513 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-03-21T11:50:28.513 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.513 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.513 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.513 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.513 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-03-21T11:50:28.513 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.513 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.513 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.513 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.513 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-03-21T11:50:28.513 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.513 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.514 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.514 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.514 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-03-21T11:50:28.514 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.514 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.514 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.514 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.514 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-03-21T11:50:28.514 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-03-21T11:50:28.514 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-03-21T11:50:28.514 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.514 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.515 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.515 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:28.515 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.515 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-03-21T11:50:28.515 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 1, 2024-03-21T11:50:28.515 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 2, 2024-03-21T11:50:28.515 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 1, 2024-03-21T11:50:28.515 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 1, 2024-03-21T11:50:28.515 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-03-21T11:50:28.515 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 0, 2024-03-21T11:50:28.515 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 0, 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 3156, 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T11:50:28.516 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T11:50:28.517 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-03-21T11:50:28.517 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T11:50:28.517 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T11:50:28.517 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-03-21T11:50:28.517 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-03-21T11:50:28.517 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-03-21T11:50:28.517 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-03-21T11:50:28.517 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T11:50:28.517 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T11:50:28.517 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T11:50:28.517 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T11:50:28.517 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T11:50:28.517 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-03-21T11:50:28.517 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-03-21T11:50:28.518 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-03-21T11:50:28.518 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T11:50:28.518 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T11:50:28.518 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-03-21T11:50:28.518 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-03-21T11:50:28.518 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-03-21T11:50:28.518 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-03-21T11:50:28.518 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T11:50:28.518 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-03-21T11:50:28.518 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T11:50:28.518 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-03-21T11:50:28.518 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 232149, 2024-03-21T11:50:28.518 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 77, 2024-03-21T11:50:28.518 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 8888, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 170, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 21061819, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 119756, 2024-03-21T11:50:28.519 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 5787, 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 21, 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 3, 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 5787, 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 5787, 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 36.459318191, 2024-03-21T11:50:28.520 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.006300210 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 5787, 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 5788 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 5787, 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 5787, 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 0, 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 3339, 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 0, 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 2446, 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 0, 2024-03-21T11:50:28.521 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 0, 2024-03-21T11:50:28.522 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-03-21T11:50:28.522 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-03-21T11:50:28.522 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-03-21T11:50:28.522 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2024-03-21T11:50:28.522 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 0, 2024-03-21T11:50:28.522 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-03-21T11:50:28.522 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-03-21T11:50:28.522 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-03-21T11:50:28.522 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 1, 2024-03-21T11:50:28.522 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 0, 2024-03-21T11:50:28.522 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-03-21T11:50:28.522 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-03-21T11:50:28.522 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-03-21T11:50:28.522 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-03-21T11:50:28.523 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 96, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 5, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 10, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 11, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 1, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 0, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 2, 2024-03-21T11:50:28.524 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 0 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 0, 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 0 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-03-21T11:50:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 0, 2024-03-21T11:50:28.526 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 0, 2024-03-21T11:50:28.526 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 0, 2024-03-21T11:50:28.526 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-03-21T11:50:28.526 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.526 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T11:50:28.526 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:28.526 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-03-21T11:50:28.526 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:28.526 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 6303, 2024-03-21T11:50:28.526 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 301800816, 2024-03-21T11:50:28.526 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:28.526 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 6303, 2024-03-21T11:50:28.526 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 6303, 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 301800816, 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-03-21T11:50:28.527 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 5787, 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 299830697, 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 5787, 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 299830697, 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:28.528 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-03-21T11:50:28.529 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:28.529 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-03-21T11:50:28.529 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-03-21T11:50:28.529 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:28.529 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:28.529 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 5787, 2024-03-21T11:50:28.529 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 5787, 2024-03-21T11:50:28.529 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 5787, 2024-03-21T11:50:28.529 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 5787, 2024-03-21T11:50:28.529 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:28.529 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.529 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.529 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.529 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:28.530 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.530 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-03-21T11:50:28.530 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:28.530 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-03-21T11:50:28.530 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:28.530 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-03-21T11:50:28.530 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-03-21T11:50:28.530 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:28.530 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:28.530 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-03-21T11:50:28.530 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-03-21T11:50:28.530 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2024-03-21T11:50:28.530 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2024-03-21T11:50:28.530 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:28.531 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.531 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.531 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.531 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:28.531 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-03-21T11:50:28.531 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x55b46d9566e0": { 2024-03-21T11:50:28.531 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-03-21T11:50:28.531 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-03-21T11:50:28.531 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-03-21T11:50:28.531 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-03-21T11:50:28.531 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-03-21T11:50:28.531 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-03-21T11:50:28.531 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-03-21T11:50:28.531 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-03-21T11:50:28.532 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-03-21T11:50:28.532 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2024-03-21T11:50:28.532 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2024-03-21T11:50:28.532 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-03-21T11:50:28.532 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-03-21T11:50:28.532 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-03-21T11:50:28.532 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-03-21T11:50:28.532 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:28.532 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-03-21T11:50:28.532 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-03-21T11:50:29.143 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:28 smithi100 ceph-mon[36245]: pgmap v1215: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T11:50:29.241 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:28 smithi102 ceph-mon[36748]: pgmap v1215: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T11:50:29.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:28 smithi047 ceph-mon[31351]: pgmap v1215: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T11:50:29.317 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-21T11:50:29.317 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-21T11:50:29.317 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-03-21T11:50:29.317 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-03-21T11:50:29.318 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-21T11:50:29.329 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-21T11:50:29.329 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:50:29.329 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T11:50:29.404 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T11:50:29.404 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T11:50:29.462 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:50:29.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.692+0000 7f7f5a600640 1 -- 172.21.15.47:0/2586961147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f54107b00 msgr2=0x7f7f54107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:29.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.692+0000 7f7f5a600640 1 --2- 172.21.15.47:0/2586961147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f54107b00 0x7f7f54107f20 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009900 tx=0x7f7f4402bc40 comp rx=0 tx=0).stop 2024-03-21T11:50:29.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.692+0000 7f7f5a600640 1 -- 172.21.15.47:0/2586961147 shutdown_connections 2024-03-21T11:50:29.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.692+0000 7f7f5a600640 1 --2- 172.21.15.47:0/2586961147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f541096b0 0x7f7f5410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:29.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.692+0000 7f7f5a600640 1 --2- 172.21.15.47:0/2586961147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f54108cf0 0x7f7f54109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:29.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.692+0000 7f7f5a600640 1 --2- 172.21.15.47:0/2586961147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f54107b00 0x7f7f54107f20 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:29.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.692+0000 7f7f5a600640 1 -- 172.21.15.47:0/2586961147 >> 172.21.15.47:0/2586961147 conn(0x7f7f54076ab0 msgr2=0x7f7f54078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.692+0000 7f7f5a600640 1 -- 172.21.15.47:0/2586961147 shutdown_connections 2024-03-21T11:50:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.692+0000 7f7f5a600640 1 -- 172.21.15.47:0/2586961147 wait complete. 2024-03-21T11:50:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.693+0000 7f7f5a600640 1 Processor -- start 2024-03-21T11:50:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.694+0000 7f7f5a600640 1 -- start start 2024-03-21T11:50:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.694+0000 7f7f5a600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54107b00 0x7f7f5419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.694+0000 7f7f5a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f54108cf0 0x7f7f5419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.694+0000 7f7f5a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f541096b0 0x7f7f541a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.694+0000 7f7f5a600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f541137c0 con 0x7f7f54107b00 2024-03-21T11:50:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.694+0000 7f7f5a600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54113640 con 0x7f7f54108cf0 2024-03-21T11:50:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.694+0000 7f7f5a600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54113940 con 0x7f7f541096b0 2024-03-21T11:50:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.694+0000 7f7f59200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54107b00 0x7f7f5419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.694+0000 7f7f59c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f541096b0 0x7f7f541a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.694+0000 7f7f59200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54107b00 0x7f7f5419de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34948/0 (socket says 172.21.15.47:34948) 2024-03-21T11:50:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.694+0000 7f7f59200640 1 -- 172.21.15.47:0/1494069790 learned_addr learned my addr 172.21.15.47:0/1494069790 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:50:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.694+0000 7f7f53e00640 1 --2- 172.21.15.47:0/1494069790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f54108cf0 0x7f7f5419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.695+0000 7f7f59c00640 1 -- 172.21.15.47:0/1494069790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f54108cf0 msgr2=0x7f7f5419e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.695+0000 7f7f59c00640 1 --2- 172.21.15.47:0/1494069790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f54108cf0 0x7f7f5419e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.695+0000 7f7f59c00640 1 -- 172.21.15.47:0/1494069790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54107b00 msgr2=0x7f7f5419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.695+0000 7f7f59c00640 1 --2- 172.21.15.47:0/1494069790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54107b00 0x7f7f5419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.695+0000 7f7f59c00640 1 -- 172.21.15.47:0/1494069790 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f541a5a50 con 0x7f7f541096b0 2024-03-21T11:50:29.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.695+0000 7f7f59c00640 1 --2- 172.21.15.47:0/1494069790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f541096b0 0x7f7f541a53e0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009fd0 tx=0x7f7f4c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:29.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.695+0000 7f7f51600640 1 -- 172.21.15.47:0/1494069790 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00ce20 con 0x7f7f541096b0 2024-03-21T11:50:29.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.695+0000 7f7f5a600640 1 -- 172.21.15.47:0/1494069790 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f541a5d40 con 0x7f7f541096b0 2024-03-21T11:50:29.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.695+0000 7f7f51600640 1 -- 172.21.15.47:0/1494069790 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f4c005020 con 0x7f7f541096b0 2024-03-21T11:50:29.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.695+0000 7f7f51600640 1 -- 172.21.15.47:0/1494069790 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c010720 con 0x7f7f541096b0 2024-03-21T11:50:29.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.696+0000 7f7f5a600640 1 -- 172.21.15.47:0/1494069790 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f541a6230 con 0x7f7f541096b0 2024-03-21T11:50:29.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.697+0000 7f7f5a600640 1 -- 172.21.15.47:0/1494069790 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5410ee20 con 0x7f7f541096b0 2024-03-21T11:50:29.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.699+0000 7f7f51600640 1 -- 172.21.15.47:0/1494069790 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7f4c005540 con 0x7f7f541096b0 2024-03-21T11:50:29.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.700+0000 7f7f51600640 1 --2- 172.21.15.47:0/1494069790 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f380a2f20 0x7f7f380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.700+0000 7f7f51600640 1 -- 172.21.15.47:0/1494069790 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f7f4c0cfea0 con 0x7f7f541096b0 2024-03-21T11:50:29.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.700+0000 7f7f59200640 1 --2- 172.21.15.47:0/1494069790 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f380a2f20 0x7f7f380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:29.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.701+0000 7f7f59200640 1 --2- 172.21.15.47:0/1494069790 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f380a2f20 0x7f7f380a53e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005ab0 tx=0x7f7f4402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:29.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.704+0000 7f7f51600640 1 -- 172.21.15.47:0/1494069790 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f4c09c8e0 con 0x7f7f541096b0 2024-03-21T11:50:29.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.921+0000 7f7f5a600640 1 -- 172.21.15.47:0/1494069790 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f540622a0 con 0x7f7f541096b0 2024-03-21T11:50:29.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.923+0000 7f7f51600640 1 -- 172.21.15.47:0/1494069790 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24747 (secure 0 0 0) 0x7f7f4c09c1e0 con 0x7f7f541096b0 2024-03-21T11:50:29.925 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:50:29.925 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":96,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:45:38.900262+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:50:29.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.926+0000 7f7f5a600640 1 -- 172.21.15.47:0/1494069790 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f380a2f20 msgr2=0x7f7f380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:29.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.926+0000 7f7f5a600640 1 --2- 172.21.15.47:0/1494069790 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f380a2f20 0x7f7f380a53e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005ab0 tx=0x7f7f4402cf90 comp rx=0 tx=0).stop 2024-03-21T11:50:29.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.926+0000 7f7f5a600640 1 -- 172.21.15.47:0/1494069790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f541096b0 msgr2=0x7f7f541a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:29.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.926+0000 7f7f5a600640 1 --2- 172.21.15.47:0/1494069790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f541096b0 0x7f7f541a53e0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009fd0 tx=0x7f7f4c00ef10 comp rx=0 tx=0).stop 2024-03-21T11:50:29.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.926+0000 7f7f5a600640 1 -- 172.21.15.47:0/1494069790 shutdown_connections 2024-03-21T11:50:29.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.926+0000 7f7f5a600640 1 --2- 172.21.15.47:0/1494069790 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f7f380a2f20 0x7f7f380a53e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:29.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.926+0000 7f7f5a600640 1 --2- 172.21.15.47:0/1494069790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f541096b0 0x7f7f541a53e0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:29.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.926+0000 7f7f5a600640 1 --2- 172.21.15.47:0/1494069790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f54108cf0 0x7f7f5419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:29.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.926+0000 7f7f5a600640 1 --2- 172.21.15.47:0/1494069790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f54107b00 0x7f7f5419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:29.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.926+0000 7f7f5a600640 1 -- 172.21.15.47:0/1494069790 >> 172.21.15.47:0/1494069790 conn(0x7f7f54076ab0 msgr2=0x7f7f5410bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:29.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.927+0000 7f7f5a600640 1 -- 172.21.15.47:0/1494069790 shutdown_connections 2024-03-21T11:50:29.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:29.927+0000 7f7f5a600640 1 -- 172.21.15.47:0/1494069790 wait complete. 2024-03-21T11:50:29.950 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T11:15:35.545245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T11:20:47.016174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T11:20:48.108506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T11:20:52.254154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T11:50:29.950 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T11:50:30.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.188+0000 7f853be00640 1 -- 172.21.15.47:0/3519813103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f853c108db0 msgr2=0x7f853c109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:30.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.188+0000 7f853be00640 1 --2- 172.21.15.47:0/3519813103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f853c108db0 0x7f853c109230 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f8524009860 tx=0x7f852402bb70 comp rx=0 tx=0).stop 2024-03-21T11:50:30.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.188+0000 7f853be00640 1 -- 172.21.15.47:0/3519813103 shutdown_connections 2024-03-21T11:50:30.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.188+0000 7f853be00640 1 --2- 172.21.15.47:0/3519813103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f853c109770 0x7f853c110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.188+0000 7f853be00640 1 --2- 172.21.15.47:0/3519813103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f853c108db0 0x7f853c109230 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.188+0000 7f853be00640 1 --2- 172.21.15.47:0/3519813103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c107bc0 0x7f853c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.188+0000 7f853be00640 1 -- 172.21.15.47:0/3519813103 >> 172.21.15.47:0/3519813103 conn(0x7f853c076b70 msgr2=0x7f853c078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.188+0000 7f853be00640 1 -- 172.21.15.47:0/3519813103 shutdown_connections 2024-03-21T11:50:30.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.188+0000 7f853be00640 1 -- 172.21.15.47:0/3519813103 wait complete. 2024-03-21T11:50:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.190+0000 7f853be00640 1 Processor -- start 2024-03-21T11:50:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.190+0000 7f853be00640 1 -- start start 2024-03-21T11:50:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.190+0000 7f853be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c107bc0 0x7f853c19de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.190+0000 7f853be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f853c108db0 0x7f853c19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.190+0000 7f853be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f853c109770 0x7f853c1a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.190+0000 7f853be00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c113820 con 0x7f853c109770 2024-03-21T11:50:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.190+0000 7f853be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c1136a0 con 0x7f853c107bc0 2024-03-21T11:50:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.190+0000 7f853be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c1139a0 con 0x7f853c108db0 2024-03-21T11:50:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.190+0000 7f8533e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f853c108db0 0x7f853c19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.190+0000 7f853aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c107bc0 0x7f853c19de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.190+0000 7f853aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c107bc0 0x7f853c19de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47742/0 (socket says 172.21.15.47:47742) 2024-03-21T11:50:30.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.190+0000 7f8533e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f853c108db0 0x7f853c19e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41808/0 (socket says 172.21.15.47:41808) 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.190+0000 7f853aa00640 1 -- 172.21.15.47:0/3203613332 learned_addr learned my addr 172.21.15.47:0/3203613332 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.191+0000 7f853b400640 1 --2- 172.21.15.47:0/3203613332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f853c109770 0x7f853c1a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.191+0000 7f8533e00640 1 -- 172.21.15.47:0/3203613332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c107bc0 msgr2=0x7f853c19de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.191+0000 7f8533e00640 1 --2- 172.21.15.47:0/3203613332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c107bc0 0x7f853c19de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.191+0000 7f8533e00640 1 -- 172.21.15.47:0/3203613332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f853c109770 msgr2=0x7f853c1a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.191+0000 7f8533e00640 1 --2- 172.21.15.47:0/3203613332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f853c109770 0x7f853c1a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.191+0000 7f8533e00640 1 -- 172.21.15.47:0/3203613332 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c1a5ab0 con 0x7f853c108db0 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.191+0000 7f853b400640 1 --2- 172.21.15.47:0/3203613332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f853c109770 0x7f853c1a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.191+0000 7f8533e00640 1 --2- 172.21.15.47:0/3203613332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f853c108db0 0x7f853c19e3d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f852402c0b0 tx=0x7f8524032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.191+0000 7f8533400640 1 -- 172.21.15.47:0/3203613332 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524041020 con 0x7f853c108db0 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.191+0000 7f8533400640 1 -- 172.21.15.47:0/3203613332 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f852402c9f0 con 0x7f853c108db0 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.191+0000 7f8533400640 1 -- 172.21.15.47:0/3203613332 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524039040 con 0x7f853c108db0 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.191+0000 7f853be00640 1 -- 172.21.15.47:0/3203613332 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853c1a5d40 con 0x7f853c108db0 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.192+0000 7f853be00640 1 -- 172.21.15.47:0/3203613332 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853c1a61d0 con 0x7f853c108db0 2024-03-21T11:50:30.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.193+0000 7f853be00640 1 -- 172.21.15.47:0/3203613332 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853c10ef60 con 0x7f853c108db0 2024-03-21T11:50:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.194+0000 7f8533400640 1 -- 172.21.15.47:0/3203613332 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8524044020 con 0x7f853c108db0 2024-03-21T11:50:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.196+0000 7f8533400640 1 --2- 172.21.15.47:0/3203613332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f85200a2f20 0x7f85200a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.196+0000 7f8533400640 1 -- 172.21.15.47:0/3203613332 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f85240f1080 con 0x7f853c108db0 2024-03-21T11:50:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.196+0000 7f853aa00640 1 --2- 172.21.15.47:0/3203613332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f85200a2f20 0x7f85200a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.197+0000 7f853aa00640 1 --2- 172.21.15.47:0/3203613332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f85200a2f20 0x7f85200a53e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f852c0096c0 tx=0x7f852c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.200+0000 7f8533400640 1 -- 172.21.15.47:0/3203613332 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f852404b020 con 0x7f853c108db0 2024-03-21T11:50:30.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:29 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1494069790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:50:30.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:29 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1494069790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:50:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.403+0000 7f853be00640 1 -- 172.21.15.47:0/3203613332 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f853c0622a0 con 0x7f853c108db0 2024-03-21T11:50:30.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.404+0000 7f8533400640 1 -- 172.21.15.47:0/3203613332 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85240bdac0 con 0x7f853c108db0 2024-03-21T11:50:30.405 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T11:50:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.407+0000 7f853be00640 1 -- 172.21.15.47:0/3203613332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f85200a2f20 msgr2=0x7f85200a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.407+0000 7f853be00640 1 --2- 172.21.15.47:0/3203613332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f85200a2f20 0x7f85200a53e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f852c0096c0 tx=0x7f852c009260 comp rx=0 tx=0).stop 2024-03-21T11:50:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.407+0000 7f853be00640 1 -- 172.21.15.47:0/3203613332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f853c108db0 msgr2=0x7f853c19e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.407+0000 7f853be00640 1 --2- 172.21.15.47:0/3203613332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f853c108db0 0x7f853c19e3d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f852402c0b0 tx=0x7f8524032430 comp rx=0 tx=0).stop 2024-03-21T11:50:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.407+0000 7f853be00640 1 -- 172.21.15.47:0/3203613332 shutdown_connections 2024-03-21T11:50:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.407+0000 7f853be00640 1 --2- 172.21.15.47:0/3203613332 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f85200a2f20 0x7f85200a53e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.407+0000 7f853be00640 1 --2- 172.21.15.47:0/3203613332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f853c109770 0x7f853c1a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.407+0000 7f853be00640 1 --2- 172.21.15.47:0/3203613332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f853c108db0 0x7f853c19e3d0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.407+0000 7f853be00640 1 --2- 172.21.15.47:0/3203613332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c107bc0 0x7f853c19de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.407+0000 7f853be00640 1 -- 172.21.15.47:0/3203613332 >> 172.21.15.47:0/3203613332 conn(0x7f853c076b70 msgr2=0x7f853c10c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.407+0000 7f853be00640 1 -- 172.21.15.47:0/3203613332 shutdown_connections 2024-03-21T11:50:30.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.407+0000 7f853be00640 1 -- 172.21.15.47:0/3203613332 wait complete. 2024-03-21T11:50:30.428 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T11:50:30.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:29 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1494069790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:50:30.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.655+0000 7f6472e00640 1 -- 172.21.15.47:0/1892944025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f646c104930 msgr2=0x7f646c106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.655+0000 7f6472e00640 1 --2- 172.21.15.47:0/1892944025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f646c104930 0x7f646c106d20 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f6458009900 tx=0x7f645802bc30 comp rx=0 tx=0).stop 2024-03-21T11:50:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.655+0000 7f6472e00640 1 -- 172.21.15.47:0/1892944025 shutdown_connections 2024-03-21T11:50:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.655+0000 7f6472e00640 1 --2- 172.21.15.47:0/1892944025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f646c107260 0x7f646c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.655+0000 7f6472e00640 1 --2- 172.21.15.47:0/1892944025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f646c104930 0x7f646c106d20 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.655+0000 7f6472e00640 1 --2- 172.21.15.47:0/1892944025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646c102000 0x7f646c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.655+0000 7f6472e00640 1 -- 172.21.15.47:0/1892944025 >> 172.21.15.47:0/1892944025 conn(0x7f646c0fba10 msgr2=0x7f646c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.655+0000 7f6472e00640 1 -- 172.21.15.47:0/1892944025 shutdown_connections 2024-03-21T11:50:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.655+0000 7f6472e00640 1 -- 172.21.15.47:0/1892944025 wait complete. 2024-03-21T11:50:30.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.657+0000 7f6472e00640 1 Processor -- start 2024-03-21T11:50:30.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.657+0000 7f6472e00640 1 -- start start 2024-03-21T11:50:30.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.657+0000 7f6472e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f646c102000 0x7f646c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:30.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.657+0000 7f6472e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f646c104930 0x7f646c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:30.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6472e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646c107260 0x7f646c1a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:30.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6472e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c10d1d0 con 0x7f646c107260 2024-03-21T11:50:30.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6472e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c10d050 con 0x7f646c104930 2024-03-21T11:50:30.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6472e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c10d350 con 0x7f646c102000 2024-03-21T11:50:30.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6471000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f646c104930 0x7f646c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:30.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6471000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f646c104930 0x7f646c19a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47762/0 (socket says 172.21.15.47:47762) 2024-03-21T11:50:30.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6471a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f646c102000 0x7f646c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:30.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6471000640 1 -- 172.21.15.47:0/754271017 learned_addr learned my addr 172.21.15.47:0/754271017 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:50:30.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6472400640 1 --2- 172.21.15.47:0/754271017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646c107260 0x7f646c1a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:30.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6471000640 1 -- 172.21.15.47:0/754271017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f646c102000 msgr2=0x7f646c199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6471000640 1 --2- 172.21.15.47:0/754271017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f646c102000 0x7f646c199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6471000640 1 -- 172.21.15.47:0/754271017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646c107260 msgr2=0x7f646c1a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6471000640 1 --2- 172.21.15.47:0/754271017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646c107260 0x7f646c1a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6471000640 1 -- 172.21.15.47:0/754271017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c1a16f0 con 0x7f646c104930 2024-03-21T11:50:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.658+0000 7f6472400640 1 --2- 172.21.15.47:0/754271017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646c107260 0x7f646c1a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:50:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.659+0000 7f6471000640 1 --2- 172.21.15.47:0/754271017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f646c104930 0x7f646c19a010 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f6458005ab0 tx=0x7f6458005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.659+0000 7f646a000640 1 -- 172.21.15.47:0/754271017 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6458041070 con 0x7f646c104930 2024-03-21T11:50:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.659+0000 7f6472e00640 1 -- 172.21.15.47:0/754271017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646c1a1980 con 0x7f646c104930 2024-03-21T11:50:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.659+0000 7f646a000640 1 -- 172.21.15.47:0/754271017 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64580054e0 con 0x7f646c104930 2024-03-21T11:50:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.659+0000 7f646a000640 1 -- 172.21.15.47:0/754271017 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6458034440 con 0x7f646c104930 2024-03-21T11:50:30.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.659+0000 7f6472e00640 1 -- 172.21.15.47:0/754271017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646c1a1e70 con 0x7f646c104930 2024-03-21T11:50:30.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.660+0000 7f6472e00640 1 -- 172.21.15.47:0/754271017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646c108db0 con 0x7f646c104930 2024-03-21T11:50:30.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.662+0000 7f646a000640 1 -- 172.21.15.47:0/754271017 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6458044070 con 0x7f646c104930 2024-03-21T11:50:30.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.664+0000 7f646a000640 1 --2- 172.21.15.47:0/754271017 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f64540a2f20 0x7f64540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:30.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.664+0000 7f646a000640 1 -- 172.21.15.47:0/754271017 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f64580f1440 con 0x7f646c104930 2024-03-21T11:50:30.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.664+0000 7f6471a00640 1 --2- 172.21.15.47:0/754271017 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f64540a2f20 0x7f64540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:30.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.665+0000 7f6471a00640 1 --2- 172.21.15.47:0/754271017 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f64540a2f20 0x7f64540a53e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6460005fd0 tx=0x7f6460009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:30.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.668+0000 7f646a000640 1 -- 172.21.15.47:0/754271017 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f64580bde80 con 0x7f646c104930 2024-03-21T11:50:30.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.879+0000 7f6472e00640 1 -- 172.21.15.47:0/754271017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f646c0622a0 con 0x7f646c104930 2024-03-21T11:50:30.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.879+0000 7f646a000640 1 -- 172.21.15.47:0/754271017 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f64580bd780 con 0x7f646c104930 2024-03-21T11:50:30.880 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:50:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.882+0000 7f6472e00640 1 -- 172.21.15.47:0/754271017 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f64540a2f20 msgr2=0x7f64540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.882+0000 7f6472e00640 1 --2- 172.21.15.47:0/754271017 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f64540a2f20 0x7f64540a53e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6460005fd0 tx=0x7f6460009760 comp rx=0 tx=0).stop 2024-03-21T11:50:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.882+0000 7f6472e00640 1 -- 172.21.15.47:0/754271017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f646c104930 msgr2=0x7f646c19a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.882+0000 7f6472e00640 1 --2- 172.21.15.47:0/754271017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f646c104930 0x7f646c19a010 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f6458005ab0 tx=0x7f6458005dc0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.882+0000 7f6472e00640 1 -- 172.21.15.47:0/754271017 shutdown_connections 2024-03-21T11:50:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.882+0000 7f6472e00640 1 --2- 172.21.15.47:0/754271017 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f64540a2f20 0x7f64540a53e0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.882+0000 7f6472e00640 1 --2- 172.21.15.47:0/754271017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f646c107260 0x7f646c1a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.882+0000 7f6472e00640 1 --2- 172.21.15.47:0/754271017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f646c104930 0x7f646c19a010 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.882+0000 7f6472e00640 1 --2- 172.21.15.47:0/754271017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f646c102000 0x7f646c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.882+0000 7f6472e00640 1 -- 172.21.15.47:0/754271017 >> 172.21.15.47:0/754271017 conn(0x7f646c0fba10 msgr2=0x7f646c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.883+0000 7f6472e00640 1 -- 172.21.15.47:0/754271017 shutdown_connections 2024-03-21T11:50:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:30.883+0000 7f6472e00640 1 -- 172.21.15.47:0/754271017 wait complete. 2024-03-21T11:50:30.903 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T11:50:31.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.127+0000 7f3a59c00640 1 -- 172.21.15.47:0/2597587900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a54107bc0 msgr2=0x7f3a54107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:31.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.127+0000 7f3a59c00640 1 --2- 172.21.15.47:0/2597587900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a54107bc0 0x7f3a54107fe0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009900 tx=0x7f3a4402bc50 comp rx=0 tx=0).stop 2024-03-21T11:50:31.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.127+0000 7f3a59c00640 1 -- 172.21.15.47:0/2597587900 shutdown_connections 2024-03-21T11:50:31.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.127+0000 7f3a59c00640 1 --2- 172.21.15.47:0/2597587900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a54109770 0x7f3a54110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.127+0000 7f3a59c00640 1 --2- 172.21.15.47:0/2597587900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a54108db0 0x7f3a54109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.127+0000 7f3a59c00640 1 --2- 172.21.15.47:0/2597587900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a54107bc0 0x7f3a54107fe0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.127+0000 7f3a59c00640 1 -- 172.21.15.47:0/2597587900 >> 172.21.15.47:0/2597587900 conn(0x7f3a54076b70 msgr2=0x7f3a54078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:31.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.127+0000 7f3a59c00640 1 -- 172.21.15.47:0/2597587900 shutdown_connections 2024-03-21T11:50:31.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.127+0000 7f3a59c00640 1 -- 172.21.15.47:0/2597587900 wait complete. 2024-03-21T11:50:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.129+0000 7f3a59c00640 1 Processor -- start 2024-03-21T11:50:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.129+0000 7f3a59c00640 1 -- start start 2024-03-21T11:50:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.129+0000 7f3a59c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a54107bc0 0x7f3a5419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.129+0000 7f3a59c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a54108db0 0x7f3a5419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.129+0000 7f3a59c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a54109770 0x7f3a541a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.129+0000 7f3a59c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a54113860 con 0x7f3a54107bc0 2024-03-21T11:50:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.129+0000 7f3a59c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a541136e0 con 0x7f3a54109770 2024-03-21T11:50:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.129+0000 7f3a59c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a541139e0 con 0x7f3a54108db0 2024-03-21T11:50:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.130+0000 7f3a53e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a54107bc0 0x7f3a5419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.130+0000 7f3a4be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a54108db0 0x7f3a5419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.130+0000 7f3a59200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a54109770 0x7f3a541a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.130+0000 7f3a53e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a54107bc0 0x7f3a5419dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34996/0 (socket says 172.21.15.47:34996) 2024-03-21T11:50:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.130+0000 7f3a53e00640 1 -- 172.21.15.47:0/1479417428 learned_addr learned my addr 172.21.15.47:0/1479417428 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:50:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.130+0000 7f3a53e00640 1 -- 172.21.15.47:0/1479417428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a54108db0 msgr2=0x7f3a5419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.130+0000 7f3a53e00640 1 --2- 172.21.15.47:0/1479417428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a54108db0 0x7f3a5419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.130+0000 7f3a53e00640 1 -- 172.21.15.47:0/1479417428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a54109770 msgr2=0x7f3a541a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.130+0000 7f3a53e00640 1 --2- 172.21.15.47:0/1479417428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a54109770 0x7f3a541a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.130+0000 7f3a53e00640 1 -- 172.21.15.47:0/1479417428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a541a5ac0 con 0x7f3a54107bc0 2024-03-21T11:50:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.130+0000 7f3a59200640 1 --2- 172.21.15.47:0/1479417428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a54109770 0x7f3a541a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:50:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.130+0000 7f3a4be00640 1 --2- 172.21.15.47:0/1479417428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a54108db0 0x7f3a5419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:50:31.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.131+0000 7f3a53e00640 1 --2- 172.21.15.47:0/1479417428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a54107bc0 0x7f3a5419dea0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3a4402c260 tx=0x7f3a44005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.131+0000 7f3a51600640 1 -- 172.21.15.47:0/1479417428 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a44041070 con 0x7f3a54107bc0 2024-03-21T11:50:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.131+0000 7f3a59c00640 1 -- 172.21.15.47:0/1479417428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a541a5d50 con 0x7f3a54107bc0 2024-03-21T11:50:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.131+0000 7f3a51600640 1 -- 172.21.15.47:0/1479417428 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a440054e0 con 0x7f3a54107bc0 2024-03-21T11:50:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.131+0000 7f3a59c00640 1 -- 172.21.15.47:0/1479417428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a541a6210 con 0x7f3a54107bc0 2024-03-21T11:50:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.131+0000 7f3a51600640 1 -- 172.21.15.47:0/1479417428 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4404b060 con 0x7f3a54107bc0 2024-03-21T11:50:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.132+0000 7f3a59c00640 1 -- 172.21.15.47:0/1479417428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5410eee0 con 0x7f3a54107bc0 2024-03-21T11:50:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.134+0000 7f3a51600640 1 -- 172.21.15.47:0/1479417428 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3a44044070 con 0x7f3a54107bc0 2024-03-21T11:50:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.136+0000 7f3a51600640 1 --2- 172.21.15.47:0/1479417428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a380a2f20 0x7f3a380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.136+0000 7f3a51600640 1 -- 172.21.15.47:0/1479417428 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f3a440f1370 con 0x7f3a54107bc0 2024-03-21T11:50:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.136+0000 7f3a4be00640 1 --2- 172.21.15.47:0/1479417428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a380a2f20 0x7f3a380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:31.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.137+0000 7f3a4be00640 1 --2- 172.21.15.47:0/1479417428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a380a2f20 0x7f3a380a53e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3a5419f3b0 tx=0x7f3a3c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:31.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.141+0000 7f3a51600640 1 -- 172.21.15.47:0/1479417428 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3a44003e20 con 0x7f3a54107bc0 2024-03-21T11:50:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:30 smithi047 ceph-mon[31351]: pgmap v1216: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-21T11:50:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:31 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3203613332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:50:31.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:31 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/754271017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:50:31.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.347+0000 7f3a59c00640 1 -- 172.21.15.47:0/1479417428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3a540622a0 con 0x7f3a54107bc0 2024-03-21T11:50:31.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.349+0000 7f3a51600640 1 -- 172.21.15.47:0/1479417428 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3a440bd760 con 0x7f3a54107bc0 2024-03-21T11:50:31.350 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T11:50:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.351+0000 7f3a59c00640 1 -- 172.21.15.47:0/1479417428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a380a2f20 msgr2=0x7f3a380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:31.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.351+0000 7f3a59c00640 1 --2- 172.21.15.47:0/1479417428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a380a2f20 0x7f3a380a53e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3a5419f3b0 tx=0x7f3a3c00b390 comp rx=0 tx=0).stop 2024-03-21T11:50:31.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.351+0000 7f3a59c00640 1 -- 172.21.15.47:0/1479417428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a54107bc0 msgr2=0x7f3a5419dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:31.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.351+0000 7f3a59c00640 1 --2- 172.21.15.47:0/1479417428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a54107bc0 0x7f3a5419dea0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3a4402c260 tx=0x7f3a44005dd0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.352+0000 7f3a59c00640 1 -- 172.21.15.47:0/1479417428 shutdown_connections 2024-03-21T11:50:31.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.352+0000 7f3a59c00640 1 --2- 172.21.15.47:0/1479417428 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f3a380a2f20 0x7f3a380a53e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.352+0000 7f3a59c00640 1 --2- 172.21.15.47:0/1479417428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a54109770 0x7f3a541a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.352+0000 7f3a59c00640 1 --2- 172.21.15.47:0/1479417428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3a54108db0 0x7f3a5419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.352+0000 7f3a59c00640 1 --2- 172.21.15.47:0/1479417428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a54107bc0 0x7f3a5419dea0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.352+0000 7f3a59c00640 1 -- 172.21.15.47:0/1479417428 >> 172.21.15.47:0/1479417428 conn(0x7f3a54076b70 msgr2=0x7f3a5410c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:31.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.352+0000 7f3a59c00640 1 -- 172.21.15.47:0/1479417428 shutdown_connections 2024-03-21T11:50:31.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.352+0000 7f3a59c00640 1 -- 172.21.15.47:0/1479417428 wait complete. 2024-03-21T11:50:31.370 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T11:50:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:31 smithi100 ceph-mon[36245]: pgmap v1216: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-21T11:50:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:31 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3203613332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:50:31.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:31 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/754271017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:50:31.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:31 smithi102 ceph-mon[36748]: pgmap v1216: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-21T11:50:31.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:31 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3203613332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T11:50:31.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:31 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/754271017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T11:50:31.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.594+0000 7fa278e00640 1 -- 172.21.15.47:0/832665282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa274108eb0 msgr2=0x7fa274109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:31.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.594+0000 7fa278e00640 1 --2- 172.21.15.47:0/832665282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa274108eb0 0x7fa274109330 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009900 tx=0x7fa25c02bc50 comp rx=0 tx=0).stop 2024-03-21T11:50:31.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.594+0000 7fa278e00640 1 -- 172.21.15.47:0/832665282 shutdown_connections 2024-03-21T11:50:31.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.594+0000 7fa278e00640 1 --2- 172.21.15.47:0/832665282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa274109870 0x7fa274110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.594+0000 7fa278e00640 1 --2- 172.21.15.47:0/832665282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa274108eb0 0x7fa274109330 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.594+0000 7fa278e00640 1 --2- 172.21.15.47:0/832665282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa274107cc0 0x7fa2741080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.594+0000 7fa278e00640 1 -- 172.21.15.47:0/832665282 >> 172.21.15.47:0/832665282 conn(0x7fa274076c60 msgr2=0x7fa2740790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:31.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.595+0000 7fa278e00640 1 -- 172.21.15.47:0/832665282 shutdown_connections 2024-03-21T11:50:31.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.595+0000 7fa278e00640 1 -- 172.21.15.47:0/832665282 wait complete. 2024-03-21T11:50:31.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.596+0000 7fa278e00640 1 Processor -- start 2024-03-21T11:50:31.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.596+0000 7fa278e00640 1 -- start start 2024-03-21T11:50:31.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.596+0000 7fa278e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa274107cc0 0x7fa27419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:31.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.596+0000 7fa278e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa274108eb0 0x7fa27419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:31.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.596+0000 7fa278e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa274109870 0x7fa2741a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:31.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.596+0000 7fa278e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274113940 con 0x7fa274109870 2024-03-21T11:50:31.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.596+0000 7fa278e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741137c0 con 0x7fa274108eb0 2024-03-21T11:50:31.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.596+0000 7fa278e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274113ac0 con 0x7fa274107cc0 2024-03-21T11:50:31.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.597+0000 7fa271e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa274107cc0 0x7fa27419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:31.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.597+0000 7fa271e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa274107cc0 0x7fa27419e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41846/0 (socket says 172.21.15.47:41846) 2024-03-21T11:50:31.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.597+0000 7fa271e00640 1 -- 172.21.15.47:0/2427032288 learned_addr learned my addr 172.21.15.47:0/2427032288 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:50:31.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.597+0000 7fa26be00640 1 --2- 172.21.15.47:0/2427032288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa274108eb0 0x7fa27419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:31.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.597+0000 7fa272800640 1 --2- 172.21.15.47:0/2427032288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa274109870 0x7fa2741a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:31.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.597+0000 7fa271e00640 1 -- 172.21.15.47:0/2427032288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa274108eb0 msgr2=0x7fa27419e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:31.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.597+0000 7fa271e00640 1 --2- 172.21.15.47:0/2427032288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa274108eb0 0x7fa27419e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.597+0000 7fa271e00640 1 -- 172.21.15.47:0/2427032288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa274109870 msgr2=0x7fa2741a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:31.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.597+0000 7fa271e00640 1 --2- 172.21.15.47:0/2427032288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa274109870 0x7fa2741a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.597+0000 7fa271e00640 1 -- 172.21.15.47:0/2427032288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2741a5c50 con 0x7fa274107cc0 2024-03-21T11:50:31.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.597+0000 7fa271e00640 1 --2- 172.21.15.47:0/2427032288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa274107cc0 0x7fa27419e030 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fa26400e9c0 tx=0x7fa26400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:31.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.598+0000 7fa26aa00640 1 -- 172.21.15.47:0/2427032288 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26400cde0 con 0x7fa274107cc0 2024-03-21T11:50:31.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.598+0000 7fa26aa00640 1 -- 172.21.15.47:0/2427032288 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa264010910 con 0x7fa274107cc0 2024-03-21T11:50:31.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.598+0000 7fa278e00640 1 -- 172.21.15.47:0/2427032288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2741a5f40 con 0x7fa274107cc0 2024-03-21T11:50:31.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.598+0000 7fa26aa00640 1 -- 172.21.15.47:0/2427032288 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa264018a10 con 0x7fa274107cc0 2024-03-21T11:50:31.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.598+0000 7fa278e00640 1 -- 172.21.15.47:0/2427032288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2741a63a0 con 0x7fa274107cc0 2024-03-21T11:50:31.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.599+0000 7fa278e00640 1 -- 172.21.15.47:0/2427032288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27410efe0 con 0x7fa274107cc0 2024-03-21T11:50:31.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.601+0000 7fa26aa00640 1 -- 172.21.15.47:0/2427032288 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa26401f020 con 0x7fa274107cc0 2024-03-21T11:50:31.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.602+0000 7fa26aa00640 1 --2- 172.21.15.47:0/2427032288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2580a2f20 0x7fa2580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:31.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.602+0000 7fa26aa00640 1 -- 172.21.15.47:0/2427032288 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fa2640cf9b0 con 0x7fa274107cc0 2024-03-21T11:50:31.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.602+0000 7fa26be00640 1 --2- 172.21.15.47:0/2427032288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2580a2f20 0x7fa2580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:31.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.603+0000 7fa26be00640 1 --2- 172.21.15.47:0/2427032288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2580a2f20 0x7fa2580a53e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa25c02c7d0 tx=0x7fa25c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:31.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.606+0000 7fa26aa00640 1 -- 172.21.15.47:0/2427032288 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa26409c3f0 con 0x7fa274107cc0 2024-03-21T11:50:31.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.808+0000 7fa278e00640 1 -- 172.21.15.47:0/2427032288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa2740623a0 con 0x7fa274107cc0 2024-03-21T11:50:31.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.809+0000 7fa26aa00640 1 -- 172.21.15.47:0/2427032288 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa26409bcf0 con 0x7fa274107cc0 2024-03-21T11:50:31.810 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T11:50:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.811+0000 7fa278e00640 1 -- 172.21.15.47:0/2427032288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2580a2f20 msgr2=0x7fa2580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.811+0000 7fa278e00640 1 --2- 172.21.15.47:0/2427032288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2580a2f20 0x7fa2580a53e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa25c02c7d0 tx=0x7fa25c005fd0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.811+0000 7fa278e00640 1 -- 172.21.15.47:0/2427032288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa274107cc0 msgr2=0x7fa27419e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.811+0000 7fa278e00640 1 --2- 172.21.15.47:0/2427032288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa274107cc0 0x7fa27419e030 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fa26400e9c0 tx=0x7fa26400ee80 comp rx=0 tx=0).stop 2024-03-21T11:50:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.812+0000 7fa278e00640 1 -- 172.21.15.47:0/2427032288 shutdown_connections 2024-03-21T11:50:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.812+0000 7fa278e00640 1 --2- 172.21.15.47:0/2427032288 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fa2580a2f20 0x7fa2580a53e0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.812+0000 7fa278e00640 1 --2- 172.21.15.47:0/2427032288 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa274109870 0x7fa2741a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.812+0000 7fa278e00640 1 --2- 172.21.15.47:0/2427032288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa274108eb0 0x7fa27419e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.812+0000 7fa278e00640 1 --2- 172.21.15.47:0/2427032288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa274107cc0 0x7fa27419e030 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.812+0000 7fa278e00640 1 -- 172.21.15.47:0/2427032288 >> 172.21.15.47:0/2427032288 conn(0x7fa274076c60 msgr2=0x7fa27410c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.812+0000 7fa278e00640 1 -- 172.21.15.47:0/2427032288 shutdown_connections 2024-03-21T11:50:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:31.812+0000 7fa278e00640 1 -- 172.21.15.47:0/2427032288 wait complete. 2024-03-21T11:50:31.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T11:50:32.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.062+0000 7f92c3200640 1 -- 172.21.15.47:0/2298679809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92c4105370 msgr2=0x7f92c410bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:32.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.062+0000 7f92c3200640 1 --2- 172.21.15.47:0/2298679809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92c4105370 0x7f92c410bc00 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009ca0 tx=0x7f92bc02c100 comp rx=0 tx=0).stop 2024-03-21T11:50:32.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.062+0000 7f92c3200640 1 -- 172.21.15.47:0/2298679809 shutdown_connections 2024-03-21T11:50:32.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.062+0000 7f92c3200640 1 --2- 172.21.15.47:0/2298679809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92c4105370 0x7f92c410bc00 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:32.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.062+0000 7f92c3200640 1 --2- 172.21.15.47:0/2298679809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92c41049b0 0x7f92c4104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:32.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.062+0000 7f92c3200640 1 --2- 172.21.15.47:0/2298679809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c41037c0 0x7f92c4103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:32.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.062+0000 7f92c3200640 1 -- 172.21.15.47:0/2298679809 >> 172.21.15.47:0/2298679809 conn(0x7f92c40fed50 msgr2=0x7f92c4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:32.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.063+0000 7f92c3200640 1 -- 172.21.15.47:0/2298679809 shutdown_connections 2024-03-21T11:50:32.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.063+0000 7f92c3200640 1 -- 172.21.15.47:0/2298679809 wait complete. 2024-03-21T11:50:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.063+0000 7f92c3200640 1 Processor -- start 2024-03-21T11:50:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.064+0000 7f92c3200640 1 -- start start 2024-03-21T11:50:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.064+0000 7f92c3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92c41037c0 0x7f92c4199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.064+0000 7f92c3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c41049b0 0x7f92c4199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.064+0000 7f92c3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92c4105370 0x7f92c41a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.064+0000 7f92c3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c410f420 con 0x7f92c41049b0 2024-03-21T11:50:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.064+0000 7f92c3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c410f2a0 con 0x7f92c41037c0 2024-03-21T11:50:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.064+0000 7f92c3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c410f5a0 con 0x7f92c4105370 2024-03-21T11:50:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.064+0000 7f92c1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92c41037c0 0x7f92c4199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.064+0000 7f92c1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92c41037c0 0x7f92c4199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:47824/0 (socket says 172.21.15.47:47824) 2024-03-21T11:50:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.064+0000 7f92c1e00640 1 -- 172.21.15.47:0/935973180 learned_addr learned my addr 172.21.15.47:0/935973180 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:50:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.064+0000 7f92c1400640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c41049b0 0x7f92c4199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.064+0000 7f92c2800640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92c4105370 0x7f92c41a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.065+0000 7f92c1e00640 1 -- 172.21.15.47:0/935973180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92c4105370 msgr2=0x7f92c41a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.065+0000 7f92c1e00640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92c4105370 0x7f92c41a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.065+0000 7f92c1e00640 1 -- 172.21.15.47:0/935973180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c41049b0 msgr2=0x7f92c4199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.065+0000 7f92c1e00640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c41049b0 0x7f92c4199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.065+0000 7f92c1e00640 1 -- 172.21.15.47:0/935973180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c41a1620 con 0x7f92c41037c0 2024-03-21T11:50:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.065+0000 7f92c1400640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c41049b0 0x7f92c4199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:50:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.065+0000 7f92c1e00640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92c41037c0 0x7f92c4199a00 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f92b400e9c0 tx=0x7f92b400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.065+0000 7f92baa00640 1 -- 172.21.15.47:0/935973180 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b400cd80 con 0x7f92c41037c0 2024-03-21T11:50:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.065+0000 7f92baa00640 1 -- 172.21.15.47:0/935973180 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92b40035f0 con 0x7f92c41037c0 2024-03-21T11:50:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.065+0000 7f92c3200640 1 -- 172.21.15.47:0/935973180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c41a1910 con 0x7f92c41037c0 2024-03-21T11:50:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.065+0000 7f92baa00640 1 -- 172.21.15.47:0/935973180 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4010690 con 0x7f92c41037c0 2024-03-21T11:50:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.065+0000 7f92c3200640 1 -- 172.21.15.47:0/935973180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c41a1eb0 con 0x7f92c41037c0 2024-03-21T11:50:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.067+0000 7f92c3200640 1 -- 172.21.15.47:0/935973180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c4068800 con 0x7f92c41037c0 2024-03-21T11:50:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.068+0000 7f92baa00640 1 -- 172.21.15.47:0/935973180 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f92b40037a0 con 0x7f92c41037c0 2024-03-21T11:50:32.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.069+0000 7f92baa00640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92a80a2fa0 0x7f92a80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:32.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.069+0000 7f92baa00640 1 -- 172.21.15.47:0/935973180 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f92b40cfbf0 con 0x7f92c41037c0 2024-03-21T11:50:32.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.070+0000 7f92c1400640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92a80a2fa0 0x7f92a80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:32.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.070+0000 7f92c1400640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92a80a2fa0 0x7f92a80a5460 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f92c419af10 tx=0x7f92b000b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.073+0000 7f92baa00640 1 -- 172.21.15.47:0/935973180 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f92b409c630 con 0x7f92c41037c0 2024-03-21T11:50:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.292+0000 7f92c3200640 1 -- 172.21.15.47:0/935973180 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f92c40622a0 con 0x7f92c41037c0 2024-03-21T11:50:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.292+0000 7f92baa00640 1 -- 172.21.15.47:0/935973180 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v96) v1 ==== 81+0+589 (secure 0 0 0) 0x7f92b409bf30 con 0x7f92c41037c0 2024-03-21T11:50:32.294 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/4277663939 2024-03-22T11:12:48.837005+0000 2024-03-21T11:50:32.294 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/185270192 2024-03-22T11:12:48.837005+0000 2024-03-21T11:50:32.294 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/185270192 2024-03-22T11:12:48.837005+0000 2024-03-21T11:50:32.294 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1272635669 2024-03-22T11:12:48.837005+0000 2024-03-21T11:50:32.294 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2508263463 2024-03-22T11:12:14.713084+0000 2024-03-21T11:50:32.294 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/3773889790 2024-03-22T11:12:48.837005+0000 2024-03-21T11:50:32.294 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/1584030551 2024-03-22T11:12:14.713084+0000 2024-03-21T11:50:32.294 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/3130329909 2024-03-22T11:12:14.713084+0000 2024-03-21T11:50:32.295 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/1584030551 2024-03-22T11:12:14.713084+0000 2024-03-21T11:50:32.295 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/335539055 2024-03-22T11:12:14.713084+0000 2024-03-21T11:50:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:listed 10 entries 2024-03-21T11:50:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.295+0000 7f92c3200640 1 -- 172.21.15.47:0/935973180 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92a80a2fa0 msgr2=0x7f92a80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.295+0000 7f92c3200640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92a80a2fa0 0x7f92a80a5460 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f92c419af10 tx=0x7f92b000b5f0 comp rx=0 tx=0).stop 2024-03-21T11:50:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.295+0000 7f92c3200640 1 -- 172.21.15.47:0/935973180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92c41037c0 msgr2=0x7f92c4199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.295+0000 7f92c3200640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92c41037c0 0x7f92c4199a00 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f92b400e9c0 tx=0x7f92b400ee80 comp rx=0 tx=0).stop 2024-03-21T11:50:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.296+0000 7f92c3200640 1 -- 172.21.15.47:0/935973180 shutdown_connections 2024-03-21T11:50:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.296+0000 7f92c3200640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f92a80a2fa0 0x7f92a80a5460 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.296+0000 7f92c3200640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92c4105370 0x7f92c41a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.296+0000 7f92c3200640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c41049b0 0x7f92c4199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.296+0000 7f92c3200640 1 --2- 172.21.15.47:0/935973180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92c41037c0 0x7f92c4199a00 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.296+0000 7f92c3200640 1 -- 172.21.15.47:0/935973180 >> 172.21.15.47:0/935973180 conn(0x7f92c40fed50 msgr2=0x7f92c4107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.296+0000 7f92c3200640 1 -- 172.21.15.47:0/935973180 shutdown_connections 2024-03-21T11:50:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:32.296+0000 7f92c3200640 1 -- 172.21.15.47:0/935973180 wait complete. 2024-03-21T11:50:32.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:32 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1479417428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:50:32.312 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:32 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2427032288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:50:32.316 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-21T11:50:32.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:50:32.316 DEBUG:teuthology.orchestra.run.smithi047:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-21T11:50:32.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:32 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1479417428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:50:32.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:32 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2427032288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:50:32.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:32 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1479417428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T11:50:32.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:32 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2427032288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T11:50:32.628 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T11:50:32.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:50:32.628 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-21T11:50:32.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:50:32.723 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-21T11:50:32.847 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-21T11:50:32.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:50:32.848 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T11:50:32.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T11:50:32.983 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-03-21T11:50:33.157 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-21T11:50:33.168 INFO:tasks.cephadm:Teardown begin 2024-03-21T11:50:33.168 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-21T11:50:33.169 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:50:33.272 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:33 smithi047 ceph-mon[31351]: pgmap v1217: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:50:33.272 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:33 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/935973180' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T11:50:33.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:33 smithi100 ceph-mon[36245]: pgmap v1217: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:50:33.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:33 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/935973180' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T11:50:33.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.393+0000 7f9fd6e00640 1 -- 172.21.15.47:0/692629043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd0105490 msgr2=0x7f9fd010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:33.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.393+0000 7f9fd6e00640 1 --2- 172.21.15.47:0/692629043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd0105490 0x7f9fd010bd20 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009ca0 tx=0x7f9fc802c100 comp rx=0 tx=0).stop 2024-03-21T11:50:33.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.394+0000 7f9fd6e00640 1 -- 172.21.15.47:0/692629043 shutdown_connections 2024-03-21T11:50:33.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.394+0000 7f9fd6e00640 1 --2- 172.21.15.47:0/692629043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd0105490 0x7f9fd010bd20 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:33.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.394+0000 7f9fd6e00640 1 --2- 172.21.15.47:0/692629043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fd0104ad0 0x7f9fd0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:33.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.394+0000 7f9fd6e00640 1 --2- 172.21.15.47:0/692629043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd01038e0 0x7f9fd0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:33.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.394+0000 7f9fd6e00640 1 -- 172.21.15.47:0/692629043 >> 172.21.15.47:0/692629043 conn(0x7f9fd00fee50 msgr2=0x7f9fd0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:33.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.394+0000 7f9fd6e00640 1 -- 172.21.15.47:0/692629043 shutdown_connections 2024-03-21T11:50:33.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.394+0000 7f9fd6e00640 1 -- 172.21.15.47:0/692629043 wait complete. 2024-03-21T11:50:33.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.395+0000 7f9fd6e00640 1 Processor -- start 2024-03-21T11:50:33.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.395+0000 7f9fd6e00640 1 -- start start 2024-03-21T11:50:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.395+0000 7f9fd6e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd01038e0 0x7f9fd0199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.395+0000 7f9fd6e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd0104ad0 0x7f9fd019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.395+0000 7f9fd6e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fd0105490 0x7f9fd01a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.395+0000 7f9fd6e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd010f540 con 0x7f9fd0104ad0 2024-03-21T11:50:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.395+0000 7f9fd6e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd010f3c0 con 0x7f9fd0105490 2024-03-21T11:50:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.395+0000 7f9fd6e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd010f6c0 con 0x7f9fd01038e0 2024-03-21T11:50:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.395+0000 7f9fcfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd01038e0 0x7f9fd0199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.395+0000 7f9fcfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd01038e0 0x7f9fd0199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41876/0 (socket says 172.21.15.47:41876) 2024-03-21T11:50:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.395+0000 7f9fcfe00640 1 -- 172.21.15.47:0/1998956290 learned_addr learned my addr 172.21.15.47:0/1998956290 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:50:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.395+0000 7f9fcf400640 1 --2- 172.21.15.47:0/1998956290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd0104ad0 0x7f9fd019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.395+0000 7f9fd4e00640 1 --2- 172.21.15.47:0/1998956290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fd0105490 0x7f9fd01a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:33.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.396+0000 7f9fcfe00640 1 -- 172.21.15.47:0/1998956290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fd0105490 msgr2=0x7f9fd01a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.396+0000 7f9fcfe00640 1 --2- 172.21.15.47:0/1998956290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fd0105490 0x7f9fd01a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.396+0000 7f9fcfe00640 1 -- 172.21.15.47:0/1998956290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd0104ad0 msgr2=0x7f9fd019a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.396+0000 7f9fcfe00640 1 --2- 172.21.15.47:0/1998956290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd0104ad0 0x7f9fd019a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.396+0000 7f9fcfe00640 1 -- 172.21.15.47:0/1998956290 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd01a17a0 con 0x7f9fd01038e0 2024-03-21T11:50:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.396+0000 7f9fcfe00640 1 --2- 172.21.15.47:0/1998956290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd01038e0 0x7f9fd0199b80 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000e9c0 tx=0x7f9fc000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.396+0000 7f9fccc00640 1 -- 172.21.15.47:0/1998956290 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000cd80 con 0x7f9fd01038e0 2024-03-21T11:50:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.396+0000 7f9fd6e00640 1 -- 172.21.15.47:0/1998956290 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd01a1a90 con 0x7f9fd01038e0 2024-03-21T11:50:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.396+0000 7f9fccc00640 1 -- 172.21.15.47:0/1998956290 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc00035f0 con 0x7f9fd01038e0 2024-03-21T11:50:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.396+0000 7f9fccc00640 1 -- 172.21.15.47:0/1998956290 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0010690 con 0x7f9fd01038e0 2024-03-21T11:50:33.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.396+0000 7f9fd6e00640 1 -- 172.21.15.47:0/1998956290 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd01a1fe0 con 0x7f9fd01038e0 2024-03-21T11:50:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.397+0000 7f9fd6e00640 1 -- 172.21.15.47:0/1998956290 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd00688e0 con 0x7f9fd01038e0 2024-03-21T11:50:33.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.400+0000 7f9fccc00640 1 -- 172.21.15.47:0/1998956290 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9fc00037a0 con 0x7f9fd01038e0 2024-03-21T11:50:33.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.401+0000 7f9fccc00640 1 --2- 172.21.15.47:0/1998956290 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9fb40a2f20 0x7f9fb40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:33.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.401+0000 7f9fccc00640 1 -- 172.21.15.47:0/1998956290 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f9fc00cfc50 con 0x7f9fd01038e0 2024-03-21T11:50:33.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.401+0000 7f9fcf400640 1 --2- 172.21.15.47:0/1998956290 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9fb40a2f20 0x7f9fb40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:33.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.402+0000 7f9fcf400640 1 --2- 172.21.15.47:0/1998956290 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9fb40a2f20 0x7f9fb40a53e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0095f0 tx=0x7f9fbc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:33.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.403+0000 7f9fccc00640 1 -- 172.21.15.47:0/1998956290 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9fc009c720 con 0x7f9fd01038e0 2024-03-21T11:50:33.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:33 smithi102 ceph-mon[36748]: pgmap v1217: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-03-21T11:50:33.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:33 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/935973180' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T11:50:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.639+0000 7f9fd6e00640 1 -- 172.21.15.47:0/1998956290 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fd010ac00 con 0x7f9fd01038e0 2024-03-21T11:50:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.640+0000 7f9fccc00640 1 -- 172.21.15.47:0/1998956290 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+11481 (secure 0 0 0) 0x7f9fc009c020 con 0x7f9fd01038e0 2024-03-21T11:50:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:50:33.641 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:49:23.191513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_32623":{"gid":32623,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4278877119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4278877119},{"type":"v1","addr":"172.21.15.100:6835","nonce":4278877119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24524,24523,24535,24581,24553]},"id":1}]} 2024-03-21T11:50:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 43 2024-03-21T11:50:33.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.642+0000 7f9fd6e00640 1 -- 172.21.15.47:0/1998956290 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9fb40a2f20 msgr2=0x7f9fb40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:33.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.642+0000 7f9fd6e00640 1 --2- 172.21.15.47:0/1998956290 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9fb40a2f20 0x7f9fb40a53e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0095f0 tx=0x7f9fbc005f90 comp rx=0 tx=0).stop 2024-03-21T11:50:33.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.642+0000 7f9fd6e00640 1 -- 172.21.15.47:0/1998956290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd01038e0 msgr2=0x7f9fd0199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:33.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.642+0000 7f9fd6e00640 1 --2- 172.21.15.47:0/1998956290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd01038e0 0x7f9fd0199b80 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000e9c0 tx=0x7f9fc000ee80 comp rx=0 tx=0).stop 2024-03-21T11:50:33.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.642+0000 7f9fd6e00640 1 -- 172.21.15.47:0/1998956290 shutdown_connections 2024-03-21T11:50:33.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.642+0000 7f9fd6e00640 1 --2- 172.21.15.47:0/1998956290 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f9fb40a2f20 0x7f9fb40a53e0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:33.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.642+0000 7f9fd6e00640 1 --2- 172.21.15.47:0/1998956290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fd0105490 0x7f9fd01a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:33.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.642+0000 7f9fd6e00640 1 --2- 172.21.15.47:0/1998956290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd0104ad0 0x7f9fd019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:33.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.642+0000 7f9fd6e00640 1 --2- 172.21.15.47:0/1998956290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd01038e0 0x7f9fd0199b80 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:33.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.642+0000 7f9fd6e00640 1 -- 172.21.15.47:0/1998956290 >> 172.21.15.47:0/1998956290 conn(0x7f9fd00fee50 msgr2=0x7f9fd0107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:33.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.643+0000 7f9fd6e00640 1 -- 172.21.15.47:0/1998956290 shutdown_connections 2024-03-21T11:50:33.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.643+0000 7f9fd6e00640 1 -- 172.21.15.47:0/1998956290 wait complete. 2024-03-21T11:50:33.661 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T11:50:33.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.880+0000 7f3449a00640 1 -- 172.21.15.47:0/30622205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444101500 msgr2=0x7f3444101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:33.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.880+0000 7f3449a00640 1 --2- 172.21.15.47:0/30622205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444101500 0x7f3444101920 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f3434009900 tx=0x7f343402bc40 comp rx=0 tx=0).stop 2024-03-21T11:50:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.880+0000 7f3449a00640 1 -- 172.21.15.47:0/30622205 shutdown_connections 2024-03-21T11:50:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.880+0000 7f3449a00640 1 --2- 172.21.15.47:0/30622205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34441030b0 0x7f3444109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.880+0000 7f3449a00640 1 --2- 172.21.15.47:0/30622205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34441026f0 0x7f3444102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.880+0000 7f3449a00640 1 --2- 172.21.15.47:0/30622205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444101500 0x7f3444101920 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.880+0000 7f3449a00640 1 -- 172.21.15.47:0/30622205 >> 172.21.15.47:0/30622205 conn(0x7f34440fca70 msgr2=0x7f34440feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.880+0000 7f3449a00640 1 -- 172.21.15.47:0/30622205 shutdown_connections 2024-03-21T11:50:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.880+0000 7f3449a00640 1 -- 172.21.15.47:0/30622205 wait complete. 2024-03-21T11:50:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.881+0000 7f3449a00640 1 Processor -- start 2024-03-21T11:50:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.881+0000 7f3449a00640 1 -- start start 2024-03-21T11:50:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f3449a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3444101500 0x7f34441999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f3449a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34441026f0 0x7f3444199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f3449a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34441030b0 0x7f34441a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f3449a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344410d1e0 con 0x7f3444101500 2024-03-21T11:50:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f3449a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344410d060 con 0x7f34441030b0 2024-03-21T11:50:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f3449a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344410d360 con 0x7f34441026f0 2024-03-21T11:50:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f3443e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3444101500 0x7f34441999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f343be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34441026f0 0x7f3444199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f3443e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3444101500 0x7f34441999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35062/0 (socket says 172.21.15.47:35062) 2024-03-21T11:50:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f3443e00640 1 -- 172.21.15.47:0/3214509971 learned_addr learned my addr 172.21.15.47:0/3214509971 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:50:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f3449000640 1 --2- 172.21.15.47:0/3214509971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34441030b0 0x7f34441a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f343be00640 1 -- 172.21.15.47:0/3214509971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34441030b0 msgr2=0x7f34441a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f343be00640 1 --2- 172.21.15.47:0/3214509971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34441030b0 0x7f34441a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f343be00640 1 -- 172.21.15.47:0/3214509971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3444101500 msgr2=0x7f34441999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f343be00640 1 --2- 172.21.15.47:0/3214509971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3444101500 0x7f34441999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.882+0000 7f343be00640 1 -- 172.21.15.47:0/3214509971 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34441a15c0 con 0x7f34441026f0 2024-03-21T11:50:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.883+0000 7f343be00640 1 --2- 172.21.15.47:0/3214509971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34441026f0 0x7f3444199ee0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d6d0 tx=0x7f342c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.883+0000 7f3441600640 1 -- 172.21.15.47:0/3214509971 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c017070 con 0x7f34441026f0 2024-03-21T11:50:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.883+0000 7f3441600640 1 -- 172.21.15.47:0/3214509971 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f342c009820 con 0x7f34441026f0 2024-03-21T11:50:33.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.883+0000 7f3441600640 1 -- 172.21.15.47:0/3214509971 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c004d00 con 0x7f34441026f0 2024-03-21T11:50:33.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.883+0000 7f3449a00640 1 -- 172.21.15.47:0/3214509971 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34441a18b0 con 0x7f34441026f0 2024-03-21T11:50:33.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.883+0000 7f3449a00640 1 -- 172.21.15.47:0/3214509971 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34441a1e00 con 0x7f34441026f0 2024-03-21T11:50:33.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.884+0000 7f3449a00640 1 -- 172.21.15.47:0/3214509971 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34441a2030 con 0x7f34441026f0 2024-03-21T11:50:33.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.886+0000 7f3441600640 1 -- 172.21.15.47:0/3214509971 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f342c0099d0 con 0x7f34441026f0 2024-03-21T11:50:33.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.887+0000 7f3441600640 1 --2- 172.21.15.47:0/3214509971 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f34280a2f20 0x7f34280a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:33.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.887+0000 7f3441600640 1 -- 172.21.15.47:0/3214509971 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7f342c0cf3e0 con 0x7f34441026f0 2024-03-21T11:50:33.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.888+0000 7f3443e00640 1 --2- 172.21.15.47:0/3214509971 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f34280a2f20 0x7f34280a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:33.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.888+0000 7f3443e00640 1 --2- 172.21.15.47:0/3214509971 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f34280a2f20 0x7f34280a53e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3434005ab0 tx=0x7f343402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:33.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:33.894+0000 7f3441600640 1 -- 172.21.15.47:0/3214509971 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f342c09b7d0 con 0x7f34441026f0 2024-03-21T11:50:34.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.133+0000 7f3449a00640 1 -- 172.21.15.47:0/3214509971 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34441a2200 con 0x7f34441026f0 2024-03-21T11:50:34.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.134+0000 7f3441600640 1 -- 172.21.15.47:0/3214509971 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+11481 (secure 0 0 0) 0x7f342c09f870 con 0x7f34441026f0 2024-03-21T11:50:34.136 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:50:34.136 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/4028210726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":4028210726},{"type":"v1","addr":"172.21.15.47:6835","nonce":4028210726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1314095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1314095873},{"type":"v1","addr":"172.21.15.47:6839","nonce":1314095873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T11:20:48.626416+0000","modified":"2024-03-21T11:49:23.191513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24581,"mds_1":24553,"mds_2":24535,"mds_3":24524,"mds_4":24523},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14466":{"gid":14466,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6841/1742084729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":1742084729},{"type":"v1","addr":"172.21.15.47:6841","nonce":1742084729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24446":{"gid":24446,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/904714287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":904714287},{"type":"v1","addr":"172.21.15.100:6833","nonce":904714287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6833/1794606435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":1794606435},{"type":"v1","addr":"172.21.15.102:6833","nonce":1794606435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6837/1846721633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":1846721633},{"type":"v1","addr":"172.21.15.47:6837","nonce":1846721633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.102:6835/1182573386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1182573386},{"type":"v1","addr":"172.21.15.102:6835","nonce":1182573386}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.100:6837/1118767699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1118767699},{"type":"v1","addr":"172.21.15.100:6837","nonce":1118767699}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.102:6837/1198140504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1198140504},{"type":"v1","addr":"172.21.15.102:6837","nonce":1198140504}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.102:6839/2885246980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":2885246980},{"type":"v1","addr":"172.21.15.102:6839","nonce":2885246980}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/2708345792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":2708345792},{"type":"v1","addr":"172.21.15.100:6839","nonce":2708345792}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_32623":{"gid":32623,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6835/4278877119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4278877119},{"type":"v1","addr":"172.21.15.100:6835","nonce":4278877119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24524,24523,24535,24581,24553]},"id":1}]} 2024-03-21T11:50:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 43 2024-03-21T11:50:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.137+0000 7f3449a00640 1 -- 172.21.15.47:0/3214509971 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f34280a2f20 msgr2=0x7f34280a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.137+0000 7f3449a00640 1 --2- 172.21.15.47:0/3214509971 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f34280a2f20 0x7f34280a53e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3434005ab0 tx=0x7f343402cf90 comp rx=0 tx=0).stop 2024-03-21T11:50:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.137+0000 7f3449a00640 1 -- 172.21.15.47:0/3214509971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34441026f0 msgr2=0x7f3444199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.137+0000 7f3449a00640 1 --2- 172.21.15.47:0/3214509971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34441026f0 0x7f3444199ee0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d6d0 tx=0x7f342c00db90 comp rx=0 tx=0).stop 2024-03-21T11:50:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.137+0000 7f3449a00640 1 -- 172.21.15.47:0/3214509971 shutdown_connections 2024-03-21T11:50:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.137+0000 7f3449a00640 1 --2- 172.21.15.47:0/3214509971 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f34280a2f20 0x7f34280a53e0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.137+0000 7f3449a00640 1 --2- 172.21.15.47:0/3214509971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34441030b0 0x7f34441a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.137+0000 7f3449a00640 1 --2- 172.21.15.47:0/3214509971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34441026f0 0x7f3444199ee0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.137+0000 7f3449a00640 1 --2- 172.21.15.47:0/3214509971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3444101500 0x7f34441999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.138+0000 7f3449a00640 1 -- 172.21.15.47:0/3214509971 >> 172.21.15.47:0/3214509971 conn(0x7f34440fca70 msgr2=0x7f34440738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.138+0000 7f3449a00640 1 -- 172.21.15.47:0/3214509971 shutdown_connections 2024-03-21T11:50:34.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.138+0000 7f3449a00640 1 -- 172.21.15.47:0/3214509971 wait complete. 2024-03-21T11:50:34.158 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T11:50:34.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:34 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1998956290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:50:34.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.371+0000 7fb995000640 1 -- 172.21.15.47:0/830355090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9900772a0 msgr2=0x7fb990077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:34.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.371+0000 7fb995000640 1 --2- 172.21.15.47:0/830355090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9900772a0 0x7fb990077700 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb978009900 tx=0x7fb97802bc30 comp rx=0 tx=0).stop 2024-03-21T11:50:34.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.372+0000 7fb995000640 1 -- 172.21.15.47:0/830355090 shutdown_connections 2024-03-21T11:50:34.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.372+0000 7fb995000640 1 --2- 172.21.15.47:0/830355090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb990077c40 0x7fb9900784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.372+0000 7fb995000640 1 --2- 172.21.15.47:0/830355090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9900772a0 0x7fb990077700 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.372+0000 7fb995000640 1 --2- 172.21.15.47:0/830355090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb990076060 0x7fb990076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.372+0000 7fb995000640 1 -- 172.21.15.47:0/830355090 >> 172.21.15.47:0/830355090 conn(0x7fb9900fee50 msgr2=0x7fb990101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:34.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.372+0000 7fb995000640 1 -- 172.21.15.47:0/830355090 shutdown_connections 2024-03-21T11:50:34.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.372+0000 7fb995000640 1 -- 172.21.15.47:0/830355090 wait complete. 2024-03-21T11:50:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.373+0000 7fb995000640 1 Processor -- start 2024-03-21T11:50:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.373+0000 7fb995000640 1 -- start start 2024-03-21T11:50:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.373+0000 7fb995000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb990076060 0x7fb99019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.373+0000 7fb995000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9900772a0 0x7fb99019e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.373+0000 7fb995000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb990077c40 0x7fb9901a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.373+0000 7fb995000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9901139b0 con 0x7fb990076060 2024-03-21T11:50:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.373+0000 7fb995000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb990113830 con 0x7fb990077c40 2024-03-21T11:50:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.373+0000 7fb995000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb990113b30 con 0x7fb9900772a0 2024-03-21T11:50:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.373+0000 7fb987e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9900772a0 0x7fb99019e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.373+0000 7fb987e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9900772a0 0x7fb99019e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41908/0 (socket says 172.21.15.47:41908) 2024-03-21T11:50:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.373+0000 7fb987e00640 1 -- 172.21.15.47:0/2802248556 learned_addr learned my addr 172.21.15.47:0/2802248556 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:50:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.373+0000 7fb98e800640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb990077c40 0x7fb9901a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.374+0000 7fb98de00640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb990076060 0x7fb99019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.374+0000 7fb987e00640 1 -- 172.21.15.47:0/2802248556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb990077c40 msgr2=0x7fb9901a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.374+0000 7fb987e00640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb990077c40 0x7fb9901a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.374+0000 7fb987e00640 1 -- 172.21.15.47:0/2802248556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb990076060 msgr2=0x7fb99019df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.374+0000 7fb987e00640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb990076060 0x7fb99019df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.374+0000 7fb987e00640 1 -- 172.21.15.47:0/2802248556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9901a5bb0 con 0x7fb9900772a0 2024-03-21T11:50:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.374+0000 7fb98de00640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb990076060 0x7fb99019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:50:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.374+0000 7fb987e00640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9900772a0 0x7fb99019e4d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fb978005ab0 tx=0x7fb97802c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.374+0000 7fb986a00640 1 -- 172.21.15.47:0/2802248556 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978043020 con 0x7fb9900772a0 2024-03-21T11:50:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.375+0000 7fb986a00640 1 -- 172.21.15.47:0/2802248556 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9780054e0 con 0x7fb9900772a0 2024-03-21T11:50:34.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.375+0000 7fb986a00640 1 -- 172.21.15.47:0/2802248556 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978003ce0 con 0x7fb9900772a0 2024-03-21T11:50:34.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.375+0000 7fb995000640 1 -- 172.21.15.47:0/2802248556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9901a5e40 con 0x7fb9900772a0 2024-03-21T11:50:34.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.375+0000 7fb995000640 1 -- 172.21.15.47:0/2802248556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9901a62b0 con 0x7fb9900772a0 2024-03-21T11:50:34.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.376+0000 7fb995000640 1 -- 172.21.15.47:0/2802248556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9901103a0 con 0x7fb9900772a0 2024-03-21T11:50:34.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.377+0000 7fb986a00640 1 -- 172.21.15.47:0/2802248556 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb978046020 con 0x7fb9900772a0 2024-03-21T11:50:34.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.379+0000 7fb986a00640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9740a2f20 0x7fb9740a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:34.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.379+0000 7fb986a00640 1 -- 172.21.15.47:0/2802248556 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fb9780f1930 con 0x7fb9900772a0 2024-03-21T11:50:34.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.379+0000 7fb98de00640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9740a2f20 0x7fb9740a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:34.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.379+0000 7fb98de00640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9740a2f20 0x7fb9740a53e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb980009c20 tx=0x7fb980005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:34.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.382+0000 7fb986a00640 1 -- 172.21.15.47:0/2802248556 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb97804d020 con 0x7fb9900772a0 2024-03-21T11:50:34.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:34 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1998956290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:50:34.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:34 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1998956290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:50:34.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.580+0000 7fb995000640 1 -- 172.21.15.47:0/2802248556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9900623a0 con 0x7fb9900772a0 2024-03-21T11:50:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.582+0000 7fb986a00640 1 -- 172.21.15.47:0/2802248556 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24747 (secure 0 0 0) 0x7fb9780be490 con 0x7fb9900772a0 2024-03-21T11:50:34.583 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T11:50:34.584 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":96,"fsid":"a6f35682-e773-11ee-95cd-87774f69a715","created":"2024-03-21T11:11:41.726008+0000","modified":"2024-03-21T11:45:38.900262+0000","last_up_change":"2024-03-21T11:19:49.706942+0000","last_in_change":"2024-03-21T11:19:30.070811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T11:15:35.545245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T11:20:47.016174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T11:20:48.108506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T11:20:52.254154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68bd29ba-c4c1-482d-b233-47e42371a12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6803","nonce":3777145460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6805","nonce":3777145460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6809","nonce":3777145460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3777145460},{"type":"v1","addr":"172.21.15.47:6807","nonce":3777145460}]},"public_addr":"172.21.15.47:6803/3777145460","cluster_addr":"172.21.15.47:6805/3777145460","heartbeat_back_addr":"172.21.15.47:6809/3777145460","heartbeat_front_addr":"172.21.15.47:6807/3777145460","state":["exists","up"]},{"osd":1,"uuid":"c44724d5-59e4-4520-83d9-7e5f4fd085a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6801","nonce":2612500436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6803","nonce":2612500436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6807","nonce":2612500436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2612500436},{"type":"v1","addr":"172.21.15.100:6805","nonce":2612500436}]},"public_addr":"172.21.15.100:6801/2612500436","cluster_addr":"172.21.15.100:6803/2612500436","heartbeat_back_addr":"172.21.15.100:6807/2612500436","heartbeat_front_addr":"172.21.15.100:6805/2612500436","state":["exists","up"]},{"osd":2,"uuid":"712cd8e5-7760-4534-8f80-71e7acd5fae9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6801","nonce":644548261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6803","nonce":644548261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6807","nonce":644548261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":644548261},{"type":"v1","addr":"172.21.15.102:6805","nonce":644548261}]},"public_addr":"172.21.15.102:6801/644548261","cluster_addr":"172.21.15.102:6803/644548261","heartbeat_back_addr":"172.21.15.102:6807/644548261","heartbeat_front_addr":"172.21.15.102:6805/644548261","state":["exists","up"]},{"osd":3,"uuid":"182c6d29-67dd-4cc1-b79f-7d106f2cc0ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6811","nonce":534982877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6813","nonce":534982877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6817","nonce":534982877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":534982877},{"type":"v1","addr":"172.21.15.47:6815","nonce":534982877}]},"public_addr":"172.21.15.47:6811/534982877","cluster_addr":"172.21.15.47:6813/534982877","heartbeat_back_addr":"172.21.15.47:6817/534982877","heartbeat_front_addr":"172.21.15.47:6815/534982877","state":["exists","up"]},{"osd":4,"uuid":"65901f21-1ccb-45d6-90a9-8f40d2dfaf28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6809","nonce":1323994971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6811","nonce":1323994971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6815","nonce":1323994971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1323994971},{"type":"v1","addr":"172.21.15.100:6813","nonce":1323994971}]},"public_addr":"172.21.15.100:6809/1323994971","cluster_addr":"172.21.15.100:6811/1323994971","heartbeat_back_addr":"172.21.15.100:6815/1323994971","heartbeat_front_addr":"172.21.15.100:6813/1323994971","state":["exists","up"]},{"osd":5,"uuid":"d02dcbc9-f19e-43ef-9b63-9c7039111849","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6809","nonce":2128847052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6811","nonce":2128847052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6815","nonce":2128847052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2128847052},{"type":"v1","addr":"172.21.15.102:6813","nonce":2128847052}]},"public_addr":"172.21.15.102:6809/2128847052","cluster_addr":"172.21.15.102:6811/2128847052","heartbeat_back_addr":"172.21.15.102:6815/2128847052","heartbeat_front_addr":"172.21.15.102:6813/2128847052","state":["exists","up"]},{"osd":6,"uuid":"fbe3b7c0-943f-4849-a8b5-5d792e1f494b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6819","nonce":2684410491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6821","nonce":2684410491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6825","nonce":2684410491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2684410491},{"type":"v1","addr":"172.21.15.47:6823","nonce":2684410491}]},"public_addr":"172.21.15.47:6819/2684410491","cluster_addr":"172.21.15.47:6821/2684410491","heartbeat_back_addr":"172.21.15.47:6825/2684410491","heartbeat_front_addr":"172.21.15.47:6823/2684410491","state":["exists","up"]},{"osd":7,"uuid":"53cea2b7-6bbe-4b15-b28e-7fdddd70607e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6817","nonce":3018830551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6819","nonce":3018830551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6823","nonce":3018830551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3018830551},{"type":"v1","addr":"172.21.15.100:6821","nonce":3018830551}]},"public_addr":"172.21.15.100:6817/3018830551","cluster_addr":"172.21.15.100:6819/3018830551","heartbeat_back_addr":"172.21.15.100:6823/3018830551","heartbeat_front_addr":"172.21.15.100:6821/3018830551","state":["exists","up"]},{"osd":8,"uuid":"20bf15de-3238-4613-9cd2-6bcc5ba6d60f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6817","nonce":1509099917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6819","nonce":1509099917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6823","nonce":1509099917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":1509099917},{"type":"v1","addr":"172.21.15.102:6821","nonce":1509099917}]},"public_addr":"172.21.15.102:6817/1509099917","cluster_addr":"172.21.15.102:6819/1509099917","heartbeat_back_addr":"172.21.15.102:6823/1509099917","heartbeat_front_addr":"172.21.15.102:6821/1509099917","state":["exists","up"]},{"osd":9,"uuid":"9f22490f-23d7-4822-b6f6-57f2da5e0514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6827","nonce":3319373812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6829","nonce":3319373812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6833","nonce":3319373812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3319373812},{"type":"v1","addr":"172.21.15.47:6831","nonce":3319373812}]},"public_addr":"172.21.15.47:6827/3319373812","cluster_addr":"172.21.15.47:6829/3319373812","heartbeat_back_addr":"172.21.15.47:6833/3319373812","heartbeat_front_addr":"172.21.15.47:6831/3319373812","state":["exists","up"]},{"osd":10,"uuid":"22ad02d6-55e9-4ec3-96e1-b24d6d1222f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6825","nonce":1645855291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6827","nonce":1645855291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6831","nonce":1645855291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1645855291},{"type":"v1","addr":"172.21.15.100:6829","nonce":1645855291}]},"public_addr":"172.21.15.100:6825/1645855291","cluster_addr":"172.21.15.100:6827/1645855291","heartbeat_back_addr":"172.21.15.100:6831/1645855291","heartbeat_front_addr":"172.21.15.100:6829/1645855291","state":["exists","up"]},{"osd":11,"uuid":"756cdfb6-361e-494f-89a0-ad0d560f9675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6825","nonce":557348355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6827","nonce":557348355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6831","nonce":557348355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":557348355},{"type":"v1","addr":"172.21.15.102:6829","nonce":557348355}]},"public_addr":"172.21.15.102:6825/557348355","cluster_addr":"172.21.15.102:6827/557348355","heartbeat_back_addr":"172.21.15.102:6831/557348355","heartbeat_front_addr":"172.21.15.102:6829/557348355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:14:43.000462+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:08.009369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:15:31.117631+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:05.015933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:30.523345+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:16:55.309129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:29.289090+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:17:55.754503+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:20.704768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:18:55.170559+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:22.925794+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T11:19:48.053886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/4277663939":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6800/185270192":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/1272635669":"2024-03-22T11:12:48.837005+0000","172.21.15.47:0/2508263463":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3773889790":"2024-03-22T11:12:48.837005+0000","172.21.15.47:6801/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/3130329909":"2024-03-22T11:12:14.713084+0000","172.21.15.47:6800/1584030551":"2024-03-22T11:12:14.713084+0000","172.21.15.47:0/335539055":"2024-03-22T11:12:14.713084+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T11:50:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.584+0000 7fb995000640 1 -- 172.21.15.47:0/2802248556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9740a2f20 msgr2=0x7fb9740a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.585+0000 7fb995000640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9740a2f20 0x7fb9740a53e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb980009c20 tx=0x7fb980005f90 comp rx=0 tx=0).stop 2024-03-21T11:50:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.585+0000 7fb995000640 1 -- 172.21.15.47:0/2802248556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9900772a0 msgr2=0x7fb99019e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.585+0000 7fb995000640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9900772a0 0x7fb99019e4d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fb978005ab0 tx=0x7fb97802c6c0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.585+0000 7fb995000640 1 -- 172.21.15.47:0/2802248556 shutdown_connections 2024-03-21T11:50:34.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.585+0000 7fb995000640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb9740a2f20 0x7fb9740a53e0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.585+0000 7fb995000640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb990077c40 0x7fb9901a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.585+0000 7fb995000640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9900772a0 0x7fb99019e4d0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.585+0000 7fb995000640 1 --2- 172.21.15.47:0/2802248556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb990076060 0x7fb99019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.585+0000 7fb995000640 1 -- 172.21.15.47:0/2802248556 >> 172.21.15.47:0/2802248556 conn(0x7fb9900fee50 msgr2=0x7fb9901004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:34.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.585+0000 7fb995000640 1 -- 172.21.15.47:0/2802248556 shutdown_connections 2024-03-21T11:50:34.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.585+0000 7fb995000640 1 -- 172.21.15.47:0/2802248556 wait complete. 2024-03-21T11:50:34.605 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-21T11:50:34.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.806+0000 7fdb01000640 1 -- 172.21.15.47:0/1891429293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc100480 msgr2=0x7fdafc1008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:34.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.806+0000 7fdb01000640 1 --2- 172.21.15.47:0/1891429293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc100480 0x7fdafc1008a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009ca0 tx=0x7fdaec02c0f0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.806+0000 7fdb01000640 1 -- 172.21.15.47:0/1891429293 shutdown_connections 2024-03-21T11:50:34.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.806+0000 7fdb01000640 1 --2- 172.21.15.47:0/1891429293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdafc102030 0x7fdafc1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.806+0000 7fdb01000640 1 --2- 172.21.15.47:0/1891429293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdafc101670 0x7fdafc101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.806+0000 7fdb01000640 1 --2- 172.21.15.47:0/1891429293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc100480 0x7fdafc1008a0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.806+0000 7fdb01000640 1 -- 172.21.15.47:0/1891429293 >> 172.21.15.47:0/1891429293 conn(0x7fdafc0fba10 msgr2=0x7fdafc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:34.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.806+0000 7fdb01000640 1 -- 172.21.15.47:0/1891429293 shutdown_connections 2024-03-21T11:50:34.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.806+0000 7fdb01000640 1 -- 172.21.15.47:0/1891429293 wait complete. 2024-03-21T11:50:34.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.807+0000 7fdb01000640 1 Processor -- start 2024-03-21T11:50:34.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.808+0000 7fdb01000640 1 -- start start 2024-03-21T11:50:34.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.808+0000 7fdb01000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdafc100480 0x7fdafc104410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:34.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.808+0000 7fdb01000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc101670 0x7fdafc102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:34.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.808+0000 7fdb01000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdafc102030 0x7fdafc102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:34.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.808+0000 7fdb01000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc10c110 con 0x7fdafc102030 2024-03-21T11:50:34.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.808+0000 7fdb01000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc10bf90 con 0x7fdafc101670 2024-03-21T11:50:34.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.808+0000 7fdb01000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc10c290 con 0x7fdafc100480 2024-03-21T11:50:34.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.808+0000 7fdafb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdafc100480 0x7fdafc104410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:34.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.808+0000 7fdafb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdafc100480 0x7fdafc104410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:41924/0 (socket says 172.21.15.47:41924) 2024-03-21T11:50:34.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.808+0000 7fdafb400640 1 -- 172.21.15.47:0/170881598 learned_addr learned my addr 172.21.15.47:0/170881598 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:50:34.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.808+0000 7fdaf2a00640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc101670 0x7fdafc102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:34.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.808+0000 7fdafbe00640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdafc102030 0x7fdafc102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:34.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.809+0000 7fdafb400640 1 -- 172.21.15.47:0/170881598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc101670 msgr2=0x7fdafc102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:34.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.809+0000 7fdafb400640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc101670 0x7fdafc102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.809+0000 7fdafb400640 1 -- 172.21.15.47:0/170881598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdafc102030 msgr2=0x7fdafc102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:34.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.809+0000 7fdafb400640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdafc102030 0x7fdafc102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:34.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.809+0000 7fdafb400640 1 -- 172.21.15.47:0/170881598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc103820 con 0x7fdafc100480 2024-03-21T11:50:34.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.809+0000 7fdafbe00640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdafc102030 0x7fdafc102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T11:50:34.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.809+0000 7fdafb400640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdafc100480 0x7fdafc104410 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fdaec000f80 tx=0x7fdaec032ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:34.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.809+0000 7fdaf8c00640 1 -- 172.21.15.47:0/170881598 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec041020 con 0x7fdafc100480 2024-03-21T11:50:34.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.809+0000 7fdaf8c00640 1 -- 172.21.15.47:0/170881598 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdaec034cd0 con 0x7fdafc100480 2024-03-21T11:50:34.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.809+0000 7fdb01000640 1 -- 172.21.15.47:0/170881598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdafc1a1950 con 0x7fdafc100480 2024-03-21T11:50:34.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.809+0000 7fdaf8c00640 1 -- 172.21.15.47:0/170881598 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec039040 con 0x7fdafc100480 2024-03-21T11:50:34.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.810+0000 7fdb01000640 1 -- 172.21.15.47:0/170881598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdafc1a1ec0 con 0x7fdafc100480 2024-03-21T11:50:34.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.811+0000 7fdb01000640 1 -- 172.21.15.47:0/170881598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdafc0683b0 con 0x7fdafc100480 2024-03-21T11:50:34.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.812+0000 7fdaf8c00640 1 -- 172.21.15.47:0/170881598 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdaec044020 con 0x7fdafc100480 2024-03-21T11:50:34.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.814+0000 7fdaf8c00640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdae00a2fa0 0x7fdae00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:34.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.814+0000 7fdaf8c00640 1 -- 172.21.15.47:0/170881598 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9150+0+0 (secure 0 0 0) 0x7fdaec0f1790 con 0x7fdafc100480 2024-03-21T11:50:34.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.814+0000 7fdaf2a00640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdae00a2fa0 0x7fdae00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:34.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.815+0000 7fdaf2a00640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdae00a2fa0 0x7fdae00a5460 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fdafc104180 tx=0x7fdae4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:34.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:34.821+0000 7fdaf8c00640 1 -- 172.21.15.47:0/170881598 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdaec04b020 con 0x7fdafc100480 2024-03-21T11:50:35.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.066+0000 7fdb01000640 1 -- 172.21.15.47:0/170881598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdafc1a2250 con 0x7fdafc100480 2024-03-21T11:50:35.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.076+0000 7fdaf8c00640 1 -- 172.21.15.47:0/170881598 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdaec0be1d0 con 0x7fdafc100480 2024-03-21T11:50:35.078 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-21T11:50:35.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.081+0000 7fdb01000640 1 -- 172.21.15.47:0/170881598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdae00a2fa0 msgr2=0x7fdae00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:35.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.081+0000 7fdb01000640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdae00a2fa0 0x7fdae00a5460 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fdafc104180 tx=0x7fdae4005f90 comp rx=0 tx=0).stop 2024-03-21T11:50:35.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.081+0000 7fdb01000640 1 -- 172.21.15.47:0/170881598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdafc100480 msgr2=0x7fdafc104410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:35.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.081+0000 7fdb01000640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdafc100480 0x7fdafc104410 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fdaec000f80 tx=0x7fdaec032ec0 comp rx=0 tx=0).stop 2024-03-21T11:50:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.082+0000 7fdb01000640 1 -- 172.21.15.47:0/170881598 shutdown_connections 2024-03-21T11:50:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.082+0000 7fdb01000640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fdae00a2fa0 0x7fdae00a5460 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.082+0000 7fdb01000640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdafc102030 0x7fdafc102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.082+0000 7fdb01000640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdafc101670 0x7fdafc102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.082+0000 7fdb01000640 1 --2- 172.21.15.47:0/170881598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdafc100480 0x7fdafc104410 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.082+0000 7fdb01000640 1 -- 172.21.15.47:0/170881598 >> 172.21.15.47:0/170881598 conn(0x7fdafc0fba10 msgr2=0x7fdafc0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.082+0000 7fdb01000640 1 -- 172.21.15.47:0/170881598 shutdown_connections 2024-03-21T11:50:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.082+0000 7fdb01000640 1 -- 172.21.15.47:0/170881598 wait complete. 2024-03-21T11:50:35.113 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-21T11:50:35.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.302+0000 7fe3c6e00640 1 -- 172.21.15.47:0/4015946681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c0076060 msgr2=0x7fe3c0076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:35.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.302+0000 7fe3c6e00640 1 --2- 172.21.15.47:0/4015946681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c0076060 0x7fe3c0076480 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009900 tx=0x7fe3b002bc30 comp rx=0 tx=0).stop 2024-03-21T11:50:35.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.303+0000 7fe3c6e00640 1 -- 172.21.15.47:0/4015946681 shutdown_connections 2024-03-21T11:50:35.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.303+0000 7fe3c6e00640 1 --2- 172.21.15.47:0/4015946681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3c0077c40 0x7fe3c00784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:35.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.303+0000 7fe3c6e00640 1 --2- 172.21.15.47:0/4015946681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c00772a0 0x7fe3c0077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:35.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.303+0000 7fe3c6e00640 1 --2- 172.21.15.47:0/4015946681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c0076060 0x7fe3c0076480 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:35.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.303+0000 7fe3c6e00640 1 -- 172.21.15.47:0/4015946681 >> 172.21.15.47:0/4015946681 conn(0x7fe3c00fee50 msgr2=0x7fe3c0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:35.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.303+0000 7fe3c6e00640 1 -- 172.21.15.47:0/4015946681 shutdown_connections 2024-03-21T11:50:35.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.303+0000 7fe3c6e00640 1 -- 172.21.15.47:0/4015946681 wait complete. 2024-03-21T11:50:35.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.304+0000 7fe3c6e00640 1 Processor -- start 2024-03-21T11:50:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.304+0000 7fe3c6e00640 1 -- start start 2024-03-21T11:50:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.304+0000 7fe3c6e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c0076060 0x7fe3c019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.304+0000 7fe3c6e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3c00772a0 0x7fe3c019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.304+0000 7fe3c6e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c0077c40 0x7fe3c01a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.304+0000 7fe3c6e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c0113970 con 0x7fe3c0076060 2024-03-21T11:50:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.304+0000 7fe3c6e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c01137f0 con 0x7fe3c00772a0 2024-03-21T11:50:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.304+0000 7fe3c6e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c0113af0 con 0x7fe3c0077c40 2024-03-21T11:50:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.304+0000 7fe3bfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c0076060 0x7fe3c019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.305+0000 7fe3bfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c0076060 0x7fe3c019e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35124/0 (socket says 172.21.15.47:35124) 2024-03-21T11:50:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.305+0000 7fe3bfe00640 1 -- 172.21.15.47:0/2800036674 learned_addr learned my addr 172.21.15.47:0/2800036674 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:50:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.305+0000 7fe3bf400640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3c00772a0 0x7fe3c019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:35.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.305+0000 7fe3c4e00640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c0077c40 0x7fe3c01a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.305+0000 7fe3bfe00640 1 -- 172.21.15.47:0/2800036674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c0077c40 msgr2=0x7fe3c01a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.305+0000 7fe3bfe00640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c0077c40 0x7fe3c01a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.305+0000 7fe3bfe00640 1 -- 172.21.15.47:0/2800036674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3c00772a0 msgr2=0x7fe3c019e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.305+0000 7fe3bfe00640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3c00772a0 0x7fe3c019e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.305+0000 7fe3bfe00640 1 -- 172.21.15.47:0/2800036674 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c01a5ca0 con 0x7fe3c0076060 2024-03-21T11:50:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.305+0000 7fe3bf400640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3c00772a0 0x7fe3c019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:50:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.305+0000 7fe3bfe00640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c0076060 0x7fe3c019e080 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0005ab0 tx=0x7fe3b002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.306+0000 7fe3bcc00640 1 -- 172.21.15.47:0/2800036674 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0041070 con 0x7fe3c0076060 2024-03-21T11:50:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.306+0000 7fe3bcc00640 1 -- 172.21.15.47:0/2800036674 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3b0004450 con 0x7fe3c0076060 2024-03-21T11:50:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.306+0000 7fe3c6e00640 1 -- 172.21.15.47:0/2800036674 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c01a5f30 con 0x7fe3c0076060 2024-03-21T11:50:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.306+0000 7fe3bcc00640 1 -- 172.21.15.47:0/2800036674 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0003e50 con 0x7fe3c0076060 2024-03-21T11:50:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.306+0000 7fe3c6e00640 1 -- 172.21.15.47:0/2800036674 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c01a6390 con 0x7fe3c0076060 2024-03-21T11:50:35.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.307+0000 7fe3c6e00640 1 -- 172.21.15.47:0/2800036674 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c0110250 con 0x7fe3c0076060 2024-03-21T11:50:35.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.309+0000 7fe3bcc00640 1 -- 172.21.15.47:0/2800036674 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe3b0044070 con 0x7fe3c0076060 2024-03-21T11:50:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.310+0000 7fe3bcc00640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe3a40a3140 0x7fe3a40a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.310+0000 7fe3bcc00640 1 -- 172.21.15.47:0/2800036674 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9580+0+0 (secure 0 0 0) 0x7fe3b00f1490 con 0x7fe3c0076060 2024-03-21T11:50:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.310+0000 7fe3bf400640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe3a40a3140 0x7fe3a40a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:35.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.311+0000 7fe3bf400640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe3a40a3140 0x7fe3a40a5600 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe3c019f590 tx=0x7fe3a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:35.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:35 smithi047 ceph-mon[31351]: pgmap v1218: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-21T11:50:35.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:35 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/3214509971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:50:35.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:35 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2802248556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:50:35.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:35 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a[31347]: 2024-03-21T11:50:35.068+0000 7ffa20c00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T11:50:35.314 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:50:35 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-d[69563]: ignoring --setuser ceph since I am not root 2024-03-21T11:50:35.314 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:50:35 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-d[69563]: ignoring --setgroup ceph since I am not root 2024-03-21T11:50:35.314 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:50:35 smithi047 ceph-mds[69574]: -- 172.21.15.47:0/2092805617 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5573833e3a40 con 0x5573833b6c00 2024-03-21T11:50:35.314 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:50:35 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-d[69563]: starting mds.d at 2024-03-21T11:50:35.315 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:50:35 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-a[70549]: ignoring --setuser ceph since I am not root 2024-03-21T11:50:35.315 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:50:35 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-a[70549]: ignoring --setgroup ceph since I am not root 2024-03-21T11:50:35.315 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:50:35 smithi047 ceph-mds[70572]: -- 172.21.15.47:0/3569533317 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5568dd249a40 con 0x5568dd21d000 2024-03-21T11:50:35.315 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:50:35 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-a[70549]: starting mds.a at 2024-03-21T11:50:35.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.315+0000 7fe3bcc00640 1 -- 172.21.15.47:0/2800036674 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe3b00bd6d0 con 0x7fe3c0076060 2024-03-21T11:50:35.350 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-mon[36245]: pgmap v1218: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-21T11:50:35.350 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/3214509971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:50:35.350 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2802248556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:50:35.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-mon[36748]: pgmap v1218: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-21T11:50:35.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/3214509971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T11:50:35.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2802248556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T11:50:35.492 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-f[67371]: ignoring --setuser ceph since I am not root 2024-03-21T11:50:35.492 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-f[67371]: ignoring --setgroup ceph since I am not root 2024-03-21T11:50:35.492 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-mds[67375]: -- 172.21.15.102:0/726640898 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5579c6e67a40 con 0x5579c6e3ac00 2024-03-21T11:50:35.492 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-f[67371]: starting mds.f at 2024-03-21T11:50:35.492 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-l[67062]: ignoring --setuser ceph since I am not root 2024-03-21T11:50:35.493 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-l[67062]: ignoring --setgroup ceph since I am not root 2024-03-21T11:50:35.493 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-mds[67066]: -- 172.21.15.102:0/311416355 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cb72937a40 con 0x55cb7290ac00 2024-03-21T11:50:35.493 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-l[67062]: starting mds.l at 2024-03-21T11:50:35.493 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-c[67682]: ignoring --setuser ceph since I am not root 2024-03-21T11:50:35.493 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-c[67682]: ignoring --setgroup ceph since I am not root 2024-03-21T11:50:35.493 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-mds[67686]: -- 172.21.15.102:0/4121303917 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b97a6ada40 con 0x55b97a680800 2024-03-21T11:50:35.493 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-c[67682]: starting mds.c at 2024-03-21T11:50:35.493 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-i[67997]: ignoring --setuser ceph since I am not root 2024-03-21T11:50:35.494 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-i[67997]: ignoring --setgroup ceph since I am not root 2024-03-21T11:50:35.494 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-mds[68001]: -- 172.21.15.102:0/3683045332 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55bd6617c4e0 con 0x55bd66153000 2024-03-21T11:50:35.494 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-mds[68001]: -- 172.21.15.102:0/3683045332 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bd6617fa40 con 0x55bd66153000 2024-03-21T11:50:35.494 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:50:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-i[67997]: starting mds.i at 2024-03-21T11:50:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:35.558+0000 7fe3c6e00640 1 -- 172.21.15.47:0/2800036674 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe3c002d2b0 con 0x7fe3c0076060 2024-03-21T11:50:35.643 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b[66324]: ignoring --setuser ceph since I am not root 2024-03-21T11:50:35.643 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b[66324]: ignoring --setgroup ceph since I am not root 2024-03-21T11:50:35.643 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-mds[66328]: -- 172.21.15.100:0/3114988120 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ea269a1a40 con 0x55ea26974c00 2024-03-21T11:50:35.643 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b[66324]: starting mds.b at 2024-03-21T11:50:35.644 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e[73627]: ignoring --setuser ceph since I am not root 2024-03-21T11:50:35.644 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e[73627]: ignoring --setgroup ceph since I am not root 2024-03-21T11:50:35.644 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-mds[73631]: -- 172.21.15.100:0/4233818122 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5600d72884e0 con 0x5600d725ec00 2024-03-21T11:50:35.644 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-mds[73631]: -- 172.21.15.100:0/4233818122 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5600d728ba40 con 0x5600d725ec00 2024-03-21T11:50:35.644 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e[73627]: starting mds.e at 2024-03-21T11:50:35.644 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-h[66002]: ignoring --setuser ceph since I am not root 2024-03-21T11:50:35.644 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-h[66002]: ignoring --setgroup ceph since I am not root 2024-03-21T11:50:35.644 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-mds[66006]: -- 172.21.15.100:0/3640481432 <== mon.1 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5626a107ba40 con 0x5626a104f000 2024-03-21T11:50:35.644 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-h[66002]: starting mds.h at 2024-03-21T11:50:35.645 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-k[65371]: ignoring --setuser ceph since I am not root 2024-03-21T11:50:35.645 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-k[65371]: ignoring --setgroup ceph since I am not root 2024-03-21T11:50:35.645 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-mds[65375]: -- 172.21.15.100:0/2834272864 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55701d787a40 con 0x55701d75a800 2024-03-21T11:50:35.645 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:50:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-k[65371]: starting mds.k at 2024-03-21T11:50:36.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.074+0000 7fe3bcc00640 1 -- 172.21.15.47:0/2800036674 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe3b00c1770 con 0x7fe3c0076060 2024-03-21T11:50:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.077+0000 7fe3c6e00640 1 -- 172.21.15.47:0/2800036674 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe3a40a3140 msgr2=0x7fe3a40a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.077+0000 7fe3c6e00640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe3a40a3140 0x7fe3a40a5600 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe3c019f590 tx=0x7fe3a8005f90 comp rx=0 tx=0).stop 2024-03-21T11:50:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.077+0000 7fe3c6e00640 1 -- 172.21.15.47:0/2800036674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c0076060 msgr2=0x7fe3c019e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.077+0000 7fe3c6e00640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c0076060 0x7fe3c019e080 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0005ab0 tx=0x7fe3b002c6c0 comp rx=0 tx=0).stop 2024-03-21T11:50:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.077+0000 7fe3c6e00640 1 -- 172.21.15.47:0/2800036674 shutdown_connections 2024-03-21T11:50:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.077+0000 7fe3c6e00640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fe3a40a3140 0x7fe3a40a5600 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.077+0000 7fe3c6e00640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c0077c40 0x7fe3c01a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.077+0000 7fe3c6e00640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3c00772a0 0x7fe3c019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.077+0000 7fe3c6e00640 1 --2- 172.21.15.47:0/2800036674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3c0076060 0x7fe3c019e080 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.077+0000 7fe3c6e00640 1 -- 172.21.15.47:0/2800036674 >> 172.21.15.47:0/2800036674 conn(0x7fe3c00fee50 msgr2=0x7fe3c0100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.077+0000 7fe3c6e00640 1 -- 172.21.15.47:0/2800036674 shutdown_connections 2024-03-21T11:50:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.077+0000 7fe3c6e00640 1 -- 172.21.15.47:0/2800036674 wait complete. 2024-03-21T11:50:36.104 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-21T11:50:36.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.292+0000 7fc437200640 1 -- 172.21.15.47:0/3340628421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc438100480 msgr2=0x7fc4381008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:36.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.292+0000 7fc437200640 1 --2- 172.21.15.47:0/3340628421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc438100480 0x7fc4381008a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc428009900 tx=0x7fc42802bc50 comp rx=0 tx=0).stop 2024-03-21T11:50:36.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.292+0000 7fc437200640 1 -- 172.21.15.47:0/3340628421 shutdown_connections 2024-03-21T11:50:36.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.292+0000 7fc437200640 1 --2- 172.21.15.47:0/3340628421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc438102030 0x7fc4381088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:36.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.292+0000 7fc437200640 1 --2- 172.21.15.47:0/3340628421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc438101670 0x7fc438101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:36.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.292+0000 7fc437200640 1 --2- 172.21.15.47:0/3340628421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc438100480 0x7fc4381008a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:36.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.292+0000 7fc437200640 1 -- 172.21.15.47:0/3340628421 >> 172.21.15.47:0/3340628421 conn(0x7fc4380fba10 msgr2=0x7fc4380fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:36.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.292+0000 7fc437200640 1 -- 172.21.15.47:0/3340628421 shutdown_connections 2024-03-21T11:50:36.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.292+0000 7fc437200640 1 -- 172.21.15.47:0/3340628421 wait complete. 2024-03-21T11:50:36.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.293+0000 7fc437200640 1 Processor -- start 2024-03-21T11:50:36.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.293+0000 7fc437200640 1 -- start start 2024-03-21T11:50:36.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.293+0000 7fc437200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc438100480 0x7fc438199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:36.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.293+0000 7fc437200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc438101670 0x7fc43819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:36.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.294+0000 7fc437200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc438102030 0x7fc4381a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:36.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.294+0000 7fc437200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43810c110 con 0x7fc438101670 2024-03-21T11:50:36.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.294+0000 7fc437200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43810bf90 con 0x7fc438100480 2024-03-21T11:50:36.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.294+0000 7fc437200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43810c290 con 0x7fc438102030 2024-03-21T11:50:36.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.294+0000 7fc435400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc438101670 0x7fc43819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:36.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.294+0000 7fc435400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc438101670 0x7fc43819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35146/0 (socket says 172.21.15.47:35146) 2024-03-21T11:50:36.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.294+0000 7fc435400640 1 -- 172.21.15.47:0/2350104860 learned_addr learned my addr 172.21.15.47:0/2350104860 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:50:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.294+0000 7fc435e00640 1 --2- 172.21.15.47:0/2350104860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc438100480 0x7fc438199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.294+0000 7fc436800640 1 --2- 172.21.15.47:0/2350104860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc438102030 0x7fc4381a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.294+0000 7fc435400640 1 -- 172.21.15.47:0/2350104860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc438102030 msgr2=0x7fc4381a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.294+0000 7fc435400640 1 --2- 172.21.15.47:0/2350104860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc438102030 0x7fc4381a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.294+0000 7fc435400640 1 -- 172.21.15.47:0/2350104860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc438100480 msgr2=0x7fc438199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.294+0000 7fc435400640 1 --2- 172.21.15.47:0/2350104860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc438100480 0x7fc438199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.294+0000 7fc435400640 1 -- 172.21.15.47:0/2350104860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4381a1790 con 0x7fc438101670 2024-03-21T11:50:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.295+0000 7fc435400640 1 --2- 172.21.15.47:0/2350104860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc438101670 0x7fc43819a0b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc42000e9f0 tx=0x7fc42000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.295+0000 7fc42ea00640 1 -- 172.21.15.47:0/2350104860 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42000cd80 con 0x7fc438101670 2024-03-21T11:50:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.295+0000 7fc42ea00640 1 -- 172.21.15.47:0/2350104860 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc420010910 con 0x7fc438101670 2024-03-21T11:50:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.295+0000 7fc437200640 1 -- 172.21.15.47:0/2350104860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4381a1a80 con 0x7fc438101670 2024-03-21T11:50:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.295+0000 7fc42ea00640 1 -- 172.21.15.47:0/2350104860 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420018990 con 0x7fc438101670 2024-03-21T11:50:36.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.295+0000 7fc437200640 1 -- 172.21.15.47:0/2350104860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4381a2020 con 0x7fc438101670 2024-03-21T11:50:36.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.296+0000 7fc437200640 1 -- 172.21.15.47:0/2350104860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4381a21f0 con 0x7fc438101670 2024-03-21T11:50:36.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.298+0000 7fc42ea00640 1 -- 172.21.15.47:0/2350104860 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc420010ac0 con 0x7fc438101670 2024-03-21T11:50:36.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.299+0000 7fc42ea00640 1 --2- 172.21.15.47:0/2350104860 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc41c0a2fa0 0x7fc41c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:36.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.300+0000 7fc42ea00640 1 -- 172.21.15.47:0/2350104860 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9580+0+0 (secure 0 0 0) 0x7fc4200cfca0 con 0x7fc438101670 2024-03-21T11:50:36.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.300+0000 7fc435e00640 1 --2- 172.21.15.47:0/2350104860 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc41c0a2fa0 0x7fc41c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:36.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.301+0000 7fc435e00640 1 --2- 172.21.15.47:0/2350104860 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc41c0a2fa0 0x7fc41c0a5460 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fc428000c00 tx=0x7fc428005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:36.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.304+0000 7fc42ea00640 1 -- 172.21.15.47:0/2350104860 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc42009c530 con 0x7fc438101670 2024-03-21T11:50:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:36 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/170881598' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T11:50:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:36 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T11:50:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:36 smithi047 ceph-mon[31351]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T11:50:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:36 smithi047 ceph-mon[31351]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T11:50:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:36 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-21T11:50:36.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:36 smithi047 ceph-mon[31351]: osdmap e97: 12 total, 12 up, 12 in 2024-03-21T11:50:36.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:36 smithi047 ceph-mon[31351]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-21T11:50:36.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:36 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2800036674' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T11:50:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:36 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/170881598' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T11:50:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:36 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T11:50:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:36 smithi100 ceph-mon[36245]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T11:50:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:36 smithi100 ceph-mon[36245]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T11:50:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:36 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-21T11:50:36.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:36 smithi100 ceph-mon[36245]: osdmap e97: 12 total, 12 up, 12 in 2024-03-21T11:50:36.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:36 smithi100 ceph-mon[36245]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-21T11:50:36.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:36 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2800036674' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T11:50:36.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:36 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/170881598' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T11:50:36.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:36 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T11:50:36.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:36 smithi102 ceph-mon[36748]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T11:50:36.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:36 smithi102 ceph-mon[36748]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T11:50:36.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:36 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-21T11:50:36.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:36 smithi102 ceph-mon[36748]: osdmap e97: 12 total, 12 up, 12 in 2024-03-21T11:50:36.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:36 smithi102 ceph-mon[36748]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-21T11:50:36.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:36 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2800036674' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T11:50:36.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:36.515+0000 7fc437200640 1 -- 172.21.15.47:0/2350104860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc4381a2460 con 0x7fc438101670 2024-03-21T11:50:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.089+0000 7fc42ea00640 1 -- 172.21.15.47:0/2350104860 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc42009be30 con 0x7fc438101670 2024-03-21T11:50:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-03-21T11:50:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.096+0000 7fc437200640 1 -- 172.21.15.47:0/2350104860 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc41c0a2fa0 msgr2=0x7fc41c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.096+0000 7fc437200640 1 --2- 172.21.15.47:0/2350104860 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc41c0a2fa0 0x7fc41c0a5460 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fc428000c00 tx=0x7fc428005fd0 comp rx=0 tx=0).stop 2024-03-21T11:50:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.096+0000 7fc437200640 1 -- 172.21.15.47:0/2350104860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc438101670 msgr2=0x7fc43819a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.096+0000 7fc437200640 1 --2- 172.21.15.47:0/2350104860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc438101670 0x7fc43819a0b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc42000e9f0 tx=0x7fc42000eeb0 comp rx=0 tx=0).stop 2024-03-21T11:50:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.096+0000 7fc437200640 1 -- 172.21.15.47:0/2350104860 shutdown_connections 2024-03-21T11:50:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.096+0000 7fc437200640 1 --2- 172.21.15.47:0/2350104860 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fc41c0a2fa0 0x7fc41c0a5460 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.096+0000 7fc437200640 1 --2- 172.21.15.47:0/2350104860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc438102030 0x7fc4381a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.096+0000 7fc437200640 1 --2- 172.21.15.47:0/2350104860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc438101670 0x7fc43819a0b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.096+0000 7fc437200640 1 --2- 172.21.15.47:0/2350104860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc438100480 0x7fc438199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.096+0000 7fc437200640 1 -- 172.21.15.47:0/2350104860 >> 172.21.15.47:0/2350104860 conn(0x7fc4380fba10 msgr2=0x7fc4380fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.097+0000 7fc437200640 1 -- 172.21.15.47:0/2350104860 shutdown_connections 2024-03-21T11:50:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.097+0000 7fc437200640 1 -- 172.21.15.47:0/2350104860 wait complete. 2024-03-21T11:50:37.123 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-21T11:50:37.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: pgmap v1220: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2024-03-21T11:50:37.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T11:50:37.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T11:50:37.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: Cluster is now healthy 2024-03-21T11:50:37.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2800036674' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-21T11:50:37.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.102:6836/1600663885,v1:172.21.15.102:6838/1600663885] up:boot 2024-03-21T11:50:37.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.102:6837/2309300128,v1:172.21.15.102:6839/2309300128] up:boot 2024-03-21T11:50:37.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.102:6833/4249338257,v1:172.21.15.102:6835/4249338257] up:boot 2024-03-21T11:50:37.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.102:6832/3650573049,v1:172.21.15.102:6834/3650573049] up:boot 2024-03-21T11:50:37.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.47:6836/3031084226,v1:172.21.15.47:6837/3031084226] up:boot 2024-03-21T11:50:37.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.47:6840/2218024412,v1:172.21.15.47:6841/2218024412] up:boot 2024-03-21T11:50:37.315 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.100:6836/3676966593,v1:172.21.15.100:6838/3676966593] up:boot 2024-03-21T11:50:37.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.100:6837/3898418826,v1:172.21.15.100:6839/3898418826] up:boot 2024-03-21T11:50:37.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.100:6832/2796286426,v1:172.21.15.100:6834/2796286426] up:boot 2024-03-21T11:50:37.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: mds.? [v2:172.21.15.100:6833/2360718774,v1:172.21.15.100:6835/2360718774] up:boot 2024-03-21T11:50:37.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: fsmap 12 up:standby 2024-03-21T11:50:37.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T11:50:37.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T11:50:37.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T11:50:37.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T11:50:37.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T11:50:37.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T11:50:37.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T11:50:37.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T11:50:37.316 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T11:50:37.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T11:50:37.317 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:37 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2350104860' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.353+0000 7fb3acc00640 1 -- 172.21.15.47:0/3737311589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3a8104ad0 msgr2=0x7fb3a8104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.353+0000 7fb3acc00640 1 --2- 172.21.15.47:0/3737311589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3a8104ad0 0x7fb3a8104f50 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fb394009860 tx=0x7fb39402bb70 comp rx=0 tx=0).stop 2024-03-21T11:50:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.353+0000 7fb3acc00640 1 -- 172.21.15.47:0/3737311589 shutdown_connections 2024-03-21T11:50:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.353+0000 7fb3acc00640 1 --2- 172.21.15.47:0/3737311589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3a8105490 0x7fb3a810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.353+0000 7fb3acc00640 1 --2- 172.21.15.47:0/3737311589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3a8104ad0 0x7fb3a8104f50 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.353+0000 7fb3acc00640 1 --2- 172.21.15.47:0/3737311589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a81038e0 0x7fb3a8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.353+0000 7fb3acc00640 1 -- 172.21.15.47:0/3737311589 >> 172.21.15.47:0/3737311589 conn(0x7fb3a80fee50 msgr2=0x7fb3a8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.353+0000 7fb3acc00640 1 -- 172.21.15.47:0/3737311589 shutdown_connections 2024-03-21T11:50:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.353+0000 7fb3acc00640 1 -- 172.21.15.47:0/3737311589 wait complete. 2024-03-21T11:50:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.354+0000 7fb3acc00640 1 Processor -- start 2024-03-21T11:50:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.354+0000 7fb3acc00640 1 -- start start 2024-03-21T11:50:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.355+0000 7fb3acc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3a81038e0 0x7fb3a8199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.355+0000 7fb3acc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a8104ad0 0x7fb3a819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.355+0000 7fb3acc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3a8105490 0x7fb3a81a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.355+0000 7fb3acc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a810f560 con 0x7fb3a8104ad0 2024-03-21T11:50:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.355+0000 7fb3acc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a810f3e0 con 0x7fb3a8105490 2024-03-21T11:50:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.355+0000 7fb3acc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a810f6e0 con 0x7fb3a81038e0 2024-03-21T11:50:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.355+0000 7fb3a5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3a81038e0 0x7fb3a8199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.355+0000 7fb3a5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a8104ad0 0x7fb3a819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.355+0000 7fb3a6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3a8105490 0x7fb3a81a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.355+0000 7fb3a5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3a81038e0 0x7fb3a8199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42016/0 (socket says 172.21.15.47:42016) 2024-03-21T11:50:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.355+0000 7fb3a5e00640 1 -- 172.21.15.47:0/81428410 learned_addr learned my addr 172.21.15.47:0/81428410 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:50:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.356+0000 7fb3a5e00640 1 -- 172.21.15.47:0/81428410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3a8105490 msgr2=0x7fb3a81a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.356+0000 7fb3a5e00640 1 --2- 172.21.15.47:0/81428410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3a8105490 0x7fb3a81a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.356+0000 7fb3a5e00640 1 -- 172.21.15.47:0/81428410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a8104ad0 msgr2=0x7fb3a819a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.356+0000 7fb3a5e00640 1 --2- 172.21.15.47:0/81428410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a8104ad0 0x7fb3a819a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.356+0000 7fb3a5e00640 1 -- 172.21.15.47:0/81428410 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a81a1800 con 0x7fb3a81038e0 2024-03-21T11:50:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.356+0000 7fb3a5400640 1 --2- 172.21.15.47:0/81428410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a8104ad0 0x7fb3a819a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:50:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.356+0000 7fb3a5e00640 1 --2- 172.21.15.47:0/81428410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3a81038e0 0x7fb3a8199be0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fb39800e9a0 tx=0x7fb39800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.356+0000 7fb39ea00640 1 -- 172.21.15.47:0/81428410 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39800cd80 con 0x7fb3a81038e0 2024-03-21T11:50:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.356+0000 7fb3acc00640 1 -- 172.21.15.47:0/81428410 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a81a1af0 con 0x7fb3a81038e0 2024-03-21T11:50:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.356+0000 7fb39ea00640 1 -- 172.21.15.47:0/81428410 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb398010910 con 0x7fb3a81038e0 2024-03-21T11:50:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.356+0000 7fb39ea00640 1 -- 172.21.15.47:0/81428410 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398018a40 con 0x7fb3a81038e0 2024-03-21T11:50:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.356+0000 7fb3acc00640 1 -- 172.21.15.47:0/81428410 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a81a2010 con 0x7fb3a81038e0 2024-03-21T11:50:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.358+0000 7fb3acc00640 1 -- 172.21.15.47:0/81428410 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a810ac20 con 0x7fb3a81038e0 2024-03-21T11:50:37.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.359+0000 7fb39ea00640 1 -- 172.21.15.47:0/81428410 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb398010ac0 con 0x7fb3a81038e0 2024-03-21T11:50:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.361+0000 7fb39ea00640 1 --2- 172.21.15.47:0/81428410 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb38c0a2f20 0x7fb38c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.361+0000 7fb39ea00640 1 -- 172.21.15.47:0/81428410 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9026+0+0 (secure 0 0 0) 0x7fb3980cfa70 con 0x7fb3a81038e0 2024-03-21T11:50:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.361+0000 7fb3a5400640 1 --2- 172.21.15.47:0/81428410 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb38c0a2f20 0x7fb38c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.362+0000 7fb3a5400640 1 --2- 172.21.15.47:0/81428410 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb38c0a2f20 0x7fb38c0a53e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb39402c180 tx=0x7fb39402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.365+0000 7fb39ea00640 1 -- 172.21.15.47:0/81428410 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb398053740 con 0x7fb3a81038e0 2024-03-21T11:50:37.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: pgmap v1220: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2024-03-21T11:50:37.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T11:50:37.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T11:50:37.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: Cluster is now healthy 2024-03-21T11:50:37.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2800036674' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-21T11:50:37.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.102:6836/1600663885,v1:172.21.15.102:6838/1600663885] up:boot 2024-03-21T11:50:37.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.102:6837/2309300128,v1:172.21.15.102:6839/2309300128] up:boot 2024-03-21T11:50:37.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.102:6833/4249338257,v1:172.21.15.102:6835/4249338257] up:boot 2024-03-21T11:50:37.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.102:6832/3650573049,v1:172.21.15.102:6834/3650573049] up:boot 2024-03-21T11:50:37.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.47:6836/3031084226,v1:172.21.15.47:6837/3031084226] up:boot 2024-03-21T11:50:37.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.47:6840/2218024412,v1:172.21.15.47:6841/2218024412] up:boot 2024-03-21T11:50:37.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.100:6836/3676966593,v1:172.21.15.100:6838/3676966593] up:boot 2024-03-21T11:50:37.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.100:6837/3898418826,v1:172.21.15.100:6839/3898418826] up:boot 2024-03-21T11:50:37.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.100:6832/2796286426,v1:172.21.15.100:6834/2796286426] up:boot 2024-03-21T11:50:37.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: mds.? [v2:172.21.15.100:6833/2360718774,v1:172.21.15.100:6835/2360718774] up:boot 2024-03-21T11:50:37.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: fsmap 12 up:standby 2024-03-21T11:50:37.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T11:50:37.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T11:50:37.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T11:50:37.394 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T11:50:37.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T11:50:37.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T11:50:37.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T11:50:37.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T11:50:37.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T11:50:37.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T11:50:37.395 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:37 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2350104860' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:37.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: pgmap v1220: 129 pgs: 129 active+clean; 528 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2024-03-21T11:50:37.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T11:50:37.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T11:50:37.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: Cluster is now healthy 2024-03-21T11:50:37.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2800036674' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-21T11:50:37.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.102:6836/1600663885,v1:172.21.15.102:6838/1600663885] up:boot 2024-03-21T11:50:37.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.102:6837/2309300128,v1:172.21.15.102:6839/2309300128] up:boot 2024-03-21T11:50:37.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.102:6833/4249338257,v1:172.21.15.102:6835/4249338257] up:boot 2024-03-21T11:50:37.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.102:6832/3650573049,v1:172.21.15.102:6834/3650573049] up:boot 2024-03-21T11:50:37.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.47:6836/3031084226,v1:172.21.15.47:6837/3031084226] up:boot 2024-03-21T11:50:37.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.47:6840/2218024412,v1:172.21.15.47:6841/2218024412] up:boot 2024-03-21T11:50:37.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.100:6836/3676966593,v1:172.21.15.100:6838/3676966593] up:boot 2024-03-21T11:50:37.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.100:6837/3898418826,v1:172.21.15.100:6839/3898418826] up:boot 2024-03-21T11:50:37.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.100:6832/2796286426,v1:172.21.15.100:6834/2796286426] up:boot 2024-03-21T11:50:37.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: mds.? [v2:172.21.15.100:6833/2360718774,v1:172.21.15.100:6835/2360718774] up:boot 2024-03-21T11:50:37.492 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: fsmap 12 up:standby 2024-03-21T11:50:37.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T11:50:37.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T11:50:37.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T11:50:37.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T11:50:37.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T11:50:37.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T11:50:37.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T11:50:37.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T11:50:37.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T11:50:37.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: from='mgr.14152 172.21.15.47:0/439001110' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T11:50:37.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:37 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2350104860' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:37.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:37.560+0000 7fb3acc00640 1 -- 172.21.15.47:0/81428410 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb3a802d070 con 0x7fb3a81038e0 2024-03-21T11:50:38.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.097+0000 7fb39ea00640 1 -- 172.21.15.47:0/81428410 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb39809c520 con 0x7fb3a81038e0 2024-03-21T11:50:38.099 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-03-21T11:50:38.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.103+0000 7fb3acc00640 1 -- 172.21.15.47:0/81428410 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb38c0a2f20 msgr2=0x7fb38c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:38.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.103+0000 7fb3acc00640 1 --2- 172.21.15.47:0/81428410 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb38c0a2f20 0x7fb38c0a53e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb39402c180 tx=0x7fb39402e040 comp rx=0 tx=0).stop 2024-03-21T11:50:38.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.103+0000 7fb3acc00640 1 -- 172.21.15.47:0/81428410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3a81038e0 msgr2=0x7fb3a8199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:38.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.103+0000 7fb3acc00640 1 --2- 172.21.15.47:0/81428410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3a81038e0 0x7fb3a8199be0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fb39800e9a0 tx=0x7fb39800ee60 comp rx=0 tx=0).stop 2024-03-21T11:50:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.104+0000 7fb3acc00640 1 -- 172.21.15.47:0/81428410 shutdown_connections 2024-03-21T11:50:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.104+0000 7fb3acc00640 1 --2- 172.21.15.47:0/81428410 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7fb38c0a2f20 0x7fb38c0a53e0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.104+0000 7fb3acc00640 1 --2- 172.21.15.47:0/81428410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3a8105490 0x7fb3a81a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.104+0000 7fb3acc00640 1 --2- 172.21.15.47:0/81428410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3a8104ad0 0x7fb3a819a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.104+0000 7fb3acc00640 1 --2- 172.21.15.47:0/81428410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3a81038e0 0x7fb3a8199be0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:38.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.104+0000 7fb3acc00640 1 -- 172.21.15.47:0/81428410 >> 172.21.15.47:0/81428410 conn(0x7fb3a80fee50 msgr2=0x7fb3a8107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:38.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.104+0000 7fb3acc00640 1 -- 172.21.15.47:0/81428410 shutdown_connections 2024-03-21T11:50:38.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.105+0000 7fb3acc00640 1 -- 172.21.15.47:0/81428410 wait complete. 2024-03-21T11:50:38.141 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-21T11:50:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:38 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/2350104860' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T11:50:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:38 smithi100 ceph-mon[36245]: osdmap e98: 12 total, 12 up, 12 in 2024-03-21T11:50:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:38 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/81428410' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:38.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:38 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:38.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.452+0000 7f28d8c00640 1 -- 172.21.15.47:0/442897609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d41038e0 msgr2=0x7f28d4103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:38.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.452+0000 7f28d8c00640 1 --2- 172.21.15.47:0/442897609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d41038e0 0x7f28d4103d00 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009900 tx=0x7f28c402bc50 comp rx=0 tx=0).stop 2024-03-21T11:50:38.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.453+0000 7f28d8c00640 1 -- 172.21.15.47:0/442897609 shutdown_connections 2024-03-21T11:50:38.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.453+0000 7f28d8c00640 1 --2- 172.21.15.47:0/442897609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28d4105490 0x7f28d410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:38.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.453+0000 7f28d8c00640 1 --2- 172.21.15.47:0/442897609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d4104ad0 0x7f28d4104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:38.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.453+0000 7f28d8c00640 1 --2- 172.21.15.47:0/442897609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d41038e0 0x7f28d4103d00 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:38.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.453+0000 7f28d8c00640 1 -- 172.21.15.47:0/442897609 >> 172.21.15.47:0/442897609 conn(0x7f28d40fee50 msgr2=0x7f28d4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.456+0000 7f28d8c00640 1 -- 172.21.15.47:0/442897609 shutdown_connections 2024-03-21T11:50:38.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.456+0000 7f28d8c00640 1 -- 172.21.15.47:0/442897609 wait complete. 2024-03-21T11:50:38.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.457+0000 7f28d8c00640 1 Processor -- start 2024-03-21T11:50:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.457+0000 7f28d8c00640 1 -- start start 2024-03-21T11:50:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.458+0000 7f28d8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28d41038e0 0x7f28d4110b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.458+0000 7f28d8c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d4104ad0 0x7f28d41110c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.458+0000 7f28d8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d4105490 0x7f28d4114110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.458+0000 7f28d8c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d4077920 con 0x7f28d4104ad0 2024-03-21T11:50:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.458+0000 7f28d8c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d40777a0 con 0x7f28d4105490 2024-03-21T11:50:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.458+0000 7f28d8c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d4077aa0 con 0x7f28d41038e0 2024-03-21T11:50:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.458+0000 7f28d2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d4105490 0x7f28d4114110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.458+0000 7f28d1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28d41038e0 0x7f28d4110b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:38.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.458+0000 7f28d1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28d41038e0 0x7f28d4110b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.47:42036/0 (socket says 172.21.15.47:42036) 2024-03-21T11:50:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.458+0000 7f28d1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d4104ad0 0x7f28d41110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.458+0000 7f28d2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d4105490 0x7f28d4114110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48026/0 (socket says 172.21.15.47:48026) 2024-03-21T11:50:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.458+0000 7f28d1e00640 1 -- 172.21.15.47:0/1653193564 learned_addr learned my addr 172.21.15.47:0/1653193564 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T11:50:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.459+0000 7f28d1e00640 1 -- 172.21.15.47:0/1653193564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d4105490 msgr2=0x7f28d4114110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.459+0000 7f28d1e00640 1 --2- 172.21.15.47:0/1653193564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d4105490 0x7f28d4114110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:38.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.459+0000 7f28d1e00640 1 -- 172.21.15.47:0/1653193564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d4104ad0 msgr2=0x7f28d41110c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.459+0000 7f28d1e00640 1 --2- 172.21.15.47:0/1653193564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d4104ad0 0x7f28d41110c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.459+0000 7f28d1e00640 1 -- 172.21.15.47:0/1653193564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d4114780 con 0x7f28d41038e0 2024-03-21T11:50:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.459+0000 7f28d1400640 1 --2- 172.21.15.47:0/1653193564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d4104ad0 0x7f28d41110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T11:50:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.460+0000 7f28d1e00640 1 --2- 172.21.15.47:0/1653193564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28d41038e0 0x7f28d4110b80 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f28c4000c00 tx=0x7f28c4005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.460+0000 7f28caa00640 1 -- 172.21.15.47:0/1653193564 <== mon.1 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4043020 con 0x7f28d41038e0 2024-03-21T11:50:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.460+0000 7f28caa00640 1 -- 172.21.15.47:0/1653193564 <== mon.1 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28c40054e0 con 0x7f28d41038e0 2024-03-21T11:50:38.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.460+0000 7f28caa00640 1 -- 172.21.15.47:0/1653193564 <== mon.1 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4003df0 con 0x7f28d41038e0 2024-03-21T11:50:38.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.464+0000 7f28d8c00640 1 -- 172.21.15.47:0/1653193564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d4114a10 con 0x7f28d41038e0 2024-03-21T11:50:38.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.464+0000 7f28d8c00640 1 -- 172.21.15.47:0/1653193564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d4114f10 con 0x7f28d41038e0 2024-03-21T11:50:38.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.466+0000 7f28d8c00640 1 -- 172.21.15.47:0/1653193564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d4115230 con 0x7f28d41038e0 2024-03-21T11:50:38.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.471+0000 7f28caa00640 1 -- 172.21.15.47:0/1653193564 <== mon.1 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f28c4046020 con 0x7f28d41038e0 2024-03-21T11:50:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.473+0000 7f28caa00640 1 --2- 172.21.15.47:0/1653193564 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28b80a3070 0x7f28b80a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T11:50:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.473+0000 7f28caa00640 1 -- 172.21.15.47:0/1653193564 <== mon.1 v2:172.21.15.102:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8496+0+0 (secure 0 0 0) 0x7f28c40f17e0 con 0x7f28d41038e0 2024-03-21T11:50:38.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.474+0000 7f28d1400640 1 --2- 172.21.15.47:0/1653193564 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28b80a3070 0x7f28b80a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T11:50:38.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.475+0000 7f28d1400640 1 --2- 172.21.15.47:0/1653193564 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28b80a3070 0x7f28b80a5530 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f28d4112090 tx=0x7f28c0009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T11:50:38.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.478+0000 7f28caa00640 1 -- 172.21.15.47:0/1653193564 <== mon.1 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f28c404d070 con 0x7f28d41038e0 2024-03-21T11:50:38.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:38 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/2350104860' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T11:50:38.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:38 smithi102 ceph-mon[36748]: osdmap e98: 12 total, 12 up, 12 in 2024-03-21T11:50:38.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:38 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/81428410' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:38.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:38 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:38.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:38 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/2350104860' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T11:50:38.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:38 smithi047 ceph-mon[31351]: osdmap e98: 12 total, 12 up, 12 in 2024-03-21T11:50:38.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:38 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/81428410' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:38.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:38 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:38.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:38.734+0000 7f28d8c00640 1 -- 172.21.15.47:0/1653193564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f28d4115430 con 0x7f28d41038e0 2024-03-21T11:50:39.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:39.114+0000 7f28caa00640 1 -- 172.21.15.47:0/1653193564 <== mon.1 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v100) v1 ==== 178+0+0 (secure 0 0 0) 0x7f28c40be590 con 0x7f28d41038e0 2024-03-21T11:50:39.119 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data_ec' removed 2024-03-21T11:50:39.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:39.120+0000 7f28d8c00640 1 -- 172.21.15.47:0/1653193564 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28b80a3070 msgr2=0x7f28b80a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:39.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:39.120+0000 7f28d8c00640 1 --2- 172.21.15.47:0/1653193564 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28b80a3070 0x7f28b80a5530 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f28d4112090 tx=0x7f28c0009110 comp rx=0 tx=0).stop 2024-03-21T11:50:39.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:39.120+0000 7f28d8c00640 1 -- 172.21.15.47:0/1653193564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28d41038e0 msgr2=0x7f28d4110b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T11:50:39.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:39.120+0000 7f28d8c00640 1 --2- 172.21.15.47:0/1653193564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28d41038e0 0x7f28d4110b80 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f28c4000c00 tx=0x7f28c4005dd0 comp rx=0 tx=0).stop 2024-03-21T11:50:39.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:39.120+0000 7f28d8c00640 1 -- 172.21.15.47:0/1653193564 shutdown_connections 2024-03-21T11:50:39.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:39.120+0000 7f28d8c00640 1 --2- 172.21.15.47:0/1653193564 >> [v2:172.21.15.47:6800/2057671763,v1:172.21.15.47:6801/2057671763] conn(0x7f28b80a3070 0x7f28b80a5530 secure :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f28d4112090 tx=0x7f28c0009110 comp rx=0 tx=0).stop 2024-03-21T11:50:39.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:39.120+0000 7f28d8c00640 1 --2- 172.21.15.47:0/1653193564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d4105490 0x7f28d4114110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:39.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:39.120+0000 7f28d8c00640 1 --2- 172.21.15.47:0/1653193564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d4104ad0 0x7f28d41110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T11:50:39.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:39.120+0000 7f28d8c00640 1 --2- 172.21.15.47:0/1653193564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28d41038e0 0x7f28d4110b80 secure :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f28c4000c00 tx=0x7f28c4005dd0 comp rx=0 tx=0).stop 2024-03-21T11:50:39.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:39.120+0000 7f28d8c00640 1 -- 172.21.15.47:0/1653193564 >> 172.21.15.47:0/1653193564 conn(0x7f28d40fee50 msgr2=0x7f28d40ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T11:50:39.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:39.121+0000 7f28d8c00640 1 -- 172.21.15.47:0/1653193564 shutdown_connections 2024-03-21T11:50:39.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T11:50:39.121+0000 7f28d8c00640 1 -- 172.21.15.47:0/1653193564 wait complete. 2024-03-21T11:50:39.155 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T11:50:39.251 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T11:50:39.291 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T11:50:39.321 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-21T11:50:39.321 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T11:50:39.371 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T11:50:39.386 DEBUG:teuthology.orchestra.run.smithi102:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T11:50:39.392 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:39 smithi100 ceph-mon[36245]: pgmap v1222: 97 pgs: 97 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:50:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:39 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T11:50:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:39 smithi100 ceph-mon[36245]: osdmap e99: 12 total, 12 up, 12 in 2024-03-21T11:50:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:39 smithi100 ceph-mon[36245]: from='client.? 172.21.15.47:0/1653193564' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:39.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:39 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:39.402 INFO:tasks.cephadm:Stopping all daemons... 2024-03-21T11:50:39.403 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-21T11:50:39.403 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.a 2024-03-21T11:50:39.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:39 smithi102 ceph-mon[36748]: pgmap v1222: 97 pgs: 97 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:50:39.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:39 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T11:50:39.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:39 smithi102 ceph-mon[36748]: osdmap e99: 12 total, 12 up, 12 in 2024-03-21T11:50:39.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:39 smithi102 ceph-mon[36748]: from='client.? 172.21.15.47:0/1653193564' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:39.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:39 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:39.510 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:39 smithi047 ceph-mon[31351]: pgmap v1222: 97 pgs: 97 active+clean; 577 KiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T11:50:39.510 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:39 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T11:50:39.510 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:39 smithi047 ceph-mon[31351]: osdmap e99: 12 total, 12 up, 12 in 2024-03-21T11:50:39.510 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:39 smithi047 ceph-mon[31351]: from='client.? 172.21.15.47:0/1653193564' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:39.510 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:39 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T11:50:39.815 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:39 smithi047 systemd[1]: Stopping Ceph mon.a for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:50:40.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:40 smithi100 ceph-mon[36245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T11:50:40.393 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:40 smithi100 ceph-mon[36245]: osdmap e100: 12 total, 12 up, 12 in 2024-03-21T11:50:40.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:40 smithi102 ceph-mon[36748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T11:50:40.491 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:40 smithi102 ceph-mon[36748]: osdmap e100: 12 total, 12 up, 12 in 2024-03-21T11:50:40.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:40 smithi047 ceph-mon[31351]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T11:50:40.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:40 smithi047 ceph-mon[31351]: osdmap e100: 12 total, 12 up, 12 in 2024-03-21T11:50:40.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:40 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a[31347]: 2024-03-21T11:50:40.132+0000 7ffa27a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:50:40.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:40 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a[31347]: 2024-03-21T11:50:40.132+0000 7ffa27a00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-21T11:50:40.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:40 smithi047 podman[124970]: 2024-03-21 11:50:40.348868761 +0000 UTC m=+0.368772987 container died af33427a459adb8d1f6af38ac16b3b39ae04f6cb4b32fc99b556d7f1a6beb74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T11:50:41.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:40 smithi047 podman[124970]: 2024-03-21 11:50:40.824510426 +0000 UTC m=+0.844414612 container cleanup af33427a459adb8d1f6af38ac16b3b39ae04f6cb4b32fc99b556d7f1a6beb74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-21T11:50:41.065 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:40 smithi047 bash[124970]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a 2024-03-21T11:50:41.421 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 11:50:41 smithi047 podman[124980]: 2024-03-21 11:50:41.159453224 +0000 UTC m=+0.795024901 container remove af33427a459adb8d1f6af38ac16b3b39ae04f6cb4b32fc99b556d7f1a6beb74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5) 2024-03-21T11:50:41.438 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.a.service' 2024-03-21T11:50:41.590 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:50:41.590 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-21T11:50:41.590 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-21T11:50:41.590 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.b 2024-03-21T11:50:41.893 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:41 smithi100 systemd[1]: Stopping Ceph mon.b for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:50:42.367 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:42 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-b[36241]: 2024-03-21T11:50:42.031+0000 7f0468905700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:50:42.367 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:42 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-b[36241]: 2024-03-21T11:50:42.031+0000 7f0468905700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-21T11:50:42.367 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:42 smithi100 podman[77891]: 2024-03-21 11:50:42.062809272 +0000 UTC m=+0.166923042 container died 5ce4875e32d6a1e20d4151e204ee00a74b0bde9be888e0f1ca4183c8fc239ee9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-21T11:50:42.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:42 smithi100 podman[77891]: 2024-03-21 11:50:42.566838594 +0000 UTC m=+0.670952382 container cleanup 5ce4875e32d6a1e20d4151e204ee00a74b0bde9be888e0f1ca4183c8fc239ee9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-b, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T11:50:42.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 11:50:42 smithi100 bash[77891]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-b 2024-03-21T11:50:42.944 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.b.service' 2024-03-21T11:50:43.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:50:43.002 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-21T11:50:43.002 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-21T11:50:43.002 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.c 2024-03-21T11:50:43.386 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:43 smithi102 systemd[1]: Stopping Ceph mon.c for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:50:43.702 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:43 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-c[36722]: 2024-03-21T11:50:43.383+0000 7fc962374700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:50:43.702 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:43 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-c[36722]: 2024-03-21T11:50:43.383+0000 7fc962374700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-21T11:50:43.702 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:43 smithi102 podman[76160]: 2024-03-21 11:50:43.434878482 +0000 UTC m=+0.169538881 container died a078e02443605c8f261a67104620ab2ccf2359441cb0ec09f640ca58acbf694d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-21T11:50:43.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:43 smithi102 podman[76160]: 2024-03-21 11:50:43.809542973 +0000 UTC m=+0.544203408 container cleanup a078e02443605c8f261a67104620ab2ccf2359441cb0ec09f640ca58acbf694d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-21T11:50:43.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Mar 21 11:50:43 smithi102 bash[76160]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mon-c 2024-03-21T11:50:44.817 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mon.c.service' 2024-03-21T11:50:44.880 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:50:44.880 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-21T11:50:44.880 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-21T11:50:44.880 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mgr.x 2024-03-21T11:50:45.314 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:50:45 smithi047 systemd[1]: Stopping Ceph mgr.x for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:50:45.814 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:50:45 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:50:45.485+0000 7f77cb800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:50:45.815 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:50:45 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x[31581]: 2024-03-21T11:50:45.485+0000 7f77cb800700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-21T11:50:45.815 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:50:45 smithi047 podman[125088]: 2024-03-21 11:50:45.579231866 +0000 UTC m=+0.244414831 container died 34a3d6ac0ad59a1753ed4986446d79db805c3c956b4bb36893995c7a48b04f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:50:46.147 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:50:46 smithi047 podman[125088]: 2024-03-21 11:50:46.054702837 +0000 UTC m=+0.719885817 container cleanup 34a3d6ac0ad59a1753ed4986446d79db805c3c956b4bb36893995c7a48b04f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-03-21T11:50:46.147 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:50:46 smithi047 bash[125088]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x 2024-03-21T11:50:46.814 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 11:50:46 smithi047 podman[125102]: 2024-03-21 11:50:46.431702481 +0000 UTC m=+0.840896774 container remove 34a3d6ac0ad59a1753ed4986446d79db805c3c956b4bb36893995c7a48b04f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-x, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-21T11:50:46.985 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-21T11:50:47.113 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:50:47.113 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-21T11:50:47.113 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-21T11:50:47.113 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mgr.y 2024-03-21T11:50:47.393 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:50:47 smithi100 systemd[1]: Stopping Ceph mgr.y for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:50:47.739 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:50:47 smithi100 podman[78011]: 2024-03-21 11:50:47.450866424 +0000 UTC m=+0.139253144 container died bdc77f29aa77ba6a1d9099388492869f69fd9cf5be4a63f73f31f5344108fb2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-21T11:50:48.091 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:50:47 smithi100 podman[78011]: 2024-03-21 11:50:47.88093049 +0000 UTC m=+0.569317215 container cleanup bdc77f29aa77ba6a1d9099388492869f69fd9cf5be4a63f73f31f5344108fb2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-21T11:50:48.091 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 11:50:47 smithi100 bash[78011]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-y 2024-03-21T11:50:48.315 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-21T11:50:48.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:50:48.353 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-21T11:50:48.354 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-21T11:50:48.354 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mgr.z 2024-03-21T11:50:48.682 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:50:48 smithi102 systemd[1]: Stopping Ceph mgr.z for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:50:48.954 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:50:48 smithi102 podman[78358]: 2024-03-21 11:50:48.682049632 +0000 UTC m=+0.129018326 container died a64d8e76571a9b7bfbd59dedc2a666ee940a879965e921f6fb77dbb0a0fbbe30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:50:49.241 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:50:49 smithi102 podman[78358]: 2024-03-21 11:50:49.045476114 +0000 UTC m=+0.492444795 container cleanup a64d8e76571a9b7bfbd59dedc2a666ee940a879965e921f6fb77dbb0a0fbbe30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-21T11:50:49.241 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:50:49 smithi102 bash[78358]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z 2024-03-21T11:50:49.521 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:50:49 smithi102 podman[78372]: 2024-03-21 11:50:49.263905463 +0000 UTC m=+0.560287714 container remove a64d8e76571a9b7bfbd59dedc2a666ee940a879965e921f6fb77dbb0a0fbbe30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mgr-z, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-21T11:50:49.521 INFO:journalctl@ceph.mgr.z.smithi102.stdout:Mar 21 11:50:49 smithi102 systemd[1]: ceph-a6f35682-e773-11ee-95cd-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-21T11:50:49.525 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mgr.z.service' 2024-03-21T11:50:49.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:50:49.564 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-21T11:50:49.564 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-21T11:50:49.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.0 2024-03-21T11:50:50.064 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:50:49 smithi047 systemd[1]: Stopping Ceph osd.0 for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:50:50.565 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:50:50 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-0[43551]: 2024-03-21T11:50:50.233+0000 7f4a6c800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:50:50.565 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:50:50 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-0[43551]: 2024-03-21T11:50:50.234+0000 7f4a6c800700 -1 osd.0 100 *** Got signal Terminated *** 2024-03-21T11:50:50.565 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:50:50 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-0[43551]: 2024-03-21T11:50:50.234+0000 7f4a6c800700 -1 osd.0 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T11:50:56.383 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:50:56 smithi047 podman[125209]: 2024-03-21 11:50:56.084590371 +0000 UTC m=+6.014638977 container died 8c62707ad3ec2ec612fb2a8b50315f5e74d2d967671d3ce01a9789e2a138f5d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-21T11:50:56.796 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:50:56 smithi047 podman[125209]: 2024-03-21 11:50:56.552444443 +0000 UTC m=+6.482493070 container cleanup 8c62707ad3ec2ec612fb2a8b50315f5e74d2d967671d3ce01a9789e2a138f5d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-21T11:50:56.796 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:50:56 smithi047 bash[125209]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-0 2024-03-21T11:50:57.064 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:50:56 smithi047 podman[125230]: 2024-03-21 11:50:56.79533555 +0000 UTC m=+0.700198790 container remove 8c62707ad3ec2ec612fb2a8b50315f5e74d2d967671d3ce01a9789e2a138f5d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-21T11:50:58.065 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:50:57 smithi047 podman[125318]: 2024-03-21 11:50:57.573252558 +0000 UTC m=+0.035945871 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:50:58.065 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:50:57 smithi047 podman[125318]: 2024-03-21 11:50:57.721075978 +0000 UTC m=+0.183769278 container create 6e810202727a9c7d7c438696dfb947d5df21a15c17f69fadac360bb02426309a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-21T11:50:58.565 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:50:58 smithi047 podman[125318]: 2024-03-21 11:50:58.227734293 +0000 UTC m=+0.690427637 container init 6e810202727a9c7d7c438696dfb947d5df21a15c17f69fadac360bb02426309a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-21T11:50:58.565 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:50:58 smithi047 podman[125318]: 2024-03-21 11:50:58.239564721 +0000 UTC m=+0.702258065 container start 6e810202727a9c7d7c438696dfb947d5df21a15c17f69fadac360bb02426309a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-21T11:50:58.566 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:50:58 smithi047 podman[125318]: 2024-03-21 11:50:58.349145699 +0000 UTC m=+0.811839086 container attach 6e810202727a9c7d7c438696dfb947d5df21a15c17f69fadac360bb02426309a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T11:50:59.049 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 11:50:58 smithi047 podman[125318]: 2024-03-21 11:50:58.692995813 +0000 UTC m=+1.155689157 container died 6e810202727a9c7d7c438696dfb947d5df21a15c17f69fadac360bb02426309a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T11:50:59.462 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.0.service' 2024-03-21T11:50:59.598 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:50:59.599 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-21T11:50:59.599 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-21T11:50:59.599 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.1 2024-03-21T11:50:59.893 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:50:59 smithi100 systemd[1]: Stopping Ceph osd.1 for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:51:00.393 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:50:59 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-1[40856]: 2024-03-21T11:50:59.930+0000 7f1bbad0c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:51:00.393 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:50:59 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-1[40856]: 2024-03-21T11:50:59.930+0000 7f1bbad0c700 -1 osd.1 100 *** Got signal Terminated *** 2024-03-21T11:51:00.393 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:50:59 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-1[40856]: 2024-03-21T11:50:59.930+0000 7f1bbad0c700 -1 osd.1 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T11:51:05.620 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:51:05 smithi100 podman[78136]: 2024-03-21 11:51:05.335658276 +0000 UTC m=+5.545922141 container died ba975c2a84e77e925507acdb38c9ff80ed929c770b3bcf25969d468a602f53a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-21T11:51:05.893 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:51:05 smithi100 podman[78136]: 2024-03-21 11:51:05.710553838 +0000 UTC m=+5.920817684 container cleanup ba975c2a84e77e925507acdb38c9ff80ed929c770b3bcf25969d468a602f53a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:51:05.893 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:51:05 smithi100 bash[78136]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-1 2024-03-21T11:51:06.363 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:51:05 smithi100 podman[78164]: 2024-03-21 11:51:05.954114276 +0000 UTC m=+0.603167023 container remove ba975c2a84e77e925507acdb38c9ff80ed929c770b3bcf25969d468a602f53a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:51:06.643 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:51:06 smithi100 podman[78254]: 2024-03-21 11:51:06.266672128 +0000 UTC m=+0.017921031 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:51:06.643 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:51:06 smithi100 podman[78254]: 2024-03-21 11:51:06.371922096 +0000 UTC m=+0.123171048 container create 001d0bf07240a5bfcd04b3ef07b8547871944bec80299d690f4fbe10cbbe89c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-21T11:51:07.143 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:51:06 smithi100 podman[78254]: 2024-03-21 11:51:06.666989342 +0000 UTC m=+0.418238271 container init 001d0bf07240a5bfcd04b3ef07b8547871944bec80299d690f4fbe10cbbe89c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-21T11:51:07.143 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:51:06 smithi100 podman[78254]: 2024-03-21 11:51:06.670287631 +0000 UTC m=+0.421536541 container start 001d0bf07240a5bfcd04b3ef07b8547871944bec80299d690f4fbe10cbbe89c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:51:07.143 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:51:06 smithi100 podman[78254]: 2024-03-21 11:51:06.800038498 +0000 UTC m=+0.551287437 container attach 001d0bf07240a5bfcd04b3ef07b8547871944bec80299d690f4fbe10cbbe89c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-21T11:51:07.586 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 11:51:07 smithi100 podman[78254]: 2024-03-21 11:51:07.185742006 +0000 UTC m=+0.936990918 container died 001d0bf07240a5bfcd04b3ef07b8547871944bec80299d690f4fbe10cbbe89c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-21T11:51:07.910 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.1.service' 2024-03-21T11:51:07.953 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:51:07.954 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-21T11:51:07.954 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-21T11:51:07.954 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.2 2024-03-21T11:51:08.241 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:51:07 smithi102 systemd[1]: Stopping Ceph osd.2 for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:51:08.741 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:51:08 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2[41926]: 2024-03-21T11:51:08.257+0000 7f965d031700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:51:08.742 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:51:08 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2[41926]: 2024-03-21T11:51:08.257+0000 7f965d031700 -1 osd.2 100 *** Got signal Terminated *** 2024-03-21T11:51:08.742 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:51:08 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2[41926]: 2024-03-21T11:51:08.257+0000 7f965d031700 -1 osd.2 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T11:51:14.101 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:51:13 smithi102 podman[78481]: 2024-03-21 11:51:13.835256105 +0000 UTC m=+5.695393577 container died 62adc2cf2f9e806ef9c4e0eaa7ce73c46acfa71ff9138e738034e5be33f2aad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-21T11:51:14.477 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:51:14 smithi102 podman[78481]: 2024-03-21 11:51:14.19211248 +0000 UTC m=+6.052249943 container cleanup 62adc2cf2f9e806ef9c4e0eaa7ce73c46acfa71ff9138e738034e5be33f2aad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-21T11:51:14.477 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:51:14 smithi102 bash[78481]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2 2024-03-21T11:51:14.741 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:51:14 smithi102 podman[78500]: 2024-03-21 11:51:14.477029566 +0000 UTC m=+0.638473628 container remove 62adc2cf2f9e806ef9c4e0eaa7ce73c46acfa71ff9138e738034e5be33f2aad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-21T11:51:15.058 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:51:14 smithi102 podman[78592]: 2024-03-21 11:51:14.708540309 +0000 UTC m=+0.017540607 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:51:15.058 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:51:14 smithi102 podman[78592]: 2024-03-21 11:51:14.811353977 +0000 UTC m=+0.120354282 container create 51664b2a7d6f11af1460c143bdb8d4ff5ffe04ab2c3e9b971801c7a39ff68add (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-03-21T11:51:15.058 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:51:15 smithi102 podman[78592]: 2024-03-21 11:51:15.05534791 +0000 UTC m=+0.364348196 container init 51664b2a7d6f11af1460c143bdb8d4ff5ffe04ab2c3e9b971801c7a39ff68add (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-21T11:51:15.058 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:51:15 smithi102 podman[78592]: 2024-03-21 11:51:15.058661231 +0000 UTC m=+0.367661518 container start 51664b2a7d6f11af1460c143bdb8d4ff5ffe04ab2c3e9b971801c7a39ff68add (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2-deactivate, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-21T11:51:15.323 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:51:15 smithi102 podman[78592]: 2024-03-21 11:51:15.155948123 +0000 UTC m=+0.464948406 container attach 51664b2a7d6f11af1460c143bdb8d4ff5ffe04ab2c3e9b971801c7a39ff68add (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True) 2024-03-21T11:51:15.685 INFO:journalctl@ceph.osd.2.smithi102.stdout:Mar 21 11:51:15 smithi102 podman[78592]: 2024-03-21 11:51:15.386637807 +0000 UTC m=+0.695638094 container died 51664b2a7d6f11af1460c143bdb8d4ff5ffe04ab2c3e9b971801c7a39ff68add (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-21T11:51:15.999 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.2.service' 2024-03-21T11:51:16.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:51:16.040 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-21T11:51:16.040 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-21T11:51:16.040 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.3 2024-03-21T11:51:16.564 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:16 smithi047 systemd[1]: Stopping Ceph osd.3 for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:51:17.064 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:16 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3[50461]: 2024-03-21T11:51:16.662+0000 7f511e600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:51:17.065 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:16 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3[50461]: 2024-03-21T11:51:16.662+0000 7f511e600700 -1 osd.3 100 *** Got signal Terminated *** 2024-03-21T11:51:17.065 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:16 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3[50461]: 2024-03-21T11:51:16.662+0000 7f511e600700 -1 osd.3 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T11:51:17.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:16 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:16.962+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:17.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:17.097+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:18.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:17 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:17.984+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:18.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:18 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:18.142+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:19.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:18 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:18.963+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:19.314 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:19 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3[50461]: 2024-03-21T11:51:19.028+0000 7f5119e00700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:51.364140+0000 front 2024-03-21T11:50:51.364959+0000 (oldest deadline 2024-03-21T11:51:18.966363+0000) 2024-03-21T11:51:19.642 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:19 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:19.156+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:20.064 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:19 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:19.576+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:20.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:19 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:19.956+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:20.392 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:20 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:20.111+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:20.564 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3[50461]: 2024-03-21T11:51:20.072+0000 7f5119e00700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:51.364140+0000 front 2024-03-21T11:50:51.364959+0000 (oldest deadline 2024-03-21T11:51:18.966363+0000) 2024-03-21T11:51:21.064 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:20 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:20.610+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:21.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:20 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:20.980+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:21.392 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:21 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:21.131+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:21.564 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:21 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3[50461]: 2024-03-21T11:51:21.072+0000 7f5119e00700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:51.364140+0000 front 2024-03-21T11:50:51.364959+0000 (oldest deadline 2024-03-21T11:51:18.966363+0000) 2024-03-21T11:51:22.064 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:21 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:21.638+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:22.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:21 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:21.988+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:22.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:22 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:22.085+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:22.874 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:22 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:22.683+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:22.875 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:22 smithi047 podman[125422]: 2024-03-21 11:51:22.597076769 +0000 UTC m=+6.100399086 container died 8418de722bbbeb42990869c53cc079e0cc4e438e2e3511fdc9cf5035937efa0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-21T11:51:23.241 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:23 smithi047 podman[125422]: 2024-03-21 11:51:23.01426582 +0000 UTC m=+6.517588137 container cleanup 8418de722bbbeb42990869c53cc079e0cc4e438e2e3511fdc9cf5035937efa0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-21T11:51:23.241 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:23 smithi047 bash[125422]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3 2024-03-21T11:51:23.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:23 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:23.121+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:23.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:23 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:23.004+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:23.564 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:23 smithi047 podman[125454]: 2024-03-21 11:51:23.240724946 +0000 UTC m=+0.630481367 container remove 8418de722bbbeb42990869c53cc079e0cc4e438e2e3511fdc9cf5035937efa0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:51:24.064 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:23 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:23.658+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:24.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:24 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:24.115+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:24.415 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:24 smithi047 podman[125539]: 2024-03-21 11:51:23.988500966 +0000 UTC m=+0.033549818 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:51:24.416 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:24 smithi047 podman[125539]: 2024-03-21 11:51:24.130489307 +0000 UTC m=+0.175538196 container create 85f98cf85489c587574a85c17e5621a4c3d16c611659d4042ab5aa985fe3924b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:51:24.416 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:24 smithi047 podman[125539]: 2024-03-21 11:51:24.414574475 +0000 UTC m=+0.459623371 container init 85f98cf85489c587574a85c17e5621a4c3d16c611659d4042ab5aa985fe3924b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-21T11:51:24.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:24 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:24.048+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:24.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:24 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:24.636+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:24.815 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:24 smithi047 podman[125539]: 2024-03-21 11:51:24.42662873 +0000 UTC m=+0.471677604 container start 85f98cf85489c587574a85c17e5621a4c3d16c611659d4042ab5aa985fe3924b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:51:24.815 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:24 smithi047 podman[125539]: 2024-03-21 11:51:24.527848644 +0000 UTC m=+0.572897567 container attach 85f98cf85489c587574a85c17e5621a4c3d16c611659d4042ab5aa985fe3924b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T11:51:25.221 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 11:51:24 smithi047 podman[125539]: 2024-03-21 11:51:24.882285263 +0000 UTC m=+0.927334145 container died 85f98cf85489c587574a85c17e5621a4c3d16c611659d4042ab5aa985fe3924b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0) 2024-03-21T11:51:25.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:25 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:25.009+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:25.631 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.3.service' 2024-03-21T11:51:25.642 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:25 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:25.166+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:25.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:51:25.763 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-21T11:51:25.763 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-21T11:51:25.763 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.4 2024-03-21T11:51:26.064 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:25 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:25.610+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:26.090 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:25 smithi100 systemd[1]: Stopping Ceph osd.4 for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:51:26.357 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:26 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:26.054+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:26.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:26 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:26.169+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:26.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:26 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:26.169+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:26.393 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:26 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4[47405]: 2024-03-21T11:51:26.089+0000 7f9c58cce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:51:26.393 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:26 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4[47405]: 2024-03-21T11:51:26.089+0000 7f9c58cce700 -1 osd.4 100 *** Got signal Terminated *** 2024-03-21T11:51:26.393 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:26 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4[47405]: 2024-03-21T11:51:26.089+0000 7f9c58cce700 -1 osd.4 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T11:51:26.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:26 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:26.408+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:27.064 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:26 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:26.659+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:27.064 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:26 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:26.659+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:27.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:27 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:27.137+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:27.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:27 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:27.137+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:27.397 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:27 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:27.072+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:27.741 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:27 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:27.397+0000 7f239e440700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:01.159364+0000 front 2024-03-21T11:51:01.159377+0000 (oldest deadline 2024-03-21T11:51:26.459371+0000) 2024-03-21T11:51:27.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:27 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:27.435+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:28.064 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:27 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:27.629+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:28.064 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:27 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:27.629+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:28.378 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:28 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:28.092+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:28.470 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:28 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:28.147+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:28.470 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:28 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:28.147+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:28.653 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:28 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:28.385+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:28.741 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:28 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:28.378+0000 7f239e440700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:01.159364+0000 front 2024-03-21T11:51:01.159377+0000 (oldest deadline 2024-03-21T11:51:26.459371+0000) 2024-03-21T11:51:28.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:28 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:28.469+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:29.064 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:28 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:28.651+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:29.064 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:28 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:28.651+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:29.111 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:28 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:28.821+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:29.417 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:29 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:29.111+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:29.439 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:29 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:29.186+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:29.439 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:29 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:29.186+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:29.741 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:29 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:29.417+0000 7f239e440700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:01.159364+0000 front 2024-03-21T11:51:01.159377+0000 (oldest deadline 2024-03-21T11:51:26.459371+0000) 2024-03-21T11:51:29.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:29 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:29.657+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:29.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:29 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:29.657+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:29.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:29 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:29.419+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:29.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:29 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:29.438+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:30.117 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:29 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:29.832+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:30.378 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:30 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:30.117+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:30.457 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:30 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4[47405]: 2024-03-21T11:51:30.424+0000 7f9c552c2700 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:04.277687+0000 front 2024-03-21T11:51:04.277706+0000 (oldest deadline 2024-03-21T11:51:29.577882+0000) 2024-03-21T11:51:30.457 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:30 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:30.405+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:30.457 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:30 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:30.195+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:30.458 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:30 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:30.195+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:30.741 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:30 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:30.378+0000 7f239e440700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:01.159364+0000 front 2024-03-21T11:51:01.159377+0000 (oldest deadline 2024-03-21T11:51:26.459371+0000) 2024-03-21T11:51:30.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:30 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:30.431+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:30.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:30 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:30.664+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:30.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:30 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:30.664+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:31.114 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:30 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:30.789+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:31.491 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:31 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:31.344+0000 7f239e440700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:01.159364+0000 front 2024-03-21T11:51:01.159377+0000 (oldest deadline 2024-03-21T11:51:26.459371+0000) 2024-03-21T11:51:31.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:31 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:31.114+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:31.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:31 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:31.367+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:31.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:31 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:31.191+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:31.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:31 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:31.191+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:31.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:31 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:31.478+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:31.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:31 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:31.700+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:31.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:31 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:31.700+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:32.024 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:31 smithi100 podman[78360]: 2024-03-21 11:51:31.724983721 +0000 UTC m=+5.765407907 container died 1fe6513c03be3a434b641b82fe0342e275a9d00ffd5ad121c0d07cadc0c90d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-21T11:51:32.101 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:31 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:31.815+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:32.364 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:32 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:32.101+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:32.383 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:32 smithi100 podman[78360]: 2024-03-21 11:51:32.123489926 +0000 UTC m=+6.163914104 container cleanup 1fe6513c03be3a434b641b82fe0342e275a9d00ffd5ad121c0d07cadc0c90d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T11:51:32.383 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:32 smithi100 bash[78360]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4 2024-03-21T11:51:32.383 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:32 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:32.150+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:32.383 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:32 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:32.150+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:32.643 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:32 smithi100 podman[78389]: 2024-03-21 11:51:32.383450457 +0000 UTC m=+0.660182335 container remove 1fe6513c03be3a434b641b82fe0342e275a9d00ffd5ad121c0d07cadc0c90d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0) 2024-03-21T11:51:32.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:32 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:32.397+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:32.741 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:32 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:32.364+0000 7f239e440700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:01.159364+0000 front 2024-03-21T11:51:01.159377+0000 (oldest deadline 2024-03-21T11:51:26.459371+0000) 2024-03-21T11:51:32.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:32 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:32.513+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:32.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:32 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:32.653+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:32.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:32 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:32.653+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:32.970 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:32 smithi100 podman[78475]: 2024-03-21 11:51:32.589419674 +0000 UTC m=+0.017873316 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:51:32.970 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:32 smithi100 podman[78475]: 2024-03-21 11:51:32.709775057 +0000 UTC m=+0.138228682 container create 5f479b78b153cc1d82f74ac40a84903274be05c312a27dfb98d7cca43bdaf4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-03-21T11:51:32.971 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:32 smithi100 podman[78475]: 2024-03-21 11:51:32.970752624 +0000 UTC m=+0.399206261 container init 5f479b78b153cc1d82f74ac40a84903274be05c312a27dfb98d7cca43bdaf4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-21T11:51:33.108 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:32 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:32.792+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:33.264 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:32 smithi100 podman[78475]: 2024-03-21 11:51:32.974012604 +0000 UTC m=+0.402466223 container start 5f479b78b153cc1d82f74ac40a84903274be05c312a27dfb98d7cca43bdaf4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:51:33.264 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:33 smithi100 podman[78475]: 2024-03-21 11:51:33.070840192 +0000 UTC m=+0.499293837 container attach 5f479b78b153cc1d82f74ac40a84903274be05c312a27dfb98d7cca43bdaf4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-21T11:51:33.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:33 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:33.146+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:33.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:33 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:33.146+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:33.376 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:33 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:33.108+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:33.575 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 11:51:33 smithi100 podman[78475]: 2024-03-21 11:51:33.264349034 +0000 UTC m=+0.692802664 container died 5f479b78b153cc1d82f74ac40a84903274be05c312a27dfb98d7cca43bdaf4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-4-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:51:33.575 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:33 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:33.421+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:33.741 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:33 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:33.376+0000 7f239e440700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:01.159364+0000 front 2024-03-21T11:51:01.159377+0000 (oldest deadline 2024-03-21T11:51:26.459371+0000) 2024-03-21T11:51:33.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:33 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:33.502+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:33.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:33 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:33.661+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:33.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:33 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:33.661+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:33.887 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.4.service' 2024-03-21T11:51:33.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:51:33.930 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-21T11:51:33.930 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-21T11:51:33.930 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.5 2024-03-21T11:51:34.090 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:33 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:33.784+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:34.090 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:33 smithi102 systemd[1]: Stopping Ceph osd.5 for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:51:34.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:34 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:34.126+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:34.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:34 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:34.126+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:34.425 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:34 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:34.089+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:34.425 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:34 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:34.252+0000 7f23a1e4c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:51:34.425 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:34 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:34.252+0000 7f23a1e4c700 -1 osd.5 100 *** Got signal Terminated *** 2024-03-21T11:51:34.425 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:34 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:34.252+0000 7f23a1e4c700 -1 osd.5 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T11:51:34.670 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:34 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:34.514+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:34.741 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:34 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:34.425+0000 7f239e440700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:01.159364+0000 front 2024-03-21T11:51:01.159377+0000 (oldest deadline 2024-03-21T11:51:26.459371+0000) 2024-03-21T11:51:34.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:34 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:34.413+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:34.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:34 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:34.413+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:35.064 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:34 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:34.669+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:35.064 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:34 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:34.669+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:35.085 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:34 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:34.775+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:35.085 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:34 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:34.775+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:35.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:35.121+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:35.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:35.121+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:35.460 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:35.085+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:35.461 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:35.085+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:35.737 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:35.460+0000 7f239e440700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:01.159364+0000 front 2024-03-21T11:51:01.159377+0000 (oldest deadline 2024-03-21T11:51:26.459371+0000) 2024-03-21T11:51:35.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:35 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:35.545+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:35.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:35 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:35.639+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:35.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:35 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:35.639+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:35.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:35.456+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:35.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:35 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:35.456+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:35.991 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:35.737+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:35.991 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:35 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:35.737+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:36.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:36 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:36.105+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:36.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:36 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:36.105+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:36.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:36 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:36.105+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:36.451 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:36 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:36.061+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:36.451 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:36 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:36.061+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:36.725 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:36 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:36.450+0000 7f239e440700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:01.159364+0000 front 2024-03-21T11:51:01.159377+0000 (oldest deadline 2024-03-21T11:51:26.459371+0000) 2024-03-21T11:51:36.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:36 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:36.496+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:36.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:36 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:36.617+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:36.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:36 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:36.617+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:36.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:36 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:36.470+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:36.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:36 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:36.470+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:36.991 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:36 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:36.725+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:36.991 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:36 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:36.725+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:37.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:37 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:37.060+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:37.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:37 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:37.060+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:37.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:37 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:37.060+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:37.472 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:37 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:37.064+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:37.473 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:37 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:37.064+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:37.741 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:37 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:37.472+0000 7f239e440700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:01.159364+0000 front 2024-03-21T11:51:01.159377+0000 (oldest deadline 2024-03-21T11:51:26.459371+0000) 2024-03-21T11:51:37.741 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:37 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:37.472+0000 7f239e440700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.261249+0000 front 2024-03-21T11:51:09.261270+0000 (oldest deadline 2024-03-21T11:51:36.862056+0000) 2024-03-21T11:51:37.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:37 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:37.509+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:37.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:37 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:37.606+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:37.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:37 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:37.606+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:37.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:37 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:37.459+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:37.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:37 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:37.459+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:38.068 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:37 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:37.767+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:38.068 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:37 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:37.767+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:38.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:38 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:38.014+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:38.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:38 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:38.014+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:38.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:38 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:38.014+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:38.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:38 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:38.068+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:38.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:38 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:38.068+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:38.774 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:38 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:38.516+0000 7f239e440700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:01.159364+0000 front 2024-03-21T11:51:01.159377+0000 (oldest deadline 2024-03-21T11:51:26.459371+0000) 2024-03-21T11:51:38.774 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:38 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5[48459]: 2024-03-21T11:51:38.516+0000 7f239e440700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.261249+0000 front 2024-03-21T11:51:09.261270+0000 (oldest deadline 2024-03-21T11:51:36.862056+0000) 2024-03-21T11:51:38.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:38 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:38.530+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:38.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:38 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:38.530+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.491670+0000 front 2024-03-21T11:51:12.491773+0000 (oldest deadline 2024-03-21T11:51:38.391882+0000) 2024-03-21T11:51:38.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:38 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:38.628+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:38.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:38 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:38.628+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:38.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:38 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:38.628+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:38.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:38 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:38.418+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:38.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:38 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:38.418+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:39.027 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:38 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:38.773+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:39.027 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:38 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:38.773+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:39.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:39 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:39.020+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:39.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:39 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:39.020+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:39.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:39 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:39.020+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:39.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:39 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:39.027+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:39.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:39 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:39.027+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:39.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:39 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:39.511+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:39.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:39 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:39.511+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.491670+0000 front 2024-03-21T11:51:12.491773+0000 (oldest deadline 2024-03-21T11:51:38.391882+0000) 2024-03-21T11:51:39.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:39 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:39.647+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:39.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:39 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:39.647+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:39.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:39 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:39.648+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:39.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:39 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:39.459+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:39.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:39 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:39.459+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:39.982 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:39 smithi102 podman[78710]: 2024-03-21 11:51:39.713764557 +0000 UTC m=+5.590855336 container died 27fc6e14ed65a8fcfa516ab2c35d859f2c3fc909402800ecca02b3d80e29fc32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-21T11:51:39.982 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:39 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:39.765+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:39.983 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:39 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:39.765+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:40.241 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:40 smithi102 podman[78710]: 2024-03-21 11:51:40.038306238 +0000 UTC m=+5.915397033 container cleanup 27fc6e14ed65a8fcfa516ab2c35d859f2c3fc909402800ecca02b3d80e29fc32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-21T11:51:40.241 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:40 smithi102 bash[78710]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5 2024-03-21T11:51:40.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:39 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:39.981+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:40.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:39 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:39.981+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:40.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:40 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:40.061+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:40.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:40 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:40.061+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:40.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:40 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:40.061+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:40.688 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:40 smithi102 podman[78727]: 2024-03-21 11:51:40.274312367 +0000 UTC m=+0.561169787 container remove 27fc6e14ed65a8fcfa516ab2c35d859f2c3fc909402800ecca02b3d80e29fc32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T11:51:40.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:40 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:40.505+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:40.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:40 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:40.505+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.491670+0000 front 2024-03-21T11:51:12.491773+0000 (oldest deadline 2024-03-21T11:51:38.391882+0000) 2024-03-21T11:51:40.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:40 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:40.606+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:40.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:40 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:40.606+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:40.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:40 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:40.606+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:40.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:40 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:40.460+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:40.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:40 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:40.460+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:40.976 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:40 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:40.791+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:40.976 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:40 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:40.791+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:40.976 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:40 smithi102 podman[78814]: 2024-03-21 11:51:40.591425581 +0000 UTC m=+0.017303896 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:51:40.976 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:40 smithi102 podman[78814]: 2024-03-21 11:51:40.71564725 +0000 UTC m=+0.141525546 container create 4673d7129866317f3fcbd16249f83c88e7d15d8961ce4e7e88a1fbe2914287b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-21T11:51:40.976 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:40 smithi102 podman[78814]: 2024-03-21 11:51:40.976574152 +0000 UTC m=+0.402452446 container init 4673d7129866317f3fcbd16249f83c88e7d15d8961ce4e7e88a1fbe2914287b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-03-21T11:51:41.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:41 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:41.027+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:41.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:41 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:41.027+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:41.242 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:40 smithi102 podman[78814]: 2024-03-21 11:51:40.979948452 +0000 UTC m=+0.405826746 container start 4673d7129866317f3fcbd16249f83c88e7d15d8961ce4e7e88a1fbe2914287b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-21T11:51:41.242 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:41 smithi102 podman[78814]: 2024-03-21 11:51:41.060006637 +0000 UTC m=+0.485884947 container attach 4673d7129866317f3fcbd16249f83c88e7d15d8961ce4e7e88a1fbe2914287b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux ) 2024-03-21T11:51:41.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:41 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:41.104+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:41.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:41 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:41.104+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:41.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:41 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:41.104+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:41.512 INFO:journalctl@ceph.osd.5.smithi102.stdout:Mar 21 11:51:41 smithi102 podman[78814]: 2024-03-21 11:51:41.262772094 +0000 UTC m=+0.688650399 container died 4673d7129866317f3fcbd16249f83c88e7d15d8961ce4e7e88a1fbe2914287b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-21T11:51:41.737 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:41 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:41.485+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:41.737 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:41 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:41.485+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:41.780 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:41 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:41.748+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:41.780 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:41 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:41.748+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:41.799 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.5.service' 2024-03-21T11:51:41.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:41 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:41.496+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:41.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:41 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:41.496+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.491670+0000 front 2024-03-21T11:51:12.491773+0000 (oldest deadline 2024-03-21T11:51:38.391882+0000) 2024-03-21T11:51:41.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:41 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:41.568+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:41.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:41 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:41.568+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:41.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:41 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:41.568+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:41.840 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:51:41.840 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-21T11:51:41.840 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-21T11:51:41.840 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.6 2024-03-21T11:51:42.314 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:41 smithi047 systemd[1]: Stopping Ceph osd.6 for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:51:42.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:42 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:42.105+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:42.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:42 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:42.105+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:42.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:42 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:42.105+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:42.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:42 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:42.059+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:42.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:42 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:42.059+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:42.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:42 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:42.614+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:42.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:42 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:42.614+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:42.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:42 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:42.614+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:42.815 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:42 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:42.474+0000 7ff863000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:51:42.815 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:42 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:42.474+0000 7ff863000700 -1 osd.6 100 *** Got signal Terminated *** 2024-03-21T11:51:42.815 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:42 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:42.474+0000 7ff863000700 -1 osd.6 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T11:51:42.815 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:42 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:42.504+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:42.815 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:42 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:42.504+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.491670+0000 front 2024-03-21T11:51:12.491773+0000 (oldest deadline 2024-03-21T11:51:38.391882+0000) 2024-03-21T11:51:42.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:42 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:42.501+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:42.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:42 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:42.501+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:43.078 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:42 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:42.786+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:43.078 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:42 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:42.786+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:43.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:43 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:43.057+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:43.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:43 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:43.057+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:43.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:43 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:43.057+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:43.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:43 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:43.057+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:43.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:43 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:43.077+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:43.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:43 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:43.077+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:43.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:43 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:43.547+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:43.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:43 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:43.547+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.491670+0000 front 2024-03-21T11:51:12.491773+0000 (oldest deadline 2024-03-21T11:51:38.391882+0000) 2024-03-21T11:51:43.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:43 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:43.595+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:43.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:43 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:43.595+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:43.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:43 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:43.595+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:43.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:43 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:43.480+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:43.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:43 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:43.480+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:44.122 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:43 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:43.815+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:44.122 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:43 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:43.815+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:44.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:44 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:44.098+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:44.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:44 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:44.098+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:44.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:44 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:44.098+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:44.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:44 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:44.098+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:44.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:44 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:44.121+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:44.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:44 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:44.121+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:44.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:44 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:44.472+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:44.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:44 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:44.472+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:44.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:44 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:44.472+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.165699+0000 front 2024-03-21T11:51:18.166057+0000 (oldest deadline 2024-03-21T11:51:44.065407+0000) 2024-03-21T11:51:45.064 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:44 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:44.564+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:45.064 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:44 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:44.564+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.491670+0000 front 2024-03-21T11:51:12.491773+0000 (oldest deadline 2024-03-21T11:51:38.391882+0000) 2024-03-21T11:51:45.065 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:44 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:44.581+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:45.065 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:44 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:44.581+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:45.065 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:44 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:44.581+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:45.109 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:44 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:44.840+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:45.109 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:44 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:44.840+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:45.377 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:45 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:45.108+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:45.377 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:45 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:45.108+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:45.377 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:45 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:45.108+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:45.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:45 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:45.099+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:45.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:45 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:45.099+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:45.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:45 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:45.099+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:45.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:45 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:45.099+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:45.814 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:45 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:45.581+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:45.815 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:45 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:45.581+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.491670+0000 front 2024-03-21T11:51:12.491773+0000 (oldest deadline 2024-03-21T11:51:38.391882+0000) 2024-03-21T11:51:45.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:45 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:45.558+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:45.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:45 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:45.558+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:45.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:45 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:45.558+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:45.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:45 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:45.558+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:45.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:45 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:45.465+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:45.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:45 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:45.465+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:45.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:45 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:45.465+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.165699+0000 front 2024-03-21T11:51:18.166057+0000 (oldest deadline 2024-03-21T11:51:44.065407+0000) 2024-03-21T11:51:46.119 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:45 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:45.810+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:46.119 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:45 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:45.810+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:46.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:46 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:46.081+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:46.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:46 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:46.081+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:46.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:46 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:46.081+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:46.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:46 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:46.081+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:46.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:46 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:46.117+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:46.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:46 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:46.117+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:46.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:46 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:46.117+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:46.815 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:46 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:46.562+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.085254+0000 front 2024-03-21T11:51:02.085356+0000 (oldest deadline 2024-03-21T11:51:27.985491+0000) 2024-03-21T11:51:46.815 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:46 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6[57031]: 2024-03-21T11:51:46.562+0000 7ff859800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.491670+0000 front 2024-03-21T11:51:12.491773+0000 (oldest deadline 2024-03-21T11:51:38.391882+0000) 2024-03-21T11:51:46.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:46 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:46.537+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:46.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:46 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:46.537+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:46.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:46 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:46.537+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:46.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:46 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:46.537+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:46.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:46 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:46.472+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:46.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:46 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:46.472+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:46.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:46 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:46.472+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.165699+0000 front 2024-03-21T11:51:18.166057+0000 (oldest deadline 2024-03-21T11:51:44.065407+0000) 2024-03-21T11:51:47.152 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:46 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:46.841+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:47.153 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:46 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:46.841+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:47.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:47 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:47.069+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:47.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:47 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:47.069+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:47.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:47 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:47.069+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:47.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:47 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:47.069+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:47.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:47 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:47.152+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:47.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:47 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:47.152+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:47.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:47 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:47.152+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:47.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:47 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:47.542+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:47.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:47 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:47.542+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:47.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:47 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:47.542+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:47.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:47 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:47.542+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:47.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:47 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:47.469+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:47.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:47 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:47.469+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:47.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:47 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:47.469+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.165699+0000 front 2024-03-21T11:51:18.166057+0000 (oldest deadline 2024-03-21T11:51:44.065407+0000) 2024-03-21T11:51:48.120 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:47 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:47.859+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:48.120 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:47 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:47.859+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:48.120 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:47 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:47.859+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:51:48.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:48 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:48.050+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:48.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:48 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:48.050+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:48.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:48 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:48.050+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:48.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:48 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:48.050+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:48.405 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:48 smithi047 podman[125643]: 2024-03-21 11:51:48.1048536 +0000 UTC m=+5.778919782 container died 038d97750d0ef1839fe58ea5ec471c9aeadd293328eb7dbeb5c42469a59edf6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:51:48.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:48 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:48.118+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:48.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:48 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:48.118+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:48.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:48 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:48.118+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:48.812 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:48 smithi047 podman[125643]: 2024-03-21 11:51:48.59268488 +0000 UTC m=+6.266751099 container cleanup 038d97750d0ef1839fe58ea5ec471c9aeadd293328eb7dbeb5c42469a59edf6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-21T11:51:48.812 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:48 smithi047 bash[125643]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6 2024-03-21T11:51:48.813 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:48 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:48.533+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:48.813 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:48 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:48.533+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:48.813 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:48 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:48.533+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:48.813 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:48 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:48.533+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:48.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:48 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:48.437+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:48.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:48 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:48.437+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:48.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:48 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:48.437+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.165699+0000 front 2024-03-21T11:51:18.166057+0000 (oldest deadline 2024-03-21T11:51:44.065407+0000) 2024-03-21T11:51:49.064 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:48 smithi047 podman[125672]: 2024-03-21 11:51:48.810319275 +0000 UTC m=+0.687314536 container remove 038d97750d0ef1839fe58ea5ec471c9aeadd293328eb7dbeb5c42469a59edf6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:51:49.152 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:48 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:48.851+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:49.153 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:48 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:48.851+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:49.153 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:48 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:48.851+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:51:49.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:49 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:49.053+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:49.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:49 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:49.053+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:49.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:49 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:49.053+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:49.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:49 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:49.053+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:49.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:49 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:49.151+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:49.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:49 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:49.151+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:49.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:49 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:49.151+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:49.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:49 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:49.546+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:49.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:49 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:49.546+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:49.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:49 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:49.546+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:49.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:49 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:49.546+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:49.815 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:49 smithi047 podman[125764]: 2024-03-21 11:51:49.522483329 +0000 UTC m=+0.039600727 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:51:49.815 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:49 smithi047 podman[125764]: 2024-03-21 11:51:49.644475288 +0000 UTC m=+0.161592719 container create a00b66603b940810d08c8622ceeea93fb30333969d2f6261796a97e544b0df21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-21T11:51:49.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:49 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:49.402+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:49.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:49 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:49.402+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:49.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:49 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:49.402+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.165699+0000 front 2024-03-21T11:51:18.166057+0000 (oldest deadline 2024-03-21T11:51:44.065407+0000) 2024-03-21T11:51:50.152 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:49 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:49.871+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:50.153 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:49 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:49.871+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:50.153 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:49 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:49.871+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:51:50.315 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:49 smithi047 podman[125764]: 2024-03-21 11:51:49.881546566 +0000 UTC m=+0.398663984 container init a00b66603b940810d08c8622ceeea93fb30333969d2f6261796a97e544b0df21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-21T11:51:50.315 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:49 smithi047 podman[125764]: 2024-03-21 11:51:49.893994067 +0000 UTC m=+0.411111452 container start a00b66603b940810d08c8622ceeea93fb30333969d2f6261796a97e544b0df21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-21T11:51:50.315 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:49 smithi047 podman[125764]: 2024-03-21 11:51:49.997125306 +0000 UTC m=+0.514242691 container attach a00b66603b940810d08c8622ceeea93fb30333969d2f6261796a97e544b0df21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True) 2024-03-21T11:51:50.373 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:50 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:50.052+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:50.373 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:50 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:50.052+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:50.373 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:50 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:50.052+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:50.373 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:50 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:50.052+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:50.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:50 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:50.151+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:50.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:50 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:50.151+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:50.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:50 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:50.151+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:50.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:50 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:50.372+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:50.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:50 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:50.372+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:50.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:50 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:50.372+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.165699+0000 front 2024-03-21T11:51:18.166057+0000 (oldest deadline 2024-03-21T11:51:44.065407+0000) 2024-03-21T11:51:50.732 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 11:51:50 smithi047 podman[125764]: 2024-03-21 11:51:50.342098907 +0000 UTC m=+0.859216324 container died a00b66603b940810d08c8622ceeea93fb30333969d2f6261796a97e544b0df21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-21T11:51:50.733 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:50 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:50.540+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:50.733 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:50 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:50.540+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:50.733 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:50 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:50.540+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:50.733 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:50 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:50.540+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:51.151 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.6.service' 2024-03-21T11:51:51.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:50 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:50.868+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:51.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:50 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:50.868+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:51.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:50 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:50.868+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:51:51.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:51 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:51.105+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:51.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:51 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:51.105+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:51.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:51 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:51.105+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:51.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:51:51.288 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-21T11:51:51.288 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-21T11:51:51.288 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.7 2024-03-21T11:51:51.334 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:51 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:51.074+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:51.334 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:51 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:51.074+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:51.334 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:51 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:51.075+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:51.335 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:51 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:51.075+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:51.602 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:51 smithi100 systemd[1]: Stopping Ceph osd.7 for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:51:51.603 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:51 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:51.365+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:51.603 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:51 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:51.365+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:51.603 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:51 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:51.365+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.165699+0000 front 2024-03-21T11:51:18.166057+0000 (oldest deadline 2024-03-21T11:51:44.065407+0000) 2024-03-21T11:51:51.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:51 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:51.502+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:51.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:51 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:51.502+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:51.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:51 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:51.502+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:51.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:51 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:51.502+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:51.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:51 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:51.602+0000 7f8fb125b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:51:51.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:51 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:51.602+0000 7f8fb125b700 -1 osd.7 100 *** Got signal Terminated *** 2024-03-21T11:51:51.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:51 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:51.602+0000 7f8fb125b700 -1 osd.7 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T11:51:52.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:51 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:51.883+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:52.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:51 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:51.883+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:52.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:51 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:51.883+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:51:52.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:52 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:52.095+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:52.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:52 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:52.095+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:52.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:52 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:52.095+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:52.377 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:52 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:52.079+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:52.377 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:52 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:52.079+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:52.377 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:52 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:52.079+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:52.377 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:52 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:52.079+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:52.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:52 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:52.377+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:52.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:52 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:52.377+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:52.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:52 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:52.377+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.165699+0000 front 2024-03-21T11:51:18.166057+0000 (oldest deadline 2024-03-21T11:51:44.065407+0000) 2024-03-21T11:51:52.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:52 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:52.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:52.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:52 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:52.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:52.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:52 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:52.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:52.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:52 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:52.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:53.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:52 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:52.882+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:53.242 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:52 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:52.882+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:53.242 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:52 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:52.882+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:51:53.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:53 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:53.049+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:53.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:53 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:53.049+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:53.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:53 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:53.049+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:53.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:53 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:53.049+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:51:53.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:53 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:53.072+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:53.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:53 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:53.072+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:53.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:53 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:53.072+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:53.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:53 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:53.072+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:53.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:53 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:53.484+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:53.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:53 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:53.484+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:53.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:53 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:53.484+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:53.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:53 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:53.484+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:53.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:53 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:53.424+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:53.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:53 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:53.424+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:53.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:53 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:53.424+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.165699+0000 front 2024-03-21T11:51:18.166057+0000 (oldest deadline 2024-03-21T11:51:44.065407+0000) 2024-03-21T11:51:53.893 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:53 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:53.424+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:26.966331+0000 front 2024-03-21T11:51:26.966298+0000 (oldest deadline 2024-03-21T11:51:52.879778+0000) 2024-03-21T11:51:54.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:53 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:53.860+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:54.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:53 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:53.860+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:54.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:53 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:53.860+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:51:54.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:54 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:54.028+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:54.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:54 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:54.028+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:54.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:54 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:54.028+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:54.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:54 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:54.028+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:51:54.378 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:54 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:54.033+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:54.378 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:54 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:54.033+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:54.378 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:54 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:54.033+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:54.378 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:54 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:54.033+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:54.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:54 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:54.378+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:54.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:54 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:54.378+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:54.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:54 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:54.378+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.165699+0000 front 2024-03-21T11:51:18.166057+0000 (oldest deadline 2024-03-21T11:51:44.065407+0000) 2024-03-21T11:51:54.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:54 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:54.378+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:26.966331+0000 front 2024-03-21T11:51:26.966298+0000 (oldest deadline 2024-03-21T11:51:52.879778+0000) 2024-03-21T11:51:54.817 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:54 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:54.486+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:54.817 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:54 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:54.486+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:54.817 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:54 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:54.486+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:54.817 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:54 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:54.486+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:54.817 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:54 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:54.486+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:51:55.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:54 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:54.892+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:55.242 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:54 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:54.892+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:55.242 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:54 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:54.892+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:51:55.242 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:54 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:54.892+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:27.476750+0000 front 2024-03-21T11:51:27.476714+0000 (oldest deadline 2024-03-21T11:51:53.877340+0000) 2024-03-21T11:51:55.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:54 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:54.991+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:55.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:54 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:54.991+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:55.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:54 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:54.991+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:55.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:54 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:54.991+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:51:55.374 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:55 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:55.071+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:55.374 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:55 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:55.071+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:55.374 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:55 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:55.071+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:55.374 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:55 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:55.071+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:55.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:55 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:55.374+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:55.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:55 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:55.374+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:55.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:55 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:55.374+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.165699+0000 front 2024-03-21T11:51:18.166057+0000 (oldest deadline 2024-03-21T11:51:44.065407+0000) 2024-03-21T11:51:55.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:55 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:55.374+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:26.966331+0000 front 2024-03-21T11:51:26.966298+0000 (oldest deadline 2024-03-21T11:51:52.879778+0000) 2024-03-21T11:51:55.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:55 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:55.486+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:55.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:55 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:55.486+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:55.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:55 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:55.486+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:55.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:55 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:55.486+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:55.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:55 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:55.486+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:51:56.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:55 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:55.898+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:56.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:55 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:55.898+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:56.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:55 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:55.898+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:51:56.242 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:55 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:55.898+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:27.476750+0000 front 2024-03-21T11:51:27.476714+0000 (oldest deadline 2024-03-21T11:51:53.877340+0000) 2024-03-21T11:51:56.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:56 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:56.017+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:56.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:56 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:56.017+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:56.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:56 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:56.017+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:56.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:56 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:56.017+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:51:56.351 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:56 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:56.080+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:56.351 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:56 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:56.080+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:56.351 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:56 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:56.080+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:56.351 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:56 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:56.080+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:56.351 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:56 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:56.080+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:51:56.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:56 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:56.350+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.061564+0000 front 2024-03-21T11:51:02.061581+0000 (oldest deadline 2024-03-21T11:51:26.161708+0000) 2024-03-21T11:51:56.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:56 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:56.350+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:11.263958+0000 front 2024-03-21T11:51:11.263898+0000 (oldest deadline 2024-03-21T11:51:34.163832+0000) 2024-03-21T11:51:56.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:56 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:56.350+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.165699+0000 front 2024-03-21T11:51:18.166057+0000 (oldest deadline 2024-03-21T11:51:44.065407+0000) 2024-03-21T11:51:56.643 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:56 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7[53977]: 2024-03-21T11:51:56.350+0000 7f8fa9847700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:26.966331+0000 front 2024-03-21T11:51:26.966298+0000 (oldest deadline 2024-03-21T11:51:52.879778+0000) 2024-03-21T11:51:56.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:56 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:56.459+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:56.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:56 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:56.459+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:56.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:56 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:56.459+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:56.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:56 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:56.459+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:56.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:56 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:56.459+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:51:57.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:56 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:56.901+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:57.267 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:56 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:56.901+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:57.267 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:56 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:56.901+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:51:57.267 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:56 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:56.901+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:27.476750+0000 front 2024-03-21T11:51:27.476714+0000 (oldest deadline 2024-03-21T11:51:53.877340+0000) 2024-03-21T11:51:57.268 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:57 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:56.999+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:57.268 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:57 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:56.999+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:57.268 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:57 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:56.999+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:57.268 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:57 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:56.999+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:51:57.393 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:57 smithi100 podman[78581]: 2024-03-21 11:51:57.137228513 +0000 UTC m=+5.660567021 container died 2f005f20133836e3dd818abd0f98b009fe1cb1ea47b5934b7146cc581a5508db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-21T11:51:57.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:57 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:57.122+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:57.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:57 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:57.122+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:57.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:57 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:57.122+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:57.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:57 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:57.122+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:57.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:57 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:57.122+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:51:57.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:57 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:57.422+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:57.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:57 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:57.422+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:57.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:57 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:57.422+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:57.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:57 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:57.422+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:57.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:57 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:57.422+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:51:57.819 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:57 smithi100 podman[78581]: 2024-03-21 11:51:57.501975247 +0000 UTC m=+6.025313756 container cleanup 2f005f20133836e3dd818abd0f98b009fe1cb1ea47b5934b7146cc581a5508db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:51:57.819 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:57 smithi100 bash[78581]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7 2024-03-21T11:51:58.112 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:57 smithi100 podman[78599]: 2024-03-21 11:51:57.819908304 +0000 UTC m=+0.675201698 container remove 2f005f20133836e3dd818abd0f98b009fe1cb1ea47b5934b7146cc581a5508db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-21T11:51:58.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:58 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:58.031+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:58.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:58 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:58.031+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:58.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:58 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:58.031+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:58.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:58 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:58.031+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:51:58.242 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:57 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:57.943+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:58.242 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:57 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:57.943+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:58.242 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:57 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:57.943+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:51:58.242 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:57 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:57.943+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:27.476750+0000 front 2024-03-21T11:51:27.476714+0000 (oldest deadline 2024-03-21T11:51:53.877340+0000) 2024-03-21T11:51:58.393 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:58 smithi100 podman[78688]: 2024-03-21 11:51:58.112652559 +0000 UTC m=+0.107653553 container create be5c5578b75d7b4b736fe286edf381394193e489aaef620192a7a7ac27ea0508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-21T11:51:58.393 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:58 smithi100 podman[78688]: 2024-03-21 11:51:58.022842787 +0000 UTC m=+0.017843791 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:51:58.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:58 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:58.144+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:58.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:58 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:58.144+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:58.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:58 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:58.144+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:58.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:58 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:58.144+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:58.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:58 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:58.144+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:51:58.711 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:58 smithi100 podman[78688]: 2024-03-21 11:51:58.420384281 +0000 UTC m=+0.415385285 container init be5c5578b75d7b4b736fe286edf381394193e489aaef620192a7a7ac27ea0508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-21T11:51:58.711 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:58 smithi100 podman[78688]: 2024-03-21 11:51:58.42419852 +0000 UTC m=+0.419199514 container start be5c5578b75d7b4b736fe286edf381394193e489aaef620192a7a7ac27ea0508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319) 2024-03-21T11:51:58.711 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:58 smithi100 podman[78688]: 2024-03-21 11:51:58.50724066 +0000 UTC m=+0.502241666 container attach be5c5578b75d7b4b736fe286edf381394193e489aaef620192a7a7ac27ea0508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-03-21T11:51:58.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:58 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:58.422+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:58.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:58 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:58.422+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:58.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:58 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:58.422+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:58.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:58 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:58.422+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:58.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:58 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:58.422+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:51:59.018 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 11:51:58 smithi100 podman[78688]: 2024-03-21 11:51:58.71138915 +0000 UTC m=+0.706390146 container died be5c5578b75d7b4b736fe286edf381394193e489aaef620192a7a7ac27ea0508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-21T11:51:59.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:58 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:58.960+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:51:59.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:58 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:58.960+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:51:59.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:58 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:58.960+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:51:59.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:58 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:58.960+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:27.476750+0000 front 2024-03-21T11:51:27.476714+0000 (oldest deadline 2024-03-21T11:51:53.877340+0000) 2024-03-21T11:51:59.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:59 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:59.063+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:51:59.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:59 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:59.063+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:51:59.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:59 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:59.063+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:51:59.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:51:59 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:51:59.063+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:51:59.302 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:59 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:59.141+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:51:59.302 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:59 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:59.141+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:51:59.303 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:59 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:59.141+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:51:59.303 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:59 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:59.141+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:51:59.303 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:51:59 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:51:59.141+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:51:59.323 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.7.service' 2024-03-21T11:51:59.365 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:51:59.365 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-21T11:51:59.365 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-21T11:51:59.366 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.8 2024-03-21T11:51:59.683 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:59 smithi102 systemd[1]: Stopping Ceph osd.8 for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:51:59.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:59 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:59.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:51:59.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:59 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:59.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:51:59.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:59 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:59.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:51:59.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:59 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:59.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:51:59.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:51:59 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:51:59.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:51:59.963 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:59 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:59.682+0000 7f71c91e6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:51:59.964 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:59 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:59.682+0000 7f71c91e6700 -1 osd.8 100 *** Got signal Terminated *** 2024-03-21T11:51:59.964 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:59 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:59.682+0000 7f71c91e6700 -1 osd.8 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T11:52:00.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:59 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:59.963+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:52:00.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:59 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:59.963+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:52:00.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:59 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:59.963+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:52:00.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:51:59 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:51:59.963+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:27.476750+0000 front 2024-03-21T11:51:27.476714+0000 (oldest deadline 2024-03-21T11:51:53.877340+0000) 2024-03-21T11:52:00.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:00 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:00.105+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:00.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:00 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:00.105+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:00.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:00 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:00.105+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:00.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:00 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:00.105+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:00.488 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:00 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:00.173+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:00.488 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:00 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:00.173+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:00.488 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:00 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:00.173+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:00.488 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:00 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:00.173+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:00.488 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:00 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:00.173+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:00.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:00 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:00.428+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:52:00.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:00 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:00.428+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:52:00.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:00 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:00.428+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:52:00.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:00 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:00.428+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:52:00.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:00 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:00.428+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:52:01.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:01 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:01.128+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:01.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:01 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:01.128+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:01.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:01 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:01.128+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:01.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:01 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:01.128+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:01.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:01 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:01.128+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:01.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:01 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:01.128+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:01.491 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:01 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:01.009+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:52:01.491 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:01 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:01.009+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:52:01.491 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:01 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:01.009+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:52:01.491 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:01 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:01.009+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:27.476750+0000 front 2024-03-21T11:51:27.476714+0000 (oldest deadline 2024-03-21T11:51:53.877340+0000) 2024-03-21T11:52:01.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:01 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:01.060+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:01.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:01 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:01.060+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:01.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:01 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:01.060+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:01.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:01 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:01.060+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:01.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:01 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:01.060+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:01.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:01 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:01.475+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:52:01.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:01 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:01.475+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:52:01.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:01 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:01.475+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:52:01.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:01 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:01.475+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:52:01.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:01 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:01.475+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:52:02.491 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:02 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:02.044+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:52:02.492 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:02 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:02.044+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:52:02.492 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:02 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:02.044+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:52:02.492 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:02 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:02.044+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:27.476750+0000 front 2024-03-21T11:51:27.476714+0000 (oldest deadline 2024-03-21T11:51:53.877340+0000) 2024-03-21T11:52:02.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:02 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:02.086+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:02.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:02 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:02.086+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:02.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:02 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:02.086+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:02.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:02 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:02.086+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:02.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:02 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:02.086+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:02.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:02 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:02.144+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:02.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:02 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:02.144+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:02.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:02 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:02.144+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:02.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:02 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:02.144+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:02.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:02 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:02.144+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:02.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:02 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:02.144+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:02.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:02 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:02.478+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:52:02.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:02 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:02.478+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:52:02.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:02 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:02.478+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:52:02.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:02 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:02.478+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:52:02.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:02 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:02.478+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:52:03.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:03 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:03.098+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:03.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:03 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:03.098+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:03.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:03 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:03.098+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:03.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:03 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:03.098+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:03.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:03 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:03.098+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:03.492 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:02 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:02.997+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:52:03.492 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:02 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:02.997+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:52:03.492 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:02 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:02.997+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:52:03.492 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:02 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:02.997+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:27.476750+0000 front 2024-03-21T11:51:27.476714+0000 (oldest deadline 2024-03-21T11:51:53.877340+0000) 2024-03-21T11:52:03.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:03 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:03.176+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:03.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:03 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:03.176+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:03.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:03 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:03.176+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:03.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:03 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:03.176+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:03.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:03 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:03.176+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:03.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:03 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:03.176+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:03.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:03 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:03.466+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:52:03.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:03 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:03.466+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:52:03.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:03 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:03.466+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:52:03.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:03 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:03.466+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:52:03.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:03 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:03.466+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:52:04.491 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:04 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:04.044+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.473002+0000 front 2024-03-21T11:51:03.473036+0000 (oldest deadline 2024-03-21T11:51:28.173109+0000) 2024-03-21T11:52:04.491 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:04 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:04.044+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:08.173508+0000 front 2024-03-21T11:51:08.173524+0000 (oldest deadline 2024-03-21T11:51:34.073763+0000) 2024-03-21T11:52:04.491 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:04 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:04.044+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:21.576270+0000 front 2024-03-21T11:51:21.576401+0000 (oldest deadline 2024-03-21T11:51:47.476092+0000) 2024-03-21T11:52:04.491 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:04 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8[55454]: 2024-03-21T11:52:04.044+0000 7f71c17d2700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:27.476750+0000 front 2024-03-21T11:51:27.476714+0000 (oldest deadline 2024-03-21T11:51:53.877340+0000) 2024-03-21T11:52:04.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:04 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:04.070+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:04.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:04 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:04.070+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:04.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:04 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:04.070+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:04.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:04 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:04.070+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:04.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:04 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:04.070+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:04.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:04 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:04.146+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:04.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:04 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:04.146+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:04.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:04 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:04.146+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:04.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:04 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:04.146+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:04.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:04 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:04.146+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:04.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:04 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:04.146+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:04.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:04 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:04.475+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:52:04.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:04 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:04.475+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:52:04.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:04 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:04.475+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:52:04.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:04 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:04.475+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:52:04.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:04 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:04.475+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:52:04.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:04 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:04.475+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:34.164251+0000 front 2024-03-21T11:51:34.164310+0000 (oldest deadline 2024-03-21T11:52:04.165942+0000) 2024-03-21T11:52:05.382 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:05 smithi102 podman[78931]: 2024-03-21 11:52:05.109234477 +0000 UTC m=+5.547943862 container died 9cabba3f6551ca0797df16e6b7340eda66e4dcd23f9b4af726bc3b2d211876fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-21T11:52:05.382 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:05 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:05.100+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:05.382 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:05 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:05.100+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:05.383 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:05 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:05.100+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:05.383 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:05 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:05.100+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:05.383 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:05 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:05.100+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:05.642 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:05 smithi102 podman[78931]: 2024-03-21 11:52:05.382496349 +0000 UTC m=+5.821205725 container cleanup 9cabba3f6551ca0797df16e6b7340eda66e4dcd23f9b4af726bc3b2d211876fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-21T11:52:05.642 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:05 smithi102 bash[78931]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8 2024-03-21T11:52:05.642 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:05 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:05.177+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:05.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:05 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:05.177+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:05.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:05 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:05.177+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:05.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:05 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:05.177+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:05.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:05 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:05.177+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:05.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:05 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:05.177+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:05.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:05 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:05.474+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:52:05.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:05 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:05.474+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:52:05.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:05 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:05.474+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:52:05.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:05 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:05.474+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:52:05.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:05 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:05.474+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:52:05.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:05 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:05.474+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:34.164251+0000 front 2024-03-21T11:51:34.164310+0000 (oldest deadline 2024-03-21T11:52:04.165942+0000) 2024-03-21T11:52:05.986 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:05 smithi102 podman[78948]: 2024-03-21 11:52:05.642162649 +0000 UTC m=+0.532596327 container remove 9cabba3f6551ca0797df16e6b7340eda66e4dcd23f9b4af726bc3b2d211876fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-21T11:52:06.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:05 smithi102 podman[79045]: 2024-03-21 11:52:05.888774696 +0000 UTC m=+0.017578110 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:52:06.241 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:06 smithi102 podman[79045]: 2024-03-21 11:52:06.001643184 +0000 UTC m=+0.130446590 container create ecf3982843d96566fb5cf44b833c3991b1212a99dee2325dcc5d65315f962bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:52:06.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:06 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:06.134+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:06.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:06 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:06.134+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:06.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:06 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:06.134+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:06.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:06 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:06.134+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:06.242 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:06 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:06.134+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:06.536 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:06 smithi102 podman[79045]: 2024-03-21 11:52:06.246053505 +0000 UTC m=+0.374856924 container init ecf3982843d96566fb5cf44b833c3991b1212a99dee2325dcc5d65315f962bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-21T11:52:06.536 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:06 smithi102 podman[79045]: 2024-03-21 11:52:06.249158955 +0000 UTC m=+0.377962352 container start ecf3982843d96566fb5cf44b833c3991b1212a99dee2325dcc5d65315f962bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T11:52:06.536 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:06 smithi102 podman[79045]: 2024-03-21 11:52:06.337525481 +0000 UTC m=+0.466328897 container attach ecf3982843d96566fb5cf44b833c3991b1212a99dee2325dcc5d65315f962bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T11:52:06.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:06 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:06.172+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:06.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:06 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:06.172+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:06.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:06 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:06.172+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:06.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:06 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:06.172+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:06.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:06 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:06.172+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:06.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:06 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:06.172+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:06.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:06 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:06.468+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:52:06.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:06 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:06.468+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:52:06.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:06 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:06.468+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:52:06.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:06 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:06.468+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:52:06.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:06 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:06.468+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:52:06.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:06 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:06.468+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:34.164251+0000 front 2024-03-21T11:51:34.164310+0000 (oldest deadline 2024-03-21T11:52:04.165942+0000) 2024-03-21T11:52:06.898 INFO:journalctl@ceph.osd.8.smithi102.stdout:Mar 21 11:52:06 smithi102 podman[79045]: 2024-03-21 11:52:06.535999256 +0000 UTC m=+0.664802686 container died ecf3982843d96566fb5cf44b833c3991b1212a99dee2325dcc5d65315f962bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-21T11:52:07.224 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:07 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:07.106+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:07.224 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:07 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:07.106+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:07.225 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:07 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:07.106+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:07.225 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:07 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:07.106+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:07.225 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:07 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:07.106+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:07.248 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.8.service' 2024-03-21T11:52:07.293 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:52:07.293 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-21T11:52:07.293 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-21T11:52:07.293 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.9 2024-03-21T11:52:07.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:07 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:07.153+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:07.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:07 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:07.153+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:07.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:07 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:07.153+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:07.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:07 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:07.153+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:07.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:07 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:07.153+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:07.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:07 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:07.153+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:07.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:07 smithi047 systemd[1]: Stopping Ceph osd.9 for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:52:07.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:07 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:07.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:52:07.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:07 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:07.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:52:07.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:07 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:07.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:52:07.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:07 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:07.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:52:07.816 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:07 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:07.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:52:07.816 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:07 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:07.457+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:34.164251+0000 front 2024-03-21T11:51:34.164310+0000 (oldest deadline 2024-03-21T11:52:04.165942+0000) 2024-03-21T11:52:08.314 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:07 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:07.835+0000 7f53dd200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:52:08.314 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:07 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:07.835+0000 7f53dd200700 -1 osd.9 100 *** Got signal Terminated *** 2024-03-21T11:52:08.315 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:07 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:07.835+0000 7f53dd200700 -1 osd.9 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T11:52:08.435 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:08 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:08.098+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:08.435 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:08 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:08.098+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:08.435 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:08 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:08.098+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:08.435 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:08 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:08.098+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:08.435 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:08 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:08.098+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:08.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:08 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:08.146+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:08.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:08 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:08.146+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:08.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:08 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:08.146+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:08.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:08 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:08.146+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:08.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:08 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:08.146+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:08.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:08 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:08.146+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:08.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:08 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:08.493+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:52:08.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:08 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:08.493+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:52:08.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:08 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:08.493+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:52:08.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:08 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:08.493+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:52:08.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:08 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:08.493+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:52:08.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:08 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:08.493+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:34.164251+0000 front 2024-03-21T11:51:34.164310+0000 (oldest deadline 2024-03-21T11:52:04.165942+0000) 2024-03-21T11:52:09.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:09 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:09.101+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:09.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:09 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:09.101+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:09.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:09 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:09.101+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:09.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:09 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:09.101+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:09.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:09 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:09.101+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:09.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:09 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:09.161+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:09.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:09 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:09.161+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:09.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:09 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:09.161+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:09.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:09 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:09.161+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:09.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:09 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:09.161+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:09.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:09 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:09.161+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:09.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:09 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:09.444+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:52:09.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:09 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:09.444+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:52:09.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:09 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:09.444+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:52:09.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:09 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:09.444+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:52:09.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:09 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:09.444+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:52:09.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:09 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:09.444+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:34.164251+0000 front 2024-03-21T11:51:34.164310+0000 (oldest deadline 2024-03-21T11:52:04.165942+0000) 2024-03-21T11:52:10.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:10 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:10.156+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:10.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:10 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:10.156+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:10.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:10 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:10.156+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:10.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:10 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:10.156+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:10.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:10 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:10.156+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:10.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:10 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:10.156+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:10.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:10 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:10.156+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:46.089713+0000 front 2024-03-21T11:51:46.089592+0000 (oldest deadline 2024-03-21T11:52:09.589438+0000) 2024-03-21T11:52:10.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:10 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:10.110+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:10.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:10 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:10.110+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:10.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:10 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:10.110+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:10.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:10 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:10.110+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:10.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:10 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:10.110+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:10.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:10 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:10.110+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:10.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:10 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:10.472+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:52:10.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:10 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:10.472+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:52:10.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:10 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:10.472+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:52:10.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:10 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:10.472+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:52:10.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:10 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:10.472+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:52:10.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:10 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:10.472+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:34.164251+0000 front 2024-03-21T11:51:34.164310+0000 (oldest deadline 2024-03-21T11:52:04.165942+0000) 2024-03-21T11:52:11.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:11 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:11.082+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:11.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:11 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:11.082+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:11.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:11 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:11.082+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:11.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:11 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:11.082+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:11.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:11 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:11.082+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:11.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:11 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:11.082+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:11.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:11 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:11.153+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:11.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:11 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:11.153+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:11.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:11 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:11.153+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:11.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:11 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:11.153+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:11.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:11 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:11.153+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:11.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:11 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:11.153+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:11.644 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:11 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:11.153+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:46.089713+0000 front 2024-03-21T11:51:46.089592+0000 (oldest deadline 2024-03-21T11:52:09.589438+0000) 2024-03-21T11:52:11.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:11 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:11.497+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:52:11.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:11 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:11.497+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:52:11.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:11 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:11.497+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:52:11.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:11 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:11.497+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:52:11.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:11 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:11.497+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:52:11.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:11 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:11.497+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:34.164251+0000 front 2024-03-21T11:51:34.164310+0000 (oldest deadline 2024-03-21T11:52:04.165942+0000) 2024-03-21T11:52:12.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:12 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:12.134+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:12.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:12 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:12.134+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:12.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:12 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:12.134+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:12.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:12 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:12.134+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:12.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:12 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:12.134+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:12.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:12 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:12.134+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:12.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:12 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:12.134+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:46.089713+0000 front 2024-03-21T11:51:46.089592+0000 (oldest deadline 2024-03-21T11:52:09.589438+0000) 2024-03-21T11:52:12.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:12 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:12.093+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:12.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:12 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:12.093+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:12.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:12 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:12.093+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:12.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:12 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:12.093+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:12.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:12 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:12.093+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:12.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:12 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:12.093+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:12.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:12 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:12.490+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.544627+0000 front 2024-03-21T11:50:54.544816+0000 (oldest deadline 2024-03-21T11:51:18.644874+0000) 2024-03-21T11:52:12.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:12 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:12.490+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:02.148231+0000 front 2024-03-21T11:51:02.148155+0000 (oldest deadline 2024-03-21T11:51:26.248375+0000) 2024-03-21T11:52:12.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:12 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:12.490+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:12.653628+0000 front 2024-03-21T11:51:12.653449+0000 (oldest deadline 2024-03-21T11:51:38.553720+0000) 2024-03-21T11:52:12.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:12 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:12.490+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.857183+0000 front 2024-03-21T11:51:20.857297+0000 (oldest deadline 2024-03-21T11:51:45.557437+0000) 2024-03-21T11:52:12.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:12 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:12.490+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.961420+0000 front 2024-03-21T11:51:28.961502+0000 (oldest deadline 2024-03-21T11:51:53.661648+0000) 2024-03-21T11:52:12.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:12 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9[63617]: 2024-03-21T11:52:12.490+0000 7f53d3a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:34.164251+0000 front 2024-03-21T11:51:34.164310+0000 (oldest deadline 2024-03-21T11:52:04.165942+0000) 2024-03-21T11:52:13.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:13 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:13.106+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:13.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:13 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:13.106+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:13.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:13 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:13.106+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:13.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:13 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:13.106+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:13.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:13 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:13.106+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:13.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:13 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:13.106+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:13.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:13 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:13.106+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:46.089713+0000 front 2024-03-21T11:51:46.089592+0000 (oldest deadline 2024-03-21T11:52:09.589438+0000) 2024-03-21T11:52:13.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:13 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:13.108+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:13.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:13 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:13.108+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:13.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:13 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:13.108+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:13.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:13 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:13.108+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:13.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:13 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:13.108+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:13.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:13 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:13.108+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:14.043 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:13 smithi047 podman[125869]: 2024-03-21 11:52:13.777703028 +0000 UTC m=+6.063500336 container died 6debc0f9d05a2c4d8f389b57957ace22f136581a994f33e5c0746c2692598859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-21T11:52:14.314 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:14 smithi047 podman[125869]: 2024-03-21 11:52:14.165667617 +0000 UTC m=+6.451464938 container cleanup 6debc0f9d05a2c4d8f389b57957ace22f136581a994f33e5c0746c2692598859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T11:52:14.314 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:14 smithi047 bash[125869]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9 2024-03-21T11:52:14.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:14 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:14.128+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:14.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:14 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:14.128+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:14.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:14 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:14.128+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:14.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:14 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:14.128+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:14.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:14 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:14.128+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:14.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:14 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:14.128+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:14.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:14 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:14.151+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:14.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:14 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:14.151+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:14.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:14 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:14.151+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:14.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:14 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:14.151+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:14.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:14 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:14.151+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:14.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:14 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:14.151+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:14.644 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:14 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:14.151+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:46.089713+0000 front 2024-03-21T11:51:46.089592+0000 (oldest deadline 2024-03-21T11:52:09.589438+0000) 2024-03-21T11:52:14.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:14 smithi047 podman[125887]: 2024-03-21 11:52:14.378847616 +0000 UTC m=+0.583542855 container remove 6debc0f9d05a2c4d8f389b57957ace22f136581a994f33e5c0746c2692598859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-21T11:52:15.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:15 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:15.165+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:15.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:15 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:15.165+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:15.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:15 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:15.165+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:15.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:15 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:15.165+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:15.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:15 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:15.165+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:15.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:15 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:15.165+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:15.509 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:15 smithi047 podman[125976]: 2024-03-21 11:52:15.127965253 +0000 UTC m=+0.038536439 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:52:15.509 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:15 smithi047 podman[125976]: 2024-03-21 11:52:15.254456713 +0000 UTC m=+0.165027897 container create 47850aa70e01412b23d8ac9cb29c08b63ab06d5d4c8b8444d836a7c30f512f58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-21T11:52:15.509 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:15 smithi047 podman[125976]: 2024-03-21 11:52:15.507959317 +0000 UTC m=+0.418530535 container init 47850aa70e01412b23d8ac9cb29c08b63ab06d5d4c8b8444d836a7c30f512f58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:52:15.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:15 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:15.186+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:15.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:15 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:15.186+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:15.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:15 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:15.186+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:15.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:15 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:15.186+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:15.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:15 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:15.186+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:15.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:15 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:15.186+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:15.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:15 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:15.186+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:46.089713+0000 front 2024-03-21T11:51:46.089592+0000 (oldest deadline 2024-03-21T11:52:09.589438+0000) 2024-03-21T11:52:15.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:15 smithi047 podman[125976]: 2024-03-21 11:52:15.519305813 +0000 UTC m=+0.429876998 container start 47850aa70e01412b23d8ac9cb29c08b63ab06d5d4c8b8444d836a7c30f512f58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-03-21T11:52:15.814 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:15 smithi047 podman[125976]: 2024-03-21 11:52:15.598460175 +0000 UTC m=+0.509031429 container attach 47850aa70e01412b23d8ac9cb29c08b63ab06d5d4c8b8444d836a7c30f512f58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-21T11:52:16.315 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 11:52:15 smithi047 podman[125976]: 2024-03-21 11:52:15.976467924 +0000 UTC m=+0.887039116 container died 47850aa70e01412b23d8ac9cb29c08b63ab06d5d4c8b8444d836a7c30f512f58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-21T11:52:16.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:16 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:16.115+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:16.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:16 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:16.115+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:16.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:16 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:16.115+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:16.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:16 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:16.115+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:16.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:16 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:16.115+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:16.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:16 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:16.115+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:16.617 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.9.service' 2024-03-21T11:52:16.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:16 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:16.181+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:16.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:16 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:16.181+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:16.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:16 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:16.181+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:16.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:16 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:16.181+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:16.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:16 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:16.181+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:16.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:16 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:16.181+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:16.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:16 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:16.181+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:46.089713+0000 front 2024-03-21T11:51:46.089592+0000 (oldest deadline 2024-03-21T11:52:09.589438+0000) 2024-03-21T11:52:16.724 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:52:16.725 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-21T11:52:16.725 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-21T11:52:16.725 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.10 2024-03-21T11:52:17.030 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:16 smithi100 systemd[1]: Stopping Ceph osd.10 for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:52:17.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:17.029+0000 7fc11aeac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:52:17.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:17.029+0000 7fc11aeac700 -1 osd.10 100 *** Got signal Terminated *** 2024-03-21T11:52:17.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:17.029+0000 7fc11aeac700 -1 osd.10 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T11:52:17.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:17.208+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:17.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:17.208+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:17.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:17.208+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:17.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:17.208+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:17.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:17.208+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:17.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:17.208+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:17.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:17 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:17.208+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:46.089713+0000 front 2024-03-21T11:51:46.089592+0000 (oldest deadline 2024-03-21T11:52:09.589438+0000) 2024-03-21T11:52:17.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:17 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:17.126+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:17.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:17 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:17.126+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:17.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:17 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:17.126+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:17.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:17 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:17.126+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:17.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:17 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:17.126+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:17.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:17 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:17.126+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:18.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:18 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:18.145+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:18.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:18 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:18.145+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:18.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:18 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:18.145+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:18.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:18 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:18.145+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:18.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:18 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:18.145+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:18.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:18 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:18.145+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:18.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:18 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:18.177+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:18.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:18 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:18.177+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:18.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:18 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:18.177+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:18.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:18 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:18.177+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:18.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:18 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:18.177+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:18.643 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:18 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:18.177+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:18.644 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:18 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:18.177+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:46.089713+0000 front 2024-03-21T11:51:46.089592+0000 (oldest deadline 2024-03-21T11:52:09.589438+0000) 2024-03-21T11:52:19.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:19 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:19.135+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:19.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:19 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:19.135+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:19.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:19 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:19.135+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:19.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:19 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:19.135+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:19.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:19 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:19.135+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:19.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:19 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:19.135+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:19.394 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:19 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:19.135+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:46.089713+0000 front 2024-03-21T11:51:46.089592+0000 (oldest deadline 2024-03-21T11:52:09.589438+0000) 2024-03-21T11:52:19.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:19 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:19.123+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:19.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:19 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:19.123+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:19.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:19 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:19.123+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:19.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:19 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:19.123+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:19.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:19 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:19.123+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:19.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:19 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:19.123+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:20.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:20 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:20.131+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:20.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:20 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:20.131+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:20.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:20 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:20.131+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:20.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:20 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:20.131+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:20.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:20 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:20.131+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:20.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:20 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:20.131+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:20.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:20 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:20.131+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:46.089713+0000 front 2024-03-21T11:51:46.089592+0000 (oldest deadline 2024-03-21T11:52:09.589438+0000) 2024-03-21T11:52:20.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:20 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:20.103+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:20.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:20 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:20.103+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:20.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:20 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:20.103+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:20.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:20 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:20.103+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:20.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:20 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:20.103+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:20.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:20 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:20.103+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:20.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:20 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:20.103+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T11:51:53.851064+0000 front 2024-03-21T11:51:53.850995+0000 (oldest deadline 2024-03-21T11:52:19.150938+0000) 2024-03-21T11:52:21.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:21 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:21.088+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:50.978899+0000 front 2024-03-21T11:50:50.978857+0000 (oldest deadline 2024-03-21T11:51:16.278731+0000) 2024-03-21T11:52:21.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:21 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:21.088+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T11:51:03.880614+0000 front 2024-03-21T11:51:03.880639+0000 (oldest deadline 2024-03-21T11:51:25.580727+0000) 2024-03-21T11:52:21.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:21 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:21.088+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:10.683037+0000 front 2024-03-21T11:51:10.683016+0000 (oldest deadline 2024-03-21T11:51:35.383088+0000) 2024-03-21T11:52:21.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:21 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:21.088+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:20.684703+0000 front 2024-03-21T11:51:20.684452+0000 (oldest deadline 2024-03-21T11:51:42.984393+0000) 2024-03-21T11:52:21.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:21 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:21.088+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:30.486546+0000 front 2024-03-21T11:51:30.486619+0000 (oldest deadline 2024-03-21T11:51:55.186653+0000) 2024-03-21T11:52:21.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:21 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:21.088+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:35.187299+0000 front 2024-03-21T11:51:35.187195+0000 (oldest deadline 2024-03-21T11:52:00.387562+0000) 2024-03-21T11:52:21.393 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:21 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10[60623]: 2024-03-21T11:52:21.088+0000 7fc113498700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:46.089713+0000 front 2024-03-21T11:51:46.089592+0000 (oldest deadline 2024-03-21T11:52:09.589438+0000) 2024-03-21T11:52:21.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:21 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:21.094+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:21.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:21 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:21.094+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:21.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:21 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:21.094+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:21.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:21 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:21.094+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:21.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:21 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:21.094+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:21.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:21 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:21.094+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:21.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:21 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:21.094+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T11:51:53.851064+0000 front 2024-03-21T11:51:53.850995+0000 (oldest deadline 2024-03-21T11:52:19.150938+0000) 2024-03-21T11:52:22.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:22 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:22.082+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:22.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:22 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:22.082+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:22.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:22 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:22.082+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:22.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:22 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:22.082+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:22.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:22 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:22.082+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:22.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:22 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:22.082+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:22.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:22 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:22.082+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T11:51:53.851064+0000 front 2024-03-21T11:51:53.850995+0000 (oldest deadline 2024-03-21T11:52:19.150938+0000) 2024-03-21T11:52:22.796 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:22 smithi100 podman[78807]: 2024-03-21 11:52:22.545594052 +0000 UTC m=+5.626849453 container died a397990779ce077268cb41712d830bf84069795a7e12156e1d9e574e3fca4d6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-21T11:52:23.143 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:22 smithi100 podman[78807]: 2024-03-21 11:52:22.895701202 +0000 UTC m=+5.976956603 container cleanup a397990779ce077268cb41712d830bf84069795a7e12156e1d9e574e3fca4d6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T11:52:23.143 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:22 smithi100 bash[78807]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10 2024-03-21T11:52:23.489 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:23 smithi100 podman[78826]: 2024-03-21 11:52:23.180249179 +0000 UTC m=+0.630235808 container remove a397990779ce077268cb41712d830bf84069795a7e12156e1d9e574e3fca4d6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-21T11:52:23.493 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:23 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:23.073+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:23.493 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:23 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:23.073+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:23.493 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:23 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:23.073+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:23.493 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:23 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:23.073+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:23.494 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:23 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:23.073+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:23.494 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:23 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:23.073+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:23.494 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:23 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:23.073+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T11:51:53.851064+0000 front 2024-03-21T11:51:53.850995+0000 (oldest deadline 2024-03-21T11:52:19.150938+0000) 2024-03-21T11:52:23.817 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:23 smithi100 podman[78921]: 2024-03-21 11:52:23.489376853 +0000 UTC m=+0.110722587 container create 2715fae56c2500aacbc551c82d43711af30c08d11e3efa60f6dbf4ecbedf7e26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-21T11:52:23.817 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:23 smithi100 podman[78921]: 2024-03-21 11:52:23.396313157 +0000 UTC m=+0.017658912 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:52:23.817 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:23 smithi100 podman[78921]: 2024-03-21 11:52:23.733592639 +0000 UTC m=+0.354938373 container init 2715fae56c2500aacbc551c82d43711af30c08d11e3efa60f6dbf4ecbedf7e26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-21T11:52:23.818 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:23 smithi100 podman[78921]: 2024-03-21 11:52:23.736850466 +0000 UTC m=+0.358196194 container start 2715fae56c2500aacbc551c82d43711af30c08d11e3efa60f6dbf4ecbedf7e26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-21T11:52:23.818 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:23 smithi100 podman[78921]: 2024-03-21 11:52:23.81671028 +0000 UTC m=+0.438056043 container attach 2715fae56c2500aacbc551c82d43711af30c08d11e3efa60f6dbf4ecbedf7e26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-21T11:52:24.143 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 11:52:24 smithi100 podman[78921]: 2024-03-21 11:52:24.018273399 +0000 UTC m=+0.639619134 container died 2715fae56c2500aacbc551c82d43711af30c08d11e3efa60f6dbf4ecbedf7e26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-21T11:52:24.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:24 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:24.023+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:24.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:24 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:24.023+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:24.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:24 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:24.023+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:24.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:24 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:24.023+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:24.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:24 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:24.023+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:24.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:24 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:24.023+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:24.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:24 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:24.023+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T11:51:53.851064+0000 front 2024-03-21T11:51:53.850995+0000 (oldest deadline 2024-03-21T11:52:19.150938+0000) 2024-03-21T11:52:24.664 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.10.service' 2024-03-21T11:52:24.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:52:24.708 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-21T11:52:24.708 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-21T11:52:24.708 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.11 2024-03-21T11:52:24.991 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:24 smithi102 systemd[1]: Stopping Ceph osd.11 for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:52:25.439 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:24 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:24.993+0000 7f4821e44700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:52:25.439 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:24 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:24.993+0000 7f4821e44700 -1 osd.11 100 *** Got signal Terminated *** 2024-03-21T11:52:25.439 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:24 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:24.993+0000 7f4821e44700 -1 osd.11 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T11:52:25.440 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:25 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:25.072+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:25.440 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:25 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:25.072+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:25.440 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:25 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:25.072+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:25.440 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:25 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:25.072+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:25.440 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:25 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:25.072+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:25.440 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:25 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:25.072+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:25.440 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:25 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:25.072+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T11:51:53.851064+0000 front 2024-03-21T11:51:53.850995+0000 (oldest deadline 2024-03-21T11:52:19.150938+0000) 2024-03-21T11:52:26.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:26 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:26.110+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:26.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:26 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:26.110+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:26.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:26 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:26.110+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:26.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:26 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:26.110+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:26.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:26 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:26.110+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:26.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:26 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:26.110+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:26.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:26 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:26.110+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T11:51:53.851064+0000 front 2024-03-21T11:51:53.850995+0000 (oldest deadline 2024-03-21T11:52:19.150938+0000) 2024-03-21T11:52:27.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:27 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:27.112+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:27.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:27 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:27.112+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:27.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:27 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:27.112+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:27.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:27 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:27.112+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:27.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:27 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:27.112+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:27.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:27 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:27.112+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:27.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:27 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:27.112+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T11:51:53.851064+0000 front 2024-03-21T11:51:53.850995+0000 (oldest deadline 2024-03-21T11:52:19.150938+0000) 2024-03-21T11:52:28.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:28 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:28.084+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:28.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:28 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:28.084+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:28.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:28 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:28.084+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:28.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:28 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:28.084+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:28.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:28 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:28.084+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:28.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:28 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:28.084+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:28.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:28 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:28.084+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T11:51:53.851064+0000 front 2024-03-21T11:51:53.850995+0000 (oldest deadline 2024-03-21T11:52:19.150938+0000) 2024-03-21T11:52:29.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:29 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:29.090+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T11:50:54.440288+0000 front 2024-03-21T11:50:54.440397+0000 (oldest deadline 2024-03-21T11:51:16.140431+0000) 2024-03-21T11:52:29.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:29 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:29.090+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.2 since back 2024-03-21T11:51:09.444149+0000 front 2024-03-21T11:51:09.444195+0000 (oldest deadline 2024-03-21T11:51:34.144337+0000) 2024-03-21T11:52:29.491 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:29 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:29.090+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T11:51:18.746064+0000 front 2024-03-21T11:51:18.746016+0000 (oldest deadline 2024-03-21T11:51:44.645862+0000) 2024-03-21T11:52:29.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:29 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:29.090+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T11:51:28.147205+0000 front 2024-03-21T11:51:28.147152+0000 (oldest deadline 2024-03-21T11:51:52.247214+0000) 2024-03-21T11:52:29.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:29 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:29.090+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.5 since back 2024-03-21T11:51:37.548292+0000 front 2024-03-21T11:51:37.548221+0000 (oldest deadline 2024-03-21T11:52:00.448434+0000) 2024-03-21T11:52:29.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:29 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:29.090+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T11:51:43.349794+0000 front 2024-03-21T11:51:43.349676+0000 (oldest deadline 2024-03-21T11:52:09.249422+0000) 2024-03-21T11:52:29.492 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:29 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11[62378]: 2024-03-21T11:52:29.090+0000 7f481a430700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T11:51:53.851064+0000 front 2024-03-21T11:51:53.850995+0000 (oldest deadline 2024-03-21T11:52:19.150938+0000) 2024-03-21T11:52:30.918 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:30 smithi102 podman[79162]: 2024-03-21 11:52:30.602586521 +0000 UTC m=+5.704254322 container died 002b68a8f12c1fff6c1060299beaf328d5e97e3425da38513233b26d68d92c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-21T11:52:31.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:30 smithi102 podman[79162]: 2024-03-21 11:52:30.918671192 +0000 UTC m=+6.020338992 container cleanup 002b68a8f12c1fff6c1060299beaf328d5e97e3425da38513233b26d68d92c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-21T11:52:31.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:30 smithi102 bash[79162]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11 2024-03-21T11:52:31.241 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:31 smithi102 podman[79179]: 2024-03-21 11:52:31.136398982 +0000 UTC m=+0.532248755 container remove 002b68a8f12c1fff6c1060299beaf328d5e97e3425da38513233b26d68d92c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:52:31.991 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:31 smithi102 podman[79272]: 2024-03-21 11:52:31.435627074 +0000 UTC m=+0.017825007 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T11:52:31.991 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:31 smithi102 podman[79272]: 2024-03-21 11:52:31.537399439 +0000 UTC m=+0.119597414 container create b8c196e399e9c0ed6e31a4fe5c7a55a9ac36d738b8948c3e9afb65d960a31c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-21T11:52:31.992 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:31 smithi102 podman[79272]: 2024-03-21 11:52:31.706397196 +0000 UTC m=+0.288595141 container init b8c196e399e9c0ed6e31a4fe5c7a55a9ac36d738b8948c3e9afb65d960a31c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:52:31.992 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:31 smithi102 podman[79272]: 2024-03-21 11:52:31.709526118 +0000 UTC m=+0.291724042 container start b8c196e399e9c0ed6e31a4fe5c7a55a9ac36d738b8948c3e9afb65d960a31c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-21T11:52:31.992 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:31 smithi102 podman[79272]: 2024-03-21 11:52:31.781125469 +0000 UTC m=+0.363323411 container attach b8c196e399e9c0ed6e31a4fe5c7a55a9ac36d738b8948c3e9afb65d960a31c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-21T11:52:32.267 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:31 smithi102 podman[79272]: 2024-03-21 11:52:31.994473455 +0000 UTC m=+0.576671378 container died b8c196e399e9c0ed6e31a4fe5c7a55a9ac36d738b8948c3e9afb65d960a31c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-21T11:52:32.522 INFO:journalctl@ceph.osd.11.smithi102.stdout:Mar 21 11:52:32 smithi102 podman[79298]: 2024-03-21 11:52:32.509698877 +0000 UTC m=+0.510081627 container remove b8c196e399e9c0ed6e31a4fe5c7a55a9ac36d738b8948c3e9afb65d960a31c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-osd-11-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-21T11:52:32.528 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@osd.11.service' 2024-03-21T11:52:32.569 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:52:32.569 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-21T11:52:32.569 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-21T11:52:32.569 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.a 2024-03-21T11:52:33.064 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:52:32 smithi047 systemd[1]: Stopping Ceph mds.a for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:52:33.564 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:52:33 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-a[70549]: 2024-03-21T11:52:33.169+0000 7f0682e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:52:33.564 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:52:33 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-a[70549]: 2024-03-21T11:52:33.169+0000 7f0682e00700 -1 mds.a *** got signal Terminated *** 2024-03-21T11:52:35.564 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:52:35 smithi047 podman[126094]: 2024-03-21 11:52:35.200444666 +0000 UTC m=+2.176648938 container died 0463adc15a5cfe7fb9e31c6e21d1a8f8cd73f07f39da46c0fc1e3d8f851dee1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-21T11:52:35.988 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:52:35 smithi047 podman[126094]: 2024-03-21 11:52:35.769841959 +0000 UTC m=+2.746046270 container cleanup 0463adc15a5cfe7fb9e31c6e21d1a8f8cd73f07f39da46c0fc1e3d8f851dee1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T11:52:35.988 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:52:35 smithi047 bash[126094]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-a 2024-03-21T11:52:36.314 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 11:52:35 smithi047 podman[126108]: 2024-03-21 11:52:35.987447809 +0000 UTC m=+0.768253219 container remove 0463adc15a5cfe7fb9e31c6e21d1a8f8cd73f07f39da46c0fc1e3d8f851dee1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T11:52:36.474 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.a.service' 2024-03-21T11:52:36.602 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:52:36.603 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-21T11:52:36.603 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-21T11:52:36.603 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.d 2024-03-21T11:52:37.064 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:52:36 smithi047 systemd[1]: Stopping Ceph mds.d for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:52:37.564 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:52:37 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-d[69563]: 2024-03-21T11:52:37.154+0000 7fd286200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:52:37.564 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:52:37 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-d[69563]: 2024-03-21T11:52:37.154+0000 7fd286200700 -1 mds.d *** got signal Terminated *** 2024-03-21T11:52:39.467 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:52:39 smithi047 podman[126214]: 2024-03-21 11:52:39.19500392 +0000 UTC m=+2.160737270 container died f6f4b37d470db8f093be83e64edacdd1fdbc502f0101310c55a25ce43850511d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T11:52:39.814 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:52:39 smithi047 podman[126214]: 2024-03-21 11:52:39.573026997 +0000 UTC m=+2.538760341 container cleanup f6f4b37d470db8f093be83e64edacdd1fdbc502f0101310c55a25ce43850511d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:52:39.814 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:52:39 smithi047 bash[126214]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-d 2024-03-21T11:52:40.314 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 11:52:39 smithi047 podman[126228]: 2024-03-21 11:52:39.941000348 +0000 UTC m=+0.726700516 container remove f6f4b37d470db8f093be83e64edacdd1fdbc502f0101310c55a25ce43850511d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-d, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:52:40.466 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.d.service' 2024-03-21T11:52:40.589 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:52:40.589 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-21T11:52:40.589 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-21T11:52:40.589 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.g 2024-03-21T11:52:41.064 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 11:52:40 smithi047 systemd[1]: Stopping Ceph mds.g for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:52:41.564 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 11:52:41 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-g[70189]: 2024-03-21T11:52:41.216+0000 7f58c0a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:52:41.564 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 11:52:41 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-g[70189]: 2024-03-21T11:52:41.216+0000 7f58c0a00700 -1 mds.g *** got signal Terminated *** 2024-03-21T11:52:43.564 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 11:52:43 smithi047 podman[126334]: 2024-03-21 11:52:43.270797324 +0000 UTC m=+2.189123145 container died d14b154447d8c489c4672eea2e32eae8d23c6c934e5ac3fbbc7429c297ef2b22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-g, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-21T11:52:43.970 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 11:52:43 smithi047 podman[126334]: 2024-03-21 11:52:43.693491186 +0000 UTC m=+2.611817000 container cleanup d14b154447d8c489c4672eea2e32eae8d23c6c934e5ac3fbbc7429c297ef2b22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-g, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T11:52:43.970 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 11:52:43 smithi047 bash[126334]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-g 2024-03-21T11:52:44.314 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 11:52:43 smithi047 podman[126348]: 2024-03-21 11:52:43.969430177 +0000 UTC m=+0.676251444 container remove d14b154447d8c489c4672eea2e32eae8d23c6c934e5ac3fbbc7429c297ef2b22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-g, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-21T11:52:44.542 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.g.service' 2024-03-21T11:52:44.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:52:44.664 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-21T11:52:44.664 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-21T11:52:44.664 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.j 2024-03-21T11:52:45.064 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 11:52:44 smithi047 systemd[1]: Stopping Ceph mds.j for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:52:45.564 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 11:52:45 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-j[69200]: 2024-03-21T11:52:45.303+0000 7f44e8200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:52:45.564 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 11:52:45 smithi047 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-j[69200]: 2024-03-21T11:52:45.303+0000 7f44e8200700 -1 mds.j *** got signal Terminated *** 2024-03-21T11:52:48.894 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 11:52:48 smithi047 podman[126453]: 2024-03-21 11:52:48.635739109 +0000 UTC m=+3.448970700 container died 8a6c3bc78d0f8b529308a4f2b49123bdaece972607138ff58af06c5ad572028b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-j, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-21T11:52:49.215 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 11:52:49 smithi047 podman[126453]: 2024-03-21 11:52:49.021876796 +0000 UTC m=+3.835108400 container cleanup 8a6c3bc78d0f8b529308a4f2b49123bdaece972607138ff58af06c5ad572028b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-j, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-21T11:52:49.216 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 11:52:49 smithi047 bash[126453]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-j 2024-03-21T11:52:49.564 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 11:52:49 smithi047 podman[126467]: 2024-03-21 11:52:49.214575538 +0000 UTC m=+0.556062397 container remove 8a6c3bc78d0f8b529308a4f2b49123bdaece972607138ff58af06c5ad572028b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-j, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-21T11:52:49.784 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.j.service' 2024-03-21T11:52:49.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:52:49.905 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-21T11:52:49.906 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-21T11:52:49.906 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.b 2024-03-21T11:52:50.212 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:52:49 smithi100 systemd[1]: Stopping Ceph mds.b for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:52:50.544 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:52:50 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b[66324]: 2024-03-21T11:52:50.211+0000 7f0bfdc07700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:52:50.544 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:52:50 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b[66324]: 2024-03-21T11:52:50.211+0000 7f0bfdc07700 -1 mds.b *** got signal Terminated *** 2024-03-21T11:52:51.621 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:52:51 smithi100 podman[79027]: 2024-03-21 11:52:51.373182883 +0000 UTC m=+1.270870429 container died 3fa90a15426b691dcf39a2f16a6fce372d8eaa4085f74f5ea55d99b95aca0a5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T11:52:51.893 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:52:51 smithi100 podman[79027]: 2024-03-21 11:52:51.762077882 +0000 UTC m=+1.659765425 container cleanup 3fa90a15426b691dcf39a2f16a6fce372d8eaa4085f74f5ea55d99b95aca0a5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T11:52:51.893 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 11:52:51 smithi100 bash[79027]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-b 2024-03-21T11:52:52.117 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.b.service' 2024-03-21T11:52:52.156 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:52:52.156 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-21T11:52:52.156 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-21T11:52:52.156 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.e 2024-03-21T11:52:52.393 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:52:52 smithi100 systemd[1]: Stopping Ceph mds.e for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:52:52.893 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:52:52 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e[73627]: 2024-03-21T11:52:52.511+0000 7f681be42700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:52:52.893 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:52:52 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e[73627]: 2024-03-21T11:52:52.511+0000 7f681be42700 -1 mds.e *** got signal Terminated *** 2024-03-21T11:52:55.643 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:52:55 smithi100 podman[79161]: 2024-03-21 11:52:55.372035204 +0000 UTC m=+3.001161725 container died 9160af86b80f6309e802251f749a79657b9a77bb7d2886319068f1f08793c040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-03-21T11:52:56.005 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:52:55 smithi100 podman[79161]: 2024-03-21 11:52:55.729388134 +0000 UTC m=+3.358514638 container cleanup 9160af86b80f6309e802251f749a79657b9a77bb7d2886319068f1f08793c040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-21T11:52:56.005 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 11:52:55 smithi100 bash[79161]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-e 2024-03-21T11:52:56.167 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.e.service' 2024-03-21T11:52:56.202 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:52:56.203 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-21T11:52:56.203 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-21T11:52:56.203 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.h 2024-03-21T11:52:56.525 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:52:56 smithi100 systemd[1]: Stopping Ceph mds.h for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:52:56.893 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:52:56 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-h[66002]: 2024-03-21T11:52:56.523+0000 7f6684539700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:52:56.893 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:52:56 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-h[66002]: 2024-03-21T11:52:56.523+0000 7f6684539700 -1 mds.h *** got signal Terminated *** 2024-03-21T11:52:59.643 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:52:59 smithi100 podman[79280]: 2024-03-21 11:52:59.37307288 +0000 UTC m=+2.958769634 container died 987da945c2f1dc6a6312d4880d23021a18b7a06261d3dc915844ab449c11eaba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-h, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:53:00.008 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:52:59 smithi100 podman[79280]: 2024-03-21 11:52:59.732891398 +0000 UTC m=+3.318588155 container cleanup 987da945c2f1dc6a6312d4880d23021a18b7a06261d3dc915844ab449c11eaba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-21T11:53:00.009 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 11:52:59 smithi100 bash[79280]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-h 2024-03-21T11:53:00.157 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.h.service' 2024-03-21T11:53:00.191 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:53:00.192 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-21T11:53:00.192 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-21T11:53:00.192 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.k 2024-03-21T11:53:00.643 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:53:00 smithi100 systemd[1]: Stopping Ceph mds.k for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:53:00.643 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:53:00 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-k[65371]: 2024-03-21T11:53:00.510+0000 7fbe12c89700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:53:00.643 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:53:00 smithi100 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-k[65371]: 2024-03-21T11:53:00.510+0000 7fbe12c89700 -1 mds.k *** got signal Terminated *** 2024-03-21T11:53:03.643 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:53:03 smithi100 podman[79402]: 2024-03-21 11:53:03.369843383 +0000 UTC m=+2.965496167 container died 17c5e3dda4ad7c4b6a4aaf08fef89d4139e774972e351a069581d509377d31a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-k, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True) 2024-03-21T11:53:03.920 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:53:03 smithi100 podman[79402]: 2024-03-21 11:53:03.711199408 +0000 UTC m=+3.306852138 container cleanup 17c5e3dda4ad7c4b6a4aaf08fef89d4139e774972e351a069581d509377d31a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-k, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:53:03.920 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 11:53:03 smithi100 bash[79402]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-k 2024-03-21T11:53:04.125 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.k.service' 2024-03-21T11:53:04.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:53:04.161 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-21T11:53:04.161 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-21T11:53:04.161 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.c 2024-03-21T11:53:04.491 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:53:04 smithi102 systemd[1]: Stopping Ceph mds.c for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:53:04.754 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:53:04 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-c[67682]: 2024-03-21T11:53:04.551+0000 7fa32fcdf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:53:04.754 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:53:04 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-c[67682]: 2024-03-21T11:53:04.551+0000 7fa32fcdf700 -1 mds.c *** got signal Terminated *** 2024-03-21T11:53:07.484 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:53:07 smithi102 podman[79377]: 2024-03-21 11:53:07.165827809 +0000 UTC m=+2.809733925 container died 7eee422f14642b5a1d3c9a9a6b2e08d905230ef8440f1d5423d4c1d8e7bbefc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-21T11:53:07.484 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:53:07 smithi102 podman[79377]: 2024-03-21 11:53:07.484648626 +0000 UTC m=+3.128554756 container cleanup 7eee422f14642b5a1d3c9a9a6b2e08d905230ef8440f1d5423d4c1d8e7bbefc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:53:07.741 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:53:07 smithi102 bash[79377]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-c 2024-03-21T11:53:07.741 INFO:journalctl@ceph.mds.c.smithi102.stdout:Mar 21 11:53:07 smithi102 podman[79400]: 2024-03-21 11:53:07.661023263 +0000 UTC m=+0.491754466 container remove 7eee422f14642b5a1d3c9a9a6b2e08d905230ef8440f1d5423d4c1d8e7bbefc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:53:07.908 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.c.service' 2024-03-21T11:53:07.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:53:07.947 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-21T11:53:07.947 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-21T11:53:07.947 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.f 2024-03-21T11:53:08.241 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:53:08 smithi102 systemd[1]: Stopping Ceph mds.f for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:53:08.741 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:53:08 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-f[67371]: 2024-03-21T11:53:08.267+0000 7fd0af34e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:53:08.741 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:53:08 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-f[67371]: 2024-03-21T11:53:08.267+0000 7fd0af34e700 -1 mds.f *** got signal Terminated *** 2024-03-21T11:53:11.474 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:53:11 smithi102 podman[79508]: 2024-03-21 11:53:11.175926529 +0000 UTC m=+3.007996912 container died f8203788be54de48b755f29a98e32430bc5f8bee388b8169d16133ce6729409c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T11:53:11.474 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:53:11 smithi102 podman[79508]: 2024-03-21 11:53:11.474270945 +0000 UTC m=+3.306341328 container cleanup f8203788be54de48b755f29a98e32430bc5f8bee388b8169d16133ce6729409c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-f, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:53:11.741 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:53:11 smithi102 bash[79508]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-f 2024-03-21T11:53:11.741 INFO:journalctl@ceph.mds.f.smithi102.stdout:Mar 21 11:53:11 smithi102 podman[79523]: 2024-03-21 11:53:11.714602372 +0000 UTC m=+0.537477913 container remove f8203788be54de48b755f29a98e32430bc5f8bee388b8169d16133ce6729409c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-21T11:53:11.818 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.f.service' 2024-03-21T11:53:11.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:53:11.854 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-21T11:53:11.854 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-21T11:53:11.854 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.i 2024-03-21T11:53:12.175 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:53:11 smithi102 systemd[1]: Stopping Ceph mds.i for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:53:12.491 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:53:12 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-i[67997]: 2024-03-21T11:53:12.174+0000 7fd114080700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:53:12.491 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:53:12 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-i[67997]: 2024-03-21T11:53:12.174+0000 7fd114080700 -1 mds.i *** got signal Terminated *** 2024-03-21T11:53:15.491 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:53:15 smithi102 podman[79631]: 2024-03-21 11:53:15.172584806 +0000 UTC m=+3.105965112 container died 19a552d32b82ab30d9e409aecd2ccd2e0370c2ba87b1a1a971d698cd490cceda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-i, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:53:15.984 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:53:15 smithi102 podman[79631]: 2024-03-21 11:53:15.733489982 +0000 UTC m=+3.666870278 container cleanup 19a552d32b82ab30d9e409aecd2ccd2e0370c2ba87b1a1a971d698cd490cceda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-i, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-21T11:53:15.984 INFO:journalctl@ceph.mds.i.smithi102.stdout:Mar 21 11:53:15 smithi102 bash[79631]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-i 2024-03-21T11:53:16.149 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.i.service' 2024-03-21T11:53:16.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:53:16.185 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-21T11:53:16.185 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-21T11:53:16.185 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.l 2024-03-21T11:53:16.741 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:53:16 smithi102 systemd[1]: Stopping Ceph mds.l for a6f35682-e773-11ee-95cd-87774f69a715... 2024-03-21T11:53:16.741 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:53:16 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-l[67062]: 2024-03-21T11:53:16.478+0000 7f9c4a41e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T11:53:16.741 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:53:16 smithi102 ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-l[67062]: 2024-03-21T11:53:16.478+0000 7f9c4a41e700 -1 mds.l *** got signal Terminated *** 2024-03-21T11:53:19.595 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:53:19 smithi102 podman[79755]: 2024-03-21 11:53:19.266963513 +0000 UTC m=+2.866859711 container died caa307b0576baaddcff0d4d1953f8d76d0d3e120e5ebdf7ac8d48f5122cb0eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=) 2024-03-21T11:53:19.949 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:53:19 smithi102 podman[79755]: 2024-03-21 11:53:19.595902875 +0000 UTC m=+3.195799064 container cleanup caa307b0576baaddcff0d4d1953f8d76d0d3e120e5ebdf7ac8d48f5122cb0eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-l, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-21T11:53:19.949 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:53:19 smithi102 bash[79755]: ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-l 2024-03-21T11:53:19.949 INFO:journalctl@ceph.mds.l.smithi102.stdout:Mar 21 11:53:19 smithi102 podman[79769]: 2024-03-21 11:53:19.830111502 +0000 UTC m=+0.561734206 container remove caa307b0576baaddcff0d4d1953f8d76d0d3e120e5ebdf7ac8d48f5122cb0eb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-a6f35682-e773-11ee-95cd-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T11:53:19.970 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a6f35682-e773-11ee-95cd-87774f69a715@mds.l.service' 2024-03-21T11:53:20.004 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T11:53:20.004 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-21T11:53:20.004 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a6f35682-e773-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T11:53:20.351 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:53:25.713 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a6f35682-e773-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T11:53:25.904 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:53:28.430 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a6f35682-e773-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T11:53:28.622 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T11:53:31.210 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T11:53:31.324 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T11:53:31.359 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T11:53:31.387 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-21T11:53:31.389 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288/remote/smithi047/crash 2024-03-21T11:53:31.390 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/crash -- . 2024-03-21T11:53:31.500 INFO:teuthology.orchestra.run.smithi047.stderr:tar: /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T11:53:31.501 INFO:teuthology.orchestra.run.smithi047.stderr:tar: Error is not recoverable: exiting now 2024-03-21T11:53:31.506 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288/remote/smithi100/crash 2024-03-21T11:53:31.507 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/crash -- . 2024-03-21T11:53:31.553 INFO:teuthology.orchestra.run.smithi100.stderr:tar: /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T11:53:31.553 INFO:teuthology.orchestra.run.smithi100.stderr:tar: Error is not recoverable: exiting now 2024-03-21T11:53:31.555 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288/remote/smithi102/crash 2024-03-21T11:53:31.556 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/crash -- . 2024-03-21T11:53:31.606 INFO:teuthology.orchestra.run.smithi102.stderr:tar: /var/lib/ceph/a6f35682-e773-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T11:53:31.606 INFO:teuthology.orchestra.run.smithi102.stderr:tar: Error is not recoverable: exiting now 2024-03-21T11:53:31.607 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-21T11:53:31.607 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-21T11:53:31.784 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-21T11:13:56.142649+0000 mon.a (mon.0) 187 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T11:53:31.786 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-21T11:53:31.786 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[SEC\]' /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-21T11:53:31.894 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]' /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-21T11:53:31.990 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[WRN\]' /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-21T11:53:32.083 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-21T11:13:56.142649+0000 mon.a (mon.0) 187 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T11:53:32.084 INFO:tasks.cephadm:Compressing logs... 2024-03-21T11:53:32.084 DEBUG:teuthology.orchestra.run.smithi047:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T11:53:32.088 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T11:53:32.091 DEBUG:teuthology.orchestra.run.smithi102:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T11:53:32.134 INFO:teuthology.orchestra.run.smithi100.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-21T11:53:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T11:53:32.142 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.10.log 2024-03-21T11:53:32.145 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T11:53:32.145 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.b.log 2024-03-21T11:53:32.145 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-21T11:53:32.145 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-21T11:53:32.145 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.k.log 2024-03-21T11:53:32.145 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.h.log 2024-03-21T11:53:32.145 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.7.log 2024-03-21T11:53:32.145 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.e.log 2024-03-21T11:53:32.148 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mgr.y.log 2024-03-21T11:53:32.148 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T11:53:32.148 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.log 2024-03-21T11:53:32.148 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T11:53:32.148 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mon.b.log 2024-03-21T11:53:32.150 INFO:teuthology.orchestra.run.smithi102.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-21T11:53:32.151 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T11:53:32.151 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.l.log 2024-03-21T11:53:32.151 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T11:53:32.151 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.i.log 2024-03-21T11:53:32.151 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.11.log 2024-03-21T11:53:32.151 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mgr.z.log 2024-03-21T11:53:32.151 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.i.log: /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.8.log 2024-03-21T11:53:32.151 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-21T11:53:32.151 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.f.log 2024-03-21T11:53:32.151 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.8.log: /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T11:53:32.151 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.log 2024-03-21T11:53:32.151 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-21T11:53:32.151 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.c.log 2024-03-21T11:53:32.151 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.log: /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T11:53:32.152 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mon.c.log 2024-03-21T11:53:32.152 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mon.c.log: 93.6% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mgr.z.log.gz 2024-03-21T11:53:32.152 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mon.b.log: 93.5% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mgr.y.log.gz 2024-03-21T11:53:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-21T11:53:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T11:53:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-21T11:53:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.6.log 2024-03-21T11:53:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mon.a.log 2024-03-21T11:53:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T11:53:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.g.log 2024-03-21T11:53:32.169 INFO:teuthology.orchestra.run.smithi047.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-21T11:53:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.j.log 2024-03-21T11:53:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-21T11:53:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mgr.x.log 2024-03-21T11:53:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-21T11:53:32.174 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T11:53:32.175 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.log 2024-03-21T11:53:32.176 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T11:53:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.9.log 2024-03-21T11:53:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-volume.log: 94.5% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.g.log.gz 2024-03-21T11:53:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.d.log 2024-03-21T11:53:32.179 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.a.log 2024-03-21T11:53:32.182 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.d.log: /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.a.log: 94.4% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.j.log.gz 2024-03-21T11:53:32.183 INFO:teuthology.orchestra.run.smithi047.stderr: 87.3% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T11:53:32.206 INFO:teuthology.orchestra.run.smithi047.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T11:53:32.262 INFO:teuthology.orchestra.run.smithi100.stderr: 83.1% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T11:53:32.343 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T11:53:32.352 INFO:teuthology.orchestra.run.smithi102.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T11:53:32.361 INFO:teuthology.orchestra.run.smithi102.stderr: 90.3% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T11:53:32.365 INFO:teuthology.orchestra.run.smithi100.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T11:53:32.395 INFO:teuthology.orchestra.run.smithi102.stderr: 91.0% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T11:53:32.400 INFO:teuthology.orchestra.run.smithi047.stderr: 90.2% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T11:53:32.420 INFO:teuthology.orchestra.run.smithi100.stderr: 90.3% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T11:53:32.425 INFO:teuthology.orchestra.run.smithi102.stderr: 87.3% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T11:53:32.478 INFO:teuthology.orchestra.run.smithi102.stderr: 83.0% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T11:53:32.526 INFO:teuthology.orchestra.run.smithi047.stderr: 90.8% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T11:53:32.759 INFO:teuthology.orchestra.run.smithi100.stderr: 91.0% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T11:53:32.895 INFO:teuthology.orchestra.run.smithi047.stderr: 83.1% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T11:53:33.220 INFO:teuthology.orchestra.run.smithi047.stderr: 90.1% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mgr.x.log.gz 2024-03-21T11:53:35.946 INFO:teuthology.orchestra.run.smithi047.stderr: 94.7% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.d.log.gz 2024-03-21T11:53:37.146 INFO:teuthology.orchestra.run.smithi047.stderr: 90.1% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mon.a.log.gz 2024-03-21T11:53:49.909 INFO:teuthology.orchestra.run.smithi100.stderr: 91.8% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mon.b.log.gz 2024-03-21T11:53:58.227 INFO:teuthology.orchestra.run.smithi047.stderr: 95.5% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.a.log.gz 2024-03-21T11:54:31.153 INFO:teuthology.orchestra.run.smithi102.stderr: 92.9% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mon.c.log.gz 2024-03-21T11:56:29.494 INFO:teuthology.orchestra.run.smithi100.stderr: 95.5% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.k.log.gz 2024-03-21T11:56:30.683 INFO:teuthology.orchestra.run.smithi047.stderr: 91.3% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-21T11:56:51.934 INFO:teuthology.orchestra.run.smithi047.stderr: 91.0% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.6.log.gz 2024-03-21T11:56:55.389 INFO:teuthology.orchestra.run.smithi047.stderr: 91.0% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-21T11:57:05.093 INFO:teuthology.orchestra.run.smithi047.stderr: 91.0% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.9.log.gz 2024-03-21T11:57:05.096 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-03-21T11:57:05.096 INFO:teuthology.orchestra.run.smithi047.stderr:real 3m32.977s 2024-03-21T11:57:05.096 INFO:teuthology.orchestra.run.smithi047.stderr:user 7m5.174s 2024-03-21T11:57:05.096 INFO:teuthology.orchestra.run.smithi047.stderr:sys 0m42.297s 2024-03-21T11:59:00.888 INFO:teuthology.orchestra.run.smithi102.stderr: 95.0% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.l.log.gz 2024-03-21T11:59:55.663 INFO:teuthology.orchestra.run.smithi100.stderr: 94.9% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.e.log.gz 2024-03-21T12:02:08.400 INFO:teuthology.orchestra.run.smithi102.stderr: 94.0% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.i.log.gz 2024-03-21T12:02:53.625 INFO:teuthology.orchestra.run.smithi100.stderr: 91.2% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.10.log.gz 2024-03-21T12:03:20.638 INFO:teuthology.orchestra.run.smithi102.stderr: 91.1% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.11.log.gz 2024-03-21T12:03:28.710 INFO:teuthology.orchestra.run.smithi100.stderr: 91.4% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-21T12:03:37.801 INFO:teuthology.orchestra.run.smithi102.stderr: 90.9% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.8.log.gz 2024-03-21T12:06:56.882 INFO:teuthology.orchestra.run.smithi102.stderr: 91.1% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-21T12:08:01.219 INFO:teuthology.orchestra.run.smithi100.stderr: 91.2% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-21T12:08:09.383 INFO:teuthology.orchestra.run.smithi100.stderr: 91.0% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.7.log.gz 2024-03-21T12:09:34.688 INFO:teuthology.orchestra.run.smithi102.stderr: 90.8% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-21T12:09:53.213 INFO:teuthology.orchestra.run.smithi102.stderr: 93.5% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.f.log.gz 2024-03-21T12:10:16.402 INFO:teuthology.orchestra.run.smithi100.stderr: 93.4% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.h.log.gz 2024-03-21T12:11:19.729 INFO:teuthology.orchestra.run.smithi102.stderr: 93.0% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.c.log.gz 2024-03-21T12:11:19.748 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-03-21T12:11:19.748 INFO:teuthology.orchestra.run.smithi102.stderr:real 17m47.643s 2024-03-21T12:11:19.748 INFO:teuthology.orchestra.run.smithi102.stderr:user 8m53.175s 2024-03-21T12:11:19.748 INFO:teuthology.orchestra.run.smithi102.stderr:sys 0m23.142s 2024-03-21T12:16:09.567 INFO:teuthology.orchestra.run.smithi100.stderr: 93.2% -- replaced with /var/log/ceph/a6f35682-e773-11ee-95cd-87774f69a715/ceph-mds.b.log.gz 2024-03-21T12:16:09.954 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-03-21T12:16:09.954 INFO:teuthology.orchestra.run.smithi100.stderr:real 22m37.848s 2024-03-21T12:16:09.954 INFO:teuthology.orchestra.run.smithi100.stderr:user 12m10.486s 2024-03-21T12:16:09.954 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m31.435s 2024-03-21T12:16:09.955 INFO:tasks.cephadm:Archiving logs... 2024-03-21T12:16:09.955 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288/remote/smithi047/log 2024-03-21T12:16:09.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T12:16:57.769 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288/remote/smithi100/log 2024-03-21T12:16:57.771 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T12:18:55.738 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288/remote/smithi102/log 2024-03-21T12:18:55.741 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T12:20:22.916 INFO:tasks.cephadm:Removing cluster... 2024-03-21T12:20:22.916 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a6f35682-e773-11ee-95cd-87774f69a715 --force 2024-03-21T12:20:23.398 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T12:20:25.151 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a6f35682-e773-11ee-95cd-87774f69a715 --force 2024-03-21T12:20:25.668 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T12:20:26.512 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a6f35682-e773-11ee-95cd-87774f69a715 --force 2024-03-21T12:20:27.035 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: a6f35682-e773-11ee-95cd-87774f69a715 2024-03-21T12:20:27.923 INFO:tasks.cephadm:Removing cephadm ... 2024-03-21T12:20:27.923 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T12:20:27.983 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T12:20:28.001 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T12:20:28.017 INFO:tasks.cephadm:Teardown complete 2024-03-21T12:20:28.017 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-21T12:20:28.028 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-21T12:20:28.029 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T12:20:28.036 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T12:20:28.044 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T12:20:28.219 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-21T12:20:28.230 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-21T12:20:28.230 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T12:20:28.238 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T12:20:28.240 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T12:20:28.254 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-03-21T12:20:28.254 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-03-21T12:20:28.286 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-03-21T12:20:28.356 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T12:20:28.356 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-03-21T12:20:28.356 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 34 +839us[ +839us] +/- 82ms 2024-03-21T12:20:28.356 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 -1078us[-1068us] +/- 80ms 2024-03-21T12:20:28.356 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +260us[ +270us] +/- 67ms 2024-03-21T12:20:28.356 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T12:20:28.358 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T12:20:28.358 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-03-21T12:20:28.358 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 34 +831us[ +831us] +/- 82ms 2024-03-21T12:20:28.358 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 -1081us[-1064us] +/- 80ms 2024-03-21T12:20:28.358 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +265us[ +281us] +/- 67ms 2024-03-21T12:20:28.359 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T12:20:28.381 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T12:20:28.381 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-03-21T12:20:28.381 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 33 +819us[ +819us] +/- 82ms 2024-03-21T12:20:28.381 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 -1096us[-1092us] +/- 80ms 2024-03-21T12:20:28.381 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +245us[ +249us] +/- 67ms 2024-03-21T12:20:28.381 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T12:20:28.385 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-21T12:20:28.395 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-21T12:20:28.396 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-21T12:20:28.428 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T12:20:28.727 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T12:20:28.837 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T12:20:28.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T12:20:29.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T12:20:29.071 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T12:20:29.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T12:20:29.107 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T12:20:29.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T12:20:29.142 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-21T12:20:29.153 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-21T12:20:29.190 INFO:teuthology.task.internal:Duration was 4677.445185 seconds 2024-03-21T12:20:29.190 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-21T12:20:29.212 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-21T12:20:29.212 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T12:20:29.220 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T12:20:29.222 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T12:20:29.283 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T12:20:29.297 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T12:20:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T12:20:29.691 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-21T12:20:29.691 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T12:20:29.691 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T12:20:29.755 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-03-21T12:20:29.756 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T12:20:29.787 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-03-21T12:20:29.788 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T12:20:29.812 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-21T12:20:29.812 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T12:20:29.820 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T12:20:29.830 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T12:20:29.931 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-21T12:20:29.931 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T12:20:29.938 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T12:20:29.940 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T12:20:30.508 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-21T12:20:30.519 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-21T12:20:30.519 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T12:20:30.641 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T12:20:30.691 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T12:20:30.736 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-21T12:20:30.747 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T12:20:30.755 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T12:20:30.757 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T12:20:30.789 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-03-21T12:20:30.792 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-03-21T12:20:30.879 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-03-21T12:20:30.949 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T12:20:31.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T12:20:31.006 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T12:20:31.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T12:20:31.022 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T12:20:31.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T12:20:31.037 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-21T12:20:31.049 INFO:teuthology.task.internal:Transferring archived files... 2024-03-21T12:20:31.050 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288/remote/smithi047 2024-03-21T12:20:31.050 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T12:20:31.173 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288/remote/smithi100 2024-03-21T12:20:31.173 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T12:20:31.214 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288/remote/smithi102 2024-03-21T12:20:31.214 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T12:20:31.253 INFO:teuthology.task.internal:Removing archive directory... 2024-03-21T12:20:31.253 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T12:20:31.261 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T12:20:31.264 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T12:20:31.313 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-21T12:20:31.325 INFO:teuthology.task.internal:Not uploading archives. 2024-03-21T12:20:31.326 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-21T12:20:31.372 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-21T12:20:31.372 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T12:20:31.380 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T12:20:31.382 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T12:20:31.397 INFO:teuthology.orchestra.run.smithi102.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 12:20 /home/ubuntu/cephtest 2024-03-21T12:20:31.398 INFO:teuthology.orchestra.run.smithi100.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 12:20 /home/ubuntu/cephtest 2024-03-21T12:20:31.430 INFO:teuthology.orchestra.run.smithi047.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 12:20 /home/ubuntu/cephtest 2024-03-21T12:20:31.433 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-21T12:20:31.445 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-21T12:20:31.496 INFO:teuthology.nuke:Checking targets against current locks 2024-03-21T12:20:31.523 DEBUG:teuthology.nuke:shortname: smithi047 2024-03-21T12:20:31.524 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T12:20:31.550 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 10:53:37.281124', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLgNA2Cv+/e+vSQiK6PBnccFdtzJeLcrO6X8o2IA6GnIzz2YmyjPyY9VqM/h7kdjJ994waGlpuljCHye3vrB+ms='} 2024-03-21T12:20:31.553 DEBUG:teuthology.nuke:shortname: smithi100 2024-03-21T12:20:31.554 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T12:20:31.576 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 10:53:37.278927', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMAKhlcMYYiUa8l05rssHJxz+MWdLxH5Db3TrI5GQrNK2JqHXA+5lxfmLOUMCPxtgSIk0zt/rI19pc1bJml8CjM='} 2024-03-21T12:20:31.580 DEBUG:teuthology.nuke:shortname: smithi102 2024-03-21T12:20:31.581 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T12:20:31.611 INFO:teuthology.orchestra.console.smithi047:Power off 2024-03-21T12:20:31.611 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T12:20:31.620 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 10:53:37.280099', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHGc7E7fcehHfHo3EXyrmhbGUh5qN0aAYfNGR18GsApBPg7ec4LJmJtKZIFZzxmkPI3YFipQT0YCUyvUM838azE='} 2024-03-21T12:20:31.645 DEBUG:teuthology.orchestra.console.smithi047:power off output: Chassis Power Control: Down/Off 2024-03-21T12:20:31.645 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T12:20:31.660 INFO:teuthology.orchestra.console.smithi100:Power off 2024-03-21T12:20:31.660 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T12:20:31.674 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on 2024-03-21T12:20:31.684 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2024-03-21T12:20:31.684 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T12:20:31.704 INFO:teuthology.orchestra.console.smithi102:Power off 2024-03-21T12:20:31.704 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T12:20:31.713 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-21T12:20:31.725 DEBUG:teuthology.orchestra.console.smithi102:power off output: Chassis Power Control: Down/Off 2024-03-21T12:20:31.725 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T12:20:31.746 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is on 2024-03-21T12:20:35.675 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T12:20:35.714 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T12:20:35.747 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T12:20:35.787 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on 2024-03-21T12:20:35.823 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-21T12:20:35.856 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is on 2024-03-21T12:20:39.791 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T12:20:39.824 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T12:20:39.857 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T12:20:39.902 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is off 2024-03-21T12:20:39.934 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-21T12:20:39.967 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is on 2024-03-21T12:20:40.003 INFO:teuthology.orchestra.console.smithi047:Power off completed 2024-03-21T12:20:43.939 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T12:20:43.968 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T12:20:44.048 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-03-21T12:20:44.080 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is off 2024-03-21T12:20:44.149 INFO:teuthology.orchestra.console.smithi100:Power off completed 2024-03-21T12:20:44.181 INFO:teuthology.orchestra.console.smithi102:Power off completed 2024-03-21T12:20:44.286 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} duration: 4677.445184707642 failure_reason: '"2024-03-21T11:13:56.142649+0000 mon.a (mon.0) 187 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: scheduled_vshankar@teuthology success: false 2024-03-21T12:20:44.287 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T12:20:44.391 INFO:teuthology.run:FAIL